2024-09-21T14:45:56.895 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-21T14:45:56.917 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-21T14:45:56.997 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-17_22:16:03-rbd-reef-distro-default-smithi/7909956 branch: reef description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-zstd supported-random-distro$/{ubuntu_20.04} thrashers/default thrashosds-health workloads/rbd_fsx_deep_copy} email: ceph-qa@ceph.com first_in_suite: false job_id: '7909956' kernel: kdb: 1 sha1: distro ktype: distro last_in_suite: false machine_type: smithi name: teuthology-2024-09-17_22:16:03-rbd-reef-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 - machine: cpus: 1 disk: 40 ram: 8000 volumes: count: 4 size: 30 os_type: ubuntu os_version: '20.04' overrides: admin_socket: branch: reef ceph: conf: global: mon client directed command retry: 5 mon warn on pool no app: false ms inject socket failures: 5000 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bluestore block size: 96636764160 bluestore compression algorithm: zstd bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from sha1: 19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: 19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: reef sha1: a477b1894d5c5a613f23260ca0e6b96d79652ba7 owner: scheduled_teuthology@teuthology priority: 930 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b seed: 4623 sha1: 19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 sleep_before_teardown: 0 subset: 80/128 suite: rbd suite_branch: reef suite_path: /home/teuthworker/src/git.ceph.com_ceph_a477b1894d5c5a613f23260ca0e6b96d79652ba7/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: a477b1894d5c5a613f23260ca0e6b96d79652ba7 targets: smithi032.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= smithi046.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= tasks: - install: null - ceph: null - thrashosds: timeout: 1200 - rbd_fsx: clients: - client.0 deep_copy: true ops: 6000 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-17_22:16:03 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2902 2024-09-21T14:45:56.997 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_a477b1894d5c5a613f23260ca0e6b96d79652ba7/qa; will attempt to use it 2024-09-21T14:45:56.998 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_a477b1894d5c5a613f23260ca0e6b96d79652ba7/qa/tasks 2024-09-21T14:45:56.998 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-21T14:45:56.999 INFO:teuthology.task.internal:Checking packages... 2024-09-21T14:45:57.018 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '19e2a2beb6f41a3ab62650e38614eb29b59dcbb9' 2024-09-21T14:45:57.018 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-21T14:45:57.018 INFO:teuthology.packaging:ref: None 2024-09-21T14:45:57.018 INFO:teuthology.packaging:tag: None 2024-09-21T14:45:57.019 INFO:teuthology.packaging:branch: reef 2024-09-21T14:45:57.019 INFO:teuthology.packaging:sha1: 19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 2024-09-21T14:45:57.019 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&ref=reef 2024-09-21T14:45:57.282 INFO:teuthology.task.internal:Found packages for ceph version 18.2.4-871-g6a1bdf85-1focal 2024-09-21T14:45:57.283 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-21T14:45:57.290 INFO:teuthology.task.internal:no buildpackages task found 2024-09-21T14:45:57.290 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-21T14:45:57.297 INFO:teuthology.task.internal:Saving configuration 2024-09-21T14:45:57.308 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-21T14:45:57.315 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-21T14:45:57.340 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi032.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-17_22:16:03-rbd-reef-distro-default-smithi/7909956', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 14:39:20.767263', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2024-09-21T14:45:57.360 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi046.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-17_22:16:03-rbd-reef-distro-default-smithi/7909956', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 14:39:20.769275', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2024-09-21T14:45:57.360 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-21T14:45:57.366 INFO:teuthology.task.internal:roles: ubuntu@smithi032.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2024-09-21T14:45:57.366 INFO:teuthology.task.internal:roles: ubuntu@smithi046.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2024-09-21T14:45:57.367 INFO:teuthology.run_tasks:Running task console_log... 2024-09-21T14:45:57.422 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f34bd72fa30>, signals=[15]) 2024-09-21T14:45:57.422 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-21T14:45:57.428 INFO:teuthology.task.internal:Opening connections... 2024-09-21T14:45:57.428 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi032.front.sepia.ceph.com 2024-09-21T14:45:57.429 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T14:45:57.517 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi046.front.sepia.ceph.com 2024-09-21T14:45:57.517 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T14:45:57.607 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-21T14:45:57.614 DEBUG:teuthology.orchestra.run.smithi032:> uname -m 2024-09-21T14:45:57.620 INFO:teuthology.orchestra.run.smithi032.stdout:x86_64 2024-09-21T14:45:57.620 DEBUG:teuthology.orchestra.run.smithi032:> cat /etc/os-release 2024-09-21T14:45:57.668 INFO:teuthology.orchestra.run.smithi032.stdout:NAME="Ubuntu" 2024-09-21T14:45:57.668 INFO:teuthology.orchestra.run.smithi032.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2024-09-21T14:45:57.669 INFO:teuthology.orchestra.run.smithi032.stdout:ID=ubuntu 2024-09-21T14:45:57.669 INFO:teuthology.orchestra.run.smithi032.stdout:ID_LIKE=debian 2024-09-21T14:45:57.669 INFO:teuthology.orchestra.run.smithi032.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2024-09-21T14:45:57.669 INFO:teuthology.orchestra.run.smithi032.stdout:VERSION_ID="20.04" 2024-09-21T14:45:57.669 INFO:teuthology.orchestra.run.smithi032.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-21T14:45:57.669 INFO:teuthology.orchestra.run.smithi032.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-21T14:45:57.669 INFO:teuthology.orchestra.run.smithi032.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-21T14:45:57.669 INFO:teuthology.orchestra.run.smithi032.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-21T14:45:57.669 INFO:teuthology.orchestra.run.smithi032.stdout:VERSION_CODENAME=focal 2024-09-21T14:45:57.669 INFO:teuthology.orchestra.run.smithi032.stdout:UBUNTU_CODENAME=focal 2024-09-21T14:45:57.670 INFO:teuthology.lock.ops:Updating smithi032.front.sepia.ceph.com on lock server 2024-09-21T14:45:57.691 DEBUG:teuthology.orchestra.run.smithi046:> uname -m 2024-09-21T14:45:57.697 INFO:teuthology.orchestra.run.smithi046.stdout:x86_64 2024-09-21T14:45:57.697 DEBUG:teuthology.orchestra.run.smithi046:> cat /etc/os-release 2024-09-21T14:45:57.749 INFO:teuthology.orchestra.run.smithi046.stdout:NAME="Ubuntu" 2024-09-21T14:45:57.749 INFO:teuthology.orchestra.run.smithi046.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2024-09-21T14:45:57.750 INFO:teuthology.orchestra.run.smithi046.stdout:ID=ubuntu 2024-09-21T14:45:57.750 INFO:teuthology.orchestra.run.smithi046.stdout:ID_LIKE=debian 2024-09-21T14:45:57.750 INFO:teuthology.orchestra.run.smithi046.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2024-09-21T14:45:57.750 INFO:teuthology.orchestra.run.smithi046.stdout:VERSION_ID="20.04" 2024-09-21T14:45:57.750 INFO:teuthology.orchestra.run.smithi046.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-21T14:45:57.750 INFO:teuthology.orchestra.run.smithi046.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-21T14:45:57.750 INFO:teuthology.orchestra.run.smithi046.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-21T14:45:57.750 INFO:teuthology.orchestra.run.smithi046.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-21T14:45:57.750 INFO:teuthology.orchestra.run.smithi046.stdout:VERSION_CODENAME=focal 2024-09-21T14:45:57.750 INFO:teuthology.orchestra.run.smithi046.stdout:UBUNTU_CODENAME=focal 2024-09-21T14:45:57.751 INFO:teuthology.lock.ops:Updating smithi046.front.sepia.ceph.com on lock server 2024-09-21T14:45:57.783 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-21T14:45:57.797 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-21T14:45:57.802 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-21T14:45:57.802 DEBUG:teuthology.orchestra.run.smithi032:> test '!' -e /home/ubuntu/cephtest 2024-09-21T14:45:57.804 DEBUG:teuthology.orchestra.run.smithi046:> test '!' -e /home/ubuntu/cephtest 2024-09-21T14:45:57.810 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-21T14:45:57.830 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-21T14:45:57.830 DEBUG:teuthology.orchestra.run.smithi032:> test -z $(ls -A /var/lib/ceph) 2024-09-21T14:45:57.832 DEBUG:teuthology.orchestra.run.smithi046:> test -z $(ls -A /var/lib/ceph) 2024-09-21T14:45:57.843 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-21T14:45:57.916 INFO:teuthology.run_tasks:Running task kernel... 2024-09-21T14:45:57.966 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-21T14:45:57.966 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-21T14:45:57.966 DEBUG:teuthology.orchestra.run.smithi032:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T14:45:57.967 DEBUG:teuthology.orchestra.run.smithi046:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T14:45:57.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T14:45:57.973 DEBUG:teuthology.orchestra.run.smithi032:> uname -r 2024-09-21T14:45:57.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T14:45:57.974 DEBUG:teuthology.orchestra.run.smithi046:> uname -r 2024-09-21T14:45:57.980 INFO:teuthology.orchestra.run.smithi032.stdout:5.4.0-124-generic 2024-09-21T14:45:57.980 INFO:teuthology.task.kernel:Running kernel on smithi032: 5.4.0-124-generic 2024-09-21T14:45:57.980 DEBUG:teuthology.orchestra.run.smithi032:> sudo apt-get clean 2024-09-21T14:45:57.981 INFO:teuthology.orchestra.run.smithi046.stdout:5.4.0-124-generic 2024-09-21T14:45:57.981 INFO:teuthology.task.kernel:Running kernel on smithi046: 5.4.0-124-generic 2024-09-21T14:45:57.981 DEBUG:teuthology.orchestra.run.smithi046:> sudo apt-get clean 2024-09-21T14:45:58.089 DEBUG:teuthology.orchestra.run.smithi046:> sudo apt-get update 2024-09-21T14:45:58.138 DEBUG:teuthology.orchestra.run.smithi032:> sudo apt-get update 2024-09-21T14:45:58.401 INFO:teuthology.orchestra.run.smithi046.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-21T14:45:58.401 INFO:teuthology.orchestra.run.smithi046.stdout:Get:2 http://security.ubuntu.com/ubuntu focal-security InRelease [128 kB] 2024-09-21T14:45:58.421 INFO:teuthology.orchestra.run.smithi032.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-21T14:45:58.422 INFO:teuthology.orchestra.run.smithi032.stdout:Get:2 http://security.ubuntu.com/ubuntu focal-security InRelease [128 kB] 2024-09-21T14:45:58.451 INFO:teuthology.orchestra.run.smithi046.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease [128 kB] 2024-09-21T14:45:58.481 INFO:teuthology.orchestra.run.smithi032.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease [128 kB] 2024-09-21T14:45:58.805 INFO:teuthology.orchestra.run.smithi046.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease [128 kB] 2024-09-21T14:45:58.839 INFO:teuthology.orchestra.run.smithi032.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease [128 kB] 2024-09-21T14:45:58.883 INFO:teuthology.orchestra.run.smithi046.stdout:Get:5 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [818 kB] 2024-09-21T14:45:58.921 INFO:teuthology.orchestra.run.smithi032.stdout:Get:5 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [818 kB] 2024-09-21T14:45:59.031 INFO:teuthology.orchestra.run.smithi046.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [3,601 kB] 2024-09-21T14:45:59.066 INFO:teuthology.orchestra.run.smithi032.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [3,601 kB] 2024-09-21T14:45:59.232 INFO:teuthology.orchestra.run.smithi046.stdout:Get:7 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [3,227 kB] 2024-09-21T14:45:59.270 INFO:teuthology.orchestra.run.smithi032.stdout:Get:7 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [3,227 kB] 2024-09-21T14:45:59.424 INFO:teuthology.orchestra.run.smithi046.stdout:Get:8 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [476 kB] 2024-09-21T14:45:59.438 INFO:teuthology.orchestra.run.smithi046.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [14.3 kB] 2024-09-21T14:45:59.439 INFO:teuthology.orchestra.run.smithi046.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [38.3 kB] 2024-09-21T14:45:59.440 INFO:teuthology.orchestra.run.smithi046.stdout:Get:11 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [3,156 kB] 2024-09-21T14:45:59.461 INFO:teuthology.orchestra.run.smithi032.stdout:Get:8 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [476 kB] 2024-09-21T14:45:59.475 INFO:teuthology.orchestra.run.smithi032.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [14.3 kB] 2024-09-21T14:45:59.476 INFO:teuthology.orchestra.run.smithi032.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [38.3 kB] 2024-09-21T14:45:59.477 INFO:teuthology.orchestra.run.smithi032.stdout:Get:11 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [3,156 kB] 2024-09-21T14:45:59.499 INFO:teuthology.orchestra.run.smithi046.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [1,037 kB] 2024-09-21T14:45:59.516 INFO:teuthology.orchestra.run.smithi046.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [554 kB] 2024-09-21T14:45:59.525 INFO:teuthology.orchestra.run.smithi046.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [17.7 kB] 2024-09-21T14:45:59.525 INFO:teuthology.orchestra.run.smithi046.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [3,283 kB] 2024-09-21T14:45:59.534 INFO:teuthology.orchestra.run.smithi032.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [1,037 kB] 2024-09-21T14:45:59.534 INFO:teuthology.orchestra.run.smithi046.stdout:Get:16 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [441 kB] 2024-09-21T14:45:59.548 INFO:teuthology.orchestra.run.smithi046.stdout:Get:17 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [548 B] 2024-09-21T14:45:59.548 INFO:teuthology.orchestra.run.smithi046.stdout:Get:18 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [681 kB] 2024-09-21T14:45:59.550 INFO:teuthology.orchestra.run.smithi032.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [554 kB] 2024-09-21T14:45:59.562 INFO:teuthology.orchestra.run.smithi032.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [17.7 kB] 2024-09-21T14:45:59.562 INFO:teuthology.orchestra.run.smithi032.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [39.6 kB] 2024-09-21T14:45:59.563 INFO:teuthology.orchestra.run.smithi032.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [3,283 kB] 2024-09-21T14:45:59.565 INFO:teuthology.orchestra.run.smithi046.stdout:Get:19 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [1,012 kB] 2024-09-21T14:45:59.571 INFO:teuthology.orchestra.run.smithi032.stdout:Get:17 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [441 kB] 2024-09-21T14:45:59.584 INFO:teuthology.orchestra.run.smithi032.stdout:Get:18 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [548 B] 2024-09-21T14:45:59.584 INFO:teuthology.orchestra.run.smithi032.stdout:Get:19 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [1,012 kB] 2024-09-21T14:45:59.596 INFO:teuthology.orchestra.run.smithi046.stdout:Get:20 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [213 kB] 2024-09-21T14:45:59.602 INFO:teuthology.orchestra.run.smithi046.stdout:Get:21 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [21.4 kB] 2024-09-21T14:45:59.602 INFO:teuthology.orchestra.run.smithi046.stdout:Get:22 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [7,204 B] 2024-09-21T14:45:59.602 INFO:teuthology.orchestra.run.smithi046.stdout:Get:23 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [24.8 kB] 2024-09-21T14:45:59.603 INFO:teuthology.orchestra.run.smithi046.stdout:Get:24 http://security.ubuntu.com/ubuntu focal-security/multiverse Translation-en [5,968 B] 2024-09-21T14:45:59.610 INFO:teuthology.orchestra.run.smithi032.stdout:Get:20 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [681 kB] 2024-09-21T14:45:59.616 INFO:teuthology.orchestra.run.smithi046.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [39.6 kB] 2024-09-21T14:45:59.616 INFO:teuthology.orchestra.run.smithi046.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [460 kB] 2024-09-21T14:45:59.624 INFO:teuthology.orchestra.run.smithi046.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [548 B] 2024-09-21T14:45:59.624 INFO:teuthology.orchestra.run.smithi046.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1,232 kB] 2024-09-21T14:45:59.631 INFO:teuthology.orchestra.run.smithi032.stdout:Get:21 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [213 kB] 2024-09-21T14:45:59.637 INFO:teuthology.orchestra.run.smithi032.stdout:Get:22 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [21.4 kB] 2024-09-21T14:45:59.637 INFO:teuthology.orchestra.run.smithi032.stdout:Get:23 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [24.8 kB] 2024-09-21T14:45:59.638 INFO:teuthology.orchestra.run.smithi032.stdout:Get:24 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [7,204 B] 2024-09-21T14:45:59.638 INFO:teuthology.orchestra.run.smithi032.stdout:Get:25 http://security.ubuntu.com/ubuntu focal-security/multiverse Translation-en [5,968 B] 2024-09-21T14:45:59.652 INFO:teuthology.orchestra.run.smithi032.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [460 kB] 2024-09-21T14:45:59.659 INFO:teuthology.orchestra.run.smithi032.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [548 B] 2024-09-21T14:45:59.659 INFO:teuthology.orchestra.run.smithi032.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1,232 kB] 2024-09-21T14:45:59.670 INFO:teuthology.orchestra.run.smithi046.stdout:Get:29 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [540 B] 2024-09-21T14:45:59.680 INFO:teuthology.orchestra.run.smithi046.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [808 kB] 2024-09-21T14:45:59.693 INFO:teuthology.orchestra.run.smithi046.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [295 kB] 2024-09-21T14:45:59.698 INFO:teuthology.orchestra.run.smithi046.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [28.3 kB] 2024-09-21T14:45:59.698 INFO:teuthology.orchestra.run.smithi046.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [8,440 B] 2024-09-21T14:45:59.698 INFO:teuthology.orchestra.run.smithi046.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [27.1 kB] 2024-09-21T14:45:59.711 INFO:teuthology.orchestra.run.smithi032.stdout:Get:29 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [540 B] 2024-09-21T14:45:59.713 INFO:teuthology.orchestra.run.smithi032.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [808 kB] 2024-09-21T14:45:59.727 INFO:teuthology.orchestra.run.smithi032.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [295 kB] 2024-09-21T14:45:59.733 INFO:teuthology.orchestra.run.smithi032.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [28.3 kB] 2024-09-21T14:45:59.733 INFO:teuthology.orchestra.run.smithi032.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [27.1 kB] 2024-09-21T14:45:59.734 INFO:teuthology.orchestra.run.smithi032.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [8,440 B] 2024-09-21T14:45:59.734 INFO:teuthology.orchestra.run.smithi032.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [7,936 B] 2024-09-21T14:45:59.761 INFO:teuthology.orchestra.run.smithi046.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [7,936 B] 2024-09-21T14:45:59.762 INFO:teuthology.orchestra.run.smithi046.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [616 B] 2024-09-21T14:45:59.762 INFO:teuthology.orchestra.run.smithi046.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [45.7 kB] 2024-09-21T14:45:59.762 INFO:teuthology.orchestra.run.smithi046.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [36.1 kB] 2024-09-21T14:45:59.796 INFO:teuthology.orchestra.run.smithi032.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [616 B] 2024-09-21T14:45:59.796 INFO:teuthology.orchestra.run.smithi032.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [36.1 kB] 2024-09-21T14:45:59.796 INFO:teuthology.orchestra.run.smithi032.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [45.7 kB] 2024-09-21T14:45:59.842 INFO:teuthology.orchestra.run.smithi046.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [1,420 B] 2024-09-21T14:45:59.843 INFO:teuthology.orchestra.run.smithi046.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [25.0 kB] 2024-09-21T14:45:59.843 INFO:teuthology.orchestra.run.smithi046.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.8 kB] 2024-09-21T14:45:59.843 INFO:teuthology.orchestra.run.smithi046.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [880 B] 2024-09-21T14:45:59.877 INFO:teuthology.orchestra.run.smithi032.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [1,420 B] 2024-09-21T14:45:59.877 INFO:teuthology.orchestra.run.smithi032.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [25.0 kB] 2024-09-21T14:45:59.878 INFO:teuthology.orchestra.run.smithi032.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.8 kB] 2024-09-21T14:45:59.878 INFO:teuthology.orchestra.run.smithi032.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [880 B] 2024-09-21T14:46:03.884 INFO:teuthology.orchestra.run.smithi046.stdout:Fetched 22.0 MB in 4s (5,807 kB/s) 2024-09-21T14:46:03.912 INFO:teuthology.orchestra.run.smithi032.stdout:Fetched 22.0 MB in 4s (5,867 kB/s) 2024-09-21T14:46:05.362 INFO:teuthology.orchestra.run.smithi046.stdout:Reading package lists... 2024-09-21T14:46:05.390 DEBUG:teuthology.orchestra.run.smithi046:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-21T14:46:05.475 INFO:teuthology.orchestra.run.smithi046.stdout:Reading package lists... 2024-09-21T14:46:05.517 INFO:teuthology.orchestra.run.smithi032.stdout:Reading package lists... 2024-09-21T14:46:05.544 DEBUG:teuthology.orchestra.run.smithi032:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-21T14:46:05.618 INFO:teuthology.orchestra.run.smithi032.stdout:Reading package lists... 2024-09-21T14:46:05.675 INFO:teuthology.orchestra.run.smithi046.stdout:Building dependency tree... 2024-09-21T14:46:05.675 INFO:teuthology.orchestra.run.smithi046.stdout:Reading state information... 2024-09-21T14:46:05.844 INFO:teuthology.orchestra.run.smithi032.stdout:Building dependency tree... 2024-09-21T14:46:05.845 INFO:teuthology.orchestra.run.smithi032.stdout:Reading state information... 2024-09-21T14:46:05.884 INFO:teuthology.orchestra.run.smithi046.stdout:The following additional packages will be installed: 2024-09-21T14:46:05.884 INFO:teuthology.orchestra.run.smithi046.stdout: linux-generic linux-headers-5.4.0-196 linux-headers-5.4.0-196-generic 2024-09-21T14:46:05.884 INFO:teuthology.orchestra.run.smithi046.stdout: linux-headers-generic linux-image-5.4.0-196-generic 2024-09-21T14:46:05.885 INFO:teuthology.orchestra.run.smithi046.stdout: linux-modules-5.4.0-196-generic linux-modules-extra-5.4.0-196-generic 2024-09-21T14:46:05.886 INFO:teuthology.orchestra.run.smithi046.stdout:Suggested packages: 2024-09-21T14:46:05.886 INFO:teuthology.orchestra.run.smithi046.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2024-09-21T14:46:05.886 INFO:teuthology.orchestra.run.smithi046.stdout:Recommended packages: 2024-09-21T14:46:05.886 INFO:teuthology.orchestra.run.smithi046.stdout: thermald 2024-09-21T14:46:05.942 INFO:teuthology.orchestra.run.smithi046.stdout:The following NEW packages will be installed: 2024-09-21T14:46:05.942 INFO:teuthology.orchestra.run.smithi046.stdout: linux-headers-5.4.0-196 linux-headers-5.4.0-196-generic 2024-09-21T14:46:05.942 INFO:teuthology.orchestra.run.smithi046.stdout: linux-image-5.4.0-196-generic linux-modules-5.4.0-196-generic 2024-09-21T14:46:05.943 INFO:teuthology.orchestra.run.smithi046.stdout: linux-modules-extra-5.4.0-196-generic 2024-09-21T14:46:05.944 INFO:teuthology.orchestra.run.smithi046.stdout:The following packages will be upgraded: 2024-09-21T14:46:05.944 INFO:teuthology.orchestra.run.smithi046.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-21T14:46:06.064 INFO:teuthology.orchestra.run.smithi032.stdout:The following additional packages will be installed: 2024-09-21T14:46:06.065 INFO:teuthology.orchestra.run.smithi032.stdout: linux-generic linux-headers-5.4.0-196 linux-headers-5.4.0-196-generic 2024-09-21T14:46:06.065 INFO:teuthology.orchestra.run.smithi032.stdout: linux-headers-generic linux-image-5.4.0-196-generic 2024-09-21T14:46:06.065 INFO:teuthology.orchestra.run.smithi032.stdout: linux-modules-5.4.0-196-generic linux-modules-extra-5.4.0-196-generic 2024-09-21T14:46:06.067 INFO:teuthology.orchestra.run.smithi032.stdout:Suggested packages: 2024-09-21T14:46:06.067 INFO:teuthology.orchestra.run.smithi032.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2024-09-21T14:46:06.067 INFO:teuthology.orchestra.run.smithi032.stdout:Recommended packages: 2024-09-21T14:46:06.067 INFO:teuthology.orchestra.run.smithi032.stdout: thermald 2024-09-21T14:46:06.123 INFO:teuthology.orchestra.run.smithi032.stdout:The following NEW packages will be installed: 2024-09-21T14:46:06.124 INFO:teuthology.orchestra.run.smithi032.stdout: linux-headers-5.4.0-196 linux-headers-5.4.0-196-generic 2024-09-21T14:46:06.124 INFO:teuthology.orchestra.run.smithi032.stdout: linux-image-5.4.0-196-generic linux-modules-5.4.0-196-generic 2024-09-21T14:46:06.124 INFO:teuthology.orchestra.run.smithi032.stdout: linux-modules-extra-5.4.0-196-generic 2024-09-21T14:46:06.125 INFO:teuthology.orchestra.run.smithi032.stdout:The following packages will be upgraded: 2024-09-21T14:46:06.125 INFO:teuthology.orchestra.run.smithi032.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-21T14:46:06.126 INFO:teuthology.orchestra.run.smithi046.stdout:3 upgraded, 5 newly installed, 0 to remove and 318 not upgraded. 2024-09-21T14:46:06.126 INFO:teuthology.orchestra.run.smithi046.stdout:Need to get 77.2 MB of archives. 2024-09-21T14:46:06.126 INFO:teuthology.orchestra.run.smithi046.stdout:After this operation, 381 MB of additional disk space will be used. 2024-09-21T14:46:06.126 INFO:teuthology.orchestra.run.smithi046.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-196-generic amd64 5.4.0-196.216 [15.0 MB] 2024-09-21T14:46:06.178 INFO:teuthology.orchestra.run.smithi032.stdout:3 upgraded, 5 newly installed, 0 to remove and 318 not upgraded. 2024-09-21T14:46:06.178 INFO:teuthology.orchestra.run.smithi032.stdout:Need to get 77.2 MB of archives. 2024-09-21T14:46:06.178 INFO:teuthology.orchestra.run.smithi032.stdout:After this operation, 381 MB of additional disk space will be used. 2024-09-21T14:46:06.178 INFO:teuthology.orchestra.run.smithi032.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-196-generic amd64 5.4.0-196.216 [15.0 MB] 2024-09-21T14:46:06.513 INFO:teuthology.orchestra.run.smithi032.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-196-generic amd64 5.4.0-196.216 [10.5 MB] 2024-09-21T14:46:06.605 INFO:teuthology.orchestra.run.smithi032.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-196-generic amd64 5.4.0-196.216 [39.2 MB] 2024-09-21T14:46:07.063 INFO:teuthology.orchestra.run.smithi032.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.196.194 [1,900 B] 2024-09-21T14:46:07.063 INFO:teuthology.orchestra.run.smithi032.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.196.194 [2,592 B] 2024-09-21T14:46:07.063 INFO:teuthology.orchestra.run.smithi032.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-196 all 5.4.0-196.216 [11.0 MB] 2024-09-21T14:46:07.155 INFO:teuthology.orchestra.run.smithi032.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-196-generic amd64 5.4.0-196.216 [1,386 kB] 2024-09-21T14:46:07.166 INFO:teuthology.orchestra.run.smithi032.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.196.194 [2,460 B] 2024-09-21T14:46:07.291 INFO:teuthology.orchestra.run.smithi046.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-196-generic amd64 5.4.0-196.216 [10.5 MB] 2024-09-21T14:46:07.562 INFO:teuthology.orchestra.run.smithi046.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-196-generic amd64 5.4.0-196.216 [39.2 MB] 2024-09-21T14:46:07.838 INFO:teuthology.orchestra.run.smithi032.stdout:Fetched 77.2 MB in 1s (74.7 MB/s) 2024-09-21T14:46:08.031 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package linux-modules-5.4.0-196-generic. 2024-09-21T14:46:08.901 INFO:teuthology.orchestra.run.smithi046.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.196.194 [1,900 B] 2024-09-21T14:46:08.902 INFO:teuthology.orchestra.run.smithi046.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.196.194 [2,592 B] 2024-09-21T14:46:08.902 INFO:teuthology.orchestra.run.smithi046.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-196 all 5.4.0-196.216 [11.0 MB] 2024-09-21T14:46:09.185 INFO:teuthology.orchestra.run.smithi046.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-196-generic amd64 5.4.0-196.216 [1,386 kB] 2024-09-21T14:46:09.199 INFO:teuthology.orchestra.run.smithi046.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.196.194 [2,460 B] 2024-09-21T14:46:09.790 INFO:teuthology.orchestra.run.smithi032.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 104373 files and directories currently installed.) 2024-09-21T14:46:09.795 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../0-linux-modules-5.4.0-196-generic_5.4.0-196.216_amd64.deb ... 2024-09-21T14:46:09.882 INFO:teuthology.orchestra.run.smithi046.stdout:Fetched 77.2 MB in 3s (23.8 MB/s) 2024-09-21T14:46:09.887 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking linux-modules-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-21T14:46:10.067 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package linux-modules-5.4.0-196-generic. 2024-09-21T14:46:11.741 INFO:teuthology.orchestra.run.smithi046.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 104373 files and directories currently installed.) 2024-09-21T14:46:11.746 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../0-linux-modules-5.4.0-196-generic_5.4.0-196.216_amd64.deb ... 2024-09-21T14:46:11.822 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking linux-modules-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-21T14:46:11.856 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package linux-image-5.4.0-196-generic. 2024-09-21T14:46:11.872 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../1-linux-image-5.4.0-196-generic_5.4.0-196.216_amd64.deb ... 2024-09-21T14:46:11.971 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking linux-image-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-21T14:46:13.068 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-196-generic. 2024-09-21T14:46:13.083 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-196-generic_5.4.0-196.216_amd64.deb ... 2024-09-21T14:46:13.126 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking linux-modules-extra-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-21T14:46:13.691 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package linux-image-5.4.0-196-generic. 2024-09-21T14:46:13.707 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../1-linux-image-5.4.0-196-generic_5.4.0-196.216_amd64.deb ... 2024-09-21T14:46:13.813 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking linux-image-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-21T14:46:14.894 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-196-generic. 2024-09-21T14:46:14.908 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-196-generic_5.4.0-196.216_amd64.deb ... 2024-09-21T14:46:14.951 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking linux-modules-extra-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-21T14:46:17.903 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../3-linux-generic_5.4.0.196.194_amd64.deb ... 2024-09-21T14:46:18.030 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking linux-generic (5.4.0.196.194) over (5.4.0.124.125) ... 2024-09-21T14:46:18.356 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.196.194_amd64.deb ... 2024-09-21T14:46:18.486 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking linux-image-generic (5.4.0.196.194) over (5.4.0.124.125) ... 2024-09-21T14:46:19.145 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package linux-headers-5.4.0-196. 2024-09-21T14:46:19.158 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../5-linux-headers-5.4.0-196_5.4.0-196.216_all.deb ... 2024-09-21T14:46:19.195 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking linux-headers-5.4.0-196 (5.4.0-196.216) ... 2024-09-21T14:46:19.970 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../3-linux-generic_5.4.0.196.194_amd64.deb ... 2024-09-21T14:46:20.088 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking linux-generic (5.4.0.196.194) over (5.4.0.124.125) ... 2024-09-21T14:46:20.438 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.196.194_amd64.deb ... 2024-09-21T14:46:20.558 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking linux-image-generic (5.4.0.196.194) over (5.4.0.124.125) ... 2024-09-21T14:46:20.911 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package linux-headers-5.4.0-196. 2024-09-21T14:46:20.926 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../5-linux-headers-5.4.0-196_5.4.0-196.216_all.deb ... 2024-09-21T14:46:20.969 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking linux-headers-5.4.0-196 (5.4.0-196.216) ... 2024-09-21T14:46:24.543 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package linux-headers-5.4.0-196-generic. 2024-09-21T14:46:24.563 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../6-linux-headers-5.4.0-196-generic_5.4.0-196.216_amd64.deb ... 2024-09-21T14:46:24.600 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking linux-headers-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-21T14:46:26.305 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package linux-headers-5.4.0-196-generic. 2024-09-21T14:46:26.329 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../6-linux-headers-5.4.0-196-generic_5.4.0-196.216_amd64.deb ... 2024-09-21T14:46:26.371 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking linux-headers-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-21T14:46:27.373 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.196.194_amd64.deb ... 2024-09-21T14:46:27.520 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking linux-headers-generic (5.4.0.196.194) over (5.4.0.124.125) ... 2024-09-21T14:46:27.862 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up linux-headers-5.4.0-196 (5.4.0-196.216) ... 2024-09-21T14:46:27.983 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up linux-headers-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-21T14:46:28.135 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up linux-headers-generic (5.4.0.196.194) ... 2024-09-21T14:46:28.261 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up linux-modules-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-21T14:46:29.169 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.196.194_amd64.deb ... 2024-09-21T14:46:29.296 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking linux-headers-generic (5.4.0.196.194) over (5.4.0.124.125) ... 2024-09-21T14:46:29.555 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up linux-image-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-21T14:46:29.650 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up linux-headers-5.4.0-196 (5.4.0-196.216) ... 2024-09-21T14:46:29.759 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up linux-headers-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-21T14:46:29.884 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up linux-headers-generic (5.4.0.196.194) ... 2024-09-21T14:46:30.010 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up linux-modules-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-21T14:46:30.751 INFO:teuthology.orchestra.run.smithi032.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-196-generic 2024-09-21T14:46:30.751 INFO:teuthology.orchestra.run.smithi032.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-196-generic 2024-09-21T14:46:30.874 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up linux-modules-extra-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-21T14:46:31.321 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up linux-image-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-21T14:46:32.092 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up linux-image-generic (5.4.0.196.194) ... 2024-09-21T14:46:32.243 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up linux-generic (5.4.0.196.194) ... 2024-09-21T14:46:32.378 INFO:teuthology.orchestra.run.smithi032.stdout:Processing triggers for linux-image-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-21T14:46:32.447 INFO:teuthology.orchestra.run.smithi032.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-21T14:46:32.447 INFO:teuthology.orchestra.run.smithi032.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-196-generic 2024-09-21T14:46:32.503 INFO:teuthology.orchestra.run.smithi046.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-196-generic 2024-09-21T14:46:32.503 INFO:teuthology.orchestra.run.smithi046.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-196-generic 2024-09-21T14:46:32.673 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up linux-modules-extra-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-21T14:46:33.874 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up linux-image-generic (5.4.0.196.194) ... 2024-09-21T14:46:33.983 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up linux-generic (5.4.0.196.194) ... 2024-09-21T14:46:34.109 INFO:teuthology.orchestra.run.smithi046.stdout:Processing triggers for linux-image-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-21T14:46:34.194 INFO:teuthology.orchestra.run.smithi046.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-21T14:46:34.194 INFO:teuthology.orchestra.run.smithi046.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-196-generic 2024-09-21T14:46:55.295 INFO:teuthology.orchestra.run.smithi032.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-21T14:46:55.295 INFO:teuthology.orchestra.run.smithi032.stdout:Sourcing file `/etc/default/grub' 2024-09-21T14:46:55.305 INFO:teuthology.orchestra.run.smithi032.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-21T14:46:55.320 INFO:teuthology.orchestra.run.smithi032.stdout:Generating grub configuration file ... 2024-09-21T14:46:55.591 INFO:teuthology.orchestra.run.smithi032.stdout:Found linux image: /boot/vmlinuz-5.4.0-196-generic 2024-09-21T14:46:55.610 INFO:teuthology.orchestra.run.smithi032.stdout:Found initrd image: /boot/initrd.img-5.4.0-196-generic 2024-09-21T14:46:55.980 INFO:teuthology.orchestra.run.smithi032.stdout:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-09-21T14:46:55.992 INFO:teuthology.orchestra.run.smithi032.stdout:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-09-21T14:46:56.385 INFO:teuthology.orchestra.run.smithi046.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-21T14:46:56.385 INFO:teuthology.orchestra.run.smithi046.stdout:Sourcing file `/etc/default/grub' 2024-09-21T14:46:56.394 INFO:teuthology.orchestra.run.smithi046.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-21T14:46:56.418 INFO:teuthology.orchestra.run.smithi046.stdout:Generating grub configuration file ... 2024-09-21T14:46:56.808 INFO:teuthology.orchestra.run.smithi046.stdout:Found linux image: /boot/vmlinuz-5.4.0-196-generic 2024-09-21T14:46:56.828 INFO:teuthology.orchestra.run.smithi046.stdout:Found initrd image: /boot/initrd.img-5.4.0-196-generic 2024-09-21T14:46:57.256 INFO:teuthology.orchestra.run.smithi046.stdout:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-09-21T14:46:57.282 INFO:teuthology.orchestra.run.smithi046.stdout:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-09-21T14:46:57.991 INFO:teuthology.orchestra.run.smithi032.stdout:done 2024-09-21T14:46:58.457 DEBUG:teuthology.orchestra.run.smithi032:> dpkg -s linux-image-generic 2024-09-21T14:46:58.503 INFO:teuthology.orchestra.run.smithi032.stdout:Package: linux-image-generic 2024-09-21T14:46:58.503 INFO:teuthology.orchestra.run.smithi032.stdout:Status: install ok installed 2024-09-21T14:46:58.503 INFO:teuthology.orchestra.run.smithi032.stdout:Priority: optional 2024-09-21T14:46:58.503 INFO:teuthology.orchestra.run.smithi032.stdout:Section: kernel 2024-09-21T14:46:58.503 INFO:teuthology.orchestra.run.smithi032.stdout:Installed-Size: 20 2024-09-21T14:46:58.503 INFO:teuthology.orchestra.run.smithi032.stdout:Maintainer: Ubuntu Kernel Team 2024-09-21T14:46:58.503 INFO:teuthology.orchestra.run.smithi032.stdout:Architecture: amd64 2024-09-21T14:46:58.504 INFO:teuthology.orchestra.run.smithi032.stdout:Source: linux-meta 2024-09-21T14:46:58.504 INFO:teuthology.orchestra.run.smithi032.stdout:Version: 5.4.0.196.194 2024-09-21T14:46:58.504 INFO:teuthology.orchestra.run.smithi032.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-09-21T14:46:58.504 INFO:teuthology.orchestra.run.smithi032.stdout:Depends: linux-image-5.4.0-196-generic, linux-modules-extra-5.4.0-196-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-21T14:46:58.504 INFO:teuthology.orchestra.run.smithi032.stdout:Recommends: thermald 2024-09-21T14:46:58.504 INFO:teuthology.orchestra.run.smithi032.stdout:Description: Generic Linux kernel image 2024-09-21T14:46:58.504 INFO:teuthology.orchestra.run.smithi032.stdout: This package will always depend on the latest generic kernel image 2024-09-21T14:46:58.504 INFO:teuthology.orchestra.run.smithi032.stdout: available. 2024-09-21T14:46:58.505 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-124-generic Expected: 5.4.0-196-generic 2024-09-21T14:46:58.505 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-21T14:46:58.505 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-21T14:46:58.505 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-21T14:46:58.505 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi032.front.sepia.ceph.com, path=None, version=distro) 2024-09-21T14:46:58.505 DEBUG:teuthology.orchestra.run.smithi032:> sudo apt-get clean 2024-09-21T14:46:58.598 DEBUG:teuthology.orchestra.run.smithi032:> sudo apt-get update 2024-09-21T14:46:58.861 INFO:teuthology.orchestra.run.smithi032.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-21T14:46:58.868 INFO:teuthology.orchestra.run.smithi032.stdout:Hit:2 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-09-21T14:46:58.931 INFO:teuthology.orchestra.run.smithi032.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-09-21T14:46:59.017 INFO:teuthology.orchestra.run.smithi032.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-09-21T14:46:59.306 INFO:teuthology.orchestra.run.smithi046.stdout:done 2024-09-21T14:46:59.751 DEBUG:teuthology.orchestra.run.smithi046:> dpkg -s linux-image-generic 2024-09-21T14:46:59.803 INFO:teuthology.orchestra.run.smithi046.stdout:Package: linux-image-generic 2024-09-21T14:46:59.803 INFO:teuthology.orchestra.run.smithi046.stdout:Status: install ok installed 2024-09-21T14:46:59.803 INFO:teuthology.orchestra.run.smithi046.stdout:Priority: optional 2024-09-21T14:46:59.803 INFO:teuthology.orchestra.run.smithi046.stdout:Section: kernel 2024-09-21T14:46:59.803 INFO:teuthology.orchestra.run.smithi046.stdout:Installed-Size: 20 2024-09-21T14:46:59.804 INFO:teuthology.orchestra.run.smithi046.stdout:Maintainer: Ubuntu Kernel Team 2024-09-21T14:46:59.804 INFO:teuthology.orchestra.run.smithi046.stdout:Architecture: amd64 2024-09-21T14:46:59.804 INFO:teuthology.orchestra.run.smithi046.stdout:Source: linux-meta 2024-09-21T14:46:59.804 INFO:teuthology.orchestra.run.smithi046.stdout:Version: 5.4.0.196.194 2024-09-21T14:46:59.804 INFO:teuthology.orchestra.run.smithi046.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-09-21T14:46:59.804 INFO:teuthology.orchestra.run.smithi046.stdout:Depends: linux-image-5.4.0-196-generic, linux-modules-extra-5.4.0-196-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-21T14:46:59.804 INFO:teuthology.orchestra.run.smithi046.stdout:Recommends: thermald 2024-09-21T14:46:59.804 INFO:teuthology.orchestra.run.smithi046.stdout:Description: Generic Linux kernel image 2024-09-21T14:46:59.804 INFO:teuthology.orchestra.run.smithi046.stdout: This package will always depend on the latest generic kernel image 2024-09-21T14:46:59.804 INFO:teuthology.orchestra.run.smithi046.stdout: available. 2024-09-21T14:46:59.804 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-124-generic Expected: 5.4.0-196-generic 2024-09-21T14:46:59.804 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-21T14:46:59.804 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-21T14:46:59.805 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-21T14:46:59.805 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi046.front.sepia.ceph.com, path=None, version=distro) 2024-09-21T14:46:59.805 DEBUG:teuthology.orchestra.run.smithi046:> sudo apt-get clean 2024-09-21T14:46:59.907 DEBUG:teuthology.orchestra.run.smithi046:> sudo apt-get update 2024-09-21T14:47:00.034 INFO:teuthology.orchestra.run.smithi046.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-21T14:47:00.034 INFO:teuthology.orchestra.run.smithi046.stdout:Hit:2 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-09-21T14:47:00.041 INFO:teuthology.orchestra.run.smithi046.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-09-21T14:47:00.058 INFO:teuthology.orchestra.run.smithi046.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-09-21T14:47:00.767 INFO:teuthology.orchestra.run.smithi032.stdout:Reading package lists... 2024-09-21T14:47:00.795 DEBUG:teuthology.orchestra.run.smithi032:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-21T14:47:00.865 INFO:teuthology.orchestra.run.smithi032.stdout:Reading package lists... 2024-09-21T14:47:01.067 INFO:teuthology.orchestra.run.smithi032.stdout:Building dependency tree... 2024-09-21T14:47:01.067 INFO:teuthology.orchestra.run.smithi032.stdout:Reading state information... 2024-09-21T14:47:01.309 INFO:teuthology.orchestra.run.smithi032.stdout:linux-image-generic is already the newest version (5.4.0.196.194). 2024-09-21T14:47:01.309 INFO:teuthology.orchestra.run.smithi032.stdout:0 upgraded, 0 newly installed, 0 to remove and 318 not upgraded. 2024-09-21T14:47:01.311 DEBUG:teuthology.orchestra.run.smithi032:> dpkg -s linux-image-generic 2024-09-21T14:47:01.333 INFO:teuthology.orchestra.run.smithi032.stdout:Package: linux-image-generic 2024-09-21T14:47:01.333 INFO:teuthology.orchestra.run.smithi032.stdout:Status: install ok installed 2024-09-21T14:47:01.333 INFO:teuthology.orchestra.run.smithi032.stdout:Priority: optional 2024-09-21T14:47:01.334 INFO:teuthology.orchestra.run.smithi032.stdout:Section: kernel 2024-09-21T14:47:01.334 INFO:teuthology.orchestra.run.smithi032.stdout:Installed-Size: 20 2024-09-21T14:47:01.334 INFO:teuthology.orchestra.run.smithi032.stdout:Maintainer: Ubuntu Kernel Team 2024-09-21T14:47:01.334 INFO:teuthology.orchestra.run.smithi032.stdout:Architecture: amd64 2024-09-21T14:47:01.334 INFO:teuthology.orchestra.run.smithi032.stdout:Source: linux-meta 2024-09-21T14:47:01.334 INFO:teuthology.orchestra.run.smithi032.stdout:Version: 5.4.0.196.194 2024-09-21T14:47:01.334 INFO:teuthology.orchestra.run.smithi032.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-09-21T14:47:01.334 INFO:teuthology.orchestra.run.smithi032.stdout:Depends: linux-image-5.4.0-196-generic, linux-modules-extra-5.4.0-196-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-21T14:47:01.334 INFO:teuthology.orchestra.run.smithi032.stdout:Recommends: thermald 2024-09-21T14:47:01.334 INFO:teuthology.orchestra.run.smithi032.stdout:Description: Generic Linux kernel image 2024-09-21T14:47:01.334 INFO:teuthology.orchestra.run.smithi032.stdout: This package will always depend on the latest generic kernel image 2024-09-21T14:47:01.334 INFO:teuthology.orchestra.run.smithi032.stdout: available. 2024-09-21T14:47:01.335 DEBUG:teuthology.orchestra.run.smithi032:> mktemp 2024-09-21T14:47:01.339 INFO:teuthology.orchestra.run.smithi032.stdout:/tmp/tmp.14IR31lGpk 2024-09-21T14:47:01.339 DEBUG:teuthology.orchestra.run.smithi032:> sudo cp /boot/grub/grub.cfg /tmp/tmp.14IR31lGpk 2024-09-21T14:47:01.402 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 0666 /tmp/tmp.14IR31lGpk 2024-09-21T14:47:01.438 DEBUG:teuthology.orchestra.remote:smithi032:/tmp/tmp.14IR31lGpk is 9KB 2024-09-21T14:47:01.452 DEBUG:teuthology.orchestra.run.smithi032:> rm -fr /tmp/tmp.14IR31lGpk 2024-09-21T14:47:01.458 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-21T14:47:01.512 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-09-21T14:47:01.512 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-21T14:47:01.512 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-21T14:47:01.550 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-196-generic 2024-09-21T14:47:01.550 DEBUG:teuthology.orchestra.run.smithi032:> sudo update-grub 2024-09-21T14:47:01.977 INFO:teuthology.orchestra.run.smithi046.stdout:Reading package lists... 2024-09-21T14:47:02.006 DEBUG:teuthology.orchestra.run.smithi046:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-21T14:47:02.077 INFO:teuthology.orchestra.run.smithi046.stdout:Reading package lists... 2024-09-21T14:47:02.300 INFO:teuthology.orchestra.run.smithi046.stdout:Building dependency tree... 2024-09-21T14:47:02.301 INFO:teuthology.orchestra.run.smithi046.stdout:Reading state information... 2024-09-21T14:47:02.557 INFO:teuthology.orchestra.run.smithi046.stdout:linux-image-generic is already the newest version (5.4.0.196.194). 2024-09-21T14:47:02.557 INFO:teuthology.orchestra.run.smithi046.stdout:0 upgraded, 0 newly installed, 0 to remove and 318 not upgraded. 2024-09-21T14:47:02.559 DEBUG:teuthology.orchestra.run.smithi046:> dpkg -s linux-image-generic 2024-09-21T14:47:02.581 INFO:teuthology.orchestra.run.smithi046.stdout:Package: linux-image-generic 2024-09-21T14:47:02.581 INFO:teuthology.orchestra.run.smithi046.stdout:Status: install ok installed 2024-09-21T14:47:02.581 INFO:teuthology.orchestra.run.smithi046.stdout:Priority: optional 2024-09-21T14:47:02.581 INFO:teuthology.orchestra.run.smithi046.stdout:Section: kernel 2024-09-21T14:47:02.581 INFO:teuthology.orchestra.run.smithi046.stdout:Installed-Size: 20 2024-09-21T14:47:02.582 INFO:teuthology.orchestra.run.smithi046.stdout:Maintainer: Ubuntu Kernel Team 2024-09-21T14:47:02.582 INFO:teuthology.orchestra.run.smithi046.stdout:Architecture: amd64 2024-09-21T14:47:02.582 INFO:teuthology.orchestra.run.smithi046.stdout:Source: linux-meta 2024-09-21T14:47:02.582 INFO:teuthology.orchestra.run.smithi046.stdout:Version: 5.4.0.196.194 2024-09-21T14:47:02.582 INFO:teuthology.orchestra.run.smithi046.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-09-21T14:47:02.582 INFO:teuthology.orchestra.run.smithi046.stdout:Depends: linux-image-5.4.0-196-generic, linux-modules-extra-5.4.0-196-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-21T14:47:02.582 INFO:teuthology.orchestra.run.smithi046.stdout:Recommends: thermald 2024-09-21T14:47:02.582 INFO:teuthology.orchestra.run.smithi046.stdout:Description: Generic Linux kernel image 2024-09-21T14:47:02.582 INFO:teuthology.orchestra.run.smithi046.stdout: This package will always depend on the latest generic kernel image 2024-09-21T14:47:02.582 INFO:teuthology.orchestra.run.smithi046.stdout: available. 2024-09-21T14:47:02.582 DEBUG:teuthology.orchestra.run.smithi046:> mktemp 2024-09-21T14:47:02.588 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/tmp.NLuwKP3bsU 2024-09-21T14:47:02.589 DEBUG:teuthology.orchestra.run.smithi046:> sudo cp /boot/grub/grub.cfg /tmp/tmp.NLuwKP3bsU 2024-09-21T14:47:02.648 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 0666 /tmp/tmp.NLuwKP3bsU 2024-09-21T14:47:02.684 DEBUG:teuthology.orchestra.remote:smithi046:/tmp/tmp.NLuwKP3bsU is 9KB 2024-09-21T14:47:02.696 DEBUG:teuthology.orchestra.run.smithi046:> rm -fr /tmp/tmp.NLuwKP3bsU 2024-09-21T14:47:02.702 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-21T14:47:02.759 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-21T14:47:02.759 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-21T14:47:02.759 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-21T14:47:02.796 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-196-generic 2024-09-21T14:47:02.796 DEBUG:teuthology.orchestra.run.smithi046:> sudo update-grub 2024-09-21T14:47:02.944 INFO:teuthology.orchestra.run.smithi032.stderr:Sourcing file `/etc/default/grub' 2024-09-21T14:47:02.952 INFO:teuthology.orchestra.run.smithi032.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-21T14:47:02.961 INFO:teuthology.orchestra.run.smithi032.stderr:Generating grub configuration file ... 2024-09-21T14:47:03.227 INFO:teuthology.orchestra.run.smithi032.stderr:Found linux image: /boot/vmlinuz-5.4.0-196-generic 2024-09-21T14:47:03.238 INFO:teuthology.orchestra.run.smithi032.stderr:Found initrd image: /boot/initrd.img-5.4.0-196-generic 2024-09-21T14:47:03.592 INFO:teuthology.orchestra.run.smithi032.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-09-21T14:47:03.604 INFO:teuthology.orchestra.run.smithi032.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-09-21T14:47:04.105 INFO:teuthology.orchestra.run.smithi046.stderr:Sourcing file `/etc/default/grub' 2024-09-21T14:47:04.115 INFO:teuthology.orchestra.run.smithi046.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-21T14:47:04.159 INFO:teuthology.orchestra.run.smithi046.stderr:Generating grub configuration file ... 2024-09-21T14:47:04.456 INFO:teuthology.orchestra.run.smithi046.stderr:Found linux image: /boot/vmlinuz-5.4.0-196-generic 2024-09-21T14:47:04.467 INFO:teuthology.orchestra.run.smithi046.stderr:Found initrd image: /boot/initrd.img-5.4.0-196-generic 2024-09-21T14:47:04.601 INFO:teuthology.orchestra.run.smithi032.stderr:done 2024-09-21T14:47:04.603 DEBUG:teuthology.orchestra.run.smithi032:> sudo shutdown -r now 2024-09-21T14:47:04.913 INFO:teuthology.orchestra.run.smithi046.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-09-21T14:47:04.925 INFO:teuthology.orchestra.run.smithi046.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-09-21T14:47:05.980 INFO:teuthology.orchestra.run.smithi046.stderr:done 2024-09-21T14:47:05.982 DEBUG:teuthology.orchestra.run.smithi046:> sudo shutdown -r now 2024-09-21T14:47:34.606 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-21T14:47:34.607 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi032.front.sepia.ceph.com' 2024-09-21T14:47:34.607 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T14:47:35.986 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-21T14:47:35.986 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-09-21T14:47:35.987 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T14:48:10.479 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.46 2024-09-21T14:48:19.486 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-09-21T14:48:19.487 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T14:48:34.624 DEBUG:teuthology.orchestra.remote:timed out 2024-09-21T14:48:43.630 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi032.front.sepia.ceph.com' 2024-09-21T14:48:43.631 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T14:48:43.872 DEBUG:teuthology.orchestra.run.smithi032:> true 2024-09-21T14:48:44.479 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi032.front.sepia.ceph.com' 2024-09-21T14:48:44.480 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.4.0-196-generic"... 2024-09-21T14:48:44.480 DEBUG:teuthology.orchestra.run.smithi032:> uname -r 2024-09-21T14:48:44.486 INFO:teuthology.orchestra.run.smithi032.stdout:5.4.0-196-generic 2024-09-21T14:48:44.486 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-196-generic vs 5.4.0-196-generic 2024-09-21T14:48:44.486 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-21T14:48:44.487 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-21T14:48:45.488 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-21T14:48:45.488 DEBUG:teuthology.orchestra.run.smithi032:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-21T14:48:45.597 INFO:teuthology.orchestra.run.smithi032.stdout:ttyS1 2024-09-21T14:48:45.625 DEBUG:teuthology.parallel:result is None 2024-09-21T14:49:19.490 DEBUG:teuthology.orchestra.remote:timed out 2024-09-21T14:49:31.491 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-09-21T14:49:31.491 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T14:49:31.749 DEBUG:teuthology.orchestra.run.smithi046:> true 2024-09-21T14:49:32.396 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-09-21T14:49:32.396 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.4.0-196-generic"... 2024-09-21T14:49:32.396 DEBUG:teuthology.orchestra.run.smithi046:> uname -r 2024-09-21T14:49:32.445 INFO:teuthology.orchestra.run.smithi046.stdout:5.4.0-196-generic 2024-09-21T14:49:32.445 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-196-generic vs 5.4.0-196-generic 2024-09-21T14:49:32.445 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-21T14:49:32.445 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-21T14:49:33.446 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-21T14:49:33.446 DEBUG:teuthology.orchestra.run.smithi046:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-21T14:49:33.544 INFO:teuthology.orchestra.run.smithi046.stdout:ttyS1 2024-09-21T14:49:33.578 DEBUG:teuthology.parallel:result is None 2024-09-21T14:49:33.578 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-21T14:49:33.588 INFO:teuthology.task.internal:Creating test directory... 2024-09-21T14:49:33.588 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T14:49:33.591 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T14:49:33.599 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-21T14:49:33.608 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-21T14:49:33.622 INFO:teuthology.task.internal:Creating archive directory... 2024-09-21T14:49:33.622 DEBUG:teuthology.orchestra.run.smithi032:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T14:49:33.641 DEBUG:teuthology.orchestra.run.smithi046:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T14:49:33.676 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-21T14:49:33.686 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-21T14:49:33.686 DEBUG:teuthology.orchestra.run.smithi032:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T14:49:33.689 DEBUG:teuthology.orchestra.run.smithi046:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T14:49:33.707 INFO:teuthology.orchestra.run.smithi046.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T14:49:33.711 INFO:teuthology.orchestra.run.smithi032.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T14:49:33.718 INFO:teuthology.orchestra.run.smithi046.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T14:49:33.723 INFO:teuthology.orchestra.run.smithi032.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T14:49:33.724 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-21T14:49:33.729 INFO:teuthology.task.internal:Configuring sudo... 2024-09-21T14:49:33.730 DEBUG:teuthology.orchestra.run.smithi032:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T14:49:33.732 DEBUG:teuthology.orchestra.run.smithi046:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T14:49:33.779 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-21T14:49:33.788 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-21T14:49:33.788 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T14:49:33.790 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T14:49:33.796 DEBUG:teuthology.orchestra.run.smithi032:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T14:49:33.802 DEBUG:teuthology.orchestra.run.smithi032:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T14:49:33.849 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-09-21T14:49:33.849 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T14:49:33.870 DEBUG:teuthology.orchestra.run.smithi046:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T14:49:33.878 DEBUG:teuthology.orchestra.run.smithi046:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T14:49:33.884 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-21T14:49:33.884 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T14:49:33.909 DEBUG:teuthology.orchestra.run.smithi032:> sudo service rsyslog restart 2024-09-21T14:49:33.911 DEBUG:teuthology.orchestra.run.smithi046:> sudo service rsyslog restart 2024-09-21T14:49:33.961 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-21T14:49:33.968 INFO:teuthology.task.internal:Starting timer... 2024-09-21T14:49:33.968 INFO:teuthology.run_tasks:Running task pcp... 2024-09-21T14:49:33.978 INFO:teuthology.run_tasks:Running task selinux... 2024-09-21T14:49:33.985 DEBUG:teuthology.task.selinux:Excluding smithi032: OS 'ubuntu' does not support SELinux 2024-09-21T14:49:33.985 DEBUG:teuthology.task.selinux:Excluding smithi046: OS 'ubuntu' does not support SELinux 2024-09-21T14:49:33.985 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-21T14:49:33.985 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-09-21T14:49:33.985 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-21T14:49:33.986 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-21T14:49:33.993 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-21T14:49:35.466 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-21T14:49:35.587 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-21T14:49:35.596 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-21T14:49:35.596 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi032.front.sepia.ceph.com,smithi046.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-21T14:55:09.722 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi032.front.sepia.ceph.com'), Remote(name='ubuntu@smithi046.front.sepia.ceph.com')] 2024-09-21T14:55:09.723 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi032.front.sepia.ceph.com' 2024-09-21T14:55:09.724 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T14:55:09.818 DEBUG:teuthology.orchestra.run.smithi032:> true 2024-09-21T14:55:09.940 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi032.front.sepia.ceph.com' 2024-09-21T14:55:09.940 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-09-21T14:55:09.941 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T14:55:10.028 DEBUG:teuthology.orchestra.run.smithi046:> true 2024-09-21T14:55:10.152 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-09-21T14:55:10.152 INFO:teuthology.run_tasks:Running task clock... 2024-09-21T14:55:10.163 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-21T14:55:10.163 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T14:55:10.163 DEBUG:teuthology.orchestra.run.smithi032:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T14:55:10.166 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T14:55:10.166 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T14:55:10.197 INFO:teuthology.orchestra.run.smithi032.stdout:21 Sep 14:55:10 ntpd[13682]: ntpd 4.2.8p12@1.3728-o (1): Starting 2024-09-21T14:55:10.197 INFO:teuthology.orchestra.run.smithi032.stdout:21 Sep 14:55:10 ntpd[13682]: Command line: ntpd -gq 2024-09-21T14:55:10.199 INFO:teuthology.orchestra.run.smithi032.stdout:21 Sep 14:55:10 ntpd[13682]: proto: precision = 0.120 usec (-23) 2024-09-21T14:55:10.199 INFO:teuthology.orchestra.run.smithi032.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-21T14:55:10.200 INFO:teuthology.orchestra.run.smithi032.stdout:21 Sep 14:55:10 ntpd[13682]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-21T14:55:10.200 INFO:teuthology.orchestra.run.smithi032.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-21T14:55:10.201 INFO:teuthology.orchestra.run.smithi032.stdout:21 Sep 14:55:10 ntpd[13682]: restrict ::: KOD does nothing without LIMITED. 2024-09-21T14:55:10.201 INFO:teuthology.orchestra.run.smithi032.stdout:21 Sep 14:55:10 ntpd[13682]: Listen and drop on 0 v6wildcard [::]:123 2024-09-21T14:55:10.201 INFO:teuthology.orchestra.run.smithi032.stdout:21 Sep 14:55:10 ntpd[13682]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-21T14:55:10.201 INFO:teuthology.orchestra.run.smithi032.stdout:21 Sep 14:55:10 ntpd[13682]: Listen normally on 2 lo 127.0.0.1:123 2024-09-21T14:55:10.201 INFO:teuthology.orchestra.run.smithi032.stdout:21 Sep 14:55:10 ntpd[13682]: Listen normally on 3 ens1f0 172.21.15.32:123 2024-09-21T14:55:10.201 INFO:teuthology.orchestra.run.smithi032.stdout:21 Sep 14:55:10 ntpd[13682]: Listen normally on 4 lo [::1]:123 2024-09-21T14:55:10.201 INFO:teuthology.orchestra.run.smithi032.stdout:21 Sep 14:55:10 ntpd[13682]: Listen normally on 5 ens1f0 [fe80::ae1f:6bff:fef8:2602%4]:123 2024-09-21T14:55:10.201 INFO:teuthology.orchestra.run.smithi032.stdout:21 Sep 14:55:10 ntpd[13682]: Listening on routing socket on fd #22 for interface updates 2024-09-21T14:55:10.202 INFO:teuthology.orchestra.run.smithi046.stdout:21 Sep 14:55:10 ntpd[13657]: ntpd 4.2.8p12@1.3728-o (1): Starting 2024-09-21T14:55:10.202 INFO:teuthology.orchestra.run.smithi046.stdout:21 Sep 14:55:10 ntpd[13657]: Command line: ntpd -gq 2024-09-21T14:55:10.202 INFO:teuthology.orchestra.run.smithi046.stdout:21 Sep 14:55:10 ntpd[13657]: proto: precision = 0.072 usec (-24) 2024-09-21T14:55:10.202 INFO:teuthology.orchestra.run.smithi046.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-21T14:55:10.203 INFO:teuthology.orchestra.run.smithi046.stdout:21 Sep 14:55:10 ntpd[13657]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-21T14:55:10.203 INFO:teuthology.orchestra.run.smithi046.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-21T14:55:10.204 INFO:teuthology.orchestra.run.smithi046.stdout:21 Sep 14:55:10 ntpd[13657]: restrict ::: KOD does nothing without LIMITED. 2024-09-21T14:55:10.204 INFO:teuthology.orchestra.run.smithi046.stdout:21 Sep 14:55:10 ntpd[13657]: Listen and drop on 0 v6wildcard [::]:123 2024-09-21T14:55:10.204 INFO:teuthology.orchestra.run.smithi046.stdout:21 Sep 14:55:10 ntpd[13657]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-21T14:55:10.204 INFO:teuthology.orchestra.run.smithi046.stdout:21 Sep 14:55:10 ntpd[13657]: Listen normally on 2 lo 127.0.0.1:123 2024-09-21T14:55:10.204 INFO:teuthology.orchestra.run.smithi046.stdout:21 Sep 14:55:10 ntpd[13657]: Listen normally on 3 ens1f0 172.21.15.46:123 2024-09-21T14:55:10.204 INFO:teuthology.orchestra.run.smithi046.stdout:21 Sep 14:55:10 ntpd[13657]: Listen normally on 4 lo [::1]:123 2024-09-21T14:55:10.204 INFO:teuthology.orchestra.run.smithi046.stdout:21 Sep 14:55:10 ntpd[13657]: Listen normally on 5 ens1f0 [fe80::ec4:7aff:febd:15f8%4]:123 2024-09-21T14:55:10.204 INFO:teuthology.orchestra.run.smithi046.stdout:21 Sep 14:55:10 ntpd[13657]: Listening on routing socket on fd #22 for interface updates 2024-09-21T14:55:11.199 INFO:teuthology.orchestra.run.smithi032.stderr:21 Sep 14:55:11 ntpd[13682]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T14:55:11.199 INFO:teuthology.orchestra.run.smithi032.stderr:21 Sep 14:55:11 ntpd[13682]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T14:55:11.200 INFO:teuthology.orchestra.run.smithi032.stderr:21 Sep 14:55:11 ntpd[13682]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T14:55:11.200 INFO:teuthology.orchestra.run.smithi032.stderr:21 Sep 14:55:11 ntpd[13682]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T14:55:11.200 INFO:teuthology.orchestra.run.smithi032.stderr:21 Sep 14:55:11 ntpd[13682]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T14:55:11.200 INFO:teuthology.orchestra.run.smithi032.stderr:21 Sep 14:55:11 ntpd[13682]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T14:55:11.200 INFO:teuthology.orchestra.run.smithi032.stderr:21 Sep 14:55:11 ntpd[13682]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T14:55:11.200 INFO:teuthology.orchestra.run.smithi032.stderr:21 Sep 14:55:11 ntpd[13682]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T14:55:11.200 INFO:teuthology.orchestra.run.smithi032.stderr:21 Sep 14:55:11 ntpd[13682]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T14:55:11.200 INFO:teuthology.orchestra.run.smithi032.stderr:21 Sep 14:55:11 ntpd[13682]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T14:55:11.200 INFO:teuthology.orchestra.run.smithi032.stderr:21 Sep 14:55:11 ntpd[13682]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T14:55:11.200 INFO:teuthology.orchestra.run.smithi032.stderr:21 Sep 14:55:11 ntpd[13682]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T14:55:11.200 INFO:teuthology.orchestra.run.smithi032.stderr:21 Sep 14:55:11 ntpd[13682]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T14:55:11.200 INFO:teuthology.orchestra.run.smithi032.stderr:21 Sep 14:55:11 ntpd[13682]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T14:55:11.200 INFO:teuthology.orchestra.run.smithi032.stderr:21 Sep 14:55:11 ntpd[13682]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T14:55:11.200 INFO:teuthology.orchestra.run.smithi032.stderr:21 Sep 14:55:11 ntpd[13682]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T14:55:11.201 INFO:teuthology.orchestra.run.smithi032.stderr:21 Sep 14:55:11 ntpd[13682]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T14:55:11.201 INFO:teuthology.orchestra.run.smithi032.stderr:21 Sep 14:55:11 ntpd[13682]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T14:55:12.198 INFO:teuthology.orchestra.run.smithi032.stderr:21 Sep 14:55:12 ntpd[13682]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T14:55:12.198 INFO:teuthology.orchestra.run.smithi032.stderr:21 Sep 14:55:12 ntpd[13682]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T14:55:12.201 INFO:teuthology.orchestra.run.smithi046.stderr:21 Sep 14:55:12 ntpd[13657]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T14:55:12.201 INFO:teuthology.orchestra.run.smithi046.stderr:21 Sep 14:55:12 ntpd[13657]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T14:55:13.199 INFO:teuthology.orchestra.run.smithi032.stderr:21 Sep 14:55:13 ntpd[13682]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T14:55:13.200 INFO:teuthology.orchestra.run.smithi032.stderr:21 Sep 14:55:13 ntpd[13682]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T14:55:13.200 INFO:teuthology.orchestra.run.smithi032.stderr:21 Sep 14:55:13 ntpd[13682]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T14:55:13.200 INFO:teuthology.orchestra.run.smithi032.stderr:21 Sep 14:55:13 ntpd[13682]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T14:55:13.200 INFO:teuthology.orchestra.run.smithi032.stderr:21 Sep 14:55:13 ntpd[13682]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T14:55:13.200 INFO:teuthology.orchestra.run.smithi032.stderr:21 Sep 14:55:13 ntpd[13682]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T14:55:13.200 INFO:teuthology.orchestra.run.smithi032.stderr:21 Sep 14:55:13 ntpd[13682]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T14:55:13.200 INFO:teuthology.orchestra.run.smithi032.stderr:21 Sep 14:55:13 ntpd[13682]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T14:55:13.200 INFO:teuthology.orchestra.run.smithi032.stderr:21 Sep 14:55:13 ntpd[13682]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T14:55:13.200 INFO:teuthology.orchestra.run.smithi032.stderr:21 Sep 14:55:13 ntpd[13682]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T14:55:13.200 INFO:teuthology.orchestra.run.smithi032.stderr:21 Sep 14:55:13 ntpd[13682]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T14:55:13.201 INFO:teuthology.orchestra.run.smithi032.stderr:21 Sep 14:55:13 ntpd[13682]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T14:55:13.201 INFO:teuthology.orchestra.run.smithi032.stderr:21 Sep 14:55:13 ntpd[13682]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T14:55:13.201 INFO:teuthology.orchestra.run.smithi032.stderr:21 Sep 14:55:13 ntpd[13682]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T14:55:13.201 INFO:teuthology.orchestra.run.smithi032.stderr:21 Sep 14:55:13 ntpd[13682]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T14:55:13.201 INFO:teuthology.orchestra.run.smithi032.stderr:21 Sep 14:55:13 ntpd[13682]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T14:55:13.201 INFO:teuthology.orchestra.run.smithi032.stderr:21 Sep 14:55:13 ntpd[13682]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T14:55:13.201 INFO:teuthology.orchestra.run.smithi032.stderr:21 Sep 14:55:13 ntpd[13682]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T14:55:14.202 INFO:teuthology.orchestra.run.smithi046.stderr:21 Sep 14:55:14 ntpd[13657]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T14:55:14.202 INFO:teuthology.orchestra.run.smithi046.stderr:21 Sep 14:55:14 ntpd[13657]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T14:55:14.202 INFO:teuthology.orchestra.run.smithi046.stderr:21 Sep 14:55:14 ntpd[13657]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T14:55:14.202 INFO:teuthology.orchestra.run.smithi046.stderr:21 Sep 14:55:14 ntpd[13657]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T14:55:14.202 INFO:teuthology.orchestra.run.smithi046.stderr:21 Sep 14:55:14 ntpd[13657]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T14:55:14.202 INFO:teuthology.orchestra.run.smithi046.stderr:21 Sep 14:55:14 ntpd[13657]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T14:55:14.203 INFO:teuthology.orchestra.run.smithi046.stderr:21 Sep 14:55:14 ntpd[13657]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T14:55:14.203 INFO:teuthology.orchestra.run.smithi046.stderr:21 Sep 14:55:14 ntpd[13657]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T14:55:14.203 INFO:teuthology.orchestra.run.smithi046.stderr:21 Sep 14:55:14 ntpd[13657]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T14:55:14.203 INFO:teuthology.orchestra.run.smithi046.stderr:21 Sep 14:55:14 ntpd[13657]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T14:55:14.203 INFO:teuthology.orchestra.run.smithi046.stderr:21 Sep 14:55:14 ntpd[13657]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T14:55:14.203 INFO:teuthology.orchestra.run.smithi046.stderr:21 Sep 14:55:14 ntpd[13657]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T14:55:15.199 INFO:teuthology.orchestra.run.smithi032.stderr:21 Sep 14:55:15 ntpd[13682]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T14:55:15.199 INFO:teuthology.orchestra.run.smithi032.stderr:21 Sep 14:55:15 ntpd[13682]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T14:55:15.199 INFO:teuthology.orchestra.run.smithi032.stderr:21 Sep 14:55:15 ntpd[13682]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T14:55:15.199 INFO:teuthology.orchestra.run.smithi032.stderr:21 Sep 14:55:15 ntpd[13682]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T14:55:15.200 INFO:teuthology.orchestra.run.smithi032.stderr:21 Sep 14:55:15 ntpd[13682]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T14:55:15.200 INFO:teuthology.orchestra.run.smithi032.stderr:21 Sep 14:55:15 ntpd[13682]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T14:55:15.200 INFO:teuthology.orchestra.run.smithi032.stderr:21 Sep 14:55:15 ntpd[13682]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T14:55:15.200 INFO:teuthology.orchestra.run.smithi032.stderr:21 Sep 14:55:15 ntpd[13682]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T14:55:15.200 INFO:teuthology.orchestra.run.smithi032.stderr:21 Sep 14:55:15 ntpd[13682]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T14:55:15.200 INFO:teuthology.orchestra.run.smithi032.stderr:21 Sep 14:55:15 ntpd[13682]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T14:55:15.200 INFO:teuthology.orchestra.run.smithi032.stderr:21 Sep 14:55:15 ntpd[13682]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T14:55:15.200 INFO:teuthology.orchestra.run.smithi032.stderr:21 Sep 14:55:15 ntpd[13682]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T14:55:15.200 INFO:teuthology.orchestra.run.smithi032.stderr:21 Sep 14:55:15 ntpd[13682]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T14:55:15.200 INFO:teuthology.orchestra.run.smithi032.stderr:21 Sep 14:55:15 ntpd[13682]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T14:55:15.200 INFO:teuthology.orchestra.run.smithi032.stderr:21 Sep 14:55:15 ntpd[13682]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T14:55:15.200 INFO:teuthology.orchestra.run.smithi032.stderr:21 Sep 14:55:15 ntpd[13682]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T14:55:15.200 INFO:teuthology.orchestra.run.smithi032.stderr:21 Sep 14:55:15 ntpd[13682]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T14:55:15.201 INFO:teuthology.orchestra.run.smithi032.stderr:21 Sep 14:55:15 ntpd[13682]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T14:55:15.201 INFO:teuthology.orchestra.run.smithi046.stderr:21 Sep 14:55:15 ntpd[13657]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T14:55:15.201 INFO:teuthology.orchestra.run.smithi046.stderr:21 Sep 14:55:15 ntpd[13657]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T14:55:15.201 INFO:teuthology.orchestra.run.smithi046.stderr:21 Sep 14:55:15 ntpd[13657]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T14:55:15.202 INFO:teuthology.orchestra.run.smithi046.stderr:21 Sep 14:55:15 ntpd[13657]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T14:55:15.202 INFO:teuthology.orchestra.run.smithi046.stderr:21 Sep 14:55:15 ntpd[13657]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T14:55:15.202 INFO:teuthology.orchestra.run.smithi046.stderr:21 Sep 14:55:15 ntpd[13657]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T14:55:16.202 INFO:teuthology.orchestra.run.smithi046.stderr:21 Sep 14:55:16 ntpd[13657]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T14:55:16.202 INFO:teuthology.orchestra.run.smithi046.stderr:21 Sep 14:55:16 ntpd[13657]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T14:55:16.202 INFO:teuthology.orchestra.run.smithi046.stderr:21 Sep 14:55:16 ntpd[13657]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T14:55:16.203 INFO:teuthology.orchestra.run.smithi046.stderr:21 Sep 14:55:16 ntpd[13657]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T14:55:16.203 INFO:teuthology.orchestra.run.smithi046.stderr:21 Sep 14:55:16 ntpd[13657]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T14:55:16.203 INFO:teuthology.orchestra.run.smithi046.stderr:21 Sep 14:55:16 ntpd[13657]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T14:55:16.203 INFO:teuthology.orchestra.run.smithi046.stderr:21 Sep 14:55:16 ntpd[13657]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T14:55:16.203 INFO:teuthology.orchestra.run.smithi046.stderr:21 Sep 14:55:16 ntpd[13657]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T14:55:16.203 INFO:teuthology.orchestra.run.smithi046.stderr:21 Sep 14:55:16 ntpd[13657]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T14:55:16.203 INFO:teuthology.orchestra.run.smithi046.stderr:21 Sep 14:55:16 ntpd[13657]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T14:55:16.203 INFO:teuthology.orchestra.run.smithi046.stderr:21 Sep 14:55:16 ntpd[13657]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T14:55:16.203 INFO:teuthology.orchestra.run.smithi046.stderr:21 Sep 14:55:16 ntpd[13657]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T14:55:17.198 INFO:teuthology.orchestra.run.smithi032.stderr:21 Sep 14:55:17 ntpd[13682]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T14:55:17.199 INFO:teuthology.orchestra.run.smithi032.stderr:21 Sep 14:55:17 ntpd[13682]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T14:55:17.199 INFO:teuthology.orchestra.run.smithi032.stderr:21 Sep 14:55:17 ntpd[13682]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T14:55:17.199 INFO:teuthology.orchestra.run.smithi032.stderr:21 Sep 14:55:17 ntpd[13682]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T14:55:17.199 INFO:teuthology.orchestra.run.smithi032.stderr:21 Sep 14:55:17 ntpd[13682]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T14:55:17.199 INFO:teuthology.orchestra.run.smithi032.stderr:21 Sep 14:55:17 ntpd[13682]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T14:55:17.199 INFO:teuthology.orchestra.run.smithi032.stderr:21 Sep 14:55:17 ntpd[13682]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T14:55:17.199 INFO:teuthology.orchestra.run.smithi032.stderr:21 Sep 14:55:17 ntpd[13682]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T14:55:17.199 INFO:teuthology.orchestra.run.smithi032.stderr:21 Sep 14:55:17 ntpd[13682]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T14:55:17.199 INFO:teuthology.orchestra.run.smithi032.stderr:21 Sep 14:55:17 ntpd[13682]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T14:55:17.200 INFO:teuthology.orchestra.run.smithi032.stdout:21 Sep 14:55:17 ntpd[13682]: ntpd: time slew +0.001205 s 2024-09-21T14:55:17.200 INFO:teuthology.orchestra.run.smithi032.stdout:ntpd: time slew +0.001205s 2024-09-21T14:55:17.200 INFO:teuthology.orchestra.run.smithi032.stderr:21 Sep 14:55:17 ntpd[13682]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-21T14:55:17.200 INFO:teuthology.orchestra.run.smithi032.stderr:21 Sep 14:55:17 ntpd[13682]: can't open /var/log/ntpstats/loopstats.20240921: Permission denied 2024-09-21T14:55:17.201 INFO:teuthology.orchestra.run.smithi046.stderr:21 Sep 14:55:17 ntpd[13657]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T14:55:17.201 INFO:teuthology.orchestra.run.smithi046.stderr:21 Sep 14:55:17 ntpd[13657]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T14:55:17.201 INFO:teuthology.orchestra.run.smithi046.stderr:21 Sep 14:55:17 ntpd[13657]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T14:55:17.202 INFO:teuthology.orchestra.run.smithi046.stderr:21 Sep 14:55:17 ntpd[13657]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T14:55:17.202 INFO:teuthology.orchestra.run.smithi046.stderr:21 Sep 14:55:17 ntpd[13657]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T14:55:17.202 INFO:teuthology.orchestra.run.smithi046.stderr:21 Sep 14:55:17 ntpd[13657]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T14:55:17.265 INFO:teuthology.orchestra.run.smithi032.stdout: remote refid st t when poll reach delay offset jitter 2024-09-21T14:55:17.265 INFO:teuthology.orchestra.run.smithi032.stdout:============================================================================== 2024-09-21T14:55:17.265 INFO:teuthology.orchestra.run.smithi032.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-21T14:55:17.265 INFO:teuthology.orchestra.run.smithi032.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-21T14:55:17.265 INFO:teuthology.orchestra.run.smithi032.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-21T14:55:17.266 INFO:teuthology.orchestra.run.smithi032.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-21T14:55:18.202 INFO:teuthology.orchestra.run.smithi046.stderr:21 Sep 14:55:18 ntpd[13657]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T14:55:18.202 INFO:teuthology.orchestra.run.smithi046.stderr:21 Sep 14:55:18 ntpd[13657]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T14:55:18.202 INFO:teuthology.orchestra.run.smithi046.stderr:21 Sep 14:55:18 ntpd[13657]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T14:55:18.202 INFO:teuthology.orchestra.run.smithi046.stderr:21 Sep 14:55:18 ntpd[13657]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T14:55:18.203 INFO:teuthology.orchestra.run.smithi046.stderr:21 Sep 14:55:18 ntpd[13657]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T14:55:18.203 INFO:teuthology.orchestra.run.smithi046.stderr:21 Sep 14:55:18 ntpd[13657]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T14:55:18.203 INFO:teuthology.orchestra.run.smithi046.stderr:21 Sep 14:55:18 ntpd[13657]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T14:55:18.203 INFO:teuthology.orchestra.run.smithi046.stderr:21 Sep 14:55:18 ntpd[13657]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T14:55:18.203 INFO:teuthology.orchestra.run.smithi046.stderr:21 Sep 14:55:18 ntpd[13657]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T14:55:18.203 INFO:teuthology.orchestra.run.smithi046.stderr:21 Sep 14:55:18 ntpd[13657]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T14:55:18.203 INFO:teuthology.orchestra.run.smithi046.stderr:21 Sep 14:55:18 ntpd[13657]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T14:55:18.203 INFO:teuthology.orchestra.run.smithi046.stderr:21 Sep 14:55:18 ntpd[13657]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T14:55:19.201 INFO:teuthology.orchestra.run.smithi046.stderr:21 Sep 14:55:19 ntpd[13657]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T14:55:19.303 INFO:teuthology.orchestra.run.smithi046.stderr:21 Sep 14:55:19 ntpd[13657]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T14:55:19.304 INFO:teuthology.orchestra.run.smithi046.stderr:21 Sep 14:55:19 ntpd[13657]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T14:55:19.304 INFO:teuthology.orchestra.run.smithi046.stderr:21 Sep 14:55:19 ntpd[13657]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T14:55:19.304 INFO:teuthology.orchestra.run.smithi046.stderr:21 Sep 14:55:19 ntpd[13657]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T14:55:19.304 INFO:teuthology.orchestra.run.smithi046.stderr:21 Sep 14:55:19 ntpd[13657]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T14:55:20.202 INFO:teuthology.orchestra.run.smithi046.stderr:21 Sep 14:55:20 ntpd[13657]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T14:55:20.202 INFO:teuthology.orchestra.run.smithi046.stderr:21 Sep 14:55:20 ntpd[13657]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T14:55:20.202 INFO:teuthology.orchestra.run.smithi046.stderr:21 Sep 14:55:20 ntpd[13657]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T14:55:20.202 INFO:teuthology.orchestra.run.smithi046.stderr:21 Sep 14:55:20 ntpd[13657]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T14:55:20.202 INFO:teuthology.orchestra.run.smithi046.stderr:21 Sep 14:55:20 ntpd[13657]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T14:55:20.202 INFO:teuthology.orchestra.run.smithi046.stderr:21 Sep 14:55:20 ntpd[13657]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T14:55:20.202 INFO:teuthology.orchestra.run.smithi046.stderr:21 Sep 14:55:20 ntpd[13657]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T14:55:20.202 INFO:teuthology.orchestra.run.smithi046.stderr:21 Sep 14:55:20 ntpd[13657]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T14:55:20.203 INFO:teuthology.orchestra.run.smithi046.stdout:21 Sep 14:55:20 ntpd[13657]: ntpd: time slew +0.004783 s 2024-09-21T14:55:20.203 INFO:teuthology.orchestra.run.smithi046.stdout:ntpd: time slew +0.004783s 2024-09-21T14:55:20.203 INFO:teuthology.orchestra.run.smithi046.stderr:21 Sep 14:55:20 ntpd[13657]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-21T14:55:20.203 INFO:teuthology.orchestra.run.smithi046.stderr:21 Sep 14:55:20 ntpd[13657]: can't open /var/log/ntpstats/loopstats.20240921: Permission denied 2024-09-21T14:55:20.265 INFO:teuthology.orchestra.run.smithi046.stdout: remote refid st t when poll reach delay offset jitter 2024-09-21T14:55:20.265 INFO:teuthology.orchestra.run.smithi046.stdout:============================================================================== 2024-09-21T14:55:20.266 INFO:teuthology.orchestra.run.smithi046.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-21T14:55:20.266 INFO:teuthology.orchestra.run.smithi046.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-21T14:55:20.266 INFO:teuthology.orchestra.run.smithi046.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-21T14:55:20.266 INFO:teuthology.orchestra.run.smithi046.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-21T14:55:20.267 INFO:teuthology.run_tasks:Running task install... 2024-09-21T14:55:20.283 DEBUG:teuthology.task.install:project ceph 2024-09-21T14:55:20.283 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '19e2a2beb6f41a3ab62650e38614eb29b59dcbb9'}} 2024-09-21T14:55:20.283 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '19e2a2beb6f41a3ab62650e38614eb29b59dcbb9'} 2024-09-21T14:55:20.283 INFO:teuthology.task.install:Using flavor: default 2024-09-21T14:55:20.288 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-21T14:55:20.288 INFO:teuthology.task.install:extra packages: [] 2024-09-21T14:55:20.289 DEBUG:teuthology.orchestra.run.smithi032:> sudo apt-key list | grep Ceph 2024-09-21T14:55:20.289 DEBUG:teuthology.orchestra.run.smithi046:> sudo apt-key list | grep Ceph 2024-09-21T14:55:20.392 INFO:teuthology.orchestra.run.smithi032.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2024-09-21T14:55:20.413 INFO:teuthology.orchestra.run.smithi046.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2024-09-21T14:55:20.453 INFO:teuthology.orchestra.run.smithi032.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-21T14:55:20.454 INFO:teuthology.orchestra.run.smithi032.stdout:uid [ unknown] Ceph.com (release key) 2024-09-21T14:55:20.455 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-09-21T14:55:20.455 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 2024-09-21T14:55:20.477 INFO:teuthology.orchestra.run.smithi046.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-21T14:55:20.477 INFO:teuthology.orchestra.run.smithi046.stdout:uid [ unknown] Ceph.com (release key) 2024-09-21T14:55:20.478 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-09-21T14:55:20.478 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 2024-09-21T14:55:20.608 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default/ 2024-09-21T14:55:20.609 INFO:teuthology.task.install.deb:Package version is 18.2.4-855-g19e2a2be-1focal 2024-09-21T14:55:20.613 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default/ 2024-09-21T14:55:20.613 INFO:teuthology.task.install.deb:Package version is 18.2.4-855-g19e2a2be-1focal 2024-09-21T14:55:20.720 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-09-21T14:55:20.720 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-21T14:55:20.724 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-21T14:55:20.724 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-21T14:55:20.737 DEBUG:teuthology.orchestra.run.smithi032:> sudo apt-get update 2024-09-21T14:55:20.741 DEBUG:teuthology.orchestra.run.smithi046:> sudo apt-get update 2024-09-21T14:55:20.880 INFO:teuthology.orchestra.run.smithi032.stdout:Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-09-21T14:55:20.883 INFO:teuthology.orchestra.run.smithi046.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-21T14:55:20.883 INFO:teuthology.orchestra.run.smithi046.stdout:Hit:2 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-09-21T14:55:20.889 INFO:teuthology.orchestra.run.smithi046.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-09-21T14:55:20.907 INFO:teuthology.orchestra.run.smithi046.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-09-21T14:55:20.995 INFO:teuthology.orchestra.run.smithi032.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-21T14:55:21.024 INFO:teuthology.orchestra.run.smithi046.stdout:Ign:5 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal InRelease 2024-09-21T14:55:21.028 INFO:teuthology.orchestra.run.smithi032.stdout:Ign:3 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal InRelease 2024-09-21T14:55:21.077 INFO:teuthology.orchestra.run.smithi032.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-09-21T14:55:21.078 INFO:teuthology.orchestra.run.smithi046.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal Release [7,662 B] 2024-09-21T14:55:21.082 INFO:teuthology.orchestra.run.smithi032.stdout:Get:5 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal Release [7,662 B] 2024-09-21T14:55:21.132 INFO:teuthology.orchestra.run.smithi046.stdout:Ign:7 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal Release.gpg 2024-09-21T14:55:21.137 INFO:teuthology.orchestra.run.smithi032.stdout:Ign:6 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal Release.gpg 2024-09-21T14:55:21.159 INFO:teuthology.orchestra.run.smithi032.stdout:Hit:7 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-09-21T14:55:21.186 INFO:teuthology.orchestra.run.smithi046.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main i386 Packages [3,426 B] 2024-09-21T14:55:21.190 INFO:teuthology.orchestra.run.smithi032.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main i386 Packages [3,426 B] 2024-09-21T14:55:21.251 INFO:teuthology.orchestra.run.smithi046.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 Packages [17.6 kB] 2024-09-21T14:55:21.254 INFO:teuthology.orchestra.run.smithi032.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 Packages [17.6 kB] 2024-09-21T14:55:21.558 INFO:teuthology.orchestra.run.smithi046.stdout:Fetched 28.7 kB in 1s (44.4 kB/s) 2024-09-21T14:55:21.559 INFO:teuthology.orchestra.run.smithi032.stdout:Fetched 28.7 kB in 1s (43.8 kB/s) 2024-09-21T14:55:22.761 INFO:teuthology.orchestra.run.smithi046.stdout:Reading package lists... 2024-09-21T14:55:22.764 INFO:teuthology.orchestra.run.smithi032.stdout:Reading package lists... 2024-09-21T14:55:22.788 DEBUG:teuthology.orchestra.run.smithi046:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.2.4-855-g19e2a2be-1focal cephadm=18.2.4-855-g19e2a2be-1focal ceph-mds=18.2.4-855-g19e2a2be-1focal ceph-mgr=18.2.4-855-g19e2a2be-1focal ceph-common=18.2.4-855-g19e2a2be-1focal ceph-fuse=18.2.4-855-g19e2a2be-1focal ceph-test=18.2.4-855-g19e2a2be-1focal ceph-volume=18.2.4-855-g19e2a2be-1focal radosgw=18.2.4-855-g19e2a2be-1focal python3-rados=18.2.4-855-g19e2a2be-1focal python3-rgw=18.2.4-855-g19e2a2be-1focal python3-cephfs=18.2.4-855-g19e2a2be-1focal python3-rbd=18.2.4-855-g19e2a2be-1focal libcephfs2=18.2.4-855-g19e2a2be-1focal libcephfs-dev=18.2.4-855-g19e2a2be-1focal librados2=18.2.4-855-g19e2a2be-1focal librbd1=18.2.4-855-g19e2a2be-1focal rbd-fuse=18.2.4-855-g19e2a2be-1focal 2024-09-21T14:55:22.792 DEBUG:teuthology.orchestra.run.smithi032:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.2.4-855-g19e2a2be-1focal cephadm=18.2.4-855-g19e2a2be-1focal ceph-mds=18.2.4-855-g19e2a2be-1focal ceph-mgr=18.2.4-855-g19e2a2be-1focal ceph-common=18.2.4-855-g19e2a2be-1focal ceph-fuse=18.2.4-855-g19e2a2be-1focal ceph-test=18.2.4-855-g19e2a2be-1focal ceph-volume=18.2.4-855-g19e2a2be-1focal radosgw=18.2.4-855-g19e2a2be-1focal python3-rados=18.2.4-855-g19e2a2be-1focal python3-rgw=18.2.4-855-g19e2a2be-1focal python3-cephfs=18.2.4-855-g19e2a2be-1focal python3-rbd=18.2.4-855-g19e2a2be-1focal libcephfs2=18.2.4-855-g19e2a2be-1focal libcephfs-dev=18.2.4-855-g19e2a2be-1focal librados2=18.2.4-855-g19e2a2be-1focal librbd1=18.2.4-855-g19e2a2be-1focal rbd-fuse=18.2.4-855-g19e2a2be-1focal 2024-09-21T14:55:22.862 INFO:teuthology.orchestra.run.smithi046.stdout:Reading package lists... 2024-09-21T14:55:22.867 INFO:teuthology.orchestra.run.smithi032.stdout:Reading package lists... 2024-09-21T14:55:23.063 INFO:teuthology.orchestra.run.smithi046.stdout:Building dependency tree... 2024-09-21T14:55:23.063 INFO:teuthology.orchestra.run.smithi046.stdout:Reading state information... 2024-09-21T14:55:23.069 INFO:teuthology.orchestra.run.smithi032.stdout:Building dependency tree... 2024-09-21T14:55:23.069 INFO:teuthology.orchestra.run.smithi032.stdout:Reading state information... 2024-09-21T14:55:23.265 INFO:teuthology.orchestra.run.smithi046.stdout:The following packages were automatically installed and are no longer required: 2024-09-21T14:55:23.267 INFO:teuthology.orchestra.run.smithi046.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2024-09-21T14:55:23.267 INFO:teuthology.orchestra.run.smithi046.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-21T14:55:23.267 INFO:teuthology.orchestra.run.smithi046.stdout:The following additional packages will be installed: 2024-09-21T14:55:23.267 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-21T14:55:23.267 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd fonts-mathjax jq 2024-09-21T14:55:23.267 INFO:teuthology.orchestra.run.smithi046.stdout: libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc libjs-underscore 2024-09-21T14:55:23.267 INFO:teuthology.orchestra.run.smithi046.stdout: liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 libpcre2-16-0 2024-09-21T14:55:23.267 INFO:teuthology.orchestra.run.smithi046.stdout: libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 libradosstriper1 2024-09-21T14:55:23.268 INFO:teuthology.orchestra.run.smithi046.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli 2024-09-21T14:55:23.268 INFO:teuthology.orchestra.run.smithi046.stdout: python-asyncssh-doc python-natsort-doc python-pastedeploy-tpl 2024-09-21T14:55:23.268 INFO:teuthology.orchestra.run.smithi046.stdout: python3-asyncssh python3-atomicwrites python3-attr python3-bcrypt 2024-09-21T14:55:23.268 INFO:teuthology.orchestra.run.smithi046.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2024-09-21T14:55:23.268 INFO:teuthology.orchestra.run.smithi046.stdout: python3-ceph-common python3-cherrypy3 python3-cryptography python3-decorator 2024-09-21T14:55:23.268 INFO:teuthology.orchestra.run.smithi046.stdout: python3-google-auth python3-html5lib python3-jinja2 python3-joblib 2024-09-21T14:55:23.268 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2024-09-21T14:55:23.268 INFO:teuthology.orchestra.run.smithi046.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2024-09-21T14:55:23.268 INFO:teuthology.orchestra.run.smithi046.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2024-09-21T14:55:23.268 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2024-09-21T14:55:23.268 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pytest 2024-09-21T14:55:23.269 INFO:teuthology.orchestra.run.smithi046.stdout: python3-repoze.lru python3-requests-oauthlib python3-routes python3-rsa 2024-09-21T14:55:23.269 INFO:teuthology.orchestra.run.smithi046.stdout: python3-scipy python3-simplegeneric python3-simplejson 2024-09-21T14:55:23.269 INFO:teuthology.orchestra.run.smithi046.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2024-09-21T14:55:23.269 INFO:teuthology.orchestra.run.smithi046.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2024-09-21T14:55:23.269 INFO:teuthology.orchestra.run.smithi046.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2024-09-21T14:55:23.269 INFO:teuthology.orchestra.run.smithi046.stdout: qttranslations5-l10n socat xmlstarlet 2024-09-21T14:55:23.269 INFO:teuthology.orchestra.run.smithi046.stdout:Suggested packages: 2024-09-21T14:55:23.269 INFO:teuthology.orchestra.run.smithi046.stdout: python3-influxdb fonts-mathjax-extras fonts-stix libjs-mathjax-doc 2024-09-21T14:55:23.269 INFO:teuthology.orchestra.run.smithi046.stdout: python-attr-doc python-blinker-doc python-cryptography-doc 2024-09-21T14:55:23.269 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2024-09-21T14:55:23.269 INFO:teuthology.orchestra.run.smithi046.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2024-09-21T14:55:23.270 INFO:teuthology.orchestra.run.smithi046.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2024-09-21T14:55:23.270 INFO:teuthology.orchestra.run.smithi046.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2024-09-21T14:55:23.270 INFO:teuthology.orchestra.run.smithi046.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2024-09-21T14:55:23.270 INFO:teuthology.orchestra.run.smithi046.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-21T14:55:23.270 INFO:teuthology.orchestra.run.smithi046.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2024-09-21T14:55:23.270 INFO:teuthology.orchestra.run.smithi046.stdout:Recommended packages: 2024-09-21T14:55:23.270 INFO:teuthology.orchestra.run.smithi046.stdout: btrfs-tools 2024-09-21T14:55:23.275 INFO:teuthology.orchestra.run.smithi032.stdout:The following packages were automatically installed and are no longer required: 2024-09-21T14:55:23.277 INFO:teuthology.orchestra.run.smithi032.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2024-09-21T14:55:23.277 INFO:teuthology.orchestra.run.smithi032.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-21T14:55:23.277 INFO:teuthology.orchestra.run.smithi032.stdout:The following additional packages will be installed: 2024-09-21T14:55:23.277 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-21T14:55:23.277 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd fonts-mathjax jq 2024-09-21T14:55:23.277 INFO:teuthology.orchestra.run.smithi032.stdout: libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc libjs-underscore 2024-09-21T14:55:23.277 INFO:teuthology.orchestra.run.smithi032.stdout: liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 libpcre2-16-0 2024-09-21T14:55:23.277 INFO:teuthology.orchestra.run.smithi032.stdout: libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 libradosstriper1 2024-09-21T14:55:23.278 INFO:teuthology.orchestra.run.smithi032.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli 2024-09-21T14:55:23.278 INFO:teuthology.orchestra.run.smithi032.stdout: python-asyncssh-doc python-natsort-doc python-pastedeploy-tpl 2024-09-21T14:55:23.278 INFO:teuthology.orchestra.run.smithi032.stdout: python3-asyncssh python3-atomicwrites python3-attr python3-bcrypt 2024-09-21T14:55:23.278 INFO:teuthology.orchestra.run.smithi032.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2024-09-21T14:55:23.278 INFO:teuthology.orchestra.run.smithi032.stdout: python3-ceph-common python3-cherrypy3 python3-cryptography python3-decorator 2024-09-21T14:55:23.278 INFO:teuthology.orchestra.run.smithi032.stdout: python3-google-auth python3-html5lib python3-jinja2 python3-joblib 2024-09-21T14:55:23.278 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2024-09-21T14:55:23.278 INFO:teuthology.orchestra.run.smithi032.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2024-09-21T14:55:23.278 INFO:teuthology.orchestra.run.smithi032.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2024-09-21T14:55:23.279 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2024-09-21T14:55:23.279 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pytest 2024-09-21T14:55:23.279 INFO:teuthology.orchestra.run.smithi032.stdout: python3-repoze.lru python3-requests-oauthlib python3-routes python3-rsa 2024-09-21T14:55:23.279 INFO:teuthology.orchestra.run.smithi032.stdout: python3-scipy python3-simplegeneric python3-simplejson 2024-09-21T14:55:23.279 INFO:teuthology.orchestra.run.smithi032.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2024-09-21T14:55:23.279 INFO:teuthology.orchestra.run.smithi032.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2024-09-21T14:55:23.279 INFO:teuthology.orchestra.run.smithi032.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2024-09-21T14:55:23.279 INFO:teuthology.orchestra.run.smithi032.stdout: qttranslations5-l10n socat xmlstarlet 2024-09-21T14:55:23.279 INFO:teuthology.orchestra.run.smithi032.stdout:Suggested packages: 2024-09-21T14:55:23.279 INFO:teuthology.orchestra.run.smithi032.stdout: python3-influxdb fonts-mathjax-extras fonts-stix libjs-mathjax-doc 2024-09-21T14:55:23.279 INFO:teuthology.orchestra.run.smithi032.stdout: python-attr-doc python-blinker-doc python-cryptography-doc 2024-09-21T14:55:23.280 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2024-09-21T14:55:23.280 INFO:teuthology.orchestra.run.smithi032.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2024-09-21T14:55:23.280 INFO:teuthology.orchestra.run.smithi032.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2024-09-21T14:55:23.280 INFO:teuthology.orchestra.run.smithi032.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2024-09-21T14:55:23.280 INFO:teuthology.orchestra.run.smithi032.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2024-09-21T14:55:23.280 INFO:teuthology.orchestra.run.smithi032.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-21T14:55:23.280 INFO:teuthology.orchestra.run.smithi032.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2024-09-21T14:55:23.280 INFO:teuthology.orchestra.run.smithi032.stdout:Recommended packages: 2024-09-21T14:55:23.280 INFO:teuthology.orchestra.run.smithi032.stdout: btrfs-tools 2024-09-21T14:55:23.393 INFO:teuthology.orchestra.run.smithi046.stdout:The following NEW packages will be installed: 2024-09-21T14:55:23.393 INFO:teuthology.orchestra.run.smithi046.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-21T14:55:23.393 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-21T14:55:23.393 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-09-21T14:55:23.393 INFO:teuthology.orchestra.run.smithi046.stdout: fonts-mathjax jq libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 2024-09-21T14:55:23.393 INFO:teuthology.orchestra.run.smithi046.stdout: libjs-mathjax libjs-sphinxdoc libjs-underscore liblbfgsb0 liblttng-ust0 2024-09-21T14:55:23.393 INFO:teuthology.orchestra.run.smithi046.stdout: liblua5.3-0 liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-09-21T14:55:23.393 INFO:teuthology.orchestra.run.smithi046.stdout: libqt5network5 librabbitmq4 libradosstriper1 librdkafka1 librgw2 2024-09-21T14:55:23.394 INFO:teuthology.orchestra.run.smithi046.stdout: libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli python-asyncssh-doc 2024-09-21T14:55:23.394 INFO:teuthology.orchestra.run.smithi046.stdout: python-natsort-doc python-pastedeploy-tpl python3-asyncssh 2024-09-21T14:55:23.394 INFO:teuthology.orchestra.run.smithi046.stdout: python3-atomicwrites python3-attr python3-bcrypt python3-blinker python3-bs4 2024-09-21T14:55:23.394 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-09-21T14:55:23.395 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cherrypy3 python3-cryptography python3-decorator python3-google-auth 2024-09-21T14:55:23.395 INFO:teuthology.orchestra.run.smithi046.stdout: python3-html5lib python3-jinja2 python3-joblib python3-jwt 2024-09-21T14:55:23.395 INFO:teuthology.orchestra.run.smithi046.stdout: python3-kubernetes python3-logutils python3-lxml python3-mako 2024-09-21T14:55:23.395 INFO:teuthology.orchestra.run.smithi046.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2024-09-21T14:55:23.395 INFO:teuthology.orchestra.run.smithi046.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2024-09-21T14:55:23.395 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2024-09-21T14:55:23.395 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pytest 2024-09-21T14:55:23.395 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests-oauthlib 2024-09-21T14:55:23.395 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rgw python3-routes python3-rsa python3-scipy python3-simplegeneric 2024-09-21T14:55:23.395 INFO:teuthology.orchestra.run.smithi046.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-09-21T14:55:23.395 INFO:teuthology.orchestra.run.smithi046.stdout: python3-sklearn-lib python3-soupsieve python3-tempita python3-waitress 2024-09-21T14:55:23.395 INFO:teuthology.orchestra.run.smithi046.stdout: python3-wcwidth python3-webencodings python3-webob python3-websocket 2024-09-21T14:55:23.395 INFO:teuthology.orchestra.run.smithi046.stdout: python3-webtest python3-werkzeug qttranslations5-l10n radosgw rbd-fuse socat 2024-09-21T14:55:23.395 INFO:teuthology.orchestra.run.smithi046.stdout: xmlstarlet 2024-09-21T14:55:23.396 INFO:teuthology.orchestra.run.smithi046.stdout:The following packages will be upgraded: 2024-09-21T14:55:23.396 INFO:teuthology.orchestra.run.smithi046.stdout: librados2 librbd1 2024-09-21T14:55:23.406 INFO:teuthology.orchestra.run.smithi032.stdout:The following NEW packages will be installed: 2024-09-21T14:55:23.406 INFO:teuthology.orchestra.run.smithi032.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-21T14:55:23.406 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-21T14:55:23.406 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-09-21T14:55:23.406 INFO:teuthology.orchestra.run.smithi032.stdout: fonts-mathjax jq libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 2024-09-21T14:55:23.407 INFO:teuthology.orchestra.run.smithi032.stdout: libjs-mathjax libjs-sphinxdoc libjs-underscore liblbfgsb0 liblttng-ust0 2024-09-21T14:55:23.407 INFO:teuthology.orchestra.run.smithi032.stdout: liblua5.3-0 liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-09-21T14:55:23.407 INFO:teuthology.orchestra.run.smithi032.stdout: libqt5network5 librabbitmq4 libradosstriper1 librdkafka1 librgw2 2024-09-21T14:55:23.407 INFO:teuthology.orchestra.run.smithi032.stdout: libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli python-asyncssh-doc 2024-09-21T14:55:23.408 INFO:teuthology.orchestra.run.smithi032.stdout: python-natsort-doc python-pastedeploy-tpl python3-asyncssh 2024-09-21T14:55:23.408 INFO:teuthology.orchestra.run.smithi032.stdout: python3-atomicwrites python3-attr python3-bcrypt python3-blinker python3-bs4 2024-09-21T14:55:23.408 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-09-21T14:55:23.408 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cherrypy3 python3-cryptography python3-decorator python3-google-auth 2024-09-21T14:55:23.408 INFO:teuthology.orchestra.run.smithi032.stdout: python3-html5lib python3-jinja2 python3-joblib python3-jwt 2024-09-21T14:55:23.408 INFO:teuthology.orchestra.run.smithi032.stdout: python3-kubernetes python3-logutils python3-lxml python3-mako 2024-09-21T14:55:23.408 INFO:teuthology.orchestra.run.smithi032.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2024-09-21T14:55:23.408 INFO:teuthology.orchestra.run.smithi032.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2024-09-21T14:55:23.408 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2024-09-21T14:55:23.408 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pytest 2024-09-21T14:55:23.408 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests-oauthlib 2024-09-21T14:55:23.408 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rgw python3-routes python3-rsa python3-scipy python3-simplegeneric 2024-09-21T14:55:23.408 INFO:teuthology.orchestra.run.smithi032.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-09-21T14:55:23.408 INFO:teuthology.orchestra.run.smithi032.stdout: python3-sklearn-lib python3-soupsieve python3-tempita python3-waitress 2024-09-21T14:55:23.408 INFO:teuthology.orchestra.run.smithi032.stdout: python3-wcwidth python3-webencodings python3-webob python3-websocket 2024-09-21T14:55:23.409 INFO:teuthology.orchestra.run.smithi032.stdout: python3-webtest python3-werkzeug qttranslations5-l10n radosgw rbd-fuse socat 2024-09-21T14:55:23.409 INFO:teuthology.orchestra.run.smithi032.stdout: xmlstarlet 2024-09-21T14:55:23.409 INFO:teuthology.orchestra.run.smithi032.stdout:The following packages will be upgraded: 2024-09-21T14:55:23.409 INFO:teuthology.orchestra.run.smithi032.stdout: librados2 librbd1 2024-09-21T14:55:23.475 INFO:teuthology.orchestra.run.smithi032.stdout:2 upgraded, 110 newly installed, 0 to remove and 297 not upgraded. 2024-09-21T14:55:23.475 INFO:teuthology.orchestra.run.smithi032.stdout:Need to get 199 MB of archives. 2024-09-21T14:55:23.475 INFO:teuthology.orchestra.run.smithi032.stdout:After this operation, 977 MB of additional disk space will be used. 2024-09-21T14:55:23.475 INFO:teuthology.orchestra.run.smithi032.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2024-09-21T14:55:23.559 INFO:teuthology.orchestra.run.smithi032.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/universe amd64 libdouble-conversion3 amd64 3.1.5-4ubuntu1 [37.9 kB] 2024-09-21T14:55:23.561 INFO:teuthology.orchestra.run.smithi032.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpcre2-16-0 amd64 10.34-7ubuntu0.1 [181 kB] 2024-09-21T14:55:23.567 INFO:teuthology.orchestra.run.smithi046.stdout:2 upgraded, 110 newly installed, 0 to remove and 297 not upgraded. 2024-09-21T14:55:23.567 INFO:teuthology.orchestra.run.smithi046.stdout:Need to get 199 MB of archives. 2024-09-21T14:55:23.567 INFO:teuthology.orchestra.run.smithi046.stdout:After this operation, 977 MB of additional disk space will be used. 2024-09-21T14:55:23.567 INFO:teuthology.orchestra.run.smithi046.stdout:Get:1 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 18.2.4-855-g19e2a2be-1focal [3,453 kB] 2024-09-21T14:55:23.578 INFO:teuthology.orchestra.run.smithi032.stdout:Get:4 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 18.2.4-855-g19e2a2be-1focal [3,453 kB] 2024-09-21T14:55:23.579 INFO:teuthology.orchestra.run.smithi032.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5core5a amd64 5.12.8+dfsg-0ubuntu2.1 [2,006 kB] 2024-09-21T14:55:23.589 INFO:teuthology.orchestra.run.smithi046.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2024-09-21T14:55:23.633 INFO:teuthology.orchestra.run.smithi032.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5dbus5 amd64 5.12.8+dfsg-0ubuntu2.1 [208 kB] 2024-09-21T14:55:23.636 INFO:teuthology.orchestra.run.smithi032.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5network5 amd64 5.12.8+dfsg-0ubuntu2.1 [673 kB] 2024-09-21T14:55:23.642 INFO:teuthology.orchestra.run.smithi032.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/universe amd64 libthrift-0.13.0 amd64 0.13.0-2build2 [249 kB] 2024-09-21T14:55:23.644 INFO:teuthology.orchestra.run.smithi032.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2024-09-21T14:55:23.644 INFO:teuthology.orchestra.run.smithi032.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2024-09-21T14:55:23.645 INFO:teuthology.orchestra.run.smithi032.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2024-09-21T14:55:23.645 INFO:teuthology.orchestra.run.smithi032.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2024-09-21T14:55:23.648 INFO:teuthology.orchestra.run.smithi032.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2024-09-21T14:55:23.649 INFO:teuthology.orchestra.run.smithi032.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2024-09-21T14:55:23.651 INFO:teuthology.orchestra.run.smithi032.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-natsort all 7.0.1-1 [29.2 kB] 2024-09-21T14:55:23.668 INFO:teuthology.orchestra.run.smithi032.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2024-09-21T14:55:23.668 INFO:teuthology.orchestra.run.smithi032.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2024-09-21T14:55:23.668 INFO:teuthology.orchestra.run.smithi032.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2024-09-21T14:55:23.670 INFO:teuthology.orchestra.run.smithi032.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-21T14:55:23.670 INFO:teuthology.orchestra.run.smithi032.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2024-09-21T14:55:23.670 INFO:teuthology.orchestra.run.smithi032.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-webob all 1:1.8.5-2ubuntu0.1 [84.5 kB] 2024-09-21T14:55:23.672 INFO:teuthology.orchestra.run.smithi032.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2024-09-21T14:55:23.673 INFO:teuthology.orchestra.run.smithi032.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2024-09-21T14:55:23.674 INFO:teuthology.orchestra.run.smithi032.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2024-09-21T14:55:23.686 INFO:teuthology.orchestra.run.smithi032.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2024-09-21T14:55:23.694 INFO:teuthology.orchestra.run.smithi032.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2024-09-21T14:55:23.700 INFO:teuthology.orchestra.run.smithi032.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2024-09-21T14:55:23.700 INFO:teuthology.orchestra.run.smithi032.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2024-09-21T14:55:23.701 INFO:teuthology.orchestra.run.smithi032.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2024-09-21T14:55:23.701 INFO:teuthology.orchestra.run.smithi032.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2024-09-21T14:55:23.703 INFO:teuthology.orchestra.run.smithi032.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-werkzeug all 0.16.1+dfsg1-2ubuntu0.2 [183 kB] 2024-09-21T14:55:23.707 INFO:teuthology.orchestra.run.smithi032.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2024-09-21T14:55:23.707 INFO:teuthology.orchestra.run.smithi032.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.3 [211 kB] 2024-09-21T14:55:23.709 INFO:teuthology.orchestra.run.smithi032.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2024-09-21T14:55:23.720 INFO:teuthology.orchestra.run.smithi032.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-asyncssh all 1.12.2-1 [152 kB] 2024-09-21T14:55:23.722 INFO:teuthology.orchestra.run.smithi032.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jinja2 all 2.10.1-2ubuntu0.3 [96.0 kB] 2024-09-21T14:55:23.723 INFO:teuthology.orchestra.run.smithi032.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2024-09-21T14:55:23.724 INFO:teuthology.orchestra.run.smithi032.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2024-09-21T14:55:23.725 INFO:teuthology.orchestra.run.smithi032.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2024-09-21T14:55:23.738 INFO:teuthology.orchestra.run.smithi032.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2024-09-21T14:55:23.739 INFO:teuthology.orchestra.run.smithi032.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2024-09-21T14:55:23.739 INFO:teuthology.orchestra.run.smithi032.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2024-09-21T14:55:23.740 INFO:teuthology.orchestra.run.smithi032.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2024-09-21T14:55:23.741 INFO:teuthology.orchestra.run.smithi032.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2024-09-21T14:55:23.755 INFO:teuthology.orchestra.run.smithi032.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-21T14:55:23.755 INFO:teuthology.orchestra.run.smithi032.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2024-09-21T14:55:23.758 INFO:teuthology.orchestra.run.smithi032.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2024-09-21T14:55:23.760 INFO:teuthology.orchestra.run.smithi032.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2024-09-21T14:55:23.762 INFO:teuthology.orchestra.run.smithi032.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2024-09-21T14:55:23.772 INFO:teuthology.orchestra.run.smithi032.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2024-09-21T14:55:23.773 INFO:teuthology.orchestra.run.smithi032.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2024-09-21T14:55:23.852 INFO:teuthology.orchestra.run.smithi032.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2024-09-21T14:55:23.868 INFO:teuthology.orchestra.run.smithi032.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2024-09-21T14:55:23.869 INFO:teuthology.orchestra.run.smithi032.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2024-09-21T14:55:23.880 INFO:teuthology.orchestra.run.smithi032.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2024-09-21T14:55:23.943 INFO:teuthology.orchestra.run.smithi032.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2024-09-21T14:55:23.944 INFO:teuthology.orchestra.run.smithi032.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2024-09-21T14:55:23.945 INFO:teuthology.orchestra.run.smithi032.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2024-09-21T14:55:23.945 INFO:teuthology.orchestra.run.smithi032.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2024-09-21T14:55:23.946 INFO:teuthology.orchestra.run.smithi032.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2024-09-21T14:55:23.946 INFO:teuthology.orchestra.run.smithi032.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2024-09-21T14:55:23.948 INFO:teuthology.orchestra.run.smithi032.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2024-09-21T14:55:23.948 INFO:teuthology.orchestra.run.smithi032.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2024-09-21T14:55:23.961 INFO:teuthology.orchestra.run.smithi032.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2024-09-21T14:55:23.967 INFO:teuthology.orchestra.run.smithi032.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2024-09-21T14:55:23.969 INFO:teuthology.orchestra.run.smithi032.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2024-09-21T14:55:23.972 INFO:teuthology.orchestra.run.smithi032.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2024-09-21T14:55:23.973 INFO:teuthology.orchestra.run.smithi032.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2024-09-21T14:55:23.976 INFO:teuthology.orchestra.run.smithi032.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2024-09-21T14:55:23.978 INFO:teuthology.orchestra.run.smithi032.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/universe amd64 fonts-mathjax all 2.7.4+dfsg-1 [2,208 kB] 2024-09-21T14:55:23.983 INFO:teuthology.orchestra.run.smithi046.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/universe amd64 libdouble-conversion3 amd64 3.1.5-4ubuntu1 [37.9 kB] 2024-09-21T14:55:23.993 INFO:teuthology.orchestra.run.smithi032.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.2 [283 kB] 2024-09-21T14:55:23.995 INFO:teuthology.orchestra.run.smithi032.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-asyncssh-doc all 1.12.2-1 [258 kB] 2024-09-21T14:55:23.997 INFO:teuthology.orchestra.run.smithi046.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpcre2-16-0 amd64 10.34-7ubuntu0.1 [181 kB] 2024-09-21T14:55:24.037 INFO:teuthology.orchestra.run.smithi032.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/universe amd64 libjs-mathjax all 2.7.4+dfsg-1 [5,654 kB] 2024-09-21T14:55:24.079 INFO:teuthology.orchestra.run.smithi032.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-natsort-doc all 7.0.1-1 [103 kB] 2024-09-21T14:55:24.079 INFO:teuthology.orchestra.run.smithi032.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2024-09-21T14:55:24.080 INFO:teuthology.orchestra.run.smithi032.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2024-09-21T14:55:24.080 INFO:teuthology.orchestra.run.smithi046.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5core5a amd64 5.12.8+dfsg-0ubuntu2.1 [2,006 kB] 2024-09-21T14:55:24.081 INFO:teuthology.orchestra.run.smithi032.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2024-09-21T14:55:24.092 INFO:teuthology.orchestra.run.smithi032.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-21T14:55:24.093 INFO:teuthology.orchestra.run.smithi032.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2024-09-21T14:55:24.094 INFO:teuthology.orchestra.run.smithi032.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2024-09-21T14:55:24.094 INFO:teuthology.orchestra.run.smithi032.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2024-09-21T14:55:24.123 INFO:teuthology.orchestra.run.smithi032.stdout:Get:82 http://archive.ubuntu.com/ubuntu focal/universe amd64 qttranslations5-l10n all 5.12.8-0ubuntu1 [1,486 kB] 2024-09-21T14:55:24.322 INFO:teuthology.orchestra.run.smithi046.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5dbus5 amd64 5.12.8+dfsg-0ubuntu2.1 [208 kB] 2024-09-21T14:55:24.331 INFO:teuthology.orchestra.run.smithi046.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5network5 amd64 5.12.8+dfsg-0ubuntu2.1 [673 kB] 2024-09-21T14:55:24.359 INFO:teuthology.orchestra.run.smithi046.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/universe amd64 libthrift-0.13.0 amd64 0.13.0-2build2 [249 kB] 2024-09-21T14:55:24.367 INFO:teuthology.orchestra.run.smithi046.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2024-09-21T14:55:24.368 INFO:teuthology.orchestra.run.smithi046.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2024-09-21T14:55:24.372 INFO:teuthology.orchestra.run.smithi046.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2024-09-21T14:55:24.373 INFO:teuthology.orchestra.run.smithi046.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2024-09-21T14:55:24.385 INFO:teuthology.orchestra.run.smithi046.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2024-09-21T14:55:24.387 INFO:teuthology.orchestra.run.smithi046.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2024-09-21T14:55:24.403 INFO:teuthology.orchestra.run.smithi046.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-natsort all 7.0.1-1 [29.2 kB] 2024-09-21T14:55:24.462 INFO:teuthology.orchestra.run.smithi046.stdout:Get:16 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 18.2.4-855-g19e2a2be-1focal [4,044 kB] 2024-09-21T14:55:24.474 INFO:teuthology.orchestra.run.smithi046.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2024-09-21T14:55:24.475 INFO:teuthology.orchestra.run.smithi046.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2024-09-21T14:55:24.476 INFO:teuthology.orchestra.run.smithi046.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2024-09-21T14:55:24.477 INFO:teuthology.orchestra.run.smithi046.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-21T14:55:24.478 INFO:teuthology.orchestra.run.smithi046.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2024-09-21T14:55:24.479 INFO:teuthology.orchestra.run.smithi046.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-webob all 1:1.8.5-2ubuntu0.1 [84.5 kB] 2024-09-21T14:55:24.482 INFO:teuthology.orchestra.run.smithi046.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2024-09-21T14:55:24.482 INFO:teuthology.orchestra.run.smithi046.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2024-09-21T14:55:24.486 INFO:teuthology.orchestra.run.smithi046.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2024-09-21T14:55:24.549 INFO:teuthology.orchestra.run.smithi032.stdout:Get:83 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 18.2.4-855-g19e2a2be-1focal [4,044 kB] 2024-09-21T14:55:24.566 INFO:teuthology.orchestra.run.smithi046.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2024-09-21T14:55:24.567 INFO:teuthology.orchestra.run.smithi046.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2024-09-21T14:55:24.576 INFO:teuthology.orchestra.run.smithi046.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2024-09-21T14:55:24.576 INFO:teuthology.orchestra.run.smithi046.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2024-09-21T14:55:24.577 INFO:teuthology.orchestra.run.smithi046.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2024-09-21T14:55:24.578 INFO:teuthology.orchestra.run.smithi046.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2024-09-21T14:55:24.580 INFO:teuthology.orchestra.run.smithi046.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-werkzeug all 0.16.1+dfsg1-2ubuntu0.2 [183 kB] 2024-09-21T14:55:24.583 INFO:teuthology.orchestra.run.smithi046.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2024-09-21T14:55:24.648 INFO:teuthology.orchestra.run.smithi046.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.3 [211 kB] 2024-09-21T14:55:24.653 INFO:teuthology.orchestra.run.smithi046.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2024-09-21T14:55:24.729 INFO:teuthology.orchestra.run.smithi046.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-asyncssh all 1.12.2-1 [152 kB] 2024-09-21T14:55:24.732 INFO:teuthology.orchestra.run.smithi046.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jinja2 all 2.10.1-2ubuntu0.3 [96.0 kB] 2024-09-21T14:55:24.734 INFO:teuthology.orchestra.run.smithi046.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2024-09-21T14:55:24.735 INFO:teuthology.orchestra.run.smithi046.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2024-09-21T14:55:24.737 INFO:teuthology.orchestra.run.smithi046.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2024-09-21T14:55:24.737 INFO:teuthology.orchestra.run.smithi046.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2024-09-21T14:55:24.740 INFO:teuthology.orchestra.run.smithi046.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2024-09-21T14:55:24.740 INFO:teuthology.orchestra.run.smithi046.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2024-09-21T14:55:24.806 INFO:teuthology.orchestra.run.smithi046.stdout:Get:44 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 18.2.4-855-g19e2a2be-1focal [746 kB] 2024-09-21T14:55:24.810 INFO:teuthology.orchestra.run.smithi046.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2024-09-21T14:55:24.811 INFO:teuthology.orchestra.run.smithi046.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2024-09-21T14:55:24.823 INFO:teuthology.orchestra.run.smithi046.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-21T14:55:24.824 INFO:teuthology.orchestra.run.smithi046.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2024-09-21T14:55:24.827 INFO:teuthology.orchestra.run.smithi046.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2024-09-21T14:55:24.855 INFO:teuthology.orchestra.run.smithi046.stdout:Get:50 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 18.2.4-855-g19e2a2be-1focal [388 kB] 2024-09-21T14:55:24.888 INFO:teuthology.orchestra.run.smithi046.stdout:Get:51 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 18.2.4-855-g19e2a2be-1focal [31.9 kB] 2024-09-21T14:55:24.890 INFO:teuthology.orchestra.run.smithi046.stdout:Get:52 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 18.2.4-855-g19e2a2be-1focal [201 kB] 2024-09-21T14:55:24.892 INFO:teuthology.orchestra.run.smithi046.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2024-09-21T14:55:24.894 INFO:teuthology.orchestra.run.smithi046.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2024-09-21T14:55:24.895 INFO:teuthology.orchestra.run.smithi046.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2024-09-21T14:55:24.896 INFO:teuthology.orchestra.run.smithi046.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2024-09-21T14:55:24.898 INFO:teuthology.orchestra.run.smithi046.stdout:Get:57 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 18.2.4-855-g19e2a2be-1focal [62.5 kB] 2024-09-21T14:55:24.924 INFO:teuthology.orchestra.run.smithi046.stdout:Get:58 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 18.2.4-855-g19e2a2be-1focal [367 kB] 2024-09-21T14:55:24.937 INFO:teuthology.orchestra.run.smithi046.stdout:Get:59 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 18.2.4-855-g19e2a2be-1focal [6,627 kB] 2024-09-21T14:55:24.937 INFO:teuthology.orchestra.run.smithi032.stdout:Get:84 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 18.2.4-855-g19e2a2be-1focal [746 kB] 2024-09-21T14:55:25.017 INFO:teuthology.orchestra.run.smithi032.stdout:Get:85 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 18.2.4-855-g19e2a2be-1focal [388 kB] 2024-09-21T14:55:25.059 INFO:teuthology.orchestra.run.smithi032.stdout:Get:86 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 18.2.4-855-g19e2a2be-1focal [31.9 kB] 2024-09-21T14:55:25.060 INFO:teuthology.orchestra.run.smithi032.stdout:Get:87 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 18.2.4-855-g19e2a2be-1focal [201 kB] 2024-09-21T14:55:25.064 INFO:teuthology.orchestra.run.smithi032.stdout:Get:88 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 18.2.4-855-g19e2a2be-1focal [62.5 kB] 2024-09-21T14:55:25.066 INFO:teuthology.orchestra.run.smithi032.stdout:Get:89 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 18.2.4-855-g19e2a2be-1focal [367 kB] 2024-09-21T14:55:25.106 INFO:teuthology.orchestra.run.smithi032.stdout:Get:90 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 18.2.4-855-g19e2a2be-1focal [6,627 kB] 2024-09-21T14:55:25.220 INFO:teuthology.orchestra.run.smithi046.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2024-09-21T14:55:25.255 INFO:teuthology.orchestra.run.smithi046.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2024-09-21T14:55:25.257 INFO:teuthology.orchestra.run.smithi046.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2024-09-21T14:55:25.321 INFO:teuthology.orchestra.run.smithi046.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2024-09-21T14:55:25.321 INFO:teuthology.orchestra.run.smithi046.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2024-09-21T14:55:25.323 INFO:teuthology.orchestra.run.smithi046.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2024-09-21T14:55:25.323 INFO:teuthology.orchestra.run.smithi046.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2024-09-21T14:55:25.323 INFO:teuthology.orchestra.run.smithi046.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2024-09-21T14:55:25.324 INFO:teuthology.orchestra.run.smithi046.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2024-09-21T14:55:25.324 INFO:teuthology.orchestra.run.smithi046.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2024-09-21T14:55:25.384 INFO:teuthology.orchestra.run.smithi046.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2024-09-21T14:55:25.384 INFO:teuthology.orchestra.run.smithi046.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2024-09-21T14:55:25.388 INFO:teuthology.orchestra.run.smithi046.stdout:Get:72 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 18.2.4-855-g19e2a2be-1focal [113 kB] 2024-09-21T14:55:25.396 INFO:teuthology.orchestra.run.smithi046.stdout:Get:73 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 18.2.4-855-g19e2a2be-1focal [478 kB] 2024-09-21T14:55:25.433 INFO:teuthology.orchestra.run.smithi046.stdout:Get:74 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 18.2.4-855-g19e2a2be-1focal [25.2 MB] 2024-09-21T14:55:25.465 INFO:teuthology.orchestra.run.smithi046.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2024-09-21T14:55:25.470 INFO:teuthology.orchestra.run.smithi046.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2024-09-21T14:55:25.474 INFO:teuthology.orchestra.run.smithi046.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2024-09-21T14:55:25.477 INFO:teuthology.orchestra.run.smithi046.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2024-09-21T14:55:25.478 INFO:teuthology.orchestra.run.smithi046.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2024-09-21T14:55:25.480 INFO:teuthology.orchestra.run.smithi046.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2024-09-21T14:55:25.483 INFO:teuthology.orchestra.run.smithi046.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal/universe amd64 fonts-mathjax all 2.7.4+dfsg-1 [2,208 kB] 2024-09-21T14:55:25.554 INFO:teuthology.orchestra.run.smithi046.stdout:Get:82 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.2 [283 kB] 2024-09-21T14:55:25.560 INFO:teuthology.orchestra.run.smithi046.stdout:Get:83 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-asyncssh-doc all 1.12.2-1 [258 kB] 2024-09-21T14:55:25.563 INFO:teuthology.orchestra.run.smithi046.stdout:Get:84 http://archive.ubuntu.com/ubuntu focal/universe amd64 libjs-mathjax all 2.7.4+dfsg-1 [5,654 kB] 2024-09-21T14:55:25.613 INFO:teuthology.orchestra.run.smithi032.stdout:Get:91 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 18.2.4-855-g19e2a2be-1focal [113 kB] 2024-09-21T14:55:25.615 INFO:teuthology.orchestra.run.smithi032.stdout:Get:92 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 18.2.4-855-g19e2a2be-1focal [478 kB] 2024-09-21T14:55:25.666 INFO:teuthology.orchestra.run.smithi032.stdout:Get:93 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 18.2.4-855-g19e2a2be-1focal [25.2 MB] 2024-09-21T14:55:25.725 INFO:teuthology.orchestra.run.smithi046.stdout:Get:85 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-natsort-doc all 7.0.1-1 [103 kB] 2024-09-21T14:55:25.726 INFO:teuthology.orchestra.run.smithi046.stdout:Get:86 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2024-09-21T14:55:25.726 INFO:teuthology.orchestra.run.smithi046.stdout:Get:87 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2024-09-21T14:55:25.727 INFO:teuthology.orchestra.run.smithi046.stdout:Get:88 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2024-09-21T14:55:25.740 INFO:teuthology.orchestra.run.smithi046.stdout:Get:89 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-21T14:55:25.740 INFO:teuthology.orchestra.run.smithi046.stdout:Get:90 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2024-09-21T14:55:25.794 INFO:teuthology.orchestra.run.smithi046.stdout:Get:91 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2024-09-21T14:55:25.795 INFO:teuthology.orchestra.run.smithi046.stdout:Get:92 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2024-09-21T14:55:25.796 INFO:teuthology.orchestra.run.smithi046.stdout:Get:93 http://archive.ubuntu.com/ubuntu focal/universe amd64 qttranslations5-l10n all 5.12.8-0ubuntu1 [1,486 kB] 2024-09-21T14:55:26.774 INFO:teuthology.orchestra.run.smithi046.stdout:Get:94 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 18.2.4-855-g19e2a2be-1focal [9,605 kB] 2024-09-21T14:55:27.103 INFO:teuthology.orchestra.run.smithi046.stdout:Get:95 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 18.2.4-855-g19e2a2be-1focal [238 kB] 2024-09-21T14:55:27.106 INFO:teuthology.orchestra.run.smithi046.stdout:Get:96 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 18.2.4-855-g19e2a2be-1focal [148 kB] 2024-09-21T14:55:27.108 INFO:teuthology.orchestra.run.smithi046.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 18.2.4-855-g19e2a2be-1focal [1,574 kB] 2024-09-21T14:55:27.153 INFO:teuthology.orchestra.run.smithi046.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 18.2.4-855-g19e2a2be-1focal [6,640 kB] 2024-09-21T14:55:27.190 INFO:teuthology.orchestra.run.smithi032.stdout:Get:94 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 18.2.4-855-g19e2a2be-1focal [9,605 kB] 2024-09-21T14:55:27.337 INFO:teuthology.orchestra.run.smithi046.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 18.2.4-855-g19e2a2be-1focal [24.5 MB] 2024-09-21T14:55:27.478 INFO:teuthology.orchestra.run.smithi032.stdout:Get:95 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 18.2.4-855-g19e2a2be-1focal [238 kB] 2024-09-21T14:55:27.525 INFO:teuthology.orchestra.run.smithi032.stdout:Get:96 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 18.2.4-855-g19e2a2be-1focal [148 kB] 2024-09-21T14:55:27.528 INFO:teuthology.orchestra.run.smithi032.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 18.2.4-855-g19e2a2be-1focal [1,574 kB] 2024-09-21T14:55:27.557 INFO:teuthology.orchestra.run.smithi032.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 18.2.4-855-g19e2a2be-1focal [6,640 kB] 2024-09-21T14:55:27.810 INFO:teuthology.orchestra.run.smithi032.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 18.2.4-855-g19e2a2be-1focal [24.5 MB] 2024-09-21T14:55:28.171 INFO:teuthology.orchestra.run.smithi046.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 18.2.4-855-g19e2a2be-1focal [13.7 kB] 2024-09-21T14:55:28.217 INFO:teuthology.orchestra.run.smithi046.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 18.2.4-855-g19e2a2be-1focal [856 kB] 2024-09-21T14:55:28.233 INFO:teuthology.orchestra.run.smithi046.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 18.2.4-855-g19e2a2be-1focal [2,295 kB] 2024-09-21T14:55:28.250 INFO:teuthology.orchestra.run.smithi046.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 18.2.4-855-g19e2a2be-1focal [239 kB] 2024-09-21T14:55:28.252 INFO:teuthology.orchestra.run.smithi046.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 18.2.4-855-g19e2a2be-1focal [143 kB] 2024-09-21T14:55:28.253 INFO:teuthology.orchestra.run.smithi046.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 18.2.4-855-g19e2a2be-1focal [7,015 kB] 2024-09-21T14:55:28.437 INFO:teuthology.orchestra.run.smithi046.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 18.2.4-855-g19e2a2be-1focal [7,893 kB] 2024-09-21T14:55:28.671 INFO:teuthology.orchestra.run.smithi046.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 18.2.4-855-g19e2a2be-1focal [13.9 kB] 2024-09-21T14:55:28.672 INFO:teuthology.orchestra.run.smithi046.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 18.2.4-855-g19e2a2be-1focal [46.9 MB] 2024-09-21T14:55:28.727 INFO:teuthology.orchestra.run.smithi032.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 18.2.4-855-g19e2a2be-1focal [13.7 kB] 2024-09-21T14:55:28.728 INFO:teuthology.orchestra.run.smithi032.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 18.2.4-855-g19e2a2be-1focal [856 kB] 2024-09-21T14:55:28.739 INFO:teuthology.orchestra.run.smithi032.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 18.2.4-855-g19e2a2be-1focal [2,295 kB] 2024-09-21T14:55:28.771 INFO:teuthology.orchestra.run.smithi032.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 18.2.4-855-g19e2a2be-1focal [239 kB] 2024-09-21T14:55:28.775 INFO:teuthology.orchestra.run.smithi032.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 18.2.4-855-g19e2a2be-1focal [143 kB] 2024-09-21T14:55:28.777 INFO:teuthology.orchestra.run.smithi032.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 18.2.4-855-g19e2a2be-1focal [7,015 kB] 2024-09-21T14:55:29.052 INFO:teuthology.orchestra.run.smithi032.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 18.2.4-855-g19e2a2be-1focal [7,893 kB] 2024-09-21T14:55:29.339 INFO:teuthology.orchestra.run.smithi032.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 18.2.4-855-g19e2a2be-1focal [13.9 kB] 2024-09-21T14:55:29.339 INFO:teuthology.orchestra.run.smithi032.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 18.2.4-855-g19e2a2be-1focal [46.9 MB] 2024-09-21T14:55:30.269 INFO:teuthology.orchestra.run.smithi046.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 ceph-volume all 18.2.4-855-g19e2a2be-1focal [120 kB] 2024-09-21T14:55:30.271 INFO:teuthology.orchestra.run.smithi046.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 18.2.4-855-g19e2a2be-1focal [37.4 kB] 2024-09-21T14:55:30.272 INFO:teuthology.orchestra.run.smithi046.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 18.2.4-855-g19e2a2be-1focal [13.5 MB] 2024-09-21T14:55:30.636 INFO:teuthology.orchestra.run.smithi046.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 18.2.4-855-g19e2a2be-1focal [108 kB] 2024-09-21T14:55:30.893 INFO:teuthology.orchestra.run.smithi032.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 ceph-volume all 18.2.4-855-g19e2a2be-1focal [120 kB] 2024-09-21T14:55:30.895 INFO:teuthology.orchestra.run.smithi032.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 18.2.4-855-g19e2a2be-1focal [37.4 kB] 2024-09-21T14:55:30.896 INFO:teuthology.orchestra.run.smithi032.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 18.2.4-855-g19e2a2be-1focal [13.5 MB] 2024-09-21T14:55:31.070 INFO:teuthology.orchestra.run.smithi032.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 18.2.4-855-g19e2a2be-1focal [108 kB] 2024-09-21T14:55:31.308 INFO:teuthology.orchestra.run.smithi046.stdout:Fetched 199 MB in 7s (27.5 MB/s) 2024-09-21T14:55:31.480 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2024-09-21T14:55:31.535 INFO:teuthology.orchestra.run.smithi046.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 141720 files and directories currently installed.) 2024-09-21T14:55:31.539 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../000-liblttng-ust0_2.11.0-1_amd64.deb ... 2024-09-21T14:55:31.629 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2024-09-21T14:55:31.734 INFO:teuthology.orchestra.run.smithi032.stdout:Fetched 199 MB in 8s (26.0 MB/s) 2024-09-21T14:55:31.905 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2024-09-21T14:55:31.965 INFO:teuthology.orchestra.run.smithi032.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 141720 files and directories currently installed.) 2024-09-21T14:55:31.969 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../000-liblttng-ust0_2.11.0-1_amd64.deb ... 2024-09-21T14:55:31.990 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-21T14:55:32.010 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.5-4ubuntu1_amd64.deb ... 2024-09-21T14:55:32.048 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-09-21T14:55:32.062 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2024-09-21T14:55:32.343 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-21T14:55:32.361 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../002-libpcre2-16-0_10.34-7ubuntu0.1_amd64.deb ... 2024-09-21T14:55:32.393 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-09-21T14:55:32.415 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-21T14:55:32.436 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.5-4ubuntu1_amd64.deb ... 2024-09-21T14:55:32.473 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-09-21T14:55:32.801 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-21T14:55:32.820 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../002-libpcre2-16-0_10.34-7ubuntu0.1_amd64.deb ... 2024-09-21T14:55:32.871 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-21T14:55:32.885 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-09-21T14:55:32.887 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../003-libqt5core5a_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-21T14:55:32.940 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-21T14:55:33.322 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-21T14:55:33.338 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../003-libqt5core5a_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-21T14:55:33.390 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-21T14:55:33.450 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-21T14:55:33.466 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../004-libqt5dbus5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-21T14:55:33.507 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-21T14:55:33.844 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-21T14:55:33.862 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../005-libqt5network5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-21T14:55:33.891 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-21T14:55:33.893 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-21T14:55:33.911 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../004-libqt5dbus5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-21T14:55:33.949 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-21T14:55:34.263 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libthrift-0.13.0. 2024-09-21T14:55:34.281 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../006-libthrift-0.13.0_0.13.0-2build2_amd64.deb ... 2024-09-21T14:55:34.303 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-21T14:55:34.321 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../005-libqt5network5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-21T14:55:34.321 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libthrift-0.13.0 (0.13.0-2build2) ... 2024-09-21T14:55:34.361 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-21T14:55:34.699 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../007-librbd1_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-21T14:55:34.723 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libthrift-0.13.0. 2024-09-21T14:55:34.739 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../006-libthrift-0.13.0_0.13.0-2build2_amd64.deb ... 2024-09-21T14:55:34.781 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libthrift-0.13.0 (0.13.0-2build2) ... 2024-09-21T14:55:34.825 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking librbd1 (18.2.4-855-g19e2a2be-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-09-21T14:55:35.130 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../007-librbd1_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-21T14:55:35.251 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking librbd1 (18.2.4-855-g19e2a2be-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-09-21T14:55:35.514 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../008-librados2_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-21T14:55:35.612 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking librados2 (18.2.4-855-g19e2a2be-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-09-21T14:55:35.949 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../008-librados2_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-21T14:55:36.097 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking librados2 (18.2.4-855-g19e2a2be-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-09-21T14:55:36.333 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libcephfs2. 2024-09-21T14:55:36.352 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../009-libcephfs2_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-21T14:55:36.391 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libcephfs2 (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:55:36.718 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-rados. 2024-09-21T14:55:36.734 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../010-python3-rados_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-21T14:55:36.792 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libcephfs2. 2024-09-21T14:55:36.801 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-rados (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:55:36.810 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../009-libcephfs2_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-21T14:55:36.850 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libcephfs2 (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:55:37.121 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-21T14:55:37.138 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../011-python3-ceph-argparse_18.2.4-855-g19e2a2be-1focal_all.deb ... 2024-09-21T14:55:37.171 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-ceph-argparse (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:55:37.178 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-rados. 2024-09-21T14:55:37.194 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../010-python3-rados_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-21T14:55:37.236 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-rados (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:55:37.373 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-cephfs. 2024-09-21T14:55:37.390 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../012-python3-cephfs_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-21T14:55:37.431 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-cephfs (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:55:37.614 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-21T14:55:37.633 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../011-python3-ceph-argparse_18.2.4-855-g19e2a2be-1focal_all.deb ... 2024-09-21T14:55:37.672 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-ceph-argparse (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:55:37.792 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-21T14:55:37.808 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../013-python3-ceph-common_18.2.4-855-g19e2a2be-1focal_all.deb ... 2024-09-21T14:55:37.850 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-ceph-common (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:55:38.002 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-cephfs. 2024-09-21T14:55:38.021 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../012-python3-cephfs_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-21T14:55:38.116 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-cephfs (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:55:38.187 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-prettytable. 2024-09-21T14:55:38.203 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../014-python3-prettytable_0.7.2-5_all.deb ... 2024-09-21T14:55:38.305 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2024-09-21T14:55:38.411 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-21T14:55:38.429 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../013-python3-ceph-common_18.2.4-855-g19e2a2be-1focal_all.deb ... 2024-09-21T14:55:38.460 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-ceph-common (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:55:38.698 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-rbd. 2024-09-21T14:55:38.717 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../015-python3-rbd_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-21T14:55:38.755 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-rbd (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:55:38.772 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-prettytable. 2024-09-21T14:55:38.790 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../014-python3-prettytable_0.7.2-5_all.deb ... 2024-09-21T14:55:38.830 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2024-09-21T14:55:39.066 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-rbd. 2024-09-21T14:55:39.075 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2024-09-21T14:55:39.085 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../015-python3-rbd_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-21T14:55:39.093 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../016-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2024-09-21T14:55:39.124 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-rbd (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:55:39.125 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-09-21T14:55:39.494 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2024-09-21T14:55:39.510 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../016-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2024-09-21T14:55:39.511 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package librabbitmq4:amd64. 2024-09-21T14:55:39.529 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../017-librabbitmq4_0.10.0-1_amd64.deb ... 2024-09-21T14:55:39.551 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-09-21T14:55:39.569 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2024-09-21T14:55:39.863 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package librabbitmq4:amd64. 2024-09-21T14:55:39.881 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../017-librabbitmq4_0.10.0-1_amd64.deb ... 2024-09-21T14:55:39.906 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-21T14:55:39.913 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2024-09-21T14:55:39.922 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../018-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2024-09-21T14:55:39.964 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-09-21T14:55:40.241 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-21T14:55:40.258 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../018-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2024-09-21T14:55:40.299 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-09-21T14:55:40.366 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package librgw2. 2024-09-21T14:55:40.383 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../019-librgw2_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-21T14:55:40.424 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking librgw2 (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:55:40.643 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package librgw2. 2024-09-21T14:55:40.664 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../019-librgw2_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-21T14:55:40.760 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking librgw2 (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:55:41.311 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-rgw. 2024-09-21T14:55:41.327 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../020-python3-rgw_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-21T14:55:41.368 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-rgw (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:55:41.646 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-rgw. 2024-09-21T14:55:41.663 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../020-python3-rgw_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-21T14:55:41.688 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package liboath0. 2024-09-21T14:55:41.704 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-rgw (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:55:41.704 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../021-liboath0_2.6.1-1.3_amd64.deb ... 2024-09-21T14:55:41.755 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2024-09-21T14:55:41.998 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package liboath0. 2024-09-21T14:55:42.007 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libradosstriper1. 2024-09-21T14:55:42.017 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../021-liboath0_2.6.1-1.3_amd64.deb ... 2024-09-21T14:55:42.023 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../022-libradosstriper1_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-21T14:55:42.056 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2024-09-21T14:55:42.065 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libradosstriper1 (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:55:42.401 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libradosstriper1. 2024-09-21T14:55:42.418 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../022-libradosstriper1_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-21T14:55:42.459 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libradosstriper1 (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:55:42.561 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-common. 2024-09-21T14:55:42.580 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../023-ceph-common_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-21T14:55:42.619 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-common (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:55:42.930 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-common. 2024-09-21T14:55:42.948 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../023-ceph-common_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-21T14:55:42.988 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-common (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:55:45.099 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-base. 2024-09-21T14:55:45.115 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../024-ceph-base_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-21T14:55:45.159 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-base (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:55:45.518 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-base. 2024-09-21T14:55:45.534 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../024-ceph-base_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-21T14:55:45.586 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-base (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:55:46.303 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-21T14:55:46.321 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../025-python3-cherrypy3_8.9.1-6_all.deb ... 2024-09-21T14:55:46.353 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2024-09-21T14:55:46.681 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-natsort. 2024-09-21T14:55:46.700 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../026-python3-natsort_7.0.1-1_all.deb ... 2024-09-21T14:55:46.731 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-natsort (7.0.1-1) ... 2024-09-21T14:55:46.814 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-21T14:55:46.831 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../025-python3-cherrypy3_8.9.1-6_all.deb ... 2024-09-21T14:55:46.864 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2024-09-21T14:55:47.000 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-logutils. 2024-09-21T14:55:47.020 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../027-python3-logutils_0.3.3-7_all.deb ... 2024-09-21T14:55:47.058 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-logutils (0.3.3-7) ... 2024-09-21T14:55:47.167 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-natsort. 2024-09-21T14:55:47.185 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../026-python3-natsort_7.0.1-1_all.deb ... 2024-09-21T14:55:47.225 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-natsort (7.0.1-1) ... 2024-09-21T14:55:47.345 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-21T14:55:47.364 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../028-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2024-09-21T14:55:47.403 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2024-09-21T14:55:47.503 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-logutils. 2024-09-21T14:55:47.519 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../027-python3-logutils_0.3.3-7_all.deb ... 2024-09-21T14:55:47.554 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-logutils (0.3.3-7) ... 2024-09-21T14:55:47.747 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-mako. 2024-09-21T14:55:47.763 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../029-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2024-09-21T14:55:47.805 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-09-21T14:55:47.856 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-21T14:55:47.874 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../028-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2024-09-21T14:55:47.914 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2024-09-21T14:55:48.117 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-21T14:55:48.136 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../030-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-21T14:55:48.175 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-21T14:55:48.200 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-mako. 2024-09-21T14:55:48.220 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../029-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2024-09-21T14:55:48.258 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-09-21T14:55:48.469 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-21T14:55:48.489 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../031-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2024-09-21T14:55:48.528 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2024-09-21T14:55:48.587 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-21T14:55:48.603 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../030-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-21T14:55:48.645 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-21T14:55:48.830 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-webob. 2024-09-21T14:55:48.846 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../032-python3-webob_1%3a1.8.5-2ubuntu0.1_all.deb ... 2024-09-21T14:55:48.888 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-webob (1:1.8.5-2ubuntu0.1) ... 2024-09-21T14:55:48.931 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-21T14:55:48.950 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../031-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2024-09-21T14:55:48.981 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2024-09-21T14:55:49.233 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-soupsieve. 2024-09-21T14:55:49.249 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../033-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2024-09-21T14:55:49.259 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-webob. 2024-09-21T14:55:49.275 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../032-python3-webob_1%3a1.8.5-2ubuntu0.1_all.deb ... 2024-09-21T14:55:49.291 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2024-09-21T14:55:49.316 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-webob (1:1.8.5-2ubuntu0.1) ... 2024-09-21T14:55:49.594 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-bs4. 2024-09-21T14:55:49.611 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../034-python3-bs4_4.8.2-1_all.deb ... 2024-09-21T14:55:49.644 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-soupsieve. 2024-09-21T14:55:49.652 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2024-09-21T14:55:49.662 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../033-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2024-09-21T14:55:49.705 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2024-09-21T14:55:49.999 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-bs4. 2024-09-21T14:55:50.016 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../034-python3-bs4_4.8.2-1_all.deb ... 2024-09-21T14:55:50.022 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-waitress. 2024-09-21T14:55:50.039 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../035-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2024-09-21T14:55:50.049 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2024-09-21T14:55:50.083 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2024-09-21T14:55:50.408 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-tempita. 2024-09-21T14:55:50.425 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../036-python3-tempita_0.5.2-6_all.deb ... 2024-09-21T14:55:50.435 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-waitress. 2024-09-21T14:55:50.456 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../035-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2024-09-21T14:55:50.466 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-tempita (0.5.2-6) ... 2024-09-21T14:55:50.496 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2024-09-21T14:55:50.796 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-paste. 2024-09-21T14:55:50.814 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../037-python3-paste_3.4.0+dfsg1-1_all.deb ... 2024-09-21T14:55:50.822 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-tempita. 2024-09-21T14:55:50.838 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../036-python3-tempita_0.5.2-6_all.deb ... 2024-09-21T14:55:50.854 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2024-09-21T14:55:50.880 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-tempita (0.5.2-6) ... 2024-09-21T14:55:51.158 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-21T14:55:51.175 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../038-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2024-09-21T14:55:51.183 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-paste. 2024-09-21T14:55:51.199 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../037-python3-paste_3.4.0+dfsg1-1_all.deb ... 2024-09-21T14:55:51.216 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2024-09-21T14:55:51.241 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2024-09-21T14:55:51.519 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-21T14:55:51.535 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../039-python3-pastedeploy_2.1.0-1_all.deb ... 2024-09-21T14:55:51.577 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-21T14:55:51.594 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../038-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2024-09-21T14:55:51.610 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2024-09-21T14:55:51.635 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2024-09-21T14:55:51.905 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-webtest. 2024-09-21T14:55:51.925 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../040-python3-webtest_2.0.32-2_all.deb ... 2024-09-21T14:55:51.938 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-21T14:55:51.954 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../039-python3-pastedeploy_2.1.0-1_all.deb ... 2024-09-21T14:55:51.963 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-webtest (2.0.32-2) ... 2024-09-21T14:55:52.013 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2024-09-21T14:55:52.274 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-pecan. 2024-09-21T14:55:52.291 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../041-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2024-09-21T14:55:52.332 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2024-09-21T14:55:52.333 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-webtest. 2024-09-21T14:55:52.350 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../040-python3-webtest_2.0.32-2_all.deb ... 2024-09-21T14:55:52.391 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-webtest (2.0.32-2) ... 2024-09-21T14:55:52.710 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-pecan. 2024-09-21T14:55:52.721 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-21T14:55:52.727 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../041-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2024-09-21T14:55:52.738 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../042-python3-werkzeug_0.16.1+dfsg1-2ubuntu0.2_all.deb ... 2024-09-21T14:55:52.768 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2024-09-21T14:55:52.779 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-09-21T14:55:53.074 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-21T14:55:53.090 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../043-ceph-mgr-modules-core_18.2.4-855-g19e2a2be-1focal_all.deb ... 2024-09-21T14:55:53.115 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-21T14:55:53.131 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../042-python3-werkzeug_0.16.1+dfsg1-2ubuntu0.2_all.deb ... 2024-09-21T14:55:53.140 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-mgr-modules-core (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:55:53.173 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-09-21T14:55:53.568 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-21T14:55:53.584 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../044-libsqlite3-mod-ceph_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-21T14:55:53.618 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libsqlite3-mod-ceph (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:55:53.838 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-21T14:55:53.857 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../043-ceph-mgr-modules-core_18.2.4-855-g19e2a2be-1focal_all.deb ... 2024-09-21T14:55:54.314 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-mgr-modules-core (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:55:54.685 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-21T14:55:54.688 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-21T14:55:54.702 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../044-libsqlite3-mod-ceph_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-21T14:55:54.709 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../045-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2024-09-21T14:55:54.743 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libsqlite3-mod-ceph (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:55:54.746 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2024-09-21T14:55:55.074 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-cryptography. 2024-09-21T14:55:55.079 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-21T14:55:55.093 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../046-python3-cryptography_2.8-3ubuntu0.3_amd64.deb ... 2024-09-21T14:55:55.097 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../045-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2024-09-21T14:55:55.132 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.3) ... 2024-09-21T14:55:55.137 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2024-09-21T14:55:55.482 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-cryptography. 2024-09-21T14:55:55.498 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../046-python3-cryptography_2.8-3ubuntu0.3_amd64.deb ... 2024-09-21T14:55:55.540 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.3) ... 2024-09-21T14:55:55.544 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-openssl. 2024-09-21T14:55:55.560 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../047-python3-openssl_19.0.0-1build1_all.deb ... 2024-09-21T14:55:55.602 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2024-09-21T14:55:55.919 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-openssl. 2024-09-21T14:55:55.935 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../047-python3-openssl_19.0.0-1build1_all.deb ... 2024-09-21T14:55:55.977 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2024-09-21T14:55:55.997 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-mgr. 2024-09-21T14:55:56.014 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../048-ceph-mgr_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-21T14:55:56.055 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-mgr (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:55:56.322 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-mgr. 2024-09-21T14:55:56.338 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../048-ceph-mgr_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-21T14:55:56.380 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-mgr (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:55:56.559 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-mon. 2024-09-21T14:55:56.575 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../049-ceph-mon_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-21T14:55:56.617 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-mon (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:55:56.892 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-mon. 2024-09-21T14:55:56.908 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../049-ceph-mon_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-21T14:55:56.950 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-mon (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:55:57.637 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-osd. 2024-09-21T14:55:57.654 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../050-ceph-osd_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-21T14:55:57.696 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-osd (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:55:57.874 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-osd. 2024-09-21T14:55:57.893 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../050-ceph-osd_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-21T14:55:57.932 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-osd (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:55:59.870 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph. 2024-09-21T14:55:59.886 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../051-ceph_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-21T14:55:59.919 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:56:00.168 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph. 2024-09-21T14:56:00.186 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../051-ceph_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-21T14:56:00.226 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:56:00.289 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-fuse. 2024-09-21T14:56:00.306 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../052-ceph-fuse_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-21T14:56:00.347 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-fuse (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:56:00.536 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-fuse. 2024-09-21T14:56:00.552 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../052-ceph-fuse_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-21T14:56:00.594 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-fuse (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:56:00.794 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-mds. 2024-09-21T14:56:00.813 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../053-ceph-mds_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-21T14:56:00.857 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-mds (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:56:01.005 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-mds. 2024-09-21T14:56:01.022 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../053-ceph-mds_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-21T14:56:01.063 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-mds (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:56:01.344 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package cephadm. 2024-09-21T14:56:01.360 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../054-cephadm_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-21T14:56:01.393 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking cephadm (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:56:01.600 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package cephadm. 2024-09-21T14:56:01.618 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../054-cephadm_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-21T14:56:01.658 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking cephadm (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:56:01.680 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-21T14:56:01.697 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../055-python3-asyncssh_1.12.2-1_all.deb ... 2024-09-21T14:56:01.738 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-asyncssh (1.12.2-1) ... 2024-09-21T14:56:01.960 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-21T14:56:01.980 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../055-python3-asyncssh_1.12.2-1_all.deb ... 2024-09-21T14:56:02.018 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-asyncssh (1.12.2-1) ... 2024-09-21T14:56:02.091 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-jinja2. 2024-09-21T14:56:02.107 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../056-python3-jinja2_2.10.1-2ubuntu0.3_all.deb ... 2024-09-21T14:56:02.150 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-09-21T14:56:02.364 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-jinja2. 2024-09-21T14:56:02.380 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../056-python3-jinja2_2.10.1-2ubuntu0.3_all.deb ... 2024-09-21T14:56:02.422 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-09-21T14:56:02.494 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-21T14:56:02.511 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../057-ceph-mgr-cephadm_18.2.4-855-g19e2a2be-1focal_all.deb ... 2024-09-21T14:56:02.552 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-mgr-cephadm (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:56:02.750 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-21T14:56:02.766 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../057-ceph-mgr-cephadm_18.2.4-855-g19e2a2be-1focal_all.deb ... 2024-09-21T14:56:02.808 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-mgr-cephadm (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:56:02.914 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-jwt. 2024-09-21T14:56:02.933 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../058-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2024-09-21T14:56:02.972 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2024-09-21T14:56:03.119 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-jwt. 2024-09-21T14:56:03.136 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../058-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2024-09-21T14:56:03.202 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2024-09-21T14:56:03.326 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-attr. 2024-09-21T14:56:03.345 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../059-python3-attr_19.3.0-2_all.deb ... 2024-09-21T14:56:03.384 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-attr (19.3.0-2) ... 2024-09-21T14:56:03.513 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-attr. 2024-09-21T14:56:03.533 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../059-python3-attr_19.3.0-2_all.deb ... 2024-09-21T14:56:03.571 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-attr (19.3.0-2) ... 2024-09-21T14:56:03.711 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-pluggy. 2024-09-21T14:56:03.727 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../060-python3-pluggy_0.13.0-2_all.deb ... 2024-09-21T14:56:03.769 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2024-09-21T14:56:03.908 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-pluggy. 2024-09-21T14:56:03.926 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../060-python3-pluggy_0.13.0-2_all.deb ... 2024-09-21T14:56:03.966 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2024-09-21T14:56:04.081 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-py. 2024-09-21T14:56:04.098 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../061-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2024-09-21T14:56:04.156 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2024-09-21T14:56:04.252 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-py. 2024-09-21T14:56:04.269 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../061-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2024-09-21T14:56:04.310 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2024-09-21T14:56:04.492 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-atomicwrites. 2024-09-21T14:56:04.510 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../062-python3-atomicwrites_1.1.5-2build1_all.deb ... 2024-09-21T14:56:04.550 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2024-09-21T14:56:04.705 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-atomicwrites. 2024-09-21T14:56:04.724 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../062-python3-atomicwrites_1.1.5-2build1_all.deb ... 2024-09-21T14:56:04.788 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2024-09-21T14:56:04.941 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-packaging. 2024-09-21T14:56:04.961 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../063-python3-packaging_20.3-1_all.deb ... 2024-09-21T14:56:05.003 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-packaging (20.3-1) ... 2024-09-21T14:56:05.091 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-packaging. 2024-09-21T14:56:05.107 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../063-python3-packaging_20.3-1_all.deb ... 2024-09-21T14:56:05.149 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-packaging (20.3-1) ... 2024-09-21T14:56:05.273 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-21T14:56:05.291 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../064-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2024-09-21T14:56:05.331 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2024-09-21T14:56:05.452 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-21T14:56:05.468 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../064-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2024-09-21T14:56:05.510 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2024-09-21T14:56:05.576 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-pytest. 2024-09-21T14:56:05.592 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../065-python3-pytest_4.6.9-1_all.deb ... 2024-09-21T14:56:05.625 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-pytest (4.6.9-1) ... 2024-09-21T14:56:05.812 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-pytest. 2024-09-21T14:56:05.829 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../065-python3-pytest_4.6.9-1_all.deb ... 2024-09-21T14:56:05.870 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-pytest (4.6.9-1) ... 2024-09-21T14:56:05.979 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-21T14:56:05.995 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../066-python3-repoze.lru_0.7-2_all.deb ... 2024-09-21T14:56:06.037 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-21T14:56:06.216 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-21T14:56:06.236 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../066-python3-repoze.lru_0.7-2_all.deb ... 2024-09-21T14:56:06.274 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-21T14:56:06.315 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libjs-underscore. 2024-09-21T14:56:06.334 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../067-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2024-09-21T14:56:06.381 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-09-21T14:56:06.535 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libjs-underscore. 2024-09-21T14:56:06.551 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../067-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2024-09-21T14:56:06.593 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-09-21T14:56:06.667 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libjs-sphinxdoc. 2024-09-21T14:56:06.685 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../068-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2024-09-21T14:56:06.725 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-09-21T14:56:06.879 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libjs-sphinxdoc. 2024-09-21T14:56:06.898 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../068-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2024-09-21T14:56:06.938 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-09-21T14:56:07.029 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-routes. 2024-09-21T14:56:07.049 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../069-python3-routes_2.4.1-2_all.deb ... 2024-09-21T14:56:07.087 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-routes (2.4.1-2) ... 2024-09-21T14:56:07.240 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-routes. 2024-09-21T14:56:07.260 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../069-python3-routes_2.4.1-2_all.deb ... 2024-09-21T14:56:07.298 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-routes (2.4.1-2) ... 2024-09-21T14:56:07.458 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-21T14:56:07.474 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../070-ceph-mgr-dashboard_18.2.4-855-g19e2a2be-1focal_all.deb ... 2024-09-21T14:56:07.524 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-mgr-dashboard (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:56:07.643 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-21T14:56:07.659 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../070-ceph-mgr-dashboard_18.2.4-855-g19e2a2be-1focal_all.deb ... 2024-09-21T14:56:07.701 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-mgr-dashboard (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:56:08.896 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-decorator. 2024-09-21T14:56:08.912 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../071-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2024-09-21T14:56:08.953 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2024-09-21T14:56:09.023 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-decorator. 2024-09-21T14:56:09.042 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../071-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2024-09-21T14:56:09.081 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2024-09-21T14:56:09.282 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2024-09-21T14:56:09.299 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../072-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2024-09-21T14:56:09.340 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-09-21T14:56:09.409 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2024-09-21T14:56:09.425 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../072-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2024-09-21T14:56:09.467 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-09-21T14:56:09.668 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-scipy. 2024-09-21T14:56:09.684 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../073-python3-scipy_1.3.3-3build1_amd64.deb ... 2024-09-21T14:56:09.726 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2024-09-21T14:56:10.078 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-scipy. 2024-09-21T14:56:10.097 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../073-python3-scipy_1.3.3-3build1_amd64.deb ... 2024-09-21T14:56:10.136 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2024-09-21T14:56:11.085 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-sklearn-lib. 2024-09-21T14:56:11.103 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../074-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2024-09-21T14:56:11.143 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-09-21T14:56:11.428 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-sklearn-lib. 2024-09-21T14:56:11.446 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../074-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2024-09-21T14:56:11.486 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-09-21T14:56:11.697 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-joblib. 2024-09-21T14:56:11.713 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../075-python3-joblib_0.14.0-3_all.deb ... 2024-09-21T14:56:11.746 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-joblib (0.14.0-3) ... 2024-09-21T14:56:12.031 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-joblib. 2024-09-21T14:56:12.049 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../075-python3-joblib_0.14.0-3_all.deb ... 2024-09-21T14:56:12.089 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-joblib (0.14.0-3) ... 2024-09-21T14:56:12.125 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-sklearn. 2024-09-21T14:56:12.143 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../076-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2024-09-21T14:56:12.183 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-09-21T14:56:12.451 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-sklearn. 2024-09-21T14:56:12.470 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../076-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2024-09-21T14:56:12.509 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-09-21T14:56:12.867 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-21T14:56:12.884 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../077-ceph-mgr-diskprediction-local_18.2.4-855-g19e2a2be-1focal_all.deb ... 2024-09-21T14:56:12.925 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-mgr-diskprediction-local (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:56:13.217 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-21T14:56:13.236 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../077-ceph-mgr-diskprediction-local_18.2.4-855-g19e2a2be-1focal_all.deb ... 2024-09-21T14:56:13.275 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-mgr-diskprediction-local (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:56:14.078 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-cachetools. 2024-09-21T14:56:14.094 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../078-python3-cachetools_4.0.0-1_all.deb ... 2024-09-21T14:56:14.135 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2024-09-21T14:56:14.402 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-cachetools. 2024-09-21T14:56:14.419 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../078-python3-cachetools_4.0.0-1_all.deb ... 2024-09-21T14:56:14.423 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-21T14:56:14.439 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../079-python3-pyasn1_0.4.2-3build1_all.deb ... 2024-09-21T14:56:14.461 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2024-09-21T14:56:14.481 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2024-09-21T14:56:14.764 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-21T14:56:14.782 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../079-python3-pyasn1_0.4.2-3build1_all.deb ... 2024-09-21T14:56:14.822 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2024-09-21T14:56:14.842 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-21T14:56:14.859 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../080-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2024-09-21T14:56:14.900 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-09-21T14:56:15.150 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-21T14:56:15.167 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../080-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2024-09-21T14:56:15.208 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-09-21T14:56:15.549 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-rsa. 2024-09-21T14:56:15.565 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../081-python3-rsa_4.0-3_all.deb ... 2024-09-21T14:56:15.622 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-rsa. 2024-09-21T14:56:15.623 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-rsa (4.0-3) ... 2024-09-21T14:56:15.639 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../081-python3-rsa_4.0-3_all.deb ... 2024-09-21T14:56:15.680 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-rsa (4.0-3) ... 2024-09-21T14:56:15.944 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-google-auth. 2024-09-21T14:56:15.962 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../082-python3-google-auth_1.5.1-2_all.deb ... 2024-09-21T14:56:16.002 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2024-09-21T14:56:16.026 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-google-auth. 2024-09-21T14:56:16.043 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../082-python3-google-auth_1.5.1-2_all.deb ... 2024-09-21T14:56:16.084 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2024-09-21T14:56:16.323 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-blinker. 2024-09-21T14:56:16.339 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../083-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2024-09-21T14:56:16.380 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-09-21T14:56:16.388 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-blinker. 2024-09-21T14:56:16.408 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../083-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2024-09-21T14:56:16.446 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-09-21T14:56:16.704 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-oauthlib. 2024-09-21T14:56:16.722 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../084-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2024-09-21T14:56:16.752 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-oauthlib. 2024-09-21T14:56:16.762 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2024-09-21T14:56:16.772 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../084-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2024-09-21T14:56:16.810 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2024-09-21T14:56:17.091 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-21T14:56:17.108 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../085-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2024-09-21T14:56:17.114 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-21T14:56:17.131 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../085-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2024-09-21T14:56:17.149 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-09-21T14:56:17.172 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-09-21T14:56:17.477 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-websocket. 2024-09-21T14:56:17.494 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../086-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2024-09-21T14:56:17.517 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-websocket. 2024-09-21T14:56:17.534 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../086-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2024-09-21T14:56:17.535 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2024-09-21T14:56:17.575 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2024-09-21T14:56:17.847 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-21T14:56:17.863 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../087-python3-kubernetes_7.0.0~a1-2_all.deb ... 2024-09-21T14:56:17.903 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-21T14:56:17.905 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2024-09-21T14:56:17.919 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../087-python3-kubernetes_7.0.0~a1-2_all.deb ... 2024-09-21T14:56:17.969 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2024-09-21T14:56:18.411 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-21T14:56:18.430 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../088-ceph-mgr-k8sevents_18.2.4-855-g19e2a2be-1focal_all.deb ... 2024-09-21T14:56:18.469 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-mgr-k8sevents (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:56:18.517 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-21T14:56:18.534 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../088-ceph-mgr-k8sevents_18.2.4-855-g19e2a2be-1focal_all.deb ... 2024-09-21T14:56:18.575 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-mgr-k8sevents (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:56:18.814 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-21T14:56:18.831 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../089-libonig5_6.9.4-1_amd64.deb ... 2024-09-21T14:56:18.871 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2024-09-21T14:56:18.911 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-21T14:56:18.927 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../089-libonig5_6.9.4-1_amd64.deb ... 2024-09-21T14:56:18.969 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2024-09-21T14:56:19.237 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-21T14:56:19.255 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../090-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-09-21T14:56:19.295 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-09-21T14:56:19.342 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-21T14:56:19.361 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../090-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-09-21T14:56:19.408 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-09-21T14:56:19.540 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package jq. 2024-09-21T14:56:19.558 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../091-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-09-21T14:56:19.598 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2024-09-21T14:56:19.686 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package jq. 2024-09-21T14:56:19.703 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../091-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-09-21T14:56:19.744 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2024-09-21T14:56:19.878 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package socat. 2024-09-21T14:56:19.897 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../092-socat_1.7.3.3-2_amd64.deb ... 2024-09-21T14:56:19.936 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking socat (1.7.3.3-2) ... 2024-09-21T14:56:19.982 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package socat. 2024-09-21T14:56:19.999 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../092-socat_1.7.3.3-2_amd64.deb ... 2024-09-21T14:56:20.040 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking socat (1.7.3.3-2) ... 2024-09-21T14:56:20.231 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package xmlstarlet. 2024-09-21T14:56:20.250 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../093-xmlstarlet_1.6.1-2build1_amd64.deb ... 2024-09-21T14:56:20.289 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2024-09-21T14:56:20.362 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package xmlstarlet. 2024-09-21T14:56:20.379 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../093-xmlstarlet_1.6.1-2build1_amd64.deb ... 2024-09-21T14:56:20.420 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2024-09-21T14:56:20.654 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-test. 2024-09-21T14:56:20.671 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../094-ceph-test_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-21T14:56:20.712 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-test (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:56:20.774 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-test. 2024-09-21T14:56:20.793 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../094-ceph-test_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-21T14:56:20.832 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-test (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:56:24.824 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-volume. 2024-09-21T14:56:24.845 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../095-ceph-volume_18.2.4-855-g19e2a2be-1focal_all.deb ... 2024-09-21T14:56:24.891 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-volume (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:56:25.045 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-volume. 2024-09-21T14:56:25.062 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../095-ceph-volume_18.2.4-855-g19e2a2be-1focal_all.deb ... 2024-09-21T14:56:25.103 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-volume (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:56:25.178 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package fonts-mathjax. 2024-09-21T14:56:25.195 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../096-fonts-mathjax_2.7.4+dfsg-1_all.deb ... 2024-09-21T14:56:25.236 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking fonts-mathjax (2.7.4+dfsg-1) ... 2024-09-21T14:56:25.431 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package fonts-mathjax. 2024-09-21T14:56:25.448 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../096-fonts-mathjax_2.7.4+dfsg-1_all.deb ... 2024-09-21T14:56:25.489 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking fonts-mathjax (2.7.4+dfsg-1) ... 2024-09-21T14:56:25.765 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-21T14:56:25.785 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../097-libcephfs-dev_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-21T14:56:25.823 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libcephfs-dev (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:56:26.001 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-21T14:56:26.018 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../097-libcephfs-dev_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-21T14:56:26.059 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libcephfs-dev (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:56:26.243 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package nvme-cli. 2024-09-21T14:56:26.259 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../098-nvme-cli_1.9-1ubuntu0.2_amd64.deb ... 2024-09-21T14:56:26.319 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking nvme-cli (1.9-1ubuntu0.2) ... 2024-09-21T14:56:26.454 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package nvme-cli. 2024-09-21T14:56:26.471 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../098-nvme-cli_1.9-1ubuntu0.2_amd64.deb ... 2024-09-21T14:56:26.530 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking nvme-cli (1.9-1ubuntu0.2) ... 2024-09-21T14:56:26.631 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-21T14:56:26.650 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../099-python-asyncssh-doc_1.12.2-1_all.deb ... 2024-09-21T14:56:26.689 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python-asyncssh-doc (1.12.2-1) ... 2024-09-21T14:56:26.825 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-21T14:56:26.842 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../099-python-asyncssh-doc_1.12.2-1_all.deb ... 2024-09-21T14:56:26.883 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python-asyncssh-doc (1.12.2-1) ... 2024-09-21T14:56:27.076 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libjs-mathjax. 2024-09-21T14:56:27.094 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../100-libjs-mathjax_2.7.4+dfsg-1_all.deb ... 2024-09-21T14:56:27.134 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libjs-mathjax (2.7.4+dfsg-1) ... 2024-09-21T14:56:27.278 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libjs-mathjax. 2024-09-21T14:56:27.296 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../100-libjs-mathjax_2.7.4+dfsg-1_all.deb ... 2024-09-21T14:56:27.336 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libjs-mathjax (2.7.4+dfsg-1) ... 2024-09-21T14:56:28.828 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python-natsort-doc. 2024-09-21T14:56:28.846 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../101-python-natsort-doc_7.0.1-1_all.deb ... 2024-09-21T14:56:28.886 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python-natsort-doc (7.0.1-1) ... 2024-09-21T14:56:28.968 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python-natsort-doc. 2024-09-21T14:56:28.987 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../101-python-natsort-doc_7.0.1-1_all.deb ... 2024-09-21T14:56:29.026 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python-natsort-doc (7.0.1-1) ... 2024-09-21T14:56:29.223 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-webencodings. 2024-09-21T14:56:29.240 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../102-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2024-09-21T14:56:29.280 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2024-09-21T14:56:29.371 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-webencodings. 2024-09-21T14:56:29.388 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../102-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2024-09-21T14:56:29.429 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2024-09-21T14:56:29.609 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-html5lib. 2024-09-21T14:56:29.626 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../103-python3-html5lib_1.0.1-2_all.deb ... 2024-09-21T14:56:29.667 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2024-09-21T14:56:29.732 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-html5lib. 2024-09-21T14:56:29.750 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../103-python3-html5lib_1.0.1-2_all.deb ... 2024-09-21T14:56:29.790 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2024-09-21T14:56:30.011 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-lxml:amd64. 2024-09-21T14:56:30.028 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../104-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2024-09-21T14:56:30.069 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-09-21T14:56:30.101 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-lxml:amd64. 2024-09-21T14:56:30.118 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../104-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2024-09-21T14:56:30.151 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-09-21T14:56:30.548 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-pastescript. 2024-09-21T14:56:30.565 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../105-python3-pastescript_2.0.2-4_all.deb ... 2024-09-21T14:56:30.596 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-pastescript. 2024-09-21T14:56:30.606 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-21T14:56:30.614 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../105-python3-pastescript_2.0.2-4_all.deb ... 2024-09-21T14:56:30.654 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-21T14:56:30.934 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-psutil. 2024-09-21T14:56:30.941 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-psutil. 2024-09-21T14:56:30.954 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../106-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2024-09-21T14:56:30.961 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../106-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2024-09-21T14:56:30.990 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2024-09-21T14:56:30.992 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2024-09-21T14:56:31.335 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-21T14:56:31.352 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../107-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2024-09-21T14:56:31.370 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-21T14:56:31.390 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../107-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2024-09-21T14:56:31.393 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-09-21T14:56:31.437 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-09-21T14:56:31.704 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-simplejson. 2024-09-21T14:56:31.723 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../108-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2024-09-21T14:56:31.748 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-simplejson. 2024-09-21T14:56:31.762 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2024-09-21T14:56:31.765 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../108-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2024-09-21T14:56:31.806 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2024-09-21T14:56:32.056 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-21T14:56:32.067 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-21T14:56:32.074 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../109-qttranslations5-l10n_5.12.8-0ubuntu1_all.deb ... 2024-09-21T14:56:32.086 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../109-qttranslations5-l10n_5.12.8-0ubuntu1_all.deb ... 2024-09-21T14:56:32.114 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-09-21T14:56:32.125 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-09-21T14:56:32.788 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package radosgw. 2024-09-21T14:56:32.808 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../110-radosgw_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-21T14:56:32.845 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking radosgw (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:56:32.847 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package radosgw. 2024-09-21T14:56:32.864 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../110-radosgw_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-21T14:56:32.904 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking radosgw (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:56:34.249 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package rbd-fuse. 2024-09-21T14:56:34.269 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../111-rbd-fuse_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-21T14:56:34.298 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking rbd-fuse (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:56:34.318 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package rbd-fuse. 2024-09-21T14:56:34.336 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../111-rbd-fuse_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-21T14:56:34.393 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking rbd-fuse (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:56:34.582 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-attr (19.3.0-2) ... 2024-09-21T14:56:34.690 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-attr (19.3.0-2) ... 2024-09-21T14:56:34.878 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2024-09-21T14:56:34.981 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2024-09-21T14:56:35.037 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-09-21T14:56:35.107 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-09-21T14:56:35.164 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up nvme-cli (1.9-1ubuntu0.2) ... 2024-09-21T14:56:35.258 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up nvme-cli (1.9-1ubuntu0.2) ... 2024-09-21T14:56:35.365 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up cephadm (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:56:35.468 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up cephadm (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:56:35.918 INFO:teuthology.orchestra.run.smithi032.stdout:Adding system user cephadm....done 2024-09-21T14:56:36.065 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up fonts-mathjax (2.7.4+dfsg-1) ... 2024-09-21T14:56:36.088 INFO:teuthology.orchestra.run.smithi046.stdout:Adding system user cephadm....done 2024-09-21T14:56:36.191 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2024-09-21T14:56:36.243 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up fonts-mathjax (2.7.4+dfsg-1) ... 2024-09-21T14:56:36.369 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2024-09-21T14:56:36.569 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-21T14:56:36.730 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-21T14:56:36.877 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libjs-mathjax (2.7.4+dfsg-1) ... 2024-09-21T14:56:37.002 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up liboath0 (2.6.1-1.3) ... 2024-09-21T14:56:37.064 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libjs-mathjax (2.7.4+dfsg-1) ... 2024-09-21T14:56:37.128 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2024-09-21T14:56:37.190 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up liboath0 (2.6.1-1.3) ... 2024-09-21T14:56:37.316 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2024-09-21T14:56:37.472 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-joblib (0.14.0-3) ... 2024-09-21T14:56:37.651 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-joblib (0.14.0-3) ... 2024-09-21T14:56:37.917 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2024-09-21T14:56:38.055 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2024-09-21T14:56:38.226 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-cachetools (4.0.0-1) ... 2024-09-21T14:56:38.331 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-cachetools (4.0.0-1) ... 2024-09-21T14:56:38.527 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2024-09-21T14:56:38.632 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2024-09-21T14:56:38.837 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-09-21T14:56:38.900 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-09-21T14:56:39.146 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-ceph-argparse (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:56:39.201 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-ceph-argparse (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:56:39.456 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-09-21T14:56:39.527 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-09-21T14:56:39.581 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2024-09-21T14:56:39.653 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2024-09-21T14:56:39.883 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2024-09-21T14:56:39.921 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2024-09-21T14:56:40.175 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-09-21T14:56:40.214 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-09-21T14:56:40.301 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2024-09-21T14:56:40.340 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2024-09-21T14:56:40.716 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-natsort (7.0.1-1) ... 2024-09-21T14:56:41.094 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-natsort (7.0.1-1) ... 2024-09-21T14:56:41.109 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2024-09-21T14:56:41.393 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2024-09-21T14:56:41.403 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2024-09-21T14:56:41.696 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2024-09-21T14:56:41.744 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2024-09-21T14:56:42.028 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-09-21T14:56:42.072 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2024-09-21T14:56:42.373 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-09-21T14:56:42.380 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-tempita (0.5.2-6) ... 2024-09-21T14:56:42.673 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-packaging (20.3-1) ... 2024-09-21T14:56:42.724 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-tempita (0.5.2-6) ... 2024-09-21T14:56:42.990 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2024-09-21T14:56:43.025 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-packaging (20.3-1) ... 2024-09-21T14:56:43.116 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-09-21T14:56:43.242 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2024-09-21T14:56:43.334 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2024-09-21T14:56:43.460 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-09-21T14:56:43.568 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2024-09-21T14:56:43.594 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2024-09-21T14:56:43.904 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2024-09-21T14:56:43.978 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-09-21T14:56:44.322 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-09-21T14:56:44.346 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-cryptography (2.8-3ubuntu0.3) ... 2024-09-21T14:56:44.699 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-cryptography (2.8-3ubuntu0.3) ... 2024-09-21T14:56:44.715 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up socat (1.7.3.3-2) ... 2024-09-21T14:56:44.841 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-ceph-common (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:56:45.067 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up socat (1.7.3.3-2) ... 2024-09-21T14:56:45.193 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-ceph-common (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:56:45.326 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-21T14:56:45.460 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-html5lib (1.0.1-2) ... 2024-09-21T14:56:45.536 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-21T14:56:45.670 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-html5lib (1.0.1-2) ... 2024-09-21T14:56:45.851 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-09-21T14:56:45.987 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2024-09-21T14:56:46.063 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-09-21T14:56:46.189 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2024-09-21T14:56:46.315 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-pluggy (0.13.0-2) ... 2024-09-21T14:56:46.501 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-pluggy (0.13.0-2) ... 2024-09-21T14:56:46.624 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-09-21T14:56:46.823 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-09-21T14:56:46.959 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-21T14:56:47.076 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-09-21T14:56:47.149 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-21T14:56:47.186 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2024-09-21T14:56:47.275 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-09-21T14:56:47.401 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2024-09-21T14:56:47.529 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2024-09-21T14:56:47.769 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2024-09-21T14:56:47.872 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-logutils (0.3.3-7) ... 2024-09-21T14:56:48.053 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-logutils (0.3.3-7) ... 2024-09-21T14:56:48.173 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2024-09-21T14:56:48.298 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-21T14:56:48.363 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2024-09-21T14:56:48.464 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-21T14:56:48.599 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-prettytable (0.7.2-5) ... 2024-09-21T14:56:48.757 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-prettytable (0.7.2-5) ... 2024-09-21T14:56:48.900 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2024-09-21T14:56:49.058 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2024-09-21T14:56:49.218 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2024-09-21T14:56:49.392 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2024-09-21T14:56:49.583 INFO:teuthology.orchestra.run.smithi032.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2024-09-21T14:56:49.767 INFO:teuthology.orchestra.run.smithi046.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2024-09-21T14:56:49.829 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2024-09-21T14:56:49.963 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-09-21T14:56:50.020 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2024-09-21T14:56:50.154 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-09-21T14:56:50.298 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2024-09-21T14:56:50.497 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2024-09-21T14:56:50.682 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-09-21T14:56:50.825 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-09-21T14:56:50.865 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-09-21T14:56:50.951 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-webob (1:1.8.5-2ubuntu0.1) ... 2024-09-21T14:56:50.991 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-09-21T14:56:51.117 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-webob (1:1.8.5-2ubuntu0.1) ... 2024-09-21T14:56:51.302 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-09-21T14:56:51.460 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-09-21T14:56:51.603 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2024-09-21T14:56:51.786 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2024-09-21T14:56:53.108 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-09-21T14:56:53.234 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-pytest (4.6.9-1) ... 2024-09-21T14:56:53.308 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-09-21T14:56:53.442 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-pytest (4.6.9-1) ... 2024-09-21T14:56:53.602 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2024-09-21T14:56:53.827 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2024-09-21T14:56:53.945 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2024-09-21T14:56:54.145 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2024-09-21T14:56:54.263 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-21T14:56:54.397 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-bs4 (4.8.2-1) ... 2024-09-21T14:56:54.455 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-21T14:56:54.564 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-bs4 (4.8.2-1) ... 2024-09-21T14:56:54.740 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2024-09-21T14:56:54.940 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2024-09-21T14:56:55.075 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-asyncssh (1.12.2-1) ... 2024-09-21T14:56:55.275 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-asyncssh (1.12.2-1) ... 2024-09-21T14:56:55.468 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-09-21T14:56:55.710 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-09-21T14:56:55.827 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-09-21T14:56:56.053 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-09-21T14:56:56.916 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-09-21T14:56:57.042 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python-natsort-doc (7.0.1-1) ... 2024-09-21T14:56:57.133 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-09-21T14:56:57.193 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2024-09-21T14:56:57.259 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python-natsort-doc (7.0.1-1) ... 2024-09-21T14:56:57.319 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-rsa (4.0-3) ... 2024-09-21T14:56:57.410 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2024-09-21T14:56:57.536 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-rsa (4.0-3) ... 2024-09-21T14:56:57.628 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libthrift-0.13.0 (0.13.0-2build2) ... 2024-09-21T14:56:57.729 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-webtest (2.0.32-2) ... 2024-09-21T14:56:57.854 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libthrift-0.13.0 (0.13.0-2build2) ... 2024-09-21T14:56:57.980 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-webtest (2.0.32-2) ... 2024-09-21T14:56:58.106 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-09-21T14:56:58.273 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-09-21T14:56:58.398 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-21T14:56:58.583 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-21T14:56:58.725 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2024-09-21T14:56:58.909 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2024-09-21T14:56:59.093 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up librados2 (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:56:59.219 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python-asyncssh-doc (1.12.2-1) ... 2024-09-21T14:56:59.311 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up librados2 (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:56:59.337 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up librgw2 (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:56:59.437 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python-asyncssh-doc (1.12.2-1) ... 2024-09-21T14:56:59.455 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libsqlite3-mod-ceph (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:56:59.571 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up librgw2 (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:56:59.658 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libcephfs2 (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:56:59.706 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libsqlite3-mod-ceph (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:56:59.783 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-routes (2.4.1-2) ... 2024-09-21T14:56:59.942 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libcephfs2 (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:57:00.085 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-routes (2.4.1-2) ... 2024-09-21T14:57:00.101 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libradosstriper1 (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:57:00.227 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-google-auth (1.5.1-2) ... 2024-09-21T14:57:00.386 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libradosstriper1 (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:57:00.512 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-google-auth (1.5.1-2) ... 2024-09-21T14:57:00.554 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up librbd1 (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:57:00.682 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-mgr-modules-core (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:57:00.807 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up librbd1 (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:57:00.808 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-fuse (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:57:00.934 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-mgr-modules-core (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:57:01.039 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-21T14:57:01.039 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-21T14:57:01.059 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-fuse (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:57:01.295 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-21T14:57:01.295 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-21T14:57:01.485 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libcephfs-dev (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:57:01.611 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-rados (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:57:01.736 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2024-09-21T14:57:01.761 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libcephfs-dev (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:57:01.887 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-rados (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:57:02.013 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2024-09-21T14:57:02.699 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-rbd (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:57:02.825 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up rbd-fuse (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:57:02.951 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-rgw (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:57:02.958 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-rbd (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:57:03.077 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-cephfs (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:57:03.109 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up rbd-fuse (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:57:03.203 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-common (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:57:03.227 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-rgw (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:57:03.353 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-cephfs (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:57:03.479 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-common (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:57:03.772 INFO:teuthology.orchestra.run.smithi032.stdout:Adding group ceph....done 2024-09-21T14:57:04.073 INFO:teuthology.orchestra.run.smithi046.stdout:Adding group ceph....done 2024-09-21T14:57:04.165 INFO:teuthology.orchestra.run.smithi032.stdout:Adding system user ceph....done 2024-09-21T14:57:04.342 INFO:teuthology.orchestra.run.smithi032.stdout:Setting system user ceph properties....done 2024-09-21T14:57:04.367 INFO:teuthology.orchestra.run.smithi032.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-21T14:57:04.458 INFO:teuthology.orchestra.run.smithi046.stdout:Adding system user ceph....done 2024-09-21T14:57:04.519 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-21T14:57:04.626 INFO:teuthology.orchestra.run.smithi046.stdout:Setting system user ceph properties....done 2024-09-21T14:57:04.648 INFO:teuthology.orchestra.run.smithi046.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-21T14:57:04.808 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-21T14:57:04.840 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-21T14:57:05.145 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-21T14:57:05.304 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-test (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:57:05.430 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up radosgw (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:57:05.605 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-test (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:57:05.739 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up radosgw (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:57:05.889 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-21T14:57:05.889 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-21T14:57:06.214 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-21T14:57:06.215 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-21T14:57:06.332 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-base (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:57:06.666 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-base (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:57:06.755 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-21T14:57:07.119 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-21T14:57:07.209 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-mds (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:57:07.458 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-21T14:57:07.458 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-21T14:57:07.585 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-mds (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:57:07.827 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-21T14:57:07.827 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-21T14:57:07.894 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-mgr (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:57:08.125 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-21T14:57:08.125 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-21T14:57:08.278 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-mgr (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:57:08.512 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-21T14:57:08.512 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-21T14:57:08.570 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-osd (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:57:08.980 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-osd (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:57:09.085 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-21T14:57:09.085 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-21T14:57:09.484 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-21T14:57:09.484 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-21T14:57:09.555 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-mgr-k8sevents (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:57:09.681 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-mgr-diskprediction-local (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:57:09.857 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-mon (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:57:09.931 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-mgr-k8sevents (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:57:10.057 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-mgr-diskprediction-local (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:57:10.089 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-21T14:57:10.089 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-21T14:57:10.192 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-mon (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:57:10.405 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-21T14:57:10.405 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-21T14:57:10.550 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-mgr-cephadm (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:57:10.701 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:57:10.826 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-mgr-cephadm (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:57:10.827 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-mgr-dashboard (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:57:10.961 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-volume (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:57:10.969 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:57:11.095 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-mgr-dashboard (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:57:11.254 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-volume (18.2.4-855-g19e2a2be-1focal) ... 2024-09-21T14:57:11.314 INFO:teuthology.orchestra.run.smithi032.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2024-09-21T14:57:11.491 INFO:teuthology.orchestra.run.smithi032.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2024-09-21T14:57:11.624 INFO:teuthology.orchestra.run.smithi046.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2024-09-21T14:57:11.775 INFO:teuthology.orchestra.run.smithi046.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2024-09-21T14:57:12.055 INFO:teuthology.orchestra.run.smithi032.stdout:Processing triggers for man-db (2.9.1-1) ... 2024-09-21T14:57:12.069 INFO:teuthology.orchestra.run.smithi046.stdout:Processing triggers for man-db (2.9.1-1) ... 2024-09-21T14:57:14.946 INFO:teuthology.orchestra.run.smithi032.stdout:Processing triggers for fontconfig (2.13.1-2ubuntu3) ... 2024-09-21T14:57:14.946 INFO:teuthology.orchestra.run.smithi046.stdout:Processing triggers for fontconfig (2.13.1-2ubuntu3) ... 2024-09-21T14:57:15.537 INFO:teuthology.orchestra.run.smithi032.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-21T14:57:15.542 DEBUG:teuthology.parallel:result is None 2024-09-21T14:57:15.557 INFO:teuthology.orchestra.run.smithi046.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-21T14:57:15.560 DEBUG:teuthology.parallel:result is None 2024-09-21T14:57:15.561 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 2024-09-21T14:57:15.709 DEBUG:teuthology.orchestra.run.smithi032:> dpkg-query -W -f '${Version}' ceph 2024-09-21T14:57:15.733 INFO:teuthology.orchestra.run.smithi032.stdout:18.2.4-855-g19e2a2be-1focal 2024-09-21T14:57:15.733 INFO:teuthology.packaging:The installed version of ceph is 18.2.4-855-g19e2a2be-1focal 2024-09-21T14:57:15.734 INFO:teuthology.task.install:The correct ceph version 18.2.4-855-g19e2a2be-1focal is installed. 2024-09-21T14:57:15.736 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 2024-09-21T14:57:15.877 DEBUG:teuthology.orchestra.run.smithi046:> dpkg-query -W -f '${Version}' ceph 2024-09-21T14:57:15.901 INFO:teuthology.orchestra.run.smithi046.stdout:18.2.4-855-g19e2a2be-1focal 2024-09-21T14:57:15.901 INFO:teuthology.packaging:The installed version of ceph is 18.2.4-855-g19e2a2be-1focal 2024-09-21T14:57:15.901 INFO:teuthology.task.install:The correct ceph version 18.2.4-855-g19e2a2be-1focal is installed. 2024-09-21T14:57:15.902 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-21T14:57:15.903 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-09-21T14:57:15.903 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T14:57:15.920 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-21T14:57:15.920 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T14:57:15.964 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-21T14:57:15.964 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-09-21T14:57:15.964 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T14:57:15.980 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T14:57:16.037 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-21T14:57:16.037 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T14:57:16.053 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T14:57:16.111 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-21T14:57:16.112 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-09-21T14:57:16.112 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T14:57:16.126 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T14:57:16.182 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-21T14:57:16.182 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T14:57:16.198 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T14:57:16.207 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-21T14:57:16.207 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-09-21T14:57:16.207 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T14:57:16.223 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T14:57:16.233 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-21T14:57:16.233 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T14:57:16.248 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T14:57:16.307 INFO:teuthology.run_tasks:Running task ceph... 2024-09-21T14:57:16.374 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-21T14:57:16.374 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 777 /var/log/ceph 2024-09-21T14:57:16.377 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 777 /var/log/ceph 2024-09-21T14:57:16.391 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-21T14:57:16.391 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-21T14:57:16.393 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-21T14:57:16.404 INFO:tasks.ceph:Creating extra log directories... 2024-09-21T14:57:16.405 DEBUG:teuthology.orchestra.run.smithi032:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-21T14:57:16.407 DEBUG:teuthology.orchestra.run.smithi046:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-21T14:57:16.469 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-21T14:57:16.469 INFO:tasks.ceph:config {'conf': {'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-21T14:57:16.470 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-09-17_22:16:03-rbd-reef-distro-default-smithi/7909956', 'branch': 'reef', 'description': 'rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-zstd supported-random-distro$/{ubuntu_20.04} thrashers/default thrashosds-health workloads/rbd_fsx_deep_copy}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7909956', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'teuthology-2024-09-17_22:16:03-rbd-reef-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 10}}, {'machine': {'cpus': 1, 'disk': 40, 'ram': 8000}, 'volumes': {'count': 4, 'size': 30}}], 'os_type': 'ubuntu', 'os_version': '20.04', 'overrides': {'admin_socket': {'branch': 'reef'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from'], 'sha1': '19e2a2beb6f41a3ab62650e38614eb29b59dcbb9'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '19e2a2beb6f41a3ab62650e38614eb29b59dcbb9'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'reef', 'sha1': 'a477b1894d5c5a613f23260ca0e6b96d79652ba7'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 930, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']], 'seed': 4623, 'sha1': '19e2a2beb6f41a3ab62650e38614eb29b59dcbb9', 'sleep_before_teardown': 0, 'subset': '80/128', 'suite': 'rbd', 'suite_branch': 'reef', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_a477b1894d5c5a613f23260ca0e6b96d79652ba7/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': 'a477b1894d5c5a613f23260ca0e6b96d79652ba7', 'targets': {'smithi032.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM=', 'smithi046.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'thrashosds': {'timeout': 1200}}, {'rbd_fsx': {'clients': ['client.0'], 'deep_copy': True, 'ops': 6000}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-17_22:16:03', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2902'} 2024-09-21T14:57:16.470 DEBUG:teuthology.orchestra.run.smithi032:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-21T14:57:16.472 DEBUG:teuthology.orchestra.run.smithi046:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-21T14:57:16.479 DEBUG:teuthology.orchestra.run.smithi032:> sudo install -d -m0777 -- /var/run/ceph 2024-09-21T14:57:16.520 DEBUG:teuthology.orchestra.run.smithi046:> sudo install -d -m0777 -- /var/run/ceph 2024-09-21T14:57:16.536 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-09-21T14:57:16.536 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T14:57:16.540 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T14:57:16.540 DEBUG:teuthology.orchestra.run.smithi032:> stat /dev/vg_nvme/lv_1 2024-09-21T14:57:16.547 INFO:teuthology.orchestra.run.smithi032.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T14:57:16.548 INFO:teuthology.orchestra.run.smithi032.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T14:57:16.548 INFO:teuthology.orchestra.run.smithi032.stdout:Device: 6h/6d Inode: 358 Links: 1 2024-09-21T14:57:16.548 INFO:teuthology.orchestra.run.smithi032.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T14:57:16.548 INFO:teuthology.orchestra.run.smithi032.stdout:Access: 2024-09-21 14:53:44.647796374 +0000 2024-09-21T14:57:16.548 INFO:teuthology.orchestra.run.smithi032.stdout:Modify: 2024-09-21 14:53:44.463807054 +0000 2024-09-21T14:57:16.548 INFO:teuthology.orchestra.run.smithi032.stdout:Change: 2024-09-21 14:53:44.463807054 +0000 2024-09-21T14:57:16.548 INFO:teuthology.orchestra.run.smithi032.stdout: Birth: - 2024-09-21T14:57:16.548 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T14:57:16.605 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records in 2024-09-21T14:57:16.605 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records out 2024-09-21T14:57:16.605 INFO:teuthology.orchestra.run.smithi032.stderr:512 bytes copied, 0.000309593 s, 1.7 MB/s 2024-09-21T14:57:16.606 DEBUG:teuthology.orchestra.run.smithi032:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T14:57:16.614 DEBUG:teuthology.orchestra.run.smithi032:> stat /dev/vg_nvme/lv_2 2024-09-21T14:57:16.620 INFO:teuthology.orchestra.run.smithi032.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T14:57:16.620 INFO:teuthology.orchestra.run.smithi032.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T14:57:16.620 INFO:teuthology.orchestra.run.smithi032.stdout:Device: 6h/6d Inode: 364 Links: 1 2024-09-21T14:57:16.620 INFO:teuthology.orchestra.run.smithi032.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T14:57:16.620 INFO:teuthology.orchestra.run.smithi032.stdout:Access: 2024-09-21 14:53:45.087770919 +0000 2024-09-21T14:57:16.620 INFO:teuthology.orchestra.run.smithi032.stdout:Modify: 2024-09-21 14:53:44.875783141 +0000 2024-09-21T14:57:16.621 INFO:teuthology.orchestra.run.smithi032.stdout:Change: 2024-09-21 14:53:44.875783141 +0000 2024-09-21T14:57:16.621 INFO:teuthology.orchestra.run.smithi032.stdout: Birth: - 2024-09-21T14:57:16.621 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T14:57:16.677 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records in 2024-09-21T14:57:16.677 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records out 2024-09-21T14:57:16.677 INFO:teuthology.orchestra.run.smithi032.stderr:512 bytes copied, 0.000310852 s, 1.6 MB/s 2024-09-21T14:57:16.678 DEBUG:teuthology.orchestra.run.smithi032:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T14:57:16.686 DEBUG:teuthology.orchestra.run.smithi032:> stat /dev/vg_nvme/lv_3 2024-09-21T14:57:16.692 INFO:teuthology.orchestra.run.smithi032.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T14:57:16.692 INFO:teuthology.orchestra.run.smithi032.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T14:57:16.692 INFO:teuthology.orchestra.run.smithi032.stdout:Device: 6h/6d Inode: 373 Links: 1 2024-09-21T14:57:16.692 INFO:teuthology.orchestra.run.smithi032.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T14:57:16.692 INFO:teuthology.orchestra.run.smithi032.stdout:Access: 2024-09-21 14:53:45.491747876 +0000 2024-09-21T14:57:16.692 INFO:teuthology.orchestra.run.smithi032.stdout:Modify: 2024-09-21 14:53:45.291759283 +0000 2024-09-21T14:57:16.692 INFO:teuthology.orchestra.run.smithi032.stdout:Change: 2024-09-21 14:53:45.291759283 +0000 2024-09-21T14:57:16.692 INFO:teuthology.orchestra.run.smithi032.stdout: Birth: - 2024-09-21T14:57:16.692 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T14:57:16.749 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records in 2024-09-21T14:57:16.749 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records out 2024-09-21T14:57:16.749 INFO:teuthology.orchestra.run.smithi032.stderr:512 bytes copied, 0.000297617 s, 1.7 MB/s 2024-09-21T14:57:16.751 DEBUG:teuthology.orchestra.run.smithi032:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T14:57:16.758 DEBUG:teuthology.orchestra.run.smithi032:> stat /dev/vg_nvme/lv_4 2024-09-21T14:57:16.764 INFO:teuthology.orchestra.run.smithi032.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T14:57:16.764 INFO:teuthology.orchestra.run.smithi032.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T14:57:16.764 INFO:teuthology.orchestra.run.smithi032.stdout:Device: 6h/6d Inode: 379 Links: 1 2024-09-21T14:57:16.765 INFO:teuthology.orchestra.run.smithi032.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T14:57:16.765 INFO:teuthology.orchestra.run.smithi032.stdout:Access: 2024-09-21 14:53:45.915723691 +0000 2024-09-21T14:57:16.765 INFO:teuthology.orchestra.run.smithi032.stdout:Modify: 2024-09-21 14:53:45.699736011 +0000 2024-09-21T14:57:16.765 INFO:teuthology.orchestra.run.smithi032.stdout:Change: 2024-09-21 14:53:45.699736011 +0000 2024-09-21T14:57:16.765 INFO:teuthology.orchestra.run.smithi032.stdout: Birth: - 2024-09-21T14:57:16.765 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T14:57:16.821 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records in 2024-09-21T14:57:16.821 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records out 2024-09-21T14:57:16.821 INFO:teuthology.orchestra.run.smithi032.stderr:512 bytes copied, 0.00030352 s, 1.7 MB/s 2024-09-21T14:57:16.822 DEBUG:teuthology.orchestra.run.smithi032:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T14:57:16.830 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-21T14:57:16.830 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-21T14:57:16.830 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T14:57:16.837 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T14:57:16.837 DEBUG:teuthology.orchestra.run.smithi046:> stat /dev/vg_nvme/lv_1 2024-09-21T14:57:16.845 INFO:teuthology.orchestra.run.smithi046.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T14:57:16.845 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T14:57:16.845 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 6h/6d Inode: 359 Links: 1 2024-09-21T14:57:16.845 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T14:57:16.845 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2024-09-21 14:53:45.586871497 +0000 2024-09-21T14:57:16.845 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2024-09-21 14:53:45.394875916 +0000 2024-09-21T14:57:16.845 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2024-09-21 14:53:45.394875916 +0000 2024-09-21T14:57:16.845 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: - 2024-09-21T14:57:16.845 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T14:57:16.902 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-09-21T14:57:16.902 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-09-21T14:57:16.902 INFO:teuthology.orchestra.run.smithi046.stderr:512 bytes copied, 0.000314397 s, 1.6 MB/s 2024-09-21T14:57:16.904 DEBUG:teuthology.orchestra.run.smithi046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T14:57:16.911 DEBUG:teuthology.orchestra.run.smithi046:> stat /dev/vg_nvme/lv_2 2024-09-21T14:57:16.915 INFO:teuthology.orchestra.run.smithi046.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T14:57:16.915 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T14:57:16.915 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 6h/6d Inode: 365 Links: 1 2024-09-21T14:57:16.916 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T14:57:16.916 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2024-09-21 14:53:46.026861372 +0000 2024-09-21T14:57:16.916 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2024-09-21 14:53:45.826865975 +0000 2024-09-21T14:57:16.916 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2024-09-21 14:53:45.826865975 +0000 2024-09-21T14:57:16.916 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: - 2024-09-21T14:57:16.916 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T14:57:16.927 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-09-21T14:57:16.927 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-09-21T14:57:16.927 INFO:teuthology.orchestra.run.smithi046.stderr:512 bytes copied, 0.000294117 s, 1.7 MB/s 2024-09-21T14:57:16.928 DEBUG:teuthology.orchestra.run.smithi046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T14:57:16.980 DEBUG:teuthology.orchestra.run.smithi046:> stat /dev/vg_nvme/lv_3 2024-09-21T14:57:16.987 INFO:teuthology.orchestra.run.smithi046.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T14:57:16.987 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T14:57:16.987 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 6h/6d Inode: 372 Links: 1 2024-09-21T14:57:16.987 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T14:57:16.987 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2024-09-21 14:53:46.478850971 +0000 2024-09-21T14:57:16.987 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2024-09-21 14:53:46.258856034 +0000 2024-09-21T14:57:16.988 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2024-09-21 14:53:46.258856034 +0000 2024-09-21T14:57:16.988 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: - 2024-09-21T14:57:16.988 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T14:57:17.003 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-09-21T14:57:17.003 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-09-21T14:57:17.003 INFO:teuthology.orchestra.run.smithi046.stderr:512 bytes copied, 0.000275074 s, 1.9 MB/s 2024-09-21T14:57:17.004 DEBUG:teuthology.orchestra.run.smithi046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T14:57:17.056 DEBUG:teuthology.orchestra.run.smithi046:> stat /dev/vg_nvme/lv_4 2024-09-21T14:57:17.063 INFO:teuthology.orchestra.run.smithi046.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T14:57:17.063 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T14:57:17.063 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 6h/6d Inode: 377 Links: 1 2024-09-21T14:57:17.063 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T14:57:17.063 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2024-09-21 14:53:46.894841397 +0000 2024-09-21T14:57:17.063 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2024-09-21 14:53:46.698845909 +0000 2024-09-21T14:57:17.063 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2024-09-21 14:53:46.698845909 +0000 2024-09-21T14:57:17.063 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: - 2024-09-21T14:57:17.063 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T14:57:17.076 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-09-21T14:57:17.076 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-09-21T14:57:17.076 INFO:teuthology.orchestra.run.smithi046.stderr:512 bytes copied, 0.000288887 s, 1.8 MB/s 2024-09-21T14:57:17.077 DEBUG:teuthology.orchestra.run.smithi046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T14:57:17.128 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-21T14:57:17.128 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi032.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi046.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-21T14:57:17.128 INFO:tasks.ceph:Generating config... 2024-09-21T14:57:17.129 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-09-21T14:57:17.129 INFO:tasks.ceph:[global] mon warn on pool no app = False 2024-09-21T14:57:17.130 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-09-21T14:57:17.130 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-21T14:57:17.130 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-21T14:57:17.130 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-21T14:57:17.130 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-21T14:57:17.130 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-21T14:57:17.130 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-21T14:57:17.130 INFO:tasks.ceph:[osd] bluestore compression algorithm = zstd 2024-09-21T14:57:17.130 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2024-09-21T14:57:17.130 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-21T14:57:17.130 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-21T14:57:17.130 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-21T14:57:17.130 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-21T14:57:17.131 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-21T14:57:17.131 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-21T14:57:17.131 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-21T14:57:17.131 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-21T14:57:17.131 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-21T14:57:17.131 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-09-21T14:57:17.131 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-21T14:57:17.131 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-09-21T14:57:17.131 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-21T14:57:17.131 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-09-21T14:57:17.131 INFO:tasks.ceph:Setting up mon.a... 2024-09-21T14:57:17.131 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-21T14:57:17.173 INFO:teuthology.orchestra.run.smithi032.stdout:creating /etc/ceph/ceph.keyring 2024-09-21T14:57:17.191 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-21T14:57:17.280 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-21T14:57:17.291 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.32'), ('mon.c', '[v2:172.21.15.32:3301,v1:172.21.15.32:6790]'), ('mon.b', '172.21.15.46')] 2024-09-21T14:57:17.291 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.32,[v2:172.21.15.32:3301,v1:172.21.15.32:6790],172.21.15.46', 'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2024-09-21T14:57:17.292 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-09-21T14:57:17.292 DEBUG:teuthology.orchestra.run.smithi032:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-21T14:57:17.298 DEBUG:teuthology.orchestra.run.smithi032:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.32 --addv c '[v2:172.21.15.32:3301,v1:172.21.15.32:6790]' --add b 172.21.15.46 --print /home/ubuntu/cephtest/ceph.monmap 2024-09-21T14:57:17.382 INFO:teuthology.orchestra.run.smithi032.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-21T14:57:17.383 INFO:teuthology.orchestra.run.smithi032.stdout:monmaptool: generated fsid caf51e55-9234-4a26-bcd2-d60811962560 2024-09-21T14:57:17.383 INFO:teuthology.orchestra.run.smithi032.stdout:setting min_mon_release = pacific 2024-09-21T14:57:17.383 INFO:teuthology.orchestra.run.smithi032.stdout:epoch 0 2024-09-21T14:57:17.383 INFO:teuthology.orchestra.run.smithi032.stdout:fsid caf51e55-9234-4a26-bcd2-d60811962560 2024-09-21T14:57:17.383 INFO:teuthology.orchestra.run.smithi032.stdout:last_changed 2024-09-21T14:57:17.381978+0000 2024-09-21T14:57:17.383 INFO:teuthology.orchestra.run.smithi032.stdout:created 2024-09-21T14:57:17.381978+0000 2024-09-21T14:57:17.383 INFO:teuthology.orchestra.run.smithi032.stdout:min_mon_release 16 (pacific) 2024-09-21T14:57:17.383 INFO:teuthology.orchestra.run.smithi032.stdout:election_strategy: 1 2024-09-21T14:57:17.383 INFO:teuthology.orchestra.run.smithi032.stdout:0: [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] mon.a 2024-09-21T14:57:17.383 INFO:teuthology.orchestra.run.smithi032.stdout:1: [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] mon.b 2024-09-21T14:57:17.383 INFO:teuthology.orchestra.run.smithi032.stdout:2: [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] mon.c 2024-09-21T14:57:17.383 INFO:teuthology.orchestra.run.smithi032.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-21T14:57:17.384 DEBUG:teuthology.orchestra.run.smithi032:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-21T14:57:17.391 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID caf51e55-9234-4a26-bcd2-d60811962560... 2024-09-21T14:57:17.392 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-21T14:57:17.394 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-21T14:57:17.427 INFO:teuthology.orchestra.run.smithi032.stdout:[global] 2024-09-21T14:57:17.427 INFO:teuthology.orchestra.run.smithi032.stdout: chdir = "" 2024-09-21T14:57:17.427 INFO:teuthology.orchestra.run.smithi032.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-21T14:57:17.428 INFO:teuthology.orchestra.run.smithi032.stdout: auth supported = cephx 2024-09-21T14:57:17.428 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-21T14:57:17.428 INFO:teuthology.orchestra.run.smithi032.stdout: filestore xattr use omap = true 2024-09-21T14:57:17.428 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-21T14:57:17.428 INFO:teuthology.orchestra.run.smithi032.stdout: mon clock drift allowed = 1.000 2024-09-21T14:57:17.428 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-21T14:57:17.428 INFO:teuthology.orchestra.run.smithi032.stdout: osd crush chooseleaf type = 0 2024-09-21T14:57:17.428 INFO:teuthology.orchestra.run.smithi032.stdout: auth debug = true 2024-09-21T14:57:17.428 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-21T14:57:17.428 INFO:teuthology.orchestra.run.smithi032.stdout: ms die on old message = true 2024-09-21T14:57:17.428 INFO:teuthology.orchestra.run.smithi032.stdout: ms die on bug = true 2024-09-21T14:57:17.428 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-21T14:57:17.428 INFO:teuthology.orchestra.run.smithi032.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-21T14:57:17.429 INFO:teuthology.orchestra.run.smithi032.stdout: mon pg warn max object skew = 0 2024-09-21T14:57:17.429 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-21T14:57:17.429 INFO:teuthology.orchestra.run.smithi032.stdout: # disable pg_autoscaler by default for new pools 2024-09-21T14:57:17.429 INFO:teuthology.orchestra.run.smithi032.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-21T14:57:17.429 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-21T14:57:17.429 INFO:teuthology.orchestra.run.smithi032.stdout: osd pool default size = 2 2024-09-21T14:57:17.429 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-21T14:57:17.429 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd allow primary affinity = true 2024-09-21T14:57:17.429 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd allow pg remap = true 2024-09-21T14:57:17.429 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on legacy crush tunables = false 2024-09-21T14:57:17.429 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on crush straw calc version zero = false 2024-09-21T14:57:17.429 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on no sortbitwise = false 2024-09-21T14:57:17.429 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on osd down out interval zero = false 2024-09-21T14:57:17.429 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on too few osds = false 2024-09-21T14:57:17.430 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-21T14:57:17.430 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-21T14:57:17.430 INFO:teuthology.orchestra.run.smithi032.stdout: mon_allow_pool_size_one = true 2024-09-21T14:57:17.430 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-21T14:57:17.430 INFO:teuthology.orchestra.run.smithi032.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-21T14:57:17.430 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-21T14:57:17.430 INFO:teuthology.orchestra.run.smithi032.stdout: osd default data pool replay window = 5 2024-09-21T14:57:17.430 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-21T14:57:17.430 INFO:teuthology.orchestra.run.smithi032.stdout: mon allow pool delete = true 2024-09-21T14:57:17.430 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-21T14:57:17.430 INFO:teuthology.orchestra.run.smithi032.stdout: mon cluster log file level = debug 2024-09-21T14:57:17.430 INFO:teuthology.orchestra.run.smithi032.stdout: debug asserts on shutdown = true 2024-09-21T14:57:17.430 INFO:teuthology.orchestra.run.smithi032.stdout: mon health detail to clog = false 2024-09-21T14:57:17.430 INFO:teuthology.orchestra.run.smithi032.stdout: mon host = "172.21.15.32,[v2:172.21.15.32:3301,v1:172.21.15.32:6790],172.21.15.46" 2024-09-21T14:57:17.431 INFO:teuthology.orchestra.run.smithi032.stdout: mon client directed command retry = 5 2024-09-21T14:57:17.431 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on pool no app = False 2024-09-21T14:57:17.431 INFO:teuthology.orchestra.run.smithi032.stdout: ms inject socket failures = 5000 2024-09-21T14:57:17.431 INFO:teuthology.orchestra.run.smithi032.stdout: fsid = caf51e55-9234-4a26-bcd2-d60811962560 2024-09-21T14:57:17.431 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-21T14:57:17.431 INFO:teuthology.orchestra.run.smithi032.stdout:[osd] 2024-09-21T14:57:17.431 INFO:teuthology.orchestra.run.smithi032.stdout: osd journal size = 100 2024-09-21T14:57:17.431 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-21T14:57:17.431 INFO:teuthology.orchestra.run.smithi032.stdout: osd scrub load threshold = 5.0 2024-09-21T14:57:17.431 INFO:teuthology.orchestra.run.smithi032.stdout: osd scrub max interval = 600 2024-09-21T14:57:17.431 INFO:teuthology.orchestra.run.smithi032.stdout: osd mclock profile = high_recovery_ops 2024-09-21T14:57:17.432 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-21T14:57:17.432 INFO:teuthology.orchestra.run.smithi032.stdout: osd recover clone overlap = true 2024-09-21T14:57:17.432 INFO:teuthology.orchestra.run.smithi032.stdout: osd recovery max chunk = 1048576 2024-09-21T14:57:17.432 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-21T14:57:17.432 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug shutdown = true 2024-09-21T14:57:17.432 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug op order = true 2024-09-21T14:57:17.432 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug verify stray on activate = true 2024-09-21T14:57:17.432 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-21T14:57:17.432 INFO:teuthology.orchestra.run.smithi032.stdout: osd open classes on start = true 2024-09-21T14:57:17.432 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug pg log writeout = true 2024-09-21T14:57:17.432 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-21T14:57:17.432 INFO:teuthology.orchestra.run.smithi032.stdout: osd deep scrub update digest min age = 30 2024-09-21T14:57:17.432 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-21T14:57:17.432 INFO:teuthology.orchestra.run.smithi032.stdout: osd map max advance = 10 2024-09-21T14:57:17.433 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-21T14:57:17.433 INFO:teuthology.orchestra.run.smithi032.stdout: journal zero on create = true 2024-09-21T14:57:17.433 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-21T14:57:17.433 INFO:teuthology.orchestra.run.smithi032.stdout: filestore ondisk finisher threads = 3 2024-09-21T14:57:17.433 INFO:teuthology.orchestra.run.smithi032.stdout: filestore apply finisher threads = 3 2024-09-21T14:57:17.433 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-21T14:57:17.433 INFO:teuthology.orchestra.run.smithi032.stdout: bdev debug aio = true 2024-09-21T14:57:17.433 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug misdirected ops = true 2024-09-21T14:57:17.433 INFO:teuthology.orchestra.run.smithi032.stdout: bluestore block size = 96636764160 2024-09-21T14:57:17.433 INFO:teuthology.orchestra.run.smithi032.stdout: bluestore compression algorithm = zstd 2024-09-21T14:57:17.433 INFO:teuthology.orchestra.run.smithi032.stdout: bluestore compression mode = aggressive 2024-09-21T14:57:17.433 INFO:teuthology.orchestra.run.smithi032.stdout: bluestore fsck on mount = True 2024-09-21T14:57:17.433 INFO:teuthology.orchestra.run.smithi032.stdout: debug bluefs = 1/20 2024-09-21T14:57:17.434 INFO:teuthology.orchestra.run.smithi032.stdout: debug bluestore = 1/20 2024-09-21T14:57:17.434 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2024-09-21T14:57:17.434 INFO:teuthology.orchestra.run.smithi032.stdout: debug osd = 20 2024-09-21T14:57:17.434 INFO:teuthology.orchestra.run.smithi032.stdout: debug rocksdb = 4/10 2024-09-21T14:57:17.434 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd backfillfull_ratio = 0.85 2024-09-21T14:57:17.434 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd full ratio = 0.9 2024-09-21T14:57:17.434 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd nearfull ratio = 0.8 2024-09-21T14:57:17.434 INFO:teuthology.orchestra.run.smithi032.stdout: osd blocked scrub grace period = 3600 2024-09-21T14:57:17.434 INFO:teuthology.orchestra.run.smithi032.stdout: osd failsafe full ratio = 0.95 2024-09-21T14:57:17.434 INFO:teuthology.orchestra.run.smithi032.stdout: osd max markdown count = 1000 2024-09-21T14:57:17.434 INFO:teuthology.orchestra.run.smithi032.stdout: osd objectstore = bluestore 2024-09-21T14:57:17.434 INFO:teuthology.orchestra.run.smithi032.stdout: osd shutdown pgref assert = True 2024-09-21T14:57:17.434 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-21T14:57:17.434 INFO:teuthology.orchestra.run.smithi032.stdout:[mgr] 2024-09-21T14:57:17.435 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2024-09-21T14:57:17.435 INFO:teuthology.orchestra.run.smithi032.stdout: debug mgr = 20 2024-09-21T14:57:17.435 INFO:teuthology.orchestra.run.smithi032.stdout: debug mon = 20 2024-09-21T14:57:17.435 INFO:teuthology.orchestra.run.smithi032.stdout: debug auth = 20 2024-09-21T14:57:17.435 INFO:teuthology.orchestra.run.smithi032.stdout: mon reweight min pgs per osd = 4 2024-09-21T14:57:17.435 INFO:teuthology.orchestra.run.smithi032.stdout: mon reweight min bytes per osd = 10 2024-09-21T14:57:17.435 INFO:teuthology.orchestra.run.smithi032.stdout: mgr/telemetry/nag = false 2024-09-21T14:57:17.435 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-21T14:57:17.435 INFO:teuthology.orchestra.run.smithi032.stdout:[mon] 2024-09-21T14:57:17.435 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2024-09-21T14:57:17.435 INFO:teuthology.orchestra.run.smithi032.stdout: debug mon = 20 2024-09-21T14:57:17.435 INFO:teuthology.orchestra.run.smithi032.stdout: debug paxos = 20 2024-09-21T14:57:17.435 INFO:teuthology.orchestra.run.smithi032.stdout: debug auth = 20 2024-09-21T14:57:17.436 INFO:teuthology.orchestra.run.smithi032.stdout: mon data avail warn = 5 2024-09-21T14:57:17.436 INFO:teuthology.orchestra.run.smithi032.stdout: mon mgr mkfs grace = 240 2024-09-21T14:57:17.436 INFO:teuthology.orchestra.run.smithi032.stdout: mon reweight min pgs per osd = 4 2024-09-21T14:57:17.436 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd reporter subtree level = osd 2024-09-21T14:57:17.436 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd prime pg temp = true 2024-09-21T14:57:17.436 INFO:teuthology.orchestra.run.smithi032.stdout: mon reweight min bytes per osd = 10 2024-09-21T14:57:17.436 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-21T14:57:17.436 INFO:teuthology.orchestra.run.smithi032.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-21T14:57:17.436 INFO:teuthology.orchestra.run.smithi032.stdout: auth mon ticket ttl = 660 # 11m 2024-09-21T14:57:17.436 INFO:teuthology.orchestra.run.smithi032.stdout: auth service ticket ttl = 240 # 4m 2024-09-21T14:57:17.436 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-21T14:57:17.436 INFO:teuthology.orchestra.run.smithi032.stdout: # don't complain about insecure global_id in the test suite 2024-09-21T14:57:17.436 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-21T14:57:17.436 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-21T14:57:17.437 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-21T14:57:17.437 INFO:teuthology.orchestra.run.smithi032.stdout: # 1m isn't quite enough 2024-09-21T14:57:17.437 INFO:teuthology.orchestra.run.smithi032.stdout: mon_down_mkfs_grace = 2m 2024-09-21T14:57:17.437 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-21T14:57:17.437 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_filestore_osds = false 2024-09-21T14:57:17.437 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-21T14:57:17.437 INFO:teuthology.orchestra.run.smithi032.stdout:[client] 2024-09-21T14:57:17.437 INFO:teuthology.orchestra.run.smithi032.stdout: rgw cache enabled = true 2024-09-21T14:57:17.437 INFO:teuthology.orchestra.run.smithi032.stdout: rgw enable ops log = true 2024-09-21T14:57:17.437 INFO:teuthology.orchestra.run.smithi032.stdout: rgw enable usage log = true 2024-09-21T14:57:17.437 INFO:teuthology.orchestra.run.smithi032.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-21T14:57:17.437 INFO:teuthology.orchestra.run.smithi032.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-21T14:57:17.437 INFO:teuthology.orchestra.run.smithi032.stdout:[mon.a] 2024-09-21T14:57:17.437 INFO:teuthology.orchestra.run.smithi032.stdout:[mon.c] 2024-09-21T14:57:17.438 INFO:teuthology.orchestra.run.smithi032.stdout:[mon.b] 2024-09-21T14:57:17.438 INFO:teuthology.orchestra.run.smithi046.stdout:[global] 2024-09-21T14:57:17.438 INFO:teuthology.orchestra.run.smithi046.stdout: chdir = "" 2024-09-21T14:57:17.438 INFO:teuthology.orchestra.run.smithi046.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-21T14:57:17.438 INFO:teuthology.orchestra.run.smithi046.stdout: auth supported = cephx 2024-09-21T14:57:17.438 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-21T14:57:17.439 INFO:teuthology.orchestra.run.smithi046.stdout: filestore xattr use omap = true 2024-09-21T14:57:17.439 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-21T14:57:17.439 INFO:teuthology.orchestra.run.smithi046.stdout: mon clock drift allowed = 1.000 2024-09-21T14:57:17.439 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-21T14:57:17.439 INFO:teuthology.orchestra.run.smithi046.stdout: osd crush chooseleaf type = 0 2024-09-21T14:57:17.439 INFO:teuthology.orchestra.run.smithi046.stdout: auth debug = true 2024-09-21T14:57:17.439 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-21T14:57:17.439 INFO:teuthology.orchestra.run.smithi046.stdout: ms die on old message = true 2024-09-21T14:57:17.439 INFO:teuthology.orchestra.run.smithi046.stdout: ms die on bug = true 2024-09-21T14:57:17.439 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-21T14:57:17.439 INFO:teuthology.orchestra.run.smithi046.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-21T14:57:17.439 INFO:teuthology.orchestra.run.smithi046.stdout: mon pg warn max object skew = 0 2024-09-21T14:57:17.439 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-21T14:57:17.439 INFO:teuthology.orchestra.run.smithi046.stdout: # disable pg_autoscaler by default for new pools 2024-09-21T14:57:17.440 INFO:teuthology.orchestra.run.smithi046.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-21T14:57:17.440 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-21T14:57:17.440 INFO:teuthology.orchestra.run.smithi046.stdout: osd pool default size = 2 2024-09-21T14:57:17.440 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-21T14:57:17.440 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd allow primary affinity = true 2024-09-21T14:57:17.440 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd allow pg remap = true 2024-09-21T14:57:17.440 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on legacy crush tunables = false 2024-09-21T14:57:17.440 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on crush straw calc version zero = false 2024-09-21T14:57:17.440 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on no sortbitwise = false 2024-09-21T14:57:17.440 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on osd down out interval zero = false 2024-09-21T14:57:17.440 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on too few osds = false 2024-09-21T14:57:17.440 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-21T14:57:17.440 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-21T14:57:17.441 INFO:teuthology.orchestra.run.smithi046.stdout: mon_allow_pool_size_one = true 2024-09-21T14:57:17.441 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-21T14:57:17.441 INFO:teuthology.orchestra.run.smithi046.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-21T14:57:17.441 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-21T14:57:17.441 INFO:teuthology.orchestra.run.smithi046.stdout: osd default data pool replay window = 5 2024-09-21T14:57:17.441 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-21T14:57:17.441 INFO:teuthology.orchestra.run.smithi046.stdout: mon allow pool delete = true 2024-09-21T14:57:17.441 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-21T14:57:17.441 INFO:teuthology.orchestra.run.smithi046.stdout: mon cluster log file level = debug 2024-09-21T14:57:17.441 INFO:teuthology.orchestra.run.smithi046.stdout: debug asserts on shutdown = true 2024-09-21T14:57:17.441 INFO:teuthology.orchestra.run.smithi046.stdout: mon health detail to clog = false 2024-09-21T14:57:17.441 INFO:teuthology.orchestra.run.smithi046.stdout: mon host = "172.21.15.32,[v2:172.21.15.32:3301,v1:172.21.15.32:6790],172.21.15.46" 2024-09-21T14:57:17.441 INFO:teuthology.orchestra.run.smithi046.stdout: mon client directed command retry = 5 2024-09-21T14:57:17.441 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on pool no app = False 2024-09-21T14:57:17.441 INFO:teuthology.orchestra.run.smithi046.stdout: ms inject socket failures = 5000 2024-09-21T14:57:17.442 INFO:teuthology.orchestra.run.smithi046.stdout: fsid = caf51e55-9234-4a26-bcd2-d60811962560 2024-09-21T14:57:17.442 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-21T14:57:17.442 INFO:teuthology.orchestra.run.smithi046.stdout:[osd] 2024-09-21T14:57:17.442 INFO:teuthology.orchestra.run.smithi046.stdout: osd journal size = 100 2024-09-21T14:57:17.442 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-21T14:57:17.442 INFO:teuthology.orchestra.run.smithi046.stdout: osd scrub load threshold = 5.0 2024-09-21T14:57:17.442 INFO:teuthology.orchestra.run.smithi046.stdout: osd scrub max interval = 600 2024-09-21T14:57:17.442 INFO:teuthology.orchestra.run.smithi046.stdout: osd mclock profile = high_recovery_ops 2024-09-21T14:57:17.442 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-21T14:57:17.442 INFO:teuthology.orchestra.run.smithi046.stdout: osd recover clone overlap = true 2024-09-21T14:57:17.442 INFO:teuthology.orchestra.run.smithi046.stdout: osd recovery max chunk = 1048576 2024-09-21T14:57:17.442 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-21T14:57:17.442 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug shutdown = true 2024-09-21T14:57:17.442 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug op order = true 2024-09-21T14:57:17.442 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug verify stray on activate = true 2024-09-21T14:57:17.443 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-21T14:57:17.443 INFO:teuthology.orchestra.run.smithi046.stdout: osd open classes on start = true 2024-09-21T14:57:17.443 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug pg log writeout = true 2024-09-21T14:57:17.443 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-21T14:57:17.443 INFO:teuthology.orchestra.run.smithi046.stdout: osd deep scrub update digest min age = 30 2024-09-21T14:57:17.443 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-21T14:57:17.443 INFO:teuthology.orchestra.run.smithi046.stdout: osd map max advance = 10 2024-09-21T14:57:17.443 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-21T14:57:17.443 INFO:teuthology.orchestra.run.smithi046.stdout: journal zero on create = true 2024-09-21T14:57:17.443 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-21T14:57:17.443 INFO:teuthology.orchestra.run.smithi046.stdout: filestore ondisk finisher threads = 3 2024-09-21T14:57:17.443 INFO:teuthology.orchestra.run.smithi046.stdout: filestore apply finisher threads = 3 2024-09-21T14:57:17.443 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-21T14:57:17.443 INFO:teuthology.orchestra.run.smithi046.stdout: bdev debug aio = true 2024-09-21T14:57:17.443 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug misdirected ops = true 2024-09-21T14:57:17.444 INFO:teuthology.orchestra.run.smithi046.stdout: bluestore block size = 96636764160 2024-09-21T14:57:17.444 INFO:teuthology.orchestra.run.smithi046.stdout: bluestore compression algorithm = zstd 2024-09-21T14:57:17.444 INFO:teuthology.orchestra.run.smithi046.stdout: bluestore compression mode = aggressive 2024-09-21T14:57:17.444 INFO:teuthology.orchestra.run.smithi046.stdout: bluestore fsck on mount = True 2024-09-21T14:57:17.444 INFO:teuthology.orchestra.run.smithi046.stdout: debug bluefs = 1/20 2024-09-21T14:57:17.444 INFO:teuthology.orchestra.run.smithi046.stdout: debug bluestore = 1/20 2024-09-21T14:57:17.444 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-09-21T14:57:17.444 INFO:teuthology.orchestra.run.smithi046.stdout: debug osd = 20 2024-09-21T14:57:17.444 INFO:teuthology.orchestra.run.smithi046.stdout: debug rocksdb = 4/10 2024-09-21T14:57:17.444 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd backfillfull_ratio = 0.85 2024-09-21T14:57:17.444 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd full ratio = 0.9 2024-09-21T14:57:17.444 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd nearfull ratio = 0.8 2024-09-21T14:57:17.444 INFO:teuthology.orchestra.run.smithi046.stdout: osd blocked scrub grace period = 3600 2024-09-21T14:57:17.444 INFO:teuthology.orchestra.run.smithi046.stdout: osd failsafe full ratio = 0.95 2024-09-21T14:57:17.444 INFO:teuthology.orchestra.run.smithi046.stdout: osd max markdown count = 1000 2024-09-21T14:57:17.445 INFO:teuthology.orchestra.run.smithi046.stdout: osd objectstore = bluestore 2024-09-21T14:57:17.445 INFO:teuthology.orchestra.run.smithi046.stdout: osd shutdown pgref assert = True 2024-09-21T14:57:17.445 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-21T14:57:17.445 INFO:teuthology.orchestra.run.smithi046.stdout:[mgr] 2024-09-21T14:57:17.445 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-09-21T14:57:17.445 INFO:teuthology.orchestra.run.smithi046.stdout: debug mgr = 20 2024-09-21T14:57:17.445 INFO:teuthology.orchestra.run.smithi046.stdout: debug mon = 20 2024-09-21T14:57:17.445 INFO:teuthology.orchestra.run.smithi046.stdout: debug auth = 20 2024-09-21T14:57:17.445 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min pgs per osd = 4 2024-09-21T14:57:17.445 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min bytes per osd = 10 2024-09-21T14:57:17.445 INFO:teuthology.orchestra.run.smithi046.stdout: mgr/telemetry/nag = false 2024-09-21T14:57:17.445 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-21T14:57:17.445 INFO:teuthology.orchestra.run.smithi046.stdout:[mon] 2024-09-21T14:57:17.445 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-09-21T14:57:17.445 INFO:teuthology.orchestra.run.smithi046.stdout: debug mon = 20 2024-09-21T14:57:17.446 INFO:teuthology.orchestra.run.smithi046.stdout: debug paxos = 20 2024-09-21T14:57:17.446 INFO:teuthology.orchestra.run.smithi046.stdout: debug auth = 20 2024-09-21T14:57:17.446 INFO:teuthology.orchestra.run.smithi046.stdout: mon data avail warn = 5 2024-09-21T14:57:17.446 INFO:teuthology.orchestra.run.smithi046.stdout: mon mgr mkfs grace = 240 2024-09-21T14:57:17.446 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min pgs per osd = 4 2024-09-21T14:57:17.446 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd reporter subtree level = osd 2024-09-21T14:57:17.446 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd prime pg temp = true 2024-09-21T14:57:17.446 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min bytes per osd = 10 2024-09-21T14:57:17.446 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-21T14:57:17.446 INFO:teuthology.orchestra.run.smithi046.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-21T14:57:17.446 INFO:teuthology.orchestra.run.smithi046.stdout: auth mon ticket ttl = 660 # 11m 2024-09-21T14:57:17.446 INFO:teuthology.orchestra.run.smithi046.stdout: auth service ticket ttl = 240 # 4m 2024-09-21T14:57:17.446 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-21T14:57:17.447 INFO:teuthology.orchestra.run.smithi046.stdout: # don't complain about insecure global_id in the test suite 2024-09-21T14:57:17.447 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-21T14:57:17.447 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-21T14:57:17.447 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-21T14:57:17.447 INFO:teuthology.orchestra.run.smithi046.stdout: # 1m isn't quite enough 2024-09-21T14:57:17.447 INFO:teuthology.orchestra.run.smithi046.stdout: mon_down_mkfs_grace = 2m 2024-09-21T14:57:17.447 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-21T14:57:17.447 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_filestore_osds = false 2024-09-21T14:57:17.447 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-21T14:57:17.447 INFO:teuthology.orchestra.run.smithi046.stdout:[client] 2024-09-21T14:57:17.447 INFO:teuthology.orchestra.run.smithi046.stdout: rgw cache enabled = true 2024-09-21T14:57:17.447 INFO:teuthology.orchestra.run.smithi046.stdout: rgw enable ops log = true 2024-09-21T14:57:17.447 INFO:teuthology.orchestra.run.smithi046.stdout: rgw enable usage log = true 2024-09-21T14:57:17.447 INFO:teuthology.orchestra.run.smithi046.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-21T14:57:17.447 INFO:teuthology.orchestra.run.smithi046.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-21T14:57:17.448 INFO:teuthology.orchestra.run.smithi046.stdout:[mon.a] 2024-09-21T14:57:17.448 INFO:teuthology.orchestra.run.smithi046.stdout:[mon.c] 2024-09-21T14:57:17.448 INFO:teuthology.orchestra.run.smithi046.stdout:[mon.b] 2024-09-21T14:57:17.449 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-21T14:57:17.449 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-21T14:57:17.503 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-21T14:57:17.504 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-09-21T14:57:17.504 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-21T14:57:17.511 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-09-21T14:57:17.511 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-21T14:57:17.517 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi032.front.sepia.ceph.com 2024-09-21T14:57:17.517 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-09-21T14:57:17.517 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-21T14:57:17.517 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-21T14:57:17.540 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-09-21T14:57:17.540 DEBUG:teuthology.orchestra.run.smithi032:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-21T14:57:17.547 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi046.front.sepia.ceph.com 2024-09-21T14:57:17.547 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-21T14:57:17.547 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-21T14:57:17.547 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-21T14:57:17.573 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-21T14:57:17.573 DEBUG:teuthology.orchestra.run.smithi046:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-21T14:57:17.578 INFO:tasks.ceph:Setting up mon nodes... 2024-09-21T14:57:17.578 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-21T14:57:17.578 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-21T14:57:17.628 INFO:teuthology.orchestra.run.smithi032.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-21T14:57:17.631 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-21T14:57:17.681 INFO:teuthology.orchestra.run.smithi046.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-21T14:57:17.692 INFO:tasks.ceph:Setting up mds nodes... 2024-09-21T14:57:17.693 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-21T14:57:17.693 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-21T14:57:17.734 INFO:teuthology.orchestra.run.smithi032.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-21T14:57:17.749 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-21T14:57:17.791 INFO:teuthology.orchestra.run.smithi046.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-21T14:57:17.803 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-21T14:57:17.804 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi032.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi046.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-21T14:57:17.804 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-21T14:57:17.819 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-21T14:57:17.819 INFO:tasks.ceph:role: osd.0 2024-09-21T14:57:17.819 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi032.front.sepia.ceph.com 2024-09-21T14:57:17.819 DEBUG:teuthology.orchestra.run.smithi032:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-21T14:57:17.910 INFO:teuthology.orchestra.run.smithi032.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T14:57:17.910 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T14:57:17.910 INFO:teuthology.orchestra.run.smithi032.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T14:57:17.911 INFO:teuthology.orchestra.run.smithi032.stdout: = reflink=1 2024-09-21T14:57:17.911 INFO:teuthology.orchestra.run.smithi032.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T14:57:17.911 INFO:teuthology.orchestra.run.smithi032.stdout: = sunit=0 swidth=0 blks 2024-09-21T14:57:17.911 INFO:teuthology.orchestra.run.smithi032.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T14:57:17.911 INFO:teuthology.orchestra.run.smithi032.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-21T14:57:17.911 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T14:57:17.911 INFO:teuthology.orchestra.run.smithi032.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T14:57:17.911 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi032.front.sepia.ceph.com -o noatime 2024-09-21T14:57:17.911 DEBUG:teuthology.orchestra.run.smithi032:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-21T14:57:17.931 DEBUG:teuthology.orchestra.run.smithi032:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-21T14:57:17.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T14:57:17.941 INFO:teuthology.orchestra.run.smithi032.stderr:sudo: /sbin/restorecon: command not found 2024-09-21T14:57:17.941 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-21T14:57:17.949 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-21T14:57:17.949 INFO:tasks.ceph:role: osd.1 2024-09-21T14:57:17.950 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi032.front.sepia.ceph.com 2024-09-21T14:57:17.950 DEBUG:teuthology.orchestra.run.smithi032:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-21T14:57:17.989 INFO:teuthology.orchestra.run.smithi032.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T14:57:17.989 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T14:57:17.989 INFO:teuthology.orchestra.run.smithi032.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T14:57:17.990 INFO:teuthology.orchestra.run.smithi032.stdout: = reflink=1 2024-09-21T14:57:17.990 INFO:teuthology.orchestra.run.smithi032.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T14:57:17.990 INFO:teuthology.orchestra.run.smithi032.stdout: = sunit=0 swidth=0 blks 2024-09-21T14:57:17.990 INFO:teuthology.orchestra.run.smithi032.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T14:57:17.990 INFO:teuthology.orchestra.run.smithi032.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-21T14:57:17.990 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T14:57:17.990 INFO:teuthology.orchestra.run.smithi032.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T14:57:17.991 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi032.front.sepia.ceph.com -o noatime 2024-09-21T14:57:17.991 DEBUG:teuthology.orchestra.run.smithi032:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-21T14:57:18.007 DEBUG:teuthology.orchestra.run.smithi032:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-21T14:57:18.017 INFO:teuthology.orchestra.run.smithi032.stderr:sudo: /sbin/restorecon: command not found 2024-09-21T14:57:18.017 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T14:57:18.018 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-21T14:57:18.073 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-21T14:57:18.073 INFO:tasks.ceph:role: osd.2 2024-09-21T14:57:18.073 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi032.front.sepia.ceph.com 2024-09-21T14:57:18.073 DEBUG:teuthology.orchestra.run.smithi032:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-21T14:57:18.115 INFO:teuthology.orchestra.run.smithi032.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T14:57:18.115 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T14:57:18.115 INFO:teuthology.orchestra.run.smithi032.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T14:57:18.115 INFO:teuthology.orchestra.run.smithi032.stdout: = reflink=1 2024-09-21T14:57:18.115 INFO:teuthology.orchestra.run.smithi032.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T14:57:18.115 INFO:teuthology.orchestra.run.smithi032.stdout: = sunit=0 swidth=0 blks 2024-09-21T14:57:18.115 INFO:teuthology.orchestra.run.smithi032.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T14:57:18.115 INFO:teuthology.orchestra.run.smithi032.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-21T14:57:18.115 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T14:57:18.116 INFO:teuthology.orchestra.run.smithi032.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T14:57:18.116 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi032.front.sepia.ceph.com -o noatime 2024-09-21T14:57:18.116 DEBUG:teuthology.orchestra.run.smithi032:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-21T14:57:18.137 DEBUG:teuthology.orchestra.run.smithi032:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-21T14:57:18.191 INFO:teuthology.orchestra.run.smithi032.stderr:sudo: /sbin/restorecon: command not found 2024-09-21T14:57:18.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T14:57:18.192 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-21T14:57:18.205 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-21T14:57:18.205 INFO:tasks.ceph:role: osd.3 2024-09-21T14:57:18.205 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi032.front.sepia.ceph.com 2024-09-21T14:57:18.205 DEBUG:teuthology.orchestra.run.smithi032:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-21T14:57:18.247 INFO:teuthology.orchestra.run.smithi032.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T14:57:18.248 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T14:57:18.248 INFO:teuthology.orchestra.run.smithi032.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T14:57:18.248 INFO:teuthology.orchestra.run.smithi032.stdout: = reflink=1 2024-09-21T14:57:18.248 INFO:teuthology.orchestra.run.smithi032.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T14:57:18.248 INFO:teuthology.orchestra.run.smithi032.stdout: = sunit=0 swidth=0 blks 2024-09-21T14:57:18.248 INFO:teuthology.orchestra.run.smithi032.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T14:57:18.248 INFO:teuthology.orchestra.run.smithi032.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-21T14:57:18.248 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T14:57:18.248 INFO:teuthology.orchestra.run.smithi032.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T14:57:18.249 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi032.front.sepia.ceph.com -o noatime 2024-09-21T14:57:18.249 DEBUG:teuthology.orchestra.run.smithi032:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-21T14:57:18.271 DEBUG:teuthology.orchestra.run.smithi032:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-21T14:57:18.330 INFO:teuthology.orchestra.run.smithi032.stderr:sudo: /sbin/restorecon: command not found 2024-09-21T14:57:18.331 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T14:57:18.331 DEBUG:teuthology.orchestra.run.smithi032:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T14:57:18.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-09-21T14:57:18.409+0000 7f3c90f10880 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-21T14:57:18.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-09-21T14:57:18.409+0000 7f3c90f10880 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-21T14:57:18.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-09-21T14:57:18.409+0000 7f3c90f10880 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-21T14:57:18.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-09-21T14:57:18.409+0000 7f3c90f10880 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-21T14:57:18.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-09-21T14:57:18.409+0000 7f3c90f10880 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-21T14:57:18.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-09-21T14:57:18.409+0000 7f3c90f10880 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-21T14:57:21.394 DEBUG:teuthology.orchestra.run.smithi032:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-21T14:57:21.409 DEBUG:teuthology.orchestra.run.smithi032:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T14:57:21.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-09-21T14:57:21.493+0000 7f879b9ec880 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-21T14:57:21.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-09-21T14:57:21.493+0000 7f879b9ec880 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-21T14:57:21.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-09-21T14:57:21.493+0000 7f879b9ec880 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-21T14:57:21.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-09-21T14:57:21.493+0000 7f879b9ec880 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-21T14:57:21.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-09-21T14:57:21.493+0000 7f879b9ec880 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-21T14:57:21.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-09-21T14:57:21.493+0000 7f879b9ec880 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-21T14:57:24.486 DEBUG:teuthology.orchestra.run.smithi032:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-21T14:57:24.502 DEBUG:teuthology.orchestra.run.smithi032:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T14:57:24.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-09-21T14:57:24.545+0000 7fa573a87880 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-21T14:57:24.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-09-21T14:57:24.545+0000 7fa573a87880 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-21T14:57:24.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-09-21T14:57:24.549+0000 7fa573a87880 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-21T14:57:24.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-09-21T14:57:24.549+0000 7fa573a87880 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-21T14:57:24.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-09-21T14:57:24.549+0000 7fa573a87880 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-21T14:57:24.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-09-21T14:57:24.549+0000 7fa573a87880 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-21T14:57:27.542 DEBUG:teuthology.orchestra.run.smithi032:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-21T14:57:27.558 DEBUG:teuthology.orchestra.run.smithi032:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T14:57:27.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-09-21T14:57:27.597+0000 7fd73000d880 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-21T14:57:27.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-09-21T14:57:27.597+0000 7fd73000d880 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-21T14:57:27.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-09-21T14:57:27.597+0000 7fd73000d880 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-21T14:57:27.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-09-21T14:57:27.597+0000 7fd73000d880 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-21T14:57:27.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-09-21T14:57:27.597+0000 7fd73000d880 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-21T14:57:27.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-09-21T14:57:27.597+0000 7fd73000d880 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-21T14:57:30.593 DEBUG:teuthology.orchestra.run.smithi032:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-21T14:57:30.610 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-21T14:57:30.627 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-21T14:57:30.627 INFO:tasks.ceph:role: osd.4 2024-09-21T14:57:30.627 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi046.front.sepia.ceph.com 2024-09-21T14:57:30.627 DEBUG:teuthology.orchestra.run.smithi046:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-21T14:57:30.716 INFO:teuthology.orchestra.run.smithi046.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T14:57:30.716 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T14:57:30.716 INFO:teuthology.orchestra.run.smithi046.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T14:57:30.716 INFO:teuthology.orchestra.run.smithi046.stdout: = reflink=1 2024-09-21T14:57:30.717 INFO:teuthology.orchestra.run.smithi046.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T14:57:30.717 INFO:teuthology.orchestra.run.smithi046.stdout: = sunit=0 swidth=0 blks 2024-09-21T14:57:30.717 INFO:teuthology.orchestra.run.smithi046.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T14:57:30.717 INFO:teuthology.orchestra.run.smithi046.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-21T14:57:30.717 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T14:57:30.717 INFO:teuthology.orchestra.run.smithi046.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T14:57:30.718 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi046.front.sepia.ceph.com -o noatime 2024-09-21T14:57:30.718 DEBUG:teuthology.orchestra.run.smithi046:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-21T14:57:30.738 DEBUG:teuthology.orchestra.run.smithi046:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-21T14:57:30.748 INFO:teuthology.orchestra.run.smithi046.stderr:sudo: /sbin/restorecon: command not found 2024-09-21T14:57:30.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T14:57:30.749 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-21T14:57:30.760 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-21T14:57:30.760 INFO:tasks.ceph:role: osd.5 2024-09-21T14:57:30.760 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi046.front.sepia.ceph.com 2024-09-21T14:57:30.760 DEBUG:teuthology.orchestra.run.smithi046:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-21T14:57:30.799 INFO:teuthology.orchestra.run.smithi046.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T14:57:30.799 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T14:57:30.799 INFO:teuthology.orchestra.run.smithi046.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T14:57:30.799 INFO:teuthology.orchestra.run.smithi046.stdout: = reflink=1 2024-09-21T14:57:30.799 INFO:teuthology.orchestra.run.smithi046.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T14:57:30.799 INFO:teuthology.orchestra.run.smithi046.stdout: = sunit=0 swidth=0 blks 2024-09-21T14:57:30.799 INFO:teuthology.orchestra.run.smithi046.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T14:57:30.799 INFO:teuthology.orchestra.run.smithi046.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-21T14:57:30.800 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T14:57:30.800 INFO:teuthology.orchestra.run.smithi046.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T14:57:30.801 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi046.front.sepia.ceph.com -o noatime 2024-09-21T14:57:30.801 DEBUG:teuthology.orchestra.run.smithi046:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-21T14:57:30.821 DEBUG:teuthology.orchestra.run.smithi046:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-21T14:57:30.875 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T14:57:30.875 INFO:teuthology.orchestra.run.smithi046.stderr:sudo: /sbin/restorecon: command not found 2024-09-21T14:57:30.875 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-21T14:57:30.886 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-21T14:57:30.886 INFO:tasks.ceph:role: osd.6 2024-09-21T14:57:30.886 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi046.front.sepia.ceph.com 2024-09-21T14:57:30.886 DEBUG:teuthology.orchestra.run.smithi046:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-21T14:57:30.928 INFO:teuthology.orchestra.run.smithi046.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T14:57:30.928 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T14:57:30.928 INFO:teuthology.orchestra.run.smithi046.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T14:57:30.928 INFO:teuthology.orchestra.run.smithi046.stdout: = reflink=1 2024-09-21T14:57:30.928 INFO:teuthology.orchestra.run.smithi046.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T14:57:30.928 INFO:teuthology.orchestra.run.smithi046.stdout: = sunit=0 swidth=0 blks 2024-09-21T14:57:30.928 INFO:teuthology.orchestra.run.smithi046.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T14:57:30.928 INFO:teuthology.orchestra.run.smithi046.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-21T14:57:30.928 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T14:57:30.929 INFO:teuthology.orchestra.run.smithi046.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T14:57:30.929 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi046.front.sepia.ceph.com -o noatime 2024-09-21T14:57:30.930 DEBUG:teuthology.orchestra.run.smithi046:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-21T14:57:30.948 DEBUG:teuthology.orchestra.run.smithi046:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-21T14:57:31.003 INFO:teuthology.orchestra.run.smithi046.stderr:sudo: /sbin/restorecon: command not found 2024-09-21T14:57:31.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T14:57:31.005 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-21T14:57:31.015 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-21T14:57:31.015 INFO:tasks.ceph:role: osd.7 2024-09-21T14:57:31.015 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi046.front.sepia.ceph.com 2024-09-21T14:57:31.015 DEBUG:teuthology.orchestra.run.smithi046:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-21T14:57:31.058 INFO:teuthology.orchestra.run.smithi046.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T14:57:31.059 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T14:57:31.059 INFO:teuthology.orchestra.run.smithi046.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T14:57:31.059 INFO:teuthology.orchestra.run.smithi046.stdout: = reflink=1 2024-09-21T14:57:31.059 INFO:teuthology.orchestra.run.smithi046.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T14:57:31.059 INFO:teuthology.orchestra.run.smithi046.stdout: = sunit=0 swidth=0 blks 2024-09-21T14:57:31.059 INFO:teuthology.orchestra.run.smithi046.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T14:57:31.059 INFO:teuthology.orchestra.run.smithi046.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-21T14:57:31.059 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T14:57:31.059 INFO:teuthology.orchestra.run.smithi046.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T14:57:31.060 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi046.front.sepia.ceph.com -o noatime 2024-09-21T14:57:31.061 DEBUG:teuthology.orchestra.run.smithi046:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-21T14:57:31.125 DEBUG:teuthology.orchestra.run.smithi046:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-21T14:57:31.136 INFO:teuthology.orchestra.run.smithi046.stderr:sudo: /sbin/restorecon: command not found 2024-09-21T14:57:31.136 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T14:57:31.137 DEBUG:teuthology.orchestra.run.smithi046:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T14:57:31.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-21T14:57:31.214+0000 7fbb6d597880 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-21T14:57:31.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-21T14:57:31.214+0000 7fbb6d597880 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-21T14:57:31.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-21T14:57:31.214+0000 7fbb6d597880 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-21T14:57:31.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-21T14:57:31.214+0000 7fbb6d597880 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-21T14:57:31.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-21T14:57:31.214+0000 7fbb6d597880 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-21T14:57:31.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-21T14:57:31.214+0000 7fbb6d597880 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-21T14:57:34.264 DEBUG:teuthology.orchestra.run.smithi046:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-21T14:57:34.279 DEBUG:teuthology.orchestra.run.smithi046:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T14:57:34.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-21T14:57:34.366+0000 7f2261836880 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-21T14:57:34.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-21T14:57:34.366+0000 7f2261836880 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-21T14:57:34.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-21T14:57:34.366+0000 7f2261836880 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-21T14:57:34.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-21T14:57:34.366+0000 7f2261836880 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-21T14:57:34.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-21T14:57:34.366+0000 7f2261836880 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-21T14:57:34.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-21T14:57:34.366+0000 7f2261836880 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-21T14:57:37.367 DEBUG:teuthology.orchestra.run.smithi046:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-21T14:57:37.385 DEBUG:teuthology.orchestra.run.smithi046:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T14:57:37.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-21T14:57:37.474+0000 7f1c9d594880 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-21T14:57:37.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-21T14:57:37.474+0000 7f1c9d594880 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-21T14:57:37.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-21T14:57:37.474+0000 7f1c9d594880 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-09-21T14:57:37.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-21T14:57:37.474+0000 7f1c9d594880 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-09-21T14:57:37.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-21T14:57:37.474+0000 7f1c9d594880 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-09-21T14:57:37.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-21T14:57:37.474+0000 7f1c9d594880 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-21T14:57:40.483 DEBUG:teuthology.orchestra.run.smithi046:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-21T14:57:40.500 DEBUG:teuthology.orchestra.run.smithi046:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T14:57:40.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-21T14:57:40.538+0000 7fd06f6ca880 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-21T14:57:40.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-21T14:57:40.538+0000 7fd06f6ca880 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-21T14:57:40.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-21T14:57:40.538+0000 7fd06f6ca880 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-09-21T14:57:40.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-21T14:57:40.538+0000 7fd06f6ca880 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-09-21T14:57:40.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-21T14:57:40.538+0000 7fd06f6ca880 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-09-21T14:57:40.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-21T14:57:40.538+0000 7fd06f6ca880 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-21T14:57:43.530 DEBUG:teuthology.orchestra.run.smithi046:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-21T14:57:43.589 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-21T14:57:43.589 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-09-21T14:57:43.589 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-21T14:57:43.606 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-09-21T14:57:43.606 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-21T14:57:43.618 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-09-21T14:57:43.618 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-21T14:57:43.627 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-09-21T14:57:43.627 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-21T14:57:43.686 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-09-21T14:57:43.686 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-21T14:57:43.698 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-21T14:57:43.698 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-21T14:57:43.714 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-21T14:57:43.714 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-21T14:57:43.772 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-21T14:57:43.772 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-21T14:57:43.787 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-21T14:57:43.787 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-21T14:57:43.797 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-21T14:57:43.797 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-21T14:57:43.856 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-09-21T14:57:43.856 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-21T14:57:43.863 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-21T14:57:43.863 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-21T14:57:43.867 INFO:tasks.ceph:Adding keys to all mons... 2024-09-21T14:57:43.867 DEBUG:teuthology.orchestra.run.smithi032:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-21T14:57:43.908 DEBUG:teuthology.orchestra.run.smithi046:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-21T14:57:43.921 INFO:teuthology.orchestra.run.smithi032.stdout:[mgr.y] 2024-09-21T14:57:43.921 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQDN3u5mpLRrJRAAurUhQ29JmD/iR7F8PGDFZQ== 2024-09-21T14:57:43.921 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.0] 2024-09-21T14:57:43.921 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQDO3u5mWb+SGBAAcmrjyCXzUT5L/1NNrGP/vQ== 2024-09-21T14:57:43.921 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.1] 2024-09-21T14:57:43.921 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQDR3u5m/JiiHRAAX1ugT8usaSkcjlB97lOwcw== 2024-09-21T14:57:43.921 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.2] 2024-09-21T14:57:43.921 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQDU3u5mxO/JIBAAjT4BDqQ7E8Ro4gHFy8+bQQ== 2024-09-21T14:57:43.921 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.3] 2024-09-21T14:57:43.921 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQDX3u5mKTu5IxAAe3bjnN1pYon9ALmC9vk0zQ== 2024-09-21T14:57:43.921 INFO:teuthology.orchestra.run.smithi032.stdout:[mgr.x] 2024-09-21T14:57:43.922 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQDN3u5mbay9KBAAVHkdGBMafAahcutQh0TeQA== 2024-09-21T14:57:43.922 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.4] 2024-09-21T14:57:43.922 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQDb3u5mQgcLDRAA4nUuxqIL+K4R3RpcunDTWQ== 2024-09-21T14:57:43.922 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.5] 2024-09-21T14:57:43.922 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQDe3u5mHLsiFhAAfE9Wgmx26nq5SVMMXYg38g== 2024-09-21T14:57:43.922 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.6] 2024-09-21T14:57:43.922 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQDh3u5mpu+DHBAAkCGMh37k7g0PLX5Q3UrUmw== 2024-09-21T14:57:43.922 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.7] 2024-09-21T14:57:43.922 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQDk3u5mBtpfIBAAhpj1eB4Lbht71O1gLvu1jw== 2024-09-21T14:57:43.922 INFO:teuthology.orchestra.run.smithi032.stdout:[client.0] 2024-09-21T14:57:43.922 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQDN3u5myIjFKxAA7hLW8fUABUhVX82I5YVYMg== 2024-09-21T14:57:43.922 INFO:teuthology.orchestra.run.smithi032.stdout:[client.1] 2024-09-21T14:57:43.922 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQDN3u5mYU9FLxAAqCqWOrosaDGFWyKB3fuJrQ== 2024-09-21T14:57:43.924 INFO:teuthology.orchestra.run.smithi046.stdout:[mgr.y] 2024-09-21T14:57:43.924 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQDN3u5mpLRrJRAAurUhQ29JmD/iR7F8PGDFZQ== 2024-09-21T14:57:43.924 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.0] 2024-09-21T14:57:43.924 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQDO3u5mWb+SGBAAcmrjyCXzUT5L/1NNrGP/vQ== 2024-09-21T14:57:43.924 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.1] 2024-09-21T14:57:43.924 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQDR3u5m/JiiHRAAX1ugT8usaSkcjlB97lOwcw== 2024-09-21T14:57:43.924 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.2] 2024-09-21T14:57:43.924 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQDU3u5mxO/JIBAAjT4BDqQ7E8Ro4gHFy8+bQQ== 2024-09-21T14:57:43.924 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.3] 2024-09-21T14:57:43.925 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQDX3u5mKTu5IxAAe3bjnN1pYon9ALmC9vk0zQ== 2024-09-21T14:57:43.925 INFO:teuthology.orchestra.run.smithi046.stdout:[mgr.x] 2024-09-21T14:57:43.925 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQDN3u5mbay9KBAAVHkdGBMafAahcutQh0TeQA== 2024-09-21T14:57:43.925 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.4] 2024-09-21T14:57:43.925 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQDb3u5mQgcLDRAA4nUuxqIL+K4R3RpcunDTWQ== 2024-09-21T14:57:43.925 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.5] 2024-09-21T14:57:43.925 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQDe3u5mHLsiFhAAfE9Wgmx26nq5SVMMXYg38g== 2024-09-21T14:57:43.925 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.6] 2024-09-21T14:57:43.925 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQDh3u5mpu+DHBAAkCGMh37k7g0PLX5Q3UrUmw== 2024-09-21T14:57:43.925 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.7] 2024-09-21T14:57:43.925 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQDk3u5mBtpfIBAAhpj1eB4Lbht71O1gLvu1jw== 2024-09-21T14:57:43.925 INFO:teuthology.orchestra.run.smithi046.stdout:[client.0] 2024-09-21T14:57:43.925 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQDN3u5myIjFKxAA7hLW8fUABUhVX82I5YVYMg== 2024-09-21T14:57:43.925 INFO:teuthology.orchestra.run.smithi046.stdout:[client.1] 2024-09-21T14:57:43.926 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQDN3u5mYU9FLxAAqCqWOrosaDGFWyKB3fuJrQ== 2024-09-21T14:57:43.926 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-21T14:57:43.928 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-21T14:57:43.977 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T14:57:43.979 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T14:57:44.028 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T14:57:44.030 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T14:57:44.081 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T14:57:44.083 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T14:57:44.134 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T14:57:44.136 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T14:57:44.181 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-21T14:57:44.184 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-21T14:57:44.230 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T14:57:44.232 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T14:57:44.281 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T14:57:44.283 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T14:57:44.333 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T14:57:44.335 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T14:57:44.380 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T14:57:44.383 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T14:57:44.472 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T14:57:44.475 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T14:57:44.528 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T14:57:44.530 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T14:57:44.583 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-21T14:57:44.583 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-21T14:57:44.599 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-21T14:57:44.685 DEBUG:teuthology.orchestra.run.smithi032:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-21T14:57:44.702 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-21T14:57:44.712 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-21T14:57:44.793 DEBUG:teuthology.orchestra.run.smithi032:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-21T14:57:44.808 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-21T14:57:44.825 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-21T14:57:44.906 DEBUG:teuthology.orchestra.run.smithi046:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-21T14:57:44.920 DEBUG:teuthology.orchestra.run.smithi032:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-21T14:57:44.923 DEBUG:teuthology.orchestra.run.smithi046:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-21T14:57:44.927 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-21T14:57:44.928 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-21T14:57:44.928 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-21T14:57:44.972 INFO:tasks.ceph.mon.a:Started 2024-09-21T14:57:44.972 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-21T14:57:44.973 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-21T14:57:44.975 INFO:tasks.ceph.mon.c:Started 2024-09-21T14:57:44.975 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-21T14:57:44.975 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-21T14:57:44.977 INFO:tasks.ceph.mon.b:Started 2024-09-21T14:57:44.977 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-21T14:57:44.977 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-21T14:57:44.977 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-21T14:57:44.979 INFO:tasks.ceph.mgr.y:Started 2024-09-21T14:57:44.980 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-21T14:57:44.980 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-21T14:57:44.982 INFO:tasks.ceph.mgr.x:Started 2024-09-21T14:57:44.982 INFO:tasks.ceph:Setting crush tunables to default 2024-09-21T14:57:44.982 DEBUG:teuthology.orchestra.run.smithi032:> sudo ceph --cluster ceph osd crush tunables default 2024-09-21T14:57:50.462 INFO:teuthology.orchestra.run.smithi032.stderr:adjusted tunables profile to default 2024-09-21T14:57:50.477 INFO:tasks.ceph:check_enable_crimson: False 2024-09-21T14:57:50.477 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-21T14:57:50.477 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-09-21T14:57:50.477 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-21T14:57:50.492 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-09-21T14:57:50.492 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-21T14:57:50.503 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-09-21T14:57:50.503 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-21T14:57:50.513 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-09-21T14:57:50.514 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-21T14:57:50.523 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-21T14:57:50.523 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-21T14:57:50.539 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-21T14:57:50.540 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-21T14:57:50.551 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-09-21T14:57:50.544+0000 7f9432b6b2c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-21T14:57:50.596 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-21T14:57:50.596 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-21T14:57:50.605 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-21T14:57:50.606 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-21T14:57:50.621 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --cluster ceph osd new a4a73316-3a94-42e9-9a51-deb46a66bb06 0 2024-09-21T14:57:50.891 INFO:teuthology.orchestra.run.smithi046.stdout:0 2024-09-21T14:57:50.907 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --cluster ceph osd new 006757a0-94d1-4a9b-aef8-73634cc5828e 1 2024-09-21T14:57:51.126 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-09-21T14:57:51.139 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --cluster ceph osd new 0b499a23-820e-4e6c-a558-adbf88b7014f 2 2024-09-21T14:57:51.314 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-09-21T14:57:51.308+0000 7f9432b6b2c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-21T14:57:51.332 INFO:tasks.ceph.mgr.x.smithi046.stderr:2024-09-21T14:57:51.329+0000 7f482bc8f2c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-21T14:57:51.344 INFO:teuthology.orchestra.run.smithi046.stdout:2 2024-09-21T14:57:51.363 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --cluster ceph osd new 5a57b3dd-4a07-45b7-b159-3cac4c7eb262 3 2024-09-21T14:57:51.377 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-09-21T14:57:51.372+0000 7f9432b6b2c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-21T14:57:51.553 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-09-21T14:57:51.548+0000 7f9432b6b2c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-21T14:57:51.571 INFO:teuthology.orchestra.run.smithi046.stdout:3 2024-09-21T14:57:51.586 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --cluster ceph osd new 39a00807-7517-474e-8842-623dd7093cc5 4 2024-09-21T14:57:51.789 INFO:teuthology.orchestra.run.smithi046.stdout:4 2024-09-21T14:57:51.791 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-09-21T14:57:51.788+0000 7f9432b6b2c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-21T14:57:51.803 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --cluster ceph osd new edf9f045-ddeb-4504-a0fc-1a0e4e853550 5 2024-09-21T14:57:51.847 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-09-21T14:57:51.844+0000 7f9432b6b2c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-21T14:57:52.010 INFO:teuthology.orchestra.run.smithi046.stdout:5 2024-09-21T14:57:52.024 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --cluster ceph osd new c820ea6b-88fe-4421-83ec-17c466691fea 6 2024-09-21T14:57:52.044 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-09-21T14:57:52.040+0000 7f9432b6b2c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-21T14:57:52.061 INFO:tasks.ceph.mgr.x.smithi046.stderr:2024-09-21T14:57:52.061+0000 7f482bc8f2c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-21T14:57:52.116 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-09-21T14:57:52.112+0000 7f9432b6b2c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-21T14:57:52.124 INFO:tasks.ceph.mgr.x.smithi046.stderr:2024-09-21T14:57:52.121+0000 7f482bc8f2c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-21T14:57:52.187 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-09-21T14:57:52.184+0000 7f9432b6b2c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-21T14:57:52.233 INFO:teuthology.orchestra.run.smithi046.stdout:6 2024-09-21T14:57:52.248 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --cluster ceph osd new 3a2f82a4-28a0-4fcb-9d6b-2285fa4d07ac 7 2024-09-21T14:57:52.305 INFO:tasks.ceph.mgr.x.smithi046.stderr:2024-09-21T14:57:52.305+0000 7f482bc8f2c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-21T14:57:52.447 INFO:teuthology.orchestra.run.smithi046.stdout:7 2024-09-21T14:57:52.460 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-21T14:57:52.460 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-21T14:57:52.463 INFO:tasks.ceph.osd.0:Started 2024-09-21T14:57:52.463 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-21T14:57:52.463 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-21T14:57:52.467 INFO:tasks.ceph.osd.1:Started 2024-09-21T14:57:52.467 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-21T14:57:52.467 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-21T14:57:52.469 INFO:tasks.ceph.osd.2:Started 2024-09-21T14:57:52.469 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-21T14:57:52.470 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-21T14:57:52.472 INFO:tasks.ceph.osd.3:Started 2024-09-21T14:57:52.472 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-21T14:57:52.472 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-21T14:57:52.474 INFO:tasks.ceph.osd.4:Started 2024-09-21T14:57:52.474 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-21T14:57:52.474 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-21T14:57:52.476 INFO:tasks.ceph.osd.5:Started 2024-09-21T14:57:52.476 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-21T14:57:52.476 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-21T14:57:52.479 INFO:tasks.ceph.osd.6:Started 2024-09-21T14:57:52.479 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-21T14:57:52.479 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-21T14:57:52.481 INFO:tasks.ceph.osd.7:Started 2024-09-21T14:57:52.481 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T14:57:52.576 INFO:tasks.ceph.mgr.x.smithi046.stderr:2024-09-21T14:57:52.573+0000 7f482bc8f2c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-21T14:57:52.629 INFO:tasks.ceph.mgr.x.smithi046.stderr:2024-09-21T14:57:52.629+0000 7f482bc8f2c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-21T14:57:52.648 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-09-21T14:57:52.644+0000 7f9432b6b2c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-21T14:57:52.677 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-21T14:57:52.677 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":10,"fsid":"caf51e55-9234-4a26-bcd2-d60811962560","created":"2024-09-21T14:57:50.338822+0000","modified":"2024-09-21T14:57:52.442036+0000","last_up_change":"0.000000","last_in_change":"2024-09-21T14:57:52.442036+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"a4a73316-3a94-42e9-9a51-deb46a66bb06","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"006757a0-94d1-4a9b-aef8-73634cc5828e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"0b499a23-820e-4e6c-a558-adbf88b7014f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"5a57b3dd-4a07-45b7-b159-3cac4c7eb262","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"39a00807-7517-474e-8842-623dd7093cc5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"edf9f045-ddeb-4504-a0fc-1a0e4e853550","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"c820ea6b-88fe-4421-83ec-17c466691fea","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"3a2f82a4-28a0-4fcb-9d6b-2285fa4d07ac","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T14:57:52.691 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-21T14:57:52.691 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-21T14:57:52.776 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-09-21T14:57:52.772+0000 7f9432b6b2c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-21T14:57:52.816 INFO:tasks.ceph.mgr.x.smithi046.stderr:2024-09-21T14:57:52.813+0000 7f482bc8f2c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-21T14:57:52.865 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:57:52.860+0000 7fa86887e880 -1 Falling back to public interface 2024-09-21T14:57:52.867 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:57:52.865+0000 7f315d409880 -1 Falling back to public interface 2024-09-21T14:57:52.867 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:57:52.865+0000 7fe95ed61880 -1 Falling back to public interface 2024-09-21T14:57:52.877 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:57:52.872+0000 7f50a8b42880 -1 Falling back to public interface 2024-09-21T14:57:52.881 INFO:tasks.ceph.mgr.x.smithi046.stderr:2024-09-21T14:57:52.881+0000 7f482bc8f2c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-21T14:57:52.893 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:57:52.888+0000 7f016ed09880 -1 Falling back to public interface 2024-09-21T14:57:52.947 INFO:tasks.ceph.mgr.x.smithi046.stderr:2024-09-21T14:57:52.945+0000 7f482bc8f2c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-21T14:57:53.093 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:57:53.088+0000 7fd8d1739880 -1 Falling back to public interface 2024-09-21T14:57:53.115 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:57:53.113+0000 7f6054310880 -1 Falling back to public interface 2024-09-21T14:57:53.115 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:57:53.113+0000 7efc0cea5880 -1 Falling back to public interface 2024-09-21T14:57:53.149 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-09-21T14:57:53.144+0000 7f9432b6b2c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-21T14:57:53.347 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-09-21T14:57:53.340+0000 7f9432b6b2c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-21T14:57:53.348 INFO:tasks.ceph.mgr.x.smithi046.stderr:2024-09-21T14:57:53.345+0000 7f482bc8f2c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-21T14:57:53.418 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-09-21T14:57:53.412+0000 7f9432b6b2c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-21T14:57:53.470 INFO:tasks.ceph.mgr.x.smithi046.stderr:2024-09-21T14:57:53.469+0000 7f482bc8f2c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-21T14:57:53.497 DEBUG:teuthology.orchestra.run.smithi032:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-21T14:57:53.582 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-09-21T14:57:53.576+0000 7f9432b6b2c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-21T14:57:53.668 INFO:teuthology.misc.health.smithi032.stdout: 2024-09-21T14:57:53.668 INFO:teuthology.misc.health.smithi032.stdout:{"epoch":10,"fsid":"caf51e55-9234-4a26-bcd2-d60811962560","created":"2024-09-21T14:57:50.338822+0000","modified":"2024-09-21T14:57:52.442036+0000","last_up_change":"0.000000","last_in_change":"2024-09-21T14:57:52.442036+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"a4a73316-3a94-42e9-9a51-deb46a66bb06","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"006757a0-94d1-4a9b-aef8-73634cc5828e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"0b499a23-820e-4e6c-a558-adbf88b7014f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"5a57b3dd-4a07-45b7-b159-3cac4c7eb262","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"39a00807-7517-474e-8842-623dd7093cc5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"edf9f045-ddeb-4504-a0fc-1a0e4e853550","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"c820ea6b-88fe-4421-83ec-17c466691fea","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"3a2f82a4-28a0-4fcb-9d6b-2285fa4d07ac","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T14:57:53.679 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-21T14:57:53.828 INFO:tasks.ceph.mgr.x.smithi046.stderr:2024-09-21T14:57:53.825+0000 7f482bc8f2c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-21T14:57:54.019 INFO:tasks.ceph.mgr.x.smithi046.stderr:2024-09-21T14:57:54.017+0000 7f482bc8f2c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-21T14:57:54.090 INFO:tasks.ceph.mgr.x.smithi046.stderr:2024-09-21T14:57:54.089+0000 7f482bc8f2c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-21T14:57:54.253 INFO:tasks.ceph.mgr.x.smithi046.stderr:2024-09-21T14:57:54.253+0000 7f482bc8f2c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-21T14:57:54.266 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-09-21T14:57:54.260+0000 7f9432b6b2c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-21T14:57:54.337 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-09-21T14:57:54.332+0000 7f9432b6b2c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-21T14:57:54.496 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-09-21T14:57:54.492+0000 7f9432b6b2c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-21T14:57:54.563 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-09-21T14:57:54.560+0000 7f9432b6b2c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-21T14:57:54.637 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-09-21T14:57:54.632+0000 7f9432b6b2c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-21T14:57:54.938 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-09-21T14:57:54.932+0000 7f9432b6b2c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-21T14:57:54.977 INFO:tasks.ceph.mgr.x.smithi046.stderr:2024-09-21T14:57:54.977+0000 7f482bc8f2c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-21T14:57:55.011 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-09-21T14:57:55.008+0000 7f9432b6b2c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-21T14:57:55.050 INFO:tasks.ceph.mgr.x.smithi046.stderr:2024-09-21T14:57:55.049+0000 7f482bc8f2c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-21T14:57:55.218 INFO:tasks.ceph.mgr.x.smithi046.stderr:2024-09-21T14:57:55.217+0000 7f482bc8f2c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-21T14:57:55.288 INFO:tasks.ceph.mgr.x.smithi046.stderr:2024-09-21T14:57:55.285+0000 7f482bc8f2c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-21T14:57:55.365 INFO:tasks.ceph.mgr.x.smithi046.stderr:2024-09-21T14:57:55.365+0000 7f482bc8f2c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-21T14:57:55.710 INFO:tasks.ceph.mgr.x.smithi046.stderr:2024-09-21T14:57:55.709+0000 7f482bc8f2c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-21T14:57:55.744 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-09-21T14:57:55.740+0000 7f9432b6b2c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-21T14:57:55.795 INFO:tasks.ceph.mgr.x.smithi046.stderr:2024-09-21T14:57:55.793+0000 7f482bc8f2c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-21T14:57:56.245 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:57:56.240+0000 7fa86887e880 -1 osd.2 0 log_to_monitors true 2024-09-21T14:57:56.481 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:57:56.481+0000 7f315d409880 -1 osd.4 0 log_to_monitors true 2024-09-21T14:57:56.481 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:57:56.476+0000 7f50a8b42880 -1 osd.0 0 log_to_monitors true 2024-09-21T14:57:56.492 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:57:56.488+0000 7f016ed09880 -1 osd.3 0 log_to_monitors true 2024-09-21T14:57:56.550 INFO:tasks.ceph.mgr.x.smithi046.stderr:2024-09-21T14:57:56.549+0000 7f482bc8f2c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-21T14:57:56.669 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:57:56.669+0000 7efc0cea5880 -1 osd.6 0 log_to_monitors true 2024-09-21T14:57:56.786 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-09-21T14:57:56.780+0000 7f94196e2700 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-21T14:57:56.788 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-09-21T14:57:56.784+0000 7f94196e2700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-21T14:57:56.796 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:57:56.793+0000 7fe95ed61880 -1 osd.5 0 log_to_monitors true 2024-09-21T14:57:56.951 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:57:56.948+0000 7fd8d1739880 -1 osd.1 0 log_to_monitors true 2024-09-21T14:57:56.966 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:57:56.965+0000 7f6054310880 -1 osd.7 0 log_to_monitors true 2024-09-21T14:57:59.114 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:57:59.113+0000 7efc0966b700 -1 osd.6 0 waiting for initial osdmap 2024-09-21T14:57:59.119 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:57:59.117+0000 7efc00793700 -1 osd.6 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T14:57:59.126 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:57:59.125+0000 7f3159248700 -1 osd.4 0 waiting for initial osdmap 2024-09-21T14:57:59.128 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:57:59.124+0000 7f50a5308700 -1 osd.0 0 waiting for initial osdmap 2024-09-21T14:57:59.130 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:57:59.129+0000 7f3150cf7700 -1 osd.4 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T14:57:59.134 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:57:59.128+0000 7f509c430700 -1 osd.0 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T14:57:59.134 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:57:59.133+0000 7fe95b527700 -1 osd.5 0 waiting for initial osdmap 2024-09-21T14:57:59.140 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:57:59.137+0000 7fe95264f700 -1 osd.5 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T14:57:59.141 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:57:59.136+0000 7f016ab48700 -1 osd.3 0 waiting for initial osdmap 2024-09-21T14:57:59.148 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:57:59.144+0000 7f01625f7700 -1 osd.3 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T14:57:59.160 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:57:59.156+0000 7fa865044700 -1 osd.2 0 waiting for initial osdmap 2024-09-21T14:57:59.165 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:57:59.160+0000 7fa85c16c700 -1 osd.2 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T14:58:00.108 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:58:00.104+0000 7fd8cd578700 -1 osd.1 0 waiting for initial osdmap 2024-09-21T14:58:00.114 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:58:00.108+0000 7fd8c5027700 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T14:58:00.176 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:58:00.173+0000 7f605014f700 -1 osd.7 0 waiting for initial osdmap 2024-09-21T14:58:00.185 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:58:00.185+0000 7f6047bfe700 -1 osd.7 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T14:58:00.485 DEBUG:teuthology.orchestra.run.smithi032:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-21T14:58:00.787 INFO:teuthology.misc.health.smithi032.stdout: 2024-09-21T14:58:00.787 INFO:teuthology.misc.health.smithi032.stdout:{"epoch":14,"fsid":"caf51e55-9234-4a26-bcd2-d60811962560","created":"2024-09-21T14:57:50.338822+0000","modified":"2024-09-21T14:58:00.116465+0000","last_up_change":"2024-09-21T14:58:00.116465+0000","last_in_change":"2024-09-21T14:57:52.442036+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"a4a73316-3a94-42e9-9a51-deb46a66bb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":4016672243},{"type":"v1","addr":"172.21.15.32:6809","nonce":4016672243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":4016672243},{"type":"v1","addr":"172.21.15.32:6811","nonce":4016672243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":4016672243},{"type":"v1","addr":"172.21.15.32:6815","nonce":4016672243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":4016672243},{"type":"v1","addr":"172.21.15.32:6813","nonce":4016672243}]},"public_addr":"172.21.15.32:6809/4016672243","cluster_addr":"172.21.15.32:6811/4016672243","heartbeat_back_addr":"172.21.15.32:6815/4016672243","heartbeat_front_addr":"172.21.15.32:6813/4016672243","state":["exists","up"]},{"osd":1,"uuid":"006757a0-94d1-4a9b-aef8-73634cc5828e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":641090577},{"type":"v1","addr":"172.21.15.32:6825","nonce":641090577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":641090577},{"type":"v1","addr":"172.21.15.32:6827","nonce":641090577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":641090577},{"type":"v1","addr":"172.21.15.32:6831","nonce":641090577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":641090577},{"type":"v1","addr":"172.21.15.32:6829","nonce":641090577}]},"public_addr":"172.21.15.32:6825/641090577","cluster_addr":"172.21.15.32:6827/641090577","heartbeat_back_addr":"172.21.15.32:6831/641090577","heartbeat_front_addr":"172.21.15.32:6829/641090577","state":["exists","up"]},{"osd":2,"uuid":"0b499a23-820e-4e6c-a558-adbf88b7014f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2609154414},{"type":"v1","addr":"172.21.15.32:6801","nonce":2609154414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2609154414},{"type":"v1","addr":"172.21.15.32:6803","nonce":2609154414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2609154414},{"type":"v1","addr":"172.21.15.32:6807","nonce":2609154414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2609154414},{"type":"v1","addr":"172.21.15.32:6805","nonce":2609154414}]},"public_addr":"172.21.15.32:6801/2609154414","cluster_addr":"172.21.15.32:6803/2609154414","heartbeat_back_addr":"172.21.15.32:6807/2609154414","heartbeat_front_addr":"172.21.15.32:6805/2609154414","state":["exists","up"]},{"osd":3,"uuid":"5a57b3dd-4a07-45b7-b159-3cac4c7eb262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":1462563376},{"type":"v1","addr":"172.21.15.32:6817","nonce":1462563376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1462563376},{"type":"v1","addr":"172.21.15.32:6819","nonce":1462563376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1462563376},{"type":"v1","addr":"172.21.15.32:6823","nonce":1462563376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1462563376},{"type":"v1","addr":"172.21.15.32:6821","nonce":1462563376}]},"public_addr":"172.21.15.32:6817/1462563376","cluster_addr":"172.21.15.32:6819/1462563376","heartbeat_back_addr":"172.21.15.32:6823/1462563376","heartbeat_front_addr":"172.21.15.32:6821/1462563376","state":["exists","up"]},{"osd":4,"uuid":"39a00807-7517-474e-8842-623dd7093cc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6801","nonce":144412042},{"type":"v1","addr":"172.21.15.46:6803","nonce":144412042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6805","nonce":144412042},{"type":"v1","addr":"172.21.15.46:6807","nonce":144412042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":144412042},{"type":"v1","addr":"172.21.15.46:6814","nonce":144412042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":144412042},{"type":"v1","addr":"172.21.15.46:6811","nonce":144412042}]},"public_addr":"172.21.15.46:6803/144412042","cluster_addr":"172.21.15.46:6807/144412042","heartbeat_back_addr":"172.21.15.46:6814/144412042","heartbeat_front_addr":"172.21.15.46:6811/144412042","state":["exists","up"]},{"osd":5,"uuid":"edf9f045-ddeb-4504-a0fc-1a0e4e853550","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":849101442},{"type":"v1","addr":"172.21.15.46:6802","nonce":849101442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":849101442},{"type":"v1","addr":"172.21.15.46:6806","nonce":849101442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":849101442},{"type":"v1","addr":"172.21.15.46:6815","nonce":849101442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":849101442},{"type":"v1","addr":"172.21.15.46:6810","nonce":849101442}]},"public_addr":"172.21.15.46:6802/849101442","cluster_addr":"172.21.15.46:6806/849101442","heartbeat_back_addr":"172.21.15.46:6815/849101442","heartbeat_front_addr":"172.21.15.46:6810/849101442","state":["exists","up"]},{"osd":6,"uuid":"c820ea6b-88fe-4421-83ec-17c466691fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":847189172},{"type":"v1","addr":"172.21.15.46:6818","nonce":847189172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":847189172},{"type":"v1","addr":"172.21.15.46:6822","nonce":847189172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6827","nonce":847189172},{"type":"v1","addr":"172.21.15.46:6828","nonce":847189172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6823","nonce":847189172},{"type":"v1","addr":"172.21.15.46:6825","nonce":847189172}]},"public_addr":"172.21.15.46:6818/847189172","cluster_addr":"172.21.15.46:6822/847189172","heartbeat_back_addr":"172.21.15.46:6828/847189172","heartbeat_front_addr":"172.21.15.46:6825/847189172","state":["exists","up"]},{"osd":7,"uuid":"3a2f82a4-28a0-4fcb-9d6b-2285fa4d07ac","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T14:58:00.806 DEBUG:teuthology.misc:7 of 8 OSDs are up 2024-09-21T14:58:07.612 DEBUG:teuthology.orchestra.run.smithi032:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-21T14:58:07.921 INFO:teuthology.misc.health.smithi032.stdout: 2024-09-21T14:58:07.921 INFO:teuthology.misc.health.smithi032.stdout:{"epoch":18,"fsid":"caf51e55-9234-4a26-bcd2-d60811962560","created":"2024-09-21T14:57:50.338822+0000","modified":"2024-09-21T14:58:04.199911+0000","last_up_change":"2024-09-21T14:58:01.177141+0000","last_in_change":"2024-09-21T14:57:52.442036+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T14:58:01.842810+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a4a73316-3a94-42e9-9a51-deb46a66bb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":4016672243},{"type":"v1","addr":"172.21.15.32:6809","nonce":4016672243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":4016672243},{"type":"v1","addr":"172.21.15.32:6811","nonce":4016672243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":4016672243},{"type":"v1","addr":"172.21.15.32:6815","nonce":4016672243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":4016672243},{"type":"v1","addr":"172.21.15.32:6813","nonce":4016672243}]},"public_addr":"172.21.15.32:6809/4016672243","cluster_addr":"172.21.15.32:6811/4016672243","heartbeat_back_addr":"172.21.15.32:6815/4016672243","heartbeat_front_addr":"172.21.15.32:6813/4016672243","state":["exists","up"]},{"osd":1,"uuid":"006757a0-94d1-4a9b-aef8-73634cc5828e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":641090577},{"type":"v1","addr":"172.21.15.32:6825","nonce":641090577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":641090577},{"type":"v1","addr":"172.21.15.32:6827","nonce":641090577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":641090577},{"type":"v1","addr":"172.21.15.32:6831","nonce":641090577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":641090577},{"type":"v1","addr":"172.21.15.32:6829","nonce":641090577}]},"public_addr":"172.21.15.32:6825/641090577","cluster_addr":"172.21.15.32:6827/641090577","heartbeat_back_addr":"172.21.15.32:6831/641090577","heartbeat_front_addr":"172.21.15.32:6829/641090577","state":["exists","up"]},{"osd":2,"uuid":"0b499a23-820e-4e6c-a558-adbf88b7014f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2609154414},{"type":"v1","addr":"172.21.15.32:6801","nonce":2609154414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2609154414},{"type":"v1","addr":"172.21.15.32:6803","nonce":2609154414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2609154414},{"type":"v1","addr":"172.21.15.32:6807","nonce":2609154414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2609154414},{"type":"v1","addr":"172.21.15.32:6805","nonce":2609154414}]},"public_addr":"172.21.15.32:6801/2609154414","cluster_addr":"172.21.15.32:6803/2609154414","heartbeat_back_addr":"172.21.15.32:6807/2609154414","heartbeat_front_addr":"172.21.15.32:6805/2609154414","state":["exists","up"]},{"osd":3,"uuid":"5a57b3dd-4a07-45b7-b159-3cac4c7eb262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":1462563376},{"type":"v1","addr":"172.21.15.32:6817","nonce":1462563376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1462563376},{"type":"v1","addr":"172.21.15.32:6819","nonce":1462563376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1462563376},{"type":"v1","addr":"172.21.15.32:6823","nonce":1462563376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1462563376},{"type":"v1","addr":"172.21.15.32:6821","nonce":1462563376}]},"public_addr":"172.21.15.32:6817/1462563376","cluster_addr":"172.21.15.32:6819/1462563376","heartbeat_back_addr":"172.21.15.32:6823/1462563376","heartbeat_front_addr":"172.21.15.32:6821/1462563376","state":["exists","up"]},{"osd":4,"uuid":"39a00807-7517-474e-8842-623dd7093cc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6801","nonce":144412042},{"type":"v1","addr":"172.21.15.46:6803","nonce":144412042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6805","nonce":144412042},{"type":"v1","addr":"172.21.15.46:6807","nonce":144412042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":144412042},{"type":"v1","addr":"172.21.15.46:6814","nonce":144412042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":144412042},{"type":"v1","addr":"172.21.15.46:6811","nonce":144412042}]},"public_addr":"172.21.15.46:6803/144412042","cluster_addr":"172.21.15.46:6807/144412042","heartbeat_back_addr":"172.21.15.46:6814/144412042","heartbeat_front_addr":"172.21.15.46:6811/144412042","state":["exists","up"]},{"osd":5,"uuid":"edf9f045-ddeb-4504-a0fc-1a0e4e853550","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":849101442},{"type":"v1","addr":"172.21.15.46:6802","nonce":849101442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":849101442},{"type":"v1","addr":"172.21.15.46:6806","nonce":849101442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":849101442},{"type":"v1","addr":"172.21.15.46:6815","nonce":849101442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":849101442},{"type":"v1","addr":"172.21.15.46:6810","nonce":849101442}]},"public_addr":"172.21.15.46:6802/849101442","cluster_addr":"172.21.15.46:6806/849101442","heartbeat_back_addr":"172.21.15.46:6815/849101442","heartbeat_front_addr":"172.21.15.46:6810/849101442","state":["exists","up"]},{"osd":6,"uuid":"c820ea6b-88fe-4421-83ec-17c466691fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":847189172},{"type":"v1","addr":"172.21.15.46:6818","nonce":847189172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":847189172},{"type":"v1","addr":"172.21.15.46:6822","nonce":847189172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6827","nonce":847189172},{"type":"v1","addr":"172.21.15.46:6828","nonce":847189172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6823","nonce":847189172},{"type":"v1","addr":"172.21.15.46:6825","nonce":847189172}]},"public_addr":"172.21.15.46:6818/847189172","cluster_addr":"172.21.15.46:6822/847189172","heartbeat_back_addr":"172.21.15.46:6828/847189172","heartbeat_front_addr":"172.21.15.46:6825/847189172","state":["exists","up"]},{"osd":7,"uuid":"3a2f82a4-28a0-4fcb-9d6b-2285fa4d07ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3894963553},{"type":"v1","addr":"172.21.15.46:6819","nonce":3894963553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":3894963553},{"type":"v1","addr":"172.21.15.46:6824","nonce":3894963553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":3894963553},{"type":"v1","addr":"172.21.15.46:6831","nonce":3894963553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":3894963553},{"type":"v1","addr":"172.21.15.46:6829","nonce":3894963553}]},"public_addr":"172.21.15.46:6819/3894963553","cluster_addr":"172.21.15.46:6824/3894963553","heartbeat_back_addr":"172.21.15.46:6831/3894963553","heartbeat_front_addr":"172.21.15.46:6829/3894963553","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T14:57:57.458317+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T14:57:57.907624+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T14:57:57.278817+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T14:57:57.500449+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T14:57:57.504325+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T14:57:57.777525+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T14:57:57.698961+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T14:57:57.932625+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T14:58:07.935 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-21T14:58:07.935 INFO:tasks.ceph:Creating RBD pool 2024-09-21T14:58:07.935 DEBUG:teuthology.orchestra.run.smithi032:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-21T14:58:08.326 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'rbd' created 2024-09-21T14:58:08.338 DEBUG:teuthology.orchestra.run.smithi032:> rbd --cluster ceph pool init rbd 2024-09-21T14:58:11.356 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-21T14:58:11.356 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-21T14:58:11.356 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-21T14:58:11.356 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-21T14:58:11.357 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-21T14:58:11.716 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-21T14:58:11.731 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":5,"flags":0,"active_gid":4104,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1616414276},{"type":"v1","addr":"172.21.15.32:6833","nonce":1616414276}]},"active_addr":"172.21.15.32:6833/1616414276","active_change":"2024-09-21T14:57:55.763912+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4123,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.32:0","nonce":4075151084}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.32:0","nonce":3411998640}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.32:0","nonce":2111471321}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.32:0","nonce":4216239891}]}]} 2024-09-21T14:58:11.734 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-21T14:58:11.734 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-21T14:58:11.734 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T14:58:12.045 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-21T14:58:12.045 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":22,"fsid":"caf51e55-9234-4a26-bcd2-d60811962560","created":"2024-09-21T14:57:50.338822+0000","modified":"2024-09-21T14:58:11.332733+0000","last_up_change":"2024-09-21T14:58:01.177141+0000","last_in_change":"2024-09-21T14:57:52.442036+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T14:58:01.842810+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-21T14:58:08.284925+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":22,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a4a73316-3a94-42e9-9a51-deb46a66bb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":4016672243},{"type":"v1","addr":"172.21.15.32:6809","nonce":4016672243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":4016672243},{"type":"v1","addr":"172.21.15.32:6811","nonce":4016672243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":4016672243},{"type":"v1","addr":"172.21.15.32:6815","nonce":4016672243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":4016672243},{"type":"v1","addr":"172.21.15.32:6813","nonce":4016672243}]},"public_addr":"172.21.15.32:6809/4016672243","cluster_addr":"172.21.15.32:6811/4016672243","heartbeat_back_addr":"172.21.15.32:6815/4016672243","heartbeat_front_addr":"172.21.15.32:6813/4016672243","state":["exists","up"]},{"osd":1,"uuid":"006757a0-94d1-4a9b-aef8-73634cc5828e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":641090577},{"type":"v1","addr":"172.21.15.32:6825","nonce":641090577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":641090577},{"type":"v1","addr":"172.21.15.32:6827","nonce":641090577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":641090577},{"type":"v1","addr":"172.21.15.32:6831","nonce":641090577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":641090577},{"type":"v1","addr":"172.21.15.32:6829","nonce":641090577}]},"public_addr":"172.21.15.32:6825/641090577","cluster_addr":"172.21.15.32:6827/641090577","heartbeat_back_addr":"172.21.15.32:6831/641090577","heartbeat_front_addr":"172.21.15.32:6829/641090577","state":["exists","up"]},{"osd":2,"uuid":"0b499a23-820e-4e6c-a558-adbf88b7014f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2609154414},{"type":"v1","addr":"172.21.15.32:6801","nonce":2609154414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2609154414},{"type":"v1","addr":"172.21.15.32:6803","nonce":2609154414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2609154414},{"type":"v1","addr":"172.21.15.32:6807","nonce":2609154414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2609154414},{"type":"v1","addr":"172.21.15.32:6805","nonce":2609154414}]},"public_addr":"172.21.15.32:6801/2609154414","cluster_addr":"172.21.15.32:6803/2609154414","heartbeat_back_addr":"172.21.15.32:6807/2609154414","heartbeat_front_addr":"172.21.15.32:6805/2609154414","state":["exists","up"]},{"osd":3,"uuid":"5a57b3dd-4a07-45b7-b159-3cac4c7eb262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":1462563376},{"type":"v1","addr":"172.21.15.32:6817","nonce":1462563376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1462563376},{"type":"v1","addr":"172.21.15.32:6819","nonce":1462563376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1462563376},{"type":"v1","addr":"172.21.15.32:6823","nonce":1462563376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1462563376},{"type":"v1","addr":"172.21.15.32:6821","nonce":1462563376}]},"public_addr":"172.21.15.32:6817/1462563376","cluster_addr":"172.21.15.32:6819/1462563376","heartbeat_back_addr":"172.21.15.32:6823/1462563376","heartbeat_front_addr":"172.21.15.32:6821/1462563376","state":["exists","up"]},{"osd":4,"uuid":"39a00807-7517-474e-8842-623dd7093cc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6801","nonce":144412042},{"type":"v1","addr":"172.21.15.46:6803","nonce":144412042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6805","nonce":144412042},{"type":"v1","addr":"172.21.15.46:6807","nonce":144412042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":144412042},{"type":"v1","addr":"172.21.15.46:6814","nonce":144412042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":144412042},{"type":"v1","addr":"172.21.15.46:6811","nonce":144412042}]},"public_addr":"172.21.15.46:6803/144412042","cluster_addr":"172.21.15.46:6807/144412042","heartbeat_back_addr":"172.21.15.46:6814/144412042","heartbeat_front_addr":"172.21.15.46:6811/144412042","state":["exists","up"]},{"osd":5,"uuid":"edf9f045-ddeb-4504-a0fc-1a0e4e853550","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":849101442},{"type":"v1","addr":"172.21.15.46:6802","nonce":849101442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":849101442},{"type":"v1","addr":"172.21.15.46:6806","nonce":849101442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":849101442},{"type":"v1","addr":"172.21.15.46:6815","nonce":849101442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":849101442},{"type":"v1","addr":"172.21.15.46:6810","nonce":849101442}]},"public_addr":"172.21.15.46:6802/849101442","cluster_addr":"172.21.15.46:6806/849101442","heartbeat_back_addr":"172.21.15.46:6815/849101442","heartbeat_front_addr":"172.21.15.46:6810/849101442","state":["exists","up"]},{"osd":6,"uuid":"c820ea6b-88fe-4421-83ec-17c466691fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":847189172},{"type":"v1","addr":"172.21.15.46:6818","nonce":847189172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":847189172},{"type":"v1","addr":"172.21.15.46:6822","nonce":847189172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6827","nonce":847189172},{"type":"v1","addr":"172.21.15.46:6828","nonce":847189172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6823","nonce":847189172},{"type":"v1","addr":"172.21.15.46:6825","nonce":847189172}]},"public_addr":"172.21.15.46:6818/847189172","cluster_addr":"172.21.15.46:6822/847189172","heartbeat_back_addr":"172.21.15.46:6828/847189172","heartbeat_front_addr":"172.21.15.46:6825/847189172","state":["exists","up"]},{"osd":7,"uuid":"3a2f82a4-28a0-4fcb-9d6b-2285fa4d07ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3894963553},{"type":"v1","addr":"172.21.15.46:6819","nonce":3894963553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":3894963553},{"type":"v1","addr":"172.21.15.46:6824","nonce":3894963553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":3894963553},{"type":"v1","addr":"172.21.15.46:6831","nonce":3894963553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":3894963553},{"type":"v1","addr":"172.21.15.46:6829","nonce":3894963553}]},"public_addr":"172.21.15.46:6819/3894963553","cluster_addr":"172.21.15.46:6824/3894963553","heartbeat_back_addr":"172.21.15.46:6831/3894963553","heartbeat_front_addr":"172.21.15.46:6829/3894963553","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T14:57:57.458317+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T14:57:57.907624+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T14:57:57.278817+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T14:57:57.500449+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T14:57:57.504325+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T14:57:57.777525+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T14:57:57.698961+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T14:57:57.932625+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T14:58:12.063 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-21T14:58:12.063 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T14:58:12.367 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-21T14:58:12.367 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":22,"fsid":"caf51e55-9234-4a26-bcd2-d60811962560","created":"2024-09-21T14:57:50.338822+0000","modified":"2024-09-21T14:58:11.332733+0000","last_up_change":"2024-09-21T14:58:01.177141+0000","last_in_change":"2024-09-21T14:57:52.442036+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T14:58:01.842810+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-21T14:58:08.284925+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":22,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a4a73316-3a94-42e9-9a51-deb46a66bb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":4016672243},{"type":"v1","addr":"172.21.15.32:6809","nonce":4016672243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":4016672243},{"type":"v1","addr":"172.21.15.32:6811","nonce":4016672243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":4016672243},{"type":"v1","addr":"172.21.15.32:6815","nonce":4016672243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":4016672243},{"type":"v1","addr":"172.21.15.32:6813","nonce":4016672243}]},"public_addr":"172.21.15.32:6809/4016672243","cluster_addr":"172.21.15.32:6811/4016672243","heartbeat_back_addr":"172.21.15.32:6815/4016672243","heartbeat_front_addr":"172.21.15.32:6813/4016672243","state":["exists","up"]},{"osd":1,"uuid":"006757a0-94d1-4a9b-aef8-73634cc5828e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":641090577},{"type":"v1","addr":"172.21.15.32:6825","nonce":641090577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":641090577},{"type":"v1","addr":"172.21.15.32:6827","nonce":641090577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":641090577},{"type":"v1","addr":"172.21.15.32:6831","nonce":641090577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":641090577},{"type":"v1","addr":"172.21.15.32:6829","nonce":641090577}]},"public_addr":"172.21.15.32:6825/641090577","cluster_addr":"172.21.15.32:6827/641090577","heartbeat_back_addr":"172.21.15.32:6831/641090577","heartbeat_front_addr":"172.21.15.32:6829/641090577","state":["exists","up"]},{"osd":2,"uuid":"0b499a23-820e-4e6c-a558-adbf88b7014f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2609154414},{"type":"v1","addr":"172.21.15.32:6801","nonce":2609154414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2609154414},{"type":"v1","addr":"172.21.15.32:6803","nonce":2609154414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2609154414},{"type":"v1","addr":"172.21.15.32:6807","nonce":2609154414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2609154414},{"type":"v1","addr":"172.21.15.32:6805","nonce":2609154414}]},"public_addr":"172.21.15.32:6801/2609154414","cluster_addr":"172.21.15.32:6803/2609154414","heartbeat_back_addr":"172.21.15.32:6807/2609154414","heartbeat_front_addr":"172.21.15.32:6805/2609154414","state":["exists","up"]},{"osd":3,"uuid":"5a57b3dd-4a07-45b7-b159-3cac4c7eb262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":1462563376},{"type":"v1","addr":"172.21.15.32:6817","nonce":1462563376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1462563376},{"type":"v1","addr":"172.21.15.32:6819","nonce":1462563376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1462563376},{"type":"v1","addr":"172.21.15.32:6823","nonce":1462563376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1462563376},{"type":"v1","addr":"172.21.15.32:6821","nonce":1462563376}]},"public_addr":"172.21.15.32:6817/1462563376","cluster_addr":"172.21.15.32:6819/1462563376","heartbeat_back_addr":"172.21.15.32:6823/1462563376","heartbeat_front_addr":"172.21.15.32:6821/1462563376","state":["exists","up"]},{"osd":4,"uuid":"39a00807-7517-474e-8842-623dd7093cc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6801","nonce":144412042},{"type":"v1","addr":"172.21.15.46:6803","nonce":144412042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6805","nonce":144412042},{"type":"v1","addr":"172.21.15.46:6807","nonce":144412042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":144412042},{"type":"v1","addr":"172.21.15.46:6814","nonce":144412042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":144412042},{"type":"v1","addr":"172.21.15.46:6811","nonce":144412042}]},"public_addr":"172.21.15.46:6803/144412042","cluster_addr":"172.21.15.46:6807/144412042","heartbeat_back_addr":"172.21.15.46:6814/144412042","heartbeat_front_addr":"172.21.15.46:6811/144412042","state":["exists","up"]},{"osd":5,"uuid":"edf9f045-ddeb-4504-a0fc-1a0e4e853550","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":849101442},{"type":"v1","addr":"172.21.15.46:6802","nonce":849101442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":849101442},{"type":"v1","addr":"172.21.15.46:6806","nonce":849101442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":849101442},{"type":"v1","addr":"172.21.15.46:6815","nonce":849101442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":849101442},{"type":"v1","addr":"172.21.15.46:6810","nonce":849101442}]},"public_addr":"172.21.15.46:6802/849101442","cluster_addr":"172.21.15.46:6806/849101442","heartbeat_back_addr":"172.21.15.46:6815/849101442","heartbeat_front_addr":"172.21.15.46:6810/849101442","state":["exists","up"]},{"osd":6,"uuid":"c820ea6b-88fe-4421-83ec-17c466691fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":847189172},{"type":"v1","addr":"172.21.15.46:6818","nonce":847189172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":847189172},{"type":"v1","addr":"172.21.15.46:6822","nonce":847189172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6827","nonce":847189172},{"type":"v1","addr":"172.21.15.46:6828","nonce":847189172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6823","nonce":847189172},{"type":"v1","addr":"172.21.15.46:6825","nonce":847189172}]},"public_addr":"172.21.15.46:6818/847189172","cluster_addr":"172.21.15.46:6822/847189172","heartbeat_back_addr":"172.21.15.46:6828/847189172","heartbeat_front_addr":"172.21.15.46:6825/847189172","state":["exists","up"]},{"osd":7,"uuid":"3a2f82a4-28a0-4fcb-9d6b-2285fa4d07ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3894963553},{"type":"v1","addr":"172.21.15.46:6819","nonce":3894963553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":3894963553},{"type":"v1","addr":"172.21.15.46:6824","nonce":3894963553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":3894963553},{"type":"v1","addr":"172.21.15.46:6831","nonce":3894963553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":3894963553},{"type":"v1","addr":"172.21.15.46:6829","nonce":3894963553}]},"public_addr":"172.21.15.46:6819/3894963553","cluster_addr":"172.21.15.46:6824/3894963553","heartbeat_back_addr":"172.21.15.46:6831/3894963553","heartbeat_front_addr":"172.21.15.46:6829/3894963553","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T14:57:57.458317+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T14:57:57.907624+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T14:57:57.278817+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T14:57:57.500449+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T14:57:57.504325+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T14:57:57.777525+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T14:57:57.698961+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T14:57:57.932625+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T14:58:12.384 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-21T14:58:12.384 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-21T14:58:12.384 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-21T14:58:12.385 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-21T14:58:12.385 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-21T14:58:12.385 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-21T14:58:12.386 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-21T14:58:12.386 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-21T14:58:12.592 INFO:teuthology.orchestra.run.smithi032.stdout:60129542148 2024-09-21T14:58:12.592 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-21T14:58:12.604 INFO:teuthology.orchestra.run.smithi032.stdout:64424509446 2024-09-21T14:58:12.604 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-21T14:58:12.610 INFO:teuthology.orchestra.run.smithi032.stdout:60129542149 2024-09-21T14:58:12.611 INFO:teuthology.orchestra.run.smithi032.stdout:60129542149 2024-09-21T14:58:12.611 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-21T14:58:12.612 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-21T14:58:12.615 INFO:teuthology.orchestra.run.smithi032.stdout:60129542148 2024-09-21T14:58:12.615 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-21T14:58:12.617 INFO:teuthology.orchestra.run.smithi032.stdout:60129542150 2024-09-21T14:58:12.617 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-21T14:58:12.619 INFO:teuthology.orchestra.run.smithi032.stdout:60129542150 2024-09-21T14:58:12.619 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-21T14:58:12.821 INFO:teuthology.orchestra.run.smithi032.stdout:60129542149 2024-09-21T14:58:12.822 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-21T14:58:13.081 INFO:teuthology.orchestra.run.smithi032.stdout:60129542147 2024-09-21T14:58:13.090 INFO:teuthology.orchestra.run.smithi032.stdout:60129542147 2024-09-21T14:58:13.090 INFO:teuthology.orchestra.run.smithi032.stdout:60129542148 2024-09-21T14:58:13.097 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542147 for osd.0 2024-09-21T14:58:13.100 INFO:teuthology.orchestra.run.smithi032.stdout:60129542148 2024-09-21T14:58:13.107 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542148 for osd.4 2024-09-21T14:58:13.108 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542147 for osd.2 2024-09-21T14:58:13.118 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542148 for osd.1 2024-09-21T14:58:13.118 INFO:teuthology.orchestra.run.smithi032.stdout:64424509444 2024-09-21T14:58:13.131 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509444 for osd.7 2024-09-21T14:58:13.161 INFO:teuthology.orchestra.run.smithi032.stdout:60129542149 2024-09-21T14:58:13.164 INFO:teuthology.orchestra.run.smithi032.stdout:60129542149 2024-09-21T14:58:13.164 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542149 for osd.5 2024-09-21T14:58:13.165 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542149 for osd.3 2024-09-21T14:58:13.221 INFO:teuthology.orchestra.run.smithi032.stdout:60129542148 2024-09-21T14:58:13.239 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542148 for osd.6 2024-09-21T14:58:14.098 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-21T14:58:14.108 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-21T14:58:14.108 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-21T14:58:14.119 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-21T14:58:14.162 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-21T14:58:14.165 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-21T14:58:14.165 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-21T14:58:14.239 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-21T14:58:14.588 INFO:teuthology.orchestra.run.smithi032.stdout:60129542148 2024-09-21T14:58:14.599 INFO:teuthology.orchestra.run.smithi032.stdout:60129542148 2024-09-21T14:58:14.606 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542148 for osd.0 2024-09-21T14:58:14.606 DEBUG:teuthology.parallel:result is None 2024-09-21T14:58:14.617 INFO:teuthology.orchestra.run.smithi032.stdout:60129542149 2024-09-21T14:58:14.618 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542148 for osd.2 2024-09-21T14:58:14.618 DEBUG:teuthology.parallel:result is None 2024-09-21T14:58:14.631 INFO:teuthology.orchestra.run.smithi032.stdout:60129542150 2024-09-21T14:58:14.634 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.4 2024-09-21T14:58:14.634 DEBUG:teuthology.parallel:result is None 2024-09-21T14:58:14.644 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.5 2024-09-21T14:58:14.644 DEBUG:teuthology.parallel:result is None 2024-09-21T14:58:14.656 INFO:teuthology.orchestra.run.smithi032.stdout:60129542150 2024-09-21T14:58:14.669 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.3 2024-09-21T14:58:14.669 DEBUG:teuthology.parallel:result is None 2024-09-21T14:58:14.679 INFO:teuthology.orchestra.run.smithi032.stdout:60129542149 2024-09-21T14:58:14.680 INFO:teuthology.orchestra.run.smithi032.stdout:60129542149 2024-09-21T14:58:14.691 INFO:teuthology.orchestra.run.smithi032.stdout:64424509446 2024-09-21T14:58:14.696 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.1 2024-09-21T14:58:14.696 DEBUG:teuthology.parallel:result is None 2024-09-21T14:58:14.697 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.6 2024-09-21T14:58:14.697 DEBUG:teuthology.parallel:result is None 2024-09-21T14:58:14.705 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.7 2024-09-21T14:58:14.705 DEBUG:teuthology.parallel:result is None 2024-09-21T14:58:14.705 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T14:58:14.705 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T14:58:15.000 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-21T14:58:15.003 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-09-21T14:58:15.014 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-09-21T14:58:13.777393+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":250836,"kb_used_data":1880,"kb_used_omap":12,"kb_used_meta":248819,"kb_avail":754723884,"statfs":{"total":773094113280,"available":772837257216,"internally_reserved":0,"allocated":1925120,"data_stored":1792486,"data_compressed":10404,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":12744,"internal_metadata":254791224},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.462601"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T14:58:11.347673+0000","last_change":"2024-09-21T14:58:11.348177+0000","last_active":"2024-09-21T14:58:11.347673+0000","last_peered":"2024-09-21T14:58:11.347673+0000","last_clean":"2024-09-21T14:58:11.347673+0000","last_became_active":"2024-09-21T14:58:09.328417+0000","last_became_peered":"2024-09-21T14:58:09.328417+0000","last_unstale":"2024-09-21T14:58:11.347673+0000","last_undegraded":"2024-09-21T14:58:11.347673+0000","last_fullsized":"2024-09-21T14:58:11.347673+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:52:22.671185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018917490000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T14:58:11.404838+0000","last_change":"2024-09-21T14:58:11.405717+0000","last_active":"2024-09-21T14:58:11.404838+0000","last_peered":"2024-09-21T14:58:11.404838+0000","last_clean":"2024-09-21T14:58:11.404838+0000","last_became_active":"2024-09-21T14:58:09.326849+0000","last_became_peered":"2024-09-21T14:58:09.326849+0000","last_unstale":"2024-09-21T14:58:11.404838+0000","last_undegraded":"2024-09-21T14:58:11.404838+0000","last_fullsized":"2024-09-21T14:58:11.404838+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:29:42.893819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012560399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T14:58:11.347781+0000","last_change":"2024-09-21T14:58:11.348182+0000","last_active":"2024-09-21T14:58:11.347781+0000","last_peered":"2024-09-21T14:58:11.347781+0000","last_clean":"2024-09-21T14:58:11.347781+0000","last_became_active":"2024-09-21T14:58:09.329036+0000","last_became_peered":"2024-09-21T14:58:09.329036+0000","last_unstale":"2024-09-21T14:58:11.347781+0000","last_undegraded":"2024-09-21T14:58:11.347781+0000","last_fullsized":"2024-09-21T14:58:11.347781+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:24:45.719970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019338770000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T14:58:11.405492+0000","last_change":"2024-09-21T14:58:11.406176+0000","last_active":"2024-09-21T14:58:11.405492+0000","last_peered":"2024-09-21T14:58:11.405492+0000","last_clean":"2024-09-21T14:58:11.405492+0000","last_became_active":"2024-09-21T14:58:09.777465+0000","last_became_peered":"2024-09-21T14:58:09.777465+0000","last_unstale":"2024-09-21T14:58:11.405492+0000","last_undegraded":"2024-09-21T14:58:11.405492+0000","last_fullsized":"2024-09-21T14:58:11.405492+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:55:04.667816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095704000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T14:58:11.347504+0000","last_change":"2024-09-21T14:58:11.347504+0000","last_active":"2024-09-21T14:58:11.347504+0000","last_peered":"2024-09-21T14:58:11.347504+0000","last_clean":"2024-09-21T14:58:11.347504+0000","last_became_active":"2024-09-21T14:58:09.775981+0000","last_became_peered":"2024-09-21T14:58:09.775981+0000","last_unstale":"2024-09-21T14:58:11.347504+0000","last_undegraded":"2024-09-21T14:58:11.347504+0000","last_fullsized":"2024-09-21T14:58:11.347504+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:47:16.286652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012568880000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T14:58:11.342210+0000","last_change":"2024-09-21T14:58:11.342210+0000","last_active":"2024-09-21T14:58:11.342210+0000","last_peered":"2024-09-21T14:58:11.342210+0000","last_clean":"2024-09-21T14:58:11.342210+0000","last_became_active":"2024-09-21T14:58:09.330624+0000","last_became_peered":"2024-09-21T14:58:09.330624+0000","last_unstale":"2024-09-21T14:58:11.342210+0000","last_undegraded":"2024-09-21T14:58:11.342210+0000","last_fullsized":"2024-09-21T14:58:11.342210+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:29:43.229538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001051151,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T14:58:11.346402+0000","last_change":"2024-09-21T14:58:11.346766+0000","last_active":"2024-09-21T14:58:11.346402+0000","last_peered":"2024-09-21T14:58:11.346402+0000","last_clean":"2024-09-21T14:58:11.346402+0000","last_became_active":"2024-09-21T14:58:09.329748+0000","last_became_peered":"2024-09-21T14:58:09.329748+0000","last_unstale":"2024-09-21T14:58:11.346402+0000","last_undegraded":"2024-09-21T14:58:11.346402+0000","last_fullsized":"2024-09-21T14:58:11.346402+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:31:20.267675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052698000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T14:58:11.347268+0000","last_change":"2024-09-21T14:58:11.347700+0000","last_active":"2024-09-21T14:58:11.347268+0000","last_peered":"2024-09-21T14:58:11.347268+0000","last_clean":"2024-09-21T14:58:11.347268+0000","last_became_active":"2024-09-21T14:58:09.438842+0000","last_became_peered":"2024-09-21T14:58:09.438842+0000","last_unstale":"2024-09-21T14:58:11.347268+0000","last_undegraded":"2024-09-21T14:58:11.347268+0000","last_fullsized":"2024-09-21T14:58:11.347268+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:41:34.626249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001426194,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"18'117","reported_seq":164,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T14:58:11.088945+0000","last_change":"2024-09-21T14:58:03.616055+0000","last_active":"2024-09-21T14:58:11.088945+0000","last_peered":"2024-09-21T14:58:11.088945+0000","last_clean":"2024-09-21T14:58:11.088945+0000","last_became_active":"2024-09-21T14:58:03.613214+0000","last_became_peered":"2024-09-21T14:58:03.613214+0000","last_unstale":"2024-09-21T14:58:11.088945+0000","last_undegraded":"2024-09-21T14:58:11.088945+0000","last_fullsized":"2024-09-21T14:58:11.088945+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:02.180162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:02.180162+0000","last_clean_scrub_stamp":"2024-09-21T14:58:02.180162+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:29:05.860361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":10404,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509446,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31344,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340496,"statfs":{"total":96636764160,"available":96604667904,"internally_reserved":0,"allocated":225280,"data_stored":76464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0269999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79900000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2849999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92200000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2410000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542149,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31340,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340500,"statfs":{"total":96636764160,"available":96604672000,"internally_reserved":0,"allocated":229376,"data_stored":76483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90600000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.054}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0229999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.141}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.401}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4319999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542150,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":282624,"data_stored":666832,"data_compressed":5202,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70799999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80400000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1910000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91200000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.139}]}]},{"osd":4,"up_from":14,"seq":60129542149,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31336,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340504,"statfs":{"total":96636764160,"available":96604676096,"internally_reserved":0,"allocated":225280,"data_stored":76464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.286}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97999999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1659999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.196}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3120000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542150,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31340,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340500,"statfs":{"total":96636764160,"available":96604672000,"internally_reserved":0,"allocated":229376,"data_stored":76483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69599999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0680000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1839999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1100000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542148,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31340,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340500,"statfs":{"total":96636764160,"available":96604672000,"internally_reserved":0,"allocated":225280,"data_stored":76464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0129999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.871}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72899999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.413}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.206}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3400000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3009999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542149,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":282624,"data_stored":666832,"data_compressed":5202,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31848901},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.751}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80200000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93899999999999995}]}]},{"osd":0,"up_from":14,"seq":60129542148,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31336,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340504,"statfs":{"total":96636764160,"available":96604676096,"internally_reserved":0,"allocated":225280,"data_stored":76464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81299999999999994}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91100000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3149999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.012}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0429999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5202,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5202,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T14:58:15.015 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T14:58:15.304 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-21T14:58:15.306 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-09-21T14:58:15.317 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-21T14:58:15.091509+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":250836,"kb_used_data":1880,"kb_used_omap":12,"kb_used_meta":248819,"kb_avail":754723884,"statfs":{"total":773094113280,"available":772837257216,"internally_reserved":0,"allocated":1925120,"data_stored":1792486,"data_compressed":10404,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":12744,"internal_metadata":254791224},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.773796"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T14:58:11.347673+0000","last_change":"2024-09-21T14:58:11.348177+0000","last_active":"2024-09-21T14:58:11.347673+0000","last_peered":"2024-09-21T14:58:11.347673+0000","last_clean":"2024-09-21T14:58:11.347673+0000","last_became_active":"2024-09-21T14:58:09.328417+0000","last_became_peered":"2024-09-21T14:58:09.328417+0000","last_unstale":"2024-09-21T14:58:11.347673+0000","last_undegraded":"2024-09-21T14:58:11.347673+0000","last_fullsized":"2024-09-21T14:58:11.347673+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:52:22.671185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018917490000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T14:58:11.404838+0000","last_change":"2024-09-21T14:58:11.405717+0000","last_active":"2024-09-21T14:58:11.404838+0000","last_peered":"2024-09-21T14:58:11.404838+0000","last_clean":"2024-09-21T14:58:11.404838+0000","last_became_active":"2024-09-21T14:58:09.326849+0000","last_became_peered":"2024-09-21T14:58:09.326849+0000","last_unstale":"2024-09-21T14:58:11.404838+0000","last_undegraded":"2024-09-21T14:58:11.404838+0000","last_fullsized":"2024-09-21T14:58:11.404838+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:29:42.893819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012560399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T14:58:11.347781+0000","last_change":"2024-09-21T14:58:11.348182+0000","last_active":"2024-09-21T14:58:11.347781+0000","last_peered":"2024-09-21T14:58:11.347781+0000","last_clean":"2024-09-21T14:58:11.347781+0000","last_became_active":"2024-09-21T14:58:09.329036+0000","last_became_peered":"2024-09-21T14:58:09.329036+0000","last_unstale":"2024-09-21T14:58:11.347781+0000","last_undegraded":"2024-09-21T14:58:11.347781+0000","last_fullsized":"2024-09-21T14:58:11.347781+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:24:45.719970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019338770000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T14:58:11.405492+0000","last_change":"2024-09-21T14:58:11.406176+0000","last_active":"2024-09-21T14:58:11.405492+0000","last_peered":"2024-09-21T14:58:11.405492+0000","last_clean":"2024-09-21T14:58:11.405492+0000","last_became_active":"2024-09-21T14:58:09.777465+0000","last_became_peered":"2024-09-21T14:58:09.777465+0000","last_unstale":"2024-09-21T14:58:11.405492+0000","last_undegraded":"2024-09-21T14:58:11.405492+0000","last_fullsized":"2024-09-21T14:58:11.405492+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:55:04.667816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095704000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T14:58:11.347504+0000","last_change":"2024-09-21T14:58:11.347504+0000","last_active":"2024-09-21T14:58:11.347504+0000","last_peered":"2024-09-21T14:58:11.347504+0000","last_clean":"2024-09-21T14:58:11.347504+0000","last_became_active":"2024-09-21T14:58:09.775981+0000","last_became_peered":"2024-09-21T14:58:09.775981+0000","last_unstale":"2024-09-21T14:58:11.347504+0000","last_undegraded":"2024-09-21T14:58:11.347504+0000","last_fullsized":"2024-09-21T14:58:11.347504+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:47:16.286652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012568880000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T14:58:11.342210+0000","last_change":"2024-09-21T14:58:11.342210+0000","last_active":"2024-09-21T14:58:11.342210+0000","last_peered":"2024-09-21T14:58:11.342210+0000","last_clean":"2024-09-21T14:58:11.342210+0000","last_became_active":"2024-09-21T14:58:09.330624+0000","last_became_peered":"2024-09-21T14:58:09.330624+0000","last_unstale":"2024-09-21T14:58:11.342210+0000","last_undegraded":"2024-09-21T14:58:11.342210+0000","last_fullsized":"2024-09-21T14:58:11.342210+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:29:43.229538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001051151,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T14:58:11.346402+0000","last_change":"2024-09-21T14:58:11.346766+0000","last_active":"2024-09-21T14:58:11.346402+0000","last_peered":"2024-09-21T14:58:11.346402+0000","last_clean":"2024-09-21T14:58:11.346402+0000","last_became_active":"2024-09-21T14:58:09.329748+0000","last_became_peered":"2024-09-21T14:58:09.329748+0000","last_unstale":"2024-09-21T14:58:11.346402+0000","last_undegraded":"2024-09-21T14:58:11.346402+0000","last_fullsized":"2024-09-21T14:58:11.346402+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:31:20.267675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052698000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T14:58:11.347268+0000","last_change":"2024-09-21T14:58:11.347700+0000","last_active":"2024-09-21T14:58:11.347268+0000","last_peered":"2024-09-21T14:58:11.347268+0000","last_clean":"2024-09-21T14:58:11.347268+0000","last_became_active":"2024-09-21T14:58:09.438842+0000","last_became_peered":"2024-09-21T14:58:09.438842+0000","last_unstale":"2024-09-21T14:58:11.347268+0000","last_undegraded":"2024-09-21T14:58:11.347268+0000","last_fullsized":"2024-09-21T14:58:11.347268+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:41:34.626249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001426194,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"18'117","reported_seq":164,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T14:58:11.088945+0000","last_change":"2024-09-21T14:58:03.616055+0000","last_active":"2024-09-21T14:58:11.088945+0000","last_peered":"2024-09-21T14:58:11.088945+0000","last_clean":"2024-09-21T14:58:11.088945+0000","last_became_active":"2024-09-21T14:58:03.613214+0000","last_became_peered":"2024-09-21T14:58:03.613214+0000","last_unstale":"2024-09-21T14:58:11.088945+0000","last_undegraded":"2024-09-21T14:58:11.088945+0000","last_fullsized":"2024-09-21T14:58:11.088945+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:02.180162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:02.180162+0000","last_clean_scrub_stamp":"2024-09-21T14:58:02.180162+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:29:05.860361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":10404,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509446,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31344,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340496,"statfs":{"total":96636764160,"available":96604667904,"internally_reserved":0,"allocated":225280,"data_stored":76464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0269999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79900000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2849999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92200000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2410000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542149,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31340,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340500,"statfs":{"total":96636764160,"available":96604672000,"internally_reserved":0,"allocated":229376,"data_stored":76483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90600000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.054}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0229999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.141}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.401}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4319999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542150,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":282624,"data_stored":666832,"data_compressed":5202,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70799999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80400000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1910000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91200000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.139}]}]},{"osd":4,"up_from":14,"seq":60129542149,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31336,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340504,"statfs":{"total":96636764160,"available":96604676096,"internally_reserved":0,"allocated":225280,"data_stored":76464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.286}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97999999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1659999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.196}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3120000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542150,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31340,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340500,"statfs":{"total":96636764160,"available":96604672000,"internally_reserved":0,"allocated":229376,"data_stored":76483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69599999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0680000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1839999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1100000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542148,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31340,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340500,"statfs":{"total":96636764160,"available":96604672000,"internally_reserved":0,"allocated":225280,"data_stored":76464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0129999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.871}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72899999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.413}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.206}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3400000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3009999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542149,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":282624,"data_stored":666832,"data_compressed":5202,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31848901},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.751}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80200000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93899999999999995}]}]},{"osd":0,"up_from":14,"seq":60129542148,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31336,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340504,"statfs":{"total":96636764160,"available":96604676096,"internally_reserved":0,"allocated":225280,"data_stored":76464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81299999999999994}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91100000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3149999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.012}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0429999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5202,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5202,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T14:58:15.318 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T14:58:15.318 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-21T14:58:15.318 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-21T14:58:15.318 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-21T14:58:15.637 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-21T14:58:15.637 INFO:teuthology.orchestra.run.smithi032.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-21T14:58:15.650 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-21T14:58:15.650 INFO:teuthology.run_tasks:Running task thrashosds... 2024-09-21T14:58:15.666 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-21T14:58:15.666 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'reef'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from'], 'sha1': '19e2a2beb6f41a3ab62650e38614eb29b59dcbb9'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '19e2a2beb6f41a3ab62650e38614eb29b59dcbb9'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'reef', 'sha1': 'a477b1894d5c5a613f23260ca0e6b96d79652ba7'}} 2024-09-21T14:58:15.667 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2024-09-21T14:58:15.667 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-09-21T14:58:15.995 INFO:tasks.thrashosds:Beginning thrashosds... 2024-09-21T14:58:15.995 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T14:58:15.995 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T14:58:16.294 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-21T14:58:16.297 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-09-21T14:58:16.308 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-09-21T14:58:15.778001+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":250836,"kb_used_data":1880,"kb_used_omap":12,"kb_used_meta":248819,"kb_avail":754723884,"statfs":{"total":773094113280,"available":772837257216,"internally_reserved":0,"allocated":1925120,"data_stored":1792486,"data_compressed":10404,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":12744,"internal_metadata":254791224},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001808"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T14:58:11.347673+0000","last_change":"2024-09-21T14:58:11.348177+0000","last_active":"2024-09-21T14:58:11.347673+0000","last_peered":"2024-09-21T14:58:11.347673+0000","last_clean":"2024-09-21T14:58:11.347673+0000","last_became_active":"2024-09-21T14:58:09.328417+0000","last_became_peered":"2024-09-21T14:58:09.328417+0000","last_unstale":"2024-09-21T14:58:11.347673+0000","last_undegraded":"2024-09-21T14:58:11.347673+0000","last_fullsized":"2024-09-21T14:58:11.347673+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:52:22.671185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018917490000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T14:58:11.404838+0000","last_change":"2024-09-21T14:58:11.405717+0000","last_active":"2024-09-21T14:58:11.404838+0000","last_peered":"2024-09-21T14:58:11.404838+0000","last_clean":"2024-09-21T14:58:11.404838+0000","last_became_active":"2024-09-21T14:58:09.326849+0000","last_became_peered":"2024-09-21T14:58:09.326849+0000","last_unstale":"2024-09-21T14:58:11.404838+0000","last_undegraded":"2024-09-21T14:58:11.404838+0000","last_fullsized":"2024-09-21T14:58:11.404838+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:29:42.893819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012560399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T14:58:11.347781+0000","last_change":"2024-09-21T14:58:11.348182+0000","last_active":"2024-09-21T14:58:11.347781+0000","last_peered":"2024-09-21T14:58:11.347781+0000","last_clean":"2024-09-21T14:58:11.347781+0000","last_became_active":"2024-09-21T14:58:09.329036+0000","last_became_peered":"2024-09-21T14:58:09.329036+0000","last_unstale":"2024-09-21T14:58:11.347781+0000","last_undegraded":"2024-09-21T14:58:11.347781+0000","last_fullsized":"2024-09-21T14:58:11.347781+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:24:45.719970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019338770000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T14:58:11.405492+0000","last_change":"2024-09-21T14:58:11.406176+0000","last_active":"2024-09-21T14:58:11.405492+0000","last_peered":"2024-09-21T14:58:11.405492+0000","last_clean":"2024-09-21T14:58:11.405492+0000","last_became_active":"2024-09-21T14:58:09.777465+0000","last_became_peered":"2024-09-21T14:58:09.777465+0000","last_unstale":"2024-09-21T14:58:11.405492+0000","last_undegraded":"2024-09-21T14:58:11.405492+0000","last_fullsized":"2024-09-21T14:58:11.405492+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:55:04.667816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095704000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T14:58:11.347504+0000","last_change":"2024-09-21T14:58:11.347504+0000","last_active":"2024-09-21T14:58:11.347504+0000","last_peered":"2024-09-21T14:58:11.347504+0000","last_clean":"2024-09-21T14:58:11.347504+0000","last_became_active":"2024-09-21T14:58:09.775981+0000","last_became_peered":"2024-09-21T14:58:09.775981+0000","last_unstale":"2024-09-21T14:58:11.347504+0000","last_undegraded":"2024-09-21T14:58:11.347504+0000","last_fullsized":"2024-09-21T14:58:11.347504+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:47:16.286652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012568880000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T14:58:11.342210+0000","last_change":"2024-09-21T14:58:11.342210+0000","last_active":"2024-09-21T14:58:11.342210+0000","last_peered":"2024-09-21T14:58:11.342210+0000","last_clean":"2024-09-21T14:58:11.342210+0000","last_became_active":"2024-09-21T14:58:09.330624+0000","last_became_peered":"2024-09-21T14:58:09.330624+0000","last_unstale":"2024-09-21T14:58:11.342210+0000","last_undegraded":"2024-09-21T14:58:11.342210+0000","last_fullsized":"2024-09-21T14:58:11.342210+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:29:43.229538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001051151,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T14:58:11.346402+0000","last_change":"2024-09-21T14:58:11.346766+0000","last_active":"2024-09-21T14:58:11.346402+0000","last_peered":"2024-09-21T14:58:11.346402+0000","last_clean":"2024-09-21T14:58:11.346402+0000","last_became_active":"2024-09-21T14:58:09.329748+0000","last_became_peered":"2024-09-21T14:58:09.329748+0000","last_unstale":"2024-09-21T14:58:11.346402+0000","last_undegraded":"2024-09-21T14:58:11.346402+0000","last_fullsized":"2024-09-21T14:58:11.346402+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:31:20.267675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052698000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T14:58:11.347268+0000","last_change":"2024-09-21T14:58:11.347700+0000","last_active":"2024-09-21T14:58:11.347268+0000","last_peered":"2024-09-21T14:58:11.347268+0000","last_clean":"2024-09-21T14:58:11.347268+0000","last_became_active":"2024-09-21T14:58:09.438842+0000","last_became_peered":"2024-09-21T14:58:09.438842+0000","last_unstale":"2024-09-21T14:58:11.347268+0000","last_undegraded":"2024-09-21T14:58:11.347268+0000","last_fullsized":"2024-09-21T14:58:11.347268+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:41:34.626249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001426194,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"18'117","reported_seq":164,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T14:58:11.088945+0000","last_change":"2024-09-21T14:58:03.616055+0000","last_active":"2024-09-21T14:58:11.088945+0000","last_peered":"2024-09-21T14:58:11.088945+0000","last_clean":"2024-09-21T14:58:11.088945+0000","last_became_active":"2024-09-21T14:58:03.613214+0000","last_became_peered":"2024-09-21T14:58:03.613214+0000","last_unstale":"2024-09-21T14:58:11.088945+0000","last_undegraded":"2024-09-21T14:58:11.088945+0000","last_fullsized":"2024-09-21T14:58:11.088945+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:02.180162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:02.180162+0000","last_clean_scrub_stamp":"2024-09-21T14:58:02.180162+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:29:05.860361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":10404,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509446,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31344,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340496,"statfs":{"total":96636764160,"available":96604667904,"internally_reserved":0,"allocated":225280,"data_stored":76464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0269999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79900000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2849999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92200000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2410000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542149,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31340,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340500,"statfs":{"total":96636764160,"available":96604672000,"internally_reserved":0,"allocated":229376,"data_stored":76483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90600000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.054}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0229999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.141}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.401}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4319999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542150,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":282624,"data_stored":666832,"data_compressed":5202,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70799999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80400000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1910000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91200000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.139}]}]},{"osd":4,"up_from":14,"seq":60129542149,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31336,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340504,"statfs":{"total":96636764160,"available":96604676096,"internally_reserved":0,"allocated":225280,"data_stored":76464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.286}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97999999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1659999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.196}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3120000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542150,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31340,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340500,"statfs":{"total":96636764160,"available":96604672000,"internally_reserved":0,"allocated":229376,"data_stored":76483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69599999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0680000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1839999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1100000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542148,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31340,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340500,"statfs":{"total":96636764160,"available":96604672000,"internally_reserved":0,"allocated":225280,"data_stored":76464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0129999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.871}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72899999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.413}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.206}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3400000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3009999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542149,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":282624,"data_stored":666832,"data_compressed":5202,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31848901},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.751}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80200000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93899999999999995}]}]},{"osd":0,"up_from":14,"seq":60129542148,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31336,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340504,"statfs":{"total":96636764160,"available":96604676096,"internally_reserved":0,"allocated":225280,"data_stored":76464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81299999999999994}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91100000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3149999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.012}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0429999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5202,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5202,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T14:58:16.309 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T14:58:16.618 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-21T14:58:16.621 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-09-21T14:58:16.633 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-09-21T14:58:15.778001+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":250836,"kb_used_data":1880,"kb_used_omap":12,"kb_used_meta":248819,"kb_avail":754723884,"statfs":{"total":773094113280,"available":772837257216,"internally_reserved":0,"allocated":1925120,"data_stored":1792486,"data_compressed":10404,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":12744,"internal_metadata":254791224},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001808"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T14:58:11.347673+0000","last_change":"2024-09-21T14:58:11.348177+0000","last_active":"2024-09-21T14:58:11.347673+0000","last_peered":"2024-09-21T14:58:11.347673+0000","last_clean":"2024-09-21T14:58:11.347673+0000","last_became_active":"2024-09-21T14:58:09.328417+0000","last_became_peered":"2024-09-21T14:58:09.328417+0000","last_unstale":"2024-09-21T14:58:11.347673+0000","last_undegraded":"2024-09-21T14:58:11.347673+0000","last_fullsized":"2024-09-21T14:58:11.347673+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:52:22.671185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018917490000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T14:58:11.404838+0000","last_change":"2024-09-21T14:58:11.405717+0000","last_active":"2024-09-21T14:58:11.404838+0000","last_peered":"2024-09-21T14:58:11.404838+0000","last_clean":"2024-09-21T14:58:11.404838+0000","last_became_active":"2024-09-21T14:58:09.326849+0000","last_became_peered":"2024-09-21T14:58:09.326849+0000","last_unstale":"2024-09-21T14:58:11.404838+0000","last_undegraded":"2024-09-21T14:58:11.404838+0000","last_fullsized":"2024-09-21T14:58:11.404838+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:29:42.893819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012560399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T14:58:11.347781+0000","last_change":"2024-09-21T14:58:11.348182+0000","last_active":"2024-09-21T14:58:11.347781+0000","last_peered":"2024-09-21T14:58:11.347781+0000","last_clean":"2024-09-21T14:58:11.347781+0000","last_became_active":"2024-09-21T14:58:09.329036+0000","last_became_peered":"2024-09-21T14:58:09.329036+0000","last_unstale":"2024-09-21T14:58:11.347781+0000","last_undegraded":"2024-09-21T14:58:11.347781+0000","last_fullsized":"2024-09-21T14:58:11.347781+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:24:45.719970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019338770000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T14:58:11.405492+0000","last_change":"2024-09-21T14:58:11.406176+0000","last_active":"2024-09-21T14:58:11.405492+0000","last_peered":"2024-09-21T14:58:11.405492+0000","last_clean":"2024-09-21T14:58:11.405492+0000","last_became_active":"2024-09-21T14:58:09.777465+0000","last_became_peered":"2024-09-21T14:58:09.777465+0000","last_unstale":"2024-09-21T14:58:11.405492+0000","last_undegraded":"2024-09-21T14:58:11.405492+0000","last_fullsized":"2024-09-21T14:58:11.405492+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:55:04.667816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095704000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T14:58:11.347504+0000","last_change":"2024-09-21T14:58:11.347504+0000","last_active":"2024-09-21T14:58:11.347504+0000","last_peered":"2024-09-21T14:58:11.347504+0000","last_clean":"2024-09-21T14:58:11.347504+0000","last_became_active":"2024-09-21T14:58:09.775981+0000","last_became_peered":"2024-09-21T14:58:09.775981+0000","last_unstale":"2024-09-21T14:58:11.347504+0000","last_undegraded":"2024-09-21T14:58:11.347504+0000","last_fullsized":"2024-09-21T14:58:11.347504+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:47:16.286652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012568880000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T14:58:11.342210+0000","last_change":"2024-09-21T14:58:11.342210+0000","last_active":"2024-09-21T14:58:11.342210+0000","last_peered":"2024-09-21T14:58:11.342210+0000","last_clean":"2024-09-21T14:58:11.342210+0000","last_became_active":"2024-09-21T14:58:09.330624+0000","last_became_peered":"2024-09-21T14:58:09.330624+0000","last_unstale":"2024-09-21T14:58:11.342210+0000","last_undegraded":"2024-09-21T14:58:11.342210+0000","last_fullsized":"2024-09-21T14:58:11.342210+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:29:43.229538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001051151,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T14:58:11.346402+0000","last_change":"2024-09-21T14:58:11.346766+0000","last_active":"2024-09-21T14:58:11.346402+0000","last_peered":"2024-09-21T14:58:11.346402+0000","last_clean":"2024-09-21T14:58:11.346402+0000","last_became_active":"2024-09-21T14:58:09.329748+0000","last_became_peered":"2024-09-21T14:58:09.329748+0000","last_unstale":"2024-09-21T14:58:11.346402+0000","last_undegraded":"2024-09-21T14:58:11.346402+0000","last_fullsized":"2024-09-21T14:58:11.346402+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:31:20.267675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052698000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T14:58:11.347268+0000","last_change":"2024-09-21T14:58:11.347700+0000","last_active":"2024-09-21T14:58:11.347268+0000","last_peered":"2024-09-21T14:58:11.347268+0000","last_clean":"2024-09-21T14:58:11.347268+0000","last_became_active":"2024-09-21T14:58:09.438842+0000","last_became_peered":"2024-09-21T14:58:09.438842+0000","last_unstale":"2024-09-21T14:58:11.347268+0000","last_undegraded":"2024-09-21T14:58:11.347268+0000","last_fullsized":"2024-09-21T14:58:11.347268+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:41:34.626249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001426194,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"18'117","reported_seq":164,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T14:58:11.088945+0000","last_change":"2024-09-21T14:58:03.616055+0000","last_active":"2024-09-21T14:58:11.088945+0000","last_peered":"2024-09-21T14:58:11.088945+0000","last_clean":"2024-09-21T14:58:11.088945+0000","last_became_active":"2024-09-21T14:58:03.613214+0000","last_became_peered":"2024-09-21T14:58:03.613214+0000","last_unstale":"2024-09-21T14:58:11.088945+0000","last_undegraded":"2024-09-21T14:58:11.088945+0000","last_fullsized":"2024-09-21T14:58:11.088945+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:02.180162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:02.180162+0000","last_clean_scrub_stamp":"2024-09-21T14:58:02.180162+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:29:05.860361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":10404,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509446,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31344,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340496,"statfs":{"total":96636764160,"available":96604667904,"internally_reserved":0,"allocated":225280,"data_stored":76464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0269999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79900000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2849999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92200000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2410000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542149,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31340,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340500,"statfs":{"total":96636764160,"available":96604672000,"internally_reserved":0,"allocated":229376,"data_stored":76483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90600000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.054}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0229999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.141}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.401}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4319999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542150,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":282624,"data_stored":666832,"data_compressed":5202,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70799999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80400000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1910000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91200000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.139}]}]},{"osd":4,"up_from":14,"seq":60129542149,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31336,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340504,"statfs":{"total":96636764160,"available":96604676096,"internally_reserved":0,"allocated":225280,"data_stored":76464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.286}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97999999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1659999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.196}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3120000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542150,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31340,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340500,"statfs":{"total":96636764160,"available":96604672000,"internally_reserved":0,"allocated":229376,"data_stored":76483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69599999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0680000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1839999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1100000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542148,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31340,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340500,"statfs":{"total":96636764160,"available":96604672000,"internally_reserved":0,"allocated":225280,"data_stored":76464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0129999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.871}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72899999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.413}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.206}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3400000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3009999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542149,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":282624,"data_stored":666832,"data_compressed":5202,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31848901},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.751}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80200000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93899999999999995}]}]},{"osd":0,"up_from":14,"seq":60129542148,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31336,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340504,"statfs":{"total":96636764160,"available":96604676096,"internally_reserved":0,"allocated":225280,"data_stored":76464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81299999999999994}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91100000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3149999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.012}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0429999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5202,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5202,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T14:58:16.634 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T14:58:16.635 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-09-21T14:58:16.939 INFO:teuthology.orchestra.run.smithi032.stdout:epoch 23 2024-09-21T14:58:16.939 INFO:teuthology.orchestra.run.smithi032.stdout:fsid caf51e55-9234-4a26-bcd2-d60811962560 2024-09-21T14:58:16.940 INFO:teuthology.orchestra.run.smithi032.stdout:created 2024-09-21T14:57:50.338822+0000 2024-09-21T14:58:16.940 INFO:teuthology.orchestra.run.smithi032.stdout:modified 2024-09-21T14:58:15.087944+0000 2024-09-21T14:58:16.940 INFO:teuthology.orchestra.run.smithi032.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-21T14:58:16.940 INFO:teuthology.orchestra.run.smithi032.stdout:crush_version 5 2024-09-21T14:58:16.940 INFO:teuthology.orchestra.run.smithi032.stdout:full_ratio 0.95 2024-09-21T14:58:16.940 INFO:teuthology.orchestra.run.smithi032.stdout:backfillfull_ratio 0.9 2024-09-21T14:58:16.940 INFO:teuthology.orchestra.run.smithi032.stdout:nearfull_ratio 0.85 2024-09-21T14:58:16.940 INFO:teuthology.orchestra.run.smithi032.stdout:require_min_compat_client luminous 2024-09-21T14:58:16.940 INFO:teuthology.orchestra.run.smithi032.stdout:min_compat_client jewel 2024-09-21T14:58:16.940 INFO:teuthology.orchestra.run.smithi032.stdout:require_osd_release reef 2024-09-21T14:58:16.940 INFO:teuthology.orchestra.run.smithi032.stdout:stretch_mode_enabled false 2024-09-21T14:58:16.940 INFO:teuthology.orchestra.run.smithi032.stdout:pool 1 '.mgr' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 18 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 8.00 2024-09-21T14:58:16.941 INFO:teuthology.orchestra.run.smithi032.stdout:pool 2 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 22 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd read_balance_score 4.98 2024-09-21T14:58:16.941 INFO:teuthology.orchestra.run.smithi032.stdout:max_osd 8 2024-09-21T14:58:16.941 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.32:6808/4016672243,v1:172.21.15.32:6809/4016672243] [v2:172.21.15.32:6810/4016672243,v1:172.21.15.32:6811/4016672243] exists,up a4a73316-3a94-42e9-9a51-deb46a66bb06 2024-09-21T14:58:16.941 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1 up in weight 1 up_from 14 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.32:6824/641090577,v1:172.21.15.32:6825/641090577] [v2:172.21.15.32:6826/641090577,v1:172.21.15.32:6827/641090577] exists,up 006757a0-94d1-4a9b-aef8-73634cc5828e 2024-09-21T14:58:16.941 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.32:6800/2609154414,v1:172.21.15.32:6801/2609154414] [v2:172.21.15.32:6802/2609154414,v1:172.21.15.32:6803/2609154414] exists,up 0b499a23-820e-4e6c-a558-adbf88b7014f 2024-09-21T14:58:16.941 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3 up in weight 1 up_from 14 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.32:6816/1462563376,v1:172.21.15.32:6817/1462563376] [v2:172.21.15.32:6818/1462563376,v1:172.21.15.32:6819/1462563376] exists,up 5a57b3dd-4a07-45b7-b159-3cac4c7eb262 2024-09-21T14:58:16.941 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4 up in weight 1 up_from 14 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.46:6801/144412042,v1:172.21.15.46:6803/144412042] [v2:172.21.15.46:6805/144412042,v1:172.21.15.46:6807/144412042] exists,up 39a00807-7517-474e-8842-623dd7093cc5 2024-09-21T14:58:16.941 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5 up in weight 1 up_from 14 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.46:6800/849101442,v1:172.21.15.46:6802/849101442] [v2:172.21.15.46:6804/849101442,v1:172.21.15.46:6806/849101442] exists,up edf9f045-ddeb-4504-a0fc-1a0e4e853550 2024-09-21T14:58:16.941 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.46:6817/847189172,v1:172.21.15.46:6818/847189172] [v2:172.21.15.46:6820/847189172,v1:172.21.15.46:6822/847189172] exists,up c820ea6b-88fe-4421-83ec-17c466691fea 2024-09-21T14:58:16.941 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7 up in weight 1 up_from 15 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.46:6816/3894963553,v1:172.21.15.46:6819/3894963553] [v2:172.21.15.46:6821/3894963553,v1:172.21.15.46:6824/3894963553] exists,up 3a2f82a4-28a0-4fcb-9d6b-2285fa4d07ac 2024-09-21T14:58:16.954 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.32:6808/4016672243,v1:172.21.15.32:6809/4016672243] [v2:172.21.15.32:6810/4016672243,v1:172.21.15.32:6811/4016672243] exists,up a4a73316-3a94-42e9-9a51-deb46a66bb06', 'osd.1 up in weight 1 up_from 14 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.32:6824/641090577,v1:172.21.15.32:6825/641090577] [v2:172.21.15.32:6826/641090577,v1:172.21.15.32:6827/641090577] exists,up 006757a0-94d1-4a9b-aef8-73634cc5828e', 'osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.32:6800/2609154414,v1:172.21.15.32:6801/2609154414] [v2:172.21.15.32:6802/2609154414,v1:172.21.15.32:6803/2609154414] exists,up 0b499a23-820e-4e6c-a558-adbf88b7014f', 'osd.3 up in weight 1 up_from 14 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.32:6816/1462563376,v1:172.21.15.32:6817/1462563376] [v2:172.21.15.32:6818/1462563376,v1:172.21.15.32:6819/1462563376] exists,up 5a57b3dd-4a07-45b7-b159-3cac4c7eb262', 'osd.4 up in weight 1 up_from 14 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.46:6801/144412042,v1:172.21.15.46:6803/144412042] [v2:172.21.15.46:6805/144412042,v1:172.21.15.46:6807/144412042] exists,up 39a00807-7517-474e-8842-623dd7093cc5', 'osd.5 up in weight 1 up_from 14 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.46:6800/849101442,v1:172.21.15.46:6802/849101442] [v2:172.21.15.46:6804/849101442,v1:172.21.15.46:6806/849101442] exists,up edf9f045-ddeb-4504-a0fc-1a0e4e853550', 'osd.6 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.46:6817/847189172,v1:172.21.15.46:6818/847189172] [v2:172.21.15.46:6820/847189172,v1:172.21.15.46:6822/847189172] exists,up c820ea6b-88fe-4421-83ec-17c466691fea', 'osd.7 up in weight 1 up_from 15 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.46:6816/3894963553,v1:172.21.15.46:6819/3894963553] [v2:172.21.15.46:6821/3894963553,v1:172.21.15.46:6824/3894963553] exists,up 3a2f82a4-28a0-4fcb-9d6b-2285fa4d07ac'] 2024-09-21T14:58:16.954 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2024-09-21T14:58:17.060 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:58:17.060 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "mon.a", 2024-09-21T14:58:17.060 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster": "ceph", 2024-09-21T14:58:17.060 INFO:teuthology.orchestra.run.smithi032.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-09-21T14:58:17.061 INFO:teuthology.orchestra.run.smithi032.stdout: "admin_socket_mode": "", 2024-09-21T14:58:17.061 INFO:teuthology.orchestra.run.smithi032.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2024-09-21T14:58:17.061 INFO:teuthology.orchestra.run.smithi032.stdout: "auth_client_required": "cephx, none", 2024-09-21T14:58:17.061 INFO:teuthology.orchestra.run.smithi032.stdout: "auth_cluster_required": "cephx", 2024-09-21T14:58:17.061 INFO:teuthology.orchestra.run.smithi032.stdout: "auth_debug": "true", 2024-09-21T14:58:17.061 INFO:teuthology.orchestra.run.smithi032.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-09-21T14:58:17.061 INFO:teuthology.orchestra.run.smithi032.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-09-21T14:58:17.061 INFO:teuthology.orchestra.run.smithi032.stdout: "auth_service_required": "cephx", 2024-09-21T14:58:17.061 INFO:teuthology.orchestra.run.smithi032.stdout: "auth_service_ticket_ttl": "240.000000", 2024-09-21T14:58:17.061 INFO:teuthology.orchestra.run.smithi032.stdout: "auth_supported": "cephx", 2024-09-21T14:58:17.061 INFO:teuthology.orchestra.run.smithi032.stdout: "bdev_aio": "true", 2024-09-21T14:58:17.061 INFO:teuthology.orchestra.run.smithi032.stdout: "bdev_aio_max_queue_depth": "1024", 2024-09-21T14:58:17.061 INFO:teuthology.orchestra.run.smithi032.stdout: "bdev_aio_poll_ms": "250", 2024-09-21T14:58:17.061 INFO:teuthology.orchestra.run.smithi032.stdout: "bdev_aio_reap_max": "16", 2024-09-21T14:58:17.062 INFO:teuthology.orchestra.run.smithi032.stdout: "bdev_async_discard": "false", 2024-09-21T14:58:17.062 INFO:teuthology.orchestra.run.smithi032.stdout: "bdev_block_size": "4096", 2024-09-21T14:58:17.062 INFO:teuthology.orchestra.run.smithi032.stdout: "bdev_debug_aio": "false", 2024-09-21T14:58:17.062 INFO:teuthology.orchestra.run.smithi032.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-09-21T14:58:17.062 INFO:teuthology.orchestra.run.smithi032.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-09-21T14:58:17.062 INFO:teuthology.orchestra.run.smithi032.stdout: "bdev_debug_inflight_ios": "false", 2024-09-21T14:58:17.062 INFO:teuthology.orchestra.run.smithi032.stdout: "bdev_enable_discard": "false", 2024-09-21T14:58:17.062 INFO:teuthology.orchestra.run.smithi032.stdout: "bdev_flock_retry": "3", 2024-09-21T14:58:17.062 INFO:teuthology.orchestra.run.smithi032.stdout: "bdev_flock_retry_interval": "0.100000", 2024-09-21T14:58:17.062 INFO:teuthology.orchestra.run.smithi032.stdout: "bdev_inject_crash": "0", 2024-09-21T14:58:17.062 INFO:teuthology.orchestra.run.smithi032.stdout: "bdev_inject_crash_flush_delay": "2", 2024-09-21T14:58:17.062 INFO:teuthology.orchestra.run.smithi032.stdout: "bdev_ioring": "false", 2024-09-21T14:58:17.062 INFO:teuthology.orchestra.run.smithi032.stdout: "bdev_ioring_hipri": "false", 2024-09-21T14:58:17.063 INFO:teuthology.orchestra.run.smithi032.stdout: "bdev_ioring_sqthread_poll": "false", 2024-09-21T14:58:17.063 INFO:teuthology.orchestra.run.smithi032.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-09-21T14:58:17.063 INFO:teuthology.orchestra.run.smithi032.stdout: "bdev_read_buffer_alignment": "4096", 2024-09-21T14:58:17.063 INFO:teuthology.orchestra.run.smithi032.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-09-21T14:58:17.063 INFO:teuthology.orchestra.run.smithi032.stdout: "bdev_type": "", 2024-09-21T14:58:17.063 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_alloc_size": "1048576", 2024-09-21T14:58:17.063 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_allocator": "hybrid", 2024-09-21T14:58:17.063 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_buffered_io": "true", 2024-09-21T14:58:17.063 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_check_for_zeros": "false", 2024-09-21T14:58:17.063 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_check_volume_selector_often": "false", 2024-09-21T14:58:17.063 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-09-21T14:58:17.063 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_compact_log_sync": "false", 2024-09-21T14:58:17.063 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-09-21T14:58:17.064 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-09-21T14:58:17.064 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_log_compact_min_size": "16777216", 2024-09-21T14:58:17.064 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_log_replay_check_allocations": "true", 2024-09-21T14:58:17.064 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_max_log_runway": "4194304", 2024-09-21T14:58:17.064 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_max_prefetch": "1048576", 2024-09-21T14:58:17.064 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_min_flush_size": "524288", 2024-09-21T14:58:17.064 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_min_log_runway": "1048576", 2024-09-21T14:58:17.064 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_replay_recovery": "false", 2024-09-21T14:58:17.064 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-09-21T14:58:17.064 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_shared_alloc_size": "65536", 2024-09-21T14:58:17.064 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_sync_write": "false", 2024-09-21T14:58:17.064 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-09-21T14:58:17.064 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-09-21T14:58:17.064 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-09-21T14:58:17.065 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_allocation_from_file": "true", 2024-09-21T14:58:17.065 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_allocator": "hybrid", 2024-09-21T14:58:17.065 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-09-21T14:58:17.065 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-09-21T14:58:17.065 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-09-21T14:58:17.065 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-09-21T14:58:17.065 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-09-21T14:58:17.065 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-09-21T14:58:17.065 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_blobid_prealloc": "10240", 2024-09-21T14:58:17.065 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_block_create": "true", 2024-09-21T14:58:17.065 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_block_db_create": "false", 2024-09-21T14:58:17.065 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_block_db_path": "", 2024-09-21T14:58:17.065 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_block_db_size": "0", 2024-09-21T14:58:17.066 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_block_path": "", 2024-09-21T14:58:17.066 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_block_preallocate_file": "false", 2024-09-21T14:58:17.066 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_block_size": "107374182400", 2024-09-21T14:58:17.066 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_block_wal_create": "false", 2024-09-21T14:58:17.066 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_block_wal_path": "", 2024-09-21T14:58:17.066 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_block_wal_size": "100663296", 2024-09-21T14:58:17.066 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_bluefs": "true", 2024-09-21T14:58:17.066 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-09-21T14:58:17.066 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_bluefs_env_mirror": "false", 2024-09-21T14:58:17.066 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-09-21T14:58:17.066 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-09-21T14:58:17.066 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-09-21T14:58:17.066 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-09-21T14:58:17.067 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-09-21T14:58:17.067 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-09-21T14:58:17.067 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_autotune": "true", 2024-09-21T14:58:17.067 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-09-21T14:58:17.067 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-09-21T14:58:17.067 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-09-21T14:58:17.067 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-09-21T14:58:17.067 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_size": "0", 2024-09-21T14:58:17.067 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-09-21T14:58:17.067 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-09-21T14:58:17.067 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-09-21T14:58:17.067 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-09-21T14:58:17.067 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_type": "2q", 2024-09-21T14:58:17.068 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-09-21T14:58:17.068 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_clone_cow": "true", 2024-09-21T14:58:17.068 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_compression_algorithm": "snappy", 2024-09-21T14:58:17.068 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_compression_max_blob_size": "0", 2024-09-21T14:58:17.068 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-09-21T14:58:17.068 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-09-21T14:58:17.068 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_compression_min_blob_size": "0", 2024-09-21T14:58:17.068 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-09-21T14:58:17.068 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-09-21T14:58:17.068 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_compression_mode": "none", 2024-09-21T14:58:17.068 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-09-21T14:58:17.068 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_csum_type": "crc32c", 2024-09-21T14:58:17.068 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_debug_enforce_settings": "default", 2024-09-21T14:58:17.068 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_debug_freelist": "false", 2024-09-21T14:58:17.069 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_debug_fsck_abort": "false", 2024-09-21T14:58:17.069 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-09-21T14:58:17.069 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_debug_inject_bug21040": "false", 2024-09-21T14:58:17.069 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-09-21T14:58:17.069 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_debug_inject_read_err": "false", 2024-09-21T14:58:17.069 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_debug_legacy_omap": "false", 2024-09-21T14:58:17.069 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-09-21T14:58:17.069 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-09-21T14:58:17.069 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-09-21T14:58:17.069 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-09-21T14:58:17.069 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_debug_prefill": "0.000000", 2024-09-21T14:58:17.069 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-09-21T14:58:17.069 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-09-21T14:58:17.070 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-09-21T14:58:17.070 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_debug_small_allocations": "0", 2024-09-21T14:58:17.070 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-09-21T14:58:17.070 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_default_buffered_read": "true", 2024-09-21T14:58:17.070 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_default_buffered_write": "false", 2024-09-21T14:58:17.070 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_deferred_batch_ops": "0", 2024-09-21T14:58:17.070 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-09-21T14:58:17.070 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-09-21T14:58:17.070 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-09-21T14:58:17.070 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-09-21T14:58:17.070 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-09-21T14:58:17.070 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-09-21T14:58:17.070 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-09-21T14:58:17.070 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_fail_eio": "false", 2024-09-21T14:58:17.071 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-09-21T14:58:17.071 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-09-21T14:58:17.071 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-09-21T14:58:17.071 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-09-21T14:58:17.071 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_fsck_on_mkfs": "true", 2024-09-21T14:58:17.071 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-09-21T14:58:17.071 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_fsck_on_mount": "false", 2024-09-21T14:58:17.071 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-09-21T14:58:17.071 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_fsck_on_umount": "false", 2024-09-21T14:58:17.071 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-09-21T14:58:17.071 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-09-21T14:58:17.071 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-09-21T14:58:17.071 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-09-21T14:58:17.071 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-09-21T14:58:17.072 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-09-21T14:58:17.072 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-09-21T14:58:17.072 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-09-21T14:58:17.072 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_ignore_data_csum": "false", 2024-09-21T14:58:17.072 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-09-21T14:58:17.072 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_kvbackend": "rocksdb", 2024-09-21T14:58:17.072 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-09-21T14:58:17.072 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-09-21T14:58:17.072 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_log_op_age": "5.000000", 2024-09-21T14:58:17.072 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_max_alloc_size": "0", 2024-09-21T14:58:17.072 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_max_blob_size": "0", 2024-09-21T14:58:17.072 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-09-21T14:58:17.072 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-09-21T14:58:17.072 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_max_defer_interval": "3.000000", 2024-09-21T14:58:17.072 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_max_deferred_txc": "32", 2024-09-21T14:58:17.073 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_min_alloc_size": "0", 2024-09-21T14:58:17.073 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-09-21T14:58:17.073 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-09-21T14:58:17.073 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_nid_prealloc": "1024", 2024-09-21T14:58:17.073 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_prefer_deferred_size": "0", 2024-09-21T14:58:17.073 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-09-21T14:58:17.073 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-09-21T14:58:17.073 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_qfsck_on_mount": "true", 2024-09-21T14:58:17.073 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_retry_disk_reads": "3", 2024-09-21T14:58:17.073 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_rocksdb_cf": "true", 2024-09-21T14:58:17.073 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2024-09-21T14:58:17.073 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2024-09-21T14:58:17.073 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_rocksdb_options_annex": "", 2024-09-21T14:58:17.073 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_spdk_coremask": "0x1", 2024-09-21T14:58:17.073 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_spdk_io_sleep": "5", 2024-09-21T14:58:17.074 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_spdk_max_io_completion": "0", 2024-09-21T14:58:17.074 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_spdk_mem": "512", 2024-09-21T14:58:17.074 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_sync_submit_transaction": "false", 2024-09-21T14:58:17.074 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_throttle_bytes": "67108864", 2024-09-21T14:58:17.074 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_throttle_cost_per_io": "0", 2024-09-21T14:58:17.074 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-09-21T14:58:17.074 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-09-21T14:58:17.074 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-09-21T14:58:17.074 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-09-21T14:58:17.074 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_tracing": "false", 2024-09-21T14:58:17.074 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-09-21T14:58:17.074 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-09-21T14:58:17.074 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_volume_selection_reserved": "0", 2024-09-21T14:58:17.074 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-09-21T14:58:17.074 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-09-21T14:58:17.075 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-09-21T14:58:17.075 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-09-21T14:58:17.075 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-09-21T14:58:17.075 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-09-21T14:58:17.075 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_zero_block_detection": "false", 2024-09-21T14:58:17.075 INFO:teuthology.orchestra.run.smithi032.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-09-21T14:58:17.075 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs_mirror_action_update_interval": "2", 2024-09-21T14:58:17.075 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-09-21T14:58:17.075 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-09-21T14:58:17.075 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-09-21T14:58:17.075 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-09-21T14:58:17.075 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs_mirror_mount_timeout": "10", 2024-09-21T14:58:17.075 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs_mirror_perf_stats_prio": "5", 2024-09-21T14:58:17.075 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-09-21T14:58:17.075 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-09-21T14:58:17.076 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-09-21T14:58:17.076 INFO:teuthology.orchestra.run.smithi032.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-09-21T14:58:17.076 INFO:teuthology.orchestra.run.smithi032.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-09-21T14:58:17.076 INFO:teuthology.orchestra.run.smithi032.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-09-21T14:58:17.076 INFO:teuthology.orchestra.run.smithi032.stdout: "cephx_cluster_require_signatures": "false", 2024-09-21T14:58:17.076 INFO:teuthology.orchestra.run.smithi032.stdout: "cephx_cluster_require_version": "2", 2024-09-21T14:58:17.076 INFO:teuthology.orchestra.run.smithi032.stdout: "cephx_require_signatures": "false", 2024-09-21T14:58:17.076 INFO:teuthology.orchestra.run.smithi032.stdout: "cephx_require_version": "2", 2024-09-21T14:58:17.076 INFO:teuthology.orchestra.run.smithi032.stdout: "cephx_service_require_signatures": "false", 2024-09-21T14:58:17.076 INFO:teuthology.orchestra.run.smithi032.stdout: "cephx_service_require_version": "2", 2024-09-21T14:58:17.076 INFO:teuthology.orchestra.run.smithi032.stdout: "cephx_sign_messages": "true", 2024-09-21T14:58:17.076 INFO:teuthology.orchestra.run.smithi032.stdout: "chdir": "", 2024-09-21T14:58:17.076 INFO:teuthology.orchestra.run.smithi032.stdout: "client_acl_type": "", 2024-09-21T14:58:17.076 INFO:teuthology.orchestra.run.smithi032.stdout: "client_asio_thread_count": "2", 2024-09-21T14:58:17.077 INFO:teuthology.orchestra.run.smithi032.stdout: "client_cache_mid": "0.750000", 2024-09-21T14:58:17.077 INFO:teuthology.orchestra.run.smithi032.stdout: "client_cache_size": "16384", 2024-09-21T14:58:17.077 INFO:teuthology.orchestra.run.smithi032.stdout: "client_caps_release_delay": "5", 2024-09-21T14:58:17.077 INFO:teuthology.orchestra.run.smithi032.stdout: "client_check_pool_perm": "true", 2024-09-21T14:58:17.077 INFO:teuthology.orchestra.run.smithi032.stdout: "client_collect_and_send_global_metrics": "false", 2024-09-21T14:58:17.077 INFO:teuthology.orchestra.run.smithi032.stdout: "client_debug_force_sync_read": "false", 2024-09-21T14:58:17.077 INFO:teuthology.orchestra.run.smithi032.stdout: "client_debug_getattr_caps": "false", 2024-09-21T14:58:17.077 INFO:teuthology.orchestra.run.smithi032.stdout: "client_debug_inject_features": "", 2024-09-21T14:58:17.077 INFO:teuthology.orchestra.run.smithi032.stdout: "client_debug_inject_tick_delay": "0", 2024-09-21T14:58:17.077 INFO:teuthology.orchestra.run.smithi032.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-09-21T14:58:17.077 INFO:teuthology.orchestra.run.smithi032.stdout: "client_die_on_failed_remount": "false", 2024-09-21T14:58:17.077 INFO:teuthology.orchestra.run.smithi032.stdout: "client_dirsize_rbytes": "true", 2024-09-21T14:58:17.077 INFO:teuthology.orchestra.run.smithi032.stdout: "client_force_lazyio": "false", 2024-09-21T14:58:17.077 INFO:teuthology.orchestra.run.smithi032.stdout: "client_fs": "", 2024-09-21T14:58:17.077 INFO:teuthology.orchestra.run.smithi032.stdout: "client_inject_fixed_oldest_tid": "false", 2024-09-21T14:58:17.077 INFO:teuthology.orchestra.run.smithi032.stdout: "client_inject_release_failure": "false", 2024-09-21T14:58:17.078 INFO:teuthology.orchestra.run.smithi032.stdout: "client_max_inline_size": "4096", 2024-09-21T14:58:17.078 INFO:teuthology.orchestra.run.smithi032.stdout: "client_max_retries_on_remount_failure": "5", 2024-09-21T14:58:17.078 INFO:teuthology.orchestra.run.smithi032.stdout: "client_mds_namespace": "", 2024-09-21T14:58:17.078 INFO:teuthology.orchestra.run.smithi032.stdout: "client_metadata": "", 2024-09-21T14:58:17.078 INFO:teuthology.orchestra.run.smithi032.stdout: "client_mount_gid": "-1", 2024-09-21T14:58:17.078 INFO:teuthology.orchestra.run.smithi032.stdout: "client_mount_timeout": "300", 2024-09-21T14:58:17.078 INFO:teuthology.orchestra.run.smithi032.stdout: "client_mount_uid": "-1", 2024-09-21T14:58:17.078 INFO:teuthology.orchestra.run.smithi032.stdout: "client_mountpoint": "/", 2024-09-21T14:58:17.078 INFO:teuthology.orchestra.run.smithi032.stdout: "client_notify_timeout": "10", 2024-09-21T14:58:17.078 INFO:teuthology.orchestra.run.smithi032.stdout: "client_oc": "true", 2024-09-21T14:58:17.078 INFO:teuthology.orchestra.run.smithi032.stdout: "client_oc_max_dirty": "104857600", 2024-09-21T14:58:17.078 INFO:teuthology.orchestra.run.smithi032.stdout: "client_oc_max_dirty_age": "5.000000", 2024-09-21T14:58:17.078 INFO:teuthology.orchestra.run.smithi032.stdout: "client_oc_max_objects": "1000", 2024-09-21T14:58:17.078 INFO:teuthology.orchestra.run.smithi032.stdout: "client_oc_size": "209715200", 2024-09-21T14:58:17.079 INFO:teuthology.orchestra.run.smithi032.stdout: "client_oc_target_dirty": "8388608", 2024-09-21T14:58:17.079 INFO:teuthology.orchestra.run.smithi032.stdout: "client_permissions": "true", 2024-09-21T14:58:17.079 INFO:teuthology.orchestra.run.smithi032.stdout: "client_quota": "true", 2024-09-21T14:58:17.079 INFO:teuthology.orchestra.run.smithi032.stdout: "client_quota_df": "true", 2024-09-21T14:58:17.079 INFO:teuthology.orchestra.run.smithi032.stdout: "client_readahead_max_bytes": "0", 2024-09-21T14:58:17.079 INFO:teuthology.orchestra.run.smithi032.stdout: "client_readahead_max_periods": "4", 2024-09-21T14:58:17.079 INFO:teuthology.orchestra.run.smithi032.stdout: "client_readahead_min": "131072", 2024-09-21T14:58:17.079 INFO:teuthology.orchestra.run.smithi032.stdout: "client_reconnect_stale": "false", 2024-09-21T14:58:17.079 INFO:teuthology.orchestra.run.smithi032.stdout: "client_shutdown_timeout": "30", 2024-09-21T14:58:17.079 INFO:teuthology.orchestra.run.smithi032.stdout: "client_snapdir": ".snap", 2024-09-21T14:58:17.079 INFO:teuthology.orchestra.run.smithi032.stdout: "client_tick_interval": "1", 2024-09-21T14:58:17.079 INFO:teuthology.orchestra.run.smithi032.stdout: "client_trace": "", 2024-09-21T14:58:17.079 INFO:teuthology.orchestra.run.smithi032.stdout: "client_try_dentry_invalidate": "false", 2024-09-21T14:58:17.079 INFO:teuthology.orchestra.run.smithi032.stdout: "client_use_faked_inos": "false", 2024-09-21T14:58:17.079 INFO:teuthology.orchestra.run.smithi032.stdout: "client_use_random_mds": "false", 2024-09-21T14:58:17.080 INFO:teuthology.orchestra.run.smithi032.stdout: "clog_to_graylog": "false", 2024-09-21T14:58:17.080 INFO:teuthology.orchestra.run.smithi032.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-09-21T14:58:17.080 INFO:teuthology.orchestra.run.smithi032.stdout: "clog_to_graylog_port": "12201", 2024-09-21T14:58:17.080 INFO:teuthology.orchestra.run.smithi032.stdout: "clog_to_monitors": "default=true", 2024-09-21T14:58:17.080 INFO:teuthology.orchestra.run.smithi032.stdout: "clog_to_syslog": "false", 2024-09-21T14:58:17.080 INFO:teuthology.orchestra.run.smithi032.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-09-21T14:58:17.080 INFO:teuthology.orchestra.run.smithi032.stdout: "clog_to_syslog_level": "info", 2024-09-21T14:58:17.080 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "-", 2024-09-21T14:58:17.080 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_network": "", 2024-09-21T14:58:17.080 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_network_interface": "", 2024-09-21T14:58:17.080 INFO:teuthology.orchestra.run.smithi032.stdout: "compressor_zlib_isal": "false", 2024-09-21T14:58:17.080 INFO:teuthology.orchestra.run.smithi032.stdout: "compressor_zlib_level": "5", 2024-09-21T14:58:17.080 INFO:teuthology.orchestra.run.smithi032.stdout: "compressor_zlib_winsize": "-15", 2024-09-21T14:58:17.080 INFO:teuthology.orchestra.run.smithi032.stdout: "compressor_zstd_level": "1", 2024-09-21T14:58:17.080 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2024-09-21T14:58:17.080 INFO:teuthology.orchestra.run.smithi032.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-09-21T14:58:17.081 INFO:teuthology.orchestra.run.smithi032.stdout: "crimson_alien_op_num_threads": "6", 2024-09-21T14:58:17.081 INFO:teuthology.orchestra.run.smithi032.stdout: "crimson_alien_thread_cpu_cores": "", 2024-09-21T14:58:17.081 INFO:teuthology.orchestra.run.smithi032.stdout: "crimson_osd_obc_lru_size": "10", 2024-09-21T14:58:17.081 INFO:teuthology.orchestra.run.smithi032.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-09-21T14:58:17.081 INFO:teuthology.orchestra.run.smithi032.stdout: "crimson_seastar_smp": "1", 2024-09-21T14:58:17.081 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_location": "", 2024-09-21T14:58:17.081 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_location_hook": "", 2024-09-21T14:58:17.081 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_location_hook_timeout": "10", 2024-09-21T14:58:17.081 INFO:teuthology.orchestra.run.smithi032.stdout: "daemonize": "false", 2024-09-21T14:58:17.081 INFO:teuthology.orchestra.run.smithi032.stdout: "daos_pool": "tank", 2024-09-21T14:58:17.081 INFO:teuthology.orchestra.run.smithi032.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2024-09-21T14:58:17.081 INFO:teuthology.orchestra.run.smithi032.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2024-09-21T14:58:17.081 INFO:teuthology.orchestra.run.smithi032.stdout: "dbstore_db_name_prefix": "dbstore", 2024-09-21T14:58:17.082 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_alienstore": "0/5", 2024-09-21T14:58:17.082 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_allow_any_pool_priority": "false", 2024-09-21T14:58:17.082 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_asok": "1/5", 2024-09-21T14:58:17.082 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_asok_assert_abort": "false", 2024-09-21T14:58:17.082 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_asserts_on_shutdown": "true", 2024-09-21T14:58:17.082 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_auth": "20/20", 2024-09-21T14:58:17.082 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_bdev": "1/3", 2024-09-21T14:58:17.082 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_bluefs": "1/5", 2024-09-21T14:58:17.082 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_bluestore": "1/5", 2024-09-21T14:58:17.082 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_buffer": "0/1", 2024-09-21T14:58:17.082 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_ceph_exporter": "1/5", 2024-09-21T14:58:17.082 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_cephfs_mirror": "0/5", 2024-09-21T14:58:17.082 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_cephsqlite": "0/5", 2024-09-21T14:58:17.082 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_client": "0/5", 2024-09-21T14:58:17.083 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_compressor": "1/5", 2024-09-21T14:58:17.083 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_context": "0/1", 2024-09-21T14:58:17.083 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_crush": "1/1", 2024-09-21T14:58:17.083 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_crypto": "1/5", 2024-09-21T14:58:17.083 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_cyanstore": "0/5", 2024-09-21T14:58:17.083 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_deliberately_leak_memory": "false", 2024-09-21T14:58:17.083 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_disable_randomized_ping": "false", 2024-09-21T14:58:17.083 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_dpdk": "1/5", 2024-09-21T14:58:17.083 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_eventtrace": "1/5", 2024-09-21T14:58:17.083 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_filer": "0/1", 2024-09-21T14:58:17.083 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_filestore": "1/3", 2024-09-21T14:58:17.083 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_finisher": "1/1", 2024-09-21T14:58:17.083 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_fuse": "1/5", 2024-09-21T14:58:17.084 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_heartbeat_testing_span": "0", 2024-09-21T14:58:17.084 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_heartbeatmap": "1/5", 2024-09-21T14:58:17.084 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_immutable_obj_cache": "0/5", 2024-09-21T14:58:17.084 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_javaclient": "1/5", 2024-09-21T14:58:17.084 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_journal": "1/3", 2024-09-21T14:58:17.084 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_journaler": "0/5", 2024-09-21T14:58:17.084 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_kstore": "1/5", 2024-09-21T14:58:17.084 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_leveldb": "4/5", 2024-09-21T14:58:17.084 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_lockdep": "0/1", 2024-09-21T14:58:17.084 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_mclock": "1/5", 2024-09-21T14:58:17.084 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_mds": "1/5", 2024-09-21T14:58:17.084 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_mds_balancer": "1/5", 2024-09-21T14:58:17.084 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_mds_locker": "1/5", 2024-09-21T14:58:17.085 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_mds_log": "1/5", 2024-09-21T14:58:17.085 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_mds_log_expire": "1/5", 2024-09-21T14:58:17.085 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_mds_migrator": "1/5", 2024-09-21T14:58:17.085 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_memstore": "1/5", 2024-09-21T14:58:17.085 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_mgr": "2/5", 2024-09-21T14:58:17.085 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_mgrc": "1/5", 2024-09-21T14:58:17.085 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_mon": "20/20", 2024-09-21T14:58:17.085 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_monc": "0/10", 2024-09-21T14:58:17.085 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_ms": "1/1", 2024-09-21T14:58:17.085 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_none": "0/5", 2024-09-21T14:58:17.085 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_objclass": "0/5", 2024-09-21T14:58:17.085 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_objectcacher": "0/5", 2024-09-21T14:58:17.085 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_objecter": "0/1", 2024-09-21T14:58:17.085 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_optracker": "0/5", 2024-09-21T14:58:17.086 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_osd": "1/5", 2024-09-21T14:58:17.086 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_paxos": "20/20", 2024-09-21T14:58:17.086 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_perfcounter": "1/5", 2024-09-21T14:58:17.086 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_prioritycache": "1/5", 2024-09-21T14:58:17.086 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_rados": "0/5", 2024-09-21T14:58:17.086 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_rbd": "0/5", 2024-09-21T14:58:17.086 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_rbd_mirror": "0/5", 2024-09-21T14:58:17.086 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_rbd_pwl": "0/5", 2024-09-21T14:58:17.086 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_rbd_replay": "0/5", 2024-09-21T14:58:17.086 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_refs": "0/0", 2024-09-21T14:58:17.087 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_reserver": "1/1", 2024-09-21T14:58:17.087 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_rgw": "1/5", 2024-09-21T14:58:17.087 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_rgw_access": "1/5", 2024-09-21T14:58:17.087 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_rgw_datacache": "1/5", 2024-09-21T14:58:17.087 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_rgw_dbstore": "1/5", 2024-09-21T14:58:17.087 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_rgw_flight": "1/5", 2024-09-21T14:58:17.087 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_rgw_sync": "1/5", 2024-09-21T14:58:17.087 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_rocksdb": "4/5", 2024-09-21T14:58:17.087 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_seastore": "0/5", 2024-09-21T14:58:17.087 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_seastore_backref": "0/5", 2024-09-21T14:58:17.087 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_seastore_cache": "0/5", 2024-09-21T14:58:17.087 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_seastore_cleaner": "0/5", 2024-09-21T14:58:17.087 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_seastore_device": "0/5", 2024-09-21T14:58:17.088 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_seastore_epm": "0/5", 2024-09-21T14:58:17.088 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_seastore_fixedkv_tree": "0/5", 2024-09-21T14:58:17.088 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_seastore_journal": "0/5", 2024-09-21T14:58:17.088 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_seastore_lba": "0/5", 2024-09-21T14:58:17.088 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_seastore_odata": "0/5", 2024-09-21T14:58:17.088 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_seastore_omap": "0/5", 2024-09-21T14:58:17.088 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_seastore_onode": "0/5", 2024-09-21T14:58:17.088 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_seastore_t": "0/5", 2024-09-21T14:58:17.088 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_seastore_tm": "0/5", 2024-09-21T14:58:17.088 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_striper": "0/1", 2024-09-21T14:58:17.088 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_test": "0/5", 2024-09-21T14:58:17.088 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_throttle": "1/1", 2024-09-21T14:58:17.088 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_timer": "0/1", 2024-09-21T14:58:17.088 INFO:teuthology.orchestra.run.smithi032.stdout: "debug_tp": "0/5", 2024-09-21T14:58:17.088 INFO:teuthology.orchestra.run.smithi032.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-09-21T14:58:17.089 INFO:teuthology.orchestra.run.smithi032.stdout: "device_failure_prediction_mode": "none", 2024-09-21T14:58:17.089 INFO:teuthology.orchestra.run.smithi032.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-09-21T14:58:17.089 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_dir": "/usr/lib/ceph/erasure-code", 2024-09-21T14:58:17.089 INFO:teuthology.orchestra.run.smithi032.stdout: "err_to_graylog": "false", 2024-09-21T14:58:17.089 INFO:teuthology.orchestra.run.smithi032.stdout: "err_to_journald": "false", 2024-09-21T14:58:17.089 INFO:teuthology.orchestra.run.smithi032.stdout: "err_to_stderr": "true", 2024-09-21T14:58:17.089 INFO:teuthology.orchestra.run.smithi032.stdout: "err_to_syslog": "false", 2024-09-21T14:58:17.089 INFO:teuthology.orchestra.run.smithi032.stdout: "event_tracing": "false", 2024-09-21T14:58:17.089 INFO:teuthology.orchestra.run.smithi032.stdout: "exporter_addr": "0.0.0.0", 2024-09-21T14:58:17.089 INFO:teuthology.orchestra.run.smithi032.stdout: "exporter_http_port": "9926", 2024-09-21T14:58:17.089 INFO:teuthology.orchestra.run.smithi032.stdout: "exporter_prio_limit": "5", 2024-09-21T14:58:17.089 INFO:teuthology.orchestra.run.smithi032.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-09-21T14:58:17.089 INFO:teuthology.orchestra.run.smithi032.stdout: "exporter_sort_metrics": "true", 2024-09-21T14:58:17.089 INFO:teuthology.orchestra.run.smithi032.stdout: "exporter_stats_period": "5", 2024-09-21T14:58:17.090 INFO:teuthology.orchestra.run.smithi032.stdout: "fake_statfs_for_testing": "0", 2024-09-21T14:58:17.090 INFO:teuthology.orchestra.run.smithi032.stdout: "fatal_signal_handlers": "true", 2024-09-21T14:58:17.090 INFO:teuthology.orchestra.run.smithi032.stdout: "filer_max_purge_ops": "10", 2024-09-21T14:58:17.090 INFO:teuthology.orchestra.run.smithi032.stdout: "filer_max_truncate_ops": "128", 2024-09-21T14:58:17.090 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_apply_finisher_threads": "1", 2024-09-21T14:58:17.090 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_blackhole": "false", 2024-09-21T14:58:17.090 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_btrfs_clone_range": "true", 2024-09-21T14:58:17.090 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_btrfs_snap": "true", 2024-09-21T14:58:17.090 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_caller_concurrency": "10", 2024-09-21T14:58:17.090 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_collect_device_partition_information": "true", 2024-09-21T14:58:17.090 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_commit_timeout": "600.000000", 2024-09-21T14:58:17.090 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_debug_inject_read_err": "false", 2024-09-21T14:58:17.090 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_debug_omap_check": "false", 2024-09-21T14:58:17.090 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_debug_random_read_err": "0.000000", 2024-09-21T14:58:17.091 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_debug_verify_split": "false", 2024-09-21T14:58:17.091 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_dump_file": "", 2024-09-21T14:58:17.091 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-09-21T14:58:17.091 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-09-21T14:58:17.091 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_fadvise": "true", 2024-09-21T14:58:17.091 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_fail_eio": "true", 2024-09-21T14:58:17.091 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_fd_cache_shards": "16", 2024-09-21T14:58:17.091 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_fd_cache_size": "128", 2024-09-21T14:58:17.091 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_fiemap": "false", 2024-09-21T14:58:17.091 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_fiemap_threshold": "4096", 2024-09-21T14:58:17.091 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-09-21T14:58:17.091 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_index_retry_probability": "0.000000", 2024-09-21T14:58:17.091 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_inject_stall": "0", 2024-09-21T14:58:17.091 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_journal_parallel": "false", 2024-09-21T14:58:17.091 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_journal_trailing": "false", 2024-09-21T14:58:17.092 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_journal_writeahead": "false", 2024-09-21T14:58:17.092 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_kill_at": "0", 2024-09-21T14:58:17.092 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-09-21T14:58:17.092 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_max_inline_xattr_size": "0", 2024-09-21T14:58:17.092 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-09-21T14:58:17.092 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-09-21T14:58:17.092 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-09-21T14:58:17.092 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_max_inline_xattrs": "0", 2024-09-21T14:58:17.092 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-09-21T14:58:17.092 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_max_inline_xattrs_other": "2", 2024-09-21T14:58:17.092 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-09-21T14:58:17.092 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_max_sync_interval": "5.000000", 2024-09-21T14:58:17.092 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_max_xattr_value_size": "0", 2024-09-21T14:58:17.093 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-09-21T14:58:17.093 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-09-21T14:58:17.093 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-09-21T14:58:17.093 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_merge_threshold": "-10", 2024-09-21T14:58:17.093 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_min_sync_interval": "0.010000", 2024-09-21T14:58:17.093 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_odsync_write": "false", 2024-09-21T14:58:17.093 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_omap_backend": "rocksdb", 2024-09-21T14:58:17.093 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_omap_backend_path": "", 2024-09-21T14:58:17.093 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_omap_header_cache_size": "1024", 2024-09-21T14:58:17.093 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_ondisk_finisher_threads": "1", 2024-09-21T14:58:17.093 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-09-21T14:58:17.093 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_op_thread_timeout": "60", 2024-09-21T14:58:17.093 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_op_threads": "2", 2024-09-21T14:58:17.094 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_punch_hole": "false", 2024-09-21T14:58:17.094 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-09-21T14:58:17.094 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-09-21T14:58:17.094 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-09-21T14:58:17.094 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-09-21T14:58:17.094 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-09-21T14:58:17.094 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_queue_max_bytes": "104857600", 2024-09-21T14:58:17.094 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-09-21T14:58:17.094 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-09-21T14:58:17.094 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-09-21T14:58:17.094 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_queue_max_ops": "50", 2024-09-21T14:58:17.094 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-09-21T14:58:17.095 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_seek_data_hole": "false", 2024-09-21T14:58:17.095 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_sloppy_crc": "false", 2024-09-21T14:58:17.095 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-09-21T14:58:17.095 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_splice": "false", 2024-09-21T14:58:17.095 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_split_multiple": "2", 2024-09-21T14:58:17.095 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_split_rand_factor": "20", 2024-09-21T14:58:17.095 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_update_to": "1000", 2024-09-21T14:58:17.095 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-09-21T14:58:17.095 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-09-21T14:58:17.095 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-09-21T14:58:17.095 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-09-21T14:58:17.095 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-09-21T14:58:17.096 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-09-21T14:58:17.096 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_wbthrottle_enable": "true", 2024-09-21T14:58:17.096 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-09-21T14:58:17.096 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-09-21T14:58:17.096 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-09-21T14:58:17.096 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-09-21T14:58:17.096 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-09-21T14:58:17.096 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-09-21T14:58:17.096 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_xfs_extsize": "false", 2024-09-21T14:58:17.096 INFO:teuthology.orchestra.run.smithi032.stdout: "filestore_zfs_snap": "false", 2024-09-21T14:58:17.096 INFO:teuthology.orchestra.run.smithi032.stdout: "fio_dir": "/tmp/fio", 2024-09-21T14:58:17.096 INFO:teuthology.orchestra.run.smithi032.stdout: "fsid": "caf51e55-9234-4a26-bcd2-d60811962560", 2024-09-21T14:58:17.097 INFO:teuthology.orchestra.run.smithi032.stdout: "fuse_allow_other": "true", 2024-09-21T14:58:17.097 INFO:teuthology.orchestra.run.smithi032.stdout: "fuse_atomic_o_trunc": "true", 2024-09-21T14:58:17.097 INFO:teuthology.orchestra.run.smithi032.stdout: "fuse_big_writes": "true", 2024-09-21T14:58:17.097 INFO:teuthology.orchestra.run.smithi032.stdout: "fuse_debug": "false", 2024-09-21T14:58:17.097 INFO:teuthology.orchestra.run.smithi032.stdout: "fuse_default_permissions": "false", 2024-09-21T14:58:17.097 INFO:teuthology.orchestra.run.smithi032.stdout: "fuse_disable_pagecache": "false", 2024-09-21T14:58:17.097 INFO:teuthology.orchestra.run.smithi032.stdout: "fuse_max_write": "0", 2024-09-21T14:58:17.097 INFO:teuthology.orchestra.run.smithi032.stdout: "fuse_multithreaded": "true", 2024-09-21T14:58:17.097 INFO:teuthology.orchestra.run.smithi032.stdout: "fuse_require_active_mds": "true", 2024-09-21T14:58:17.097 INFO:teuthology.orchestra.run.smithi032.stdout: "fuse_set_user_groups": "true", 2024-09-21T14:58:17.097 INFO:teuthology.orchestra.run.smithi032.stdout: "fuse_splice_move": "true", 2024-09-21T14:58:17.097 INFO:teuthology.orchestra.run.smithi032.stdout: "fuse_splice_read": "true", 2024-09-21T14:58:17.098 INFO:teuthology.orchestra.run.smithi032.stdout: "fuse_splice_write": "true", 2024-09-21T14:58:17.098 INFO:teuthology.orchestra.run.smithi032.stdout: "fuse_syncfs_on_mksnap": "true", 2024-09-21T14:58:17.098 INFO:teuthology.orchestra.run.smithi032.stdout: "fuse_use_invalidate_cb": "true", 2024-09-21T14:58:17.098 INFO:teuthology.orchestra.run.smithi032.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-09-21T14:58:17.098 INFO:teuthology.orchestra.run.smithi032.stdout: "gss_target_name": "ceph", 2024-09-21T14:58:17.098 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_file": "", 2024-09-21T14:58:17.098 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_inject_failure": "0", 2024-09-21T14:58:17.098 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_interval": "5", 2024-09-21T14:58:17.098 INFO:teuthology.orchestra.run.smithi032.stdout: "host": "smithi032", 2024-09-21T14:58:17.098 INFO:teuthology.orchestra.run.smithi032.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-09-21T14:58:17.098 INFO:teuthology.orchestra.run.smithi032.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-09-21T14:58:17.098 INFO:teuthology.orchestra.run.smithi032.stdout: "immutable_object_cache_max_size": "1073741824", 2024-09-21T14:58:17.098 INFO:teuthology.orchestra.run.smithi032.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-09-21T14:58:17.098 INFO:teuthology.orchestra.run.smithi032.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-09-21T14:58:17.099 INFO:teuthology.orchestra.run.smithi032.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-09-21T14:58:17.099 INFO:teuthology.orchestra.run.smithi032.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-09-21T14:58:17.099 INFO:teuthology.orchestra.run.smithi032.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-09-21T14:58:17.099 INFO:teuthology.orchestra.run.smithi032.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-09-21T14:58:17.099 INFO:teuthology.orchestra.run.smithi032.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-09-21T14:58:17.099 INFO:teuthology.orchestra.run.smithi032.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-09-21T14:58:17.099 INFO:teuthology.orchestra.run.smithi032.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-09-21T14:58:17.099 INFO:teuthology.orchestra.run.smithi032.stdout: "immutable_object_cache_watermark": "0.900000", 2024-09-21T14:58:17.099 INFO:teuthology.orchestra.run.smithi032.stdout: "inject_early_sigterm": "false", 2024-09-21T14:58:17.099 INFO:teuthology.orchestra.run.smithi032.stdout: "jaeger_agent_port": "6799", 2024-09-21T14:58:17.099 INFO:teuthology.orchestra.run.smithi032.stdout: "jaeger_tracing_enable": "false", 2024-09-21T14:58:17.099 INFO:teuthology.orchestra.run.smithi032.stdout: "journal_aio": "true", 2024-09-21T14:58:17.099 INFO:teuthology.orchestra.run.smithi032.stdout: "journal_align_min_size": "65536", 2024-09-21T14:58:17.100 INFO:teuthology.orchestra.run.smithi032.stdout: "journal_block_align": "true", 2024-09-21T14:58:17.100 INFO:teuthology.orchestra.run.smithi032.stdout: "journal_block_size": "4096", 2024-09-21T14:58:17.100 INFO:teuthology.orchestra.run.smithi032.stdout: "journal_dio": "true", 2024-09-21T14:58:17.100 INFO:teuthology.orchestra.run.smithi032.stdout: "journal_discard": "false", 2024-09-21T14:58:17.100 INFO:teuthology.orchestra.run.smithi032.stdout: "journal_force_aio": "false", 2024-09-21T14:58:17.100 INFO:teuthology.orchestra.run.smithi032.stdout: "journal_ignore_corruption": "false", 2024-09-21T14:58:17.100 INFO:teuthology.orchestra.run.smithi032.stdout: "journal_max_write_bytes": "10485760", 2024-09-21T14:58:17.100 INFO:teuthology.orchestra.run.smithi032.stdout: "journal_max_write_entries": "100", 2024-09-21T14:58:17.100 INFO:teuthology.orchestra.run.smithi032.stdout: "journal_replay_from": "0", 2024-09-21T14:58:17.100 INFO:teuthology.orchestra.run.smithi032.stdout: "journal_throttle_high_multiple": "0.000000", 2024-09-21T14:58:17.100 INFO:teuthology.orchestra.run.smithi032.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-09-21T14:58:17.100 INFO:teuthology.orchestra.run.smithi032.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-09-21T14:58:17.100 INFO:teuthology.orchestra.run.smithi032.stdout: "journal_throttle_max_multiple": "0.000000", 2024-09-21T14:58:17.100 INFO:teuthology.orchestra.run.smithi032.stdout: "journal_write_header_frequency": "0", 2024-09-21T14:58:17.101 INFO:teuthology.orchestra.run.smithi032.stdout: "journal_zero_on_create": "false", 2024-09-21T14:58:17.101 INFO:teuthology.orchestra.run.smithi032.stdout: "journaler_prefetch_periods": "10", 2024-09-21T14:58:17.101 INFO:teuthology.orchestra.run.smithi032.stdout: "journaler_prezero_periods": "5", 2024-09-21T14:58:17.101 INFO:teuthology.orchestra.run.smithi032.stdout: "journaler_write_head_interval": "15", 2024-09-21T14:58:17.101 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "", 2024-09-21T14:58:17.101 INFO:teuthology.orchestra.run.smithi032.stdout: "keyfile": "", 2024-09-21T14:58:17.101 INFO:teuthology.orchestra.run.smithi032.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-09-21T14:58:17.101 INFO:teuthology.orchestra.run.smithi032.stdout: "kstore_backend": "rocksdb", 2024-09-21T14:58:17.101 INFO:teuthology.orchestra.run.smithi032.stdout: "kstore_default_stripe_size": "65536", 2024-09-21T14:58:17.101 INFO:teuthology.orchestra.run.smithi032.stdout: "kstore_fsck_on_mount": "false", 2024-09-21T14:58:17.101 INFO:teuthology.orchestra.run.smithi032.stdout: "kstore_fsck_on_mount_deep": "true", 2024-09-21T14:58:17.101 INFO:teuthology.orchestra.run.smithi032.stdout: "kstore_max_bytes": "67108864", 2024-09-21T14:58:17.101 INFO:teuthology.orchestra.run.smithi032.stdout: "kstore_max_ops": "512", 2024-09-21T14:58:17.101 INFO:teuthology.orchestra.run.smithi032.stdout: "kstore_nid_prealloc": "1024", 2024-09-21T14:58:17.102 INFO:teuthology.orchestra.run.smithi032.stdout: "kstore_onode_map_size": "1024", 2024-09-21T14:58:17.102 INFO:teuthology.orchestra.run.smithi032.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-09-21T14:58:17.102 INFO:teuthology.orchestra.run.smithi032.stdout: "kstore_sync_submit_transaction": "false", 2024-09-21T14:58:17.102 INFO:teuthology.orchestra.run.smithi032.stdout: "kstore_sync_transaction": "false", 2024-09-21T14:58:17.102 INFO:teuthology.orchestra.run.smithi032.stdout: "leveldb_block_size": "65536", 2024-09-21T14:58:17.102 INFO:teuthology.orchestra.run.smithi032.stdout: "leveldb_bloom_size": "0", 2024-09-21T14:58:17.102 INFO:teuthology.orchestra.run.smithi032.stdout: "leveldb_cache_size": "536870912", 2024-09-21T14:58:17.103 INFO:teuthology.orchestra.run.smithi032.stdout: "leveldb_compact_on_mount": "false", 2024-09-21T14:58:17.103 INFO:teuthology.orchestra.run.smithi032.stdout: "leveldb_compression": "false", 2024-09-21T14:58:17.103 INFO:teuthology.orchestra.run.smithi032.stdout: "leveldb_log": "", 2024-09-21T14:58:17.103 INFO:teuthology.orchestra.run.smithi032.stdout: "leveldb_log_to_ceph_log": "true", 2024-09-21T14:58:17.103 INFO:teuthology.orchestra.run.smithi032.stdout: "leveldb_max_open_files": "0", 2024-09-21T14:58:17.103 INFO:teuthology.orchestra.run.smithi032.stdout: "leveldb_paranoid": "false", 2024-09-21T14:58:17.103 INFO:teuthology.orchestra.run.smithi032.stdout: "leveldb_write_buffer_size": "33554432", 2024-09-21T14:58:17.104 INFO:teuthology.orchestra.run.smithi032.stdout: "librados_thread_count": "2", 2024-09-21T14:58:17.104 INFO:teuthology.orchestra.run.smithi032.stdout: "lockdep": "false", 2024-09-21T14:58:17.104 INFO:teuthology.orchestra.run.smithi032.stdout: "lockdep_force_backtrace": "false", 2024-09-21T14:58:17.104 INFO:teuthology.orchestra.run.smithi032.stdout: "log_coarse_timestamps": "true", 2024-09-21T14:58:17.104 INFO:teuthology.orchestra.run.smithi032.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-09-21T14:58:17.104 INFO:teuthology.orchestra.run.smithi032.stdout: "log_flush_on_exit": "false", 2024-09-21T14:58:17.104 INFO:teuthology.orchestra.run.smithi032.stdout: "log_graylog_host": "127.0.0.1", 2024-09-21T14:58:17.104 INFO:teuthology.orchestra.run.smithi032.stdout: "log_graylog_port": "12201", 2024-09-21T14:58:17.104 INFO:teuthology.orchestra.run.smithi032.stdout: "log_max_new": "1000", 2024-09-21T14:58:17.105 INFO:teuthology.orchestra.run.smithi032.stdout: "log_max_recent": "10000", 2024-09-21T14:58:17.105 INFO:teuthology.orchestra.run.smithi032.stdout: "log_stderr_prefix": "", 2024-09-21T14:58:17.105 INFO:teuthology.orchestra.run.smithi032.stdout: "log_stop_at_utilization": "0.970000", 2024-09-21T14:58:17.105 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": "true", 2024-09-21T14:58:17.105 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_graylog": "false", 2024-09-21T14:58:17.105 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_journald": "false", 2024-09-21T14:58:17.105 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_stderr": "false", 2024-09-21T14:58:17.105 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_syslog": "false", 2024-09-21T14:58:17.105 INFO:teuthology.orchestra.run.smithi032.stdout: "max_rotating_auth_attempts": "10", 2024-09-21T14:58:17.106 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-09-21T14:58:17.106 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_action_on_write_error": "1", 2024-09-21T14:58:17.106 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_alternate_name_max": "8192", 2024-09-21T14:58:17.106 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_asio_thread_count": "2", 2024-09-21T14:58:17.106 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_bal_export_pin": "true", 2024-09-21T14:58:17.106 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_bal_fragment_dirs": "true", 2024-09-21T14:58:17.107 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-09-21T14:58:17.107 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_bal_fragment_interval": "5", 2024-09-21T14:58:17.107 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_bal_fragment_size_max": "100000", 2024-09-21T14:58:17.107 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_bal_idle_threshold": "0.000000", 2024-09-21T14:58:17.107 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_bal_interval": "10", 2024-09-21T14:58:17.107 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_bal_max": "-1", 2024-09-21T14:58:17.107 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_bal_max_until": "-1", 2024-09-21T14:58:17.108 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_bal_merge_size": "50", 2024-09-21T14:58:17.108 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_bal_midchunk": "0.300000", 2024-09-21T14:58:17.108 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_bal_min_rebalance": "0.100000", 2024-09-21T14:58:17.108 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_bal_min_start": "0.200000", 2024-09-21T14:58:17.108 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_bal_minchunk": "0.001000", 2024-09-21T14:58:17.108 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_bal_mode": "0", 2024-09-21T14:58:17.108 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_bal_need_max": "1.200000", 2024-09-21T14:58:17.108 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_bal_need_min": "0.800000", 2024-09-21T14:58:17.108 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-09-21T14:58:17.108 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_bal_sample_interval": "3.000000", 2024-09-21T14:58:17.108 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_bal_split_bits": "3", 2024-09-21T14:58:17.109 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_bal_split_rd": "25000.000000", 2024-09-21T14:58:17.109 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_bal_split_size": "10000", 2024-09-21T14:58:17.109 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_bal_split_wr": "10000.000000", 2024-09-21T14:58:17.109 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_bal_target_decay": "10.000000", 2024-09-21T14:58:17.109 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-09-21T14:58:17.109 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_beacon_grace": "15.000000", 2024-09-21T14:58:17.109 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_beacon_interval": "4.000000", 2024-09-21T14:58:17.109 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_beacon_mon_down_grace": "60", 2024-09-21T14:58:17.109 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_cache_memory_limit": "4294967296", 2024-09-21T14:58:17.109 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_cache_mid": "0.700000", 2024-09-21T14:58:17.109 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_cache_release_free_interval": "10", 2024-09-21T14:58:17.109 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_cache_reservation": "0.050000", 2024-09-21T14:58:17.109 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-09-21T14:58:17.109 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_cache_trim_interval": "1", 2024-09-21T14:58:17.109 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_cache_trim_threshold": "262144", 2024-09-21T14:58:17.110 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-09-21T14:58:17.110 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-09-21T14:58:17.110 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_client_delegate_inos_pct": "50", 2024-09-21T14:58:17.110 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_client_prealloc_inos": "1000", 2024-09-21T14:58:17.110 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-09-21T14:58:17.110 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_connect_bootstrapping": "false", 2024-09-21T14:58:17.110 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_damage_table_max_entries": "10000", 2024-09-21T14:58:17.110 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-09-21T14:58:17.110 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_debug_auth_pins": "false", 2024-09-21T14:58:17.110 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_debug_frag": "false", 2024-09-21T14:58:17.110 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_debug_scatterstat": "false", 2024-09-21T14:58:17.110 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_debug_subtrees": "false", 2024-09-21T14:58:17.110 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_decay_halflife": "5.000000", 2024-09-21T14:58:17.111 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_default_dir_hash": "2", 2024-09-21T14:58:17.111 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_defer_session_stale": "true", 2024-09-21T14:58:17.111 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_deny_all_reconnect": "false", 2024-09-21T14:58:17.111 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_dir_keys_per_op": "16384", 2024-09-21T14:58:17.111 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_dir_max_commit_size": "10", 2024-09-21T14:58:17.111 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_dir_max_entries": "0", 2024-09-21T14:58:17.111 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_dir_prefetch": "true", 2024-09-21T14:58:17.111 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_dirstat_min_interval": "1.000000", 2024-09-21T14:58:17.111 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_dump_cache_after_rejoin": "false", 2024-09-21T14:58:17.111 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_dump_cache_on_map": "false", 2024-09-21T14:58:17.111 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_dump_cache_threshold_file": "0", 2024-09-21T14:58:17.111 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-09-21T14:58:17.111 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_early_reply": "true", 2024-09-21T14:58:17.111 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_enable_op_tracker": "true", 2024-09-21T14:58:17.111 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_enforce_unique_name": "true", 2024-09-21T14:58:17.112 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_export_ephemeral_distributed": "true", 2024-09-21T14:58:17.112 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-09-21T14:58:17.112 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_export_ephemeral_random": "true", 2024-09-21T14:58:17.112 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-09-21T14:58:17.112 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_extraordinary_events_dump_interval": "0", 2024-09-21T14:58:17.112 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_forward_all_requests_to_auth": "false", 2024-09-21T14:58:17.112 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-09-21T14:58:17.112 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_fscrypt_last_block_max_size": "4096", 2024-09-21T14:58:17.112 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-09-21T14:58:17.112 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-09-21T14:58:17.112 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_health_cache_threshold": "1.500000", 2024-09-21T14:58:17.112 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_health_summarize_threshold": "10", 2024-09-21T14:58:17.112 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_heartbeat_grace": "15.000000", 2024-09-21T14:58:17.112 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_heartbeat_reset_grace": "1000", 2024-09-21T14:58:17.112 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_inject_health_dummy": "false", 2024-09-21T14:58:17.113 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-09-21T14:58:17.113 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_inject_migrator_session_race": "false", 2024-09-21T14:58:17.113 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-09-21T14:58:17.113 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-09-21T14:58:17.113 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-09-21T14:58:17.113 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_join_fs": "", 2024-09-21T14:58:17.113 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_journal_format": "1", 2024-09-21T14:58:17.113 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_kill_after_journal_logs_flushed": "false", 2024-09-21T14:58:17.113 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_kill_create_at": "0", 2024-09-21T14:58:17.113 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_kill_export_at": "0", 2024-09-21T14:58:17.113 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_kill_import_at": "0", 2024-09-21T14:58:17.113 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_kill_journal_at": "0", 2024-09-21T14:58:17.113 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_kill_journal_expire_at": "0", 2024-09-21T14:58:17.113 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_kill_journal_replay_at": "0", 2024-09-21T14:58:17.113 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_kill_link_at": "0", 2024-09-21T14:58:17.113 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_kill_mdstable_at": "0", 2024-09-21T14:58:17.114 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_kill_openc_at": "0", 2024-09-21T14:58:17.114 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_kill_rename_at": "0", 2024-09-21T14:58:17.114 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_log_events_per_segment": "1024", 2024-09-21T14:58:17.114 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_log_max_events": "-1", 2024-09-21T14:58:17.114 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_log_max_segments": "128", 2024-09-21T14:58:17.114 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_log_pause": "false", 2024-09-21T14:58:17.114 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_log_segment_size": "0", 2024-09-21T14:58:17.114 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_log_skip_corrupt_events": "false", 2024-09-21T14:58:17.114 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_log_warn_factor": "2.000000", 2024-09-21T14:58:17.114 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_max_caps_per_client": "1048576", 2024-09-21T14:58:17.114 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_max_completed_flushes": "100000", 2024-09-21T14:58:17.114 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_max_completed_requests": "100000", 2024-09-21T14:58:17.114 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_max_export_size": "20971520", 2024-09-21T14:58:17.115 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_max_file_recover": "32", 2024-09-21T14:58:17.115 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_max_purge_files": "64", 2024-09-21T14:58:17.115 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_max_purge_ops": "8192", 2024-09-21T14:58:17.115 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-09-21T14:58:17.115 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-09-21T14:58:17.115 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_max_snaps_per_dir": "100", 2024-09-21T14:58:17.115 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_metrics_update_interval": "2", 2024-09-21T14:58:17.115 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_min_caps_per_client": "100", 2024-09-21T14:58:17.115 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_min_caps_working_set": "10000", 2024-09-21T14:58:17.115 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-09-21T14:58:17.115 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_numa_node": "-1", 2024-09-21T14:58:17.115 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-09-21T14:58:17.115 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_op_complaint_time": "30.000000", 2024-09-21T14:58:17.115 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_op_history_duration": "600", 2024-09-21T14:58:17.115 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_op_history_size": "20", 2024-09-21T14:58:17.116 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_op_history_slow_op_size": "20", 2024-09-21T14:58:17.116 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_op_history_slow_op_threshold": "10", 2024-09-21T14:58:17.116 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_op_log_threshold": "5", 2024-09-21T14:58:17.116 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_ping_grace": "15", 2024-09-21T14:58:17.116 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_ping_interval": "5", 2024-09-21T14:58:17.116 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-09-21T14:58:17.116 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-09-21T14:58:17.116 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_recall_max_caps": "30000", 2024-09-21T14:58:17.116 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-09-21T14:58:17.116 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_recall_max_decay_threshold": "131072", 2024-09-21T14:58:17.116 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-09-21T14:58:17.116 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_recall_warning_threshold": "262144", 2024-09-21T14:58:17.116 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_reconnect_timeout": "45.000000", 2024-09-21T14:58:17.116 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_replay_interval": "1.000000", 2024-09-21T14:58:17.116 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-09-21T14:58:17.117 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-09-21T14:58:17.117 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_root_ino_gid": "0", 2024-09-21T14:58:17.117 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_root_ino_uid": "0", 2024-09-21T14:58:17.117 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-09-21T14:58:17.117 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_session_blocklist_on_evict": "true", 2024-09-21T14:58:17.117 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_session_blocklist_on_timeout": "true", 2024-09-21T14:58:17.117 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-09-21T14:58:17.117 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-09-21T14:58:17.117 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-09-21T14:58:17.117 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-09-21T14:58:17.117 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-09-21T14:58:17.117 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_session_metadata_threshold": "16777216", 2024-09-21T14:58:17.117 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-09-21T14:58:17.117 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_shutdown_check": "0", 2024-09-21T14:58:17.117 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_skip_ino": "0", 2024-09-21T14:58:17.118 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_sleep_rank_change": "0.000000", 2024-09-21T14:58:17.118 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_snap_max_uid": "4294967294", 2024-09-21T14:58:17.118 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_snap_min_uid": "0", 2024-09-21T14:58:17.118 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_snap_rstat": "false", 2024-09-21T14:58:17.118 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_standby_replay_damaged": "false", 2024-09-21T14:58:17.118 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_symlink_recovery": "true", 2024-09-21T14:58:17.118 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_task_status_update_interval": "2.000000", 2024-09-21T14:58:17.118 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_thrash_exports": "0", 2024-09-21T14:58:17.118 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_thrash_fragments": "0", 2024-09-21T14:58:17.118 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_tick_interval": "5.000000", 2024-09-21T14:58:17.118 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_valgrind_exit": "false", 2024-09-21T14:58:17.118 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_verify_backtrace": "1", 2024-09-21T14:58:17.118 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_verify_scatter": "false", 2024-09-21T14:58:17.119 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_wipe_ino_prealloc": "false", 2024-09-21T14:58:17.119 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_wipe_sessions": "false", 2024-09-21T14:58:17.119 INFO:teuthology.orchestra.run.smithi032.stdout: "mempool_debug": "false", 2024-09-21T14:58:17.119 INFO:teuthology.orchestra.run.smithi032.stdout: "memstore_debug_omit_block_device_write": "false", 2024-09-21T14:58:17.119 INFO:teuthology.orchestra.run.smithi032.stdout: "memstore_device_bytes": "1073741824", 2024-09-21T14:58:17.119 INFO:teuthology.orchestra.run.smithi032.stdout: "memstore_page_set": "false", 2024-09-21T14:58:17.119 INFO:teuthology.orchestra.run.smithi032.stdout: "memstore_page_size": "65536", 2024-09-21T14:58:17.119 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_client_bytes": "134217728", 2024-09-21T14:58:17.119 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_client_messages": "512", 2024-09-21T14:58:17.119 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-09-21T14:58:17.119 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_connect_retry_interval": "1.000000", 2024-09-21T14:58:17.119 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-09-21T14:58:17.119 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-09-21T14:58:17.119 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_disabled_modules": "", 2024-09-21T14:58:17.119 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-09-21T14:58:17.120 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_max_pg_num_change": "128", 2024-09-21T14:58:17.120 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_mds_bytes": "134217728", 2024-09-21T14:58:17.120 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_mds_messages": "128", 2024-09-21T14:58:17.120 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-09-21T14:58:17.120 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_mon_bytes": "134217728", 2024-09-21T14:58:17.120 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_mon_messages": "128", 2024-09-21T14:58:17.120 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_osd_bytes": "536870912", 2024-09-21T14:58:17.120 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_osd_messages": "8192", 2024-09-21T14:58:17.120 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_pool": "true", 2024-09-21T14:58:17.120 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_service_beacon_grace": "60.000000", 2024-09-21T14:58:17.120 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_standby_modules": "true", 2024-09-21T14:58:17.120 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_stats_period": "5", 2024-09-21T14:58:17.120 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_stats_threshold": "5", 2024-09-21T14:58:17.120 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_tick_period": "2", 2024-09-21T14:58:17.120 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-09-21T14:58:17.121 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_accept_timeout_factor": "2.000000", 2024-09-21T14:58:17.121 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_allow_pool_delete": "true", 2024-09-21T14:58:17.121 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_allow_pool_size_one": "true", 2024-09-21T14:58:17.121 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_auth_validate_all_caps": "true", 2024-09-21T14:58:17.121 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-09-21T14:58:17.121 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-09-21T14:58:17.121 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_client_bytes": "104857600", 2024-09-21T14:58:17.121 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_client_directed_command_retry": "5", 2024-09-21T14:58:17.121 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_client_hunt_interval": "3.000000", 2024-09-21T14:58:17.121 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-09-21T14:58:17.121 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-09-21T14:58:17.121 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-09-21T14:58:17.121 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_client_hunt_parallel": "3", 2024-09-21T14:58:17.121 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_client_log_interval": "1.000000", 2024-09-21T14:58:17.121 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-09-21T14:58:17.121 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_client_ping_interval": "10.000000", 2024-09-21T14:58:17.122 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_client_ping_timeout": "30.000000", 2024-09-21T14:58:17.122 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_clock_drift_allowed": "1.000000", 2024-09-21T14:58:17.122 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-09-21T14:58:17.122 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-09-21T14:58:17.122 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_cluster_log_file_level": "debug", 2024-09-21T14:58:17.122 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_cluster_log_to_file": "true", 2024-09-21T14:58:17.122 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_cluster_log_to_graylog": "false", 2024-09-21T14:58:17.122 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-09-21T14:58:17.122 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-09-21T14:58:17.122 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_cluster_log_to_journald": "false", 2024-09-21T14:58:17.122 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_cluster_log_to_stderr": "false", 2024-09-21T14:58:17.122 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-09-21T14:58:17.122 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-09-21T14:58:17.122 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_cluster_log_to_syslog_level": "info", 2024-09-21T14:58:17.123 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_compact_on_bootstrap": "false", 2024-09-21T14:58:17.123 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_compact_on_start": "false", 2024-09-21T14:58:17.123 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_compact_on_trim": "true", 2024-09-21T14:58:17.123 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_con_tracker_persist_interval": "10", 2024-09-21T14:58:17.123 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_con_tracker_score_halflife": "43200", 2024-09-21T14:58:17.123 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_config_key_max_entry_size": "65536", 2024-09-21T14:58:17.123 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_cpu_threads": "4", 2024-09-21T14:58:17.123 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_crush_min_required_version": "hammer", 2024-09-21T14:58:17.123 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_daemon_bytes": "419430400", 2024-09-21T14:58:17.123 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-09-21T14:58:17.123 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_data_avail_crit": "5", 2024-09-21T14:58:17.123 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_data_avail_warn": "5", 2024-09-21T14:58:17.123 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_data_size_warn": "16106127360", 2024-09-21T14:58:17.123 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_debug_block_osdmap_trim": "false", 2024-09-21T14:58:17.124 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-09-21T14:58:17.124 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_debug_dump_json": "false", 2024-09-21T14:58:17.124 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-09-21T14:58:17.124 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_debug_dump_transactions": "false", 2024-09-21T14:58:17.124 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_debug_extra_checks": "false", 2024-09-21T14:58:17.124 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-09-21T14:58:17.124 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-09-21T14:58:17.124 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_debug_no_require_quincy": "false", 2024-09-21T14:58:17.124 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_debug_no_require_reef": "false", 2024-09-21T14:58:17.124 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-09-21T14:58:17.124 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_delta_reset_interval": "10.000000", 2024-09-21T14:58:17.124 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_dns_srv_name": "ceph-mon", 2024-09-21T14:58:17.124 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_down_mkfs_grace": "120", 2024-09-21T14:58:17.124 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_election_default_strategy": "1", 2024-09-21T14:58:17.125 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_election_timeout": "5.000000", 2024-09-21T14:58:17.125 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-09-21T14:58:17.125 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_elector_ping_divisor": "2", 2024-09-21T14:58:17.125 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_elector_ping_timeout": "2.000000", 2024-09-21T14:58:17.125 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_enable_op_tracker": "true", 2024-09-21T14:58:17.125 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_fake_pool_delete": "false", 2024-09-21T14:58:17.125 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_force_quorum_join": "false", 2024-09-21T14:58:17.125 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_globalid_prealloc": "10000", 2024-09-21T14:58:17.125 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_health_detail_to_clog": "false", 2024-09-21T14:58:17.125 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_health_log_update_period": "5", 2024-09-21T14:58:17.125 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_health_max_detail": "50", 2024-09-21T14:58:17.125 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_health_to_clog": "true", 2024-09-21T14:58:17.125 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_health_to_clog_interval": "600", 2024-09-21T14:58:17.125 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-09-21T14:58:17.126 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_host": "172.21.15.32,[v2:172.21.15.32:3301,v1:172.21.15.32:6790],172.21.15.46", 2024-09-21T14:58:17.126 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_host_override": "", 2024-09-21T14:58:17.126 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_initial_members": "", 2024-09-21T14:58:17.126 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-09-21T14:58:17.126 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-09-21T14:58:17.126 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-09-21T14:58:17.126 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-09-21T14:58:17.126 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_keyvaluedb": "rocksdb", 2024-09-21T14:58:17.126 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_lease": "5.000000", 2024-09-21T14:58:17.126 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-09-21T14:58:17.126 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-09-21T14:58:17.126 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_log_full_interval": "50", 2024-09-21T14:58:17.126 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_log_max": "10000", 2024-09-21T14:58:17.127 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_log_max_summary": "50", 2024-09-21T14:58:17.127 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_max_log_entries_per_event": "4096", 2024-09-21T14:58:17.127 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_max_log_epochs": "500", 2024-09-21T14:58:17.127 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_max_mdsmap_epochs": "500", 2024-09-21T14:58:17.127 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_max_mgrmap_epochs": "500", 2024-09-21T14:58:17.127 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_max_osd": "10000", 2024-09-21T14:58:17.127 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_max_pg_per_osd": "10000", 2024-09-21T14:58:17.127 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_max_pool_pg_num": "65536", 2024-09-21T14:58:17.127 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-09-21T14:58:17.127 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-09-21T14:58:17.127 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_mds_force_trim_to": "0", 2024-09-21T14:58:17.127 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_mds_skip_sanity": "false", 2024-09-21T14:58:17.127 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_memory_autotune": "true", 2024-09-21T14:58:17.128 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_memory_target": "2147483648", 2024-09-21T14:58:17.128 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_mgr_beacon_grace": "30", 2024-09-21T14:58:17.128 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-09-21T14:58:17.128 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_mgr_digest_period": "5", 2024-09-21T14:58:17.128 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_mgr_inactive_grace": "60", 2024-09-21T14:58:17.128 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_mgr_mkfs_grace": "240", 2024-09-21T14:58:17.128 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-09-21T14:58:17.128 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_min_osdmap_epochs": "500", 2024-09-21T14:58:17.128 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_op_complaint_time": "30", 2024-09-21T14:58:17.128 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_op_history_duration": "600", 2024-09-21T14:58:17.128 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_op_history_size": "20", 2024-09-21T14:58:17.128 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_op_history_slow_op_size": "20", 2024-09-21T14:58:17.129 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_op_history_slow_op_threshold": "10", 2024-09-21T14:58:17.129 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_op_log_threshold": "5", 2024-09-21T14:58:17.129 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-09-21T14:58:17.129 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-09-21T14:58:17.129 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-09-21T14:58:17.129 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_auto_mark_in": "false", 2024-09-21T14:58:17.129 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_auto_mark_new_in": "true", 2024-09-21T14:58:17.129 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-09-21T14:58:17.129 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-09-21T14:58:17.129 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_cache_size": "500", 2024-09-21T14:58:17.129 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_cache_size_min": "134217728", 2024-09-21T14:58:17.129 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_crush_smoke_test": "true", 2024-09-21T14:58:17.129 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_destroyed_out_interval": "600", 2024-09-21T14:58:17.130 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_down_out_interval": "600", 2024-09-21T14:58:17.130 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-09-21T14:58:17.130 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-09-21T14:58:17.130 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_force_trim_to": "0", 2024-09-21T14:58:17.130 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_full_ratio": "0.950000", 2024-09-21T14:58:17.130 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-09-21T14:58:17.130 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_laggy_halflife": "3600", 2024-09-21T14:58:17.130 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_laggy_max_interval": "300", 2024-09-21T14:58:17.130 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_laggy_weight": "0.300000", 2024-09-21T14:58:17.130 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-09-21T14:58:17.130 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_max_creating_pgs": "1024", 2024-09-21T14:58:17.130 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_max_initial_pgs": "1024", 2024-09-21T14:58:17.130 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_min_down_reporters": "2", 2024-09-21T14:58:17.131 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-09-21T14:58:17.131 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-09-21T14:58:17.131 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-09-21T14:58:17.131 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_prime_pg_temp": "true", 2024-09-21T14:58:17.131 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-09-21T14:58:17.131 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-09-21T14:58:17.131 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_report_timeout": "900", 2024-09-21T14:58:17.131 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-09-21T14:58:17.131 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-09-21T14:58:17.131 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_warn_num_repaired": "10", 2024-09-21T14:58:17.131 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osd_warn_op_age": "32.000000", 2024-09-21T14:58:17.131 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-09-21T14:58:17.131 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osdmap_full_prune_interval": "10", 2024-09-21T14:58:17.131 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osdmap_full_prune_min": "10000", 2024-09-21T14:58:17.132 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_osdmap_full_prune_txsize": "100", 2024-09-21T14:58:17.132 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-09-21T14:58:17.132 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_pg_stuck_threshold": "60", 2024-09-21T14:58:17.132 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-09-21T14:58:17.132 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_pg_warn_min_objects": "10000", 2024-09-21T14:58:17.132 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_pg_warn_min_per_osd": "0", 2024-09-21T14:58:17.132 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-09-21T14:58:17.132 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_pool_quota_crit_threshold": "0", 2024-09-21T14:58:17.132 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_pool_quota_warn_threshold": "0", 2024-09-21T14:58:17.132 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_probe_timeout": "2.000000", 2024-09-21T14:58:17.132 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_reweight_max_change": "0.050000", 2024-09-21T14:58:17.132 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_reweight_max_osds": "4", 2024-09-21T14:58:17.132 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-09-21T14:58:17.132 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-09-21T14:58:17.132 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-09-21T14:58:17.132 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-09-21T14:58:17.133 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-09-21T14:58:17.133 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_scrub_interval": "86400", 2024-09-21T14:58:17.133 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_scrub_max_keys": "100", 2024-09-21T14:58:17.133 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_scrub_timeout": "300", 2024-09-21T14:58:17.133 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_session_timeout": "300", 2024-09-21T14:58:17.133 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_smart_report_timeout": "5", 2024-09-21T14:58:17.133 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_stat_smooth_intervals": "6", 2024-09-21T14:58:17.133 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-09-21T14:58:17.133 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_stretch_pool_min_size": "2", 2024-09-21T14:58:17.133 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_stretch_pool_size": "4", 2024-09-21T14:58:17.133 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-09-21T14:58:17.133 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_subscribe_interval": "86400.000000", 2024-09-21T14:58:17.133 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_sync_debug": "false", 2024-09-21T14:58:17.133 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_sync_max_payload_keys": "2000", 2024-09-21T14:58:17.133 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_sync_max_payload_size": "1048576", 2024-09-21T14:58:17.133 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_sync_provider_kill_at": "0", 2024-09-21T14:58:17.134 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_sync_requester_kill_at": "0", 2024-09-21T14:58:17.134 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_sync_timeout": "60.000000", 2024-09-21T14:58:17.134 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_target_pg_per_osd": "100", 2024-09-21T14:58:17.134 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_tick_interval": "5", 2024-09-21T14:58:17.134 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_timecheck_interval": "300.000000", 2024-09-21T14:58:17.134 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-09-21T14:58:17.134 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_use_min_delay_socket": "false", 2024-09-21T14:58:17.134 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_warn_older_version_delay": "604800", 2024-09-21T14:58:17.134 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-09-21T14:58:17.134 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-09-21T14:58:17.134 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-09-21T14:58:17.135 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_warn_on_filestore_osds": "false", 2024-09-21T14:58:17.135 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-09-21T14:58:17.135 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-09-21T14:58:17.135 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-09-21T14:58:17.135 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_warn_on_misplaced": "false", 2024-09-21T14:58:17.135 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-09-21T14:58:17.135 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_warn_on_older_version": "true", 2024-09-21T14:58:17.135 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-09-21T14:58:17.135 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_warn_on_pool_no_app": "false", 2024-09-21T14:58:17.135 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_warn_on_pool_no_redundancy": "false", 2024-09-21T14:58:17.135 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-09-21T14:58:17.135 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-09-21T14:58:17.135 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-09-21T14:58:17.135 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_warn_on_too_few_osds": "false", 2024-09-21T14:58:17.135 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-09-21T14:58:17.136 INFO:teuthology.orchestra.run.smithi032.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-09-21T14:58:17.136 INFO:teuthology.orchestra.run.smithi032.stdout: "monmap": "", 2024-09-21T14:58:17.136 INFO:teuthology.orchestra.run.smithi032.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-21T14:58:17.136 INFO:teuthology.orchestra.run.smithi032.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2024-09-21T14:58:17.136 INFO:teuthology.orchestra.run.smithi032.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2024-09-21T14:58:17.136 INFO:teuthology.orchestra.run.smithi032.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-21T14:58:17.136 INFO:teuthology.orchestra.run.smithi032.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2024-09-21T14:58:17.136 INFO:teuthology.orchestra.run.smithi032.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2024-09-21T14:58:17.136 INFO:teuthology.orchestra.run.smithi032.stdout: "motr_tracing_enabled": "false", 2024-09-21T14:58:17.136 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_async_op_threads": "3", 2024-09-21T14:58:17.136 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_async_rdma_buffer_size": "131072", 2024-09-21T14:58:17.136 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_async_rdma_cm": "false", 2024-09-21T14:58:17.136 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_async_rdma_device_name": "", 2024-09-21T14:58:17.136 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_async_rdma_dscp": "96", 2024-09-21T14:58:17.136 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-09-21T14:58:17.137 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_async_rdma_gid_idx": "0", 2024-09-21T14:58:17.137 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_async_rdma_local_gid": "", 2024-09-21T14:58:17.137 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_async_rdma_polling_us": "1000", 2024-09-21T14:58:17.137 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_async_rdma_port_num": "1", 2024-09-21T14:58:17.137 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-09-21T14:58:17.137 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-09-21T14:58:17.137 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_async_rdma_roce_ver": "1", 2024-09-21T14:58:17.137 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_async_rdma_send_buffers": "1024", 2024-09-21T14:58:17.137 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_async_rdma_sl": "3", 2024-09-21T14:58:17.137 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_async_rdma_support_srq": "true", 2024-09-21T14:58:17.137 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_async_rdma_type": "ib", 2024-09-21T14:58:17.137 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_async_reap_threshold": "5", 2024-09-21T14:58:17.137 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_bind_before_connect": "false", 2024-09-21T14:58:17.137 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_bind_ipv4": "true", 2024-09-21T14:58:17.137 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_bind_ipv6": "false", 2024-09-21T14:58:17.138 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_bind_msgr1": "true", 2024-09-21T14:58:17.138 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_bind_msgr2": "true", 2024-09-21T14:58:17.138 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_bind_port_max": "7568", 2024-09-21T14:58:17.138 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_bind_port_min": "6800", 2024-09-21T14:58:17.138 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_bind_prefer_ipv4": "false", 2024-09-21T14:58:17.138 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_bind_retry_count": "3", 2024-09-21T14:58:17.138 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_bind_retry_delay": "5", 2024-09-21T14:58:17.138 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_blackhole_client": "false", 2024-09-21T14:58:17.138 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_blackhole_mds": "false", 2024-09-21T14:58:17.138 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_blackhole_mgr": "false", 2024-09-21T14:58:17.138 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_blackhole_mon": "false", 2024-09-21T14:58:17.138 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_blackhole_osd": "false", 2024-09-21T14:58:17.138 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_client_mode": "crc secure", 2024-09-21T14:58:17.138 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_cluster_mode": "crc secure", 2024-09-21T14:58:17.139 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_cluster_type": "", 2024-09-21T14:58:17.139 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_compress_secure": "false", 2024-09-21T14:58:17.139 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_connection_idle_timeout": "900", 2024-09-21T14:58:17.139 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_connection_ready_timeout": "10", 2024-09-21T14:58:17.139 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_crc_data": "true", 2024-09-21T14:58:17.139 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_crc_header": "true", 2024-09-21T14:58:17.139 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_die_on_bad_msg": "false", 2024-09-21T14:58:17.139 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_die_on_bug": "true", 2024-09-21T14:58:17.139 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_die_on_old_message": "true", 2024-09-21T14:58:17.139 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_die_on_skipped_message": "false", 2024-09-21T14:58:17.139 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_die_on_unhandled_msg": "false", 2024-09-21T14:58:17.139 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-09-21T14:58:17.139 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_dpdk_coremask": "0xF", 2024-09-21T14:58:17.139 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-09-21T14:58:17.139 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_dpdk_devs_allowlist": "", 2024-09-21T14:58:17.140 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_dpdk_enable_tso": "true", 2024-09-21T14:58:17.140 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-09-21T14:58:17.140 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-09-21T14:58:17.140 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_dpdk_hugepages": "", 2024-09-21T14:58:17.140 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_dpdk_hw_flow_control": "true", 2024-09-21T14:58:17.140 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-09-21T14:58:17.140 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_dpdk_lro": "true", 2024-09-21T14:58:17.140 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_dpdk_memory_channel": "4", 2024-09-21T14:58:17.140 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-09-21T14:58:17.140 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_dpdk_pmd": "", 2024-09-21T14:58:17.140 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_dpdk_port_id": "0", 2024-09-21T14:58:17.140 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-09-21T14:58:17.140 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_dump_corrupt_message_level": "1", 2024-09-21T14:58:17.140 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_dump_on_send": "false", 2024-09-21T14:58:17.140 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_initial_backoff": "0.200000", 2024-09-21T14:58:17.141 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_inject_delay_max": "1.000000", 2024-09-21T14:58:17.141 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_inject_delay_probability": "0.000000", 2024-09-21T14:58:17.141 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_inject_delay_type": "", 2024-09-21T14:58:17.141 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_inject_internal_delays": "0.000000", 2024-09-21T14:58:17.141 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_inject_network_congestion": "0", 2024-09-21T14:58:17.141 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_inject_socket_failures": "5000", 2024-09-21T14:58:17.141 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_learn_addr_from_peer": "true", 2024-09-21T14:58:17.141 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_max_accept_failures": "4", 2024-09-21T14:58:17.141 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_max_backoff": "15.000000", 2024-09-21T14:58:17.141 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_mon_client_mode": "secure crc", 2024-09-21T14:58:17.141 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_mon_cluster_mode": "secure crc", 2024-09-21T14:58:17.141 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_mon_service_mode": "secure crc", 2024-09-21T14:58:17.141 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_osd_compress_min_size": "1024", 2024-09-21T14:58:17.141 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_osd_compress_mode": "none", 2024-09-21T14:58:17.141 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_osd_compression_algorithm": "snappy", 2024-09-21T14:58:17.141 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-09-21T14:58:17.142 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_pq_min_cost": "65536", 2024-09-21T14:58:17.142 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_public_type": "", 2024-09-21T14:58:17.142 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_service_mode": "crc secure", 2024-09-21T14:58:17.142 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_tcp_listen_backlog": "512", 2024-09-21T14:58:17.142 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_tcp_nodelay": "true", 2024-09-21T14:58:17.142 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-09-21T14:58:17.142 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_tcp_rcvbuf": "0", 2024-09-21T14:58:17.142 INFO:teuthology.orchestra.run.smithi032.stdout: "ms_type": "async+posix", 2024-09-21T14:58:17.142 INFO:teuthology.orchestra.run.smithi032.stdout: "no_config_file": "false", 2024-09-21T14:58:17.142 INFO:teuthology.orchestra.run.smithi032.stdout: "objecter_completion_locks_per_session": "32", 2024-09-21T14:58:17.142 INFO:teuthology.orchestra.run.smithi032.stdout: "objecter_debug_inject_relock_delay": "false", 2024-09-21T14:58:17.142 INFO:teuthology.orchestra.run.smithi032.stdout: "objecter_inflight_op_bytes": "104857600", 2024-09-21T14:58:17.142 INFO:teuthology.orchestra.run.smithi032.stdout: "objecter_inflight_ops": "1024", 2024-09-21T14:58:17.142 INFO:teuthology.orchestra.run.smithi032.stdout: "objecter_inject_no_watch_ping": "false", 2024-09-21T14:58:17.143 INFO:teuthology.orchestra.run.smithi032.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-09-21T14:58:17.143 INFO:teuthology.orchestra.run.smithi032.stdout: "objecter_tick_interval": "5.000000", 2024-09-21T14:58:17.143 INFO:teuthology.orchestra.run.smithi032.stdout: "objecter_timeout": "10.000000", 2024-09-21T14:58:17.143 INFO:teuthology.orchestra.run.smithi032.stdout: "objectstore_blackhole": "false", 2024-09-21T14:58:17.143 INFO:teuthology.orchestra.run.smithi032.stdout: "openssl_engine_opts": "", 2024-09-21T14:58:17.143 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_agent_delay_time": "5.000000", 2024-09-21T14:58:17.143 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_agent_hist_halflife": "1000", 2024-09-21T14:58:17.143 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_agent_max_low_ops": "2", 2024-09-21T14:58:17.143 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_agent_max_ops": "4", 2024-09-21T14:58:17.143 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-09-21T14:58:17.143 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_agent_quantize_effort": "0.100000", 2024-09-21T14:58:17.143 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_agent_slop": "0.020000", 2024-09-21T14:58:17.143 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-09-21T14:58:17.143 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_allow_recovery_below_min_size": "true", 2024-09-21T14:58:17.143 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_asio_thread_count": "2", 2024-09-21T14:58:17.144 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_async_recovery_min_cost": "100", 2024-09-21T14:58:17.144 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_auto_mark_unfound_lost": "false", 2024-09-21T14:58:17.144 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_backfill_retry_interval": "30.000000", 2024-09-21T14:58:17.144 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_backfill_scan_max": "512", 2024-09-21T14:58:17.144 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_backfill_scan_min": "64", 2024-09-21T14:58:17.144 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_backoff_on_degraded": "false", 2024-09-21T14:58:17.144 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_backoff_on_peering": "false", 2024-09-21T14:58:17.144 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_backoff_on_unfound": "true", 2024-09-21T14:58:17.144 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_beacon_report_interval": "300", 2024-09-21T14:58:17.144 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_bench_duration": "30", 2024-09-21T14:58:17.144 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-09-21T14:58:17.144 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_bench_max_block_size": "67108864", 2024-09-21T14:58:17.144 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_bench_small_size_max_iops": "100", 2024-09-21T14:58:17.144 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_blkin_trace_all": "false", 2024-09-21T14:58:17.144 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_blocked_scrub_grace_period": "120", 2024-09-21T14:58:17.145 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-09-21T14:58:17.145 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2024-09-21T14:58:17.145 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-09-21T14:58:17.145 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_check_for_log_corruption": "false", 2024-09-21T14:58:17.145 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-09-21T14:58:17.145 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-21T14:58:17.145 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_class_dir": "/usr/lib/rados-classes", 2024-09-21T14:58:17.145 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-21T14:58:17.145 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_class_update_on_start": "true", 2024-09-21T14:58:17.145 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_client_message_cap": "256", 2024-09-21T14:58:17.145 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_client_message_size_cap": "524288000", 2024-09-21T14:58:17.145 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_client_op_priority": "63", 2024-09-21T14:58:17.145 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_client_watch_timeout": "30", 2024-09-21T14:58:17.145 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_command_max_records": "256", 2024-09-21T14:58:17.145 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_compact_on_start": "false", 2024-09-21T14:58:17.146 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-09-21T14:58:17.146 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_crush_chooseleaf_type": "0", 2024-09-21T14:58:17.146 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_crush_initial_weight": "-1.000000", 2024-09-21T14:58:17.146 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_crush_update_on_start": "true", 2024-09-21T14:58:17.146 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_crush_update_weight_set": "true", 2024-09-21T14:58:17.146 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-09-21T14:58:17.146 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-09-21T14:58:17.146 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-09-21T14:58:17.146 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_debug_drop_ping_duration": "0", 2024-09-21T14:58:17.146 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-09-21T14:58:17.146 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_debug_feed_pullee": "-1", 2024-09-21T14:58:17.146 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-09-21T14:58:17.146 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-09-21T14:58:17.146 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-09-21T14:58:17.146 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_debug_misdirected_ops": "false", 2024-09-21T14:58:17.147 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_debug_no_acting_change": "false", 2024-09-21T14:58:17.147 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_debug_no_purge_strays": "false", 2024-09-21T14:58:17.147 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_debug_op_order": "false", 2024-09-21T14:58:17.147 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_debug_pg_log_writeout": "false", 2024-09-21T14:58:17.147 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_debug_pretend_recovery_active": "false", 2024-09-21T14:58:17.147 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-09-21T14:58:17.147 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-09-21T14:58:17.147 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_debug_shutdown": "false", 2024-09-21T14:58:17.147 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-09-21T14:58:17.147 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-09-21T14:58:17.147 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_debug_verify_cached_snaps": "false", 2024-09-21T14:58:17.147 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_debug_verify_missing_on_start": "false", 2024-09-21T14:58:17.147 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_debug_verify_snaps": "false", 2024-09-21T14:58:17.147 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-09-21T14:58:17.148 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-09-21T14:58:17.148 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_deep_scrub_keys": "1024", 2024-09-21T14:58:17.148 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-09-21T14:58:17.148 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-09-21T14:58:17.148 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-09-21T14:58:17.148 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_deep_scrub_stride": "524288", 2024-09-21T14:58:17.148 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-09-21T14:58:17.148 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_default_data_pool_replay_window": "5", 2024-09-21T14:58:17.148 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_default_notify_timeout": "30", 2024-09-21T14:58:17.148 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_delete_sleep": "0.000000", 2024-09-21T14:58:17.148 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-09-21T14:58:17.148 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-09-21T14:58:17.148 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-09-21T14:58:17.148 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_discard_disconnected_ops": "true", 2024-09-21T14:58:17.148 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_enable_op_tracker": "true", 2024-09-21T14:58:17.148 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-09-21T14:58:17.149 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_extblkdev_plugins": "vdo", 2024-09-21T14:58:17.149 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-09-21T14:58:17.149 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-09-21T14:58:17.149 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_fast_info": "true", 2024-09-21T14:58:17.149 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_fast_shutdown": "true", 2024-09-21T14:58:17.149 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-09-21T14:58:17.149 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_fast_shutdown_timeout": "15", 2024-09-21T14:58:17.149 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-09-21T14:58:17.149 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-09-21T14:58:17.149 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-09-21T14:58:17.149 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_function_tracing": "false", 2024-09-21T14:58:17.149 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_heartbeat_grace": "20", 2024-09-21T14:58:17.149 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_heartbeat_interval": "6", 2024-09-21T14:58:17.149 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-09-21T14:58:17.149 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_heartbeat_min_peers": "10", 2024-09-21T14:58:17.150 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_heartbeat_min_size": "2000", 2024-09-21T14:58:17.150 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_heartbeat_stale": "600", 2024-09-21T14:58:17.150 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-09-21T14:58:17.150 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_hit_set_max_size": "100000", 2024-09-21T14:58:17.150 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_hit_set_min_size": "1000", 2024-09-21T14:58:17.150 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-09-21T14:58:17.150 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-09-21T14:58:17.150 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-09-21T14:58:17.150 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-09-21T14:58:17.150 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-09-21T14:58:17.150 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_journal_flush_on_shutdown": "true", 2024-09-21T14:58:17.150 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_journal_size": "5120", 2024-09-21T14:58:17.150 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_kill_backfill_at": "0", 2024-09-21T14:58:17.150 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_loop_before_reset_tphandle": "64", 2024-09-21T14:58:17.151 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_map_cache_size": "50", 2024-09-21T14:58:17.151 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_map_dedup": "true", 2024-09-21T14:58:17.151 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_map_message_max": "40", 2024-09-21T14:58:17.151 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_map_message_max_bytes": "10485760", 2024-09-21T14:58:17.151 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_map_share_max_epochs": "40", 2024-09-21T14:58:17.151 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_max_attr_name_len": "100", 2024-09-21T14:58:17.151 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_max_attr_size": "0", 2024-09-21T14:58:17.151 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_max_backfills": "1", 2024-09-21T14:58:17.151 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_max_markdown_count": "5", 2024-09-21T14:58:17.151 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_max_markdown_period": "600", 2024-09-21T14:58:17.151 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_max_object_name_len": "2048", 2024-09-21T14:58:17.151 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_max_object_namespace_len": "256", 2024-09-21T14:58:17.151 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_max_object_size": "134217728", 2024-09-21T14:58:17.151 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-09-21T14:58:17.151 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_max_omap_entries_per_request": "1024", 2024-09-21T14:58:17.152 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_max_pg_blocked_by": "16", 2024-09-21T14:58:17.152 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_max_pg_log_entries": "10000", 2024-09-21T14:58:17.152 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-09-21T14:58:17.152 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_max_pgls": "1024", 2024-09-21T14:58:17.152 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_max_push_cost": "8388608", 2024-09-21T14:58:17.152 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_max_push_objects": "10", 2024-09-21T14:58:17.152 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_max_scrubs": "3", 2024-09-21T14:58:17.152 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-09-21T14:58:17.152 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_max_trimming_pgs": "2", 2024-09-21T14:58:17.152 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_max_write_op_reply_len": "64", 2024-09-21T14:58:17.152 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_max_write_size": "90", 2024-09-21T14:58:17.152 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-09-21T14:58:17.152 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-09-21T14:58:17.152 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-09-21T14:58:17.152 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-09-21T14:58:17.153 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-09-21T14:58:17.153 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-09-21T14:58:17.153 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-09-21T14:58:17.153 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mclock_override_recovery_settings": "false", 2024-09-21T14:58:17.153 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mclock_profile": "balanced", 2024-09-21T14:58:17.153 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-09-21T14:58:17.153 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-09-21T14:58:17.153 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-09-21T14:58:17.153 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-09-21T14:58:17.153 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-09-21T14:58:17.153 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-09-21T14:58:17.153 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-09-21T14:58:17.153 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-09-21T14:58:17.153 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-09-21T14:58:17.154 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-09-21T14:58:17.154 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mclock_skip_benchmark": "false", 2024-09-21T14:58:17.154 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_memory_base": "805306368", 2024-09-21T14:58:17.154 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_memory_cache_min": "134217728", 2024-09-21T14:58:17.154 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-09-21T14:58:17.154 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-09-21T14:58:17.154 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_memory_target": "4294967296", 2024-09-21T14:58:17.154 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_memory_target_autotune": "false", 2024-09-21T14:58:17.154 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-09-21T14:58:17.154 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_min_pg_log_entries": "250", 2024-09-21T14:58:17.154 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_min_recovery_priority": "0", 2024-09-21T14:58:17.154 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mon_heartbeat_interval": "30", 2024-09-21T14:58:17.154 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-09-21T14:58:17.155 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mon_report_interval": "5", 2024-09-21T14:58:17.155 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mon_report_max_in_flight": "2", 2024-09-21T14:58:17.155 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-09-21T14:58:17.155 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_num_cache_shards": "32", 2024-09-21T14:58:17.155 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_num_op_tracker_shard": "32", 2024-09-21T14:58:17.155 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_numa_auto_affinity": "true", 2024-09-21T14:58:17.155 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_numa_node": "-1", 2024-09-21T14:58:17.155 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_numa_prefer_iface": "true", 2024-09-21T14:58:17.155 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-09-21T14:58:17.155 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_objecter_finishers": "1", 2024-09-21T14:58:17.155 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_objectstore": "bluestore", 2024-09-21T14:58:17.155 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_objectstore_fuse": "false", 2024-09-21T14:58:17.155 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_objectstore_tracing": "false", 2024-09-21T14:58:17.155 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_op_complaint_time": "30.000000", 2024-09-21T14:58:17.155 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_op_history_duration": "600", 2024-09-21T14:58:17.156 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_op_history_size": "20", 2024-09-21T14:58:17.156 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_op_history_slow_op_size": "20", 2024-09-21T14:58:17.156 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-09-21T14:58:17.156 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_op_log_threshold": "5", 2024-09-21T14:58:17.156 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_op_num_shards": "0", 2024-09-21T14:58:17.156 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_op_num_shards_hdd": "5", 2024-09-21T14:58:17.156 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_op_num_shards_ssd": "8", 2024-09-21T14:58:17.156 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_op_num_threads_per_shard": "0", 2024-09-21T14:58:17.156 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-09-21T14:58:17.156 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-09-21T14:58:17.156 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-09-21T14:58:17.156 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_op_pq_min_cost": "65536", 2024-09-21T14:58:17.156 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_op_queue": "mclock_scheduler", 2024-09-21T14:58:17.156 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_op_queue_cut_off": "high", 2024-09-21T14:58:17.156 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_op_thread_suicide_timeout": "150", 2024-09-21T14:58:17.157 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_op_thread_timeout": "15", 2024-09-21T14:58:17.157 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_open_classes_on_start": "true", 2024-09-21T14:58:17.157 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_os_flags": "0", 2024-09-21T14:58:17.157 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_peering_op_priority": "255", 2024-09-21T14:58:17.157 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pg_delete_cost": "1048576", 2024-09-21T14:58:17.157 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pg_delete_priority": "5", 2024-09-21T14:58:17.157 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-09-21T14:58:17.157 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-09-21T14:58:17.157 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pg_log_dups_tracked": "3000", 2024-09-21T14:58:17.157 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pg_log_trim_max": "10000", 2024-09-21T14:58:17.157 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pg_log_trim_min": "100", 2024-09-21T14:58:17.157 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-09-21T14:58:17.157 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pg_object_context_cache_count": "64", 2024-09-21T14:58:17.157 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pg_stat_report_interval_max": "500", 2024-09-21T14:58:17.157 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-09-21T14:58:17.157 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-09-21T14:58:17.158 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-09-21T14:58:17.158 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-09-21T14:58:17.158 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-09-21T14:58:17.158 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-09-21T14:58:17.158 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pool_default_crimson": "false", 2024-09-21T14:58:17.158 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pool_default_crush_rule": "-1", 2024-09-21T14:58:17.158 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pool_default_ec_fast_read": "false", 2024-09-21T14:58:17.158 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-09-21T14:58:17.158 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pool_default_flag_bulk": "false", 2024-09-21T14:58:17.158 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-09-21T14:58:17.159 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pool_default_flag_nodelete": "false", 2024-09-21T14:58:17.159 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-09-21T14:58:17.159 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-09-21T14:58:17.159 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pool_default_flags": "0", 2024-09-21T14:58:17.159 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-09-21T14:58:17.159 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pool_default_min_size": "0", 2024-09-21T14:58:17.159 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-09-21T14:58:17.159 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pool_default_pg_num": "32", 2024-09-21T14:58:17.159 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pool_default_pgp_num": "0", 2024-09-21T14:58:17.159 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-09-21T14:58:17.159 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pool_default_size": "2", 2024-09-21T14:58:17.160 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pool_default_type": "replicated", 2024-09-21T14:58:17.160 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-09-21T14:58:17.160 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pool_use_gmt_hitset": "true", 2024-09-21T14:58:17.160 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_push_per_object_cost": "1000", 2024-09-21T14:58:17.160 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_read_ec_check_for_errors": "false", 2024-09-21T14:58:17.160 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_recover_clone_overlap": "true", 2024-09-21T14:58:17.160 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_recover_clone_overlap_limit": "10", 2024-09-21T14:58:17.160 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_recovery_cost": "20971520", 2024-09-21T14:58:17.160 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_recovery_delay_start": "0.000000", 2024-09-21T14:58:17.160 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_recovery_max_active": "0", 2024-09-21T14:58:17.160 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_recovery_max_active_hdd": "3", 2024-09-21T14:58:17.160 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_recovery_max_active_ssd": "10", 2024-09-21T14:58:17.160 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_recovery_max_chunk": "8388608", 2024-09-21T14:58:17.161 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-09-21T14:58:17.161 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_recovery_max_single_start": "1", 2024-09-21T14:58:17.161 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_recovery_op_priority": "3", 2024-09-21T14:58:17.161 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_recovery_op_warn_multiple": "16", 2024-09-21T14:58:17.161 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_recovery_priority": "5", 2024-09-21T14:58:17.161 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_recovery_retry_interval": "30.000000", 2024-09-21T14:58:17.161 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_recovery_sleep": "0.000000", 2024-09-21T14:58:17.161 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-09-21T14:58:17.161 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-09-21T14:58:17.161 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-09-21T14:58:17.161 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_repair_during_recovery": "false", 2024-09-21T14:58:17.161 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_requested_scrub_priority": "120", 2024-09-21T14:58:17.161 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-09-21T14:58:17.162 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_rollback_to_cluster_snap": "", 2024-09-21T14:58:17.162 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_scrub_auto_repair": "false", 2024-09-21T14:58:17.162 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-09-21T14:58:17.162 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-09-21T14:58:17.162 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_scrub_begin_hour": "0", 2024-09-21T14:58:17.162 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_scrub_begin_week_day": "0", 2024-09-21T14:58:17.162 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_scrub_chunk_max": "25", 2024-09-21T14:58:17.162 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_scrub_chunk_min": "5", 2024-09-21T14:58:17.162 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_scrub_cost": "52428800", 2024-09-21T14:58:17.162 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_scrub_during_recovery": "false", 2024-09-21T14:58:17.162 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_scrub_end_hour": "0", 2024-09-21T14:58:17.162 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_scrub_end_week_day": "0", 2024-09-21T14:58:17.163 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_scrub_event_cost": "4096", 2024-09-21T14:58:17.163 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-09-21T14:58:17.163 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-09-21T14:58:17.163 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_scrub_invalid_stats": "true", 2024-09-21T14:58:17.163 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_scrub_load_threshold": "0.500000", 2024-09-21T14:58:17.163 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_scrub_max_interval": "604800.000000", 2024-09-21T14:58:17.163 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_scrub_max_preemptions": "5", 2024-09-21T14:58:17.163 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_scrub_min_interval": "86400.000000", 2024-09-21T14:58:17.163 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_scrub_priority": "5", 2024-09-21T14:58:17.163 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_scrub_reservation_timeout": "5000", 2024-09-21T14:58:17.163 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_scrub_sleep": "0.000000", 2024-09-21T14:58:17.163 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_scrub_slow_reservation_response": "2200", 2024-09-21T14:58:17.163 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_shallow_scrub_chunk_max": "100", 2024-09-21T14:58:17.163 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_shallow_scrub_chunk_min": "50", 2024-09-21T14:58:17.164 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_shutdown_pgref_assert": "false", 2024-09-21T14:58:17.164 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_skip_data_digest": "false", 2024-09-21T14:58:17.164 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_smart_report_timeout": "5", 2024-09-21T14:58:17.164 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_snap_trim_cost": "1048576", 2024-09-21T14:58:17.164 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_snap_trim_priority": "5", 2024-09-21T14:58:17.164 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_snap_trim_sleep": "0.000000", 2024-09-21T14:58:17.164 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-09-21T14:58:17.164 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-09-21T14:58:17.164 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-09-21T14:58:17.164 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_stats_update_period_not_scrubbing": "120", 2024-09-21T14:58:17.164 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_stats_update_period_scrubbing": "15", 2024-09-21T14:58:17.164 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-09-21T14:58:17.164 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_target_transaction_size": "30", 2024-09-21T14:58:17.164 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-09-21T14:58:17.165 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-09-21T14:58:17.165 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-09-21T14:58:17.165 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-09-21T14:58:17.165 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-09-21T14:58:17.165 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-09-21T14:58:17.165 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-09-21T14:58:17.165 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_tier_default_cache_mode": "writeback", 2024-09-21T14:58:17.165 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-09-21T14:58:17.165 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-09-21T14:58:17.165 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_tracing": "false", 2024-09-21T14:58:17.165 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_use_stale_snap": "false", 2024-09-21T14:58:17.165 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-09-21T14:58:17.165 INFO:teuthology.orchestra.run.smithi032.stdout: "osdc_blkin_trace_all": "false", 2024-09-21T14:58:17.165 INFO:teuthology.orchestra.run.smithi032.stdout: "paxos_kill_at": "0", 2024-09-21T14:58:17.166 INFO:teuthology.orchestra.run.smithi032.stdout: "paxos_max_join_drift": "10", 2024-09-21T14:58:17.166 INFO:teuthology.orchestra.run.smithi032.stdout: "paxos_min": "500", 2024-09-21T14:58:17.166 INFO:teuthology.orchestra.run.smithi032.stdout: "paxos_min_wait": "0.050000", 2024-09-21T14:58:17.166 INFO:teuthology.orchestra.run.smithi032.stdout: "paxos_propose_interval": "1.000000", 2024-09-21T14:58:17.166 INFO:teuthology.orchestra.run.smithi032.stdout: "paxos_service_trim_max": "500", 2024-09-21T14:58:17.166 INFO:teuthology.orchestra.run.smithi032.stdout: "paxos_service_trim_max_multiplier": "20", 2024-09-21T14:58:17.166 INFO:teuthology.orchestra.run.smithi032.stdout: "paxos_service_trim_min": "250", 2024-09-21T14:58:17.166 INFO:teuthology.orchestra.run.smithi032.stdout: "paxos_stash_full_interval": "25", 2024-09-21T14:58:17.166 INFO:teuthology.orchestra.run.smithi032.stdout: "paxos_trim_max": "500", 2024-09-21T14:58:17.166 INFO:teuthology.orchestra.run.smithi032.stdout: "paxos_trim_min": "250", 2024-09-21T14:58:17.166 INFO:teuthology.orchestra.run.smithi032.stdout: "perf": "true", 2024-09-21T14:58:17.166 INFO:teuthology.orchestra.run.smithi032.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2024-09-21T14:58:17.166 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-09-21T14:58:17.167 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin_dir": "/usr/lib/ceph", 2024-09-21T14:58:17.167 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "-", 2024-09-21T14:58:17.167 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrv": "", 2024-09-21T14:58:17.167 INFO:teuthology.orchestra.run.smithi032.stdout: "public_bind_addr": "-", 2024-09-21T14:58:17.167 INFO:teuthology.orchestra.run.smithi032.stdout: "public_network": "", 2024-09-21T14:58:17.167 INFO:teuthology.orchestra.run.smithi032.stdout: "public_network_interface": "", 2024-09-21T14:58:17.167 INFO:teuthology.orchestra.run.smithi032.stdout: "qat_compressor_enabled": "false", 2024-09-21T14:58:17.167 INFO:teuthology.orchestra.run.smithi032.stdout: "qat_compressor_session_max_number": "256", 2024-09-21T14:58:17.167 INFO:teuthology.orchestra.run.smithi032.stdout: "rados_mon_op_timeout": "0", 2024-09-21T14:58:17.167 INFO:teuthology.orchestra.run.smithi032.stdout: "rados_osd_op_timeout": "0", 2024-09-21T14:58:17.167 INFO:teuthology.orchestra.run.smithi032.stdout: "rados_tracing": "false", 2024-09-21T14:58:17.167 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_atime_update_interval": "60", 2024-09-21T14:58:17.167 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-09-21T14:58:17.167 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_balance_parent_reads": "false", 2024-09-21T14:58:17.168 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_balance_snap_reads": "false", 2024-09-21T14:58:17.168 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_blkin_trace_all": "false", 2024-09-21T14:58:17.168 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_blocklist_expire_seconds": "0", 2024-09-21T14:58:17.168 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_blocklist_on_break_lock": "true", 2024-09-21T14:58:17.168 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_cache": "true", 2024-09-21T14:58:17.168 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_cache_block_writes_upfront": "false", 2024-09-21T14:58:17.168 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_cache_max_dirty": "25165824", 2024-09-21T14:58:17.168 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-09-21T14:58:17.168 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_cache_max_dirty_object": "0", 2024-09-21T14:58:17.168 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_cache_policy": "writearound", 2024-09-21T14:58:17.168 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_cache_size": "33554432", 2024-09-21T14:58:17.168 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_cache_target_dirty": "16777216", 2024-09-21T14:58:17.168 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-09-21T14:58:17.168 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_clone_copy_on_read": "false", 2024-09-21T14:58:17.169 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_compression_hint": "none", 2024-09-21T14:58:17.169 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_concurrent_management_ops": "10", 2024-09-21T14:58:17.169 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-09-21T14:58:17.169 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_default_clone_format": "auto", 2024-09-21T14:58:17.169 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_default_data_pool": "", 2024-09-21T14:58:17.169 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_default_features": "61", 2024-09-21T14:58:17.169 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_default_format": "2", 2024-09-21T14:58:17.169 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_default_map_options": "", 2024-09-21T14:58:17.169 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_default_order": "22", 2024-09-21T14:58:17.169 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_default_pool": "rbd", 2024-09-21T14:58:17.169 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-09-21T14:58:17.170 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_default_stripe_count": "0", 2024-09-21T14:58:17.170 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_default_stripe_unit": "0", 2024-09-21T14:58:17.170 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_disable_zero_copy_writes": "true", 2024-09-21T14:58:17.170 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_discard_granularity_bytes": "65536", 2024-09-21T14:58:17.170 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-09-21T14:58:17.170 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_enable_alloc_hint": "true", 2024-09-21T14:58:17.170 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-09-21T14:58:17.170 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_io_scheduler": "simple", 2024-09-21T14:58:17.170 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-09-21T14:58:17.170 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_journal_commit_age": "5.000000", 2024-09-21T14:58:17.170 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-09-21T14:58:17.170 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-09-21T14:58:17.170 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-09-21T14:58:17.170 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-09-21T14:58:17.171 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_journal_object_flush_interval": "0", 2024-09-21T14:58:17.171 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-09-21T14:58:17.171 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-09-21T14:58:17.171 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_journal_order": "24", 2024-09-21T14:58:17.171 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_journal_pool": "", 2024-09-21T14:58:17.171 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_journal_splay_width": "4", 2024-09-21T14:58:17.171 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_localize_parent_reads": "false", 2024-09-21T14:58:17.171 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_localize_snap_reads": "false", 2024-09-21T14:58:17.171 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-09-21T14:58:17.171 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-09-21T14:58:17.171 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-09-21T14:58:17.171 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-09-21T14:58:17.171 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-09-21T14:58:17.171 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-09-21T14:58:17.171 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_mirror_image_policy_type": "simple", 2024-09-21T14:58:17.172 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-09-21T14:58:17.172 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-09-21T14:58:17.172 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-09-21T14:58:17.172 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-09-21T14:58:17.172 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-09-21T14:58:17.172 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-09-21T14:58:17.172 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-09-21T14:58:17.172 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_mirror_memory_autotune": "true", 2024-09-21T14:58:17.172 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_mirror_memory_base": "805306368", 2024-09-21T14:58:17.172 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-09-21T14:58:17.172 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-09-21T14:58:17.172 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-09-21T14:58:17.172 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-09-21T14:58:17.172 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_mirror_memory_target": "4294967296", 2024-09-21T14:58:17.172 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-09-21T14:58:17.173 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-09-21T14:58:17.173 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-09-21T14:58:17.173 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_mirroring_delete_delay": "0", 2024-09-21T14:58:17.173 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-09-21T14:58:17.173 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_mirroring_replay_delay": "0", 2024-09-21T14:58:17.173 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-09-21T14:58:17.173 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-09-21T14:58:17.173 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_move_to_trash_on_remove": "false", 2024-09-21T14:58:17.173 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-09-21T14:58:17.173 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_mtime_update_interval": "60", 2024-09-21T14:58:17.173 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_non_blocking_aio": "true", 2024-09-21T14:58:17.173 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_op_thread_timeout": "60", 2024-09-21T14:58:17.173 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_op_threads": "1", 2024-09-21T14:58:17.173 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_parent_cache_enabled": "false", 2024-09-21T14:58:17.173 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_persistent_cache_mode": "disabled", 2024-09-21T14:58:17.174 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_persistent_cache_path": "/tmp", 2024-09-21T14:58:17.174 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_persistent_cache_size": "1073741824", 2024-09-21T14:58:17.174 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_plugins": "", 2024-09-21T14:58:17.174 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_qos_bps_burst": "0", 2024-09-21T14:58:17.174 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-09-21T14:58:17.174 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_qos_bps_limit": "0", 2024-09-21T14:58:17.174 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_qos_exclude_ops": "0", 2024-09-21T14:58:17.174 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_qos_iops_burst": "0", 2024-09-21T14:58:17.174 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-09-21T14:58:17.174 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_qos_iops_limit": "0", 2024-09-21T14:58:17.174 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_qos_read_bps_burst": "0", 2024-09-21T14:58:17.174 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-09-21T14:58:17.174 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_qos_read_bps_limit": "0", 2024-09-21T14:58:17.174 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_qos_read_iops_burst": "0", 2024-09-21T14:58:17.175 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-09-21T14:58:17.175 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_qos_read_iops_limit": "0", 2024-09-21T14:58:17.175 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_qos_schedule_tick_min": "50", 2024-09-21T14:58:17.175 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_qos_write_bps_burst": "0", 2024-09-21T14:58:17.175 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-09-21T14:58:17.175 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_qos_write_bps_limit": "0", 2024-09-21T14:58:17.175 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_qos_write_iops_burst": "0", 2024-09-21T14:58:17.175 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-09-21T14:58:17.175 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_qos_write_iops_limit": "0", 2024-09-21T14:58:17.175 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_quiesce_notification_attempts": "10", 2024-09-21T14:58:17.175 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_read_from_replica_policy": "default", 2024-09-21T14:58:17.175 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-09-21T14:58:17.175 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_readahead_max_bytes": "524288", 2024-09-21T14:58:17.175 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_readahead_trigger_requests": "10", 2024-09-21T14:58:17.176 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_request_timed_out_seconds": "30", 2024-09-21T14:58:17.176 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_skip_partial_discard": "true", 2024-09-21T14:58:17.176 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-09-21T14:58:17.176 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_tracing": "false", 2024-09-21T14:58:17.176 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_validate_names": "true", 2024-09-21T14:58:17.176 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_validate_pool": "true", 2024-09-21T14:58:17.176 INFO:teuthology.orchestra.run.smithi032.stdout: "restapi_base_url": "", 2024-09-21T14:58:17.176 INFO:teuthology.orchestra.run.smithi032.stdout: "restapi_log_level": "", 2024-09-21T14:58:17.176 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_acl_grants_max_num": "100", 2024-09-21T14:58:17.176 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_admin_entry": "admin", 2024-09-21T14:58:17.176 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2024-09-21T14:58:17.176 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_backend_store": "rados", 2024-09-21T14:58:17.176 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_barbican_url": "", 2024-09-21T14:58:17.176 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_beast_enable_async": "true", 2024-09-21T14:58:17.177 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-09-21T14:58:17.177 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-09-21T14:58:17.177 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_bucket_index_max_aio": "128", 2024-09-21T14:58:17.177 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2024-09-21T14:58:17.177 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-09-21T14:58:17.177 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_bucket_quota_ttl": "600", 2024-09-21T14:58:17.177 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_bucket_sync_spawn_window": "20", 2024-09-21T14:58:17.177 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_cache_enabled": "true", 2024-09-21T14:58:17.177 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_cache_expiry_interval": "900", 2024-09-21T14:58:17.177 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_cache_lru_size": "10000", 2024-09-21T14:58:17.177 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_config_store": "rados", 2024-09-21T14:58:17.177 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_content_length_compat": "false", 2024-09-21T14:58:17.177 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_copy_obj_progress": "true", 2024-09-21T14:58:17.177 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-09-21T14:58:17.177 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_cors_rules_max_num": "100", 2024-09-21T14:58:17.177 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_cross_domain_policy": "", 2024-09-21T14:58:17.178 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_default_encryption_key": "", 2024-09-21T14:58:17.178 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_kmip_addr": "", 2024-09-21T14:58:17.178 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_kmip_ca_path": "", 2024-09-21T14:58:17.178 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_kmip_client_cert": "", 2024-09-21T14:58:17.178 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_kmip_client_key": "", 2024-09-21T14:58:17.178 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-09-21T14:58:17.178 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_kmip_password": "", 2024-09-21T14:58:17.178 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-09-21T14:58:17.178 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_kmip_username": "", 2024-09-21T14:58:17.178 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_require_ssl": "true", 2024-09-21T14:58:17.178 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-09-21T14:58:17.178 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-09-21T14:58:17.178 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-09-21T14:58:17.178 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-09-21T14:58:17.179 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-09-21T14:58:17.179 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-09-21T14:58:17.179 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-09-21T14:58:17.179 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-09-21T14:58:17.179 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-09-21T14:58:17.179 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-09-21T14:58:17.179 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-09-21T14:58:17.179 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-09-21T14:58:17.179 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-09-21T14:58:17.179 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-09-21T14:58:17.179 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_suppress_logs": "true", 2024-09-21T14:58:17.180 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_vault_addr": "", 2024-09-21T14:58:17.180 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_vault_auth": "token", 2024-09-21T14:58:17.180 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_vault_namespace": "", 2024-09-21T14:58:17.180 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_vault_prefix": "", 2024-09-21T14:58:17.180 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-09-21T14:58:17.180 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-09-21T14:58:17.180 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-09-21T14:58:17.180 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-09-21T14:58:17.180 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_vault_token_file": "", 2024-09-21T14:58:17.180 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-09-21T14:58:17.180 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_curl_buffersize": "524288", 2024-09-21T14:58:17.180 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_curl_low_speed_limit": "1024", 2024-09-21T14:58:17.180 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_curl_low_speed_time": "300", 2024-09-21T14:58:17.180 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_curl_tcp_keepalive": "0", 2024-09-21T14:58:17.181 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-09-21T14:58:17.181 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-09-21T14:58:17.181 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-09-21T14:58:17.181 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-09-21T14:58:17.181 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-09-21T14:58:17.181 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_d3n_l1_fadvise": "4", 2024-09-21T14:58:17.181 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-09-21T14:58:17.181 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-09-21T14:58:17.181 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-09-21T14:58:17.181 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-09-21T14:58:17.181 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_data_log_changes_size": "1000", 2024-09-21T14:58:17.181 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_data_log_num_shards": "128", 2024-09-21T14:58:17.181 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-09-21T14:58:17.181 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_data_log_window": "30", 2024-09-21T14:58:17.181 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_data_notify_interval_msec": "0", 2024-09-21T14:58:17.181 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_data_sync_poll_interval": "20", 2024-09-21T14:58:17.182 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_data_sync_spawn_window": "20", 2024-09-21T14:58:17.182 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-09-21T14:58:17.182 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-09-21T14:58:17.182 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_default_data_log_backing": "fifo", 2024-09-21T14:58:17.182 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-09-21T14:58:17.182 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_default_region_info_oid": "default.region", 2024-09-21T14:58:17.182 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-09-21T14:58:17.182 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-09-21T14:58:17.182 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_defer_to_bucket_acls": "", 2024-09-21T14:58:17.182 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-09-21T14:58:17.182 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-09-21T14:58:17.182 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-09-21T14:58:17.182 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-09-21T14:58:17.182 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-09-21T14:58:17.183 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-09-21T14:58:17.183 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-09-21T14:58:17.183 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-09-21T14:58:17.183 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_dmclock_data_res": "500.000000", 2024-09-21T14:58:17.183 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-09-21T14:58:17.183 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-09-21T14:58:17.183 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-09-21T14:58:17.183 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-09-21T14:58:17.183 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_dns_name": "", 2024-09-21T14:58:17.183 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_dns_s3website_name": "", 2024-09-21T14:58:17.183 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_dynamic_resharding": "true", 2024-09-21T14:58:17.183 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-09-21T14:58:17.183 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_enable_gc_threads": "true", 2024-09-21T14:58:17.183 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_enable_lc_threads": "true", 2024-09-21T14:58:17.184 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_enable_ops_log": "false", 2024-09-21T14:58:17.184 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_enable_quota_threads": "true", 2024-09-21T14:58:17.184 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_enable_static_website": "false", 2024-09-21T14:58:17.184 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_enable_usage_log": "false", 2024-09-21T14:58:17.184 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_enforce_swift_acls": "true", 2024-09-21T14:58:17.184 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_exit_timeout_secs": "120", 2024-09-21T14:58:17.184 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_expose_bucket": "false", 2024-09-21T14:58:17.184 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_extended_http_attrs": "", 2024-09-21T14:58:17.184 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_filter": "none", 2024-09-21T14:58:17.184 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-09-21T14:58:17.184 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_frontends": "beast port=7480", 2024-09-21T14:58:17.184 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_gc_max_concurrent_io": "10", 2024-09-21T14:58:17.184 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_gc_max_deferred": "50", 2024-09-21T14:58:17.184 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-09-21T14:58:17.185 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_gc_max_objs": "32", 2024-09-21T14:58:17.185 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_gc_max_queue_size": "134213632", 2024-09-21T14:58:17.185 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_gc_max_trim_chunk": "16", 2024-09-21T14:58:17.185 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_gc_obj_min_wait": "7200", 2024-09-21T14:58:17.185 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_gc_processor_max_time": "3600", 2024-09-21T14:58:17.185 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_gc_processor_period": "3600", 2024-09-21T14:58:17.185 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-09-21T14:58:17.185 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_get_obj_window_size": "16777216", 2024-09-21T14:58:17.185 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_healthcheck_disabling_path": "", 2024-09-21T14:58:17.185 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_ignore_get_invalid_range": "false", 2024-09-21T14:58:17.185 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_init_timeout": "300", 2024-09-21T14:58:17.185 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-09-21T14:58:17.185 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2024-09-21T14:58:17.185 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-09-21T14:58:17.185 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-09-21T14:58:17.186 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_keystone_admin_domain": "", 2024-09-21T14:58:17.186 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_keystone_admin_password": "", 2024-09-21T14:58:17.186 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_keystone_admin_password_path": "", 2024-09-21T14:58:17.186 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_keystone_admin_project": "", 2024-09-21T14:58:17.186 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_keystone_admin_tenant": "", 2024-09-21T14:58:17.186 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_keystone_admin_token": "", 2024-09-21T14:58:17.186 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_keystone_admin_token_path": "", 2024-09-21T14:58:17.186 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_keystone_admin_user": "", 2024-09-21T14:58:17.186 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_keystone_api_version": "2", 2024-09-21T14:58:17.186 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_keystone_barbican_domain": "", 2024-09-21T14:58:17.186 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_keystone_barbican_password": "", 2024-09-21T14:58:17.186 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_keystone_barbican_project": "", 2024-09-21T14:58:17.186 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_keystone_barbican_tenant": "", 2024-09-21T14:58:17.187 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_keystone_barbican_user": "", 2024-09-21T14:58:17.187 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-09-21T14:58:17.187 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_keystone_implicit_tenants": "false", 2024-09-21T14:58:17.187 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-09-21T14:58:17.187 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_keystone_service_token_enabled": "false", 2024-09-21T14:58:17.187 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_keystone_token_cache_size": "10000", 2024-09-21T14:58:17.187 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_keystone_url": "", 2024-09-21T14:58:17.187 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_keystone_verify_ssl": "true", 2024-09-21T14:58:17.187 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_lc_debug_interval": "-1", 2024-09-21T14:58:17.187 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_lc_lock_max_time": "90", 2024-09-21T14:58:17.187 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_lc_max_objs": "32", 2024-09-21T14:58:17.187 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_lc_max_rules": "1000", 2024-09-21T14:58:17.188 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_lc_max_worker": "3", 2024-09-21T14:58:17.188 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_lc_max_wp_worker": "3", 2024-09-21T14:58:17.188 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_lc_thread_delay": "0", 2024-09-21T14:58:17.188 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-09-21T14:58:17.188 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_ldap_dnattr": "uid", 2024-09-21T14:58:17.188 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-09-21T14:58:17.188 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_ldap_searchfilter": "", 2024-09-21T14:58:17.188 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-09-21T14:58:17.188 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_ldap_uri": "ldaps://", 2024-09-21T14:58:17.188 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-09-21T14:58:17.188 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-09-21T14:58:17.188 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-09-21T14:58:17.188 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_log_http_headers": "", 2024-09-21T14:58:17.188 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_log_nonexistent_bucket": "false", 2024-09-21T14:58:17.188 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-09-21T14:58:17.189 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_log_object_name_utc": "false", 2024-09-21T14:58:17.189 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_luarocks_location": "/tmp/luarocks", 2024-09-21T14:58:17.189 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_max_attr_name_len": "0", 2024-09-21T14:58:17.189 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_max_attr_size": "0", 2024-09-21T14:58:17.189 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_max_attrs_num_in_req": "0", 2024-09-21T14:58:17.189 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_max_chunk_size": "4194304", 2024-09-21T14:58:17.189 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_max_concurrent_requests": "1024", 2024-09-21T14:58:17.189 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2024-09-21T14:58:17.189 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_max_dynamic_shards": "1999", 2024-09-21T14:58:17.189 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_max_listing_results": "1000", 2024-09-21T14:58:17.189 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_max_notify_retries": "10", 2024-09-21T14:58:17.189 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_max_objs_per_shard": "100000", 2024-09-21T14:58:17.189 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_max_put_param_size": "1048576", 2024-09-21T14:58:17.189 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_max_put_size": "5368709120", 2024-09-21T14:58:17.189 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_max_slo_entries": "1000", 2024-09-21T14:58:17.190 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_md_log_max_shards": "64", 2024-09-21T14:58:17.190 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_md_notify_interval_msec": "200", 2024-09-21T14:58:17.190 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_meta_sync_poll_interval": "20", 2024-09-21T14:58:17.190 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_meta_sync_spawn_window": "20", 2024-09-21T14:58:17.190 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-09-21T14:58:17.190 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_mp_lock_max_time": "600", 2024-09-21T14:58:17.190 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-09-21T14:58:17.190 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_multipart_min_part_size": "5242880", 2024-09-21T14:58:17.190 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-09-21T14:58:17.190 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-09-21T14:58:17.191 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_nfs_fhcache_size": "2017", 2024-09-21T14:58:17.191 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_nfs_frontends": "rgw-nfs", 2024-09-21T14:58:17.191 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-09-21T14:58:17.191 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_nfs_lru_lanes": "5", 2024-09-21T14:58:17.191 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_nfs_max_gc": "300", 2024-09-21T14:58:17.191 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-09-21T14:58:17.191 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_nfs_run_gc_threads": "false", 2024-09-21T14:58:17.191 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_nfs_run_lc_threads": "false", 2024-09-21T14:58:17.191 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_nfs_run_quota_threads": "false", 2024-09-21T14:58:17.191 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_nfs_run_sync_thread": "false", 2024-09-21T14:58:17.191 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-09-21T14:58:17.191 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-09-21T14:58:17.191 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_num_async_rados_threads": "32", 2024-09-21T14:58:17.191 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_num_control_oids": "8", 2024-09-21T14:58:17.192 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_numa_node": "-1", 2024-09-21T14:58:17.192 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_obj_stripe_size": "4194304", 2024-09-21T14:58:17.192 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-09-21T14:58:17.192 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_objexp_chunk_size": "100", 2024-09-21T14:58:17.192 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_objexp_gc_interval": "600", 2024-09-21T14:58:17.192 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_objexp_hints_num_shards": "127", 2024-09-21T14:58:17.192 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-09-21T14:58:17.192 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-09-21T14:58:17.192 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_op_thread_timeout": "600", 2024-09-21T14:58:17.192 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_op_tracing": "false", 2024-09-21T14:58:17.192 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_opa_token": "", 2024-09-21T14:58:17.192 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_opa_url": "", 2024-09-21T14:58:17.192 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_opa_verify_ssl": "true", 2024-09-21T14:58:17.192 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-09-21T14:58:17.193 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2024-09-21T14:58:17.193 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_ops_log_rados": "false", 2024-09-21T14:58:17.193 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_ops_log_socket_path": "", 2024-09-21T14:58:17.193 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-09-21T14:58:17.193 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2024-09-21T14:58:17.193 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-09-21T14:58:17.193 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_period_push_interval": "2.000000", 2024-09-21T14:58:17.193 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_period_push_interval_max": "30.000000", 2024-09-21T14:58:17.193 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_period_root_pool": ".rgw.root", 2024-09-21T14:58:17.193 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_policy_reject_invalid_principals": "true", 2024-09-21T14:58:17.193 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_print_continue": "true", 2024-09-21T14:58:17.193 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_print_prohibited_content_length": "false", 2024-09-21T14:58:17.193 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-09-21T14:58:17.193 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-09-21T14:58:17.193 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-09-21T14:58:17.194 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-09-21T14:58:17.194 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_rados_tracing": "false", 2024-09-21T14:58:17.194 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_realm": "", 2024-09-21T14:58:17.194 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_realm_id": "", 2024-09-21T14:58:17.194 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-09-21T14:58:17.194 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_region": "", 2024-09-21T14:58:17.194 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_region_root_pool": ".rgw.root", 2024-09-21T14:58:17.194 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_relaxed_region_enforcement": "false", 2024-09-21T14:58:17.194 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-09-21T14:58:17.194 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-09-21T14:58:17.194 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_request_uri": "", 2024-09-21T14:58:17.195 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_reshard_batch_size": "64", 2024-09-21T14:58:17.195 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-09-21T14:58:17.195 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_reshard_max_aio": "128", 2024-09-21T14:58:17.195 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_reshard_num_logs": "16", 2024-09-21T14:58:17.195 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_reshard_thread_interval": "600", 2024-09-21T14:58:17.195 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_resolve_cname": "false", 2024-09-21T14:58:17.195 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_rest_getusage_op_compat": "false", 2024-09-21T14:58:17.195 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_run_sync_thread": "true", 2024-09-21T14:58:17.195 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-09-21T14:58:17.195 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_s3_auth_use_keystone": "false", 2024-09-21T14:58:17.195 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_s3_auth_use_ldap": "false", 2024-09-21T14:58:17.195 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_s3_auth_use_rados": "true", 2024-09-21T14:58:17.195 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_s3_auth_use_sts": "false", 2024-09-21T14:58:17.196 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-09-21T14:58:17.196 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_s3_success_create_obj_status": "0", 2024-09-21T14:58:17.196 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-09-21T14:58:17.196 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_scheduler_type": "throttler", 2024-09-21T14:58:17.196 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_script_uri": "", 2024-09-21T14:58:17.196 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_service_provider_name": "", 2024-09-21T14:58:17.196 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-09-21T14:58:17.196 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_sts_client_id": "", 2024-09-21T14:58:17.196 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_sts_client_secret": "", 2024-09-21T14:58:17.196 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_sts_entry": "sts", 2024-09-21T14:58:17.196 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_sts_key": "sts", 2024-09-21T14:58:17.196 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_sts_max_session_duration": "43200", 2024-09-21T14:58:17.196 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_sts_min_session_duration": "900", 2024-09-21T14:58:17.197 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_sts_token_introspection_url": "", 2024-09-21T14:58:17.197 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_swift_account_in_url": "false", 2024-09-21T14:58:17.197 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_swift_auth_entry": "auth", 2024-09-21T14:58:17.197 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_swift_auth_url": "", 2024-09-21T14:58:17.197 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_swift_custom_header": "", 2024-09-21T14:58:17.197 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_swift_enforce_content_length": "false", 2024-09-21T14:58:17.197 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_swift_need_stats": "true", 2024-09-21T14:58:17.197 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_swift_tenant_name": "", 2024-09-21T14:58:17.197 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_swift_token_expiration": "86400", 2024-09-21T14:58:17.197 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_swift_url": "", 2024-09-21T14:58:17.197 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_swift_url_prefix": "swift", 2024-09-21T14:58:17.197 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_swift_versioning_enabled": "false", 2024-09-21T14:58:17.197 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2024-09-21T14:58:17.198 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-09-21T14:58:17.198 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_sync_lease_period": "120", 2024-09-21T14:58:17.198 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-09-21T14:58:17.198 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_sync_log_trim_interval": "1200", 2024-09-21T14:58:17.198 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-09-21T14:58:17.198 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-09-21T14:58:17.198 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-09-21T14:58:17.198 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_sync_obj_etag_verify": "false", 2024-09-21T14:58:17.198 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_sync_trace_history_size": "4096", 2024-09-21T14:58:17.198 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-09-21T14:58:17.198 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-09-21T14:58:17.198 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_thread_pool_size": "512", 2024-09-21T14:58:17.198 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_torrent_comment": "", 2024-09-21T14:58:17.199 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_torrent_createby": "", 2024-09-21T14:58:17.199 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_torrent_encoding": "", 2024-09-21T14:58:17.199 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_torrent_flag": "false", 2024-09-21T14:58:17.199 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_torrent_origin": "", 2024-09-21T14:58:17.199 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_torrent_sha_unit": "524288", 2024-09-21T14:58:17.199 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_torrent_tracker": "", 2024-09-21T14:58:17.199 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_trust_forwarded_https": "false", 2024-09-21T14:58:17.199 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-09-21T14:58:17.199 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_usage_log_tick_interval": "30", 2024-09-21T14:58:17.199 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_usage_max_shards": "32", 2024-09-21T14:58:17.199 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_usage_max_user_shards": "1", 2024-09-21T14:58:17.199 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_use_opa_authz": "false", 2024-09-21T14:58:17.199 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-09-21T14:58:17.199 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_user_default_quota_max_size": "-1", 2024-09-21T14:58:17.200 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_user_max_buckets": "1000", 2024-09-21T14:58:17.200 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_user_policies_max_num": "100", 2024-09-21T14:58:17.200 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-09-21T14:58:17.200 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-09-21T14:58:17.200 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_user_quota_sync_interval": "86400", 2024-09-21T14:58:17.200 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-09-21T14:58:17.200 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_user_unique_email": "true", 2024-09-21T14:58:17.200 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_verify_ssl": "true", 2024-09-21T14:58:17.200 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_website_routing_rules_max_num": "50", 2024-09-21T14:58:17.200 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_zone": "", 2024-09-21T14:58:17.200 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_zone_id": "", 2024-09-21T14:58:17.201 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-09-21T14:58:17.201 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_zonegroup": "", 2024-09-21T14:58:17.201 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_zonegroup_id": "", 2024-09-21T14:58:17.201 INFO:teuthology.orchestra.run.smithi032.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-09-21T14:58:17.201 INFO:teuthology.orchestra.run.smithi032.stdout: "rgwlc_auto_session_clear": "true", 2024-09-21T14:58:17.201 INFO:teuthology.orchestra.run.smithi032.stdout: "rgwlc_skip_bucket_step": "false", 2024-09-21T14:58:17.201 INFO:teuthology.orchestra.run.smithi032.stdout: "rocksdb_block_size": "4096", 2024-09-21T14:58:17.201 INFO:teuthology.orchestra.run.smithi032.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-09-21T14:58:17.201 INFO:teuthology.orchestra.run.smithi032.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-09-21T14:58:17.201 INFO:teuthology.orchestra.run.smithi032.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-09-21T14:58:17.201 INFO:teuthology.orchestra.run.smithi032.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-09-21T14:58:17.201 INFO:teuthology.orchestra.run.smithi032.stdout: "rocksdb_cache_shard_bits": "4", 2024-09-21T14:58:17.201 INFO:teuthology.orchestra.run.smithi032.stdout: "rocksdb_cache_size": "536870912", 2024-09-21T14:58:17.201 INFO:teuthology.orchestra.run.smithi032.stdout: "rocksdb_cache_type": "binned_lru", 2024-09-21T14:58:17.202 INFO:teuthology.orchestra.run.smithi032.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-09-21T14:58:17.202 INFO:teuthology.orchestra.run.smithi032.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-09-21T14:58:17.202 INFO:teuthology.orchestra.run.smithi032.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-09-21T14:58:17.202 INFO:teuthology.orchestra.run.smithi032.stdout: "rocksdb_collect_compaction_stats": "false", 2024-09-21T14:58:17.202 INFO:teuthology.orchestra.run.smithi032.stdout: "rocksdb_collect_extended_stats": "false", 2024-09-21T14:58:17.202 INFO:teuthology.orchestra.run.smithi032.stdout: "rocksdb_collect_memory_stats": "false", 2024-09-21T14:58:17.202 INFO:teuthology.orchestra.run.smithi032.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-09-21T14:58:17.202 INFO:teuthology.orchestra.run.smithi032.stdout: "rocksdb_index_type": "binary_search", 2024-09-21T14:58:17.202 INFO:teuthology.orchestra.run.smithi032.stdout: "rocksdb_log_to_ceph_log": "true", 2024-09-21T14:58:17.202 INFO:teuthology.orchestra.run.smithi032.stdout: "rocksdb_metadata_block_size": "4096", 2024-09-21T14:58:17.202 INFO:teuthology.orchestra.run.smithi032.stdout: "rocksdb_partition_filters": "false", 2024-09-21T14:58:17.202 INFO:teuthology.orchestra.run.smithi032.stdout: "rocksdb_perf": "false", 2024-09-21T14:58:17.202 INFO:teuthology.orchestra.run.smithi032.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-09-21T14:58:17.203 INFO:teuthology.orchestra.run.smithi032.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-09-21T14:58:17.203 INFO:teuthology.orchestra.run.smithi032.stdout: "rotating_keys_renewal_timeout": "10", 2024-09-21T14:58:17.203 INFO:teuthology.orchestra.run.smithi032.stdout: "run_dir": "/var/run/ceph", 2024-09-21T14:58:17.203 INFO:teuthology.orchestra.run.smithi032.stdout: "seastore_block_create": "true", 2024-09-21T14:58:17.203 INFO:teuthology.orchestra.run.smithi032.stdout: "seastore_cache_lru_size": "67108864", 2024-09-21T14:58:17.203 INFO:teuthology.orchestra.run.smithi032.stdout: "seastore_cbjournal_size": "5368709120", 2024-09-21T14:58:17.203 INFO:teuthology.orchestra.run.smithi032.stdout: "seastore_default_max_object_size": "16777216", 2024-09-21T14:58:17.203 INFO:teuthology.orchestra.run.smithi032.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-09-21T14:58:17.203 INFO:teuthology.orchestra.run.smithi032.stdout: "seastore_device_size": "53687091200", 2024-09-21T14:58:17.203 INFO:teuthology.orchestra.run.smithi032.stdout: "seastore_journal_batch_capacity": "16", 2024-09-21T14:58:17.203 INFO:teuthology.orchestra.run.smithi032.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-09-21T14:58:17.203 INFO:teuthology.orchestra.run.smithi032.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-09-21T14:58:17.203 INFO:teuthology.orchestra.run.smithi032.stdout: "seastore_journal_iodepth_limit": "5", 2024-09-21T14:58:17.204 INFO:teuthology.orchestra.run.smithi032.stdout: "seastore_main_device_type": "SSD", 2024-09-21T14:58:17.204 INFO:teuthology.orchestra.run.smithi032.stdout: "seastore_max_concurrent_transactions": "8", 2024-09-21T14:58:17.204 INFO:teuthology.orchestra.run.smithi032.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2024-09-21T14:58:17.204 INFO:teuthology.orchestra.run.smithi032.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2024-09-21T14:58:17.204 INFO:teuthology.orchestra.run.smithi032.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2024-09-21T14:58:17.204 INFO:teuthology.orchestra.run.smithi032.stdout: "seastore_obj_data_write_amplification": "1.250000", 2024-09-21T14:58:17.204 INFO:teuthology.orchestra.run.smithi032.stdout: "seastore_segment_size": "67108864", 2024-09-21T14:58:17.204 INFO:teuthology.orchestra.run.smithi032.stdout: "set_keepcaps": "false", 2024-09-21T14:58:17.204 INFO:teuthology.orchestra.run.smithi032.stdout: "setgroup": "", 2024-09-21T14:58:17.204 INFO:teuthology.orchestra.run.smithi032.stdout: "setuser": "", 2024-09-21T14:58:17.204 INFO:teuthology.orchestra.run.smithi032.stdout: "setuser_match_path": "", 2024-09-21T14:58:17.204 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_misplaced_ratio": "0.050000", 2024-09-21T14:58:17.204 INFO:teuthology.orchestra.run.smithi032.stdout: "thp": "false", 2024-09-21T14:58:17.204 INFO:teuthology.orchestra.run.smithi032.stdout: "threadpool_default_timeout": "60", 2024-09-21T14:58:17.205 INFO:teuthology.orchestra.run.smithi032.stdout: "threadpool_empty_queue_max_wait": "2", 2024-09-21T14:58:17.205 INFO:teuthology.orchestra.run.smithi032.stdout: "throttler_perf_counter": "true" 2024-09-21T14:58:17.205 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:58:17.206 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-09-21T14:58:17.348 INFO:teuthology.orchestra.run.smithi032.stdout:mon.a: {} 2024-09-21T14:58:17.364 INFO:teuthology.orchestra.run.smithi032.stdout:mon.b: {} 2024-09-21T14:58:17.375 INFO:teuthology.orchestra.run.smithi032.stdout:mon.c: {} 2024-09-21T14:58:17.377 INFO:teuthology.orchestra.run.smithi032.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-21T14:58:17.378 INFO:teuthology.orchestra.run.smithi032.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-21T14:58:17.378 INFO:teuthology.orchestra.run.smithi032.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-21T14:58:17.389 DEBUG:teuthology.orchestra.run.smithi032:> type ceph-objectstore-tool 2024-09-21T14:58:17.393 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-21T14:58:17.393 DEBUG:teuthology.orchestra.run.smithi046:> type ceph-objectstore-tool 2024-09-21T14:58:17.399 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-21T14:58:17.400 INFO:teuthology.run_tasks:Running task rbd_fsx... 2024-09-21T14:58:17.412 INFO:tasks.rbd_fsx:starting rbd_fsx... 2024-09-21T14:58:17.412 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-09-21T14:58:17.412 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T14:58:17.413 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-21T14:58:17.413 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-09-21T14:58:17.413 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-09-21T14:58:17.413 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T14:58:17.413 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-09-21T14:58:17.414 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T14:58:17.414 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-09-21T14:58:17.414 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T14:58:17.415 DEBUG:teuthology.orchestra.run.smithi032:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx --cluster ceph --id 0 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 1 -w 1 -t 1 -h 1 -l 250000000 -S 0 -N 6000 -g pool_client.0 image_client.0 2024-09-21T14:58:17.416 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:58:17.408+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:58:17.450 INFO:teuthology.orchestra.run.smithi032.stdout:mapped writes DISABLED 2024-09-21T14:58:17.450 INFO:teuthology.orchestra.run.smithi032.stdout:mapped reads DISABLED 2024-09-21T14:58:17.450 INFO:teuthology.orchestra.run.smithi032.stdout:Seed set to 1957 2024-09-21T14:58:17.515 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:58:17.513+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:58:17.556 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T14:58:17.561 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:17.561 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:17.573 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T14:58:17.575 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T14:58:17.604 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T14:58:17.616 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:58:17.613+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:58:17.629 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T14:58:17.650 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-21T14:58:17.675 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T14:58:17.677 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:17.677 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:17.687 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T14:58:17.699 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T14:58:17.717 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:58:17.717+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:58:17.723 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T14:58:17.725 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:58:17.725 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:58:17.725 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:58:17.725 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:58:17.725 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:58:17.725 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:58:17.725 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:58:17.725 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:58:17.811 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:17.811 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:17.817 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:58:17.812+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:58:17.821 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T14:58:17.919 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:58:17.917+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:58:17.927 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:17.927 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:17.937 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T14:58:18.019 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:58:18.017+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:58:18.047 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:18.047 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:18.056 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T14:58:18.120 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:58:18.117+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:58:18.163 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:18.163 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:18.172 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T14:58:18.221 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:58:18.221+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:58:18.282 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:18.282 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:18.292 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T14:58:18.322 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:58:18.316+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:58:18.403 INFO:teuthology.orchestra.run.smithi032.stderr:reweighted osd.0 to 1 (10000) 2024-09-21T14:58:18.407 INFO:teuthology.orchestra.run.smithi032.stderr:noscrub is set 2024-09-21T14:58:18.410 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:18.410 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:18.415 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-21T14:58:18.421 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T14:58:18.423 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:58:18.421+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:58:18.523 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:58:18.520+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:58:18.583 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:18.583 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:18.594 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T14:58:18.639 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:58:18.636+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:58:18.694 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:18.695 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:18.705 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T14:58:18.725 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:58:18.720+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:58:18.815 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:18.815 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:18.825 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T14:58:18.826 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:58:18.825+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:58:18.927 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:58:18.924+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:58:18.929 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:18.929 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:18.939 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T14:58:19.028 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:58:19.024+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:58:19.052 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:19.052 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:19.062 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T14:58:19.129 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:58:19.124+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:58:19.170 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:19.170 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:19.179 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T14:58:19.230 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:58:19.229+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:58:19.285 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:19.285 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:19.295 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T14:58:19.330 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:58:19.329+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:58:19.411 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:19.411 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:19.419 INFO:teuthology.orchestra.run.smithi032.stderr:reweighted osd.1 to 1 (10000) 2024-09-21T14:58:19.423 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T14:58:19.427 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-21T14:58:19.465 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:58:19.465+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:58:19.532 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:58:19.528+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:58:19.574 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:19.574 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:19.584 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T14:58:19.634 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:58:19.628+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:58:19.689 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:19.689 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:19.699 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T14:58:19.734 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:58:19.728+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:58:19.740 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T14:58:19.811 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:19.811 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:19.821 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T14:58:19.835 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:58:19.833+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:58:19.885 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T14:58:19.903 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T14:58:19.921 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T14:58:19.922 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:19.922 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:19.932 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T14:58:19.936 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:58:19.932+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:58:19.941 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T14:58:19.964 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-21T14:58:19.985 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T14:58:20.007 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T14:58:20.028 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T14:58:20.031 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:58:20.031 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:58:20.031 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:58:20.031 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:58:20.031 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:58:20.031 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:58:20.031 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:58:20.031 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:58:20.036 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:58:20.037+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:58:20.041 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:58:20.041 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T14:58:20.041 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T14:58:20.041 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T14:58:20.041 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:20.042 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4281.0:5 2.2 e21/19 2:5cea7035:::rbd_info:head v 21'1, mlcod=0'0)", 2024-09-21T14:58:20.042 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:10.343460+0000", 2024-09-21T14:58:20.042 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.6987766690000008, 2024-09-21T14:58:20.042 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0024752820000000001, 2024-09-21T14:58:20.042 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:20.042 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:20.042 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:20.042 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:20.042 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:20.042 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:10.343460+0000", 2024-09-21T14:58:20.042 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:20.042 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:20.042 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:20.042 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:20.043 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:10.343460+0000", 2024-09-21T14:58:20.043 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:20.043 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:20.043 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:20.043 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:20.043 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:10.343459+0000", 2024-09-21T14:58:20.043 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:20.043 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:20.043 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:20.043 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:20.043 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:10.343499+0000", 2024-09-21T14:58:20.043 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9983e-05 2024-09-21T14:58:20.043 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:20.043 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:20.044 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:20.044 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:10.343521+0000", 2024-09-21T14:58:20.044 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1858999999999999e-05 2024-09-21T14:58:20.044 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:20.044 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:20.044 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:20.044 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:10.343540+0000", 2024-09-21T14:58:20.044 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9426e-05 2024-09-21T14:58:20.044 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:20.044 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:20.044 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:20.044 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:10.343647+0000", 2024-09-21T14:58:20.044 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000106612 2024-09-21T14:58:20.044 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:20.044 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:20.045 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:20.045 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:10.343723+0000", 2024-09-21T14:58:20.045 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6230999999999996e-05 2024-09-21T14:58:20.045 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:20.045 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:20.045 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:20.045 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:10.345869+0000", 2024-09-21T14:58:20.045 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0021460450000000001 2024-09-21T14:58:20.045 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:20.045 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:20.045 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:20.045 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:10.345935+0000", 2024-09-21T14:58:20.045 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5776000000000002e-05 2024-09-21T14:58:20.045 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:20.046 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:20.046 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:20.046 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:20.046 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:20.046 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4281.0:7 2.2 e22/19 2:5cea7035:::rbd_info:head v 22'2, mlcod=0'0)", 2024-09-21T14:58:20.046 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:11.349430+0000", 2024-09-21T14:58:20.046 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 8.6928060340000002, 2024-09-21T14:58:20.046 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0015191499999999999, 2024-09-21T14:58:20.046 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:20.046 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:20.046 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:20.046 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:20.046 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:20.047 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:11.349430+0000", 2024-09-21T14:58:20.047 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:20.047 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:20.047 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:20.047 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:20.047 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:11.349430+0000", 2024-09-21T14:58:20.047 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:20.047 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:20.047 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:20.047 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:20.047 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:11.349429+0000", 2024-09-21T14:58:20.047 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:20.047 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:20.047 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:20.048 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:20.048 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:11.349450+0000", 2024-09-21T14:58:20.048 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0873000000000002e-05 2024-09-21T14:58:20.048 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:20.048 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:20.048 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:20.048 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:11.349471+0000", 2024-09-21T14:58:20.048 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1016e-05 2024-09-21T14:58:20.048 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:20.048 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:20.048 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:20.048 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:11.349490+0000", 2024-09-21T14:58:20.048 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8736000000000002e-05 2024-09-21T14:58:20.048 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:20.048 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:20.049 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:20.049 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:11.349585+0000", 2024-09-21T14:58:20.049 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5377999999999994e-05 2024-09-21T14:58:20.049 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:20.049 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:20.049 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:20.049 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:11.349634+0000", 2024-09-21T14:58:20.049 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9449e-05 2024-09-21T14:58:20.049 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:20.049 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:20.049 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:20.049 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:11.350896+0000", 2024-09-21T14:58:20.049 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001261119 2024-09-21T14:58:20.049 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:20.050 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:20.050 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:20.050 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:11.350949+0000", 2024-09-21T14:58:20.050 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3749000000000003e-05 2024-09-21T14:58:20.050 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:20.050 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:20.050 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:20.050 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:20.050 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:20.050 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:58:20.052 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T14:58:20.138 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:58:20.137+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:58:20.160 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:58:20.161 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:58:20.161 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T14:58:20.161 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:58:20.170 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T14:58:20.238 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:58:20.237+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:58:20.328 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:58:20.329 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:58:20.329 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T14:58:20.329 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T14:58:20.329 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:58:20.339 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T14:58:20.339 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:58:20.337+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:58:20.412 INFO:teuthology.orchestra.run.smithi032.stderr:reweighted osd.2 to 1 (10000) 2024-09-21T14:58:20.422 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T14:58:20.425 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-21T14:58:20.439 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:58:20.436+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:58:20.448 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:58:20.448 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T14:58:20.449 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T14:58:20.449 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [] 2024-09-21T14:58:20.449 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:58:20.458 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T14:58:20.540 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:58:20.536+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:58:20.602 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:58:20.602 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:58:20.602 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T14:58:20.602 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:58:20.612 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T14:58:20.641 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:58:20.636+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:58:20.739 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:58:20.739 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:58:20.739 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T14:58:20.739 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T14:58:20.739 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:58:20.742 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:58:20.741+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:58:20.749 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T14:58:20.842 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:58:20.841+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:58:20.848 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:58:20.849 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T14:58:20.849 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T14:58:20.849 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [] 2024-09-21T14:58:20.849 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:58:20.859 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T14:58:20.943 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:58:20.941+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:58:20.964 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:58:20.964 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:58:20.964 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T14:58:20.964 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:58:20.974 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T14:58:21.044 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:58:21.040+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:58:21.080 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:58:21.080 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:58:21.081 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T14:58:21.081 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T14:58:21.081 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:58:21.090 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T14:58:21.144 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:58:21.140+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:58:21.208 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:58:21.208 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T14:58:21.209 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T14:58:21.209 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T14:58:21.209 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.209 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T14:58:21.209 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.205754+0000", 2024-09-21T14:58:21.209 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 17.997306336000001, 2024-09-21T14:58:21.209 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.41117094199999998, 2024-09-21T14:58:21.209 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:21.209 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:21.209 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:21.209 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:21.209 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:21.209 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1 2024-09-21T14:58:21.210 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.210 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:21.210 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.210 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:21.210 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205754+0000", 2024-09-21T14:58:21.210 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:21.210 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.210 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.210 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:21.210 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205754+0000", 2024-09-21T14:58:21.210 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:21.210 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.210 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.211 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:21.211 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205752+0000", 2024-09-21T14:58:21.211 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T14:58:21.211 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.211 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.211 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:21.211 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205765+0000", 2024-09-21T14:58:21.211 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3230999999999999e-05 2024-09-21T14:58:21.211 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.211 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.211 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:21.211 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205782+0000", 2024-09-21T14:58:21.211 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6634e-05 2024-09-21T14:58:21.212 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.212 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.212 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:21.212 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205793+0000", 2024-09-21T14:58:21.212 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1297e-05 2024-09-21T14:58:21.212 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.212 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.212 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:21.212 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205867+0000", 2024-09-21T14:58:21.212 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4269000000000003e-05 2024-09-21T14:58:21.212 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.212 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.212 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T14:58:21.212 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205877+0000", 2024-09-21T14:58:21.213 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2690000000000005e-06 2024-09-21T14:58:21.213 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.213 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.213 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:21.213 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.613700+0000", 2024-09-21T14:58:21.213 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.407823455 2024-09-21T14:58:21.213 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.213 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.213 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:21.213 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.614203+0000", 2024-09-21T14:58:21.213 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00050311500000000005 2024-09-21T14:58:21.213 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.213 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.213 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:21.213 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.614750+0000", 2024-09-21T14:58:21.214 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00054738699999999998 2024-09-21T14:58:21.214 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.214 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.214 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:21.214 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616367+0000", 2024-09-21T14:58:21.214 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001616634 2024-09-21T14:58:21.214 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.214 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.214 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:21.214 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616574+0000", 2024-09-21T14:58:21.214 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00020668499999999999 2024-09-21T14:58:21.214 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.214 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.215 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:21.215 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616802+0000", 2024-09-21T14:58:21.215 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00022817900000000001 2024-09-21T14:58:21.215 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.215 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.215 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:21.215 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616925+0000", 2024-09-21T14:58:21.215 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012309699999999999 2024-09-21T14:58:21.215 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:21.215 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:21.215 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:21.215 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.215 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.215 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T14:58:21.216 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.666480+0000", 2024-09-21T14:58:21.216 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 17.536580879999999, 2024-09-21T14:58:21.216 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0098753969999999993, 2024-09-21T14:58:21.216 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:21.216 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:21.216 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:21.216 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:21.216 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:21.216 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 35 2024-09-21T14:58:21.216 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.216 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:21.216 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.216 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:21.216 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.666480+0000", 2024-09-21T14:58:21.217 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:21.217 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.217 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.217 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:21.217 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.666480+0000", 2024-09-21T14:58:21.217 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:21.217 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.217 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.217 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:21.217 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.666478+0000", 2024-09-21T14:58:21.217 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:21.217 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.217 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.218 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:21.218 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.666626+0000", 2024-09-21T14:58:21.218 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014799100000000001 2024-09-21T14:58:21.218 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.218 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.218 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:21.218 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.666675+0000", 2024-09-21T14:58:21.218 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9086999999999999e-05 2024-09-21T14:58:21.218 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.218 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.218 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:21.218 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.666696+0000", 2024-09-21T14:58:21.218 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0448000000000001e-05 2024-09-21T14:58:21.218 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.218 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.219 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:21.219 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.672759+0000", 2024-09-21T14:58:21.219 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0060628410000000002 2024-09-21T14:58:21.219 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.219 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.219 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:21.219 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.672891+0000", 2024-09-21T14:58:21.219 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013294700000000001 2024-09-21T14:58:21.219 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.219 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.219 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:21.219 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.673110+0000", 2024-09-21T14:58:21.219 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00021842500000000001 2024-09-21T14:58:21.219 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.219 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.220 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:21.220 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.676136+0000", 2024-09-21T14:58:21.220 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0030259739999999999 2024-09-21T14:58:21.220 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.220 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.220 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:21.220 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.676192+0000", 2024-09-21T14:58:21.220 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5795999999999999e-05 2024-09-21T14:58:21.220 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.220 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.220 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:21.220 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.676255+0000", 2024-09-21T14:58:21.220 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3548999999999998e-05 2024-09-21T14:58:21.220 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.220 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.221 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:21.221 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.676355+0000", 2024-09-21T14:58:21.221 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9786999999999999e-05 2024-09-21T14:58:21.221 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:21.221 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:21.221 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:21.221 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.221 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.221 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T14:58:21.221 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.666714+0000", 2024-09-21T14:58:21.221 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 17.536346258999998, 2024-09-21T14:58:21.221 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.010286645000000001, 2024-09-21T14:58:21.221 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:21.222 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:21.222 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:21.222 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:21.222 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:21.222 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 36 2024-09-21T14:58:21.222 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.222 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:21.222 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.222 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:21.222 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.666714+0000", 2024-09-21T14:58:21.222 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:21.222 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.222 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.223 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:21.223 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.666714+0000", 2024-09-21T14:58:21.223 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:21.223 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.223 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.223 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:21.223 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.666713+0000", 2024-09-21T14:58:21.223 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:21.223 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.223 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.223 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:21.223 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.666724+0000", 2024-09-21T14:58:21.223 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0939e-05 2024-09-21T14:58:21.223 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.224 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.224 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:21.224 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.666752+0000", 2024-09-21T14:58:21.224 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8725e-05 2024-09-21T14:58:21.224 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.224 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.224 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:21.224 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.666768+0000", 2024-09-21T14:58:21.224 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5274e-05 2024-09-21T14:58:21.224 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.224 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.224 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:21.224 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.674731+0000", 2024-09-21T14:58:21.224 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0079634179999999999 2024-09-21T14:58:21.224 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.225 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.225 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:21.225 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.674840+0000", 2024-09-21T14:58:21.225 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000108639 2024-09-21T14:58:21.225 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.225 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.225 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:21.225 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.674959+0000", 2024-09-21T14:58:21.225 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00011884599999999999 2024-09-21T14:58:21.225 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.225 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.225 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:21.225 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.676146+0000", 2024-09-21T14:58:21.226 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0011873680000000001 2024-09-21T14:58:21.226 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.226 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.226 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:21.226 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.676924+0000", 2024-09-21T14:58:21.226 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00077794099999999998 2024-09-21T14:58:21.226 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.226 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.226 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:21.226 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.676966+0000", 2024-09-21T14:58:21.226 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2172999999999998e-05 2024-09-21T14:58:21.226 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.226 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.226 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:21.227 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.677001+0000", 2024-09-21T14:58:21.227 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4755999999999997e-05 2024-09-21T14:58:21.227 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:21.227 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:21.227 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:21.227 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.227 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.227 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T14:58:21.227 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.666789+0000", 2024-09-21T14:58:21.227 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 17.536271189000001, 2024-09-21T14:58:21.227 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.010759082999999999, 2024-09-21T14:58:21.227 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:21.227 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:21.227 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:21.227 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:21.228 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:21.228 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 37 2024-09-21T14:58:21.228 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.228 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:21.228 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.228 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:21.228 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.666789+0000", 2024-09-21T14:58:21.228 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:21.228 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.228 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.228 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:21.228 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.666789+0000", 2024-09-21T14:58:21.228 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:21.228 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.228 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.229 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:21.229 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.666788+0000", 2024-09-21T14:58:21.229 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:58:21.229 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.229 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.229 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:21.229 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.666799+0000", 2024-09-21T14:58:21.229 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0532e-05 2024-09-21T14:58:21.229 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.229 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.229 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:21.229 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.666822+0000", 2024-09-21T14:58:21.229 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.323e-05 2024-09-21T14:58:21.229 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.230 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.230 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:21.230 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.666875+0000", 2024-09-21T14:58:21.230 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3071000000000001e-05 2024-09-21T14:58:21.230 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.230 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.230 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:21.230 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.675669+0000", 2024-09-21T14:58:21.230 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0087942860000000001 2024-09-21T14:58:21.230 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.230 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.230 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:21.230 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.675749+0000", 2024-09-21T14:58:21.230 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9260999999999999e-05 2024-09-21T14:58:21.231 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.231 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.231 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:21.231 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.675872+0000", 2024-09-21T14:58:21.231 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000123072 2024-09-21T14:58:21.231 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.231 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.231 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:21.231 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.676749+0000", 2024-09-21T14:58:21.231 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00087719600000000001 2024-09-21T14:58:21.231 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.231 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.231 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:21.231 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.677408+0000", 2024-09-21T14:58:21.231 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00065863300000000005 2024-09-21T14:58:21.232 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.232 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.232 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:21.232 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.677492+0000", 2024-09-21T14:58:21.232 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4934999999999999e-05 2024-09-21T14:58:21.232 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.232 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.232 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:21.232 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.677548+0000", 2024-09-21T14:58:21.232 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5881999999999998e-05 2024-09-21T14:58:21.232 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:21.232 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:21.232 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:21.232 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.232 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.232 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T14:58:21.233 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.748110+0000", 2024-09-21T14:58:21.233 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 17.454950063999998, 2024-09-21T14:58:21.233 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.010071220000000001, 2024-09-21T14:58:21.233 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:21.233 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:21.233 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:21.233 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:21.233 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:21.233 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 88 2024-09-21T14:58:21.233 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.233 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:21.233 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.233 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:21.233 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.748110+0000", 2024-09-21T14:58:21.233 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:21.234 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.234 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.234 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:21.234 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.748110+0000", 2024-09-21T14:58:21.234 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:21.234 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.234 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.234 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:21.234 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.748110+0000", 2024-09-21T14:58:21.234 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:58:21.234 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.234 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.234 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:21.234 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.748116+0000", 2024-09-21T14:58:21.235 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5230000000000001e-06 2024-09-21T14:58:21.235 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.235 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.235 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:21.235 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.748130+0000", 2024-09-21T14:58:21.235 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4248e-05 2024-09-21T14:58:21.235 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.235 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.235 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:21.235 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.748159+0000", 2024-09-21T14:58:21.235 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8931000000000001e-05 2024-09-21T14:58:21.235 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.235 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.235 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:21.236 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.755598+0000", 2024-09-21T14:58:21.236 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0074386820000000003 2024-09-21T14:58:21.236 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.236 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.236 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:21.236 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.755778+0000", 2024-09-21T14:58:21.236 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00018027399999999999 2024-09-21T14:58:21.236 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.236 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.236 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:21.236 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.756059+0000", 2024-09-21T14:58:21.236 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00028083600000000002 2024-09-21T14:58:21.236 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.236 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.236 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:21.237 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.757447+0000", 2024-09-21T14:58:21.237 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0013877939999999999 2024-09-21T14:58:21.237 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.237 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.237 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:21.237 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.758054+0000", 2024-09-21T14:58:21.237 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000606979 2024-09-21T14:58:21.237 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.237 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.237 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:21.237 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.758128+0000", 2024-09-21T14:58:21.237 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4493999999999995e-05 2024-09-21T14:58:21.237 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.238 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.238 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:21.238 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.758182+0000", 2024-09-21T14:58:21.238 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3247e-05 2024-09-21T14:58:21.238 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:21.238 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:21.238 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:21.238 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.238 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.238 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T14:58:21.238 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.954952+0000", 2024-09-21T14:58:21.238 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 17.248108499000001, 2024-09-21T14:58:21.238 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0097988899999999993, 2024-09-21T14:58:21.239 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:21.239 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:21.239 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:21.239 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:21.239 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:21.239 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 111 2024-09-21T14:58:21.239 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.239 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:21.239 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.239 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:21.239 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.954952+0000", 2024-09-21T14:58:21.239 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:21.239 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.239 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.240 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:21.240 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.954952+0000", 2024-09-21T14:58:21.240 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:21.240 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.240 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.240 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:21.240 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.954952+0000", 2024-09-21T14:58:21.240 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:21.240 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.240 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.240 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:21.240 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.954955+0000", 2024-09-21T14:58:21.240 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2440000000000002e-06 2024-09-21T14:58:21.241 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.241 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.241 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:21.241 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.954960+0000", 2024-09-21T14:58:21.241 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8119999999999998e-06 2024-09-21T14:58:21.241 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.241 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.241 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:21.241 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.954963+0000", 2024-09-21T14:58:21.241 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4029999999999999e-06 2024-09-21T14:58:21.241 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.241 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.241 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:21.242 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.960407+0000", 2024-09-21T14:58:21.242 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0054443360000000001 2024-09-21T14:58:21.242 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.242 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.242 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:21.242 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.960496+0000", 2024-09-21T14:58:21.242 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8978000000000001e-05 2024-09-21T14:58:21.242 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.242 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.242 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:21.242 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.960635+0000", 2024-09-21T14:58:21.242 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013853300000000001 2024-09-21T14:58:21.242 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.243 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.243 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:21.243 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.964189+0000", 2024-09-21T14:58:21.243 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0035545590000000001 2024-09-21T14:58:21.243 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.243 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.243 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:21.243 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.964571+0000", 2024-09-21T14:58:21.243 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00038135799999999999 2024-09-21T14:58:21.243 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.243 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.243 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:21.243 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.964685+0000", 2024-09-21T14:58:21.243 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00011391300000000001 2024-09-21T14:58:21.244 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.244 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.244 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:21.244 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.964751+0000", 2024-09-21T14:58:21.244 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6111000000000005e-05 2024-09-21T14:58:21.244 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:21.244 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:21.244 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:21.244 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.244 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.244 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T14:58:21.244 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.955076+0000", 2024-09-21T14:58:21.244 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 17.247984581000001, 2024-09-21T14:58:21.244 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.010426615, 2024-09-21T14:58:21.245 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:21.245 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:21.245 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:21.245 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:21.245 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:21.245 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 112 2024-09-21T14:58:21.245 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.245 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:21.245 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.245 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:21.245 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955076+0000", 2024-09-21T14:58:21.245 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:21.245 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.245 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.246 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:21.246 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955076+0000", 2024-09-21T14:58:21.246 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:21.246 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.246 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.246 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:21.246 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955075+0000", 2024-09-21T14:58:21.246 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:58:21.246 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.246 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.246 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:21.246 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955122+0000", 2024-09-21T14:58:21.246 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6802999999999999e-05 2024-09-21T14:58:21.246 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.247 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.247 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:21.247 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955130+0000", 2024-09-21T14:58:21.247 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0490000000000004e-06 2024-09-21T14:58:21.247 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.247 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.247 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:21.247 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955136+0000", 2024-09-21T14:58:21.247 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9059999999999999e-06 2024-09-21T14:58:21.247 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.247 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.247 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:21.247 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.961454+0000", 2024-09-21T14:58:21.247 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0063180429999999997 2024-09-21T14:58:21.248 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.248 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.248 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:21.248 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.961600+0000", 2024-09-21T14:58:21.248 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014651700000000001 2024-09-21T14:58:21.248 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.248 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.248 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:21.248 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.961798+0000", 2024-09-21T14:58:21.248 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00019784100000000001 2024-09-21T14:58:21.248 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.248 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.248 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:21.248 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.964785+0000", 2024-09-21T14:58:21.248 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.002986561 2024-09-21T14:58:21.248 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.249 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.249 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:21.249 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.965422+0000", 2024-09-21T14:58:21.249 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00063701800000000005 2024-09-21T14:58:21.249 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.249 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.249 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:21.249 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.965466+0000", 2024-09-21T14:58:21.249 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4020999999999997e-05 2024-09-21T14:58:21.249 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.249 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.249 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:21.249 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.965503+0000", 2024-09-21T14:58:21.249 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6814999999999999e-05 2024-09-21T14:58:21.249 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:21.250 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:21.250 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:21.250 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.250 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.250 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T14:58:21.250 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.955142+0000", 2024-09-21T14:58:21.250 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 17.247918688999999, 2024-09-21T14:58:21.250 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012388923, 2024-09-21T14:58:21.250 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:21.250 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:21.250 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:21.250 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:21.250 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:21.250 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 113 2024-09-21T14:58:21.251 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.251 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:21.251 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.251 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:21.251 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955142+0000", 2024-09-21T14:58:21.251 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:21.251 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.251 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.251 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:21.251 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955142+0000", 2024-09-21T14:58:21.251 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:21.251 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.251 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.251 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:21.252 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955141+0000", 2024-09-21T14:58:21.252 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:21.252 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.252 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.252 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:21.252 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955145+0000", 2024-09-21T14:58:21.252 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2109999999999998e-06 2024-09-21T14:58:21.252 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.252 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.252 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:21.252 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955150+0000", 2024-09-21T14:58:21.252 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4090000000000002e-06 2024-09-21T14:58:21.252 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.252 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.252 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:21.253 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955153+0000", 2024-09-21T14:58:21.253 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1240000000000001e-06 2024-09-21T14:58:21.253 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.253 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.253 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:21.253 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.963026+0000", 2024-09-21T14:58:21.253 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0078724060000000002 2024-09-21T14:58:21.253 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.253 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.253 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:21.253 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.963126+0000", 2024-09-21T14:58:21.253 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000100206 2024-09-21T14:58:21.253 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.253 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.253 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:21.254 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.963277+0000", 2024-09-21T14:58:21.254 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015159200000000001 2024-09-21T14:58:21.254 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.254 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.254 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:21.254 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.965519+0000", 2024-09-21T14:58:21.254 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0022418440000000002 2024-09-21T14:58:21.254 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.254 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.254 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:21.254 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967305+0000", 2024-09-21T14:58:21.254 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001785525 2024-09-21T14:58:21.254 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.254 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.254 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:21.254 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967438+0000", 2024-09-21T14:58:21.255 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013284399999999999 2024-09-21T14:58:21.255 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.255 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.255 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:21.255 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967531+0000", 2024-09-21T14:58:21.255 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.3140000000000006e-05 2024-09-21T14:58:21.255 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:21.255 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:21.255 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:21.255 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.255 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.255 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T14:58:21.255 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.955387+0000", 2024-09-21T14:58:21.256 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 17.247673755000001, 2024-09-21T14:58:21.256 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012319478999999999, 2024-09-21T14:58:21.256 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:21.256 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:21.256 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:21.256 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:21.256 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:21.256 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 114 2024-09-21T14:58:21.256 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.256 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:21.256 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.256 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:21.256 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955387+0000", 2024-09-21T14:58:21.256 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:21.256 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.257 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.257 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:21.257 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955387+0000", 2024-09-21T14:58:21.257 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:21.257 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.257 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.257 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:21.257 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955386+0000", 2024-09-21T14:58:21.257 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:58:21.257 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.257 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.257 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:21.257 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955392+0000", 2024-09-21T14:58:21.257 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4740000000000002e-06 2024-09-21T14:58:21.257 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.258 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.258 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:21.258 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955400+0000", 2024-09-21T14:58:21.258 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2139999999999996e-06 2024-09-21T14:58:21.258 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.258 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.258 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:21.258 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955407+0000", 2024-09-21T14:58:21.258 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2299999999999996e-06 2024-09-21T14:58:21.258 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.258 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.258 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:21.258 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.964840+0000", 2024-09-21T14:58:21.258 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0094330079999999997 2024-09-21T14:58:21.259 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.259 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.259 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:21.259 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.964917+0000", 2024-09-21T14:58:21.259 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7071999999999997e-05 2024-09-21T14:58:21.259 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.259 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.259 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:21.259 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.965026+0000", 2024-09-21T14:58:21.259 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000109646 2024-09-21T14:58:21.259 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.259 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.259 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:21.259 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967609+0000", 2024-09-21T14:58:21.259 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0025824929999999999 2024-09-21T14:58:21.260 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.260 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.260 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:21.260 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967629+0000", 2024-09-21T14:58:21.260 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0486999999999999e-05 2024-09-21T14:58:21.260 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.260 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.260 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:21.260 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967672+0000", 2024-09-21T14:58:21.260 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2314999999999997e-05 2024-09-21T14:58:21.260 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.260 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.260 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:21.260 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967706+0000", 2024-09-21T14:58:21.260 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4483999999999997e-05 2024-09-21T14:58:21.261 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:21.261 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:21.261 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:21.261 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.261 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.261 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T14:58:21.261 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.955609+0000", 2024-09-21T14:58:21.261 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 17.247451948999998, 2024-09-21T14:58:21.261 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013624153999999999, 2024-09-21T14:58:21.261 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:21.261 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:21.261 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:21.261 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:21.261 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:21.261 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 115 2024-09-21T14:58:21.262 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.262 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:21.262 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.262 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:21.262 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955609+0000", 2024-09-21T14:58:21.262 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:21.262 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.262 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.262 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:21.262 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955609+0000", 2024-09-21T14:58:21.262 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:21.262 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.262 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.262 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:21.262 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955607+0000", 2024-09-21T14:58:21.263 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-21T14:58:21.263 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.263 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.263 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:21.263 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955732+0000", 2024-09-21T14:58:21.263 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000125566 2024-09-21T14:58:21.263 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.263 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.263 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:21.263 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955752+0000", 2024-09-21T14:58:21.263 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9916999999999999e-05 2024-09-21T14:58:21.263 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.263 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.263 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:21.263 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955767+0000", 2024-09-21T14:58:21.264 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4687e-05 2024-09-21T14:58:21.264 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.264 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.264 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:21.264 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.965574+0000", 2024-09-21T14:58:21.264 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0098072850000000007 2024-09-21T14:58:21.264 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.264 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.264 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:21.264 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.965642+0000", 2024-09-21T14:58:21.264 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7784e-05 2024-09-21T14:58:21.264 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.264 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.264 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:21.264 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.965753+0000", 2024-09-21T14:58:21.265 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000111111 2024-09-21T14:58:21.265 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.265 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.265 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:21.265 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967717+0000", 2024-09-21T14:58:21.265 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0019635339999999999 2024-09-21T14:58:21.265 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.265 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.265 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:21.265 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.969014+0000", 2024-09-21T14:58:21.265 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001297093 2024-09-21T14:58:21.265 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.265 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.265 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:21.265 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.969130+0000", 2024-09-21T14:58:21.266 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000116011 2024-09-21T14:58:21.266 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.266 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.266 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:21.266 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.969233+0000", 2024-09-21T14:58:21.266 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000102982 2024-09-21T14:58:21.266 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:21.266 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:21.266 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:21.266 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.266 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.266 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T14:58:21.266 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.955783+0000", 2024-09-21T14:58:21.266 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 17.247277279999999, 2024-09-21T14:58:21.267 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014630241, 2024-09-21T14:58:21.267 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:21.267 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:21.267 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:21.267 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:21.267 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:21.267 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 116 2024-09-21T14:58:21.267 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.267 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:21.267 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.267 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:21.267 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955783+0000", 2024-09-21T14:58:21.267 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:21.267 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.268 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.268 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:21.268 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955783+0000", 2024-09-21T14:58:21.268 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:21.268 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.268 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.268 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:21.268 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955782+0000", 2024-09-21T14:58:21.268 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:58:21.268 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.268 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.268 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:21.268 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955791+0000", 2024-09-21T14:58:21.268 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8089999999999999e-06 2024-09-21T14:58:21.269 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.269 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.269 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:21.269 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955804+0000", 2024-09-21T14:58:21.269 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2548000000000001e-05 2024-09-21T14:58:21.269 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.269 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.269 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:21.269 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955814+0000", 2024-09-21T14:58:21.269 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0158e-05 2024-09-21T14:58:21.269 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.269 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.269 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:21.269 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.966515+0000", 2024-09-21T14:58:21.270 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.010701756 2024-09-21T14:58:21.270 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.270 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.270 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:21.270 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.966606+0000", 2024-09-21T14:58:21.270 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0761999999999996e-05 2024-09-21T14:58:21.270 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.270 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.270 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:21.270 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.966740+0000", 2024-09-21T14:58:21.270 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000133693 2024-09-21T14:58:21.270 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.270 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.270 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:21.270 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.968920+0000", 2024-09-21T14:58:21.271 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0021799290000000002 2024-09-21T14:58:21.271 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.271 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.271 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:21.271 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.970320+0000", 2024-09-21T14:58:21.271 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0014001759999999999 2024-09-21T14:58:21.271 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.271 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.271 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:21.271 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.970376+0000", 2024-09-21T14:58:21.271 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5696999999999999e-05 2024-09-21T14:58:21.271 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.271 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.271 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:21.272 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.970413+0000", 2024-09-21T14:58:21.272 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7750999999999998e-05 2024-09-21T14:58:21.272 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:21.272 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:21.272 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:21.272 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.272 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.272 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T14:58:21.272 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.955829+0000", 2024-09-21T14:58:21.272 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 17.247231809999999, 2024-09-21T14:58:21.272 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014919597999999999, 2024-09-21T14:58:21.272 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:21.272 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:21.272 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:21.272 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:21.272 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:21.273 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 117 2024-09-21T14:58:21.273 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.273 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:21.273 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.273 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:21.273 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955829+0000", 2024-09-21T14:58:21.273 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:21.273 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.273 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.273 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:21.273 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955829+0000", 2024-09-21T14:58:21.273 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:21.273 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.273 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.273 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:21.274 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955827+0000", 2024-09-21T14:58:21.274 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:21.274 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.274 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.274 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:21.274 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955838+0000", 2024-09-21T14:58:21.274 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.111e-05 2024-09-21T14:58:21.274 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.274 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.274 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:21.274 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955875+0000", 2024-09-21T14:58:21.274 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6211999999999998e-05 2024-09-21T14:58:21.274 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.274 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.275 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:21.275 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955899+0000", 2024-09-21T14:58:21.275 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4128999999999999e-05 2024-09-21T14:58:21.275 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.275 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.275 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:21.275 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967767+0000", 2024-09-21T14:58:21.275 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.011868251999999999 2024-09-21T14:58:21.275 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.275 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.275 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:21.275 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967866+0000", 2024-09-21T14:58:21.275 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8664000000000003e-05 2024-09-21T14:58:21.275 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.276 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.276 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:21.276 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.968128+0000", 2024-09-21T14:58:21.276 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00026243600000000001 2024-09-21T14:58:21.276 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.276 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.276 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:21.276 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.969393+0000", 2024-09-21T14:58:21.276 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0012650350000000001 2024-09-21T14:58:21.276 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.276 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.276 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:21.276 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.970523+0000", 2024-09-21T14:58:21.276 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001129557 2024-09-21T14:58:21.276 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.277 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.277 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:21.277 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.970603+0000", 2024-09-21T14:58:21.277 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0090000000000001e-05 2024-09-21T14:58:21.277 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.277 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.277 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:21.277 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.970748+0000", 2024-09-21T14:58:21.277 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014552799999999999 2024-09-21T14:58:21.277 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:21.277 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:21.277 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:21.277 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.277 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.278 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T14:58:21.278 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:04.492934+0000", 2024-09-21T14:58:21.278 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 16.710126562999999, 2024-09-21T14:58:21.278 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.010845886000000001, 2024-09-21T14:58:21.278 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:21.278 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:21.278 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:21.278 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:21.278 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:21.278 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 134 2024-09-21T14:58:21.278 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.278 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:21.278 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.278 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:21.278 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.492934+0000", 2024-09-21T14:58:21.279 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:21.279 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.279 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.279 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:21.279 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.492934+0000", 2024-09-21T14:58:21.279 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:21.279 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.279 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.279 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:21.279 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.492933+0000", 2024-09-21T14:58:21.279 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:58:21.279 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.279 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.279 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:21.279 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493094+0000", 2024-09-21T14:58:21.280 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00016101599999999999 2024-09-21T14:58:21.280 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.280 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.280 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:21.280 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493110+0000", 2024-09-21T14:58:21.280 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5447999999999998e-05 2024-09-21T14:58:21.280 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.280 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.280 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:21.280 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493119+0000", 2024-09-21T14:58:21.280 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.3170000000000008e-06 2024-09-21T14:58:21.280 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.280 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.280 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:21.280 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.498294+0000", 2024-09-21T14:58:21.281 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0051747700000000004 2024-09-21T14:58:21.281 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.281 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.281 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:21.281 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.498456+0000", 2024-09-21T14:58:21.281 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00016253599999999999 2024-09-21T14:58:21.281 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.281 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.281 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:21.281 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.498626+0000", 2024-09-21T14:58:21.281 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00016929900000000001 2024-09-21T14:58:21.281 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.281 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.281 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:21.281 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.502513+0000", 2024-09-21T14:58:21.281 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0038875530000000002 2024-09-21T14:58:21.282 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.282 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.282 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:21.282 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.503655+0000", 2024-09-21T14:58:21.282 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001142405 2024-09-21T14:58:21.282 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.282 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.282 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:21.282 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.503741+0000", 2024-09-21T14:58:21.282 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5918e-05 2024-09-21T14:58:21.282 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.282 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.282 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:21.282 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.503780+0000", 2024-09-21T14:58:21.283 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.8401e-05 2024-09-21T14:58:21.283 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:21.283 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:21.283 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:21.283 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.283 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.283 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T14:58:21.283 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:04.493130+0000", 2024-09-21T14:58:21.283 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 16.709930405000001, 2024-09-21T14:58:21.283 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012013845, 2024-09-21T14:58:21.283 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:21.283 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:21.283 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:21.283 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:21.283 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:21.284 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 135 2024-09-21T14:58:21.284 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.284 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:21.284 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.284 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:21.284 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493130+0000", 2024-09-21T14:58:21.284 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:21.284 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.284 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.284 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:21.284 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493130+0000", 2024-09-21T14:58:21.284 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:21.284 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.284 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.284 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:21.284 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493129+0000", 2024-09-21T14:58:21.285 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:58:21.285 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.285 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.285 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:21.285 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493135+0000", 2024-09-21T14:58:21.285 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1290000000000001e-06 2024-09-21T14:58:21.285 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.285 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.285 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:21.285 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493143+0000", 2024-09-21T14:58:21.285 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0139999999999998e-06 2024-09-21T14:58:21.285 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.285 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.285 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:21.285 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493150+0000", 2024-09-21T14:58:21.286 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.674e-06 2024-09-21T14:58:21.286 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.286 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.286 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:21.286 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.500089+0000", 2024-09-21T14:58:21.286 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0069391599999999998 2024-09-21T14:58:21.286 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.286 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.286 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:21.286 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.500277+0000", 2024-09-21T14:58:21.286 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00018736999999999999 2024-09-21T14:58:21.286 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.286 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.286 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:21.287 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.500426+0000", 2024-09-21T14:58:21.287 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014981900000000001 2024-09-21T14:58:21.287 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.287 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.287 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:21.287 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.503817+0000", 2024-09-21T14:58:21.287 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0033905150000000002 2024-09-21T14:58:21.287 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.287 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.287 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:21.287 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.504994+0000", 2024-09-21T14:58:21.287 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0011767500000000001 2024-09-21T14:58:21.287 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.287 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.288 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:21.288 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.505083+0000", 2024-09-21T14:58:21.288 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.9339999999999995e-05 2024-09-21T14:58:21.288 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.288 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.288 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:21.288 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.505144+0000", 2024-09-21T14:58:21.288 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0934000000000003e-05 2024-09-21T14:58:21.288 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:21.288 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:21.288 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:21.288 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.288 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.288 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T14:58:21.288 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:04.493201+0000", 2024-09-21T14:58:21.289 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 16.709859061, 2024-09-21T14:58:21.289 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.01208676, 2024-09-21T14:58:21.289 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:21.289 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:21.289 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:21.289 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:21.289 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:21.289 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 136 2024-09-21T14:58:21.289 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.289 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:21.289 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.289 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:21.289 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493201+0000", 2024-09-21T14:58:21.289 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:21.290 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.290 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.290 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:21.290 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493201+0000", 2024-09-21T14:58:21.290 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:21.290 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.290 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.290 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:21.290 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493200+0000", 2024-09-21T14:58:21.290 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:58:21.290 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.290 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.290 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:21.290 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493214+0000", 2024-09-21T14:58:21.291 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3488e-05 2024-09-21T14:58:21.291 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.291 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.291 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:21.291 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493244+0000", 2024-09-21T14:58:21.291 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9848e-05 2024-09-21T14:58:21.291 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.291 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.291 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:21.291 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493263+0000", 2024-09-21T14:58:21.291 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9071000000000001e-05 2024-09-21T14:58:21.291 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.291 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.291 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:21.292 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.501197+0000", 2024-09-21T14:58:21.292 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.007934623 2024-09-21T14:58:21.292 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.292 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.292 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:21.292 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.501327+0000", 2024-09-21T14:58:21.292 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000129605 2024-09-21T14:58:21.292 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.292 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.292 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:21.292 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.501518+0000", 2024-09-21T14:58:21.292 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000190662 2024-09-21T14:58:21.292 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.292 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.292 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:21.293 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.505175+0000", 2024-09-21T14:58:21.293 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0036576989999999999 2024-09-21T14:58:21.293 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.293 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.293 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:21.293 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.505216+0000", 2024-09-21T14:58:21.293 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1097000000000003e-05 2024-09-21T14:58:21.293 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.293 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.293 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:21.293 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.505251+0000", 2024-09-21T14:58:21.293 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4758000000000002e-05 2024-09-21T14:58:21.293 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.293 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.293 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:21.294 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.505288+0000", 2024-09-21T14:58:21.294 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6913999999999999e-05 2024-09-21T14:58:21.294 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:21.294 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:21.294 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:21.294 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.294 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.294 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T14:58:21.294 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:04.493315+0000", 2024-09-21T14:58:21.294 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 16.709744977, 2024-09-21T14:58:21.294 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014055766000000001, 2024-09-21T14:58:21.294 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:21.294 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:21.294 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:21.295 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:21.295 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:21.295 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 137 2024-09-21T14:58:21.295 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.295 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:21.295 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.295 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:21.295 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493315+0000", 2024-09-21T14:58:21.295 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:21.295 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.295 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.295 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:21.295 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493315+0000", 2024-09-21T14:58:21.295 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:21.295 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.296 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.296 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:21.296 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493314+0000", 2024-09-21T14:58:21.296 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:58:21.296 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.296 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.296 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:21.296 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493507+0000", 2024-09-21T14:58:21.296 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000192444 2024-09-21T14:58:21.296 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.296 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.296 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:21.296 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493528+0000", 2024-09-21T14:58:21.296 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0845000000000001e-05 2024-09-21T14:58:21.297 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.297 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.297 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:21.297 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493543+0000", 2024-09-21T14:58:21.297 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5800000000000001e-05 2024-09-21T14:58:21.297 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.297 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.297 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:21.297 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.502575+0000", 2024-09-21T14:58:21.297 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0090312389999999999 2024-09-21T14:58:21.297 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.297 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.297 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:21.297 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.502677+0000", 2024-09-21T14:58:21.297 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000102271 2024-09-21T14:58:21.298 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.298 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.298 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:21.298 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.502825+0000", 2024-09-21T14:58:21.298 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000148171 2024-09-21T14:58:21.298 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.298 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.298 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:21.298 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507251+0000", 2024-09-21T14:58:21.298 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0044253879999999997 2024-09-21T14:58:21.298 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.298 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.298 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:21.298 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507313+0000", 2024-09-21T14:58:21.299 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2170999999999998e-05 2024-09-21T14:58:21.299 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.299 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.299 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:21.299 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507346+0000", 2024-09-21T14:58:21.299 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3079e-05 2024-09-21T14:58:21.299 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.299 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.299 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:21.299 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507371+0000", 2024-09-21T14:58:21.299 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5477999999999999e-05 2024-09-21T14:58:21.299 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:21.299 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:21.299 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:21.299 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.300 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.300 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T14:58:21.300 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:04.493570+0000", 2024-09-21T14:58:21.300 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 16.709490572, 2024-09-21T14:58:21.300 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014061686, 2024-09-21T14:58:21.300 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:21.300 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:21.300 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:21.300 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:21.300 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:21.300 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 138 2024-09-21T14:58:21.300 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.300 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:21.300 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.300 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:21.301 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493570+0000", 2024-09-21T14:58:21.301 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:21.301 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.301 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.301 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:21.301 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493570+0000", 2024-09-21T14:58:21.301 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:21.301 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.301 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.301 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:21.301 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493568+0000", 2024-09-21T14:58:21.301 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:21.301 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.301 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.302 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:21.302 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493579+0000", 2024-09-21T14:58:21.302 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0574000000000001e-05 2024-09-21T14:58:21.302 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.302 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.302 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:21.302 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493595+0000", 2024-09-21T14:58:21.302 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6504999999999999e-05 2024-09-21T14:58:21.302 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.302 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.302 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:21.302 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493608+0000", 2024-09-21T14:58:21.302 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2836999999999999e-05 2024-09-21T14:58:21.302 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.303 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.303 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:21.303 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.503909+0000", 2024-09-21T14:58:21.303 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.01030118 2024-09-21T14:58:21.303 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.303 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.303 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:21.303 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.504064+0000", 2024-09-21T14:58:21.303 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015493000000000001 2024-09-21T14:58:21.303 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.303 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.303 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:21.303 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.504299+0000", 2024-09-21T14:58:21.303 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00023416100000000001 2024-09-21T14:58:21.304 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.304 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.304 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:21.304 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507466+0000", 2024-09-21T14:58:21.304 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0031678380000000001 2024-09-21T14:58:21.304 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.305 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.305 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:21.306 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507483+0000", 2024-09-21T14:58:21.306 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6606999999999999e-05 2024-09-21T14:58:21.306 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.306 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.306 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:21.306 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507566+0000", 2024-09-21T14:58:21.306 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3110999999999998e-05 2024-09-21T14:58:21.306 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.306 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.306 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:21.306 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507632+0000", 2024-09-21T14:58:21.306 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5480000000000003e-05 2024-09-21T14:58:21.306 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:21.307 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:21.307 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:21.307 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.307 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.307 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T14:58:21.307 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:04.493625+0000", 2024-09-21T14:58:21.307 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 16.709435884000001, 2024-09-21T14:58:21.307 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014665378999999999, 2024-09-21T14:58:21.307 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:21.307 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:21.307 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:21.307 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:21.307 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:21.308 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139 2024-09-21T14:58:21.308 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.308 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:21.308 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.308 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:21.308 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493625+0000", 2024-09-21T14:58:21.308 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:21.308 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.308 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.308 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:21.308 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493625+0000", 2024-09-21T14:58:21.308 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:21.308 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.308 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.309 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:21.309 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493624+0000", 2024-09-21T14:58:21.309 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:58:21.309 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.309 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.309 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:21.309 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493638+0000", 2024-09-21T14:58:21.309 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4282e-05 2024-09-21T14:58:21.309 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.309 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.309 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:21.309 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493654+0000", 2024-09-21T14:58:21.309 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5739999999999998e-05 2024-09-21T14:58:21.310 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.310 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.310 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:21.310 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493666+0000", 2024-09-21T14:58:21.310 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2865e-05 2024-09-21T14:58:21.310 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.310 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.310 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:21.310 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.505329+0000", 2024-09-21T14:58:21.310 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.011662237000000001 2024-09-21T14:58:21.310 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.310 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.310 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:21.310 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.505377+0000", 2024-09-21T14:58:21.311 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7862999999999999e-05 2024-09-21T14:58:21.311 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.311 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.311 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:21.311 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.505455+0000", 2024-09-21T14:58:21.311 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8918000000000006e-05 2024-09-21T14:58:21.311 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.311 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.311 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:21.311 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507650+0000", 2024-09-21T14:58:21.311 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.002195027 2024-09-21T14:58:21.311 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.311 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.311 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:21.312 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508149+0000", 2024-09-21T14:58:21.312 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00049828399999999999 2024-09-21T14:58:21.312 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.312 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.312 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:21.312 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508230+0000", 2024-09-21T14:58:21.312 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1376000000000002e-05 2024-09-21T14:58:21.312 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.312 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.312 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:21.312 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508290+0000", 2024-09-21T14:58:21.312 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9852999999999998e-05 2024-09-21T14:58:21.312 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:21.312 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:21.313 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:21.313 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.313 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.313 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T14:58:21.313 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:04.493679+0000", 2024-09-21T14:58:21.313 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 16.709380985999999, 2024-09-21T14:58:21.313 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014869869000000001, 2024-09-21T14:58:21.313 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:21.313 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:21.313 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:21.313 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:21.313 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:21.313 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140 2024-09-21T14:58:21.313 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.314 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:21.314 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.314 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:21.314 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493679+0000", 2024-09-21T14:58:21.314 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:21.314 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.314 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.314 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:21.314 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493679+0000", 2024-09-21T14:58:21.314 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:21.314 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.314 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.314 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:21.314 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493679+0000", 2024-09-21T14:58:21.315 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:58:21.315 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.315 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.315 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:21.315 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493889+0000", 2024-09-21T14:58:21.315 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00021032700000000001 2024-09-21T14:58:21.315 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.315 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.315 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:21.315 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493931+0000", 2024-09-21T14:58:21.315 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1971000000000002e-05 2024-09-21T14:58:21.315 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.315 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.315 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:21.315 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493947+0000", 2024-09-21T14:58:21.316 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6385e-05 2024-09-21T14:58:21.316 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.316 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.316 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:21.316 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.505804+0000", 2024-09-21T14:58:21.316 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.011857136000000001 2024-09-21T14:58:21.316 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.316 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.316 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:21.316 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.505992+0000", 2024-09-21T14:58:21.316 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000187665 2024-09-21T14:58:21.316 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.316 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.316 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:21.316 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.506281+0000", 2024-09-21T14:58:21.317 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000288712 2024-09-21T14:58:21.317 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.317 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.317 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:21.317 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508349+0000", 2024-09-21T14:58:21.317 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0020685790000000001 2024-09-21T14:58:21.317 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.317 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.317 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:21.317 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508371+0000", 2024-09-21T14:58:21.317 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1747e-05 2024-09-21T14:58:21.317 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.317 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.317 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:21.318 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508450+0000", 2024-09-21T14:58:21.318 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9332000000000005e-05 2024-09-21T14:58:21.318 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.318 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.318 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:21.318 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508549+0000", 2024-09-21T14:58:21.318 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8918000000000004e-05 2024-09-21T14:58:21.318 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:21.318 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:21.318 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:21.318 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.318 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.318 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T14:58:21.318 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:04.493967+0000", 2024-09-21T14:58:21.319 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 16.709093949, 2024-09-21T14:58:21.319 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.015281075999999999, 2024-09-21T14:58:21.319 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:21.319 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:21.319 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:21.319 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:21.319 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:21.319 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 141 2024-09-21T14:58:21.319 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.319 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:21.319 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.319 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:21.319 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493967+0000", 2024-09-21T14:58:21.319 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:21.319 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.320 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.320 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:21.320 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493967+0000", 2024-09-21T14:58:21.320 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:21.320 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.320 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.320 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:21.320 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493965+0000", 2024-09-21T14:58:21.320 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:21.320 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.320 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.320 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:21.320 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493987+0000", 2024-09-21T14:58:21.320 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1662999999999999e-05 2024-09-21T14:58:21.320 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.320 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.321 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:21.321 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.494004+0000", 2024-09-21T14:58:21.321 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7311000000000001e-05 2024-09-21T14:58:21.321 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.321 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.321 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:21.321 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.494017+0000", 2024-09-21T14:58:21.321 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3202e-05 2024-09-21T14:58:21.321 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.321 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.321 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:21.321 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507678+0000", 2024-09-21T14:58:21.321 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013660432 2024-09-21T14:58:21.321 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.321 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.322 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:21.322 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507726+0000", 2024-09-21T14:58:21.322 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8154e-05 2024-09-21T14:58:21.322 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.322 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.322 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:21.322 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507797+0000", 2024-09-21T14:58:21.322 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.1110000000000002e-05 2024-09-21T14:58:21.322 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.322 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.322 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:21.322 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508710+0000", 2024-09-21T14:58:21.322 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00091339299999999997 2024-09-21T14:58:21.322 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.322 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.323 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:21.323 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.509108+0000", 2024-09-21T14:58:21.323 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00039718300000000001 2024-09-21T14:58:21.323 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.323 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.323 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:21.323 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.509186+0000", 2024-09-21T14:58:21.323 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8592999999999998e-05 2024-09-21T14:58:21.323 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.323 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.323 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:21.323 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.509248+0000", 2024-09-21T14:58:21.323 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1450000000000003e-05 2024-09-21T14:58:21.323 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:21.323 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:21.324 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:21.324 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:21.324 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:21.324 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:58:21.325 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T14:58:21.325 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:58:21.325+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:58:21.419 INFO:teuthology.orchestra.run.smithi032.stderr:reweighted osd.3 to 1 (10000) 2024-09-21T14:58:21.419 INFO:teuthology.orchestra.run.smithi032.stderr:nodeep-scrub is set 2024-09-21T14:58:21.425 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:58:21.425+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:58:21.429 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-21T14:58:21.459 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:58:21.459 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:58:21.459 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T14:58:21.459 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:58:21.470 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T14:58:21.527 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:58:21.525+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:58:21.587 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:58:21.588 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:58:21.588 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T14:58:21.588 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T14:58:21.588 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:58:21.600 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T14:58:21.628 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:58:21.625+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:58:21.705 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:58:21.705 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T14:58:21.705 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T14:58:21.705 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T14:58:21.706 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.706 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4281.0:1 2.3 e20/19 2:c4c92e5a:::rbd_trash:head v 20'1, mlcod=0'0)", 2024-09-21T14:58:21.706 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:09.440433+0000", 2024-09-21T14:58:21.706 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 12.263816951000001, 2024-09-21T14:58:21.706 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0013672630000000001, 2024-09-21T14:58:21.706 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:21.706 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:21.706 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:21.706 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.706 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:21.706 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.440433+0000", 2024-09-21T14:58:21.706 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:21.706 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.707 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.707 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:21.707 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.440433+0000", 2024-09-21T14:58:21.707 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:21.707 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.707 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.707 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:21.707 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.440432+0000", 2024-09-21T14:58:21.707 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:21.707 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.707 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.707 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:21.707 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.440450+0000", 2024-09-21T14:58:21.707 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7886000000000001e-05 2024-09-21T14:58:21.708 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.708 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.708 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:21.708 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.440470+0000", 2024-09-21T14:58:21.708 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9599999999999999e-05 2024-09-21T14:58:21.708 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.708 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.708 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:21.708 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.440488+0000", 2024-09-21T14:58:21.708 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8425999999999999e-05 2024-09-21T14:58:21.708 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.708 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.708 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:21.708 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.440578+0000", 2024-09-21T14:58:21.709 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.9535000000000002e-05 2024-09-21T14:58:21.709 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.709 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.709 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:21.709 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.440646+0000", 2024-09-21T14:58:21.709 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8205000000000002e-05 2024-09-21T14:58:21.709 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.709 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.709 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:21.709 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.441641+0000", 2024-09-21T14:58:21.709 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00099473399999999994 2024-09-21T14:58:21.709 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:21.709 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:21.709 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:21.709 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.441800+0000", 2024-09-21T14:58:21.710 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000159567 2024-09-21T14:58:21.710 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:21.710 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:21.710 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:21.710 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:21.710 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:21.710 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:58:21.715 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T14:58:21.728 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:58:21.724+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:58:21.822 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:58:21.822 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:58:21.822 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T14:58:21.822 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:58:21.829 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:58:21.829+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:58:21.832 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T14:58:21.930 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:58:21.929+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:58:21.934 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:58:21.934 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:58:21.934 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T14:58:21.934 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T14:58:21.934 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:58:21.943 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T14:58:22.031 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:58:22.028+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:58:22.045 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:58:22.045 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T14:58:22.045 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T14:58:22.045 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T14:58:22.045 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.045 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4281.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-21T14:58:22.046 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:09.324416+0000", 2024-09-21T14:58:22.046 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 12.718684854999999, 2024-09-21T14:58:22.046 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.117837947, 2024-09-21T14:58:22.046 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:22.046 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:22.046 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:22.046 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4281", 2024-09-21T14:58:22.046 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/1518520303", 2024-09-21T14:58:22.046 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1 2024-09-21T14:58:22.046 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.046 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:22.046 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.047 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:22.047 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324416+0000", 2024-09-21T14:58:22.047 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:22.047 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.047 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.047 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:22.047 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324416+0000", 2024-09-21T14:58:22.047 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:22.047 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.047 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.047 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:22.047 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324414+0000", 2024-09-21T14:58:22.047 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T14:58:22.047 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.048 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.048 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:22.048 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324429+0000", 2024-09-21T14:58:22.048 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5637e-05 2024-09-21T14:58:22.048 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.048 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.048 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:22.048 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324448+0000", 2024-09-21T14:58:22.048 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9042000000000002e-05 2024-09-21T14:58:22.048 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.048 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.048 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:22.048 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324465+0000", 2024-09-21T14:58:22.048 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6767999999999999e-05 2024-09-21T14:58:22.048 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.049 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.049 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:22.049 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324571+0000", 2024-09-21T14:58:22.049 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000105714 2024-09-21T14:58:22.049 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.049 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.049 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op must wait for map", 2024-09-21T14:58:22.049 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324587+0000", 2024-09-21T14:58:22.049 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5753999999999999e-05 2024-09-21T14:58:22.049 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.049 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.049 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:22.049 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.325864+0000", 2024-09-21T14:58:22.049 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0012773350000000001 2024-09-21T14:58:22.050 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.050 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.050 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T14:58:22.050 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.325878+0000", 2024-09-21T14:58:22.050 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4498000000000001e-05 2024-09-21T14:58:22.050 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.050 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.050 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:22.050 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.439461+0000", 2024-09-21T14:58:22.050 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.11358212500000001 2024-09-21T14:58:22.050 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.050 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.050 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:22.050 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.439848+0000", 2024-09-21T14:58:22.051 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00038747800000000002 2024-09-21T14:58:22.051 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.051 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.051 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:58:22.051 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.440175+0000", 2024-09-21T14:58:22.051 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00032721900000000001 2024-09-21T14:58:22.051 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.051 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.051 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:22.051 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.441770+0000", 2024-09-21T14:58:22.051 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0015949409999999999 2024-09-21T14:58:22.051 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.051 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.051 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:22.051 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.442086+0000", 2024-09-21T14:58:22.052 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00031586299999999999 2024-09-21T14:58:22.052 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.052 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.052 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:22.052 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.442195+0000", 2024-09-21T14:58:22.052 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000108791 2024-09-21T14:58:22.052 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.052 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.052 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:22.052 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.442254+0000", 2024-09-21T14:58:22.052 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9038e-05 2024-09-21T14:58:22.052 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:22.052 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:22.052 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:22.052 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.053 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.053 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop_reply(client.4281.0:1 2.3 e20/19 ondisk, result = 0)", 2024-09-21T14:58:22.053 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:09.441861+0000", 2024-09-21T14:58:22.053 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 12.601239361999999, 2024-09-21T14:58:22.053 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00040704400000000001, 2024-09-21T14:58:22.053 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:22.053 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:22.053 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:22.053 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.053 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:22.053 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.441861+0000", 2024-09-21T14:58:22.053 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:22.053 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.053 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.053 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:22.054 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.441861+0000", 2024-09-21T14:58:22.054 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:22.054 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.054 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.054 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:22.054 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.441860+0000", 2024-09-21T14:58:22.054 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-21T14:58:22.054 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.054 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.054 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:22.054 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.441872+0000", 2024-09-21T14:58:22.054 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2405999999999999e-05 2024-09-21T14:58:22.054 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.055 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.055 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:22.055 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.441908+0000", 2024-09-21T14:58:22.055 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6269999999999997e-05 2024-09-21T14:58:22.055 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.055 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.055 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:22.055 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.441934+0000", 2024-09-21T14:58:22.055 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5329000000000001e-05 2024-09-21T14:58:22.055 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.055 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.055 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:22.055 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.442053+0000", 2024-09-21T14:58:22.056 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000119821 2024-09-21T14:58:22.056 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.056 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.056 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:22.056 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.442072+0000", 2024-09-21T14:58:22.056 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8709999999999999e-05 2024-09-21T14:58:22.056 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.056 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.056 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:22.056 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.442268+0000", 2024-09-21T14:58:22.056 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000196295 2024-09-21T14:58:22.056 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:22.056 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:22.057 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:22.057 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.057 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.057 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4281.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-21T14:58:22.057 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:09.447828+0000", 2024-09-21T14:58:22.057 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 12.595272853999999, 2024-09-21T14:58:22.057 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.32904393700000001, 2024-09-21T14:58:22.057 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:22.057 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:58:22.057 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:22.057 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4281", 2024-09-21T14:58:22.057 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/1518520303", 2024-09-21T14:58:22.057 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 2 2024-09-21T14:58:22.057 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.058 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:22.058 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.058 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:22.058 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447828+0000", 2024-09-21T14:58:22.058 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:22.058 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.058 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.058 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:22.058 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447828+0000", 2024-09-21T14:58:22.058 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:22.058 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.058 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.058 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:22.058 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447825+0000", 2024-09-21T14:58:22.059 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T14:58:22.059 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.059 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.059 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:22.059 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447841+0000", 2024-09-21T14:58:22.059 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5359999999999999e-05 2024-09-21T14:58:22.059 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.059 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.059 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:22.059 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447875+0000", 2024-09-21T14:58:22.059 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4388999999999999e-05 2024-09-21T14:58:22.059 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.059 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.059 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:22.059 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447903+0000", 2024-09-21T14:58:22.060 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7373e-05 2024-09-21T14:58:22.060 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.060 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.060 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:22.060 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447996+0000", 2024-09-21T14:58:22.060 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2954999999999993e-05 2024-09-21T14:58:22.060 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.060 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.060 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T14:58:22.060 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.448011+0000", 2024-09-21T14:58:22.060 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5038999999999999e-05 2024-09-21T14:58:22.060 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.060 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.060 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:22.061 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.776501+0000", 2024-09-21T14:58:22.061 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.32849012500000002 2024-09-21T14:58:22.061 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.061 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.061 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:22.061 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.776872+0000", 2024-09-21T14:58:22.061 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00037119600000000002 2024-09-21T14:58:22.061 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:22.061 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:22.061 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:22.061 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.061 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.061 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4281.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-21T14:58:22.061 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:09.777483+0000", 2024-09-21T14:58:22.061 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 12.265618258, 2024-09-21T14:58:22.062 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00042328299999999999, 2024-09-21T14:58:22.062 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:22.062 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:58:22.062 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:22.062 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4281", 2024-09-21T14:58:22.062 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/1518520303", 2024-09-21T14:58:22.062 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 3 2024-09-21T14:58:22.062 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.062 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:22.062 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.062 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:22.062 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.777483+0000", 2024-09-21T14:58:22.062 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:22.062 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.063 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.063 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:22.063 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.777483+0000", 2024-09-21T14:58:22.063 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:22.063 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.063 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.063 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:22.063 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.777479+0000", 2024-09-21T14:58:22.063 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999962 2024-09-21T14:58:22.063 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.063 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.063 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:22.063 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.777496+0000", 2024-09-21T14:58:22.063 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7209000000000001e-05 2024-09-21T14:58:22.064 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.064 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.064 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:22.064 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.777543+0000", 2024-09-21T14:58:22.064 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7243000000000002e-05 2024-09-21T14:58:22.064 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.064 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.064 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:22.064 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.777566+0000", 2024-09-21T14:58:22.064 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2631999999999999e-05 2024-09-21T14:58:22.064 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.064 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.064 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:22.064 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.777694+0000", 2024-09-21T14:58:22.064 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012810100000000001 2024-09-21T14:58:22.065 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.065 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.065 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:22.065 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.777906+0000", 2024-09-21T14:58:22.065 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00021180099999999999 2024-09-21T14:58:22.065 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:22.065 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:22.065 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:22.065 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.065 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.065 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4281.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-21T14:58:22.065 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:10.338028+0000", 2024-09-21T14:58:22.065 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 11.705072527, 2024-09-21T14:58:22.066 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0061049110000000002, 2024-09-21T14:58:22.066 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:22.066 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:22.066 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:22.066 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4281", 2024-09-21T14:58:22.066 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/1518520303", 2024-09-21T14:58:22.066 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5 2024-09-21T14:58:22.066 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.066 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:22.066 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.066 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:22.066 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:10.338028+0000", 2024-09-21T14:58:22.066 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:22.066 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.067 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.067 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:22.067 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:10.338028+0000", 2024-09-21T14:58:22.067 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:22.067 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.067 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.067 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:22.067 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:10.338026+0000", 2024-09-21T14:58:22.067 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T14:58:22.067 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.067 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.067 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:22.067 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:10.338042+0000", 2024-09-21T14:58:22.067 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5586999999999998e-05 2024-09-21T14:58:22.068 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.068 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.068 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:22.068 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:10.338069+0000", 2024-09-21T14:58:22.068 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7229e-05 2024-09-21T14:58:22.068 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.068 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.068 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:22.068 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:10.338081+0000", 2024-09-21T14:58:22.068 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2593e-05 2024-09-21T14:58:22.068 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.068 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.068 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:22.068 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:10.338232+0000", 2024-09-21T14:58:22.068 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000150123 2024-09-21T14:58:22.068 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.069 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.069 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op must wait for map", 2024-09-21T14:58:22.069 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:10.338249+0000", 2024-09-21T14:58:22.069 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7346999999999998e-05 2024-09-21T14:58:22.069 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.069 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.069 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:22.069 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:10.340791+0000", 2024-09-21T14:58:22.069 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0025422740000000002 2024-09-21T14:58:22.069 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.069 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.069 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:22.069 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:10.340929+0000", 2024-09-21T14:58:22.069 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013819400000000001 2024-09-21T14:58:22.070 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.070 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.070 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 6", 2024-09-21T14:58:22.070 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:10.341054+0000", 2024-09-21T14:58:22.070 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012472699999999999 2024-09-21T14:58:22.070 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.070 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.070 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:22.070 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:10.342074+0000", 2024-09-21T14:58:22.070 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0010203370000000001 2024-09-21T14:58:22.070 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.070 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.070 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:22.070 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:10.344039+0000", 2024-09-21T14:58:22.071 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0019648909999999999 2024-09-21T14:58:22.071 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.071 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.071 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:22.071 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:10.344096+0000", 2024-09-21T14:58:22.071 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6644999999999997e-05 2024-09-21T14:58:22.071 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.071 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.071 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:22.071 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:10.344133+0000", 2024-09-21T14:58:22.071 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7246999999999998e-05 2024-09-21T14:58:22.071 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:22.071 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:22.072 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:22.072 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.072 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.072 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop_reply(client.4281.0:5 2.2 e20/19 ondisk, result = 0)", 2024-09-21T14:58:22.072 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:10.343933+0000", 2024-09-21T14:58:22.072 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 11.699167504, 2024-09-21T14:58:22.072 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00020858199999999999, 2024-09-21T14:58:22.072 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:22.072 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:22.072 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:22.072 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.072 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:22.072 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:10.343933+0000", 2024-09-21T14:58:22.072 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:22.072 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.072 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.073 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:22.073 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:10.343933+0000", 2024-09-21T14:58:22.073 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:22.073 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.073 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.073 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:22.073 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:10.343932+0000", 2024-09-21T14:58:22.073 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:58:22.073 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.073 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.073 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:22.073 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:10.343940+0000", 2024-09-21T14:58:22.073 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.362e-06 2024-09-21T14:58:22.073 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.074 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.074 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:22.074 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:10.343957+0000", 2024-09-21T14:58:22.074 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7180000000000002e-05 2024-09-21T14:58:22.074 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.074 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.074 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:22.074 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:10.343965+0000", 2024-09-21T14:58:22.074 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6009999999999995e-06 2024-09-21T14:58:22.074 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.074 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.074 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:22.074 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:10.344025+0000", 2024-09-21T14:58:22.074 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9474e-05 2024-09-21T14:58:22.075 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.075 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.075 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:22.075 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:10.344034+0000", 2024-09-21T14:58:22.075 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0340000000000002e-06 2024-09-21T14:58:22.075 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.075 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.075 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:22.075 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:10.344142+0000", 2024-09-21T14:58:22.075 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010808 2024-09-21T14:58:22.075 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:22.075 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:22.075 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:22.076 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.076 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.076 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4281.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-09-21T14:58:22.076 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:11.342188+0000", 2024-09-21T14:58:22.076 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.700912967000001, 2024-09-21T14:58:22.076 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0069221070000000003, 2024-09-21T14:58:22.076 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:22.076 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:22.076 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:22.076 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4281", 2024-09-21T14:58:22.076 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/1518520303", 2024-09-21T14:58:22.076 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 7 2024-09-21T14:58:22.076 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.076 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:22.076 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.076 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:22.077 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:11.342188+0000", 2024-09-21T14:58:22.077 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:22.077 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.077 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.077 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:22.077 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:11.342188+0000", 2024-09-21T14:58:22.077 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:22.077 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.077 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.077 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:22.077 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:11.342184+0000", 2024-09-21T14:58:22.077 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999967 2024-09-21T14:58:22.077 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.077 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.077 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:22.077 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:11.342201+0000", 2024-09-21T14:58:22.078 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6764000000000001e-05 2024-09-21T14:58:22.078 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.078 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.078 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:22.078 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:11.342235+0000", 2024-09-21T14:58:22.078 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3531999999999997e-05 2024-09-21T14:58:22.078 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.078 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.078 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:22.078 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:11.342250+0000", 2024-09-21T14:58:22.078 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5225000000000001e-05 2024-09-21T14:58:22.078 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.078 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.078 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:22.079 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:11.342366+0000", 2024-09-21T14:58:22.079 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000116462 2024-09-21T14:58:22.079 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.079 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.079 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op must wait for map", 2024-09-21T14:58:22.079 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:11.342376+0000", 2024-09-21T14:58:22.079 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8330000000000006e-06 2024-09-21T14:58:22.079 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.079 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.079 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:22.079 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:11.346408+0000", 2024-09-21T14:58:22.079 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0040313579999999996 2024-09-21T14:58:22.079 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.079 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.079 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:22.080 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:11.346537+0000", 2024-09-21T14:58:22.080 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012978299999999999 2024-09-21T14:58:22.080 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.080 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.080 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 6", 2024-09-21T14:58:22.080 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:11.346730+0000", 2024-09-21T14:58:22.080 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00019226700000000001 2024-09-21T14:58:22.080 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.080 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.080 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:22.080 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:11.347727+0000", 2024-09-21T14:58:22.080 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00099706600000000001 2024-09-21T14:58:22.080 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.080 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.080 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:22.080 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:11.349025+0000", 2024-09-21T14:58:22.081 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0012984069999999999 2024-09-21T14:58:22.081 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.081 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.081 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:22.081 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:11.349064+0000", 2024-09-21T14:58:22.081 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.8897999999999999e-05 2024-09-21T14:58:22.081 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.081 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.081 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:22.081 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:11.349110+0000", 2024-09-21T14:58:22.081 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5969e-05 2024-09-21T14:58:22.081 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:22.081 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:22.081 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:22.081 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.082 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.082 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop_reply(client.4281.0:7 2.2 e22/19 ondisk, result = 0)", 2024-09-21T14:58:22.082 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:11.348919+0000", 2024-09-21T14:58:22.082 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.694181934, 2024-09-21T14:58:22.082 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00019911500000000001, 2024-09-21T14:58:22.082 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:22.082 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:22.082 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:22.082 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.082 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:22.082 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:11.348919+0000", 2024-09-21T14:58:22.082 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:22.082 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.082 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.082 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:22.083 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:11.348919+0000", 2024-09-21T14:58:22.083 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:22.083 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.083 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.083 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:22.083 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:11.348917+0000", 2024-09-21T14:58:22.083 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-21T14:58:22.083 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.083 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.083 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:22.083 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:11.348931+0000", 2024-09-21T14:58:22.083 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3946e-05 2024-09-21T14:58:22.083 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.083 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.083 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:22.084 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:11.348956+0000", 2024-09-21T14:58:22.084 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4644999999999999e-05 2024-09-21T14:58:22.084 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.084 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.084 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:22.084 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:11.348971+0000", 2024-09-21T14:58:22.084 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5387999999999999e-05 2024-09-21T14:58:22.084 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.084 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.084 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:22.084 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:11.349015+0000", 2024-09-21T14:58:22.084 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3943000000000003e-05 2024-09-21T14:58:22.084 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.084 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.084 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:22.084 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:11.349021+0000", 2024-09-21T14:58:22.085 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3690000000000003e-06 2024-09-21T14:58:22.085 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:22.085 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:22.085 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:22.085 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:11.349118+0000", 2024-09-21T14:58:22.085 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6745999999999998e-05 2024-09-21T14:58:22.085 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:22.085 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:22.085 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:22.085 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:22.085 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:22.085 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:58:22.085 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T14:58:22.087 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T14:58:22.133 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:58:22.133+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:58:22.200 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:58:22.200 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:58:22.200 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T14:58:22.200 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:58:22.210 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T14:58:22.222 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T14:58:22.232 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:58:22.229+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:58:22.242 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T14:58:22.263 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T14:58:22.284 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T14:58:22.304 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-21T14:58:22.313 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:22.313 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:22.323 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T14:58:22.328 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T14:58:22.333 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:58:22.328+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:58:22.356 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T14:58:22.377 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T14:58:22.379 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:58:22.380 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:58:22.380 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:58:22.380 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:58:22.380 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:58:22.380 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:58:22.380 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:58:22.380 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:58:22.429 INFO:teuthology.orchestra.run.smithi032.stderr:reweighted osd.4 to 1 (10000) 2024-09-21T14:58:22.430 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:22.430 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:22.434 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:58:22.433+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:58:22.439 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-21T14:58:22.441 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T14:58:22.535 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:58:22.528+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:58:22.593 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:22.594 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:22.604 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T14:58:22.636 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:58:22.632+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:58:22.714 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:22.714 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:22.724 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T14:58:22.736 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:58:22.732+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:58:22.836 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:22.836 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:22.837 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:58:22.832+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:58:22.846 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T14:58:22.939 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:58:22.932+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:58:22.952 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:22.952 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:22.961 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T14:58:23.040 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:58:23.036+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:58:23.069 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:23.069 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:23.079 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T14:58:23.141 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:58:23.141+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:58:23.191 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:23.191 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:23.202 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T14:58:23.242 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:58:23.236+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:58:23.310 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:23.310 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:23.320 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T14:58:23.343 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:58:23.340+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:58:23.428 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:23.428 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:23.432 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T14:58:23.433 INFO:teuthology.orchestra.run.smithi032.stderr:reweighted osd.5 to 1 (10000) 2024-09-21T14:58:23.438 INFO:teuthology.orchestra.run.smithi032.stdout:1 write 0x502963c thru 0x502ccd5 (0x369a bytes) 2024-09-21T14:58:23.439 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T14:58:23.444 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:58:23.441+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:58:23.445 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-21T14:58:23.447 INFO:teuthology.orchestra.run.smithi032.stdout:truncating to largest ever: 0xbf00c89 2024-09-21T14:58:23.458 INFO:teuthology.orchestra.run.smithi032.stdout:2 trunc from 0x502ccd6 to 0xbf00c89 2024-09-21T14:58:23.480 INFO:teuthology.orchestra.run.smithi032.stdout:4 write 0xd525202 thru 0xd52df80 (0x8d7f bytes) 2024-09-21T14:58:23.487 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:23.487 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:23.487 INFO:teuthology.orchestra.run.smithi032.stdout:7 punch from 0xcdc3f5 to 0xce24e6, (0x60f1 bytes) 2024-09-21T14:58:23.487 INFO:teuthology.orchestra.run.smithi032.stdout:truncating to largest ever: 0xd7105b5 2024-09-21T14:58:23.487 INFO:teuthology.orchestra.run.smithi032.stdout:8 trunc from 0xd52df81 to 0xd7105b5 2024-09-21T14:58:23.498 INFO:teuthology.orchestra.run.smithi032.stdout:9 read 0x1711266 thru 0x1711835 (0x5d0 bytes) 2024-09-21T14:58:23.498 INFO:teuthology.orchestra.run.smithi032.stdout:10 read 0x5c3c185 thru 0x5c487e8 (0xc664 bytes) 2024-09-21T14:58:23.498 INFO:teuthology.orchestra.run.smithi032.stdout:11 write 0xb617bc thru 0xb71044 (0xf889 bytes) 2024-09-21T14:58:23.506 INFO:teuthology.orchestra.run.smithi032.stdout:12 read 0x8e62ad3 thru 0x8e6461f (0x1b4d bytes) 2024-09-21T14:58:23.506 INFO:teuthology.orchestra.run.smithi032.stdout:13 writesame 0x10b3f18 thru 0x10bd2ed data_size 0x7f(0x93d6 bytes) 2024-09-21T14:58:23.512 INFO:teuthology.orchestra.run.smithi032.stdout:14 punch from 0xcb6767d to 0xcb742f5, (0xcc78 bytes) 2024-09-21T14:58:23.512 INFO:teuthology.orchestra.run.smithi032.stdout:15 write 0x2510a28 thru 0x2518630 (0x7c09 bytes) 2024-09-21T14:58:23.519 INFO:teuthology.orchestra.run.smithi032.stdout:16 read 0x9d579a4 thru 0x9d6027b (0x88d8 bytes) 2024-09-21T14:58:23.519 INFO:teuthology.orchestra.run.smithi032.stdout:19 write 0x46140d2 thru 0x461812f (0x405e bytes) 2024-09-21T14:58:23.524 INFO:teuthology.orchestra.run.smithi032.stdout:20 write 0x5e61641 thru 0x5e6da08 (0xc3c8 bytes) 2024-09-21T14:58:23.530 INFO:teuthology.orchestra.run.smithi032.stdout:21 punch from 0x180dccc to 0x181a104, (0xc438 bytes) 2024-09-21T14:58:23.530 INFO:teuthology.orchestra.run.smithi032.stdout:23 writesame 0x9afc3fc thru 0x9afe4d5 data_size 0x1d(0x20da bytes) 2024-09-21T14:58:23.542 INFO:teuthology.orchestra.run.smithi032.stdout:24 trunc from 0xd7105b5 to 0x9ca49ae 2024-09-21T14:58:23.545 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:58:23.545+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:58:23.551 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:23.551 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:23.561 INFO:teuthology.orchestra.run.smithi032.stdout:25 read 0x53deec thru 0x54d5b6 (0xf6cb bytes) 2024-09-21T14:58:23.561 INFO:teuthology.orchestra.run.smithi032.stdout:26 punch from 0x979eb33 to 0x97ac312, (0xd7df bytes) 2024-09-21T14:58:23.561 INFO:teuthology.orchestra.run.smithi032.stdout:27 write 0x483031e thru 0x483809a (0x7d7d bytes) 2024-09-21T14:58:23.562 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T14:58:23.567 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:23.569 INFO:teuthology.orchestra.run.smithi032.stdout:29 trunc from 0x9ca49ae to 0x7de78c3 2024-09-21T14:58:23.592 INFO:teuthology.orchestra.run.smithi032.stdout:30 trunc from 0x7de78c3 to 0x349c888 2024-09-21T14:58:23.613 INFO:teuthology.orchestra.run.smithi032.stdout:31 punch from 0x1d0e74b to 0x1d17971, (0x9226 bytes) 2024-09-21T14:58:23.613 INFO:teuthology.orchestra.run.smithi032.stdout:32 read 0x223b885 thru 0x223ec5e (0x33da bytes) 2024-09-21T14:58:23.628 INFO:teuthology.orchestra.run.smithi032.stdout:33 write 0x5e77b02 thru 0x5e7ef77 (0x7476 bytes) 2024-09-21T14:58:23.647 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:58:23.645+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:58:23.657 INFO:teuthology.orchestra.run.smithi032.stdout:36 writesame 0xcf377ee thru 0xcf41eaf data_size 0xf(0xa6c2 bytes) 2024-09-21T14:58:23.657 INFO:teuthology.orchestra.run.smithi032.stdout:37 write 0xbe32a92 thru 0xbe35086 (0x25f5 bytes) 2024-09-21T14:58:23.671 INFO:teuthology.orchestra.run.smithi032.stdout:39 trunc from 0xcf41eb0 to 0x84f27f6 2024-09-21T14:58:23.692 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:23.692 INFO:teuthology.orchestra.run.smithi032.stdout:41 punch from 0x717be60 to 0x7189b31, (0xdcd1 bytes) 2024-09-21T14:58:23.692 INFO:teuthology.orchestra.run.smithi032.stdout:42 read 0x24e6e07 thru 0x24f6432 (0xf62c bytes) 2024-09-21T14:58:23.693 INFO:teuthology.orchestra.run.smithi032.stdout:44 read 0x20400ad thru 0x2040c5c (0xbb0 bytes) 2024-09-21T14:58:23.693 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:23.693 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:23.693 INFO:teuthology.orchestra.run.smithi032.stdout:50 read 0x49604bb thru 0x496c1e5 (0xbd2b bytes) 2024-09-21T14:58:23.693 INFO:teuthology.orchestra.run.smithi032.stdout:51 punch from 0x10a8121 to 0x10b4055, (0xbf34 bytes) 2024-09-21T14:58:23.693 INFO:teuthology.orchestra.run.smithi032.stdout:52 read 0xea3957 thru 0xeae93c (0xafe6 bytes) 2024-09-21T14:58:23.693 INFO:teuthology.orchestra.run.smithi032.stdout:truncating to largest ever: 0xdd4d710 2024-09-21T14:58:23.702 INFO:teuthology.orchestra.run.smithi032.stdout:53 trunc from 0x84f27f6 to 0xdd4d710 2024-09-21T14:58:23.712 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:23.712 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:23.714 INFO:teuthology.orchestra.run.smithi032.stdout:54 read 0xb4285f1 thru 0xb42af3c (0x294c bytes) 2024-09-21T14:58:23.721 INFO:teuthology.orchestra.run.smithi032.stdout:55 trunc from 0xdd4d710 to 0x8477239 2024-09-21T14:58:23.727 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T14:58:23.732 INFO:teuthology.orchestra.run.smithi032.stdout:56 writesame 0x406bd8d thru 0x40782dc data_size 0x38(0xc550 bytes) 2024-09-21T14:58:23.740 INFO:teuthology.orchestra.run.smithi032.stdout:57 write 0x7b179a thru 0x7b4694 (0x2efb bytes) 2024-09-21T14:58:23.746 INFO:teuthology.orchestra.run.smithi032.stdout:58 write 0x766e95d thru 0x767a427 (0xbacb bytes) 2024-09-21T14:58:23.747 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:58:23.745+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:58:23.753 INFO:teuthology.orchestra.run.smithi032.stdout:59 read 0x227fccc thru 0x22820f6 (0x242b bytes) 2024-09-21T14:58:23.754 INFO:teuthology.orchestra.run.smithi032.stdout:60 trunc from 0x8477239 to 0x979828b 2024-09-21T14:58:23.782 INFO:teuthology.orchestra.run.smithi032.stdout:61 write 0xc9b2870 thru 0xc9b3afe (0x128f bytes) 2024-09-21T14:58:23.789 INFO:teuthology.orchestra.run.smithi032.stdout:62 write 0xc5886b8 thru 0xc58fb47 (0x7490 bytes) 2024-09-21T14:58:23.795 INFO:teuthology.orchestra.run.smithi032.stdout:64 punch from 0x5d904aa to 0x5d92cda, (0x2830 bytes) 2024-09-21T14:58:23.795 INFO:teuthology.orchestra.run.smithi032.stdout:65 writesame 0xc86e10e thru 0xc8708d0 data_size 0x27(0x27c3 bytes) 2024-09-21T14:58:23.813 INFO:teuthology.orchestra.run.smithi032.stdout:66 trunc from 0xc9b3aff to 0x34213cd 2024-09-21T14:58:23.835 INFO:teuthology.orchestra.run.smithi032.stdout:69 read 0x39097e thru 0x39cd3e (0xc3c1 bytes) 2024-09-21T14:58:23.836 INFO:teuthology.orchestra.run.smithi032.stdout:71 read 0x336448c thru 0x336e7bf (0xa334 bytes) 2024-09-21T14:58:23.836 INFO:teuthology.orchestra.run.smithi032.stdout:72 read 0x39362a thru 0x3a0b62 (0xd539 bytes) 2024-09-21T14:58:23.847 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:23.847 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:23.848 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:58:23.844+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:58:23.859 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T14:58:23.863 INFO:teuthology.orchestra.run.smithi032.stdout:73 write 0xe5b6db9 thru 0xe5b9576 (0x27be bytes) 2024-09-21T14:58:23.870 INFO:teuthology.orchestra.run.smithi032.stdout:74 read 0x5e6082c thru 0x5e6ee03 (0xe5d8 bytes) 2024-09-21T14:58:23.870 INFO:teuthology.orchestra.run.smithi032.stdout:75 clone 1 order 24 su 1048576 sc 11 2024-09-21T14:58:23.949 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:58:23.949+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:58:23.965 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:23.965 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:23.975 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T14:58:24.050 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:58:24.044+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:58:24.130 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:24.131 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:24.141 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T14:58:24.150 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:58:24.144+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:58:24.240 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:24.240 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:24.249 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T14:58:24.291 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:58:24.288+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:58:24.352 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:58:24.348+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:58:24.396 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T14:58:24.402 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:24.402 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:24.412 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T14:58:24.454 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:58:24.453+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:58:24.455 INFO:teuthology.orchestra.run.smithi032.stderr:reweighted osd.6 to 1 (10000) 2024-09-21T14:58:24.455 INFO:teuthology.orchestra.run.smithi032.stderr:noscrub is unset 2024-09-21T14:58:24.462 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-21T14:58:24.544 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T14:58:24.554 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:58:24.553+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:58:24.558 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:24.558 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:24.562 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T14:58:24.568 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T14:58:24.583 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T14:58:24.604 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T14:58:24.655 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:58:24.653+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:58:24.671 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:58:24.671 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:58:24.672 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T14:58:24.672 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T14:58:24.672 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:58:24.681 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T14:58:24.756 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:58:24.753+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:58:24.780 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:58:24.780 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T14:58:24.781 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T14:58:24.781 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T14:58:24.781 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:24.781 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4281.0:1 2.3 e20/19 2:c4c92e5a:::rbd_trash:head v 20'1, mlcod=0'0)", 2024-09-21T14:58:24.781 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:09.440433+0000", 2024-09-21T14:58:24.781 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 15.338984908, 2024-09-21T14:58:24.781 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0013672630000000001, 2024-09-21T14:58:24.781 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:24.781 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:24.781 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:24.781 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:24.781 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:24.781 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.440433+0000", 2024-09-21T14:58:24.781 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:24.782 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:24.782 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:24.782 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:24.782 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.440433+0000", 2024-09-21T14:58:24.782 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:24.782 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:24.782 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:24.782 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:24.782 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.440432+0000", 2024-09-21T14:58:24.782 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:24.782 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:24.782 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:24.782 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:24.782 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.440450+0000", 2024-09-21T14:58:24.783 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7886000000000001e-05 2024-09-21T14:58:24.783 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:24.783 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:24.783 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:24.783 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.440470+0000", 2024-09-21T14:58:24.783 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9599999999999999e-05 2024-09-21T14:58:24.783 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:24.783 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:24.783 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:24.783 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.440488+0000", 2024-09-21T14:58:24.783 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8425999999999999e-05 2024-09-21T14:58:24.783 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:24.783 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:24.784 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:24.784 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.440578+0000", 2024-09-21T14:58:24.784 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.9535000000000002e-05 2024-09-21T14:58:24.784 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:24.784 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:24.784 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:24.784 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.440646+0000", 2024-09-21T14:58:24.784 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8205000000000002e-05 2024-09-21T14:58:24.784 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:24.784 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:24.784 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:24.784 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.441641+0000", 2024-09-21T14:58:24.784 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00099473399999999994 2024-09-21T14:58:24.784 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:24.784 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:24.785 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:24.785 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.441800+0000", 2024-09-21T14:58:24.785 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000159567 2024-09-21T14:58:24.785 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:24.785 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:24.785 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:24.785 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:24.785 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:24.785 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:58:24.790 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T14:58:24.833 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-21T14:58:24.857 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T14:58:24.857 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:58:24.852+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:58:24.880 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T14:58:24.893 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:58:24.893 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:58:24.893 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T14:58:24.893 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:58:24.904 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T14:58:24.904 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T14:58:24.906 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:58:24.906 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:58:24.906 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:58:24.906 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:58:24.907 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:58:24.907 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:58:24.907 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:58:24.907 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:58:24.959 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:58:24.957+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:58:25.012 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:58:25.012 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:58:25.012 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T14:58:25.012 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T14:58:25.012 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:58:25.022 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T14:58:25.071 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:58:25.068+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:58:25.133 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:58:25.133 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T14:58:25.133 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T14:58:25.133 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T14:58:25.134 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.134 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4281.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-21T14:58:25.134 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:09.324416+0000", 2024-09-21T14:58:25.134 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 15.806714385999999, 2024-09-21T14:58:25.134 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.117837947, 2024-09-21T14:58:25.134 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:25.134 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:25.134 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:25.134 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4281", 2024-09-21T14:58:25.134 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/1518520303", 2024-09-21T14:58:25.134 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1 2024-09-21T14:58:25.134 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.135 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:25.135 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.135 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:25.135 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324416+0000", 2024-09-21T14:58:25.135 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:25.135 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.135 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.135 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:25.135 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324416+0000", 2024-09-21T14:58:25.135 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:25.135 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.135 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.135 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:25.135 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324414+0000", 2024-09-21T14:58:25.135 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T14:58:25.136 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.136 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.136 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:25.136 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324429+0000", 2024-09-21T14:58:25.136 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5637e-05 2024-09-21T14:58:25.136 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.136 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.136 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:25.136 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324448+0000", 2024-09-21T14:58:25.136 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9042000000000002e-05 2024-09-21T14:58:25.136 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.136 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.136 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:25.136 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324465+0000", 2024-09-21T14:58:25.136 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6767999999999999e-05 2024-09-21T14:58:25.137 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.137 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.137 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:25.137 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324571+0000", 2024-09-21T14:58:25.137 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000105714 2024-09-21T14:58:25.137 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.137 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.137 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op must wait for map", 2024-09-21T14:58:25.137 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324587+0000", 2024-09-21T14:58:25.137 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5753999999999999e-05 2024-09-21T14:58:25.137 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.137 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.137 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:25.137 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.325864+0000", 2024-09-21T14:58:25.138 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0012773350000000001 2024-09-21T14:58:25.138 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.138 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.138 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T14:58:25.138 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.325878+0000", 2024-09-21T14:58:25.138 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4498000000000001e-05 2024-09-21T14:58:25.138 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.138 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.138 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:25.138 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.439461+0000", 2024-09-21T14:58:25.138 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.11358212500000001 2024-09-21T14:58:25.138 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.138 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.138 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:25.139 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.439848+0000", 2024-09-21T14:58:25.139 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00038747800000000002 2024-09-21T14:58:25.139 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.139 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.139 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:58:25.139 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.440175+0000", 2024-09-21T14:58:25.139 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00032721900000000001 2024-09-21T14:58:25.139 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.139 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.139 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:25.139 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.441770+0000", 2024-09-21T14:58:25.139 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0015949409999999999 2024-09-21T14:58:25.139 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.139 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.139 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:25.140 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.442086+0000", 2024-09-21T14:58:25.140 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00031586299999999999 2024-09-21T14:58:25.140 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.140 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.140 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:25.140 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.442195+0000", 2024-09-21T14:58:25.140 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000108791 2024-09-21T14:58:25.140 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.140 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.140 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:25.140 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.442254+0000", 2024-09-21T14:58:25.140 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9038e-05 2024-09-21T14:58:25.140 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:25.140 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:25.141 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:25.141 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.141 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.141 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop_reply(client.4281.0:1 2.3 e20/19 ondisk, result = 0)", 2024-09-21T14:58:25.141 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:09.441861+0000", 2024-09-21T14:58:25.141 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 15.689268892999999, 2024-09-21T14:58:25.141 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00040704400000000001, 2024-09-21T14:58:25.141 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:25.141 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:25.141 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:25.141 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.141 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:25.141 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.441861+0000", 2024-09-21T14:58:25.141 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:25.141 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.142 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.142 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:25.142 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.441861+0000", 2024-09-21T14:58:25.142 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:25.142 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.142 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.142 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:25.142 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.441860+0000", 2024-09-21T14:58:25.142 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-21T14:58:25.142 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.142 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.142 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:25.142 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.441872+0000", 2024-09-21T14:58:25.143 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2405999999999999e-05 2024-09-21T14:58:25.143 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.143 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.143 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:25.143 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.441908+0000", 2024-09-21T14:58:25.143 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6269999999999997e-05 2024-09-21T14:58:25.143 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.143 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.143 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:25.143 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.441934+0000", 2024-09-21T14:58:25.143 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5329000000000001e-05 2024-09-21T14:58:25.143 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.143 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.143 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:25.144 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.442053+0000", 2024-09-21T14:58:25.144 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000119821 2024-09-21T14:58:25.144 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.144 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.144 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:25.144 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.442072+0000", 2024-09-21T14:58:25.144 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8709999999999999e-05 2024-09-21T14:58:25.144 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.144 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.144 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:25.144 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.442268+0000", 2024-09-21T14:58:25.144 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000196295 2024-09-21T14:58:25.144 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:25.144 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:25.145 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:25.145 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.145 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.145 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4281.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-21T14:58:25.145 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:09.447828+0000", 2024-09-21T14:58:25.145 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 15.683302384999999, 2024-09-21T14:58:25.145 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.32904393700000001, 2024-09-21T14:58:25.145 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:25.145 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:58:25.145 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:25.145 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4281", 2024-09-21T14:58:25.145 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/1518520303", 2024-09-21T14:58:25.145 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 2 2024-09-21T14:58:25.145 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.146 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:25.146 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.146 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:25.146 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447828+0000", 2024-09-21T14:58:25.146 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:25.146 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.146 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.146 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:25.146 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447828+0000", 2024-09-21T14:58:25.146 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:25.146 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.146 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.146 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:25.146 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447825+0000", 2024-09-21T14:58:25.147 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T14:58:25.147 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.147 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.147 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:25.147 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447841+0000", 2024-09-21T14:58:25.147 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5359999999999999e-05 2024-09-21T14:58:25.147 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.147 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.147 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:25.147 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447875+0000", 2024-09-21T14:58:25.147 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4388999999999999e-05 2024-09-21T14:58:25.147 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.147 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.148 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:25.148 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447903+0000", 2024-09-21T14:58:25.148 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7373e-05 2024-09-21T14:58:25.148 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.148 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.148 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:25.148 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447996+0000", 2024-09-21T14:58:25.148 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2954999999999993e-05 2024-09-21T14:58:25.148 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.148 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.148 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T14:58:25.148 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.448011+0000", 2024-09-21T14:58:25.148 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5038999999999999e-05 2024-09-21T14:58:25.149 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.149 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.149 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:25.149 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.776501+0000", 2024-09-21T14:58:25.149 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.32849012500000002 2024-09-21T14:58:25.149 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.149 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.149 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:25.149 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.776872+0000", 2024-09-21T14:58:25.149 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00037119600000000002 2024-09-21T14:58:25.149 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:25.149 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:25.149 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:25.149 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.150 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.150 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4281.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-21T14:58:25.150 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:09.777483+0000", 2024-09-21T14:58:25.150 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 15.353647789, 2024-09-21T14:58:25.150 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00042328299999999999, 2024-09-21T14:58:25.150 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:25.150 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:58:25.150 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:25.150 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4281", 2024-09-21T14:58:25.150 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/1518520303", 2024-09-21T14:58:25.150 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 3 2024-09-21T14:58:25.150 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.150 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:25.150 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.151 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:25.151 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.777483+0000", 2024-09-21T14:58:25.151 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:25.151 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.151 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.151 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:25.151 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.777483+0000", 2024-09-21T14:58:25.151 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:25.151 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.151 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.151 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:25.151 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.777479+0000", 2024-09-21T14:58:25.151 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999962 2024-09-21T14:58:25.151 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.152 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.152 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:25.152 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.777496+0000", 2024-09-21T14:58:25.152 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7209000000000001e-05 2024-09-21T14:58:25.152 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.152 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.152 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:25.152 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.777543+0000", 2024-09-21T14:58:25.152 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7243000000000002e-05 2024-09-21T14:58:25.152 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.152 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.152 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:25.152 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.777566+0000", 2024-09-21T14:58:25.152 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2631999999999999e-05 2024-09-21T14:58:25.153 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.153 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.153 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:25.153 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.777694+0000", 2024-09-21T14:58:25.153 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012810100000000001 2024-09-21T14:58:25.153 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.153 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.153 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:25.153 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.777906+0000", 2024-09-21T14:58:25.153 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00021180099999999999 2024-09-21T14:58:25.153 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:25.153 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:25.153 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:25.153 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.154 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.154 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4281.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-21T14:58:25.154 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:10.338028+0000", 2024-09-21T14:58:25.154 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.793102058000001, 2024-09-21T14:58:25.154 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0061049110000000002, 2024-09-21T14:58:25.154 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:25.154 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:25.154 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:25.154 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4281", 2024-09-21T14:58:25.154 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/1518520303", 2024-09-21T14:58:25.154 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 5 2024-09-21T14:58:25.154 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.155 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:25.155 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.155 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:25.155 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:10.338028+0000", 2024-09-21T14:58:25.155 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:25.155 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.155 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.155 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:25.155 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:10.338028+0000", 2024-09-21T14:58:25.155 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:25.155 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.155 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.155 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:25.155 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:10.338026+0000", 2024-09-21T14:58:25.156 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T14:58:25.156 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.156 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.156 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:25.156 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:10.338042+0000", 2024-09-21T14:58:25.156 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5586999999999998e-05 2024-09-21T14:58:25.156 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.156 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.156 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:25.156 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:10.338069+0000", 2024-09-21T14:58:25.156 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7229e-05 2024-09-21T14:58:25.156 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.156 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.156 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:25.157 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:10.338081+0000", 2024-09-21T14:58:25.157 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2593e-05 2024-09-21T14:58:25.157 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.157 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.157 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:25.157 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:10.338232+0000", 2024-09-21T14:58:25.157 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000150123 2024-09-21T14:58:25.157 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.157 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.157 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op must wait for map", 2024-09-21T14:58:25.157 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:10.338249+0000", 2024-09-21T14:58:25.157 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7346999999999998e-05 2024-09-21T14:58:25.157 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.157 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.158 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:25.158 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:10.340791+0000", 2024-09-21T14:58:25.158 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0025422740000000002 2024-09-21T14:58:25.158 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.158 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.158 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:25.158 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:10.340929+0000", 2024-09-21T14:58:25.158 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013819400000000001 2024-09-21T14:58:25.158 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.158 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.158 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 6", 2024-09-21T14:58:25.158 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:10.341054+0000", 2024-09-21T14:58:25.158 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012472699999999999 2024-09-21T14:58:25.158 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.159 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.159 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:25.159 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:10.342074+0000", 2024-09-21T14:58:25.159 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0010203370000000001 2024-09-21T14:58:25.159 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.159 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.159 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:25.159 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:10.344039+0000", 2024-09-21T14:58:25.159 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0019648909999999999 2024-09-21T14:58:25.159 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.159 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.159 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:25.159 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:10.344096+0000", 2024-09-21T14:58:25.159 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6644999999999997e-05 2024-09-21T14:58:25.160 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.160 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.160 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:25.160 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:10.344133+0000", 2024-09-21T14:58:25.160 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7246999999999998e-05 2024-09-21T14:58:25.160 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:25.160 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:25.160 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:25.160 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.160 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.160 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop_reply(client.4281.0:5 2.2 e20/19 ondisk, result = 0)", 2024-09-21T14:58:25.160 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:10.343933+0000", 2024-09-21T14:58:25.160 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.787197035, 2024-09-21T14:58:25.160 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00020858199999999999, 2024-09-21T14:58:25.161 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:25.161 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:25.161 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:25.161 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.161 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:25.161 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:10.343933+0000", 2024-09-21T14:58:25.161 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:25.161 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.161 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.161 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:25.161 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:10.343933+0000", 2024-09-21T14:58:25.161 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:25.161 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.161 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.162 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:25.162 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:10.343932+0000", 2024-09-21T14:58:25.162 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:58:25.162 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.162 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.162 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:25.162 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:10.343940+0000", 2024-09-21T14:58:25.162 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.362e-06 2024-09-21T14:58:25.162 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.162 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.162 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:25.162 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:10.343957+0000", 2024-09-21T14:58:25.162 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7180000000000002e-05 2024-09-21T14:58:25.163 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.163 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.163 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:25.163 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:10.343965+0000", 2024-09-21T14:58:25.163 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6009999999999995e-06 2024-09-21T14:58:25.163 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.163 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.163 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:25.163 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:10.344025+0000", 2024-09-21T14:58:25.163 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9474e-05 2024-09-21T14:58:25.163 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.163 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.163 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:25.163 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:10.344034+0000", 2024-09-21T14:58:25.164 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0340000000000002e-06 2024-09-21T14:58:25.164 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.164 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.164 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:25.164 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:10.344142+0000", 2024-09-21T14:58:25.164 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010808 2024-09-21T14:58:25.164 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:25.164 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:25.164 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:25.164 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.164 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.164 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4281.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-09-21T14:58:25.164 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:11.342188+0000", 2024-09-21T14:58:25.165 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 13.788942498000001, 2024-09-21T14:58:25.165 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0069221070000000003, 2024-09-21T14:58:25.165 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:25.165 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:25.165 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:25.165 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4281", 2024-09-21T14:58:25.165 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/1518520303", 2024-09-21T14:58:25.165 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 7 2024-09-21T14:58:25.165 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.165 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:25.165 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.165 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:25.165 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:11.342188+0000", 2024-09-21T14:58:25.165 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:25.166 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.166 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.166 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:25.166 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:11.342188+0000", 2024-09-21T14:58:25.166 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:25.166 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.166 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.166 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:25.166 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:11.342184+0000", 2024-09-21T14:58:25.166 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999967 2024-09-21T14:58:25.166 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.166 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.166 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:25.167 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:11.342201+0000", 2024-09-21T14:58:25.167 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6764000000000001e-05 2024-09-21T14:58:25.167 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.167 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.167 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:25.167 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:11.342235+0000", 2024-09-21T14:58:25.167 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3531999999999997e-05 2024-09-21T14:58:25.167 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.167 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.167 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:25.167 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:11.342250+0000", 2024-09-21T14:58:25.167 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5225000000000001e-05 2024-09-21T14:58:25.167 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.167 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.168 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:25.168 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:11.342366+0000", 2024-09-21T14:58:25.168 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000116462 2024-09-21T14:58:25.168 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.168 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.168 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op must wait for map", 2024-09-21T14:58:25.168 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:11.342376+0000", 2024-09-21T14:58:25.168 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8330000000000006e-06 2024-09-21T14:58:25.168 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.168 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.168 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:25.168 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:11.346408+0000", 2024-09-21T14:58:25.168 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0040313579999999996 2024-09-21T14:58:25.168 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.169 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.169 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:25.169 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:11.346537+0000", 2024-09-21T14:58:25.169 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012978299999999999 2024-09-21T14:58:25.169 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.169 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.169 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 6", 2024-09-21T14:58:25.169 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:11.346730+0000", 2024-09-21T14:58:25.169 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00019226700000000001 2024-09-21T14:58:25.169 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.169 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.169 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:25.169 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:11.347727+0000", 2024-09-21T14:58:25.169 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00099706600000000001 2024-09-21T14:58:25.170 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.170 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.170 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:25.170 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:11.349025+0000", 2024-09-21T14:58:25.170 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0012984069999999999 2024-09-21T14:58:25.170 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.170 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.170 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:25.170 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:11.349064+0000", 2024-09-21T14:58:25.170 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.8897999999999999e-05 2024-09-21T14:58:25.170 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.170 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.170 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:25.171 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:11.349110+0000", 2024-09-21T14:58:25.171 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5969e-05 2024-09-21T14:58:25.171 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:25.171 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:25.171 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:25.171 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.171 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.171 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop_reply(client.4281.0:7 2.2 e22/19 ondisk, result = 0)", 2024-09-21T14:58:25.171 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:11.348919+0000", 2024-09-21T14:58:25.171 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 13.782211465, 2024-09-21T14:58:25.171 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00019911500000000001, 2024-09-21T14:58:25.171 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:25.171 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:25.171 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:25.171 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.172 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:25.172 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:11.348919+0000", 2024-09-21T14:58:25.172 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:25.172 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.172 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.172 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:25.172 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:11.348919+0000", 2024-09-21T14:58:25.172 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:25.172 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.172 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.172 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:25.172 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:11.348917+0000", 2024-09-21T14:58:25.172 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-21T14:58:25.172 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.172 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.173 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:25.173 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:11.348931+0000", 2024-09-21T14:58:25.173 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3946e-05 2024-09-21T14:58:25.173 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.173 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.173 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:25.173 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:11.348956+0000", 2024-09-21T14:58:25.173 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4644999999999999e-05 2024-09-21T14:58:25.173 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.173 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.173 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:25.173 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:11.348971+0000", 2024-09-21T14:58:25.173 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5387999999999999e-05 2024-09-21T14:58:25.173 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.174 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.174 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:25.174 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:11.349015+0000", 2024-09-21T14:58:25.174 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3943000000000003e-05 2024-09-21T14:58:25.174 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.174 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.174 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:25.174 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:11.349021+0000", 2024-09-21T14:58:25.174 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3690000000000003e-06 2024-09-21T14:58:25.174 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:25.174 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:25.174 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:25.174 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:11.349118+0000", 2024-09-21T14:58:25.174 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6745999999999998e-05 2024-09-21T14:58:25.175 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:25.175 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:25.175 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:25.175 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:25.175 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:25.175 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:58:25.176 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T14:58:25.177 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:58:25.172+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:58:25.277 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:58:25.272+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:58:25.295 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:58:25.295 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:58:25.295 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T14:58:25.295 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:58:25.305 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T14:58:25.378 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:58:25.377+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:58:25.411 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:58:25.411 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:58:25.411 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T14:58:25.411 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T14:58:25.411 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:58:25.420 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T14:58:25.451 INFO:teuthology.orchestra.run.smithi032.stderr:reweighted osd.7 to 1 (10000) 2024-09-21T14:58:25.467 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-21T14:58:25.467 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T14:58:25.479 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:58:25.477+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:58:25.540 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:58:25.540 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T14:58:25.540 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T14:58:25.540 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T14:58:25.540 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.540 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12 3.6 3:6748301c:::rbd_header.11367ffd86fe:head [create,call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T14:58:25.540 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:21.422589+0000", 2024-09-21T14:58:25.540 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.1166317020000003, 2024-09-21T14:58:25.540 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0059205330000000004, 2024-09-21T14:58:25.540 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:25.541 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:25.541 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:25.541 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:25.541 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:25.541 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12 2024-09-21T14:58:25.541 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.541 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:25.541 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.541 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:25.541 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.422589+0000", 2024-09-21T14:58:25.541 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:25.541 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.541 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.541 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:25.542 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.422589+0000", 2024-09-21T14:58:25.542 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:25.542 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.542 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.542 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:25.542 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.422587+0000", 2024-09-21T14:58:25.542 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T14:58:25.542 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.542 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.542 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:25.542 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.422598+0000", 2024-09-21T14:58:25.542 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0889e-05 2024-09-21T14:58:25.542 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.542 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.543 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:25.543 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.422614+0000", 2024-09-21T14:58:25.543 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5869999999999999e-05 2024-09-21T14:58:25.543 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.543 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.543 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:25.543 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.422632+0000", 2024-09-21T14:58:25.543 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8172000000000002e-05 2024-09-21T14:58:25.543 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.543 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.543 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:25.543 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.422709+0000", 2024-09-21T14:58:25.543 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6993e-05 2024-09-21T14:58:25.544 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.544 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.544 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op must wait for map", 2024-09-21T14:58:25.544 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.422714+0000", 2024-09-21T14:58:25.544 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.369e-06 2024-09-21T14:58:25.544 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.544 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.544 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:25.544 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.425359+0000", 2024-09-21T14:58:25.544 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0026451970000000002 2024-09-21T14:58:25.544 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.544 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.544 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:25.544 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.425855+0000", 2024-09-21T14:58:25.545 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000496591 2024-09-21T14:58:25.545 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.545 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.545 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 7", 2024-09-21T14:58:25.545 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.426174+0000", 2024-09-21T14:58:25.545 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00031870899999999997 2024-09-21T14:58:25.545 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.545 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.545 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:58:25.545 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.427445+0000", 2024-09-21T14:58:25.545 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001270497 2024-09-21T14:58:25.545 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.545 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.546 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:25.546 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.428336+0000", 2024-09-21T14:58:25.546 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00089194000000000003 2024-09-21T14:58:25.546 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.546 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.546 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:25.546 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.428443+0000", 2024-09-21T14:58:25.546 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000106279 2024-09-21T14:58:25.546 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.546 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.546 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:25.546 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.428510+0000", 2024-09-21T14:58:25.546 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7070999999999995e-05 2024-09-21T14:58:25.547 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:25.547 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:25.547 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:25.547 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.547 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.547 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop_reply(client.4406.0:12 3.6 e26/24 ondisk, result = 0)", 2024-09-21T14:58:25.547 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:21.428198+0000", 2024-09-21T14:58:25.547 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.111022749, 2024-09-21T14:58:25.547 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000328754, 2024-09-21T14:58:25.547 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:25.547 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:25.547 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:25.547 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.547 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:25.548 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.428198+0000", 2024-09-21T14:58:25.548 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:25.548 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.548 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.548 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:25.548 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.428198+0000", 2024-09-21T14:58:25.548 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:25.548 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.548 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.548 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:25.548 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.428197+0000", 2024-09-21T14:58:25.548 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:25.548 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.548 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.549 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:25.549 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.428205+0000", 2024-09-21T14:58:25.549 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1780000000000003e-06 2024-09-21T14:58:25.549 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.549 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.549 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:25.549 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.428228+0000", 2024-09-21T14:58:25.549 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3071999999999999e-05 2024-09-21T14:58:25.549 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.549 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.549 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:25.549 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.428241+0000", 2024-09-21T14:58:25.549 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2425999999999999e-05 2024-09-21T14:58:25.549 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.550 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.550 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:25.550 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.428314+0000", 2024-09-21T14:58:25.550 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3139999999999994e-05 2024-09-21T14:58:25.550 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.550 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.550 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:25.550 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.428327+0000", 2024-09-21T14:58:25.550 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3298e-05 2024-09-21T14:58:25.550 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.550 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.550 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:25.550 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.428527+0000", 2024-09-21T14:58:25.551 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019987999999999999 2024-09-21T14:58:25.551 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:25.551 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:25.551 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:25.551 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.551 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.551 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:16 3.6 3:6748301c:::rbd_header.11367ffd86fe:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-21T14:58:25.551 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:21.435367+0000", 2024-09-21T14:58:25.551 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.1038536209999998, 2024-09-21T14:58:25.551 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00053224900000000005, 2024-09-21T14:58:25.551 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:25.551 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:25.551 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:25.551 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:25.551 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:25.552 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 16 2024-09-21T14:58:25.552 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.552 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:25.552 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.552 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:25.552 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.435367+0000", 2024-09-21T14:58:25.552 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:25.552 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.552 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.552 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:25.552 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.435367+0000", 2024-09-21T14:58:25.552 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:25.552 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.552 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.552 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:25.553 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.435365+0000", 2024-09-21T14:58:25.553 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T14:58:25.553 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.553 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.553 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:25.553 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.435378+0000", 2024-09-21T14:58:25.553 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2218999999999999e-05 2024-09-21T14:58:25.553 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.553 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.553 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:25.553 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.435400+0000", 2024-09-21T14:58:25.553 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2416e-05 2024-09-21T14:58:25.553 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.553 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.553 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:25.554 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.435411+0000", 2024-09-21T14:58:25.554 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0900000000000001e-05 2024-09-21T14:58:25.554 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.554 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.554 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:25.554 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.435485+0000", 2024-09-21T14:58:25.554 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4530000000000006e-05 2024-09-21T14:58:25.554 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.554 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.554 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:25.554 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.435595+0000", 2024-09-21T14:58:25.554 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000109714 2024-09-21T14:58:25.554 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.554 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.555 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:25.555 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.435900+0000", 2024-09-21T14:58:25.555 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00030448200000000002 2024-09-21T14:58:25.555 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:25.555 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:25.555 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:25.555 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.555 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.555 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:17 3.6 3:6748301c:::rbd_header.11367ffd86fe:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-21T14:58:25.555 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:21.436080+0000", 2024-09-21T14:58:25.555 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.1031409280000002, 2024-09-21T14:58:25.555 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00035921800000000002, 2024-09-21T14:58:25.555 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:25.555 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:25.556 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:25.556 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:25.556 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:25.556 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 17 2024-09-21T14:58:25.556 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.556 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:25.556 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.556 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:25.556 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.436080+0000", 2024-09-21T14:58:25.556 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:25.556 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.556 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.556 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:25.556 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.436080+0000", 2024-09-21T14:58:25.556 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:25.557 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.557 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.557 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:25.557 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.436079+0000", 2024-09-21T14:58:25.557 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:25.557 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.557 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.557 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:25.557 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.436088+0000", 2024-09-21T14:58:25.557 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3179999999999996e-06 2024-09-21T14:58:25.557 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.557 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.557 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:25.557 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.436106+0000", 2024-09-21T14:58:25.557 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7612999999999999e-05 2024-09-21T14:58:25.558 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.558 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.558 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:25.558 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.436115+0000", 2024-09-21T14:58:25.558 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3160000000000003e-06 2024-09-21T14:58:25.558 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.558 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.558 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:25.558 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.436177+0000", 2024-09-21T14:58:25.558 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2533000000000006e-05 2024-09-21T14:58:25.558 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.558 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.558 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:25.558 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.436274+0000", 2024-09-21T14:58:25.559 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6372000000000005e-05 2024-09-21T14:58:25.559 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.559 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.559 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:25.559 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.436439+0000", 2024-09-21T14:58:25.559 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016550399999999999 2024-09-21T14:58:25.559 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:25.559 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:25.559 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:25.559 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.559 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.559 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:18 3.6 3:6748301c:::rbd_header.11367ffd86fe:head [call,call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-21T14:58:25.559 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:21.436749+0000", 2024-09-21T14:58:25.559 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.102472326, 2024-09-21T14:58:25.560 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00037552199999999999, 2024-09-21T14:58:25.560 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:25.560 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:25.560 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:25.560 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:25.560 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:25.560 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 18 2024-09-21T14:58:25.560 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.560 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:25.560 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.560 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:25.560 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.436749+0000", 2024-09-21T14:58:25.560 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:25.560 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.561 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.561 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:25.561 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.436749+0000", 2024-09-21T14:58:25.561 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:25.561 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.561 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.561 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:25.561 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.436747+0000", 2024-09-21T14:58:25.561 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:25.561 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.561 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.561 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:25.561 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.436757+0000", 2024-09-21T14:58:25.561 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9739999999999997e-06 2024-09-21T14:58:25.562 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.562 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.562 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:25.562 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.436777+0000", 2024-09-21T14:58:25.562 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0355000000000001e-05 2024-09-21T14:58:25.562 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.562 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.562 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:25.562 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.436792+0000", 2024-09-21T14:58:25.562 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4756000000000001e-05 2024-09-21T14:58:25.562 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.562 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.562 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:25.562 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.436854+0000", 2024-09-21T14:58:25.563 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2111000000000002e-05 2024-09-21T14:58:25.563 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.563 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.563 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:25.563 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.436953+0000", 2024-09-21T14:58:25.563 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.8881000000000004e-05 2024-09-21T14:58:25.563 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.563 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.563 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:25.563 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.437124+0000", 2024-09-21T14:58:25.563 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017094699999999999 2024-09-21T14:58:25.563 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:25.563 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:25.564 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:25.564 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.564 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.564 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:19 3.6 3:6748301c:::rbd_header.11367ffd86fe:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-21T14:58:25.564 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:21.437521+0000", 2024-09-21T14:58:25.564 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.1016996360000002, 2024-09-21T14:58:25.564 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00039898100000000002, 2024-09-21T14:58:25.564 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:25.564 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:25.564 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:25.564 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:25.564 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:25.564 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 19 2024-09-21T14:58:25.564 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.565 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:25.565 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.565 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:25.565 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.437521+0000", 2024-09-21T14:58:25.565 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:25.565 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.565 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.565 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:25.565 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.437521+0000", 2024-09-21T14:58:25.565 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:25.565 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.565 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.565 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:25.566 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.437520+0000", 2024-09-21T14:58:25.566 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:25.566 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.566 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.566 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:25.566 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.437528+0000", 2024-09-21T14:58:25.566 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5469999999999999e-06 2024-09-21T14:58:25.566 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.566 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.566 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:25.566 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.437546+0000", 2024-09-21T14:58:25.566 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8386e-05 2024-09-21T14:58:25.566 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.567 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.567 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:25.567 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.437555+0000", 2024-09-21T14:58:25.567 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5339999999999999e-06 2024-09-21T14:58:25.567 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.567 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.567 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:25.567 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.437627+0000", 2024-09-21T14:58:25.567 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2546000000000006e-05 2024-09-21T14:58:25.567 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.567 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.567 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:25.567 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.437722+0000", 2024-09-21T14:58:25.568 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4802000000000005e-05 2024-09-21T14:58:25.568 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.568 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.568 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:25.568 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.437920+0000", 2024-09-21T14:58:25.568 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019826199999999999 2024-09-21T14:58:25.568 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:25.568 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:25.568 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:25.568 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.568 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.568 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:20 3.6 3:6748301c:::rbd_header.11367ffd86fe:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-21T14:58:25.569 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:21.438875+0000", 2024-09-21T14:58:25.569 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.1003455170000001, 2024-09-21T14:58:25.569 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00067049799999999995, 2024-09-21T14:58:25.569 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:25.569 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:25.569 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:25.569 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:25.569 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:25.569 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 20 2024-09-21T14:58:25.569 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.569 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:25.569 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.569 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:25.569 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.438875+0000", 2024-09-21T14:58:25.570 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:25.570 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.570 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.570 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:25.570 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.438875+0000", 2024-09-21T14:58:25.570 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:25.570 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.570 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.570 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:25.570 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.438873+0000", 2024-09-21T14:58:25.570 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T14:58:25.570 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.570 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.570 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:25.571 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.438887+0000", 2024-09-21T14:58:25.571 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3884e-05 2024-09-21T14:58:25.571 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.571 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.571 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:25.571 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.438914+0000", 2024-09-21T14:58:25.571 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6384999999999999e-05 2024-09-21T14:58:25.571 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.571 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.571 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:25.571 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.438940+0000", 2024-09-21T14:58:25.571 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6137999999999999e-05 2024-09-21T14:58:25.571 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.572 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.572 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:25.572 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.438994+0000", 2024-09-21T14:58:25.572 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4172999999999998e-05 2024-09-21T14:58:25.572 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.572 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.572 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:25.572 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.439102+0000", 2024-09-21T14:58:25.572 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000108385 2024-09-21T14:58:25.572 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.572 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.572 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:25.572 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.439546+0000", 2024-09-21T14:58:25.572 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00044351699999999999 2024-09-21T14:58:25.573 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:25.573 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:25.573 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:25.573 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.573 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.573 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:21 3.6 3:6748301c:::rbd_header.11367ffd86fe:head [call,call rbd.parent_overlap_get in=21b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-21T14:58:25.573 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:21.439743+0000", 2024-09-21T14:58:25.573 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.0994775390000004, 2024-09-21T14:58:25.573 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00054151700000000004, 2024-09-21T14:58:25.573 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:25.573 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:25.573 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:25.573 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:25.573 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:25.574 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 21 2024-09-21T14:58:25.574 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.574 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:25.574 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.574 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:25.574 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.439743+0000", 2024-09-21T14:58:25.574 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:25.574 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.574 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.574 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:25.574 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.439743+0000", 2024-09-21T14:58:25.574 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:25.574 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.574 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.575 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:25.575 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.439742+0000", 2024-09-21T14:58:25.575 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:25.575 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.575 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.575 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:25.575 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.439752+0000", 2024-09-21T14:58:25.575 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4339999999999998e-06 2024-09-21T14:58:25.575 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.575 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.575 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:25.575 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.439772+0000", 2024-09-21T14:58:25.575 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0786999999999999e-05 2024-09-21T14:58:25.575 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.576 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.576 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:25.576 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.439786+0000", 2024-09-21T14:58:25.576 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3216e-05 2024-09-21T14:58:25.576 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.576 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.576 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:25.576 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.439851+0000", 2024-09-21T14:58:25.576 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5763e-05 2024-09-21T14:58:25.576 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.576 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.576 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:25.576 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.439950+0000", 2024-09-21T14:58:25.577 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.8265999999999997e-05 2024-09-21T14:58:25.577 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.577 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.577 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:25.577 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.440285+0000", 2024-09-21T14:58:25.577 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00033541300000000001 2024-09-21T14:58:25.577 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:25.577 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:25.577 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:25.577 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.577 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.577 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22 3.6 3:6748301c:::rbd_header.11367ffd86fe:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-21T14:58:25.577 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:21.440440+0000", 2024-09-21T14:58:25.577 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.0987806689999999, 2024-09-21T14:58:25.577 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00033491400000000001, 2024-09-21T14:58:25.578 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:25.578 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:25.578 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:25.578 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:25.578 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:25.578 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22 2024-09-21T14:58:25.578 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.578 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:25.578 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.578 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:25.578 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.440440+0000", 2024-09-21T14:58:25.578 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:25.578 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.578 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.578 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:25.579 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.440440+0000", 2024-09-21T14:58:25.579 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:25.579 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.579 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.579 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:25.579 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.440439+0000", 2024-09-21T14:58:25.579 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:25.579 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.579 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.579 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:25.579 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.440447+0000", 2024-09-21T14:58:25.579 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2989999999999997e-06 2024-09-21T14:58:25.579 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.579 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.580 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:25.580 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.440466+0000", 2024-09-21T14:58:25.580 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.859e-05 2024-09-21T14:58:25.580 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.580 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.580 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:25.580 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.440474+0000", 2024-09-21T14:58:25.580 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1459999999999996e-06 2024-09-21T14:58:25.580 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.580 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.580 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:25.580 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.440525+0000", 2024-09-21T14:58:25.580 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0998999999999997e-05 2024-09-21T14:58:25.580 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.580 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.580 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:25.581 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.440615+0000", 2024-09-21T14:58:25.581 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0045999999999998e-05 2024-09-21T14:58:25.581 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.581 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.581 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:25.581 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.440775+0000", 2024-09-21T14:58:25.581 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016007799999999999 2024-09-21T14:58:25.581 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:25.581 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:25.581 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:25.581 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.581 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.581 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:24 3.6 3:6748301c:::rbd_header.11367ffd86fe:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-21T14:58:25.581 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:21.441553+0000", 2024-09-21T14:58:25.582 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.0976683410000003, 2024-09-21T14:58:25.582 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000330131, 2024-09-21T14:58:25.582 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:25.582 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:25.582 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:25.582 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:25.582 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:25.582 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 24 2024-09-21T14:58:25.582 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.582 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:25.582 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.582 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:25.582 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.441553+0000", 2024-09-21T14:58:25.582 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:25.583 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.583 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.583 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:25.583 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.441553+0000", 2024-09-21T14:58:25.583 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:25.583 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.583 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.583 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:25.583 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.441551+0000", 2024-09-21T14:58:25.583 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:25.583 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.583 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.583 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:25.583 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.441561+0000", 2024-09-21T14:58:25.584 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6160000000000008e-06 2024-09-21T14:58:25.584 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.584 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.584 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:25.584 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.441583+0000", 2024-09-21T14:58:25.584 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2064000000000001e-05 2024-09-21T14:58:25.584 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.584 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.584 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:25.584 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.441596+0000", 2024-09-21T14:58:25.584 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2948999999999999e-05 2024-09-21T14:58:25.584 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.584 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.584 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:25.585 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.441648+0000", 2024-09-21T14:58:25.585 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1826999999999997e-05 2024-09-21T14:58:25.585 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.585 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.585 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:25.585 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.441736+0000", 2024-09-21T14:58:25.585 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8463999999999999e-05 2024-09-21T14:58:25.585 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.585 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.585 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:25.585 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.441883+0000", 2024-09-21T14:58:25.585 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000146599 2024-09-21T14:58:25.585 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:25.585 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:25.585 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:25.586 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.586 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.586 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:25 3.6 3:6748301c:::rbd_header.11367ffd86fe:head [watch watch cookie 139864871832192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T14:58:25.586 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:21.442192+0000", 2024-09-21T14:58:25.586 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.0970287599999997, 2024-09-21T14:58:25.586 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00187035, 2024-09-21T14:58:25.586 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:25.586 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:25.586 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:25.586 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:25.586 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:25.586 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 25 2024-09-21T14:58:25.586 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.586 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:25.587 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.587 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:25.587 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.442192+0000", 2024-09-21T14:58:25.587 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:25.587 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.587 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.587 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:25.587 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.442192+0000", 2024-09-21T14:58:25.587 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:25.587 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.587 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.587 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:25.587 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.442191+0000", 2024-09-21T14:58:25.587 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:25.588 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.588 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.588 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:25.588 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.442200+0000", 2024-09-21T14:58:25.588 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1020000000000003e-06 2024-09-21T14:58:25.588 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.588 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.588 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:25.588 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.442219+0000", 2024-09-21T14:58:25.588 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9459000000000001e-05 2024-09-21T14:58:25.588 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.588 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.588 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:25.588 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.442229+0000", 2024-09-21T14:58:25.589 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0599999999999997e-06 2024-09-21T14:58:25.589 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.589 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.589 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:25.589 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.442306+0000", 2024-09-21T14:58:25.589 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7194000000000007e-05 2024-09-21T14:58:25.589 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.589 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.589 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:25.589 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.442408+0000", 2024-09-21T14:58:25.589 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010183000000000001 2024-09-21T14:58:25.589 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.589 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.589 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 7", 2024-09-21T14:58:25.589 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.442570+0000", 2024-09-21T14:58:25.590 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000161939 2024-09-21T14:58:25.590 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.590 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.590 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:58:25.590 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.443352+0000", 2024-09-21T14:58:25.590 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00078295700000000003 2024-09-21T14:58:25.590 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.590 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.590 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:25.590 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.443927+0000", 2024-09-21T14:58:25.590 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00057451400000000004 2024-09-21T14:58:25.590 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.590 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.590 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:25.591 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.443981+0000", 2024-09-21T14:58:25.591 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3678999999999999e-05 2024-09-21T14:58:25.591 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.591 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.591 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:25.591 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.444063+0000", 2024-09-21T14:58:25.591 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1908000000000003e-05 2024-09-21T14:58:25.591 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:25.591 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:25.591 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:25.591 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.591 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.591 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop_reply(client.4406.0:25 3.6 e27/24 ondisk, result = 0)", 2024-09-21T14:58:25.591 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:21.443806+0000", 2024-09-21T14:58:25.592 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.0954150739999999, 2024-09-21T14:58:25.592 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00026796399999999998, 2024-09-21T14:58:25.592 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:25.592 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:25.592 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:25.592 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.592 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:25.592 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.443806+0000", 2024-09-21T14:58:25.592 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:25.592 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.592 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.592 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:25.592 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.443806+0000", 2024-09-21T14:58:25.592 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:25.593 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.593 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.593 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:25.593 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.443805+0000", 2024-09-21T14:58:25.593 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:25.593 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.593 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.593 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:25.593 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.443813+0000", 2024-09-21T14:58:25.593 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9199999999999993e-06 2024-09-21T14:58:25.593 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.593 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.593 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:25.593 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.443832+0000", 2024-09-21T14:58:25.593 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8632999999999999e-05 2024-09-21T14:58:25.594 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.594 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.594 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:25.594 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.443841+0000", 2024-09-21T14:58:25.594 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7059999999999993e-06 2024-09-21T14:58:25.594 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.594 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.594 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:25.594 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.443909+0000", 2024-09-21T14:58:25.594 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8347000000000001e-05 2024-09-21T14:58:25.594 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.594 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.594 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:25.594 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.443921+0000", 2024-09-21T14:58:25.595 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2351000000000001e-05 2024-09-21T14:58:25.595 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:25.595 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:25.595 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:25.595 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.444074+0000", 2024-09-21T14:58:25.595 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015237900000000001 2024-09-21T14:58:25.595 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:25.595 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:25.595 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:25.595 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:25.595 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:25.595 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:58:25.596 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T14:58:25.597 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:58:25.593+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:58:25.697 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:58:25.692+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:58:25.754 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:58:25.754 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:58:25.754 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T14:58:25.755 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:58:25.764 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T14:58:25.798 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:58:25.797+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:58:25.832 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-21T14:58:25.835 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-09-21T14:58:25.848 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":37,"stamp":"2024-09-21T14:58:25.780604+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590395,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":162,"num_write_kb":2419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":121,"ondisk_log_size":121,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":34,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":251176,"kb_used_data":2220,"kb_used_omap":12,"kb_used_meta":248819,"kb_avail":754723544,"statfs":{"total":773094113280,"available":772836909056,"internally_reserved":0,"allocated":2273280,"data_stored":1959298,"data_compressed":10404,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":12744,"internal_metadata":254791224},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001219"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T14:58:20.414668+0000","last_change":"2024-09-21T14:58:19.410326+0000","last_active":"2024-09-21T14:58:20.414668+0000","last_peered":"2024-09-21T14:58:20.414668+0000","last_clean":"2024-09-21T14:58:20.414668+0000","last_became_active":"2024-09-21T14:58:19.410055+0000","last_became_peered":"2024-09-21T14:58:19.410055+0000","last_unstale":"2024-09-21T14:58:20.414668+0000","last_undegraded":"2024-09-21T14:58:20.414668+0000","last_fullsized":"2024-09-21T14:58:20.414668+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:49:13.944219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T14:58:20.414506+0000","last_change":"2024-09-21T14:58:19.412656+0000","last_active":"2024-09-21T14:58:20.414506+0000","last_peered":"2024-09-21T14:58:20.414506+0000","last_clean":"2024-09-21T14:58:20.414506+0000","last_became_active":"2024-09-21T14:58:19.412353+0000","last_became_peered":"2024-09-21T14:58:19.412353+0000","last_unstale":"2024-09-21T14:58:20.414506+0000","last_undegraded":"2024-09-21T14:58:20.414506+0000","last_fullsized":"2024-09-21T14:58:20.414506+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:38:15.532795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-21T14:58:19.412848+0000","last_change":"2024-09-21T14:58:19.411705+0000","last_active":"2024-09-21T14:58:19.412848+0000","last_peered":"2024-09-21T14:58:19.412848+0000","last_clean":"2024-09-21T14:58:19.412848+0000","last_became_active":"2024-09-21T14:58:19.411387+0000","last_became_peered":"2024-09-21T14:58:19.411387+0000","last_unstale":"2024-09-21T14:58:19.412848+0000","last_undegraded":"2024-09-21T14:58:19.412848+0000","last_fullsized":"2024-09-21T14:58:19.412848+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:37:57.597069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T14:58:20.408502+0000","last_change":"2024-09-21T14:58:19.411642+0000","last_active":"2024-09-21T14:58:20.408502+0000","last_peered":"2024-09-21T14:58:20.408502+0000","last_clean":"2024-09-21T14:58:20.408502+0000","last_became_active":"2024-09-21T14:58:19.411306+0000","last_became_peered":"2024-09-21T14:58:19.411306+0000","last_unstale":"2024-09-21T14:58:20.408502+0000","last_undegraded":"2024-09-21T14:58:20.408502+0000","last_fullsized":"2024-09-21T14:58:20.408502+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:20:27.079795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T14:58:20.493010+0000","last_change":"2024-09-21T14:58:19.413315+0000","last_active":"2024-09-21T14:58:20.493010+0000","last_peered":"2024-09-21T14:58:20.493010+0000","last_clean":"2024-09-21T14:58:20.493010+0000","last_became_active":"2024-09-21T14:58:19.413053+0000","last_became_peered":"2024-09-21T14:58:19.413053+0000","last_unstale":"2024-09-21T14:58:20.493010+0000","last_undegraded":"2024-09-21T14:58:20.493010+0000","last_fullsized":"2024-09-21T14:58:20.493010+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:07:12.901354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"26'1","reported_seq":11,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T14:58:20.411013+0000","last_change":"2024-09-21T14:58:19.409889+0000","last_active":"2024-09-21T14:58:20.411013+0000","last_peered":"2024-09-21T14:58:20.411013+0000","last_clean":"2024-09-21T14:58:20.411013+0000","last_became_active":"2024-09-21T14:58:19.409415+0000","last_became_peered":"2024-09-21T14:58:19.409415+0000","last_unstale":"2024-09-21T14:58:20.411013+0000","last_undegraded":"2024-09-21T14:58:20.411013+0000","last_fullsized":"2024-09-21T14:58:20.411013+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:28:46.336301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T14:58:20.493005+0000","last_change":"2024-09-21T14:58:19.410435+0000","last_active":"2024-09-21T14:58:20.493005+0000","last_peered":"2024-09-21T14:58:20.493005+0000","last_clean":"2024-09-21T14:58:20.493005+0000","last_became_active":"2024-09-21T14:58:19.410062+0000","last_became_peered":"2024-09-21T14:58:19.410062+0000","last_unstale":"2024-09-21T14:58:20.493005+0000","last_undegraded":"2024-09-21T14:58:20.493005+0000","last_fullsized":"2024-09-21T14:58:20.493005+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:12:02.923321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T14:58:20.414573+0000","last_change":"2024-09-21T14:58:19.409695+0000","last_active":"2024-09-21T14:58:20.414573+0000","last_peered":"2024-09-21T14:58:20.414573+0000","last_clean":"2024-09-21T14:58:20.414573+0000","last_became_active":"2024-09-21T14:58:19.409371+0000","last_became_peered":"2024-09-21T14:58:19.409371+0000","last_unstale":"2024-09-21T14:58:20.414573+0000","last_undegraded":"2024-09-21T14:58:20.414573+0000","last_fullsized":"2024-09-21T14:58:20.414573+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:35:27.143445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T14:58:20.414681+0000","last_change":"2024-09-21T14:58:19.410571+0000","last_active":"2024-09-21T14:58:20.414681+0000","last_peered":"2024-09-21T14:58:20.414681+0000","last_clean":"2024-09-21T14:58:20.414681+0000","last_became_active":"2024-09-21T14:58:19.410306+0000","last_became_peered":"2024-09-21T14:58:19.410306+0000","last_unstale":"2024-09-21T14:58:20.414681+0000","last_undegraded":"2024-09-21T14:58:20.414681+0000","last_fullsized":"2024-09-21T14:58:20.414681+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:33:50.089165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-21T14:58:19.416075+0000","last_change":"2024-09-21T14:58:19.414327+0000","last_active":"2024-09-21T14:58:19.416075+0000","last_peered":"2024-09-21T14:58:19.416075+0000","last_clean":"2024-09-21T14:58:19.416075+0000","last_became_active":"2024-09-21T14:58:19.413935+0000","last_became_peered":"2024-09-21T14:58:19.413935+0000","last_unstale":"2024-09-21T14:58:19.416075+0000","last_undegraded":"2024-09-21T14:58:19.416075+0000","last_fullsized":"2024-09-21T14:58:19.416075+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:15:28.307418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T14:58:20.403904+0000","last_change":"2024-09-21T14:58:19.410964+0000","last_active":"2024-09-21T14:58:20.403904+0000","last_peered":"2024-09-21T14:58:20.403904+0000","last_clean":"2024-09-21T14:58:20.403904+0000","last_became_active":"2024-09-21T14:58:19.410687+0000","last_became_peered":"2024-09-21T14:58:19.410687+0000","last_unstale":"2024-09-21T14:58:20.403904+0000","last_undegraded":"2024-09-21T14:58:20.403904+0000","last_fullsized":"2024-09-21T14:58:20.403904+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:21:42.226982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-21T14:58:19.410932+0000","last_change":"2024-09-21T14:58:19.407635+0000","last_active":"2024-09-21T14:58:19.410932+0000","last_peered":"2024-09-21T14:58:19.410932+0000","last_clean":"2024-09-21T14:58:19.410932+0000","last_became_active":"2024-09-21T14:58:19.407310+0000","last_became_peered":"2024-09-21T14:58:19.407310+0000","last_unstale":"2024-09-21T14:58:19.410932+0000","last_undegraded":"2024-09-21T14:58:19.410932+0000","last_fullsized":"2024-09-21T14:58:19.410932+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:47:46.079115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-21T14:58:18.398550+0000","last_change":"2024-09-21T14:58:11.348177+0000","last_active":"2024-09-21T14:58:18.398550+0000","last_peered":"2024-09-21T14:58:18.398550+0000","last_clean":"2024-09-21T14:58:18.398550+0000","last_became_active":"2024-09-21T14:58:09.328417+0000","last_became_peered":"2024-09-21T14:58:09.328417+0000","last_unstale":"2024-09-21T14:58:18.398550+0000","last_undegraded":"2024-09-21T14:58:18.398550+0000","last_fullsized":"2024-09-21T14:58:18.398550+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:52:22.671185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018917490000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T14:58:20.493247+0000","last_change":"2024-09-21T14:58:19.404157+0000","last_active":"2024-09-21T14:58:20.493247+0000","last_peered":"2024-09-21T14:58:20.493247+0000","last_clean":"2024-09-21T14:58:20.493247+0000","last_became_active":"2024-09-21T14:58:19.403868+0000","last_became_peered":"2024-09-21T14:58:19.403868+0000","last_unstale":"2024-09-21T14:58:20.493247+0000","last_undegraded":"2024-09-21T14:58:20.493247+0000","last_fullsized":"2024-09-21T14:58:20.493247+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:52:08.672416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-21T14:58:19.397919+0000","last_change":"2024-09-21T14:58:11.405717+0000","last_active":"2024-09-21T14:58:19.397919+0000","last_peered":"2024-09-21T14:58:19.397919+0000","last_clean":"2024-09-21T14:58:19.397919+0000","last_became_active":"2024-09-21T14:58:09.326849+0000","last_became_peered":"2024-09-21T14:58:09.326849+0000","last_unstale":"2024-09-21T14:58:19.397919+0000","last_undegraded":"2024-09-21T14:58:19.397919+0000","last_fullsized":"2024-09-21T14:58:19.397919+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:29:42.893819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012560399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T14:58:20.493076+0000","last_change":"2024-09-21T14:58:19.411316+0000","last_active":"2024-09-21T14:58:20.493076+0000","last_peered":"2024-09-21T14:58:20.493076+0000","last_clean":"2024-09-21T14:58:20.493076+0000","last_became_active":"2024-09-21T14:58:19.410954+0000","last_became_peered":"2024-09-21T14:58:19.410954+0000","last_unstale":"2024-09-21T14:58:20.493076+0000","last_undegraded":"2024-09-21T14:58:20.493076+0000","last_fullsized":"2024-09-21T14:58:20.493076+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:26:59.084415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-21T14:58:18.398580+0000","last_change":"2024-09-21T14:58:11.348182+0000","last_active":"2024-09-21T14:58:18.398580+0000","last_peered":"2024-09-21T14:58:18.398580+0000","last_clean":"2024-09-21T14:58:18.398580+0000","last_became_active":"2024-09-21T14:58:09.329036+0000","last_became_peered":"2024-09-21T14:58:09.329036+0000","last_unstale":"2024-09-21T14:58:18.398580+0000","last_undegraded":"2024-09-21T14:58:18.398580+0000","last_fullsized":"2024-09-21T14:58:18.398580+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:24:45.719970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019338770000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T14:58:20.404878+0000","last_change":"2024-09-21T14:58:19.409494+0000","last_active":"2024-09-21T14:58:20.404878+0000","last_peered":"2024-09-21T14:58:20.404878+0000","last_clean":"2024-09-21T14:58:20.404878+0000","last_became_active":"2024-09-21T14:58:19.409130+0000","last_became_peered":"2024-09-21T14:58:19.409130+0000","last_unstale":"2024-09-21T14:58:20.404878+0000","last_undegraded":"2024-09-21T14:58:20.404878+0000","last_fullsized":"2024-09-21T14:58:20.404878+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:47:27.991651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-21T14:58:19.399284+0000","last_change":"2024-09-21T14:58:11.406176+0000","last_active":"2024-09-21T14:58:19.399284+0000","last_peered":"2024-09-21T14:58:19.399284+0000","last_clean":"2024-09-21T14:58:19.399284+0000","last_became_active":"2024-09-21T14:58:09.777465+0000","last_became_peered":"2024-09-21T14:58:09.777465+0000","last_unstale":"2024-09-21T14:58:19.399284+0000","last_undegraded":"2024-09-21T14:58:19.399284+0000","last_fullsized":"2024-09-21T14:58:19.399284+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:55:04.667816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095704000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-21T14:58:19.415848+0000","last_change":"2024-09-21T14:58:19.409395+0000","last_active":"2024-09-21T14:58:19.415848+0000","last_peered":"2024-09-21T14:58:19.415848+0000","last_clean":"2024-09-21T14:58:19.415848+0000","last_became_active":"2024-09-21T14:58:19.408971+0000","last_became_peered":"2024-09-21T14:58:19.408971+0000","last_unstale":"2024-09-21T14:58:19.415848+0000","last_undegraded":"2024-09-21T14:58:19.415848+0000","last_fullsized":"2024-09-21T14:58:19.415848+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:29:52.642121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T14:58:20.403664+0000","last_change":"2024-09-21T14:58:19.418130+0000","last_active":"2024-09-21T14:58:20.403664+0000","last_peered":"2024-09-21T14:58:20.403664+0000","last_clean":"2024-09-21T14:58:20.403664+0000","last_became_active":"2024-09-21T14:58:19.417453+0000","last_became_peered":"2024-09-21T14:58:19.417453+0000","last_unstale":"2024-09-21T14:58:20.403664+0000","last_undegraded":"2024-09-21T14:58:20.403664+0000","last_fullsized":"2024-09-21T14:58:20.403664+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:19:27.087272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-21T14:58:18.398306+0000","last_change":"2024-09-21T14:58:11.347504+0000","last_active":"2024-09-21T14:58:18.398306+0000","last_peered":"2024-09-21T14:58:18.398306+0000","last_clean":"2024-09-21T14:58:18.398306+0000","last_became_active":"2024-09-21T14:58:09.775981+0000","last_became_peered":"2024-09-21T14:58:09.775981+0000","last_unstale":"2024-09-21T14:58:18.398306+0000","last_undegraded":"2024-09-21T14:58:18.398306+0000","last_fullsized":"2024-09-21T14:58:18.398306+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:47:16.286652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012568880000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T14:58:20.414303+0000","last_change":"2024-09-21T14:58:19.406072+0000","last_active":"2024-09-21T14:58:20.414303+0000","last_peered":"2024-09-21T14:58:20.414303+0000","last_clean":"2024-09-21T14:58:20.414303+0000","last_became_active":"2024-09-21T14:58:19.405811+0000","last_became_peered":"2024-09-21T14:58:19.405811+0000","last_unstale":"2024-09-21T14:58:20.414303+0000","last_undegraded":"2024-09-21T14:58:20.414303+0000","last_fullsized":"2024-09-21T14:58:20.414303+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:36:25.551607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T14:58:20.405084+0000","last_change":"2024-09-21T14:58:11.342210+0000","last_active":"2024-09-21T14:58:20.405084+0000","last_peered":"2024-09-21T14:58:20.405084+0000","last_clean":"2024-09-21T14:58:20.405084+0000","last_became_active":"2024-09-21T14:58:09.330624+0000","last_became_peered":"2024-09-21T14:58:09.330624+0000","last_unstale":"2024-09-21T14:58:20.405084+0000","last_undegraded":"2024-09-21T14:58:20.405084+0000","last_fullsized":"2024-09-21T14:58:20.405084+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:29:43.229538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001051151,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T14:58:20.404718+0000","last_change":"2024-09-21T14:58:19.417824+0000","last_active":"2024-09-21T14:58:20.404718+0000","last_peered":"2024-09-21T14:58:20.404718+0000","last_clean":"2024-09-21T14:58:20.404718+0000","last_became_active":"2024-09-21T14:58:19.417529+0000","last_became_peered":"2024-09-21T14:58:19.417529+0000","last_unstale":"2024-09-21T14:58:20.404718+0000","last_undegraded":"2024-09-21T14:58:20.404718+0000","last_fullsized":"2024-09-21T14:58:20.404718+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:38:57.961654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-21T14:58:18.398245+0000","last_change":"2024-09-21T14:58:11.346766+0000","last_active":"2024-09-21T14:58:18.398245+0000","last_peered":"2024-09-21T14:58:18.398245+0000","last_clean":"2024-09-21T14:58:18.398245+0000","last_became_active":"2024-09-21T14:58:09.329748+0000","last_became_peered":"2024-09-21T14:58:09.329748+0000","last_unstale":"2024-09-21T14:58:18.398245+0000","last_undegraded":"2024-09-21T14:58:18.398245+0000","last_fullsized":"2024-09-21T14:58:18.398245+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:31:20.267675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052698000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T14:58:20.493199+0000","last_change":"2024-09-21T14:58:19.416737+0000","last_active":"2024-09-21T14:58:20.493199+0000","last_peered":"2024-09-21T14:58:20.493199+0000","last_clean":"2024-09-21T14:58:20.493199+0000","last_became_active":"2024-09-21T14:58:19.416313+0000","last_became_peered":"2024-09-21T14:58:19.416313+0000","last_unstale":"2024-09-21T14:58:20.493199+0000","last_undegraded":"2024-09-21T14:58:20.493199+0000","last_fullsized":"2024-09-21T14:58:20.493199+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:34:38.858491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-21T14:58:18.398390+0000","last_change":"2024-09-21T14:58:11.347700+0000","last_active":"2024-09-21T14:58:18.398390+0000","last_peered":"2024-09-21T14:58:18.398390+0000","last_clean":"2024-09-21T14:58:18.398390+0000","last_became_active":"2024-09-21T14:58:09.438842+0000","last_became_peered":"2024-09-21T14:58:09.438842+0000","last_unstale":"2024-09-21T14:58:18.398390+0000","last_undegraded":"2024-09-21T14:58:18.398390+0000","last_fullsized":"2024-09-21T14:58:18.398390+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:41:34.626249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001426194,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":168,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-21T14:58:19.402645+0000","last_change":"2024-09-21T14:58:03.616055+0000","last_active":"2024-09-21T14:58:19.402645+0000","last_peered":"2024-09-21T14:58:19.402645+0000","last_clean":"2024-09-21T14:58:19.402645+0000","last_became_active":"2024-09-21T14:58:03.613214+0000","last_became_peered":"2024-09-21T14:58:03.613214+0000","last_unstale":"2024-09-21T14:58:19.402645+0000","last_undegraded":"2024-09-21T14:58:19.402645+0000","last_fullsized":"2024-09-21T14:58:19.402645+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:02.180162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:02.180162+0000","last_clean_scrub_stamp":"2024-09-21T14:58:02.180162+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:29:05.860361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T14:58:20.492821+0000","last_change":"2024-09-21T14:58:19.404346+0000","last_active":"2024-09-21T14:58:20.492821+0000","last_peered":"2024-09-21T14:58:20.492821+0000","last_clean":"2024-09-21T14:58:20.492821+0000","last_became_active":"2024-09-21T14:58:19.403887+0000","last_became_peered":"2024-09-21T14:58:19.403887+0000","last_unstale":"2024-09-21T14:58:20.492821+0000","last_undegraded":"2024-09-21T14:58:20.492821+0000","last_fullsized":"2024-09-21T14:58:20.492821+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:25:35.185774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-21T14:58:19.407546+0000","last_change":"2024-09-21T14:58:19.400189+0000","last_active":"2024-09-21T14:58:19.407546+0000","last_peered":"2024-09-21T14:58:19.407546+0000","last_clean":"2024-09-21T14:58:19.407546+0000","last_became_active":"2024-09-21T14:58:19.399712+0000","last_became_peered":"2024-09-21T14:58:19.399712+0000","last_unstale":"2024-09-21T14:58:19.407546+0000","last_undegraded":"2024-09-21T14:58:19.407546+0000","last_fullsized":"2024-09-21T14:58:19.407546+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:50:47.558139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-21T14:58:19.411881+0000","last_change":"2024-09-21T14:58:19.408125+0000","last_active":"2024-09-21T14:58:19.411881+0000","last_peered":"2024-09-21T14:58:19.411881+0000","last_clean":"2024-09-21T14:58:19.411881+0000","last_became_active":"2024-09-21T14:58:19.407847+0000","last_became_peered":"2024-09-21T14:58:19.407847+0000","last_unstale":"2024-09-21T14:58:19.411881+0000","last_undegraded":"2024-09-21T14:58:19.411881+0000","last_fullsized":"2024-09-21T14:58:19.411881+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:45:44.979236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T14:58:21.429355+0000","last_change":"2024-09-21T14:58:21.429355+0000","last_active":"2024-09-21T14:58:21.429355+0000","last_peered":"2024-09-21T14:58:21.429355+0000","last_clean":"2024-09-21T14:58:21.429355+0000","last_became_active":"2024-09-21T14:58:19.410679+0000","last_became_peered":"2024-09-21T14:58:19.410679+0000","last_unstale":"2024-09-21T14:58:21.429355+0000","last_undegraded":"2024-09-21T14:58:21.429355+0000","last_fullsized":"2024-09-21T14:58:21.429355+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:07:40.758932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037710199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T14:58:20.403569+0000","last_change":"2024-09-21T14:58:19.411282+0000","last_active":"2024-09-21T14:58:20.403569+0000","last_peered":"2024-09-21T14:58:20.403569+0000","last_clean":"2024-09-21T14:58:20.403569+0000","last_became_active":"2024-09-21T14:58:19.411026+0000","last_became_peered":"2024-09-21T14:58:19.411026+0000","last_unstale":"2024-09-21T14:58:20.403569+0000","last_undegraded":"2024-09-21T14:58:20.403569+0000","last_fullsized":"2024-09-21T14:58:20.403569+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:38:03.448592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-21T14:58:19.412036+0000","last_change":"2024-09-21T14:58:19.408202+0000","last_active":"2024-09-21T14:58:19.412036+0000","last_peered":"2024-09-21T14:58:19.412036+0000","last_clean":"2024-09-21T14:58:19.412036+0000","last_became_active":"2024-09-21T14:58:19.407884+0000","last_became_peered":"2024-09-21T14:58:19.407884+0000","last_unstale":"2024-09-21T14:58:19.412036+0000","last_undegraded":"2024-09-21T14:58:19.412036+0000","last_fullsized":"2024-09-21T14:58:19.412036+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:28:57.111615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-21T14:58:19.416219+0000","last_change":"2024-09-21T14:58:19.414690+0000","last_active":"2024-09-21T14:58:19.416219+0000","last_peered":"2024-09-21T14:58:19.416219+0000","last_clean":"2024-09-21T14:58:19.416219+0000","last_became_active":"2024-09-21T14:58:19.414418+0000","last_became_peered":"2024-09-21T14:58:19.414418+0000","last_unstale":"2024-09-21T14:58:19.416219+0000","last_undegraded":"2024-09-21T14:58:19.416219+0000","last_fullsized":"2024-09-21T14:58:19.416219+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:36:41.583214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T14:58:20.404009+0000","last_change":"2024-09-21T14:58:19.410709+0000","last_active":"2024-09-21T14:58:20.404009+0000","last_peered":"2024-09-21T14:58:20.404009+0000","last_clean":"2024-09-21T14:58:20.404009+0000","last_became_active":"2024-09-21T14:58:19.410406+0000","last_became_peered":"2024-09-21T14:58:19.410406+0000","last_unstale":"2024-09-21T14:58:20.404009+0000","last_undegraded":"2024-09-21T14:58:20.404009+0000","last_fullsized":"2024-09-21T14:58:20.404009+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:59:16.026939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T14:58:20.408692+0000","last_change":"2024-09-21T14:58:19.410187+0000","last_active":"2024-09-21T14:58:20.408692+0000","last_peered":"2024-09-21T14:58:20.408692+0000","last_clean":"2024-09-21T14:58:20.408692+0000","last_became_active":"2024-09-21T14:58:19.409935+0000","last_became_peered":"2024-09-21T14:58:19.409935+0000","last_unstale":"2024-09-21T14:58:20.408692+0000","last_undegraded":"2024-09-21T14:58:20.408692+0000","last_fullsized":"2024-09-21T14:58:20.408692+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:10:36.252629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-21T14:58:19.409847+0000","last_change":"2024-09-21T14:58:19.406446+0000","last_active":"2024-09-21T14:58:19.409847+0000","last_peered":"2024-09-21T14:58:19.409847+0000","last_clean":"2024-09-21T14:58:19.409847+0000","last_became_active":"2024-09-21T14:58:19.406164+0000","last_became_peered":"2024-09-21T14:58:19.406164+0000","last_unstale":"2024-09-21T14:58:19.409847+0000","last_undegraded":"2024-09-21T14:58:19.409847+0000","last_fullsized":"2024-09-21T14:58:19.409847+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:26:32.857391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T14:58:20.404717+0000","last_change":"2024-09-21T14:58:19.412340+0000","last_active":"2024-09-21T14:58:20.404717+0000","last_peered":"2024-09-21T14:58:20.404717+0000","last_clean":"2024-09-21T14:58:20.404717+0000","last_became_active":"2024-09-21T14:58:19.412138+0000","last_became_peered":"2024-09-21T14:58:19.412138+0000","last_unstale":"2024-09-21T14:58:20.404717+0000","last_undegraded":"2024-09-21T14:58:20.404717+0000","last_fullsized":"2024-09-21T14:58:20.404717+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:34:16.989941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-21T14:58:19.413068+0000","last_change":"2024-09-21T14:58:19.412295+0000","last_active":"2024-09-21T14:58:19.413068+0000","last_peered":"2024-09-21T14:58:19.413068+0000","last_clean":"2024-09-21T14:58:19.413068+0000","last_became_active":"2024-09-21T14:58:19.412030+0000","last_became_peered":"2024-09-21T14:58:19.412030+0000","last_unstale":"2024-09-21T14:58:19.413068+0000","last_undegraded":"2024-09-21T14:58:19.413068+0000","last_fullsized":"2024-09-21T14:58:19.413068+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:44:29.282391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":8,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":16,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":10404,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509448,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":274432,"data_stored":100374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84299999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70899999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0509999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0780000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542151,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":278528,"data_stored":100393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1279999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69299999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1779999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2749999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542152,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31452,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340388,"statfs":{"total":96636764160,"available":96604557312,"internally_reserved":0,"allocated":335872,"data_stored":690750,"data_compressed":5202,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81100000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0489999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84899999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2230000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2010000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3620000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542151,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31376,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340464,"statfs":{"total":96636764160,"available":96604635136,"internally_reserved":0,"allocated":262144,"data_stored":94255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0429999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.214}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.143}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2609999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3160000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542152,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31380,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340460,"statfs":{"total":96636764160,"available":96604631040,"internally_reserved":0,"allocated":266240,"data_stored":94274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82399999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.753}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99399999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3220000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.22}]}]},{"osd":2,"up_from":14,"seq":60129542150,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31372,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340468,"statfs":{"total":96636764160,"available":96604639232,"internally_reserved":0,"allocated":262144,"data_stored":94255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79700000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84099999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78300000000000003}]}]},{"osd":1,"up_from":14,"seq":60129542151,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31432,"kb_used_data":312,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340408,"statfs":{"total":96636764160,"available":96604577792,"internally_reserved":0,"allocated":319488,"data_stored":684623,"data_compressed":5202,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31848901},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91700000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1080000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84899999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.075}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79100000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88700000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542150,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31384,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340456,"statfs":{"total":96636764160,"available":96604626944,"internally_reserved":0,"allocated":274432,"data_stored":100374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1160000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.141}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93799999999999994}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5202,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5202,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":8,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":8,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T14:58:25.850 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T14:58:25.874 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:58:25.874 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:58:25.874 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T14:58:25.874 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T14:58:25.874 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:58:25.884 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T14:58:25.899 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:58:25.896+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:58:26.001 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:58:26.001+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:58:26.040 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:58:26.040 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T14:58:26.040 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T14:58:26.040 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T14:58:26.040 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.040 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4209.0:20 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'10, mlcod=17'8)", 2024-09-21T14:58:26.040 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:03.652481+0000", 2024-09-21T14:58:26.040 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.385367632000001, 2024-09-21T14:58:26.040 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0025744560000000001, 2024-09-21T14:58:26.040 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:26.041 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:26.041 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:26.041 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.041 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:26.041 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.652481+0000", 2024-09-21T14:58:26.041 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:26.041 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.041 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.041 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:26.041 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.652481+0000", 2024-09-21T14:58:26.041 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:26.041 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.041 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.041 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:26.042 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.652479+0000", 2024-09-21T14:58:26.042 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T14:58:26.042 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.042 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.042 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:26.042 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.652662+0000", 2024-09-21T14:58:26.042 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000182857 2024-09-21T14:58:26.042 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.042 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.042 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:26.042 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.652694+0000", 2024-09-21T14:58:26.042 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1628999999999998e-05 2024-09-21T14:58:26.042 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.043 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.043 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:26.043 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.652706+0000", 2024-09-21T14:58:26.043 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.274e-05 2024-09-21T14:58:26.043 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.043 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.043 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:26.043 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.652790+0000", 2024-09-21T14:58:26.043 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3500999999999999e-05 2024-09-21T14:58:26.043 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.043 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.043 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:26.043 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.652832+0000", 2024-09-21T14:58:26.044 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1888999999999998e-05 2024-09-21T14:58:26.044 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.044 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.044 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:26.044 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.654949+0000", 2024-09-21T14:58:26.044 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0021170619999999999 2024-09-21T14:58:26.044 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.044 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.044 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:26.044 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.655055+0000", 2024-09-21T14:58:26.044 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010655800000000001 2024-09-21T14:58:26.044 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:26.044 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:26.044 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:26.045 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.045 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.045 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4209.0:32 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'18, mlcod=17'14)", 2024-09-21T14:58:26.045 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:03.671413+0000", 2024-09-21T14:58:26.045 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.366434995999999, 2024-09-21T14:58:26.045 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0024865389999999999, 2024-09-21T14:58:26.045 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:26.045 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:26.045 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:26.045 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.045 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:26.045 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.671413+0000", 2024-09-21T14:58:26.045 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:26.045 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.046 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.046 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:26.046 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.671413+0000", 2024-09-21T14:58:26.046 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:26.046 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.046 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.046 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:26.046 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.671412+0000", 2024-09-21T14:58:26.046 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T14:58:26.046 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.046 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.046 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:26.046 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.671510+0000", 2024-09-21T14:58:26.047 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.8160999999999997e-05 2024-09-21T14:58:26.047 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.047 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.047 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:26.047 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.671532+0000", 2024-09-21T14:58:26.047 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2036999999999999e-05 2024-09-21T14:58:26.047 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.047 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.047 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:26.047 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.671542+0000", 2024-09-21T14:58:26.047 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0242e-05 2024-09-21T14:58:26.047 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.047 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.048 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:26.048 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.671625+0000", 2024-09-21T14:58:26.048 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3404000000000003e-05 2024-09-21T14:58:26.048 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.048 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.048 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:26.048 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.671665+0000", 2024-09-21T14:58:26.048 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9016e-05 2024-09-21T14:58:26.048 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.048 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.048 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:26.048 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.673813+0000", 2024-09-21T14:58:26.049 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0021483439999999999 2024-09-21T14:58:26.049 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.049 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.049 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:26.049 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.673900+0000", 2024-09-21T14:58:26.049 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7144000000000005e-05 2024-09-21T14:58:26.049 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:26.049 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:26.049 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:26.049 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.049 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.049 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4209.0:68 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'42, mlcod=17'39)", 2024-09-21T14:58:26.049 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:03.727004+0000", 2024-09-21T14:58:26.049 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.310844656, 2024-09-21T14:58:26.050 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.002609428, 2024-09-21T14:58:26.050 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:26.050 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:26.050 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:26.050 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.050 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:26.050 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.727004+0000", 2024-09-21T14:58:26.050 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:26.050 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.050 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.050 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:26.050 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.727004+0000", 2024-09-21T14:58:26.051 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:26.051 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.051 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.051 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:26.051 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.727002+0000", 2024-09-21T14:58:26.051 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:26.051 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.051 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.051 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:26.051 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.727018+0000", 2024-09-21T14:58:26.051 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5531000000000001e-05 2024-09-21T14:58:26.051 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.051 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.051 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:26.052 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.727049+0000", 2024-09-21T14:58:26.052 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1541999999999997e-05 2024-09-21T14:58:26.052 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.052 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.052 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:26.052 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.727064+0000", 2024-09-21T14:58:26.052 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4606e-05 2024-09-21T14:58:26.052 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.052 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.052 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:26.052 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.727133+0000", 2024-09-21T14:58:26.052 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8986999999999995e-05 2024-09-21T14:58:26.053 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.053 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.053 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:26.053 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.727158+0000", 2024-09-21T14:58:26.053 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5048000000000001e-05 2024-09-21T14:58:26.053 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.053 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.053 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:26.053 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.729528+0000", 2024-09-21T14:58:26.053 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0023699530000000002 2024-09-21T14:58:26.053 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.053 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.053 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:26.053 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.729613+0000", 2024-09-21T14:58:26.054 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5237999999999999e-05 2024-09-21T14:58:26.054 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:26.054 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:26.054 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:26.054 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.054 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.054 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4209.0:83 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'53, mlcod=17'50)", 2024-09-21T14:58:26.054 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:03.751652+0000", 2024-09-21T14:58:26.054 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.286196516, 2024-09-21T14:58:26.054 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0023993119999999998, 2024-09-21T14:58:26.054 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:26.054 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:26.054 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:26.055 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.055 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:26.055 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.751652+0000", 2024-09-21T14:58:26.055 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:26.055 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.055 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.055 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:26.055 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.751652+0000", 2024-09-21T14:58:26.055 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:26.055 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.055 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.055 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:26.055 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.751650+0000", 2024-09-21T14:58:26.056 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T14:58:26.056 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.056 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.056 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:26.056 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.751669+0000", 2024-09-21T14:58:26.056 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9236e-05 2024-09-21T14:58:26.056 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.057 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.057 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:26.057 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.751708+0000", 2024-09-21T14:58:26.057 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8343000000000002e-05 2024-09-21T14:58:26.057 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.057 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.057 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:26.058 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.751732+0000", 2024-09-21T14:58:26.058 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4736000000000002e-05 2024-09-21T14:58:26.058 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.058 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.058 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:26.058 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.751818+0000", 2024-09-21T14:58:26.058 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5493000000000003e-05 2024-09-21T14:58:26.059 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.059 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.059 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:26.059 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.751851+0000", 2024-09-21T14:58:26.059 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3167999999999998e-05 2024-09-21T14:58:26.059 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.060 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.060 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:26.060 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.753977+0000", 2024-09-21T14:58:26.060 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0021262939999999999 2024-09-21T14:58:26.060 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.060 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.061 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:26.061 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.754051+0000", 2024-09-21T14:58:26.061 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3892000000000003e-05 2024-09-21T14:58:26.061 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:26.061 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:26.061 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:26.061 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.062 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.062 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4209.0:104 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'66, mlcod=17'64)", 2024-09-21T14:58:26.062 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:03.956537+0000", 2024-09-21T14:58:26.062 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.081311034999999, 2024-09-21T14:58:26.062 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0028947130000000001, 2024-09-21T14:58:26.062 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:26.062 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:26.063 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:26.063 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.063 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:26.063 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.956537+0000", 2024-09-21T14:58:26.063 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:26.063 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.063 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.064 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:26.064 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.956537+0000", 2024-09-21T14:58:26.064 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:26.064 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.064 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.064 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:26.064 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.956536+0000", 2024-09-21T14:58:26.065 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T14:58:26.065 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.065 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.065 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:26.065 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.956550+0000", 2024-09-21T14:58:26.065 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4122e-05 2024-09-21T14:58:26.066 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.066 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.066 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:26.066 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.956581+0000", 2024-09-21T14:58:26.066 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1643000000000002e-05 2024-09-21T14:58:26.066 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.066 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.067 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:26.067 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.956594+0000", 2024-09-21T14:58:26.067 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2500000000000001e-05 2024-09-21T14:58:26.067 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.067 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.067 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:26.068 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.956689+0000", 2024-09-21T14:58:26.068 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5180999999999996e-05 2024-09-21T14:58:26.068 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.068 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.068 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:26.068 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.956727+0000", 2024-09-21T14:58:26.068 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8009999999999997e-05 2024-09-21T14:58:26.069 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.069 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.069 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:26.069 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.959348+0000", 2024-09-21T14:58:26.069 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0026211540000000001 2024-09-21T14:58:26.069 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.069 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.070 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:26.070 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.959432+0000", 2024-09-21T14:58:26.070 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3849000000000003e-05 2024-09-21T14:58:26.070 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:26.070 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:26.070 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:26.071 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.071 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.071 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4209.0:105 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'67, mlcod=17'64)", 2024-09-21T14:58:26.071 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:03.957110+0000", 2024-09-21T14:58:26.071 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.080737986999999, 2024-09-21T14:58:26.071 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0031464470000000001, 2024-09-21T14:58:26.072 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:26.072 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:26.072 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:26.072 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.072 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:26.072 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.957110+0000", 2024-09-21T14:58:26.072 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:26.073 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.073 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.073 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:26.073 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.957110+0000", 2024-09-21T14:58:26.073 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:26.073 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.073 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.074 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:26.074 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.957109+0000", 2024-09-21T14:58:26.074 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T14:58:26.074 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.074 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.074 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:26.074 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.957124+0000", 2024-09-21T14:58:26.075 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5639000000000001e-05 2024-09-21T14:58:26.075 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.075 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.075 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:26.075 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.957159+0000", 2024-09-21T14:58:26.075 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4530000000000003e-05 2024-09-21T14:58:26.076 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.076 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.076 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:26.076 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.957173+0000", 2024-09-21T14:58:26.076 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3869000000000001e-05 2024-09-21T14:58:26.076 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.077 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.077 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:26.077 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.957287+0000", 2024-09-21T14:58:26.077 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011453199999999999 2024-09-21T14:58:26.077 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.077 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.077 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:26.078 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.957358+0000", 2024-09-21T14:58:26.078 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0401000000000006e-05 2024-09-21T14:58:26.078 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.078 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.078 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:26.078 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.960177+0000", 2024-09-21T14:58:26.078 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0028189140000000001 2024-09-21T14:58:26.079 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.079 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.079 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:26.079 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.960257+0000", 2024-09-21T14:58:26.079 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0282000000000001e-05 2024-09-21T14:58:26.079 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:26.080 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:26.080 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:26.080 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.080 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.080 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4209.0:106 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'68, mlcod=17'64)", 2024-09-21T14:58:26.080 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:03.957977+0000", 2024-09-21T14:58:26.080 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.079871349000001, 2024-09-21T14:58:26.080 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0028674130000000001, 2024-09-21T14:58:26.081 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:26.081 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:26.081 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:26.081 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.081 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:26.081 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.957977+0000", 2024-09-21T14:58:26.082 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:26.082 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.082 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.082 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:26.082 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.957977+0000", 2024-09-21T14:58:26.082 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:26.082 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.083 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.083 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:26.083 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.957975+0000", 2024-09-21T14:58:26.083 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T14:58:26.083 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.083 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.083 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:26.084 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.958027+0000", 2024-09-21T14:58:26.084 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1925999999999997e-05 2024-09-21T14:58:26.084 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.084 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.084 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:26.084 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.958077+0000", 2024-09-21T14:58:26.084 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9289000000000002e-05 2024-09-21T14:58:26.085 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.085 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.085 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:26.085 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.958089+0000", 2024-09-21T14:58:26.085 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.276e-05 2024-09-21T14:58:26.085 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.086 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.086 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:26.086 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.958278+0000", 2024-09-21T14:58:26.086 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018880599999999999 2024-09-21T14:58:26.086 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.086 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.086 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:26.087 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.958318+0000", 2024-09-21T14:58:26.087 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9388999999999998e-05 2024-09-21T14:58:26.087 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.087 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.087 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:26.087 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.960790+0000", 2024-09-21T14:58:26.088 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0024728720000000001 2024-09-21T14:58:26.088 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.088 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.088 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:26.088 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.960845+0000", 2024-09-21T14:58:26.088 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4098000000000003e-05 2024-09-21T14:58:26.088 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:26.089 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:26.089 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:26.089 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.089 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.089 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4209.0:107 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'69, mlcod=17'64)", 2024-09-21T14:58:26.089 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:03.959170+0000", 2024-09-21T14:58:26.089 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.078678341, 2024-09-21T14:58:26.089 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0030848759999999999, 2024-09-21T14:58:26.089 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:26.090 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:26.090 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:26.090 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.090 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:26.090 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.959170+0000", 2024-09-21T14:58:26.090 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:26.091 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.091 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.091 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:26.091 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.959170+0000", 2024-09-21T14:58:26.091 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:26.091 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.091 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.092 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:26.092 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.959168+0000", 2024-09-21T14:58:26.092 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T14:58:26.092 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.092 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.092 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:26.092 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.959186+0000", 2024-09-21T14:58:26.093 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7294000000000001e-05 2024-09-21T14:58:26.093 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.093 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.093 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:26.093 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.959221+0000", 2024-09-21T14:58:26.093 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4925000000000002e-05 2024-09-21T14:58:26.093 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.093 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.094 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:26.094 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.959238+0000", 2024-09-21T14:58:26.094 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7079999999999999e-05 2024-09-21T14:58:26.094 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.094 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.094 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:26.095 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.959474+0000", 2024-09-21T14:58:26.095 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00023679199999999999 2024-09-21T14:58:26.095 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.095 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.095 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:26.095 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.959508+0000", 2024-09-21T14:58:26.095 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3195999999999999e-05 2024-09-21T14:58:26.095 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.096 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.096 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:26.096 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.962179+0000", 2024-09-21T14:58:26.096 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0026716700000000001 2024-09-21T14:58:26.096 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.096 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.096 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:26.097 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.962255+0000", 2024-09-21T14:58:26.097 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5739000000000001e-05 2024-09-21T14:58:26.097 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:26.097 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:26.097 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:26.097 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.097 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.098 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4209.0:108 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'70, mlcod=17'64)", 2024-09-21T14:58:26.098 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:03.959894+0000", 2024-09-21T14:58:26.098 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.077954492, 2024-09-21T14:58:26.098 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0026229949999999999, 2024-09-21T14:58:26.098 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:26.098 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:26.099 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:26.099 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.099 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:26.099 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.959894+0000", 2024-09-21T14:58:26.099 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:26.099 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.099 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.100 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:26.100 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.959894+0000", 2024-09-21T14:58:26.100 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:26.100 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.100 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.100 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:26.100 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.959892+0000", 2024-09-21T14:58:26.101 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T14:58:26.101 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.101 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.101 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:26.101 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.959910+0000", 2024-09-21T14:58:26.101 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8423e-05 2024-09-21T14:58:26.101 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.101 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.102 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:26.102 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.959959+0000", 2024-09-21T14:58:26.102 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8785999999999997e-05 2024-09-21T14:58:26.102 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.102 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.102 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:26.103 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.959998+0000", 2024-09-21T14:58:26.103 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8562000000000001e-05 2024-09-21T14:58:26.103 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.103 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.103 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:26.103 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.960325+0000", 2024-09-21T14:58:26.103 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00032692000000000002 2024-09-21T14:58:26.104 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.104 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.104 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:26.104 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.960351+0000", 2024-09-21T14:58:26.104 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6746000000000001e-05 2024-09-21T14:58:26.104 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.104 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.105 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:26.105 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.962399+0000", 2024-09-21T14:58:26.105 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0020471640000000002 2024-09-21T14:58:26.105 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.105 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.105 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:26.105 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.962517+0000", 2024-09-21T14:58:26.106 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000118404 2024-09-21T14:58:26.106 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:26.106 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:26.106 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:26.106 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.106 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.107 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4209.0:109 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'71, mlcod=17'64)", 2024-09-21T14:58:26.107 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:03.960745+0000", 2024-09-21T14:58:26.107 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.077103514000001, 2024-09-21T14:58:26.107 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0033225070000000002, 2024-09-21T14:58:26.107 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:26.107 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:26.107 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:26.108 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.108 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:26.108 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.960745+0000", 2024-09-21T14:58:26.108 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:26.108 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.108 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.108 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:26.109 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.960745+0000", 2024-09-21T14:58:26.109 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:26.109 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.109 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.109 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:26.109 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.960743+0000", 2024-09-21T14:58:26.109 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:26.110 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.110 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.110 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:26.110 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.960798+0000", 2024-09-21T14:58:26.110 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5186000000000003e-05 2024-09-21T14:58:26.110 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.111 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.111 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:26.111 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.960821+0000", 2024-09-21T14:58:26.111 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2353000000000001e-05 2024-09-21T14:58:26.111 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.111 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.112 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:26.112 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.960832+0000", 2024-09-21T14:58:26.112 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1473e-05 2024-09-21T14:58:26.112 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.112 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.112 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:26.113 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.960888+0000", 2024-09-21T14:58:26.113 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5964999999999997e-05 2024-09-21T14:58:26.113 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.113 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.113 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:26.113 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.960917+0000", 2024-09-21T14:58:26.113 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9090000000000001e-05 2024-09-21T14:58:26.114 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.114 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.114 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:26.114 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.963979+0000", 2024-09-21T14:58:26.114 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0030617650000000001 2024-09-21T14:58:26.114 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.115 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.115 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:26.115 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.964067+0000", 2024-09-21T14:58:26.115 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8337999999999993e-05 2024-09-21T14:58:26.115 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:26.115 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:26.116 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:26.116 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.116 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.116 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4209.0:110 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'72, mlcod=17'67)", 2024-09-21T14:58:26.116 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:03.962329+0000", 2024-09-21T14:58:26.116 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.075519066999998, 2024-09-21T14:58:26.117 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0034337909999999998, 2024-09-21T14:58:26.117 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:26.117 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:26.117 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:26.117 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.117 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:26.118 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.962329+0000", 2024-09-21T14:58:26.118 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:26.118 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.118 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.118 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:26.118 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.962329+0000", 2024-09-21T14:58:26.119 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:26.119 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.119 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.119 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:26.119 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.962328+0000", 2024-09-21T14:58:26.119 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:26.119 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.120 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.120 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:26.120 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.962345+0000", 2024-09-21T14:58:26.120 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7479999999999999e-05 2024-09-21T14:58:26.120 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.120 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.121 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:26.121 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.962371+0000", 2024-09-21T14:58:26.121 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5252000000000001e-05 2024-09-21T14:58:26.121 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.121 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.121 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:26.121 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.962393+0000", 2024-09-21T14:58:26.122 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2174000000000001e-05 2024-09-21T14:58:26.122 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.122 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.122 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:26.122 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.962571+0000", 2024-09-21T14:58:26.122 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000177749 2024-09-21T14:58:26.123 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.123 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.123 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:26.123 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.962602+0000", 2024-09-21T14:58:26.123 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1776000000000002e-05 2024-09-21T14:58:26.123 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.123 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.123 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:26.123 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.965690+0000", 2024-09-21T14:58:26.124 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0030873850000000002 2024-09-21T14:58:26.124 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.124 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.124 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:26.124 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.965763+0000", 2024-09-21T14:58:26.124 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3487999999999998e-05 2024-09-21T14:58:26.124 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:26.124 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:26.124 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:26.124 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.124 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.124 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4209.0:111 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'73, mlcod=17'67)", 2024-09-21T14:58:26.124 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:03.962973+0000", 2024-09-21T14:58:26.124 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.074875623000001, 2024-09-21T14:58:26.125 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0028287170000000001, 2024-09-21T14:58:26.125 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:26.125 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:26.125 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:26.125 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.125 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:26.125 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.962973+0000", 2024-09-21T14:58:26.125 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:26.125 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.125 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.125 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:26.125 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.962973+0000", 2024-09-21T14:58:26.125 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:26.125 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.125 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.126 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:26.126 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.962971+0000", 2024-09-21T14:58:26.126 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T14:58:26.126 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.126 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.126 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:26.126 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.962983+0000", 2024-09-21T14:58:26.126 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2353e-05 2024-09-21T14:58:26.126 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.126 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.126 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:26.126 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.963008+0000", 2024-09-21T14:58:26.126 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4626000000000001e-05 2024-09-21T14:58:26.127 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.127 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.127 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:26.127 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.963019+0000", 2024-09-21T14:58:26.127 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1343e-05 2024-09-21T14:58:26.127 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.127 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.127 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:26.127 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.963179+0000", 2024-09-21T14:58:26.127 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015977 2024-09-21T14:58:26.127 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.127 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.127 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:26.127 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.963213+0000", 2024-09-21T14:58:26.128 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4088999999999998e-05 2024-09-21T14:58:26.128 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.128 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.128 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:26.128 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.965766+0000", 2024-09-21T14:58:26.128 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0025524699999999998 2024-09-21T14:58:26.128 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.128 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.128 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:26.128 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.965802+0000", 2024-09-21T14:58:26.128 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.5782999999999999e-05 2024-09-21T14:58:26.128 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:26.128 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:26.128 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:26.129 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.129 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.129 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4209.0:112 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'74, mlcod=17'69)", 2024-09-21T14:58:26.129 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:03.964193+0000", 2024-09-21T14:58:26.129 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.073655122000002, 2024-09-21T14:58:26.129 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0025785349999999999, 2024-09-21T14:58:26.129 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:26.129 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:26.129 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:26.129 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.129 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:26.129 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.964193+0000", 2024-09-21T14:58:26.129 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:26.129 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.130 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.130 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:26.130 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.964193+0000", 2024-09-21T14:58:26.130 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:26.130 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.130 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.130 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:26.130 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.964191+0000", 2024-09-21T14:58:26.130 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-21T14:58:26.130 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.130 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.130 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:26.130 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.964303+0000", 2024-09-21T14:58:26.130 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011144700000000001 2024-09-21T14:58:26.131 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.131 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.131 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:26.131 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.964327+0000", 2024-09-21T14:58:26.131 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4576999999999999e-05 2024-09-21T14:58:26.131 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.131 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.131 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:26.131 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.964344+0000", 2024-09-21T14:58:26.131 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6826000000000001e-05 2024-09-21T14:58:26.131 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.131 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.131 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:26.131 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.964559+0000", 2024-09-21T14:58:26.131 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021451200000000001 2024-09-21T14:58:26.132 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.132 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.132 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:26.132 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.964597+0000", 2024-09-21T14:58:26.132 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8266000000000003e-05 2024-09-21T14:58:26.132 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.132 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.132 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:26.132 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.966654+0000", 2024-09-21T14:58:26.132 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0020573129999999999 2024-09-21T14:58:26.132 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.132 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.132 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:26.132 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.966772+0000", 2024-09-21T14:58:26.133 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000117741 2024-09-21T14:58:26.133 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:26.133 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:26.133 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:26.133 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.133 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.133 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4209.0:116 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'78, mlcod=17'73)", 2024-09-21T14:58:26.133 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:03.969123+0000", 2024-09-21T14:58:26.133 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.068725392000001, 2024-09-21T14:58:26.133 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0032522969999999999, 2024-09-21T14:58:26.133 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:26.133 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:26.133 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:26.133 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.134 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:26.134 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.969123+0000", 2024-09-21T14:58:26.134 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:26.134 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.134 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.134 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:26.134 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.969123+0000", 2024-09-21T14:58:26.134 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:26.134 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.134 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.134 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:26.134 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.969122+0000", 2024-09-21T14:58:26.134 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:26.134 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.135 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.135 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:26.135 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.969132+0000", 2024-09-21T14:58:26.135 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5810000000000003e-06 2024-09-21T14:58:26.135 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.135 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.135 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:26.135 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.969145+0000", 2024-09-21T14:58:26.135 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3029e-05 2024-09-21T14:58:26.135 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.135 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.135 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:26.135 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.969151+0000", 2024-09-21T14:58:26.135 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9279999999999998e-06 2024-09-21T14:58:26.136 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.136 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.136 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:26.136 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.969321+0000", 2024-09-21T14:58:26.136 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017070900000000001 2024-09-21T14:58:26.136 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.136 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.136 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:26.136 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.969390+0000", 2024-09-21T14:58:26.136 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8548999999999997e-05 2024-09-21T14:58:26.136 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.136 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.137 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:26.137 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.972257+0000", 2024-09-21T14:58:26.137 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0028669390000000002 2024-09-21T14:58:26.137 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.137 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.137 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:26.137 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.972375+0000", 2024-09-21T14:58:26.137 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011844400000000001 2024-09-21T14:58:26.137 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:26.137 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:26.137 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:26.137 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.137 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.138 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4209.0:130 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'88, mlcod=17'85)", 2024-09-21T14:58:26.138 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:04.496317+0000", 2024-09-21T14:58:26.138 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 21.541531844000001, 2024-09-21T14:58:26.138 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.002431744, 2024-09-21T14:58:26.138 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:26.138 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:26.138 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:26.138 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.138 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:26.138 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:04.496317+0000", 2024-09-21T14:58:26.138 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:26.138 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.138 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.138 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:26.139 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:04.496317+0000", 2024-09-21T14:58:26.139 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:26.139 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.139 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.139 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:26.139 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:04.496315+0000", 2024-09-21T14:58:26.139 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T14:58:26.139 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.139 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.139 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:26.139 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:04.496329+0000", 2024-09-21T14:58:26.139 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.431e-05 2024-09-21T14:58:26.139 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.139 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.140 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:26.140 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:04.496358+0000", 2024-09-21T14:58:26.140 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8436e-05 2024-09-21T14:58:26.140 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.140 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.140 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:26.140 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:04.496370+0000", 2024-09-21T14:58:26.140 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2576e-05 2024-09-21T14:58:26.140 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.140 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.140 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:26.140 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:04.496445+0000", 2024-09-21T14:58:26.140 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4534000000000002e-05 2024-09-21T14:58:26.140 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.140 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.141 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:26.141 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:04.496469+0000", 2024-09-21T14:58:26.141 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4700000000000001e-05 2024-09-21T14:58:26.141 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.141 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.141 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:26.141 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:04.498692+0000", 2024-09-21T14:58:26.141 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0022227530000000001 2024-09-21T14:58:26.141 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.141 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.141 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:26.141 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:04.498748+0000", 2024-09-21T14:58:26.141 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6221999999999998e-05 2024-09-21T14:58:26.141 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:26.141 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:26.142 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:26.142 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.142 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.142 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4209.0:131 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'89, mlcod=17'85)", 2024-09-21T14:58:26.142 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:04.497490+0000", 2024-09-21T14:58:26.142 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 21.540358505, 2024-09-21T14:58:26.142 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0024435889999999999, 2024-09-21T14:58:26.142 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:26.142 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:26.142 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:26.142 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.142 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:26.142 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:04.497490+0000", 2024-09-21T14:58:26.142 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:26.143 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.143 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.143 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:26.143 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:04.497490+0000", 2024-09-21T14:58:26.143 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:26.143 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.143 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.143 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:26.143 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:04.497488+0000", 2024-09-21T14:58:26.143 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T14:58:26.143 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.143 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.143 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:26.143 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:04.497580+0000", 2024-09-21T14:58:26.143 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2251999999999997e-05 2024-09-21T14:58:26.144 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.144 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.144 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:26.144 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:04.497614+0000", 2024-09-21T14:58:26.144 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3206e-05 2024-09-21T14:58:26.144 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.144 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.144 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:26.144 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:04.497627+0000", 2024-09-21T14:58:26.144 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3149999999999999e-05 2024-09-21T14:58:26.144 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.144 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.144 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:26.144 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:04.497723+0000", 2024-09-21T14:58:26.144 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6442999999999998e-05 2024-09-21T14:58:26.145 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.145 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.145 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:26.145 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:04.497758+0000", 2024-09-21T14:58:26.145 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.5102999999999999e-05 2024-09-21T14:58:26.145 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.145 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.145 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:26.145 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:04.499870+0000", 2024-09-21T14:58:26.145 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0021113410000000001 2024-09-21T14:58:26.145 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.145 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.145 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:26.145 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:04.499934+0000", 2024-09-21T14:58:26.145 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3887999999999995e-05 2024-09-21T14:58:26.146 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:26.146 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:26.146 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:26.146 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.146 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.146 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4209.0:134 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'92, mlcod=18'87)", 2024-09-21T14:58:26.146 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:04.501035+0000", 2024-09-21T14:58:26.146 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 21.536813763000001, 2024-09-21T14:58:26.146 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0030509510000000001, 2024-09-21T14:58:26.146 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:26.146 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:26.146 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:26.146 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.146 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:26.147 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:04.501035+0000", 2024-09-21T14:58:26.147 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:26.147 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.147 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.147 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:26.147 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:04.501035+0000", 2024-09-21T14:58:26.147 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:26.147 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.147 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.147 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:26.147 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:04.501033+0000", 2024-09-21T14:58:26.147 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:26.147 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.147 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.148 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:26.148 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:04.501115+0000", 2024-09-21T14:58:26.148 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1252000000000001e-05 2024-09-21T14:58:26.148 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.148 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.148 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:26.148 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:04.501131+0000", 2024-09-21T14:58:26.148 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5985999999999999e-05 2024-09-21T14:58:26.148 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.148 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.148 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:26.148 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:04.501143+0000", 2024-09-21T14:58:26.148 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2593e-05 2024-09-21T14:58:26.148 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.148 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.149 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:26.149 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:04.501217+0000", 2024-09-21T14:58:26.149 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3503000000000004e-05 2024-09-21T14:58:26.149 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.149 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.149 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:26.149 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:04.501246+0000", 2024-09-21T14:58:26.149 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8785e-05 2024-09-21T14:58:26.149 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.149 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.149 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:26.149 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:04.503975+0000", 2024-09-21T14:58:26.149 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0027293259999999998 2024-09-21T14:58:26.149 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.149 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.150 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:26.150 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:04.504086+0000", 2024-09-21T14:58:26.150 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000110776 2024-09-21T14:58:26.150 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:26.150 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:26.150 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:26.150 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.150 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.150 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4209.0:136 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'94, mlcod=18'89)", 2024-09-21T14:58:26.150 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:04.503869+0000", 2024-09-21T14:58:26.150 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 21.533979711000001, 2024-09-21T14:58:26.150 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0024804580000000001, 2024-09-21T14:58:26.150 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:26.150 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:26.151 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:26.151 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.151 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:26.151 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:04.503869+0000", 2024-09-21T14:58:26.151 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:26.151 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.151 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.151 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:26.151 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:04.503869+0000", 2024-09-21T14:58:26.151 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:26.151 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.151 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.151 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:26.152 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:04.503868+0000", 2024-09-21T14:58:26.152 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:26.152 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.152 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.152 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:26.152 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:04.503882+0000", 2024-09-21T14:58:26.152 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4657e-05 2024-09-21T14:58:26.152 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.152 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.152 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:26.152 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:04.503907+0000", 2024-09-21T14:58:26.152 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4751000000000001e-05 2024-09-21T14:58:26.152 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.152 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.153 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:26.153 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:04.503920+0000", 2024-09-21T14:58:26.153 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2660000000000001e-05 2024-09-21T14:58:26.153 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.153 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.153 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:26.153 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:04.504127+0000", 2024-09-21T14:58:26.153 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000206885 2024-09-21T14:58:26.153 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.153 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.153 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:26.153 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:04.504163+0000", 2024-09-21T14:58:26.153 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6035999999999999e-05 2024-09-21T14:58:26.153 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.154 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.154 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:26.154 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:04.506299+0000", 2024-09-21T14:58:26.154 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0021365070000000002 2024-09-21T14:58:26.154 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.154 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.154 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:26.154 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:04.506349+0000", 2024-09-21T14:58:26.154 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9919000000000001e-05 2024-09-21T14:58:26.154 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:26.154 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:26.154 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:26.154 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.154 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.155 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4209.0:155 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'109, mlcod=18'105)", 2024-09-21T14:58:26.155 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:05.314656+0000", 2024-09-21T14:58:26.155 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 20.723192580999999, 2024-09-21T14:58:26.155 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0024543859999999998, 2024-09-21T14:58:26.155 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:26.155 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:26.155 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:26.155 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.155 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:26.155 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:05.314656+0000", 2024-09-21T14:58:26.155 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:26.155 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.155 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.155 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:26.156 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:05.314656+0000", 2024-09-21T14:58:26.156 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:26.156 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.156 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.156 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:26.156 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:05.314655+0000", 2024-09-21T14:58:26.156 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:26.156 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.156 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.156 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:26.156 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:05.314848+0000", 2024-09-21T14:58:26.156 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000192967 2024-09-21T14:58:26.156 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.156 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.157 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:26.157 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:05.314861+0000", 2024-09-21T14:58:26.157 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.365e-05 2024-09-21T14:58:26.157 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.157 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.157 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:26.157 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:05.314870+0000", 2024-09-21T14:58:26.157 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3499999999999997e-06 2024-09-21T14:58:26.157 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.157 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.157 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:26.157 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:05.314943+0000", 2024-09-21T14:58:26.157 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3653999999999996e-05 2024-09-21T14:58:26.157 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.157 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.158 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:26.158 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:05.314964+0000", 2024-09-21T14:58:26.158 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0562000000000001e-05 2024-09-21T14:58:26.158 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.158 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.158 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:26.158 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:05.317035+0000", 2024-09-21T14:58:26.158 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0020709399999999998 2024-09-21T14:58:26.158 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.158 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.158 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:26.158 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:05.317110+0000", 2024-09-21T14:58:26.158 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5468000000000003e-05 2024-09-21T14:58:26.158 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:26.158 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:26.159 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:26.159 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.159 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.159 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:502 3.4 e31/24 3:23331720:::rbd_object_map.113693316f36.0000000000000005:head v 31'4, mlcod=0'0)", 2024-09-21T14:58:26.159 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:25.472005+0000", 2024-09-21T14:58:26.159 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.56584389099999999, 2024-09-21T14:58:26.159 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0024445249999999999, 2024-09-21T14:58:26.159 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:26.159 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:26.159 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:26.159 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.159 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:26.159 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.472005+0000", 2024-09-21T14:58:26.159 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:26.160 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.160 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.160 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:26.160 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.472005+0000", 2024-09-21T14:58:26.160 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:26.160 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.160 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.160 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:26.160 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.472004+0000", 2024-09-21T14:58:26.160 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:26.160 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.160 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.160 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:26.160 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.472015+0000", 2024-09-21T14:58:26.161 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1714e-05 2024-09-21T14:58:26.161 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.161 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.161 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:26.161 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.472026+0000", 2024-09-21T14:58:26.161 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0190000000000001e-05 2024-09-21T14:58:26.161 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.161 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.161 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:26.161 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.472033+0000", 2024-09-21T14:58:26.161 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1779999999999997e-06 2024-09-21T14:58:26.161 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.161 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.161 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:26.161 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.472884+0000", 2024-09-21T14:58:26.162 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00085105700000000001 2024-09-21T14:58:26.162 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.162 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.162 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:26.162 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.472905+0000", 2024-09-21T14:58:26.162 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1243e-05 2024-09-21T14:58:26.162 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.162 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.162 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:26.162 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.474412+0000", 2024-09-21T14:58:26.162 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0015071660000000001 2024-09-21T14:58:26.162 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.162 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.162 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:26.163 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.474449+0000", 2024-09-21T14:58:26.163 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6804999999999997e-05 2024-09-21T14:58:26.163 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:26.163 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:26.163 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:26.163 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:26.163 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:26.163 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:58:26.164 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T14:58:26.165 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:58:26.161+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:58:26.168 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-21T14:58:26.172 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-09-21T14:58:26.183 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":37,"stamp":"2024-09-21T14:58:25.780604+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590395,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":162,"num_write_kb":2419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":121,"ondisk_log_size":121,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":34,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":251176,"kb_used_data":2220,"kb_used_omap":12,"kb_used_meta":248819,"kb_avail":754723544,"statfs":{"total":773094113280,"available":772836909056,"internally_reserved":0,"allocated":2273280,"data_stored":1959298,"data_compressed":10404,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":12744,"internal_metadata":254791224},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001219"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T14:58:20.414668+0000","last_change":"2024-09-21T14:58:19.410326+0000","last_active":"2024-09-21T14:58:20.414668+0000","last_peered":"2024-09-21T14:58:20.414668+0000","last_clean":"2024-09-21T14:58:20.414668+0000","last_became_active":"2024-09-21T14:58:19.410055+0000","last_became_peered":"2024-09-21T14:58:19.410055+0000","last_unstale":"2024-09-21T14:58:20.414668+0000","last_undegraded":"2024-09-21T14:58:20.414668+0000","last_fullsized":"2024-09-21T14:58:20.414668+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:49:13.944219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T14:58:20.414506+0000","last_change":"2024-09-21T14:58:19.412656+0000","last_active":"2024-09-21T14:58:20.414506+0000","last_peered":"2024-09-21T14:58:20.414506+0000","last_clean":"2024-09-21T14:58:20.414506+0000","last_became_active":"2024-09-21T14:58:19.412353+0000","last_became_peered":"2024-09-21T14:58:19.412353+0000","last_unstale":"2024-09-21T14:58:20.414506+0000","last_undegraded":"2024-09-21T14:58:20.414506+0000","last_fullsized":"2024-09-21T14:58:20.414506+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:38:15.532795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-21T14:58:19.412848+0000","last_change":"2024-09-21T14:58:19.411705+0000","last_active":"2024-09-21T14:58:19.412848+0000","last_peered":"2024-09-21T14:58:19.412848+0000","last_clean":"2024-09-21T14:58:19.412848+0000","last_became_active":"2024-09-21T14:58:19.411387+0000","last_became_peered":"2024-09-21T14:58:19.411387+0000","last_unstale":"2024-09-21T14:58:19.412848+0000","last_undegraded":"2024-09-21T14:58:19.412848+0000","last_fullsized":"2024-09-21T14:58:19.412848+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:37:57.597069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T14:58:20.408502+0000","last_change":"2024-09-21T14:58:19.411642+0000","last_active":"2024-09-21T14:58:20.408502+0000","last_peered":"2024-09-21T14:58:20.408502+0000","last_clean":"2024-09-21T14:58:20.408502+0000","last_became_active":"2024-09-21T14:58:19.411306+0000","last_became_peered":"2024-09-21T14:58:19.411306+0000","last_unstale":"2024-09-21T14:58:20.408502+0000","last_undegraded":"2024-09-21T14:58:20.408502+0000","last_fullsized":"2024-09-21T14:58:20.408502+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:20:27.079795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T14:58:20.493010+0000","last_change":"2024-09-21T14:58:19.413315+0000","last_active":"2024-09-21T14:58:20.493010+0000","last_peered":"2024-09-21T14:58:20.493010+0000","last_clean":"2024-09-21T14:58:20.493010+0000","last_became_active":"2024-09-21T14:58:19.413053+0000","last_became_peered":"2024-09-21T14:58:19.413053+0000","last_unstale":"2024-09-21T14:58:20.493010+0000","last_undegraded":"2024-09-21T14:58:20.493010+0000","last_fullsized":"2024-09-21T14:58:20.493010+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:07:12.901354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"26'1","reported_seq":11,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T14:58:20.411013+0000","last_change":"2024-09-21T14:58:19.409889+0000","last_active":"2024-09-21T14:58:20.411013+0000","last_peered":"2024-09-21T14:58:20.411013+0000","last_clean":"2024-09-21T14:58:20.411013+0000","last_became_active":"2024-09-21T14:58:19.409415+0000","last_became_peered":"2024-09-21T14:58:19.409415+0000","last_unstale":"2024-09-21T14:58:20.411013+0000","last_undegraded":"2024-09-21T14:58:20.411013+0000","last_fullsized":"2024-09-21T14:58:20.411013+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:28:46.336301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T14:58:20.493005+0000","last_change":"2024-09-21T14:58:19.410435+0000","last_active":"2024-09-21T14:58:20.493005+0000","last_peered":"2024-09-21T14:58:20.493005+0000","last_clean":"2024-09-21T14:58:20.493005+0000","last_became_active":"2024-09-21T14:58:19.410062+0000","last_became_peered":"2024-09-21T14:58:19.410062+0000","last_unstale":"2024-09-21T14:58:20.493005+0000","last_undegraded":"2024-09-21T14:58:20.493005+0000","last_fullsized":"2024-09-21T14:58:20.493005+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:12:02.923321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T14:58:20.414573+0000","last_change":"2024-09-21T14:58:19.409695+0000","last_active":"2024-09-21T14:58:20.414573+0000","last_peered":"2024-09-21T14:58:20.414573+0000","last_clean":"2024-09-21T14:58:20.414573+0000","last_became_active":"2024-09-21T14:58:19.409371+0000","last_became_peered":"2024-09-21T14:58:19.409371+0000","last_unstale":"2024-09-21T14:58:20.414573+0000","last_undegraded":"2024-09-21T14:58:20.414573+0000","last_fullsized":"2024-09-21T14:58:20.414573+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:35:27.143445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T14:58:20.414681+0000","last_change":"2024-09-21T14:58:19.410571+0000","last_active":"2024-09-21T14:58:20.414681+0000","last_peered":"2024-09-21T14:58:20.414681+0000","last_clean":"2024-09-21T14:58:20.414681+0000","last_became_active":"2024-09-21T14:58:19.410306+0000","last_became_peered":"2024-09-21T14:58:19.410306+0000","last_unstale":"2024-09-21T14:58:20.414681+0000","last_undegraded":"2024-09-21T14:58:20.414681+0000","last_fullsized":"2024-09-21T14:58:20.414681+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:33:50.089165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-21T14:58:19.416075+0000","last_change":"2024-09-21T14:58:19.414327+0000","last_active":"2024-09-21T14:58:19.416075+0000","last_peered":"2024-09-21T14:58:19.416075+0000","last_clean":"2024-09-21T14:58:19.416075+0000","last_became_active":"2024-09-21T14:58:19.413935+0000","last_became_peered":"2024-09-21T14:58:19.413935+0000","last_unstale":"2024-09-21T14:58:19.416075+0000","last_undegraded":"2024-09-21T14:58:19.416075+0000","last_fullsized":"2024-09-21T14:58:19.416075+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:15:28.307418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T14:58:20.403904+0000","last_change":"2024-09-21T14:58:19.410964+0000","last_active":"2024-09-21T14:58:20.403904+0000","last_peered":"2024-09-21T14:58:20.403904+0000","last_clean":"2024-09-21T14:58:20.403904+0000","last_became_active":"2024-09-21T14:58:19.410687+0000","last_became_peered":"2024-09-21T14:58:19.410687+0000","last_unstale":"2024-09-21T14:58:20.403904+0000","last_undegraded":"2024-09-21T14:58:20.403904+0000","last_fullsized":"2024-09-21T14:58:20.403904+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:21:42.226982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-21T14:58:19.410932+0000","last_change":"2024-09-21T14:58:19.407635+0000","last_active":"2024-09-21T14:58:19.410932+0000","last_peered":"2024-09-21T14:58:19.410932+0000","last_clean":"2024-09-21T14:58:19.410932+0000","last_became_active":"2024-09-21T14:58:19.407310+0000","last_became_peered":"2024-09-21T14:58:19.407310+0000","last_unstale":"2024-09-21T14:58:19.410932+0000","last_undegraded":"2024-09-21T14:58:19.410932+0000","last_fullsized":"2024-09-21T14:58:19.410932+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:47:46.079115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-21T14:58:18.398550+0000","last_change":"2024-09-21T14:58:11.348177+0000","last_active":"2024-09-21T14:58:18.398550+0000","last_peered":"2024-09-21T14:58:18.398550+0000","last_clean":"2024-09-21T14:58:18.398550+0000","last_became_active":"2024-09-21T14:58:09.328417+0000","last_became_peered":"2024-09-21T14:58:09.328417+0000","last_unstale":"2024-09-21T14:58:18.398550+0000","last_undegraded":"2024-09-21T14:58:18.398550+0000","last_fullsized":"2024-09-21T14:58:18.398550+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:52:22.671185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018917490000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T14:58:20.493247+0000","last_change":"2024-09-21T14:58:19.404157+0000","last_active":"2024-09-21T14:58:20.493247+0000","last_peered":"2024-09-21T14:58:20.493247+0000","last_clean":"2024-09-21T14:58:20.493247+0000","last_became_active":"2024-09-21T14:58:19.403868+0000","last_became_peered":"2024-09-21T14:58:19.403868+0000","last_unstale":"2024-09-21T14:58:20.493247+0000","last_undegraded":"2024-09-21T14:58:20.493247+0000","last_fullsized":"2024-09-21T14:58:20.493247+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:52:08.672416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-21T14:58:19.397919+0000","last_change":"2024-09-21T14:58:11.405717+0000","last_active":"2024-09-21T14:58:19.397919+0000","last_peered":"2024-09-21T14:58:19.397919+0000","last_clean":"2024-09-21T14:58:19.397919+0000","last_became_active":"2024-09-21T14:58:09.326849+0000","last_became_peered":"2024-09-21T14:58:09.326849+0000","last_unstale":"2024-09-21T14:58:19.397919+0000","last_undegraded":"2024-09-21T14:58:19.397919+0000","last_fullsized":"2024-09-21T14:58:19.397919+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:29:42.893819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012560399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T14:58:20.493076+0000","last_change":"2024-09-21T14:58:19.411316+0000","last_active":"2024-09-21T14:58:20.493076+0000","last_peered":"2024-09-21T14:58:20.493076+0000","last_clean":"2024-09-21T14:58:20.493076+0000","last_became_active":"2024-09-21T14:58:19.410954+0000","last_became_peered":"2024-09-21T14:58:19.410954+0000","last_unstale":"2024-09-21T14:58:20.493076+0000","last_undegraded":"2024-09-21T14:58:20.493076+0000","last_fullsized":"2024-09-21T14:58:20.493076+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:26:59.084415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-21T14:58:18.398580+0000","last_change":"2024-09-21T14:58:11.348182+0000","last_active":"2024-09-21T14:58:18.398580+0000","last_peered":"2024-09-21T14:58:18.398580+0000","last_clean":"2024-09-21T14:58:18.398580+0000","last_became_active":"2024-09-21T14:58:09.329036+0000","last_became_peered":"2024-09-21T14:58:09.329036+0000","last_unstale":"2024-09-21T14:58:18.398580+0000","last_undegraded":"2024-09-21T14:58:18.398580+0000","last_fullsized":"2024-09-21T14:58:18.398580+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:24:45.719970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019338770000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T14:58:20.404878+0000","last_change":"2024-09-21T14:58:19.409494+0000","last_active":"2024-09-21T14:58:20.404878+0000","last_peered":"2024-09-21T14:58:20.404878+0000","last_clean":"2024-09-21T14:58:20.404878+0000","last_became_active":"2024-09-21T14:58:19.409130+0000","last_became_peered":"2024-09-21T14:58:19.409130+0000","last_unstale":"2024-09-21T14:58:20.404878+0000","last_undegraded":"2024-09-21T14:58:20.404878+0000","last_fullsized":"2024-09-21T14:58:20.404878+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:47:27.991651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-21T14:58:19.399284+0000","last_change":"2024-09-21T14:58:11.406176+0000","last_active":"2024-09-21T14:58:19.399284+0000","last_peered":"2024-09-21T14:58:19.399284+0000","last_clean":"2024-09-21T14:58:19.399284+0000","last_became_active":"2024-09-21T14:58:09.777465+0000","last_became_peered":"2024-09-21T14:58:09.777465+0000","last_unstale":"2024-09-21T14:58:19.399284+0000","last_undegraded":"2024-09-21T14:58:19.399284+0000","last_fullsized":"2024-09-21T14:58:19.399284+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:55:04.667816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095704000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-21T14:58:19.415848+0000","last_change":"2024-09-21T14:58:19.409395+0000","last_active":"2024-09-21T14:58:19.415848+0000","last_peered":"2024-09-21T14:58:19.415848+0000","last_clean":"2024-09-21T14:58:19.415848+0000","last_became_active":"2024-09-21T14:58:19.408971+0000","last_became_peered":"2024-09-21T14:58:19.408971+0000","last_unstale":"2024-09-21T14:58:19.415848+0000","last_undegraded":"2024-09-21T14:58:19.415848+0000","last_fullsized":"2024-09-21T14:58:19.415848+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:29:52.642121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T14:58:20.403664+0000","last_change":"2024-09-21T14:58:19.418130+0000","last_active":"2024-09-21T14:58:20.403664+0000","last_peered":"2024-09-21T14:58:20.403664+0000","last_clean":"2024-09-21T14:58:20.403664+0000","last_became_active":"2024-09-21T14:58:19.417453+0000","last_became_peered":"2024-09-21T14:58:19.417453+0000","last_unstale":"2024-09-21T14:58:20.403664+0000","last_undegraded":"2024-09-21T14:58:20.403664+0000","last_fullsized":"2024-09-21T14:58:20.403664+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:19:27.087272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-21T14:58:18.398306+0000","last_change":"2024-09-21T14:58:11.347504+0000","last_active":"2024-09-21T14:58:18.398306+0000","last_peered":"2024-09-21T14:58:18.398306+0000","last_clean":"2024-09-21T14:58:18.398306+0000","last_became_active":"2024-09-21T14:58:09.775981+0000","last_became_peered":"2024-09-21T14:58:09.775981+0000","last_unstale":"2024-09-21T14:58:18.398306+0000","last_undegraded":"2024-09-21T14:58:18.398306+0000","last_fullsized":"2024-09-21T14:58:18.398306+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:47:16.286652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012568880000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T14:58:20.414303+0000","last_change":"2024-09-21T14:58:19.406072+0000","last_active":"2024-09-21T14:58:20.414303+0000","last_peered":"2024-09-21T14:58:20.414303+0000","last_clean":"2024-09-21T14:58:20.414303+0000","last_became_active":"2024-09-21T14:58:19.405811+0000","last_became_peered":"2024-09-21T14:58:19.405811+0000","last_unstale":"2024-09-21T14:58:20.414303+0000","last_undegraded":"2024-09-21T14:58:20.414303+0000","last_fullsized":"2024-09-21T14:58:20.414303+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:36:25.551607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T14:58:20.405084+0000","last_change":"2024-09-21T14:58:11.342210+0000","last_active":"2024-09-21T14:58:20.405084+0000","last_peered":"2024-09-21T14:58:20.405084+0000","last_clean":"2024-09-21T14:58:20.405084+0000","last_became_active":"2024-09-21T14:58:09.330624+0000","last_became_peered":"2024-09-21T14:58:09.330624+0000","last_unstale":"2024-09-21T14:58:20.405084+0000","last_undegraded":"2024-09-21T14:58:20.405084+0000","last_fullsized":"2024-09-21T14:58:20.405084+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:29:43.229538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001051151,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T14:58:20.404718+0000","last_change":"2024-09-21T14:58:19.417824+0000","last_active":"2024-09-21T14:58:20.404718+0000","last_peered":"2024-09-21T14:58:20.404718+0000","last_clean":"2024-09-21T14:58:20.404718+0000","last_became_active":"2024-09-21T14:58:19.417529+0000","last_became_peered":"2024-09-21T14:58:19.417529+0000","last_unstale":"2024-09-21T14:58:20.404718+0000","last_undegraded":"2024-09-21T14:58:20.404718+0000","last_fullsized":"2024-09-21T14:58:20.404718+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:38:57.961654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-21T14:58:18.398245+0000","last_change":"2024-09-21T14:58:11.346766+0000","last_active":"2024-09-21T14:58:18.398245+0000","last_peered":"2024-09-21T14:58:18.398245+0000","last_clean":"2024-09-21T14:58:18.398245+0000","last_became_active":"2024-09-21T14:58:09.329748+0000","last_became_peered":"2024-09-21T14:58:09.329748+0000","last_unstale":"2024-09-21T14:58:18.398245+0000","last_undegraded":"2024-09-21T14:58:18.398245+0000","last_fullsized":"2024-09-21T14:58:18.398245+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:31:20.267675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052698000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T14:58:20.493199+0000","last_change":"2024-09-21T14:58:19.416737+0000","last_active":"2024-09-21T14:58:20.493199+0000","last_peered":"2024-09-21T14:58:20.493199+0000","last_clean":"2024-09-21T14:58:20.493199+0000","last_became_active":"2024-09-21T14:58:19.416313+0000","last_became_peered":"2024-09-21T14:58:19.416313+0000","last_unstale":"2024-09-21T14:58:20.493199+0000","last_undegraded":"2024-09-21T14:58:20.493199+0000","last_fullsized":"2024-09-21T14:58:20.493199+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:34:38.858491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-21T14:58:18.398390+0000","last_change":"2024-09-21T14:58:11.347700+0000","last_active":"2024-09-21T14:58:18.398390+0000","last_peered":"2024-09-21T14:58:18.398390+0000","last_clean":"2024-09-21T14:58:18.398390+0000","last_became_active":"2024-09-21T14:58:09.438842+0000","last_became_peered":"2024-09-21T14:58:09.438842+0000","last_unstale":"2024-09-21T14:58:18.398390+0000","last_undegraded":"2024-09-21T14:58:18.398390+0000","last_fullsized":"2024-09-21T14:58:18.398390+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:41:34.626249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001426194,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":168,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-21T14:58:19.402645+0000","last_change":"2024-09-21T14:58:03.616055+0000","last_active":"2024-09-21T14:58:19.402645+0000","last_peered":"2024-09-21T14:58:19.402645+0000","last_clean":"2024-09-21T14:58:19.402645+0000","last_became_active":"2024-09-21T14:58:03.613214+0000","last_became_peered":"2024-09-21T14:58:03.613214+0000","last_unstale":"2024-09-21T14:58:19.402645+0000","last_undegraded":"2024-09-21T14:58:19.402645+0000","last_fullsized":"2024-09-21T14:58:19.402645+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:02.180162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:02.180162+0000","last_clean_scrub_stamp":"2024-09-21T14:58:02.180162+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:29:05.860361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T14:58:20.492821+0000","last_change":"2024-09-21T14:58:19.404346+0000","last_active":"2024-09-21T14:58:20.492821+0000","last_peered":"2024-09-21T14:58:20.492821+0000","last_clean":"2024-09-21T14:58:20.492821+0000","last_became_active":"2024-09-21T14:58:19.403887+0000","last_became_peered":"2024-09-21T14:58:19.403887+0000","last_unstale":"2024-09-21T14:58:20.492821+0000","last_undegraded":"2024-09-21T14:58:20.492821+0000","last_fullsized":"2024-09-21T14:58:20.492821+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:25:35.185774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-21T14:58:19.407546+0000","last_change":"2024-09-21T14:58:19.400189+0000","last_active":"2024-09-21T14:58:19.407546+0000","last_peered":"2024-09-21T14:58:19.407546+0000","last_clean":"2024-09-21T14:58:19.407546+0000","last_became_active":"2024-09-21T14:58:19.399712+0000","last_became_peered":"2024-09-21T14:58:19.399712+0000","last_unstale":"2024-09-21T14:58:19.407546+0000","last_undegraded":"2024-09-21T14:58:19.407546+0000","last_fullsized":"2024-09-21T14:58:19.407546+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:50:47.558139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-21T14:58:19.411881+0000","last_change":"2024-09-21T14:58:19.408125+0000","last_active":"2024-09-21T14:58:19.411881+0000","last_peered":"2024-09-21T14:58:19.411881+0000","last_clean":"2024-09-21T14:58:19.411881+0000","last_became_active":"2024-09-21T14:58:19.407847+0000","last_became_peered":"2024-09-21T14:58:19.407847+0000","last_unstale":"2024-09-21T14:58:19.411881+0000","last_undegraded":"2024-09-21T14:58:19.411881+0000","last_fullsized":"2024-09-21T14:58:19.411881+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:45:44.979236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T14:58:21.429355+0000","last_change":"2024-09-21T14:58:21.429355+0000","last_active":"2024-09-21T14:58:21.429355+0000","last_peered":"2024-09-21T14:58:21.429355+0000","last_clean":"2024-09-21T14:58:21.429355+0000","last_became_active":"2024-09-21T14:58:19.410679+0000","last_became_peered":"2024-09-21T14:58:19.410679+0000","last_unstale":"2024-09-21T14:58:21.429355+0000","last_undegraded":"2024-09-21T14:58:21.429355+0000","last_fullsized":"2024-09-21T14:58:21.429355+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:07:40.758932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037710199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T14:58:20.403569+0000","last_change":"2024-09-21T14:58:19.411282+0000","last_active":"2024-09-21T14:58:20.403569+0000","last_peered":"2024-09-21T14:58:20.403569+0000","last_clean":"2024-09-21T14:58:20.403569+0000","last_became_active":"2024-09-21T14:58:19.411026+0000","last_became_peered":"2024-09-21T14:58:19.411026+0000","last_unstale":"2024-09-21T14:58:20.403569+0000","last_undegraded":"2024-09-21T14:58:20.403569+0000","last_fullsized":"2024-09-21T14:58:20.403569+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:38:03.448592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-21T14:58:19.412036+0000","last_change":"2024-09-21T14:58:19.408202+0000","last_active":"2024-09-21T14:58:19.412036+0000","last_peered":"2024-09-21T14:58:19.412036+0000","last_clean":"2024-09-21T14:58:19.412036+0000","last_became_active":"2024-09-21T14:58:19.407884+0000","last_became_peered":"2024-09-21T14:58:19.407884+0000","last_unstale":"2024-09-21T14:58:19.412036+0000","last_undegraded":"2024-09-21T14:58:19.412036+0000","last_fullsized":"2024-09-21T14:58:19.412036+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:28:57.111615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-21T14:58:19.416219+0000","last_change":"2024-09-21T14:58:19.414690+0000","last_active":"2024-09-21T14:58:19.416219+0000","last_peered":"2024-09-21T14:58:19.416219+0000","last_clean":"2024-09-21T14:58:19.416219+0000","last_became_active":"2024-09-21T14:58:19.414418+0000","last_became_peered":"2024-09-21T14:58:19.414418+0000","last_unstale":"2024-09-21T14:58:19.416219+0000","last_undegraded":"2024-09-21T14:58:19.416219+0000","last_fullsized":"2024-09-21T14:58:19.416219+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:36:41.583214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T14:58:20.404009+0000","last_change":"2024-09-21T14:58:19.410709+0000","last_active":"2024-09-21T14:58:20.404009+0000","last_peered":"2024-09-21T14:58:20.404009+0000","last_clean":"2024-09-21T14:58:20.404009+0000","last_became_active":"2024-09-21T14:58:19.410406+0000","last_became_peered":"2024-09-21T14:58:19.410406+0000","last_unstale":"2024-09-21T14:58:20.404009+0000","last_undegraded":"2024-09-21T14:58:20.404009+0000","last_fullsized":"2024-09-21T14:58:20.404009+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:59:16.026939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T14:58:20.408692+0000","last_change":"2024-09-21T14:58:19.410187+0000","last_active":"2024-09-21T14:58:20.408692+0000","last_peered":"2024-09-21T14:58:20.408692+0000","last_clean":"2024-09-21T14:58:20.408692+0000","last_became_active":"2024-09-21T14:58:19.409935+0000","last_became_peered":"2024-09-21T14:58:19.409935+0000","last_unstale":"2024-09-21T14:58:20.408692+0000","last_undegraded":"2024-09-21T14:58:20.408692+0000","last_fullsized":"2024-09-21T14:58:20.408692+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:10:36.252629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-21T14:58:19.409847+0000","last_change":"2024-09-21T14:58:19.406446+0000","last_active":"2024-09-21T14:58:19.409847+0000","last_peered":"2024-09-21T14:58:19.409847+0000","last_clean":"2024-09-21T14:58:19.409847+0000","last_became_active":"2024-09-21T14:58:19.406164+0000","last_became_peered":"2024-09-21T14:58:19.406164+0000","last_unstale":"2024-09-21T14:58:19.409847+0000","last_undegraded":"2024-09-21T14:58:19.409847+0000","last_fullsized":"2024-09-21T14:58:19.409847+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:26:32.857391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-21T14:58:20.404717+0000","last_change":"2024-09-21T14:58:19.412340+0000","last_active":"2024-09-21T14:58:20.404717+0000","last_peered":"2024-09-21T14:58:20.404717+0000","last_clean":"2024-09-21T14:58:20.404717+0000","last_became_active":"2024-09-21T14:58:19.412138+0000","last_became_peered":"2024-09-21T14:58:19.412138+0000","last_unstale":"2024-09-21T14:58:20.404717+0000","last_undegraded":"2024-09-21T14:58:20.404717+0000","last_fullsized":"2024-09-21T14:58:20.404717+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:34:16.989941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-21T14:58:19.413068+0000","last_change":"2024-09-21T14:58:19.412295+0000","last_active":"2024-09-21T14:58:19.413068+0000","last_peered":"2024-09-21T14:58:19.413068+0000","last_clean":"2024-09-21T14:58:19.413068+0000","last_became_active":"2024-09-21T14:58:19.412030+0000","last_became_peered":"2024-09-21T14:58:19.412030+0000","last_unstale":"2024-09-21T14:58:19.413068+0000","last_undegraded":"2024-09-21T14:58:19.413068+0000","last_fullsized":"2024-09-21T14:58:19.413068+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:44:29.282391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":8,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":16,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":10404,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509448,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":274432,"data_stored":100374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84299999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70899999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0509999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0780000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542151,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":278528,"data_stored":100393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1279999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69299999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1779999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2749999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542152,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31452,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340388,"statfs":{"total":96636764160,"available":96604557312,"internally_reserved":0,"allocated":335872,"data_stored":690750,"data_compressed":5202,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81100000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0489999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84899999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2230000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2010000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3620000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542151,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31376,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340464,"statfs":{"total":96636764160,"available":96604635136,"internally_reserved":0,"allocated":262144,"data_stored":94255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0429999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.214}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.143}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2609999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3160000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542152,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31380,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340460,"statfs":{"total":96636764160,"available":96604631040,"internally_reserved":0,"allocated":266240,"data_stored":94274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82399999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.753}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99399999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3220000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.22}]}]},{"osd":2,"up_from":14,"seq":60129542150,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31372,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340468,"statfs":{"total":96636764160,"available":96604639232,"internally_reserved":0,"allocated":262144,"data_stored":94255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79700000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84099999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78300000000000003}]}]},{"osd":1,"up_from":14,"seq":60129542151,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31432,"kb_used_data":312,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340408,"statfs":{"total":96636764160,"available":96604577792,"internally_reserved":0,"allocated":319488,"data_stored":684623,"data_compressed":5202,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31848901},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91700000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1080000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84899999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.075}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79100000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88700000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542150,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31384,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340456,"statfs":{"total":96636764160,"available":96604626944,"internally_reserved":0,"allocated":274432,"data_stored":100374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1160000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.141}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93799999999999994}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5202,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5202,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":8,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":8,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T14:58:26.186 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-21T14:58:26.186 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T14:58:26.186 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '1000', 'type': 'test-reweight-by-utilization'} 2024-09-21T14:58:26.186 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 1000 1.0 2024-09-21T14:58:26.265 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:58:26.265+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:58:26.324 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:58:26.324 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:58:26.324 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T14:58:26.324 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:58:26.334 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T14:58:26.366 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:58:26.365+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:58:26.443 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:58:26.443 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:58:26.443 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T14:58:26.443 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T14:58:26.443 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:58:26.453 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T14:58:26.463 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T14:58:26.497 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:58:26.497+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:58:26.501 INFO:teuthology.orchestra.run.smithi032.stdout:moved 0 / 82 (0%) 2024-09-21T14:58:26.501 INFO:teuthology.orchestra.run.smithi032.stdout:avg 10.25 2024-09-21T14:58:26.501 INFO:teuthology.orchestra.run.smithi032.stdout:stddev 2.4367 -> 2.4367 (expected baseline 2.99479) 2024-09-21T14:58:26.502 INFO:teuthology.orchestra.run.smithi032.stdout:min osd.7 with 6 -> 6 pgs (0.585366 -> 0.585366 * mean) 2024-09-21T14:58:26.502 INFO:teuthology.orchestra.run.smithi032.stdout:max osd.3 with 15 -> 15 pgs (1.46341 -> 1.46341 * mean) 2024-09-21T14:58:26.502 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-21T14:58:26.502 INFO:teuthology.orchestra.run.smithi032.stdout:oload 1000 2024-09-21T14:58:26.502 INFO:teuthology.orchestra.run.smithi032.stdout:max_change 1 2024-09-21T14:58:26.502 INFO:teuthology.orchestra.run.smithi032.stdout:max_change_osds 4 2024-09-21T14:58:26.502 INFO:teuthology.orchestra.run.smithi032.stdout:average_utilization 0.0003 2024-09-21T14:58:26.502 INFO:teuthology.orchestra.run.smithi032.stdout:overload_utilization 0.0033 2024-09-21T14:58:26.505 INFO:teuthology.orchestra.run.smithi032.stderr:no change 2024-09-21T14:58:26.517 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2024-09-21T14:58:26.517 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 110 1.0 2024-09-21T14:58:26.569 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:58:26.569+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:58:26.609 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:58:26.609 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T14:58:26.609 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T14:58:26.609 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T14:58:26.609 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.609 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4281.0:5 2.2 e21/19 2:5cea7035:::rbd_info:head v 21'1, mlcod=0'0)", 2024-09-21T14:58:26.610 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:10.343460+0000", 2024-09-21T14:58:26.610 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 16.264890519000001, 2024-09-21T14:58:26.610 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0024752820000000001, 2024-09-21T14:58:26.610 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:26.610 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:26.610 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:26.610 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.610 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:26.610 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:10.343460+0000", 2024-09-21T14:58:26.610 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:26.610 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.610 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.610 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:26.610 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:10.343460+0000", 2024-09-21T14:58:26.611 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:26.611 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.611 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.611 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:26.611 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:10.343459+0000", 2024-09-21T14:58:26.611 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:26.611 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.611 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.611 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:26.611 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:10.343499+0000", 2024-09-21T14:58:26.611 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9983e-05 2024-09-21T14:58:26.611 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.611 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.611 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:26.611 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:10.343521+0000", 2024-09-21T14:58:26.612 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1858999999999999e-05 2024-09-21T14:58:26.612 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.612 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.612 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:26.612 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:10.343540+0000", 2024-09-21T14:58:26.612 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9426e-05 2024-09-21T14:58:26.612 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.612 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.612 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:26.612 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:10.343647+0000", 2024-09-21T14:58:26.612 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000106612 2024-09-21T14:58:26.612 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.612 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.612 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:26.613 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:10.343723+0000", 2024-09-21T14:58:26.613 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6230999999999996e-05 2024-09-21T14:58:26.613 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.613 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.613 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:26.613 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:10.345869+0000", 2024-09-21T14:58:26.613 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0021460450000000001 2024-09-21T14:58:26.613 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.613 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.613 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:26.613 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:10.345935+0000", 2024-09-21T14:58:26.613 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5776000000000002e-05 2024-09-21T14:58:26.613 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:26.613 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:26.614 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:26.614 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.614 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.614 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4281.0:7 2.2 e22/19 2:5cea7035:::rbd_info:head v 22'2, mlcod=0'0)", 2024-09-21T14:58:26.614 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:11.349430+0000", 2024-09-21T14:58:26.614 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 15.258919884000001, 2024-09-21T14:58:26.614 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0015191499999999999, 2024-09-21T14:58:26.614 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:26.614 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:26.614 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:26.614 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.614 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:26.614 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:11.349430+0000", 2024-09-21T14:58:26.614 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:26.615 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.615 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.615 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:26.615 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:11.349430+0000", 2024-09-21T14:58:26.615 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:26.615 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.615 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.615 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:26.615 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:11.349429+0000", 2024-09-21T14:58:26.615 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:26.615 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.615 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.615 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:26.615 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:11.349450+0000", 2024-09-21T14:58:26.616 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0873000000000002e-05 2024-09-21T14:58:26.616 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.616 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.616 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:26.616 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:11.349471+0000", 2024-09-21T14:58:26.616 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1016e-05 2024-09-21T14:58:26.616 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.616 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.616 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:26.616 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:11.349490+0000", 2024-09-21T14:58:26.616 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8736000000000002e-05 2024-09-21T14:58:26.616 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.616 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.616 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:26.616 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:11.349585+0000", 2024-09-21T14:58:26.617 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5377999999999994e-05 2024-09-21T14:58:26.617 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.617 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.617 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:26.617 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:11.349634+0000", 2024-09-21T14:58:26.617 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9449e-05 2024-09-21T14:58:26.617 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.617 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.617 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:26.617 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:11.350896+0000", 2024-09-21T14:58:26.617 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001261119 2024-09-21T14:58:26.617 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.617 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.617 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:26.618 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:11.350949+0000", 2024-09-21T14:58:26.618 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3749000000000003e-05 2024-09-21T14:58:26.618 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:26.618 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:26.618 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:26.618 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.618 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.618 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:13 3.15 e27/24 3:a8a0cd24:::rbd_object_map.11367ffd86fe:head v 27'1, mlcod=0'0)", 2024-09-21T14:58:26.618 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:21.429587+0000", 2024-09-21T14:58:26.618 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.178762925, 2024-09-21T14:58:26.618 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0013429080000000001, 2024-09-21T14:58:26.618 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:26.618 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:26.618 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:26.619 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.619 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:26.619 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.429587+0000", 2024-09-21T14:58:26.619 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:26.619 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.619 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.619 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:26.619 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.429587+0000", 2024-09-21T14:58:26.619 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:26.619 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.619 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.619 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:26.619 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.429586+0000", 2024-09-21T14:58:26.619 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:26.620 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.620 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.620 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:26.620 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.429595+0000", 2024-09-21T14:58:26.620 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9819999999999997e-06 2024-09-21T14:58:26.620 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.620 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.620 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:26.620 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.429613+0000", 2024-09-21T14:58:26.620 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7158000000000001e-05 2024-09-21T14:58:26.620 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.620 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.620 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:26.620 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.429626+0000", 2024-09-21T14:58:26.621 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3538e-05 2024-09-21T14:58:26.621 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.621 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.621 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:26.621 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.429731+0000", 2024-09-21T14:58:26.621 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000104351 2024-09-21T14:58:26.621 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.621 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.621 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:26.621 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.429758+0000", 2024-09-21T14:58:26.621 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7611999999999999e-05 2024-09-21T14:58:26.621 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.621 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.621 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:26.621 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.430833+0000", 2024-09-21T14:58:26.622 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001074817 2024-09-21T14:58:26.622 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.622 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.622 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:26.622 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.430930+0000", 2024-09-21T14:58:26.622 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7094000000000002e-05 2024-09-21T14:58:26.622 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:26.622 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:26.622 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:26.622 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.622 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.622 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:324 3.4 3:23331720:::rbd_object_map.113693316f36.0000000000000005:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T14:58:26.622 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:25.450424+0000", 2024-09-21T14:58:26.622 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.157926105, 2024-09-21T14:58:26.623 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0045158519999999999, 2024-09-21T14:58:26.623 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:26.623 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:26.623 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:26.623 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:26.623 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:26.623 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 324 2024-09-21T14:58:26.623 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.623 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:26.623 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.623 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:26.623 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.450424+0000", 2024-09-21T14:58:26.623 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:26.623 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.624 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.625 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:26.625 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.450424+0000", 2024-09-21T14:58:26.625 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:26.625 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.626 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.626 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:26.626 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.450422+0000", 2024-09-21T14:58:26.626 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-21T14:58:26.626 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.626 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.626 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:26.626 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.450436+0000", 2024-09-21T14:58:26.626 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4663e-05 2024-09-21T14:58:26.626 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.626 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.626 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:26.626 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.450469+0000", 2024-09-21T14:58:26.626 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2981000000000002e-05 2024-09-21T14:58:26.627 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.627 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.627 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:26.627 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.450482+0000", 2024-09-21T14:58:26.627 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.305e-05 2024-09-21T14:58:26.627 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.627 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.627 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:26.627 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.450544+0000", 2024-09-21T14:58:26.627 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2260000000000004e-05 2024-09-21T14:58:26.627 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.627 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.627 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op must wait for map", 2024-09-21T14:58:26.627 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.450549+0000", 2024-09-21T14:58:26.628 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7870000000000001e-06 2024-09-21T14:58:26.628 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.628 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.628 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:26.628 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.453041+0000", 2024-09-21T14:58:26.628 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0024921639999999998 2024-09-21T14:58:26.628 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.628 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.628 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:26.628 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.453157+0000", 2024-09-21T14:58:26.628 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000115419 2024-09-21T14:58:26.628 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.628 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.628 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:26.628 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.453289+0000", 2024-09-21T14:58:26.629 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013237200000000001 2024-09-21T14:58:26.629 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.629 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.629 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:58:26.629 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.454356+0000", 2024-09-21T14:58:26.629 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0010672139999999999 2024-09-21T14:58:26.629 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.629 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.629 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:26.629 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.454815+0000", 2024-09-21T14:58:26.629 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00045882200000000001 2024-09-21T14:58:26.629 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.629 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.629 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:26.630 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.454891+0000", 2024-09-21T14:58:26.630 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6070999999999997e-05 2024-09-21T14:58:26.630 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.630 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.630 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:26.630 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.454940+0000", 2024-09-21T14:58:26.630 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8466e-05 2024-09-21T14:58:26.630 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:26.630 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:26.630 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:26.630 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.630 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.630 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:400 3.12 3:4ef17d39:::rbd_data.11367ffd86fe.0000000000000014:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T14:58:26.630 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:25.462953+0000", 2024-09-21T14:58:26.631 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.145397226, 2024-09-21T14:58:26.631 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00060437299999999995, 2024-09-21T14:58:26.631 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:26.631 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:58:26.631 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:26.631 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:26.631 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:26.631 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 400 2024-09-21T14:58:26.631 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.631 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:26.631 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.631 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:26.631 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.462953+0000", 2024-09-21T14:58:26.631 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:26.632 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.632 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.632 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:26.632 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.462953+0000", 2024-09-21T14:58:26.632 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:26.632 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.632 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.632 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:26.632 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.462952+0000", 2024-09-21T14:58:26.632 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:26.632 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.632 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.632 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:26.632 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.462957+0000", 2024-09-21T14:58:26.633 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0969999999999996e-06 2024-09-21T14:58:26.633 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.633 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.633 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:26.633 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.462974+0000", 2024-09-21T14:58:26.633 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7068e-05 2024-09-21T14:58:26.633 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.633 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.633 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:26.633 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.462984+0000", 2024-09-21T14:58:26.633 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0254e-05 2024-09-21T14:58:26.633 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.633 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.633 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:26.633 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.463418+0000", 2024-09-21T14:58:26.634 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00043319799999999998 2024-09-21T14:58:26.634 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.634 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.634 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:26.634 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.463557+0000", 2024-09-21T14:58:26.634 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000139553 2024-09-21T14:58:26.634 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:26.634 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:26.634 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:26.634 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.634 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.634 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:406 3.12 3:4b779880:::rbd_data.11367ffd86fe.0000000000000025:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T14:58:26.634 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:25.462994+0000", 2024-09-21T14:58:26.634 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.1453563520000001, 2024-09-21T14:58:26.635 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00082261599999999997, 2024-09-21T14:58:26.635 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:26.635 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:58:26.635 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:26.635 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:26.635 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:26.635 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 406 2024-09-21T14:58:26.635 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.635 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:26.635 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.635 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:26.635 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.462994+0000", 2024-09-21T14:58:26.635 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:26.635 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.636 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.636 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:26.636 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.462994+0000", 2024-09-21T14:58:26.636 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:26.636 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.636 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.636 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:26.636 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.462993+0000", 2024-09-21T14:58:26.636 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:26.636 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.636 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.636 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:26.636 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.462998+0000", 2024-09-21T14:58:26.636 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3970000000000002e-06 2024-09-21T14:58:26.636 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.637 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.637 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:26.637 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.463013+0000", 2024-09-21T14:58:26.637 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4572999999999999e-05 2024-09-21T14:58:26.637 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.637 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.637 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:26.637 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.463023+0000", 2024-09-21T14:58:26.637 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9070000000000002e-06 2024-09-21T14:58:26.637 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.637 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.637 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:26.637 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.463612+0000", 2024-09-21T14:58:26.637 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00058929900000000005 2024-09-21T14:58:26.637 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.638 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.638 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:26.638 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.463816+0000", 2024-09-21T14:58:26.638 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00020416999999999999 2024-09-21T14:58:26.638 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:26.638 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:26.638 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:26.638 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.638 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.638 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:438 3.12 3:4b779880:::rbd_data.11367ffd86fe.0000000000000025:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T14:58:26.638 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:25.466153+0000", 2024-09-21T14:58:26.638 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.142196706, 2024-09-21T14:58:26.638 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00064124600000000005, 2024-09-21T14:58:26.639 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:26.639 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:58:26.639 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:26.639 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:26.639 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:26.639 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 438 2024-09-21T14:58:26.639 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.639 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:26.639 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.639 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:26.639 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.466153+0000", 2024-09-21T14:58:26.639 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:26.639 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.639 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.639 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:26.640 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.466153+0000", 2024-09-21T14:58:26.640 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:26.640 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.640 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.640 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:26.640 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.466153+0000", 2024-09-21T14:58:26.640 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:26.640 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.640 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.640 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:26.640 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.466157+0000", 2024-09-21T14:58:26.640 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.352e-06 2024-09-21T14:58:26.640 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.640 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.641 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:26.641 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.466167+0000", 2024-09-21T14:58:26.641 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9839999999999996e-06 2024-09-21T14:58:26.641 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.641 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.641 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:26.641 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.466175+0000", 2024-09-21T14:58:26.641 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1000000000000004e-06 2024-09-21T14:58:26.641 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.641 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.641 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:26.641 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.466665+0000", 2024-09-21T14:58:26.641 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00048993300000000001 2024-09-21T14:58:26.641 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.641 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.642 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:26.642 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.466795+0000", 2024-09-21T14:58:26.642 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000129663 2024-09-21T14:58:26.642 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:26.642 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:26.642 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:26.642 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.642 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.642 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:499 3.4 3:23331720:::rbd_object_map.113693316f36.0000000000000005:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T14:58:26.642 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:25.469014+0000", 2024-09-21T14:58:26.642 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.139336543, 2024-09-21T14:58:26.642 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0061707929999999999, 2024-09-21T14:58:26.642 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:26.642 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:26.643 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:26.643 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:26.643 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:26.643 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 499 2024-09-21T14:58:26.643 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.643 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:26.643 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.643 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:26.643 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.469014+0000", 2024-09-21T14:58:26.643 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:26.643 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.643 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.643 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:26.643 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.469014+0000", 2024-09-21T14:58:26.644 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:26.644 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.644 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.644 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:26.644 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.469012+0000", 2024-09-21T14:58:26.644 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:26.644 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.644 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.644 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:26.644 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.469021+0000", 2024-09-21T14:58:26.644 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7399999999999993e-06 2024-09-21T14:58:26.644 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.644 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.644 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:26.645 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.469042+0000", 2024-09-21T14:58:26.645 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1687000000000001e-05 2024-09-21T14:58:26.645 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.645 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.645 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:26.645 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.469066+0000", 2024-09-21T14:58:26.645 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3635999999999999e-05 2024-09-21T14:58:26.645 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.645 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.645 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:26.645 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.469120+0000", 2024-09-21T14:58:26.645 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3811999999999999e-05 2024-09-21T14:58:26.645 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.645 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.645 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:26.646 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.469248+0000", 2024-09-21T14:58:26.646 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000128501 2024-09-21T14:58:26.646 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.646 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.646 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:26.646 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.469850+0000", 2024-09-21T14:58:26.646 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00060168099999999996 2024-09-21T14:58:26.646 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.646 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.646 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:26.646 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.472543+0000", 2024-09-21T14:58:26.646 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0026930130000000002 2024-09-21T14:58:26.646 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.646 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.647 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:58:26.647 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.475085+0000", 2024-09-21T14:58:26.647 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0025416359999999999 2024-09-21T14:58:26.647 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.647 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.647 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:26.647 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.475140+0000", 2024-09-21T14:58:26.647 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5238999999999997e-05 2024-09-21T14:58:26.647 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.647 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.647 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:26.647 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.475184+0000", 2024-09-21T14:58:26.647 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4459000000000002e-05 2024-09-21T14:58:26.648 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:26.648 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:26.648 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:26.648 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.648 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.648 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:500 3.4 3:23331720:::rbd_object_map.113693316f36.0000000000000005:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T14:58:26.648 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:25.469105+0000", 2024-09-21T14:58:26.648 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.1392447800000001, 2024-09-21T14:58:26.648 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0061870459999999999, 2024-09-21T14:58:26.648 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:26.648 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:26.648 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:26.648 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:26.648 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:26.648 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 500 2024-09-21T14:58:26.649 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.649 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:26.649 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.649 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:26.649 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.469105+0000", 2024-09-21T14:58:26.649 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:26.649 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.649 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.649 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:26.649 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.469105+0000", 2024-09-21T14:58:26.649 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:26.649 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.649 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.649 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:26.650 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.469104+0000", 2024-09-21T14:58:26.650 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:26.650 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.650 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.650 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:26.650 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.469122+0000", 2024-09-21T14:58:26.650 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7655e-05 2024-09-21T14:58:26.650 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.650 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.650 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:26.650 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.469133+0000", 2024-09-21T14:58:26.650 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1422e-05 2024-09-21T14:58:26.650 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.650 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.650 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:26.651 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.469143+0000", 2024-09-21T14:58:26.651 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.8199999999999992e-06 2024-09-21T14:58:26.651 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.651 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.651 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:26.651 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.470410+0000", 2024-09-21T14:58:26.651 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001266959 2024-09-21T14:58:26.651 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.651 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.651 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:26.651 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.470511+0000", 2024-09-21T14:58:26.651 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000100721 2024-09-21T14:58:26.651 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.651 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.652 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:26.652 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.470907+0000", 2024-09-21T14:58:26.652 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00039660200000000002 2024-09-21T14:58:26.652 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.652 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.652 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:26.652 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.475041+0000", 2024-09-21T14:58:26.652 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0041341490000000002 2024-09-21T14:58:26.652 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.652 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.652 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:58:26.652 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.475202+0000", 2024-09-21T14:58:26.652 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000160132 2024-09-21T14:58:26.652 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.653 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.653 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:26.653 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.475249+0000", 2024-09-21T14:58:26.653 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7341e-05 2024-09-21T14:58:26.653 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.653 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.653 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:26.653 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.475292+0000", 2024-09-21T14:58:26.653 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3399000000000002e-05 2024-09-21T14:58:26.653 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:26.653 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:26.653 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:26.653 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.653 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.653 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:502 3.4 3:23331720:::rbd_object_map.113693316f36.0000000000000005:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T14:58:26.654 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:25.469155+0000", 2024-09-21T14:58:26.654 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.1391947309999999, 2024-09-21T14:58:26.654 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0062263259999999999, 2024-09-21T14:58:26.654 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:26.654 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:26.654 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:26.654 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:26.654 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:26.654 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 502 2024-09-21T14:58:26.654 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.654 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:26.654 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.654 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:26.655 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.469155+0000", 2024-09-21T14:58:26.655 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:26.655 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.655 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.655 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:26.655 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.469155+0000", 2024-09-21T14:58:26.655 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:26.655 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.655 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.655 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:26.655 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.469155+0000", 2024-09-21T14:58:26.655 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:26.655 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.655 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.655 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:26.656 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.469159+0000", 2024-09-21T14:58:26.656 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8069999999999999e-06 2024-09-21T14:58:26.656 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.656 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.656 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:26.656 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.469168+0000", 2024-09-21T14:58:26.656 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5799999999999992e-06 2024-09-21T14:58:26.656 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.656 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.656 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:26.656 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.469176+0000", 2024-09-21T14:58:26.656 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2679999999999994e-06 2024-09-21T14:58:26.656 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.656 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.657 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:26.657 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.471364+0000", 2024-09-21T14:58:26.657 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0021876930000000001 2024-09-21T14:58:26.657 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.657 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.657 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:26.657 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.471493+0000", 2024-09-21T14:58:26.657 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000129426 2024-09-21T14:58:26.657 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.657 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.657 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:26.657 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.471898+0000", 2024-09-21T14:58:26.657 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000404254 2024-09-21T14:58:26.657 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.657 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.658 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:26.658 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.475077+0000", 2024-09-21T14:58:26.658 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0031798880000000001 2024-09-21T14:58:26.658 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.658 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.658 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:58:26.658 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.475300+0000", 2024-09-21T14:58:26.658 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00022288700000000001 2024-09-21T14:58:26.658 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.658 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.658 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:26.658 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.475343+0000", 2024-09-21T14:58:26.658 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3046000000000001e-05 2024-09-21T14:58:26.658 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.659 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.659 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:26.659 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.475382+0000", 2024-09-21T14:58:26.659 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8253000000000001e-05 2024-09-21T14:58:26.659 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:26.659 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:26.659 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:26.659 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.659 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.659 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:512 3.4 3:265e2476:::rbd_data.11367ffd86fe.0000000000000047:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T14:58:26.659 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:25.469342+0000", 2024-09-21T14:58:26.659 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.1390080819999999, 2024-09-21T14:58:26.659 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0031520649999999999, 2024-09-21T14:58:26.659 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:26.660 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:58:26.660 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:26.660 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:26.660 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:26.660 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 512 2024-09-21T14:58:26.660 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.660 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:26.660 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.660 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:26.660 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.469342+0000", 2024-09-21T14:58:26.660 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:26.660 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.660 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.660 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:26.660 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.469342+0000", 2024-09-21T14:58:26.661 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:26.661 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.661 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.661 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:26.661 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.469341+0000", 2024-09-21T14:58:26.661 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:26.661 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.661 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.661 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:26.661 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.469347+0000", 2024-09-21T14:58:26.661 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5019999999999998e-06 2024-09-21T14:58:26.661 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.661 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.661 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:26.661 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.469359+0000", 2024-09-21T14:58:26.662 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1770000000000001e-05 2024-09-21T14:58:26.662 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.662 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.662 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:26.662 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.469367+0000", 2024-09-21T14:58:26.662 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7970000000000001e-06 2024-09-21T14:58:26.662 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.662 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.662 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:26.662 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.472364+0000", 2024-09-21T14:58:26.662 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0029971730000000001 2024-09-21T14:58:26.662 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.662 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.662 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:26.662 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.472494+0000", 2024-09-21T14:58:26.663 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012981000000000001 2024-09-21T14:58:26.663 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:26.663 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:26.663 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:26.663 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.663 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.663 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:518 3.4 3:23331720:::rbd_object_map.113693316f36.0000000000000005:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T14:58:26.663 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:25.469843+0000", 2024-09-21T14:58:26.663 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.138507291, 2024-09-21T14:58:26.663 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0056936850000000004, 2024-09-21T14:58:26.663 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:26.663 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:26.663 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:26.663 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:26.663 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:26.664 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 518 2024-09-21T14:58:26.664 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.664 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:26.664 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.664 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:26.664 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.469843+0000", 2024-09-21T14:58:26.664 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:26.664 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.664 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.664 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:26.664 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.469843+0000", 2024-09-21T14:58:26.664 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:26.664 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.664 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.664 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:26.665 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.469842+0000", 2024-09-21T14:58:26.665 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:26.665 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.665 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.665 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:26.665 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.469849+0000", 2024-09-21T14:58:26.665 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8369999999999997e-06 2024-09-21T14:58:26.665 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.665 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.665 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:26.665 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.469873+0000", 2024-09-21T14:58:26.665 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3918e-05 2024-09-21T14:58:26.665 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.665 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.666 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:26.666 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.469882+0000", 2024-09-21T14:58:26.666 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7050000000000005e-06 2024-09-21T14:58:26.666 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.666 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.666 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:26.666 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.472577+0000", 2024-09-21T14:58:26.666 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0026946660000000001 2024-09-21T14:58:26.666 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.666 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.666 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:26.666 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.472707+0000", 2024-09-21T14:58:26.666 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013014400000000001 2024-09-21T14:58:26.666 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.667 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.667 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:26.667 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.473113+0000", 2024-09-21T14:58:26.667 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00040620899999999999 2024-09-21T14:58:26.667 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.667 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.667 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:58:26.667 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.475412+0000", 2024-09-21T14:58:26.667 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0022989920000000001 2024-09-21T14:58:26.667 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.667 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.667 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:26.667 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.475448+0000", 2024-09-21T14:58:26.667 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.5788999999999999e-05 2024-09-21T14:58:26.668 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.668 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.668 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:26.668 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.475485+0000", 2024-09-21T14:58:26.668 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7214e-05 2024-09-21T14:58:26.668 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.668 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.668 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:26.668 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.475536+0000", 2024-09-21T14:58:26.668 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1388999999999999e-05 2024-09-21T14:58:26.668 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:26.668 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:26.668 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:26.668 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.668 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.669 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:519 3.4 3:23331720:::rbd_object_map.113693316f36.0000000000000005:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T14:58:26.669 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:25.469890+0000", 2024-09-21T14:58:26.669 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.1384595710000001, 2024-09-21T14:58:26.669 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.006042108, 2024-09-21T14:58:26.669 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:26.669 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:26.669 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:26.669 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:26.669 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:26.669 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 519 2024-09-21T14:58:26.669 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.669 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:26.669 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.669 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:26.670 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.469890+0000", 2024-09-21T14:58:26.670 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:26.670 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.670 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.670 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:26.670 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.469890+0000", 2024-09-21T14:58:26.670 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:26.670 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.670 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.670 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:26.670 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.469890+0000", 2024-09-21T14:58:26.670 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:26.670 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.670 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.670 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:26.671 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.469895+0000", 2024-09-21T14:58:26.671 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8330000000000001e-06 2024-09-21T14:58:26.671 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.671 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.671 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:26.671 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.469903+0000", 2024-09-21T14:58:26.671 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0430000000000008e-06 2024-09-21T14:58:26.671 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.671 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.671 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:26.671 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.469909+0000", 2024-09-21T14:58:26.671 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9630000000000003e-06 2024-09-21T14:58:26.671 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.671 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.671 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:26.672 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.473541+0000", 2024-09-21T14:58:26.672 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0036322860000000002 2024-09-21T14:58:26.672 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.672 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.672 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:26.672 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.473651+0000", 2024-09-21T14:58:26.672 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000109855 2024-09-21T14:58:26.672 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.672 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.672 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:26.672 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.474176+0000", 2024-09-21T14:58:26.672 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00052483599999999999 2024-09-21T14:58:26.672 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.672 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.673 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:26.673 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.475820+0000", 2024-09-21T14:58:26.673 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001644791 2024-09-21T14:58:26.673 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.673 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.673 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:58:26.673 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.475828+0000", 2024-09-21T14:58:26.673 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4919999999999996e-06 2024-09-21T14:58:26.673 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.673 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.673 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:26.673 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.475877+0000", 2024-09-21T14:58:26.673 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9293999999999999e-05 2024-09-21T14:58:26.673 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.673 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.674 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:26.674 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.475933+0000", 2024-09-21T14:58:26.674 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5379e-05 2024-09-21T14:58:26.674 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:26.674 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:26.674 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:26.674 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.674 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.674 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop_reply(client.4406.0:499 3.4 e31/24 ondisk, result = 0)", 2024-09-21T14:58:26.674 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:25.471616+0000", 2024-09-21T14:58:26.674 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.1367339999999999, 2024-09-21T14:58:26.674 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00093048999999999996, 2024-09-21T14:58:26.674 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:26.674 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:26.675 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:26.675 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.675 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:26.675 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.471616+0000", 2024-09-21T14:58:26.675 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:26.675 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.675 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.675 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:26.675 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.471616+0000", 2024-09-21T14:58:26.675 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:26.675 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.675 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.675 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:26.675 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.471612+0000", 2024-09-21T14:58:26.676 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999962 2024-09-21T14:58:26.676 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.676 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.676 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:26.676 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.471622+0000", 2024-09-21T14:58:26.676 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.268e-06 2024-09-21T14:58:26.676 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.676 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.676 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:26.676 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.471634+0000", 2024-09-21T14:58:26.676 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1914e-05 2024-09-21T14:58:26.676 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.676 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.676 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:26.676 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.471648+0000", 2024-09-21T14:58:26.677 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4482e-05 2024-09-21T14:58:26.677 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.677 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.677 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:26.677 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.472526+0000", 2024-09-21T14:58:26.677 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00087754499999999997 2024-09-21T14:58:26.677 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.677 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.677 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:26.677 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.472535+0000", 2024-09-21T14:58:26.677 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7669999999999998e-06 2024-09-21T14:58:26.677 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.677 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.677 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:26.677 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.472547+0000", 2024-09-21T14:58:26.678 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1119e-05 2024-09-21T14:58:26.678 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:26.678 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:26.678 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:26.678 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.678 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.678 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop_reply(client.4406.0:500 3.4 e31/24 ondisk, result = 0)", 2024-09-21T14:58:26.678 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:25.472918+0000", 2024-09-21T14:58:26.678 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.135432459, 2024-09-21T14:58:26.678 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0021287400000000001, 2024-09-21T14:58:26.678 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:26.678 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:26.678 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:26.678 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.679 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:26.679 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.472918+0000", 2024-09-21T14:58:26.679 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:26.679 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.679 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.679 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:26.679 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.472918+0000", 2024-09-21T14:58:26.679 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:26.679 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.679 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.679 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:26.679 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.472917+0000", 2024-09-21T14:58:26.679 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:26.679 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.679 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.680 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:26.680 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.472923+0000", 2024-09-21T14:58:26.680 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3840000000000002e-06 2024-09-21T14:58:26.680 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.680 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.680 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:26.680 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.472935+0000", 2024-09-21T14:58:26.680 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2099000000000001e-05 2024-09-21T14:58:26.680 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.680 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.680 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:26.680 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.472943+0000", 2024-09-21T14:58:26.680 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7079999999999997e-06 2024-09-21T14:58:26.680 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.680 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.681 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:26.681 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.475026+0000", 2024-09-21T14:58:26.681 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0020835049999999998 2024-09-21T14:58:26.681 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.681 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.681 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:26.681 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.475034+0000", 2024-09-21T14:58:26.681 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2400000000000001e-06 2024-09-21T14:58:26.681 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.681 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.681 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:26.681 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.475046+0000", 2024-09-21T14:58:26.681 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2679e-05 2024-09-21T14:58:26.681 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:26.682 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:26.682 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:26.682 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.682 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.682 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop_reply(client.4406.0:502 3.4 e31/24 ondisk, result = 0)", 2024-09-21T14:58:26.682 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:25.474538+0000", 2024-09-21T14:58:26.682 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.133812394, 2024-09-21T14:58:26.682 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00087569999999999998, 2024-09-21T14:58:26.682 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:26.682 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:26.682 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:26.682 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.682 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:26.682 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.474538+0000", 2024-09-21T14:58:26.683 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:26.683 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.683 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.683 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:26.683 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.474538+0000", 2024-09-21T14:58:26.683 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:26.683 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.683 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.683 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:26.683 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.474537+0000", 2024-09-21T14:58:26.683 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:26.683 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.683 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.683 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:26.683 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.474543+0000", 2024-09-21T14:58:26.684 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3130000000000004e-06 2024-09-21T14:58:26.684 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.684 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.684 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:26.684 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.474558+0000", 2024-09-21T14:58:26.684 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.523e-05 2024-09-21T14:58:26.684 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.684 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.684 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:26.684 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.474565+0000", 2024-09-21T14:58:26.684 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8800000000000002e-06 2024-09-21T14:58:26.684 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.684 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.684 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:26.684 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.475067+0000", 2024-09-21T14:58:26.685 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00050130300000000004 2024-09-21T14:58:26.685 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.685 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.685 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:26.685 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.475073+0000", 2024-09-21T14:58:26.685 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9819999999999996e-06 2024-09-21T14:58:26.685 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.685 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.685 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:26.685 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.475413+0000", 2024-09-21T14:58:26.685 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00034079899999999997 2024-09-21T14:58:26.685 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:26.685 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:26.685 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:26.685 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.686 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.686 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop_reply(client.4406.0:518 3.4 e31/24 ondisk, result = 0)", 2024-09-21T14:58:26.686 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:25.474575+0000", 2024-09-21T14:58:26.686 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.133774743, 2024-09-21T14:58:26.686 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00096987400000000002, 2024-09-21T14:58:26.686 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:26.686 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:26.686 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:26.686 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.686 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:26.686 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.474575+0000", 2024-09-21T14:58:26.686 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:26.686 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.686 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.687 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:26.687 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.474575+0000", 2024-09-21T14:58:26.687 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:26.687 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.687 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.687 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:26.687 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.474575+0000", 2024-09-21T14:58:26.687 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:26.687 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.687 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.687 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:26.687 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.474578+0000", 2024-09-21T14:58:26.687 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7130000000000001e-06 2024-09-21T14:58:26.687 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.687 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.688 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:26.688 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.474584+0000", 2024-09-21T14:58:26.688 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9410000000000004e-06 2024-09-21T14:58:26.688 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.688 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.688 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:26.688 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.474596+0000", 2024-09-21T14:58:26.688 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1955e-05 2024-09-21T14:58:26.688 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.688 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.688 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:26.688 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.475437+0000", 2024-09-21T14:58:26.688 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00084097800000000004 2024-09-21T14:58:26.688 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.688 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.689 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:26.689 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.475443+0000", 2024-09-21T14:58:26.689 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5590000000000002e-06 2024-09-21T14:58:26.689 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.689 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.689 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:26.689 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.475545+0000", 2024-09-21T14:58:26.689 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000102085 2024-09-21T14:58:26.689 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:26.689 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:26.689 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:26.689 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.689 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.689 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:671 3.4 3:23331720:::rbd_object_map.113693316f36.0000000000000005:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T14:58:26.689 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:25.478363+0000", 2024-09-21T14:58:26.690 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.1299871459999999, 2024-09-21T14:58:26.690 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0019101929999999999, 2024-09-21T14:58:26.690 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:26.690 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:26.690 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:26.690 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:26.690 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:26.690 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 671 2024-09-21T14:58:26.690 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.690 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:26.690 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.690 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:26.690 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.478363+0000", 2024-09-21T14:58:26.690 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:26.691 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.691 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.691 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:26.691 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.478363+0000", 2024-09-21T14:58:26.691 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:26.691 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.691 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.691 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:26.691 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.478362+0000", 2024-09-21T14:58:26.691 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:26.691 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.691 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.691 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:26.691 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.478377+0000", 2024-09-21T14:58:26.691 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5332000000000002e-05 2024-09-21T14:58:26.692 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.692 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.692 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:26.692 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.478395+0000", 2024-09-21T14:58:26.692 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7612999999999999e-05 2024-09-21T14:58:26.692 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.692 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.692 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:26.692 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.478403+0000", 2024-09-21T14:58:26.692 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3410000000000003e-06 2024-09-21T14:58:26.692 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.692 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.692 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:26.692 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.478457+0000", 2024-09-21T14:58:26.693 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4237999999999998e-05 2024-09-21T14:58:26.693 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.693 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.693 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:26.693 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.478544+0000", 2024-09-21T14:58:26.693 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6136000000000003e-05 2024-09-21T14:58:26.693 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.693 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.693 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:26.693 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.478983+0000", 2024-09-21T14:58:26.693 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00043967799999999999 2024-09-21T14:58:26.693 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.693 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.693 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:58:26.693 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.479779+0000", 2024-09-21T14:58:26.694 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00079532500000000005 2024-09-21T14:58:26.694 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.694 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.694 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:26.694 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.480219+0000", 2024-09-21T14:58:26.694 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00044051000000000002 2024-09-21T14:58:26.694 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.694 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.694 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:26.694 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.480250+0000", 2024-09-21T14:58:26.694 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0805000000000001e-05 2024-09-21T14:58:26.694 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.694 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.694 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:26.694 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.480273+0000", 2024-09-21T14:58:26.695 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3155000000000001e-05 2024-09-21T14:58:26.695 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:26.695 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:26.695 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:26.695 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.695 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.695 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:672 3.4 3:23331720:::rbd_object_map.113693316f36.0000000000000005:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T14:58:26.695 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:25.478976+0000", 2024-09-21T14:58:26.695 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.1293744910000001, 2024-09-21T14:58:26.695 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0016728859999999999, 2024-09-21T14:58:26.695 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:26.695 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:26.695 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:26.695 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:26.696 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:26.696 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 672 2024-09-21T14:58:26.696 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.696 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:26.696 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.696 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:26.696 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.478976+0000", 2024-09-21T14:58:26.696 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:26.696 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.696 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.696 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:26.696 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.478976+0000", 2024-09-21T14:58:26.696 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:26.696 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.697 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.697 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:26.697 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.478975+0000", 2024-09-21T14:58:26.697 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:26.697 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.697 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.697 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:26.697 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.478981+0000", 2024-09-21T14:58:26.697 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9390000000000002e-06 2024-09-21T14:58:26.697 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.697 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.697 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:26.697 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.478993+0000", 2024-09-21T14:58:26.697 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2563e-05 2024-09-21T14:58:26.697 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.698 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.698 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:26.698 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.479000+0000", 2024-09-21T14:58:26.698 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5490000000000003e-06 2024-09-21T14:58:26.698 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.698 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.698 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:26.698 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.479344+0000", 2024-09-21T14:58:26.698 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00034405500000000001 2024-09-21T14:58:26.698 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.698 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.698 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:26.698 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.479391+0000", 2024-09-21T14:58:26.698 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7403e-05 2024-09-21T14:58:26.699 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.699 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.699 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:26.699 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.479560+0000", 2024-09-21T14:58:26.699 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016838399999999999 2024-09-21T14:58:26.699 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.699 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.699 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:58:26.699 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.480293+0000", 2024-09-21T14:58:26.699 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00073291999999999997 2024-09-21T14:58:26.699 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.699 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.699 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:26.699 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.480598+0000", 2024-09-21T14:58:26.699 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00030574299999999998 2024-09-21T14:58:26.700 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.700 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.700 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:26.700 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.480626+0000", 2024-09-21T14:58:26.700 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.796e-05 2024-09-21T14:58:26.700 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.700 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.700 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:26.700 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.480648+0000", 2024-09-21T14:58:26.700 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2154000000000002e-05 2024-09-21T14:58:26.700 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:26.700 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:26.700 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:26.700 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.700 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.701 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:721 3.4 3:23331720:::rbd_object_map.113693316f36.0000000000000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T14:58:26.701 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:25.529175+0000", 2024-09-21T14:58:26.701 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.079175091, 2024-09-21T14:58:26.701 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0019794589999999998, 2024-09-21T14:58:26.701 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:26.701 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:26.701 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:26.701 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:26.701 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:26.701 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 721 2024-09-21T14:58:26.701 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.701 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:26.701 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.701 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:26.702 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.529175+0000", 2024-09-21T14:58:26.702 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:26.702 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.702 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.702 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:26.702 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.529175+0000", 2024-09-21T14:58:26.702 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:26.702 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.702 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.702 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:26.702 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.529173+0000", 2024-09-21T14:58:26.702 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T14:58:26.702 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.702 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.702 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:26.703 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.529183+0000", 2024-09-21T14:58:26.703 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9340000000000001e-06 2024-09-21T14:58:26.703 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.703 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.703 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:26.703 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.529207+0000", 2024-09-21T14:58:26.703 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3643e-05 2024-09-21T14:58:26.703 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.703 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.703 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:26.703 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.529226+0000", 2024-09-21T14:58:26.703 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9582e-05 2024-09-21T14:58:26.703 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.703 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.703 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:26.704 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.529324+0000", 2024-09-21T14:58:26.704 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7262000000000005e-05 2024-09-21T14:58:26.704 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.704 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.704 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:26.704 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.529438+0000", 2024-09-21T14:58:26.704 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000114892 2024-09-21T14:58:26.704 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.704 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.704 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:26.704 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.529593+0000", 2024-09-21T14:58:26.704 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015426599999999999 2024-09-21T14:58:26.704 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.704 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.704 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:58:26.705 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.530582+0000", 2024-09-21T14:58:26.705 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00098970999999999998 2024-09-21T14:58:26.705 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.705 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.705 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:26.705 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.530989+0000", 2024-09-21T14:58:26.705 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00040641000000000001 2024-09-21T14:58:26.705 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.705 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.705 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:26.705 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.531059+0000", 2024-09-21T14:58:26.705 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9676000000000002e-05 2024-09-21T14:58:26.705 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:26.705 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:26.706 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:26.706 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.531154+0000", 2024-09-21T14:58:26.706 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5880999999999999e-05 2024-09-21T14:58:26.706 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:26.706 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:26.706 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:26.706 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:26.706 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:26.706 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:58:26.707 INFO:teuthology.orchestra.run.smithi032.stdout:77 writesame 0xc0401c2 thru 0xc04e436 data_size 0x1(0xe275 bytes) 2024-09-21T14:58:26.707 INFO:teuthology.orchestra.run.smithi032.stdout:78 trunc from 0xe5b9577 to 0x9d63e5 2024-09-21T14:58:26.708 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:58:26.704+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:58:26.708 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T14:58:26.716 INFO:teuthology.orchestra.run.smithi032.stdout:truncating to largest ever: 0xe712634 2024-09-21T14:58:26.738 INFO:teuthology.orchestra.run.smithi032.stdout:79 trunc from 0x9d63e5 to 0xe712634 2024-09-21T14:58:26.747 INFO:teuthology.orchestra.run.smithi032.stdout:81 clone 2 order 22 su 524288 sc 4 2024-09-21T14:58:26.809 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:58:26.809+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:58:26.817 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:58:26.817 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:58:26.817 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T14:58:26.817 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:58:26.827 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T14:58:26.878 INFO:teuthology.orchestra.run.smithi032.stdout:moved 0 / 82 (0%) 2024-09-21T14:58:26.878 INFO:teuthology.orchestra.run.smithi032.stdout:avg 10.25 2024-09-21T14:58:26.878 INFO:teuthology.orchestra.run.smithi032.stdout:stddev 2.4367 -> 2.4367 (expected baseline 2.99479) 2024-09-21T14:58:26.878 INFO:teuthology.orchestra.run.smithi032.stdout:min osd.7 with 6 -> 6 pgs (0.585366 -> 0.585366 * mean) 2024-09-21T14:58:26.878 INFO:teuthology.orchestra.run.smithi032.stdout:max osd.3 with 15 -> 15 pgs (1.46341 -> 1.46341 * mean) 2024-09-21T14:58:26.879 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-21T14:58:26.879 INFO:teuthology.orchestra.run.smithi032.stdout:oload 110 2024-09-21T14:58:26.879 INFO:teuthology.orchestra.run.smithi032.stdout:max_change 1 2024-09-21T14:58:26.879 INFO:teuthology.orchestra.run.smithi032.stdout:max_change_osds 4 2024-09-21T14:58:26.879 INFO:teuthology.orchestra.run.smithi032.stdout:average_utilization 0.0003 2024-09-21T14:58:26.879 INFO:teuthology.orchestra.run.smithi032.stdout:overload_utilization 0.0004 2024-09-21T14:58:26.881 INFO:teuthology.orchestra.run.smithi032.stderr:no change 2024-09-21T14:58:26.891 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-09-21T14:58:26.891 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 1.0 2024-09-21T14:58:26.910 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:58:26.909+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:58:26.917 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T14:58:26.936 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:58:26.937 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:58:26.937 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T14:58:26.937 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T14:58:26.937 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:58:26.946 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T14:58:27.010 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:58:27.004+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:58:27.068 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T14:58:27.086 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T14:58:27.103 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:58:27.103 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T14:58:27.103 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T14:58:27.103 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T14:58:27.103 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.103 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:12 3.6 e27/24 3:6748301c:::rbd_header.11367ffd86fe:head v 27'1, mlcod=0'0)", 2024-09-21T14:58:27.104 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:21.426395+0000", 2024-09-21T14:58:27.104 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.6747060789999999, 2024-09-21T14:58:27.104 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0015710940000000001, 2024-09-21T14:58:27.104 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:27.104 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:27.104 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:27.104 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.104 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:27.104 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.426395+0000", 2024-09-21T14:58:27.104 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:27.104 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.104 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.104 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:27.105 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.426395+0000", 2024-09-21T14:58:27.105 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:27.105 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.105 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.105 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:27.105 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.426394+0000", 2024-09-21T14:58:27.105 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:27.105 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.105 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.105 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:27.105 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.426405+0000", 2024-09-21T14:58:27.105 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0811e-05 2024-09-21T14:58:27.105 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.105 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.106 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:27.106 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.426438+0000", 2024-09-21T14:58:27.106 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3241999999999998e-05 2024-09-21T14:58:27.106 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.106 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.106 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:27.106 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.426455+0000", 2024-09-21T14:58:27.106 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6506000000000001e-05 2024-09-21T14:58:27.106 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.106 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.106 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:27.106 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.426546+0000", 2024-09-21T14:58:27.107 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1042e-05 2024-09-21T14:58:27.107 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.107 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.107 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:27.107 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.426598+0000", 2024-09-21T14:58:27.107 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1870999999999999e-05 2024-09-21T14:58:27.107 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.107 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.108 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:27.108 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.427884+0000", 2024-09-21T14:58:27.108 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0012864490000000001 2024-09-21T14:58:27.108 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.108 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.108 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:27.108 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.427966+0000", 2024-09-21T14:58:27.108 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1876e-05 2024-09-21T14:58:27.108 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:27.108 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:27.108 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:27.108 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.108 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.109 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:25 3.6 e27/24 3:6748301c:::rbd_header.11367ffd86fe:head v 27'2, mlcod=0'0)", 2024-09-21T14:58:27.109 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:21.442732+0000", 2024-09-21T14:58:27.109 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.6583692909999996, 2024-09-21T14:58:27.109 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0010074070000000001, 2024-09-21T14:58:27.109 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:27.109 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:27.109 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:27.109 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.109 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:27.109 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.442732+0000", 2024-09-21T14:58:27.109 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:27.109 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.109 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.109 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:27.110 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.442732+0000", 2024-09-21T14:58:27.110 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:27.110 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.110 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.110 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:27.110 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.442730+0000", 2024-09-21T14:58:27.110 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:27.110 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.110 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.110 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:27.110 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.442746+0000", 2024-09-21T14:58:27.110 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5330000000000001e-05 2024-09-21T14:58:27.110 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.111 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.111 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:27.111 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.442774+0000", 2024-09-21T14:58:27.111 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8603999999999999e-05 2024-09-21T14:58:27.111 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.111 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.111 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:27.111 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.442787+0000", 2024-09-21T14:58:27.111 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2377e-05 2024-09-21T14:58:27.111 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.111 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.111 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:27.111 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.442871+0000", 2024-09-21T14:58:27.111 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3881999999999994e-05 2024-09-21T14:58:27.112 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.112 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.112 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:27.112 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.442905+0000", 2024-09-21T14:58:27.112 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.464e-05 2024-09-21T14:58:27.112 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.112 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.112 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:27.112 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.443684+0000", 2024-09-21T14:58:27.112 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00077904500000000002 2024-09-21T14:58:27.112 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.112 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.112 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:27.112 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.443739+0000", 2024-09-21T14:58:27.112 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4925e-05 2024-09-21T14:58:27.113 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:27.113 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:27.113 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:27.113 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.113 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.113 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:341 3.b 3:d4183149:::rbd_data.11367ffd86fe.0000000000000021:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T14:58:27.113 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:25.462758+0000", 2024-09-21T14:58:27.113 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.6383430479999999, 2024-09-21T14:58:27.113 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0044292250000000002, 2024-09-21T14:58:27.113 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:27.113 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:58:27.113 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:27.113 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:27.113 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:27.114 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 341 2024-09-21T14:58:27.114 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.114 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:27.114 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.114 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:27.114 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.462758+0000", 2024-09-21T14:58:27.114 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:27.114 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.114 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.114 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:27.114 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.462758+0000", 2024-09-21T14:58:27.114 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:27.114 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.115 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.115 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:27.115 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.462756+0000", 2024-09-21T14:58:27.115 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-21T14:58:27.115 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.115 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.115 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:27.115 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.462768+0000", 2024-09-21T14:58:27.115 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2384e-05 2024-09-21T14:58:27.115 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.115 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.115 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:27.115 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.462797+0000", 2024-09-21T14:58:27.115 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8666999999999998e-05 2024-09-21T14:58:27.116 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.116 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.116 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:27.116 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.462814+0000", 2024-09-21T14:58:27.116 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6730000000000001e-05 2024-09-21T14:58:27.116 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.116 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.116 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:27.116 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.463191+0000", 2024-09-21T14:58:27.117 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000377096 2024-09-21T14:58:27.117 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.117 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.117 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op must wait for map", 2024-09-21T14:58:27.117 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.463204+0000", 2024-09-21T14:58:27.117 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3777999999999999e-05 2024-09-21T14:58:27.117 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.117 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.118 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:27.118 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.467079+0000", 2024-09-21T14:58:27.118 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.003874307 2024-09-21T14:58:27.118 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.118 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.118 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:27.118 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.467187+0000", 2024-09-21T14:58:27.118 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000108451 2024-09-21T14:58:27.118 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:27.118 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:27.118 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:27.118 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.119 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.119 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:357 3.b 3:d4183149:::rbd_data.11367ffd86fe.0000000000000021:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T14:58:27.119 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:25.462843+0000", 2024-09-21T14:58:27.119 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.6382584759999999, 2024-09-21T14:58:27.119 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0044394409999999997, 2024-09-21T14:58:27.119 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:27.119 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:58:27.119 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:27.119 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:27.119 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:27.119 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 357 2024-09-21T14:58:27.119 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.119 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:27.119 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.120 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:27.120 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.462843+0000", 2024-09-21T14:58:27.120 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:27.120 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.120 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.120 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:27.120 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.462843+0000", 2024-09-21T14:58:27.120 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:27.120 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.120 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.120 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:27.120 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.462842+0000", 2024-09-21T14:58:27.120 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:27.120 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.121 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.121 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:27.121 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.462848+0000", 2024-09-21T14:58:27.121 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9020000000000002e-06 2024-09-21T14:58:27.121 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.121 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.121 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:27.121 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.462868+0000", 2024-09-21T14:58:27.121 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0088000000000001e-05 2024-09-21T14:58:27.121 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.121 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.121 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:27.121 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.462897+0000", 2024-09-21T14:58:27.121 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8595e-05 2024-09-21T14:58:27.122 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.122 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.122 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:27.122 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.463327+0000", 2024-09-21T14:58:27.122 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00042943899999999999 2024-09-21T14:58:27.122 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.122 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.122 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting_for_map not empty", 2024-09-21T14:58:27.122 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.463339+0000", 2024-09-21T14:58:27.122 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2767e-05 2024-09-21T14:58:27.122 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.122 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.122 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:27.123 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.467209+0000", 2024-09-21T14:58:27.123 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.003869295 2024-09-21T14:58:27.123 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.123 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.123 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:27.123 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.467282+0000", 2024-09-21T14:58:27.123 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3399000000000007e-05 2024-09-21T14:58:27.123 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:27.123 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:27.123 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:27.123 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.123 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.123 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:436 3.b 3:d1ff3b72:::rbd_data.11367ffd86fe.0000000000000020:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T14:58:27.124 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:25.466633+0000", 2024-09-21T14:58:27.124 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.6344678159999999, 2024-09-21T14:58:27.124 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00086452999999999996, 2024-09-21T14:58:27.124 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:27.124 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:58:27.124 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:27.124 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:27.124 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:27.124 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 436 2024-09-21T14:58:27.124 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.124 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:27.124 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.124 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:27.124 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.466633+0000", 2024-09-21T14:58:27.125 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:27.125 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.125 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.125 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:27.125 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.466633+0000", 2024-09-21T14:58:27.125 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:27.125 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.125 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.125 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:27.125 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.466632+0000", 2024-09-21T14:58:27.125 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:27.125 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.125 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.125 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:27.126 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.466638+0000", 2024-09-21T14:58:27.126 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4589999999999997e-06 2024-09-21T14:58:27.126 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.126 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.126 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:27.126 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.466649+0000", 2024-09-21T14:58:27.126 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0829000000000001e-05 2024-09-21T14:58:27.126 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.126 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.126 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:27.126 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.466668+0000", 2024-09-21T14:58:27.126 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9318e-05 2024-09-21T14:58:27.127 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.127 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.127 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:27.127 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.467405+0000", 2024-09-21T14:58:27.127 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00073700100000000004 2024-09-21T14:58:27.127 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.127 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.127 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:27.127 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.467498+0000", 2024-09-21T14:58:27.127 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2731000000000004e-05 2024-09-21T14:58:27.127 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:27.127 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:27.127 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:27.127 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.128 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.128 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:452 3.b 3:d1ff3b72:::rbd_data.11367ffd86fe.0000000000000020:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T14:58:27.128 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:25.466676+0000", 2024-09-21T14:58:27.128 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.6344246390000001, 2024-09-21T14:58:27.128 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00094031099999999999, 2024-09-21T14:58:27.128 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:27.128 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:58:27.128 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:27.128 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:27.128 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:27.128 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 452 2024-09-21T14:58:27.128 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.128 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:27.128 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.129 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:27.129 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.466676+0000", 2024-09-21T14:58:27.129 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:27.129 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.129 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.129 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:27.129 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.466676+0000", 2024-09-21T14:58:27.129 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:27.129 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.129 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.129 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:27.129 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.466676+0000", 2024-09-21T14:58:27.129 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:27.129 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.129 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.130 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:27.130 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.466681+0000", 2024-09-21T14:58:27.130 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7400000000000004e-06 2024-09-21T14:58:27.130 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.130 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.130 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:27.130 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.466691+0000", 2024-09-21T14:58:27.130 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0086e-05 2024-09-21T14:58:27.130 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.130 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.130 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:27.130 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.466698+0000", 2024-09-21T14:58:27.130 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8439999999999998e-06 2024-09-21T14:58:27.130 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.131 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.131 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:27.131 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.467519+0000", 2024-09-21T14:58:27.131 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00082056200000000005 2024-09-21T14:58:27.131 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.131 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.131 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:27.131 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.467617+0000", 2024-09-21T14:58:27.131 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7684999999999997e-05 2024-09-21T14:58:27.131 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:27.131 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:27.131 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:27.131 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.131 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.132 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:668 3.6 e31/24 3:620f9ae4:::rbd_data.113693316f36.0000000000000007:head v 31'30, mlcod=30'28)", 2024-09-21T14:58:27.132 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:25.477337+0000", 2024-09-21T14:58:27.132 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.6237636449999999, 2024-09-21T14:58:27.132 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0012337439999999999, 2024-09-21T14:58:27.132 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:27.132 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:27.132 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:27.132 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.132 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:27.132 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.477337+0000", 2024-09-21T14:58:27.132 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:27.132 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.132 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.132 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:27.132 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.477337+0000", 2024-09-21T14:58:27.133 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:27.133 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.133 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.133 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:27.133 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.477337+0000", 2024-09-21T14:58:27.133 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:27.133 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.133 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.133 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:27.133 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.477363+0000", 2024-09-21T14:58:27.133 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5967999999999999e-05 2024-09-21T14:58:27.133 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.133 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.134 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:27.134 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.477374+0000", 2024-09-21T14:58:27.134 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1306e-05 2024-09-21T14:58:27.134 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.134 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.134 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:27.134 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.477382+0000", 2024-09-21T14:58:27.134 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2600000000000005e-06 2024-09-21T14:58:27.134 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.134 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.134 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:27.134 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.477437+0000", 2024-09-21T14:58:27.134 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4296000000000003e-05 2024-09-21T14:58:27.134 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.135 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.135 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:27.135 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.477465+0000", 2024-09-21T14:58:27.135 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7915999999999998e-05 2024-09-21T14:58:27.135 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.135 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.135 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:27.135 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.478530+0000", 2024-09-21T14:58:27.135 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00106492 2024-09-21T14:58:27.135 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.135 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.135 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:27.135 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.478571+0000", 2024-09-21T14:58:27.135 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1502999999999999e-05 2024-09-21T14:58:27.136 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:27.136 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:27.136 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:27.136 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.136 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.136 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:673 3.6 e31/24 3:60fbaa5f:::rbd_data.113693316f36.0000000000000014:head v 31'31, mlcod=30'29)", 2024-09-21T14:58:27.136 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:25.481119+0000", 2024-09-21T14:58:27.136 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.619982432, 2024-09-21T14:58:27.136 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00086776300000000002, 2024-09-21T14:58:27.136 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:27.136 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:27.136 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:27.137 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.137 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:27.137 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.481119+0000", 2024-09-21T14:58:27.137 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:27.137 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.137 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.137 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:27.137 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.481119+0000", 2024-09-21T14:58:27.137 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:27.137 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.137 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.137 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:27.137 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.481118+0000", 2024-09-21T14:58:27.138 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:27.138 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.138 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.138 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:27.138 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.481134+0000", 2024-09-21T14:58:27.138 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5724000000000001e-05 2024-09-21T14:58:27.138 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.138 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.138 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:27.138 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.481146+0000", 2024-09-21T14:58:27.138 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2301e-05 2024-09-21T14:58:27.138 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.138 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.139 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:27.139 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.481153+0000", 2024-09-21T14:58:27.139 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3010000000000001e-06 2024-09-21T14:58:27.139 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.139 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.139 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:27.139 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.481194+0000", 2024-09-21T14:58:27.139 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1134000000000003e-05 2024-09-21T14:58:27.139 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.139 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.139 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:27.139 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.481215+0000", 2024-09-21T14:58:27.139 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0407999999999998e-05 2024-09-21T14:58:27.140 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.140 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.140 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:27.140 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.481945+0000", 2024-09-21T14:58:27.140 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00073013699999999995 2024-09-21T14:58:27.140 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.140 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.140 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:27.140 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.481986+0000", 2024-09-21T14:58:27.140 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1486000000000002e-05 2024-09-21T14:58:27.140 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:27.140 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:27.140 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:27.140 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.140 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.141 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:713 3.6 e31/24 3:63a18c8a:::rbd_object_map.113693316f36:head v 31'35, mlcod=31'33)", 2024-09-21T14:58:27.141 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:25.520342+0000", 2024-09-21T14:58:27.141 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.580759384, 2024-09-21T14:58:27.141 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0011569250000000001, 2024-09-21T14:58:27.141 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:27.141 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:27.141 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:27.141 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.141 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:27.141 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.520342+0000", 2024-09-21T14:58:27.141 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:27.141 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.141 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.141 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:27.142 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.520342+0000", 2024-09-21T14:58:27.142 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:27.142 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.142 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.142 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:27.142 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.520340+0000", 2024-09-21T14:58:27.142 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:27.142 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.142 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.142 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:27.142 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.520365+0000", 2024-09-21T14:58:27.142 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4950999999999999e-05 2024-09-21T14:58:27.142 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.143 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.143 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:27.143 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.520390+0000", 2024-09-21T14:58:27.143 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4683999999999999e-05 2024-09-21T14:58:27.143 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.143 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.143 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:27.143 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.520400+0000", 2024-09-21T14:58:27.143 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0738e-05 2024-09-21T14:58:27.143 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.143 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.143 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:27.143 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.520493+0000", 2024-09-21T14:58:27.143 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2361000000000006e-05 2024-09-21T14:58:27.143 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.144 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.144 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:27.144 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.520523+0000", 2024-09-21T14:58:27.144 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0617999999999997e-05 2024-09-21T14:58:27.144 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.144 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.144 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:27.144 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.521433+0000", 2024-09-21T14:58:27.144 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00091005199999999995 2024-09-21T14:58:27.144 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.144 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.144 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:27.144 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.521499+0000", 2024-09-21T14:58:27.144 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5145000000000001e-05 2024-09-21T14:58:27.145 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:27.145 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:27.145 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:27.145 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.145 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.145 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "pg_update_log_missing(3.6 epoch 31/24 rep_tid 41 entries 31'36 (0'0) error 3:63a18c8a:::rbd_object_map.113693316f36:head by client.4406.0:720 0.000000 0 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 31'34)", 2024-09-21T14:58:27.145 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:25.527607+0000", 2024-09-21T14:58:27.145 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.5734938089999999, 2024-09-21T14:58:27.145 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00090469900000000002, 2024-09-21T14:58:27.145 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:27.145 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:58:27.146 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:27.146 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.146 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:27.146 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.527607+0000", 2024-09-21T14:58:27.146 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:27.146 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.146 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.146 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:27.146 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.527607+0000", 2024-09-21T14:58:27.147 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:27.147 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.147 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.147 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:27.147 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.527606+0000", 2024-09-21T14:58:27.147 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T14:58:27.147 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.147 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.148 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:27.148 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.527643+0000", 2024-09-21T14:58:27.148 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7919000000000001e-05 2024-09-21T14:58:27.148 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.148 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.148 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:27.148 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.527684+0000", 2024-09-21T14:58:27.148 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0324999999999998e-05 2024-09-21T14:58:27.148 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.149 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.149 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:27.149 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.527713+0000", 2024-09-21T14:58:27.149 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9274000000000001e-05 2024-09-21T14:58:27.149 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.149 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.149 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:27.149 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.527808+0000", 2024-09-21T14:58:27.150 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4866999999999998e-05 2024-09-21T14:58:27.150 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.150 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.150 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:27.150 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.528512+0000", 2024-09-21T14:58:27.150 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00070402599999999996 2024-09-21T14:58:27.150 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:27.150 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:27.151 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:27.151 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.151 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.151 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:726 3.6 e32/24 3:63a18c8a:::rbd_object_map.113693316f36:head v 32'37, mlcod=31'35)", 2024-09-21T14:58:27.151 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:26.467264+0000", 2024-09-21T14:58:27.151 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.63383663700000004, 2024-09-21T14:58:27.151 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0013553009999999999, 2024-09-21T14:58:27.151 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:27.151 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:27.152 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:27.152 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.152 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:27.152 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.467264+0000", 2024-09-21T14:58:27.152 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:27.152 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.152 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.152 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:27.152 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.467264+0000", 2024-09-21T14:58:27.153 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:27.153 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.153 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.153 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:27.153 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.467264+0000", 2024-09-21T14:58:27.153 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:27.153 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.153 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.153 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:27.154 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.467270+0000", 2024-09-21T14:58:27.154 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4049999999999997e-06 2024-09-21T14:58:27.154 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.154 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.154 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:27.154 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.467283+0000", 2024-09-21T14:58:27.154 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2843999999999999e-05 2024-09-21T14:58:27.154 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.155 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.155 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:27.155 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.467294+0000", 2024-09-21T14:58:27.155 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0567999999999999e-05 2024-09-21T14:58:27.155 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.155 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.155 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:27.155 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.467409+0000", 2024-09-21T14:58:27.155 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000115295 2024-09-21T14:58:27.156 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.156 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.156 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:27.156 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.467469+0000", 2024-09-21T14:58:27.156 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9641e-05 2024-09-21T14:58:27.156 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.156 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.156 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:27.157 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.468522+0000", 2024-09-21T14:58:27.157 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001053525 2024-09-21T14:58:27.157 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.157 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.157 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:27.157 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.468620+0000", 2024-09-21T14:58:27.157 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7497999999999994e-05 2024-09-21T14:58:27.157 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:27.157 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:27.158 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:27.158 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.158 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.158 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:731 3.6 e32/24 3:6748301c:::rbd_header.11367ffd86fe:head v 32'38, mlcod=31'36)", 2024-09-21T14:58:27.158 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:26.603426+0000", 2024-09-21T14:58:27.158 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.49767492299999999, 2024-09-21T14:58:27.158 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0016267930000000001, 2024-09-21T14:58:27.158 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:27.159 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:27.159 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:27.159 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.159 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:27.159 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.603426+0000", 2024-09-21T14:58:27.159 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:27.159 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.159 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.159 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:27.160 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.603426+0000", 2024-09-21T14:58:27.160 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:27.160 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.160 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.160 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:27.160 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.603423+0000", 2024-09-21T14:58:27.160 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999971 2024-09-21T14:58:27.160 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.160 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.161 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:27.161 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.603447+0000", 2024-09-21T14:58:27.161 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3592999999999999e-05 2024-09-21T14:58:27.161 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.161 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.161 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:27.161 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.603494+0000", 2024-09-21T14:58:27.161 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6891999999999998e-05 2024-09-21T14:58:27.161 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.161 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.161 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:27.161 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.603514+0000", 2024-09-21T14:58:27.161 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0176000000000001e-05 2024-09-21T14:58:27.162 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.162 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.162 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:27.162 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.603664+0000", 2024-09-21T14:58:27.162 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014961099999999999 2024-09-21T14:58:27.162 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.162 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.162 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:27.162 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.603729+0000", 2024-09-21T14:58:27.162 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5362000000000002e-05 2024-09-21T14:58:27.162 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.162 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.162 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:27.162 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.604979+0000", 2024-09-21T14:58:27.163 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0012499500000000001 2024-09-21T14:58:27.163 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.163 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.163 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:27.163 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.605053+0000", 2024-09-21T14:58:27.163 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3912000000000007e-05 2024-09-21T14:58:27.163 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:27.163 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:27.163 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:27.163 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.163 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.163 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:747 3.6 e32/24 3:63a18c8a:::rbd_object_map.113693316f36:head v 32'39, mlcod=32'37)", 2024-09-21T14:58:27.163 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:26.621788+0000", 2024-09-21T14:58:27.164 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.47931270199999998, 2024-09-21T14:58:27.164 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0015379980000000001, 2024-09-21T14:58:27.164 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:27.164 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:27.164 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:27.164 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.164 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:27.164 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.621788+0000", 2024-09-21T14:58:27.164 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:27.164 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.164 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.164 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:27.164 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.621788+0000", 2024-09-21T14:58:27.164 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:27.164 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.165 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.165 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:27.165 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.621786+0000", 2024-09-21T14:58:27.165 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-21T14:58:27.165 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.165 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.165 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:27.165 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.621808+0000", 2024-09-21T14:58:27.165 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2194000000000001e-05 2024-09-21T14:58:27.165 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.165 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.165 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:27.165 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.621851+0000", 2024-09-21T14:58:27.165 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2938999999999997e-05 2024-09-21T14:58:27.166 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.166 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.166 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:27.166 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.621873+0000", 2024-09-21T14:58:27.166 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1691999999999998e-05 2024-09-21T14:58:27.166 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.166 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.166 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:27.166 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.622009+0000", 2024-09-21T14:58:27.166 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000136358 2024-09-21T14:58:27.166 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.166 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.166 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:27.167 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.622098+0000", 2024-09-21T14:58:27.167 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8781000000000003e-05 2024-09-21T14:58:27.167 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.167 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.167 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:27.167 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.623253+0000", 2024-09-21T14:58:27.167 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0011549430000000001 2024-09-21T14:58:27.167 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.167 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.167 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:27.167 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.623326+0000", 2024-09-21T14:58:27.167 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3458e-05 2024-09-21T14:58:27.167 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:27.167 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:27.168 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:27.168 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.168 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.168 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:750 3.6 e32/24 3:63a18c8a:::rbd_object_map.113693316f36:head v 32'40, mlcod=32'38)", 2024-09-21T14:58:27.168 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:26.625746+0000", 2024-09-21T14:58:27.168 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.47535487799999998, 2024-09-21T14:58:27.168 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0013943969999999999, 2024-09-21T14:58:27.168 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:27.168 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:27.168 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:27.168 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.168 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:27.168 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.625746+0000", 2024-09-21T14:58:27.168 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:27.169 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.169 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.169 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:27.169 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.625746+0000", 2024-09-21T14:58:27.169 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:27.169 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.169 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.169 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:27.169 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.625744+0000", 2024-09-21T14:58:27.169 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T14:58:27.169 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.169 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.169 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:27.169 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.625763+0000", 2024-09-21T14:58:27.170 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8827000000000001e-05 2024-09-21T14:58:27.170 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.170 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.170 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:27.170 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.625801+0000", 2024-09-21T14:58:27.170 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7545000000000001e-05 2024-09-21T14:58:27.170 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.170 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.170 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:27.170 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.625820+0000", 2024-09-21T14:58:27.170 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9006999999999999e-05 2024-09-21T14:58:27.170 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.170 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.171 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:27.171 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.625904+0000", 2024-09-21T14:58:27.171 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3956999999999996e-05 2024-09-21T14:58:27.171 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.171 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.171 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:27.171 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.625939+0000", 2024-09-21T14:58:27.171 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4841000000000001e-05 2024-09-21T14:58:27.171 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.171 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.171 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:27.171 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.627057+0000", 2024-09-21T14:58:27.171 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001118406 2024-09-21T14:58:27.171 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.172 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.172 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:27.172 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.627141+0000", 2024-09-21T14:58:27.172 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3521000000000002e-05 2024-09-21T14:58:27.172 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:27.172 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:27.172 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:27.172 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.172 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.172 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:753 3.6 e32/24 3:63a18c8a:::rbd_object_map.113693316f36:head v 32'41, mlcod=32'39)", 2024-09-21T14:58:27.172 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:26.694470+0000", 2024-09-21T14:58:27.172 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.406630942, 2024-09-21T14:58:27.172 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001849961, 2024-09-21T14:58:27.172 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:27.173 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:27.173 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:27.173 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.173 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:27.173 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.694470+0000", 2024-09-21T14:58:27.173 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:27.173 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.173 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.173 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:27.173 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.694470+0000", 2024-09-21T14:58:27.173 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:27.173 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.173 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.173 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:27.174 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.694468+0000", 2024-09-21T14:58:27.174 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T14:58:27.174 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.174 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.174 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:27.174 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.694488+0000", 2024-09-21T14:58:27.174 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0202000000000001e-05 2024-09-21T14:58:27.174 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.174 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.174 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:27.174 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.694516+0000", 2024-09-21T14:58:27.174 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7699999999999999e-05 2024-09-21T14:58:27.174 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.175 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.175 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:27.175 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.694533+0000", 2024-09-21T14:58:27.175 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7235999999999999e-05 2024-09-21T14:58:27.175 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.175 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.175 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:27.175 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.694677+0000", 2024-09-21T14:58:27.175 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014342099999999999 2024-09-21T14:58:27.175 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.175 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.175 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:27.175 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.694721+0000", 2024-09-21T14:58:27.175 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4022e-05 2024-09-21T14:58:27.176 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.176 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.176 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:27.176 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.696247+0000", 2024-09-21T14:58:27.176 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0015265879999999999 2024-09-21T14:58:27.176 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.176 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.176 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:27.176 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.696320+0000", 2024-09-21T14:58:27.176 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2662000000000003e-05 2024-09-21T14:58:27.176 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:27.176 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:27.176 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:27.177 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.177 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.177 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:756 3.6 e32/24 3:60fbaa5f:::rbd_data.113693316f36.0000000000000014:head v 32'42, mlcod=32'40)", 2024-09-21T14:58:27.177 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:26.697592+0000", 2024-09-21T14:58:27.177 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.40350945799999999, 2024-09-21T14:58:27.177 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0016480189999999999, 2024-09-21T14:58:27.177 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:27.177 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:27.177 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:27.177 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.177 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:27.177 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.697592+0000", 2024-09-21T14:58:27.177 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:27.177 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.177 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.178 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:27.178 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.697592+0000", 2024-09-21T14:58:27.178 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:27.178 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.178 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.178 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:27.178 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.697590+0000", 2024-09-21T14:58:27.178 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T14:58:27.178 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.178 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.178 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:27.178 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.697606+0000", 2024-09-21T14:58:27.178 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6062999999999999e-05 2024-09-21T14:58:27.178 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.179 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.179 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:27.179 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.697641+0000", 2024-09-21T14:58:27.179 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.5216000000000003e-05 2024-09-21T14:58:27.179 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.179 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.179 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:27.179 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.697676+0000", 2024-09-21T14:58:27.179 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4393000000000001e-05 2024-09-21T14:58:27.179 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.179 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.179 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:27.179 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.697765+0000", 2024-09-21T14:58:27.180 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9918000000000002e-05 2024-09-21T14:58:27.180 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.180 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.180 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:27.180 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.697803+0000", 2024-09-21T14:58:27.180 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7045999999999997e-05 2024-09-21T14:58:27.180 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.180 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.180 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:27.180 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.699163+0000", 2024-09-21T14:58:27.180 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001360874 2024-09-21T14:58:27.180 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.180 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.180 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:27.181 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.699240+0000", 2024-09-21T14:58:27.181 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6188999999999998e-05 2024-09-21T14:58:27.181 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:27.181 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:27.181 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:27.181 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.181 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.181 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:757 3.6 e32/24 3:63a18c8a:::rbd_object_map.113693316f36:head v 32'43, mlcod=32'41)", 2024-09-21T14:58:27.181 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:26.700725+0000", 2024-09-21T14:58:27.181 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.40037610400000001, 2024-09-21T14:58:27.181 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0019856880000000002, 2024-09-21T14:58:27.181 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:27.181 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:27.181 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:27.182 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.182 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:27.182 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.700725+0000", 2024-09-21T14:58:27.182 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:27.182 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.182 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.182 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:27.182 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.700725+0000", 2024-09-21T14:58:27.182 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:27.182 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.182 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.182 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:27.182 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.700723+0000", 2024-09-21T14:58:27.182 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:27.183 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.183 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.183 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:27.183 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.700739+0000", 2024-09-21T14:58:27.183 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5953000000000001e-05 2024-09-21T14:58:27.183 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.183 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.183 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:27.183 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.700776+0000", 2024-09-21T14:58:27.183 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6223000000000002e-05 2024-09-21T14:58:27.183 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.183 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.183 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:27.183 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.700792+0000", 2024-09-21T14:58:27.184 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6765999999999998e-05 2024-09-21T14:58:27.184 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.184 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.184 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:27.184 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.700907+0000", 2024-09-21T14:58:27.184 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011500100000000001 2024-09-21T14:58:27.184 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.184 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.184 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:27.184 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.700965+0000", 2024-09-21T14:58:27.184 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8004999999999998e-05 2024-09-21T14:58:27.184 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.184 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.184 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:27.185 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.702645+0000", 2024-09-21T14:58:27.185 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0016795320000000001 2024-09-21T14:58:27.185 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.185 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.185 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:27.185 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.702711+0000", 2024-09-21T14:58:27.185 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5797999999999996e-05 2024-09-21T14:58:27.185 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:27.185 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:27.185 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:27.185 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.185 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.185 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:759 3.6 e32/24 3:63a18c8a:::rbd_object_map.113693316f36:head v 32'44, mlcod=32'42)", 2024-09-21T14:58:27.185 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:26.704445+0000", 2024-09-21T14:58:27.186 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.396655798, 2024-09-21T14:58:27.186 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001742464, 2024-09-21T14:58:27.186 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:27.186 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:27.186 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:27.186 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.186 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:27.186 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.704445+0000", 2024-09-21T14:58:27.186 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:27.186 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.186 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.186 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:27.186 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.704445+0000", 2024-09-21T14:58:27.187 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:27.187 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.187 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.187 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:27.187 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.704444+0000", 2024-09-21T14:58:27.187 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:27.187 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.187 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.187 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:27.187 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.704454+0000", 2024-09-21T14:58:27.187 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9399999999999997e-06 2024-09-21T14:58:27.187 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.187 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.187 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:27.188 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.704473+0000", 2024-09-21T14:58:27.188 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8593999999999998e-05 2024-09-21T14:58:27.188 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.188 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.188 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:27.188 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.704480+0000", 2024-09-21T14:58:27.188 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8860000000000004e-06 2024-09-21T14:58:27.188 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.188 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.188 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:27.188 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.704604+0000", 2024-09-21T14:58:27.188 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000123643 2024-09-21T14:58:27.189 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.189 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.189 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:27.189 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.704634+0000", 2024-09-21T14:58:27.189 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0145e-05 2024-09-21T14:58:27.189 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.189 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.189 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:27.189 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.706121+0000", 2024-09-21T14:58:27.189 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0014864489999999999 2024-09-21T14:58:27.189 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.189 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.189 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:27.189 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.706188+0000", 2024-09-21T14:58:27.190 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6989000000000005e-05 2024-09-21T14:58:27.190 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:27.190 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:27.190 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:27.190 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.190 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.190 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:765 3.6 e32/24 3:63a18c8a:::rbd_object_map.113693316f36:head v 32'46, mlcod=32'44)", 2024-09-21T14:58:27.190 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:26.711872+0000", 2024-09-21T14:58:27.190 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.38922920100000002, 2024-09-21T14:58:27.190 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0009865570000000001, 2024-09-21T14:58:27.190 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:27.190 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:27.190 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:27.190 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.191 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:27.191 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.711872+0000", 2024-09-21T14:58:27.191 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:27.191 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.191 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.191 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:27.191 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.711872+0000", 2024-09-21T14:58:27.191 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:27.191 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.191 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.191 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:27.191 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.711871+0000", 2024-09-21T14:58:27.191 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:27.191 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.191 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.192 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:27.192 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.711881+0000", 2024-09-21T14:58:27.192 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0441999999999999e-05 2024-09-21T14:58:27.192 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.192 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.192 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:27.192 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.711904+0000", 2024-09-21T14:58:27.192 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2549e-05 2024-09-21T14:58:27.192 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.192 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.192 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:27.192 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.711913+0000", 2024-09-21T14:58:27.192 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0620000000000007e-06 2024-09-21T14:58:27.192 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.193 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.193 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:27.193 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.711977+0000", 2024-09-21T14:58:27.193 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4701000000000003e-05 2024-09-21T14:58:27.193 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.193 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.193 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:27.193 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.711994+0000", 2024-09-21T14:58:27.193 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7153999999999999e-05 2024-09-21T14:58:27.193 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.193 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.193 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:27.193 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.712814+0000", 2024-09-21T14:58:27.194 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00081953099999999997 2024-09-21T14:58:27.194 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.194 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.194 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:27.194 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.712858+0000", 2024-09-21T14:58:27.194 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4356e-05 2024-09-21T14:58:27.194 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:27.194 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:27.194 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:27.194 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.194 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.194 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:773 3.6 e32/24 3:63a18c8a:::rbd_object_map.113693316f36:head v 32'47, mlcod=32'45)", 2024-09-21T14:58:27.194 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:26.740462+0000", 2024-09-21T14:58:27.194 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.36063881599999997, 2024-09-21T14:58:27.195 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0017884859999999999, 2024-09-21T14:58:27.195 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:27.195 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:27.195 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:27.195 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.195 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:27.195 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.740462+0000", 2024-09-21T14:58:27.195 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:27.195 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.195 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.195 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:27.195 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.740462+0000", 2024-09-21T14:58:27.195 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:27.195 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.196 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.196 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:27.196 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.740461+0000", 2024-09-21T14:58:27.196 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:27.196 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.196 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.196 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:27.196 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.740475+0000", 2024-09-21T14:58:27.196 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4283e-05 2024-09-21T14:58:27.196 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.196 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.196 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:27.196 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.740501+0000", 2024-09-21T14:58:27.196 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5626000000000002e-05 2024-09-21T14:58:27.197 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.197 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.197 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:27.197 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.740512+0000", 2024-09-21T14:58:27.197 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0882e-05 2024-09-21T14:58:27.197 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.197 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.197 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:27.197 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.740624+0000", 2024-09-21T14:58:27.197 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000112448 2024-09-21T14:58:27.197 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.197 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.197 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:27.197 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.740690+0000", 2024-09-21T14:58:27.198 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6476000000000005e-05 2024-09-21T14:58:27.198 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.198 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.198 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:27.198 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.742177+0000", 2024-09-21T14:58:27.198 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0014866790000000001 2024-09-21T14:58:27.198 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:27.198 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:27.198 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:27.198 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.742251+0000", 2024-09-21T14:58:27.198 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3564000000000002e-05 2024-09-21T14:58:27.198 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:27.198 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:27.198 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:27.199 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:27.199 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:27.199 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:58:27.200 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T14:58:27.200 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T14:58:27.200 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-21T14:58:27.200 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T14:58:27.201 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T14:58:27.201 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:58:27.197+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:58:27.201 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T14:58:27.219 INFO:teuthology.orchestra.run.smithi032.stdout:moved 0 / 82 (0%) 2024-09-21T14:58:27.219 INFO:teuthology.orchestra.run.smithi032.stdout:avg 10.25 2024-09-21T14:58:27.219 INFO:teuthology.orchestra.run.smithi032.stdout:stddev 2.4367 -> 2.4367 (expected baseline 2.99479) 2024-09-21T14:58:27.220 INFO:teuthology.orchestra.run.smithi032.stdout:min osd.7 with 6 -> 6 pgs (0.585366 -> 0.585366 * mean) 2024-09-21T14:58:27.220 INFO:teuthology.orchestra.run.smithi032.stdout:max osd.3 with 15 -> 15 pgs (1.46341 -> 1.46341 * mean) 2024-09-21T14:58:27.220 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-21T14:58:27.220 INFO:teuthology.orchestra.run.smithi032.stdout:oload 110 2024-09-21T14:58:27.220 INFO:teuthology.orchestra.run.smithi032.stdout:max_change 1 2024-09-21T14:58:27.220 INFO:teuthology.orchestra.run.smithi032.stdout:max_change_osds 4 2024-09-21T14:58:27.220 INFO:teuthology.orchestra.run.smithi032.stdout:average_utilization 0.0003 2024-09-21T14:58:27.220 INFO:teuthology.orchestra.run.smithi032.stdout:overload_utilization 0.0004 2024-09-21T14:58:27.223 INFO:teuthology.orchestra.run.smithi032.stderr:no change 2024-09-21T14:58:27.223 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T14:58:27.225 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:58:27.225 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:58:27.225 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:58:27.225 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:58:27.225 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:58:27.225 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:58:27.225 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:58:27.226 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:58:27.234 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2024-09-21T14:58:27.234 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 110 1.0 2024-09-21T14:58:27.301 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:58:27.301+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:58:27.313 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:27.313 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:27.324 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T14:58:27.402 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:58:27.396+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:58:27.432 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:27.432 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:27.442 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T14:58:27.466 INFO:teuthology.orchestra.run.smithi032.stderr:nodeep-scrub is unset 2024-09-21T14:58:27.503 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:58:27.496+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:58:27.555 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:27.555 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:27.565 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T14:58:27.602 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-09-21T14:58:27.596+0000 7f94196e2700 -1 mgr.server reply reply (1) Operation not permitted no change 2024-09-21T14:58:27.602 INFO:teuthology.orchestra.run.smithi032.stdout:moved 2 / 82 (2.43902%) 2024-09-21T14:58:27.602 INFO:teuthology.orchestra.run.smithi032.stdout:avg 10.25 2024-09-21T14:58:27.602 INFO:teuthology.orchestra.run.smithi032.stdout:stddev 2.4367 -> 2.90474 (expected baseline 2.99479) 2024-09-21T14:58:27.602 INFO:teuthology.orchestra.run.smithi032.stdout:min osd.7 with 6 -> 4 pgs (0.585366 -> 0.390244 * mean) 2024-09-21T14:58:27.602 INFO:teuthology.orchestra.run.smithi032.stdout:max osd.3 with 15 -> 15 pgs (1.46341 -> 1.46341 * mean) 2024-09-21T14:58:27.603 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-21T14:58:27.603 INFO:teuthology.orchestra.run.smithi032.stdout:oload 110 2024-09-21T14:58:27.603 INFO:teuthology.orchestra.run.smithi032.stdout:max_change 1 2024-09-21T14:58:27.603 INFO:teuthology.orchestra.run.smithi032.stdout:max_change_osds 4 2024-09-21T14:58:27.603 INFO:teuthology.orchestra.run.smithi032.stdout:average_utilization 0.0003 2024-09-21T14:58:27.603 INFO:teuthology.orchestra.run.smithi032.stdout:overload_utilization 0.0004 2024-09-21T14:58:27.603 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7 weight 1.0000 -> 0.8997 2024-09-21T14:58:27.604 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:58:27.600+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:58:27.605 INFO:teuthology.orchestra.run.smithi032.stderr:no change 2024-09-21T14:58:27.616 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-09-21T14:58:27.616 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 0.05 2024-09-21T14:58:27.705 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:58:27.700+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:58:27.713 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:27.713 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:27.724 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T14:58:27.806 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:58:27.805+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:58:27.828 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:27.828 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:27.838 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T14:58:27.907 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:58:27.900+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:58:27.940 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:27.940 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:27.951 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T14:58:28.007 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:58:28.005+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:58:28.056 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:28.057 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:28.066 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T14:58:28.109 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:58:28.109+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:58:28.173 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:28.174 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:28.183 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T14:58:28.209 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:58:28.204+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:58:28.290 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:28.290 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:28.299 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T14:58:28.311 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:58:28.309+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:58:28.407 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:28.407 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:28.411 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:58:28.408+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:58:28.417 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T14:58:28.473 INFO:teuthology.orchestra.run.smithi032.stdout:moved 0 / 82 (0%) 2024-09-21T14:58:28.473 INFO:teuthology.orchestra.run.smithi032.stdout:avg 10.25 2024-09-21T14:58:28.474 INFO:teuthology.orchestra.run.smithi032.stdout:stddev 2.4367 -> 2.4367 (expected baseline 2.99479) 2024-09-21T14:58:28.474 INFO:teuthology.orchestra.run.smithi032.stdout:min osd.7 with 6 -> 6 pgs (0.585366 -> 0.585366 * mean) 2024-09-21T14:58:28.474 INFO:teuthology.orchestra.run.smithi032.stdout:max osd.3 with 15 -> 15 pgs (1.46341 -> 1.46341 * mean) 2024-09-21T14:58:28.474 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-21T14:58:28.474 INFO:teuthology.orchestra.run.smithi032.stdout:oload 110 2024-09-21T14:58:28.474 INFO:teuthology.orchestra.run.smithi032.stdout:max_change 0.05 2024-09-21T14:58:28.474 INFO:teuthology.orchestra.run.smithi032.stdout:max_change_osds 4 2024-09-21T14:58:28.475 INFO:teuthology.orchestra.run.smithi032.stdout:average_utilization 0.0003 2024-09-21T14:58:28.475 INFO:teuthology.orchestra.run.smithi032.stdout:overload_utilization 0.0004 2024-09-21T14:58:28.475 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7 weight 1.0000 -> 0.9500 2024-09-21T14:58:28.512 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:58:28.508+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:58:28.540 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:28.540 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:28.551 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T14:58:28.613 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:58:28.608+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:58:28.706 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:28.707 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:28.714 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:58:28.708+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:58:28.716 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T14:58:28.815 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:58:28.813+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:58:28.833 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:28.833 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:28.844 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T14:58:28.916 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:58:28.913+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:58:28.957 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:28.957 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:28.967 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T14:58:29.017 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:58:29.012+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:58:29.118 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:58:29.116+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:58:29.122 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:29.122 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:29.132 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T14:58:29.219 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:58:29.217+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:58:29.237 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T14:58:29.238 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:29.238 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:29.248 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T14:58:29.320 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:58:29.317+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:58:29.391 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T14:58:29.402 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:29.402 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:29.410 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T14:58:29.412 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T14:58:29.430 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T14:58:29.454 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T14:58:29.457 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:58:29.457+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:58:29.480 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-21T14:58:29.482 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T14:58:29.505 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T14:58:29.521 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:58:29.516+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:58:29.529 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:58:29.529 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T14:58:29.529 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T14:58:29.530 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T14:58:29.530 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.530 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4209.0:20 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'10, mlcod=17'8)", 2024-09-21T14:58:29.530 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:03.652481+0000", 2024-09-21T14:58:29.530 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 25.87365879, 2024-09-21T14:58:29.530 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0025744560000000001, 2024-09-21T14:58:29.530 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:29.530 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:29.530 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:29.530 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.530 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:29.530 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.652481+0000", 2024-09-21T14:58:29.530 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:29.531 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.531 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.531 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:29.531 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.652481+0000", 2024-09-21T14:58:29.531 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:29.531 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.531 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.531 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:29.531 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.652479+0000", 2024-09-21T14:58:29.531 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T14:58:29.532 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.532 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.532 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:29.532 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.652662+0000", 2024-09-21T14:58:29.532 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000182857 2024-09-21T14:58:29.532 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.532 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.532 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:29.532 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.652694+0000", 2024-09-21T14:58:29.532 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1628999999999998e-05 2024-09-21T14:58:29.533 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.533 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.533 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:29.533 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.652706+0000", 2024-09-21T14:58:29.533 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.274e-05 2024-09-21T14:58:29.533 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.533 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.533 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:29.533 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.652790+0000", 2024-09-21T14:58:29.533 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3500999999999999e-05 2024-09-21T14:58:29.533 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.533 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.533 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:29.534 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.652832+0000", 2024-09-21T14:58:29.534 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1888999999999998e-05 2024-09-21T14:58:29.534 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.534 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.534 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:29.534 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.654949+0000", 2024-09-21T14:58:29.534 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0021170619999999999 2024-09-21T14:58:29.534 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.534 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.534 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:29.534 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.655055+0000", 2024-09-21T14:58:29.534 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010655800000000001 2024-09-21T14:58:29.534 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:29.534 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:29.535 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:29.535 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.535 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.535 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4209.0:32 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'18, mlcod=17'14)", 2024-09-21T14:58:29.535 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:03.671413+0000", 2024-09-21T14:58:29.535 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 25.854726154000002, 2024-09-21T14:58:29.535 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0024865389999999999, 2024-09-21T14:58:29.535 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:29.535 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:29.535 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:29.535 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.535 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:29.535 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.671413+0000", 2024-09-21T14:58:29.535 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:29.535 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.536 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.536 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:29.536 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.671413+0000", 2024-09-21T14:58:29.536 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:29.536 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.536 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.536 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:29.536 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.671412+0000", 2024-09-21T14:58:29.536 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T14:58:29.536 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.536 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.537 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:29.537 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.671510+0000", 2024-09-21T14:58:29.537 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.8160999999999997e-05 2024-09-21T14:58:29.537 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.537 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.537 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:29.537 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.671532+0000", 2024-09-21T14:58:29.537 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2036999999999999e-05 2024-09-21T14:58:29.537 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.537 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.537 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:29.537 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.671542+0000", 2024-09-21T14:58:29.537 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0242e-05 2024-09-21T14:58:29.537 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.537 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.538 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:29.538 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.671625+0000", 2024-09-21T14:58:29.538 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3404000000000003e-05 2024-09-21T14:58:29.538 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.538 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.538 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:29.538 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.671665+0000", 2024-09-21T14:58:29.538 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9016e-05 2024-09-21T14:58:29.538 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.538 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.538 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:29.538 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.673813+0000", 2024-09-21T14:58:29.538 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0021483439999999999 2024-09-21T14:58:29.538 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.539 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.539 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:29.539 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.673900+0000", 2024-09-21T14:58:29.539 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7144000000000005e-05 2024-09-21T14:58:29.539 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:29.539 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:29.539 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:29.539 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.539 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.539 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4209.0:68 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'42, mlcod=17'39)", 2024-09-21T14:58:29.539 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:03.727004+0000", 2024-09-21T14:58:29.539 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 25.799135814, 2024-09-21T14:58:29.539 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.002609428, 2024-09-21T14:58:29.539 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:29.540 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:29.540 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:29.540 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.540 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:29.540 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.727004+0000", 2024-09-21T14:58:29.540 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:29.540 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.540 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.540 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:29.540 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.727004+0000", 2024-09-21T14:58:29.540 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:29.540 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.540 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.540 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:29.540 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.727002+0000", 2024-09-21T14:58:29.541 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:29.541 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.541 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.541 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:29.541 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.727018+0000", 2024-09-21T14:58:29.541 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5531000000000001e-05 2024-09-21T14:58:29.541 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.541 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.541 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:29.541 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.727049+0000", 2024-09-21T14:58:29.541 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1541999999999997e-05 2024-09-21T14:58:29.541 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.541 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.541 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:29.542 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.727064+0000", 2024-09-21T14:58:29.542 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4606e-05 2024-09-21T14:58:29.542 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.542 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.542 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:29.542 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.727133+0000", 2024-09-21T14:58:29.542 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8986999999999995e-05 2024-09-21T14:58:29.542 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.542 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.542 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:29.542 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.727158+0000", 2024-09-21T14:58:29.542 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5048000000000001e-05 2024-09-21T14:58:29.542 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.542 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.543 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:29.543 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.729528+0000", 2024-09-21T14:58:29.543 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0023699530000000002 2024-09-21T14:58:29.543 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.543 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.543 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:29.543 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.729613+0000", 2024-09-21T14:58:29.543 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5237999999999999e-05 2024-09-21T14:58:29.543 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:29.543 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:29.543 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:29.543 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.543 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.543 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4209.0:83 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'53, mlcod=17'50)", 2024-09-21T14:58:29.544 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:03.751652+0000", 2024-09-21T14:58:29.544 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 25.774487674, 2024-09-21T14:58:29.544 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0023993119999999998, 2024-09-21T14:58:29.544 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:29.544 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:29.544 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:29.544 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.544 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:29.544 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.751652+0000", 2024-09-21T14:58:29.544 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:29.544 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.544 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.544 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:29.544 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.751652+0000", 2024-09-21T14:58:29.544 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:29.545 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.545 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.545 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:29.545 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.751650+0000", 2024-09-21T14:58:29.545 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T14:58:29.545 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.545 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.545 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:29.545 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.751669+0000", 2024-09-21T14:58:29.545 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9236e-05 2024-09-21T14:58:29.545 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.545 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.545 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:29.545 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.751708+0000", 2024-09-21T14:58:29.546 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8343000000000002e-05 2024-09-21T14:58:29.546 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.546 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.546 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:29.546 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.751732+0000", 2024-09-21T14:58:29.546 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4736000000000002e-05 2024-09-21T14:58:29.546 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.546 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.546 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:29.546 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.751818+0000", 2024-09-21T14:58:29.546 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5493000000000003e-05 2024-09-21T14:58:29.546 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.546 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.546 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:29.547 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.751851+0000", 2024-09-21T14:58:29.547 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3167999999999998e-05 2024-09-21T14:58:29.547 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.547 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.547 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:29.547 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.753977+0000", 2024-09-21T14:58:29.547 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0021262939999999999 2024-09-21T14:58:29.547 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.547 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.547 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:29.547 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.754051+0000", 2024-09-21T14:58:29.547 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3892000000000003e-05 2024-09-21T14:58:29.547 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:29.547 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:29.548 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:29.548 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.548 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.548 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4209.0:104 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'66, mlcod=17'64)", 2024-09-21T14:58:29.548 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:03.956537+0000", 2024-09-21T14:58:29.548 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 25.569602193000001, 2024-09-21T14:58:29.548 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0028947130000000001, 2024-09-21T14:58:29.548 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:29.548 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:29.548 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:29.548 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.548 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:29.548 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.956537+0000", 2024-09-21T14:58:29.549 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:29.549 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.549 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.549 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:29.549 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.956537+0000", 2024-09-21T14:58:29.549 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:29.549 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.549 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.549 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:29.549 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.956536+0000", 2024-09-21T14:58:29.549 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T14:58:29.549 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.549 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.549 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:29.550 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.956550+0000", 2024-09-21T14:58:29.550 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4122e-05 2024-09-21T14:58:29.550 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.550 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.550 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:29.550 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.956581+0000", 2024-09-21T14:58:29.550 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1643000000000002e-05 2024-09-21T14:58:29.550 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.550 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.550 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:29.550 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.956594+0000", 2024-09-21T14:58:29.550 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2500000000000001e-05 2024-09-21T14:58:29.550 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.550 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.551 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:29.551 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.956689+0000", 2024-09-21T14:58:29.551 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5180999999999996e-05 2024-09-21T14:58:29.551 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.551 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.551 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:29.551 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.956727+0000", 2024-09-21T14:58:29.551 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8009999999999997e-05 2024-09-21T14:58:29.551 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.551 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.551 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:29.551 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.959348+0000", 2024-09-21T14:58:29.551 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0026211540000000001 2024-09-21T14:58:29.551 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.552 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.552 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:29.552 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.959432+0000", 2024-09-21T14:58:29.552 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3849000000000003e-05 2024-09-21T14:58:29.552 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:29.552 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:29.552 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:29.552 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.552 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.552 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4209.0:105 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'67, mlcod=17'64)", 2024-09-21T14:58:29.552 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:03.957110+0000", 2024-09-21T14:58:29.552 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 25.569029145000002, 2024-09-21T14:58:29.552 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0031464470000000001, 2024-09-21T14:58:29.552 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:29.553 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:29.553 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:29.553 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.553 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:29.553 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.957110+0000", 2024-09-21T14:58:29.553 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:29.553 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.553 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.553 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:29.553 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.957110+0000", 2024-09-21T14:58:29.553 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:29.553 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.553 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.553 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:29.553 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.957109+0000", 2024-09-21T14:58:29.554 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T14:58:29.554 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.554 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.554 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:29.554 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.957124+0000", 2024-09-21T14:58:29.554 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5639000000000001e-05 2024-09-21T14:58:29.554 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.554 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.554 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:29.554 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.957159+0000", 2024-09-21T14:58:29.554 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4530000000000003e-05 2024-09-21T14:58:29.554 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.554 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.554 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:29.554 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.957173+0000", 2024-09-21T14:58:29.555 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3869000000000001e-05 2024-09-21T14:58:29.555 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.555 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.555 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:29.555 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.957287+0000", 2024-09-21T14:58:29.555 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011453199999999999 2024-09-21T14:58:29.555 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.555 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.555 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:29.555 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.957358+0000", 2024-09-21T14:58:29.555 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0401000000000006e-05 2024-09-21T14:58:29.555 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.555 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.555 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:29.556 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.960177+0000", 2024-09-21T14:58:29.556 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0028189140000000001 2024-09-21T14:58:29.556 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.556 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.556 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:29.556 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.960257+0000", 2024-09-21T14:58:29.556 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0282000000000001e-05 2024-09-21T14:58:29.556 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:29.556 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:29.556 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:29.556 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.556 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.556 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4209.0:106 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'68, mlcod=17'64)", 2024-09-21T14:58:29.556 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:03.957977+0000", 2024-09-21T14:58:29.557 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 25.568162507, 2024-09-21T14:58:29.557 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0028674130000000001, 2024-09-21T14:58:29.557 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:29.557 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:29.557 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:29.557 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.557 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:29.557 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.957977+0000", 2024-09-21T14:58:29.557 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:29.557 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.557 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.557 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:29.557 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.957977+0000", 2024-09-21T14:58:29.557 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:29.557 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.558 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.558 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:29.558 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.957975+0000", 2024-09-21T14:58:29.558 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T14:58:29.558 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.558 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.558 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:29.558 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.958027+0000", 2024-09-21T14:58:29.558 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1925999999999997e-05 2024-09-21T14:58:29.558 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.558 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.558 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:29.558 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.958077+0000", 2024-09-21T14:58:29.559 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9289000000000002e-05 2024-09-21T14:58:29.559 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.559 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.559 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:29.559 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.958089+0000", 2024-09-21T14:58:29.559 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.276e-05 2024-09-21T14:58:29.559 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.559 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.559 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:29.559 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.958278+0000", 2024-09-21T14:58:29.559 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018880599999999999 2024-09-21T14:58:29.559 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.559 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.559 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:29.559 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.958318+0000", 2024-09-21T14:58:29.560 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9388999999999998e-05 2024-09-21T14:58:29.560 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.560 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.560 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:29.560 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.960790+0000", 2024-09-21T14:58:29.560 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0024728720000000001 2024-09-21T14:58:29.560 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.560 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.560 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:29.560 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.960845+0000", 2024-09-21T14:58:29.560 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4098000000000003e-05 2024-09-21T14:58:29.560 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:29.560 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:29.560 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:29.561 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.561 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.561 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4209.0:107 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'69, mlcod=17'64)", 2024-09-21T14:58:29.561 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:03.959170+0000", 2024-09-21T14:58:29.561 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 25.566969498999999, 2024-09-21T14:58:29.561 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0030848759999999999, 2024-09-21T14:58:29.561 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:29.561 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:29.561 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:29.561 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.561 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:29.561 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.959170+0000", 2024-09-21T14:58:29.561 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:29.561 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.561 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.562 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:29.562 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.959170+0000", 2024-09-21T14:58:29.562 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:29.562 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.562 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.562 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:29.562 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.959168+0000", 2024-09-21T14:58:29.562 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T14:58:29.562 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.562 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.562 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:29.562 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.959186+0000", 2024-09-21T14:58:29.562 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7294000000000001e-05 2024-09-21T14:58:29.562 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.563 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.563 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:29.563 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.959221+0000", 2024-09-21T14:58:29.563 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4925000000000002e-05 2024-09-21T14:58:29.563 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.563 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.563 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:29.563 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.959238+0000", 2024-09-21T14:58:29.563 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7079999999999999e-05 2024-09-21T14:58:29.563 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.563 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.563 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:29.563 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.959474+0000", 2024-09-21T14:58:29.563 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00023679199999999999 2024-09-21T14:58:29.564 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.564 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.564 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:29.564 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.959508+0000", 2024-09-21T14:58:29.564 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3195999999999999e-05 2024-09-21T14:58:29.564 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.564 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.564 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:29.564 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.962179+0000", 2024-09-21T14:58:29.564 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0026716700000000001 2024-09-21T14:58:29.564 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.564 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.564 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:29.564 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.962255+0000", 2024-09-21T14:58:29.565 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5739000000000001e-05 2024-09-21T14:58:29.565 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:29.565 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:29.565 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:29.565 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.565 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.565 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4209.0:108 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'70, mlcod=17'64)", 2024-09-21T14:58:29.565 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:03.959894+0000", 2024-09-21T14:58:29.565 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 25.566245649999999, 2024-09-21T14:58:29.565 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0026229949999999999, 2024-09-21T14:58:29.565 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:29.565 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:29.565 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:29.566 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.566 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:29.566 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.959894+0000", 2024-09-21T14:58:29.566 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:29.566 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.566 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.566 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:29.566 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.959894+0000", 2024-09-21T14:58:29.566 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:29.566 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.566 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.566 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:29.566 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.959892+0000", 2024-09-21T14:58:29.566 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T14:58:29.567 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.567 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.567 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:29.567 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.959910+0000", 2024-09-21T14:58:29.567 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8423e-05 2024-09-21T14:58:29.567 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.567 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.567 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:29.567 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.959959+0000", 2024-09-21T14:58:29.567 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8785999999999997e-05 2024-09-21T14:58:29.567 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.567 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.567 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:29.567 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.959998+0000", 2024-09-21T14:58:29.567 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8562000000000001e-05 2024-09-21T14:58:29.568 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.568 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.568 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:29.568 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.960325+0000", 2024-09-21T14:58:29.568 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00032692000000000002 2024-09-21T14:58:29.568 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.568 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.568 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:29.568 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.960351+0000", 2024-09-21T14:58:29.568 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6746000000000001e-05 2024-09-21T14:58:29.568 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.568 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.568 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:29.568 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.962399+0000", 2024-09-21T14:58:29.569 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0020471640000000002 2024-09-21T14:58:29.569 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.569 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.569 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:29.569 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.962517+0000", 2024-09-21T14:58:29.569 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000118404 2024-09-21T14:58:29.569 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:29.569 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:29.569 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:29.569 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.569 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.569 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4209.0:109 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'71, mlcod=17'64)", 2024-09-21T14:58:29.569 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:03.960745+0000", 2024-09-21T14:58:29.569 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 25.565394672, 2024-09-21T14:58:29.569 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0033225070000000002, 2024-09-21T14:58:29.570 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:29.570 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:29.570 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:29.570 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.570 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:29.570 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.960745+0000", 2024-09-21T14:58:29.570 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:29.570 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.570 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.570 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:29.570 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.960745+0000", 2024-09-21T14:58:29.570 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:29.570 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.570 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.571 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:29.571 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.960743+0000", 2024-09-21T14:58:29.571 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:29.571 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.571 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.571 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:29.571 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.960798+0000", 2024-09-21T14:58:29.571 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5186000000000003e-05 2024-09-21T14:58:29.571 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.571 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.571 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:29.571 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.960821+0000", 2024-09-21T14:58:29.571 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2353000000000001e-05 2024-09-21T14:58:29.571 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.572 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.572 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:29.572 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.960832+0000", 2024-09-21T14:58:29.572 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1473e-05 2024-09-21T14:58:29.572 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.572 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.572 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:29.572 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.960888+0000", 2024-09-21T14:58:29.572 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5964999999999997e-05 2024-09-21T14:58:29.572 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.572 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.572 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:29.572 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.960917+0000", 2024-09-21T14:58:29.572 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9090000000000001e-05 2024-09-21T14:58:29.573 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.573 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.573 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:29.573 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.963979+0000", 2024-09-21T14:58:29.573 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0030617650000000001 2024-09-21T14:58:29.573 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.573 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.573 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:29.573 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.964067+0000", 2024-09-21T14:58:29.573 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8337999999999993e-05 2024-09-21T14:58:29.573 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:29.573 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:29.573 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:29.573 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.574 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.574 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4209.0:110 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'72, mlcod=17'67)", 2024-09-21T14:58:29.574 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:03.962329+0000", 2024-09-21T14:58:29.574 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 25.563810225000001, 2024-09-21T14:58:29.574 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0034337909999999998, 2024-09-21T14:58:29.574 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:29.574 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:29.574 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:29.574 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.574 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:29.574 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.962329+0000", 2024-09-21T14:58:29.574 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:29.574 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.574 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.574 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:29.575 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.962329+0000", 2024-09-21T14:58:29.575 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:29.575 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.575 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.575 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:29.575 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.962328+0000", 2024-09-21T14:58:29.575 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:29.575 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.575 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.575 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:29.575 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.962345+0000", 2024-09-21T14:58:29.575 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7479999999999999e-05 2024-09-21T14:58:29.575 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.575 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.576 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:29.576 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.962371+0000", 2024-09-21T14:58:29.576 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5252000000000001e-05 2024-09-21T14:58:29.576 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.576 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.576 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:29.576 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.962393+0000", 2024-09-21T14:58:29.576 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2174000000000001e-05 2024-09-21T14:58:29.576 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.576 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.576 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:29.576 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.962571+0000", 2024-09-21T14:58:29.576 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000177749 2024-09-21T14:58:29.576 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.576 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.577 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:29.577 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.962602+0000", 2024-09-21T14:58:29.577 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1776000000000002e-05 2024-09-21T14:58:29.577 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.577 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.577 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:29.577 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.965690+0000", 2024-09-21T14:58:29.577 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0030873850000000002 2024-09-21T14:58:29.577 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.577 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.577 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:29.577 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.965763+0000", 2024-09-21T14:58:29.577 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3487999999999998e-05 2024-09-21T14:58:29.577 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:29.578 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:29.578 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:29.578 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.578 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.578 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4209.0:111 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'73, mlcod=17'67)", 2024-09-21T14:58:29.578 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:03.962973+0000", 2024-09-21T14:58:29.578 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 25.563166781, 2024-09-21T14:58:29.578 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0028287170000000001, 2024-09-21T14:58:29.578 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:29.578 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:29.578 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:29.578 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.578 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:29.578 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.962973+0000", 2024-09-21T14:58:29.579 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:29.579 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.579 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.579 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:29.579 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.962973+0000", 2024-09-21T14:58:29.579 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:29.579 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.579 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.579 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:29.579 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.962971+0000", 2024-09-21T14:58:29.579 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T14:58:29.579 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.579 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.579 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:29.579 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.962983+0000", 2024-09-21T14:58:29.580 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2353e-05 2024-09-21T14:58:29.580 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.580 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.580 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:29.580 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.963008+0000", 2024-09-21T14:58:29.580 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4626000000000001e-05 2024-09-21T14:58:29.580 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.580 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.580 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:29.580 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.963019+0000", 2024-09-21T14:58:29.580 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1343e-05 2024-09-21T14:58:29.580 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.580 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.580 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:29.581 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.963179+0000", 2024-09-21T14:58:29.581 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015977 2024-09-21T14:58:29.581 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.581 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.581 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:29.581 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.963213+0000", 2024-09-21T14:58:29.581 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4088999999999998e-05 2024-09-21T14:58:29.581 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.581 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.581 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:29.581 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.965766+0000", 2024-09-21T14:58:29.581 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0025524699999999998 2024-09-21T14:58:29.581 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.581 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.581 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:29.582 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.965802+0000", 2024-09-21T14:58:29.582 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.5782999999999999e-05 2024-09-21T14:58:29.582 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:29.582 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:29.582 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:29.582 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.582 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.582 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4209.0:112 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'74, mlcod=17'69)", 2024-09-21T14:58:29.582 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:03.964193+0000", 2024-09-21T14:58:29.582 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 25.561946280000001, 2024-09-21T14:58:29.582 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0025785349999999999, 2024-09-21T14:58:29.582 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:29.582 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:29.582 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:29.583 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.583 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:29.583 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.964193+0000", 2024-09-21T14:58:29.583 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:29.583 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.583 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.583 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:29.583 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.964193+0000", 2024-09-21T14:58:29.583 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:29.583 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.583 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.583 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:29.583 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.964191+0000", 2024-09-21T14:58:29.583 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-21T14:58:29.583 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.584 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.584 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:29.584 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.964303+0000", 2024-09-21T14:58:29.584 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011144700000000001 2024-09-21T14:58:29.584 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.584 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.584 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:29.584 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.964327+0000", 2024-09-21T14:58:29.584 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4576999999999999e-05 2024-09-21T14:58:29.584 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.584 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.584 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:29.584 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.964344+0000", 2024-09-21T14:58:29.584 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6826000000000001e-05 2024-09-21T14:58:29.585 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.585 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.585 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:29.585 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.964559+0000", 2024-09-21T14:58:29.585 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021451200000000001 2024-09-21T14:58:29.585 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.585 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.585 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:29.585 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.964597+0000", 2024-09-21T14:58:29.585 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8266000000000003e-05 2024-09-21T14:58:29.585 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.585 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.585 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:29.585 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.966654+0000", 2024-09-21T14:58:29.585 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0020573129999999999 2024-09-21T14:58:29.586 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.586 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.586 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:29.586 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.966772+0000", 2024-09-21T14:58:29.586 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000117741 2024-09-21T14:58:29.586 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:29.586 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:29.586 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:29.586 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.586 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.586 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4209.0:116 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'78, mlcod=17'73)", 2024-09-21T14:58:29.586 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:03.969123+0000", 2024-09-21T14:58:29.586 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 25.55701655, 2024-09-21T14:58:29.586 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0032522969999999999, 2024-09-21T14:58:29.587 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:29.587 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:29.587 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:29.587 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.587 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:29.587 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.969123+0000", 2024-09-21T14:58:29.587 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:29.587 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.587 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.587 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:29.587 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.969123+0000", 2024-09-21T14:58:29.587 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:29.587 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.587 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.588 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:29.588 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.969122+0000", 2024-09-21T14:58:29.588 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:29.588 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.588 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.588 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:29.588 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.969132+0000", 2024-09-21T14:58:29.588 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5810000000000003e-06 2024-09-21T14:58:29.588 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.588 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.588 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:29.588 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.969145+0000", 2024-09-21T14:58:29.588 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3029e-05 2024-09-21T14:58:29.588 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.589 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.589 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:29.589 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.969151+0000", 2024-09-21T14:58:29.589 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9279999999999998e-06 2024-09-21T14:58:29.589 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.589 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.589 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:29.589 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.969321+0000", 2024-09-21T14:58:29.589 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017070900000000001 2024-09-21T14:58:29.589 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.589 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.589 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:29.589 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.969390+0000", 2024-09-21T14:58:29.589 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8548999999999997e-05 2024-09-21T14:58:29.589 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.590 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.590 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:29.590 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.972257+0000", 2024-09-21T14:58:29.590 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0028669390000000002 2024-09-21T14:58:29.590 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.590 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.590 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:29.590 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:03.972375+0000", 2024-09-21T14:58:29.590 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011844400000000001 2024-09-21T14:58:29.590 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:29.590 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:29.590 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:29.590 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.590 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.591 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4209.0:130 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'88, mlcod=17'85)", 2024-09-21T14:58:29.591 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:04.496317+0000", 2024-09-21T14:58:29.591 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 25.029823002000001, 2024-09-21T14:58:29.591 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.002431744, 2024-09-21T14:58:29.591 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:29.591 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:29.591 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:29.591 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.591 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:29.591 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:04.496317+0000", 2024-09-21T14:58:29.591 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:29.591 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.591 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.591 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:29.592 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:04.496317+0000", 2024-09-21T14:58:29.592 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:29.592 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.592 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.592 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:29.592 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:04.496315+0000", 2024-09-21T14:58:29.592 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T14:58:29.592 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.592 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.592 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:29.592 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:04.496329+0000", 2024-09-21T14:58:29.592 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.431e-05 2024-09-21T14:58:29.592 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.592 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.592 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:29.593 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:04.496358+0000", 2024-09-21T14:58:29.593 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8436e-05 2024-09-21T14:58:29.593 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.593 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.593 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:29.593 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:04.496370+0000", 2024-09-21T14:58:29.593 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2576e-05 2024-09-21T14:58:29.593 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.593 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.593 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:29.593 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:04.496445+0000", 2024-09-21T14:58:29.593 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4534000000000002e-05 2024-09-21T14:58:29.593 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.593 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.594 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:29.594 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:04.496469+0000", 2024-09-21T14:58:29.594 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4700000000000001e-05 2024-09-21T14:58:29.594 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.594 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.594 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:29.594 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:04.498692+0000", 2024-09-21T14:58:29.594 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0022227530000000001 2024-09-21T14:58:29.594 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.594 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.594 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:29.594 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:04.498748+0000", 2024-09-21T14:58:29.594 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6221999999999998e-05 2024-09-21T14:58:29.594 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:29.594 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:29.595 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:29.595 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.595 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.595 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4209.0:131 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'89, mlcod=17'85)", 2024-09-21T14:58:29.595 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:04.497490+0000", 2024-09-21T14:58:29.595 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 25.028649662999999, 2024-09-21T14:58:29.595 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0024435889999999999, 2024-09-21T14:58:29.595 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:29.595 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:29.595 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:29.595 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.595 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:29.595 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:04.497490+0000", 2024-09-21T14:58:29.595 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:29.596 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.596 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.596 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:29.596 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:04.497490+0000", 2024-09-21T14:58:29.596 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:29.596 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.596 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.596 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:29.596 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:04.497488+0000", 2024-09-21T14:58:29.596 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T14:58:29.596 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.596 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.596 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:29.596 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:04.497580+0000", 2024-09-21T14:58:29.597 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2251999999999997e-05 2024-09-21T14:58:29.597 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.597 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.597 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:29.597 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:04.497614+0000", 2024-09-21T14:58:29.597 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3206e-05 2024-09-21T14:58:29.597 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.597 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.597 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:29.597 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:04.497627+0000", 2024-09-21T14:58:29.597 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3149999999999999e-05 2024-09-21T14:58:29.597 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.597 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.597 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:29.598 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:04.497723+0000", 2024-09-21T14:58:29.598 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6442999999999998e-05 2024-09-21T14:58:29.598 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.598 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.598 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:29.598 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:04.497758+0000", 2024-09-21T14:58:29.598 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.5102999999999999e-05 2024-09-21T14:58:29.598 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.598 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.598 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:29.598 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:04.499870+0000", 2024-09-21T14:58:29.598 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0021113410000000001 2024-09-21T14:58:29.598 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.598 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.599 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:29.599 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:04.499934+0000", 2024-09-21T14:58:29.599 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3887999999999995e-05 2024-09-21T14:58:29.599 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:29.599 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:29.599 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:29.599 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.599 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.599 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4209.0:134 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'92, mlcod=18'87)", 2024-09-21T14:58:29.599 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:04.501035+0000", 2024-09-21T14:58:29.599 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 25.025104921000001, 2024-09-21T14:58:29.599 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0030509510000000001, 2024-09-21T14:58:29.599 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:29.599 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:29.599 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:29.600 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.600 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:29.600 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:04.501035+0000", 2024-09-21T14:58:29.600 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:29.600 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.600 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.600 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:29.600 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:04.501035+0000", 2024-09-21T14:58:29.600 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:29.600 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.600 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.600 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:29.600 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:04.501033+0000", 2024-09-21T14:58:29.600 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:29.600 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.601 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.601 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:29.601 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:04.501115+0000", 2024-09-21T14:58:29.601 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1252000000000001e-05 2024-09-21T14:58:29.601 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.601 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.601 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:29.601 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:04.501131+0000", 2024-09-21T14:58:29.601 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5985999999999999e-05 2024-09-21T14:58:29.601 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.601 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.601 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:29.601 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:04.501143+0000", 2024-09-21T14:58:29.601 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2593e-05 2024-09-21T14:58:29.601 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.602 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.602 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:29.602 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:04.501217+0000", 2024-09-21T14:58:29.602 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3503000000000004e-05 2024-09-21T14:58:29.602 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.602 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.602 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:29.602 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:04.501246+0000", 2024-09-21T14:58:29.602 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8785e-05 2024-09-21T14:58:29.602 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.602 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.602 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:29.602 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:04.503975+0000", 2024-09-21T14:58:29.602 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0027293259999999998 2024-09-21T14:58:29.603 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.603 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.603 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:29.603 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:04.504086+0000", 2024-09-21T14:58:29.603 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000110776 2024-09-21T14:58:29.603 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:29.603 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:29.603 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:29.603 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.603 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.603 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4209.0:136 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'94, mlcod=18'89)", 2024-09-21T14:58:29.603 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:04.503869+0000", 2024-09-21T14:58:29.603 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 25.022270869, 2024-09-21T14:58:29.603 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0024804580000000001, 2024-09-21T14:58:29.603 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:29.604 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:29.604 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:29.604 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.604 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:29.604 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:04.503869+0000", 2024-09-21T14:58:29.604 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:29.604 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.604 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.604 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:29.604 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:04.503869+0000", 2024-09-21T14:58:29.604 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:29.604 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.604 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.604 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:29.605 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:04.503868+0000", 2024-09-21T14:58:29.605 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:29.605 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.605 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.605 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:29.605 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:04.503882+0000", 2024-09-21T14:58:29.605 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4657e-05 2024-09-21T14:58:29.605 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.605 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.605 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:29.605 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:04.503907+0000", 2024-09-21T14:58:29.605 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4751000000000001e-05 2024-09-21T14:58:29.605 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.605 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.606 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:29.606 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:04.503920+0000", 2024-09-21T14:58:29.606 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2660000000000001e-05 2024-09-21T14:58:29.606 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.606 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.606 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:29.606 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:04.504127+0000", 2024-09-21T14:58:29.606 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000206885 2024-09-21T14:58:29.606 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.606 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.606 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:29.606 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:04.504163+0000", 2024-09-21T14:58:29.606 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6035999999999999e-05 2024-09-21T14:58:29.607 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.607 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.607 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:29.607 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:04.506299+0000", 2024-09-21T14:58:29.607 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0021365070000000002 2024-09-21T14:58:29.607 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.607 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.607 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:29.607 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:04.506349+0000", 2024-09-21T14:58:29.607 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9919000000000001e-05 2024-09-21T14:58:29.607 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:29.607 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:29.607 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:29.607 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.607 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.608 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4209.0:155 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'109, mlcod=18'105)", 2024-09-21T14:58:29.608 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:05.314656+0000", 2024-09-21T14:58:29.608 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 24.211483738999998, 2024-09-21T14:58:29.608 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0024543859999999998, 2024-09-21T14:58:29.608 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:29.608 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:29.608 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:29.608 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.608 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:29.608 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:05.314656+0000", 2024-09-21T14:58:29.608 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:29.608 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.608 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.609 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:29.609 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:05.314656+0000", 2024-09-21T14:58:29.609 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:29.609 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.609 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.609 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:29.609 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:05.314655+0000", 2024-09-21T14:58:29.609 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:29.609 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.609 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.609 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:29.609 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:05.314848+0000", 2024-09-21T14:58:29.609 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000192967 2024-09-21T14:58:29.609 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.609 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.610 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:29.610 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:05.314861+0000", 2024-09-21T14:58:29.610 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.365e-05 2024-09-21T14:58:29.610 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.610 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.610 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:29.610 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:05.314870+0000", 2024-09-21T14:58:29.610 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3499999999999997e-06 2024-09-21T14:58:29.610 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.610 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.610 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:29.610 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:05.314943+0000", 2024-09-21T14:58:29.610 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3653999999999996e-05 2024-09-21T14:58:29.611 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.611 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.611 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:29.611 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:05.314964+0000", 2024-09-21T14:58:29.611 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0562000000000001e-05 2024-09-21T14:58:29.611 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.611 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.611 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:29.611 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:05.317035+0000", 2024-09-21T14:58:29.611 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0020709399999999998 2024-09-21T14:58:29.611 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.611 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.611 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:29.611 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:05.317110+0000", 2024-09-21T14:58:29.612 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5468000000000003e-05 2024-09-21T14:58:29.612 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:29.612 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:29.612 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:29.612 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.612 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.612 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:502 3.4 e31/24 3:23331720:::rbd_object_map.113693316f36.0000000000000005:head v 31'4, mlcod=0'0)", 2024-09-21T14:58:29.612 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:25.472005+0000", 2024-09-21T14:58:29.612 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.0541350490000001, 2024-09-21T14:58:29.612 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0024445249999999999, 2024-09-21T14:58:29.612 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:29.612 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:29.612 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:29.612 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.612 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:29.613 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.472005+0000", 2024-09-21T14:58:29.613 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:29.613 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.613 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.613 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:29.613 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.472005+0000", 2024-09-21T14:58:29.613 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:29.613 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.613 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.613 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:29.613 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.472004+0000", 2024-09-21T14:58:29.613 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:29.613 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.613 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.614 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:29.614 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.472015+0000", 2024-09-21T14:58:29.614 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1714e-05 2024-09-21T14:58:29.614 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.614 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.614 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:29.614 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.472026+0000", 2024-09-21T14:58:29.614 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0190000000000001e-05 2024-09-21T14:58:29.614 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.614 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.614 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:29.614 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.472033+0000", 2024-09-21T14:58:29.614 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1779999999999997e-06 2024-09-21T14:58:29.614 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.615 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.615 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:29.615 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.472884+0000", 2024-09-21T14:58:29.615 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00085105700000000001 2024-09-21T14:58:29.615 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.615 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.615 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:29.615 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.472905+0000", 2024-09-21T14:58:29.615 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1243e-05 2024-09-21T14:58:29.615 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.615 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.615 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:29.615 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.474412+0000", 2024-09-21T14:58:29.615 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0015071660000000001 2024-09-21T14:58:29.616 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.616 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.616 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:29.616 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.474449+0000", 2024-09-21T14:58:29.616 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6804999999999997e-05 2024-09-21T14:58:29.616 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:29.616 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:29.616 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:29.616 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:29.616 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:29.616 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:58:29.617 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T14:58:29.617 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T14:58:29.618 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T14:58:29.619 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:58:29.619 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:58:29.619 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:58:29.619 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:58:29.619 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:58:29.619 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:58:29.619 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:58:29.619 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:58:29.623 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:58:29.620+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:58:29.662 INFO:teuthology.orchestra.run.smithi032.stdout:checking clone #0, image image_client.0 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent1 2024-09-21T14:58:29.724 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:58:29.720+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:58:29.733 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:58:29.733 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:58:29.733 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T14:58:29.733 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:58:29.743 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T14:58:29.825 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:58:29.825+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:58:29.850 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:58:29.850 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:58:29.850 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T14:58:29.850 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T14:58:29.850 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:58:29.860 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T14:58:29.926 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:58:29.925+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:58:29.973 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:58:29.973 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T14:58:29.973 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T14:58:29.973 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T14:58:29.973 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.973 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4281.0:5 2.2 e21/19 2:5cea7035:::rbd_info:head v 21'1, mlcod=0'0)", 2024-09-21T14:58:29.974 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:10.343460+0000", 2024-09-21T14:58:29.974 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 19.626330303, 2024-09-21T14:58:29.974 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0024752820000000001, 2024-09-21T14:58:29.974 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:29.974 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:29.974 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:29.975 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.975 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:29.975 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:10.343460+0000", 2024-09-21T14:58:29.975 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:29.975 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.975 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.976 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:29.976 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:10.343460+0000", 2024-09-21T14:58:29.976 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:29.976 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.976 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.976 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:29.976 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:10.343459+0000", 2024-09-21T14:58:29.977 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:29.977 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.977 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.977 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:29.977 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:10.343499+0000", 2024-09-21T14:58:29.977 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9983e-05 2024-09-21T14:58:29.977 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.978 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.978 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:29.978 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:10.343521+0000", 2024-09-21T14:58:29.978 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1858999999999999e-05 2024-09-21T14:58:29.978 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.978 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.979 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:29.979 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:10.343540+0000", 2024-09-21T14:58:29.979 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9426e-05 2024-09-21T14:58:29.979 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.979 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.979 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:29.980 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:10.343647+0000", 2024-09-21T14:58:29.980 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000106612 2024-09-21T14:58:29.980 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.980 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.980 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:29.980 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:10.343723+0000", 2024-09-21T14:58:29.981 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6230999999999996e-05 2024-09-21T14:58:29.981 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.981 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.981 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:29.981 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:10.345869+0000", 2024-09-21T14:58:29.981 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0021460450000000001 2024-09-21T14:58:29.982 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.982 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.982 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:29.982 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:10.345935+0000", 2024-09-21T14:58:29.982 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5776000000000002e-05 2024-09-21T14:58:29.982 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:29.982 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:29.983 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:29.983 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.983 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.983 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4281.0:7 2.2 e22/19 2:5cea7035:::rbd_info:head v 22'2, mlcod=0'0)", 2024-09-21T14:58:29.983 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:11.349430+0000", 2024-09-21T14:58:29.983 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 18.620359667999999, 2024-09-21T14:58:29.984 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0015191499999999999, 2024-09-21T14:58:29.984 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:29.984 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:29.984 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:29.984 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.984 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:29.985 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:11.349430+0000", 2024-09-21T14:58:29.985 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:29.985 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.985 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.985 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:29.985 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:11.349430+0000", 2024-09-21T14:58:29.985 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:29.986 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.986 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.986 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:29.986 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:11.349429+0000", 2024-09-21T14:58:29.986 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:29.986 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.986 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.987 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:29.987 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:11.349450+0000", 2024-09-21T14:58:29.987 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0873000000000002e-05 2024-09-21T14:58:29.987 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.987 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.987 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:29.987 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:11.349471+0000", 2024-09-21T14:58:29.988 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1016e-05 2024-09-21T14:58:29.988 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.988 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.988 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:29.988 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:11.349490+0000", 2024-09-21T14:58:29.988 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8736000000000002e-05 2024-09-21T14:58:29.988 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.988 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.989 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:29.989 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:11.349585+0000", 2024-09-21T14:58:29.989 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5377999999999994e-05 2024-09-21T14:58:29.989 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.989 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.989 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:29.989 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:11.349634+0000", 2024-09-21T14:58:29.989 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9449e-05 2024-09-21T14:58:29.989 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.989 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.989 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:29.989 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:11.350896+0000", 2024-09-21T14:58:29.989 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001261119 2024-09-21T14:58:29.989 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.990 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.990 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:29.990 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:11.350949+0000", 2024-09-21T14:58:29.990 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3749000000000003e-05 2024-09-21T14:58:29.990 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:29.990 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:29.990 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:29.990 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.990 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.990 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:13 3.15 e27/24 3:a8a0cd24:::rbd_object_map.11367ffd86fe:head v 27'1, mlcod=0'0)", 2024-09-21T14:58:29.990 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:21.429587+0000", 2024-09-21T14:58:29.990 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 8.5402027090000008, 2024-09-21T14:58:29.990 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0013429080000000001, 2024-09-21T14:58:29.991 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:29.991 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:29.991 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:29.991 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.991 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:29.991 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.429587+0000", 2024-09-21T14:58:29.991 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:29.991 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.991 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.991 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:29.991 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.429587+0000", 2024-09-21T14:58:29.991 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:29.991 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.991 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.991 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:29.992 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.429586+0000", 2024-09-21T14:58:29.992 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:29.992 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.992 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.992 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:29.992 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.429595+0000", 2024-09-21T14:58:29.992 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9819999999999997e-06 2024-09-21T14:58:29.992 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.992 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.992 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:29.992 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.429613+0000", 2024-09-21T14:58:29.992 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7158000000000001e-05 2024-09-21T14:58:29.992 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.992 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.993 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:29.993 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.429626+0000", 2024-09-21T14:58:29.993 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3538e-05 2024-09-21T14:58:29.993 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.993 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.993 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:29.993 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.429731+0000", 2024-09-21T14:58:29.993 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000104351 2024-09-21T14:58:29.993 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.993 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.993 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:29.993 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.429758+0000", 2024-09-21T14:58:29.993 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7611999999999999e-05 2024-09-21T14:58:29.994 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.994 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.994 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:29.994 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.430833+0000", 2024-09-21T14:58:29.994 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001074817 2024-09-21T14:58:29.994 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.994 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.994 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:29.994 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.430930+0000", 2024-09-21T14:58:29.994 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7094000000000002e-05 2024-09-21T14:58:29.994 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:29.994 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:29.994 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:29.995 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.995 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.995 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:324 3.4 3:23331720:::rbd_object_map.113693316f36.0000000000000005:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T14:58:29.995 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:25.450424+0000", 2024-09-21T14:58:29.995 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.5193658890000004, 2024-09-21T14:58:29.995 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0045158519999999999, 2024-09-21T14:58:29.995 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:29.995 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:29.995 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:29.995 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:29.995 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:29.995 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 324 2024-09-21T14:58:29.996 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.996 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:29.996 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.996 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:29.996 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.450424+0000", 2024-09-21T14:58:29.996 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:29.996 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.996 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.996 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:29.996 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.450424+0000", 2024-09-21T14:58:29.996 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:29.996 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.996 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.996 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:29.996 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.450422+0000", 2024-09-21T14:58:29.997 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-21T14:58:29.997 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.997 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.997 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:29.997 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.450436+0000", 2024-09-21T14:58:29.997 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4663e-05 2024-09-21T14:58:29.997 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.997 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.997 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:29.997 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.450469+0000", 2024-09-21T14:58:29.997 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2981000000000002e-05 2024-09-21T14:58:29.997 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.997 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.997 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:29.997 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.450482+0000", 2024-09-21T14:58:29.997 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.305e-05 2024-09-21T14:58:29.998 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.998 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.998 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:29.998 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.450544+0000", 2024-09-21T14:58:29.998 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2260000000000004e-05 2024-09-21T14:58:29.998 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.998 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.998 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op must wait for map", 2024-09-21T14:58:29.998 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.450549+0000", 2024-09-21T14:58:29.998 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7870000000000001e-06 2024-09-21T14:58:29.998 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.998 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.998 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:29.998 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.453041+0000", 2024-09-21T14:58:29.999 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0024921639999999998 2024-09-21T14:58:29.999 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.999 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.999 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:29.999 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.453157+0000", 2024-09-21T14:58:29.999 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000115419 2024-09-21T14:58:29.999 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.999 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.999 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:29.999 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.453289+0000", 2024-09-21T14:58:29.999 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013237200000000001 2024-09-21T14:58:29.999 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:29.999 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:29.999 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:58:30.000 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.454356+0000", 2024-09-21T14:58:30.000 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0010672139999999999 2024-09-21T14:58:30.000 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.000 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.000 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:30.000 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.454815+0000", 2024-09-21T14:58:30.000 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00045882200000000001 2024-09-21T14:58:30.000 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.000 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.000 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:30.000 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.454891+0000", 2024-09-21T14:58:30.000 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6070999999999997e-05 2024-09-21T14:58:30.000 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.000 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.000 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:30.001 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.454940+0000", 2024-09-21T14:58:30.001 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8466e-05 2024-09-21T14:58:30.001 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:30.001 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:30.001 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:30.001 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.001 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.001 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:400 3.12 3:4ef17d39:::rbd_data.11367ffd86fe.0000000000000014:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T14:58:30.001 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:25.462953+0000", 2024-09-21T14:58:30.001 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.5068370099999999, 2024-09-21T14:58:30.001 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00060437299999999995, 2024-09-21T14:58:30.001 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:30.001 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:58:30.001 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:30.001 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:30.002 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:30.002 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 400 2024-09-21T14:58:30.002 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.002 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:30.002 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.002 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:30.002 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.462953+0000", 2024-09-21T14:58:30.002 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:30.002 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.002 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.003 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:30.003 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.462953+0000", 2024-09-21T14:58:30.003 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:30.003 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.003 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.003 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:30.003 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.462952+0000", 2024-09-21T14:58:30.003 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:30.003 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.003 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.003 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:30.003 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.462957+0000", 2024-09-21T14:58:30.003 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0969999999999996e-06 2024-09-21T14:58:30.004 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.004 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.004 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:30.004 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.462974+0000", 2024-09-21T14:58:30.004 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7068e-05 2024-09-21T14:58:30.004 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.004 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.004 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:30.004 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.462984+0000", 2024-09-21T14:58:30.004 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0254e-05 2024-09-21T14:58:30.004 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.004 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.004 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:30.004 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.463418+0000", 2024-09-21T14:58:30.005 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00043319799999999998 2024-09-21T14:58:30.005 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.005 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.005 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:30.005 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.463557+0000", 2024-09-21T14:58:30.005 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000139553 2024-09-21T14:58:30.005 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:30.005 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:30.005 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:30.005 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.005 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.005 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:406 3.12 3:4b779880:::rbd_data.11367ffd86fe.0000000000000025:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T14:58:30.005 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:25.462994+0000", 2024-09-21T14:58:30.005 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.5067961360000002, 2024-09-21T14:58:30.006 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00082261599999999997, 2024-09-21T14:58:30.006 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:30.006 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:58:30.006 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:30.006 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:30.006 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:30.006 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 406 2024-09-21T14:58:30.006 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.006 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:30.006 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.006 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:30.007 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.462994+0000", 2024-09-21T14:58:30.007 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:30.007 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.007 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.007 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:30.007 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.462994+0000", 2024-09-21T14:58:30.007 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:30.007 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.007 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.007 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:30.007 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.462993+0000", 2024-09-21T14:58:30.007 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:30.007 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.007 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.007 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:30.008 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.462998+0000", 2024-09-21T14:58:30.008 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3970000000000002e-06 2024-09-21T14:58:30.008 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.008 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.008 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:30.008 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.463013+0000", 2024-09-21T14:58:30.008 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4572999999999999e-05 2024-09-21T14:58:30.008 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.008 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.008 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:30.008 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.463023+0000", 2024-09-21T14:58:30.008 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9070000000000002e-06 2024-09-21T14:58:30.008 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.008 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.008 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:30.009 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.463612+0000", 2024-09-21T14:58:30.009 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00058929900000000005 2024-09-21T14:58:30.009 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.009 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.009 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:30.009 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.463816+0000", 2024-09-21T14:58:30.009 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00020416999999999999 2024-09-21T14:58:30.009 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:30.009 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:30.009 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:30.009 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.009 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.009 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:438 3.12 3:4b779880:::rbd_data.11367ffd86fe.0000000000000025:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T14:58:30.009 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:25.466153+0000", 2024-09-21T14:58:30.010 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.5036364899999999, 2024-09-21T14:58:30.010 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00064124600000000005, 2024-09-21T14:58:30.010 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:30.010 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:58:30.010 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:30.010 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:30.010 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:30.010 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 438 2024-09-21T14:58:30.010 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.010 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:30.011 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.011 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:30.011 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.466153+0000", 2024-09-21T14:58:30.011 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:30.011 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.011 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.011 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:30.011 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.466153+0000", 2024-09-21T14:58:30.011 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:30.011 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.011 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.011 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:30.011 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.466153+0000", 2024-09-21T14:58:30.012 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:30.012 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.012 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.012 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:30.012 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.466157+0000", 2024-09-21T14:58:30.012 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.352e-06 2024-09-21T14:58:30.012 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.012 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.012 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:30.012 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.466167+0000", 2024-09-21T14:58:30.012 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9839999999999996e-06 2024-09-21T14:58:30.012 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.012 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.012 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:30.013 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.466175+0000", 2024-09-21T14:58:30.013 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1000000000000004e-06 2024-09-21T14:58:30.013 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.013 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.013 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:30.013 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.466665+0000", 2024-09-21T14:58:30.013 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00048993300000000001 2024-09-21T14:58:30.013 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.013 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.013 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:30.013 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.466795+0000", 2024-09-21T14:58:30.013 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000129663 2024-09-21T14:58:30.013 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:30.013 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:30.014 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:30.014 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.014 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.014 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:499 3.4 3:23331720:::rbd_object_map.113693316f36.0000000000000005:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T14:58:30.014 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:25.469014+0000", 2024-09-21T14:58:30.014 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.5007763269999996, 2024-09-21T14:58:30.014 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0061707929999999999, 2024-09-21T14:58:30.014 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:30.014 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:30.014 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:30.014 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:30.014 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:30.014 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 499 2024-09-21T14:58:30.015 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.015 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:30.015 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.015 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:30.015 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.469014+0000", 2024-09-21T14:58:30.015 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:30.015 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.015 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.015 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:30.015 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.469014+0000", 2024-09-21T14:58:30.015 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:30.015 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.015 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.015 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:30.015 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.469012+0000", 2024-09-21T14:58:30.016 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:30.016 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.016 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.016 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:30.016 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.469021+0000", 2024-09-21T14:58:30.016 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7399999999999993e-06 2024-09-21T14:58:30.016 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.016 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.016 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:30.016 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.469042+0000", 2024-09-21T14:58:30.016 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1687000000000001e-05 2024-09-21T14:58:30.016 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.016 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.016 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:30.017 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.469066+0000", 2024-09-21T14:58:30.017 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3635999999999999e-05 2024-09-21T14:58:30.017 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.017 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.017 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:30.017 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.469120+0000", 2024-09-21T14:58:30.017 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3811999999999999e-05 2024-09-21T14:58:30.017 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.017 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.017 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:30.017 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.469248+0000", 2024-09-21T14:58:30.017 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000128501 2024-09-21T14:58:30.017 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.017 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.018 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:30.018 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.469850+0000", 2024-09-21T14:58:30.018 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00060168099999999996 2024-09-21T14:58:30.018 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.018 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.018 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:30.018 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.472543+0000", 2024-09-21T14:58:30.018 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0026930130000000002 2024-09-21T14:58:30.018 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.018 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.018 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:58:30.018 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.475085+0000", 2024-09-21T14:58:30.018 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0025416359999999999 2024-09-21T14:58:30.019 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.019 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.019 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:30.019 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.475140+0000", 2024-09-21T14:58:30.019 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5238999999999997e-05 2024-09-21T14:58:30.019 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.019 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.019 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:30.019 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.475184+0000", 2024-09-21T14:58:30.019 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4459000000000002e-05 2024-09-21T14:58:30.019 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:30.019 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:30.019 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:30.019 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.020 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.020 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:500 3.4 3:23331720:::rbd_object_map.113693316f36.0000000000000005:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T14:58:30.020 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:25.469105+0000", 2024-09-21T14:58:30.020 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.5006845640000002, 2024-09-21T14:58:30.020 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0061870459999999999, 2024-09-21T14:58:30.020 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:30.020 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:30.020 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:30.020 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:30.020 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:30.020 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 500 2024-09-21T14:58:30.020 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.020 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:30.020 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.021 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:30.021 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.469105+0000", 2024-09-21T14:58:30.021 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:30.021 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.021 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.021 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:30.021 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.469105+0000", 2024-09-21T14:58:30.021 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:30.021 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.021 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.021 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:30.021 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.469104+0000", 2024-09-21T14:58:30.021 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:30.021 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.022 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.022 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:30.022 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.469122+0000", 2024-09-21T14:58:30.022 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7655e-05 2024-09-21T14:58:30.022 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.022 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.022 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:30.022 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.469133+0000", 2024-09-21T14:58:30.022 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1422e-05 2024-09-21T14:58:30.022 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.022 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.022 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:30.022 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.469143+0000", 2024-09-21T14:58:30.022 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.8199999999999992e-06 2024-09-21T14:58:30.023 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.023 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.023 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:30.023 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.470410+0000", 2024-09-21T14:58:30.023 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001266959 2024-09-21T14:58:30.023 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.023 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.023 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:30.023 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.470511+0000", 2024-09-21T14:58:30.023 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000100721 2024-09-21T14:58:30.023 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.023 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.023 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:30.024 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.470907+0000", 2024-09-21T14:58:30.024 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00039660200000000002 2024-09-21T14:58:30.024 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.024 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.024 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:30.024 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.475041+0000", 2024-09-21T14:58:30.024 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0041341490000000002 2024-09-21T14:58:30.024 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.024 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.024 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:58:30.024 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.475202+0000", 2024-09-21T14:58:30.024 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000160132 2024-09-21T14:58:30.024 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.025 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.025 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:30.025 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.475249+0000", 2024-09-21T14:58:30.025 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7341e-05 2024-09-21T14:58:30.025 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.025 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.025 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:30.025 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.475292+0000", 2024-09-21T14:58:30.025 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3399000000000002e-05 2024-09-21T14:58:30.025 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:30.025 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:30.025 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:30.025 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.025 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.025 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:502 3.4 3:23331720:::rbd_object_map.113693316f36.0000000000000005:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T14:58:30.026 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:25.469155+0000", 2024-09-21T14:58:30.026 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.5006345149999998, 2024-09-21T14:58:30.026 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0062263259999999999, 2024-09-21T14:58:30.026 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:30.026 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:30.026 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:30.026 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:30.026 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:30.026 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 502 2024-09-21T14:58:30.026 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.026 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:30.026 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.027 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:30.027 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.469155+0000", 2024-09-21T14:58:30.027 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:30.027 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.027 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.027 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:30.027 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.469155+0000", 2024-09-21T14:58:30.027 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:30.027 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.027 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.027 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:30.027 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.469155+0000", 2024-09-21T14:58:30.027 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:30.027 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.028 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.028 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:30.028 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.469159+0000", 2024-09-21T14:58:30.028 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8069999999999999e-06 2024-09-21T14:58:30.028 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.028 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.028 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:30.028 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.469168+0000", 2024-09-21T14:58:30.028 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5799999999999992e-06 2024-09-21T14:58:30.028 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.028 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.028 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:30.028 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.469176+0000", 2024-09-21T14:58:30.028 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2679999999999994e-06 2024-09-21T14:58:30.029 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.029 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.029 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:30.029 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.471364+0000", 2024-09-21T14:58:30.029 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0021876930000000001 2024-09-21T14:58:30.029 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.029 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.029 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:30.029 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.471493+0000", 2024-09-21T14:58:30.029 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000129426 2024-09-21T14:58:30.029 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.029 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.029 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:30.030 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.471898+0000", 2024-09-21T14:58:30.030 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000404254 2024-09-21T14:58:30.030 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.030 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.030 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:30.030 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.475077+0000", 2024-09-21T14:58:30.030 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0031798880000000001 2024-09-21T14:58:30.030 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.030 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.030 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:58:30.030 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.475300+0000", 2024-09-21T14:58:30.030 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00022288700000000001 2024-09-21T14:58:30.030 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.031 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.031 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:30.031 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.475343+0000", 2024-09-21T14:58:30.031 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3046000000000001e-05 2024-09-21T14:58:30.031 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.031 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.031 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:30.031 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.475382+0000", 2024-09-21T14:58:30.031 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8253000000000001e-05 2024-09-21T14:58:30.031 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:30.031 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:30.031 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:30.031 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.032 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.032 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:512 3.4 3:265e2476:::rbd_data.11367ffd86fe.0000000000000047:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T14:58:30.032 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:25.469342+0000", 2024-09-21T14:58:30.032 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.500447866, 2024-09-21T14:58:30.032 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0031520649999999999, 2024-09-21T14:58:30.032 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:30.032 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:58:30.032 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:30.032 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:30.032 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:30.032 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 512 2024-09-21T14:58:30.032 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.032 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:30.032 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.033 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:30.033 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.469342+0000", 2024-09-21T14:58:30.033 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:30.033 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.033 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.033 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:30.033 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.469342+0000", 2024-09-21T14:58:30.033 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:30.033 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.033 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.033 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:30.033 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.469341+0000", 2024-09-21T14:58:30.033 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:30.033 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.033 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.034 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:30.034 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.469347+0000", 2024-09-21T14:58:30.034 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5019999999999998e-06 2024-09-21T14:58:30.034 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.034 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.034 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:30.034 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.469359+0000", 2024-09-21T14:58:30.034 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1770000000000001e-05 2024-09-21T14:58:30.034 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.034 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.034 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:30.034 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.469367+0000", 2024-09-21T14:58:30.034 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7970000000000001e-06 2024-09-21T14:58:30.035 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.035 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.035 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:30.035 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.472364+0000", 2024-09-21T14:58:30.035 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0029971730000000001 2024-09-21T14:58:30.035 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.035 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.035 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:30.035 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.472494+0000", 2024-09-21T14:58:30.035 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012981000000000001 2024-09-21T14:58:30.035 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:30.035 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:30.035 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:30.035 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.036 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.036 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:518 3.4 3:23331720:::rbd_object_map.113693316f36.0000000000000005:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T14:58:30.036 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:25.469843+0000", 2024-09-21T14:58:30.036 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.4999470749999997, 2024-09-21T14:58:30.036 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0056936850000000004, 2024-09-21T14:58:30.036 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:30.036 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:30.036 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:30.036 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:30.036 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:30.036 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 518 2024-09-21T14:58:30.036 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.036 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:30.036 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.036 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:30.037 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.469843+0000", 2024-09-21T14:58:30.037 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:30.037 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.037 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.037 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:30.037 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.469843+0000", 2024-09-21T14:58:30.037 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:30.037 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.037 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.037 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:30.037 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.469842+0000", 2024-09-21T14:58:30.037 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:30.037 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.037 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.037 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:30.038 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.469849+0000", 2024-09-21T14:58:30.038 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8369999999999997e-06 2024-09-21T14:58:30.038 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.038 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.038 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:30.038 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.469873+0000", 2024-09-21T14:58:30.038 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3918e-05 2024-09-21T14:58:30.038 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.038 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.038 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:30.038 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.469882+0000", 2024-09-21T14:58:30.038 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7050000000000005e-06 2024-09-21T14:58:30.038 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.038 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.038 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:30.039 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.472577+0000", 2024-09-21T14:58:30.039 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0026946660000000001 2024-09-21T14:58:30.039 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.039 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.039 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:30.039 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.472707+0000", 2024-09-21T14:58:30.039 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013014400000000001 2024-09-21T14:58:30.039 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.039 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.039 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:30.039 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.473113+0000", 2024-09-21T14:58:30.039 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00040620899999999999 2024-09-21T14:58:30.040 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.040 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.040 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:58:30.040 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.475412+0000", 2024-09-21T14:58:30.040 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0022989920000000001 2024-09-21T14:58:30.040 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.040 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.040 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:30.040 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.475448+0000", 2024-09-21T14:58:30.040 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.5788999999999999e-05 2024-09-21T14:58:30.040 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.040 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.040 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:30.040 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.475485+0000", 2024-09-21T14:58:30.041 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7214e-05 2024-09-21T14:58:30.041 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.041 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.041 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:30.041 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.475536+0000", 2024-09-21T14:58:30.041 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1388999999999999e-05 2024-09-21T14:58:30.041 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:30.041 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:30.041 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:30.041 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.041 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.041 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:519 3.4 3:23331720:::rbd_object_map.113693316f36.0000000000000005:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T14:58:30.041 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:25.469890+0000", 2024-09-21T14:58:30.041 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.4998993550000002, 2024-09-21T14:58:30.042 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.006042108, 2024-09-21T14:58:30.042 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:30.042 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:30.042 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:30.042 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:30.042 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:30.042 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 519 2024-09-21T14:58:30.042 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.042 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:30.042 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.042 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:30.042 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.469890+0000", 2024-09-21T14:58:30.042 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:30.042 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.043 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.043 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:30.043 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.469890+0000", 2024-09-21T14:58:30.043 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:30.043 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.043 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.043 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:30.043 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.469890+0000", 2024-09-21T14:58:30.043 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:30.043 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.043 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.043 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:30.043 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.469895+0000", 2024-09-21T14:58:30.043 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8330000000000001e-06 2024-09-21T14:58:30.044 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.044 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.044 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:30.044 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.469903+0000", 2024-09-21T14:58:30.044 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0430000000000008e-06 2024-09-21T14:58:30.044 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.044 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.044 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:30.044 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.469909+0000", 2024-09-21T14:58:30.044 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9630000000000003e-06 2024-09-21T14:58:30.044 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.044 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.044 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:30.044 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.473541+0000", 2024-09-21T14:58:30.044 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0036322860000000002 2024-09-21T14:58:30.044 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.045 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.045 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:30.045 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.473651+0000", 2024-09-21T14:58:30.045 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000109855 2024-09-21T14:58:30.045 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.045 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.045 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:30.045 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.474176+0000", 2024-09-21T14:58:30.045 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00052483599999999999 2024-09-21T14:58:30.045 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.045 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.045 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:30.045 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.475820+0000", 2024-09-21T14:58:30.045 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001644791 2024-09-21T14:58:30.046 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.046 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.046 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:58:30.046 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.475828+0000", 2024-09-21T14:58:30.046 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4919999999999996e-06 2024-09-21T14:58:30.046 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.046 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.046 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:30.046 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.475877+0000", 2024-09-21T14:58:30.046 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9293999999999999e-05 2024-09-21T14:58:30.046 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.046 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.046 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:30.046 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.475933+0000", 2024-09-21T14:58:30.046 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5379e-05 2024-09-21T14:58:30.047 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:30.047 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:30.047 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:30.047 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.047 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.047 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop_reply(client.4406.0:499 3.4 e31/24 ondisk, result = 0)", 2024-09-21T14:58:30.047 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:25.471616+0000", 2024-09-21T14:58:30.047 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.4981737840000005, 2024-09-21T14:58:30.047 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00093048999999999996, 2024-09-21T14:58:30.047 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:30.047 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:30.047 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:30.047 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.047 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:30.047 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.471616+0000", 2024-09-21T14:58:30.048 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:30.048 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.048 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.048 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:30.048 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.471616+0000", 2024-09-21T14:58:30.048 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:30.048 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.048 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.048 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:30.048 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.471612+0000", 2024-09-21T14:58:30.048 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999962 2024-09-21T14:58:30.048 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.048 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.048 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:30.048 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.471622+0000", 2024-09-21T14:58:30.048 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.268e-06 2024-09-21T14:58:30.049 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.049 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.049 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:30.049 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.471634+0000", 2024-09-21T14:58:30.049 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1914e-05 2024-09-21T14:58:30.049 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.049 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.049 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:30.049 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.471648+0000", 2024-09-21T14:58:30.049 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4482e-05 2024-09-21T14:58:30.049 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.049 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.049 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:30.049 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.472526+0000", 2024-09-21T14:58:30.049 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00087754499999999997 2024-09-21T14:58:30.050 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.050 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.050 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:30.050 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.472535+0000", 2024-09-21T14:58:30.050 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7669999999999998e-06 2024-09-21T14:58:30.050 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.050 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.050 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:30.050 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.472547+0000", 2024-09-21T14:58:30.050 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1119e-05 2024-09-21T14:58:30.050 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:30.050 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:30.050 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:30.050 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.051 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.051 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop_reply(client.4406.0:500 3.4 e31/24 ondisk, result = 0)", 2024-09-21T14:58:30.051 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:25.472918+0000", 2024-09-21T14:58:30.051 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.4968722430000003, 2024-09-21T14:58:30.051 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0021287400000000001, 2024-09-21T14:58:30.051 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:30.051 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:30.051 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:30.051 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.051 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:30.051 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.472918+0000", 2024-09-21T14:58:30.051 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:30.051 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.051 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.051 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:30.052 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.472918+0000", 2024-09-21T14:58:30.052 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:30.052 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.052 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.052 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:30.052 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.472917+0000", 2024-09-21T14:58:30.052 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:30.052 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.052 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.052 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:30.052 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.472923+0000", 2024-09-21T14:58:30.052 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3840000000000002e-06 2024-09-21T14:58:30.052 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.053 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.053 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:30.053 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.472935+0000", 2024-09-21T14:58:30.053 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2099000000000001e-05 2024-09-21T14:58:30.053 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.053 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.053 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:30.053 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.472943+0000", 2024-09-21T14:58:30.053 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7079999999999997e-06 2024-09-21T14:58:30.053 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.053 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.053 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:30.053 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.475026+0000", 2024-09-21T14:58:30.053 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0020835049999999998 2024-09-21T14:58:30.053 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.054 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.054 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:30.054 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.475034+0000", 2024-09-21T14:58:30.054 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2400000000000001e-06 2024-09-21T14:58:30.054 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.054 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.054 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:30.054 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.475046+0000", 2024-09-21T14:58:30.054 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2679e-05 2024-09-21T14:58:30.054 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:30.054 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:30.054 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:30.054 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.055 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.055 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop_reply(client.4406.0:502 3.4 e31/24 ondisk, result = 0)", 2024-09-21T14:58:30.055 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:25.474538+0000", 2024-09-21T14:58:30.055 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.4952521780000003, 2024-09-21T14:58:30.055 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00087569999999999998, 2024-09-21T14:58:30.055 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:30.055 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:30.055 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:30.055 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.055 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:30.055 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.474538+0000", 2024-09-21T14:58:30.055 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:30.055 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.056 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.056 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:30.056 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.474538+0000", 2024-09-21T14:58:30.056 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:30.056 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.056 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.056 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:30.056 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.474537+0000", 2024-09-21T14:58:30.056 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:30.056 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.056 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.056 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:30.056 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.474543+0000", 2024-09-21T14:58:30.056 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3130000000000004e-06 2024-09-21T14:58:30.057 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.057 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.057 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:30.057 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.474558+0000", 2024-09-21T14:58:30.057 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.523e-05 2024-09-21T14:58:30.057 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.057 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.057 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:30.057 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.474565+0000", 2024-09-21T14:58:30.057 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8800000000000002e-06 2024-09-21T14:58:30.057 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.057 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.057 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:30.057 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.475067+0000", 2024-09-21T14:58:30.058 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00050130300000000004 2024-09-21T14:58:30.058 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.058 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.058 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:30.058 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.475073+0000", 2024-09-21T14:58:30.058 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9819999999999996e-06 2024-09-21T14:58:30.058 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.059 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.059 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:30.059 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.475413+0000", 2024-09-21T14:58:30.059 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00034079899999999997 2024-09-21T14:58:30.059 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:30.059 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:30.059 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:30.059 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.059 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.060 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop_reply(client.4406.0:518 3.4 e31/24 ondisk, result = 0)", 2024-09-21T14:58:30.060 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:25.474575+0000", 2024-09-21T14:58:30.060 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.4952145269999999, 2024-09-21T14:58:30.060 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00096987400000000002, 2024-09-21T14:58:30.060 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:30.060 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:30.060 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:30.060 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.060 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:30.060 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.474575+0000", 2024-09-21T14:58:30.060 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:30.060 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.060 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.061 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:30.061 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.474575+0000", 2024-09-21T14:58:30.061 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:30.061 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.061 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.061 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:30.061 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.474575+0000", 2024-09-21T14:58:30.061 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:30.061 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.061 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.061 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:30.061 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.474578+0000", 2024-09-21T14:58:30.061 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7130000000000001e-06 2024-09-21T14:58:30.061 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.062 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.062 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:30.062 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.474584+0000", 2024-09-21T14:58:30.062 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9410000000000004e-06 2024-09-21T14:58:30.062 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.062 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.062 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:30.062 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.474596+0000", 2024-09-21T14:58:30.062 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1955e-05 2024-09-21T14:58:30.062 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.062 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.062 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:30.062 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.475437+0000", 2024-09-21T14:58:30.063 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00084097800000000004 2024-09-21T14:58:30.063 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.063 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.063 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:30.063 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.475443+0000", 2024-09-21T14:58:30.063 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5590000000000002e-06 2024-09-21T14:58:30.063 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.063 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.063 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:30.063 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.475545+0000", 2024-09-21T14:58:30.064 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000102085 2024-09-21T14:58:30.065 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:30.065 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:30.065 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:30.065 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.065 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.065 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:671 3.4 3:23331720:::rbd_object_map.113693316f36.0000000000000005:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T14:58:30.065 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:25.478363+0000", 2024-09-21T14:58:30.066 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.4914269300000003, 2024-09-21T14:58:30.066 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0019101929999999999, 2024-09-21T14:58:30.066 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:30.066 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:30.066 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:30.066 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:30.066 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:30.066 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 671 2024-09-21T14:58:30.066 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.066 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:30.066 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.066 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:30.066 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.478363+0000", 2024-09-21T14:58:30.067 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:30.067 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.067 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.067 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:30.067 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.478363+0000", 2024-09-21T14:58:30.067 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:30.067 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.067 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.067 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:30.067 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.478362+0000", 2024-09-21T14:58:30.067 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:30.067 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.067 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.067 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:30.067 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.478377+0000", 2024-09-21T14:58:30.068 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5332000000000002e-05 2024-09-21T14:58:30.068 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.068 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.068 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:30.068 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.478395+0000", 2024-09-21T14:58:30.068 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7612999999999999e-05 2024-09-21T14:58:30.068 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.068 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.068 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:30.068 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.478403+0000", 2024-09-21T14:58:30.068 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3410000000000003e-06 2024-09-21T14:58:30.068 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.068 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.068 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:30.069 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.478457+0000", 2024-09-21T14:58:30.069 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4237999999999998e-05 2024-09-21T14:58:30.069 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.069 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.069 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:30.069 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.478544+0000", 2024-09-21T14:58:30.069 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6136000000000003e-05 2024-09-21T14:58:30.069 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.069 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.069 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:30.069 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.478983+0000", 2024-09-21T14:58:30.069 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00043967799999999999 2024-09-21T14:58:30.069 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.069 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.070 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:58:30.070 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.479779+0000", 2024-09-21T14:58:30.070 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00079532500000000005 2024-09-21T14:58:30.070 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.070 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.070 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:30.070 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.480219+0000", 2024-09-21T14:58:30.070 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00044051000000000002 2024-09-21T14:58:30.070 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.070 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.070 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:30.070 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.480250+0000", 2024-09-21T14:58:30.070 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0805000000000001e-05 2024-09-21T14:58:30.071 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.071 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.071 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:30.071 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.480273+0000", 2024-09-21T14:58:30.071 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3155000000000001e-05 2024-09-21T14:58:30.071 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:30.071 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:30.071 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:30.071 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.071 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.071 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:672 3.4 3:23331720:::rbd_object_map.113693316f36.0000000000000005:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T14:58:30.071 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:25.478976+0000", 2024-09-21T14:58:30.071 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.490814275, 2024-09-21T14:58:30.072 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0016728859999999999, 2024-09-21T14:58:30.072 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:30.072 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:30.072 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:30.072 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:30.072 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:30.072 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 672 2024-09-21T14:58:30.072 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.072 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:30.072 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.072 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:30.072 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.478976+0000", 2024-09-21T14:58:30.072 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:30.072 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.073 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.073 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:30.073 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.478976+0000", 2024-09-21T14:58:30.073 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:30.073 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.073 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.073 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:30.073 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.478975+0000", 2024-09-21T14:58:30.073 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:30.073 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.073 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.073 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:30.073 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.478981+0000", 2024-09-21T14:58:30.073 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9390000000000002e-06 2024-09-21T14:58:30.073 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.074 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.074 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:30.074 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.478993+0000", 2024-09-21T14:58:30.074 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2563e-05 2024-09-21T14:58:30.074 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.074 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.074 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:30.074 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.479000+0000", 2024-09-21T14:58:30.074 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5490000000000003e-06 2024-09-21T14:58:30.074 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.074 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.074 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:30.074 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.479344+0000", 2024-09-21T14:58:30.074 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00034405500000000001 2024-09-21T14:58:30.075 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.075 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.075 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:30.075 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.479391+0000", 2024-09-21T14:58:30.075 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7403e-05 2024-09-21T14:58:30.075 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.075 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.075 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:30.076 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.479560+0000", 2024-09-21T14:58:30.076 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016838399999999999 2024-09-21T14:58:30.076 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.076 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.076 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:58:30.076 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.480293+0000", 2024-09-21T14:58:30.076 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00073291999999999997 2024-09-21T14:58:30.076 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.076 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.076 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:30.076 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.480598+0000", 2024-09-21T14:58:30.077 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00030574299999999998 2024-09-21T14:58:30.077 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.077 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.077 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:30.077 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.480626+0000", 2024-09-21T14:58:30.077 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.796e-05 2024-09-21T14:58:30.077 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.077 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.077 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:30.077 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.480648+0000", 2024-09-21T14:58:30.077 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2154000000000002e-05 2024-09-21T14:58:30.077 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:30.077 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:30.077 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:30.077 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.077 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.078 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:721 3.4 3:23331720:::rbd_object_map.113693316f36.0000000000000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T14:58:30.078 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:25.529175+0000", 2024-09-21T14:58:30.078 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.4406148749999996, 2024-09-21T14:58:30.078 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0019794589999999998, 2024-09-21T14:58:30.078 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:30.078 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:30.078 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:30.078 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:30.078 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:30.078 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 721 2024-09-21T14:58:30.078 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.078 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:30.078 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.079 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:30.079 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.529175+0000", 2024-09-21T14:58:30.079 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:30.079 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.079 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.079 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:30.079 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.529175+0000", 2024-09-21T14:58:30.079 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:30.079 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.079 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.079 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:30.079 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.529173+0000", 2024-09-21T14:58:30.079 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T14:58:30.079 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.079 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.080 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:30.080 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.529183+0000", 2024-09-21T14:58:30.080 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9340000000000001e-06 2024-09-21T14:58:30.080 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.080 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.080 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:30.080 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.529207+0000", 2024-09-21T14:58:30.080 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3643e-05 2024-09-21T14:58:30.080 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.081 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.081 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:30.081 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.529226+0000", 2024-09-21T14:58:30.081 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9582e-05 2024-09-21T14:58:30.081 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.081 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.081 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:30.081 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.529324+0000", 2024-09-21T14:58:30.081 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7262000000000005e-05 2024-09-21T14:58:30.082 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.082 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.082 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:30.082 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.529438+0000", 2024-09-21T14:58:30.082 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000114892 2024-09-21T14:58:30.082 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.082 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.082 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:30.083 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.529593+0000", 2024-09-21T14:58:30.083 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015426599999999999 2024-09-21T14:58:30.083 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.083 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.083 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:58:30.083 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.530582+0000", 2024-09-21T14:58:30.083 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00098970999999999998 2024-09-21T14:58:30.083 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.084 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.084 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:30.084 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.530989+0000", 2024-09-21T14:58:30.084 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00040641000000000001 2024-09-21T14:58:30.084 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.084 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.084 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:30.084 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.531059+0000", 2024-09-21T14:58:30.085 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9676000000000002e-05 2024-09-21T14:58:30.085 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.085 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.085 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:30.085 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.531154+0000", 2024-09-21T14:58:30.085 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5880999999999999e-05 2024-09-21T14:58:30.085 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:30.085 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:30.085 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:30.086 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:30.086 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:30.086 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:58:30.088 INFO:teuthology.orchestra.run.smithi032.stdout:83 write 0x4aa08cb thru 0x4aa9f05 (0x963b bytes) 2024-09-21T14:58:30.088 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:30.088 INFO:teuthology.orchestra.run.smithi032.stdout:85 writesame 0xd82a920 thru 0xd82ba50 data_size 0x9(0x1131 bytes) 2024-09-21T14:58:30.089 INFO:teuthology.orchestra.run.smithi032.stdout:86 read 0x16da38a thru 0x16ea23f (0xfeb6 bytes) 2024-09-21T14:58:30.089 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:30.089 INFO:teuthology.orchestra.run.smithi032.stdout:90 read 0x93339a4 thru 0x9336930 (0x2f8d bytes) 2024-09-21T14:58:30.089 INFO:teuthology.orchestra.run.smithi032.stdout:91 writesame 0x84bcc06 thru 0x84c41b3 data_size 0x3(0x75ae bytes) 2024-09-21T14:58:30.090 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T14:58:30.090 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:58:30.085+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:58:30.097 INFO:teuthology.orchestra.run.smithi032.stdout:93 read 0x8d6baef thru 0x8d7b83b (0xfd4d bytes) 2024-09-21T14:58:30.097 INFO:teuthology.orchestra.run.smithi032.stdout:97 write 0xa79b1cc thru 0xa79bbbe (0x9f3 bytes) 2024-09-21T14:58:30.104 INFO:teuthology.orchestra.run.smithi032.stdout:99 read 0xdae8e9 thru 0xdb5138 (0x6850 bytes) 2024-09-21T14:58:30.104 INFO:teuthology.orchestra.run.smithi032.stderr:noscrub is set 2024-09-21T14:58:30.105 INFO:teuthology.orchestra.run.smithi032.stdout:100 write 0xd13ee2c thru 0xd14c778 (0xd94d bytes) 2024-09-21T14:58:30.112 INFO:teuthology.orchestra.run.smithi032.stdout:101 writesame 0xc7c0baf thru 0xc7ca8ac data_size 0x5(0x9cfe bytes) 2024-09-21T14:58:30.119 INFO:teuthology.orchestra.run.smithi032.stdout:102 write 0x5b40266 thru 0x5b4c40b (0xc1a6 bytes) 2024-09-21T14:58:30.127 INFO:teuthology.orchestra.run.smithi032.stdout:103 read 0x178c719 thru 0x17984bc (0xbda4 bytes) 2024-09-21T14:58:30.127 INFO:teuthology.orchestra.run.smithi032.stdout:107 punch from 0xdeb3517 to 0xdeb6155, (0x2c3e bytes) 2024-09-21T14:58:30.128 INFO:teuthology.orchestra.run.smithi032.stdout:108 read 0x8393a69 thru 0x83a1d53 (0xe2eb bytes) 2024-09-21T14:58:30.151 INFO:teuthology.orchestra.run.smithi032.stdout:109 trunc from 0xe712634 to 0x5b5687 2024-09-21T14:58:30.189 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:58:30.189+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:58:30.209 INFO:teuthology.orchestra.run.smithi032.stdout:110 writesame 0xb268746 thru 0xb27764e data_size 0x1(0xef09 bytes) 2024-09-21T14:58:30.210 INFO:teuthology.orchestra.run.smithi032.stdout:113 read 0x79f1215 thru 0x79f26ed (0x14d9 bytes) 2024-09-21T14:58:30.224 INFO:teuthology.orchestra.run.smithi032.stdout:114 writesame 0xca8ad2d thru 0xca91c95 data_size 0x3(0x6f69 bytes) 2024-09-21T14:58:30.224 INFO:teuthology.orchestra.run.smithi032.stdout:117 write 0x9c2ccd1 thru 0x9c38d8f (0xc0bf bytes) 2024-09-21T14:58:30.231 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:30.232 INFO:teuthology.orchestra.run.smithi032.stdout:119 trunc from 0xca91c96 to 0xd097709 2024-09-21T14:58:30.245 INFO:teuthology.orchestra.run.smithi032.stdout:120 read 0xa7da16e thru 0xa7de307 (0x419a bytes) 2024-09-21T14:58:30.245 INFO:teuthology.orchestra.run.smithi032.stdout:122 write 0xc7dc102 thru 0xc7e2847 (0x6746 bytes) 2024-09-21T14:58:30.253 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:58:30.253 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:58:30.254 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T14:58:30.254 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:58:30.257 INFO:teuthology.orchestra.run.smithi032.stdout:124 trunc from 0xd097709 to 0xba5bef2 2024-09-21T14:58:30.264 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T14:58:30.281 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:30.281 INFO:teuthology.orchestra.run.smithi032.stdout:126 write 0x4d70621 thru 0x4d7c7fe (0xc1de bytes) 2024-09-21T14:58:30.288 INFO:teuthology.orchestra.run.smithi032.stdout:127 writesame 0x47b7535 thru 0x47c4259 data_size 0x1(0xcd25 bytes) 2024-09-21T14:58:30.309 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:58:30.309+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:58:30.336 INFO:teuthology.orchestra.run.smithi032.stdout:128 writesame 0xa91ae2d thru 0xa926c97 data_size 0x1(0xbe6b bytes) 2024-09-21T14:58:30.374 INFO:teuthology.orchestra.run.smithi032.stdout:129 punch from 0x9aecbd to 0x9bb27b, (0xc5be bytes) 2024-09-21T14:58:30.375 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:30.375 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:30.375 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:30.375 INFO:teuthology.orchestra.run.smithi032.stdout:133 writesame 0x5ec2e7a thru 0x5ec5fea data_size 0x1(0x3171 bytes) 2024-09-21T14:58:30.377 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:58:30.377 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:58:30.377 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T14:58:30.377 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T14:58:30.377 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:58:30.385 INFO:teuthology.orchestra.run.smithi032.stdout:134 read 0x61e67af thru 0x61f1860 (0xb0b2 bytes) 2024-09-21T14:58:30.385 INFO:teuthology.orchestra.run.smithi032.stdout:135 punch from 0x3292702 to 0x32987e8, (0x60e6 bytes) 2024-09-21T14:58:30.389 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T14:58:30.390 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:58:30.384+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:58:30.404 INFO:teuthology.orchestra.run.smithi032.stdout:136 writesame 0xe96df03 thru 0xe976261 data_size 0xd(0x835f bytes) 2024-09-21T14:58:30.405 INFO:teuthology.orchestra.run.smithi032.stdout:137 write 0x3a1b094 thru 0x3a1e1c0 (0x312d bytes) 2024-09-21T14:58:30.411 INFO:teuthology.orchestra.run.smithi032.stdout:138 read 0x3586704 thru 0x3593cc6 (0xd5c3 bytes) 2024-09-21T14:58:30.411 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:30.411 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:30.411 INFO:teuthology.orchestra.run.smithi032.stdout:142 write 0xe1793e0 thru 0xe18223b (0x8e5c bytes) 2024-09-21T14:58:30.419 INFO:teuthology.orchestra.run.smithi032.stdout:143 writesame 0x6bdc29f thru 0x6beabe2 data_size 0x2(0xe944 bytes) 2024-09-21T14:58:30.444 INFO:teuthology.orchestra.run.smithi032.stdout:144 read 0xe23524f thru 0xe240173 (0xaf25 bytes) 2024-09-21T14:58:30.444 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:30.444 INFO:teuthology.orchestra.run.smithi032.stdout:146 writesame 0xcf74893 thru 0xcf77602 data_size 0x10(0x2d70 bytes) 2024-09-21T14:58:30.449 INFO:teuthology.orchestra.run.smithi032.stdout:147 read 0x126a75e thru 0x1278588 (0xde2b bytes) 2024-09-21T14:58:30.450 INFO:teuthology.orchestra.run.smithi032.stdout:149 write 0x41d6a55 thru 0x41e4198 (0xd744 bytes) 2024-09-21T14:58:30.479 INFO:teuthology.orchestra.run.smithi032.stdout:154 trunc from 0xe976262 to 0x149e26a 2024-09-21T14:58:30.490 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:58:30.489+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:58:30.496 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:30.500 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:58:30.500 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T14:58:30.500 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T14:58:30.500 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T14:58:30.500 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.501 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:12 3.6 e27/24 3:6748301c:::rbd_header.11367ffd86fe:head v 27'1, mlcod=0'0)", 2024-09-21T14:58:30.501 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:21.426395+0000", 2024-09-21T14:58:30.501 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.0730880309999993, 2024-09-21T14:58:30.501 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0015710940000000001, 2024-09-21T14:58:30.501 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:30.501 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:30.501 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:30.501 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.501 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:30.501 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.426395+0000", 2024-09-21T14:58:30.501 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:30.501 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.501 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.501 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:30.502 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.426395+0000", 2024-09-21T14:58:30.502 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:30.502 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.502 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.502 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:30.502 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.426394+0000", 2024-09-21T14:58:30.502 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:30.502 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.502 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.502 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:30.502 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.426405+0000", 2024-09-21T14:58:30.502 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0811e-05 2024-09-21T14:58:30.502 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.502 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.503 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:30.503 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.426438+0000", 2024-09-21T14:58:30.503 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3241999999999998e-05 2024-09-21T14:58:30.503 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.503 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.503 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:30.503 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.426455+0000", 2024-09-21T14:58:30.503 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6506000000000001e-05 2024-09-21T14:58:30.503 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.503 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.503 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:30.503 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.426546+0000", 2024-09-21T14:58:30.504 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1042e-05 2024-09-21T14:58:30.504 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.504 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.504 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:30.504 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.426598+0000", 2024-09-21T14:58:30.504 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1870999999999999e-05 2024-09-21T14:58:30.504 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.504 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.504 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:30.504 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.427884+0000", 2024-09-21T14:58:30.504 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0012864490000000001 2024-09-21T14:58:30.504 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.504 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.504 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:30.505 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.427966+0000", 2024-09-21T14:58:30.505 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1876e-05 2024-09-21T14:58:30.505 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:30.505 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:30.505 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:30.505 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.505 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.505 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:341 3.b 3:d4183149:::rbd_data.11367ffd86fe.0000000000000021:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T14:58:30.505 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:25.462758+0000", 2024-09-21T14:58:30.505 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.0367249999999997, 2024-09-21T14:58:30.505 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0044292250000000002, 2024-09-21T14:58:30.505 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:30.505 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:58:30.505 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:30.505 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:30.506 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:30.506 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 341 2024-09-21T14:58:30.506 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.506 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:30.506 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.506 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:30.506 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.462758+0000", 2024-09-21T14:58:30.506 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:30.506 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.506 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.506 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:30.506 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.462758+0000", 2024-09-21T14:58:30.506 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:30.506 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.507 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.507 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:30.507 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.462756+0000", 2024-09-21T14:58:30.507 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-21T14:58:30.507 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.507 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.507 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:30.507 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.462768+0000", 2024-09-21T14:58:30.507 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2384e-05 2024-09-21T14:58:30.507 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.507 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.507 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:30.507 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.462797+0000", 2024-09-21T14:58:30.507 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8666999999999998e-05 2024-09-21T14:58:30.508 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.508 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.508 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:30.508 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.462814+0000", 2024-09-21T14:58:30.508 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6730000000000001e-05 2024-09-21T14:58:30.508 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.508 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.508 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:30.508 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.463191+0000", 2024-09-21T14:58:30.508 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000377096 2024-09-21T14:58:30.508 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.508 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.508 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op must wait for map", 2024-09-21T14:58:30.508 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.463204+0000", 2024-09-21T14:58:30.509 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3777999999999999e-05 2024-09-21T14:58:30.509 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.509 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.509 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:30.509 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.467079+0000", 2024-09-21T14:58:30.509 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.003874307 2024-09-21T14:58:30.509 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.509 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.509 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:30.509 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.467187+0000", 2024-09-21T14:58:30.509 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000108451 2024-09-21T14:58:30.509 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:30.509 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:30.509 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:30.509 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.509 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.510 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:357 3.b 3:d4183149:::rbd_data.11367ffd86fe.0000000000000021:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T14:58:30.510 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:25.462843+0000", 2024-09-21T14:58:30.510 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.0366404280000001, 2024-09-21T14:58:30.510 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0044394409999999997, 2024-09-21T14:58:30.510 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:30.510 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:58:30.510 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:30.510 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:30.510 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:30.510 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 357 2024-09-21T14:58:30.510 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.510 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:30.510 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.510 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:30.510 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.462843+0000", 2024-09-21T14:58:30.511 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:30.511 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.511 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.511 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:30.511 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.462843+0000", 2024-09-21T14:58:30.511 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:30.511 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.511 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.511 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:30.511 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.462842+0000", 2024-09-21T14:58:30.511 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:30.511 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.511 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.511 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:30.512 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.462848+0000", 2024-09-21T14:58:30.512 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9020000000000002e-06 2024-09-21T14:58:30.512 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.512 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.512 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:30.512 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.462868+0000", 2024-09-21T14:58:30.512 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0088000000000001e-05 2024-09-21T14:58:30.512 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.512 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.512 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:30.512 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.462897+0000", 2024-09-21T14:58:30.512 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8595e-05 2024-09-21T14:58:30.512 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.512 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.512 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:30.512 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.463327+0000", 2024-09-21T14:58:30.513 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00042943899999999999 2024-09-21T14:58:30.513 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.513 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.513 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting_for_map not empty", 2024-09-21T14:58:30.513 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.463339+0000", 2024-09-21T14:58:30.513 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2767e-05 2024-09-21T14:58:30.513 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.513 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.513 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:30.513 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.467209+0000", 2024-09-21T14:58:30.513 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.003869295 2024-09-21T14:58:30.513 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.513 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.514 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:30.514 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.467282+0000", 2024-09-21T14:58:30.514 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3399000000000007e-05 2024-09-21T14:58:30.514 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:30.514 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:30.514 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:30.514 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.514 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.514 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:726 3.6 e32/24 3:63a18c8a:::rbd_object_map.113693316f36:head v 32'37, mlcod=31'35)", 2024-09-21T14:58:30.514 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:26.467264+0000", 2024-09-21T14:58:30.514 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.0322185890000002, 2024-09-21T14:58:30.514 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0013553009999999999, 2024-09-21T14:58:30.514 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:30.514 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:30.515 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:30.515 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.515 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:30.515 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.467264+0000", 2024-09-21T14:58:30.515 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:30.515 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.515 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.515 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:30.515 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.467264+0000", 2024-09-21T14:58:30.515 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:30.515 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.515 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.515 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:30.515 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.467264+0000", 2024-09-21T14:58:30.516 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:30.516 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.516 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.516 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:30.516 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.467270+0000", 2024-09-21T14:58:30.517 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4049999999999997e-06 2024-09-21T14:58:30.517 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.517 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.517 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:30.517 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.467283+0000", 2024-09-21T14:58:30.517 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2843999999999999e-05 2024-09-21T14:58:30.517 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.518 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.518 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:30.518 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.467294+0000", 2024-09-21T14:58:30.518 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0567999999999999e-05 2024-09-21T14:58:30.518 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.518 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.518 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:30.518 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.467409+0000", 2024-09-21T14:58:30.518 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000115295 2024-09-21T14:58:30.518 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.518 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.518 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:30.518 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.467469+0000", 2024-09-21T14:58:30.518 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9641e-05 2024-09-21T14:58:30.519 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.519 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.519 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:30.519 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.468522+0000", 2024-09-21T14:58:30.519 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001053525 2024-09-21T14:58:30.519 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.519 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.519 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:30.519 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.468620+0000", 2024-09-21T14:58:30.519 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7497999999999994e-05 2024-09-21T14:58:30.519 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:30.519 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:30.519 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:30.519 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.520 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.520 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:731 3.6 e32/24 3:6748301c:::rbd_header.11367ffd86fe:head v 32'38, mlcod=31'36)", 2024-09-21T14:58:30.520 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:26.603426+0000", 2024-09-21T14:58:30.520 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.8960568750000002, 2024-09-21T14:58:30.520 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0016267930000000001, 2024-09-21T14:58:30.520 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:30.520 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:30.520 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:30.520 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.520 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:30.520 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.603426+0000", 2024-09-21T14:58:30.520 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:30.520 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.521 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.521 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:30.521 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.603426+0000", 2024-09-21T14:58:30.521 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:30.521 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.521 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.521 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:30.521 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.603423+0000", 2024-09-21T14:58:30.521 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999971 2024-09-21T14:58:30.521 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.521 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.521 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:30.521 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.603447+0000", 2024-09-21T14:58:30.522 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3592999999999999e-05 2024-09-21T14:58:30.522 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.522 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.522 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:30.522 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.603494+0000", 2024-09-21T14:58:30.522 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6891999999999998e-05 2024-09-21T14:58:30.522 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.522 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.522 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:30.522 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.603514+0000", 2024-09-21T14:58:30.522 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0176000000000001e-05 2024-09-21T14:58:30.522 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.522 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.523 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:30.523 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.603664+0000", 2024-09-21T14:58:30.523 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014961099999999999 2024-09-21T14:58:30.523 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.523 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.523 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:30.523 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.603729+0000", 2024-09-21T14:58:30.523 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5362000000000002e-05 2024-09-21T14:58:30.523 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.523 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.523 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:30.523 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.604979+0000", 2024-09-21T14:58:30.523 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0012499500000000001 2024-09-21T14:58:30.524 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.524 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.524 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:30.524 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.605053+0000", 2024-09-21T14:58:30.524 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3912000000000007e-05 2024-09-21T14:58:30.524 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:30.524 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:30.524 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:30.524 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.524 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.524 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:747 3.6 e32/24 3:63a18c8a:::rbd_object_map.113693316f36:head v 32'39, mlcod=32'37)", 2024-09-21T14:58:30.524 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:26.621788+0000", 2024-09-21T14:58:30.524 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.8776946539999999, 2024-09-21T14:58:30.524 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0015379980000000001, 2024-09-21T14:58:30.525 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:30.525 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:30.525 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:30.525 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.525 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:30.525 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.621788+0000", 2024-09-21T14:58:30.525 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:30.525 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.525 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.525 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:30.525 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.621788+0000", 2024-09-21T14:58:30.525 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:30.525 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.526 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.526 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:30.526 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.621786+0000", 2024-09-21T14:58:30.526 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-21T14:58:30.526 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.526 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.526 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:30.526 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.621808+0000", 2024-09-21T14:58:30.526 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2194000000000001e-05 2024-09-21T14:58:30.526 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.526 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.526 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:30.527 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.621851+0000", 2024-09-21T14:58:30.527 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2938999999999997e-05 2024-09-21T14:58:30.527 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.527 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.527 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:30.527 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.621873+0000", 2024-09-21T14:58:30.527 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1691999999999998e-05 2024-09-21T14:58:30.527 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.527 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.527 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:30.527 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.622009+0000", 2024-09-21T14:58:30.527 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000136358 2024-09-21T14:58:30.527 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.528 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.528 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:30.528 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.622098+0000", 2024-09-21T14:58:30.528 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8781000000000003e-05 2024-09-21T14:58:30.528 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.528 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.528 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:30.528 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.623253+0000", 2024-09-21T14:58:30.528 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0011549430000000001 2024-09-21T14:58:30.528 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.528 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.528 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:30.528 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.623326+0000", 2024-09-21T14:58:30.528 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3458e-05 2024-09-21T14:58:30.529 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:30.529 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:30.529 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:30.529 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.529 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.529 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:750 3.6 e32/24 3:63a18c8a:::rbd_object_map.113693316f36:head v 32'40, mlcod=32'38)", 2024-09-21T14:58:30.529 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:26.625746+0000", 2024-09-21T14:58:30.529 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.8737368299999999, 2024-09-21T14:58:30.529 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0013943969999999999, 2024-09-21T14:58:30.529 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:30.529 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:30.529 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:30.529 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.529 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:30.529 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.625746+0000", 2024-09-21T14:58:30.529 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:30.530 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.530 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.530 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:30.530 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.625746+0000", 2024-09-21T14:58:30.530 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:30.530 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.530 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.530 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:30.530 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.625744+0000", 2024-09-21T14:58:30.530 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T14:58:30.530 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.530 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.530 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:30.530 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.625763+0000", 2024-09-21T14:58:30.531 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8827000000000001e-05 2024-09-21T14:58:30.531 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.531 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.531 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:30.531 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.625801+0000", 2024-09-21T14:58:30.531 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7545000000000001e-05 2024-09-21T14:58:30.531 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.531 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.531 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:30.531 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.625820+0000", 2024-09-21T14:58:30.531 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9006999999999999e-05 2024-09-21T14:58:30.531 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.531 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.531 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:30.531 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.625904+0000", 2024-09-21T14:58:30.532 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3956999999999996e-05 2024-09-21T14:58:30.532 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.532 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.532 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:30.532 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.625939+0000", 2024-09-21T14:58:30.532 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4841000000000001e-05 2024-09-21T14:58:30.532 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.532 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.532 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:30.532 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.627057+0000", 2024-09-21T14:58:30.532 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001118406 2024-09-21T14:58:30.532 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.532 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.532 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:30.532 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.627141+0000", 2024-09-21T14:58:30.533 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3521000000000002e-05 2024-09-21T14:58:30.533 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:30.533 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:30.533 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:30.533 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.533 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.533 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:753 3.6 e32/24 3:63a18c8a:::rbd_object_map.113693316f36:head v 32'41, mlcod=32'39)", 2024-09-21T14:58:30.533 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:26.694470+0000", 2024-09-21T14:58:30.533 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.8050128939999999, 2024-09-21T14:58:30.533 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001849961, 2024-09-21T14:58:30.533 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:30.533 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:30.533 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:30.533 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.534 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:30.534 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.694470+0000", 2024-09-21T14:58:30.534 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:30.534 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.534 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.534 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:30.534 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.694470+0000", 2024-09-21T14:58:30.534 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:30.534 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.534 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.534 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:30.534 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.694468+0000", 2024-09-21T14:58:30.534 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T14:58:30.534 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.535 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.535 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:30.535 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.694488+0000", 2024-09-21T14:58:30.535 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0202000000000001e-05 2024-09-21T14:58:30.535 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.535 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.535 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:30.535 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.694516+0000", 2024-09-21T14:58:30.535 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7699999999999999e-05 2024-09-21T14:58:30.535 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.535 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.535 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:30.535 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.694533+0000", 2024-09-21T14:58:30.535 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7235999999999999e-05 2024-09-21T14:58:30.535 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.536 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.536 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:30.536 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.694677+0000", 2024-09-21T14:58:30.536 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014342099999999999 2024-09-21T14:58:30.536 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.536 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.536 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:30.536 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.694721+0000", 2024-09-21T14:58:30.536 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4022e-05 2024-09-21T14:58:30.536 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.536 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.536 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:30.536 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.696247+0000", 2024-09-21T14:58:30.536 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0015265879999999999 2024-09-21T14:58:30.537 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.537 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.537 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:30.537 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.696320+0000", 2024-09-21T14:58:30.537 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2662000000000003e-05 2024-09-21T14:58:30.537 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:30.537 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:30.537 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:30.537 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.537 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.537 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:756 3.6 e32/24 3:60fbaa5f:::rbd_data.113693316f36.0000000000000014:head v 32'42, mlcod=32'40)", 2024-09-21T14:58:30.537 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:26.697592+0000", 2024-09-21T14:58:30.537 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.8018914100000001, 2024-09-21T14:58:30.537 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0016480189999999999, 2024-09-21T14:58:30.537 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:30.537 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:30.538 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:30.538 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.538 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:30.538 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.697592+0000", 2024-09-21T14:58:30.538 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:30.538 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.538 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.538 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:30.538 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.697592+0000", 2024-09-21T14:58:30.538 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:30.538 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.538 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.538 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:30.539 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.697590+0000", 2024-09-21T14:58:30.539 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T14:58:30.539 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.539 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.539 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:30.539 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.697606+0000", 2024-09-21T14:58:30.539 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6062999999999999e-05 2024-09-21T14:58:30.539 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.539 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.539 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:30.539 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.697641+0000", 2024-09-21T14:58:30.539 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.5216000000000003e-05 2024-09-21T14:58:30.539 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.539 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.540 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:30.540 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.697676+0000", 2024-09-21T14:58:30.540 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4393000000000001e-05 2024-09-21T14:58:30.540 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.540 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.540 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:30.540 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.697765+0000", 2024-09-21T14:58:30.540 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9918000000000002e-05 2024-09-21T14:58:30.540 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.540 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.540 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:30.540 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.697803+0000", 2024-09-21T14:58:30.540 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7045999999999997e-05 2024-09-21T14:58:30.540 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.540 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.540 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:30.541 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.699163+0000", 2024-09-21T14:58:30.541 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001360874 2024-09-21T14:58:30.541 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.541 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.541 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:30.541 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.699240+0000", 2024-09-21T14:58:30.541 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6188999999999998e-05 2024-09-21T14:58:30.541 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:30.541 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:30.541 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:30.541 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.541 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.541 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:757 3.6 e32/24 3:63a18c8a:::rbd_object_map.113693316f36:head v 32'43, mlcod=32'41)", 2024-09-21T14:58:30.541 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:26.700725+0000", 2024-09-21T14:58:30.541 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.798758056, 2024-09-21T14:58:30.542 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0019856880000000002, 2024-09-21T14:58:30.542 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:30.542 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:30.542 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:30.542 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.542 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:30.542 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.700725+0000", 2024-09-21T14:58:30.542 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:30.542 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.542 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.542 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:30.542 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.700725+0000", 2024-09-21T14:58:30.542 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:30.543 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.543 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.543 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:30.543 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.700723+0000", 2024-09-21T14:58:30.543 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:30.543 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.543 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.543 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:30.543 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.700739+0000", 2024-09-21T14:58:30.543 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5953000000000001e-05 2024-09-21T14:58:30.543 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.543 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.543 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:30.544 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.700776+0000", 2024-09-21T14:58:30.544 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6223000000000002e-05 2024-09-21T14:58:30.544 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.544 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.544 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:30.544 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.700792+0000", 2024-09-21T14:58:30.544 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6765999999999998e-05 2024-09-21T14:58:30.544 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.544 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.544 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:30.544 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.700907+0000", 2024-09-21T14:58:30.544 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011500100000000001 2024-09-21T14:58:30.544 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.544 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.544 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:30.545 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.700965+0000", 2024-09-21T14:58:30.545 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8004999999999998e-05 2024-09-21T14:58:30.545 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.545 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.545 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:30.545 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.702645+0000", 2024-09-21T14:58:30.545 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0016795320000000001 2024-09-21T14:58:30.545 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.545 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.545 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:30.545 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.702711+0000", 2024-09-21T14:58:30.545 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5797999999999996e-05 2024-09-21T14:58:30.545 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:30.545 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:30.545 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:30.546 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.546 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.546 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:759 3.6 e32/24 3:63a18c8a:::rbd_object_map.113693316f36:head v 32'44, mlcod=32'42)", 2024-09-21T14:58:30.546 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:26.704445+0000", 2024-09-21T14:58:30.546 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.7950377500000001, 2024-09-21T14:58:30.546 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001742464, 2024-09-21T14:58:30.546 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:30.546 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:30.546 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:30.546 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.546 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:30.546 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.704445+0000", 2024-09-21T14:58:30.546 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:30.546 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.547 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.547 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:30.547 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.704445+0000", 2024-09-21T14:58:30.547 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:30.547 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.547 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.547 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:30.547 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.704444+0000", 2024-09-21T14:58:30.547 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:30.547 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.547 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.547 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:30.547 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.704454+0000", 2024-09-21T14:58:30.547 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9399999999999997e-06 2024-09-21T14:58:30.547 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.547 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.548 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:30.548 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.704473+0000", 2024-09-21T14:58:30.548 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8593999999999998e-05 2024-09-21T14:58:30.548 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.548 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.548 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:30.548 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.704480+0000", 2024-09-21T14:58:30.548 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8860000000000004e-06 2024-09-21T14:58:30.548 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.548 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.548 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:30.548 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.704604+0000", 2024-09-21T14:58:30.548 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000123643 2024-09-21T14:58:30.548 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.548 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.549 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:30.549 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.704634+0000", 2024-09-21T14:58:30.549 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0145e-05 2024-09-21T14:58:30.549 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.549 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.549 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:30.549 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.706121+0000", 2024-09-21T14:58:30.549 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0014864489999999999 2024-09-21T14:58:30.549 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.549 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.549 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:30.549 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.706188+0000", 2024-09-21T14:58:30.549 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6989000000000005e-05 2024-09-21T14:58:30.549 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:30.550 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:30.550 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:30.550 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.550 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.550 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:773 3.6 e32/24 3:63a18c8a:::rbd_object_map.113693316f36:head v 32'47, mlcod=32'45)", 2024-09-21T14:58:30.550 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:26.740462+0000", 2024-09-21T14:58:30.550 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.7590207680000001, 2024-09-21T14:58:30.550 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0017884859999999999, 2024-09-21T14:58:30.550 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:30.550 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:30.550 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:30.550 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.550 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:30.551 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.740462+0000", 2024-09-21T14:58:30.551 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:30.551 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.551 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.551 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:30.551 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.740462+0000", 2024-09-21T14:58:30.551 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:30.551 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.551 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.551 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:30.551 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.740461+0000", 2024-09-21T14:58:30.551 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:30.551 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.551 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.551 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:30.551 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.740475+0000", 2024-09-21T14:58:30.552 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4283e-05 2024-09-21T14:58:30.552 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.552 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.552 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:30.552 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.740501+0000", 2024-09-21T14:58:30.552 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5626000000000002e-05 2024-09-21T14:58:30.552 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.552 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.552 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:30.552 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.740512+0000", 2024-09-21T14:58:30.552 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0882e-05 2024-09-21T14:58:30.552 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.552 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.552 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:30.552 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.740624+0000", 2024-09-21T14:58:30.553 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000112448 2024-09-21T14:58:30.553 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.553 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.553 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:30.553 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.740690+0000", 2024-09-21T14:58:30.553 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6476000000000005e-05 2024-09-21T14:58:30.553 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.553 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.553 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:30.553 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.742177+0000", 2024-09-21T14:58:30.553 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0014866790000000001 2024-09-21T14:58:30.553 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.553 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.553 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:30.553 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.742251+0000", 2024-09-21T14:58:30.553 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3564000000000002e-05 2024-09-21T14:58:30.554 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:30.554 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:30.554 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:30.554 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.554 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.554 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:1300 3.6 e35/24 3:6748301c:::rbd_header.11367ffd86fe:head v 35'50, mlcod=33'48)", 2024-09-21T14:58:30.554 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:29.661518+0000", 2024-09-21T14:58:30.554 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.83796501700000003, 2024-09-21T14:58:30.554 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001354844, 2024-09-21T14:58:30.554 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:30.554 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:30.554 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:30.554 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.554 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:30.555 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:29.661518+0000", 2024-09-21T14:58:30.555 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:30.555 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.555 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.555 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:30.555 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:29.661518+0000", 2024-09-21T14:58:30.555 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:30.555 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.555 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.555 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:30.555 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:29.661517+0000", 2024-09-21T14:58:30.555 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:30.555 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.555 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.555 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:30.555 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:29.661527+0000", 2024-09-21T14:58:30.556 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3170000000000008e-06 2024-09-21T14:58:30.556 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.556 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.556 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:30.556 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:29.661576+0000", 2024-09-21T14:58:30.556 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9018999999999999e-05 2024-09-21T14:58:30.556 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.556 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.556 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:30.556 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:29.661597+0000", 2024-09-21T14:58:30.556 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0794000000000001e-05 2024-09-21T14:58:30.556 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.556 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.556 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:30.556 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:29.661740+0000", 2024-09-21T14:58:30.556 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014338400000000001 2024-09-21T14:58:30.557 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.557 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.557 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:30.557 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:29.661808+0000", 2024-09-21T14:58:30.557 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7764999999999999e-05 2024-09-21T14:58:30.557 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.557 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.557 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:30.557 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:29.662803+0000", 2024-09-21T14:58:30.557 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00099558999999999997 2024-09-21T14:58:30.557 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.557 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.557 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:30.557 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:29.662873+0000", 2024-09-21T14:58:30.558 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9564e-05 2024-09-21T14:58:30.558 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:30.558 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:30.558 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:30.558 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.558 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.558 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:1359 3.6 e35/24 3:6748301c:::rbd_header.11367ffd86fe:head v 35'51, mlcod=33'49)", 2024-09-21T14:58:30.558 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:29.868778+0000", 2024-09-21T14:58:30.558 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.63070495299999996, 2024-09-21T14:58:30.558 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001465208, 2024-09-21T14:58:30.558 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:30.558 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:30.558 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:30.558 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.559 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:30.559 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:29.868778+0000", 2024-09-21T14:58:30.559 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:30.559 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.559 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.559 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:30.559 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:29.868778+0000", 2024-09-21T14:58:30.559 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:30.559 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.559 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.559 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:30.559 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:29.868775+0000", 2024-09-21T14:58:30.559 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999971 2024-09-21T14:58:30.559 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.559 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.560 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:30.560 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:29.868797+0000", 2024-09-21T14:58:30.560 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1645999999999999e-05 2024-09-21T14:58:30.560 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.560 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.560 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:30.560 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:29.868844+0000", 2024-09-21T14:58:30.560 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7493000000000001e-05 2024-09-21T14:58:30.560 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.560 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.560 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:30.560 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:29.868865+0000", 2024-09-21T14:58:30.560 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0519000000000001e-05 2024-09-21T14:58:30.560 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.560 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.560 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:30.561 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:29.869002+0000", 2024-09-21T14:58:30.561 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000136907 2024-09-21T14:58:30.561 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.561 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.561 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:30.561 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:29.869046+0000", 2024-09-21T14:58:30.561 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4353e-05 2024-09-21T14:58:30.561 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.561 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.561 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:30.561 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:29.870201+0000", 2024-09-21T14:58:30.561 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0011551459999999999 2024-09-21T14:58:30.561 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.561 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.562 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:30.562 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:29.870243+0000", 2024-09-21T14:58:30.562 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1916999999999999e-05 2024-09-21T14:58:30.562 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:30.562 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:30.562 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:30.562 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.562 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.562 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:1367 3.b 3:d2b92ac8:::rbd_data.1136515ecb6b.0000000000000011:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2754763~38459] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-21T14:58:30.562 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:30.074394+0000", 2024-09-21T14:58:30.562 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.42508871999999998, 2024-09-21T14:58:30.562 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0036624069999999999, 2024-09-21T14:58:30.562 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:30.562 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:30.563 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:30.563 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:30.563 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:30.563 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1367 2024-09-21T14:58:30.563 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.563 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:30.563 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.563 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:30.563 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.074394+0000", 2024-09-21T14:58:30.563 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:30.563 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.563 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.563 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:30.564 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.074394+0000", 2024-09-21T14:58:30.564 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:30.564 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.564 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.564 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:30.564 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.074392+0000", 2024-09-21T14:58:30.564 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-21T14:58:30.564 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.564 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.564 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:30.564 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.074509+0000", 2024-09-21T14:58:30.564 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000117061 2024-09-21T14:58:30.564 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.564 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.565 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:30.565 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.074545+0000", 2024-09-21T14:58:30.565 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.5722000000000001e-05 2024-09-21T14:58:30.565 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.565 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.565 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:30.565 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.074563+0000", 2024-09-21T14:58:30.565 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8119999999999999e-05 2024-09-21T14:58:30.565 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.565 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.565 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:30.565 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.074684+0000", 2024-09-21T14:58:30.565 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000121724 2024-09-21T14:58:30.565 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.566 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.566 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:30.566 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.074941+0000", 2024-09-21T14:58:30.566 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00025672899999999997 2024-09-21T14:58:30.566 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.566 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.566 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T14:58:30.566 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.075150+0000", 2024-09-21T14:58:30.566 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00020905899999999999 2024-09-21T14:58:30.566 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.566 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.566 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:58:30.566 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.077349+0000", 2024-09-21T14:58:30.566 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0021984130000000002 2024-09-21T14:58:30.567 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.567 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.567 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:30.567 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.077875+0000", 2024-09-21T14:58:30.567 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00052637500000000004 2024-09-21T14:58:30.567 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.567 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.567 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:30.567 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.077976+0000", 2024-09-21T14:58:30.567 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000101371 2024-09-21T14:58:30.567 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.567 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.567 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:30.567 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.078057+0000", 2024-09-21T14:58:30.567 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0298999999999998e-05 2024-09-21T14:58:30.567 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:30.568 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:30.568 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:30.568 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.568 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.568 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:1373 3.8 e36/24 3:13ba05d9:::rbd_data.1136515ecb6b.000000000000002b:head v 36'3, mlcod=29'1)", 2024-09-21T14:58:30.568 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:30.103488+0000", 2024-09-21T14:58:30.568 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.39599486099999998, 2024-09-21T14:58:30.568 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0012395170000000001, 2024-09-21T14:58:30.568 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:30.568 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:30.568 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:30.568 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.568 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:30.568 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.103488+0000", 2024-09-21T14:58:30.569 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:30.569 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.569 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.569 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:30.569 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.103488+0000", 2024-09-21T14:58:30.569 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:30.569 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.569 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.569 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:30.569 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.103487+0000", 2024-09-21T14:58:30.569 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:30.569 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.569 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.570 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:30.570 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.103506+0000", 2024-09-21T14:58:30.570 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8746e-05 2024-09-21T14:58:30.570 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.570 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.570 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:30.570 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.103535+0000", 2024-09-21T14:58:30.570 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.853e-05 2024-09-21T14:58:30.570 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.570 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.570 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:30.570 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.103557+0000", 2024-09-21T14:58:30.570 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2515999999999999e-05 2024-09-21T14:58:30.570 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.571 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.571 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:30.571 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.103727+0000", 2024-09-21T14:58:30.571 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016950600000000001 2024-09-21T14:58:30.571 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.571 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.571 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:30.571 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.103760+0000", 2024-09-21T14:58:30.571 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3636000000000001e-05 2024-09-21T14:58:30.571 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.571 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.571 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:30.571 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.104627+0000", 2024-09-21T14:58:30.571 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000867149 2024-09-21T14:58:30.571 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.571 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.572 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:30.572 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.104728+0000", 2024-09-21T14:58:30.572 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010017399999999999 2024-09-21T14:58:30.572 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:30.572 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:30.572 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:30.572 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.572 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.572 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:1376 3.c e36/24 3:35bac63f:::rbd_data.1136515ecb6b.0000000000000036:head v 36'1, mlcod=0'0)", 2024-09-21T14:58:30.572 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:30.111873+0000", 2024-09-21T14:58:30.572 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.38761005100000001, 2024-09-21T14:58:30.572 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0013990140000000001, 2024-09-21T14:58:30.572 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:30.572 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:30.572 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:30.573 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.573 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:30.573 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.111873+0000", 2024-09-21T14:58:30.573 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:30.573 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.573 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.573 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:30.573 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.111873+0000", 2024-09-21T14:58:30.573 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:30.573 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.573 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.573 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:30.574 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.111873+0000", 2024-09-21T14:58:30.574 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:30.574 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.574 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.574 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:30.574 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.112007+0000", 2024-09-21T14:58:30.574 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000134838 2024-09-21T14:58:30.574 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.574 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.574 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:30.574 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.112021+0000", 2024-09-21T14:58:30.575 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3984e-05 2024-09-21T14:58:30.575 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.575 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.575 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:30.575 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.112028+0000", 2024-09-21T14:58:30.575 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7850000000000003e-06 2024-09-21T14:58:30.575 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.575 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.575 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:30.575 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.112077+0000", 2024-09-21T14:58:30.575 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8257000000000002e-05 2024-09-21T14:58:30.575 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.575 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.576 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:30.576 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.112099+0000", 2024-09-21T14:58:30.576 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2685e-05 2024-09-21T14:58:30.576 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.576 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.576 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:30.576 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.113205+0000", 2024-09-21T14:58:30.576 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001105885 2024-09-21T14:58:30.576 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.576 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.576 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:30.576 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.113272+0000", 2024-09-21T14:58:30.576 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6870000000000002e-05 2024-09-21T14:58:30.576 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:30.576 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:30.576 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:30.577 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.577 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.577 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:1382 3.b 3:d2b92ac8:::rbd_data.1136515ecb6b.0000000000000011:head [delete] snapc 8=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T14:58:30.577 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:30.155959+0000", 2024-09-21T14:58:30.577 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.343524037, 2024-09-21T14:58:30.577 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0036011789999999999, 2024-09-21T14:58:30.577 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:30.577 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:30.577 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:30.577 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:30.577 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:30.577 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1382 2024-09-21T14:58:30.577 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.577 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:30.577 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.577 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:30.578 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.155959+0000", 2024-09-21T14:58:30.578 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:30.578 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.578 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.578 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:30.578 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.155959+0000", 2024-09-21T14:58:30.578 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:30.578 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.578 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.578 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:30.578 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.155956+0000", 2024-09-21T14:58:30.578 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-21T14:58:30.578 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.578 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.578 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:30.578 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.155974+0000", 2024-09-21T14:58:30.579 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7846000000000002e-05 2024-09-21T14:58:30.579 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.579 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.579 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:30.579 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.156012+0000", 2024-09-21T14:58:30.579 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7889000000000003e-05 2024-09-21T14:58:30.579 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.579 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.579 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:30.579 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.156028+0000", 2024-09-21T14:58:30.579 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.596e-05 2024-09-21T14:58:30.579 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.579 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.579 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:30.579 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.156135+0000", 2024-09-21T14:58:30.580 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000106761 2024-09-21T14:58:30.580 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.580 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.580 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:30.580 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.156255+0000", 2024-09-21T14:58:30.580 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000120536 2024-09-21T14:58:30.580 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.580 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.580 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T14:58:30.580 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.156390+0000", 2024-09-21T14:58:30.580 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013479400000000001 2024-09-21T14:58:30.580 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.580 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.580 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:58:30.580 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.158601+0000", 2024-09-21T14:58:30.581 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0022104619999999998 2024-09-21T14:58:30.581 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.581 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.581 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:30.581 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.159341+0000", 2024-09-21T14:58:30.581 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00074053900000000002 2024-09-21T14:58:30.581 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.581 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.581 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:30.581 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.159466+0000", 2024-09-21T14:58:30.581 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012490299999999999 2024-09-21T14:58:30.581 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.581 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.581 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:30.581 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.159560+0000", 2024-09-21T14:58:30.582 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4011999999999994e-05 2024-09-21T14:58:30.582 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:30.582 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:30.582 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:30.582 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.582 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.582 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:1385 3.8 e36/24 3:13ba05d9:::rbd_data.1136515ecb6b.000000000000002b:head v 36'4, mlcod=29'2)", 2024-09-21T14:58:30.582 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:30.156535+0000", 2024-09-21T14:58:30.582 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.34294846600000001, 2024-09-21T14:58:30.582 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0027476929999999998, 2024-09-21T14:58:30.582 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:30.582 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:30.582 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:30.582 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.583 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:30.583 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.156535+0000", 2024-09-21T14:58:30.583 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:30.583 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.583 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.583 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:30.583 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.156535+0000", 2024-09-21T14:58:30.583 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:30.583 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.583 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.583 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:30.584 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.156533+0000", 2024-09-21T14:58:30.584 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T14:58:30.584 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.584 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.584 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:30.584 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.156553+0000", 2024-09-21T14:58:30.584 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0123999999999999e-05 2024-09-21T14:58:30.584 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.584 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.584 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:30.584 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.156580+0000", 2024-09-21T14:58:30.584 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7219000000000001e-05 2024-09-21T14:58:30.584 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.584 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.584 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:30.585 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.156670+0000", 2024-09-21T14:58:30.585 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9652000000000002e-05 2024-09-21T14:58:30.585 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.585 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.585 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:30.585 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.156850+0000", 2024-09-21T14:58:30.585 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000180972 2024-09-21T14:58:30.585 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.585 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.585 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:30.585 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.156907+0000", 2024-09-21T14:58:30.585 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6344999999999997e-05 2024-09-21T14:58:30.585 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.585 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.586 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:30.586 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.159008+0000", 2024-09-21T14:58:30.586 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0021007999999999999 2024-09-21T14:58:30.586 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.586 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.586 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:30.586 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.159282+0000", 2024-09-21T14:58:30.586 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00027457099999999998 2024-09-21T14:58:30.586 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:30.586 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:30.586 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:30.586 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.587 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.587 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:1388 3.c e36/24 3:35bac63f:::rbd_data.1136515ecb6b.0000000000000036:head v 36'2, mlcod=0'0)", 2024-09-21T14:58:30.587 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:30.156719+0000", 2024-09-21T14:58:30.587 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.34276416799999998, 2024-09-21T14:58:30.587 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0025134559999999999, 2024-09-21T14:58:30.587 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:30.587 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:30.587 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:30.588 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.588 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:30.588 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.156719+0000", 2024-09-21T14:58:30.588 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:30.588 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.589 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.589 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:30.589 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.156719+0000", 2024-09-21T14:58:30.589 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:30.589 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.589 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.589 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:30.589 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.156717+0000", 2024-09-21T14:58:30.589 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:30.589 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.589 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.590 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:30.590 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.156734+0000", 2024-09-21T14:58:30.590 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6884e-05 2024-09-21T14:58:30.590 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.590 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.590 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:30.590 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.156759+0000", 2024-09-21T14:58:30.590 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4808e-05 2024-09-21T14:58:30.590 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.590 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.590 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:30.590 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.156773+0000", 2024-09-21T14:58:30.590 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4302e-05 2024-09-21T14:58:30.590 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.590 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.591 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:30.591 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.156978+0000", 2024-09-21T14:58:30.591 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00020522499999999999 2024-09-21T14:58:30.591 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.591 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.591 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:30.591 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.157020+0000", 2024-09-21T14:58:30.591 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1717000000000001e-05 2024-09-21T14:58:30.591 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.591 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.591 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:30.591 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.159150+0000", 2024-09-21T14:58:30.591 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0021301190000000002 2024-09-21T14:58:30.591 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:30.591 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:30.592 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:30.592 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.159232+0000", 2024-09-21T14:58:30.592 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2045000000000005e-05 2024-09-21T14:58:30.592 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:30.592 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:30.592 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:30.592 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:30.592 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:30.592 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:58:30.594 INFO:teuthology.orchestra.run.smithi032.stdout:157 write 0xdf16bb1 thru 0xdf256fc (0xeb4c bytes) 2024-09-21T14:58:30.594 INFO:teuthology.orchestra.run.smithi032.stdout:158 trunc from 0xdf256fd to 0xb77a5b3 2024-09-21T14:58:30.594 INFO:teuthology.orchestra.run.smithi032.stdout:160 punch from 0x6d51121 to 0x6d59b5f, (0x8a3e bytes) 2024-09-21T14:58:30.594 INFO:teuthology.orchestra.run.smithi032.stdout:161 writesame 0xdbb22da thru 0xdbb6c89 data_size 0x9(0x49b0 bytes) 2024-09-21T14:58:30.594 INFO:teuthology.orchestra.run.smithi032.stdout:162 write 0x7d688cc thru 0x7d6b98b (0x30c0 bytes) 2024-09-21T14:58:30.594 INFO:teuthology.orchestra.run.smithi032.stdout:164 read 0xa80143c thru 0xa80a7b6 (0x937b bytes) 2024-09-21T14:58:30.594 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:58:30.593+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:58:30.595 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T14:58:30.605 INFO:teuthology.orchestra.run.smithi032.stdout:165 trunc from 0xdbb6c8a to 0x1657d82 2024-09-21T14:58:30.626 INFO:teuthology.orchestra.run.smithi032.stdout:168 read 0x81b4b7 thru 0x8264dd (0xb027 bytes) 2024-09-21T14:58:30.627 INFO:teuthology.orchestra.run.smithi032.stdout:169 writesame 0x1554001 thru 0x155c424 data_size 0x6(0x8424 bytes) 2024-09-21T14:58:30.661 INFO:teuthology.orchestra.run.smithi032.stdout:170 write 0x89255a8 thru 0x892c7f9 (0x7252 bytes) 2024-09-21T14:58:30.667 INFO:teuthology.orchestra.run.smithi032.stdout:171 trunc from 0x892c7fa to 0x8e85b92 2024-09-21T14:58:30.679 INFO:teuthology.orchestra.run.smithi032.stdout:172 writesame 0x44f385 thru 0x45b4cd data_size 0x1(0xc149 bytes) 2024-09-21T14:58:30.694 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:58:30.693+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:58:30.709 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:58:30.709 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:58:30.709 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T14:58:30.709 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:58:30.719 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T14:58:30.741 INFO:teuthology.orchestra.run.smithi032.stdout:174 write 0xd04f8ac thru 0xd0542e3 (0x4a38 bytes) 2024-09-21T14:58:30.746 INFO:teuthology.orchestra.run.smithi032.stdout:175 write 0x51dd857 thru 0x51e4b65 (0x730f bytes) 2024-09-21T14:58:30.753 INFO:teuthology.orchestra.run.smithi032.stdout:176 write 0x3958bf3 thru 0x396137d (0x878b bytes) 2024-09-21T14:58:30.759 INFO:teuthology.orchestra.run.smithi032.stdout:177 write 0x1d704a4 thru 0x1d7b41b (0xaf78 bytes) 2024-09-21T14:58:30.762 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:30.784 INFO:teuthology.orchestra.run.smithi032.stdout:179 trunc from 0xd0542e4 to 0x118cb7a 2024-09-21T14:58:30.794 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:58:30.793+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:58:30.807 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:30.822 INFO:teuthology.orchestra.run.smithi032.stdout:182 writesame 0x1c975e3 thru 0x1c99131 data_size 0x1(0x1b4f bytes) 2024-09-21T14:58:30.846 INFO:teuthology.orchestra.run.smithi032.stdout:183 write 0x74ec785 thru 0x74f44ec (0x7d68 bytes) 2024-09-21T14:58:30.852 INFO:teuthology.orchestra.run.smithi032.stdout:184 write 0x15b3213 thru 0x15bf149 (0xbf37 bytes) 2024-09-21T14:58:30.854 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:58:30.854 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:58:30.855 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T14:58:30.855 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T14:58:30.855 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:58:30.867 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T14:58:30.879 INFO:teuthology.orchestra.run.smithi032.stdout:185 write 0xc86e687 thru 0xc87540a (0x6d84 bytes) 2024-09-21T14:58:30.895 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:58:30.893+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:58:30.900 INFO:teuthology.orchestra.run.smithi032.stdout:186 write 0xe551109 thru 0xe5594ee (0x83e6 bytes) 2024-09-21T14:58:30.905 INFO:teuthology.orchestra.run.smithi032.stdout:188 write 0xbeb7519 thru 0xbec3c77 (0xc75f bytes) 2024-09-21T14:58:30.910 INFO:teuthology.orchestra.run.smithi032.stdout:189 read 0xb2d1257 thru 0xb2d3d5c (0x2b06 bytes) 2024-09-21T14:58:30.911 INFO:teuthology.orchestra.run.smithi032.stdout:190 write 0x690588c thru 0x6909a17 (0x418c bytes) 2024-09-21T14:58:30.916 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:30.917 INFO:teuthology.orchestra.run.smithi032.stdout:194 writesame 0x1cf7697 thru 0x1cfa181 data_size 0x1(0x2aeb bytes) 2024-09-21T14:58:30.930 INFO:teuthology.orchestra.run.smithi032.stdout:195 read 0x4a3c1f thru 0x4a9b8d (0x5f6f bytes) 2024-09-21T14:58:30.930 INFO:teuthology.orchestra.run.smithi032.stdout:198 writesame 0xb4aa0b thru 0xb4f2a3 data_size 0x3f(0x4899 bytes) 2024-09-21T14:58:30.934 INFO:teuthology.orchestra.run.smithi032.stdout:199 write 0x66a8a3a thru 0x66ae232 (0x57f9 bytes) 2024-09-21T14:58:30.940 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:30.940 INFO:teuthology.orchestra.run.smithi032.stdout:201 writesame 0xdbbc014 thru 0xdbbed79 data_size 0x3(0x2d66 bytes) 2024-09-21T14:58:30.949 INFO:teuthology.orchestra.run.smithi032.stdout:202 punch from 0x2af5dda to 0x2aff72a, (0x9950 bytes) 2024-09-21T14:58:30.949 INFO:teuthology.orchestra.run.smithi032.stdout:203 writesame 0x418a72a thru 0x4190590 data_size 0xd(0x5e67 bytes) 2024-09-21T14:58:30.956 INFO:teuthology.orchestra.run.smithi032.stdout:204 write 0x1afbd31 thru 0x1b01e6d (0x613d bytes) 2024-09-21T14:58:30.967 INFO:teuthology.orchestra.run.smithi032.stdout:206 trunc from 0xe5594ef to 0x9a05e55 2024-09-21T14:58:30.986 INFO:teuthology.orchestra.run.smithi032.stdout:207 read 0x5b9f20f thru 0x5babc80 (0xca72 bytes) 2024-09-21T14:58:30.986 INFO:teuthology.orchestra.run.smithi032.stdout:212 punch from 0x958dec4 to 0x9596fcd, (0x9109 bytes) 2024-09-21T14:58:30.986 INFO:teuthology.orchestra.run.smithi032.stdout:214 writesame 0x3fb5ec8 thru 0x3fc37e8 data_size 0x1(0xd921 bytes) 2024-09-21T14:58:30.996 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:58:30.993+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:58:30.999 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:58:30.999 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T14:58:30.999 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T14:58:30.999 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T14:58:30.999 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:30.999 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:13 3.15 3:a8a0cd24:::rbd_object_map.11367ffd86fe:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T14:58:30.999 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:21.426693+0000", 2024-09-21T14:58:30.999 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 9.5678466859999993, 2024-09-21T14:58:30.999 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0024896340000000001, 2024-09-21T14:58:30.999 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:31.000 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:31.000 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:31.000 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:31.000 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:31.000 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 13 2024-09-21T14:58:31.000 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.000 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:31.000 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.000 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:31.000 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:21.426693+0000", 2024-09-21T14:58:31.000 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:31.000 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.000 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.000 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:31.001 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:21.426693+0000", 2024-09-21T14:58:31.001 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:31.001 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.001 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.001 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:31.001 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:21.426691+0000", 2024-09-21T14:58:31.001 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-21T14:58:31.001 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.001 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.001 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:31.001 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:21.426705+0000", 2024-09-21T14:58:31.002 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4398e-05 2024-09-21T14:58:31.002 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.002 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.002 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:31.002 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:21.426727+0000", 2024-09-21T14:58:31.002 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1761000000000001e-05 2024-09-21T14:58:31.002 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.002 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.002 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:31.002 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:21.426738+0000", 2024-09-21T14:58:31.002 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0987e-05 2024-09-21T14:58:31.002 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.002 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.003 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:31.003 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:21.426810+0000", 2024-09-21T14:58:31.003 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2537e-05 2024-09-21T14:58:31.003 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.003 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.003 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:31.003 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:21.427036+0000", 2024-09-21T14:58:31.003 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000226028 2024-09-21T14:58:31.003 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.003 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.003 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 6", 2024-09-21T14:58:31.003 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:21.427287+0000", 2024-09-21T14:58:31.003 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00025095899999999998 2024-09-21T14:58:31.004 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.004 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.004 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:31.004 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:21.428308+0000", 2024-09-21T14:58:31.004 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0010210130000000001 2024-09-21T14:58:31.004 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.004 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.004 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:31.004 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:21.429068+0000", 2024-09-21T14:58:31.004 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00075976400000000001 2024-09-21T14:58:31.004 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.004 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.005 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:31.005 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:21.429133+0000", 2024-09-21T14:58:31.005 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.444e-05 2024-09-21T14:58:31.005 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.005 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.005 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:31.005 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:21.429182+0000", 2024-09-21T14:58:31.005 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9700999999999997e-05 2024-09-21T14:58:31.005 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:31.005 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:31.005 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:31.005 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.005 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.005 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:337 3.3 3:c554b425:::rbd_data.11367ffd86fe.0000000000000016:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T14:58:31.006 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:25.457552+0000", 2024-09-21T14:58:31.006 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 5.5369876979999999, 2024-09-21T14:58:31.006 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0041465709999999999, 2024-09-21T14:58:31.006 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:31.006 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:58:31.006 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:31.006 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:31.006 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:31.007 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 337 2024-09-21T14:58:31.007 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.007 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:31.007 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.007 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:31.007 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:25.457552+0000", 2024-09-21T14:58:31.007 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:31.007 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.007 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.007 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:31.007 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:25.457552+0000", 2024-09-21T14:58:31.008 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:31.008 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.008 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.008 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:31.008 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:25.457549+0000", 2024-09-21T14:58:31.008 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-21T14:58:31.008 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.008 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.008 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:31.008 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:25.457566+0000", 2024-09-21T14:58:31.008 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7031e-05 2024-09-21T14:58:31.008 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.009 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.009 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:31.009 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:25.457600+0000", 2024-09-21T14:58:31.009 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4254000000000001e-05 2024-09-21T14:58:31.009 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.009 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.009 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:31.009 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:25.457617+0000", 2024-09-21T14:58:31.009 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6813999999999999e-05 2024-09-21T14:58:31.009 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.009 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.009 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:31.010 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:25.457705+0000", 2024-09-21T14:58:31.010 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.865e-05 2024-09-21T14:58:31.010 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.010 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.010 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op must wait for map", 2024-09-21T14:58:31.010 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:25.457714+0000", 2024-09-21T14:58:31.010 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0989999999999999e-06 2024-09-21T14:58:31.010 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.010 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.010 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:31.010 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:25.461574+0000", 2024-09-21T14:58:31.010 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0038602749999999998 2024-09-21T14:58:31.011 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.011 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.011 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:31.011 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:25.461698+0000", 2024-09-21T14:58:31.011 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012446000000000001 2024-09-21T14:58:31.011 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:31.011 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:31.011 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:31.011 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.011 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.011 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:344 3.15 3:a83bbcfa:::rbd_data.11367ffd86fe.0000000000000029:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T14:58:31.011 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:25.459558+0000", 2024-09-21T14:58:31.011 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 5.5349810850000001, 2024-09-21T14:58:31.012 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0030343900000000001, 2024-09-21T14:58:31.012 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:31.012 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:58:31.012 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:31.012 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:31.012 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:31.012 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 344 2024-09-21T14:58:31.012 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.012 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:31.012 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.012 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:31.012 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:25.459558+0000", 2024-09-21T14:58:31.012 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:31.012 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.013 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.013 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:31.013 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:25.459558+0000", 2024-09-21T14:58:31.013 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:31.013 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.013 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.013 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:31.013 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:25.459557+0000", 2024-09-21T14:58:31.013 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:31.013 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.013 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.013 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:31.013 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:25.459565+0000", 2024-09-21T14:58:31.014 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6619999999999994e-06 2024-09-21T14:58:31.014 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.014 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.014 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:31.014 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:25.459591+0000", 2024-09-21T14:58:31.014 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5389e-05 2024-09-21T14:58:31.014 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.014 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.014 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:31.014 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:25.459601+0000", 2024-09-21T14:58:31.014 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6889999999999999e-06 2024-09-21T14:58:31.015 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.015 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.015 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:31.015 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:25.459659+0000", 2024-09-21T14:58:31.015 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8712999999999998e-05 2024-09-21T14:58:31.015 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.015 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.015 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op must wait for map", 2024-09-21T14:58:31.015 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:25.459669+0000", 2024-09-21T14:58:31.015 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0159999999999999e-05 2024-09-21T14:58:31.015 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.015 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.015 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:31.015 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:25.462463+0000", 2024-09-21T14:58:31.016 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0027938110000000002 2024-09-21T14:58:31.016 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.016 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.016 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:31.016 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:25.462593+0000", 2024-09-21T14:58:31.016 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012943899999999999 2024-09-21T14:58:31.016 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:31.016 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:31.016 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:31.016 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.016 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.016 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:351 3.f 3:f6015596:::rbd_data.11367ffd86fe.0000000000000011:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T14:58:31.017 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:25.459617+0000", 2024-09-21T14:58:31.017 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 5.5349226219999998, 2024-09-21T14:58:31.017 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0025482040000000001, 2024-09-21T14:58:31.017 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:31.017 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:58:31.017 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:31.017 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:31.017 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:31.017 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 351 2024-09-21T14:58:31.017 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.017 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:31.017 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.017 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:31.018 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:25.459617+0000", 2024-09-21T14:58:31.018 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:31.018 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.018 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.018 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:31.018 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:25.459617+0000", 2024-09-21T14:58:31.018 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:31.018 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.018 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.018 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:31.018 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:25.459616+0000", 2024-09-21T14:58:31.018 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:31.018 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.019 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.019 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:31.019 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:25.459620+0000", 2024-09-21T14:58:31.019 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5209999999999998e-06 2024-09-21T14:58:31.019 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.019 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.019 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:31.019 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:25.459625+0000", 2024-09-21T14:58:31.019 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3800000000000002e-06 2024-09-21T14:58:31.019 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.019 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.019 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:31.019 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:25.459631+0000", 2024-09-21T14:58:31.020 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.84e-06 2024-09-21T14:58:31.020 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.020 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.020 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:31.020 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:25.459680+0000", 2024-09-21T14:58:31.020 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9419000000000002e-05 2024-09-21T14:58:31.020 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.020 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.020 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op must wait for map", 2024-09-21T14:58:31.020 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:25.459689+0000", 2024-09-21T14:58:31.020 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4309999999999994e-06 2024-09-21T14:58:31.020 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.020 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.020 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:31.021 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:25.462082+0000", 2024-09-21T14:58:31.021 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0023934389999999998 2024-09-21T14:58:31.021 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.021 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.021 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:31.021 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:25.462165+0000", 2024-09-21T14:58:31.021 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2609999999999997e-05 2024-09-21T14:58:31.021 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:31.021 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:31.021 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:31.021 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.021 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.021 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:367 3.f 3:f6015596:::rbd_data.11367ffd86fe.0000000000000011:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T14:58:31.022 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:25.459676+0000", 2024-09-21T14:58:31.022 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 5.5348633520000003, 2024-09-21T14:58:31.022 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0025619050000000002, 2024-09-21T14:58:31.022 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:31.022 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:58:31.022 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:31.022 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:31.022 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:31.022 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 367 2024-09-21T14:58:31.022 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.022 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:31.022 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.022 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:31.023 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:25.459676+0000", 2024-09-21T14:58:31.023 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:31.023 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.023 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.023 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:31.023 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:25.459676+0000", 2024-09-21T14:58:31.023 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:31.023 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.023 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.023 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:31.023 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:25.459676+0000", 2024-09-21T14:58:31.023 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:31.023 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.023 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.024 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:31.024 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:25.459679+0000", 2024-09-21T14:58:31.024 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.997e-06 2024-09-21T14:58:31.024 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.024 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.024 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:31.024 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:25.459689+0000", 2024-09-21T14:58:31.024 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2019999999999993e-06 2024-09-21T14:58:31.024 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.024 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.024 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:31.024 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:25.459694+0000", 2024-09-21T14:58:31.024 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.429e-06 2024-09-21T14:58:31.025 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.025 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.025 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:31.025 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:25.459752+0000", 2024-09-21T14:58:31.025 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7744000000000001e-05 2024-09-21T14:58:31.025 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.025 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.025 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting_for_map not empty", 2024-09-21T14:58:31.025 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:25.459757+0000", 2024-09-21T14:58:31.025 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9470000000000002e-06 2024-09-21T14:58:31.025 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.025 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.025 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:31.026 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:25.462182+0000", 2024-09-21T14:58:31.026 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.002425544 2024-09-21T14:58:31.026 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.026 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.026 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:31.026 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:25.462238+0000", 2024-09-21T14:58:31.026 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5560999999999999e-05 2024-09-21T14:58:31.026 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:31.026 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:31.026 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:31.026 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.026 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.026 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:1376 3.c 3:35bac63f:::rbd_data.1136515ecb6b.0000000000000036:head [set-alloc-hint object_size 4194304 write_size 4194304,write 257580~55629] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T14:58:31.026 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:30.106566+0000", 2024-09-21T14:58:31.027 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.88797382300000005, 2024-09-21T14:58:31.027 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0049454169999999997, 2024-09-21T14:58:31.027 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:31.027 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:31.027 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:31.027 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:31.027 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:31.027 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1376 2024-09-21T14:58:31.027 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.027 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:31.027 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.027 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:31.027 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.106566+0000", 2024-09-21T14:58:31.028 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:31.028 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.028 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.028 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:31.028 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.106566+0000", 2024-09-21T14:58:31.028 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:31.028 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.028 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.028 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:31.028 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.106564+0000", 2024-09-21T14:58:31.028 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-21T14:58:31.028 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.028 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.029 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:31.029 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.106608+0000", 2024-09-21T14:58:31.029 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4150999999999998e-05 2024-09-21T14:58:31.029 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.029 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.029 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:31.029 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.106627+0000", 2024-09-21T14:58:31.029 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9205e-05 2024-09-21T14:58:31.029 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.029 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.029 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:31.029 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.106636+0000", 2024-09-21T14:58:31.029 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4910000000000005e-06 2024-09-21T14:58:31.030 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.030 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.030 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:31.030 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.106698+0000", 2024-09-21T14:58:31.030 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2181000000000007e-05 2024-09-21T14:58:31.030 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.030 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.030 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op must wait for map", 2024-09-21T14:58:31.030 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.106702+0000", 2024-09-21T14:58:31.030 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3080000000000001e-06 2024-09-21T14:58:31.030 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.030 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.031 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:31.031 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.108858+0000", 2024-09-21T14:58:31.031 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0021558049999999998 2024-09-21T14:58:31.031 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.031 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.031 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:31.031 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.109008+0000", 2024-09-21T14:58:31.031 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015012700000000001 2024-09-21T14:58:31.031 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.031 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.031 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 7", 2024-09-21T14:58:31.031 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.109110+0000", 2024-09-21T14:58:31.031 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010155899999999999 2024-09-21T14:58:31.032 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.032 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.032 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:31.032 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.110173+0000", 2024-09-21T14:58:31.032 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0010639289999999999 2024-09-21T14:58:31.032 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.032 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.032 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:31.032 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.111428+0000", 2024-09-21T14:58:31.032 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0012550180000000001 2024-09-21T14:58:31.032 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.032 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.032 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:31.032 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.111461+0000", 2024-09-21T14:58:31.033 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2851000000000001e-05 2024-09-21T14:58:31.033 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.033 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.033 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:31.033 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.111511+0000", 2024-09-21T14:58:31.033 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9650000000000001e-05 2024-09-21T14:58:31.033 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:31.033 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:31.033 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:31.033 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.033 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.033 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:1388 3.c 3:35bac63f:::rbd_data.1136515ecb6b.0000000000000036:head [delete] snapc 8=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T14:58:31.033 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:30.153949+0000", 2024-09-21T14:58:31.034 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.84059071500000004, 2024-09-21T14:58:31.034 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0035509600000000001, 2024-09-21T14:58:31.034 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:31.034 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:31.034 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:31.034 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:31.034 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:31.034 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1388 2024-09-21T14:58:31.034 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.034 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:31.034 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.034 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:31.034 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.153949+0000", 2024-09-21T14:58:31.035 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:31.035 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.035 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.035 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:31.035 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.153949+0000", 2024-09-21T14:58:31.035 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:31.035 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.035 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.035 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:31.035 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.153946+0000", 2024-09-21T14:58:31.035 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T14:58:31.035 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.035 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.035 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:31.036 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.153956+0000", 2024-09-21T14:58:31.036 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6840000000000008e-06 2024-09-21T14:58:31.036 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.036 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.036 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:31.036 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.153975+0000", 2024-09-21T14:58:31.036 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9012999999999999e-05 2024-09-21T14:58:31.036 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.036 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.036 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:31.036 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.153985+0000", 2024-09-21T14:58:31.036 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0484e-05 2024-09-21T14:58:31.036 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.037 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.037 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:31.037 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.154048+0000", 2024-09-21T14:58:31.037 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2102999999999998e-05 2024-09-21T14:58:31.037 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.037 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.037 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:31.037 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.154115+0000", 2024-09-21T14:58:31.037 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7868999999999997e-05 2024-09-21T14:58:31.037 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.037 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.037 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 7", 2024-09-21T14:58:31.037 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.154224+0000", 2024-09-21T14:58:31.037 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010902 2024-09-21T14:58:31.038 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.038 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.038 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:31.038 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.155115+0000", 2024-09-21T14:58:31.038 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00089033099999999996 2024-09-21T14:58:31.038 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.038 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.038 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:31.038 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.157431+0000", 2024-09-21T14:58:31.038 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0023162059999999999 2024-09-21T14:58:31.038 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.038 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.038 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:31.039 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.157459+0000", 2024-09-21T14:58:31.039 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8262000000000001e-05 2024-09-21T14:58:31.039 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.039 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.039 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:31.039 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.157500+0000", 2024-09-21T14:58:31.039 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.0423000000000002e-05 2024-09-21T14:58:31.039 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:31.039 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:31.039 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:31.039 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.039 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.039 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:1425 3.1b e36/24 3:df30c629:::rbd_data.1136515ecb6b.0000000000000024:head v 36'1, mlcod=0'0)", 2024-09-21T14:58:31.039 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:30.227957+0000", 2024-09-21T14:58:31.040 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.766582596, 2024-09-21T14:58:31.040 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0025708979999999998, 2024-09-21T14:58:31.040 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:31.040 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:31.040 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:31.040 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.040 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:31.040 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.227957+0000", 2024-09-21T14:58:31.040 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:31.040 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.040 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.040 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:31.040 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.227957+0000", 2024-09-21T14:58:31.040 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:31.041 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.041 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.041 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:31.041 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.227956+0000", 2024-09-21T14:58:31.041 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:31.041 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.041 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.041 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:31.041 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.228128+0000", 2024-09-21T14:58:31.041 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000172046 2024-09-21T14:58:31.041 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.041 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.041 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:31.042 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.228151+0000", 2024-09-21T14:58:31.042 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2869000000000001e-05 2024-09-21T14:58:31.042 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.042 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.042 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:31.042 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.228163+0000", 2024-09-21T14:58:31.042 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1389999999999999e-05 2024-09-21T14:58:31.042 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.042 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.042 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:31.042 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.228255+0000", 2024-09-21T14:58:31.042 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2748e-05 2024-09-21T14:58:31.042 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.043 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.043 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:31.043 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.228296+0000", 2024-09-21T14:58:31.043 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.0240000000000001e-05 2024-09-21T14:58:31.043 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.043 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.043 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:31.043 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.230448+0000", 2024-09-21T14:58:31.043 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0021522210000000002 2024-09-21T14:58:31.043 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.043 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.043 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:31.044 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.230528+0000", 2024-09-21T14:58:31.044 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9773999999999999e-05 2024-09-21T14:58:31.044 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:31.044 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:31.044 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:31.044 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.044 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.044 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:1454 3.e e36/24 3:71be909a:::rbd_data.1136515ecb6b.000000000000002a:head v 36'4, mlcod=32'2)", 2024-09-21T14:58:31.044 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:30.362356+0000", 2024-09-21T14:58:31.044 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.63218323099999996, 2024-09-21T14:58:31.044 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0026075579999999998, 2024-09-21T14:58:31.044 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:31.045 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:31.045 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:31.045 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.045 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:31.045 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.362356+0000", 2024-09-21T14:58:31.045 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:31.045 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.045 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.045 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:31.045 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.362356+0000", 2024-09-21T14:58:31.045 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:31.045 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.045 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.046 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:31.046 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.362354+0000", 2024-09-21T14:58:31.046 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T14:58:31.046 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.046 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.046 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:31.046 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.363141+0000", 2024-09-21T14:58:31.046 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00078737900000000005 2024-09-21T14:58:31.046 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.046 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.046 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:31.046 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.363179+0000", 2024-09-21T14:58:31.046 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7749000000000001e-05 2024-09-21T14:58:31.047 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.047 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.047 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:31.047 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.363195+0000", 2024-09-21T14:58:31.047 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6555999999999999e-05 2024-09-21T14:58:31.047 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.047 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.047 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:31.047 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.363271+0000", 2024-09-21T14:58:31.047 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5998e-05 2024-09-21T14:58:31.047 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.047 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.047 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:31.048 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.363302+0000", 2024-09-21T14:58:31.048 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1093999999999997e-05 2024-09-21T14:58:31.048 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.048 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.048 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:31.048 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.364906+0000", 2024-09-21T14:58:31.048 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001603855 2024-09-21T14:58:31.048 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.048 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.048 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:31.048 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.364964+0000", 2024-09-21T14:58:31.048 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7426999999999998e-05 2024-09-21T14:58:31.048 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:31.049 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:31.049 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:31.049 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.049 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.049 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:1470 3.15 3:afe7411f:::rbd_data.1136515ecb6b.000000000000003a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 496608~27680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T14:58:31.049 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:30.414273+0000", 2024-09-21T14:58:31.049 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.58026657100000001, 2024-09-21T14:58:31.049 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0039222340000000001, 2024-09-21T14:58:31.049 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:31.049 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:31.049 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:31.049 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:31.049 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:31.049 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1470 2024-09-21T14:58:31.050 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.050 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:31.050 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.050 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:31.050 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.414273+0000", 2024-09-21T14:58:31.050 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:31.050 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.050 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.050 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:31.050 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.414273+0000", 2024-09-21T14:58:31.050 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:31.050 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.050 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.051 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:31.051 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.414271+0000", 2024-09-21T14:58:31.051 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T14:58:31.051 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.051 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.051 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:31.051 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.414298+0000", 2024-09-21T14:58:31.051 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7131000000000002e-05 2024-09-21T14:58:31.051 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.051 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.051 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:31.051 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.414323+0000", 2024-09-21T14:58:31.051 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4973999999999998e-05 2024-09-21T14:58:31.052 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.052 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.052 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:31.052 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.414334+0000", 2024-09-21T14:58:31.052 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1657e-05 2024-09-21T14:58:31.052 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.052 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.052 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:31.052 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.414474+0000", 2024-09-21T14:58:31.052 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013978100000000001 2024-09-21T14:58:31.052 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.052 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.052 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:31.052 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.414839+0000", 2024-09-21T14:58:31.053 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000365149 2024-09-21T14:58:31.053 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.053 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.053 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 6", 2024-09-21T14:58:31.053 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.415056+0000", 2024-09-21T14:58:31.053 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00021713200000000001 2024-09-21T14:58:31.053 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.053 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.053 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:31.053 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.416939+0000", 2024-09-21T14:58:31.053 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001882945 2024-09-21T14:58:31.053 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.053 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.053 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:31.054 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.418070+0000", 2024-09-21T14:58:31.054 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0011310249999999999 2024-09-21T14:58:31.054 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.054 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.054 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:31.054 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.418143+0000", 2024-09-21T14:58:31.054 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2261000000000005e-05 2024-09-21T14:58:31.054 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.054 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.054 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:31.054 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.418195+0000", 2024-09-21T14:58:31.055 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2361000000000002e-05 2024-09-21T14:58:31.055 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:31.055 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:31.055 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:31.055 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.055 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.055 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:1473 3.15 3:af2afa6a:::rbd_data.1136515ecb6b.000000000000001b:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T14:58:31.055 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:30.421730+0000", 2024-09-21T14:58:31.055 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.57280915600000004, 2024-09-21T14:58:31.055 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021478937, 2024-09-21T14:58:31.055 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:31.055 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:31.055 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:31.055 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:31.056 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:31.056 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1473 2024-09-21T14:58:31.056 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.056 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:31.056 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.056 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:31.056 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421730+0000", 2024-09-21T14:58:31.056 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:31.056 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.056 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.056 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:31.056 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421730+0000", 2024-09-21T14:58:31.056 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:31.057 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.057 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.057 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:31.057 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421728+0000", 2024-09-21T14:58:31.057 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-21T14:58:31.057 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.057 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.057 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:31.057 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421739+0000", 2024-09-21T14:58:31.057 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0502e-05 2024-09-21T14:58:31.057 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.057 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.057 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:31.058 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421762+0000", 2024-09-21T14:58:31.058 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3405999999999999e-05 2024-09-21T14:58:31.058 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.058 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.058 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:31.058 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421773+0000", 2024-09-21T14:58:31.058 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0744000000000001e-05 2024-09-21T14:58:31.058 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.058 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.058 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:31.058 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421838+0000", 2024-09-21T14:58:31.058 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5486000000000003e-05 2024-09-21T14:58:31.058 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.059 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.059 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:31.059 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421955+0000", 2024-09-21T14:58:31.059 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000116765 2024-09-21T14:58:31.059 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.059 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.059 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 6", 2024-09-21T14:58:31.059 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.425572+0000", 2024-09-21T14:58:31.059 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0036171380000000002 2024-09-21T14:58:31.059 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.059 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.059 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:31.060 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.442973+0000", 2024-09-21T14:58:31.060 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.017400268 2024-09-21T14:58:31.060 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.060 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.060 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:31.060 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.442996+0000", 2024-09-21T14:58:31.060 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3139e-05 2024-09-21T14:58:31.060 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.060 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.060 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:31.060 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.443079+0000", 2024-09-21T14:58:31.060 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2832999999999998e-05 2024-09-21T14:58:31.060 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.061 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.061 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:31.061 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.443209+0000", 2024-09-21T14:58:31.061 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013052499999999999 2024-09-21T14:58:31.061 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:31.061 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:31.061 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:31.061 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.061 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.061 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:1482 3.15 3:af2afa6a:::rbd_data.1136515ecb6b.000000000000001b:head [delete] snapc 8=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T14:58:31.061 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:30.481917+0000", 2024-09-21T14:58:31.061 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.51262256699999997, 2024-09-21T14:58:31.061 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0036932309999999999, 2024-09-21T14:58:31.061 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:31.062 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:31.062 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:31.062 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:31.062 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:31.062 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1482 2024-09-21T14:58:31.062 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.062 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:31.062 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.062 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:31.062 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.481917+0000", 2024-09-21T14:58:31.062 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:31.062 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.063 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.063 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:31.063 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.481917+0000", 2024-09-21T14:58:31.063 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:31.063 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.063 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.063 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:31.063 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.481914+0000", 2024-09-21T14:58:31.063 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T14:58:31.063 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.063 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.064 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:31.064 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.481925+0000", 2024-09-21T14:58:31.064 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0964999999999999e-05 2024-09-21T14:58:31.064 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.064 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.064 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:31.064 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.481944+0000", 2024-09-21T14:58:31.064 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8482999999999999e-05 2024-09-21T14:58:31.064 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.064 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.064 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:31.064 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.481961+0000", 2024-09-21T14:58:31.065 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7159999999999998e-05 2024-09-21T14:58:31.065 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.065 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.065 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:31.065 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.482015+0000", 2024-09-21T14:58:31.065 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4249999999999997e-05 2024-09-21T14:58:31.065 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.065 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.065 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:31.065 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.482114+0000", 2024-09-21T14:58:31.065 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8684999999999995e-05 2024-09-21T14:58:31.065 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.065 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.066 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 6", 2024-09-21T14:58:31.066 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.482184+0000", 2024-09-21T14:58:31.066 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0604000000000004e-05 2024-09-21T14:58:31.066 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.066 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.066 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:31.066 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.483583+0000", 2024-09-21T14:58:31.066 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0013982440000000001 2024-09-21T14:58:31.066 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.066 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.066 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:31.067 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.485547+0000", 2024-09-21T14:58:31.067 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0019639100000000001 2024-09-21T14:58:31.067 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.067 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.067 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:31.067 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.485585+0000", 2024-09-21T14:58:31.067 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.8804999999999999e-05 2024-09-21T14:58:31.067 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.067 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.067 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:31.067 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.485610+0000", 2024-09-21T14:58:31.067 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4678999999999999e-05 2024-09-21T14:58:31.067 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:31.068 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:31.068 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:31.068 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.068 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.068 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:1486 3.15 3:afe7411f:::rbd_data.1136515ecb6b.000000000000003a:head [delete] snapc 8=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T14:58:31.068 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:30.481989+0000", 2024-09-21T14:58:31.068 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.51254989299999998, 2024-09-21T14:58:31.068 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0040307399999999997, 2024-09-21T14:58:31.068 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:31.068 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:31.068 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:31.068 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:31.068 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:31.069 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1486 2024-09-21T14:58:31.069 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.069 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:31.069 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.069 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:31.069 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.481989+0000", 2024-09-21T14:58:31.069 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:31.069 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.069 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.069 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:31.069 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.481989+0000", 2024-09-21T14:58:31.069 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:31.069 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.069 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.070 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:31.070 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.481989+0000", 2024-09-21T14:58:31.070 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:58:31.070 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.070 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.070 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:31.070 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.481994+0000", 2024-09-21T14:58:31.070 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8330000000000001e-06 2024-09-21T14:58:31.070 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.070 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.070 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:31.070 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.482001+0000", 2024-09-21T14:58:31.070 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4250000000000001e-06 2024-09-21T14:58:31.070 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.071 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.071 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:31.071 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.482007+0000", 2024-09-21T14:58:31.071 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8860000000000001e-06 2024-09-21T14:58:31.071 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.071 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.071 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:31.071 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.482485+0000", 2024-09-21T14:58:31.071 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00047820000000000002 2024-09-21T14:58:31.071 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.071 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.071 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:31.071 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.482735+0000", 2024-09-21T14:58:31.072 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000249862 2024-09-21T14:58:31.072 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.072 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.072 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 6", 2024-09-21T14:58:31.072 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.482974+0000", 2024-09-21T14:58:31.072 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00023892999999999999 2024-09-21T14:58:31.072 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.072 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.072 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:31.072 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.484286+0000", 2024-09-21T14:58:31.072 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0013121470000000001 2024-09-21T14:58:31.072 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.072 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.072 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:31.073 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.485925+0000", 2024-09-21T14:58:31.073 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001639148 2024-09-21T14:58:31.073 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.073 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.073 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:31.073 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.485978+0000", 2024-09-21T14:58:31.073 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2332e-05 2024-09-21T14:58:31.073 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.073 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.073 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:31.073 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.486020+0000", 2024-09-21T14:58:31.073 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2710999999999998e-05 2024-09-21T14:58:31.074 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:31.074 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:31.074 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:31.074 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.074 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.074 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:1508 3.c 3:35bac63f:::rbd_data.1136515ecb6b.0000000000000036:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3763121~60236] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T14:58:31.074 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:30.528427+0000", 2024-09-21T14:58:31.074 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.466112099, 2024-09-21T14:58:31.074 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.003484348, 2024-09-21T14:58:31.074 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:31.074 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:31.074 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:31.074 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:31.075 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:31.075 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1508 2024-09-21T14:58:31.075 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.075 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:31.075 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.075 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:31.075 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.528427+0000", 2024-09-21T14:58:31.075 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:31.075 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.075 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.075 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:31.075 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.528427+0000", 2024-09-21T14:58:31.075 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:31.076 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.076 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.076 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:31.076 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.528425+0000", 2024-09-21T14:58:31.076 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-21T14:58:31.076 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.076 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.076 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:31.076 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.528485+0000", 2024-09-21T14:58:31.076 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0004999999999999e-05 2024-09-21T14:58:31.076 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.076 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.076 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:31.076 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.528513+0000", 2024-09-21T14:58:31.077 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7605000000000001e-05 2024-09-21T14:58:31.077 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.077 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.077 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:31.077 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.528525+0000", 2024-09-21T14:58:31.077 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2164000000000001e-05 2024-09-21T14:58:31.077 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.077 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.077 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:31.077 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.528643+0000", 2024-09-21T14:58:31.077 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000118316 2024-09-21T14:58:31.077 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.077 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.077 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:31.078 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.528864+0000", 2024-09-21T14:58:31.078 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00022093199999999999 2024-09-21T14:58:31.078 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.078 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.078 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 7", 2024-09-21T14:58:31.078 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.529243+0000", 2024-09-21T14:58:31.078 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000378346 2024-09-21T14:58:31.078 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.078 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.078 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:31.078 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.530946+0000", 2024-09-21T14:58:31.078 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001703567 2024-09-21T14:58:31.079 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.079 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.079 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:31.079 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.531682+0000", 2024-09-21T14:58:31.079 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00073578000000000001 2024-09-21T14:58:31.079 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.079 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.079 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:31.079 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.531816+0000", 2024-09-21T14:58:31.079 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013375999999999999 2024-09-21T14:58:31.079 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.079 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.080 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:31.080 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.531912+0000", 2024-09-21T14:58:31.080 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5835000000000007e-05 2024-09-21T14:58:31.080 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:31.080 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:31.080 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:31.080 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.080 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.080 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:1510 3.c 3:35bac63f:::rbd_data.1136515ecb6b.0000000000000036:head [delete] snapc 8=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T14:58:31.080 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:30.539674+0000", 2024-09-21T14:58:31.080 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.454865507, 2024-09-21T14:58:31.080 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0023304100000000002, 2024-09-21T14:58:31.080 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:31.081 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:31.081 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:31.081 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:31.081 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:31.081 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1510 2024-09-21T14:58:31.081 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.081 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:31.081 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.081 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:31.081 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.539674+0000", 2024-09-21T14:58:31.081 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:31.081 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.081 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.082 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:31.082 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.539674+0000", 2024-09-21T14:58:31.082 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:31.082 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.082 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.082 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:31.082 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.539671+0000", 2024-09-21T14:58:31.082 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T14:58:31.082 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.082 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.082 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:31.082 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.539686+0000", 2024-09-21T14:58:31.083 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4457000000000001e-05 2024-09-21T14:58:31.083 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.083 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.083 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:31.083 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.539719+0000", 2024-09-21T14:58:31.083 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2907999999999997e-05 2024-09-21T14:58:31.083 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.083 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.083 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:31.083 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.539734+0000", 2024-09-21T14:58:31.083 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.556e-05 2024-09-21T14:58:31.083 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.084 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.084 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:31.084 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.539802+0000", 2024-09-21T14:58:31.084 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8140999999999997e-05 2024-09-21T14:58:31.084 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.084 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.084 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:31.084 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.539898+0000", 2024-09-21T14:58:31.084 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5519999999999993e-05 2024-09-21T14:58:31.084 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.084 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.084 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 7", 2024-09-21T14:58:31.084 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.540008+0000", 2024-09-21T14:58:31.085 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000110142 2024-09-21T14:58:31.085 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.085 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.085 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:31.085 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.540936+0000", 2024-09-21T14:58:31.085 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00092826400000000004 2024-09-21T14:58:31.085 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.085 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.085 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:31.085 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.541799+0000", 2024-09-21T14:58:31.085 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00086268100000000002 2024-09-21T14:58:31.085 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.086 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.086 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:31.086 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.541924+0000", 2024-09-21T14:58:31.086 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012483199999999999 2024-09-21T14:58:31.086 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.086 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.086 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:31.086 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.542004+0000", 2024-09-21T14:58:31.086 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0351000000000004e-05 2024-09-21T14:58:31.086 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:31.086 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:31.086 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:31.086 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.087 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.087 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:1531 3.15 3:aca5faff:::rbd_data.1136515ecb6b.000000000000001e:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3573964~12480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T14:58:31.087 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:30.578477+0000", 2024-09-21T14:58:31.087 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.41606257600000002, 2024-09-21T14:58:31.087 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0039711809999999998, 2024-09-21T14:58:31.087 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:31.087 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:31.087 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:31.087 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:31.087 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:31.087 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1531 2024-09-21T14:58:31.087 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.088 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:31.088 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.088 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:31.088 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.578477+0000", 2024-09-21T14:58:31.088 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:31.088 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.088 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.088 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:31.088 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.578477+0000", 2024-09-21T14:58:31.088 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:31.088 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.088 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.089 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:31.089 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.578474+0000", 2024-09-21T14:58:31.089 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-21T14:58:31.089 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.089 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.089 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:31.089 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.578523+0000", 2024-09-21T14:58:31.089 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9076000000000002e-05 2024-09-21T14:58:31.089 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.089 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.089 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:31.089 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.578560+0000", 2024-09-21T14:58:31.089 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6996000000000003e-05 2024-09-21T14:58:31.090 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.090 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.090 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:31.090 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.578576+0000", 2024-09-21T14:58:31.090 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6079999999999999e-05 2024-09-21T14:58:31.090 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.090 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.090 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:31.090 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.578693+0000", 2024-09-21T14:58:31.090 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00011671 2024-09-21T14:58:31.090 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.090 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.091 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:31.091 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.578926+0000", 2024-09-21T14:58:31.091 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00023317499999999999 2024-09-21T14:58:31.091 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.091 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.091 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 6", 2024-09-21T14:58:31.091 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.579138+0000", 2024-09-21T14:58:31.091 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00021226400000000001 2024-09-21T14:58:31.091 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.091 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.091 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:31.092 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.580886+0000", 2024-09-21T14:58:31.092 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0017482229999999999 2024-09-21T14:58:31.092 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.092 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.092 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:31.092 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.582319+0000", 2024-09-21T14:58:31.092 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001432032 2024-09-21T14:58:31.092 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.092 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.092 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:31.092 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.582387+0000", 2024-09-21T14:58:31.092 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8194999999999993e-05 2024-09-21T14:58:31.092 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.093 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.093 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:31.093 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.582448+0000", 2024-09-21T14:58:31.093 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1273999999999996e-05 2024-09-21T14:58:31.093 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:31.093 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:31.093 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:31.093 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.093 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.093 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:1546 3.3 3:c7c3230f:::rbd_data.1136515ecb6b.0000000000000006:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T14:58:31.093 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:30.629102+0000", 2024-09-21T14:58:31.093 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.36543749800000003, 2024-09-21T14:58:31.093 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0047496329999999996, 2024-09-21T14:58:31.094 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:31.094 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:31.094 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:31.094 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:31.094 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:31.094 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1546 2024-09-21T14:58:31.094 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.094 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:31.094 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.094 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:31.094 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.629102+0000", 2024-09-21T14:58:31.094 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:31.094 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.095 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.095 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:31.095 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.629102+0000", 2024-09-21T14:58:31.095 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:31.095 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.095 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.095 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:31.095 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.629100+0000", 2024-09-21T14:58:31.095 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-21T14:58:31.095 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.095 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.095 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:31.095 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.629111+0000", 2024-09-21T14:58:31.096 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1181e-05 2024-09-21T14:58:31.096 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.096 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.096 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:31.096 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.629134+0000", 2024-09-21T14:58:31.096 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.281e-05 2024-09-21T14:58:31.096 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.096 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.096 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:31.096 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.629145+0000", 2024-09-21T14:58:31.096 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0913999999999999e-05 2024-09-21T14:58:31.096 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.096 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.097 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:31.097 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.629201+0000", 2024-09-21T14:58:31.097 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6382999999999999e-05 2024-09-21T14:58:31.097 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.097 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.097 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:31.097 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.629342+0000", 2024-09-21T14:58:31.097 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014090500000000001 2024-09-21T14:58:31.097 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.097 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.097 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:58:31.097 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.630140+0000", 2024-09-21T14:58:31.097 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00079839899999999998 2024-09-21T14:58:31.098 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.098 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.098 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:31.098 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.633546+0000", 2024-09-21T14:58:31.098 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0034059950000000002 2024-09-21T14:58:31.098 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.098 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.098 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:31.098 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.633741+0000", 2024-09-21T14:58:31.098 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00019487500000000001 2024-09-21T14:58:31.098 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.098 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.098 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:31.099 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.633805+0000", 2024-09-21T14:58:31.099 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3787000000000004e-05 2024-09-21T14:58:31.099 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.099 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.099 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:31.099 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.633852+0000", 2024-09-21T14:58:31.099 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6502999999999998e-05 2024-09-21T14:58:31.099 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:31.099 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:31.099 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:31.099 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.099 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.099 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:1644 3.15 3:afe7411f:::rbd_data.1136515ecb6b.000000000000003a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1380617~33766] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T14:58:31.100 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:30.901572+0000", 2024-09-21T14:58:31.100 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.092967039000000001, 2024-09-21T14:58:31.100 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.002860277, 2024-09-21T14:58:31.100 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:31.100 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:31.100 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:31.100 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:31.100 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:31.100 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1644 2024-09-21T14:58:31.100 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.100 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:31.100 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.100 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:31.100 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.901572+0000", 2024-09-21T14:58:31.101 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:31.101 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.101 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.101 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:31.101 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.901572+0000", 2024-09-21T14:58:31.101 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:31.101 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.101 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.101 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:31.101 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.901570+0000", 2024-09-21T14:58:31.101 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T14:58:31.101 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.101 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.101 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:31.102 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.901607+0000", 2024-09-21T14:58:31.102 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7605000000000003e-05 2024-09-21T14:58:31.102 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.102 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.102 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:31.102 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.901640+0000", 2024-09-21T14:58:31.102 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2319e-05 2024-09-21T14:58:31.102 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.102 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.102 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:31.102 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.901654+0000", 2024-09-21T14:58:31.102 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.428e-05 2024-09-21T14:58:31.102 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.103 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.103 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:31.103 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.901720+0000", 2024-09-21T14:58:31.103 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5493999999999993e-05 2024-09-21T14:58:31.103 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.103 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.103 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:31.103 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.901816+0000", 2024-09-21T14:58:31.103 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6328000000000003e-05 2024-09-21T14:58:31.103 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.103 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.103 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 6", 2024-09-21T14:58:31.104 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.901987+0000", 2024-09-21T14:58:31.104 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017092600000000001 2024-09-21T14:58:31.104 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.104 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.104 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:31.104 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.903154+0000", 2024-09-21T14:58:31.104 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0011669549999999999 2024-09-21T14:58:31.104 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.104 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.104 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:31.104 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.904317+0000", 2024-09-21T14:58:31.104 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0011630099999999999 2024-09-21T14:58:31.105 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.105 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.105 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:31.105 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.904383+0000", 2024-09-21T14:58:31.105 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6017000000000002e-05 2024-09-21T14:58:31.105 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.105 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.105 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:31.105 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.904433+0000", 2024-09-21T14:58:31.105 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9793999999999998e-05 2024-09-21T14:58:31.105 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:31.105 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:31.105 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:31.105 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.106 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.106 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:1661 3.3 3:c7c3230f:::rbd_data.1136515ecb6b.0000000000000006:head [write 2621440~7790] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T14:58:31.106 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:30.955864+0000", 2024-09-21T14:58:31.106 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.038675653999999997, 2024-09-21T14:58:31.106 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.002351452, 2024-09-21T14:58:31.106 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:31.106 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:31.106 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:31.106 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:31.106 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:31.106 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1661 2024-09-21T14:58:31.106 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.106 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:31.107 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.107 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:31.107 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.955864+0000", 2024-09-21T14:58:31.107 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:31.107 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.107 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.107 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:31.107 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.955864+0000", 2024-09-21T14:58:31.107 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:31.107 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.107 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.107 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:31.107 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.955861+0000", 2024-09-21T14:58:31.107 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-21T14:58:31.108 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.108 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.108 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:31.108 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.955892+0000", 2024-09-21T14:58:31.108 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1458999999999998e-05 2024-09-21T14:58:31.108 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.108 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.108 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:31.108 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.955926+0000", 2024-09-21T14:58:31.108 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3339e-05 2024-09-21T14:58:31.108 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.108 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.108 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:31.108 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.955941+0000", 2024-09-21T14:58:31.109 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5325999999999999e-05 2024-09-21T14:58:31.109 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.109 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.109 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:31.109 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.956024+0000", 2024-09-21T14:58:31.109 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3110999999999998e-05 2024-09-21T14:58:31.109 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.109 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.109 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:31.109 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.956159+0000", 2024-09-21T14:58:31.109 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013503300000000001 2024-09-21T14:58:31.109 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.109 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.110 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:58:31.110 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.956351+0000", 2024-09-21T14:58:31.110 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00019205399999999999 2024-09-21T14:58:31.110 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.110 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.110 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:31.110 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.957402+0000", 2024-09-21T14:58:31.110 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001050346 2024-09-21T14:58:31.110 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.110 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.110 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:31.110 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.958155+0000", 2024-09-21T14:58:31.110 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00075382800000000001 2024-09-21T14:58:31.111 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.111 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.111 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:31.111 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.958187+0000", 2024-09-21T14:58:31.111 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1946000000000002e-05 2024-09-21T14:58:31.111 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.111 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.111 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:31.111 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.958215+0000", 2024-09-21T14:58:31.111 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7804e-05 2024-09-21T14:58:31.111 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:31.111 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:31.111 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:31.111 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.112 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.112 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:1666 3.15 3:afe7411f:::rbd_data.1136515ecb6b.000000000000003a:head [delete] snapc 8=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T14:58:31.112 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:30.970385+0000", 2024-09-21T14:58:31.112 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.024153994000000002, 2024-09-21T14:58:31.112 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0026097400000000001, 2024-09-21T14:58:31.112 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:31.112 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:31.112 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:31.112 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:31.112 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:31.112 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1666 2024-09-21T14:58:31.112 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.112 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:31.112 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.113 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:31.113 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.970385+0000", 2024-09-21T14:58:31.113 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:31.113 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.113 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.113 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:31.113 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.970385+0000", 2024-09-21T14:58:31.113 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:31.113 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.113 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.113 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:31.113 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.970383+0000", 2024-09-21T14:58:31.113 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T14:58:31.114 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.114 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.114 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:31.114 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.970397+0000", 2024-09-21T14:58:31.114 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4085e-05 2024-09-21T14:58:31.114 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.114 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.114 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:31.114 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.970418+0000", 2024-09-21T14:58:31.114 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0437000000000001e-05 2024-09-21T14:58:31.114 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.114 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.114 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:31.115 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.970439+0000", 2024-09-21T14:58:31.115 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1101e-05 2024-09-21T14:58:31.115 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.115 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.115 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:31.115 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.970504+0000", 2024-09-21T14:58:31.115 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4740999999999996e-05 2024-09-21T14:58:31.115 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.115 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.115 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:31.115 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.970586+0000", 2024-09-21T14:58:31.115 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1976000000000003e-05 2024-09-21T14:58:31.115 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.116 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.116 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 6", 2024-09-21T14:58:31.116 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.970692+0000", 2024-09-21T14:58:31.116 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000106427 2024-09-21T14:58:31.116 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.116 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.116 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:31.116 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.971894+0000", 2024-09-21T14:58:31.116 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0012021810000000001 2024-09-21T14:58:31.116 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.116 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.116 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:31.117 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.972932+0000", 2024-09-21T14:58:31.117 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001037493 2024-09-21T14:58:31.117 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.117 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.117 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:31.117 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.972961+0000", 2024-09-21T14:58:31.117 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9492000000000002e-05 2024-09-21T14:58:31.117 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.117 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.117 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:31.117 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.972995+0000", 2024-09-21T14:58:31.117 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3997e-05 2024-09-21T14:58:31.117 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:31.118 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:31.118 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:31.118 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:31.118 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:31.118 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:58:31.120 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T14:58:31.121 INFO:teuthology.orchestra.run.smithi032.stdout:215 write 0x50fe101 thru 0x5101e27 (0x3d27 bytes) 2024-09-21T14:58:31.121 INFO:teuthology.orchestra.run.smithi032.stdout:216 write 0x92aac2f thru 0x92b688d (0xbc5f bytes) 2024-09-21T14:58:31.121 INFO:teuthology.orchestra.run.smithi032.stdout:218 read 0x3af9f64 thru 0x3afd8f8 (0x3995 bytes) 2024-09-21T14:58:31.121 INFO:teuthology.orchestra.run.smithi032.stdout:219 writesame 0x7354d42 thru 0x735ead6 data_size 0x33(0x9d95 bytes) 2024-09-21T14:58:31.121 INFO:teuthology.orchestra.run.smithi032.stdout:220 writesame 0xe305c60 thru 0xe31407c data_size 0x1(0xe41d bytes) 2024-09-21T14:58:31.121 INFO:teuthology.orchestra.run.smithi032.stdout:222 write 0x3dbaa5c thru 0x3dbe156 (0x36fb bytes) 2024-09-21T14:58:31.121 INFO:teuthology.orchestra.run.smithi032.stdout:225 write 0x949873e thru 0x949cd83 (0x4646 bytes) 2024-09-21T14:58:31.121 INFO:teuthology.orchestra.run.smithi032.stdout:226 read 0xaee7180 thru 0xaee73b7 (0x238 bytes) 2024-09-21T14:58:31.121 INFO:teuthology.orchestra.run.smithi032.stdout:227 clone 3 order 18 su 16384 sc 7 2024-09-21T14:58:31.121 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:58:31.116+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:58:31.220 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:58:31.215+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:58:31.286 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:58:31.286 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:58:31.286 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T14:58:31.286 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:58:31.296 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T14:58:31.321 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:58:31.315+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:58:31.407 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:58:31.407 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:58:31.407 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T14:58:31.407 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T14:58:31.407 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:58:31.417 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T14:58:31.422 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:58:31.421+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:58:31.523 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:58:31.521+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:58:31.581 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:58:31.581 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T14:58:31.581 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T14:58:31.581 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T14:58:31.581 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.582 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T14:58:31.582 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.205754+0000", 2024-09-21T14:58:31.582 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 28.37027286, 2024-09-21T14:58:31.582 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.41117094199999998, 2024-09-21T14:58:31.582 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:31.582 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:31.582 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:31.582 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:31.582 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:31.582 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1 2024-09-21T14:58:31.582 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.582 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:31.582 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.582 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:31.583 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205754+0000", 2024-09-21T14:58:31.583 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:31.583 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.583 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.583 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:31.583 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205754+0000", 2024-09-21T14:58:31.583 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:31.583 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.583 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.583 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:31.583 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205752+0000", 2024-09-21T14:58:31.583 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T14:58:31.583 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.584 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.584 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:31.584 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205765+0000", 2024-09-21T14:58:31.584 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3230999999999999e-05 2024-09-21T14:58:31.584 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.584 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.584 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:31.584 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205782+0000", 2024-09-21T14:58:31.584 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6634e-05 2024-09-21T14:58:31.584 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.584 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.584 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:31.584 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205793+0000", 2024-09-21T14:58:31.584 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1297e-05 2024-09-21T14:58:31.585 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.585 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.585 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:31.585 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205867+0000", 2024-09-21T14:58:31.585 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4269000000000003e-05 2024-09-21T14:58:31.585 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.585 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.585 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T14:58:31.585 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205877+0000", 2024-09-21T14:58:31.585 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2690000000000005e-06 2024-09-21T14:58:31.585 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.585 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.585 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:31.585 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.613700+0000", 2024-09-21T14:58:31.586 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.407823455 2024-09-21T14:58:31.586 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.586 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.586 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:31.586 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.614203+0000", 2024-09-21T14:58:31.586 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00050311500000000005 2024-09-21T14:58:31.586 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.586 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.586 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:31.586 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.614750+0000", 2024-09-21T14:58:31.586 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00054738699999999998 2024-09-21T14:58:31.586 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.586 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.587 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:31.587 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616367+0000", 2024-09-21T14:58:31.587 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001616634 2024-09-21T14:58:31.587 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.587 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.587 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:31.587 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616574+0000", 2024-09-21T14:58:31.587 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00020668499999999999 2024-09-21T14:58:31.587 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.587 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.587 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:31.587 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616802+0000", 2024-09-21T14:58:31.588 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00022817900000000001 2024-09-21T14:58:31.588 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.588 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.588 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:31.588 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616925+0000", 2024-09-21T14:58:31.588 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012309699999999999 2024-09-21T14:58:31.588 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:31.588 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:31.588 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:31.588 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.588 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.588 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T14:58:31.589 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.666480+0000", 2024-09-21T14:58:31.589 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 27.909547404000001, 2024-09-21T14:58:31.589 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0098753969999999993, 2024-09-21T14:58:31.589 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:31.589 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:31.589 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:31.589 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:31.589 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:31.589 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 35 2024-09-21T14:58:31.589 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.589 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:31.589 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.589 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:31.589 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.666480+0000", 2024-09-21T14:58:31.590 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:31.590 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.590 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.590 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:31.590 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.666480+0000", 2024-09-21T14:58:31.590 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:31.590 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.590 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.590 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:31.590 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.666478+0000", 2024-09-21T14:58:31.590 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:31.590 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.590 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.591 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:31.591 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.666626+0000", 2024-09-21T14:58:31.591 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014799100000000001 2024-09-21T14:58:31.591 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.591 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.591 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:31.591 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.666675+0000", 2024-09-21T14:58:31.591 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9086999999999999e-05 2024-09-21T14:58:31.591 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.591 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.591 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:31.591 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.666696+0000", 2024-09-21T14:58:31.591 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0448000000000001e-05 2024-09-21T14:58:31.591 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.592 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.592 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:31.592 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.672759+0000", 2024-09-21T14:58:31.592 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0060628410000000002 2024-09-21T14:58:31.592 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.592 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.592 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:31.592 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.672891+0000", 2024-09-21T14:58:31.592 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013294700000000001 2024-09-21T14:58:31.592 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.592 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.592 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:31.592 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.673110+0000", 2024-09-21T14:58:31.592 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00021842500000000001 2024-09-21T14:58:31.593 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.593 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.593 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:31.593 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.676136+0000", 2024-09-21T14:58:31.593 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0030259739999999999 2024-09-21T14:58:31.593 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.593 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.593 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:31.593 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.676192+0000", 2024-09-21T14:58:31.593 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5795999999999999e-05 2024-09-21T14:58:31.593 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.593 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.593 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:31.593 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.676255+0000", 2024-09-21T14:58:31.593 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3548999999999998e-05 2024-09-21T14:58:31.594 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.594 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.594 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:31.594 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.676355+0000", 2024-09-21T14:58:31.594 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9786999999999999e-05 2024-09-21T14:58:31.594 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:31.594 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:31.594 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:31.594 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.594 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.594 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T14:58:31.594 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.666714+0000", 2024-09-21T14:58:31.594 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 27.909312783000001, 2024-09-21T14:58:31.594 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.010286645000000001, 2024-09-21T14:58:31.594 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:31.595 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:31.595 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:31.595 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:31.595 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:31.595 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 36 2024-09-21T14:58:31.595 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.595 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:31.595 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.595 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:31.595 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.666714+0000", 2024-09-21T14:58:31.595 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:31.595 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.595 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.596 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:31.596 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.666714+0000", 2024-09-21T14:58:31.596 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:31.596 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.596 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.596 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:31.596 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.666713+0000", 2024-09-21T14:58:31.596 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:31.596 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.596 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.596 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:31.596 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.666724+0000", 2024-09-21T14:58:31.596 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0939e-05 2024-09-21T14:58:31.596 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.597 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.597 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:31.597 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.666752+0000", 2024-09-21T14:58:31.597 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8725e-05 2024-09-21T14:58:31.597 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.597 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.597 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:31.597 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.666768+0000", 2024-09-21T14:58:31.597 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5274e-05 2024-09-21T14:58:31.597 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.597 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.597 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:31.597 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.674731+0000", 2024-09-21T14:58:31.598 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0079634179999999999 2024-09-21T14:58:31.598 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.598 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.598 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:31.598 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.674840+0000", 2024-09-21T14:58:31.598 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000108639 2024-09-21T14:58:31.598 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.598 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.598 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:31.598 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.674959+0000", 2024-09-21T14:58:31.598 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00011884599999999999 2024-09-21T14:58:31.598 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.599 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.599 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:31.599 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.676146+0000", 2024-09-21T14:58:31.599 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0011873680000000001 2024-09-21T14:58:31.599 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.599 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.599 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:31.599 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.676924+0000", 2024-09-21T14:58:31.599 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00077794099999999998 2024-09-21T14:58:31.599 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.600 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.600 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:31.600 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.676966+0000", 2024-09-21T14:58:31.600 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2172999999999998e-05 2024-09-21T14:58:31.600 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.600 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.600 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:31.600 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.677001+0000", 2024-09-21T14:58:31.600 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4755999999999997e-05 2024-09-21T14:58:31.600 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:31.600 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:31.600 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:31.600 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.600 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.601 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T14:58:31.601 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.666789+0000", 2024-09-21T14:58:31.601 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 27.909237713, 2024-09-21T14:58:31.601 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.010759082999999999, 2024-09-21T14:58:31.601 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:31.601 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:31.601 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:31.601 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:31.601 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:31.601 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 37 2024-09-21T14:58:31.601 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.601 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:31.601 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.602 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:31.602 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.666789+0000", 2024-09-21T14:58:31.602 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:31.602 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.602 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.602 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:31.602 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.666789+0000", 2024-09-21T14:58:31.602 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:31.602 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.602 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.602 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:31.602 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.666788+0000", 2024-09-21T14:58:31.603 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:58:31.603 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.603 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.603 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:31.603 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.666799+0000", 2024-09-21T14:58:31.603 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0532e-05 2024-09-21T14:58:31.603 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.603 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.603 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:31.603 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.666822+0000", 2024-09-21T14:58:31.603 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.323e-05 2024-09-21T14:58:31.604 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.604 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.604 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:31.604 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.666875+0000", 2024-09-21T14:58:31.604 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3071000000000001e-05 2024-09-21T14:58:31.604 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.604 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.604 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:31.604 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.675669+0000", 2024-09-21T14:58:31.604 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0087942860000000001 2024-09-21T14:58:31.604 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.604 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.604 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:31.605 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.675749+0000", 2024-09-21T14:58:31.605 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9260999999999999e-05 2024-09-21T14:58:31.605 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.605 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.605 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:31.605 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.675872+0000", 2024-09-21T14:58:31.605 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000123072 2024-09-21T14:58:31.605 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.605 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.605 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:31.605 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.676749+0000", 2024-09-21T14:58:31.605 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00087719600000000001 2024-09-21T14:58:31.605 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.606 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.606 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:31.606 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.677408+0000", 2024-09-21T14:58:31.606 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00065863300000000005 2024-09-21T14:58:31.606 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.606 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.606 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:31.606 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.677492+0000", 2024-09-21T14:58:31.606 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4934999999999999e-05 2024-09-21T14:58:31.606 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.606 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.606 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:31.606 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.677548+0000", 2024-09-21T14:58:31.606 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5881999999999998e-05 2024-09-21T14:58:31.607 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:31.607 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:31.607 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:31.607 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.607 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.607 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T14:58:31.607 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.748110+0000", 2024-09-21T14:58:31.607 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 27.827916588000001, 2024-09-21T14:58:31.607 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.010071220000000001, 2024-09-21T14:58:31.607 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:31.607 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:31.607 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:31.607 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:31.607 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:31.608 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 88 2024-09-21T14:58:31.608 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.608 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:31.608 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.608 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:31.608 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.748110+0000", 2024-09-21T14:58:31.608 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:31.608 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.608 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.608 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:31.608 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.748110+0000", 2024-09-21T14:58:31.608 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:31.608 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.608 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.609 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:31.609 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.748110+0000", 2024-09-21T14:58:31.609 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:58:31.609 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.609 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.609 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:31.609 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.748116+0000", 2024-09-21T14:58:31.609 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5230000000000001e-06 2024-09-21T14:58:31.609 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.609 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.609 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:31.609 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.748130+0000", 2024-09-21T14:58:31.609 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4248e-05 2024-09-21T14:58:31.609 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.610 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.610 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:31.610 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.748159+0000", 2024-09-21T14:58:31.610 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8931000000000001e-05 2024-09-21T14:58:31.610 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.610 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.610 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:31.610 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.755598+0000", 2024-09-21T14:58:31.610 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0074386820000000003 2024-09-21T14:58:31.610 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.610 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.610 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:31.610 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.755778+0000", 2024-09-21T14:58:31.610 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00018027399999999999 2024-09-21T14:58:31.610 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.611 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.611 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:31.611 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.756059+0000", 2024-09-21T14:58:31.611 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00028083600000000002 2024-09-21T14:58:31.611 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.611 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.611 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:31.611 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.757447+0000", 2024-09-21T14:58:31.611 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0013877939999999999 2024-09-21T14:58:31.611 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.611 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.611 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:31.611 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.758054+0000", 2024-09-21T14:58:31.611 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000606979 2024-09-21T14:58:31.611 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.612 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.612 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:31.612 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.758128+0000", 2024-09-21T14:58:31.612 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4493999999999995e-05 2024-09-21T14:58:31.612 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.612 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.612 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:31.612 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.758182+0000", 2024-09-21T14:58:31.612 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3247e-05 2024-09-21T14:58:31.612 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:31.612 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:31.612 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:31.612 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.612 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.612 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T14:58:31.613 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.954952+0000", 2024-09-21T14:58:31.613 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 27.621075023, 2024-09-21T14:58:31.613 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0097988899999999993, 2024-09-21T14:58:31.613 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:31.613 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:31.613 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:31.613 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:31.613 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:31.613 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 111 2024-09-21T14:58:31.613 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.613 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:31.613 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.613 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:31.613 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.954952+0000", 2024-09-21T14:58:31.613 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:31.614 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.614 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.614 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:31.614 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.954952+0000", 2024-09-21T14:58:31.614 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:31.614 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.614 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.614 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:31.614 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.954952+0000", 2024-09-21T14:58:31.614 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:31.614 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.614 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.614 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:31.614 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.954955+0000", 2024-09-21T14:58:31.615 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2440000000000002e-06 2024-09-21T14:58:31.615 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.615 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.615 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:31.615 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.954960+0000", 2024-09-21T14:58:31.615 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8119999999999998e-06 2024-09-21T14:58:31.615 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.615 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.615 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:31.615 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.954963+0000", 2024-09-21T14:58:31.616 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4029999999999999e-06 2024-09-21T14:58:31.616 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.616 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.616 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:31.616 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.960407+0000", 2024-09-21T14:58:31.616 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0054443360000000001 2024-09-21T14:58:31.616 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.616 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.616 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:31.616 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.960496+0000", 2024-09-21T14:58:31.616 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8978000000000001e-05 2024-09-21T14:58:31.616 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.616 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.616 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:31.617 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.960635+0000", 2024-09-21T14:58:31.617 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013853300000000001 2024-09-21T14:58:31.617 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.617 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.617 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:31.617 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.964189+0000", 2024-09-21T14:58:31.617 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0035545590000000001 2024-09-21T14:58:31.617 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.617 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.617 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:31.617 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.964571+0000", 2024-09-21T14:58:31.617 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00038135799999999999 2024-09-21T14:58:31.617 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.617 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.617 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:31.617 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.964685+0000", 2024-09-21T14:58:31.618 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00011391300000000001 2024-09-21T14:58:31.618 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.618 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.618 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:31.618 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.964751+0000", 2024-09-21T14:58:31.618 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6111000000000005e-05 2024-09-21T14:58:31.618 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:31.618 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:31.618 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:31.618 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.618 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.618 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T14:58:31.618 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.955076+0000", 2024-09-21T14:58:31.618 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 27.620951105, 2024-09-21T14:58:31.618 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.010426615, 2024-09-21T14:58:31.619 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:31.619 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:31.619 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:31.619 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:31.619 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:31.619 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 112 2024-09-21T14:58:31.619 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.619 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:31.619 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.619 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:31.619 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955076+0000", 2024-09-21T14:58:31.619 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:31.619 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.619 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.619 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:31.620 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955076+0000", 2024-09-21T14:58:31.620 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:31.620 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.620 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.620 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:31.620 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955075+0000", 2024-09-21T14:58:31.620 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:58:31.620 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.620 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.620 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:31.620 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955122+0000", 2024-09-21T14:58:31.620 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6802999999999999e-05 2024-09-21T14:58:31.621 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.621 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.621 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:31.621 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955130+0000", 2024-09-21T14:58:31.621 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0490000000000004e-06 2024-09-21T14:58:31.621 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.621 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.621 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:31.621 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955136+0000", 2024-09-21T14:58:31.621 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9059999999999999e-06 2024-09-21T14:58:31.621 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.621 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.621 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:31.621 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.961454+0000", 2024-09-21T14:58:31.621 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0063180429999999997 2024-09-21T14:58:31.621 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.622 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.622 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:31.622 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.961600+0000", 2024-09-21T14:58:31.622 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014651700000000001 2024-09-21T14:58:31.622 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.622 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.622 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:31.622 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.961798+0000", 2024-09-21T14:58:31.622 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00019784100000000001 2024-09-21T14:58:31.622 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.622 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.622 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:31.622 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.964785+0000", 2024-09-21T14:58:31.622 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.002986561 2024-09-21T14:58:31.622 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.623 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.623 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:31.623 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.965422+0000", 2024-09-21T14:58:31.623 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00063701800000000005 2024-09-21T14:58:31.623 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.623 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.623 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:31.623 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.965466+0000", 2024-09-21T14:58:31.623 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4020999999999997e-05 2024-09-21T14:58:31.623 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.623 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.623 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:31.623 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.965503+0000", 2024-09-21T14:58:31.624 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6814999999999999e-05 2024-09-21T14:58:31.624 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:31.624 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:31.624 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:31.624 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.624 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.624 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T14:58:31.624 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.955142+0000", 2024-09-21T14:58:31.624 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 27.620885213000001, 2024-09-21T14:58:31.624 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012388923, 2024-09-21T14:58:31.624 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:31.624 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:31.624 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:31.624 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:31.625 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:31.625 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 113 2024-09-21T14:58:31.625 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.625 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:31.625 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.625 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:31.625 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955142+0000", 2024-09-21T14:58:31.625 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:31.625 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.625 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.625 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:31.626 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955142+0000", 2024-09-21T14:58:31.626 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:31.626 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.626 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.626 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:31.626 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955141+0000", 2024-09-21T14:58:31.626 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:31.626 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.626 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.626 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:31.626 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955145+0000", 2024-09-21T14:58:31.626 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2109999999999998e-06 2024-09-21T14:58:31.626 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.626 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.627 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:31.627 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955150+0000", 2024-09-21T14:58:31.627 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4090000000000002e-06 2024-09-21T14:58:31.627 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.627 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.627 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:31.627 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955153+0000", 2024-09-21T14:58:31.627 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1240000000000001e-06 2024-09-21T14:58:31.627 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.627 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.627 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:31.627 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.963026+0000", 2024-09-21T14:58:31.627 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0078724060000000002 2024-09-21T14:58:31.627 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.628 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.628 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:31.628 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.963126+0000", 2024-09-21T14:58:31.628 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000100206 2024-09-21T14:58:31.628 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.628 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.628 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:31.628 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.963277+0000", 2024-09-21T14:58:31.628 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015159200000000001 2024-09-21T14:58:31.628 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.628 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.628 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:31.629 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.965519+0000", 2024-09-21T14:58:31.629 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0022418440000000002 2024-09-21T14:58:31.629 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.629 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.629 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:31.629 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967305+0000", 2024-09-21T14:58:31.629 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001785525 2024-09-21T14:58:31.629 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.629 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.629 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:31.629 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967438+0000", 2024-09-21T14:58:31.629 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013284399999999999 2024-09-21T14:58:31.629 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.629 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.629 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:31.629 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967531+0000", 2024-09-21T14:58:31.630 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.3140000000000006e-05 2024-09-21T14:58:31.630 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:31.630 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:31.630 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:31.630 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.630 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.630 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T14:58:31.630 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.955387+0000", 2024-09-21T14:58:31.630 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 27.620640279, 2024-09-21T14:58:31.630 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012319478999999999, 2024-09-21T14:58:31.630 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:31.630 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:31.630 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:31.630 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:31.631 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:31.631 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 114 2024-09-21T14:58:31.631 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.631 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:31.631 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.631 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:31.631 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955387+0000", 2024-09-21T14:58:31.631 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:31.631 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.631 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.631 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:31.631 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955387+0000", 2024-09-21T14:58:31.631 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:31.631 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.632 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.632 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:31.632 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955386+0000", 2024-09-21T14:58:31.632 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:58:31.632 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.632 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.632 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:31.632 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955392+0000", 2024-09-21T14:58:31.632 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4740000000000002e-06 2024-09-21T14:58:31.632 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.632 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.632 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:31.632 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955400+0000", 2024-09-21T14:58:31.632 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2139999999999996e-06 2024-09-21T14:58:31.632 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.633 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.633 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:31.633 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955407+0000", 2024-09-21T14:58:31.633 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2299999999999996e-06 2024-09-21T14:58:31.633 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.633 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.633 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:31.633 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.964840+0000", 2024-09-21T14:58:31.633 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0094330079999999997 2024-09-21T14:58:31.633 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.633 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.633 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:31.633 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.964917+0000", 2024-09-21T14:58:31.633 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7071999999999997e-05 2024-09-21T14:58:31.634 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.634 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.634 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:31.634 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.965026+0000", 2024-09-21T14:58:31.634 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000109646 2024-09-21T14:58:31.634 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.634 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.634 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:31.634 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967609+0000", 2024-09-21T14:58:31.634 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0025824929999999999 2024-09-21T14:58:31.634 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.634 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.634 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:31.634 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967629+0000", 2024-09-21T14:58:31.635 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0486999999999999e-05 2024-09-21T14:58:31.635 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.635 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.635 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:31.635 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967672+0000", 2024-09-21T14:58:31.635 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2314999999999997e-05 2024-09-21T14:58:31.635 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.635 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.635 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:31.635 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967706+0000", 2024-09-21T14:58:31.635 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4483999999999997e-05 2024-09-21T14:58:31.635 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:31.635 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:31.635 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:31.635 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.635 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.635 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T14:58:31.636 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.955609+0000", 2024-09-21T14:58:31.636 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 27.620418473000001, 2024-09-21T14:58:31.636 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013624153999999999, 2024-09-21T14:58:31.636 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:31.636 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:31.636 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:31.636 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:31.636 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:31.636 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 115 2024-09-21T14:58:31.636 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.636 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:31.636 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.637 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:31.637 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955609+0000", 2024-09-21T14:58:31.637 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:31.637 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.637 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.637 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:31.637 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955609+0000", 2024-09-21T14:58:31.637 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:31.637 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.637 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.637 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:31.637 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955607+0000", 2024-09-21T14:58:31.637 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-21T14:58:31.638 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.638 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.638 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:31.638 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955732+0000", 2024-09-21T14:58:31.638 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000125566 2024-09-21T14:58:31.638 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.638 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.638 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:31.638 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955752+0000", 2024-09-21T14:58:31.638 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9916999999999999e-05 2024-09-21T14:58:31.638 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.638 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.638 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:31.638 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955767+0000", 2024-09-21T14:58:31.638 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4687e-05 2024-09-21T14:58:31.638 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.639 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.639 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:31.639 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.965574+0000", 2024-09-21T14:58:31.639 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0098072850000000007 2024-09-21T14:58:31.639 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.639 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.639 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:31.639 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.965642+0000", 2024-09-21T14:58:31.639 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7784e-05 2024-09-21T14:58:31.639 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.639 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.639 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:31.639 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.965753+0000", 2024-09-21T14:58:31.639 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000111111 2024-09-21T14:58:31.640 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.640 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.640 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:31.640 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967717+0000", 2024-09-21T14:58:31.640 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0019635339999999999 2024-09-21T14:58:31.640 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.640 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.640 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:31.640 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.969014+0000", 2024-09-21T14:58:31.640 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001297093 2024-09-21T14:58:31.640 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.640 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.640 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:31.640 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.969130+0000", 2024-09-21T14:58:31.640 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000116011 2024-09-21T14:58:31.641 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.641 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.641 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:31.641 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.969233+0000", 2024-09-21T14:58:31.641 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000102982 2024-09-21T14:58:31.641 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:31.641 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:31.641 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:31.641 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.641 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.641 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T14:58:31.641 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.955783+0000", 2024-09-21T14:58:31.641 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 27.620243804000001, 2024-09-21T14:58:31.641 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014630241, 2024-09-21T14:58:31.641 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:31.641 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:31.642 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:31.642 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:31.642 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:31.642 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 116 2024-09-21T14:58:31.642 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.642 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:31.642 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.642 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:31.642 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955783+0000", 2024-09-21T14:58:31.642 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:31.642 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.642 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.642 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:31.642 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955783+0000", 2024-09-21T14:58:31.643 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:31.643 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.643 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.643 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:31.643 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955782+0000", 2024-09-21T14:58:31.643 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:58:31.643 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.643 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.643 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:31.643 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955791+0000", 2024-09-21T14:58:31.643 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8089999999999999e-06 2024-09-21T14:58:31.643 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.643 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.643 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:31.643 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955804+0000", 2024-09-21T14:58:31.643 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2548000000000001e-05 2024-09-21T14:58:31.644 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.644 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.644 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:31.644 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955814+0000", 2024-09-21T14:58:31.644 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0158e-05 2024-09-21T14:58:31.644 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.644 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.644 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:31.644 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.966515+0000", 2024-09-21T14:58:31.644 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.010701756 2024-09-21T14:58:31.644 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.644 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.644 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:31.644 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.966606+0000", 2024-09-21T14:58:31.645 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0761999999999996e-05 2024-09-21T14:58:31.645 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.645 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.645 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:31.645 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.966740+0000", 2024-09-21T14:58:31.645 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000133693 2024-09-21T14:58:31.645 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.645 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.645 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:31.645 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.968920+0000", 2024-09-21T14:58:31.645 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0021799290000000002 2024-09-21T14:58:31.645 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.645 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.645 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:31.645 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.970320+0000", 2024-09-21T14:58:31.645 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0014001759999999999 2024-09-21T14:58:31.646 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.646 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.646 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:31.646 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.970376+0000", 2024-09-21T14:58:31.646 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5696999999999999e-05 2024-09-21T14:58:31.646 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.646 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.646 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:31.646 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.970413+0000", 2024-09-21T14:58:31.646 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7750999999999998e-05 2024-09-21T14:58:31.646 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:31.646 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:31.646 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:31.646 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.647 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.647 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T14:58:31.647 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.955829+0000", 2024-09-21T14:58:31.647 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 27.620198334000001, 2024-09-21T14:58:31.647 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014919597999999999, 2024-09-21T14:58:31.647 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:31.647 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:31.647 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:31.647 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:31.647 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:31.647 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 117 2024-09-21T14:58:31.647 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.647 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:31.647 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.648 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:31.648 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955829+0000", 2024-09-21T14:58:31.648 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:31.648 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.648 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.648 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:31.648 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955829+0000", 2024-09-21T14:58:31.648 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:31.648 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.648 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.648 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:31.648 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955827+0000", 2024-09-21T14:58:31.648 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:31.648 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.649 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.649 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:31.649 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955838+0000", 2024-09-21T14:58:31.649 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.111e-05 2024-09-21T14:58:31.649 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.649 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.649 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:31.649 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955875+0000", 2024-09-21T14:58:31.649 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6211999999999998e-05 2024-09-21T14:58:31.649 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.649 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.649 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:31.650 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955899+0000", 2024-09-21T14:58:31.650 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4128999999999999e-05 2024-09-21T14:58:31.650 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.650 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.650 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:31.650 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967767+0000", 2024-09-21T14:58:31.650 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.011868251999999999 2024-09-21T14:58:31.650 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.650 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.650 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:31.650 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967866+0000", 2024-09-21T14:58:31.650 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8664000000000003e-05 2024-09-21T14:58:31.650 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.651 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.651 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:31.651 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.968128+0000", 2024-09-21T14:58:31.651 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00026243600000000001 2024-09-21T14:58:31.651 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.651 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.651 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:31.651 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.969393+0000", 2024-09-21T14:58:31.651 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0012650350000000001 2024-09-21T14:58:31.651 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.651 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.651 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:31.651 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.970523+0000", 2024-09-21T14:58:31.651 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001129557 2024-09-21T14:58:31.651 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.652 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.652 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:31.652 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.970603+0000", 2024-09-21T14:58:31.652 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0090000000000001e-05 2024-09-21T14:58:31.652 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.652 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.652 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:31.652 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.970748+0000", 2024-09-21T14:58:31.652 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014552799999999999 2024-09-21T14:58:31.652 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:31.652 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:31.652 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:31.652 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.652 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.652 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T14:58:31.653 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:04.492934+0000", 2024-09-21T14:58:31.653 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 27.083093087000002, 2024-09-21T14:58:31.653 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.010845886000000001, 2024-09-21T14:58:31.653 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:31.653 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:31.653 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:31.653 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:31.653 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:31.653 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 134 2024-09-21T14:58:31.653 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.653 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:31.653 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.653 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:31.653 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.492934+0000", 2024-09-21T14:58:31.653 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:31.653 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.654 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.654 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:31.654 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.492934+0000", 2024-09-21T14:58:31.654 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:31.654 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.654 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.654 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:31.654 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.492933+0000", 2024-09-21T14:58:31.654 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:58:31.654 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.654 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.654 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:31.654 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493094+0000", 2024-09-21T14:58:31.654 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00016101599999999999 2024-09-21T14:58:31.654 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.655 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.655 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:31.655 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493110+0000", 2024-09-21T14:58:31.655 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5447999999999998e-05 2024-09-21T14:58:31.655 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.655 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.655 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:31.655 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493119+0000", 2024-09-21T14:58:31.655 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.3170000000000008e-06 2024-09-21T14:58:31.655 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.655 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.655 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:31.655 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.498294+0000", 2024-09-21T14:58:31.655 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0051747700000000004 2024-09-21T14:58:31.655 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.656 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.656 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:31.656 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.498456+0000", 2024-09-21T14:58:31.656 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00016253599999999999 2024-09-21T14:58:31.656 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.656 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.656 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:31.656 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.498626+0000", 2024-09-21T14:58:31.656 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00016929900000000001 2024-09-21T14:58:31.656 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.656 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.656 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:31.656 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.502513+0000", 2024-09-21T14:58:31.656 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0038875530000000002 2024-09-21T14:58:31.657 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.657 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.657 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:31.657 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.503655+0000", 2024-09-21T14:58:31.657 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001142405 2024-09-21T14:58:31.657 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.657 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.657 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:31.657 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.503741+0000", 2024-09-21T14:58:31.657 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5918e-05 2024-09-21T14:58:31.657 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.657 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.657 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:31.657 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.503780+0000", 2024-09-21T14:58:31.658 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.8401e-05 2024-09-21T14:58:31.658 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:31.658 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:31.658 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:31.658 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.658 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.658 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T14:58:31.658 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:04.493130+0000", 2024-09-21T14:58:31.658 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 27.082896929, 2024-09-21T14:58:31.658 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012013845, 2024-09-21T14:58:31.658 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:31.658 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:31.658 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:31.658 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:31.658 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:31.659 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 135 2024-09-21T14:58:31.659 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.659 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:31.659 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.659 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:31.659 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493130+0000", 2024-09-21T14:58:31.659 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:31.659 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.659 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.659 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:31.659 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493130+0000", 2024-09-21T14:58:31.659 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:31.659 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.659 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.659 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:31.660 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493129+0000", 2024-09-21T14:58:31.660 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:58:31.660 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.660 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.660 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:31.660 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493135+0000", 2024-09-21T14:58:31.660 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1290000000000001e-06 2024-09-21T14:58:31.660 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.660 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.660 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:31.660 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493143+0000", 2024-09-21T14:58:31.660 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0139999999999998e-06 2024-09-21T14:58:31.660 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.660 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.660 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:31.661 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493150+0000", 2024-09-21T14:58:31.661 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.674e-06 2024-09-21T14:58:31.661 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.661 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.661 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:31.661 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.500089+0000", 2024-09-21T14:58:31.661 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0069391599999999998 2024-09-21T14:58:31.661 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.661 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.661 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:31.661 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.500277+0000", 2024-09-21T14:58:31.661 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00018736999999999999 2024-09-21T14:58:31.661 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.661 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.662 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:31.662 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.500426+0000", 2024-09-21T14:58:31.662 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014981900000000001 2024-09-21T14:58:31.662 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.662 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.662 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:31.662 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.503817+0000", 2024-09-21T14:58:31.662 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0033905150000000002 2024-09-21T14:58:31.662 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.662 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.662 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:31.662 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.504994+0000", 2024-09-21T14:58:31.663 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0011767500000000001 2024-09-21T14:58:31.663 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.663 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.663 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:31.663 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.505083+0000", 2024-09-21T14:58:31.663 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.9339999999999995e-05 2024-09-21T14:58:31.663 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.663 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.663 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:31.663 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.505144+0000", 2024-09-21T14:58:31.663 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0934000000000003e-05 2024-09-21T14:58:31.663 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:31.663 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:31.663 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:31.663 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.664 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.664 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T14:58:31.664 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:04.493201+0000", 2024-09-21T14:58:31.664 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 27.082825584999998, 2024-09-21T14:58:31.664 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.01208676, 2024-09-21T14:58:31.664 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:31.664 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:31.664 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:31.664 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:31.664 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:31.664 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 136 2024-09-21T14:58:31.664 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.664 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:31.664 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.664 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:31.665 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493201+0000", 2024-09-21T14:58:31.665 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:31.665 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.665 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.665 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:31.665 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493201+0000", 2024-09-21T14:58:31.665 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:31.665 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.665 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.665 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:31.665 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493200+0000", 2024-09-21T14:58:31.665 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:58:31.665 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.665 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.665 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:31.666 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493214+0000", 2024-09-21T14:58:31.666 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3488e-05 2024-09-21T14:58:31.666 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.666 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.666 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:31.666 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493244+0000", 2024-09-21T14:58:31.666 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9848e-05 2024-09-21T14:58:31.666 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.666 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.666 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:31.666 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493263+0000", 2024-09-21T14:58:31.666 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9071000000000001e-05 2024-09-21T14:58:31.666 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.666 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.667 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:31.667 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.501197+0000", 2024-09-21T14:58:31.667 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.007934623 2024-09-21T14:58:31.667 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.667 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.667 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:31.667 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.501327+0000", 2024-09-21T14:58:31.667 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000129605 2024-09-21T14:58:31.667 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.667 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.667 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:31.667 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.501518+0000", 2024-09-21T14:58:31.667 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000190662 2024-09-21T14:58:31.667 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.667 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.667 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:31.668 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.505175+0000", 2024-09-21T14:58:31.668 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0036576989999999999 2024-09-21T14:58:31.668 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.668 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.668 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:31.668 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.505216+0000", 2024-09-21T14:58:31.668 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1097000000000003e-05 2024-09-21T14:58:31.668 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.668 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.668 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:31.668 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.505251+0000", 2024-09-21T14:58:31.668 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4758000000000002e-05 2024-09-21T14:58:31.668 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.669 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.669 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:31.669 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.505288+0000", 2024-09-21T14:58:31.669 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6913999999999999e-05 2024-09-21T14:58:31.669 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:31.669 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:31.669 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:31.669 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.669 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.669 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T14:58:31.669 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:04.493315+0000", 2024-09-21T14:58:31.669 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 27.082711500999999, 2024-09-21T14:58:31.669 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014055766000000001, 2024-09-21T14:58:31.669 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:31.670 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:31.670 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:31.670 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:31.670 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:31.670 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 137 2024-09-21T14:58:31.670 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.670 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:31.670 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.670 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:31.670 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493315+0000", 2024-09-21T14:58:31.670 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:31.670 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.670 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.670 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:31.671 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493315+0000", 2024-09-21T14:58:31.671 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:31.671 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.671 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.671 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:31.671 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493314+0000", 2024-09-21T14:58:31.671 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:58:31.671 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.671 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.671 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:31.671 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493507+0000", 2024-09-21T14:58:31.671 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000192444 2024-09-21T14:58:31.671 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.671 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.672 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:31.672 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493528+0000", 2024-09-21T14:58:31.672 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0845000000000001e-05 2024-09-21T14:58:31.672 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.672 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.672 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:31.672 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493543+0000", 2024-09-21T14:58:31.672 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5800000000000001e-05 2024-09-21T14:58:31.672 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.672 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.672 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:31.672 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.502575+0000", 2024-09-21T14:58:31.672 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0090312389999999999 2024-09-21T14:58:31.672 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.672 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.673 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:31.673 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.502677+0000", 2024-09-21T14:58:31.673 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000102271 2024-09-21T14:58:31.673 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.673 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.673 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:31.673 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.502825+0000", 2024-09-21T14:58:31.673 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000148171 2024-09-21T14:58:31.673 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.673 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.673 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:31.673 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507251+0000", 2024-09-21T14:58:31.674 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0044253879999999997 2024-09-21T14:58:31.674 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.674 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.674 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:31.674 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507313+0000", 2024-09-21T14:58:31.674 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2170999999999998e-05 2024-09-21T14:58:31.674 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.674 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.675 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:31.675 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507346+0000", 2024-09-21T14:58:31.675 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3079e-05 2024-09-21T14:58:31.675 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.675 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.675 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:31.675 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507371+0000", 2024-09-21T14:58:31.675 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5477999999999999e-05 2024-09-21T14:58:31.675 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:31.675 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:31.675 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:31.675 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.675 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.675 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T14:58:31.676 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:04.493570+0000", 2024-09-21T14:58:31.676 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 27.082457095999999, 2024-09-21T14:58:31.676 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014061686, 2024-09-21T14:58:31.676 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:31.676 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:31.676 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:31.676 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:31.676 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:31.676 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 138 2024-09-21T14:58:31.676 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.676 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:31.676 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.676 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:31.676 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493570+0000", 2024-09-21T14:58:31.676 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:31.677 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.677 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.677 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:31.677 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493570+0000", 2024-09-21T14:58:31.677 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:31.677 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.677 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.677 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:31.677 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493568+0000", 2024-09-21T14:58:31.677 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:31.677 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.677 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.677 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:31.677 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493579+0000", 2024-09-21T14:58:31.677 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0574000000000001e-05 2024-09-21T14:58:31.677 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.678 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.678 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:31.678 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493595+0000", 2024-09-21T14:58:31.678 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6504999999999999e-05 2024-09-21T14:58:31.678 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.678 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.678 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:31.678 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493608+0000", 2024-09-21T14:58:31.678 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2836999999999999e-05 2024-09-21T14:58:31.678 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.678 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.678 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:31.678 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.503909+0000", 2024-09-21T14:58:31.678 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.01030118 2024-09-21T14:58:31.678 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.679 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.679 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:31.679 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.504064+0000", 2024-09-21T14:58:31.679 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015493000000000001 2024-09-21T14:58:31.679 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.679 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.679 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:31.679 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.504299+0000", 2024-09-21T14:58:31.679 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00023416100000000001 2024-09-21T14:58:31.679 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.679 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.679 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:31.679 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507466+0000", 2024-09-21T14:58:31.679 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0031678380000000001 2024-09-21T14:58:31.680 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.680 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.680 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:31.680 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507483+0000", 2024-09-21T14:58:31.680 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6606999999999999e-05 2024-09-21T14:58:31.680 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.680 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.680 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:31.680 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507566+0000", 2024-09-21T14:58:31.680 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3110999999999998e-05 2024-09-21T14:58:31.680 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.680 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.680 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:31.681 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507632+0000", 2024-09-21T14:58:31.681 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5480000000000003e-05 2024-09-21T14:58:31.681 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:31.681 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:31.681 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:31.681 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.681 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.681 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T14:58:31.681 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:04.493625+0000", 2024-09-21T14:58:31.681 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 27.082402408, 2024-09-21T14:58:31.681 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014665378999999999, 2024-09-21T14:58:31.681 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:31.682 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:31.682 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:31.682 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:31.682 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:31.682 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139 2024-09-21T14:58:31.682 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.682 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:31.682 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.682 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:31.682 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493625+0000", 2024-09-21T14:58:31.682 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:31.682 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.682 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.682 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:31.683 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493625+0000", 2024-09-21T14:58:31.683 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:31.683 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.683 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.683 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:31.683 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493624+0000", 2024-09-21T14:58:31.683 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:58:31.683 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.683 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.683 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:31.683 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493638+0000", 2024-09-21T14:58:31.683 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4282e-05 2024-09-21T14:58:31.683 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.683 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.684 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:31.684 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493654+0000", 2024-09-21T14:58:31.684 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5739999999999998e-05 2024-09-21T14:58:31.684 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.684 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.684 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:31.684 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493666+0000", 2024-09-21T14:58:31.684 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2865e-05 2024-09-21T14:58:31.684 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.684 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.684 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:31.684 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.505329+0000", 2024-09-21T14:58:31.684 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.011662237000000001 2024-09-21T14:58:31.684 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.685 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.685 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:31.685 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.505377+0000", 2024-09-21T14:58:31.685 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7862999999999999e-05 2024-09-21T14:58:31.685 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.685 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.685 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:31.685 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.505455+0000", 2024-09-21T14:58:31.685 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8918000000000006e-05 2024-09-21T14:58:31.685 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.685 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.685 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:31.685 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507650+0000", 2024-09-21T14:58:31.685 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.002195027 2024-09-21T14:58:31.686 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.686 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.686 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:31.686 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508149+0000", 2024-09-21T14:58:31.686 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00049828399999999999 2024-09-21T14:58:31.686 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.686 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.686 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:31.686 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508230+0000", 2024-09-21T14:58:31.686 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1376000000000002e-05 2024-09-21T14:58:31.687 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.687 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.687 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:31.687 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508290+0000", 2024-09-21T14:58:31.687 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9852999999999998e-05 2024-09-21T14:58:31.687 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:31.687 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:31.687 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:31.687 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.687 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.687 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T14:58:31.687 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:04.493679+0000", 2024-09-21T14:58:31.687 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 27.082347510000002, 2024-09-21T14:58:31.688 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014869869000000001, 2024-09-21T14:58:31.688 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:31.688 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:31.688 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:31.688 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:31.688 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:31.688 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140 2024-09-21T14:58:31.688 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.688 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:31.688 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.688 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:31.688 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493679+0000", 2024-09-21T14:58:31.688 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:31.688 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.689 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.689 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:31.689 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493679+0000", 2024-09-21T14:58:31.689 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:31.689 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.689 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.689 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:31.689 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493679+0000", 2024-09-21T14:58:31.689 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:58:31.689 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.689 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.689 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:31.689 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493889+0000", 2024-09-21T14:58:31.689 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00021032700000000001 2024-09-21T14:58:31.689 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.689 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.690 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:31.690 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493931+0000", 2024-09-21T14:58:31.690 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1971000000000002e-05 2024-09-21T14:58:31.690 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.690 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.690 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:31.690 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493947+0000", 2024-09-21T14:58:31.690 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6385e-05 2024-09-21T14:58:31.690 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.690 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.690 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:31.690 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.505804+0000", 2024-09-21T14:58:31.690 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.011857136000000001 2024-09-21T14:58:31.690 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.691 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.691 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:31.691 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.505992+0000", 2024-09-21T14:58:31.691 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000187665 2024-09-21T14:58:31.691 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.691 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.691 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:31.691 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.506281+0000", 2024-09-21T14:58:31.691 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000288712 2024-09-21T14:58:31.691 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.691 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.691 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:31.691 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508349+0000", 2024-09-21T14:58:31.691 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0020685790000000001 2024-09-21T14:58:31.691 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.692 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.692 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:31.692 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508371+0000", 2024-09-21T14:58:31.692 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1747e-05 2024-09-21T14:58:31.692 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.692 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.692 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:31.692 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508450+0000", 2024-09-21T14:58:31.692 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9332000000000005e-05 2024-09-21T14:58:31.692 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.692 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.692 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:31.692 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508549+0000", 2024-09-21T14:58:31.692 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8918000000000004e-05 2024-09-21T14:58:31.693 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:31.693 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:31.693 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:31.693 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.693 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.693 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T14:58:31.693 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:04.493967+0000", 2024-09-21T14:58:31.693 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 27.082060472999999, 2024-09-21T14:58:31.693 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.015281075999999999, 2024-09-21T14:58:31.693 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:31.693 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:31.693 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:31.694 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:31.694 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:31.694 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 141 2024-09-21T14:58:31.694 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.694 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:31.694 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.694 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:31.694 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493967+0000", 2024-09-21T14:58:31.694 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:31.694 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.694 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.694 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:31.694 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493967+0000", 2024-09-21T14:58:31.694 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:31.695 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.695 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.695 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:31.695 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493965+0000", 2024-09-21T14:58:31.695 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:31.695 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.695 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.695 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:31.695 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493987+0000", 2024-09-21T14:58:31.695 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1662999999999999e-05 2024-09-21T14:58:31.695 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.695 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.696 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:31.696 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.494004+0000", 2024-09-21T14:58:31.696 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7311000000000001e-05 2024-09-21T14:58:31.696 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.696 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.696 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:31.696 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.494017+0000", 2024-09-21T14:58:31.696 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3202e-05 2024-09-21T14:58:31.696 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.696 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.696 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:31.696 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507678+0000", 2024-09-21T14:58:31.696 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013660432 2024-09-21T14:58:31.697 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.697 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.697 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:31.697 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507726+0000", 2024-09-21T14:58:31.697 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8154e-05 2024-09-21T14:58:31.697 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.697 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.697 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:31.697 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507797+0000", 2024-09-21T14:58:31.697 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.1110000000000002e-05 2024-09-21T14:58:31.697 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.697 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.697 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:31.697 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508710+0000", 2024-09-21T14:58:31.698 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00091339299999999997 2024-09-21T14:58:31.698 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.698 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.698 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:31.698 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.509108+0000", 2024-09-21T14:58:31.698 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00039718300000000001 2024-09-21T14:58:31.698 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.698 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.698 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:31.698 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.509186+0000", 2024-09-21T14:58:31.698 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8592999999999998e-05 2024-09-21T14:58:31.698 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:31.699 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:31.699 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:31.699 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.509248+0000", 2024-09-21T14:58:31.699 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1450000000000003e-05 2024-09-21T14:58:31.699 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:31.699 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:31.699 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:31.699 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:31.699 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:31.699 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:58:31.700 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T14:58:31.701 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T14:58:31.702 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:58:31.695+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:58:31.801 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:58:31.795+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:58:31.815 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:58:31.815 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:58:31.815 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T14:58:31.815 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:58:31.825 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T14:58:31.862 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T14:58:31.893 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T14:58:31.902 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:58:31.901+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:58:31.914 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T14:58:31.934 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:31.934 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:31.934 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T14:58:31.944 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T14:58:31.958 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-21T14:58:31.981 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T14:58:32.003 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T14:58:32.003 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:58:31.999+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:58:32.026 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T14:58:32.028 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:58:32.028 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:58:32.028 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:58:32.028 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:58:32.028 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:58:32.028 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:58:32.028 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:58:32.028 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:58:32.047 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:32.047 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:32.057 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T14:58:32.105 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:58:32.105+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:58:32.118 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T14:58:32.206 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:58:32.199+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:58:32.236 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:32.236 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:32.246 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T14:58:32.307 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:58:32.305+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:58:32.393 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:32.393 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:32.403 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T14:58:32.408 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:58:32.403+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:58:32.503 INFO:teuthology.orchestra.run.smithi032.stderr:nodeep-scrub is set 2024-09-21T14:58:32.508 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:58:32.505+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:58:32.525 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:32.525 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:32.540 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T14:58:32.609 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:58:32.603+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:58:32.669 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:32.669 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:32.683 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T14:58:32.710 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:58:32.709+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:58:32.808 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:32.808 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:32.810 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:58:32.809+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:58:32.824 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T14:58:32.911 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:58:32.907+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:58:32.950 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:32.951 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:32.961 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T14:58:33.012 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:58:33.009+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:58:33.062 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:33.063 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:33.072 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T14:58:33.114 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:58:33.107+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:58:33.177 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:33.177 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:33.187 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T14:58:33.233 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:58:33.233+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:58:33.295 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:33.296 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:33.305 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T14:58:33.316 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:58:33.313+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:58:33.411 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:33.411 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:33.416 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:58:33.411+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:58:33.420 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T14:58:33.493 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T14:58:33.493 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T14:58:33.493 INFO:tasks.thrashosds.thrasher:inject_pause on osd.1 2024-09-21T14:58:33.493 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-21T14:58:33.493 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-21T14:58:33.493 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok config set filestore_inject_stall 3 2024-09-21T14:58:33.537 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:33.537 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:33.539 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:58:33.535+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:58:33.547 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T14:58:33.614 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:58:33.614 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-21T14:58:33.614 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:58:33.617 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:58:33.617+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:58:33.651 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:33.651 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:33.661 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T14:58:33.695 INFO:teuthology.orchestra.run.smithi032.stdout:checking clone #1, image image_client.0-clone1 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent2 2024-09-21T14:58:33.718 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:58:33.717+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:58:33.764 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:33.765 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:33.774 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T14:58:33.819 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:58:33.817+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:58:33.919 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:58:33.915+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:58:33.928 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:33.928 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:33.938 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T14:58:34.020 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:58:34.017+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:58:34.045 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T14:58:34.046 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:34.047 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:34.056 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T14:58:34.091 INFO:teuthology.orchestra.run.smithi032.stdout:228 punch from 0x5da4f32 to 0x5db38f6, (0xe9c4 bytes) 2024-09-21T14:58:34.099 INFO:teuthology.orchestra.run.smithi032.stdout:229 write 0x63adcc5 thru 0x63bc5ee (0xe92a bytes) 2024-09-21T14:58:34.105 INFO:teuthology.orchestra.run.smithi032.stdout:230 writesame 0xca42e1 thru 0xcaf623 data_size 0x3(0xb343 bytes) 2024-09-21T14:58:34.120 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:34.120 INFO:teuthology.orchestra.run.smithi032.stdout:232 writesame 0x3896c9c thru 0x3898b1a data_size 0x1(0x1e7f bytes) 2024-09-21T14:58:34.121 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:58:34.117+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:58:34.126 INFO:teuthology.orchestra.run.smithi032.stdout:233 read 0x811a6 thru 0x83ba2 (0x29fd bytes) 2024-09-21T14:58:34.127 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:34.127 INFO:teuthology.orchestra.run.smithi032.stdout:235 read 0x74bc80e thru 0x74c0102 (0x38f5 bytes) 2024-09-21T14:58:34.128 INFO:teuthology.orchestra.run.smithi032.stdout:236 punch from 0x93f4f74 to 0x93fda9b, (0x8b27 bytes) 2024-09-21T14:58:34.144 INFO:teuthology.orchestra.run.smithi032.stdout:237 trunc from 0xe31407d to 0x4b1d7cd 2024-09-21T14:58:34.193 INFO:teuthology.orchestra.run.smithi032.stdout:239 writesame 0x6917e6b thru 0x6921dd7 data_size 0x1(0x9f6d bytes) 2024-09-21T14:58:34.201 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T14:58:34.216 INFO:teuthology.orchestra.run.smithi032.stdout:240 write 0xe772da2 thru 0xe77fbf5 (0xce54 bytes) 2024-09-21T14:58:34.221 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:58:34.215+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:58:34.224 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T14:58:34.227 INFO:teuthology.orchestra.run.smithi032.stdout:241 read 0x1498d73 thru 0x149f44b (0x66d9 bytes) 2024-09-21T14:58:34.227 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:58:34.227 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T14:58:34.227 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.228 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6137 3.15 3:ac7dc405:::rbd_data.11361bed1f78.000000000000039e:head [set-alloc-hint object_size 262144 write_size 262144,write 65536~16384 in=16384b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-21T14:58:34.228 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:34.223493+0000", 2024-09-21T14:58:34.228 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0027125539999999998, 2024-09-21T14:58:34.228 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0027637209999999998, 2024-09-21T14:58:34.228 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:34.228 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "waiting for sub ops", 2024-09-21T14:58:34.228 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:34.228 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:34.228 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:34.228 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6137 2024-09-21T14:58:34.228 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.228 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:34.228 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.228 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:34.228 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.223493+0000", 2024-09-21T14:58:34.229 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:34.229 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.229 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.229 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:34.229 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.223493+0000", 2024-09-21T14:58:34.229 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:34.229 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.229 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.229 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:34.229 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.223491+0000", 2024-09-21T14:58:34.229 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-21T14:58:34.229 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.229 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.229 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:34.230 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.223525+0000", 2024-09-21T14:58:34.230 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3207999999999998e-05 2024-09-21T14:58:34.230 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.230 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.230 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:34.230 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.223542+0000", 2024-09-21T14:58:34.230 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7116e-05 2024-09-21T14:58:34.230 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.230 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.230 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:34.230 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.223552+0000", 2024-09-21T14:58:34.230 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0406e-05 2024-09-21T14:58:34.230 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.230 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.231 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:34.231 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.223656+0000", 2024-09-21T14:58:34.231 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000104022 2024-09-21T14:58:34.231 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.231 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.231 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:34.231 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.223954+0000", 2024-09-21T14:58:34.231 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00029771700000000002 2024-09-21T14:58:34.231 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.231 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.231 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 6", 2024-09-21T14:58:34.231 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.224242+0000", 2024-09-21T14:58:34.231 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000288196 2024-09-21T14:58:34.231 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.232 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.232 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:34.232 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.225216+0000", 2024-09-21T14:58:34.232 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00097429899999999998 2024-09-21T14:58:34.232 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:34.232 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:34.232 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:34.232 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.232 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.232 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop_reply(client.4406.0:6137 3.15 e39/24 ondisk, result = 0)", 2024-09-21T14:58:34.232 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:34.226242+0000", 2024-09-21T14:58:34.232 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4294967295.9999638, 2024-09-21T14:58:34.232 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017585999999999999, 2024-09-21T14:58:34.232 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:34.233 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:34.233 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:34.233 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.233 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:34.233 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.226242+0000", 2024-09-21T14:58:34.233 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:34.233 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.233 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.233 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:34.233 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.226242+0000", 2024-09-21T14:58:34.233 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:34.233 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.233 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.233 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:34.234 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.226241+0000", 2024-09-21T14:58:34.234 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:34.234 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.234 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.234 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:34.234 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.226250+0000", 2024-09-21T14:58:34.234 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8559999999999996e-06 2024-09-21T14:58:34.234 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.234 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.234 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:34.234 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.226268+0000", 2024-09-21T14:58:34.234 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8493999999999999e-05 2024-09-21T14:58:34.235 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.235 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.235 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:34.235 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.226281+0000", 2024-09-21T14:58:34.235 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2801e-05 2024-09-21T14:58:34.235 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.235 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.235 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:34.235 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.226352+0000", 2024-09-21T14:58:34.235 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.1239e-05 2024-09-21T14:58:34.235 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.235 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.235 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:34.236 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.226358+0000", 2024-09-21T14:58:34.236 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0310000000000004e-06 2024-09-21T14:58:34.236 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:34.236 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:34.236 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:34.236 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:34.236 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-21T14:58:34.236 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 2 2024-09-21T14:58:34.236 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:58:34.236 INFO:teuthology.orchestra.run.smithi032.stdout:243 write 0xceaab21 thru 0xceb717e (0xc65e bytes) 2024-09-21T14:58:34.238 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T14:58:34.239 INFO:teuthology.orchestra.run.smithi032.stdout:244 read 0xa48f331 thru 0xa498d19 (0x99e9 bytes) 2024-09-21T14:58:34.239 INFO:teuthology.orchestra.run.smithi032.stdout:245 punch from 0x8e32dcd to 0x8e37ee9, (0x511c bytes) 2024-09-21T14:58:34.239 INFO:teuthology.orchestra.run.smithi032.stdout:247 writesame 0x6264423 thru 0x6271767 data_size 0x1d(0xd345 bytes) 2024-09-21T14:58:34.247 INFO:teuthology.orchestra.run.smithi032.stdout:248 punch from 0x7c2860e to 0x7c32346, (0x9d38 bytes) 2024-09-21T14:58:34.247 INFO:teuthology.orchestra.run.smithi032.stdout:249 write 0x9dcbac7 thru 0x9dd7360 (0xb89a bytes) 2024-09-21T14:58:34.248 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T14:58:34.268 INFO:teuthology.orchestra.run.smithi032.stdout:250 trunc from 0xe77fbf6 to 0x89b08b5 2024-09-21T14:58:34.269 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T14:58:34.292 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-21T14:58:34.305 INFO:teuthology.orchestra.run.smithi032.stdout:251 writesame 0xb46378d thru 0xb464f19 data_size 0x1(0x178d bytes) 2024-09-21T14:58:34.305 INFO:teuthology.orchestra.run.smithi032.stdout:252 writesame 0xa8a0d71 thru 0xa8a6e4a data_size 0x62(0x60da bytes) 2024-09-21T14:58:34.313 INFO:teuthology.orchestra.run.smithi032.stdout:253 read 0x57d293a thru 0x57dff3e (0xd605 bytes) 2024-09-21T14:58:34.313 INFO:teuthology.orchestra.run.smithi032.stdout:255 write 0xa1477d0 thru 0xa14da94 (0x62c5 bytes) 2024-09-21T14:58:34.314 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T14:58:34.321 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:58:34.315+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:58:34.323 INFO:teuthology.orchestra.run.smithi032.stdout:256 read 0x5e91826 thru 0x5ea1451 (0xfc2c bytes) 2024-09-21T14:58:34.323 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:34.323 INFO:teuthology.orchestra.run.smithi032.stdout:265 write 0x32dedcf thru 0x32e4975 (0x5ba7 bytes) 2024-09-21T14:58:34.329 INFO:teuthology.orchestra.run.smithi032.stdout:266 read 0x56ccf5a thru 0x56dcbba (0xfc61 bytes) 2024-09-21T14:58:34.329 INFO:teuthology.orchestra.run.smithi032.stdout:268 read 0xad34862 thru 0xad3835a (0x3af9 bytes) 2024-09-21T14:58:34.329 INFO:teuthology.orchestra.run.smithi032.stdout:270 read 0x29db3c1 thru 0x29e645c (0xb09c bytes) 2024-09-21T14:58:34.338 INFO:teuthology.orchestra.run.smithi032.stdout:271 trunc from 0xb464f1a to 0x5f0fa95 2024-09-21T14:58:34.345 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T14:58:34.359 INFO:teuthology.orchestra.run.smithi032.stdout:272 read 0x12c1509 thru 0x12c539e (0x3e96 bytes) 2024-09-21T14:58:34.360 INFO:teuthology.orchestra.run.smithi032.stdout:273 write 0x111abdf thru 0x111fad4 (0x4ef6 bytes) 2024-09-21T14:58:34.363 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:58:34.363 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:58:34.363 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T14:58:34.363 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T14:58:34.363 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:58:34.367 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T14:58:34.369 INFO:teuthology.orchestra.run.smithi032.stdout:274 trunc from 0x5f0fa95 to 0x273ecb6 2024-09-21T14:58:34.370 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:58:34.370 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:58:34.370 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:58:34.371 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:58:34.371 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:58:34.371 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:58:34.371 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:58:34.371 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:58:34.380 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T14:58:34.402 INFO:teuthology.orchestra.run.smithi032.stdout:275 clone 4 order 19 su 131072 sc 4 2024-09-21T14:58:34.422 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:58:34.421+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:58:34.508 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:58:34.508 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T14:58:34.508 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T14:58:34.508 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T14:58:34.509 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.509 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:337 3.3 3:c554b425:::rbd_data.11367ffd86fe.0000000000000016:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T14:58:34.509 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:25.457552+0000", 2024-09-21T14:58:34.509 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 9.0454850530000002, 2024-09-21T14:58:34.509 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0041465709999999999, 2024-09-21T14:58:34.509 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:34.509 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:58:34.509 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:34.509 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:34.509 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:34.509 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 337 2024-09-21T14:58:34.509 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.509 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:34.510 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.510 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:34.510 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:25.457552+0000", 2024-09-21T14:58:34.510 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:34.510 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.510 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.510 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:34.510 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:25.457552+0000", 2024-09-21T14:58:34.510 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:34.510 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.510 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.510 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:34.511 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:25.457549+0000", 2024-09-21T14:58:34.511 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-21T14:58:34.511 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.511 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.511 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:34.511 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:25.457566+0000", 2024-09-21T14:58:34.511 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7031e-05 2024-09-21T14:58:34.511 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.511 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.511 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:34.511 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:25.457600+0000", 2024-09-21T14:58:34.511 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4254000000000001e-05 2024-09-21T14:58:34.511 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.511 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.511 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:34.512 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:25.457617+0000", 2024-09-21T14:58:34.512 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6813999999999999e-05 2024-09-21T14:58:34.512 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.512 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.512 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:34.512 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:25.457705+0000", 2024-09-21T14:58:34.512 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.865e-05 2024-09-21T14:58:34.512 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.512 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.512 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op must wait for map", 2024-09-21T14:58:34.512 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:25.457714+0000", 2024-09-21T14:58:34.512 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0989999999999999e-06 2024-09-21T14:58:34.512 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.513 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.513 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:34.513 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:25.461574+0000", 2024-09-21T14:58:34.513 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0038602749999999998 2024-09-21T14:58:34.513 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.513 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.513 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:34.513 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:25.461698+0000", 2024-09-21T14:58:34.513 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012446000000000001 2024-09-21T14:58:34.513 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:34.513 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:34.513 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:34.513 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.513 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.513 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:344 3.15 3:a83bbcfa:::rbd_data.11367ffd86fe.0000000000000029:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T14:58:34.514 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:25.459558+0000", 2024-09-21T14:58:34.514 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 9.0434784399999995, 2024-09-21T14:58:34.514 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0030343900000000001, 2024-09-21T14:58:34.514 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:34.514 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:58:34.514 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:34.514 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:34.514 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:34.514 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 344 2024-09-21T14:58:34.514 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.514 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:34.514 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.514 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:34.515 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:25.459558+0000", 2024-09-21T14:58:34.515 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:34.515 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.515 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.515 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:34.515 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:25.459558+0000", 2024-09-21T14:58:34.515 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:34.515 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.515 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.515 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:34.515 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:25.459557+0000", 2024-09-21T14:58:34.515 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:34.516 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.516 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.516 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:34.516 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:25.459565+0000", 2024-09-21T14:58:34.516 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6619999999999994e-06 2024-09-21T14:58:34.516 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.516 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.516 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:34.516 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:25.459591+0000", 2024-09-21T14:58:34.516 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5389e-05 2024-09-21T14:58:34.516 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.516 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.516 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:34.516 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:25.459601+0000", 2024-09-21T14:58:34.516 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6889999999999999e-06 2024-09-21T14:58:34.516 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.517 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.517 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:34.517 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:25.459659+0000", 2024-09-21T14:58:34.517 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8712999999999998e-05 2024-09-21T14:58:34.517 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.517 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.517 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op must wait for map", 2024-09-21T14:58:34.517 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:25.459669+0000", 2024-09-21T14:58:34.517 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0159999999999999e-05 2024-09-21T14:58:34.517 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.517 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.517 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:34.517 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:25.462463+0000", 2024-09-21T14:58:34.517 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0027938110000000002 2024-09-21T14:58:34.518 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.518 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.518 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:34.518 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:25.462593+0000", 2024-09-21T14:58:34.518 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012943899999999999 2024-09-21T14:58:34.518 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:34.518 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:34.518 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:34.518 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.518 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.518 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:1376 3.c 3:35bac63f:::rbd_data.1136515ecb6b.0000000000000036:head [set-alloc-hint object_size 4194304 write_size 4194304,write 257580~55629] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T14:58:34.518 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:30.106566+0000", 2024-09-21T14:58:34.518 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.3964711779999996, 2024-09-21T14:58:34.518 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0049454169999999997, 2024-09-21T14:58:34.519 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:34.519 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:34.519 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:34.519 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:34.519 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:34.519 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1376 2024-09-21T14:58:34.519 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.519 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:34.519 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.519 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:34.519 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.106566+0000", 2024-09-21T14:58:34.519 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:34.519 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.519 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.519 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:34.520 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.106566+0000", 2024-09-21T14:58:34.520 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:34.520 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.520 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.520 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:34.520 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.106564+0000", 2024-09-21T14:58:34.520 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-21T14:58:34.520 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.520 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.520 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:34.520 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.106608+0000", 2024-09-21T14:58:34.520 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4150999999999998e-05 2024-09-21T14:58:34.520 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.520 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.520 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:34.521 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.106627+0000", 2024-09-21T14:58:34.521 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9205e-05 2024-09-21T14:58:34.521 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.521 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.521 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:34.521 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.106636+0000", 2024-09-21T14:58:34.521 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4910000000000005e-06 2024-09-21T14:58:34.521 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.521 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.521 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:34.521 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.106698+0000", 2024-09-21T14:58:34.521 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2181000000000007e-05 2024-09-21T14:58:34.521 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.522 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.522 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op must wait for map", 2024-09-21T14:58:34.522 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.106702+0000", 2024-09-21T14:58:34.522 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3080000000000001e-06 2024-09-21T14:58:34.522 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.522 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.522 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:34.522 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.108858+0000", 2024-09-21T14:58:34.522 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0021558049999999998 2024-09-21T14:58:34.522 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.522 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.522 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:34.522 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.109008+0000", 2024-09-21T14:58:34.522 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015012700000000001 2024-09-21T14:58:34.523 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.523 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.523 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 7", 2024-09-21T14:58:34.523 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.109110+0000", 2024-09-21T14:58:34.523 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010155899999999999 2024-09-21T14:58:34.523 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.523 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.523 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:34.523 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.110173+0000", 2024-09-21T14:58:34.523 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0010639289999999999 2024-09-21T14:58:34.523 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.523 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.523 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:34.523 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.111428+0000", 2024-09-21T14:58:34.523 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0012550180000000001 2024-09-21T14:58:34.524 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.524 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.524 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:34.524 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.111461+0000", 2024-09-21T14:58:34.524 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2851000000000001e-05 2024-09-21T14:58:34.524 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.524 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.524 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:34.524 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.111511+0000", 2024-09-21T14:58:34.524 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9650000000000001e-05 2024-09-21T14:58:34.524 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:34.524 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:34.524 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:34.524 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.524 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.524 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:1388 3.c 3:35bac63f:::rbd_data.1136515ecb6b.0000000000000036:head [delete] snapc 8=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T14:58:34.525 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:30.153949+0000", 2024-09-21T14:58:34.525 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.3490880699999996, 2024-09-21T14:58:34.525 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0035509600000000001, 2024-09-21T14:58:34.525 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:34.525 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:34.525 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:34.525 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:34.525 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:34.525 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1388 2024-09-21T14:58:34.525 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.525 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:34.525 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.525 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:34.525 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.153949+0000", 2024-09-21T14:58:34.526 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:34.526 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.526 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.526 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:34.526 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.153949+0000", 2024-09-21T14:58:34.526 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:34.526 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.526 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.526 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:34.526 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.153946+0000", 2024-09-21T14:58:34.526 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T14:58:34.526 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.526 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.526 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:34.527 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.153956+0000", 2024-09-21T14:58:34.527 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6840000000000008e-06 2024-09-21T14:58:34.527 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.527 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.527 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:34.527 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.153975+0000", 2024-09-21T14:58:34.527 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9012999999999999e-05 2024-09-21T14:58:34.527 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.527 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.527 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:34.527 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.153985+0000", 2024-09-21T14:58:34.527 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0484e-05 2024-09-21T14:58:34.527 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.527 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.528 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:34.528 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.154048+0000", 2024-09-21T14:58:34.528 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2102999999999998e-05 2024-09-21T14:58:34.528 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.528 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.528 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:34.528 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.154115+0000", 2024-09-21T14:58:34.528 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7868999999999997e-05 2024-09-21T14:58:34.528 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.528 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.528 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 7", 2024-09-21T14:58:34.528 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.154224+0000", 2024-09-21T14:58:34.528 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010902 2024-09-21T14:58:34.528 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.529 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.529 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:34.529 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.155115+0000", 2024-09-21T14:58:34.529 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00089033099999999996 2024-09-21T14:58:34.529 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.529 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.529 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:34.529 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.157431+0000", 2024-09-21T14:58:34.529 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0023162059999999999 2024-09-21T14:58:34.529 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.529 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.529 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:34.529 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.157459+0000", 2024-09-21T14:58:34.529 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8262000000000001e-05 2024-09-21T14:58:34.529 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.530 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.530 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:34.530 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.157500+0000", 2024-09-21T14:58:34.530 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.0423000000000002e-05 2024-09-21T14:58:34.530 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:34.530 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:34.530 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:34.530 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.530 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.530 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:1470 3.15 3:afe7411f:::rbd_data.1136515ecb6b.000000000000003a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 496608~27680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T14:58:34.530 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:30.414273+0000", 2024-09-21T14:58:34.530 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.0887639260000004, 2024-09-21T14:58:34.530 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0039222340000000001, 2024-09-21T14:58:34.530 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:34.531 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:34.531 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:34.531 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:34.531 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:34.531 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1470 2024-09-21T14:58:34.531 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.531 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:34.531 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.531 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:34.531 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.414273+0000", 2024-09-21T14:58:34.531 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:34.531 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.531 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.531 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:34.531 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.414273+0000", 2024-09-21T14:58:34.532 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:34.532 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.532 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.532 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:34.532 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.414271+0000", 2024-09-21T14:58:34.532 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T14:58:34.532 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.532 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.532 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:34.532 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.414298+0000", 2024-09-21T14:58:34.532 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7131000000000002e-05 2024-09-21T14:58:34.532 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.532 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.532 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:34.533 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.414323+0000", 2024-09-21T14:58:34.533 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4973999999999998e-05 2024-09-21T14:58:34.533 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.533 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.533 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:34.533 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.414334+0000", 2024-09-21T14:58:34.533 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1657e-05 2024-09-21T14:58:34.533 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.533 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.533 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:34.533 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.414474+0000", 2024-09-21T14:58:34.533 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013978100000000001 2024-09-21T14:58:34.533 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.533 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.533 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:34.534 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.414839+0000", 2024-09-21T14:58:34.534 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000365149 2024-09-21T14:58:34.534 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.534 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.534 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 6", 2024-09-21T14:58:34.534 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.415056+0000", 2024-09-21T14:58:34.534 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00021713200000000001 2024-09-21T14:58:34.534 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.534 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.534 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:34.534 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.416939+0000", 2024-09-21T14:58:34.534 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001882945 2024-09-21T14:58:34.534 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.534 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.534 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:34.535 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.418070+0000", 2024-09-21T14:58:34.535 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0011310249999999999 2024-09-21T14:58:34.535 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.535 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.535 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:34.535 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.418143+0000", 2024-09-21T14:58:34.535 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2261000000000005e-05 2024-09-21T14:58:34.535 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.535 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.535 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:34.535 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.418195+0000", 2024-09-21T14:58:34.535 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2361000000000002e-05 2024-09-21T14:58:34.535 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:34.535 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:34.536 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:34.536 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.536 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.536 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:1473 3.15 3:af2afa6a:::rbd_data.1136515ecb6b.000000000000001b:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T14:58:34.536 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:30.421730+0000", 2024-09-21T14:58:34.536 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.0813065110000002, 2024-09-21T14:58:34.536 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021478937, 2024-09-21T14:58:34.536 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:34.536 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:34.536 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:34.536 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:34.536 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:34.536 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1473 2024-09-21T14:58:34.536 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.536 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:34.537 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.537 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:34.537 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421730+0000", 2024-09-21T14:58:34.537 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:34.537 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.537 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.537 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:34.537 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421730+0000", 2024-09-21T14:58:34.537 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:34.537 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.537 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.537 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:34.537 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421728+0000", 2024-09-21T14:58:34.537 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-21T14:58:34.538 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.538 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.538 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:34.538 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421739+0000", 2024-09-21T14:58:34.538 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0502e-05 2024-09-21T14:58:34.538 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.538 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.538 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:34.538 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421762+0000", 2024-09-21T14:58:34.538 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3405999999999999e-05 2024-09-21T14:58:34.538 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.538 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.538 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:34.538 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421773+0000", 2024-09-21T14:58:34.539 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0744000000000001e-05 2024-09-21T14:58:34.539 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.539 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.539 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:34.539 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421838+0000", 2024-09-21T14:58:34.539 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5486000000000003e-05 2024-09-21T14:58:34.539 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.539 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.539 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:34.539 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421955+0000", 2024-09-21T14:58:34.539 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000116765 2024-09-21T14:58:34.539 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.539 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.539 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 6", 2024-09-21T14:58:34.540 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.425572+0000", 2024-09-21T14:58:34.540 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0036171380000000002 2024-09-21T14:58:34.540 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.540 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.540 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:34.540 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.442973+0000", 2024-09-21T14:58:34.540 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.017400268 2024-09-21T14:58:34.540 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.540 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.540 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:34.540 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.442996+0000", 2024-09-21T14:58:34.540 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3139e-05 2024-09-21T14:58:34.540 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.540 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.540 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:34.541 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.443079+0000", 2024-09-21T14:58:34.541 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2832999999999998e-05 2024-09-21T14:58:34.541 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.541 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.541 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:34.541 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.443209+0000", 2024-09-21T14:58:34.541 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013052499999999999 2024-09-21T14:58:34.541 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:34.541 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:34.541 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:34.541 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.541 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.541 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:1482 3.15 3:af2afa6a:::rbd_data.1136515ecb6b.000000000000001b:head [delete] snapc 8=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T14:58:34.541 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:30.481917+0000", 2024-09-21T14:58:34.542 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.0211199219999996, 2024-09-21T14:58:34.542 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0036932309999999999, 2024-09-21T14:58:34.542 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:34.542 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:34.542 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:34.542 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:34.542 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:34.542 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1482 2024-09-21T14:58:34.542 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.542 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:34.542 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.542 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:34.542 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.481917+0000", 2024-09-21T14:58:34.542 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:34.543 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.543 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.543 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:34.543 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.481917+0000", 2024-09-21T14:58:34.543 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:34.543 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.543 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.543 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:34.543 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.481914+0000", 2024-09-21T14:58:34.543 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T14:58:34.543 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.543 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.543 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:34.543 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.481925+0000", 2024-09-21T14:58:34.544 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0964999999999999e-05 2024-09-21T14:58:34.544 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.544 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.544 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:34.544 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.481944+0000", 2024-09-21T14:58:34.544 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8482999999999999e-05 2024-09-21T14:58:34.544 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.544 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.544 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:34.544 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.481961+0000", 2024-09-21T14:58:34.544 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7159999999999998e-05 2024-09-21T14:58:34.544 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.544 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.544 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:34.544 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.482015+0000", 2024-09-21T14:58:34.545 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4249999999999997e-05 2024-09-21T14:58:34.545 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.545 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.545 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:34.545 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.482114+0000", 2024-09-21T14:58:34.545 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8684999999999995e-05 2024-09-21T14:58:34.545 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.545 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.545 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 6", 2024-09-21T14:58:34.545 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.482184+0000", 2024-09-21T14:58:34.545 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0604000000000004e-05 2024-09-21T14:58:34.545 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.545 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.545 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:34.545 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.483583+0000", 2024-09-21T14:58:34.546 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0013982440000000001 2024-09-21T14:58:34.546 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.546 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.546 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:34.546 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.485547+0000", 2024-09-21T14:58:34.546 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0019639100000000001 2024-09-21T14:58:34.546 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.546 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.546 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:34.546 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.485585+0000", 2024-09-21T14:58:34.546 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.8804999999999999e-05 2024-09-21T14:58:34.546 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.546 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.546 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:34.547 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.485610+0000", 2024-09-21T14:58:34.547 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4678999999999999e-05 2024-09-21T14:58:34.547 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:34.547 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:34.547 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:34.547 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.547 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.547 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:1486 3.15 3:afe7411f:::rbd_data.1136515ecb6b.000000000000003a:head [delete] snapc 8=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T14:58:34.547 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:30.481989+0000", 2024-09-21T14:58:34.547 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.0210472480000004, 2024-09-21T14:58:34.547 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0040307399999999997, 2024-09-21T14:58:34.547 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:34.547 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:34.547 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:34.548 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:34.548 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:34.548 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1486 2024-09-21T14:58:34.548 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.548 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:34.548 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.548 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:34.548 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.481989+0000", 2024-09-21T14:58:34.548 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:34.548 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.548 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.548 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:34.548 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.481989+0000", 2024-09-21T14:58:34.548 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:34.548 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.549 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.549 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:34.549 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.481989+0000", 2024-09-21T14:58:34.549 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:58:34.549 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.549 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.549 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:34.549 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.481994+0000", 2024-09-21T14:58:34.549 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8330000000000001e-06 2024-09-21T14:58:34.549 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.549 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.549 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:34.549 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.482001+0000", 2024-09-21T14:58:34.549 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4250000000000001e-06 2024-09-21T14:58:34.550 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.550 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.550 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:34.550 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.482007+0000", 2024-09-21T14:58:34.550 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8860000000000001e-06 2024-09-21T14:58:34.550 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.550 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.550 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:34.550 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.482485+0000", 2024-09-21T14:58:34.550 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00047820000000000002 2024-09-21T14:58:34.550 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.550 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.550 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:34.550 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.482735+0000", 2024-09-21T14:58:34.551 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000249862 2024-09-21T14:58:34.551 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.551 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.551 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 6", 2024-09-21T14:58:34.551 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.482974+0000", 2024-09-21T14:58:34.551 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00023892999999999999 2024-09-21T14:58:34.551 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.551 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.551 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:34.551 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.484286+0000", 2024-09-21T14:58:34.551 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0013121470000000001 2024-09-21T14:58:34.551 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.551 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.551 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:34.552 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.485925+0000", 2024-09-21T14:58:34.552 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001639148 2024-09-21T14:58:34.552 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.552 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.552 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:34.552 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.485978+0000", 2024-09-21T14:58:34.552 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2332e-05 2024-09-21T14:58:34.552 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.552 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.552 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:34.552 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.486020+0000", 2024-09-21T14:58:34.552 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2710999999999998e-05 2024-09-21T14:58:34.552 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:34.552 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:34.553 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:34.553 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.553 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.553 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:1508 3.c 3:35bac63f:::rbd_data.1136515ecb6b.0000000000000036:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3763121~60236] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T14:58:34.553 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:30.528427+0000", 2024-09-21T14:58:34.553 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 3.9746094539999999, 2024-09-21T14:58:34.553 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.003484348, 2024-09-21T14:58:34.553 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:34.553 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:34.553 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:34.553 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:34.553 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:34.553 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1508 2024-09-21T14:58:34.553 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.553 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:34.554 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.554 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:34.554 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.528427+0000", 2024-09-21T14:58:34.554 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:34.554 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.554 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.554 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:34.554 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.528427+0000", 2024-09-21T14:58:34.554 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:34.554 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.554 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.554 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:34.554 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.528425+0000", 2024-09-21T14:58:34.554 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-21T14:58:34.555 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.555 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.555 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:34.555 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.528485+0000", 2024-09-21T14:58:34.555 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0004999999999999e-05 2024-09-21T14:58:34.555 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.555 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.555 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:34.555 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.528513+0000", 2024-09-21T14:58:34.555 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7605000000000001e-05 2024-09-21T14:58:34.555 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.555 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.555 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:34.555 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.528525+0000", 2024-09-21T14:58:34.556 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2164000000000001e-05 2024-09-21T14:58:34.556 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.556 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.556 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:34.556 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.528643+0000", 2024-09-21T14:58:34.556 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000118316 2024-09-21T14:58:34.556 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.556 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.556 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:34.556 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.528864+0000", 2024-09-21T14:58:34.556 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00022093199999999999 2024-09-21T14:58:34.556 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.556 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.556 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 7", 2024-09-21T14:58:34.556 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.529243+0000", 2024-09-21T14:58:34.557 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000378346 2024-09-21T14:58:34.557 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.557 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.557 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:34.557 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.530946+0000", 2024-09-21T14:58:34.557 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001703567 2024-09-21T14:58:34.557 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.557 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.557 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:34.557 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.531682+0000", 2024-09-21T14:58:34.557 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00073578000000000001 2024-09-21T14:58:34.557 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.557 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.557 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:34.558 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.531816+0000", 2024-09-21T14:58:34.558 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013375999999999999 2024-09-21T14:58:34.558 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.558 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.558 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:34.558 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.531912+0000", 2024-09-21T14:58:34.558 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5835000000000007e-05 2024-09-21T14:58:34.558 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:34.558 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:34.558 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:34.558 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.558 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.558 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:1531 3.15 3:aca5faff:::rbd_data.1136515ecb6b.000000000000001e:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3573964~12480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T14:58:34.558 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:30.578477+0000", 2024-09-21T14:58:34.559 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 3.9245599310000001, 2024-09-21T14:58:34.559 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0039711809999999998, 2024-09-21T14:58:34.559 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:34.559 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:34.559 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:34.559 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:34.559 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:34.559 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1531 2024-09-21T14:58:34.559 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.559 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:34.559 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.559 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:34.559 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.578477+0000", 2024-09-21T14:58:34.559 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:34.560 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.560 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.560 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:34.560 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.578477+0000", 2024-09-21T14:58:34.560 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:34.560 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.560 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.560 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:34.560 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.578474+0000", 2024-09-21T14:58:34.560 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-21T14:58:34.560 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.560 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.560 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:34.560 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.578523+0000", 2024-09-21T14:58:34.561 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9076000000000002e-05 2024-09-21T14:58:34.561 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.561 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.561 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:34.561 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.578560+0000", 2024-09-21T14:58:34.561 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6996000000000003e-05 2024-09-21T14:58:34.561 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.561 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.561 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:34.561 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.578576+0000", 2024-09-21T14:58:34.561 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6079999999999999e-05 2024-09-21T14:58:34.561 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.561 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.561 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:34.562 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.578693+0000", 2024-09-21T14:58:34.562 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00011671 2024-09-21T14:58:34.562 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.562 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.562 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:34.562 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.578926+0000", 2024-09-21T14:58:34.562 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00023317499999999999 2024-09-21T14:58:34.562 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.562 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.562 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 6", 2024-09-21T14:58:34.562 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.579138+0000", 2024-09-21T14:58:34.562 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00021226400000000001 2024-09-21T14:58:34.562 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.563 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.563 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:34.563 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.580886+0000", 2024-09-21T14:58:34.563 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0017482229999999999 2024-09-21T14:58:34.563 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.563 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.563 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:34.563 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.582319+0000", 2024-09-21T14:58:34.563 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001432032 2024-09-21T14:58:34.563 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.563 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.563 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:34.563 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.582387+0000", 2024-09-21T14:58:34.563 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8194999999999993e-05 2024-09-21T14:58:34.564 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.564 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.564 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:34.564 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.582448+0000", 2024-09-21T14:58:34.564 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1273999999999996e-05 2024-09-21T14:58:34.564 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:34.564 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:34.564 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:34.564 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.564 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.564 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:1546 3.3 3:c7c3230f:::rbd_data.1136515ecb6b.0000000000000006:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T14:58:34.564 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:30.629102+0000", 2024-09-21T14:58:34.564 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 3.8739348529999997, 2024-09-21T14:58:34.564 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0047496329999999996, 2024-09-21T14:58:34.565 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:34.565 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:34.565 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:34.565 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:34.565 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:34.565 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1546 2024-09-21T14:58:34.565 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.565 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:34.565 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.565 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:34.565 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.629102+0000", 2024-09-21T14:58:34.565 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:34.565 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.565 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.565 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:34.566 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.629102+0000", 2024-09-21T14:58:34.566 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:34.566 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.566 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.566 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:34.566 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.629100+0000", 2024-09-21T14:58:34.566 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-21T14:58:34.566 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.566 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.566 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:34.566 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.629111+0000", 2024-09-21T14:58:34.566 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1181e-05 2024-09-21T14:58:34.566 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.566 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.567 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:34.567 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.629134+0000", 2024-09-21T14:58:34.567 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.281e-05 2024-09-21T14:58:34.567 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.567 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.567 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:34.567 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.629145+0000", 2024-09-21T14:58:34.567 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0913999999999999e-05 2024-09-21T14:58:34.567 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.567 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.567 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:34.567 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.629201+0000", 2024-09-21T14:58:34.567 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6382999999999999e-05 2024-09-21T14:58:34.567 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.568 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.568 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:34.568 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.629342+0000", 2024-09-21T14:58:34.568 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014090500000000001 2024-09-21T14:58:34.568 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.568 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.568 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:58:34.568 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.630140+0000", 2024-09-21T14:58:34.568 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00079839899999999998 2024-09-21T14:58:34.568 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.568 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.568 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:34.568 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.633546+0000", 2024-09-21T14:58:34.568 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0034059950000000002 2024-09-21T14:58:34.569 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.569 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.569 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:34.569 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.633741+0000", 2024-09-21T14:58:34.569 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00019487500000000001 2024-09-21T14:58:34.569 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.569 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.569 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:34.569 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.633805+0000", 2024-09-21T14:58:34.569 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3787000000000004e-05 2024-09-21T14:58:34.569 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.569 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.569 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:34.569 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.633852+0000", 2024-09-21T14:58:34.570 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6502999999999998e-05 2024-09-21T14:58:34.570 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:34.570 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:34.570 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:34.570 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.570 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.570 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:1678 3.c 3:377cc7e1:::rbd_data.1136515ecb6b.000000000000000f:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T14:58:34.570 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:30.988085+0000", 2024-09-21T14:58:34.570 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 3.5149519169999999, 2024-09-21T14:58:34.570 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.032974802999999997, 2024-09-21T14:58:34.570 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:34.570 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:34.570 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:34.570 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:34.570 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:34.571 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1678 2024-09-21T14:58:34.571 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.571 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:34.571 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.571 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:34.571 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988085+0000", 2024-09-21T14:58:34.571 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:34.571 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.571 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.571 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:34.571 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988085+0000", 2024-09-21T14:58:34.571 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:34.571 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.571 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.572 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:34.572 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988083+0000", 2024-09-21T14:58:34.572 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-21T14:58:34.572 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.572 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.572 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:34.572 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988093+0000", 2024-09-21T14:58:34.572 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8800000000000003e-06 2024-09-21T14:58:34.572 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.572 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.572 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:34.572 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988113+0000", 2024-09-21T14:58:34.572 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0217999999999999e-05 2024-09-21T14:58:34.572 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.572 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.573 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:34.573 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988123+0000", 2024-09-21T14:58:34.573 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5859999999999994e-06 2024-09-21T14:58:34.573 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.573 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.573 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:34.573 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988203+0000", 2024-09-21T14:58:34.573 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0834000000000006e-05 2024-09-21T14:58:34.573 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.573 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.573 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:34.573 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988376+0000", 2024-09-21T14:58:34.573 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017301299999999999 2024-09-21T14:58:34.573 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.573 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.574 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 7", 2024-09-21T14:58:34.574 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.996338+0000", 2024-09-21T14:58:34.574 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0079618520000000002 2024-09-21T14:58:34.574 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.574 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.574 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:34.574 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.020881+0000", 2024-09-21T14:58:34.574 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024542731000000002 2024-09-21T14:58:34.574 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.574 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.574 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:34.574 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.020904+0000", 2024-09-21T14:58:34.574 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2861e-05 2024-09-21T14:58:34.574 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.575 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.575 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:34.575 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.020987+0000", 2024-09-21T14:58:34.575 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2806000000000006e-05 2024-09-21T14:58:34.575 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.575 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.575 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:34.575 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.021060+0000", 2024-09-21T14:58:34.575 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2871000000000001e-05 2024-09-21T14:58:34.575 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:34.575 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:34.575 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:34.575 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.575 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.576 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop_reply(client.4406.0:1678 3.c e36/24 ondisk, result = 0)", 2024-09-21T14:58:34.576 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:31.016656+0000", 2024-09-21T14:58:34.576 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 3.486380982, 2024-09-21T14:58:34.576 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0042336919999999998, 2024-09-21T14:58:34.576 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:34.576 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:34.576 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:34.576 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.576 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:34.576 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.016656+0000", 2024-09-21T14:58:34.576 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:34.576 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.576 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.576 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:34.577 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.016656+0000", 2024-09-21T14:58:34.577 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:34.577 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.577 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.577 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:34.577 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.016655+0000", 2024-09-21T14:58:34.577 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:58:34.577 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.577 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.577 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:34.577 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.016661+0000", 2024-09-21T14:58:34.577 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7240000000000003e-06 2024-09-21T14:58:34.577 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.577 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.577 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:34.578 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.016680+0000", 2024-09-21T14:58:34.578 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.842e-05 2024-09-21T14:58:34.578 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.578 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.578 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:34.578 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.016690+0000", 2024-09-21T14:58:34.578 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.7429999999999997e-06 2024-09-21T14:58:34.578 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.578 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.578 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:34.578 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.020854+0000", 2024-09-21T14:58:34.578 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0041642370000000003 2024-09-21T14:58:34.578 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.578 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.579 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:34.579 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.020870+0000", 2024-09-21T14:58:34.579 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5894999999999999e-05 2024-09-21T14:58:34.579 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.579 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.579 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:34.579 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.020889+0000", 2024-09-21T14:58:34.579 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9687e-05 2024-09-21T14:58:34.579 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:34.579 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:34.579 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:34.579 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.579 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.579 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:1686 3.15 3:aca5faff:::rbd_data.1136515ecb6b.000000000000001e:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T14:58:34.580 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:31.035289+0000", 2024-09-21T14:58:34.580 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 3.4677481210000001, 2024-09-21T14:58:34.580 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0029747089999999999, 2024-09-21T14:58:34.580 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:34.580 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:34.580 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:34.580 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:34.580 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:34.580 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1686 2024-09-21T14:58:34.580 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.580 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:34.580 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.580 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:34.581 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.035289+0000", 2024-09-21T14:58:34.581 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:34.581 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.581 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.581 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:34.581 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.035289+0000", 2024-09-21T14:58:34.581 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:34.581 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.581 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.581 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:34.581 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.035287+0000", 2024-09-21T14:58:34.581 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-21T14:58:34.581 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.581 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.581 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:34.582 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.035298+0000", 2024-09-21T14:58:34.582 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.079e-05 2024-09-21T14:58:34.582 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.582 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.582 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:34.582 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.035320+0000", 2024-09-21T14:58:34.582 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2475000000000001e-05 2024-09-21T14:58:34.582 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.582 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.582 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:34.582 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.035331+0000", 2024-09-21T14:58:34.582 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0818999999999999e-05 2024-09-21T14:58:34.582 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.582 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.583 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:34.583 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.035404+0000", 2024-09-21T14:58:34.583 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3008000000000003e-05 2024-09-21T14:58:34.583 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.583 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.583 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:34.583 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.035510+0000", 2024-09-21T14:58:34.583 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000105682 2024-09-21T14:58:34.583 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.583 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.583 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 6", 2024-09-21T14:58:34.583 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.035786+0000", 2024-09-21T14:58:34.583 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00027606700000000001 2024-09-21T14:58:34.583 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.584 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.584 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:34.584 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.037249+0000", 2024-09-21T14:58:34.584 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0014633789999999999 2024-09-21T14:58:34.584 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.584 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.584 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:34.584 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.038198+0000", 2024-09-21T14:58:34.584 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00094863399999999996 2024-09-21T14:58:34.584 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.584 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.584 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:34.584 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.038235+0000", 2024-09-21T14:58:34.584 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6955000000000001e-05 2024-09-21T14:58:34.585 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.585 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.585 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:34.585 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.038263+0000", 2024-09-21T14:58:34.585 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8612e-05 2024-09-21T14:58:34.585 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:34.585 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:34.585 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:34.585 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.585 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.585 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:1696 3.c 3:377cc7e1:::rbd_data.1136515ecb6b.000000000000000f:head [write 3385948~14075] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T14:58:34.585 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:31.059184+0000", 2024-09-21T14:58:34.585 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 3.4438525040000001, 2024-09-21T14:58:34.585 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.003402937, 2024-09-21T14:58:34.585 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:34.586 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:34.586 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:34.586 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:34.586 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:34.586 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1696 2024-09-21T14:58:34.586 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.586 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:34.586 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.586 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:34.586 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.059184+0000", 2024-09-21T14:58:34.586 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:34.586 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.586 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.586 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:34.587 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.059184+0000", 2024-09-21T14:58:34.587 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:34.587 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.587 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.587 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:34.587 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.059182+0000", 2024-09-21T14:58:34.587 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-21T14:58:34.587 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.587 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.587 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:34.587 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.059205+0000", 2024-09-21T14:58:34.587 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3082999999999999e-05 2024-09-21T14:58:34.587 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.587 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.588 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:34.588 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.059230+0000", 2024-09-21T14:58:34.588 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4859999999999999e-05 2024-09-21T14:58:34.588 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.588 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.588 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:34.588 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.059241+0000", 2024-09-21T14:58:34.588 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1289e-05 2024-09-21T14:58:34.588 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.588 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.588 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:34.588 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.059311+0000", 2024-09-21T14:58:34.588 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9496e-05 2024-09-21T14:58:34.588 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.588 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.589 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:34.589 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.059424+0000", 2024-09-21T14:58:34.589 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000113298 2024-09-21T14:58:34.589 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.589 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.589 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 7", 2024-09-21T14:58:34.589 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.059577+0000", 2024-09-21T14:58:34.589 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000153306 2024-09-21T14:58:34.589 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.589 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.589 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:34.589 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.060921+0000", 2024-09-21T14:58:34.589 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001343218 2024-09-21T14:58:34.589 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.590 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.590 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:34.590 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.062452+0000", 2024-09-21T14:58:34.590 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0015309900000000001 2024-09-21T14:58:34.590 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.590 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.590 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:34.590 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.062530+0000", 2024-09-21T14:58:34.590 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8156000000000002e-05 2024-09-21T14:58:34.590 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.590 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.590 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:34.590 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.062587+0000", 2024-09-21T14:58:34.591 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7300999999999999e-05 2024-09-21T14:58:34.591 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:34.591 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:34.591 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:34.591 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.591 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.591 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6137 3.15 3:ac7dc405:::rbd_data.11361bed1f78.000000000000039e:head [set-alloc-hint object_size 262144 write_size 262144,write 65536~16384 in=16384b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-21T14:58:34.591 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:34.223493+0000", 2024-09-21T14:58:34.591 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.27954346899999999, 2024-09-21T14:58:34.591 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0029755649999999999, 2024-09-21T14:58:34.591 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:34.591 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:34.591 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:34.591 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:34.591 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:34.592 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6137 2024-09-21T14:58:34.592 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.592 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:34.592 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.592 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:34.592 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.223493+0000", 2024-09-21T14:58:34.592 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:34.592 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.592 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.592 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:34.592 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.223493+0000", 2024-09-21T14:58:34.592 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:34.592 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.593 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.593 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:34.593 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.223491+0000", 2024-09-21T14:58:34.593 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-21T14:58:34.593 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.593 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.593 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:34.593 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.223525+0000", 2024-09-21T14:58:34.593 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3207999999999998e-05 2024-09-21T14:58:34.593 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.593 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.593 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:34.593 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.223542+0000", 2024-09-21T14:58:34.593 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7116e-05 2024-09-21T14:58:34.593 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.594 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.594 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:34.594 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.223552+0000", 2024-09-21T14:58:34.594 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0406e-05 2024-09-21T14:58:34.594 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.594 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.594 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:34.594 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.223656+0000", 2024-09-21T14:58:34.594 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000104022 2024-09-21T14:58:34.594 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.594 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.594 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:34.594 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.223954+0000", 2024-09-21T14:58:34.594 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00029771700000000002 2024-09-21T14:58:34.595 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.595 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.595 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 6", 2024-09-21T14:58:34.595 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.224242+0000", 2024-09-21T14:58:34.595 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000288196 2024-09-21T14:58:34.595 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.595 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.595 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:34.595 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.225216+0000", 2024-09-21T14:58:34.595 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00097429899999999998 2024-09-21T14:58:34.595 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.595 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.595 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:34.595 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.226362+0000", 2024-09-21T14:58:34.596 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001145868 2024-09-21T14:58:34.596 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.596 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.596 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:34.596 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.226442+0000", 2024-09-21T14:58:34.596 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0289000000000003e-05 2024-09-21T14:58:34.596 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.596 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.596 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:34.596 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.226469+0000", 2024-09-21T14:58:34.596 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6407999999999999e-05 2024-09-21T14:58:34.596 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:34.596 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:34.597 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:34.597 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.597 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.597 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6213 3.15 3:a929f9d1:::rbd_data.11361bed1f78.000000000000018d:head [delete] snapc b=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-21T14:58:34.597 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:34.340865+0000", 2024-09-21T14:58:34.597 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.162172224, 2024-09-21T14:58:34.597 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0030554599999999999, 2024-09-21T14:58:34.597 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:34.597 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:34.597 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:34.597 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:34.597 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:34.597 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6213 2024-09-21T14:58:34.597 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.598 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:34.598 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.598 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:34.598 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.340865+0000", 2024-09-21T14:58:34.598 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:34.598 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.598 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.598 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:34.598 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.340865+0000", 2024-09-21T14:58:34.598 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:34.598 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.598 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.598 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:34.598 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.340862+0000", 2024-09-21T14:58:34.598 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T14:58:34.599 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.599 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.599 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:34.599 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.340879+0000", 2024-09-21T14:58:34.599 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7164999999999999e-05 2024-09-21T14:58:34.599 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.599 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.599 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:34.599 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.340911+0000", 2024-09-21T14:58:34.599 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1921999999999997e-05 2024-09-21T14:58:34.599 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.599 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.599 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:34.599 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.340924+0000", 2024-09-21T14:58:34.600 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3424999999999999e-05 2024-09-21T14:58:34.600 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.600 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.600 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:34.600 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.340991+0000", 2024-09-21T14:58:34.600 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6821000000000003e-05 2024-09-21T14:58:34.600 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.600 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.600 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:34.600 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.341074+0000", 2024-09-21T14:58:34.600 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2589999999999994e-05 2024-09-21T14:58:34.600 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.600 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.600 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 6", 2024-09-21T14:58:34.601 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.341158+0000", 2024-09-21T14:58:34.601 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4081999999999999e-05 2024-09-21T14:58:34.601 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.601 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.601 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:34.601 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.342572+0000", 2024-09-21T14:58:34.601 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0014137209999999999 2024-09-21T14:58:34.601 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.601 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.601 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:34.601 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.343803+0000", 2024-09-21T14:58:34.601 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0012308519999999999 2024-09-21T14:58:34.601 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.601 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.602 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:34.602 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.343867+0000", 2024-09-21T14:58:34.602 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4604999999999996e-05 2024-09-21T14:58:34.602 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.602 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.602 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:34.602 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.343920+0000", 2024-09-21T14:58:34.602 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2830000000000001e-05 2024-09-21T14:58:34.602 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:34.602 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:34.602 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:34.602 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.602 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.603 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6214 3.15 3:ade75988:::rbd_data.11361bed1f78.000000000000018e:head [delete] snapc b=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-21T14:58:34.603 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:34.340943+0000", 2024-09-21T14:58:34.603 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.162093717, 2024-09-21T14:58:34.603 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0042430130000000003, 2024-09-21T14:58:34.603 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:34.603 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:34.603 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:34.603 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:34.603 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:34.603 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6214 2024-09-21T14:58:34.603 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.603 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:34.603 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.603 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:34.603 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.340943+0000", 2024-09-21T14:58:34.604 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:34.604 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.604 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.604 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:34.604 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.340943+0000", 2024-09-21T14:58:34.604 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:34.604 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.604 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.604 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:34.604 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.340942+0000", 2024-09-21T14:58:34.604 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:34.604 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.604 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.604 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:34.605 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.340949+0000", 2024-09-21T14:58:34.605 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3419999999999996e-06 2024-09-21T14:58:34.605 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.605 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.605 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:34.605 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.340958+0000", 2024-09-21T14:58:34.605 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5179999999999993e-06 2024-09-21T14:58:34.605 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.605 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.605 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:34.605 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.340965+0000", 2024-09-21T14:58:34.605 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5050000000000004e-06 2024-09-21T14:58:34.605 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.605 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.606 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:34.606 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.341506+0000", 2024-09-21T14:58:34.606 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00054140299999999998 2024-09-21T14:58:34.606 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.606 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.606 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:34.606 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.341713+0000", 2024-09-21T14:58:34.606 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00020653199999999999 2024-09-21T14:58:34.606 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.606 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.606 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 6", 2024-09-21T14:58:34.606 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.341968+0000", 2024-09-21T14:58:34.606 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00025488900000000001 2024-09-21T14:58:34.606 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.607 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.607 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:34.607 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.342968+0000", 2024-09-21T14:58:34.607 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0010007830000000001 2024-09-21T14:58:34.607 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.607 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.607 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:34.607 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.345077+0000", 2024-09-21T14:58:34.607 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0021083640000000002 2024-09-21T14:58:34.607 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.607 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.607 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:34.607 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.345138+0000", 2024-09-21T14:58:34.607 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1098000000000004e-05 2024-09-21T14:58:34.608 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.608 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.608 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:34.608 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.345186+0000", 2024-09-21T14:58:34.608 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8124999999999997e-05 2024-09-21T14:58:34.608 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:34.608 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:34.608 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:34.608 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.608 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.608 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6256 3.15 3:af5b726b:::rbd_data.11361bed1f78.00000000000000d9:head [delete] snapc b=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-21T14:58:34.608 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:34.378949+0000", 2024-09-21T14:58:34.608 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.124088095, 2024-09-21T14:58:34.608 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0033881480000000001, 2024-09-21T14:58:34.609 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:34.609 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:34.609 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:34.609 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:34.609 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:34.609 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6256 2024-09-21T14:58:34.609 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.609 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:34.609 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.609 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:34.609 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.378949+0000", 2024-09-21T14:58:34.609 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:34.609 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.609 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.610 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:34.610 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.378949+0000", 2024-09-21T14:58:34.610 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:34.610 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.610 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.610 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:34.610 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.378947+0000", 2024-09-21T14:58:34.610 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:34.610 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.610 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.610 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:34.610 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.378954+0000", 2024-09-21T14:58:34.610 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3810000000000004e-06 2024-09-21T14:58:34.610 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.611 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.611 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:34.611 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.378971+0000", 2024-09-21T14:58:34.611 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7260000000000001e-05 2024-09-21T14:58:34.611 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.611 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.611 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:34.611 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.378979+0000", 2024-09-21T14:58:34.611 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.259e-06 2024-09-21T14:58:34.611 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.611 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.611 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:34.611 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.379057+0000", 2024-09-21T14:58:34.612 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7415000000000003e-05 2024-09-21T14:58:34.612 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.612 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.612 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:34.612 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.379181+0000", 2024-09-21T14:58:34.612 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012419500000000001 2024-09-21T14:58:34.612 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.612 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.612 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 6", 2024-09-21T14:58:34.612 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.379286+0000", 2024-09-21T14:58:34.612 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000105444 2024-09-21T14:58:34.612 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.612 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.612 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:34.613 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.380511+0000", 2024-09-21T14:58:34.613 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0012243989999999999 2024-09-21T14:58:34.613 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.613 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.613 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:34.613 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.382269+0000", 2024-09-21T14:58:34.613 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001758424 2024-09-21T14:58:34.613 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.613 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.613 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:34.613 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.382308+0000", 2024-09-21T14:58:34.613 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.9384000000000001e-05 2024-09-21T14:58:34.613 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.613 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.614 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:34.614 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.382337+0000", 2024-09-21T14:58:34.614 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8357e-05 2024-09-21T14:58:34.614 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:34.614 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:34.614 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:34.614 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.614 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.614 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6260 3.15 3:aa002667:::rbd_data.11361bed1f78.00000000000000dd:head [delete] snapc b=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-21T14:58:34.614 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:34.379515+0000", 2024-09-21T14:58:34.614 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.12352189, 2024-09-21T14:58:34.614 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0029519540000000001, 2024-09-21T14:58:34.615 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:34.615 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:34.615 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:34.615 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:34.615 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:34.615 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6260 2024-09-21T14:58:34.615 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.615 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:34.615 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.615 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:34.615 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.379515+0000", 2024-09-21T14:58:34.615 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:34.615 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.615 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.616 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:34.616 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.379515+0000", 2024-09-21T14:58:34.616 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:34.616 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.616 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.616 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:34.616 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.379514+0000", 2024-09-21T14:58:34.616 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:34.616 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.616 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.616 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:34.616 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.379521+0000", 2024-09-21T14:58:34.616 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3329999999999999e-06 2024-09-21T14:58:34.616 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.616 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.617 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:34.617 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.379565+0000", 2024-09-21T14:58:34.617 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4523e-05 2024-09-21T14:58:34.617 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.617 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.617 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:34.617 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.379575+0000", 2024-09-21T14:58:34.617 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2259999999999994e-06 2024-09-21T14:58:34.617 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.617 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.617 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:34.617 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.379602+0000", 2024-09-21T14:58:34.617 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7076e-05 2024-09-21T14:58:34.617 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.617 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.618 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:34.618 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.379659+0000", 2024-09-21T14:58:34.618 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7701999999999997e-05 2024-09-21T14:58:34.618 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.618 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.618 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 6", 2024-09-21T14:58:34.618 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.379779+0000", 2024-09-21T14:58:34.618 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000119707 2024-09-21T14:58:34.618 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.618 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.618 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:34.618 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.380566+0000", 2024-09-21T14:58:34.618 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00078656799999999997 2024-09-21T14:58:34.618 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.619 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.619 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:34.619 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.382384+0000", 2024-09-21T14:58:34.619 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001817988 2024-09-21T14:58:34.619 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.619 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.619 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:34.619 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.382433+0000", 2024-09-21T14:58:34.619 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9432999999999999e-05 2024-09-21T14:58:34.619 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.619 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.619 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:34.619 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.382467+0000", 2024-09-21T14:58:34.619 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3679000000000001e-05 2024-09-21T14:58:34.619 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:34.620 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:34.620 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:34.620 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:34.620 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:34.620 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:58:34.620 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T14:58:34.622 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T14:58:34.622 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:58:34.615+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:58:34.723 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:58:34.715+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:58:34.740 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:58:34.740 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:58:34.740 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T14:58:34.741 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:58:34.751 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T14:58:34.824 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:58:34.820+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:58:34.845 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:58:34.846 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:58:34.846 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T14:58:34.846 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T14:58:34.846 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:58:34.855 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T14:58:34.925 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:58:34.924+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:58:34.968 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:58:34.968 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T14:58:34.968 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T14:58:34.968 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T14:58:34.968 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.968 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T14:58:34.969 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.205754+0000", 2024-09-21T14:58:34.969 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 31.757630429999999, 2024-09-21T14:58:34.969 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.41117094199999998, 2024-09-21T14:58:34.969 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:34.969 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:34.969 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:34.969 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:34.969 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:34.969 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1 2024-09-21T14:58:34.969 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.969 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:34.969 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.969 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:34.969 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205754+0000", 2024-09-21T14:58:34.970 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:34.970 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.970 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.970 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:34.970 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205754+0000", 2024-09-21T14:58:34.970 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:34.970 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.970 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.970 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:34.970 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205752+0000", 2024-09-21T14:58:34.970 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T14:58:34.970 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.970 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.971 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:34.971 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205765+0000", 2024-09-21T14:58:34.971 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3230999999999999e-05 2024-09-21T14:58:34.971 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.971 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.971 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:34.971 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205782+0000", 2024-09-21T14:58:34.971 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6634e-05 2024-09-21T14:58:34.971 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.971 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.971 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:34.971 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205793+0000", 2024-09-21T14:58:34.971 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1297e-05 2024-09-21T14:58:34.972 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.972 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.972 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:34.972 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205867+0000", 2024-09-21T14:58:34.972 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4269000000000003e-05 2024-09-21T14:58:34.972 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.972 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.972 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T14:58:34.972 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205877+0000", 2024-09-21T14:58:34.972 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2690000000000005e-06 2024-09-21T14:58:34.972 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.972 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.972 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:34.972 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.613700+0000", 2024-09-21T14:58:34.973 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.407823455 2024-09-21T14:58:34.973 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.973 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.973 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:34.973 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.614203+0000", 2024-09-21T14:58:34.973 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00050311500000000005 2024-09-21T14:58:34.973 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.973 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.973 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:34.973 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.614750+0000", 2024-09-21T14:58:34.973 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00054738699999999998 2024-09-21T14:58:34.973 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.973 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.973 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:34.974 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616367+0000", 2024-09-21T14:58:34.974 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001616634 2024-09-21T14:58:34.974 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.974 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.974 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:34.974 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616574+0000", 2024-09-21T14:58:34.974 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00020668499999999999 2024-09-21T14:58:34.974 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.974 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.974 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:34.974 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616802+0000", 2024-09-21T14:58:34.974 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00022817900000000001 2024-09-21T14:58:34.974 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.974 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.975 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:34.975 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616925+0000", 2024-09-21T14:58:34.975 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012309699999999999 2024-09-21T14:58:34.975 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:34.975 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:34.975 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:34.975 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.975 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.975 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T14:58:34.975 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.666480+0000", 2024-09-21T14:58:34.975 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 31.296904974, 2024-09-21T14:58:34.975 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0098753969999999993, 2024-09-21T14:58:34.975 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:34.975 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:34.976 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:34.976 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:34.976 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:34.976 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 35 2024-09-21T14:58:34.976 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.976 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:34.976 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.976 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:34.976 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.666480+0000", 2024-09-21T14:58:34.976 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:34.976 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.976 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.976 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:34.976 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.666480+0000", 2024-09-21T14:58:34.976 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:34.977 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.977 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.977 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:34.977 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.666478+0000", 2024-09-21T14:58:34.977 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:34.977 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.977 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.977 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:34.977 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.666626+0000", 2024-09-21T14:58:34.977 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014799100000000001 2024-09-21T14:58:34.977 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.977 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.977 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:34.978 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.666675+0000", 2024-09-21T14:58:34.978 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9086999999999999e-05 2024-09-21T14:58:34.978 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.978 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.978 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:34.978 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.666696+0000", 2024-09-21T14:58:34.978 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0448000000000001e-05 2024-09-21T14:58:34.978 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.978 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.978 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:34.978 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.672759+0000", 2024-09-21T14:58:34.978 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0060628410000000002 2024-09-21T14:58:34.978 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.979 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.979 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:34.979 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.672891+0000", 2024-09-21T14:58:34.979 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013294700000000001 2024-09-21T14:58:34.979 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.979 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.979 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:34.979 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.673110+0000", 2024-09-21T14:58:34.979 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00021842500000000001 2024-09-21T14:58:34.979 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.979 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.979 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:34.979 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.676136+0000", 2024-09-21T14:58:34.979 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0030259739999999999 2024-09-21T14:58:34.979 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.980 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.980 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:34.980 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.676192+0000", 2024-09-21T14:58:34.980 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5795999999999999e-05 2024-09-21T14:58:34.980 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.980 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.980 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:34.980 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.676255+0000", 2024-09-21T14:58:34.980 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3548999999999998e-05 2024-09-21T14:58:34.980 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.980 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.980 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:34.980 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.676355+0000", 2024-09-21T14:58:34.980 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9786999999999999e-05 2024-09-21T14:58:34.981 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:34.981 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:34.981 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:34.981 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.981 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.981 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T14:58:34.981 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.666714+0000", 2024-09-21T14:58:34.981 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 31.296670353, 2024-09-21T14:58:34.981 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.010286645000000001, 2024-09-21T14:58:34.981 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:34.981 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:34.981 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:34.981 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:34.981 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:34.982 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 36 2024-09-21T14:58:34.982 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.982 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:34.982 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.982 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:34.982 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.666714+0000", 2024-09-21T14:58:34.982 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:34.982 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.982 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.982 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:34.982 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.666714+0000", 2024-09-21T14:58:34.982 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:34.982 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.982 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.983 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:34.983 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.666713+0000", 2024-09-21T14:58:34.983 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:34.983 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.983 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.983 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:34.983 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.666724+0000", 2024-09-21T14:58:34.983 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0939e-05 2024-09-21T14:58:34.983 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.983 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.983 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:34.983 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.666752+0000", 2024-09-21T14:58:34.983 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8725e-05 2024-09-21T14:58:34.983 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.984 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.984 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:34.984 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.666768+0000", 2024-09-21T14:58:34.984 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5274e-05 2024-09-21T14:58:34.984 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.984 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.984 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:34.984 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.674731+0000", 2024-09-21T14:58:34.984 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0079634179999999999 2024-09-21T14:58:34.984 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.984 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.984 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:34.984 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.674840+0000", 2024-09-21T14:58:34.984 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000108639 2024-09-21T14:58:34.985 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.985 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.985 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:34.985 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.674959+0000", 2024-09-21T14:58:34.985 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00011884599999999999 2024-09-21T14:58:34.985 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.985 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.985 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:34.985 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.676146+0000", 2024-09-21T14:58:34.985 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0011873680000000001 2024-09-21T14:58:34.985 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.985 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.985 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:34.985 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.676924+0000", 2024-09-21T14:58:34.986 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00077794099999999998 2024-09-21T14:58:34.986 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.986 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.986 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:34.986 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.676966+0000", 2024-09-21T14:58:34.986 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2172999999999998e-05 2024-09-21T14:58:34.986 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.986 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.986 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:34.986 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.677001+0000", 2024-09-21T14:58:34.986 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4755999999999997e-05 2024-09-21T14:58:34.986 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:34.986 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:34.987 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:34.987 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.987 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.987 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T14:58:34.987 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.666789+0000", 2024-09-21T14:58:34.987 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 31.296595282999998, 2024-09-21T14:58:34.987 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.010759082999999999, 2024-09-21T14:58:34.987 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:34.987 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:34.987 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:34.987 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:34.987 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:34.987 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 37 2024-09-21T14:58:34.987 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.988 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:34.988 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.988 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:34.988 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.666789+0000", 2024-09-21T14:58:34.988 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:34.988 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.988 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.988 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:34.988 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.666789+0000", 2024-09-21T14:58:34.988 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:34.988 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.988 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.988 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:34.988 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.666788+0000", 2024-09-21T14:58:34.989 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:58:34.989 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.989 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.989 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:34.989 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.666799+0000", 2024-09-21T14:58:34.989 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0532e-05 2024-09-21T14:58:34.989 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.989 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.989 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:34.989 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.666822+0000", 2024-09-21T14:58:34.989 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.323e-05 2024-09-21T14:58:34.989 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.989 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.989 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:34.990 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.666875+0000", 2024-09-21T14:58:34.990 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3071000000000001e-05 2024-09-21T14:58:34.990 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.990 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.990 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:34.990 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.675669+0000", 2024-09-21T14:58:34.990 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0087942860000000001 2024-09-21T14:58:34.990 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.990 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.990 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:34.990 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.675749+0000", 2024-09-21T14:58:34.990 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9260999999999999e-05 2024-09-21T14:58:34.990 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.990 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.991 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:34.991 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.675872+0000", 2024-09-21T14:58:34.991 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000123072 2024-09-21T14:58:34.991 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.991 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.991 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:34.991 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.676749+0000", 2024-09-21T14:58:34.991 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00087719600000000001 2024-09-21T14:58:34.991 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.991 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.991 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:34.991 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.677408+0000", 2024-09-21T14:58:34.991 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00065863300000000005 2024-09-21T14:58:34.991 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.992 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.992 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:34.992 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.677492+0000", 2024-09-21T14:58:34.992 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4934999999999999e-05 2024-09-21T14:58:34.992 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.992 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.992 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:34.992 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.677548+0000", 2024-09-21T14:58:34.992 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5881999999999998e-05 2024-09-21T14:58:34.992 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:34.992 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:34.992 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:34.992 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.993 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.993 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T14:58:34.993 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.748110+0000", 2024-09-21T14:58:34.993 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 31.215274158, 2024-09-21T14:58:34.993 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.010071220000000001, 2024-09-21T14:58:34.993 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:34.993 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:34.993 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:34.993 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:34.993 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:34.993 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 88 2024-09-21T14:58:34.993 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.993 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:34.993 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.993 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:34.994 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.748110+0000", 2024-09-21T14:58:34.994 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:34.994 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.994 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.994 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:34.994 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.748110+0000", 2024-09-21T14:58:34.994 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:34.994 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.994 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.994 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:34.994 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.748110+0000", 2024-09-21T14:58:34.994 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:58:34.994 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.994 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.995 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:34.995 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.748116+0000", 2024-09-21T14:58:34.995 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5230000000000001e-06 2024-09-21T14:58:34.995 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.995 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.995 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:34.995 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.748130+0000", 2024-09-21T14:58:34.995 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4248e-05 2024-09-21T14:58:34.995 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.995 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.995 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:34.995 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.748159+0000", 2024-09-21T14:58:34.995 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8931000000000001e-05 2024-09-21T14:58:34.995 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.996 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.996 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:34.996 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.755598+0000", 2024-09-21T14:58:34.996 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0074386820000000003 2024-09-21T14:58:34.996 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.996 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.996 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:34.996 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.755778+0000", 2024-09-21T14:58:34.996 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00018027399999999999 2024-09-21T14:58:34.996 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.996 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.996 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:34.996 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.756059+0000", 2024-09-21T14:58:34.996 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00028083600000000002 2024-09-21T14:58:34.997 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.997 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.997 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:34.997 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.757447+0000", 2024-09-21T14:58:34.997 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0013877939999999999 2024-09-21T14:58:34.997 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.997 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.997 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:34.997 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.758054+0000", 2024-09-21T14:58:34.997 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000606979 2024-09-21T14:58:34.997 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.997 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.997 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:34.997 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.758128+0000", 2024-09-21T14:58:34.998 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4493999999999995e-05 2024-09-21T14:58:34.998 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.998 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.998 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:34.998 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.758182+0000", 2024-09-21T14:58:34.998 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3247e-05 2024-09-21T14:58:34.998 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:34.998 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:34.998 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:34.998 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.998 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.998 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T14:58:34.998 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.954952+0000", 2024-09-21T14:58:34.998 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 31.008432592999998, 2024-09-21T14:58:34.999 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0097988899999999993, 2024-09-21T14:58:34.999 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:34.999 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:34.999 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:34.999 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:34.999 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:34.999 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 111 2024-09-21T14:58:34.999 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.999 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:34.999 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:34.999 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:34.999 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.954952+0000", 2024-09-21T14:58:34.999 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:34.999 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:34.999 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.000 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:35.000 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.954952+0000", 2024-09-21T14:58:35.000 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:35.000 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.000 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.000 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:35.000 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.954952+0000", 2024-09-21T14:58:35.000 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:35.000 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.000 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.000 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:35.000 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.954955+0000", 2024-09-21T14:58:35.000 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2440000000000002e-06 2024-09-21T14:58:35.000 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.001 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.001 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:35.001 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.954960+0000", 2024-09-21T14:58:35.001 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8119999999999998e-06 2024-09-21T14:58:35.001 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.001 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.001 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:35.001 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.954963+0000", 2024-09-21T14:58:35.001 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4029999999999999e-06 2024-09-21T14:58:35.001 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.001 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.001 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:35.001 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.960407+0000", 2024-09-21T14:58:35.001 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0054443360000000001 2024-09-21T14:58:35.002 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.002 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.002 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:35.002 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.960496+0000", 2024-09-21T14:58:35.002 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8978000000000001e-05 2024-09-21T14:58:35.002 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.002 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.002 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:35.002 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.960635+0000", 2024-09-21T14:58:35.002 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013853300000000001 2024-09-21T14:58:35.002 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.002 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.002 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:35.002 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.964189+0000", 2024-09-21T14:58:35.003 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0035545590000000001 2024-09-21T14:58:35.003 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.003 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.003 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:35.003 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.964571+0000", 2024-09-21T14:58:35.003 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00038135799999999999 2024-09-21T14:58:35.003 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.003 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.003 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:35.003 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.964685+0000", 2024-09-21T14:58:35.003 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00011391300000000001 2024-09-21T14:58:35.003 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.003 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.003 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:35.004 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.964751+0000", 2024-09-21T14:58:35.004 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6111000000000005e-05 2024-09-21T14:58:35.004 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.004 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:35.004 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.004 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.004 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.004 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T14:58:35.004 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.955076+0000", 2024-09-21T14:58:35.004 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 31.008308674999999, 2024-09-21T14:58:35.004 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.010426615, 2024-09-21T14:58:35.004 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:35.004 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:35.004 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:35.005 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:35.005 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:35.005 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 112 2024-09-21T14:58:35.005 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.005 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:35.005 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.005 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:35.005 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955076+0000", 2024-09-21T14:58:35.005 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:35.005 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.005 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.005 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:35.005 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955076+0000", 2024-09-21T14:58:35.005 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:35.005 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.006 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.006 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:35.006 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955075+0000", 2024-09-21T14:58:35.006 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:58:35.006 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.006 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.006 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:35.006 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955122+0000", 2024-09-21T14:58:35.006 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6802999999999999e-05 2024-09-21T14:58:35.006 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.006 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.006 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:35.006 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955130+0000", 2024-09-21T14:58:35.006 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0490000000000004e-06 2024-09-21T14:58:35.007 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.007 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.007 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:35.007 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955136+0000", 2024-09-21T14:58:35.007 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9059999999999999e-06 2024-09-21T14:58:35.007 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.007 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.007 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:35.007 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.961454+0000", 2024-09-21T14:58:35.007 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0063180429999999997 2024-09-21T14:58:35.007 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.007 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.007 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:35.007 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.961600+0000", 2024-09-21T14:58:35.008 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014651700000000001 2024-09-21T14:58:35.008 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.008 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.008 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:35.008 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.961798+0000", 2024-09-21T14:58:35.008 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00019784100000000001 2024-09-21T14:58:35.008 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.008 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.008 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:35.008 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.964785+0000", 2024-09-21T14:58:35.008 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.002986561 2024-09-21T14:58:35.008 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.008 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.008 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:35.009 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.965422+0000", 2024-09-21T14:58:35.009 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00063701800000000005 2024-09-21T14:58:35.009 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.009 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.009 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:35.009 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.965466+0000", 2024-09-21T14:58:35.009 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4020999999999997e-05 2024-09-21T14:58:35.009 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.009 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.009 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:35.009 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.965503+0000", 2024-09-21T14:58:35.009 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6814999999999999e-05 2024-09-21T14:58:35.009 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.009 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:35.010 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.010 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.010 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.010 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T14:58:35.010 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.955142+0000", 2024-09-21T14:58:35.010 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 31.008242783, 2024-09-21T14:58:35.010 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012388923, 2024-09-21T14:58:35.010 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:35.010 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:35.010 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:35.010 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:35.010 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:35.010 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 113 2024-09-21T14:58:35.010 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.011 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:35.011 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.011 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:35.011 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955142+0000", 2024-09-21T14:58:35.011 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:35.011 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.011 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.011 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:35.011 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955142+0000", 2024-09-21T14:58:35.011 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:35.011 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.011 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.011 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:35.011 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955141+0000", 2024-09-21T14:58:35.012 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:35.012 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.012 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.012 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:35.012 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955145+0000", 2024-09-21T14:58:35.012 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2109999999999998e-06 2024-09-21T14:58:35.012 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.012 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.012 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:35.012 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955150+0000", 2024-09-21T14:58:35.012 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4090000000000002e-06 2024-09-21T14:58:35.012 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.012 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.013 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:35.013 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955153+0000", 2024-09-21T14:58:35.013 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1240000000000001e-06 2024-09-21T14:58:35.013 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.013 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.013 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:35.013 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.963026+0000", 2024-09-21T14:58:35.013 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0078724060000000002 2024-09-21T14:58:35.013 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.013 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.013 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:35.013 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.963126+0000", 2024-09-21T14:58:35.013 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000100206 2024-09-21T14:58:35.013 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.013 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.014 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:35.014 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.963277+0000", 2024-09-21T14:58:35.014 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015159200000000001 2024-09-21T14:58:35.014 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.014 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.014 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:35.014 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.965519+0000", 2024-09-21T14:58:35.014 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0022418440000000002 2024-09-21T14:58:35.014 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.014 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.014 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:35.014 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967305+0000", 2024-09-21T14:58:35.014 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001785525 2024-09-21T14:58:35.015 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.015 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.018 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:35.018 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967438+0000", 2024-09-21T14:58:35.018 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013284399999999999 2024-09-21T14:58:35.018 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.018 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.019 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:35.019 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967531+0000", 2024-09-21T14:58:35.019 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.3140000000000006e-05 2024-09-21T14:58:35.019 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.019 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:35.019 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.019 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.019 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.019 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T14:58:35.019 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.955387+0000", 2024-09-21T14:58:35.019 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 31.007997848999999, 2024-09-21T14:58:35.019 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012319478999999999, 2024-09-21T14:58:35.019 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:35.019 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:35.020 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:35.020 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:35.020 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:35.020 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 114 2024-09-21T14:58:35.020 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.020 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:35.020 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.020 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:35.020 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955387+0000", 2024-09-21T14:58:35.020 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:35.020 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.020 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.020 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:35.020 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955387+0000", 2024-09-21T14:58:35.020 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:35.021 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.021 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.021 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:35.021 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955386+0000", 2024-09-21T14:58:35.021 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:58:35.021 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.021 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.021 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:35.021 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955392+0000", 2024-09-21T14:58:35.021 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4740000000000002e-06 2024-09-21T14:58:35.021 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.021 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.021 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:35.021 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955400+0000", 2024-09-21T14:58:35.022 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2139999999999996e-06 2024-09-21T14:58:35.022 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.022 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.022 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:35.022 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955407+0000", 2024-09-21T14:58:35.022 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2299999999999996e-06 2024-09-21T14:58:35.022 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.022 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.022 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:35.022 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.964840+0000", 2024-09-21T14:58:35.022 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0094330079999999997 2024-09-21T14:58:35.022 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.022 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.022 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:35.023 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.964917+0000", 2024-09-21T14:58:35.023 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7071999999999997e-05 2024-09-21T14:58:35.023 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.023 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.023 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:35.023 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.965026+0000", 2024-09-21T14:58:35.023 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000109646 2024-09-21T14:58:35.023 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.023 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.023 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:35.023 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967609+0000", 2024-09-21T14:58:35.023 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0025824929999999999 2024-09-21T14:58:35.023 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.023 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.024 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:35.024 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967629+0000", 2024-09-21T14:58:35.024 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0486999999999999e-05 2024-09-21T14:58:35.024 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.024 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.024 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:35.024 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967672+0000", 2024-09-21T14:58:35.024 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2314999999999997e-05 2024-09-21T14:58:35.024 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.024 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.024 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:35.024 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967706+0000", 2024-09-21T14:58:35.024 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4483999999999997e-05 2024-09-21T14:58:35.024 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.025 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:35.025 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.025 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.025 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.025 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T14:58:35.025 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.955609+0000", 2024-09-21T14:58:35.025 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 31.007776043, 2024-09-21T14:58:35.025 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013624153999999999, 2024-09-21T14:58:35.025 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:35.025 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:35.025 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:35.025 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:35.025 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:35.026 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 115 2024-09-21T14:58:35.026 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.026 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:35.026 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.026 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:35.026 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955609+0000", 2024-09-21T14:58:35.026 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:35.026 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.026 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.026 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:35.026 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955609+0000", 2024-09-21T14:58:35.026 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:35.026 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.027 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.027 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:35.027 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955607+0000", 2024-09-21T14:58:35.027 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-21T14:58:35.027 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.027 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.027 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:35.027 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955732+0000", 2024-09-21T14:58:35.027 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000125566 2024-09-21T14:58:35.027 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.027 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.027 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:35.027 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955752+0000", 2024-09-21T14:58:35.027 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9916999999999999e-05 2024-09-21T14:58:35.028 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.028 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.028 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:35.028 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955767+0000", 2024-09-21T14:58:35.028 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4687e-05 2024-09-21T14:58:35.028 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.028 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.028 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:35.028 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.965574+0000", 2024-09-21T14:58:35.028 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0098072850000000007 2024-09-21T14:58:35.028 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.028 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.028 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:35.028 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.965642+0000", 2024-09-21T14:58:35.029 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7784e-05 2024-09-21T14:58:35.029 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.029 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.029 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:35.029 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.965753+0000", 2024-09-21T14:58:35.029 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000111111 2024-09-21T14:58:35.029 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.029 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.029 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:35.029 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967717+0000", 2024-09-21T14:58:35.029 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0019635339999999999 2024-09-21T14:58:35.029 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.029 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.029 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:35.030 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.969014+0000", 2024-09-21T14:58:35.030 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001297093 2024-09-21T14:58:35.030 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.030 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.030 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:35.030 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.969130+0000", 2024-09-21T14:58:35.030 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000116011 2024-09-21T14:58:35.030 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.030 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.030 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:35.030 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.969233+0000", 2024-09-21T14:58:35.030 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000102982 2024-09-21T14:58:35.030 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.030 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:35.031 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.031 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.031 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.031 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T14:58:35.031 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.955783+0000", 2024-09-21T14:58:35.031 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 31.007601374, 2024-09-21T14:58:35.031 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014630241, 2024-09-21T14:58:35.031 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:35.031 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:35.031 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:35.031 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:35.031 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:35.031 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 116 2024-09-21T14:58:35.031 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.032 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:35.032 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.032 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:35.032 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955783+0000", 2024-09-21T14:58:35.032 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:35.032 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.032 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.032 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:35.032 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955783+0000", 2024-09-21T14:58:35.032 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:35.032 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.032 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.032 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:35.033 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955782+0000", 2024-09-21T14:58:35.033 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:58:35.033 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.033 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.033 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:35.033 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955791+0000", 2024-09-21T14:58:35.033 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8089999999999999e-06 2024-09-21T14:58:35.033 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.033 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.033 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:35.033 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955804+0000", 2024-09-21T14:58:35.033 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2548000000000001e-05 2024-09-21T14:58:35.033 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.033 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.034 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:35.034 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955814+0000", 2024-09-21T14:58:35.034 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0158e-05 2024-09-21T14:58:35.034 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.034 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.034 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:35.034 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.966515+0000", 2024-09-21T14:58:35.034 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.010701756 2024-09-21T14:58:35.034 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.034 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.034 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:35.034 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.966606+0000", 2024-09-21T14:58:35.034 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0761999999999996e-05 2024-09-21T14:58:35.034 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.034 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.035 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:35.035 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.966740+0000", 2024-09-21T14:58:35.035 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000133693 2024-09-21T14:58:35.035 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.035 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.035 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:35.035 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.968920+0000", 2024-09-21T14:58:35.035 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0021799290000000002 2024-09-21T14:58:35.035 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.035 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.035 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:35.035 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.970320+0000", 2024-09-21T14:58:35.035 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0014001759999999999 2024-09-21T14:58:35.035 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.036 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.036 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:35.036 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.970376+0000", 2024-09-21T14:58:35.036 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5696999999999999e-05 2024-09-21T14:58:35.036 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.036 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.036 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:35.036 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.970413+0000", 2024-09-21T14:58:35.036 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7750999999999998e-05 2024-09-21T14:58:35.036 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.036 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:35.036 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.036 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.036 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.037 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T14:58:35.037 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.955829+0000", 2024-09-21T14:58:35.037 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 31.007555904, 2024-09-21T14:58:35.037 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014919597999999999, 2024-09-21T14:58:35.037 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:35.037 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:35.037 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:35.037 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:35.037 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:35.037 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 117 2024-09-21T14:58:35.037 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.037 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:35.037 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.037 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:35.037 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955829+0000", 2024-09-21T14:58:35.038 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:35.038 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.038 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.038 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:35.038 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955829+0000", 2024-09-21T14:58:35.038 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:35.038 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.038 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.038 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:35.038 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955827+0000", 2024-09-21T14:58:35.038 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:35.038 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.038 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.039 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:35.039 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955838+0000", 2024-09-21T14:58:35.039 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.111e-05 2024-09-21T14:58:35.039 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.039 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.039 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:35.039 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955875+0000", 2024-09-21T14:58:35.039 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6211999999999998e-05 2024-09-21T14:58:35.039 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.039 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.039 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:35.039 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955899+0000", 2024-09-21T14:58:35.039 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4128999999999999e-05 2024-09-21T14:58:35.039 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.040 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.040 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:35.040 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967767+0000", 2024-09-21T14:58:35.040 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.011868251999999999 2024-09-21T14:58:35.040 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.040 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.040 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:35.040 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967866+0000", 2024-09-21T14:58:35.040 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8664000000000003e-05 2024-09-21T14:58:35.040 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.040 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.040 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:35.040 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.968128+0000", 2024-09-21T14:58:35.040 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00026243600000000001 2024-09-21T14:58:35.041 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.041 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.041 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:35.041 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.969393+0000", 2024-09-21T14:58:35.041 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0012650350000000001 2024-09-21T14:58:35.041 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.041 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.041 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:35.041 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.970523+0000", 2024-09-21T14:58:35.041 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001129557 2024-09-21T14:58:35.041 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.041 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.041 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:35.041 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.970603+0000", 2024-09-21T14:58:35.042 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0090000000000001e-05 2024-09-21T14:58:35.042 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.042 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.042 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:35.042 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.970748+0000", 2024-09-21T14:58:35.042 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014552799999999999 2024-09-21T14:58:35.042 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.042 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:35.042 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.042 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.042 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.042 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T14:58:35.042 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:04.492934+0000", 2024-09-21T14:58:35.042 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 30.470450657000001, 2024-09-21T14:58:35.043 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.010845886000000001, 2024-09-21T14:58:35.043 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:35.043 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:35.043 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:35.043 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:35.043 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:35.043 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 134 2024-09-21T14:58:35.043 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.043 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:35.043 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.043 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:35.043 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.492934+0000", 2024-09-21T14:58:35.043 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:35.044 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.044 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.044 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:35.044 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.492934+0000", 2024-09-21T14:58:35.044 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:35.044 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.044 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.044 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:35.044 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.492933+0000", 2024-09-21T14:58:35.044 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:58:35.044 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.044 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.044 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:35.044 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493094+0000", 2024-09-21T14:58:35.044 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00016101599999999999 2024-09-21T14:58:35.045 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.045 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.045 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:35.045 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493110+0000", 2024-09-21T14:58:35.045 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5447999999999998e-05 2024-09-21T14:58:35.045 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.045 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.045 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:35.045 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493119+0000", 2024-09-21T14:58:35.045 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.3170000000000008e-06 2024-09-21T14:58:35.045 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.045 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.045 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:35.045 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.498294+0000", 2024-09-21T14:58:35.046 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0051747700000000004 2024-09-21T14:58:35.046 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.046 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.046 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:35.046 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.498456+0000", 2024-09-21T14:58:35.046 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00016253599999999999 2024-09-21T14:58:35.046 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.046 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.046 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:35.046 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.498626+0000", 2024-09-21T14:58:35.046 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00016929900000000001 2024-09-21T14:58:35.046 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.046 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.046 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:35.047 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.502513+0000", 2024-09-21T14:58:35.047 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0038875530000000002 2024-09-21T14:58:35.047 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.047 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.047 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:35.047 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.503655+0000", 2024-09-21T14:58:35.047 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001142405 2024-09-21T14:58:35.047 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.047 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.047 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:35.047 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.503741+0000", 2024-09-21T14:58:35.047 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5918e-05 2024-09-21T14:58:35.047 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.047 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.048 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:35.048 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.503780+0000", 2024-09-21T14:58:35.048 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.8401e-05 2024-09-21T14:58:35.048 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.048 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:35.048 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.048 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.048 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.048 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T14:58:35.048 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:04.493130+0000", 2024-09-21T14:58:35.048 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 30.470254498999999, 2024-09-21T14:58:35.048 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012013845, 2024-09-21T14:58:35.048 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:35.048 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:35.049 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:35.049 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:35.049 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:35.049 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 135 2024-09-21T14:58:35.049 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.049 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:35.049 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.049 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:35.049 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493130+0000", 2024-09-21T14:58:35.049 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:35.049 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.049 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.049 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:35.049 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493130+0000", 2024-09-21T14:58:35.049 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:35.050 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.050 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.050 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:35.050 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493129+0000", 2024-09-21T14:58:35.050 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:58:35.050 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.050 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.050 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:35.050 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493135+0000", 2024-09-21T14:58:35.050 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1290000000000001e-06 2024-09-21T14:58:35.050 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.050 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.050 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:35.050 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493143+0000", 2024-09-21T14:58:35.051 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0139999999999998e-06 2024-09-21T14:58:35.051 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.051 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.051 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:35.051 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493150+0000", 2024-09-21T14:58:35.051 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.674e-06 2024-09-21T14:58:35.051 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.051 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.051 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:35.051 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.500089+0000", 2024-09-21T14:58:35.051 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0069391599999999998 2024-09-21T14:58:35.051 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.051 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.051 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:35.052 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.500277+0000", 2024-09-21T14:58:35.052 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00018736999999999999 2024-09-21T14:58:35.052 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.052 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.052 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:35.052 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.500426+0000", 2024-09-21T14:58:35.052 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014981900000000001 2024-09-21T14:58:35.052 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.052 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.052 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:35.052 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.503817+0000", 2024-09-21T14:58:35.052 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0033905150000000002 2024-09-21T14:58:35.052 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.052 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.052 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:35.053 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.504994+0000", 2024-09-21T14:58:35.053 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0011767500000000001 2024-09-21T14:58:35.053 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.053 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.053 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:35.053 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.505083+0000", 2024-09-21T14:58:35.053 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.9339999999999995e-05 2024-09-21T14:58:35.053 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.053 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.053 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:35.053 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.505144+0000", 2024-09-21T14:58:35.053 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0934000000000003e-05 2024-09-21T14:58:35.053 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.053 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:35.054 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.054 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.054 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.054 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T14:58:35.054 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:04.493201+0000", 2024-09-21T14:58:35.054 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 30.470183155000001, 2024-09-21T14:58:35.054 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.01208676, 2024-09-21T14:58:35.054 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:35.054 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:35.054 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:35.054 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:35.054 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:35.054 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 136 2024-09-21T14:58:35.054 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.054 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:35.055 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.055 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:35.055 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493201+0000", 2024-09-21T14:58:35.055 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:35.055 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.055 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.055 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:35.055 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493201+0000", 2024-09-21T14:58:35.055 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:35.055 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.055 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.055 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:35.055 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493200+0000", 2024-09-21T14:58:35.055 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:58:35.056 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.056 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.056 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:35.056 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493214+0000", 2024-09-21T14:58:35.056 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3488e-05 2024-09-21T14:58:35.056 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.056 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.056 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:35.056 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493244+0000", 2024-09-21T14:58:35.056 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9848e-05 2024-09-21T14:58:35.056 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.056 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.056 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:35.056 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493263+0000", 2024-09-21T14:58:35.057 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9071000000000001e-05 2024-09-21T14:58:35.057 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.057 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.057 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:35.057 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.501197+0000", 2024-09-21T14:58:35.057 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.007934623 2024-09-21T14:58:35.057 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.057 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.057 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:35.057 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.501327+0000", 2024-09-21T14:58:35.057 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000129605 2024-09-21T14:58:35.057 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.057 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.057 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:35.058 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.501518+0000", 2024-09-21T14:58:35.058 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000190662 2024-09-21T14:58:35.058 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.058 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.058 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:35.058 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.505175+0000", 2024-09-21T14:58:35.058 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0036576989999999999 2024-09-21T14:58:35.058 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.058 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.058 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:35.058 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.505216+0000", 2024-09-21T14:58:35.058 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1097000000000003e-05 2024-09-21T14:58:35.058 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.058 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.059 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:35.059 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.505251+0000", 2024-09-21T14:58:35.059 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4758000000000002e-05 2024-09-21T14:58:35.059 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.059 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.059 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:35.059 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.505288+0000", 2024-09-21T14:58:35.059 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6913999999999999e-05 2024-09-21T14:58:35.059 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.059 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:35.059 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.059 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.059 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.059 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T14:58:35.059 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:04.493315+0000", 2024-09-21T14:58:35.060 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 30.470069071000001, 2024-09-21T14:58:35.060 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014055766000000001, 2024-09-21T14:58:35.060 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:35.060 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:35.060 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:35.060 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:35.060 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:35.060 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 137 2024-09-21T14:58:35.060 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.060 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:35.060 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.060 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:35.060 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493315+0000", 2024-09-21T14:58:35.060 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:35.061 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.061 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.061 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:35.061 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493315+0000", 2024-09-21T14:58:35.061 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:35.061 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.061 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.061 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:35.061 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493314+0000", 2024-09-21T14:58:35.061 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:58:35.061 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.061 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.061 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:35.061 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493507+0000", 2024-09-21T14:58:35.062 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000192444 2024-09-21T14:58:35.062 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.062 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.062 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:35.062 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493528+0000", 2024-09-21T14:58:35.062 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0845000000000001e-05 2024-09-21T14:58:35.062 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.062 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.062 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:35.062 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493543+0000", 2024-09-21T14:58:35.062 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5800000000000001e-05 2024-09-21T14:58:35.062 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.062 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.062 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:35.063 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.502575+0000", 2024-09-21T14:58:35.063 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0090312389999999999 2024-09-21T14:58:35.063 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.063 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.063 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:35.063 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.502677+0000", 2024-09-21T14:58:35.063 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000102271 2024-09-21T14:58:35.063 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.063 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.063 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:35.063 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.502825+0000", 2024-09-21T14:58:35.063 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000148171 2024-09-21T14:58:35.063 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.063 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.064 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:35.064 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507251+0000", 2024-09-21T14:58:35.064 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0044253879999999997 2024-09-21T14:58:35.064 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.064 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.064 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:35.064 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507313+0000", 2024-09-21T14:58:35.064 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2170999999999998e-05 2024-09-21T14:58:35.064 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.064 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.064 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:35.064 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507346+0000", 2024-09-21T14:58:35.064 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3079e-05 2024-09-21T14:58:35.064 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.065 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.065 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:35.065 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507371+0000", 2024-09-21T14:58:35.065 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5477999999999999e-05 2024-09-21T14:58:35.065 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.065 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:35.065 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.065 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.065 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.065 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T14:58:35.065 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:04.493570+0000", 2024-09-21T14:58:35.065 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 30.469814666000001, 2024-09-21T14:58:35.065 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014061686, 2024-09-21T14:58:35.065 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:35.066 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:35.066 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:35.066 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:35.066 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:35.066 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 138 2024-09-21T14:58:35.066 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.066 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:35.066 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.066 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:35.066 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493570+0000", 2024-09-21T14:58:35.066 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:35.066 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.066 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.066 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:35.067 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493570+0000", 2024-09-21T14:58:35.067 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:35.067 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.067 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.067 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:35.067 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493568+0000", 2024-09-21T14:58:35.067 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:35.067 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.067 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.067 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:35.067 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493579+0000", 2024-09-21T14:58:35.067 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0574000000000001e-05 2024-09-21T14:58:35.067 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.067 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.068 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:35.068 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493595+0000", 2024-09-21T14:58:35.068 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6504999999999999e-05 2024-09-21T14:58:35.068 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.068 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.068 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:35.068 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493608+0000", 2024-09-21T14:58:35.068 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2836999999999999e-05 2024-09-21T14:58:35.068 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.068 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.068 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:35.068 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.503909+0000", 2024-09-21T14:58:35.068 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.01030118 2024-09-21T14:58:35.068 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.069 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.069 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:35.069 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.504064+0000", 2024-09-21T14:58:35.069 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015493000000000001 2024-09-21T14:58:35.069 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.069 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.069 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:35.069 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.504299+0000", 2024-09-21T14:58:35.069 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00023416100000000001 2024-09-21T14:58:35.069 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.069 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.069 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:35.069 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507466+0000", 2024-09-21T14:58:35.069 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0031678380000000001 2024-09-21T14:58:35.070 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.070 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.070 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:35.070 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507483+0000", 2024-09-21T14:58:35.070 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6606999999999999e-05 2024-09-21T14:58:35.070 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.070 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.070 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:35.070 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507566+0000", 2024-09-21T14:58:35.070 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3110999999999998e-05 2024-09-21T14:58:35.070 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.070 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.070 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:35.070 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507632+0000", 2024-09-21T14:58:35.071 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5480000000000003e-05 2024-09-21T14:58:35.071 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.071 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:35.071 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.071 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.071 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.071 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T14:58:35.071 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:04.493625+0000", 2024-09-21T14:58:35.071 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 30.469759977999999, 2024-09-21T14:58:35.071 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014665378999999999, 2024-09-21T14:58:35.071 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:35.071 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:35.071 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:35.071 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:35.072 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:35.072 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139 2024-09-21T14:58:35.072 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.072 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:35.072 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.072 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:35.072 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493625+0000", 2024-09-21T14:58:35.072 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:35.072 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.072 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.072 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:35.072 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493625+0000", 2024-09-21T14:58:35.072 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:35.073 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.073 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.073 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:35.073 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493624+0000", 2024-09-21T14:58:35.073 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:58:35.073 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.073 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.073 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:35.073 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493638+0000", 2024-09-21T14:58:35.073 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4282e-05 2024-09-21T14:58:35.073 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.073 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.073 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:35.073 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493654+0000", 2024-09-21T14:58:35.073 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5739999999999998e-05 2024-09-21T14:58:35.074 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.074 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.074 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:35.074 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493666+0000", 2024-09-21T14:58:35.074 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2865e-05 2024-09-21T14:58:35.074 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.074 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.074 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:35.074 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.505329+0000", 2024-09-21T14:58:35.074 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.011662237000000001 2024-09-21T14:58:35.074 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.074 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.074 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:35.074 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.505377+0000", 2024-09-21T14:58:35.075 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7862999999999999e-05 2024-09-21T14:58:35.075 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.075 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.075 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:35.075 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.505455+0000", 2024-09-21T14:58:35.075 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8918000000000006e-05 2024-09-21T14:58:35.075 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.075 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.075 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:35.075 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507650+0000", 2024-09-21T14:58:35.075 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.002195027 2024-09-21T14:58:35.075 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.075 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.076 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:35.076 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508149+0000", 2024-09-21T14:58:35.076 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00049828399999999999 2024-09-21T14:58:35.076 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.076 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.076 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:35.076 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508230+0000", 2024-09-21T14:58:35.076 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1376000000000002e-05 2024-09-21T14:58:35.076 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.076 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.076 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:35.076 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508290+0000", 2024-09-21T14:58:35.076 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9852999999999998e-05 2024-09-21T14:58:35.076 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.077 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:35.077 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.077 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.077 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.077 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T14:58:35.077 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:04.493679+0000", 2024-09-21T14:58:35.077 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 30.469705080000001, 2024-09-21T14:58:35.077 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014869869000000001, 2024-09-21T14:58:35.077 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:35.077 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:35.077 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:35.077 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:35.077 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:35.077 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140 2024-09-21T14:58:35.078 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.078 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:35.078 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.078 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:35.078 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493679+0000", 2024-09-21T14:58:35.078 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:35.078 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.078 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.078 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:35.078 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493679+0000", 2024-09-21T14:58:35.078 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:35.078 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.079 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.079 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:35.079 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493679+0000", 2024-09-21T14:58:35.079 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:58:35.079 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.079 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.079 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:35.079 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493889+0000", 2024-09-21T14:58:35.079 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00021032700000000001 2024-09-21T14:58:35.079 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.079 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.079 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:35.079 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493931+0000", 2024-09-21T14:58:35.080 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1971000000000002e-05 2024-09-21T14:58:35.080 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.080 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.080 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:35.080 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493947+0000", 2024-09-21T14:58:35.080 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6385e-05 2024-09-21T14:58:35.080 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.080 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.080 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:35.080 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.505804+0000", 2024-09-21T14:58:35.080 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.011857136000000001 2024-09-21T14:58:35.080 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.080 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.080 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:35.081 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.505992+0000", 2024-09-21T14:58:35.081 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000187665 2024-09-21T14:58:35.081 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.081 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.081 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:35.081 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.506281+0000", 2024-09-21T14:58:35.081 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000288712 2024-09-21T14:58:35.081 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.081 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.081 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:35.081 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508349+0000", 2024-09-21T14:58:35.081 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0020685790000000001 2024-09-21T14:58:35.081 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.082 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.082 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:35.082 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508371+0000", 2024-09-21T14:58:35.082 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1747e-05 2024-09-21T14:58:35.082 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.082 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.082 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:35.082 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508450+0000", 2024-09-21T14:58:35.082 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9332000000000005e-05 2024-09-21T14:58:35.082 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.082 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.082 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:35.082 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508549+0000", 2024-09-21T14:58:35.082 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8918000000000004e-05 2024-09-21T14:58:35.083 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.083 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:35.083 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.083 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.083 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.083 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T14:58:35.083 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:04.493967+0000", 2024-09-21T14:58:35.083 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 30.469418043000001, 2024-09-21T14:58:35.083 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.015281075999999999, 2024-09-21T14:58:35.083 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:35.083 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:35.083 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:35.083 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:35.083 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:35.084 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 141 2024-09-21T14:58:35.084 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.084 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:35.084 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.084 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:35.084 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493967+0000", 2024-09-21T14:58:35.084 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:35.084 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.084 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.084 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:35.084 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493967+0000", 2024-09-21T14:58:35.084 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:35.084 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.084 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.085 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:35.085 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493965+0000", 2024-09-21T14:58:35.085 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:35.085 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.085 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.085 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:35.085 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493987+0000", 2024-09-21T14:58:35.085 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1662999999999999e-05 2024-09-21T14:58:35.085 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.085 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.085 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:35.085 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.494004+0000", 2024-09-21T14:58:35.085 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7311000000000001e-05 2024-09-21T14:58:35.085 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.086 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.086 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:35.086 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.494017+0000", 2024-09-21T14:58:35.086 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3202e-05 2024-09-21T14:58:35.086 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.086 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.086 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:35.086 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507678+0000", 2024-09-21T14:58:35.086 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013660432 2024-09-21T14:58:35.086 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.086 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.086 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:35.086 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507726+0000", 2024-09-21T14:58:35.086 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8154e-05 2024-09-21T14:58:35.087 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.087 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.087 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:35.087 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507797+0000", 2024-09-21T14:58:35.087 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.1110000000000002e-05 2024-09-21T14:58:35.087 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.087 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.087 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:35.087 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508710+0000", 2024-09-21T14:58:35.087 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00091339299999999997 2024-09-21T14:58:35.087 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.087 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.087 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:35.087 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.509108+0000", 2024-09-21T14:58:35.088 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00039718300000000001 2024-09-21T14:58:35.088 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.088 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.088 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:35.088 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.509186+0000", 2024-09-21T14:58:35.088 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8592999999999998e-05 2024-09-21T14:58:35.088 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.088 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.088 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:35.088 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.509248+0000", 2024-09-21T14:58:35.088 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1450000000000003e-05 2024-09-21T14:58:35.088 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.088 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:35.088 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.088 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.089 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:35.089 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:58:35.090 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T14:58:35.090 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:58:35.083+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:58:35.111 INFO:teuthology.orchestra.run.smithi032.stderr:noscrub is unset 2024-09-21T14:58:35.191 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:58:35.188+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:58:35.256 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:58:35.256 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:58:35.257 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T14:58:35.257 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:58:35.267 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T14:58:35.292 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:58:35.288+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:58:35.367 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:58:35.368 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:58:35.368 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T14:58:35.368 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T14:58:35.368 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:58:35.377 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T14:58:35.393 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:58:35.392+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:58:35.488 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:58:35.489 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T14:58:35.489 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T14:58:35.489 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T14:58:35.489 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.489 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:723 3.16 3:6e95a6fe:::rbd_header.113693316f36:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-21T14:58:35.489 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:26.454676+0000", 2024-09-21T14:58:35.489 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 9.0310630599999993, 2024-09-21T14:58:35.489 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.004911047, 2024-09-21T14:58:35.489 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:35.489 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:35.490 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:35.490 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:35.490 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:35.490 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 723 2024-09-21T14:58:35.490 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.490 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:35.490 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.490 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:35.490 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.454676+0000", 2024-09-21T14:58:35.490 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:35.491 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.491 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.491 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:35.491 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.454676+0000", 2024-09-21T14:58:35.491 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:35.491 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.491 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.491 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:35.491 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.454674+0000", 2024-09-21T14:58:35.491 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-21T14:58:35.491 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.491 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.492 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:35.492 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.454683+0000", 2024-09-21T14:58:35.492 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8880000000000003e-06 2024-09-21T14:58:35.492 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.492 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.492 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:35.492 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.454705+0000", 2024-09-21T14:58:35.492 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2393999999999999e-05 2024-09-21T14:58:35.492 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.493 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.493 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:35.493 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.454716+0000", 2024-09-21T14:58:35.493 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1304999999999999e-05 2024-09-21T14:58:35.493 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.493 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.493 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:35.493 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.454782+0000", 2024-09-21T14:58:35.493 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5963999999999994e-05 2024-09-21T14:58:35.493 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.494 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.494 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op must wait for map", 2024-09-21T14:58:35.494 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.454796+0000", 2024-09-21T14:58:35.494 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3925e-05 2024-09-21T14:58:35.494 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.494 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.494 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:35.494 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.457268+0000", 2024-09-21T14:58:35.494 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0024721130000000002 2024-09-21T14:58:35.494 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.494 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.494 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:35.495 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.457413+0000", 2024-09-21T14:58:35.495 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000145005 2024-09-21T14:58:35.495 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.495 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.495 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 3", 2024-09-21T14:58:35.495 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.457912+0000", 2024-09-21T14:58:35.495 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00049899999999999999 2024-09-21T14:58:35.495 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.495 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.495 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:35.495 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.458705+0000", 2024-09-21T14:58:35.495 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00079238400000000004 2024-09-21T14:58:35.495 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.495 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.496 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:35.496 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.459522+0000", 2024-09-21T14:58:35.496 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00081716299999999998 2024-09-21T14:58:35.496 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.496 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.496 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:35.496 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.459561+0000", 2024-09-21T14:58:35.496 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.8838000000000003e-05 2024-09-21T14:58:35.496 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.496 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.496 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:35.496 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.459587+0000", 2024-09-21T14:58:35.496 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6145000000000001e-05 2024-09-21T14:58:35.496 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.496 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:35.497 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.497 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.497 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.497 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:1454 3.e 3:71be909a:::rbd_data.1136515ecb6b.000000000000002a:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T14:58:35.497 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:30.339995+0000", 2024-09-21T14:58:35.497 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 5.1457438629999999, 2024-09-21T14:58:35.497 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.033615974999999999, 2024-09-21T14:58:35.497 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:35.497 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:35.497 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:35.497 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:35.497 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:35.497 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1454 2024-09-21T14:58:35.497 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.498 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:35.498 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.498 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:35.498 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.339995+0000", 2024-09-21T14:58:35.498 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:35.498 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.498 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.498 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:35.498 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.339995+0000", 2024-09-21T14:58:35.498 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:35.498 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.498 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.498 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:35.499 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.339992+0000", 2024-09-21T14:58:35.499 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-21T14:58:35.499 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.499 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.499 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:35.499 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.340008+0000", 2024-09-21T14:58:35.499 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5906e-05 2024-09-21T14:58:35.499 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.499 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.499 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:35.499 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.340042+0000", 2024-09-21T14:58:35.499 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4066000000000002e-05 2024-09-21T14:58:35.499 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.499 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.499 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:35.500 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.340056+0000", 2024-09-21T14:58:35.500 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4389999999999999e-05 2024-09-21T14:58:35.500 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.500 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.500 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:35.500 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.340178+0000", 2024-09-21T14:58:35.500 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012202 2024-09-21T14:58:35.500 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.500 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.500 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:35.500 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.340424+0000", 2024-09-21T14:58:35.500 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000245682 2024-09-21T14:58:35.500 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.500 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.500 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 0", 2024-09-21T14:58:35.501 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.348708+0000", 2024-09-21T14:58:35.501 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0082834390000000001 2024-09-21T14:58:35.501 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.501 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.501 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:35.501 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.373176+0000", 2024-09-21T14:58:35.501 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024468888000000001 2024-09-21T14:58:35.501 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.501 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.501 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:35.501 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.373306+0000", 2024-09-21T14:58:35.501 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012991700000000001 2024-09-21T14:58:35.501 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.501 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.502 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:35.502 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.373464+0000", 2024-09-21T14:58:35.502 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000157913 2024-09-21T14:58:35.502 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.502 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.502 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:35.502 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.373611+0000", 2024-09-21T14:58:35.502 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014681699999999999 2024-09-21T14:58:35.502 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.502 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:35.502 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.502 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.502 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.502 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:1657 3.16 3:681a4391:::rbd_data.1136515ecb6b.0000000000000037:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T14:58:35.503 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:30.943122+0000", 2024-09-21T14:58:35.503 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.5426167800000004, 2024-09-21T14:58:35.503 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0049027680000000001, 2024-09-21T14:58:35.503 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:35.503 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:35.503 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:35.503 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:35.503 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:35.503 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1657 2024-09-21T14:58:35.503 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.503 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:35.503 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.503 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:35.503 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.943122+0000", 2024-09-21T14:58:35.504 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:35.504 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.504 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.504 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:35.504 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.943122+0000", 2024-09-21T14:58:35.504 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:35.504 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.504 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.504 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:35.504 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.943120+0000", 2024-09-21T14:58:35.504 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T14:58:35.504 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.504 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.504 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:35.504 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.943133+0000", 2024-09-21T14:58:35.505 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3574000000000001e-05 2024-09-21T14:58:35.505 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.505 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.505 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:35.505 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.943159+0000", 2024-09-21T14:58:35.505 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5797000000000001e-05 2024-09-21T14:58:35.505 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.505 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.505 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:35.505 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.943177+0000", 2024-09-21T14:58:35.505 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8402000000000001e-05 2024-09-21T14:58:35.505 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.505 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.505 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:35.506 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.943268+0000", 2024-09-21T14:58:35.506 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0132999999999999e-05 2024-09-21T14:58:35.506 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.506 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.506 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:35.506 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.943506+0000", 2024-09-21T14:58:35.506 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00023839999999999999 2024-09-21T14:58:35.506 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.506 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.506 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 3", 2024-09-21T14:58:35.506 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.944415+0000", 2024-09-21T14:58:35.506 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00090852700000000001 2024-09-21T14:58:35.506 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.507 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.507 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:35.507 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.947720+0000", 2024-09-21T14:58:35.507 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0033050050000000002 2024-09-21T14:58:35.507 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.507 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.507 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:35.507 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.947815+0000", 2024-09-21T14:58:35.507 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5291999999999995e-05 2024-09-21T14:58:35.507 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.507 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.507 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:35.507 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.947960+0000", 2024-09-21T14:58:35.507 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014545899999999999 2024-09-21T14:58:35.507 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.508 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.508 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:35.508 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.948025+0000", 2024-09-21T14:58:35.508 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4659000000000006e-05 2024-09-21T14:58:35.508 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.508 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:35.508 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.508 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.508 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.508 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6244 3.16 3:68aadb42:::rbd_data.11361bed1f78.00000000000000cd:head [delete] snapc b=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-21T14:58:35.508 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:34.372480+0000", 2024-09-21T14:58:35.508 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.1132589129999999, 2024-09-21T14:58:35.508 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0055254279999999998, 2024-09-21T14:58:35.508 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:35.509 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:35.509 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:35.509 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:35.509 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:35.509 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6244 2024-09-21T14:58:35.509 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.509 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:35.509 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.509 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:35.509 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.372480+0000", 2024-09-21T14:58:35.509 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:35.509 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.509 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.509 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:35.510 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.372480+0000", 2024-09-21T14:58:35.510 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:35.510 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.510 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.510 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:35.510 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.372478+0000", 2024-09-21T14:58:35.510 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-21T14:58:35.510 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.510 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.510 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:35.510 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.372489+0000", 2024-09-21T14:58:35.510 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0859999999999999e-05 2024-09-21T14:58:35.510 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.511 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.511 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:35.511 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.372509+0000", 2024-09-21T14:58:35.511 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0169e-05 2024-09-21T14:58:35.511 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.511 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.511 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:35.511 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.372523+0000", 2024-09-21T14:58:35.511 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4100000000000001e-05 2024-09-21T14:58:35.511 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.511 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.511 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:35.511 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.373759+0000", 2024-09-21T14:58:35.511 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0012352400000000001 2024-09-21T14:58:35.512 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.512 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.512 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:35.512 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.373953+0000", 2024-09-21T14:58:35.512 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000193931 2024-09-21T14:58:35.512 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.512 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.512 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 3", 2024-09-21T14:58:35.512 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.374177+0000", 2024-09-21T14:58:35.512 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00022431299999999999 2024-09-21T14:58:35.512 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.512 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.512 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:35.512 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.376615+0000", 2024-09-21T14:58:35.512 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0024385880000000002 2024-09-21T14:58:35.513 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.513 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.513 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:35.513 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.377868+0000", 2024-09-21T14:58:35.513 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0012527829999999999 2024-09-21T14:58:35.513 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.513 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.513 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:35.513 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.377954+0000", 2024-09-21T14:58:35.513 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6125999999999995e-05 2024-09-21T14:58:35.513 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.513 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.514 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:35.514 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:34.378005+0000", 2024-09-21T14:58:35.514 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1184000000000003e-05 2024-09-21T14:58:35.514 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.514 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:35.514 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.514 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.514 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.514 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6440 3.16 3:68aaef8d:::rbd_data.11361bed1f78.000000000000000a:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:35.514 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.119780+0000", 2024-09-21T14:58:35.514 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.36595913400000002, 2024-09-21T14:58:35.515 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0048968730000000004, 2024-09-21T14:58:35.515 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:35.515 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:35.515 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:35.515 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:35.515 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:35.515 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6440 2024-09-21T14:58:35.515 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.515 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:35.515 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.515 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:35.515 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.119780+0000", 2024-09-21T14:58:35.515 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:35.516 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.516 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.516 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:35.516 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.119780+0000", 2024-09-21T14:58:35.516 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:35.516 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.516 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.516 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:35.516 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.119779+0000", 2024-09-21T14:58:35.516 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:35.516 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.517 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.517 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:35.517 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.119783+0000", 2024-09-21T14:58:35.517 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.428e-06 2024-09-21T14:58:35.517 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.517 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.517 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:35.517 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.119791+0000", 2024-09-21T14:58:35.517 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7740000000000004e-06 2024-09-21T14:58:35.517 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.517 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.517 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:35.517 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.119797+0000", 2024-09-21T14:58:35.518 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9370000000000001e-06 2024-09-21T14:58:35.518 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.518 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.518 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:35.518 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.119837+0000", 2024-09-21T14:58:35.518 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.0874999999999997e-05 2024-09-21T14:58:35.518 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.518 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.518 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting_for_map not empty", 2024-09-21T14:58:35.518 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.119842+0000", 2024-09-21T14:58:35.519 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0660000000000003e-06 2024-09-21T14:58:35.519 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.519 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.519 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:35.519 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.124407+0000", 2024-09-21T14:58:35.519 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0045644880000000002 2024-09-21T14:58:35.519 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.519 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.519 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:35.519 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.124511+0000", 2024-09-21T14:58:35.519 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000103763 2024-09-21T14:58:35.519 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.519 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.520 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:35.520 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.124677+0000", 2024-09-21T14:58:35.520 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00016598900000000001 2024-09-21T14:58:35.520 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.520 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:35.520 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.520 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.520 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.520 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6449 3.16 3:6eb18e6c:::rbd_data.11361bed1f78.0000000000000013:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:35.520 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.119822+0000", 2024-09-21T14:58:35.520 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.36591682399999997, 2024-09-21T14:58:35.521 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0052806579999999997, 2024-09-21T14:58:35.521 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:35.521 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:35.521 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:35.521 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:35.521 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:35.521 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6449 2024-09-21T14:58:35.521 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.521 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:35.521 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.521 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:35.522 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.119822+0000", 2024-09-21T14:58:35.522 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:35.522 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.522 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.522 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:35.522 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.119822+0000", 2024-09-21T14:58:35.522 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:35.522 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.522 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.522 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:35.522 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.119822+0000", 2024-09-21T14:58:35.522 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:35.523 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.523 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.523 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:35.523 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.119826+0000", 2024-09-21T14:58:35.523 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.0609999999999997e-06 2024-09-21T14:58:35.523 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.523 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.523 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:35.523 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.119835+0000", 2024-09-21T14:58:35.523 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0860000000000008e-06 2024-09-21T14:58:35.524 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.524 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.524 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:35.524 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.119842+0000", 2024-09-21T14:58:35.524 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3200000000000002e-06 2024-09-21T14:58:35.524 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.524 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.524 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:35.524 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.119884+0000", 2024-09-21T14:58:35.524 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1999999999999998e-05 2024-09-21T14:58:35.524 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.525 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.525 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting_for_map not empty", 2024-09-21T14:58:35.525 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.119889+0000", 2024-09-21T14:58:35.525 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6399999999999996e-06 2024-09-21T14:58:35.525 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.525 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.525 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:35.525 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.124726+0000", 2024-09-21T14:58:35.525 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0048374869999999997 2024-09-21T14:58:35.526 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.526 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.526 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:35.526 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.124909+0000", 2024-09-21T14:58:35.526 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00018336099999999999 2024-09-21T14:58:35.526 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.526 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.526 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:35.526 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.125103+0000", 2024-09-21T14:58:35.526 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00019335700000000001 2024-09-21T14:58:35.526 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.526 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:35.526 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.527 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.527 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.527 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6454 3.16 3:68aaef8d:::rbd_data.11361bed1f78.000000000000000a:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:35.527 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.119861+0000", 2024-09-21T14:58:35.527 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.36587757199999998, 2024-09-21T14:58:35.527 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00553616, 2024-09-21T14:58:35.527 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:35.527 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:35.527 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:35.527 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:35.527 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:35.527 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6454 2024-09-21T14:58:35.527 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.527 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:35.528 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.528 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:35.528 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.119861+0000", 2024-09-21T14:58:35.528 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:35.528 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.528 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.528 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:35.528 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.119861+0000", 2024-09-21T14:58:35.528 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:35.528 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.528 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.528 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:35.528 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.119861+0000", 2024-09-21T14:58:35.528 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:35.528 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.529 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.529 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:35.529 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.119864+0000", 2024-09-21T14:58:35.529 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4800000000000001e-06 2024-09-21T14:58:35.529 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.529 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.529 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:35.529 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.119874+0000", 2024-09-21T14:58:35.529 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5370000000000003e-06 2024-09-21T14:58:35.529 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.529 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.529 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:35.529 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.119880+0000", 2024-09-21T14:58:35.529 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5880000000000003e-06 2024-09-21T14:58:35.530 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.530 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.530 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:35.530 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.119905+0000", 2024-09-21T14:58:35.530 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4932000000000001e-05 2024-09-21T14:58:35.530 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.530 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.530 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting_for_map not empty", 2024-09-21T14:58:35.530 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.119908+0000", 2024-09-21T14:58:35.530 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5780000000000001e-06 2024-09-21T14:58:35.530 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.530 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.530 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:35.530 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.125155+0000", 2024-09-21T14:58:35.531 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0052474089999999998 2024-09-21T14:58:35.531 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.531 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.531 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:35.531 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.125284+0000", 2024-09-21T14:58:35.531 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012902500000000001 2024-09-21T14:58:35.531 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.531 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.531 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:35.531 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.125398+0000", 2024-09-21T14:58:35.531 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00011314799999999999 2024-09-21T14:58:35.532 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.532 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:35.532 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.532 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.532 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.532 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6463 3.16 3:6eb18e6c:::rbd_data.11361bed1f78.0000000000000013:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:35.532 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.119887+0000", 2024-09-21T14:58:35.532 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.36585192100000002, 2024-09-21T14:58:35.532 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0057856499999999998, 2024-09-21T14:58:35.532 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:35.532 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:35.532 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:35.532 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:35.532 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:35.533 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6463 2024-09-21T14:58:35.533 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.533 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:35.533 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.533 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:35.533 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.119887+0000", 2024-09-21T14:58:35.533 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:35.533 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.533 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.533 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:35.533 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.119887+0000", 2024-09-21T14:58:35.534 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:35.534 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.534 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.534 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:35.534 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.119887+0000", 2024-09-21T14:58:35.534 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:35.534 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.534 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.534 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:35.534 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.119890+0000", 2024-09-21T14:58:35.534 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0769999999999999e-06 2024-09-21T14:58:35.534 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.535 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.535 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:35.535 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.119897+0000", 2024-09-21T14:58:35.535 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7689999999999996e-06 2024-09-21T14:58:35.535 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.535 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.535 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:35.535 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.119905+0000", 2024-09-21T14:58:35.535 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4279999999999999e-06 2024-09-21T14:58:35.535 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.535 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.535 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:35.535 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.119939+0000", 2024-09-21T14:58:35.536 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3866999999999999e-05 2024-09-21T14:58:35.536 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.536 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.536 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting_for_map not empty", 2024-09-21T14:58:35.536 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.119943+0000", 2024-09-21T14:58:35.536 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5000000000000001e-06 2024-09-21T14:58:35.536 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.536 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.536 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:35.536 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.125437+0000", 2024-09-21T14:58:35.536 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0054933669999999999 2024-09-21T14:58:35.536 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.536 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.537 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:35.537 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.125557+0000", 2024-09-21T14:58:35.537 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012080599999999999 2024-09-21T14:58:35.537 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.537 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.537 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:35.537 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.125673+0000", 2024-09-21T14:58:35.537 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000115272 2024-09-21T14:58:35.537 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.537 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:35.537 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.537 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.537 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.537 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6468 3.16 3:68aaef8d:::rbd_data.11361bed1f78.000000000000000a:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:35.538 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.119941+0000", 2024-09-21T14:58:35.538 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.36579760500000003, 2024-09-21T14:58:35.538 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0060084200000000004, 2024-09-21T14:58:35.538 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:35.538 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:35.538 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:35.538 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:35.538 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:35.538 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6468 2024-09-21T14:58:35.538 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.538 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:35.538 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.538 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:35.538 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.119941+0000", 2024-09-21T14:58:35.539 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:35.539 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.539 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.539 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:35.539 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.119941+0000", 2024-09-21T14:58:35.539 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:35.539 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.539 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.539 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:35.539 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.119941+0000", 2024-09-21T14:58:35.539 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:58:35.539 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.539 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.539 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:35.540 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.119945+0000", 2024-09-21T14:58:35.540 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7999999999999998e-06 2024-09-21T14:58:35.540 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.540 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.540 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:35.540 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.119956+0000", 2024-09-21T14:58:35.540 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0064999999999999e-05 2024-09-21T14:58:35.540 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.540 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.540 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:35.540 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.119963+0000", 2024-09-21T14:58:35.540 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2729999999999997e-06 2024-09-21T14:58:35.540 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.540 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.540 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:35.541 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.119994+0000", 2024-09-21T14:58:35.541 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1721999999999999e-05 2024-09-21T14:58:35.541 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.541 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.541 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting_for_map not empty", 2024-09-21T14:58:35.541 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.119997+0000", 2024-09-21T14:58:35.541 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8509999999999999e-06 2024-09-21T14:58:35.541 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.541 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.541 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:35.541 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.125713+0000", 2024-09-21T14:58:35.541 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0057158529999999999 2024-09-21T14:58:35.541 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.541 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.542 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:35.542 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.125834+0000", 2024-09-21T14:58:35.542 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012065099999999999 2024-09-21T14:58:35.542 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.542 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.542 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:35.542 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.125950+0000", 2024-09-21T14:58:35.542 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000115942 2024-09-21T14:58:35.542 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.542 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:35.542 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.542 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.542 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.542 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6477 3.16 3:6eb18e6c:::rbd_data.11361bed1f78.0000000000000013:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:35.543 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.119977+0000", 2024-09-21T14:58:35.543 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.36576229900000001, 2024-09-21T14:58:35.543 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0062386309999999997, 2024-09-21T14:58:35.543 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:35.543 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:35.543 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:35.543 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:35.543 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:35.543 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6477 2024-09-21T14:58:35.543 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.543 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:35.543 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.543 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:35.543 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.119977+0000", 2024-09-21T14:58:35.544 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:35.544 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.544 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.544 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:35.544 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.119977+0000", 2024-09-21T14:58:35.544 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:35.544 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.544 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.544 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:35.544 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.119976+0000", 2024-09-21T14:58:35.544 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:35.544 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.544 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.544 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:35.545 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.119980+0000", 2024-09-21T14:58:35.545 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3610000000000001e-06 2024-09-21T14:58:35.545 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.545 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.545 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:35.545 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.119988+0000", 2024-09-21T14:58:35.545 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2630000000000003e-06 2024-09-21T14:58:35.545 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.545 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.545 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:35.545 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.119994+0000", 2024-09-21T14:58:35.545 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4540000000000004e-06 2024-09-21T14:58:35.545 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.546 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.546 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:35.546 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.120013+0000", 2024-09-21T14:58:35.546 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8859999999999999e-05 2024-09-21T14:58:35.546 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.546 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.546 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting_for_map not empty", 2024-09-21T14:58:35.546 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.120016+0000", 2024-09-21T14:58:35.546 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.683e-06 2024-09-21T14:58:35.546 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.546 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.546 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:35.546 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.125992+0000", 2024-09-21T14:58:35.547 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0059764579999999996 2024-09-21T14:58:35.547 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.547 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.547 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:35.547 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.126094+0000", 2024-09-21T14:58:35.547 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000101269 2024-09-21T14:58:35.547 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.547 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.547 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:35.547 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.126215+0000", 2024-09-21T14:58:35.547 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012178799999999999 2024-09-21T14:58:35.547 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.547 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:35.547 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.548 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.548 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.548 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6484 3.16 3:6eb18e6c:::rbd_data.11361bed1f78.0000000000000013:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:35.548 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.120002+0000", 2024-09-21T14:58:35.548 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.36573729900000002, 2024-09-21T14:58:35.548 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0065176710000000001, 2024-09-21T14:58:35.548 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:35.548 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:35.548 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:35.548 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:35.549 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:35.549 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6484 2024-09-21T14:58:35.549 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.549 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:35.549 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.549 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:35.549 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.120002+0000", 2024-09-21T14:58:35.549 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:35.549 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.549 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.549 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:35.549 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.120002+0000", 2024-09-21T14:58:35.549 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:35.549 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.550 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.550 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:35.550 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.120001+0000", 2024-09-21T14:58:35.550 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:35.550 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.550 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.550 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:35.550 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.120004+0000", 2024-09-21T14:58:35.550 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1580000000000001e-06 2024-09-21T14:58:35.550 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.550 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.551 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:35.551 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.120013+0000", 2024-09-21T14:58:35.551 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2560000000000002e-06 2024-09-21T14:58:35.551 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.551 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.551 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:35.551 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.120020+0000", 2024-09-21T14:58:35.551 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6029999999999999e-06 2024-09-21T14:58:35.551 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.551 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.551 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:35.551 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.120053+0000", 2024-09-21T14:58:35.551 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2524999999999998e-05 2024-09-21T14:58:35.552 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.552 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.552 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting_for_map not empty", 2024-09-21T14:58:35.552 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.120059+0000", 2024-09-21T14:58:35.552 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6869999999999997e-06 2024-09-21T14:58:35.552 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.552 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.552 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:35.552 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.126262+0000", 2024-09-21T14:58:35.552 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0062021940000000003 2024-09-21T14:58:35.553 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.553 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.553 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:35.553 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.126391+0000", 2024-09-21T14:58:35.553 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000129011 2024-09-21T14:58:35.553 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.553 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.553 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:35.553 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.126519+0000", 2024-09-21T14:58:35.553 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012872 2024-09-21T14:58:35.553 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.553 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:35.553 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.554 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.554 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.554 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6498 3.16 3:6eb18e6c:::rbd_data.11361bed1f78.0000000000000013:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:35.554 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.120036+0000", 2024-09-21T14:58:35.554 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.36570338299999999, 2024-09-21T14:58:35.554 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0067525479999999997, 2024-09-21T14:58:35.554 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:35.554 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:35.554 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:35.554 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:35.554 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:35.555 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6498 2024-09-21T14:58:35.555 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.555 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:35.555 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.555 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:35.555 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.120036+0000", 2024-09-21T14:58:35.555 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:35.555 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.555 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.555 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:35.556 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.120036+0000", 2024-09-21T14:58:35.556 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:35.556 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.556 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.556 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:35.556 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.120035+0000", 2024-09-21T14:58:35.556 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:35.556 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.556 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.556 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:35.556 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.120039+0000", 2024-09-21T14:58:35.556 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.8569999999999997e-06 2024-09-21T14:58:35.557 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.557 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.557 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:35.557 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.120047+0000", 2024-09-21T14:58:35.557 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2199999999999992e-06 2024-09-21T14:58:35.557 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.557 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.557 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:35.557 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.120060+0000", 2024-09-21T14:58:35.557 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3314e-05 2024-09-21T14:58:35.557 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.557 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.558 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:35.558 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.120086+0000", 2024-09-21T14:58:35.558 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5323000000000001e-05 2024-09-21T14:58:35.558 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.558 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.558 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting_for_map not empty", 2024-09-21T14:58:35.558 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.120088+0000", 2024-09-21T14:58:35.558 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6749999999999998e-06 2024-09-21T14:58:35.558 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.558 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.558 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:35.558 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.126582+0000", 2024-09-21T14:58:35.558 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0064932779999999999 2024-09-21T14:58:35.559 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.559 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.559 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:35.559 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.126692+0000", 2024-09-21T14:58:35.559 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00011026 2024-09-21T14:58:35.559 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.559 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.559 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:35.559 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.126788+0000", 2024-09-21T14:58:35.559 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6201999999999998e-05 2024-09-21T14:58:35.560 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.560 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:35.560 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.560 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.560 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.560 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6701 3.16 3:6eb18e6c:::rbd_data.11361bed1f78.0000000000000013:d [read 32768~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:35.560 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.133561+0000", 2024-09-21T14:58:35.560 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.35217781199999998, 2024-09-21T14:58:35.560 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0048288979999999999, 2024-09-21T14:58:35.560 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:35.560 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:35.561 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:35.561 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:35.561 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:35.561 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6701 2024-09-21T14:58:35.561 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.561 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:35.561 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.561 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:35.561 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133561+0000", 2024-09-21T14:58:35.562 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:35.562 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.562 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.562 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:35.562 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133561+0000", 2024-09-21T14:58:35.562 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:35.562 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.562 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.562 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:35.562 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133560+0000", 2024-09-21T14:58:35.562 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:58:35.562 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.562 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.562 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:35.563 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133567+0000", 2024-09-21T14:58:35.563 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.401e-06 2024-09-21T14:58:35.563 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.563 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.563 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:35.563 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133580+0000", 2024-09-21T14:58:35.563 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2850000000000001e-05 2024-09-21T14:58:35.563 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.563 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.563 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:35.563 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133590+0000", 2024-09-21T14:58:35.563 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0193e-05 2024-09-21T14:58:35.563 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.563 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.564 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:35.564 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.136259+0000", 2024-09-21T14:58:35.564 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0026689779999999998 2024-09-21T14:58:35.564 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.564 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.564 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:35.564 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.136335+0000", 2024-09-21T14:58:35.564 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5388000000000004e-05 2024-09-21T14:58:35.564 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.564 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.564 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:35.564 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.138390+0000", 2024-09-21T14:58:35.564 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0020552330000000001 2024-09-21T14:58:35.564 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.565 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:35.565 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.565 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.565 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.565 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6712 3.16 3:68aaef8d:::rbd_data.11361bed1f78.000000000000000a:d [read 98304~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:35.565 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.133599+0000", 2024-09-21T14:58:35.565 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.35214019699999999, 2024-09-21T14:58:35.565 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0052813549999999997, 2024-09-21T14:58:35.565 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:35.565 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:35.565 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:35.565 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:35.565 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:35.566 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6712 2024-09-21T14:58:35.566 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.566 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:35.566 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.566 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:35.566 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133599+0000", 2024-09-21T14:58:35.566 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:35.566 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.566 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.566 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:35.566 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133599+0000", 2024-09-21T14:58:35.566 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:35.566 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.566 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.567 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:35.567 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133598+0000", 2024-09-21T14:58:35.567 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:35.567 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.567 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.567 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:35.567 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133602+0000", 2024-09-21T14:58:35.567 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.9890000000000003e-06 2024-09-21T14:58:35.567 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.567 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.567 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:35.567 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133610+0000", 2024-09-21T14:58:35.567 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6760000000000007e-06 2024-09-21T14:58:35.567 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.568 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.568 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:35.568 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133615+0000", 2024-09-21T14:58:35.568 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5829999999999997e-06 2024-09-21T14:58:35.568 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.568 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.568 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:35.568 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.138418+0000", 2024-09-21T14:58:35.568 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0048024720000000003 2024-09-21T14:58:35.568 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.568 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.568 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:35.568 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.138505+0000", 2024-09-21T14:58:35.568 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.7285000000000002e-05 2024-09-21T14:58:35.568 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.569 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.569 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:35.569 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.138880+0000", 2024-09-21T14:58:35.569 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00037490499999999998 2024-09-21T14:58:35.569 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.569 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:35.569 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.569 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.569 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.569 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6713 3.16 3:68aaef8d:::rbd_data.11361bed1f78.000000000000000a:d [read 163840~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:35.569 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.133621+0000", 2024-09-21T14:58:35.569 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.35211771200000003, 2024-09-21T14:58:35.569 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0054748490000000004, 2024-09-21T14:58:35.569 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:35.570 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:35.570 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:35.570 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:35.570 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:35.570 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6713 2024-09-21T14:58:35.570 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.570 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:35.570 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.570 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:35.570 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133621+0000", 2024-09-21T14:58:35.570 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:35.570 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.570 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.571 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:35.571 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133621+0000", 2024-09-21T14:58:35.571 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:35.571 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.571 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.571 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:35.571 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133621+0000", 2024-09-21T14:58:35.571 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:35.571 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.571 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.571 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:35.571 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133624+0000", 2024-09-21T14:58:35.571 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5530000000000001e-06 2024-09-21T14:58:35.571 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.571 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.572 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:35.572 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133631+0000", 2024-09-21T14:58:35.572 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3570000000000003e-06 2024-09-21T14:58:35.572 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.572 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.572 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:35.572 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133636+0000", 2024-09-21T14:58:35.572 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3290000000000001e-06 2024-09-21T14:58:35.572 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.572 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.572 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:35.572 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.138908+0000", 2024-09-21T14:58:35.572 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0052724410000000001 2024-09-21T14:58:35.572 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.573 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.573 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:35.573 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.139004+0000", 2024-09-21T14:58:35.573 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5758e-05 2024-09-21T14:58:35.573 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.573 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.573 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:35.573 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.139096+0000", 2024-09-21T14:58:35.573 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1885000000000005e-05 2024-09-21T14:58:35.573 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.573 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:35.573 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.573 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.573 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.573 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6714 3.16 3:68aaef8d:::rbd_data.11361bed1f78.000000000000000a:d [read 245760~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:35.574 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.133641+0000", 2024-09-21T14:58:35.574 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.35209770400000001, 2024-09-21T14:58:35.574 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0059002120000000002, 2024-09-21T14:58:35.574 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:35.574 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:35.574 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:35.574 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:35.574 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:35.574 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6714 2024-09-21T14:58:35.574 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.574 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:35.574 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.574 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:35.575 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133641+0000", 2024-09-21T14:58:35.575 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:35.575 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.575 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.575 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:35.575 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133641+0000", 2024-09-21T14:58:35.575 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:35.575 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.575 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.575 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:35.575 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133641+0000", 2024-09-21T14:58:35.575 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:35.575 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.575 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.575 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:35.576 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133644+0000", 2024-09-21T14:58:35.576 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9069999999999999e-06 2024-09-21T14:58:35.576 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.576 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.576 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:35.576 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133649+0000", 2024-09-21T14:58:35.576 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6130000000000003e-06 2024-09-21T14:58:35.576 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.576 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.576 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:35.576 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133655+0000", 2024-09-21T14:58:35.576 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.083e-06 2024-09-21T14:58:35.576 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.576 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.577 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:35.577 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.139122+0000", 2024-09-21T14:58:35.577 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.005466767 2024-09-21T14:58:35.577 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.577 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.577 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:35.577 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.139193+0000", 2024-09-21T14:58:35.577 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0448e-05 2024-09-21T14:58:35.577 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.577 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.577 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:35.577 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.139541+0000", 2024-09-21T14:58:35.577 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00034885500000000002 2024-09-21T14:58:35.577 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.578 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:35.578 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.578 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.578 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.578 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6729 3.16 3:6eb18e6c:::rbd_data.11361bed1f78.0000000000000013:d [read 49152~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:35.578 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.133660+0000", 2024-09-21T14:58:35.578 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.35207928900000002, 2024-09-21T14:58:35.578 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0062381800000000003, 2024-09-21T14:58:35.578 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:35.578 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:35.578 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:35.578 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:35.578 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:35.578 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6729 2024-09-21T14:58:35.579 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.579 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:35.579 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.579 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:35.579 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133660+0000", 2024-09-21T14:58:35.579 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:35.579 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.579 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.579 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:35.579 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133660+0000", 2024-09-21T14:58:35.579 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:35.579 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.579 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.579 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:35.580 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133659+0000", 2024-09-21T14:58:35.580 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:35.580 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.580 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.580 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:35.580 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133662+0000", 2024-09-21T14:58:35.580 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.463e-06 2024-09-21T14:58:35.580 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.580 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.580 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:35.580 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133667+0000", 2024-09-21T14:58:35.580 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.344e-06 2024-09-21T14:58:35.580 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.581 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.581 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:35.581 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133672+0000", 2024-09-21T14:58:35.581 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6280000000000004e-06 2024-09-21T14:58:35.581 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.581 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.581 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:35.581 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.139570+0000", 2024-09-21T14:58:35.581 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.005898225 2024-09-21T14:58:35.581 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.581 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.581 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:35.581 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.139651+0000", 2024-09-21T14:58:35.581 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1342999999999997e-05 2024-09-21T14:58:35.582 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.582 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.582 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:35.582 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.139898+0000", 2024-09-21T14:58:35.582 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00024648500000000001 2024-09-21T14:58:35.582 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.582 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:35.582 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.582 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.582 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.582 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6746 3.16 3:6eb18e6c:::rbd_data.11361bed1f78.0000000000000013:d [read 65536~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:35.582 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.133676+0000", 2024-09-21T14:58:35.582 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.35206321899999998, 2024-09-21T14:58:35.582 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0065617979999999998, 2024-09-21T14:58:35.583 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:35.583 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:35.583 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:35.583 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:35.583 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:35.583 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6746 2024-09-21T14:58:35.583 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.583 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:35.584 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.584 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:35.584 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133676+0000", 2024-09-21T14:58:35.584 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:35.584 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.584 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.584 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:35.584 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133676+0000", 2024-09-21T14:58:35.584 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:35.584 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.584 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.584 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:35.584 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133675+0000", 2024-09-21T14:58:35.584 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:35.585 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.585 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.585 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:35.585 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133678+0000", 2024-09-21T14:58:35.585 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3149999999999999e-06 2024-09-21T14:58:35.585 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.585 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.585 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:35.585 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133683+0000", 2024-09-21T14:58:35.585 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4439999999999999e-06 2024-09-21T14:58:35.585 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.585 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.586 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:35.586 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133687+0000", 2024-09-21T14:58:35.586 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1649999999999999e-06 2024-09-21T14:58:35.586 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.586 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.586 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:35.586 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.139932+0000", 2024-09-21T14:58:35.586 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0062448920000000002 2024-09-21T14:58:35.586 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.586 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.586 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:35.586 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.140009+0000", 2024-09-21T14:58:35.587 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6734000000000001e-05 2024-09-21T14:58:35.587 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.587 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.587 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:35.587 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.140238+0000", 2024-09-21T14:58:35.587 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00022854599999999999 2024-09-21T14:58:35.587 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.587 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:35.587 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.587 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.588 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.588 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6747 3.16 3:6eb18e6c:::rbd_data.11361bed1f78.0000000000000013:d [read 147456~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:35.588 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.133691+0000", 2024-09-21T14:58:35.588 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.35204762099999998, 2024-09-21T14:58:35.588 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.006764936, 2024-09-21T14:58:35.588 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:35.588 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:35.588 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:35.588 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:35.588 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:35.588 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6747 2024-09-21T14:58:35.588 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.588 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:35.589 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.589 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:35.589 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133691+0000", 2024-09-21T14:58:35.589 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:35.589 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.589 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.589 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:35.589 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133691+0000", 2024-09-21T14:58:35.589 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:35.589 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.589 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.590 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:35.590 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133691+0000", 2024-09-21T14:58:35.590 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:35.590 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.590 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.590 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:35.590 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133693+0000", 2024-09-21T14:58:35.590 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2390000000000001e-06 2024-09-21T14:58:35.590 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.590 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.590 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:35.590 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133699+0000", 2024-09-21T14:58:35.591 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2519999999999999e-06 2024-09-21T14:58:35.591 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.591 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.591 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:35.591 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133703+0000", 2024-09-21T14:58:35.591 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3660000000000002e-06 2024-09-21T14:58:35.591 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.591 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.591 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:35.591 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.140270+0000", 2024-09-21T14:58:35.592 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0065673499999999996 2024-09-21T14:58:35.592 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.592 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.592 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:35.592 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.140351+0000", 2024-09-21T14:58:35.592 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0975000000000003e-05 2024-09-21T14:58:35.592 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.592 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.592 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:35.593 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.140456+0000", 2024-09-21T14:58:35.593 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000105086 2024-09-21T14:58:35.593 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.593 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:35.593 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.593 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.593 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.593 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6748 3.16 3:6eb18e6c:::rbd_data.11361bed1f78.0000000000000013:d [read 212992~32768 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:35.593 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.133707+0000", 2024-09-21T14:58:35.594 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.35203209200000002, 2024-09-21T14:58:35.594 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0071255779999999996, 2024-09-21T14:58:35.594 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:35.594 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:35.594 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:35.594 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:35.594 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:35.594 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6748 2024-09-21T14:58:35.594 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.594 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:35.594 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.594 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:35.594 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133707+0000", 2024-09-21T14:58:35.595 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:35.595 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.595 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.595 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:35.595 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133707+0000", 2024-09-21T14:58:35.595 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:35.595 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.595 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.595 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:35.595 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133707+0000", 2024-09-21T14:58:35.595 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:35.595 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.595 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.595 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:35.596 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133709+0000", 2024-09-21T14:58:35.596 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2419999999999999e-06 2024-09-21T14:58:35.596 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.596 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.596 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:35.596 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133714+0000", 2024-09-21T14:58:35.596 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.361e-06 2024-09-21T14:58:35.596 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.596 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.596 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:35.596 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133718+0000", 2024-09-21T14:58:35.596 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2549999999999999e-06 2024-09-21T14:58:35.596 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.596 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.597 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:35.597 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.140541+0000", 2024-09-21T14:58:35.597 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0068226040000000003 2024-09-21T14:58:35.597 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.597 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.597 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:35.597 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.140634+0000", 2024-09-21T14:58:35.597 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2748e-05 2024-09-21T14:58:35.597 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.597 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.597 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:35.597 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.140832+0000", 2024-09-21T14:58:35.597 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00019867299999999999 2024-09-21T14:58:35.597 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.598 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:35.598 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.598 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.598 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:35.598 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:58:35.599 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T14:58:35.600 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:58:35.595+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:58:35.700 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:58:35.696+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:58:35.709 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:58:35.709 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:58:35.710 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T14:58:35.710 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:58:35.719 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T14:58:35.801 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:58:35.795+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:58:35.828 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:58:35.828 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:58:35.828 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T14:58:35.829 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T14:58:35.829 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:58:35.838 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T14:58:35.902 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:58:35.900+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:58:35.950 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:58:35.950 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T14:58:35.950 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T14:58:35.950 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T14:58:35.950 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.951 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4281.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-21T14:58:35.951 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:09.324416+0000", 2024-09-21T14:58:35.951 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 26.621266890000001, 2024-09-21T14:58:35.951 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.117837947, 2024-09-21T14:58:35.951 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:35.951 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:35.951 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:35.951 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4281", 2024-09-21T14:58:35.951 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/1518520303", 2024-09-21T14:58:35.951 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1 2024-09-21T14:58:35.951 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.951 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:35.951 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.952 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:35.952 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324416+0000", 2024-09-21T14:58:35.952 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:35.952 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.952 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.952 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:35.952 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324416+0000", 2024-09-21T14:58:35.952 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:35.952 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.952 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.952 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:35.952 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324414+0000", 2024-09-21T14:58:35.952 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T14:58:35.952 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.953 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.953 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:35.953 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324429+0000", 2024-09-21T14:58:35.953 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5637e-05 2024-09-21T14:58:35.953 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.953 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.953 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:35.953 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324448+0000", 2024-09-21T14:58:35.953 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9042000000000002e-05 2024-09-21T14:58:35.953 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.953 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.953 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:35.953 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324465+0000", 2024-09-21T14:58:35.953 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6767999999999999e-05 2024-09-21T14:58:35.954 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.954 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.954 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:35.954 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324571+0000", 2024-09-21T14:58:35.954 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000105714 2024-09-21T14:58:35.954 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.954 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.954 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op must wait for map", 2024-09-21T14:58:35.954 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324587+0000", 2024-09-21T14:58:35.954 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5753999999999999e-05 2024-09-21T14:58:35.954 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.954 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.954 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:35.955 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.325864+0000", 2024-09-21T14:58:35.955 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0012773350000000001 2024-09-21T14:58:35.955 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.955 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.955 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T14:58:35.955 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.325878+0000", 2024-09-21T14:58:35.955 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4498000000000001e-05 2024-09-21T14:58:35.955 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.955 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.955 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:35.955 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.439461+0000", 2024-09-21T14:58:35.955 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.11358212500000001 2024-09-21T14:58:35.955 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.955 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.956 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:35.956 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.439848+0000", 2024-09-21T14:58:35.956 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00038747800000000002 2024-09-21T14:58:35.956 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.956 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.956 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:58:35.956 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.440175+0000", 2024-09-21T14:58:35.956 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00032721900000000001 2024-09-21T14:58:35.956 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.956 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.956 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:35.956 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.441770+0000", 2024-09-21T14:58:35.956 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0015949409999999999 2024-09-21T14:58:35.957 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.957 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.957 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:35.957 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.442086+0000", 2024-09-21T14:58:35.957 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00031586299999999999 2024-09-21T14:58:35.957 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.957 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.957 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:35.957 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.442195+0000", 2024-09-21T14:58:35.957 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000108791 2024-09-21T14:58:35.957 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.957 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.957 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:35.957 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.442254+0000", 2024-09-21T14:58:35.958 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9038e-05 2024-09-21T14:58:35.958 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.958 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:35.958 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.958 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.958 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.958 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4281.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-21T14:58:35.958 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:09.447828+0000", 2024-09-21T14:58:35.958 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 26.497854888999999, 2024-09-21T14:58:35.958 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.32904393700000001, 2024-09-21T14:58:35.958 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:35.958 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:58:35.959 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:35.959 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4281", 2024-09-21T14:58:35.959 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/1518520303", 2024-09-21T14:58:35.959 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 2 2024-09-21T14:58:35.959 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.959 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:35.959 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.959 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:35.959 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447828+0000", 2024-09-21T14:58:35.959 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:35.959 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.959 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.959 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:35.959 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447828+0000", 2024-09-21T14:58:35.960 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:35.960 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.960 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.960 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:35.960 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447825+0000", 2024-09-21T14:58:35.960 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T14:58:35.960 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.960 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.960 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:35.960 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447841+0000", 2024-09-21T14:58:35.960 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5359999999999999e-05 2024-09-21T14:58:35.960 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.960 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.960 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:35.961 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447875+0000", 2024-09-21T14:58:35.961 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4388999999999999e-05 2024-09-21T14:58:35.961 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.961 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.961 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:35.961 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447903+0000", 2024-09-21T14:58:35.961 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7373e-05 2024-09-21T14:58:35.961 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.961 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.961 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:35.961 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447996+0000", 2024-09-21T14:58:35.961 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2954999999999993e-05 2024-09-21T14:58:35.961 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.961 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.962 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T14:58:35.962 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.448011+0000", 2024-09-21T14:58:35.962 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5038999999999999e-05 2024-09-21T14:58:35.962 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.962 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.962 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:35.962 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.776501+0000", 2024-09-21T14:58:35.962 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.32849012500000002 2024-09-21T14:58:35.962 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.963 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.963 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:35.963 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.776872+0000", 2024-09-21T14:58:35.963 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00037119600000000002 2024-09-21T14:58:35.963 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.963 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:35.963 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.963 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.963 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.963 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:752 3.14 3:28fc02e5:::rbd_data.113693316f36.0000000000000010:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-21T14:58:35.963 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:26.625579+0000", 2024-09-21T14:58:35.963 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 9.3201040339999999, 2024-09-21T14:58:35.963 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042965426000000001, 2024-09-21T14:58:35.963 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:35.964 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:35.964 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:35.964 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:35.964 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:35.964 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 752 2024-09-21T14:58:35.964 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.964 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:35.964 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.964 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:35.964 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625579+0000", 2024-09-21T14:58:35.964 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:35.965 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.965 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.965 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:35.965 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625579+0000", 2024-09-21T14:58:35.965 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:35.965 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.965 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.965 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:35.965 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625577+0000", 2024-09-21T14:58:35.965 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T14:58:35.965 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.965 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.966 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:35.966 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625588+0000", 2024-09-21T14:58:35.966 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1555e-05 2024-09-21T14:58:35.966 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.966 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.966 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:35.966 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625610+0000", 2024-09-21T14:58:35.966 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2107999999999999e-05 2024-09-21T14:58:35.966 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.966 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.966 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:35.967 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625621+0000", 2024-09-21T14:58:35.967 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0762e-05 2024-09-21T14:58:35.967 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.967 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.967 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:35.967 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625692+0000", 2024-09-21T14:58:35.967 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.1203000000000003e-05 2024-09-21T14:58:35.967 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.967 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.967 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:35.967 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625872+0000", 2024-09-21T14:58:35.968 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017962800000000001 2024-09-21T14:58:35.968 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.968 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.968 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:58:35.968 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.633188+0000", 2024-09-21T14:58:35.968 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.007316014 2024-09-21T14:58:35.968 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.968 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.968 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:35.968 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.668357+0000", 2024-09-21T14:58:35.968 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035168933999999999 2024-09-21T14:58:35.968 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.969 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.969 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:35.969 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.668418+0000", 2024-09-21T14:58:35.969 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1282e-05 2024-09-21T14:58:35.969 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.969 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.969 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:35.969 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.668486+0000", 2024-09-21T14:58:35.969 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8468999999999998e-05 2024-09-21T14:58:35.969 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.969 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.970 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:35.970 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.668544+0000", 2024-09-21T14:58:35.970 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7785000000000003e-05 2024-09-21T14:58:35.970 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.970 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:35.970 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.970 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.970 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.970 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop_reply(client.4406.0:752 3.14 e32/24 ondisk, result = 0)", 2024-09-21T14:58:35.970 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:26.653670+0000", 2024-09-21T14:58:35.971 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 9.2920127879999992, 2024-09-21T14:58:35.971 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014884797, 2024-09-21T14:58:35.971 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:35.971 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:35.971 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:35.971 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.971 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:35.971 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.653670+0000", 2024-09-21T14:58:35.971 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:35.971 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.972 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.972 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:35.972 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.653670+0000", 2024-09-21T14:58:35.972 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:35.972 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.972 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.972 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:35.972 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.653669+0000", 2024-09-21T14:58:35.972 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:58:35.972 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.972 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.973 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:35.973 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.653676+0000", 2024-09-21T14:58:35.973 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.1629999999999999e-06 2024-09-21T14:58:35.973 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.973 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.973 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:35.973 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.653695+0000", 2024-09-21T14:58:35.973 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8760999999999999e-05 2024-09-21T14:58:35.974 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.974 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.974 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:35.974 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.653704+0000", 2024-09-21T14:58:35.974 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6139999999999998e-06 2024-09-21T14:58:35.974 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.974 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.974 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:35.974 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.668393+0000", 2024-09-21T14:58:35.975 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014688728 2024-09-21T14:58:35.975 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.975 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.975 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:35.975 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.668410+0000", 2024-09-21T14:58:35.975 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6858e-05 2024-09-21T14:58:35.975 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.975 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.975 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:35.975 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.668555+0000", 2024-09-21T14:58:35.975 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014483700000000001 2024-09-21T14:58:35.975 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.976 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:35.976 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.976 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.976 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.976 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6504 3.d 3:b2f690c6:::rbd_data.11361bed1f78.0000000000000019:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:35.976 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.121932+0000", 2024-09-21T14:58:35.976 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.82375036599999996, 2024-09-21T14:58:35.976 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0097717470000000008, 2024-09-21T14:58:35.976 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:35.976 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:35.976 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:35.976 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:35.976 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:35.976 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6504 2024-09-21T14:58:35.976 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.977 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:35.977 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.977 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:35.977 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.121932+0000", 2024-09-21T14:58:35.977 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:35.977 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.977 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.977 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:35.977 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.121932+0000", 2024-09-21T14:58:35.977 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:35.977 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.977 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.977 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:35.977 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.121932+0000", 2024-09-21T14:58:35.977 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:35.978 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.978 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.978 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:35.978 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.121934+0000", 2024-09-21T14:58:35.978 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3190000000000002e-06 2024-09-21T14:58:35.978 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.978 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.978 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:35.978 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.121939+0000", 2024-09-21T14:58:35.978 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3159999999999998e-06 2024-09-21T14:58:35.978 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.978 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.978 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:35.979 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.121950+0000", 2024-09-21T14:58:35.979 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1476e-05 2024-09-21T14:58:35.979 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.979 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.979 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:35.979 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.122788+0000", 2024-09-21T14:58:35.979 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00083814499999999999 2024-09-21T14:58:35.979 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.979 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.979 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting_for_map not empty", 2024-09-21T14:58:35.979 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.122793+0000", 2024-09-21T14:58:35.979 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2729999999999996e-06 2024-09-21T14:58:35.979 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.980 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.980 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:35.980 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131544+0000", 2024-09-21T14:58:35.980 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.008751768 2024-09-21T14:58:35.980 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.980 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.980 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:35.980 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131629+0000", 2024-09-21T14:58:35.980 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4536000000000005e-05 2024-09-21T14:58:35.980 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.980 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.980 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:35.980 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131704+0000", 2024-09-21T14:58:35.980 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5346999999999995e-05 2024-09-21T14:58:35.981 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.981 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:35.981 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.981 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.981 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.981 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6682 3.1d 3:b8f5ac1a:::rbd_data.11361bed1f78.0000000000000009:d [read 147456~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:35.981 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.134499+0000", 2024-09-21T14:58:35.981 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.81118348200000001, 2024-09-21T14:58:35.981 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0094333049999999995, 2024-09-21T14:58:35.981 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:35.981 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:35.981 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:35.981 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:35.981 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:35.982 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6682 2024-09-21T14:58:35.982 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.982 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:35.982 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.982 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:35.982 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134499+0000", 2024-09-21T14:58:35.982 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:35.982 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.982 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.982 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:35.982 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134499+0000", 2024-09-21T14:58:35.982 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:35.982 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.982 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.983 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:35.983 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134499+0000", 2024-09-21T14:58:35.983 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:35.983 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.983 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.983 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:35.983 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134502+0000", 2024-09-21T14:58:35.983 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8399999999999999e-06 2024-09-21T14:58:35.983 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.983 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.983 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:35.983 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134509+0000", 2024-09-21T14:58:35.983 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6539999999999999e-06 2024-09-21T14:58:35.983 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.984 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.984 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:35.984 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134516+0000", 2024-09-21T14:58:35.984 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2689999999999996e-06 2024-09-21T14:58:35.984 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.984 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.984 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:35.984 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.143777+0000", 2024-09-21T14:58:35.984 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0092615270000000003 2024-09-21T14:58:35.984 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.984 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.984 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:35.984 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.143849+0000", 2024-09-21T14:58:35.984 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2138000000000006e-05 2024-09-21T14:58:35.985 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.985 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.985 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:35.985 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.143933+0000", 2024-09-21T14:58:35.985 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3354000000000002e-05 2024-09-21T14:58:35.985 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.985 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:35.985 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.985 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.985 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.985 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6683 3.1d 3:b8f5ac1a:::rbd_data.11361bed1f78.0000000000000009:d [read 229376~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:35.985 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.134520+0000", 2024-09-21T14:58:35.985 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.81116263700000002, 2024-09-21T14:58:35.985 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0097861619999999993, 2024-09-21T14:58:35.986 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:35.986 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:35.986 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:35.986 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:35.986 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:35.986 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6683 2024-09-21T14:58:35.986 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.986 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:35.986 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.986 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:35.986 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134520+0000", 2024-09-21T14:58:35.986 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:35.986 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.987 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.987 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:35.987 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134520+0000", 2024-09-21T14:58:35.987 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:35.987 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.987 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.987 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:35.987 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134520+0000", 2024-09-21T14:58:35.987 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:35.987 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.987 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.987 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:35.987 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134522+0000", 2024-09-21T14:58:35.987 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4059999999999999e-06 2024-09-21T14:58:35.987 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.988 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.988 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:35.988 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134529+0000", 2024-09-21T14:58:35.988 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0890000000000002e-06 2024-09-21T14:58:35.988 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.988 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.988 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:35.988 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134534+0000", 2024-09-21T14:58:35.988 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.617e-06 2024-09-21T14:58:35.988 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.988 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.988 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:35.988 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.143957+0000", 2024-09-21T14:58:35.989 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0094232459999999997 2024-09-21T14:58:35.989 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.989 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.989 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:35.989 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.144046+0000", 2024-09-21T14:58:35.989 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8690000000000006e-05 2024-09-21T14:58:35.989 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.989 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.989 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:35.989 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.144306+0000", 2024-09-21T14:58:35.989 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00026042300000000002 2024-09-21T14:58:35.989 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.989 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:35.989 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.989 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.990 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.990 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6698 3.1d 3:b9c0cfe7:::rbd_data.11361bed1f78.0000000000000010:d [read 32768~32768 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:35.990 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.134539+0000", 2024-09-21T14:58:35.990 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.81114356200000004, 2024-09-21T14:58:35.990 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.010677103, 2024-09-21T14:58:35.990 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:35.990 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:35.990 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:35.990 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:35.990 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:35.990 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6698 2024-09-21T14:58:35.990 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.990 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:35.991 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.991 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:35.991 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134539+0000", 2024-09-21T14:58:35.991 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:35.991 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.991 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.991 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:35.991 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134539+0000", 2024-09-21T14:58:35.991 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:35.991 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.991 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.991 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:35.991 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134539+0000", 2024-09-21T14:58:35.991 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:35.992 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.992 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.992 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:35.992 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134541+0000", 2024-09-21T14:58:35.992 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3209999999999999e-06 2024-09-21T14:58:35.992 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.992 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.992 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:35.992 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134547+0000", 2024-09-21T14:58:35.992 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8799999999999996e-06 2024-09-21T14:58:35.992 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.992 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.992 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:35.992 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134557+0000", 2024-09-21T14:58:35.993 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0045e-05 2024-09-21T14:58:35.993 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.993 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.993 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:35.993 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.144524+0000", 2024-09-21T14:58:35.993 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0099663229999999992 2024-09-21T14:58:35.993 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.993 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.993 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:35.993 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.144625+0000", 2024-09-21T14:58:35.993 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000101376 2024-09-21T14:58:35.993 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.993 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.993 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:35.993 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.145216+0000", 2024-09-21T14:58:35.994 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00059145599999999997 2024-09-21T14:58:35.994 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.994 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:35.994 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.994 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.994 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.994 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6706 3.d 3:b2fe5261:::rbd_data.11361bed1f78.0000000000000008:d [read 98304~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:35.994 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.134652+0000", 2024-09-21T14:58:35.994 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.81103087100000004, 2024-09-21T14:58:35.995 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.010156419999999999, 2024-09-21T14:58:35.995 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:35.995 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:35.995 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:35.995 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:35.995 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:35.995 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6706 2024-09-21T14:58:35.995 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.995 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:35.995 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.995 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:35.995 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134652+0000", 2024-09-21T14:58:35.995 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:35.996 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.996 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.996 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:35.996 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134652+0000", 2024-09-21T14:58:35.996 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:35.996 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.996 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.996 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:35.996 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134652+0000", 2024-09-21T14:58:35.996 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:35.996 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.997 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.997 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:35.997 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134654+0000", 2024-09-21T14:58:35.997 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2199999999999999e-06 2024-09-21T14:58:35.997 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.997 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.997 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:35.997 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134660+0000", 2024-09-21T14:58:35.997 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7450000000000001e-06 2024-09-21T14:58:35.997 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.998 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.998 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:35.998 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134664+0000", 2024-09-21T14:58:35.998 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.549e-06 2024-09-21T14:58:35.998 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.998 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.998 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:35.998 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.144334+0000", 2024-09-21T14:58:35.998 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0096699009999999998 2024-09-21T14:58:35.998 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.998 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.998 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:35.998 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.144417+0000", 2024-09-21T14:58:35.999 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3088000000000001e-05 2024-09-21T14:58:35.999 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.999 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:35.999 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:35.999 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.144808+0000", 2024-09-21T14:58:35.999 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00039123499999999998 2024-09-21T14:58:35.999 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.999 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:35.999 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:35.999 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:35.999 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.000 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6707 3.d 3:b2fe5261:::rbd_data.11361bed1f78.0000000000000008:d [read 180224~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:36.000 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.134669+0000", 2024-09-21T14:58:36.000 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.81101404399999999, 2024-09-21T14:58:36.000 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.010282932, 2024-09-21T14:58:36.000 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:36.000 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:36.000 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:36.000 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:36.000 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:36.000 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6707 2024-09-21T14:58:36.000 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.000 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:36.001 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.001 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:36.001 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134669+0000", 2024-09-21T14:58:36.001 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:36.001 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.001 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.001 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:36.001 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134669+0000", 2024-09-21T14:58:36.001 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:36.001 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.001 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.002 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:36.002 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134668+0000", 2024-09-21T14:58:36.002 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:36.002 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.002 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.002 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:36.002 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134675+0000", 2024-09-21T14:58:36.002 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4239999999999998e-06 2024-09-21T14:58:36.002 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.002 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.002 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:36.002 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134681+0000", 2024-09-21T14:58:36.003 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8100000000000003e-06 2024-09-21T14:58:36.003 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.003 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.003 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:36.003 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134686+0000", 2024-09-21T14:58:36.003 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2349999999999999e-06 2024-09-21T14:58:36.003 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.003 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.003 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:36.003 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.144832+0000", 2024-09-21T14:58:36.003 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.010146159 2024-09-21T14:58:36.004 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.004 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.004 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:36.004 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.144886+0000", 2024-09-21T14:58:36.004 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3739999999999997e-05 2024-09-21T14:58:36.004 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.004 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.004 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:36.004 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.144952+0000", 2024-09-21T14:58:36.005 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6049000000000004e-05 2024-09-21T14:58:36.005 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:36.005 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:36.005 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:36.005 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.005 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.005 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6708 3.d 3:b2fe5261:::rbd_data.11361bed1f78.0000000000000008:d [read 245760~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:36.005 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.134694+0000", 2024-09-21T14:58:36.005 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.81098893599999999, 2024-09-21T14:58:36.005 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012651897, 2024-09-21T14:58:36.005 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:36.005 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:36.006 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:36.006 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:36.006 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:36.006 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6708 2024-09-21T14:58:36.006 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.006 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:36.006 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.006 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:36.006 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134694+0000", 2024-09-21T14:58:36.006 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:36.006 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.006 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.007 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:36.007 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134694+0000", 2024-09-21T14:58:36.007 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:36.007 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.007 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.007 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:36.007 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134693+0000", 2024-09-21T14:58:36.007 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:36.007 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.007 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.007 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:36.007 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134696+0000", 2024-09-21T14:58:36.007 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.061e-06 2024-09-21T14:58:36.007 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.008 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.008 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:36.008 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134703+0000", 2024-09-21T14:58:36.008 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8970000000000002e-06 2024-09-21T14:58:36.008 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.008 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.008 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:36.008 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134708+0000", 2024-09-21T14:58:36.008 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8119999999999998e-06 2024-09-21T14:58:36.008 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.008 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.009 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:36.009 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.144979+0000", 2024-09-21T14:58:36.009 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.010270975999999999 2024-09-21T14:58:36.009 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.009 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.009 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:36.009 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.145049+0000", 2024-09-21T14:58:36.009 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9849000000000002e-05 2024-09-21T14:58:36.010 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.010 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.010 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:36.010 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147346+0000", 2024-09-21T14:58:36.010 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0022968799999999998 2024-09-21T14:58:36.010 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:36.010 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:36.010 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:36.010 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.010 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.010 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6709 3.1d 3:b8f5ac1a:::rbd_data.11361bed1f78.0000000000000009:d [read 98304~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:36.010 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.134712+0000", 2024-09-21T14:58:36.010 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.81097051600000003, 2024-09-21T14:58:36.011 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.010787688, 2024-09-21T14:58:36.011 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:36.011 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:36.011 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:36.011 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:36.011 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:36.011 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6709 2024-09-21T14:58:36.011 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.011 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:36.011 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.011 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:36.011 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134712+0000", 2024-09-21T14:58:36.011 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:36.011 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.012 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.012 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:36.012 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134712+0000", 2024-09-21T14:58:36.012 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:36.012 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.012 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.012 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:36.012 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134712+0000", 2024-09-21T14:58:36.012 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:36.012 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.012 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.012 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:36.012 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134720+0000", 2024-09-21T14:58:36.012 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8960000000000003e-06 2024-09-21T14:58:36.013 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.013 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.013 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:36.013 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134726+0000", 2024-09-21T14:58:36.013 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8699999999999997e-06 2024-09-21T14:58:36.013 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.013 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.013 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:36.013 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134737+0000", 2024-09-21T14:58:36.013 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0939e-05 2024-09-21T14:58:36.013 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.013 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.013 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:36.013 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.145232+0000", 2024-09-21T14:58:36.014 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.010494837999999999 2024-09-21T14:58:36.014 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.014 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.014 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:36.014 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.145310+0000", 2024-09-21T14:58:36.014 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8004999999999996e-05 2024-09-21T14:58:36.014 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.014 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.014 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:36.014 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.145500+0000", 2024-09-21T14:58:36.014 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000190466 2024-09-21T14:58:36.014 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:36.014 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:36.015 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:36.015 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.015 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.015 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6710 3.1d 3:b8f5ac1a:::rbd_data.11361bed1f78.0000000000000009:d [read 163840~32768 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:36.015 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.134742+0000", 2024-09-21T14:58:36.015 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.81094044600000004, 2024-09-21T14:58:36.015 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.010932667, 2024-09-21T14:58:36.015 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:36.015 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:36.015 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:36.015 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:36.015 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:36.015 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6710 2024-09-21T14:58:36.015 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.015 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:36.016 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.016 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:36.016 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134742+0000", 2024-09-21T14:58:36.016 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:36.016 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.016 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.016 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:36.016 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134742+0000", 2024-09-21T14:58:36.016 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:36.016 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.016 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.016 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:36.016 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134742+0000", 2024-09-21T14:58:36.016 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:36.017 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.017 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.017 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:36.017 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134745+0000", 2024-09-21T14:58:36.017 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8700000000000001e-06 2024-09-21T14:58:36.017 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.017 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.017 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:36.017 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134751+0000", 2024-09-21T14:58:36.017 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4309999999999999e-06 2024-09-21T14:58:36.017 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.017 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.017 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:36.017 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134757+0000", 2024-09-21T14:58:36.018 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5940000000000001e-06 2024-09-21T14:58:36.018 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.018 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.018 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:36.018 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.145522+0000", 2024-09-21T14:58:36.018 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.010765389 2024-09-21T14:58:36.018 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.018 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.018 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:36.018 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.145589+0000", 2024-09-21T14:58:36.018 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7005e-05 2024-09-21T14:58:36.018 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.018 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.018 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:36.019 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.145675+0000", 2024-09-21T14:58:36.019 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5785999999999995e-05 2024-09-21T14:58:36.019 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:36.019 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:36.019 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:36.019 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.019 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.019 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6711 3.1d 3:b8f5ac1a:::rbd_data.11361bed1f78.0000000000000009:d [read 245760~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:36.019 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.134795+0000", 2024-09-21T14:58:36.019 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.81088829900000003, 2024-09-21T14:58:36.019 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.011222032, 2024-09-21T14:58:36.019 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:36.019 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:36.020 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:36.020 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:36.020 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:36.020 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6711 2024-09-21T14:58:36.020 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.020 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:36.020 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.020 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:36.020 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134795+0000", 2024-09-21T14:58:36.020 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:36.020 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.020 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.020 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:36.020 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134795+0000", 2024-09-21T14:58:36.021 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:36.021 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.021 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.021 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:36.021 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134794+0000", 2024-09-21T14:58:36.021 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:36.021 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.021 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.021 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:36.021 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134797+0000", 2024-09-21T14:58:36.021 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4989999999999999e-06 2024-09-21T14:58:36.021 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.021 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.021 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:36.021 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134805+0000", 2024-09-21T14:58:36.022 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.1289999999999998e-06 2024-09-21T14:58:36.022 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.022 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.022 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:36.022 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134810+0000", 2024-09-21T14:58:36.022 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0490000000000003e-06 2024-09-21T14:58:36.022 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.022 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.022 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:36.022 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.145697+0000", 2024-09-21T14:58:36.022 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.010887841000000001 2024-09-21T14:58:36.022 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.022 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.023 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:36.023 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.145768+0000", 2024-09-21T14:58:36.023 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0112999999999998e-05 2024-09-21T14:58:36.023 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.023 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.023 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:36.023 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146017+0000", 2024-09-21T14:58:36.023 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00024907 2024-09-21T14:58:36.023 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:36.023 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:36.023 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:36.023 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.023 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.023 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6726 3.1d 3:b9c0cfe7:::rbd_data.11361bed1f78.0000000000000010:d [read 65536~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:36.023 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.134814+0000", 2024-09-21T14:58:36.024 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.81086839200000005, 2024-09-21T14:58:36.024 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.011517806, 2024-09-21T14:58:36.024 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:36.024 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:36.024 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:36.024 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:36.024 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:36.024 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6726 2024-09-21T14:58:36.024 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.024 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:36.024 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.024 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:36.024 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134814+0000", 2024-09-21T14:58:36.024 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:36.025 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.025 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.025 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:36.025 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134814+0000", 2024-09-21T14:58:36.025 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:36.025 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.025 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.025 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:36.025 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134814+0000", 2024-09-21T14:58:36.025 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:36.025 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.025 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.025 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:36.025 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134817+0000", 2024-09-21T14:58:36.026 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7389999999999999e-06 2024-09-21T14:58:36.026 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.026 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.026 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:36.026 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134822+0000", 2024-09-21T14:58:36.026 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6169999999999998e-06 2024-09-21T14:58:36.026 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.026 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.026 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:36.026 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134827+0000", 2024-09-21T14:58:36.026 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4660000000000001e-06 2024-09-21T14:58:36.026 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.026 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.027 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:36.027 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146039+0000", 2024-09-21T14:58:36.027 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.01121221 2024-09-21T14:58:36.027 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.027 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.027 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:36.027 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146112+0000", 2024-09-21T14:58:36.027 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2615999999999997e-05 2024-09-21T14:58:36.027 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.027 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.027 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:36.027 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146332+0000", 2024-09-21T14:58:36.027 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00022061300000000001 2024-09-21T14:58:36.027 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:36.028 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:36.028 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:36.028 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.028 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.028 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6737 3.1d 3:b9c0cfe7:::rbd_data.11361bed1f78.0000000000000010:d [read 81920~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:36.028 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.134866+0000", 2024-09-21T14:58:36.028 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.81081647000000001, 2024-09-21T14:58:36.028 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.011799451000000001, 2024-09-21T14:58:36.028 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:36.028 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:36.028 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:36.028 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:36.028 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:36.028 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6737 2024-09-21T14:58:36.029 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.029 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:36.029 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.029 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:36.029 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134866+0000", 2024-09-21T14:58:36.029 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:36.029 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.029 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.029 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:36.030 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134866+0000", 2024-09-21T14:58:36.030 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:36.030 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.030 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.030 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:36.030 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134866+0000", 2024-09-21T14:58:36.030 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:36.030 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.030 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.030 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:36.030 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134868+0000", 2024-09-21T14:58:36.030 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.401e-06 2024-09-21T14:58:36.030 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.031 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.031 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:36.031 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134874+0000", 2024-09-21T14:58:36.031 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8000000000000004e-06 2024-09-21T14:58:36.031 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.031 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.031 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:36.031 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134880+0000", 2024-09-21T14:58:36.031 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8329999999999999e-06 2024-09-21T14:58:36.032 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.032 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.032 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:36.032 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146355+0000", 2024-09-21T14:58:36.032 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.011474652 2024-09-21T14:58:36.032 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.032 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.032 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:36.032 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146424+0000", 2024-09-21T14:58:36.032 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9054000000000007e-05 2024-09-21T14:58:36.032 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.032 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.032 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:36.033 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146666+0000", 2024-09-21T14:58:36.033 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000242091 2024-09-21T14:58:36.033 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:36.033 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:36.033 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:36.033 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.033 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.033 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6738 3.1d 3:b9c0cfe7:::rbd_data.11361bed1f78.0000000000000010:d [read 147456~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:36.033 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.134885+0000", 2024-09-21T14:58:36.033 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.81079828200000004, 2024-09-21T14:58:36.033 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.011996183000000001, 2024-09-21T14:58:36.034 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:36.034 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:36.034 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:36.034 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:36.034 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:36.034 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6738 2024-09-21T14:58:36.034 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.034 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:36.034 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.034 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:36.035 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134885+0000", 2024-09-21T14:58:36.035 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:36.035 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.035 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.035 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:36.035 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134885+0000", 2024-09-21T14:58:36.035 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:36.035 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.035 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.035 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:36.035 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134884+0000", 2024-09-21T14:58:36.035 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:36.036 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.036 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.036 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:36.036 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134887+0000", 2024-09-21T14:58:36.036 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2649999999999999e-06 2024-09-21T14:58:36.036 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.036 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.036 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:36.036 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134892+0000", 2024-09-21T14:58:36.036 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9479999999999996e-06 2024-09-21T14:58:36.036 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.037 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.037 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:36.037 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134897+0000", 2024-09-21T14:58:36.037 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8849999999999998e-06 2024-09-21T14:58:36.037 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.037 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.037 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:36.037 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146691+0000", 2024-09-21T14:58:36.037 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.011794001 2024-09-21T14:58:36.037 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.038 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.038 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:36.038 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146793+0000", 2024-09-21T14:58:36.038 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010184 2024-09-21T14:58:36.038 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.038 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.038 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:36.038 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146881+0000", 2024-09-21T14:58:36.038 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.7563000000000002e-05 2024-09-21T14:58:36.038 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:36.038 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:36.038 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:36.039 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.039 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.039 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6739 3.1d 3:b9c0cfe7:::rbd_data.11361bed1f78.0000000000000010:d [read 229376~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:36.039 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.134904+0000", 2024-09-21T14:58:36.039 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.81077888200000003, 2024-09-21T14:58:36.039 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012287384, 2024-09-21T14:58:36.039 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:36.039 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:36.039 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:36.039 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:36.040 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:36.040 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6739 2024-09-21T14:58:36.040 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.040 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:36.040 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.040 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:36.040 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134904+0000", 2024-09-21T14:58:36.040 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:36.040 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.040 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.041 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:36.041 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134904+0000", 2024-09-21T14:58:36.041 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:36.041 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.041 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.041 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:36.041 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134904+0000", 2024-09-21T14:58:36.041 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:36.041 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.042 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.042 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:36.042 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134906+0000", 2024-09-21T14:58:36.042 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5229999999999998e-06 2024-09-21T14:58:36.042 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.042 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.042 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:36.042 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134912+0000", 2024-09-21T14:58:36.042 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3199999999999996e-06 2024-09-21T14:58:36.042 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.042 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.042 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:36.042 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134918+0000", 2024-09-21T14:58:36.043 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0560000000000001e-06 2024-09-21T14:58:36.043 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.043 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.043 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:36.043 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146905+0000", 2024-09-21T14:58:36.043 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.011986949 2024-09-21T14:58:36.043 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.043 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.043 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:36.043 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146975+0000", 2024-09-21T14:58:36.043 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9806000000000002e-05 2024-09-21T14:58:36.043 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.043 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.044 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:36.044 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147191+0000", 2024-09-21T14:58:36.044 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000216097 2024-09-21T14:58:36.044 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:36.044 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:36.044 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:36.044 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.044 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.044 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6754 3.d 3:b04809f7:::rbd_data.11361bed1f78.0000000000000017:d [read 32768~32768 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:36.044 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.135064+0000", 2024-09-21T14:58:36.044 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.81061892599999996, 2024-09-21T14:58:36.044 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012621561999999999, 2024-09-21T14:58:36.044 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:36.045 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:36.045 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:36.045 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:36.045 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:36.045 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6754 2024-09-21T14:58:36.045 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.045 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:36.045 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.045 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:36.045 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.135064+0000", 2024-09-21T14:58:36.045 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:36.045 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.045 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.045 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:36.046 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.135064+0000", 2024-09-21T14:58:36.046 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:36.046 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.046 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.046 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:36.046 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.135064+0000", 2024-09-21T14:58:36.046 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:36.046 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.046 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.046 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:36.046 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.135068+0000", 2024-09-21T14:58:36.046 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5630000000000002e-06 2024-09-21T14:58:36.047 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.047 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.047 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:36.047 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.135074+0000", 2024-09-21T14:58:36.047 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1979999999999998e-06 2024-09-21T14:58:36.047 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.047 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.047 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:36.047 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.135080+0000", 2024-09-21T14:58:36.047 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.147e-06 2024-09-21T14:58:36.047 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.047 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.047 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:36.047 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147380+0000", 2024-09-21T14:58:36.048 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012300075000000001 2024-09-21T14:58:36.048 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.048 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.048 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:36.048 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147460+0000", 2024-09-21T14:58:36.048 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0289000000000003e-05 2024-09-21T14:58:36.048 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.048 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.048 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:36.048 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147685+0000", 2024-09-21T14:58:36.048 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00022560899999999999 2024-09-21T14:58:36.048 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:36.048 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:36.048 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:36.048 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.049 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.049 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6756 3.d 3:b2f690c6:::rbd_data.11361bed1f78.0000000000000019:d [read 32768~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:36.049 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.135084+0000", 2024-09-21T14:58:36.049 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.81059838500000003, 2024-09-21T14:58:36.049 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013162790000000001, 2024-09-21T14:58:36.049 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:36.049 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:36.049 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:36.049 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:36.049 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:36.049 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6756 2024-09-21T14:58:36.049 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.049 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:36.049 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.050 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:36.050 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.135084+0000", 2024-09-21T14:58:36.050 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:36.050 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.050 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.050 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:36.050 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.135084+0000", 2024-09-21T14:58:36.050 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:36.050 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.050 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.050 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:36.050 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.135084+0000", 2024-09-21T14:58:36.050 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:36.051 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.051 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.051 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:36.051 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.135087+0000", 2024-09-21T14:58:36.051 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.768e-06 2024-09-21T14:58:36.051 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.051 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.051 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:36.051 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.135092+0000", 2024-09-21T14:58:36.051 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4410000000000001e-06 2024-09-21T14:58:36.051 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.051 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.051 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:36.051 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.135097+0000", 2024-09-21T14:58:36.052 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6430000000000003e-06 2024-09-21T14:58:36.052 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.052 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.052 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:36.052 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147922+0000", 2024-09-21T14:58:36.052 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012824991000000001 2024-09-21T14:58:36.052 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.052 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.052 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:36.052 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.148033+0000", 2024-09-21T14:58:36.052 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000111394 2024-09-21T14:58:36.052 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:36.052 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:36.052 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:36.053 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.148247+0000", 2024-09-21T14:58:36.053 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00021402500000000001 2024-09-21T14:58:36.053 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:36.053 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:36.053 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:36.053 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:36.053 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:36.053 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:58:36.054 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T14:58:36.055 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:58:36.051+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:58:36.155 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:58:36.151+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:58:36.175 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:58:36.175 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:58:36.175 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T14:58:36.175 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:58:36.184 INFO:teuthology.orchestra.run.smithi032.stdout:checking clone #2, image image_client.0-clone2 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent3 2024-09-21T14:58:36.185 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T14:58:36.256 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:58:36.251+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:58:36.287 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:58:36.287 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:58:36.287 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T14:58:36.287 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T14:58:36.287 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:58:36.299 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T14:58:36.357 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:58:36.356+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:58:36.385 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T14:58:36.399 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:58:36.399 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T14:58:36.399 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T14:58:36.400 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T14:58:36.400 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.400 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12 3.6 3:6748301c:::rbd_header.11367ffd86fe:head [create,call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T14:58:36.400 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:21.422589+0000", 2024-09-21T14:58:36.400 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.974081653000001, 2024-09-21T14:58:36.400 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0059205330000000004, 2024-09-21T14:58:36.400 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:36.400 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:36.400 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:36.400 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:36.400 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:36.401 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12 2024-09-21T14:58:36.401 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.401 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:36.401 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.401 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:36.401 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.422589+0000", 2024-09-21T14:58:36.401 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:36.401 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.401 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.401 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:36.402 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.422589+0000", 2024-09-21T14:58:36.402 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:36.402 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.402 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.402 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:36.402 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.422587+0000", 2024-09-21T14:58:36.402 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T14:58:36.402 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.402 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.402 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:36.402 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.422598+0000", 2024-09-21T14:58:36.403 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0889e-05 2024-09-21T14:58:36.403 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.403 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.403 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:36.403 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.422614+0000", 2024-09-21T14:58:36.403 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5869999999999999e-05 2024-09-21T14:58:36.403 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.403 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.403 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:36.403 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.422632+0000", 2024-09-21T14:58:36.404 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8172000000000002e-05 2024-09-21T14:58:36.404 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.404 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.404 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:36.404 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.422709+0000", 2024-09-21T14:58:36.404 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6993e-05 2024-09-21T14:58:36.404 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.404 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.404 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op must wait for map", 2024-09-21T14:58:36.404 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.422714+0000", 2024-09-21T14:58:36.404 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.369e-06 2024-09-21T14:58:36.404 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.404 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.404 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:36.405 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.425359+0000", 2024-09-21T14:58:36.405 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0026451970000000002 2024-09-21T14:58:36.405 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.405 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.405 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:36.405 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.425855+0000", 2024-09-21T14:58:36.405 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000496591 2024-09-21T14:58:36.405 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.405 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.405 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 7", 2024-09-21T14:58:36.405 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.426174+0000", 2024-09-21T14:58:36.405 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00031870899999999997 2024-09-21T14:58:36.405 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.405 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.405 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:58:36.406 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.427445+0000", 2024-09-21T14:58:36.406 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001270497 2024-09-21T14:58:36.406 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.406 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.406 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:36.406 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.428336+0000", 2024-09-21T14:58:36.406 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00089194000000000003 2024-09-21T14:58:36.406 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.406 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.406 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:36.406 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.428443+0000", 2024-09-21T14:58:36.406 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000106279 2024-09-21T14:58:36.406 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.406 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.406 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:36.407 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:21.428510+0000", 2024-09-21T14:58:36.407 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7070999999999995e-05 2024-09-21T14:58:36.407 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:36.407 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:36.407 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:36.407 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.407 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.407 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:726 3.6 3:63a18c8a:::rbd_object_map.113693316f36:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-21T14:58:36.407 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:26.463417+0000", 2024-09-21T14:58:36.407 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.9332536640000004, 2024-09-21T14:58:36.407 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0055121629999999996, 2024-09-21T14:58:36.407 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:36.407 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:36.407 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:36.407 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:36.408 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:36.408 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 726 2024-09-21T14:58:36.408 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.408 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:36.408 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.408 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:36.408 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.463417+0000", 2024-09-21T14:58:36.408 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:36.408 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.408 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.408 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:36.408 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.463417+0000", 2024-09-21T14:58:36.408 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:36.408 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.409 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.409 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:36.409 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.463415+0000", 2024-09-21T14:58:36.409 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999971 2024-09-21T14:58:36.409 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.409 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.409 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:36.409 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.463432+0000", 2024-09-21T14:58:36.409 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7070000000000001e-05 2024-09-21T14:58:36.409 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.409 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.409 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:36.409 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.463465+0000", 2024-09-21T14:58:36.409 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3339e-05 2024-09-21T14:58:36.409 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.410 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.410 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:36.410 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.463488+0000", 2024-09-21T14:58:36.410 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2742999999999999e-05 2024-09-21T14:58:36.410 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.410 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.410 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:36.410 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.463622+0000", 2024-09-21T14:58:36.410 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013426399999999999 2024-09-21T14:58:36.410 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.410 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.410 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op must wait for map", 2024-09-21T14:58:36.410 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.463632+0000", 2024-09-21T14:58:36.410 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6460000000000005e-06 2024-09-21T14:58:36.410 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.411 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.411 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:36.411 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.466384+0000", 2024-09-21T14:58:36.411 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0027527799999999998 2024-09-21T14:58:36.411 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.411 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.411 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:36.411 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.466610+0000", 2024-09-21T14:58:36.411 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00022558699999999999 2024-09-21T14:58:36.411 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.411 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.411 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 7", 2024-09-21T14:58:36.411 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.467102+0000", 2024-09-21T14:58:36.411 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00049241499999999995 2024-09-21T14:58:36.411 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.411 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.412 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:58:36.412 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.468599+0000", 2024-09-21T14:58:36.412 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00149614 2024-09-21T14:58:36.412 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.412 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.412 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:36.412 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.468824+0000", 2024-09-21T14:58:36.412 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00022584699999999999 2024-09-21T14:58:36.412 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.412 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.412 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:36.412 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.468888+0000", 2024-09-21T14:58:36.412 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3678999999999998e-05 2024-09-21T14:58:36.412 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.412 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.413 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:36.413 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:26.468929+0000", 2024-09-21T14:58:36.413 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1285999999999997e-05 2024-09-21T14:58:36.413 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:36.413 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:36.413 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:36.413 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.413 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.413 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:1452 3.19 3:9d40fe72:::rbd_data.1136515ecb6b.0000000000000013:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T14:58:36.413 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:30.292755+0000", 2024-09-21T14:58:36.413 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 6.1039157880000001, 2024-09-21T14:58:36.413 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043852191999999998, 2024-09-21T14:58:36.413 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:36.413 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:36.414 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:36.414 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:36.414 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:36.414 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1452 2024-09-21T14:58:36.414 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.414 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:36.414 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.414 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:36.414 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292755+0000", 2024-09-21T14:58:36.414 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:36.414 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.414 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.414 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:36.414 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292755+0000", 2024-09-21T14:58:36.415 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:36.415 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.415 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.415 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:36.415 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292754+0000", 2024-09-21T14:58:36.415 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:36.415 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.415 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.415 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:36.415 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292760+0000", 2024-09-21T14:58:36.415 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5029999999999997e-06 2024-09-21T14:58:36.415 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.415 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.415 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:36.415 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292771+0000", 2024-09-21T14:58:36.416 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1551e-05 2024-09-21T14:58:36.416 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.416 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.416 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:36.416 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292779+0000", 2024-09-21T14:58:36.416 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1559999999999998e-06 2024-09-21T14:58:36.416 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.416 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.416 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:36.416 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292896+0000", 2024-09-21T14:58:36.416 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000117304 2024-09-21T14:58:36.416 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.416 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.416 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:36.416 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.293124+0000", 2024-09-21T14:58:36.417 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00022819999999999999 2024-09-21T14:58:36.417 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.417 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.417 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 1", 2024-09-21T14:58:36.417 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.299967+0000", 2024-09-21T14:58:36.417 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0068427009999999996 2024-09-21T14:58:36.417 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.417 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.417 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:36.417 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336463+0000", 2024-09-21T14:58:36.417 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.036495842000000001 2024-09-21T14:58:36.417 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.417 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.417 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:58:36.417 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336480+0000", 2024-09-21T14:58:36.417 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7419999999999999e-05 2024-09-21T14:58:36.418 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.418 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.418 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:36.418 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336550+0000", 2024-09-21T14:58:36.418 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9951999999999997e-05 2024-09-21T14:58:36.418 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.418 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.418 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:36.418 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336607+0000", 2024-09-21T14:58:36.418 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7423000000000002e-05 2024-09-21T14:58:36.418 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:36.418 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:36.418 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:36.418 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.418 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.419 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop_reply(client.4406.0:1452 3.19 e36/24 ondisk, result = 0)", 2024-09-21T14:58:36.419 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:30.322854+0000", 2024-09-21T14:58:36.419 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 6.0738171120000004, 2024-09-21T14:58:36.419 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013613079, 2024-09-21T14:58:36.419 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:36.419 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:36.419 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:36.419 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.419 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:36.419 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.322854+0000", 2024-09-21T14:58:36.419 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:36.419 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.419 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.419 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:36.419 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.322854+0000", 2024-09-21T14:58:36.419 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:36.420 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.420 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.420 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:36.420 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.322852+0000", 2024-09-21T14:58:36.420 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:36.420 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.420 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.420 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:36.420 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.322860+0000", 2024-09-21T14:58:36.420 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.605e-06 2024-09-21T14:58:36.420 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.420 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.420 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:36.420 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.322878+0000", 2024-09-21T14:58:36.420 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8204000000000001e-05 2024-09-21T14:58:36.421 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.421 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.421 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:36.421 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.322888+0000", 2024-09-21T14:58:36.421 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4099999999999997e-06 2024-09-21T14:58:36.421 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.421 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.421 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:36.421 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336440+0000", 2024-09-21T14:58:36.421 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013551848 2024-09-21T14:58:36.421 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.421 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.421 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:36.422 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336453+0000", 2024-09-21T14:58:36.422 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3465000000000001e-05 2024-09-21T14:58:36.422 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.422 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.422 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:36.422 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336467+0000", 2024-09-21T14:58:36.422 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3913999999999999e-05 2024-09-21T14:58:36.422 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:36.422 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:36.422 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:36.422 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.422 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.422 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:1456 3.2 3:4346589c:::rbd_data.1136515ecb6b.0000000000000015:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T14:58:36.422 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:30.378684+0000", 2024-09-21T14:58:36.423 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 6.0179872559999996, 2024-09-21T14:58:36.423 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0080153619999999998, 2024-09-21T14:58:36.423 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:36.423 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:36.423 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:36.423 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:36.423 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:36.423 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1456 2024-09-21T14:58:36.423 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.423 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:36.423 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.423 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:36.423 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.378684+0000", 2024-09-21T14:58:36.423 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:36.424 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.424 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.424 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:36.424 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.378684+0000", 2024-09-21T14:58:36.424 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:36.424 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.424 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.424 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:36.424 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.378681+0000", 2024-09-21T14:58:36.424 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-21T14:58:36.424 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.425 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.425 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:36.425 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.378701+0000", 2024-09-21T14:58:36.425 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9066999999999999e-05 2024-09-21T14:58:36.425 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.425 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.425 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:36.425 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.378727+0000", 2024-09-21T14:58:36.425 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6517999999999999e-05 2024-09-21T14:58:36.425 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.425 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.425 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:36.426 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.378745+0000", 2024-09-21T14:58:36.426 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7965000000000002e-05 2024-09-21T14:58:36.426 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.426 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.426 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:36.426 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.378848+0000", 2024-09-21T14:58:36.426 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000103257 2024-09-21T14:58:36.426 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.426 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.426 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:36.426 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.379077+0000", 2024-09-21T14:58:36.426 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000228843 2024-09-21T14:58:36.426 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.427 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.427 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 2", 2024-09-21T14:58:36.427 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.380388+0000", 2024-09-21T14:58:36.427 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0013109370000000001 2024-09-21T14:58:36.427 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.427 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.427 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:36.427 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.386570+0000", 2024-09-21T14:58:36.427 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0061821050000000002 2024-09-21T14:58:36.427 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.427 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.427 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:58:36.427 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.386590+0000", 2024-09-21T14:58:36.428 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0251e-05 2024-09-21T14:58:36.428 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.428 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.428 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:36.428 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.386650+0000", 2024-09-21T14:58:36.428 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9466000000000003e-05 2024-09-21T14:58:36.428 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.428 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.428 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:36.428 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.386699+0000", 2024-09-21T14:58:36.428 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9175000000000003e-05 2024-09-21T14:58:36.428 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:36.429 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:36.429 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:36.429 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.429 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.429 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:1567 3.8 3:139d3d55:::rbd_data.1136515ecb6b.0000000000000000:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T14:58:36.429 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:30.681087+0000", 2024-09-21T14:58:36.429 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.715583573, 2024-09-21T14:58:36.429 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.039009224000000002, 2024-09-21T14:58:36.429 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:36.429 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:36.429 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:36.430 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:36.430 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:36.430 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1567 2024-09-21T14:58:36.430 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.430 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:36.430 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.430 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:36.430 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681087+0000", 2024-09-21T14:58:36.430 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:36.430 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.430 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.431 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:36.431 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681087+0000", 2024-09-21T14:58:36.431 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:36.431 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.431 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.431 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:36.431 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681086+0000", 2024-09-21T14:58:36.431 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:36.431 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.431 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.431 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:36.432 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681097+0000", 2024-09-21T14:58:36.432 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.134e-05 2024-09-21T14:58:36.432 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.432 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.432 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:36.432 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681122+0000", 2024-09-21T14:58:36.432 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5313e-05 2024-09-21T14:58:36.432 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.432 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.432 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:36.432 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681138+0000", 2024-09-21T14:58:36.432 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5916000000000001e-05 2024-09-21T14:58:36.432 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.432 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.433 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:36.433 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681270+0000", 2024-09-21T14:58:36.433 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000131474 2024-09-21T14:58:36.433 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.433 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.433 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:36.433 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681416+0000", 2024-09-21T14:58:36.433 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014637 2024-09-21T14:58:36.433 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.433 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.433 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 7", 2024-09-21T14:58:36.433 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.693163+0000", 2024-09-21T14:58:36.433 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011747241 2024-09-21T14:58:36.433 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.433 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.434 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:58:36.434 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.719750+0000", 2024-09-21T14:58:36.434 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.026586852000000001 2024-09-21T14:58:36.434 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.434 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.434 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:36.434 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.719859+0000", 2024-09-21T14:58:36.434 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010925999999999999 2024-09-21T14:58:36.434 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.435 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.435 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:36.435 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.719951+0000", 2024-09-21T14:58:36.435 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1294999999999999e-05 2024-09-21T14:58:36.435 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.435 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.435 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:36.435 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.720097+0000", 2024-09-21T14:58:36.435 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014583 2024-09-21T14:58:36.436 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:36.436 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:36.436 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:36.436 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.436 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.436 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:1651 3.8 3:120e2780:::rbd_data.1136515ecb6b.0000000000000005:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T14:58:36.436 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:30.920757+0000", 2024-09-21T14:58:36.436 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.4759141519999996, 2024-09-21T14:58:36.436 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010533140999999999, 2024-09-21T14:58:36.436 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:36.436 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:36.436 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:36.436 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:36.436 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:36.436 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1651 2024-09-21T14:58:36.437 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.437 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:36.437 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.437 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:36.437 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.920757+0000", 2024-09-21T14:58:36.437 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:36.437 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.437 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.437 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:36.437 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.920757+0000", 2024-09-21T14:58:36.437 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:36.437 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.437 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.437 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:36.438 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.920755+0000", 2024-09-21T14:58:36.438 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T14:58:36.438 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.438 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.438 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:36.438 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.920769+0000", 2024-09-21T14:58:36.438 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4156e-05 2024-09-21T14:58:36.438 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.438 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.438 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:36.438 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.920804+0000", 2024-09-21T14:58:36.438 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4898999999999999e-05 2024-09-21T14:58:36.438 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.438 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.438 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:36.438 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.920818+0000", 2024-09-21T14:58:36.439 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4676e-05 2024-09-21T14:58:36.439 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.439 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.439 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:36.439 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.920931+0000", 2024-09-21T14:58:36.439 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000112465 2024-09-21T14:58:36.439 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.439 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.439 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:36.439 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.921191+0000", 2024-09-21T14:58:36.439 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00025991099999999999 2024-09-21T14:58:36.439 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.439 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.439 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 7", 2024-09-21T14:58:36.439 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.923372+0000", 2024-09-21T14:58:36.440 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0021811980000000001 2024-09-21T14:58:36.440 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.440 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.440 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:36.440 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.930804+0000", 2024-09-21T14:58:36.440 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.007431844 2024-09-21T14:58:36.440 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.440 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.440 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:58:36.440 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.931137+0000", 2024-09-21T14:58:36.440 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00033363700000000002 2024-09-21T14:58:36.440 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.440 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.440 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:36.440 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.931231+0000", 2024-09-21T14:58:36.440 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3251999999999994e-05 2024-09-21T14:58:36.441 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.441 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.441 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:36.441 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.931290+0000", 2024-09-21T14:58:36.441 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9236e-05 2024-09-21T14:58:36.441 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:36.441 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:36.441 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:36.441 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.441 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.441 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:6202 3.7 3:e1c9b41d:::rbd_object_map.11361bed1f78:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-21T14:58:36.441 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:34.315052+0000", 2024-09-21T14:58:36.441 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 2.0816185030000001, 2024-09-21T14:58:36.441 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0054141980000000003, 2024-09-21T14:58:36.441 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:36.442 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:36.442 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:36.442 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:36.442 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:36.442 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 6202 2024-09-21T14:58:36.442 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.442 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:36.442 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.442 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:36.442 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:34.315052+0000", 2024-09-21T14:58:36.442 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:36.442 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.442 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.442 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:36.442 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:34.315052+0000", 2024-09-21T14:58:36.443 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:36.443 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.443 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.443 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:36.443 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:34.315051+0000", 2024-09-21T14:58:36.443 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:36.443 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.443 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.443 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:36.443 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:34.315061+0000", 2024-09-21T14:58:36.443 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.594e-06 2024-09-21T14:58:36.443 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.443 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.444 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:36.444 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:34.315078+0000", 2024-09-21T14:58:36.444 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7119999999999999e-05 2024-09-21T14:58:36.444 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.444 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.444 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:36.444 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:34.315091+0000", 2024-09-21T14:58:36.444 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2968e-05 2024-09-21T14:58:36.444 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.444 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.444 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:36.444 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:34.316750+0000", 2024-09-21T14:58:36.444 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0016591609999999999 2024-09-21T14:58:36.444 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.445 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.445 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:36.445 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:34.316830+0000", 2024-09-21T14:58:36.445 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0364999999999994e-05 2024-09-21T14:58:36.445 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.445 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.445 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T14:58:36.445 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:34.317216+0000", 2024-09-21T14:58:36.445 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000385955 2024-09-21T14:58:36.445 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.445 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.445 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:58:36.445 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:34.319970+0000", 2024-09-21T14:58:36.445 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0027540680000000001 2024-09-21T14:58:36.445 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.446 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.446 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:36.446 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:34.320296+0000", 2024-09-21T14:58:36.446 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00032523699999999998 2024-09-21T14:58:36.446 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.446 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.446 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:36.446 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:34.320374+0000", 2024-09-21T14:58:36.446 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8186e-05 2024-09-21T14:58:36.446 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.446 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.446 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:36.446 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:34.320467+0000", 2024-09-21T14:58:36.446 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2812999999999994e-05 2024-09-21T14:58:36.446 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:36.447 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:36.447 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:36.447 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.447 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.447 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:6203 3.7 3:e1c9b41d:::rbd_object_map.11361bed1f78:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-21T14:58:36.447 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:34.315132+0000", 2024-09-21T14:58:36.447 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 2.0815386299999998, 2024-09-21T14:58:36.447 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0062492169999999996, 2024-09-21T14:58:36.447 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:36.447 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:36.447 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:36.447 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:36.448 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:36.448 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 6203 2024-09-21T14:58:36.448 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.448 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:36.448 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.448 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:36.448 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:34.315132+0000", 2024-09-21T14:58:36.448 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:36.448 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.448 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.448 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:36.448 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:34.315132+0000", 2024-09-21T14:58:36.448 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:36.448 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.448 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.448 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:36.449 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:34.315131+0000", 2024-09-21T14:58:36.449 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:36.449 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.449 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.449 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:36.449 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:34.315140+0000", 2024-09-21T14:58:36.449 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9199999999999993e-06 2024-09-21T14:58:36.449 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.449 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.449 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:36.449 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:34.315157+0000", 2024-09-21T14:58:36.449 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6657e-05 2024-09-21T14:58:36.449 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.449 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.450 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:36.450 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:34.315170+0000", 2024-09-21T14:58:36.450 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3719e-05 2024-09-21T14:58:36.450 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.450 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.450 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:36.450 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:34.318182+0000", 2024-09-21T14:58:36.450 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.003011508 2024-09-21T14:58:36.450 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.450 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.450 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:36.450 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:34.318302+0000", 2024-09-21T14:58:36.450 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012045 2024-09-21T14:58:36.450 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.450 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.451 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T14:58:36.451 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:34.319049+0000", 2024-09-21T14:58:36.451 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00074637 2024-09-21T14:58:36.451 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.451 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.451 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:58:36.451 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:34.320496+0000", 2024-09-21T14:58:36.451 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0014472840000000001 2024-09-21T14:58:36.451 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.451 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.451 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:36.451 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:34.321216+0000", 2024-09-21T14:58:36.451 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00071985300000000001 2024-09-21T14:58:36.451 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.451 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.451 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:36.452 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:34.321310+0000", 2024-09-21T14:58:36.452 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4572000000000002e-05 2024-09-21T14:58:36.452 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.452 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.452 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:36.452 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:34.321382+0000", 2024-09-21T14:58:36.452 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1132999999999998e-05 2024-09-21T14:58:36.452 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:36.452 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:36.452 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:36.452 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.452 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.452 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:6413 3.7 3:e306c1b4:::rbd_data.11361bed1f78.0000000000000004:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:36.452 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:35.121116+0000", 2024-09-21T14:58:36.452 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.275554989, 2024-09-21T14:58:36.452 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0054503989999999999, 2024-09-21T14:58:36.453 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:36.453 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:36.453 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:36.453 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:36.453 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:36.453 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 6413 2024-09-21T14:58:36.453 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.453 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:36.453 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.453 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:36.453 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.121116+0000", 2024-09-21T14:58:36.453 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:36.453 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.453 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.453 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:36.453 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.121116+0000", 2024-09-21T14:58:36.454 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:36.454 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.454 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.454 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:36.454 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.121115+0000", 2024-09-21T14:58:36.454 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:36.454 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.454 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.454 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:36.454 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.121120+0000", 2024-09-21T14:58:36.454 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.062e-06 2024-09-21T14:58:36.455 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.455 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.455 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:36.455 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.121132+0000", 2024-09-21T14:58:36.455 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1637999999999999e-05 2024-09-21T14:58:36.455 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.455 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.455 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:36.455 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.121143+0000", 2024-09-21T14:58:36.455 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0937999999999999e-05 2024-09-21T14:58:36.455 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.456 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.456 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:36.456 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.121340+0000", 2024-09-21T14:58:36.456 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019743699999999999 2024-09-21T14:58:36.456 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.456 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.456 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting_for_map not empty", 2024-09-21T14:58:36.456 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.121354+0000", 2024-09-21T14:58:36.456 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3424000000000001e-05 2024-09-21T14:58:36.456 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.456 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.456 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:36.456 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.126352+0000", 2024-09-21T14:58:36.456 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0049982389999999998 2024-09-21T14:58:36.457 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.457 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.457 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:36.457 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.126472+0000", 2024-09-21T14:58:36.457 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000120339 2024-09-21T14:58:36.457 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.457 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.457 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:36.457 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.126566+0000", 2024-09-21T14:58:36.457 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4060000000000004e-05 2024-09-21T14:58:36.458 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:36.458 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:36.458 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:36.458 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.458 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.458 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:6637 3.7 3:e306c1b4:::rbd_data.11361bed1f78.0000000000000004:d [read 147456~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:36.458 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:35.132562+0000", 2024-09-21T14:58:36.458 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.264108834, 2024-09-21T14:58:36.458 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0057056310000000001, 2024-09-21T14:58:36.458 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:36.458 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:36.458 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:36.458 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:36.458 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:36.459 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 6637 2024-09-21T14:58:36.459 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.459 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:36.459 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.459 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:36.459 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.132562+0000", 2024-09-21T14:58:36.459 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:36.459 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.459 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.459 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:36.459 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.132562+0000", 2024-09-21T14:58:36.460 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:36.460 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.460 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.460 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:36.460 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.132562+0000", 2024-09-21T14:58:36.460 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:36.460 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.460 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.460 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:36.460 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.132566+0000", 2024-09-21T14:58:36.460 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9820000000000002e-06 2024-09-21T14:58:36.460 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.460 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.460 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:36.461 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.132575+0000", 2024-09-21T14:58:36.461 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.628e-06 2024-09-21T14:58:36.461 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.461 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.461 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:36.461 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.132595+0000", 2024-09-21T14:58:36.461 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9613e-05 2024-09-21T14:58:36.461 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.461 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.461 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:36.461 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.137835+0000", 2024-09-21T14:58:36.462 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0052404030000000002 2024-09-21T14:58:36.462 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.462 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.462 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:36.462 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.137959+0000", 2024-09-21T14:58:36.462 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012413499999999999 2024-09-21T14:58:36.462 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.462 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.462 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:36.462 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.138268+0000", 2024-09-21T14:58:36.462 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00030842100000000002 2024-09-21T14:58:36.462 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:36.463 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:36.463 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:36.463 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.463 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.463 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:6638 3.7 3:e306c1b4:::rbd_data.11361bed1f78.0000000000000004:d [read 229376~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:36.463 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:35.132615+0000", 2024-09-21T14:58:36.463 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.264055532, 2024-09-21T14:58:36.463 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0058856639999999997, 2024-09-21T14:58:36.463 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:36.463 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:36.463 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:36.464 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:36.464 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:36.464 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 6638 2024-09-21T14:58:36.464 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.464 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:36.464 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.464 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:36.464 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.132615+0000", 2024-09-21T14:58:36.464 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:36.464 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.465 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.465 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:36.465 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.132615+0000", 2024-09-21T14:58:36.465 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:36.465 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.465 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.465 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:36.465 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.132615+0000", 2024-09-21T14:58:36.465 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:36.465 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.466 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.466 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:36.466 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.132623+0000", 2024-09-21T14:58:36.466 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0900000000000005e-06 2024-09-21T14:58:36.466 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.466 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.466 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:36.466 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.132644+0000", 2024-09-21T14:58:36.466 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0509999999999998e-05 2024-09-21T14:58:36.466 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.466 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.467 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:36.467 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.132652+0000", 2024-09-21T14:58:36.467 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6680000000000007e-06 2024-09-21T14:58:36.467 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.467 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.467 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:36.467 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.138285+0000", 2024-09-21T14:58:36.467 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0056329279999999997 2024-09-21T14:58:36.467 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.467 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.467 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:36.468 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.138333+0000", 2024-09-21T14:58:36.468 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8214999999999998e-05 2024-09-21T14:58:36.468 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.468 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.468 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:36.468 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.138501+0000", 2024-09-21T14:58:36.468 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016766900000000001 2024-09-21T14:58:36.468 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:36.468 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:36.468 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:36.468 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.468 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.468 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:6661 3.7 3:e2aed099:::rbd_data.11361bed1f78.000000000000000b:d [read 196608~32768 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:36.468 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:35.134125+0000", 2024-09-21T14:58:36.469 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.262546247, 2024-09-21T14:58:36.469 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0055712019999999999, 2024-09-21T14:58:36.469 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:36.469 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:36.469 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:36.469 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:36.469 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:36.469 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 6661 2024-09-21T14:58:36.469 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.469 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:36.469 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.469 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:36.469 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.134125+0000", 2024-09-21T14:58:36.469 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:36.469 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.470 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.470 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:36.470 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.134125+0000", 2024-09-21T14:58:36.470 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:36.470 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.470 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.470 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:36.470 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.134124+0000", 2024-09-21T14:58:36.470 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:36.470 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.470 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.470 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:36.470 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.134128+0000", 2024-09-21T14:58:36.470 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2230000000000001e-06 2024-09-21T14:58:36.471 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.471 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.471 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:36.471 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.134137+0000", 2024-09-21T14:58:36.471 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1880000000000008e-06 2024-09-21T14:58:36.471 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.471 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.471 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:36.471 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.134145+0000", 2024-09-21T14:58:36.471 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3440000000000003e-06 2024-09-21T14:58:36.471 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.471 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.471 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:36.471 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.139260+0000", 2024-09-21T14:58:36.471 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0051159380000000004 2024-09-21T14:58:36.472 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.472 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.472 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:36.472 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.139393+0000", 2024-09-21T14:58:36.472 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013229800000000001 2024-09-21T14:58:36.472 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.472 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.472 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:36.472 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.139696+0000", 2024-09-21T14:58:36.472 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000303128 2024-09-21T14:58:36.472 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:36.472 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:36.472 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:36.472 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.473 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.473 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:6687 3.7 3:e2aed099:::rbd_data.11361bed1f78.000000000000000b:d [read 65536~32768 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:36.473 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:35.134544+0000", 2024-09-21T14:58:36.473 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.262127223, 2024-09-21T14:58:36.473 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0054244330000000002, 2024-09-21T14:58:36.473 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:36.473 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:36.473 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:36.473 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:36.473 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:36.473 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 6687 2024-09-21T14:58:36.473 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.473 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:36.473 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.473 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:36.474 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.134544+0000", 2024-09-21T14:58:36.474 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:36.474 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.474 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.474 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:36.474 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.134544+0000", 2024-09-21T14:58:36.474 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:36.474 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.474 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.474 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:36.474 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.134542+0000", 2024-09-21T14:58:36.475 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:36.475 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.475 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.475 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:36.475 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.134550+0000", 2024-09-21T14:58:36.475 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9049999999999997e-06 2024-09-21T14:58:36.475 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.475 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.475 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:36.475 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.134565+0000", 2024-09-21T14:58:36.475 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4263e-05 2024-09-21T14:58:36.475 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.475 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.475 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:36.476 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.134576+0000", 2024-09-21T14:58:36.476 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1214e-05 2024-09-21T14:58:36.476 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.476 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.476 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:36.476 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.139744+0000", 2024-09-21T14:58:36.476 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0051683090000000003 2024-09-21T14:58:36.476 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.476 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.476 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:36.476 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.139799+0000", 2024-09-21T14:58:36.476 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4515999999999998e-05 2024-09-21T14:58:36.476 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.476 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.477 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:36.477 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.139968+0000", 2024-09-21T14:58:36.477 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000169471 2024-09-21T14:58:36.477 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:36.477 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:36.477 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:36.477 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.477 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.477 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:6688 3.7 3:e2aed099:::rbd_data.11361bed1f78.000000000000000b:d [read 147456~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:36.477 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:35.134586+0000", 2024-09-21T14:58:36.477 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.262085329, 2024-09-21T14:58:36.477 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0057116390000000001, 2024-09-21T14:58:36.477 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:36.478 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:36.478 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:36.478 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:36.478 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:36.478 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 6688 2024-09-21T14:58:36.478 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.478 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:36.478 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.478 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:36.478 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.134586+0000", 2024-09-21T14:58:36.478 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:36.478 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.479 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.479 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:36.479 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.134586+0000", 2024-09-21T14:58:36.479 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:36.479 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.479 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.479 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:36.479 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.134585+0000", 2024-09-21T14:58:36.479 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:36.479 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.479 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.479 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:36.479 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.134590+0000", 2024-09-21T14:58:36.479 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9919999999999998e-06 2024-09-21T14:58:36.480 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.480 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.480 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:36.480 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.134599+0000", 2024-09-21T14:58:36.480 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1179999999999998e-06 2024-09-21T14:58:36.480 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.480 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.480 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:36.480 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.134607+0000", 2024-09-21T14:58:36.480 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8080000000000005e-06 2024-09-21T14:58:36.480 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.480 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.480 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:36.481 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.140021+0000", 2024-09-21T14:58:36.481 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0054145030000000002 2024-09-21T14:58:36.481 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.481 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.481 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:36.481 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.140146+0000", 2024-09-21T14:58:36.481 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012510900000000001 2024-09-21T14:58:36.481 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.481 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.481 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:36.481 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.140297+0000", 2024-09-21T14:58:36.481 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015104800000000001 2024-09-21T14:58:36.481 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:36.481 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:36.482 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:36.482 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.482 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.482 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:6689 3.7 3:e2aed099:::rbd_data.11361bed1f78.000000000000000b:d [read 229376~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:36.482 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:35.134614+0000", 2024-09-21T14:58:36.482 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.2620573880000001, 2024-09-21T14:58:36.482 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0061794800000000002, 2024-09-21T14:58:36.482 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:36.482 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:36.482 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:36.482 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:36.482 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:36.482 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 6689 2024-09-21T14:58:36.482 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.483 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:36.483 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.483 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:36.483 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.134614+0000", 2024-09-21T14:58:36.483 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:36.483 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.483 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.483 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:36.483 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.134614+0000", 2024-09-21T14:58:36.483 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:36.483 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.483 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.483 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:36.484 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.134613+0000", 2024-09-21T14:58:36.484 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:36.484 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.484 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.484 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:36.484 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.134617+0000", 2024-09-21T14:58:36.484 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2769999999999999e-06 2024-09-21T14:58:36.484 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.484 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.484 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:36.484 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.134625+0000", 2024-09-21T14:58:36.484 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1899999999999995e-06 2024-09-21T14:58:36.484 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.485 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.485 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:36.485 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.134632+0000", 2024-09-21T14:58:36.485 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.793e-06 2024-09-21T14:58:36.485 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.485 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.485 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:36.485 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.140340+0000", 2024-09-21T14:58:36.485 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0057082330000000001 2024-09-21T14:58:36.485 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.485 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.485 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:36.485 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.140462+0000", 2024-09-21T14:58:36.485 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012144199999999999 2024-09-21T14:58:36.486 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.486 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.486 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:36.486 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.140793+0000", 2024-09-21T14:58:36.486 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000331157 2024-09-21T14:58:36.486 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:36.486 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:36.486 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:36.486 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.486 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.486 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:6715 3.7 3:e2aed099:::rbd_data.11361bed1f78.000000000000000b:d [read 98304~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:36.486 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:35.135264+0000", 2024-09-21T14:58:36.487 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.2614068330000001, 2024-09-21T14:58:36.487 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0058929200000000003, 2024-09-21T14:58:36.487 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:36.487 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:36.487 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:36.487 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:36.487 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:36.487 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 6715 2024-09-21T14:58:36.487 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.487 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:36.488 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.488 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:36.488 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.135264+0000", 2024-09-21T14:58:36.488 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:36.488 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.488 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.488 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:36.488 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.135264+0000", 2024-09-21T14:58:36.488 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:36.488 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.488 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.488 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:36.488 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.135263+0000", 2024-09-21T14:58:36.488 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:36.489 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.489 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.489 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:36.489 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.135267+0000", 2024-09-21T14:58:36.489 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1409999999999998e-06 2024-09-21T14:58:36.489 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.489 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.489 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:36.489 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.135274+0000", 2024-09-21T14:58:36.489 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6499999999999999e-06 2024-09-21T14:58:36.489 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.490 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.490 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:36.490 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.135279+0000", 2024-09-21T14:58:36.490 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5469999999999998e-06 2024-09-21T14:58:36.490 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.490 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.490 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:36.490 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.140856+0000", 2024-09-21T14:58:36.490 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0055774350000000004 2024-09-21T14:58:36.490 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.490 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.490 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:36.491 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.140938+0000", 2024-09-21T14:58:36.491 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1963000000000001e-05 2024-09-21T14:58:36.491 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.491 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.491 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:36.491 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.141157+0000", 2024-09-21T14:58:36.491 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000219027 2024-09-21T14:58:36.491 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:36.491 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:36.491 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:36.492 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.492 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.492 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:6716 3.7 3:e2aed099:::rbd_data.11361bed1f78.000000000000000b:d [read 163840~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:36.492 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:35.135282+0000", 2024-09-21T14:58:36.492 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.2613887580000001, 2024-09-21T14:58:36.492 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0061110729999999999, 2024-09-21T14:58:36.492 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:36.492 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:36.492 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:36.492 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:36.492 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:36.493 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 6716 2024-09-21T14:58:36.493 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.493 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:36.493 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.493 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:36.493 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.135282+0000", 2024-09-21T14:58:36.493 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:36.493 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.493 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.493 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:36.493 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.135282+0000", 2024-09-21T14:58:36.494 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:36.494 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.494 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.494 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:36.494 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.135282+0000", 2024-09-21T14:58:36.494 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:36.494 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.494 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.494 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:36.494 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.135285+0000", 2024-09-21T14:58:36.494 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.667e-06 2024-09-21T14:58:36.494 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.495 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.495 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:36.495 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.135292+0000", 2024-09-21T14:58:36.495 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2169999999999997e-06 2024-09-21T14:58:36.495 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.495 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.495 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:36.495 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.135300+0000", 2024-09-21T14:58:36.495 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9410000000000007e-06 2024-09-21T14:58:36.495 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.495 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.496 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:36.496 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.141190+0000", 2024-09-21T14:58:36.496 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0058907539999999998 2024-09-21T14:58:36.496 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.496 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.496 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:36.496 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.141266+0000", 2024-09-21T14:58:36.496 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6033999999999997e-05 2024-09-21T14:58:36.496 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.496 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.497 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:36.497 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.141393+0000", 2024-09-21T14:58:36.497 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000126754 2024-09-21T14:58:36.497 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:36.497 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:36.497 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:36.497 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.497 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.497 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:6717 3.7 3:e2aed099:::rbd_data.11361bed1f78.000000000000000b:d [read 245760~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:36.498 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:35.135306+0000", 2024-09-21T14:58:36.498 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.261365243, 2024-09-21T14:58:36.498 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.006388459, 2024-09-21T14:58:36.498 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:36.498 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:36.498 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:36.498 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:36.498 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:36.498 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 6717 2024-09-21T14:58:36.499 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.499 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:36.499 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.499 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:36.499 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.135306+0000", 2024-09-21T14:58:36.499 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:36.499 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.499 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.499 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:36.499 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.135306+0000", 2024-09-21T14:58:36.500 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:36.500 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.500 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.500 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:36.500 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.135305+0000", 2024-09-21T14:58:36.500 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:36.500 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.500 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.500 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:36.500 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.135308+0000", 2024-09-21T14:58:36.500 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4839999999999998e-06 2024-09-21T14:58:36.500 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.500 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.500 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:36.500 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.135314+0000", 2024-09-21T14:58:36.501 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0580000000000003e-06 2024-09-21T14:58:36.501 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.501 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.501 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:36.501 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.135320+0000", 2024-09-21T14:58:36.501 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7239999999999997e-06 2024-09-21T14:58:36.501 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.501 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.501 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:36.501 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.141437+0000", 2024-09-21T14:58:36.501 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0061173019999999998 2024-09-21T14:58:36.501 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.502 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.502 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:36.502 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.141515+0000", 2024-09-21T14:58:36.502 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8263000000000006e-05 2024-09-21T14:58:36.502 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.502 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.502 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:36.502 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.141694+0000", 2024-09-21T14:58:36.502 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017904600000000001 2024-09-21T14:58:36.502 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:36.502 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:36.502 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:36.502 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.502 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.502 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:6792 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:head [write 0~524288 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:36.503 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:35.148607+0000", 2024-09-21T14:58:36.503 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.248064045, 2024-09-21T14:58:36.503 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0074821489999999996, 2024-09-21T14:58:36.503 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:36.503 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:36.503 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:36.503 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:36.503 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:36.503 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 6792 2024-09-21T14:58:36.503 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.503 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:36.503 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.503 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:36.503 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.148607+0000", 2024-09-21T14:58:36.504 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:36.504 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.504 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.504 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:36.504 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.148607+0000", 2024-09-21T14:58:36.504 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:36.504 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.504 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.504 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:36.504 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.148605+0000", 2024-09-21T14:58:36.504 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:36.504 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.504 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.504 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:36.504 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.149630+0000", 2024-09-21T14:58:36.505 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0010244589999999999 2024-09-21T14:58:36.505 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.505 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.505 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:36.505 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.149658+0000", 2024-09-21T14:58:36.505 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8221e-05 2024-09-21T14:58:36.505 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.505 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.505 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:36.505 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.149672+0000", 2024-09-21T14:58:36.505 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3482999999999999e-05 2024-09-21T14:58:36.505 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.505 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.505 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:36.506 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.150006+0000", 2024-09-21T14:58:36.506 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00033429199999999998 2024-09-21T14:58:36.506 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.506 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.506 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:36.506 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.150242+0000", 2024-09-21T14:58:36.506 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000235904 2024-09-21T14:58:36.506 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.506 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.506 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 2", 2024-09-21T14:58:36.506 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.150423+0000", 2024-09-21T14:58:36.506 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000181557 2024-09-21T14:58:36.506 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.506 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.506 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:58:36.507 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.153406+0000", 2024-09-21T14:58:36.507 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0029829129999999998 2024-09-21T14:58:36.507 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.507 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.507 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:36.507 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.155957+0000", 2024-09-21T14:58:36.507 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0025512249999999998 2024-09-21T14:58:36.507 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.507 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.507 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:36.507 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.156026+0000", 2024-09-21T14:58:36.507 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8174000000000002e-05 2024-09-21T14:58:36.507 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:36.507 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:36.507 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:36.508 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.156089+0000", 2024-09-21T14:58:36.508 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3429000000000005e-05 2024-09-21T14:58:36.508 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:36.508 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:36.508 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:36.508 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:36.508 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:36.508 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:58:36.509 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T14:58:36.510 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:58:36.503+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:58:36.535 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T14:58:36.556 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T14:58:36.578 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T14:58:36.598 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T14:58:36.610 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:58:36.608+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:58:36.619 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-21T14:58:36.625 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:58:36.625 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:58:36.625 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T14:58:36.625 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:58:36.629 INFO:teuthology.orchestra.run.smithi032.stdout:278 write 0xe0cf705 thru 0xe0d50ff (0x59fb bytes) 2024-09-21T14:58:36.635 INFO:teuthology.orchestra.run.smithi032.stdout:280 read 0x2fe0eb3 thru 0x2ff037c (0xf4ca bytes) 2024-09-21T14:58:36.635 INFO:teuthology.orchestra.run.smithi032.stdout:281 read 0x4f0fcba thru 0x4f125b6 (0x28fd bytes) 2024-09-21T14:58:36.635 INFO:teuthology.orchestra.run.smithi032.stdout:282 writesame 0x1065800 thru 0x106faee data_size 0x1(0xa2ef bytes) 2024-09-21T14:58:36.636 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T14:58:36.644 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T14:58:36.660 INFO:teuthology.orchestra.run.smithi032.stdout:284 punch from 0x8011ee0 to 0x801c4e7, (0xa607 bytes) 2024-09-21T14:58:36.661 INFO:teuthology.orchestra.run.smithi032.stdout:285 write 0x459e7f thru 0x45f7b8 (0x593a bytes) 2024-09-21T14:58:36.662 INFO:teuthology.orchestra.run.smithi032.stdout:287 writesame 0x8e75047 thru 0x8e7c744 data_size 0x3(0x76fe bytes) 2024-09-21T14:58:36.668 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T14:58:36.675 INFO:teuthology.orchestra.run.smithi032.stdout:288 write 0xca7dae9 thru 0xca868e4 (0x8dfc bytes) 2024-09-21T14:58:36.682 INFO:teuthology.orchestra.run.smithi032.stdout:289 write 0x19537e0 thru 0x1957957 (0x4178 bytes) 2024-09-21T14:58:36.684 INFO:teuthology.orchestra.run.smithi032.stdout:290 write 0x64daa40 thru 0x64ea1de (0xf79f bytes) 2024-09-21T14:58:36.697 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T14:58:36.699 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:58:36.699 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:58:36.699 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:58:36.699 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:58:36.700 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:58:36.700 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:58:36.700 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:58:36.700 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:58:36.700 INFO:teuthology.orchestra.run.smithi032.stdout:291 write 0xe5bda27 thru 0xe5ccb98 (0xf172 bytes) 2024-09-21T14:58:36.707 INFO:teuthology.orchestra.run.smithi032.stdout:292 punch from 0x8b34b3f to 0x8b42735, (0xdbf6 bytes) 2024-09-21T14:58:36.707 INFO:teuthology.orchestra.run.smithi032.stdout:293 read 0x561a6f thru 0x565919 (0x3eab bytes) 2024-09-21T14:58:36.708 INFO:teuthology.orchestra.run.smithi032.stdout:294 read 0x542d365 thru 0x54357ba (0x8456 bytes) 2024-09-21T14:58:36.708 INFO:teuthology.orchestra.run.smithi032.stdout:295 write 0x9bb64d5 thru 0x9bc3c7d (0xd7a9 bytes) 2024-09-21T14:58:36.711 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:58:36.707+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:58:36.714 INFO:teuthology.orchestra.run.smithi032.stdout:296 writesame 0xdce4adb thru 0xdce511c data_size 0x9(0x642 bytes) 2024-09-21T14:58:36.718 INFO:teuthology.orchestra.run.smithi032.stdout:298 write 0x9c53aa thru 0x9d328d (0xdee4 bytes) 2024-09-21T14:58:36.722 INFO:teuthology.orchestra.run.smithi032.stdout:299 read 0x22e48b4 thru 0x22ece45 (0x8592 bytes) 2024-09-21T14:58:36.722 INFO:teuthology.orchestra.run.smithi032.stdout:300 writesame 0xd859efb thru 0xd85dc5c data_size 0x9(0x3d62 bytes) 2024-09-21T14:58:36.732 INFO:teuthology.orchestra.run.smithi032.stdout:301 trunc from 0xe5ccb99 to 0xbd549db 2024-09-21T14:58:36.746 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:36.746 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:36.749 INFO:teuthology.orchestra.run.smithi032.stdout:302 read 0x562e919 thru 0x5636d08 (0x83f0 bytes) 2024-09-21T14:58:36.750 INFO:teuthology.orchestra.run.smithi032.stdout:303 writesame 0x60e4108 thru 0x60f2979 data_size 0x1(0xe872 bytes) 2024-09-21T14:58:36.756 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T14:58:36.787 INFO:teuthology.orchestra.run.smithi032.stdout:304 write 0xbb0a740 thru 0xbb0d332 (0x2bf3 bytes) 2024-09-21T14:58:36.812 INFO:teuthology.orchestra.run.smithi032.stdout:305 trunc from 0xbd549db to 0x2181049 2024-09-21T14:58:36.812 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:58:36.808+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:58:36.829 INFO:teuthology.orchestra.run.smithi032.stdout:306 read 0x4b5ba3 thru 0x4bf085 (0x94e3 bytes) 2024-09-21T14:58:36.830 INFO:teuthology.orchestra.run.smithi032.stdout:308 read 0x16a3580 thru 0x16a671d (0x319e bytes) 2024-09-21T14:58:36.831 INFO:teuthology.orchestra.run.smithi032.stdout:309 punch from 0x773c9a to 0x77d47a, (0x97e0 bytes) 2024-09-21T14:58:36.831 INFO:teuthology.orchestra.run.smithi032.stdout:310 read 0x1872f47 thru 0x187af91 (0x804b bytes) 2024-09-21T14:58:36.832 INFO:teuthology.orchestra.run.smithi032.stdout:312 write 0xc16633 thru 0xc234d7 (0xcea5 bytes) 2024-09-21T14:58:36.836 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:36.836 INFO:teuthology.orchestra.run.smithi032.stdout:314 read 0x8381bf thru 0x83e416 (0x6258 bytes) 2024-09-21T14:58:36.855 INFO:teuthology.orchestra.run.smithi032.stdout:315 write 0x561379b thru 0x5617718 (0x3f7e bytes) 2024-09-21T14:58:36.862 INFO:teuthology.orchestra.run.smithi032.stdout:316 read 0x357a4f0 thru 0x35886b2 (0xe1c3 bytes) 2024-09-21T14:58:36.862 INFO:teuthology.orchestra.run.smithi032.stdout:317 read 0xdf01d2 thru 0xdf1892 (0x16c1 bytes) 2024-09-21T14:58:36.863 INFO:teuthology.orchestra.run.smithi032.stdout:319 write 0xe35d40 thru 0xe45b9e (0xfe5f bytes) 2024-09-21T14:58:36.866 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:36.866 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:36.870 INFO:teuthology.orchestra.run.smithi032.stdout:320 read 0x51a03e thru 0x523530 (0x94f3 bytes) 2024-09-21T14:58:36.872 INFO:teuthology.orchestra.run.smithi032.stdout:321 read 0xdaac3e thru 0xdadaa1 (0x2e64 bytes) 2024-09-21T14:58:36.872 INFO:teuthology.orchestra.run.smithi032.stdout:324 read 0x27cada8 thru 0x27cbc7a (0xed3 bytes) 2024-09-21T14:58:36.872 INFO:teuthology.orchestra.run.smithi032.stdout:325 punch from 0x4595735 to 0x459b917, (0x61e2 bytes) 2024-09-21T14:58:36.876 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T14:58:36.900 INFO:teuthology.orchestra.run.smithi032.stdout:326 writesame 0xd6c7520 thru 0xd6ca9f0 data_size 0x1(0x34d1 bytes) 2024-09-21T14:58:36.900 INFO:teuthology.orchestra.run.smithi032.stdout:327 read 0xc069b45 thru 0xc072614 (0x8ad0 bytes) 2024-09-21T14:58:36.900 INFO:teuthology.orchestra.run.smithi032.stdout:329 read 0x4ce8be5 thru 0x4ce9989 (0xda5 bytes) 2024-09-21T14:58:36.913 INFO:teuthology.orchestra.run.smithi032.stdout:330 writesame 0xea5b9a2 thru 0xea5c683 data_size 0x2(0xce2 bytes) 2024-09-21T14:58:36.913 INFO:teuthology.orchestra.run.smithi032.stdout:333 punch from 0xe0e1e50 to 0xe0e3c16, (0x1dc6 bytes) 2024-09-21T14:58:36.913 INFO:teuthology.orchestra.run.smithi032.stdout:334 punch from 0xad6b897 to 0xad75951, (0xa0ba bytes) 2024-09-21T14:58:36.913 INFO:teuthology.orchestra.run.smithi032.stdout:335 write 0x54eecc8 thru 0x54fcfa6 (0xe2df bytes) 2024-09-21T14:58:36.913 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:58:36.907+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:58:36.918 INFO:teuthology.orchestra.run.smithi032.stdout:336 write 0x54f005c thru 0x54fa611 (0xa5b6 bytes) 2024-09-21T14:58:36.922 INFO:teuthology.orchestra.run.smithi032.stdout:337 read 0xbcdf55e thru 0xbce6254 (0x6cf7 bytes) 2024-09-21T14:58:36.922 INFO:teuthology.orchestra.run.smithi032.stdout:338 read 0xbd6e8e thru 0xbe2655 (0xb7c8 bytes) 2024-09-21T14:58:36.923 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:36.943 INFO:teuthology.orchestra.run.smithi032.stdout:340 trunc from 0xea5c684 to 0x47bd937 2024-09-21T14:58:36.967 INFO:teuthology.orchestra.run.smithi032.stdout:343 trunc from 0x47bd937 to 0x2a24af4 2024-09-21T14:58:36.977 INFO:teuthology.orchestra.run.smithi032.stdout:344 punch from 0x1d925f8 to 0x1d9bf69, (0x9971 bytes) 2024-09-21T14:58:36.977 INFO:teuthology.orchestra.run.smithi032.stdout:345 write 0x1f18e93 thru 0x1f2333f (0xa4ad bytes) 2024-09-21T14:58:36.984 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:36.984 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:36.985 INFO:teuthology.orchestra.run.smithi032.stdout:346 punch from 0x254633f to 0x254a2ec, (0x3fad bytes) 2024-09-21T14:58:36.985 INFO:teuthology.orchestra.run.smithi032.stdout:347 read 0xb3f54d thru 0xb46abe (0x7572 bytes) 2024-09-21T14:58:36.986 INFO:teuthology.orchestra.run.smithi032.stdout:349 read 0x1e3d9f thru 0x1f10a2 (0xd304 bytes) 2024-09-21T14:58:36.987 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:36.995 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T14:58:37.001 INFO:teuthology.orchestra.run.smithi032.stdout:352 write 0x3da6e94 thru 0x3db11f5 (0xa362 bytes) 2024-09-21T14:58:37.014 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:58:37.012+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:58:37.034 INFO:teuthology.orchestra.run.smithi032.stdout:353 writesame 0xb122a61 thru 0xb129341 data_size 0x1(0x68e1 bytes) 2024-09-21T14:58:37.035 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:37.035 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:37.035 INFO:teuthology.orchestra.run.smithi032.stdout:357 writesame 0x5320b13 thru 0x532ee7d data_size 0x1(0xe36b bytes) 2024-09-21T14:58:37.087 INFO:teuthology.orchestra.run.smithi032.stdout:358 writesame 0xcce7cd8 thru 0xccf3d1a data_size 0x1(0xc043 bytes) 2024-09-21T14:58:37.087 INFO:teuthology.orchestra.run.smithi032.stdout:359 write 0x3d085e4 thru 0x3d17283 (0xeca0 bytes) 2024-09-21T14:58:37.093 INFO:teuthology.orchestra.run.smithi032.stdout:362 write 0x94ef8f0 thru 0x94f833c (0x8a4d bytes) 2024-09-21T14:58:37.099 INFO:teuthology.orchestra.run.smithi032.stdout:363 write 0x800c561 thru 0x801278b (0x622b bytes) 2024-09-21T14:58:37.107 INFO:teuthology.orchestra.run.smithi032.stdout:364 punch from 0x4dff81b to 0x4e05535, (0x5d1a bytes) 2024-09-21T14:58:37.107 INFO:teuthology.orchestra.run.smithi032.stdout:365 read 0xc2d6b74 thru 0xc2dbcd5 (0x5162 bytes) 2024-09-21T14:58:37.107 INFO:teuthology.orchestra.run.smithi032.stdout:366 read 0x8c0ccc thru 0x8cadf4 (0xa129 bytes) 2024-09-21T14:58:37.108 INFO:teuthology.orchestra.run.smithi032.stdout:367 writesame 0x23d351b thru 0x23d8a8a data_size 0x8(0x5570 bytes) 2024-09-21T14:58:37.111 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:37.111 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:37.115 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:58:37.112+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:58:37.115 INFO:teuthology.orchestra.run.smithi032.stdout:368 write 0x4ddc7e9 thru 0x4ddffac (0x37c4 bytes) 2024-09-21T14:58:37.122 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T14:58:37.122 INFO:teuthology.orchestra.run.smithi032.stdout:369 write 0x3f1bb97 thru 0x3f28517 (0xc981 bytes) 2024-09-21T14:58:37.128 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T14:58:37.130 INFO:teuthology.orchestra.run.smithi032.stdout:370 punch from 0x34432ed to 0x344913a, (0x5e4d bytes) 2024-09-21T14:58:37.131 INFO:teuthology.orchestra.run.smithi032.stdout:371 write 0x8ef275f thru 0x8f01fbc (0xf85e bytes) 2024-09-21T14:58:37.138 INFO:teuthology.orchestra.run.smithi032.stdout:373 punch from 0x49b6364 to 0x49b8a23, (0x26bf bytes) 2024-09-21T14:58:37.138 INFO:teuthology.orchestra.run.smithi032.stdout:374 write 0x9ac0fbf thru 0x9ad0d1c (0xfd5e bytes) 2024-09-21T14:58:37.145 INFO:teuthology.orchestra.run.smithi032.stdout:375 read 0x51b93bc thru 0x51c0eb6 (0x7afb bytes) 2024-09-21T14:58:37.145 INFO:teuthology.orchestra.run.smithi032.stdout:376 read 0xb2e4354 thru 0xb2eb6d9 (0x7386 bytes) 2024-09-21T14:58:37.145 INFO:teuthology.orchestra.run.smithi032.stdout:378 write 0x7540d70 thru 0x754c6e4 (0xb975 bytes) 2024-09-21T14:58:37.151 INFO:teuthology.orchestra.run.smithi032.stdout:380 read 0xb5babae thru 0xb5c7444 (0xc897 bytes) 2024-09-21T14:58:37.151 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:37.151 INFO:teuthology.orchestra.run.smithi032.stdout:382 punch from 0x4dd3aa6 to 0x4dd8f9a, (0x54f4 bytes) 2024-09-21T14:58:37.151 INFO:teuthology.orchestra.run.smithi032.stdout:384 write 0x541c7f3 thru 0x54264be (0x9ccc bytes) 2024-09-21T14:58:37.157 INFO:teuthology.orchestra.run.smithi032.stdout:385 write 0xb65c55f thru 0xb665f17 (0x99b9 bytes) 2024-09-21T14:58:37.163 INFO:teuthology.orchestra.run.smithi032.stdout:387 writesame 0x19cebd3 thru 0x19d8b7a data_size 0x68(0x9fa8 bytes) 2024-09-21T14:58:37.166 INFO:teuthology.orchestra.run.smithi032.stdout:388 read 0x7063ed9 thru 0x706b362 (0x748a bytes) 2024-09-21T14:58:37.166 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:37.166 INFO:teuthology.orchestra.run.smithi032.stdout:392 write 0x66dd501 thru 0x66e2cd7 (0x57d7 bytes) 2024-09-21T14:58:37.173 INFO:teuthology.orchestra.run.smithi032.stdout:395 write 0x61ac9bf thru 0x61ba137 (0xd779 bytes) 2024-09-21T14:58:37.179 INFO:teuthology.orchestra.run.smithi032.stdout:396 writesame 0x21ae1d0 thru 0x21b20e7 data_size 0xc(0x3f18 bytes) 2024-09-21T14:58:37.185 INFO:teuthology.orchestra.run.smithi032.stdout:398 write 0x7ad1a1 thru 0x7b2881 (0x56e1 bytes) 2024-09-21T14:58:37.188 INFO:teuthology.orchestra.run.smithi032.stdout:399 write 0x3f3232a thru 0x3f3d94c (0xb623 bytes) 2024-09-21T14:58:37.192 INFO:teuthology.orchestra.run.smithi032.stdout:400 writesame 0x31cf835 thru 0x31d8a96 data_size 0x2(0x9262 bytes) 2024-09-21T14:58:37.210 INFO:teuthology.orchestra.run.smithi032.stdout:401 read 0xa9b77a5 thru 0xa9bf195 (0x79f1 bytes) 2024-09-21T14:58:37.211 INFO:teuthology.orchestra.run.smithi032.stdout:402 punch from 0xa4ade9d to 0xa4ae0f8, (0x25b bytes) 2024-09-21T14:58:37.211 INFO:teuthology.orchestra.run.smithi032.stdout:406 write 0xc8e2f74 thru 0xc8ee5c9 (0xb656 bytes) 2024-09-21T14:58:37.216 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:58:37.212+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:58:37.217 INFO:teuthology.orchestra.run.smithi032.stdout:407 write 0x779667d thru 0x77966a1 (0x25 bytes) 2024-09-21T14:58:37.222 INFO:teuthology.orchestra.run.smithi032.stdout:408 writesame 0x88e9a71 thru 0x88f20f2 data_size 0x9(0x8682 bytes) 2024-09-21T14:58:37.228 INFO:teuthology.orchestra.run.smithi032.stdout:409 read 0x7a07d1e thru 0x7a0e026 (0x6309 bytes) 2024-09-21T14:58:37.241 INFO:teuthology.orchestra.run.smithi032.stdout:410 writesame 0xee390ff thru 0xee3b914 data_size 0x7(0x2816 bytes) 2024-09-21T14:58:37.242 INFO:teuthology.orchestra.run.smithi032.stdout:411 write 0x8c4dd32 thru 0x8c5da9c (0xfd6b bytes) 2024-09-21T14:58:37.248 INFO:teuthology.orchestra.run.smithi032.stdout:412 write 0x6dace95 thru 0x6db769d (0xa809 bytes) 2024-09-21T14:58:37.252 INFO:teuthology.orchestra.run.smithi032.stdout:413 write 0x845d572 thru 0x8465ec8 (0x8957 bytes) 2024-09-21T14:58:37.259 INFO:teuthology.orchestra.run.smithi032.stdout:415 write 0xae43e6a thru 0xae49f4a (0x60e1 bytes) 2024-09-21T14:58:37.264 INFO:teuthology.orchestra.run.smithi032.stdout:417 punch from 0x8ff96cf to 0x8ffa13a, (0xa6b bytes) 2024-09-21T14:58:37.264 INFO:teuthology.orchestra.run.smithi032.stdout:418 read 0xd75714b thru 0xd758223 (0x10d9 bytes) 2024-09-21T14:58:37.264 INFO:teuthology.orchestra.run.smithi032.stdout:419 write 0x57acf6d thru 0x57aeff2 (0x2086 bytes) 2024-09-21T14:58:37.268 INFO:teuthology.orchestra.run.smithi032.stdout:420 punch from 0x893c19f to 0x894ace4, (0xeb45 bytes) 2024-09-21T14:58:37.268 INFO:teuthology.orchestra.run.smithi032.stdout:424 write 0xc767849 thru 0xc768eb1 (0x1669 bytes) 2024-09-21T14:58:37.272 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:37.272 INFO:teuthology.orchestra.run.smithi032.stdout:427 read 0x834aab5 thru 0x8359814 (0xed60 bytes) 2024-09-21T14:58:37.291 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:37.291 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:37.291 INFO:teuthology.orchestra.run.smithi032.stdout:429 trunc from 0xee3b915 to 0x666e5d8 2024-09-21T14:58:37.302 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T14:58:37.312 INFO:teuthology.orchestra.run.smithi032.stdout:431 punch from 0x591ca29 to 0x591ffd4, (0x35ab bytes) 2024-09-21T14:58:37.312 INFO:teuthology.orchestra.run.smithi032.stdout:432 read 0x56d1d74 thru 0x56dcf94 (0xb221 bytes) 2024-09-21T14:58:37.317 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:58:37.312+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:58:37.335 INFO:teuthology.orchestra.run.smithi032.stdout:433 write 0xe376af6 thru 0xe3860e5 (0xf5f0 bytes) 2024-09-21T14:58:37.343 INFO:teuthology.orchestra.run.smithi032.stdout:436 read 0x1e4f86e thru 0x1e58ae1 (0x9274 bytes) 2024-09-21T14:58:37.343 INFO:teuthology.orchestra.run.smithi032.stdout:439 read 0x93e3744 thru 0x93e579b (0x2058 bytes) 2024-09-21T14:58:37.343 INFO:teuthology.orchestra.run.smithi032.stdout:440 read 0x628a46 thru 0x637009 (0xe5c4 bytes) 2024-09-21T14:58:37.344 INFO:teuthology.orchestra.run.smithi032.stdout:441 read 0x7373e33 thru 0x738368e (0xf85c bytes) 2024-09-21T14:58:37.344 INFO:teuthology.orchestra.run.smithi032.stdout:442 writesame 0x3076aeb thru 0x308669b data_size 0x1(0xfbb1 bytes) 2024-09-21T14:58:37.373 INFO:teuthology.orchestra.run.smithi032.stdout:443 punch from 0x48f611a to 0x48fd2dd, (0x71c3 bytes) 2024-09-21T14:58:37.374 INFO:teuthology.orchestra.run.smithi032.stdout:444 punch from 0x835a871 to 0x835dc26, (0x33b5 bytes) 2024-09-21T14:58:37.374 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:37.374 INFO:teuthology.orchestra.run.smithi032.stdout:448 punch from 0x16cc9d9 to 0x16dbb37, (0xf15e bytes) 2024-09-21T14:58:37.374 INFO:teuthology.orchestra.run.smithi032.stdout:449 read 0x1d19c9b thru 0x1d1f894 (0x5bfa bytes) 2024-09-21T14:58:37.374 INFO:teuthology.orchestra.run.smithi032.stdout:450 read 0x12c6019 thru 0x12d1f85 (0xbf6d bytes) 2024-09-21T14:58:37.374 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:37.374 INFO:teuthology.orchestra.run.smithi032.stdout:452 write 0x2184406 thru 0x218b893 (0x748e bytes) 2024-09-21T14:58:37.379 INFO:teuthology.orchestra.run.smithi032.stdout:453 read 0xd32ed87 thru 0xd3350fb (0x6375 bytes) 2024-09-21T14:58:37.379 INFO:teuthology.orchestra.run.smithi032.stdout:455 writesame 0xcdf7520 thru 0xce05f23 data_size 0x22(0xea04 bytes) 2024-09-21T14:58:37.385 INFO:teuthology.orchestra.run.smithi032.stdout:456 write 0xa31b276 thru 0xa3237f4 (0x857f bytes) 2024-09-21T14:58:37.392 INFO:teuthology.orchestra.run.smithi032.stdout:457 write 0x72518e5 thru 0x725b54b (0x9c67 bytes) 2024-09-21T14:58:37.396 INFO:teuthology.orchestra.run.smithi032.stdout:459 write 0xddedf7b thru 0xddf0d33 (0x2db9 bytes) 2024-09-21T14:58:37.401 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:37.402 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:37.412 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T14:58:37.418 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:58:37.412+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:58:37.420 INFO:teuthology.orchestra.run.smithi032.stdout:460 trunc from 0xe3860e6 to 0x1e27f76 2024-09-21T14:58:37.442 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:37.447 INFO:teuthology.orchestra.run.smithi032.stdout:463 trunc from 0x1e27f76 to 0x548a0e4 2024-09-21T14:58:37.455 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:37.455 INFO:teuthology.orchestra.run.smithi032.stdout:465 read 0x3580c38 thru 0x3590701 (0xfaca bytes) 2024-09-21T14:58:37.455 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:37.455 INFO:teuthology.orchestra.run.smithi032.stdout:467 read 0x11bca7e thru 0x11c6a51 (0x9fd4 bytes) 2024-09-21T14:58:37.455 INFO:teuthology.orchestra.run.smithi032.stdout:468 read 0x4c86be thru 0x4d7e04 (0xf747 bytes) 2024-09-21T14:58:37.456 INFO:teuthology.orchestra.run.smithi032.stdout:469 write 0x11e285b thru 0x11e4204 (0x19aa bytes) 2024-09-21T14:58:37.480 INFO:teuthology.orchestra.run.smithi032.stdout:470 write 0xc4da385 thru 0xc4dc687 (0x2303 bytes) 2024-09-21T14:58:37.485 INFO:teuthology.orchestra.run.smithi032.stdout:472 read 0x609e0cc thru 0x60a148e (0x33c3 bytes) 2024-09-21T14:58:37.486 INFO:teuthology.orchestra.run.smithi032.stdout:473 punch from 0x633f11 to 0x6382a4, (0x4393 bytes) 2024-09-21T14:58:37.486 INFO:teuthology.orchestra.run.smithi032.stdout:474 punch from 0xdefc5 to 0xeead1, (0xfb0c bytes) 2024-09-21T14:58:37.486 INFO:teuthology.orchestra.run.smithi032.stdout:475 read 0x42f3bdc thru 0x43004f9 (0xc91e bytes) 2024-09-21T14:58:37.486 INFO:teuthology.orchestra.run.smithi032.stdout:476 write 0x5095054 thru 0x509d541 (0x84ee bytes) 2024-09-21T14:58:37.491 INFO:teuthology.orchestra.run.smithi032.stdout:477 punch from 0x254f9a0 to 0x2559cd0, (0xa330 bytes) 2024-09-21T14:58:37.491 INFO:teuthology.orchestra.run.smithi032.stdout:478 write 0x592d776 thru 0x5935382 (0x7c0d bytes) 2024-09-21T14:58:37.496 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:37.497 INFO:teuthology.orchestra.run.smithi032.stdout:480 write 0x2c948a4 thru 0x2ca2345 (0xdaa2 bytes) 2024-09-21T14:58:37.503 INFO:teuthology.orchestra.run.smithi032.stdout:482 punch from 0x8ed721d to 0x8edbf80, (0x4d63 bytes) 2024-09-21T14:58:37.504 INFO:teuthology.orchestra.run.smithi032.stdout:483 writesame 0x99f01b7 thru 0x99f3c02 data_size 0x4(0x3a4c bytes) 2024-09-21T14:58:37.513 INFO:teuthology.orchestra.run.smithi032.stdout:484 punch from 0x991fd58 to 0x992bb64, (0xbe0c bytes) 2024-09-21T14:58:37.518 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:58:37.516+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:58:37.524 INFO:teuthology.orchestra.run.smithi032.stdout:486 trunc from 0xc4dc688 to 0x6e70f00 2024-09-21T14:58:37.524 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:37.524 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:37.535 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T14:58:37.540 INFO:teuthology.orchestra.run.smithi032.stdout:487 read 0x55dad20 thru 0x55de950 (0x3c31 bytes) 2024-09-21T14:58:37.540 INFO:teuthology.orchestra.run.smithi032.stdout:488 write 0x34704ba thru 0x34785dd (0x8124 bytes) 2024-09-21T14:58:37.546 INFO:teuthology.orchestra.run.smithi032.stdout:489 read 0x37df27c thru 0x37ef14b (0xfed0 bytes) 2024-09-21T14:58:37.547 INFO:teuthology.orchestra.run.smithi032.stdout:490 read 0x16f9cde thru 0x170346b (0x978e bytes) 2024-09-21T14:58:37.548 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:37.570 INFO:teuthology.orchestra.run.smithi032.stdout:492 write 0xd084e77 thru 0xd09096b (0xbaf5 bytes) 2024-09-21T14:58:37.575 INFO:teuthology.orchestra.run.smithi032.stdout:493 read 0x2f89729 thru 0x2f90dec (0x76c4 bytes) 2024-09-21T14:58:37.576 INFO:teuthology.orchestra.run.smithi032.stdout:494 read 0x8415a20 thru 0x841ac34 (0x5215 bytes) 2024-09-21T14:58:37.576 INFO:teuthology.orchestra.run.smithi032.stdout:497 writesame 0x67f6e2c thru 0x68017b0 data_size 0x1(0xa985 bytes) 2024-09-21T14:58:37.614 INFO:teuthology.orchestra.run.smithi032.stdout:498 writesame 0x34bbd8f thru 0x34c3327 data_size 0x2d(0x7599 bytes) 2024-09-21T14:58:37.619 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:58:37.616+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:58:37.624 INFO:teuthology.orchestra.run.smithi032.stdout:500 read 0x504abac thru 0x504fd92 (0x51e7 bytes) 2024-09-21T14:58:37.630 INFO:teuthology.orchestra.run.smithi032.stdout:501 trunc from 0xd09096c to 0xadd5521 2024-09-21T14:58:37.645 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:37.645 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:37.649 INFO:teuthology.orchestra.run.smithi032.stdout:502 read 0x9900be1 thru 0x990fcce (0xf0ee bytes) 2024-09-21T14:58:37.652 INFO:teuthology.orchestra.run.smithi032.stdout:503 trunc from 0xadd5521 to 0x9d398fa 2024-09-21T14:58:37.655 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T14:58:37.679 INFO:teuthology.orchestra.run.smithi032.stdout:504 write 0xd478573 thru 0xd481a74 (0x9502 bytes) 2024-09-21T14:58:37.687 INFO:teuthology.orchestra.run.smithi032.stdout:505 punch from 0xb92371a to 0xb92b71e, (0x8004 bytes) 2024-09-21T14:58:37.687 INFO:teuthology.orchestra.run.smithi032.stdout:507 read 0x2c612e9 thru 0x2c6a08a (0x8da2 bytes) 2024-09-21T14:58:37.687 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:37.687 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:37.687 INFO:teuthology.orchestra.run.smithi032.stdout:512 write 0x581e493 thru 0x582a9b7 (0xc525 bytes) 2024-09-21T14:58:37.696 INFO:teuthology.orchestra.run.smithi032.stdout:513 trunc from 0xd481a75 to 0xb73d7a8 2024-09-21T14:58:37.714 INFO:teuthology.orchestra.run.smithi032.stdout:514 punch from 0x6886091 to 0x689119b, (0xb10a bytes) 2024-09-21T14:58:37.714 INFO:teuthology.orchestra.run.smithi032.stdout:515 punch from 0x72954ed to 0x729641f, (0xf32 bytes) 2024-09-21T14:58:37.714 INFO:teuthology.orchestra.run.smithi032.stdout:516 read 0x3db5488 thru 0x3db7c4b (0x27c4 bytes) 2024-09-21T14:58:37.714 INFO:teuthology.orchestra.run.smithi032.stdout:517 clone 5 order 18 su 8192 sc 13 2024-09-21T14:58:37.720 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:58:37.716+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:58:37.772 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:37.773 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:37.782 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T14:58:37.822 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:58:37.820+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:58:37.923 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:58:37.920+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:58:37.939 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:37.939 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:37.949 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T14:58:38.024 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:58:38.020+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:58:38.057 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:38.057 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:38.066 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T14:58:38.118 INFO:teuthology.orchestra.run.smithi032.stderr:nodeep-scrub is unset 2024-09-21T14:58:38.125 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:58:38.124+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:58:38.184 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:38.184 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:38.194 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T14:58:38.226 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:58:38.224+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:58:38.296 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:38.296 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:38.306 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T14:58:38.326 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:58:38.324+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:58:38.416 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:38.416 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:38.426 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T14:58:38.427 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:58:38.420+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:58:38.527 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:58:38.524+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:58:38.536 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:38.536 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:38.546 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T14:58:38.624 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T14:58:38.625 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T14:58:38.625 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-21T14:58:38.625 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-21T14:58:38.628 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:58:38.624+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:58:38.701 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:38.701 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:38.712 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T14:58:38.715 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T14:58:38.763 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:58:38.760+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:58:38.812 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:38.812 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:38.822 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T14:58:38.829 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:58:38.824+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:58:38.851 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T14:58:38.876 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T14:58:38.894 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T14:58:38.914 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T14:58:38.918 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:58:38.918 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:58:38.918 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T14:58:38.918 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:58:38.928 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T14:58:38.930 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:58:38.928+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:58:38.942 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-21T14:58:38.942 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:58:38.943 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 43, 2024-09-21T14:58:38.943 INFO:teuthology.orchestra.run.smithi032.stdout: "fsid": "caf51e55-9234-4a26-bcd2-d60811962560", 2024-09-21T14:58:38.943 INFO:teuthology.orchestra.run.smithi032.stdout: "created": "2024-09-21T14:57:50.338822+0000", 2024-09-21T14:58:38.943 INFO:teuthology.orchestra.run.smithi032.stdout: "modified": "2024-09-21T14:58:38.100916+0000", 2024-09-21T14:58:38.943 INFO:teuthology.orchestra.run.smithi032.stdout: "last_up_change": "2024-09-21T14:58:01.177141+0000", 2024-09-21T14:58:38.943 INFO:teuthology.orchestra.run.smithi032.stdout: "last_in_change": "2024-09-21T14:57:52.442036+0000", 2024-09-21T14:58:38.943 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-21T14:58:38.943 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_num": 5799936, 2024-09-21T14:58:38.943 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_set": [ 2024-09-21T14:58:38.943 INFO:teuthology.orchestra.run.smithi032.stdout: "pglog_hardlimit", 2024-09-21T14:58:38.943 INFO:teuthology.orchestra.run.smithi032.stdout: "purged_snapdirs", 2024-09-21T14:58:38.943 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_deletes", 2024-09-21T14:58:38.943 INFO:teuthology.orchestra.run.smithi032.stdout: "sortbitwise" 2024-09-21T14:58:38.943 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-21T14:58:38.944 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_version": 5, 2024-09-21T14:58:38.944 INFO:teuthology.orchestra.run.smithi032.stdout: "full_ratio": 0.94999998807907104, 2024-09-21T14:58:38.944 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-21T14:58:38.944 INFO:teuthology.orchestra.run.smithi032.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-21T14:58:38.944 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_snapshot": "", 2024-09-21T14:58:38.944 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_max": 3, 2024-09-21T14:58:38.944 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd": 8, 2024-09-21T14:58:38.944 INFO:teuthology.orchestra.run.smithi032.stdout: "require_min_compat_client": "luminous", 2024-09-21T14:58:38.944 INFO:teuthology.orchestra.run.smithi032.stdout: "min_compat_client": "jewel", 2024-09-21T14:58:38.944 INFO:teuthology.orchestra.run.smithi032.stdout: "require_osd_release": "reef", 2024-09-21T14:58:38.944 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_crimson": false, 2024-09-21T14:58:38.944 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2024-09-21T14:58:38.944 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:38.944 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 1, 2024-09-21T14:58:38.945 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": ".mgr", 2024-09-21T14:58:38.945 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-09-21T14:58:01.842810+0000", 2024-09-21T14:58:38.945 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-09-21T14:58:38.945 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2024-09-21T14:58:38.945 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-09-21T14:58:38.945 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-09-21T14:58:38.945 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-09-21T14:58:38.945 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-09-21T14:58:38.945 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-09-21T14:58:38.945 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-09-21T14:58:38.945 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T14:58:38.945 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T14:58:38.945 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-09-21T14:58:38.946 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-09-21T14:58:38.946 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 1, 2024-09-21T14:58:38.946 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 1, 2024-09-21T14:58:38.946 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 1, 2024-09-21T14:58:38.946 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 1, 2024-09-21T14:58:38.946 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 1, 2024-09-21T14:58:38.946 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-09-21T14:58:38.946 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-09-21T14:58:38.946 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-09-21T14:58:38.946 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-09-21T14:58:38.946 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-09-21T14:58:38.946 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-09-21T14:58:38.947 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-09-21T14:58:38.947 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:38.947 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "18", 2024-09-21T14:58:38.947 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-09-21T14:58:38.947 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T14:58:38.947 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T14:58:38.947 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-09-21T14:58:38.947 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-09-21T14:58:38.947 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-09-21T14:58:38.947 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-09-21T14:58:38.947 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-09-21T14:58:38.947 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-09-21T14:58:38.947 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-09-21T14:58:38.947 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-09-21T14:58:38.948 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-09-21T14:58:38.948 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-09-21T14:58:38.948 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-09-21T14:58:38.948 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-09-21T14:58:38.948 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-09-21T14:58:38.948 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-09-21T14:58:38.948 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-09-21T14:58:38.948 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T14:58:38.948 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T14:58:38.948 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T14:58:38.948 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-09-21T14:58:38.949 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-09-21T14:58:38.949 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-09-21T14:58:38.949 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-09-21T14:58:38.949 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-09-21T14:58:38.949 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:38.949 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-09-21T14:58:38.949 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-09-21T14:58:38.949 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-09-21T14:58:38.949 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-09-21T14:58:38.949 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-09-21T14:58:38.949 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T14:58:38.949 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-09-21T14:58:38.949 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-09-21T14:58:38.949 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-09-21T14:58:38.950 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-09-21T14:58:38.950 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-09-21T14:58:38.950 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-09-21T14:58:38.950 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_max": 32, 2024-09-21T14:58:38.950 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 1 2024-09-21T14:58:38.950 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:38.950 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-09-21T14:58:38.950 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": {} 2024-09-21T14:58:38.950 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:38.950 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-09-21T14:58:38.950 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 8, 2024-09-21T14:58:38.950 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 8, 2024-09-21T14:58:38.950 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 0.25, 2024-09-21T14:58:38.951 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 2, 2024-09-21T14:58:38.951 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 2, 2024-09-21T14:58:38.951 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-09-21T14:58:38.951 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-09-21T14:58:38.951 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-09-21T14:58:38.951 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:38.951 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:38.951 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:38.951 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 2, 2024-09-21T14:58:38.951 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "rbd", 2024-09-21T14:58:38.951 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-09-21T14:58:08.284925+0000", 2024-09-21T14:58:38.951 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 8193, 2024-09-21T14:58:38.951 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T14:58:38.951 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-09-21T14:58:38.951 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-09-21T14:58:38.952 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-09-21T14:58:38.952 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-09-21T14:58:38.952 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-09-21T14:58:38.952 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-09-21T14:58:38.952 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T14:58:38.952 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T14:58:38.952 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-09-21T14:58:38.952 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-09-21T14:58:38.952 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2024-09-21T14:58:38.952 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2024-09-21T14:58:38.952 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2024-09-21T14:58:38.952 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2024-09-21T14:58:38.952 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2024-09-21T14:58:38.952 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-09-21T14:58:38.952 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-09-21T14:58:38.953 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-09-21T14:58:38.953 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-09-21T14:58:38.953 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-09-21T14:58:38.953 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-09-21T14:58:38.953 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-09-21T14:58:38.953 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:38.953 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "22", 2024-09-21T14:58:38.953 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-09-21T14:58:38.953 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T14:58:38.953 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T14:58:38.953 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-09-21T14:58:38.953 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-09-21T14:58:38.953 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 3, 2024-09-21T14:58:38.953 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 22, 2024-09-21T14:58:38.954 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-09-21T14:58:38.954 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-09-21T14:58:38.954 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-09-21T14:58:38.954 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-09-21T14:58:38.954 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-09-21T14:58:38.954 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-09-21T14:58:38.954 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-09-21T14:58:38.954 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-09-21T14:58:38.954 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-09-21T14:58:38.954 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-09-21T14:58:38.954 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-09-21T14:58:38.954 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T14:58:38.954 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T14:58:38.954 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T14:58:38.955 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-09-21T14:58:38.955 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-09-21T14:58:38.955 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-09-21T14:58:38.955 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-09-21T14:58:38.955 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-09-21T14:58:38.955 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:38.955 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-09-21T14:58:38.955 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-09-21T14:58:38.955 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-09-21T14:58:38.955 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-09-21T14:58:38.955 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-09-21T14:58:38.955 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T14:58:38.955 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-09-21T14:58:38.955 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-09-21T14:58:38.956 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-09-21T14:58:38.956 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-09-21T14:58:38.956 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-09-21T14:58:38.956 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2024-09-21T14:58:38.956 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-09-21T14:58:38.956 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd": {} 2024-09-21T14:58:38.956 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:38.956 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-09-21T14:58:38.956 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 4.9800000190734863, 2024-09-21T14:58:38.956 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 4.9800000190734863, 2024-09-21T14:58:38.956 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 0.87999999523162842, 2024-09-21T14:58:38.956 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 4.380000114440918, 2024-09-21T14:58:38.956 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 4.380000114440918, 2024-09-21T14:58:38.956 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-09-21T14:58:38.956 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-09-21T14:58:38.957 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-09-21T14:58:38.957 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:38.957 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:38.957 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:38.957 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 3, 2024-09-21T14:58:38.957 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "pool_client.0", 2024-09-21T14:58:38.957 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-09-21T14:58:17.510709+0000", 2024-09-21T14:58:38.957 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 8193, 2024-09-21T14:58:38.957 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T14:58:38.957 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-09-21T14:58:38.957 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-09-21T14:58:38.957 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-09-21T14:58:38.957 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-09-21T14:58:38.957 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-09-21T14:58:38.957 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-09-21T14:58:38.958 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T14:58:38.958 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T14:58:38.958 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-09-21T14:58:38.958 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-09-21T14:58:38.958 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 32, 2024-09-21T14:58:38.958 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 32, 2024-09-21T14:58:38.958 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 32, 2024-09-21T14:58:38.958 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 32, 2024-09-21T14:58:38.958 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 32, 2024-09-21T14:58:38.958 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-09-21T14:58:38.958 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-09-21T14:58:38.958 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-09-21T14:58:38.958 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-09-21T14:58:38.958 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-09-21T14:58:38.959 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-09-21T14:58:38.959 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-09-21T14:58:38.959 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:38.959 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "43", 2024-09-21T14:58:38.959 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-09-21T14:58:38.959 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T14:58:38.959 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T14:58:38.959 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-09-21T14:58:38.959 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-09-21T14:58:39.039 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 16, 2024-09-21T14:58:39.039 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 43, 2024-09-21T14:58:39.039 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-09-21T14:58:39.039 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-09-21T14:58:39.039 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-09-21T14:58:39.039 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-09-21T14:58:39.039 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-09-21T14:58:39.040 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-09-21T14:58:39.040 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-09-21T14:58:39.040 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-09-21T14:58:39.040 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-09-21T14:58:39.040 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-09-21T14:58:39.040 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-09-21T14:58:39.040 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T14:58:39.040 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T14:58:39.040 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T14:58:39.040 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-09-21T14:58:39.040 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-09-21T14:58:39.040 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-09-21T14:58:39.040 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-09-21T14:58:39.040 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-09-21T14:58:39.041 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.041 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-09-21T14:58:39.041 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-09-21T14:58:39.041 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-09-21T14:58:39.041 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-09-21T14:58:39.041 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-09-21T14:58:39.041 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T14:58:39.041 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-09-21T14:58:39.041 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-09-21T14:58:39.041 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-09-21T14:58:39.041 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-09-21T14:58:39.041 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-09-21T14:58:39.041 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2024-09-21T14:58:39.041 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-09-21T14:58:39.041 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd": {} 2024-09-21T14:58:39.042 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.042 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-09-21T14:58:39.042 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 1.75, 2024-09-21T14:58:39.042 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 1.75, 2024-09-21T14:58:39.042 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 1, 2024-09-21T14:58:39.042 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 1.75, 2024-09-21T14:58:39.042 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 1.75, 2024-09-21T14:58:39.042 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2024-09-21T14:58:39.042 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2024-09-21T14:58:39.042 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2024-09-21T14:58:39.042 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:39.042 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:39.042 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-21T14:58:39.043 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2024-09-21T14:58:39.043 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.043 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-09-21T14:58:39.043 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "a4a73316-3a94-42e9-9a51-deb46a66bb06", 2024-09-21T14:58:39.043 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-21T14:58:39.043 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-21T14:58:39.043 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-21T14:58:39.043 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-21T14:58:39.043 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-21T14:58:39.043 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-21T14:58:39.043 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2024-09-21T14:58:39.043 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 24, 2024-09-21T14:58:39.043 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-21T14:58:39.043 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-21T14:58:39.044 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-21T14:58:39.044 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:39.044 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.044 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:39.044 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6808", 2024-09-21T14:58:39.044 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4016672243 2024-09-21T14:58:39.044 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.044 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.044 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:39.044 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6809", 2024-09-21T14:58:39.044 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4016672243 2024-09-21T14:58:39.044 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:39.045 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:39.045 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.045 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-21T14:58:39.045 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:39.045 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.045 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:39.045 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6810", 2024-09-21T14:58:39.045 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4016672243 2024-09-21T14:58:39.045 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.046 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.046 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:39.046 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6811", 2024-09-21T14:58:39.046 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4016672243 2024-09-21T14:58:39.046 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:39.046 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:39.046 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.046 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-21T14:58:39.046 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:39.046 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.046 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:39.046 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6814", 2024-09-21T14:58:39.047 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4016672243 2024-09-21T14:58:39.047 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.047 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.047 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:39.047 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6815", 2024-09-21T14:58:39.047 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4016672243 2024-09-21T14:58:39.047 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:39.047 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:39.047 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.047 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-21T14:58:39.047 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:39.047 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.047 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:39.047 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6812", 2024-09-21T14:58:39.048 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4016672243 2024-09-21T14:58:39.048 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.048 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.048 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:39.048 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6813", 2024-09-21T14:58:39.048 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4016672243 2024-09-21T14:58:39.048 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:39.048 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:39.048 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.048 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6809/4016672243", 2024-09-21T14:58:39.048 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6811/4016672243", 2024-09-21T14:58:39.048 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6815/4016672243", 2024-09-21T14:58:39.048 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6813/4016672243", 2024-09-21T14:58:39.048 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-21T14:58:39.048 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-21T14:58:39.049 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-21T14:58:39.049 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:39.049 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.049 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.049 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-09-21T14:58:39.049 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "006757a0-94d1-4a9b-aef8-73634cc5828e", 2024-09-21T14:58:39.049 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-21T14:58:39.049 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-21T14:58:39.049 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-21T14:58:39.049 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-21T14:58:39.049 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-21T14:58:39.049 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-21T14:58:39.049 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2024-09-21T14:58:39.049 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 24, 2024-09-21T14:58:39.049 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-21T14:58:39.050 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-21T14:58:39.050 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-21T14:58:39.050 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:39.050 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.050 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:39.050 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6824", 2024-09-21T14:58:39.050 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 641090577 2024-09-21T14:58:39.050 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.050 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.050 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:39.050 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6825", 2024-09-21T14:58:39.050 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 641090577 2024-09-21T14:58:39.050 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:39.050 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:39.050 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.051 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-21T14:58:39.051 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:39.051 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.051 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:39.051 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6826", 2024-09-21T14:58:39.051 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 641090577 2024-09-21T14:58:39.051 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.051 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.051 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:39.051 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6827", 2024-09-21T14:58:39.051 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 641090577 2024-09-21T14:58:39.051 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:39.051 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:39.051 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.051 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-21T14:58:39.052 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:39.052 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.052 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:39.052 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6830", 2024-09-21T14:58:39.052 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 641090577 2024-09-21T14:58:39.052 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.052 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.052 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:39.052 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6831", 2024-09-21T14:58:39.052 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 641090577 2024-09-21T14:58:39.052 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:39.052 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:39.052 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.052 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-21T14:58:39.053 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:39.053 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.053 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:39.053 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6828", 2024-09-21T14:58:39.053 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 641090577 2024-09-21T14:58:39.053 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.053 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.053 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:39.053 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6829", 2024-09-21T14:58:39.053 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 641090577 2024-09-21T14:58:39.053 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:39.053 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:39.053 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.053 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6825/641090577", 2024-09-21T14:58:39.053 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6827/641090577", 2024-09-21T14:58:39.053 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6831/641090577", 2024-09-21T14:58:39.054 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6829/641090577", 2024-09-21T14:58:39.054 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-21T14:58:39.054 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-21T14:58:39.054 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-21T14:58:39.054 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:39.054 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.054 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.054 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-09-21T14:58:39.054 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "0b499a23-820e-4e6c-a558-adbf88b7014f", 2024-09-21T14:58:39.054 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-21T14:58:39.054 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-21T14:58:39.054 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-21T14:58:39.054 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-21T14:58:39.054 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-21T14:58:39.055 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-21T14:58:39.055 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2024-09-21T14:58:39.055 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 24, 2024-09-21T14:58:39.055 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-21T14:58:39.055 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-21T14:58:39.055 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-21T14:58:39.055 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:39.055 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.055 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:39.055 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6800", 2024-09-21T14:58:39.055 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2609154414 2024-09-21T14:58:39.055 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.055 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.055 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:39.055 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6801", 2024-09-21T14:58:39.056 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2609154414 2024-09-21T14:58:39.056 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:39.056 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:39.056 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.056 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-21T14:58:39.056 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:39.056 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.056 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:39.056 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6802", 2024-09-21T14:58:39.056 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2609154414 2024-09-21T14:58:39.056 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.056 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.056 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:39.056 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6803", 2024-09-21T14:58:39.056 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2609154414 2024-09-21T14:58:39.057 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:39.057 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:39.057 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.057 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-21T14:58:39.057 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:39.057 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.057 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:39.057 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6806", 2024-09-21T14:58:39.057 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2609154414 2024-09-21T14:58:39.057 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.057 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.057 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:39.057 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6807", 2024-09-21T14:58:39.057 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2609154414 2024-09-21T14:58:39.057 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:39.058 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:39.058 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.058 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-21T14:58:39.058 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:39.058 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.058 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:39.058 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6804", 2024-09-21T14:58:39.058 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2609154414 2024-09-21T14:58:39.058 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.058 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.058 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:39.058 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6805", 2024-09-21T14:58:39.058 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2609154414 2024-09-21T14:58:39.058 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:39.058 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:39.059 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.059 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6801/2609154414", 2024-09-21T14:58:39.059 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6803/2609154414", 2024-09-21T14:58:39.059 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6807/2609154414", 2024-09-21T14:58:39.059 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6805/2609154414", 2024-09-21T14:58:39.059 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-21T14:58:39.059 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-21T14:58:39.059 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-21T14:58:39.059 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:39.059 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.059 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.059 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-09-21T14:58:39.059 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "5a57b3dd-4a07-45b7-b159-3cac4c7eb262", 2024-09-21T14:58:39.059 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-21T14:58:39.059 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-21T14:58:39.060 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-21T14:58:39.060 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-21T14:58:39.060 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-21T14:58:39.060 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-21T14:58:39.060 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2024-09-21T14:58:39.060 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 24, 2024-09-21T14:58:39.060 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-21T14:58:39.060 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-21T14:58:39.060 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-21T14:58:39.060 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:39.060 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.060 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:39.060 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6816", 2024-09-21T14:58:39.060 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1462563376 2024-09-21T14:58:39.060 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.061 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.061 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:39.061 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6817", 2024-09-21T14:58:39.061 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1462563376 2024-09-21T14:58:39.061 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:39.061 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:39.061 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.061 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-21T14:58:39.061 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:39.061 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.061 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:39.061 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6818", 2024-09-21T14:58:39.061 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1462563376 2024-09-21T14:58:39.061 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.061 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.062 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:39.062 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6819", 2024-09-21T14:58:39.062 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1462563376 2024-09-21T14:58:39.062 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:39.062 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:39.062 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.062 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-21T14:58:39.062 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:39.062 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.062 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:39.062 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6822", 2024-09-21T14:58:39.062 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1462563376 2024-09-21T14:58:39.062 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.062 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.063 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:39.063 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6823", 2024-09-21T14:58:39.063 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1462563376 2024-09-21T14:58:39.063 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:39.063 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:39.063 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.063 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-21T14:58:39.063 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:39.063 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.063 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:39.063 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6820", 2024-09-21T14:58:39.063 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1462563376 2024-09-21T14:58:39.064 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.064 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.064 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:39.064 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6821", 2024-09-21T14:58:39.064 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1462563376 2024-09-21T14:58:39.064 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:39.064 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:39.064 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.064 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6817/1462563376", 2024-09-21T14:58:39.064 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6819/1462563376", 2024-09-21T14:58:39.064 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6823/1462563376", 2024-09-21T14:58:39.064 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6821/1462563376", 2024-09-21T14:58:39.064 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-21T14:58:39.065 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-21T14:58:39.065 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-21T14:58:39.065 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:39.065 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.065 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.065 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-09-21T14:58:39.065 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "39a00807-7517-474e-8842-623dd7093cc5", 2024-09-21T14:58:39.065 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-21T14:58:39.065 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-21T14:58:39.065 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-21T14:58:39.065 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-21T14:58:39.065 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-21T14:58:39.065 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-21T14:58:39.065 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2024-09-21T14:58:39.066 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 24, 2024-09-21T14:58:39.066 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-21T14:58:39.066 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-21T14:58:39.066 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-21T14:58:39.066 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:39.066 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.066 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:39.066 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6801", 2024-09-21T14:58:39.066 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 144412042 2024-09-21T14:58:39.066 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.066 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.066 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:39.067 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6803", 2024-09-21T14:58:39.067 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 144412042 2024-09-21T14:58:39.067 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:39.067 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:39.067 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.067 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-21T14:58:39.067 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:39.067 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.067 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:39.067 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6805", 2024-09-21T14:58:39.067 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 144412042 2024-09-21T14:58:39.067 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.067 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.067 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:39.068 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6807", 2024-09-21T14:58:39.068 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 144412042 2024-09-21T14:58:39.068 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:39.068 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:39.068 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.068 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-21T14:58:39.068 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:39.068 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.068 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:39.068 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6812", 2024-09-21T14:58:39.068 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 144412042 2024-09-21T14:58:39.068 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.068 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.069 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:39.069 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6814", 2024-09-21T14:58:39.069 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 144412042 2024-09-21T14:58:39.069 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:39.069 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:39.069 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.069 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-21T14:58:39.069 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:39.069 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.069 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:39.069 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6809", 2024-09-21T14:58:39.069 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 144412042 2024-09-21T14:58:39.069 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.069 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.069 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:39.069 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6811", 2024-09-21T14:58:39.070 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 144412042 2024-09-21T14:58:39.070 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:39.070 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:39.070 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.070 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.46:6803/144412042", 2024-09-21T14:58:39.070 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.46:6807/144412042", 2024-09-21T14:58:39.070 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.46:6814/144412042", 2024-09-21T14:58:39.070 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.46:6811/144412042", 2024-09-21T14:58:39.070 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-21T14:58:39.070 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-21T14:58:39.070 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-21T14:58:39.070 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:39.070 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.070 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.071 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-09-21T14:58:39.071 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "edf9f045-ddeb-4504-a0fc-1a0e4e853550", 2024-09-21T14:58:39.071 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-21T14:58:39.071 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-21T14:58:39.071 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-21T14:58:39.071 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-21T14:58:39.071 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-21T14:58:39.071 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-21T14:58:39.071 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2024-09-21T14:58:39.071 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 24, 2024-09-21T14:58:39.071 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-21T14:58:39.071 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-21T14:58:39.071 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-21T14:58:39.071 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:39.072 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.072 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:39.072 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6800", 2024-09-21T14:58:39.072 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 849101442 2024-09-21T14:58:39.072 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.072 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.072 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:39.072 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6802", 2024-09-21T14:58:39.072 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 849101442 2024-09-21T14:58:39.072 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:39.072 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:39.072 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.072 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-21T14:58:39.073 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:39.073 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.073 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:39.073 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6804", 2024-09-21T14:58:39.073 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 849101442 2024-09-21T14:58:39.073 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.073 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.073 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:39.073 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6806", 2024-09-21T14:58:39.073 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 849101442 2024-09-21T14:58:39.073 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:39.073 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:39.073 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.074 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-21T14:58:39.074 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:39.074 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.074 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:39.074 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6813", 2024-09-21T14:58:39.074 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 849101442 2024-09-21T14:58:39.074 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.074 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.074 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:39.074 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6815", 2024-09-21T14:58:39.074 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 849101442 2024-09-21T14:58:39.074 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:39.074 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:39.074 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.075 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-21T14:58:39.075 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:39.075 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.075 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:39.075 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6808", 2024-09-21T14:58:39.075 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 849101442 2024-09-21T14:58:39.075 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.075 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.075 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:39.075 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6810", 2024-09-21T14:58:39.075 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 849101442 2024-09-21T14:58:39.075 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:39.075 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:39.075 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.075 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.46:6802/849101442", 2024-09-21T14:58:39.076 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.46:6806/849101442", 2024-09-21T14:58:39.076 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.46:6815/849101442", 2024-09-21T14:58:39.076 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.46:6810/849101442", 2024-09-21T14:58:39.076 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-21T14:58:39.076 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-21T14:58:39.076 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-21T14:58:39.076 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:39.076 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.076 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.076 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-09-21T14:58:39.076 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "c820ea6b-88fe-4421-83ec-17c466691fea", 2024-09-21T14:58:39.076 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-21T14:58:39.076 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-21T14:58:39.076 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-21T14:58:39.076 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-21T14:58:39.076 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-21T14:58:39.077 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-21T14:58:39.077 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2024-09-21T14:58:39.077 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 24, 2024-09-21T14:58:39.077 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-21T14:58:39.077 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-21T14:58:39.077 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-21T14:58:39.077 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:39.077 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.077 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:39.077 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6817", 2024-09-21T14:58:39.077 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 847189172 2024-09-21T14:58:39.077 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.077 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.077 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:39.078 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6818", 2024-09-21T14:58:39.078 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 847189172 2024-09-21T14:58:39.078 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:39.078 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:39.078 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.078 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-21T14:58:39.078 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:39.078 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.078 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:39.078 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6820", 2024-09-21T14:58:39.078 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 847189172 2024-09-21T14:58:39.078 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.078 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.078 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:39.078 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6822", 2024-09-21T14:58:39.079 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 847189172 2024-09-21T14:58:39.079 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:39.079 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:39.079 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.079 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-21T14:58:39.079 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:39.079 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.079 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:39.079 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6827", 2024-09-21T14:58:39.079 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 847189172 2024-09-21T14:58:39.079 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.079 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.079 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:39.079 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6828", 2024-09-21T14:58:39.079 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 847189172 2024-09-21T14:58:39.080 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:39.080 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:39.080 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.080 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-21T14:58:39.080 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:39.080 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.080 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:39.080 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6823", 2024-09-21T14:58:39.080 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 847189172 2024-09-21T14:58:39.080 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.080 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.080 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:39.080 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6825", 2024-09-21T14:58:39.080 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 847189172 2024-09-21T14:58:39.080 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:39.080 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:39.081 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.081 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.46:6818/847189172", 2024-09-21T14:58:39.081 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.46:6822/847189172", 2024-09-21T14:58:39.081 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.46:6828/847189172", 2024-09-21T14:58:39.081 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.46:6825/847189172", 2024-09-21T14:58:39.081 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-21T14:58:39.081 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-21T14:58:39.081 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-21T14:58:39.081 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:39.081 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.081 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.081 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-09-21T14:58:39.081 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "3a2f82a4-28a0-4fcb-9d6b-2285fa4d07ac", 2024-09-21T14:58:39.081 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-21T14:58:39.081 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-21T14:58:39.081 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 0.95001220703125, 2024-09-21T14:58:39.082 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-21T14:58:39.082 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-21T14:58:39.082 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-21T14:58:39.082 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 15, 2024-09-21T14:58:39.082 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 24, 2024-09-21T14:58:39.082 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-21T14:58:39.082 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-21T14:58:39.082 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-21T14:58:39.082 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:39.082 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.082 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:39.082 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6816", 2024-09-21T14:58:39.082 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3894963553 2024-09-21T14:58:39.082 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.083 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.083 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:39.083 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6819", 2024-09-21T14:58:39.083 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3894963553 2024-09-21T14:58:39.083 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:39.083 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:39.083 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.083 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-21T14:58:39.083 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:39.083 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.083 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:39.083 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6821", 2024-09-21T14:58:39.083 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3894963553 2024-09-21T14:58:39.083 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.083 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.084 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:39.084 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6824", 2024-09-21T14:58:39.084 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3894963553 2024-09-21T14:58:39.084 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:39.084 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:39.084 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.084 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-21T14:58:39.084 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:39.084 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.084 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:39.084 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6830", 2024-09-21T14:58:39.084 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3894963553 2024-09-21T14:58:39.084 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.084 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.084 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:39.085 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6831", 2024-09-21T14:58:39.085 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3894963553 2024-09-21T14:58:39.085 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:39.085 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:39.085 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.085 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-21T14:58:39.085 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:39.085 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.085 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:39.085 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6826", 2024-09-21T14:58:39.085 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3894963553 2024-09-21T14:58:39.085 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.085 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.085 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:39.085 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6829", 2024-09-21T14:58:39.085 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3894963553 2024-09-21T14:58:39.086 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:39.086 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:39.086 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.086 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.46:6819/3894963553", 2024-09-21T14:58:39.086 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.46:6824/3894963553", 2024-09-21T14:58:39.086 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.46:6831/3894963553", 2024-09-21T14:58:39.086 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.46:6829/3894963553", 2024-09-21T14:58:39.086 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-21T14:58:39.086 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-21T14:58:39.086 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-21T14:58:39.086 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:39.086 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:39.086 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-21T14:58:39.086 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_xinfo": [ 2024-09-21T14:58:39.087 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.087 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-09-21T14:58:39.087 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-21T14:58:39.087 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-21T14:58:39.087 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-21T14:58:39.087 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-21T14:58:39.087 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-21T14:58:39.087 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-21T14:57:57.458317+0000", 2024-09-21T14:58:39.087 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-21T14:58:39.087 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.087 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.087 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-09-21T14:58:39.087 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-21T14:58:39.087 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-21T14:58:39.087 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-21T14:58:39.088 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-21T14:58:39.088 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-21T14:58:39.088 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-21T14:57:57.907624+0000", 2024-09-21T14:58:39.088 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-21T14:58:39.088 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.088 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.088 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-09-21T14:58:39.088 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-21T14:58:39.088 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-21T14:58:39.088 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-21T14:58:39.088 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-21T14:58:39.088 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-21T14:58:39.088 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-21T14:57:57.278817+0000", 2024-09-21T14:58:39.089 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-21T14:58:39.089 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.089 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.089 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-09-21T14:58:39.089 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-21T14:58:39.089 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-21T14:58:39.089 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-21T14:58:39.089 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-21T14:58:39.089 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-21T14:58:39.089 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-21T14:57:57.500449+0000", 2024-09-21T14:58:39.089 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-21T14:58:39.089 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.089 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.089 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-09-21T14:58:39.089 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-21T14:58:39.090 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-21T14:58:39.090 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-21T14:58:39.090 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-21T14:58:39.090 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-21T14:58:39.090 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-21T14:57:57.504325+0000", 2024-09-21T14:58:39.090 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-21T14:58:39.090 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.090 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.090 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-09-21T14:58:39.090 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-21T14:58:39.090 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-21T14:58:39.090 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-21T14:58:39.090 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-21T14:58:39.091 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-21T14:58:39.091 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-21T14:57:57.777525+0000", 2024-09-21T14:58:39.091 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-21T14:58:39.091 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.091 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.091 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-09-21T14:58:39.091 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-21T14:58:39.091 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-21T14:58:39.091 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-21T14:58:39.091 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-21T14:58:39.091 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-21T14:58:39.091 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-21T14:57:57.698961+0000", 2024-09-21T14:58:39.091 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-21T14:58:39.091 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.091 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.091 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-09-21T14:58:39.092 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-21T14:58:39.092 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-21T14:58:39.092 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-21T14:58:39.092 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-21T14:58:39.092 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-21T14:58:39.092 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-21T14:57:57.932625+0000", 2024-09-21T14:58:39.092 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-21T14:58:39.092 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:39.092 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-21T14:58:39.092 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap": [], 2024-09-21T14:58:39.092 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_items": [], 2024-09-21T14:58:39.092 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_primaries": [], 2024-09-21T14:58:39.092 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_temp": [], 2024-09-21T14:58:39.093 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_temp": [], 2024-09-21T14:58:39.093 INFO:teuthology.orchestra.run.smithi032.stdout: "blocklist": {}, 2024-09-21T14:58:39.093 INFO:teuthology.orchestra.run.smithi032.stdout: "range_blocklist": {}, 2024-09-21T14:58:39.093 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profiles": { 2024-09-21T14:58:39.093 INFO:teuthology.orchestra.run.smithi032.stdout: "default": { 2024-09-21T14:58:39.093 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2024-09-21T14:58:39.093 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2024-09-21T14:58:39.093 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "1", 2024-09-21T14:58:39.093 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2024-09-21T14:58:39.093 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van" 2024-09-21T14:58:39.093 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:39.093 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.093 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps_queue": [ 2024-09-21T14:58:39.093 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.093 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 3, 2024-09-21T14:58:39.093 INFO:teuthology.orchestra.run.smithi032.stdout: "snaps": [ 2024-09-21T14:58:39.094 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.094 INFO:teuthology.orchestra.run.smithi032.stdout: "begin": 11, 2024-09-21T14:58:39.094 INFO:teuthology.orchestra.run.smithi032.stdout: "length": 1 2024-09-21T14:58:39.094 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.094 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.094 INFO:teuthology.orchestra.run.smithi032.stdout: "begin": 14, 2024-09-21T14:58:39.094 INFO:teuthology.orchestra.run.smithi032.stdout: "length": 1 2024-09-21T14:58:39.094 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:39.094 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:39.094 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:39.094 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-21T14:58:39.094 INFO:teuthology.orchestra.run.smithi032.stdout: "new_removed_snaps": [], 2024-09-21T14:58:39.094 INFO:teuthology.orchestra.run.smithi032.stdout: "new_purged_snaps": [], 2024-09-21T14:58:39.094 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_node_flags": {}, 2024-09-21T14:58:39.094 INFO:teuthology.orchestra.run.smithi032.stdout: "device_class_flags": {}, 2024-09-21T14:58:39.095 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode": { 2024-09-21T14:58:39.095 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_enabled": false, 2024-09-21T14:58:39.095 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_bucket_count": 0, 2024-09-21T14:58:39.095 INFO:teuthology.orchestra.run.smithi032.stdout: "degraded_stretch_mode": 0, 2024-09-21T14:58:39.095 INFO:teuthology.orchestra.run.smithi032.stdout: "recovering_stretch_mode": 0, 2024-09-21T14:58:39.095 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_bucket": 0 2024-09-21T14:58:39.095 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:39.095 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:58:39.097 INFO:tasks.thrashosds.thrasher:j is {'epoch': 43, 'fsid': 'caf51e55-9234-4a26-bcd2-d60811962560', 'created': '2024-09-21T14:57:50.338822+0000', 'modified': '2024-09-21T14:58:38.100916+0000', 'last_up_change': '2024-09-21T14:58:01.177141+0000', 'last_in_change': '2024-09-21T14:57:52.442036+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T14:58:01.842810+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-21T14:58:08.284925+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 22, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-21T14:58:17.510709+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '43', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 16, 'snap_epoch': 43, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': 'a4a73316-3a94-42e9-9a51-deb46a66bb06', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6808', 'nonce': 4016672243}, {'type': 'v1', 'addr': '172.21.15.32:6809', 'nonce': 4016672243}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6810', 'nonce': 4016672243}, {'type': 'v1', 'addr': '172.21.15.32:6811', 'nonce': 4016672243}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6814', 'nonce': 4016672243}, {'type': 'v1', 'addr': '172.21.15.32:6815', 'nonce': 4016672243}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6812', 'nonce': 4016672243}, {'type': 'v1', 'addr': '172.21.15.32:6813', 'nonce': 4016672243}]}, 'public_addr': '172.21.15.32:6809/4016672243', 'cluster_addr': '172.21.15.32:6811/4016672243', 'heartbeat_back_addr': '172.21.15.32:6815/4016672243', 'heartbeat_front_addr': '172.21.15.32:6813/4016672243', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '006757a0-94d1-4a9b-aef8-73634cc5828e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6824', 'nonce': 641090577}, {'type': 'v1', 'addr': '172.21.15.32:6825', 'nonce': 641090577}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6826', 'nonce': 641090577}, {'type': 'v1', 'addr': '172.21.15.32:6827', 'nonce': 641090577}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6830', 'nonce': 641090577}, {'type': 'v1', 'addr': '172.21.15.32:6831', 'nonce': 641090577}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6828', 'nonce': 641090577}, {'type': 'v1', 'addr': '172.21.15.32:6829', 'nonce': 641090577}]}, 'public_addr': '172.21.15.32:6825/641090577', 'cluster_addr': '172.21.15.32:6827/641090577', 'heartbeat_back_addr': '172.21.15.32:6831/641090577', 'heartbeat_front_addr': '172.21.15.32:6829/641090577', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '0b499a23-820e-4e6c-a558-adbf88b7014f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6800', 'nonce': 2609154414}, {'type': 'v1', 'addr': '172.21.15.32:6801', 'nonce': 2609154414}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6802', 'nonce': 2609154414}, {'type': 'v1', 'addr': '172.21.15.32:6803', 'nonce': 2609154414}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6806', 'nonce': 2609154414}, {'type': 'v1', 'addr': '172.21.15.32:6807', 'nonce': 2609154414}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6804', 'nonce': 2609154414}, {'type': 'v1', 'addr': '172.21.15.32:6805', 'nonce': 2609154414}]}, 'public_addr': '172.21.15.32:6801/2609154414', 'cluster_addr': '172.21.15.32:6803/2609154414', 'heartbeat_back_addr': '172.21.15.32:6807/2609154414', 'heartbeat_front_addr': '172.21.15.32:6805/2609154414', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '5a57b3dd-4a07-45b7-b159-3cac4c7eb262', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6816', 'nonce': 1462563376}, {'type': 'v1', 'addr': '172.21.15.32:6817', 'nonce': 1462563376}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6818', 'nonce': 1462563376}, {'type': 'v1', 'addr': '172.21.15.32:6819', 'nonce': 1462563376}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6822', 'nonce': 1462563376}, {'type': 'v1', 'addr': '172.21.15.32:6823', 'nonce': 1462563376}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6820', 'nonce': 1462563376}, {'type': 'v1', 'addr': '172.21.15.32:6821', 'nonce': 1462563376}]}, 'public_addr': '172.21.15.32:6817/1462563376', 'cluster_addr': '172.21.15.32:6819/1462563376', 'heartbeat_back_addr': '172.21.15.32:6823/1462563376', 'heartbeat_front_addr': '172.21.15.32:6821/1462563376', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '39a00807-7517-474e-8842-623dd7093cc5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6801', 'nonce': 144412042}, {'type': 'v1', 'addr': '172.21.15.46:6803', 'nonce': 144412042}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6805', 'nonce': 144412042}, {'type': 'v1', 'addr': '172.21.15.46:6807', 'nonce': 144412042}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6812', 'nonce': 144412042}, {'type': 'v1', 'addr': '172.21.15.46:6814', 'nonce': 144412042}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6809', 'nonce': 144412042}, {'type': 'v1', 'addr': '172.21.15.46:6811', 'nonce': 144412042}]}, 'public_addr': '172.21.15.46:6803/144412042', 'cluster_addr': '172.21.15.46:6807/144412042', 'heartbeat_back_addr': '172.21.15.46:6814/144412042', 'heartbeat_front_addr': '172.21.15.46:6811/144412042', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'edf9f045-ddeb-4504-a0fc-1a0e4e853550', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6800', 'nonce': 849101442}, {'type': 'v1', 'addr': '172.21.15.46:6802', 'nonce': 849101442}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6804', 'nonce': 849101442}, {'type': 'v1', 'addr': '172.21.15.46:6806', 'nonce': 849101442}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6813', 'nonce': 849101442}, {'type': 'v1', 'addr': '172.21.15.46:6815', 'nonce': 849101442}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6808', 'nonce': 849101442}, {'type': 'v1', 'addr': '172.21.15.46:6810', 'nonce': 849101442}]}, 'public_addr': '172.21.15.46:6802/849101442', 'cluster_addr': '172.21.15.46:6806/849101442', 'heartbeat_back_addr': '172.21.15.46:6815/849101442', 'heartbeat_front_addr': '172.21.15.46:6810/849101442', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'c820ea6b-88fe-4421-83ec-17c466691fea', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6817', 'nonce': 847189172}, {'type': 'v1', 'addr': '172.21.15.46:6818', 'nonce': 847189172}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6820', 'nonce': 847189172}, {'type': 'v1', 'addr': '172.21.15.46:6822', 'nonce': 847189172}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6827', 'nonce': 847189172}, {'type': 'v1', 'addr': '172.21.15.46:6828', 'nonce': 847189172}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6823', 'nonce': 847189172}, {'type': 'v1', 'addr': '172.21.15.46:6825', 'nonce': 847189172}]}, 'public_addr': '172.21.15.46:6818/847189172', 'cluster_addr': '172.21.15.46:6822/847189172', 'heartbeat_back_addr': '172.21.15.46:6828/847189172', 'heartbeat_front_addr': '172.21.15.46:6825/847189172', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '3a2f82a4-28a0-4fcb-9d6b-2285fa4d07ac', 'up': 1, 'in': 1, 'weight': 0.95001220703125, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6816', 'nonce': 3894963553}, {'type': 'v1', 'addr': '172.21.15.46:6819', 'nonce': 3894963553}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6821', 'nonce': 3894963553}, {'type': 'v1', 'addr': '172.21.15.46:6824', 'nonce': 3894963553}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6830', 'nonce': 3894963553}, {'type': 'v1', 'addr': '172.21.15.46:6831', 'nonce': 3894963553}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6826', 'nonce': 3894963553}, {'type': 'v1', 'addr': '172.21.15.46:6829', 'nonce': 3894963553}]}, 'public_addr': '172.21.15.46:6819/3894963553', 'cluster_addr': '172.21.15.46:6824/3894963553', 'heartbeat_back_addr': '172.21.15.46:6831/3894963553', 'heartbeat_front_addr': '172.21.15.46:6829/3894963553', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:57:57.458317+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:57:57.907624+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:57:57.278817+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:57:57.500449+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:57:57.504325+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:57:57.777525+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:57:57.698961+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:57:57.932625+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 11, 'length': 1}, {'begin': 14, 'length': 1}]}], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-21T14:58:39.097 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T14:58:39.098 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:58:39.098 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:58:39.098 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T14:58:39.099 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T14:58:39.099 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:58:39.099 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T14:58:39.099 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:58:39.092+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:58:39.143 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-21T14:58:39.179 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T14:58:39.198 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:58:39.196+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:58:39.216 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T14:58:39.267 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T14:58:39.270 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:58:39.270 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:58:39.270 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:58:39.270 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:58:39.270 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:58:39.270 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:58:39.270 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:58:39.270 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:58:39.297 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:58:39.296+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:58:39.310 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:58:39.310 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T14:58:39.311 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T14:58:39.311 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T14:58:39.311 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.311 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4281.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-21T14:58:39.311 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:09.324416+0000", 2024-09-21T14:58:39.311 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 29.983903570999999, 2024-09-21T14:58:39.311 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.117837947, 2024-09-21T14:58:39.311 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:39.311 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:39.311 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:39.311 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4281", 2024-09-21T14:58:39.311 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/1518520303", 2024-09-21T14:58:39.311 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1 2024-09-21T14:58:39.311 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.312 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:39.312 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.312 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:39.312 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324416+0000", 2024-09-21T14:58:39.312 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:39.312 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.312 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.312 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:39.312 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324416+0000", 2024-09-21T14:58:39.312 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:39.312 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.312 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.312 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:39.312 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324414+0000", 2024-09-21T14:58:39.312 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T14:58:39.313 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.313 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.313 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:39.313 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324429+0000", 2024-09-21T14:58:39.313 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5637e-05 2024-09-21T14:58:39.313 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.313 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.313 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:39.313 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324448+0000", 2024-09-21T14:58:39.313 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9042000000000002e-05 2024-09-21T14:58:39.313 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.313 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.313 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:39.313 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324465+0000", 2024-09-21T14:58:39.313 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6767999999999999e-05 2024-09-21T14:58:39.313 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.314 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.314 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:39.314 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324571+0000", 2024-09-21T14:58:39.314 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000105714 2024-09-21T14:58:39.314 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.314 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.314 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op must wait for map", 2024-09-21T14:58:39.314 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324587+0000", 2024-09-21T14:58:39.314 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5753999999999999e-05 2024-09-21T14:58:39.314 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.314 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.314 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:39.314 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.325864+0000", 2024-09-21T14:58:39.314 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0012773350000000001 2024-09-21T14:58:39.314 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.315 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.315 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T14:58:39.315 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.325878+0000", 2024-09-21T14:58:39.315 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4498000000000001e-05 2024-09-21T14:58:39.315 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.315 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.315 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:39.315 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.439461+0000", 2024-09-21T14:58:39.315 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.11358212500000001 2024-09-21T14:58:39.315 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.315 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.315 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:39.315 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.439848+0000", 2024-09-21T14:58:39.315 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00038747800000000002 2024-09-21T14:58:39.316 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.316 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.316 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:58:39.316 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.440175+0000", 2024-09-21T14:58:39.316 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00032721900000000001 2024-09-21T14:58:39.316 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.316 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.316 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:39.316 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.441770+0000", 2024-09-21T14:58:39.316 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0015949409999999999 2024-09-21T14:58:39.316 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.316 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.316 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:39.316 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.442086+0000", 2024-09-21T14:58:39.316 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00031586299999999999 2024-09-21T14:58:39.316 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.317 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.317 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:39.317 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.442195+0000", 2024-09-21T14:58:39.317 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000108791 2024-09-21T14:58:39.317 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.317 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.317 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:39.317 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.442254+0000", 2024-09-21T14:58:39.317 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9038e-05 2024-09-21T14:58:39.317 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:39.317 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:39.317 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:39.317 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.317 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.317 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4281.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-21T14:58:39.317 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:09.447828+0000", 2024-09-21T14:58:39.318 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 29.860491570000001, 2024-09-21T14:58:39.318 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.32904393700000001, 2024-09-21T14:58:39.318 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:39.318 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:58:39.318 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:39.318 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4281", 2024-09-21T14:58:39.318 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/1518520303", 2024-09-21T14:58:39.318 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 2 2024-09-21T14:58:39.318 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.318 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:39.318 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.318 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:39.318 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447828+0000", 2024-09-21T14:58:39.318 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:39.318 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.319 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.319 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:39.319 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447828+0000", 2024-09-21T14:58:39.319 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:39.319 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.319 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.319 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:39.319 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447825+0000", 2024-09-21T14:58:39.319 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T14:58:39.319 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.319 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.319 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:39.319 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447841+0000", 2024-09-21T14:58:39.319 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5359999999999999e-05 2024-09-21T14:58:39.319 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.319 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.320 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:39.320 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447875+0000", 2024-09-21T14:58:39.320 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4388999999999999e-05 2024-09-21T14:58:39.320 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.320 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.320 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:39.320 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447903+0000", 2024-09-21T14:58:39.320 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7373e-05 2024-09-21T14:58:39.320 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.320 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.320 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:39.320 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447996+0000", 2024-09-21T14:58:39.320 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2954999999999993e-05 2024-09-21T14:58:39.320 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.320 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.320 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T14:58:39.321 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.448011+0000", 2024-09-21T14:58:39.321 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5038999999999999e-05 2024-09-21T14:58:39.321 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.321 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.321 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:39.321 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.776501+0000", 2024-09-21T14:58:39.321 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.32849012500000002 2024-09-21T14:58:39.321 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.321 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.321 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:39.321 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.776872+0000", 2024-09-21T14:58:39.321 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00037119600000000002 2024-09-21T14:58:39.321 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:39.321 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:39.321 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:39.321 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.322 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.322 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:752 3.14 3:28fc02e5:::rbd_data.113693316f36.0000000000000010:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-21T14:58:39.322 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:26.625579+0000", 2024-09-21T14:58:39.322 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 12.682740715, 2024-09-21T14:58:39.322 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042965426000000001, 2024-09-21T14:58:39.322 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:39.322 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:39.322 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:39.322 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:39.322 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:39.322 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 752 2024-09-21T14:58:39.322 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.322 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:39.322 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.322 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:39.323 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625579+0000", 2024-09-21T14:58:39.323 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:39.323 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.323 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.323 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:39.323 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625579+0000", 2024-09-21T14:58:39.323 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:39.323 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.323 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.323 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:39.323 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625577+0000", 2024-09-21T14:58:39.323 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T14:58:39.323 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.323 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.323 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:39.323 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625588+0000", 2024-09-21T14:58:39.324 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1555e-05 2024-09-21T14:58:39.324 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.324 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.324 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:39.324 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625610+0000", 2024-09-21T14:58:39.324 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2107999999999999e-05 2024-09-21T14:58:39.324 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.324 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.324 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:39.324 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625621+0000", 2024-09-21T14:58:39.324 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0762e-05 2024-09-21T14:58:39.324 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.324 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.324 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:39.324 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625692+0000", 2024-09-21T14:58:39.324 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.1203000000000003e-05 2024-09-21T14:58:39.325 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.325 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.325 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:39.325 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625872+0000", 2024-09-21T14:58:39.325 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017962800000000001 2024-09-21T14:58:39.325 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.325 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.325 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:58:39.325 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.633188+0000", 2024-09-21T14:58:39.325 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.007316014 2024-09-21T14:58:39.325 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.325 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.325 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:39.325 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.668357+0000", 2024-09-21T14:58:39.325 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035168933999999999 2024-09-21T14:58:39.326 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.326 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.326 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:39.326 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.668418+0000", 2024-09-21T14:58:39.326 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1282e-05 2024-09-21T14:58:39.326 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.326 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.326 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:39.326 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.668486+0000", 2024-09-21T14:58:39.326 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8468999999999998e-05 2024-09-21T14:58:39.326 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.326 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.326 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:39.326 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.668544+0000", 2024-09-21T14:58:39.326 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7785000000000003e-05 2024-09-21T14:58:39.326 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:39.327 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:39.327 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:39.327 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.327 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.327 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop_reply(client.4406.0:752 3.14 e32/24 ondisk, result = 0)", 2024-09-21T14:58:39.327 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:26.653670+0000", 2024-09-21T14:58:39.327 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 12.654649469000001, 2024-09-21T14:58:39.327 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014884797, 2024-09-21T14:58:39.327 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:39.327 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:39.327 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:39.327 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.327 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:39.327 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.653670+0000", 2024-09-21T14:58:39.327 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:39.328 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.328 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.328 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:39.328 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.653670+0000", 2024-09-21T14:58:39.328 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:39.328 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.328 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.328 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:39.328 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.653669+0000", 2024-09-21T14:58:39.328 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:58:39.328 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.328 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.328 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:39.328 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.653676+0000", 2024-09-21T14:58:39.328 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.1629999999999999e-06 2024-09-21T14:58:39.328 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.329 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.329 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:39.329 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.653695+0000", 2024-09-21T14:58:39.329 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8760999999999999e-05 2024-09-21T14:58:39.329 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.329 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.329 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:39.329 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.653704+0000", 2024-09-21T14:58:39.329 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6139999999999998e-06 2024-09-21T14:58:39.329 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.329 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.329 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:39.329 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.668393+0000", 2024-09-21T14:58:39.329 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014688728 2024-09-21T14:58:39.329 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.330 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.330 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:39.330 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.668410+0000", 2024-09-21T14:58:39.330 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6858e-05 2024-09-21T14:58:39.330 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.330 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.330 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:39.330 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.668555+0000", 2024-09-21T14:58:39.330 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014483700000000001 2024-09-21T14:58:39.330 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:39.330 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:39.330 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:39.330 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.330 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.330 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6504 3.d 3:b2f690c6:::rbd_data.11361bed1f78.0000000000000019:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:39.330 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.121932+0000", 2024-09-21T14:58:39.331 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.1863870470000002, 2024-09-21T14:58:39.331 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0097717470000000008, 2024-09-21T14:58:39.331 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:39.331 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:39.331 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:39.331 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:39.331 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:39.331 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6504 2024-09-21T14:58:39.331 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.331 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:39.331 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.331 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:39.331 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.121932+0000", 2024-09-21T14:58:39.331 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:39.331 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.331 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.332 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:39.332 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.121932+0000", 2024-09-21T14:58:39.332 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:39.332 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.332 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.332 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:39.332 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.121932+0000", 2024-09-21T14:58:39.332 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:39.332 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.332 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.332 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:39.332 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.121934+0000", 2024-09-21T14:58:39.332 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3190000000000002e-06 2024-09-21T14:58:39.332 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.332 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.332 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:39.333 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.121939+0000", 2024-09-21T14:58:39.333 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3159999999999998e-06 2024-09-21T14:58:39.333 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.333 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.333 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:39.333 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.121950+0000", 2024-09-21T14:58:39.333 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1476e-05 2024-09-21T14:58:39.333 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.333 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.333 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:39.333 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.122788+0000", 2024-09-21T14:58:39.333 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00083814499999999999 2024-09-21T14:58:39.333 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.333 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.333 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting_for_map not empty", 2024-09-21T14:58:39.334 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.122793+0000", 2024-09-21T14:58:39.334 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2729999999999996e-06 2024-09-21T14:58:39.334 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.334 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.334 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:39.334 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131544+0000", 2024-09-21T14:58:39.334 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.008751768 2024-09-21T14:58:39.334 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.334 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.334 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:39.334 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131629+0000", 2024-09-21T14:58:39.334 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4536000000000005e-05 2024-09-21T14:58:39.334 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.334 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.334 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:39.334 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131704+0000", 2024-09-21T14:58:39.335 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5346999999999995e-05 2024-09-21T14:58:39.335 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:39.335 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:39.335 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:39.335 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.335 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.335 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6682 3.1d 3:b8f5ac1a:::rbd_data.11361bed1f78.0000000000000009:d [read 147456~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:39.335 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.134499+0000", 2024-09-21T14:58:39.335 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.1738201630000002, 2024-09-21T14:58:39.335 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0094333049999999995, 2024-09-21T14:58:39.335 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:39.335 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:39.335 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:39.335 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:39.335 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:39.335 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6682 2024-09-21T14:58:39.336 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.336 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:39.336 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.336 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:39.336 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134499+0000", 2024-09-21T14:58:39.336 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:39.336 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.336 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.336 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:39.336 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134499+0000", 2024-09-21T14:58:39.336 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:39.336 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.336 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.336 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:39.336 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134499+0000", 2024-09-21T14:58:39.336 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:39.337 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.337 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.337 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:39.337 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134502+0000", 2024-09-21T14:58:39.337 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8399999999999999e-06 2024-09-21T14:58:39.337 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.337 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.337 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:39.337 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134509+0000", 2024-09-21T14:58:39.337 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6539999999999999e-06 2024-09-21T14:58:39.337 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.337 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.337 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:39.337 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134516+0000", 2024-09-21T14:58:39.337 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2689999999999996e-06 2024-09-21T14:58:39.337 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.338 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.338 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:39.338 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.143777+0000", 2024-09-21T14:58:39.338 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0092615270000000003 2024-09-21T14:58:39.338 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.338 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.338 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:39.338 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.143849+0000", 2024-09-21T14:58:39.338 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2138000000000006e-05 2024-09-21T14:58:39.338 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.338 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.338 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:39.338 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.143933+0000", 2024-09-21T14:58:39.338 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3354000000000002e-05 2024-09-21T14:58:39.338 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:39.338 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:39.339 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:39.339 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.339 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.339 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6683 3.1d 3:b8f5ac1a:::rbd_data.11361bed1f78.0000000000000009:d [read 229376~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:39.339 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.134520+0000", 2024-09-21T14:58:39.339 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.1737993180000004, 2024-09-21T14:58:39.339 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0097861619999999993, 2024-09-21T14:58:39.339 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:39.339 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:39.339 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:39.339 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:39.339 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:39.339 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6683 2024-09-21T14:58:39.339 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.339 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:39.339 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.340 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:39.340 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134520+0000", 2024-09-21T14:58:39.340 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:39.340 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.340 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.340 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:39.340 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134520+0000", 2024-09-21T14:58:39.340 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:39.340 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.340 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.340 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:39.340 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134520+0000", 2024-09-21T14:58:39.340 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:39.340 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.340 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.341 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:39.341 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134522+0000", 2024-09-21T14:58:39.341 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4059999999999999e-06 2024-09-21T14:58:39.341 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.341 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.341 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:39.341 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134529+0000", 2024-09-21T14:58:39.341 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0890000000000002e-06 2024-09-21T14:58:39.341 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.341 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.341 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:39.341 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134534+0000", 2024-09-21T14:58:39.341 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.617e-06 2024-09-21T14:58:39.341 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.341 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.341 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:39.342 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.143957+0000", 2024-09-21T14:58:39.342 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0094232459999999997 2024-09-21T14:58:39.342 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.342 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.342 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:39.342 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.144046+0000", 2024-09-21T14:58:39.342 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8690000000000006e-05 2024-09-21T14:58:39.342 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.342 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.342 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:39.342 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.144306+0000", 2024-09-21T14:58:39.342 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00026042300000000002 2024-09-21T14:58:39.342 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:39.342 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:39.342 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:39.343 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.343 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.343 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6698 3.1d 3:b9c0cfe7:::rbd_data.11361bed1f78.0000000000000010:d [read 32768~32768 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:39.343 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.134539+0000", 2024-09-21T14:58:39.343 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.1737802430000004, 2024-09-21T14:58:39.343 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.010677103, 2024-09-21T14:58:39.343 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:39.343 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:39.343 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:39.343 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:39.343 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:39.343 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6698 2024-09-21T14:58:39.343 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.343 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:39.343 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.344 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:39.344 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134539+0000", 2024-09-21T14:58:39.344 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:39.344 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.344 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.344 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:39.344 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134539+0000", 2024-09-21T14:58:39.344 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:39.344 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.344 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.344 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:39.344 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134539+0000", 2024-09-21T14:58:39.344 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:39.344 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.344 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.344 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:39.345 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134541+0000", 2024-09-21T14:58:39.345 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3209999999999999e-06 2024-09-21T14:58:39.345 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.345 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.345 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:39.345 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134547+0000", 2024-09-21T14:58:39.345 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8799999999999996e-06 2024-09-21T14:58:39.345 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.345 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.345 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:39.345 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134557+0000", 2024-09-21T14:58:39.345 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0045e-05 2024-09-21T14:58:39.345 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.345 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.345 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:39.345 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.144524+0000", 2024-09-21T14:58:39.345 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0099663229999999992 2024-09-21T14:58:39.346 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.346 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.346 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:39.346 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.144625+0000", 2024-09-21T14:58:39.346 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000101376 2024-09-21T14:58:39.346 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.346 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.346 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:39.346 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.145216+0000", 2024-09-21T14:58:39.346 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00059145599999999997 2024-09-21T14:58:39.346 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:39.346 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:39.346 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:39.346 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.347 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.347 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6706 3.d 3:b2fe5261:::rbd_data.11361bed1f78.0000000000000008:d [read 98304~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:39.347 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.134652+0000", 2024-09-21T14:58:39.347 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.1736675520000004, 2024-09-21T14:58:39.347 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.010156419999999999, 2024-09-21T14:58:39.347 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:39.347 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:39.347 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:39.347 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:39.347 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:39.347 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6706 2024-09-21T14:58:39.347 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.347 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:39.347 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.348 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:39.348 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134652+0000", 2024-09-21T14:58:39.348 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:39.348 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.348 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.348 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:39.348 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134652+0000", 2024-09-21T14:58:39.348 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:39.348 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.348 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.348 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:39.348 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134652+0000", 2024-09-21T14:58:39.348 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:39.348 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.348 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.348 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:39.349 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134654+0000", 2024-09-21T14:58:39.349 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2199999999999999e-06 2024-09-21T14:58:39.349 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.349 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.349 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:39.349 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134660+0000", 2024-09-21T14:58:39.349 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7450000000000001e-06 2024-09-21T14:58:39.349 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.349 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.349 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:39.349 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134664+0000", 2024-09-21T14:58:39.349 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.549e-06 2024-09-21T14:58:39.349 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.349 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.349 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:39.350 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.144334+0000", 2024-09-21T14:58:39.350 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0096699009999999998 2024-09-21T14:58:39.350 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.350 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.350 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:39.350 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.144417+0000", 2024-09-21T14:58:39.350 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3088000000000001e-05 2024-09-21T14:58:39.350 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.350 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.350 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:39.350 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.144808+0000", 2024-09-21T14:58:39.350 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00039123499999999998 2024-09-21T14:58:39.350 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:39.350 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:39.350 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:39.351 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.351 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.351 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6707 3.d 3:b2fe5261:::rbd_data.11361bed1f78.0000000000000008:d [read 180224~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:39.351 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.134669+0000", 2024-09-21T14:58:39.351 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.1736507249999999, 2024-09-21T14:58:39.351 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.010282932, 2024-09-21T14:58:39.351 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:39.351 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:39.351 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:39.351 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:39.351 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:39.351 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6707 2024-09-21T14:58:39.351 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.351 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:39.352 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.352 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:39.352 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134669+0000", 2024-09-21T14:58:39.352 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:39.352 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.352 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.352 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:39.352 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134669+0000", 2024-09-21T14:58:39.352 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:39.352 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.352 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.352 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:39.352 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134668+0000", 2024-09-21T14:58:39.352 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:39.352 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.352 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.353 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:39.353 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134675+0000", 2024-09-21T14:58:39.353 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4239999999999998e-06 2024-09-21T14:58:39.353 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.353 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.353 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:39.353 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134681+0000", 2024-09-21T14:58:39.353 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8100000000000003e-06 2024-09-21T14:58:39.353 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.353 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.353 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:39.353 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134686+0000", 2024-09-21T14:58:39.353 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2349999999999999e-06 2024-09-21T14:58:39.353 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.353 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.354 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:39.354 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.144832+0000", 2024-09-21T14:58:39.354 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.010146159 2024-09-21T14:58:39.354 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.354 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.354 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:39.354 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.144886+0000", 2024-09-21T14:58:39.354 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3739999999999997e-05 2024-09-21T14:58:39.354 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.354 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.354 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:39.354 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.144952+0000", 2024-09-21T14:58:39.354 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6049000000000004e-05 2024-09-21T14:58:39.354 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:39.354 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:39.355 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:39.355 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.355 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.355 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6708 3.d 3:b2fe5261:::rbd_data.11361bed1f78.0000000000000008:d [read 245760~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:39.355 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.134694+0000", 2024-09-21T14:58:39.355 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.1736256169999999, 2024-09-21T14:58:39.355 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012651897, 2024-09-21T14:58:39.355 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:39.355 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:39.355 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:39.355 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:39.355 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:39.355 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6708 2024-09-21T14:58:39.355 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.355 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:39.355 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.356 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:39.356 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134694+0000", 2024-09-21T14:58:39.356 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:39.356 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.356 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.356 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:39.356 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134694+0000", 2024-09-21T14:58:39.356 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:39.356 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.356 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.356 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:39.356 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134693+0000", 2024-09-21T14:58:39.356 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:39.356 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.357 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.357 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:39.357 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134696+0000", 2024-09-21T14:58:39.357 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.061e-06 2024-09-21T14:58:39.357 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.357 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.357 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:39.357 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134703+0000", 2024-09-21T14:58:39.357 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8970000000000002e-06 2024-09-21T14:58:39.357 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.357 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.357 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:39.357 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134708+0000", 2024-09-21T14:58:39.357 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8119999999999998e-06 2024-09-21T14:58:39.357 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.357 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.358 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:39.358 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.144979+0000", 2024-09-21T14:58:39.358 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.010270975999999999 2024-09-21T14:58:39.358 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.358 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.358 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:39.358 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.145049+0000", 2024-09-21T14:58:39.358 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9849000000000002e-05 2024-09-21T14:58:39.358 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.358 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.358 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:39.358 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147346+0000", 2024-09-21T14:58:39.358 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0022968799999999998 2024-09-21T14:58:39.358 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:39.358 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:39.358 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:39.359 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.359 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.359 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6709 3.1d 3:b8f5ac1a:::rbd_data.11361bed1f78.0000000000000009:d [read 98304~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:39.359 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.134712+0000", 2024-09-21T14:58:39.359 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.1736071969999999, 2024-09-21T14:58:39.359 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.010787688, 2024-09-21T14:58:39.359 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:39.359 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:39.359 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:39.359 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:39.359 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:39.359 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6709 2024-09-21T14:58:39.359 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.359 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:39.359 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.359 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:39.360 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134712+0000", 2024-09-21T14:58:39.360 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:39.360 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.360 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.360 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:39.360 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134712+0000", 2024-09-21T14:58:39.360 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:39.360 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.360 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.360 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:39.360 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134712+0000", 2024-09-21T14:58:39.360 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:39.360 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.360 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.360 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:39.360 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134720+0000", 2024-09-21T14:58:39.361 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8960000000000003e-06 2024-09-21T14:58:39.361 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.361 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.361 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:39.361 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134726+0000", 2024-09-21T14:58:39.361 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8699999999999997e-06 2024-09-21T14:58:39.361 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.361 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.361 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:39.361 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134737+0000", 2024-09-21T14:58:39.361 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0939e-05 2024-09-21T14:58:39.361 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.361 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.361 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:39.362 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.145232+0000", 2024-09-21T14:58:39.362 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.010494837999999999 2024-09-21T14:58:39.362 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.362 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.362 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:39.362 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.145310+0000", 2024-09-21T14:58:39.362 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8004999999999996e-05 2024-09-21T14:58:39.362 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.362 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.362 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:39.362 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.145500+0000", 2024-09-21T14:58:39.362 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000190466 2024-09-21T14:58:39.362 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:39.362 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:39.362 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:39.363 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.363 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.363 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6710 3.1d 3:b8f5ac1a:::rbd_data.11361bed1f78.0000000000000009:d [read 163840~32768 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:39.363 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.134742+0000", 2024-09-21T14:58:39.363 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.1735771269999997, 2024-09-21T14:58:39.363 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.010932667, 2024-09-21T14:58:39.363 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:39.363 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:39.363 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:39.363 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:39.363 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:39.363 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6710 2024-09-21T14:58:39.363 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.363 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:39.363 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.363 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:39.364 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134742+0000", 2024-09-21T14:58:39.364 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:39.364 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.364 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.364 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:39.364 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134742+0000", 2024-09-21T14:58:39.364 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:39.364 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.364 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.364 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:39.364 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134742+0000", 2024-09-21T14:58:39.364 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:39.364 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.364 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.364 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:39.365 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134745+0000", 2024-09-21T14:58:39.365 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8700000000000001e-06 2024-09-21T14:58:39.365 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.365 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.365 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:39.365 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134751+0000", 2024-09-21T14:58:39.365 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4309999999999999e-06 2024-09-21T14:58:39.365 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.365 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.365 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:39.365 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134757+0000", 2024-09-21T14:58:39.365 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5940000000000001e-06 2024-09-21T14:58:39.366 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.366 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.366 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:39.366 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.145522+0000", 2024-09-21T14:58:39.366 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.010765389 2024-09-21T14:58:39.366 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.366 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.366 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:39.366 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.145589+0000", 2024-09-21T14:58:39.366 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7005e-05 2024-09-21T14:58:39.366 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.366 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.366 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:39.366 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.145675+0000", 2024-09-21T14:58:39.366 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5785999999999995e-05 2024-09-21T14:58:39.366 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:39.367 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:39.367 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:39.367 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.367 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.367 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6711 3.1d 3:b8f5ac1a:::rbd_data.11361bed1f78.0000000000000009:d [read 245760~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:39.367 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.134795+0000", 2024-09-21T14:58:39.367 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.1735249799999998, 2024-09-21T14:58:39.367 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.011222032, 2024-09-21T14:58:39.367 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:39.367 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:39.367 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:39.367 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:39.367 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:39.367 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6711 2024-09-21T14:58:39.368 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.368 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:39.368 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.368 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:39.368 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134795+0000", 2024-09-21T14:58:39.368 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:39.368 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.368 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.368 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:39.368 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134795+0000", 2024-09-21T14:58:39.368 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:39.368 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.368 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.368 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:39.368 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134794+0000", 2024-09-21T14:58:39.368 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:39.369 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.369 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.369 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:39.369 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134797+0000", 2024-09-21T14:58:39.369 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4989999999999999e-06 2024-09-21T14:58:39.369 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.369 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.369 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:39.369 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134805+0000", 2024-09-21T14:58:39.369 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.1289999999999998e-06 2024-09-21T14:58:39.369 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.369 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.369 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:39.369 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134810+0000", 2024-09-21T14:58:39.369 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0490000000000003e-06 2024-09-21T14:58:39.369 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.370 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.370 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:39.370 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.145697+0000", 2024-09-21T14:58:39.370 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.010887841000000001 2024-09-21T14:58:39.370 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.370 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.370 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:39.370 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.145768+0000", 2024-09-21T14:58:39.370 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0112999999999998e-05 2024-09-21T14:58:39.370 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.370 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.370 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:39.370 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146017+0000", 2024-09-21T14:58:39.370 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00024907 2024-09-21T14:58:39.370 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:39.371 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:39.371 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:39.371 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.371 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.371 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6726 3.1d 3:b9c0cfe7:::rbd_data.11361bed1f78.0000000000000010:d [read 65536~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:39.371 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.134814+0000", 2024-09-21T14:58:39.371 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.1735050730000003, 2024-09-21T14:58:39.371 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.011517806, 2024-09-21T14:58:39.371 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:39.371 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:39.371 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:39.371 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:39.371 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:39.371 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6726 2024-09-21T14:58:39.371 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.371 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:39.372 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.372 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:39.372 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134814+0000", 2024-09-21T14:58:39.372 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:39.372 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.372 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.372 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:39.372 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134814+0000", 2024-09-21T14:58:39.372 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:39.372 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.372 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.372 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:39.372 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134814+0000", 2024-09-21T14:58:39.372 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:39.372 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.372 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.373 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:39.373 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134817+0000", 2024-09-21T14:58:39.373 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7389999999999999e-06 2024-09-21T14:58:39.373 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.373 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.373 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:39.373 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134822+0000", 2024-09-21T14:58:39.373 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6169999999999998e-06 2024-09-21T14:58:39.373 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.373 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.373 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:39.373 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134827+0000", 2024-09-21T14:58:39.373 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4660000000000001e-06 2024-09-21T14:58:39.373 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.373 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.374 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:39.374 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146039+0000", 2024-09-21T14:58:39.374 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.01121221 2024-09-21T14:58:39.374 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.374 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.374 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:39.374 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146112+0000", 2024-09-21T14:58:39.374 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2615999999999997e-05 2024-09-21T14:58:39.374 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.374 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.374 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:39.374 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146332+0000", 2024-09-21T14:58:39.374 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00022061300000000001 2024-09-21T14:58:39.374 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:39.374 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:39.375 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:39.375 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.375 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.375 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6737 3.1d 3:b9c0cfe7:::rbd_data.11361bed1f78.0000000000000010:d [read 81920~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:39.375 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.134866+0000", 2024-09-21T14:58:39.375 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.1734531510000004, 2024-09-21T14:58:39.375 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.011799451000000001, 2024-09-21T14:58:39.375 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:39.375 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:39.375 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:39.375 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:39.375 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:39.375 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6737 2024-09-21T14:58:39.375 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.376 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:39.376 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.376 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:39.376 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134866+0000", 2024-09-21T14:58:39.376 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:39.376 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.376 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.376 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:39.376 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134866+0000", 2024-09-21T14:58:39.376 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:39.376 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.376 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.376 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:39.376 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134866+0000", 2024-09-21T14:58:39.376 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:39.377 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.377 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.377 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:39.377 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134868+0000", 2024-09-21T14:58:39.377 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.401e-06 2024-09-21T14:58:39.377 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.377 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.377 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:39.377 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134874+0000", 2024-09-21T14:58:39.377 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8000000000000004e-06 2024-09-21T14:58:39.377 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.377 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.377 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:39.377 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134880+0000", 2024-09-21T14:58:39.377 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8329999999999999e-06 2024-09-21T14:58:39.378 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.378 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.378 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:39.378 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146355+0000", 2024-09-21T14:58:39.378 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.011474652 2024-09-21T14:58:39.378 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.378 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.378 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:39.378 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146424+0000", 2024-09-21T14:58:39.378 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9054000000000007e-05 2024-09-21T14:58:39.378 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.378 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.378 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:39.379 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146666+0000", 2024-09-21T14:58:39.379 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000242091 2024-09-21T14:58:39.379 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:39.379 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:39.379 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:39.379 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.379 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.379 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6738 3.1d 3:b9c0cfe7:::rbd_data.11361bed1f78.0000000000000010:d [read 147456~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:39.379 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.134885+0000", 2024-09-21T14:58:39.379 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.1734349630000001, 2024-09-21T14:58:39.379 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.011996183000000001, 2024-09-21T14:58:39.379 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:39.379 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:39.379 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:39.380 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:39.380 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:39.380 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6738 2024-09-21T14:58:39.380 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.380 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:39.380 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.380 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:39.380 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134885+0000", 2024-09-21T14:58:39.380 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:39.380 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.380 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.380 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:39.380 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134885+0000", 2024-09-21T14:58:39.380 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:39.380 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.380 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.381 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:39.381 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134884+0000", 2024-09-21T14:58:39.381 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:39.381 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.381 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.381 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:39.381 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134887+0000", 2024-09-21T14:58:39.381 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2649999999999999e-06 2024-09-21T14:58:39.381 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.381 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.381 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:39.381 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134892+0000", 2024-09-21T14:58:39.381 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9479999999999996e-06 2024-09-21T14:58:39.381 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.381 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.382 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:39.382 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134897+0000", 2024-09-21T14:58:39.382 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8849999999999998e-06 2024-09-21T14:58:39.382 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.382 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.382 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:39.382 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146691+0000", 2024-09-21T14:58:39.382 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.011794001 2024-09-21T14:58:39.382 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.382 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.382 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:39.382 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146793+0000", 2024-09-21T14:58:39.382 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010184 2024-09-21T14:58:39.382 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.383 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.383 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:39.383 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146881+0000", 2024-09-21T14:58:39.383 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.7563000000000002e-05 2024-09-21T14:58:39.383 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:39.383 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:39.383 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:39.383 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.383 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.383 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6739 3.1d 3:b9c0cfe7:::rbd_data.11361bed1f78.0000000000000010:d [read 229376~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:39.383 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.134904+0000", 2024-09-21T14:58:39.383 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.1734155629999998, 2024-09-21T14:58:39.383 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012287384, 2024-09-21T14:58:39.383 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:39.384 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:39.384 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:39.384 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:39.384 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:39.384 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6739 2024-09-21T14:58:39.384 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.384 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:39.384 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.384 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:39.384 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134904+0000", 2024-09-21T14:58:39.384 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:39.384 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.384 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.384 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:39.385 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134904+0000", 2024-09-21T14:58:39.385 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:39.385 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.385 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.385 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:39.385 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134904+0000", 2024-09-21T14:58:39.385 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:39.385 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.385 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.385 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:39.385 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134906+0000", 2024-09-21T14:58:39.385 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5229999999999998e-06 2024-09-21T14:58:39.385 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.385 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.385 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:39.385 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134912+0000", 2024-09-21T14:58:39.386 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3199999999999996e-06 2024-09-21T14:58:39.386 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.386 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.386 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:39.386 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.134918+0000", 2024-09-21T14:58:39.386 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0560000000000001e-06 2024-09-21T14:58:39.386 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.386 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.386 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:39.386 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146905+0000", 2024-09-21T14:58:39.386 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.011986949 2024-09-21T14:58:39.386 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.386 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.387 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:39.387 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146975+0000", 2024-09-21T14:58:39.387 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9806000000000002e-05 2024-09-21T14:58:39.387 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.387 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.387 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:39.387 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147191+0000", 2024-09-21T14:58:39.387 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000216097 2024-09-21T14:58:39.387 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:39.387 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:39.387 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:39.387 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.387 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.387 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6754 3.d 3:b04809f7:::rbd_data.11361bed1f78.0000000000000017:d [read 32768~32768 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:39.388 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.135064+0000", 2024-09-21T14:58:39.388 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.1732556069999998, 2024-09-21T14:58:39.388 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012621561999999999, 2024-09-21T14:58:39.388 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:39.388 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:39.388 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:39.388 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:39.388 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:39.388 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6754 2024-09-21T14:58:39.388 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.388 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:39.388 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.388 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:39.388 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.135064+0000", 2024-09-21T14:58:39.389 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:39.389 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.389 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.389 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:39.389 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.135064+0000", 2024-09-21T14:58:39.389 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:39.389 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.389 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.389 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:39.389 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.135064+0000", 2024-09-21T14:58:39.389 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:39.389 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.389 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.390 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:39.390 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.135068+0000", 2024-09-21T14:58:39.390 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5630000000000002e-06 2024-09-21T14:58:39.390 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.390 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.390 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:39.390 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.135074+0000", 2024-09-21T14:58:39.390 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1979999999999998e-06 2024-09-21T14:58:39.390 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.390 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.390 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:39.390 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.135080+0000", 2024-09-21T14:58:39.390 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.147e-06 2024-09-21T14:58:39.390 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.391 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.391 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:39.391 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147380+0000", 2024-09-21T14:58:39.391 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012300075000000001 2024-09-21T14:58:39.391 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.391 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.391 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:39.391 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147460+0000", 2024-09-21T14:58:39.391 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0289000000000003e-05 2024-09-21T14:58:39.391 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.391 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.391 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:39.392 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147685+0000", 2024-09-21T14:58:39.392 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00022560899999999999 2024-09-21T14:58:39.392 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:39.392 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:39.392 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:39.392 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.392 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.392 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6756 3.d 3:b2f690c6:::rbd_data.11361bed1f78.0000000000000019:d [read 32768~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:39.392 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.135084+0000", 2024-09-21T14:58:39.392 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.1732350660000002, 2024-09-21T14:58:39.392 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013162790000000001, 2024-09-21T14:58:39.392 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:39.392 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:39.392 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:39.392 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:39.393 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:39.393 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6756 2024-09-21T14:58:39.393 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.393 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:39.393 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.393 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:39.393 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.135084+0000", 2024-09-21T14:58:39.393 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:39.393 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.393 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.393 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:39.393 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.135084+0000", 2024-09-21T14:58:39.393 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:39.393 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.393 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.393 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:39.394 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.135084+0000", 2024-09-21T14:58:39.394 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:39.394 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.394 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.394 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:39.394 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.135087+0000", 2024-09-21T14:58:39.394 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.768e-06 2024-09-21T14:58:39.394 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.394 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.394 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:39.394 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.135092+0000", 2024-09-21T14:58:39.394 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4410000000000001e-06 2024-09-21T14:58:39.394 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.394 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.394 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:39.395 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.135097+0000", 2024-09-21T14:58:39.395 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6430000000000003e-06 2024-09-21T14:58:39.395 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.395 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.395 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:39.395 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147922+0000", 2024-09-21T14:58:39.395 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012824991000000001 2024-09-21T14:58:39.395 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.395 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.395 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:39.395 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.148033+0000", 2024-09-21T14:58:39.395 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000111394 2024-09-21T14:58:39.395 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:39.395 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:39.395 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:39.396 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.148247+0000", 2024-09-21T14:58:39.396 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00021402500000000001 2024-09-21T14:58:39.396 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:39.396 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:39.396 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:39.396 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:39.396 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:39.396 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:58:39.397 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T14:58:39.398 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:58:39.396+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:58:39.499 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:58:39.496+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:58:39.505 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-21T14:58:39.510 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-09-21T14:58:39.528 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:58:39.528 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T14:58:39.528 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.529 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:11129 3.1b 3:dcc59e10:::rbd_data.11363a21d2be.0000000000000020:10 [read 81920~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:39.529 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.528231+0000", 2024-09-21T14:58:39.529 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.00042717599999999998, 2024-09-21T14:58:39.529 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00052764200000000004, 2024-09-21T14:58:39.529 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:39.529 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:39.529 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:39.529 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:39.529 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:39.529 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 11129 2024-09-21T14:58:39.529 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.529 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:39.529 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.529 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:39.530 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.528231+0000", 2024-09-21T14:58:39.530 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:39.530 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.530 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.530 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:39.530 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.528231+0000", 2024-09-21T14:58:39.530 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:39.530 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.530 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.530 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:39.530 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.528230+0000", 2024-09-21T14:58:39.530 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:39.530 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.530 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.530 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:39.531 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.528236+0000", 2024-09-21T14:58:39.531 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3700000000000003e-06 2024-09-21T14:58:39.531 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.531 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.531 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:39.531 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.528246+0000", 2024-09-21T14:58:39.531 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0108e-05 2024-09-21T14:58:39.531 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.531 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.531 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:39.531 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.528254+0000", 2024-09-21T14:58:39.531 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8949999999999998e-06 2024-09-21T14:58:39.531 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.531 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.531 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:39.531 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.528560+0000", 2024-09-21T14:58:39.532 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00030655800000000001 2024-09-21T14:58:39.532 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.532 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.532 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:39.532 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.528651+0000", 2024-09-21T14:58:39.532 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0625999999999996e-05 2024-09-21T14:58:39.532 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:39.532 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:39.532 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:39.532 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.532 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.532 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:11143 3.1b 3.879a33b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:39.532 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.528294+0000", 2024-09-21T14:58:39.532 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.00036398300000000001, 2024-09-21T14:58:39.532 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00058597200000000003, 2024-09-21T14:58:39.532 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:39.533 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:58:39.533 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:39.533 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:39.533 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:39.533 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 11143 2024-09-21T14:58:39.533 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.533 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:39.533 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.533 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:39.533 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.528294+0000", 2024-09-21T14:58:39.533 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:39.533 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.533 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.533 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:39.533 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.528294+0000", 2024-09-21T14:58:39.533 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:39.534 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.534 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.534 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:39.534 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.528294+0000", 2024-09-21T14:58:39.534 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:39.534 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.534 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.534 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:39.534 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.528297+0000", 2024-09-21T14:58:39.534 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7270000000000001e-06 2024-09-21T14:58:39.534 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.534 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.534 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:39.534 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.528306+0000", 2024-09-21T14:58:39.534 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4530000000000002e-06 2024-09-21T14:58:39.535 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.535 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.535 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:39.535 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.528314+0000", 2024-09-21T14:58:39.535 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7009999999999996e-06 2024-09-21T14:58:39.535 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.535 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.535 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:39.535 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.528869+0000", 2024-09-21T14:58:39.535 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00055589099999999998 2024-09-21T14:58:39.535 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:39.535 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:39.535 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:39.535 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.535 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.535 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:11155 3.1b 3.879a33b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:39.536 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.528911+0000", 2024-09-21T14:58:39.536 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4294967295.9997478, 2024-09-21T14:58:39.536 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3530999999999998e-05, 2024-09-21T14:58:39.536 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:39.536 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-21T14:58:39.536 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:39.536 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:39.536 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:39.536 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 11155 2024-09-21T14:58:39.536 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.536 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:39.536 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.536 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:39.536 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.528911+0000", 2024-09-21T14:58:39.536 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:39.536 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.537 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.537 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:39.537 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.528911+0000", 2024-09-21T14:58:39.537 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:39.537 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.537 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.537 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:39.537 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.528910+0000", 2024-09-21T14:58:39.537 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:39.537 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.537 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.537 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:39.537 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.528915+0000", 2024-09-21T14:58:39.537 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7030000000000003e-06 2024-09-21T14:58:39.537 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.538 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.538 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:39.538 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.528926+0000", 2024-09-21T14:58:39.538 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0849999999999999e-05 2024-09-21T14:58:39.538 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.538 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.538 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:39.538 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.528936+0000", 2024-09-21T14:58:39.538 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0217999999999999e-05 2024-09-21T14:58:39.538 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:39.538 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:39.538 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:39.538 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:39.538 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-21T14:58:39.538 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 3 2024-09-21T14:58:39.538 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:58:39.539 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":56,"stamp":"2024-09-21T14:58:39.117987+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":84298956,"num_objects":215,"num_object_clones":0,"num_object_copies":430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":215,"num_whiteouts":0,"num_read":9111,"num_read_kb":48251,"num_write":2004,"num_write_kb":49649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1484,"ondisk_log_size":1484,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":278856,"kb_used_data":9392,"kb_used_omap":12,"kb_used_meta":269299,"kb_avail":754695864,"statfs":{"total":773094113280,"available":772808564736,"internally_reserved":0,"allocated":9617408,"data_stored":69126746,"data_compressed":574746,"data_compressed_allocated":5218304,"data_compressed_original":67273236,"omap_allocated":12744,"internal_metadata":275762744},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":65075381,"num_objects":186,"num_object_clones":0,"num_object_copies":372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":186,"num_whiteouts":0,"num_read":6913,"num_read_kb":46193,"num_write":1422,"num_write_kb":45281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.609373"},"pg_stats":[{"pgid":"3.1f","version":"41'113","reported_seq":281,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T14:58:35.291875+0000","last_change":"2024-09-21T14:58:33.510215+0000","last_active":"2024-09-21T14:58:35.291875+0000","last_peered":"2024-09-21T14:58:35.291875+0000","last_clean":"2024-09-21T14:58:35.291875+0000","last_became_active":"2024-09-21T14:58:19.410055+0000","last_became_peered":"2024-09-21T14:58:19.410055+0000","last_unstale":"2024-09-21T14:58:35.291875+0000","last_undegraded":"2024-09-21T14:58:35.291875+0000","last_fullsized":"2024-09-21T14:58:35.291875+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":113,"log_dups_size":0,"ondisk_log_size":113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:49:13.944219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0044157320000000003,"stat_sum":{"num_bytes":524304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":632,"num_read_kb":521,"num_write":172,"num_write_kb":854,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"b","length":"1"}]},{"pgid":"3.1e","version":"41'30","reported_seq":160,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T14:58:35.198115+0000","last_change":"2024-09-21T14:58:33.510458+0000","last_active":"2024-09-21T14:58:35.198115+0000","last_peered":"2024-09-21T14:58:35.198115+0000","last_clean":"2024-09-21T14:58:35.198115+0000","last_became_active":"2024-09-21T14:58:19.412353+0000","last_became_peered":"2024-09-21T14:58:19.412353+0000","last_unstale":"2024-09-21T14:58:35.198115+0000","last_undegraded":"2024-09-21T14:58:35.198115+0000","last_fullsized":"2024-09-21T14:58:35.198115+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:38:15.532795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041653640000000004,"stat_sum":{"num_bytes":3795966,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":87,"num_read_kb":426,"num_write":30,"num_write_kb":1769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"b","length":"1"}]},{"pgid":"3.1d","version":"41'26","reported_seq":377,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T14:58:35.197026+0000","last_change":"2024-09-21T14:58:33.506738+0000","last_active":"2024-09-21T14:58:35.197026+0000","last_peered":"2024-09-21T14:58:35.197026+0000","last_clean":"2024-09-21T14:58:35.197026+0000","last_became_active":"2024-09-21T14:58:19.411387+0000","last_became_peered":"2024-09-21T14:58:19.411387+0000","last_unstale":"2024-09-21T14:58:35.197026+0000","last_undegraded":"2024-09-21T14:58:35.197026+0000","last_fullsized":"2024-09-21T14:58:35.197026+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:37:57.597069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013229439999999999,"stat_sum":{"num_bytes":4449608,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":309,"num_read_kb":3704,"num_write":27,"num_write_kb":1464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"b","length":"1"}]},{"pgid":"3.1c","version":"41'18","reported_seq":85,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T14:58:35.239193+0000","last_change":"2024-09-21T14:58:33.511110+0000","last_active":"2024-09-21T14:58:35.239193+0000","last_peered":"2024-09-21T14:58:35.239193+0000","last_clean":"2024-09-21T14:58:35.239193+0000","last_became_active":"2024-09-21T14:58:19.411306+0000","last_became_peered":"2024-09-21T14:58:19.411306+0000","last_unstale":"2024-09-21T14:58:35.239193+0000","last_undegraded":"2024-09-21T14:58:35.239193+0000","last_fullsized":"2024-09-21T14:58:35.239193+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:20:27.079795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089448299999999995,"stat_sum":{"num_bytes":786432,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":34,"num_read_kb":266,"num_write":18,"num_write_kb":1039,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"b","length":"1"}]},{"pgid":"3.1b","version":"41'23","reported_seq":114,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T14:58:35.205550+0000","last_change":"2024-09-21T14:58:33.515860+0000","last_active":"2024-09-21T14:58:35.205550+0000","last_peered":"2024-09-21T14:58:35.205550+0000","last_clean":"2024-09-21T14:58:35.205550+0000","last_became_active":"2024-09-21T14:58:19.413053+0000","last_became_peered":"2024-09-21T14:58:19.413053+0000","last_unstale":"2024-09-21T14:58:35.205550+0000","last_undegraded":"2024-09-21T14:58:35.205550+0000","last_fullsized":"2024-09-21T14:58:35.205550+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:07:12.901354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065231700000000002,"stat_sum":{"num_bytes":966656,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":48,"num_read_kb":432,"num_write":24,"num_write_kb":1328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"b","length":"1"}]},{"pgid":"3.1a","version":"41'35","reported_seq":282,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T14:58:35.288694+0000","last_change":"2024-09-21T14:58:33.510448+0000","last_active":"2024-09-21T14:58:35.288694+0000","last_peered":"2024-09-21T14:58:35.288694+0000","last_clean":"2024-09-21T14:58:35.288694+0000","last_became_active":"2024-09-21T14:58:19.409415+0000","last_became_peered":"2024-09-21T14:58:19.409415+0000","last_unstale":"2024-09-21T14:58:35.288694+0000","last_undegraded":"2024-09-21T14:58:35.288694+0000","last_fullsized":"2024-09-21T14:58:35.288694+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:28:46.336301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043556999999999998,"stat_sum":{"num_bytes":1409043,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":279,"num_read_kb":1170,"num_write":39,"num_write_kb":1284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"b","length":"1"}]},{"pgid":"3.19","version":"41'63","reported_seq":453,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T14:58:35.241276+0000","last_change":"2024-09-21T14:58:33.516001+0000","last_active":"2024-09-21T14:58:35.241276+0000","last_peered":"2024-09-21T14:58:35.241276+0000","last_clean":"2024-09-21T14:58:35.241276+0000","last_became_active":"2024-09-21T14:58:19.410062+0000","last_became_peered":"2024-09-21T14:58:19.410062+0000","last_unstale":"2024-09-21T14:58:35.241276+0000","last_undegraded":"2024-09-21T14:58:35.241276+0000","last_fullsized":"2024-09-21T14:58:35.241276+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:12:02.923321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077683600000000004,"stat_sum":{"num_bytes":1754513,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":715,"num_read_kb":1316,"num_write":63,"num_write_kb":1702,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"b","length":"1"}]},{"pgid":"3.18","version":"41'14","reported_seq":133,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T14:58:35.234012+0000","last_change":"2024-09-21T14:58:33.510109+0000","last_active":"2024-09-21T14:58:35.234012+0000","last_peered":"2024-09-21T14:58:35.234012+0000","last_clean":"2024-09-21T14:58:35.234012+0000","last_became_active":"2024-09-21T14:58:19.409371+0000","last_became_peered":"2024-09-21T14:58:19.409371+0000","last_unstale":"2024-09-21T14:58:35.234012+0000","last_undegraded":"2024-09-21T14:58:35.234012+0000","last_fullsized":"2024-09-21T14:58:35.234012+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:35:27.143445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041105439999999998,"stat_sum":{"num_bytes":979620,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":76,"num_read_kb":749,"num_write":15,"num_write_kb":816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"b","length":"1"}]},{"pgid":"3.17","version":"41'20","reported_seq":75,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T14:58:35.162905+0000","last_change":"2024-09-21T14:58:33.510397+0000","last_active":"2024-09-21T14:58:35.162905+0000","last_peered":"2024-09-21T14:58:35.162905+0000","last_clean":"2024-09-21T14:58:35.162905+0000","last_became_active":"2024-09-21T14:58:19.410306+0000","last_became_peered":"2024-09-21T14:58:19.410306+0000","last_unstale":"2024-09-21T14:58:35.162905+0000","last_undegraded":"2024-09-21T14:58:35.162905+0000","last_fullsized":"2024-09-21T14:58:35.162905+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:33:50.089165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0042075300000000001,"stat_sum":{"num_bytes":458752,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":64,"num_write":20,"num_write_kb":876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"b","length":"1"}]},{"pgid":"3.16","version":"41'49","reported_seq":259,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T14:58:35.247372+0000","last_change":"2024-09-21T14:58:33.584269+0000","last_active":"2024-09-21T14:58:35.247372+0000","last_peered":"2024-09-21T14:58:35.247372+0000","last_clean":"2024-09-21T14:58:35.247372+0000","last_became_active":"2024-09-21T14:58:19.413935+0000","last_became_peered":"2024-09-21T14:58:19.413935+0000","last_unstale":"2024-09-21T14:58:35.247372+0000","last_undegraded":"2024-09-21T14:58:35.247372+0000","last_fullsized":"2024-09-21T14:58:35.247372+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:15:28.307418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034519100000000001,"stat_sum":{"num_bytes":1565058,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":345,"num_read_kb":998,"num_write":48,"num_write_kb":1236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"b","length":"1"}]},{"pgid":"3.15","version":"41'86","reported_seq":281,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T14:58:35.231611+0000","last_change":"2024-09-21T14:58:33.583891+0000","last_active":"2024-09-21T14:58:35.231611+0000","last_peered":"2024-09-21T14:58:35.231611+0000","last_clean":"2024-09-21T14:58:35.231611+0000","last_became_active":"2024-09-21T14:58:19.410687+0000","last_became_peered":"2024-09-21T14:58:19.410687+0000","last_unstale":"2024-09-21T14:58:35.231611+0000","last_undegraded":"2024-09-21T14:58:35.231611+0000","last_fullsized":"2024-09-21T14:58:35.231611+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:21:42.226982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059394000000000005,"stat_sum":{"num_bytes":1698608,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":377,"num_read_kb":1341,"num_write":113,"num_write_kb":1710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"b","length":"1"}]},{"pgid":"3.14","version":"39'111","reported_seq":160,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T14:58:35.125345+0000","last_change":"2024-09-21T14:58:33.506814+0000","last_active":"2024-09-21T14:58:35.125345+0000","last_peered":"2024-09-21T14:58:35.125345+0000","last_clean":"2024-09-21T14:58:35.125345+0000","last_became_active":"2024-09-21T14:58:19.407310+0000","last_became_peered":"2024-09-21T14:58:19.407310+0000","last_unstale":"2024-09-21T14:58:35.125345+0000","last_undegraded":"2024-09-21T14:58:35.125345+0000","last_fullsized":"2024-09-21T14:58:35.125345+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:47:46.079115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001364482,"stat_sum":{"num_bytes":49,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":452,"num_read_kb":360,"num_write":170,"num_write_kb":452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"b","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":34,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T14:58:33.505919+0000","last_change":"2024-09-21T14:58:11.348177+0000","last_active":"2024-09-21T14:58:33.505919+0000","last_peered":"2024-09-21T14:58:33.505919+0000","last_clean":"2024-09-21T14:58:33.505919+0000","last_became_active":"2024-09-21T14:58:09.328417+0000","last_became_peered":"2024-09-21T14:58:09.328417+0000","last_unstale":"2024-09-21T14:58:33.505919+0000","last_undegraded":"2024-09-21T14:58:33.505919+0000","last_fullsized":"2024-09-21T14:58:33.505919+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:52:22.671185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018917490000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"41'69","reported_seq":363,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T14:58:35.250307+0000","last_change":"2024-09-21T14:58:33.515728+0000","last_active":"2024-09-21T14:58:35.250307+0000","last_peered":"2024-09-21T14:58:35.250307+0000","last_clean":"2024-09-21T14:58:35.250307+0000","last_became_active":"2024-09-21T14:58:19.403868+0000","last_became_peered":"2024-09-21T14:58:19.403868+0000","last_unstale":"2024-09-21T14:58:35.250307+0000","last_undegraded":"2024-09-21T14:58:35.250307+0000","last_fullsized":"2024-09-21T14:58:35.250307+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:52:08.672416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000531392,"stat_sum":{"num_bytes":6506173,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":587,"num_read_kb":1019,"num_write":70,"num_write_kb":1687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"b","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":35,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T14:58:34.514614+0000","last_change":"2024-09-21T14:58:11.405717+0000","last_active":"2024-09-21T14:58:34.514614+0000","last_peered":"2024-09-21T14:58:34.514614+0000","last_clean":"2024-09-21T14:58:34.514614+0000","last_became_active":"2024-09-21T14:58:09.326849+0000","last_became_peered":"2024-09-21T14:58:09.326849+0000","last_unstale":"2024-09-21T14:58:34.514614+0000","last_undegraded":"2024-09-21T14:58:34.514614+0000","last_fullsized":"2024-09-21T14:58:34.514614+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:29:42.893819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012560399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"41'73","reported_seq":404,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T14:58:35.249518+0000","last_change":"2024-09-21T14:58:33.516512+0000","last_active":"2024-09-21T14:58:35.249518+0000","last_peered":"2024-09-21T14:58:35.249518+0000","last_clean":"2024-09-21T14:58:35.249518+0000","last_became_active":"2024-09-21T14:58:19.410954+0000","last_became_peered":"2024-09-21T14:58:19.410954+0000","last_unstale":"2024-09-21T14:58:35.249518+0000","last_undegraded":"2024-09-21T14:58:35.249518+0000","last_fullsized":"2024-09-21T14:58:35.249518+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:26:59.084415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011156860000000001,"stat_sum":{"num_bytes":3626706,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":519,"num_read_kb":3004,"num_write":108,"num_write_kb":2080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"b","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":34,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T14:58:33.505914+0000","last_change":"2024-09-21T14:58:11.348182+0000","last_active":"2024-09-21T14:58:33.505914+0000","last_peered":"2024-09-21T14:58:33.505914+0000","last_clean":"2024-09-21T14:58:33.505914+0000","last_became_active":"2024-09-21T14:58:09.329036+0000","last_became_peered":"2024-09-21T14:58:09.329036+0000","last_unstale":"2024-09-21T14:58:33.505914+0000","last_undegraded":"2024-09-21T14:58:33.505914+0000","last_fullsized":"2024-09-21T14:58:33.505914+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:24:45.719970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019338770000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"41'37","reported_seq":116,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T14:58:35.231878+0000","last_change":"2024-09-21T14:58:33.506912+0000","last_active":"2024-09-21T14:58:35.231878+0000","last_peered":"2024-09-21T14:58:35.231878+0000","last_clean":"2024-09-21T14:58:35.231878+0000","last_became_active":"2024-09-21T14:58:19.409130+0000","last_became_peered":"2024-09-21T14:58:19.409130+0000","last_unstale":"2024-09-21T14:58:35.231878+0000","last_undegraded":"2024-09-21T14:58:35.231878+0000","last_fullsized":"2024-09-21T14:58:35.231878+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:47:27.991651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042473000000000001,"stat_sum":{"num_bytes":2179072,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":73,"num_read_kb":366,"num_write":46,"num_write_kb":2575,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"b","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":35,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T14:58:34.518918+0000","last_change":"2024-09-21T14:58:11.406176+0000","last_active":"2024-09-21T14:58:34.518918+0000","last_peered":"2024-09-21T14:58:34.518918+0000","last_clean":"2024-09-21T14:58:34.518918+0000","last_became_active":"2024-09-21T14:58:09.777465+0000","last_became_peered":"2024-09-21T14:58:09.777465+0000","last_unstale":"2024-09-21T14:58:34.518918+0000","last_undegraded":"2024-09-21T14:58:34.518918+0000","last_fullsized":"2024-09-21T14:58:34.518918+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:55:04.667816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095704000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"41'11","reported_seq":118,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T14:58:35.247318+0000","last_change":"2024-09-21T14:58:33.582915+0000","last_active":"2024-09-21T14:58:35.247318+0000","last_peered":"2024-09-21T14:58:35.247318+0000","last_clean":"2024-09-21T14:58:35.247318+0000","last_became_active":"2024-09-21T14:58:19.408971+0000","last_became_peered":"2024-09-21T14:58:19.408971+0000","last_unstale":"2024-09-21T14:58:35.247318+0000","last_undegraded":"2024-09-21T14:58:35.247318+0000","last_fullsized":"2024-09-21T14:58:35.247318+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:29:52.642121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048702400000000002,"stat_sum":{"num_bytes":1425424,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":69,"num_read_kb":627,"num_write":12,"num_write_kb":1329,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"b","length":"1"}]},{"pgid":"3.3","version":"41'16","reported_seq":371,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T14:58:35.222876+0000","last_change":"2024-09-21T14:58:33.583610+0000","last_active":"2024-09-21T14:58:35.222876+0000","last_peered":"2024-09-21T14:58:35.222876+0000","last_clean":"2024-09-21T14:58:35.222876+0000","last_became_active":"2024-09-21T14:58:19.417453+0000","last_became_peered":"2024-09-21T14:58:19.417453+0000","last_unstale":"2024-09-21T14:58:35.222876+0000","last_undegraded":"2024-09-21T14:58:35.222876+0000","last_fullsized":"2024-09-21T14:58:35.222876+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:19:27.087272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001562419,"stat_sum":{"num_bytes":5070446,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":312,"num_read_kb":3640,"num_write":18,"num_write_kb":1990,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"b","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":36,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T14:58:33.505421+0000","last_change":"2024-09-21T14:58:11.347504+0000","last_active":"2024-09-21T14:58:33.505421+0000","last_peered":"2024-09-21T14:58:33.505421+0000","last_clean":"2024-09-21T14:58:33.505421+0000","last_became_active":"2024-09-21T14:58:09.775981+0000","last_became_peered":"2024-09-21T14:58:09.775981+0000","last_unstale":"2024-09-21T14:58:33.505421+0000","last_undegraded":"2024-09-21T14:58:33.505421+0000","last_fullsized":"2024-09-21T14:58:33.505421+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:47:16.286652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012568880000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"41'16","reported_seq":135,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T14:58:35.248417+0000","last_change":"2024-09-21T14:58:33.510670+0000","last_active":"2024-09-21T14:58:35.248417+0000","last_peered":"2024-09-21T14:58:35.248417+0000","last_clean":"2024-09-21T14:58:35.248417+0000","last_became_active":"2024-09-21T14:58:19.405811+0000","last_became_peered":"2024-09-21T14:58:19.405811+0000","last_unstale":"2024-09-21T14:58:35.248417+0000","last_undegraded":"2024-09-21T14:58:35.248417+0000","last_fullsized":"2024-09-21T14:58:35.248417+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:36:25.551607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041792640000000002,"stat_sum":{"num_bytes":1343579,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":83,"num_read_kb":805,"num_write":18,"num_write_kb":1153,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"b","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":36,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T14:58:35.149448+0000","last_change":"2024-09-21T14:58:11.342210+0000","last_active":"2024-09-21T14:58:35.149448+0000","last_peered":"2024-09-21T14:58:35.149448+0000","last_clean":"2024-09-21T14:58:35.149448+0000","last_became_active":"2024-09-21T14:58:09.330624+0000","last_became_peered":"2024-09-21T14:58:09.330624+0000","last_unstale":"2024-09-21T14:58:35.149448+0000","last_undegraded":"2024-09-21T14:58:35.149448+0000","last_fullsized":"2024-09-21T14:58:35.149448+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:29:43.229538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001051151,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"41'20","reported_seq":110,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T14:58:35.212252+0000","last_change":"2024-09-21T14:58:33.506165+0000","last_active":"2024-09-21T14:58:35.212252+0000","last_peered":"2024-09-21T14:58:35.212252+0000","last_clean":"2024-09-21T14:58:35.212252+0000","last_became_active":"2024-09-21T14:58:19.417529+0000","last_became_peered":"2024-09-21T14:58:19.417529+0000","last_unstale":"2024-09-21T14:58:35.212252+0000","last_undegraded":"2024-09-21T14:58:35.212252+0000","last_fullsized":"2024-09-21T14:58:35.212252+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:38:57.961654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055905500000000003,"stat_sum":{"num_bytes":1097728,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":47,"num_read_kb":461,"num_write":21,"num_write_kb":1385,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"b","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":34,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T14:58:33.504045+0000","last_change":"2024-09-21T14:58:11.346766+0000","last_active":"2024-09-21T14:58:33.504045+0000","last_peered":"2024-09-21T14:58:33.504045+0000","last_clean":"2024-09-21T14:58:33.504045+0000","last_became_active":"2024-09-21T14:58:09.329748+0000","last_became_peered":"2024-09-21T14:58:09.329748+0000","last_unstale":"2024-09-21T14:58:33.504045+0000","last_undegraded":"2024-09-21T14:58:33.504045+0000","last_fullsized":"2024-09-21T14:58:33.504045+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:31:20.267675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052698000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"41'17","reported_seq":624,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T14:58:35.234223+0000","last_change":"2024-09-21T14:58:33.515506+0000","last_active":"2024-09-21T14:58:35.234223+0000","last_peered":"2024-09-21T14:58:35.234223+0000","last_clean":"2024-09-21T14:58:35.234223+0000","last_became_active":"2024-09-21T14:58:19.416313+0000","last_became_peered":"2024-09-21T14:58:19.416313+0000","last_unstale":"2024-09-21T14:58:35.234223+0000","last_undegraded":"2024-09-21T14:58:35.234223+0000","last_fullsized":"2024-09-21T14:58:35.234223+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:34:38.858491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038044400000000002,"stat_sum":{"num_bytes":9065488,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":570,"num_read_kb":4709,"num_write":16,"num_write_kb":1180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"b","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":35,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-21T14:58:33.506092+0000","last_change":"2024-09-21T14:58:11.347700+0000","last_active":"2024-09-21T14:58:33.506092+0000","last_peered":"2024-09-21T14:58:33.506092+0000","last_clean":"2024-09-21T14:58:33.506092+0000","last_became_active":"2024-09-21T14:58:09.438842+0000","last_became_peered":"2024-09-21T14:58:09.438842+0000","last_unstale":"2024-09-21T14:58:33.506092+0000","last_undegraded":"2024-09-21T14:58:33.506092+0000","last_fullsized":"2024-09-21T14:58:33.506092+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:41:34.626249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001426194,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":183,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T14:58:34.519849+0000","last_change":"2024-09-21T14:58:03.616055+0000","last_active":"2024-09-21T14:58:34.519849+0000","last_peered":"2024-09-21T14:58:34.519849+0000","last_clean":"2024-09-21T14:58:34.519849+0000","last_became_active":"2024-09-21T14:58:03.613214+0000","last_became_peered":"2024-09-21T14:58:03.613214+0000","last_unstale":"2024-09-21T14:58:34.519849+0000","last_undegraded":"2024-09-21T14:58:34.519849+0000","last_fullsized":"2024-09-21T14:58:34.519849+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:02.180162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:02.180162+0000","last_clean_scrub_stamp":"2024-09-21T14:58:02.180162+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:29:05.860361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"40'23","reported_seq":731,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T14:58:35.266446+0000","last_change":"2024-09-21T14:58:33.515448+0000","last_active":"2024-09-21T14:58:35.266446+0000","last_peered":"2024-09-21T14:58:35.266446+0000","last_clean":"2024-09-21T14:58:35.266446+0000","last_became_active":"2024-09-21T14:58:19.403887+0000","last_became_peered":"2024-09-21T14:58:19.403887+0000","last_unstale":"2024-09-21T14:58:35.266446+0000","last_undegraded":"2024-09-21T14:58:35.266446+0000","last_fullsized":"2024-09-21T14:58:35.266446+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:25:35.185774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035317900000000001,"stat_sum":{"num_bytes":7272595,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":669,"num_read_kb":7170,"num_write":23,"num_write_kb":845,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"b","length":"1"}]},{"pgid":"3.9","version":"41'19","reported_seq":146,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T14:58:35.245371+0000","last_change":"2024-09-21T14:58:33.505043+0000","last_active":"2024-09-21T14:58:35.245371+0000","last_peered":"2024-09-21T14:58:35.245371+0000","last_clean":"2024-09-21T14:58:35.245371+0000","last_became_active":"2024-09-21T14:58:19.399712+0000","last_became_peered":"2024-09-21T14:58:19.399712+0000","last_unstale":"2024-09-21T14:58:35.245371+0000","last_undegraded":"2024-09-21T14:58:35.245371+0000","last_fullsized":"2024-09-21T14:58:35.245371+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:50:47.558139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065766999999999996,"stat_sum":{"num_bytes":2498158,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":85,"num_read_kb":526,"num_write":22,"num_write_kb":1597,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"b","length":"1"}]},{"pgid":"3.a","version":"41'24","reported_seq":388,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T14:58:35.222222+0000","last_change":"2024-09-21T14:58:33.506138+0000","last_active":"2024-09-21T14:58:35.222222+0000","last_peered":"2024-09-21T14:58:35.222222+0000","last_clean":"2024-09-21T14:58:35.222222+0000","last_became_active":"2024-09-21T14:58:19.407847+0000","last_became_peered":"2024-09-21T14:58:19.407847+0000","last_unstale":"2024-09-21T14:58:35.222222+0000","last_undegraded":"2024-09-21T14:58:35.222222+0000","last_fullsized":"2024-09-21T14:58:35.222222+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:45:44.979236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017328300000000001,"stat_sum":{"num_bytes":6330904,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":322,"num_read_kb":3111,"num_write":28,"num_write_kb":2842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"b","length":"1"}]},{"pgid":"3.b","version":"41'26","reported_seq":160,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-21T14:58:36.831499+0000","last_change":"2024-09-21T14:58:36.107267+0000","last_active":"2024-09-21T14:58:36.831499+0000","last_peered":"2024-09-21T14:58:36.831499+0000","last_clean":"2024-09-21T14:58:36.831499+0000","last_became_active":"2024-09-21T14:58:19.410679+0000","last_became_peered":"2024-09-21T14:58:19.410679+0000","last_unstale":"2024-09-21T14:58:36.831499+0000","last_undegraded":"2024-09-21T14:58:36.831499+0000","last_fullsized":"2024-09-21T14:58:36.831499+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:07:40.758932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002543399,"stat_sum":{"num_bytes":2774663,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":86,"num_read_kb":883,"num_write":28,"num_write_kb":2555,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"b","length":"1"},{"start":"e","length":"1"}]},{"pgid":"3.c","version":"41'21","reported_seq":461,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T14:58:35.246746+0000","last_change":"2024-09-21T14:58:33.583530+0000","last_active":"2024-09-21T14:58:35.246746+0000","last_peered":"2024-09-21T14:58:35.246746+0000","last_clean":"2024-09-21T14:58:35.246746+0000","last_became_active":"2024-09-21T14:58:19.411026+0000","last_became_peered":"2024-09-21T14:58:19.411026+0000","last_unstale":"2024-09-21T14:58:35.246746+0000","last_undegraded":"2024-09-21T14:58:35.246746+0000","last_fullsized":"2024-09-21T14:58:35.246746+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:38:03.448592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061813000000000005,"stat_sum":{"num_bytes":5975565,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":397,"num_read_kb":4926,"num_write":23,"num_write_kb":1797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"b","length":"1"}]},{"pgid":"3.d","version":"41'8","reported_seq":117,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T14:58:35.171081+0000","last_change":"2024-09-21T14:58:33.507175+0000","last_active":"2024-09-21T14:58:35.171081+0000","last_peered":"2024-09-21T14:58:35.171081+0000","last_clean":"2024-09-21T14:58:35.171081+0000","last_became_active":"2024-09-21T14:58:19.407884+0000","last_became_peered":"2024-09-21T14:58:19.407884+0000","last_unstale":"2024-09-21T14:58:35.171081+0000","last_undegraded":"2024-09-21T14:58:35.171081+0000","last_fullsized":"2024-09-21T14:58:35.171081+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:28:57.111615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015230409999999999,"stat_sum":{"num_bytes":933888,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":67,"num_read_kb":528,"num_write":9,"num_write_kb":864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"b","length":"1"}]},{"pgid":"3.e","version":"41'276","reported_seq":396,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T14:58:35.224116+0000","last_change":"2024-09-21T14:58:33.584412+0000","last_active":"2024-09-21T14:58:35.224116+0000","last_peered":"2024-09-21T14:58:35.224116+0000","last_clean":"2024-09-21T14:58:35.224116+0000","last_became_active":"2024-09-21T14:58:19.414418+0000","last_became_peered":"2024-09-21T14:58:19.414418+0000","last_unstale":"2024-09-21T14:58:35.224116+0000","last_undegraded":"2024-09-21T14:58:35.224116+0000","last_fullsized":"2024-09-21T14:58:35.224116+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":276,"log_dups_size":0,"ondisk_log_size":276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:36:41.583214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033136199999999998,"stat_sum":{"num_bytes":2360881,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1379,"num_read_kb":1856,"num_write":536,"num_write_kb":1622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"b","length":"1"}]},{"pgid":"3.f","version":"41'32","reported_seq":148,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T14:58:35.285809+0000","last_change":"2024-09-21T14:58:33.582014+0000","last_active":"2024-09-21T14:58:35.285809+0000","last_peered":"2024-09-21T14:58:35.285809+0000","last_clean":"2024-09-21T14:58:35.285809+0000","last_became_active":"2024-09-21T14:58:19.410406+0000","last_became_peered":"2024-09-21T14:58:19.410406+0000","last_unstale":"2024-09-21T14:58:35.285809+0000","last_undegraded":"2024-09-21T14:58:35.285809+0000","last_fullsized":"2024-09-21T14:58:35.285809+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:59:16.026939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000400412,"stat_sum":{"num_bytes":1647032,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":84,"num_read_kb":809,"num_write":32,"num_write_kb":1685,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"b","length":"1"}]},{"pgid":"3.10","version":"41'22","reported_seq":180,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T14:58:35.223187+0000","last_change":"2024-09-21T14:58:33.510443+0000","last_active":"2024-09-21T14:58:35.223187+0000","last_peered":"2024-09-21T14:58:35.223187+0000","last_clean":"2024-09-21T14:58:35.223187+0000","last_became_active":"2024-09-21T14:58:19.409935+0000","last_became_peered":"2024-09-21T14:58:19.409935+0000","last_unstale":"2024-09-21T14:58:35.223187+0000","last_undegraded":"2024-09-21T14:58:35.223187+0000","last_fullsized":"2024-09-21T14:58:35.223187+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:10:36.252629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064831999999999997,"stat_sum":{"num_bytes":1826393,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":117,"num_read_kb":506,"num_write":24,"num_write_kb":2118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"b","length":"1"}]},{"pgid":"3.11","version":"39'25","reported_seq":154,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T14:58:35.239195+0000","last_change":"2024-09-21T14:58:33.504759+0000","last_active":"2024-09-21T14:58:35.239195+0000","last_peered":"2024-09-21T14:58:35.239195+0000","last_clean":"2024-09-21T14:58:35.239195+0000","last_became_active":"2024-09-21T14:58:19.406164+0000","last_became_peered":"2024-09-21T14:58:19.406164+0000","last_unstale":"2024-09-21T14:58:35.239195+0000","last_undegraded":"2024-09-21T14:58:35.239195+0000","last_fullsized":"2024-09-21T14:58:35.239195+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:26:32.857391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059497599999999999,"stat_sum":{"num_bytes":1749340,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":87,"num_read_kb":1026,"num_write":26,"num_write_kb":1390,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"b","length":"1"}]},{"pgid":"3.12","version":"41'16","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T14:58:35.182883+0000","last_change":"2024-09-21T14:58:33.506505+0000","last_active":"2024-09-21T14:58:35.182883+0000","last_peered":"2024-09-21T14:58:35.182883+0000","last_clean":"2024-09-21T14:58:35.182883+0000","last_became_active":"2024-09-21T14:58:19.412138+0000","last_became_peered":"2024-09-21T14:58:19.412138+0000","last_unstale":"2024-09-21T14:58:35.182883+0000","last_undegraded":"2024-09-21T14:58:35.182883+0000","last_fullsized":"2024-09-21T14:58:35.182883+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:34:16.989941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001045682,"stat_sum":{"num_bytes":849493,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":12,"num_read_kb":64,"num_write":17,"num_write_kb":957,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"b","length":"1"}]},{"pgid":"3.13","version":"39'25","reported_seq":131,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T14:58:35.197227+0000","last_change":"2024-09-21T14:58:33.506236+0000","last_active":"2024-09-21T14:58:35.197227+0000","last_peered":"2024-09-21T14:58:35.197227+0000","last_clean":"2024-09-21T14:58:35.197227+0000","last_became_active":"2024-09-21T14:58:19.412030+0000","last_became_peered":"2024-09-21T14:58:19.412030+0000","last_unstale":"2024-09-21T14:58:35.197227+0000","last_undegraded":"2024-09-21T14:58:35.197227+0000","last_fullsized":"2024-09-21T14:58:35.197227+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:44:29.282391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090279499999999996,"stat_sum":{"num_bytes":786432,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":64,"num_read_kb":768,"num_write":27,"num_write_kb":1050,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"b","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":83708569,"num_objects":211,"num_object_clones":0,"num_object_copies":422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":211,"num_whiteouts":0,"num_read":8995,"num_read_kb":48151,"num_write":1843,"num_write_kb":47231,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":5836800,"data_stored":66423076,"data_compressed":572074,"data_compressed_allocated":5152768,"data_compressed_original":66138644,"omap_allocated":0,"internal_metadata":0},"log_size":1364,"ondisk_log_size":1364,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":10404,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509451,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36232,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335608,"statfs":{"total":96636764160,"available":96599662592,"internally_reserved":0,"allocated":1036288,"data_stored":6303873,"data_compressed":47097,"data_compressed_allocated":475136,"data_compressed_original":6070272,"omap_allocated":1589,"internal_metadata":36043211},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42699999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.312}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34200000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51500000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]}]},{"osd":6,"up_from":14,"seq":60129542154,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32308,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339532,"statfs":{"total":96636764160,"available":96603680768,"internally_reserved":0,"allocated":1204224,"data_stored":9653265,"data_compressed":49004,"data_compressed_allocated":700416,"data_compressed_original":9453568,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65700000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.874}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78500000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0049999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.216}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1719999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542155,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36288,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335552,"statfs":{"total":96636764160,"available":96599605248,"internally_reserved":0,"allocated":1101824,"data_stored":8372245,"data_compressed":20181,"data_compressed_allocated":573440,"data_compressed_original":8146944,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73999999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67500000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86299999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89800000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.103}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2350000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542154,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32376,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339464,"statfs":{"total":96636764160,"available":96603611136,"internally_reserved":0,"allocated":1273856,"data_stored":7776091,"data_compressed":152732,"data_compressed_allocated":671744,"data_compressed_original":7512064,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85799999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90900000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76800000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]}]},{"osd":3,"up_from":14,"seq":60129542155,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32232,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339608,"statfs":{"total":96636764160,"available":96603758592,"internally_reserved":0,"allocated":1142784,"data_stored":8571033,"data_compressed":20845,"data_compressed_allocated":602112,"data_compressed_original":8343818,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84299999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80300000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.056}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.018}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.208}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.327}]}]},{"osd":2,"up_from":14,"seq":60129542153,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36500,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335340,"statfs":{"total":96636764160,"available":96599388160,"internally_reserved":0,"allocated":1323008,"data_stored":8639755,"data_compressed":151000,"data_compressed_allocated":733184,"data_compressed_original":8385835,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1259999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81699999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78600000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.378}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]}]},{"osd":1,"up_from":14,"seq":60129542154,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36412,"kb_used_data":1200,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335428,"statfs":{"total":96636764160,"available":96599478272,"internally_reserved":0,"allocated":1228800,"data_stored":10129532,"data_compressed":40068,"data_compressed_allocated":704512,"data_compressed_original":9904128,"omap_allocated":1595,"internal_metadata":36043205},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45000000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80700000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93100000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.014}]}]},{"osd":0,"up_from":14,"seq":60129542153,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36508,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335332,"statfs":{"total":96636764160,"available":96599379968,"internally_reserved":0,"allocated":1306624,"data_stored":9680952,"data_compressed":93819,"data_compressed_allocated":757760,"data_compressed_original":9456607,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93700000000000006}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79600000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.069}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2190000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3819999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1140000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5202,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5202,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":847872,"data_stored":9488899,"data_compressed":93819,"data_compressed_allocated":757760,"data_compressed_original":9456607,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":712704,"data_stored":9347111,"data_compressed":34866,"data_compressed_allocated":667648,"data_compressed_original":9330688,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":864256,"data_stored":8447702,"data_compressed":151000,"data_compressed_allocated":733184,"data_compressed_original":8385835,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":679936,"data_stored":8378961,"data_compressed":20845,"data_compressed_allocated":602112,"data_compressed_original":8343818,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":815104,"data_stored":7584038,"data_compressed":152732,"data_compressed_allocated":671744,"data_compressed_original":7512064,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":585728,"data_stored":7589824,"data_compressed":14979,"data_compressed_allocated":536576,"data_compressed_original":7573504,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":741376,"data_stored":9461193,"data_compressed":49004,"data_compressed_allocated":700416,"data_compressed_original":9453568,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":589824,"data_stored":6125348,"data_compressed":54829,"data_compressed_allocated":483328,"data_compressed_original":6082560,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T14:58:39.541 INFO:tasks.thrashosds.thrasher:Setting 3.2 to [2, 3] 2024-09-21T14:58:39.541 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.2', '2', '3'] 2024-09-21T14:58:39.541 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.2 2 3 2024-09-21T14:58:39.542 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T14:58:39.599 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:58:39.596+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:58:39.670 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:58:39.670 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:58:39.670 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T14:58:39.670 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T14:58:39.670 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:58:39.681 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T14:58:39.700 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:58:39.696+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:58:39.800 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:58:39.796+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:58:39.804 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:58:39.804 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T14:58:39.804 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T14:58:39.804 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T14:58:39.804 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.805 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:1452 3.19 3:9d40fe72:::rbd_data.1136515ecb6b.0000000000000013:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T14:58:39.805 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:30.292755+0000", 2024-09-21T14:58:39.805 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.5095019309999991, 2024-09-21T14:58:39.805 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043852191999999998, 2024-09-21T14:58:39.805 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:39.805 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:39.805 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:39.805 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:39.805 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:39.805 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1452 2024-09-21T14:58:39.805 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.805 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:39.805 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.805 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:39.805 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292755+0000", 2024-09-21T14:58:39.806 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:39.806 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.806 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.806 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:39.806 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292755+0000", 2024-09-21T14:58:39.806 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:39.806 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.806 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.806 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:39.806 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292754+0000", 2024-09-21T14:58:39.806 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:39.806 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.806 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.806 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:39.806 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292760+0000", 2024-09-21T14:58:39.807 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5029999999999997e-06 2024-09-21T14:58:39.807 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.807 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.807 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:39.807 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292771+0000", 2024-09-21T14:58:39.807 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1551e-05 2024-09-21T14:58:39.807 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.807 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.807 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:39.807 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292779+0000", 2024-09-21T14:58:39.807 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1559999999999998e-06 2024-09-21T14:58:39.807 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.807 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.808 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:39.808 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292896+0000", 2024-09-21T14:58:39.808 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000117304 2024-09-21T14:58:39.808 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.808 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.808 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:39.808 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.293124+0000", 2024-09-21T14:58:39.808 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00022819999999999999 2024-09-21T14:58:39.808 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.808 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.808 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 1", 2024-09-21T14:58:39.808 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.299967+0000", 2024-09-21T14:58:39.808 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0068427009999999996 2024-09-21T14:58:39.808 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.808 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.808 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:39.809 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336463+0000", 2024-09-21T14:58:39.809 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.036495842000000001 2024-09-21T14:58:39.809 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.809 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.809 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:58:39.809 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336480+0000", 2024-09-21T14:58:39.809 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7419999999999999e-05 2024-09-21T14:58:39.809 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.809 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.809 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:39.809 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336550+0000", 2024-09-21T14:58:39.809 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9951999999999997e-05 2024-09-21T14:58:39.809 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.809 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.809 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:39.809 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336607+0000", 2024-09-21T14:58:39.810 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7423000000000002e-05 2024-09-21T14:58:39.810 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:39.810 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:39.810 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:39.810 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.810 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.810 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop_reply(client.4406.0:1452 3.19 e36/24 ondisk, result = 0)", 2024-09-21T14:58:39.810 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:30.322854+0000", 2024-09-21T14:58:39.810 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.4794032549999994, 2024-09-21T14:58:39.810 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013613079, 2024-09-21T14:58:39.810 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:39.810 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:39.810 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:39.810 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.810 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:39.811 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.322854+0000", 2024-09-21T14:58:39.811 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:39.811 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.811 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.811 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:39.811 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.322854+0000", 2024-09-21T14:58:39.811 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:39.811 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.811 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.811 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:39.811 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.322852+0000", 2024-09-21T14:58:39.811 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:39.811 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.811 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.811 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:39.811 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.322860+0000", 2024-09-21T14:58:39.812 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.605e-06 2024-09-21T14:58:39.812 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.812 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.812 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:39.812 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.322878+0000", 2024-09-21T14:58:39.812 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8204000000000001e-05 2024-09-21T14:58:39.812 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.812 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.812 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:39.812 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.322888+0000", 2024-09-21T14:58:39.812 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4099999999999997e-06 2024-09-21T14:58:39.812 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.812 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.812 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:39.812 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336440+0000", 2024-09-21T14:58:39.813 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013551848 2024-09-21T14:58:39.813 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.813 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.813 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:39.813 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336453+0000", 2024-09-21T14:58:39.813 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3465000000000001e-05 2024-09-21T14:58:39.813 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.813 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.813 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:39.813 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336467+0000", 2024-09-21T14:58:39.813 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3913999999999999e-05 2024-09-21T14:58:39.813 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:39.813 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:39.813 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:39.813 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.814 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.814 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:1567 3.8 3:139d3d55:::rbd_data.1136515ecb6b.0000000000000000:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T14:58:39.814 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:30.681087+0000", 2024-09-21T14:58:39.814 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.1211697160000007, 2024-09-21T14:58:39.814 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.039009224000000002, 2024-09-21T14:58:39.814 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:39.814 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:39.814 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:39.814 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:39.814 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:39.814 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1567 2024-09-21T14:58:39.814 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.814 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:39.815 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.815 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:39.815 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681087+0000", 2024-09-21T14:58:39.815 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:39.815 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.815 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.815 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:39.815 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681087+0000", 2024-09-21T14:58:39.815 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:39.815 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.815 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.815 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:39.815 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681086+0000", 2024-09-21T14:58:39.815 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:39.815 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.816 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.816 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:39.816 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681097+0000", 2024-09-21T14:58:39.816 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.134e-05 2024-09-21T14:58:39.816 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.816 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.816 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:39.816 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681122+0000", 2024-09-21T14:58:39.816 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5313e-05 2024-09-21T14:58:39.816 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.816 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.816 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:39.816 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681138+0000", 2024-09-21T14:58:39.816 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5916000000000001e-05 2024-09-21T14:58:39.816 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.817 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.817 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:39.817 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681270+0000", 2024-09-21T14:58:39.817 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000131474 2024-09-21T14:58:39.817 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.817 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.817 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:39.817 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681416+0000", 2024-09-21T14:58:39.817 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014637 2024-09-21T14:58:39.817 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.817 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.817 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 7", 2024-09-21T14:58:39.817 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.693163+0000", 2024-09-21T14:58:39.817 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011747241 2024-09-21T14:58:39.817 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.817 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.818 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:58:39.818 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.719750+0000", 2024-09-21T14:58:39.818 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.026586852000000001 2024-09-21T14:58:39.818 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.818 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.818 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:39.818 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.719859+0000", 2024-09-21T14:58:39.818 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010925999999999999 2024-09-21T14:58:39.818 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.818 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.818 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:39.818 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.719951+0000", 2024-09-21T14:58:39.818 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1294999999999999e-05 2024-09-21T14:58:39.818 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.819 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.819 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:39.819 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.720097+0000", 2024-09-21T14:58:39.819 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014583 2024-09-21T14:58:39.819 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:39.819 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:39.819 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:39.819 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.819 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.819 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9288 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 385024~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:39.819 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158023+0000", 2024-09-21T14:58:39.819 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.64423425300000003, 2024-09-21T14:58:39.819 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011313961000000001, 2024-09-21T14:58:39.819 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:39.819 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:39.820 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:39.820 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:39.820 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:39.820 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9288 2024-09-21T14:58:39.820 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.820 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:39.820 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.820 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:39.820 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158023+0000", 2024-09-21T14:58:39.820 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:39.820 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.820 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.820 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:39.820 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158023+0000", 2024-09-21T14:58:39.820 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:39.820 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.821 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.821 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:39.821 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158022+0000", 2024-09-21T14:58:39.821 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:39.821 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.821 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.821 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:39.821 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158034+0000", 2024-09-21T14:58:39.821 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1185999999999999e-05 2024-09-21T14:58:39.821 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.821 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.821 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:39.821 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158071+0000", 2024-09-21T14:58:39.821 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7110999999999998e-05 2024-09-21T14:58:39.821 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.821 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.822 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:39.822 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158081+0000", 2024-09-21T14:58:39.822 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0684e-05 2024-09-21T14:58:39.822 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.822 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.822 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:39.822 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.169101+0000", 2024-09-21T14:58:39.822 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011019986000000001 2024-09-21T14:58:39.822 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.822 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.822 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:39.822 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.169161+0000", 2024-09-21T14:58:39.822 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0024e-05 2024-09-21T14:58:39.822 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.822 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.823 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:39.823 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.169337+0000", 2024-09-21T14:58:39.823 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000175348 2024-09-21T14:58:39.823 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:39.823 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:39.823 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:39.823 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.823 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.823 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9289 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 417792~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:39.823 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158091+0000", 2024-09-21T14:58:39.823 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.64416614800000005, 2024-09-21T14:58:39.823 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011594068000000001, 2024-09-21T14:58:39.823 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:39.823 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:39.823 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:39.824 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:39.824 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:39.824 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9289 2024-09-21T14:58:39.824 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.824 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:39.824 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.824 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:39.824 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158091+0000", 2024-09-21T14:58:39.824 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:39.824 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.824 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.824 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:39.824 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158091+0000", 2024-09-21T14:58:39.824 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:39.824 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.824 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.825 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:39.825 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158090+0000", 2024-09-21T14:58:39.825 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:39.825 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.825 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.825 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:39.825 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158096+0000", 2024-09-21T14:58:39.825 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6439999999999997e-06 2024-09-21T14:58:39.825 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.825 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.825 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:39.825 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158107+0000", 2024-09-21T14:58:39.825 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0930999999999999e-05 2024-09-21T14:58:39.825 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.825 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.825 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:39.826 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158123+0000", 2024-09-21T14:58:39.826 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5937999999999999e-05 2024-09-21T14:58:39.826 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.826 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.826 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:39.826 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.169368+0000", 2024-09-21T14:58:39.826 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011244891999999999 2024-09-21T14:58:39.826 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.826 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.826 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:39.826 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.169468+0000", 2024-09-21T14:58:39.826 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000100388 2024-09-21T14:58:39.826 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.826 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.826 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:39.827 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.169685+0000", 2024-09-21T14:58:39.827 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000217111 2024-09-21T14:58:39.827 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:39.827 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:39.827 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:39.827 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.827 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.827 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9315 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 8192~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:39.827 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158130+0000", 2024-09-21T14:58:39.827 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.64412694699999995, 2024-09-21T14:58:39.827 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011893473, 2024-09-21T14:58:39.827 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:39.827 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:39.827 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:39.827 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:39.828 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:39.828 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9315 2024-09-21T14:58:39.828 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.828 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:39.828 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.828 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:39.828 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158130+0000", 2024-09-21T14:58:39.828 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:39.828 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.828 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.828 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:39.828 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158130+0000", 2024-09-21T14:58:39.828 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:39.828 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.828 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.828 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:39.829 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158130+0000", 2024-09-21T14:58:39.829 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:39.829 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.829 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.829 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:39.829 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158133+0000", 2024-09-21T14:58:39.829 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4999999999999999e-06 2024-09-21T14:58:39.829 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.829 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.829 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:39.829 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158139+0000", 2024-09-21T14:58:39.829 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9599999999999997e-06 2024-09-21T14:58:39.829 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.829 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.829 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:39.829 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158144+0000", 2024-09-21T14:58:39.829 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5419999999999999e-06 2024-09-21T14:58:39.830 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.830 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.830 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:39.830 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.169730+0000", 2024-09-21T14:58:39.830 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011586391999999999 2024-09-21T14:58:39.830 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.830 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.830 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:39.830 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.169826+0000", 2024-09-21T14:58:39.830 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5667000000000004e-05 2024-09-21T14:58:39.830 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.830 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.830 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:39.830 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.170024+0000", 2024-09-21T14:58:39.831 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019792 2024-09-21T14:58:39.831 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:39.831 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:39.831 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:39.831 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.831 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.831 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9316 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 114688~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:39.831 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158147+0000", 2024-09-21T14:58:39.831 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.64410973199999999, 2024-09-21T14:58:39.831 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012199521, 2024-09-21T14:58:39.831 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:39.831 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:39.831 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:39.831 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:39.832 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:39.832 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9316 2024-09-21T14:58:39.832 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.832 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:39.832 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.832 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:39.832 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158147+0000", 2024-09-21T14:58:39.832 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:39.832 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.832 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.832 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:39.832 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158147+0000", 2024-09-21T14:58:39.832 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:39.832 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.832 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.832 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:39.833 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158147+0000", 2024-09-21T14:58:39.833 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:39.833 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.833 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.833 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:39.833 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158150+0000", 2024-09-21T14:58:39.833 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6400000000000001e-06 2024-09-21T14:58:39.833 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.833 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.833 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:39.833 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158154+0000", 2024-09-21T14:58:39.833 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7380000000000002e-06 2024-09-21T14:58:39.833 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.833 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.833 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:39.833 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158160+0000", 2024-09-21T14:58:39.834 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8479999999999997e-06 2024-09-21T14:58:39.834 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.834 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.834 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:39.834 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.170072+0000", 2024-09-21T14:58:39.834 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011911713000000001 2024-09-21T14:58:39.834 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.834 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.834 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:39.834 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.170129+0000", 2024-09-21T14:58:39.834 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7522999999999998e-05 2024-09-21T14:58:39.834 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.834 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.834 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:39.834 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.170347+0000", 2024-09-21T14:58:39.835 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021744099999999999 2024-09-21T14:58:39.835 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:39.835 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:39.835 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:39.835 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.835 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.835 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9317 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 147456~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:39.835 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158169+0000", 2024-09-21T14:58:39.835 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.64408769799999999, 2024-09-21T14:58:39.835 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012441407, 2024-09-21T14:58:39.835 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:39.835 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:39.835 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:39.835 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:39.835 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:39.835 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9317 2024-09-21T14:58:39.836 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.836 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:39.836 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.836 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:39.836 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158169+0000", 2024-09-21T14:58:39.836 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:39.836 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.836 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.836 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:39.836 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158169+0000", 2024-09-21T14:58:39.836 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:39.836 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.836 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.836 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:39.836 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158168+0000", 2024-09-21T14:58:39.836 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:39.837 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.837 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.837 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:39.837 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158172+0000", 2024-09-21T14:58:39.837 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1980000000000003e-06 2024-09-21T14:58:39.837 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.837 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.837 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:39.837 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158180+0000", 2024-09-21T14:58:39.837 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7719999999999994e-06 2024-09-21T14:58:39.837 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.837 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.837 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:39.837 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158185+0000", 2024-09-21T14:58:39.837 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2839999999999997e-06 2024-09-21T14:58:39.837 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.838 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.838 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:39.838 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.170401+0000", 2024-09-21T14:58:39.838 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012215630999999999 2024-09-21T14:58:39.838 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.838 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.838 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:39.838 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.170448+0000", 2024-09-21T14:58:39.838 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6794e-05 2024-09-21T14:58:39.838 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.838 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.838 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:39.838 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.170611+0000", 2024-09-21T14:58:39.838 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016305599999999999 2024-09-21T14:58:39.838 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:39.838 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:39.839 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:39.839 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.839 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.839 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9318 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 253952~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:39.839 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158192+0000", 2024-09-21T14:58:39.839 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.64406469499999996, 2024-09-21T14:58:39.839 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012712467, 2024-09-21T14:58:39.839 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:39.839 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:39.839 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:39.839 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:39.839 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:39.839 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9318 2024-09-21T14:58:39.839 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.839 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:39.840 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.840 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:39.840 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158192+0000", 2024-09-21T14:58:39.840 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:39.840 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.840 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.840 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:39.840 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158192+0000", 2024-09-21T14:58:39.840 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:39.840 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.840 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.840 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:39.840 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158192+0000", 2024-09-21T14:58:39.840 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:39.840 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.840 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.840 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:39.841 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158195+0000", 2024-09-21T14:58:39.841 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.946e-06 2024-09-21T14:58:39.841 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.841 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.841 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:39.841 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158205+0000", 2024-09-21T14:58:39.841 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5799999999999998e-06 2024-09-21T14:58:39.841 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.841 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.841 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:39.841 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158212+0000", 2024-09-21T14:58:39.841 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.092e-06 2024-09-21T14:58:39.841 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.841 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.841 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:39.841 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.170646+0000", 2024-09-21T14:58:39.842 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012433623 2024-09-21T14:58:39.842 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.842 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.842 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:39.842 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.170696+0000", 2024-09-21T14:58:39.842 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9901000000000002e-05 2024-09-21T14:58:39.842 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.842 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.842 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:39.842 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.170905+0000", 2024-09-21T14:58:39.842 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00020919499999999999 2024-09-21T14:58:39.842 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:39.842 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:39.842 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:39.842 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.843 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.843 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9319 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 286720~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:39.843 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158219+0000", 2024-09-21T14:58:39.843 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.64403821299999997, 2024-09-21T14:58:39.843 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013038543, 2024-09-21T14:58:39.843 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:39.843 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:39.843 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:39.843 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:39.843 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:39.843 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9319 2024-09-21T14:58:39.843 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.843 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:39.843 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.843 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:39.844 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158219+0000", 2024-09-21T14:58:39.844 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:39.844 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.844 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.844 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:39.844 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158219+0000", 2024-09-21T14:58:39.844 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:39.844 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.844 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.844 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:39.844 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158218+0000", 2024-09-21T14:58:39.844 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:39.844 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.844 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.844 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:39.845 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158222+0000", 2024-09-21T14:58:39.845 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9859999999999996e-06 2024-09-21T14:58:39.845 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.845 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.845 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:39.845 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158232+0000", 2024-09-21T14:58:39.845 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5680000000000005e-06 2024-09-21T14:58:39.845 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.845 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.845 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:39.845 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158239+0000", 2024-09-21T14:58:39.845 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.469e-06 2024-09-21T14:58:39.845 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.845 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.845 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:39.846 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.170967+0000", 2024-09-21T14:58:39.846 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012728242000000001 2024-09-21T14:58:39.846 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.846 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.846 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:39.846 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.171037+0000", 2024-09-21T14:58:39.846 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9704999999999998e-05 2024-09-21T14:58:39.846 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.846 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.846 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:39.846 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.171257+0000", 2024-09-21T14:58:39.846 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00022030500000000001 2024-09-21T14:58:39.846 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:39.846 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:39.846 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:39.847 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.847 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.847 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9320 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 425984~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:39.847 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158246+0000", 2024-09-21T14:58:39.847 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.64401111, 2024-09-21T14:58:39.847 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013343911, 2024-09-21T14:58:39.847 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:39.847 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:39.847 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:39.847 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:39.847 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:39.847 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9320 2024-09-21T14:58:39.847 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.847 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:39.847 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.847 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:39.848 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158246+0000", 2024-09-21T14:58:39.848 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:39.848 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.848 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.848 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:39.848 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158246+0000", 2024-09-21T14:58:39.848 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:39.848 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.848 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.848 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:39.848 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158245+0000", 2024-09-21T14:58:39.848 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:39.848 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.848 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.849 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:39.849 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158249+0000", 2024-09-21T14:58:39.849 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8659999999999999e-06 2024-09-21T14:58:39.849 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.849 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.849 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:39.849 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158258+0000", 2024-09-21T14:58:39.849 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0720000000000006e-06 2024-09-21T14:58:39.849 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.849 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.849 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:39.849 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158266+0000", 2024-09-21T14:58:39.849 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0039999999999999e-06 2024-09-21T14:58:39.849 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.850 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.850 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:39.850 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.171302+0000", 2024-09-21T14:58:39.850 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013035972999999999 2024-09-21T14:58:39.850 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.850 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.850 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:39.850 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.171395+0000", 2024-09-21T14:58:39.850 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3071000000000004e-05 2024-09-21T14:58:39.850 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.850 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.850 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:39.850 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.171590+0000", 2024-09-21T14:58:39.850 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000194525 2024-09-21T14:58:39.850 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:39.850 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:39.851 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:39.851 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.851 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.851 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9347 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 16384~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:39.851 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158291+0000", 2024-09-21T14:58:39.851 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.64396621499999995, 2024-09-21T14:58:39.851 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013600958, 2024-09-21T14:58:39.851 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:39.851 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:39.851 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:39.851 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:39.851 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:39.851 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9347 2024-09-21T14:58:39.851 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.852 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:39.852 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.852 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:39.852 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158291+0000", 2024-09-21T14:58:39.852 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:39.852 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.852 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.852 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:39.852 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158291+0000", 2024-09-21T14:58:39.852 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:39.852 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.852 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.852 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:39.852 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158290+0000", 2024-09-21T14:58:39.852 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:39.852 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.853 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.853 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:39.853 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158295+0000", 2024-09-21T14:58:39.853 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2399999999999998e-06 2024-09-21T14:58:39.853 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.853 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.853 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:39.853 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158312+0000", 2024-09-21T14:58:39.853 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6929999999999999e-05 2024-09-21T14:58:39.853 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.853 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.853 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:39.853 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158353+0000", 2024-09-21T14:58:39.853 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0805e-05 2024-09-21T14:58:39.853 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.853 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.853 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:39.854 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.171634+0000", 2024-09-21T14:58:39.854 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.01328085 2024-09-21T14:58:39.854 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.854 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.854 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:39.854 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.171722+0000", 2024-09-21T14:58:39.854 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8115000000000006e-05 2024-09-21T14:58:39.854 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.854 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.854 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:39.854 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.171892+0000", 2024-09-21T14:58:39.854 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000169722 2024-09-21T14:58:39.854 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:39.855 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:39.855 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:39.855 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.855 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.855 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9348 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 122880~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:39.855 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158369+0000", 2024-09-21T14:58:39.855 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.64388804700000002, 2024-09-21T14:58:39.855 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.01388847, 2024-09-21T14:58:39.855 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:39.855 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:39.855 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:39.855 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:39.855 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:39.855 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9348 2024-09-21T14:58:39.855 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.856 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:39.856 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.856 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:39.856 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158369+0000", 2024-09-21T14:58:39.856 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:39.856 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.856 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.856 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:39.856 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158369+0000", 2024-09-21T14:58:39.856 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:39.856 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.856 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.856 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:39.856 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158368+0000", 2024-09-21T14:58:39.856 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:39.857 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.857 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.857 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:39.857 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158374+0000", 2024-09-21T14:58:39.857 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0499999999999997e-06 2024-09-21T14:58:39.857 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.857 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.857 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:39.857 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158386+0000", 2024-09-21T14:58:39.857 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1579999999999999e-05 2024-09-21T14:58:39.857 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.857 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.857 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:39.857 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158396+0000", 2024-09-21T14:58:39.857 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.8260000000000005e-06 2024-09-21T14:58:39.857 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.858 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.858 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:39.858 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.171967+0000", 2024-09-21T14:58:39.858 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013571318000000001 2024-09-21T14:58:39.858 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.858 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.858 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:39.858 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172034+0000", 2024-09-21T14:58:39.858 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.656e-05 2024-09-21T14:58:39.858 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.858 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.858 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:39.858 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172258+0000", 2024-09-21T14:58:39.858 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00022395999999999999 2024-09-21T14:58:39.859 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:39.859 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:39.859 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:39.859 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.859 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.859 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9349 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 155648~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:39.859 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158403+0000", 2024-09-21T14:58:39.859 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.64385430099999996, 2024-09-21T14:58:39.859 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014229851, 2024-09-21T14:58:39.859 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:39.859 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:39.859 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:39.859 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:39.859 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:39.859 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9349 2024-09-21T14:58:39.859 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.860 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:39.860 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.860 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:39.860 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158403+0000", 2024-09-21T14:58:39.860 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:39.860 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.860 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.860 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:39.860 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158403+0000", 2024-09-21T14:58:39.860 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:39.860 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.860 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.860 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:39.860 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158402+0000", 2024-09-21T14:58:39.860 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:39.860 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.861 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.861 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:39.861 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158406+0000", 2024-09-21T14:58:39.861 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1339999999999997e-06 2024-09-21T14:58:39.861 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.861 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.861 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:39.861 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158416+0000", 2024-09-21T14:58:39.861 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3149999999999998e-06 2024-09-21T14:58:39.861 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.861 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.861 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:39.861 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158439+0000", 2024-09-21T14:58:39.861 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3255999999999999e-05 2024-09-21T14:58:39.861 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.861 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.862 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:39.862 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172305+0000", 2024-09-21T14:58:39.862 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013866603999999999 2024-09-21T14:58:39.862 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.862 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.862 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:39.862 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172413+0000", 2024-09-21T14:58:39.862 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000107664 2024-09-21T14:58:39.862 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.862 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.862 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:39.862 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172633+0000", 2024-09-21T14:58:39.862 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021950800000000001 2024-09-21T14:58:39.862 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:39.862 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:39.863 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:39.863 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.863 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.863 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9350 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 294912~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:39.863 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158448+0000", 2024-09-21T14:58:39.863 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.64380943000000002, 2024-09-21T14:58:39.863 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014471525000000001, 2024-09-21T14:58:39.863 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:39.863 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:39.863 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:39.863 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:39.863 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:39.863 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9350 2024-09-21T14:58:39.863 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.863 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:39.864 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.864 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:39.864 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158448+0000", 2024-09-21T14:58:39.864 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:39.864 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.864 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.864 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:39.864 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158448+0000", 2024-09-21T14:58:39.864 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:39.864 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.864 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.864 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:39.864 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158447+0000", 2024-09-21T14:58:39.864 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:39.864 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.865 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.865 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:39.865 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158452+0000", 2024-09-21T14:58:39.865 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8400000000000002e-06 2024-09-21T14:58:39.865 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.865 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.865 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:39.865 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158475+0000", 2024-09-21T14:58:39.865 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3091e-05 2024-09-21T14:58:39.865 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.865 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.865 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:39.865 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158515+0000", 2024-09-21T14:58:39.865 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0265000000000002e-05 2024-09-21T14:58:39.865 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.866 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.866 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:39.866 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172682+0000", 2024-09-21T14:58:39.866 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014166761999999999 2024-09-21T14:58:39.866 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.866 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.866 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:39.866 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172752+0000", 2024-09-21T14:58:39.866 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9698999999999998e-05 2024-09-21T14:58:39.866 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.866 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.866 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:39.866 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172919+0000", 2024-09-21T14:58:39.866 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016759199999999999 2024-09-21T14:58:39.866 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:39.867 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:39.867 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:39.867 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.867 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.867 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9351 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 434176~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:39.867 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158526+0000", 2024-09-21T14:58:39.867 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.64373071199999998, 2024-09-21T14:58:39.867 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014662107000000001, 2024-09-21T14:58:39.867 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:39.867 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:39.867 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:39.867 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:39.867 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:39.867 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9351 2024-09-21T14:58:39.867 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.868 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:39.868 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.868 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:39.868 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158526+0000", 2024-09-21T14:58:39.868 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:39.868 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.868 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.868 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:39.868 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158526+0000", 2024-09-21T14:58:39.868 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:39.868 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.868 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.868 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:39.868 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158525+0000", 2024-09-21T14:58:39.868 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:39.869 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.869 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.869 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:39.869 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158531+0000", 2024-09-21T14:58:39.869 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5840000000000002e-06 2024-09-21T14:58:39.869 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.869 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.869 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:39.869 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158564+0000", 2024-09-21T14:58:39.869 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3195000000000003e-05 2024-09-21T14:58:39.869 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.869 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.869 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:39.869 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158577+0000", 2024-09-21T14:58:39.869 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2330000000000001e-05 2024-09-21T14:58:39.870 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.870 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.870 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:39.870 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172956+0000", 2024-09-21T14:58:39.870 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014379438 2024-09-21T14:58:39.870 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.870 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.870 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:39.870 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173020+0000", 2024-09-21T14:58:39.870 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4401999999999998e-05 2024-09-21T14:58:39.870 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.870 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.870 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:39.870 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173188+0000", 2024-09-21T14:58:39.870 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016804 2024-09-21T14:58:39.870 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:39.871 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:39.871 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:39.871 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.871 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.871 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9379 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 24576~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:39.871 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158586+0000", 2024-09-21T14:58:39.871 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.64367145199999998, 2024-09-21T14:58:39.871 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014865173000000001, 2024-09-21T14:58:39.871 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:39.871 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:39.871 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:39.871 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:39.871 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:39.871 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9379 2024-09-21T14:58:39.871 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.871 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:39.872 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.872 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:39.872 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158586+0000", 2024-09-21T14:58:39.872 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:39.872 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.872 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.872 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:39.872 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158586+0000", 2024-09-21T14:58:39.872 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:39.872 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.872 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.872 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:39.872 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158585+0000", 2024-09-21T14:58:39.872 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:39.872 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.872 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.873 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:39.873 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158590+0000", 2024-09-21T14:58:39.873 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5820000000000001e-06 2024-09-21T14:58:39.873 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.873 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.873 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:39.873 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158676+0000", 2024-09-21T14:58:39.873 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5075999999999997e-05 2024-09-21T14:58:39.873 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.873 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.873 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:39.873 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158695+0000", 2024-09-21T14:58:39.873 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9137999999999999e-05 2024-09-21T14:58:39.873 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.873 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.873 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:39.874 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173219+0000", 2024-09-21T14:58:39.874 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014524034999999999 2024-09-21T14:58:39.874 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.874 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.874 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:39.874 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173276+0000", 2024-09-21T14:58:39.874 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7442000000000003e-05 2024-09-21T14:58:39.874 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.874 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.874 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:39.874 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173451+0000", 2024-09-21T14:58:39.874 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017464800000000001 2024-09-21T14:58:39.874 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:39.874 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:39.875 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:39.875 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.875 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.875 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9380 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 163840~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:39.875 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158706+0000", 2024-09-21T14:58:39.875 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.64355094199999996, 2024-09-21T14:58:39.875 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015096381000000001, 2024-09-21T14:58:39.875 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:39.875 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:39.875 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:39.875 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:39.875 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:39.875 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9380 2024-09-21T14:58:39.875 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.875 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:39.875 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.876 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:39.876 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158706+0000", 2024-09-21T14:58:39.876 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:39.876 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.876 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.876 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:39.876 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158706+0000", 2024-09-21T14:58:39.876 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:39.876 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.876 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.876 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:39.876 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158705+0000", 2024-09-21T14:58:39.876 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:39.876 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.876 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.876 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:39.877 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158712+0000", 2024-09-21T14:58:39.877 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7240000000000003e-06 2024-09-21T14:58:39.877 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.877 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.877 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:39.877 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158755+0000", 2024-09-21T14:58:39.877 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2953e-05 2024-09-21T14:58:39.877 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.877 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.877 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:39.877 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158765+0000", 2024-09-21T14:58:39.877 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0149000000000001e-05 2024-09-21T14:58:39.877 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.877 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.877 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:39.877 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173492+0000", 2024-09-21T14:58:39.878 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014727105000000001 2024-09-21T14:58:39.878 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.878 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.878 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:39.878 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173564+0000", 2024-09-21T14:58:39.878 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1789e-05 2024-09-21T14:58:39.878 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.878 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.878 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:39.878 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173803+0000", 2024-09-21T14:58:39.878 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00023867100000000001 2024-09-21T14:58:39.878 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:39.878 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:39.878 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:39.878 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.879 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.879 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9381 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 303104~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:39.879 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158774+0000", 2024-09-21T14:58:39.879 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.64348286200000004, 2024-09-21T14:58:39.879 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015309095999999999, 2024-09-21T14:58:39.879 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:39.879 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:39.879 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:39.879 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:39.879 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:39.879 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9381 2024-09-21T14:58:39.879 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.879 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:39.879 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.879 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:39.879 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158774+0000", 2024-09-21T14:58:39.880 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:39.880 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.880 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.880 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:39.880 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158774+0000", 2024-09-21T14:58:39.880 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:39.880 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.880 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.880 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:39.880 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158773+0000", 2024-09-21T14:58:39.880 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:39.880 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.880 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.880 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:39.880 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158779+0000", 2024-09-21T14:58:39.880 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6999999999999996e-06 2024-09-21T14:58:39.881 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.881 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.881 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:39.881 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158790+0000", 2024-09-21T14:58:39.881 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0594000000000001e-05 2024-09-21T14:58:39.881 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.881 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.881 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:39.881 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158798+0000", 2024-09-21T14:58:39.881 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5860000000000005e-06 2024-09-21T14:58:39.881 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.881 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.881 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:39.881 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173829+0000", 2024-09-21T14:58:39.881 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015030349 2024-09-21T14:58:39.881 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.882 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.882 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:39.882 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173918+0000", 2024-09-21T14:58:39.882 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9375000000000004e-05 2024-09-21T14:58:39.882 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.882 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.882 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:39.882 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.174083+0000", 2024-09-21T14:58:39.882 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016536200000000001 2024-09-21T14:58:39.882 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:39.882 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:39.882 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:39.882 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.882 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.882 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9382 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 442368~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:39.882 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158805+0000", 2024-09-21T14:58:39.883 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.64345195700000002, 2024-09-21T14:58:39.883 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015592623, 2024-09-21T14:58:39.883 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:39.883 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:39.883 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:39.883 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:39.883 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:39.883 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9382 2024-09-21T14:58:39.883 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.883 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:39.883 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.883 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:39.883 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158805+0000", 2024-09-21T14:58:39.883 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:39.883 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.884 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.884 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:39.884 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158805+0000", 2024-09-21T14:58:39.884 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:39.884 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.884 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.884 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:39.884 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158805+0000", 2024-09-21T14:58:39.884 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:39.884 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.884 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.884 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:39.884 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158808+0000", 2024-09-21T14:58:39.884 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9179999999999996e-06 2024-09-21T14:58:39.884 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.885 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.885 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:39.885 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158817+0000", 2024-09-21T14:58:39.885 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9299999999999992e-06 2024-09-21T14:58:39.885 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.885 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.885 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:39.885 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158825+0000", 2024-09-21T14:58:39.885 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6739999999999997e-06 2024-09-21T14:58:39.885 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.885 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.885 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:39.885 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.174122+0000", 2024-09-21T14:58:39.885 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.01529694 2024-09-21T14:58:39.885 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.885 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.886 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:39.886 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.174185+0000", 2024-09-21T14:58:39.886 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3015000000000006e-05 2024-09-21T14:58:39.886 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:39.886 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:39.886 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:39.886 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.174398+0000", 2024-09-21T14:58:39.886 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021274799999999999 2024-09-21T14:58:39.886 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:39.886 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:39.886 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:39.886 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:39.886 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:39.886 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:58:39.887 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T14:58:39.901 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:58:39.896+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:58:40.001 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:58:40.001 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:58:40.002 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T14:58:40.002 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:58:40.002 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:58:39.996+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:58:40.012 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T14:58:40.105 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:58:40.100+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:58:40.113 INFO:teuthology.orchestra.run.smithi032.stderr:set 3.2 pg_upmap mapping to [2,3] 2024-09-21T14:58:40.128 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:58:40.128 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:58:40.128 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T14:58:40.128 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T14:58:40.129 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:58:40.131 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T14:58:40.138 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T14:58:40.204 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:58:40.200+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:58:40.264 INFO:teuthology.orchestra.run.smithi032.stdout:checking clone #3, image image_client.0-clone3 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent4 2024-09-21T14:58:40.298 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:58:40.298 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T14:58:40.298 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T14:58:40.298 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T14:58:40.299 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.299 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:6316 3.1f 3:ff210ee2:::rbd_header.11361bed1f78:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-21T14:58:40.299 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:34.515200+0000", 2024-09-21T14:58:40.299 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.7821103130000004, 2024-09-21T14:58:40.299 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.006353376, 2024-09-21T14:58:40.299 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:40.299 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:40.299 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:40.299 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:40.299 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:40.299 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 6316 2024-09-21T14:58:40.299 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.299 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:40.300 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.300 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:40.300 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:34.515200+0000", 2024-09-21T14:58:40.300 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:40.300 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.300 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.300 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:40.300 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:34.515200+0000", 2024-09-21T14:58:40.300 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:40.300 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.300 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.300 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:40.300 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:34.515197+0000", 2024-09-21T14:58:40.300 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999971 2024-09-21T14:58:40.300 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.300 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.301 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:40.301 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:34.515213+0000", 2024-09-21T14:58:40.301 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.615e-05 2024-09-21T14:58:40.301 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.301 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.301 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:40.301 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:34.515249+0000", 2024-09-21T14:58:40.301 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.5846000000000002e-05 2024-09-21T14:58:40.301 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.301 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.301 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:40.301 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:34.515273+0000", 2024-09-21T14:58:40.301 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3303e-05 2024-09-21T14:58:40.301 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.301 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.302 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:40.302 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:34.515447+0000", 2024-09-21T14:58:40.302 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017410599999999999 2024-09-21T14:58:40.302 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.302 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.302 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op must wait for map", 2024-09-21T14:58:40.302 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:34.515471+0000", 2024-09-21T14:58:40.302 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4723e-05 2024-09-21T14:58:40.302 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.302 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.302 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:40.302 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:34.519150+0000", 2024-09-21T14:58:40.302 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0036784560000000001 2024-09-21T14:58:40.302 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.302 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.303 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:40.303 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:34.519252+0000", 2024-09-21T14:58:40.303 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000102001 2024-09-21T14:58:40.303 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.303 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.303 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 1", 2024-09-21T14:58:40.303 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:34.519693+0000", 2024-09-21T14:58:40.303 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00044087599999999998 2024-09-21T14:58:40.303 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.303 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.303 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:58:40.303 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:34.520694+0000", 2024-09-21T14:58:40.303 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001000887 2024-09-21T14:58:40.303 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.303 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.304 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:40.304 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:34.521464+0000", 2024-09-21T14:58:40.304 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00077060900000000001 2024-09-21T14:58:40.304 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.304 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.304 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:40.304 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:34.521511+0000", 2024-09-21T14:58:40.304 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6533000000000003e-05 2024-09-21T14:58:40.304 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.304 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.304 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:40.304 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:34.521553+0000", 2024-09-21T14:58:40.304 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2522999999999999e-05 2024-09-21T14:58:40.304 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:40.304 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:40.305 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:40.305 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.305 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.305 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:6788 3.0 3:068c236c:::rbd_data.11363a21d2be.0000000000000001:head [write 0~524288 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:40.305 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:35.144365+0000", 2024-09-21T14:58:40.305 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.1529452060000001, 2024-09-21T14:58:40.305 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0063227639999999998, 2024-09-21T14:58:40.305 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:40.305 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:40.305 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:40.305 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:40.305 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:40.305 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 6788 2024-09-21T14:58:40.305 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.305 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:40.306 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.306 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:40.306 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.144365+0000", 2024-09-21T14:58:40.306 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:40.306 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.306 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.306 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:40.306 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.144365+0000", 2024-09-21T14:58:40.306 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:40.306 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.306 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.306 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:40.306 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.144364+0000", 2024-09-21T14:58:40.306 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:40.306 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.307 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.307 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:40.307 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.145263+0000", 2024-09-21T14:58:40.307 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00089863900000000001 2024-09-21T14:58:40.307 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.307 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.307 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:40.307 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.145282+0000", 2024-09-21T14:58:40.307 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9194e-05 2024-09-21T14:58:40.307 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.307 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.307 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:40.307 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.145291+0000", 2024-09-21T14:58:40.307 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9970000000000004e-06 2024-09-21T14:58:40.307 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.307 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.308 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:40.308 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.145399+0000", 2024-09-21T14:58:40.308 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000108595 2024-09-21T14:58:40.308 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.308 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.308 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:40.308 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.145553+0000", 2024-09-21T14:58:40.308 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015402100000000001 2024-09-21T14:58:40.308 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.308 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.308 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 7", 2024-09-21T14:58:40.308 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.145680+0000", 2024-09-21T14:58:40.308 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012625200000000001 2024-09-21T14:58:40.308 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.308 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.308 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:58:40.309 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.149189+0000", 2024-09-21T14:58:40.309 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0035092830000000002 2024-09-21T14:58:40.309 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.309 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.309 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:40.309 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.150600+0000", 2024-09-21T14:58:40.309 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0014114049999999999 2024-09-21T14:58:40.309 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.309 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.309 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:40.309 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.150647+0000", 2024-09-21T14:58:40.309 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.693e-05 2024-09-21T14:58:40.309 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.309 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.309 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:40.309 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.150688+0000", 2024-09-21T14:58:40.310 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0510999999999999e-05 2024-09-21T14:58:40.310 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:40.310 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:40.310 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:40.310 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.310 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.310 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:6811 3.1f 3:f97aaf24:::rbd_object_map.11363a21d2be.000000000000000e:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:40.310 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:35.151188+0000", 2024-09-21T14:58:40.310 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.1461219160000002, 2024-09-21T14:58:40.310 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0061405499999999998, 2024-09-21T14:58:40.310 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:40.310 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:40.310 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:40.310 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:40.310 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:40.311 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 6811 2024-09-21T14:58:40.311 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.311 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:40.311 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.311 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:40.311 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.151188+0000", 2024-09-21T14:58:40.311 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:40.311 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.311 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.311 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:40.311 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.151188+0000", 2024-09-21T14:58:40.311 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:40.311 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.311 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.311 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:40.311 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.151187+0000", 2024-09-21T14:58:40.312 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:40.312 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.312 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.312 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:40.312 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.151193+0000", 2024-09-21T14:58:40.312 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6960000000000002e-06 2024-09-21T14:58:40.312 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.312 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.312 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:40.312 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.151202+0000", 2024-09-21T14:58:40.312 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1209999999999996e-06 2024-09-21T14:58:40.312 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.312 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.312 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:40.312 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.151219+0000", 2024-09-21T14:58:40.312 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.681e-05 2024-09-21T14:58:40.313 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.313 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.313 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:40.313 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.152461+0000", 2024-09-21T14:58:40.313 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001242384 2024-09-21T14:58:40.313 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.313 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.313 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:40.313 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.152636+0000", 2024-09-21T14:58:40.313 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017426 2024-09-21T14:58:40.313 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.313 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.313 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 1", 2024-09-21T14:58:40.313 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.153303+0000", 2024-09-21T14:58:40.314 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00066699100000000002 2024-09-21T14:58:40.314 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.314 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.314 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:40.314 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.156549+0000", 2024-09-21T14:58:40.314 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0032462020000000001 2024-09-21T14:58:40.314 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.314 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.314 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:58:40.314 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.157225+0000", 2024-09-21T14:58:40.314 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00067670999999999996 2024-09-21T14:58:40.314 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.314 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.314 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:40.314 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.157276+0000", 2024-09-21T14:58:40.315 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0890999999999998e-05 2024-09-21T14:58:40.315 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.315 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.315 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:40.315 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.157329+0000", 2024-09-21T14:58:40.315 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2465e-05 2024-09-21T14:58:40.315 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:40.315 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:40.315 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:40.315 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.315 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.315 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:7275 3.1f 3:f97aaf24:::rbd_object_map.11363a21d2be.000000000000000e:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:40.315 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:35.182927+0000", 2024-09-21T14:58:40.315 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.1143832299999996, 2024-09-21T14:58:40.315 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0068328759999999999, 2024-09-21T14:58:40.315 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:40.316 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:40.316 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:40.316 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:40.316 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:40.316 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 7275 2024-09-21T14:58:40.316 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.316 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:40.316 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.316 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:40.316 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.182927+0000", 2024-09-21T14:58:40.316 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:40.316 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.316 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.316 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:40.316 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.182927+0000", 2024-09-21T14:58:40.316 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:40.317 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.317 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.317 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:40.317 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.182926+0000", 2024-09-21T14:58:40.317 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:40.317 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.317 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.317 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:40.317 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.183174+0000", 2024-09-21T14:58:40.317 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00024831899999999999 2024-09-21T14:58:40.317 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.317 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.317 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:40.317 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.183186+0000", 2024-09-21T14:58:40.317 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1793e-05 2024-09-21T14:58:40.318 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.318 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.318 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:40.318 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.183194+0000", 2024-09-21T14:58:40.318 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9060000000000002e-06 2024-09-21T14:58:40.318 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.318 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.318 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:40.318 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.184776+0000", 2024-09-21T14:58:40.318 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001582073 2024-09-21T14:58:40.318 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.318 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.318 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:40.318 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.184894+0000", 2024-09-21T14:58:40.318 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000118093 2024-09-21T14:58:40.319 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.319 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.319 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 1", 2024-09-21T14:58:40.319 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.185118+0000", 2024-09-21T14:58:40.319 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000223601 2024-09-21T14:58:40.319 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.319 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.319 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:58:40.319 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.189602+0000", 2024-09-21T14:58:40.319 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0044845969999999999 2024-09-21T14:58:40.319 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.319 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.319 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:40.319 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.189656+0000", 2024-09-21T14:58:40.319 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3835000000000002e-05 2024-09-21T14:58:40.319 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.320 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.320 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:40.320 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.189715+0000", 2024-09-21T14:58:40.320 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9168e-05 2024-09-21T14:58:40.320 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.320 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.320 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:40.320 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.189760+0000", 2024-09-21T14:58:40.320 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4477000000000001e-05 2024-09-21T14:58:40.320 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:40.320 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:40.320 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:40.320 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.320 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.320 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:7276 3.1f 3:f97aaf24:::rbd_object_map.11363a21d2be.000000000000000e:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:40.320 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:35.183628+0000", 2024-09-21T14:58:40.321 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.1136825269999999, 2024-09-21T14:58:40.321 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0062540319999999996, 2024-09-21T14:58:40.321 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:40.321 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:40.321 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:40.321 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:40.321 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:40.321 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 7276 2024-09-21T14:58:40.321 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.321 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:40.321 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.321 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:40.321 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.183628+0000", 2024-09-21T14:58:40.321 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:40.321 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.322 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.322 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:40.322 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.183628+0000", 2024-09-21T14:58:40.322 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:40.322 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.322 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.322 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:40.322 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.183627+0000", 2024-09-21T14:58:40.322 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:40.322 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.322 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.322 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:40.322 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.183635+0000", 2024-09-21T14:58:40.323 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7949999999999996e-06 2024-09-21T14:58:40.323 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.323 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.323 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:40.323 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.184433+0000", 2024-09-21T14:58:40.323 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00079790199999999996 2024-09-21T14:58:40.323 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.323 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.323 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:40.323 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.184445+0000", 2024-09-21T14:58:40.323 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2057e-05 2024-09-21T14:58:40.323 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.323 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.323 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:40.323 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.185762+0000", 2024-09-21T14:58:40.324 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0013168990000000001 2024-09-21T14:58:40.324 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.324 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.324 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:40.325 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.185815+0000", 2024-09-21T14:58:40.325 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2689999999999999e-05 2024-09-21T14:58:40.325 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.325 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.325 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 1", 2024-09-21T14:58:40.325 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.186114+0000", 2024-09-21T14:58:40.325 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00029914500000000001 2024-09-21T14:58:40.325 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.325 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.326 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:58:40.326 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.189607+0000", 2024-09-21T14:58:40.326 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0034932650000000002 2024-09-21T14:58:40.326 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.326 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.326 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:40.326 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.189807+0000", 2024-09-21T14:58:40.326 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019965799999999999 2024-09-21T14:58:40.326 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.326 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.326 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:40.326 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.189842+0000", 2024-09-21T14:58:40.326 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.5108000000000003e-05 2024-09-21T14:58:40.326 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.326 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.327 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:40.327 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.189882+0000", 2024-09-21T14:58:40.327 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9580999999999999e-05 2024-09-21T14:58:40.327 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:40.327 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:40.327 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:40.327 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.327 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.327 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:7608 3.1f 3:f97aaf24:::rbd_object_map.11363a21d2be.000000000000000e:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:40.327 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:35.202202+0000", 2024-09-21T14:58:40.327 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.0951080529999997, 2024-09-21T14:58:40.327 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010060813, 2024-09-21T14:58:40.327 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:40.327 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:40.327 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:40.327 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:40.328 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:40.328 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 7608 2024-09-21T14:58:40.328 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.328 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:40.328 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.328 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:40.328 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.202202+0000", 2024-09-21T14:58:40.328 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:40.328 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.328 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.328 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:40.328 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.202202+0000", 2024-09-21T14:58:40.328 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:40.328 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.328 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.328 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:40.329 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.202201+0000", 2024-09-21T14:58:40.329 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:40.329 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.329 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.329 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:40.329 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.202209+0000", 2024-09-21T14:58:40.329 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5039999999999996e-06 2024-09-21T14:58:40.329 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.329 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.329 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:40.329 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.202220+0000", 2024-09-21T14:58:40.329 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1192999999999999e-05 2024-09-21T14:58:40.329 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.329 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.329 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:40.329 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.202229+0000", 2024-09-21T14:58:40.330 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3749999999999992e-06 2024-09-21T14:58:40.330 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.330 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.330 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:40.330 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.202295+0000", 2024-09-21T14:58:40.330 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5333000000000006e-05 2024-09-21T14:58:40.330 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.330 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.330 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:40.330 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.202461+0000", 2024-09-21T14:58:40.330 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016598599999999999 2024-09-21T14:58:40.330 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.330 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.330 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 1", 2024-09-21T14:58:40.330 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.203061+0000", 2024-09-21T14:58:40.331 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00060068100000000004 2024-09-21T14:58:40.331 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.331 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.331 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:58:40.331 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.204034+0000", 2024-09-21T14:58:40.331 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00097254800000000005 2024-09-21T14:58:40.331 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.331 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.331 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:40.331 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.212176+0000", 2024-09-21T14:58:40.331 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0081418740000000003 2024-09-21T14:58:40.331 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.331 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.331 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:40.331 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.212224+0000", 2024-09-21T14:58:40.331 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8430999999999998e-05 2024-09-21T14:58:40.332 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.332 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.332 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:40.332 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.212263+0000", 2024-09-21T14:58:40.332 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.892e-05 2024-09-21T14:58:40.332 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:40.332 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:40.332 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:40.332 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.332 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.332 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:7693 3.1f 3:f97aaf24:::rbd_object_map.11363a21d2be.000000000000000e:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:40.332 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:35.205166+0000", 2024-09-21T14:58:40.332 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.0921441850000004, 2024-09-21T14:58:40.332 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0075195219999999998, 2024-09-21T14:58:40.332 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:40.332 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:40.333 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:40.333 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:40.333 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:40.333 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 7693 2024-09-21T14:58:40.333 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.333 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:40.333 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.333 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:40.333 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.205166+0000", 2024-09-21T14:58:40.333 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:40.333 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.333 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.333 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:40.333 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.205166+0000", 2024-09-21T14:58:40.333 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:40.333 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.334 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.334 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:40.334 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.205165+0000", 2024-09-21T14:58:40.334 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:40.334 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.334 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.334 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:40.334 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.205179+0000", 2024-09-21T14:58:40.334 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4025e-05 2024-09-21T14:58:40.334 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.334 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.334 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:40.334 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.205194+0000", 2024-09-21T14:58:40.334 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4171e-05 2024-09-21T14:58:40.334 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.334 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.335 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:40.335 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.205200+0000", 2024-09-21T14:58:40.335 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0009999999999998e-06 2024-09-21T14:58:40.335 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.335 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.335 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:40.335 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.205282+0000", 2024-09-21T14:58:40.335 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2076999999999994e-05 2024-09-21T14:58:40.335 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.335 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.335 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:40.335 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.205366+0000", 2024-09-21T14:58:40.335 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4233999999999993e-05 2024-09-21T14:58:40.336 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.336 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.336 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 1", 2024-09-21T14:58:40.336 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.206007+0000", 2024-09-21T14:58:40.336 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00064105399999999995 2024-09-21T14:58:40.336 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.336 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.336 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:58:40.336 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.207649+0000", 2024-09-21T14:58:40.336 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001641923 2024-09-21T14:58:40.336 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.336 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.336 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:40.336 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.212576+0000", 2024-09-21T14:58:40.337 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.004927285 2024-09-21T14:58:40.337 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.337 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.337 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:40.337 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.212633+0000", 2024-09-21T14:58:40.337 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6613999999999997e-05 2024-09-21T14:58:40.337 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.337 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.337 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:40.337 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.212686+0000", 2024-09-21T14:58:40.337 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2809000000000002e-05 2024-09-21T14:58:40.337 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:40.337 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:40.337 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:40.338 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.338 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.338 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:7694 3.1f 3:f97aaf24:::rbd_object_map.11363a21d2be.000000000000000e:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:40.338 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:35.205214+0000", 2024-09-21T14:58:40.338 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.0920959669999997, 2024-09-21T14:58:40.338 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0076555299999999998, 2024-09-21T14:58:40.338 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:40.338 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:40.338 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:40.338 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:40.338 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:40.338 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 7694 2024-09-21T14:58:40.338 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.339 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:40.339 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.339 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:40.339 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.205214+0000", 2024-09-21T14:58:40.339 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:40.339 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.339 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.339 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:40.339 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.205214+0000", 2024-09-21T14:58:40.339 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:40.339 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.339 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.339 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:40.339 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.205214+0000", 2024-09-21T14:58:40.340 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:40.340 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.340 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.340 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:40.340 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.205218+0000", 2024-09-21T14:58:40.340 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4209999999999997e-06 2024-09-21T14:58:40.340 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.340 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.340 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:40.340 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.205225+0000", 2024-09-21T14:58:40.340 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6710000000000002e-06 2024-09-21T14:58:40.340 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.340 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.340 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:40.340 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.205230+0000", 2024-09-21T14:58:40.341 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4319999999999998e-06 2024-09-21T14:58:40.341 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.341 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.341 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:40.341 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.206525+0000", 2024-09-21T14:58:40.341 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001294426 2024-09-21T14:58:40.341 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.341 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.341 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:40.341 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.206611+0000", 2024-09-21T14:58:40.341 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6244999999999998e-05 2024-09-21T14:58:40.341 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.341 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.341 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 1", 2024-09-21T14:58:40.341 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.207192+0000", 2024-09-21T14:58:40.342 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00058100800000000005 2024-09-21T14:58:40.342 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.342 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.342 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:58:40.342 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.208385+0000", 2024-09-21T14:58:40.342 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001193288 2024-09-21T14:58:40.342 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.342 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.342 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:40.342 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.212778+0000", 2024-09-21T14:58:40.342 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0043923499999999997 2024-09-21T14:58:40.342 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.342 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.342 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:40.343 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.212833+0000", 2024-09-21T14:58:40.343 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5687999999999999e-05 2024-09-21T14:58:40.343 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.343 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.343 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:40.343 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.212870+0000", 2024-09-21T14:58:40.343 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6522e-05 2024-09-21T14:58:40.343 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:40.343 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:40.343 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:40.343 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.343 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.343 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:8130 3.1f 3:f97aaf24:::rbd_object_map.11363a21d2be.000000000000000e:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:40.343 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:35.236830+0000", 2024-09-21T14:58:40.343 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.0604800699999997, 2024-09-21T14:58:40.344 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0058089049999999996, 2024-09-21T14:58:40.344 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:40.344 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:40.344 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:40.344 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:40.344 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:40.344 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 8130 2024-09-21T14:58:40.344 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.344 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:40.344 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.344 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:40.344 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.236830+0000", 2024-09-21T14:58:40.344 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:40.344 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.344 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.345 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:40.345 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.236830+0000", 2024-09-21T14:58:40.345 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:40.345 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.345 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.345 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:40.345 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.236829+0000", 2024-09-21T14:58:40.345 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:40.345 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.345 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.345 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:40.345 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.236837+0000", 2024-09-21T14:58:40.345 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4560000000000003e-06 2024-09-21T14:58:40.346 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.346 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.346 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:40.346 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.236853+0000", 2024-09-21T14:58:40.346 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5948e-05 2024-09-21T14:58:40.346 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.346 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.346 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:40.346 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.236861+0000", 2024-09-21T14:58:40.346 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0639999999999994e-06 2024-09-21T14:58:40.346 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.346 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.346 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:40.346 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.237695+0000", 2024-09-21T14:58:40.347 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00083468099999999999 2024-09-21T14:58:40.347 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.347 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.347 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:40.347 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.237782+0000", 2024-09-21T14:58:40.347 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7107000000000005e-05 2024-09-21T14:58:40.347 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.347 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.347 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 1", 2024-09-21T14:58:40.347 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.238498+0000", 2024-09-21T14:58:40.347 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00071560000000000005 2024-09-21T14:58:40.347 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.347 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.347 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:40.348 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.242540+0000", 2024-09-21T14:58:40.348 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0040419460000000003 2024-09-21T14:58:40.348 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.348 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.348 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:58:40.348 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.242551+0000", 2024-09-21T14:58:40.348 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1533e-05 2024-09-21T14:58:40.348 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.348 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.348 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:40.348 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.242600+0000", 2024-09-21T14:58:40.348 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8127999999999997e-05 2024-09-21T14:58:40.348 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.348 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.349 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:40.349 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.242639+0000", 2024-09-21T14:58:40.349 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9444999999999999e-05 2024-09-21T14:58:40.349 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:40.349 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:40.349 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:40.349 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.349 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.349 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9565 3.0 3:068c236c:::rbd_data.11363a21d2be.0000000000000001:10 [read 352256~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:40.349 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.196990+0000", 2024-09-21T14:58:40.350 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.1003199690000001, 2024-09-21T14:58:40.350 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0060517849999999996, 2024-09-21T14:58:40.350 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:40.350 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:40.350 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:40.350 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:40.350 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:40.350 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9565 2024-09-21T14:58:40.350 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.350 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:40.350 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.350 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:40.350 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.196990+0000", 2024-09-21T14:58:40.351 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:40.351 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.351 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.351 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:40.351 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.196990+0000", 2024-09-21T14:58:40.351 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:40.351 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.351 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.351 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:40.351 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.196990+0000", 2024-09-21T14:58:40.351 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:40.351 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.351 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.351 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:40.352 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.196993+0000", 2024-09-21T14:58:40.352 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3299999999999999e-06 2024-09-21T14:58:40.352 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.352 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.352 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:40.352 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.197002+0000", 2024-09-21T14:58:40.352 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3160000000000003e-06 2024-09-21T14:58:40.352 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.352 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.352 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:40.352 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.197010+0000", 2024-09-21T14:58:40.352 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2069999999999998e-06 2024-09-21T14:58:40.352 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.353 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.353 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:40.353 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.202598+0000", 2024-09-21T14:58:40.353 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0055885199999999996 2024-09-21T14:58:40.353 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.353 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.353 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:40.353 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.202682+0000", 2024-09-21T14:58:40.353 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4029999999999993e-05 2024-09-21T14:58:40.353 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.353 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.353 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:40.353 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.203042+0000", 2024-09-21T14:58:40.354 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00035991499999999999 2024-09-21T14:58:40.354 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:40.354 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:40.354 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:40.354 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.354 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.354 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9566 3.0 3:068c236c:::rbd_data.11363a21d2be.0000000000000001:10 [read 491520~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:40.354 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.197415+0000", 2024-09-21T14:58:40.354 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.099895538, 2024-09-21T14:58:40.354 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0058796259999999998, 2024-09-21T14:58:40.354 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:40.355 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:40.355 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:40.355 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:40.355 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:40.355 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9566 2024-09-21T14:58:40.355 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.355 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:40.355 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.355 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:40.355 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.197415+0000", 2024-09-21T14:58:40.355 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:40.355 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.355 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.355 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:40.355 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.197415+0000", 2024-09-21T14:58:40.356 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:40.356 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.356 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.356 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:40.356 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.197414+0000", 2024-09-21T14:58:40.356 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:40.356 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.356 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.356 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:40.356 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.197421+0000", 2024-09-21T14:58:40.356 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3180000000000001e-06 2024-09-21T14:58:40.356 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.356 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.356 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:40.357 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.197436+0000", 2024-09-21T14:58:40.357 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4565e-05 2024-09-21T14:58:40.357 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.357 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.357 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:40.357 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.197452+0000", 2024-09-21T14:58:40.357 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6087999999999999e-05 2024-09-21T14:58:40.357 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.357 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.357 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:40.357 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.203067+0000", 2024-09-21T14:58:40.357 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0056151159999999999 2024-09-21T14:58:40.357 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.357 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.357 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:40.358 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.203116+0000", 2024-09-21T14:58:40.358 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9395999999999999e-05 2024-09-21T14:58:40.358 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.358 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.358 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:40.358 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.203294+0000", 2024-09-21T14:58:40.358 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017811199999999999 2024-09-21T14:58:40.358 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:40.358 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:40.358 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:40.358 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.358 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.358 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9581 3.1e 3:7e97476a:::rbd_data.11363a21d2be.0000000000000005:10 [read 245760~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:40.358 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.197593+0000", 2024-09-21T14:58:40.359 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.0997168290000001, 2024-09-21T14:58:40.359 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0058210029999999999, 2024-09-21T14:58:40.359 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:40.359 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:40.359 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:40.359 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:40.359 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:40.359 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9581 2024-09-21T14:58:40.359 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.359 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:40.359 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.359 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:40.359 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.197593+0000", 2024-09-21T14:58:40.360 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:40.360 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.360 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.360 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:40.360 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.197593+0000", 2024-09-21T14:58:40.360 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:40.360 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.360 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.360 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:40.360 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.197593+0000", 2024-09-21T14:58:40.360 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:40.360 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.360 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.360 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:40.360 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.197597+0000", 2024-09-21T14:58:40.361 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7339999999999999e-06 2024-09-21T14:58:40.361 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.361 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.361 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:40.361 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.197608+0000", 2024-09-21T14:58:40.361 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0672999999999999e-05 2024-09-21T14:58:40.361 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.361 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.361 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:40.361 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.197616+0000", 2024-09-21T14:58:40.361 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4760000000000001e-06 2024-09-21T14:58:40.361 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.361 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.361 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:40.361 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.203053+0000", 2024-09-21T14:58:40.362 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.005437618 2024-09-21T14:58:40.362 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.362 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.362 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:40.362 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.203166+0000", 2024-09-21T14:58:40.362 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000112976 2024-09-21T14:58:40.362 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.362 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.362 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:40.362 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.203414+0000", 2024-09-21T14:58:40.362 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000248151 2024-09-21T14:58:40.362 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:40.362 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:40.362 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:40.363 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.363 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.363 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9582 3.1e 3:7e97476a:::rbd_data.11363a21d2be.0000000000000005:10 [read 278528~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:40.363 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.197622+0000", 2024-09-21T14:58:40.363 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.099688207, 2024-09-21T14:58:40.363 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0060363609999999996, 2024-09-21T14:58:40.363 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:40.363 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:40.363 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:40.363 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:40.363 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:40.363 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9582 2024-09-21T14:58:40.363 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.363 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:40.363 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.364 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:40.364 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.197622+0000", 2024-09-21T14:58:40.364 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:40.364 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.364 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.364 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:40.364 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.197622+0000", 2024-09-21T14:58:40.364 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:40.364 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.364 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.364 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:40.364 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.197621+0000", 2024-09-21T14:58:40.364 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:40.364 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.364 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.364 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:40.365 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.197625+0000", 2024-09-21T14:58:40.365 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9759999999999997e-06 2024-09-21T14:58:40.365 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.365 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.365 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:40.365 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.197634+0000", 2024-09-21T14:58:40.365 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3139999999999993e-06 2024-09-21T14:58:40.365 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.365 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.365 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:40.365 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.197642+0000", 2024-09-21T14:58:40.365 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6240000000000002e-06 2024-09-21T14:58:40.366 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.366 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.366 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:40.366 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.203453+0000", 2024-09-21T14:58:40.366 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0058112579999999997 2024-09-21T14:58:40.366 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.366 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.366 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:40.366 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.203502+0000", 2024-09-21T14:58:40.366 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8903000000000003e-05 2024-09-21T14:58:40.366 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.366 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.367 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:40.367 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.203658+0000", 2024-09-21T14:58:40.367 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0001561 2024-09-21T14:58:40.367 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:40.367 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:40.367 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:40.367 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.367 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.367 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9583 3.1e 3:7e97476a:::rbd_data.11363a21d2be.0000000000000005:10 [read 385024~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:40.367 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.197648+0000", 2024-09-21T14:58:40.367 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.0996617419999999, 2024-09-21T14:58:40.367 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0062802810000000004, 2024-09-21T14:58:40.368 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:40.368 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:40.368 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:40.368 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:40.368 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:40.368 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9583 2024-09-21T14:58:40.368 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.368 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:40.368 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.368 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:40.368 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.197648+0000", 2024-09-21T14:58:40.368 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:40.368 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.368 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.368 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:40.369 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.197648+0000", 2024-09-21T14:58:40.369 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:40.369 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.369 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.369 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:40.369 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.197648+0000", 2024-09-21T14:58:40.369 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:40.369 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.369 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.370 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:40.370 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.197652+0000", 2024-09-21T14:58:40.370 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7790000000000002e-06 2024-09-21T14:58:40.370 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.370 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.370 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:40.370 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.197660+0000", 2024-09-21T14:58:40.370 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5820000000000002e-06 2024-09-21T14:58:40.370 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.371 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.371 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:40.371 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.197668+0000", 2024-09-21T14:58:40.371 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3320000000000003e-06 2024-09-21T14:58:40.371 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.371 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.371 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:40.371 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.203691+0000", 2024-09-21T14:58:40.371 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0060237720000000002 2024-09-21T14:58:40.371 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.372 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.372 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:40.372 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.203739+0000", 2024-09-21T14:58:40.372 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7528999999999998e-05 2024-09-21T14:58:40.372 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.372 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.372 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:40.372 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.203929+0000", 2024-09-21T14:58:40.372 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018985000000000001 2024-09-21T14:58:40.372 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:40.373 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:40.373 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:40.373 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.373 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.373 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9589 3.1e 3:7e97476a:::rbd_data.11363a21d2be.0000000000000005:10 [read 425984~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:40.373 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.197674+0000", 2024-09-21T14:58:40.373 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.0996361130000001, 2024-09-21T14:58:40.373 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0066266889999999998, 2024-09-21T14:58:40.373 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:40.374 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:40.374 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:40.374 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:40.374 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:40.374 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9589 2024-09-21T14:58:40.374 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.374 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:40.374 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.374 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:40.375 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.197674+0000", 2024-09-21T14:58:40.375 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:40.375 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.375 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.375 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:40.375 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.197674+0000", 2024-09-21T14:58:40.375 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:40.375 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.375 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.376 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:40.376 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.197674+0000", 2024-09-21T14:58:40.376 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:40.376 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.376 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.376 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:40.376 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.197677+0000", 2024-09-21T14:58:40.376 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.693e-06 2024-09-21T14:58:40.376 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.377 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.377 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:40.377 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.197686+0000", 2024-09-21T14:58:40.377 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4910000000000005e-06 2024-09-21T14:58:40.377 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.377 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.377 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:40.377 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.197693+0000", 2024-09-21T14:58:40.377 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2350000000000002e-06 2024-09-21T14:58:40.377 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.378 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.378 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:40.378 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.203962+0000", 2024-09-21T14:58:40.378 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0062692240000000003 2024-09-21T14:58:40.378 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.378 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.378 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:40.378 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.204059+0000", 2024-09-21T14:58:40.378 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6735000000000001e-05 2024-09-21T14:58:40.379 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.379 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.379 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:40.379 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.204301+0000", 2024-09-21T14:58:40.379 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000241902 2024-09-21T14:58:40.379 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:40.379 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:40.379 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:40.379 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.379 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.380 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9618 3.1e 3:7e97476a:::rbd_data.11363a21d2be.0000000000000005:10 [read 434176~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:40.380 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.198728+0000", 2024-09-21T14:58:40.380 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.098582239, 2024-09-21T14:58:40.380 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0059398879999999999, 2024-09-21T14:58:40.380 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:40.380 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:40.380 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:40.380 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:40.380 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:40.380 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9618 2024-09-21T14:58:40.381 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.381 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:40.381 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.381 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:40.381 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.198728+0000", 2024-09-21T14:58:40.381 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:40.381 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.381 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.381 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:40.382 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.198728+0000", 2024-09-21T14:58:40.382 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:40.382 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.382 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.382 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:40.382 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.198727+0000", 2024-09-21T14:58:40.382 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:40.382 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.382 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.383 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:40.383 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.198754+0000", 2024-09-21T14:58:40.383 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7653000000000001e-05 2024-09-21T14:58:40.383 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.383 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.383 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:40.383 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.198772+0000", 2024-09-21T14:58:40.383 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7575e-05 2024-09-21T14:58:40.383 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.383 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.383 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:40.384 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.198784+0000", 2024-09-21T14:58:40.384 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1771e-05 2024-09-21T14:58:40.384 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.384 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.384 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:40.384 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.204344+0000", 2024-09-21T14:58:40.384 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0055603500000000004 2024-09-21T14:58:40.384 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.384 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.384 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:40.384 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.204468+0000", 2024-09-21T14:58:40.384 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012389399999999999 2024-09-21T14:58:40.384 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.384 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.384 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:40.385 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.204668+0000", 2024-09-21T14:58:40.385 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00020008999999999999 2024-09-21T14:58:40.385 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:40.385 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:40.385 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:40.385 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.385 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.385 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:11086 3.1f 3:fddf67f4:::rbd_data.11363a21d2be.000000000000001e:10 [read 409600~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:40.385 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.525165+0000", 2024-09-21T14:58:40.385 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.77214480799999996, 2024-09-21T14:58:40.385 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0058864349999999998, 2024-09-21T14:58:40.385 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:40.386 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:40.386 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:40.386 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:40.386 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:40.386 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 11086 2024-09-21T14:58:40.386 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.386 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:40.386 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.386 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:40.386 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.525165+0000", 2024-09-21T14:58:40.386 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:40.386 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.386 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.387 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:40.387 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.525165+0000", 2024-09-21T14:58:40.387 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:40.387 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.387 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.387 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:40.387 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.525165+0000", 2024-09-21T14:58:40.387 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:40.387 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.387 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.387 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:40.387 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.525167+0000", 2024-09-21T14:58:40.387 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.905e-06 2024-09-21T14:58:40.388 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.388 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.388 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:40.388 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.525171+0000", 2024-09-21T14:58:40.388 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1509999999999997e-06 2024-09-21T14:58:40.388 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.388 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.388 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:40.388 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.525178+0000", 2024-09-21T14:58:40.388 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0849999999999999e-06 2024-09-21T14:58:40.388 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.388 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.388 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:40.388 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.530864+0000", 2024-09-21T14:58:40.389 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0056856290000000002 2024-09-21T14:58:40.389 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.389 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.389 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:40.389 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.530914+0000", 2024-09-21T14:58:40.389 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9840999999999999e-05 2024-09-21T14:58:40.389 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.389 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.389 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:40.389 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.531052+0000", 2024-09-21T14:58:40.389 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013817699999999999 2024-09-21T14:58:40.389 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:40.389 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:40.389 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:40.390 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.390 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.390 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:11087 3.1f 3:fddf67f4:::rbd_data.11363a21d2be.000000000000001e:10 [read 516096~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:40.390 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.525184+0000", 2024-09-21T14:58:40.390 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.77212584699999998, 2024-09-21T14:58:40.390 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0060770909999999997, 2024-09-21T14:58:40.390 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:40.390 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:40.390 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:40.390 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:40.390 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:40.390 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 11087 2024-09-21T14:58:40.391 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.391 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:40.391 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.391 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:40.391 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.525184+0000", 2024-09-21T14:58:40.391 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:40.391 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.391 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.391 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:40.391 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.525184+0000", 2024-09-21T14:58:40.391 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:40.391 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.391 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.391 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:40.391 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.525184+0000", 2024-09-21T14:58:40.392 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:40.392 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.392 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.392 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:40.392 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.525192+0000", 2024-09-21T14:58:40.392 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1650000000000006e-06 2024-09-21T14:58:40.392 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.392 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.392 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:40.392 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.525202+0000", 2024-09-21T14:58:40.392 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4779999999999997e-06 2024-09-21T14:58:40.392 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.392 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.392 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:40.392 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.525210+0000", 2024-09-21T14:58:40.393 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3999999999999992e-06 2024-09-21T14:58:40.393 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.393 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.393 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:40.393 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.531080+0000", 2024-09-21T14:58:40.393 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0058705099999999998 2024-09-21T14:58:40.393 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.393 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.393 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:40.393 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.531130+0000", 2024-09-21T14:58:40.393 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9039000000000003e-05 2024-09-21T14:58:40.393 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.393 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.393 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:40.393 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.531261+0000", 2024-09-21T14:58:40.393 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000131941 2024-09-21T14:58:40.393 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:40.394 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:40.394 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:40.394 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.394 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.394 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:11147 3.1f 3:fddf67f4:::rbd_data.11363a21d2be.000000000000001e:10 [read 286720~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:40.394 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.528327+0000", 2024-09-21T14:58:40.394 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.76898354099999999, 2024-09-21T14:58:40.394 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0058834810000000003, 2024-09-21T14:58:40.394 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:40.394 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:40.394 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:40.394 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:40.395 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:40.395 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 11147 2024-09-21T14:58:40.395 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.395 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:40.395 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.395 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:40.395 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.528327+0000", 2024-09-21T14:58:40.395 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:40.395 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.395 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.395 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:40.395 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.528327+0000", 2024-09-21T14:58:40.395 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:40.395 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.395 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.396 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:40.396 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.528326+0000", 2024-09-21T14:58:40.396 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:40.396 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.396 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.396 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:40.396 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.528330+0000", 2024-09-21T14:58:40.396 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7910000000000002e-06 2024-09-21T14:58:40.396 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.396 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.396 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:40.396 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.528339+0000", 2024-09-21T14:58:40.396 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8040000000000007e-06 2024-09-21T14:58:40.396 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.396 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.396 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:40.397 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.528346+0000", 2024-09-21T14:58:40.397 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1180000000000003e-06 2024-09-21T14:58:40.397 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.397 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.397 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:40.397 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.534066+0000", 2024-09-21T14:58:40.397 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0057197250000000002 2024-09-21T14:58:40.397 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.397 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.397 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:40.397 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.534129+0000", 2024-09-21T14:58:40.397 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3503999999999994e-05 2024-09-21T14:58:40.397 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.397 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.397 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:40.398 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.534210+0000", 2024-09-21T14:58:40.398 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1057000000000007e-05 2024-09-21T14:58:40.398 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:40.398 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:40.398 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:40.398 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.398 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.398 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:11148 3.1f 3:fddf67f4:::rbd_data.11363a21d2be.000000000000001e:10 [read 425984~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:40.398 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.528365+0000", 2024-09-21T14:58:40.398 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.76894509300000002, 2024-09-21T14:58:40.398 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0061269250000000001, 2024-09-21T14:58:40.398 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:40.398 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:40.398 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:40.398 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:40.398 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:40.399 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 11148 2024-09-21T14:58:40.399 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.399 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:40.399 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.399 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:40.399 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.528365+0000", 2024-09-21T14:58:40.399 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:40.399 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.399 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.399 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:40.399 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.528365+0000", 2024-09-21T14:58:40.399 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:40.399 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.399 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.399 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:40.400 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.528364+0000", 2024-09-21T14:58:40.400 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:40.400 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.400 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.400 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:40.400 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.528369+0000", 2024-09-21T14:58:40.400 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7849999999999999e-06 2024-09-21T14:58:40.400 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.400 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.400 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:40.400 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.528379+0000", 2024-09-21T14:58:40.400 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0086e-05 2024-09-21T14:58:40.400 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.400 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.400 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:40.400 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.528387+0000", 2024-09-21T14:58:40.401 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3710000000000002e-06 2024-09-21T14:58:40.401 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.401 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.401 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:40.401 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.534226+0000", 2024-09-21T14:58:40.401 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.005839103 2024-09-21T14:58:40.401 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.401 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.401 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:40.401 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.534274+0000", 2024-09-21T14:58:40.401 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8013000000000003e-05 2024-09-21T14:58:40.401 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.401 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.401 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:40.401 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.534492+0000", 2024-09-21T14:58:40.401 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000218307 2024-09-21T14:58:40.402 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:40.402 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:40.402 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:40.402 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:40.402 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:40.402 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:58:40.403 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T14:58:40.403 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:58:40.400+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:58:40.504 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:58:40.500+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:58:40.564 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:58:40.564 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:58:40.564 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T14:58:40.564 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:58:40.574 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T14:58:40.604 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:58:40.600+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:58:40.678 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:58:40.679 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:58:40.679 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T14:58:40.679 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T14:58:40.679 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:58:40.688 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T14:58:40.705 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:58:40.704+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:58:40.797 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:58:40.798 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T14:58:40.798 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T14:58:40.798 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T14:58:40.798 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.798 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:10651 3.12 3:4e9d23af:::rbd_data.1136f5ed5dd6.0000000000000020:head [write 0~81920 [fadvise_sequential+fadvise_nocache],write 90112~8192 [fadvise_sequential+fadvise_nocache],write 122880~122880 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:40.798 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.292476+0000", 2024-09-21T14:58:40.798 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.5026113780000001, 2024-09-21T14:58:40.798 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012765669, 2024-09-21T14:58:40.798 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:40.798 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:40.798 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:40.798 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:40.798 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:40.799 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 10651 2024-09-21T14:58:40.799 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.799 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:40.799 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.799 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:40.799 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292476+0000", 2024-09-21T14:58:40.799 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:40.799 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.799 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.799 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:40.799 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292476+0000", 2024-09-21T14:58:40.799 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:40.799 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.799 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.800 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:40.800 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292475+0000", 2024-09-21T14:58:40.800 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:40.800 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.800 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.800 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:40.800 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292696+0000", 2024-09-21T14:58:40.800 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000220201 2024-09-21T14:58:40.800 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.800 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.800 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:40.800 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292711+0000", 2024-09-21T14:58:40.800 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5803000000000001e-05 2024-09-21T14:58:40.800 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.800 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.801 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:40.801 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292721+0000", 2024-09-21T14:58:40.801 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6160000000000008e-06 2024-09-21T14:58:40.801 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.801 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.801 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:40.801 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292757+0000", 2024-09-21T14:58:40.801 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6211999999999998e-05 2024-09-21T14:58:40.801 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.801 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.801 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:40.801 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292878+0000", 2024-09-21T14:58:40.801 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000120306 2024-09-21T14:58:40.801 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.802 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.802 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 3", 2024-09-21T14:58:40.802 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.293019+0000", 2024-09-21T14:58:40.802 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014186400000000001 2024-09-21T14:58:40.802 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.802 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.802 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:58:40.802 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.294898+0000", 2024-09-21T14:58:40.802 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001878434 2024-09-21T14:58:40.802 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.802 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.802 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:40.802 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.305162+0000", 2024-09-21T14:58:40.802 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.01026373 2024-09-21T14:58:40.802 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.803 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.803 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:40.803 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.305209+0000", 2024-09-21T14:58:40.803 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7615999999999999e-05 2024-09-21T14:58:40.803 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.803 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.803 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:40.803 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.305242+0000", 2024-09-21T14:58:40.803 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2979999999999999e-05 2024-09-21T14:58:40.803 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:40.803 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:40.803 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:40.803 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.803 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.803 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:11285 3.12 3:4edf73d2:::rbd_data.1136f5ed5dd6.0000000000000039:head [write 90112~32768 [fadvise_sequential+fadvise_nocache],write 139264~24576 [fadvise_sequential+fadvise_nocache],write 180224~24576 [fadvise_sequential+fadvise_nocache],write 221184~40960 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:40.804 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.539470+0000", 2024-09-21T14:58:40.804 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.255617626, 2024-09-21T14:58:40.804 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0083360099999999996, 2024-09-21T14:58:40.804 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:40.804 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:40.804 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:40.804 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:40.804 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:40.804 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 11285 2024-09-21T14:58:40.804 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.804 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:40.804 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.804 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:40.804 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.539470+0000", 2024-09-21T14:58:40.804 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:40.804 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.805 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.805 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:40.805 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.539470+0000", 2024-09-21T14:58:40.805 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:40.805 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.805 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.805 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:40.805 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.539469+0000", 2024-09-21T14:58:40.805 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:40.805 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.805 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.805 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:40.805 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.539652+0000", 2024-09-21T14:58:40.805 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000183084 2024-09-21T14:58:40.805 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.805 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.806 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:40.806 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.539666+0000", 2024-09-21T14:58:40.806 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3947e-05 2024-09-21T14:58:40.806 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.806 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.806 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:40.806 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.539673+0000", 2024-09-21T14:58:40.806 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7309999999999996e-06 2024-09-21T14:58:40.806 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.806 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.806 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:40.806 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.539714+0000", 2024-09-21T14:58:40.806 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1539000000000003e-05 2024-09-21T14:58:40.806 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.807 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.807 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:40.807 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.539832+0000", 2024-09-21T14:58:40.807 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011753500000000001 2024-09-21T14:58:40.807 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.807 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.807 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 3", 2024-09-21T14:58:40.807 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.540005+0000", 2024-09-21T14:58:40.807 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000173031 2024-09-21T14:58:40.807 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.807 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.807 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:58:40.807 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.541606+0000", 2024-09-21T14:58:40.807 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001600899 2024-09-21T14:58:40.807 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.807 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.808 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:40.808 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.547709+0000", 2024-09-21T14:58:40.808 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0061027110000000002 2024-09-21T14:58:40.808 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.808 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.808 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:40.808 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.547767+0000", 2024-09-21T14:58:40.808 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8248999999999997e-05 2024-09-21T14:58:40.808 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.808 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.808 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:40.808 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.547806+0000", 2024-09-21T14:58:40.808 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9341000000000002e-05 2024-09-21T14:58:40.808 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:40.808 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:40.809 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:40.809 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.809 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.809 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12189 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 40960~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:40.809 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.618667+0000", 2024-09-21T14:58:40.809 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.1764211760000001, 2024-09-21T14:58:40.809 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0083014549999999993, 2024-09-21T14:58:40.809 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:40.809 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:40.809 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:40.809 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:40.809 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:40.809 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12189 2024-09-21T14:58:40.809 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.809 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:40.809 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.810 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:40.810 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618667+0000", 2024-09-21T14:58:40.810 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:40.810 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.810 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.810 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:40.810 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618667+0000", 2024-09-21T14:58:40.810 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:40.810 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.810 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.810 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:40.810 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618666+0000", 2024-09-21T14:58:40.810 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:40.810 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.811 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.811 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:40.811 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618669+0000", 2024-09-21T14:58:40.811 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.554e-06 2024-09-21T14:58:40.811 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.811 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.811 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:40.811 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618675+0000", 2024-09-21T14:58:40.811 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2570000000000004e-06 2024-09-21T14:58:40.811 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.811 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.811 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:40.811 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618680+0000", 2024-09-21T14:58:40.811 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6539999999999998e-06 2024-09-21T14:58:40.811 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.812 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.812 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:40.812 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.626683+0000", 2024-09-21T14:58:40.812 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0080035880000000007 2024-09-21T14:58:40.812 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.812 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.812 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:40.812 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.626764+0000", 2024-09-21T14:58:40.812 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0672000000000003e-05 2024-09-21T14:58:40.812 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.812 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.812 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:40.812 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.626968+0000", 2024-09-21T14:58:40.812 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00020407499999999999 2024-09-21T14:58:40.812 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:40.813 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:40.813 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:40.813 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.813 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.813 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12193 3.4 3:25437d51:::rbd_data.11363a21d2be.0000000000000025:10 [read 425984~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:40.813 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.618755+0000", 2024-09-21T14:58:40.813 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.1763330949999999, 2024-09-21T14:58:40.813 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086461379999999994, 2024-09-21T14:58:40.813 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:40.813 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:40.813 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:40.813 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:40.813 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:40.813 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12193 2024-09-21T14:58:40.813 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.813 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:40.814 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.814 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:40.814 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618755+0000", 2024-09-21T14:58:40.814 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:40.814 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.814 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.814 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:40.814 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618755+0000", 2024-09-21T14:58:40.814 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:40.814 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.814 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.814 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:40.814 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618754+0000", 2024-09-21T14:58:40.814 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:40.814 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.814 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.815 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:40.815 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618759+0000", 2024-09-21T14:58:40.815 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4020000000000004e-06 2024-09-21T14:58:40.815 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.815 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.815 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:40.815 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618767+0000", 2024-09-21T14:58:40.815 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7490000000000004e-06 2024-09-21T14:58:40.815 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.815 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.815 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:40.815 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618773+0000", 2024-09-21T14:58:40.815 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0939999999999996e-06 2024-09-21T14:58:40.815 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.815 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.815 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:40.816 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.627115+0000", 2024-09-21T14:58:40.816 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0083416840000000003 2024-09-21T14:58:40.816 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.816 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.816 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:40.816 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.627214+0000", 2024-09-21T14:58:40.816 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9042000000000005e-05 2024-09-21T14:58:40.816 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.816 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.816 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:40.816 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.627401+0000", 2024-09-21T14:58:40.816 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018677200000000001 2024-09-21T14:58:40.816 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:40.816 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:40.816 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:40.816 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.817 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.817 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12212 3.4 3:21b43b45:::rbd_data.11363a21d2be.000000000000002b:10 [read 237568~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:40.817 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.619922+0000", 2024-09-21T14:58:40.817 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.175166111, 2024-09-21T14:58:40.817 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.008324372, 2024-09-21T14:58:40.817 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:40.817 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:40.817 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:40.817 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:40.817 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:40.817 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12212 2024-09-21T14:58:40.817 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.817 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:40.817 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.818 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:40.818 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619922+0000", 2024-09-21T14:58:40.818 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:40.818 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.818 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.818 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:40.818 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619922+0000", 2024-09-21T14:58:40.818 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:40.818 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.818 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.818 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:40.818 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619921+0000", 2024-09-21T14:58:40.818 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:40.818 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.818 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.819 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:40.819 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619924+0000", 2024-09-21T14:58:40.819 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.999e-06 2024-09-21T14:58:40.819 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.819 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.819 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:40.819 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619934+0000", 2024-09-21T14:58:40.819 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.8120000000000002e-06 2024-09-21T14:58:40.819 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.819 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.819 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:40.819 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619939+0000", 2024-09-21T14:58:40.819 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7729999999999999e-06 2024-09-21T14:58:40.819 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.820 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.820 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:40.820 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628005+0000", 2024-09-21T14:58:40.820 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0080663330000000002 2024-09-21T14:58:40.820 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.820 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.820 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:40.820 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628073+0000", 2024-09-21T14:58:40.820 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7479999999999998e-05 2024-09-21T14:58:40.820 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.820 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.820 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:40.820 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628246+0000", 2024-09-21T14:58:40.820 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017337400000000001 2024-09-21T14:58:40.820 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:40.820 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:40.821 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:40.821 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.821 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.821 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12213 3.4 3:21b43b45:::rbd_data.11363a21d2be.000000000000002b:10 [read 270336~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:40.821 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.619945+0000", 2024-09-21T14:58:40.821 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.1751428289999999, 2024-09-21T14:58:40.821 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086195049999999995, 2024-09-21T14:58:40.821 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:40.821 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:40.821 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:40.821 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:40.821 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:40.821 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12213 2024-09-21T14:58:40.821 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.821 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:40.822 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.822 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:40.822 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619945+0000", 2024-09-21T14:58:40.822 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:40.822 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.822 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.822 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:40.822 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619945+0000", 2024-09-21T14:58:40.822 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:40.822 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.822 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.822 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:40.822 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619945+0000", 2024-09-21T14:58:40.822 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:40.822 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.823 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.823 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:40.823 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619947+0000", 2024-09-21T14:58:40.823 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5780000000000001e-06 2024-09-21T14:58:40.823 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.823 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.823 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:40.823 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619957+0000", 2024-09-21T14:58:40.823 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3500000000000003e-06 2024-09-21T14:58:40.823 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.823 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.823 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:40.823 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619964+0000", 2024-09-21T14:58:40.823 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.199e-06 2024-09-21T14:58:40.824 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.824 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.824 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:40.824 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628285+0000", 2024-09-21T14:58:40.824 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0083210939999999994 2024-09-21T14:58:40.824 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.824 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.824 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:40.824 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628361+0000", 2024-09-21T14:58:40.824 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5873999999999999e-05 2024-09-21T14:58:40.824 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.824 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.824 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:40.824 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628565+0000", 2024-09-21T14:58:40.824 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000203799 2024-09-21T14:58:40.825 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:40.825 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:40.825 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:40.825 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.825 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.825 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12214 3.4 3:21b43b45:::rbd_data.11363a21d2be.000000000000002b:10 [read 376832~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:40.825 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.619969+0000", 2024-09-21T14:58:40.825 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.175118828, 2024-09-21T14:58:40.825 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0090292089999999998, 2024-09-21T14:58:40.825 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:40.825 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:40.825 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:40.825 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:40.825 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:40.825 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12214 2024-09-21T14:58:40.826 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.826 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:40.826 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.826 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:40.826 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619969+0000", 2024-09-21T14:58:40.826 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:40.826 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.826 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.826 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:40.826 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619969+0000", 2024-09-21T14:58:40.826 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:40.826 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.826 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.827 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:40.827 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619969+0000", 2024-09-21T14:58:40.827 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:40.827 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.827 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.827 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:40.827 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619971+0000", 2024-09-21T14:58:40.827 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1830000000000001e-06 2024-09-21T14:58:40.827 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.827 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.827 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:40.827 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619976+0000", 2024-09-21T14:58:40.827 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7129999999999996e-06 2024-09-21T14:58:40.827 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.828 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.828 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:40.828 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619981+0000", 2024-09-21T14:58:40.828 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2460000000000003e-06 2024-09-21T14:58:40.828 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.828 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.828 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:40.828 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628602+0000", 2024-09-21T14:58:40.828 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086209519999999994 2024-09-21T14:58:40.828 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.828 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.828 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:40.828 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628734+0000", 2024-09-21T14:58:40.828 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000132242 2024-09-21T14:58:40.828 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.829 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.829 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:40.829 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628998+0000", 2024-09-21T14:58:40.829 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00026419799999999999 2024-09-21T14:58:40.829 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:40.829 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:40.829 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:40.829 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.829 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.829 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12215 3.4 3:21b43b45:::rbd_data.11363a21d2be.000000000000002b:10 [read 409600~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:40.829 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.619985+0000", 2024-09-21T14:58:40.829 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.1751027620000001, 2024-09-21T14:58:40.829 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0095457319999999995, 2024-09-21T14:58:40.829 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:40.829 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:40.830 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:40.830 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:40.830 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:40.830 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12215 2024-09-21T14:58:40.830 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.830 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:40.830 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.830 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:40.830 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619985+0000", 2024-09-21T14:58:40.830 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:40.830 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.830 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.830 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:40.830 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619985+0000", 2024-09-21T14:58:40.831 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:40.831 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.831 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.831 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:40.831 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619985+0000", 2024-09-21T14:58:40.831 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:40.831 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.831 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.831 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:40.831 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619987+0000", 2024-09-21T14:58:40.831 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6129999999999998e-06 2024-09-21T14:58:40.831 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.831 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.831 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:40.831 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619993+0000", 2024-09-21T14:58:40.832 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0820000000000004e-06 2024-09-21T14:58:40.832 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.832 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.832 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:40.832 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619998+0000", 2024-09-21T14:58:40.832 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7759999999999997e-06 2024-09-21T14:58:40.832 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.832 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.832 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:40.832 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629022+0000", 2024-09-21T14:58:40.832 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0090237489999999993 2024-09-21T14:58:40.832 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.832 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.832 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:40.832 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629098+0000", 2024-09-21T14:58:40.832 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5735999999999995e-05 2024-09-21T14:58:40.833 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.833 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.833 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:40.833 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629531+0000", 2024-09-21T14:58:40.833 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00043310100000000003 2024-09-21T14:58:40.833 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:40.833 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:40.833 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:40.833 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.833 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.833 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12216 3.4 3:21b43b45:::rbd_data.11363a21d2be.000000000000002b:10 [read 516096~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:40.833 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.620003+0000", 2024-09-21T14:58:40.833 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.1750847659999999, 2024-09-21T14:58:40.833 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0098850169999999994, 2024-09-21T14:58:40.833 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:40.834 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:40.834 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:40.834 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:40.834 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:40.834 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12216 2024-09-21T14:58:40.834 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.834 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:40.834 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.834 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:40.834 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620003+0000", 2024-09-21T14:58:40.834 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:40.834 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.834 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.834 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:40.834 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620003+0000", 2024-09-21T14:58:40.835 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:40.835 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.835 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.835 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:40.835 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620003+0000", 2024-09-21T14:58:40.835 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:40.835 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.835 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.835 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:40.835 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620006+0000", 2024-09-21T14:58:40.835 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4350000000000001e-06 2024-09-21T14:58:40.835 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.835 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.835 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:40.835 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620013+0000", 2024-09-21T14:58:40.836 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4219999999999997e-06 2024-09-21T14:58:40.836 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.836 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.836 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:40.836 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620017+0000", 2024-09-21T14:58:40.836 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.865e-06 2024-09-21T14:58:40.836 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.836 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.836 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:40.836 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629581+0000", 2024-09-21T14:58:40.836 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0095631989999999997 2024-09-21T14:58:40.836 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.836 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.836 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:40.836 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629691+0000", 2024-09-21T14:58:40.837 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000110586 2024-09-21T14:58:40.837 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.837 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.837 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:40.837 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629888+0000", 2024-09-21T14:58:40.837 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019691199999999999 2024-09-21T14:58:40.837 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:40.837 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:40.837 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:40.837 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.837 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.837 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12222 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 49152~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:40.837 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.620027+0000", 2024-09-21T14:58:40.837 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.1750604199999999, 2024-09-21T14:58:40.837 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010228131999999999, 2024-09-21T14:58:40.838 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:40.838 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:40.838 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:40.838 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:40.838 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:40.838 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12222 2024-09-21T14:58:40.838 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.838 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:40.838 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.838 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:40.838 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620027+0000", 2024-09-21T14:58:40.838 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:40.838 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.838 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.838 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:40.839 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620027+0000", 2024-09-21T14:58:40.839 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:40.839 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.839 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.839 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:40.839 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620027+0000", 2024-09-21T14:58:40.839 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:40.839 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.839 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.839 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:40.839 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620030+0000", 2024-09-21T14:58:40.839 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0400000000000001e-06 2024-09-21T14:58:40.839 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.839 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.839 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:40.840 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620037+0000", 2024-09-21T14:58:40.840 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8220000000000001e-06 2024-09-21T14:58:40.840 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.840 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.840 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:40.840 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620042+0000", 2024-09-21T14:58:40.840 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.843e-06 2024-09-21T14:58:40.840 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.840 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.840 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:40.840 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629922+0000", 2024-09-21T14:58:40.840 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0098805100000000003 2024-09-21T14:58:40.840 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.840 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.840 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:40.840 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.630020+0000", 2024-09-21T14:58:40.841 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7596999999999994e-05 2024-09-21T14:58:40.841 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.841 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.841 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:40.841 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.630256+0000", 2024-09-21T14:58:40.841 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00023573299999999999 2024-09-21T14:58:40.841 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:40.841 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:40.841 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:40.841 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.841 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.841 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12251 3.4 3:21b43b45:::rbd_data.11363a21d2be.000000000000002b:10 [read 278528~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:40.841 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.623999+0000", 2024-09-21T14:58:40.841 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.1710892669999999, 2024-09-21T14:58:40.841 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0081074140000000003, 2024-09-21T14:58:40.841 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:40.842 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:40.842 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:40.842 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:40.842 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:40.842 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12251 2024-09-21T14:58:40.842 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.842 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:40.842 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.842 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:40.842 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.623999+0000", 2024-09-21T14:58:40.842 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:40.842 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.842 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.842 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:40.842 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.623999+0000", 2024-09-21T14:58:40.843 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:40.843 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.843 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.843 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:40.843 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.623998+0000", 2024-09-21T14:58:40.843 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:40.843 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.843 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.843 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:40.843 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624000+0000", 2024-09-21T14:58:40.843 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1119999999999999e-06 2024-09-21T14:58:40.843 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.843 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.843 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:40.843 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624006+0000", 2024-09-21T14:58:40.843 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5559999999999998e-06 2024-09-21T14:58:40.844 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.844 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.844 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:40.844 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624028+0000", 2024-09-21T14:58:40.844 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1600999999999999e-05 2024-09-21T14:58:40.844 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.844 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.844 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:40.844 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.631819+0000", 2024-09-21T14:58:40.844 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0077913380000000001 2024-09-21T14:58:40.844 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.844 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.844 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:40.844 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.631903+0000", 2024-09-21T14:58:40.844 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4313000000000004e-05 2024-09-21T14:58:40.844 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.845 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.845 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:40.845 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.632106+0000", 2024-09-21T14:58:40.845 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00020284 2024-09-21T14:58:40.845 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:40.845 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:40.845 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:40.845 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.845 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.845 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12253 3.4 3:21b43b45:::rbd_data.11363a21d2be.000000000000002b:10 [read 417792~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:40.845 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.624578+0000", 2024-09-21T14:58:40.845 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.170510052, 2024-09-21T14:58:40.845 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0085112079999999993, 2024-09-21T14:58:40.845 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:40.845 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:40.845 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:40.846 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:40.846 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:40.846 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12253 2024-09-21T14:58:40.846 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.846 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:40.846 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.846 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:40.846 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624578+0000", 2024-09-21T14:58:40.846 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:40.846 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.846 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.846 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:40.846 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624578+0000", 2024-09-21T14:58:40.846 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:40.847 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.847 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.847 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:40.847 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624578+0000", 2024-09-21T14:58:40.847 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:40.847 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.847 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.847 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:40.847 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624579+0000", 2024-09-21T14:58:40.847 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.688e-06 2024-09-21T14:58:40.847 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.847 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.847 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:40.847 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624582+0000", 2024-09-21T14:58:40.848 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2260000000000001e-06 2024-09-21T14:58:40.848 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.848 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.848 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:40.848 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624585+0000", 2024-09-21T14:58:40.848 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4820000000000001e-06 2024-09-21T14:58:40.848 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.848 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.848 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:40.848 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.632580+0000", 2024-09-21T14:58:40.848 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0079948959999999996 2024-09-21T14:58:40.848 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.848 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.848 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:40.849 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.632683+0000", 2024-09-21T14:58:40.849 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010308 2024-09-21T14:58:40.849 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.849 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.849 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:40.849 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.633089+0000", 2024-09-21T14:58:40.849 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00040612599999999998 2024-09-21T14:58:40.849 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:40.849 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:40.849 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:40.849 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.849 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.849 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12260 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 57344~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:40.849 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.624587+0000", 2024-09-21T14:58:40.850 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.170500565, 2024-09-21T14:58:40.850 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0088282069999999994, 2024-09-21T14:58:40.850 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:40.850 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:40.850 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:40.850 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:40.850 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:40.850 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12260 2024-09-21T14:58:40.850 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.850 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:40.850 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.850 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:40.850 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624587+0000", 2024-09-21T14:58:40.850 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:40.851 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.851 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.851 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:40.851 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624587+0000", 2024-09-21T14:58:40.851 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:40.851 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.851 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.851 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:40.851 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624587+0000", 2024-09-21T14:58:40.851 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-21T14:58:40.851 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.851 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.851 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:40.851 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624589+0000", 2024-09-21T14:58:40.852 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4339999999999999e-06 2024-09-21T14:58:40.852 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.852 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.852 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:40.852 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624593+0000", 2024-09-21T14:58:40.852 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5859999999999998e-06 2024-09-21T14:58:40.852 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.852 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.852 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:40.852 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624596+0000", 2024-09-21T14:58:40.852 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5830000000000001e-06 2024-09-21T14:58:40.852 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.852 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.852 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:40.852 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.633132+0000", 2024-09-21T14:58:40.853 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.008536666 2024-09-21T14:58:40.853 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.853 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.853 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:40.853 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.633216+0000", 2024-09-21T14:58:40.853 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3387999999999995e-05 2024-09-21T14:58:40.853 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.853 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.853 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:40.853 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.633415+0000", 2024-09-21T14:58:40.853 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019976799999999999 2024-09-21T14:58:40.853 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:40.853 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:40.853 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:40.854 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.854 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.854 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12516 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 245760~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:40.854 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645194+0000", 2024-09-21T14:58:40.854 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.1498940280000001, 2024-09-21T14:58:40.854 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0084284990000000008, 2024-09-21T14:58:40.854 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:40.854 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:40.854 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:40.854 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:40.854 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:40.854 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12516 2024-09-21T14:58:40.854 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.854 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:40.854 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.855 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:40.855 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645194+0000", 2024-09-21T14:58:40.855 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:40.855 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.855 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.855 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:40.855 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645194+0000", 2024-09-21T14:58:40.855 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:40.855 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.855 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.855 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:40.855 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645193+0000", 2024-09-21T14:58:40.855 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:40.855 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.855 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.856 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:40.856 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645196+0000", 2024-09-21T14:58:40.856 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1030000000000002e-06 2024-09-21T14:58:40.856 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.856 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.856 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:40.856 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645205+0000", 2024-09-21T14:58:40.856 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6200000000000005e-06 2024-09-21T14:58:40.856 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.856 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.856 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:40.856 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645210+0000", 2024-09-21T14:58:40.856 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6280000000000004e-06 2024-09-21T14:58:40.856 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.856 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.857 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:40.857 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.653305+0000", 2024-09-21T14:58:40.857 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0080957879999999996 2024-09-21T14:58:40.857 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.857 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.857 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:40.857 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.653389+0000", 2024-09-21T14:58:40.857 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3242e-05 2024-09-21T14:58:40.857 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.857 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.857 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:40.857 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.653622+0000", 2024-09-21T14:58:40.857 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00023361599999999999 2024-09-21T14:58:40.857 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:40.858 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:40.858 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:40.858 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.858 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.858 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12517 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 278528~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:40.858 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645214+0000", 2024-09-21T14:58:40.858 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.1498739149999999, 2024-09-21T14:58:40.858 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086832899999999998, 2024-09-21T14:58:40.858 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:40.858 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:40.858 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:40.858 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:40.858 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:40.858 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12517 2024-09-21T14:58:40.858 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.859 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:40.859 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.859 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:40.859 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645214+0000", 2024-09-21T14:58:40.859 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:40.859 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.859 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.859 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:40.859 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645214+0000", 2024-09-21T14:58:40.859 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:40.859 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.859 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.859 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:40.859 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645214+0000", 2024-09-21T14:58:40.859 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:40.859 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.860 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.860 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:40.860 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645216+0000", 2024-09-21T14:58:40.860 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5579999999999999e-06 2024-09-21T14:58:40.860 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.860 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.860 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:40.860 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645224+0000", 2024-09-21T14:58:40.860 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2439999999999993e-06 2024-09-21T14:58:40.860 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.860 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.860 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:40.860 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645231+0000", 2024-09-21T14:58:40.860 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1859999999999997e-06 2024-09-21T14:58:40.860 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.860 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.861 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:40.861 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.653651+0000", 2024-09-21T14:58:40.861 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0084204220000000003 2024-09-21T14:58:40.861 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.861 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.861 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:40.861 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.653726+0000", 2024-09-21T14:58:40.861 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5356000000000002e-05 2024-09-21T14:58:40.861 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.861 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.861 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:40.861 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.653897+0000", 2024-09-21T14:58:40.861 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000170848 2024-09-21T14:58:40.861 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:40.861 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:40.861 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:40.862 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.862 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.862 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12518 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 385024~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:40.862 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645235+0000", 2024-09-21T14:58:40.862 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.149852689, 2024-09-21T14:58:40.862 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0089841020000000008, 2024-09-21T14:58:40.862 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:40.862 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:40.862 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:40.862 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:40.862 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:40.862 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12518 2024-09-21T14:58:40.862 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.862 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:40.862 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.863 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:40.863 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645235+0000", 2024-09-21T14:58:40.863 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:40.863 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.863 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.863 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:40.863 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645235+0000", 2024-09-21T14:58:40.863 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:40.863 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.863 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.863 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:40.863 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645235+0000", 2024-09-21T14:58:40.863 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:40.863 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.863 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.863 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:40.864 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645238+0000", 2024-09-21T14:58:40.864 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.869e-06 2024-09-21T14:58:40.864 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.864 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.864 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:40.864 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645244+0000", 2024-09-21T14:58:40.864 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8930000000000002e-06 2024-09-21T14:58:40.864 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.864 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.864 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:40.864 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645249+0000", 2024-09-21T14:58:40.864 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2859999999999999e-06 2024-09-21T14:58:40.864 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.864 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.864 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:40.864 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.653930+0000", 2024-09-21T14:58:40.865 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086815320000000005 2024-09-21T14:58:40.865 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.865 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.865 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:40.865 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654028+0000", 2024-09-21T14:58:40.865 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7578000000000007e-05 2024-09-21T14:58:40.865 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.865 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.865 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:40.865 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654219+0000", 2024-09-21T14:58:40.865 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019132099999999999 2024-09-21T14:58:40.865 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:40.865 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:40.865 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:40.865 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.865 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.866 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12538 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 147456~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:40.866 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645253+0000", 2024-09-21T14:58:40.866 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.149835065, 2024-09-21T14:58:40.866 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0092095360000000008, 2024-09-21T14:58:40.866 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:40.866 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:40.866 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:40.866 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:40.866 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:40.866 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12538 2024-09-21T14:58:40.866 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.866 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:40.866 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.866 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:40.866 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645253+0000", 2024-09-21T14:58:40.867 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:40.867 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.867 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.867 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:40.867 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645253+0000", 2024-09-21T14:58:40.867 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:40.867 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.867 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.867 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:40.867 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645252+0000", 2024-09-21T14:58:40.867 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:40.867 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.867 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.867 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:40.868 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645255+0000", 2024-09-21T14:58:40.868 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2290000000000002e-06 2024-09-21T14:58:40.868 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.868 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.868 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:40.868 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645267+0000", 2024-09-21T14:58:40.868 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2631e-05 2024-09-21T14:58:40.868 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.868 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.868 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:40.868 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645278+0000", 2024-09-21T14:58:40.868 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0847e-05 2024-09-21T14:58:40.868 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.868 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.868 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:40.869 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654251+0000", 2024-09-21T14:58:40.869 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0089731340000000007 2024-09-21T14:58:40.869 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.869 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.869 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:40.869 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654296+0000", 2024-09-21T14:58:40.869 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4353e-05 2024-09-21T14:58:40.869 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.869 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.869 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:40.869 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654462+0000", 2024-09-21T14:58:40.869 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016668499999999999 2024-09-21T14:58:40.869 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:40.869 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:40.869 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:40.870 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.870 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.870 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12539 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 253952~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:40.870 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645284+0000", 2024-09-21T14:58:40.870 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.149803844, 2024-09-21T14:58:40.870 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0094524029999999998, 2024-09-21T14:58:40.870 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:40.870 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:40.870 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:40.870 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:40.870 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:40.870 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12539 2024-09-21T14:58:40.870 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.870 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:40.870 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.870 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:40.871 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645284+0000", 2024-09-21T14:58:40.871 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:40.871 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.871 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.871 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:40.871 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645284+0000", 2024-09-21T14:58:40.871 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:40.871 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.871 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.871 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:40.871 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645284+0000", 2024-09-21T14:58:40.871 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:40.871 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.871 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.871 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:40.872 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645286+0000", 2024-09-21T14:58:40.872 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7159999999999999e-06 2024-09-21T14:58:40.872 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.872 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.872 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:40.872 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645292+0000", 2024-09-21T14:58:40.872 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7400000000000001e-06 2024-09-21T14:58:40.872 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.872 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.872 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:40.872 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645297+0000", 2024-09-21T14:58:40.872 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9019999999999998e-06 2024-09-21T14:58:40.872 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.872 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.872 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:40.872 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654490+0000", 2024-09-21T14:58:40.873 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0091932360000000005 2024-09-21T14:58:40.873 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.873 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.873 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:40.873 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654565+0000", 2024-09-21T14:58:40.873 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4657e-05 2024-09-21T14:58:40.873 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.873 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.873 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:40.873 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654736+0000", 2024-09-21T14:58:40.873 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017156199999999999 2024-09-21T14:58:40.873 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:40.873 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:40.873 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:40.873 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.873 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.874 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12540 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 286720~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:40.874 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645302+0000", 2024-09-21T14:58:40.874 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.1497855750000001, 2024-09-21T14:58:40.874 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0096603090000000006, 2024-09-21T14:58:40.874 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:40.874 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:40.874 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:40.874 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:40.874 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:40.874 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12540 2024-09-21T14:58:40.874 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.874 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:40.874 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.874 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:40.874 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645302+0000", 2024-09-21T14:58:40.875 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:40.875 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.875 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.875 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:40.875 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645302+0000", 2024-09-21T14:58:40.875 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:40.875 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.875 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.875 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:40.875 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645302+0000", 2024-09-21T14:58:40.875 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:40.875 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.875 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.875 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:40.875 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645305+0000", 2024-09-21T14:58:40.875 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7329999999999999e-06 2024-09-21T14:58:40.876 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.876 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.876 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:40.876 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645316+0000", 2024-09-21T14:58:40.876 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1832e-05 2024-09-21T14:58:40.876 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.876 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.876 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:40.876 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645322+0000", 2024-09-21T14:58:40.876 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.169e-06 2024-09-21T14:58:40.876 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.876 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.876 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:40.876 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654778+0000", 2024-09-21T14:58:40.876 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0094559090000000002 2024-09-21T14:58:40.876 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.877 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.877 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:40.877 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654828+0000", 2024-09-21T14:58:40.877 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0461999999999999e-05 2024-09-21T14:58:40.877 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.877 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.877 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:40.877 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654963+0000", 2024-09-21T14:58:40.877 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000134583 2024-09-21T14:58:40.877 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:40.877 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:40.877 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:40.877 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.877 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.877 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12561 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:40.878 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645742+0000", 2024-09-21T14:58:40.878 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.149346309, 2024-09-21T14:58:40.878 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0093583810000000007, 2024-09-21T14:58:40.878 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:40.878 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:40.878 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:40.878 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:40.878 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:40.878 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12561 2024-09-21T14:58:40.878 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.878 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:40.878 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.878 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:40.878 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645742+0000", 2024-09-21T14:58:40.878 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:40.878 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.879 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.879 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:40.879 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645742+0000", 2024-09-21T14:58:40.879 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:40.879 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.879 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.879 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:40.879 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645741+0000", 2024-09-21T14:58:40.879 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:40.879 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.879 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.879 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:40.879 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645746+0000", 2024-09-21T14:58:40.879 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8379999999999998e-06 2024-09-21T14:58:40.879 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.879 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.880 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:40.880 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645757+0000", 2024-09-21T14:58:40.880 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0705e-05 2024-09-21T14:58:40.880 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.880 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.880 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:40.880 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645765+0000", 2024-09-21T14:58:40.880 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6729999999999992e-06 2024-09-21T14:58:40.880 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.880 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.880 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:40.880 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654993+0000", 2024-09-21T14:58:40.880 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0092287480000000002 2024-09-21T14:58:40.880 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.880 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.880 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:40.880 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.655050+0000", 2024-09-21T14:58:40.881 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6394999999999998e-05 2024-09-21T14:58:40.881 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:40.881 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:40.881 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:40.881 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.655100+0000", 2024-09-21T14:58:40.881 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0049000000000001e-05 2024-09-21T14:58:40.881 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:40.881 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:40.881 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:40.881 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:40.881 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:40.881 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:58:40.882 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T14:58:40.883 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:58:40.880+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:58:40.983 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:58:40.980+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:58:41.043 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:58:41.043 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:58:41.043 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T14:58:41.043 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:58:41.053 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T14:58:41.083 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:58:41.080+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:58:41.113 INFO:teuthology.orchestra.run.smithi032.stderr:noscrub is set 2024-09-21T14:58:41.166 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:58:41.166 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:58:41.166 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T14:58:41.166 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T14:58:41.166 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:58:41.176 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T14:58:41.184 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:58:41.180+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:58:41.282 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:58:41.283 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T14:58:41.283 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T14:58:41.283 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T14:58:41.283 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.283 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:341 3.b 3:d4183149:::rbd_data.11367ffd86fe.0000000000000021:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T14:58:41.283 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:25.462758+0000", 2024-09-21T14:58:41.283 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 15.817318243999999, 2024-09-21T14:58:41.283 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0044292250000000002, 2024-09-21T14:58:41.283 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:41.283 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:58:41.283 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:41.283 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:41.283 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:41.283 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 341 2024-09-21T14:58:41.284 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.284 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:41.284 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.284 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:41.284 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.462758+0000", 2024-09-21T14:58:41.284 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:41.284 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.284 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.284 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:41.284 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.462758+0000", 2024-09-21T14:58:41.284 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:41.284 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.284 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.284 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:41.284 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.462756+0000", 2024-09-21T14:58:41.285 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-21T14:58:41.285 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.285 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.285 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:41.285 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.462768+0000", 2024-09-21T14:58:41.285 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2384e-05 2024-09-21T14:58:41.285 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.285 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.285 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:41.285 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.462797+0000", 2024-09-21T14:58:41.285 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8666999999999998e-05 2024-09-21T14:58:41.285 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.285 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.285 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:41.285 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.462814+0000", 2024-09-21T14:58:41.285 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6730000000000001e-05 2024-09-21T14:58:41.286 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.286 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.286 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:41.286 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.463191+0000", 2024-09-21T14:58:41.286 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000377096 2024-09-21T14:58:41.286 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.286 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.286 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op must wait for map", 2024-09-21T14:58:41.286 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.463204+0000", 2024-09-21T14:58:41.286 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3777999999999999e-05 2024-09-21T14:58:41.286 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.286 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.286 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:41.287 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.467079+0000", 2024-09-21T14:58:41.287 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.003874307 2024-09-21T14:58:41.287 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.287 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.287 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:41.287 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.467187+0000", 2024-09-21T14:58:41.287 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000108451 2024-09-21T14:58:41.287 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:41.287 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:41.287 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:41.287 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.287 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.287 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:357 3.b 3:d4183149:::rbd_data.11367ffd86fe.0000000000000021:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T14:58:41.287 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:25.462843+0000", 2024-09-21T14:58:41.287 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 15.817233672, 2024-09-21T14:58:41.288 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0044394409999999997, 2024-09-21T14:58:41.288 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:41.288 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:58:41.288 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:41.288 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:41.288 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:41.288 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 357 2024-09-21T14:58:41.288 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.288 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:41.288 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.288 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:41.288 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.462843+0000", 2024-09-21T14:58:41.288 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:41.288 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.288 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.288 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:41.289 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.462843+0000", 2024-09-21T14:58:41.289 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:41.289 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.289 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.289 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:41.289 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.462842+0000", 2024-09-21T14:58:41.289 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:41.289 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.289 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.289 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:41.289 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.462848+0000", 2024-09-21T14:58:41.289 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9020000000000002e-06 2024-09-21T14:58:41.290 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.290 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.290 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:41.290 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.462868+0000", 2024-09-21T14:58:41.290 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0088000000000001e-05 2024-09-21T14:58:41.290 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.290 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.290 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:41.290 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.462897+0000", 2024-09-21T14:58:41.290 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8595e-05 2024-09-21T14:58:41.290 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.290 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.290 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:41.290 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.463327+0000", 2024-09-21T14:58:41.291 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00042943899999999999 2024-09-21T14:58:41.291 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.291 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.291 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting_for_map not empty", 2024-09-21T14:58:41.291 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.463339+0000", 2024-09-21T14:58:41.291 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2767e-05 2024-09-21T14:58:41.291 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.291 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.291 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:41.291 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.467209+0000", 2024-09-21T14:58:41.291 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.003869295 2024-09-21T14:58:41.291 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.291 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.291 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:41.291 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:25.467282+0000", 2024-09-21T14:58:41.291 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3399000000000007e-05 2024-09-21T14:58:41.292 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:41.292 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:41.292 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:41.292 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.292 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.292 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:7969 3.b 3:d2c9f41f:::rbd_data.11361bed1f78.000000000000005b:d [read 0~32768 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:41.292 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:35.227077+0000", 2024-09-21T14:58:41.292 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 6.052999485, 2024-09-21T14:58:41.292 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0045264379999999998, 2024-09-21T14:58:41.292 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:41.292 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:41.292 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:41.292 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:41.292 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:41.292 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 7969 2024-09-21T14:58:41.292 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.293 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:41.293 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.293 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:41.293 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227077+0000", 2024-09-21T14:58:41.293 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:41.293 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.293 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.293 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:41.293 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227077+0000", 2024-09-21T14:58:41.293 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:41.293 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.293 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.293 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:41.293 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227076+0000", 2024-09-21T14:58:41.293 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:41.293 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.294 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.294 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:41.294 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227079+0000", 2024-09-21T14:58:41.294 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3520000000000001e-06 2024-09-21T14:58:41.294 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.294 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.294 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:41.294 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227089+0000", 2024-09-21T14:58:41.294 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0365e-05 2024-09-21T14:58:41.294 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.294 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.294 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:41.294 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227095+0000", 2024-09-21T14:58:41.294 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7289999999999997e-06 2024-09-21T14:58:41.294 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.295 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.295 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:41.295 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.228787+0000", 2024-09-21T14:58:41.295 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0016917919999999999 2024-09-21T14:58:41.295 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.295 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.295 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:41.295 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.228865+0000", 2024-09-21T14:58:41.295 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8784999999999999e-05 2024-09-21T14:58:41.295 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.295 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.295 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:41.295 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.231603+0000", 2024-09-21T14:58:41.295 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.002737766 2024-09-21T14:58:41.295 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:41.295 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:41.296 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:41.296 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.296 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.296 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:7985 3.b 3:d03c2ae1:::rbd_data.11361bed1f78.0000000000000056:d [read 212992~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:41.296 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:35.227119+0000", 2024-09-21T14:58:41.296 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 6.0529574210000003, 2024-09-21T14:58:41.296 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0042232390000000002, 2024-09-21T14:58:41.296 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:41.296 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:41.296 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:41.296 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:41.296 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:41.296 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 7985 2024-09-21T14:58:41.296 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.297 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:41.297 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.297 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:41.297 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227119+0000", 2024-09-21T14:58:41.297 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:41.297 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.297 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.297 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:41.297 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227119+0000", 2024-09-21T14:58:41.297 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:41.297 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.297 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.297 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:41.297 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227118+0000", 2024-09-21T14:58:41.297 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:41.298 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.298 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.298 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:41.298 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227121+0000", 2024-09-21T14:58:41.298 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0479999999999999e-06 2024-09-21T14:58:41.298 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.298 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.298 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:41.298 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227132+0000", 2024-09-21T14:58:41.298 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.04e-05 2024-09-21T14:58:41.298 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.298 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.298 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:41.298 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227138+0000", 2024-09-21T14:58:41.299 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0270000000000001e-06 2024-09-21T14:58:41.299 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.299 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.299 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:41.299 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.231120+0000", 2024-09-21T14:58:41.299 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0039821750000000001 2024-09-21T14:58:41.299 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.299 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.299 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:41.299 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.231225+0000", 2024-09-21T14:58:41.299 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010473 2024-09-21T14:58:41.299 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.299 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.299 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:41.299 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.231342+0000", 2024-09-21T14:58:41.300 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000117244 2024-09-21T14:58:41.300 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:41.300 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:41.300 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:41.300 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.300 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.300 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:7990 3.b 3:d2c9f41f:::rbd_data.11361bed1f78.000000000000005b:d [read 32768~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:41.300 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:35.227143+0000", 2024-09-21T14:58:41.300 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 6.052932974, 2024-09-21T14:58:41.300 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0045399860000000002, 2024-09-21T14:58:41.300 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:41.300 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:41.301 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:41.301 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:41.301 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:41.301 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 7990 2024-09-21T14:58:41.301 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.301 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:41.301 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.301 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:41.301 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227143+0000", 2024-09-21T14:58:41.301 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:41.301 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.301 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.301 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:41.301 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227143+0000", 2024-09-21T14:58:41.301 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:41.301 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.302 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.302 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:41.302 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227143+0000", 2024-09-21T14:58:41.302 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:41.302 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.302 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.302 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:41.302 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227145+0000", 2024-09-21T14:58:41.302 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7130000000000001e-06 2024-09-21T14:58:41.302 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.302 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.302 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:41.302 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227157+0000", 2024-09-21T14:58:41.303 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1501999999999999e-05 2024-09-21T14:58:41.303 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.303 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.303 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:41.303 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227163+0000", 2024-09-21T14:58:41.303 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5450000000000003e-06 2024-09-21T14:58:41.303 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.303 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.303 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:41.303 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.231368+0000", 2024-09-21T14:58:41.303 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0042056430000000002 2024-09-21T14:58:41.303 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.303 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.303 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:41.303 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.231464+0000", 2024-09-21T14:58:41.304 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6043000000000002e-05 2024-09-21T14:58:41.304 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.304 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.304 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:41.304 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.231683+0000", 2024-09-21T14:58:41.304 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000219008 2024-09-21T14:58:41.304 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:41.304 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:41.304 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:41.304 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.304 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.304 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:7991 3.b 3:d2c9f41f:::rbd_data.11361bed1f78.000000000000005b:d [read 98304~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:41.304 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:35.227168+0000", 2024-09-21T14:58:41.304 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 6.0529083039999998, 2024-09-21T14:58:41.305 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0048739430000000004, 2024-09-21T14:58:41.305 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:41.305 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:41.305 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:41.305 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:41.305 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:41.305 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 7991 2024-09-21T14:58:41.305 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.305 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:41.305 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.305 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:41.305 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227168+0000", 2024-09-21T14:58:41.305 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:41.305 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.305 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.305 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:41.306 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227168+0000", 2024-09-21T14:58:41.306 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:41.306 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.306 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.306 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:41.306 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227167+0000", 2024-09-21T14:58:41.306 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:41.306 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.306 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.306 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:41.306 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227172+0000", 2024-09-21T14:58:41.306 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.561e-06 2024-09-21T14:58:41.306 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.306 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.307 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:41.307 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227179+0000", 2024-09-21T14:58:41.307 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1119999999999998e-06 2024-09-21T14:58:41.307 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.307 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.307 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:41.307 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227186+0000", 2024-09-21T14:58:41.307 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.046e-06 2024-09-21T14:58:41.307 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.307 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.307 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:41.307 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.231717+0000", 2024-09-21T14:58:41.307 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0045312190000000004 2024-09-21T14:58:41.307 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.308 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.308 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:41.308 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.231823+0000", 2024-09-21T14:58:41.308 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000105303 2024-09-21T14:58:41.308 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.308 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.308 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:41.308 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.232042+0000", 2024-09-21T14:58:41.308 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021928599999999999 2024-09-21T14:58:41.308 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:41.308 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:41.308 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:41.308 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.308 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.308 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:8004 3.b 3:d2c9f41f:::rbd_data.11361bed1f78.000000000000005b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:41.309 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:35.227192+0000", 2024-09-21T14:58:41.309 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 6.0528845100000002, 2024-09-21T14:58:41.309 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0052132289999999998, 2024-09-21T14:58:41.309 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:41.309 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:41.309 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:41.309 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:41.309 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:41.309 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 8004 2024-09-21T14:58:41.309 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.309 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:41.309 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.309 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:41.309 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227192+0000", 2024-09-21T14:58:41.309 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:41.310 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.310 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.310 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:41.310 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227192+0000", 2024-09-21T14:58:41.310 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:41.310 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.310 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.310 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:41.310 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227191+0000", 2024-09-21T14:58:41.310 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:41.310 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.310 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.310 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:41.311 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227194+0000", 2024-09-21T14:58:41.311 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8990000000000001e-06 2024-09-21T14:58:41.311 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.311 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.311 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:41.311 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227201+0000", 2024-09-21T14:58:41.311 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5810000000000002e-06 2024-09-21T14:58:41.311 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.311 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.311 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:41.311 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227206+0000", 2024-09-21T14:58:41.311 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.694e-06 2024-09-21T14:58:41.312 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.312 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.312 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:41.312 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.232148+0000", 2024-09-21T14:58:41.312 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00494174 2024-09-21T14:58:41.312 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.312 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.312 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:41.312 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.232308+0000", 2024-09-21T14:58:41.312 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015929599999999999 2024-09-21T14:58:41.312 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.312 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.313 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:41.313 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.232405+0000", 2024-09-21T14:58:41.313 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7415000000000002e-05 2024-09-21T14:58:41.313 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:41.313 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:41.313 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:41.313 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.313 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.313 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:8011 3.b 3:d1b12ef9:::rbd_data.11361bed1f78.0000000000000062:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:41.313 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:35.227212+0000", 2024-09-21T14:58:41.313 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 6.0528646500000001, 2024-09-21T14:58:41.313 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0054249789999999999, 2024-09-21T14:58:41.313 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:41.314 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:41.314 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:41.314 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:41.314 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:41.314 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 8011 2024-09-21T14:58:41.314 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.314 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:41.314 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.314 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:41.314 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227212+0000", 2024-09-21T14:58:41.314 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:41.314 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.314 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.315 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:41.315 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227212+0000", 2024-09-21T14:58:41.315 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:41.315 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.315 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.315 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:41.315 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227211+0000", 2024-09-21T14:58:41.315 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:41.315 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.315 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.315 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:41.315 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227214+0000", 2024-09-21T14:58:41.315 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7999999999999999e-06 2024-09-21T14:58:41.315 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.315 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.316 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:41.316 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227220+0000", 2024-09-21T14:58:41.316 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0569999999999998e-06 2024-09-21T14:58:41.316 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.316 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.316 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:41.316 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227223+0000", 2024-09-21T14:58:41.316 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3000000000000002e-06 2024-09-21T14:58:41.316 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.316 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.316 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:41.316 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.232445+0000", 2024-09-21T14:58:41.316 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0052220160000000003 2024-09-21T14:58:41.316 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.317 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.317 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:41.317 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.232535+0000", 2024-09-21T14:58:41.317 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9866999999999999e-05 2024-09-21T14:58:41.317 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.317 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.317 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:41.317 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.232637+0000", 2024-09-21T14:58:41.317 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010155 2024-09-21T14:58:41.317 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:41.317 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:41.317 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:41.317 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.317 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.318 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:8019 3.b 3:d1b12ef9:::rbd_data.11361bed1f78.0000000000000062:d [read 0~32768 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:41.318 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:35.227226+0000", 2024-09-21T14:58:41.318 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 6.0528500489999999, 2024-09-21T14:58:41.318 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.005662696, 2024-09-21T14:58:41.318 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:41.318 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:41.318 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:41.318 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:41.318 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:41.318 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 8019 2024-09-21T14:58:41.318 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.318 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:41.318 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.318 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:41.319 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227226+0000", 2024-09-21T14:58:41.319 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:41.319 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.319 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.319 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:41.319 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227226+0000", 2024-09-21T14:58:41.319 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:41.319 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.319 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.319 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:41.319 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227226+0000", 2024-09-21T14:58:41.319 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:41.319 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.319 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.319 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:41.319 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227228+0000", 2024-09-21T14:58:41.320 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5919999999999999e-06 2024-09-21T14:58:41.320 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.320 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.320 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:41.320 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227235+0000", 2024-09-21T14:58:41.320 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2650000000000002e-06 2024-09-21T14:58:41.320 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.320 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.320 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:41.320 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227241+0000", 2024-09-21T14:58:41.320 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9139999999999996e-06 2024-09-21T14:58:41.320 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.321 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.321 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:41.321 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.232671+0000", 2024-09-21T14:58:41.321 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0054301879999999999 2024-09-21T14:58:41.321 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.321 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.321 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:41.321 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.232747+0000", 2024-09-21T14:58:41.321 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5988999999999994e-05 2024-09-21T14:58:41.321 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.321 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.321 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:41.321 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.232889+0000", 2024-09-21T14:58:41.321 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014205199999999999 2024-09-21T14:58:41.321 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:41.321 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:41.322 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:41.322 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.322 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.322 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:8020 3.b 3:d1b12ef9:::rbd_data.11361bed1f78.0000000000000062:d [read 81920~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:41.322 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:35.227246+0000", 2024-09-21T14:58:41.322 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 6.0528304559999997, 2024-09-21T14:58:41.322 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0061395770000000002, 2024-09-21T14:58:41.322 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:41.322 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:41.322 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:41.322 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:41.322 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:41.322 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 8020 2024-09-21T14:58:41.322 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.323 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:41.323 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.323 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:41.323 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227246+0000", 2024-09-21T14:58:41.323 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:41.323 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.323 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.323 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:41.323 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227246+0000", 2024-09-21T14:58:41.323 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:41.323 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.323 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.323 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:41.323 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227245+0000", 2024-09-21T14:58:41.324 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:41.324 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.324 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.324 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:41.324 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227248+0000", 2024-09-21T14:58:41.324 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8119999999999999e-06 2024-09-21T14:58:41.324 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.324 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.324 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:41.324 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227255+0000", 2024-09-21T14:58:41.324 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7549999999999997e-06 2024-09-21T14:58:41.324 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.324 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.324 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:41.324 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227260+0000", 2024-09-21T14:58:41.325 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.271e-06 2024-09-21T14:58:41.325 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.325 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.325 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:41.325 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.232967+0000", 2024-09-21T14:58:41.325 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0057065179999999998 2024-09-21T14:58:41.325 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.325 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.325 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:41.325 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.233108+0000", 2024-09-21T14:58:41.325 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000141123 2024-09-21T14:58:41.325 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.325 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.325 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:41.325 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.233385+0000", 2024-09-21T14:58:41.326 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00027743400000000001 2024-09-21T14:58:41.326 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:41.326 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:41.326 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:41.326 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.326 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.326 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:8034 3.b 3:d1b12ef9:::rbd_data.11361bed1f78.0000000000000062:d [read 32768~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:41.326 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:35.227265+0000", 2024-09-21T14:58:41.326 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 6.0528110890000004, 2024-09-21T14:58:41.326 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.006525968, 2024-09-21T14:58:41.326 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:41.326 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:41.326 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:41.326 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:41.327 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:41.327 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 8034 2024-09-21T14:58:41.327 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.327 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:41.327 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.327 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:41.327 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227265+0000", 2024-09-21T14:58:41.327 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:41.327 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.327 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.327 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:41.327 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227265+0000", 2024-09-21T14:58:41.327 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:41.328 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.328 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.328 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:41.328 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227265+0000", 2024-09-21T14:58:41.328 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:41.328 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.328 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.328 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:41.328 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227268+0000", 2024-09-21T14:58:41.328 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8990000000000001e-06 2024-09-21T14:58:41.328 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.328 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.328 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:41.329 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227274+0000", 2024-09-21T14:58:41.329 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4470000000000003e-06 2024-09-21T14:58:41.329 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.329 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.329 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:41.329 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227288+0000", 2024-09-21T14:58:41.329 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3550000000000001e-05 2024-09-21T14:58:41.329 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.329 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.329 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:41.329 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.233418+0000", 2024-09-21T14:58:41.329 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0061307870000000004 2024-09-21T14:58:41.329 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.330 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.330 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:41.330 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.233553+0000", 2024-09-21T14:58:41.330 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013475600000000001 2024-09-21T14:58:41.330 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.330 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.330 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:41.330 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.233791+0000", 2024-09-21T14:58:41.331 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00023788499999999999 2024-09-21T14:58:41.331 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:41.331 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:41.331 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:41.331 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.331 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.331 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:8035 3.b 3:d1b12ef9:::rbd_data.11361bed1f78.0000000000000062:d [read 98304~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:41.331 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:35.227295+0000", 2024-09-21T14:58:41.331 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 6.0527808839999997, 2024-09-21T14:58:41.331 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0069673820000000003, 2024-09-21T14:58:41.331 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:41.331 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:41.331 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:41.331 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:41.331 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:41.332 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 8035 2024-09-21T14:58:41.332 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.332 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:41.332 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.332 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:41.332 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227295+0000", 2024-09-21T14:58:41.332 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:41.332 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.332 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.332 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:41.332 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227295+0000", 2024-09-21T14:58:41.332 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:41.332 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.332 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.333 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:41.333 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227295+0000", 2024-09-21T14:58:41.333 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:41.333 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.333 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.333 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:41.333 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227298+0000", 2024-09-21T14:58:41.333 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2339999999999999e-06 2024-09-21T14:58:41.333 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.333 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.333 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:41.333 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227305+0000", 2024-09-21T14:58:41.333 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0389999999999999e-06 2024-09-21T14:58:41.333 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.333 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.334 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:41.334 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227311+0000", 2024-09-21T14:58:41.334 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8470000000000001e-06 2024-09-21T14:58:41.334 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.334 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.334 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:41.334 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.233824+0000", 2024-09-21T14:58:41.334 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0065132920000000004 2024-09-21T14:58:41.334 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.334 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.334 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:41.334 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.233922+0000", 2024-09-21T14:58:41.334 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.8073999999999996e-05 2024-09-21T14:58:41.334 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.335 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.335 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:41.335 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.234263+0000", 2024-09-21T14:58:41.335 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00034027999999999999 2024-09-21T14:58:41.335 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:41.335 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:41.335 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:41.335 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.335 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.335 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:8048 3.b 3:d1b12ef9:::rbd_data.11361bed1f78.0000000000000062:d [read 49152~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:41.335 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:35.227316+0000", 2024-09-21T14:58:41.335 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 6.0527600919999998, 2024-09-21T14:58:41.335 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0076197799999999996, 2024-09-21T14:58:41.335 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:41.336 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:41.336 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:41.336 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:41.336 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:41.336 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 8048 2024-09-21T14:58:41.336 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.336 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:41.336 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.336 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:41.336 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227316+0000", 2024-09-21T14:58:41.336 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:41.336 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.336 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.336 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:41.336 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227316+0000", 2024-09-21T14:58:41.336 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:41.337 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.337 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.337 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:41.337 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227316+0000", 2024-09-21T14:58:41.337 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:41.337 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.337 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.337 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:41.337 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227324+0000", 2024-09-21T14:58:41.337 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1009999999999992e-06 2024-09-21T14:58:41.337 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.337 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.337 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:41.337 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227346+0000", 2024-09-21T14:58:41.338 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2347000000000001e-05 2024-09-21T14:58:41.338 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.338 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.338 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:41.338 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227360+0000", 2024-09-21T14:58:41.338 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.403e-05 2024-09-21T14:58:41.338 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.338 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.338 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:41.338 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.234320+0000", 2024-09-21T14:58:41.338 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0069597019999999999 2024-09-21T14:58:41.338 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.338 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.338 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:41.338 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.234730+0000", 2024-09-21T14:58:41.339 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00041006300000000001 2024-09-21T14:58:41.339 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.339 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.339 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:41.339 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.234936+0000", 2024-09-21T14:58:41.339 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00020586700000000001 2024-09-21T14:58:41.339 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:41.339 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:41.339 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:41.339 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.339 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.339 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:8049 3.b 3:d1b12ef9:::rbd_data.11361bed1f78.0000000000000062:d [read 114688~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:41.339 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:35.227371+0000", 2024-09-21T14:58:41.339 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 6.052704759, 2024-09-21T14:58:41.339 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0078533609999999997, 2024-09-21T14:58:41.340 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:41.340 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:41.340 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:41.340 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:41.340 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:41.340 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 8049 2024-09-21T14:58:41.340 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.340 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:41.340 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.340 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:41.340 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227371+0000", 2024-09-21T14:58:41.340 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:41.340 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.340 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.340 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:41.340 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227371+0000", 2024-09-21T14:58:41.341 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:41.341 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.341 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.341 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:41.341 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227370+0000", 2024-09-21T14:58:41.341 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:41.341 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.341 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.341 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:41.341 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227378+0000", 2024-09-21T14:58:41.341 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.379e-06 2024-09-21T14:58:41.341 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.341 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.342 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:41.342 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227388+0000", 2024-09-21T14:58:41.342 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0175e-05 2024-09-21T14:58:41.342 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.342 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.342 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:41.342 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227393+0000", 2024-09-21T14:58:41.342 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6690000000000003e-06 2024-09-21T14:58:41.342 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.342 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.342 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:41.342 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.234964+0000", 2024-09-21T14:58:41.343 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0075705319999999996 2024-09-21T14:58:41.343 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.343 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.343 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:41.343 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.235041+0000", 2024-09-21T14:58:41.343 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7170999999999997e-05 2024-09-21T14:58:41.343 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.343 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.343 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:41.343 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.235225+0000", 2024-09-21T14:58:41.343 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000183735 2024-09-21T14:58:41.343 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:41.343 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:41.343 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:41.343 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.343 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.344 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:8064 3.b 3:d1b12ef9:::rbd_data.11361bed1f78.0000000000000062:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:41.344 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:35.230805+0000", 2024-09-21T14:58:41.344 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 6.0492715090000004, 2024-09-21T14:58:41.344 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0046229139999999997, 2024-09-21T14:58:41.344 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:41.344 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:41.344 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:41.344 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:41.344 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:41.344 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 8064 2024-09-21T14:58:41.344 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.344 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:41.344 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.344 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:41.344 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.230805+0000", 2024-09-21T14:58:41.345 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:41.345 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.345 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.345 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:41.345 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.230805+0000", 2024-09-21T14:58:41.345 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:41.345 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.345 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.345 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:41.345 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.230803+0000", 2024-09-21T14:58:41.345 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:41.345 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.345 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.345 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:41.346 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.230811+0000", 2024-09-21T14:58:41.346 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1179999999999992e-06 2024-09-21T14:58:41.346 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.346 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.346 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:41.346 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.230843+0000", 2024-09-21T14:58:41.346 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1359999999999998e-05 2024-09-21T14:58:41.346 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.346 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.347 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:41.347 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.230853+0000", 2024-09-21T14:58:41.347 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0203e-05 2024-09-21T14:58:41.347 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.347 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.347 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:41.347 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.235257+0000", 2024-09-21T14:58:41.347 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0044038660000000002 2024-09-21T14:58:41.347 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.347 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.348 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:41.348 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.235344+0000", 2024-09-21T14:58:41.348 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7781999999999994e-05 2024-09-21T14:58:41.348 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.348 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.348 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:41.348 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.235428+0000", 2024-09-21T14:58:41.348 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3175000000000002e-05 2024-09-21T14:58:41.348 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:41.349 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:41.349 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:41.349 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.349 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.349 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:8110 3.b 3:d7ca7d68:::rbd_data.11363a21d2be.000000000000002d:head [write 0~393216 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:41.349 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:35.234817+0000", 2024-09-21T14:58:41.349 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 6.0452589379999999, 2024-09-21T14:58:41.349 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0045433920000000003, 2024-09-21T14:58:41.349 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:41.349 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:41.350 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:41.350 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:41.350 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:41.350 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 8110 2024-09-21T14:58:41.350 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.350 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:41.350 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.350 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:41.350 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.234817+0000", 2024-09-21T14:58:41.351 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:41.351 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.351 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.351 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:41.351 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.234817+0000", 2024-09-21T14:58:41.351 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:41.351 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.351 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.352 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:41.352 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.234816+0000", 2024-09-21T14:58:41.352 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:41.352 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.352 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.352 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:41.352 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.234968+0000", 2024-09-21T14:58:41.352 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015134 2024-09-21T14:58:41.352 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.352 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.353 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:41.353 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.234979+0000", 2024-09-21T14:58:41.353 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0997e-05 2024-09-21T14:58:41.353 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.353 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.353 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:41.353 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.234988+0000", 2024-09-21T14:58:41.353 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9930000000000001e-06 2024-09-21T14:58:41.353 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.354 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.354 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:41.354 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.236562+0000", 2024-09-21T14:58:41.354 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001574527 2024-09-21T14:58:41.354 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.354 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.354 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:41.354 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.236729+0000", 2024-09-21T14:58:41.354 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016653699999999999 2024-09-21T14:58:41.355 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.355 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.355 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T14:58:41.355 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.236815+0000", 2024-09-21T14:58:41.355 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6351e-05 2024-09-21T14:58:41.355 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.355 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.355 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:41.355 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.239244+0000", 2024-09-21T14:58:41.356 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0024289469999999999 2024-09-21T14:58:41.356 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.356 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.356 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:58:41.356 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.239256+0000", 2024-09-21T14:58:41.356 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2206e-05 2024-09-21T14:58:41.356 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.356 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.356 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:41.357 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.239308+0000", 2024-09-21T14:58:41.357 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1725999999999999e-05 2024-09-21T14:58:41.357 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.357 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.357 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:41.357 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.239361+0000", 2024-09-21T14:58:41.357 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2641e-05 2024-09-21T14:58:41.357 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:41.357 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:41.357 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:41.358 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.358 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.358 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:13062 3.b 3:d4de1489:::rbd_data.1136f5ed5dd6.0000000000000067:head [write 0~57344 [fadvise_sequential+fadvise_nocache],write 98304~122880 [fadvise_sequential+fadvise_nocache],write 237568~8192 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:41.358 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.683197+0000", 2024-09-21T14:58:41.358 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.596879626, 2024-09-21T14:58:41.358 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0041119429999999998, 2024-09-21T14:58:41.358 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:41.358 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:41.359 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:41.359 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:41.359 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:41.359 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 13062 2024-09-21T14:58:41.359 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.359 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:41.359 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.359 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:41.359 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.683197+0000", 2024-09-21T14:58:41.359 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:41.360 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.360 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.360 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:41.360 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.683197+0000", 2024-09-21T14:58:41.360 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:41.360 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.360 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.360 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:41.360 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.683196+0000", 2024-09-21T14:58:41.360 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:41.360 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.361 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.361 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:41.361 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.683366+0000", 2024-09-21T14:58:41.361 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000170612 2024-09-21T14:58:41.361 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.361 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.361 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:41.361 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.683381+0000", 2024-09-21T14:58:41.361 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4386000000000001e-05 2024-09-21T14:58:41.361 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.361 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.361 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:41.361 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.683389+0000", 2024-09-21T14:58:41.361 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6039999999999993e-06 2024-09-21T14:58:41.361 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.362 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.362 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:41.362 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.685094+0000", 2024-09-21T14:58:41.362 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001704891 2024-09-21T14:58:41.362 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.362 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.362 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:41.362 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.685226+0000", 2024-09-21T14:58:41.362 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000132242 2024-09-21T14:58:41.362 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.362 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.362 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T14:58:41.362 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.685430+0000", 2024-09-21T14:58:41.362 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00020393599999999999 2024-09-21T14:58:41.363 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.363 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.363 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:58:41.363 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.686567+0000", 2024-09-21T14:58:41.363 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0011368470000000001 2024-09-21T14:58:41.363 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.363 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.363 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:41.363 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.687250+0000", 2024-09-21T14:58:41.363 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00068261199999999998 2024-09-21T14:58:41.363 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.363 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.363 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:41.364 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.687287+0000", 2024-09-21T14:58:41.364 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7064999999999998e-05 2024-09-21T14:58:41.364 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.364 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.364 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:41.364 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.687309+0000", 2024-09-21T14:58:41.364 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1585000000000001e-05 2024-09-21T14:58:41.364 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:41.364 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:41.364 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:41.364 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.364 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.364 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:15118 3.b 3:d2c9f41f:::rbd_data.11361bed1f78.000000000000005b:head [sparse-read 0~114688] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-21T14:58:41.364 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:40.290224+0000", 2024-09-21T14:58:41.364 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.98985270800000003, 2024-09-21T14:58:41.364 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0042569189999999996, 2024-09-21T14:58:41.365 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:41.365 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:41.365 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:41.365 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:41.365 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:41.365 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 15118 2024-09-21T14:58:41.365 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.365 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:41.365 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.365 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:41.365 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:40.290224+0000", 2024-09-21T14:58:41.365 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:41.365 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.365 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.365 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:41.365 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:40.290224+0000", 2024-09-21T14:58:41.366 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:41.366 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.366 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.366 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:41.366 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:40.290223+0000", 2024-09-21T14:58:41.366 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:41.366 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.366 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.366 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:41.366 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:40.290229+0000", 2024-09-21T14:58:41.366 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1999999999999999e-06 2024-09-21T14:58:41.366 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.366 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.366 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:41.367 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:40.290244+0000", 2024-09-21T14:58:41.367 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5438999999999999e-05 2024-09-21T14:58:41.367 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.367 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.367 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:41.367 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:40.290257+0000", 2024-09-21T14:58:41.367 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2746e-05 2024-09-21T14:58:41.367 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.367 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.367 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:41.367 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:40.293634+0000", 2024-09-21T14:58:41.367 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0033766769999999998 2024-09-21T14:58:41.367 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.367 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.367 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:41.368 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:40.293825+0000", 2024-09-21T14:58:41.368 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000191363 2024-09-21T14:58:41.368 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.368 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.368 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:41.368 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:40.294480+0000", 2024-09-21T14:58:41.368 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000655428 2024-09-21T14:58:41.368 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:41.368 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:41.368 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:41.368 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.368 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.368 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:15125 3.b 3:d1b12ef9:::rbd_data.11361bed1f78.0000000000000062:head [sparse-read 0~163840] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-21T14:58:41.368 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:40.290268+0000", 2024-09-21T14:58:41.368 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.98980838900000001, 2024-09-21T14:58:41.368 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.005421192, 2024-09-21T14:58:41.369 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:41.369 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:41.369 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:41.369 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:41.369 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:41.369 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 15125 2024-09-21T14:58:41.369 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.369 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:41.369 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.369 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:41.369 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:40.290268+0000", 2024-09-21T14:58:41.369 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:41.369 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.370 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.370 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:41.370 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:40.290268+0000", 2024-09-21T14:58:41.370 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:41.370 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.370 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.370 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:41.370 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:40.290267+0000", 2024-09-21T14:58:41.370 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:41.370 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.370 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.370 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:41.370 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:40.290272+0000", 2024-09-21T14:58:41.370 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4090000000000002e-06 2024-09-21T14:58:41.370 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.371 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.371 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:41.371 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:40.290286+0000", 2024-09-21T14:58:41.371 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3939e-05 2024-09-21T14:58:41.371 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.371 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.371 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:41.371 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:40.290299+0000", 2024-09-21T14:58:41.371 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2473999999999999e-05 2024-09-21T14:58:41.371 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.371 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.371 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:41.371 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:40.294552+0000", 2024-09-21T14:58:41.371 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0042534529999999999 2024-09-21T14:58:41.371 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.371 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.371 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:41.372 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:40.294739+0000", 2024-09-21T14:58:41.372 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018671200000000001 2024-09-21T14:58:41.372 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.372 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.372 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:41.372 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:40.295689+0000", 2024-09-21T14:58:41.372 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00095006799999999998 2024-09-21T14:58:41.372 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:41.372 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:41.372 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:41.372 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.372 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.372 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:15174 3.b 3:d11a5903:::rbd_data.11361bed1f78.0000000000000093:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-21T14:58:41.372 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:40.290309+0000", 2024-09-21T14:58:41.372 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.98976680399999994, 2024-09-21T14:58:41.373 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0060508960000000001, 2024-09-21T14:58:41.373 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:41.373 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:58:41.373 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:41.373 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:41.373 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:41.373 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 15174 2024-09-21T14:58:41.373 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.373 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:41.373 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.373 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:41.373 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:40.290309+0000", 2024-09-21T14:58:41.373 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:41.373 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.373 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.373 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:41.373 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:40.290309+0000", 2024-09-21T14:58:41.374 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:41.374 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.374 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.374 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:41.374 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:40.290309+0000", 2024-09-21T14:58:41.374 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:41.374 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.374 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.374 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:41.374 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:40.290314+0000", 2024-09-21T14:58:41.374 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5219999999999998e-06 2024-09-21T14:58:41.374 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.374 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.374 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:41.374 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:40.290330+0000", 2024-09-21T14:58:41.375 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5511000000000001e-05 2024-09-21T14:58:41.375 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.375 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.375 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:41.375 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:40.290341+0000", 2024-09-21T14:58:41.375 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1729000000000001e-05 2024-09-21T14:58:41.375 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.375 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.375 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:41.375 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:40.296093+0000", 2024-09-21T14:58:41.375 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.005751847 2024-09-21T14:58:41.375 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:41.375 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:41.375 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:41.376 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:40.296360+0000", 2024-09-21T14:58:41.376 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00026712099999999999 2024-09-21T14:58:41.376 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:41.376 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:41.376 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:41.376 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:41.376 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:41.376 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:58:41.376 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T14:58:41.377 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:58:41.376+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:58:41.378 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T14:58:41.476 INFO:teuthology.orchestra.run.smithi032.stdout:518 read 0x2237cc5 thru 0x2239845 (0x1b81 bytes) 2024-09-21T14:58:41.477 INFO:teuthology.orchestra.run.smithi032.stdout:520 write 0x77b3807 thru 0x77c01b6 (0xc9b0 bytes) 2024-09-21T14:58:41.478 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:58:41.476+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:58:41.511 INFO:teuthology.orchestra.run.smithi032.stdout:522 write 0xbf50410 thru 0xbf57468 (0x7059 bytes) 2024-09-21T14:58:41.516 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:58:41.516 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T14:58:41.517 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:41.517 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:15225 3.13 e46/24 3:cd55680c:::rbd_object_map.1136f5ed5dd6:head v 46'62, mlcod=46'57)", 2024-09-21T14:58:41.517 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:41.515534+0000", 2024-09-21T14:58:41.517 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.00056120900000000001, 2024-09-21T14:58:41.517 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00059012400000000003, 2024-09-21T14:58:41.517 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:41.517 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:41.517 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:41.517 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:41.517 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:41.517 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.515534+0000", 2024-09-21T14:58:41.517 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:41.517 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:41.517 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:41.518 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:41.518 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.515534+0000", 2024-09-21T14:58:41.518 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:41.518 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:41.518 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:41.518 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:41.518 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.515533+0000", 2024-09-21T14:58:41.518 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:41.518 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:41.518 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:41.518 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:41.518 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.515541+0000", 2024-09-21T14:58:41.518 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5830000000000001e-06 2024-09-21T14:58:41.519 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:41.519 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:41.519 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:41.519 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.515552+0000", 2024-09-21T14:58:41.519 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0814e-05 2024-09-21T14:58:41.519 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:41.519 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:41.519 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:41.519 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.515557+0000", 2024-09-21T14:58:41.519 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0889999999999999e-06 2024-09-21T14:58:41.519 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:41.519 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:41.519 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:41.520 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.515597+0000", 2024-09-21T14:58:41.520 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.0547999999999999e-05 2024-09-21T14:58:41.520 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:41.520 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:41.520 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:41.520 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.515611+0000", 2024-09-21T14:58:41.520 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3518e-05 2024-09-21T14:58:41.520 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:41.520 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:41.520 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:41.520 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:41.520 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-21T14:58:41.520 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 1 2024-09-21T14:58:41.520 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:58:41.523 INFO:teuthology.orchestra.run.smithi032.stdout:526 write 0xbe88237 thru 0xbe978be (0xf688 bytes) 2024-09-21T14:58:41.531 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T14:58:41.539 INFO:teuthology.orchestra.run.smithi032.stdout:527 write 0x7597e43 thru 0x75a6ba5 (0xed63 bytes) 2024-09-21T14:58:41.552 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T14:58:41.553 INFO:teuthology.orchestra.run.smithi032.stdout:528 trunc from 0xbf57469 to 0x9b0e4e4 2024-09-21T14:58:41.573 INFO:teuthology.orchestra.run.smithi032.stdout:529 read 0x246d687 thru 0x24738ad (0x6227 bytes) 2024-09-21T14:58:41.573 INFO:teuthology.orchestra.run.smithi032.stdout:530 write 0xeda636 thru 0xee3257 (0x8c22 bytes) 2024-09-21T14:58:41.576 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T14:58:41.577 INFO:teuthology.orchestra.run.smithi032.stdout:532 read 0x3a41a85 thru 0x3a4af1e (0x949a bytes) 2024-09-21T14:58:41.577 INFO:teuthology.orchestra.run.smithi032.stdout:533 punch from 0x76f79c4 to 0x7702232, (0xa86e bytes) 2024-09-21T14:58:41.578 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:58:41.576+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:58:41.599 INFO:teuthology.orchestra.run.smithi032.stdout:536 writesame 0xe890533 thru 0xe8984be data_size 0x9(0x7f8c bytes) 2024-09-21T14:58:41.600 INFO:teuthology.orchestra.run.smithi032.stdout:537 write 0x4e03713 thru 0x4e09f77 (0x6865 bytes) 2024-09-21T14:58:41.602 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T14:58:41.603 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:41.603 INFO:teuthology.orchestra.run.smithi032.stdout:540 punch from 0xda559cd to 0xda57d5c, (0x238f bytes) 2024-09-21T14:58:41.624 INFO:teuthology.orchestra.run.smithi032.stdout:542 trunc from 0xe8984bf to 0x17bdd84 2024-09-21T14:58:41.636 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T14:58:41.658 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:41.658 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:41.669 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-21T14:58:41.671 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T14:58:41.672 INFO:teuthology.orchestra.run.smithi032.stdout:543 writesame 0x1504400 thru 0x151119e data_size 0x1(0xcd9f bytes) 2024-09-21T14:58:41.678 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:58:41.676+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:58:41.691 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T14:58:41.696 INFO:teuthology.orchestra.run.smithi032.stdout:544 write 0x422e3b6 thru 0x4238e26 (0xaa71 bytes) 2024-09-21T14:58:41.709 INFO:teuthology.orchestra.run.smithi032.stdout:545 read 0x27ae61d thru 0x27b2dd7 (0x47bb bytes) 2024-09-21T14:58:41.709 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:41.709 INFO:teuthology.orchestra.run.smithi032.stdout:547 read 0x3c3e920 thru 0x3c40042 (0x1723 bytes) 2024-09-21T14:58:41.709 INFO:teuthology.orchestra.run.smithi032.stdout:548 read 0x1ef7095 thru 0x1efd25d (0x61c9 bytes) 2024-09-21T14:58:41.709 INFO:teuthology.orchestra.run.smithi032.stdout:549 read 0x8fbbf thru 0x93e55 (0x4297 bytes) 2024-09-21T14:58:41.709 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:41.710 INFO:teuthology.orchestra.run.smithi032.stdout:551 clone 6 order 18 su 32768 sc 2 2024-09-21T14:58:41.717 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T14:58:41.742 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T14:58:41.744 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:58:41.744 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:58:41.744 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:58:41.744 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:58:41.744 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:58:41.744 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:58:41.744 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:58:41.744 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:58:41.779 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:41.779 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:41.779 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:58:41.776+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:58:41.788 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T14:58:41.880 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:58:41.876+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:58:41.898 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:41.899 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:41.908 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T14:58:41.981 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:58:41.980+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:58:42.018 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:42.018 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:42.028 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T14:58:42.082 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:58:42.080+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:58:42.139 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:42.139 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:42.150 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T14:58:42.194 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:58:42.192+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:58:42.250 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:42.251 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:42.260 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T14:58:42.284 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:58:42.280+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:58:42.368 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:42.368 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:42.378 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T14:58:42.384 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:58:42.380+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:58:42.485 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:58:42.484+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:58:42.486 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:42.487 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:42.497 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T14:58:42.586 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:58:42.584+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:58:42.649 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:42.650 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:42.659 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T14:58:42.687 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:58:42.684+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:58:42.763 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:42.763 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:42.774 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T14:58:42.787 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:58:42.784+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:58:42.880 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:42.880 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:42.889 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:58:42.888+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:58:42.890 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T14:58:42.990 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:58:42.988+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:58:43.009 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:43.009 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:43.019 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T14:58:43.090 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:58:43.088+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:58:43.123 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:43.123 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:43.126 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T14:58:43.133 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T14:58:43.191 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:58:43.188+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:58:43.266 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:43.266 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:43.281 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T14:58:43.293 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:58:43.288+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:58:43.393 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:58:43.388+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:58:43.452 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:43.452 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:43.462 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T14:58:43.493 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:58:43.492+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:58:43.566 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:43.566 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:43.575 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T14:58:43.621 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:58:43.620+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:58:43.684 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:43.684 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:43.694 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T14:58:43.696 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:58:43.692+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:58:43.755 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T14:58:43.796 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:58:43.792+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:58:43.804 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:43.804 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:43.814 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T14:58:43.898 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:58:43.896+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:58:43.912 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T14:58:43.923 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:43.923 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:43.933 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T14:58:43.934 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T14:58:43.998 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:58:43.996+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:58:44.089 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:44.089 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:44.099 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T14:58:44.100 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:58:44.096+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:58:44.160 INFO:teuthology.orchestra.run.smithi032.stderr:nodeep-scrub is set 2024-09-21T14:58:44.167 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T14:58:44.186 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T14:58:44.200 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:58:44.196+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:58:44.206 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-21T14:58:44.217 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:44.217 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:44.227 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T14:58:44.232 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T14:58:44.252 INFO:teuthology.orchestra.run.smithi032.stdout:checking clone #4, image image_client.0-clone4 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent5 2024-09-21T14:58:44.254 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T14:58:44.276 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T14:58:44.279 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:58:44.279 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:58:44.279 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:58:44.279 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:58:44.279 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:58:44.279 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:58:44.279 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:58:44.279 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:58:44.300 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:58:44.296+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:58:44.331 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:58:44.331 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:58:44.331 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T14:58:44.331 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T14:58:44.331 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:58:44.342 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T14:58:44.401 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:58:44.400+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:58:44.499 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:58:44.499 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T14:58:44.499 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T14:58:44.499 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T14:58:44.499 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.499 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:7990 3.b 3:d2c9f41f:::rbd_data.11361bed1f78.000000000000005b:d [read 32768~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:44.500 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:35.227143+0000", 2024-09-21T14:58:44.500 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.269964109, 2024-09-21T14:58:44.500 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0045399860000000002, 2024-09-21T14:58:44.500 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:44.500 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:44.500 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:44.500 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:44.500 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:44.500 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 7990 2024-09-21T14:58:44.500 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.500 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:44.500 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.500 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:44.500 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227143+0000", 2024-09-21T14:58:44.501 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:44.501 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.501 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.501 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:44.501 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227143+0000", 2024-09-21T14:58:44.501 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:44.501 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.501 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.501 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:44.501 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227143+0000", 2024-09-21T14:58:44.501 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:44.501 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.501 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.501 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:44.501 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227145+0000", 2024-09-21T14:58:44.502 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7130000000000001e-06 2024-09-21T14:58:44.502 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.502 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.502 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:44.502 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227157+0000", 2024-09-21T14:58:44.502 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1501999999999999e-05 2024-09-21T14:58:44.502 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.502 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.502 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:44.502 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227163+0000", 2024-09-21T14:58:44.502 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5450000000000003e-06 2024-09-21T14:58:44.502 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.502 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.503 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:44.503 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.231368+0000", 2024-09-21T14:58:44.503 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0042056430000000002 2024-09-21T14:58:44.503 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.503 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.503 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:44.503 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.231464+0000", 2024-09-21T14:58:44.503 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6043000000000002e-05 2024-09-21T14:58:44.503 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.503 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.503 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:44.503 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.231683+0000", 2024-09-21T14:58:44.503 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000219008 2024-09-21T14:58:44.503 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:44.503 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:44.504 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:44.504 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.504 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.504 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:7991 3.b 3:d2c9f41f:::rbd_data.11361bed1f78.000000000000005b:d [read 98304~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:44.504 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:35.227168+0000", 2024-09-21T14:58:44.504 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.2699394389999998, 2024-09-21T14:58:44.504 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0048739430000000004, 2024-09-21T14:58:44.504 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:44.504 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:44.504 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:44.504 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:44.504 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:44.504 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 7991 2024-09-21T14:58:44.504 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.504 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:44.505 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.505 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:44.505 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227168+0000", 2024-09-21T14:58:44.505 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:44.505 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.505 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.505 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:44.505 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227168+0000", 2024-09-21T14:58:44.505 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:44.505 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.505 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.505 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:44.505 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227167+0000", 2024-09-21T14:58:44.505 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:44.505 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.506 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.506 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:44.506 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227172+0000", 2024-09-21T14:58:44.506 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.561e-06 2024-09-21T14:58:44.506 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.506 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.506 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:44.506 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227179+0000", 2024-09-21T14:58:44.506 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1119999999999998e-06 2024-09-21T14:58:44.506 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.506 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.506 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:44.506 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227186+0000", 2024-09-21T14:58:44.506 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.046e-06 2024-09-21T14:58:44.507 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.507 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.507 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:44.507 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.231717+0000", 2024-09-21T14:58:44.507 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0045312190000000004 2024-09-21T14:58:44.507 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.507 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.507 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:44.507 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.231823+0000", 2024-09-21T14:58:44.507 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000105303 2024-09-21T14:58:44.507 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.507 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.507 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:44.508 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.232042+0000", 2024-09-21T14:58:44.508 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021928599999999999 2024-09-21T14:58:44.508 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:44.508 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:44.508 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:44.508 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.508 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.508 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:8004 3.b 3:d2c9f41f:::rbd_data.11361bed1f78.000000000000005b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:44.508 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:35.227192+0000", 2024-09-21T14:58:44.508 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.2699156449999993, 2024-09-21T14:58:44.508 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0052132289999999998, 2024-09-21T14:58:44.508 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:44.508 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:44.508 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:44.508 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:44.509 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:44.509 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 8004 2024-09-21T14:58:44.509 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.509 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:44.509 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.509 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:44.509 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227192+0000", 2024-09-21T14:58:44.509 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:44.509 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.509 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.509 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:44.509 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227192+0000", 2024-09-21T14:58:44.509 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:44.509 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.509 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.510 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:44.510 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227191+0000", 2024-09-21T14:58:44.510 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:44.510 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.510 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.510 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:44.510 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227194+0000", 2024-09-21T14:58:44.510 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8990000000000001e-06 2024-09-21T14:58:44.510 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.510 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.510 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:44.510 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227201+0000", 2024-09-21T14:58:44.510 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5810000000000002e-06 2024-09-21T14:58:44.510 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.510 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.511 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:44.511 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227206+0000", 2024-09-21T14:58:44.511 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.694e-06 2024-09-21T14:58:44.511 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.511 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.511 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:44.511 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.232148+0000", 2024-09-21T14:58:44.511 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00494174 2024-09-21T14:58:44.511 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.511 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.511 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:44.511 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.232308+0000", 2024-09-21T14:58:44.511 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015929599999999999 2024-09-21T14:58:44.511 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.511 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.512 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:44.512 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.232405+0000", 2024-09-21T14:58:44.512 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7415000000000002e-05 2024-09-21T14:58:44.512 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:44.512 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:44.512 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:44.512 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.512 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.512 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:8011 3.b 3:d1b12ef9:::rbd_data.11361bed1f78.0000000000000062:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:44.512 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:35.227212+0000", 2024-09-21T14:58:44.512 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.2698957849999992, 2024-09-21T14:58:44.512 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0054249789999999999, 2024-09-21T14:58:44.512 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:44.512 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:44.513 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:44.513 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:44.513 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:44.513 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 8011 2024-09-21T14:58:44.513 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.513 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:44.513 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.513 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:44.513 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227212+0000", 2024-09-21T14:58:44.513 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:44.513 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.513 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.513 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:44.513 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227212+0000", 2024-09-21T14:58:44.514 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:44.514 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.514 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.514 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:44.514 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227211+0000", 2024-09-21T14:58:44.514 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:44.514 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.514 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.514 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:44.514 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227214+0000", 2024-09-21T14:58:44.514 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7999999999999999e-06 2024-09-21T14:58:44.514 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.514 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.514 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:44.515 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227220+0000", 2024-09-21T14:58:44.515 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0569999999999998e-06 2024-09-21T14:58:44.515 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.515 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.515 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:44.515 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227223+0000", 2024-09-21T14:58:44.515 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3000000000000002e-06 2024-09-21T14:58:44.515 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.515 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.515 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:44.515 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.232445+0000", 2024-09-21T14:58:44.515 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0052220160000000003 2024-09-21T14:58:44.515 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.515 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.515 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:44.516 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.232535+0000", 2024-09-21T14:58:44.516 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9866999999999999e-05 2024-09-21T14:58:44.516 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.516 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.516 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:44.516 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.232637+0000", 2024-09-21T14:58:44.516 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010155 2024-09-21T14:58:44.516 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:44.516 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:44.516 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:44.516 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.516 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.516 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:8019 3.b 3:d1b12ef9:::rbd_data.11361bed1f78.0000000000000062:d [read 0~32768 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:44.516 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:35.227226+0000", 2024-09-21T14:58:44.517 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.2698811840000008, 2024-09-21T14:58:44.517 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.005662696, 2024-09-21T14:58:44.517 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:44.517 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:44.517 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:44.517 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:44.517 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:44.517 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 8019 2024-09-21T14:58:44.517 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.517 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:44.517 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.517 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:44.517 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227226+0000", 2024-09-21T14:58:44.517 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:44.518 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.518 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.518 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:44.518 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227226+0000", 2024-09-21T14:58:44.518 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:44.518 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.518 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.518 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:44.518 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227226+0000", 2024-09-21T14:58:44.518 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:44.518 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.518 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.518 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:44.518 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227228+0000", 2024-09-21T14:58:44.519 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5919999999999999e-06 2024-09-21T14:58:44.519 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.519 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.519 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:44.519 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227235+0000", 2024-09-21T14:58:44.519 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2650000000000002e-06 2024-09-21T14:58:44.519 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.519 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.519 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:44.519 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227241+0000", 2024-09-21T14:58:44.519 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9139999999999996e-06 2024-09-21T14:58:44.519 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.519 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.519 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:44.520 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.232671+0000", 2024-09-21T14:58:44.520 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0054301879999999999 2024-09-21T14:58:44.520 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.520 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.520 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:44.520 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.232747+0000", 2024-09-21T14:58:44.520 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5988999999999994e-05 2024-09-21T14:58:44.520 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.520 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.520 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:44.520 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.232889+0000", 2024-09-21T14:58:44.520 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014205199999999999 2024-09-21T14:58:44.520 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:44.520 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:44.520 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:44.521 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.521 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.521 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:8020 3.b 3:d1b12ef9:::rbd_data.11361bed1f78.0000000000000062:d [read 81920~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:44.521 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:35.227246+0000", 2024-09-21T14:58:44.521 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.2698615909999997, 2024-09-21T14:58:44.521 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0061395770000000002, 2024-09-21T14:58:44.521 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:44.521 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:44.521 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:44.521 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:44.521 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:44.521 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 8020 2024-09-21T14:58:44.521 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.521 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:44.522 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.522 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:44.522 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227246+0000", 2024-09-21T14:58:44.522 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:44.522 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.522 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.522 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:44.522 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227246+0000", 2024-09-21T14:58:44.522 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:44.522 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.522 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.522 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:44.522 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227245+0000", 2024-09-21T14:58:44.523 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:44.523 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.523 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.523 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:44.523 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227248+0000", 2024-09-21T14:58:44.523 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8119999999999999e-06 2024-09-21T14:58:44.523 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.523 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.523 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:44.523 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227255+0000", 2024-09-21T14:58:44.523 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7549999999999997e-06 2024-09-21T14:58:44.523 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.523 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.523 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:44.523 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227260+0000", 2024-09-21T14:58:44.524 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.271e-06 2024-09-21T14:58:44.524 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.524 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.524 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:44.524 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.232967+0000", 2024-09-21T14:58:44.524 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0057065179999999998 2024-09-21T14:58:44.524 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.524 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.524 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:44.524 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.233108+0000", 2024-09-21T14:58:44.524 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000141123 2024-09-21T14:58:44.524 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.524 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.524 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:44.524 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.233385+0000", 2024-09-21T14:58:44.524 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00027743400000000001 2024-09-21T14:58:44.525 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:44.525 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:44.525 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:44.525 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.525 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.525 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:8034 3.b 3:d1b12ef9:::rbd_data.11361bed1f78.0000000000000062:d [read 32768~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:44.525 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:35.227265+0000", 2024-09-21T14:58:44.525 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.2698422239999996, 2024-09-21T14:58:44.525 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.006525968, 2024-09-21T14:58:44.525 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:44.525 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:44.525 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:44.525 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:44.525 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:44.525 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 8034 2024-09-21T14:58:44.526 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.526 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:44.526 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.526 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:44.526 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227265+0000", 2024-09-21T14:58:44.526 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:44.526 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.526 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.526 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:44.526 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227265+0000", 2024-09-21T14:58:44.526 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:44.526 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.526 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.526 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:44.527 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227265+0000", 2024-09-21T14:58:44.527 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:44.527 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.527 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.527 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:44.527 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227268+0000", 2024-09-21T14:58:44.527 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8990000000000001e-06 2024-09-21T14:58:44.527 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.527 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.527 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:44.527 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227274+0000", 2024-09-21T14:58:44.527 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4470000000000003e-06 2024-09-21T14:58:44.527 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.527 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.527 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:44.528 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227288+0000", 2024-09-21T14:58:44.528 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3550000000000001e-05 2024-09-21T14:58:44.528 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.528 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.528 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:44.528 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.233418+0000", 2024-09-21T14:58:44.528 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0061307870000000004 2024-09-21T14:58:44.528 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.528 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.528 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:44.528 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.233553+0000", 2024-09-21T14:58:44.528 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013475600000000001 2024-09-21T14:58:44.528 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.528 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.528 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:44.529 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.233791+0000", 2024-09-21T14:58:44.529 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00023788499999999999 2024-09-21T14:58:44.529 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:44.529 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:44.529 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:44.529 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.529 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.529 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:8035 3.b 3:d1b12ef9:::rbd_data.11361bed1f78.0000000000000062:d [read 98304~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:44.529 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:35.227295+0000", 2024-09-21T14:58:44.529 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.2698120189999997, 2024-09-21T14:58:44.529 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0069673820000000003, 2024-09-21T14:58:44.529 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:44.529 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:44.529 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:44.530 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:44.530 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:44.530 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 8035 2024-09-21T14:58:44.530 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.530 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:44.530 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.530 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:44.530 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227295+0000", 2024-09-21T14:58:44.530 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:44.530 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.530 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.530 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:44.530 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227295+0000", 2024-09-21T14:58:44.530 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:44.531 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.531 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.531 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:44.531 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227295+0000", 2024-09-21T14:58:44.531 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:44.531 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.531 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.531 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:44.531 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227298+0000", 2024-09-21T14:58:44.531 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2339999999999999e-06 2024-09-21T14:58:44.531 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.531 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.531 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:44.531 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227305+0000", 2024-09-21T14:58:44.531 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0389999999999999e-06 2024-09-21T14:58:44.532 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.532 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.532 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:44.532 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227311+0000", 2024-09-21T14:58:44.532 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8470000000000001e-06 2024-09-21T14:58:44.532 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.532 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.532 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:44.532 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.233824+0000", 2024-09-21T14:58:44.532 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0065132920000000004 2024-09-21T14:58:44.532 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.532 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.532 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:44.532 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.233922+0000", 2024-09-21T14:58:44.532 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.8073999999999996e-05 2024-09-21T14:58:44.533 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.533 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.533 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:44.533 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.234263+0000", 2024-09-21T14:58:44.533 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00034027999999999999 2024-09-21T14:58:44.533 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:44.533 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:44.533 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:44.533 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.533 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.533 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:8048 3.b 3:d1b12ef9:::rbd_data.11361bed1f78.0000000000000062:d [read 49152~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:44.533 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:35.227316+0000", 2024-09-21T14:58:44.533 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.2697912270000007, 2024-09-21T14:58:44.533 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0076197799999999996, 2024-09-21T14:58:44.534 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:44.534 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:44.534 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:44.534 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:44.534 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:44.534 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 8048 2024-09-21T14:58:44.534 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.534 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:44.534 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.534 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:44.534 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227316+0000", 2024-09-21T14:58:44.534 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:44.534 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.534 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.535 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:44.535 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227316+0000", 2024-09-21T14:58:44.535 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:44.535 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.535 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.535 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:44.535 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227316+0000", 2024-09-21T14:58:44.535 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:44.535 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.535 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.535 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:44.535 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227324+0000", 2024-09-21T14:58:44.535 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1009999999999992e-06 2024-09-21T14:58:44.536 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.536 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.536 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:44.536 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227346+0000", 2024-09-21T14:58:44.536 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2347000000000001e-05 2024-09-21T14:58:44.536 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.536 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.536 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:44.536 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227360+0000", 2024-09-21T14:58:44.536 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.403e-05 2024-09-21T14:58:44.536 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.536 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.536 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:44.536 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.234320+0000", 2024-09-21T14:58:44.536 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0069597019999999999 2024-09-21T14:58:44.537 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.537 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.537 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:44.537 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.234730+0000", 2024-09-21T14:58:44.537 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00041006300000000001 2024-09-21T14:58:44.537 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.537 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.537 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:44.537 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.234936+0000", 2024-09-21T14:58:44.537 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00020586700000000001 2024-09-21T14:58:44.537 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:44.537 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:44.537 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:44.537 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.538 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.538 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:8049 3.b 3:d1b12ef9:::rbd_data.11361bed1f78.0000000000000062:d [read 114688~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:44.538 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:35.227371+0000", 2024-09-21T14:58:44.538 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.2697358940000001, 2024-09-21T14:58:44.538 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0078533609999999997, 2024-09-21T14:58:44.538 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:44.538 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:44.538 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:44.538 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:44.538 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:44.538 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 8049 2024-09-21T14:58:44.538 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.538 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:44.538 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.539 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:44.539 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227371+0000", 2024-09-21T14:58:44.539 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:44.539 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.539 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.539 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:44.539 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227371+0000", 2024-09-21T14:58:44.539 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:44.539 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.539 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.539 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:44.539 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227370+0000", 2024-09-21T14:58:44.539 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:44.539 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.540 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.540 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:44.540 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227378+0000", 2024-09-21T14:58:44.540 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.379e-06 2024-09-21T14:58:44.540 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.540 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.540 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:44.540 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227388+0000", 2024-09-21T14:58:44.540 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0175e-05 2024-09-21T14:58:44.540 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.540 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.540 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:44.540 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227393+0000", 2024-09-21T14:58:44.540 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6690000000000003e-06 2024-09-21T14:58:44.541 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.541 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.541 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:44.541 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.234964+0000", 2024-09-21T14:58:44.541 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0075705319999999996 2024-09-21T14:58:44.541 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.541 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.541 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:44.541 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.235041+0000", 2024-09-21T14:58:44.541 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7170999999999997e-05 2024-09-21T14:58:44.541 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.541 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.541 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:44.541 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.235225+0000", 2024-09-21T14:58:44.542 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000183735 2024-09-21T14:58:44.542 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:44.542 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:44.542 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:44.542 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.542 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.542 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:8064 3.b 3:d1b12ef9:::rbd_data.11361bed1f78.0000000000000062:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:44.542 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:35.230805+0000", 2024-09-21T14:58:44.542 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.2663026439999996, 2024-09-21T14:58:44.542 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0046229139999999997, 2024-09-21T14:58:44.542 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:44.542 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:44.542 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:44.543 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:44.543 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:44.543 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 8064 2024-09-21T14:58:44.543 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.543 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:44.543 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.543 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:44.543 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.230805+0000", 2024-09-21T14:58:44.543 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:44.543 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.543 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.543 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:44.543 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.230805+0000", 2024-09-21T14:58:44.543 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:44.544 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.544 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.544 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:44.544 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.230803+0000", 2024-09-21T14:58:44.544 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:44.544 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.544 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.544 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:44.544 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.230811+0000", 2024-09-21T14:58:44.544 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1179999999999992e-06 2024-09-21T14:58:44.544 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.544 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.544 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:44.544 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.230843+0000", 2024-09-21T14:58:44.545 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1359999999999998e-05 2024-09-21T14:58:44.545 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.545 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.545 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:44.545 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.230853+0000", 2024-09-21T14:58:44.545 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0203e-05 2024-09-21T14:58:44.545 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.545 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.545 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:44.545 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.235257+0000", 2024-09-21T14:58:44.545 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0044038660000000002 2024-09-21T14:58:44.545 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.545 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.546 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:44.546 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.235344+0000", 2024-09-21T14:58:44.546 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7781999999999994e-05 2024-09-21T14:58:44.546 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.546 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.546 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:44.546 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.235428+0000", 2024-09-21T14:58:44.546 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3175000000000002e-05 2024-09-21T14:58:44.546 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:44.546 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:44.546 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:44.546 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.546 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.546 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:8110 3.b 3:d7ca7d68:::rbd_data.11363a21d2be.000000000000002d:head [write 0~393216 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:44.546 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:35.234817+0000", 2024-09-21T14:58:44.547 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.2622900730000008, 2024-09-21T14:58:44.547 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0045433920000000003, 2024-09-21T14:58:44.547 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:44.547 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:44.547 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:44.547 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:44.547 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:44.547 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 8110 2024-09-21T14:58:44.547 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.547 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:44.547 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.547 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:44.547 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.234817+0000", 2024-09-21T14:58:44.548 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:44.548 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.548 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.548 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:44.548 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.234817+0000", 2024-09-21T14:58:44.548 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:44.548 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.548 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.548 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:44.548 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.234816+0000", 2024-09-21T14:58:44.548 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:44.548 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.548 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.548 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:44.549 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.234968+0000", 2024-09-21T14:58:44.549 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015134 2024-09-21T14:58:44.549 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.549 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.549 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:44.549 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.234979+0000", 2024-09-21T14:58:44.549 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0997e-05 2024-09-21T14:58:44.549 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.549 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.549 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:44.549 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.234988+0000", 2024-09-21T14:58:44.549 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9930000000000001e-06 2024-09-21T14:58:44.549 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.549 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.549 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:44.550 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.236562+0000", 2024-09-21T14:58:44.550 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001574527 2024-09-21T14:58:44.550 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.550 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.550 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:44.550 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.236729+0000", 2024-09-21T14:58:44.550 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016653699999999999 2024-09-21T14:58:44.550 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.550 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.550 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T14:58:44.550 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.236815+0000", 2024-09-21T14:58:44.551 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6351e-05 2024-09-21T14:58:44.551 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.551 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.551 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:44.551 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.239244+0000", 2024-09-21T14:58:44.551 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0024289469999999999 2024-09-21T14:58:44.551 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.551 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.551 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:58:44.551 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.239256+0000", 2024-09-21T14:58:44.551 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2206e-05 2024-09-21T14:58:44.551 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.551 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.552 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:44.552 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.239308+0000", 2024-09-21T14:58:44.552 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1725999999999999e-05 2024-09-21T14:58:44.552 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.552 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.552 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:44.552 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.239361+0000", 2024-09-21T14:58:44.552 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2641e-05 2024-09-21T14:58:44.552 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:44.552 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:44.552 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:44.552 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.552 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.552 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:15125 3.b 3:d1b12ef9:::rbd_data.11361bed1f78.0000000000000062:head [sparse-read 0~163840] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-21T14:58:44.553 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:40.290268+0000", 2024-09-21T14:58:44.553 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.2068395240000003, 2024-09-21T14:58:44.553 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.005421192, 2024-09-21T14:58:44.553 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:44.553 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:44.553 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:44.553 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:44.553 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:44.553 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 15125 2024-09-21T14:58:44.553 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.553 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:44.553 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.553 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:44.553 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:40.290268+0000", 2024-09-21T14:58:44.554 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:44.554 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.554 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.554 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:44.554 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:40.290268+0000", 2024-09-21T14:58:44.554 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:44.554 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.554 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.554 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:44.554 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:40.290267+0000", 2024-09-21T14:58:44.554 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:44.554 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.554 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.555 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:44.555 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:40.290272+0000", 2024-09-21T14:58:44.555 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4090000000000002e-06 2024-09-21T14:58:44.555 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.555 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.555 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:44.555 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:40.290286+0000", 2024-09-21T14:58:44.555 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3939e-05 2024-09-21T14:58:44.555 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.555 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.555 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:44.555 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:40.290299+0000", 2024-09-21T14:58:44.555 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2473999999999999e-05 2024-09-21T14:58:44.555 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.556 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.556 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:44.556 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:40.294552+0000", 2024-09-21T14:58:44.556 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0042534529999999999 2024-09-21T14:58:44.556 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.556 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.556 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:44.556 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:40.294739+0000", 2024-09-21T14:58:44.556 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018671200000000001 2024-09-21T14:58:44.556 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.556 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.556 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:44.556 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:40.295689+0000", 2024-09-21T14:58:44.556 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00095006799999999998 2024-09-21T14:58:44.557 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:44.557 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:44.557 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:44.557 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.557 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.557 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:15174 3.b 3:d11a5903:::rbd_data.11361bed1f78.0000000000000093:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-21T14:58:44.557 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:40.290309+0000", 2024-09-21T14:58:44.557 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.2067979390000003, 2024-09-21T14:58:44.557 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0060508960000000001, 2024-09-21T14:58:44.557 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:44.557 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:58:44.557 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:44.557 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:44.557 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:44.557 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 15174 2024-09-21T14:58:44.557 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.558 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:44.558 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.558 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:44.558 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:40.290309+0000", 2024-09-21T14:58:44.558 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:44.558 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.558 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.558 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:44.558 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:40.290309+0000", 2024-09-21T14:58:44.558 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:44.558 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.558 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.558 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:44.559 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:40.290309+0000", 2024-09-21T14:58:44.559 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:44.559 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.559 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.559 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:44.559 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:40.290314+0000", 2024-09-21T14:58:44.559 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5219999999999998e-06 2024-09-21T14:58:44.559 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.559 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.559 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:44.559 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:40.290330+0000", 2024-09-21T14:58:44.559 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5511000000000001e-05 2024-09-21T14:58:44.559 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.559 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.559 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:44.559 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:40.290341+0000", 2024-09-21T14:58:44.560 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1729000000000001e-05 2024-09-21T14:58:44.560 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.560 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.560 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:44.560 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:40.296093+0000", 2024-09-21T14:58:44.560 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.005751847 2024-09-21T14:58:44.560 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.560 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.560 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:44.560 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:40.296360+0000", 2024-09-21T14:58:44.560 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00026712099999999999 2024-09-21T14:58:44.560 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:44.560 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:44.560 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:44.561 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.561 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.561 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:19869 3.b 3:d7ca7d68:::rbd_data.11363a21d2be.000000000000002d:head [sparse-read 0~393216] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T14:58:44.561 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:44.340896+0000", 2024-09-21T14:58:44.561 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.156211497, 2024-09-21T14:58:44.561 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0071749170000000003, 2024-09-21T14:58:44.561 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:44.561 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:44.561 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:44.561 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:44.561 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:44.561 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 19869 2024-09-21T14:58:44.561 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.561 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:44.561 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.562 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:44.562 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340896+0000", 2024-09-21T14:58:44.562 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:44.562 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.562 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.562 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:44.562 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340896+0000", 2024-09-21T14:58:44.562 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:44.562 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.562 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.562 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:44.562 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340894+0000", 2024-09-21T14:58:44.562 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:44.562 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.563 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.563 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:44.563 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340903+0000", 2024-09-21T14:58:44.563 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0259999999999996e-06 2024-09-21T14:58:44.563 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.563 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.563 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:44.563 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340923+0000", 2024-09-21T14:58:44.563 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0013999999999998e-05 2024-09-21T14:58:44.563 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.563 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.563 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:44.563 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340938+0000", 2024-09-21T14:58:44.563 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4569000000000001e-05 2024-09-21T14:58:44.564 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.564 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.564 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:44.564 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342092+0000", 2024-09-21T14:58:44.564 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00115354 2024-09-21T14:58:44.564 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.564 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.564 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:44.564 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342290+0000", 2024-09-21T14:58:44.564 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019834299999999999 2024-09-21T14:58:44.564 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.564 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.564 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:44.564 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.348071+0000", 2024-09-21T14:58:44.564 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.005780861 2024-09-21T14:58:44.565 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:44.565 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:44.565 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:44.565 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.565 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.565 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:19872 3.b 3:d38ac3f6:::rbd_data.11363a21d2be.0000000000000030:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T14:58:44.565 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:44.340951+0000", 2024-09-21T14:58:44.565 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.15615633800000001, 2024-09-21T14:58:44.565 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0069228680000000004, 2024-09-21T14:58:44.565 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:44.565 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:44.565 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:44.565 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:44.565 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:44.566 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 19872 2024-09-21T14:58:44.566 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.566 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:44.566 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.566 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:44.566 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340951+0000", 2024-09-21T14:58:44.566 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:44.566 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.566 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.566 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:44.566 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340951+0000", 2024-09-21T14:58:44.566 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:44.566 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.566 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.567 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:44.567 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340950+0000", 2024-09-21T14:58:44.567 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:44.567 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.567 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.567 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:44.567 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340957+0000", 2024-09-21T14:58:44.567 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8430000000000004e-06 2024-09-21T14:58:44.567 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.567 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.567 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:44.567 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340974+0000", 2024-09-21T14:58:44.567 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7082999999999999e-05 2024-09-21T14:58:44.567 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.568 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.568 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:44.568 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340987+0000", 2024-09-21T14:58:44.568 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2763e-05 2024-09-21T14:58:44.568 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.568 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.568 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:44.568 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.347149+0000", 2024-09-21T14:58:44.568 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0061626600000000004 2024-09-21T14:58:44.568 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.568 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.568 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:44.568 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.347354+0000", 2024-09-21T14:58:44.568 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00020441799999999999 2024-09-21T14:58:44.569 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.569 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.569 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:44.569 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.347874+0000", 2024-09-21T14:58:44.569 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00052019399999999997 2024-09-21T14:58:44.569 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:44.569 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:44.569 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:44.569 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.569 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.569 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:19877 3.b 3:d5fc5a5a:::rbd_data.11363a21d2be.0000000000000035:head [sparse-read 0~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T14:58:44.569 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:44.340998+0000", 2024-09-21T14:58:44.569 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.15610950200000001, 2024-09-21T14:58:44.569 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0073322129999999998, 2024-09-21T14:58:44.569 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:44.569 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:44.570 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:44.570 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:44.570 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:44.570 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 19877 2024-09-21T14:58:44.570 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.570 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:44.570 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.570 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:44.570 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340998+0000", 2024-09-21T14:58:44.570 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:44.570 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.570 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.570 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:44.570 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340998+0000", 2024-09-21T14:58:44.571 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:44.571 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.571 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.571 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:44.571 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340997+0000", 2024-09-21T14:58:44.571 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:44.571 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.571 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.571 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:44.571 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.341003+0000", 2024-09-21T14:58:44.571 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0070000000000003e-06 2024-09-21T14:58:44.571 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.571 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.571 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:44.571 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.341018+0000", 2024-09-21T14:58:44.572 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5472999999999999e-05 2024-09-21T14:58:44.572 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.572 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.572 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:44.572 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.341040+0000", 2024-09-21T14:58:44.572 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1568000000000001e-05 2024-09-21T14:58:44.572 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.572 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.572 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:44.572 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.347901+0000", 2024-09-21T14:58:44.572 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0068608300000000001 2024-09-21T14:58:44.572 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.572 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.572 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:44.573 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.347973+0000", 2024-09-21T14:58:44.573 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2327000000000001e-05 2024-09-21T14:58:44.573 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.573 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.573 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:44.573 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.348330+0000", 2024-09-21T14:58:44.573 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00035695799999999999 2024-09-21T14:58:44.573 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:44.573 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:44.573 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:44.573 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.573 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.573 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:19941 3.b 3:d7b21c30:::rbd_data.11363a21d2be.0000000000000075:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T14:58:44.573 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:44.342141+0000", 2024-09-21T14:58:44.574 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.154965935, 2024-09-21T14:58:44.574 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0063087439999999998, 2024-09-21T14:58:44.574 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:44.574 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:58:44.574 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:44.574 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:44.574 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:44.574 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 19941 2024-09-21T14:58:44.574 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.574 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:44.574 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.574 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:44.574 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342141+0000", 2024-09-21T14:58:44.574 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:44.575 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.575 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.575 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:44.575 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342141+0000", 2024-09-21T14:58:44.575 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:44.575 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.575 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.575 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:44.575 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342139+0000", 2024-09-21T14:58:44.575 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T14:58:44.575 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.575 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.575 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:44.575 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342151+0000", 2024-09-21T14:58:44.576 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1242e-05 2024-09-21T14:58:44.576 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.576 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.576 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:44.576 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342175+0000", 2024-09-21T14:58:44.576 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4491e-05 2024-09-21T14:58:44.576 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.576 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.576 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:44.576 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342191+0000", 2024-09-21T14:58:44.576 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6288000000000001e-05 2024-09-21T14:58:44.576 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.576 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.576 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:44.577 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.348358+0000", 2024-09-21T14:58:44.577 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0061665449999999998 2024-09-21T14:58:44.577 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.577 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.577 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:44.577 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.348450+0000", 2024-09-21T14:58:44.577 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2245999999999997e-05 2024-09-21T14:58:44.577 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:44.577 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:44.577 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:44.577 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.577 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.577 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:19998 3.b 3:d4c6ec57:::rbd_data.11363a21d2be.00000000000000ae:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T14:58:44.577 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:44.342204+0000", 2024-09-21T14:58:44.578 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.15490373099999999, 2024-09-21T14:58:44.578 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0063531860000000002, 2024-09-21T14:58:44.578 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:44.578 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:58:44.578 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:44.578 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:44.578 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:44.578 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 19998 2024-09-21T14:58:44.578 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.578 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:44.578 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.578 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:44.578 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342204+0000", 2024-09-21T14:58:44.578 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:44.579 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.579 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.579 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:44.579 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342204+0000", 2024-09-21T14:58:44.579 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:44.579 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.579 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.579 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:44.579 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342203+0000", 2024-09-21T14:58:44.579 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:44.579 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.579 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.579 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:44.579 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342209+0000", 2024-09-21T14:58:44.579 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1739999999999997e-06 2024-09-21T14:58:44.580 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.580 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.580 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:44.580 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342223+0000", 2024-09-21T14:58:44.580 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4065e-05 2024-09-21T14:58:44.580 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.580 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.580 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:44.580 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342236+0000", 2024-09-21T14:58:44.580 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2537e-05 2024-09-21T14:58:44.580 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.580 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.580 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:44.580 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.348476+0000", 2024-09-21T14:58:44.581 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0062405760000000003 2024-09-21T14:58:44.581 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.581 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.581 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:44.581 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.348557+0000", 2024-09-21T14:58:44.581 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0716000000000004e-05 2024-09-21T14:58:44.581 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:44.581 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:44.581 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:44.581 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.581 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.581 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:19999 3.b 3:d4069488:::rbd_data.11363a21d2be.00000000000000af:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T14:58:44.581 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:44.342246+0000", 2024-09-21T14:58:44.581 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.15486156600000001, 2024-09-21T14:58:44.582 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0064139210000000004, 2024-09-21T14:58:44.582 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:44.582 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:58:44.582 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:44.582 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:44.582 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:44.582 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 19999 2024-09-21T14:58:44.582 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.582 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:44.582 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.582 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:44.582 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342246+0000", 2024-09-21T14:58:44.582 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:44.582 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.583 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.583 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:44.583 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342246+0000", 2024-09-21T14:58:44.583 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:44.583 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.583 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.583 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:44.583 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342245+0000", 2024-09-21T14:58:44.583 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:44.583 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.583 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.583 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:44.583 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342251+0000", 2024-09-21T14:58:44.583 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8660000000000002e-06 2024-09-21T14:58:44.583 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.584 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.584 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:44.584 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342264+0000", 2024-09-21T14:58:44.584 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3716999999999999e-05 2024-09-21T14:58:44.584 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.584 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.584 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:44.584 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342275+0000", 2024-09-21T14:58:44.584 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1071e-05 2024-09-21T14:58:44.584 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.584 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.584 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:44.584 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.348581+0000", 2024-09-21T14:58:44.585 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0063053730000000004 2024-09-21T14:58:44.585 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:44.585 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:44.585 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:44.585 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.348660+0000", 2024-09-21T14:58:44.585 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8854000000000001e-05 2024-09-21T14:58:44.585 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:44.585 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:44.585 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:44.585 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:44.585 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:44.585 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:58:44.586 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T14:58:44.587 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:58:44.584+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:58:44.600 INFO:teuthology.orchestra.run.smithi032.stdout:552 trunc from 0x4238e27 to 0xd27edb2 2024-09-21T14:58:44.620 INFO:teuthology.orchestra.run.smithi032.stdout:555 write 0x9fac44c thru 0x9faf1a8 (0x2d5d bytes) 2024-09-21T14:58:44.626 INFO:teuthology.orchestra.run.smithi032.stdout:556 writesame 0x5115274 thru 0x51192c3 data_size 0x62(0x4050 bytes) 2024-09-21T14:58:44.636 INFO:teuthology.orchestra.run.smithi032.stdout:558 trunc from 0xd27edb2 to 0xc9997e9 2024-09-21T14:58:44.647 INFO:teuthology.orchestra.run.smithi032.stdout:559 write 0x10e11e thru 0x115794 (0x7677 bytes) 2024-09-21T14:58:44.651 INFO:teuthology.orchestra.run.smithi032.stdout:560 write 0x604fefe thru 0x605f22c (0xf32f bytes) 2024-09-21T14:58:44.660 INFO:teuthology.orchestra.run.smithi032.stdout:562 read 0xb284652 thru 0xb288dd6 (0x4785 bytes) 2024-09-21T14:58:44.670 INFO:teuthology.orchestra.run.smithi032.stdout:564 trunc from 0xc9997e9 to 0x77654ca 2024-09-21T14:58:44.687 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:58:44.684+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:58:44.693 INFO:teuthology.orchestra.run.smithi032.stdout:565 read 0x65dad63 thru 0x65e6ac7 (0xbd65 bytes) 2024-09-21T14:58:44.705 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:58:44.705 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:58:44.706 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T14:58:44.706 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:58:44.711 INFO:teuthology.orchestra.run.smithi032.stdout:567 write 0x9f5c470 thru 0x9f5f494 (0x3025 bytes) 2024-09-21T14:58:44.716 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T14:58:44.719 INFO:teuthology.orchestra.run.smithi032.stdout:568 read 0x1b74845 thru 0x1b79fb9 (0x5775 bytes) 2024-09-21T14:58:44.725 INFO:teuthology.orchestra.run.smithi032.stdout:571 trunc from 0x9f5f495 to 0x6dd04fe 2024-09-21T14:58:44.747 INFO:teuthology.orchestra.run.smithi032.stdout:573 read 0x2e58f4 thru 0x2e631c (0xa29 bytes) 2024-09-21T14:58:44.765 INFO:teuthology.orchestra.run.smithi032.stdout:575 writesame 0x8f0211e thru 0x8f0fbde data_size 0x1(0xdac1 bytes) 2024-09-21T14:58:44.787 INFO:teuthology.orchestra.run.smithi032.stdout:576 writesame 0xd08a231 thru 0xd0960df data_size 0xd(0xbeaf bytes) 2024-09-21T14:58:44.787 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:44.788 INFO:teuthology.orchestra.run.smithi032.stdout:578 punch from 0x980393f to 0x9803d3d, (0x3fe bytes) 2024-09-21T14:58:44.788 INFO:teuthology.orchestra.run.smithi032.stdout:581 punch from 0x90ead44 to 0x90f3a76, (0x8d32 bytes) 2024-09-21T14:58:44.788 INFO:teuthology.orchestra.run.smithi032.stdout:583 punch from 0x7979f4f to 0x797b24f, (0x1300 bytes) 2024-09-21T14:58:44.788 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:44.788 INFO:teuthology.orchestra.run.smithi032.stdout:586 punch from 0xf04006 to 0xf07ffc, (0x3ff6 bytes) 2024-09-21T14:58:44.788 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:44.788 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:44.788 INFO:teuthology.orchestra.run.smithi032.stdout:590 write 0x2cb995f thru 0x2cbe5db (0x4c7d bytes) 2024-09-21T14:58:44.788 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:58:44.784+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:58:44.795 INFO:teuthology.orchestra.run.smithi032.stdout:591 writesame 0x796ccd thru 0x7a5505 data_size 0x11(0xe839 bytes) 2024-09-21T14:58:44.800 INFO:teuthology.orchestra.run.smithi032.stdout:592 punch from 0x55adfda to 0x55b5733, (0x7759 bytes) 2024-09-21T14:58:44.800 INFO:teuthology.orchestra.run.smithi032.stdout:593 punch from 0xb09923b to 0xb0a6403, (0xd1c8 bytes) 2024-09-21T14:58:44.800 INFO:teuthology.orchestra.run.smithi032.stdout:595 writesame 0x72c9c3e thru 0x72cbf56 data_size 0x5(0x2319 bytes) 2024-09-21T14:58:44.808 INFO:teuthology.orchestra.run.smithi032.stdout:596 writesame 0x3db5b79 thru 0x3dc0a8e data_size 0x49(0xaf16 bytes) 2024-09-21T14:58:44.817 INFO:teuthology.orchestra.run.smithi032.stdout:598 punch from 0x2687765 to 0x2692864, (0xb0ff bytes) 2024-09-21T14:58:44.817 INFO:teuthology.orchestra.run.smithi032.stdout:599 read 0x6e24f0c thru 0x6e2e07f (0x9174 bytes) 2024-09-21T14:58:44.833 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:58:44.833 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:58:44.833 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T14:58:44.833 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T14:58:44.833 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:58:44.837 INFO:teuthology.orchestra.run.smithi032.stdout:604 trunc from 0xd0960e0 to 0x120c8c8 2024-09-21T14:58:44.845 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T14:58:44.864 INFO:teuthology.orchestra.run.smithi032.stdout:605 read 0xb75318 thru 0xb7983f (0x4528 bytes) 2024-09-21T14:58:44.865 INFO:teuthology.orchestra.run.smithi032.stdout:606 punch from 0xd29260 to 0xd31ed4, (0x8c74 bytes) 2024-09-21T14:58:44.873 INFO:teuthology.orchestra.run.smithi032.stdout:609 trunc from 0x120c8c8 to 0x5bb5730 2024-09-21T14:58:44.883 INFO:teuthology.orchestra.run.smithi032.stdout:610 punch from 0x312d37e to 0x312ea61, (0x16e3 bytes) 2024-09-21T14:58:44.888 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:58:44.884+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:58:44.912 INFO:teuthology.orchestra.run.smithi032.stdout:613 write 0xeb97deb thru 0xeb99ca4 (0x1eba bytes) 2024-09-21T14:58:44.921 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:44.921 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:44.921 INFO:teuthology.orchestra.run.smithi032.stdout:617 write 0x162fe8f thru 0x16359a4 (0x5b16 bytes) 2024-09-21T14:58:44.937 INFO:teuthology.orchestra.run.smithi032.stdout:619 trunc from 0xeb99ca5 to 0xa5ad385 2024-09-21T14:58:44.959 INFO:teuthology.orchestra.run.smithi032.stdout:620 punch from 0x8f0ad1a to 0x8f0d0bd, (0x23a3 bytes) 2024-09-21T14:58:44.959 INFO:teuthology.orchestra.run.smithi032.stdout:621 writesame 0x7a1a7d5 thru 0x7a1c089 data_size 0x19(0x18b5 bytes) 2024-09-21T14:58:44.966 INFO:teuthology.orchestra.run.smithi032.stdout:622 read 0x9c07df2 thru 0x9c0d3b3 (0x55c2 bytes) 2024-09-21T14:58:44.966 INFO:teuthology.orchestra.run.smithi032.stdout:623 read 0xfa4c38 thru 0xfab6cd (0x6a96 bytes) 2024-09-21T14:58:44.967 INFO:teuthology.orchestra.run.smithi032.stdout:624 writesame 0x6a48544 thru 0x6a53015 data_size 0x5(0xaad2 bytes) 2024-09-21T14:58:44.976 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:58:44.976 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T14:58:44.976 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T14:58:44.976 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T14:58:44.976 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:44.977 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:1473 3.15 3:af2afa6a:::rbd_data.1136515ecb6b.000000000000001b:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T14:58:44.977 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:30.421730+0000", 2024-09-21T14:58:44.977 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.550897737, 2024-09-21T14:58:44.977 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021478937, 2024-09-21T14:58:44.977 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:44.977 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:44.977 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:44.977 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:44.977 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:44.977 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1473 2024-09-21T14:58:44.977 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:44.977 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:44.977 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:44.977 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:44.978 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421730+0000", 2024-09-21T14:58:44.978 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:44.978 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:44.978 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:44.978 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:44.978 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421730+0000", 2024-09-21T14:58:44.978 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:44.978 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:44.978 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:44.978 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:44.978 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421728+0000", 2024-09-21T14:58:44.978 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-21T14:58:44.978 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:44.978 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:44.978 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:44.979 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421739+0000", 2024-09-21T14:58:44.979 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0502e-05 2024-09-21T14:58:44.979 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:44.979 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:44.979 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:44.979 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421762+0000", 2024-09-21T14:58:44.979 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3405999999999999e-05 2024-09-21T14:58:44.979 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:44.979 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:44.979 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:44.979 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421773+0000", 2024-09-21T14:58:44.979 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0744000000000001e-05 2024-09-21T14:58:44.979 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:44.979 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:44.980 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:44.980 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421838+0000", 2024-09-21T14:58:44.980 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5486000000000003e-05 2024-09-21T14:58:44.980 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:44.980 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:44.980 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:44.980 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421955+0000", 2024-09-21T14:58:44.980 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000116765 2024-09-21T14:58:44.980 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:44.980 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:44.980 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 6", 2024-09-21T14:58:44.980 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.425572+0000", 2024-09-21T14:58:44.980 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0036171380000000002 2024-09-21T14:58:44.980 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:44.981 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:44.981 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:44.981 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.442973+0000", 2024-09-21T14:58:44.981 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.017400268 2024-09-21T14:58:44.981 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:44.981 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:44.981 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:44.981 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.442996+0000", 2024-09-21T14:58:44.981 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3139e-05 2024-09-21T14:58:44.981 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:44.981 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:44.981 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:44.981 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.443079+0000", 2024-09-21T14:58:44.981 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2832999999999998e-05 2024-09-21T14:58:44.981 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:44.982 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:44.982 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:44.982 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.443209+0000", 2024-09-21T14:58:44.982 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013052499999999999 2024-09-21T14:58:44.982 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:44.982 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:44.982 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:44.982 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:44.982 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:44.982 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:1678 3.c 3:377cc7e1:::rbd_data.1136515ecb6b.000000000000000f:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T14:58:44.982 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:30.988085+0000", 2024-09-21T14:58:44.982 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 13.984543143, 2024-09-21T14:58:44.982 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.032974802999999997, 2024-09-21T14:58:44.982 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:44.983 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:44.983 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:44.983 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:44.983 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:44.983 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1678 2024-09-21T14:58:44.983 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:44.983 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:44.983 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:44.983 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:44.983 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988085+0000", 2024-09-21T14:58:44.983 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:44.983 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:44.983 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:44.983 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:44.983 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988085+0000", 2024-09-21T14:58:44.984 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:44.984 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:44.984 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:44.984 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:44.984 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988083+0000", 2024-09-21T14:58:44.984 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-21T14:58:44.984 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:44.984 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:44.984 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:44.984 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988093+0000", 2024-09-21T14:58:44.984 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8800000000000003e-06 2024-09-21T14:58:44.984 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:44.984 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:44.984 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:44.985 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988113+0000", 2024-09-21T14:58:44.985 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0217999999999999e-05 2024-09-21T14:58:44.985 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:44.985 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:44.985 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:44.985 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988123+0000", 2024-09-21T14:58:44.985 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5859999999999994e-06 2024-09-21T14:58:44.985 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:44.985 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:44.985 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:44.985 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988203+0000", 2024-09-21T14:58:44.985 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0834000000000006e-05 2024-09-21T14:58:44.985 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:44.985 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:44.985 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:44.985 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988376+0000", 2024-09-21T14:58:44.986 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017301299999999999 2024-09-21T14:58:44.986 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:44.986 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:44.986 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 7", 2024-09-21T14:58:44.986 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.996338+0000", 2024-09-21T14:58:44.986 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0079618520000000002 2024-09-21T14:58:44.986 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:44.986 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:44.986 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:44.986 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.020881+0000", 2024-09-21T14:58:44.986 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024542731000000002 2024-09-21T14:58:44.986 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:44.986 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:44.987 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:44.987 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.020904+0000", 2024-09-21T14:58:44.987 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2861e-05 2024-09-21T14:58:44.987 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:44.987 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:44.987 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:44.987 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.020987+0000", 2024-09-21T14:58:44.987 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2806000000000006e-05 2024-09-21T14:58:44.987 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:44.987 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:44.987 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:44.987 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.021060+0000", 2024-09-21T14:58:44.987 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2871000000000001e-05 2024-09-21T14:58:44.987 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:44.987 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:44.988 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:44.988 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:44.988 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:44.988 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9342 3.3 3:c0bf6a9b:::rbd_data.11363a21d2be.0000000000000000:10 [read 483328~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:44.988 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157188+0000", 2024-09-21T14:58:44.988 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 5.8154397229999999, 2024-09-21T14:58:44.988 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018001409999999999, 2024-09-21T14:58:44.988 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:44.988 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:44.988 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:44.988 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:44.988 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:44.988 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9342 2024-09-21T14:58:44.988 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:44.988 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:44.989 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:44.989 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:44.989 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157188+0000", 2024-09-21T14:58:44.989 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:44.989 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:44.989 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:44.989 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:44.989 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157188+0000", 2024-09-21T14:58:44.989 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:44.989 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:44.989 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:44.989 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:44.989 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157188+0000", 2024-09-21T14:58:44.990 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:44.990 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:44.990 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:44.990 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:44.990 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157191+0000", 2024-09-21T14:58:44.990 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7209999999999999e-06 2024-09-21T14:58:44.990 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:44.990 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:44.990 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:44.990 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157196+0000", 2024-09-21T14:58:44.990 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9139999999999996e-06 2024-09-21T14:58:44.990 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:44.990 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:44.990 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:44.991 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157232+0000", 2024-09-21T14:58:44.991 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5598e-05 2024-09-21T14:58:44.991 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:44.991 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:44.991 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:44.991 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175014+0000", 2024-09-21T14:58:44.991 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.017782128000000001 2024-09-21T14:58:44.991 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:44.991 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:44.991 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:44.991 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175058+0000", 2024-09-21T14:58:44.991 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3837000000000001e-05 2024-09-21T14:58:44.991 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:44.992 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:44.992 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:44.992 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175190+0000", 2024-09-21T14:58:44.992 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013161299999999999 2024-09-21T14:58:44.992 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:44.992 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:44.992 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:44.992 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:44.992 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:44.992 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9369 3.3 3:c725132c:::rbd_data.11363a21d2be.0000000000000007:10 [read 24576~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:44.992 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157241+0000", 2024-09-21T14:58:44.992 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 5.8153866970000001, 2024-09-21T14:58:44.992 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018289156000000001, 2024-09-21T14:58:44.992 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:44.992 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:44.992 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:44.993 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:44.993 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:44.993 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9369 2024-09-21T14:58:44.993 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:44.993 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:44.993 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:44.993 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:44.993 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157241+0000", 2024-09-21T14:58:44.993 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:44.993 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:44.993 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:44.993 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:44.993 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157241+0000", 2024-09-21T14:58:44.993 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:44.993 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:44.993 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:44.994 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:44.994 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157241+0000", 2024-09-21T14:58:44.994 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:44.994 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:44.994 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:44.994 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:44.994 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157244+0000", 2024-09-21T14:58:44.994 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5049999999999999e-06 2024-09-21T14:58:44.994 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:44.994 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:44.994 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:44.994 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157255+0000", 2024-09-21T14:58:44.994 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0797e-05 2024-09-21T14:58:44.994 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:44.995 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:44.995 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:44.995 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157260+0000", 2024-09-21T14:58:44.995 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1749999999999996e-06 2024-09-21T14:58:44.995 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:44.995 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:44.995 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:44.995 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175237+0000", 2024-09-21T14:58:44.995 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.017976405000000001 2024-09-21T14:58:44.995 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:44.995 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:44.995 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:44.995 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175331+0000", 2024-09-21T14:58:44.996 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4508999999999999e-05 2024-09-21T14:58:44.996 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:44.996 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:44.996 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:44.996 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175530+0000", 2024-09-21T14:58:44.996 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00019928200000000001 2024-09-21T14:58:44.996 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:44.996 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:44.996 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:44.996 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:44.996 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:44.996 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9370 3.3 3:c725132c:::rbd_data.11363a21d2be.0000000000000007:10 [read 163840~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:44.996 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157279+0000", 2024-09-21T14:58:44.996 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 5.8153485190000005, 2024-09-21T14:58:44.997 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018472379000000001, 2024-09-21T14:58:44.997 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:44.997 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:44.997 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:44.997 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:44.997 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:44.997 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9370 2024-09-21T14:58:44.997 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:44.997 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:44.997 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:44.997 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:44.997 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157279+0000", 2024-09-21T14:58:44.997 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:44.997 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:44.997 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:44.998 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:44.998 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157279+0000", 2024-09-21T14:58:44.998 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:44.998 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:44.998 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:44.998 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:44.998 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157279+0000", 2024-09-21T14:58:44.998 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:44.998 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:44.998 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:44.998 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:44.998 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157282+0000", 2024-09-21T14:58:44.998 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2229999999999999e-06 2024-09-21T14:58:44.998 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:44.998 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:44.999 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:44.999 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157294+0000", 2024-09-21T14:58:44.999 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2173e-05 2024-09-21T14:58:44.999 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:44.999 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:44.999 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:44.999 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157300+0000", 2024-09-21T14:58:44.999 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.93e-06 2024-09-21T14:58:44.999 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:44.999 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:44.999 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:44.999 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175569+0000", 2024-09-21T14:58:44.999 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018268376999999999 2024-09-21T14:58:44.999 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.000 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.000 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:45.000 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175622+0000", 2024-09-21T14:58:45.000 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3808999999999999e-05 2024-09-21T14:58:45.000 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.000 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.000 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:45.000 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175752+0000", 2024-09-21T14:58:45.000 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012938700000000001 2024-09-21T14:58:45.000 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:45.000 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:45.000 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:45.000 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.000 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.000 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9371 3.3 3:c0bf6a9b:::rbd_data.11363a21d2be.0000000000000000:10 [read 73728~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:45.001 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157306+0000", 2024-09-21T14:58:45.001 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 5.8153224080000001, 2024-09-21T14:58:45.001 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018755473000000002, 2024-09-21T14:58:45.001 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:45.001 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:45.001 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:45.001 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:45.001 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:45.001 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9371 2024-09-21T14:58:45.001 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.001 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:45.001 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.001 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:45.001 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157306+0000", 2024-09-21T14:58:45.002 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:45.002 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.002 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.002 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:45.002 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157306+0000", 2024-09-21T14:58:45.002 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:45.002 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.002 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.002 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:45.002 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157305+0000", 2024-09-21T14:58:45.002 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:45.002 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.002 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.002 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:45.002 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157312+0000", 2024-09-21T14:58:45.003 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3289999999999996e-06 2024-09-21T14:58:45.003 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.003 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.003 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:45.003 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157324+0000", 2024-09-21T14:58:45.003 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1725e-05 2024-09-21T14:58:45.003 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.003 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.003 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:45.003 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157339+0000", 2024-09-21T14:58:45.003 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5211999999999999e-05 2024-09-21T14:58:45.003 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.003 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.003 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:45.004 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175798+0000", 2024-09-21T14:58:45.004 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018458781 2024-09-21T14:58:45.004 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.004 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.004 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:45.004 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175899+0000", 2024-09-21T14:58:45.004 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000100784 2024-09-21T14:58:45.004 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.004 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.004 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:45.004 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176061+0000", 2024-09-21T14:58:45.004 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00016216099999999999 2024-09-21T14:58:45.004 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:45.005 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:45.005 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:45.005 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.005 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.005 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9372 3.3 3:c0bf6a9b:::rbd_data.11363a21d2be.0000000000000000:10 [read 212992~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:45.005 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157345+0000", 2024-09-21T14:58:45.005 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 5.8152830619999998, 2024-09-21T14:58:45.005 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018926493999999999, 2024-09-21T14:58:45.005 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:45.005 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:45.005 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:45.005 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:45.005 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:45.005 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9372 2024-09-21T14:58:45.005 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.006 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:45.006 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.006 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:45.006 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157345+0000", 2024-09-21T14:58:45.006 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:45.006 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.006 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.006 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:45.006 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157345+0000", 2024-09-21T14:58:45.006 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:45.006 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.006 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.006 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:45.006 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157344+0000", 2024-09-21T14:58:45.007 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:45.007 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.007 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.007 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:45.007 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157348+0000", 2024-09-21T14:58:45.007 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1389999999999999e-06 2024-09-21T14:58:45.007 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.007 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.007 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:45.007 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157372+0000", 2024-09-21T14:58:45.007 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4711000000000001e-05 2024-09-21T14:58:45.007 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.007 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.007 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:45.008 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157378+0000", 2024-09-21T14:58:45.008 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9909999999999999e-06 2024-09-21T14:58:45.008 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.008 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.008 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:45.008 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176090+0000", 2024-09-21T14:58:45.008 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018712111999999999 2024-09-21T14:58:45.008 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.008 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.008 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:45.008 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176134+0000", 2024-09-21T14:58:45.008 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3377999999999997e-05 2024-09-21T14:58:45.008 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.008 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.009 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:45.009 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176271+0000", 2024-09-21T14:58:45.009 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013763499999999999 2024-09-21T14:58:45.009 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:45.009 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:45.009 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:45.009 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.009 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.009 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9373 3.3 3:c0bf6a9b:::rbd_data.11363a21d2be.0000000000000000:10 [read 352256~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:45.009 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157384+0000", 2024-09-21T14:58:45.009 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 5.8152442899999999, 2024-09-21T14:58:45.009 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019154937, 2024-09-21T14:58:45.009 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:45.009 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:45.010 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:45.010 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:45.010 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:45.010 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9373 2024-09-21T14:58:45.010 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.010 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:45.010 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.010 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:45.010 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157384+0000", 2024-09-21T14:58:45.010 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:45.010 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.010 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.010 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:45.011 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157384+0000", 2024-09-21T14:58:45.011 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:45.011 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.011 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.011 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:45.011 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157383+0000", 2024-09-21T14:58:45.011 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:45.011 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.011 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.011 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:45.011 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157386+0000", 2024-09-21T14:58:45.011 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.191e-06 2024-09-21T14:58:45.011 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.011 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.012 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:45.012 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157393+0000", 2024-09-21T14:58:45.012 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5339999999999996e-06 2024-09-21T14:58:45.012 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.012 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.012 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:45.012 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157397+0000", 2024-09-21T14:58:45.012 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5220000000000001e-06 2024-09-21T14:58:45.012 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.012 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.012 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:45.012 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176302+0000", 2024-09-21T14:58:45.012 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018905113000000001 2024-09-21T14:58:45.012 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.013 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.013 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:45.013 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176379+0000", 2024-09-21T14:58:45.013 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6372999999999995e-05 2024-09-21T14:58:45.013 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.013 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.013 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:45.013 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176539+0000", 2024-09-21T14:58:45.013 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015973800000000001 2024-09-21T14:58:45.013 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:45.013 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:45.013 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:45.013 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.013 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.013 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9374 3.3 3:c0bf6a9b:::rbd_data.11363a21d2be.0000000000000000:10 [read 491520~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:45.014 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157401+0000", 2024-09-21T14:58:45.014 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 5.8152265060000001, 2024-09-21T14:58:45.014 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019387755, 2024-09-21T14:58:45.014 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:45.014 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:45.014 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:45.014 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:45.014 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:45.014 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9374 2024-09-21T14:58:45.014 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.014 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:45.014 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.014 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:45.014 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157401+0000", 2024-09-21T14:58:45.014 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:45.015 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.015 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.015 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:45.015 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157401+0000", 2024-09-21T14:58:45.015 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:45.015 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.015 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.015 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:45.015 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157401+0000", 2024-09-21T14:58:45.015 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:45.015 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.015 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.015 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:45.015 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157414+0000", 2024-09-21T14:58:45.015 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2417e-05 2024-09-21T14:58:45.015 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.016 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.016 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:45.016 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157424+0000", 2024-09-21T14:58:45.016 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0861e-05 2024-09-21T14:58:45.016 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.016 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.016 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:45.016 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157429+0000", 2024-09-21T14:58:45.016 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0499999999999999e-06 2024-09-21T14:58:45.016 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.016 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.016 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:45.016 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176596+0000", 2024-09-21T14:58:45.016 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019166552 2024-09-21T14:58:45.016 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.017 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.017 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:45.017 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176647+0000", 2024-09-21T14:58:45.017 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1112000000000001e-05 2024-09-21T14:58:45.017 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.017 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.017 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:45.017 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176789+0000", 2024-09-21T14:58:45.017 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014209300000000001 2024-09-21T14:58:45.017 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:45.017 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:45.017 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:45.017 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.017 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.017 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9401 3.3 3:c725132c:::rbd_data.11363a21d2be.0000000000000007:10 [read 32768~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:45.018 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157621+0000", 2024-09-21T14:58:45.018 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 5.8150066970000003, 2024-09-21T14:58:45.018 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019381711999999999, 2024-09-21T14:58:45.018 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:45.018 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:45.018 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:45.018 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:45.018 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:45.018 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9401 2024-09-21T14:58:45.018 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.018 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:45.018 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.018 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:45.018 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157621+0000", 2024-09-21T14:58:45.019 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:45.019 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.019 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.019 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:45.019 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157621+0000", 2024-09-21T14:58:45.019 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:45.019 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.019 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.019 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:45.019 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157620+0000", 2024-09-21T14:58:45.019 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:58:45.019 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.019 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.019 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:45.019 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157626+0000", 2024-09-21T14:58:45.020 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7570000000000001e-06 2024-09-21T14:58:45.020 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.020 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.020 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:45.020 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157636+0000", 2024-09-21T14:58:45.020 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8339999999999994e-06 2024-09-21T14:58:45.020 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.020 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.020 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:45.020 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157643+0000", 2024-09-21T14:58:45.020 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9439999999999999e-06 2024-09-21T14:58:45.020 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.020 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.020 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:45.020 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176816+0000", 2024-09-21T14:58:45.021 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019172872000000001 2024-09-21T14:58:45.021 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.021 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.021 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:45.021 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176878+0000", 2024-09-21T14:58:45.021 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1734000000000002e-05 2024-09-21T14:58:45.021 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.021 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.021 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:45.021 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177003+0000", 2024-09-21T14:58:45.021 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012537400000000001 2024-09-21T14:58:45.021 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:45.021 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:45.021 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:45.021 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.022 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.022 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9402 3.3 3:c725132c:::rbd_data.11363a21d2be.0000000000000007:10 [read 172032~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:45.022 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157649+0000", 2024-09-21T14:58:45.022 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 5.8149788249999999, 2024-09-21T14:58:45.022 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019567404, 2024-09-21T14:58:45.022 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:45.022 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:45.022 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:45.022 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:45.022 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:45.022 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9402 2024-09-21T14:58:45.022 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.022 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:45.022 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.022 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:45.023 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157649+0000", 2024-09-21T14:58:45.023 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:45.023 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.023 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.023 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:45.023 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157649+0000", 2024-09-21T14:58:45.026 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:45.026 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.026 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.026 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:45.026 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157649+0000", 2024-09-21T14:58:45.026 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:45.026 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.026 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.026 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:45.026 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157652+0000", 2024-09-21T14:58:45.027 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8779999999999998e-06 2024-09-21T14:58:45.027 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.027 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.027 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:45.027 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157657+0000", 2024-09-21T14:58:45.027 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6659999999999996e-06 2024-09-21T14:58:45.027 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.027 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.027 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:45.027 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157662+0000", 2024-09-21T14:58:45.027 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5519999999999998e-06 2024-09-21T14:58:45.027 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.027 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.028 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:45.028 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177040+0000", 2024-09-21T14:58:45.028 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019377807 2024-09-21T14:58:45.028 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.028 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.028 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:45.028 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177085+0000", 2024-09-21T14:58:45.028 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5195999999999999e-05 2024-09-21T14:58:45.028 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.028 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.028 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:45.028 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177217+0000", 2024-09-21T14:58:45.028 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013172600000000001 2024-09-21T14:58:45.028 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:45.029 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:45.029 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:45.029 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.029 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.029 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9403 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:45.029 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157949+0000", 2024-09-21T14:58:45.029 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 5.814678496, 2024-09-21T14:58:45.029 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023540373999999999, 2024-09-21T14:58:45.029 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:45.029 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:45.029 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:45.029 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:45.029 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:45.029 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9403 2024-09-21T14:58:45.029 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.030 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:45.030 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.030 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:45.030 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157949+0000", 2024-09-21T14:58:45.030 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:45.030 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.030 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.030 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:45.030 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157949+0000", 2024-09-21T14:58:45.030 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:45.030 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.030 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.030 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:45.030 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157948+0000", 2024-09-21T14:58:45.031 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:45.031 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.031 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.031 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:45.031 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157956+0000", 2024-09-21T14:58:45.031 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7999999999999999e-06 2024-09-21T14:58:45.031 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.031 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.031 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:45.031 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157969+0000", 2024-09-21T14:58:45.031 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2792000000000001e-05 2024-09-21T14:58:45.031 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.031 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.031 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:45.032 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157977+0000", 2024-09-21T14:58:45.032 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.704e-06 2024-09-21T14:58:45.032 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.032 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.032 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:45.032 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177246+0000", 2024-09-21T14:58:45.032 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019268264 2024-09-21T14:58:45.032 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.032 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.032 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:45.032 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177315+0000", 2024-09-21T14:58:45.032 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9480999999999994e-05 2024-09-21T14:58:45.032 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.032 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.032 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:58:45.033 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177615+0000", 2024-09-21T14:58:45.033 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00029929499999999999 2024-09-21T14:58:45.033 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.033 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.033 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:45.033 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181333+0000", 2024-09-21T14:58:45.033 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0037183149999999998 2024-09-21T14:58:45.033 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.033 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.033 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:45.033 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181374+0000", 2024-09-21T14:58:45.033 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.0723999999999998e-05 2024-09-21T14:58:45.033 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.033 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.033 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:45.033 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181434+0000", 2024-09-21T14:58:45.034 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0024e-05 2024-09-21T14:58:45.034 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.034 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.034 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:45.034 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181490+0000", 2024-09-21T14:58:45.034 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6245000000000001e-05 2024-09-21T14:58:45.034 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:45.034 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:45.034 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:45.034 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.034 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.034 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9404 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:45.034 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.160480+0000", 2024-09-21T14:58:45.035 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 5.8121481460000002, 2024-09-21T14:58:45.035 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021137686999999999, 2024-09-21T14:58:45.035 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:45.035 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:45.035 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:45.035 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:45.035 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:45.035 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9404 2024-09-21T14:58:45.035 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.035 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:45.035 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.035 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:45.035 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160480+0000", 2024-09-21T14:58:45.035 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:45.036 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.036 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.036 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:45.036 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160480+0000", 2024-09-21T14:58:45.036 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:45.036 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.036 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.036 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:45.036 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160478+0000", 2024-09-21T14:58:45.036 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:45.036 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.036 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.036 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:45.036 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160486+0000", 2024-09-21T14:58:45.037 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9969999999999998e-06 2024-09-21T14:58:45.037 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.037 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.037 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:45.037 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160502+0000", 2024-09-21T14:58:45.037 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5650000000000001e-05 2024-09-21T14:58:45.037 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.037 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.037 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:45.037 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160511+0000", 2024-09-21T14:58:45.037 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.3610000000000007e-06 2024-09-21T14:58:45.037 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.037 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.037 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:45.038 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.178035+0000", 2024-09-21T14:58:45.038 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.017523520000000001 2024-09-21T14:58:45.038 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.038 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.038 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:45.038 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.178151+0000", 2024-09-21T14:58:45.038 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000116013 2024-09-21T14:58:45.038 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.038 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.038 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:58:45.038 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.178408+0000", 2024-09-21T14:58:45.038 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00025743400000000002 2024-09-21T14:58:45.038 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.038 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.038 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:45.039 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181501+0000", 2024-09-21T14:58:45.039 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.003092891 2024-09-21T14:58:45.039 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.039 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.039 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:45.039 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181538+0000", 2024-09-21T14:58:45.039 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6977999999999997e-05 2024-09-21T14:58:45.039 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.039 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.039 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:45.039 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181582+0000", 2024-09-21T14:58:45.039 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3849e-05 2024-09-21T14:58:45.039 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.040 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.040 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:45.040 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181617+0000", 2024-09-21T14:58:45.040 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5423000000000003e-05 2024-09-21T14:58:45.040 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:45.040 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:45.040 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:45.040 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.040 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.040 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9406 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:45.040 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.166040+0000", 2024-09-21T14:58:45.040 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 5.8065882670000004, 2024-09-21T14:58:45.040 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018094524000000001, 2024-09-21T14:58:45.040 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:45.040 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:45.041 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:45.041 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:45.041 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:45.041 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9406 2024-09-21T14:58:45.041 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.041 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:45.041 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.041 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:45.041 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.166040+0000", 2024-09-21T14:58:45.041 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:45.041 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.041 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.041 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:45.041 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.166040+0000", 2024-09-21T14:58:45.041 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:45.042 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.042 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.042 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:45.042 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.166038+0000", 2024-09-21T14:58:45.042 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:45.042 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.042 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.042 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:45.042 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.166051+0000", 2024-09-21T14:58:45.042 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2638e-05 2024-09-21T14:58:45.042 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.042 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.042 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:45.042 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.166070+0000", 2024-09-21T14:58:45.043 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8691000000000001e-05 2024-09-21T14:58:45.043 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.043 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.043 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:45.043 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.166089+0000", 2024-09-21T14:58:45.043 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9286000000000001e-05 2024-09-21T14:58:45.043 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.043 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.043 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:45.043 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.180194+0000", 2024-09-21T14:58:45.043 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014105039999999999 2024-09-21T14:58:45.043 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.043 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.043 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:45.043 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.180340+0000", 2024-09-21T14:58:45.044 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014623199999999999 2024-09-21T14:58:45.044 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.044 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.044 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:58:45.044 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.180818+0000", 2024-09-21T14:58:45.044 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000477489 2024-09-21T14:58:45.044 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.044 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.044 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:45.044 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.183808+0000", 2024-09-21T14:58:45.044 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0029897610000000001 2024-09-21T14:58:45.044 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.044 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.044 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:45.044 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.183821+0000", 2024-09-21T14:58:45.045 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3251999999999999e-05 2024-09-21T14:58:45.045 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.045 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.045 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:45.045 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.183872+0000", 2024-09-21T14:58:45.045 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0900999999999999e-05 2024-09-21T14:58:45.045 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.045 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.045 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:45.045 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.184134+0000", 2024-09-21T14:58:45.045 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000262484 2024-09-21T14:58:45.045 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:45.045 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:45.045 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:45.045 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.045 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.046 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9407 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:45.046 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.168094+0000", 2024-09-21T14:58:45.046 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 5.8045336469999995, 2024-09-21T14:58:45.046 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019444860000000001, 2024-09-21T14:58:45.046 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:45.046 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:45.046 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:45.046 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:45.046 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:45.046 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9407 2024-09-21T14:58:45.046 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.046 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:45.046 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.046 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:45.047 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.168094+0000", 2024-09-21T14:58:45.047 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:45.047 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.047 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.047 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:45.047 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.168094+0000", 2024-09-21T14:58:45.047 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:45.047 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.047 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.047 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:45.047 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.168093+0000", 2024-09-21T14:58:45.047 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:58:45.047 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.047 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.048 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:45.048 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.168119+0000", 2024-09-21T14:58:45.048 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5803999999999999e-05 2024-09-21T14:58:45.048 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.048 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.048 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:45.048 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.168135+0000", 2024-09-21T14:58:45.048 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6359999999999999e-05 2024-09-21T14:58:45.048 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.048 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.048 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:45.048 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.168144+0000", 2024-09-21T14:58:45.048 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4270000000000008e-06 2024-09-21T14:58:45.048 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.049 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.049 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:45.049 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181943+0000", 2024-09-21T14:58:45.049 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013799617 2024-09-21T14:58:45.049 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.049 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.049 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:45.049 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.182052+0000", 2024-09-21T14:58:45.049 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000108924 2024-09-21T14:58:45.049 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.049 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.049 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:58:45.049 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.183431+0000", 2024-09-21T14:58:45.049 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0013783619999999999 2024-09-21T14:58:45.050 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.050 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.050 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:45.050 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.187276+0000", 2024-09-21T14:58:45.050 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0038450469999999999 2024-09-21T14:58:45.050 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.050 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.050 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:45.050 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.187288+0000", 2024-09-21T14:58:45.050 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2414e-05 2024-09-21T14:58:45.050 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.050 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.050 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:45.050 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.187414+0000", 2024-09-21T14:58:45.051 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012614399999999999 2024-09-21T14:58:45.051 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.051 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.051 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:45.051 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.187539+0000", 2024-09-21T14:58:45.051 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012481800000000001 2024-09-21T14:58:45.051 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:45.051 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:45.051 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:45.051 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.051 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.051 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9410 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:45.051 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.174103+0000", 2024-09-21T14:58:45.051 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 5.7985247380000002, 2024-09-21T14:58:45.052 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025493703, 2024-09-21T14:58:45.052 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:45.052 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:45.052 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:45.052 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:45.052 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:45.052 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9410 2024-09-21T14:58:45.052 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.052 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:45.052 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.052 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:45.052 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.174103+0000", 2024-09-21T14:58:45.052 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:45.052 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.052 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.053 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:45.053 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.174103+0000", 2024-09-21T14:58:45.053 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:45.053 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.053 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.053 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:45.053 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.174102+0000", 2024-09-21T14:58:45.053 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:58:45.053 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.053 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.053 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:45.053 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.174108+0000", 2024-09-21T14:58:45.053 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1149999999999999e-06 2024-09-21T14:58:45.053 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.054 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.054 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:45.054 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.174117+0000", 2024-09-21T14:58:45.054 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4870000000000002e-06 2024-09-21T14:58:45.054 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.054 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.054 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:45.054 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.174122+0000", 2024-09-21T14:58:45.054 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9790000000000001e-06 2024-09-21T14:58:45.054 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.054 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.054 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:45.054 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.187830+0000", 2024-09-21T14:58:45.055 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013708048 2024-09-21T14:58:45.055 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.055 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.055 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:45.055 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.187913+0000", 2024-09-21T14:58:45.055 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2763999999999996e-05 2024-09-21T14:58:45.055 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.055 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.055 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:58:45.055 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.188230+0000", 2024-09-21T14:58:45.055 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00031745100000000001 2024-09-21T14:58:45.055 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.055 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.055 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:45.056 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.192708+0000", 2024-09-21T14:58:45.056 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0044783890000000002 2024-09-21T14:58:45.056 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.056 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.056 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:45.056 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199487+0000", 2024-09-21T14:58:45.056 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0067782149999999998 2024-09-21T14:58:45.056 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.056 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.056 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:45.056 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199560+0000", 2024-09-21T14:58:45.056 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3752999999999997e-05 2024-09-21T14:58:45.056 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.056 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.056 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:45.057 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199597+0000", 2024-09-21T14:58:45.057 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6526999999999998e-05 2024-09-21T14:58:45.057 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:45.057 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:45.057 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:45.057 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.057 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.057 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9411 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:45.057 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.175943+0000", 2024-09-21T14:58:45.057 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 5.7966847189999999, 2024-09-21T14:58:45.057 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023773181000000001, 2024-09-21T14:58:45.057 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:45.057 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:45.057 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:45.057 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:45.057 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:45.058 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9411 2024-09-21T14:58:45.058 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.058 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:45.058 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.058 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:45.058 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175943+0000", 2024-09-21T14:58:45.058 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:45.058 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.058 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.058 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:45.058 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175943+0000", 2024-09-21T14:58:45.058 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:45.058 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.058 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.058 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:45.059 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175942+0000", 2024-09-21T14:58:45.059 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:58:45.059 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.059 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.059 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:45.059 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175948+0000", 2024-09-21T14:58:45.059 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9689999999999999e-06 2024-09-21T14:58:45.059 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.059 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.059 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:45.059 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175960+0000", 2024-09-21T14:58:45.059 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1745e-05 2024-09-21T14:58:45.059 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.060 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.060 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:45.060 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175965+0000", 2024-09-21T14:58:45.060 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.203e-06 2024-09-21T14:58:45.060 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.060 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.060 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:45.060 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.189900+0000", 2024-09-21T14:58:45.060 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013934940999999999 2024-09-21T14:58:45.060 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.060 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.060 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:45.060 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.189988+0000", 2024-09-21T14:58:45.060 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8419999999999997e-05 2024-09-21T14:58:45.060 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.061 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.061 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:58:45.061 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.190321+0000", 2024-09-21T14:58:45.061 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00033231800000000001 2024-09-21T14:58:45.061 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.061 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.061 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:45.061 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.192746+0000", 2024-09-21T14:58:45.061 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0024254400000000001 2024-09-21T14:58:45.061 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.061 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.061 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:45.061 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199605+0000", 2024-09-21T14:58:45.061 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0068582809999999999 2024-09-21T14:58:45.062 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.062 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.062 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:45.062 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199680+0000", 2024-09-21T14:58:45.062 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5630000000000006e-05 2024-09-21T14:58:45.062 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.062 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.062 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:45.062 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199716+0000", 2024-09-21T14:58:45.062 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6248999999999998e-05 2024-09-21T14:58:45.062 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:45.062 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:45.062 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:45.062 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.063 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.063 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9412 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:45.063 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.177402+0000", 2024-09-21T14:58:45.063 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 5.7952256220000002, 2024-09-21T14:58:45.063 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022445590000000001, 2024-09-21T14:58:45.063 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:45.063 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:45.063 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:45.063 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:45.063 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:45.063 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9412 2024-09-21T14:58:45.063 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.063 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:45.064 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.064 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:45.064 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177402+0000", 2024-09-21T14:58:45.064 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:45.064 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.064 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.064 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:45.064 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177402+0000", 2024-09-21T14:58:45.064 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:45.064 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.064 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.064 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:45.064 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177401+0000", 2024-09-21T14:58:45.064 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:58:45.065 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.065 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.065 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:45.065 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177409+0000", 2024-09-21T14:58:45.065 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4710000000000001e-06 2024-09-21T14:58:45.065 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.065 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.065 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:45.065 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177419+0000", 2024-09-21T14:58:45.065 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0351e-05 2024-09-21T14:58:45.065 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.065 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.065 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:45.065 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177424+0000", 2024-09-21T14:58:45.065 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1919999999999996e-06 2024-09-21T14:58:45.066 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.066 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.066 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:45.066 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.191343+0000", 2024-09-21T14:58:45.066 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.01391878 2024-09-21T14:58:45.066 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.066 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.066 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:45.066 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.191446+0000", 2024-09-21T14:58:45.066 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000102453 2024-09-21T14:58:45.066 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.066 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.066 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:58:45.066 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.192200+0000", 2024-09-21T14:58:45.067 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00075476899999999997 2024-09-21T14:58:45.067 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.067 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.067 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:45.067 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.193866+0000", 2024-09-21T14:58:45.067 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0016657519999999999 2024-09-21T14:58:45.067 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.067 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.067 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:45.067 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199724+0000", 2024-09-21T14:58:45.067 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0058579529999999999 2024-09-21T14:58:45.067 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.067 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.068 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:45.068 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199803+0000", 2024-09-21T14:58:45.068 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8662e-05 2024-09-21T14:58:45.068 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.068 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.068 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:45.068 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199848+0000", 2024-09-21T14:58:45.068 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5216000000000002e-05 2024-09-21T14:58:45.068 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:45.068 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:45.068 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:45.068 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.068 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.068 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:10445 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:45.068 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.271980+0000", 2024-09-21T14:58:45.069 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 5.7006482570000001, 2024-09-21T14:58:45.069 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018664256000000001, 2024-09-21T14:58:45.069 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:45.069 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:45.069 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:45.069 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:45.069 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:45.069 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 10445 2024-09-21T14:58:45.069 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.069 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:45.069 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.069 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:45.069 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.271980+0000", 2024-09-21T14:58:45.069 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:45.070 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.070 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.070 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:45.070 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.271980+0000", 2024-09-21T14:58:45.070 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:45.070 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.070 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.070 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:45.070 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.271979+0000", 2024-09-21T14:58:45.070 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:45.070 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.070 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.070 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:45.070 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.271982+0000", 2024-09-21T14:58:45.071 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7240000000000001e-06 2024-09-21T14:58:45.071 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.071 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.071 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:45.071 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.271987+0000", 2024-09-21T14:58:45.071 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3639999999999998e-06 2024-09-21T14:58:45.071 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.071 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.071 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:45.071 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.271992+0000", 2024-09-21T14:58:45.071 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7110000000000003e-06 2024-09-21T14:58:45.071 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.071 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.071 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:45.072 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.277567+0000", 2024-09-21T14:58:45.072 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0055745009999999999 2024-09-21T14:58:45.072 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.072 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.072 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:45.072 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.277655+0000", 2024-09-21T14:58:45.072 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8465000000000001e-05 2024-09-21T14:58:45.072 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.072 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.072 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:58:45.072 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.278720+0000", 2024-09-21T14:58:45.072 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0010650569999999999 2024-09-21T14:58:45.072 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.072 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.073 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:45.073 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.281153+0000", 2024-09-21T14:58:45.073 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0024326389999999999 2024-09-21T14:58:45.073 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.073 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.073 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:45.073 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.290574+0000", 2024-09-21T14:58:45.073 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0094209489999999996 2024-09-21T14:58:45.073 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.073 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.073 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:45.073 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.290613+0000", 2024-09-21T14:58:45.073 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.8803000000000001e-05 2024-09-21T14:58:45.073 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.073 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.073 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:45.074 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.290644+0000", 2024-09-21T14:58:45.074 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1404e-05 2024-09-21T14:58:45.074 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:45.074 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:45.074 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:45.074 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.074 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.074 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:10446 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:45.074 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.272708+0000", 2024-09-21T14:58:45.074 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 5.6999203920000001, 2024-09-21T14:58:45.074 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018025565, 2024-09-21T14:58:45.074 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:45.074 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:45.074 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:45.074 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:45.075 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:45.075 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 10446 2024-09-21T14:58:45.075 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.075 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:45.075 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.075 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:45.075 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.272708+0000", 2024-09-21T14:58:45.075 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:45.075 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.075 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.075 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:45.075 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.272708+0000", 2024-09-21T14:58:45.075 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:45.075 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.076 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.076 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:45.076 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.272706+0000", 2024-09-21T14:58:45.076 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:58:45.076 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.076 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.076 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:45.076 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.272714+0000", 2024-09-21T14:58:45.076 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2089999999999999e-06 2024-09-21T14:58:45.076 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.076 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.076 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:45.076 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.272726+0000", 2024-09-21T14:58:45.076 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1993e-05 2024-09-21T14:58:45.076 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.077 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.077 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:45.077 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.272736+0000", 2024-09-21T14:58:45.077 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0467e-05 2024-09-21T14:58:45.077 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.077 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.077 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:45.077 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.279120+0000", 2024-09-21T14:58:45.077 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0063839190000000001 2024-09-21T14:58:45.077 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.077 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.077 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:45.077 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.279203+0000", 2024-09-21T14:58:45.077 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3097999999999996e-05 2024-09-21T14:58:45.077 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.078 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.078 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:58:45.078 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.279607+0000", 2024-09-21T14:58:45.078 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00040362699999999999 2024-09-21T14:58:45.078 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.078 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.078 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:45.078 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.281972+0000", 2024-09-21T14:58:45.078 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0023649130000000002 2024-09-21T14:58:45.078 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.078 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.078 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:45.078 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.290656+0000", 2024-09-21T14:58:45.078 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0086847480000000008 2024-09-21T14:58:45.079 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.079 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.079 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:45.079 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.290696+0000", 2024-09-21T14:58:45.079 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.9929999999999999e-05 2024-09-21T14:58:45.079 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.079 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.079 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:45.079 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.290733+0000", 2024-09-21T14:58:45.079 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6779999999999997e-05 2024-09-21T14:58:45.079 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:45.079 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:45.079 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:45.079 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:45.079 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:45.080 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:58:45.080 INFO:teuthology.orchestra.run.smithi032.stdout:625 punch from 0x196df3f to 0x19755a2, (0x7663 bytes) 2024-09-21T14:58:45.080 INFO:teuthology.orchestra.run.smithi032.stdout:626 read 0x8c0d9c0 thru 0x8c1d7d5 (0xfe16 bytes) 2024-09-21T14:58:45.081 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T14:58:45.082 INFO:teuthology.orchestra.run.smithi032.stdout:627 write 0xc8026c3 thru 0xc806a35 (0x4373 bytes) 2024-09-21T14:58:45.082 INFO:teuthology.orchestra.run.smithi032.stdout:628 punch from 0x1457ec5 to 0x1460ed2, (0x900d bytes) 2024-09-21T14:58:45.083 INFO:teuthology.orchestra.run.smithi032.stdout:629 read 0x567694d thru 0x567e791 (0x7e45 bytes) 2024-09-21T14:58:45.083 INFO:teuthology.orchestra.run.smithi032.stdout:630 writesame 0x2c018b4 thru 0x2c0b433 data_size 0x40(0x9b80 bytes) 2024-09-21T14:58:45.083 INFO:teuthology.orchestra.run.smithi032.stdout:631 write 0x71ca589 thru 0x71d92e9 (0xed61 bytes) 2024-09-21T14:58:45.083 INFO:teuthology.orchestra.run.smithi032.stdout:632 read 0x84df9a5 thru 0x84e3e7a (0x44d6 bytes) 2024-09-21T14:58:45.083 INFO:teuthology.orchestra.run.smithi032.stdout:633 write 0x37ecb0e thru 0x37f2e43 (0x6336 bytes) 2024-09-21T14:58:45.083 INFO:teuthology.orchestra.run.smithi032.stdout:634 read 0x9c70741 thru 0x9c76e6e (0x672e bytes) 2024-09-21T14:58:45.083 INFO:teuthology.orchestra.run.smithi032.stdout:635 writesame 0xee5006f thru 0xee562ab data_size 0x53(0x623d bytes) 2024-09-21T14:58:45.083 INFO:teuthology.orchestra.run.smithi032.stdout:637 trunc from 0xee562ac to 0x57b8272 2024-09-21T14:58:45.083 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:58:45.076+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:58:45.120 INFO:teuthology.orchestra.run.smithi032.stdout:639 writesame 0xd0c542c thru 0xd0c9065 data_size 0xd(0x3c3a bytes) 2024-09-21T14:58:45.126 INFO:teuthology.orchestra.run.smithi032.stdout:640 trunc from 0xd0c9066 to 0x9fa9e5c 2024-09-21T14:58:45.128 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T14:58:45.128 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T14:58:45.129 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-21T14:58:45.129 INFO:tasks.thrashosds.thrasher:Setting osd 6 primary_affinity to 0.050137 2024-09-21T14:58:45.129 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 0.05013688398801752 2024-09-21T14:58:45.137 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:45.137 INFO:teuthology.orchestra.run.smithi032.stdout:642 writesame 0x56558e4 thru 0x5657b9c data_size 0x1(0x22b9 bytes) 2024-09-21T14:58:45.148 INFO:teuthology.orchestra.run.smithi032.stdout:643 punch from 0x6c4abbe to 0x6c4bd3c, (0x117e bytes) 2024-09-21T14:58:45.149 INFO:teuthology.orchestra.run.smithi032.stdout:644 write 0x64f6dc1 thru 0x6506cf2 (0xff32 bytes) 2024-09-21T14:58:45.177 INFO:teuthology.orchestra.run.smithi032.stdout:645 trunc from 0x9fa9e5c to 0xd07aa9 2024-09-21T14:58:45.181 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:58:45.180+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:58:45.224 INFO:teuthology.orchestra.run.smithi032.stdout:646 write 0x4e8038d thru 0x4e8c1cc (0xbe40 bytes) 2024-09-21T14:58:45.234 INFO:teuthology.orchestra.run.smithi032.stdout:648 write 0x4d934d0 thru 0x4d9a6c5 (0x71f6 bytes) 2024-09-21T14:58:45.238 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:58:45.238 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T14:58:45.238 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.238 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:20526 3.1a 3:5b69faf7:::rbd_data.1136c84d9a13.0000000000000136:head [set-alloc-hint object_size 262144 write_size 262144,write 46288~19248 in=19248b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-21T14:58:45.238 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:45.236810+0000", 2024-09-21T14:58:45.238 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.001208881, 2024-09-21T14:58:45.239 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001267998, 2024-09-21T14:58:45.239 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:45.239 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "waiting for sub ops", 2024-09-21T14:58:45.239 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:45.239 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:45.239 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:45.239 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 20526 2024-09-21T14:58:45.239 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.239 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:45.239 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.239 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:45.239 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:45.236810+0000", 2024-09-21T14:58:45.239 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:45.239 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.239 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.240 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:45.240 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:45.236810+0000", 2024-09-21T14:58:45.240 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:45.240 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.240 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.240 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:45.240 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:45.236809+0000", 2024-09-21T14:58:45.240 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:45.240 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.240 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.240 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:45.240 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:45.236827+0000", 2024-09-21T14:58:45.240 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8445999999999999e-05 2024-09-21T14:58:45.240 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.240 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.240 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:45.241 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:45.236848+0000", 2024-09-21T14:58:45.241 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0990000000000001e-05 2024-09-21T14:58:45.241 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.241 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.241 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:45.241 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:45.236858+0000", 2024-09-21T14:58:45.241 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.01e-05 2024-09-21T14:58:45.241 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.241 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.241 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:45.241 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:45.236931+0000", 2024-09-21T14:58:45.241 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2432e-05 2024-09-21T14:58:45.241 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.241 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.241 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:45.242 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:45.237186+0000", 2024-09-21T14:58:45.242 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00025521799999999998 2024-09-21T14:58:45.242 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.242 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.242 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:45.242 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:45.237481+0000", 2024-09-21T14:58:45.242 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000295161 2024-09-21T14:58:45.242 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:45.242 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:45.242 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:45.242 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:45.242 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-21T14:58:45.242 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 1 2024-09-21T14:58:45.242 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:58:45.243 INFO:teuthology.orchestra.run.smithi032.stdout:650 write 0x36770d6 thru 0x367bd95 (0x4cc0 bytes) 2024-09-21T14:58:45.246 INFO:teuthology.orchestra.run.smithi032.stdout:651 read 0x147842c thru 0x147f40d (0x6fe2 bytes) 2024-09-21T14:58:45.246 INFO:teuthology.orchestra.run.smithi032.stdout:653 write 0xc61e7c thru 0xc6a07a (0x81ff bytes) 2024-09-21T14:58:45.249 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T14:58:45.266 INFO:teuthology.orchestra.run.smithi032.stdout:654 write 0x9d0880b thru 0x9d0df8a (0x5780 bytes) 2024-09-21T14:58:45.271 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:45.271 INFO:teuthology.orchestra.run.smithi032.stdout:656 read 0x360147a thru 0x36048a1 (0x3428 bytes) 2024-09-21T14:58:45.282 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:58:45.280+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:58:45.290 INFO:teuthology.orchestra.run.smithi032.stdout:657 writesame 0xdd8a03a thru 0xdd92543 data_size 0x1(0x850a bytes) 2024-09-21T14:58:45.290 INFO:teuthology.orchestra.run.smithi032.stdout:659 read 0x526538 thru 0x526a49 (0x512 bytes) 2024-09-21T14:58:45.306 INFO:teuthology.orchestra.run.smithi032.stdout:660 trunc from 0xdd92544 to 0x5177cf1 2024-09-21T14:58:45.350 INFO:teuthology.orchestra.run.smithi032.stdout:661 write 0x89f149d thru 0x89f709d (0x5c01 bytes) 2024-09-21T14:58:45.379 INFO:teuthology.orchestra.run.smithi032.stdout:663 write 0xd56f2f8 thru 0xd578dcb (0x9ad4 bytes) 2024-09-21T14:58:45.382 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:58:45.380+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:58:45.386 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:58:45.387 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:58:45.387 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T14:58:45.387 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T14:58:45.387 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:58:45.388 INFO:teuthology.orchestra.run.smithi032.stdout:664 read 0x3cea4d7 thru 0x3cf430c (0x9e36 bytes) 2024-09-21T14:58:45.388 INFO:teuthology.orchestra.run.smithi032.stdout:665 write 0x96a59df thru 0x96a9b2f (0x4151 bytes) 2024-09-21T14:58:45.397 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T14:58:45.397 INFO:teuthology.orchestra.run.smithi032.stdout:667 read 0xb819fb6 thru 0xb81e0d5 (0x4120 bytes) 2024-09-21T14:58:45.397 INFO:teuthology.orchestra.run.smithi032.stdout:668 writesame 0x355b55f thru 0x35654f7 data_size 0x1(0x9f99 bytes) 2024-09-21T14:58:45.420 INFO:teuthology.orchestra.run.smithi032.stdout:669 read 0x215f237 thru 0x21673e2 (0x81ac bytes) 2024-09-21T14:58:45.420 INFO:teuthology.orchestra.run.smithi032.stdout:670 read 0x97cd46 thru 0x984d2d (0x7fe8 bytes) 2024-09-21T14:58:45.421 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:45.421 INFO:teuthology.orchestra.run.smithi032.stdout:675 write 0x2f0b8fd thru 0x2f1a939 (0xf03d bytes) 2024-09-21T14:58:45.430 INFO:teuthology.orchestra.run.smithi032.stdout:677 read 0x3a8ae11 thru 0x3a98abe (0xdcae bytes) 2024-09-21T14:58:45.430 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:45.430 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:45.430 INFO:teuthology.orchestra.run.smithi032.stdout:686 write 0xa0cd977 thru 0xa0d68c2 (0x8f4c bytes) 2024-09-21T14:58:45.438 INFO:teuthology.orchestra.run.smithi032.stdout:687 writesame 0x64134cd thru 0x64143b4 data_size 0x24(0xee8 bytes) 2024-09-21T14:58:45.444 INFO:teuthology.orchestra.run.smithi032.stdout:688 writesame 0x835eb48 thru 0x83605e6 data_size 0x5(0x1a9f bytes) 2024-09-21T14:58:45.464 INFO:teuthology.orchestra.run.smithi032.stdout:690 writesame 0xda55530 thru 0xda6076d data_size 0xa9(0xb23e bytes) 2024-09-21T14:58:45.464 INFO:teuthology.orchestra.run.smithi032.stdout:691 punch from 0xc03df60 to 0xc044a05, (0x6aa5 bytes) 2024-09-21T14:58:45.464 INFO:teuthology.orchestra.run.smithi032.stdout:692 read 0x6c7072a thru 0x6c78d4d (0x8624 bytes) 2024-09-21T14:58:45.464 INFO:teuthology.orchestra.run.smithi032.stdout:694 read 0xcdf305a thru 0xcdfb27e (0x8225 bytes) 2024-09-21T14:58:45.464 INFO:teuthology.orchestra.run.smithi032.stdout:695 read 0x433858 thru 0x43bd5f (0x8508 bytes) 2024-09-21T14:58:45.465 INFO:teuthology.orchestra.run.smithi032.stdout:696 write 0xb1b697 thru 0xb2988f (0xe1f9 bytes) 2024-09-21T14:58:45.470 INFO:teuthology.orchestra.run.smithi032.stdout:697 read 0x88fc096 thru 0x8908a7d (0xc9e8 bytes) 2024-09-21T14:58:45.470 INFO:teuthology.orchestra.run.smithi032.stdout:698 punch from 0x1b1df8c to 0x1b244df, (0x6553 bytes) 2024-09-21T14:58:45.483 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:58:45.480+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:58:45.491 INFO:teuthology.orchestra.run.smithi032.stdout:699 trunc from 0xda6076e to 0x13f5732 2024-09-21T14:58:45.522 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:58:45.522 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T14:58:45.522 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T14:58:45.522 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T14:58:45.522 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.523 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T14:58:45.523 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.205754+0000", 2024-09-21T14:58:45.523 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 42.314740618000002, 2024-09-21T14:58:45.523 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.41117094199999998, 2024-09-21T14:58:45.523 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:45.523 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:45.523 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:45.523 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:45.523 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:45.523 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1 2024-09-21T14:58:45.523 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.523 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:45.523 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.523 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:45.524 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205754+0000", 2024-09-21T14:58:45.524 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:45.524 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.524 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.524 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:45.524 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205754+0000", 2024-09-21T14:58:45.524 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:45.524 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.524 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.524 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:45.524 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205752+0000", 2024-09-21T14:58:45.524 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T14:58:45.524 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.525 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.525 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:45.525 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205765+0000", 2024-09-21T14:58:45.525 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3230999999999999e-05 2024-09-21T14:58:45.525 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.525 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.525 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:45.525 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205782+0000", 2024-09-21T14:58:45.525 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6634e-05 2024-09-21T14:58:45.525 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.525 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.525 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:45.525 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205793+0000", 2024-09-21T14:58:45.525 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1297e-05 2024-09-21T14:58:45.525 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.526 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.526 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:45.526 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205867+0000", 2024-09-21T14:58:45.526 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4269000000000003e-05 2024-09-21T14:58:45.526 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.526 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.526 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T14:58:45.526 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205877+0000", 2024-09-21T14:58:45.526 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2690000000000005e-06 2024-09-21T14:58:45.526 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.526 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.526 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:45.526 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.613700+0000", 2024-09-21T14:58:45.527 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.407823455 2024-09-21T14:58:45.527 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.527 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.527 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:45.527 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.614203+0000", 2024-09-21T14:58:45.527 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00050311500000000005 2024-09-21T14:58:45.527 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.527 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.527 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:45.527 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.614750+0000", 2024-09-21T14:58:45.527 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00054738699999999998 2024-09-21T14:58:45.527 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.527 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.527 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:45.528 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616367+0000", 2024-09-21T14:58:45.528 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001616634 2024-09-21T14:58:45.528 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.528 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.528 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:45.528 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616574+0000", 2024-09-21T14:58:45.528 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00020668499999999999 2024-09-21T14:58:45.528 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.528 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.528 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:45.528 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616802+0000", 2024-09-21T14:58:45.528 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00022817900000000001 2024-09-21T14:58:45.528 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.528 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.529 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:45.529 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616925+0000", 2024-09-21T14:58:45.529 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012309699999999999 2024-09-21T14:58:45.529 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:45.529 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:45.529 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:45.529 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.529 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.529 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T14:58:45.529 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.955142+0000", 2024-09-21T14:58:45.529 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 41.565352971000003, 2024-09-21T14:58:45.529 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012388923, 2024-09-21T14:58:45.529 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:45.529 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:45.530 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:45.530 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:45.530 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:45.530 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 113 2024-09-21T14:58:45.530 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.530 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:45.530 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.530 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:45.530 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955142+0000", 2024-09-21T14:58:45.530 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:45.530 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.530 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.530 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:45.531 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955142+0000", 2024-09-21T14:58:45.531 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:45.531 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.531 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.531 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:45.531 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955141+0000", 2024-09-21T14:58:45.531 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:45.531 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.531 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.531 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:45.531 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955145+0000", 2024-09-21T14:58:45.531 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2109999999999998e-06 2024-09-21T14:58:45.531 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.531 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.531 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:45.532 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955150+0000", 2024-09-21T14:58:45.532 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4090000000000002e-06 2024-09-21T14:58:45.532 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.532 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.532 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:45.532 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955153+0000", 2024-09-21T14:58:45.532 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1240000000000001e-06 2024-09-21T14:58:45.532 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.532 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.532 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:45.532 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.963026+0000", 2024-09-21T14:58:45.532 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0078724060000000002 2024-09-21T14:58:45.532 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.532 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.532 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:45.532 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.963126+0000", 2024-09-21T14:58:45.533 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000100206 2024-09-21T14:58:45.533 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.533 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.533 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:45.533 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.963277+0000", 2024-09-21T14:58:45.533 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015159200000000001 2024-09-21T14:58:45.533 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.533 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.533 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:45.533 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.965519+0000", 2024-09-21T14:58:45.533 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0022418440000000002 2024-09-21T14:58:45.533 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.533 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.533 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:45.534 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967305+0000", 2024-09-21T14:58:45.534 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001785525 2024-09-21T14:58:45.534 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.534 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.534 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:45.534 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967438+0000", 2024-09-21T14:58:45.534 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013284399999999999 2024-09-21T14:58:45.534 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.534 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.534 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:45.534 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967531+0000", 2024-09-21T14:58:45.534 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.3140000000000006e-05 2024-09-21T14:58:45.534 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:45.535 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:45.535 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:45.535 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.535 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.535 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T14:58:45.535 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.955387+0000", 2024-09-21T14:58:45.535 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 41.565108037000002, 2024-09-21T14:58:45.535 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012319478999999999, 2024-09-21T14:58:45.535 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:45.535 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:45.535 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:45.535 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:45.535 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:45.535 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 114 2024-09-21T14:58:45.535 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.536 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:45.536 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.536 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:45.536 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955387+0000", 2024-09-21T14:58:45.536 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:45.536 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.536 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.536 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:45.536 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955387+0000", 2024-09-21T14:58:45.536 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:45.536 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.536 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.536 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:45.536 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955386+0000", 2024-09-21T14:58:45.536 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:58:45.536 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.537 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.537 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:45.537 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955392+0000", 2024-09-21T14:58:45.537 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4740000000000002e-06 2024-09-21T14:58:45.537 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.537 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.537 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:45.537 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955400+0000", 2024-09-21T14:58:45.537 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2139999999999996e-06 2024-09-21T14:58:45.537 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.537 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.537 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:45.537 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955407+0000", 2024-09-21T14:58:45.537 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2299999999999996e-06 2024-09-21T14:58:45.537 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.538 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.538 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:45.538 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.964840+0000", 2024-09-21T14:58:45.538 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0094330079999999997 2024-09-21T14:58:45.538 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.538 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.538 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:45.538 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.964917+0000", 2024-09-21T14:58:45.538 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7071999999999997e-05 2024-09-21T14:58:45.538 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.538 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.538 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:45.538 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.965026+0000", 2024-09-21T14:58:45.538 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000109646 2024-09-21T14:58:45.538 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.538 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.539 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:45.539 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967609+0000", 2024-09-21T14:58:45.539 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0025824929999999999 2024-09-21T14:58:45.539 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.539 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.539 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:45.539 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967629+0000", 2024-09-21T14:58:45.539 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0486999999999999e-05 2024-09-21T14:58:45.539 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.539 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.539 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:45.539 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967672+0000", 2024-09-21T14:58:45.539 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2314999999999997e-05 2024-09-21T14:58:45.539 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.540 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.540 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:45.540 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967706+0000", 2024-09-21T14:58:45.540 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4483999999999997e-05 2024-09-21T14:58:45.540 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:45.540 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:45.540 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:45.540 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.540 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.540 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T14:58:45.540 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.955609+0000", 2024-09-21T14:58:45.540 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 41.564886231000003, 2024-09-21T14:58:45.540 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013624153999999999, 2024-09-21T14:58:45.540 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:45.540 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:45.540 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:45.541 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:45.541 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:45.541 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 115 2024-09-21T14:58:45.541 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.541 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:45.541 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.541 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:45.541 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955609+0000", 2024-09-21T14:58:45.541 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:45.541 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.541 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.541 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:45.541 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955609+0000", 2024-09-21T14:58:45.541 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:45.541 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.541 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.542 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:45.542 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955607+0000", 2024-09-21T14:58:45.542 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-21T14:58:45.542 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.542 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.542 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:45.542 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955732+0000", 2024-09-21T14:58:45.542 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000125566 2024-09-21T14:58:45.542 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.542 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.542 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:45.542 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955752+0000", 2024-09-21T14:58:45.542 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9916999999999999e-05 2024-09-21T14:58:45.542 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.543 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.543 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:45.543 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955767+0000", 2024-09-21T14:58:45.543 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4687e-05 2024-09-21T14:58:45.543 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.543 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.543 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:45.543 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.965574+0000", 2024-09-21T14:58:45.543 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0098072850000000007 2024-09-21T14:58:45.543 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.543 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.543 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:45.543 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.965642+0000", 2024-09-21T14:58:45.543 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7784e-05 2024-09-21T14:58:45.543 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.544 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.544 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:45.544 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.965753+0000", 2024-09-21T14:58:45.544 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000111111 2024-09-21T14:58:45.544 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.544 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.544 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:45.544 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967717+0000", 2024-09-21T14:58:45.544 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0019635339999999999 2024-09-21T14:58:45.544 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.544 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.544 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:45.544 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.969014+0000", 2024-09-21T14:58:45.544 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001297093 2024-09-21T14:58:45.544 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.544 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.545 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:45.545 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.969130+0000", 2024-09-21T14:58:45.545 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000116011 2024-09-21T14:58:45.545 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.545 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.545 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:45.545 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.969233+0000", 2024-09-21T14:58:45.545 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000102982 2024-09-21T14:58:45.545 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:45.545 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:45.545 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:45.545 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.545 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.545 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T14:58:45.545 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.955783+0000", 2024-09-21T14:58:45.545 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 41.564711561999999, 2024-09-21T14:58:45.546 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014630241, 2024-09-21T14:58:45.546 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:45.546 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:45.546 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:45.546 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:45.546 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:45.546 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 116 2024-09-21T14:58:45.546 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.546 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:45.546 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.546 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:45.546 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955783+0000", 2024-09-21T14:58:45.546 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:45.546 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.547 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.547 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:45.547 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955783+0000", 2024-09-21T14:58:45.547 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:45.547 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.547 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.547 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:45.547 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955782+0000", 2024-09-21T14:58:45.547 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:58:45.547 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.547 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.547 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:45.547 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955791+0000", 2024-09-21T14:58:45.547 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8089999999999999e-06 2024-09-21T14:58:45.547 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.547 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.548 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:45.548 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955804+0000", 2024-09-21T14:58:45.548 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2548000000000001e-05 2024-09-21T14:58:45.548 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.548 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.548 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:45.548 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955814+0000", 2024-09-21T14:58:45.548 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0158e-05 2024-09-21T14:58:45.548 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.548 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.548 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:45.548 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.966515+0000", 2024-09-21T14:58:45.548 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.010701756 2024-09-21T14:58:45.548 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.548 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.549 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:45.549 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.966606+0000", 2024-09-21T14:58:45.549 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0761999999999996e-05 2024-09-21T14:58:45.549 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.549 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.549 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:45.549 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.966740+0000", 2024-09-21T14:58:45.549 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000133693 2024-09-21T14:58:45.549 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.549 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.549 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:45.549 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.968920+0000", 2024-09-21T14:58:45.549 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0021799290000000002 2024-09-21T14:58:45.549 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.549 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.549 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:45.549 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.970320+0000", 2024-09-21T14:58:45.550 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0014001759999999999 2024-09-21T14:58:45.550 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.550 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.550 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:45.550 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.970376+0000", 2024-09-21T14:58:45.550 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5696999999999999e-05 2024-09-21T14:58:45.550 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.550 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.550 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:45.550 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.970413+0000", 2024-09-21T14:58:45.550 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7750999999999998e-05 2024-09-21T14:58:45.550 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:45.550 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:45.550 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:45.550 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.551 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.551 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T14:58:45.551 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.955829+0000", 2024-09-21T14:58:45.551 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 41.564666092000003, 2024-09-21T14:58:45.551 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014919597999999999, 2024-09-21T14:58:45.551 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:45.551 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:45.551 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:45.551 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:45.551 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:45.551 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 117 2024-09-21T14:58:45.551 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.551 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:45.551 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.552 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:45.552 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955829+0000", 2024-09-21T14:58:45.552 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:45.552 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.552 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.552 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:45.552 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955829+0000", 2024-09-21T14:58:45.552 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:45.552 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.552 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.552 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:45.552 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955827+0000", 2024-09-21T14:58:45.552 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:45.552 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.552 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.552 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:45.552 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955838+0000", 2024-09-21T14:58:45.553 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.111e-05 2024-09-21T14:58:45.553 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.553 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.553 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:45.553 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955875+0000", 2024-09-21T14:58:45.553 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6211999999999998e-05 2024-09-21T14:58:45.553 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.553 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.553 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:45.553 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955899+0000", 2024-09-21T14:58:45.553 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4128999999999999e-05 2024-09-21T14:58:45.553 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.553 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.553 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:45.554 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967767+0000", 2024-09-21T14:58:45.554 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.011868251999999999 2024-09-21T14:58:45.554 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.554 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.554 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:45.554 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967866+0000", 2024-09-21T14:58:45.554 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8664000000000003e-05 2024-09-21T14:58:45.554 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.554 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.554 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:45.554 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.968128+0000", 2024-09-21T14:58:45.554 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00026243600000000001 2024-09-21T14:58:45.554 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.554 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.554 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:45.555 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.969393+0000", 2024-09-21T14:58:45.555 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0012650350000000001 2024-09-21T14:58:45.555 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.555 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.555 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:45.555 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.970523+0000", 2024-09-21T14:58:45.555 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001129557 2024-09-21T14:58:45.555 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.555 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.555 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:45.555 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.970603+0000", 2024-09-21T14:58:45.555 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0090000000000001e-05 2024-09-21T14:58:45.555 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.555 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.555 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:45.555 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.970748+0000", 2024-09-21T14:58:45.556 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014552799999999999 2024-09-21T14:58:45.556 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:45.556 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:45.556 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:45.556 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.556 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.556 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T14:58:45.556 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:04.493315+0000", 2024-09-21T14:58:45.556 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 41.027179259, 2024-09-21T14:58:45.556 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014055766000000001, 2024-09-21T14:58:45.556 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:45.556 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:45.556 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:45.556 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:45.556 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:45.557 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 137 2024-09-21T14:58:45.557 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.557 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:45.557 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.557 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:45.557 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493315+0000", 2024-09-21T14:58:45.557 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:45.557 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.557 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.557 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:45.557 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493315+0000", 2024-09-21T14:58:45.557 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:45.557 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.557 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.557 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:45.558 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493314+0000", 2024-09-21T14:58:45.558 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:58:45.558 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.558 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.558 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:45.558 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493507+0000", 2024-09-21T14:58:45.558 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000192444 2024-09-21T14:58:45.558 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.558 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.558 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:45.558 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493528+0000", 2024-09-21T14:58:45.558 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0845000000000001e-05 2024-09-21T14:58:45.558 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.559 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.559 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:45.559 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493543+0000", 2024-09-21T14:58:45.559 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5800000000000001e-05 2024-09-21T14:58:45.559 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.559 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.559 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:45.559 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.502575+0000", 2024-09-21T14:58:45.559 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0090312389999999999 2024-09-21T14:58:45.559 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.559 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.559 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:45.559 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.502677+0000", 2024-09-21T14:58:45.560 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000102271 2024-09-21T14:58:45.560 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.560 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.560 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:45.560 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.502825+0000", 2024-09-21T14:58:45.560 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000148171 2024-09-21T14:58:45.560 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.560 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.560 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:45.560 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507251+0000", 2024-09-21T14:58:45.560 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0044253879999999997 2024-09-21T14:58:45.560 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.560 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.560 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:45.561 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507313+0000", 2024-09-21T14:58:45.561 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2170999999999998e-05 2024-09-21T14:58:45.561 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.561 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.561 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:45.561 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507346+0000", 2024-09-21T14:58:45.561 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3079e-05 2024-09-21T14:58:45.561 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.561 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.561 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:45.561 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507371+0000", 2024-09-21T14:58:45.561 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5477999999999999e-05 2024-09-21T14:58:45.561 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:45.561 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:45.561 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:45.562 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.562 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.562 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T14:58:45.562 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:04.493570+0000", 2024-09-21T14:58:45.562 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 41.026924854000001, 2024-09-21T14:58:45.562 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014061686, 2024-09-21T14:58:45.562 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:45.562 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:45.562 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:45.562 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:45.562 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:45.562 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 138 2024-09-21T14:58:45.562 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.563 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:45.563 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.563 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:45.563 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493570+0000", 2024-09-21T14:58:45.563 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:45.563 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.563 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.563 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:45.563 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493570+0000", 2024-09-21T14:58:45.563 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:45.563 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.563 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.563 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:45.563 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493568+0000", 2024-09-21T14:58:45.564 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:45.564 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.564 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.564 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:45.564 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493579+0000", 2024-09-21T14:58:45.564 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0574000000000001e-05 2024-09-21T14:58:45.564 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.564 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.564 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:45.564 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493595+0000", 2024-09-21T14:58:45.564 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6504999999999999e-05 2024-09-21T14:58:45.564 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.564 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.564 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:45.565 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493608+0000", 2024-09-21T14:58:45.565 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2836999999999999e-05 2024-09-21T14:58:45.565 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.565 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.565 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:45.565 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.503909+0000", 2024-09-21T14:58:45.565 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.01030118 2024-09-21T14:58:45.565 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.565 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.565 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:45.565 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.504064+0000", 2024-09-21T14:58:45.565 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015493000000000001 2024-09-21T14:58:45.565 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.565 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.566 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:45.566 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.504299+0000", 2024-09-21T14:58:45.566 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00023416100000000001 2024-09-21T14:58:45.566 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.566 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.566 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:45.566 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507466+0000", 2024-09-21T14:58:45.566 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0031678380000000001 2024-09-21T14:58:45.566 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.566 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.567 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:45.567 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507483+0000", 2024-09-21T14:58:45.567 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6606999999999999e-05 2024-09-21T14:58:45.567 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.567 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.567 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:45.567 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507566+0000", 2024-09-21T14:58:45.567 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3110999999999998e-05 2024-09-21T14:58:45.567 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.567 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.567 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:45.567 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507632+0000", 2024-09-21T14:58:45.567 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5480000000000003e-05 2024-09-21T14:58:45.567 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:45.567 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:45.567 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:45.568 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.568 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.568 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T14:58:45.568 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:04.493625+0000", 2024-09-21T14:58:45.568 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 41.026870166000002, 2024-09-21T14:58:45.568 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014665378999999999, 2024-09-21T14:58:45.568 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:45.568 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:45.568 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:45.568 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:45.568 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:45.568 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139 2024-09-21T14:58:45.568 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.568 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:45.568 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.568 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:45.569 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493625+0000", 2024-09-21T14:58:45.569 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:45.569 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.569 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.569 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:45.569 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493625+0000", 2024-09-21T14:58:45.569 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:45.569 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.569 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.569 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:45.569 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493624+0000", 2024-09-21T14:58:45.569 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:58:45.569 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.569 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.569 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:45.569 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493638+0000", 2024-09-21T14:58:45.570 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4282e-05 2024-09-21T14:58:45.570 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.570 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.570 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:45.570 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493654+0000", 2024-09-21T14:58:45.570 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5739999999999998e-05 2024-09-21T14:58:45.570 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.570 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.570 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:45.570 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493666+0000", 2024-09-21T14:58:45.570 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2865e-05 2024-09-21T14:58:45.570 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.570 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.570 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:45.570 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.505329+0000", 2024-09-21T14:58:45.570 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.011662237000000001 2024-09-21T14:58:45.571 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.571 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.571 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:45.571 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.505377+0000", 2024-09-21T14:58:45.571 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7862999999999999e-05 2024-09-21T14:58:45.571 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.571 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.571 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:45.571 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.505455+0000", 2024-09-21T14:58:45.571 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8918000000000006e-05 2024-09-21T14:58:45.571 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.571 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.571 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:45.571 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507650+0000", 2024-09-21T14:58:45.571 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.002195027 2024-09-21T14:58:45.572 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.572 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.572 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:45.572 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508149+0000", 2024-09-21T14:58:45.572 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00049828399999999999 2024-09-21T14:58:45.572 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.572 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.572 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:45.572 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508230+0000", 2024-09-21T14:58:45.572 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1376000000000002e-05 2024-09-21T14:58:45.572 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.572 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.572 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:45.572 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508290+0000", 2024-09-21T14:58:45.572 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9852999999999998e-05 2024-09-21T14:58:45.572 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:45.573 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:45.573 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:45.573 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.573 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.573 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T14:58:45.573 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:04.493679+0000", 2024-09-21T14:58:45.573 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 41.026815268, 2024-09-21T14:58:45.573 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014869869000000001, 2024-09-21T14:58:45.573 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:45.573 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:45.573 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:45.573 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:45.573 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:45.573 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140 2024-09-21T14:58:45.573 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.573 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:45.574 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.574 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:45.574 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493679+0000", 2024-09-21T14:58:45.574 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:45.574 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.574 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.574 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:45.574 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493679+0000", 2024-09-21T14:58:45.574 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:45.574 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.574 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.574 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:45.574 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493679+0000", 2024-09-21T14:58:45.574 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:58:45.575 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.575 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.575 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:45.575 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493889+0000", 2024-09-21T14:58:45.575 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00021032700000000001 2024-09-21T14:58:45.575 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.575 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.575 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:45.575 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493931+0000", 2024-09-21T14:58:45.575 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1971000000000002e-05 2024-09-21T14:58:45.575 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.575 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.575 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:45.575 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493947+0000", 2024-09-21T14:58:45.576 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6385e-05 2024-09-21T14:58:45.576 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.576 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.576 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:45.576 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.505804+0000", 2024-09-21T14:58:45.576 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.011857136000000001 2024-09-21T14:58:45.576 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.576 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.576 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:45.576 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.505992+0000", 2024-09-21T14:58:45.576 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000187665 2024-09-21T14:58:45.576 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.576 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.576 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:45.576 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.506281+0000", 2024-09-21T14:58:45.576 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000288712 2024-09-21T14:58:45.577 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.577 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.577 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:45.577 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508349+0000", 2024-09-21T14:58:45.577 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0020685790000000001 2024-09-21T14:58:45.577 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.577 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.577 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:45.577 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508371+0000", 2024-09-21T14:58:45.577 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1747e-05 2024-09-21T14:58:45.577 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.577 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.577 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:45.577 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508450+0000", 2024-09-21T14:58:45.577 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9332000000000005e-05 2024-09-21T14:58:45.577 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.578 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.578 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:45.578 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508549+0000", 2024-09-21T14:58:45.578 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8918000000000004e-05 2024-09-21T14:58:45.578 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:45.578 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:45.578 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:45.578 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.578 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.578 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T14:58:45.578 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:04.493967+0000", 2024-09-21T14:58:45.578 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 41.026528231, 2024-09-21T14:58:45.578 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.015281075999999999, 2024-09-21T14:58:45.578 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:45.578 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:45.578 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:45.579 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:45.579 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:45.579 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 141 2024-09-21T14:58:45.579 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.579 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:45.579 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.579 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:45.579 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493967+0000", 2024-09-21T14:58:45.579 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:45.579 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.579 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.579 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:45.579 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493967+0000", 2024-09-21T14:58:45.579 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:45.579 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.580 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.580 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:45.580 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493965+0000", 2024-09-21T14:58:45.580 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:45.580 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.580 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.580 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:45.580 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493987+0000", 2024-09-21T14:58:45.580 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1662999999999999e-05 2024-09-21T14:58:45.580 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.580 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.580 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:45.580 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.494004+0000", 2024-09-21T14:58:45.580 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7311000000000001e-05 2024-09-21T14:58:45.580 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.580 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.581 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:45.581 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.494017+0000", 2024-09-21T14:58:45.581 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3202e-05 2024-09-21T14:58:45.581 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.581 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.581 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:45.581 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507678+0000", 2024-09-21T14:58:45.581 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013660432 2024-09-21T14:58:45.581 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.581 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.581 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:45.581 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507726+0000", 2024-09-21T14:58:45.581 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8154e-05 2024-09-21T14:58:45.581 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.581 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.581 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:45.581 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507797+0000", 2024-09-21T14:58:45.582 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.1110000000000002e-05 2024-09-21T14:58:45.582 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.582 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.582 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:45.582 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508710+0000", 2024-09-21T14:58:45.582 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00091339299999999997 2024-09-21T14:58:45.582 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.582 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.582 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:45.582 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.509108+0000", 2024-09-21T14:58:45.582 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00039718300000000001 2024-09-21T14:58:45.582 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.582 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.582 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:45.582 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.509186+0000", 2024-09-21T14:58:45.583 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8592999999999998e-05 2024-09-21T14:58:45.583 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.583 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.583 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:45.583 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.509248+0000", 2024-09-21T14:58:45.583 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1450000000000003e-05 2024-09-21T14:58:45.583 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:45.583 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:45.583 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:45.583 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.583 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.583 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6626 3.1a 3:5adee3e5:::rbd_data.11361bed1f78.0000000000000001:d [read 229376~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:45.583 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131593+0000", 2024-09-21T14:58:45.584 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.388901653, 2024-09-21T14:58:45.584 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012576725, 2024-09-21T14:58:45.584 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:45.584 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:45.584 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:45.584 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:45.584 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:45.584 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6626 2024-09-21T14:58:45.584 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.584 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:45.584 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.584 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:45.584 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131593+0000", 2024-09-21T14:58:45.585 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:45.585 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.585 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.585 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:45.585 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131593+0000", 2024-09-21T14:58:45.585 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:45.585 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.585 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.585 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:45.585 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131593+0000", 2024-09-21T14:58:45.585 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T14:58:45.585 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.585 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.585 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:45.585 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131598+0000", 2024-09-21T14:58:45.585 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7849999999999999e-06 2024-09-21T14:58:45.586 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.586 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.586 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:45.586 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131601+0000", 2024-09-21T14:58:45.586 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1949999999999999e-06 2024-09-21T14:58:45.586 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.586 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.586 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:45.586 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131604+0000", 2024-09-21T14:58:45.586 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6350000000000002e-06 2024-09-21T14:58:45.586 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.586 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.586 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:45.586 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.143902+0000", 2024-09-21T14:58:45.586 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012298216000000001 2024-09-21T14:58:45.587 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.587 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.587 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:45.587 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.143993+0000", 2024-09-21T14:58:45.587 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1073e-05 2024-09-21T14:58:45.587 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.587 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.587 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:45.587 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.144170+0000", 2024-09-21T14:58:45.587 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017703 2024-09-21T14:58:45.587 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:45.587 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:45.587 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:45.588 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.588 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.588 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6631 3.1a 3:5d547f54:::rbd_data.11361bed1f78.0000000000000003:d [read 0~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:45.588 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131606+0000", 2024-09-21T14:58:45.588 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.388888953, 2024-09-21T14:58:45.588 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013903148000000001, 2024-09-21T14:58:45.588 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:45.588 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:45.588 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:45.588 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:45.588 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:45.588 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6631 2024-09-21T14:58:45.588 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.588 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:45.588 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.588 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:45.589 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131606+0000", 2024-09-21T14:58:45.589 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:45.589 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.589 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.589 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:45.589 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131606+0000", 2024-09-21T14:58:45.589 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:45.589 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.589 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.589 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:45.589 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131606+0000", 2024-09-21T14:58:45.589 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T14:58:45.589 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.589 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.590 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:45.590 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131607+0000", 2024-09-21T14:58:45.590 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2699999999999999e-06 2024-09-21T14:58:45.590 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.590 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.590 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:45.590 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131610+0000", 2024-09-21T14:58:45.590 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.88e-06 2024-09-21T14:58:45.590 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.590 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.590 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:45.590 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131612+0000", 2024-09-21T14:58:45.590 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5170000000000002e-06 2024-09-21T14:58:45.591 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.591 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.591 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:45.591 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.144205+0000", 2024-09-21T14:58:45.591 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012592763 2024-09-21T14:58:45.591 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.591 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.591 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:45.591 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.144332+0000", 2024-09-21T14:58:45.591 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012694200000000001 2024-09-21T14:58:45.591 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.591 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.591 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:45.591 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.145509+0000", 2024-09-21T14:58:45.591 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001176992 2024-09-21T14:58:45.592 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:45.592 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:45.592 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:45.592 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.592 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.592 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6632 3.1a 3:5d547f54:::rbd_data.11361bed1f78.0000000000000003:d [read 81920~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:45.592 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131614+0000", 2024-09-21T14:58:45.592 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.388880345, 2024-09-21T14:58:45.592 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014093553999999999, 2024-09-21T14:58:45.592 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:45.592 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:45.592 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:45.592 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:45.592 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:45.593 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6632 2024-09-21T14:58:45.593 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.593 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:45.593 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.593 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:45.593 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131614+0000", 2024-09-21T14:58:45.593 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:45.593 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.593 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.593 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:45.593 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131614+0000", 2024-09-21T14:58:45.593 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:45.593 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.594 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.594 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:45.594 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131614+0000", 2024-09-21T14:58:45.594 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T14:58:45.594 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.594 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.594 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:45.594 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131615+0000", 2024-09-21T14:58:45.594 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.218e-06 2024-09-21T14:58:45.594 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.594 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.594 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:45.594 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131618+0000", 2024-09-21T14:58:45.594 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9050000000000001e-06 2024-09-21T14:58:45.594 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.595 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.595 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:45.595 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131621+0000", 2024-09-21T14:58:45.595 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7439999999999999e-06 2024-09-21T14:58:45.595 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.595 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.595 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:45.595 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.145538+0000", 2024-09-21T14:58:45.595 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013916618 2024-09-21T14:58:45.595 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.595 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.595 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:45.595 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.145619+0000", 2024-09-21T14:58:45.595 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1812999999999997e-05 2024-09-21T14:58:45.595 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.596 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.596 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:45.596 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.145708+0000", 2024-09-21T14:58:45.596 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8460000000000003e-05 2024-09-21T14:58:45.596 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:45.596 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:45.596 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:45.596 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.596 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.596 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6633 3.1a 3:5d547f54:::rbd_data.11361bed1f78.0000000000000003:d [read 147456~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:45.596 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131623+0000", 2024-09-21T14:58:45.596 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.388871378999999, 2024-09-21T14:58:45.596 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014395845000000001, 2024-09-21T14:58:45.596 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:45.597 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:45.597 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:45.597 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:45.597 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:45.597 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6633 2024-09-21T14:58:45.597 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.597 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:45.597 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.597 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:45.597 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131623+0000", 2024-09-21T14:58:45.597 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:45.597 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.597 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.597 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:45.598 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131623+0000", 2024-09-21T14:58:45.598 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:45.598 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.598 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.598 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:45.598 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131623+0000", 2024-09-21T14:58:45.598 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T14:58:45.598 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.598 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.598 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:45.598 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131628+0000", 2024-09-21T14:58:45.598 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0370000000000002e-06 2024-09-21T14:58:45.598 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.598 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.599 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:45.599 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131632+0000", 2024-09-21T14:58:45.599 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3519999999999998e-06 2024-09-21T14:58:45.599 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.599 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.599 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:45.599 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131634+0000", 2024-09-21T14:58:45.599 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5019999999999999e-06 2024-09-21T14:58:45.599 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.599 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.599 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:45.599 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.145734+0000", 2024-09-21T14:58:45.599 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014100118 2024-09-21T14:58:45.599 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.600 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.600 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:45.600 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.145804+0000", 2024-09-21T14:58:45.600 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0229999999999997e-05 2024-09-21T14:58:45.600 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.600 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.600 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:45.600 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146019+0000", 2024-09-21T14:58:45.600 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00021481699999999999 2024-09-21T14:58:45.600 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:45.600 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:45.600 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:45.600 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.600 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.600 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6634 3.1a 3:5d547f54:::rbd_data.11361bed1f78.0000000000000003:d [read 229376~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:45.601 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131636+0000", 2024-09-21T14:58:45.601 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.388858269, 2024-09-21T14:58:45.601 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014645843, 2024-09-21T14:58:45.601 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:45.601 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:45.601 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:45.601 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:45.601 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:45.601 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6634 2024-09-21T14:58:45.601 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.601 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:45.601 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.602 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:45.602 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131636+0000", 2024-09-21T14:58:45.602 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:45.602 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.602 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.602 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:45.602 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131636+0000", 2024-09-21T14:58:45.602 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:45.602 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.602 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.602 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:45.602 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131636+0000", 2024-09-21T14:58:45.602 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:45.603 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.603 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.603 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:45.603 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131641+0000", 2024-09-21T14:58:45.603 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8489999999999997e-06 2024-09-21T14:58:45.603 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.603 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.603 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:45.603 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131644+0000", 2024-09-21T14:58:45.603 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.179e-06 2024-09-21T14:58:45.603 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.604 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.604 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:45.604 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131647+0000", 2024-09-21T14:58:45.604 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8009999999999999e-06 2024-09-21T14:58:45.604 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.604 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.604 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:45.604 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146046+0000", 2024-09-21T14:58:45.604 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014399018 2024-09-21T14:58:45.604 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.604 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.604 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:45.604 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146118+0000", 2024-09-21T14:58:45.604 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2044000000000003e-05 2024-09-21T14:58:45.605 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.605 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.605 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:45.605 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146282+0000", 2024-09-21T14:58:45.605 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000164196 2024-09-21T14:58:45.605 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:45.605 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:45.605 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:45.605 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.605 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.605 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6639 3.1a 3:592d80ed:::rbd_data.11361bed1f78.0000000000000005:d [read 0~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:45.605 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131650+0000", 2024-09-21T14:58:45.605 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.38884507, 2024-09-21T14:58:45.605 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016002832000000002, 2024-09-21T14:58:45.605 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:45.606 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:45.606 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:45.606 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:45.606 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:45.606 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6639 2024-09-21T14:58:45.606 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.606 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:45.606 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.606 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:45.606 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131650+0000", 2024-09-21T14:58:45.606 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:45.606 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.606 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.606 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:45.606 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131650+0000", 2024-09-21T14:58:45.607 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:45.607 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.607 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.607 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:45.607 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131649+0000", 2024-09-21T14:58:45.607 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:45.607 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.607 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.607 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:45.607 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131651+0000", 2024-09-21T14:58:45.607 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9199999999999998e-06 2024-09-21T14:58:45.607 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.607 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.607 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:45.607 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131656+0000", 2024-09-21T14:58:45.607 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9899999999999997e-06 2024-09-21T14:58:45.608 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.608 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.608 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:45.608 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131665+0000", 2024-09-21T14:58:45.608 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0999999999999993e-06 2024-09-21T14:58:45.608 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.608 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.608 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:45.608 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146309+0000", 2024-09-21T14:58:45.608 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014643893999999999 2024-09-21T14:58:45.608 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.608 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.608 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:45.608 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146392+0000", 2024-09-21T14:58:45.608 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2531e-05 2024-09-21T14:58:45.608 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.609 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.609 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:45.609 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147653+0000", 2024-09-21T14:58:45.609 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0012606710000000001 2024-09-21T14:58:45.609 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:45.609 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:45.609 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:45.609 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.609 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.609 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6640 3.1a 3:592d80ed:::rbd_data.11361bed1f78.0000000000000005:d [read 65536~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:45.609 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131670+0000", 2024-09-21T14:58:45.609 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.388824770999999, 2024-09-21T14:58:45.609 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016188859, 2024-09-21T14:58:45.609 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:45.609 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:45.609 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:45.610 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:45.610 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:45.610 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6640 2024-09-21T14:58:45.610 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.610 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:45.610 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.610 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:45.610 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131670+0000", 2024-09-21T14:58:45.610 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:45.610 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.610 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.610 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:45.610 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131670+0000", 2024-09-21T14:58:45.610 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:45.610 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.611 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.611 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:45.611 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131670+0000", 2024-09-21T14:58:45.611 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T14:58:45.611 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.611 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.611 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:45.611 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131672+0000", 2024-09-21T14:58:45.611 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.818e-06 2024-09-21T14:58:45.611 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.611 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.611 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:45.611 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131675+0000", 2024-09-21T14:58:45.611 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5930000000000002e-06 2024-09-21T14:58:45.611 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.611 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.612 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:45.612 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131678+0000", 2024-09-21T14:58:45.612 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6790000000000001e-06 2024-09-21T14:58:45.612 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.612 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.612 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:45.612 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147681+0000", 2024-09-21T14:58:45.612 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016002755 2024-09-21T14:58:45.612 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.612 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.612 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:45.612 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147762+0000", 2024-09-21T14:58:45.612 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1052999999999998e-05 2024-09-21T14:58:45.612 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.612 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.612 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:45.613 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147859+0000", 2024-09-21T14:58:45.613 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.7199000000000002e-05 2024-09-21T14:58:45.613 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:45.613 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:45.613 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:45.613 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.613 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.613 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6641 3.1a 3:592d80ed:::rbd_data.11361bed1f78.0000000000000005:d [read 147456~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:45.613 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131680+0000", 2024-09-21T14:58:45.613 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.388814483000001, 2024-09-21T14:58:45.613 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016431296000000001, 2024-09-21T14:58:45.613 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:45.613 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:45.613 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:45.613 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:45.614 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:45.614 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6641 2024-09-21T14:58:45.614 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.614 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:45.614 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.614 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:45.614 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131680+0000", 2024-09-21T14:58:45.614 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:45.614 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.614 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.614 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:45.614 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131680+0000", 2024-09-21T14:58:45.614 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:45.614 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.615 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.615 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:45.615 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131680+0000", 2024-09-21T14:58:45.615 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T14:58:45.615 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.615 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.615 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:45.615 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131681+0000", 2024-09-21T14:58:45.615 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2610000000000001e-06 2024-09-21T14:58:45.615 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.615 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.615 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:45.615 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131685+0000", 2024-09-21T14:58:45.615 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.8299999999999998e-06 2024-09-21T14:58:45.615 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.616 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.616 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:45.616 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131688+0000", 2024-09-21T14:58:45.616 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6189999999999998e-06 2024-09-21T14:58:45.616 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.616 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.616 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:45.616 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147884+0000", 2024-09-21T14:58:45.616 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016196465 2024-09-21T14:58:45.616 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.616 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.616 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:45.616 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147949+0000", 2024-09-21T14:58:45.616 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4867000000000001e-05 2024-09-21T14:58:45.616 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.616 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.617 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:45.617 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.148112+0000", 2024-09-21T14:58:45.617 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00016247700000000001 2024-09-21T14:58:45.617 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:45.617 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:45.617 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:45.617 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.617 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.617 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6642 3.1a 3:592d80ed:::rbd_data.11361bed1f78.0000000000000005:d [read 212992~32768 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:45.617 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131690+0000", 2024-09-21T14:58:45.617 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.388804605000001, 2024-09-21T14:58:45.617 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016751335999999999, 2024-09-21T14:58:45.617 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:45.617 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:45.618 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:45.618 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:45.618 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:45.618 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6642 2024-09-21T14:58:45.618 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.618 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:45.618 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.618 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:45.618 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131690+0000", 2024-09-21T14:58:45.618 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:45.618 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.618 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.618 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:45.618 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131690+0000", 2024-09-21T14:58:45.619 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:45.619 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.619 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.619 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:45.619 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131690+0000", 2024-09-21T14:58:45.619 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T14:58:45.619 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.619 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.619 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:45.619 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131691+0000", 2024-09-21T14:58:45.619 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3120000000000001e-06 2024-09-21T14:58:45.619 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.619 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.619 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:45.619 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131694+0000", 2024-09-21T14:58:45.620 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0070000000000002e-06 2024-09-21T14:58:45.620 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.620 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.620 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:45.620 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131700+0000", 2024-09-21T14:58:45.620 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0800000000000002e-06 2024-09-21T14:58:45.620 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.620 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.620 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:45.620 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.148135+0000", 2024-09-21T14:58:45.620 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016434250000000001 2024-09-21T14:58:45.620 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.620 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.620 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:45.620 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.148204+0000", 2024-09-21T14:58:45.621 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9466000000000002e-05 2024-09-21T14:58:45.621 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:45.621 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:45.621 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:45.621 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.148441+0000", 2024-09-21T14:58:45.621 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00023743700000000001 2024-09-21T14:58:45.621 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:45.621 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:45.621 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:45.621 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:45.621 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:45.621 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:58:45.622 INFO:teuthology.orchestra.run.smithi032.stdout:702 write 0x3dbadaa thru 0x3dca912 (0xfb69 bytes) 2024-09-21T14:58:45.623 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T14:58:45.624 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:45.624 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:45.624 INFO:teuthology.orchestra.run.smithi032.stdout:706 write 0x8d48e4f thru 0x8d55bc8 (0xcd7a bytes) 2024-09-21T14:58:45.624 INFO:teuthology.orchestra.run.smithi032.stdout:707 read 0x7e52c41 thru 0x7e60be4 (0xdfa4 bytes) 2024-09-21T14:58:45.624 INFO:teuthology.orchestra.run.smithi032.stdout:708 punch from 0x1177f4e to 0x1185776, (0xd828 bytes) 2024-09-21T14:58:45.624 INFO:teuthology.orchestra.run.smithi032.stdout:709 read 0x349d275 thru 0x34ab9ac (0xe738 bytes) 2024-09-21T14:58:45.624 INFO:teuthology.orchestra.run.smithi032.stdout:714 read 0x41bfe71 thru 0x41cd4e0 (0xd670 bytes) 2024-09-21T14:58:45.624 INFO:teuthology.orchestra.run.smithi032.stdout:715 read 0x8746156 thru 0x87558a0 (0xf74b bytes) 2024-09-21T14:58:45.625 INFO:teuthology.orchestra.run.smithi032.stdout:716 clone 7 order 20 su 65536 sc 5 2024-09-21T14:58:45.625 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:58:45.620+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:58:45.723 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:58:45.720+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:58:45.783 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:58:45.783 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:58:45.783 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T14:58:45.783 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:58:45.794 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T14:58:45.842 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:58:45.840+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:58:45.900 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:58:45.900 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:58:45.900 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T14:58:45.900 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T14:58:45.900 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:58:45.910 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T14:58:45.924 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:58:45.920+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:58:46.021 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:58:46.021 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T14:58:46.021 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T14:58:46.021 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T14:58:46.021 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.021 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:1454 3.e 3:71be909a:::rbd_data.1136515ecb6b.000000000000002a:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T14:58:46.021 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:30.339995+0000", 2024-09-21T14:58:46.021 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 15.677818791, 2024-09-21T14:58:46.021 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.033615974999999999, 2024-09-21T14:58:46.021 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:46.021 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:46.021 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:46.021 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:46.021 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:46.022 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1454 2024-09-21T14:58:46.022 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.022 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:46.022 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.022 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:46.022 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.339995+0000", 2024-09-21T14:58:46.022 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:46.022 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.022 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.022 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:46.022 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.339995+0000", 2024-09-21T14:58:46.022 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:46.022 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.022 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.022 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:46.023 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.339992+0000", 2024-09-21T14:58:46.023 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-21T14:58:46.023 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.023 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.023 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:46.023 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.340008+0000", 2024-09-21T14:58:46.023 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5906e-05 2024-09-21T14:58:46.023 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.023 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.023 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:46.023 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.340042+0000", 2024-09-21T14:58:46.023 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4066000000000002e-05 2024-09-21T14:58:46.023 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.023 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.024 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:46.024 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.340056+0000", 2024-09-21T14:58:46.024 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4389999999999999e-05 2024-09-21T14:58:46.024 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.024 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.024 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:46.024 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.340178+0000", 2024-09-21T14:58:46.024 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012202 2024-09-21T14:58:46.024 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.024 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.024 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:46.024 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.340424+0000", 2024-09-21T14:58:46.024 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000245682 2024-09-21T14:58:46.025 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.025 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.025 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 0", 2024-09-21T14:58:46.025 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.348708+0000", 2024-09-21T14:58:46.025 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0082834390000000001 2024-09-21T14:58:46.025 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.025 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.025 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:46.025 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.373176+0000", 2024-09-21T14:58:46.025 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024468888000000001 2024-09-21T14:58:46.025 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.025 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.025 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:46.025 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.373306+0000", 2024-09-21T14:58:46.025 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012991700000000001 2024-09-21T14:58:46.026 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.026 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.026 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:46.026 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.373464+0000", 2024-09-21T14:58:46.026 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000157913 2024-09-21T14:58:46.026 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.026 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.026 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:46.026 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.373611+0000", 2024-09-21T14:58:46.026 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014681699999999999 2024-09-21T14:58:46.026 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:46.026 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:46.026 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:46.026 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.027 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.027 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6468 3.16 3:68aaef8d:::rbd_data.11361bed1f78.000000000000000a:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:46.027 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.119941+0000", 2024-09-21T14:58:46.027 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.897872532999999, 2024-09-21T14:58:46.027 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0060084200000000004, 2024-09-21T14:58:46.027 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:46.027 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:46.027 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:46.027 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:46.027 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:46.027 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6468 2024-09-21T14:58:46.027 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.027 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:46.028 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.028 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:46.028 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.119941+0000", 2024-09-21T14:58:46.028 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:46.028 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.028 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.028 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:46.028 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.119941+0000", 2024-09-21T14:58:46.028 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:46.028 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.028 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.028 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:46.028 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.119941+0000", 2024-09-21T14:58:46.028 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:58:46.028 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.029 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.029 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:46.029 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.119945+0000", 2024-09-21T14:58:46.029 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7999999999999998e-06 2024-09-21T14:58:46.029 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.029 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.029 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:46.029 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.119956+0000", 2024-09-21T14:58:46.029 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0064999999999999e-05 2024-09-21T14:58:46.029 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.029 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.029 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:46.029 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.119963+0000", 2024-09-21T14:58:46.029 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2729999999999997e-06 2024-09-21T14:58:46.029 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.030 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.030 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:46.030 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.119994+0000", 2024-09-21T14:58:46.030 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1721999999999999e-05 2024-09-21T14:58:46.030 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.030 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.030 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting_for_map not empty", 2024-09-21T14:58:46.030 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.119997+0000", 2024-09-21T14:58:46.030 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8509999999999999e-06 2024-09-21T14:58:46.030 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.030 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.030 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:46.030 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.125713+0000", 2024-09-21T14:58:46.030 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0057158529999999999 2024-09-21T14:58:46.031 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.031 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.031 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:46.031 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.125834+0000", 2024-09-21T14:58:46.031 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012065099999999999 2024-09-21T14:58:46.031 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.031 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.031 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:46.031 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.125950+0000", 2024-09-21T14:58:46.031 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000115942 2024-09-21T14:58:46.031 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:46.031 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:46.031 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:46.031 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.032 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.032 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6477 3.16 3:6eb18e6c:::rbd_data.11361bed1f78.0000000000000013:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:46.032 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.119977+0000", 2024-09-21T14:58:46.032 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.897837227, 2024-09-21T14:58:46.032 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0062386309999999997, 2024-09-21T14:58:46.032 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:46.032 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:46.032 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:46.032 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:46.032 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:46.032 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6477 2024-09-21T14:58:46.032 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.032 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:46.032 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.032 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:46.033 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.119977+0000", 2024-09-21T14:58:46.033 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:46.033 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.033 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.033 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:46.033 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.119977+0000", 2024-09-21T14:58:46.033 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:46.033 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.033 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.033 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:46.033 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.119976+0000", 2024-09-21T14:58:46.033 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:46.033 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.033 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.034 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:46.034 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.119980+0000", 2024-09-21T14:58:46.034 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3610000000000001e-06 2024-09-21T14:58:46.034 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.034 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.034 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:46.034 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.119988+0000", 2024-09-21T14:58:46.034 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2630000000000003e-06 2024-09-21T14:58:46.034 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.034 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.034 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:46.034 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.119994+0000", 2024-09-21T14:58:46.034 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4540000000000004e-06 2024-09-21T14:58:46.035 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.035 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.035 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:46.035 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.120013+0000", 2024-09-21T14:58:46.035 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8859999999999999e-05 2024-09-21T14:58:46.035 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.035 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.035 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting_for_map not empty", 2024-09-21T14:58:46.035 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.120016+0000", 2024-09-21T14:58:46.035 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.683e-06 2024-09-21T14:58:46.035 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.035 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.035 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:46.035 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.125992+0000", 2024-09-21T14:58:46.035 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0059764579999999996 2024-09-21T14:58:46.036 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.036 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.036 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:46.036 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.126094+0000", 2024-09-21T14:58:46.036 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000101269 2024-09-21T14:58:46.036 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.036 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.036 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:46.036 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.126215+0000", 2024-09-21T14:58:46.036 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012178799999999999 2024-09-21T14:58:46.036 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:46.036 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:46.036 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:46.036 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.036 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.037 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6484 3.16 3:6eb18e6c:::rbd_data.11361bed1f78.0000000000000013:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:46.037 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.120002+0000", 2024-09-21T14:58:46.037 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.897812226999999, 2024-09-21T14:58:46.037 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0065176710000000001, 2024-09-21T14:58:46.037 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:46.037 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:46.037 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:46.037 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:46.037 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:46.037 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6484 2024-09-21T14:58:46.037 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.037 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:46.037 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.037 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:46.037 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.120002+0000", 2024-09-21T14:58:46.038 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:46.038 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.038 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.038 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:46.038 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.120002+0000", 2024-09-21T14:58:46.038 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:46.038 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.038 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.038 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:46.038 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.120001+0000", 2024-09-21T14:58:46.038 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:46.038 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.038 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.038 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:46.038 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.120004+0000", 2024-09-21T14:58:46.038 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1580000000000001e-06 2024-09-21T14:58:46.039 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.039 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.039 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:46.039 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.120013+0000", 2024-09-21T14:58:46.039 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2560000000000002e-06 2024-09-21T14:58:46.039 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.039 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.039 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:46.039 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.120020+0000", 2024-09-21T14:58:46.039 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6029999999999999e-06 2024-09-21T14:58:46.039 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.039 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.039 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:46.039 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.120053+0000", 2024-09-21T14:58:46.039 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2524999999999998e-05 2024-09-21T14:58:46.039 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.040 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.040 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting_for_map not empty", 2024-09-21T14:58:46.040 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.120059+0000", 2024-09-21T14:58:46.040 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6869999999999997e-06 2024-09-21T14:58:46.040 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.040 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.040 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:46.040 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.126262+0000", 2024-09-21T14:58:46.040 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0062021940000000003 2024-09-21T14:58:46.040 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.040 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.040 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:46.040 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.126391+0000", 2024-09-21T14:58:46.040 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000129011 2024-09-21T14:58:46.041 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.041 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.041 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:46.041 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.126519+0000", 2024-09-21T14:58:46.041 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012872 2024-09-21T14:58:46.041 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:46.041 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:46.041 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:46.041 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.041 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.041 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6498 3.16 3:6eb18e6c:::rbd_data.11361bed1f78.0000000000000013:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:46.041 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.120036+0000", 2024-09-21T14:58:46.041 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.897778311, 2024-09-21T14:58:46.041 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0067525479999999997, 2024-09-21T14:58:46.042 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:46.042 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:46.042 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:46.042 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:46.042 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:46.042 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6498 2024-09-21T14:58:46.042 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.042 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:46.042 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.042 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:46.042 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.120036+0000", 2024-09-21T14:58:46.042 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:46.042 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.042 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.043 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:46.043 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.120036+0000", 2024-09-21T14:58:46.043 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:46.043 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.043 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.043 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:46.043 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.120035+0000", 2024-09-21T14:58:46.043 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:46.043 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.043 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.043 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:46.043 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.120039+0000", 2024-09-21T14:58:46.043 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.8569999999999997e-06 2024-09-21T14:58:46.043 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.043 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.044 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:46.044 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.120047+0000", 2024-09-21T14:58:46.044 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2199999999999992e-06 2024-09-21T14:58:46.044 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.044 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.044 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:46.044 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.120060+0000", 2024-09-21T14:58:46.044 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3314e-05 2024-09-21T14:58:46.044 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.044 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.044 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:46.044 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.120086+0000", 2024-09-21T14:58:46.044 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5323000000000001e-05 2024-09-21T14:58:46.044 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.045 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.045 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting_for_map not empty", 2024-09-21T14:58:46.045 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.120088+0000", 2024-09-21T14:58:46.045 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6749999999999998e-06 2024-09-21T14:58:46.045 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.045 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.045 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:46.045 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.126582+0000", 2024-09-21T14:58:46.045 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0064932779999999999 2024-09-21T14:58:46.045 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.045 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.045 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:46.045 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.126692+0000", 2024-09-21T14:58:46.045 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00011026 2024-09-21T14:58:46.045 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.045 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.046 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:46.046 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.126788+0000", 2024-09-21T14:58:46.046 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6201999999999998e-05 2024-09-21T14:58:46.046 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:46.046 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:46.046 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:46.046 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.046 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.046 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6714 3.16 3:68aaef8d:::rbd_data.11361bed1f78.000000000000000a:d [read 245760~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:46.046 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.133641+0000", 2024-09-21T14:58:46.046 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.884172632, 2024-09-21T14:58:46.046 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0059002120000000002, 2024-09-21T14:58:46.046 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:46.046 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:46.046 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:46.047 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:46.047 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:46.047 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6714 2024-09-21T14:58:46.047 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.047 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:46.047 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.047 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:46.047 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133641+0000", 2024-09-21T14:58:46.047 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:46.047 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.047 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.047 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:46.047 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133641+0000", 2024-09-21T14:58:46.047 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:46.047 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.047 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.048 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:46.048 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133641+0000", 2024-09-21T14:58:46.048 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:46.048 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.048 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.048 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:46.048 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133644+0000", 2024-09-21T14:58:46.048 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9069999999999999e-06 2024-09-21T14:58:46.048 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.048 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.048 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:46.048 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133649+0000", 2024-09-21T14:58:46.048 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6130000000000003e-06 2024-09-21T14:58:46.048 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.048 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.048 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:46.049 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133655+0000", 2024-09-21T14:58:46.049 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.083e-06 2024-09-21T14:58:46.049 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.049 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.049 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:46.049 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.139122+0000", 2024-09-21T14:58:46.049 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.005466767 2024-09-21T14:58:46.049 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.049 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.049 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:46.049 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.139193+0000", 2024-09-21T14:58:46.049 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0448e-05 2024-09-21T14:58:46.049 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.049 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.049 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:46.050 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.139541+0000", 2024-09-21T14:58:46.050 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00034885500000000002 2024-09-21T14:58:46.050 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:46.050 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:46.050 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:46.050 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.050 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.050 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6729 3.16 3:6eb18e6c:::rbd_data.11361bed1f78.0000000000000013:d [read 49152~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:46.050 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.133660+0000", 2024-09-21T14:58:46.050 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.884154217000001, 2024-09-21T14:58:46.050 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0062381800000000003, 2024-09-21T14:58:46.050 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:46.050 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:46.050 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:46.050 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:46.050 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:46.051 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6729 2024-09-21T14:58:46.051 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.051 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:46.051 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.051 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:46.051 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133660+0000", 2024-09-21T14:58:46.051 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:46.051 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.051 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.051 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:46.051 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133660+0000", 2024-09-21T14:58:46.051 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:46.051 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.051 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.051 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:46.051 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133659+0000", 2024-09-21T14:58:46.052 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:46.052 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.052 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.052 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:46.052 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133662+0000", 2024-09-21T14:58:46.052 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.463e-06 2024-09-21T14:58:46.052 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.052 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.052 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:46.052 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133667+0000", 2024-09-21T14:58:46.052 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.344e-06 2024-09-21T14:58:46.052 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.052 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.052 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:46.052 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133672+0000", 2024-09-21T14:58:46.052 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6280000000000004e-06 2024-09-21T14:58:46.053 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.053 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.053 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:46.053 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.139570+0000", 2024-09-21T14:58:46.053 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.005898225 2024-09-21T14:58:46.053 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.053 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.053 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:46.053 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.139651+0000", 2024-09-21T14:58:46.053 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1342999999999997e-05 2024-09-21T14:58:46.053 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.053 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.053 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:46.053 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.139898+0000", 2024-09-21T14:58:46.053 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00024648500000000001 2024-09-21T14:58:46.053 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:46.054 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:46.054 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:46.054 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.054 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.054 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6746 3.16 3:6eb18e6c:::rbd_data.11361bed1f78.0000000000000013:d [read 65536~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:46.054 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.133676+0000", 2024-09-21T14:58:46.054 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.884138147, 2024-09-21T14:58:46.054 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0065617979999999998, 2024-09-21T14:58:46.054 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:46.054 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:46.054 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:46.054 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:46.054 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:46.054 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6746 2024-09-21T14:58:46.054 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.055 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:46.055 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.055 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:46.055 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133676+0000", 2024-09-21T14:58:46.055 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:46.055 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.055 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.055 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:46.055 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133676+0000", 2024-09-21T14:58:46.055 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:46.055 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.055 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.055 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:46.055 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133675+0000", 2024-09-21T14:58:46.055 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:46.056 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.056 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.056 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:46.056 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133678+0000", 2024-09-21T14:58:46.056 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3149999999999999e-06 2024-09-21T14:58:46.056 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.056 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.056 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:46.056 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133683+0000", 2024-09-21T14:58:46.056 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4439999999999999e-06 2024-09-21T14:58:46.056 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.056 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.056 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:46.056 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133687+0000", 2024-09-21T14:58:46.056 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1649999999999999e-06 2024-09-21T14:58:46.056 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.057 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.057 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:46.057 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.139932+0000", 2024-09-21T14:58:46.057 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0062448920000000002 2024-09-21T14:58:46.057 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.057 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.057 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:46.057 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.140009+0000", 2024-09-21T14:58:46.057 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6734000000000001e-05 2024-09-21T14:58:46.057 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.057 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.057 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:46.057 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.140238+0000", 2024-09-21T14:58:46.057 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00022854599999999999 2024-09-21T14:58:46.057 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:46.057 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:46.058 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:46.058 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.058 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.058 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6747 3.16 3:6eb18e6c:::rbd_data.11361bed1f78.0000000000000013:d [read 147456~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:46.058 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.133691+0000", 2024-09-21T14:58:46.058 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.884122549000001, 2024-09-21T14:58:46.058 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.006764936, 2024-09-21T14:58:46.058 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:46.058 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:46.058 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:46.058 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:46.058 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:46.058 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6747 2024-09-21T14:58:46.058 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.059 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:46.059 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.059 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:46.059 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133691+0000", 2024-09-21T14:58:46.059 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:46.059 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.059 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.059 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:46.059 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133691+0000", 2024-09-21T14:58:46.059 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:46.059 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.059 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.059 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:46.059 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133691+0000", 2024-09-21T14:58:46.059 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:46.059 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.060 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.060 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:46.060 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133693+0000", 2024-09-21T14:58:46.060 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2390000000000001e-06 2024-09-21T14:58:46.060 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.060 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.060 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:46.060 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133699+0000", 2024-09-21T14:58:46.060 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2519999999999999e-06 2024-09-21T14:58:46.060 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.060 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.060 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:46.060 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133703+0000", 2024-09-21T14:58:46.060 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3660000000000002e-06 2024-09-21T14:58:46.060 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.060 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.061 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:46.061 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.140270+0000", 2024-09-21T14:58:46.061 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0065673499999999996 2024-09-21T14:58:46.061 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.061 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.061 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:46.061 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.140351+0000", 2024-09-21T14:58:46.061 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0975000000000003e-05 2024-09-21T14:58:46.061 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.061 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.061 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:46.061 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.140456+0000", 2024-09-21T14:58:46.061 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000105086 2024-09-21T14:58:46.061 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:46.061 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:46.061 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:46.062 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.062 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.062 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6748 3.16 3:6eb18e6c:::rbd_data.11361bed1f78.0000000000000013:d [read 212992~32768 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:46.062 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.133707+0000", 2024-09-21T14:58:46.062 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.88410702, 2024-09-21T14:58:46.062 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0071255779999999996, 2024-09-21T14:58:46.062 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:46.062 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:46.062 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:46.062 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:46.062 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:46.062 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6748 2024-09-21T14:58:46.062 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.063 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:46.063 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.063 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:46.063 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133707+0000", 2024-09-21T14:58:46.063 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:46.063 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.063 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.063 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:46.063 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133707+0000", 2024-09-21T14:58:46.063 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:46.063 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.063 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.063 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:46.063 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133707+0000", 2024-09-21T14:58:46.063 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:46.064 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.064 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.064 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:46.064 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133709+0000", 2024-09-21T14:58:46.064 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2419999999999999e-06 2024-09-21T14:58:46.064 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.064 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.064 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:46.064 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133714+0000", 2024-09-21T14:58:46.064 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.361e-06 2024-09-21T14:58:46.064 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.064 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.064 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:46.064 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133718+0000", 2024-09-21T14:58:46.064 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2549999999999999e-06 2024-09-21T14:58:46.065 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.065 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.065 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:46.065 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.140541+0000", 2024-09-21T14:58:46.065 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0068226040000000003 2024-09-21T14:58:46.065 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.065 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.065 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:46.065 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.140634+0000", 2024-09-21T14:58:46.065 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2748e-05 2024-09-21T14:58:46.065 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.065 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.065 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:46.065 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.140832+0000", 2024-09-21T14:58:46.065 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00019867299999999999 2024-09-21T14:58:46.066 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:46.066 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:46.066 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:46.066 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.066 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.066 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9390 3.e 3:730ca76c:::rbd_data.11363a21d2be.0000000000000004:10 [read 0~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:46.066 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.160531+0000", 2024-09-21T14:58:46.066 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 6.8572834069999997, 2024-09-21T14:58:46.066 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0059367109999999999, 2024-09-21T14:58:46.066 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:46.066 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:46.066 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:46.066 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:46.066 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:46.066 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9390 2024-09-21T14:58:46.066 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.067 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:46.067 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.067 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:46.067 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160531+0000", 2024-09-21T14:58:46.067 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:46.067 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.067 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.067 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:46.067 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160531+0000", 2024-09-21T14:58:46.067 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:46.067 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.067 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.067 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:46.067 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160530+0000", 2024-09-21T14:58:46.067 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:46.067 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.068 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.068 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:46.068 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160533+0000", 2024-09-21T14:58:46.068 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4880000000000001e-06 2024-09-21T14:58:46.068 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.068 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.068 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:46.068 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160539+0000", 2024-09-21T14:58:46.068 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9359999999999996e-06 2024-09-21T14:58:46.068 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.068 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.068 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:46.068 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160543+0000", 2024-09-21T14:58:46.068 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6609999999999999e-06 2024-09-21T14:58:46.068 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.068 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.069 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:46.069 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.166238+0000", 2024-09-21T14:58:46.069 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0056944309999999998 2024-09-21T14:58:46.069 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.069 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.069 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:46.069 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.166299+0000", 2024-09-21T14:58:46.069 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0977000000000002e-05 2024-09-21T14:58:46.069 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.069 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.069 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:46.069 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.166467+0000", 2024-09-21T14:58:46.069 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00016854899999999999 2024-09-21T14:58:46.069 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:46.069 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:46.069 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:46.069 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.070 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.070 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9391 3.e 3:730ca76c:::rbd_data.11363a21d2be.0000000000000004:10 [read 106496~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:46.070 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.160547+0000", 2024-09-21T14:58:46.070 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 6.8572668920000002, 2024-09-21T14:58:46.070 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0064124159999999998, 2024-09-21T14:58:46.070 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:46.070 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:46.070 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:46.070 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:46.070 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:46.070 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9391 2024-09-21T14:58:46.070 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.070 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:46.070 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.071 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:46.071 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160547+0000", 2024-09-21T14:58:46.071 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:46.071 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.071 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.071 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:46.071 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160547+0000", 2024-09-21T14:58:46.071 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:46.071 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.071 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.071 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:46.071 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160547+0000", 2024-09-21T14:58:46.071 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:46.071 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.071 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.072 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:46.072 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160549+0000", 2024-09-21T14:58:46.072 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3369999999999998e-06 2024-09-21T14:58:46.072 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.072 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.072 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:46.072 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160557+0000", 2024-09-21T14:58:46.072 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6490000000000008e-06 2024-09-21T14:58:46.072 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.072 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.072 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:46.072 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160562+0000", 2024-09-21T14:58:46.072 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8999999999999997e-06 2024-09-21T14:58:46.072 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.072 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.073 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:46.073 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.166612+0000", 2024-09-21T14:58:46.073 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0060500780000000004 2024-09-21T14:58:46.073 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.073 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.073 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:46.073 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.166723+0000", 2024-09-21T14:58:46.073 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00011092899999999999 2024-09-21T14:58:46.073 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.073 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.073 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:46.073 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.166959+0000", 2024-09-21T14:58:46.073 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00023686700000000001 2024-09-21T14:58:46.073 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:46.073 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:46.073 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:46.074 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.074 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.074 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9392 3.e 3:730ca76c:::rbd_data.11363a21d2be.0000000000000004:10 [read 139264~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:46.074 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.160566+0000", 2024-09-21T14:58:46.074 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 6.8572481779999999, 2024-09-21T14:58:46.074 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0067674689999999999, 2024-09-21T14:58:46.074 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:46.074 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:46.074 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:46.074 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:46.074 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:46.074 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9392 2024-09-21T14:58:46.074 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.074 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:46.075 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.075 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:46.075 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160566+0000", 2024-09-21T14:58:46.075 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:46.075 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.075 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.075 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:46.075 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160566+0000", 2024-09-21T14:58:46.075 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:46.075 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.075 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.075 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:46.075 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160565+0000", 2024-09-21T14:58:46.075 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:46.076 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.076 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.076 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:46.076 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160568+0000", 2024-09-21T14:58:46.076 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.367e-06 2024-09-21T14:58:46.076 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.076 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.076 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:46.076 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160575+0000", 2024-09-21T14:58:46.076 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8669999999999996e-06 2024-09-21T14:58:46.076 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.076 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.076 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:46.076 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160581+0000", 2024-09-21T14:58:46.076 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.93e-06 2024-09-21T14:58:46.077 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.077 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.077 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:46.077 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.166990+0000", 2024-09-21T14:58:46.077 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0064090299999999996 2024-09-21T14:58:46.077 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.077 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.077 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:46.077 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.167097+0000", 2024-09-21T14:58:46.077 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000107841 2024-09-21T14:58:46.077 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.077 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.077 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:46.077 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.167333+0000", 2024-09-21T14:58:46.077 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00023578299999999999 2024-09-21T14:58:46.077 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:46.078 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:46.078 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:46.078 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.078 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.078 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9393 3.e 3:730ca76c:::rbd_data.11363a21d2be.0000000000000004:10 [read 245760~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:46.078 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.160585+0000", 2024-09-21T14:58:46.078 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 6.8572289619999998, 2024-09-21T14:58:46.078 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0071208850000000004, 2024-09-21T14:58:46.078 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:46.078 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:46.078 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:46.078 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:46.078 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:46.078 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9393 2024-09-21T14:58:46.078 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.079 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:46.079 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.079 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:46.079 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160585+0000", 2024-09-21T14:58:46.079 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:46.079 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.079 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.079 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:46.079 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160585+0000", 2024-09-21T14:58:46.079 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:46.079 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.079 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.079 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:46.079 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160585+0000", 2024-09-21T14:58:46.079 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:46.079 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.080 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.080 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:46.080 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160587+0000", 2024-09-21T14:58:46.080 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3970000000000001e-06 2024-09-21T14:58:46.080 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.080 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.080 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:46.080 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160592+0000", 2024-09-21T14:58:46.080 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9250000000000003e-06 2024-09-21T14:58:46.080 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.080 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.080 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:46.080 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160597+0000", 2024-09-21T14:58:46.080 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7809999999999996e-06 2024-09-21T14:58:46.080 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.080 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.081 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:46.081 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.167364+0000", 2024-09-21T14:58:46.081 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0067670669999999999 2024-09-21T14:58:46.081 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.081 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.081 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:46.081 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.167480+0000", 2024-09-21T14:58:46.081 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000116345 2024-09-21T14:58:46.081 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.081 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.081 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:46.081 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.167706+0000", 2024-09-21T14:58:46.081 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00022575199999999999 2024-09-21T14:58:46.081 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:46.081 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:46.082 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:46.082 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.082 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.082 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9394 3.e 3:730ca76c:::rbd_data.11363a21d2be.0000000000000004:10 [read 278528~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:46.082 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.160601+0000", 2024-09-21T14:58:46.082 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 6.8572133869999998, 2024-09-21T14:58:46.082 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0074910879999999999, 2024-09-21T14:58:46.082 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:46.082 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:46.082 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:46.082 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:46.082 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:46.082 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9394 2024-09-21T14:58:46.082 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.082 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:46.083 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.083 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:46.083 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160601+0000", 2024-09-21T14:58:46.083 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:46.083 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.083 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.083 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:46.083 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160601+0000", 2024-09-21T14:58:46.083 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:46.083 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.083 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.083 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:46.083 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160600+0000", 2024-09-21T14:58:46.083 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:46.083 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.084 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.084 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:46.084 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160603+0000", 2024-09-21T14:58:46.084 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6529999999999998e-06 2024-09-21T14:58:46.084 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.084 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.084 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:46.084 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160609+0000", 2024-09-21T14:58:46.084 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7080000000000002e-06 2024-09-21T14:58:46.084 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.084 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.084 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:46.084 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160616+0000", 2024-09-21T14:58:46.084 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9739999999999996e-06 2024-09-21T14:58:46.085 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.085 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.085 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:46.085 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.167740+0000", 2024-09-21T14:58:46.085 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0071245019999999996 2024-09-21T14:58:46.085 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.085 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.085 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:46.085 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.167869+0000", 2024-09-21T14:58:46.085 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000129002 2024-09-21T14:58:46.085 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.085 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.085 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:46.085 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.168092+0000", 2024-09-21T14:58:46.086 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000222568 2024-09-21T14:58:46.086 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:46.086 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:46.086 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:46.086 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.086 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.086 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9395 3.e 3:730ca76c:::rbd_data.11363a21d2be.0000000000000004:10 [read 385024~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:46.086 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.160620+0000", 2024-09-21T14:58:46.086 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 6.8571940969999998, 2024-09-21T14:58:46.086 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0078786570000000007, 2024-09-21T14:58:46.086 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:46.086 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:46.086 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:46.087 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:46.087 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:46.087 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9395 2024-09-21T14:58:46.087 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.087 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:46.087 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.087 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:46.087 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160620+0000", 2024-09-21T14:58:46.087 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:46.087 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.087 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.087 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:46.087 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160620+0000", 2024-09-21T14:58:46.087 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:46.087 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.088 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.088 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:46.088 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160619+0000", 2024-09-21T14:58:46.088 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:46.088 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.088 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.088 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:46.088 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160622+0000", 2024-09-21T14:58:46.088 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.463e-06 2024-09-21T14:58:46.088 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.088 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.088 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:46.088 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160626+0000", 2024-09-21T14:58:46.088 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.0260000000000001e-06 2024-09-21T14:58:46.089 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.089 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.089 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:46.089 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160628+0000", 2024-09-21T14:58:46.089 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5610000000000001e-06 2024-09-21T14:58:46.089 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.089 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.089 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:46.089 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.168151+0000", 2024-09-21T14:58:46.089 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0075221890000000003 2024-09-21T14:58:46.089 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.089 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.089 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:46.090 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.168253+0000", 2024-09-21T14:58:46.090 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000102157 2024-09-21T14:58:46.090 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.090 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.090 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:46.090 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.168498+0000", 2024-09-21T14:58:46.090 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00024563200000000002 2024-09-21T14:58:46.090 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:46.090 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:46.090 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:46.090 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.090 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.090 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9645 3.16 3:6fc5e53c:::rbd_data.11363a21d2be.000000000000000e:10 [read 57344~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:46.090 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.200690+0000", 2024-09-21T14:58:46.091 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 6.8171235660000002, 2024-09-21T14:58:46.091 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0062915949999999997, 2024-09-21T14:58:46.091 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:46.091 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:46.091 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:46.091 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:46.091 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:46.091 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9645 2024-09-21T14:58:46.091 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.091 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:46.091 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.091 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:46.091 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.200690+0000", 2024-09-21T14:58:46.091 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:46.092 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.092 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.092 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:46.092 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.200690+0000", 2024-09-21T14:58:46.092 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:46.092 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.092 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.092 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:46.092 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.200689+0000", 2024-09-21T14:58:46.092 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:46.092 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.092 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.092 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:46.093 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.200696+0000", 2024-09-21T14:58:46.093 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8909999999999998e-06 2024-09-21T14:58:46.093 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.093 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.093 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:46.093 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.200707+0000", 2024-09-21T14:58:46.093 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1573000000000001e-05 2024-09-21T14:58:46.093 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.093 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.093 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:46.093 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.200715+0000", 2024-09-21T14:58:46.093 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8450000000000003e-06 2024-09-21T14:58:46.093 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.093 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.094 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:46.094 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.200880+0000", 2024-09-21T14:58:46.094 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00016440899999999999 2024-09-21T14:58:46.094 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.094 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.094 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:46.094 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.206880+0000", 2024-09-21T14:58:46.094 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0060000089999999997 2024-09-21T14:58:46.094 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.094 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.094 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:46.094 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.206982+0000", 2024-09-21T14:58:46.094 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000102302 2024-09-21T14:58:46.094 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:46.095 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:46.095 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:46.095 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.095 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.095 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9679 3.16 3:6fc5e53c:::rbd_data.11363a21d2be.000000000000000e:10 [read 65536~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:46.095 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.201730+0000", 2024-09-21T14:58:46.095 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 6.8160843570000003, 2024-09-21T14:58:46.095 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0058735239999999998, 2024-09-21T14:58:46.095 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:46.095 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:46.095 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:46.095 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:46.095 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:46.095 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9679 2024-09-21T14:58:46.095 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.096 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:46.096 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.096 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:46.096 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.201730+0000", 2024-09-21T14:58:46.096 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:46.096 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.096 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.096 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:46.096 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.201730+0000", 2024-09-21T14:58:46.096 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:46.096 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.096 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.096 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:46.096 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.201729+0000", 2024-09-21T14:58:46.097 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:58:46.097 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.097 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.097 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:46.097 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.201734+0000", 2024-09-21T14:58:46.097 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7939999999999999e-06 2024-09-21T14:58:46.097 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.097 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.097 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:46.097 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.201744+0000", 2024-09-21T14:58:46.097 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4769999999999992e-06 2024-09-21T14:58:46.097 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.097 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.097 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:46.098 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.201761+0000", 2024-09-21T14:58:46.098 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7504e-05 2024-09-21T14:58:46.098 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.098 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.098 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:46.098 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.207365+0000", 2024-09-21T14:58:46.098 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0056033970000000004 2024-09-21T14:58:46.098 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.098 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.098 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:46.098 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.207437+0000", 2024-09-21T14:58:46.098 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2138999999999995e-05 2024-09-21T14:58:46.098 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.099 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.099 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:46.099 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.207603+0000", 2024-09-21T14:58:46.099 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00016622499999999999 2024-09-21T14:58:46.099 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:46.099 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:46.099 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:46.099 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.099 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.099 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:15130 3.2 3:47f1bbdb:::rbd_data.11361bed1f78.0000000000000067:head [sparse-read 0~147456] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-21T14:58:46.099 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:40.288284+0000", 2024-09-21T14:58:46.099 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 5.7295294700000001, 2024-09-21T14:58:46.099 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1142935039999999, 2024-09-21T14:58:46.099 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:46.100 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:46.100 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:46.100 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:46.100 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:46.100 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 15130 2024-09-21T14:58:46.100 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.100 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:46.100 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.100 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:46.100 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288284+0000", 2024-09-21T14:58:46.100 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:46.100 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.100 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.100 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:46.101 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288284+0000", 2024-09-21T14:58:46.101 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:46.101 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.101 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.101 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:46.101 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288284+0000", 2024-09-21T14:58:46.101 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:46.101 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.101 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.101 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:46.101 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288288+0000", 2024-09-21T14:58:46.101 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5829999999999998e-06 2024-09-21T14:58:46.101 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.102 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.102 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:46.102 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288297+0000", 2024-09-21T14:58:46.102 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0669999999999998e-06 2024-09-21T14:58:46.102 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.102 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.102 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:46.102 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288305+0000", 2024-09-21T14:58:46.102 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8860000000000004e-06 2024-09-21T14:58:46.102 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.102 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.102 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:46.102 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288417+0000", 2024-09-21T14:58:46.102 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000112952 2024-09-21T14:58:46.103 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.103 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.103 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T14:58:46.103 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288428+0000", 2024-09-21T14:58:46.103 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0631e-05 2024-09-21T14:58:46.103 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.103 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.103 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:46.103 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.401876+0000", 2024-09-21T14:58:46.103 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1134479610000001 2024-09-21T14:58:46.103 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.103 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.103 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:46.103 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.402076+0000", 2024-09-21T14:58:46.103 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00019944200000000001 2024-09-21T14:58:46.104 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.104 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.104 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:46.104 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.402578+0000", 2024-09-21T14:58:46.104 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00050242899999999998 2024-09-21T14:58:46.104 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:46.104 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:46.104 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:46.104 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.104 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.104 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:20598 3.2 3:403f4568:::rbd_data.1136c84d9a13.00000000000000d4:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-21T14:58:46.104 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:45.401322+0000", 2024-09-21T14:58:46.104 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.616491754, 2024-09-21T14:58:46.104 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014768051000000001, 2024-09-21T14:58:46.104 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:46.105 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:46.105 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:46.105 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:46.105 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:46.105 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 20598 2024-09-21T14:58:46.105 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.105 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:46.105 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.105 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:46.105 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:45.401322+0000", 2024-09-21T14:58:46.105 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:46.105 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.105 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.105 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:46.105 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:45.401322+0000", 2024-09-21T14:58:46.106 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:46.106 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.106 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.106 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:46.106 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:45.401320+0000", 2024-09-21T14:58:46.106 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T14:58:46.106 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.106 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.106 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:46.106 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:45.401330+0000", 2024-09-21T14:58:46.106 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0108999999999999e-05 2024-09-21T14:58:46.106 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.106 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.106 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:46.106 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:45.401351+0000", 2024-09-21T14:58:46.107 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1191000000000001e-05 2024-09-21T14:58:46.107 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.107 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.107 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:46.107 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:45.401361+0000", 2024-09-21T14:58:46.107 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5130000000000002e-06 2024-09-21T14:58:46.107 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.107 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.107 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:46.107 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:45.401476+0000", 2024-09-21T14:58:46.107 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000115684 2024-09-21T14:58:46.107 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.107 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.107 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:46.107 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:45.401717+0000", 2024-09-21T14:58:46.108 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000240256 2024-09-21T14:58:46.108 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.108 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.108 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 3", 2024-09-21T14:58:46.108 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:45.406009+0000", 2024-09-21T14:58:46.108 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0042920780000000004 2024-09-21T14:58:46.108 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.108 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.108 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:46.108 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:45.415983+0000", 2024-09-21T14:58:46.108 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0099743269999999998 2024-09-21T14:58:46.108 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.108 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.108 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:46.109 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:45.415998+0000", 2024-09-21T14:58:46.109 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.49e-05 2024-09-21T14:58:46.109 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.109 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.109 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:46.109 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:45.416049+0000", 2024-09-21T14:58:46.109 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0657e-05 2024-09-21T14:58:46.109 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:46.109 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:46.109 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:46.109 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:45.416090+0000", 2024-09-21T14:58:46.109 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1569000000000001e-05 2024-09-21T14:58:46.109 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:46.109 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:46.109 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:46.109 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:46.110 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:46.110 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:58:46.111 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T14:58:46.112 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:58:46.108+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:58:46.172 INFO:teuthology.orchestra.run.smithi032.stderr:set osd.6 primary-affinity to 0.0501369 (832852) 2024-09-21T14:58:46.176 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T14:58:46.230 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:58:46.228+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:58:46.266 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:58:46.266 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:58:46.266 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T14:58:46.266 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:58:46.276 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T14:58:46.292 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T14:58:46.312 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:58:46.308+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:58:46.395 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:58:46.395 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:58:46.395 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T14:58:46.396 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T14:58:46.396 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:58:46.407 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T14:58:46.413 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:58:46.412+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:58:46.422 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T14:58:46.440 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T14:58:46.459 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T14:58:46.479 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T14:58:46.500 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-21T14:58:46.514 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:58:46.512+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:58:46.524 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T14:58:46.539 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:46.539 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:46.546 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T14:58:46.549 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T14:58:46.569 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T14:58:46.571 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:58:46.572 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:58:46.572 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:58:46.572 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:58:46.572 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:58:46.572 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:58:46.572 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:58:46.572 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:58:46.626 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:58:46.624+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:58:46.665 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:46.665 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:46.675 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T14:58:46.716 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:58:46.712+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:58:46.786 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:46.786 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:46.796 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T14:58:46.817 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:58:46.816+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:58:46.902 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:46.902 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:46.911 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T14:58:46.918 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:58:46.916+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:58:47.019 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:58:47.016+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:58:47.023 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:47.023 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:47.033 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T14:58:47.120 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:58:47.116+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:58:47.139 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:47.139 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:47.149 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T14:58:47.178 INFO:teuthology.orchestra.run.smithi032.stderr:noscrub is unset 2024-09-21T14:58:47.243 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:58:47.240+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:58:47.272 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:47.272 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:47.282 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T14:58:47.322 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:58:47.320+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:58:47.386 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:47.386 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:47.396 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T14:58:47.422 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:58:47.420+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:58:47.502 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:47.502 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:47.512 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T14:58:47.557 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:58:47.556+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:58:47.622 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:47.622 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:47.625 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:58:47.624+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:58:47.632 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T14:58:47.725 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:58:47.724+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:58:47.741 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:47.742 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:47.751 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T14:58:47.826 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:58:47.824+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:58:47.857 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:47.857 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:47.866 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T14:58:47.927 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:58:47.924+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:58:47.974 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:47.974 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:47.984 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T14:58:48.028 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:58:48.024+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:58:48.103 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:48.103 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:48.113 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T14:58:48.129 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:58:48.128+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:58:48.225 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:48.225 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:48.229 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:58:48.228+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:58:48.236 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T14:58:48.324 INFO:teuthology.orchestra.run.smithi032.stdout:checking clone #5, image image_client.0-clone5 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent6 2024-09-21T14:58:48.330 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:58:48.328+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:58:48.355 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:48.356 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:48.373 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T14:58:48.431 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:58:48.428+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:58:48.473 INFO:teuthology.orchestra.run.smithi032.stdout:717 write 0x229d202 thru 0x22a2334 (0x5133 bytes) 2024-09-21T14:58:48.473 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:48.474 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:48.485 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T14:58:48.487 INFO:teuthology.orchestra.run.smithi032.stdout:718 writesame 0x43b14e1 thru 0x43c091e data_size 0x33(0xf43e bytes) 2024-09-21T14:58:48.495 INFO:teuthology.orchestra.run.smithi032.stdout:720 clone 8 order 21 su 16384 sc 4 2024-09-21T14:58:48.532 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:58:48.528+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:58:48.583 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T14:58:48.592 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:48.592 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:48.603 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T14:58:48.646 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:58:48.644+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:58:48.724 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T14:58:48.734 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:58:48.732+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:58:48.745 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T14:58:48.756 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:58:48.757 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T14:58:48.757 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T14:58:48.757 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T14:58:48.757 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.757 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T14:58:48.757 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.205754+0000", 2024-09-21T14:58:48.757 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 45.548365109999999, 2024-09-21T14:58:48.757 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.41117094199999998, 2024-09-21T14:58:48.757 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:48.757 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:48.757 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:48.757 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:48.757 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:48.758 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1 2024-09-21T14:58:48.758 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.758 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:48.758 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.758 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:48.758 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205754+0000", 2024-09-21T14:58:48.758 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:48.758 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.758 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.758 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:48.758 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205754+0000", 2024-09-21T14:58:48.758 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:48.758 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.758 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.759 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:48.759 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205752+0000", 2024-09-21T14:58:48.759 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T14:58:48.759 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.759 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.759 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:48.759 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205765+0000", 2024-09-21T14:58:48.759 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3230999999999999e-05 2024-09-21T14:58:48.759 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.759 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.759 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:48.759 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205782+0000", 2024-09-21T14:58:48.759 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6634e-05 2024-09-21T14:58:48.759 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.759 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.760 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:48.760 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205793+0000", 2024-09-21T14:58:48.760 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1297e-05 2024-09-21T14:58:48.760 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.760 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.760 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:48.760 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205867+0000", 2024-09-21T14:58:48.760 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4269000000000003e-05 2024-09-21T14:58:48.760 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.760 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.760 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T14:58:48.760 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205877+0000", 2024-09-21T14:58:48.760 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2690000000000005e-06 2024-09-21T14:58:48.760 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.760 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.761 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:48.761 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.613700+0000", 2024-09-21T14:58:48.761 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.407823455 2024-09-21T14:58:48.761 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.761 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.761 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:48.761 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.614203+0000", 2024-09-21T14:58:48.761 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00050311500000000005 2024-09-21T14:58:48.761 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.761 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.761 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:48.761 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.614750+0000", 2024-09-21T14:58:48.761 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00054738699999999998 2024-09-21T14:58:48.761 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.761 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.762 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:48.762 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616367+0000", 2024-09-21T14:58:48.762 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001616634 2024-09-21T14:58:48.762 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.762 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.762 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:48.762 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616574+0000", 2024-09-21T14:58:48.762 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00020668499999999999 2024-09-21T14:58:48.762 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.762 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.762 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:48.762 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616802+0000", 2024-09-21T14:58:48.762 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00022817900000000001 2024-09-21T14:58:48.762 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.763 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.763 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:48.763 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616925+0000", 2024-09-21T14:58:48.763 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012309699999999999 2024-09-21T14:58:48.763 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:48.763 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:48.763 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:48.763 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.763 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.763 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T14:58:48.763 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.955142+0000", 2024-09-21T14:58:48.763 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 44.798977463, 2024-09-21T14:58:48.763 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012388923, 2024-09-21T14:58:48.764 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:48.764 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:48.764 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:48.764 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:48.764 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:48.764 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 113 2024-09-21T14:58:48.764 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.764 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:48.764 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.764 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:48.764 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955142+0000", 2024-09-21T14:58:48.764 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:48.764 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.764 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.764 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:48.764 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955142+0000", 2024-09-21T14:58:48.765 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:48.765 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.765 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.765 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:48.765 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955141+0000", 2024-09-21T14:58:48.765 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:48.765 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.765 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.765 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:48.765 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955145+0000", 2024-09-21T14:58:48.765 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2109999999999998e-06 2024-09-21T14:58:48.765 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.765 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.766 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:48.766 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955150+0000", 2024-09-21T14:58:48.766 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4090000000000002e-06 2024-09-21T14:58:48.766 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.766 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.766 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:48.766 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955153+0000", 2024-09-21T14:58:48.766 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1240000000000001e-06 2024-09-21T14:58:48.766 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.766 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.766 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:48.766 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.963026+0000", 2024-09-21T14:58:48.766 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0078724060000000002 2024-09-21T14:58:48.766 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.766 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.767 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:48.767 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.963126+0000", 2024-09-21T14:58:48.767 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000100206 2024-09-21T14:58:48.767 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.767 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.767 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:48.767 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.963277+0000", 2024-09-21T14:58:48.767 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015159200000000001 2024-09-21T14:58:48.767 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.767 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.767 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:48.767 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.965519+0000", 2024-09-21T14:58:48.767 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0022418440000000002 2024-09-21T14:58:48.767 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.767 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.768 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:48.768 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967305+0000", 2024-09-21T14:58:48.768 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001785525 2024-09-21T14:58:48.768 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.768 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.768 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:48.768 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967438+0000", 2024-09-21T14:58:48.768 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013284399999999999 2024-09-21T14:58:48.768 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.768 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.768 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:48.768 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967531+0000", 2024-09-21T14:58:48.768 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.3140000000000006e-05 2024-09-21T14:58:48.768 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:48.769 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:48.769 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:48.769 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.769 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.769 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T14:58:48.769 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.955387+0000", 2024-09-21T14:58:48.769 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 44.798732528999999, 2024-09-21T14:58:48.769 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012319478999999999, 2024-09-21T14:58:48.769 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:48.769 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:48.769 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:48.769 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:48.769 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:48.769 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 114 2024-09-21T14:58:48.770 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.770 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:48.770 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.770 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:48.770 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955387+0000", 2024-09-21T14:58:48.770 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:48.770 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.770 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.770 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:48.770 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955387+0000", 2024-09-21T14:58:48.770 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:48.770 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.770 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.770 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:48.770 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955386+0000", 2024-09-21T14:58:48.771 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:58:48.771 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.771 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.771 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:48.771 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955392+0000", 2024-09-21T14:58:48.771 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4740000000000002e-06 2024-09-21T14:58:48.771 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.771 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.771 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:48.771 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955400+0000", 2024-09-21T14:58:48.771 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2139999999999996e-06 2024-09-21T14:58:48.771 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.771 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.771 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:48.772 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955407+0000", 2024-09-21T14:58:48.772 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2299999999999996e-06 2024-09-21T14:58:48.772 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.772 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.772 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:48.772 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.964840+0000", 2024-09-21T14:58:48.772 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0094330079999999997 2024-09-21T14:58:48.772 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.772 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.772 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:48.772 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.964917+0000", 2024-09-21T14:58:48.772 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7071999999999997e-05 2024-09-21T14:58:48.772 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.772 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.772 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:48.773 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.965026+0000", 2024-09-21T14:58:48.773 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000109646 2024-09-21T14:58:48.773 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.773 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.773 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:48.773 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967609+0000", 2024-09-21T14:58:48.773 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0025824929999999999 2024-09-21T14:58:48.773 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.773 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.773 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:48.773 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967629+0000", 2024-09-21T14:58:48.773 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0486999999999999e-05 2024-09-21T14:58:48.773 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.773 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.773 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:48.774 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967672+0000", 2024-09-21T14:58:48.774 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2314999999999997e-05 2024-09-21T14:58:48.774 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.774 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.774 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:48.774 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967706+0000", 2024-09-21T14:58:48.774 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4483999999999997e-05 2024-09-21T14:58:48.774 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:48.774 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:48.774 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:48.774 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.774 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.774 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T14:58:48.774 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.955609+0000", 2024-09-21T14:58:48.774 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 44.798510723, 2024-09-21T14:58:48.775 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013624153999999999, 2024-09-21T14:58:48.775 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:48.775 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:48.775 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:48.775 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:48.775 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:48.775 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 115 2024-09-21T14:58:48.775 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.775 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:48.775 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.775 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:48.775 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955609+0000", 2024-09-21T14:58:48.775 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:48.776 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.776 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.776 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:48.776 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955609+0000", 2024-09-21T14:58:48.776 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:48.776 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.776 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.776 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:48.776 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955607+0000", 2024-09-21T14:58:48.776 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-21T14:58:48.776 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.776 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.776 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:48.776 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955732+0000", 2024-09-21T14:58:48.777 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000125566 2024-09-21T14:58:48.777 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.777 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.777 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:48.777 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955752+0000", 2024-09-21T14:58:48.777 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9916999999999999e-05 2024-09-21T14:58:48.777 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.777 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.777 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:48.777 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955767+0000", 2024-09-21T14:58:48.777 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4687e-05 2024-09-21T14:58:48.777 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.777 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.777 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:48.777 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.965574+0000", 2024-09-21T14:58:48.778 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0098072850000000007 2024-09-21T14:58:48.778 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.778 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.778 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:48.778 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.965642+0000", 2024-09-21T14:58:48.778 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7784e-05 2024-09-21T14:58:48.778 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.778 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.778 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:48.778 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.965753+0000", 2024-09-21T14:58:48.778 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000111111 2024-09-21T14:58:48.778 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.778 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.778 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:48.778 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967717+0000", 2024-09-21T14:58:48.778 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0019635339999999999 2024-09-21T14:58:48.779 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.779 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.779 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:48.779 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.969014+0000", 2024-09-21T14:58:48.779 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001297093 2024-09-21T14:58:48.779 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.779 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.779 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:48.779 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.969130+0000", 2024-09-21T14:58:48.779 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000116011 2024-09-21T14:58:48.779 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.779 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.779 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:48.779 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.969233+0000", 2024-09-21T14:58:48.780 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000102982 2024-09-21T14:58:48.780 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:48.780 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:48.780 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:48.780 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.780 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.780 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T14:58:48.780 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.955783+0000", 2024-09-21T14:58:48.780 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 44.798336054000004, 2024-09-21T14:58:48.780 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014630241, 2024-09-21T14:58:48.780 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:48.780 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:48.780 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:48.780 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:48.781 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:48.781 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 116 2024-09-21T14:58:48.781 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.781 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:48.781 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.781 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:48.781 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955783+0000", 2024-09-21T14:58:48.781 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:48.781 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.781 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.781 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:48.781 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955783+0000", 2024-09-21T14:58:48.781 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:48.781 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.782 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.782 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:48.782 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955782+0000", 2024-09-21T14:58:48.782 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:58:48.782 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.782 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.782 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:48.782 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955791+0000", 2024-09-21T14:58:48.782 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8089999999999999e-06 2024-09-21T14:58:48.782 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.782 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.782 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:48.782 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955804+0000", 2024-09-21T14:58:48.783 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2548000000000001e-05 2024-09-21T14:58:48.783 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.783 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.783 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:48.783 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955814+0000", 2024-09-21T14:58:48.783 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0158e-05 2024-09-21T14:58:48.783 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.783 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.783 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:48.783 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.966515+0000", 2024-09-21T14:58:48.783 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.010701756 2024-09-21T14:58:48.783 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.783 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.784 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:48.784 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.966606+0000", 2024-09-21T14:58:48.784 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0761999999999996e-05 2024-09-21T14:58:48.784 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.784 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.784 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:48.784 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.966740+0000", 2024-09-21T14:58:48.784 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000133693 2024-09-21T14:58:48.784 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.784 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.784 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:48.784 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.968920+0000", 2024-09-21T14:58:48.784 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0021799290000000002 2024-09-21T14:58:48.784 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.785 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.785 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:48.785 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.970320+0000", 2024-09-21T14:58:48.785 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0014001759999999999 2024-09-21T14:58:48.785 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.785 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.785 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:48.785 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.970376+0000", 2024-09-21T14:58:48.785 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5696999999999999e-05 2024-09-21T14:58:48.785 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.785 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.785 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:48.785 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.970413+0000", 2024-09-21T14:58:48.785 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7750999999999998e-05 2024-09-21T14:58:48.785 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:48.786 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:48.786 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:48.786 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.786 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.786 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T14:58:48.786 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.955829+0000", 2024-09-21T14:58:48.786 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 44.798290584, 2024-09-21T14:58:48.786 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014919597999999999, 2024-09-21T14:58:48.786 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:48.786 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:48.786 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:48.786 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:48.786 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:48.787 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 117 2024-09-21T14:58:48.787 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.787 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:48.787 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.787 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:48.787 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955829+0000", 2024-09-21T14:58:48.787 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:48.787 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.787 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.787 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:48.787 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955829+0000", 2024-09-21T14:58:48.787 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:48.787 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.787 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.788 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:48.788 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955827+0000", 2024-09-21T14:58:48.788 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:48.788 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.788 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.788 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:48.788 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955838+0000", 2024-09-21T14:58:48.788 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.111e-05 2024-09-21T14:58:48.788 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.788 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.788 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:48.788 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955875+0000", 2024-09-21T14:58:48.788 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6211999999999998e-05 2024-09-21T14:58:48.789 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.789 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.789 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:48.789 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955899+0000", 2024-09-21T14:58:48.789 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4128999999999999e-05 2024-09-21T14:58:48.789 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.789 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.789 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:48.789 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967767+0000", 2024-09-21T14:58:48.789 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.011868251999999999 2024-09-21T14:58:48.789 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.789 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.789 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:48.789 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967866+0000", 2024-09-21T14:58:48.789 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8664000000000003e-05 2024-09-21T14:58:48.790 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.790 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.790 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:48.790 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.968128+0000", 2024-09-21T14:58:48.790 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00026243600000000001 2024-09-21T14:58:48.790 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.790 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.790 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:48.790 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.969393+0000", 2024-09-21T14:58:48.790 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0012650350000000001 2024-09-21T14:58:48.790 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.790 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.790 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:48.790 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.970523+0000", 2024-09-21T14:58:48.790 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001129557 2024-09-21T14:58:48.791 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.791 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.791 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:48.791 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.970603+0000", 2024-09-21T14:58:48.791 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0090000000000001e-05 2024-09-21T14:58:48.791 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.791 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.791 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:48.791 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.970748+0000", 2024-09-21T14:58:48.791 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014552799999999999 2024-09-21T14:58:48.791 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:48.791 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:48.791 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:48.791 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.791 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.792 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T14:58:48.792 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:04.493315+0000", 2024-09-21T14:58:48.792 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 44.260803750999997, 2024-09-21T14:58:48.792 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014055766000000001, 2024-09-21T14:58:48.792 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:48.792 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:48.792 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:48.792 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:48.792 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:48.792 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 137 2024-09-21T14:58:48.792 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.792 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:48.792 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.792 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:48.792 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493315+0000", 2024-09-21T14:58:48.792 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:48.793 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.793 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.793 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:48.793 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493315+0000", 2024-09-21T14:58:48.793 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:48.793 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.793 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.793 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:48.793 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493314+0000", 2024-09-21T14:58:48.793 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:58:48.793 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.793 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.793 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:48.793 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493507+0000", 2024-09-21T14:58:48.794 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000192444 2024-09-21T14:58:48.794 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.794 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.794 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:48.794 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493528+0000", 2024-09-21T14:58:48.794 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0845000000000001e-05 2024-09-21T14:58:48.794 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.794 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.794 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:48.794 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493543+0000", 2024-09-21T14:58:48.794 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5800000000000001e-05 2024-09-21T14:58:48.794 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.794 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.794 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:48.795 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.502575+0000", 2024-09-21T14:58:48.795 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0090312389999999999 2024-09-21T14:58:48.795 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.795 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.795 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:48.795 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.502677+0000", 2024-09-21T14:58:48.795 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000102271 2024-09-21T14:58:48.795 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.795 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.795 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:48.795 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.502825+0000", 2024-09-21T14:58:48.795 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000148171 2024-09-21T14:58:48.795 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.795 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.796 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:48.796 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507251+0000", 2024-09-21T14:58:48.796 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0044253879999999997 2024-09-21T14:58:48.796 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.796 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.796 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:48.796 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507313+0000", 2024-09-21T14:58:48.796 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2170999999999998e-05 2024-09-21T14:58:48.796 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.796 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.796 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:48.796 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507346+0000", 2024-09-21T14:58:48.796 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3079e-05 2024-09-21T14:58:48.796 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.796 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.797 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:48.797 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507371+0000", 2024-09-21T14:58:48.797 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5477999999999999e-05 2024-09-21T14:58:48.797 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:48.797 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:48.797 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:48.797 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.797 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.797 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T14:58:48.797 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:04.493570+0000", 2024-09-21T14:58:48.797 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 44.260549345999998, 2024-09-21T14:58:48.797 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014061686, 2024-09-21T14:58:48.797 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:48.797 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:48.797 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:48.797 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:48.798 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:48.798 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 138 2024-09-21T14:58:48.798 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.798 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:48.798 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.798 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:48.798 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493570+0000", 2024-09-21T14:58:48.798 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:48.798 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.798 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.798 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:48.798 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493570+0000", 2024-09-21T14:58:48.798 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:48.798 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.799 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.799 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:48.799 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493568+0000", 2024-09-21T14:58:48.799 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:48.799 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.799 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.799 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:48.799 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493579+0000", 2024-09-21T14:58:48.799 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0574000000000001e-05 2024-09-21T14:58:48.799 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.799 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.799 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:48.799 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493595+0000", 2024-09-21T14:58:48.799 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6504999999999999e-05 2024-09-21T14:58:48.799 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.800 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.800 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:48.800 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493608+0000", 2024-09-21T14:58:48.800 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2836999999999999e-05 2024-09-21T14:58:48.800 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.800 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.800 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:48.800 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.503909+0000", 2024-09-21T14:58:48.800 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.01030118 2024-09-21T14:58:48.800 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.800 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.800 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:48.800 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.504064+0000", 2024-09-21T14:58:48.800 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015493000000000001 2024-09-21T14:58:48.800 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.801 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.801 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:48.801 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.504299+0000", 2024-09-21T14:58:48.801 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00023416100000000001 2024-09-21T14:58:48.801 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.801 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.801 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:48.801 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507466+0000", 2024-09-21T14:58:48.801 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0031678380000000001 2024-09-21T14:58:48.801 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.801 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.801 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:48.801 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507483+0000", 2024-09-21T14:58:48.801 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6606999999999999e-05 2024-09-21T14:58:48.801 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.801 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.802 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:48.802 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507566+0000", 2024-09-21T14:58:48.802 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3110999999999998e-05 2024-09-21T14:58:48.802 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.802 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.802 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:48.802 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507632+0000", 2024-09-21T14:58:48.802 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5480000000000003e-05 2024-09-21T14:58:48.802 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:48.802 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:48.802 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:48.802 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.802 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.803 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T14:58:48.803 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:04.493625+0000", 2024-09-21T14:58:48.803 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 44.260494657999999, 2024-09-21T14:58:48.803 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014665378999999999, 2024-09-21T14:58:48.803 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:48.803 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:48.803 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:48.803 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:48.803 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:48.803 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139 2024-09-21T14:58:48.803 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.803 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:48.803 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.804 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:48.804 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493625+0000", 2024-09-21T14:58:48.804 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:48.804 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.804 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.804 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:48.804 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493625+0000", 2024-09-21T14:58:48.804 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:48.804 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.804 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.804 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:48.804 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493624+0000", 2024-09-21T14:58:48.804 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:58:48.804 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.804 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.804 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:48.805 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493638+0000", 2024-09-21T14:58:48.805 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4282e-05 2024-09-21T14:58:48.805 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.805 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.805 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:48.805 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493654+0000", 2024-09-21T14:58:48.805 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5739999999999998e-05 2024-09-21T14:58:48.805 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.805 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.805 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:48.805 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493666+0000", 2024-09-21T14:58:48.805 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2865e-05 2024-09-21T14:58:48.805 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.805 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.805 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:48.805 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.505329+0000", 2024-09-21T14:58:48.806 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.011662237000000001 2024-09-21T14:58:48.806 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.806 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.806 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:48.806 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.505377+0000", 2024-09-21T14:58:48.806 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7862999999999999e-05 2024-09-21T14:58:48.806 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.806 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.806 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:48.806 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.505455+0000", 2024-09-21T14:58:48.806 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8918000000000006e-05 2024-09-21T14:58:48.806 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.806 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.806 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:48.806 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507650+0000", 2024-09-21T14:58:48.807 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.002195027 2024-09-21T14:58:48.807 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.807 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.807 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:48.807 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508149+0000", 2024-09-21T14:58:48.807 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00049828399999999999 2024-09-21T14:58:48.807 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.807 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.807 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:48.807 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508230+0000", 2024-09-21T14:58:48.807 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1376000000000002e-05 2024-09-21T14:58:48.807 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.807 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.807 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:48.807 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508290+0000", 2024-09-21T14:58:48.807 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9852999999999998e-05 2024-09-21T14:58:48.808 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:48.808 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:48.808 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:48.808 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.808 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.808 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T14:58:48.808 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:04.493679+0000", 2024-09-21T14:58:48.808 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 44.260439759999997, 2024-09-21T14:58:48.808 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014869869000000001, 2024-09-21T14:58:48.808 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:48.808 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:48.808 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:48.808 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:48.808 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:48.808 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140 2024-09-21T14:58:48.809 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.809 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:48.809 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.809 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:48.809 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493679+0000", 2024-09-21T14:58:48.809 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:48.809 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.809 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.809 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:48.809 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493679+0000", 2024-09-21T14:58:48.809 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:48.809 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.809 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.809 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:48.809 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493679+0000", 2024-09-21T14:58:48.809 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:58:48.810 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.810 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.810 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:48.810 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493889+0000", 2024-09-21T14:58:48.810 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00021032700000000001 2024-09-21T14:58:48.810 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.810 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.810 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:48.810 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493931+0000", 2024-09-21T14:58:48.810 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1971000000000002e-05 2024-09-21T14:58:48.810 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.810 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.810 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:48.810 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493947+0000", 2024-09-21T14:58:48.811 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6385e-05 2024-09-21T14:58:48.811 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.811 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.811 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:48.811 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.505804+0000", 2024-09-21T14:58:48.811 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.011857136000000001 2024-09-21T14:58:48.811 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.811 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.811 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:48.811 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.505992+0000", 2024-09-21T14:58:48.811 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000187665 2024-09-21T14:58:48.811 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.811 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.811 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:48.811 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.506281+0000", 2024-09-21T14:58:48.811 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000288712 2024-09-21T14:58:48.812 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.812 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.812 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:48.812 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508349+0000", 2024-09-21T14:58:48.812 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0020685790000000001 2024-09-21T14:58:48.812 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.812 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.812 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:48.812 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508371+0000", 2024-09-21T14:58:48.812 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1747e-05 2024-09-21T14:58:48.812 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.812 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.812 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:48.812 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508450+0000", 2024-09-21T14:58:48.812 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9332000000000005e-05 2024-09-21T14:58:48.812 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.813 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.813 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:48.813 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508549+0000", 2024-09-21T14:58:48.813 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8918000000000004e-05 2024-09-21T14:58:48.813 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:48.813 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:48.813 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:48.813 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.813 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.813 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T14:58:48.813 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:04.493967+0000", 2024-09-21T14:58:48.813 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 44.260152722999997, 2024-09-21T14:58:48.813 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.015281075999999999, 2024-09-21T14:58:48.813 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:48.813 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:48.813 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:48.814 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:48.814 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:48.814 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 141 2024-09-21T14:58:48.814 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.814 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:48.814 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.814 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:48.814 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493967+0000", 2024-09-21T14:58:48.814 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:48.814 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.814 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.814 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:48.814 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493967+0000", 2024-09-21T14:58:48.814 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:48.815 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.815 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.815 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:48.815 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493965+0000", 2024-09-21T14:58:48.815 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:48.815 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.815 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.815 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:48.815 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493987+0000", 2024-09-21T14:58:48.815 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1662999999999999e-05 2024-09-21T14:58:48.815 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.815 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.815 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:48.815 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.494004+0000", 2024-09-21T14:58:48.815 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7311000000000001e-05 2024-09-21T14:58:48.815 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.816 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.816 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:48.816 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.494017+0000", 2024-09-21T14:58:48.816 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3202e-05 2024-09-21T14:58:48.816 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.816 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.816 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:48.816 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507678+0000", 2024-09-21T14:58:48.816 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013660432 2024-09-21T14:58:48.816 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.816 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.816 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:48.816 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507726+0000", 2024-09-21T14:58:48.816 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8154e-05 2024-09-21T14:58:48.816 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.816 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.817 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:48.817 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507797+0000", 2024-09-21T14:58:48.817 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.1110000000000002e-05 2024-09-21T14:58:48.817 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.817 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.817 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:48.817 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508710+0000", 2024-09-21T14:58:48.817 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00091339299999999997 2024-09-21T14:58:48.817 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.817 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.817 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:48.817 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.509108+0000", 2024-09-21T14:58:48.817 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00039718300000000001 2024-09-21T14:58:48.817 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.818 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.818 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:48.818 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.509186+0000", 2024-09-21T14:58:48.818 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8592999999999998e-05 2024-09-21T14:58:48.818 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.818 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.818 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:48.818 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.509248+0000", 2024-09-21T14:58:48.818 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1450000000000003e-05 2024-09-21T14:58:48.818 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:48.818 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:48.818 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:48.818 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.819 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.819 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6626 3.1a 3:5adee3e5:::rbd_data.11361bed1f78.0000000000000001:d [read 229376~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:48.819 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131593+0000", 2024-09-21T14:58:48.819 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 13.622526145, 2024-09-21T14:58:48.819 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012576725, 2024-09-21T14:58:48.819 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:48.819 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:48.819 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:48.819 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:48.819 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:48.819 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6626 2024-09-21T14:58:48.819 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.819 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:48.819 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.819 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:48.820 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131593+0000", 2024-09-21T14:58:48.820 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:48.820 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.820 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.820 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:48.820 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131593+0000", 2024-09-21T14:58:48.820 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:48.820 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.820 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.820 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:48.820 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131593+0000", 2024-09-21T14:58:48.820 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T14:58:48.820 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.820 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.820 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:48.820 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131598+0000", 2024-09-21T14:58:48.821 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7849999999999999e-06 2024-09-21T14:58:48.821 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.821 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.821 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:48.821 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131601+0000", 2024-09-21T14:58:48.821 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1949999999999999e-06 2024-09-21T14:58:48.821 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.821 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.821 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:48.821 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131604+0000", 2024-09-21T14:58:48.821 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6350000000000002e-06 2024-09-21T14:58:48.821 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.821 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.821 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:48.822 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.143902+0000", 2024-09-21T14:58:48.822 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012298216000000001 2024-09-21T14:58:48.822 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.822 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.822 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:48.822 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.143993+0000", 2024-09-21T14:58:48.822 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1073e-05 2024-09-21T14:58:48.822 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.822 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.822 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:48.822 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.144170+0000", 2024-09-21T14:58:48.822 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017703 2024-09-21T14:58:48.822 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:48.822 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:48.823 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:48.823 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.823 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.823 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6631 3.1a 3:5d547f54:::rbd_data.11361bed1f78.0000000000000003:d [read 0~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:48.823 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131606+0000", 2024-09-21T14:58:48.823 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 13.622513444999999, 2024-09-21T14:58:48.823 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013903148000000001, 2024-09-21T14:58:48.823 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:48.823 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:48.823 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:48.823 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:48.823 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:48.823 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6631 2024-09-21T14:58:48.823 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.823 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:48.824 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.824 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:48.824 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131606+0000", 2024-09-21T14:58:48.824 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:48.824 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.824 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.824 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:48.824 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131606+0000", 2024-09-21T14:58:48.824 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:48.824 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.824 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.824 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:48.824 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131606+0000", 2024-09-21T14:58:48.824 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T14:58:48.824 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.825 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.825 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:48.825 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131607+0000", 2024-09-21T14:58:48.825 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2699999999999999e-06 2024-09-21T14:58:48.825 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.825 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.825 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:48.825 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131610+0000", 2024-09-21T14:58:48.825 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.88e-06 2024-09-21T14:58:48.825 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.825 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.825 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:48.825 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131612+0000", 2024-09-21T14:58:48.825 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5170000000000002e-06 2024-09-21T14:58:48.825 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.825 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.825 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:48.826 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.144205+0000", 2024-09-21T14:58:48.826 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012592763 2024-09-21T14:58:48.826 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.826 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.826 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:48.826 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.144332+0000", 2024-09-21T14:58:48.826 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012694200000000001 2024-09-21T14:58:48.826 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.826 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.826 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:48.826 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.145509+0000", 2024-09-21T14:58:48.826 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001176992 2024-09-21T14:58:48.826 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:48.826 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:48.827 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:48.827 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.827 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.827 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6632 3.1a 3:5d547f54:::rbd_data.11361bed1f78.0000000000000003:d [read 81920~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:48.827 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131614+0000", 2024-09-21T14:58:48.827 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 13.622504836999999, 2024-09-21T14:58:48.827 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014093553999999999, 2024-09-21T14:58:48.827 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:48.827 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:48.827 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:48.827 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:48.827 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:48.827 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6632 2024-09-21T14:58:48.827 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.827 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:48.828 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.828 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:48.828 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131614+0000", 2024-09-21T14:58:48.828 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:48.828 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.828 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.828 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:48.828 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131614+0000", 2024-09-21T14:58:48.828 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:48.828 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.828 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.828 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:48.828 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131614+0000", 2024-09-21T14:58:48.828 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T14:58:48.828 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.828 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.828 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:48.829 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131615+0000", 2024-09-21T14:58:48.829 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.218e-06 2024-09-21T14:58:48.829 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.829 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.829 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:48.829 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131618+0000", 2024-09-21T14:58:48.829 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9050000000000001e-06 2024-09-21T14:58:48.829 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.829 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.829 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:48.829 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131621+0000", 2024-09-21T14:58:48.829 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7439999999999999e-06 2024-09-21T14:58:48.829 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.829 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.829 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:48.830 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.145538+0000", 2024-09-21T14:58:48.830 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013916618 2024-09-21T14:58:48.830 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.830 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.830 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:48.830 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.145619+0000", 2024-09-21T14:58:48.830 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1812999999999997e-05 2024-09-21T14:58:48.830 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.830 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.830 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:48.830 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.145708+0000", 2024-09-21T14:58:48.830 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8460000000000003e-05 2024-09-21T14:58:48.830 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:48.830 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:48.831 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:48.831 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.831 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.831 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6633 3.1a 3:5d547f54:::rbd_data.11361bed1f78.0000000000000003:d [read 147456~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:48.831 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131623+0000", 2024-09-21T14:58:48.831 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 13.622495871, 2024-09-21T14:58:48.831 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014395845000000001, 2024-09-21T14:58:48.831 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:48.831 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:48.831 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:48.831 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:48.831 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:48.831 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6633 2024-09-21T14:58:48.832 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.832 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:48.832 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.832 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:48.832 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131623+0000", 2024-09-21T14:58:48.832 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:48.832 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.832 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.832 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:48.832 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131623+0000", 2024-09-21T14:58:48.832 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:48.832 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.832 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.832 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:48.832 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131623+0000", 2024-09-21T14:58:48.833 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T14:58:48.833 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.833 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.833 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:48.833 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131628+0000", 2024-09-21T14:58:48.833 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0370000000000002e-06 2024-09-21T14:58:48.833 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.833 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.833 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:48.833 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131632+0000", 2024-09-21T14:58:48.833 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3519999999999998e-06 2024-09-21T14:58:48.833 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.833 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.833 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:48.833 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131634+0000", 2024-09-21T14:58:48.834 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5019999999999999e-06 2024-09-21T14:58:48.834 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.834 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.834 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:48.834 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.145734+0000", 2024-09-21T14:58:48.834 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014100118 2024-09-21T14:58:48.834 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.834 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.834 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:48.834 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.145804+0000", 2024-09-21T14:58:48.834 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0229999999999997e-05 2024-09-21T14:58:48.834 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.834 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.835 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:48.835 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146019+0000", 2024-09-21T14:58:48.835 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00021481699999999999 2024-09-21T14:58:48.835 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:48.835 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:48.835 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:48.835 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.835 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.835 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6634 3.1a 3:5d547f54:::rbd_data.11361bed1f78.0000000000000003:d [read 229376~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:48.835 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131636+0000", 2024-09-21T14:58:48.835 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 13.622482761000001, 2024-09-21T14:58:48.835 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014645843, 2024-09-21T14:58:48.835 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:48.836 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:48.836 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:48.836 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:48.836 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:48.836 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6634 2024-09-21T14:58:48.836 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.836 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:48.836 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.836 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:48.836 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131636+0000", 2024-09-21T14:58:48.836 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:48.836 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.836 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.836 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:48.836 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131636+0000", 2024-09-21T14:58:48.837 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:48.837 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.837 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.837 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:48.837 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131636+0000", 2024-09-21T14:58:48.837 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:48.837 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.837 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.837 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:48.837 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131641+0000", 2024-09-21T14:58:48.837 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8489999999999997e-06 2024-09-21T14:58:48.837 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.837 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.837 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:48.838 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131644+0000", 2024-09-21T14:58:48.838 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.179e-06 2024-09-21T14:58:48.838 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.838 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.838 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:48.838 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131647+0000", 2024-09-21T14:58:48.838 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8009999999999999e-06 2024-09-21T14:58:48.838 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.838 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.838 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:48.838 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146046+0000", 2024-09-21T14:58:48.838 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014399018 2024-09-21T14:58:48.838 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.839 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.839 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:48.839 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146118+0000", 2024-09-21T14:58:48.839 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2044000000000003e-05 2024-09-21T14:58:48.839 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.839 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.839 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:48.839 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146282+0000", 2024-09-21T14:58:48.839 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000164196 2024-09-21T14:58:48.839 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:48.839 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:48.839 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:48.839 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.839 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.840 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6639 3.1a 3:592d80ed:::rbd_data.11361bed1f78.0000000000000005:d [read 0~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:48.840 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131650+0000", 2024-09-21T14:58:48.840 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 13.622469561999999, 2024-09-21T14:58:48.840 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016002832000000002, 2024-09-21T14:58:48.840 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:48.840 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:48.840 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:48.840 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:48.840 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:48.840 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6639 2024-09-21T14:58:48.840 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.840 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:48.840 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.840 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:48.841 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131650+0000", 2024-09-21T14:58:48.841 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:48.841 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.841 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.841 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:48.841 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131650+0000", 2024-09-21T14:58:48.841 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:48.841 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.841 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.841 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:48.841 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131649+0000", 2024-09-21T14:58:48.841 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:48.841 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.841 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.841 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:48.841 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131651+0000", 2024-09-21T14:58:48.841 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9199999999999998e-06 2024-09-21T14:58:48.842 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.842 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.842 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:48.842 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131656+0000", 2024-09-21T14:58:48.842 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9899999999999997e-06 2024-09-21T14:58:48.842 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.842 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.842 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:48.842 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131665+0000", 2024-09-21T14:58:48.842 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0999999999999993e-06 2024-09-21T14:58:48.842 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.842 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.842 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:48.842 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146309+0000", 2024-09-21T14:58:48.842 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014643893999999999 2024-09-21T14:58:48.843 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.843 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.843 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:48.843 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146392+0000", 2024-09-21T14:58:48.843 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2531e-05 2024-09-21T14:58:48.843 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.843 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.843 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:48.843 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147653+0000", 2024-09-21T14:58:48.843 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0012606710000000001 2024-09-21T14:58:48.843 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:48.843 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:48.843 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:48.843 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.843 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.843 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6640 3.1a 3:592d80ed:::rbd_data.11361bed1f78.0000000000000005:d [read 65536~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:48.844 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131670+0000", 2024-09-21T14:58:48.844 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 13.622449263, 2024-09-21T14:58:48.844 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016188859, 2024-09-21T14:58:48.844 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:48.844 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:48.844 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:48.844 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:48.844 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:48.844 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6640 2024-09-21T14:58:48.844 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.844 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:48.844 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.844 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:48.844 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131670+0000", 2024-09-21T14:58:48.844 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:48.844 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.845 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.845 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:48.845 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131670+0000", 2024-09-21T14:58:48.845 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:48.845 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.845 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.845 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:48.845 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131670+0000", 2024-09-21T14:58:48.845 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T14:58:48.845 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.845 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.845 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:48.845 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131672+0000", 2024-09-21T14:58:48.845 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.818e-06 2024-09-21T14:58:48.845 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.845 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.846 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:48.846 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131675+0000", 2024-09-21T14:58:48.846 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5930000000000002e-06 2024-09-21T14:58:48.846 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.846 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.846 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:48.846 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131678+0000", 2024-09-21T14:58:48.846 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6790000000000001e-06 2024-09-21T14:58:48.846 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.846 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.846 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:48.846 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147681+0000", 2024-09-21T14:58:48.846 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016002755 2024-09-21T14:58:48.846 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.847 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.847 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:48.847 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147762+0000", 2024-09-21T14:58:48.847 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1052999999999998e-05 2024-09-21T14:58:48.847 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.847 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.847 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:48.847 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147859+0000", 2024-09-21T14:58:48.847 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.7199000000000002e-05 2024-09-21T14:58:48.847 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:48.847 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:48.847 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:48.847 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.847 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.847 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6641 3.1a 3:592d80ed:::rbd_data.11361bed1f78.0000000000000005:d [read 147456~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:48.847 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131680+0000", 2024-09-21T14:58:48.848 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 13.622438975, 2024-09-21T14:58:48.848 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016431296000000001, 2024-09-21T14:58:48.848 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:48.848 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:48.848 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:48.848 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:48.848 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:48.848 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6641 2024-09-21T14:58:48.848 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.848 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:48.848 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.848 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:48.848 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131680+0000", 2024-09-21T14:58:48.848 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:48.848 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.849 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.849 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:48.849 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131680+0000", 2024-09-21T14:58:48.849 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:48.849 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.849 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.849 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:48.849 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131680+0000", 2024-09-21T14:58:48.849 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T14:58:48.849 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.849 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.849 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:48.849 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131681+0000", 2024-09-21T14:58:48.849 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2610000000000001e-06 2024-09-21T14:58:48.849 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.849 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.850 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:48.850 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131685+0000", 2024-09-21T14:58:48.850 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.8299999999999998e-06 2024-09-21T14:58:48.850 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.850 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.850 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:48.850 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131688+0000", 2024-09-21T14:58:48.850 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6189999999999998e-06 2024-09-21T14:58:48.850 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.850 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.850 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:48.850 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147884+0000", 2024-09-21T14:58:48.850 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016196465 2024-09-21T14:58:48.850 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.850 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.851 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:48.851 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147949+0000", 2024-09-21T14:58:48.851 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4867000000000001e-05 2024-09-21T14:58:48.851 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.851 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.851 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:48.851 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.148112+0000", 2024-09-21T14:58:48.851 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00016247700000000001 2024-09-21T14:58:48.851 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:48.851 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:48.851 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:48.851 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.851 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.851 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6642 3.1a 3:592d80ed:::rbd_data.11361bed1f78.0000000000000005:d [read 212992~32768 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:48.852 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131690+0000", 2024-09-21T14:58:48.852 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 13.622429096999999, 2024-09-21T14:58:48.852 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016751335999999999, 2024-09-21T14:58:48.852 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:48.852 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:48.852 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:48.852 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:48.852 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:48.852 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6642 2024-09-21T14:58:48.852 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.852 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:48.852 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.852 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:48.852 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131690+0000", 2024-09-21T14:58:48.852 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:48.853 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.853 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.853 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:48.853 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131690+0000", 2024-09-21T14:58:48.853 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:48.853 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.853 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.853 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:48.853 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131690+0000", 2024-09-21T14:58:48.853 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T14:58:48.853 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.853 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.853 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:48.853 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131691+0000", 2024-09-21T14:58:48.853 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3120000000000001e-06 2024-09-21T14:58:48.853 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.853 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.854 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:48.854 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131694+0000", 2024-09-21T14:58:48.854 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0070000000000002e-06 2024-09-21T14:58:48.854 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.854 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.854 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:48.854 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131700+0000", 2024-09-21T14:58:48.854 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0800000000000002e-06 2024-09-21T14:58:48.854 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.854 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.854 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:48.854 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.148135+0000", 2024-09-21T14:58:48.854 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016434250000000001 2024-09-21T14:58:48.854 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.854 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.855 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:48.855 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.148204+0000", 2024-09-21T14:58:48.855 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9466000000000002e-05 2024-09-21T14:58:48.855 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:48.855 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:48.855 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:48.855 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.148441+0000", 2024-09-21T14:58:48.855 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00023743700000000001 2024-09-21T14:58:48.855 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:48.855 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:48.855 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:48.855 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:48.855 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:48.855 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:58:48.856 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T14:58:48.857 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T14:58:48.857 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T14:58:48.858 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-21T14:58:48.858 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T14:58:48.858 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T14:58:48.858 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:58:48.852+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:58:48.870 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T14:58:48.873 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:58:48.873 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:58:48.873 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:58:48.873 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:58:48.873 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:58:48.873 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:58:48.873 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:58:48.873 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:58:48.957 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:58:48.956+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:58:48.970 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:58:48.970 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:58:48.970 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T14:58:48.970 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:58:48.980 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T14:58:49.058 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:58:49.056+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:58:49.091 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:58:49.091 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:58:49.091 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T14:58:49.091 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T14:58:49.091 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:58:49.101 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T14:58:49.160 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:58:49.156+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:58:49.197 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T14:58:49.222 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:58:49.222 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T14:58:49.222 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T14:58:49.222 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T14:58:49.222 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.222 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:1454 3.e 3:71be909a:::rbd_data.1136515ecb6b.000000000000002a:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T14:58:49.222 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:30.339995+0000", 2024-09-21T14:58:49.222 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.88032844, 2024-09-21T14:58:49.223 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.033615974999999999, 2024-09-21T14:58:49.223 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:49.223 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:49.223 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:49.223 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:49.223 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:49.223 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1454 2024-09-21T14:58:49.223 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.223 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:49.223 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.223 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:49.223 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.339995+0000", 2024-09-21T14:58:49.223 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:49.224 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.224 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.224 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:49.224 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.339995+0000", 2024-09-21T14:58:49.224 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:49.224 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.224 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.224 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:49.224 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.339992+0000", 2024-09-21T14:58:49.224 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-21T14:58:49.224 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.224 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.224 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:49.224 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.340008+0000", 2024-09-21T14:58:49.225 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5906e-05 2024-09-21T14:58:49.225 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.225 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.225 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:49.225 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.340042+0000", 2024-09-21T14:58:49.225 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4066000000000002e-05 2024-09-21T14:58:49.225 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.225 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.225 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:49.225 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.340056+0000", 2024-09-21T14:58:49.225 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4389999999999999e-05 2024-09-21T14:58:49.225 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.225 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.225 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:49.225 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.340178+0000", 2024-09-21T14:58:49.226 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012202 2024-09-21T14:58:49.226 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.226 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.226 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:49.226 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.340424+0000", 2024-09-21T14:58:49.226 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000245682 2024-09-21T14:58:49.226 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.226 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.226 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 0", 2024-09-21T14:58:49.226 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.348708+0000", 2024-09-21T14:58:49.226 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0082834390000000001 2024-09-21T14:58:49.226 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.226 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.226 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:49.226 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.373176+0000", 2024-09-21T14:58:49.227 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024468888000000001 2024-09-21T14:58:49.227 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.227 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.227 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:49.227 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.373306+0000", 2024-09-21T14:58:49.227 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012991700000000001 2024-09-21T14:58:49.227 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.227 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.227 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:49.227 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.373464+0000", 2024-09-21T14:58:49.227 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000157913 2024-09-21T14:58:49.227 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.227 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.227 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:49.228 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.373611+0000", 2024-09-21T14:58:49.228 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014681699999999999 2024-09-21T14:58:49.228 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:49.228 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:49.228 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:49.228 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.228 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.228 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6468 3.16 3:68aaef8d:::rbd_data.11361bed1f78.000000000000000a:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:49.228 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.119941+0000", 2024-09-21T14:58:49.228 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.100382182000001, 2024-09-21T14:58:49.228 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0060084200000000004, 2024-09-21T14:58:49.228 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:49.228 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:49.228 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:49.228 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:49.229 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:49.229 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6468 2024-09-21T14:58:49.229 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.229 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:49.229 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.229 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:49.229 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.119941+0000", 2024-09-21T14:58:49.229 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:49.229 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.229 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.229 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:49.229 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.119941+0000", 2024-09-21T14:58:49.229 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:49.229 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.229 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.229 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:49.230 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.119941+0000", 2024-09-21T14:58:49.230 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:58:49.230 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.230 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.230 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:49.230 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.119945+0000", 2024-09-21T14:58:49.230 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7999999999999998e-06 2024-09-21T14:58:49.230 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.230 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.230 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:49.230 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.119956+0000", 2024-09-21T14:58:49.230 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0064999999999999e-05 2024-09-21T14:58:49.230 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.230 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.230 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:49.231 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.119963+0000", 2024-09-21T14:58:49.231 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2729999999999997e-06 2024-09-21T14:58:49.231 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.231 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.231 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:49.231 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.119994+0000", 2024-09-21T14:58:49.231 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1721999999999999e-05 2024-09-21T14:58:49.231 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.231 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.231 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting_for_map not empty", 2024-09-21T14:58:49.231 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.119997+0000", 2024-09-21T14:58:49.231 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8509999999999999e-06 2024-09-21T14:58:49.231 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.231 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.231 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:49.231 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.125713+0000", 2024-09-21T14:58:49.232 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0057158529999999999 2024-09-21T14:58:49.232 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.232 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.232 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:49.232 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.125834+0000", 2024-09-21T14:58:49.232 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012065099999999999 2024-09-21T14:58:49.232 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.232 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.232 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:49.232 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.125950+0000", 2024-09-21T14:58:49.232 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000115942 2024-09-21T14:58:49.232 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:49.232 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:49.232 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:49.232 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.232 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.233 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6477 3.16 3:6eb18e6c:::rbd_data.11361bed1f78.0000000000000013:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:49.233 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.119977+0000", 2024-09-21T14:58:49.233 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.100346876, 2024-09-21T14:58:49.233 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0062386309999999997, 2024-09-21T14:58:49.233 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:49.233 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:49.233 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:49.233 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:49.233 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:49.233 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6477 2024-09-21T14:58:49.233 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.233 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:49.233 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.234 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:49.234 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.119977+0000", 2024-09-21T14:58:49.234 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:49.234 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.234 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.234 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:49.234 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.119977+0000", 2024-09-21T14:58:49.234 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:49.234 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.234 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.234 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:49.234 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.119976+0000", 2024-09-21T14:58:49.234 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:49.234 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.234 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.235 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:49.235 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.119980+0000", 2024-09-21T14:58:49.235 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3610000000000001e-06 2024-09-21T14:58:49.235 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.235 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.235 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:49.235 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.119988+0000", 2024-09-21T14:58:49.235 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2630000000000003e-06 2024-09-21T14:58:49.235 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.235 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.235 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:49.235 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.119994+0000", 2024-09-21T14:58:49.235 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4540000000000004e-06 2024-09-21T14:58:49.235 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.235 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.236 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:49.236 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.120013+0000", 2024-09-21T14:58:49.236 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8859999999999999e-05 2024-09-21T14:58:49.236 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.236 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.236 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting_for_map not empty", 2024-09-21T14:58:49.236 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.120016+0000", 2024-09-21T14:58:49.236 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.683e-06 2024-09-21T14:58:49.236 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.236 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.236 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:49.236 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.125992+0000", 2024-09-21T14:58:49.236 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0059764579999999996 2024-09-21T14:58:49.237 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.237 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.237 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:49.237 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.126094+0000", 2024-09-21T14:58:49.237 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000101269 2024-09-21T14:58:49.237 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.237 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.237 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:49.237 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.126215+0000", 2024-09-21T14:58:49.237 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012178799999999999 2024-09-21T14:58:49.237 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:49.237 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:49.237 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:49.237 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.237 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.237 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6484 3.16 3:6eb18e6c:::rbd_data.11361bed1f78.0000000000000013:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:49.238 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.120002+0000", 2024-09-21T14:58:49.238 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.100321876000001, 2024-09-21T14:58:49.238 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0065176710000000001, 2024-09-21T14:58:49.238 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:49.238 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:49.238 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:49.238 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:49.238 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:49.238 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6484 2024-09-21T14:58:49.238 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.238 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:49.238 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.238 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:49.238 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.120002+0000", 2024-09-21T14:58:49.238 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:49.239 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.239 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.239 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:49.239 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.120002+0000", 2024-09-21T14:58:49.239 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:49.239 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.239 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.239 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:49.239 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.120001+0000", 2024-09-21T14:58:49.239 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:49.239 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.239 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.239 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:49.240 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.120004+0000", 2024-09-21T14:58:49.240 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1580000000000001e-06 2024-09-21T14:58:49.240 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.240 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.240 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:49.240 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.120013+0000", 2024-09-21T14:58:49.240 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2560000000000002e-06 2024-09-21T14:58:49.240 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.240 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.240 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:49.240 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.120020+0000", 2024-09-21T14:58:49.240 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6029999999999999e-06 2024-09-21T14:58:49.240 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.240 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.241 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:49.241 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.120053+0000", 2024-09-21T14:58:49.241 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2524999999999998e-05 2024-09-21T14:58:49.241 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.241 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.241 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting_for_map not empty", 2024-09-21T14:58:49.241 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.120059+0000", 2024-09-21T14:58:49.241 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6869999999999997e-06 2024-09-21T14:58:49.241 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.241 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.241 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:49.241 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.126262+0000", 2024-09-21T14:58:49.241 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0062021940000000003 2024-09-21T14:58:49.241 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.241 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.242 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:49.242 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.126391+0000", 2024-09-21T14:58:49.242 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000129011 2024-09-21T14:58:49.242 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.242 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.242 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:49.242 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.126519+0000", 2024-09-21T14:58:49.242 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012872 2024-09-21T14:58:49.242 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:49.242 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:49.242 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:49.242 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.242 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.242 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6498 3.16 3:6eb18e6c:::rbd_data.11361bed1f78.0000000000000013:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:49.242 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.120036+0000", 2024-09-21T14:58:49.243 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.100287959999999, 2024-09-21T14:58:49.243 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0067525479999999997, 2024-09-21T14:58:49.243 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:49.243 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:49.243 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:49.243 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:49.243 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:49.243 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6498 2024-09-21T14:58:49.243 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.243 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:49.243 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.243 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:49.243 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.120036+0000", 2024-09-21T14:58:49.243 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:49.243 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.243 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.243 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:49.244 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.120036+0000", 2024-09-21T14:58:49.244 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:49.244 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.244 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.244 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:49.244 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.120035+0000", 2024-09-21T14:58:49.244 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:49.244 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.244 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.244 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:49.244 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.120039+0000", 2024-09-21T14:58:49.244 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.8569999999999997e-06 2024-09-21T14:58:49.244 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.244 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.244 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:49.244 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.120047+0000", 2024-09-21T14:58:49.245 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2199999999999992e-06 2024-09-21T14:58:49.245 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.245 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.245 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:49.245 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.120060+0000", 2024-09-21T14:58:49.245 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3314e-05 2024-09-21T14:58:49.245 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.245 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.245 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:49.245 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.120086+0000", 2024-09-21T14:58:49.245 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5323000000000001e-05 2024-09-21T14:58:49.245 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.245 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.245 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting_for_map not empty", 2024-09-21T14:58:49.246 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.120088+0000", 2024-09-21T14:58:49.246 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6749999999999998e-06 2024-09-21T14:58:49.246 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.246 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.246 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:49.246 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.126582+0000", 2024-09-21T14:58:49.246 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0064932779999999999 2024-09-21T14:58:49.246 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.246 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.246 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:49.246 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.126692+0000", 2024-09-21T14:58:49.246 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00011026 2024-09-21T14:58:49.246 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.246 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.246 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:49.247 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.126788+0000", 2024-09-21T14:58:49.247 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6201999999999998e-05 2024-09-21T14:58:49.247 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:49.247 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:49.247 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:49.247 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.247 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.247 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6714 3.16 3:68aaef8d:::rbd_data.11361bed1f78.000000000000000a:d [read 245760~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:49.247 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.133641+0000", 2024-09-21T14:58:49.247 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.086682281, 2024-09-21T14:58:49.247 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0059002120000000002, 2024-09-21T14:58:49.247 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:49.247 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:49.247 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:49.248 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:49.248 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:49.248 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6714 2024-09-21T14:58:49.248 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.248 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:49.248 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.248 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:49.248 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133641+0000", 2024-09-21T14:58:49.248 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:49.248 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.248 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.248 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:49.248 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133641+0000", 2024-09-21T14:58:49.249 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:49.249 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.249 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.249 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:49.249 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133641+0000", 2024-09-21T14:58:49.249 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:49.249 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.249 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.249 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:49.249 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133644+0000", 2024-09-21T14:58:49.250 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9069999999999999e-06 2024-09-21T14:58:49.250 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.250 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.250 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:49.250 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133649+0000", 2024-09-21T14:58:49.250 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6130000000000003e-06 2024-09-21T14:58:49.250 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.250 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.250 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:49.250 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133655+0000", 2024-09-21T14:58:49.250 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.083e-06 2024-09-21T14:58:49.250 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.251 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.251 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:49.251 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.139122+0000", 2024-09-21T14:58:49.251 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.005466767 2024-09-21T14:58:49.251 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.251 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.251 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:49.251 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.139193+0000", 2024-09-21T14:58:49.251 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0448e-05 2024-09-21T14:58:49.251 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.251 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.251 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:49.251 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.139541+0000", 2024-09-21T14:58:49.251 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00034885500000000002 2024-09-21T14:58:49.252 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:49.252 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:49.252 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:49.252 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.252 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.252 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6729 3.16 3:6eb18e6c:::rbd_data.11361bed1f78.0000000000000013:d [read 49152~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:49.252 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.133660+0000", 2024-09-21T14:58:49.252 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.086663866, 2024-09-21T14:58:49.252 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0062381800000000003, 2024-09-21T14:58:49.252 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:49.252 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:49.252 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:49.252 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:49.252 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:49.252 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6729 2024-09-21T14:58:49.253 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.253 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:49.253 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.253 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:49.253 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133660+0000", 2024-09-21T14:58:49.253 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:49.253 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.253 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.253 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:49.253 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133660+0000", 2024-09-21T14:58:49.253 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:49.253 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.253 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.253 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:49.254 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133659+0000", 2024-09-21T14:58:49.254 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:49.254 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.254 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.254 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:49.254 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133662+0000", 2024-09-21T14:58:49.254 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.463e-06 2024-09-21T14:58:49.254 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.254 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.254 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:49.254 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133667+0000", 2024-09-21T14:58:49.254 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.344e-06 2024-09-21T14:58:49.254 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.254 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.254 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:49.255 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133672+0000", 2024-09-21T14:58:49.255 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6280000000000004e-06 2024-09-21T14:58:49.255 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.255 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.255 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:49.255 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.139570+0000", 2024-09-21T14:58:49.255 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.005898225 2024-09-21T14:58:49.255 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.255 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.255 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:49.255 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.139651+0000", 2024-09-21T14:58:49.255 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1342999999999997e-05 2024-09-21T14:58:49.256 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.256 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.256 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:49.256 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.139898+0000", 2024-09-21T14:58:49.256 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00024648500000000001 2024-09-21T14:58:49.256 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:49.256 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:49.256 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:49.256 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.256 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.256 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6746 3.16 3:6eb18e6c:::rbd_data.11361bed1f78.0000000000000013:d [read 65536~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:49.256 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.133676+0000", 2024-09-21T14:58:49.256 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.086647795999999, 2024-09-21T14:58:49.256 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0065617979999999998, 2024-09-21T14:58:49.256 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:49.257 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:49.257 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:49.257 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:49.257 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:49.257 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6746 2024-09-21T14:58:49.257 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.257 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:49.257 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.257 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:49.257 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133676+0000", 2024-09-21T14:58:49.257 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:49.257 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.257 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.257 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:49.257 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133676+0000", 2024-09-21T14:58:49.258 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:49.258 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.258 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.258 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:49.258 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133675+0000", 2024-09-21T14:58:49.258 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:49.258 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.258 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.258 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:49.258 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133678+0000", 2024-09-21T14:58:49.258 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3149999999999999e-06 2024-09-21T14:58:49.258 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.258 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.258 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:49.259 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133683+0000", 2024-09-21T14:58:49.259 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4439999999999999e-06 2024-09-21T14:58:49.259 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.259 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.259 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:49.259 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133687+0000", 2024-09-21T14:58:49.259 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1649999999999999e-06 2024-09-21T14:58:49.259 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.259 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.259 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:49.259 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.139932+0000", 2024-09-21T14:58:49.259 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0062448920000000002 2024-09-21T14:58:49.259 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.259 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.259 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:49.260 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.140009+0000", 2024-09-21T14:58:49.260 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6734000000000001e-05 2024-09-21T14:58:49.260 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.260 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.260 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:49.260 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.140238+0000", 2024-09-21T14:58:49.260 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00022854599999999999 2024-09-21T14:58:49.260 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:49.260 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:49.260 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:49.260 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.260 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.260 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6747 3.16 3:6eb18e6c:::rbd_data.11361bed1f78.0000000000000013:d [read 147456~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:49.260 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.133691+0000", 2024-09-21T14:58:49.260 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.086632198, 2024-09-21T14:58:49.261 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.006764936, 2024-09-21T14:58:49.261 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:49.261 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:49.261 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:49.261 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:49.261 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:49.261 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6747 2024-09-21T14:58:49.261 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.261 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:49.261 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.261 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:49.261 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133691+0000", 2024-09-21T14:58:49.261 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:49.261 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.262 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.262 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:49.262 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133691+0000", 2024-09-21T14:58:49.262 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:49.262 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.262 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.262 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:49.262 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133691+0000", 2024-09-21T14:58:49.262 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:49.262 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.262 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.262 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:49.262 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133693+0000", 2024-09-21T14:58:49.262 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2390000000000001e-06 2024-09-21T14:58:49.263 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.263 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.263 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:49.263 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133699+0000", 2024-09-21T14:58:49.263 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2519999999999999e-06 2024-09-21T14:58:49.263 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.263 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.263 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:49.263 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133703+0000", 2024-09-21T14:58:49.263 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3660000000000002e-06 2024-09-21T14:58:49.263 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.263 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.263 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:49.263 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.140270+0000", 2024-09-21T14:58:49.263 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0065673499999999996 2024-09-21T14:58:49.264 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.264 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.264 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:49.264 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.140351+0000", 2024-09-21T14:58:49.264 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0975000000000003e-05 2024-09-21T14:58:49.264 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.264 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.264 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:49.264 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.140456+0000", 2024-09-21T14:58:49.264 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000105086 2024-09-21T14:58:49.264 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:49.264 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:49.264 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:49.265 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.265 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.265 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6748 3.16 3:6eb18e6c:::rbd_data.11361bed1f78.0000000000000013:d [read 212992~32768 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:49.265 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.133707+0000", 2024-09-21T14:58:49.265 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.086616669, 2024-09-21T14:58:49.265 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0071255779999999996, 2024-09-21T14:58:49.265 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:49.265 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:49.265 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:49.265 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:49.265 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:49.265 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6748 2024-09-21T14:58:49.265 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.265 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:49.265 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.266 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:49.266 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133707+0000", 2024-09-21T14:58:49.266 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:49.266 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.266 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.266 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:49.266 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133707+0000", 2024-09-21T14:58:49.266 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:49.266 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.266 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.266 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:49.266 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133707+0000", 2024-09-21T14:58:49.267 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:49.267 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.267 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.267 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:49.267 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133709+0000", 2024-09-21T14:58:49.267 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2419999999999999e-06 2024-09-21T14:58:49.267 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.267 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.267 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:49.267 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133714+0000", 2024-09-21T14:58:49.267 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.361e-06 2024-09-21T14:58:49.267 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.267 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.267 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:49.268 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.133718+0000", 2024-09-21T14:58:49.268 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2549999999999999e-06 2024-09-21T14:58:49.268 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.268 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.268 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:49.268 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.140541+0000", 2024-09-21T14:58:49.268 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0068226040000000003 2024-09-21T14:58:49.268 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.268 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.268 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:49.268 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.140634+0000", 2024-09-21T14:58:49.268 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2748e-05 2024-09-21T14:58:49.268 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.268 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.268 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:49.269 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.140832+0000", 2024-09-21T14:58:49.269 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00019867299999999999 2024-09-21T14:58:49.269 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:49.269 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:49.269 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:49.269 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.269 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.269 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9390 3.e 3:730ca76c:::rbd_data.11363a21d2be.0000000000000004:10 [read 0~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:49.269 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.160531+0000", 2024-09-21T14:58:49.269 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.059793056, 2024-09-21T14:58:49.269 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0059367109999999999, 2024-09-21T14:58:49.269 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:49.269 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:49.269 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:49.269 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:49.270 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:49.270 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9390 2024-09-21T14:58:49.270 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.270 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:49.270 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.270 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:49.270 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160531+0000", 2024-09-21T14:58:49.270 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:49.270 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.270 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.270 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:49.270 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160531+0000", 2024-09-21T14:58:49.270 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:49.270 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.270 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.271 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:49.271 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160530+0000", 2024-09-21T14:58:49.271 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:49.271 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.271 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.271 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:49.271 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160533+0000", 2024-09-21T14:58:49.271 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4880000000000001e-06 2024-09-21T14:58:49.271 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.271 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.271 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:49.271 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160539+0000", 2024-09-21T14:58:49.271 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9359999999999996e-06 2024-09-21T14:58:49.271 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.272 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.272 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:49.272 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160543+0000", 2024-09-21T14:58:49.272 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6609999999999999e-06 2024-09-21T14:58:49.272 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.272 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.272 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:49.272 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.166238+0000", 2024-09-21T14:58:49.272 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0056944309999999998 2024-09-21T14:58:49.272 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.272 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.272 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:49.272 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.166299+0000", 2024-09-21T14:58:49.272 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0977000000000002e-05 2024-09-21T14:58:49.272 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.272 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.273 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:49.273 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.166467+0000", 2024-09-21T14:58:49.273 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00016854899999999999 2024-09-21T14:58:49.273 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:49.273 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:49.273 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:49.273 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.273 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.273 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9391 3.e 3:730ca76c:::rbd_data.11363a21d2be.0000000000000004:10 [read 106496~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:49.273 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.160547+0000", 2024-09-21T14:58:49.273 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.059776541, 2024-09-21T14:58:49.273 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0064124159999999998, 2024-09-21T14:58:49.273 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:49.273 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:49.274 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:49.274 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:49.274 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:49.274 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9391 2024-09-21T14:58:49.274 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.274 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:49.274 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.274 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:49.274 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160547+0000", 2024-09-21T14:58:49.274 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:49.274 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.274 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.274 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:49.274 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160547+0000", 2024-09-21T14:58:49.275 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:49.275 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.275 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.275 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:49.275 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160547+0000", 2024-09-21T14:58:49.275 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:49.275 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.275 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.275 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:49.275 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160549+0000", 2024-09-21T14:58:49.275 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3369999999999998e-06 2024-09-21T14:58:49.275 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.275 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.276 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:49.276 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160557+0000", 2024-09-21T14:58:49.276 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6490000000000008e-06 2024-09-21T14:58:49.276 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.276 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.276 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:49.276 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160562+0000", 2024-09-21T14:58:49.276 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8999999999999997e-06 2024-09-21T14:58:49.276 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.276 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.276 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:49.276 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.166612+0000", 2024-09-21T14:58:49.276 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0060500780000000004 2024-09-21T14:58:49.276 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.277 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.277 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:49.277 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.166723+0000", 2024-09-21T14:58:49.277 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00011092899999999999 2024-09-21T14:58:49.277 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.277 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.277 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:49.277 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.166959+0000", 2024-09-21T14:58:49.277 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00023686700000000001 2024-09-21T14:58:49.277 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:49.277 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:49.277 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:49.277 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.277 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.278 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9392 3.e 3:730ca76c:::rbd_data.11363a21d2be.0000000000000004:10 [read 139264~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:49.278 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.160566+0000", 2024-09-21T14:58:49.278 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.059757827, 2024-09-21T14:58:49.278 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0067674689999999999, 2024-09-21T14:58:49.278 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:49.278 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:49.278 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:49.278 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:49.278 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:49.278 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9392 2024-09-21T14:58:49.278 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.278 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:49.278 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.279 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:49.279 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160566+0000", 2024-09-21T14:58:49.279 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:49.279 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.279 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.279 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:49.279 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160566+0000", 2024-09-21T14:58:49.279 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:49.279 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.279 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.279 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:49.279 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160565+0000", 2024-09-21T14:58:49.279 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:49.279 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.279 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.280 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:49.280 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160568+0000", 2024-09-21T14:58:49.280 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.367e-06 2024-09-21T14:58:49.280 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.280 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.280 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:49.280 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160575+0000", 2024-09-21T14:58:49.280 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8669999999999996e-06 2024-09-21T14:58:49.280 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.280 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.280 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:49.280 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160581+0000", 2024-09-21T14:58:49.280 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.93e-06 2024-09-21T14:58:49.280 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.281 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.281 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:49.281 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.166990+0000", 2024-09-21T14:58:49.281 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0064090299999999996 2024-09-21T14:58:49.281 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.281 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.281 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:49.281 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.167097+0000", 2024-09-21T14:58:49.281 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000107841 2024-09-21T14:58:49.281 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.281 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.281 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:49.281 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.167333+0000", 2024-09-21T14:58:49.281 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00023578299999999999 2024-09-21T14:58:49.281 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:49.282 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:49.282 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:49.282 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.282 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.282 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9393 3.e 3:730ca76c:::rbd_data.11363a21d2be.0000000000000004:10 [read 245760~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:49.282 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.160585+0000", 2024-09-21T14:58:49.282 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.059738611, 2024-09-21T14:58:49.282 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0071208850000000004, 2024-09-21T14:58:49.282 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:49.282 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:49.282 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:49.282 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:49.282 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:49.282 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9393 2024-09-21T14:58:49.283 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.283 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:49.283 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.283 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:49.283 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160585+0000", 2024-09-21T14:58:49.283 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:49.283 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.283 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.283 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:49.283 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160585+0000", 2024-09-21T14:58:49.283 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:49.283 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.283 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.283 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:49.284 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160585+0000", 2024-09-21T14:58:49.284 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:49.284 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.284 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.284 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:49.284 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160587+0000", 2024-09-21T14:58:49.284 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3970000000000001e-06 2024-09-21T14:58:49.284 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.284 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.284 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:49.284 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160592+0000", 2024-09-21T14:58:49.284 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9250000000000003e-06 2024-09-21T14:58:49.284 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.284 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.285 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:49.285 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160597+0000", 2024-09-21T14:58:49.285 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7809999999999996e-06 2024-09-21T14:58:49.285 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.285 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.285 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:49.285 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.167364+0000", 2024-09-21T14:58:49.285 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0067670669999999999 2024-09-21T14:58:49.285 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.285 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.285 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:49.285 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.167480+0000", 2024-09-21T14:58:49.285 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000116345 2024-09-21T14:58:49.285 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.285 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.286 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:49.286 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.167706+0000", 2024-09-21T14:58:49.286 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00022575199999999999 2024-09-21T14:58:49.286 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:49.286 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:49.286 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:49.286 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.286 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.286 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9394 3.e 3:730ca76c:::rbd_data.11363a21d2be.0000000000000004:10 [read 278528~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:49.286 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.160601+0000", 2024-09-21T14:58:49.286 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.059723035999999, 2024-09-21T14:58:49.286 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0074910879999999999, 2024-09-21T14:58:49.286 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:49.286 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:49.286 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:49.287 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:49.287 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:49.287 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9394 2024-09-21T14:58:49.287 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.287 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:49.287 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.287 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:49.287 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160601+0000", 2024-09-21T14:58:49.287 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:49.287 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.287 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.287 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:49.287 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160601+0000", 2024-09-21T14:58:49.287 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:49.287 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.288 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.288 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:49.288 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160600+0000", 2024-09-21T14:58:49.288 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:49.288 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.288 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.288 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:49.288 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160603+0000", 2024-09-21T14:58:49.288 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6529999999999998e-06 2024-09-21T14:58:49.288 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.288 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.288 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:49.288 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160609+0000", 2024-09-21T14:58:49.289 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7080000000000002e-06 2024-09-21T14:58:49.289 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.289 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.289 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:49.289 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160616+0000", 2024-09-21T14:58:49.289 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9739999999999996e-06 2024-09-21T14:58:49.289 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.289 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.289 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:49.289 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.167740+0000", 2024-09-21T14:58:49.289 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0071245019999999996 2024-09-21T14:58:49.289 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.289 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.289 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:49.289 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.167869+0000", 2024-09-21T14:58:49.290 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000129002 2024-09-21T14:58:49.290 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.290 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.290 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:49.290 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.168092+0000", 2024-09-21T14:58:49.290 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000222568 2024-09-21T14:58:49.290 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:49.290 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:49.290 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:49.290 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.290 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.290 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9395 3.e 3:730ca76c:::rbd_data.11363a21d2be.0000000000000004:10 [read 385024~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:49.290 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.160620+0000", 2024-09-21T14:58:49.290 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.059703746, 2024-09-21T14:58:49.291 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0078786570000000007, 2024-09-21T14:58:49.291 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:49.291 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:49.291 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:49.291 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:49.291 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:49.291 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9395 2024-09-21T14:58:49.291 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.291 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:49.291 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.291 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:49.291 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160620+0000", 2024-09-21T14:58:49.291 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:49.291 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.291 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.292 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:49.292 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160620+0000", 2024-09-21T14:58:49.292 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:49.292 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.292 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.292 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:49.292 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160619+0000", 2024-09-21T14:58:49.292 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:49.292 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.292 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.292 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:49.292 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160622+0000", 2024-09-21T14:58:49.292 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.463e-06 2024-09-21T14:58:49.292 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.293 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.293 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:49.293 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160626+0000", 2024-09-21T14:58:49.293 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.0260000000000001e-06 2024-09-21T14:58:49.293 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.293 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.293 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:49.293 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160628+0000", 2024-09-21T14:58:49.293 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5610000000000001e-06 2024-09-21T14:58:49.293 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.293 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.293 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:49.293 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.168151+0000", 2024-09-21T14:58:49.293 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0075221890000000003 2024-09-21T14:58:49.293 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.294 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.294 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:49.294 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.168253+0000", 2024-09-21T14:58:49.294 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000102157 2024-09-21T14:58:49.294 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.294 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.294 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:49.294 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.168498+0000", 2024-09-21T14:58:49.294 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00024563200000000002 2024-09-21T14:58:49.294 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:49.294 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:49.294 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:49.294 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.295 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.295 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9645 3.16 3:6fc5e53c:::rbd_data.11363a21d2be.000000000000000e:10 [read 57344~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:49.295 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.200690+0000", 2024-09-21T14:58:49.295 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.019633215000001, 2024-09-21T14:58:49.295 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0062915949999999997, 2024-09-21T14:58:49.295 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:49.295 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:49.295 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:49.295 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:49.295 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:49.295 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9645 2024-09-21T14:58:49.295 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.295 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:49.295 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.296 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:49.296 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.200690+0000", 2024-09-21T14:58:49.296 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:49.296 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.296 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.296 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:49.296 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.200690+0000", 2024-09-21T14:58:49.296 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:49.296 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.296 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.296 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:49.296 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.200689+0000", 2024-09-21T14:58:49.296 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:49.296 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.297 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.297 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:49.297 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.200696+0000", 2024-09-21T14:58:49.297 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8909999999999998e-06 2024-09-21T14:58:49.297 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.297 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.297 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:49.297 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.200707+0000", 2024-09-21T14:58:49.297 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1573000000000001e-05 2024-09-21T14:58:49.297 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.297 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.297 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:49.297 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.200715+0000", 2024-09-21T14:58:49.297 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8450000000000003e-06 2024-09-21T14:58:49.297 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.298 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.298 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:49.298 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.200880+0000", 2024-09-21T14:58:49.298 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00016440899999999999 2024-09-21T14:58:49.298 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.298 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.298 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:49.298 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.206880+0000", 2024-09-21T14:58:49.298 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0060000089999999997 2024-09-21T14:58:49.298 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.298 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.298 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:49.298 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.206982+0000", 2024-09-21T14:58:49.298 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000102302 2024-09-21T14:58:49.299 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:49.299 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:49.299 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:49.299 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.299 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.299 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9679 3.16 3:6fc5e53c:::rbd_data.11363a21d2be.000000000000000e:10 [read 65536~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:49.299 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.201730+0000", 2024-09-21T14:58:49.299 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.018594006000001, 2024-09-21T14:58:49.299 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0058735239999999998, 2024-09-21T14:58:49.299 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:49.299 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:49.299 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:49.299 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:49.299 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:49.300 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9679 2024-09-21T14:58:49.300 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.300 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:49.300 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.300 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:49.300 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.201730+0000", 2024-09-21T14:58:49.300 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:49.300 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.300 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.300 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:49.300 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.201730+0000", 2024-09-21T14:58:49.300 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:49.300 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.300 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.300 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:49.300 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.201729+0000", 2024-09-21T14:58:49.301 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:58:49.301 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.301 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.301 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:49.301 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.201734+0000", 2024-09-21T14:58:49.301 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7939999999999999e-06 2024-09-21T14:58:49.301 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.301 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.301 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:49.301 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.201744+0000", 2024-09-21T14:58:49.301 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4769999999999992e-06 2024-09-21T14:58:49.301 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.301 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.301 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:49.301 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.201761+0000", 2024-09-21T14:58:49.301 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7504e-05 2024-09-21T14:58:49.302 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.302 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.302 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:49.302 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.207365+0000", 2024-09-21T14:58:49.302 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0056033970000000004 2024-09-21T14:58:49.302 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.302 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.302 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:49.302 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.207437+0000", 2024-09-21T14:58:49.302 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2138999999999995e-05 2024-09-21T14:58:49.302 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.302 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.302 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:49.302 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.207603+0000", 2024-09-21T14:58:49.302 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00016622499999999999 2024-09-21T14:58:49.303 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:49.303 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:49.303 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:49.303 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.303 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.303 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:15130 3.2 3:47f1bbdb:::rbd_data.11361bed1f78.0000000000000067:head [sparse-read 0~147456] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-21T14:58:49.303 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:40.288284+0000", 2024-09-21T14:58:49.303 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 8.9320391190000006, 2024-09-21T14:58:49.303 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1142935039999999, 2024-09-21T14:58:49.303 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:49.303 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:49.303 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:49.303 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:49.303 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:49.304 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 15130 2024-09-21T14:58:49.304 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.304 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:49.304 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.304 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:49.304 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288284+0000", 2024-09-21T14:58:49.304 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:49.304 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.304 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.304 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:49.304 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288284+0000", 2024-09-21T14:58:49.304 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:49.304 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.304 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.304 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:49.304 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288284+0000", 2024-09-21T14:58:49.305 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:49.305 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.305 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.305 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:49.305 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288288+0000", 2024-09-21T14:58:49.305 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5829999999999998e-06 2024-09-21T14:58:49.305 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.305 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.305 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:49.305 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288297+0000", 2024-09-21T14:58:49.305 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0669999999999998e-06 2024-09-21T14:58:49.305 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.305 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.305 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:49.306 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288305+0000", 2024-09-21T14:58:49.306 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8860000000000004e-06 2024-09-21T14:58:49.306 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.306 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.306 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:49.306 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288417+0000", 2024-09-21T14:58:49.306 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000112952 2024-09-21T14:58:49.306 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.306 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.306 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T14:58:49.306 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288428+0000", 2024-09-21T14:58:49.306 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0631e-05 2024-09-21T14:58:49.306 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.306 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.307 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:49.307 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.401876+0000", 2024-09-21T14:58:49.307 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1134479610000001 2024-09-21T14:58:49.307 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.307 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.307 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:49.307 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.402076+0000", 2024-09-21T14:58:49.307 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00019944200000000001 2024-09-21T14:58:49.307 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.307 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.307 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:49.307 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.402578+0000", 2024-09-21T14:58:49.307 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00050242899999999998 2024-09-21T14:58:49.307 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:49.307 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:49.308 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:49.308 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.308 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.308 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:20598 3.2 3:403f4568:::rbd_data.1136c84d9a13.00000000000000d4:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-21T14:58:49.308 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:45.401322+0000", 2024-09-21T14:58:49.308 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 3.8190014030000001, 2024-09-21T14:58:49.308 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014768051000000001, 2024-09-21T14:58:49.308 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:49.308 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:49.308 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:49.308 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:49.308 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:49.308 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 20598 2024-09-21T14:58:49.308 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.309 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:49.309 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.309 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:49.309 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:45.401322+0000", 2024-09-21T14:58:49.309 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:49.309 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.309 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.309 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:49.309 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:45.401322+0000", 2024-09-21T14:58:49.309 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:49.309 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.309 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.309 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:49.310 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:45.401320+0000", 2024-09-21T14:58:49.310 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T14:58:49.310 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.310 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.310 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:49.310 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:45.401330+0000", 2024-09-21T14:58:49.310 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0108999999999999e-05 2024-09-21T14:58:49.310 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.310 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.310 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:49.310 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:45.401351+0000", 2024-09-21T14:58:49.310 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1191000000000001e-05 2024-09-21T14:58:49.310 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.310 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.311 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:49.311 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:45.401361+0000", 2024-09-21T14:58:49.311 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5130000000000002e-06 2024-09-21T14:58:49.311 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.311 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.311 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:49.311 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:45.401476+0000", 2024-09-21T14:58:49.311 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000115684 2024-09-21T14:58:49.311 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.311 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.311 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:49.311 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:45.401717+0000", 2024-09-21T14:58:49.311 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000240256 2024-09-21T14:58:49.311 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.311 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.312 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 3", 2024-09-21T14:58:49.312 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:45.406009+0000", 2024-09-21T14:58:49.312 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0042920780000000004 2024-09-21T14:58:49.312 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.312 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.312 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:49.312 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:45.415983+0000", 2024-09-21T14:58:49.312 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0099743269999999998 2024-09-21T14:58:49.312 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.312 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.312 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:49.312 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:45.415998+0000", 2024-09-21T14:58:49.312 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.49e-05 2024-09-21T14:58:49.312 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.312 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.312 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:49.313 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:45.416049+0000", 2024-09-21T14:58:49.313 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0657e-05 2024-09-21T14:58:49.313 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.313 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.313 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:49.313 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:45.416090+0000", 2024-09-21T14:58:49.313 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1569000000000001e-05 2024-09-21T14:58:49.313 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:49.313 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:49.313 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:49.313 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:49.313 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:49.313 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:58:49.315 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T14:58:49.315 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:58:49.312+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:58:49.415 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:58:49.412+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:58:49.473 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:58:49.473 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:58:49.473 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T14:58:49.473 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:58:49.483 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T14:58:49.516 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:58:49.512+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:58:49.589 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:58:49.589 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:58:49.589 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T14:58:49.589 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T14:58:49.589 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:58:49.599 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T14:58:49.617 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:58:49.612+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:58:49.706 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:58:49.706 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T14:58:49.706 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T14:58:49.706 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T14:58:49.707 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.707 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4281.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-21T14:58:49.707 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:09.324416+0000", 2024-09-21T14:58:49.707 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 40.379100235000003, 2024-09-21T14:58:49.707 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.117837947, 2024-09-21T14:58:49.707 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:49.707 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:49.707 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:49.707 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4281", 2024-09-21T14:58:49.707 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/1518520303", 2024-09-21T14:58:49.707 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1 2024-09-21T14:58:49.707 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.707 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:49.707 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.707 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:49.707 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324416+0000", 2024-09-21T14:58:49.708 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:49.708 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.708 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.708 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:49.708 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324416+0000", 2024-09-21T14:58:49.708 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:49.708 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.708 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.708 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:49.708 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324414+0000", 2024-09-21T14:58:49.708 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T14:58:49.708 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.708 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.708 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:49.708 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324429+0000", 2024-09-21T14:58:49.709 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5637e-05 2024-09-21T14:58:49.709 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.709 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.709 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:49.709 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324448+0000", 2024-09-21T14:58:49.709 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9042000000000002e-05 2024-09-21T14:58:49.709 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.709 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.709 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:49.709 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324465+0000", 2024-09-21T14:58:49.709 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6767999999999999e-05 2024-09-21T14:58:49.709 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.709 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.709 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:49.709 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324571+0000", 2024-09-21T14:58:49.709 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000105714 2024-09-21T14:58:49.710 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.710 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.710 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op must wait for map", 2024-09-21T14:58:49.710 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324587+0000", 2024-09-21T14:58:49.710 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5753999999999999e-05 2024-09-21T14:58:49.710 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.710 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.710 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:49.710 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.325864+0000", 2024-09-21T14:58:49.710 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0012773350000000001 2024-09-21T14:58:49.710 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.710 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.710 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T14:58:49.710 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.325878+0000", 2024-09-21T14:58:49.711 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4498000000000001e-05 2024-09-21T14:58:49.711 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.711 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.711 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:49.711 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.439461+0000", 2024-09-21T14:58:49.711 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.11358212500000001 2024-09-21T14:58:49.711 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.711 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.711 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:49.711 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.439848+0000", 2024-09-21T14:58:49.711 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00038747800000000002 2024-09-21T14:58:49.711 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.711 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.711 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:58:49.711 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.440175+0000", 2024-09-21T14:58:49.712 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00032721900000000001 2024-09-21T14:58:49.712 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.712 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.712 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:49.712 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.441770+0000", 2024-09-21T14:58:49.712 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0015949409999999999 2024-09-21T14:58:49.712 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.712 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.712 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:49.712 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.442086+0000", 2024-09-21T14:58:49.712 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00031586299999999999 2024-09-21T14:58:49.712 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.712 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.712 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:49.712 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.442195+0000", 2024-09-21T14:58:49.712 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000108791 2024-09-21T14:58:49.713 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.713 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.713 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:49.713 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.442254+0000", 2024-09-21T14:58:49.713 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9038e-05 2024-09-21T14:58:49.713 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:49.713 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:49.713 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:49.713 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.713 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.713 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4281.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-21T14:58:49.713 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:09.447828+0000", 2024-09-21T14:58:49.713 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 40.255688233999997, 2024-09-21T14:58:49.713 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.32904393700000001, 2024-09-21T14:58:49.713 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:49.713 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:58:49.714 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:49.714 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4281", 2024-09-21T14:58:49.714 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/1518520303", 2024-09-21T14:58:49.714 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 2 2024-09-21T14:58:49.714 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.714 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:49.714 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.714 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:49.714 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447828+0000", 2024-09-21T14:58:49.714 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:49.714 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.714 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.714 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:49.714 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447828+0000", 2024-09-21T14:58:49.714 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:49.715 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.715 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.715 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:49.715 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447825+0000", 2024-09-21T14:58:49.715 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T14:58:49.715 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.715 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.715 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:49.715 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447841+0000", 2024-09-21T14:58:49.715 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5359999999999999e-05 2024-09-21T14:58:49.715 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.715 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.715 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:49.715 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447875+0000", 2024-09-21T14:58:49.715 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4388999999999999e-05 2024-09-21T14:58:49.715 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.716 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.716 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:49.716 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447903+0000", 2024-09-21T14:58:49.716 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7373e-05 2024-09-21T14:58:49.716 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.716 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.716 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:49.716 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447996+0000", 2024-09-21T14:58:49.716 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2954999999999993e-05 2024-09-21T14:58:49.716 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.716 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.716 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T14:58:49.716 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.448011+0000", 2024-09-21T14:58:49.716 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5038999999999999e-05 2024-09-21T14:58:49.716 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.716 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.717 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:49.717 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.776501+0000", 2024-09-21T14:58:49.717 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.32849012500000002 2024-09-21T14:58:49.717 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.717 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.717 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:49.717 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.776872+0000", 2024-09-21T14:58:49.717 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00037119600000000002 2024-09-21T14:58:49.717 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:49.717 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:49.717 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:49.717 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.717 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.717 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:752 3.14 3:28fc02e5:::rbd_data.113693316f36.0000000000000010:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-21T14:58:49.717 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:26.625579+0000", 2024-09-21T14:58:49.717 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 23.077937379000002, 2024-09-21T14:58:49.718 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042965426000000001, 2024-09-21T14:58:49.718 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:49.718 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:49.718 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:49.718 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:49.718 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:49.718 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 752 2024-09-21T14:58:49.718 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.718 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:49.718 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.718 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:49.718 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625579+0000", 2024-09-21T14:58:49.718 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:49.718 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.719 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.719 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:49.719 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625579+0000", 2024-09-21T14:58:49.719 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:49.719 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.719 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.719 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:49.719 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625577+0000", 2024-09-21T14:58:49.719 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T14:58:49.719 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.719 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.719 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:49.719 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625588+0000", 2024-09-21T14:58:49.719 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1555e-05 2024-09-21T14:58:49.719 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.719 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.720 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:49.720 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625610+0000", 2024-09-21T14:58:49.720 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2107999999999999e-05 2024-09-21T14:58:49.720 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.720 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.720 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:49.720 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625621+0000", 2024-09-21T14:58:49.720 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0762e-05 2024-09-21T14:58:49.720 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.720 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.720 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:49.720 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625692+0000", 2024-09-21T14:58:49.720 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.1203000000000003e-05 2024-09-21T14:58:49.720 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.720 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.720 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:49.721 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625872+0000", 2024-09-21T14:58:49.721 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017962800000000001 2024-09-21T14:58:49.721 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.721 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.721 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:58:49.721 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.633188+0000", 2024-09-21T14:58:49.721 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.007316014 2024-09-21T14:58:49.721 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.721 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.721 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:49.721 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.668357+0000", 2024-09-21T14:58:49.721 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035168933999999999 2024-09-21T14:58:49.721 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.721 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.721 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:49.721 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.668418+0000", 2024-09-21T14:58:49.722 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1282e-05 2024-09-21T14:58:49.722 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.722 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.722 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:49.722 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.668486+0000", 2024-09-21T14:58:49.722 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8468999999999998e-05 2024-09-21T14:58:49.722 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.722 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.722 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:49.722 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.668544+0000", 2024-09-21T14:58:49.722 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7785000000000003e-05 2024-09-21T14:58:49.722 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:49.722 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:49.722 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:49.722 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.722 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.723 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:11682 3.a 3:5753fc6a:::rbd_data.11363a21d2be.0000000000000022:10 [read 483328~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:49.723 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.564765+0000", 2024-09-21T14:58:49.723 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.138750828999999, 2024-09-21T14:58:49.723 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024379739000000001, 2024-09-21T14:58:49.723 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:49.723 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:49.723 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:49.723 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:49.723 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:49.723 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 11682 2024-09-21T14:58:49.723 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.723 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:49.723 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.723 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:49.723 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.564765+0000", 2024-09-21T14:58:49.724 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:49.724 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.724 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.724 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:49.724 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.564765+0000", 2024-09-21T14:58:49.724 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:49.724 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.724 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.724 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:49.724 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.564765+0000", 2024-09-21T14:58:49.724 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:49.724 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.724 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.724 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:49.725 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.564768+0000", 2024-09-21T14:58:49.725 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2119999999999999e-06 2024-09-21T14:58:49.725 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.725 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.725 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:49.725 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.564775+0000", 2024-09-21T14:58:49.725 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5479999999999998e-06 2024-09-21T14:58:49.725 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.725 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.725 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:49.725 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.564779+0000", 2024-09-21T14:58:49.725 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.702e-06 2024-09-21T14:58:49.725 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.726 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.726 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:49.726 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.588887+0000", 2024-09-21T14:58:49.726 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024107625000000001 2024-09-21T14:58:49.726 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.726 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.726 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:49.726 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.588984+0000", 2024-09-21T14:58:49.726 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.7351999999999999e-05 2024-09-21T14:58:49.726 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.726 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.726 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:49.726 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.589145+0000", 2024-09-21T14:58:49.726 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000160824 2024-09-21T14:58:49.726 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:49.727 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:49.727 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:49.727 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.727 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.727 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:11683 3.a 3:50f81070:::rbd_data.11363a21d2be.0000000000000023:10 [read 40960~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:49.727 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.564784+0000", 2024-09-21T14:58:49.727 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.138732212000001, 2024-09-21T14:58:49.727 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024654982999999998, 2024-09-21T14:58:49.727 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:49.727 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:49.727 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:49.727 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:49.727 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:49.727 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 11683 2024-09-21T14:58:49.727 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.727 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:49.728 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.728 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:49.728 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.564784+0000", 2024-09-21T14:58:49.728 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:49.728 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.728 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.729 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:49.729 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.564784+0000", 2024-09-21T14:58:49.729 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:49.729 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.729 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.729 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:49.730 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.564784+0000", 2024-09-21T14:58:49.730 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:49.730 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.730 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.730 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:49.730 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.564786+0000", 2024-09-21T14:58:49.730 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7489999999999999e-06 2024-09-21T14:58:49.730 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.730 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.730 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:49.730 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.564792+0000", 2024-09-21T14:58:49.730 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8889999999999999e-06 2024-09-21T14:58:49.730 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.730 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.731 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:49.731 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.564797+0000", 2024-09-21T14:58:49.731 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.198e-06 2024-09-21T14:58:49.731 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.731 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.731 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:49.731 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.589180+0000", 2024-09-21T14:58:49.731 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024382622 2024-09-21T14:58:49.731 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.731 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.731 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:49.731 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.589238+0000", 2024-09-21T14:58:49.731 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8211999999999998e-05 2024-09-21T14:58:49.731 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.731 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.732 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:49.732 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.589439+0000", 2024-09-21T14:58:49.732 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000200671 2024-09-21T14:58:49.732 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:49.732 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:49.732 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:49.732 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.732 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.732 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:11684 3.a 3:50f81070:::rbd_data.11363a21d2be.0000000000000023:10 [read 180224~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:49.732 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.564801+0000", 2024-09-21T14:58:49.732 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.138714822000001, 2024-09-21T14:58:49.732 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024935196999999999, 2024-09-21T14:58:49.732 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:49.732 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:49.733 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:49.733 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:49.733 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:49.733 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 11684 2024-09-21T14:58:49.733 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.733 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:49.733 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.733 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:49.733 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.564801+0000", 2024-09-21T14:58:49.733 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:49.733 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.733 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.733 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:49.733 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.564801+0000", 2024-09-21T14:58:49.733 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:49.734 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.734 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.734 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:49.734 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.564801+0000", 2024-09-21T14:58:49.734 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:49.734 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.734 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.734 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:49.734 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.564804+0000", 2024-09-21T14:58:49.734 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5399999999999998e-06 2024-09-21T14:58:49.734 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.734 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.734 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:49.735 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.564809+0000", 2024-09-21T14:58:49.735 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5890000000000002e-06 2024-09-21T14:58:49.735 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.735 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.735 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:49.735 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.564814+0000", 2024-09-21T14:58:49.735 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.865e-06 2024-09-21T14:58:49.735 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.735 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.735 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:49.735 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.589481+0000", 2024-09-21T14:58:49.735 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024666527000000001 2024-09-21T14:58:49.735 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.735 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.735 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:49.735 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.589558+0000", 2024-09-21T14:58:49.736 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7176999999999996e-05 2024-09-21T14:58:49.736 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.736 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.736 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:49.736 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.589737+0000", 2024-09-21T14:58:49.736 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000178825 2024-09-21T14:58:49.736 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:49.736 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:49.736 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:49.736 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.736 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.736 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:11685 3.a 3:50f81070:::rbd_data.11363a21d2be.0000000000000023:10 [read 319488~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:49.736 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.564818+0000", 2024-09-21T14:58:49.736 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.138697826, 2024-09-21T14:58:49.737 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025265926000000001, 2024-09-21T14:58:49.737 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:49.737 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:49.737 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:49.737 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:49.737 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:49.737 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 11685 2024-09-21T14:58:49.737 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.737 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:49.737 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.737 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:49.737 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.564818+0000", 2024-09-21T14:58:49.737 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:49.737 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.737 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.737 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:49.738 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.564818+0000", 2024-09-21T14:58:49.738 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:49.738 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.738 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.738 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:49.738 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.564818+0000", 2024-09-21T14:58:49.738 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:49.738 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.738 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.738 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:49.738 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.564821+0000", 2024-09-21T14:58:49.738 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4980000000000001e-06 2024-09-21T14:58:49.738 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.739 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.739 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:49.739 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.564826+0000", 2024-09-21T14:58:49.739 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.395e-06 2024-09-21T14:58:49.739 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.739 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.739 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:49.739 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.564831+0000", 2024-09-21T14:58:49.739 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6070000000000001e-06 2024-09-21T14:58:49.739 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.739 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.740 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:49.740 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.589764+0000", 2024-09-21T14:58:49.740 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024933264 2024-09-21T14:58:49.740 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.740 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.740 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:49.740 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.589816+0000", 2024-09-21T14:58:49.740 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2407000000000002e-05 2024-09-21T14:58:49.740 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.740 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.740 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:49.740 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.590084+0000", 2024-09-21T14:58:49.740 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00026811800000000002 2024-09-21T14:58:49.740 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:49.741 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:49.741 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:49.741 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.741 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.741 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:11686 3.a 3:50f81070:::rbd_data.11363a21d2be.0000000000000023:10 [read 458752~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:49.741 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.564835+0000", 2024-09-21T14:58:49.741 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.138681493, 2024-09-21T14:58:49.741 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025511987999999999, 2024-09-21T14:58:49.741 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:49.741 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:49.741 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:49.741 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:49.741 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:49.741 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 11686 2024-09-21T14:58:49.742 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.742 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:49.742 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.742 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:49.742 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.564835+0000", 2024-09-21T14:58:49.742 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:49.742 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.742 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.742 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:49.742 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.564835+0000", 2024-09-21T14:58:49.742 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:49.742 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.743 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.743 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:49.743 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.564834+0000", 2024-09-21T14:58:49.743 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:49.743 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.743 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.743 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:49.743 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.564837+0000", 2024-09-21T14:58:49.743 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5220000000000002e-06 2024-09-21T14:58:49.743 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.743 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.743 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:49.743 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.564843+0000", 2024-09-21T14:58:49.743 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8159999999999999e-06 2024-09-21T14:58:49.744 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.744 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.744 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:49.744 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.564852+0000", 2024-09-21T14:58:49.744 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.3910000000000004e-06 2024-09-21T14:58:49.744 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.744 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.744 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:49.744 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.590115+0000", 2024-09-21T14:58:49.744 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025263094 2024-09-21T14:58:49.744 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.744 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.744 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:49.744 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.590195+0000", 2024-09-21T14:58:49.745 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9850000000000003e-05 2024-09-21T14:58:49.745 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.745 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.745 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:49.745 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.590347+0000", 2024-09-21T14:58:49.745 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015171600000000001 2024-09-21T14:58:49.745 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:49.745 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:49.745 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:49.745 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.745 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.745 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:11687 3.a 3:5692637c:::rbd_data.11363a21d2be.0000000000000024:10 [read 40960~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:49.745 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.564857+0000", 2024-09-21T14:58:49.745 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.138659606999999, 2024-09-21T14:58:49.746 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025851223, 2024-09-21T14:58:49.746 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:49.746 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:49.746 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:49.746 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:49.746 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:49.746 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 11687 2024-09-21T14:58:49.746 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.746 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:49.746 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.746 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:49.746 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.564857+0000", 2024-09-21T14:58:49.746 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:49.746 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.747 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.747 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:49.747 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.564857+0000", 2024-09-21T14:58:49.747 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:49.747 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.747 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.747 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:49.747 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.564856+0000", 2024-09-21T14:58:49.747 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:49.747 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.747 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.747 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:49.747 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.564860+0000", 2024-09-21T14:58:49.747 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6739999999999999e-06 2024-09-21T14:58:49.748 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.748 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.748 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:49.748 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.564866+0000", 2024-09-21T14:58:49.748 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1850000000000001e-06 2024-09-21T14:58:49.748 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.748 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.748 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:49.748 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.564870+0000", 2024-09-21T14:58:49.748 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6380000000000003e-06 2024-09-21T14:58:49.748 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.748 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.748 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:49.748 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.590375+0000", 2024-09-21T14:58:49.749 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025504608000000002 2024-09-21T14:58:49.749 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.749 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.749 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:49.749 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.590450+0000", 2024-09-21T14:58:49.749 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4757000000000003e-05 2024-09-21T14:58:49.749 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.749 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.749 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:49.749 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.590708+0000", 2024-09-21T14:58:49.749 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00025795899999999999 2024-09-21T14:58:49.749 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:49.749 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:49.749 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:49.749 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.750 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.750 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:11688 3.a 3:5692637c:::rbd_data.11363a21d2be.0000000000000024:10 [read 180224~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:49.750 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.564875+0000", 2024-09-21T14:58:49.750 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.138641032000001, 2024-09-21T14:58:49.750 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026297360999999998, 2024-09-21T14:58:49.750 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:49.750 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:49.750 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:49.750 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:49.750 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:49.750 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 11688 2024-09-21T14:58:49.750 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.750 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:49.751 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.751 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:49.751 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.564875+0000", 2024-09-21T14:58:49.751 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:49.751 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.751 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.751 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:49.751 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.564875+0000", 2024-09-21T14:58:49.751 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:49.751 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.751 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.751 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:49.751 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.564875+0000", 2024-09-21T14:58:49.751 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:49.751 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.752 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.752 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:49.752 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.564878+0000", 2024-09-21T14:58:49.752 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.729e-06 2024-09-21T14:58:49.752 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.752 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.752 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:49.752 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.564883+0000", 2024-09-21T14:58:49.752 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6659999999999996e-06 2024-09-21T14:58:49.752 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.752 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.752 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:49.752 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.564890+0000", 2024-09-21T14:58:49.752 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4130000000000003e-06 2024-09-21T14:58:49.753 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.753 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.753 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:49.753 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.590832+0000", 2024-09-21T14:58:49.753 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025942535999999999 2024-09-21T14:58:49.753 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.753 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.753 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:49.753 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.590957+0000", 2024-09-21T14:58:49.753 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012464400000000001 2024-09-21T14:58:49.753 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.753 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.753 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:49.753 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.591173+0000", 2024-09-21T14:58:49.754 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00021570199999999999 2024-09-21T14:58:49.754 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:49.754 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:49.754 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:49.754 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.754 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.754 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:11689 3.a 3:5692637c:::rbd_data.11363a21d2be.0000000000000024:10 [read 319488~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:49.754 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.564895+0000", 2024-09-21T14:58:49.754 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.138621678, 2024-09-21T14:58:49.754 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026531776, 2024-09-21T14:58:49.754 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:49.754 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:49.754 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:49.754 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:49.755 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:49.755 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 11689 2024-09-21T14:58:49.755 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.755 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:49.755 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.755 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:49.755 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.564895+0000", 2024-09-21T14:58:49.755 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:49.755 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.755 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.755 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:49.755 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.564895+0000", 2024-09-21T14:58:49.755 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:49.755 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.756 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.756 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:49.756 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.564894+0000", 2024-09-21T14:58:49.756 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:49.756 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.756 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.756 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:49.756 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.564897+0000", 2024-09-21T14:58:49.756 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5179999999999999e-06 2024-09-21T14:58:49.756 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.756 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.756 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:49.756 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.564907+0000", 2024-09-21T14:58:49.756 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0227000000000001e-05 2024-09-21T14:58:49.756 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.757 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.757 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:49.757 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.564914+0000", 2024-09-21T14:58:49.757 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.674e-06 2024-09-21T14:58:49.757 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.757 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.757 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:49.757 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.591212+0000", 2024-09-21T14:58:49.757 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026298478 2024-09-21T14:58:49.757 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.757 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.757 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:49.757 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.591259+0000", 2024-09-21T14:58:49.757 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7076999999999997e-05 2024-09-21T14:58:49.758 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.758 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.758 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:49.758 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.591426+0000", 2024-09-21T14:58:49.758 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00016719900000000001 2024-09-21T14:58:49.758 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:49.758 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:49.758 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:49.758 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.758 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.758 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:11706 3.a 3:50f81070:::rbd_data.11363a21d2be.0000000000000023:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:49.758 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.567549+0000", 2024-09-21T14:58:49.758 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.135966723999999, 2024-09-21T14:58:49.759 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024216367999999999, 2024-09-21T14:58:49.759 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:49.759 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:49.759 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:49.759 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:49.759 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:49.759 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 11706 2024-09-21T14:58:49.759 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.759 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:49.759 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.759 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:49.759 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.567549+0000", 2024-09-21T14:58:49.759 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:49.759 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.760 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.760 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:49.760 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.567549+0000", 2024-09-21T14:58:49.760 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:49.760 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.760 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.760 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:49.760 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.567549+0000", 2024-09-21T14:58:49.760 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:49.760 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.760 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.760 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:49.760 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.567553+0000", 2024-09-21T14:58:49.760 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.8600000000000003e-06 2024-09-21T14:58:49.761 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.761 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.761 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:49.761 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.567560+0000", 2024-09-21T14:58:49.761 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2359999999999998e-06 2024-09-21T14:58:49.761 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.761 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.761 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:49.761 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.567564+0000", 2024-09-21T14:58:49.761 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.614e-06 2024-09-21T14:58:49.761 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.761 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.761 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:49.761 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.591637+0000", 2024-09-21T14:58:49.761 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024073431999999999 2024-09-21T14:58:49.762 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.762 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.762 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:49.762 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.591706+0000", 2024-09-21T14:58:49.762 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9078999999999994e-05 2024-09-21T14:58:49.762 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.762 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.762 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:49.762 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.591766+0000", 2024-09-21T14:58:49.762 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9664000000000003e-05 2024-09-21T14:58:49.762 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:49.762 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:49.762 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:49.762 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.762 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.763 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:11707 3.a 3:5692637c:::rbd_data.11363a21d2be.0000000000000024:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:49.763 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.567567+0000", 2024-09-21T14:58:49.763 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.135949329000001, 2024-09-21T14:58:49.763 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025398673, 2024-09-21T14:58:49.763 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:49.763 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:49.763 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:49.763 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:49.763 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:49.763 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 11707 2024-09-21T14:58:49.763 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.763 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:49.763 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.763 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:49.764 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.567567+0000", 2024-09-21T14:58:49.764 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:49.764 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.764 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.764 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:49.764 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.567567+0000", 2024-09-21T14:58:49.764 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:49.764 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.764 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.764 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:49.764 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.567567+0000", 2024-09-21T14:58:49.764 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:49.764 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.764 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.765 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:49.765 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.567569+0000", 2024-09-21T14:58:49.765 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9630000000000001e-06 2024-09-21T14:58:49.765 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.765 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.765 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:49.765 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.567572+0000", 2024-09-21T14:58:49.765 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0189999999999998e-06 2024-09-21T14:58:49.765 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.765 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.765 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:49.765 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.567576+0000", 2024-09-21T14:58:49.766 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.9750000000000001e-06 2024-09-21T14:58:49.766 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.766 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.766 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:49.766 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.592804+0000", 2024-09-21T14:58:49.766 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.02522841 2024-09-21T14:58:49.766 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.766 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.766 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:49.766 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.592884+0000", 2024-09-21T14:58:49.766 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0130999999999996e-05 2024-09-21T14:58:49.766 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.766 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.766 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:49.767 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.592966+0000", 2024-09-21T14:58:49.767 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1429999999999998e-05 2024-09-21T14:58:49.767 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:49.767 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:49.767 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:49.767 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.767 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.767 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:11790 3.a 3:53f19a70:::rbd_data.1136f5ed5dd6.0000000000000045:head [write 0~262144 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:49.767 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.576965+0000", 2024-09-21T14:58:49.767 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.126551544, 2024-09-21T14:58:49.767 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026804264000000001, 2024-09-21T14:58:49.767 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:49.767 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:49.767 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:49.768 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:49.768 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:49.768 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 11790 2024-09-21T14:58:49.768 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.768 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:49.768 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.768 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:49.768 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.576965+0000", 2024-09-21T14:58:49.768 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:49.768 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.768 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.768 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:49.768 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.576965+0000", 2024-09-21T14:58:49.768 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:49.768 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.768 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.769 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:49.769 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.576964+0000", 2024-09-21T14:58:49.769 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:49.769 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.769 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.769 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:49.769 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.577036+0000", 2024-09-21T14:58:49.769 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.224e-05 2024-09-21T14:58:49.769 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.769 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.769 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:49.769 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.577045+0000", 2024-09-21T14:58:49.769 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5939999999999994e-06 2024-09-21T14:58:49.769 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.769 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.770 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:49.770 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.577051+0000", 2024-09-21T14:58:49.770 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7379999999999999e-06 2024-09-21T14:58:49.770 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.770 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.770 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:49.770 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.600966+0000", 2024-09-21T14:58:49.770 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023914973999999999 2024-09-21T14:58:49.770 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.770 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.770 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:49.770 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.601105+0000", 2024-09-21T14:58:49.770 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000139417 2024-09-21T14:58:49.770 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.770 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.771 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 1", 2024-09-21T14:58:49.771 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.601233+0000", 2024-09-21T14:58:49.771 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012790999999999999 2024-09-21T14:58:49.771 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.771 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.771 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:49.771 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.603287+0000", 2024-09-21T14:58:49.771 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0020542550000000001 2024-09-21T14:58:49.771 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.771 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.771 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:49.771 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.603682+0000", 2024-09-21T14:58:49.771 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00039470400000000001 2024-09-21T14:58:49.771 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.772 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.772 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:49.772 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.603721+0000", 2024-09-21T14:58:49.772 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.8906999999999999e-05 2024-09-21T14:58:49.772 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.772 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.772 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:49.772 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.603769+0000", 2024-09-21T14:58:49.772 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7922e-05 2024-09-21T14:58:49.772 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:49.772 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:49.772 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:49.772 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.773 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.773 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:11806 3.a 3:50f81070:::rbd_data.11363a21d2be.0000000000000023:10 [read 204800~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:49.773 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.581185+0000", 2024-09-21T14:58:49.773 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.122331525, 2024-09-21T14:58:49.773 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024130908, 2024-09-21T14:58:49.773 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:49.773 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:49.773 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:49.773 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:49.773 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:49.773 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 11806 2024-09-21T14:58:49.773 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.773 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:49.773 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.774 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:49.774 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.581185+0000", 2024-09-21T14:58:49.774 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:49.774 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.774 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.774 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:49.774 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.581185+0000", 2024-09-21T14:58:49.774 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:49.774 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.774 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.774 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:49.774 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.581184+0000", 2024-09-21T14:58:49.774 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:49.775 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.775 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.775 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:49.775 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.581186+0000", 2024-09-21T14:58:49.775 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9829999999999999e-06 2024-09-21T14:58:49.775 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.775 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.775 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:49.775 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.581192+0000", 2024-09-21T14:58:49.775 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1440000000000002e-06 2024-09-21T14:58:49.775 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.775 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.775 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:49.775 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.581195+0000", 2024-09-21T14:58:49.776 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.9430000000000002e-06 2024-09-21T14:58:49.776 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.776 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.776 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:49.776 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.605042+0000", 2024-09-21T14:58:49.776 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023846835 2024-09-21T14:58:49.776 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.776 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.776 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:49.776 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.605127+0000", 2024-09-21T14:58:49.776 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4226000000000003e-05 2024-09-21T14:58:49.776 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.776 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.777 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:49.777 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.605316+0000", 2024-09-21T14:58:49.777 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00018905899999999999 2024-09-21T14:58:49.777 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:49.777 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:49.777 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:49.777 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.777 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.777 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:11807 3.a 3:50f81070:::rbd_data.11363a21d2be.0000000000000023:10 [read 344064~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:49.777 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.581199+0000", 2024-09-21T14:58:49.777 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.122317256000001, 2024-09-21T14:58:49.777 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024419730000000001, 2024-09-21T14:58:49.777 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:49.778 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:49.778 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:49.778 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:49.778 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:49.778 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 11807 2024-09-21T14:58:49.778 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.778 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:49.778 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.778 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:49.778 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.581199+0000", 2024-09-21T14:58:49.778 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:49.778 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.778 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.778 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:49.779 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.581199+0000", 2024-09-21T14:58:49.779 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:49.779 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.779 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.779 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:49.779 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.581199+0000", 2024-09-21T14:58:49.779 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:49.779 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.779 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.779 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:49.779 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.581201+0000", 2024-09-21T14:58:49.779 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0679999999999999e-06 2024-09-21T14:58:49.779 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.779 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.780 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:49.780 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.581206+0000", 2024-09-21T14:58:49.780 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1610000000000002e-06 2024-09-21T14:58:49.780 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.780 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.780 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:49.780 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.581210+0000", 2024-09-21T14:58:49.780 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.9029999999999997e-06 2024-09-21T14:58:49.780 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.780 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.780 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:49.780 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.605388+0000", 2024-09-21T14:58:49.780 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024178076999999999 2024-09-21T14:58:49.781 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.781 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.781 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:49.781 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.605464+0000", 2024-09-21T14:58:49.781 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6076999999999997e-05 2024-09-21T14:58:49.781 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.781 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.781 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:49.781 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.605619+0000", 2024-09-21T14:58:49.781 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015472899999999999 2024-09-21T14:58:49.781 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:49.781 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:49.781 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:49.782 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.782 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.782 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:11808 3.a 3:50f81070:::rbd_data.11363a21d2be.0000000000000023:10 [read 483328~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:49.782 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.581213+0000", 2024-09-21T14:58:49.782 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.122302733, 2024-09-21T14:58:49.782 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024932673999999998, 2024-09-21T14:58:49.782 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:49.782 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:49.782 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:49.782 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:49.782 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:49.782 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 11808 2024-09-21T14:58:49.782 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.783 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:49.783 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.783 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:49.783 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.581213+0000", 2024-09-21T14:58:49.783 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:49.783 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.783 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.783 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:49.783 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.581213+0000", 2024-09-21T14:58:49.783 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:49.783 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.783 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.783 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:49.783 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.581213+0000", 2024-09-21T14:58:49.784 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:49.784 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.784 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.784 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:49.784 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.581215+0000", 2024-09-21T14:58:49.784 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1119999999999999e-06 2024-09-21T14:58:49.784 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.784 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.784 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:49.784 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.581221+0000", 2024-09-21T14:58:49.784 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5280000000000003e-06 2024-09-21T14:58:49.784 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.784 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.784 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:49.784 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.581225+0000", 2024-09-21T14:58:49.785 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4519999999999999e-06 2024-09-21T14:58:49.785 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.785 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.785 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:49.785 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.605661+0000", 2024-09-21T14:58:49.785 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024435243999999998 2024-09-21T14:58:49.785 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.785 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.785 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:49.785 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.605741+0000", 2024-09-21T14:58:49.785 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0944000000000003e-05 2024-09-21T14:58:49.785 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.785 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.785 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:49.786 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.606146+0000", 2024-09-21T14:58:49.786 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000404674 2024-09-21T14:58:49.786 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:49.786 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:49.786 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:49.786 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.786 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.786 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:11809 3.a 3:5692637c:::rbd_data.11363a21d2be.0000000000000024:10 [read 65536~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:49.786 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.581229+0000", 2024-09-21T14:58:49.786 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.122286971999999, 2024-09-21T14:58:49.786 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025129536000000001, 2024-09-21T14:58:49.786 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:49.786 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:49.786 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:49.787 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:49.787 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:49.787 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 11809 2024-09-21T14:58:49.787 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.787 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:49.787 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.787 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:49.787 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.581229+0000", 2024-09-21T14:58:49.787 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:49.787 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.787 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.787 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:49.787 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.581229+0000", 2024-09-21T14:58:49.787 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:49.788 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.788 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.788 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:49.788 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.581229+0000", 2024-09-21T14:58:49.788 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:49.788 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.788 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.788 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:49.788 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.581231+0000", 2024-09-21T14:58:49.788 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2280000000000001e-06 2024-09-21T14:58:49.788 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.788 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.788 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:49.788 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.581236+0000", 2024-09-21T14:58:49.788 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.101e-06 2024-09-21T14:58:49.789 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.789 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.789 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:49.789 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.581241+0000", 2024-09-21T14:58:49.789 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4270000000000001e-06 2024-09-21T14:58:49.789 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.789 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.789 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:49.789 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.606183+0000", 2024-09-21T14:58:49.789 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.02494232 2024-09-21T14:58:49.789 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.789 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.789 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:49.789 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.606229+0000", 2024-09-21T14:58:49.790 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5775999999999997e-05 2024-09-21T14:58:49.790 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.790 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.790 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:49.790 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.606359+0000", 2024-09-21T14:58:49.790 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000130023 2024-09-21T14:58:49.790 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:49.790 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:49.790 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:49.790 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.790 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.790 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:11810 3.a 3:5692637c:::rbd_data.11363a21d2be.0000000000000024:10 [read 204800~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:49.790 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.581244+0000", 2024-09-21T14:58:49.790 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.122271999000001, 2024-09-21T14:58:49.791 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025349567, 2024-09-21T14:58:49.791 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:49.791 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:49.791 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:49.791 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:49.791 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:49.791 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 11810 2024-09-21T14:58:49.791 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.791 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:49.791 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.791 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:49.791 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.581244+0000", 2024-09-21T14:58:49.791 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:49.791 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.792 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.792 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:49.792 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.581244+0000", 2024-09-21T14:58:49.792 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:49.792 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.792 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.792 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:49.792 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.581244+0000", 2024-09-21T14:58:49.792 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:49.792 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.792 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.792 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:49.792 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.581246+0000", 2024-09-21T14:58:49.792 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.95e-06 2024-09-21T14:58:49.793 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.793 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.793 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:49.793 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.581251+0000", 2024-09-21T14:58:49.793 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6879999999999998e-06 2024-09-21T14:58:49.793 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.793 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.793 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:49.793 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.581255+0000", 2024-09-21T14:58:49.793 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.0330000000000002e-06 2024-09-21T14:58:49.793 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.793 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.793 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:49.793 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.606392+0000", 2024-09-21T14:58:49.793 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025136992 2024-09-21T14:58:49.794 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.794 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.794 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:49.794 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.606461+0000", 2024-09-21T14:58:49.794 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8999999999999997e-05 2024-09-21T14:58:49.794 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.794 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.794 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:49.794 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.606594+0000", 2024-09-21T14:58:49.794 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000133194 2024-09-21T14:58:49.794 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:49.794 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:49.794 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:49.795 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.795 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.795 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:11811 3.a 3:5692637c:::rbd_data.11363a21d2be.0000000000000024:10 [read 344064~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:49.795 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.581258+0000", 2024-09-21T14:58:49.795 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.122257888, 2024-09-21T14:58:49.795 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025562213, 2024-09-21T14:58:49.795 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:49.795 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:49.795 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:49.795 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:49.795 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:49.795 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 11811 2024-09-21T14:58:49.795 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.795 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:49.796 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.796 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:49.796 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.581258+0000", 2024-09-21T14:58:49.796 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:49.796 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.796 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.796 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:49.796 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.581258+0000", 2024-09-21T14:58:49.796 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:49.796 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.796 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.796 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:49.796 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.581258+0000", 2024-09-21T14:58:49.796 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:49.797 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.797 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.797 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:49.797 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.581265+0000", 2024-09-21T14:58:49.797 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6499999999999999e-06 2024-09-21T14:58:49.797 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.797 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.797 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:49.797 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.581270+0000", 2024-09-21T14:58:49.797 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.694e-06 2024-09-21T14:58:49.797 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.797 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.797 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:49.797 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.581275+0000", 2024-09-21T14:58:49.798 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6519999999999997e-06 2024-09-21T14:58:49.798 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.798 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.798 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:49.798 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.606638+0000", 2024-09-21T14:58:49.798 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025362941 2024-09-21T14:58:49.798 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.798 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.798 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:49.798 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.606698+0000", 2024-09-21T14:58:49.798 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9892000000000002e-05 2024-09-21T14:58:49.798 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:49.798 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:49.798 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:49.798 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.606821+0000", 2024-09-21T14:58:49.799 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012279500000000001 2024-09-21T14:58:49.799 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:49.799 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:49.799 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:49.799 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:49.799 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:49.799 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:58:49.800 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T14:58:49.801 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:58:49.796+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:58:49.901 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:58:49.900+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:58:49.918 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:58:49.918 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:58:49.918 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T14:58:49.918 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:58:49.929 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T14:58:50.004 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:58:50.000+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:58:50.041 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:58:50.041 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:58:50.041 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T14:58:50.041 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T14:58:50.041 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:58:50.050 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T14:58:50.104 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:58:50.101+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:58:50.115 INFO:teuthology.orchestra.run.smithi032.stderr:nodeep-scrub is unset 2024-09-21T14:58:50.205 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:58:50.204+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:58:50.235 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:58:50.235 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T14:58:50.235 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T14:58:50.235 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T14:58:50.236 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.236 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:1452 3.19 3:9d40fe72:::rbd_data.1136515ecb6b.0000000000000013:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T14:58:50.236 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:30.292755+0000", 2024-09-21T14:58:50.236 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 19.941547745000001, 2024-09-21T14:58:50.236 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043852191999999998, 2024-09-21T14:58:50.236 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:50.236 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:50.236 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:50.236 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:50.236 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:50.236 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1452 2024-09-21T14:58:50.236 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.236 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:50.237 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.237 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:50.237 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292755+0000", 2024-09-21T14:58:50.237 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:50.237 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.237 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.237 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:50.237 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292755+0000", 2024-09-21T14:58:50.237 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:50.237 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.237 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.237 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:50.237 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292754+0000", 2024-09-21T14:58:50.238 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:50.238 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.238 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.238 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:50.238 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292760+0000", 2024-09-21T14:58:50.238 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5029999999999997e-06 2024-09-21T14:58:50.238 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.238 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.238 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:50.238 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292771+0000", 2024-09-21T14:58:50.238 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1551e-05 2024-09-21T14:58:50.238 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.238 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.239 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:50.239 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292779+0000", 2024-09-21T14:58:50.239 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1559999999999998e-06 2024-09-21T14:58:50.239 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.239 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.239 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:50.239 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292896+0000", 2024-09-21T14:58:50.239 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000117304 2024-09-21T14:58:50.239 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.239 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.239 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:50.239 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.293124+0000", 2024-09-21T14:58:50.240 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00022819999999999999 2024-09-21T14:58:50.240 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.240 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.240 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 1", 2024-09-21T14:58:50.240 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.299967+0000", 2024-09-21T14:58:50.240 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0068427009999999996 2024-09-21T14:58:50.240 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.240 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.240 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:50.240 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336463+0000", 2024-09-21T14:58:50.240 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.036495842000000001 2024-09-21T14:58:50.240 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.240 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.241 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:58:50.241 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336480+0000", 2024-09-21T14:58:50.241 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7419999999999999e-05 2024-09-21T14:58:50.241 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.241 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.241 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:50.241 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336550+0000", 2024-09-21T14:58:50.241 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9951999999999997e-05 2024-09-21T14:58:50.241 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.241 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.241 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:50.242 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336607+0000", 2024-09-21T14:58:50.242 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7423000000000002e-05 2024-09-21T14:58:50.242 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:50.242 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:50.242 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:50.242 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.242 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.242 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop_reply(client.4406.0:1452 3.19 e36/24 ondisk, result = 0)", 2024-09-21T14:58:50.242 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:30.322854+0000", 2024-09-21T14:58:50.242 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 19.911449069, 2024-09-21T14:58:50.242 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013613079, 2024-09-21T14:58:50.242 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:50.242 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:50.243 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:50.243 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.243 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:50.243 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.322854+0000", 2024-09-21T14:58:50.243 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:50.243 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.243 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.243 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:50.243 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.322854+0000", 2024-09-21T14:58:50.243 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:50.243 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.243 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.243 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:50.244 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.322852+0000", 2024-09-21T14:58:50.244 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:50.244 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.244 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.244 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:50.244 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.322860+0000", 2024-09-21T14:58:50.244 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.605e-06 2024-09-21T14:58:50.244 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.244 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.244 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:50.244 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.322878+0000", 2024-09-21T14:58:50.244 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8204000000000001e-05 2024-09-21T14:58:50.244 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.244 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.245 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:50.245 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.322888+0000", 2024-09-21T14:58:50.245 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4099999999999997e-06 2024-09-21T14:58:50.245 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.245 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.245 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:50.245 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336440+0000", 2024-09-21T14:58:50.245 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013551848 2024-09-21T14:58:50.245 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.245 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.245 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:50.245 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336453+0000", 2024-09-21T14:58:50.246 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3465000000000001e-05 2024-09-21T14:58:50.246 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.246 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.246 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:50.246 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336467+0000", 2024-09-21T14:58:50.246 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3913999999999999e-05 2024-09-21T14:58:50.246 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:50.246 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:50.246 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:50.246 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.246 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.246 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:1567 3.8 3:139d3d55:::rbd_data.1136515ecb6b.0000000000000000:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T14:58:50.246 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:30.681087+0000", 2024-09-21T14:58:50.246 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 19.553215529999999, 2024-09-21T14:58:50.247 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.039009224000000002, 2024-09-21T14:58:50.247 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:50.247 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:50.247 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:50.247 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:50.247 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:50.247 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1567 2024-09-21T14:58:50.247 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.247 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:50.247 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.247 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:50.247 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681087+0000", 2024-09-21T14:58:50.247 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:50.247 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.248 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.248 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:50.248 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681087+0000", 2024-09-21T14:58:50.248 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:50.248 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.248 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.248 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:50.248 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681086+0000", 2024-09-21T14:58:50.248 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:50.248 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.248 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.248 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:50.248 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681097+0000", 2024-09-21T14:58:50.248 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.134e-05 2024-09-21T14:58:50.249 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.249 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.249 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:50.249 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681122+0000", 2024-09-21T14:58:50.249 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5313e-05 2024-09-21T14:58:50.249 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.249 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.249 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:50.249 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681138+0000", 2024-09-21T14:58:50.249 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5916000000000001e-05 2024-09-21T14:58:50.249 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.249 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.249 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:50.249 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681270+0000", 2024-09-21T14:58:50.250 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000131474 2024-09-21T14:58:50.250 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.250 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.250 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:50.250 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681416+0000", 2024-09-21T14:58:50.250 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014637 2024-09-21T14:58:50.250 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.250 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.250 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 7", 2024-09-21T14:58:50.250 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.693163+0000", 2024-09-21T14:58:50.250 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011747241 2024-09-21T14:58:50.250 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.250 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.251 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:58:50.251 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.719750+0000", 2024-09-21T14:58:50.251 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.026586852000000001 2024-09-21T14:58:50.251 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.251 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.251 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:50.251 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.719859+0000", 2024-09-21T14:58:50.251 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010925999999999999 2024-09-21T14:58:50.251 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.251 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.251 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:50.251 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.719951+0000", 2024-09-21T14:58:50.251 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1294999999999999e-05 2024-09-21T14:58:50.251 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.252 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.252 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:50.252 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.720097+0000", 2024-09-21T14:58:50.252 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014583 2024-09-21T14:58:50.252 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:50.252 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:50.252 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:50.252 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.252 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.252 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9320 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 425984~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:50.252 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158246+0000", 2024-09-21T14:58:50.252 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 11.076056924, 2024-09-21T14:58:50.252 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013343911, 2024-09-21T14:58:50.252 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:50.253 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:50.253 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:50.253 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:50.253 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:50.253 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9320 2024-09-21T14:58:50.253 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.253 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:50.253 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.253 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:50.253 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158246+0000", 2024-09-21T14:58:50.253 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:50.253 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.253 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.253 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:50.254 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158246+0000", 2024-09-21T14:58:50.254 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:50.254 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.254 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.254 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:50.254 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158245+0000", 2024-09-21T14:58:50.254 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:50.254 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.254 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.254 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:50.254 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158249+0000", 2024-09-21T14:58:50.254 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8659999999999999e-06 2024-09-21T14:58:50.254 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.255 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.255 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:50.255 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158258+0000", 2024-09-21T14:58:50.255 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0720000000000006e-06 2024-09-21T14:58:50.255 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.255 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.255 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:50.255 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158266+0000", 2024-09-21T14:58:50.255 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0039999999999999e-06 2024-09-21T14:58:50.255 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.255 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.255 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:50.255 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.171302+0000", 2024-09-21T14:58:50.255 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013035972999999999 2024-09-21T14:58:50.256 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.256 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.256 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:50.256 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.171395+0000", 2024-09-21T14:58:50.256 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3071000000000004e-05 2024-09-21T14:58:50.256 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.256 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.256 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:50.256 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.171590+0000", 2024-09-21T14:58:50.256 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000194525 2024-09-21T14:58:50.256 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:50.256 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:50.256 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:50.256 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.256 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.257 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9347 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 16384~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:50.257 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158291+0000", 2024-09-21T14:58:50.257 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 11.076012028999999, 2024-09-21T14:58:50.257 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013600958, 2024-09-21T14:58:50.257 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:50.257 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:50.257 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:50.257 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:50.257 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:50.257 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9347 2024-09-21T14:58:50.257 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.257 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:50.257 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.257 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:50.258 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158291+0000", 2024-09-21T14:58:50.258 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:50.258 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.258 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.258 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:50.258 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158291+0000", 2024-09-21T14:58:50.258 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:50.258 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.258 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.258 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:50.258 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158290+0000", 2024-09-21T14:58:50.258 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:50.258 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.258 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.259 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:50.259 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158295+0000", 2024-09-21T14:58:50.259 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2399999999999998e-06 2024-09-21T14:58:50.259 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.259 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.259 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:50.259 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158312+0000", 2024-09-21T14:58:50.259 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6929999999999999e-05 2024-09-21T14:58:50.259 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.259 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.259 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:50.259 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158353+0000", 2024-09-21T14:58:50.259 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0805e-05 2024-09-21T14:58:50.259 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.260 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.260 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:50.260 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.171634+0000", 2024-09-21T14:58:50.260 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.01328085 2024-09-21T14:58:50.260 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.260 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.260 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:50.260 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.171722+0000", 2024-09-21T14:58:50.260 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8115000000000006e-05 2024-09-21T14:58:50.260 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.260 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.260 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:50.260 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.171892+0000", 2024-09-21T14:58:50.260 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000169722 2024-09-21T14:58:50.260 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:50.260 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:50.261 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:50.261 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.261 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.261 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9348 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 122880~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:50.261 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158369+0000", 2024-09-21T14:58:50.261 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 11.075933860999999, 2024-09-21T14:58:50.261 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.01388847, 2024-09-21T14:58:50.261 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:50.261 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:50.261 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:50.261 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:50.261 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:50.261 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9348 2024-09-21T14:58:50.261 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.261 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:50.262 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.262 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:50.262 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158369+0000", 2024-09-21T14:58:50.262 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:50.262 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.262 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.262 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:50.262 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158369+0000", 2024-09-21T14:58:50.262 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:50.262 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.262 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.262 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:50.262 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158368+0000", 2024-09-21T14:58:50.262 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:50.262 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.263 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.263 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:50.263 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158374+0000", 2024-09-21T14:58:50.263 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0499999999999997e-06 2024-09-21T14:58:50.263 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.263 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.263 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:50.263 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158386+0000", 2024-09-21T14:58:50.263 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1579999999999999e-05 2024-09-21T14:58:50.263 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.263 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.263 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:50.264 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158396+0000", 2024-09-21T14:58:50.264 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.8260000000000005e-06 2024-09-21T14:58:50.264 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.264 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.264 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:50.264 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.171967+0000", 2024-09-21T14:58:50.264 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013571318000000001 2024-09-21T14:58:50.264 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.264 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.264 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:50.264 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172034+0000", 2024-09-21T14:58:50.264 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.656e-05 2024-09-21T14:58:50.264 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.264 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.264 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:50.265 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172258+0000", 2024-09-21T14:58:50.265 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00022395999999999999 2024-09-21T14:58:50.265 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:50.265 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:50.265 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:50.265 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.265 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.265 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9349 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 155648~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:50.265 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158403+0000", 2024-09-21T14:58:50.265 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 11.075900115, 2024-09-21T14:58:50.265 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014229851, 2024-09-21T14:58:50.265 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:50.265 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:50.265 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:50.265 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:50.266 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:50.266 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9349 2024-09-21T14:58:50.266 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.266 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:50.266 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.266 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:50.266 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158403+0000", 2024-09-21T14:58:50.266 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:50.266 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.266 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.266 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:50.266 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158403+0000", 2024-09-21T14:58:50.266 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:50.266 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.267 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.267 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:50.267 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158402+0000", 2024-09-21T14:58:50.267 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:50.267 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.267 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.267 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:50.267 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158406+0000", 2024-09-21T14:58:50.267 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1339999999999997e-06 2024-09-21T14:58:50.267 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.267 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.267 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:50.267 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158416+0000", 2024-09-21T14:58:50.267 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3149999999999998e-06 2024-09-21T14:58:50.267 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.268 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.268 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:50.268 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158439+0000", 2024-09-21T14:58:50.268 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3255999999999999e-05 2024-09-21T14:58:50.268 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.268 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.268 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:50.268 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172305+0000", 2024-09-21T14:58:50.268 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013866603999999999 2024-09-21T14:58:50.268 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.268 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.268 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:50.268 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172413+0000", 2024-09-21T14:58:50.268 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000107664 2024-09-21T14:58:50.269 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.269 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.269 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:50.269 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172633+0000", 2024-09-21T14:58:50.269 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021950800000000001 2024-09-21T14:58:50.269 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:50.269 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:50.269 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:50.269 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.269 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.269 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9350 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 294912~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:50.269 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158448+0000", 2024-09-21T14:58:50.269 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 11.075855244, 2024-09-21T14:58:50.269 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014471525000000001, 2024-09-21T14:58:50.269 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:50.270 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:50.270 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:50.270 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:50.270 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:50.270 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9350 2024-09-21T14:58:50.270 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.270 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:50.270 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.270 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:50.270 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158448+0000", 2024-09-21T14:58:50.270 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:50.270 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.270 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.270 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:50.270 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158448+0000", 2024-09-21T14:58:50.271 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:50.271 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.271 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.271 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:50.271 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158447+0000", 2024-09-21T14:58:50.271 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:50.271 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.271 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.271 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:50.271 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158452+0000", 2024-09-21T14:58:50.271 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8400000000000002e-06 2024-09-21T14:58:50.271 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.271 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.271 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:50.272 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158475+0000", 2024-09-21T14:58:50.272 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3091e-05 2024-09-21T14:58:50.272 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.272 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.272 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:50.272 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158515+0000", 2024-09-21T14:58:50.272 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0265000000000002e-05 2024-09-21T14:58:50.272 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.272 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.272 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:50.272 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172682+0000", 2024-09-21T14:58:50.272 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014166761999999999 2024-09-21T14:58:50.272 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.272 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.272 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:50.273 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172752+0000", 2024-09-21T14:58:50.273 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9698999999999998e-05 2024-09-21T14:58:50.273 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.273 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.273 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:50.273 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172919+0000", 2024-09-21T14:58:50.273 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016759199999999999 2024-09-21T14:58:50.273 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:50.273 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:50.273 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:50.273 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.273 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.273 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9351 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 434176~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:50.273 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158526+0000", 2024-09-21T14:58:50.273 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 11.075776526, 2024-09-21T14:58:50.274 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014662107000000001, 2024-09-21T14:58:50.274 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:50.274 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:50.274 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:50.274 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:50.274 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:50.274 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9351 2024-09-21T14:58:50.274 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.274 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:50.274 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.274 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:50.274 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158526+0000", 2024-09-21T14:58:50.274 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:50.274 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.275 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.275 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:50.275 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158526+0000", 2024-09-21T14:58:50.275 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:50.275 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.275 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.275 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:50.275 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158525+0000", 2024-09-21T14:58:50.275 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:50.275 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.275 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.275 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:50.275 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158531+0000", 2024-09-21T14:58:50.275 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5840000000000002e-06 2024-09-21T14:58:50.275 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.275 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.276 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:50.276 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158564+0000", 2024-09-21T14:58:50.276 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3195000000000003e-05 2024-09-21T14:58:50.276 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.276 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.276 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:50.276 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158577+0000", 2024-09-21T14:58:50.276 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2330000000000001e-05 2024-09-21T14:58:50.276 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.276 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.276 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:50.276 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172956+0000", 2024-09-21T14:58:50.276 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014379438 2024-09-21T14:58:50.276 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.276 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.276 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:50.277 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173020+0000", 2024-09-21T14:58:50.277 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4401999999999998e-05 2024-09-21T14:58:50.277 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.277 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.277 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:50.277 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173188+0000", 2024-09-21T14:58:50.277 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016804 2024-09-21T14:58:50.277 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:50.277 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:50.277 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:50.277 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.277 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.277 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9379 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 24576~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:50.277 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158586+0000", 2024-09-21T14:58:50.278 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 11.075717266, 2024-09-21T14:58:50.278 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014865173000000001, 2024-09-21T14:58:50.278 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:50.278 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:50.278 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:50.278 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:50.278 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:50.278 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9379 2024-09-21T14:58:50.278 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.278 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:50.278 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.278 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:50.278 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158586+0000", 2024-09-21T14:58:50.278 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:50.278 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.278 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.279 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:50.279 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158586+0000", 2024-09-21T14:58:50.279 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:50.279 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.279 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.279 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:50.279 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158585+0000", 2024-09-21T14:58:50.279 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:50.279 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.279 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.279 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:50.279 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158590+0000", 2024-09-21T14:58:50.279 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5820000000000001e-06 2024-09-21T14:58:50.279 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.279 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.280 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:50.280 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158676+0000", 2024-09-21T14:58:50.280 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5075999999999997e-05 2024-09-21T14:58:50.280 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.280 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.280 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:50.280 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158695+0000", 2024-09-21T14:58:50.280 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9137999999999999e-05 2024-09-21T14:58:50.280 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.280 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.280 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:50.280 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173219+0000", 2024-09-21T14:58:50.280 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014524034999999999 2024-09-21T14:58:50.280 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.280 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.281 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:50.281 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173276+0000", 2024-09-21T14:58:50.281 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7442000000000003e-05 2024-09-21T14:58:50.281 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.281 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.281 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:50.281 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173451+0000", 2024-09-21T14:58:50.281 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017464800000000001 2024-09-21T14:58:50.281 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:50.281 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:50.281 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:50.281 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.281 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.281 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9380 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 163840~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:50.282 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158706+0000", 2024-09-21T14:58:50.282 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 11.075596755999999, 2024-09-21T14:58:50.282 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015096381000000001, 2024-09-21T14:58:50.282 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:50.282 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:50.282 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:50.282 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:50.282 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:50.282 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9380 2024-09-21T14:58:50.282 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.282 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:50.282 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.282 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:50.282 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158706+0000", 2024-09-21T14:58:50.282 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:50.283 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.283 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.283 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:50.283 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158706+0000", 2024-09-21T14:58:50.283 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:50.283 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.283 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.283 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:50.283 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158705+0000", 2024-09-21T14:58:50.283 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:50.283 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.283 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.283 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:50.283 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158712+0000", 2024-09-21T14:58:50.283 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7240000000000003e-06 2024-09-21T14:58:50.284 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.284 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.284 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:50.284 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158755+0000", 2024-09-21T14:58:50.284 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2953e-05 2024-09-21T14:58:50.284 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.284 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.284 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:50.284 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158765+0000", 2024-09-21T14:58:50.284 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0149000000000001e-05 2024-09-21T14:58:50.284 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.284 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.284 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:50.284 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173492+0000", 2024-09-21T14:58:50.285 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014727105000000001 2024-09-21T14:58:50.285 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.285 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.285 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:50.285 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173564+0000", 2024-09-21T14:58:50.285 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1789e-05 2024-09-21T14:58:50.285 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.285 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.285 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:50.285 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173803+0000", 2024-09-21T14:58:50.285 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00023867100000000001 2024-09-21T14:58:50.285 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:50.285 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:50.285 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:50.286 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.286 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.286 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9381 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 303104~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:50.286 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158774+0000", 2024-09-21T14:58:50.286 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 11.075528675999999, 2024-09-21T14:58:50.286 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015309095999999999, 2024-09-21T14:58:50.286 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:50.286 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:50.286 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:50.286 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:50.286 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:50.286 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9381 2024-09-21T14:58:50.286 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.286 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:50.287 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.287 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:50.287 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158774+0000", 2024-09-21T14:58:50.287 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:50.287 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.287 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.287 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:50.287 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158774+0000", 2024-09-21T14:58:50.287 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:50.287 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.287 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.287 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:50.287 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158773+0000", 2024-09-21T14:58:50.287 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:50.288 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.288 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.288 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:50.288 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158779+0000", 2024-09-21T14:58:50.288 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6999999999999996e-06 2024-09-21T14:58:50.288 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.288 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.288 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:50.288 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158790+0000", 2024-09-21T14:58:50.288 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0594000000000001e-05 2024-09-21T14:58:50.288 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.288 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.288 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:50.288 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158798+0000", 2024-09-21T14:58:50.289 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5860000000000005e-06 2024-09-21T14:58:50.289 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.289 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.289 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:50.289 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173829+0000", 2024-09-21T14:58:50.289 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015030349 2024-09-21T14:58:50.289 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.289 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.289 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:50.289 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173918+0000", 2024-09-21T14:58:50.289 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9375000000000004e-05 2024-09-21T14:58:50.289 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.289 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.289 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:50.289 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.174083+0000", 2024-09-21T14:58:50.289 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016536200000000001 2024-09-21T14:58:50.290 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:50.290 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:50.290 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:50.290 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.290 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.290 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9382 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 442368~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:50.290 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158805+0000", 2024-09-21T14:58:50.290 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 11.075497771, 2024-09-21T14:58:50.290 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015592623, 2024-09-21T14:58:50.290 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:50.290 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:50.290 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:50.290 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:50.290 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:50.291 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9382 2024-09-21T14:58:50.291 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.291 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:50.291 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.291 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:50.291 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158805+0000", 2024-09-21T14:58:50.291 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:50.291 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.291 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.291 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:50.291 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158805+0000", 2024-09-21T14:58:50.291 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:50.291 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.291 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.291 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:50.292 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158805+0000", 2024-09-21T14:58:50.292 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:50.292 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.292 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.292 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:50.292 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158808+0000", 2024-09-21T14:58:50.292 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9179999999999996e-06 2024-09-21T14:58:50.292 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.292 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.292 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:50.292 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158817+0000", 2024-09-21T14:58:50.292 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9299999999999992e-06 2024-09-21T14:58:50.292 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.292 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.293 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:50.293 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158825+0000", 2024-09-21T14:58:50.293 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6739999999999997e-06 2024-09-21T14:58:50.293 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.293 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.293 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:50.293 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.174122+0000", 2024-09-21T14:58:50.293 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.01529694 2024-09-21T14:58:50.293 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.293 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.293 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:50.293 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.174185+0000", 2024-09-21T14:58:50.293 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3015000000000006e-05 2024-09-21T14:58:50.294 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.294 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.294 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:50.294 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.174398+0000", 2024-09-21T14:58:50.294 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021274799999999999 2024-09-21T14:58:50.294 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:50.294 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:50.294 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:50.294 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.294 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.294 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22699 3.19 3:98de45b8:::rbd_data.11366acc894.0000000000000000:19 [read 638976~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:58:50.294 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.140173+0000", 2024-09-21T14:58:50.294 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.094129673999999997, 2024-09-21T14:58:50.295 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013511366, 2024-09-21T14:58:50.295 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:50.295 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:50.295 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:50.295 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:50.295 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:50.295 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22699 2024-09-21T14:58:50.295 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.295 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:50.295 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.295 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:50.295 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140173+0000", 2024-09-21T14:58:50.295 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:50.295 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.296 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.296 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:50.296 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140173+0000", 2024-09-21T14:58:50.296 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:50.296 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.296 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.296 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:50.296 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140173+0000", 2024-09-21T14:58:50.296 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:50.296 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.296 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.296 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:50.296 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140175+0000", 2024-09-21T14:58:50.296 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5859999999999999e-06 2024-09-21T14:58:50.297 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.297 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.297 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:50.297 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140181+0000", 2024-09-21T14:58:50.297 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0340000000000002e-06 2024-09-21T14:58:50.297 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.297 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.297 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:50.297 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140186+0000", 2024-09-21T14:58:50.297 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.04e-06 2024-09-21T14:58:50.297 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.297 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.297 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:50.297 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.153426+0000", 2024-09-21T14:58:50.298 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013240001 2024-09-21T14:58:50.298 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.298 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.298 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:50.298 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.153513+0000", 2024-09-21T14:58:50.298 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6038999999999994e-05 2024-09-21T14:58:50.298 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.298 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.298 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:50.298 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.153685+0000", 2024-09-21T14:58:50.298 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000172067 2024-09-21T14:58:50.298 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:50.298 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:50.298 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:50.299 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.299 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.299 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22700 3.19 3:98de45b8:::rbd_data.11366acc894.0000000000000000:19 [read 704512~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:58:50.299 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.140191+0000", 2024-09-21T14:58:50.299 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.094112157000000002, 2024-09-21T14:58:50.299 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013770486, 2024-09-21T14:58:50.299 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:50.299 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:50.299 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:50.299 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:50.299 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:50.299 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22700 2024-09-21T14:58:50.299 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.299 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:50.299 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.300 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:50.300 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140191+0000", 2024-09-21T14:58:50.300 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:50.300 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.300 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.300 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:50.300 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140191+0000", 2024-09-21T14:58:50.300 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:50.300 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.300 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.300 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:50.300 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140190+0000", 2024-09-21T14:58:50.300 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:50.300 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.300 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.301 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:50.301 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140199+0000", 2024-09-21T14:58:50.301 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3340000000000002e-06 2024-09-21T14:58:50.301 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.301 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.301 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:50.301 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140205+0000", 2024-09-21T14:58:50.301 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3820000000000001e-06 2024-09-21T14:58:50.301 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.301 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.301 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:50.301 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140210+0000", 2024-09-21T14:58:50.301 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0969999999999996e-06 2024-09-21T14:58:50.301 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.301 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.302 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:50.302 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.153713+0000", 2024-09-21T14:58:50.302 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013502713 2024-09-21T14:58:50.302 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.302 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.302 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:50.302 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.153786+0000", 2024-09-21T14:58:50.302 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3022999999999995e-05 2024-09-21T14:58:50.302 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.302 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.302 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:50.302 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.153961+0000", 2024-09-21T14:58:50.302 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000175274 2024-09-21T14:58:50.302 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:50.303 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:50.303 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:50.303 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.303 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.303 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22701 3.19 3:98de45b8:::rbd_data.11366acc894.0000000000000000:19 [read 770048~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:58:50.303 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.140215+0000", 2024-09-21T14:58:50.303 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.094088340000000006, 2024-09-21T14:58:50.303 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014072497, 2024-09-21T14:58:50.303 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:50.303 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:50.303 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:50.303 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:50.303 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:50.303 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22701 2024-09-21T14:58:50.303 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.304 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:50.304 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.304 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:50.304 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140215+0000", 2024-09-21T14:58:50.304 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:50.304 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.304 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.304 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:50.304 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140215+0000", 2024-09-21T14:58:50.304 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:50.304 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.304 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.304 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:50.304 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140214+0000", 2024-09-21T14:58:50.305 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:50.305 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.305 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.305 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:50.305 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140217+0000", 2024-09-21T14:58:50.305 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.74e-06 2024-09-21T14:58:50.305 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.305 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.305 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:50.305 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140223+0000", 2024-09-21T14:58:50.305 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0929999999999999e-06 2024-09-21T14:58:50.305 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.306 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.306 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:50.306 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140228+0000", 2024-09-21T14:58:50.306 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2279999999999998e-06 2024-09-21T14:58:50.306 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.306 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.306 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:50.306 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.153991+0000", 2024-09-21T14:58:50.306 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013762329 2024-09-21T14:58:50.306 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.306 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.306 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:50.306 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.154098+0000", 2024-09-21T14:58:50.306 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000107117 2024-09-21T14:58:50.307 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.307 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.307 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:50.307 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.154287+0000", 2024-09-21T14:58:50.307 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000189431 2024-09-21T14:58:50.307 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:50.307 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:50.307 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:50.307 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.307 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.307 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22702 3.19 3:98de45b8:::rbd_data.11366acc894.0000000000000000:19 [read 835584~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:58:50.307 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.140232+0000", 2024-09-21T14:58:50.307 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.094070487999999994, 2024-09-21T14:58:50.307 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015338742000000001, 2024-09-21T14:58:50.308 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:50.308 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:50.308 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:50.308 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:50.308 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:50.308 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22702 2024-09-21T14:58:50.308 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.308 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:50.308 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.308 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:50.308 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140232+0000", 2024-09-21T14:58:50.308 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:50.308 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.308 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.309 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:50.309 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140232+0000", 2024-09-21T14:58:50.309 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:50.309 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.309 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.309 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:50.309 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140232+0000", 2024-09-21T14:58:50.309 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:50.309 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.309 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.309 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:50.309 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140234+0000", 2024-09-21T14:58:50.309 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.402e-06 2024-09-21T14:58:50.309 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.309 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.310 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:50.310 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140241+0000", 2024-09-21T14:58:50.310 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.054e-06 2024-09-21T14:58:50.310 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.310 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.310 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:50.310 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140246+0000", 2024-09-21T14:58:50.310 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0050000000000004e-06 2024-09-21T14:58:50.310 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.310 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.310 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:50.310 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.154315+0000", 2024-09-21T14:58:50.310 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014069440000000001 2024-09-21T14:58:50.310 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.310 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.311 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:50.311 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.154383+0000", 2024-09-21T14:58:50.311 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8151000000000005e-05 2024-09-21T14:58:50.311 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.311 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.311 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:50.311 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155571+0000", 2024-09-21T14:58:50.311 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001188029 2024-09-21T14:58:50.311 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:50.311 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:50.311 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:50.311 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.311 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.311 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22703 3.19 3:98de45b8:::rbd_data.11366acc894.0000000000000000:19 [read 901120~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:58:50.311 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.140250+0000", 2024-09-21T14:58:50.312 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.094053330000000004, 2024-09-21T14:58:50.312 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014992169, 2024-09-21T14:58:50.312 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:50.312 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:50.312 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:50.312 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:50.312 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:50.312 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22703 2024-09-21T14:58:50.312 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.312 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:50.312 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.312 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:50.312 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140250+0000", 2024-09-21T14:58:50.312 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:50.312 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.313 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.313 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:50.313 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140250+0000", 2024-09-21T14:58:50.313 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:50.313 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.313 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.313 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:50.313 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140249+0000", 2024-09-21T14:58:50.313 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:50.313 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.313 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.313 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:50.313 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140251+0000", 2024-09-21T14:58:50.314 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.136e-06 2024-09-21T14:58:50.314 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.314 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.314 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:50.314 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140257+0000", 2024-09-21T14:58:50.314 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9610000000000002e-06 2024-09-21T14:58:50.314 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.314 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.314 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:50.314 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140262+0000", 2024-09-21T14:58:50.314 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4549999999999997e-06 2024-09-21T14:58:50.314 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.314 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.314 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:50.315 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155006+0000", 2024-09-21T14:58:50.315 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014744189 2024-09-21T14:58:50.315 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.315 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.315 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:50.315 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155081+0000", 2024-09-21T14:58:50.315 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4548000000000005e-05 2024-09-21T14:58:50.315 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.315 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.315 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:50.315 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155242+0000", 2024-09-21T14:58:50.315 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000161204 2024-09-21T14:58:50.315 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:50.315 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:50.315 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:50.316 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.316 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.316 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22704 3.19 3:98de45b8:::rbd_data.11366acc894.0000000000000000:19 [read 966656~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:58:50.316 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.140266+0000", 2024-09-21T14:58:50.316 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.094037013000000003, 2024-09-21T14:58:50.316 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015223443999999999, 2024-09-21T14:58:50.316 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:50.316 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:50.316 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:50.316 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:50.316 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:50.316 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22704 2024-09-21T14:58:50.316 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.316 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:50.316 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.316 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:50.317 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140266+0000", 2024-09-21T14:58:50.317 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:50.317 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.317 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.317 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:50.317 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140266+0000", 2024-09-21T14:58:50.317 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:50.317 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.317 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.317 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:50.317 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140266+0000", 2024-09-21T14:58:50.317 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:50.317 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.317 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.318 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:50.318 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140268+0000", 2024-09-21T14:58:50.318 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.475e-06 2024-09-21T14:58:50.318 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.318 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.318 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:50.318 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140274+0000", 2024-09-21T14:58:50.318 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.319e-06 2024-09-21T14:58:50.318 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.318 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.318 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:50.318 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140279+0000", 2024-09-21T14:58:50.318 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.882e-06 2024-09-21T14:58:50.318 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.319 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.319 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:50.319 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155264+0000", 2024-09-21T14:58:50.319 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014984655 2024-09-21T14:58:50.319 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.319 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.319 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:50.319 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155338+0000", 2024-09-21T14:58:50.319 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4091999999999995e-05 2024-09-21T14:58:50.319 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.319 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.319 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:50.319 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155489+0000", 2024-09-21T14:58:50.319 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000151349 2024-09-21T14:58:50.319 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:50.320 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:50.320 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:50.320 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.320 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.320 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22705 3.19 3:98de45b8:::rbd_data.11366acc894.0000000000000000:19 [read 1032192~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:58:50.320 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.140284+0000", 2024-09-21T14:58:50.320 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.094019156000000007, 2024-09-21T14:58:50.320 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015467896, 2024-09-21T14:58:50.320 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:50.320 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:50.320 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:50.320 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:50.320 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:50.320 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22705 2024-09-21T14:58:50.320 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.321 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:50.321 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.321 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:50.321 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140284+0000", 2024-09-21T14:58:50.321 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:50.321 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.321 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.321 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:50.321 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140284+0000", 2024-09-21T14:58:50.321 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:50.321 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.321 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.321 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:50.321 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140283+0000", 2024-09-21T14:58:50.321 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:50.321 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.322 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.322 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:50.322 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140286+0000", 2024-09-21T14:58:50.322 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7130000000000001e-06 2024-09-21T14:58:50.322 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.322 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.322 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:50.322 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140292+0000", 2024-09-21T14:58:50.322 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3430000000000001e-06 2024-09-21T14:58:50.322 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.322 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.322 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:50.322 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140298+0000", 2024-09-21T14:58:50.322 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9730000000000002e-06 2024-09-21T14:58:50.323 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.323 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.323 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:50.323 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155510+0000", 2024-09-21T14:58:50.323 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015211738000000001 2024-09-21T14:58:50.323 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.323 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.323 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:50.323 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155598+0000", 2024-09-21T14:58:50.323 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7822e-05 2024-09-21T14:58:50.323 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.323 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.323 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:50.323 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155752+0000", 2024-09-21T14:58:50.323 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0001537 2024-09-21T14:58:50.324 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:50.324 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:50.324 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:50.324 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:50.324 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:50.324 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:58:50.325 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T14:58:50.326 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:58:50.321+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:58:50.426 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:58:50.424+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:58:50.441 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:58:50.441 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:58:50.441 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T14:58:50.441 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:58:50.451 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T14:58:50.527 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:58:50.525+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:58:50.558 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:58:50.558 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:58:50.559 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T14:58:50.559 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T14:58:50.559 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:58:50.568 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T14:58:50.628 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:58:50.625+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:58:50.677 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:58:50.677 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T14:58:50.677 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T14:58:50.677 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T14:58:50.677 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.677 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22719 3.1e 3:79caef05:::rbd_data.11366acc894.0000000000000001:19 [read 901120~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:58:50.677 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.140027+0000", 2024-09-21T14:58:50.677 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.534564606, 2024-09-21T14:58:50.677 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015081021, 2024-09-21T14:58:50.677 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:50.677 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:50.677 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:50.678 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:50.678 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:50.678 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22719 2024-09-21T14:58:50.678 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.678 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:50.678 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.678 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:50.678 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140027+0000", 2024-09-21T14:58:50.678 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:50.678 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.678 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.678 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:50.678 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140027+0000", 2024-09-21T14:58:50.678 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:50.678 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.679 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.679 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:50.679 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140026+0000", 2024-09-21T14:58:50.679 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:50.679 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.679 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.679 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:50.679 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140042+0000", 2024-09-21T14:58:50.679 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5381999999999999e-05 2024-09-21T14:58:50.679 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.679 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.679 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:50.679 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140051+0000", 2024-09-21T14:58:50.679 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3500000000000003e-06 2024-09-21T14:58:50.679 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.680 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.680 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:50.680 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140066+0000", 2024-09-21T14:58:50.680 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4582e-05 2024-09-21T14:58:50.680 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.680 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.680 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:50.680 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.154602+0000", 2024-09-21T14:58:50.680 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014535881 2024-09-21T14:58:50.680 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.680 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.680 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:50.680 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.154674+0000", 2024-09-21T14:58:50.680 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2303000000000002e-05 2024-09-21T14:58:50.680 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.680 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.681 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:50.681 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155108+0000", 2024-09-21T14:58:50.681 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00043396500000000002 2024-09-21T14:58:50.681 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:50.681 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:50.681 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:50.681 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.681 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.681 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22720 3.1e 3:79caef05:::rbd_data.11366acc894.0000000000000001:19 [read 966656~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:58:50.681 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.140073+0000", 2024-09-21T14:58:50.681 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.534518041, 2024-09-21T14:58:50.681 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015292312000000001, 2024-09-21T14:58:50.681 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:50.681 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:50.681 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:50.681 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:50.682 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:50.682 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22720 2024-09-21T14:58:50.682 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.682 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:50.682 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.682 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:50.682 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140073+0000", 2024-09-21T14:58:50.682 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:50.682 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.682 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.682 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:50.682 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140073+0000", 2024-09-21T14:58:50.682 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:50.682 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.683 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.683 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:50.683 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140073+0000", 2024-09-21T14:58:50.683 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:50.683 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.683 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.683 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:50.683 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140076+0000", 2024-09-21T14:58:50.683 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4999999999999999e-06 2024-09-21T14:58:50.683 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.683 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.683 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:50.683 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140084+0000", 2024-09-21T14:58:50.683 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1310000000000006e-06 2024-09-21T14:58:50.684 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.684 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.684 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:50.684 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140091+0000", 2024-09-21T14:58:50.684 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.674e-06 2024-09-21T14:58:50.684 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.684 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.684 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:50.684 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155135+0000", 2024-09-21T14:58:50.684 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015044224 2024-09-21T14:58:50.684 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.684 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.684 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:50.684 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155205+0000", 2024-09-21T14:58:50.684 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9751000000000004e-05 2024-09-21T14:58:50.685 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.685 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.685 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:50.685 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155366+0000", 2024-09-21T14:58:50.685 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016073899999999999 2024-09-21T14:58:50.685 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:50.685 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:50.685 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:50.685 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.685 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.685 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22721 3.1e 3:79caef05:::rbd_data.11366acc894.0000000000000001:19 [read 1032192~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:58:50.685 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.140096+0000", 2024-09-21T14:58:50.685 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.53449505600000002, 2024-09-21T14:58:50.685 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015563158000000001, 2024-09-21T14:58:50.686 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:50.686 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:50.686 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:50.686 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:50.686 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:50.686 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22721 2024-09-21T14:58:50.686 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.686 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:50.686 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.686 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:50.686 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140096+0000", 2024-09-21T14:58:50.686 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:50.686 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.686 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.687 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:50.687 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140096+0000", 2024-09-21T14:58:50.687 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:50.687 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.687 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.687 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:50.687 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140096+0000", 2024-09-21T14:58:50.687 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:50.687 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.687 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.687 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:50.687 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140099+0000", 2024-09-21T14:58:50.687 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.5080000000000001e-06 2024-09-21T14:58:50.687 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.687 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.688 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:50.688 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140106+0000", 2024-09-21T14:58:50.688 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8630000000000002e-06 2024-09-21T14:58:50.688 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.688 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.688 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:50.688 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140112+0000", 2024-09-21T14:58:50.688 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6509999999999998e-06 2024-09-21T14:58:50.688 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.688 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.688 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:50.688 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155392+0000", 2024-09-21T14:58:50.688 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015280375000000001 2024-09-21T14:58:50.688 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.689 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.689 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:50.689 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155459+0000", 2024-09-21T14:58:50.689 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7040999999999997e-05 2024-09-21T14:58:50.689 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.689 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.689 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:50.689 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155660+0000", 2024-09-21T14:58:50.689 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000200115 2024-09-21T14:58:50.689 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:50.689 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:50.689 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:50.689 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.689 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.689 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22865 3.1e 3:7ca947c4:::rbd_data.11366acc894.000000000000000b:19 [read 393216~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:58:50.690 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.142757+0000", 2024-09-21T14:58:50.690 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.53183438500000002, 2024-09-21T14:58:50.690 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014907012000000001, 2024-09-21T14:58:50.690 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:50.690 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:50.690 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:50.690 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:50.690 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:50.690 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22865 2024-09-21T14:58:50.690 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.690 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:50.690 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.690 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:50.690 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.142757+0000", 2024-09-21T14:58:50.691 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:50.691 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.691 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.691 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:50.691 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.142757+0000", 2024-09-21T14:58:50.691 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:50.691 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.691 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.691 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:50.691 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.142757+0000", 2024-09-21T14:58:50.691 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:50.691 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.691 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.691 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:50.692 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.142759+0000", 2024-09-21T14:58:50.692 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3800000000000001e-06 2024-09-21T14:58:50.692 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.692 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.692 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:50.692 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.142765+0000", 2024-09-21T14:58:50.692 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7819999999999999e-06 2024-09-21T14:58:50.692 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.692 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.692 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:50.692 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.142770+0000", 2024-09-21T14:58:50.692 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9230000000000001e-06 2024-09-21T14:58:50.692 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.692 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.692 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:50.693 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.157393+0000", 2024-09-21T14:58:50.693 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014623662000000001 2024-09-21T14:58:50.693 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.693 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.693 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:50.693 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.157469+0000", 2024-09-21T14:58:50.693 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5383000000000007e-05 2024-09-21T14:58:50.693 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.693 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.693 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:50.693 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.157664+0000", 2024-09-21T14:58:50.693 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019522100000000001 2024-09-21T14:58:50.693 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:50.693 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:50.693 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:50.693 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.694 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.694 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22866 3.1e 3:7ca947c4:::rbd_data.11366acc894.000000000000000b:19 [read 458752~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:58:50.694 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.142774+0000", 2024-09-21T14:58:50.694 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.53181727499999998, 2024-09-21T14:58:50.694 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.01521982, 2024-09-21T14:58:50.694 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:50.694 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:50.694 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:50.694 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:50.694 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:50.694 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22866 2024-09-21T14:58:50.694 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.694 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:50.694 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.695 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:50.695 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.142774+0000", 2024-09-21T14:58:50.695 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:50.695 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.695 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.695 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:50.695 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.142774+0000", 2024-09-21T14:58:50.695 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:50.695 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.695 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.695 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:50.695 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.142774+0000", 2024-09-21T14:58:50.695 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:50.695 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.695 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.696 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:50.696 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.142776+0000", 2024-09-21T14:58:50.696 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.588e-06 2024-09-21T14:58:50.696 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.696 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.696 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:50.696 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.142783+0000", 2024-09-21T14:58:50.696 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2729999999999999e-06 2024-09-21T14:58:50.696 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.696 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.696 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:50.696 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.142789+0000", 2024-09-21T14:58:50.696 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3160000000000002e-06 2024-09-21T14:58:50.696 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.696 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.697 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:50.697 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.157705+0000", 2024-09-21T14:58:50.697 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014915586999999999 2024-09-21T14:58:50.697 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.697 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.697 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:50.697 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.157792+0000", 2024-09-21T14:58:50.697 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7322000000000002e-05 2024-09-21T14:58:50.697 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.697 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.697 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:50.697 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.157994+0000", 2024-09-21T14:58:50.697 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000202093 2024-09-21T14:58:50.697 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:50.697 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:50.698 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:50.698 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.698 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.698 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22867 3.1e 3:7ca947c4:::rbd_data.11366acc894.000000000000000b:19 [read 524288~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:58:50.698 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.142794+0000", 2024-09-21T14:58:50.698 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.53179753500000004, 2024-09-21T14:58:50.698 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015569725, 2024-09-21T14:58:50.698 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:50.698 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:50.698 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:50.698 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:50.698 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:50.698 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22867 2024-09-21T14:58:50.698 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.698 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:50.699 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.699 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:50.699 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.142794+0000", 2024-09-21T14:58:50.699 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:50.699 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.699 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.699 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:50.699 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.142794+0000", 2024-09-21T14:58:50.699 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:50.699 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.699 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.699 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:50.699 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.142793+0000", 2024-09-21T14:58:50.699 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:50.699 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.699 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.700 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:50.700 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.142801+0000", 2024-09-21T14:58:50.700 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8509999999999999e-06 2024-09-21T14:58:50.700 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.700 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.700 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:50.700 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.142807+0000", 2024-09-21T14:58:50.700 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1210000000000003e-06 2024-09-21T14:58:50.700 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.700 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.700 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:50.700 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.142812+0000", 2024-09-21T14:58:50.700 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2809999999999999e-06 2024-09-21T14:58:50.700 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.700 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.700 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:50.701 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.158021+0000", 2024-09-21T14:58:50.701 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015208038 2024-09-21T14:58:50.701 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.701 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.701 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:50.701 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.158098+0000", 2024-09-21T14:58:50.701 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7888999999999999e-05 2024-09-21T14:58:50.701 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.701 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.701 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:50.701 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.158364+0000", 2024-09-21T14:58:50.701 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00026517400000000002 2024-09-21T14:58:50.701 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:50.701 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:50.701 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:50.701 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.702 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.702 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:23017 3.1e 3:7ca947c4:::rbd_data.11366acc894.000000000000000b:19 [read 229376~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:58:50.702 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.147628+0000", 2024-09-21T14:58:50.702 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.52696351500000005, 2024-09-21T14:58:50.702 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014791732, 2024-09-21T14:58:50.702 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:50.702 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:50.702 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:50.702 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:50.702 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:50.702 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 23017 2024-09-21T14:58:50.702 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.702 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:50.702 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.702 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:50.703 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147628+0000", 2024-09-21T14:58:50.703 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:50.703 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.703 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.703 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:50.703 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147628+0000", 2024-09-21T14:58:50.703 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:50.703 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.703 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.703 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:50.703 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147628+0000", 2024-09-21T14:58:50.703 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:50.703 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.703 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.703 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:50.704 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147635+0000", 2024-09-21T14:58:50.704 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3459999999999996e-06 2024-09-21T14:58:50.704 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.704 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.704 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:50.704 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147642+0000", 2024-09-21T14:58:50.704 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0249999999999997e-06 2024-09-21T14:58:50.704 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.704 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.704 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:50.704 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147647+0000", 2024-09-21T14:58:50.704 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.549e-06 2024-09-21T14:58:50.704 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.704 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.705 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:50.705 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.161946+0000", 2024-09-21T14:58:50.705 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.01429997 2024-09-21T14:58:50.705 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.705 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.705 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:50.705 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.162034+0000", 2024-09-21T14:58:50.705 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7676000000000006e-05 2024-09-21T14:58:50.705 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.705 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.705 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:50.705 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.162420+0000", 2024-09-21T14:58:50.705 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00038544499999999999 2024-09-21T14:58:50.705 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:50.706 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:50.706 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:50.706 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.706 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.706 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:23018 3.1e 3:7ca947c4:::rbd_data.11366acc894.000000000000000b:19 [read 294912~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:58:50.706 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.147650+0000", 2024-09-21T14:58:50.706 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.52694108699999997, 2024-09-21T14:58:50.706 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015515414999999999, 2024-09-21T14:58:50.706 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:50.706 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:50.706 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:50.706 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:50.707 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:50.707 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 23018 2024-09-21T14:58:50.707 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.707 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:50.707 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.707 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:50.707 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147650+0000", 2024-09-21T14:58:50.707 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:50.707 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.707 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.707 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:50.707 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147650+0000", 2024-09-21T14:58:50.707 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:50.708 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.708 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.708 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:50.708 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147650+0000", 2024-09-21T14:58:50.708 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:50.708 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.708 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.708 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:50.708 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147652+0000", 2024-09-21T14:58:50.708 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1469999999999999e-06 2024-09-21T14:58:50.708 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.708 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.708 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:50.708 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147657+0000", 2024-09-21T14:58:50.709 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3449999999999996e-06 2024-09-21T14:58:50.709 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.709 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.709 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:50.709 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147663+0000", 2024-09-21T14:58:50.709 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1309999999999997e-06 2024-09-21T14:58:50.709 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.709 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.709 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:50.709 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.162466+0000", 2024-09-21T14:58:50.709 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014803474 2024-09-21T14:58:50.709 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.709 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.709 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:50.710 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.162562+0000", 2024-09-21T14:58:50.710 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6272000000000003e-05 2024-09-21T14:58:50.710 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.710 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.710 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:50.710 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.163166+0000", 2024-09-21T14:58:50.710 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00060336000000000005 2024-09-21T14:58:50.710 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:50.710 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:50.710 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:50.710 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.710 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.710 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:23019 3.1e 3:7ca947c4:::rbd_data.11366acc894.000000000000000b:19 [read 360448~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:58:50.711 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.147666+0000", 2024-09-21T14:58:50.711 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.52692504299999998, 2024-09-21T14:58:50.711 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015788706, 2024-09-21T14:58:50.711 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:50.711 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:50.711 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:50.711 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:50.711 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:50.711 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 23019 2024-09-21T14:58:50.711 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.711 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:50.711 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.711 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:50.712 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147666+0000", 2024-09-21T14:58:50.712 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:50.712 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.712 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.712 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:50.712 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147666+0000", 2024-09-21T14:58:50.712 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:50.712 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.712 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.712 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:50.712 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147666+0000", 2024-09-21T14:58:50.712 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:50.713 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.713 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.713 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:50.713 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147668+0000", 2024-09-21T14:58:50.713 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1969999999999999e-06 2024-09-21T14:58:50.713 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.713 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.713 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:50.713 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147673+0000", 2024-09-21T14:58:50.713 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2220000000000001e-06 2024-09-21T14:58:50.713 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.713 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.713 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:50.714 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147677+0000", 2024-09-21T14:58:50.714 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9620000000000004e-06 2024-09-21T14:58:50.714 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.714 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.714 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:50.714 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.163200+0000", 2024-09-21T14:58:50.714 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.01552215 2024-09-21T14:58:50.714 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.714 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.714 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:50.714 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.163294+0000", 2024-09-21T14:58:50.714 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4017000000000005e-05 2024-09-21T14:58:50.714 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.714 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.714 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:50.715 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.163455+0000", 2024-09-21T14:58:50.715 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000161478 2024-09-21T14:58:50.715 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:50.715 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:50.715 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:50.715 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.715 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.715 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:23020 3.1e 3:7ca947c4:::rbd_data.11366acc894.000000000000000b:19 [read 425984~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:58:50.715 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.147681+0000", 2024-09-21T14:58:50.715 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.52691031300000002, 2024-09-21T14:58:50.715 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.016190145, 2024-09-21T14:58:50.715 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:50.715 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:50.715 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:50.715 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:50.716 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:50.716 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 23020 2024-09-21T14:58:50.716 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.716 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:50.716 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.716 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:50.716 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147681+0000", 2024-09-21T14:58:50.716 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:50.716 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.716 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.716 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:50.716 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147681+0000", 2024-09-21T14:58:50.716 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:50.716 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.716 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.717 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:50.717 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147681+0000", 2024-09-21T14:58:50.717 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:50.717 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.717 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.717 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:50.717 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147683+0000", 2024-09-21T14:58:50.717 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3470000000000001e-06 2024-09-21T14:58:50.717 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.717 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.717 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:50.717 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147689+0000", 2024-09-21T14:58:50.717 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5849999999999999e-06 2024-09-21T14:58:50.717 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.717 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.718 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:50.718 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147695+0000", 2024-09-21T14:58:50.718 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.964e-06 2024-09-21T14:58:50.718 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.718 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.718 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:50.718 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.163488+0000", 2024-09-21T14:58:50.718 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015793564999999999 2024-09-21T14:58:50.718 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.718 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.718 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:50.718 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.163532+0000", 2024-09-21T14:58:50.718 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3841000000000003e-05 2024-09-21T14:58:50.718 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.719 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.719 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:50.719 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.163871+0000", 2024-09-21T14:58:50.719 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00033914500000000001 2024-09-21T14:58:50.719 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:50.719 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:50.719 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:50.719 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.719 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.719 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:23021 3.1e 3:7ca947c4:::rbd_data.11366acc894.000000000000000b:19 [read 491520~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:58:50.719 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.147698+0000", 2024-09-21T14:58:50.719 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.52689291500000002, 2024-09-21T14:58:50.719 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.016449135, 2024-09-21T14:58:50.719 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:50.719 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:50.719 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:50.720 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:50.720 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:50.720 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 23021 2024-09-21T14:58:50.720 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.720 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:50.720 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.720 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:50.720 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147698+0000", 2024-09-21T14:58:50.720 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:50.720 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.720 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.720 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:50.720 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147698+0000", 2024-09-21T14:58:50.720 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:50.720 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.720 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.721 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:50.721 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147698+0000", 2024-09-21T14:58:50.721 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:50.721 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.721 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.721 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:50.721 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147701+0000", 2024-09-21T14:58:50.721 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6299999999999998e-06 2024-09-21T14:58:50.721 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.721 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.721 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:50.721 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147706+0000", 2024-09-21T14:58:50.721 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3059999999999997e-06 2024-09-21T14:58:50.721 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.721 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.721 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:50.722 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147711+0000", 2024-09-21T14:58:50.722 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0860000000000001e-06 2024-09-21T14:58:50.722 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.722 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.722 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:50.722 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.163897+0000", 2024-09-21T14:58:50.722 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.016186282 2024-09-21T14:58:50.722 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.722 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.722 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:50.722 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.163986+0000", 2024-09-21T14:58:50.722 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8676000000000003e-05 2024-09-21T14:58:50.722 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.722 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.722 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:50.722 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.164148+0000", 2024-09-21T14:58:50.723 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016148000000000001 2024-09-21T14:58:50.723 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:50.723 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:50.723 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:50.723 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.723 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.723 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:23022 3.1e 3:7ca947c4:::rbd_data.11366acc894.000000000000000b:19 [read 557056~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:58:50.723 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.148289+0000", 2024-09-21T14:58:50.723 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.52630203099999995, 2024-09-21T14:58:50.723 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.016062979000000002, 2024-09-21T14:58:50.723 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:50.723 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:50.723 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:50.723 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:50.724 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:50.724 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 23022 2024-09-21T14:58:50.724 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.724 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:50.724 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.724 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:50.724 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.148289+0000", 2024-09-21T14:58:50.724 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:50.724 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.724 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.724 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:50.724 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.148289+0000", 2024-09-21T14:58:50.724 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:50.724 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.724 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.724 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:50.725 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.148288+0000", 2024-09-21T14:58:50.725 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:50.725 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.725 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.725 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:50.725 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.148295+0000", 2024-09-21T14:58:50.725 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8340000000000001e-06 2024-09-21T14:58:50.725 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.725 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.725 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:50.725 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.148304+0000", 2024-09-21T14:58:50.725 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0129999999999999e-06 2024-09-21T14:58:50.725 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.725 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.725 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:50.726 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.148311+0000", 2024-09-21T14:58:50.726 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1439999999999997e-06 2024-09-21T14:58:50.726 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.726 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.726 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:50.726 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.164182+0000", 2024-09-21T14:58:50.726 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015870825000000002 2024-09-21T14:58:50.726 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.726 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.726 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:50.726 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.164229+0000", 2024-09-21T14:58:50.726 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6508999999999998e-05 2024-09-21T14:58:50.726 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.727 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.727 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:50.727 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.164352+0000", 2024-09-21T14:58:50.727 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000123664 2024-09-21T14:58:50.727 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:50.727 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:50.727 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:50.727 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.727 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.727 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:23115 3.1e 3:7ca947c4:::rbd_data.11366acc894.000000000000000b:19 [read 114688~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:58:50.727 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.150277+0000", 2024-09-21T14:58:50.727 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.52431439499999999, 2024-09-21T14:58:50.727 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014686462000000001, 2024-09-21T14:58:50.727 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:50.727 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:50.727 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:50.728 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:50.728 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:50.728 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 23115 2024-09-21T14:58:50.728 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.728 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:50.728 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.728 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:50.728 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150277+0000", 2024-09-21T14:58:50.728 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:50.728 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.728 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.728 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:50.728 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150277+0000", 2024-09-21T14:58:50.728 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:50.728 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.728 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.729 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:50.729 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150277+0000", 2024-09-21T14:58:50.729 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:50.729 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.729 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.729 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:50.729 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150280+0000", 2024-09-21T14:58:50.729 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2349999999999999e-06 2024-09-21T14:58:50.729 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.729 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.729 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:50.729 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150286+0000", 2024-09-21T14:58:50.729 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5969999999999997e-06 2024-09-21T14:58:50.729 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.729 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.729 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:50.730 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150291+0000", 2024-09-21T14:58:50.730 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9359999999999998e-06 2024-09-21T14:58:50.730 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.730 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.730 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:50.730 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.164697+0000", 2024-09-21T14:58:50.730 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014406014999999999 2024-09-21T14:58:50.730 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.730 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.730 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:50.730 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.164784+0000", 2024-09-21T14:58:50.730 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6954999999999997e-05 2024-09-21T14:58:50.730 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.730 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.730 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:50.731 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.164963+0000", 2024-09-21T14:58:50.731 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017918 2024-09-21T14:58:50.731 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:50.731 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:50.731 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:50.731 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.731 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.731 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:23116 3.1e 3:7ca947c4:::rbd_data.11366acc894.000000000000000b:19 [read 180224~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:58:50.731 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.150296+0000", 2024-09-21T14:58:50.731 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.52429568800000004, 2024-09-21T14:58:50.731 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015003996, 2024-09-21T14:58:50.731 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:50.731 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:50.731 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:50.731 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:50.731 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:50.732 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 23116 2024-09-21T14:58:50.732 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.732 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:50.732 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.732 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:50.732 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150296+0000", 2024-09-21T14:58:50.732 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:50.732 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.732 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.732 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:50.732 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150296+0000", 2024-09-21T14:58:50.732 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:50.732 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.732 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.732 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:50.732 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150295+0000", 2024-09-21T14:58:50.733 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:50.733 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.733 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.733 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:50.733 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150298+0000", 2024-09-21T14:58:50.733 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6749999999999998e-06 2024-09-21T14:58:50.733 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.733 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.733 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:50.733 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150304+0000", 2024-09-21T14:58:50.733 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6359999999999999e-06 2024-09-21T14:58:50.733 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.733 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.733 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:50.733 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150308+0000", 2024-09-21T14:58:50.734 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6260000000000003e-06 2024-09-21T14:58:50.734 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.734 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.734 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:50.734 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.165003+0000", 2024-09-21T14:58:50.734 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014694377 2024-09-21T14:58:50.734 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.734 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.734 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:50.734 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.165093+0000", 2024-09-21T14:58:50.734 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0137999999999997e-05 2024-09-21T14:58:50.734 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.734 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.735 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:50.735 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.165300+0000", 2024-09-21T14:58:50.735 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00020688799999999999 2024-09-21T14:58:50.735 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:50.735 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:50.735 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:50.735 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.735 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.735 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:23117 3.1e 3:7ca947c4:::rbd_data.11366acc894.000000000000000b:19 [read 245760~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:58:50.735 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.150312+0000", 2024-09-21T14:58:50.735 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.52427935199999998, 2024-09-21T14:58:50.735 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015236939, 2024-09-21T14:58:50.735 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:50.735 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:50.735 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:50.735 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:50.736 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:50.736 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 23117 2024-09-21T14:58:50.736 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.736 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:50.736 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.736 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:50.736 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150312+0000", 2024-09-21T14:58:50.736 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:50.736 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.736 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.736 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:50.736 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150312+0000", 2024-09-21T14:58:50.736 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:50.736 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.736 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.736 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:50.737 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150312+0000", 2024-09-21T14:58:50.737 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:50.737 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.737 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.737 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:50.737 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150314+0000", 2024-09-21T14:58:50.737 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.154e-06 2024-09-21T14:58:50.737 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.737 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.737 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:50.737 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150320+0000", 2024-09-21T14:58:50.737 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6640000000000003e-06 2024-09-21T14:58:50.737 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.737 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.737 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:50.737 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150324+0000", 2024-09-21T14:58:50.738 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5669999999999996e-06 2024-09-21T14:58:50.738 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.738 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.738 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:50.738 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.165317+0000", 2024-09-21T14:58:50.738 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014993328 2024-09-21T14:58:50.738 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.738 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.738 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:50.738 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.165370+0000", 2024-09-21T14:58:50.738 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2036999999999997e-05 2024-09-21T14:58:50.738 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.738 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.738 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:50.738 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.165549+0000", 2024-09-21T14:58:50.739 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017947500000000001 2024-09-21T14:58:50.739 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:50.739 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:50.739 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:50.739 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.739 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.739 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:23118 3.1e 3:7ca947c4:::rbd_data.11366acc894.000000000000000b:19 [read 311296~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:58:50.739 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.150328+0000", 2024-09-21T14:58:50.739 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.52426320699999995, 2024-09-21T14:58:50.739 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015505385999999999, 2024-09-21T14:58:50.739 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:50.739 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:50.739 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:50.739 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:50.739 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:50.739 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 23118 2024-09-21T14:58:50.739 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.740 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:50.740 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.740 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:50.740 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150328+0000", 2024-09-21T14:58:50.740 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:50.740 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.740 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.740 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:50.740 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150328+0000", 2024-09-21T14:58:50.740 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:50.740 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.740 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.740 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:50.740 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150328+0000", 2024-09-21T14:58:50.740 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:50.741 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.741 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.741 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:50.741 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150330+0000", 2024-09-21T14:58:50.741 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3410000000000001e-06 2024-09-21T14:58:50.741 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.741 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.741 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:50.741 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150336+0000", 2024-09-21T14:58:50.741 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.694e-06 2024-09-21T14:58:50.741 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.741 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.741 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:50.741 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150343+0000", 2024-09-21T14:58:50.741 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3050000000000004e-06 2024-09-21T14:58:50.741 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.742 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.742 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:50.742 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.165607+0000", 2024-09-21T14:58:50.742 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015264534 2024-09-21T14:58:50.742 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.742 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.742 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:50.742 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.165661+0000", 2024-09-21T14:58:50.742 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3949000000000001e-05 2024-09-21T14:58:50.742 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.742 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.742 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:50.742 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.165834+0000", 2024-09-21T14:58:50.742 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000172139 2024-09-21T14:58:50.742 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:50.743 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:50.743 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:50.743 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.743 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.743 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:23119 3.1e 3:7ca947c4:::rbd_data.11366acc894.000000000000000b:19 [read 376832~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:58:50.743 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.150347+0000", 2024-09-21T14:58:50.743 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.52424418800000006, 2024-09-21T14:58:50.743 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015790017, 2024-09-21T14:58:50.743 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:50.743 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:50.743 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:50.743 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:50.743 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:50.743 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 23119 2024-09-21T14:58:50.743 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.743 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:50.744 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.744 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:50.744 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150347+0000", 2024-09-21T14:58:50.744 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:50.744 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.744 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.744 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:50.744 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150347+0000", 2024-09-21T14:58:50.744 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:50.744 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.744 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.744 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:50.744 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150347+0000", 2024-09-21T14:58:50.744 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:50.744 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.745 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.745 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:50.745 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150349+0000", 2024-09-21T14:58:50.745 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1730000000000002e-06 2024-09-21T14:58:50.745 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.745 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.745 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:50.745 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150355+0000", 2024-09-21T14:58:50.745 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9320000000000001e-06 2024-09-21T14:58:50.745 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.745 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.745 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:50.745 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150360+0000", 2024-09-21T14:58:50.745 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9450000000000001e-06 2024-09-21T14:58:50.745 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.746 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.746 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:50.746 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.165867+0000", 2024-09-21T14:58:50.746 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015507191 2024-09-21T14:58:50.746 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.746 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.746 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:50.746 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.165956+0000", 2024-09-21T14:58:50.746 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9235999999999997e-05 2024-09-21T14:58:50.746 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.746 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.746 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:50.746 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.166137+0000", 2024-09-21T14:58:50.746 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018088699999999999 2024-09-21T14:58:50.747 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:50.747 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:50.747 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:50.747 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.747 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.747 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:23120 3.1e 3:7ca947c4:::rbd_data.11366acc894.000000000000000b:19 [read 442368~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:58:50.747 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.150364+0000", 2024-09-21T14:58:50.747 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.52422751300000003, 2024-09-21T14:58:50.747 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.016001277000000001, 2024-09-21T14:58:50.747 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:50.747 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:50.747 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:50.747 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:50.747 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:50.747 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 23120 2024-09-21T14:58:50.747 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.748 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:50.748 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.748 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:50.748 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150364+0000", 2024-09-21T14:58:50.748 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:50.748 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.748 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.748 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:50.748 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150364+0000", 2024-09-21T14:58:50.748 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:50.748 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.748 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.748 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:50.748 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150363+0000", 2024-09-21T14:58:50.748 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:50.749 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.749 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.749 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:50.749 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150366+0000", 2024-09-21T14:58:50.749 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1969999999999999e-06 2024-09-21T14:58:50.749 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.749 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.749 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:50.749 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150371+0000", 2024-09-21T14:58:50.749 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7459999999999997e-06 2024-09-21T14:58:50.749 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.749 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.749 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:50.749 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150376+0000", 2024-09-21T14:58:50.749 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6519999999999997e-06 2024-09-21T14:58:50.749 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.750 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.750 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:50.750 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.166166+0000", 2024-09-21T14:58:50.750 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015790286000000001 2024-09-21T14:58:50.750 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.750 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.750 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:50.750 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.166226+0000", 2024-09-21T14:58:50.750 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9682000000000002e-05 2024-09-21T14:58:50.750 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.750 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.750 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:50.750 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.166365+0000", 2024-09-21T14:58:50.750 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013911800000000001 2024-09-21T14:58:50.750 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:50.751 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:50.751 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:50.751 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.751 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.751 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:23121 3.1e 3:7ca947c4:::rbd_data.11366acc894.000000000000000b:19 [read 507904~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:58:50.751 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.150380+0000", 2024-09-21T14:58:50.751 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.52421138199999995, 2024-09-21T14:58:50.751 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.016302104000000001, 2024-09-21T14:58:50.751 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:50.751 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:50.751 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:50.751 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:50.751 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:50.751 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 23121 2024-09-21T14:58:50.751 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.752 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:50.752 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.752 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:50.752 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150380+0000", 2024-09-21T14:58:50.752 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:50.752 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.752 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.752 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:50.752 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150380+0000", 2024-09-21T14:58:50.752 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:50.752 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.752 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.752 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:50.752 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150380+0000", 2024-09-21T14:58:50.752 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:50.752 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.753 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.753 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:50.753 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150382+0000", 2024-09-21T14:58:50.753 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1720000000000001e-06 2024-09-21T14:58:50.753 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.753 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.753 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:50.753 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150387+0000", 2024-09-21T14:58:50.753 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2569999999999998e-06 2024-09-21T14:58:50.753 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.753 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.753 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:50.753 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150392+0000", 2024-09-21T14:58:50.753 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1100000000000002e-06 2024-09-21T14:58:50.753 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.753 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.754 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:50.754 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.166392+0000", 2024-09-21T14:58:50.754 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015999394 2024-09-21T14:58:50.754 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.754 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.754 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:50.754 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.166461+0000", 2024-09-21T14:58:50.754 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9035999999999994e-05 2024-09-21T14:58:50.754 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.754 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.754 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:50.754 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.166682+0000", 2024-09-21T14:58:50.754 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00022143900000000001 2024-09-21T14:58:50.754 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:50.755 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:50.755 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:50.755 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.755 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.755 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:23122 3.1e 3:7ca947c4:::rbd_data.11366acc894.000000000000000b:19 [read 573440~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:58:50.755 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.150396+0000", 2024-09-21T14:58:50.755 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.52419545899999997, 2024-09-21T14:58:50.755 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.016563735999999999, 2024-09-21T14:58:50.755 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:50.755 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:50.755 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:50.755 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:50.755 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:50.755 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 23122 2024-09-21T14:58:50.755 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.756 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:50.756 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.756 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:50.756 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150396+0000", 2024-09-21T14:58:50.756 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:50.756 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.756 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.756 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:50.756 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150396+0000", 2024-09-21T14:58:50.756 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:50.756 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.756 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.756 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:50.756 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150396+0000", 2024-09-21T14:58:50.756 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:50.756 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.757 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.757 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:50.757 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150398+0000", 2024-09-21T14:58:50.757 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1739999999999999e-06 2024-09-21T14:58:50.757 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.757 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.757 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:50.757 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150403+0000", 2024-09-21T14:58:50.757 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1490000000000002e-06 2024-09-21T14:58:50.757 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.757 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.757 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:50.757 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150408+0000", 2024-09-21T14:58:50.757 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6460000000000001e-06 2024-09-21T14:58:50.757 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.758 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.758 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:50.758 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.166716+0000", 2024-09-21T14:58:50.758 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.016308804999999999 2024-09-21T14:58:50.758 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.758 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.758 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:50.758 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.166792+0000", 2024-09-21T14:58:50.758 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5479e-05 2024-09-21T14:58:50.758 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:50.758 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:50.758 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:50.758 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.166960+0000", 2024-09-21T14:58:50.758 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000167799 2024-09-21T14:58:50.758 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:50.759 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:50.759 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:50.759 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:50.759 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:50.759 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:58:50.760 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T14:58:50.760 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:58:50.756+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:58:50.861 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:58:50.860+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:58:50.886 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T14:58:50.916 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:58:50.916 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:58:50.916 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T14:58:50.916 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:58:50.926 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T14:58:50.961 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:58:50.960+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:58:51.038 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:58:51.038 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:58:51.038 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T14:58:51.038 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T14:58:51.038 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:58:51.044 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T14:58:51.048 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T14:58:51.064 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T14:58:51.084 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T14:58:51.089 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:58:51.088+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:58:51.103 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T14:58:51.123 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-21T14:58:51.145 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T14:58:51.163 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:58:51.161+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:58:51.169 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T14:58:51.187 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T14:58:51.187 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T14:58:51.187 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-21T14:58:51.187 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-21T14:58:51.193 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T14:58:51.197 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:58:51.197 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:58:51.197 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:58:51.197 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:58:51.197 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:58:51.197 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:58:51.197 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:58:51.197 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:58:51.201 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:51.201 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:51.213 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T14:58:51.264 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:58:51.260+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:58:51.313 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:51.313 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:51.323 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T14:58:51.332 INFO:teuthology.orchestra.run.smithi032.stdout:checking clone #6, image image_client.0-clone6 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent7 2024-09-21T14:58:51.365 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:58:51.361+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:58:51.466 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:58:51.465+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:58:51.484 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:51.484 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:51.494 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T14:58:51.509 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-21T14:58:51.509 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:58:51.509 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 56, 2024-09-21T14:58:51.509 INFO:teuthology.orchestra.run.smithi032.stdout: "fsid": "caf51e55-9234-4a26-bcd2-d60811962560", 2024-09-21T14:58:51.509 INFO:teuthology.orchestra.run.smithi032.stdout: "created": "2024-09-21T14:57:50.338822+0000", 2024-09-21T14:58:51.509 INFO:teuthology.orchestra.run.smithi032.stdout: "modified": "2024-09-21T14:58:51.192423+0000", 2024-09-21T14:58:51.509 INFO:teuthology.orchestra.run.smithi032.stdout: "last_up_change": "2024-09-21T14:58:01.177141+0000", 2024-09-21T14:58:51.509 INFO:teuthology.orchestra.run.smithi032.stdout: "last_in_change": "2024-09-21T14:57:52.442036+0000", 2024-09-21T14:58:51.509 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-21T14:58:51.509 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_num": 5799936, 2024-09-21T14:58:51.509 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_set": [ 2024-09-21T14:58:51.510 INFO:teuthology.orchestra.run.smithi032.stdout: "pglog_hardlimit", 2024-09-21T14:58:51.510 INFO:teuthology.orchestra.run.smithi032.stdout: "purged_snapdirs", 2024-09-21T14:58:51.510 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_deletes", 2024-09-21T14:58:51.510 INFO:teuthology.orchestra.run.smithi032.stdout: "sortbitwise" 2024-09-21T14:58:51.510 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-21T14:58:51.510 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_version": 5, 2024-09-21T14:58:51.510 INFO:teuthology.orchestra.run.smithi032.stdout: "full_ratio": 0.94999998807907104, 2024-09-21T14:58:51.510 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-21T14:58:51.510 INFO:teuthology.orchestra.run.smithi032.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-21T14:58:51.510 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_snapshot": "", 2024-09-21T14:58:51.510 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_max": 3, 2024-09-21T14:58:51.510 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd": 8, 2024-09-21T14:58:51.510 INFO:teuthology.orchestra.run.smithi032.stdout: "require_min_compat_client": "luminous", 2024-09-21T14:58:51.510 INFO:teuthology.orchestra.run.smithi032.stdout: "min_compat_client": "luminous", 2024-09-21T14:58:51.510 INFO:teuthology.orchestra.run.smithi032.stdout: "require_osd_release": "reef", 2024-09-21T14:58:51.511 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_crimson": false, 2024-09-21T14:58:51.511 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2024-09-21T14:58:51.511 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:51.511 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 1, 2024-09-21T14:58:51.511 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": ".mgr", 2024-09-21T14:58:51.511 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-09-21T14:58:01.842810+0000", 2024-09-21T14:58:51.511 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-09-21T14:58:51.511 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2024-09-21T14:58:51.511 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-09-21T14:58:51.511 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-09-21T14:58:51.511 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-09-21T14:58:51.511 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-09-21T14:58:51.511 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-09-21T14:58:51.511 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-09-21T14:58:51.511 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T14:58:51.512 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T14:58:51.512 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-09-21T14:58:51.512 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-09-21T14:58:51.512 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 1, 2024-09-21T14:58:51.512 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 1, 2024-09-21T14:58:51.512 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 1, 2024-09-21T14:58:51.512 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 1, 2024-09-21T14:58:51.512 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 1, 2024-09-21T14:58:51.512 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-09-21T14:58:51.512 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-09-21T14:58:51.512 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-09-21T14:58:51.512 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-09-21T14:58:51.512 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-09-21T14:58:51.512 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-09-21T14:58:51.513 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-09-21T14:58:51.513 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.513 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "18", 2024-09-21T14:58:51.513 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-09-21T14:58:51.513 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T14:58:51.513 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T14:58:51.513 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-09-21T14:58:51.513 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-09-21T14:58:51.513 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-09-21T14:58:51.513 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-09-21T14:58:51.513 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-09-21T14:58:51.513 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-09-21T14:58:51.513 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-09-21T14:58:51.513 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-09-21T14:58:51.513 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-09-21T14:58:51.513 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-09-21T14:58:51.514 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-09-21T14:58:51.514 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-09-21T14:58:51.514 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-09-21T14:58:51.514 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-09-21T14:58:51.514 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-09-21T14:58:51.514 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T14:58:51.514 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T14:58:51.514 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T14:58:51.514 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-09-21T14:58:51.514 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-09-21T14:58:51.514 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-09-21T14:58:51.514 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-09-21T14:58:51.514 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-09-21T14:58:51.514 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.514 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-09-21T14:58:51.515 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-09-21T14:58:51.515 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-09-21T14:58:51.515 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-09-21T14:58:51.515 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-09-21T14:58:51.515 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T14:58:51.515 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-09-21T14:58:51.515 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-09-21T14:58:51.515 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-09-21T14:58:51.515 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-09-21T14:58:51.515 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-09-21T14:58:51.515 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-09-21T14:58:51.515 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_max": 32, 2024-09-21T14:58:51.515 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 1 2024-09-21T14:58:51.515 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.516 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-09-21T14:58:51.516 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": {} 2024-09-21T14:58:51.516 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.516 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-09-21T14:58:51.516 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 8, 2024-09-21T14:58:51.516 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 8, 2024-09-21T14:58:51.516 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 0.25, 2024-09-21T14:58:51.516 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 2, 2024-09-21T14:58:51.516 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 2, 2024-09-21T14:58:51.516 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 0.88126564025878906, 2024-09-21T14:58:51.516 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 0.87999999523162842, 2024-09-21T14:58:51.516 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-21T14:58:51.516 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:51.516 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.516 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:51.517 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 2, 2024-09-21T14:58:51.517 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "rbd", 2024-09-21T14:58:51.517 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-09-21T14:58:08.284925+0000", 2024-09-21T14:58:51.517 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 8193, 2024-09-21T14:58:51.517 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T14:58:51.517 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-09-21T14:58:51.517 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-09-21T14:58:51.517 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-09-21T14:58:51.517 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-09-21T14:58:51.517 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-09-21T14:58:51.517 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-09-21T14:58:51.517 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T14:58:51.517 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T14:58:51.518 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-09-21T14:58:51.518 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-09-21T14:58:51.518 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2024-09-21T14:58:51.518 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2024-09-21T14:58:51.518 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2024-09-21T14:58:51.518 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2024-09-21T14:58:51.518 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2024-09-21T14:58:51.518 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-09-21T14:58:51.518 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-09-21T14:58:51.518 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-09-21T14:58:51.518 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-09-21T14:58:51.518 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-09-21T14:58:51.518 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-09-21T14:58:51.518 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-09-21T14:58:51.518 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.519 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "22", 2024-09-21T14:58:51.519 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-09-21T14:58:51.519 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T14:58:51.519 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T14:58:51.519 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-09-21T14:58:51.519 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-09-21T14:58:51.519 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 3, 2024-09-21T14:58:51.519 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 22, 2024-09-21T14:58:51.519 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-09-21T14:58:51.519 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-09-21T14:58:51.519 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-09-21T14:58:51.519 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-09-21T14:58:51.519 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-09-21T14:58:51.519 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-09-21T14:58:51.519 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-09-21T14:58:51.520 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-09-21T14:58:51.520 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-09-21T14:58:51.520 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-09-21T14:58:51.520 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-09-21T14:58:51.520 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T14:58:51.520 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T14:58:51.520 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T14:58:51.520 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-09-21T14:58:51.520 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-09-21T14:58:51.520 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-09-21T14:58:51.520 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-09-21T14:58:51.520 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-09-21T14:58:51.520 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.520 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-09-21T14:58:51.520 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-09-21T14:58:51.520 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-09-21T14:58:51.521 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-09-21T14:58:51.521 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-09-21T14:58:51.521 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T14:58:51.521 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-09-21T14:58:51.521 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-09-21T14:58:51.521 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-09-21T14:58:51.521 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-09-21T14:58:51.521 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-09-21T14:58:51.521 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2024-09-21T14:58:51.521 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-09-21T14:58:51.521 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd": {} 2024-09-21T14:58:51.521 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.521 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-09-21T14:58:51.521 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 4.9800000190734863, 2024-09-21T14:58:51.521 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 4.9800000190734863, 2024-09-21T14:58:51.521 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 0.87999999523162842, 2024-09-21T14:58:51.522 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 4.380000114440918, 2024-09-21T14:58:51.522 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 4.380000114440918, 2024-09-21T14:58:51.522 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 0.88126564025878906, 2024-09-21T14:58:51.522 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 0.87999999523162842, 2024-09-21T14:58:51.522 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-21T14:58:51.522 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:51.522 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.522 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:51.522 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 3, 2024-09-21T14:58:51.522 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "pool_client.0", 2024-09-21T14:58:51.522 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-09-21T14:58:17.510709+0000", 2024-09-21T14:58:51.522 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 8193, 2024-09-21T14:58:51.522 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T14:58:51.522 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-09-21T14:58:51.522 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-09-21T14:58:51.523 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-09-21T14:58:51.523 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-09-21T14:58:51.523 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-09-21T14:58:51.523 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-09-21T14:58:51.523 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T14:58:51.523 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T14:58:51.523 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-09-21T14:58:51.523 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-09-21T14:58:51.523 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 32, 2024-09-21T14:58:51.523 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 32, 2024-09-21T14:58:51.523 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 32, 2024-09-21T14:58:51.523 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 32, 2024-09-21T14:58:51.523 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 32, 2024-09-21T14:58:51.523 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-09-21T14:58:51.523 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-09-21T14:58:51.523 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-09-21T14:58:51.523 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-09-21T14:58:51.524 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-09-21T14:58:51.524 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-09-21T14:58:51.524 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-09-21T14:58:51.524 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.524 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "56", 2024-09-21T14:58:51.524 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-09-21T14:58:51.524 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T14:58:51.524 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T14:58:51.524 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-09-21T14:58:51.524 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-09-21T14:58:51.524 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 27, 2024-09-21T14:58:51.524 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 56, 2024-09-21T14:58:51.524 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-09-21T14:58:51.524 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-09-21T14:58:51.525 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-09-21T14:58:51.525 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-09-21T14:58:51.525 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-09-21T14:58:51.525 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-09-21T14:58:51.525 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-09-21T14:58:51.525 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-09-21T14:58:51.525 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-09-21T14:58:51.525 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-09-21T14:58:51.525 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-09-21T14:58:51.525 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T14:58:51.525 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T14:58:51.525 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T14:58:51.525 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-09-21T14:58:51.525 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-09-21T14:58:51.525 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-09-21T14:58:51.525 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-09-21T14:58:51.525 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-09-21T14:58:51.526 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.526 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-09-21T14:58:51.526 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-09-21T14:58:51.526 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-09-21T14:58:51.526 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-09-21T14:58:51.526 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-09-21T14:58:51.526 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T14:58:51.526 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-09-21T14:58:51.526 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-09-21T14:58:51.526 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-09-21T14:58:51.526 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-09-21T14:58:51.526 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-09-21T14:58:51.526 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2024-09-21T14:58:51.526 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-09-21T14:58:51.526 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd": {} 2024-09-21T14:58:51.527 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.527 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-09-21T14:58:51.527 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 2, 2024-09-21T14:58:51.527 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 2, 2024-09-21T14:58:51.527 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 1, 2024-09-21T14:58:51.527 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 2, 2024-09-21T14:58:51.527 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 2, 2024-09-21T14:58:51.527 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 0.88126564025878906, 2024-09-21T14:58:51.527 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 0.87999999523162842, 2024-09-21T14:58:51.527 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-21T14:58:51.527 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:51.527 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:51.527 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-21T14:58:51.527 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2024-09-21T14:58:51.527 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:51.527 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-09-21T14:58:51.528 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "a4a73316-3a94-42e9-9a51-deb46a66bb06", 2024-09-21T14:58:51.528 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-21T14:58:51.528 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-21T14:58:51.528 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-21T14:58:51.528 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-21T14:58:51.528 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-21T14:58:51.528 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-21T14:58:51.528 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2024-09-21T14:58:51.528 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 24, 2024-09-21T14:58:51.528 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-21T14:58:51.528 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-21T14:58:51.528 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-21T14:58:51.528 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:51.528 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:51.528 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:51.528 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6808", 2024-09-21T14:58:51.528 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4016672243 2024-09-21T14:58:51.529 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.529 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:51.529 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:51.529 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6809", 2024-09-21T14:58:51.529 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4016672243 2024-09-21T14:58:51.529 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:51.529 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:51.529 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.529 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-21T14:58:51.529 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:51.529 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:51.529 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:51.529 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6810", 2024-09-21T14:58:51.529 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4016672243 2024-09-21T14:58:51.529 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.529 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:51.530 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:51.530 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6811", 2024-09-21T14:58:51.530 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4016672243 2024-09-21T14:58:51.530 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:51.530 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:51.530 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.530 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-21T14:58:51.530 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:51.530 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:51.530 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:51.530 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6814", 2024-09-21T14:58:51.530 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4016672243 2024-09-21T14:58:51.530 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.531 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:51.531 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:51.531 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6815", 2024-09-21T14:58:51.531 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4016672243 2024-09-21T14:58:51.531 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:51.531 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:51.531 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.531 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-21T14:58:51.531 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:51.531 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:51.531 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:51.531 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6812", 2024-09-21T14:58:51.531 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4016672243 2024-09-21T14:58:51.531 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.531 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:51.531 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:51.532 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6813", 2024-09-21T14:58:51.532 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4016672243 2024-09-21T14:58:51.532 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:51.532 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:51.532 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.532 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6809/4016672243", 2024-09-21T14:58:51.532 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6811/4016672243", 2024-09-21T14:58:51.532 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6815/4016672243", 2024-09-21T14:58:51.532 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6813/4016672243", 2024-09-21T14:58:51.532 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-21T14:58:51.532 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-21T14:58:51.532 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-21T14:58:51.532 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:51.532 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.532 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:51.533 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-09-21T14:58:51.533 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "006757a0-94d1-4a9b-aef8-73634cc5828e", 2024-09-21T14:58:51.533 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-21T14:58:51.533 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-21T14:58:51.533 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-21T14:58:51.533 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-21T14:58:51.533 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-21T14:58:51.533 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-21T14:58:51.533 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2024-09-21T14:58:51.533 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 24, 2024-09-21T14:58:51.533 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-21T14:58:51.533 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-21T14:58:51.533 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-21T14:58:51.534 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:51.534 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:51.534 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:51.534 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6824", 2024-09-21T14:58:51.534 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 641090577 2024-09-21T14:58:51.534 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.534 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:51.534 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:51.534 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6825", 2024-09-21T14:58:51.534 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 641090577 2024-09-21T14:58:51.534 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:51.534 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:51.534 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.534 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-21T14:58:51.535 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:51.535 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:51.535 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:51.535 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6826", 2024-09-21T14:58:51.535 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 641090577 2024-09-21T14:58:51.535 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.535 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:51.535 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:51.535 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6827", 2024-09-21T14:58:51.535 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 641090577 2024-09-21T14:58:51.535 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:51.535 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:51.535 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.535 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-21T14:58:51.535 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:51.536 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:51.536 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:51.536 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6830", 2024-09-21T14:58:51.536 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 641090577 2024-09-21T14:58:51.536 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.536 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:51.536 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:51.536 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6831", 2024-09-21T14:58:51.536 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 641090577 2024-09-21T14:58:51.536 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:51.536 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:51.536 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.536 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-21T14:58:51.536 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:51.536 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:51.536 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:51.537 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6828", 2024-09-21T14:58:51.537 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 641090577 2024-09-21T14:58:51.537 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.537 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:51.537 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:51.537 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6829", 2024-09-21T14:58:51.537 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 641090577 2024-09-21T14:58:51.537 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:51.537 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:51.537 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.537 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6825/641090577", 2024-09-21T14:58:51.537 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6827/641090577", 2024-09-21T14:58:51.537 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6831/641090577", 2024-09-21T14:58:51.537 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6829/641090577", 2024-09-21T14:58:51.537 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-21T14:58:51.537 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-21T14:58:51.538 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-21T14:58:51.538 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:51.538 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.538 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:51.538 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-09-21T14:58:51.538 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "0b499a23-820e-4e6c-a558-adbf88b7014f", 2024-09-21T14:58:51.538 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-21T14:58:51.538 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-21T14:58:51.538 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-21T14:58:51.538 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-21T14:58:51.538 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-21T14:58:51.538 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-21T14:58:51.538 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2024-09-21T14:58:51.538 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 51, 2024-09-21T14:58:51.538 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-21T14:58:51.538 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-21T14:58:51.539 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-21T14:58:51.539 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:51.539 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:51.539 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:51.539 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6800", 2024-09-21T14:58:51.539 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2609154414 2024-09-21T14:58:51.539 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.539 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:51.539 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:51.539 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6801", 2024-09-21T14:58:51.539 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2609154414 2024-09-21T14:58:51.539 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:51.539 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:51.539 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.539 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-21T14:58:51.539 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:51.540 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:51.540 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:51.540 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6802", 2024-09-21T14:58:51.540 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2609154414 2024-09-21T14:58:51.540 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.540 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:51.540 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:51.540 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6803", 2024-09-21T14:58:51.540 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2609154414 2024-09-21T14:58:51.540 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:51.540 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:51.540 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.540 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-21T14:58:51.540 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:51.540 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:51.540 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:51.541 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6806", 2024-09-21T14:58:51.541 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2609154414 2024-09-21T14:58:51.541 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.541 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:51.541 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:51.541 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6807", 2024-09-21T14:58:51.541 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2609154414 2024-09-21T14:58:51.541 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:51.541 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:51.541 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.541 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-21T14:58:51.541 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:51.541 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:51.541 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:51.541 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6804", 2024-09-21T14:58:51.541 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2609154414 2024-09-21T14:58:51.542 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.542 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:51.542 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:51.542 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6805", 2024-09-21T14:58:51.542 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2609154414 2024-09-21T14:58:51.542 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:51.542 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:51.542 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.542 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6801/2609154414", 2024-09-21T14:58:51.542 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6803/2609154414", 2024-09-21T14:58:51.542 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6807/2609154414", 2024-09-21T14:58:51.542 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6805/2609154414", 2024-09-21T14:58:51.542 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-21T14:58:51.542 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-21T14:58:51.543 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-21T14:58:51.543 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:51.543 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.543 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:51.543 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-09-21T14:58:51.543 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "5a57b3dd-4a07-45b7-b159-3cac4c7eb262", 2024-09-21T14:58:51.543 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-21T14:58:51.543 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-21T14:58:51.543 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-21T14:58:51.543 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-21T14:58:51.543 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-21T14:58:51.543 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-21T14:58:51.543 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2024-09-21T14:58:51.543 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 51, 2024-09-21T14:58:51.543 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-21T14:58:51.544 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-21T14:58:51.544 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-21T14:58:51.544 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:51.544 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:51.544 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:51.544 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6816", 2024-09-21T14:58:51.544 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1462563376 2024-09-21T14:58:51.544 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.544 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:51.544 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:51.544 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6817", 2024-09-21T14:58:51.544 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1462563376 2024-09-21T14:58:51.544 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:51.544 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:51.544 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.544 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-21T14:58:51.545 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:51.545 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:51.545 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:51.545 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6818", 2024-09-21T14:58:51.545 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1462563376 2024-09-21T14:58:51.545 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.545 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:51.545 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:51.545 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6819", 2024-09-21T14:58:51.545 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1462563376 2024-09-21T14:58:51.545 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:51.545 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:51.545 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.545 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-21T14:58:51.545 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:51.545 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:51.546 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:51.546 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6822", 2024-09-21T14:58:51.546 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1462563376 2024-09-21T14:58:51.546 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.546 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:51.546 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:51.546 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6823", 2024-09-21T14:58:51.546 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1462563376 2024-09-21T14:58:51.546 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:51.546 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:51.546 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.546 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-21T14:58:51.546 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:51.546 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:51.546 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:51.547 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6820", 2024-09-21T14:58:51.547 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1462563376 2024-09-21T14:58:51.547 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.547 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:51.547 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:51.547 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6821", 2024-09-21T14:58:51.547 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1462563376 2024-09-21T14:58:51.547 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:51.547 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:51.547 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.547 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6817/1462563376", 2024-09-21T14:58:51.547 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6819/1462563376", 2024-09-21T14:58:51.547 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6823/1462563376", 2024-09-21T14:58:51.547 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6821/1462563376", 2024-09-21T14:58:51.547 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-21T14:58:51.548 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-21T14:58:51.548 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-21T14:58:51.548 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:51.548 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.548 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:51.548 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-09-21T14:58:51.548 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "39a00807-7517-474e-8842-623dd7093cc5", 2024-09-21T14:58:51.548 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-21T14:58:51.548 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-21T14:58:51.548 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-21T14:58:51.548 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-21T14:58:51.548 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-21T14:58:51.548 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-21T14:58:51.548 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2024-09-21T14:58:51.548 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 24, 2024-09-21T14:58:51.549 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-21T14:58:51.549 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-21T14:58:51.549 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-21T14:58:51.549 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:51.549 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:51.549 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:51.549 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6801", 2024-09-21T14:58:51.549 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 144412042 2024-09-21T14:58:51.549 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.549 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:51.549 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:51.549 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6803", 2024-09-21T14:58:51.549 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 144412042 2024-09-21T14:58:51.549 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:51.549 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:51.549 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.550 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-21T14:58:51.550 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:51.550 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:51.550 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:51.550 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6805", 2024-09-21T14:58:51.550 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 144412042 2024-09-21T14:58:51.550 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.550 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:51.550 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:51.550 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6807", 2024-09-21T14:58:51.550 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 144412042 2024-09-21T14:58:51.550 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:51.550 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:51.550 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.550 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-21T14:58:51.551 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:51.551 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:51.551 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:51.551 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6812", 2024-09-21T14:58:51.551 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 144412042 2024-09-21T14:58:51.551 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.551 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:51.551 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:51.551 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6814", 2024-09-21T14:58:51.551 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 144412042 2024-09-21T14:58:51.551 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:51.551 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:51.551 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.552 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-21T14:58:51.552 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:51.552 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:51.552 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:51.552 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6809", 2024-09-21T14:58:51.552 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 144412042 2024-09-21T14:58:51.552 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.552 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:51.552 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:51.552 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6811", 2024-09-21T14:58:51.552 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 144412042 2024-09-21T14:58:51.552 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:51.552 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:51.552 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.552 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.46:6803/144412042", 2024-09-21T14:58:51.552 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.46:6807/144412042", 2024-09-21T14:58:51.553 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.46:6814/144412042", 2024-09-21T14:58:51.553 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.46:6811/144412042", 2024-09-21T14:58:51.553 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-21T14:58:51.553 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-21T14:58:51.553 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-21T14:58:51.553 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:51.553 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.553 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:51.553 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-09-21T14:58:51.553 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "edf9f045-ddeb-4504-a0fc-1a0e4e853550", 2024-09-21T14:58:51.553 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-21T14:58:51.553 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-21T14:58:51.553 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-21T14:58:51.553 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-21T14:58:51.553 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-21T14:58:51.554 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-21T14:58:51.554 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2024-09-21T14:58:51.554 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 51, 2024-09-21T14:58:51.554 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-21T14:58:51.554 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-21T14:58:51.554 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-21T14:58:51.554 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:51.554 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:51.554 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:51.554 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6800", 2024-09-21T14:58:51.554 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 849101442 2024-09-21T14:58:51.554 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.554 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:51.554 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:51.555 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6802", 2024-09-21T14:58:51.555 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 849101442 2024-09-21T14:58:51.555 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:51.555 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:51.555 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.555 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-21T14:58:51.555 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:51.555 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:51.555 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:51.555 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6804", 2024-09-21T14:58:51.555 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 849101442 2024-09-21T14:58:51.555 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.555 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:51.555 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:51.555 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6806", 2024-09-21T14:58:51.556 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 849101442 2024-09-21T14:58:51.556 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:51.556 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:51.556 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.556 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-21T14:58:51.556 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:51.556 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:51.556 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:51.556 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6813", 2024-09-21T14:58:51.556 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 849101442 2024-09-21T14:58:51.556 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.556 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:51.556 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:51.556 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6815", 2024-09-21T14:58:51.556 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 849101442 2024-09-21T14:58:51.556 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:51.557 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:51.557 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.557 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-21T14:58:51.557 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:51.557 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:51.557 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:51.557 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6808", 2024-09-21T14:58:51.557 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 849101442 2024-09-21T14:58:51.557 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.557 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:51.557 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:51.557 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6810", 2024-09-21T14:58:51.557 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 849101442 2024-09-21T14:58:51.557 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:51.558 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:51.558 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.558 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.46:6802/849101442", 2024-09-21T14:58:51.558 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.46:6806/849101442", 2024-09-21T14:58:51.558 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.46:6815/849101442", 2024-09-21T14:58:51.558 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.46:6810/849101442", 2024-09-21T14:58:51.558 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-21T14:58:51.558 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-21T14:58:51.558 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-21T14:58:51.558 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:51.558 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.558 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:51.558 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-09-21T14:58:51.558 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "c820ea6b-88fe-4421-83ec-17c466691fea", 2024-09-21T14:58:51.558 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-21T14:58:51.559 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-21T14:58:51.559 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-21T14:58:51.559 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 0.0501251220703125, 2024-09-21T14:58:51.559 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-21T14:58:51.559 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-21T14:58:51.559 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2024-09-21T14:58:51.559 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 24, 2024-09-21T14:58:51.559 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-21T14:58:51.559 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-21T14:58:51.559 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-21T14:58:51.559 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:51.559 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:51.559 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:51.559 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6817", 2024-09-21T14:58:51.560 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 847189172 2024-09-21T14:58:51.560 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.560 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:51.560 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:51.560 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6818", 2024-09-21T14:58:51.560 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 847189172 2024-09-21T14:58:51.560 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:51.560 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:51.560 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.560 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-21T14:58:51.560 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:51.560 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:51.560 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:51.560 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6820", 2024-09-21T14:58:51.560 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 847189172 2024-09-21T14:58:51.561 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.561 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:51.561 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:51.561 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6822", 2024-09-21T14:58:51.561 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 847189172 2024-09-21T14:58:51.561 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:51.561 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:51.561 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.561 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-21T14:58:51.561 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:51.561 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:51.561 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:51.561 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6827", 2024-09-21T14:58:51.561 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 847189172 2024-09-21T14:58:51.562 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.562 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:51.562 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:51.562 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6828", 2024-09-21T14:58:51.562 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 847189172 2024-09-21T14:58:51.562 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:51.562 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:51.562 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.562 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-21T14:58:51.562 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:51.562 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:51.562 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:51.562 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6823", 2024-09-21T14:58:51.562 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 847189172 2024-09-21T14:58:51.563 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.563 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:51.563 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:51.563 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6825", 2024-09-21T14:58:51.563 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 847189172 2024-09-21T14:58:51.563 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:51.563 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:51.563 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.563 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.46:6818/847189172", 2024-09-21T14:58:51.563 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.46:6822/847189172", 2024-09-21T14:58:51.563 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.46:6828/847189172", 2024-09-21T14:58:51.563 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.46:6825/847189172", 2024-09-21T14:58:51.563 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-21T14:58:51.563 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-21T14:58:51.564 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-21T14:58:51.564 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:51.564 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.564 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:51.564 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-09-21T14:58:51.564 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "3a2f82a4-28a0-4fcb-9d6b-2285fa4d07ac", 2024-09-21T14:58:51.564 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-21T14:58:51.564 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-21T14:58:51.564 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 0.95001220703125, 2024-09-21T14:58:51.564 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-21T14:58:51.564 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-21T14:58:51.564 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-21T14:58:51.564 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 15, 2024-09-21T14:58:51.564 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 24, 2024-09-21T14:58:51.564 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-21T14:58:51.565 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-21T14:58:51.565 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-21T14:58:51.565 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:51.565 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:51.565 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:51.565 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6816", 2024-09-21T14:58:51.565 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3894963553 2024-09-21T14:58:51.565 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.565 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:51.565 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:51.565 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6819", 2024-09-21T14:58:51.565 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3894963553 2024-09-21T14:58:51.565 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:51.565 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:51.565 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.566 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-21T14:58:51.566 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:51.566 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:51.566 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:51.566 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6821", 2024-09-21T14:58:51.566 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3894963553 2024-09-21T14:58:51.566 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.566 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:51.566 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:51.566 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6824", 2024-09-21T14:58:51.566 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3894963553 2024-09-21T14:58:51.566 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:51.566 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:51.566 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.567 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-21T14:58:51.567 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:51.567 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:51.567 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:51.567 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6830", 2024-09-21T14:58:51.567 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3894963553 2024-09-21T14:58:51.567 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.567 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:51.567 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:51.567 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6831", 2024-09-21T14:58:51.567 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3894963553 2024-09-21T14:58:51.567 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:51.567 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:51.567 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.568 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-21T14:58:51.568 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:51.568 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:51.568 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:51.568 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6826", 2024-09-21T14:58:51.568 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3894963553 2024-09-21T14:58:51.568 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.568 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:51.568 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:51.568 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6829", 2024-09-21T14:58:51.568 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3894963553 2024-09-21T14:58:51.568 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:51.568 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:51.568 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.568 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.46:6819/3894963553", 2024-09-21T14:58:51.568 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.46:6824/3894963553", 2024-09-21T14:58:51.569 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.46:6831/3894963553", 2024-09-21T14:58:51.569 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.46:6829/3894963553", 2024-09-21T14:58:51.569 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-21T14:58:51.569 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-21T14:58:51.569 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-21T14:58:51.569 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:51.569 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:51.569 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-21T14:58:51.569 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_xinfo": [ 2024-09-21T14:58:51.569 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:51.569 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-09-21T14:58:51.569 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-21T14:58:51.569 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-21T14:58:51.570 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-21T14:58:51.570 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-21T14:58:51.570 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-21T14:58:51.570 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-21T14:57:57.458317+0000", 2024-09-21T14:58:51.570 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-21T14:58:51.570 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.570 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:51.570 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-09-21T14:58:51.570 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-21T14:58:51.570 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-21T14:58:51.570 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-21T14:58:51.570 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-21T14:58:51.570 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-21T14:58:51.570 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-21T14:57:57.907624+0000", 2024-09-21T14:58:51.571 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-21T14:58:51.571 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.571 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:51.571 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-09-21T14:58:51.571 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-21T14:58:51.571 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-21T14:58:51.571 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-21T14:58:51.571 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-21T14:58:51.571 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-21T14:58:51.571 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-21T14:57:57.278817+0000", 2024-09-21T14:58:51.571 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-21T14:58:51.571 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.571 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:51.571 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-09-21T14:58:51.572 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-21T14:58:51.572 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-21T14:58:51.572 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-21T14:58:51.572 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-21T14:58:51.572 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-21T14:58:51.572 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-21T14:57:57.500449+0000", 2024-09-21T14:58:51.572 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-21T14:58:51.572 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.572 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:51.572 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-09-21T14:58:51.572 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-21T14:58:51.572 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-21T14:58:51.572 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-21T14:58:51.572 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-21T14:58:51.572 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-21T14:58:51.573 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-21T14:57:57.504325+0000", 2024-09-21T14:58:51.573 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-21T14:58:51.573 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.573 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:51.573 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-09-21T14:58:51.573 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-21T14:58:51.573 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-21T14:58:51.573 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-21T14:58:51.573 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-21T14:58:51.573 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-21T14:58:51.573 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-21T14:57:57.777525+0000", 2024-09-21T14:58:51.573 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-21T14:58:51.573 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.573 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:51.574 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-09-21T14:58:51.574 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-21T14:58:51.574 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-21T14:58:51.574 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-21T14:58:51.574 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-21T14:58:51.574 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-21T14:58:51.574 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-21T14:57:57.698961+0000", 2024-09-21T14:58:51.574 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-21T14:58:51.574 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.574 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:51.574 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-09-21T14:58:51.574 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-21T14:58:51.574 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-21T14:58:51.575 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-21T14:58:51.575 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-21T14:58:51.575 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-21T14:58:51.575 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-21T14:57:57.932625+0000", 2024-09-21T14:58:51.575 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-21T14:58:51.575 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:51.575 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-21T14:58:51.575 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap": [ 2024-09-21T14:58:51.575 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:51.575 INFO:teuthology.orchestra.run.smithi032.stdout: "pgid": "3.2", 2024-09-21T14:58:51.575 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2024-09-21T14:58:51.575 INFO:teuthology.orchestra.run.smithi032.stdout: 2, 2024-09-21T14:58:51.575 INFO:teuthology.orchestra.run.smithi032.stdout: 3 2024-09-21T14:58:51.575 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:51.576 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:51.576 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-21T14:58:51.576 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_items": [], 2024-09-21T14:58:51.576 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_primaries": [], 2024-09-21T14:58:51.576 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_temp": [], 2024-09-21T14:58:51.576 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_temp": [], 2024-09-21T14:58:51.576 INFO:teuthology.orchestra.run.smithi032.stdout: "blocklist": {}, 2024-09-21T14:58:51.576 INFO:teuthology.orchestra.run.smithi032.stdout: "range_blocklist": {}, 2024-09-21T14:58:51.576 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profiles": { 2024-09-21T14:58:51.576 INFO:teuthology.orchestra.run.smithi032.stdout: "default": { 2024-09-21T14:58:51.576 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2024-09-21T14:58:51.576 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2024-09-21T14:58:51.576 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "1", 2024-09-21T14:58:51.576 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2024-09-21T14:58:51.576 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van" 2024-09-21T14:58:51.577 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:51.577 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.577 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps_queue": [ 2024-09-21T14:58:51.577 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:51.577 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 3, 2024-09-21T14:58:51.577 INFO:teuthology.orchestra.run.smithi032.stdout: "snaps": [ 2024-09-21T14:58:51.577 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:51.577 INFO:teuthology.orchestra.run.smithi032.stdout: "begin": 23, 2024-09-21T14:58:51.577 INFO:teuthology.orchestra.run.smithi032.stdout: "length": 1 2024-09-21T14:58:51.577 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:51.577 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:51.577 INFO:teuthology.orchestra.run.smithi032.stdout: "begin": 26, 2024-09-21T14:58:51.577 INFO:teuthology.orchestra.run.smithi032.stdout: "length": 1 2024-09-21T14:58:51.577 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:51.577 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:51.577 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:51.578 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-21T14:58:51.578 INFO:teuthology.orchestra.run.smithi032.stdout: "new_removed_snaps": [ 2024-09-21T14:58:51.578 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:51.578 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 3, 2024-09-21T14:58:51.578 INFO:teuthology.orchestra.run.smithi032.stdout: "snaps": [ 2024-09-21T14:58:51.578 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:51.578 INFO:teuthology.orchestra.run.smithi032.stdout: "begin": 26, 2024-09-21T14:58:51.578 INFO:teuthology.orchestra.run.smithi032.stdout: "length": 1 2024-09-21T14:58:51.578 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:51.578 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:51.578 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:51.578 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-21T14:58:51.578 INFO:teuthology.orchestra.run.smithi032.stdout: "new_purged_snaps": [], 2024-09-21T14:58:51.578 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_node_flags": {}, 2024-09-21T14:58:51.579 INFO:teuthology.orchestra.run.smithi032.stdout: "device_class_flags": {}, 2024-09-21T14:58:51.579 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode": { 2024-09-21T14:58:51.579 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_enabled": false, 2024-09-21T14:58:51.579 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_bucket_count": 0, 2024-09-21T14:58:51.579 INFO:teuthology.orchestra.run.smithi032.stdout: "degraded_stretch_mode": 0, 2024-09-21T14:58:51.579 INFO:teuthology.orchestra.run.smithi032.stdout: "recovering_stretch_mode": 0, 2024-09-21T14:58:51.579 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_bucket": 0 2024-09-21T14:58:51.579 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:51.579 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:58:51.581 INFO:tasks.thrashosds.thrasher:j is {'epoch': 56, 'fsid': 'caf51e55-9234-4a26-bcd2-d60811962560', 'created': '2024-09-21T14:57:50.338822+0000', 'modified': '2024-09-21T14:58:51.192423+0000', 'last_up_change': '2024-09-21T14:58:01.177141+0000', 'last_in_change': '2024-09-21T14:57:52.442036+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T14:58:01.842810+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.8812656402587891, 'average_primary_affinity': 0.8799999952316284, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-21T14:58:08.284925+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 22, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 0.8812656402587891, 'average_primary_affinity': 0.8799999952316284, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-21T14:58:17.510709+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '56', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 27, 'snap_epoch': 56, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.8812656402587891, 'average_primary_affinity': 0.8799999952316284, 'average_primary_affinity_weighted': 0.8799999952316284}}], 'osds': [{'osd': 0, 'uuid': 'a4a73316-3a94-42e9-9a51-deb46a66bb06', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6808', 'nonce': 4016672243}, {'type': 'v1', 'addr': '172.21.15.32:6809', 'nonce': 4016672243}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6810', 'nonce': 4016672243}, {'type': 'v1', 'addr': '172.21.15.32:6811', 'nonce': 4016672243}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6814', 'nonce': 4016672243}, {'type': 'v1', 'addr': '172.21.15.32:6815', 'nonce': 4016672243}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6812', 'nonce': 4016672243}, {'type': 'v1', 'addr': '172.21.15.32:6813', 'nonce': 4016672243}]}, 'public_addr': '172.21.15.32:6809/4016672243', 'cluster_addr': '172.21.15.32:6811/4016672243', 'heartbeat_back_addr': '172.21.15.32:6815/4016672243', 'heartbeat_front_addr': '172.21.15.32:6813/4016672243', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '006757a0-94d1-4a9b-aef8-73634cc5828e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6824', 'nonce': 641090577}, {'type': 'v1', 'addr': '172.21.15.32:6825', 'nonce': 641090577}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6826', 'nonce': 641090577}, {'type': 'v1', 'addr': '172.21.15.32:6827', 'nonce': 641090577}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6830', 'nonce': 641090577}, {'type': 'v1', 'addr': '172.21.15.32:6831', 'nonce': 641090577}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6828', 'nonce': 641090577}, {'type': 'v1', 'addr': '172.21.15.32:6829', 'nonce': 641090577}]}, 'public_addr': '172.21.15.32:6825/641090577', 'cluster_addr': '172.21.15.32:6827/641090577', 'heartbeat_back_addr': '172.21.15.32:6831/641090577', 'heartbeat_front_addr': '172.21.15.32:6829/641090577', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '0b499a23-820e-4e6c-a558-adbf88b7014f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 51, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6800', 'nonce': 2609154414}, {'type': 'v1', 'addr': '172.21.15.32:6801', 'nonce': 2609154414}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6802', 'nonce': 2609154414}, {'type': 'v1', 'addr': '172.21.15.32:6803', 'nonce': 2609154414}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6806', 'nonce': 2609154414}, {'type': 'v1', 'addr': '172.21.15.32:6807', 'nonce': 2609154414}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6804', 'nonce': 2609154414}, {'type': 'v1', 'addr': '172.21.15.32:6805', 'nonce': 2609154414}]}, 'public_addr': '172.21.15.32:6801/2609154414', 'cluster_addr': '172.21.15.32:6803/2609154414', 'heartbeat_back_addr': '172.21.15.32:6807/2609154414', 'heartbeat_front_addr': '172.21.15.32:6805/2609154414', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '5a57b3dd-4a07-45b7-b159-3cac4c7eb262', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 51, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6816', 'nonce': 1462563376}, {'type': 'v1', 'addr': '172.21.15.32:6817', 'nonce': 1462563376}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6818', 'nonce': 1462563376}, {'type': 'v1', 'addr': '172.21.15.32:6819', 'nonce': 1462563376}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6822', 'nonce': 1462563376}, {'type': 'v1', 'addr': '172.21.15.32:6823', 'nonce': 1462563376}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6820', 'nonce': 1462563376}, {'type': 'v1', 'addr': '172.21.15.32:6821', 'nonce': 1462563376}]}, 'public_addr': '172.21.15.32:6817/1462563376', 'cluster_addr': '172.21.15.32:6819/1462563376', 'heartbeat_back_addr': '172.21.15.32:6823/1462563376', 'heartbeat_front_addr': '172.21.15.32:6821/1462563376', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '39a00807-7517-474e-8842-623dd7093cc5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6801', 'nonce': 144412042}, {'type': 'v1', 'addr': '172.21.15.46:6803', 'nonce': 144412042}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6805', 'nonce': 144412042}, {'type': 'v1', 'addr': '172.21.15.46:6807', 'nonce': 144412042}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6812', 'nonce': 144412042}, {'type': 'v1', 'addr': '172.21.15.46:6814', 'nonce': 144412042}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6809', 'nonce': 144412042}, {'type': 'v1', 'addr': '172.21.15.46:6811', 'nonce': 144412042}]}, 'public_addr': '172.21.15.46:6803/144412042', 'cluster_addr': '172.21.15.46:6807/144412042', 'heartbeat_back_addr': '172.21.15.46:6814/144412042', 'heartbeat_front_addr': '172.21.15.46:6811/144412042', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'edf9f045-ddeb-4504-a0fc-1a0e4e853550', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 51, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6800', 'nonce': 849101442}, {'type': 'v1', 'addr': '172.21.15.46:6802', 'nonce': 849101442}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6804', 'nonce': 849101442}, {'type': 'v1', 'addr': '172.21.15.46:6806', 'nonce': 849101442}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6813', 'nonce': 849101442}, {'type': 'v1', 'addr': '172.21.15.46:6815', 'nonce': 849101442}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6808', 'nonce': 849101442}, {'type': 'v1', 'addr': '172.21.15.46:6810', 'nonce': 849101442}]}, 'public_addr': '172.21.15.46:6802/849101442', 'cluster_addr': '172.21.15.46:6806/849101442', 'heartbeat_back_addr': '172.21.15.46:6815/849101442', 'heartbeat_front_addr': '172.21.15.46:6810/849101442', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'c820ea6b-88fe-4421-83ec-17c466691fea', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.0501251220703125, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6817', 'nonce': 847189172}, {'type': 'v1', 'addr': '172.21.15.46:6818', 'nonce': 847189172}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6820', 'nonce': 847189172}, {'type': 'v1', 'addr': '172.21.15.46:6822', 'nonce': 847189172}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6827', 'nonce': 847189172}, {'type': 'v1', 'addr': '172.21.15.46:6828', 'nonce': 847189172}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6823', 'nonce': 847189172}, {'type': 'v1', 'addr': '172.21.15.46:6825', 'nonce': 847189172}]}, 'public_addr': '172.21.15.46:6818/847189172', 'cluster_addr': '172.21.15.46:6822/847189172', 'heartbeat_back_addr': '172.21.15.46:6828/847189172', 'heartbeat_front_addr': '172.21.15.46:6825/847189172', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '3a2f82a4-28a0-4fcb-9d6b-2285fa4d07ac', 'up': 1, 'in': 1, 'weight': 0.95001220703125, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6816', 'nonce': 3894963553}, {'type': 'v1', 'addr': '172.21.15.46:6819', 'nonce': 3894963553}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6821', 'nonce': 3894963553}, {'type': 'v1', 'addr': '172.21.15.46:6824', 'nonce': 3894963553}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6830', 'nonce': 3894963553}, {'type': 'v1', 'addr': '172.21.15.46:6831', 'nonce': 3894963553}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6826', 'nonce': 3894963553}, {'type': 'v1', 'addr': '172.21.15.46:6829', 'nonce': 3894963553}]}, 'public_addr': '172.21.15.46:6819/3894963553', 'cluster_addr': '172.21.15.46:6824/3894963553', 'heartbeat_back_addr': '172.21.15.46:6831/3894963553', 'heartbeat_front_addr': '172.21.15.46:6829/3894963553', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:57:57.458317+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:57:57.907624+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:57:57.278817+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:57:57.500449+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:57:57.504325+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:57:57.777525+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:57:57.698961+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:57:57.932625+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.2', 'osds': [2, 3]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 23, 'length': 1}, {'begin': 26, 'length': 1}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 26, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-21T14:58:51.581 INFO:tasks.thrashosds.thrasher:No pg_upmap entries; doing nothing 2024-09-21T14:58:51.581 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:58:51.577+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:58:51.598 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:51.598 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:51.601 INFO:teuthology.orchestra.run.smithi032.stdout:722 punch from 0x73ade2a to 0x73ba189, (0xc35f bytes) 2024-09-21T14:58:51.608 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T14:58:51.621 INFO:teuthology.orchestra.run.smithi032.stdout:723 write 0xbda9c7c thru 0xbdae9e2 (0x4d67 bytes) 2024-09-21T14:58:51.628 INFO:teuthology.orchestra.run.smithi032.stdout:724 writesame 0x510a615 thru 0x5111aca data_size 0x1(0x74b6 bytes) 2024-09-21T14:58:51.647 INFO:teuthology.orchestra.run.smithi032.stdout:725 writesame 0xbd80157 thru 0xbd8bd22 data_size 0x44(0xbbcc bytes) 2024-09-21T14:58:51.654 INFO:teuthology.orchestra.run.smithi032.stdout:726 write 0x5cb6f58 thru 0x5cb8521 (0x15ca bytes) 2024-09-21T14:58:51.660 INFO:teuthology.orchestra.run.smithi032.stdout:727 read 0x32b59e6 thru 0x32bb144 (0x575f bytes) 2024-09-21T14:58:51.661 INFO:teuthology.orchestra.run.smithi032.stdout:729 read 0x5808cf8 thru 0x5817c62 (0xef6b bytes) 2024-09-21T14:58:51.661 INFO:teuthology.orchestra.run.smithi032.stdout:730 writesame 0x8f41b2b thru 0x8f50047 data_size 0x31(0xe51d bytes) 2024-09-21T14:58:51.665 INFO:teuthology.orchestra.run.smithi032.stdout:731 read 0x6b9a3b3 thru 0x6ba097c (0x65ca bytes) 2024-09-21T14:58:51.665 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:51.665 INFO:teuthology.orchestra.run.smithi032.stdout:733 read 0x974d091 thru 0x97529cd (0x593d bytes) 2024-09-21T14:58:51.665 INFO:teuthology.orchestra.run.smithi032.stdout:734 write 0x36133c3 thru 0x362264c (0xf28a bytes) 2024-09-21T14:58:51.676 INFO:teuthology.orchestra.run.smithi032.stdout:736 trunc from 0xbdae9e3 to 0xd184273 2024-09-21T14:58:51.682 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:58:51.680+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:58:51.684 INFO:teuthology.orchestra.run.smithi032.stdout:740 writesame 0xcb73a74 thru 0xcb81ea9 data_size 0x6(0xe436 bytes) 2024-09-21T14:58:51.696 INFO:teuthology.orchestra.run.smithi032.stdout:742 read 0x456167f thru 0x4562eff (0x1881 bytes) 2024-09-21T14:58:51.697 INFO:teuthology.orchestra.run.smithi032.stdout:744 read 0x7c99af9 thru 0x7ca7f51 (0xe459 bytes) 2024-09-21T14:58:51.697 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:51.698 INFO:teuthology.orchestra.run.smithi032.stdout:748 write 0x5492446 thru 0x5492c4e (0x809 bytes) 2024-09-21T14:58:51.709 INFO:teuthology.orchestra.run.smithi032.stdout:750 write 0xe8889ea thru 0xe88f416 (0x6a2d bytes) 2024-09-21T14:58:51.724 INFO:teuthology.orchestra.run.smithi032.stdout:751 trunc from 0xe88f417 to 0x960a9bf 2024-09-21T14:58:51.737 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:51.738 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:51.743 INFO:teuthology.orchestra.run.smithi032.stdout:752 write 0x54ea24c thru 0x54f562a (0xb3df bytes) 2024-09-21T14:58:51.748 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:51.749 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T14:58:51.759 INFO:teuthology.orchestra.run.smithi032.stdout:754 write 0xa541f38 thru 0xa54e9ef (0xcab8 bytes) 2024-09-21T14:58:51.781 INFO:teuthology.orchestra.run.smithi032.stdout:755 write 0xd01b244 thru 0xd0202ea (0x50a7 bytes) 2024-09-21T14:58:51.782 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:58:51.781+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:58:51.803 INFO:teuthology.orchestra.run.smithi032.stdout:757 write 0xe2c3d8b thru 0xe2c4446 (0x6bc bytes) 2024-09-21T14:58:51.809 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:51.809 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:51.809 INFO:teuthology.orchestra.run.smithi032.stdout:762 read 0x8b037aa thru 0x8b0eca0 (0xb4f7 bytes) 2024-09-21T14:58:51.810 INFO:teuthology.orchestra.run.smithi032.stdout:763 read 0x8d583d5 thru 0x8d6498e (0xc5ba bytes) 2024-09-21T14:58:51.811 INFO:teuthology.orchestra.run.smithi032.stdout:764 read 0x6eb8274 thru 0x6ec21dd (0x9f6a bytes) 2024-09-21T14:58:51.811 INFO:teuthology.orchestra.run.smithi032.stdout:765 read 0x2fca40 thru 0x305f08 (0x94c9 bytes) 2024-09-21T14:58:51.812 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:51.821 INFO:teuthology.orchestra.run.smithi032.stdout:768 write 0xeb8a4ac thru 0xeb8c103 (0x1c58 bytes) 2024-09-21T14:58:51.842 INFO:teuthology.orchestra.run.smithi032.stdout:769 trunc from 0xeb8c104 to 0x4d69e06 2024-09-21T14:58:51.881 INFO:teuthology.orchestra.run.smithi032.stdout:770 write 0xd83f204 thru 0xd847415 (0x8212 bytes) 2024-09-21T14:58:51.882 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:58:51.880+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:58:51.884 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:51.884 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:51.889 INFO:teuthology.orchestra.run.smithi032.stdout:771 punch from 0xbc0fb71 to 0xbc1f548, (0xf9d7 bytes) 2024-09-21T14:58:51.890 INFO:teuthology.orchestra.run.smithi032.stdout:772 read 0xd498eb7 thru 0xd4a15fd (0x8747 bytes) 2024-09-21T14:58:51.890 INFO:teuthology.orchestra.run.smithi032.stdout:774 write 0xccdf3ee thru 0xcce3dc1 (0x49d4 bytes) 2024-09-21T14:58:51.896 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T14:58:51.917 INFO:teuthology.orchestra.run.smithi032.stdout:775 trunc from 0xd847416 to 0x65720e 2024-09-21T14:58:51.942 INFO:teuthology.orchestra.run.smithi032.stdout:776 read 0x214fdc thru 0x21721f (0x2244 bytes) 2024-09-21T14:58:51.950 INFO:teuthology.orchestra.run.smithi032.stdout:777 write 0xc8eeb3 thru 0xc913ff (0x254d bytes) 2024-09-21T14:58:51.968 INFO:teuthology.orchestra.run.smithi032.stdout:779 writesame 0x3d50d79 thru 0x3d5aa7a data_size 0x3(0x9d02 bytes) 2024-09-21T14:58:51.982 INFO:teuthology.orchestra.run.smithi032.stdout:780 writesame 0x6fc1806 thru 0x6fc2b53 data_size 0x7(0x134e bytes) 2024-09-21T14:58:51.983 INFO:teuthology.orchestra.run.smithi032.stdout:781 read 0xeb1e57 thru 0xeb54ab (0x3655 bytes) 2024-09-21T14:58:51.983 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:58:51.981+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:58:51.994 INFO:teuthology.orchestra.run.smithi032.stdout:782 write 0x7c504cb thru 0x7c5e62d (0xe163 bytes) 2024-09-21T14:58:52.004 INFO:teuthology.orchestra.run.smithi032.stdout:785 read 0xe06006 thru 0xe0f088 (0x9083 bytes) 2024-09-21T14:58:52.005 INFO:teuthology.orchestra.run.smithi032.stdout:786 read 0xea551d thru 0xea6486 (0xf6a bytes) 2024-09-21T14:58:52.006 INFO:teuthology.orchestra.run.smithi032.stdout:790 trunc from 0x7c5e62e to 0x8c40892 2024-09-21T14:58:52.015 INFO:teuthology.orchestra.run.smithi032.stdout:791 write 0x2d95719 thru 0x2da4aa0 (0xf388 bytes) 2024-09-21T14:58:52.026 INFO:teuthology.orchestra.run.smithi032.stdout:792 punch from 0x6d3beca to 0x6d46be8, (0xad1e bytes) 2024-09-21T14:58:52.036 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:52.036 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:52.038 INFO:teuthology.orchestra.run.smithi032.stdout:793 write 0xae5868c thru 0xae5c4e6 (0x3e5b bytes) 2024-09-21T14:58:52.044 INFO:teuthology.orchestra.run.smithi032.stdout:794 read 0x7a28177 thru 0x7a2cbcf (0x4a59 bytes) 2024-09-21T14:58:52.045 INFO:teuthology.orchestra.run.smithi032.stdout:795 read 0x2b2cc87 thru 0x2b2df16 (0x1290 bytes) 2024-09-21T14:58:52.048 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T14:58:52.053 INFO:teuthology.orchestra.run.smithi032.stdout:796 trunc from 0xae5c4e7 to 0x6f26c22 2024-09-21T14:58:52.070 INFO:teuthology.orchestra.run.smithi032.stdout:797 punch from 0x5037e68 to 0x5042c10, (0xada8 bytes) 2024-09-21T14:58:52.070 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:52.070 INFO:teuthology.orchestra.run.smithi032.stdout:800 punch from 0x2db4efd to 0x2dba0c8, (0x51cb bytes) 2024-09-21T14:58:52.070 INFO:teuthology.orchestra.run.smithi032.stdout:801 writesame 0x69b717e thru 0x69b7bcf data_size 0x1(0xa52 bytes) 2024-09-21T14:58:52.076 INFO:teuthology.orchestra.run.smithi032.stdout:802 write 0x20b349 thru 0x2142ea (0x8fa2 bytes) 2024-09-21T14:58:52.080 INFO:teuthology.orchestra.run.smithi032.stdout:803 write 0x10fa850 thru 0x11006c2 (0x5e73 bytes) 2024-09-21T14:58:52.084 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:58:52.081+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:58:52.110 INFO:teuthology.orchestra.run.smithi032.stdout:806 write 0xe2a74f2 thru 0xe2ace24 (0x5933 bytes) 2024-09-21T14:58:52.117 INFO:teuthology.orchestra.run.smithi032.stdout:808 read 0xc9690c8 thru 0xc96b24a (0x2183 bytes) 2024-09-21T14:58:52.117 INFO:teuthology.orchestra.run.smithi032.stdout:810 writesame 0x1ab1bc2 thru 0x1ab8185 data_size 0x4e(0x65c4 bytes) 2024-09-21T14:58:52.124 INFO:teuthology.orchestra.run.smithi032.stdout:811 read 0x84f40a2 thru 0x84f4c61 (0xbc0 bytes) 2024-09-21T14:58:52.124 INFO:teuthology.orchestra.run.smithi032.stdout:813 read 0x5b7e9b0 thru 0x5b8a4bb (0xbb0c bytes) 2024-09-21T14:58:52.124 INFO:teuthology.orchestra.run.smithi032.stdout:816 read 0xa132efb thru 0xa132f73 (0x79 bytes) 2024-09-21T14:58:52.124 INFO:teuthology.orchestra.run.smithi032.stdout:817 punch from 0x1aa5768 to 0x1aa7b0f, (0x23a7 bytes) 2024-09-21T14:58:52.124 INFO:teuthology.orchestra.run.smithi032.stdout:818 writesame 0xc9c2f51 thru 0xc9ca033 data_size 0x75(0x70e3 bytes) 2024-09-21T14:58:52.130 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T14:58:52.131 INFO:teuthology.orchestra.run.smithi032.stdout:819 read 0x630ed9e thru 0x631d752 (0xe9b5 bytes) 2024-09-21T14:58:52.141 INFO:teuthology.orchestra.run.smithi032.stdout:820 writesame 0xee66a6a thru 0xee6a888 data_size 0x9(0x3e1f bytes) 2024-09-21T14:58:52.141 INFO:teuthology.orchestra.run.smithi032.stdout:821 read 0x16620c9 thru 0x166286b (0x7a3 bytes) 2024-09-21T14:58:52.141 INFO:teuthology.orchestra.run.smithi032.stdout:823 read 0x3ae7cf0 thru 0x3af6b5a (0xee6b bytes) 2024-09-21T14:58:52.142 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:52.142 INFO:teuthology.orchestra.run.smithi032.stdout:825 write 0x2b3c902 thru 0x2b3ebba (0x22b9 bytes) 2024-09-21T14:58:52.144 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:52.144 INFO:teuthology.orchestra.run.smithi032.stdout:827 writesame 0xdc62ed5 thru 0xdc6528c data_size 0x24(0x23b8 bytes) 2024-09-21T14:58:52.151 INFO:teuthology.orchestra.run.smithi032.stdout:828 write 0x18b21ca thru 0x18c143f (0xf276 bytes) 2024-09-21T14:58:52.156 INFO:teuthology.orchestra.run.smithi032.stdout:829 read 0x7ab5e80 thru 0x7abe11b (0x829c bytes) 2024-09-21T14:58:52.156 INFO:teuthology.orchestra.run.smithi032.stdout:830 write 0xe7ab304 thru 0xe7b18b1 (0x65ae bytes) 2024-09-21T14:58:52.162 INFO:teuthology.orchestra.run.smithi032.stdout:831 write 0xc372281 thru 0xc37cd3b (0xaabb bytes) 2024-09-21T14:58:52.170 INFO:teuthology.orchestra.run.smithi032.stdout:833 punch from 0x586da19 to 0x587af05, (0xd4ec bytes) 2024-09-21T14:58:52.170 INFO:teuthology.orchestra.run.smithi032.stdout:834 write 0xac5b120 thru 0xac67b1c (0xc9fd bytes) 2024-09-21T14:58:52.177 INFO:teuthology.orchestra.run.smithi032.stdout:835 read 0x1fdb89f thru 0x1fdc7ee (0xf50 bytes) 2024-09-21T14:58:52.178 INFO:teuthology.orchestra.run.smithi032.stdout:838 write 0x3a1eecd thru 0x3a270ec (0x8220 bytes) 2024-09-21T14:58:52.179 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:52.179 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:52.185 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:58:52.181+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:58:52.185 INFO:teuthology.orchestra.run.smithi032.stdout:839 write 0xbeaba6e thru 0xbead9f3 (0x1f86 bytes) 2024-09-21T14:58:52.191 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T14:58:52.192 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:52.192 INFO:teuthology.orchestra.run.smithi032.stdout:842 write 0x64cd201 thru 0x64d65b4 (0x93b4 bytes) 2024-09-21T14:58:52.218 INFO:teuthology.orchestra.run.smithi032.stdout:843 trunc from 0xee6a889 to 0x4245f15 2024-09-21T14:58:52.241 INFO:teuthology.orchestra.run.smithi032.stdout:844 trunc from 0x4245f15 to 0x20cdcee 2024-09-21T14:58:52.255 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:52.255 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:52.255 INFO:teuthology.orchestra.run.smithi032.stdout:847 read 0x18168c5 thru 0x181c1e6 (0x5922 bytes) 2024-09-21T14:58:52.256 INFO:teuthology.orchestra.run.smithi032.stdout:849 read 0x147d65e thru 0x147d778 (0x11b bytes) 2024-09-21T14:58:52.267 INFO:teuthology.orchestra.run.smithi032.stdout:850 write 0x3adf81c thru 0x3ae3a0c (0x41f1 bytes) 2024-09-21T14:58:52.286 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:58:52.285+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:58:52.287 INFO:teuthology.orchestra.run.smithi032.stdout:851 trunc from 0x3ae3a0d to 0xc912307 2024-09-21T14:58:52.303 INFO:teuthology.orchestra.run.smithi032.stdout:852 write 0xcc0ed3d thru 0xcc0edb5 (0x79 bytes) 2024-09-21T14:58:52.308 INFO:teuthology.orchestra.run.smithi032.stdout:853 write 0x3f67c3c thru 0x3f6d2ea (0x56af bytes) 2024-09-21T14:58:52.309 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:52.309 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:52.314 INFO:teuthology.orchestra.run.smithi032.stdout:854 write 0xbe8e403 thru 0xbe9c678 (0xe276 bytes) 2024-09-21T14:58:52.322 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T14:58:52.334 INFO:teuthology.orchestra.run.smithi032.stdout:855 trunc from 0xcc0edb6 to 0x695d5c6 2024-09-21T14:58:52.369 INFO:teuthology.orchestra.run.smithi032.stdout:856 writesame 0xd08aaea thru 0xd093c75 data_size 0xa2(0x918c bytes) 2024-09-21T14:58:52.369 INFO:teuthology.orchestra.run.smithi032.stdout:857 read 0x4d09479 thru 0x4d10573 (0x70fb bytes) 2024-09-21T14:58:52.380 INFO:teuthology.orchestra.run.smithi032.stdout:858 write 0xe919da2 thru 0xe9287dd (0xea3c bytes) 2024-09-21T14:58:52.387 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:58:52.384+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:58:52.391 INFO:teuthology.orchestra.run.smithi032.stdout:859 read 0xb8a31a4 thru 0xb8a4f7e (0x1ddb bytes) 2024-09-21T14:58:52.391 INFO:teuthology.orchestra.run.smithi032.stdout:861 read 0x664d85f thru 0x665d0f0 (0xf892 bytes) 2024-09-21T14:58:52.391 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:52.391 INFO:teuthology.orchestra.run.smithi032.stdout:864 write 0xc297515 thru 0xc29ac9a (0x3786 bytes) 2024-09-21T14:58:52.398 INFO:teuthology.orchestra.run.smithi032.stdout:865 writesame 0x58d75e0 thru 0x58df8d5 data_size 0x1(0x82f6 bytes) 2024-09-21T14:58:52.414 INFO:teuthology.orchestra.run.smithi032.stdout:868 write 0x3c60048 thru 0x3c62e58 (0x2e11 bytes) 2024-09-21T14:58:52.416 INFO:teuthology.orchestra.run.smithi032.stdout:870 write 0x8032277 thru 0x803be02 (0x9b8c bytes) 2024-09-21T14:58:52.424 INFO:teuthology.orchestra.run.smithi032.stdout:871 read 0x5ad7b59 thru 0x5adbbb9 (0x4061 bytes) 2024-09-21T14:58:52.425 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:52.437 INFO:teuthology.orchestra.run.smithi032.stdout:873 trunc from 0xe9287de to 0x7224ea0 2024-09-21T14:58:52.453 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:52.453 INFO:teuthology.orchestra.run.smithi032.stdout:876 punch from 0x2ce64bf to 0x2ced8b6, (0x73f7 bytes) 2024-09-21T14:58:52.453 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:52.454 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:52.455 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:52.459 INFO:teuthology.orchestra.run.smithi032.stdout:878 trunc from 0x7224ea0 to 0x30bffa0 2024-09-21T14:58:52.467 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T14:58:52.474 INFO:teuthology.orchestra.run.smithi032.stdout:879 trunc from 0x30bffa0 to 0x342cf8c 2024-09-21T14:58:52.481 INFO:teuthology.orchestra.run.smithi032.stdout:880 read 0x2279c58 thru 0x227ef0a (0x52b3 bytes) 2024-09-21T14:58:52.486 INFO:teuthology.orchestra.run.smithi032.stdout:881 trunc from 0x342cf8c to 0x217ea6 2024-09-21T14:58:52.487 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:58:52.485+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:58:52.533 INFO:teuthology.orchestra.run.smithi032.stdout:882 write 0xe4a7e46 thru 0xe4abaec (0x3ca7 bytes) 2024-09-21T14:58:52.540 INFO:teuthology.orchestra.run.smithi032.stdout:884 writesame 0x5181a00 thru 0x518b046 data_size 0x1f(0x9647 bytes) 2024-09-21T14:58:52.547 INFO:teuthology.orchestra.run.smithi032.stdout:886 write 0xb2a56d9 thru 0xb2a56fa (0x22 bytes) 2024-09-21T14:58:52.551 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:52.551 INFO:teuthology.orchestra.run.smithi032.stdout:888 read 0x179fdbb thru 0x17a414f (0x4395 bytes) 2024-09-21T14:58:52.551 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:52.551 INFO:teuthology.orchestra.run.smithi032.stdout:890 punch from 0xab3adf0 to 0xab3b850, (0xa60 bytes) 2024-09-21T14:58:52.551 INFO:teuthology.orchestra.run.smithi032.stdout:891 write 0x5c028f5 thru 0x5c0e806 (0xbf12 bytes) 2024-09-21T14:58:52.559 INFO:teuthology.orchestra.run.smithi032.stdout:894 write 0xc1bee33 thru 0xc1cc53a (0xd708 bytes) 2024-09-21T14:58:52.569 INFO:teuthology.orchestra.run.smithi032.stdout:895 read 0xd9db037 thru 0xd9e3253 (0x821d bytes) 2024-09-21T14:58:52.569 INFO:teuthology.orchestra.run.smithi032.stdout:896 punch from 0x3d9bced to 0x3da19c4, (0x5cd7 bytes) 2024-09-21T14:58:52.569 INFO:teuthology.orchestra.run.smithi032.stdout:899 write 0x15fed52 thru 0x1608a7d (0x9d2c bytes) 2024-09-21T14:58:52.577 INFO:teuthology.orchestra.run.smithi032.stdout:901 read 0x553e314 thru 0x554b0c9 (0xcdb6 bytes) 2024-09-21T14:58:52.587 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:58:52.585+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:58:52.589 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:52.589 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:52.594 INFO:teuthology.orchestra.run.smithi032.stdout:902 trunc from 0xe4abaed to 0x45cd5ca 2024-09-21T14:58:52.603 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T14:58:52.610 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:52.610 INFO:teuthology.orchestra.run.smithi032.stdout:906 read 0x227d2ca thru 0x227e328 (0x105f bytes) 2024-09-21T14:58:52.610 INFO:teuthology.orchestra.run.smithi032.stdout:907 write 0x1aaa147 thru 0x1ab47f8 (0xa6b2 bytes) 2024-09-21T14:58:52.628 INFO:teuthology.orchestra.run.smithi032.stdout:908 writesame 0x59ce8a6 thru 0x59d9dfb data_size 0x9(0xb556 bytes) 2024-09-21T14:58:52.629 INFO:teuthology.orchestra.run.smithi032.stdout:909 write 0x421b3fe thru 0x4227f94 (0xcb97 bytes) 2024-09-21T14:58:52.637 INFO:teuthology.orchestra.run.smithi032.stdout:910 write 0x3bcd350 thru 0x3bd733e (0x9fef bytes) 2024-09-21T14:58:52.646 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:52.659 INFO:teuthology.orchestra.run.smithi032.stdout:912 writesame 0x85099ad thru 0x850c0f2 data_size 0xb(0x2746 bytes) 2024-09-21T14:58:52.677 INFO:teuthology.orchestra.run.smithi032.stdout:914 write 0xd381f5b thru 0xd382f1a (0xfc0 bytes) 2024-09-21T14:58:52.682 INFO:teuthology.orchestra.run.smithi032.stdout:915 punch from 0x7a10799 to 0x7a1792f, (0x7196 bytes) 2024-09-21T14:58:52.682 INFO:teuthology.orchestra.run.smithi032.stdout:918 punch from 0x8296e55 to 0x829af75, (0x4120 bytes) 2024-09-21T14:58:52.683 INFO:teuthology.orchestra.run.smithi032.stdout:919 read 0xcfe9c48 thru 0xcfeb42b (0x17e4 bytes) 2024-09-21T14:58:52.683 INFO:teuthology.orchestra.run.smithi032.stdout:920 write 0x32e9685 thru 0x32f8212 (0xeb8e bytes) 2024-09-21T14:58:52.688 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:58:52.684+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:58:52.694 INFO:teuthology.orchestra.run.smithi032.stdout:921 writesame 0xa03f7fc thru 0xa04b09b data_size 0x2(0xb8a0 bytes) 2024-09-21T14:58:52.711 INFO:teuthology.orchestra.run.smithi032.stdout:922 write 0x117974b thru 0x1187243 (0xdaf9 bytes) 2024-09-21T14:58:52.715 INFO:teuthology.orchestra.run.smithi032.stdout:923 read 0xd1d77ce thru 0xd1e0cdc (0x950f bytes) 2024-09-21T14:58:52.716 INFO:teuthology.orchestra.run.smithi032.stdout:924 writesame 0x791eb40 thru 0x7925ae6 data_size 0x1(0x6fa7 bytes) 2024-09-21T14:58:52.729 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:52.729 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:52.732 INFO:teuthology.orchestra.run.smithi032.stdout:926 write 0x595b6bb thru 0x595df12 (0x2858 bytes) 2024-09-21T14:58:52.738 INFO:teuthology.orchestra.run.smithi032.stdout:927 write 0x6f1ae41 thru 0x6f24aee (0x9cae bytes) 2024-09-21T14:58:52.741 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T14:58:52.747 INFO:teuthology.orchestra.run.smithi032.stdout:929 punch from 0x9b99431 to 0x9b9aa54, (0x1623 bytes) 2024-09-21T14:58:52.747 INFO:teuthology.orchestra.run.smithi032.stdout:930 write 0x7bdfcde thru 0x7be265a (0x297d bytes) 2024-09-21T14:58:52.749 INFO:teuthology.orchestra.run.smithi032.stdout:931 write 0x5a95411 thru 0x5a972af (0x1e9f bytes) 2024-09-21T14:58:52.754 INFO:teuthology.orchestra.run.smithi032.stdout:933 write 0xba469df thru 0xba56652 (0xfc74 bytes) 2024-09-21T14:58:52.762 INFO:teuthology.orchestra.run.smithi032.stdout:934 writesame 0x6dccd38 thru 0x6dd0cf3 data_size 0x2(0x3fbc bytes) 2024-09-21T14:58:52.767 INFO:teuthology.orchestra.run.smithi032.stdout:937 punch from 0x74b88ce to 0x74be504, (0x5c36 bytes) 2024-09-21T14:58:52.776 INFO:teuthology.orchestra.run.smithi032.stdout:939 writesame 0xe2cceed thru 0xe2d5e2d data_size 0x5b(0x8f41 bytes) 2024-09-21T14:58:52.776 INFO:teuthology.orchestra.run.smithi032.stdout:940 read 0x40522b4 thru 0x4055ff8 (0x3d45 bytes) 2024-09-21T14:58:52.776 INFO:teuthology.orchestra.run.smithi032.stdout:941 read 0xd5739ad thru 0xd57a6fa (0x6d4e bytes) 2024-09-21T14:58:52.777 INFO:teuthology.orchestra.run.smithi032.stdout:943 punch from 0xe2d0fe6 to 0xe2d5e2e, (0x4e48 bytes) 2024-09-21T14:58:52.777 INFO:teuthology.orchestra.run.smithi032.stdout:944 write 0x1a6bbb6 thru 0x1a74b05 (0x8f50 bytes) 2024-09-21T14:58:52.782 INFO:teuthology.orchestra.run.smithi032.stdout:951 read 0x3cc8731 thru 0x3cd8501 (0xfdd1 bytes) 2024-09-21T14:58:52.782 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:52.789 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:58:52.785+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:58:52.794 INFO:teuthology.orchestra.run.smithi032.stdout:957 trunc from 0xe2d5e2e to 0x75e87b7 2024-09-21T14:58:52.813 INFO:teuthology.orchestra.run.smithi032.stdout:959 write 0x25cb68b thru 0x25ce4d9 (0x2e4f bytes) 2024-09-21T14:58:52.819 INFO:teuthology.orchestra.run.smithi032.stdout:960 punch from 0x6ae004b to 0x6ae63ef, (0x63a4 bytes) 2024-09-21T14:58:52.819 INFO:teuthology.orchestra.run.smithi032.stdout:961 punch from 0x407f733 to 0x408d381, (0xdc4e bytes) 2024-09-21T14:58:52.836 INFO:teuthology.orchestra.run.smithi032.stdout:964 writesame 0xdeaa5dc thru 0xdeb1bba data_size 0x47(0x75df bytes) 2024-09-21T14:58:52.836 INFO:teuthology.orchestra.run.smithi032.stdout:965 read 0xa0da1cd thru 0xa0ddc94 (0x3ac8 bytes) 2024-09-21T14:58:52.836 INFO:teuthology.orchestra.run.smithi032.stdout:966 read 0xa459f85 thru 0xa467f6a (0xdfe6 bytes) 2024-09-21T14:58:52.856 INFO:teuthology.orchestra.run.smithi032.stdout:967 trunc from 0xdeb1bbb to 0x17a2492 2024-09-21T14:58:52.864 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:52.864 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:52.877 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T14:58:52.878 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:52.896 INFO:teuthology.orchestra.run.smithi032.stdout:969 writesame 0x8a7d2b8 thru 0x8a85291 data_size 0xf(0x7fda bytes) 2024-09-21T14:58:52.896 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:52.897 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:58:52.893+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:58:52.903 INFO:teuthology.orchestra.run.smithi032.stdout:971 trunc from 0x8a85292 to 0x486c69e 2024-09-21T14:58:52.911 INFO:teuthology.orchestra.run.smithi032.stdout:972 punch from 0x2e451df to 0x2e4560f, (0x430 bytes) 2024-09-21T14:58:52.929 INFO:teuthology.orchestra.run.smithi032.stdout:974 writesame 0xad25801 thru 0xad3151a data_size 0x67(0xbd1a bytes) 2024-09-21T14:58:52.945 INFO:teuthology.orchestra.run.smithi032.stdout:975 write 0xe799ae2 thru 0xe7a6eb4 (0xd3d3 bytes) 2024-09-21T14:58:52.955 INFO:teuthology.orchestra.run.smithi032.stdout:976 read 0xc666d29 thru 0xc66eaab (0x7d83 bytes) 2024-09-21T14:58:52.955 INFO:teuthology.orchestra.run.smithi032.stdout:977 write 0xd92d7cc thru 0xd92d860 (0x95 bytes) 2024-09-21T14:58:52.961 INFO:teuthology.orchestra.run.smithi032.stdout:978 read 0x14dd2c7 thru 0x14e9161 (0xbe9b bytes) 2024-09-21T14:58:52.962 INFO:teuthology.orchestra.run.smithi032.stdout:979 write 0x6771b36 thru 0x677ca5d (0xaf28 bytes) 2024-09-21T14:58:52.972 INFO:teuthology.orchestra.run.smithi032.stdout:980 write 0x9ba5c19 thru 0x9baab8d (0x4f75 bytes) 2024-09-21T14:58:52.980 INFO:teuthology.orchestra.run.smithi032.stdout:981 punch from 0xbc482f6 to 0xbc48f74, (0xc7e bytes) 2024-09-21T14:58:52.980 INFO:teuthology.orchestra.run.smithi032.stdout:982 writesame 0x30b061b thru 0x30bcf4b data_size 0x1(0xc931 bytes) 2024-09-21T14:58:52.990 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:58:52.989+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:58:52.999 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:52.999 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:53.008 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T14:58:53.009 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:53.009 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:53.009 INFO:teuthology.orchestra.run.smithi032.stdout:986 read 0x91ff338 thru 0x920b72b (0xc3f4 bytes) 2024-09-21T14:58:53.009 INFO:teuthology.orchestra.run.smithi032.stdout:987 write 0x6fe0b0a thru 0x6fe0c9b (0x192 bytes) 2024-09-21T14:58:53.013 INFO:teuthology.orchestra.run.smithi032.stdout:988 write 0x93d8806 thru 0x93ddbc1 (0x53bc bytes) 2024-09-21T14:58:53.022 INFO:teuthology.orchestra.run.smithi032.stdout:989 trunc from 0xe7a6eb5 to 0xd2a073f 2024-09-21T14:58:53.036 INFO:teuthology.orchestra.run.smithi032.stdout:990 write 0xb26e939 thru 0xb27632f (0x79f7 bytes) 2024-09-21T14:58:53.054 INFO:teuthology.orchestra.run.smithi032.stdout:991 writesame 0xdfcb4e9 thru 0xdfd6f50 data_size 0x14(0xba68 bytes) 2024-09-21T14:58:53.055 INFO:teuthology.orchestra.run.smithi032.stdout:992 trunc from 0xdfd6f51 to 0xe251b84 2024-09-21T14:58:53.063 INFO:teuthology.orchestra.run.smithi032.stdout:994 read 0xb47c411 thru 0xb488c37 (0xc827 bytes) 2024-09-21T14:58:53.064 INFO:teuthology.orchestra.run.smithi032.stdout:995 punch from 0xa549a4a to 0xa551d54, (0x830a bytes) 2024-09-21T14:58:53.064 INFO:teuthology.orchestra.run.smithi032.stdout:996 read 0x5dd6fcc thru 0x5de0fe4 (0xa019 bytes) 2024-09-21T14:58:53.064 INFO:teuthology.orchestra.run.smithi032.stdout:997 read 0x8770302 thru 0x8776eee (0x6bed bytes) 2024-09-21T14:58:53.064 INFO:teuthology.orchestra.run.smithi032.stdout:999 writesame 0x7d60c95 thru 0x7d6649e data_size 0x3b(0x580a bytes) 2024-09-21T14:58:53.072 INFO:teuthology.orchestra.run.smithi032.stdout:1001 punch from 0x4419232 to 0x4425c8c, (0xca5a bytes) 2024-09-21T14:58:53.072 INFO:teuthology.orchestra.run.smithi032.stdout:1002 clone 9 order 24 su 1048576 sc 10 2024-09-21T14:58:53.091 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:58:53.088+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:58:53.119 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:53.120 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:53.129 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T14:58:53.192 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:58:53.189+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:58:53.217 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T14:58:53.225 INFO:teuthology.orchestra.run.smithi032.stderr:noscrub is set 2024-09-21T14:58:53.293 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:58:53.293+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:58:53.327 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:53.327 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:53.340 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T14:58:53.386 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T14:58:53.394 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:58:53.393+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:58:53.407 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T14:58:53.429 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T14:58:53.448 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T14:58:53.456 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:53.456 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:53.466 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T14:58:53.468 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-21T14:58:53.491 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T14:58:53.495 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:58:53.493+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:58:53.513 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T14:58:53.537 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T14:58:53.539 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:58:53.539 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:58:53.539 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:58:53.539 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:58:53.539 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:58:53.539 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:58:53.540 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:58:53.540 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:58:53.570 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:58:53.570 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:58:53.570 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T14:58:53.570 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T14:58:53.570 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:58:53.579 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T14:58:53.596 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:58:53.592+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:58:53.689 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:58:53.689 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T14:58:53.689 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T14:58:53.689 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T14:58:53.689 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.689 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:1452 3.19 3:9d40fe72:::rbd_data.1136515ecb6b.0000000000000013:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T14:58:53.689 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:30.292755+0000", 2024-09-21T14:58:53.689 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.393076804, 2024-09-21T14:58:53.689 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043852191999999998, 2024-09-21T14:58:53.689 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:53.689 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:53.689 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:53.690 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:53.690 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:53.690 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1452 2024-09-21T14:58:53.690 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.690 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:53.690 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.690 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:53.690 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292755+0000", 2024-09-21T14:58:53.690 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:53.690 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.690 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.690 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:53.690 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292755+0000", 2024-09-21T14:58:53.690 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:53.691 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.691 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.691 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:53.691 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292754+0000", 2024-09-21T14:58:53.691 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:53.691 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.691 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.691 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:53.691 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292760+0000", 2024-09-21T14:58:53.691 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5029999999999997e-06 2024-09-21T14:58:53.691 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.691 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.691 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:53.691 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292771+0000", 2024-09-21T14:58:53.691 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1551e-05 2024-09-21T14:58:53.692 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.692 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.692 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:53.692 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292779+0000", 2024-09-21T14:58:53.692 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1559999999999998e-06 2024-09-21T14:58:53.692 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.692 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.692 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:53.692 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292896+0000", 2024-09-21T14:58:53.692 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000117304 2024-09-21T14:58:53.692 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.692 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.692 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:53.692 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.293124+0000", 2024-09-21T14:58:53.692 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00022819999999999999 2024-09-21T14:58:53.693 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.693 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.693 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 1", 2024-09-21T14:58:53.693 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.299967+0000", 2024-09-21T14:58:53.693 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0068427009999999996 2024-09-21T14:58:53.693 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.693 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.693 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:53.693 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336463+0000", 2024-09-21T14:58:53.693 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.036495842000000001 2024-09-21T14:58:53.693 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.693 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.693 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:58:53.693 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336480+0000", 2024-09-21T14:58:53.693 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7419999999999999e-05 2024-09-21T14:58:53.693 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.694 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.694 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:53.694 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336550+0000", 2024-09-21T14:58:53.694 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9951999999999997e-05 2024-09-21T14:58:53.694 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.694 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.694 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:53.694 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336607+0000", 2024-09-21T14:58:53.694 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7423000000000002e-05 2024-09-21T14:58:53.694 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:53.694 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:53.694 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:53.694 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.694 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.694 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop_reply(client.4406.0:1452 3.19 e36/24 ondisk, result = 0)", 2024-09-21T14:58:53.695 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:30.322854+0000", 2024-09-21T14:58:53.695 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.362978128000002, 2024-09-21T14:58:53.695 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013613079, 2024-09-21T14:58:53.695 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:53.695 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:53.695 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:53.695 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.695 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:53.695 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.322854+0000", 2024-09-21T14:58:53.695 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:53.695 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.695 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.695 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:53.695 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.322854+0000", 2024-09-21T14:58:53.695 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:53.696 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.696 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.696 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:53.696 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.322852+0000", 2024-09-21T14:58:53.696 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:53.696 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.696 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.696 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:53.696 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.322860+0000", 2024-09-21T14:58:53.696 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.605e-06 2024-09-21T14:58:53.696 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.696 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.696 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:53.696 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.322878+0000", 2024-09-21T14:58:53.696 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8204000000000001e-05 2024-09-21T14:58:53.697 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.697 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.697 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:53.697 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.322888+0000", 2024-09-21T14:58:53.697 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4099999999999997e-06 2024-09-21T14:58:53.697 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.697 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.697 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:53.697 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336440+0000", 2024-09-21T14:58:53.697 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013551848 2024-09-21T14:58:53.697 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.697 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.697 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:53.697 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336453+0000", 2024-09-21T14:58:53.697 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3465000000000001e-05 2024-09-21T14:58:53.697 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.698 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.698 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:53.698 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336467+0000", 2024-09-21T14:58:53.698 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3913999999999999e-05 2024-09-21T14:58:53.698 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:53.698 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:53.698 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:53.698 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.698 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.698 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:1567 3.8 3:139d3d55:::rbd_data.1136515ecb6b.0000000000000000:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T14:58:53.698 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:30.681087+0000", 2024-09-21T14:58:53.698 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.004744589000001, 2024-09-21T14:58:53.698 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.039009224000000002, 2024-09-21T14:58:53.699 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:53.699 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:53.699 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:53.699 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:53.699 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:53.699 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1567 2024-09-21T14:58:53.699 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.699 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:53.699 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.699 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:53.699 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681087+0000", 2024-09-21T14:58:53.699 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:53.699 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.699 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.699 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:53.700 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681087+0000", 2024-09-21T14:58:53.700 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:53.700 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.700 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.700 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:53.700 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681086+0000", 2024-09-21T14:58:53.700 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:53.700 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.700 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.700 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:53.700 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681097+0000", 2024-09-21T14:58:53.700 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.134e-05 2024-09-21T14:58:53.700 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.700 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.700 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:53.700 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681122+0000", 2024-09-21T14:58:53.701 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5313e-05 2024-09-21T14:58:53.701 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.701 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.701 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:53.701 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681138+0000", 2024-09-21T14:58:53.701 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5916000000000001e-05 2024-09-21T14:58:53.701 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.701 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.701 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:53.701 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681270+0000", 2024-09-21T14:58:53.701 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000131474 2024-09-21T14:58:53.701 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.701 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.701 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:53.701 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681416+0000", 2024-09-21T14:58:53.702 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014637 2024-09-21T14:58:53.702 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.702 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.702 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 7", 2024-09-21T14:58:53.702 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.693163+0000", 2024-09-21T14:58:53.702 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011747241 2024-09-21T14:58:53.702 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.702 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.702 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:58:53.702 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.719750+0000", 2024-09-21T14:58:53.702 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.026586852000000001 2024-09-21T14:58:53.702 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.702 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.702 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:53.703 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.719859+0000", 2024-09-21T14:58:53.703 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010925999999999999 2024-09-21T14:58:53.703 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.703 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.703 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:53.703 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.719951+0000", 2024-09-21T14:58:53.703 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1294999999999999e-05 2024-09-21T14:58:53.703 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.703 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.703 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:53.703 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.720097+0000", 2024-09-21T14:58:53.703 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014583 2024-09-21T14:58:53.703 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:53.703 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:53.703 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:53.703 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.704 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.704 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9320 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 425984~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:53.704 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158246+0000", 2024-09-21T14:58:53.704 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.527585983, 2024-09-21T14:58:53.704 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013343911, 2024-09-21T14:58:53.704 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:53.704 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:53.704 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:53.704 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:53.704 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:53.704 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9320 2024-09-21T14:58:53.704 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.704 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:53.704 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.704 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:53.705 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158246+0000", 2024-09-21T14:58:53.705 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:53.705 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.705 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.705 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:53.705 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158246+0000", 2024-09-21T14:58:53.705 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:53.705 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.705 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.705 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:53.705 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158245+0000", 2024-09-21T14:58:53.705 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:53.705 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.706 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.706 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:53.706 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158249+0000", 2024-09-21T14:58:53.706 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8659999999999999e-06 2024-09-21T14:58:53.706 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.706 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.706 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:53.706 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158258+0000", 2024-09-21T14:58:53.706 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0720000000000006e-06 2024-09-21T14:58:53.706 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.706 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.706 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:53.706 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158266+0000", 2024-09-21T14:58:53.706 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0039999999999999e-06 2024-09-21T14:58:53.707 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.707 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.707 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:53.707 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.171302+0000", 2024-09-21T14:58:53.707 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013035972999999999 2024-09-21T14:58:53.707 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.707 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.707 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:53.707 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.171395+0000", 2024-09-21T14:58:53.707 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3071000000000004e-05 2024-09-21T14:58:53.707 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.707 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.707 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:53.707 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.171590+0000", 2024-09-21T14:58:53.707 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000194525 2024-09-21T14:58:53.707 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:53.708 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:53.708 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:53.708 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.708 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.708 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9347 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 16384~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:53.708 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158291+0000", 2024-09-21T14:58:53.708 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.527541088, 2024-09-21T14:58:53.708 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013600958, 2024-09-21T14:58:53.708 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:53.708 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:53.708 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:53.708 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:53.708 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:53.708 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9347 2024-09-21T14:58:53.708 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.709 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:53.709 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.709 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:53.709 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158291+0000", 2024-09-21T14:58:53.709 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:53.709 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.709 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.709 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:53.709 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158291+0000", 2024-09-21T14:58:53.709 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:53.709 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.709 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.709 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:53.709 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158290+0000", 2024-09-21T14:58:53.709 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:53.710 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.710 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.710 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:53.710 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158295+0000", 2024-09-21T14:58:53.710 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2399999999999998e-06 2024-09-21T14:58:53.710 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.710 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.710 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:53.710 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158312+0000", 2024-09-21T14:58:53.710 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6929999999999999e-05 2024-09-21T14:58:53.710 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.710 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.710 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:53.710 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158353+0000", 2024-09-21T14:58:53.710 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0805e-05 2024-09-21T14:58:53.711 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.711 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.711 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:53.711 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.171634+0000", 2024-09-21T14:58:53.711 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.01328085 2024-09-21T14:58:53.711 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.711 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.711 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:53.711 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.171722+0000", 2024-09-21T14:58:53.711 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8115000000000006e-05 2024-09-21T14:58:53.711 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.711 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.711 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:53.711 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.171892+0000", 2024-09-21T14:58:53.711 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000169722 2024-09-21T14:58:53.712 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:53.712 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:53.712 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:53.712 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.712 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.712 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9348 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 122880~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:53.712 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158369+0000", 2024-09-21T14:58:53.712 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.52746292, 2024-09-21T14:58:53.712 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.01388847, 2024-09-21T14:58:53.712 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:53.712 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:53.712 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:53.712 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:53.712 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:53.713 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9348 2024-09-21T14:58:53.713 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.713 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:53.713 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.713 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:53.713 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158369+0000", 2024-09-21T14:58:53.713 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:53.713 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.713 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.713 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:53.713 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158369+0000", 2024-09-21T14:58:53.713 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:53.713 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.713 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.713 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:53.714 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158368+0000", 2024-09-21T14:58:53.714 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:53.714 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.714 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.714 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:53.714 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158374+0000", 2024-09-21T14:58:53.714 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0499999999999997e-06 2024-09-21T14:58:53.714 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.714 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.714 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:53.714 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158386+0000", 2024-09-21T14:58:53.714 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1579999999999999e-05 2024-09-21T14:58:53.714 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.715 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.715 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:53.715 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158396+0000", 2024-09-21T14:58:53.715 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.8260000000000005e-06 2024-09-21T14:58:53.715 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.715 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.715 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:53.715 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.171967+0000", 2024-09-21T14:58:53.715 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013571318000000001 2024-09-21T14:58:53.715 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.715 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.715 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:53.715 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172034+0000", 2024-09-21T14:58:53.715 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.656e-05 2024-09-21T14:58:53.716 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.716 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.716 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:53.716 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172258+0000", 2024-09-21T14:58:53.716 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00022395999999999999 2024-09-21T14:58:53.716 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:53.716 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:53.716 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:53.716 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.716 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.716 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9349 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 155648~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:53.716 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158403+0000", 2024-09-21T14:58:53.716 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.527429174, 2024-09-21T14:58:53.716 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014229851, 2024-09-21T14:58:53.717 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:53.717 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:53.717 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:53.717 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:53.717 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:53.717 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9349 2024-09-21T14:58:53.717 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.717 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:53.717 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.717 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:53.717 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158403+0000", 2024-09-21T14:58:53.717 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:53.717 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.717 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.718 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:53.718 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158403+0000", 2024-09-21T14:58:53.718 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:53.718 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.718 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.718 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:53.718 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158402+0000", 2024-09-21T14:58:53.718 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:53.718 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.718 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.718 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:53.718 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158406+0000", 2024-09-21T14:58:53.718 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1339999999999997e-06 2024-09-21T14:58:53.718 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.719 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.719 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:53.719 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158416+0000", 2024-09-21T14:58:53.719 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3149999999999998e-06 2024-09-21T14:58:53.719 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.719 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.719 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:53.719 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158439+0000", 2024-09-21T14:58:53.719 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3255999999999999e-05 2024-09-21T14:58:53.719 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.719 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.719 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:53.719 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172305+0000", 2024-09-21T14:58:53.719 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013866603999999999 2024-09-21T14:58:53.720 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.720 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.720 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:53.720 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172413+0000", 2024-09-21T14:58:53.720 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000107664 2024-09-21T14:58:53.720 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.720 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.720 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:53.720 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172633+0000", 2024-09-21T14:58:53.720 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021950800000000001 2024-09-21T14:58:53.720 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:53.720 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:53.721 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:53.721 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.721 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.721 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9350 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 294912~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:53.721 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158448+0000", 2024-09-21T14:58:53.721 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.527384303, 2024-09-21T14:58:53.721 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014471525000000001, 2024-09-21T14:58:53.721 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:53.721 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:53.721 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:53.721 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:53.721 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:53.721 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9350 2024-09-21T14:58:53.722 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.722 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:53.722 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.722 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:53.722 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158448+0000", 2024-09-21T14:58:53.722 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:53.722 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.722 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.722 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:53.722 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158448+0000", 2024-09-21T14:58:53.722 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:53.722 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.722 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.723 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:53.723 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158447+0000", 2024-09-21T14:58:53.723 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:53.723 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.723 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.723 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:53.723 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158452+0000", 2024-09-21T14:58:53.723 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8400000000000002e-06 2024-09-21T14:58:53.723 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.723 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.723 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:53.723 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158475+0000", 2024-09-21T14:58:53.723 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3091e-05 2024-09-21T14:58:53.723 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.724 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.724 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:53.724 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158515+0000", 2024-09-21T14:58:53.724 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0265000000000002e-05 2024-09-21T14:58:53.724 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.724 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.724 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:53.724 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172682+0000", 2024-09-21T14:58:53.724 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014166761999999999 2024-09-21T14:58:53.724 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.724 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.724 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:53.724 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172752+0000", 2024-09-21T14:58:53.725 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9698999999999998e-05 2024-09-21T14:58:53.725 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.725 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.725 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:53.725 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172919+0000", 2024-09-21T14:58:53.725 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016759199999999999 2024-09-21T14:58:53.725 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:53.725 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:53.725 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:53.725 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.725 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.725 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9351 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 434176~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:53.725 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158526+0000", 2024-09-21T14:58:53.725 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.527305585000001, 2024-09-21T14:58:53.726 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014662107000000001, 2024-09-21T14:58:53.726 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:53.726 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:53.726 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:53.726 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:53.726 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:53.726 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9351 2024-09-21T14:58:53.726 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.726 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:53.726 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.726 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:53.726 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158526+0000", 2024-09-21T14:58:53.726 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:53.726 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.727 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.727 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:53.727 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158526+0000", 2024-09-21T14:58:53.727 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:53.727 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.727 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.727 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:53.727 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158525+0000", 2024-09-21T14:58:53.727 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:53.727 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.727 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.727 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:53.727 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158531+0000", 2024-09-21T14:58:53.727 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5840000000000002e-06 2024-09-21T14:58:53.728 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.728 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.728 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:53.728 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158564+0000", 2024-09-21T14:58:53.728 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3195000000000003e-05 2024-09-21T14:58:53.728 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.728 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.728 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:53.728 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158577+0000", 2024-09-21T14:58:53.728 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2330000000000001e-05 2024-09-21T14:58:53.728 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.728 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.728 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:53.728 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172956+0000", 2024-09-21T14:58:53.729 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014379438 2024-09-21T14:58:53.729 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.729 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.729 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:53.729 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173020+0000", 2024-09-21T14:58:53.729 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4401999999999998e-05 2024-09-21T14:58:53.729 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.729 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.729 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:53.729 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173188+0000", 2024-09-21T14:58:53.729 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016804 2024-09-21T14:58:53.729 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:53.729 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:53.730 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:53.730 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.730 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.730 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9379 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 24576~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:53.730 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158586+0000", 2024-09-21T14:58:53.730 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.527246325, 2024-09-21T14:58:53.730 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014865173000000001, 2024-09-21T14:58:53.730 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:53.730 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:53.730 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:53.730 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:53.730 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:53.730 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9379 2024-09-21T14:58:53.730 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.731 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:53.731 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.731 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:53.731 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158586+0000", 2024-09-21T14:58:53.731 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:53.731 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.731 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.731 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:53.731 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158586+0000", 2024-09-21T14:58:53.731 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:53.731 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.731 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.731 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:53.732 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158585+0000", 2024-09-21T14:58:53.732 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:53.732 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.732 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.732 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:53.732 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158590+0000", 2024-09-21T14:58:53.732 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5820000000000001e-06 2024-09-21T14:58:53.732 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.732 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.732 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:53.732 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158676+0000", 2024-09-21T14:58:53.732 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5075999999999997e-05 2024-09-21T14:58:53.732 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.732 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.733 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:53.733 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158695+0000", 2024-09-21T14:58:53.733 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9137999999999999e-05 2024-09-21T14:58:53.733 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.733 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.733 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:53.733 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173219+0000", 2024-09-21T14:58:53.733 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014524034999999999 2024-09-21T14:58:53.733 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.733 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.733 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:53.733 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173276+0000", 2024-09-21T14:58:53.733 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7442000000000003e-05 2024-09-21T14:58:53.734 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.734 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.734 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:53.734 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173451+0000", 2024-09-21T14:58:53.734 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017464800000000001 2024-09-21T14:58:53.734 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:53.734 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:53.734 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:53.734 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.734 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.734 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9380 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 163840~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:53.734 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158706+0000", 2024-09-21T14:58:53.734 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.527125815, 2024-09-21T14:58:53.734 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015096381000000001, 2024-09-21T14:58:53.735 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:53.735 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:53.735 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:53.735 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:53.735 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:53.735 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9380 2024-09-21T14:58:53.735 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.735 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:53.735 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.735 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:53.735 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158706+0000", 2024-09-21T14:58:53.735 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:53.735 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.735 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.736 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:53.736 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158706+0000", 2024-09-21T14:58:53.736 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:53.736 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.736 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.736 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:53.736 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158705+0000", 2024-09-21T14:58:53.736 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:53.736 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.736 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.736 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:53.736 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158712+0000", 2024-09-21T14:58:53.736 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7240000000000003e-06 2024-09-21T14:58:53.737 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.737 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.737 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:53.737 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158755+0000", 2024-09-21T14:58:53.737 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2953e-05 2024-09-21T14:58:53.737 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.737 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.737 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:53.737 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158765+0000", 2024-09-21T14:58:53.737 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0149000000000001e-05 2024-09-21T14:58:53.737 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.737 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.737 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:53.738 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173492+0000", 2024-09-21T14:58:53.738 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014727105000000001 2024-09-21T14:58:53.738 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.738 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.738 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:53.738 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173564+0000", 2024-09-21T14:58:53.738 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1789e-05 2024-09-21T14:58:53.738 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.738 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.738 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:53.738 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173803+0000", 2024-09-21T14:58:53.738 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00023867100000000001 2024-09-21T14:58:53.738 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:53.738 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:53.738 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:53.739 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.739 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.739 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9381 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 303104~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:53.739 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158774+0000", 2024-09-21T14:58:53.739 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.527057735, 2024-09-21T14:58:53.739 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015309095999999999, 2024-09-21T14:58:53.739 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:53.739 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:53.739 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:53.739 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:53.739 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:53.739 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9381 2024-09-21T14:58:53.739 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.739 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:53.740 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.740 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:53.740 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158774+0000", 2024-09-21T14:58:53.740 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:53.740 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.740 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.740 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:53.740 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158774+0000", 2024-09-21T14:58:53.740 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:53.740 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.740 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.740 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:53.740 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158773+0000", 2024-09-21T14:58:53.741 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:53.741 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.741 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.741 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:53.741 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158779+0000", 2024-09-21T14:58:53.741 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6999999999999996e-06 2024-09-21T14:58:53.741 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.741 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.741 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:53.741 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158790+0000", 2024-09-21T14:58:53.741 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0594000000000001e-05 2024-09-21T14:58:53.741 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.741 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.741 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:53.741 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158798+0000", 2024-09-21T14:58:53.742 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5860000000000005e-06 2024-09-21T14:58:53.742 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.742 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.742 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:53.742 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173829+0000", 2024-09-21T14:58:53.742 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015030349 2024-09-21T14:58:53.742 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.742 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.742 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:53.742 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173918+0000", 2024-09-21T14:58:53.742 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9375000000000004e-05 2024-09-21T14:58:53.742 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.742 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.743 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:53.743 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.174083+0000", 2024-09-21T14:58:53.743 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016536200000000001 2024-09-21T14:58:53.743 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:53.743 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:53.743 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:53.743 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.743 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.744 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9382 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 442368~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:53.745 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158805+0000", 2024-09-21T14:58:53.745 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.527026830000001, 2024-09-21T14:58:53.745 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015592623, 2024-09-21T14:58:53.745 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:53.745 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:53.745 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:53.745 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:53.745 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:53.745 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9382 2024-09-21T14:58:53.745 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.745 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:53.745 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.746 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:53.746 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158805+0000", 2024-09-21T14:58:53.746 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:53.746 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.746 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.746 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:53.746 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158805+0000", 2024-09-21T14:58:53.746 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:53.746 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.746 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.746 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:53.746 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158805+0000", 2024-09-21T14:58:53.746 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:53.747 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.747 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.747 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:53.747 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158808+0000", 2024-09-21T14:58:53.747 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9179999999999996e-06 2024-09-21T14:58:53.747 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.747 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.747 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:53.747 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158817+0000", 2024-09-21T14:58:53.747 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9299999999999992e-06 2024-09-21T14:58:53.747 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.747 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.747 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:53.747 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158825+0000", 2024-09-21T14:58:53.748 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6739999999999997e-06 2024-09-21T14:58:53.748 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.748 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.748 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:53.748 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.174122+0000", 2024-09-21T14:58:53.748 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.01529694 2024-09-21T14:58:53.748 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.748 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.748 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:53.748 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.174185+0000", 2024-09-21T14:58:53.748 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3015000000000006e-05 2024-09-21T14:58:53.748 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.748 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.748 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:53.749 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.174398+0000", 2024-09-21T14:58:53.749 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021274799999999999 2024-09-21T14:58:53.749 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:53.749 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:53.749 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:53.749 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.749 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.749 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22699 3.19 3:98de45b8:::rbd_data.11366acc894.0000000000000000:19 [read 638976~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:58:53.749 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.140173+0000", 2024-09-21T14:58:53.749 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.5456587329999998, 2024-09-21T14:58:53.749 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013511366, 2024-09-21T14:58:53.749 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:53.749 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:53.749 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:53.750 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:53.750 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:53.750 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22699 2024-09-21T14:58:53.750 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.750 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:53.750 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.750 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:53.750 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140173+0000", 2024-09-21T14:58:53.750 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:53.750 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.750 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.750 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:53.750 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140173+0000", 2024-09-21T14:58:53.750 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:53.751 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.751 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.751 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:53.751 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140173+0000", 2024-09-21T14:58:53.751 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:53.751 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.751 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.751 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:53.751 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140175+0000", 2024-09-21T14:58:53.751 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5859999999999999e-06 2024-09-21T14:58:53.751 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.751 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.751 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:53.751 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140181+0000", 2024-09-21T14:58:53.752 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0340000000000002e-06 2024-09-21T14:58:53.752 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.752 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.752 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:53.752 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140186+0000", 2024-09-21T14:58:53.752 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.04e-06 2024-09-21T14:58:53.752 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.752 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.752 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:53.752 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.153426+0000", 2024-09-21T14:58:53.752 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013240001 2024-09-21T14:58:53.752 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.752 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.752 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:53.753 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.153513+0000", 2024-09-21T14:58:53.753 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6038999999999994e-05 2024-09-21T14:58:53.753 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.753 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.753 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:53.753 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.153685+0000", 2024-09-21T14:58:53.753 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000172067 2024-09-21T14:58:53.753 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:53.753 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:53.753 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:53.753 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.753 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.753 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22700 3.19 3:98de45b8:::rbd_data.11366acc894.0000000000000000:19 [read 704512~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:58:53.753 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.140191+0000", 2024-09-21T14:58:53.754 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.5456412159999999, 2024-09-21T14:58:53.754 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013770486, 2024-09-21T14:58:53.754 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:53.754 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:53.754 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:53.754 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:53.754 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:53.754 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22700 2024-09-21T14:58:53.754 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.754 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:53.754 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.754 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:53.754 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140191+0000", 2024-09-21T14:58:53.754 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:53.755 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.755 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.755 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:53.755 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140191+0000", 2024-09-21T14:58:53.755 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:53.755 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.755 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.755 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:53.755 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140190+0000", 2024-09-21T14:58:53.755 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:53.755 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.755 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.755 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:53.755 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140199+0000", 2024-09-21T14:58:53.755 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3340000000000002e-06 2024-09-21T14:58:53.756 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.756 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.756 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:53.756 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140205+0000", 2024-09-21T14:58:53.756 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3820000000000001e-06 2024-09-21T14:58:53.756 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.756 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.756 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:53.756 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140210+0000", 2024-09-21T14:58:53.756 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0969999999999996e-06 2024-09-21T14:58:53.756 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.756 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.757 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:53.757 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.153713+0000", 2024-09-21T14:58:53.757 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013502713 2024-09-21T14:58:53.757 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.757 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.757 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:53.757 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.153786+0000", 2024-09-21T14:58:53.757 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3022999999999995e-05 2024-09-21T14:58:53.757 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.757 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.757 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:53.757 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.153961+0000", 2024-09-21T14:58:53.757 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000175274 2024-09-21T14:58:53.757 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:53.758 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:53.758 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:53.758 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.758 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.758 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22701 3.19 3:98de45b8:::rbd_data.11366acc894.0000000000000000:19 [read 770048~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:58:53.758 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.140215+0000", 2024-09-21T14:58:53.758 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.5456173990000002, 2024-09-21T14:58:53.758 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014072497, 2024-09-21T14:58:53.758 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:53.758 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:53.758 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:53.758 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:53.758 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:53.759 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22701 2024-09-21T14:58:53.759 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.759 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:53.759 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.759 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:53.759 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140215+0000", 2024-09-21T14:58:53.759 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:53.759 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.759 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.759 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:53.759 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140215+0000", 2024-09-21T14:58:53.759 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:53.759 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.759 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.760 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:53.760 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140214+0000", 2024-09-21T14:58:53.760 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:53.760 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.760 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.760 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:53.760 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140217+0000", 2024-09-21T14:58:53.760 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.74e-06 2024-09-21T14:58:53.760 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.760 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.760 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:53.760 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140223+0000", 2024-09-21T14:58:53.760 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0929999999999999e-06 2024-09-21T14:58:53.760 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.760 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.761 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:53.761 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140228+0000", 2024-09-21T14:58:53.761 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2279999999999998e-06 2024-09-21T14:58:53.761 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.761 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.761 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:53.761 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.153991+0000", 2024-09-21T14:58:53.761 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013762329 2024-09-21T14:58:53.761 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.761 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.761 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:53.761 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.154098+0000", 2024-09-21T14:58:53.761 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000107117 2024-09-21T14:58:53.761 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.761 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.762 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:53.762 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.154287+0000", 2024-09-21T14:58:53.762 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000189431 2024-09-21T14:58:53.762 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:53.762 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:53.762 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:53.762 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.762 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.762 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22702 3.19 3:98de45b8:::rbd_data.11366acc894.0000000000000000:19 [read 835584~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:58:53.762 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.140232+0000", 2024-09-21T14:58:53.762 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.5455995470000001, 2024-09-21T14:58:53.762 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015338742000000001, 2024-09-21T14:58:53.762 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:53.762 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:53.763 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:53.763 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:53.763 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:53.763 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22702 2024-09-21T14:58:53.763 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.763 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:53.763 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.763 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:53.763 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140232+0000", 2024-09-21T14:58:53.763 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:53.763 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.763 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.763 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:53.763 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140232+0000", 2024-09-21T14:58:53.764 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:53.764 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.764 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.764 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:53.764 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140232+0000", 2024-09-21T14:58:53.764 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:53.764 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.764 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.764 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:53.764 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140234+0000", 2024-09-21T14:58:53.764 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.402e-06 2024-09-21T14:58:53.764 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.764 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.764 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:53.764 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140241+0000", 2024-09-21T14:58:53.765 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.054e-06 2024-09-21T14:58:53.765 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.765 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.765 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:53.765 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140246+0000", 2024-09-21T14:58:53.765 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0050000000000004e-06 2024-09-21T14:58:53.765 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.765 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.765 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:53.765 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.154315+0000", 2024-09-21T14:58:53.765 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014069440000000001 2024-09-21T14:58:53.765 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.765 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.765 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:53.766 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.154383+0000", 2024-09-21T14:58:53.766 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8151000000000005e-05 2024-09-21T14:58:53.766 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.766 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.766 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:53.766 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155571+0000", 2024-09-21T14:58:53.766 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001188029 2024-09-21T14:58:53.766 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:53.766 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:53.766 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:53.766 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.766 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.766 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22703 3.19 3:98de45b8:::rbd_data.11366acc894.0000000000000000:19 [read 901120~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:58:53.766 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.140250+0000", 2024-09-21T14:58:53.766 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.5455823889999998, 2024-09-21T14:58:53.767 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014992169, 2024-09-21T14:58:53.767 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:53.767 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:53.767 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:53.767 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:53.767 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:53.767 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22703 2024-09-21T14:58:53.767 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.767 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:53.767 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.767 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:53.767 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140250+0000", 2024-09-21T14:58:53.767 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:53.768 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.768 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.768 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:53.768 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140250+0000", 2024-09-21T14:58:53.768 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:53.768 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.768 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.768 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:53.768 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140249+0000", 2024-09-21T14:58:53.768 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:53.768 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.768 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.768 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:53.768 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140251+0000", 2024-09-21T14:58:53.768 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.136e-06 2024-09-21T14:58:53.768 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.769 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.769 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:53.769 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140257+0000", 2024-09-21T14:58:53.769 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9610000000000002e-06 2024-09-21T14:58:53.769 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.769 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.769 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:53.769 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140262+0000", 2024-09-21T14:58:53.769 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4549999999999997e-06 2024-09-21T14:58:53.769 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.769 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.769 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:53.769 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155006+0000", 2024-09-21T14:58:53.770 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014744189 2024-09-21T14:58:53.770 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.770 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.770 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:53.770 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155081+0000", 2024-09-21T14:58:53.770 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4548000000000005e-05 2024-09-21T14:58:53.770 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.770 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.770 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:53.770 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155242+0000", 2024-09-21T14:58:53.770 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000161204 2024-09-21T14:58:53.770 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:53.770 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:53.770 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:53.771 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.771 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.771 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22704 3.19 3:98de45b8:::rbd_data.11366acc894.0000000000000000:19 [read 966656~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:58:53.771 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.140266+0000", 2024-09-21T14:58:53.771 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.5455660719999997, 2024-09-21T14:58:53.771 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015223443999999999, 2024-09-21T14:58:53.771 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:53.771 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:53.771 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:53.771 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:53.771 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:53.771 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22704 2024-09-21T14:58:53.772 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.772 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:53.772 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.772 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:53.772 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140266+0000", 2024-09-21T14:58:53.772 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:53.772 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.772 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.772 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:53.772 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140266+0000", 2024-09-21T14:58:53.772 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:53.772 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.772 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.772 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:53.773 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140266+0000", 2024-09-21T14:58:53.773 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:53.773 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.773 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.773 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:53.773 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140268+0000", 2024-09-21T14:58:53.773 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.475e-06 2024-09-21T14:58:53.773 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.773 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.773 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:53.773 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140274+0000", 2024-09-21T14:58:53.774 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.319e-06 2024-09-21T14:58:53.774 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.774 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.774 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:53.774 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140279+0000", 2024-09-21T14:58:53.774 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.882e-06 2024-09-21T14:58:53.774 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.774 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.774 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:53.774 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155264+0000", 2024-09-21T14:58:53.774 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014984655 2024-09-21T14:58:53.774 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.775 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.775 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:53.775 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155338+0000", 2024-09-21T14:58:53.775 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4091999999999995e-05 2024-09-21T14:58:53.775 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.775 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.775 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:53.775 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155489+0000", 2024-09-21T14:58:53.775 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000151349 2024-09-21T14:58:53.775 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:53.775 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:53.775 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:53.775 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.776 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.776 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22705 3.19 3:98de45b8:::rbd_data.11366acc894.0000000000000000:19 [read 1032192~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:58:53.776 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.140284+0000", 2024-09-21T14:58:53.776 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.5455482150000002, 2024-09-21T14:58:53.776 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015467896, 2024-09-21T14:58:53.776 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:53.776 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:53.776 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:53.776 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:53.776 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:53.776 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22705 2024-09-21T14:58:53.776 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.776 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:53.777 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.777 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:53.777 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140284+0000", 2024-09-21T14:58:53.777 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:53.777 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.777 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.777 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:53.777 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140284+0000", 2024-09-21T14:58:53.777 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:53.777 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.777 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.777 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:53.777 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140283+0000", 2024-09-21T14:58:53.777 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:53.778 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.778 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.778 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:53.778 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140286+0000", 2024-09-21T14:58:53.778 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7130000000000001e-06 2024-09-21T14:58:53.778 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.778 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.778 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:53.778 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140292+0000", 2024-09-21T14:58:53.778 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3430000000000001e-06 2024-09-21T14:58:53.778 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.778 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.778 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:53.778 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140298+0000", 2024-09-21T14:58:53.779 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9730000000000002e-06 2024-09-21T14:58:53.779 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.779 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.779 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:53.779 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155510+0000", 2024-09-21T14:58:53.779 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015211738000000001 2024-09-21T14:58:53.779 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.779 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.779 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:53.779 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155598+0000", 2024-09-21T14:58:53.779 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7822e-05 2024-09-21T14:58:53.779 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:53.779 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:53.779 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:53.779 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155752+0000", 2024-09-21T14:58:53.780 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0001537 2024-09-21T14:58:53.780 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:53.780 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:53.780 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:53.780 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:53.780 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:53.780 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:58:53.780 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T14:58:53.781 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:58:53.740+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:58:53.845 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:58:53.845+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:58:53.902 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:58:53.902 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:58:53.902 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T14:58:53.902 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:58:53.912 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T14:58:53.946 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:58:53.945+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:58:54.019 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:58:54.019 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:58:54.019 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T14:58:54.019 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T14:58:54.019 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:58:54.029 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T14:58:54.073 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:58:54.072+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:58:54.138 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:58:54.139 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T14:58:54.139 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T14:58:54.139 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T14:58:54.139 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.139 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22719 3.1e 3:79caef05:::rbd_data.11366acc894.0000000000000001:19 [read 901120~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:58:54.139 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.140027+0000", 2024-09-21T14:58:54.139 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.9975414580000002, 2024-09-21T14:58:54.139 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015081021, 2024-09-21T14:58:54.139 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:54.139 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:54.139 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:54.139 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:54.139 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:54.140 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22719 2024-09-21T14:58:54.140 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.140 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:54.140 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.140 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:54.140 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140027+0000", 2024-09-21T14:58:54.140 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:54.140 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.140 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.140 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:54.140 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140027+0000", 2024-09-21T14:58:54.140 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:54.140 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.140 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.141 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:54.141 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140026+0000", 2024-09-21T14:58:54.141 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:54.141 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.141 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.141 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:54.141 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140042+0000", 2024-09-21T14:58:54.141 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5381999999999999e-05 2024-09-21T14:58:54.141 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.141 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.141 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:54.141 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140051+0000", 2024-09-21T14:58:54.141 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3500000000000003e-06 2024-09-21T14:58:54.141 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.142 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.142 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:54.142 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140066+0000", 2024-09-21T14:58:54.142 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4582e-05 2024-09-21T14:58:54.142 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.142 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.142 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:54.142 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.154602+0000", 2024-09-21T14:58:54.142 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014535881 2024-09-21T14:58:54.142 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.142 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.142 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:54.142 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.154674+0000", 2024-09-21T14:58:54.143 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2303000000000002e-05 2024-09-21T14:58:54.143 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.143 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.143 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:54.143 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155108+0000", 2024-09-21T14:58:54.143 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00043396500000000002 2024-09-21T14:58:54.143 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:54.143 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:54.143 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:54.143 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.143 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.143 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22720 3.1e 3:79caef05:::rbd_data.11366acc894.0000000000000001:19 [read 966656~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:58:54.143 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.140073+0000", 2024-09-21T14:58:54.143 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.9974948929999998, 2024-09-21T14:58:54.144 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015292312000000001, 2024-09-21T14:58:54.144 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:54.144 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:54.144 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:54.144 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:54.144 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:54.144 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22720 2024-09-21T14:58:54.144 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.144 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:54.144 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.144 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:54.144 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140073+0000", 2024-09-21T14:58:54.144 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:54.144 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.144 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.145 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:54.145 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140073+0000", 2024-09-21T14:58:54.145 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:54.145 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.145 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.145 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:54.145 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140073+0000", 2024-09-21T14:58:54.145 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:54.145 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.145 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.145 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:54.145 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140076+0000", 2024-09-21T14:58:54.145 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4999999999999999e-06 2024-09-21T14:58:54.145 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.146 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.146 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:54.146 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140084+0000", 2024-09-21T14:58:54.146 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1310000000000006e-06 2024-09-21T14:58:54.146 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.146 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.146 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:54.146 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140091+0000", 2024-09-21T14:58:54.146 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.674e-06 2024-09-21T14:58:54.146 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.146 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.146 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:54.146 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155135+0000", 2024-09-21T14:58:54.147 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015044224 2024-09-21T14:58:54.147 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.147 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.147 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:54.147 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155205+0000", 2024-09-21T14:58:54.147 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9751000000000004e-05 2024-09-21T14:58:54.147 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.147 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.147 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:54.147 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155366+0000", 2024-09-21T14:58:54.147 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016073899999999999 2024-09-21T14:58:54.147 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:54.147 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:54.147 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:54.148 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.148 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.148 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22721 3.1e 3:79caef05:::rbd_data.11366acc894.0000000000000001:19 [read 1032192~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:58:54.148 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.140096+0000", 2024-09-21T14:58:54.148 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.9974719080000001, 2024-09-21T14:58:54.148 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015563158000000001, 2024-09-21T14:58:54.148 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:54.148 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:54.148 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:54.148 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:54.148 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:54.148 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22721 2024-09-21T14:58:54.148 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.148 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:54.149 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.149 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:54.149 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140096+0000", 2024-09-21T14:58:54.149 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:54.149 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.149 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.149 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:54.149 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140096+0000", 2024-09-21T14:58:54.149 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:54.149 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.149 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.149 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:54.149 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140096+0000", 2024-09-21T14:58:54.149 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:54.149 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.150 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.150 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:54.150 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140099+0000", 2024-09-21T14:58:54.150 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.5080000000000001e-06 2024-09-21T14:58:54.150 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.150 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.150 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:54.150 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140106+0000", 2024-09-21T14:58:54.150 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8630000000000002e-06 2024-09-21T14:58:54.150 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.150 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.150 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:54.150 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140112+0000", 2024-09-21T14:58:54.151 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6509999999999998e-06 2024-09-21T14:58:54.151 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.151 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.151 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:54.151 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155392+0000", 2024-09-21T14:58:54.151 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015280375000000001 2024-09-21T14:58:54.151 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.151 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.151 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:54.151 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155459+0000", 2024-09-21T14:58:54.151 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7040999999999997e-05 2024-09-21T14:58:54.151 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.151 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.151 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:54.152 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155660+0000", 2024-09-21T14:58:54.152 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000200115 2024-09-21T14:58:54.152 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:54.152 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:54.152 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:54.152 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.152 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.152 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22865 3.1e 3:7ca947c4:::rbd_data.11366acc894.000000000000000b:19 [read 393216~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:58:54.152 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.142757+0000", 2024-09-21T14:58:54.152 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.994811237, 2024-09-21T14:58:54.152 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014907012000000001, 2024-09-21T14:58:54.152 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:54.152 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:54.152 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:54.153 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:54.153 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:54.153 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22865 2024-09-21T14:58:54.153 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.153 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:54.153 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.153 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:54.153 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.142757+0000", 2024-09-21T14:58:54.153 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:54.153 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.153 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.153 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:54.153 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.142757+0000", 2024-09-21T14:58:54.154 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:54.154 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.154 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.154 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:54.154 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.142757+0000", 2024-09-21T14:58:54.154 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:54.154 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.154 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.154 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:54.154 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.142759+0000", 2024-09-21T14:58:54.154 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3800000000000001e-06 2024-09-21T14:58:54.154 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.154 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.155 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:54.155 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.142765+0000", 2024-09-21T14:58:54.155 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7819999999999999e-06 2024-09-21T14:58:54.155 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.155 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.155 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:54.155 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.142770+0000", 2024-09-21T14:58:54.155 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9230000000000001e-06 2024-09-21T14:58:54.155 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.155 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.155 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:54.155 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.157393+0000", 2024-09-21T14:58:54.155 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014623662000000001 2024-09-21T14:58:54.156 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.156 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.156 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:54.156 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.157469+0000", 2024-09-21T14:58:54.156 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5383000000000007e-05 2024-09-21T14:58:54.156 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.156 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.156 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:54.156 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.157664+0000", 2024-09-21T14:58:54.156 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019522100000000001 2024-09-21T14:58:54.156 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:54.156 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:54.156 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:54.157 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.157 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.157 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22866 3.1e 3:7ca947c4:::rbd_data.11366acc894.000000000000000b:19 [read 458752~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:58:54.157 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.142774+0000", 2024-09-21T14:58:54.157 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.994794127, 2024-09-21T14:58:54.157 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.01521982, 2024-09-21T14:58:54.157 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:54.157 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:54.157 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:54.157 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:54.157 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:54.157 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22866 2024-09-21T14:58:54.157 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.158 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:54.158 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.158 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:54.158 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.142774+0000", 2024-09-21T14:58:54.158 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:54.158 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.158 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.158 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:54.158 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.142774+0000", 2024-09-21T14:58:54.158 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:54.158 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.158 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.158 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:54.159 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.142774+0000", 2024-09-21T14:58:54.159 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:54.159 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.159 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.159 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:54.159 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.142776+0000", 2024-09-21T14:58:54.159 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.588e-06 2024-09-21T14:58:54.159 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.159 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.159 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:54.159 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.142783+0000", 2024-09-21T14:58:54.159 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2729999999999999e-06 2024-09-21T14:58:54.159 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.159 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.160 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:54.160 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.142789+0000", 2024-09-21T14:58:54.160 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3160000000000002e-06 2024-09-21T14:58:54.160 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.160 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.160 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:54.160 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.157705+0000", 2024-09-21T14:58:54.160 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014915586999999999 2024-09-21T14:58:54.160 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.160 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.160 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:54.160 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.157792+0000", 2024-09-21T14:58:54.160 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7322000000000002e-05 2024-09-21T14:58:54.160 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.161 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.161 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:54.161 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.157994+0000", 2024-09-21T14:58:54.161 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000202093 2024-09-21T14:58:54.161 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:54.161 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:54.161 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:54.161 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.161 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.161 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22867 3.1e 3:7ca947c4:::rbd_data.11366acc894.000000000000000b:19 [read 524288~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:58:54.161 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.142794+0000", 2024-09-21T14:58:54.161 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.9947743870000001, 2024-09-21T14:58:54.161 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015569725, 2024-09-21T14:58:54.161 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:54.161 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:54.162 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:54.162 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:54.162 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:54.162 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22867 2024-09-21T14:58:54.162 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.162 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:54.162 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.162 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:54.162 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.142794+0000", 2024-09-21T14:58:54.162 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:54.162 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.162 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.162 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:54.162 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.142794+0000", 2024-09-21T14:58:54.163 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:54.163 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.163 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.163 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:54.163 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.142793+0000", 2024-09-21T14:58:54.163 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:54.163 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.163 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.163 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:54.163 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.142801+0000", 2024-09-21T14:58:54.163 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8509999999999999e-06 2024-09-21T14:58:54.163 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.163 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.163 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:54.164 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.142807+0000", 2024-09-21T14:58:54.164 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1210000000000003e-06 2024-09-21T14:58:54.164 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.164 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.164 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:54.164 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.142812+0000", 2024-09-21T14:58:54.164 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2809999999999999e-06 2024-09-21T14:58:54.164 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.164 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.164 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:54.164 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.158021+0000", 2024-09-21T14:58:54.164 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015208038 2024-09-21T14:58:54.164 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.164 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.165 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:54.165 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.158098+0000", 2024-09-21T14:58:54.165 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7888999999999999e-05 2024-09-21T14:58:54.165 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.165 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.165 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:54.165 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.158364+0000", 2024-09-21T14:58:54.165 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00026517400000000002 2024-09-21T14:58:54.165 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:54.165 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:54.165 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:54.165 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.165 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.165 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:23017 3.1e 3:7ca947c4:::rbd_data.11366acc894.000000000000000b:19 [read 229376~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:58:54.166 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.147628+0000", 2024-09-21T14:58:54.166 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.989940367, 2024-09-21T14:58:54.166 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014791732, 2024-09-21T14:58:54.166 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:54.166 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:54.166 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:54.166 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:54.166 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:54.166 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 23017 2024-09-21T14:58:54.166 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.166 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:54.166 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.166 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:54.167 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147628+0000", 2024-09-21T14:58:54.167 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:54.167 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.167 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.167 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:54.167 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147628+0000", 2024-09-21T14:58:54.167 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:54.167 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.167 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.167 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:54.167 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147628+0000", 2024-09-21T14:58:54.167 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:54.168 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.168 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.168 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:54.168 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147635+0000", 2024-09-21T14:58:54.168 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3459999999999996e-06 2024-09-21T14:58:54.168 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.168 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.168 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:54.168 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147642+0000", 2024-09-21T14:58:54.168 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0249999999999997e-06 2024-09-21T14:58:54.168 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.168 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.168 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:54.168 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147647+0000", 2024-09-21T14:58:54.168 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.549e-06 2024-09-21T14:58:54.169 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.169 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.169 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:54.169 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.161946+0000", 2024-09-21T14:58:54.169 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.01429997 2024-09-21T14:58:54.169 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.169 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.169 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:54.169 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.162034+0000", 2024-09-21T14:58:54.169 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7676000000000006e-05 2024-09-21T14:58:54.169 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.169 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.169 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:54.170 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.162420+0000", 2024-09-21T14:58:54.170 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00038544499999999999 2024-09-21T14:58:54.170 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:54.170 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:54.170 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:54.170 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.170 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.170 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:23018 3.1e 3:7ca947c4:::rbd_data.11366acc894.000000000000000b:19 [read 294912~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:58:54.170 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.147650+0000", 2024-09-21T14:58:54.170 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.9899179390000001, 2024-09-21T14:58:54.170 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015515414999999999, 2024-09-21T14:58:54.170 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:54.170 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:54.171 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:54.171 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:54.171 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:54.171 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 23018 2024-09-21T14:58:54.171 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.171 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:54.171 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.171 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:54.171 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147650+0000", 2024-09-21T14:58:54.171 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:54.171 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.171 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.171 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:54.172 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147650+0000", 2024-09-21T14:58:54.172 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:54.172 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.172 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.172 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:54.172 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147650+0000", 2024-09-21T14:58:54.172 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:54.172 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.172 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.172 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:54.172 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147652+0000", 2024-09-21T14:58:54.172 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1469999999999999e-06 2024-09-21T14:58:54.172 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.172 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.173 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:54.173 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147657+0000", 2024-09-21T14:58:54.173 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3449999999999996e-06 2024-09-21T14:58:54.173 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.173 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.173 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:54.173 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147663+0000", 2024-09-21T14:58:54.173 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1309999999999997e-06 2024-09-21T14:58:54.173 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.173 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.173 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:54.173 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.162466+0000", 2024-09-21T14:58:54.173 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014803474 2024-09-21T14:58:54.174 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.174 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.174 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:54.174 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.162562+0000", 2024-09-21T14:58:54.174 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6272000000000003e-05 2024-09-21T14:58:54.174 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.174 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.174 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:54.174 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.163166+0000", 2024-09-21T14:58:54.174 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00060336000000000005 2024-09-21T14:58:54.174 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:54.174 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:54.174 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:54.174 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.175 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.175 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:23019 3.1e 3:7ca947c4:::rbd_data.11366acc894.000000000000000b:19 [read 360448~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:58:54.175 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.147666+0000", 2024-09-21T14:58:54.175 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.989901895, 2024-09-21T14:58:54.175 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015788706, 2024-09-21T14:58:54.175 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:54.175 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:54.175 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:54.175 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:54.175 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:54.175 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 23019 2024-09-21T14:58:54.175 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.175 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:54.176 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.176 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:54.176 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147666+0000", 2024-09-21T14:58:54.176 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:54.176 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.176 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.176 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:54.176 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147666+0000", 2024-09-21T14:58:54.176 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:54.176 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.176 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.176 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:54.177 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147666+0000", 2024-09-21T14:58:54.177 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:54.177 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.177 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.177 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:54.177 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147668+0000", 2024-09-21T14:58:54.177 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1969999999999999e-06 2024-09-21T14:58:54.177 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.177 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.177 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:54.177 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147673+0000", 2024-09-21T14:58:54.177 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2220000000000001e-06 2024-09-21T14:58:54.177 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.178 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.178 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:54.178 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147677+0000", 2024-09-21T14:58:54.178 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9620000000000004e-06 2024-09-21T14:58:54.178 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.178 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.178 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:54.178 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.163200+0000", 2024-09-21T14:58:54.178 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.01552215 2024-09-21T14:58:54.178 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.178 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.178 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:54.178 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.163294+0000", 2024-09-21T14:58:54.179 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4017000000000005e-05 2024-09-21T14:58:54.179 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.179 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.179 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:54.179 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.163455+0000", 2024-09-21T14:58:54.179 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000161478 2024-09-21T14:58:54.179 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:54.179 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:54.179 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:54.179 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.179 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.179 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:23020 3.1e 3:7ca947c4:::rbd_data.11366acc894.000000000000000b:19 [read 425984~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:58:54.179 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.147681+0000", 2024-09-21T14:58:54.179 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.9898871649999998, 2024-09-21T14:58:54.180 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.016190145, 2024-09-21T14:58:54.180 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:54.180 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:54.180 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:54.180 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:54.180 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:54.180 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 23020 2024-09-21T14:58:54.180 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.180 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:54.180 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.180 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:54.180 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147681+0000", 2024-09-21T14:58:54.181 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:54.181 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.181 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.181 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:54.181 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147681+0000", 2024-09-21T14:58:54.181 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:54.181 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.181 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.181 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:54.181 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147681+0000", 2024-09-21T14:58:54.181 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:54.181 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.181 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.181 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:54.182 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147683+0000", 2024-09-21T14:58:54.182 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3470000000000001e-06 2024-09-21T14:58:54.182 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.182 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.182 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:54.182 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147689+0000", 2024-09-21T14:58:54.182 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5849999999999999e-06 2024-09-21T14:58:54.182 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.182 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.182 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:54.182 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147695+0000", 2024-09-21T14:58:54.182 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.964e-06 2024-09-21T14:58:54.182 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.182 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.183 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:54.183 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.163488+0000", 2024-09-21T14:58:54.183 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015793564999999999 2024-09-21T14:58:54.183 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.183 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.183 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:54.183 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.163532+0000", 2024-09-21T14:58:54.183 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3841000000000003e-05 2024-09-21T14:58:54.183 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.183 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.183 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:54.183 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.163871+0000", 2024-09-21T14:58:54.184 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00033914500000000001 2024-09-21T14:58:54.184 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:54.184 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:54.184 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:54.184 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.184 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.184 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:23021 3.1e 3:7ca947c4:::rbd_data.11366acc894.000000000000000b:19 [read 491520~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:58:54.184 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.147698+0000", 2024-09-21T14:58:54.184 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.9898697670000001, 2024-09-21T14:58:54.184 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.016449135, 2024-09-21T14:58:54.184 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:54.184 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:54.184 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:54.184 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:54.185 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:54.185 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 23021 2024-09-21T14:58:54.185 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.185 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:54.185 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.185 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:54.185 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147698+0000", 2024-09-21T14:58:54.185 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:54.185 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.185 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.185 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:54.185 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147698+0000", 2024-09-21T14:58:54.185 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:54.185 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.186 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.186 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:54.186 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147698+0000", 2024-09-21T14:58:54.186 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:54.186 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.186 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.186 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:54.186 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147701+0000", 2024-09-21T14:58:54.186 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6299999999999998e-06 2024-09-21T14:58:54.186 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.186 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.186 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:54.186 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147706+0000", 2024-09-21T14:58:54.187 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3059999999999997e-06 2024-09-21T14:58:54.187 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.187 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.187 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:54.187 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147711+0000", 2024-09-21T14:58:54.187 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0860000000000001e-06 2024-09-21T14:58:54.187 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.187 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.187 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:54.187 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.163897+0000", 2024-09-21T14:58:54.187 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.016186282 2024-09-21T14:58:54.187 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.187 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.187 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:54.188 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.163986+0000", 2024-09-21T14:58:54.188 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8676000000000003e-05 2024-09-21T14:58:54.188 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.188 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.188 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:54.188 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.164148+0000", 2024-09-21T14:58:54.188 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016148000000000001 2024-09-21T14:58:54.188 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:54.188 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:54.188 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:54.188 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.188 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.188 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:23022 3.1e 3:7ca947c4:::rbd_data.11366acc894.000000000000000b:19 [read 557056~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:58:54.189 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.148289+0000", 2024-09-21T14:58:54.189 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.9892788829999999, 2024-09-21T14:58:54.189 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.016062979000000002, 2024-09-21T14:58:54.189 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:54.189 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:54.189 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:54.189 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:54.189 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:54.189 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 23022 2024-09-21T14:58:54.189 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.189 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:54.189 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.189 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:54.189 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.148289+0000", 2024-09-21T14:58:54.190 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:54.190 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.190 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.190 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:54.190 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.148289+0000", 2024-09-21T14:58:54.190 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:54.190 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.190 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.190 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:54.190 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.148288+0000", 2024-09-21T14:58:54.190 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:54.190 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.190 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.190 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:54.191 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.148295+0000", 2024-09-21T14:58:54.191 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8340000000000001e-06 2024-09-21T14:58:54.191 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.191 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.191 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:54.191 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.148304+0000", 2024-09-21T14:58:54.191 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0129999999999999e-06 2024-09-21T14:58:54.191 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.191 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.191 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:54.191 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.148311+0000", 2024-09-21T14:58:54.191 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1439999999999997e-06 2024-09-21T14:58:54.191 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.191 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.192 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:54.192 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.164182+0000", 2024-09-21T14:58:54.192 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015870825000000002 2024-09-21T14:58:54.192 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.192 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.192 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:54.192 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.164229+0000", 2024-09-21T14:58:54.192 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6508999999999998e-05 2024-09-21T14:58:54.192 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.192 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.192 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:54.192 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.164352+0000", 2024-09-21T14:58:54.192 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000123664 2024-09-21T14:58:54.192 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:54.193 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:54.193 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:54.193 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.193 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.193 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:23115 3.1e 3:7ca947c4:::rbd_data.11366acc894.000000000000000b:19 [read 114688~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:58:54.193 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.150277+0000", 2024-09-21T14:58:54.193 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.9872912469999999, 2024-09-21T14:58:54.193 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014686462000000001, 2024-09-21T14:58:54.193 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:54.193 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:54.193 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:54.193 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:54.193 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:54.193 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 23115 2024-09-21T14:58:54.194 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.194 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:54.194 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.194 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:54.194 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150277+0000", 2024-09-21T14:58:54.194 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:54.194 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.194 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.194 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:54.194 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150277+0000", 2024-09-21T14:58:54.194 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:54.194 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.194 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.195 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:54.195 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150277+0000", 2024-09-21T14:58:54.195 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:54.195 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.195 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.195 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:54.195 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150280+0000", 2024-09-21T14:58:54.195 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2349999999999999e-06 2024-09-21T14:58:54.195 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.195 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.195 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:54.195 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150286+0000", 2024-09-21T14:58:54.195 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5969999999999997e-06 2024-09-21T14:58:54.196 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.196 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.196 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:54.196 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150291+0000", 2024-09-21T14:58:54.196 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9359999999999998e-06 2024-09-21T14:58:54.196 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.196 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.196 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:54.196 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.164697+0000", 2024-09-21T14:58:54.196 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014406014999999999 2024-09-21T14:58:54.196 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.196 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.196 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:54.197 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.164784+0000", 2024-09-21T14:58:54.197 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6954999999999997e-05 2024-09-21T14:58:54.197 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.197 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.197 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:54.197 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.164963+0000", 2024-09-21T14:58:54.197 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017918 2024-09-21T14:58:54.197 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:54.197 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:54.197 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:54.197 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.197 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.197 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:23116 3.1e 3:7ca947c4:::rbd_data.11366acc894.000000000000000b:19 [read 180224~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:58:54.197 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.150296+0000", 2024-09-21T14:58:54.198 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.9872725400000002, 2024-09-21T14:58:54.198 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015003996, 2024-09-21T14:58:54.198 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:54.198 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:54.198 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:54.198 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:54.198 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:54.198 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 23116 2024-09-21T14:58:54.198 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.198 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:54.198 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.198 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:54.198 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150296+0000", 2024-09-21T14:58:54.198 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:54.199 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.199 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.199 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:54.199 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150296+0000", 2024-09-21T14:58:54.199 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:54.199 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.199 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.199 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:54.199 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150295+0000", 2024-09-21T14:58:54.199 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:54.199 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.199 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.199 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:54.199 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150298+0000", 2024-09-21T14:58:54.200 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6749999999999998e-06 2024-09-21T14:58:54.200 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.200 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.200 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:54.200 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150304+0000", 2024-09-21T14:58:54.200 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6359999999999999e-06 2024-09-21T14:58:54.200 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.200 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.200 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:54.200 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150308+0000", 2024-09-21T14:58:54.200 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6260000000000003e-06 2024-09-21T14:58:54.200 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.200 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.200 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:54.201 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.165003+0000", 2024-09-21T14:58:54.201 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014694377 2024-09-21T14:58:54.201 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.201 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.201 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:54.201 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.165093+0000", 2024-09-21T14:58:54.201 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0137999999999997e-05 2024-09-21T14:58:54.201 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.201 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.201 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:54.201 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.165300+0000", 2024-09-21T14:58:54.201 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00020688799999999999 2024-09-21T14:58:54.201 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:54.202 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:54.202 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:54.202 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.202 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.202 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:23117 3.1e 3:7ca947c4:::rbd_data.11366acc894.000000000000000b:19 [read 245760~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:58:54.202 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.150312+0000", 2024-09-21T14:58:54.202 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.9872562039999999, 2024-09-21T14:58:54.202 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015236939, 2024-09-21T14:58:54.202 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:54.202 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:54.202 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:54.202 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:54.202 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:54.203 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 23117 2024-09-21T14:58:54.203 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.203 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:54.203 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.203 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:54.203 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150312+0000", 2024-09-21T14:58:54.203 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:54.203 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.203 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.203 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:54.203 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150312+0000", 2024-09-21T14:58:54.203 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:54.203 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.204 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.204 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:54.204 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150312+0000", 2024-09-21T14:58:54.204 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:54.204 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.204 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.204 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:54.204 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150314+0000", 2024-09-21T14:58:54.204 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.154e-06 2024-09-21T14:58:54.204 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.204 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.204 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:54.204 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150320+0000", 2024-09-21T14:58:54.204 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6640000000000003e-06 2024-09-21T14:58:54.205 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.205 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.205 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:54.205 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150324+0000", 2024-09-21T14:58:54.205 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5669999999999996e-06 2024-09-21T14:58:54.205 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.205 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.205 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:54.205 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.165317+0000", 2024-09-21T14:58:54.205 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014993328 2024-09-21T14:58:54.205 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.205 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.205 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:54.205 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.165370+0000", 2024-09-21T14:58:54.206 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2036999999999997e-05 2024-09-21T14:58:54.206 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.206 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.206 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:54.206 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.165549+0000", 2024-09-21T14:58:54.206 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017947500000000001 2024-09-21T14:58:54.206 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:54.206 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:54.206 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:54.206 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.206 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.206 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:23118 3.1e 3:7ca947c4:::rbd_data.11366acc894.000000000000000b:19 [read 311296~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:58:54.206 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.150328+0000", 2024-09-21T14:58:54.206 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.9872400589999999, 2024-09-21T14:58:54.207 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015505385999999999, 2024-09-21T14:58:54.207 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:54.207 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:54.207 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:54.207 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:54.207 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:54.207 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 23118 2024-09-21T14:58:54.207 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.207 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:54.207 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.207 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:54.207 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150328+0000", 2024-09-21T14:58:54.207 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:54.207 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.208 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.208 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:54.208 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150328+0000", 2024-09-21T14:58:54.208 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:54.208 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.208 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.208 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:54.208 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150328+0000", 2024-09-21T14:58:54.208 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:54.208 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.208 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.208 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:54.208 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150330+0000", 2024-09-21T14:58:54.209 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3410000000000001e-06 2024-09-21T14:58:54.209 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.209 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.209 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:54.209 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150336+0000", 2024-09-21T14:58:54.209 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.694e-06 2024-09-21T14:58:54.209 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.209 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.209 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:54.209 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150343+0000", 2024-09-21T14:58:54.209 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3050000000000004e-06 2024-09-21T14:58:54.209 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.209 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.209 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:54.209 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.165607+0000", 2024-09-21T14:58:54.210 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015264534 2024-09-21T14:58:54.210 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.210 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.210 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:54.210 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.165661+0000", 2024-09-21T14:58:54.210 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3949000000000001e-05 2024-09-21T14:58:54.210 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.210 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.210 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:54.210 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.165834+0000", 2024-09-21T14:58:54.210 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000172139 2024-09-21T14:58:54.210 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:54.210 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:54.211 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:54.211 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.211 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.211 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:23119 3.1e 3:7ca947c4:::rbd_data.11366acc894.000000000000000b:19 [read 376832~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:58:54.211 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.150347+0000", 2024-09-21T14:58:54.211 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.9872210400000001, 2024-09-21T14:58:54.211 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015790017, 2024-09-21T14:58:54.211 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:54.211 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:54.211 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:54.211 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:54.211 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:54.211 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 23119 2024-09-21T14:58:54.211 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.211 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:54.212 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.212 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:54.212 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150347+0000", 2024-09-21T14:58:54.212 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:54.212 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.212 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.212 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:54.212 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150347+0000", 2024-09-21T14:58:54.212 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:54.212 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.212 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.212 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:54.212 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150347+0000", 2024-09-21T14:58:54.212 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:54.212 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.213 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.213 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:54.213 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150349+0000", 2024-09-21T14:58:54.213 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1730000000000002e-06 2024-09-21T14:58:54.213 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.213 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.213 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:54.213 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150355+0000", 2024-09-21T14:58:54.213 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9320000000000001e-06 2024-09-21T14:58:54.213 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.213 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.213 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:54.213 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150360+0000", 2024-09-21T14:58:54.214 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9450000000000001e-06 2024-09-21T14:58:54.214 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.214 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.214 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:54.214 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.165867+0000", 2024-09-21T14:58:54.214 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015507191 2024-09-21T14:58:54.214 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.214 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.214 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:54.214 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.165956+0000", 2024-09-21T14:58:54.214 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9235999999999997e-05 2024-09-21T14:58:54.214 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.214 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.214 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:54.214 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.166137+0000", 2024-09-21T14:58:54.215 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018088699999999999 2024-09-21T14:58:54.215 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:54.215 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:54.215 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:54.215 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.215 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.215 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:23120 3.1e 3:7ca947c4:::rbd_data.11366acc894.000000000000000b:19 [read 442368~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:58:54.215 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.150364+0000", 2024-09-21T14:58:54.215 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.9872043650000002, 2024-09-21T14:58:54.215 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.016001277000000001, 2024-09-21T14:58:54.215 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:54.215 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:54.215 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:54.215 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:54.215 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:54.216 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 23120 2024-09-21T14:58:54.216 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.216 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:54.216 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.216 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:54.216 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150364+0000", 2024-09-21T14:58:54.216 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:54.216 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.216 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.216 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:54.216 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150364+0000", 2024-09-21T14:58:54.216 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:54.216 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.216 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.216 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:54.217 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150363+0000", 2024-09-21T14:58:54.217 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:54.217 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.217 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.217 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:54.217 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150366+0000", 2024-09-21T14:58:54.217 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1969999999999999e-06 2024-09-21T14:58:54.217 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.217 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.217 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:54.217 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150371+0000", 2024-09-21T14:58:54.217 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7459999999999997e-06 2024-09-21T14:58:54.217 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.217 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.217 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:54.217 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150376+0000", 2024-09-21T14:58:54.218 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6519999999999997e-06 2024-09-21T14:58:54.218 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.218 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.218 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:54.218 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.166166+0000", 2024-09-21T14:58:54.218 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015790286000000001 2024-09-21T14:58:54.218 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.218 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.218 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:54.218 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.166226+0000", 2024-09-21T14:58:54.218 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9682000000000002e-05 2024-09-21T14:58:54.218 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.218 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.218 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:54.218 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.166365+0000", 2024-09-21T14:58:54.219 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013911800000000001 2024-09-21T14:58:54.219 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:54.219 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:54.219 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:54.219 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.219 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.219 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:23121 3.1e 3:7ca947c4:::rbd_data.11366acc894.000000000000000b:19 [read 507904~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:58:54.219 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.150380+0000", 2024-09-21T14:58:54.219 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.987188234, 2024-09-21T14:58:54.219 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.016302104000000001, 2024-09-21T14:58:54.219 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:54.219 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:54.219 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:54.219 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:54.219 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:54.220 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 23121 2024-09-21T14:58:54.220 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.220 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:54.220 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.220 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:54.220 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150380+0000", 2024-09-21T14:58:54.220 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:54.220 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.220 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.220 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:54.220 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150380+0000", 2024-09-21T14:58:54.220 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:54.220 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.221 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.221 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:54.221 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150380+0000", 2024-09-21T14:58:54.221 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:54.221 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.221 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.221 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:54.221 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150382+0000", 2024-09-21T14:58:54.221 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1720000000000001e-06 2024-09-21T14:58:54.221 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.221 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.221 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:54.221 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150387+0000", 2024-09-21T14:58:54.221 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2569999999999998e-06 2024-09-21T14:58:54.221 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.221 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.221 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:54.222 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150392+0000", 2024-09-21T14:58:54.222 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1100000000000002e-06 2024-09-21T14:58:54.222 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.222 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.222 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:54.222 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.166392+0000", 2024-09-21T14:58:54.222 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015999394 2024-09-21T14:58:54.222 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.222 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.222 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:54.222 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.166461+0000", 2024-09-21T14:58:54.222 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9035999999999994e-05 2024-09-21T14:58:54.222 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.222 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.222 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:54.223 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.166682+0000", 2024-09-21T14:58:54.223 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00022143900000000001 2024-09-21T14:58:54.223 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:54.223 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:54.223 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:54.223 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.223 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.223 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:23122 3.1e 3:7ca947c4:::rbd_data.11366acc894.000000000000000b:19 [read 573440~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:58:54.223 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.150396+0000", 2024-09-21T14:58:54.223 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.9871723110000001, 2024-09-21T14:58:54.223 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.016563735999999999, 2024-09-21T14:58:54.223 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:54.224 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:54.224 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:54.224 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:54.224 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:54.224 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 23122 2024-09-21T14:58:54.224 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.224 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:54.224 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.224 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:54.224 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150396+0000", 2024-09-21T14:58:54.224 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:54.224 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.224 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.224 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:54.224 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150396+0000", 2024-09-21T14:58:54.225 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:54.225 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.225 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.225 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:54.225 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150396+0000", 2024-09-21T14:58:54.225 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:54.225 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.225 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.225 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:54.225 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150398+0000", 2024-09-21T14:58:54.225 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1739999999999999e-06 2024-09-21T14:58:54.225 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.225 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.225 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:54.225 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150403+0000", 2024-09-21T14:58:54.225 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1490000000000002e-06 2024-09-21T14:58:54.226 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.226 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.226 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:54.226 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150408+0000", 2024-09-21T14:58:54.226 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6460000000000001e-06 2024-09-21T14:58:54.226 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.226 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.226 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:54.226 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.166716+0000", 2024-09-21T14:58:54.226 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.016308804999999999 2024-09-21T14:58:54.226 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.226 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.226 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:54.227 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.166792+0000", 2024-09-21T14:58:54.227 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5479e-05 2024-09-21T14:58:54.227 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.227 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.227 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:54.227 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.166960+0000", 2024-09-21T14:58:54.227 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000167799 2024-09-21T14:58:54.227 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:54.227 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:54.227 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:54.227 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:54.227 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:54.227 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:58:54.229 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T14:58:54.229 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:58:54.228+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:58:54.330 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:58:54.329+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:58:54.357 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:58:54.357 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:58:54.357 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T14:58:54.357 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:58:54.367 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T14:58:54.431 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:58:54.429+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:58:54.468 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:58:54.468 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:58:54.468 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T14:58:54.469 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T14:58:54.469 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:58:54.478 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T14:58:54.531 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:58:54.529+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:58:54.588 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:58:54.588 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T14:58:54.588 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T14:58:54.588 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T14:58:54.588 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.588 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:10651 3.12 3:4e9d23af:::rbd_data.1136f5ed5dd6.0000000000000020:head [write 0~81920 [fadvise_sequential+fadvise_nocache],write 90112~8192 [fadvise_sequential+fadvise_nocache],write 122880~122880 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:54.588 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.292476+0000", 2024-09-21T14:58:54.588 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 15.294500323999999, 2024-09-21T14:58:54.588 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012765669, 2024-09-21T14:58:54.588 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:54.588 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:54.589 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:54.589 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:54.589 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:54.589 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 10651 2024-09-21T14:58:54.589 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.589 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:54.589 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.589 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:54.589 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292476+0000", 2024-09-21T14:58:54.589 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:54.589 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.589 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.589 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:54.589 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292476+0000", 2024-09-21T14:58:54.589 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:54.590 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.590 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.590 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:54.590 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292475+0000", 2024-09-21T14:58:54.590 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:54.590 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.590 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.590 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:54.590 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292696+0000", 2024-09-21T14:58:54.590 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000220201 2024-09-21T14:58:54.590 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.590 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.590 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:54.590 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292711+0000", 2024-09-21T14:58:54.591 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5803000000000001e-05 2024-09-21T14:58:54.591 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.591 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.591 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:54.591 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292721+0000", 2024-09-21T14:58:54.591 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6160000000000008e-06 2024-09-21T14:58:54.591 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.591 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.591 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:54.591 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292757+0000", 2024-09-21T14:58:54.591 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6211999999999998e-05 2024-09-21T14:58:54.591 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.591 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.591 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:54.591 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292878+0000", 2024-09-21T14:58:54.592 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000120306 2024-09-21T14:58:54.592 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.592 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.592 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 3", 2024-09-21T14:58:54.592 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.293019+0000", 2024-09-21T14:58:54.592 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014186400000000001 2024-09-21T14:58:54.592 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.592 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.592 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:58:54.592 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.294898+0000", 2024-09-21T14:58:54.592 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001878434 2024-09-21T14:58:54.592 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.592 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.592 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:54.592 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.305162+0000", 2024-09-21T14:58:54.593 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.01026373 2024-09-21T14:58:54.593 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.593 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.593 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:54.593 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.305209+0000", 2024-09-21T14:58:54.593 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7615999999999999e-05 2024-09-21T14:58:54.593 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.593 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.593 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:54.593 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.305242+0000", 2024-09-21T14:58:54.593 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2979999999999999e-05 2024-09-21T14:58:54.593 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:54.593 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:54.593 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:54.593 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.594 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.594 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:11285 3.12 3:4edf73d2:::rbd_data.1136f5ed5dd6.0000000000000039:head [write 90112~32768 [fadvise_sequential+fadvise_nocache],write 139264~24576 [fadvise_sequential+fadvise_nocache],write 180224~24576 [fadvise_sequential+fadvise_nocache],write 221184~40960 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:54.594 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.539470+0000", 2024-09-21T14:58:54.594 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 15.047506572, 2024-09-21T14:58:54.594 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0083360099999999996, 2024-09-21T14:58:54.594 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:54.594 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:54.594 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:54.594 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:54.594 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:54.594 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 11285 2024-09-21T14:58:54.594 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.594 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:54.595 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.595 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:54.595 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.539470+0000", 2024-09-21T14:58:54.595 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:54.595 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.595 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.595 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:54.595 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.539470+0000", 2024-09-21T14:58:54.595 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:54.595 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.595 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.595 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:54.595 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.539469+0000", 2024-09-21T14:58:54.595 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:54.596 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.596 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.596 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:54.596 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.539652+0000", 2024-09-21T14:58:54.596 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000183084 2024-09-21T14:58:54.596 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.596 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.596 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:54.596 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.539666+0000", 2024-09-21T14:58:54.596 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3947e-05 2024-09-21T14:58:54.596 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.596 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.596 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:54.597 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.539673+0000", 2024-09-21T14:58:54.597 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7309999999999996e-06 2024-09-21T14:58:54.597 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.597 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.597 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:54.597 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.539714+0000", 2024-09-21T14:58:54.597 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1539000000000003e-05 2024-09-21T14:58:54.597 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.597 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.597 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:54.597 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.539832+0000", 2024-09-21T14:58:54.597 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011753500000000001 2024-09-21T14:58:54.597 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.597 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.597 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 3", 2024-09-21T14:58:54.597 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.540005+0000", 2024-09-21T14:58:54.598 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000173031 2024-09-21T14:58:54.598 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.598 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.598 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:58:54.598 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.541606+0000", 2024-09-21T14:58:54.598 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001600899 2024-09-21T14:58:54.598 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.598 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.598 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:54.598 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.547709+0000", 2024-09-21T14:58:54.598 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0061027110000000002 2024-09-21T14:58:54.598 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.598 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.598 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:54.598 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.547767+0000", 2024-09-21T14:58:54.599 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8248999999999997e-05 2024-09-21T14:58:54.599 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.599 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.599 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:54.599 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.547806+0000", 2024-09-21T14:58:54.599 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9341000000000002e-05 2024-09-21T14:58:54.599 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:54.599 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:54.599 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:54.599 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.599 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.599 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12189 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 40960~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:54.599 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.618667+0000", 2024-09-21T14:58:54.599 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.968310122, 2024-09-21T14:58:54.599 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0083014549999999993, 2024-09-21T14:58:54.599 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:54.600 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:54.600 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:54.600 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:54.600 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:54.600 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12189 2024-09-21T14:58:54.600 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.600 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:54.600 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.600 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:54.600 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618667+0000", 2024-09-21T14:58:54.600 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:54.600 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.600 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.600 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:54.601 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618667+0000", 2024-09-21T14:58:54.601 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:54.601 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.601 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.601 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:54.601 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618666+0000", 2024-09-21T14:58:54.601 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:54.601 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.601 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.601 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:54.601 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618669+0000", 2024-09-21T14:58:54.601 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.554e-06 2024-09-21T14:58:54.601 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.601 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.601 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:54.602 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618675+0000", 2024-09-21T14:58:54.602 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2570000000000004e-06 2024-09-21T14:58:54.602 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.602 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.602 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:54.602 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618680+0000", 2024-09-21T14:58:54.602 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6539999999999998e-06 2024-09-21T14:58:54.602 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.602 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.602 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:54.602 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.626683+0000", 2024-09-21T14:58:54.602 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0080035880000000007 2024-09-21T14:58:54.602 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.602 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.602 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:54.603 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.626764+0000", 2024-09-21T14:58:54.603 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0672000000000003e-05 2024-09-21T14:58:54.603 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.603 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.603 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:54.603 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.626968+0000", 2024-09-21T14:58:54.603 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00020407499999999999 2024-09-21T14:58:54.603 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:54.603 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:54.603 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:54.603 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.603 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.603 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12193 3.4 3:25437d51:::rbd_data.11363a21d2be.0000000000000025:10 [read 425984~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:54.603 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.618755+0000", 2024-09-21T14:58:54.604 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.968222041000001, 2024-09-21T14:58:54.604 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086461379999999994, 2024-09-21T14:58:54.604 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:54.604 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:54.604 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:54.604 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:54.604 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:54.604 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12193 2024-09-21T14:58:54.604 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.604 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:54.604 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.604 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:54.604 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618755+0000", 2024-09-21T14:58:54.604 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:54.604 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.604 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.605 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:54.605 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618755+0000", 2024-09-21T14:58:54.605 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:54.605 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.605 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.605 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:54.605 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618754+0000", 2024-09-21T14:58:54.605 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:54.605 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.605 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.605 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:54.605 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618759+0000", 2024-09-21T14:58:54.605 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4020000000000004e-06 2024-09-21T14:58:54.605 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.605 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.606 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:54.606 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618767+0000", 2024-09-21T14:58:54.606 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7490000000000004e-06 2024-09-21T14:58:54.606 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.606 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.606 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:54.606 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618773+0000", 2024-09-21T14:58:54.606 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0939999999999996e-06 2024-09-21T14:58:54.606 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.606 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.606 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:54.606 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.627115+0000", 2024-09-21T14:58:54.606 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0083416840000000003 2024-09-21T14:58:54.606 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.606 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.607 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:54.607 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.627214+0000", 2024-09-21T14:58:54.607 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9042000000000005e-05 2024-09-21T14:58:54.607 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.607 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.607 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:54.607 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.627401+0000", 2024-09-21T14:58:54.607 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018677200000000001 2024-09-21T14:58:54.607 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:54.607 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:54.607 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:54.607 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.607 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.608 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12212 3.4 3:21b43b45:::rbd_data.11363a21d2be.000000000000002b:10 [read 237568~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:54.608 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.619922+0000", 2024-09-21T14:58:54.608 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.967055057, 2024-09-21T14:58:54.608 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.008324372, 2024-09-21T14:58:54.608 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:54.608 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:54.608 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:54.608 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:54.608 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:54.608 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12212 2024-09-21T14:58:54.608 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.608 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:54.608 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.608 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:54.608 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619922+0000", 2024-09-21T14:58:54.609 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:54.609 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.609 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.609 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:54.609 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619922+0000", 2024-09-21T14:58:54.609 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:54.609 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.609 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.609 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:54.609 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619921+0000", 2024-09-21T14:58:54.609 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:54.609 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.609 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.609 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:54.609 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619924+0000", 2024-09-21T14:58:54.610 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.999e-06 2024-09-21T14:58:54.610 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.610 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.610 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:54.610 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619934+0000", 2024-09-21T14:58:54.610 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.8120000000000002e-06 2024-09-21T14:58:54.610 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.610 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.610 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:54.610 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619939+0000", 2024-09-21T14:58:54.610 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7729999999999999e-06 2024-09-21T14:58:54.610 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.610 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.610 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:54.611 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628005+0000", 2024-09-21T14:58:54.611 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0080663330000000002 2024-09-21T14:58:54.611 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.611 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.611 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:54.611 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628073+0000", 2024-09-21T14:58:54.611 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7479999999999998e-05 2024-09-21T14:58:54.611 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.611 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.611 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:54.611 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628246+0000", 2024-09-21T14:58:54.611 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017337400000000001 2024-09-21T14:58:54.611 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:54.611 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:54.612 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:54.612 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.612 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.612 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12213 3.4 3:21b43b45:::rbd_data.11363a21d2be.000000000000002b:10 [read 270336~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:54.612 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.619945+0000", 2024-09-21T14:58:54.612 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.967031775000001, 2024-09-21T14:58:54.612 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086195049999999995, 2024-09-21T14:58:54.612 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:54.612 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:54.612 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:54.612 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:54.612 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:54.612 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12213 2024-09-21T14:58:54.612 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.612 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:54.613 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.613 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:54.613 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619945+0000", 2024-09-21T14:58:54.613 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:54.613 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.613 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.613 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:54.613 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619945+0000", 2024-09-21T14:58:54.613 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:54.613 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.613 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.613 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:54.614 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619945+0000", 2024-09-21T14:58:54.614 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:54.614 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.614 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.614 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:54.614 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619947+0000", 2024-09-21T14:58:54.614 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5780000000000001e-06 2024-09-21T14:58:54.614 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.614 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.614 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:54.614 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619957+0000", 2024-09-21T14:58:54.614 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3500000000000003e-06 2024-09-21T14:58:54.614 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.614 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.615 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:54.615 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619964+0000", 2024-09-21T14:58:54.615 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.199e-06 2024-09-21T14:58:54.615 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.615 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.615 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:54.615 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628285+0000", 2024-09-21T14:58:54.615 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0083210939999999994 2024-09-21T14:58:54.615 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.615 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.615 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:54.615 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628361+0000", 2024-09-21T14:58:54.615 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5873999999999999e-05 2024-09-21T14:58:54.616 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.616 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.616 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:54.616 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628565+0000", 2024-09-21T14:58:54.616 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000203799 2024-09-21T14:58:54.616 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:54.616 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:54.616 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:54.616 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.616 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.616 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12214 3.4 3:21b43b45:::rbd_data.11363a21d2be.000000000000002b:10 [read 376832~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:54.616 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.619969+0000", 2024-09-21T14:58:54.616 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.967007774000001, 2024-09-21T14:58:54.617 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0090292089999999998, 2024-09-21T14:58:54.617 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:54.617 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:54.617 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:54.617 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:54.617 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:54.617 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12214 2024-09-21T14:58:54.617 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.617 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:54.617 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.617 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:54.617 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619969+0000", 2024-09-21T14:58:54.617 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:54.618 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.618 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.618 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:54.618 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619969+0000", 2024-09-21T14:58:54.618 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:54.618 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.618 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.618 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:54.618 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619969+0000", 2024-09-21T14:58:54.618 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:54.618 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.618 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.618 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:54.618 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619971+0000", 2024-09-21T14:58:54.619 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1830000000000001e-06 2024-09-21T14:58:54.619 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.619 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.619 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:54.619 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619976+0000", 2024-09-21T14:58:54.619 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7129999999999996e-06 2024-09-21T14:58:54.619 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.619 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.619 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:54.619 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619981+0000", 2024-09-21T14:58:54.619 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2460000000000003e-06 2024-09-21T14:58:54.619 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.619 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.619 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:54.620 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628602+0000", 2024-09-21T14:58:54.620 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086209519999999994 2024-09-21T14:58:54.620 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.620 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.620 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:54.620 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628734+0000", 2024-09-21T14:58:54.620 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000132242 2024-09-21T14:58:54.620 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.620 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.620 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:54.620 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628998+0000", 2024-09-21T14:58:54.620 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00026419799999999999 2024-09-21T14:58:54.620 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:54.621 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:54.621 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:54.621 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.621 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.621 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12215 3.4 3:21b43b45:::rbd_data.11363a21d2be.000000000000002b:10 [read 409600~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:54.621 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.619985+0000", 2024-09-21T14:58:54.621 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.966991708, 2024-09-21T14:58:54.621 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0095457319999999995, 2024-09-21T14:58:54.621 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:54.621 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:54.621 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:54.621 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:54.621 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:54.621 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12215 2024-09-21T14:58:54.621 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.622 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:54.622 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.622 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:54.622 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619985+0000", 2024-09-21T14:58:54.622 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:54.622 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.622 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.622 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:54.622 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619985+0000", 2024-09-21T14:58:54.622 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:54.622 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.622 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.622 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:54.622 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619985+0000", 2024-09-21T14:58:54.622 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:54.623 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.623 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.623 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:54.623 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619987+0000", 2024-09-21T14:58:54.623 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6129999999999998e-06 2024-09-21T14:58:54.623 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.623 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.623 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:54.623 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619993+0000", 2024-09-21T14:58:54.623 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0820000000000004e-06 2024-09-21T14:58:54.623 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.623 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.623 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:54.624 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619998+0000", 2024-09-21T14:58:54.624 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7759999999999997e-06 2024-09-21T14:58:54.624 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.624 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.624 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:54.624 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629022+0000", 2024-09-21T14:58:54.624 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0090237489999999993 2024-09-21T14:58:54.624 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.624 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.624 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:54.624 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629098+0000", 2024-09-21T14:58:54.624 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5735999999999995e-05 2024-09-21T14:58:54.624 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.624 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.625 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:54.625 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629531+0000", 2024-09-21T14:58:54.625 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00043310100000000003 2024-09-21T14:58:54.625 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:54.625 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:54.625 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:54.625 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.625 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.625 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12216 3.4 3:21b43b45:::rbd_data.11363a21d2be.000000000000002b:10 [read 516096~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:54.625 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.620003+0000", 2024-09-21T14:58:54.625 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.966973712, 2024-09-21T14:58:54.625 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0098850169999999994, 2024-09-21T14:58:54.625 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:54.625 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:54.626 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:54.626 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:54.626 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:54.626 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12216 2024-09-21T14:58:54.626 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.626 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:54.626 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.626 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:54.626 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620003+0000", 2024-09-21T14:58:54.626 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:54.626 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.626 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.626 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:54.626 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620003+0000", 2024-09-21T14:58:54.626 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:54.627 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.627 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.627 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:54.627 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620003+0000", 2024-09-21T14:58:54.627 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:54.627 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.627 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.627 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:54.627 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620006+0000", 2024-09-21T14:58:54.627 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4350000000000001e-06 2024-09-21T14:58:54.627 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.627 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.627 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:54.627 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620013+0000", 2024-09-21T14:58:54.628 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4219999999999997e-06 2024-09-21T14:58:54.628 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.628 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.628 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:54.628 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620017+0000", 2024-09-21T14:58:54.628 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.865e-06 2024-09-21T14:58:54.628 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.628 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.628 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:54.628 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629581+0000", 2024-09-21T14:58:54.628 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0095631989999999997 2024-09-21T14:58:54.628 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.628 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.628 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:54.629 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629691+0000", 2024-09-21T14:58:54.629 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000110586 2024-09-21T14:58:54.629 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.629 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.629 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:54.629 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629888+0000", 2024-09-21T14:58:54.629 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019691199999999999 2024-09-21T14:58:54.629 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:54.629 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:54.629 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:54.629 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.629 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.629 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12222 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 49152~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:54.629 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.620027+0000", 2024-09-21T14:58:54.629 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.966949366, 2024-09-21T14:58:54.629 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010228131999999999, 2024-09-21T14:58:54.630 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:54.630 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:54.630 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:54.630 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:54.630 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:54.630 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12222 2024-09-21T14:58:54.630 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.630 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:54.630 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.630 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:54.630 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620027+0000", 2024-09-21T14:58:54.630 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:54.630 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.630 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.631 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:54.631 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620027+0000", 2024-09-21T14:58:54.631 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:54.631 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.631 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.631 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:54.631 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620027+0000", 2024-09-21T14:58:54.631 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:54.631 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.631 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.631 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:54.631 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620030+0000", 2024-09-21T14:58:54.631 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0400000000000001e-06 2024-09-21T14:58:54.631 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.631 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.632 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:54.632 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620037+0000", 2024-09-21T14:58:54.632 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8220000000000001e-06 2024-09-21T14:58:54.632 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.632 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.632 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:54.632 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620042+0000", 2024-09-21T14:58:54.632 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.843e-06 2024-09-21T14:58:54.632 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.632 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.632 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:54.632 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629922+0000", 2024-09-21T14:58:54.632 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0098805100000000003 2024-09-21T14:58:54.633 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.633 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.633 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:54.633 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.630020+0000", 2024-09-21T14:58:54.633 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7596999999999994e-05 2024-09-21T14:58:54.633 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.633 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.633 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:54.633 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.630256+0000", 2024-09-21T14:58:54.633 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00023573299999999999 2024-09-21T14:58:54.633 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:54.633 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:54.633 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:54.633 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.633 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.634 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12253 3.4 3:21b43b45:::rbd_data.11363a21d2be.000000000000002b:10 [read 417792~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:54.634 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.624578+0000", 2024-09-21T14:58:54.634 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.962398997999999, 2024-09-21T14:58:54.634 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0085112079999999993, 2024-09-21T14:58:54.634 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:54.634 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:54.634 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:54.634 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:54.634 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:54.634 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12253 2024-09-21T14:58:54.634 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.634 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:54.634 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.635 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:54.635 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624578+0000", 2024-09-21T14:58:54.635 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:54.635 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.635 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.635 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:54.635 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624578+0000", 2024-09-21T14:58:54.635 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:54.635 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.635 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.635 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:54.635 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624578+0000", 2024-09-21T14:58:54.635 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:54.635 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.636 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.636 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:54.636 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624579+0000", 2024-09-21T14:58:54.636 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.688e-06 2024-09-21T14:58:54.636 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.636 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.636 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:54.636 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624582+0000", 2024-09-21T14:58:54.636 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2260000000000001e-06 2024-09-21T14:58:54.636 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.636 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.636 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:54.636 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624585+0000", 2024-09-21T14:58:54.636 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4820000000000001e-06 2024-09-21T14:58:54.637 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.637 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.637 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:54.637 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.632580+0000", 2024-09-21T14:58:54.637 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0079948959999999996 2024-09-21T14:58:54.637 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.637 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.637 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:54.637 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.632683+0000", 2024-09-21T14:58:54.637 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010308 2024-09-21T14:58:54.637 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.637 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.637 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:54.637 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.633089+0000", 2024-09-21T14:58:54.637 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00040612599999999998 2024-09-21T14:58:54.637 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:54.638 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:54.638 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:54.638 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.638 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.638 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12260 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 57344~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:54.638 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.624587+0000", 2024-09-21T14:58:54.638 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.962389511, 2024-09-21T14:58:54.638 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0088282069999999994, 2024-09-21T14:58:54.638 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:54.638 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:54.638 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:54.638 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:54.638 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:54.639 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12260 2024-09-21T14:58:54.639 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.639 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:54.639 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.639 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:54.639 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624587+0000", 2024-09-21T14:58:54.639 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:54.639 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.639 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.639 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:54.639 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624587+0000", 2024-09-21T14:58:54.639 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:54.639 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.639 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.639 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:54.640 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624587+0000", 2024-09-21T14:58:54.640 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-21T14:58:54.640 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.640 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.640 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:54.640 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624589+0000", 2024-09-21T14:58:54.640 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4339999999999999e-06 2024-09-21T14:58:54.640 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.640 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.640 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:54.640 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624593+0000", 2024-09-21T14:58:54.640 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5859999999999998e-06 2024-09-21T14:58:54.640 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.640 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.640 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:54.640 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624596+0000", 2024-09-21T14:58:54.641 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5830000000000001e-06 2024-09-21T14:58:54.641 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.641 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.641 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:54.641 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.633132+0000", 2024-09-21T14:58:54.641 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.008536666 2024-09-21T14:58:54.641 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.641 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.641 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:54.641 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.633216+0000", 2024-09-21T14:58:54.641 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3387999999999995e-05 2024-09-21T14:58:54.641 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.641 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.641 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:54.642 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.633415+0000", 2024-09-21T14:58:54.642 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019976799999999999 2024-09-21T14:58:54.642 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:54.642 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:54.642 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:54.642 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.642 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.642 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12516 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 245760~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:54.642 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645194+0000", 2024-09-21T14:58:54.642 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.941782974000001, 2024-09-21T14:58:54.642 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0084284990000000008, 2024-09-21T14:58:54.642 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:54.642 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:54.643 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:54.643 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:54.643 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:54.643 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12516 2024-09-21T14:58:54.643 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.643 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:54.643 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.643 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:54.643 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645194+0000", 2024-09-21T14:58:54.643 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:54.643 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.643 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.643 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:54.643 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645194+0000", 2024-09-21T14:58:54.644 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:54.644 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.644 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.644 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:54.644 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645193+0000", 2024-09-21T14:58:54.644 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:54.644 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.644 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.644 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:54.644 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645196+0000", 2024-09-21T14:58:54.644 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1030000000000002e-06 2024-09-21T14:58:54.644 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.644 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.644 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:54.645 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645205+0000", 2024-09-21T14:58:54.645 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6200000000000005e-06 2024-09-21T14:58:54.645 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.645 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.645 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:54.645 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645210+0000", 2024-09-21T14:58:54.645 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6280000000000004e-06 2024-09-21T14:58:54.645 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.645 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.645 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:54.645 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.653305+0000", 2024-09-21T14:58:54.645 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0080957879999999996 2024-09-21T14:58:54.645 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.645 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.645 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:54.646 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.653389+0000", 2024-09-21T14:58:54.646 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3242e-05 2024-09-21T14:58:54.646 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.646 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.646 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:54.646 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.653622+0000", 2024-09-21T14:58:54.646 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00023361599999999999 2024-09-21T14:58:54.646 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:54.646 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:54.646 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:54.646 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.646 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.646 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12517 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 278528~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:54.646 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645214+0000", 2024-09-21T14:58:54.646 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.941762861000001, 2024-09-21T14:58:54.647 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086832899999999998, 2024-09-21T14:58:54.647 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:54.647 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:54.647 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:54.647 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:54.647 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:54.647 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12517 2024-09-21T14:58:54.647 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.647 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:54.647 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.647 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:54.647 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645214+0000", 2024-09-21T14:58:54.647 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:54.647 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.647 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.647 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:54.648 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645214+0000", 2024-09-21T14:58:54.648 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:54.648 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.648 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.648 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:54.648 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645214+0000", 2024-09-21T14:58:54.648 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:54.648 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.648 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.648 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:54.648 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645216+0000", 2024-09-21T14:58:54.648 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5579999999999999e-06 2024-09-21T14:58:54.648 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.648 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.648 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:54.649 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645224+0000", 2024-09-21T14:58:54.649 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2439999999999993e-06 2024-09-21T14:58:54.649 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.649 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.649 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:54.649 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645231+0000", 2024-09-21T14:58:54.649 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1859999999999997e-06 2024-09-21T14:58:54.649 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.649 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.649 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:54.649 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.653651+0000", 2024-09-21T14:58:54.649 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0084204220000000003 2024-09-21T14:58:54.649 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.649 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.649 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:54.650 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.653726+0000", 2024-09-21T14:58:54.650 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5356000000000002e-05 2024-09-21T14:58:54.650 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.650 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.650 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:54.650 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.653897+0000", 2024-09-21T14:58:54.650 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000170848 2024-09-21T14:58:54.650 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:54.650 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:54.650 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:54.650 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.650 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.650 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12518 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 385024~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:54.650 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645235+0000", 2024-09-21T14:58:54.651 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.941741635, 2024-09-21T14:58:54.651 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0089841020000000008, 2024-09-21T14:58:54.651 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:54.651 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:54.651 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:54.651 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:54.651 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:54.651 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12518 2024-09-21T14:58:54.651 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.651 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:54.651 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.651 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:54.651 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645235+0000", 2024-09-21T14:58:54.651 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:54.651 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.652 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.652 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:54.652 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645235+0000", 2024-09-21T14:58:54.652 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:54.652 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.652 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.652 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:54.652 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645235+0000", 2024-09-21T14:58:54.652 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:54.652 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.652 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.652 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:54.652 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645238+0000", 2024-09-21T14:58:54.652 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.869e-06 2024-09-21T14:58:54.652 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.652 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.653 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:54.653 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645244+0000", 2024-09-21T14:58:54.653 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8930000000000002e-06 2024-09-21T14:58:54.653 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.653 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.653 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:54.653 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645249+0000", 2024-09-21T14:58:54.653 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2859999999999999e-06 2024-09-21T14:58:54.653 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.653 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.653 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:54.653 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.653930+0000", 2024-09-21T14:58:54.653 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086815320000000005 2024-09-21T14:58:54.654 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.654 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.654 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:54.654 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654028+0000", 2024-09-21T14:58:54.654 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7578000000000007e-05 2024-09-21T14:58:54.654 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.654 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.654 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:54.654 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654219+0000", 2024-09-21T14:58:54.654 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019132099999999999 2024-09-21T14:58:54.654 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:54.654 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:54.654 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:54.655 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.655 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.655 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12538 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 147456~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:54.655 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645253+0000", 2024-09-21T14:58:54.655 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.941724011, 2024-09-21T14:58:54.655 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0092095360000000008, 2024-09-21T14:58:54.655 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:54.655 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:54.655 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:54.655 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:54.655 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:54.655 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12538 2024-09-21T14:58:54.655 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.655 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:54.656 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.656 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:54.656 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645253+0000", 2024-09-21T14:58:54.656 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:54.656 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.656 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.656 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:54.656 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645253+0000", 2024-09-21T14:58:54.656 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:54.656 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.656 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.656 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:54.656 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645252+0000", 2024-09-21T14:58:54.656 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:54.656 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.657 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.657 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:54.657 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645255+0000", 2024-09-21T14:58:54.657 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2290000000000002e-06 2024-09-21T14:58:54.657 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.657 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.657 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:54.657 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645267+0000", 2024-09-21T14:58:54.657 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2631e-05 2024-09-21T14:58:54.657 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.657 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.657 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:54.657 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645278+0000", 2024-09-21T14:58:54.657 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0847e-05 2024-09-21T14:58:54.658 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.658 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.658 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:54.658 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654251+0000", 2024-09-21T14:58:54.658 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0089731340000000007 2024-09-21T14:58:54.658 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.658 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.658 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:54.658 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654296+0000", 2024-09-21T14:58:54.658 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4353e-05 2024-09-21T14:58:54.658 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.658 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.658 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:54.658 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654462+0000", 2024-09-21T14:58:54.659 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016668499999999999 2024-09-21T14:58:54.659 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:54.659 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:54.659 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:54.659 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.659 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.659 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12539 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 253952~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:54.659 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645284+0000", 2024-09-21T14:58:54.659 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.941692789999999, 2024-09-21T14:58:54.659 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0094524029999999998, 2024-09-21T14:58:54.659 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:54.659 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:54.659 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:54.659 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:54.660 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:54.660 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12539 2024-09-21T14:58:54.660 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.660 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:54.660 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.660 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:54.660 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645284+0000", 2024-09-21T14:58:54.660 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:54.660 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.660 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.660 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:54.660 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645284+0000", 2024-09-21T14:58:54.660 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:54.660 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.660 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.661 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:54.661 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645284+0000", 2024-09-21T14:58:54.661 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:54.661 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.661 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.661 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:54.661 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645286+0000", 2024-09-21T14:58:54.661 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7159999999999999e-06 2024-09-21T14:58:54.661 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.661 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.661 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:54.661 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645292+0000", 2024-09-21T14:58:54.661 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7400000000000001e-06 2024-09-21T14:58:54.661 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.661 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.661 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:54.662 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645297+0000", 2024-09-21T14:58:54.662 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9019999999999998e-06 2024-09-21T14:58:54.662 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.662 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.662 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:54.662 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654490+0000", 2024-09-21T14:58:54.662 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0091932360000000005 2024-09-21T14:58:54.662 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.662 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.662 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:54.662 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654565+0000", 2024-09-21T14:58:54.662 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4657e-05 2024-09-21T14:58:54.662 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.662 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.662 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:54.663 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654736+0000", 2024-09-21T14:58:54.663 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017156199999999999 2024-09-21T14:58:54.663 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:54.663 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:54.663 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:54.663 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.663 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.663 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12540 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 286720~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:54.663 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645302+0000", 2024-09-21T14:58:54.663 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.941674520999999, 2024-09-21T14:58:54.663 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0096603090000000006, 2024-09-21T14:58:54.663 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:54.663 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:54.663 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:54.663 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:54.664 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:54.664 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12540 2024-09-21T14:58:54.664 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.664 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:54.664 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.664 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:54.664 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645302+0000", 2024-09-21T14:58:54.664 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:54.664 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.664 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.664 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:54.664 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645302+0000", 2024-09-21T14:58:54.664 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:54.664 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.664 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.665 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:54.665 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645302+0000", 2024-09-21T14:58:54.665 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:54.665 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.665 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.665 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:54.665 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645305+0000", 2024-09-21T14:58:54.665 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7329999999999999e-06 2024-09-21T14:58:54.665 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.665 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.665 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:54.665 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645316+0000", 2024-09-21T14:58:54.665 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1832e-05 2024-09-21T14:58:54.665 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.665 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.666 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:54.666 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645322+0000", 2024-09-21T14:58:54.666 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.169e-06 2024-09-21T14:58:54.666 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.666 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.666 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:54.666 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654778+0000", 2024-09-21T14:58:54.666 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0094559090000000002 2024-09-21T14:58:54.666 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.666 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.666 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:54.666 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654828+0000", 2024-09-21T14:58:54.666 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0461999999999999e-05 2024-09-21T14:58:54.666 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.666 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.666 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:54.667 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654963+0000", 2024-09-21T14:58:54.667 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000134583 2024-09-21T14:58:54.667 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:54.667 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:54.667 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:54.667 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.667 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.667 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12561 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:54.667 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645742+0000", 2024-09-21T14:58:54.667 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.941235255, 2024-09-21T14:58:54.667 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0093583810000000007, 2024-09-21T14:58:54.667 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:54.667 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:54.667 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:54.667 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:54.667 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:54.668 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12561 2024-09-21T14:58:54.668 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.668 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:54.668 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.668 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:54.668 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645742+0000", 2024-09-21T14:58:54.668 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:54.668 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.668 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.668 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:54.668 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645742+0000", 2024-09-21T14:58:54.668 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:54.668 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.668 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.668 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:54.668 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645741+0000", 2024-09-21T14:58:54.669 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:54.669 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.669 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.669 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:54.669 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645746+0000", 2024-09-21T14:58:54.669 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8379999999999998e-06 2024-09-21T14:58:54.669 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.669 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.669 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:54.669 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645757+0000", 2024-09-21T14:58:54.669 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0705e-05 2024-09-21T14:58:54.669 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.669 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.669 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:54.669 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645765+0000", 2024-09-21T14:58:54.670 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6729999999999992e-06 2024-09-21T14:58:54.670 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.670 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.670 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:54.670 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654993+0000", 2024-09-21T14:58:54.670 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0092287480000000002 2024-09-21T14:58:54.670 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.670 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.670 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:54.670 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.655050+0000", 2024-09-21T14:58:54.670 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6394999999999998e-05 2024-09-21T14:58:54.670 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.670 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.670 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:54.671 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.655100+0000", 2024-09-21T14:58:54.671 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0049000000000001e-05 2024-09-21T14:58:54.671 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:54.671 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:54.671 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:54.671 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.671 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.671 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:15431 3.12 3:4bc7d206:::rbd_data.1136f5ed5dd6.0000000000000051:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:58:54.671 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:41.673445+0000", 2024-09-21T14:58:54.671 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 12.913531921000001, 2024-09-21T14:58:54.671 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086103610000000004, 2024-09-21T14:58:54.671 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:54.671 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:54.671 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:54.671 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:54.672 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:54.672 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 15431 2024-09-21T14:58:54.672 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.672 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:54.672 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.672 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:54.672 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.673445+0000", 2024-09-21T14:58:54.672 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:54.672 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.672 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.672 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:54.672 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.673445+0000", 2024-09-21T14:58:54.672 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:54.672 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.672 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.672 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:54.672 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.673444+0000", 2024-09-21T14:58:54.673 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:54.673 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.673 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.673 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:54.673 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.673451+0000", 2024-09-21T14:58:54.673 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4739999999999999e-06 2024-09-21T14:58:54.673 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.673 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.673 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:54.673 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.673472+0000", 2024-09-21T14:58:54.673 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0687e-05 2024-09-21T14:58:54.673 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.673 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.673 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:54.673 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.673481+0000", 2024-09-21T14:58:54.674 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3349999999999996e-06 2024-09-21T14:58:54.674 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.674 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.674 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:54.674 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.673565+0000", 2024-09-21T14:58:54.674 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4011000000000006e-05 2024-09-21T14:58:54.674 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.674 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.674 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:54.674 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.673650+0000", 2024-09-21T14:58:54.674 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4949000000000003e-05 2024-09-21T14:58:54.674 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.675 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.675 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 3", 2024-09-21T14:58:54.675 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.675014+0000", 2024-09-21T14:58:54.675 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0013636620000000001 2024-09-21T14:58:54.675 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.675 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.675 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:54.675 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.681866+0000", 2024-09-21T14:58:54.675 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0068525640000000002 2024-09-21T14:58:54.675 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.675 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.675 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:58:54.675 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.681885+0000", 2024-09-21T14:58:54.675 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8519999999999999e-05 2024-09-21T14:58:54.675 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.675 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.676 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:54.676 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.681961+0000", 2024-09-21T14:58:54.676 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6248000000000006e-05 2024-09-21T14:58:54.676 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:54.676 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:54.676 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:54.676 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.682055+0000", 2024-09-21T14:58:54.676 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4184000000000005e-05 2024-09-21T14:58:54.676 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:54.676 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:54.676 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:54.676 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:54.676 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:54.676 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:58:54.678 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T14:58:54.678 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:58:54.676+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:58:54.779 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:58:54.777+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:58:54.834 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:58:54.834 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:58:54.834 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T14:58:54.834 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:58:54.844 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T14:58:54.880 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:58:54.877+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:58:54.955 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:58:54.955 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:58:54.956 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T14:58:54.956 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T14:58:54.956 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:58:54.965 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T14:58:54.981 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:58:54.977+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:58:55.072 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:58:55.072 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T14:58:55.072 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T14:58:55.072 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T14:58:55.072 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.073 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:8011 3.b 3:d1b12ef9:::rbd_data.11361bed1f78.0000000000000062:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:55.073 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:35.227212+0000", 2024-09-21T14:58:55.073 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 19.842589187000002, 2024-09-21T14:58:55.073 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0054249789999999999, 2024-09-21T14:58:55.073 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:55.073 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:55.073 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:55.073 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:55.073 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:55.073 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 8011 2024-09-21T14:58:55.073 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.073 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:55.073 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.073 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:55.073 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227212+0000", 2024-09-21T14:58:55.074 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:55.074 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.074 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.074 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:55.074 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227212+0000", 2024-09-21T14:58:55.074 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:55.074 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.074 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.074 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:55.074 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227211+0000", 2024-09-21T14:58:55.074 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:55.074 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.074 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.075 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:55.075 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227214+0000", 2024-09-21T14:58:55.075 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7999999999999999e-06 2024-09-21T14:58:55.075 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.075 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.075 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:55.075 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227220+0000", 2024-09-21T14:58:55.075 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0569999999999998e-06 2024-09-21T14:58:55.075 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.075 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.075 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:55.075 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227223+0000", 2024-09-21T14:58:55.075 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3000000000000002e-06 2024-09-21T14:58:55.075 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.076 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.076 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:55.076 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.232445+0000", 2024-09-21T14:58:55.076 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0052220160000000003 2024-09-21T14:58:55.076 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.076 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.076 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:55.076 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.232535+0000", 2024-09-21T14:58:55.076 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9866999999999999e-05 2024-09-21T14:58:55.076 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.076 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.076 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:55.076 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.232637+0000", 2024-09-21T14:58:55.076 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010155 2024-09-21T14:58:55.076 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:55.077 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:55.077 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:55.077 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.077 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.077 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:8019 3.b 3:d1b12ef9:::rbd_data.11361bed1f78.0000000000000062:d [read 0~32768 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:55.077 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:35.227226+0000", 2024-09-21T14:58:55.077 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 19.842574586000001, 2024-09-21T14:58:55.077 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.005662696, 2024-09-21T14:58:55.077 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:55.077 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:55.077 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:55.077 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:55.077 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:55.077 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 8019 2024-09-21T14:58:55.078 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.078 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:55.078 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.078 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:55.078 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227226+0000", 2024-09-21T14:58:55.078 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:55.078 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.078 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.078 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:55.078 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227226+0000", 2024-09-21T14:58:55.078 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:55.078 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.079 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.079 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:55.079 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227226+0000", 2024-09-21T14:58:55.079 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:55.079 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.079 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.079 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:55.079 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227228+0000", 2024-09-21T14:58:55.079 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5919999999999999e-06 2024-09-21T14:58:55.079 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.079 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.079 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:55.079 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227235+0000", 2024-09-21T14:58:55.079 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2650000000000002e-06 2024-09-21T14:58:55.080 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.080 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.080 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:55.080 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227241+0000", 2024-09-21T14:58:55.080 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9139999999999996e-06 2024-09-21T14:58:55.080 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.080 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.080 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:55.080 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.232671+0000", 2024-09-21T14:58:55.080 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0054301879999999999 2024-09-21T14:58:55.080 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.080 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.080 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:55.080 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.232747+0000", 2024-09-21T14:58:55.081 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5988999999999994e-05 2024-09-21T14:58:55.081 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.081 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.081 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:55.081 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.232889+0000", 2024-09-21T14:58:55.081 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014205199999999999 2024-09-21T14:58:55.081 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:55.081 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:55.081 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:55.081 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.081 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.081 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:8020 3.b 3:d1b12ef9:::rbd_data.11361bed1f78.0000000000000062:d [read 81920~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:55.081 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:35.227246+0000", 2024-09-21T14:58:55.081 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 19.842554993, 2024-09-21T14:58:55.081 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0061395770000000002, 2024-09-21T14:58:55.082 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:55.082 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:55.082 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:55.082 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:55.082 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:55.082 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 8020 2024-09-21T14:58:55.082 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.082 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:55.082 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.082 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:55.082 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227246+0000", 2024-09-21T14:58:55.082 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:55.082 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.082 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.082 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:55.083 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227246+0000", 2024-09-21T14:58:55.083 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:55.083 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.083 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.083 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:55.083 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227245+0000", 2024-09-21T14:58:55.083 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:55.083 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.083 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.083 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:55.083 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227248+0000", 2024-09-21T14:58:55.083 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8119999999999999e-06 2024-09-21T14:58:55.083 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.084 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.084 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:55.084 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227255+0000", 2024-09-21T14:58:55.084 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7549999999999997e-06 2024-09-21T14:58:55.084 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.084 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.084 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:55.084 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227260+0000", 2024-09-21T14:58:55.084 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.271e-06 2024-09-21T14:58:55.084 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.084 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.084 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:55.084 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.232967+0000", 2024-09-21T14:58:55.084 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0057065179999999998 2024-09-21T14:58:55.085 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.085 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.085 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:55.085 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.233108+0000", 2024-09-21T14:58:55.085 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000141123 2024-09-21T14:58:55.085 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.085 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.085 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:55.085 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.233385+0000", 2024-09-21T14:58:55.085 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00027743400000000001 2024-09-21T14:58:55.085 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:55.085 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:55.085 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:55.085 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.086 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.086 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:8034 3.b 3:d1b12ef9:::rbd_data.11361bed1f78.0000000000000062:d [read 32768~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:55.086 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:35.227265+0000", 2024-09-21T14:58:55.086 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 19.842535626, 2024-09-21T14:58:55.086 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.006525968, 2024-09-21T14:58:55.086 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:55.086 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:55.086 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:55.086 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:55.086 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:55.086 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 8034 2024-09-21T14:58:55.086 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.086 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:55.087 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.087 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:55.087 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227265+0000", 2024-09-21T14:58:55.087 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:55.087 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.087 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.087 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:55.087 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227265+0000", 2024-09-21T14:58:55.087 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:55.087 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.087 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.087 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:55.087 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227265+0000", 2024-09-21T14:58:55.087 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:55.087 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.088 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.088 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:55.088 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227268+0000", 2024-09-21T14:58:55.088 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8990000000000001e-06 2024-09-21T14:58:55.088 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.088 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.088 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:55.088 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227274+0000", 2024-09-21T14:58:55.088 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4470000000000003e-06 2024-09-21T14:58:55.088 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.088 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.088 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:55.088 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227288+0000", 2024-09-21T14:58:55.088 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3550000000000001e-05 2024-09-21T14:58:55.089 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.089 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.089 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:55.089 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.233418+0000", 2024-09-21T14:58:55.089 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0061307870000000004 2024-09-21T14:58:55.089 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.089 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.089 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:55.089 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.233553+0000", 2024-09-21T14:58:55.089 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013475600000000001 2024-09-21T14:58:55.089 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.089 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.089 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:55.089 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.233791+0000", 2024-09-21T14:58:55.089 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00023788499999999999 2024-09-21T14:58:55.090 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:55.090 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:55.090 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:55.090 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.090 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.090 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:8035 3.b 3:d1b12ef9:::rbd_data.11361bed1f78.0000000000000062:d [read 98304~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:55.090 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:35.227295+0000", 2024-09-21T14:58:55.090 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 19.842505420999998, 2024-09-21T14:58:55.090 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0069673820000000003, 2024-09-21T14:58:55.090 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:55.090 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:55.090 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:55.090 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:55.091 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:55.091 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 8035 2024-09-21T14:58:55.091 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.091 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:55.091 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.091 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:55.091 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227295+0000", 2024-09-21T14:58:55.091 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:55.091 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.091 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.091 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:55.091 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227295+0000", 2024-09-21T14:58:55.091 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:55.091 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.091 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.092 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:55.092 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227295+0000", 2024-09-21T14:58:55.092 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:55.092 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.092 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.092 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:55.092 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227298+0000", 2024-09-21T14:58:55.092 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2339999999999999e-06 2024-09-21T14:58:55.092 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.092 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.092 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:55.092 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227305+0000", 2024-09-21T14:58:55.092 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0389999999999999e-06 2024-09-21T14:58:55.092 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.092 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.093 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:55.093 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227311+0000", 2024-09-21T14:58:55.093 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8470000000000001e-06 2024-09-21T14:58:55.093 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.093 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.093 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:55.093 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.233824+0000", 2024-09-21T14:58:55.093 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0065132920000000004 2024-09-21T14:58:55.093 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.093 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.093 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:55.093 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.233922+0000", 2024-09-21T14:58:55.093 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.8073999999999996e-05 2024-09-21T14:58:55.093 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.093 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.094 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:55.094 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.234263+0000", 2024-09-21T14:58:55.094 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00034027999999999999 2024-09-21T14:58:55.094 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:55.094 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:55.094 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:55.094 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.094 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.094 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:8048 3.b 3:d1b12ef9:::rbd_data.11361bed1f78.0000000000000062:d [read 49152~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:55.094 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:35.227316+0000", 2024-09-21T14:58:55.094 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 19.842484629000001, 2024-09-21T14:58:55.094 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0076197799999999996, 2024-09-21T14:58:55.094 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:55.094 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:55.095 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:55.095 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:55.095 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:55.095 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 8048 2024-09-21T14:58:55.095 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.095 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:55.095 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.095 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:55.095 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227316+0000", 2024-09-21T14:58:55.095 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:55.095 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.095 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.095 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:55.096 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227316+0000", 2024-09-21T14:58:55.096 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:55.096 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.096 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.096 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:55.096 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227316+0000", 2024-09-21T14:58:55.096 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:55.096 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.096 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.096 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:55.096 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227324+0000", 2024-09-21T14:58:55.096 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1009999999999992e-06 2024-09-21T14:58:55.096 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.097 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.097 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:55.097 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227346+0000", 2024-09-21T14:58:55.097 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2347000000000001e-05 2024-09-21T14:58:55.097 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.097 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.097 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:55.097 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227360+0000", 2024-09-21T14:58:55.097 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.403e-05 2024-09-21T14:58:55.097 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.097 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.097 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:55.097 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.234320+0000", 2024-09-21T14:58:55.097 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0069597019999999999 2024-09-21T14:58:55.097 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.097 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.098 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:55.098 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.234730+0000", 2024-09-21T14:58:55.098 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00041006300000000001 2024-09-21T14:58:55.098 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.098 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.098 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:55.098 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.234936+0000", 2024-09-21T14:58:55.098 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00020586700000000001 2024-09-21T14:58:55.098 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:55.098 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:55.098 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:55.098 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.098 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.098 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:8049 3.b 3:d1b12ef9:::rbd_data.11361bed1f78.0000000000000062:d [read 114688~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:55.098 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:35.227371+0000", 2024-09-21T14:58:55.099 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 19.842429295999999, 2024-09-21T14:58:55.099 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0078533609999999997, 2024-09-21T14:58:55.099 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:55.099 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:55.099 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:55.099 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:55.099 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:55.099 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 8049 2024-09-21T14:58:55.099 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.099 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:55.099 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.099 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:55.099 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227371+0000", 2024-09-21T14:58:55.099 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:55.099 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.099 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.100 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:55.100 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227371+0000", 2024-09-21T14:58:55.100 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:55.100 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.100 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.100 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:55.100 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227370+0000", 2024-09-21T14:58:55.100 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:55.100 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.100 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.100 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:55.100 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227378+0000", 2024-09-21T14:58:55.100 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.379e-06 2024-09-21T14:58:55.100 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.101 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.101 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:55.101 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227388+0000", 2024-09-21T14:58:55.101 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0175e-05 2024-09-21T14:58:55.101 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.101 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.101 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:55.101 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227393+0000", 2024-09-21T14:58:55.101 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6690000000000003e-06 2024-09-21T14:58:55.101 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.101 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.101 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:55.101 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.234964+0000", 2024-09-21T14:58:55.101 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0075705319999999996 2024-09-21T14:58:55.101 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.101 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.102 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:55.102 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.235041+0000", 2024-09-21T14:58:55.102 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7170999999999997e-05 2024-09-21T14:58:55.102 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.102 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.102 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:55.102 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.235225+0000", 2024-09-21T14:58:55.102 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000183735 2024-09-21T14:58:55.102 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:55.102 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:55.102 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:55.102 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.102 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.103 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:15174 3.b 3:d11a5903:::rbd_data.11361bed1f78.0000000000000093:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-21T14:58:55.103 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:40.290309+0000", 2024-09-21T14:58:55.103 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.779491341, 2024-09-21T14:58:55.103 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0060508960000000001, 2024-09-21T14:58:55.103 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:55.103 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:58:55.103 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:55.103 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:55.103 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:55.103 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 15174 2024-09-21T14:58:55.103 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.103 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:55.103 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.103 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:55.103 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:40.290309+0000", 2024-09-21T14:58:55.104 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:55.104 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.104 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.104 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:55.104 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:40.290309+0000", 2024-09-21T14:58:55.104 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:55.104 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.104 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.104 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:55.104 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:40.290309+0000", 2024-09-21T14:58:55.104 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:55.104 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.104 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.104 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:55.104 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:40.290314+0000", 2024-09-21T14:58:55.105 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5219999999999998e-06 2024-09-21T14:58:55.105 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.105 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.105 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:55.105 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:40.290330+0000", 2024-09-21T14:58:55.105 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5511000000000001e-05 2024-09-21T14:58:55.105 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.105 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.105 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:55.105 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:40.290341+0000", 2024-09-21T14:58:55.105 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1729000000000001e-05 2024-09-21T14:58:55.105 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.105 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.105 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:55.105 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:40.296093+0000", 2024-09-21T14:58:55.106 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.005751847 2024-09-21T14:58:55.106 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.106 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.106 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:55.106 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:40.296360+0000", 2024-09-21T14:58:55.106 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00026712099999999999 2024-09-21T14:58:55.106 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:55.106 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:55.106 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:55.106 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.106 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.106 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:19869 3.b 3:d7ca7d68:::rbd_data.11363a21d2be.000000000000002d:head [sparse-read 0~393216] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T14:58:55.106 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:44.340896+0000", 2024-09-21T14:58:55.106 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 10.728904899, 2024-09-21T14:58:55.106 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0071749170000000003, 2024-09-21T14:58:55.107 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:55.107 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:55.107 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:55.107 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:55.107 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:55.107 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 19869 2024-09-21T14:58:55.107 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.107 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:55.107 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.107 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:55.107 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340896+0000", 2024-09-21T14:58:55.107 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:55.107 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.107 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.107 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:55.108 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340896+0000", 2024-09-21T14:58:55.108 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:55.108 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.108 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.108 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:55.108 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340894+0000", 2024-09-21T14:58:55.108 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:55.108 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.108 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.108 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:55.108 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340903+0000", 2024-09-21T14:58:55.108 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0259999999999996e-06 2024-09-21T14:58:55.108 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.108 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.108 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:55.108 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340923+0000", 2024-09-21T14:58:55.109 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0013999999999998e-05 2024-09-21T14:58:55.109 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.109 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.109 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:55.109 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340938+0000", 2024-09-21T14:58:55.109 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4569000000000001e-05 2024-09-21T14:58:55.109 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.109 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.109 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:55.109 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342092+0000", 2024-09-21T14:58:55.109 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00115354 2024-09-21T14:58:55.109 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.109 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.109 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:55.110 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342290+0000", 2024-09-21T14:58:55.110 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019834299999999999 2024-09-21T14:58:55.110 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.110 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.110 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:55.110 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.348071+0000", 2024-09-21T14:58:55.110 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.005780861 2024-09-21T14:58:55.110 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:55.110 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:55.110 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:55.110 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.110 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.110 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:19872 3.b 3:d38ac3f6:::rbd_data.11363a21d2be.0000000000000030:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T14:58:55.110 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:44.340951+0000", 2024-09-21T14:58:55.111 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 10.728849739999999, 2024-09-21T14:58:55.111 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0069228680000000004, 2024-09-21T14:58:55.111 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:55.111 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:55.111 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:55.111 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:55.111 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:55.111 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 19872 2024-09-21T14:58:55.111 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.111 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:55.111 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.111 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:55.111 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340951+0000", 2024-09-21T14:58:55.112 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:55.112 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.112 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.112 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:55.112 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340951+0000", 2024-09-21T14:58:55.112 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:55.112 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.112 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.112 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:55.112 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340950+0000", 2024-09-21T14:58:55.112 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:55.112 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.112 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.112 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:55.112 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340957+0000", 2024-09-21T14:58:55.113 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8430000000000004e-06 2024-09-21T14:58:55.113 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.113 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.113 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:55.113 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340974+0000", 2024-09-21T14:58:55.113 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7082999999999999e-05 2024-09-21T14:58:55.113 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.113 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.113 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:55.113 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340987+0000", 2024-09-21T14:58:55.113 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2763e-05 2024-09-21T14:58:55.113 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.113 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.114 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:55.114 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.347149+0000", 2024-09-21T14:58:55.114 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0061626600000000004 2024-09-21T14:58:55.114 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.114 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.114 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:55.114 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.347354+0000", 2024-09-21T14:58:55.114 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00020441799999999999 2024-09-21T14:58:55.114 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.114 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.114 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:55.114 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.347874+0000", 2024-09-21T14:58:55.114 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00052019399999999997 2024-09-21T14:58:55.115 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:55.115 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:55.115 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:55.115 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.115 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.115 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:19877 3.b 3:d5fc5a5a:::rbd_data.11363a21d2be.0000000000000035:head [sparse-read 0~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T14:58:55.115 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:44.340998+0000", 2024-09-21T14:58:55.115 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 10.728802904, 2024-09-21T14:58:55.115 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0073322129999999998, 2024-09-21T14:58:55.115 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:55.115 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:55.115 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:55.115 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:55.115 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:55.116 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 19877 2024-09-21T14:58:55.116 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.116 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:55.116 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.116 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:55.116 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340998+0000", 2024-09-21T14:58:55.116 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:55.116 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.116 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.116 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:55.116 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340998+0000", 2024-09-21T14:58:55.116 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:55.116 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.117 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.117 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:55.117 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340997+0000", 2024-09-21T14:58:55.117 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:55.117 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.117 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.117 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:55.117 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.341003+0000", 2024-09-21T14:58:55.117 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0070000000000003e-06 2024-09-21T14:58:55.117 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.117 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.117 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:55.117 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.341018+0000", 2024-09-21T14:58:55.117 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5472999999999999e-05 2024-09-21T14:58:55.118 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.118 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.118 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:55.118 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.341040+0000", 2024-09-21T14:58:55.118 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1568000000000001e-05 2024-09-21T14:58:55.118 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.118 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.118 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:55.118 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.347901+0000", 2024-09-21T14:58:55.118 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0068608300000000001 2024-09-21T14:58:55.118 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.118 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.118 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:55.119 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.347973+0000", 2024-09-21T14:58:55.119 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2327000000000001e-05 2024-09-21T14:58:55.119 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.119 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.119 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:55.119 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.348330+0000", 2024-09-21T14:58:55.119 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00035695799999999999 2024-09-21T14:58:55.119 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:55.119 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:55.119 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:55.119 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.119 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.119 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:19941 3.b 3:d7b21c30:::rbd_data.11363a21d2be.0000000000000075:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T14:58:55.119 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:44.342141+0000", 2024-09-21T14:58:55.120 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 10.727659337, 2024-09-21T14:58:55.120 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0063087439999999998, 2024-09-21T14:58:55.120 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:55.120 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:58:55.120 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:55.120 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:55.120 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:55.120 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 19941 2024-09-21T14:58:55.120 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.120 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:55.120 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.120 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:55.120 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342141+0000", 2024-09-21T14:58:55.121 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:55.121 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.121 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.121 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:55.121 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342141+0000", 2024-09-21T14:58:55.121 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:55.121 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.121 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.121 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:55.121 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342139+0000", 2024-09-21T14:58:55.121 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T14:58:55.121 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.121 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.121 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:55.122 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342151+0000", 2024-09-21T14:58:55.122 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1242e-05 2024-09-21T14:58:55.122 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.122 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.122 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:55.122 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342175+0000", 2024-09-21T14:58:55.122 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4491e-05 2024-09-21T14:58:55.122 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.122 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.122 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:55.122 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342191+0000", 2024-09-21T14:58:55.122 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6288000000000001e-05 2024-09-21T14:58:55.122 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.123 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.123 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:55.123 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.348358+0000", 2024-09-21T14:58:55.123 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0061665449999999998 2024-09-21T14:58:55.123 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.123 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.123 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:55.123 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.348450+0000", 2024-09-21T14:58:55.123 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2245999999999997e-05 2024-09-21T14:58:55.123 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:55.123 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:55.123 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:55.123 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.123 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.124 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:19998 3.b 3:d4c6ec57:::rbd_data.11363a21d2be.00000000000000ae:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T14:58:55.124 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:44.342204+0000", 2024-09-21T14:58:55.124 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 10.727597133, 2024-09-21T14:58:55.124 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0063531860000000002, 2024-09-21T14:58:55.124 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:55.124 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:58:55.124 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:55.124 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:55.124 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:55.124 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 19998 2024-09-21T14:58:55.124 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.124 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:55.124 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.124 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:55.124 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342204+0000", 2024-09-21T14:58:55.125 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:55.125 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.125 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.125 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:55.125 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342204+0000", 2024-09-21T14:58:55.125 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:55.125 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.125 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.125 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:55.125 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342203+0000", 2024-09-21T14:58:55.125 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:55.125 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.125 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.125 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:55.126 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342209+0000", 2024-09-21T14:58:55.126 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1739999999999997e-06 2024-09-21T14:58:55.126 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.126 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.126 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:55.126 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342223+0000", 2024-09-21T14:58:55.126 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4065e-05 2024-09-21T14:58:55.126 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.126 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.126 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:55.126 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342236+0000", 2024-09-21T14:58:55.126 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2537e-05 2024-09-21T14:58:55.126 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.126 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.127 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:55.127 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.348476+0000", 2024-09-21T14:58:55.127 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0062405760000000003 2024-09-21T14:58:55.127 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.127 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.127 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:55.127 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.348557+0000", 2024-09-21T14:58:55.127 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0716000000000004e-05 2024-09-21T14:58:55.127 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:55.127 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:55.127 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:55.127 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.127 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.128 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:19999 3.b 3:d4069488:::rbd_data.11363a21d2be.00000000000000af:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T14:58:55.128 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:44.342246+0000", 2024-09-21T14:58:55.128 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 10.727554968, 2024-09-21T14:58:55.128 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0064139210000000004, 2024-09-21T14:58:55.128 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:55.128 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:58:55.128 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:55.128 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:55.128 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:55.128 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 19999 2024-09-21T14:58:55.128 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.128 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:55.128 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.128 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:55.129 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342246+0000", 2024-09-21T14:58:55.129 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:55.129 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.129 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.129 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:55.129 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342246+0000", 2024-09-21T14:58:55.129 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:55.129 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.129 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.129 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:55.129 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342245+0000", 2024-09-21T14:58:55.129 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:55.129 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.129 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.129 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:55.130 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342251+0000", 2024-09-21T14:58:55.130 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8660000000000002e-06 2024-09-21T14:58:55.130 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.130 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.130 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:55.130 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342264+0000", 2024-09-21T14:58:55.130 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3716999999999999e-05 2024-09-21T14:58:55.130 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.130 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.130 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:55.130 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342275+0000", 2024-09-21T14:58:55.130 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1071e-05 2024-09-21T14:58:55.130 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.131 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.131 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:55.131 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.348581+0000", 2024-09-21T14:58:55.131 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0063053730000000004 2024-09-21T14:58:55.131 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.131 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.131 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:55.131 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.348660+0000", 2024-09-21T14:58:55.131 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8854000000000001e-05 2024-09-21T14:58:55.131 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:55.131 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:55.131 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:55.131 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.132 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.132 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:20407 3.b 3:d6a6f68e:::rbd_object_map.1136c84d9a13:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-21T14:58:55.132 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:45.012351+0000", 2024-09-21T14:58:55.132 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 10.057449371000001, 2024-09-21T14:58:55.132 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0056810080000000004, 2024-09-21T14:58:55.132 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:55.132 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:55.132 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:55.132 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:55.132 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:55.132 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 20407 2024-09-21T14:58:55.132 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.132 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:55.132 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.133 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:55.133 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.012351+0000", 2024-09-21T14:58:55.133 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:55.133 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.133 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.133 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:55.133 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.012351+0000", 2024-09-21T14:58:55.133 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:55.133 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.133 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.133 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:55.133 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.012350+0000", 2024-09-21T14:58:55.133 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:55.133 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.134 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.134 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:55.134 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.012360+0000", 2024-09-21T14:58:55.134 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0311e-05 2024-09-21T14:58:55.134 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.134 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.134 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:55.134 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.012383+0000", 2024-09-21T14:58:55.134 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2408e-05 2024-09-21T14:58:55.134 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.134 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.134 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:55.134 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.012398+0000", 2024-09-21T14:58:55.135 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4919999999999999e-05 2024-09-21T14:58:55.135 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.135 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.135 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:55.135 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.014378+0000", 2024-09-21T14:58:55.135 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0019804890000000002 2024-09-21T14:58:55.135 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.135 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.135 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:55.135 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.014640+0000", 2024-09-21T14:58:55.135 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000261434 2024-09-21T14:58:55.135 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.135 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.135 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T14:58:55.136 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.015591+0000", 2024-09-21T14:58:55.136 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00095092199999999999 2024-09-21T14:58:55.136 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.136 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.136 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:58:55.136 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.017084+0000", 2024-09-21T14:58:55.136 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0014932089999999999 2024-09-21T14:58:55.136 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.136 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.136 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:55.136 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.017850+0000", 2024-09-21T14:58:55.136 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00076654999999999996 2024-09-21T14:58:55.136 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.136 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.136 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:55.137 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.017928+0000", 2024-09-21T14:58:55.137 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7811000000000004e-05 2024-09-21T14:58:55.137 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.137 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.137 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:55.137 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.018032+0000", 2024-09-21T14:58:55.137 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000104248 2024-09-21T14:58:55.137 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:55.137 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:55.137 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:55.137 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.137 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.137 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:20440 3.b 3:d6a6f68e:::rbd_object_map.1136c84d9a13:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-21T14:58:55.137 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:45.106428+0000", 2024-09-21T14:58:55.138 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.9633731220000001, 2024-09-21T14:58:55.138 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0076349679999999998, 2024-09-21T14:58:55.138 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:55.138 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:55.138 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:55.138 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:55.138 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:55.138 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 20440 2024-09-21T14:58:55.138 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.138 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:55.138 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.138 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:55.138 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.106428+0000", 2024-09-21T14:58:55.139 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:55.139 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.139 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.139 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:55.139 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.106428+0000", 2024-09-21T14:58:55.139 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:55.139 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.139 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.139 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:55.139 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.106425+0000", 2024-09-21T14:58:55.139 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-21T14:58:55.139 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.139 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.140 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:55.140 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.106442+0000", 2024-09-21T14:58:55.140 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7237000000000001e-05 2024-09-21T14:58:55.140 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.140 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.140 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:55.140 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.106476+0000", 2024-09-21T14:58:55.140 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4004999999999997e-05 2024-09-21T14:58:55.140 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.140 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.140 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:55.140 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.106491+0000", 2024-09-21T14:58:55.140 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4865000000000001e-05 2024-09-21T14:58:55.140 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.140 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.141 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:55.141 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.106602+0000", 2024-09-21T14:58:55.141 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011113200000000001 2024-09-21T14:58:55.141 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.141 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.141 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op must wait for map", 2024-09-21T14:58:55.141 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.106611+0000", 2024-09-21T14:58:55.141 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9199999999999993e-06 2024-09-21T14:58:55.141 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.141 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.141 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:55.141 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.110475+0000", 2024-09-21T14:58:55.141 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0038633259999999998 2024-09-21T14:58:55.141 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.141 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.142 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:55.142 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.110622+0000", 2024-09-21T14:58:55.142 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014734200000000001 2024-09-21T14:58:55.142 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.142 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.142 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T14:58:55.142 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.111136+0000", 2024-09-21T14:58:55.142 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00051382299999999995 2024-09-21T14:58:55.142 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.142 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.142 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:58:55.142 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.112514+0000", 2024-09-21T14:58:55.142 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001378287 2024-09-21T14:58:55.142 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.143 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.143 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:55.143 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.113830+0000", 2024-09-21T14:58:55.143 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0013157939999999999 2024-09-21T14:58:55.143 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.143 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.143 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:55.143 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.113969+0000", 2024-09-21T14:58:55.143 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000138849 2024-09-21T14:58:55.143 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.143 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.143 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:55.143 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.114063+0000", 2024-09-21T14:58:55.143 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3999999999999994e-05 2024-09-21T14:58:55.144 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:55.144 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:55.144 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:55.144 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.144 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.144 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:20460 3.b 3:d6a6f68e:::rbd_object_map.1136c84d9a13:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-21T14:58:55.144 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:45.150183+0000", 2024-09-21T14:58:55.144 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.9196173039999991, 2024-09-21T14:58:55.144 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0063818800000000004, 2024-09-21T14:58:55.144 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:55.144 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:55.144 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:55.144 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:55.145 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:55.145 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 20460 2024-09-21T14:58:55.145 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.145 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:55.145 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.145 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:55.145 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150183+0000", 2024-09-21T14:58:55.145 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:55.145 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.145 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.145 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:55.145 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150183+0000", 2024-09-21T14:58:55.145 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:55.145 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.145 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.145 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:55.146 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150180+0000", 2024-09-21T14:58:55.146 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999971 2024-09-21T14:58:55.146 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.146 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.146 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:55.146 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150211+0000", 2024-09-21T14:58:55.146 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0993e-05 2024-09-21T14:58:55.146 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.146 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.146 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:55.146 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150241+0000", 2024-09-21T14:58:55.146 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9589000000000001e-05 2024-09-21T14:58:55.146 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.146 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.147 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:55.147 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150261+0000", 2024-09-21T14:58:55.147 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.967e-05 2024-09-21T14:58:55.147 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.147 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.147 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:55.147 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150411+0000", 2024-09-21T14:58:55.147 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015004300000000001 2024-09-21T14:58:55.147 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.147 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.147 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:55.147 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150598+0000", 2024-09-21T14:58:55.147 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018712399999999999 2024-09-21T14:58:55.147 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.148 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.148 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T14:58:55.148 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.151390+0000", 2024-09-21T14:58:55.148 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00079194599999999997 2024-09-21T14:58:55.148 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.148 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.148 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:55.148 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.156302+0000", 2024-09-21T14:58:55.148 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0049123960000000003 2024-09-21T14:58:55.148 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.148 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.148 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:58:55.148 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.156435+0000", 2024-09-21T14:58:55.148 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013287300000000001 2024-09-21T14:58:55.148 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.148 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.149 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:55.149 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.156508+0000", 2024-09-21T14:58:55.149 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2504999999999998e-05 2024-09-21T14:58:55.149 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.149 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.149 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:55.149 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.156565+0000", 2024-09-21T14:58:55.149 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7773999999999999e-05 2024-09-21T14:58:55.149 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:55.149 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:55.149 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:55.149 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.149 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.149 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:20461 3.b 3:d6a6f68e:::rbd_object_map.1136c84d9a13:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-21T14:58:55.149 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:45.150284+0000", 2024-09-21T14:58:55.150 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.9195164830000007, 2024-09-21T14:58:55.150 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0064029040000000001, 2024-09-21T14:58:55.150 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:55.150 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:55.150 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:55.150 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:55.150 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:55.150 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 20461 2024-09-21T14:58:55.150 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.150 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:55.150 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.150 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:55.150 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150284+0000", 2024-09-21T14:58:55.150 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:55.150 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.151 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.151 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:55.151 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150284+0000", 2024-09-21T14:58:55.151 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:55.151 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.151 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.151 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:55.151 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150283+0000", 2024-09-21T14:58:55.151 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:55.151 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.151 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.151 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:55.151 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150289+0000", 2024-09-21T14:58:55.151 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8799999999999996e-06 2024-09-21T14:58:55.151 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.151 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.152 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:55.152 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150319+0000", 2024-09-21T14:58:55.152 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9828999999999999e-05 2024-09-21T14:58:55.152 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.152 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.152 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:55.152 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150328+0000", 2024-09-21T14:58:55.152 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4770000000000003e-06 2024-09-21T14:58:55.152 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.152 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.152 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:55.152 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.152355+0000", 2024-09-21T14:58:55.152 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0020279870000000002 2024-09-21T14:58:55.152 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.152 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.153 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:55.153 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.152652+0000", 2024-09-21T14:58:55.153 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00029681 2024-09-21T14:58:55.153 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.153 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.153 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T14:58:55.153 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.153318+0000", 2024-09-21T14:58:55.153 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000666101 2024-09-21T14:58:55.153 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.153 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.153 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:55.153 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.156416+0000", 2024-09-21T14:58:55.153 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0030980729999999998 2024-09-21T14:58:55.153 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.153 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.154 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:58:55.154 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.156577+0000", 2024-09-21T14:58:55.154 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016049700000000001 2024-09-21T14:58:55.154 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.154 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.154 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:55.154 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.156628+0000", 2024-09-21T14:58:55.154 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0642000000000001e-05 2024-09-21T14:58:55.154 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.154 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.154 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:55.154 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.156687+0000", 2024-09-21T14:58:55.154 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9571000000000003e-05 2024-09-21T14:58:55.154 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:55.155 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:55.155 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:55.155 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.155 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.155 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:20462 3.b 3:d6a6f68e:::rbd_object_map.1136c84d9a13:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-21T14:58:55.155 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:45.150372+0000", 2024-09-21T14:58:55.155 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.9194284009999993, 2024-09-21T14:58:55.155 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0075518110000000003, 2024-09-21T14:58:55.155 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:55.155 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:55.155 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:55.155 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:55.155 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:55.155 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 20462 2024-09-21T14:58:55.155 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.156 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:55.156 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.156 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:55.156 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150372+0000", 2024-09-21T14:58:55.156 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:55.156 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.156 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.156 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:55.156 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150372+0000", 2024-09-21T14:58:55.156 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:55.156 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.156 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.156 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:55.156 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150372+0000", 2024-09-21T14:58:55.156 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:55.157 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.157 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.157 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:55.157 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150378+0000", 2024-09-21T14:58:55.157 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3330000000000002e-06 2024-09-21T14:58:55.157 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.157 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.157 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:55.157 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150413+0000", 2024-09-21T14:58:55.157 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4798999999999997e-05 2024-09-21T14:58:55.157 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.157 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.157 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:55.157 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150423+0000", 2024-09-21T14:58:55.157 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.8099999999999992e-06 2024-09-21T14:58:55.158 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.158 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.158 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:55.158 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.154550+0000", 2024-09-21T14:58:55.158 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0041275959999999999 2024-09-21T14:58:55.158 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.158 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.158 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:55.158 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.154758+0000", 2024-09-21T14:58:55.158 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00020815499999999999 2024-09-21T14:58:55.158 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.158 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.158 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T14:58:55.158 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.155323+0000", 2024-09-21T14:58:55.159 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00056425400000000004 2024-09-21T14:58:55.159 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.159 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.159 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:58:55.159 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.156702+0000", 2024-09-21T14:58:55.159 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001379676 2024-09-21T14:58:55.159 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.159 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.159 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:55.159 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.157662+0000", 2024-09-21T14:58:55.159 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00095948099999999998 2024-09-21T14:58:55.159 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.159 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.159 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:55.160 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.157816+0000", 2024-09-21T14:58:55.160 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000154666 2024-09-21T14:58:55.160 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.160 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.160 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:55.160 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.157924+0000", 2024-09-21T14:58:55.160 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000107798 2024-09-21T14:58:55.160 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:55.160 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:55.160 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:55.160 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.160 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.160 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:20603 3.b 3:d6a6f68e:::rbd_object_map.1136c84d9a13:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-21T14:58:55.160 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:45.422857+0000", 2024-09-21T14:58:55.160 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.6469436860000002, 2024-09-21T14:58:55.160 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0061270960000000003, 2024-09-21T14:58:55.160 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:55.161 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:55.161 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:55.161 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:55.161 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:55.161 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 20603 2024-09-21T14:58:55.161 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.161 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:55.161 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.161 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:55.161 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.422857+0000", 2024-09-21T14:58:55.161 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:55.161 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.161 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.161 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:55.162 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.422857+0000", 2024-09-21T14:58:55.162 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:55.162 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.162 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.162 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:55.162 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.422856+0000", 2024-09-21T14:58:55.162 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:55.162 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.162 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.162 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:55.162 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.422866+0000", 2024-09-21T14:58:55.162 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0360000000000001e-05 2024-09-21T14:58:55.162 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.162 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.162 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:55.163 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.422885+0000", 2024-09-21T14:58:55.163 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8576999999999998e-05 2024-09-21T14:58:55.163 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.163 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.163 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:55.163 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.422899+0000", 2024-09-21T14:58:55.163 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4711e-05 2024-09-21T14:58:55.163 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.163 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.163 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:55.163 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.424217+0000", 2024-09-21T14:58:55.163 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001318035 2024-09-21T14:58:55.163 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.164 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.164 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:55.164 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.424433+0000", 2024-09-21T14:58:55.164 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000215596 2024-09-21T14:58:55.164 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.164 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.164 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T14:58:55.164 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.425172+0000", 2024-09-21T14:58:55.164 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00073899299999999996 2024-09-21T14:58:55.164 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.164 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.164 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:58:55.164 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.426780+0000", 2024-09-21T14:58:55.164 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0016078760000000001 2024-09-21T14:58:55.164 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.164 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.165 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:55.165 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.428828+0000", 2024-09-21T14:58:55.165 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0020481179999999998 2024-09-21T14:58:55.165 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.165 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.165 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:55.165 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.428929+0000", 2024-09-21T14:58:55.165 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000100603 2024-09-21T14:58:55.165 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:55.165 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:55.165 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:55.165 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.428984+0000", 2024-09-21T14:58:55.165 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5587000000000002e-05 2024-09-21T14:58:55.166 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:55.166 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:55.166 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:55.166 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:55.166 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:55.166 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:58:55.167 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:58:55.164+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:58:55.168 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T14:58:55.239 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T14:58:55.268 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:58:55.265+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:58:55.284 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:58:55.284 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:58:55.284 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T14:58:55.284 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:58:55.303 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T14:58:55.311 INFO:teuthology.orchestra.run.smithi032.stdout:checking clone #7, image image_client.0-clone7 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent8 2024-09-21T14:58:55.369 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:58:55.365+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:58:55.429 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:58:55.429 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:58:55.429 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T14:58:55.430 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T14:58:55.430 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:58:55.441 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T14:58:55.470 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:58:55.468+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:58:55.546 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:58:55.546 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T14:58:55.546 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T14:58:55.546 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T14:58:55.546 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.546 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:1473 3.15 3:af2afa6a:::rbd_data.1136515ecb6b.000000000000001b:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T14:58:55.546 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:30.421730+0000", 2024-09-21T14:58:55.546 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 25.121521163000001, 2024-09-21T14:58:55.546 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021478937, 2024-09-21T14:58:55.547 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:55.547 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:55.547 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:55.547 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:55.547 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:55.547 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1473 2024-09-21T14:58:55.547 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.547 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:55.547 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.547 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:55.547 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421730+0000", 2024-09-21T14:58:55.547 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:55.547 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.547 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.547 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:55.547 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421730+0000", 2024-09-21T14:58:55.548 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:55.548 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.548 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.548 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:55.548 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421728+0000", 2024-09-21T14:58:55.548 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-21T14:58:55.548 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.548 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.548 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:55.548 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421739+0000", 2024-09-21T14:58:55.548 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0502e-05 2024-09-21T14:58:55.548 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.548 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.548 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:55.549 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421762+0000", 2024-09-21T14:58:55.549 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3405999999999999e-05 2024-09-21T14:58:55.549 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.549 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.549 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:55.549 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421773+0000", 2024-09-21T14:58:55.549 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0744000000000001e-05 2024-09-21T14:58:55.549 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.549 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.549 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:55.549 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421838+0000", 2024-09-21T14:58:55.549 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5486000000000003e-05 2024-09-21T14:58:55.549 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.549 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.549 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:55.550 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421955+0000", 2024-09-21T14:58:55.550 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000116765 2024-09-21T14:58:55.550 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.550 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.550 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 6", 2024-09-21T14:58:55.550 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.425572+0000", 2024-09-21T14:58:55.550 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0036171380000000002 2024-09-21T14:58:55.550 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.550 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.550 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:55.550 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.442973+0000", 2024-09-21T14:58:55.550 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.017400268 2024-09-21T14:58:55.550 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.550 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.551 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:55.551 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.442996+0000", 2024-09-21T14:58:55.551 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3139e-05 2024-09-21T14:58:55.551 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.551 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.551 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:55.551 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.443079+0000", 2024-09-21T14:58:55.551 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2832999999999998e-05 2024-09-21T14:58:55.551 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.551 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.551 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:55.551 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.443209+0000", 2024-09-21T14:58:55.551 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013052499999999999 2024-09-21T14:58:55.551 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:55.552 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:55.552 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:55.552 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.552 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.552 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:1678 3.c 3:377cc7e1:::rbd_data.1136515ecb6b.000000000000000f:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T14:58:55.552 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:30.988085+0000", 2024-09-21T14:58:55.552 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 24.555166569000001, 2024-09-21T14:58:55.552 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.032974802999999997, 2024-09-21T14:58:55.552 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:55.552 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:55.552 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:55.552 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:55.552 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:55.552 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1678 2024-09-21T14:58:55.552 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.552 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:55.553 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.553 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:55.553 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988085+0000", 2024-09-21T14:58:55.553 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:55.553 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.553 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.553 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:55.553 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988085+0000", 2024-09-21T14:58:55.553 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:55.553 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.553 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.553 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:55.553 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988083+0000", 2024-09-21T14:58:55.553 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-21T14:58:55.553 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.554 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.554 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:55.554 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988093+0000", 2024-09-21T14:58:55.554 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8800000000000003e-06 2024-09-21T14:58:55.554 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.554 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.554 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:55.554 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988113+0000", 2024-09-21T14:58:55.554 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0217999999999999e-05 2024-09-21T14:58:55.554 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.554 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.554 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:55.554 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988123+0000", 2024-09-21T14:58:55.555 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5859999999999994e-06 2024-09-21T14:58:55.555 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.555 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.555 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:55.555 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988203+0000", 2024-09-21T14:58:55.555 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0834000000000006e-05 2024-09-21T14:58:55.555 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.555 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.555 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:55.555 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988376+0000", 2024-09-21T14:58:55.555 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017301299999999999 2024-09-21T14:58:55.555 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.555 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.555 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 7", 2024-09-21T14:58:55.555 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.996338+0000", 2024-09-21T14:58:55.556 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0079618520000000002 2024-09-21T14:58:55.556 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.556 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.556 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:55.556 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.020881+0000", 2024-09-21T14:58:55.556 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024542731000000002 2024-09-21T14:58:55.556 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.556 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.556 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:55.556 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.020904+0000", 2024-09-21T14:58:55.556 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2861e-05 2024-09-21T14:58:55.556 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.556 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.556 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:55.556 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.020987+0000", 2024-09-21T14:58:55.556 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2806000000000006e-05 2024-09-21T14:58:55.557 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.557 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.557 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:55.557 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.021060+0000", 2024-09-21T14:58:55.557 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2871000000000001e-05 2024-09-21T14:58:55.557 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:55.557 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:55.557 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:55.557 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.557 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.557 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9369 3.3 3:c725132c:::rbd_data.11363a21d2be.0000000000000007:10 [read 24576~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:55.557 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157241+0000", 2024-09-21T14:58:55.557 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 16.386010122999998, 2024-09-21T14:58:55.557 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018289156000000001, 2024-09-21T14:58:55.558 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:55.558 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:55.558 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:55.558 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:55.558 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:55.558 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9369 2024-09-21T14:58:55.558 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.558 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:55.558 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.558 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:55.558 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157241+0000", 2024-09-21T14:58:55.558 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:55.558 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.558 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.558 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:55.559 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157241+0000", 2024-09-21T14:58:55.559 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:55.559 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.559 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.559 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:55.559 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157241+0000", 2024-09-21T14:58:55.559 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:55.559 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.559 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.559 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:55.559 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157244+0000", 2024-09-21T14:58:55.559 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5049999999999999e-06 2024-09-21T14:58:55.559 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.559 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.560 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:55.560 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157255+0000", 2024-09-21T14:58:55.560 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0797e-05 2024-09-21T14:58:55.560 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.560 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.560 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:55.560 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157260+0000", 2024-09-21T14:58:55.560 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1749999999999996e-06 2024-09-21T14:58:55.560 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.560 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.560 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:55.560 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175237+0000", 2024-09-21T14:58:55.560 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.017976405000000001 2024-09-21T14:58:55.560 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.560 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.561 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:55.561 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175331+0000", 2024-09-21T14:58:55.561 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4508999999999999e-05 2024-09-21T14:58:55.561 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.561 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.561 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:55.561 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175530+0000", 2024-09-21T14:58:55.561 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00019928200000000001 2024-09-21T14:58:55.561 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:55.561 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:55.561 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:55.561 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.561 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.561 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9370 3.3 3:c725132c:::rbd_data.11363a21d2be.0000000000000007:10 [read 163840~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:55.561 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157279+0000", 2024-09-21T14:58:55.561 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 16.385971945000001, 2024-09-21T14:58:55.562 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018472379000000001, 2024-09-21T14:58:55.562 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:55.562 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:55.562 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:55.562 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:55.562 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:55.562 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9370 2024-09-21T14:58:55.562 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.562 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:55.562 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.562 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:55.562 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157279+0000", 2024-09-21T14:58:55.562 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:55.562 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.562 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.563 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:55.563 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157279+0000", 2024-09-21T14:58:55.563 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:55.563 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.563 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.563 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:55.563 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157279+0000", 2024-09-21T14:58:55.563 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:55.563 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.563 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.563 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:55.563 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157282+0000", 2024-09-21T14:58:55.563 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2229999999999999e-06 2024-09-21T14:58:55.563 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.563 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.563 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:55.564 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157294+0000", 2024-09-21T14:58:55.564 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2173e-05 2024-09-21T14:58:55.564 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.564 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.564 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:55.564 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157300+0000", 2024-09-21T14:58:55.564 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.93e-06 2024-09-21T14:58:55.564 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.564 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.564 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:55.564 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175569+0000", 2024-09-21T14:58:55.564 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018268376999999999 2024-09-21T14:58:55.564 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.564 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.564 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:55.565 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175622+0000", 2024-09-21T14:58:55.565 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3808999999999999e-05 2024-09-21T14:58:55.565 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.565 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.565 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:55.565 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175752+0000", 2024-09-21T14:58:55.565 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012938700000000001 2024-09-21T14:58:55.565 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:55.565 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:55.565 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:55.565 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.565 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.565 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9371 3.3 3:c0bf6a9b:::rbd_data.11363a21d2be.0000000000000000:10 [read 73728~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:55.565 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157306+0000", 2024-09-21T14:58:55.565 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 16.385945834000001, 2024-09-21T14:58:55.565 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018755473000000002, 2024-09-21T14:58:55.566 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:55.566 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:55.566 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:55.566 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:55.566 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:55.566 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9371 2024-09-21T14:58:55.566 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.566 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:55.566 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.566 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:55.566 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157306+0000", 2024-09-21T14:58:55.566 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:55.566 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.566 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.567 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:55.567 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157306+0000", 2024-09-21T14:58:55.567 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:55.567 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.567 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.567 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:55.567 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157305+0000", 2024-09-21T14:58:55.567 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:55.567 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.567 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.567 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:55.567 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157312+0000", 2024-09-21T14:58:55.567 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3289999999999996e-06 2024-09-21T14:58:55.567 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.567 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.568 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:55.568 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157324+0000", 2024-09-21T14:58:55.568 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1725e-05 2024-09-21T14:58:55.568 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.568 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.568 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:55.568 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157339+0000", 2024-09-21T14:58:55.568 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5211999999999999e-05 2024-09-21T14:58:55.568 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.568 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.568 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:55.568 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175798+0000", 2024-09-21T14:58:55.568 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018458781 2024-09-21T14:58:55.568 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.568 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.569 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:55.569 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175899+0000", 2024-09-21T14:58:55.569 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000100784 2024-09-21T14:58:55.569 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.569 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.569 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:55.569 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176061+0000", 2024-09-21T14:58:55.569 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00016216099999999999 2024-09-21T14:58:55.569 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:55.569 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:55.569 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:55.569 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.569 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.569 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9372 3.3 3:c0bf6a9b:::rbd_data.11363a21d2be.0000000000000000:10 [read 212992~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:55.569 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157345+0000", 2024-09-21T14:58:55.569 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 16.385906488, 2024-09-21T14:58:55.570 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018926493999999999, 2024-09-21T14:58:55.570 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:55.570 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:55.570 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:55.570 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:55.570 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:55.570 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9372 2024-09-21T14:58:55.570 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.570 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:55.570 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.570 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:55.570 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157345+0000", 2024-09-21T14:58:55.570 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:55.571 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.571 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.571 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:55.571 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157345+0000", 2024-09-21T14:58:55.571 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:55.571 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.571 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.571 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:55.571 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157344+0000", 2024-09-21T14:58:55.571 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:55.571 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.571 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.571 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:55.571 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157348+0000", 2024-09-21T14:58:55.572 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1389999999999999e-06 2024-09-21T14:58:55.572 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.572 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.572 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:55.572 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157372+0000", 2024-09-21T14:58:55.572 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4711000000000001e-05 2024-09-21T14:58:55.572 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.572 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.572 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:55.572 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157378+0000", 2024-09-21T14:58:55.572 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9909999999999999e-06 2024-09-21T14:58:55.572 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.572 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.572 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:55.572 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176090+0000", 2024-09-21T14:58:55.572 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018712111999999999 2024-09-21T14:58:55.573 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.573 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.573 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:55.573 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176134+0000", 2024-09-21T14:58:55.573 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3377999999999997e-05 2024-09-21T14:58:55.573 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.573 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.573 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:55.573 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176271+0000", 2024-09-21T14:58:55.573 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013763499999999999 2024-09-21T14:58:55.573 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:55.573 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:55.573 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:55.574 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.574 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.574 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9373 3.3 3:c0bf6a9b:::rbd_data.11363a21d2be.0000000000000000:10 [read 352256~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:55.574 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157384+0000", 2024-09-21T14:58:55.574 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 16.385867716, 2024-09-21T14:58:55.574 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019154937, 2024-09-21T14:58:55.574 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:55.574 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:55.574 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:55.574 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:55.574 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:55.574 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9373 2024-09-21T14:58:55.574 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.574 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:55.575 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.575 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:55.575 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157384+0000", 2024-09-21T14:58:55.575 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:55.575 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.575 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.575 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:55.575 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157384+0000", 2024-09-21T14:58:55.575 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:55.575 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.575 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.575 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:55.575 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157383+0000", 2024-09-21T14:58:55.575 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:55.576 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.576 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.576 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:55.576 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157386+0000", 2024-09-21T14:58:55.576 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.191e-06 2024-09-21T14:58:55.576 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.576 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.576 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:55.576 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157393+0000", 2024-09-21T14:58:55.576 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5339999999999996e-06 2024-09-21T14:58:55.576 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.576 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.576 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:55.576 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157397+0000", 2024-09-21T14:58:55.576 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5220000000000001e-06 2024-09-21T14:58:55.577 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.577 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.577 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:55.577 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176302+0000", 2024-09-21T14:58:55.577 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018905113000000001 2024-09-21T14:58:55.577 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.577 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.577 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:55.577 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176379+0000", 2024-09-21T14:58:55.577 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6372999999999995e-05 2024-09-21T14:58:55.577 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.577 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.577 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:55.577 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176539+0000", 2024-09-21T14:58:55.577 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015973800000000001 2024-09-21T14:58:55.578 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:55.578 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:55.578 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:55.578 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.578 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.578 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9374 3.3 3:c0bf6a9b:::rbd_data.11363a21d2be.0000000000000000:10 [read 491520~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:55.578 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157401+0000", 2024-09-21T14:58:55.578 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 16.385849931999999, 2024-09-21T14:58:55.578 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019387755, 2024-09-21T14:58:55.578 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:55.578 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:55.578 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:55.578 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:55.578 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:55.578 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9374 2024-09-21T14:58:55.579 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.579 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:55.579 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.579 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:55.579 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157401+0000", 2024-09-21T14:58:55.579 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:55.579 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.579 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.579 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:55.579 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157401+0000", 2024-09-21T14:58:55.579 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:55.579 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.579 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.580 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:55.580 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157401+0000", 2024-09-21T14:58:55.580 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:55.580 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.580 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.580 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:55.580 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157414+0000", 2024-09-21T14:58:55.580 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2417e-05 2024-09-21T14:58:55.580 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.580 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.580 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:55.580 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157424+0000", 2024-09-21T14:58:55.580 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0861e-05 2024-09-21T14:58:55.580 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.580 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.580 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:55.581 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157429+0000", 2024-09-21T14:58:55.581 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0499999999999999e-06 2024-09-21T14:58:55.581 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.581 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.581 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:55.581 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176596+0000", 2024-09-21T14:58:55.581 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019166552 2024-09-21T14:58:55.581 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.581 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.581 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:55.581 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176647+0000", 2024-09-21T14:58:55.581 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1112000000000001e-05 2024-09-21T14:58:55.581 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.581 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.581 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:55.582 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176789+0000", 2024-09-21T14:58:55.582 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014209300000000001 2024-09-21T14:58:55.582 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:55.582 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:55.582 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:55.582 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.582 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.582 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9401 3.3 3:c725132c:::rbd_data.11363a21d2be.0000000000000007:10 [read 32768~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:55.582 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157621+0000", 2024-09-21T14:58:55.582 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 16.385630122999999, 2024-09-21T14:58:55.582 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019381711999999999, 2024-09-21T14:58:55.582 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:55.582 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:55.582 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:55.582 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:55.582 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:55.583 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9401 2024-09-21T14:58:55.583 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.583 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:55.583 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.583 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:55.583 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157621+0000", 2024-09-21T14:58:55.583 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:55.583 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.583 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.583 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:55.583 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157621+0000", 2024-09-21T14:58:55.583 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:55.583 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.583 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.584 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:55.584 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157620+0000", 2024-09-21T14:58:55.584 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:58:55.584 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.584 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.584 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:55.584 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157626+0000", 2024-09-21T14:58:55.584 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7570000000000001e-06 2024-09-21T14:58:55.584 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.584 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.584 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:55.584 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157636+0000", 2024-09-21T14:58:55.584 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8339999999999994e-06 2024-09-21T14:58:55.584 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.585 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.585 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:55.585 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157643+0000", 2024-09-21T14:58:55.585 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9439999999999999e-06 2024-09-21T14:58:55.585 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.585 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.585 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:55.585 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176816+0000", 2024-09-21T14:58:55.585 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019172872000000001 2024-09-21T14:58:55.585 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.585 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.585 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:55.585 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176878+0000", 2024-09-21T14:58:55.586 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1734000000000002e-05 2024-09-21T14:58:55.586 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.586 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.586 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:55.586 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177003+0000", 2024-09-21T14:58:55.586 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012537400000000001 2024-09-21T14:58:55.586 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:55.586 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:55.586 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:55.586 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.586 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.586 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9402 3.3 3:c725132c:::rbd_data.11363a21d2be.0000000000000007:10 [read 172032~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:55.586 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157649+0000", 2024-09-21T14:58:55.587 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 16.385602251000002, 2024-09-21T14:58:55.587 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019567404, 2024-09-21T14:58:55.587 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:55.587 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:55.587 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:55.587 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:55.587 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:55.587 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9402 2024-09-21T14:58:55.587 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.587 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:55.587 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.587 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:55.587 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157649+0000", 2024-09-21T14:58:55.588 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:55.588 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.588 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.588 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:55.588 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157649+0000", 2024-09-21T14:58:55.588 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:55.588 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.588 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.588 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:55.588 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157649+0000", 2024-09-21T14:58:55.588 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:55.588 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.588 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.588 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:55.588 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157652+0000", 2024-09-21T14:58:55.589 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8779999999999998e-06 2024-09-21T14:58:55.589 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.589 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.589 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:55.589 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157657+0000", 2024-09-21T14:58:55.589 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6659999999999996e-06 2024-09-21T14:58:55.589 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.589 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.589 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:55.589 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157662+0000", 2024-09-21T14:58:55.589 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5519999999999998e-06 2024-09-21T14:58:55.589 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.589 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.589 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:55.590 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177040+0000", 2024-09-21T14:58:55.590 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019377807 2024-09-21T14:58:55.590 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.590 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.590 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:55.590 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177085+0000", 2024-09-21T14:58:55.590 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5195999999999999e-05 2024-09-21T14:58:55.590 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.590 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.590 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:55.590 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177217+0000", 2024-09-21T14:58:55.590 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013172600000000001 2024-09-21T14:58:55.590 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:55.590 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:55.591 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:55.591 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.591 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.591 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9403 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:55.591 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157949+0000", 2024-09-21T14:58:55.591 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 16.385301922, 2024-09-21T14:58:55.591 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023540373999999999, 2024-09-21T14:58:55.591 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:55.591 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:55.591 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:55.591 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:55.591 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:55.591 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9403 2024-09-21T14:58:55.592 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.592 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:55.592 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.592 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:55.592 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157949+0000", 2024-09-21T14:58:55.592 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:55.592 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.592 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.592 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:55.592 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157949+0000", 2024-09-21T14:58:55.592 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:55.592 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.592 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.593 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:55.593 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157948+0000", 2024-09-21T14:58:55.593 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:55.593 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.593 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.593 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:55.593 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157956+0000", 2024-09-21T14:58:55.593 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7999999999999999e-06 2024-09-21T14:58:55.593 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.593 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.593 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:55.593 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157969+0000", 2024-09-21T14:58:55.593 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2792000000000001e-05 2024-09-21T14:58:55.594 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.594 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.594 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:55.594 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157977+0000", 2024-09-21T14:58:55.594 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.704e-06 2024-09-21T14:58:55.594 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.594 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.594 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:55.594 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177246+0000", 2024-09-21T14:58:55.594 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019268264 2024-09-21T14:58:55.594 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.594 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.594 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:55.594 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177315+0000", 2024-09-21T14:58:55.595 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9480999999999994e-05 2024-09-21T14:58:55.595 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.595 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.595 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:58:55.595 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177615+0000", 2024-09-21T14:58:55.595 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00029929499999999999 2024-09-21T14:58:55.595 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.595 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.595 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:55.595 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181333+0000", 2024-09-21T14:58:55.595 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0037183149999999998 2024-09-21T14:58:55.595 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.595 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.595 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:55.596 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181374+0000", 2024-09-21T14:58:55.596 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.0723999999999998e-05 2024-09-21T14:58:55.596 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.596 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.596 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:55.596 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181434+0000", 2024-09-21T14:58:55.596 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0024e-05 2024-09-21T14:58:55.596 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.596 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.596 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:55.596 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181490+0000", 2024-09-21T14:58:55.596 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6245000000000001e-05 2024-09-21T14:58:55.596 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:55.596 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:55.596 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:55.596 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.597 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.597 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9404 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:55.597 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.160480+0000", 2024-09-21T14:58:55.597 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 16.382771571999999, 2024-09-21T14:58:55.597 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021137686999999999, 2024-09-21T14:58:55.597 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:55.597 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:55.597 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:55.597 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:55.597 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:55.597 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9404 2024-09-21T14:58:55.597 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.597 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:55.597 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.597 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:55.598 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160480+0000", 2024-09-21T14:58:55.598 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:55.598 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.598 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.598 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:55.598 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160480+0000", 2024-09-21T14:58:55.598 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:55.598 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.598 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.598 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:55.598 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160478+0000", 2024-09-21T14:58:55.598 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:55.598 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.598 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.598 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:55.599 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160486+0000", 2024-09-21T14:58:55.599 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9969999999999998e-06 2024-09-21T14:58:55.599 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.599 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.599 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:55.599 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160502+0000", 2024-09-21T14:58:55.599 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5650000000000001e-05 2024-09-21T14:58:55.599 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.599 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.599 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:55.599 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160511+0000", 2024-09-21T14:58:55.599 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.3610000000000007e-06 2024-09-21T14:58:55.599 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.599 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.599 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:55.600 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.178035+0000", 2024-09-21T14:58:55.600 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.017523520000000001 2024-09-21T14:58:55.600 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.600 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.600 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:55.600 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.178151+0000", 2024-09-21T14:58:55.600 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000116013 2024-09-21T14:58:55.600 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.600 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.600 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:58:55.600 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.178408+0000", 2024-09-21T14:58:55.600 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00025743400000000002 2024-09-21T14:58:55.600 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.600 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.600 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:55.600 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181501+0000", 2024-09-21T14:58:55.600 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.003092891 2024-09-21T14:58:55.601 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.601 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.601 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:55.601 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181538+0000", 2024-09-21T14:58:55.601 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6977999999999997e-05 2024-09-21T14:58:55.601 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.601 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.601 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:55.601 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181582+0000", 2024-09-21T14:58:55.601 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3849e-05 2024-09-21T14:58:55.601 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.601 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.601 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:55.601 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181617+0000", 2024-09-21T14:58:55.601 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5423000000000003e-05 2024-09-21T14:58:55.601 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:55.602 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:55.602 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:55.602 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.602 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.602 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9406 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:55.602 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.166040+0000", 2024-09-21T14:58:55.602 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 16.377211693, 2024-09-21T14:58:55.602 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018094524000000001, 2024-09-21T14:58:55.602 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:55.602 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:55.602 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:55.602 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:55.602 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:55.602 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9406 2024-09-21T14:58:55.602 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.602 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:55.603 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.603 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:55.603 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.166040+0000", 2024-09-21T14:58:55.603 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:55.603 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.603 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.603 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:55.603 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.166040+0000", 2024-09-21T14:58:55.603 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:55.603 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.603 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.603 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:55.603 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.166038+0000", 2024-09-21T14:58:55.603 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:55.603 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.604 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.604 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:55.604 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.166051+0000", 2024-09-21T14:58:55.604 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2638e-05 2024-09-21T14:58:55.604 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.604 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.604 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:55.604 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.166070+0000", 2024-09-21T14:58:55.604 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8691000000000001e-05 2024-09-21T14:58:55.604 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.604 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.604 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:55.604 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.166089+0000", 2024-09-21T14:58:55.604 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9286000000000001e-05 2024-09-21T14:58:55.604 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.605 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.605 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:55.605 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.180194+0000", 2024-09-21T14:58:55.605 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014105039999999999 2024-09-21T14:58:55.605 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.605 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.605 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:55.605 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.180340+0000", 2024-09-21T14:58:55.605 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014623199999999999 2024-09-21T14:58:55.605 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.605 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.605 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:58:55.605 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.180818+0000", 2024-09-21T14:58:55.605 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000477489 2024-09-21T14:58:55.605 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.605 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.606 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:55.606 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.183808+0000", 2024-09-21T14:58:55.606 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0029897610000000001 2024-09-21T14:58:55.606 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.606 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.606 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:55.606 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.183821+0000", 2024-09-21T14:58:55.606 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3251999999999999e-05 2024-09-21T14:58:55.606 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.606 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.606 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:55.606 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.183872+0000", 2024-09-21T14:58:55.606 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0900999999999999e-05 2024-09-21T14:58:55.606 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.607 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.607 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:55.607 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.184134+0000", 2024-09-21T14:58:55.607 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000262484 2024-09-21T14:58:55.607 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:55.607 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:55.607 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:55.607 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.607 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.607 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9407 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:55.607 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.168094+0000", 2024-09-21T14:58:55.607 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 16.375157073, 2024-09-21T14:58:55.607 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019444860000000001, 2024-09-21T14:58:55.607 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:55.607 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:55.608 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:55.608 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:55.608 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:55.608 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9407 2024-09-21T14:58:55.608 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.608 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:55.608 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.608 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:55.608 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.168094+0000", 2024-09-21T14:58:55.608 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:55.608 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.608 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.608 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:55.608 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.168094+0000", 2024-09-21T14:58:55.608 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:55.608 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.609 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.609 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:55.609 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.168093+0000", 2024-09-21T14:58:55.609 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:58:55.609 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.609 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.609 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:55.609 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.168119+0000", 2024-09-21T14:58:55.609 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5803999999999999e-05 2024-09-21T14:58:55.609 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.609 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.609 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:55.609 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.168135+0000", 2024-09-21T14:58:55.609 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6359999999999999e-05 2024-09-21T14:58:55.609 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.610 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.610 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:55.610 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.168144+0000", 2024-09-21T14:58:55.610 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4270000000000008e-06 2024-09-21T14:58:55.610 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.610 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.610 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:55.610 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181943+0000", 2024-09-21T14:58:55.610 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013799617 2024-09-21T14:58:55.610 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.610 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.610 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:55.610 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.182052+0000", 2024-09-21T14:58:55.611 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000108924 2024-09-21T14:58:55.611 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.611 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.611 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:58:55.611 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.183431+0000", 2024-09-21T14:58:55.611 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0013783619999999999 2024-09-21T14:58:55.611 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.611 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.611 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:55.611 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.187276+0000", 2024-09-21T14:58:55.611 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0038450469999999999 2024-09-21T14:58:55.611 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.611 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.612 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:55.612 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.187288+0000", 2024-09-21T14:58:55.612 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2414e-05 2024-09-21T14:58:55.612 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.612 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.612 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:55.612 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.187414+0000", 2024-09-21T14:58:55.612 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012614399999999999 2024-09-21T14:58:55.612 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.612 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.612 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:55.612 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.187539+0000", 2024-09-21T14:58:55.612 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012481800000000001 2024-09-21T14:58:55.612 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:55.613 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:55.613 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:55.613 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.613 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.613 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9410 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:55.613 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.174103+0000", 2024-09-21T14:58:55.613 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 16.369148163999999, 2024-09-21T14:58:55.613 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025493703, 2024-09-21T14:58:55.613 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:55.613 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:55.613 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:55.613 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:55.613 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:55.613 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9410 2024-09-21T14:58:55.613 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.613 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:55.614 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.614 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:55.614 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.174103+0000", 2024-09-21T14:58:55.614 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:55.614 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.614 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.614 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:55.614 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.174103+0000", 2024-09-21T14:58:55.614 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:55.614 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.614 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.614 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:55.615 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.174102+0000", 2024-09-21T14:58:55.615 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:58:55.615 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.615 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.615 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:55.615 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.174108+0000", 2024-09-21T14:58:55.615 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1149999999999999e-06 2024-09-21T14:58:55.615 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.615 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.615 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:55.615 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.174117+0000", 2024-09-21T14:58:55.615 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4870000000000002e-06 2024-09-21T14:58:55.615 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.615 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.615 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:55.616 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.174122+0000", 2024-09-21T14:58:55.616 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9790000000000001e-06 2024-09-21T14:58:55.616 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.616 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.616 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:55.616 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.187830+0000", 2024-09-21T14:58:55.616 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013708048 2024-09-21T14:58:55.616 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.616 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.616 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:55.616 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.187913+0000", 2024-09-21T14:58:55.616 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2763999999999996e-05 2024-09-21T14:58:55.616 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.616 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.616 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:58:55.617 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.188230+0000", 2024-09-21T14:58:55.617 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00031745100000000001 2024-09-21T14:58:55.617 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.617 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.617 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:55.617 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.192708+0000", 2024-09-21T14:58:55.617 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0044783890000000002 2024-09-21T14:58:55.617 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.617 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.617 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:55.617 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199487+0000", 2024-09-21T14:58:55.617 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0067782149999999998 2024-09-21T14:58:55.617 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.617 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.618 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:55.618 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199560+0000", 2024-09-21T14:58:55.618 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3752999999999997e-05 2024-09-21T14:58:55.618 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.618 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.618 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:55.618 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199597+0000", 2024-09-21T14:58:55.618 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6526999999999998e-05 2024-09-21T14:58:55.618 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:55.618 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:55.618 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:55.618 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.618 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.619 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9411 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:55.619 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.175943+0000", 2024-09-21T14:58:55.619 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 16.367308144999999, 2024-09-21T14:58:55.619 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023773181000000001, 2024-09-21T14:58:55.619 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:55.619 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:55.619 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:55.619 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:55.619 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:55.619 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9411 2024-09-21T14:58:55.619 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.619 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:55.619 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.619 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:55.619 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175943+0000", 2024-09-21T14:58:55.620 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:55.620 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.620 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.620 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:55.620 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175943+0000", 2024-09-21T14:58:55.620 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:55.620 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.620 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.620 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:55.620 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175942+0000", 2024-09-21T14:58:55.620 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:58:55.620 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.620 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.620 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:55.620 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175948+0000", 2024-09-21T14:58:55.621 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9689999999999999e-06 2024-09-21T14:58:55.621 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.621 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.621 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:55.621 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175960+0000", 2024-09-21T14:58:55.621 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1745e-05 2024-09-21T14:58:55.621 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.621 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.621 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:55.621 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175965+0000", 2024-09-21T14:58:55.621 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.203e-06 2024-09-21T14:58:55.621 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.621 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.621 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:55.622 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.189900+0000", 2024-09-21T14:58:55.622 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013934940999999999 2024-09-21T14:58:55.622 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.622 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.622 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:55.622 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.189988+0000", 2024-09-21T14:58:55.622 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8419999999999997e-05 2024-09-21T14:58:55.622 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.622 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.622 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:58:55.622 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.190321+0000", 2024-09-21T14:58:55.622 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00033231800000000001 2024-09-21T14:58:55.622 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.622 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.622 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:55.623 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.192746+0000", 2024-09-21T14:58:55.623 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0024254400000000001 2024-09-21T14:58:55.623 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.623 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.623 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:55.623 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199605+0000", 2024-09-21T14:58:55.623 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0068582809999999999 2024-09-21T14:58:55.623 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.623 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.623 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:55.623 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199680+0000", 2024-09-21T14:58:55.623 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5630000000000006e-05 2024-09-21T14:58:55.623 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.623 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.624 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:55.624 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199716+0000", 2024-09-21T14:58:55.624 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6248999999999998e-05 2024-09-21T14:58:55.624 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:55.624 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:55.624 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:55.624 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.624 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.624 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9412 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:55.624 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.177402+0000", 2024-09-21T14:58:55.624 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 16.365849048000001, 2024-09-21T14:58:55.624 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022445590000000001, 2024-09-21T14:58:55.624 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:55.624 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:55.625 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:55.625 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:55.625 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:55.625 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9412 2024-09-21T14:58:55.625 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.625 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:55.625 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.625 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:55.625 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177402+0000", 2024-09-21T14:58:55.625 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:55.625 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.625 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.625 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:55.625 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177402+0000", 2024-09-21T14:58:55.625 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:55.626 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.626 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.626 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:55.626 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177401+0000", 2024-09-21T14:58:55.626 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:58:55.626 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.626 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.626 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:55.626 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177409+0000", 2024-09-21T14:58:55.626 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4710000000000001e-06 2024-09-21T14:58:55.626 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.626 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.626 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:55.626 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177419+0000", 2024-09-21T14:58:55.627 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0351e-05 2024-09-21T14:58:55.627 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.627 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.627 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:55.627 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177424+0000", 2024-09-21T14:58:55.627 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1919999999999996e-06 2024-09-21T14:58:55.627 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.627 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.627 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:55.627 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.191343+0000", 2024-09-21T14:58:55.627 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.01391878 2024-09-21T14:58:55.627 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.627 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.627 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:55.627 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.191446+0000", 2024-09-21T14:58:55.628 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000102453 2024-09-21T14:58:55.628 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.628 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.628 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:58:55.628 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.192200+0000", 2024-09-21T14:58:55.628 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00075476899999999997 2024-09-21T14:58:55.628 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.628 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.628 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:55.628 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.193866+0000", 2024-09-21T14:58:55.628 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0016657519999999999 2024-09-21T14:58:55.628 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.628 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.628 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:55.628 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199724+0000", 2024-09-21T14:58:55.628 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0058579529999999999 2024-09-21T14:58:55.629 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.629 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.629 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:55.629 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199803+0000", 2024-09-21T14:58:55.629 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8662e-05 2024-09-21T14:58:55.629 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.629 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.629 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:55.629 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199848+0000", 2024-09-21T14:58:55.629 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5216000000000002e-05 2024-09-21T14:58:55.629 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:55.629 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:55.629 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:55.629 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.630 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.630 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:10445 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:55.630 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.271980+0000", 2024-09-21T14:58:55.630 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 16.271271682999998, 2024-09-21T14:58:55.630 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018664256000000001, 2024-09-21T14:58:55.630 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:55.630 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:55.630 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:55.630 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:55.630 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:55.630 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 10445 2024-09-21T14:58:55.630 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.630 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:55.630 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.630 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:55.631 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.271980+0000", 2024-09-21T14:58:55.631 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:55.631 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.631 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.631 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:55.631 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.271980+0000", 2024-09-21T14:58:55.631 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:55.631 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.631 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.631 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:55.631 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.271979+0000", 2024-09-21T14:58:55.631 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:55.631 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.631 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.631 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:55.632 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.271982+0000", 2024-09-21T14:58:55.632 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7240000000000001e-06 2024-09-21T14:58:55.632 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.632 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.632 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:55.632 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.271987+0000", 2024-09-21T14:58:55.632 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3639999999999998e-06 2024-09-21T14:58:55.632 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.632 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.632 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:55.632 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.271992+0000", 2024-09-21T14:58:55.632 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7110000000000003e-06 2024-09-21T14:58:55.632 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.632 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.632 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:55.633 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.277567+0000", 2024-09-21T14:58:55.633 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0055745009999999999 2024-09-21T14:58:55.633 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.633 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.633 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:55.633 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.277655+0000", 2024-09-21T14:58:55.633 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8465000000000001e-05 2024-09-21T14:58:55.633 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.633 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.633 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:58:55.633 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.278720+0000", 2024-09-21T14:58:55.633 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0010650569999999999 2024-09-21T14:58:55.633 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.633 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.634 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:55.634 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.281153+0000", 2024-09-21T14:58:55.634 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0024326389999999999 2024-09-21T14:58:55.634 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.634 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.634 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:55.634 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.290574+0000", 2024-09-21T14:58:55.634 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0094209489999999996 2024-09-21T14:58:55.634 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.634 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.634 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:55.634 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.290613+0000", 2024-09-21T14:58:55.634 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.8803000000000001e-05 2024-09-21T14:58:55.634 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.635 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.635 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:55.635 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.290644+0000", 2024-09-21T14:58:55.635 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1404e-05 2024-09-21T14:58:55.635 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:55.635 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:55.635 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:55.635 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.635 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.635 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22769 3.f 3:f39bd56f:::rbd_data.11366acc894.0000000000000004:19 [read 1032192~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:58:55.635 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.140207+0000", 2024-09-21T14:58:55.635 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 5.4030439430000001, 2024-09-21T14:58:55.635 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018031170999999999, 2024-09-21T14:58:55.635 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:55.635 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:55.636 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:55.636 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:55.636 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:55.636 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22769 2024-09-21T14:58:55.636 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.636 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:55.636 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.636 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:55.636 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.140207+0000", 2024-09-21T14:58:55.636 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:55.636 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.636 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.636 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:55.636 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.140207+0000", 2024-09-21T14:58:55.636 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:55.636 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.637 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.637 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:55.637 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.140207+0000", 2024-09-21T14:58:55.637 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:55.637 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.637 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.637 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:55.637 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.140209+0000", 2024-09-21T14:58:55.637 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.305e-06 2024-09-21T14:58:55.637 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.637 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.637 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:55.637 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.140214+0000", 2024-09-21T14:58:55.637 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9570000000000001e-06 2024-09-21T14:58:55.637 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.638 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.638 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:55.638 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.140219+0000", 2024-09-21T14:58:55.638 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2250000000000002e-06 2024-09-21T14:58:55.638 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.638 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.638 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:55.638 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.157990+0000", 2024-09-21T14:58:55.638 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.017770939999999999 2024-09-21T14:58:55.638 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.638 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.638 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:55.638 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.158062+0000", 2024-09-21T14:58:55.638 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2652999999999997e-05 2024-09-21T14:58:55.639 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.639 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.639 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:55.639 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.158239+0000", 2024-09-21T14:58:55.639 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017640599999999999 2024-09-21T14:58:55.639 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:55.639 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:55.639 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:55.639 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.639 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.639 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:23084 3.3 3:c55e343c:::rbd_data.11366acc894.0000000000000006:19 [read 1032192~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:58:55.639 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.150724+0000", 2024-09-21T14:58:55.639 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 5.3925271080000003, 2024-09-21T14:58:55.639 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018916460999999999, 2024-09-21T14:58:55.639 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:55.639 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:55.640 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:55.640 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:55.640 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:55.640 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 23084 2024-09-21T14:58:55.640 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.640 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:55.640 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.640 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:55.640 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.150724+0000", 2024-09-21T14:58:55.640 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:55.640 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.640 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.640 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:55.640 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.150724+0000", 2024-09-21T14:58:55.640 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:55.640 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.641 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.641 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:55.641 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.150724+0000", 2024-09-21T14:58:55.641 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:55.641 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.641 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.641 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:55.641 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.150727+0000", 2024-09-21T14:58:55.641 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7250000000000002e-06 2024-09-21T14:58:55.641 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.641 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.641 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:55.641 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.150732+0000", 2024-09-21T14:58:55.641 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5809999999999996e-06 2024-09-21T14:58:55.641 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.642 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.642 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:55.642 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.150737+0000", 2024-09-21T14:58:55.642 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3590000000000001e-06 2024-09-21T14:58:55.642 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.642 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.642 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:55.642 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.166694+0000", 2024-09-21T14:58:55.642 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.015957720000000002 2024-09-21T14:58:55.642 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.642 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.642 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:55.642 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.166769+0000", 2024-09-21T14:58:55.642 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5143999999999997e-05 2024-09-21T14:58:55.643 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:55.643 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:55.643 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:55.643 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.169641+0000", 2024-09-21T14:58:55.643 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0028713089999999998 2024-09-21T14:58:55.643 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:55.643 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:55.643 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:55.643 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:55.643 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:55.643 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:58:55.643 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T14:58:55.645 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T14:58:55.646 INFO:teuthology.orchestra.run.smithi032.stdout:1007 writesame 0xea5d485 thru 0xea6820b data_size 0x1(0xad87 bytes) 2024-09-21T14:58:55.646 INFO:teuthology.orchestra.run.smithi032.stdout:1009 read 0x29ff41d thru 0x2a0dc84 (0xe868 bytes) 2024-09-21T14:58:55.646 INFO:teuthology.orchestra.run.smithi032.stdout:1011 writesame 0xbe52f8d thru 0xbe59312 data_size 0x1(0x6386 bytes) 2024-09-21T14:58:55.646 INFO:teuthology.orchestra.run.smithi032.stdout:1012 trunc from 0xea6820c to 0xe003592 2024-09-21T14:58:55.646 INFO:teuthology.orchestra.run.smithi032.stdout:1013 trunc from 0xe003592 to 0xc3da6dd 2024-09-21T14:58:55.646 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:58:55.641+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:58:55.660 INFO:teuthology.orchestra.run.smithi032.stdout:1014 trunc from 0xc3da6dd to 0xc43995c 2024-09-21T14:58:55.691 INFO:teuthology.orchestra.run.smithi032.stdout:1015 write 0xe21cdab thru 0xe2299df (0xcc35 bytes) 2024-09-21T14:58:55.694 INFO:teuthology.orchestra.run.smithi032.stdout:1016 writesame 0xcee5ca6 thru 0xceef21c data_size 0x1(0x9577 bytes) 2024-09-21T14:58:55.722 INFO:teuthology.orchestra.run.smithi032.stdout:1018 write 0x32966f0 thru 0x329e457 (0x7d68 bytes) 2024-09-21T14:58:55.725 INFO:teuthology.orchestra.run.smithi032.stdout:1019 write 0x91f8123 thru 0x91fbdb8 (0x3c96 bytes) 2024-09-21T14:58:55.728 INFO:teuthology.orchestra.run.smithi032.stdout:1021 read 0x13a8a01 thru 0x13ab290 (0x2890 bytes) 2024-09-21T14:58:55.729 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:55.740 INFO:teuthology.orchestra.run.smithi032.stdout:1024 write 0xea3ba31 thru 0xea42f56 (0x7526 bytes) 2024-09-21T14:58:55.746 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:58:55.744+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:58:55.747 INFO:teuthology.orchestra.run.smithi032.stdout:1025 write 0x17d5437 thru 0x17e4d33 (0xf8fd bytes) 2024-09-21T14:58:55.750 INFO:teuthology.orchestra.run.smithi032.stdout:1026 read 0xa75e06c thru 0xa76286a (0x47ff bytes) 2024-09-21T14:58:55.750 INFO:teuthology.orchestra.run.smithi032.stdout:1029 read 0xbe69012 thru 0xbe6f495 (0x6484 bytes) 2024-09-21T14:58:55.751 INFO:teuthology.orchestra.run.smithi032.stdout:1032 writesame 0x76dfead thru 0x76e6fb0 data_size 0x6(0x7104 bytes) 2024-09-21T14:58:55.755 INFO:teuthology.orchestra.run.smithi032.stdout:1035 read 0x15d5610 thru 0x15e2b34 (0xd525 bytes) 2024-09-21T14:58:55.756 INFO:teuthology.orchestra.run.smithi032.stdout:1036 write 0x9fe6a53 thru 0x9feb837 (0x4de5 bytes) 2024-09-21T14:58:55.762 INFO:teuthology.orchestra.run.smithi032.stdout:1037 punch from 0x3e3bfe5 to 0x3e4648d, (0xa4a8 bytes) 2024-09-21T14:58:55.762 INFO:teuthology.orchestra.run.smithi032.stdout:1038 punch from 0x657cb50 to 0x6584ebe, (0x836e bytes) 2024-09-21T14:58:55.762 INFO:teuthology.orchestra.run.smithi032.stdout:1039 write 0xb316a5c thru 0xb3251b8 (0xe75d bytes) 2024-09-21T14:58:55.767 INFO:teuthology.orchestra.run.smithi032.stdout:1040 read 0x541b99a thru 0x541fa5e (0x40c5 bytes) 2024-09-21T14:58:55.768 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:55.768 INFO:teuthology.orchestra.run.smithi032.stdout:1042 read 0x7cdfd6c thru 0x7ce46a6 (0x493b bytes) 2024-09-21T14:58:55.769 INFO:teuthology.orchestra.run.smithi032.stdout:1043 read 0xb9a9cec thru 0xb9ada32 (0x3d47 bytes) 2024-09-21T14:58:55.769 INFO:teuthology.orchestra.run.smithi032.stdout:1044 write 0xd3fbf13 thru 0xd4092e9 (0xd3d7 bytes) 2024-09-21T14:58:55.778 INFO:teuthology.orchestra.run.smithi032.stdout:1045 write 0x4a35f3d thru 0x4a379ed (0x1ab1 bytes) 2024-09-21T14:58:55.785 INFO:teuthology.orchestra.run.smithi032.stdout:1046 writesame 0x2182fc8 thru 0x2184f93 data_size 0x14(0x1fcc bytes) 2024-09-21T14:58:55.787 INFO:teuthology.orchestra.run.smithi032.stdout:1047 read 0xd543bd5 thru 0xd551bf9 (0xe025 bytes) 2024-09-21T14:58:55.787 INFO:teuthology.orchestra.run.smithi032.stdout:1049 read 0x8d2a636 thru 0x8d3707f (0xca4a bytes) 2024-09-21T14:58:55.788 INFO:teuthology.orchestra.run.smithi032.stdout:1050 read 0xc8862a thru 0xc89413 (0xdea bytes) 2024-09-21T14:58:55.788 INFO:teuthology.orchestra.run.smithi032.stdout:1051 punch from 0x6a96982 to 0x6aa12ba, (0xa938 bytes) 2024-09-21T14:58:55.789 INFO:teuthology.orchestra.run.smithi032.stdout:1052 write 0xba11e9a thru 0xba20dcb (0xef32 bytes) 2024-09-21T14:58:55.791 INFO:teuthology.orchestra.run.smithi032.stdout:1053 write 0x1dba01a thru 0x1dc52a1 (0xb288 bytes) 2024-09-21T14:58:55.791 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:58:55.791 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:58:55.792 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T14:58:55.792 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:58:55.795 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:55.795 INFO:teuthology.orchestra.run.smithi032.stdout:1057 writesame 0x25ac3b3 thru 0x25b2366 data_size 0x2(0x5fb4 bytes) 2024-09-21T14:58:55.802 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T14:58:55.806 INFO:teuthology.orchestra.run.smithi032.stdout:1058 read 0x93b5076 thru 0x93be985 (0x9910 bytes) 2024-09-21T14:58:55.807 INFO:teuthology.orchestra.run.smithi032.stdout:1059 punch from 0x34d20c8 to 0x34d65ec, (0x4524 bytes) 2024-09-21T14:58:55.807 INFO:teuthology.orchestra.run.smithi032.stdout:1060 write 0xcacf653 thru 0xcacf93f (0x2ed bytes) 2024-09-21T14:58:55.809 INFO:teuthology.orchestra.run.smithi032.stdout:1062 read 0x204e637 thru 0x2052665 (0x402f bytes) 2024-09-21T14:58:55.810 INFO:teuthology.orchestra.run.smithi032.stdout:1063 punch from 0x970300 to 0x97b49d, (0xb19d bytes) 2024-09-21T14:58:55.810 INFO:teuthology.orchestra.run.smithi032.stdout:1064 read 0x7e74387 thru 0x7e7ae42 (0x6abc bytes) 2024-09-21T14:58:55.810 INFO:teuthology.orchestra.run.smithi032.stdout:1065 punch from 0x1a2114f to 0x1a2704f, (0x5f00 bytes) 2024-09-21T14:58:55.810 INFO:teuthology.orchestra.run.smithi032.stdout:1069 write 0x19de19f thru 0x19e6043 (0x7ea5 bytes) 2024-09-21T14:58:55.812 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T14:58:55.813 INFO:teuthology.orchestra.run.smithi032.stdout:1070 write 0x2337b1 thru 0x23b67a (0x7eca bytes) 2024-09-21T14:58:55.817 INFO:teuthology.orchestra.run.smithi032.stdout:1071 read 0x6637aac thru 0x6642bd2 (0xb127 bytes) 2024-09-21T14:58:55.818 INFO:teuthology.orchestra.run.smithi032.stdout:1072 write 0xcee2120 thru 0xceed4e1 (0xb3c2 bytes) 2024-09-21T14:58:55.821 INFO:teuthology.orchestra.run.smithi032.stdout:1073 writesame 0xb1bcf25 thru 0xb1c13d7 data_size 0x1(0x44b3 bytes) 2024-09-21T14:58:55.832 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:55.832 INFO:teuthology.orchestra.run.smithi032.stdout:1075 write 0x921d6b thru 0x9229a7 (0xc3d bytes) 2024-09-21T14:58:55.834 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T14:58:55.847 INFO:teuthology.orchestra.run.smithi032.stdout:1077 trunc from 0xea42f57 to 0x70e027a 2024-09-21T14:58:55.847 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:58:55.844+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:58:55.854 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T14:58:55.870 INFO:teuthology.orchestra.run.smithi032.stdout:1081 trunc from 0x70e027a to 0x8598652 2024-09-21T14:58:55.877 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T14:58:55.879 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:55.891 INFO:teuthology.orchestra.run.smithi032.stdout:1083 trunc from 0x8598652 to 0x17625da 2024-09-21T14:58:55.908 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:55.908 INFO:teuthology.orchestra.run.smithi032.stdout:1085 read 0xfe3656 thru 0xfe538f (0x1d3a bytes) 2024-09-21T14:58:55.909 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-21T14:58:55.919 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:55.919 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:55.924 INFO:teuthology.orchestra.run.smithi032.stdout:1086 write 0x4f2814c thru 0x4f29ce5 (0x1b9a bytes) 2024-09-21T14:58:55.929 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T14:58:55.933 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T14:58:55.948 INFO:teuthology.orchestra.run.smithi032.stdout:1088 writesame 0xa3445bd thru 0xa34f5f2 data_size 0x1a(0xb036 bytes) 2024-09-21T14:58:55.948 INFO:teuthology.orchestra.run.smithi032.stdout:1090 write 0x7a94298 thru 0x7a99c57 (0x59c0 bytes) 2024-09-21T14:58:55.948 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:58:55.945+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:58:55.951 INFO:teuthology.orchestra.run.smithi032.stdout:1092 punch from 0x7181bf9 to 0x719087b, (0xec82 bytes) 2024-09-21T14:58:55.951 INFO:teuthology.orchestra.run.smithi032.stdout:1093 read 0x95d9f1e thru 0x95e047b (0x655e bytes) 2024-09-21T14:58:55.951 INFO:teuthology.orchestra.run.smithi032.stdout:1094 punch from 0x74f01fa to 0x74f255e, (0x2364 bytes) 2024-09-21T14:58:55.951 INFO:teuthology.orchestra.run.smithi032.stdout:1096 read 0x27bf9ae thru 0x27ce2ce (0xe921 bytes) 2024-09-21T14:58:55.952 INFO:teuthology.orchestra.run.smithi032.stdout:1098 write 0x261ee4a thru 0x2621e98 (0x304f bytes) 2024-09-21T14:58:55.955 INFO:teuthology.orchestra.run.smithi032.stdout:1099 punch from 0x493d5dd to 0x494d08b, (0xfaae bytes) 2024-09-21T14:58:55.955 INFO:teuthology.orchestra.run.smithi032.stdout:1100 read 0x5b061df thru 0x5b07cde (0x1b00 bytes) 2024-09-21T14:58:55.955 INFO:teuthology.orchestra.run.smithi032.stdout:1102 write 0x4120b9e thru 0x4125e9d (0x5300 bytes) 2024-09-21T14:58:55.958 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T14:58:55.958 INFO:teuthology.orchestra.run.smithi032.stdout:1103 write 0x71c5713 thru 0x71cd4af (0x7d9d bytes) 2024-09-21T14:58:55.961 INFO:teuthology.orchestra.run.smithi032.stdout:1104 punch from 0x805e461 to 0x806971e, (0xb2bd bytes) 2024-09-21T14:58:55.975 INFO:teuthology.orchestra.run.smithi032.stdout:1105 write 0xb21de40 thru 0xb22143e (0x35ff bytes) 2024-09-21T14:58:55.979 INFO:teuthology.orchestra.run.smithi032.stdout:1106 writesame 0x80f266a thru 0x80f4227 data_size 0x2(0x1bbe bytes) 2024-09-21T14:58:55.982 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T14:58:55.982 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:55.982 INFO:teuthology.orchestra.run.smithi032.stdout:1108 writesame 0x71ee1e8 thru 0x71f4bcb data_size 0x36(0x69e4 bytes) 2024-09-21T14:58:55.985 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:58:55.985 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:58:55.985 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:58:55.985 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:58:55.985 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:58:55.985 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:58:55.985 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:58:55.985 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:58:55.985 INFO:teuthology.orchestra.run.smithi032.stdout:1109 clone 10 order 19 su 32768 sc 8 2024-09-21T14:58:56.037 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:56.037 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:56.047 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T14:58:56.048 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:58:56.044+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:58:56.149 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:58:56.145+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:58:56.203 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:56.204 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:56.213 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T14:58:56.246 INFO:teuthology.orchestra.run.smithi032.stderr:nodeep-scrub is set 2024-09-21T14:58:56.250 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:58:56.248+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:58:56.333 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:56.333 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:56.343 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T14:58:56.350 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:58:56.348+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:58:56.450 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:56.450 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:56.451 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:58:56.449+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:58:56.461 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T14:58:56.552 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:58:56.549+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:58:56.569 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:56.570 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:56.580 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T14:58:56.581 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T14:58:56.581 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T14:58:56.581 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-21T14:58:56.581 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-21T14:58:56.653 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:58:56.653+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:58:56.695 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:56.695 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:56.705 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T14:58:56.754 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:58:56.752+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:58:56.804 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:56.804 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:56.814 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T14:58:56.861 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:58:56.860+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:58:56.904 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-21T14:58:56.904 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:58:56.904 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 60, 2024-09-21T14:58:56.904 INFO:teuthology.orchestra.run.smithi032.stdout: "fsid": "caf51e55-9234-4a26-bcd2-d60811962560", 2024-09-21T14:58:56.904 INFO:teuthology.orchestra.run.smithi032.stdout: "created": "2024-09-21T14:57:50.338822+0000", 2024-09-21T14:58:56.904 INFO:teuthology.orchestra.run.smithi032.stdout: "modified": "2024-09-21T14:58:56.229252+0000", 2024-09-21T14:58:56.904 INFO:teuthology.orchestra.run.smithi032.stdout: "last_up_change": "2024-09-21T14:58:01.177141+0000", 2024-09-21T14:58:56.904 INFO:teuthology.orchestra.run.smithi032.stdout: "last_in_change": "2024-09-21T14:57:52.442036+0000", 2024-09-21T14:58:56.905 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-21T14:58:56.905 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_num": 5806080, 2024-09-21T14:58:56.905 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_set": [ 2024-09-21T14:58:56.905 INFO:teuthology.orchestra.run.smithi032.stdout: "nodeep-scrub", 2024-09-21T14:58:56.905 INFO:teuthology.orchestra.run.smithi032.stdout: "noscrub", 2024-09-21T14:58:56.905 INFO:teuthology.orchestra.run.smithi032.stdout: "pglog_hardlimit", 2024-09-21T14:58:56.905 INFO:teuthology.orchestra.run.smithi032.stdout: "purged_snapdirs", 2024-09-21T14:58:56.905 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_deletes", 2024-09-21T14:58:56.905 INFO:teuthology.orchestra.run.smithi032.stdout: "sortbitwise" 2024-09-21T14:58:56.905 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-21T14:58:56.905 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_version": 5, 2024-09-21T14:58:56.905 INFO:teuthology.orchestra.run.smithi032.stdout: "full_ratio": 0.94999998807907104, 2024-09-21T14:58:56.905 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-21T14:58:56.906 INFO:teuthology.orchestra.run.smithi032.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-21T14:58:56.906 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_snapshot": "", 2024-09-21T14:58:56.906 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_max": 3, 2024-09-21T14:58:56.906 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd": 8, 2024-09-21T14:58:56.906 INFO:teuthology.orchestra.run.smithi032.stdout: "require_min_compat_client": "luminous", 2024-09-21T14:58:56.906 INFO:teuthology.orchestra.run.smithi032.stdout: "min_compat_client": "luminous", 2024-09-21T14:58:56.906 INFO:teuthology.orchestra.run.smithi032.stdout: "require_osd_release": "reef", 2024-09-21T14:58:56.906 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_crimson": false, 2024-09-21T14:58:56.906 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2024-09-21T14:58:56.906 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:56.906 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 1, 2024-09-21T14:58:56.906 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": ".mgr", 2024-09-21T14:58:56.907 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-09-21T14:58:01.842810+0000", 2024-09-21T14:58:56.907 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-09-21T14:58:56.907 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2024-09-21T14:58:56.907 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-09-21T14:58:56.907 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-09-21T14:58:56.907 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-09-21T14:58:56.907 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-09-21T14:58:56.907 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-09-21T14:58:56.907 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-09-21T14:58:56.907 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T14:58:56.907 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T14:58:56.907 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-09-21T14:58:56.907 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-09-21T14:58:56.908 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 1, 2024-09-21T14:58:56.908 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 1, 2024-09-21T14:58:56.908 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 1, 2024-09-21T14:58:56.908 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 1, 2024-09-21T14:58:56.908 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 1, 2024-09-21T14:58:56.908 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-09-21T14:58:56.908 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-09-21T14:58:56.908 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-09-21T14:58:56.908 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-09-21T14:58:56.908 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-09-21T14:58:56.908 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-09-21T14:58:56.908 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-09-21T14:58:56.909 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.909 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "18", 2024-09-21T14:58:56.909 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-09-21T14:58:56.909 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T14:58:56.909 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T14:58:56.909 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-09-21T14:58:56.909 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-09-21T14:58:56.909 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-09-21T14:58:56.909 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-09-21T14:58:56.909 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-09-21T14:58:56.909 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-09-21T14:58:56.909 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-09-21T14:58:56.909 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-09-21T14:58:56.910 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-09-21T14:58:56.910 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-09-21T14:58:56.910 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-09-21T14:58:56.910 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-09-21T14:58:56.910 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-09-21T14:58:56.910 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-09-21T14:58:56.910 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-09-21T14:58:56.910 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T14:58:56.910 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T14:58:56.910 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T14:58:56.910 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-09-21T14:58:56.910 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-09-21T14:58:56.911 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-09-21T14:58:56.911 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-09-21T14:58:56.911 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-09-21T14:58:56.911 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.911 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-09-21T14:58:56.911 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-09-21T14:58:56.911 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-09-21T14:58:56.911 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-09-21T14:58:56.911 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-09-21T14:58:56.911 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T14:58:56.911 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-09-21T14:58:56.911 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-09-21T14:58:56.912 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-09-21T14:58:56.912 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-09-21T14:58:56.912 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-09-21T14:58:56.912 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-09-21T14:58:56.912 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_max": 32, 2024-09-21T14:58:56.912 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 1 2024-09-21T14:58:56.912 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.912 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-09-21T14:58:56.912 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": {} 2024-09-21T14:58:56.912 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.912 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-09-21T14:58:56.912 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 8, 2024-09-21T14:58:56.912 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 8, 2024-09-21T14:58:56.913 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 0.25, 2024-09-21T14:58:56.913 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 2, 2024-09-21T14:58:56.913 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 2, 2024-09-21T14:58:56.913 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 0.88126564025878906, 2024-09-21T14:58:56.913 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 0.87999999523162842, 2024-09-21T14:58:56.913 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-21T14:58:56.913 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:56.913 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.913 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:56.913 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 2, 2024-09-21T14:58:56.913 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "rbd", 2024-09-21T14:58:56.913 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-09-21T14:58:08.284925+0000", 2024-09-21T14:58:56.914 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 8193, 2024-09-21T14:58:56.914 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T14:58:56.914 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-09-21T14:58:56.914 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-09-21T14:58:56.914 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-09-21T14:58:56.914 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-09-21T14:58:56.914 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-09-21T14:58:56.914 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-09-21T14:58:56.914 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T14:58:56.914 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T14:58:56.914 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-09-21T14:58:56.915 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-09-21T14:58:56.915 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2024-09-21T14:58:56.915 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2024-09-21T14:58:56.915 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2024-09-21T14:58:56.915 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2024-09-21T14:58:56.915 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2024-09-21T14:58:56.915 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-09-21T14:58:56.915 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-09-21T14:58:56.915 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-09-21T14:58:56.915 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-09-21T14:58:56.915 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-09-21T14:58:56.915 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-09-21T14:58:56.915 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-09-21T14:58:56.916 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.916 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "22", 2024-09-21T14:58:56.916 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-09-21T14:58:56.916 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T14:58:56.916 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T14:58:56.916 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-09-21T14:58:56.916 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-09-21T14:58:56.916 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 3, 2024-09-21T14:58:56.916 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 22, 2024-09-21T14:58:56.916 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-09-21T14:58:56.916 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-09-21T14:58:56.916 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-09-21T14:58:56.916 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-09-21T14:58:56.917 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-09-21T14:58:56.917 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-09-21T14:58:56.917 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-09-21T14:58:56.917 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-09-21T14:58:56.917 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-09-21T14:58:56.917 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-09-21T14:58:56.917 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-09-21T14:58:56.917 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T14:58:56.917 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T14:58:56.917 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T14:58:56.917 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-09-21T14:58:56.917 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-09-21T14:58:56.918 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-09-21T14:58:56.918 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-09-21T14:58:56.918 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-09-21T14:58:56.918 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.918 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-09-21T14:58:56.918 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-09-21T14:58:56.918 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-09-21T14:58:56.918 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-09-21T14:58:56.918 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-09-21T14:58:56.918 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T14:58:56.918 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-09-21T14:58:56.918 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-09-21T14:58:56.919 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-09-21T14:58:56.919 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-09-21T14:58:56.919 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-09-21T14:58:56.919 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2024-09-21T14:58:56.919 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-09-21T14:58:56.919 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd": {} 2024-09-21T14:58:56.919 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.919 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-09-21T14:58:56.919 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 4.9800000190734863, 2024-09-21T14:58:56.919 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 4.9800000190734863, 2024-09-21T14:58:56.919 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 0.87999999523162842, 2024-09-21T14:58:56.919 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 4.380000114440918, 2024-09-21T14:58:56.920 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 4.380000114440918, 2024-09-21T14:58:56.920 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 0.88126564025878906, 2024-09-21T14:58:56.920 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 0.87999999523162842, 2024-09-21T14:58:56.920 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-21T14:58:56.920 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:56.920 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.920 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:56.920 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 3, 2024-09-21T14:58:56.920 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "pool_client.0", 2024-09-21T14:58:56.920 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-09-21T14:58:17.510709+0000", 2024-09-21T14:58:56.920 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 8193, 2024-09-21T14:58:56.920 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T14:58:56.920 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-09-21T14:58:56.921 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-09-21T14:58:56.921 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-09-21T14:58:56.921 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-09-21T14:58:56.921 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-09-21T14:58:56.921 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-09-21T14:58:56.921 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T14:58:56.921 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T14:58:56.921 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-09-21T14:58:56.921 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-09-21T14:58:56.921 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 32, 2024-09-21T14:58:56.921 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 32, 2024-09-21T14:58:56.922 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 32, 2024-09-21T14:58:56.922 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 32, 2024-09-21T14:58:56.922 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 32, 2024-09-21T14:58:56.922 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-09-21T14:58:56.922 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-09-21T14:58:56.922 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-09-21T14:58:56.922 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-09-21T14:58:56.922 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-09-21T14:58:56.922 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-09-21T14:58:56.922 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-09-21T14:58:56.922 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.922 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "60", 2024-09-21T14:58:56.923 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-09-21T14:58:56.923 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T14:58:56.923 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T14:58:56.923 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-09-21T14:58:56.923 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-09-21T14:58:56.923 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 31, 2024-09-21T14:58:56.923 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 60, 2024-09-21T14:58:56.923 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-09-21T14:58:56.923 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-09-21T14:58:56.923 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-09-21T14:58:56.923 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-09-21T14:58:56.923 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-09-21T14:58:56.924 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-09-21T14:58:56.924 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-09-21T14:58:56.924 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-09-21T14:58:56.924 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-09-21T14:58:56.924 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-09-21T14:58:56.924 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-09-21T14:58:56.924 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T14:58:56.924 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T14:58:56.924 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T14:58:56.924 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-09-21T14:58:56.924 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-09-21T14:58:56.925 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-09-21T14:58:56.925 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-09-21T14:58:56.925 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-09-21T14:58:56.925 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.925 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-09-21T14:58:56.925 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-09-21T14:58:56.925 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-09-21T14:58:56.925 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-09-21T14:58:56.925 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-09-21T14:58:56.925 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T14:58:56.925 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-09-21T14:58:56.925 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-09-21T14:58:56.925 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-09-21T14:58:56.926 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-09-21T14:58:56.926 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-09-21T14:58:56.926 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2024-09-21T14:58:56.926 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-09-21T14:58:56.926 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd": {} 2024-09-21T14:58:56.926 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.926 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-09-21T14:58:56.926 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 2, 2024-09-21T14:58:56.926 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 2, 2024-09-21T14:58:56.926 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 1, 2024-09-21T14:58:56.926 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 2, 2024-09-21T14:58:56.926 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 2, 2024-09-21T14:58:56.926 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 0.88126564025878906, 2024-09-21T14:58:56.927 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 0.87999999523162842, 2024-09-21T14:58:56.927 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-21T14:58:56.927 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:56.927 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:56.927 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-21T14:58:56.927 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2024-09-21T14:58:56.927 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:56.927 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-09-21T14:58:56.927 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "a4a73316-3a94-42e9-9a51-deb46a66bb06", 2024-09-21T14:58:56.927 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-21T14:58:56.927 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-21T14:58:56.927 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-21T14:58:56.928 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-21T14:58:56.928 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-21T14:58:56.928 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-21T14:58:56.928 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2024-09-21T14:58:56.928 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 24, 2024-09-21T14:58:56.928 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-21T14:58:56.928 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-21T14:58:56.928 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-21T14:58:56.928 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:56.928 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:56.928 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:56.928 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6808", 2024-09-21T14:58:56.928 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4016672243 2024-09-21T14:58:56.929 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.929 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:56.929 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:56.929 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6809", 2024-09-21T14:58:56.929 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4016672243 2024-09-21T14:58:56.929 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:56.929 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:56.929 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.929 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-21T14:58:56.929 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:56.929 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:56.930 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:56.930 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6810", 2024-09-21T14:58:56.930 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4016672243 2024-09-21T14:58:56.930 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.930 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:56.930 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:56.930 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6811", 2024-09-21T14:58:56.930 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4016672243 2024-09-21T14:58:56.930 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:56.930 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:56.930 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.931 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-21T14:58:56.931 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:56.931 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:56.931 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:56.931 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6814", 2024-09-21T14:58:56.931 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4016672243 2024-09-21T14:58:56.931 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.931 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:56.931 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:56.931 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6815", 2024-09-21T14:58:56.931 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4016672243 2024-09-21T14:58:56.931 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:56.931 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:56.932 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.932 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-21T14:58:56.932 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:56.932 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:56.932 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:56.932 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6812", 2024-09-21T14:58:56.932 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4016672243 2024-09-21T14:58:56.932 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.932 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:56.932 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:56.932 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6813", 2024-09-21T14:58:56.932 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4016672243 2024-09-21T14:58:56.932 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:56.933 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:56.933 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.933 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6809/4016672243", 2024-09-21T14:58:56.933 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6811/4016672243", 2024-09-21T14:58:56.933 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6815/4016672243", 2024-09-21T14:58:56.933 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6813/4016672243", 2024-09-21T14:58:56.933 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-21T14:58:56.933 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-21T14:58:56.933 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-21T14:58:56.933 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:56.933 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.933 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:56.934 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-09-21T14:58:56.934 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "006757a0-94d1-4a9b-aef8-73634cc5828e", 2024-09-21T14:58:56.934 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-21T14:58:56.934 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-21T14:58:56.934 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-21T14:58:56.934 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-21T14:58:56.934 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-21T14:58:56.934 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-21T14:58:56.934 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2024-09-21T14:58:56.934 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 24, 2024-09-21T14:58:56.934 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-21T14:58:56.935 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-21T14:58:56.935 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-21T14:58:56.935 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:56.935 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:56.935 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:56.935 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6824", 2024-09-21T14:58:56.935 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 641090577 2024-09-21T14:58:56.935 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.935 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:56.935 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:56.935 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6825", 2024-09-21T14:58:56.935 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 641090577 2024-09-21T14:58:56.935 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:56.936 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:56.936 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.936 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-21T14:58:56.936 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:56.936 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:56.936 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:56.936 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6826", 2024-09-21T14:58:56.936 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 641090577 2024-09-21T14:58:56.936 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.936 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:56.936 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:56.936 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6827", 2024-09-21T14:58:56.936 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 641090577 2024-09-21T14:58:56.936 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:56.937 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:56.937 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.937 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-21T14:58:56.937 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:56.937 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:56.937 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:56.937 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6830", 2024-09-21T14:58:56.937 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 641090577 2024-09-21T14:58:56.937 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.937 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:56.937 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:56.937 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6831", 2024-09-21T14:58:56.937 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 641090577 2024-09-21T14:58:56.938 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:56.938 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:56.938 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.938 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-21T14:58:56.938 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:56.938 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:56.938 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:56.938 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6828", 2024-09-21T14:58:56.938 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 641090577 2024-09-21T14:58:56.938 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.938 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:56.939 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:56.939 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6829", 2024-09-21T14:58:56.939 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 641090577 2024-09-21T14:58:56.939 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:56.939 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:56.939 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.939 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6825/641090577", 2024-09-21T14:58:56.939 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6827/641090577", 2024-09-21T14:58:56.939 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6831/641090577", 2024-09-21T14:58:56.939 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6829/641090577", 2024-09-21T14:58:56.939 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-21T14:58:56.939 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-21T14:58:56.939 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-21T14:58:56.940 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:56.940 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.940 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:56.940 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-09-21T14:58:56.940 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "0b499a23-820e-4e6c-a558-adbf88b7014f", 2024-09-21T14:58:56.940 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-21T14:58:56.940 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-21T14:58:56.940 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-21T14:58:56.940 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-21T14:58:56.940 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-21T14:58:56.940 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-21T14:58:56.940 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2024-09-21T14:58:56.940 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 51, 2024-09-21T14:58:56.941 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-21T14:58:56.941 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-21T14:58:56.941 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-21T14:58:56.941 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:56.941 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:56.941 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:56.941 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6800", 2024-09-21T14:58:56.941 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2609154414 2024-09-21T14:58:56.941 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.941 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:56.941 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:56.941 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6801", 2024-09-21T14:58:56.942 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2609154414 2024-09-21T14:58:56.942 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:56.942 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:56.942 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.942 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-21T14:58:56.942 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:56.942 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:56.942 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:56.942 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6802", 2024-09-21T14:58:56.942 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2609154414 2024-09-21T14:58:56.942 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.942 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:56.943 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:56.943 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6803", 2024-09-21T14:58:56.943 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2609154414 2024-09-21T14:58:56.943 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:56.943 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:56.943 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.943 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-21T14:58:56.943 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:56.943 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:56.943 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:56.943 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6806", 2024-09-21T14:58:56.943 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2609154414 2024-09-21T14:58:56.944 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.944 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:56.944 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:56.944 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6807", 2024-09-21T14:58:56.944 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2609154414 2024-09-21T14:58:56.944 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:56.944 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:56.944 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.944 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-21T14:58:56.944 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:56.944 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:56.945 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:56.945 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6804", 2024-09-21T14:58:56.945 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2609154414 2024-09-21T14:58:56.945 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.945 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:56.945 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:56.945 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6805", 2024-09-21T14:58:56.945 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2609154414 2024-09-21T14:58:56.945 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:56.945 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:56.945 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.946 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6801/2609154414", 2024-09-21T14:58:56.946 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6803/2609154414", 2024-09-21T14:58:56.946 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6807/2609154414", 2024-09-21T14:58:56.946 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6805/2609154414", 2024-09-21T14:58:56.946 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-21T14:58:56.946 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-21T14:58:56.946 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-21T14:58:56.946 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:56.946 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.946 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:56.946 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-09-21T14:58:56.947 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "5a57b3dd-4a07-45b7-b159-3cac4c7eb262", 2024-09-21T14:58:56.947 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-21T14:58:56.947 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-21T14:58:56.947 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-21T14:58:56.947 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-21T14:58:56.947 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-21T14:58:56.947 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-21T14:58:56.947 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2024-09-21T14:58:56.947 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 51, 2024-09-21T14:58:56.947 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-21T14:58:56.947 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-21T14:58:56.948 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-21T14:58:56.948 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:56.948 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:56.948 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:56.948 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6816", 2024-09-21T14:58:56.948 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1462563376 2024-09-21T14:58:56.948 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.948 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:56.948 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:56.948 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6817", 2024-09-21T14:58:56.948 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1462563376 2024-09-21T14:58:56.948 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:56.949 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:56.949 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.949 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-21T14:58:56.949 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:56.949 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:56.949 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:56.949 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6818", 2024-09-21T14:58:56.949 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1462563376 2024-09-21T14:58:56.949 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.949 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:56.949 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:56.949 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6819", 2024-09-21T14:58:56.950 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1462563376 2024-09-21T14:58:56.950 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:56.950 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:56.950 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.950 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-21T14:58:56.950 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:56.950 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:56.950 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:56.950 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6822", 2024-09-21T14:58:56.950 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1462563376 2024-09-21T14:58:56.950 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.950 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:56.950 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:56.951 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6823", 2024-09-21T14:58:56.951 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1462563376 2024-09-21T14:58:56.951 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:56.951 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:56.951 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.951 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-21T14:58:56.951 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:56.951 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:56.951 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:56.951 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6820", 2024-09-21T14:58:56.951 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1462563376 2024-09-21T14:58:56.951 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.951 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:56.952 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:56.952 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6821", 2024-09-21T14:58:56.952 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1462563376 2024-09-21T14:58:56.952 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:56.952 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:56.952 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.952 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6817/1462563376", 2024-09-21T14:58:56.952 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6819/1462563376", 2024-09-21T14:58:56.952 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6823/1462563376", 2024-09-21T14:58:56.952 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6821/1462563376", 2024-09-21T14:58:56.952 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-21T14:58:56.952 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-21T14:58:56.952 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-21T14:58:56.953 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:56.953 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.953 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:56.953 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-09-21T14:58:56.953 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "39a00807-7517-474e-8842-623dd7093cc5", 2024-09-21T14:58:56.953 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-21T14:58:56.953 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-21T14:58:56.953 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-21T14:58:56.953 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-21T14:58:56.953 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-21T14:58:56.953 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-21T14:58:56.954 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2024-09-21T14:58:56.954 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 24, 2024-09-21T14:58:56.954 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-21T14:58:56.954 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-21T14:58:56.954 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-21T14:58:56.954 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:56.954 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:56.954 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:56.954 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6801", 2024-09-21T14:58:56.954 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 144412042 2024-09-21T14:58:56.954 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.955 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:56.955 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:56.955 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6803", 2024-09-21T14:58:56.955 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 144412042 2024-09-21T14:58:56.955 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:56.955 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:56.955 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.955 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-21T14:58:56.955 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:56.955 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:56.955 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:56.955 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6805", 2024-09-21T14:58:56.955 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 144412042 2024-09-21T14:58:56.956 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.956 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:56.956 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:56.956 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6807", 2024-09-21T14:58:56.956 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 144412042 2024-09-21T14:58:56.956 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:56.956 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:56.956 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.956 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-21T14:58:56.956 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:56.956 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:56.956 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:56.956 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6812", 2024-09-21T14:58:56.956 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 144412042 2024-09-21T14:58:56.957 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.957 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:56.957 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:56.957 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6814", 2024-09-21T14:58:56.957 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 144412042 2024-09-21T14:58:56.957 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:56.957 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:56.957 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.957 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-21T14:58:56.957 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:56.957 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:56.957 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:56.957 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6809", 2024-09-21T14:58:56.958 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 144412042 2024-09-21T14:58:56.958 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.958 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:56.958 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:56.958 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6811", 2024-09-21T14:58:56.958 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 144412042 2024-09-21T14:58:56.958 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:56.958 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:56.958 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.958 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.46:6803/144412042", 2024-09-21T14:58:56.958 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.46:6807/144412042", 2024-09-21T14:58:56.958 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.46:6814/144412042", 2024-09-21T14:58:56.958 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.46:6811/144412042", 2024-09-21T14:58:56.959 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-21T14:58:56.959 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-21T14:58:56.959 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-21T14:58:56.959 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:56.959 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.959 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:56.959 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-09-21T14:58:56.959 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "edf9f045-ddeb-4504-a0fc-1a0e4e853550", 2024-09-21T14:58:56.959 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-21T14:58:56.959 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-21T14:58:56.959 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-21T14:58:56.959 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-21T14:58:56.959 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-21T14:58:56.959 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-21T14:58:56.960 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2024-09-21T14:58:56.960 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 51, 2024-09-21T14:58:56.960 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-21T14:58:56.960 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-21T14:58:56.960 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-21T14:58:56.960 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:56.960 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:56.960 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:56.960 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6800", 2024-09-21T14:58:56.960 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 849101442 2024-09-21T14:58:56.960 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.960 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:56.960 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:56.961 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6802", 2024-09-21T14:58:56.961 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 849101442 2024-09-21T14:58:56.961 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:56.961 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:56.961 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.961 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-21T14:58:56.961 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:56.961 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:56.961 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:56.961 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6804", 2024-09-21T14:58:56.961 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 849101442 2024-09-21T14:58:56.961 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.961 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:56.961 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:56.962 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6806", 2024-09-21T14:58:56.962 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 849101442 2024-09-21T14:58:56.962 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:56.962 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:56.962 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.962 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-21T14:58:56.962 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:56.962 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:56.962 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:56.962 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6813", 2024-09-21T14:58:56.962 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 849101442 2024-09-21T14:58:56.962 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.962 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:56.963 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:56.963 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6815", 2024-09-21T14:58:56.963 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 849101442 2024-09-21T14:58:56.963 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:56.963 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:56.963 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.963 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-21T14:58:56.963 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:56.963 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:56.963 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:56.963 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6808", 2024-09-21T14:58:56.963 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 849101442 2024-09-21T14:58:56.964 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.964 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:56.964 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:56.964 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6810", 2024-09-21T14:58:56.964 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 849101442 2024-09-21T14:58:56.964 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:56.964 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:56.964 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.964 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.46:6802/849101442", 2024-09-21T14:58:56.964 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.46:6806/849101442", 2024-09-21T14:58:56.964 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.46:6815/849101442", 2024-09-21T14:58:56.965 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.46:6810/849101442", 2024-09-21T14:58:56.965 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-21T14:58:56.965 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-21T14:58:56.965 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-21T14:58:56.965 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:56.965 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.965 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:56.965 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-09-21T14:58:56.965 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "c820ea6b-88fe-4421-83ec-17c466691fea", 2024-09-21T14:58:56.965 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-21T14:58:56.965 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-21T14:58:56.965 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-21T14:58:56.966 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 0.0501251220703125, 2024-09-21T14:58:56.966 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-21T14:58:56.966 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-21T14:58:56.966 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2024-09-21T14:58:56.966 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 24, 2024-09-21T14:58:56.966 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-21T14:58:56.966 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-21T14:58:56.966 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-21T14:58:56.966 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:56.966 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:56.966 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:56.967 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6817", 2024-09-21T14:58:56.967 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 847189172 2024-09-21T14:58:56.967 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.967 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:56.967 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:56.967 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6818", 2024-09-21T14:58:56.967 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 847189172 2024-09-21T14:58:56.967 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:56.967 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:56.967 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.967 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-21T14:58:56.968 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:56.968 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:56.968 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:56.968 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6820", 2024-09-21T14:58:56.968 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 847189172 2024-09-21T14:58:56.968 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.968 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:56.968 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:56.968 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6822", 2024-09-21T14:58:56.968 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 847189172 2024-09-21T14:58:56.968 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:56.969 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:56.969 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.969 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-21T14:58:56.969 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:56.969 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:56.969 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:56.969 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6827", 2024-09-21T14:58:56.969 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 847189172 2024-09-21T14:58:56.969 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.969 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:56.969 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:56.969 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6828", 2024-09-21T14:58:56.970 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 847189172 2024-09-21T14:58:56.970 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:56.970 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:56.970 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.970 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-21T14:58:56.970 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:56.970 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:56.970 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:56.970 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6823", 2024-09-21T14:58:56.970 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 847189172 2024-09-21T14:58:56.970 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.971 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:56.971 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:56.971 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6825", 2024-09-21T14:58:56.971 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 847189172 2024-09-21T14:58:56.971 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:56.971 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:56.971 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.971 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.46:6818/847189172", 2024-09-21T14:58:56.971 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.46:6822/847189172", 2024-09-21T14:58:56.971 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.46:6828/847189172", 2024-09-21T14:58:56.971 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.46:6825/847189172", 2024-09-21T14:58:56.971 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-21T14:58:56.971 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-21T14:58:56.972 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-21T14:58:56.972 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:56.972 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.972 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:56.972 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-09-21T14:58:56.972 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "3a2f82a4-28a0-4fcb-9d6b-2285fa4d07ac", 2024-09-21T14:58:56.972 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-21T14:58:56.972 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-21T14:58:56.972 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 0.95001220703125, 2024-09-21T14:58:56.972 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-21T14:58:56.972 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-21T14:58:56.972 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-21T14:58:56.972 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 15, 2024-09-21T14:58:56.973 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 24, 2024-09-21T14:58:56.973 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-21T14:58:56.973 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-21T14:58:56.973 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-21T14:58:56.973 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:56.973 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:56.973 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:56.973 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6816", 2024-09-21T14:58:56.973 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3894963553 2024-09-21T14:58:56.973 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.973 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:56.973 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:56.974 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6819", 2024-09-21T14:58:56.974 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3894963553 2024-09-21T14:58:56.974 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:56.974 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:56.974 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.974 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-21T14:58:56.974 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:56.974 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:56.974 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:56.974 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6821", 2024-09-21T14:58:56.974 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3894963553 2024-09-21T14:58:56.974 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.975 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:56.975 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:56.975 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6824", 2024-09-21T14:58:56.975 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3894963553 2024-09-21T14:58:56.975 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:56.975 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:56.975 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.975 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-21T14:58:56.975 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:56.975 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:56.975 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:56.975 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6830", 2024-09-21T14:58:56.975 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3894963553 2024-09-21T14:58:56.976 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.976 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:56.976 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:56.976 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6831", 2024-09-21T14:58:56.976 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3894963553 2024-09-21T14:58:56.976 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:56.976 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:56.976 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.976 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-21T14:58:56.976 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:58:56.976 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:56.976 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:58:56.977 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6826", 2024-09-21T14:58:56.977 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3894963553 2024-09-21T14:58:56.977 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.977 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:56.977 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:58:56.977 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6829", 2024-09-21T14:58:56.977 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3894963553 2024-09-21T14:58:56.977 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:56.977 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:56.977 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.977 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.46:6819/3894963553", 2024-09-21T14:58:56.977 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.46:6824/3894963553", 2024-09-21T14:58:56.977 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.46:6831/3894963553", 2024-09-21T14:58:56.978 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.46:6829/3894963553", 2024-09-21T14:58:56.978 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-21T14:58:56.978 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-21T14:58:56.978 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-21T14:58:56.978 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:56.978 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:56.978 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-21T14:58:56.978 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_xinfo": [ 2024-09-21T14:58:56.978 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:56.978 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-09-21T14:58:56.978 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-21T14:58:56.978 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-21T14:58:56.978 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-21T14:58:56.979 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-21T14:58:56.979 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-21T14:58:56.979 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-21T14:57:57.458317+0000", 2024-09-21T14:58:56.979 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-21T14:58:56.979 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.979 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:56.979 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-09-21T14:58:56.979 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-21T14:58:56.979 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-21T14:58:56.979 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-21T14:58:56.979 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-21T14:58:56.979 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-21T14:58:56.980 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-21T14:57:57.907624+0000", 2024-09-21T14:58:56.980 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-21T14:58:56.980 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.980 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:56.980 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-09-21T14:58:56.980 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-21T14:58:56.980 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-21T14:58:56.980 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-21T14:58:56.980 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-21T14:58:56.980 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-21T14:58:56.980 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-21T14:57:57.278817+0000", 2024-09-21T14:58:56.980 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-21T14:58:56.981 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.981 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:56.981 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-09-21T14:58:56.981 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-21T14:58:56.981 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-21T14:58:56.981 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-21T14:58:56.981 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-21T14:58:56.981 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-21T14:58:56.981 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-21T14:57:57.500449+0000", 2024-09-21T14:58:56.981 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-21T14:58:56.981 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.981 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:56.981 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-09-21T14:58:56.982 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-21T14:58:56.982 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-21T14:58:56.982 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-21T14:58:56.982 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-21T14:58:56.982 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-21T14:58:56.982 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-21T14:57:57.504325+0000", 2024-09-21T14:58:56.982 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-21T14:58:56.982 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.982 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:56.982 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-09-21T14:58:56.982 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-21T14:58:56.982 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-21T14:58:56.982 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-21T14:58:56.983 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-21T14:58:56.983 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-21T14:58:56.983 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-21T14:57:57.777525+0000", 2024-09-21T14:58:56.983 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-21T14:58:56.983 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.983 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:56.983 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-09-21T14:58:56.983 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-21T14:58:56.983 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-21T14:58:56.983 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-21T14:58:56.983 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-21T14:58:56.984 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-21T14:58:56.984 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-21T14:57:57.698961+0000", 2024-09-21T14:58:56.984 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-21T14:58:56.984 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.984 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:56.984 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-09-21T14:58:56.984 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-21T14:58:56.984 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-21T14:58:56.984 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-21T14:58:56.984 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-21T14:58:56.984 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-21T14:58:56.985 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-21T14:57:57.932625+0000", 2024-09-21T14:58:56.985 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-21T14:58:56.985 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:56.985 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-21T14:58:56.985 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap": [ 2024-09-21T14:58:56.985 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:56.985 INFO:teuthology.orchestra.run.smithi032.stdout: "pgid": "3.2", 2024-09-21T14:58:56.985 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2024-09-21T14:58:56.985 INFO:teuthology.orchestra.run.smithi032.stdout: 2, 2024-09-21T14:58:56.985 INFO:teuthology.orchestra.run.smithi032.stdout: 3 2024-09-21T14:58:56.985 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:56.985 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:56.985 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-21T14:58:56.986 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_items": [], 2024-09-21T14:58:56.986 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_primaries": [], 2024-09-21T14:58:56.986 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_temp": [], 2024-09-21T14:58:56.986 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_temp": [], 2024-09-21T14:58:56.986 INFO:teuthology.orchestra.run.smithi032.stdout: "blocklist": {}, 2024-09-21T14:58:56.986 INFO:teuthology.orchestra.run.smithi032.stdout: "range_blocklist": {}, 2024-09-21T14:58:56.986 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profiles": { 2024-09-21T14:58:56.986 INFO:teuthology.orchestra.run.smithi032.stdout: "default": { 2024-09-21T14:58:56.986 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2024-09-21T14:58:56.986 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2024-09-21T14:58:56.986 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "1", 2024-09-21T14:58:56.987 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2024-09-21T14:58:56.987 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van" 2024-09-21T14:58:56.987 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:56.987 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.987 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps_queue": [ 2024-09-21T14:58:56.987 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:56.987 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 3, 2024-09-21T14:58:56.987 INFO:teuthology.orchestra.run.smithi032.stdout: "snaps": [ 2024-09-21T14:58:56.987 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:56.987 INFO:teuthology.orchestra.run.smithi032.stdout: "begin": 26, 2024-09-21T14:58:56.987 INFO:teuthology.orchestra.run.smithi032.stdout: "length": 1 2024-09-21T14:58:56.988 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:56.988 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:56.988 INFO:teuthology.orchestra.run.smithi032.stdout: "begin": 29, 2024-09-21T14:58:56.988 INFO:teuthology.orchestra.run.smithi032.stdout: "length": 1 2024-09-21T14:58:56.988 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:56.988 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:56.988 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:56.988 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-21T14:58:56.988 INFO:teuthology.orchestra.run.smithi032.stdout: "new_removed_snaps": [], 2024-09-21T14:58:56.988 INFO:teuthology.orchestra.run.smithi032.stdout: "new_purged_snaps": [], 2024-09-21T14:58:56.988 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_node_flags": {}, 2024-09-21T14:58:56.988 INFO:teuthology.orchestra.run.smithi032.stdout: "device_class_flags": {}, 2024-09-21T14:58:56.989 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode": { 2024-09-21T14:58:56.989 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_enabled": false, 2024-09-21T14:58:56.989 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_bucket_count": 0, 2024-09-21T14:58:56.989 INFO:teuthology.orchestra.run.smithi032.stdout: "degraded_stretch_mode": 0, 2024-09-21T14:58:56.989 INFO:teuthology.orchestra.run.smithi032.stdout: "recovering_stretch_mode": 0, 2024-09-21T14:58:56.989 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_bucket": 0 2024-09-21T14:58:56.989 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:56.989 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:58:56.991 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:56.991 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:56.992 INFO:tasks.thrashosds.thrasher:j is {'epoch': 60, 'fsid': 'caf51e55-9234-4a26-bcd2-d60811962560', 'created': '2024-09-21T14:57:50.338822+0000', 'modified': '2024-09-21T14:58:56.229252+0000', 'last_up_change': '2024-09-21T14:58:01.177141+0000', 'last_in_change': '2024-09-21T14:57:52.442036+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T14:58:01.842810+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.8812656402587891, 'average_primary_affinity': 0.8799999952316284, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-21T14:58:08.284925+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 22, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 0.8812656402587891, 'average_primary_affinity': 0.8799999952316284, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-21T14:58:17.510709+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 31, 'snap_epoch': 60, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.8812656402587891, 'average_primary_affinity': 0.8799999952316284, 'average_primary_affinity_weighted': 0.8799999952316284}}], 'osds': [{'osd': 0, 'uuid': 'a4a73316-3a94-42e9-9a51-deb46a66bb06', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6808', 'nonce': 4016672243}, {'type': 'v1', 'addr': '172.21.15.32:6809', 'nonce': 4016672243}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6810', 'nonce': 4016672243}, {'type': 'v1', 'addr': '172.21.15.32:6811', 'nonce': 4016672243}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6814', 'nonce': 4016672243}, {'type': 'v1', 'addr': '172.21.15.32:6815', 'nonce': 4016672243}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6812', 'nonce': 4016672243}, {'type': 'v1', 'addr': '172.21.15.32:6813', 'nonce': 4016672243}]}, 'public_addr': '172.21.15.32:6809/4016672243', 'cluster_addr': '172.21.15.32:6811/4016672243', 'heartbeat_back_addr': '172.21.15.32:6815/4016672243', 'heartbeat_front_addr': '172.21.15.32:6813/4016672243', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '006757a0-94d1-4a9b-aef8-73634cc5828e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6824', 'nonce': 641090577}, {'type': 'v1', 'addr': '172.21.15.32:6825', 'nonce': 641090577}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6826', 'nonce': 641090577}, {'type': 'v1', 'addr': '172.21.15.32:6827', 'nonce': 641090577}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6830', 'nonce': 641090577}, {'type': 'v1', 'addr': '172.21.15.32:6831', 'nonce': 641090577}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6828', 'nonce': 641090577}, {'type': 'v1', 'addr': '172.21.15.32:6829', 'nonce': 641090577}]}, 'public_addr': '172.21.15.32:6825/641090577', 'cluster_addr': '172.21.15.32:6827/641090577', 'heartbeat_back_addr': '172.21.15.32:6831/641090577', 'heartbeat_front_addr': '172.21.15.32:6829/641090577', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '0b499a23-820e-4e6c-a558-adbf88b7014f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 51, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6800', 'nonce': 2609154414}, {'type': 'v1', 'addr': '172.21.15.32:6801', 'nonce': 2609154414}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6802', 'nonce': 2609154414}, {'type': 'v1', 'addr': '172.21.15.32:6803', 'nonce': 2609154414}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6806', 'nonce': 2609154414}, {'type': 'v1', 'addr': '172.21.15.32:6807', 'nonce': 2609154414}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6804', 'nonce': 2609154414}, {'type': 'v1', 'addr': '172.21.15.32:6805', 'nonce': 2609154414}]}, 'public_addr': '172.21.15.32:6801/2609154414', 'cluster_addr': '172.21.15.32:6803/2609154414', 'heartbeat_back_addr': '172.21.15.32:6807/2609154414', 'heartbeat_front_addr': '172.21.15.32:6805/2609154414', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '5a57b3dd-4a07-45b7-b159-3cac4c7eb262', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 51, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6816', 'nonce': 1462563376}, {'type': 'v1', 'addr': '172.21.15.32:6817', 'nonce': 1462563376}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6818', 'nonce': 1462563376}, {'type': 'v1', 'addr': '172.21.15.32:6819', 'nonce': 1462563376}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6822', 'nonce': 1462563376}, {'type': 'v1', 'addr': '172.21.15.32:6823', 'nonce': 1462563376}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6820', 'nonce': 1462563376}, {'type': 'v1', 'addr': '172.21.15.32:6821', 'nonce': 1462563376}]}, 'public_addr': '172.21.15.32:6817/1462563376', 'cluster_addr': '172.21.15.32:6819/1462563376', 'heartbeat_back_addr': '172.21.15.32:6823/1462563376', 'heartbeat_front_addr': '172.21.15.32:6821/1462563376', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '39a00807-7517-474e-8842-623dd7093cc5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6801', 'nonce': 144412042}, {'type': 'v1', 'addr': '172.21.15.46:6803', 'nonce': 144412042}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6805', 'nonce': 144412042}, {'type': 'v1', 'addr': '172.21.15.46:6807', 'nonce': 144412042}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6812', 'nonce': 144412042}, {'type': 'v1', 'addr': '172.21.15.46:6814', 'nonce': 144412042}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6809', 'nonce': 144412042}, {'type': 'v1', 'addr': '172.21.15.46:6811', 'nonce': 144412042}]}, 'public_addr': '172.21.15.46:6803/144412042', 'cluster_addr': '172.21.15.46:6807/144412042', 'heartbeat_back_addr': '172.21.15.46:6814/144412042', 'heartbeat_front_addr': '172.21.15.46:6811/144412042', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'edf9f045-ddeb-4504-a0fc-1a0e4e853550', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 51, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6800', 'nonce': 849101442}, {'type': 'v1', 'addr': '172.21.15.46:6802', 'nonce': 849101442}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6804', 'nonce': 849101442}, {'type': 'v1', 'addr': '172.21.15.46:6806', 'nonce': 849101442}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6813', 'nonce': 849101442}, {'type': 'v1', 'addr': '172.21.15.46:6815', 'nonce': 849101442}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6808', 'nonce': 849101442}, {'type': 'v1', 'addr': '172.21.15.46:6810', 'nonce': 849101442}]}, 'public_addr': '172.21.15.46:6802/849101442', 'cluster_addr': '172.21.15.46:6806/849101442', 'heartbeat_back_addr': '172.21.15.46:6815/849101442', 'heartbeat_front_addr': '172.21.15.46:6810/849101442', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'c820ea6b-88fe-4421-83ec-17c466691fea', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.0501251220703125, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6817', 'nonce': 847189172}, {'type': 'v1', 'addr': '172.21.15.46:6818', 'nonce': 847189172}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6820', 'nonce': 847189172}, {'type': 'v1', 'addr': '172.21.15.46:6822', 'nonce': 847189172}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6827', 'nonce': 847189172}, {'type': 'v1', 'addr': '172.21.15.46:6828', 'nonce': 847189172}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6823', 'nonce': 847189172}, {'type': 'v1', 'addr': '172.21.15.46:6825', 'nonce': 847189172}]}, 'public_addr': '172.21.15.46:6818/847189172', 'cluster_addr': '172.21.15.46:6822/847189172', 'heartbeat_back_addr': '172.21.15.46:6828/847189172', 'heartbeat_front_addr': '172.21.15.46:6825/847189172', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '3a2f82a4-28a0-4fcb-9d6b-2285fa4d07ac', 'up': 1, 'in': 1, 'weight': 0.95001220703125, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6816', 'nonce': 3894963553}, {'type': 'v1', 'addr': '172.21.15.46:6819', 'nonce': 3894963553}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6821', 'nonce': 3894963553}, {'type': 'v1', 'addr': '172.21.15.46:6824', 'nonce': 3894963553}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6830', 'nonce': 3894963553}, {'type': 'v1', 'addr': '172.21.15.46:6831', 'nonce': 3894963553}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6826', 'nonce': 3894963553}, {'type': 'v1', 'addr': '172.21.15.46:6829', 'nonce': 3894963553}]}, 'public_addr': '172.21.15.46:6819/3894963553', 'cluster_addr': '172.21.15.46:6824/3894963553', 'heartbeat_back_addr': '172.21.15.46:6831/3894963553', 'heartbeat_front_addr': '172.21.15.46:6829/3894963553', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:57:57.458317+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:57:57.907624+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:57:57.278817+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:57:57.500449+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:57:57.504325+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:57:57.777525+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:57:57.698961+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:57:57.932625+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.2', 'osds': [2, 3]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 26, 'length': 1}, {'begin': 29, 'length': 1}]}], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-21T14:58:56.992 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T14:58:56.993 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T14:58:56.993 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:58:56.988+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:58:57.091 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:58:57.089+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:58:57.115 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:57.115 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:57.126 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T14:58:57.192 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:58:57.189+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:58:57.231 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:57.231 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:57.240 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T14:58:57.292 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:58:57.289+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:58:57.347 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-21T14:58:57.352 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-09-21T14:58:57.363 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:57.363 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:57.369 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":82,"stamp":"2024-09-21T14:58:57.246152+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":281663140,"num_objects":479,"num_object_clones":0,"num_object_copies":958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":479,"num_whiteouts":0,"num_read":28575,"num_read_kb":223162,"num_write":5646,"num_write_kb":144000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1403348,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4024,"ondisk_log_size":4024,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":315524,"kb_used_data":21516,"kb_used_omap":12,"kb_used_meta":293875,"kb_avail":754659196,"statfs":{"total":773094113280,"available":772771016704,"internally_reserved":0,"allocated":22032384,"data_stored":176108648,"data_compressed":2642249,"data_compressed_allocated":14471168,"data_compressed_original":172535302,"omap_allocated":12744,"internal_metadata":300928568},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":95993299,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":3398,"num_read_kb":35546,"num_write":1069,"num_write_kb":6023,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.033300"},"pg_stats":[{"pgid":"3.1f","version":"59'160","reported_seq":570,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T14:58:55.747782+0000","last_change":"2024-09-21T14:58:55.286361+0000","last_active":"2024-09-21T14:58:55.747782+0000","last_peered":"2024-09-21T14:58:55.747782+0000","last_clean":"2024-09-21T14:58:55.747782+0000","last_became_active":"2024-09-21T14:58:19.410055+0000","last_became_peered":"2024-09-21T14:58:19.410055+0000","last_unstale":"2024-09-21T14:58:55.747782+0000","last_undegraded":"2024-09-21T14:58:55.747782+0000","last_fullsized":"2024-09-21T14:58:55.747782+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:49:13.944219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016915649999999999,"stat_sum":{"num_bytes":1583395,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":853,"num_read_kb":2924,"num_write":227,"num_write_kb":2877,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1a","length":"1"},{"start":"1d","length":"1"}]},{"pgid":"3.1e","version":"56'48","reported_seq":671,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T14:58:55.391402+0000","last_change":"2024-09-21T14:58:55.285624+0000","last_active":"2024-09-21T14:58:55.391402+0000","last_peered":"2024-09-21T14:58:55.391402+0000","last_clean":"2024-09-21T14:58:55.391402+0000","last_became_active":"2024-09-21T14:58:19.412353+0000","last_became_peered":"2024-09-21T14:58:19.412353+0000","last_unstale":"2024-09-21T14:58:55.391402+0000","last_undegraded":"2024-09-21T14:58:55.391402+0000","last_fullsized":"2024-09-21T14:58:55.391402+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:38:15.532795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015286119999999999,"stat_sum":{"num_bytes":8932350,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":534,"num_read_kb":7925,"num_write":62,"num_write_kb":4350,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1a","length":"1"},{"start":"1d","length":"1"}]},{"pgid":"3.1d","version":"56'56","reported_seq":796,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T14:58:55.125285+0000","last_change":"2024-09-21T14:58:55.125592+0000","last_active":"2024-09-21T14:58:55.125285+0000","last_peered":"2024-09-21T14:58:55.125285+0000","last_clean":"2024-09-21T14:58:55.125285+0000","last_became_active":"2024-09-21T14:58:19.411387+0000","last_became_peered":"2024-09-21T14:58:19.411387+0000","last_unstale":"2024-09-21T14:58:55.125285+0000","last_undegraded":"2024-09-21T14:58:55.125285+0000","last_fullsized":"2024-09-21T14:58:55.125285+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:37:57.597069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017451719999999999,"stat_sum":{"num_bytes":5866824,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":652,"num_read_kb":7966,"num_write":74,"num_write_kb":3111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1a","length":"1"},{"start":"1d","length":"1"}]},{"pgid":"3.1c","version":"59'100","reported_seq":688,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T14:58:55.312245+0000","last_change":"2024-09-21T14:58:55.111677+0000","last_active":"2024-09-21T14:58:55.312245+0000","last_peered":"2024-09-21T14:58:55.312245+0000","last_clean":"2024-09-21T14:58:55.312245+0000","last_became_active":"2024-09-21T14:58:19.411306+0000","last_became_peered":"2024-09-21T14:58:19.411306+0000","last_unstale":"2024-09-21T14:58:55.312245+0000","last_undegraded":"2024-09-21T14:58:55.312245+0000","last_fullsized":"2024-09-21T14:58:55.312245+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:20:27.079795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013063720000000001,"stat_sum":{"num_bytes":4519337,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":676,"num_read_kb":5556,"num_write":109,"num_write_kb":6725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1a","length":"1"},{"start":"1d","length":"1"}]},{"pgid":"3.1b","version":"56'135","reported_seq":763,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T14:58:55.384124+0000","last_change":"2024-09-21T14:58:55.118410+0000","last_active":"2024-09-21T14:58:55.384124+0000","last_peered":"2024-09-21T14:58:55.384124+0000","last_clean":"2024-09-21T14:58:55.384124+0000","last_became_active":"2024-09-21T14:58:19.413053+0000","last_became_peered":"2024-09-21T14:58:19.413053+0000","last_unstale":"2024-09-21T14:58:55.384124+0000","last_undegraded":"2024-09-21T14:58:55.384124+0000","last_fullsized":"2024-09-21T14:58:55.384124+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:07:12.901354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023564689999999999,"stat_sum":{"num_bytes":2801664,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1000,"num_read_kb":7014,"num_write":151,"num_write_kb":4545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1a","length":"1"},{"start":"1d","length":"1"}]},{"pgid":"3.1a","version":"56'127","reported_seq":1024,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T14:58:55.691215+0000","last_change":"2024-09-21T14:58:55.111916+0000","last_active":"2024-09-21T14:58:55.691215+0000","last_peered":"2024-09-21T14:58:55.691215+0000","last_clean":"2024-09-21T14:58:55.691215+0000","last_became_active":"2024-09-21T14:58:19.409415+0000","last_became_peered":"2024-09-21T14:58:19.409415+0000","last_unstale":"2024-09-21T14:58:55.691215+0000","last_undegraded":"2024-09-21T14:58:55.691215+0000","last_fullsized":"2024-09-21T14:58:55.691215+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:28:46.336301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001486624,"stat_sum":{"num_bytes":5049519,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":1237,"num_read_kb":6804,"num_write":141,"num_write_kb":4348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1a","length":"1"},{"start":"1d","length":"1"}]},{"pgid":"3.19","version":"59'136","reported_seq":1225,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T14:58:55.817743+0000","last_change":"2024-09-21T14:58:55.118319+0000","last_active":"2024-09-21T14:58:55.817743+0000","last_peered":"2024-09-21T14:58:55.817743+0000","last_clean":"2024-09-21T14:58:55.817743+0000","last_became_active":"2024-09-21T14:58:19.410062+0000","last_became_peered":"2024-09-21T14:58:19.410062+0000","last_unstale":"2024-09-21T14:58:55.817743+0000","last_undegraded":"2024-09-21T14:58:55.817743+0000","last_fullsized":"2024-09-21T14:58:55.817743+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":136,"log_dups_size":0,"ondisk_log_size":136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:12:02.923321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023120219999999999,"stat_sum":{"num_bytes":7196687,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":1556,"num_read_kb":10847,"num_write":154,"num_write_kb":5126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1a","length":"1"},{"start":"1d","length":"1"}]},{"pgid":"3.18","version":"59'107","reported_seq":707,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T14:58:55.751994+0000","last_change":"2024-09-21T14:58:55.287270+0000","last_active":"2024-09-21T14:58:55.751994+0000","last_peered":"2024-09-21T14:58:55.751994+0000","last_clean":"2024-09-21T14:58:55.751994+0000","last_became_active":"2024-09-21T14:58:19.409371+0000","last_became_peered":"2024-09-21T14:58:19.409371+0000","last_unstale":"2024-09-21T14:58:55.751994+0000","last_undegraded":"2024-09-21T14:58:55.751994+0000","last_fullsized":"2024-09-21T14:58:55.751994+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:35:27.143445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014270229999999999,"stat_sum":{"num_bytes":3098214,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":725,"num_read_kb":4805,"num_write":109,"num_write_kb":3280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1a","length":"1"},{"start":"1d","length":"1"}]},{"pgid":"3.17","version":"56'49","reported_seq":414,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T14:58:55.285743+0000","last_change":"2024-09-21T14:58:55.286240+0000","last_active":"2024-09-21T14:58:55.285743+0000","last_peered":"2024-09-21T14:58:55.285743+0000","last_clean":"2024-09-21T14:58:55.285743+0000","last_became_active":"2024-09-21T14:58:19.410306+0000","last_became_peered":"2024-09-21T14:58:19.410306+0000","last_unstale":"2024-09-21T14:58:55.285743+0000","last_undegraded":"2024-09-21T14:58:55.285743+0000","last_fullsized":"2024-09-21T14:58:55.285743+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:33:50.089165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002023618,"stat_sum":{"num_bytes":1815182,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":276,"num_read_kb":3251,"num_write":57,"num_write_kb":2731,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1a","length":"1"},{"start":"1d","length":"1"}]},{"pgid":"3.16","version":"58'351","reported_seq":960,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T14:58:55.113218+0000","last_change":"2024-09-21T14:58:55.113218+0000","last_active":"2024-09-21T14:58:55.113218+0000","last_peered":"2024-09-21T14:58:55.113218+0000","last_clean":"2024-09-21T14:58:55.113218+0000","last_became_active":"2024-09-21T14:58:19.413935+0000","last_became_peered":"2024-09-21T14:58:19.413935+0000","last_unstale":"2024-09-21T14:58:55.113218+0000","last_undegraded":"2024-09-21T14:58:55.113218+0000","last_fullsized":"2024-09-21T14:58:55.113218+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":351,"log_dups_size":0,"ondisk_log_size":351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:15:28.307418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080905599999999995,"stat_sum":{"num_bytes":4397029,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":2169,"num_read_kb":7976,"num_write":576,"num_write_kb":4224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"1a","length":"1"},{"start":"1d","length":"1"}]},{"pgid":"3.15","version":"59'125","reported_seq":738,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T14:58:55.788973+0000","last_change":"2024-09-21T14:58:55.127686+0000","last_active":"2024-09-21T14:58:55.788973+0000","last_peered":"2024-09-21T14:58:55.788973+0000","last_clean":"2024-09-21T14:58:55.788973+0000","last_became_active":"2024-09-21T14:58:19.410687+0000","last_became_peered":"2024-09-21T14:58:19.410687+0000","last_unstale":"2024-09-21T14:58:55.788973+0000","last_undegraded":"2024-09-21T14:58:55.788973+0000","last_fullsized":"2024-09-21T14:58:55.788973+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":125,"log_dups_size":0,"ondisk_log_size":125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:21:42.226982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072615400000000001,"stat_sum":{"num_bytes":16494064,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":749,"num_read_kb":6990,"num_write":164,"num_write_kb":6940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1a","length":"1"},{"start":"1d","length":"1"}]},{"pgid":"3.14","version":"59'160","reported_seq":407,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T14:58:55.853129+0000","last_change":"2024-09-21T14:58:55.124688+0000","last_active":"2024-09-21T14:58:55.853129+0000","last_peered":"2024-09-21T14:58:55.853129+0000","last_clean":"2024-09-21T14:58:55.853129+0000","last_became_active":"2024-09-21T14:58:19.407310+0000","last_became_peered":"2024-09-21T14:58:19.407310+0000","last_unstale":"2024-09-21T14:58:55.853129+0000","last_undegraded":"2024-09-21T14:58:55.853129+0000","last_fullsized":"2024-09-21T14:58:55.853129+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:47:46.079115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001245557,"stat_sum":{"num_bytes":13228018,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":610,"num_read_kb":1558,"num_write":223,"num_write_kb":1676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1a","length":"1"},{"start":"1d","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":52,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-21T14:58:54.232407+0000","last_change":"2024-09-21T14:58:11.348177+0000","last_active":"2024-09-21T14:58:54.232407+0000","last_peered":"2024-09-21T14:58:54.232407+0000","last_clean":"2024-09-21T14:58:54.232407+0000","last_became_active":"2024-09-21T14:58:09.328417+0000","last_became_peered":"2024-09-21T14:58:09.328417+0000","last_unstale":"2024-09-21T14:58:54.232407+0000","last_undegraded":"2024-09-21T14:58:54.232407+0000","last_fullsized":"2024-09-21T14:58:54.232407+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:52:22.671185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018917490000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"59'134","reported_seq":895,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T14:58:55.810387+0000","last_change":"2024-09-21T14:58:55.119239+0000","last_active":"2024-09-21T14:58:55.810387+0000","last_peered":"2024-09-21T14:58:55.810387+0000","last_clean":"2024-09-21T14:58:55.810387+0000","last_became_active":"2024-09-21T14:58:19.403868+0000","last_became_peered":"2024-09-21T14:58:19.403868+0000","last_unstale":"2024-09-21T14:58:55.810387+0000","last_undegraded":"2024-09-21T14:58:55.810387+0000","last_fullsized":"2024-09-21T14:58:55.810387+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:52:08.672416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032380759999999999,"stat_sum":{"num_bytes":9070454,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":1150,"num_read_kb":7512,"num_write":170,"num_write_kb":3739,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1a","length":"1"},{"start":"1d","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":53,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-21T14:58:54.237641+0000","last_change":"2024-09-21T14:58:11.405717+0000","last_active":"2024-09-21T14:58:54.237641+0000","last_peered":"2024-09-21T14:58:54.237641+0000","last_clean":"2024-09-21T14:58:54.237641+0000","last_became_active":"2024-09-21T14:58:09.326849+0000","last_became_peered":"2024-09-21T14:58:09.326849+0000","last_unstale":"2024-09-21T14:58:54.237641+0000","last_undegraded":"2024-09-21T14:58:54.237641+0000","last_fullsized":"2024-09-21T14:58:54.237641+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:29:42.893819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012560399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"59'115","reported_seq":836,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T14:58:55.118978+0000","last_change":"2024-09-21T14:58:55.118978+0000","last_active":"2024-09-21T14:58:55.118978+0000","last_peered":"2024-09-21T14:58:55.118978+0000","last_clean":"2024-09-21T14:58:55.118978+0000","last_became_active":"2024-09-21T14:58:19.410954+0000","last_became_peered":"2024-09-21T14:58:19.410954+0000","last_unstale":"2024-09-21T14:58:55.118978+0000","last_undegraded":"2024-09-21T14:58:55.118978+0000","last_fullsized":"2024-09-21T14:58:55.118978+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:26:59.084415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025360600000000001,"stat_sum":{"num_bytes":16400317,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":863,"num_read_kb":8751,"num_write":156,"num_write_kb":3948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1a","length":"1"},{"start":"1d","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":52,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-21T14:58:54.232292+0000","last_change":"2024-09-21T14:58:11.348182+0000","last_active":"2024-09-21T14:58:54.232292+0000","last_peered":"2024-09-21T14:58:54.232292+0000","last_clean":"2024-09-21T14:58:54.232292+0000","last_became_active":"2024-09-21T14:58:09.329036+0000","last_became_peered":"2024-09-21T14:58:09.329036+0000","last_unstale":"2024-09-21T14:58:54.232292+0000","last_undegraded":"2024-09-21T14:58:54.232292+0000","last_fullsized":"2024-09-21T14:58:54.232292+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:24:45.719970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019338770000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"56'70","reported_seq":671,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T14:58:55.285504+0000","last_change":"2024-09-21T14:58:55.285504+0000","last_active":"2024-09-21T14:58:55.285504+0000","last_peered":"2024-09-21T14:58:55.285504+0000","last_clean":"2024-09-21T14:58:55.285504+0000","last_became_active":"2024-09-21T14:58:47.169989+0000","last_became_peered":"2024-09-21T14:58:47.169989+0000","last_unstale":"2024-09-21T14:58:55.285504+0000","last_undegraded":"2024-09-21T14:58:55.285504+0000","last_fullsized":"2024-09-21T14:58:55.285504+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:52:16.238600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015747580000000001,"stat_sum":{"num_bytes":5996544,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":542,"num_read_kb":7166,"num_write":84,"num_write_kb":6267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1a","length":"1"},{"start":"1d","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":53,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-21T14:58:54.231211+0000","last_change":"2024-09-21T14:58:11.406176+0000","last_active":"2024-09-21T14:58:54.231211+0000","last_peered":"2024-09-21T14:58:54.231211+0000","last_clean":"2024-09-21T14:58:54.231211+0000","last_became_active":"2024-09-21T14:58:09.777465+0000","last_became_peered":"2024-09-21T14:58:09.777465+0000","last_unstale":"2024-09-21T14:58:54.231211+0000","last_undegraded":"2024-09-21T14:58:54.231211+0000","last_fullsized":"2024-09-21T14:58:54.231211+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:55:04.667816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095704000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"56'72","reported_seq":689,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T14:58:55.383619+0000","last_change":"2024-09-21T14:58:55.113770+0000","last_active":"2024-09-21T14:58:55.383619+0000","last_peered":"2024-09-21T14:58:55.383619+0000","last_clean":"2024-09-21T14:58:55.383619+0000","last_became_active":"2024-09-21T14:58:19.408971+0000","last_became_peered":"2024-09-21T14:58:19.408971+0000","last_unstale":"2024-09-21T14:58:55.383619+0000","last_undegraded":"2024-09-21T14:58:55.383619+0000","last_fullsized":"2024-09-21T14:58:55.383619+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:29:52.642121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011371980000000001,"stat_sum":{"num_bytes":5741545,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":667,"num_read_kb":6820,"num_write":111,"num_write_kb":4606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"1a","length":"1"},{"start":"1d","length":"1"}]},{"pgid":"3.3","version":"56'250","reported_seq":1147,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T14:58:55.382860+0000","last_change":"2024-09-21T14:58:55.128204+0000","last_active":"2024-09-21T14:58:55.382860+0000","last_peered":"2024-09-21T14:58:55.382860+0000","last_clean":"2024-09-21T14:58:55.382860+0000","last_became_active":"2024-09-21T14:58:19.417453+0000","last_became_peered":"2024-09-21T14:58:19.417453+0000","last_unstale":"2024-09-21T14:58:55.382860+0000","last_undegraded":"2024-09-21T14:58:55.382860+0000","last_fullsized":"2024-09-21T14:58:55.382860+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":250,"log_dups_size":0,"ondisk_log_size":250,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:19:27.087272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093747999999999998,"stat_sum":{"num_bytes":7758521,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":1790,"num_read_kb":12038,"num_write":456,"num_write_kb":4772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1a","length":"1"},{"start":"1d","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":54,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-21T14:58:54.232050+0000","last_change":"2024-09-21T14:58:11.347504+0000","last_active":"2024-09-21T14:58:54.232050+0000","last_peered":"2024-09-21T14:58:54.232050+0000","last_clean":"2024-09-21T14:58:54.232050+0000","last_became_active":"2024-09-21T14:58:09.775981+0000","last_became_peered":"2024-09-21T14:58:09.775981+0000","last_unstale":"2024-09-21T14:58:54.232050+0000","last_undegraded":"2024-09-21T14:58:54.232050+0000","last_fullsized":"2024-09-21T14:58:54.232050+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:47:16.286652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012568880000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"57'59","reported_seq":586,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T14:58:55.385173+0000","last_change":"2024-09-21T14:58:55.286828+0000","last_active":"2024-09-21T14:58:55.385173+0000","last_peered":"2024-09-21T14:58:55.385173+0000","last_clean":"2024-09-21T14:58:55.385173+0000","last_became_active":"2024-09-21T14:58:19.405811+0000","last_became_peered":"2024-09-21T14:58:19.405811+0000","last_unstale":"2024-09-21T14:58:55.385173+0000","last_undegraded":"2024-09-21T14:58:55.385173+0000","last_fullsized":"2024-09-21T14:58:55.385173+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:36:25.551607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021055430000000001,"stat_sum":{"num_bytes":5963883,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":451,"num_read_kb":6778,"num_write":81,"num_write_kb":4015,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1a","length":"1"},{"start":"1d","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":54,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T14:58:55.119896+0000","last_change":"2024-09-21T14:58:11.342210+0000","last_active":"2024-09-21T14:58:55.119896+0000","last_peered":"2024-09-21T14:58:55.119896+0000","last_clean":"2024-09-21T14:58:55.119896+0000","last_became_active":"2024-09-21T14:58:09.330624+0000","last_became_peered":"2024-09-21T14:58:09.330624+0000","last_unstale":"2024-09-21T14:58:55.119896+0000","last_undegraded":"2024-09-21T14:58:55.119896+0000","last_fullsized":"2024-09-21T14:58:55.119896+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:29:43.229538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001051151,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"59'56","reported_seq":606,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T14:58:55.729575+0000","last_change":"2024-09-21T14:58:55.113112+0000","last_active":"2024-09-21T14:58:55.729575+0000","last_peered":"2024-09-21T14:58:55.729575+0000","last_clean":"2024-09-21T14:58:55.729575+0000","last_became_active":"2024-09-21T14:58:47.176173+0000","last_became_peered":"2024-09-21T14:58:47.176173+0000","last_unstale":"2024-09-21T14:58:55.729575+0000","last_undegraded":"2024-09-21T14:58:55.729575+0000","last_fullsized":"2024-09-21T14:58:55.729575+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:08:13.453044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001656362,"stat_sum":{"num_bytes":21221076,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":454,"num_read_kb":4904,"num_write":63,"num_write_kb":4663,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"1a","length":"1"},{"start":"1d","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":52,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-21T14:58:54.231554+0000","last_change":"2024-09-21T14:58:11.346766+0000","last_active":"2024-09-21T14:58:54.231554+0000","last_peered":"2024-09-21T14:58:54.231554+0000","last_clean":"2024-09-21T14:58:54.231554+0000","last_became_active":"2024-09-21T14:58:09.329748+0000","last_became_peered":"2024-09-21T14:58:09.329748+0000","last_unstale":"2024-09-21T14:58:54.231554+0000","last_undegraded":"2024-09-21T14:58:54.231554+0000","last_fullsized":"2024-09-21T14:58:54.231554+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:31:20.267675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052698000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"56'58","reported_seq":996,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T14:58:55.113472+0000","last_change":"2024-09-21T14:58:55.113472+0000","last_active":"2024-09-21T14:58:55.113472+0000","last_peered":"2024-09-21T14:58:55.113472+0000","last_clean":"2024-09-21T14:58:55.113472+0000","last_became_active":"2024-09-21T14:58:41.401003+0000","last_became_peered":"2024-09-21T14:58:41.401003+0000","last_unstale":"2024-09-21T14:58:55.113472+0000","last_undegraded":"2024-09-21T14:58:55.113472+0000","last_fullsized":"2024-09-21T14:58:55.113472+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:05:37.644835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001017142,"stat_sum":{"num_bytes":11036693,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":780,"num_read_kb":7644,"num_write":59,"num_write_kb":2203,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1403348,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"1a","length":"1"},{"start":"1d","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":53,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-21T14:58:54.232242+0000","last_change":"2024-09-21T14:58:11.347700+0000","last_active":"2024-09-21T14:58:54.232242+0000","last_peered":"2024-09-21T14:58:54.232242+0000","last_clean":"2024-09-21T14:58:54.232242+0000","last_became_active":"2024-09-21T14:58:09.438842+0000","last_became_peered":"2024-09-21T14:58:09.438842+0000","last_unstale":"2024-09-21T14:58:54.232242+0000","last_undegraded":"2024-09-21T14:58:54.232242+0000","last_fullsized":"2024-09-21T14:58:54.232242+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:41:34.626249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001426194,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":201,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-21T14:58:54.241055+0000","last_change":"2024-09-21T14:58:03.616055+0000","last_active":"2024-09-21T14:58:54.241055+0000","last_peered":"2024-09-21T14:58:54.241055+0000","last_clean":"2024-09-21T14:58:54.241055+0000","last_became_active":"2024-09-21T14:58:03.613214+0000","last_became_peered":"2024-09-21T14:58:03.613214+0000","last_unstale":"2024-09-21T14:58:54.241055+0000","last_undegraded":"2024-09-21T14:58:54.241055+0000","last_fullsized":"2024-09-21T14:58:54.241055+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:02.180162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:02.180162+0000","last_clean_scrub_stamp":"2024-09-21T14:58:02.180162+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:29:05.860361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"56'60","reported_seq":1054,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T14:58:55.116380+0000","last_change":"2024-09-21T14:58:55.117083+0000","last_active":"2024-09-21T14:58:55.116380+0000","last_peered":"2024-09-21T14:58:55.116380+0000","last_clean":"2024-09-21T14:58:55.116380+0000","last_became_active":"2024-09-21T14:58:19.403887+0000","last_became_peered":"2024-09-21T14:58:19.403887+0000","last_unstale":"2024-09-21T14:58:55.116380+0000","last_undegraded":"2024-09-21T14:58:55.116380+0000","last_fullsized":"2024-09-21T14:58:55.116380+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:25:35.185774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001077616,"stat_sum":{"num_bytes":11386011,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":911,"num_read_kb":11738,"num_write":66,"num_write_kb":3845,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1a","length":"1"},{"start":"1d","length":"1"}]},{"pgid":"3.9","version":"50'52","reported_seq":512,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T14:58:55.123462+0000","last_change":"2024-09-21T14:58:55.124245+0000","last_active":"2024-09-21T14:58:55.123462+0000","last_peered":"2024-09-21T14:58:55.123462+0000","last_clean":"2024-09-21T14:58:55.123462+0000","last_became_active":"2024-09-21T14:58:19.399712+0000","last_became_peered":"2024-09-21T14:58:19.399712+0000","last_unstale":"2024-09-21T14:58:55.123462+0000","last_undegraded":"2024-09-21T14:58:55.123462+0000","last_fullsized":"2024-09-21T14:58:55.123462+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:50:47.558139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010873670000000001,"stat_sum":{"num_bytes":3522158,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":372,"num_read_kb":4459,"num_write":64,"num_write_kb":3247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1a","length":"1"},{"start":"1d","length":"1"}]},{"pgid":"3.a","version":"56'176","reported_seq":1299,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T14:58:55.379401+0000","last_change":"2024-09-21T14:58:55.125365+0000","last_active":"2024-09-21T14:58:55.379401+0000","last_peered":"2024-09-21T14:58:55.379401+0000","last_clean":"2024-09-21T14:58:55.379401+0000","last_became_active":"2024-09-21T14:58:19.407847+0000","last_became_peered":"2024-09-21T14:58:19.407847+0000","last_unstale":"2024-09-21T14:58:55.379401+0000","last_undegraded":"2024-09-21T14:58:55.379401+0000","last_fullsized":"2024-09-21T14:58:55.379401+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:45:44.979236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016626119999999999,"stat_sum":{"num_bytes":11732566,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":1549,"num_read_kb":16117,"num_write":295,"num_write_kb":9743,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1a","length":"1"},{"start":"1d","length":"1"}]},{"pgid":"3.b","version":"56'163","reported_seq":1004,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T14:58:55.303231+0000","last_change":"2024-09-21T14:58:55.120432+0000","last_active":"2024-09-21T14:58:55.303231+0000","last_peered":"2024-09-21T14:58:55.303231+0000","last_clean":"2024-09-21T14:58:55.303231+0000","last_became_active":"2024-09-21T14:58:19.410679+0000","last_became_peered":"2024-09-21T14:58:19.410679+0000","last_unstale":"2024-09-21T14:58:55.303231+0000","last_undegraded":"2024-09-21T14:58:55.303231+0000","last_fullsized":"2024-09-21T14:58:55.303231+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":163,"log_dups_size":0,"ondisk_log_size":163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:07:40.758932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035648,"stat_sum":{"num_bytes":6983144,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":1284,"num_read_kb":9151,"num_write":245,"num_write_kb":5827,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1a","length":"1"},{"start":"1d","length":"1"}]},{"pgid":"3.c","version":"59'61","reported_seq":915,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T14:58:55.833530+0000","last_change":"2024-09-21T14:58:55.128294+0000","last_active":"2024-09-21T14:58:55.833530+0000","last_peered":"2024-09-21T14:58:55.833530+0000","last_clean":"2024-09-21T14:58:55.833530+0000","last_became_active":"2024-09-21T14:58:19.411026+0000","last_became_peered":"2024-09-21T14:58:19.411026+0000","last_unstale":"2024-09-21T14:58:55.833530+0000","last_undegraded":"2024-09-21T14:58:55.833530+0000","last_fullsized":"2024-09-21T14:58:55.833530+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:38:03.448592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012784579999999999,"stat_sum":{"num_bytes":8751567,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":765,"num_read_kb":10258,"num_write":68,"num_write_kb":3985,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1a","length":"1"},{"start":"1d","length":"1"}]},{"pgid":"3.d","version":"59'88","reported_seq":523,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T14:58:55.860081+0000","last_change":"2024-09-21T14:58:55.125006+0000","last_active":"2024-09-21T14:58:55.860081+0000","last_peered":"2024-09-21T14:58:55.860081+0000","last_clean":"2024-09-21T14:58:55.860081+0000","last_became_active":"2024-09-21T14:58:19.407884+0000","last_became_peered":"2024-09-21T14:58:19.407884+0000","last_unstale":"2024-09-21T14:58:55.860081+0000","last_undegraded":"2024-09-21T14:58:55.860081+0000","last_fullsized":"2024-09-21T14:58:55.860081+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:28:57.111615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013445009999999999,"stat_sum":{"num_bytes":27945235,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":347,"num_read_kb":5153,"num_write":102,"num_write_kb":5257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1a","length":"1"},{"start":"1d","length":"1"}]},{"pgid":"3.e","version":"59'310","reported_seq":672,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T14:58:55.807057+0000","last_change":"2024-09-21T14:58:55.113776+0000","last_active":"2024-09-21T14:58:55.807057+0000","last_peered":"2024-09-21T14:58:55.807057+0000","last_clean":"2024-09-21T14:58:55.807057+0000","last_became_active":"2024-09-21T14:58:19.414418+0000","last_became_peered":"2024-09-21T14:58:19.414418+0000","last_unstale":"2024-09-21T14:58:55.807057+0000","last_undegraded":"2024-09-21T14:58:55.807057+0000","last_fullsized":"2024-09-21T14:58:55.807057+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":310,"log_dups_size":0,"ondisk_log_size":310,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:36:41.583214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072234899999999995,"stat_sum":{"num_bytes":18391681,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":1619,"num_read_kb":4080,"num_write":586,"num_write_kb":2869,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"1a","length":"1"},{"start":"1d","length":"1"}]},{"pgid":"3.f","version":"59'204","reported_seq":891,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T14:58:55.768447+0000","last_change":"2024-09-21T14:58:55.127649+0000","last_active":"2024-09-21T14:58:55.768447+0000","last_peered":"2024-09-21T14:58:55.768447+0000","last_clean":"2024-09-21T14:58:55.768447+0000","last_became_active":"2024-09-21T14:58:19.410406+0000","last_became_peered":"2024-09-21T14:58:19.410406+0000","last_unstale":"2024-09-21T14:58:55.768447+0000","last_undegraded":"2024-09-21T14:58:55.768447+0000","last_fullsized":"2024-09-21T14:58:55.768447+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":204,"log_dups_size":0,"ondisk_log_size":204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:59:16.026939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076921300000000004,"stat_sum":{"num_bytes":4981730,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":1251,"num_read_kb":7996,"num_write":313,"num_write_kb":5104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1a","length":"1"},{"start":"1d","length":"1"}]},{"pgid":"3.10","version":"59'84","reported_seq":653,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T14:58:55.796015+0000","last_change":"2024-09-21T14:58:55.111149+0000","last_active":"2024-09-21T14:58:55.796015+0000","last_peered":"2024-09-21T14:58:55.796015+0000","last_clean":"2024-09-21T14:58:55.796015+0000","last_became_active":"2024-09-21T14:58:19.409935+0000","last_became_peered":"2024-09-21T14:58:19.409935+0000","last_unstale":"2024-09-21T14:58:55.796015+0000","last_undegraded":"2024-09-21T14:58:55.796015+0000","last_fullsized":"2024-09-21T14:58:55.796015+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:10:36.252629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098319399999999995,"stat_sum":{"num_bytes":19961143,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":482,"num_read_kb":5531,"num_write":100,"num_write_kb":6102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1a","length":"1"},{"start":"1d","length":"1"}]},{"pgid":"3.11","version":"56'66","reported_seq":432,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T14:58:55.123394+0000","last_change":"2024-09-21T14:58:55.124011+0000","last_active":"2024-09-21T14:58:55.123394+0000","last_peered":"2024-09-21T14:58:55.123394+0000","last_clean":"2024-09-21T14:58:55.123394+0000","last_became_active":"2024-09-21T14:58:19.406164+0000","last_became_peered":"2024-09-21T14:58:19.406164+0000","last_unstale":"2024-09-21T14:58:55.123394+0000","last_undegraded":"2024-09-21T14:58:55.123394+0000","last_fullsized":"2024-09-21T14:58:55.123394+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:26:32.857391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089422099999999995,"stat_sum":{"num_bytes":3427333,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":278,"num_read_kb":4827,"num_write":71,"num_write_kb":3799,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1a","length":"1"},{"start":"1d","length":"1"}]},{"pgid":"3.12","version":"59'51","reported_seq":316,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T14:58:55.384709+0000","last_change":"2024-09-21T14:58:55.125203+0000","last_active":"2024-09-21T14:58:55.384709+0000","last_peered":"2024-09-21T14:58:55.384709+0000","last_clean":"2024-09-21T14:58:55.384709+0000","last_became_active":"2024-09-21T14:58:47.175860+0000","last_became_peered":"2024-09-21T14:58:47.175860+0000","last_unstale":"2024-09-21T14:58:55.384709+0000","last_undegraded":"2024-09-21T14:58:55.384709+0000","last_fullsized":"2024-09-21T14:58:55.384709+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:04:46.485874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001523943,"stat_sum":{"num_bytes":2475425,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":180,"num_read_kb":2011,"num_write":75,"num_write_kb":2365,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1a","length":"1"},{"start":"1d","length":"1"}]},{"pgid":"3.13","version":"59'221","reported_seq":981,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T14:58:55.859665+0000","last_change":"2024-09-21T14:58:55.124786+0000","last_active":"2024-09-21T14:58:55.859665+0000","last_peered":"2024-09-21T14:58:55.859665+0000","last_clean":"2024-09-21T14:58:55.859665+0000","last_became_active":"2024-09-21T14:58:19.412030+0000","last_became_peered":"2024-09-21T14:58:19.412030+0000","last_unstale":"2024-09-21T14:58:55.859665+0000","last_undegraded":"2024-09-21T14:58:55.859665+0000","last_fullsized":"2024-09-21T14:58:55.859665+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":221,"log_dups_size":0,"ondisk_log_size":221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:44:29.282391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00160272,"stat_sum":{"num_bytes":3343444,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":1687,"num_read_kb":6512,"num_write":273,"num_write_kb":5293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1a","length":"1"},{"start":"1d","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":281072753,"num_objects":475,"num_object_clones":0,"num_object_copies":950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":475,"num_whiteouts":0,"num_read":28459,"num_read_kb":223062,"num_write":5485,"num_write_kb":141582,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1403348,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16846848,"data_stored":172922076,"data_compressed":2815826,"data_compressed_allocated":14602240,"data_compressed_original":171703814,"omap_allocated":0,"internal_metadata":0},"log_size":3904,"ondisk_log_size":3904,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":10404,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509455,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37208,"kb_used_data":1992,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334632,"statfs":{"total":96636764160,"available":96598663168,"internally_reserved":0,"allocated":2039808,"data_stored":15137543,"data_compressed":151754,"data_compressed_allocated":1171456,"data_compressed_original":14746802,"omap_allocated":1589,"internal_metadata":36043211},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27900000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40400000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35799999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66500000000000004}]}]},{"osd":6,"up_from":14,"seq":60129542158,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38032,"kb_used_data":2820,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94333808,"statfs":{"total":96636764160,"available":96597819392,"internally_reserved":0,"allocated":2887680,"data_stored":23029391,"data_compressed":426481,"data_compressed_allocated":1978368,"data_compressed_original":22614713,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99199999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.878}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.196}]}]},{"osd":5,"up_from":14,"seq":60129542159,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37672,"kb_used_data":2456,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334168,"statfs":{"total":96636764160,"available":96598188032,"internally_reserved":0,"allocated":2514944,"data_stored":20239022,"data_compressed":271821,"data_compressed_allocated":1634304,"data_compressed_original":19826920,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.054}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92600000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83499999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97899999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.304}]}]},{"osd":4,"up_from":14,"seq":60129542158,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37388,"kb_used_data":2172,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334452,"statfs":{"total":96636764160,"available":96598478848,"internally_reserved":0,"allocated":2224128,"data_stored":17595023,"data_compressed":194543,"data_compressed_allocated":1372160,"data_compressed_original":17182591,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81000000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78100000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70599999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95799999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88700000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92300000000000004}]}]},{"osd":3,"up_from":14,"seq":60129542159,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38480,"kb_used_data":3264,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94333360,"statfs":{"total":96636764160,"available":96597360640,"internally_reserved":0,"allocated":3342336,"data_stored":26914273,"data_compressed":441659,"data_compressed_allocated":2277376,"data_compressed_original":26396896,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72099999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.878}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97499999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1990000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.151}]}]},{"osd":2,"up_from":14,"seq":60129542157,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42152,"kb_used_data":2840,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94329688,"statfs":{"total":96636764160,"available":96593600512,"internally_reserved":0,"allocated":2908160,"data_stored":21091045,"data_compressed":443012,"data_compressed_allocated":1871872,"data_compressed_original":20596718,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95899999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94699999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.135}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.089}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92300000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89600000000000002}]}]},{"osd":1,"up_from":14,"seq":60129542158,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42300,"kb_used_data":2988,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94329540,"statfs":{"total":96636764160,"available":96593448960,"internally_reserved":0,"allocated":3059712,"data_stored":27428037,"data_compressed":284802,"data_compressed_allocated":2093056,"data_compressed_original":26955519,"omap_allocated":1595,"internal_metadata":40237509},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71299999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91100000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80300000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90100000000000002}]}]},{"osd":0,"up_from":14,"seq":60129542157,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42292,"kb_used_data":2984,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94329548,"statfs":{"total":96636764160,"available":96593457152,"internally_reserved":0,"allocated":3055616,"data_stored":24674314,"data_compressed":428177,"data_compressed_allocated":2072576,"data_compressed_original":24215143,"omap_allocated":1593,"internal_metadata":40237511},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84399999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79900000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94399999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5202,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5202,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2387968,"data_stored":24395231,"data_compressed":428213,"data_compressed_allocated":2076672,"data_compressed_original":24235623,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2514944,"data_stored":26745635,"data_compressed":386983,"data_compressed_allocated":2174976,"data_compressed_original":26554111,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2199552,"data_stored":20777777,"data_compressed":422468,"data_compressed_allocated":1851392,"data_compressed_original":20588526,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2715648,"data_stored":26683431,"data_compressed":477405,"data_compressed_allocated":2314240,"data_compressed_original":26454240,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1585152,"data_stored":17330731,"data_compressed":212451,"data_compressed_allocated":1392640,"data_compressed_original":17211263,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1810432,"data_stored":19368430,"data_compressed":292154,"data_compressed_allocated":1622016,"data_compressed_original":19269864,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2273280,"data_stored":22786881,"data_compressed":444398,"data_compressed_allocated":1998848,"data_compressed_original":22643385,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1359872,"data_stored":14833960,"data_compressed":151754,"data_compressed_allocated":1171456,"data_compressed_original":14746802,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T14:58:57.371 INFO:tasks.thrashosds.thrasher:Setting 3.c to [0, 6, 4, 1] 2024-09-21T14:58:57.372 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.c', '0', '6', '4', '1'] 2024-09-21T14:58:57.372 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.c 0 6 4 1 2024-09-21T14:58:57.377 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T14:58:57.413 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:58:57.413+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:58:57.494 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:58:57.493+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:58:57.505 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:57.505 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:57.518 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T14:58:57.595 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:58:57.593+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:58:57.657 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:57.657 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:57.669 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T14:58:57.713 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:58:57.712+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:58:57.779 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:57.779 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:57.789 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T14:58:57.797 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:58:57.792+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:58:57.895 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:57.895 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:57.898 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:58:57.897+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:58:57.905 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T14:58:57.998 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T14:58:57.999 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:58:57.997+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:58:58.062 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:58.062 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:58.072 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T14:58:58.099 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:58:58.096+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:58:58.149 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T14:58:58.169 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T14:58:58.182 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:58:58.182 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:58:58.189 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T14:58:58.192 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T14:58:58.210 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T14:58:58.232 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-21T14:58:58.233 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:58:58.232+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:58:58.259 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T14:58:58.260 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T14:58:58.261 INFO:teuthology.orchestra.run.smithi032.stderr:set 3.c pg_upmap_items mapping to [0->6,4->1] 2024-09-21T14:58:58.286 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T14:58:58.301 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:58:58.301+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:58:58.306 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T14:58:58.308 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:58:58.308 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:58:58.308 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:58:58.308 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:58:58.308 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:58:58.308 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:58:58.308 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:58:58.308 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:58:58.347 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:58:58.347 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:58:58.347 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T14:58:58.347 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T14:58:58.347 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:58:58.356 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T14:58:58.402 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:58:58.400+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:58:58.404 INFO:teuthology.orchestra.run.smithi032.stdout:checking clone #8, image image_client.0-clone8 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent9 2024-09-21T14:58:58.463 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:58:58.463 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T14:58:58.463 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T14:58:58.463 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T14:58:58.464 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.464 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:8011 3.b 3:d1b12ef9:::rbd_data.11361bed1f78.0000000000000062:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:58.464 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:35.227212+0000", 2024-09-21T14:58:58.464 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.235021066000002, 2024-09-21T14:58:58.464 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0054249789999999999, 2024-09-21T14:58:58.464 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:58.464 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:58.464 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:58.464 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:58.464 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:58.464 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 8011 2024-09-21T14:58:58.464 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.464 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:58.464 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.465 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:58.465 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227212+0000", 2024-09-21T14:58:58.465 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:58.465 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.465 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.465 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:58.465 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227212+0000", 2024-09-21T14:58:58.465 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:58.465 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.466 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.466 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:58.466 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227211+0000", 2024-09-21T14:58:58.466 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:58.466 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.466 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.466 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:58.466 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227214+0000", 2024-09-21T14:58:58.466 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7999999999999999e-06 2024-09-21T14:58:58.466 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.466 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.466 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:58.466 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227220+0000", 2024-09-21T14:58:58.466 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0569999999999998e-06 2024-09-21T14:58:58.467 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.467 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.467 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:58.467 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227223+0000", 2024-09-21T14:58:58.467 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3000000000000002e-06 2024-09-21T14:58:58.467 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.467 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.467 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:58.467 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.232445+0000", 2024-09-21T14:58:58.467 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0052220160000000003 2024-09-21T14:58:58.467 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.467 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.467 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:58.467 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.232535+0000", 2024-09-21T14:58:58.467 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9866999999999999e-05 2024-09-21T14:58:58.468 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.468 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.468 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:58.468 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.232637+0000", 2024-09-21T14:58:58.468 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010155 2024-09-21T14:58:58.468 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:58.468 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:58.468 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:58.468 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.468 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.468 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:8019 3.b 3:d1b12ef9:::rbd_data.11361bed1f78.0000000000000062:d [read 0~32768 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:58.468 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:35.227226+0000", 2024-09-21T14:58:58.468 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.235006465000001, 2024-09-21T14:58:58.468 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.005662696, 2024-09-21T14:58:58.469 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:58.469 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:58.469 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:58.469 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:58.469 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:58.469 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 8019 2024-09-21T14:58:58.469 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.469 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:58.469 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.469 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:58.469 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227226+0000", 2024-09-21T14:58:58.469 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:58.469 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.469 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.470 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:58.470 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227226+0000", 2024-09-21T14:58:58.470 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:58.470 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.470 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.470 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:58.470 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227226+0000", 2024-09-21T14:58:58.470 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:58.470 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.470 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.470 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:58.470 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227228+0000", 2024-09-21T14:58:58.470 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5919999999999999e-06 2024-09-21T14:58:58.470 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.471 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.471 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:58.471 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227235+0000", 2024-09-21T14:58:58.471 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2650000000000002e-06 2024-09-21T14:58:58.471 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.471 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.471 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:58.471 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227241+0000", 2024-09-21T14:58:58.471 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9139999999999996e-06 2024-09-21T14:58:58.471 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.471 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.471 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:58.471 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.232671+0000", 2024-09-21T14:58:58.471 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0054301879999999999 2024-09-21T14:58:58.471 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.472 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.472 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:58.472 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.232747+0000", 2024-09-21T14:58:58.472 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5988999999999994e-05 2024-09-21T14:58:58.472 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.472 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.472 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:58.472 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.232889+0000", 2024-09-21T14:58:58.472 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014205199999999999 2024-09-21T14:58:58.472 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:58.472 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:58.472 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:58.472 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.472 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.473 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:8020 3.b 3:d1b12ef9:::rbd_data.11361bed1f78.0000000000000062:d [read 81920~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:58.473 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:35.227246+0000", 2024-09-21T14:58:58.473 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.234986872, 2024-09-21T14:58:58.473 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0061395770000000002, 2024-09-21T14:58:58.473 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:58.473 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:58.473 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:58.473 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:58.473 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:58.473 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 8020 2024-09-21T14:58:58.473 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.473 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:58.474 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.474 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:58.474 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227246+0000", 2024-09-21T14:58:58.474 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:58.474 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.474 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.474 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:58.474 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227246+0000", 2024-09-21T14:58:58.474 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:58.474 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.474 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.474 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:58.474 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227245+0000", 2024-09-21T14:58:58.474 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:58.475 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.475 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.475 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:58.475 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227248+0000", 2024-09-21T14:58:58.475 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8119999999999999e-06 2024-09-21T14:58:58.475 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.475 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.475 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:58.475 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227255+0000", 2024-09-21T14:58:58.475 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7549999999999997e-06 2024-09-21T14:58:58.475 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.476 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.476 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:58.476 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227260+0000", 2024-09-21T14:58:58.476 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.271e-06 2024-09-21T14:58:58.476 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.476 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.476 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:58.476 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.232967+0000", 2024-09-21T14:58:58.476 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0057065179999999998 2024-09-21T14:58:58.476 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.476 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.476 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:58.477 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.233108+0000", 2024-09-21T14:58:58.477 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000141123 2024-09-21T14:58:58.477 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.477 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.477 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:58.477 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.233385+0000", 2024-09-21T14:58:58.477 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00027743400000000001 2024-09-21T14:58:58.477 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:58.477 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:58.477 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:58.477 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.477 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.477 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:8034 3.b 3:d1b12ef9:::rbd_data.11361bed1f78.0000000000000062:d [read 32768~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:58.477 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:35.227265+0000", 2024-09-21T14:58:58.477 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.234967505, 2024-09-21T14:58:58.478 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.006525968, 2024-09-21T14:58:58.478 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:58.478 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:58.478 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:58.478 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:58.478 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:58.478 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 8034 2024-09-21T14:58:58.478 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.478 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:58.478 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.478 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:58.478 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227265+0000", 2024-09-21T14:58:58.478 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:58.478 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.479 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.479 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:58.479 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227265+0000", 2024-09-21T14:58:58.479 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:58.479 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.479 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.479 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:58.479 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227265+0000", 2024-09-21T14:58:58.479 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:58.479 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.479 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.479 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:58.479 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227268+0000", 2024-09-21T14:58:58.479 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8990000000000001e-06 2024-09-21T14:58:58.480 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.480 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.480 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:58.480 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227274+0000", 2024-09-21T14:58:58.480 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4470000000000003e-06 2024-09-21T14:58:58.480 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.480 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.480 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:58.480 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227288+0000", 2024-09-21T14:58:58.480 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3550000000000001e-05 2024-09-21T14:58:58.480 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.480 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.480 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:58.480 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.233418+0000", 2024-09-21T14:58:58.481 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0061307870000000004 2024-09-21T14:58:58.481 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.481 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.481 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:58.481 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.233553+0000", 2024-09-21T14:58:58.481 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013475600000000001 2024-09-21T14:58:58.481 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.481 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.481 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:58.481 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.233791+0000", 2024-09-21T14:58:58.481 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00023788499999999999 2024-09-21T14:58:58.481 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:58.481 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:58.481 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:58.481 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.481 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.482 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:8035 3.b 3:d1b12ef9:::rbd_data.11361bed1f78.0000000000000062:d [read 98304~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:58.482 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:35.227295+0000", 2024-09-21T14:58:58.482 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.234937299999999, 2024-09-21T14:58:58.482 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0069673820000000003, 2024-09-21T14:58:58.482 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:58.482 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:58.482 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:58.482 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:58.482 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:58.482 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 8035 2024-09-21T14:58:58.482 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.482 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:58.482 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.483 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:58.483 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227295+0000", 2024-09-21T14:58:58.483 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:58.483 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.483 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.483 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:58.483 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227295+0000", 2024-09-21T14:58:58.483 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:58.483 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.483 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.483 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:58.483 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227295+0000", 2024-09-21T14:58:58.483 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:58.483 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.484 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.484 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:58.484 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227298+0000", 2024-09-21T14:58:58.484 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2339999999999999e-06 2024-09-21T14:58:58.484 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.484 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.484 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:58.484 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227305+0000", 2024-09-21T14:58:58.484 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0389999999999999e-06 2024-09-21T14:58:58.484 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.484 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.484 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:58.484 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227311+0000", 2024-09-21T14:58:58.484 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8470000000000001e-06 2024-09-21T14:58:58.485 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.485 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.485 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:58.485 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.233824+0000", 2024-09-21T14:58:58.485 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0065132920000000004 2024-09-21T14:58:58.485 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.485 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.485 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:58.485 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.233922+0000", 2024-09-21T14:58:58.485 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.8073999999999996e-05 2024-09-21T14:58:58.485 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.485 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.485 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:58.485 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.234263+0000", 2024-09-21T14:58:58.486 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00034027999999999999 2024-09-21T14:58:58.486 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:58.486 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:58.486 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:58.486 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.486 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.486 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:8048 3.b 3:d1b12ef9:::rbd_data.11361bed1f78.0000000000000062:d [read 49152~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:58.486 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:35.227316+0000", 2024-09-21T14:58:58.486 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.234916508000001, 2024-09-21T14:58:58.486 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0076197799999999996, 2024-09-21T14:58:58.486 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:58.486 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:58.486 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:58.487 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:58.487 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:58.487 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 8048 2024-09-21T14:58:58.487 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.487 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:58.487 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.487 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:58.487 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227316+0000", 2024-09-21T14:58:58.487 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:58.487 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.487 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.487 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:58.487 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227316+0000", 2024-09-21T14:58:58.488 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:58.488 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.488 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.488 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:58.488 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227316+0000", 2024-09-21T14:58:58.488 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:58.488 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.488 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.488 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:58.488 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227324+0000", 2024-09-21T14:58:58.488 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1009999999999992e-06 2024-09-21T14:58:58.488 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.488 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.488 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:58.489 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227346+0000", 2024-09-21T14:58:58.489 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2347000000000001e-05 2024-09-21T14:58:58.489 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.489 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.489 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:58.489 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227360+0000", 2024-09-21T14:58:58.489 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.403e-05 2024-09-21T14:58:58.489 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.489 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.489 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:58.489 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.234320+0000", 2024-09-21T14:58:58.489 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0069597019999999999 2024-09-21T14:58:58.489 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.489 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.490 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:58.490 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.234730+0000", 2024-09-21T14:58:58.490 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00041006300000000001 2024-09-21T14:58:58.490 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.490 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.490 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:58.490 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.234936+0000", 2024-09-21T14:58:58.490 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00020586700000000001 2024-09-21T14:58:58.490 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:58.490 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:58.490 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:58.490 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.490 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.491 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:8049 3.b 3:d1b12ef9:::rbd_data.11361bed1f78.0000000000000062:d [read 114688~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:58.491 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:35.227371+0000", 2024-09-21T14:58:58.491 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.234861174999999, 2024-09-21T14:58:58.491 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0078533609999999997, 2024-09-21T14:58:58.491 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:58.491 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:58.491 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:58.491 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:58.491 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:58.491 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 8049 2024-09-21T14:58:58.491 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.491 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:58.491 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.491 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:58.492 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227371+0000", 2024-09-21T14:58:58.492 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:58.492 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.492 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.492 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:58.492 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227371+0000", 2024-09-21T14:58:58.492 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:58.492 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.492 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.492 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:58.492 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227370+0000", 2024-09-21T14:58:58.492 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:58.492 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.492 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.493 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:58.493 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227378+0000", 2024-09-21T14:58:58.493 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.379e-06 2024-09-21T14:58:58.493 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.493 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.493 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:58.493 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227388+0000", 2024-09-21T14:58:58.493 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0175e-05 2024-09-21T14:58:58.493 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.493 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.493 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:58.493 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227393+0000", 2024-09-21T14:58:58.493 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6690000000000003e-06 2024-09-21T14:58:58.493 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.494 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.494 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:58.494 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.234964+0000", 2024-09-21T14:58:58.494 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0075705319999999996 2024-09-21T14:58:58.494 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.494 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.494 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:58.494 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.235041+0000", 2024-09-21T14:58:58.494 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7170999999999997e-05 2024-09-21T14:58:58.494 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.494 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.494 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:58.494 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.235225+0000", 2024-09-21T14:58:58.494 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000183735 2024-09-21T14:58:58.495 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:58.495 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:58.495 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:58.495 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.495 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.495 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:15174 3.b 3:d11a5903:::rbd_data.11361bed1f78.0000000000000093:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-21T14:58:58.495 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:40.290309+0000", 2024-09-21T14:58:58.495 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 18.17192322, 2024-09-21T14:58:58.495 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0060508960000000001, 2024-09-21T14:58:58.495 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:58.495 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:58:58.495 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:58.496 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:58.496 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:58.496 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 15174 2024-09-21T14:58:58.496 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.496 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:58.496 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.496 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:58.496 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:40.290309+0000", 2024-09-21T14:58:58.496 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:58.496 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.496 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.496 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:58.496 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:40.290309+0000", 2024-09-21T14:58:58.496 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:58.496 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.497 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.497 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:58.497 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:40.290309+0000", 2024-09-21T14:58:58.497 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:58.497 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.497 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.497 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:58.497 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:40.290314+0000", 2024-09-21T14:58:58.497 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5219999999999998e-06 2024-09-21T14:58:58.497 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.497 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.497 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:58.497 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:40.290330+0000", 2024-09-21T14:58:58.497 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5511000000000001e-05 2024-09-21T14:58:58.497 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.498 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.498 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:58.498 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:40.290341+0000", 2024-09-21T14:58:58.498 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1729000000000001e-05 2024-09-21T14:58:58.498 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.498 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.498 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:58.498 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:40.296093+0000", 2024-09-21T14:58:58.498 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.005751847 2024-09-21T14:58:58.498 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.498 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.499 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:58.499 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:40.296360+0000", 2024-09-21T14:58:58.499 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00026712099999999999 2024-09-21T14:58:58.499 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:58.499 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:58.499 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:58.499 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.499 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.499 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:19869 3.b 3:d7ca7d68:::rbd_data.11363a21d2be.000000000000002d:head [sparse-read 0~393216] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T14:58:58.499 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:44.340896+0000", 2024-09-21T14:58:58.499 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.121336778, 2024-09-21T14:58:58.499 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0071749170000000003, 2024-09-21T14:58:58.499 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:58.499 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:58.500 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:58.500 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:58.500 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:58.500 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 19869 2024-09-21T14:58:58.500 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.500 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:58.500 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.500 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:58.500 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340896+0000", 2024-09-21T14:58:58.500 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:58.500 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.500 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.500 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:58.500 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340896+0000", 2024-09-21T14:58:58.500 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:58.500 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.501 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.501 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:58.501 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340894+0000", 2024-09-21T14:58:58.501 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:58.501 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.501 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.501 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:58.501 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340903+0000", 2024-09-21T14:58:58.501 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0259999999999996e-06 2024-09-21T14:58:58.501 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.501 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.501 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:58.501 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340923+0000", 2024-09-21T14:58:58.501 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0013999999999998e-05 2024-09-21T14:58:58.501 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.501 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.502 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:58.502 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340938+0000", 2024-09-21T14:58:58.502 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4569000000000001e-05 2024-09-21T14:58:58.502 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.502 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.502 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:58.502 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342092+0000", 2024-09-21T14:58:58.502 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00115354 2024-09-21T14:58:58.502 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.502 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.502 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:58.502 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342290+0000", 2024-09-21T14:58:58.502 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019834299999999999 2024-09-21T14:58:58.502 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.503 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.503 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:58.503 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.348071+0000", 2024-09-21T14:58:58.503 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.005780861 2024-09-21T14:58:58.503 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:58.503 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:58.503 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:58.503 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.503 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.503 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:19872 3.b 3:d38ac3f6:::rbd_data.11363a21d2be.0000000000000030:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T14:58:58.503 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:44.340951+0000", 2024-09-21T14:58:58.503 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.121281618999999, 2024-09-21T14:58:58.503 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0069228680000000004, 2024-09-21T14:58:58.503 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:58.503 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:58.503 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:58.504 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:58.504 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:58.504 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 19872 2024-09-21T14:58:58.504 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.504 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:58.504 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.504 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:58.504 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340951+0000", 2024-09-21T14:58:58.504 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:58.504 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.504 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.504 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:58.504 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340951+0000", 2024-09-21T14:58:58.504 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:58.504 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.504 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.505 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:58.505 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340950+0000", 2024-09-21T14:58:58.505 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:58.505 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.505 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.505 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:58.505 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340957+0000", 2024-09-21T14:58:58.505 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8430000000000004e-06 2024-09-21T14:58:58.505 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.505 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.505 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:58.505 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340974+0000", 2024-09-21T14:58:58.505 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7082999999999999e-05 2024-09-21T14:58:58.505 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.505 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.506 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:58.506 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340987+0000", 2024-09-21T14:58:58.506 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2763e-05 2024-09-21T14:58:58.506 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.506 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.506 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:58.506 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.347149+0000", 2024-09-21T14:58:58.506 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0061626600000000004 2024-09-21T14:58:58.506 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.506 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.506 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:58.506 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.347354+0000", 2024-09-21T14:58:58.506 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00020441799999999999 2024-09-21T14:58:58.506 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.507 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.507 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:58.507 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.347874+0000", 2024-09-21T14:58:58.507 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00052019399999999997 2024-09-21T14:58:58.507 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:58.507 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:58.507 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:58.507 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.507 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.507 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:19877 3.b 3:d5fc5a5a:::rbd_data.11363a21d2be.0000000000000035:head [sparse-read 0~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T14:58:58.507 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:44.340998+0000", 2024-09-21T14:58:58.507 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.121234783, 2024-09-21T14:58:58.507 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0073322129999999998, 2024-09-21T14:58:58.507 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:58.507 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:58:58.507 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:58.508 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:58.508 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:58.508 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 19877 2024-09-21T14:58:58.508 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.508 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:58.508 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.508 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:58.508 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340998+0000", 2024-09-21T14:58:58.508 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:58.508 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.508 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.508 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:58.508 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340998+0000", 2024-09-21T14:58:58.508 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:58.508 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.509 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.509 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:58.509 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340997+0000", 2024-09-21T14:58:58.509 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:58.509 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.509 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.509 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:58.509 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.341003+0000", 2024-09-21T14:58:58.509 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0070000000000003e-06 2024-09-21T14:58:58.509 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.509 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.509 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:58.509 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.341018+0000", 2024-09-21T14:58:58.509 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5472999999999999e-05 2024-09-21T14:58:58.509 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.510 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.510 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:58.510 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.341040+0000", 2024-09-21T14:58:58.510 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1568000000000001e-05 2024-09-21T14:58:58.510 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.510 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.510 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:58.510 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.347901+0000", 2024-09-21T14:58:58.510 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0068608300000000001 2024-09-21T14:58:58.510 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.510 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.510 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:58.510 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.347973+0000", 2024-09-21T14:58:58.510 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2327000000000001e-05 2024-09-21T14:58:58.510 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.511 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.511 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:58.511 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.348330+0000", 2024-09-21T14:58:58.511 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00035695799999999999 2024-09-21T14:58:58.511 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:58.511 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:58.511 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:58.511 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.511 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.511 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:19941 3.b 3:d7b21c30:::rbd_data.11363a21d2be.0000000000000075:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T14:58:58.511 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:44.342141+0000", 2024-09-21T14:58:58.511 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.120091216000001, 2024-09-21T14:58:58.511 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0063087439999999998, 2024-09-21T14:58:58.511 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:58.511 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:58:58.512 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:58.512 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:58.512 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:58.512 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 19941 2024-09-21T14:58:58.512 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.512 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:58.512 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.512 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:58.512 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342141+0000", 2024-09-21T14:58:58.512 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:58.512 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.512 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.512 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:58.512 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342141+0000", 2024-09-21T14:58:58.512 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:58.513 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.513 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.513 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:58.513 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342139+0000", 2024-09-21T14:58:58.513 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T14:58:58.513 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.513 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.513 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:58.513 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342151+0000", 2024-09-21T14:58:58.513 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1242e-05 2024-09-21T14:58:58.513 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.513 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.513 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:58.514 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342175+0000", 2024-09-21T14:58:58.514 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4491e-05 2024-09-21T14:58:58.514 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.514 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.514 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:58.514 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342191+0000", 2024-09-21T14:58:58.514 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6288000000000001e-05 2024-09-21T14:58:58.514 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.514 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.514 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:58.514 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.348358+0000", 2024-09-21T14:58:58.514 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0061665449999999998 2024-09-21T14:58:58.515 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.515 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.515 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:58.515 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.348450+0000", 2024-09-21T14:58:58.515 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2245999999999997e-05 2024-09-21T14:58:58.515 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:58.515 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:58.515 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:58.515 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.515 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.515 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:19998 3.b 3:d4c6ec57:::rbd_data.11363a21d2be.00000000000000ae:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T14:58:58.515 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:44.342204+0000", 2024-09-21T14:58:58.515 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.120029012, 2024-09-21T14:58:58.515 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0063531860000000002, 2024-09-21T14:58:58.516 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:58.516 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:58:58.516 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:58.516 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:58.516 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:58.516 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 19998 2024-09-21T14:58:58.516 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.516 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:58.516 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.516 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:58.516 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342204+0000", 2024-09-21T14:58:58.516 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:58.516 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.516 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.517 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:58.517 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342204+0000", 2024-09-21T14:58:58.517 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:58.517 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.517 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.517 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:58.517 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342203+0000", 2024-09-21T14:58:58.517 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:58.517 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.517 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.517 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:58.517 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342209+0000", 2024-09-21T14:58:58.517 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1739999999999997e-06 2024-09-21T14:58:58.517 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.518 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.518 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:58.518 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342223+0000", 2024-09-21T14:58:58.518 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4065e-05 2024-09-21T14:58:58.518 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.518 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.518 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:58.518 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342236+0000", 2024-09-21T14:58:58.518 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2537e-05 2024-09-21T14:58:58.518 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.518 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.518 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:58.518 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.348476+0000", 2024-09-21T14:58:58.519 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0062405760000000003 2024-09-21T14:58:58.519 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.519 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.519 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:58.519 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.348557+0000", 2024-09-21T14:58:58.519 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0716000000000004e-05 2024-09-21T14:58:58.519 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:58.519 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:58.519 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:58.519 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.519 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.519 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:19999 3.b 3:d4069488:::rbd_data.11363a21d2be.00000000000000af:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T14:58:58.519 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:44.342246+0000", 2024-09-21T14:58:58.520 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.119986847, 2024-09-21T14:58:58.520 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0064139210000000004, 2024-09-21T14:58:58.520 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:58.520 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:58:58.520 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:58.520 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:58.520 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:58.520 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 19999 2024-09-21T14:58:58.520 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.520 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:58.520 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.520 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:58.521 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342246+0000", 2024-09-21T14:58:58.521 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:58.521 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.521 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.521 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:58.521 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342246+0000", 2024-09-21T14:58:58.521 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:58.521 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.521 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.521 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:58.521 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342245+0000", 2024-09-21T14:58:58.521 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:58.521 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.521 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.522 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:58.522 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342251+0000", 2024-09-21T14:58:58.522 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8660000000000002e-06 2024-09-21T14:58:58.522 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.522 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.522 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:58.522 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342264+0000", 2024-09-21T14:58:58.522 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3716999999999999e-05 2024-09-21T14:58:58.522 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.522 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.522 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:58.522 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342275+0000", 2024-09-21T14:58:58.523 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1071e-05 2024-09-21T14:58:58.523 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.523 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.523 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:58.523 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.348581+0000", 2024-09-21T14:58:58.523 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0063053730000000004 2024-09-21T14:58:58.523 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.523 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.523 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:58.523 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.348660+0000", 2024-09-21T14:58:58.523 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8854000000000001e-05 2024-09-21T14:58:58.523 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:58.523 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:58.524 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:58.524 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.524 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.524 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:20407 3.b 3:d6a6f68e:::rbd_object_map.1136c84d9a13:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-21T14:58:58.524 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:45.012351+0000", 2024-09-21T14:58:58.524 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 13.449881250000001, 2024-09-21T14:58:58.524 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0056810080000000004, 2024-09-21T14:58:58.524 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:58.524 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:58.524 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:58.524 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:58.524 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:58.524 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 20407 2024-09-21T14:58:58.524 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.524 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:58.525 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.525 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:58.525 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.012351+0000", 2024-09-21T14:58:58.525 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:58.525 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.525 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.525 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:58.525 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.012351+0000", 2024-09-21T14:58:58.525 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:58.525 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.525 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.525 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:58.525 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.012350+0000", 2024-09-21T14:58:58.525 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:58.525 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.525 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.525 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:58.526 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.012360+0000", 2024-09-21T14:58:58.526 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0311e-05 2024-09-21T14:58:58.526 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.526 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.526 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:58.526 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.012383+0000", 2024-09-21T14:58:58.526 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2408e-05 2024-09-21T14:58:58.526 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.526 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.526 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:58.526 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.012398+0000", 2024-09-21T14:58:58.526 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4919999999999999e-05 2024-09-21T14:58:58.526 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.526 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.526 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:58.527 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.014378+0000", 2024-09-21T14:58:58.527 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0019804890000000002 2024-09-21T14:58:58.527 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.527 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.527 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:58.527 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.014640+0000", 2024-09-21T14:58:58.527 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000261434 2024-09-21T14:58:58.527 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.527 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.527 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T14:58:58.527 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.015591+0000", 2024-09-21T14:58:58.527 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00095092199999999999 2024-09-21T14:58:58.527 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.527 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.527 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:58:58.527 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.017084+0000", 2024-09-21T14:58:58.527 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0014932089999999999 2024-09-21T14:58:58.528 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.528 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.528 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:58.528 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.017850+0000", 2024-09-21T14:58:58.528 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00076654999999999996 2024-09-21T14:58:58.528 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.528 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.528 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:58.528 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.017928+0000", 2024-09-21T14:58:58.528 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7811000000000004e-05 2024-09-21T14:58:58.528 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.528 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.528 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:58.528 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.018032+0000", 2024-09-21T14:58:58.528 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000104248 2024-09-21T14:58:58.528 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:58.529 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:58.529 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:58.529 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.529 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.529 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:20440 3.b 3:d6a6f68e:::rbd_object_map.1136c84d9a13:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-21T14:58:58.529 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:45.106428+0000", 2024-09-21T14:58:58.529 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 13.355805001, 2024-09-21T14:58:58.529 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0076349679999999998, 2024-09-21T14:58:58.529 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:58.529 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:58.529 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:58.529 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:58.529 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:58.529 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 20440 2024-09-21T14:58:58.529 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.529 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:58.529 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.530 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:58.530 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.106428+0000", 2024-09-21T14:58:58.530 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:58.530 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.530 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.530 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:58.530 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.106428+0000", 2024-09-21T14:58:58.530 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:58.530 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.530 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.530 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:58.530 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.106425+0000", 2024-09-21T14:58:58.530 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-21T14:58:58.530 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.530 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.530 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:58.531 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.106442+0000", 2024-09-21T14:58:58.531 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7237000000000001e-05 2024-09-21T14:58:58.531 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.531 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.531 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:58.531 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.106476+0000", 2024-09-21T14:58:58.531 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4004999999999997e-05 2024-09-21T14:58:58.531 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.531 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.531 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:58.531 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.106491+0000", 2024-09-21T14:58:58.531 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4865000000000001e-05 2024-09-21T14:58:58.531 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.531 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.531 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:58.531 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.106602+0000", 2024-09-21T14:58:58.532 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011113200000000001 2024-09-21T14:58:58.532 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.532 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.532 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op must wait for map", 2024-09-21T14:58:58.532 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.106611+0000", 2024-09-21T14:58:58.532 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9199999999999993e-06 2024-09-21T14:58:58.532 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.532 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.532 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:58.532 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.110475+0000", 2024-09-21T14:58:58.532 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0038633259999999998 2024-09-21T14:58:58.532 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.532 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.532 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:58.532 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.110622+0000", 2024-09-21T14:58:58.532 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014734200000000001 2024-09-21T14:58:58.533 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.533 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.533 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T14:58:58.533 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.111136+0000", 2024-09-21T14:58:58.533 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00051382299999999995 2024-09-21T14:58:58.533 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.533 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.533 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:58:58.533 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.112514+0000", 2024-09-21T14:58:58.533 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001378287 2024-09-21T14:58:58.533 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.533 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.533 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:58.533 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.113830+0000", 2024-09-21T14:58:58.534 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0013157939999999999 2024-09-21T14:58:58.534 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.534 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.534 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:58.534 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.113969+0000", 2024-09-21T14:58:58.534 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000138849 2024-09-21T14:58:58.534 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.534 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.534 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:58.534 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.114063+0000", 2024-09-21T14:58:58.534 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3999999999999994e-05 2024-09-21T14:58:58.534 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:58.534 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:58.534 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:58.534 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.535 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.535 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:20460 3.b 3:d6a6f68e:::rbd_object_map.1136c84d9a13:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-21T14:58:58.535 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:45.150183+0000", 2024-09-21T14:58:58.535 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 13.312049182999999, 2024-09-21T14:58:58.535 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0063818800000000004, 2024-09-21T14:58:58.535 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:58.535 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:58.535 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:58.535 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:58.535 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:58.535 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 20460 2024-09-21T14:58:58.535 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.535 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:58.535 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.535 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:58.535 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150183+0000", 2024-09-21T14:58:58.536 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:58.536 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.536 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.536 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:58.536 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150183+0000", 2024-09-21T14:58:58.536 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:58.536 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.536 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.536 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:58.536 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150180+0000", 2024-09-21T14:58:58.536 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999971 2024-09-21T14:58:58.536 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.536 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.536 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:58.536 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150211+0000", 2024-09-21T14:58:58.537 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0993e-05 2024-09-21T14:58:58.537 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.537 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.537 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:58.537 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150241+0000", 2024-09-21T14:58:58.537 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9589000000000001e-05 2024-09-21T14:58:58.537 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.537 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.537 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:58.537 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150261+0000", 2024-09-21T14:58:58.537 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.967e-05 2024-09-21T14:58:58.537 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.537 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.537 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:58.537 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150411+0000", 2024-09-21T14:58:58.537 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015004300000000001 2024-09-21T14:58:58.538 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.538 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.538 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:58.538 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150598+0000", 2024-09-21T14:58:58.538 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018712399999999999 2024-09-21T14:58:58.538 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.538 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.538 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T14:58:58.538 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.151390+0000", 2024-09-21T14:58:58.538 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00079194599999999997 2024-09-21T14:58:58.538 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.538 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.538 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:58.538 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.156302+0000", 2024-09-21T14:58:58.538 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0049123960000000003 2024-09-21T14:58:58.539 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.539 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.539 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:58:58.539 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.156435+0000", 2024-09-21T14:58:58.539 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013287300000000001 2024-09-21T14:58:58.539 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.539 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.539 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:58.539 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.156508+0000", 2024-09-21T14:58:58.539 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2504999999999998e-05 2024-09-21T14:58:58.539 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.539 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.539 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:58.539 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.156565+0000", 2024-09-21T14:58:58.540 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7773999999999999e-05 2024-09-21T14:58:58.540 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:58.540 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:58.540 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:58.540 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.540 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.540 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:20461 3.b 3:d6a6f68e:::rbd_object_map.1136c84d9a13:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-21T14:58:58.540 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:45.150284+0000", 2024-09-21T14:58:58.540 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 13.311948362000001, 2024-09-21T14:58:58.540 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0064029040000000001, 2024-09-21T14:58:58.540 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:58.540 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:58.540 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:58.540 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:58.540 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:58.540 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 20461 2024-09-21T14:58:58.540 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.541 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:58.541 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.541 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:58.541 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150284+0000", 2024-09-21T14:58:58.541 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:58.541 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.541 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.541 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:58.541 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150284+0000", 2024-09-21T14:58:58.541 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:58.541 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.541 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.541 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:58.541 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150283+0000", 2024-09-21T14:58:58.541 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:58.541 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.542 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.542 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:58.542 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150289+0000", 2024-09-21T14:58:58.542 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8799999999999996e-06 2024-09-21T14:58:58.542 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.542 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.542 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:58.542 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150319+0000", 2024-09-21T14:58:58.542 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9828999999999999e-05 2024-09-21T14:58:58.542 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.542 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.542 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:58.542 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150328+0000", 2024-09-21T14:58:58.542 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4770000000000003e-06 2024-09-21T14:58:58.542 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.542 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.543 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:58.543 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.152355+0000", 2024-09-21T14:58:58.543 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0020279870000000002 2024-09-21T14:58:58.543 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.543 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.543 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:58.543 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.152652+0000", 2024-09-21T14:58:58.543 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00029681 2024-09-21T14:58:58.543 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.543 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.543 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T14:58:58.543 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.153318+0000", 2024-09-21T14:58:58.543 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000666101 2024-09-21T14:58:58.543 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.543 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.544 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:58.544 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.156416+0000", 2024-09-21T14:58:58.544 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0030980729999999998 2024-09-21T14:58:58.544 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.544 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.544 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:58:58.544 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.156577+0000", 2024-09-21T14:58:58.544 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016049700000000001 2024-09-21T14:58:58.544 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.544 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.544 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:58.544 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.156628+0000", 2024-09-21T14:58:58.544 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0642000000000001e-05 2024-09-21T14:58:58.544 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.544 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.544 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:58.545 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.156687+0000", 2024-09-21T14:58:58.545 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9571000000000003e-05 2024-09-21T14:58:58.545 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:58.545 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:58.545 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:58.545 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.545 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.545 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:20462 3.b 3:d6a6f68e:::rbd_object_map.1136c84d9a13:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-21T14:58:58.545 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:45.150372+0000", 2024-09-21T14:58:58.545 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 13.311860279999999, 2024-09-21T14:58:58.545 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0075518110000000003, 2024-09-21T14:58:58.545 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:58.545 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:58.545 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:58.545 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:58.545 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:58.545 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 20462 2024-09-21T14:58:58.546 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.546 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:58.546 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.546 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:58.546 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150372+0000", 2024-09-21T14:58:58.546 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:58.546 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.546 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.546 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:58.546 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150372+0000", 2024-09-21T14:58:58.546 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:58.546 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.546 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.546 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:58.546 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150372+0000", 2024-09-21T14:58:58.546 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:58:58.547 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.547 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.547 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:58.547 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150378+0000", 2024-09-21T14:58:58.547 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3330000000000002e-06 2024-09-21T14:58:58.547 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.547 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.547 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:58.547 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150413+0000", 2024-09-21T14:58:58.547 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4798999999999997e-05 2024-09-21T14:58:58.547 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.547 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.547 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:58.547 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150423+0000", 2024-09-21T14:58:58.547 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.8099999999999992e-06 2024-09-21T14:58:58.548 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.548 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.548 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:58.548 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.154550+0000", 2024-09-21T14:58:58.548 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0041275959999999999 2024-09-21T14:58:58.548 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.548 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.548 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:58.548 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.154758+0000", 2024-09-21T14:58:58.548 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00020815499999999999 2024-09-21T14:58:58.548 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.548 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.548 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T14:58:58.548 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.155323+0000", 2024-09-21T14:58:58.548 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00056425400000000004 2024-09-21T14:58:58.549 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.549 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.549 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:58:58.549 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.156702+0000", 2024-09-21T14:58:58.549 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001379676 2024-09-21T14:58:58.549 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.549 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.549 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:58.549 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.157662+0000", 2024-09-21T14:58:58.549 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00095948099999999998 2024-09-21T14:58:58.549 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.549 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.549 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:58.549 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.157816+0000", 2024-09-21T14:58:58.549 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000154666 2024-09-21T14:58:58.549 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.549 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.550 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:58.550 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.157924+0000", 2024-09-21T14:58:58.550 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000107798 2024-09-21T14:58:58.550 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:58.550 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:58.550 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:58.550 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.550 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.550 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:20603 3.b 3:d6a6f68e:::rbd_object_map.1136c84d9a13:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-21T14:58:58.550 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:45.422857+0000", 2024-09-21T14:58:58.550 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 13.039375565, 2024-09-21T14:58:58.550 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0061270960000000003, 2024-09-21T14:58:58.550 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:58:58.550 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:58.550 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:58:58.551 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:58:58.551 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:58.551 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 20603 2024-09-21T14:58:58.551 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.551 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:58:58.551 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.551 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:58:58.551 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.422857+0000", 2024-09-21T14:58:58.551 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:58.551 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.551 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.551 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:58:58.551 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.422857+0000", 2024-09-21T14:58:58.551 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:58:58.552 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.552 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.552 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:58:58.552 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.422856+0000", 2024-09-21T14:58:58.552 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:58.552 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.552 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.552 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:58:58.552 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.422866+0000", 2024-09-21T14:58:58.552 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0360000000000001e-05 2024-09-21T14:58:58.552 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.552 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.552 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:58:58.552 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.422885+0000", 2024-09-21T14:58:58.553 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8576999999999998e-05 2024-09-21T14:58:58.553 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.553 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.553 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:58:58.553 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.422899+0000", 2024-09-21T14:58:58.553 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4711e-05 2024-09-21T14:58:58.553 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.553 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.553 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:58:58.553 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.424217+0000", 2024-09-21T14:58:58.553 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001318035 2024-09-21T14:58:58.553 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.553 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.553 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:58:58.554 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.424433+0000", 2024-09-21T14:58:58.554 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000215596 2024-09-21T14:58:58.554 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.554 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.554 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T14:58:58.554 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.425172+0000", 2024-09-21T14:58:58.554 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00073899299999999996 2024-09-21T14:58:58.554 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.554 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.554 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:58:58.554 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.426780+0000", 2024-09-21T14:58:58.554 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0016078760000000001 2024-09-21T14:58:58.554 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.554 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.555 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:58.555 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.428828+0000", 2024-09-21T14:58:58.555 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0020481179999999998 2024-09-21T14:58:58.555 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.555 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.555 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:58:58.555 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.428929+0000", 2024-09-21T14:58:58.555 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000100603 2024-09-21T14:58:58.555 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:58:58.555 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:58:58.555 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:58:58.555 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.428984+0000", 2024-09-21T14:58:58.555 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5587000000000002e-05 2024-09-21T14:58:58.555 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:58.555 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:58.556 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:58.556 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:58:58.556 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:58:58.556 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:58:58.557 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T14:58:58.558 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:58:58.553+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:58:58.658 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:58:58.656+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:58:58.665 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:58:58.665 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:58:58.665 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T14:58:58.665 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:58:58.675 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T14:58:58.759 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:58:58.757+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:58:58.782 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:58:58.782 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:58:58.782 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T14:58:58.782 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T14:58:58.782 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:58:58.791 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T14:58:58.860 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:58:58.856+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:58:58.901 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:58:58.901 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T14:58:58.901 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T14:58:58.901 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T14:58:58.901 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.901 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:1473 3.15 3:af2afa6a:::rbd_data.1136515ecb6b.000000000000001b:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T14:58:58.901 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:30.421730+0000", 2024-09-21T14:58:58.901 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 28.476732917, 2024-09-21T14:58:58.901 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021478937, 2024-09-21T14:58:58.901 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:58.901 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:58.901 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:58.902 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:58.902 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:58.902 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1473 2024-09-21T14:58:58.902 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.902 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:58.902 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.902 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:58.902 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421730+0000", 2024-09-21T14:58:58.902 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:58.902 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.902 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.902 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:58.903 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421730+0000", 2024-09-21T14:58:58.903 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:58.903 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.903 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.903 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:58.903 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421728+0000", 2024-09-21T14:58:58.903 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-21T14:58:58.903 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.903 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.903 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:58.903 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421739+0000", 2024-09-21T14:58:58.903 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0502e-05 2024-09-21T14:58:58.903 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.904 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.904 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:58.904 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421762+0000", 2024-09-21T14:58:58.904 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3405999999999999e-05 2024-09-21T14:58:58.904 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.904 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.904 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:58.904 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421773+0000", 2024-09-21T14:58:58.904 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0744000000000001e-05 2024-09-21T14:58:58.904 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.904 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.904 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:58.904 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421838+0000", 2024-09-21T14:58:58.904 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5486000000000003e-05 2024-09-21T14:58:58.905 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.905 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.905 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:58.905 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421955+0000", 2024-09-21T14:58:58.905 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000116765 2024-09-21T14:58:58.905 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.905 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.905 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 6", 2024-09-21T14:58:58.905 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.425572+0000", 2024-09-21T14:58:58.905 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0036171380000000002 2024-09-21T14:58:58.905 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.905 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.905 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:58.906 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.442973+0000", 2024-09-21T14:58:58.906 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.017400268 2024-09-21T14:58:58.906 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.906 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.906 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:58.906 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.442996+0000", 2024-09-21T14:58:58.906 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3139e-05 2024-09-21T14:58:58.906 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.906 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.906 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:58.906 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.443079+0000", 2024-09-21T14:58:58.906 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2832999999999998e-05 2024-09-21T14:58:58.906 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.906 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.907 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:58.907 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.443209+0000", 2024-09-21T14:58:58.907 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013052499999999999 2024-09-21T14:58:58.907 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:58.907 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:58.907 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:58.907 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.907 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.907 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:1678 3.c 3:377cc7e1:::rbd_data.1136515ecb6b.000000000000000f:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T14:58:58.907 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:30.988085+0000", 2024-09-21T14:58:58.907 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 27.910378323, 2024-09-21T14:58:58.907 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.032974802999999997, 2024-09-21T14:58:58.907 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:58.907 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:58.908 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:58.908 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:58.908 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:58.908 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1678 2024-09-21T14:58:58.908 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.908 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:58.908 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.908 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:58.908 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988085+0000", 2024-09-21T14:58:58.908 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:58.908 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.908 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.908 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:58.909 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988085+0000", 2024-09-21T14:58:58.909 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:58.909 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.909 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.909 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:58.909 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988083+0000", 2024-09-21T14:58:58.909 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-21T14:58:58.909 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.909 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.909 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:58.909 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988093+0000", 2024-09-21T14:58:58.909 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8800000000000003e-06 2024-09-21T14:58:58.909 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.909 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.909 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:58.910 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988113+0000", 2024-09-21T14:58:58.910 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0217999999999999e-05 2024-09-21T14:58:58.910 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.910 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.910 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:58.910 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988123+0000", 2024-09-21T14:58:58.910 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5859999999999994e-06 2024-09-21T14:58:58.910 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.910 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.910 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:58.910 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988203+0000", 2024-09-21T14:58:58.910 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0834000000000006e-05 2024-09-21T14:58:58.910 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.910 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.911 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:58.911 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988376+0000", 2024-09-21T14:58:58.911 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017301299999999999 2024-09-21T14:58:58.911 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.911 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.911 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 7", 2024-09-21T14:58:58.911 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.996338+0000", 2024-09-21T14:58:58.911 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0079618520000000002 2024-09-21T14:58:58.911 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.911 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.911 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:58.911 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.020881+0000", 2024-09-21T14:58:58.911 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024542731000000002 2024-09-21T14:58:58.911 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.912 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.912 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:58.912 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.020904+0000", 2024-09-21T14:58:58.912 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2861e-05 2024-09-21T14:58:58.912 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.912 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.912 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:58.912 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.020987+0000", 2024-09-21T14:58:58.912 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2806000000000006e-05 2024-09-21T14:58:58.912 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.912 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.912 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:58.912 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.021060+0000", 2024-09-21T14:58:58.912 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2871000000000001e-05 2024-09-21T14:58:58.912 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:58.913 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:58.913 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:58.913 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.913 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.913 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9369 3.3 3:c725132c:::rbd_data.11363a21d2be.0000000000000007:10 [read 24576~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:58.913 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157241+0000", 2024-09-21T14:58:58.913 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 19.741221877000001, 2024-09-21T14:58:58.913 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018289156000000001, 2024-09-21T14:58:58.913 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:58.913 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:58.913 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:58.913 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:58.913 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:58.913 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9369 2024-09-21T14:58:58.913 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.913 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:58.914 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.914 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:58.914 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157241+0000", 2024-09-21T14:58:58.914 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:58.914 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.914 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.914 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:58.914 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157241+0000", 2024-09-21T14:58:58.914 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:58.914 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.914 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.914 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:58.914 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157241+0000", 2024-09-21T14:58:58.915 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:58.915 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.915 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.915 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:58.915 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157244+0000", 2024-09-21T14:58:58.915 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5049999999999999e-06 2024-09-21T14:58:58.915 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.915 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.915 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:58.915 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157255+0000", 2024-09-21T14:58:58.915 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0797e-05 2024-09-21T14:58:58.915 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.915 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.916 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:58.916 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157260+0000", 2024-09-21T14:58:58.916 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1749999999999996e-06 2024-09-21T14:58:58.916 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.916 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.916 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:58.916 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175237+0000", 2024-09-21T14:58:58.916 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.017976405000000001 2024-09-21T14:58:58.916 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.916 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.916 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:58.916 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175331+0000", 2024-09-21T14:58:58.916 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4508999999999999e-05 2024-09-21T14:58:58.916 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.916 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.916 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:58.917 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175530+0000", 2024-09-21T14:58:58.917 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00019928200000000001 2024-09-21T14:58:58.917 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:58.917 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:58.917 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:58.917 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.917 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.917 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9370 3.3 3:c725132c:::rbd_data.11363a21d2be.0000000000000007:10 [read 163840~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:58.917 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157279+0000", 2024-09-21T14:58:58.917 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 19.741183699, 2024-09-21T14:58:58.917 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018472379000000001, 2024-09-21T14:58:58.917 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:58.917 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:58.917 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:58.917 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:58.918 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:58.918 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9370 2024-09-21T14:58:58.918 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.918 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:58.918 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.918 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:58.918 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157279+0000", 2024-09-21T14:58:58.918 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:58.918 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.918 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.918 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:58.918 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157279+0000", 2024-09-21T14:58:58.918 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:58.919 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.919 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.919 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:58.919 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157279+0000", 2024-09-21T14:58:58.919 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:58.919 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.919 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.919 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:58.919 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157282+0000", 2024-09-21T14:58:58.919 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2229999999999999e-06 2024-09-21T14:58:58.919 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.919 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.919 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:58.919 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157294+0000", 2024-09-21T14:58:58.919 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2173e-05 2024-09-21T14:58:58.920 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.920 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.920 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:58.920 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157300+0000", 2024-09-21T14:58:58.920 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.93e-06 2024-09-21T14:58:58.920 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.920 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.920 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:58.920 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175569+0000", 2024-09-21T14:58:58.920 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018268376999999999 2024-09-21T14:58:58.920 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.920 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.920 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:58.920 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175622+0000", 2024-09-21T14:58:58.920 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3808999999999999e-05 2024-09-21T14:58:58.921 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.921 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.921 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:58.921 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175752+0000", 2024-09-21T14:58:58.921 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012938700000000001 2024-09-21T14:58:58.921 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:58.921 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:58.921 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:58.921 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.921 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.921 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9371 3.3 3:c0bf6a9b:::rbd_data.11363a21d2be.0000000000000000:10 [read 73728~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:58.921 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157306+0000", 2024-09-21T14:58:58.922 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 19.741157588, 2024-09-21T14:58:58.922 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018755473000000002, 2024-09-21T14:58:58.922 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:58.922 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:58.922 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:58.922 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:58.922 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:58.922 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9371 2024-09-21T14:58:58.922 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.922 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:58.922 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.922 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:58.922 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157306+0000", 2024-09-21T14:58:58.922 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:58.923 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.923 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.923 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:58.923 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157306+0000", 2024-09-21T14:58:58.923 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:58.923 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.923 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.923 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:58.923 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157305+0000", 2024-09-21T14:58:58.923 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:58.923 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.923 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.923 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:58.923 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157312+0000", 2024-09-21T14:58:58.924 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3289999999999996e-06 2024-09-21T14:58:58.924 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.924 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.924 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:58.924 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157324+0000", 2024-09-21T14:58:58.924 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1725e-05 2024-09-21T14:58:58.924 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.924 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.924 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:58.924 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157339+0000", 2024-09-21T14:58:58.924 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5211999999999999e-05 2024-09-21T14:58:58.924 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.924 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.924 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:58.925 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175798+0000", 2024-09-21T14:58:58.925 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018458781 2024-09-21T14:58:58.925 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.925 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.925 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:58.925 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175899+0000", 2024-09-21T14:58:58.925 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000100784 2024-09-21T14:58:58.925 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.925 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.925 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:58.925 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176061+0000", 2024-09-21T14:58:58.925 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00016216099999999999 2024-09-21T14:58:58.925 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:58.925 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:58.926 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:58.926 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.926 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.926 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9372 3.3 3:c0bf6a9b:::rbd_data.11363a21d2be.0000000000000000:10 [read 212992~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:58.926 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157345+0000", 2024-09-21T14:58:58.926 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 19.741118241999999, 2024-09-21T14:58:58.926 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018926493999999999, 2024-09-21T14:58:58.926 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:58.926 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:58.926 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:58.926 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:58.926 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:58.926 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9372 2024-09-21T14:58:58.926 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.927 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:58.927 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.927 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:58.927 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157345+0000", 2024-09-21T14:58:58.927 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:58.927 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.927 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.927 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:58.927 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157345+0000", 2024-09-21T14:58:58.927 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:58.927 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.927 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.927 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:58.928 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157344+0000", 2024-09-21T14:58:58.928 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:58.928 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.928 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.928 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:58.928 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157348+0000", 2024-09-21T14:58:58.928 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1389999999999999e-06 2024-09-21T14:58:58.928 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.928 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.928 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:58.928 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157372+0000", 2024-09-21T14:58:58.928 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4711000000000001e-05 2024-09-21T14:58:58.928 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.929 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.929 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:58.929 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157378+0000", 2024-09-21T14:58:58.929 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9909999999999999e-06 2024-09-21T14:58:58.929 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.929 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.929 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:58.929 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176090+0000", 2024-09-21T14:58:58.929 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018712111999999999 2024-09-21T14:58:58.929 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.929 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.929 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:58.929 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176134+0000", 2024-09-21T14:58:58.929 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3377999999999997e-05 2024-09-21T14:58:58.929 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.930 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.930 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:58.930 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176271+0000", 2024-09-21T14:58:58.930 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013763499999999999 2024-09-21T14:58:58.930 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:58.930 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:58.930 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:58.930 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.930 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.930 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9373 3.3 3:c0bf6a9b:::rbd_data.11363a21d2be.0000000000000000:10 [read 352256~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:58.930 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157384+0000", 2024-09-21T14:58:58.930 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 19.741079469999999, 2024-09-21T14:58:58.931 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019154937, 2024-09-21T14:58:58.931 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:58.931 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:58.931 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:58.931 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:58.931 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:58.931 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9373 2024-09-21T14:58:58.931 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.931 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:58.931 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.931 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:58.931 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157384+0000", 2024-09-21T14:58:58.931 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:58.931 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.931 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.932 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:58.932 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157384+0000", 2024-09-21T14:58:58.932 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:58.932 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.932 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.932 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:58.932 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157383+0000", 2024-09-21T14:58:58.932 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:58.932 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.932 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.932 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:58.932 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157386+0000", 2024-09-21T14:58:58.932 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.191e-06 2024-09-21T14:58:58.932 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.932 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.932 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:58.933 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157393+0000", 2024-09-21T14:58:58.933 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5339999999999996e-06 2024-09-21T14:58:58.933 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.933 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.933 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:58.933 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157397+0000", 2024-09-21T14:58:58.933 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5220000000000001e-06 2024-09-21T14:58:58.933 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.933 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.933 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:58.933 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176302+0000", 2024-09-21T14:58:58.933 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018905113000000001 2024-09-21T14:58:58.933 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.933 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.934 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:58.934 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176379+0000", 2024-09-21T14:58:58.934 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6372999999999995e-05 2024-09-21T14:58:58.934 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.934 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.934 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:58.934 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176539+0000", 2024-09-21T14:58:58.934 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015973800000000001 2024-09-21T14:58:58.934 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:58.934 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:58.934 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:58.934 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.934 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.934 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9374 3.3 3:c0bf6a9b:::rbd_data.11363a21d2be.0000000000000000:10 [read 491520~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:58.935 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157401+0000", 2024-09-21T14:58:58.935 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 19.741061685999998, 2024-09-21T14:58:58.935 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019387755, 2024-09-21T14:58:58.935 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:58.935 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:58.935 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:58.935 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:58.935 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:58.935 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9374 2024-09-21T14:58:58.935 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.935 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:58.935 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.935 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:58.935 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157401+0000", 2024-09-21T14:58:58.935 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:58.936 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.936 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.936 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:58.936 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157401+0000", 2024-09-21T14:58:58.936 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:58.936 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.936 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.936 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:58.936 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157401+0000", 2024-09-21T14:58:58.936 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:58.936 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.936 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.936 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:58.936 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157414+0000", 2024-09-21T14:58:58.937 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2417e-05 2024-09-21T14:58:58.937 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.937 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.937 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:58.937 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157424+0000", 2024-09-21T14:58:58.937 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0861e-05 2024-09-21T14:58:58.937 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.937 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.937 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:58.937 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157429+0000", 2024-09-21T14:58:58.937 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0499999999999999e-06 2024-09-21T14:58:58.937 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.937 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.937 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:58.937 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176596+0000", 2024-09-21T14:58:58.938 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019166552 2024-09-21T14:58:58.938 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.938 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.938 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:58.938 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176647+0000", 2024-09-21T14:58:58.938 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1112000000000001e-05 2024-09-21T14:58:58.938 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.938 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.938 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:58.938 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176789+0000", 2024-09-21T14:58:58.938 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014209300000000001 2024-09-21T14:58:58.938 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:58.938 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:58.938 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:58.939 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.939 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.939 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9401 3.3 3:c725132c:::rbd_data.11363a21d2be.0000000000000007:10 [read 32768~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:58.939 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157621+0000", 2024-09-21T14:58:58.939 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 19.740841877000001, 2024-09-21T14:58:58.939 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019381711999999999, 2024-09-21T14:58:58.939 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:58.939 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:58.939 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:58.939 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:58.939 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:58.939 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9401 2024-09-21T14:58:58.939 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.940 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:58.940 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.940 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:58.940 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157621+0000", 2024-09-21T14:58:58.940 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:58.940 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.940 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.940 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:58.940 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157621+0000", 2024-09-21T14:58:58.940 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:58.940 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.940 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.940 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:58.940 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157620+0000", 2024-09-21T14:58:58.941 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:58:58.941 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.941 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.941 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:58.941 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157626+0000", 2024-09-21T14:58:58.941 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7570000000000001e-06 2024-09-21T14:58:58.941 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.941 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.941 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:58.941 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157636+0000", 2024-09-21T14:58:58.941 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8339999999999994e-06 2024-09-21T14:58:58.941 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.941 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.941 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:58.942 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157643+0000", 2024-09-21T14:58:58.942 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9439999999999999e-06 2024-09-21T14:58:58.942 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.942 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.942 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:58.942 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176816+0000", 2024-09-21T14:58:58.942 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019172872000000001 2024-09-21T14:58:58.942 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.942 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.942 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:58.942 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176878+0000", 2024-09-21T14:58:58.942 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1734000000000002e-05 2024-09-21T14:58:58.942 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.943 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.943 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:58.943 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177003+0000", 2024-09-21T14:58:58.943 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012537400000000001 2024-09-21T14:58:58.943 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:58.943 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:58.943 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:58.943 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.943 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.943 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9402 3.3 3:c725132c:::rbd_data.11363a21d2be.0000000000000007:10 [read 172032~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:58.943 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157649+0000", 2024-09-21T14:58:58.943 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 19.740814005000001, 2024-09-21T14:58:58.944 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019567404, 2024-09-21T14:58:58.944 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:58.944 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:58.944 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:58.944 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:58.944 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:58.944 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9402 2024-09-21T14:58:58.944 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.944 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:58.944 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.944 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:58.944 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157649+0000", 2024-09-21T14:58:58.944 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:58.944 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.944 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.945 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:58.945 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157649+0000", 2024-09-21T14:58:58.945 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:58.945 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.945 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.945 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:58.945 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157649+0000", 2024-09-21T14:58:58.945 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:58.945 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.945 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.945 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:58.945 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157652+0000", 2024-09-21T14:58:58.945 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8779999999999998e-06 2024-09-21T14:58:58.945 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.946 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.946 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:58.946 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157657+0000", 2024-09-21T14:58:58.946 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6659999999999996e-06 2024-09-21T14:58:58.946 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.946 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.946 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:58.946 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157662+0000", 2024-09-21T14:58:58.946 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5519999999999998e-06 2024-09-21T14:58:58.946 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.946 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.946 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:58.946 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177040+0000", 2024-09-21T14:58:58.946 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019377807 2024-09-21T14:58:58.947 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.947 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.947 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:58.947 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177085+0000", 2024-09-21T14:58:58.947 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5195999999999999e-05 2024-09-21T14:58:58.947 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.947 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.947 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:58.947 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177217+0000", 2024-09-21T14:58:58.947 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013172600000000001 2024-09-21T14:58:58.947 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:58.947 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:58.947 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:58.947 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.947 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.947 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9403 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:58.948 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157949+0000", 2024-09-21T14:58:58.948 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 19.740513675999999, 2024-09-21T14:58:58.948 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023540373999999999, 2024-09-21T14:58:58.948 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:58.948 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:58.948 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:58.948 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:58.948 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:58.948 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9403 2024-09-21T14:58:58.948 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.948 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:58.948 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.948 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:58.948 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157949+0000", 2024-09-21T14:58:58.949 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:58.949 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.949 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.949 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:58.949 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157949+0000", 2024-09-21T14:58:58.949 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:58.949 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.949 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.949 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:58.949 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157948+0000", 2024-09-21T14:58:58.949 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:58.949 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.949 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.949 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:58.949 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157956+0000", 2024-09-21T14:58:58.949 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7999999999999999e-06 2024-09-21T14:58:58.950 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.950 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.950 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:58.950 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157969+0000", 2024-09-21T14:58:58.950 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2792000000000001e-05 2024-09-21T14:58:58.950 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.950 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.950 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:58.950 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157977+0000", 2024-09-21T14:58:58.950 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.704e-06 2024-09-21T14:58:58.950 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.950 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.950 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:58.950 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177246+0000", 2024-09-21T14:58:58.950 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019268264 2024-09-21T14:58:58.951 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.951 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.951 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:58.951 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177315+0000", 2024-09-21T14:58:58.951 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9480999999999994e-05 2024-09-21T14:58:58.951 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.951 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.951 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:58:58.951 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177615+0000", 2024-09-21T14:58:58.951 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00029929499999999999 2024-09-21T14:58:58.951 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.951 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.951 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:58.951 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181333+0000", 2024-09-21T14:58:58.952 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0037183149999999998 2024-09-21T14:58:58.952 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.952 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.952 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:58.952 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181374+0000", 2024-09-21T14:58:58.952 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.0723999999999998e-05 2024-09-21T14:58:58.952 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.952 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.952 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:58.952 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181434+0000", 2024-09-21T14:58:58.952 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0024e-05 2024-09-21T14:58:58.952 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.952 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.952 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:58.952 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181490+0000", 2024-09-21T14:58:58.952 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6245000000000001e-05 2024-09-21T14:58:58.953 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:58.953 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:58.953 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:58.953 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.953 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.953 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9404 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:58.953 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.160480+0000", 2024-09-21T14:58:58.953 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 19.737983325999998, 2024-09-21T14:58:58.953 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021137686999999999, 2024-09-21T14:58:58.953 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:58.953 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:58.953 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:58.953 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:58.953 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:58.953 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9404 2024-09-21T14:58:58.954 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.954 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:58.954 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.954 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:58.954 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160480+0000", 2024-09-21T14:58:58.954 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:58.954 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.954 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.954 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:58.954 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160480+0000", 2024-09-21T14:58:58.954 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:58.954 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.954 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.955 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:58.955 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160478+0000", 2024-09-21T14:58:58.955 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:58.955 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.955 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.955 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:58.955 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160486+0000", 2024-09-21T14:58:58.955 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9969999999999998e-06 2024-09-21T14:58:58.955 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.955 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.955 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:58.955 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160502+0000", 2024-09-21T14:58:58.955 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5650000000000001e-05 2024-09-21T14:58:58.955 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.955 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.956 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:58.956 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160511+0000", 2024-09-21T14:58:58.956 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.3610000000000007e-06 2024-09-21T14:58:58.956 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.956 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.956 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:58.956 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.178035+0000", 2024-09-21T14:58:58.956 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.017523520000000001 2024-09-21T14:58:58.956 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.956 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.956 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:58.956 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.178151+0000", 2024-09-21T14:58:58.956 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000116013 2024-09-21T14:58:58.956 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.956 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.957 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:58:58.957 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.178408+0000", 2024-09-21T14:58:58.957 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00025743400000000002 2024-09-21T14:58:58.957 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.957 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.957 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:58.957 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181501+0000", 2024-09-21T14:58:58.957 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.003092891 2024-09-21T14:58:58.957 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.957 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.957 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:58.957 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181538+0000", 2024-09-21T14:58:58.957 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6977999999999997e-05 2024-09-21T14:58:58.957 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.958 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.958 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:58.958 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181582+0000", 2024-09-21T14:58:58.958 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3849e-05 2024-09-21T14:58:58.958 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.958 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.958 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:58.958 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181617+0000", 2024-09-21T14:58:58.958 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5423000000000003e-05 2024-09-21T14:58:58.958 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:58.958 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:58.958 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:58.958 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.958 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.959 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9406 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:58.959 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.166040+0000", 2024-09-21T14:58:58.959 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 19.732423446999999, 2024-09-21T14:58:58.959 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018094524000000001, 2024-09-21T14:58:58.959 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:58.959 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:58.959 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:58.959 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:58.959 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:58.959 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9406 2024-09-21T14:58:58.959 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.959 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:58.959 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.959 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:58.959 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.166040+0000", 2024-09-21T14:58:58.960 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:58.960 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.960 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.960 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:58.960 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.166040+0000", 2024-09-21T14:58:58.960 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:58.960 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.960 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.960 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:58.960 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.166038+0000", 2024-09-21T14:58:58.960 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:58.960 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.960 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.960 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:58.960 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.166051+0000", 2024-09-21T14:58:58.961 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2638e-05 2024-09-21T14:58:58.961 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.961 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.961 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:58.961 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.166070+0000", 2024-09-21T14:58:58.961 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8691000000000001e-05 2024-09-21T14:58:58.961 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.961 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.961 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:58.961 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.166089+0000", 2024-09-21T14:58:58.961 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9286000000000001e-05 2024-09-21T14:58:58.961 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.961 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.961 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:58.961 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.180194+0000", 2024-09-21T14:58:58.962 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014105039999999999 2024-09-21T14:58:58.962 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.962 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.962 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:58.962 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.180340+0000", 2024-09-21T14:58:58.962 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014623199999999999 2024-09-21T14:58:58.962 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.962 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.962 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:58:58.962 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.180818+0000", 2024-09-21T14:58:58.962 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000477489 2024-09-21T14:58:58.962 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.962 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.962 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:58.962 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.183808+0000", 2024-09-21T14:58:58.963 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0029897610000000001 2024-09-21T14:58:58.963 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.963 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.963 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:58.963 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.183821+0000", 2024-09-21T14:58:58.963 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3251999999999999e-05 2024-09-21T14:58:58.963 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.963 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.963 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:58.963 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.183872+0000", 2024-09-21T14:58:58.963 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0900999999999999e-05 2024-09-21T14:58:58.963 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.963 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.963 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:58.964 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.184134+0000", 2024-09-21T14:58:58.964 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000262484 2024-09-21T14:58:58.964 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:58.964 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:58.964 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:58.964 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.964 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.964 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9407 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:58.964 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.168094+0000", 2024-09-21T14:58:58.964 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 19.730368826999999, 2024-09-21T14:58:58.964 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019444860000000001, 2024-09-21T14:58:58.964 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:58.964 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:58.964 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:58.964 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:58.965 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:58.965 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9407 2024-09-21T14:58:58.965 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.965 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:58.965 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.965 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:58.965 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.168094+0000", 2024-09-21T14:58:58.965 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:58.965 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.965 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.965 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:58.965 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.168094+0000", 2024-09-21T14:58:58.965 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:58.966 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.966 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.966 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:58.966 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.168093+0000", 2024-09-21T14:58:58.966 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:58:58.966 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.966 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.966 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:58.966 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.168119+0000", 2024-09-21T14:58:58.966 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5803999999999999e-05 2024-09-21T14:58:58.966 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.966 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.966 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:58.966 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.168135+0000", 2024-09-21T14:58:58.967 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6359999999999999e-05 2024-09-21T14:58:58.967 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.967 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.967 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:58.967 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.168144+0000", 2024-09-21T14:58:58.967 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4270000000000008e-06 2024-09-21T14:58:58.967 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.967 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.967 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:58.967 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181943+0000", 2024-09-21T14:58:58.967 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013799617 2024-09-21T14:58:58.967 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.967 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.967 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:58.967 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.182052+0000", 2024-09-21T14:58:58.967 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000108924 2024-09-21T14:58:58.968 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.968 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.968 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:58:58.968 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.183431+0000", 2024-09-21T14:58:58.968 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0013783619999999999 2024-09-21T14:58:58.968 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.968 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.968 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:58.968 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.187276+0000", 2024-09-21T14:58:58.968 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0038450469999999999 2024-09-21T14:58:58.968 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.968 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.968 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:58.968 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.187288+0000", 2024-09-21T14:58:58.968 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2414e-05 2024-09-21T14:58:58.969 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.969 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.969 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:58.969 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.187414+0000", 2024-09-21T14:58:58.969 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012614399999999999 2024-09-21T14:58:58.969 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.969 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.969 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:58.969 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.187539+0000", 2024-09-21T14:58:58.969 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012481800000000001 2024-09-21T14:58:58.969 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:58.969 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:58.969 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:58.970 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.970 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.970 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9410 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:58.970 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.174103+0000", 2024-09-21T14:58:58.970 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 19.724359918000001, 2024-09-21T14:58:58.970 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025493703, 2024-09-21T14:58:58.970 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:58.970 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:58.970 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:58.970 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:58.970 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:58.970 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9410 2024-09-21T14:58:58.970 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.970 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:58.970 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.971 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:58.971 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.174103+0000", 2024-09-21T14:58:58.971 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:58.971 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.971 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.971 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:58.971 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.174103+0000", 2024-09-21T14:58:58.971 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:58.971 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.971 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.971 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:58.971 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.174102+0000", 2024-09-21T14:58:58.971 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:58:58.972 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.972 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.972 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:58.972 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.174108+0000", 2024-09-21T14:58:58.972 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1149999999999999e-06 2024-09-21T14:58:58.972 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.972 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.972 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:58.972 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.174117+0000", 2024-09-21T14:58:58.972 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4870000000000002e-06 2024-09-21T14:58:58.972 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.972 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.972 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:58.972 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.174122+0000", 2024-09-21T14:58:58.972 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9790000000000001e-06 2024-09-21T14:58:58.972 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.973 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.973 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:58.973 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.187830+0000", 2024-09-21T14:58:58.973 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013708048 2024-09-21T14:58:58.973 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.973 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.973 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:58.973 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.187913+0000", 2024-09-21T14:58:58.973 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2763999999999996e-05 2024-09-21T14:58:58.973 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.973 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.973 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:58:58.973 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.188230+0000", 2024-09-21T14:58:58.973 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00031745100000000001 2024-09-21T14:58:58.974 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.974 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.974 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:58.974 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.192708+0000", 2024-09-21T14:58:58.974 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0044783890000000002 2024-09-21T14:58:58.974 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.974 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.974 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:58.974 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199487+0000", 2024-09-21T14:58:58.974 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0067782149999999998 2024-09-21T14:58:58.974 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.974 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.974 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:58.975 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199560+0000", 2024-09-21T14:58:58.975 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3752999999999997e-05 2024-09-21T14:58:58.975 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.975 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.975 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:58.975 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199597+0000", 2024-09-21T14:58:58.975 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6526999999999998e-05 2024-09-21T14:58:58.975 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:58.975 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:58.975 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:58.975 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.975 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.975 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9411 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:58.975 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.175943+0000", 2024-09-21T14:58:58.975 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 19.722519899000002, 2024-09-21T14:58:58.976 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023773181000000001, 2024-09-21T14:58:58.976 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:58.976 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:58.976 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:58.976 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:58.976 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:58.976 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9411 2024-09-21T14:58:58.976 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.976 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:58.976 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.976 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:58.976 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175943+0000", 2024-09-21T14:58:58.976 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:58.976 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.977 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.977 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:58.977 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175943+0000", 2024-09-21T14:58:58.977 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:58.977 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.977 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.977 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:58.977 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175942+0000", 2024-09-21T14:58:58.977 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:58:58.977 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.977 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.977 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:58.977 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175948+0000", 2024-09-21T14:58:58.977 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9689999999999999e-06 2024-09-21T14:58:58.977 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.978 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.978 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:58.978 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175960+0000", 2024-09-21T14:58:58.978 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1745e-05 2024-09-21T14:58:58.978 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.978 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.978 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:58.978 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175965+0000", 2024-09-21T14:58:58.978 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.203e-06 2024-09-21T14:58:58.978 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.978 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.978 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:58.978 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.189900+0000", 2024-09-21T14:58:58.978 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013934940999999999 2024-09-21T14:58:58.978 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.979 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.979 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:58.979 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.189988+0000", 2024-09-21T14:58:58.979 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8419999999999997e-05 2024-09-21T14:58:58.979 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.979 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.979 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:58:58.979 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.190321+0000", 2024-09-21T14:58:58.979 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00033231800000000001 2024-09-21T14:58:58.979 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.979 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.979 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:58.979 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.192746+0000", 2024-09-21T14:58:58.979 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0024254400000000001 2024-09-21T14:58:58.979 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.980 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.980 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:58.980 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199605+0000", 2024-09-21T14:58:58.980 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0068582809999999999 2024-09-21T14:58:58.980 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.980 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.980 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:58.980 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199680+0000", 2024-09-21T14:58:58.980 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5630000000000006e-05 2024-09-21T14:58:58.980 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.980 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.980 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:58.980 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199716+0000", 2024-09-21T14:58:58.980 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6248999999999998e-05 2024-09-21T14:58:58.980 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:58.980 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:58.980 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:58.981 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.981 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.981 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9412 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:58.981 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.177402+0000", 2024-09-21T14:58:58.981 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 19.721060802, 2024-09-21T14:58:58.981 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022445590000000001, 2024-09-21T14:58:58.981 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:58.981 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:58.981 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:58.981 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:58.981 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:58.981 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9412 2024-09-21T14:58:58.981 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.981 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:58.982 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.982 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:58.982 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177402+0000", 2024-09-21T14:58:58.982 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:58.982 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.982 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.982 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:58.982 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177402+0000", 2024-09-21T14:58:58.982 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:58.982 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.982 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.982 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:58.982 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177401+0000", 2024-09-21T14:58:58.982 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:58:58.982 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.983 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.983 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:58.983 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177409+0000", 2024-09-21T14:58:58.983 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4710000000000001e-06 2024-09-21T14:58:58.983 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.983 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.983 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:58.983 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177419+0000", 2024-09-21T14:58:58.983 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0351e-05 2024-09-21T14:58:58.983 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.983 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.983 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:58.983 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177424+0000", 2024-09-21T14:58:58.983 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1919999999999996e-06 2024-09-21T14:58:58.983 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.984 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.984 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:58.984 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.191343+0000", 2024-09-21T14:58:58.984 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.01391878 2024-09-21T14:58:58.984 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.984 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.984 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:58.984 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.191446+0000", 2024-09-21T14:58:58.984 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000102453 2024-09-21T14:58:58.984 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.984 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.984 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:58:58.984 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.192200+0000", 2024-09-21T14:58:58.984 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00075476899999999997 2024-09-21T14:58:58.984 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.985 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.985 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:58.985 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.193866+0000", 2024-09-21T14:58:58.985 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0016657519999999999 2024-09-21T14:58:58.985 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.985 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.985 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:58.985 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199724+0000", 2024-09-21T14:58:58.985 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0058579529999999999 2024-09-21T14:58:58.985 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.985 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.985 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:58.985 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199803+0000", 2024-09-21T14:58:58.985 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8662e-05 2024-09-21T14:58:58.985 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.985 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.986 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:58.986 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199848+0000", 2024-09-21T14:58:58.986 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5216000000000002e-05 2024-09-21T14:58:58.986 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:58.986 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:58.986 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:58.986 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.986 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.986 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:10445 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:58:58.986 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.271980+0000", 2024-09-21T14:58:58.986 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 19.626483437000001, 2024-09-21T14:58:58.986 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018664256000000001, 2024-09-21T14:58:58.986 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:58.986 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:58.986 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:58.986 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:58.987 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:58.987 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 10445 2024-09-21T14:58:58.987 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.987 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:58.987 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.987 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:58.987 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.271980+0000", 2024-09-21T14:58:58.987 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:58.987 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.987 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.987 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:58.987 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.271980+0000", 2024-09-21T14:58:58.987 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:58.987 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.988 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.988 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:58.988 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.271979+0000", 2024-09-21T14:58:58.988 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:58.988 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.988 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.988 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:58.988 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.271982+0000", 2024-09-21T14:58:58.988 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7240000000000001e-06 2024-09-21T14:58:58.988 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.988 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.988 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:58.988 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.271987+0000", 2024-09-21T14:58:58.988 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3639999999999998e-06 2024-09-21T14:58:58.989 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.989 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.989 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:58.989 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.271992+0000", 2024-09-21T14:58:58.989 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7110000000000003e-06 2024-09-21T14:58:58.989 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.989 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.989 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:58.989 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.277567+0000", 2024-09-21T14:58:58.989 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0055745009999999999 2024-09-21T14:58:58.989 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.989 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.989 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:58.989 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.277655+0000", 2024-09-21T14:58:58.989 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8465000000000001e-05 2024-09-21T14:58:58.989 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.990 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.990 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:58:58.990 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.278720+0000", 2024-09-21T14:58:58.990 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0010650569999999999 2024-09-21T14:58:58.990 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.990 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.990 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:58.990 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.281153+0000", 2024-09-21T14:58:58.990 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0024326389999999999 2024-09-21T14:58:58.990 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.990 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.990 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:58.990 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.290574+0000", 2024-09-21T14:58:58.990 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0094209489999999996 2024-09-21T14:58:58.991 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.991 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.991 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:58.991 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.290613+0000", 2024-09-21T14:58:58.991 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.8803000000000001e-05 2024-09-21T14:58:58.991 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.991 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.991 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:58.991 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.290644+0000", 2024-09-21T14:58:58.991 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1404e-05 2024-09-21T14:58:58.991 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:58.991 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:58.991 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:58.991 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.991 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.992 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22769 3.f 3:f39bd56f:::rbd_data.11366acc894.0000000000000004:19 [read 1032192~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:58:58.992 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.140207+0000", 2024-09-21T14:58:58.992 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 8.7582556969999992, 2024-09-21T14:58:58.992 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018031170999999999, 2024-09-21T14:58:58.992 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:58.992 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:58.992 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:58.992 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:58.992 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:58.992 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22769 2024-09-21T14:58:58.992 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.992 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:58.992 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.992 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:58.992 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.140207+0000", 2024-09-21T14:58:58.993 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:58.993 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.993 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.993 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:58.993 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.140207+0000", 2024-09-21T14:58:58.993 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:58.993 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.993 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.993 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:58.993 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.140207+0000", 2024-09-21T14:58:58.993 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:58.993 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.993 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.993 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:58.994 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.140209+0000", 2024-09-21T14:58:58.994 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.305e-06 2024-09-21T14:58:58.994 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.994 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.994 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:58.994 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.140214+0000", 2024-09-21T14:58:58.994 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9570000000000001e-06 2024-09-21T14:58:58.994 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.994 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.994 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:58.994 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.140219+0000", 2024-09-21T14:58:58.994 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2250000000000002e-06 2024-09-21T14:58:58.994 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.994 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.995 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:58.995 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.157990+0000", 2024-09-21T14:58:58.995 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.017770939999999999 2024-09-21T14:58:58.995 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.995 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.995 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:58.995 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.158062+0000", 2024-09-21T14:58:58.995 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2652999999999997e-05 2024-09-21T14:58:58.995 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.995 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.995 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:58.995 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.158239+0000", 2024-09-21T14:58:58.995 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017640599999999999 2024-09-21T14:58:58.996 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:58.996 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:58.996 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:58.996 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.996 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.996 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:23084 3.3 3:c55e343c:::rbd_data.11366acc894.0000000000000006:19 [read 1032192~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:58:58.996 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.150724+0000", 2024-09-21T14:58:58.996 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 8.7477388620000003, 2024-09-21T14:58:58.996 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018916460999999999, 2024-09-21T14:58:58.996 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:58.996 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:58.996 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:58.996 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:58.996 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:58.996 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 23084 2024-09-21T14:58:58.997 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.997 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:58.997 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.997 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:58.997 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.150724+0000", 2024-09-21T14:58:58.997 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:58.997 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.997 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.997 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:58.997 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.150724+0000", 2024-09-21T14:58:58.997 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:58.997 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.997 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.997 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:58.997 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.150724+0000", 2024-09-21T14:58:58.997 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:58.997 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.998 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.998 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:58.998 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.150727+0000", 2024-09-21T14:58:58.998 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7250000000000002e-06 2024-09-21T14:58:58.998 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.998 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.998 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:58.998 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.150732+0000", 2024-09-21T14:58:58.998 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5809999999999996e-06 2024-09-21T14:58:58.998 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.998 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.998 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:58.998 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.150737+0000", 2024-09-21T14:58:58.998 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3590000000000001e-06 2024-09-21T14:58:58.998 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.998 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.999 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:58.999 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.166694+0000", 2024-09-21T14:58:58.999 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.015957720000000002 2024-09-21T14:58:58.999 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.999 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.999 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:58.999 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.166769+0000", 2024-09-21T14:58:58.999 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5143999999999997e-05 2024-09-21T14:58:58.999 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:58.999 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:58.999 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:58.999 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.169641+0000", 2024-09-21T14:58:58.999 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0028713089999999998 2024-09-21T14:58:58.999 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:59.000 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:59.000 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:59.000 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:59.000 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:59.000 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:58:59.001 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T14:58:59.002 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:58:59.000+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:58:59.102 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:58:59.100+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:58:59.119 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:58:59.119 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:58:59.119 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T14:58:59.120 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:58:59.129 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T14:58:59.203 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:58:59.200+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:58:59.238 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:58:59.238 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:58:59.238 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T14:58:59.238 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T14:58:59.238 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:58:59.248 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T14:58:59.269 INFO:teuthology.orchestra.run.smithi032.stderr:noscrub is unset 2024-09-21T14:58:59.304 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:58:59.300+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:58:59.357 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:58:59.357 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T14:58:59.357 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T14:58:59.357 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T14:58:59.357 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.357 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T14:58:59.357 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.205754+0000", 2024-09-21T14:58:59.358 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 56.148906154000002, 2024-09-21T14:58:59.358 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.41117094199999998, 2024-09-21T14:58:59.358 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:59.358 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:59.358 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:59.358 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:59.358 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:59.358 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1 2024-09-21T14:58:59.358 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.358 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:59.358 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.358 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:59.358 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205754+0000", 2024-09-21T14:58:59.358 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:59.359 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.359 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.359 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:59.359 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205754+0000", 2024-09-21T14:58:59.359 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:59.359 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.359 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.359 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:59.359 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205752+0000", 2024-09-21T14:58:59.359 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T14:58:59.359 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.359 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.359 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:59.359 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205765+0000", 2024-09-21T14:58:59.359 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3230999999999999e-05 2024-09-21T14:58:59.359 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.360 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.360 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:59.360 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205782+0000", 2024-09-21T14:58:59.360 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6634e-05 2024-09-21T14:58:59.360 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.360 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.360 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:59.360 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205793+0000", 2024-09-21T14:58:59.360 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1297e-05 2024-09-21T14:58:59.360 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.360 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.360 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:59.360 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205867+0000", 2024-09-21T14:58:59.360 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4269000000000003e-05 2024-09-21T14:58:59.360 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.360 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.361 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T14:58:59.361 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205877+0000", 2024-09-21T14:58:59.361 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2690000000000005e-06 2024-09-21T14:58:59.361 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.361 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.361 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:59.361 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.613700+0000", 2024-09-21T14:58:59.361 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.407823455 2024-09-21T14:58:59.361 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.361 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.361 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:59.361 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.614203+0000", 2024-09-21T14:58:59.361 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00050311500000000005 2024-09-21T14:58:59.361 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.361 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.362 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:59.362 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.614750+0000", 2024-09-21T14:58:59.362 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00054738699999999998 2024-09-21T14:58:59.362 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.362 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.362 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:59.362 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616367+0000", 2024-09-21T14:58:59.362 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001616634 2024-09-21T14:58:59.362 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.362 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.362 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:59.362 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616574+0000", 2024-09-21T14:58:59.362 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00020668499999999999 2024-09-21T14:58:59.362 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.363 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.363 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:59.363 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616802+0000", 2024-09-21T14:58:59.363 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00022817900000000001 2024-09-21T14:58:59.363 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.363 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.363 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:59.363 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616925+0000", 2024-09-21T14:58:59.363 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012309699999999999 2024-09-21T14:58:59.363 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:59.363 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:59.363 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:59.363 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.363 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.363 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T14:58:59.363 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.955142+0000", 2024-09-21T14:58:59.364 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 55.399518507000003, 2024-09-21T14:58:59.364 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012388923, 2024-09-21T14:58:59.364 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:59.364 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:59.364 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:59.364 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:59.364 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:59.364 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 113 2024-09-21T14:58:59.364 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.364 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:59.364 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.364 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:59.364 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955142+0000", 2024-09-21T14:58:59.364 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:59.365 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.365 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.365 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:59.365 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955142+0000", 2024-09-21T14:58:59.365 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:59.365 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.365 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.365 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:59.365 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955141+0000", 2024-09-21T14:58:59.365 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:59.365 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.365 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.365 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:59.365 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955145+0000", 2024-09-21T14:58:59.365 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2109999999999998e-06 2024-09-21T14:58:59.366 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.366 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.366 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:59.366 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955150+0000", 2024-09-21T14:58:59.366 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4090000000000002e-06 2024-09-21T14:58:59.366 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.366 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.366 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:59.366 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955153+0000", 2024-09-21T14:58:59.366 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1240000000000001e-06 2024-09-21T14:58:59.366 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.366 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.366 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:59.366 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.963026+0000", 2024-09-21T14:58:59.366 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0078724060000000002 2024-09-21T14:58:59.367 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.367 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.367 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:59.367 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.963126+0000", 2024-09-21T14:58:59.367 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000100206 2024-09-21T14:58:59.367 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.367 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.367 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:59.367 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.963277+0000", 2024-09-21T14:58:59.367 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015159200000000001 2024-09-21T14:58:59.367 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.367 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.367 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:59.367 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.965519+0000", 2024-09-21T14:58:59.367 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0022418440000000002 2024-09-21T14:58:59.367 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.368 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.368 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:59.368 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967305+0000", 2024-09-21T14:58:59.368 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001785525 2024-09-21T14:58:59.368 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.368 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.368 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:59.368 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967438+0000", 2024-09-21T14:58:59.368 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013284399999999999 2024-09-21T14:58:59.368 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.368 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.368 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:59.368 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967531+0000", 2024-09-21T14:58:59.368 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.3140000000000006e-05 2024-09-21T14:58:59.368 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:59.369 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:59.369 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:59.369 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.369 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.369 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T14:58:59.369 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.955387+0000", 2024-09-21T14:58:59.369 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 55.399273573000002, 2024-09-21T14:58:59.369 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012319478999999999, 2024-09-21T14:58:59.369 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:59.369 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:59.369 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:59.369 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:59.369 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:59.369 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 114 2024-09-21T14:58:59.369 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.370 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:59.370 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.370 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:59.370 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955387+0000", 2024-09-21T14:58:59.370 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:59.370 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.370 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.370 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:59.370 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955387+0000", 2024-09-21T14:58:59.370 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:59.370 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.370 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.370 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:59.370 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955386+0000", 2024-09-21T14:58:59.371 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:58:59.371 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.371 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.371 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:59.371 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955392+0000", 2024-09-21T14:58:59.371 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4740000000000002e-06 2024-09-21T14:58:59.371 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.371 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.371 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:59.371 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955400+0000", 2024-09-21T14:58:59.371 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2139999999999996e-06 2024-09-21T14:58:59.371 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.371 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.372 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:59.372 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955407+0000", 2024-09-21T14:58:59.372 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2299999999999996e-06 2024-09-21T14:58:59.372 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.372 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.372 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:59.372 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.964840+0000", 2024-09-21T14:58:59.372 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0094330079999999997 2024-09-21T14:58:59.372 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.372 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.372 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:59.372 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.964917+0000", 2024-09-21T14:58:59.372 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7071999999999997e-05 2024-09-21T14:58:59.372 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.372 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.373 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:59.373 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.965026+0000", 2024-09-21T14:58:59.373 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000109646 2024-09-21T14:58:59.373 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.373 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.373 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:59.373 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967609+0000", 2024-09-21T14:58:59.373 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0025824929999999999 2024-09-21T14:58:59.373 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.373 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.373 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:59.373 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967629+0000", 2024-09-21T14:58:59.373 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0486999999999999e-05 2024-09-21T14:58:59.373 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.373 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.374 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:59.374 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967672+0000", 2024-09-21T14:58:59.374 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2314999999999997e-05 2024-09-21T14:58:59.374 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.374 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.374 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:59.374 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967706+0000", 2024-09-21T14:58:59.374 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4483999999999997e-05 2024-09-21T14:58:59.374 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:59.374 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:59.374 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:59.374 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.374 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.374 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T14:58:59.375 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.955609+0000", 2024-09-21T14:58:59.375 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 55.399051767000003, 2024-09-21T14:58:59.375 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013624153999999999, 2024-09-21T14:58:59.375 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:59.375 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:59.375 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:59.375 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:59.375 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:59.375 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 115 2024-09-21T14:58:59.375 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.375 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:59.375 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.375 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:59.375 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955609+0000", 2024-09-21T14:58:59.376 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:59.376 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.376 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.376 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:59.376 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955609+0000", 2024-09-21T14:58:59.376 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:59.376 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.376 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.376 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:59.376 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955607+0000", 2024-09-21T14:58:59.376 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-21T14:58:59.376 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.376 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.376 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:59.376 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955732+0000", 2024-09-21T14:58:59.377 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000125566 2024-09-21T14:58:59.377 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.377 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.377 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:59.377 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955752+0000", 2024-09-21T14:58:59.377 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9916999999999999e-05 2024-09-21T14:58:59.377 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.377 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.377 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:59.377 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955767+0000", 2024-09-21T14:58:59.377 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4687e-05 2024-09-21T14:58:59.377 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.377 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.377 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:59.378 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.965574+0000", 2024-09-21T14:58:59.378 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0098072850000000007 2024-09-21T14:58:59.378 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.378 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.378 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:59.378 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.965642+0000", 2024-09-21T14:58:59.378 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7784e-05 2024-09-21T14:58:59.378 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.378 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.378 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:59.378 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.965753+0000", 2024-09-21T14:58:59.378 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000111111 2024-09-21T14:58:59.378 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.378 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.379 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:59.379 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967717+0000", 2024-09-21T14:58:59.379 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0019635339999999999 2024-09-21T14:58:59.379 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.379 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.379 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:59.379 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.969014+0000", 2024-09-21T14:58:59.379 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001297093 2024-09-21T14:58:59.379 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.379 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.379 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:59.379 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.969130+0000", 2024-09-21T14:58:59.379 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000116011 2024-09-21T14:58:59.379 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.379 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.380 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:59.380 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.969233+0000", 2024-09-21T14:58:59.380 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000102982 2024-09-21T14:58:59.380 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:59.380 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:59.380 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:59.380 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.380 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.380 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T14:58:59.380 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.955783+0000", 2024-09-21T14:58:59.380 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 55.398877098, 2024-09-21T14:58:59.380 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014630241, 2024-09-21T14:58:59.380 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:59.380 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:59.380 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:59.381 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:59.381 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:59.381 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 116 2024-09-21T14:58:59.381 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.381 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:59.381 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.381 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:59.381 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955783+0000", 2024-09-21T14:58:59.381 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:59.381 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.381 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.381 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:59.381 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955783+0000", 2024-09-21T14:58:59.381 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:59.381 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.382 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.382 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:59.382 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955782+0000", 2024-09-21T14:58:59.382 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:58:59.382 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.382 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.382 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:59.382 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955791+0000", 2024-09-21T14:58:59.382 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8089999999999999e-06 2024-09-21T14:58:59.382 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.382 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.382 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:59.382 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955804+0000", 2024-09-21T14:58:59.382 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2548000000000001e-05 2024-09-21T14:58:59.383 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.383 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.383 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:59.383 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955814+0000", 2024-09-21T14:58:59.383 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0158e-05 2024-09-21T14:58:59.383 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.383 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.383 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:59.383 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.966515+0000", 2024-09-21T14:58:59.383 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.010701756 2024-09-21T14:58:59.383 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.383 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.383 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:59.384 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.966606+0000", 2024-09-21T14:58:59.384 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0761999999999996e-05 2024-09-21T14:58:59.384 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.384 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.384 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:59.384 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.966740+0000", 2024-09-21T14:58:59.384 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000133693 2024-09-21T14:58:59.384 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.384 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.384 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:59.384 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.968920+0000", 2024-09-21T14:58:59.384 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0021799290000000002 2024-09-21T14:58:59.384 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.384 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.385 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:59.385 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.970320+0000", 2024-09-21T14:58:59.385 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0014001759999999999 2024-09-21T14:58:59.385 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.385 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.385 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:59.385 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.970376+0000", 2024-09-21T14:58:59.385 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5696999999999999e-05 2024-09-21T14:58:59.385 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.385 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.385 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:59.385 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.970413+0000", 2024-09-21T14:58:59.385 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7750999999999998e-05 2024-09-21T14:58:59.385 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:59.386 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:59.386 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:59.386 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.386 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.386 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T14:58:59.386 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.955829+0000", 2024-09-21T14:58:59.386 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 55.398831628000003, 2024-09-21T14:58:59.386 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014919597999999999, 2024-09-21T14:58:59.386 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:59.386 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:59.386 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:59.386 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:59.386 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:59.386 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 117 2024-09-21T14:58:59.387 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.387 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:59.387 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.387 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:59.387 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955829+0000", 2024-09-21T14:58:59.387 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:59.387 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.387 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.387 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:59.387 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955829+0000", 2024-09-21T14:58:59.387 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:59.387 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.387 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.387 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:59.387 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955827+0000", 2024-09-21T14:58:59.388 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:59.388 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.388 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.388 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:59.388 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955838+0000", 2024-09-21T14:58:59.388 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.111e-05 2024-09-21T14:58:59.388 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.388 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.388 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:59.388 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955875+0000", 2024-09-21T14:58:59.388 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6211999999999998e-05 2024-09-21T14:58:59.388 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.388 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.388 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:59.389 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955899+0000", 2024-09-21T14:58:59.389 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4128999999999999e-05 2024-09-21T14:58:59.389 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.389 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.389 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:59.389 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967767+0000", 2024-09-21T14:58:59.389 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.011868251999999999 2024-09-21T14:58:59.389 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.389 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.389 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:59.389 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967866+0000", 2024-09-21T14:58:59.389 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8664000000000003e-05 2024-09-21T14:58:59.389 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.389 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.389 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:59.390 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.968128+0000", 2024-09-21T14:58:59.390 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00026243600000000001 2024-09-21T14:58:59.390 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.390 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.390 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:59.390 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.969393+0000", 2024-09-21T14:58:59.390 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0012650350000000001 2024-09-21T14:58:59.390 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.390 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.390 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:59.390 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.970523+0000", 2024-09-21T14:58:59.390 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001129557 2024-09-21T14:58:59.390 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.390 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.391 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:59.391 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.970603+0000", 2024-09-21T14:58:59.391 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0090000000000001e-05 2024-09-21T14:58:59.391 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.391 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.391 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:59.391 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.970748+0000", 2024-09-21T14:58:59.391 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014552799999999999 2024-09-21T14:58:59.391 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:59.391 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:59.391 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:59.391 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.391 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.391 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T14:58:59.392 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:04.493315+0000", 2024-09-21T14:58:59.392 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 54.861344795000001, 2024-09-21T14:58:59.392 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014055766000000001, 2024-09-21T14:58:59.392 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:59.392 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:59.392 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:59.392 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:59.392 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:59.392 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 137 2024-09-21T14:58:59.392 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.392 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:59.392 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.392 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:59.392 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493315+0000", 2024-09-21T14:58:59.393 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:59.393 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.393 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.393 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:59.393 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493315+0000", 2024-09-21T14:58:59.393 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:59.393 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.393 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.393 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:59.393 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493314+0000", 2024-09-21T14:58:59.393 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:58:59.393 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.393 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.393 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:59.393 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493507+0000", 2024-09-21T14:58:59.394 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000192444 2024-09-21T14:58:59.394 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.394 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.394 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:59.394 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493528+0000", 2024-09-21T14:58:59.394 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0845000000000001e-05 2024-09-21T14:58:59.394 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.394 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.394 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:59.394 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493543+0000", 2024-09-21T14:58:59.394 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5800000000000001e-05 2024-09-21T14:58:59.394 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.394 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.394 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:59.395 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.502575+0000", 2024-09-21T14:58:59.395 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0090312389999999999 2024-09-21T14:58:59.395 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.395 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.395 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:59.395 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.502677+0000", 2024-09-21T14:58:59.395 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000102271 2024-09-21T14:58:59.395 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.395 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.395 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:59.395 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.502825+0000", 2024-09-21T14:58:59.395 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000148171 2024-09-21T14:58:59.395 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.395 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.395 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:59.395 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507251+0000", 2024-09-21T14:58:59.396 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0044253879999999997 2024-09-21T14:58:59.396 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.396 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.396 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:59.396 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507313+0000", 2024-09-21T14:58:59.396 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2170999999999998e-05 2024-09-21T14:58:59.396 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.396 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.396 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:59.396 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507346+0000", 2024-09-21T14:58:59.396 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3079e-05 2024-09-21T14:58:59.396 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.396 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.396 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:59.397 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507371+0000", 2024-09-21T14:58:59.397 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5477999999999999e-05 2024-09-21T14:58:59.397 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:59.397 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:59.397 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:59.397 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.397 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.397 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T14:58:59.397 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:04.493570+0000", 2024-09-21T14:58:59.397 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 54.861090390000001, 2024-09-21T14:58:59.397 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014061686, 2024-09-21T14:58:59.397 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:59.397 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:59.397 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:59.398 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:59.398 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:59.398 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 138 2024-09-21T14:58:59.398 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.398 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:59.398 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.398 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:59.398 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493570+0000", 2024-09-21T14:58:59.398 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:59.398 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.398 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.398 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:59.398 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493570+0000", 2024-09-21T14:58:59.398 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:59.398 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.399 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.399 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:59.399 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493568+0000", 2024-09-21T14:58:59.399 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:59.399 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.399 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.399 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:59.399 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493579+0000", 2024-09-21T14:58:59.399 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0574000000000001e-05 2024-09-21T14:58:59.399 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.399 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.399 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:59.399 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493595+0000", 2024-09-21T14:58:59.399 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6504999999999999e-05 2024-09-21T14:58:59.399 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.400 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.400 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:59.400 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493608+0000", 2024-09-21T14:58:59.400 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2836999999999999e-05 2024-09-21T14:58:59.400 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.400 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.400 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:59.400 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.503909+0000", 2024-09-21T14:58:59.400 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.01030118 2024-09-21T14:58:59.400 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.400 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.400 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:59.400 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.504064+0000", 2024-09-21T14:58:59.400 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015493000000000001 2024-09-21T14:58:59.400 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.401 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.401 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:59.401 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.504299+0000", 2024-09-21T14:58:59.401 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00023416100000000001 2024-09-21T14:58:59.401 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.401 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.401 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:59.401 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507466+0000", 2024-09-21T14:58:59.401 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0031678380000000001 2024-09-21T14:58:59.401 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.401 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.401 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:59.401 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507483+0000", 2024-09-21T14:58:59.401 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6606999999999999e-05 2024-09-21T14:58:59.401 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.401 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.402 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:59.402 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507566+0000", 2024-09-21T14:58:59.402 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3110999999999998e-05 2024-09-21T14:58:59.402 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.402 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.402 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:59.402 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507632+0000", 2024-09-21T14:58:59.402 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5480000000000003e-05 2024-09-21T14:58:59.402 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:59.402 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:59.402 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:59.402 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.402 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.403 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T14:58:59.403 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:04.493625+0000", 2024-09-21T14:58:59.403 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 54.861035702000002, 2024-09-21T14:58:59.403 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014665378999999999, 2024-09-21T14:58:59.403 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:59.403 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:59.403 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:59.403 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:59.403 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:59.403 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139 2024-09-21T14:58:59.403 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.403 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:59.403 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.403 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:59.403 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493625+0000", 2024-09-21T14:58:59.404 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:59.404 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.404 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.404 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:59.404 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493625+0000", 2024-09-21T14:58:59.404 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:59.404 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.404 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.404 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:59.404 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493624+0000", 2024-09-21T14:58:59.404 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:58:59.404 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.404 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.404 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:59.405 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493638+0000", 2024-09-21T14:58:59.405 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4282e-05 2024-09-21T14:58:59.405 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.405 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.405 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:59.405 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493654+0000", 2024-09-21T14:58:59.405 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5739999999999998e-05 2024-09-21T14:58:59.405 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.405 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.405 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:59.405 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493666+0000", 2024-09-21T14:58:59.405 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2865e-05 2024-09-21T14:58:59.405 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.405 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.406 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:59.406 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.505329+0000", 2024-09-21T14:58:59.406 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.011662237000000001 2024-09-21T14:58:59.406 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.406 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.406 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:59.406 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.505377+0000", 2024-09-21T14:58:59.406 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7862999999999999e-05 2024-09-21T14:58:59.406 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.406 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.406 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:59.406 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.505455+0000", 2024-09-21T14:58:59.406 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8918000000000006e-05 2024-09-21T14:58:59.406 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.407 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.407 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:59.407 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507650+0000", 2024-09-21T14:58:59.407 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.002195027 2024-09-21T14:58:59.407 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.407 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.407 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:59.407 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508149+0000", 2024-09-21T14:58:59.407 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00049828399999999999 2024-09-21T14:58:59.407 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.407 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.407 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:59.407 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508230+0000", 2024-09-21T14:58:59.408 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1376000000000002e-05 2024-09-21T14:58:59.408 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.408 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.408 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:59.408 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508290+0000", 2024-09-21T14:58:59.408 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9852999999999998e-05 2024-09-21T14:58:59.408 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:59.408 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:59.408 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:59.408 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.408 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.408 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T14:58:59.408 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:04.493679+0000", 2024-09-21T14:58:59.408 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 54.860980804, 2024-09-21T14:58:59.409 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014869869000000001, 2024-09-21T14:58:59.409 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:59.409 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:59.409 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:59.409 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:59.409 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:59.409 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140 2024-09-21T14:58:59.409 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.409 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:59.409 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.409 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:59.409 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493679+0000", 2024-09-21T14:58:59.409 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:59.409 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.410 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.410 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:59.410 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493679+0000", 2024-09-21T14:58:59.410 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:59.410 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.410 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.410 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:59.410 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493679+0000", 2024-09-21T14:58:59.410 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:58:59.410 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.410 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.410 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:59.410 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493889+0000", 2024-09-21T14:58:59.410 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00021032700000000001 2024-09-21T14:58:59.410 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.411 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.411 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:59.411 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493931+0000", 2024-09-21T14:58:59.411 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1971000000000002e-05 2024-09-21T14:58:59.411 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.411 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.411 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:59.411 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493947+0000", 2024-09-21T14:58:59.411 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6385e-05 2024-09-21T14:58:59.411 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.411 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.411 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:59.411 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.505804+0000", 2024-09-21T14:58:59.411 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.011857136000000001 2024-09-21T14:58:59.411 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.412 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.412 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:59.412 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.505992+0000", 2024-09-21T14:58:59.412 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000187665 2024-09-21T14:58:59.412 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.412 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.412 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:59.412 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.506281+0000", 2024-09-21T14:58:59.412 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000288712 2024-09-21T14:58:59.412 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.412 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.412 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:59.412 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508349+0000", 2024-09-21T14:58:59.412 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0020685790000000001 2024-09-21T14:58:59.412 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.412 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.413 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:59.413 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508371+0000", 2024-09-21T14:58:59.413 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1747e-05 2024-09-21T14:58:59.413 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.413 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.413 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:59.413 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508450+0000", 2024-09-21T14:58:59.413 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9332000000000005e-05 2024-09-21T14:58:59.413 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.413 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.413 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:59.413 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508549+0000", 2024-09-21T14:58:59.413 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8918000000000004e-05 2024-09-21T14:58:59.413 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:59.413 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:59.413 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:59.414 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.414 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.414 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T14:58:59.414 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:04.493967+0000", 2024-09-21T14:58:59.414 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 54.860693767000001, 2024-09-21T14:58:59.414 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.015281075999999999, 2024-09-21T14:58:59.414 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:59.414 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:59.414 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:59.414 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:58:59.414 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:58:59.414 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 141 2024-09-21T14:58:59.414 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.414 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:59.415 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.415 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:59.415 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493967+0000", 2024-09-21T14:58:59.415 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:59.415 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.415 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.415 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:59.415 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493967+0000", 2024-09-21T14:58:59.415 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:59.415 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.415 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.415 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:59.415 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493965+0000", 2024-09-21T14:58:59.415 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T14:58:59.415 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.415 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.416 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:59.416 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493987+0000", 2024-09-21T14:58:59.416 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1662999999999999e-05 2024-09-21T14:58:59.416 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.416 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.416 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:59.416 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.494004+0000", 2024-09-21T14:58:59.416 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7311000000000001e-05 2024-09-21T14:58:59.416 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.416 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.416 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:59.416 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.494017+0000", 2024-09-21T14:58:59.416 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3202e-05 2024-09-21T14:58:59.416 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.417 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.417 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:59.417 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507678+0000", 2024-09-21T14:58:59.417 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013660432 2024-09-21T14:58:59.417 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.417 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.417 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:59.417 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507726+0000", 2024-09-21T14:58:59.417 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8154e-05 2024-09-21T14:58:59.417 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.417 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.417 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:58:59.418 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507797+0000", 2024-09-21T14:58:59.418 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.1110000000000002e-05 2024-09-21T14:58:59.418 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.418 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.418 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:59.418 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508710+0000", 2024-09-21T14:58:59.418 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00091339299999999997 2024-09-21T14:58:59.418 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.418 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.418 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:59.418 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.509108+0000", 2024-09-21T14:58:59.418 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00039718300000000001 2024-09-21T14:58:59.418 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.419 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.419 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:59.419 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.509186+0000", 2024-09-21T14:58:59.419 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8592999999999998e-05 2024-09-21T14:58:59.419 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.419 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.419 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:59.419 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.509248+0000", 2024-09-21T14:58:59.419 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1450000000000003e-05 2024-09-21T14:58:59.419 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:59.419 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:59.419 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:59.419 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.419 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.419 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6626 3.1a 3:5adee3e5:::rbd_data.11361bed1f78.0000000000000001:d [read 229376~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:59.420 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131593+0000", 2024-09-21T14:58:59.420 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 24.223067188999998, 2024-09-21T14:58:59.420 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012576725, 2024-09-21T14:58:59.420 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:59.420 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:59.420 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:59.420 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:59.420 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:59.420 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6626 2024-09-21T14:58:59.420 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.420 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:59.420 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.420 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:59.420 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131593+0000", 2024-09-21T14:58:59.420 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:59.421 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.421 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.421 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:59.421 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131593+0000", 2024-09-21T14:58:59.421 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:59.421 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.421 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.421 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:59.421 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131593+0000", 2024-09-21T14:58:59.421 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T14:58:59.421 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.421 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.421 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:59.421 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131598+0000", 2024-09-21T14:58:59.421 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7849999999999999e-06 2024-09-21T14:58:59.422 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.422 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.422 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:59.422 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131601+0000", 2024-09-21T14:58:59.422 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1949999999999999e-06 2024-09-21T14:58:59.422 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.422 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.422 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:59.422 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131604+0000", 2024-09-21T14:58:59.422 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6350000000000002e-06 2024-09-21T14:58:59.422 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.422 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.422 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:59.423 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.143902+0000", 2024-09-21T14:58:59.423 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012298216000000001 2024-09-21T14:58:59.423 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.423 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.423 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:59.423 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.143993+0000", 2024-09-21T14:58:59.423 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1073e-05 2024-09-21T14:58:59.423 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.423 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.423 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:59.423 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.144170+0000", 2024-09-21T14:58:59.423 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017703 2024-09-21T14:58:59.423 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:59.423 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:59.424 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:59.424 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.424 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.424 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6631 3.1a 3:5d547f54:::rbd_data.11361bed1f78.0000000000000003:d [read 0~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:59.424 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131606+0000", 2024-09-21T14:58:59.424 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 24.223054488999999, 2024-09-21T14:58:59.424 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013903148000000001, 2024-09-21T14:58:59.424 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:59.424 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:59.424 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:59.424 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:59.424 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:59.424 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6631 2024-09-21T14:58:59.425 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.425 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:59.425 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.425 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:59.425 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131606+0000", 2024-09-21T14:58:59.425 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:59.425 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.425 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.425 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:59.425 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131606+0000", 2024-09-21T14:58:59.425 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:59.425 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.425 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.425 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:59.426 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131606+0000", 2024-09-21T14:58:59.426 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T14:58:59.426 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.426 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.426 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:59.426 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131607+0000", 2024-09-21T14:58:59.426 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2699999999999999e-06 2024-09-21T14:58:59.426 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.426 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.426 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:59.426 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131610+0000", 2024-09-21T14:58:59.426 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.88e-06 2024-09-21T14:58:59.426 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.426 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.427 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:59.427 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131612+0000", 2024-09-21T14:58:59.427 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5170000000000002e-06 2024-09-21T14:58:59.427 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.427 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.427 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:59.427 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.144205+0000", 2024-09-21T14:58:59.427 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012592763 2024-09-21T14:58:59.427 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.427 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.427 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:59.427 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.144332+0000", 2024-09-21T14:58:59.427 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012694200000000001 2024-09-21T14:58:59.427 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.427 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.428 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:59.428 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.145509+0000", 2024-09-21T14:58:59.428 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001176992 2024-09-21T14:58:59.428 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:59.428 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:59.428 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:59.428 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.428 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.428 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6632 3.1a 3:5d547f54:::rbd_data.11361bed1f78.0000000000000003:d [read 81920~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:59.428 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131614+0000", 2024-09-21T14:58:59.428 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 24.223045881000001, 2024-09-21T14:58:59.428 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014093553999999999, 2024-09-21T14:58:59.428 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:59.428 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:59.428 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:59.429 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:59.429 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:59.429 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6632 2024-09-21T14:58:59.429 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.429 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:59.429 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.429 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:59.429 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131614+0000", 2024-09-21T14:58:59.429 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:59.429 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.429 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.429 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:59.429 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131614+0000", 2024-09-21T14:58:59.429 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:59.429 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.429 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.430 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:59.430 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131614+0000", 2024-09-21T14:58:59.430 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T14:58:59.430 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.430 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.430 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:59.430 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131615+0000", 2024-09-21T14:58:59.430 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.218e-06 2024-09-21T14:58:59.430 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.430 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.430 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:59.430 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131618+0000", 2024-09-21T14:58:59.430 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9050000000000001e-06 2024-09-21T14:58:59.430 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.431 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.431 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:59.431 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131621+0000", 2024-09-21T14:58:59.431 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7439999999999999e-06 2024-09-21T14:58:59.431 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.431 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.431 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:59.431 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.145538+0000", 2024-09-21T14:58:59.431 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013916618 2024-09-21T14:58:59.431 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.431 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.431 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:59.431 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.145619+0000", 2024-09-21T14:58:59.431 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1812999999999997e-05 2024-09-21T14:58:59.431 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.431 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.432 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:59.432 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.145708+0000", 2024-09-21T14:58:59.432 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8460000000000003e-05 2024-09-21T14:58:59.432 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:59.432 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:59.432 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:59.432 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.432 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.432 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6633 3.1a 3:5d547f54:::rbd_data.11361bed1f78.0000000000000003:d [read 147456~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:59.432 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131623+0000", 2024-09-21T14:58:59.432 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 24.223036915000002, 2024-09-21T14:58:59.432 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014395845000000001, 2024-09-21T14:58:59.432 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:59.432 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:59.432 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:59.433 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:59.433 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:59.433 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6633 2024-09-21T14:58:59.433 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.433 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:59.433 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.433 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:59.433 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131623+0000", 2024-09-21T14:58:59.433 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:59.433 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.433 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.433 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:59.433 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131623+0000", 2024-09-21T14:58:59.433 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:59.434 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.434 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.434 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:59.434 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131623+0000", 2024-09-21T14:58:59.434 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T14:58:59.434 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.434 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.434 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:59.434 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131628+0000", 2024-09-21T14:58:59.434 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0370000000000002e-06 2024-09-21T14:58:59.434 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.434 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.434 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:59.434 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131632+0000", 2024-09-21T14:58:59.435 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3519999999999998e-06 2024-09-21T14:58:59.435 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.435 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.435 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:59.435 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131634+0000", 2024-09-21T14:58:59.435 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5019999999999999e-06 2024-09-21T14:58:59.435 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.435 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.435 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:59.435 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.145734+0000", 2024-09-21T14:58:59.435 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014100118 2024-09-21T14:58:59.435 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.435 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.435 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:59.435 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.145804+0000", 2024-09-21T14:58:59.435 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0229999999999997e-05 2024-09-21T14:58:59.436 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.436 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.436 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:59.436 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146019+0000", 2024-09-21T14:58:59.436 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00021481699999999999 2024-09-21T14:58:59.436 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:59.436 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:59.436 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:59.436 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.436 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.436 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6634 3.1a 3:5d547f54:::rbd_data.11361bed1f78.0000000000000003:d [read 229376~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:59.436 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131636+0000", 2024-09-21T14:58:59.436 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 24.223023805, 2024-09-21T14:58:59.436 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014645843, 2024-09-21T14:58:59.437 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:59.437 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:59.437 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:59.437 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:59.437 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:59.437 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6634 2024-09-21T14:58:59.437 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.437 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:59.437 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.437 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:59.437 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131636+0000", 2024-09-21T14:58:59.437 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:59.437 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.438 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.438 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:59.438 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131636+0000", 2024-09-21T14:58:59.438 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:59.438 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.438 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.438 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:59.438 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131636+0000", 2024-09-21T14:58:59.438 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:59.438 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.438 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.438 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:59.438 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131641+0000", 2024-09-21T14:58:59.438 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8489999999999997e-06 2024-09-21T14:58:59.439 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.439 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.439 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:59.439 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131644+0000", 2024-09-21T14:58:59.439 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.179e-06 2024-09-21T14:58:59.439 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.439 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.439 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:59.440 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131647+0000", 2024-09-21T14:58:59.440 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8009999999999999e-06 2024-09-21T14:58:59.440 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.440 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.440 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:59.440 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146046+0000", 2024-09-21T14:58:59.440 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014399018 2024-09-21T14:58:59.440 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.440 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.440 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:59.440 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146118+0000", 2024-09-21T14:58:59.440 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2044000000000003e-05 2024-09-21T14:58:59.440 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.440 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.441 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:59.441 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146282+0000", 2024-09-21T14:58:59.441 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000164196 2024-09-21T14:58:59.441 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:59.441 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:59.441 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:59.441 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.441 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.441 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6639 3.1a 3:592d80ed:::rbd_data.11361bed1f78.0000000000000005:d [read 0~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:59.441 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131650+0000", 2024-09-21T14:58:59.441 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 24.223010605999999, 2024-09-21T14:58:59.441 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016002832000000002, 2024-09-21T14:58:59.441 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:59.441 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:59.441 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:59.442 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:59.442 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:59.442 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6639 2024-09-21T14:58:59.442 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.442 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:59.442 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.442 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:59.442 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131650+0000", 2024-09-21T14:58:59.442 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:59.442 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.442 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.442 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:59.442 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131650+0000", 2024-09-21T14:58:59.443 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:59.443 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.443 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.443 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:59.443 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131649+0000", 2024-09-21T14:58:59.443 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:59.443 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.443 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.443 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:59.443 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131651+0000", 2024-09-21T14:58:59.443 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9199999999999998e-06 2024-09-21T14:58:59.443 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.443 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.443 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:59.444 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131656+0000", 2024-09-21T14:58:59.444 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9899999999999997e-06 2024-09-21T14:58:59.444 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.444 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.444 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:59.444 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131665+0000", 2024-09-21T14:58:59.444 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0999999999999993e-06 2024-09-21T14:58:59.444 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.444 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.444 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:59.444 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146309+0000", 2024-09-21T14:58:59.444 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014643893999999999 2024-09-21T14:58:59.444 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.445 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.445 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:59.445 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146392+0000", 2024-09-21T14:58:59.445 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2531e-05 2024-09-21T14:58:59.445 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.445 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.445 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:59.445 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147653+0000", 2024-09-21T14:58:59.445 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0012606710000000001 2024-09-21T14:58:59.445 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:59.445 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:59.445 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:59.445 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.445 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.446 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6640 3.1a 3:592d80ed:::rbd_data.11361bed1f78.0000000000000005:d [read 65536~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:59.446 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131670+0000", 2024-09-21T14:58:59.446 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 24.222990307, 2024-09-21T14:58:59.446 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016188859, 2024-09-21T14:58:59.446 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:59.446 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:59.446 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:59.446 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:59.446 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:59.446 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6640 2024-09-21T14:58:59.446 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.446 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:59.446 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.447 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:59.447 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131670+0000", 2024-09-21T14:58:59.447 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:59.447 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.447 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.447 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:59.447 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131670+0000", 2024-09-21T14:58:59.447 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:59.447 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.447 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.447 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:59.447 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131670+0000", 2024-09-21T14:58:59.447 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T14:58:59.447 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.448 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.448 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:59.448 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131672+0000", 2024-09-21T14:58:59.448 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.818e-06 2024-09-21T14:58:59.448 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.448 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.448 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:59.448 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131675+0000", 2024-09-21T14:58:59.448 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5930000000000002e-06 2024-09-21T14:58:59.448 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.448 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.448 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:59.448 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131678+0000", 2024-09-21T14:58:59.448 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6790000000000001e-06 2024-09-21T14:58:59.448 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.449 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.449 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:59.449 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147681+0000", 2024-09-21T14:58:59.449 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016002755 2024-09-21T14:58:59.449 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.449 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.449 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:59.449 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147762+0000", 2024-09-21T14:58:59.449 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1052999999999998e-05 2024-09-21T14:58:59.449 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.449 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.449 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:59.449 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147859+0000", 2024-09-21T14:58:59.449 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.7199000000000002e-05 2024-09-21T14:58:59.449 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:59.449 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:59.450 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:59.450 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.450 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.450 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6641 3.1a 3:592d80ed:::rbd_data.11361bed1f78.0000000000000005:d [read 147456~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:59.450 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131680+0000", 2024-09-21T14:58:59.450 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 24.222980019000001, 2024-09-21T14:58:59.450 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016431296000000001, 2024-09-21T14:58:59.450 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:59.450 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:59.450 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:59.450 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:59.450 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:59.450 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6641 2024-09-21T14:58:59.450 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.451 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:59.451 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.451 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:59.451 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131680+0000", 2024-09-21T14:58:59.451 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:59.451 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.451 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.451 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:59.451 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131680+0000", 2024-09-21T14:58:59.451 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:59.451 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.451 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.451 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:59.451 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131680+0000", 2024-09-21T14:58:59.451 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T14:58:59.451 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.452 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.452 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:59.452 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131681+0000", 2024-09-21T14:58:59.452 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2610000000000001e-06 2024-09-21T14:58:59.452 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.452 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.452 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:59.452 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131685+0000", 2024-09-21T14:58:59.452 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.8299999999999998e-06 2024-09-21T14:58:59.452 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.452 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.452 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:59.452 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131688+0000", 2024-09-21T14:58:59.452 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6189999999999998e-06 2024-09-21T14:58:59.452 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.453 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.453 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:59.453 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147884+0000", 2024-09-21T14:58:59.453 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016196465 2024-09-21T14:58:59.453 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.453 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.453 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:59.453 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147949+0000", 2024-09-21T14:58:59.453 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4867000000000001e-05 2024-09-21T14:58:59.453 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.453 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.453 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:59.453 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.148112+0000", 2024-09-21T14:58:59.453 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00016247700000000001 2024-09-21T14:58:59.453 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:59.454 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:59.454 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:59.454 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.454 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.454 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6642 3.1a 3:592d80ed:::rbd_data.11361bed1f78.0000000000000005:d [read 212992~32768 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:58:59.454 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131690+0000", 2024-09-21T14:58:59.454 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 24.222970141000001, 2024-09-21T14:58:59.454 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016751335999999999, 2024-09-21T14:58:59.454 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:59.454 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:59.454 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:59.454 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:59.454 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:59.455 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6642 2024-09-21T14:58:59.455 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.455 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:59.455 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.455 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:59.455 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131690+0000", 2024-09-21T14:58:59.455 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:59.455 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.455 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.455 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:59.455 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131690+0000", 2024-09-21T14:58:59.455 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:59.455 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.455 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.455 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:59.455 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131690+0000", 2024-09-21T14:58:59.456 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T14:58:59.456 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.456 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.456 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:59.456 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131691+0000", 2024-09-21T14:58:59.456 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3120000000000001e-06 2024-09-21T14:58:59.456 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.456 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.456 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:59.456 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131694+0000", 2024-09-21T14:58:59.456 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0070000000000002e-06 2024-09-21T14:58:59.456 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.456 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.456 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:59.456 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131700+0000", 2024-09-21T14:58:59.457 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0800000000000002e-06 2024-09-21T14:58:59.457 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.457 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.457 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:59.457 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.148135+0000", 2024-09-21T14:58:59.457 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016434250000000001 2024-09-21T14:58:59.457 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.457 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.457 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:59.457 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.148204+0000", 2024-09-21T14:58:59.457 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9466000000000002e-05 2024-09-21T14:58:59.457 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.457 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.458 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:59.458 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.148441+0000", 2024-09-21T14:58:59.458 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00023743700000000001 2024-09-21T14:58:59.458 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:59.458 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:59.458 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:59.458 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:59.458 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:59.458 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:58:59.460 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T14:58:59.460 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:58:59.457+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:58:59.560 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:58:59.557+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:58:59.618 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:58:59.618 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:58:59.618 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T14:58:59.618 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:58:59.628 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T14:58:59.661 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:58:59.656+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:58:59.735 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:58:59.735 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:58:59.735 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T14:58:59.735 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T14:58:59.735 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:58:59.746 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T14:58:59.762 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:58:59.761+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:58:59.800 INFO:teuthology.orchestra.run.smithi032.stdout:1110 punch from 0x8ff1f34 to 0x8ffe158, (0xc224 bytes) 2024-09-21T14:58:59.808 INFO:teuthology.orchestra.run.smithi032.stdout:1111 punch from 0x2438cde to 0x243b6d3, (0x29f5 bytes) 2024-09-21T14:58:59.808 INFO:teuthology.orchestra.run.smithi032.stdout:1112 read 0x1cb2191 thru 0x1cb8e45 (0x6cb5 bytes) 2024-09-21T14:58:59.812 INFO:teuthology.orchestra.run.smithi032.stdout:1113 write 0x9b15f69 thru 0x9b1a403 (0x449b bytes) 2024-09-21T14:58:59.821 INFO:teuthology.orchestra.run.smithi032.stdout:1114 read 0x9ce27c8 thru 0x9ce3ac6 (0x12ff bytes) 2024-09-21T14:58:59.821 INFO:teuthology.orchestra.run.smithi032.stdout:1117 read 0x330d79c thru 0x33157cf (0x8034 bytes) 2024-09-21T14:58:59.822 INFO:teuthology.orchestra.run.smithi032.stdout:1120 read 0x28af96c thru 0x28be4b8 (0xeb4d bytes) 2024-09-21T14:58:59.823 INFO:teuthology.orchestra.run.smithi032.stdout:1121 punch from 0x3330afc to 0x333420c, (0x3710 bytes) 2024-09-21T14:58:59.823 INFO:teuthology.orchestra.run.smithi032.stdout:1122 punch from 0x73dec2 to 0x7419e9, (0x3b27 bytes) 2024-09-21T14:58:59.824 INFO:teuthology.orchestra.run.smithi032.stdout:1123 read 0x506da69 thru 0x5071e6a (0x4402 bytes) 2024-09-21T14:58:59.824 INFO:teuthology.orchestra.run.smithi032.stdout:1125 punch from 0x339c334 to 0x33a1ada, (0x57a6 bytes) 2024-09-21T14:58:59.841 INFO:teuthology.orchestra.run.smithi032.stdout:1127 trunc from 0xb22143f to 0x11d3621 2024-09-21T14:58:59.863 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:58:59.861+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:58:59.873 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:58:59.873 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T14:58:59.873 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T14:58:59.873 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T14:58:59.873 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.873 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:1454 3.e 3:71be909a:::rbd_data.1136515ecb6b.000000000000002a:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T14:58:59.873 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:30.339995+0000", 2024-09-21T14:58:59.873 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 29.531472331, 2024-09-21T14:58:59.873 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.033615974999999999, 2024-09-21T14:58:59.873 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:59.873 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:59.874 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:59.874 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:59.874 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:59.874 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1454 2024-09-21T14:58:59.874 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.874 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:59.874 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.874 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:59.874 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.339995+0000", 2024-09-21T14:58:59.874 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:59.874 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.874 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.874 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:59.874 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.339995+0000", 2024-09-21T14:58:59.875 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:59.875 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.875 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.875 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:59.875 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.339992+0000", 2024-09-21T14:58:59.875 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-21T14:58:59.875 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.875 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.875 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:59.875 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.340008+0000", 2024-09-21T14:58:59.875 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5906e-05 2024-09-21T14:58:59.875 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.875 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.875 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:59.876 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.340042+0000", 2024-09-21T14:58:59.876 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4066000000000002e-05 2024-09-21T14:58:59.876 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.876 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.876 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:59.876 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.340056+0000", 2024-09-21T14:58:59.876 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4389999999999999e-05 2024-09-21T14:58:59.876 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.876 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.876 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:59.876 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.340178+0000", 2024-09-21T14:58:59.876 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012202 2024-09-21T14:58:59.876 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.876 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.877 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:59.877 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.340424+0000", 2024-09-21T14:58:59.877 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000245682 2024-09-21T14:58:59.877 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.877 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.877 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 0", 2024-09-21T14:58:59.877 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.348708+0000", 2024-09-21T14:58:59.877 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0082834390000000001 2024-09-21T14:58:59.877 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.877 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.877 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:59.877 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.373176+0000", 2024-09-21T14:58:59.877 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024468888000000001 2024-09-21T14:58:59.877 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.877 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.877 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:59.878 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.373306+0000", 2024-09-21T14:58:59.878 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012991700000000001 2024-09-21T14:58:59.878 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.878 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.878 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:59.878 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.373464+0000", 2024-09-21T14:58:59.878 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000157913 2024-09-21T14:58:59.878 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.878 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.878 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:59.878 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.373611+0000", 2024-09-21T14:58:59.878 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014681699999999999 2024-09-21T14:58:59.878 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:59.878 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:59.879 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:59.879 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.879 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.879 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:15130 3.2 3:47f1bbdb:::rbd_data.11361bed1f78.0000000000000067:head [sparse-read 0~147456] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-21T14:58:59.879 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:40.288284+0000", 2024-09-21T14:58:59.879 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 19.583183009999999, 2024-09-21T14:58:59.879 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1142935039999999, 2024-09-21T14:58:59.879 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:59.879 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:59.879 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:59.879 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:59.879 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:59.879 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 15130 2024-09-21T14:58:59.879 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.880 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:59.880 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.880 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:59.880 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288284+0000", 2024-09-21T14:58:59.880 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:59.880 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.880 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.880 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:59.880 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288284+0000", 2024-09-21T14:58:59.880 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:59.880 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.880 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.880 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:59.880 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288284+0000", 2024-09-21T14:58:59.880 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:59.881 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.881 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.881 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:59.881 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288288+0000", 2024-09-21T14:58:59.881 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5829999999999998e-06 2024-09-21T14:58:59.881 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.881 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.881 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:59.881 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288297+0000", 2024-09-21T14:58:59.881 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0669999999999998e-06 2024-09-21T14:58:59.881 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.881 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.881 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:59.881 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288305+0000", 2024-09-21T14:58:59.882 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8860000000000004e-06 2024-09-21T14:58:59.882 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.882 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.882 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:59.882 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288417+0000", 2024-09-21T14:58:59.882 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000112952 2024-09-21T14:58:59.882 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.882 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.882 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T14:58:59.882 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288428+0000", 2024-09-21T14:58:59.882 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0631e-05 2024-09-21T14:58:59.882 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.882 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.882 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:59.883 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.401876+0000", 2024-09-21T14:58:59.883 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1134479610000001 2024-09-21T14:58:59.883 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.883 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.883 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:59.883 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.402076+0000", 2024-09-21T14:58:59.883 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00019944200000000001 2024-09-21T14:58:59.883 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.883 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.883 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:59.883 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.402578+0000", 2024-09-21T14:58:59.883 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00050242899999999998 2024-09-21T14:58:59.883 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:59.883 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:59.884 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:59.884 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.884 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.884 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:20598 3.2 3:403f4568:::rbd_data.1136c84d9a13.00000000000000d4:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-21T14:58:59.884 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:45.401322+0000", 2024-09-21T14:58:59.884 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.470145294, 2024-09-21T14:58:59.884 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014768051000000001, 2024-09-21T14:58:59.884 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:59.884 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:59.884 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:59.884 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:59.884 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:59.884 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 20598 2024-09-21T14:58:59.885 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.885 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:59.885 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.885 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:59.885 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:45.401322+0000", 2024-09-21T14:58:59.885 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:59.885 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.885 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.885 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:59.885 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:45.401322+0000", 2024-09-21T14:58:59.885 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:59.885 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.885 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.885 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:59.886 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:45.401320+0000", 2024-09-21T14:58:59.886 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T14:58:59.886 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.886 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.886 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:59.886 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:45.401330+0000", 2024-09-21T14:58:59.886 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0108999999999999e-05 2024-09-21T14:58:59.886 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.886 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.886 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:59.886 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:45.401351+0000", 2024-09-21T14:58:59.886 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1191000000000001e-05 2024-09-21T14:58:59.886 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.887 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.887 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:59.887 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:45.401361+0000", 2024-09-21T14:58:59.887 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5130000000000002e-06 2024-09-21T14:58:59.887 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.887 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.887 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:59.887 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:45.401476+0000", 2024-09-21T14:58:59.887 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000115684 2024-09-21T14:58:59.887 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.887 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.887 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:59.887 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:45.401717+0000", 2024-09-21T14:58:59.887 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000240256 2024-09-21T14:58:59.888 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.888 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.888 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 3", 2024-09-21T14:58:59.888 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:45.406009+0000", 2024-09-21T14:58:59.888 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0042920780000000004 2024-09-21T14:58:59.888 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.888 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.888 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:59.888 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:45.415983+0000", 2024-09-21T14:58:59.888 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0099743269999999998 2024-09-21T14:58:59.888 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.888 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.889 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:59.889 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:45.415998+0000", 2024-09-21T14:58:59.889 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.49e-05 2024-09-21T14:58:59.889 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.889 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.889 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:59.889 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:45.416049+0000", 2024-09-21T14:58:59.889 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0657e-05 2024-09-21T14:58:59.889 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.889 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.889 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:59.889 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:45.416090+0000", 2024-09-21T14:58:59.889 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1569000000000001e-05 2024-09-21T14:58:59.889 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:59.890 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:59.890 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:59.890 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.890 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.890 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22841 3.16 3:6fa60bdd:::rbd_data.11366acc894.0000000000000008:19 [read 917504~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:58:59.890 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.142001+0000", 2024-09-21T14:58:59.890 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 9.7294661130000009, 2024-09-21T14:58:59.890 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.011779609, 2024-09-21T14:58:59.890 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:59.890 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:59.890 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:59.890 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:59.890 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:59.891 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22841 2024-09-21T14:58:59.891 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.891 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:59.891 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.891 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:59.891 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.142001+0000", 2024-09-21T14:58:59.891 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:59.891 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.891 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.891 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:59.891 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.142001+0000", 2024-09-21T14:58:59.891 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:59.891 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.892 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.892 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:59.892 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.142001+0000", 2024-09-21T14:58:59.892 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:59.892 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.892 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.892 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:59.892 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.142003+0000", 2024-09-21T14:58:59.892 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4109999999999999e-06 2024-09-21T14:58:59.892 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.892 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.892 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:59.892 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.142009+0000", 2024-09-21T14:58:59.892 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6219999999999997e-06 2024-09-21T14:58:59.893 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.893 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.893 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:59.893 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.142013+0000", 2024-09-21T14:58:59.893 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2069999999999997e-06 2024-09-21T14:58:59.893 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.893 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.893 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:59.893 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.152044+0000", 2024-09-21T14:58:59.893 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.010030435000000001 2024-09-21T14:58:59.893 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.893 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.893 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:59.894 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.152123+0000", 2024-09-21T14:58:59.894 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8813000000000006e-05 2024-09-21T14:58:59.894 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.894 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.894 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:59.894 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.153781+0000", 2024-09-21T14:58:59.894 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001658412 2024-09-21T14:58:59.894 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:59.894 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:59.894 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:59.894 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.894 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.894 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22842 3.16 3:6fa60bdd:::rbd_data.11366acc894.0000000000000008:19 [read 983040~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:58:59.895 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.142018+0000", 2024-09-21T14:58:59.895 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 9.7294498520000001, 2024-09-21T14:58:59.895 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012170459, 2024-09-21T14:58:59.895 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:59.895 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:59.895 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:59.895 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:59.895 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:59.895 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22842 2024-09-21T14:58:59.895 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.895 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:59.895 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.895 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:59.896 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.142018+0000", 2024-09-21T14:58:59.896 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:59.896 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.896 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.896 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:59.896 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.142018+0000", 2024-09-21T14:58:59.896 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:59.896 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.896 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.896 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:59.896 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.142017+0000", 2024-09-21T14:58:59.896 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:59.896 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.896 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.897 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:59.897 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.142020+0000", 2024-09-21T14:58:59.897 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8119999999999999e-06 2024-09-21T14:58:59.897 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.897 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.897 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:59.897 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.142026+0000", 2024-09-21T14:58:59.897 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8150000000000002e-06 2024-09-21T14:58:59.897 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.897 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.897 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:59.897 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.142031+0000", 2024-09-21T14:58:59.897 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7040000000000002e-06 2024-09-21T14:58:59.897 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.897 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.897 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:59.898 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.153822+0000", 2024-09-21T14:58:59.898 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.011791658999999999 2024-09-21T14:58:59.898 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.898 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.898 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:59.898 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.153929+0000", 2024-09-21T14:58:59.898 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000106413 2024-09-21T14:58:59.898 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.898 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.898 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:59.898 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.154188+0000", 2024-09-21T14:58:59.898 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000259423 2024-09-21T14:58:59.898 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:59.898 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:59.899 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:59.899 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.899 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.899 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22914 3.16 3:6fa60bdd:::rbd_data.11366acc894.0000000000000008:19 [read 671744~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:58:59.899 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.143191+0000", 2024-09-21T14:58:59.899 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 9.7282767690000007, 2024-09-21T14:58:59.899 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.011598498, 2024-09-21T14:58:59.899 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:59.899 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:59.899 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:59.899 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:59.899 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:59.899 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22914 2024-09-21T14:58:59.900 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.900 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:59.900 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.900 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:59.900 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.143191+0000", 2024-09-21T14:58:59.900 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:59.900 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.900 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.900 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:59.900 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.143191+0000", 2024-09-21T14:58:59.900 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:59.900 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.900 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.901 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:59.901 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.143190+0000", 2024-09-21T14:58:59.901 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:58:59.901 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.901 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.901 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:59.901 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.143195+0000", 2024-09-21T14:58:59.901 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0710000000000002e-06 2024-09-21T14:58:59.901 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.901 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.901 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:59.901 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.143205+0000", 2024-09-21T14:58:59.901 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8300000000000008e-06 2024-09-21T14:58:59.902 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.902 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.902 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:59.902 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.143212+0000", 2024-09-21T14:58:59.902 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.063e-06 2024-09-21T14:58:59.902 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.902 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.902 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:59.902 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.154561+0000", 2024-09-21T14:58:59.902 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.011348829 2024-09-21T14:58:59.902 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.902 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.902 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:59.903 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.154624+0000", 2024-09-21T14:58:59.903 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3688000000000004e-05 2024-09-21T14:58:59.903 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.903 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.903 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:59.903 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.154789+0000", 2024-09-21T14:58:59.903 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000164764 2024-09-21T14:58:59.903 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:59.903 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:59.903 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:59.903 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.903 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.903 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22915 3.16 3:6fa60bdd:::rbd_data.11366acc894.0000000000000008:19 [read 737280~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:58:59.904 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.143218+0000", 2024-09-21T14:58:59.904 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 9.7282499100000006, 2024-09-21T14:58:59.904 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.011862391, 2024-09-21T14:58:59.904 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:59.904 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:59.904 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:59.904 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:59.904 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:59.904 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22915 2024-09-21T14:58:59.904 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.904 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:59.904 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.904 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:59.905 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.143218+0000", 2024-09-21T14:58:59.905 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:59.905 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.905 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.905 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:59.905 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.143218+0000", 2024-09-21T14:58:59.905 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:59.905 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.905 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.905 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:59.905 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.143217+0000", 2024-09-21T14:58:59.905 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:59.905 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.906 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.906 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:59.906 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.143220+0000", 2024-09-21T14:58:59.906 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2140000000000001e-06 2024-09-21T14:58:59.906 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.906 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.906 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:59.906 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.143226+0000", 2024-09-21T14:58:59.906 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2009999999999996e-06 2024-09-21T14:58:59.906 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.906 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.906 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:59.906 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.143232+0000", 2024-09-21T14:58:59.907 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1819999999999997e-06 2024-09-21T14:58:59.907 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.907 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.907 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:59.907 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.154823+0000", 2024-09-21T14:58:59.907 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.011591308999999999 2024-09-21T14:58:59.907 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.907 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.907 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:59.907 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.154898+0000", 2024-09-21T14:58:59.907 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5430000000000001e-05 2024-09-21T14:58:59.907 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.908 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.908 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:59.908 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.155080+0000", 2024-09-21T14:58:59.908 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000181569 2024-09-21T14:58:59.908 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:59.908 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:59.908 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:59.908 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.908 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.908 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22916 3.16 3:6fa60bdd:::rbd_data.11366acc894.0000000000000008:19 [read 802816~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:58:59.908 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.143236+0000", 2024-09-21T14:58:59.908 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 9.7282313380000005, 2024-09-21T14:58:59.908 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012700477, 2024-09-21T14:58:59.908 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:59.908 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:59.909 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:59.909 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:59.909 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:59.909 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22916 2024-09-21T14:58:59.909 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.909 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:59.909 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.909 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:59.909 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.143236+0000", 2024-09-21T14:58:59.909 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:59.909 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.909 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.909 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:59.909 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.143236+0000", 2024-09-21T14:58:59.909 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:59.910 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.910 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.910 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:59.910 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.143236+0000", 2024-09-21T14:58:59.910 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:59.910 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.910 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.910 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:59.910 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.143238+0000", 2024-09-21T14:58:59.910 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5059999999999998e-06 2024-09-21T14:58:59.910 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.910 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.910 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:59.911 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.143265+0000", 2024-09-21T14:58:59.911 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7039999999999999e-05 2024-09-21T14:58:59.911 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.911 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.911 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:59.911 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.143272+0000", 2024-09-21T14:58:59.911 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4350000000000002e-06 2024-09-21T14:58:59.911 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.911 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.911 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:59.911 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.155362+0000", 2024-09-21T14:58:59.911 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012090485999999999 2024-09-21T14:58:59.911 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.911 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.911 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:59.912 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.155453+0000", 2024-09-21T14:58:59.912 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1001999999999994e-05 2024-09-21T14:58:59.912 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.912 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.912 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:59.912 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.155937+0000", 2024-09-21T14:58:59.912 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00048336100000000002 2024-09-21T14:58:59.912 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:59.912 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:59.912 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:59.912 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.912 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.912 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22917 3.16 3:6fa60bdd:::rbd_data.11366acc894.0000000000000008:19 [read 868352~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:58:59.912 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.143279+0000", 2024-09-21T14:58:59.913 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 9.7281888829999996, 2024-09-21T14:58:59.913 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012979479, 2024-09-21T14:58:59.913 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:59.913 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:59.913 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:59.913 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:59.913 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:59.913 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22917 2024-09-21T14:58:59.913 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.913 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:59.913 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.913 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:59.913 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.143279+0000", 2024-09-21T14:58:59.913 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:59.913 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.914 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.914 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:59.914 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.143279+0000", 2024-09-21T14:58:59.914 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:59.914 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.914 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.914 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:59.914 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.143278+0000", 2024-09-21T14:58:59.914 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:59.914 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.914 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.914 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:59.914 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.143282+0000", 2024-09-21T14:58:59.914 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.681e-06 2024-09-21T14:58:59.915 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.915 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.915 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:59.915 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.143288+0000", 2024-09-21T14:58:59.915 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7220000000000002e-06 2024-09-21T14:58:59.915 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.915 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.915 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:59.915 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.143298+0000", 2024-09-21T14:58:59.915 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.533e-06 2024-09-21T14:58:59.915 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.915 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.915 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:59.915 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.155966+0000", 2024-09-21T14:58:59.915 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012667754 2024-09-21T14:58:59.916 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.916 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.916 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:59.916 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.156039+0000", 2024-09-21T14:58:59.916 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3506999999999999e-05 2024-09-21T14:58:59.916 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.916 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.916 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:59.916 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.156258+0000", 2024-09-21T14:58:59.916 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00021886200000000001 2024-09-21T14:58:59.916 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:59.916 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:59.916 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:59.916 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.916 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.917 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22918 3.16 3:6fa60bdd:::rbd_data.11366acc894.0000000000000008:19 [read 933888~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:58:59.917 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.143303+0000", 2024-09-21T14:58:59.917 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 9.7281648290000007, 2024-09-21T14:58:59.917 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013274182000000001, 2024-09-21T14:58:59.917 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:59.917 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:59.917 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:59.917 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:59.917 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:59.917 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22918 2024-09-21T14:58:59.917 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.917 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:59.917 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.917 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:59.917 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.143303+0000", 2024-09-21T14:58:59.918 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:59.918 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.918 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.918 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:59.918 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.143303+0000", 2024-09-21T14:58:59.918 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:59.918 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.918 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.918 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:59.918 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.143302+0000", 2024-09-21T14:58:59.918 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:59.918 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.918 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.918 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:59.919 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.143305+0000", 2024-09-21T14:58:59.919 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7949999999999999e-06 2024-09-21T14:58:59.919 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.919 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.919 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:59.919 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.143311+0000", 2024-09-21T14:58:59.919 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.699e-06 2024-09-21T14:58:59.919 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.919 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.919 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:59.919 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.143315+0000", 2024-09-21T14:58:59.919 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.092e-06 2024-09-21T14:58:59.919 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.919 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.920 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:59.920 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.156288+0000", 2024-09-21T14:58:59.920 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012972727999999999 2024-09-21T14:58:59.920 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.920 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.920 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:59.920 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.156362+0000", 2024-09-21T14:58:59.920 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4201000000000003e-05 2024-09-21T14:58:59.920 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.920 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.920 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:59.920 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.156577+0000", 2024-09-21T14:58:59.920 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000215052 2024-09-21T14:58:59.920 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:59.921 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:59.921 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:59.921 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.921 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.921 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22919 3.16 3:6fa60bdd:::rbd_data.11366acc894.0000000000000008:19 [read 999424~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:58:59.921 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.143320+0000", 2024-09-21T14:58:59.921 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 9.7281478610000001, 2024-09-21T14:58:59.921 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013576809, 2024-09-21T14:58:59.921 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:59.921 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:59.921 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:59.921 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:59.921 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:59.921 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22919 2024-09-21T14:58:59.921 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.922 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:59.922 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.922 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:59.922 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.143320+0000", 2024-09-21T14:58:59.922 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:59.922 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.922 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.922 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:59.922 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.143320+0000", 2024-09-21T14:58:59.922 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:59.922 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.922 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.922 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:59.922 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.143319+0000", 2024-09-21T14:58:59.922 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:59.923 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.923 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.923 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:59.923 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.143322+0000", 2024-09-21T14:58:59.923 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3e-06 2024-09-21T14:58:59.923 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.923 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.923 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:59.923 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.143327+0000", 2024-09-21T14:58:59.923 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7980000000000002e-06 2024-09-21T14:58:59.923 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.923 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.923 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:59.923 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.143332+0000", 2024-09-21T14:58:59.924 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2649999999999998e-06 2024-09-21T14:58:59.924 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.924 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.924 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:59.924 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.156604+0000", 2024-09-21T14:58:59.924 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013272885 2024-09-21T14:58:59.924 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.924 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.924 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:59.924 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.156677+0000", 2024-09-21T14:58:59.924 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2929999999999995e-05 2024-09-21T14:58:59.924 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.924 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.924 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:59.924 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.156896+0000", 2024-09-21T14:58:59.925 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000218975 2024-09-21T14:58:59.925 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:59.925 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:59.925 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:59.925 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.925 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.925 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22995 3.16 3:6fa60bdd:::rbd_data.11366acc894.0000000000000008:19 [read 884736~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:58:59.925 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.146618+0000", 2024-09-21T14:58:59.925 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 9.7248490259999993, 2024-09-21T14:58:59.925 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.011828758999999999, 2024-09-21T14:58:59.925 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:59.925 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:59.925 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:59.925 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:59.925 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:59.926 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22995 2024-09-21T14:58:59.926 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.926 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:59.926 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.926 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:59.926 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.146618+0000", 2024-09-21T14:58:59.926 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:59.926 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.926 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.926 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:59.926 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.146618+0000", 2024-09-21T14:58:59.926 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:59.927 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.927 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.927 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:59.927 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.146618+0000", 2024-09-21T14:58:59.927 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:59.927 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.927 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.927 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:59.927 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.146647+0000", 2024-09-21T14:58:59.927 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8975e-05 2024-09-21T14:58:59.927 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.927 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.927 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:59.928 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.146655+0000", 2024-09-21T14:58:59.928 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1240000000000005e-06 2024-09-21T14:58:59.928 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.928 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.928 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:59.928 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.146661+0000", 2024-09-21T14:58:59.928 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9089999999999997e-06 2024-09-21T14:58:59.928 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.928 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.928 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:59.928 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.158188+0000", 2024-09-21T14:58:59.928 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.011526508 2024-09-21T14:58:59.928 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.928 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.928 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:59.929 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.158261+0000", 2024-09-21T14:58:59.929 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3342000000000004e-05 2024-09-21T14:58:59.929 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.929 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.929 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:59.929 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.158447+0000", 2024-09-21T14:58:59.929 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000186176 2024-09-21T14:58:59.929 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:59.929 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:59.929 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:59.929 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.929 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.929 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22996 3.16 3:6fa60bdd:::rbd_data.11366acc894.0000000000000008:19 [read 950272~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:58:59.929 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.146667+0000", 2024-09-21T14:58:59.929 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 9.7248004619999993, 2024-09-21T14:58:59.929 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012144227, 2024-09-21T14:58:59.930 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:59.930 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:59.930 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:59.930 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:59.930 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:59.930 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22996 2024-09-21T14:58:59.930 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.930 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:59.930 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.930 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:59.930 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.146667+0000", 2024-09-21T14:58:59.930 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:59.930 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.930 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.930 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:59.931 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.146667+0000", 2024-09-21T14:58:59.931 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:59.931 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.931 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.931 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:59.931 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.146666+0000", 2024-09-21T14:58:59.931 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:59.931 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.931 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.931 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:59.931 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.146669+0000", 2024-09-21T14:58:59.931 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7800000000000001e-06 2024-09-21T14:58:59.931 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.931 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.931 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:59.931 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.146675+0000", 2024-09-21T14:58:59.932 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6910000000000002e-06 2024-09-21T14:58:59.932 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.932 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.932 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:59.932 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.146696+0000", 2024-09-21T14:58:59.932 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1200999999999999e-05 2024-09-21T14:58:59.932 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.932 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.932 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:59.932 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.158476+0000", 2024-09-21T14:58:59.932 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.011779707 2024-09-21T14:58:59.932 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.932 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.933 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:59.933 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.158607+0000", 2024-09-21T14:58:59.933 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013140299999999999 2024-09-21T14:58:59.933 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.933 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.933 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:59.933 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.158811+0000", 2024-09-21T14:58:59.933 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00020397899999999999 2024-09-21T14:58:59.933 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:59.933 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:59.933 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:59.933 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.933 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.933 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22997 3.16 3:6fa60bdd:::rbd_data.11366acc894.0000000000000008:19 [read 1015808~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:58:59.933 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.146702+0000", 2024-09-21T14:58:59.934 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 9.7247656619999994, 2024-09-21T14:58:59.934 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012590647999999999, 2024-09-21T14:58:59.934 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:59.934 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:59.934 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:59.934 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:59.934 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:59.934 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22997 2024-09-21T14:58:59.934 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.934 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:59.934 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.934 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:59.934 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.146702+0000", 2024-09-21T14:58:59.935 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:59.935 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.935 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.935 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:59.935 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.146702+0000", 2024-09-21T14:58:59.935 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:59.935 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.935 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.935 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:59.935 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.146701+0000", 2024-09-21T14:58:59.935 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:59.935 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.935 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.935 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:59.935 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.146705+0000", 2024-09-21T14:58:59.935 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4189999999999998e-06 2024-09-21T14:58:59.936 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.936 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.936 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:59.936 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.146712+0000", 2024-09-21T14:58:59.936 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0759999999999997e-06 2024-09-21T14:58:59.936 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.936 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.936 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:59.936 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.146717+0000", 2024-09-21T14:58:59.936 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0579999999999997e-06 2024-09-21T14:58:59.936 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.936 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.936 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:59.936 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.158850+0000", 2024-09-21T14:58:59.936 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012133632 2024-09-21T14:58:59.936 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.936 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.937 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:59.937 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.158956+0000", 2024-09-21T14:58:59.937 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000105334 2024-09-21T14:58:59.937 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.937 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.937 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:59.937 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.159292+0000", 2024-09-21T14:58:59.937 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000336762 2024-09-21T14:58:59.937 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:59.937 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:59.937 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:59.937 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.937 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.938 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:23096 3.16 3:6fa60bdd:::rbd_data.11366acc894.0000000000000008:19 [read 966656~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:58:59.938 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.149395+0000", 2024-09-21T14:58:59.938 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 9.7220728619999992, 2024-09-21T14:58:59.938 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.011817510999999999, 2024-09-21T14:58:59.938 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:59.938 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:59.938 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:59.938 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:59.938 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:59.938 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 23096 2024-09-21T14:58:59.938 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.938 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:59.938 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.938 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:59.938 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.149395+0000", 2024-09-21T14:58:59.939 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:59.939 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.939 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.939 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:59.939 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.149395+0000", 2024-09-21T14:58:59.939 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:59.939 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.939 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.939 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:59.939 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.149394+0000", 2024-09-21T14:58:59.939 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:59.939 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.939 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.939 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:59.939 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.149397+0000", 2024-09-21T14:58:59.940 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6620000000000001e-06 2024-09-21T14:58:59.940 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.940 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.940 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:59.940 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.149403+0000", 2024-09-21T14:58:59.940 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.75e-06 2024-09-21T14:58:59.940 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.940 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.940 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:59.940 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.149407+0000", 2024-09-21T14:58:59.940 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6450000000000004e-06 2024-09-21T14:58:59.940 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.940 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.940 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:59.940 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.160962+0000", 2024-09-21T14:58:59.940 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.011554627 2024-09-21T14:58:59.941 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.941 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.941 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:59.941 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.161034+0000", 2024-09-21T14:58:59.941 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2446000000000004e-05 2024-09-21T14:58:59.941 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.941 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.941 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:59.941 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.161212+0000", 2024-09-21T14:58:59.941 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017779499999999999 2024-09-21T14:58:59.941 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:59.941 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:59.941 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:59.941 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.942 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.942 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:23097 3.16 3:6fa60bdd:::rbd_data.11366acc894.0000000000000008:19 [read 1032192~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:58:59.942 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.149411+0000", 2024-09-21T14:58:59.942 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 9.7220562519999998, 2024-09-21T14:58:59.942 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012012066, 2024-09-21T14:58:59.942 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:59.942 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:58:59.942 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:59.942 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:59.942 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:59.942 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 23097 2024-09-21T14:58:59.942 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.942 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:59.942 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.942 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:59.942 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.149411+0000", 2024-09-21T14:58:59.943 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:59.943 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.943 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.943 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:59.943 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.149411+0000", 2024-09-21T14:58:59.943 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:59.943 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.943 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.943 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:59.943 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.149411+0000", 2024-09-21T14:58:59.943 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:59.943 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.943 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.944 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:59.944 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.149416+0000", 2024-09-21T14:58:59.944 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9899999999999997e-06 2024-09-21T14:58:59.944 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.944 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.944 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:59.944 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.149427+0000", 2024-09-21T14:58:59.944 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1112e-05 2024-09-21T14:58:59.944 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.944 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.944 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:59.944 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.149438+0000", 2024-09-21T14:58:59.944 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0721e-05 2024-09-21T14:58:59.945 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.945 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.945 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:59.945 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.161234+0000", 2024-09-21T14:58:59.945 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.011796602 2024-09-21T14:58:59.945 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.945 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.945 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:59.945 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.161277+0000", 2024-09-21T14:58:59.945 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2542e-05 2024-09-21T14:58:59.945 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.945 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.945 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:59.945 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.161423+0000", 2024-09-21T14:58:59.945 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014637799999999999 2024-09-21T14:58:59.945 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:59.946 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:59.946 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:59.946 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.946 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.946 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:26360 3.16 3:6b7d3323:::rbd_object_map.11367fb81126.000000000000001d:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-21T14:58:59.946 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:54.282410+0000", 2024-09-21T14:58:59.946 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 5.5890578089999998, 2024-09-21T14:58:59.946 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.01170494, 2024-09-21T14:58:59.946 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:59.946 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:59.946 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:59.946 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:59.946 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:59.947 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 26360 2024-09-21T14:58:59.947 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.947 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:59.947 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.947 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:59.947 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.282410+0000", 2024-09-21T14:58:59.947 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:59.947 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.947 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.947 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:59.947 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.282410+0000", 2024-09-21T14:58:59.947 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:59.947 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.947 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.947 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:59.947 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.282409+0000", 2024-09-21T14:58:59.948 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:58:59.948 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.948 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.948 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:59.948 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.282417+0000", 2024-09-21T14:58:59.948 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1470000000000001e-06 2024-09-21T14:58:59.948 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.948 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.948 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:59.948 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.282439+0000", 2024-09-21T14:58:59.948 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1878e-05 2024-09-21T14:58:59.948 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.948 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.948 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:59.948 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.282446+0000", 2024-09-21T14:58:59.948 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2810000000000003e-06 2024-09-21T14:58:59.949 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.949 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.949 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:59.949 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.287889+0000", 2024-09-21T14:58:59.949 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0054428469999999998 2024-09-21T14:58:59.949 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.949 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.949 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:59.949 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.290591+0000", 2024-09-21T14:58:59.949 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0027019510000000002 2024-09-21T14:58:59.949 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.949 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.949 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 3", 2024-09-21T14:58:59.949 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.290936+0000", 2024-09-21T14:58:59.949 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00034511500000000001 2024-09-21T14:58:59.950 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.950 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.950 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:59.950 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.293945+0000", 2024-09-21T14:58:59.950 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0030093849999999998 2024-09-21T14:58:59.950 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.950 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.950 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:59.950 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.294016+0000", 2024-09-21T14:58:59.950 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0555999999999993e-05 2024-09-21T14:58:59.950 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.950 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.950 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:59.950 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.294071+0000", 2024-09-21T14:58:59.950 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5711999999999998e-05 2024-09-21T14:58:59.951 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.951 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.951 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:59.951 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.294115+0000", 2024-09-21T14:58:59.951 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3105000000000001e-05 2024-09-21T14:58:59.951 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:59.951 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:59.951 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:59.951 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.951 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.951 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:26361 3.16 3:6b7d3323:::rbd_object_map.11367fb81126.000000000000001d:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-21T14:58:59.951 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:54.282895+0000", 2024-09-21T14:58:59.951 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 5.588572772, 2024-09-21T14:58:59.951 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012623578999999999, 2024-09-21T14:58:59.951 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:59.952 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:59.952 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:59.952 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:59.952 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:59.952 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 26361 2024-09-21T14:58:59.953 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.953 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:59.953 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.953 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:59.953 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.282895+0000", 2024-09-21T14:58:59.953 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:59.954 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.954 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.954 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:59.954 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.282895+0000", 2024-09-21T14:58:59.954 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:59.954 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.954 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.954 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:59.954 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.282894+0000", 2024-09-21T14:58:59.954 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:58:59.954 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.954 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.954 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:59.954 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.282901+0000", 2024-09-21T14:58:59.955 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8750000000000002e-06 2024-09-21T14:58:59.955 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.955 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.955 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:59.955 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.282911+0000", 2024-09-21T14:58:59.955 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0710000000000001e-05 2024-09-21T14:58:59.955 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.955 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.955 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:59.955 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.282918+0000", 2024-09-21T14:58:59.955 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7769999999999997e-06 2024-09-21T14:58:59.955 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.955 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.955 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:59.955 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.291985+0000", 2024-09-21T14:58:59.956 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0090669989999999992 2024-09-21T14:58:59.956 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.956 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.956 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:59.956 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.292071+0000", 2024-09-21T14:58:59.956 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6292999999999995e-05 2024-09-21T14:58:59.956 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.956 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.956 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 3", 2024-09-21T14:58:59.956 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.292421+0000", 2024-09-21T14:58:59.956 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000349821 2024-09-21T14:58:59.956 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.956 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.956 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:59.956 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.295201+0000", 2024-09-21T14:58:59.956 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0027803490000000001 2024-09-21T14:58:59.957 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.957 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.957 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:59.957 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.295443+0000", 2024-09-21T14:58:59.957 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000241296 2024-09-21T14:58:59.957 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.957 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.957 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:59.957 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.295483+0000", 2024-09-21T14:58:59.957 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.0469999999999997e-05 2024-09-21T14:58:59.957 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.957 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.957 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:59.957 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.295518+0000", 2024-09-21T14:58:59.957 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4995e-05 2024-09-21T14:58:59.958 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:59.958 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:59.958 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:59.958 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.958 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.958 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:26362 3.16 3:6b7d3323:::rbd_object_map.11367fb81126.000000000000001d:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-21T14:58:59.958 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:54.282929+0000", 2024-09-21T14:58:59.958 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 5.588538571, 2024-09-21T14:58:59.958 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012685663, 2024-09-21T14:58:59.958 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:59.958 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:59.958 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:59.958 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:59.958 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:59.958 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 26362 2024-09-21T14:58:59.958 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.959 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:59.959 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.959 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:59.959 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.282929+0000", 2024-09-21T14:58:59.959 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:59.959 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.959 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.959 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:59.959 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.282929+0000", 2024-09-21T14:58:59.959 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:59.959 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.959 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.959 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:59.959 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.282928+0000", 2024-09-21T14:58:59.959 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:58:59.959 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.960 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.960 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:59.960 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.282933+0000", 2024-09-21T14:58:59.960 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3719999999999998e-06 2024-09-21T14:58:59.960 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.960 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.960 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:59.960 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.282940+0000", 2024-09-21T14:58:59.960 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9940000000000003e-06 2024-09-21T14:58:59.960 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.960 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.960 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:59.960 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.282946+0000", 2024-09-21T14:58:59.960 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7009999999999999e-06 2024-09-21T14:58:59.960 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.960 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.961 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:59.961 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.292982+0000", 2024-09-21T14:58:59.961 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.010035661 2024-09-21T14:58:59.961 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.961 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.961 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:59.961 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.293068+0000", 2024-09-21T14:58:59.961 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6211999999999994e-05 2024-09-21T14:58:59.961 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.961 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.961 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 3", 2024-09-21T14:58:59.961 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.293388+0000", 2024-09-21T14:58:59.961 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00031920499999999998 2024-09-21T14:58:59.961 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.961 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.962 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:59.962 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.295397+0000", 2024-09-21T14:58:59.962 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0020094079999999999 2024-09-21T14:58:59.962 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.962 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.962 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:59.962 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.295545+0000", 2024-09-21T14:58:59.962 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014850700000000001 2024-09-21T14:58:59.962 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.962 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.962 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:59.962 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.295579+0000", 2024-09-21T14:58:59.962 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3346000000000002e-05 2024-09-21T14:58:59.962 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.962 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.963 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:59.963 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.295615+0000", 2024-09-21T14:58:59.963 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5743e-05 2024-09-21T14:58:59.963 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:59.963 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:59.963 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:59.963 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.963 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.963 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:26363 3.16 3:6b7d3323:::rbd_object_map.11367fb81126.000000000000001d:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-21T14:58:59.963 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:54.283268+0000", 2024-09-21T14:58:59.963 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 5.5881995260000004, 2024-09-21T14:58:59.963 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013363939, 2024-09-21T14:58:59.963 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:58:59.963 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:58:59.963 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:58:59.963 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:58:59.964 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:58:59.964 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 26363 2024-09-21T14:58:59.964 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.964 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:58:59.964 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.964 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:58:59.964 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.283268+0000", 2024-09-21T14:58:59.964 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:59.964 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.964 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.964 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:58:59.964 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.283268+0000", 2024-09-21T14:58:59.964 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:58:59.964 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.964 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.965 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:58:59.965 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.283267+0000", 2024-09-21T14:58:59.965 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:58:59.965 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.965 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.965 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:58:59.965 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.283273+0000", 2024-09-21T14:58:59.965 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4330000000000001e-06 2024-09-21T14:58:59.965 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.965 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.965 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:58:59.965 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.283319+0000", 2024-09-21T14:58:59.965 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6171999999999998e-05 2024-09-21T14:58:59.965 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.965 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.965 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:58:59.966 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.283328+0000", 2024-09-21T14:58:59.966 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3590000000000008e-06 2024-09-21T14:58:59.966 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.966 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.966 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:58:59.966 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.294182+0000", 2024-09-21T14:58:59.966 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.010854074 2024-09-21T14:58:59.966 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.966 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.966 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:58:59.966 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.294264+0000", 2024-09-21T14:58:59.966 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2544000000000001e-05 2024-09-21T14:58:59.966 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.966 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.967 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 3", 2024-09-21T14:58:59.967 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.294723+0000", 2024-09-21T14:58:59.967 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000458181 2024-09-21T14:58:59.967 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.967 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.967 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:58:59.967 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.296044+0000", 2024-09-21T14:58:59.967 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001321486 2024-09-21T14:58:59.967 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.967 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.967 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:58:59.967 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.296560+0000", 2024-09-21T14:58:59.967 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00051559700000000002 2024-09-21T14:58:59.967 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.967 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.967 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:58:59.968 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.296596+0000", 2024-09-21T14:58:59.968 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6145e-05 2024-09-21T14:58:59.968 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:58:59.968 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:58:59.968 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:58:59.968 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.296632+0000", 2024-09-21T14:58:59.968 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6038000000000003e-05 2024-09-21T14:58:59.968 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:59.968 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:59.968 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:59.968 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:58:59.968 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:58:59.968 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:58:59.970 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T14:58:59.971 INFO:teuthology.orchestra.run.smithi032.stdout:1128 punch from 0x55b5bd to 0x564cb8, (0x96fb bytes) 2024-09-21T14:58:59.971 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:58:59.971 INFO:teuthology.orchestra.run.smithi032.stdout:1130 writesame 0x2d31684 thru 0x2d39927 data_size 0x12(0x82a4 bytes) 2024-09-21T14:58:59.971 INFO:teuthology.orchestra.run.smithi032.stdout:1131 punch from 0x24db7e7 to 0x24de802, (0x301b bytes) 2024-09-21T14:58:59.971 INFO:teuthology.orchestra.run.smithi032.stdout:1133 write 0x8be952b thru 0x8be98e3 (0x3b9 bytes) 2024-09-21T14:58:59.971 INFO:teuthology.orchestra.run.smithi032.stdout:1135 trunc from 0x8be98e4 to 0x110dc10 2024-09-21T14:58:59.971 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:58:59.969+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:58:59.992 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:00.016 INFO:teuthology.orchestra.run.smithi032.stdout:1137 trunc from 0x110dc10 to 0xbf3acde 2024-09-21T14:59:00.046 INFO:teuthology.orchestra.run.smithi032.stdout:1138 trunc from 0xbf3acde to 0x2402348 2024-09-21T14:59:00.059 INFO:teuthology.orchestra.run.smithi032.stdout:1139 read 0x18ce8ee thru 0x18dd44e (0xeb61 bytes) 2024-09-21T14:59:00.059 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:00.059 INFO:teuthology.orchestra.run.smithi032.stdout:1141 read 0xb959cc thru 0xba4339 (0xe96e bytes) 2024-09-21T14:59:00.062 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:00.062 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:00.062 INFO:teuthology.orchestra.run.smithi032.stdout:1144 read 0x637cdd thru 0x641ee9 (0xa20d bytes) 2024-09-21T14:59:00.070 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:59:00.068+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:59:00.077 INFO:teuthology.orchestra.run.smithi032.stdout:1145 writesame 0x3c8cf59 thru 0x3c91f5a data_size 0x4d(0x5002 bytes) 2024-09-21T14:59:00.077 INFO:teuthology.orchestra.run.smithi032.stdout:1146 read 0x2908ef6 thru 0x2918155 (0xf260 bytes) 2024-09-21T14:59:00.077 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:00.077 INFO:teuthology.orchestra.run.smithi032.stdout:1148 write 0x104674f thru 0x10492e0 (0x2b92 bytes) 2024-09-21T14:59:00.081 INFO:teuthology.orchestra.run.smithi032.stdout:1150 punch from 0x24383fc to 0x2438475, (0x79 bytes) 2024-09-21T14:59:00.081 INFO:teuthology.orchestra.run.smithi032.stdout:1151 read 0x26c91ee thru 0x26cded1 (0x4ce4 bytes) 2024-09-21T14:59:00.096 INFO:teuthology.orchestra.run.smithi032.stdout:1152 trunc from 0x3c91f5b to 0xcb87ffc 2024-09-21T14:59:00.110 INFO:teuthology.orchestra.run.smithi032.stdout:1153 write 0x1b71818 thru 0x1b73432 (0x1c1b bytes) 2024-09-21T14:59:00.119 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:00.119 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:00.119 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:59:00.120 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T14:59:00.120 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:00.134 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T14:59:00.134 INFO:teuthology.orchestra.run.smithi032.stdout:1155 trunc from 0xcb87ffc to 0x2ccab3a 2024-09-21T14:59:00.148 INFO:teuthology.orchestra.run.smithi032.stdout:1159 trunc from 0x2ccab3a to 0x4de5c4 2024-09-21T14:59:00.170 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:59:00.168+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:59:00.175 INFO:teuthology.orchestra.run.smithi032.stdout:1161 read 0x290011 thru 0x29147a (0x146a bytes) 2024-09-21T14:59:00.176 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:00.176 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:00.192 INFO:teuthology.orchestra.run.smithi032.stdout:1165 write 0x499faea thru 0x49a0a20 (0xf37 bytes) 2024-09-21T14:59:00.199 INFO:teuthology.orchestra.run.smithi032.stdout:1166 write 0xd532c7 thru 0xd5cd41 (0x9a7b bytes) 2024-09-21T14:59:00.231 INFO:teuthology.orchestra.run.smithi032.stdout:1167 write 0x9f11ce0 thru 0x9f1de85 (0xc1a6 bytes) 2024-09-21T14:59:00.240 INFO:teuthology.orchestra.run.smithi032.stdout:1168 read 0x8747630 thru 0x8753f0b (0xc8dc bytes) 2024-09-21T14:59:00.253 INFO:teuthology.orchestra.run.smithi032.stdout:1169 write 0xa8d5f06 thru 0xa8d70f8 (0x11f3 bytes) 2024-09-21T14:59:00.261 INFO:teuthology.orchestra.run.smithi032.stdout:1170 read 0x90bf702 thru 0x90cdf31 (0xe830 bytes) 2024-09-21T14:59:00.261 INFO:teuthology.orchestra.run.smithi032.stdout:1171 punch from 0x61706e9 to 0x61805b3, (0xfeca bytes) 2024-09-21T14:59:00.265 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:00.266 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:59:00.266 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T14:59:00.266 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T14:59:00.266 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:00.270 INFO:teuthology.orchestra.run.smithi032.stdout:1173 trunc from 0xa8d70f9 to 0x5afae3a 2024-09-21T14:59:00.272 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:59:00.268+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:59:00.279 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T14:59:00.316 INFO:teuthology.orchestra.run.smithi032.stdout:1176 write 0xd3253c7 thru 0xd32f50f (0xa149 bytes) 2024-09-21T14:59:00.320 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T14:59:00.335 INFO:teuthology.orchestra.run.smithi032.stdout:1177 trunc from 0xd32f510 to 0x8704352 2024-09-21T14:59:00.354 INFO:teuthology.orchestra.run.smithi032.stdout:1178 read 0x693d59a thru 0x6946782 (0x91e9 bytes) 2024-09-21T14:59:00.373 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:59:00.368+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:59:00.375 INFO:teuthology.orchestra.run.smithi032.stdout:1179 write 0xc672c2b thru 0xc681368 (0xe73e bytes) 2024-09-21T14:59:00.387 INFO:teuthology.orchestra.run.smithi032.stdout:1180 writesame 0x1abacad thru 0x1abf39b data_size 0x1(0x46ef bytes) 2024-09-21T14:59:00.405 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:00.405 INFO:teuthology.orchestra.run.smithi032.stdout:1183 punch from 0x9353238 to 0x93595ef, (0x63b7 bytes) 2024-09-21T14:59:00.405 INFO:teuthology.orchestra.run.smithi032.stdout:1184 read 0x98a66f7 thru 0x98b020d (0x9b17 bytes) 2024-09-21T14:59:00.406 INFO:teuthology.orchestra.run.smithi032.stdout:truncating to largest ever: 0xea70eff 2024-09-21T14:59:00.410 INFO:teuthology.orchestra.run.smithi032.stdout:1185 trunc from 0xc681369 to 0xea70eff 2024-09-21T14:59:00.420 INFO:teuthology.orchestra.run.smithi032.stdout:1186 read 0x304f64b thru 0x305e053 (0xea09 bytes) 2024-09-21T14:59:00.420 INFO:teuthology.orchestra.run.smithi032.stdout:1187 writesame 0xc9091cc thru 0xc90b805 data_size 0x15(0x263a bytes) 2024-09-21T14:59:00.426 INFO:teuthology.orchestra.run.smithi032.stdout:1191 read 0x9c90e3b thru 0x9c96da8 (0x5f6e bytes) 2024-09-21T14:59:00.426 INFO:teuthology.orchestra.run.smithi032.stdout:1192 clone 11 order 18 su 2048 sc 7 2024-09-21T14:59:00.435 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:00.435 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T14:59:00.435 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T14:59:00.435 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T14:59:00.435 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.435 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4281.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-21T14:59:00.435 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:09.324416+0000", 2024-09-21T14:59:00.435 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 51.109257477, 2024-09-21T14:59:00.436 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.117837947, 2024-09-21T14:59:00.436 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:00.436 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:00.436 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:00.436 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4281", 2024-09-21T14:59:00.436 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/1518520303", 2024-09-21T14:59:00.436 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1 2024-09-21T14:59:00.436 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.436 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:00.436 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.436 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:00.436 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324416+0000", 2024-09-21T14:59:00.437 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:00.437 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.437 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.437 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:00.437 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324416+0000", 2024-09-21T14:59:00.437 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:00.437 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.437 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.437 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:00.437 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324414+0000", 2024-09-21T14:59:00.437 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T14:59:00.437 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.437 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.438 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:00.438 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324429+0000", 2024-09-21T14:59:00.438 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5637e-05 2024-09-21T14:59:00.438 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.438 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.438 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:00.438 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324448+0000", 2024-09-21T14:59:00.438 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9042000000000002e-05 2024-09-21T14:59:00.438 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.438 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.438 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:00.438 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324465+0000", 2024-09-21T14:59:00.439 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6767999999999999e-05 2024-09-21T14:59:00.439 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.439 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.439 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:00.439 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324571+0000", 2024-09-21T14:59:00.439 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000105714 2024-09-21T14:59:00.439 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.439 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.439 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op must wait for map", 2024-09-21T14:59:00.439 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324587+0000", 2024-09-21T14:59:00.439 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5753999999999999e-05 2024-09-21T14:59:00.440 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.440 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.440 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:00.440 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.325864+0000", 2024-09-21T14:59:00.440 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0012773350000000001 2024-09-21T14:59:00.440 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.440 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.440 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T14:59:00.440 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.325878+0000", 2024-09-21T14:59:00.440 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4498000000000001e-05 2024-09-21T14:59:00.441 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.441 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.441 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:00.441 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.439461+0000", 2024-09-21T14:59:00.441 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.11358212500000001 2024-09-21T14:59:00.441 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.441 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.441 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:00.441 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.439848+0000", 2024-09-21T14:59:00.441 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00038747800000000002 2024-09-21T14:59:00.442 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.442 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.442 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:59:00.442 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.440175+0000", 2024-09-21T14:59:00.442 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00032721900000000001 2024-09-21T14:59:00.442 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.442 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.442 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:00.442 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.441770+0000", 2024-09-21T14:59:00.442 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0015949409999999999 2024-09-21T14:59:00.442 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.443 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.443 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:00.443 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.442086+0000", 2024-09-21T14:59:00.443 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00031586299999999999 2024-09-21T14:59:00.443 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.443 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.443 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:00.443 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.442195+0000", 2024-09-21T14:59:00.443 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000108791 2024-09-21T14:59:00.443 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.444 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.444 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:00.444 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.442254+0000", 2024-09-21T14:59:00.444 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9038e-05 2024-09-21T14:59:00.444 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:00.444 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:00.444 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:00.444 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.444 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.444 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4281.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-21T14:59:00.444 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:09.447828+0000", 2024-09-21T14:59:00.444 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 50.985845476000001, 2024-09-21T14:59:00.444 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.32904393700000001, 2024-09-21T14:59:00.444 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:00.445 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:00.445 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:00.445 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4281", 2024-09-21T14:59:00.445 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/1518520303", 2024-09-21T14:59:00.445 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 2 2024-09-21T14:59:00.445 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.445 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:00.445 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.445 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:00.445 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447828+0000", 2024-09-21T14:59:00.445 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:00.445 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.445 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.446 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:00.446 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447828+0000", 2024-09-21T14:59:00.446 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:00.446 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.446 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.446 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:00.446 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447825+0000", 2024-09-21T14:59:00.446 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T14:59:00.446 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.446 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.446 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:00.446 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447841+0000", 2024-09-21T14:59:00.446 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5359999999999999e-05 2024-09-21T14:59:00.447 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.447 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.447 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:00.447 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447875+0000", 2024-09-21T14:59:00.447 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4388999999999999e-05 2024-09-21T14:59:00.447 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.447 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.447 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:00.447 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447903+0000", 2024-09-21T14:59:00.447 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7373e-05 2024-09-21T14:59:00.447 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.447 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.447 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:00.448 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447996+0000", 2024-09-21T14:59:00.448 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2954999999999993e-05 2024-09-21T14:59:00.448 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.448 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.448 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T14:59:00.448 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.448011+0000", 2024-09-21T14:59:00.448 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5038999999999999e-05 2024-09-21T14:59:00.448 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.448 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.448 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:00.448 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.776501+0000", 2024-09-21T14:59:00.448 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.32849012500000002 2024-09-21T14:59:00.448 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.449 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.449 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:00.449 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.776872+0000", 2024-09-21T14:59:00.449 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00037119600000000002 2024-09-21T14:59:00.449 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:00.449 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:00.449 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:00.449 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.449 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.449 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:752 3.14 3:28fc02e5:::rbd_data.113693316f36.0000000000000010:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-21T14:59:00.449 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:26.625579+0000", 2024-09-21T14:59:00.449 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 33.808094621000002, 2024-09-21T14:59:00.449 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042965426000000001, 2024-09-21T14:59:00.449 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:00.449 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:00.450 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:00.450 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:00.450 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:00.450 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 752 2024-09-21T14:59:00.450 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.450 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:00.450 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.450 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:00.450 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625579+0000", 2024-09-21T14:59:00.450 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:00.450 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.450 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.450 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:00.450 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625579+0000", 2024-09-21T14:59:00.451 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:00.451 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.451 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.451 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:00.451 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625577+0000", 2024-09-21T14:59:00.451 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T14:59:00.451 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.451 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.451 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:00.451 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625588+0000", 2024-09-21T14:59:00.451 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1555e-05 2024-09-21T14:59:00.451 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.451 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.452 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:00.452 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625610+0000", 2024-09-21T14:59:00.452 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2107999999999999e-05 2024-09-21T14:59:00.452 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.452 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.452 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:00.452 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625621+0000", 2024-09-21T14:59:00.452 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0762e-05 2024-09-21T14:59:00.452 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.452 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.452 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:00.452 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625692+0000", 2024-09-21T14:59:00.453 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.1203000000000003e-05 2024-09-21T14:59:00.453 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.453 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.453 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:00.453 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625872+0000", 2024-09-21T14:59:00.453 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017962800000000001 2024-09-21T14:59:00.453 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.453 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.453 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:59:00.453 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.633188+0000", 2024-09-21T14:59:00.453 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.007316014 2024-09-21T14:59:00.453 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.453 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.454 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:00.454 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.668357+0000", 2024-09-21T14:59:00.454 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035168933999999999 2024-09-21T14:59:00.454 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.454 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.454 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:00.454 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.668418+0000", 2024-09-21T14:59:00.454 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1282e-05 2024-09-21T14:59:00.454 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.454 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.454 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:00.454 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.668486+0000", 2024-09-21T14:59:00.454 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8468999999999998e-05 2024-09-21T14:59:00.455 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.455 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.455 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:00.455 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.668544+0000", 2024-09-21T14:59:00.455 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7785000000000003e-05 2024-09-21T14:59:00.455 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:00.455 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:00.455 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:00.455 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.455 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.455 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22737 3.a 3:574c7a60:::rbd_data.11366acc894.0000000000000002:19 [read 1032192~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:00.455 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141028+0000", 2024-09-21T14:59:00.455 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.292645655999999, 2024-09-21T14:59:00.456 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035206281999999998, 2024-09-21T14:59:00.456 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:00.456 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:00.456 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:00.456 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:00.456 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:00.456 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22737 2024-09-21T14:59:00.456 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.456 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:00.456 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.456 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:00.456 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141028+0000", 2024-09-21T14:59:00.456 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:00.456 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.457 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.457 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:00.457 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141028+0000", 2024-09-21T14:59:00.457 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:00.457 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.457 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.457 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:00.457 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141027+0000", 2024-09-21T14:59:00.457 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:00.457 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.457 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.457 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:00.457 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141030+0000", 2024-09-21T14:59:00.457 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6189999999999998e-06 2024-09-21T14:59:00.458 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.458 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.458 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:00.458 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141036+0000", 2024-09-21T14:59:00.458 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5110000000000003e-06 2024-09-21T14:59:00.458 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.458 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.458 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:00.458 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141040+0000", 2024-09-21T14:59:00.458 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.34e-06 2024-09-21T14:59:00.458 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.458 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.458 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:00.458 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.175948+0000", 2024-09-21T14:59:00.459 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034908540000000002 2024-09-21T14:59:00.459 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.459 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.459 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:00.459 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176025+0000", 2024-09-21T14:59:00.459 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6412e-05 2024-09-21T14:59:00.459 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.459 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.459 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:00.459 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176234+0000", 2024-09-21T14:59:00.459 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000209247 2024-09-21T14:59:00.459 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:00.459 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:00.459 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:00.460 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.460 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.460 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22738 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 49152~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:00.460 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141049+0000", 2024-09-21T14:59:00.460 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.292624478, 2024-09-21T14:59:00.460 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035428754999999999, 2024-09-21T14:59:00.460 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:00.460 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:00.460 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:00.460 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:00.460 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:00.460 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22738 2024-09-21T14:59:00.460 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.461 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:00.461 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.461 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:00.461 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141049+0000", 2024-09-21T14:59:00.461 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:00.461 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.461 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.461 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:00.461 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141049+0000", 2024-09-21T14:59:00.461 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:00.461 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.461 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.461 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:00.461 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141049+0000", 2024-09-21T14:59:00.461 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:00.461 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.462 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.462 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:00.462 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141051+0000", 2024-09-21T14:59:00.462 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4980000000000001e-06 2024-09-21T14:59:00.462 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.462 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.462 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:00.462 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141057+0000", 2024-09-21T14:59:00.462 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8030000000000002e-06 2024-09-21T14:59:00.462 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.462 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.462 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:00.462 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141066+0000", 2024-09-21T14:59:00.462 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1160000000000005e-06 2024-09-21T14:59:00.462 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.463 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.463 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:00.463 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176264+0000", 2024-09-21T14:59:00.463 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035197580999999999 2024-09-21T14:59:00.463 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.463 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.463 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:00.463 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176327+0000", 2024-09-21T14:59:00.463 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3237000000000005e-05 2024-09-21T14:59:00.463 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.463 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.463 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:00.463 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176478+0000", 2024-09-21T14:59:00.463 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000150851 2024-09-21T14:59:00.463 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:00.464 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:00.464 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:00.464 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.464 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.464 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22739 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 114688~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:00.464 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141070+0000", 2024-09-21T14:59:00.464 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.292603236, 2024-09-21T14:59:00.464 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035679094000000001, 2024-09-21T14:59:00.464 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:00.464 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:00.464 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:00.464 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:00.464 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:00.464 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22739 2024-09-21T14:59:00.465 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.465 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:00.465 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.465 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:00.465 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141070+0000", 2024-09-21T14:59:00.465 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:00.465 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.465 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.465 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:00.465 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141070+0000", 2024-09-21T14:59:00.465 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:00.465 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.465 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.465 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:00.466 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141070+0000", 2024-09-21T14:59:00.466 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:00.466 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.466 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.466 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:00.466 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141072+0000", 2024-09-21T14:59:00.466 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.379e-06 2024-09-21T14:59:00.466 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.466 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.466 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:00.466 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141078+0000", 2024-09-21T14:59:00.466 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6710000000000004e-06 2024-09-21T14:59:00.466 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.466 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.467 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:00.467 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141083+0000", 2024-09-21T14:59:00.467 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7249999999999997e-06 2024-09-21T14:59:00.467 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.467 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.467 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:00.467 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176506+0000", 2024-09-21T14:59:00.467 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035423076999999997 2024-09-21T14:59:00.467 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.467 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.467 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:00.467 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176582+0000", 2024-09-21T14:59:00.467 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6305999999999998e-05 2024-09-21T14:59:00.467 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.467 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.468 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:00.468 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176749+0000", 2024-09-21T14:59:00.468 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000167262 2024-09-21T14:59:00.468 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:00.468 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:00.468 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:00.468 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.468 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.468 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22740 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 180224~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:00.468 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141090+0000", 2024-09-21T14:59:00.468 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.292583832, 2024-09-21T14:59:00.468 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036129535999999997, 2024-09-21T14:59:00.468 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:00.468 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:00.468 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:00.468 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:00.469 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:00.469 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22740 2024-09-21T14:59:00.469 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.469 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:00.469 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.469 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:00.469 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141090+0000", 2024-09-21T14:59:00.469 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:00.469 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.469 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.469 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:00.469 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141090+0000", 2024-09-21T14:59:00.469 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:00.469 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.469 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.470 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:00.470 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141089+0000", 2024-09-21T14:59:00.470 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:00.470 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.470 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.470 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:00.470 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141092+0000", 2024-09-21T14:59:00.470 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7709999999999998e-06 2024-09-21T14:59:00.470 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.470 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.470 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:00.470 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141098+0000", 2024-09-21T14:59:00.470 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.015e-06 2024-09-21T14:59:00.470 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.471 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.471 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:00.471 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141103+0000", 2024-09-21T14:59:00.471 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8629999999999999e-06 2024-09-21T14:59:00.471 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.471 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.471 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:00.471 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176778+0000", 2024-09-21T14:59:00.471 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035674984 2024-09-21T14:59:00.471 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.471 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.471 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:00.471 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176851+0000", 2024-09-21T14:59:00.471 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2861000000000006e-05 2024-09-21T14:59:00.471 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.471 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.472 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:00.472 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177219+0000", 2024-09-21T14:59:00.472 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00036845999999999998 2024-09-21T14:59:00.472 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:00.472 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:00.472 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:00.472 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.472 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.472 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22741 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 245760~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:00.472 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141107+0000", 2024-09-21T14:59:00.472 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.292566260999999, 2024-09-21T14:59:00.472 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036407729, 2024-09-21T14:59:00.472 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:00.472 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:00.472 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:00.473 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:00.473 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:00.473 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22741 2024-09-21T14:59:00.473 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.473 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:00.473 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.473 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:00.473 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141107+0000", 2024-09-21T14:59:00.473 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:00.473 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.473 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.473 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:00.473 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141107+0000", 2024-09-21T14:59:00.473 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:00.474 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.474 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.474 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:00.474 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141107+0000", 2024-09-21T14:59:00.474 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:00.474 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.474 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.474 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:00.474 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141109+0000", 2024-09-21T14:59:00.474 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6529999999999998e-06 2024-09-21T14:59:00.474 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.474 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.474 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:00.474 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141115+0000", 2024-09-21T14:59:00.475 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9649999999999996e-06 2024-09-21T14:59:00.475 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.475 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.475 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:00.475 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141121+0000", 2024-09-21T14:59:00.475 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1209999999999998e-06 2024-09-21T14:59:00.475 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.475 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.475 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:00.475 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177242+0000", 2024-09-21T14:59:00.475 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036121706000000003 2024-09-21T14:59:00.475 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.475 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.475 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:00.476 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177301+0000", 2024-09-21T14:59:00.476 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9008000000000002e-05 2024-09-21T14:59:00.476 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.476 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.476 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:00.476 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177515+0000", 2024-09-21T14:59:00.476 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00021362 2024-09-21T14:59:00.476 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:00.476 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:00.476 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:00.476 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.476 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.476 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22742 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 311296~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:00.476 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141126+0000", 2024-09-21T14:59:00.477 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.292547906999999, 2024-09-21T14:59:00.477 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036653537, 2024-09-21T14:59:00.477 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:00.477 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:00.477 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:00.477 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:00.477 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:00.477 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22742 2024-09-21T14:59:00.477 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.477 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:00.477 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.477 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:00.477 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141126+0000", 2024-09-21T14:59:00.477 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:00.477 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.478 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.478 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:00.478 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141126+0000", 2024-09-21T14:59:00.478 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:00.478 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.478 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.478 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:00.478 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141125+0000", 2024-09-21T14:59:00.478 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:00.478 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.478 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.478 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:00.478 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141128+0000", 2024-09-21T14:59:00.478 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.886e-06 2024-09-21T14:59:00.478 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.479 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.479 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:00.479 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141137+0000", 2024-09-21T14:59:00.479 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4399999999999994e-06 2024-09-21T14:59:00.479 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.479 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.479 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:00.479 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141143+0000", 2024-09-21T14:59:00.479 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1410000000000004e-06 2024-09-21T14:59:00.479 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.479 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.479 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:00.479 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177543+0000", 2024-09-21T14:59:00.479 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036400544999999999 2024-09-21T14:59:00.480 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.480 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.480 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:00.480 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177605+0000", 2024-09-21T14:59:00.480 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2206999999999996e-05 2024-09-21T14:59:00.480 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.480 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.480 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:00.480 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177779+0000", 2024-09-21T14:59:00.480 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017368099999999999 2024-09-21T14:59:00.480 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:00.480 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:00.480 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:00.480 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.480 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.480 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22743 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 376832~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:00.481 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141147+0000", 2024-09-21T14:59:00.481 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.292526423, 2024-09-21T14:59:00.481 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036915158000000003, 2024-09-21T14:59:00.481 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:00.481 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:00.481 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:00.481 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:00.481 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:00.481 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22743 2024-09-21T14:59:00.481 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.481 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:00.481 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.481 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:00.481 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141147+0000", 2024-09-21T14:59:00.482 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:00.482 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.482 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.482 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:00.482 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141147+0000", 2024-09-21T14:59:00.482 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:00.482 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.482 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.482 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:00.482 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141147+0000", 2024-09-21T14:59:00.482 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:00.482 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.482 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.482 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:00.483 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141149+0000", 2024-09-21T14:59:00.483 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.79e-06 2024-09-21T14:59:00.483 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.483 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.483 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:00.483 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141155+0000", 2024-09-21T14:59:00.483 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6699999999999999e-06 2024-09-21T14:59:00.483 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.483 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.483 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:00.483 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141160+0000", 2024-09-21T14:59:00.483 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5759999999999999e-06 2024-09-21T14:59:00.483 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.483 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.483 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:00.484 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177817+0000", 2024-09-21T14:59:00.484 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036657404999999997 2024-09-21T14:59:00.484 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.484 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.484 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:00.484 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177892+0000", 2024-09-21T14:59:00.484 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4602000000000002e-05 2024-09-21T14:59:00.484 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.484 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.484 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:00.484 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178062+0000", 2024-09-21T14:59:00.484 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000170464 2024-09-21T14:59:00.484 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:00.484 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:00.484 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:00.485 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.485 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.485 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22744 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 442368~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:00.485 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141164+0000", 2024-09-21T14:59:00.485 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.29250929, 2024-09-21T14:59:00.485 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037120806999999999, 2024-09-21T14:59:00.485 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:00.485 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:00.485 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:00.485 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:00.485 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:00.485 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22744 2024-09-21T14:59:00.485 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.485 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:00.486 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.486 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:00.486 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141164+0000", 2024-09-21T14:59:00.486 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:00.486 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.486 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.486 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:00.486 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141164+0000", 2024-09-21T14:59:00.486 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:00.486 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.486 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.486 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:00.486 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141164+0000", 2024-09-21T14:59:00.486 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:00.486 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.487 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.487 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:00.487 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141167+0000", 2024-09-21T14:59:00.487 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7159999999999999e-06 2024-09-21T14:59:00.487 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.487 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.487 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:00.487 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141173+0000", 2024-09-21T14:59:00.487 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.071e-06 2024-09-21T14:59:00.487 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.487 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.487 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:00.487 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141177+0000", 2024-09-21T14:59:00.487 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3610000000000003e-06 2024-09-21T14:59:00.488 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.488 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.488 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:00.488 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178083+0000", 2024-09-21T14:59:00.488 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036906405000000003 2024-09-21T14:59:00.488 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.488 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.488 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:00.488 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178143+0000", 2024-09-21T14:59:00.488 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0056999999999998e-05 2024-09-21T14:59:00.488 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.488 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.488 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:00.488 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178285+0000", 2024-09-21T14:59:00.488 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000141555 2024-09-21T14:59:00.489 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:00.489 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:00.489 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:00.489 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.489 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.489 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22745 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 507904~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:00.489 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141182+0000", 2024-09-21T14:59:00.489 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.292490969999999, 2024-09-21T14:59:00.489 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037332331000000003, 2024-09-21T14:59:00.489 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:00.489 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:00.489 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:00.489 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:00.489 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:00.489 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22745 2024-09-21T14:59:00.490 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.490 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:00.490 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.490 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:00.490 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141182+0000", 2024-09-21T14:59:00.490 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:00.490 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.490 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.490 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:00.490 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141182+0000", 2024-09-21T14:59:00.490 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:00.490 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.490 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.490 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:00.491 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141182+0000", 2024-09-21T14:59:00.491 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:00.491 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.491 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.491 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:00.491 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141185+0000", 2024-09-21T14:59:00.491 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.464e-06 2024-09-21T14:59:00.491 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.491 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.491 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:00.491 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141200+0000", 2024-09-21T14:59:00.491 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5886999999999999e-05 2024-09-21T14:59:00.491 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.491 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.492 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:00.492 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141571+0000", 2024-09-21T14:59:00.492 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000370909 2024-09-21T14:59:00.492 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.492 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.492 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:00.492 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178318+0000", 2024-09-21T14:59:00.492 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036746244999999997 2024-09-21T14:59:00.492 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.492 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.492 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:00.492 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178367+0000", 2024-09-21T14:59:00.492 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9317000000000002e-05 2024-09-21T14:59:00.492 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.493 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.493 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:00.493 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178515+0000", 2024-09-21T14:59:00.493 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014784600000000001 2024-09-21T14:59:00.493 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:00.493 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:00.493 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:00.493 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.493 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.493 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22746 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 573440~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:00.493 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141599+0000", 2024-09-21T14:59:00.493 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.292074338000001, 2024-09-21T14:59:00.493 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037402249999999998, 2024-09-21T14:59:00.493 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:00.493 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:00.494 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:00.494 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:00.494 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:00.494 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22746 2024-09-21T14:59:00.494 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.494 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:00.494 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.494 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:00.494 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141599+0000", 2024-09-21T14:59:00.494 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:00.494 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.494 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.494 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:00.494 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141599+0000", 2024-09-21T14:59:00.495 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:00.495 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.495 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.495 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:00.495 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141598+0000", 2024-09-21T14:59:00.495 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:59:00.495 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.495 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.495 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:00.495 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141637+0000", 2024-09-21T14:59:00.495 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.9119999999999998e-05 2024-09-21T14:59:00.495 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.495 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.496 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:00.496 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141647+0000", 2024-09-21T14:59:00.496 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5570000000000001e-06 2024-09-21T14:59:00.496 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.496 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.496 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:00.496 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141654+0000", 2024-09-21T14:59:00.496 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2049999999999996e-06 2024-09-21T14:59:00.496 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.496 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.496 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:00.496 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178553+0000", 2024-09-21T14:59:00.496 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036898528 2024-09-21T14:59:00.496 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.496 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.497 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:00.497 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178625+0000", 2024-09-21T14:59:00.497 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2669999999999994e-05 2024-09-21T14:59:00.497 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.497 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.497 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:00.497 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179001+0000", 2024-09-21T14:59:00.497 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000375977 2024-09-21T14:59:00.497 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:00.497 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:00.497 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:00.497 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.497 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.497 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22747 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 638976~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:00.498 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141660+0000", 2024-09-21T14:59:00.498 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.292013195999999, 2024-09-21T14:59:00.498 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037617352999999999, 2024-09-21T14:59:00.498 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:00.498 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:00.498 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:00.498 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:00.498 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:00.498 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22747 2024-09-21T14:59:00.498 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.498 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:00.498 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.498 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:00.498 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141660+0000", 2024-09-21T14:59:00.499 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:00.499 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.499 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.499 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:00.499 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141660+0000", 2024-09-21T14:59:00.499 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:00.499 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.499 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.499 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:00.499 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141660+0000", 2024-09-21T14:59:00.499 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:00.499 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.499 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.500 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:00.500 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141663+0000", 2024-09-21T14:59:00.500 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.569e-06 2024-09-21T14:59:00.500 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.500 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.500 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:00.500 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141670+0000", 2024-09-21T14:59:00.500 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5719999999999999e-06 2024-09-21T14:59:00.500 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.500 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.500 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:00.500 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141675+0000", 2024-09-21T14:59:00.500 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2889999999999997e-06 2024-09-21T14:59:00.500 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.501 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.501 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:00.501 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179028+0000", 2024-09-21T14:59:00.501 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037353381999999997 2024-09-21T14:59:00.501 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.501 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.501 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:00.501 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179099+0000", 2024-09-21T14:59:00.501 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0259000000000006e-05 2024-09-21T14:59:00.501 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.501 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.501 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:00.501 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179278+0000", 2024-09-21T14:59:00.501 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000178908 2024-09-21T14:59:00.501 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:00.502 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:00.502 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:00.502 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.502 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.502 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22748 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 704512~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:00.502 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141680+0000", 2024-09-21T14:59:00.502 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.291993558, 2024-09-21T14:59:00.502 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037806166000000002, 2024-09-21T14:59:00.502 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:00.502 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:00.502 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:00.502 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:00.502 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:00.503 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22748 2024-09-21T14:59:00.503 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.503 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:00.503 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.503 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:00.503 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141680+0000", 2024-09-21T14:59:00.503 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:00.503 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.503 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.503 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:00.503 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141680+0000", 2024-09-21T14:59:00.503 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:00.503 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.503 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.504 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:00.504 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141679+0000", 2024-09-21T14:59:00.504 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:00.504 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.504 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.504 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:00.504 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141683+0000", 2024-09-21T14:59:00.504 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3230000000000002e-06 2024-09-21T14:59:00.504 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.504 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.504 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:00.504 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141690+0000", 2024-09-21T14:59:00.504 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8680000000000001e-06 2024-09-21T14:59:00.504 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.504 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.505 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:00.505 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141694+0000", 2024-09-21T14:59:00.505 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6770000000000003e-06 2024-09-21T14:59:00.505 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.505 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.505 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:00.505 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179300+0000", 2024-09-21T14:59:00.505 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037605567999999999 2024-09-21T14:59:00.505 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.505 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.505 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:00.505 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179355+0000", 2024-09-21T14:59:00.505 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4771000000000001e-05 2024-09-21T14:59:00.505 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.506 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.506 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:00.506 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179486+0000", 2024-09-21T14:59:00.506 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013141699999999999 2024-09-21T14:59:00.506 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:00.506 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:00.506 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:00.506 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.506 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.506 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22749 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 770048~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:00.506 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141698+0000", 2024-09-21T14:59:00.506 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.291975186, 2024-09-21T14:59:00.506 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037967813000000003, 2024-09-21T14:59:00.506 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:00.506 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:00.507 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:00.507 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:00.507 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:00.507 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22749 2024-09-21T14:59:00.507 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.507 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:00.507 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.507 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:00.507 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141698+0000", 2024-09-21T14:59:00.507 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:00.507 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.507 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.507 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:00.507 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141698+0000", 2024-09-21T14:59:00.508 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:00.508 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.508 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.508 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:00.508 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141698+0000", 2024-09-21T14:59:00.508 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:00.508 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.508 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.508 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:00.508 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141701+0000", 2024-09-21T14:59:00.508 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8399999999999999e-06 2024-09-21T14:59:00.508 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.508 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.508 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:00.508 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141706+0000", 2024-09-21T14:59:00.509 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4979999999999997e-06 2024-09-21T14:59:00.509 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.509 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.509 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:00.509 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141711+0000", 2024-09-21T14:59:00.509 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3850000000000004e-06 2024-09-21T14:59:00.509 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.509 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.509 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:00.509 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179504+0000", 2024-09-21T14:59:00.509 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037793514 2024-09-21T14:59:00.509 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.509 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.509 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:00.509 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179546+0000", 2024-09-21T14:59:00.510 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1467999999999997e-05 2024-09-21T14:59:00.510 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.510 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.510 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:00.510 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179666+0000", 2024-09-21T14:59:00.510 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000120499 2024-09-21T14:59:00.510 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:00.510 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:00.510 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:00.510 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.510 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.510 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22750 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 835584~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:00.510 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141715+0000", 2024-09-21T14:59:00.510 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.291958885, 2024-09-21T14:59:00.511 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038173179000000002, 2024-09-21T14:59:00.511 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:00.511 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:00.511 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:00.511 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:00.511 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:00.511 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22750 2024-09-21T14:59:00.511 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.511 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:00.511 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.511 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:00.511 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141715+0000", 2024-09-21T14:59:00.511 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:00.511 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.512 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.512 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:00.512 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141715+0000", 2024-09-21T14:59:00.512 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:00.512 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.512 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.512 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:00.512 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141714+0000", 2024-09-21T14:59:00.512 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:00.512 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.512 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.512 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:00.512 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141716+0000", 2024-09-21T14:59:00.512 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1730000000000002e-06 2024-09-21T14:59:00.513 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.513 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.513 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:00.513 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141722+0000", 2024-09-21T14:59:00.513 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.417e-06 2024-09-21T14:59:00.513 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.513 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.513 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:00.513 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141726+0000", 2024-09-21T14:59:00.513 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4830000000000001e-06 2024-09-21T14:59:00.513 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.513 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.513 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:00.514 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179683+0000", 2024-09-21T14:59:00.514 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037956977000000003 2024-09-21T14:59:00.514 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.514 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.514 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:00.514 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179751+0000", 2024-09-21T14:59:00.514 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7762000000000006e-05 2024-09-21T14:59:00.514 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.514 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.514 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:00.514 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179888+0000", 2024-09-21T14:59:00.514 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000136635 2024-09-21T14:59:00.514 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:00.515 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:00.515 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:00.515 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.515 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.515 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22751 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 901120~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:00.515 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141730+0000", 2024-09-21T14:59:00.515 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.291943086, 2024-09-21T14:59:00.515 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038357011000000003, 2024-09-21T14:59:00.515 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:00.515 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:00.515 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:00.515 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:00.515 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:00.516 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22751 2024-09-21T14:59:00.516 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.516 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:00.516 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.516 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:00.516 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141730+0000", 2024-09-21T14:59:00.516 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:00.516 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.516 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.516 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:00.516 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141730+0000", 2024-09-21T14:59:00.516 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:00.516 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.516 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.516 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:00.517 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141730+0000", 2024-09-21T14:59:00.517 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:00.517 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.517 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.517 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:00.517 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141737+0000", 2024-09-21T14:59:00.517 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.64e-06 2024-09-21T14:59:00.517 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.517 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.517 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:00.517 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141743+0000", 2024-09-21T14:59:00.517 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8130000000000001e-06 2024-09-21T14:59:00.517 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.517 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.518 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:00.518 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141747+0000", 2024-09-21T14:59:00.518 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4920000000000004e-06 2024-09-21T14:59:00.518 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.518 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.518 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:00.518 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179918+0000", 2024-09-21T14:59:00.518 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038170822 2024-09-21T14:59:00.518 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.518 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.518 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:00.518 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179960+0000", 2024-09-21T14:59:00.518 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1999000000000002e-05 2024-09-21T14:59:00.518 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.519 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.519 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:00.519 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180087+0000", 2024-09-21T14:59:00.519 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012752799999999999 2024-09-21T14:59:00.519 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:00.519 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:00.519 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:00.519 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.519 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.519 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22752 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 966656~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:00.519 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141751+0000", 2024-09-21T14:59:00.519 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.291922766000001, 2024-09-21T14:59:00.519 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038553084000000001, 2024-09-21T14:59:00.520 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:00.520 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:00.520 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:00.520 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:00.520 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:00.520 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22752 2024-09-21T14:59:00.520 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.520 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:00.520 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.520 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:00.520 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141751+0000", 2024-09-21T14:59:00.520 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:00.520 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.520 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.521 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:00.521 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141751+0000", 2024-09-21T14:59:00.521 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:00.521 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.521 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.521 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:00.521 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141750+0000", 2024-09-21T14:59:00.521 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:00.521 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.521 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.521 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:00.521 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141753+0000", 2024-09-21T14:59:00.521 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5900000000000002e-06 2024-09-21T14:59:00.521 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.522 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.522 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:00.522 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141759+0000", 2024-09-21T14:59:00.522 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.947e-06 2024-09-21T14:59:00.522 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.522 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.522 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:00.522 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141764+0000", 2024-09-21T14:59:00.522 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8319999999999996e-06 2024-09-21T14:59:00.522 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.522 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.522 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:00.522 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180108+0000", 2024-09-21T14:59:00.522 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038344095000000002 2024-09-21T14:59:00.523 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.523 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.523 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:00.523 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180168+0000", 2024-09-21T14:59:00.523 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0047000000000003e-05 2024-09-21T14:59:00.523 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.523 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.523 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:00.523 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180304+0000", 2024-09-21T14:59:00.523 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013592500000000001 2024-09-21T14:59:00.523 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:00.523 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:00.523 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:00.523 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.524 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.524 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22753 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 1032192~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:00.524 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141768+0000", 2024-09-21T14:59:00.524 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.291905892000001, 2024-09-21T14:59:00.524 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038818810000000002, 2024-09-21T14:59:00.524 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:00.524 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:00.524 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:00.524 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:00.524 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:00.524 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22753 2024-09-21T14:59:00.524 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.524 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:00.525 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.525 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:00.525 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141768+0000", 2024-09-21T14:59:00.525 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:00.525 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.525 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.525 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:00.525 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141768+0000", 2024-09-21T14:59:00.525 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:00.525 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.525 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.525 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:00.525 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141767+0000", 2024-09-21T14:59:00.525 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:00.526 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.526 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.526 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:00.526 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141770+0000", 2024-09-21T14:59:00.526 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3329999999999999e-06 2024-09-21T14:59:00.526 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.526 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.526 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:00.526 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141775+0000", 2024-09-21T14:59:00.526 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2669999999999997e-06 2024-09-21T14:59:00.526 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.526 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.526 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:00.526 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141779+0000", 2024-09-21T14:59:00.527 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2259999999999999e-06 2024-09-21T14:59:00.527 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.527 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.527 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:00.527 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180325+0000", 2024-09-21T14:59:00.527 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038546428000000001 2024-09-21T14:59:00.527 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.527 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.527 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:00.527 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180401+0000", 2024-09-21T14:59:00.527 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5437000000000003e-05 2024-09-21T14:59:00.527 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:00.527 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:00.527 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:00.528 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180586+0000", 2024-09-21T14:59:00.528 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000185429 2024-09-21T14:59:00.528 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:00.528 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:00.528 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:00.528 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:00.528 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:00.528 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:00.529 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T14:59:00.530 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T14:59:00.530 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T14:59:00.530 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T14:59:00.531 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:59:00.528+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:59:00.537 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T14:59:00.561 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-21T14:59:00.584 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T14:59:00.608 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T14:59:00.630 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:59:00.629+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:59:00.632 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T14:59:00.634 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:59:00.634 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:59:00.634 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:59:00.635 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:59:00.635 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:59:00.635 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:59:00.635 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:59:00.635 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:59:00.649 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:00.649 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:00.659 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T14:59:00.731 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:59:00.728+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:59:00.762 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:00.763 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:00.772 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T14:59:00.833 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:59:00.829+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:59:00.877 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:00.877 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:00.888 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T14:59:00.934 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:59:00.932+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:59:00.993 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:00.993 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:01.003 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T14:59:01.034 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:59:01.033+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:59:01.135 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:59:01.132+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:59:01.154 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:01.154 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:01.164 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T14:59:01.236 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:59:01.233+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:59:01.273 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:01.274 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:01.283 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T14:59:01.284 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T14:59:01.337 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:59:01.337+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:59:01.390 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:01.391 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:01.400 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T14:59:01.438 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:59:01.436+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:59:01.506 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:01.506 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:01.515 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T14:59:01.557 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:59:01.556+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:59:01.640 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:59:01.637+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:59:01.668 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:01.668 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:01.678 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T14:59:01.741 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:59:01.741+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:59:01.789 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:01.789 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:01.799 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T14:59:01.841 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:59:01.840+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:59:01.907 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:01.908 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:01.917 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T14:59:01.943 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:59:01.941+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:59:02.025 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:02.025 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:02.034 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T14:59:02.081 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:59:02.081+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:59:02.144 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:59:02.141+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:59:02.149 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:02.149 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:02.160 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T14:59:02.246 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:59:02.245+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:59:02.269 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:02.269 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:02.278 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T14:59:02.295 INFO:teuthology.orchestra.run.smithi032.stderr:nodeep-scrub is unset 2024-09-21T14:59:02.347 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:59:02.345+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:59:02.412 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:02.413 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:02.424 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T14:59:02.465 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:59:02.465+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:59:02.548 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:59:02.545+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:59:02.566 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:02.566 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:02.576 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T14:59:02.649 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:59:02.649+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:59:02.652 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T14:59:02.704 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:02.705 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:02.721 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T14:59:02.749 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:59:02.748+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:59:02.850 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T14:59:02.850 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:59:02.848+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:59:02.869 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T14:59:02.874 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:02.875 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T14:59:02.875 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T14:59:02.875 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T14:59:02.875 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.875 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T14:59:02.875 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.205754+0000", 2024-09-21T14:59:02.875 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 59.66810795, 2024-09-21T14:59:02.875 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.41117094199999998, 2024-09-21T14:59:02.875 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:02.875 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:02.875 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:02.875 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:59:02.875 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:59:02.876 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1 2024-09-21T14:59:02.876 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.876 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:02.876 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.876 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:02.876 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205754+0000", 2024-09-21T14:59:02.876 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:02.876 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.876 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.876 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:02.876 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205754+0000", 2024-09-21T14:59:02.876 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:02.876 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.876 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.877 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:02.877 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205752+0000", 2024-09-21T14:59:02.877 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T14:59:02.877 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.877 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.877 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:02.877 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205765+0000", 2024-09-21T14:59:02.877 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3230999999999999e-05 2024-09-21T14:59:02.877 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.877 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.877 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:02.877 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205782+0000", 2024-09-21T14:59:02.877 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6634e-05 2024-09-21T14:59:02.878 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.878 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.878 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:02.878 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205793+0000", 2024-09-21T14:59:02.878 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1297e-05 2024-09-21T14:59:02.878 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.878 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.878 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:02.878 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205867+0000", 2024-09-21T14:59:02.878 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4269000000000003e-05 2024-09-21T14:59:02.878 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.878 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.878 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T14:59:02.878 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205877+0000", 2024-09-21T14:59:02.879 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2690000000000005e-06 2024-09-21T14:59:02.879 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.879 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.879 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:02.879 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.613700+0000", 2024-09-21T14:59:02.879 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.407823455 2024-09-21T14:59:02.879 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.879 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.879 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:02.879 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.614203+0000", 2024-09-21T14:59:02.879 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00050311500000000005 2024-09-21T14:59:02.879 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.879 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.879 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:59:02.880 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.614750+0000", 2024-09-21T14:59:02.880 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00054738699999999998 2024-09-21T14:59:02.880 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.880 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.880 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:02.880 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616367+0000", 2024-09-21T14:59:02.880 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001616634 2024-09-21T14:59:02.880 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.880 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.880 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:02.880 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616574+0000", 2024-09-21T14:59:02.880 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00020668499999999999 2024-09-21T14:59:02.880 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.880 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.880 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:02.881 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616802+0000", 2024-09-21T14:59:02.881 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00022817900000000001 2024-09-21T14:59:02.881 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.881 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.881 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:02.881 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616925+0000", 2024-09-21T14:59:02.881 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012309699999999999 2024-09-21T14:59:02.881 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:02.881 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:02.881 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:02.881 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.881 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.881 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T14:59:02.881 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.955142+0000", 2024-09-21T14:59:02.881 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 58.918720303000001, 2024-09-21T14:59:02.882 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012388923, 2024-09-21T14:59:02.882 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:02.882 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:02.882 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:02.882 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:59:02.882 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:59:02.882 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 113 2024-09-21T14:59:02.882 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.882 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:02.882 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.882 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:02.882 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955142+0000", 2024-09-21T14:59:02.883 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:02.883 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.883 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.883 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:02.883 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955142+0000", 2024-09-21T14:59:02.883 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:02.883 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.883 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.883 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:02.883 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955141+0000", 2024-09-21T14:59:02.883 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:02.883 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.883 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.883 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:02.884 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955145+0000", 2024-09-21T14:59:02.884 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2109999999999998e-06 2024-09-21T14:59:02.884 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.884 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.884 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:02.884 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955150+0000", 2024-09-21T14:59:02.884 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4090000000000002e-06 2024-09-21T14:59:02.884 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.884 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.884 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:02.884 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955153+0000", 2024-09-21T14:59:02.884 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1240000000000001e-06 2024-09-21T14:59:02.884 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.884 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.885 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:02.885 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.963026+0000", 2024-09-21T14:59:02.885 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0078724060000000002 2024-09-21T14:59:02.885 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.885 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.885 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:02.885 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.963126+0000", 2024-09-21T14:59:02.885 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000100206 2024-09-21T14:59:02.885 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.885 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.885 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:59:02.885 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.963277+0000", 2024-09-21T14:59:02.885 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015159200000000001 2024-09-21T14:59:02.885 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.885 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.886 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:02.886 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.965519+0000", 2024-09-21T14:59:02.886 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0022418440000000002 2024-09-21T14:59:02.886 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.886 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.886 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:02.886 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967305+0000", 2024-09-21T14:59:02.886 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001785525 2024-09-21T14:59:02.886 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.886 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.886 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:02.886 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967438+0000", 2024-09-21T14:59:02.886 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013284399999999999 2024-09-21T14:59:02.887 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.887 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.887 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:02.887 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967531+0000", 2024-09-21T14:59:02.887 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.3140000000000006e-05 2024-09-21T14:59:02.887 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:02.887 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:02.887 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:02.887 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.887 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.887 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T14:59:02.887 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.955387+0000", 2024-09-21T14:59:02.887 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 58.918475368999999, 2024-09-21T14:59:02.888 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012319478999999999, 2024-09-21T14:59:02.888 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:02.888 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:02.888 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:02.888 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:59:02.888 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:59:02.888 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 114 2024-09-21T14:59:02.888 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.888 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:02.888 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.888 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:02.888 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955387+0000", 2024-09-21T14:59:02.888 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:02.888 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.889 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.889 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:02.889 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955387+0000", 2024-09-21T14:59:02.889 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:02.889 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.889 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.889 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:02.889 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955386+0000", 2024-09-21T14:59:02.889 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:59:02.889 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.889 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.889 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:02.889 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955392+0000", 2024-09-21T14:59:02.889 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4740000000000002e-06 2024-09-21T14:59:02.889 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.890 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.890 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:02.890 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955400+0000", 2024-09-21T14:59:02.890 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2139999999999996e-06 2024-09-21T14:59:02.890 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.890 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.890 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:02.890 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955407+0000", 2024-09-21T14:59:02.890 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2299999999999996e-06 2024-09-21T14:59:02.890 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.890 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.890 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:02.890 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.964840+0000", 2024-09-21T14:59:02.891 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0094330079999999997 2024-09-21T14:59:02.891 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.891 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.891 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:02.891 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.964917+0000", 2024-09-21T14:59:02.891 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7071999999999997e-05 2024-09-21T14:59:02.891 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.891 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.891 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:59:02.891 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.965026+0000", 2024-09-21T14:59:02.891 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000109646 2024-09-21T14:59:02.891 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.891 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.891 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:02.892 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967609+0000", 2024-09-21T14:59:02.892 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0025824929999999999 2024-09-21T14:59:02.892 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.892 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.892 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:02.892 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967629+0000", 2024-09-21T14:59:02.892 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0486999999999999e-05 2024-09-21T14:59:02.892 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.892 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.892 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:02.892 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967672+0000", 2024-09-21T14:59:02.892 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2314999999999997e-05 2024-09-21T14:59:02.892 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.892 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.893 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:02.893 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967706+0000", 2024-09-21T14:59:02.893 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4483999999999997e-05 2024-09-21T14:59:02.893 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:02.893 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:02.893 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:02.893 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.893 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.893 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T14:59:02.893 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.955609+0000", 2024-09-21T14:59:02.893 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 58.918253563, 2024-09-21T14:59:02.893 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013624153999999999, 2024-09-21T14:59:02.893 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:02.894 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:02.894 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:02.894 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:59:02.894 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:59:02.894 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 115 2024-09-21T14:59:02.894 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.894 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:02.894 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.894 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:02.894 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955609+0000", 2024-09-21T14:59:02.894 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:02.894 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.894 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.895 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:02.895 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955609+0000", 2024-09-21T14:59:02.895 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:02.895 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.895 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.895 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:02.895 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955607+0000", 2024-09-21T14:59:02.895 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-21T14:59:02.895 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.895 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.895 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:02.895 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955732+0000", 2024-09-21T14:59:02.895 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000125566 2024-09-21T14:59:02.895 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.896 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.896 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:02.896 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955752+0000", 2024-09-21T14:59:02.896 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9916999999999999e-05 2024-09-21T14:59:02.896 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.896 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.896 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:02.896 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955767+0000", 2024-09-21T14:59:02.896 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4687e-05 2024-09-21T14:59:02.896 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.896 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.896 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:02.896 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.965574+0000", 2024-09-21T14:59:02.897 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0098072850000000007 2024-09-21T14:59:02.897 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.897 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.897 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:02.897 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.965642+0000", 2024-09-21T14:59:02.897 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7784e-05 2024-09-21T14:59:02.897 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.897 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.897 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:59:02.897 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.965753+0000", 2024-09-21T14:59:02.897 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000111111 2024-09-21T14:59:02.897 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.897 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.897 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:02.898 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967717+0000", 2024-09-21T14:59:02.898 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0019635339999999999 2024-09-21T14:59:02.898 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.898 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.898 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:02.898 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.969014+0000", 2024-09-21T14:59:02.898 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001297093 2024-09-21T14:59:02.898 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.898 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.898 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:02.898 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.969130+0000", 2024-09-21T14:59:02.898 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000116011 2024-09-21T14:59:02.898 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.898 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.899 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:02.899 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.969233+0000", 2024-09-21T14:59:02.899 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000102982 2024-09-21T14:59:02.899 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:02.899 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:02.899 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:02.899 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.899 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.899 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T14:59:02.899 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.955783+0000", 2024-09-21T14:59:02.899 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 58.918078893999997, 2024-09-21T14:59:02.899 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014630241, 2024-09-21T14:59:02.899 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:02.899 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:02.900 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:02.900 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:59:02.900 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:59:02.900 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 116 2024-09-21T14:59:02.900 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.900 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:02.900 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.900 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:02.900 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955783+0000", 2024-09-21T14:59:02.900 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:02.900 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.900 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.900 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:02.901 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955783+0000", 2024-09-21T14:59:02.901 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:02.901 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.901 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.901 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:02.901 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955782+0000", 2024-09-21T14:59:02.901 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:59:02.901 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.901 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.901 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:02.901 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955791+0000", 2024-09-21T14:59:02.901 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8089999999999999e-06 2024-09-21T14:59:02.901 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.901 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.902 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:02.902 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955804+0000", 2024-09-21T14:59:02.902 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2548000000000001e-05 2024-09-21T14:59:02.902 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.902 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.902 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:02.902 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955814+0000", 2024-09-21T14:59:02.902 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0158e-05 2024-09-21T14:59:02.902 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.902 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.902 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:02.902 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.966515+0000", 2024-09-21T14:59:02.902 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.010701756 2024-09-21T14:59:02.903 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.903 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.903 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:02.903 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.966606+0000", 2024-09-21T14:59:02.903 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0761999999999996e-05 2024-09-21T14:59:02.903 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.903 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.903 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:59:02.903 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.966740+0000", 2024-09-21T14:59:02.903 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000133693 2024-09-21T14:59:02.903 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.903 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.903 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:02.903 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.968920+0000", 2024-09-21T14:59:02.904 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0021799290000000002 2024-09-21T14:59:02.904 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.904 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.904 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:02.904 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.970320+0000", 2024-09-21T14:59:02.904 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0014001759999999999 2024-09-21T14:59:02.904 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.904 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.904 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:02.904 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.970376+0000", 2024-09-21T14:59:02.904 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5696999999999999e-05 2024-09-21T14:59:02.904 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.904 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.904 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:02.905 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.970413+0000", 2024-09-21T14:59:02.905 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7750999999999998e-05 2024-09-21T14:59:02.905 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:02.905 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:02.905 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:02.905 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.905 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.905 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T14:59:02.905 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.955829+0000", 2024-09-21T14:59:02.905 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 58.918033424000001, 2024-09-21T14:59:02.905 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014919597999999999, 2024-09-21T14:59:02.905 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:02.905 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:02.905 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:02.906 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:59:02.906 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:59:02.906 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 117 2024-09-21T14:59:02.906 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.906 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:02.906 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.906 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:02.906 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955829+0000", 2024-09-21T14:59:02.906 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:02.906 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.906 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.906 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:02.906 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955829+0000", 2024-09-21T14:59:02.907 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:02.907 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.907 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.907 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:02.907 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955827+0000", 2024-09-21T14:59:02.907 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T14:59:02.907 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.907 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.907 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:02.907 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955838+0000", 2024-09-21T14:59:02.907 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.111e-05 2024-09-21T14:59:02.907 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.907 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.908 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:02.908 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955875+0000", 2024-09-21T14:59:02.908 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6211999999999998e-05 2024-09-21T14:59:02.908 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.908 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.908 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:02.908 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955899+0000", 2024-09-21T14:59:02.908 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4128999999999999e-05 2024-09-21T14:59:02.908 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.908 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.908 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:02.908 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967767+0000", 2024-09-21T14:59:02.908 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.011868251999999999 2024-09-21T14:59:02.908 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.909 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.909 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:02.909 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967866+0000", 2024-09-21T14:59:02.909 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8664000000000003e-05 2024-09-21T14:59:02.909 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.909 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.909 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:59:02.909 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.968128+0000", 2024-09-21T14:59:02.909 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00026243600000000001 2024-09-21T14:59:02.909 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.909 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.909 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:02.909 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.969393+0000", 2024-09-21T14:59:02.909 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0012650350000000001 2024-09-21T14:59:02.910 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.910 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.910 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:02.910 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.970523+0000", 2024-09-21T14:59:02.910 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001129557 2024-09-21T14:59:02.910 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.910 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.910 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:02.910 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.970603+0000", 2024-09-21T14:59:02.910 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0090000000000001e-05 2024-09-21T14:59:02.910 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.910 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.910 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:02.911 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.970748+0000", 2024-09-21T14:59:02.911 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014552799999999999 2024-09-21T14:59:02.911 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:02.911 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:02.911 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:02.911 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.911 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.911 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T14:59:02.911 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:04.493315+0000", 2024-09-21T14:59:02.911 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 58.380546590999998, 2024-09-21T14:59:02.911 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014055766000000001, 2024-09-21T14:59:02.911 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:02.911 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:02.911 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:02.912 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:59:02.912 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:59:02.912 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 137 2024-09-21T14:59:02.912 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.912 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:02.912 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.912 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:02.912 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493315+0000", 2024-09-21T14:59:02.912 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:02.912 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.912 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.912 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:02.912 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493315+0000", 2024-09-21T14:59:02.913 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:02.913 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.913 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.913 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:02.913 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493314+0000", 2024-09-21T14:59:02.913 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:59:02.913 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.913 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.913 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:02.913 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493507+0000", 2024-09-21T14:59:02.913 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000192444 2024-09-21T14:59:02.913 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.913 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.913 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:02.914 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493528+0000", 2024-09-21T14:59:02.914 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0845000000000001e-05 2024-09-21T14:59:02.914 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.914 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.914 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:02.914 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493543+0000", 2024-09-21T14:59:02.914 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5800000000000001e-05 2024-09-21T14:59:02.914 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.914 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.914 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:02.914 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.502575+0000", 2024-09-21T14:59:02.914 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0090312389999999999 2024-09-21T14:59:02.914 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.915 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.915 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:02.915 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.502677+0000", 2024-09-21T14:59:02.915 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000102271 2024-09-21T14:59:02.915 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.915 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.915 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:59:02.915 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.502825+0000", 2024-09-21T14:59:02.915 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000148171 2024-09-21T14:59:02.915 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.915 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.915 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:02.915 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507251+0000", 2024-09-21T14:59:02.915 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0044253879999999997 2024-09-21T14:59:02.916 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.916 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.916 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:02.916 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507313+0000", 2024-09-21T14:59:02.916 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2170999999999998e-05 2024-09-21T14:59:02.916 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.916 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.916 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:02.916 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507346+0000", 2024-09-21T14:59:02.916 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3079e-05 2024-09-21T14:59:02.916 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.916 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.916 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:02.916 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507371+0000", 2024-09-21T14:59:02.917 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5477999999999999e-05 2024-09-21T14:59:02.917 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:02.917 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:02.917 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:02.917 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.917 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.917 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T14:59:02.917 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:04.493570+0000", 2024-09-21T14:59:02.917 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 58.380292185999998, 2024-09-21T14:59:02.917 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014061686, 2024-09-21T14:59:02.917 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:02.917 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:02.917 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:02.917 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:59:02.918 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:59:02.918 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 138 2024-09-21T14:59:02.918 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.918 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:02.918 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.918 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:02.918 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493570+0000", 2024-09-21T14:59:02.918 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:02.918 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.918 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.918 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:02.918 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493570+0000", 2024-09-21T14:59:02.918 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:02.918 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.919 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.919 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:02.919 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493568+0000", 2024-09-21T14:59:02.919 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T14:59:02.919 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.919 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.919 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:02.919 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493579+0000", 2024-09-21T14:59:02.919 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0574000000000001e-05 2024-09-21T14:59:02.919 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.919 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.919 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:02.919 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493595+0000", 2024-09-21T14:59:02.919 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6504999999999999e-05 2024-09-21T14:59:02.920 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.920 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.920 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:02.920 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493608+0000", 2024-09-21T14:59:02.920 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2836999999999999e-05 2024-09-21T14:59:02.920 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.920 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.920 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:02.920 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.503909+0000", 2024-09-21T14:59:02.920 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.01030118 2024-09-21T14:59:02.920 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.920 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.920 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:02.920 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.504064+0000", 2024-09-21T14:59:02.920 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015493000000000001 2024-09-21T14:59:02.921 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.921 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.921 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:59:02.921 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.504299+0000", 2024-09-21T14:59:02.921 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00023416100000000001 2024-09-21T14:59:02.921 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.921 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.921 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:02.921 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507466+0000", 2024-09-21T14:59:02.921 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0031678380000000001 2024-09-21T14:59:02.921 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.921 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.921 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:02.921 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507483+0000", 2024-09-21T14:59:02.922 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6606999999999999e-05 2024-09-21T14:59:02.922 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.922 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.922 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:02.922 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507566+0000", 2024-09-21T14:59:02.922 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3110999999999998e-05 2024-09-21T14:59:02.922 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.922 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.922 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:02.922 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507632+0000", 2024-09-21T14:59:02.922 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5480000000000003e-05 2024-09-21T14:59:02.922 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:02.922 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:02.922 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:02.922 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.923 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.923 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T14:59:02.923 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:04.493625+0000", 2024-09-21T14:59:02.923 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 58.380237498, 2024-09-21T14:59:02.923 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014665378999999999, 2024-09-21T14:59:02.923 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:02.923 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:02.923 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:02.923 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:59:02.923 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:59:02.923 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139 2024-09-21T14:59:02.923 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.923 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:02.923 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.924 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:02.924 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493625+0000", 2024-09-21T14:59:02.924 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:02.924 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.924 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.924 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:02.924 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493625+0000", 2024-09-21T14:59:02.924 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:02.924 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.924 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.924 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:02.924 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493624+0000", 2024-09-21T14:59:02.924 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:59:02.924 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.924 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.925 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:02.925 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493638+0000", 2024-09-21T14:59:02.925 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4282e-05 2024-09-21T14:59:02.925 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.925 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.925 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:02.925 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493654+0000", 2024-09-21T14:59:02.925 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5739999999999998e-05 2024-09-21T14:59:02.925 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.925 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.925 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:02.925 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493666+0000", 2024-09-21T14:59:02.925 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2865e-05 2024-09-21T14:59:02.926 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.926 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.926 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:02.926 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.505329+0000", 2024-09-21T14:59:02.926 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.011662237000000001 2024-09-21T14:59:02.926 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.926 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.926 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:02.926 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.505377+0000", 2024-09-21T14:59:02.926 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7862999999999999e-05 2024-09-21T14:59:02.926 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.926 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.926 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:59:02.926 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.505455+0000", 2024-09-21T14:59:02.927 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8918000000000006e-05 2024-09-21T14:59:02.927 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.927 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.927 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:02.927 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507650+0000", 2024-09-21T14:59:02.927 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.002195027 2024-09-21T14:59:02.927 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.927 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.927 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:02.927 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508149+0000", 2024-09-21T14:59:02.927 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00049828399999999999 2024-09-21T14:59:02.927 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.927 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.927 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:02.928 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508230+0000", 2024-09-21T14:59:02.928 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1376000000000002e-05 2024-09-21T14:59:02.928 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.928 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.928 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:02.928 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508290+0000", 2024-09-21T14:59:02.928 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9852999999999998e-05 2024-09-21T14:59:02.928 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:02.928 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:02.928 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:02.928 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.928 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.928 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T14:59:02.928 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:04.493679+0000", 2024-09-21T14:59:02.929 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 58.380182599999998, 2024-09-21T14:59:02.929 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014869869000000001, 2024-09-21T14:59:02.929 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:02.929 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:02.929 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:02.929 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:59:02.929 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:59:02.929 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140 2024-09-21T14:59:02.929 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.929 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:02.929 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.929 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:02.929 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493679+0000", 2024-09-21T14:59:02.930 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:02.930 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.930 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.930 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:02.930 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493679+0000", 2024-09-21T14:59:02.930 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:02.930 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.930 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.930 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:02.930 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493679+0000", 2024-09-21T14:59:02.930 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:59:02.930 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.931 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.931 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:02.931 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493889+0000", 2024-09-21T14:59:02.931 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00021032700000000001 2024-09-21T14:59:02.931 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.931 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.931 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:02.931 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493931+0000", 2024-09-21T14:59:02.931 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1971000000000002e-05 2024-09-21T14:59:02.931 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.931 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.931 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:02.931 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493947+0000", 2024-09-21T14:59:02.931 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6385e-05 2024-09-21T14:59:02.932 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.932 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.932 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:02.932 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.505804+0000", 2024-09-21T14:59:02.932 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.011857136000000001 2024-09-21T14:59:02.932 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.932 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.932 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:02.932 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.505992+0000", 2024-09-21T14:59:02.932 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000187665 2024-09-21T14:59:02.932 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.932 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.932 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:59:02.932 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.506281+0000", 2024-09-21T14:59:02.933 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000288712 2024-09-21T14:59:02.933 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.933 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.933 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:02.933 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508349+0000", 2024-09-21T14:59:02.933 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0020685790000000001 2024-09-21T14:59:02.933 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.933 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.933 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:02.933 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508371+0000", 2024-09-21T14:59:02.933 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1747e-05 2024-09-21T14:59:02.933 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.933 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.933 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:02.934 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508450+0000", 2024-09-21T14:59:02.934 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9332000000000005e-05 2024-09-21T14:59:02.934 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.934 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.934 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:02.934 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508549+0000", 2024-09-21T14:59:02.934 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8918000000000004e-05 2024-09-21T14:59:02.934 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:02.934 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:02.934 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:02.934 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.934 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.934 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T14:59:02.935 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:04.493967+0000", 2024-09-21T14:59:02.935 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 58.379895562999998, 2024-09-21T14:59:02.935 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.015281075999999999, 2024-09-21T14:59:02.935 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:02.935 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:02.935 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:02.935 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:59:02.935 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:59:02.935 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 141 2024-09-21T14:59:02.935 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.935 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:02.935 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.935 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:02.935 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493967+0000", 2024-09-21T14:59:02.936 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:02.936 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.936 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.936 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:02.936 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493967+0000", 2024-09-21T14:59:02.936 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:02.936 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.936 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.936 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:02.936 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493965+0000", 2024-09-21T14:59:02.936 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T14:59:02.936 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.936 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.936 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:02.937 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493987+0000", 2024-09-21T14:59:02.937 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1662999999999999e-05 2024-09-21T14:59:02.937 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.937 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.937 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:02.937 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.494004+0000", 2024-09-21T14:59:02.937 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7311000000000001e-05 2024-09-21T14:59:02.937 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.937 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.937 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:02.937 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.494017+0000", 2024-09-21T14:59:02.937 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3202e-05 2024-09-21T14:59:02.937 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.937 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.938 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:02.938 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507678+0000", 2024-09-21T14:59:02.938 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013660432 2024-09-21T14:59:02.938 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.938 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.938 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:02.938 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507726+0000", 2024-09-21T14:59:02.938 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8154e-05 2024-09-21T14:59:02.938 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.938 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.938 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:59:02.938 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507797+0000", 2024-09-21T14:59:02.938 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.1110000000000002e-05 2024-09-21T14:59:02.939 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.939 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.939 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:02.939 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508710+0000", 2024-09-21T14:59:02.939 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00091339299999999997 2024-09-21T14:59:02.939 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.939 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.939 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:02.939 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.509108+0000", 2024-09-21T14:59:02.939 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00039718300000000001 2024-09-21T14:59:02.939 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.939 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.939 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:02.939 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.509186+0000", 2024-09-21T14:59:02.940 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8592999999999998e-05 2024-09-21T14:59:02.940 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.940 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.940 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:02.940 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.509248+0000", 2024-09-21T14:59:02.940 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1450000000000003e-05 2024-09-21T14:59:02.940 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:02.940 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:02.940 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:02.940 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.940 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.940 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6626 3.1a 3:5adee3e5:::rbd_data.11361bed1f78.0000000000000001:d [read 229376~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:59:02.940 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131593+0000", 2024-09-21T14:59:02.940 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 27.742268984999999, 2024-09-21T14:59:02.941 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012576725, 2024-09-21T14:59:02.941 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:02.941 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:02.941 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:02.941 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:02.941 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:02.941 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6626 2024-09-21T14:59:02.941 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.941 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:02.941 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.941 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:02.941 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131593+0000", 2024-09-21T14:59:02.941 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:02.941 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.942 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.942 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:02.942 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131593+0000", 2024-09-21T14:59:02.942 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:02.942 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.942 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.942 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:02.942 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131593+0000", 2024-09-21T14:59:02.942 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T14:59:02.942 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.942 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.942 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:02.942 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131598+0000", 2024-09-21T14:59:02.942 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7849999999999999e-06 2024-09-21T14:59:02.943 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.943 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.943 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:02.943 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131601+0000", 2024-09-21T14:59:02.943 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1949999999999999e-06 2024-09-21T14:59:02.943 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.943 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.943 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:02.943 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131604+0000", 2024-09-21T14:59:02.943 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6350000000000002e-06 2024-09-21T14:59:02.943 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.943 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.943 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:02.944 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.143902+0000", 2024-09-21T14:59:02.944 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012298216000000001 2024-09-21T14:59:02.944 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.944 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.944 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:02.944 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.143993+0000", 2024-09-21T14:59:02.944 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1073e-05 2024-09-21T14:59:02.944 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.944 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.944 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:02.944 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.144170+0000", 2024-09-21T14:59:02.944 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017703 2024-09-21T14:59:02.944 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:02.944 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:02.945 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:02.945 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.945 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.945 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6631 3.1a 3:5d547f54:::rbd_data.11361bed1f78.0000000000000003:d [read 0~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:59:02.945 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131606+0000", 2024-09-21T14:59:02.945 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 27.742256285, 2024-09-21T14:59:02.945 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013903148000000001, 2024-09-21T14:59:02.945 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:02.945 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:02.945 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:02.945 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:02.945 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:02.945 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6631 2024-09-21T14:59:02.945 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.945 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:02.946 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.946 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:02.946 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131606+0000", 2024-09-21T14:59:02.946 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:02.946 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.946 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.946 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:02.946 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131606+0000", 2024-09-21T14:59:02.946 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:02.946 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.946 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.946 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:02.946 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131606+0000", 2024-09-21T14:59:02.946 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T14:59:02.947 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.947 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.947 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:02.947 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131607+0000", 2024-09-21T14:59:02.947 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2699999999999999e-06 2024-09-21T14:59:02.947 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.947 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.947 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:02.947 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131610+0000", 2024-09-21T14:59:02.947 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.88e-06 2024-09-21T14:59:02.947 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.947 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.947 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:02.948 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131612+0000", 2024-09-21T14:59:02.948 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5170000000000002e-06 2024-09-21T14:59:02.948 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.948 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.948 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:02.948 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.144205+0000", 2024-09-21T14:59:02.948 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012592763 2024-09-21T14:59:02.948 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.948 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.948 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:02.948 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.144332+0000", 2024-09-21T14:59:02.948 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012694200000000001 2024-09-21T14:59:02.948 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.949 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.949 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:02.949 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.145509+0000", 2024-09-21T14:59:02.949 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001176992 2024-09-21T14:59:02.949 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:02.949 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:02.949 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:02.949 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.949 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.949 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6632 3.1a 3:5d547f54:::rbd_data.11361bed1f78.0000000000000003:d [read 81920~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:59:02.949 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131614+0000", 2024-09-21T14:59:02.949 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 27.742247677000002, 2024-09-21T14:59:02.949 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014093553999999999, 2024-09-21T14:59:02.950 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:02.950 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:02.950 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:02.950 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:02.950 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:02.950 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6632 2024-09-21T14:59:02.950 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.950 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:02.950 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.950 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:02.950 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131614+0000", 2024-09-21T14:59:02.950 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:02.950 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.951 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.951 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:02.951 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131614+0000", 2024-09-21T14:59:02.951 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:02.951 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.951 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.951 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:02.951 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131614+0000", 2024-09-21T14:59:02.951 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T14:59:02.951 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.951 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.951 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:02.951 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131615+0000", 2024-09-21T14:59:02.951 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.218e-06 2024-09-21T14:59:02.951 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.952 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.952 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:02.952 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131618+0000", 2024-09-21T14:59:02.952 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9050000000000001e-06 2024-09-21T14:59:02.952 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.952 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.952 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:02.952 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131621+0000", 2024-09-21T14:59:02.952 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7439999999999999e-06 2024-09-21T14:59:02.952 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.952 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.952 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:02.952 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.145538+0000", 2024-09-21T14:59:02.952 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013916618 2024-09-21T14:59:02.953 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.953 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.953 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:02.953 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.145619+0000", 2024-09-21T14:59:02.953 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1812999999999997e-05 2024-09-21T14:59:02.953 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.953 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.953 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:02.953 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.145708+0000", 2024-09-21T14:59:02.953 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8460000000000003e-05 2024-09-21T14:59:02.953 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:02.953 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:02.953 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:02.953 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.954 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.954 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6633 3.1a 3:5d547f54:::rbd_data.11361bed1f78.0000000000000003:d [read 147456~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:59:02.954 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131623+0000", 2024-09-21T14:59:02.954 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 27.742238710999999, 2024-09-21T14:59:02.954 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014395845000000001, 2024-09-21T14:59:02.954 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:02.954 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:02.954 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:02.954 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:02.954 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:02.954 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6633 2024-09-21T14:59:02.954 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.954 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:02.954 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.954 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:02.955 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131623+0000", 2024-09-21T14:59:02.955 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:02.955 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.955 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.955 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:02.955 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131623+0000", 2024-09-21T14:59:02.955 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:02.955 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.955 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.955 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:02.955 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131623+0000", 2024-09-21T14:59:02.955 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T14:59:02.955 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.955 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.955 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:02.956 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131628+0000", 2024-09-21T14:59:02.956 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0370000000000002e-06 2024-09-21T14:59:02.956 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.956 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.956 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:02.956 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131632+0000", 2024-09-21T14:59:02.956 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3519999999999998e-06 2024-09-21T14:59:02.956 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.956 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.956 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:02.956 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131634+0000", 2024-09-21T14:59:02.956 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5019999999999999e-06 2024-09-21T14:59:02.956 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.957 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.957 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:02.957 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.145734+0000", 2024-09-21T14:59:02.957 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014100118 2024-09-21T14:59:02.957 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.957 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.957 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:02.957 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.145804+0000", 2024-09-21T14:59:02.957 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0229999999999997e-05 2024-09-21T14:59:02.957 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.957 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.957 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:02.957 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146019+0000", 2024-09-21T14:59:02.957 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00021481699999999999 2024-09-21T14:59:02.958 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:02.958 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:02.958 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:02.958 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.958 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.958 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6634 3.1a 3:5d547f54:::rbd_data.11361bed1f78.0000000000000003:d [read 229376~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:59:02.958 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131636+0000", 2024-09-21T14:59:02.958 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 27.742225601000001, 2024-09-21T14:59:02.958 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014645843, 2024-09-21T14:59:02.958 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:02.958 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:02.958 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:02.958 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:02.958 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:02.959 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6634 2024-09-21T14:59:02.959 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.959 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:02.959 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.959 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:02.959 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131636+0000", 2024-09-21T14:59:02.959 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:02.959 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.959 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.959 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:02.959 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131636+0000", 2024-09-21T14:59:02.959 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:02.959 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.960 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.960 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:02.960 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131636+0000", 2024-09-21T14:59:02.960 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:02.960 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.960 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.960 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:02.960 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131641+0000", 2024-09-21T14:59:02.960 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8489999999999997e-06 2024-09-21T14:59:02.960 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.960 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.960 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:02.960 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131644+0000", 2024-09-21T14:59:02.960 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.179e-06 2024-09-21T14:59:02.961 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.961 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.961 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:02.961 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131647+0000", 2024-09-21T14:59:02.961 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8009999999999999e-06 2024-09-21T14:59:02.961 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.961 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.961 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:02.961 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146046+0000", 2024-09-21T14:59:02.961 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014399018 2024-09-21T14:59:02.961 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.961 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.961 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:02.961 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146118+0000", 2024-09-21T14:59:02.962 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2044000000000003e-05 2024-09-21T14:59:02.962 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.962 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.962 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:02.962 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146282+0000", 2024-09-21T14:59:02.962 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000164196 2024-09-21T14:59:02.962 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:02.962 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:02.962 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:02.962 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.962 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.962 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6639 3.1a 3:592d80ed:::rbd_data.11361bed1f78.0000000000000005:d [read 0~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:59:02.962 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131650+0000", 2024-09-21T14:59:02.962 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 27.742212402, 2024-09-21T14:59:02.963 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016002832000000002, 2024-09-21T14:59:02.963 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:02.963 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:02.963 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:02.963 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:02.963 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:02.963 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6639 2024-09-21T14:59:02.963 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.963 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:02.963 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.963 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:02.963 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131650+0000", 2024-09-21T14:59:02.963 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:02.963 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.964 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.964 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:02.964 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131650+0000", 2024-09-21T14:59:02.964 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:02.964 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.964 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.964 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:02.964 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131649+0000", 2024-09-21T14:59:02.964 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:02.964 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.964 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.964 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:02.964 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131651+0000", 2024-09-21T14:59:02.964 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9199999999999998e-06 2024-09-21T14:59:02.965 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.965 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.965 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:02.965 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131656+0000", 2024-09-21T14:59:02.965 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9899999999999997e-06 2024-09-21T14:59:02.965 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.965 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.965 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:02.965 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131665+0000", 2024-09-21T14:59:02.965 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0999999999999993e-06 2024-09-21T14:59:02.965 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.965 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.965 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:02.965 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146309+0000", 2024-09-21T14:59:02.966 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014643893999999999 2024-09-21T14:59:02.966 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.966 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.966 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:02.966 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146392+0000", 2024-09-21T14:59:02.966 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2531e-05 2024-09-21T14:59:02.966 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.966 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.966 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:02.966 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147653+0000", 2024-09-21T14:59:02.966 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0012606710000000001 2024-09-21T14:59:02.966 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:02.966 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:02.967 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:02.967 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.967 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.967 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6640 3.1a 3:592d80ed:::rbd_data.11361bed1f78.0000000000000005:d [read 65536~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:59:02.967 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131670+0000", 2024-09-21T14:59:02.967 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 27.742192103000001, 2024-09-21T14:59:02.967 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016188859, 2024-09-21T14:59:02.967 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:02.967 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:02.967 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:02.967 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:02.967 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:02.967 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6640 2024-09-21T14:59:02.967 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.968 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:02.968 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.968 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:02.968 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131670+0000", 2024-09-21T14:59:02.968 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:02.968 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.968 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.968 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:02.968 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131670+0000", 2024-09-21T14:59:02.968 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:02.968 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.968 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.968 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:02.968 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131670+0000", 2024-09-21T14:59:02.969 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T14:59:02.969 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.969 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.969 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:02.969 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131672+0000", 2024-09-21T14:59:02.969 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.818e-06 2024-09-21T14:59:02.969 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.969 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.969 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:02.969 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131675+0000", 2024-09-21T14:59:02.969 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5930000000000002e-06 2024-09-21T14:59:02.969 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.969 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.969 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:02.970 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131678+0000", 2024-09-21T14:59:02.970 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6790000000000001e-06 2024-09-21T14:59:02.970 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.970 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.970 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:02.970 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147681+0000", 2024-09-21T14:59:02.970 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016002755 2024-09-21T14:59:02.970 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.970 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.970 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:02.970 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147762+0000", 2024-09-21T14:59:02.970 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1052999999999998e-05 2024-09-21T14:59:02.970 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.970 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.971 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:02.971 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147859+0000", 2024-09-21T14:59:02.971 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.7199000000000002e-05 2024-09-21T14:59:02.971 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:02.971 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:02.971 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:02.971 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.971 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.971 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6641 3.1a 3:592d80ed:::rbd_data.11361bed1f78.0000000000000005:d [read 147456~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:59:02.971 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131680+0000", 2024-09-21T14:59:02.971 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 27.742181814999999, 2024-09-21T14:59:02.971 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016431296000000001, 2024-09-21T14:59:02.971 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:02.971 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:02.972 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:02.972 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:02.972 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:02.972 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6641 2024-09-21T14:59:02.972 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.972 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:02.972 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.972 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:02.972 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131680+0000", 2024-09-21T14:59:02.972 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:02.972 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.972 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.972 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:02.972 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131680+0000", 2024-09-21T14:59:02.973 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:02.973 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.973 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.973 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:02.973 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131680+0000", 2024-09-21T14:59:02.973 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T14:59:02.973 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.973 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.973 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:02.973 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131681+0000", 2024-09-21T14:59:02.973 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2610000000000001e-06 2024-09-21T14:59:02.973 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.973 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.973 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:02.974 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131685+0000", 2024-09-21T14:59:02.974 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.8299999999999998e-06 2024-09-21T14:59:02.974 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.974 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.974 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:02.974 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131688+0000", 2024-09-21T14:59:02.974 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6189999999999998e-06 2024-09-21T14:59:02.974 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.974 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.974 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:02.974 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147884+0000", 2024-09-21T14:59:02.974 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016196465 2024-09-21T14:59:02.974 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.974 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.975 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:02.975 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147949+0000", 2024-09-21T14:59:02.975 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4867000000000001e-05 2024-09-21T14:59:02.975 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.975 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.975 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:02.975 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.148112+0000", 2024-09-21T14:59:02.975 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00016247700000000001 2024-09-21T14:59:02.975 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:02.975 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:02.975 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:02.975 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.975 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.976 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6642 3.1a 3:592d80ed:::rbd_data.11361bed1f78.0000000000000005:d [read 212992~32768 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:59:02.976 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131690+0000", 2024-09-21T14:59:02.976 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 27.742171936999998, 2024-09-21T14:59:02.976 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016751335999999999, 2024-09-21T14:59:02.976 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:02.976 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:02.976 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:02.976 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:02.976 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:02.976 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6642 2024-09-21T14:59:02.976 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.976 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:02.976 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.976 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:02.977 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131690+0000", 2024-09-21T14:59:02.977 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:02.977 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.977 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.977 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:02.977 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131690+0000", 2024-09-21T14:59:02.977 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:02.977 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.977 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.977 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:02.977 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131690+0000", 2024-09-21T14:59:02.977 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T14:59:02.977 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.977 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.978 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:02.978 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131691+0000", 2024-09-21T14:59:02.978 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3120000000000001e-06 2024-09-21T14:59:02.978 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.978 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.978 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:02.978 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131694+0000", 2024-09-21T14:59:02.978 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0070000000000002e-06 2024-09-21T14:59:02.978 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.978 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.978 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:02.978 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131700+0000", 2024-09-21T14:59:02.978 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0800000000000002e-06 2024-09-21T14:59:02.979 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.979 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.979 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:02.979 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.148135+0000", 2024-09-21T14:59:02.979 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016434250000000001 2024-09-21T14:59:02.979 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.979 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.979 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:02.979 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.148204+0000", 2024-09-21T14:59:02.979 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9466000000000002e-05 2024-09-21T14:59:02.979 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:02.979 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:02.979 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:02.979 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.148441+0000", 2024-09-21T14:59:02.980 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00023743700000000001 2024-09-21T14:59:02.980 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:02.980 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:02.980 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:02.980 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:02.980 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:02.980 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:02.982 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T14:59:02.982 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T14:59:02.982 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T14:59:02.982 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-21T14:59:02.983 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T14:59:02.983 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:59:02.980+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:59:02.993 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T14:59:03.015 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T14:59:03.017 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:59:03.017 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:59:03.017 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:59:03.017 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:59:03.018 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:59:03.018 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:59:03.018 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:59:03.018 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:59:03.082 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:59:03.080+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:59:03.088 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:03.088 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:59:03.088 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T14:59:03.088 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:03.098 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T14:59:03.183 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:59:03.180+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:59:03.208 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:03.208 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:59:03.209 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T14:59:03.209 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T14:59:03.209 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:03.218 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T14:59:03.283 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T14:59:03.284 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T14:59:03.284 INFO:tasks.thrashosds.thrasher:inject_pause on osd.0 2024-09-21T14:59:03.284 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-21T14:59:03.284 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-21T14:59:03.284 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok config set filestore_inject_stall 3 2024-09-21T14:59:03.285 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:59:03.281+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:59:03.333 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:03.333 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T14:59:03.333 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T14:59:03.333 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T14:59:03.333 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.333 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:1454 3.e 3:71be909a:::rbd_data.1136515ecb6b.000000000000002a:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T14:59:03.334 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:30.339995+0000", 2024-09-21T14:59:03.334 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 32.992472552999999, 2024-09-21T14:59:03.334 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.033615974999999999, 2024-09-21T14:59:03.334 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:03.334 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:03.334 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:03.334 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:03.334 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:03.334 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1454 2024-09-21T14:59:03.334 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.334 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:03.335 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.335 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:03.335 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.339995+0000", 2024-09-21T14:59:03.335 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:03.335 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.335 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.335 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:03.335 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.339995+0000", 2024-09-21T14:59:03.335 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:03.335 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.335 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.335 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:03.336 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.339992+0000", 2024-09-21T14:59:03.336 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-21T14:59:03.336 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.336 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.336 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:03.336 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.340008+0000", 2024-09-21T14:59:03.336 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5906e-05 2024-09-21T14:59:03.336 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.336 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.336 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:03.336 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.340042+0000", 2024-09-21T14:59:03.336 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4066000000000002e-05 2024-09-21T14:59:03.337 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.337 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.337 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:03.337 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.340056+0000", 2024-09-21T14:59:03.337 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4389999999999999e-05 2024-09-21T14:59:03.337 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.337 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.337 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:03.337 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.340178+0000", 2024-09-21T14:59:03.337 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012202 2024-09-21T14:59:03.337 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.337 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.337 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:03.338 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.340424+0000", 2024-09-21T14:59:03.338 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000245682 2024-09-21T14:59:03.338 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.338 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.338 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 0", 2024-09-21T14:59:03.338 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.348708+0000", 2024-09-21T14:59:03.338 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0082834390000000001 2024-09-21T14:59:03.338 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.338 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.338 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:03.338 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.373176+0000", 2024-09-21T14:59:03.338 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024468888000000001 2024-09-21T14:59:03.339 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.339 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.339 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:03.339 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.373306+0000", 2024-09-21T14:59:03.339 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012991700000000001 2024-09-21T14:59:03.339 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.339 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.339 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:03.339 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.373464+0000", 2024-09-21T14:59:03.339 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000157913 2024-09-21T14:59:03.339 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.339 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.340 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:03.340 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.373611+0000", 2024-09-21T14:59:03.340 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014681699999999999 2024-09-21T14:59:03.340 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:03.340 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:03.340 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:03.340 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.340 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.340 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:15130 3.2 3:47f1bbdb:::rbd_data.11361bed1f78.0000000000000067:head [sparse-read 0~147456] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-21T14:59:03.340 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:40.288284+0000", 2024-09-21T14:59:03.340 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 23.044183232000002, 2024-09-21T14:59:03.341 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1142935039999999, 2024-09-21T14:59:03.341 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:03.341 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:03.341 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:03.341 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:03.341 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:03.341 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 15130 2024-09-21T14:59:03.341 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.341 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:03.341 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.341 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:03.341 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288284+0000", 2024-09-21T14:59:03.341 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:03.342 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.342 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.342 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:03.342 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288284+0000", 2024-09-21T14:59:03.342 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:03.342 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.342 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.342 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:03.342 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288284+0000", 2024-09-21T14:59:03.342 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:03.342 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.342 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.343 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:03.343 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288288+0000", 2024-09-21T14:59:03.343 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5829999999999998e-06 2024-09-21T14:59:03.343 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.343 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.343 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:03.343 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288297+0000", 2024-09-21T14:59:03.343 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0669999999999998e-06 2024-09-21T14:59:03.343 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.343 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.343 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:03.343 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288305+0000", 2024-09-21T14:59:03.344 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8860000000000004e-06 2024-09-21T14:59:03.344 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.344 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.344 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:03.344 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288417+0000", 2024-09-21T14:59:03.344 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000112952 2024-09-21T14:59:03.344 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.344 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.344 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T14:59:03.344 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288428+0000", 2024-09-21T14:59:03.344 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0631e-05 2024-09-21T14:59:03.345 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.345 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.345 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:03.345 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.401876+0000", 2024-09-21T14:59:03.345 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1134479610000001 2024-09-21T14:59:03.345 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.345 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.345 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:03.345 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.402076+0000", 2024-09-21T14:59:03.345 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00019944200000000001 2024-09-21T14:59:03.345 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.345 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.346 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:03.346 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.402578+0000", 2024-09-21T14:59:03.346 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00050242899999999998 2024-09-21T14:59:03.346 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:03.346 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:03.346 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:03.346 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.346 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.346 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:20598 3.2 3:403f4568:::rbd_data.1136c84d9a13.00000000000000d4:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-21T14:59:03.346 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:45.401322+0000", 2024-09-21T14:59:03.346 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 17.931145516000001, 2024-09-21T14:59:03.346 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014768051000000001, 2024-09-21T14:59:03.346 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:03.347 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:03.347 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:03.347 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:03.347 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:03.347 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 20598 2024-09-21T14:59:03.347 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.347 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:03.347 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.347 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:03.347 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:45.401322+0000", 2024-09-21T14:59:03.347 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:03.348 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.348 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.348 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:03.348 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:45.401322+0000", 2024-09-21T14:59:03.348 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:03.348 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.348 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.348 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:03.348 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:45.401320+0000", 2024-09-21T14:59:03.348 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T14:59:03.348 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.348 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.349 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:03.349 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:45.401330+0000", 2024-09-21T14:59:03.349 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0108999999999999e-05 2024-09-21T14:59:03.349 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.349 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.349 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:03.349 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:45.401351+0000", 2024-09-21T14:59:03.349 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1191000000000001e-05 2024-09-21T14:59:03.349 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.349 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.349 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:03.350 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:45.401361+0000", 2024-09-21T14:59:03.350 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5130000000000002e-06 2024-09-21T14:59:03.350 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.350 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.350 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:03.350 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:45.401476+0000", 2024-09-21T14:59:03.350 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000115684 2024-09-21T14:59:03.350 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.350 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.350 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:03.351 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:45.401717+0000", 2024-09-21T14:59:03.351 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000240256 2024-09-21T14:59:03.351 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.351 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.351 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 3", 2024-09-21T14:59:03.351 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:45.406009+0000", 2024-09-21T14:59:03.351 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0042920780000000004 2024-09-21T14:59:03.351 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.351 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.351 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:03.351 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:45.415983+0000", 2024-09-21T14:59:03.351 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0099743269999999998 2024-09-21T14:59:03.352 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.352 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.352 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:03.352 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:45.415998+0000", 2024-09-21T14:59:03.352 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.49e-05 2024-09-21T14:59:03.352 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.352 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.352 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:03.353 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:45.416049+0000", 2024-09-21T14:59:03.353 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0657e-05 2024-09-21T14:59:03.353 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.353 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.353 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:03.353 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:45.416090+0000", 2024-09-21T14:59:03.353 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1569000000000001e-05 2024-09-21T14:59:03.353 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:03.353 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:03.353 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:03.353 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.353 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.354 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22841 3.16 3:6fa60bdd:::rbd_data.11366acc894.0000000000000008:19 [read 917504~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:03.354 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.142001+0000", 2024-09-21T14:59:03.354 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 13.190466335, 2024-09-21T14:59:03.354 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.011779609, 2024-09-21T14:59:03.354 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:03.354 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:03.354 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:03.354 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:03.354 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:03.354 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22841 2024-09-21T14:59:03.354 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.354 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:03.355 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.355 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:03.355 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.142001+0000", 2024-09-21T14:59:03.355 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:03.355 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.355 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.355 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:03.355 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.142001+0000", 2024-09-21T14:59:03.355 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:03.355 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.355 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.355 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:03.356 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.142001+0000", 2024-09-21T14:59:03.356 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:03.356 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.356 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.356 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:03.356 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.142003+0000", 2024-09-21T14:59:03.356 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4109999999999999e-06 2024-09-21T14:59:03.356 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.356 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.356 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:03.356 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.142009+0000", 2024-09-21T14:59:03.357 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6219999999999997e-06 2024-09-21T14:59:03.357 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.357 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.357 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:03.357 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.142013+0000", 2024-09-21T14:59:03.357 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2069999999999997e-06 2024-09-21T14:59:03.357 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.357 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.357 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:03.357 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.152044+0000", 2024-09-21T14:59:03.357 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.010030435000000001 2024-09-21T14:59:03.357 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.358 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.358 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:03.358 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.152123+0000", 2024-09-21T14:59:03.358 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8813000000000006e-05 2024-09-21T14:59:03.358 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.358 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.358 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:03.358 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.153781+0000", 2024-09-21T14:59:03.358 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001658412 2024-09-21T14:59:03.358 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:03.358 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:03.358 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:03.359 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.359 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.359 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22842 3.16 3:6fa60bdd:::rbd_data.11366acc894.0000000000000008:19 [read 983040~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:03.359 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.142018+0000", 2024-09-21T14:59:03.359 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 13.190450073999999, 2024-09-21T14:59:03.359 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012170459, 2024-09-21T14:59:03.359 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:03.359 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:03.359 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:03.359 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:03.359 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:03.360 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22842 2024-09-21T14:59:03.360 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.360 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:03.360 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.360 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:03.360 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.142018+0000", 2024-09-21T14:59:03.360 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:03.360 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.360 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.360 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:03.360 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.142018+0000", 2024-09-21T14:59:03.361 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:03.361 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.361 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.361 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:03.361 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.142017+0000", 2024-09-21T14:59:03.361 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:03.361 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.361 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.361 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:03.361 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.142020+0000", 2024-09-21T14:59:03.361 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8119999999999999e-06 2024-09-21T14:59:03.361 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.362 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.362 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:03.362 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.142026+0000", 2024-09-21T14:59:03.362 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8150000000000002e-06 2024-09-21T14:59:03.362 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.362 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.362 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:03.362 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.142031+0000", 2024-09-21T14:59:03.362 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7040000000000002e-06 2024-09-21T14:59:03.362 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.363 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.363 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:03.363 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.153822+0000", 2024-09-21T14:59:03.363 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.011791658999999999 2024-09-21T14:59:03.363 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.363 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.363 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:03.363 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.153929+0000", 2024-09-21T14:59:03.363 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000106413 2024-09-21T14:59:03.363 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.363 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.363 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:03.364 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.154188+0000", 2024-09-21T14:59:03.364 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000259423 2024-09-21T14:59:03.364 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:03.364 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:03.364 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:03.364 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.364 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.364 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22914 3.16 3:6fa60bdd:::rbd_data.11366acc894.0000000000000008:19 [read 671744~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:03.364 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.143191+0000", 2024-09-21T14:59:03.364 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 13.189276991, 2024-09-21T14:59:03.364 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.011598498, 2024-09-21T14:59:03.365 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:03.365 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:03.365 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:03.365 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:03.365 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:03.365 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22914 2024-09-21T14:59:03.365 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.365 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:03.365 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.365 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:03.365 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.143191+0000", 2024-09-21T14:59:03.366 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:03.366 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.366 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.366 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:03.366 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.143191+0000", 2024-09-21T14:59:03.366 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:03.366 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.366 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.366 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:03.366 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.143190+0000", 2024-09-21T14:59:03.366 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:59:03.366 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.367 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.367 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:03.367 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.143195+0000", 2024-09-21T14:59:03.367 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0710000000000002e-06 2024-09-21T14:59:03.367 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.367 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.367 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:03.367 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.143205+0000", 2024-09-21T14:59:03.367 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8300000000000008e-06 2024-09-21T14:59:03.367 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.367 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.368 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:03.368 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.143212+0000", 2024-09-21T14:59:03.368 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.063e-06 2024-09-21T14:59:03.368 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.368 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.368 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:03.368 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.154561+0000", 2024-09-21T14:59:03.368 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.011348829 2024-09-21T14:59:03.368 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.368 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.368 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:03.369 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.154624+0000", 2024-09-21T14:59:03.369 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3688000000000004e-05 2024-09-21T14:59:03.369 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.369 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.369 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:03.369 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.154789+0000", 2024-09-21T14:59:03.369 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000164764 2024-09-21T14:59:03.369 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:03.369 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:03.369 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:03.369 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.369 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.369 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22915 3.16 3:6fa60bdd:::rbd_data.11366acc894.0000000000000008:19 [read 737280~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:03.370 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.143218+0000", 2024-09-21T14:59:03.370 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 13.189250132, 2024-09-21T14:59:03.370 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.011862391, 2024-09-21T14:59:03.370 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:03.370 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:03.370 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:03.370 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:03.370 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:03.370 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22915 2024-09-21T14:59:03.370 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.370 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:03.370 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.371 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:03.371 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.143218+0000", 2024-09-21T14:59:03.371 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:03.371 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.371 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.371 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:03.371 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.143218+0000", 2024-09-21T14:59:03.371 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:03.371 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.371 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.371 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:03.371 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.143217+0000", 2024-09-21T14:59:03.371 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:03.371 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.372 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.372 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:03.372 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.143220+0000", 2024-09-21T14:59:03.372 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2140000000000001e-06 2024-09-21T14:59:03.372 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.372 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.372 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:03.372 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.143226+0000", 2024-09-21T14:59:03.372 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2009999999999996e-06 2024-09-21T14:59:03.372 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.372 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.372 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:03.372 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.143232+0000", 2024-09-21T14:59:03.373 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1819999999999997e-06 2024-09-21T14:59:03.373 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.373 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.373 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:03.373 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.154823+0000", 2024-09-21T14:59:03.373 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.011591308999999999 2024-09-21T14:59:03.373 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.373 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.373 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:03.373 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.154898+0000", 2024-09-21T14:59:03.373 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5430000000000001e-05 2024-09-21T14:59:03.373 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.374 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.374 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:03.374 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.155080+0000", 2024-09-21T14:59:03.374 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000181569 2024-09-21T14:59:03.374 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:03.374 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:03.374 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:03.374 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.374 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.374 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22916 3.16 3:6fa60bdd:::rbd_data.11366acc894.0000000000000008:19 [read 802816~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:03.374 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.143236+0000", 2024-09-21T14:59:03.374 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 13.18923156, 2024-09-21T14:59:03.375 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012700477, 2024-09-21T14:59:03.375 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:03.375 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:03.375 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:03.375 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:03.375 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:03.375 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22916 2024-09-21T14:59:03.375 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.375 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:03.375 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.375 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:03.375 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.143236+0000", 2024-09-21T14:59:03.376 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:03.376 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.376 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.376 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:03.376 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.143236+0000", 2024-09-21T14:59:03.376 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:03.376 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.376 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.376 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:03.376 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.143236+0000", 2024-09-21T14:59:03.376 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:03.376 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.376 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.377 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:03.377 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.143238+0000", 2024-09-21T14:59:03.377 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5059999999999998e-06 2024-09-21T14:59:03.377 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.377 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.377 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:03.377 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.143265+0000", 2024-09-21T14:59:03.377 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7039999999999999e-05 2024-09-21T14:59:03.377 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.377 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.377 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:03.377 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.143272+0000", 2024-09-21T14:59:03.377 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4350000000000002e-06 2024-09-21T14:59:03.377 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.378 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.378 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:03.378 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.155362+0000", 2024-09-21T14:59:03.378 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012090485999999999 2024-09-21T14:59:03.378 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.378 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.378 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:03.378 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.155453+0000", 2024-09-21T14:59:03.378 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1001999999999994e-05 2024-09-21T14:59:03.378 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.378 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.378 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:03.378 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.155937+0000", 2024-09-21T14:59:03.379 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00048336100000000002 2024-09-21T14:59:03.379 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:03.379 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:03.379 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:03.379 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.379 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.379 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22917 3.16 3:6fa60bdd:::rbd_data.11366acc894.0000000000000008:19 [read 868352~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:03.379 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.143279+0000", 2024-09-21T14:59:03.379 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 13.189189105000001, 2024-09-21T14:59:03.379 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012979479, 2024-09-21T14:59:03.379 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:03.379 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:03.379 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:03.380 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:03.380 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:03.380 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22917 2024-09-21T14:59:03.380 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.380 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:03.380 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.380 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:03.380 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.143279+0000", 2024-09-21T14:59:03.380 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:03.380 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.380 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.380 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:03.380 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.143279+0000", 2024-09-21T14:59:03.380 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:03.381 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.381 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.381 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:03.381 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.143278+0000", 2024-09-21T14:59:03.381 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:03.381 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.381 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.381 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:03.381 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.143282+0000", 2024-09-21T14:59:03.381 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.681e-06 2024-09-21T14:59:03.381 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.381 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.381 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:03.381 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.143288+0000", 2024-09-21T14:59:03.382 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7220000000000002e-06 2024-09-21T14:59:03.382 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.382 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.382 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:03.382 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.143298+0000", 2024-09-21T14:59:03.382 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.533e-06 2024-09-21T14:59:03.382 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.382 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.382 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:03.382 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.155966+0000", 2024-09-21T14:59:03.382 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012667754 2024-09-21T14:59:03.383 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.383 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.383 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:03.383 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.156039+0000", 2024-09-21T14:59:03.383 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3506999999999999e-05 2024-09-21T14:59:03.383 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.383 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.383 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:03.383 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.156258+0000", 2024-09-21T14:59:03.383 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00021886200000000001 2024-09-21T14:59:03.383 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:03.384 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:03.384 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:03.384 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.384 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.384 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22918 3.16 3:6fa60bdd:::rbd_data.11366acc894.0000000000000008:19 [read 933888~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:03.384 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.143303+0000", 2024-09-21T14:59:03.384 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 13.189165051, 2024-09-21T14:59:03.384 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013274182000000001, 2024-09-21T14:59:03.384 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:03.384 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:03.384 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:03.385 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:03.385 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:03.385 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22918 2024-09-21T14:59:03.385 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.385 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:03.385 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.385 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:03.385 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.143303+0000", 2024-09-21T14:59:03.385 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:03.385 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.385 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.385 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:03.386 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.143303+0000", 2024-09-21T14:59:03.386 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:03.386 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.386 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.386 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:03.386 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.143302+0000", 2024-09-21T14:59:03.386 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:03.386 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.386 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.386 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:03.386 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.143305+0000", 2024-09-21T14:59:03.387 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7949999999999999e-06 2024-09-21T14:59:03.387 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.387 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.387 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:03.387 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.143311+0000", 2024-09-21T14:59:03.387 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.699e-06 2024-09-21T14:59:03.387 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.387 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.387 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:03.387 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.143315+0000", 2024-09-21T14:59:03.387 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.092e-06 2024-09-21T14:59:03.387 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.388 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.388 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:03.388 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.156288+0000", 2024-09-21T14:59:03.388 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012972727999999999 2024-09-21T14:59:03.388 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.388 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.388 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:03.388 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.156362+0000", 2024-09-21T14:59:03.388 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4201000000000003e-05 2024-09-21T14:59:03.388 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.388 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.388 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:03.389 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.156577+0000", 2024-09-21T14:59:03.389 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000215052 2024-09-21T14:59:03.389 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:03.389 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:03.389 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:03.389 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.389 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.389 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22919 3.16 3:6fa60bdd:::rbd_data.11366acc894.0000000000000008:19 [read 999424~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:03.389 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.143320+0000", 2024-09-21T14:59:03.389 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 13.189148082999999, 2024-09-21T14:59:03.389 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013576809, 2024-09-21T14:59:03.390 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:03.390 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:03.390 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:03.390 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:03.390 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:03.390 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22919 2024-09-21T14:59:03.390 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.390 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:03.390 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.390 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:03.390 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.143320+0000", 2024-09-21T14:59:03.390 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:03.391 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.391 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.391 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:03.391 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.143320+0000", 2024-09-21T14:59:03.391 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:03.391 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.391 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.391 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:03.391 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.143319+0000", 2024-09-21T14:59:03.391 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:03.391 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.391 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.392 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:03.392 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.143322+0000", 2024-09-21T14:59:03.392 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3e-06 2024-09-21T14:59:03.392 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.392 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.392 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:03.392 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.143327+0000", 2024-09-21T14:59:03.392 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7980000000000002e-06 2024-09-21T14:59:03.392 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.392 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.392 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:03.392 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.143332+0000", 2024-09-21T14:59:03.392 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2649999999999998e-06 2024-09-21T14:59:03.393 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.393 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.393 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:03.393 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.156604+0000", 2024-09-21T14:59:03.393 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013272885 2024-09-21T14:59:03.393 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.393 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.393 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:03.393 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.156677+0000", 2024-09-21T14:59:03.393 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2929999999999995e-05 2024-09-21T14:59:03.393 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.393 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.394 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:03.394 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.156896+0000", 2024-09-21T14:59:03.394 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000218975 2024-09-21T14:59:03.394 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:03.394 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:03.394 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:03.394 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.394 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.394 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22995 3.16 3:6fa60bdd:::rbd_data.11366acc894.0000000000000008:19 [read 884736~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:03.394 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.146618+0000", 2024-09-21T14:59:03.394 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 13.185849248, 2024-09-21T14:59:03.394 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.011828758999999999, 2024-09-21T14:59:03.395 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:03.395 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:03.395 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:03.395 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:03.395 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:03.395 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22995 2024-09-21T14:59:03.395 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.395 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:03.395 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.395 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:03.395 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.146618+0000", 2024-09-21T14:59:03.395 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:03.396 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.396 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.396 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:03.396 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.146618+0000", 2024-09-21T14:59:03.396 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:03.396 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.396 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.396 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:03.396 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.146618+0000", 2024-09-21T14:59:03.396 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:03.396 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.396 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.397 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:03.397 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.146647+0000", 2024-09-21T14:59:03.397 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8975e-05 2024-09-21T14:59:03.397 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.397 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.397 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:03.397 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.146655+0000", 2024-09-21T14:59:03.397 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1240000000000005e-06 2024-09-21T14:59:03.397 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.397 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.397 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:03.397 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.146661+0000", 2024-09-21T14:59:03.397 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9089999999999997e-06 2024-09-21T14:59:03.398 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.398 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.398 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:03.398 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.158188+0000", 2024-09-21T14:59:03.398 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.011526508 2024-09-21T14:59:03.398 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.398 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.398 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:03.398 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.158261+0000", 2024-09-21T14:59:03.398 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3342000000000004e-05 2024-09-21T14:59:03.398 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.398 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.399 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:03.399 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.158447+0000", 2024-09-21T14:59:03.399 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000186176 2024-09-21T14:59:03.399 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:03.399 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:03.399 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:03.399 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.399 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.399 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22996 3.16 3:6fa60bdd:::rbd_data.11366acc894.0000000000000008:19 [read 950272~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:03.399 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.146667+0000", 2024-09-21T14:59:03.399 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 13.185800684, 2024-09-21T14:59:03.400 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012144227, 2024-09-21T14:59:03.400 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:03.400 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:03.400 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:03.400 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:03.400 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:03.400 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22996 2024-09-21T14:59:03.400 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.400 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:03.400 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.400 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:03.400 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.146667+0000", 2024-09-21T14:59:03.400 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:03.400 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.401 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.401 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:03.401 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.146667+0000", 2024-09-21T14:59:03.401 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:03.401 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.401 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.401 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:03.401 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.146666+0000", 2024-09-21T14:59:03.401 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:03.401 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.401 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.401 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:03.401 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.146669+0000", 2024-09-21T14:59:03.402 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7800000000000001e-06 2024-09-21T14:59:03.402 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.402 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.402 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:03.402 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.146675+0000", 2024-09-21T14:59:03.402 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6910000000000002e-06 2024-09-21T14:59:03.402 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.402 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.402 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:03.402 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.146696+0000", 2024-09-21T14:59:03.402 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1200999999999999e-05 2024-09-21T14:59:03.402 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.402 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.403 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:03.403 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.158476+0000", 2024-09-21T14:59:03.403 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.011779707 2024-09-21T14:59:03.403 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.403 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.403 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:03.403 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.158607+0000", 2024-09-21T14:59:03.403 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013140299999999999 2024-09-21T14:59:03.403 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.403 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.403 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:03.403 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.158811+0000", 2024-09-21T14:59:03.404 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00020397899999999999 2024-09-21T14:59:03.404 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:03.404 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:03.404 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:03.404 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.404 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.404 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22997 3.16 3:6fa60bdd:::rbd_data.11366acc894.0000000000000008:19 [read 1015808~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:03.404 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.146702+0000", 2024-09-21T14:59:03.404 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 13.185765884, 2024-09-21T14:59:03.404 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012590647999999999, 2024-09-21T14:59:03.404 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:03.404 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:03.404 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:03.405 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:03.405 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:03.405 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22997 2024-09-21T14:59:03.405 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.405 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:03.405 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.405 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:03.405 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.146702+0000", 2024-09-21T14:59:03.405 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:03.405 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.405 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.405 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:03.405 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.146702+0000", 2024-09-21T14:59:03.406 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:03.406 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.406 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.406 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:03.406 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.146701+0000", 2024-09-21T14:59:03.406 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:03.406 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.406 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.406 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:03.406 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.146705+0000", 2024-09-21T14:59:03.406 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4189999999999998e-06 2024-09-21T14:59:03.406 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.406 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.406 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:03.407 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.146712+0000", 2024-09-21T14:59:03.407 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0759999999999997e-06 2024-09-21T14:59:03.407 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.407 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.407 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:03.407 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.146717+0000", 2024-09-21T14:59:03.407 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0579999999999997e-06 2024-09-21T14:59:03.407 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.407 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.407 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:03.407 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.158850+0000", 2024-09-21T14:59:03.407 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012133632 2024-09-21T14:59:03.407 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.408 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.408 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:03.408 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.158956+0000", 2024-09-21T14:59:03.408 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000105334 2024-09-21T14:59:03.408 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.408 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.408 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:03.408 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.159292+0000", 2024-09-21T14:59:03.408 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000336762 2024-09-21T14:59:03.408 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:03.408 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:03.408 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:03.408 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.408 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.409 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:23096 3.16 3:6fa60bdd:::rbd_data.11366acc894.0000000000000008:19 [read 966656~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:03.409 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.149395+0000", 2024-09-21T14:59:03.409 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 13.183073084, 2024-09-21T14:59:03.409 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.011817510999999999, 2024-09-21T14:59:03.409 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:03.409 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:03.409 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:03.409 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:03.409 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:03.409 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 23096 2024-09-21T14:59:03.409 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.409 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:03.409 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.409 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:03.410 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.149395+0000", 2024-09-21T14:59:03.410 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:03.410 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.410 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.410 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:03.410 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.149395+0000", 2024-09-21T14:59:03.410 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:03.410 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.410 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.410 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:03.410 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.149394+0000", 2024-09-21T14:59:03.410 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:03.410 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.411 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.411 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:03.411 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.149397+0000", 2024-09-21T14:59:03.411 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6620000000000001e-06 2024-09-21T14:59:03.411 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.411 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.411 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:03.411 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.149403+0000", 2024-09-21T14:59:03.411 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.75e-06 2024-09-21T14:59:03.411 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.411 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.411 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:03.411 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.149407+0000", 2024-09-21T14:59:03.411 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6450000000000004e-06 2024-09-21T14:59:03.412 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.412 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.412 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:03.412 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.160962+0000", 2024-09-21T14:59:03.412 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.011554627 2024-09-21T14:59:03.412 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.412 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.412 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:03.412 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.161034+0000", 2024-09-21T14:59:03.412 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2446000000000004e-05 2024-09-21T14:59:03.412 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.412 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.412 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:03.413 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.161212+0000", 2024-09-21T14:59:03.413 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017779499999999999 2024-09-21T14:59:03.413 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:03.413 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:03.413 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:03.413 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.413 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.413 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:23097 3.16 3:6fa60bdd:::rbd_data.11366acc894.0000000000000008:19 [read 1032192~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:03.413 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.149411+0000", 2024-09-21T14:59:03.413 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 13.183056474000001, 2024-09-21T14:59:03.413 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012012066, 2024-09-21T14:59:03.413 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:03.413 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:03.414 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:03.414 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:03.414 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:03.414 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 23097 2024-09-21T14:59:03.414 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.414 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:03.414 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.414 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:03.414 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.149411+0000", 2024-09-21T14:59:03.414 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:03.414 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.414 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.415 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:03.415 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.149411+0000", 2024-09-21T14:59:03.415 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:03.415 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.415 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.415 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:03.415 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.149411+0000", 2024-09-21T14:59:03.415 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:03.415 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.415 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.415 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:03.416 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.149416+0000", 2024-09-21T14:59:03.416 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9899999999999997e-06 2024-09-21T14:59:03.416 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.416 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.416 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:03.416 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.149427+0000", 2024-09-21T14:59:03.416 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1112e-05 2024-09-21T14:59:03.416 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.416 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.416 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:03.416 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.149438+0000", 2024-09-21T14:59:03.416 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0721e-05 2024-09-21T14:59:03.416 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.416 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.417 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:03.417 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.161234+0000", 2024-09-21T14:59:03.417 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.011796602 2024-09-21T14:59:03.417 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.417 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.417 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:03.417 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.161277+0000", 2024-09-21T14:59:03.417 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2542e-05 2024-09-21T14:59:03.417 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.417 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.417 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:03.417 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.161423+0000", 2024-09-21T14:59:03.417 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014637799999999999 2024-09-21T14:59:03.417 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:03.418 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:03.418 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:03.418 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.418 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.418 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:26360 3.16 3:6b7d3323:::rbd_object_map.11367fb81126.000000000000001d:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-21T14:59:03.418 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:54.282410+0000", 2024-09-21T14:59:03.418 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 9.0500580310000007, 2024-09-21T14:59:03.418 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.01170494, 2024-09-21T14:59:03.418 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:03.418 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:03.418 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:03.418 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:03.418 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:03.419 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 26360 2024-09-21T14:59:03.419 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.419 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:03.419 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.419 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:03.419 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.282410+0000", 2024-09-21T14:59:03.419 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:03.419 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.419 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.419 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:03.419 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.282410+0000", 2024-09-21T14:59:03.419 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:03.419 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.419 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.420 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:03.420 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.282409+0000", 2024-09-21T14:59:03.420 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:59:03.420 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.420 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.420 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:03.420 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.282417+0000", 2024-09-21T14:59:03.420 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1470000000000001e-06 2024-09-21T14:59:03.420 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.420 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.420 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:03.420 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.282439+0000", 2024-09-21T14:59:03.420 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1878e-05 2024-09-21T14:59:03.421 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.421 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.421 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:03.421 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.282446+0000", 2024-09-21T14:59:03.421 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2810000000000003e-06 2024-09-21T14:59:03.421 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.421 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.421 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:03.421 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.287889+0000", 2024-09-21T14:59:03.421 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0054428469999999998 2024-09-21T14:59:03.421 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.421 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.421 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:03.422 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.290591+0000", 2024-09-21T14:59:03.422 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0027019510000000002 2024-09-21T14:59:03.422 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.422 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.422 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 3", 2024-09-21T14:59:03.422 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.290936+0000", 2024-09-21T14:59:03.422 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00034511500000000001 2024-09-21T14:59:03.422 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.422 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.422 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:03.422 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.293945+0000", 2024-09-21T14:59:03.422 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0030093849999999998 2024-09-21T14:59:03.423 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.423 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.423 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:03.423 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.294016+0000", 2024-09-21T14:59:03.423 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0555999999999993e-05 2024-09-21T14:59:03.423 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.423 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.423 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:03.423 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.294071+0000", 2024-09-21T14:59:03.423 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5711999999999998e-05 2024-09-21T14:59:03.423 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.423 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.423 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:03.423 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.294115+0000", 2024-09-21T14:59:03.424 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3105000000000001e-05 2024-09-21T14:59:03.424 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:03.424 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:03.424 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:03.424 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.424 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.424 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:26361 3.16 3:6b7d3323:::rbd_object_map.11367fb81126.000000000000001d:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-21T14:59:03.424 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:54.282895+0000", 2024-09-21T14:59:03.424 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 9.049572994, 2024-09-21T14:59:03.424 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012623578999999999, 2024-09-21T14:59:03.424 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:03.424 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:03.424 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:03.425 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:03.425 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:03.425 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 26361 2024-09-21T14:59:03.425 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.425 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:03.425 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.425 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:03.425 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.282895+0000", 2024-09-21T14:59:03.425 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:03.425 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.425 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.425 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:03.425 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.282895+0000", 2024-09-21T14:59:03.426 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:03.426 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.426 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.426 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:03.426 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.282894+0000", 2024-09-21T14:59:03.426 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:59:03.426 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.426 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.426 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:03.426 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.282901+0000", 2024-09-21T14:59:03.426 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8750000000000002e-06 2024-09-21T14:59:03.426 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.426 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.427 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:03.427 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.282911+0000", 2024-09-21T14:59:03.427 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0710000000000001e-05 2024-09-21T14:59:03.427 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.427 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.427 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:03.427 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.282918+0000", 2024-09-21T14:59:03.427 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7769999999999997e-06 2024-09-21T14:59:03.427 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.427 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.427 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:03.427 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.291985+0000", 2024-09-21T14:59:03.427 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0090669989999999992 2024-09-21T14:59:03.428 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.428 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.428 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:03.428 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.292071+0000", 2024-09-21T14:59:03.428 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6292999999999995e-05 2024-09-21T14:59:03.428 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.428 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.428 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 3", 2024-09-21T14:59:03.428 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.292421+0000", 2024-09-21T14:59:03.428 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000349821 2024-09-21T14:59:03.428 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.428 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.428 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:03.429 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.295201+0000", 2024-09-21T14:59:03.429 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0027803490000000001 2024-09-21T14:59:03.429 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.429 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.429 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:03.429 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.295443+0000", 2024-09-21T14:59:03.429 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000241296 2024-09-21T14:59:03.429 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.429 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.429 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:03.429 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.295483+0000", 2024-09-21T14:59:03.429 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.0469999999999997e-05 2024-09-21T14:59:03.430 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.430 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.430 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:03.430 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.295518+0000", 2024-09-21T14:59:03.430 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4995e-05 2024-09-21T14:59:03.430 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:03.430 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:03.430 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:03.430 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.430 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.430 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:26362 3.16 3:6b7d3323:::rbd_object_map.11367fb81126.000000000000001d:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-21T14:59:03.431 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:54.282929+0000", 2024-09-21T14:59:03.431 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 9.049538793, 2024-09-21T14:59:03.431 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012685663, 2024-09-21T14:59:03.431 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:03.431 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:03.431 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:03.431 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:03.431 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:03.431 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 26362 2024-09-21T14:59:03.431 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.431 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:03.431 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.432 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:03.432 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.282929+0000", 2024-09-21T14:59:03.432 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:03.432 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.432 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.432 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:03.432 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.282929+0000", 2024-09-21T14:59:03.432 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:03.432 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.432 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.432 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:03.432 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.282928+0000", 2024-09-21T14:59:03.433 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:03.433 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.433 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.433 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:03.433 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.282933+0000", 2024-09-21T14:59:03.433 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3719999999999998e-06 2024-09-21T14:59:03.433 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.433 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.433 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:03.433 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.282940+0000", 2024-09-21T14:59:03.433 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9940000000000003e-06 2024-09-21T14:59:03.433 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.434 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.434 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:03.434 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.282946+0000", 2024-09-21T14:59:03.434 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7009999999999999e-06 2024-09-21T14:59:03.434 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.434 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.434 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:03.434 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.292982+0000", 2024-09-21T14:59:03.434 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.010035661 2024-09-21T14:59:03.434 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.434 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.435 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:03.435 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.293068+0000", 2024-09-21T14:59:03.435 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6211999999999994e-05 2024-09-21T14:59:03.435 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.435 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.435 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 3", 2024-09-21T14:59:03.435 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.293388+0000", 2024-09-21T14:59:03.435 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00031920499999999998 2024-09-21T14:59:03.435 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.435 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.435 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:03.436 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.295397+0000", 2024-09-21T14:59:03.436 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0020094079999999999 2024-09-21T14:59:03.436 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.436 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.436 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:03.436 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.295545+0000", 2024-09-21T14:59:03.436 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014850700000000001 2024-09-21T14:59:03.436 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.436 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.436 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:03.436 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.295579+0000", 2024-09-21T14:59:03.436 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3346000000000002e-05 2024-09-21T14:59:03.437 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.437 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.437 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:03.437 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.295615+0000", 2024-09-21T14:59:03.437 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5743e-05 2024-09-21T14:59:03.437 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:03.437 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:03.437 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:03.437 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.437 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.437 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:26363 3.16 3:6b7d3323:::rbd_object_map.11367fb81126.000000000000001d:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-21T14:59:03.437 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:54.283268+0000", 2024-09-21T14:59:03.438 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 9.0491997479999995, 2024-09-21T14:59:03.438 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013363939, 2024-09-21T14:59:03.438 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:03.438 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:03.438 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:03.438 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:03.438 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:03.438 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 26363 2024-09-21T14:59:03.438 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.438 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:03.438 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.438 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:03.439 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.283268+0000", 2024-09-21T14:59:03.439 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:03.439 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.439 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.439 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:03.439 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.283268+0000", 2024-09-21T14:59:03.439 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:03.439 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.439 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.439 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:03.439 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.283267+0000", 2024-09-21T14:59:03.439 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:59:03.440 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.440 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.440 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:03.440 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.283273+0000", 2024-09-21T14:59:03.440 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4330000000000001e-06 2024-09-21T14:59:03.440 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.440 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.440 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:03.440 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.283319+0000", 2024-09-21T14:59:03.440 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6171999999999998e-05 2024-09-21T14:59:03.440 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.440 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.440 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:03.441 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.283328+0000", 2024-09-21T14:59:03.441 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3590000000000008e-06 2024-09-21T14:59:03.441 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.441 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.441 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:03.441 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.294182+0000", 2024-09-21T14:59:03.441 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.010854074 2024-09-21T14:59:03.441 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.441 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.441 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:03.441 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.294264+0000", 2024-09-21T14:59:03.441 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2544000000000001e-05 2024-09-21T14:59:03.442 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.442 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.442 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 3", 2024-09-21T14:59:03.442 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.294723+0000", 2024-09-21T14:59:03.442 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000458181 2024-09-21T14:59:03.442 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.442 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.442 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:03.442 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.296044+0000", 2024-09-21T14:59:03.442 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001321486 2024-09-21T14:59:03.442 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.442 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.443 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:03.443 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.296560+0000", 2024-09-21T14:59:03.443 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00051559700000000002 2024-09-21T14:59:03.443 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.443 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.443 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:03.443 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.296596+0000", 2024-09-21T14:59:03.443 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6145e-05 2024-09-21T14:59:03.443 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.443 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.443 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:03.443 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:54.296632+0000", 2024-09-21T14:59:03.444 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6038000000000003e-05 2024-09-21T14:59:03.444 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:03.444 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:03.444 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:03.444 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:03.444 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:03.444 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:03.446 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T14:59:03.447 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:03.447 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-21T14:59:03.447 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:03.448 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:59:03.444+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:59:03.478 INFO:teuthology.orchestra.run.smithi032.stdout:checking clone #9, image image_client.0-clone9 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent10 2024-09-21T14:59:03.546 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:59:03.545+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:59:03.564 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:03.565 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:59:03.565 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T14:59:03.565 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:03.581 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T14:59:03.647 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:59:03.645+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:59:03.739 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:03.740 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:59:03.740 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T14:59:03.740 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T14:59:03.740 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:03.748 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:59:03.744+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:59:03.750 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T14:59:03.793 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:03.793 INFO:teuthology.orchestra.run.smithi032.stdout:1195 read 0x45a1823 thru 0x45ab40a (0x9be8 bytes) 2024-09-21T14:59:03.794 INFO:teuthology.orchestra.run.smithi032.stdout:1196 read 0x7142218 thru 0x714c419 (0xa202 bytes) 2024-09-21T14:59:03.800 INFO:teuthology.orchestra.run.smithi032.stdout:1199 trunc from 0xea70eff to 0xafac19e 2024-09-21T14:59:03.835 INFO:teuthology.orchestra.run.smithi032.stdout:1202 punch from 0xc5b8fd to 0xc5ddee, (0x24f1 bytes) 2024-09-21T14:59:03.835 INFO:teuthology.orchestra.run.smithi032.stdout:1203 write 0x6de5e2a thru 0x6dea761 (0x4938 bytes) 2024-09-21T14:59:03.849 INFO:teuthology.orchestra.run.smithi032.stdout:1205 read 0xedd85 thru 0xeefb2 (0x122e bytes) 2024-09-21T14:59:03.849 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:59:03.849+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:59:03.849 INFO:teuthology.orchestra.run.smithi032.stdout:1208 write 0x2c2eb97 thru 0x2c3588d (0x6cf7 bytes) 2024-09-21T14:59:03.863 INFO:teuthology.orchestra.run.smithi032.stdout:1209 write 0x94ae638 thru 0x94b43c8 (0x5d91 bytes) 2024-09-21T14:59:03.874 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:03.874 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T14:59:03.874 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T14:59:03.874 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T14:59:03.874 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.874 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4281.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-21T14:59:03.874 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:09.324416+0000", 2024-09-21T14:59:03.874 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 54.547864891000003, 2024-09-21T14:59:03.874 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.117837947, 2024-09-21T14:59:03.874 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:03.875 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:03.875 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:03.875 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4281", 2024-09-21T14:59:03.875 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/1518520303", 2024-09-21T14:59:03.875 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1 2024-09-21T14:59:03.875 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.875 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:03.875 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.875 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:03.875 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324416+0000", 2024-09-21T14:59:03.875 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:03.875 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.875 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.875 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:03.875 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324416+0000", 2024-09-21T14:59:03.875 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:03.876 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.876 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.876 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:03.876 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324414+0000", 2024-09-21T14:59:03.876 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T14:59:03.876 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.876 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.876 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:03.876 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324429+0000", 2024-09-21T14:59:03.876 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5637e-05 2024-09-21T14:59:03.876 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.876 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.876 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:03.876 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324448+0000", 2024-09-21T14:59:03.876 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9042000000000002e-05 2024-09-21T14:59:03.877 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.877 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.877 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:03.877 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324465+0000", 2024-09-21T14:59:03.877 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6767999999999999e-05 2024-09-21T14:59:03.877 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.877 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.877 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:03.877 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324571+0000", 2024-09-21T14:59:03.877 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000105714 2024-09-21T14:59:03.877 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.877 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.877 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op must wait for map", 2024-09-21T14:59:03.877 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324587+0000", 2024-09-21T14:59:03.877 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5753999999999999e-05 2024-09-21T14:59:03.878 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.878 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.878 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:03.878 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.325864+0000", 2024-09-21T14:59:03.878 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0012773350000000001 2024-09-21T14:59:03.878 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.878 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.878 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T14:59:03.878 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.325878+0000", 2024-09-21T14:59:03.878 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4498000000000001e-05 2024-09-21T14:59:03.878 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.878 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.878 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:03.878 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.439461+0000", 2024-09-21T14:59:03.879 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.11358212500000001 2024-09-21T14:59:03.879 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.879 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.879 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:03.879 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.439848+0000", 2024-09-21T14:59:03.879 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00038747800000000002 2024-09-21T14:59:03.879 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.879 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.879 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:59:03.879 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.440175+0000", 2024-09-21T14:59:03.879 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00032721900000000001 2024-09-21T14:59:03.879 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.879 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.879 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:03.879 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.441770+0000", 2024-09-21T14:59:03.879 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0015949409999999999 2024-09-21T14:59:03.880 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.880 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.880 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:03.880 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.442086+0000", 2024-09-21T14:59:03.880 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00031586299999999999 2024-09-21T14:59:03.880 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.880 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.880 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:03.880 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.442195+0000", 2024-09-21T14:59:03.880 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000108791 2024-09-21T14:59:03.880 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.880 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.880 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:03.880 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.442254+0000", 2024-09-21T14:59:03.880 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9038e-05 2024-09-21T14:59:03.881 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:03.881 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:03.881 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:03.881 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.881 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.881 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4281.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-21T14:59:03.881 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:09.447828+0000", 2024-09-21T14:59:03.881 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 54.424452889999998, 2024-09-21T14:59:03.881 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.32904393700000001, 2024-09-21T14:59:03.881 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:03.881 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:03.881 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:03.882 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4281", 2024-09-21T14:59:03.882 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/1518520303", 2024-09-21T14:59:03.882 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 2 2024-09-21T14:59:03.882 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.882 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:03.882 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.882 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:03.882 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447828+0000", 2024-09-21T14:59:03.882 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:03.882 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.882 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.882 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:03.882 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447828+0000", 2024-09-21T14:59:03.882 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:03.882 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.883 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.883 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:03.883 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447825+0000", 2024-09-21T14:59:03.883 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T14:59:03.883 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.883 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.883 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:03.883 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447841+0000", 2024-09-21T14:59:03.883 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5359999999999999e-05 2024-09-21T14:59:03.883 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.883 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.883 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:03.883 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447875+0000", 2024-09-21T14:59:03.883 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4388999999999999e-05 2024-09-21T14:59:03.884 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.884 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.884 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:03.884 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447903+0000", 2024-09-21T14:59:03.884 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7373e-05 2024-09-21T14:59:03.884 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.884 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.884 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:03.884 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447996+0000", 2024-09-21T14:59:03.884 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2954999999999993e-05 2024-09-21T14:59:03.884 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.884 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.884 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T14:59:03.884 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.448011+0000", 2024-09-21T14:59:03.885 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5038999999999999e-05 2024-09-21T14:59:03.885 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.885 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.885 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:03.885 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.776501+0000", 2024-09-21T14:59:03.885 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.32849012500000002 2024-09-21T14:59:03.885 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.885 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.885 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:03.885 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.776872+0000", 2024-09-21T14:59:03.885 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00037119600000000002 2024-09-21T14:59:03.885 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:03.885 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:03.885 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:03.885 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.885 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.886 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:752 3.14 3:28fc02e5:::rbd_data.113693316f36.0000000000000010:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-21T14:59:03.886 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:26.625579+0000", 2024-09-21T14:59:03.886 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 37.246702034999998, 2024-09-21T14:59:03.886 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042965426000000001, 2024-09-21T14:59:03.886 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:03.886 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:03.886 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:03.886 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:03.886 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:03.886 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 752 2024-09-21T14:59:03.886 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.886 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:03.886 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.886 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:03.887 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625579+0000", 2024-09-21T14:59:03.887 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:03.887 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.887 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.887 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:03.887 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625579+0000", 2024-09-21T14:59:03.887 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:03.887 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.887 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.887 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:03.887 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625577+0000", 2024-09-21T14:59:03.887 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T14:59:03.887 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.888 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.888 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:03.888 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625588+0000", 2024-09-21T14:59:03.888 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1555e-05 2024-09-21T14:59:03.888 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.888 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.888 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:03.888 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625610+0000", 2024-09-21T14:59:03.888 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2107999999999999e-05 2024-09-21T14:59:03.888 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.888 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.888 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:03.889 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625621+0000", 2024-09-21T14:59:03.889 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0762e-05 2024-09-21T14:59:03.889 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.889 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.889 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:03.889 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625692+0000", 2024-09-21T14:59:03.889 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.1203000000000003e-05 2024-09-21T14:59:03.889 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.889 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.889 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:03.889 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625872+0000", 2024-09-21T14:59:03.889 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017962800000000001 2024-09-21T14:59:03.889 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.889 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.889 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:59:03.890 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.633188+0000", 2024-09-21T14:59:03.890 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.007316014 2024-09-21T14:59:03.890 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.890 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.890 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:03.890 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.668357+0000", 2024-09-21T14:59:03.890 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035168933999999999 2024-09-21T14:59:03.890 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.890 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.890 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:03.890 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.668418+0000", 2024-09-21T14:59:03.890 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1282e-05 2024-09-21T14:59:03.890 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.890 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.890 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:03.891 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.668486+0000", 2024-09-21T14:59:03.891 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8468999999999998e-05 2024-09-21T14:59:03.891 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.891 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.891 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:03.891 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.668544+0000", 2024-09-21T14:59:03.891 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7785000000000003e-05 2024-09-21T14:59:03.891 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:03.891 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:03.891 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:03.891 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.891 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.891 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22737 3.a 3:574c7a60:::rbd_data.11366acc894.0000000000000002:19 [read 1032192~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:03.891 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141028+0000", 2024-09-21T14:59:03.892 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 13.731253069999999, 2024-09-21T14:59:03.892 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035206281999999998, 2024-09-21T14:59:03.892 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:03.892 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:03.892 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:03.892 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:03.892 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:03.892 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22737 2024-09-21T14:59:03.892 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.892 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:03.892 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.892 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:03.892 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141028+0000", 2024-09-21T14:59:03.892 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:03.892 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.892 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.893 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:03.893 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141028+0000", 2024-09-21T14:59:03.893 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:03.893 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.893 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.893 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:03.893 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141027+0000", 2024-09-21T14:59:03.893 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:03.893 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.893 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.893 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:03.893 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141030+0000", 2024-09-21T14:59:03.893 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6189999999999998e-06 2024-09-21T14:59:03.893 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.893 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.894 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:03.894 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141036+0000", 2024-09-21T14:59:03.894 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5110000000000003e-06 2024-09-21T14:59:03.894 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.894 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.894 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:03.894 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141040+0000", 2024-09-21T14:59:03.894 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.34e-06 2024-09-21T14:59:03.894 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.894 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.894 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:03.894 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.175948+0000", 2024-09-21T14:59:03.894 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034908540000000002 2024-09-21T14:59:03.894 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.895 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.895 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:03.895 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176025+0000", 2024-09-21T14:59:03.895 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6412e-05 2024-09-21T14:59:03.895 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.895 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.895 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:03.895 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176234+0000", 2024-09-21T14:59:03.895 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000209247 2024-09-21T14:59:03.895 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:03.895 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:03.895 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:03.895 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.895 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.895 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22738 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 49152~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:03.896 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141049+0000", 2024-09-21T14:59:03.896 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 13.731231892, 2024-09-21T14:59:03.896 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035428754999999999, 2024-09-21T14:59:03.896 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:03.896 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:03.896 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:03.896 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:03.896 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:03.896 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22738 2024-09-21T14:59:03.896 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.896 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:03.896 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.896 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:03.896 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141049+0000", 2024-09-21T14:59:03.896 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:03.896 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.897 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.897 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:03.897 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141049+0000", 2024-09-21T14:59:03.897 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:03.897 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.897 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.897 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:03.897 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141049+0000", 2024-09-21T14:59:03.897 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:03.897 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.897 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.897 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:03.897 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141051+0000", 2024-09-21T14:59:03.897 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4980000000000001e-06 2024-09-21T14:59:03.897 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.897 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.898 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:03.898 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141057+0000", 2024-09-21T14:59:03.898 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8030000000000002e-06 2024-09-21T14:59:03.898 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.898 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.898 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:03.898 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141066+0000", 2024-09-21T14:59:03.898 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1160000000000005e-06 2024-09-21T14:59:03.898 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.898 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.898 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:03.898 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176264+0000", 2024-09-21T14:59:03.898 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035197580999999999 2024-09-21T14:59:03.898 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.898 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.899 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:03.899 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176327+0000", 2024-09-21T14:59:03.899 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3237000000000005e-05 2024-09-21T14:59:03.899 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.899 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.899 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:03.899 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176478+0000", 2024-09-21T14:59:03.899 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000150851 2024-09-21T14:59:03.899 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:03.899 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:03.899 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:03.899 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.899 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.899 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22739 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 114688~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:03.900 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141070+0000", 2024-09-21T14:59:03.900 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 13.73121065, 2024-09-21T14:59:03.900 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035679094000000001, 2024-09-21T14:59:03.900 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:03.900 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:03.900 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:03.900 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:03.900 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:03.900 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22739 2024-09-21T14:59:03.900 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.900 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:03.900 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.901 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:03.901 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141070+0000", 2024-09-21T14:59:03.901 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:03.901 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.901 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.901 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:03.901 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141070+0000", 2024-09-21T14:59:03.901 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:03.901 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.901 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.901 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:03.901 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141070+0000", 2024-09-21T14:59:03.901 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:03.901 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.901 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.901 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:03.902 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141072+0000", 2024-09-21T14:59:03.902 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.379e-06 2024-09-21T14:59:03.902 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.902 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.902 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:03.902 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141078+0000", 2024-09-21T14:59:03.902 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6710000000000004e-06 2024-09-21T14:59:03.902 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.902 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.902 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:03.902 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141083+0000", 2024-09-21T14:59:03.902 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7249999999999997e-06 2024-09-21T14:59:03.902 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.902 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.902 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:03.903 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176506+0000", 2024-09-21T14:59:03.903 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035423076999999997 2024-09-21T14:59:03.903 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.903 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.903 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:03.903 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176582+0000", 2024-09-21T14:59:03.903 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6305999999999998e-05 2024-09-21T14:59:03.903 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.903 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.903 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:03.903 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176749+0000", 2024-09-21T14:59:03.903 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000167262 2024-09-21T14:59:03.903 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:03.903 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:03.903 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:03.903 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.904 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.904 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22740 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 180224~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:03.904 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141090+0000", 2024-09-21T14:59:03.904 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 13.731191246, 2024-09-21T14:59:03.904 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036129535999999997, 2024-09-21T14:59:03.904 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:03.904 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:03.904 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:03.904 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:03.904 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:03.904 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22740 2024-09-21T14:59:03.904 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.904 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:03.904 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.904 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:03.905 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141090+0000", 2024-09-21T14:59:03.905 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:03.905 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.905 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.905 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:03.905 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141090+0000", 2024-09-21T14:59:03.905 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:03.905 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.905 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.905 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:03.905 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141089+0000", 2024-09-21T14:59:03.905 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:03.905 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.905 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.905 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:03.905 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141092+0000", 2024-09-21T14:59:03.906 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7709999999999998e-06 2024-09-21T14:59:03.906 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.906 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.906 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:03.906 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141098+0000", 2024-09-21T14:59:03.906 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.015e-06 2024-09-21T14:59:03.906 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.906 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.906 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:03.906 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141103+0000", 2024-09-21T14:59:03.906 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8629999999999999e-06 2024-09-21T14:59:03.906 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.906 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.907 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:03.907 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176778+0000", 2024-09-21T14:59:03.907 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035674984 2024-09-21T14:59:03.907 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.907 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.907 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:03.907 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176851+0000", 2024-09-21T14:59:03.907 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2861000000000006e-05 2024-09-21T14:59:03.907 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.907 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.907 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:03.907 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177219+0000", 2024-09-21T14:59:03.907 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00036845999999999998 2024-09-21T14:59:03.907 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:03.908 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:03.908 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:03.908 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.908 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.908 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22741 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 245760~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:03.908 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141107+0000", 2024-09-21T14:59:03.908 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 13.731173675000001, 2024-09-21T14:59:03.908 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036407729, 2024-09-21T14:59:03.908 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:03.908 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:03.908 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:03.908 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:03.908 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:03.908 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22741 2024-09-21T14:59:03.908 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.908 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:03.909 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.909 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:03.909 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141107+0000", 2024-09-21T14:59:03.909 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:03.909 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.909 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.909 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:03.909 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141107+0000", 2024-09-21T14:59:03.909 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:03.909 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.909 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.909 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:03.909 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141107+0000", 2024-09-21T14:59:03.909 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:03.910 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.910 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.910 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:03.910 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141109+0000", 2024-09-21T14:59:03.910 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6529999999999998e-06 2024-09-21T14:59:03.910 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.910 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.910 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:03.910 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141115+0000", 2024-09-21T14:59:03.910 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9649999999999996e-06 2024-09-21T14:59:03.910 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.910 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.910 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:03.910 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141121+0000", 2024-09-21T14:59:03.911 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1209999999999998e-06 2024-09-21T14:59:03.911 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.911 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.911 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:03.911 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177242+0000", 2024-09-21T14:59:03.911 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036121706000000003 2024-09-21T14:59:03.911 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.911 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.911 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:03.911 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177301+0000", 2024-09-21T14:59:03.911 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9008000000000002e-05 2024-09-21T14:59:03.911 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.911 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.911 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:03.911 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177515+0000", 2024-09-21T14:59:03.912 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00021362 2024-09-21T14:59:03.912 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:03.912 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:03.912 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:03.912 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.912 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.912 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22742 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 311296~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:03.912 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141126+0000", 2024-09-21T14:59:03.912 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 13.731155320999999, 2024-09-21T14:59:03.912 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036653537, 2024-09-21T14:59:03.912 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:03.912 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:03.912 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:03.912 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:03.913 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:03.913 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22742 2024-09-21T14:59:03.913 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.913 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:03.913 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.913 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:03.913 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141126+0000", 2024-09-21T14:59:03.913 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:03.913 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.913 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.913 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:03.913 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141126+0000", 2024-09-21T14:59:03.913 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:03.913 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.914 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.914 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:03.914 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141125+0000", 2024-09-21T14:59:03.914 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:03.914 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.914 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.914 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:03.914 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141128+0000", 2024-09-21T14:59:03.914 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.886e-06 2024-09-21T14:59:03.914 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.914 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.914 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:03.914 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141137+0000", 2024-09-21T14:59:03.914 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4399999999999994e-06 2024-09-21T14:59:03.915 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.915 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.915 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:03.915 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141143+0000", 2024-09-21T14:59:03.915 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1410000000000004e-06 2024-09-21T14:59:03.915 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.915 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.915 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:03.915 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177543+0000", 2024-09-21T14:59:03.915 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036400544999999999 2024-09-21T14:59:03.915 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.915 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.915 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:03.916 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177605+0000", 2024-09-21T14:59:03.916 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2206999999999996e-05 2024-09-21T14:59:03.916 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.916 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.916 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:03.916 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177779+0000", 2024-09-21T14:59:03.916 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017368099999999999 2024-09-21T14:59:03.916 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:03.916 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:03.916 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:03.916 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.916 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.916 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22743 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 376832~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:03.916 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141147+0000", 2024-09-21T14:59:03.916 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 13.731133837, 2024-09-21T14:59:03.917 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036915158000000003, 2024-09-21T14:59:03.917 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:03.917 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:03.917 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:03.917 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:03.917 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:03.917 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22743 2024-09-21T14:59:03.917 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.917 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:03.917 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.917 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:03.917 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141147+0000", 2024-09-21T14:59:03.917 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:03.917 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.918 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.918 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:03.918 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141147+0000", 2024-09-21T14:59:03.918 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:03.918 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.918 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.918 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:03.918 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141147+0000", 2024-09-21T14:59:03.918 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:03.918 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.918 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.918 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:03.918 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141149+0000", 2024-09-21T14:59:03.918 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.79e-06 2024-09-21T14:59:03.919 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.919 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.919 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:03.919 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141155+0000", 2024-09-21T14:59:03.919 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6699999999999999e-06 2024-09-21T14:59:03.919 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.919 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.919 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:03.919 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141160+0000", 2024-09-21T14:59:03.919 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5759999999999999e-06 2024-09-21T14:59:03.919 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.919 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.919 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:03.919 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177817+0000", 2024-09-21T14:59:03.919 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036657404999999997 2024-09-21T14:59:03.920 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.920 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.920 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:03.920 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177892+0000", 2024-09-21T14:59:03.920 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4602000000000002e-05 2024-09-21T14:59:03.920 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.920 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.920 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:03.920 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178062+0000", 2024-09-21T14:59:03.920 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000170464 2024-09-21T14:59:03.920 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:03.920 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:03.920 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:03.920 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.921 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.921 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22744 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 442368~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:03.921 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141164+0000", 2024-09-21T14:59:03.921 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 13.731116704, 2024-09-21T14:59:03.921 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037120806999999999, 2024-09-21T14:59:03.921 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:03.921 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:03.921 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:03.921 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:03.921 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:03.921 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22744 2024-09-21T14:59:03.921 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.921 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:03.921 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.921 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:03.922 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141164+0000", 2024-09-21T14:59:03.922 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:03.922 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.922 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.922 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:03.922 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141164+0000", 2024-09-21T14:59:03.922 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:03.922 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.922 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.922 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:03.922 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141164+0000", 2024-09-21T14:59:03.922 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:03.922 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.922 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.922 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:03.923 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141167+0000", 2024-09-21T14:59:03.923 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7159999999999999e-06 2024-09-21T14:59:03.923 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.923 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.923 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:03.923 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141173+0000", 2024-09-21T14:59:03.923 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.071e-06 2024-09-21T14:59:03.923 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.923 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.923 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:03.923 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141177+0000", 2024-09-21T14:59:03.923 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3610000000000003e-06 2024-09-21T14:59:03.923 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.923 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.924 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:03.924 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178083+0000", 2024-09-21T14:59:03.924 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036906405000000003 2024-09-21T14:59:03.924 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.924 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.924 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:03.924 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178143+0000", 2024-09-21T14:59:03.924 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0056999999999998e-05 2024-09-21T14:59:03.924 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.924 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.924 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:03.924 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178285+0000", 2024-09-21T14:59:03.924 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000141555 2024-09-21T14:59:03.925 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:03.925 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:03.925 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:03.925 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.925 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.925 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22745 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 507904~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:03.925 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141182+0000", 2024-09-21T14:59:03.925 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 13.731098383999999, 2024-09-21T14:59:03.925 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037332331000000003, 2024-09-21T14:59:03.925 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:03.925 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:03.925 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:03.925 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:03.925 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:03.925 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22745 2024-09-21T14:59:03.925 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.926 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:03.926 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.926 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:03.926 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141182+0000", 2024-09-21T14:59:03.926 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:03.926 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.926 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.926 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:03.926 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141182+0000", 2024-09-21T14:59:03.926 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:03.926 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.926 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.926 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:03.926 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141182+0000", 2024-09-21T14:59:03.927 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:03.927 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.927 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.927 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:03.927 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141185+0000", 2024-09-21T14:59:03.927 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.464e-06 2024-09-21T14:59:03.927 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.927 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.927 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:03.927 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141200+0000", 2024-09-21T14:59:03.927 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5886999999999999e-05 2024-09-21T14:59:03.927 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.927 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.928 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:03.928 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141571+0000", 2024-09-21T14:59:03.928 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000370909 2024-09-21T14:59:03.928 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.928 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.928 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:03.928 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178318+0000", 2024-09-21T14:59:03.928 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036746244999999997 2024-09-21T14:59:03.928 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.928 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.928 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:03.928 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178367+0000", 2024-09-21T14:59:03.928 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9317000000000002e-05 2024-09-21T14:59:03.929 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.929 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.929 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:03.929 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178515+0000", 2024-09-21T14:59:03.929 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014784600000000001 2024-09-21T14:59:03.929 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:03.929 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:03.929 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:03.929 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.929 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.929 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22746 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 573440~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:03.929 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141599+0000", 2024-09-21T14:59:03.929 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 13.730681752000001, 2024-09-21T14:59:03.929 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037402249999999998, 2024-09-21T14:59:03.929 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:03.930 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:03.930 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:03.930 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:03.930 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:03.930 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22746 2024-09-21T14:59:03.930 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.930 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:03.930 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.930 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:03.930 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141599+0000", 2024-09-21T14:59:03.930 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:03.930 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.930 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.930 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:03.931 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141599+0000", 2024-09-21T14:59:03.931 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:03.931 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.931 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.931 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:03.931 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141598+0000", 2024-09-21T14:59:03.931 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:59:03.931 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.931 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.931 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:03.931 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141637+0000", 2024-09-21T14:59:03.931 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.9119999999999998e-05 2024-09-21T14:59:03.931 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.931 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.932 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:03.932 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141647+0000", 2024-09-21T14:59:03.932 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5570000000000001e-06 2024-09-21T14:59:03.932 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.932 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.932 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:03.932 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141654+0000", 2024-09-21T14:59:03.932 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2049999999999996e-06 2024-09-21T14:59:03.932 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.932 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.932 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:03.932 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178553+0000", 2024-09-21T14:59:03.932 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036898528 2024-09-21T14:59:03.933 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.933 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.933 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:03.933 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178625+0000", 2024-09-21T14:59:03.933 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2669999999999994e-05 2024-09-21T14:59:03.933 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.933 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.933 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:03.933 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179001+0000", 2024-09-21T14:59:03.934 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000375977 2024-09-21T14:59:03.934 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:03.934 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:03.934 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:03.934 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.934 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.934 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22747 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 638976~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:03.934 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141660+0000", 2024-09-21T14:59:03.934 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 13.730620610000001, 2024-09-21T14:59:03.934 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037617352999999999, 2024-09-21T14:59:03.934 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:03.935 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:03.935 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:03.935 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:03.935 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:03.935 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22747 2024-09-21T14:59:03.935 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.935 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:03.935 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.935 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:03.935 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141660+0000", 2024-09-21T14:59:03.935 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:03.935 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.935 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.935 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:03.935 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141660+0000", 2024-09-21T14:59:03.936 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:03.936 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.936 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.936 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:03.936 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141660+0000", 2024-09-21T14:59:03.936 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:03.936 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.936 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.936 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:03.936 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141663+0000", 2024-09-21T14:59:03.936 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.569e-06 2024-09-21T14:59:03.936 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.936 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.936 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:03.936 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141670+0000", 2024-09-21T14:59:03.937 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5719999999999999e-06 2024-09-21T14:59:03.937 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.937 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.937 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:03.937 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141675+0000", 2024-09-21T14:59:03.937 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2889999999999997e-06 2024-09-21T14:59:03.937 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.937 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.937 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:03.937 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179028+0000", 2024-09-21T14:59:03.937 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037353381999999997 2024-09-21T14:59:03.937 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.937 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.937 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:03.937 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179099+0000", 2024-09-21T14:59:03.937 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0259000000000006e-05 2024-09-21T14:59:03.938 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.938 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.938 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:03.938 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179278+0000", 2024-09-21T14:59:03.938 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000178908 2024-09-21T14:59:03.938 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:03.938 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:03.938 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:03.938 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.938 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.938 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22748 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 704512~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:03.938 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141680+0000", 2024-09-21T14:59:03.938 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 13.730600972, 2024-09-21T14:59:03.938 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037806166000000002, 2024-09-21T14:59:03.938 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:03.939 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:03.939 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:03.939 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:03.939 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:03.939 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22748 2024-09-21T14:59:03.939 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.939 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:03.939 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.939 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:03.939 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141680+0000", 2024-09-21T14:59:03.939 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:03.939 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.939 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.939 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:03.939 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141680+0000", 2024-09-21T14:59:03.939 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:03.940 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.940 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.940 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:03.940 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141679+0000", 2024-09-21T14:59:03.940 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:03.940 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.940 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.940 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:03.940 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141683+0000", 2024-09-21T14:59:03.940 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3230000000000002e-06 2024-09-21T14:59:03.940 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.940 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.940 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:03.940 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141690+0000", 2024-09-21T14:59:03.940 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8680000000000001e-06 2024-09-21T14:59:03.941 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.941 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.941 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:03.941 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141694+0000", 2024-09-21T14:59:03.941 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6770000000000003e-06 2024-09-21T14:59:03.941 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.941 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.941 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:03.941 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179300+0000", 2024-09-21T14:59:03.941 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037605567999999999 2024-09-21T14:59:03.941 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.941 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.941 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:03.941 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179355+0000", 2024-09-21T14:59:03.942 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4771000000000001e-05 2024-09-21T14:59:03.942 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.942 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.942 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:03.942 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179486+0000", 2024-09-21T14:59:03.942 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013141699999999999 2024-09-21T14:59:03.942 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:03.942 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:03.942 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:03.942 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.942 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.942 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22749 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 770048~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:03.942 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141698+0000", 2024-09-21T14:59:03.943 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 13.7305826, 2024-09-21T14:59:03.943 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037967813000000003, 2024-09-21T14:59:03.943 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:03.943 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:03.943 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:03.943 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:03.943 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:03.943 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22749 2024-09-21T14:59:03.943 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.943 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:03.943 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.943 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:03.943 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141698+0000", 2024-09-21T14:59:03.943 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:03.944 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.944 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.944 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:03.944 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141698+0000", 2024-09-21T14:59:03.944 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:03.944 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.944 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.944 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:03.944 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141698+0000", 2024-09-21T14:59:03.944 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:03.944 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.944 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.944 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:03.944 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141701+0000", 2024-09-21T14:59:03.945 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8399999999999999e-06 2024-09-21T14:59:03.945 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.945 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.945 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:03.945 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141706+0000", 2024-09-21T14:59:03.945 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4979999999999997e-06 2024-09-21T14:59:03.945 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.945 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.945 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:03.945 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141711+0000", 2024-09-21T14:59:03.945 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3850000000000004e-06 2024-09-21T14:59:03.945 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.945 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.945 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:03.946 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179504+0000", 2024-09-21T14:59:03.946 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037793514 2024-09-21T14:59:03.946 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.946 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.946 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:03.946 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179546+0000", 2024-09-21T14:59:03.946 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1467999999999997e-05 2024-09-21T14:59:03.946 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.946 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.946 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:03.946 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179666+0000", 2024-09-21T14:59:03.946 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000120499 2024-09-21T14:59:03.946 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:03.946 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:03.947 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:03.947 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.947 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.947 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22750 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 835584~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:03.947 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141715+0000", 2024-09-21T14:59:03.947 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 13.730566298999999, 2024-09-21T14:59:03.947 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038173179000000002, 2024-09-21T14:59:03.947 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:03.947 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:03.947 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:03.947 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:03.947 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:03.947 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22750 2024-09-21T14:59:03.947 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.948 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:03.948 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.948 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:03.948 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141715+0000", 2024-09-21T14:59:03.948 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:03.948 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.948 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.948 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:03.948 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141715+0000", 2024-09-21T14:59:03.948 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:03.948 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.948 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.948 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:03.949 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141714+0000", 2024-09-21T14:59:03.949 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:03.949 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.949 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.949 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:03.949 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141716+0000", 2024-09-21T14:59:03.949 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1730000000000002e-06 2024-09-21T14:59:03.949 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.949 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.949 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:03.949 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141722+0000", 2024-09-21T14:59:03.949 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.417e-06 2024-09-21T14:59:03.949 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.949 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.949 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:03.950 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141726+0000", 2024-09-21T14:59:03.950 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4830000000000001e-06 2024-09-21T14:59:03.950 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.950 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.950 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:03.950 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179683+0000", 2024-09-21T14:59:03.950 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037956977000000003 2024-09-21T14:59:03.950 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.950 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.950 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:03.950 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179751+0000", 2024-09-21T14:59:03.950 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7762000000000006e-05 2024-09-21T14:59:03.950 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.951 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.951 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:03.951 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179888+0000", 2024-09-21T14:59:03.951 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000136635 2024-09-21T14:59:03.951 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:03.951 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:03.951 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:03.951 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.951 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.951 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22751 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 901120~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:03.951 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141730+0000", 2024-09-21T14:59:03.951 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 13.7305505, 2024-09-21T14:59:03.951 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038357011000000003, 2024-09-21T14:59:03.952 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:03.952 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:03.952 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:03.952 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:03.952 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:03.952 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22751 2024-09-21T14:59:03.952 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.952 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:03.952 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.952 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:03.952 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141730+0000", 2024-09-21T14:59:03.952 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:03.952 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.952 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.952 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:03.952 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141730+0000", 2024-09-21T14:59:03.952 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:03.953 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.953 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.953 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:03.953 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141730+0000", 2024-09-21T14:59:03.953 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:03.953 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.953 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.953 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:03.953 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141737+0000", 2024-09-21T14:59:03.953 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.64e-06 2024-09-21T14:59:03.953 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.953 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.953 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:03.953 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141743+0000", 2024-09-21T14:59:03.953 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8130000000000001e-06 2024-09-21T14:59:03.953 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.954 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.954 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:03.954 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141747+0000", 2024-09-21T14:59:03.954 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4920000000000004e-06 2024-09-21T14:59:03.954 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.954 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.954 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:03.954 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179918+0000", 2024-09-21T14:59:03.954 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038170822 2024-09-21T14:59:03.954 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.954 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.954 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:03.954 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179960+0000", 2024-09-21T14:59:03.954 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1999000000000002e-05 2024-09-21T14:59:03.955 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.955 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.955 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:03.955 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180087+0000", 2024-09-21T14:59:03.955 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012752799999999999 2024-09-21T14:59:03.955 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:03.955 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:03.955 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:03.955 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.955 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.955 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22752 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 966656~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:03.955 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141751+0000", 2024-09-21T14:59:03.955 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 13.730530180000001, 2024-09-21T14:59:03.955 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038553084000000001, 2024-09-21T14:59:03.956 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:03.956 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:03.956 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:03.956 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:03.956 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:03.956 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22752 2024-09-21T14:59:03.956 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.956 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:03.956 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.956 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:03.956 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141751+0000", 2024-09-21T14:59:03.956 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:03.956 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.956 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.957 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:03.957 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141751+0000", 2024-09-21T14:59:03.957 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:03.957 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.957 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.957 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:03.957 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141750+0000", 2024-09-21T14:59:03.957 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:03.957 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.957 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.957 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:03.957 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141753+0000", 2024-09-21T14:59:03.957 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5900000000000002e-06 2024-09-21T14:59:03.957 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.958 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.958 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:03.958 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141759+0000", 2024-09-21T14:59:03.958 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.947e-06 2024-09-21T14:59:03.958 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.958 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.958 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:03.958 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141764+0000", 2024-09-21T14:59:03.958 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8319999999999996e-06 2024-09-21T14:59:03.958 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.958 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.958 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:03.958 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180108+0000", 2024-09-21T14:59:03.959 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038344095000000002 2024-09-21T14:59:03.959 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.959 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.959 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:03.959 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180168+0000", 2024-09-21T14:59:03.959 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0047000000000003e-05 2024-09-21T14:59:03.959 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.959 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.959 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:03.959 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180304+0000", 2024-09-21T14:59:03.959 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013592500000000001 2024-09-21T14:59:03.960 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:03.960 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:03.960 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:03.960 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.960 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.960 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22753 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 1032192~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:03.960 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141768+0000", 2024-09-21T14:59:03.960 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 13.730513306000001, 2024-09-21T14:59:03.960 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038818810000000002, 2024-09-21T14:59:03.960 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:03.960 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:03.960 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:03.960 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:03.960 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:03.960 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22753 2024-09-21T14:59:03.961 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.961 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:03.961 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.961 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:03.961 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141768+0000", 2024-09-21T14:59:03.961 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:03.961 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.961 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.961 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:03.961 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141768+0000", 2024-09-21T14:59:03.961 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:03.961 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.961 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.961 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:03.962 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141767+0000", 2024-09-21T14:59:03.962 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:03.962 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.962 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.962 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:03.962 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141770+0000", 2024-09-21T14:59:03.962 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3329999999999999e-06 2024-09-21T14:59:03.962 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.962 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.962 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:03.962 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141775+0000", 2024-09-21T14:59:03.962 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2669999999999997e-06 2024-09-21T14:59:03.962 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.962 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.963 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:03.963 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141779+0000", 2024-09-21T14:59:03.963 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2259999999999999e-06 2024-09-21T14:59:03.963 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.963 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.963 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:03.963 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180325+0000", 2024-09-21T14:59:03.963 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038546428000000001 2024-09-21T14:59:03.963 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.963 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.963 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:03.963 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180401+0000", 2024-09-21T14:59:03.963 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5437000000000003e-05 2024-09-21T14:59:03.963 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:03.964 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:03.964 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:03.964 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180586+0000", 2024-09-21T14:59:03.964 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000185429 2024-09-21T14:59:03.964 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:03.964 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:03.964 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:03.964 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:03.964 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:03.964 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:03.965 INFO:teuthology.orchestra.run.smithi032.stdout:1210 trunc from 0xafac19e to 0xaf3cc44 2024-09-21T14:59:03.966 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T14:59:03.967 INFO:teuthology.orchestra.run.smithi032.stdout:1211 writesame 0xa28b5ca thru 0xa2950b6 data_size 0x1(0x9aed bytes) 2024-09-21T14:59:03.967 INFO:teuthology.orchestra.run.smithi032.stdout:1212 read 0x524d337 thru 0x5253441 (0x610b bytes) 2024-09-21T14:59:03.967 INFO:teuthology.orchestra.run.smithi032.stdout:1213 trunc from 0xaf3cc44 to 0x6f8f9da 2024-09-21T14:59:03.967 INFO:teuthology.orchestra.run.smithi032.stdout:1214 trunc from 0x6f8f9da to 0xa7fd589 2024-09-21T14:59:03.967 INFO:teuthology.orchestra.run.smithi032.stdout:1215 writesame 0x71acf4a thru 0x71b57af data_size 0x22(0x8866 bytes) 2024-09-21T14:59:03.967 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:59:03.964+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:59:03.980 INFO:teuthology.orchestra.run.smithi032.stdout:1216 trunc from 0xa7fd589 to 0x8c20ee4 2024-09-21T14:59:03.992 INFO:teuthology.orchestra.run.smithi032.stdout:1217 read 0x882732 thru 0x88e58f (0xbe5e bytes) 2024-09-21T14:59:03.992 INFO:teuthology.orchestra.run.smithi032.stdout:1219 read 0x4e50b97 thru 0x4e5ab4c (0x9fb6 bytes) 2024-09-21T14:59:03.994 INFO:teuthology.orchestra.run.smithi032.stdout:1220 trunc from 0x8c20ee4 to 0x9725f60 2024-09-21T14:59:04.006 INFO:teuthology.orchestra.run.smithi032.stdout:1222 write 0x66e8d73 thru 0x66f0a5b (0x7ce9 bytes) 2024-09-21T14:59:04.024 INFO:teuthology.orchestra.run.smithi032.stdout:1223 read 0x8073061 thru 0x807dd80 (0xad20 bytes) 2024-09-21T14:59:04.024 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:04.024 INFO:teuthology.orchestra.run.smithi032.stdout:1225 writesame 0x30654ec thru 0x30737a2 data_size 0x1(0xe2b7 bytes) 2024-09-21T14:59:04.049 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:04.049 INFO:teuthology.orchestra.run.smithi032.stdout:1230 write 0x24b464 thru 0x252e2b (0x79c8 bytes) 2024-09-21T14:59:04.052 INFO:teuthology.orchestra.run.smithi032.stdout:1231 writesame 0x524d576 thru 0x52520a1 data_size 0x22(0x4b2c bytes) 2024-09-21T14:59:04.065 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:04.065 INFO:teuthology.orchestra.run.smithi032.stdout:1234 punch from 0x11f8df6 to 0x11fa300, (0x150a bytes) 2024-09-21T14:59:04.065 INFO:teuthology.orchestra.run.smithi032.stdout:1237 punch from 0x4e9557e to 0x4e9b8ae, (0x6330 bytes) 2024-09-21T14:59:04.065 INFO:teuthology.orchestra.run.smithi032.stdout:1238 read 0x7e52906 thru 0x7e5ea41 (0xc13c bytes) 2024-09-21T14:59:04.066 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:59:04.065+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:59:04.085 INFO:teuthology.orchestra.run.smithi032.stdout:1239 write 0xebc9a09 thru 0xebd7598 (0xdb90 bytes) 2024-09-21T14:59:04.089 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:59:04.090 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:59:04.090 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T14:59:04.090 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:59:04.102 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T14:59:04.103 INFO:teuthology.orchestra.run.smithi032.stdout:1242 read 0x4db5cdd thru 0x4db6806 (0xb2a bytes) 2024-09-21T14:59:04.103 INFO:teuthology.orchestra.run.smithi032.stdout:1243 write 0x46fdc08 thru 0x470b77d (0xdb76 bytes) 2024-09-21T14:59:04.142 INFO:teuthology.orchestra.run.smithi032.stdout:1247 trunc from 0xebd7599 to 0xa2567e 2024-09-21T14:59:04.166 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:59:04.165+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:59:04.185 INFO:teuthology.orchestra.run.smithi032.stdout:1249 read 0x9dacee thru 0x9ddd61 (0x3074 bytes) 2024-09-21T14:59:04.185 INFO:teuthology.orchestra.run.smithi032.stdout:1250 read 0x7582cd thru 0x75e467 (0x619b bytes) 2024-09-21T14:59:04.185 INFO:teuthology.orchestra.run.smithi032.stdout:1251 punch from 0x1d6399 to 0x1e3faa, (0xdc11 bytes) 2024-09-21T14:59:04.185 INFO:teuthology.orchestra.run.smithi032.stdout:1253 read 0x4323af thru 0x440f8a (0xebdc bytes) 2024-09-21T14:59:04.185 INFO:teuthology.orchestra.run.smithi032.stdout:1255 read 0x7bb222 thru 0x7c6342 (0xb121 bytes) 2024-09-21T14:59:04.185 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:04.211 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:59:04.211 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:59:04.211 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T14:59:04.212 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T14:59:04.212 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:59:04.212 INFO:teuthology.orchestra.run.smithi032.stdout:1257 writesame 0xbdce486 thru 0xbdce718 data_size 0x1(0x293 bytes) 2024-09-21T14:59:04.212 INFO:teuthology.orchestra.run.smithi032.stdout:1258 writesame 0x7d23579 thru 0x7d2ca29 data_size 0x17(0x94b1 bytes) 2024-09-21T14:59:04.224 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T14:59:04.226 INFO:teuthology.orchestra.run.smithi032.stdout:1259 read 0x51a33a9 thru 0x51a7634 (0x428c bytes) 2024-09-21T14:59:04.226 INFO:teuthology.orchestra.run.smithi032.stdout:1260 trunc from 0xbdce719 to 0xbcde796 2024-09-21T14:59:04.251 INFO:teuthology.orchestra.run.smithi032.stdout:1261 writesame 0xe19e682 thru 0xe1a7a5b data_size 0x19(0x93da bytes) 2024-09-21T14:59:04.251 INFO:teuthology.orchestra.run.smithi032.stdout:1262 writesame 0x9749ba6 thru 0x974c240 data_size 0x1(0x269b bytes) 2024-09-21T14:59:04.265 INFO:teuthology.orchestra.run.smithi032.stdout:1263 punch from 0x937eb44 to 0x93897e1, (0xac9d bytes) 2024-09-21T14:59:04.265 INFO:teuthology.orchestra.run.smithi032.stdout:1265 read 0x727ebb2 thru 0x7283b47 (0x4f96 bytes) 2024-09-21T14:59:04.265 INFO:teuthology.orchestra.run.smithi032.stdout:1266 read 0xa0f9e1c thru 0xa103911 (0x9af6 bytes) 2024-09-21T14:59:04.265 INFO:teuthology.orchestra.run.smithi032.stdout:1269 read 0x1387bcd thru 0x138cb76 (0x4faa bytes) 2024-09-21T14:59:04.265 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:04.266 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:04.266 INFO:teuthology.orchestra.run.smithi032.stdout:1272 read 0xc52db17 thru 0xc53603f (0x8529 bytes) 2024-09-21T14:59:04.266 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:04.266 INFO:teuthology.orchestra.run.smithi032.stdout:1275 writesame 0xc69da71 thru 0xc6a90f7 data_size 0x1(0xb687 bytes) 2024-09-21T14:59:04.267 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:59:04.264+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:59:04.284 INFO:teuthology.orchestra.run.smithi032.stdout:1276 read 0xc822a9c thru 0xc82eedb (0xc440 bytes) 2024-09-21T14:59:04.300 INFO:teuthology.orchestra.run.smithi032.stdout:1277 trunc from 0xe1a7a5c to 0x6ae205d 2024-09-21T14:59:04.312 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T14:59:04.321 INFO:teuthology.orchestra.run.smithi032.stdout:1278 write 0xf301cc thru 0xf32ebd (0x2cf2 bytes) 2024-09-21T14:59:04.333 INFO:teuthology.orchestra.run.smithi032.stdout:1282 punch from 0x10df3db to 0x10ee841, (0xf466 bytes) 2024-09-21T14:59:04.333 INFO:teuthology.orchestra.run.smithi032.stdout:1283 punch from 0x3a896cd to 0x3a8f365, (0x5c98 bytes) 2024-09-21T14:59:04.340 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:59:04.340 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T14:59:04.340 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T14:59:04.340 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T14:59:04.341 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.341 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:1452 3.19 3:9d40fe72:::rbd_data.1136515ecb6b.0000000000000013:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T14:59:04.341 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:30.292755+0000", 2024-09-21T14:59:04.341 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 34.046794818999999, 2024-09-21T14:59:04.341 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043852191999999998, 2024-09-21T14:59:04.341 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:04.341 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:04.341 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:04.341 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:04.341 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:04.341 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1452 2024-09-21T14:59:04.341 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.341 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:04.341 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.342 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:04.342 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292755+0000", 2024-09-21T14:59:04.342 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:04.342 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.342 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.342 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:04.342 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292755+0000", 2024-09-21T14:59:04.342 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:04.342 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.342 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.342 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:04.342 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292754+0000", 2024-09-21T14:59:04.342 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:04.342 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.342 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.343 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:04.343 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292760+0000", 2024-09-21T14:59:04.343 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5029999999999997e-06 2024-09-21T14:59:04.343 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.343 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.343 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:04.343 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292771+0000", 2024-09-21T14:59:04.343 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1551e-05 2024-09-21T14:59:04.343 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.343 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.343 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:04.343 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292779+0000", 2024-09-21T14:59:04.343 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1559999999999998e-06 2024-09-21T14:59:04.343 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.344 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.344 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:04.344 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292896+0000", 2024-09-21T14:59:04.344 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000117304 2024-09-21T14:59:04.344 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.344 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.344 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:04.344 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.293124+0000", 2024-09-21T14:59:04.344 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00022819999999999999 2024-09-21T14:59:04.344 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.344 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.344 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 1", 2024-09-21T14:59:04.344 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.299967+0000", 2024-09-21T14:59:04.344 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0068427009999999996 2024-09-21T14:59:04.344 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.345 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.345 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:04.345 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336463+0000", 2024-09-21T14:59:04.345 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.036495842000000001 2024-09-21T14:59:04.345 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.345 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.345 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:59:04.345 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336480+0000", 2024-09-21T14:59:04.345 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7419999999999999e-05 2024-09-21T14:59:04.345 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.345 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.345 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:59:04.345 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336550+0000", 2024-09-21T14:59:04.345 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9951999999999997e-05 2024-09-21T14:59:04.346 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.346 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.346 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:04.346 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336607+0000", 2024-09-21T14:59:04.346 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7423000000000002e-05 2024-09-21T14:59:04.346 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:04.346 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:04.346 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:04.346 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.346 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.346 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop_reply(client.4406.0:1452 3.19 e36/24 ondisk, result = 0)", 2024-09-21T14:59:04.346 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:30.322854+0000", 2024-09-21T14:59:04.346 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 34.016696142999997, 2024-09-21T14:59:04.346 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013613079, 2024-09-21T14:59:04.347 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:04.347 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:04.347 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:04.347 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.347 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:04.347 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.322854+0000", 2024-09-21T14:59:04.347 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:04.347 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.347 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.347 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:04.347 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.322854+0000", 2024-09-21T14:59:04.347 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:04.347 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.347 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.347 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:04.348 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.322852+0000", 2024-09-21T14:59:04.348 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:59:04.348 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.348 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.348 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:04.348 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.322860+0000", 2024-09-21T14:59:04.348 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.605e-06 2024-09-21T14:59:04.348 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.348 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.348 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:04.348 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.322878+0000", 2024-09-21T14:59:04.348 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8204000000000001e-05 2024-09-21T14:59:04.348 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.348 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.348 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:04.349 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.322888+0000", 2024-09-21T14:59:04.349 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4099999999999997e-06 2024-09-21T14:59:04.349 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.349 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.349 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:04.349 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336440+0000", 2024-09-21T14:59:04.349 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013551848 2024-09-21T14:59:04.349 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.349 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.349 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:04.349 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336453+0000", 2024-09-21T14:59:04.349 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3465000000000001e-05 2024-09-21T14:59:04.349 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.349 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.350 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:04.350 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336467+0000", 2024-09-21T14:59:04.350 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3913999999999999e-05 2024-09-21T14:59:04.350 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:04.350 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:04.350 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:04.350 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.350 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.350 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:1567 3.8 3:139d3d55:::rbd_data.1136515ecb6b.0000000000000000:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T14:59:04.350 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:30.681087+0000", 2024-09-21T14:59:04.350 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 33.658462604, 2024-09-21T14:59:04.350 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.039009224000000002, 2024-09-21T14:59:04.350 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:04.350 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:04.350 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:04.351 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:04.351 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:04.351 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1567 2024-09-21T14:59:04.351 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.351 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:04.351 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.351 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:04.351 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681087+0000", 2024-09-21T14:59:04.351 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:04.351 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.351 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.351 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:04.351 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681087+0000", 2024-09-21T14:59:04.351 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:04.351 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.352 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.352 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:04.352 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681086+0000", 2024-09-21T14:59:04.352 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:59:04.352 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.352 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.352 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:04.352 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681097+0000", 2024-09-21T14:59:04.352 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.134e-05 2024-09-21T14:59:04.352 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.352 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.352 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:04.352 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681122+0000", 2024-09-21T14:59:04.352 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5313e-05 2024-09-21T14:59:04.352 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.353 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.353 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:04.353 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681138+0000", 2024-09-21T14:59:04.353 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5916000000000001e-05 2024-09-21T14:59:04.353 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.353 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.353 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:04.353 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681270+0000", 2024-09-21T14:59:04.353 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000131474 2024-09-21T14:59:04.353 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.353 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.353 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:04.353 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681416+0000", 2024-09-21T14:59:04.353 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014637 2024-09-21T14:59:04.353 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.354 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.354 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 7", 2024-09-21T14:59:04.354 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.693163+0000", 2024-09-21T14:59:04.354 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011747241 2024-09-21T14:59:04.354 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.354 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.354 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:59:04.354 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.719750+0000", 2024-09-21T14:59:04.354 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.026586852000000001 2024-09-21T14:59:04.354 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.354 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.354 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:04.354 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.719859+0000", 2024-09-21T14:59:04.355 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010925999999999999 2024-09-21T14:59:04.355 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.355 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.355 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:59:04.355 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.719951+0000", 2024-09-21T14:59:04.355 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1294999999999999e-05 2024-09-21T14:59:04.355 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.355 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.355 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:04.355 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.720097+0000", 2024-09-21T14:59:04.355 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014583 2024-09-21T14:59:04.355 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:04.355 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:04.355 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:04.356 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.356 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.356 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9347 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 16384~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:04.356 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158291+0000", 2024-09-21T14:59:04.356 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 25.181259102999999, 2024-09-21T14:59:04.356 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013600958, 2024-09-21T14:59:04.356 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:04.356 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:04.356 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:04.356 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:04.356 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:04.356 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9347 2024-09-21T14:59:04.356 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.356 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:04.356 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.357 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:04.357 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158291+0000", 2024-09-21T14:59:04.357 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:04.357 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.357 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.357 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:04.357 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158291+0000", 2024-09-21T14:59:04.357 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:04.357 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.357 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.357 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:04.357 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158290+0000", 2024-09-21T14:59:04.357 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:04.357 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.358 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.358 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:04.358 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158295+0000", 2024-09-21T14:59:04.358 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2399999999999998e-06 2024-09-21T14:59:04.358 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.358 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.358 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:04.358 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158312+0000", 2024-09-21T14:59:04.358 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6929999999999999e-05 2024-09-21T14:59:04.358 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.358 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.358 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:04.358 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158353+0000", 2024-09-21T14:59:04.358 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0805e-05 2024-09-21T14:59:04.359 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.359 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.359 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:04.359 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.171634+0000", 2024-09-21T14:59:04.359 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.01328085 2024-09-21T14:59:04.359 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.359 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.359 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:04.359 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.171722+0000", 2024-09-21T14:59:04.359 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8115000000000006e-05 2024-09-21T14:59:04.359 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.359 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.359 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:04.359 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.171892+0000", 2024-09-21T14:59:04.359 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000169722 2024-09-21T14:59:04.360 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:04.360 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:04.360 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:04.360 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.360 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.360 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9348 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 122880~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:04.360 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158369+0000", 2024-09-21T14:59:04.360 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 25.181180935, 2024-09-21T14:59:04.360 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.01388847, 2024-09-21T14:59:04.360 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:04.360 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:04.360 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:04.360 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:04.360 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:04.361 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9348 2024-09-21T14:59:04.361 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.361 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:04.361 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.361 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:04.361 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158369+0000", 2024-09-21T14:59:04.361 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:04.361 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.361 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.361 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:04.361 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158369+0000", 2024-09-21T14:59:04.361 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:04.361 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.361 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.362 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:04.362 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158368+0000", 2024-09-21T14:59:04.362 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:04.362 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.362 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.362 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:04.362 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158374+0000", 2024-09-21T14:59:04.362 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0499999999999997e-06 2024-09-21T14:59:04.362 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.362 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.362 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:04.362 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158386+0000", 2024-09-21T14:59:04.362 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1579999999999999e-05 2024-09-21T14:59:04.362 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.362 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.363 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:04.363 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158396+0000", 2024-09-21T14:59:04.363 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.8260000000000005e-06 2024-09-21T14:59:04.363 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.363 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.363 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:04.363 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.171967+0000", 2024-09-21T14:59:04.363 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013571318000000001 2024-09-21T14:59:04.363 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.363 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.363 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:04.363 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172034+0000", 2024-09-21T14:59:04.363 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.656e-05 2024-09-21T14:59:04.363 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.364 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.364 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:04.364 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172258+0000", 2024-09-21T14:59:04.364 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00022395999999999999 2024-09-21T14:59:04.364 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:04.364 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:04.364 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:04.364 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.364 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.364 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9349 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 155648~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:04.364 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158403+0000", 2024-09-21T14:59:04.364 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 25.181147189000001, 2024-09-21T14:59:04.364 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014229851, 2024-09-21T14:59:04.365 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:04.365 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:04.365 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:04.365 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:04.365 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:04.365 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9349 2024-09-21T14:59:04.365 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.365 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:04.365 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.365 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:04.365 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158403+0000", 2024-09-21T14:59:04.365 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:04.365 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.365 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.365 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:04.366 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158403+0000", 2024-09-21T14:59:04.366 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:04.366 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.366 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.366 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:04.366 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158402+0000", 2024-09-21T14:59:04.366 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:04.366 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.366 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.366 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:04.366 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158406+0000", 2024-09-21T14:59:04.366 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1339999999999997e-06 2024-09-21T14:59:04.366 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.367 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.367 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:04.367 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158416+0000", 2024-09-21T14:59:04.367 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3149999999999998e-06 2024-09-21T14:59:04.367 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.367 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.367 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:04.367 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158439+0000", 2024-09-21T14:59:04.367 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3255999999999999e-05 2024-09-21T14:59:04.367 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.367 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.367 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:04.367 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172305+0000", 2024-09-21T14:59:04.368 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013866603999999999 2024-09-21T14:59:04.368 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.368 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.368 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:04.368 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172413+0000", 2024-09-21T14:59:04.368 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000107664 2024-09-21T14:59:04.368 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.368 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.368 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:04.368 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172633+0000", 2024-09-21T14:59:04.368 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021950800000000001 2024-09-21T14:59:04.368 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:04.368 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:04.368 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:04.368 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.369 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.369 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9350 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 294912~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:04.369 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158448+0000", 2024-09-21T14:59:04.369 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 25.181102318000001, 2024-09-21T14:59:04.369 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014471525000000001, 2024-09-21T14:59:04.369 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:04.369 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:04.369 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:04.369 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:04.369 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:04.369 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9350 2024-09-21T14:59:04.369 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.369 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:04.369 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.369 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:04.370 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158448+0000", 2024-09-21T14:59:04.370 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:04.370 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.370 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.370 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:04.370 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158448+0000", 2024-09-21T14:59:04.370 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:04.370 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.370 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.370 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:04.370 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158447+0000", 2024-09-21T14:59:04.370 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:04.370 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.370 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.371 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:04.371 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158452+0000", 2024-09-21T14:59:04.371 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8400000000000002e-06 2024-09-21T14:59:04.371 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.371 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.371 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:04.371 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158475+0000", 2024-09-21T14:59:04.371 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3091e-05 2024-09-21T14:59:04.371 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.371 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.371 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:04.371 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158515+0000", 2024-09-21T14:59:04.371 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0265000000000002e-05 2024-09-21T14:59:04.371 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.372 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.372 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:04.372 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172682+0000", 2024-09-21T14:59:04.372 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014166761999999999 2024-09-21T14:59:04.372 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.372 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.372 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:04.372 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172752+0000", 2024-09-21T14:59:04.372 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9698999999999998e-05 2024-09-21T14:59:04.372 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.372 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.372 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:04.372 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172919+0000", 2024-09-21T14:59:04.372 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016759199999999999 2024-09-21T14:59:04.372 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:04.373 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:04.373 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:04.373 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.373 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.373 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9351 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 434176~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:04.373 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158526+0000", 2024-09-21T14:59:04.373 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 25.1810236, 2024-09-21T14:59:04.373 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014662107000000001, 2024-09-21T14:59:04.373 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:04.373 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:04.373 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:04.373 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:04.373 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:04.373 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9351 2024-09-21T14:59:04.374 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.374 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:04.374 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.374 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:04.374 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158526+0000", 2024-09-21T14:59:04.374 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:04.374 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.374 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.374 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:04.374 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158526+0000", 2024-09-21T14:59:04.374 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:04.374 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.374 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.375 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:04.375 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158525+0000", 2024-09-21T14:59:04.375 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:04.375 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.375 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.375 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:04.375 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158531+0000", 2024-09-21T14:59:04.375 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5840000000000002e-06 2024-09-21T14:59:04.375 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.375 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.375 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:04.375 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158564+0000", 2024-09-21T14:59:04.375 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3195000000000003e-05 2024-09-21T14:59:04.375 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.375 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.376 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:04.376 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158577+0000", 2024-09-21T14:59:04.376 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2330000000000001e-05 2024-09-21T14:59:04.376 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.376 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.376 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:04.376 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172956+0000", 2024-09-21T14:59:04.376 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014379438 2024-09-21T14:59:04.376 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.376 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.376 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:04.376 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173020+0000", 2024-09-21T14:59:04.376 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4401999999999998e-05 2024-09-21T14:59:04.376 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.377 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.377 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:04.377 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173188+0000", 2024-09-21T14:59:04.377 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016804 2024-09-21T14:59:04.377 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:04.377 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:04.377 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:04.377 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.377 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.377 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9379 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 24576~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:04.377 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158586+0000", 2024-09-21T14:59:04.377 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 25.180964339999999, 2024-09-21T14:59:04.377 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014865173000000001, 2024-09-21T14:59:04.377 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:04.378 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:04.378 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:04.378 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:04.378 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:04.378 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9379 2024-09-21T14:59:04.378 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.378 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:04.378 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.378 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:04.378 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158586+0000", 2024-09-21T14:59:04.378 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:04.378 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.378 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.378 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:04.379 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158586+0000", 2024-09-21T14:59:04.379 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:04.379 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.379 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.379 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:04.379 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158585+0000", 2024-09-21T14:59:04.379 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:04.379 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.379 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.379 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:04.379 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158590+0000", 2024-09-21T14:59:04.379 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5820000000000001e-06 2024-09-21T14:59:04.379 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.379 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.380 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:04.380 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158676+0000", 2024-09-21T14:59:04.380 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5075999999999997e-05 2024-09-21T14:59:04.380 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.380 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.380 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:04.380 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158695+0000", 2024-09-21T14:59:04.380 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9137999999999999e-05 2024-09-21T14:59:04.380 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.380 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.380 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:04.380 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173219+0000", 2024-09-21T14:59:04.380 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014524034999999999 2024-09-21T14:59:04.381 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.381 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.381 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:04.381 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173276+0000", 2024-09-21T14:59:04.381 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7442000000000003e-05 2024-09-21T14:59:04.381 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.381 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.381 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:04.381 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173451+0000", 2024-09-21T14:59:04.381 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017464800000000001 2024-09-21T14:59:04.381 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:04.381 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:04.381 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:04.381 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.382 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.382 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9380 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 163840~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:04.382 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158706+0000", 2024-09-21T14:59:04.382 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 25.180843830000001, 2024-09-21T14:59:04.382 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015096381000000001, 2024-09-21T14:59:04.382 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:04.382 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:04.382 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:04.382 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:04.382 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:04.382 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9380 2024-09-21T14:59:04.382 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.382 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:04.383 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.383 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:04.383 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158706+0000", 2024-09-21T14:59:04.383 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:04.383 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.383 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.383 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:04.383 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158706+0000", 2024-09-21T14:59:04.383 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:04.383 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.383 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.383 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:04.383 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158705+0000", 2024-09-21T14:59:04.384 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:04.384 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.384 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.384 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:04.384 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158712+0000", 2024-09-21T14:59:04.384 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7240000000000003e-06 2024-09-21T14:59:04.384 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.384 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.384 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:04.384 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158755+0000", 2024-09-21T14:59:04.384 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2953e-05 2024-09-21T14:59:04.384 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.384 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.384 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:04.385 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158765+0000", 2024-09-21T14:59:04.385 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0149000000000001e-05 2024-09-21T14:59:04.385 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.385 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.385 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:04.385 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173492+0000", 2024-09-21T14:59:04.385 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014727105000000001 2024-09-21T14:59:04.385 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.385 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.385 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:04.385 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173564+0000", 2024-09-21T14:59:04.385 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1789e-05 2024-09-21T14:59:04.385 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.385 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.385 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:04.386 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173803+0000", 2024-09-21T14:59:04.386 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00023867100000000001 2024-09-21T14:59:04.386 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:04.386 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:04.386 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:04.386 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.386 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.386 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9381 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 303104~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:04.386 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158774+0000", 2024-09-21T14:59:04.386 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 25.180775749999999, 2024-09-21T14:59:04.386 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015309095999999999, 2024-09-21T14:59:04.386 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:04.386 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:04.386 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:04.387 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:04.387 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:04.387 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9381 2024-09-21T14:59:04.387 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.387 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:04.387 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.387 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:04.387 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158774+0000", 2024-09-21T14:59:04.387 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:04.387 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.387 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.387 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:04.387 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158774+0000", 2024-09-21T14:59:04.388 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:04.388 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.388 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.388 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:04.388 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158773+0000", 2024-09-21T14:59:04.388 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:04.388 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.388 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.388 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:04.388 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158779+0000", 2024-09-21T14:59:04.388 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6999999999999996e-06 2024-09-21T14:59:04.388 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.388 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.389 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:04.389 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158790+0000", 2024-09-21T14:59:04.389 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0594000000000001e-05 2024-09-21T14:59:04.389 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.389 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.389 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:04.389 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158798+0000", 2024-09-21T14:59:04.389 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5860000000000005e-06 2024-09-21T14:59:04.389 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.389 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.389 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:04.389 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173829+0000", 2024-09-21T14:59:04.389 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015030349 2024-09-21T14:59:04.390 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.390 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.390 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:04.390 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173918+0000", 2024-09-21T14:59:04.390 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9375000000000004e-05 2024-09-21T14:59:04.390 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.390 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.390 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:04.390 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.174083+0000", 2024-09-21T14:59:04.390 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016536200000000001 2024-09-21T14:59:04.390 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:04.390 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:04.390 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:04.390 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.391 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.391 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9382 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 442368~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:04.391 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158805+0000", 2024-09-21T14:59:04.391 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 25.180744845, 2024-09-21T14:59:04.391 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015592623, 2024-09-21T14:59:04.391 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:04.391 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:04.391 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:04.391 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:04.391 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:04.391 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9382 2024-09-21T14:59:04.391 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.391 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:04.391 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.392 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:04.392 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158805+0000", 2024-09-21T14:59:04.392 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:04.392 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.392 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.392 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:04.392 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158805+0000", 2024-09-21T14:59:04.392 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:04.392 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.392 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.392 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:04.392 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158805+0000", 2024-09-21T14:59:04.392 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:04.392 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.393 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.393 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:04.393 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158808+0000", 2024-09-21T14:59:04.393 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9179999999999996e-06 2024-09-21T14:59:04.393 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.393 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.393 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:04.393 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158817+0000", 2024-09-21T14:59:04.393 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9299999999999992e-06 2024-09-21T14:59:04.393 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.393 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.393 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:04.393 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158825+0000", 2024-09-21T14:59:04.393 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6739999999999997e-06 2024-09-21T14:59:04.393 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.394 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.394 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:04.394 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.174122+0000", 2024-09-21T14:59:04.394 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.01529694 2024-09-21T14:59:04.394 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.394 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.394 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:04.394 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.174185+0000", 2024-09-21T14:59:04.394 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3015000000000006e-05 2024-09-21T14:59:04.394 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.394 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.394 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:04.394 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.174398+0000", 2024-09-21T14:59:04.395 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021274799999999999 2024-09-21T14:59:04.395 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:04.395 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:04.395 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:04.395 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.395 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.395 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22699 3.19 3:98de45b8:::rbd_data.11366acc894.0000000000000000:19 [read 638976~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:04.395 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.140173+0000", 2024-09-21T14:59:04.395 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.199376748000001, 2024-09-21T14:59:04.395 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013511366, 2024-09-21T14:59:04.395 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:04.395 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:04.395 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:04.395 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:04.396 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:04.396 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22699 2024-09-21T14:59:04.396 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.396 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:04.396 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.396 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:04.396 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140173+0000", 2024-09-21T14:59:04.396 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:04.396 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.396 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.396 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:04.396 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140173+0000", 2024-09-21T14:59:04.396 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:04.396 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.397 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.397 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:04.397 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140173+0000", 2024-09-21T14:59:04.397 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:04.397 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.397 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.397 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:04.397 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140175+0000", 2024-09-21T14:59:04.397 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5859999999999999e-06 2024-09-21T14:59:04.397 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.397 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.397 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:04.397 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140181+0000", 2024-09-21T14:59:04.398 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0340000000000002e-06 2024-09-21T14:59:04.398 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.398 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.398 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:04.398 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140186+0000", 2024-09-21T14:59:04.398 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.04e-06 2024-09-21T14:59:04.398 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.398 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.398 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:04.398 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.153426+0000", 2024-09-21T14:59:04.398 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013240001 2024-09-21T14:59:04.398 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.398 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.398 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:04.399 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.153513+0000", 2024-09-21T14:59:04.399 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6038999999999994e-05 2024-09-21T14:59:04.399 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.399 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.399 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:04.399 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.153685+0000", 2024-09-21T14:59:04.399 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000172067 2024-09-21T14:59:04.399 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:04.399 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:04.399 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:04.399 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.399 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.399 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22700 3.19 3:98de45b8:::rbd_data.11366acc894.0000000000000000:19 [read 704512~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:04.400 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.140191+0000", 2024-09-21T14:59:04.400 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.199359231000001, 2024-09-21T14:59:04.400 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013770486, 2024-09-21T14:59:04.400 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:04.400 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:04.400 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:04.400 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:04.400 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:04.400 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22700 2024-09-21T14:59:04.400 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.400 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:04.400 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.400 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:04.400 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140191+0000", 2024-09-21T14:59:04.401 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:04.401 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.401 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.401 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:04.401 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140191+0000", 2024-09-21T14:59:04.401 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:04.401 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.401 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.401 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:04.401 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140190+0000", 2024-09-21T14:59:04.401 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:04.401 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.401 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.401 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:04.402 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140199+0000", 2024-09-21T14:59:04.402 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3340000000000002e-06 2024-09-21T14:59:04.402 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.402 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.402 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:04.402 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140205+0000", 2024-09-21T14:59:04.402 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3820000000000001e-06 2024-09-21T14:59:04.402 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.402 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.402 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:04.402 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140210+0000", 2024-09-21T14:59:04.402 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0969999999999996e-06 2024-09-21T14:59:04.402 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.402 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.403 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:04.403 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.153713+0000", 2024-09-21T14:59:04.403 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013502713 2024-09-21T14:59:04.403 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.403 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.403 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:04.403 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.153786+0000", 2024-09-21T14:59:04.403 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3022999999999995e-05 2024-09-21T14:59:04.403 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.403 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.403 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:04.403 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.153961+0000", 2024-09-21T14:59:04.403 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000175274 2024-09-21T14:59:04.403 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:04.404 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:04.404 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:04.404 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.404 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.404 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22701 3.19 3:98de45b8:::rbd_data.11366acc894.0000000000000000:19 [read 770048~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:04.404 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.140215+0000", 2024-09-21T14:59:04.404 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.199335414, 2024-09-21T14:59:04.404 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014072497, 2024-09-21T14:59:04.404 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:04.404 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:04.404 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:04.404 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:04.404 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:04.404 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22701 2024-09-21T14:59:04.405 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.405 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:04.405 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.405 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:04.405 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140215+0000", 2024-09-21T14:59:04.405 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:04.405 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.405 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.405 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:04.405 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140215+0000", 2024-09-21T14:59:04.405 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:04.405 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.405 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.405 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:04.405 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140214+0000", 2024-09-21T14:59:04.406 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:04.406 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.406 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.406 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:04.406 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140217+0000", 2024-09-21T14:59:04.406 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.74e-06 2024-09-21T14:59:04.406 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.406 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.406 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:04.406 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140223+0000", 2024-09-21T14:59:04.406 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0929999999999999e-06 2024-09-21T14:59:04.406 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.406 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.406 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:04.407 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140228+0000", 2024-09-21T14:59:04.407 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2279999999999998e-06 2024-09-21T14:59:04.407 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.407 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.407 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:04.407 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.153991+0000", 2024-09-21T14:59:04.407 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013762329 2024-09-21T14:59:04.407 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.407 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.407 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:04.407 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.154098+0000", 2024-09-21T14:59:04.407 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000107117 2024-09-21T14:59:04.407 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.407 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.407 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:04.408 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.154287+0000", 2024-09-21T14:59:04.408 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000189431 2024-09-21T14:59:04.408 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:04.408 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:04.408 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:04.408 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.408 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.408 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22702 3.19 3:98de45b8:::rbd_data.11366acc894.0000000000000000:19 [read 835584~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:04.408 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.140232+0000", 2024-09-21T14:59:04.408 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.199317561999999, 2024-09-21T14:59:04.408 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015338742000000001, 2024-09-21T14:59:04.408 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:04.408 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:04.408 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:04.408 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:04.409 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:04.409 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22702 2024-09-21T14:59:04.409 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.409 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:04.409 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.409 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:04.409 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140232+0000", 2024-09-21T14:59:04.409 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:04.409 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.409 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.409 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:04.409 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140232+0000", 2024-09-21T14:59:04.409 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:04.409 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.409 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.410 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:04.410 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140232+0000", 2024-09-21T14:59:04.410 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:04.410 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.410 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.410 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:04.410 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140234+0000", 2024-09-21T14:59:04.410 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.402e-06 2024-09-21T14:59:04.410 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.410 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.410 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:04.410 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140241+0000", 2024-09-21T14:59:04.410 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.054e-06 2024-09-21T14:59:04.410 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.411 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.411 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:04.411 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140246+0000", 2024-09-21T14:59:04.411 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0050000000000004e-06 2024-09-21T14:59:04.411 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.411 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.411 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:04.411 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.154315+0000", 2024-09-21T14:59:04.411 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014069440000000001 2024-09-21T14:59:04.411 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.411 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.411 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:04.411 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.154383+0000", 2024-09-21T14:59:04.411 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8151000000000005e-05 2024-09-21T14:59:04.411 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.412 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.412 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:04.412 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155571+0000", 2024-09-21T14:59:04.412 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001188029 2024-09-21T14:59:04.412 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:04.412 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:04.412 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:04.412 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.412 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.412 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22703 3.19 3:98de45b8:::rbd_data.11366acc894.0000000000000000:19 [read 901120~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:04.412 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.140250+0000", 2024-09-21T14:59:04.412 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.199300404000001, 2024-09-21T14:59:04.412 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014992169, 2024-09-21T14:59:04.412 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:04.413 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:04.413 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:04.413 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:04.413 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:04.413 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22703 2024-09-21T14:59:04.413 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.413 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:04.413 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.413 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:04.413 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140250+0000", 2024-09-21T14:59:04.413 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:04.413 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.413 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.413 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:04.414 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140250+0000", 2024-09-21T14:59:04.414 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:04.414 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.414 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.414 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:04.414 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140249+0000", 2024-09-21T14:59:04.414 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:04.414 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.414 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.414 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:04.414 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140251+0000", 2024-09-21T14:59:04.414 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.136e-06 2024-09-21T14:59:04.414 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.414 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.415 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:04.415 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140257+0000", 2024-09-21T14:59:04.415 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9610000000000002e-06 2024-09-21T14:59:04.415 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.415 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.415 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:04.415 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140262+0000", 2024-09-21T14:59:04.415 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4549999999999997e-06 2024-09-21T14:59:04.415 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.415 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.415 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:04.415 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155006+0000", 2024-09-21T14:59:04.415 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014744189 2024-09-21T14:59:04.415 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.416 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.416 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:04.416 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155081+0000", 2024-09-21T14:59:04.416 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4548000000000005e-05 2024-09-21T14:59:04.416 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.416 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.416 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:04.416 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155242+0000", 2024-09-21T14:59:04.416 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000161204 2024-09-21T14:59:04.416 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:04.416 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:04.416 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:04.416 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.416 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.417 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22704 3.19 3:98de45b8:::rbd_data.11366acc894.0000000000000000:19 [read 966656~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:04.417 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.140266+0000", 2024-09-21T14:59:04.417 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.199284087000001, 2024-09-21T14:59:04.417 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015223443999999999, 2024-09-21T14:59:04.417 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:04.417 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:04.417 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:04.417 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:04.417 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:04.417 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22704 2024-09-21T14:59:04.417 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.417 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:04.417 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.417 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:04.418 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140266+0000", 2024-09-21T14:59:04.418 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:04.418 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.418 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.418 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:04.418 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140266+0000", 2024-09-21T14:59:04.418 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:04.418 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.418 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.418 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:04.418 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140266+0000", 2024-09-21T14:59:04.418 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:04.418 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.418 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.419 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:04.419 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140268+0000", 2024-09-21T14:59:04.419 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.475e-06 2024-09-21T14:59:04.419 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.419 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.419 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:04.419 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140274+0000", 2024-09-21T14:59:04.419 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.319e-06 2024-09-21T14:59:04.419 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.419 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.419 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:04.419 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140279+0000", 2024-09-21T14:59:04.419 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.882e-06 2024-09-21T14:59:04.419 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.420 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.420 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:04.420 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155264+0000", 2024-09-21T14:59:04.420 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014984655 2024-09-21T14:59:04.420 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.420 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.420 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:04.420 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155338+0000", 2024-09-21T14:59:04.420 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4091999999999995e-05 2024-09-21T14:59:04.420 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.420 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.420 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:04.420 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155489+0000", 2024-09-21T14:59:04.420 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000151349 2024-09-21T14:59:04.420 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:04.421 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:04.421 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:04.421 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.421 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.421 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22705 3.19 3:98de45b8:::rbd_data.11366acc894.0000000000000000:19 [read 1032192~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:04.421 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.140284+0000", 2024-09-21T14:59:04.421 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.199266229999999, 2024-09-21T14:59:04.421 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015467896, 2024-09-21T14:59:04.421 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:04.421 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:04.421 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:04.421 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:04.421 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:04.421 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22705 2024-09-21T14:59:04.421 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.422 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:04.422 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.422 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:04.422 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140284+0000", 2024-09-21T14:59:04.422 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:04.422 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.422 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.422 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:04.422 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140284+0000", 2024-09-21T14:59:04.422 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:04.422 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.422 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.422 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:04.423 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140283+0000", 2024-09-21T14:59:04.423 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:04.423 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.423 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.423 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:04.423 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140286+0000", 2024-09-21T14:59:04.423 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7130000000000001e-06 2024-09-21T14:59:04.423 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.423 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.423 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:04.423 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140292+0000", 2024-09-21T14:59:04.423 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3430000000000001e-06 2024-09-21T14:59:04.423 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.423 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.424 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:04.424 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140298+0000", 2024-09-21T14:59:04.424 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9730000000000002e-06 2024-09-21T14:59:04.424 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.424 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.424 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:04.424 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155510+0000", 2024-09-21T14:59:04.424 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015211738000000001 2024-09-21T14:59:04.424 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.424 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.424 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:04.424 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155598+0000", 2024-09-21T14:59:04.424 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7822e-05 2024-09-21T14:59:04.424 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.425 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.425 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:04.425 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155752+0000", 2024-09-21T14:59:04.425 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0001537 2024-09-21T14:59:04.425 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:04.425 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:04.425 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:04.425 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.425 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.425 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:37892 3.1b 3:d963c97f:::rbd_data.11365d19aa81.00000000000000be:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-21T14:59:04.425 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:04.029946+0000", 2024-09-21T14:59:04.425 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.309604463, 2024-09-21T14:59:04.425 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.017686021999999999, 2024-09-21T14:59:04.425 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:04.425 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:04.426 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:04.426 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:04.426 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:04.426 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 37892 2024-09-21T14:59:04.426 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.426 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:04.426 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.426 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:04.426 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.029946+0000", 2024-09-21T14:59:04.426 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:04.426 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.426 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.426 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:04.426 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.029946+0000", 2024-09-21T14:59:04.426 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:04.427 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.427 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.427 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:04.427 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.029944+0000", 2024-09-21T14:59:04.427 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T14:59:04.427 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.427 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.427 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:04.427 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.029953+0000", 2024-09-21T14:59:04.427 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7589999999999992e-06 2024-09-21T14:59:04.427 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.427 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.427 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:04.427 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.029971+0000", 2024-09-21T14:59:04.428 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7795000000000002e-05 2024-09-21T14:59:04.428 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.428 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.428 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:04.428 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.029984+0000", 2024-09-21T14:59:04.428 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.27e-05 2024-09-21T14:59:04.428 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.428 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.428 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:04.428 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.030072+0000", 2024-09-21T14:59:04.428 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8082000000000001e-05 2024-09-21T14:59:04.428 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.428 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.428 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:04.428 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.030276+0000", 2024-09-21T14:59:04.429 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00020375699999999999 2024-09-21T14:59:04.429 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.429 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.429 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-21T14:59:04.429 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.031634+0000", 2024-09-21T14:59:04.429 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0013586049999999999 2024-09-21T14:59:04.429 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.429 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.429 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:59:04.429 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.037554+0000", 2024-09-21T14:59:04.429 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0059197449999999997 2024-09-21T14:59:04.429 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.429 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.429 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:04.429 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.047490+0000", 2024-09-21T14:59:04.429 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0099361680000000004 2024-09-21T14:59:04.430 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.430 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.430 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:59:04.430 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.047569+0000", 2024-09-21T14:59:04.430 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8298000000000001e-05 2024-09-21T14:59:04.430 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.430 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.430 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:04.430 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.047632+0000", 2024-09-21T14:59:04.430 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3006e-05 2024-09-21T14:59:04.430 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:04.430 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:04.430 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:04.430 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:04.430 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:04.431 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:59:04.433 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T14:59:04.433 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:59:04.428+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:59:04.434 INFO:teuthology.orchestra.run.smithi032.stdout:1285 write 0xc0701b4 thru 0xc07319c (0x2fe9 bytes) 2024-09-21T14:59:04.434 INFO:teuthology.orchestra.run.smithi032.stdout:1287 read 0x49d1c86 thru 0x49d28c0 (0xc3b bytes) 2024-09-21T14:59:04.434 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:04.434 INFO:teuthology.orchestra.run.smithi032.stdout:1291 punch from 0x415f37f to 0x416eade, (0xf75f bytes) 2024-09-21T14:59:04.434 INFO:teuthology.orchestra.run.smithi032.stdout:1292 write 0xa2c252d thru 0xa2d16dd (0xf1b1 bytes) 2024-09-21T14:59:04.434 INFO:teuthology.orchestra.run.smithi032.stdout:1294 read 0x3548823 thru 0x3548cc7 (0x4a5 bytes) 2024-09-21T14:59:04.434 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:04.434 INFO:teuthology.orchestra.run.smithi032.stdout:1296 read 0x1ba6ca7 thru 0x1bb6b8a (0xfee4 bytes) 2024-09-21T14:59:04.435 INFO:teuthology.orchestra.run.smithi032.stdout:1297 punch from 0x3688017 to 0x368ab99, (0x2b82 bytes) 2024-09-21T14:59:04.435 INFO:teuthology.orchestra.run.smithi032.stdout:1298 writesame 0xa9566a2 thru 0xa95cfe3 data_size 0x1b(0x6942 bytes) 2024-09-21T14:59:04.435 INFO:teuthology.orchestra.run.smithi032.stdout:1299 write 0x2552931 thru 0x255e325 (0xb9f5 bytes) 2024-09-21T14:59:04.435 INFO:teuthology.orchestra.run.smithi032.stdout:1301 writesame 0x13b9806 thru 0x13c0de0 data_size 0x59(0x75db bytes) 2024-09-21T14:59:04.435 INFO:teuthology.orchestra.run.smithi032.stdout:1302 writesame 0xc88e2e2 thru 0xc898ec3 data_size 0x31(0xabe2 bytes) 2024-09-21T14:59:04.435 INFO:teuthology.orchestra.run.smithi032.stdout:1303 read 0x410ce4b thru 0x410edfb (0x1fb1 bytes) 2024-09-21T14:59:04.435 INFO:teuthology.orchestra.run.smithi032.stdout:1304 write 0x123c90f thru 0x1240770 (0x3e62 bytes) 2024-09-21T14:59:04.439 INFO:teuthology.orchestra.run.smithi032.stdout:1305 clone 12 order 20 su 16384 sc 4 2024-09-21T14:59:04.533 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:59:04.528+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:59:04.543 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:59:04.543 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:59:04.543 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T14:59:04.543 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:59:04.553 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T14:59:04.636 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:59:04.633+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:59:04.863 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:59:04.863 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:59:04.863 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T14:59:04.863 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T14:59:04.864 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:59:04.864 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T14:59:04.865 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:59:04.861+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:59:04.964 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:59:04.960+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:59:04.984 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:59:04.984 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T14:59:04.984 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T14:59:04.984 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T14:59:04.984 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.984 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22719 3.1e 3:79caef05:::rbd_data.11366acc894.0000000000000001:19 [read 901120~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:04.984 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.140027+0000", 2024-09-21T14:59:04.984 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.841612393, 2024-09-21T14:59:04.984 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015081021, 2024-09-21T14:59:04.985 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:04.985 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:04.985 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:04.985 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:04.985 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:04.985 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22719 2024-09-21T14:59:04.985 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.985 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:04.985 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.985 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:04.985 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140027+0000", 2024-09-21T14:59:04.985 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:04.985 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.986 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.986 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:04.986 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140027+0000", 2024-09-21T14:59:04.986 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:04.986 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.986 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.986 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:04.986 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140026+0000", 2024-09-21T14:59:04.986 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:04.986 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.986 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.986 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:04.986 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140042+0000", 2024-09-21T14:59:04.986 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5381999999999999e-05 2024-09-21T14:59:04.987 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.987 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.987 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:04.987 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140051+0000", 2024-09-21T14:59:04.987 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3500000000000003e-06 2024-09-21T14:59:04.987 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.987 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.987 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:04.987 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140066+0000", 2024-09-21T14:59:04.987 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4582e-05 2024-09-21T14:59:04.987 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.987 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.987 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:04.987 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.154602+0000", 2024-09-21T14:59:04.988 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014535881 2024-09-21T14:59:04.988 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.988 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.988 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:04.988 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.154674+0000", 2024-09-21T14:59:04.988 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2303000000000002e-05 2024-09-21T14:59:04.988 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.988 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.988 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:04.988 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155108+0000", 2024-09-21T14:59:04.988 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00043396500000000002 2024-09-21T14:59:04.988 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:04.988 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:04.988 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:04.989 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.989 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.989 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22720 3.1e 3:79caef05:::rbd_data.11366acc894.0000000000000001:19 [read 966656~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:04.989 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.140073+0000", 2024-09-21T14:59:04.989 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.841565828, 2024-09-21T14:59:04.989 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015292312000000001, 2024-09-21T14:59:04.989 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:04.989 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:04.989 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:04.989 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:04.989 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:04.989 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22720 2024-09-21T14:59:04.989 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.989 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:04.990 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.990 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:04.990 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140073+0000", 2024-09-21T14:59:04.990 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:04.990 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.990 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.990 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:04.990 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140073+0000", 2024-09-21T14:59:04.990 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:04.990 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.990 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.990 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:04.990 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140073+0000", 2024-09-21T14:59:04.990 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:04.991 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.991 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.991 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:04.991 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140076+0000", 2024-09-21T14:59:04.991 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4999999999999999e-06 2024-09-21T14:59:04.991 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.991 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.991 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:04.991 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140084+0000", 2024-09-21T14:59:04.991 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1310000000000006e-06 2024-09-21T14:59:04.991 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.991 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.991 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:04.991 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140091+0000", 2024-09-21T14:59:04.991 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.674e-06 2024-09-21T14:59:04.992 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.992 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.992 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:04.992 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155135+0000", 2024-09-21T14:59:04.992 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015044224 2024-09-21T14:59:04.992 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.992 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.992 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:04.992 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155205+0000", 2024-09-21T14:59:04.992 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9751000000000004e-05 2024-09-21T14:59:04.992 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.992 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.992 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:04.992 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155366+0000", 2024-09-21T14:59:04.992 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016073899999999999 2024-09-21T14:59:04.992 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:04.992 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:04.993 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:04.993 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.993 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.993 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22721 3.1e 3:79caef05:::rbd_data.11366acc894.0000000000000001:19 [read 1032192~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:04.993 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.140096+0000", 2024-09-21T14:59:04.993 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.841542842999999, 2024-09-21T14:59:04.993 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015563158000000001, 2024-09-21T14:59:04.993 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:04.993 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:04.993 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:04.993 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:04.993 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:04.993 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22721 2024-09-21T14:59:04.993 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.994 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:04.994 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.994 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:04.994 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140096+0000", 2024-09-21T14:59:04.994 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:04.994 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.994 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.994 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:04.994 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140096+0000", 2024-09-21T14:59:04.994 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:04.994 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.994 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.995 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:04.995 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140096+0000", 2024-09-21T14:59:04.995 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:04.995 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.995 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.995 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:04.995 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140099+0000", 2024-09-21T14:59:04.995 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.5080000000000001e-06 2024-09-21T14:59:04.995 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.995 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.995 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:04.995 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140106+0000", 2024-09-21T14:59:04.995 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8630000000000002e-06 2024-09-21T14:59:04.995 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.995 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.995 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:04.996 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140112+0000", 2024-09-21T14:59:04.996 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6509999999999998e-06 2024-09-21T14:59:04.996 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.996 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.996 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:04.996 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155392+0000", 2024-09-21T14:59:04.996 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015280375000000001 2024-09-21T14:59:04.996 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.996 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.996 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:04.996 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155459+0000", 2024-09-21T14:59:04.996 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7040999999999997e-05 2024-09-21T14:59:04.996 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.996 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.996 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:04.997 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155660+0000", 2024-09-21T14:59:04.997 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000200115 2024-09-21T14:59:04.997 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:04.997 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:04.997 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:04.997 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.997 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.997 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22865 3.1e 3:7ca947c4:::rbd_data.11366acc894.000000000000000b:19 [read 393216~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:04.997 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.142757+0000", 2024-09-21T14:59:04.997 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.838882172, 2024-09-21T14:59:04.997 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014907012000000001, 2024-09-21T14:59:04.997 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:04.997 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:04.997 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:04.997 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:04.997 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:04.998 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22865 2024-09-21T14:59:04.998 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.998 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:04.998 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.998 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:04.998 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.142757+0000", 2024-09-21T14:59:04.998 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:04.998 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.998 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.998 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:04.998 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.142757+0000", 2024-09-21T14:59:04.998 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:04.998 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.998 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.998 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:04.999 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.142757+0000", 2024-09-21T14:59:04.999 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:04.999 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.999 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.999 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:04.999 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.142759+0000", 2024-09-21T14:59:04.999 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3800000000000001e-06 2024-09-21T14:59:04.999 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.999 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:04.999 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:04.999 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.142765+0000", 2024-09-21T14:59:04.999 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7819999999999999e-06 2024-09-21T14:59:04.999 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:04.999 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.000 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:05.000 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.142770+0000", 2024-09-21T14:59:05.000 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9230000000000001e-06 2024-09-21T14:59:05.000 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.000 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.000 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:05.000 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.157393+0000", 2024-09-21T14:59:05.000 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014623662000000001 2024-09-21T14:59:05.000 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.000 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.000 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:05.000 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.157469+0000", 2024-09-21T14:59:05.000 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5383000000000007e-05 2024-09-21T14:59:05.000 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.000 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.000 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:05.001 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.157664+0000", 2024-09-21T14:59:05.001 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019522100000000001 2024-09-21T14:59:05.001 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:05.001 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:05.001 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:05.001 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.001 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.001 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22866 3.1e 3:7ca947c4:::rbd_data.11366acc894.000000000000000b:19 [read 458752~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:05.001 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.142774+0000", 2024-09-21T14:59:05.001 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.838865062, 2024-09-21T14:59:05.001 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.01521982, 2024-09-21T14:59:05.001 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:05.001 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:05.001 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:05.002 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:05.002 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:05.002 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22866 2024-09-21T14:59:05.002 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.002 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:05.002 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.002 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:05.002 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.142774+0000", 2024-09-21T14:59:05.002 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:05.002 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.002 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.002 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:05.002 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.142774+0000", 2024-09-21T14:59:05.002 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:05.003 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.003 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.003 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:05.003 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.142774+0000", 2024-09-21T14:59:05.003 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:05.003 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.003 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.003 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:05.003 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.142776+0000", 2024-09-21T14:59:05.003 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.588e-06 2024-09-21T14:59:05.003 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.003 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.003 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:05.003 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.142783+0000", 2024-09-21T14:59:05.004 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2729999999999999e-06 2024-09-21T14:59:05.004 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.004 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.004 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:05.004 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.142789+0000", 2024-09-21T14:59:05.004 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3160000000000002e-06 2024-09-21T14:59:05.004 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.004 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.004 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:05.004 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.157705+0000", 2024-09-21T14:59:05.004 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014915586999999999 2024-09-21T14:59:05.004 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.004 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.004 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:05.004 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.157792+0000", 2024-09-21T14:59:05.004 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7322000000000002e-05 2024-09-21T14:59:05.005 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.005 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.005 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:05.005 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.157994+0000", 2024-09-21T14:59:05.005 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000202093 2024-09-21T14:59:05.005 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:05.005 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:05.005 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:05.005 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.005 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.005 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22867 3.1e 3:7ca947c4:::rbd_data.11366acc894.000000000000000b:19 [read 524288~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:05.005 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.142794+0000", 2024-09-21T14:59:05.005 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.838845321999999, 2024-09-21T14:59:05.006 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015569725, 2024-09-21T14:59:05.006 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:05.006 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:05.006 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:05.006 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:05.006 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:05.006 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22867 2024-09-21T14:59:05.006 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.006 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:05.006 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.006 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:05.006 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.142794+0000", 2024-09-21T14:59:05.006 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:05.006 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.007 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.007 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:05.007 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.142794+0000", 2024-09-21T14:59:05.007 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:05.007 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.007 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.007 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:05.007 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.142793+0000", 2024-09-21T14:59:05.007 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:05.007 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.007 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.007 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:05.007 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.142801+0000", 2024-09-21T14:59:05.007 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8509999999999999e-06 2024-09-21T14:59:05.008 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.008 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.008 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:05.008 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.142807+0000", 2024-09-21T14:59:05.008 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1210000000000003e-06 2024-09-21T14:59:05.008 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.008 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.008 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:05.008 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.142812+0000", 2024-09-21T14:59:05.008 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2809999999999999e-06 2024-09-21T14:59:05.008 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.008 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.008 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:05.008 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.158021+0000", 2024-09-21T14:59:05.009 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015208038 2024-09-21T14:59:05.009 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.009 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.009 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:05.009 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.158098+0000", 2024-09-21T14:59:05.009 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7888999999999999e-05 2024-09-21T14:59:05.009 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.009 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.009 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:05.009 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.158364+0000", 2024-09-21T14:59:05.009 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00026517400000000002 2024-09-21T14:59:05.009 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:05.009 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:05.009 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:05.010 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.010 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.010 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:23017 3.1e 3:7ca947c4:::rbd_data.11366acc894.000000000000000b:19 [read 229376~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:05.010 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.147628+0000", 2024-09-21T14:59:05.010 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.834011302, 2024-09-21T14:59:05.010 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014791732, 2024-09-21T14:59:05.010 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:05.010 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:05.010 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:05.010 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:05.010 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:05.010 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 23017 2024-09-21T14:59:05.010 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.010 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:05.011 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.011 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:05.011 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147628+0000", 2024-09-21T14:59:05.011 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:05.011 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.011 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.011 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:05.011 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147628+0000", 2024-09-21T14:59:05.011 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:05.011 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.011 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.011 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:05.011 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147628+0000", 2024-09-21T14:59:05.012 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:05.012 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.012 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.012 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:05.012 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147635+0000", 2024-09-21T14:59:05.012 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3459999999999996e-06 2024-09-21T14:59:05.012 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.012 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.012 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:05.012 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147642+0000", 2024-09-21T14:59:05.012 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0249999999999997e-06 2024-09-21T14:59:05.012 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.012 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.012 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:05.012 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147647+0000", 2024-09-21T14:59:05.013 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.549e-06 2024-09-21T14:59:05.013 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.013 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.013 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:05.013 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.161946+0000", 2024-09-21T14:59:05.013 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.01429997 2024-09-21T14:59:05.013 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.013 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.013 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:05.013 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.162034+0000", 2024-09-21T14:59:05.013 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7676000000000006e-05 2024-09-21T14:59:05.013 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.013 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.013 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:05.014 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.162420+0000", 2024-09-21T14:59:05.014 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00038544499999999999 2024-09-21T14:59:05.014 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:05.014 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:05.014 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:05.014 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.014 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.014 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:23018 3.1e 3:7ca947c4:::rbd_data.11366acc894.000000000000000b:19 [read 294912~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:05.014 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.147650+0000", 2024-09-21T14:59:05.014 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.833988873999999, 2024-09-21T14:59:05.014 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015515414999999999, 2024-09-21T14:59:05.014 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:05.014 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:05.014 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:05.015 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:05.015 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:05.015 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 23018 2024-09-21T14:59:05.015 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.015 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:05.015 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.015 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:05.015 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147650+0000", 2024-09-21T14:59:05.015 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:05.015 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.015 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.015 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:05.015 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147650+0000", 2024-09-21T14:59:05.015 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:05.015 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.016 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.016 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:05.016 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147650+0000", 2024-09-21T14:59:05.016 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:05.016 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.016 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.016 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:05.016 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147652+0000", 2024-09-21T14:59:05.016 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1469999999999999e-06 2024-09-21T14:59:05.016 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.016 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.016 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:05.016 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147657+0000", 2024-09-21T14:59:05.016 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3449999999999996e-06 2024-09-21T14:59:05.016 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.017 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.017 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:05.017 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147663+0000", 2024-09-21T14:59:05.017 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1309999999999997e-06 2024-09-21T14:59:05.017 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.017 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.017 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:05.017 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.162466+0000", 2024-09-21T14:59:05.017 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014803474 2024-09-21T14:59:05.017 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.017 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.017 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:05.017 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.162562+0000", 2024-09-21T14:59:05.017 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6272000000000003e-05 2024-09-21T14:59:05.018 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.018 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.018 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:05.018 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.163166+0000", 2024-09-21T14:59:05.018 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00060336000000000005 2024-09-21T14:59:05.018 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:05.018 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:05.018 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:05.018 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.018 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.018 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:23019 3.1e 3:7ca947c4:::rbd_data.11366acc894.000000000000000b:19 [read 360448~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:05.018 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.147666+0000", 2024-09-21T14:59:05.018 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.83397283, 2024-09-21T14:59:05.018 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015788706, 2024-09-21T14:59:05.018 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:05.019 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:05.019 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:05.019 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:05.019 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:05.019 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 23019 2024-09-21T14:59:05.019 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.019 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:05.019 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.019 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:05.019 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147666+0000", 2024-09-21T14:59:05.019 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:05.019 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.019 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.020 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:05.020 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147666+0000", 2024-09-21T14:59:05.020 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:05.020 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.020 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.020 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:05.020 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147666+0000", 2024-09-21T14:59:05.020 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:05.020 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.020 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.020 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:05.020 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147668+0000", 2024-09-21T14:59:05.020 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1969999999999999e-06 2024-09-21T14:59:05.021 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.021 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.021 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:05.021 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147673+0000", 2024-09-21T14:59:05.021 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2220000000000001e-06 2024-09-21T14:59:05.021 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.021 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.021 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:05.021 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147677+0000", 2024-09-21T14:59:05.021 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9620000000000004e-06 2024-09-21T14:59:05.021 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.021 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.021 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:05.021 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.163200+0000", 2024-09-21T14:59:05.022 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.01552215 2024-09-21T14:59:05.022 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.022 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.022 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:05.022 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.163294+0000", 2024-09-21T14:59:05.022 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4017000000000005e-05 2024-09-21T14:59:05.022 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.022 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.022 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:05.022 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.163455+0000", 2024-09-21T14:59:05.022 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000161478 2024-09-21T14:59:05.022 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:05.022 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:05.022 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:05.022 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.023 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.023 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:23020 3.1e 3:7ca947c4:::rbd_data.11366acc894.000000000000000b:19 [read 425984~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:05.023 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.147681+0000", 2024-09-21T14:59:05.023 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.8339581, 2024-09-21T14:59:05.023 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.016190145, 2024-09-21T14:59:05.023 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:05.023 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:05.023 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:05.023 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:05.023 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:05.023 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 23020 2024-09-21T14:59:05.023 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.023 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:05.023 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.023 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:05.024 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147681+0000", 2024-09-21T14:59:05.024 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:05.024 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.024 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.024 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:05.024 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147681+0000", 2024-09-21T14:59:05.024 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:05.024 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.024 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.024 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:05.024 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147681+0000", 2024-09-21T14:59:05.024 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:05.024 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.024 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.024 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:05.024 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147683+0000", 2024-09-21T14:59:05.025 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3470000000000001e-06 2024-09-21T14:59:05.025 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.025 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.025 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:05.025 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147689+0000", 2024-09-21T14:59:05.025 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5849999999999999e-06 2024-09-21T14:59:05.025 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.025 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.025 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:05.025 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147695+0000", 2024-09-21T14:59:05.025 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.964e-06 2024-09-21T14:59:05.025 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.025 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.025 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:05.025 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.163488+0000", 2024-09-21T14:59:05.026 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015793564999999999 2024-09-21T14:59:05.026 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.026 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.026 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:05.026 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.163532+0000", 2024-09-21T14:59:05.026 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3841000000000003e-05 2024-09-21T14:59:05.026 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.026 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.026 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:05.026 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.163871+0000", 2024-09-21T14:59:05.026 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00033914500000000001 2024-09-21T14:59:05.026 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:05.026 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:05.026 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:05.026 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.027 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.027 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:23021 3.1e 3:7ca947c4:::rbd_data.11366acc894.000000000000000b:19 [read 491520~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:05.027 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.147698+0000", 2024-09-21T14:59:05.027 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.833940702, 2024-09-21T14:59:05.027 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.016449135, 2024-09-21T14:59:05.027 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:05.027 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:05.027 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:05.027 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:05.027 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:05.027 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 23021 2024-09-21T14:59:05.027 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.027 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:05.027 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.027 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:05.028 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147698+0000", 2024-09-21T14:59:05.028 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:05.028 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.028 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.028 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:05.028 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147698+0000", 2024-09-21T14:59:05.028 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:05.028 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.028 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.028 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:05.028 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147698+0000", 2024-09-21T14:59:05.028 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:05.028 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.028 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.028 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:05.028 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147701+0000", 2024-09-21T14:59:05.029 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6299999999999998e-06 2024-09-21T14:59:05.029 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.029 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.029 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:05.029 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147706+0000", 2024-09-21T14:59:05.029 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3059999999999997e-06 2024-09-21T14:59:05.029 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.029 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.029 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:05.029 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147711+0000", 2024-09-21T14:59:05.029 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0860000000000001e-06 2024-09-21T14:59:05.029 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.029 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.029 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:05.029 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.163897+0000", 2024-09-21T14:59:05.030 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.016186282 2024-09-21T14:59:05.030 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.030 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.030 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:05.030 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.163986+0000", 2024-09-21T14:59:05.030 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8676000000000003e-05 2024-09-21T14:59:05.030 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.030 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.030 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:05.030 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.164148+0000", 2024-09-21T14:59:05.030 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016148000000000001 2024-09-21T14:59:05.030 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:05.030 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:05.031 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:05.031 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.031 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.031 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:23022 3.1e 3:7ca947c4:::rbd_data.11366acc894.000000000000000b:19 [read 557056~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:05.031 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.148289+0000", 2024-09-21T14:59:05.031 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.833349818, 2024-09-21T14:59:05.031 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.016062979000000002, 2024-09-21T14:59:05.031 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:05.031 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:05.031 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:05.031 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:05.031 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:05.031 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 23022 2024-09-21T14:59:05.031 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.031 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:05.032 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.032 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:05.032 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.148289+0000", 2024-09-21T14:59:05.032 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:05.032 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.032 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.032 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:05.032 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.148289+0000", 2024-09-21T14:59:05.032 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:05.032 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.032 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.032 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:05.032 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.148288+0000", 2024-09-21T14:59:05.032 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:05.032 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.033 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.033 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:05.033 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.148295+0000", 2024-09-21T14:59:05.033 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8340000000000001e-06 2024-09-21T14:59:05.033 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.033 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.033 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:05.033 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.148304+0000", 2024-09-21T14:59:05.033 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0129999999999999e-06 2024-09-21T14:59:05.033 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.033 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.033 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:05.033 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.148311+0000", 2024-09-21T14:59:05.033 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1439999999999997e-06 2024-09-21T14:59:05.033 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.033 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.033 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:05.034 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.164182+0000", 2024-09-21T14:59:05.034 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015870825000000002 2024-09-21T14:59:05.034 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.034 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.034 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:05.034 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.164229+0000", 2024-09-21T14:59:05.034 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6508999999999998e-05 2024-09-21T14:59:05.034 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.034 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.034 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:05.034 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.164352+0000", 2024-09-21T14:59:05.034 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000123664 2024-09-21T14:59:05.034 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:05.034 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:05.035 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:05.035 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.035 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.035 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:23116 3.1e 3:7ca947c4:::rbd_data.11366acc894.000000000000000b:19 [read 180224~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:05.035 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.150296+0000", 2024-09-21T14:59:05.035 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.831343475000001, 2024-09-21T14:59:05.035 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015003996, 2024-09-21T14:59:05.035 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:05.035 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:05.035 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:05.035 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:05.035 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:05.035 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 23116 2024-09-21T14:59:05.035 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.036 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:05.036 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.036 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:05.036 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150296+0000", 2024-09-21T14:59:05.036 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:05.036 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.036 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.036 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:05.036 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150296+0000", 2024-09-21T14:59:05.036 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:05.036 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.036 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.036 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:05.036 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150295+0000", 2024-09-21T14:59:05.036 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:05.036 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.037 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.037 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:05.037 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150298+0000", 2024-09-21T14:59:05.037 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6749999999999998e-06 2024-09-21T14:59:05.037 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.037 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.037 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:05.037 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150304+0000", 2024-09-21T14:59:05.037 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6359999999999999e-06 2024-09-21T14:59:05.037 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.037 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.037 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:05.037 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150308+0000", 2024-09-21T14:59:05.037 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6260000000000003e-06 2024-09-21T14:59:05.038 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.038 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.038 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:05.038 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.165003+0000", 2024-09-21T14:59:05.038 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014694377 2024-09-21T14:59:05.038 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.038 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.038 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:05.038 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.165093+0000", 2024-09-21T14:59:05.038 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0137999999999997e-05 2024-09-21T14:59:05.038 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.038 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.038 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:05.038 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.165300+0000", 2024-09-21T14:59:05.039 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00020688799999999999 2024-09-21T14:59:05.039 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:05.039 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:05.039 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:05.039 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.039 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.039 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:23117 3.1e 3:7ca947c4:::rbd_data.11366acc894.000000000000000b:19 [read 245760~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:05.039 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.150312+0000", 2024-09-21T14:59:05.039 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.831327139000001, 2024-09-21T14:59:05.039 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015236939, 2024-09-21T14:59:05.039 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:05.039 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:05.039 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:05.039 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:05.039 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:05.039 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 23117 2024-09-21T14:59:05.040 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.040 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:05.040 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.040 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:05.040 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150312+0000", 2024-09-21T14:59:05.040 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:05.040 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.040 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.040 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:05.040 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150312+0000", 2024-09-21T14:59:05.040 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:05.040 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.040 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.040 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:05.040 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150312+0000", 2024-09-21T14:59:05.040 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:05.041 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.041 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.041 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:05.041 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150314+0000", 2024-09-21T14:59:05.041 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.154e-06 2024-09-21T14:59:05.041 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.041 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.041 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:05.041 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150320+0000", 2024-09-21T14:59:05.041 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6640000000000003e-06 2024-09-21T14:59:05.041 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.041 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.041 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:05.041 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150324+0000", 2024-09-21T14:59:05.041 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5669999999999996e-06 2024-09-21T14:59:05.042 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.042 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.042 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:05.042 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.165317+0000", 2024-09-21T14:59:05.042 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014993328 2024-09-21T14:59:05.042 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.042 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.042 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:05.042 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.165370+0000", 2024-09-21T14:59:05.042 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2036999999999997e-05 2024-09-21T14:59:05.042 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.042 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.042 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:05.042 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.165549+0000", 2024-09-21T14:59:05.043 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017947500000000001 2024-09-21T14:59:05.043 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:05.043 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:05.043 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:05.043 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.043 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.043 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:23118 3.1e 3:7ca947c4:::rbd_data.11366acc894.000000000000000b:19 [read 311296~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:05.043 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.150328+0000", 2024-09-21T14:59:05.043 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.831310994000001, 2024-09-21T14:59:05.043 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015505385999999999, 2024-09-21T14:59:05.043 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:05.043 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:05.043 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:05.043 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:05.043 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:05.044 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 23118 2024-09-21T14:59:05.044 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.044 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:05.044 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.044 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:05.044 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150328+0000", 2024-09-21T14:59:05.044 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:05.044 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.044 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.044 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:05.044 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150328+0000", 2024-09-21T14:59:05.044 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:05.044 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.044 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.045 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:05.045 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150328+0000", 2024-09-21T14:59:05.045 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:05.045 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.045 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.045 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:05.045 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150330+0000", 2024-09-21T14:59:05.045 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3410000000000001e-06 2024-09-21T14:59:05.045 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.045 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.045 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:05.045 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150336+0000", 2024-09-21T14:59:05.045 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.694e-06 2024-09-21T14:59:05.045 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.045 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.046 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:05.046 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150343+0000", 2024-09-21T14:59:05.046 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3050000000000004e-06 2024-09-21T14:59:05.046 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.046 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.046 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:05.046 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.165607+0000", 2024-09-21T14:59:05.046 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015264534 2024-09-21T14:59:05.046 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.046 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.046 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:05.046 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.165661+0000", 2024-09-21T14:59:05.046 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3949000000000001e-05 2024-09-21T14:59:05.046 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.046 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.046 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:05.047 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.165834+0000", 2024-09-21T14:59:05.047 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000172139 2024-09-21T14:59:05.047 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:05.047 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:05.047 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:05.047 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.047 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.047 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:23119 3.1e 3:7ca947c4:::rbd_data.11366acc894.000000000000000b:19 [read 376832~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:05.047 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.150347+0000", 2024-09-21T14:59:05.047 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.831291974999999, 2024-09-21T14:59:05.047 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015790017, 2024-09-21T14:59:05.047 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:05.047 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:05.047 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:05.047 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:05.048 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:05.048 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 23119 2024-09-21T14:59:05.048 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.048 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:05.048 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.048 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:05.048 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150347+0000", 2024-09-21T14:59:05.048 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:05.048 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.048 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.048 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:05.048 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150347+0000", 2024-09-21T14:59:05.048 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:05.048 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.048 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.048 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:05.049 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150347+0000", 2024-09-21T14:59:05.049 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:05.049 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.049 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.049 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:05.049 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150349+0000", 2024-09-21T14:59:05.049 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1730000000000002e-06 2024-09-21T14:59:05.049 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.049 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.049 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:05.049 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150355+0000", 2024-09-21T14:59:05.049 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9320000000000001e-06 2024-09-21T14:59:05.049 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.049 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.049 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:05.049 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150360+0000", 2024-09-21T14:59:05.050 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9450000000000001e-06 2024-09-21T14:59:05.050 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.050 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.050 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:05.050 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.165867+0000", 2024-09-21T14:59:05.050 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015507191 2024-09-21T14:59:05.050 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.050 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.050 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:05.050 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.165956+0000", 2024-09-21T14:59:05.050 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9235999999999997e-05 2024-09-21T14:59:05.050 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.050 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.050 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:05.050 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.166137+0000", 2024-09-21T14:59:05.051 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018088699999999999 2024-09-21T14:59:05.051 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:05.051 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:05.051 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:05.051 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.051 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.051 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:23120 3.1e 3:7ca947c4:::rbd_data.11366acc894.000000000000000b:19 [read 442368~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:05.051 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.150364+0000", 2024-09-21T14:59:05.051 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.8312753, 2024-09-21T14:59:05.051 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.016001277000000001, 2024-09-21T14:59:05.051 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:05.051 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:05.051 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:05.051 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:05.052 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:05.052 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 23120 2024-09-21T14:59:05.052 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.052 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:05.052 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.052 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:05.052 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150364+0000", 2024-09-21T14:59:05.052 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:05.052 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.052 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.052 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:05.052 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150364+0000", 2024-09-21T14:59:05.052 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:05.052 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.052 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.052 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:05.053 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150363+0000", 2024-09-21T14:59:05.053 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:05.053 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.053 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.053 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:05.053 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150366+0000", 2024-09-21T14:59:05.053 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1969999999999999e-06 2024-09-21T14:59:05.053 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.053 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.053 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:05.053 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150371+0000", 2024-09-21T14:59:05.053 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7459999999999997e-06 2024-09-21T14:59:05.053 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.053 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.053 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:05.053 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150376+0000", 2024-09-21T14:59:05.054 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6519999999999997e-06 2024-09-21T14:59:05.054 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.054 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.054 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:05.054 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.166166+0000", 2024-09-21T14:59:05.054 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015790286000000001 2024-09-21T14:59:05.054 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.054 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.054 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:05.054 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.166226+0000", 2024-09-21T14:59:05.054 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9682000000000002e-05 2024-09-21T14:59:05.054 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.054 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.054 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:05.054 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.166365+0000", 2024-09-21T14:59:05.055 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013911800000000001 2024-09-21T14:59:05.055 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:05.055 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:05.055 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:05.055 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.055 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.055 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:23121 3.1e 3:7ca947c4:::rbd_data.11366acc894.000000000000000b:19 [read 507904~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:05.055 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.150380+0000", 2024-09-21T14:59:05.055 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.831259168999999, 2024-09-21T14:59:05.055 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.016302104000000001, 2024-09-21T14:59:05.055 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:05.055 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:05.055 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:05.055 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:05.055 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:05.056 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 23121 2024-09-21T14:59:05.056 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.056 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:05.056 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.056 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:05.056 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150380+0000", 2024-09-21T14:59:05.056 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:05.056 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.056 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.056 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:05.056 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150380+0000", 2024-09-21T14:59:05.056 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:05.056 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.056 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.056 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:05.057 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150380+0000", 2024-09-21T14:59:05.057 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:05.057 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.057 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.057 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:05.057 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150382+0000", 2024-09-21T14:59:05.057 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1720000000000001e-06 2024-09-21T14:59:05.057 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.057 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.057 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:05.057 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150387+0000", 2024-09-21T14:59:05.057 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2569999999999998e-06 2024-09-21T14:59:05.057 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.057 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.058 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:05.058 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150392+0000", 2024-09-21T14:59:05.058 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1100000000000002e-06 2024-09-21T14:59:05.058 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.058 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.058 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:05.058 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.166392+0000", 2024-09-21T14:59:05.058 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015999394 2024-09-21T14:59:05.058 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.058 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.058 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:05.058 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.166461+0000", 2024-09-21T14:59:05.058 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9035999999999994e-05 2024-09-21T14:59:05.058 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.058 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.058 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:05.059 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.166682+0000", 2024-09-21T14:59:05.059 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00022143900000000001 2024-09-21T14:59:05.059 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:05.059 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:05.059 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:05.059 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.059 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.059 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:23122 3.1e 3:7ca947c4:::rbd_data.11366acc894.000000000000000b:19 [read 573440~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:05.059 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.150396+0000", 2024-09-21T14:59:05.059 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.831243246, 2024-09-21T14:59:05.059 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.016563735999999999, 2024-09-21T14:59:05.059 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:05.059 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:05.059 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:05.060 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:05.060 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:05.060 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 23122 2024-09-21T14:59:05.060 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.060 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:05.060 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.060 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:05.060 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150396+0000", 2024-09-21T14:59:05.060 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:05.060 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.060 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.060 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:05.060 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150396+0000", 2024-09-21T14:59:05.060 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:05.061 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.061 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.061 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:05.061 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150396+0000", 2024-09-21T14:59:05.061 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:05.061 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.061 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.061 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:05.061 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150398+0000", 2024-09-21T14:59:05.061 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1739999999999999e-06 2024-09-21T14:59:05.061 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.061 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.061 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:05.061 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150403+0000", 2024-09-21T14:59:05.062 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1490000000000002e-06 2024-09-21T14:59:05.062 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.062 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.062 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:05.062 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150408+0000", 2024-09-21T14:59:05.062 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6460000000000001e-06 2024-09-21T14:59:05.062 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.062 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.062 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:05.062 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.166716+0000", 2024-09-21T14:59:05.062 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.016308804999999999 2024-09-21T14:59:05.062 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.062 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.062 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:05.062 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.166792+0000", 2024-09-21T14:59:05.063 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5479e-05 2024-09-21T14:59:05.063 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.063 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.063 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:05.063 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.166960+0000", 2024-09-21T14:59:05.063 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000167799 2024-09-21T14:59:05.063 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:05.063 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:05.063 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:05.063 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.063 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.063 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:27274 3.18 3:1fcceaa3:::rbd_data.11367fb81126.000000000000000a:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-21T14:59:05.063 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:55.596241+0000", 2024-09-21T14:59:05.063 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.3853977450000006, 2024-09-21T14:59:05.064 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.024271564999999998, 2024-09-21T14:59:05.064 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:05.064 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:05.064 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:05.064 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:05.065 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:05.065 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 27274 2024-09-21T14:59:05.065 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.065 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:05.065 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.066 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:05.066 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:55.596241+0000", 2024-09-21T14:59:05.066 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:05.066 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.066 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.066 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:05.066 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:55.596241+0000", 2024-09-21T14:59:05.066 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:05.066 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.066 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.066 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:05.066 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:55.596239+0000", 2024-09-21T14:59:05.066 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-21T14:59:05.066 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.066 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.067 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:05.067 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:55.596252+0000", 2024-09-21T14:59:05.067 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3169e-05 2024-09-21T14:59:05.067 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.067 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.067 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:05.067 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:55.596286+0000", 2024-09-21T14:59:05.067 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.311e-05 2024-09-21T14:59:05.067 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.067 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.067 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:05.067 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:55.596300+0000", 2024-09-21T14:59:05.067 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4369000000000001e-05 2024-09-21T14:59:05.067 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.067 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.067 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:05.068 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:55.596397+0000", 2024-09-21T14:59:05.068 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6924999999999998e-05 2024-09-21T14:59:05.068 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.068 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.068 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:05.068 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:55.596626+0000", 2024-09-21T14:59:05.068 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00022904499999999999 2024-09-21T14:59:05.068 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.068 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.068 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T14:59:05.068 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:55.602376+0000", 2024-09-21T14:59:05.068 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.005750196 2024-09-21T14:59:05.068 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.068 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.068 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:59:05.068 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:55.620263+0000", 2024-09-21T14:59:05.069 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.017886868 2024-09-21T14:59:05.069 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.069 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.069 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:05.069 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:55.620353+0000", 2024-09-21T14:59:05.069 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0105999999999994e-05 2024-09-21T14:59:05.069 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.069 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.069 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:59:05.069 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:55.620449+0000", 2024-09-21T14:59:05.069 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5514999999999996e-05 2024-09-21T14:59:05.069 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:05.069 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:05.069 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:05.069 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:55.620513+0000", 2024-09-21T14:59:05.069 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4451999999999999e-05 2024-09-21T14:59:05.070 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:05.070 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:05.070 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:05.070 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:05.070 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:05.070 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:59:05.070 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T14:59:05.072 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T14:59:05.072 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:59:05.068+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:59:05.120 INFO:teuthology.orchestra.run.smithi032.stderr:noscrub is set 2024-09-21T14:59:05.172 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:59:05.168+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:59:05.200 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:59:05.200 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:59:05.200 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T14:59:05.200 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:59:05.210 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T14:59:05.227 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T14:59:05.245 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T14:59:05.266 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T14:59:05.273 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:59:05.273+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:59:05.289 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T14:59:05.313 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-21T14:59:05.324 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:59:05.324 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:59:05.324 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T14:59:05.324 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T14:59:05.324 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:59:05.335 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T14:59:05.337 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T14:59:05.360 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T14:59:05.374 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:59:05.372+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:59:05.380 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T14:59:05.383 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:59:05.383 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:59:05.383 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:59:05.383 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:59:05.384 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:59:05.384 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:59:05.384 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:59:05.384 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:59:05.437 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:05.437 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:05.448 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T14:59:05.494 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:59:05.492+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:59:05.555 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:05.556 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:05.566 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T14:59:05.576 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:59:05.572+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:59:05.672 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:05.672 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:05.676 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:59:05.673+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:59:05.682 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T14:59:05.777 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:59:05.777+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:59:05.792 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:05.792 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:05.802 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T14:59:05.878 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:59:05.877+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:59:05.923 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:05.924 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:05.934 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T14:59:05.980 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:59:05.980+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:59:06.043 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:06.043 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:06.053 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T14:59:06.080 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:59:06.081+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:59:06.186 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:59:06.181+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:59:06.187 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:06.187 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:06.213 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T14:59:06.283 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:59:06.280+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:59:06.384 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:59:06.380+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:59:06.389 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:06.389 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:06.402 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T14:59:06.485 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:59:06.480+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:59:06.533 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:06.533 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:06.543 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T14:59:06.586 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:59:06.585+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:59:06.643 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:06.643 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:06.652 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T14:59:06.686 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:59:06.684+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:59:06.758 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:06.758 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:06.768 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T14:59:06.788 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:59:06.785+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:59:06.877 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:06.878 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:06.887 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T14:59:06.888 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:59:06.884+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:59:06.990 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:59:06.988+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:59:06.992 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:06.992 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:07.003 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T14:59:07.090 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:59:07.088+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:59:07.111 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:07.111 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:07.122 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T14:59:07.134 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T14:59:07.191 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:59:07.188+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:59:07.239 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:07.239 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:07.248 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T14:59:07.292 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:59:07.288+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:59:07.356 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:07.356 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:07.367 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T14:59:07.393 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:59:07.393+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:59:07.395 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T14:59:07.477 INFO:teuthology.orchestra.run.smithi032.stdout:checking clone #10, image image_client.0-clone10 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent11 2024-09-21T14:59:07.493 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:07.493 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:07.493 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:59:07.493+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:59:07.503 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T14:59:07.540 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T14:59:07.562 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T14:59:07.582 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T14:59:07.593 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:59:07.592+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:59:07.612 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:07.612 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:07.622 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T14:59:07.624 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T14:59:07.649 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-21T14:59:07.671 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T14:59:07.694 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T14:59:07.694 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:59:07.693+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:59:07.715 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T14:59:07.717 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:59:07.717 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:59:07.717 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:59:07.718 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:59:07.718 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:59:07.718 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:59:07.718 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:59:07.718 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:59:07.729 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:59:07.729 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T14:59:07.729 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T14:59:07.729 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T14:59:07.729 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.729 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:1452 3.19 3:9d40fe72:::rbd_data.1136515ecb6b.0000000000000013:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T14:59:07.729 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:30.292755+0000", 2024-09-21T14:59:07.729 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 37.433992199000002, 2024-09-21T14:59:07.729 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043852191999999998, 2024-09-21T14:59:07.730 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:07.730 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:07.730 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:07.730 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:07.730 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:07.730 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1452 2024-09-21T14:59:07.730 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.730 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:07.730 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.730 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:07.730 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292755+0000", 2024-09-21T14:59:07.730 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:07.730 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.730 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.731 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:07.731 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292755+0000", 2024-09-21T14:59:07.731 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:07.731 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.731 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.731 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:07.731 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292754+0000", 2024-09-21T14:59:07.731 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:07.731 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.731 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.731 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:07.731 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292760+0000", 2024-09-21T14:59:07.731 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5029999999999997e-06 2024-09-21T14:59:07.731 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.731 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.732 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:07.732 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292771+0000", 2024-09-21T14:59:07.732 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1551e-05 2024-09-21T14:59:07.732 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.732 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.732 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:07.732 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292779+0000", 2024-09-21T14:59:07.732 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1559999999999998e-06 2024-09-21T14:59:07.732 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.732 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.732 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:07.732 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292896+0000", 2024-09-21T14:59:07.732 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000117304 2024-09-21T14:59:07.732 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.733 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.733 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:07.733 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.293124+0000", 2024-09-21T14:59:07.733 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00022819999999999999 2024-09-21T14:59:07.733 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.733 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.733 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 1", 2024-09-21T14:59:07.733 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.299967+0000", 2024-09-21T14:59:07.733 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0068427009999999996 2024-09-21T14:59:07.733 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.733 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.733 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:07.733 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336463+0000", 2024-09-21T14:59:07.733 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.036495842000000001 2024-09-21T14:59:07.734 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.734 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.734 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:59:07.734 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336480+0000", 2024-09-21T14:59:07.734 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7419999999999999e-05 2024-09-21T14:59:07.734 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.734 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.734 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:59:07.734 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336550+0000", 2024-09-21T14:59:07.734 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9951999999999997e-05 2024-09-21T14:59:07.734 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.734 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.734 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:07.735 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336607+0000", 2024-09-21T14:59:07.735 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7423000000000002e-05 2024-09-21T14:59:07.735 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:07.735 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:07.735 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:07.735 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.735 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.735 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop_reply(client.4406.0:1452 3.19 e36/24 ondisk, result = 0)", 2024-09-21T14:59:07.735 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:30.322854+0000", 2024-09-21T14:59:07.735 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 37.403893523000001, 2024-09-21T14:59:07.735 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013613079, 2024-09-21T14:59:07.735 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:07.735 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:07.735 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:07.736 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.736 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:07.736 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.322854+0000", 2024-09-21T14:59:07.736 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:07.736 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.736 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.736 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:07.736 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.322854+0000", 2024-09-21T14:59:07.736 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:07.736 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.736 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.736 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:07.736 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.322852+0000", 2024-09-21T14:59:07.736 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:59:07.737 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.737 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.737 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:07.737 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.322860+0000", 2024-09-21T14:59:07.737 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.605e-06 2024-09-21T14:59:07.737 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.737 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.737 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:07.737 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.322878+0000", 2024-09-21T14:59:07.737 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8204000000000001e-05 2024-09-21T14:59:07.737 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.737 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.737 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:07.737 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.322888+0000", 2024-09-21T14:59:07.738 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4099999999999997e-06 2024-09-21T14:59:07.738 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.738 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.738 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:07.738 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336440+0000", 2024-09-21T14:59:07.738 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013551848 2024-09-21T14:59:07.738 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.738 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.738 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:07.738 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336453+0000", 2024-09-21T14:59:07.738 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3465000000000001e-05 2024-09-21T14:59:07.738 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.738 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.738 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:07.739 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336467+0000", 2024-09-21T14:59:07.739 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3913999999999999e-05 2024-09-21T14:59:07.739 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:07.739 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:07.739 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:07.739 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.739 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.739 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:1567 3.8 3:139d3d55:::rbd_data.1136515ecb6b.0000000000000000:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T14:59:07.739 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:30.681087+0000", 2024-09-21T14:59:07.739 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 37.045659983999997, 2024-09-21T14:59:07.739 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.039009224000000002, 2024-09-21T14:59:07.739 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:07.739 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:07.739 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:07.740 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:07.740 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:07.740 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1567 2024-09-21T14:59:07.740 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.740 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:07.740 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.740 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:07.740 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681087+0000", 2024-09-21T14:59:07.740 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:07.740 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.740 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.740 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:07.740 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681087+0000", 2024-09-21T14:59:07.740 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:07.741 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.741 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.741 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:07.741 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681086+0000", 2024-09-21T14:59:07.741 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:59:07.741 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.741 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.741 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:07.741 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681097+0000", 2024-09-21T14:59:07.741 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.134e-05 2024-09-21T14:59:07.741 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.741 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.741 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:07.741 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681122+0000", 2024-09-21T14:59:07.741 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5313e-05 2024-09-21T14:59:07.742 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.742 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.742 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:07.742 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681138+0000", 2024-09-21T14:59:07.742 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5916000000000001e-05 2024-09-21T14:59:07.742 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.742 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.742 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:07.742 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681270+0000", 2024-09-21T14:59:07.742 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000131474 2024-09-21T14:59:07.742 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.742 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.742 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:07.742 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681416+0000", 2024-09-21T14:59:07.742 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014637 2024-09-21T14:59:07.742 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.743 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.743 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 7", 2024-09-21T14:59:07.743 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.693163+0000", 2024-09-21T14:59:07.743 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011747241 2024-09-21T14:59:07.743 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.743 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.743 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:59:07.743 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.719750+0000", 2024-09-21T14:59:07.743 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.026586852000000001 2024-09-21T14:59:07.743 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.743 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.743 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:07.743 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.719859+0000", 2024-09-21T14:59:07.743 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010925999999999999 2024-09-21T14:59:07.744 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.744 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.744 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:59:07.744 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.719951+0000", 2024-09-21T14:59:07.744 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1294999999999999e-05 2024-09-21T14:59:07.744 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.744 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.744 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:07.744 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.720097+0000", 2024-09-21T14:59:07.744 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014583 2024-09-21T14:59:07.744 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:07.744 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:07.744 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:07.744 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.744 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.745 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9347 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 16384~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:07.745 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158291+0000", 2024-09-21T14:59:07.745 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 28.568456482999999, 2024-09-21T14:59:07.745 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013600958, 2024-09-21T14:59:07.745 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:07.745 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:07.745 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:07.745 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:07.745 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:07.745 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9347 2024-09-21T14:59:07.745 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.745 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:07.745 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.745 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:07.745 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158291+0000", 2024-09-21T14:59:07.745 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:07.746 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.746 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.746 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:07.746 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158291+0000", 2024-09-21T14:59:07.746 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:07.746 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.746 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.746 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:07.746 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158290+0000", 2024-09-21T14:59:07.746 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:07.746 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.746 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.746 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:07.746 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158295+0000", 2024-09-21T14:59:07.746 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2399999999999998e-06 2024-09-21T14:59:07.747 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.747 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.747 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:07.747 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158312+0000", 2024-09-21T14:59:07.747 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6929999999999999e-05 2024-09-21T14:59:07.747 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.747 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.747 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:07.747 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158353+0000", 2024-09-21T14:59:07.747 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0805e-05 2024-09-21T14:59:07.747 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.747 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.747 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:07.747 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.171634+0000", 2024-09-21T14:59:07.747 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.01328085 2024-09-21T14:59:07.747 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.748 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.748 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:07.748 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.171722+0000", 2024-09-21T14:59:07.748 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8115000000000006e-05 2024-09-21T14:59:07.748 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.748 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.748 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:07.748 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.171892+0000", 2024-09-21T14:59:07.748 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000169722 2024-09-21T14:59:07.748 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:07.748 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:07.748 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:07.748 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.748 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.749 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9348 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 122880~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:07.749 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158369+0000", 2024-09-21T14:59:07.749 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 28.568378315, 2024-09-21T14:59:07.749 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.01388847, 2024-09-21T14:59:07.749 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:07.749 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:07.749 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:07.749 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:07.749 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:07.749 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9348 2024-09-21T14:59:07.749 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.749 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:07.749 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.749 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:07.749 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158369+0000", 2024-09-21T14:59:07.749 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:07.750 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.750 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.750 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:07.750 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158369+0000", 2024-09-21T14:59:07.750 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:07.750 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.750 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.750 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:07.750 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158368+0000", 2024-09-21T14:59:07.750 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:07.750 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.750 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.750 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:07.751 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158374+0000", 2024-09-21T14:59:07.751 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0499999999999997e-06 2024-09-21T14:59:07.751 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.751 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.751 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:07.751 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158386+0000", 2024-09-21T14:59:07.751 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1579999999999999e-05 2024-09-21T14:59:07.751 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.751 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.751 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:07.751 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158396+0000", 2024-09-21T14:59:07.751 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.8260000000000005e-06 2024-09-21T14:59:07.751 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.751 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.751 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:07.752 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.171967+0000", 2024-09-21T14:59:07.752 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013571318000000001 2024-09-21T14:59:07.752 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.752 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.752 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:07.752 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172034+0000", 2024-09-21T14:59:07.752 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.656e-05 2024-09-21T14:59:07.752 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.752 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.752 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:07.752 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172258+0000", 2024-09-21T14:59:07.752 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00022395999999999999 2024-09-21T14:59:07.752 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:07.752 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:07.752 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:07.753 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.753 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.753 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9349 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 155648~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:07.753 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158403+0000", 2024-09-21T14:59:07.753 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 28.568344569000001, 2024-09-21T14:59:07.753 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014229851, 2024-09-21T14:59:07.753 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:07.753 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:07.753 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:07.753 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:07.753 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:07.753 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9349 2024-09-21T14:59:07.753 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.753 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:07.754 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.754 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:07.754 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158403+0000", 2024-09-21T14:59:07.754 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:07.754 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.754 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.754 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:07.754 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158403+0000", 2024-09-21T14:59:07.754 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:07.754 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.754 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.754 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:07.754 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158402+0000", 2024-09-21T14:59:07.755 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:07.755 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.755 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.755 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:07.755 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158406+0000", 2024-09-21T14:59:07.755 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1339999999999997e-06 2024-09-21T14:59:07.755 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.755 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.755 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:07.755 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158416+0000", 2024-09-21T14:59:07.755 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3149999999999998e-06 2024-09-21T14:59:07.755 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.755 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.755 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:07.755 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158439+0000", 2024-09-21T14:59:07.756 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3255999999999999e-05 2024-09-21T14:59:07.756 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.756 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.756 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:07.756 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172305+0000", 2024-09-21T14:59:07.756 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013866603999999999 2024-09-21T14:59:07.756 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.756 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.756 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:07.756 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172413+0000", 2024-09-21T14:59:07.756 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000107664 2024-09-21T14:59:07.756 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.756 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.756 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:07.756 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172633+0000", 2024-09-21T14:59:07.757 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021950800000000001 2024-09-21T14:59:07.757 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:07.757 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:07.757 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:07.757 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.757 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.757 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9350 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 294912~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:07.757 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158448+0000", 2024-09-21T14:59:07.757 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 28.568299698000001, 2024-09-21T14:59:07.757 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014471525000000001, 2024-09-21T14:59:07.757 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:07.757 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:07.757 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:07.757 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:07.757 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:07.758 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9350 2024-09-21T14:59:07.758 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.758 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:07.758 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.758 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:07.758 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158448+0000", 2024-09-21T14:59:07.758 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:07.758 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.758 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.758 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:07.758 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158448+0000", 2024-09-21T14:59:07.758 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:07.758 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.759 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.759 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:07.759 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158447+0000", 2024-09-21T14:59:07.759 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:07.759 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.759 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.759 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:07.759 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158452+0000", 2024-09-21T14:59:07.759 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8400000000000002e-06 2024-09-21T14:59:07.759 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.759 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.759 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:07.759 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158475+0000", 2024-09-21T14:59:07.759 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3091e-05 2024-09-21T14:59:07.760 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.760 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.760 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:07.760 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158515+0000", 2024-09-21T14:59:07.760 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0265000000000002e-05 2024-09-21T14:59:07.760 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.760 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.760 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:07.760 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172682+0000", 2024-09-21T14:59:07.760 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014166761999999999 2024-09-21T14:59:07.760 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.760 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.760 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:07.760 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172752+0000", 2024-09-21T14:59:07.760 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9698999999999998e-05 2024-09-21T14:59:07.760 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.761 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.761 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:07.761 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172919+0000", 2024-09-21T14:59:07.761 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016759199999999999 2024-09-21T14:59:07.761 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:07.761 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:07.761 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:07.761 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.761 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.761 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9351 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 434176~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:07.761 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158526+0000", 2024-09-21T14:59:07.761 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 28.56822098, 2024-09-21T14:59:07.761 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014662107000000001, 2024-09-21T14:59:07.761 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:07.761 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:07.761 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:07.761 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:07.762 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:07.762 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9351 2024-09-21T14:59:07.762 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.762 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:07.762 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.762 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:07.762 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158526+0000", 2024-09-21T14:59:07.762 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:07.762 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.762 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.762 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:07.762 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158526+0000", 2024-09-21T14:59:07.762 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:07.762 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.763 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.763 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:07.763 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158525+0000", 2024-09-21T14:59:07.763 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:07.763 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.763 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.763 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:07.763 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158531+0000", 2024-09-21T14:59:07.763 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5840000000000002e-06 2024-09-21T14:59:07.763 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.763 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.763 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:07.763 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158564+0000", 2024-09-21T14:59:07.763 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3195000000000003e-05 2024-09-21T14:59:07.764 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.764 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.764 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:07.764 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158577+0000", 2024-09-21T14:59:07.764 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2330000000000001e-05 2024-09-21T14:59:07.764 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.764 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.764 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:07.764 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172956+0000", 2024-09-21T14:59:07.764 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014379438 2024-09-21T14:59:07.764 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.764 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.764 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:07.764 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173020+0000", 2024-09-21T14:59:07.765 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4401999999999998e-05 2024-09-21T14:59:07.765 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.765 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.765 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:07.765 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173188+0000", 2024-09-21T14:59:07.765 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016804 2024-09-21T14:59:07.765 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:07.765 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:07.765 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:07.765 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.765 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.765 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9379 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 24576~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:07.765 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158586+0000", 2024-09-21T14:59:07.765 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 28.568161719999999, 2024-09-21T14:59:07.765 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014865173000000001, 2024-09-21T14:59:07.766 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:07.766 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:07.766 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:07.766 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:07.766 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:07.766 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9379 2024-09-21T14:59:07.766 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.766 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:07.766 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.766 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:07.766 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158586+0000", 2024-09-21T14:59:07.766 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:07.766 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.766 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.766 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:07.767 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158586+0000", 2024-09-21T14:59:07.767 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:07.767 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.767 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.767 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:07.767 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158585+0000", 2024-09-21T14:59:07.767 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:07.767 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.767 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.767 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:07.768 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158590+0000", 2024-09-21T14:59:07.768 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5820000000000001e-06 2024-09-21T14:59:07.768 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.768 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.768 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:07.768 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158676+0000", 2024-09-21T14:59:07.768 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5075999999999997e-05 2024-09-21T14:59:07.768 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.768 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.768 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:07.768 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158695+0000", 2024-09-21T14:59:07.768 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9137999999999999e-05 2024-09-21T14:59:07.768 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.768 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.768 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:07.768 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173219+0000", 2024-09-21T14:59:07.769 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014524034999999999 2024-09-21T14:59:07.769 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.769 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.769 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:07.769 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173276+0000", 2024-09-21T14:59:07.769 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7442000000000003e-05 2024-09-21T14:59:07.769 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.769 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.769 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:07.769 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173451+0000", 2024-09-21T14:59:07.769 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017464800000000001 2024-09-21T14:59:07.769 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:07.769 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:07.769 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:07.769 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.769 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.770 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9380 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 163840~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:07.770 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158706+0000", 2024-09-21T14:59:07.770 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 28.568041210000001, 2024-09-21T14:59:07.770 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015096381000000001, 2024-09-21T14:59:07.770 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:07.770 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:07.770 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:07.770 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:07.770 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:07.770 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9380 2024-09-21T14:59:07.770 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.770 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:07.770 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.770 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:07.770 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158706+0000", 2024-09-21T14:59:07.771 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:07.771 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.771 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.771 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:07.771 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158706+0000", 2024-09-21T14:59:07.771 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:07.771 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.771 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.771 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:07.771 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158705+0000", 2024-09-21T14:59:07.771 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:07.771 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.771 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.771 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:07.771 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158712+0000", 2024-09-21T14:59:07.771 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7240000000000003e-06 2024-09-21T14:59:07.772 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.772 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.772 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:07.772 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158755+0000", 2024-09-21T14:59:07.772 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2953e-05 2024-09-21T14:59:07.772 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.772 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.772 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:07.772 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158765+0000", 2024-09-21T14:59:07.772 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0149000000000001e-05 2024-09-21T14:59:07.772 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.772 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.772 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:07.772 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173492+0000", 2024-09-21T14:59:07.772 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014727105000000001 2024-09-21T14:59:07.772 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.773 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.773 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:07.773 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173564+0000", 2024-09-21T14:59:07.773 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1789e-05 2024-09-21T14:59:07.773 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.773 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.773 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:07.773 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173803+0000", 2024-09-21T14:59:07.773 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00023867100000000001 2024-09-21T14:59:07.773 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:07.773 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:07.773 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:07.773 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.773 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.773 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9381 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 303104~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:07.774 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158774+0000", 2024-09-21T14:59:07.774 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 28.567973129999999, 2024-09-21T14:59:07.774 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015309095999999999, 2024-09-21T14:59:07.774 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:07.774 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:07.774 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:07.774 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:07.774 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:07.774 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9381 2024-09-21T14:59:07.774 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.774 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:07.774 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.775 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:07.775 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158774+0000", 2024-09-21T14:59:07.775 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:07.775 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.775 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.775 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:07.775 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158774+0000", 2024-09-21T14:59:07.775 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:07.775 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.775 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.775 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:07.775 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158773+0000", 2024-09-21T14:59:07.775 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:07.775 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.775 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.776 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:07.776 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158779+0000", 2024-09-21T14:59:07.776 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6999999999999996e-06 2024-09-21T14:59:07.776 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.776 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.776 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:07.776 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158790+0000", 2024-09-21T14:59:07.776 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0594000000000001e-05 2024-09-21T14:59:07.776 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.776 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.776 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:07.776 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158798+0000", 2024-09-21T14:59:07.776 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5860000000000005e-06 2024-09-21T14:59:07.776 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.776 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.776 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:07.777 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173829+0000", 2024-09-21T14:59:07.777 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015030349 2024-09-21T14:59:07.777 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.777 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.777 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:07.777 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173918+0000", 2024-09-21T14:59:07.777 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9375000000000004e-05 2024-09-21T14:59:07.777 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.777 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.777 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:07.777 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.174083+0000", 2024-09-21T14:59:07.777 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016536200000000001 2024-09-21T14:59:07.777 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:07.777 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:07.777 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:07.777 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.778 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.778 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9382 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 442368~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:07.778 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158805+0000", 2024-09-21T14:59:07.778 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 28.567942224999999, 2024-09-21T14:59:07.778 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015592623, 2024-09-21T14:59:07.778 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:07.778 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:07.778 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:07.778 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:07.778 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:07.778 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9382 2024-09-21T14:59:07.778 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.778 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:07.778 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.779 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:07.779 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158805+0000", 2024-09-21T14:59:07.779 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:07.779 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.779 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.779 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:07.779 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158805+0000", 2024-09-21T14:59:07.779 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:07.779 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.779 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.779 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:07.779 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158805+0000", 2024-09-21T14:59:07.779 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:07.779 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.780 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.780 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:07.780 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158808+0000", 2024-09-21T14:59:07.780 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9179999999999996e-06 2024-09-21T14:59:07.780 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.780 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.780 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:07.780 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158817+0000", 2024-09-21T14:59:07.780 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9299999999999992e-06 2024-09-21T14:59:07.780 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.780 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.780 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:07.780 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158825+0000", 2024-09-21T14:59:07.780 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6739999999999997e-06 2024-09-21T14:59:07.780 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.780 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.780 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:07.781 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.174122+0000", 2024-09-21T14:59:07.781 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.01529694 2024-09-21T14:59:07.781 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.781 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.781 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:07.781 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.174185+0000", 2024-09-21T14:59:07.781 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3015000000000006e-05 2024-09-21T14:59:07.781 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.781 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.781 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:07.781 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.174398+0000", 2024-09-21T14:59:07.781 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021274799999999999 2024-09-21T14:59:07.781 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:07.781 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:07.781 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:07.781 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.782 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.782 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22699 3.19 3:98de45b8:::rbd_data.11366acc894.0000000000000000:19 [read 638976~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:07.782 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.140173+0000", 2024-09-21T14:59:07.782 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 17.586574127999999, 2024-09-21T14:59:07.782 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013511366, 2024-09-21T14:59:07.782 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:07.782 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:07.782 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:07.782 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:07.782 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:07.782 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22699 2024-09-21T14:59:07.782 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.782 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:07.782 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.782 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:07.783 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140173+0000", 2024-09-21T14:59:07.783 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:07.783 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.783 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.783 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:07.783 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140173+0000", 2024-09-21T14:59:07.783 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:07.783 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.783 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.783 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:07.783 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140173+0000", 2024-09-21T14:59:07.783 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:07.783 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.783 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.783 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:07.784 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140175+0000", 2024-09-21T14:59:07.784 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5859999999999999e-06 2024-09-21T14:59:07.784 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.784 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.784 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:07.784 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140181+0000", 2024-09-21T14:59:07.784 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0340000000000002e-06 2024-09-21T14:59:07.784 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.784 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.784 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:07.784 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140186+0000", 2024-09-21T14:59:07.784 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.04e-06 2024-09-21T14:59:07.784 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.784 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.785 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:07.785 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.153426+0000", 2024-09-21T14:59:07.785 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013240001 2024-09-21T14:59:07.785 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.785 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.785 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:07.785 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.153513+0000", 2024-09-21T14:59:07.785 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6038999999999994e-05 2024-09-21T14:59:07.785 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.785 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.785 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:07.785 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.153685+0000", 2024-09-21T14:59:07.785 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000172067 2024-09-21T14:59:07.785 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:07.785 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:07.785 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:07.786 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.786 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.786 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22700 3.19 3:98de45b8:::rbd_data.11366acc894.0000000000000000:19 [read 704512~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:07.786 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.140191+0000", 2024-09-21T14:59:07.786 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 17.586556610999999, 2024-09-21T14:59:07.786 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013770486, 2024-09-21T14:59:07.786 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:07.786 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:07.786 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:07.786 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:07.786 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:07.786 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22700 2024-09-21T14:59:07.786 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.786 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:07.787 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.787 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:07.787 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140191+0000", 2024-09-21T14:59:07.787 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:07.787 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.787 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.787 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:07.787 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140191+0000", 2024-09-21T14:59:07.787 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:07.787 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.787 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.787 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:07.787 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140190+0000", 2024-09-21T14:59:07.787 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:07.787 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.788 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.788 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:07.788 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140199+0000", 2024-09-21T14:59:07.788 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3340000000000002e-06 2024-09-21T14:59:07.788 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.788 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.788 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:07.788 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140205+0000", 2024-09-21T14:59:07.788 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3820000000000001e-06 2024-09-21T14:59:07.788 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.788 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.788 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:07.788 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140210+0000", 2024-09-21T14:59:07.788 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0969999999999996e-06 2024-09-21T14:59:07.789 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.789 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.789 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:07.789 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.153713+0000", 2024-09-21T14:59:07.789 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013502713 2024-09-21T14:59:07.789 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.789 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.789 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:07.789 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.153786+0000", 2024-09-21T14:59:07.789 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3022999999999995e-05 2024-09-21T14:59:07.789 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.789 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.789 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:07.789 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.153961+0000", 2024-09-21T14:59:07.790 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000175274 2024-09-21T14:59:07.790 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:07.790 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:07.790 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:07.790 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.790 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.790 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22701 3.19 3:98de45b8:::rbd_data.11366acc894.0000000000000000:19 [read 770048~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:07.790 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.140215+0000", 2024-09-21T14:59:07.790 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 17.586532794, 2024-09-21T14:59:07.790 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014072497, 2024-09-21T14:59:07.790 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:07.790 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:07.790 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:07.790 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:07.791 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:07.791 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22701 2024-09-21T14:59:07.791 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.791 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:07.791 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.791 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:07.791 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140215+0000", 2024-09-21T14:59:07.791 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:07.791 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.791 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.791 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:07.791 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140215+0000", 2024-09-21T14:59:07.791 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:07.791 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.791 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.792 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:07.792 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140214+0000", 2024-09-21T14:59:07.792 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:07.792 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.792 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.792 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:07.792 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140217+0000", 2024-09-21T14:59:07.792 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.74e-06 2024-09-21T14:59:07.792 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.792 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.792 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:07.792 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140223+0000", 2024-09-21T14:59:07.792 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0929999999999999e-06 2024-09-21T14:59:07.792 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.792 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.793 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:07.793 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140228+0000", 2024-09-21T14:59:07.793 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2279999999999998e-06 2024-09-21T14:59:07.793 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.793 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.793 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:07.793 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.153991+0000", 2024-09-21T14:59:07.793 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013762329 2024-09-21T14:59:07.793 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.793 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.793 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:07.793 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.154098+0000", 2024-09-21T14:59:07.793 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000107117 2024-09-21T14:59:07.793 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.793 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.794 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:07.794 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.154287+0000", 2024-09-21T14:59:07.794 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000189431 2024-09-21T14:59:07.794 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:07.794 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:07.794 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:07.794 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.794 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.794 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22702 3.19 3:98de45b8:::rbd_data.11366acc894.0000000000000000:19 [read 835584~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:07.794 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.140232+0000", 2024-09-21T14:59:07.794 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 17.586514942000001, 2024-09-21T14:59:07.794 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015338742000000001, 2024-09-21T14:59:07.794 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:07.795 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:07.795 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:07.795 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:07.795 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:07.795 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22702 2024-09-21T14:59:07.795 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.795 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:07.795 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.795 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:07.795 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140232+0000", 2024-09-21T14:59:07.795 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:07.795 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.795 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.795 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:07.795 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140232+0000", 2024-09-21T14:59:07.796 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:07.796 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.796 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.796 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:07.796 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140232+0000", 2024-09-21T14:59:07.796 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:07.796 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.796 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.796 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:07.796 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140234+0000", 2024-09-21T14:59:07.796 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.402e-06 2024-09-21T14:59:07.796 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.796 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.796 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:07.796 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140241+0000", 2024-09-21T14:59:07.796 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.054e-06 2024-09-21T14:59:07.797 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.797 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.797 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:07.797 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140246+0000", 2024-09-21T14:59:07.797 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0050000000000004e-06 2024-09-21T14:59:07.797 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.797 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.797 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:07.797 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.154315+0000", 2024-09-21T14:59:07.797 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014069440000000001 2024-09-21T14:59:07.797 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.797 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.797 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:07.797 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.154383+0000", 2024-09-21T14:59:07.798 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8151000000000005e-05 2024-09-21T14:59:07.798 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.798 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.798 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:07.798 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155571+0000", 2024-09-21T14:59:07.798 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001188029 2024-09-21T14:59:07.798 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:07.798 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:07.798 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:07.798 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.798 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.798 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22703 3.19 3:98de45b8:::rbd_data.11366acc894.0000000000000000:19 [read 901120~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:07.798 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.140250+0000", 2024-09-21T14:59:07.799 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 17.586497783999999, 2024-09-21T14:59:07.799 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014992169, 2024-09-21T14:59:07.799 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:07.799 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:07.799 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:07.799 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:07.799 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:07.799 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22703 2024-09-21T14:59:07.799 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.799 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:07.799 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.799 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:07.799 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140250+0000", 2024-09-21T14:59:07.799 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:07.799 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.799 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.800 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:07.800 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140250+0000", 2024-09-21T14:59:07.800 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:07.800 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.800 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.800 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:07.800 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140249+0000", 2024-09-21T14:59:07.800 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:07.800 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.800 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.800 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:07.800 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140251+0000", 2024-09-21T14:59:07.800 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.136e-06 2024-09-21T14:59:07.800 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.801 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.801 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:07.801 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140257+0000", 2024-09-21T14:59:07.801 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9610000000000002e-06 2024-09-21T14:59:07.801 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.801 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.801 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:07.801 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140262+0000", 2024-09-21T14:59:07.801 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4549999999999997e-06 2024-09-21T14:59:07.801 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.801 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.801 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:07.801 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155006+0000", 2024-09-21T14:59:07.801 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014744189 2024-09-21T14:59:07.801 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.802 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.802 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:07.802 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155081+0000", 2024-09-21T14:59:07.802 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4548000000000005e-05 2024-09-21T14:59:07.802 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.802 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.802 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:07.802 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155242+0000", 2024-09-21T14:59:07.802 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000161204 2024-09-21T14:59:07.802 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:07.802 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:07.802 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:07.802 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.802 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.803 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22704 3.19 3:98de45b8:::rbd_data.11366acc894.0000000000000000:19 [read 966656~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:07.803 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.140266+0000", 2024-09-21T14:59:07.803 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 17.586481466999999, 2024-09-21T14:59:07.803 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015223443999999999, 2024-09-21T14:59:07.803 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:07.803 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:07.803 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:07.803 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:07.803 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:07.803 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22704 2024-09-21T14:59:07.803 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.803 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:07.803 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.803 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:07.803 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140266+0000", 2024-09-21T14:59:07.804 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:07.804 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.804 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.804 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:07.804 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140266+0000", 2024-09-21T14:59:07.804 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:07.804 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.804 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.804 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:07.804 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140266+0000", 2024-09-21T14:59:07.804 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:07.804 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.804 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.804 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:07.805 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140268+0000", 2024-09-21T14:59:07.805 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.475e-06 2024-09-21T14:59:07.805 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.805 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.805 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:07.805 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140274+0000", 2024-09-21T14:59:07.805 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.319e-06 2024-09-21T14:59:07.805 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.805 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.805 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:07.805 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140279+0000", 2024-09-21T14:59:07.805 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.882e-06 2024-09-21T14:59:07.805 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.805 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.805 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:07.805 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155264+0000", 2024-09-21T14:59:07.806 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014984655 2024-09-21T14:59:07.806 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.806 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.806 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:07.806 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155338+0000", 2024-09-21T14:59:07.806 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4091999999999995e-05 2024-09-21T14:59:07.806 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.806 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.806 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:07.806 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155489+0000", 2024-09-21T14:59:07.806 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000151349 2024-09-21T14:59:07.806 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:07.806 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:07.806 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:07.807 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.807 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.807 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22705 3.19 3:98de45b8:::rbd_data.11366acc894.0000000000000000:19 [read 1032192~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:07.807 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.140284+0000", 2024-09-21T14:59:07.807 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 17.586463609999999, 2024-09-21T14:59:07.807 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015467896, 2024-09-21T14:59:07.807 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:07.807 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:07.807 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:07.807 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:07.807 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:07.807 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22705 2024-09-21T14:59:07.807 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.807 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:07.808 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.808 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:07.808 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140284+0000", 2024-09-21T14:59:07.808 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:07.808 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.808 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.808 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:07.808 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140284+0000", 2024-09-21T14:59:07.808 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:07.808 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.808 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.808 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:07.808 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140283+0000", 2024-09-21T14:59:07.808 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:07.808 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.808 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.809 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:07.809 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140286+0000", 2024-09-21T14:59:07.809 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7130000000000001e-06 2024-09-21T14:59:07.809 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.809 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.809 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:07.809 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140292+0000", 2024-09-21T14:59:07.809 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3430000000000001e-06 2024-09-21T14:59:07.809 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.809 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.809 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:07.809 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140298+0000", 2024-09-21T14:59:07.809 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9730000000000002e-06 2024-09-21T14:59:07.809 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.810 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.810 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:07.810 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155510+0000", 2024-09-21T14:59:07.810 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015211738000000001 2024-09-21T14:59:07.810 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.810 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.810 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:07.810 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155598+0000", 2024-09-21T14:59:07.810 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7822e-05 2024-09-21T14:59:07.810 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.810 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.810 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:07.810 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155752+0000", 2024-09-21T14:59:07.810 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0001537 2024-09-21T14:59:07.811 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:07.811 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:07.811 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:07.811 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.811 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.811 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:37892 3.1b 3:d963c97f:::rbd_data.11365d19aa81.00000000000000be:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-21T14:59:07.811 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:04.029946+0000", 2024-09-21T14:59:07.811 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.6968018430000003, 2024-09-21T14:59:07.811 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.017686021999999999, 2024-09-21T14:59:07.811 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:07.811 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:07.811 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:07.811 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:07.811 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:07.811 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 37892 2024-09-21T14:59:07.812 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.812 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:07.812 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.812 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:07.812 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.029946+0000", 2024-09-21T14:59:07.812 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:07.812 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.812 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.812 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:07.812 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.029946+0000", 2024-09-21T14:59:07.812 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:07.812 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.812 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.812 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:07.813 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.029944+0000", 2024-09-21T14:59:07.813 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T14:59:07.813 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.813 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.813 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:07.813 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.029953+0000", 2024-09-21T14:59:07.813 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7589999999999992e-06 2024-09-21T14:59:07.813 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.813 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.813 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:07.813 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.029971+0000", 2024-09-21T14:59:07.813 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7795000000000002e-05 2024-09-21T14:59:07.813 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.813 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.814 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:07.814 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.029984+0000", 2024-09-21T14:59:07.814 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.27e-05 2024-09-21T14:59:07.814 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.814 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.814 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:07.814 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.030072+0000", 2024-09-21T14:59:07.814 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8082000000000001e-05 2024-09-21T14:59:07.814 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.814 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.814 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:07.814 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.030276+0000", 2024-09-21T14:59:07.814 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00020375699999999999 2024-09-21T14:59:07.814 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.815 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.815 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-21T14:59:07.815 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.031634+0000", 2024-09-21T14:59:07.815 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0013586049999999999 2024-09-21T14:59:07.815 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.815 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.815 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:59:07.815 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.037554+0000", 2024-09-21T14:59:07.815 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0059197449999999997 2024-09-21T14:59:07.815 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.815 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.815 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:07.815 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.047490+0000", 2024-09-21T14:59:07.815 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0099361680000000004 2024-09-21T14:59:07.816 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.816 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.816 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:59:07.816 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.047569+0000", 2024-09-21T14:59:07.816 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8298000000000001e-05 2024-09-21T14:59:07.816 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:07.816 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:07.816 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:07.816 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.047632+0000", 2024-09-21T14:59:07.816 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3006e-05 2024-09-21T14:59:07.816 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:07.816 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:07.816 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:07.816 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:07.817 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:07.817 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:59:07.818 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T14:59:07.819 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:59:07.816+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:59:07.899 INFO:teuthology.orchestra.run.smithi032.stdout:1306 read 0x44010ed thru 0x440c869 (0xb77d bytes) 2024-09-21T14:59:07.900 INFO:teuthology.orchestra.run.smithi032.stdout:1307 read 0x7357284 thru 0x7361a89 (0xa806 bytes) 2024-09-21T14:59:07.901 INFO:teuthology.orchestra.run.smithi032.stdout:truncating to largest ever: 0xed250be 2024-09-21T14:59:07.904 INFO:teuthology.orchestra.run.smithi032.stdout:1309 trunc from 0xc898ec4 to 0xed250be 2024-09-21T14:59:07.918 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:59:07.917+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:59:07.923 INFO:teuthology.orchestra.run.smithi032.stdout:1310 write 0x3a10b9d thru 0x3a10e95 (0x2f9 bytes) 2024-09-21T14:59:07.935 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:59:07.936 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:59:07.936 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T14:59:07.936 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:59:07.939 INFO:teuthology.orchestra.run.smithi032.stdout:1311 trunc from 0xed250be to 0xa39fde3 2024-09-21T14:59:07.946 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T14:59:07.963 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:07.963 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:07.963 INFO:teuthology.orchestra.run.smithi032.stdout:1314 write 0x689d7b5 thru 0x68a8a8c (0xb2d8 bytes) 2024-09-21T14:59:07.976 INFO:teuthology.orchestra.run.smithi032.stdout:1315 read 0x6ca270d thru 0x6cac489 (0x9d7d bytes) 2024-09-21T14:59:07.976 INFO:teuthology.orchestra.run.smithi032.stdout:1316 read 0x7bb66fb thru 0x7bbd5c1 (0x6ec7 bytes) 2024-09-21T14:59:07.979 INFO:teuthology.orchestra.run.smithi032.stdout:1317 trunc from 0xa39fde3 to 0xba4aae0 2024-09-21T14:59:07.990 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:07.990 INFO:teuthology.orchestra.run.smithi032.stdout:1322 punch from 0xa3d3a39 to 0xa3db3c0, (0x7987 bytes) 2024-09-21T14:59:07.995 INFO:teuthology.orchestra.run.smithi032.stdout:1323 trunc from 0xba4aae0 to 0x93e0992 2024-09-21T14:59:08.016 INFO:teuthology.orchestra.run.smithi032.stdout:1325 read 0x8a29b9e thru 0x8a31dc0 (0x8223 bytes) 2024-09-21T14:59:08.016 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:08.016 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:08.019 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:59:08.016+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:59:08.026 INFO:teuthology.orchestra.run.smithi032.stdout:1330 trunc from 0x93e0992 to 0xd582d68 2024-09-21T14:59:08.036 INFO:teuthology.orchestra.run.smithi032.stdout:1331 writesame 0x537234f thru 0x5380e76 data_size 0x64(0xeb28 bytes) 2024-09-21T14:59:08.051 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:08.055 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:59:08.055 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:59:08.055 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T14:59:08.055 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T14:59:08.055 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:59:08.065 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T14:59:08.072 INFO:teuthology.orchestra.run.smithi032.stdout:1334 trunc from 0xd582d68 to 0x1005603 2024-09-21T14:59:08.106 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:08.106 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:08.121 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:59:08.116+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:59:08.123 INFO:teuthology.orchestra.run.smithi032.stdout:1337 write 0x3b782fc thru 0x3b86b2a (0xe82f bytes) 2024-09-21T14:59:08.157 INFO:teuthology.orchestra.run.smithi032.stdout:1338 trunc from 0x3b86b2b to 0xdedd172 2024-09-21T14:59:08.168 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:08.168 INFO:teuthology.orchestra.run.smithi032.stdout:1340 write 0x3558750 thru 0x35635b6 (0xae67 bytes) 2024-09-21T14:59:08.179 INFO:teuthology.orchestra.run.smithi032.stdout:1341 punch from 0xb3a0b23 to 0xb3a2aad, (0x1f8a bytes) 2024-09-21T14:59:08.179 INFO:teuthology.orchestra.run.smithi032.stdout:1342 punch from 0x4884820 to 0x4892706, (0xdee6 bytes) 2024-09-21T14:59:08.179 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:08.192 INFO:teuthology.orchestra.run.smithi032.stdout:1346 writesame 0xe69f5fa thru 0xe6a3091 data_size 0x28(0x3a98 bytes) 2024-09-21T14:59:08.192 INFO:teuthology.orchestra.run.smithi032.stdout:1348 write 0x7817c94 thru 0x78214c6 (0x9833 bytes) 2024-09-21T14:59:08.202 INFO:teuthology.orchestra.run.smithi032.stdout:1349 read 0x992ab24 thru 0x9934867 (0x9d44 bytes) 2024-09-21T14:59:08.203 INFO:teuthology.orchestra.run.smithi032.stdout:1351 write 0x19394f1 thru 0x1942177 (0x8c87 bytes) 2024-09-21T14:59:08.211 INFO:teuthology.orchestra.run.smithi032.stdout:1352 read 0xd5825d8 thru 0xd58ad5b (0x8784 bytes) 2024-09-21T14:59:08.211 INFO:teuthology.orchestra.run.smithi032.stdout:1353 write 0x15ff899 thru 0x160483b (0x4fa3 bytes) 2024-09-21T14:59:08.218 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:08.218 INFO:teuthology.orchestra.run.smithi032.stdout:1359 read 0x9c7bc10 thru 0x9c85692 (0x9a83 bytes) 2024-09-21T14:59:08.218 INFO:teuthology.orchestra.run.smithi032.stdout:1360 write 0x55222d thru 0x55390e (0x16e2 bytes) 2024-09-21T14:59:08.221 INFO:teuthology.orchestra.run.smithi032.stdout:1361 punch from 0x39ae276 to 0x39bace0, (0xca6a bytes) 2024-09-21T14:59:08.221 INFO:teuthology.orchestra.run.smithi032.stdout:1362 write 0xbacf86b thru 0xbad868b (0x8e21 bytes) 2024-09-21T14:59:08.221 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:59:08.220+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:59:08.231 INFO:teuthology.orchestra.run.smithi032.stdout:1363 write 0xc618da thru 0xc6c639 (0xad60 bytes) 2024-09-21T14:59:08.234 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:59:08.234 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T14:59:08.234 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T14:59:08.234 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T14:59:08.234 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.235 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22719 3.1e 3:79caef05:::rbd_data.11366acc894.0000000000000001:19 [read 901120~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:08.235 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.140027+0000", 2024-09-21T14:59:08.235 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 18.092058571999999, 2024-09-21T14:59:08.235 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015081021, 2024-09-21T14:59:08.235 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:08.235 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:08.235 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:08.235 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:08.235 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:08.235 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22719 2024-09-21T14:59:08.235 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.235 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:08.235 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.236 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:08.236 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140027+0000", 2024-09-21T14:59:08.236 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:08.236 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.236 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.236 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:08.236 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140027+0000", 2024-09-21T14:59:08.236 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:08.236 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.236 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.236 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:08.236 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140026+0000", 2024-09-21T14:59:08.236 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:08.236 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.237 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.237 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:08.237 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140042+0000", 2024-09-21T14:59:08.237 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5381999999999999e-05 2024-09-21T14:59:08.237 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.237 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.237 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:08.237 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140051+0000", 2024-09-21T14:59:08.237 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3500000000000003e-06 2024-09-21T14:59:08.237 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.237 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.237 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:08.237 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140066+0000", 2024-09-21T14:59:08.237 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4582e-05 2024-09-21T14:59:08.238 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.238 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.238 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:08.238 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.154602+0000", 2024-09-21T14:59:08.238 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014535881 2024-09-21T14:59:08.238 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.238 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.238 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:08.238 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.154674+0000", 2024-09-21T14:59:08.238 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2303000000000002e-05 2024-09-21T14:59:08.238 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.238 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.238 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:08.239 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155108+0000", 2024-09-21T14:59:08.239 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00043396500000000002 2024-09-21T14:59:08.239 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:08.239 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:08.239 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:08.239 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.239 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.239 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22720 3.1e 3:79caef05:::rbd_data.11366acc894.0000000000000001:19 [read 966656~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:08.239 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.140073+0000", 2024-09-21T14:59:08.239 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 18.092012007000001, 2024-09-21T14:59:08.239 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015292312000000001, 2024-09-21T14:59:08.239 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:08.239 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:08.239 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:08.240 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:08.240 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:08.240 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22720 2024-09-21T14:59:08.240 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.240 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:08.240 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.240 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:08.240 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140073+0000", 2024-09-21T14:59:08.240 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:08.240 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.240 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.240 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:08.240 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140073+0000", 2024-09-21T14:59:08.240 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:08.240 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.240 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.241 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:08.241 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140073+0000", 2024-09-21T14:59:08.241 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:08.241 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.241 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.241 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:08.241 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140076+0000", 2024-09-21T14:59:08.241 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4999999999999999e-06 2024-09-21T14:59:08.241 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.241 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.241 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:08.241 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140084+0000", 2024-09-21T14:59:08.241 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1310000000000006e-06 2024-09-21T14:59:08.241 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.241 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.241 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:08.242 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140091+0000", 2024-09-21T14:59:08.242 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.674e-06 2024-09-21T14:59:08.242 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.242 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.242 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:08.242 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155135+0000", 2024-09-21T14:59:08.242 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015044224 2024-09-21T14:59:08.242 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.242 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.242 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:08.242 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155205+0000", 2024-09-21T14:59:08.242 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9751000000000004e-05 2024-09-21T14:59:08.242 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.242 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.242 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:08.243 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155366+0000", 2024-09-21T14:59:08.243 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016073899999999999 2024-09-21T14:59:08.243 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:08.243 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:08.243 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:08.243 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.243 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.243 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22721 3.1e 3:79caef05:::rbd_data.11366acc894.0000000000000001:19 [read 1032192~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:08.243 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.140096+0000", 2024-09-21T14:59:08.243 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 18.091989022, 2024-09-21T14:59:08.243 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015563158000000001, 2024-09-21T14:59:08.243 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:08.243 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:08.243 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:08.244 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:08.244 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:08.244 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22721 2024-09-21T14:59:08.244 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.244 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:08.244 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.244 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:08.244 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140096+0000", 2024-09-21T14:59:08.244 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:08.244 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.244 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.244 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:08.244 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140096+0000", 2024-09-21T14:59:08.245 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:08.245 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.245 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.245 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:08.245 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140096+0000", 2024-09-21T14:59:08.245 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:08.245 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.245 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.245 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:08.245 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140099+0000", 2024-09-21T14:59:08.245 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.5080000000000001e-06 2024-09-21T14:59:08.245 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.245 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.245 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:08.245 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140106+0000", 2024-09-21T14:59:08.245 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8630000000000002e-06 2024-09-21T14:59:08.245 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.246 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.246 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:08.246 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140112+0000", 2024-09-21T14:59:08.246 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6509999999999998e-06 2024-09-21T14:59:08.246 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.246 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.246 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:08.246 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155392+0000", 2024-09-21T14:59:08.246 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015280375000000001 2024-09-21T14:59:08.246 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.246 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.246 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:08.246 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155459+0000", 2024-09-21T14:59:08.246 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7040999999999997e-05 2024-09-21T14:59:08.247 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.247 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.247 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:08.247 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155660+0000", 2024-09-21T14:59:08.247 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000200115 2024-09-21T14:59:08.247 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:08.247 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:08.247 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:08.247 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.247 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.247 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22865 3.1e 3:7ca947c4:::rbd_data.11366acc894.000000000000000b:19 [read 393216~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:08.247 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.142757+0000", 2024-09-21T14:59:08.247 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 18.089328350999999, 2024-09-21T14:59:08.247 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014907012000000001, 2024-09-21T14:59:08.247 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:08.248 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:08.248 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:08.248 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:08.248 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:08.248 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22865 2024-09-21T14:59:08.248 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.248 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:08.248 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.248 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:08.248 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.142757+0000", 2024-09-21T14:59:08.248 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:08.248 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.248 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.248 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:08.248 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.142757+0000", 2024-09-21T14:59:08.248 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:08.249 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.249 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.249 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:08.249 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.142757+0000", 2024-09-21T14:59:08.249 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:08.249 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.249 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.249 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:08.249 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.142759+0000", 2024-09-21T14:59:08.249 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3800000000000001e-06 2024-09-21T14:59:08.249 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.249 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.249 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:08.249 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.142765+0000", 2024-09-21T14:59:08.250 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7819999999999999e-06 2024-09-21T14:59:08.250 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.250 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.250 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:08.250 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.142770+0000", 2024-09-21T14:59:08.250 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9230000000000001e-06 2024-09-21T14:59:08.250 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.250 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.250 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:08.250 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.157393+0000", 2024-09-21T14:59:08.250 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014623662000000001 2024-09-21T14:59:08.250 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.250 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.250 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:08.250 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.157469+0000", 2024-09-21T14:59:08.251 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5383000000000007e-05 2024-09-21T14:59:08.251 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.251 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.251 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:08.251 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.157664+0000", 2024-09-21T14:59:08.251 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019522100000000001 2024-09-21T14:59:08.251 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:08.251 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:08.251 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:08.251 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.251 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.251 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22866 3.1e 3:7ca947c4:::rbd_data.11366acc894.000000000000000b:19 [read 458752~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:08.251 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.142774+0000", 2024-09-21T14:59:08.251 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 18.089311241000001, 2024-09-21T14:59:08.252 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.01521982, 2024-09-21T14:59:08.252 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:08.252 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:08.252 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:08.252 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:08.252 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:08.252 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22866 2024-09-21T14:59:08.252 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.252 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:08.252 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.252 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:08.252 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.142774+0000", 2024-09-21T14:59:08.252 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:08.253 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.253 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.253 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:08.253 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.142774+0000", 2024-09-21T14:59:08.253 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:08.253 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.253 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.253 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:08.253 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.142774+0000", 2024-09-21T14:59:08.253 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:08.253 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.253 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.253 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:08.253 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.142776+0000", 2024-09-21T14:59:08.254 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.588e-06 2024-09-21T14:59:08.254 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.254 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.254 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:08.254 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.142783+0000", 2024-09-21T14:59:08.254 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2729999999999999e-06 2024-09-21T14:59:08.254 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.254 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.254 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:08.254 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.142789+0000", 2024-09-21T14:59:08.254 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3160000000000002e-06 2024-09-21T14:59:08.254 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.254 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.255 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:08.255 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.157705+0000", 2024-09-21T14:59:08.255 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014915586999999999 2024-09-21T14:59:08.255 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.255 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.255 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:08.255 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.157792+0000", 2024-09-21T14:59:08.255 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7322000000000002e-05 2024-09-21T14:59:08.255 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.255 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.255 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:08.255 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.157994+0000", 2024-09-21T14:59:08.255 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000202093 2024-09-21T14:59:08.255 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:08.256 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:08.256 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:08.256 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.256 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.256 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22867 3.1e 3:7ca947c4:::rbd_data.11366acc894.000000000000000b:19 [read 524288~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:08.256 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.142794+0000", 2024-09-21T14:59:08.256 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 18.089291501000002, 2024-09-21T14:59:08.256 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015569725, 2024-09-21T14:59:08.256 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:08.256 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:08.256 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:08.256 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:08.256 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:08.257 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22867 2024-09-21T14:59:08.257 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.257 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:08.257 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.257 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:08.257 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.142794+0000", 2024-09-21T14:59:08.257 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:08.257 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.257 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.257 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:08.257 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.142794+0000", 2024-09-21T14:59:08.257 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:08.257 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.257 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.258 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:08.258 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.142793+0000", 2024-09-21T14:59:08.258 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:08.258 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.258 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.258 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:08.258 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.142801+0000", 2024-09-21T14:59:08.258 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8509999999999999e-06 2024-09-21T14:59:08.258 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.258 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.258 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:08.258 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.142807+0000", 2024-09-21T14:59:08.258 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1210000000000003e-06 2024-09-21T14:59:08.259 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.259 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.259 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:08.259 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.142812+0000", 2024-09-21T14:59:08.259 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2809999999999999e-06 2024-09-21T14:59:08.259 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.259 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.259 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:08.259 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.158021+0000", 2024-09-21T14:59:08.259 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015208038 2024-09-21T14:59:08.259 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.259 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.259 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:08.259 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.158098+0000", 2024-09-21T14:59:08.259 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7888999999999999e-05 2024-09-21T14:59:08.260 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.260 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.260 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:08.260 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.158364+0000", 2024-09-21T14:59:08.260 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00026517400000000002 2024-09-21T14:59:08.260 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:08.260 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:08.260 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:08.260 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.260 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.260 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:23017 3.1e 3:7ca947c4:::rbd_data.11366acc894.000000000000000b:19 [read 229376~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:08.260 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.147628+0000", 2024-09-21T14:59:08.260 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 18.084457481000001, 2024-09-21T14:59:08.260 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014791732, 2024-09-21T14:59:08.260 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:08.260 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:08.261 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:08.261 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:08.261 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:08.261 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 23017 2024-09-21T14:59:08.261 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.261 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:08.261 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.261 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:08.261 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147628+0000", 2024-09-21T14:59:08.261 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:08.261 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.261 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.261 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:08.261 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147628+0000", 2024-09-21T14:59:08.261 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:08.261 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.262 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.262 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:08.262 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147628+0000", 2024-09-21T14:59:08.262 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:08.262 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.262 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.262 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:08.262 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147635+0000", 2024-09-21T14:59:08.262 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3459999999999996e-06 2024-09-21T14:59:08.262 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.262 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.262 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:08.262 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147642+0000", 2024-09-21T14:59:08.262 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0249999999999997e-06 2024-09-21T14:59:08.262 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.263 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.263 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:08.263 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147647+0000", 2024-09-21T14:59:08.263 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.549e-06 2024-09-21T14:59:08.263 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.263 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.263 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:08.263 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.161946+0000", 2024-09-21T14:59:08.263 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.01429997 2024-09-21T14:59:08.263 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.263 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.263 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:08.263 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.162034+0000", 2024-09-21T14:59:08.263 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7676000000000006e-05 2024-09-21T14:59:08.263 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.264 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.264 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:08.264 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.162420+0000", 2024-09-21T14:59:08.264 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00038544499999999999 2024-09-21T14:59:08.264 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:08.264 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:08.264 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:08.264 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.264 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.264 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:23018 3.1e 3:7ca947c4:::rbd_data.11366acc894.000000000000000b:19 [read 294912~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:08.264 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.147650+0000", 2024-09-21T14:59:08.264 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 18.084435053, 2024-09-21T14:59:08.264 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015515414999999999, 2024-09-21T14:59:08.264 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:08.264 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:08.265 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:08.265 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:08.265 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:08.265 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 23018 2024-09-21T14:59:08.265 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.265 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:08.265 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.265 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:08.265 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147650+0000", 2024-09-21T14:59:08.265 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:08.265 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.265 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.265 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:08.265 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147650+0000", 2024-09-21T14:59:08.265 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:08.265 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.266 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.266 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:08.266 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147650+0000", 2024-09-21T14:59:08.266 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:08.266 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.266 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.266 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:08.266 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147652+0000", 2024-09-21T14:59:08.266 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1469999999999999e-06 2024-09-21T14:59:08.266 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.266 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.266 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:08.266 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147657+0000", 2024-09-21T14:59:08.266 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3449999999999996e-06 2024-09-21T14:59:08.266 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.267 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.267 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:08.267 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147663+0000", 2024-09-21T14:59:08.267 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1309999999999997e-06 2024-09-21T14:59:08.267 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.267 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.267 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:08.267 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.162466+0000", 2024-09-21T14:59:08.267 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014803474 2024-09-21T14:59:08.267 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.267 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.267 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:08.267 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.162562+0000", 2024-09-21T14:59:08.267 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6272000000000003e-05 2024-09-21T14:59:08.267 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.268 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.268 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:08.268 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.163166+0000", 2024-09-21T14:59:08.268 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00060336000000000005 2024-09-21T14:59:08.268 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:08.268 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:08.268 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:08.268 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.268 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.268 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:23019 3.1e 3:7ca947c4:::rbd_data.11366acc894.000000000000000b:19 [read 360448~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:08.268 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.147666+0000", 2024-09-21T14:59:08.268 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 18.084419009000001, 2024-09-21T14:59:08.268 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015788706, 2024-09-21T14:59:08.268 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:08.268 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:08.268 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:08.269 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:08.269 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:08.269 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 23019 2024-09-21T14:59:08.269 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.269 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:08.269 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.269 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:08.269 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147666+0000", 2024-09-21T14:59:08.269 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:08.269 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.269 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.269 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:08.269 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147666+0000", 2024-09-21T14:59:08.269 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:08.270 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.270 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.270 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:08.270 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147666+0000", 2024-09-21T14:59:08.270 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:08.270 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.270 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.270 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:08.270 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147668+0000", 2024-09-21T14:59:08.270 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1969999999999999e-06 2024-09-21T14:59:08.270 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.270 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.270 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:08.270 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147673+0000", 2024-09-21T14:59:08.271 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2220000000000001e-06 2024-09-21T14:59:08.271 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.271 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.271 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:08.271 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147677+0000", 2024-09-21T14:59:08.271 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9620000000000004e-06 2024-09-21T14:59:08.271 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.271 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.271 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:08.271 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.163200+0000", 2024-09-21T14:59:08.271 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.01552215 2024-09-21T14:59:08.271 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.271 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.271 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:08.272 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.163294+0000", 2024-09-21T14:59:08.272 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4017000000000005e-05 2024-09-21T14:59:08.272 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.272 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.272 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:08.272 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.163455+0000", 2024-09-21T14:59:08.272 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000161478 2024-09-21T14:59:08.272 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:08.272 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:08.272 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:08.272 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.272 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.272 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:23020 3.1e 3:7ca947c4:::rbd_data.11366acc894.000000000000000b:19 [read 425984~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:08.272 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.147681+0000", 2024-09-21T14:59:08.272 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 18.084404279000001, 2024-09-21T14:59:08.272 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.016190145, 2024-09-21T14:59:08.273 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:08.273 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:08.273 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:08.273 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:08.273 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:08.273 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 23020 2024-09-21T14:59:08.273 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.273 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:08.273 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.273 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:08.273 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147681+0000", 2024-09-21T14:59:08.273 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:08.273 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.273 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.273 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:08.273 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147681+0000", 2024-09-21T14:59:08.274 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:08.274 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.274 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.274 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:08.274 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147681+0000", 2024-09-21T14:59:08.274 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:08.274 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.274 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.274 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:08.274 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147683+0000", 2024-09-21T14:59:08.274 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3470000000000001e-06 2024-09-21T14:59:08.274 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.274 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.275 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:08.275 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147689+0000", 2024-09-21T14:59:08.275 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5849999999999999e-06 2024-09-21T14:59:08.275 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.275 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.275 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:08.275 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147695+0000", 2024-09-21T14:59:08.275 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.964e-06 2024-09-21T14:59:08.275 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.275 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.275 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:08.275 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.163488+0000", 2024-09-21T14:59:08.275 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015793564999999999 2024-09-21T14:59:08.275 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.275 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.275 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:08.276 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.163532+0000", 2024-09-21T14:59:08.276 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3841000000000003e-05 2024-09-21T14:59:08.276 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.276 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.276 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:08.276 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.163871+0000", 2024-09-21T14:59:08.276 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00033914500000000001 2024-09-21T14:59:08.276 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:08.276 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:08.276 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:08.276 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.276 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.276 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:23021 3.1e 3:7ca947c4:::rbd_data.11366acc894.000000000000000b:19 [read 491520~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:08.276 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.147698+0000", 2024-09-21T14:59:08.276 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 18.084386881, 2024-09-21T14:59:08.276 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.016449135, 2024-09-21T14:59:08.277 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:08.277 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:08.277 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:08.277 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:08.277 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:08.277 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 23021 2024-09-21T14:59:08.277 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.277 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:08.277 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.277 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:08.277 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147698+0000", 2024-09-21T14:59:08.277 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:08.277 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.277 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.277 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:08.277 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147698+0000", 2024-09-21T14:59:08.278 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:08.278 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.278 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.278 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:08.278 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147698+0000", 2024-09-21T14:59:08.278 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:08.278 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.278 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.278 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:08.278 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147701+0000", 2024-09-21T14:59:08.278 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6299999999999998e-06 2024-09-21T14:59:08.278 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.278 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.278 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:08.278 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147706+0000", 2024-09-21T14:59:08.278 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3059999999999997e-06 2024-09-21T14:59:08.279 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.279 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.279 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:08.279 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.147711+0000", 2024-09-21T14:59:08.279 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0860000000000001e-06 2024-09-21T14:59:08.279 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.279 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.279 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:08.279 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.163897+0000", 2024-09-21T14:59:08.279 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.016186282 2024-09-21T14:59:08.279 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.279 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.279 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:08.279 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.163986+0000", 2024-09-21T14:59:08.279 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8676000000000003e-05 2024-09-21T14:59:08.280 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.280 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.280 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:08.280 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.164148+0000", 2024-09-21T14:59:08.280 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016148000000000001 2024-09-21T14:59:08.280 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:08.280 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:08.280 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:08.280 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.280 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.280 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:23022 3.1e 3:7ca947c4:::rbd_data.11366acc894.000000000000000b:19 [read 557056~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:08.280 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.148289+0000", 2024-09-21T14:59:08.280 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 18.083795996999999, 2024-09-21T14:59:08.280 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.016062979000000002, 2024-09-21T14:59:08.280 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:08.280 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:08.281 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:08.281 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:08.281 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:08.281 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 23022 2024-09-21T14:59:08.281 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.281 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:08.281 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.281 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:08.281 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.148289+0000", 2024-09-21T14:59:08.281 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:08.281 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.281 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.281 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:08.281 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.148289+0000", 2024-09-21T14:59:08.281 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:08.281 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.282 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.282 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:08.282 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.148288+0000", 2024-09-21T14:59:08.282 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:08.282 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.282 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.282 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:08.282 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.148295+0000", 2024-09-21T14:59:08.282 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8340000000000001e-06 2024-09-21T14:59:08.282 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.282 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.282 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:08.282 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.148304+0000", 2024-09-21T14:59:08.282 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0129999999999999e-06 2024-09-21T14:59:08.283 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.283 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.283 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:08.283 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.148311+0000", 2024-09-21T14:59:08.283 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1439999999999997e-06 2024-09-21T14:59:08.283 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.283 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.283 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:08.283 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.164182+0000", 2024-09-21T14:59:08.283 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015870825000000002 2024-09-21T14:59:08.283 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.283 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.283 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:08.283 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.164229+0000", 2024-09-21T14:59:08.284 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6508999999999998e-05 2024-09-21T14:59:08.284 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.284 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.284 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:08.284 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.164352+0000", 2024-09-21T14:59:08.284 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000123664 2024-09-21T14:59:08.284 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:08.284 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:08.284 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:08.284 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.284 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.284 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:23116 3.1e 3:7ca947c4:::rbd_data.11366acc894.000000000000000b:19 [read 180224~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:08.284 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.150296+0000", 2024-09-21T14:59:08.284 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 18.081789654000001, 2024-09-21T14:59:08.285 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015003996, 2024-09-21T14:59:08.285 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:08.285 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:08.285 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:08.285 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:08.285 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:08.285 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 23116 2024-09-21T14:59:08.285 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.285 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:08.285 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.285 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:08.285 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150296+0000", 2024-09-21T14:59:08.285 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:08.285 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.285 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.285 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:08.286 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150296+0000", 2024-09-21T14:59:08.286 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:08.286 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.286 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.286 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:08.286 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150295+0000", 2024-09-21T14:59:08.286 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:08.286 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.286 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.286 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:08.286 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150298+0000", 2024-09-21T14:59:08.286 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6749999999999998e-06 2024-09-21T14:59:08.286 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.286 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.286 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:08.287 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150304+0000", 2024-09-21T14:59:08.287 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6359999999999999e-06 2024-09-21T14:59:08.287 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.287 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.287 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:08.287 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150308+0000", 2024-09-21T14:59:08.287 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6260000000000003e-06 2024-09-21T14:59:08.287 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.287 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.287 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:08.287 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.165003+0000", 2024-09-21T14:59:08.287 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014694377 2024-09-21T14:59:08.287 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.287 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.287 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:08.288 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.165093+0000", 2024-09-21T14:59:08.288 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0137999999999997e-05 2024-09-21T14:59:08.288 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.288 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.288 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:08.288 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.165300+0000", 2024-09-21T14:59:08.288 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00020688799999999999 2024-09-21T14:59:08.288 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:08.288 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:08.288 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:08.288 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.288 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.288 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:23117 3.1e 3:7ca947c4:::rbd_data.11366acc894.000000000000000b:19 [read 245760~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:08.288 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.150312+0000", 2024-09-21T14:59:08.288 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 18.081773318, 2024-09-21T14:59:08.289 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015236939, 2024-09-21T14:59:08.289 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:08.289 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:08.289 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:08.289 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:08.289 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:08.289 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 23117 2024-09-21T14:59:08.289 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.289 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:08.289 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.289 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:08.289 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150312+0000", 2024-09-21T14:59:08.289 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:08.289 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.289 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.289 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:08.290 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150312+0000", 2024-09-21T14:59:08.290 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:08.290 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.290 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.290 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:08.290 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150312+0000", 2024-09-21T14:59:08.290 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:08.290 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.290 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.290 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:08.290 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150314+0000", 2024-09-21T14:59:08.290 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.154e-06 2024-09-21T14:59:08.290 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.291 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.291 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:08.291 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150320+0000", 2024-09-21T14:59:08.291 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6640000000000003e-06 2024-09-21T14:59:08.291 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.291 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.291 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:08.291 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150324+0000", 2024-09-21T14:59:08.291 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5669999999999996e-06 2024-09-21T14:59:08.291 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.291 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.291 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:08.291 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.165317+0000", 2024-09-21T14:59:08.291 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014993328 2024-09-21T14:59:08.291 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.292 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.292 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:08.292 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.165370+0000", 2024-09-21T14:59:08.292 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2036999999999997e-05 2024-09-21T14:59:08.292 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.292 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.292 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:08.292 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.165549+0000", 2024-09-21T14:59:08.292 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017947500000000001 2024-09-21T14:59:08.292 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:08.292 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:08.292 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:08.292 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.292 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.293 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:23118 3.1e 3:7ca947c4:::rbd_data.11366acc894.000000000000000b:19 [read 311296~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:08.293 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.150328+0000", 2024-09-21T14:59:08.293 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 18.081757173, 2024-09-21T14:59:08.293 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015505385999999999, 2024-09-21T14:59:08.293 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:08.293 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:08.293 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:08.293 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:08.293 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:08.293 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 23118 2024-09-21T14:59:08.293 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.293 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:08.293 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.293 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:08.293 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150328+0000", 2024-09-21T14:59:08.294 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:08.294 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.294 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.294 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:08.294 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150328+0000", 2024-09-21T14:59:08.294 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:08.294 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.294 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.294 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:08.294 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150328+0000", 2024-09-21T14:59:08.294 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:08.294 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.294 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.295 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:08.295 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150330+0000", 2024-09-21T14:59:08.295 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3410000000000001e-06 2024-09-21T14:59:08.295 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.295 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.295 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:08.295 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150336+0000", 2024-09-21T14:59:08.295 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.694e-06 2024-09-21T14:59:08.295 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.295 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.295 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:08.295 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150343+0000", 2024-09-21T14:59:08.295 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3050000000000004e-06 2024-09-21T14:59:08.295 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.296 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.296 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:08.296 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.165607+0000", 2024-09-21T14:59:08.296 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015264534 2024-09-21T14:59:08.296 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.296 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.296 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:08.296 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.165661+0000", 2024-09-21T14:59:08.296 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3949000000000001e-05 2024-09-21T14:59:08.296 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.296 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.296 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:08.296 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.165834+0000", 2024-09-21T14:59:08.296 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000172139 2024-09-21T14:59:08.297 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:08.297 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:08.297 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:08.297 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.297 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.297 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:23119 3.1e 3:7ca947c4:::rbd_data.11366acc894.000000000000000b:19 [read 376832~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:08.297 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.150347+0000", 2024-09-21T14:59:08.297 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 18.081738154, 2024-09-21T14:59:08.297 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015790017, 2024-09-21T14:59:08.297 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:08.297 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:08.297 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:08.297 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:08.298 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:08.298 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 23119 2024-09-21T14:59:08.298 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.298 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:08.298 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.298 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:08.298 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150347+0000", 2024-09-21T14:59:08.298 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:08.298 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.298 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.298 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:08.298 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150347+0000", 2024-09-21T14:59:08.298 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:08.298 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.298 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.299 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:08.299 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150347+0000", 2024-09-21T14:59:08.299 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:08.299 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.299 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.299 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:08.299 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150349+0000", 2024-09-21T14:59:08.299 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1730000000000002e-06 2024-09-21T14:59:08.299 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.299 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.299 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:08.299 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150355+0000", 2024-09-21T14:59:08.299 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9320000000000001e-06 2024-09-21T14:59:08.299 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.300 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.300 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:08.300 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150360+0000", 2024-09-21T14:59:08.300 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9450000000000001e-06 2024-09-21T14:59:08.300 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.300 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.300 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:08.300 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.165867+0000", 2024-09-21T14:59:08.300 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015507191 2024-09-21T14:59:08.300 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.300 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.300 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:08.300 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.165956+0000", 2024-09-21T14:59:08.300 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9235999999999997e-05 2024-09-21T14:59:08.300 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.301 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.301 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:08.301 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.166137+0000", 2024-09-21T14:59:08.301 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018088699999999999 2024-09-21T14:59:08.301 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:08.301 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:08.301 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:08.301 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.301 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.301 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:23120 3.1e 3:7ca947c4:::rbd_data.11366acc894.000000000000000b:19 [read 442368~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:08.301 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.150364+0000", 2024-09-21T14:59:08.301 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 18.081721478999999, 2024-09-21T14:59:08.301 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.016001277000000001, 2024-09-21T14:59:08.301 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:08.301 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:08.302 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:08.302 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:08.302 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:08.302 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 23120 2024-09-21T14:59:08.302 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.302 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:08.302 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.302 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:08.302 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150364+0000", 2024-09-21T14:59:08.302 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:08.302 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.302 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.302 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:08.302 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150364+0000", 2024-09-21T14:59:08.302 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:08.303 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.303 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.303 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:08.303 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150363+0000", 2024-09-21T14:59:08.303 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:08.303 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.303 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.303 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:08.303 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150366+0000", 2024-09-21T14:59:08.303 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1969999999999999e-06 2024-09-21T14:59:08.303 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.303 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.303 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:08.303 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150371+0000", 2024-09-21T14:59:08.304 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7459999999999997e-06 2024-09-21T14:59:08.304 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.304 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.304 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:08.304 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150376+0000", 2024-09-21T14:59:08.304 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6519999999999997e-06 2024-09-21T14:59:08.304 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.304 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.304 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:08.304 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.166166+0000", 2024-09-21T14:59:08.304 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015790286000000001 2024-09-21T14:59:08.304 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.304 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.304 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:08.305 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.166226+0000", 2024-09-21T14:59:08.305 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9682000000000002e-05 2024-09-21T14:59:08.305 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.305 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.305 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:08.305 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.166365+0000", 2024-09-21T14:59:08.305 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013911800000000001 2024-09-21T14:59:08.305 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:08.305 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:08.305 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:08.305 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.305 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.305 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:23121 3.1e 3:7ca947c4:::rbd_data.11366acc894.000000000000000b:19 [read 507904~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:08.305 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.150380+0000", 2024-09-21T14:59:08.306 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 18.081705348, 2024-09-21T14:59:08.306 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.016302104000000001, 2024-09-21T14:59:08.306 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:08.306 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:08.306 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:08.306 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:08.306 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:08.306 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 23121 2024-09-21T14:59:08.306 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.306 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:08.306 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.306 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:08.306 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150380+0000", 2024-09-21T14:59:08.307 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:08.307 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.307 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.307 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:08.307 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150380+0000", 2024-09-21T14:59:08.307 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:08.307 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.307 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.307 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:08.307 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150380+0000", 2024-09-21T14:59:08.307 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:08.307 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.307 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.307 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:08.307 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150382+0000", 2024-09-21T14:59:08.308 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1720000000000001e-06 2024-09-21T14:59:08.308 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.308 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.308 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:08.308 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150387+0000", 2024-09-21T14:59:08.308 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2569999999999998e-06 2024-09-21T14:59:08.308 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.308 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.308 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:08.308 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150392+0000", 2024-09-21T14:59:08.308 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1100000000000002e-06 2024-09-21T14:59:08.308 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.308 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.308 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:08.308 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.166392+0000", 2024-09-21T14:59:08.309 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015999394 2024-09-21T14:59:08.309 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.309 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.309 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:08.309 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.166461+0000", 2024-09-21T14:59:08.309 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9035999999999994e-05 2024-09-21T14:59:08.309 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.309 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.309 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:08.309 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.166682+0000", 2024-09-21T14:59:08.309 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00022143900000000001 2024-09-21T14:59:08.309 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:08.309 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:08.309 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:08.309 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.310 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.310 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:23122 3.1e 3:7ca947c4:::rbd_data.11366acc894.000000000000000b:19 [read 573440~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:08.310 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.150396+0000", 2024-09-21T14:59:08.310 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 18.081689425, 2024-09-21T14:59:08.310 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.016563735999999999, 2024-09-21T14:59:08.310 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:08.310 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:08.310 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:08.310 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:08.310 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:08.310 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 23122 2024-09-21T14:59:08.310 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.310 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:08.310 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.311 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:08.311 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150396+0000", 2024-09-21T14:59:08.311 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:08.311 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.311 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.311 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:08.311 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150396+0000", 2024-09-21T14:59:08.311 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:08.311 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.311 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.311 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:08.311 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150396+0000", 2024-09-21T14:59:08.311 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:08.311 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.311 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.312 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:08.312 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150398+0000", 2024-09-21T14:59:08.312 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1739999999999999e-06 2024-09-21T14:59:08.312 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.312 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.312 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:08.312 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150403+0000", 2024-09-21T14:59:08.312 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1490000000000002e-06 2024-09-21T14:59:08.312 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.312 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.312 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:08.312 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.150408+0000", 2024-09-21T14:59:08.312 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6460000000000001e-06 2024-09-21T14:59:08.312 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.312 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.312 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:08.313 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.166716+0000", 2024-09-21T14:59:08.313 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.016308804999999999 2024-09-21T14:59:08.313 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.313 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.313 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:08.313 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.166792+0000", 2024-09-21T14:59:08.313 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5479e-05 2024-09-21T14:59:08.313 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.313 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.313 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:08.313 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.166960+0000", 2024-09-21T14:59:08.313 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000167799 2024-09-21T14:59:08.313 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:08.314 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:08.314 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:08.314 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.314 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.314 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:27274 3.18 3:1fcceaa3:::rbd_data.11367fb81126.000000000000000a:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-21T14:59:08.314 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:55.596241+0000", 2024-09-21T14:59:08.314 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 12.635843924, 2024-09-21T14:59:08.314 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.024271564999999998, 2024-09-21T14:59:08.314 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:08.314 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:08.314 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:08.314 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:08.314 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:08.315 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 27274 2024-09-21T14:59:08.315 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.315 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:08.315 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.315 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:08.315 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:55.596241+0000", 2024-09-21T14:59:08.315 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:08.315 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.315 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.315 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:08.315 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:55.596241+0000", 2024-09-21T14:59:08.315 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:08.315 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.315 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.315 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:08.315 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:55.596239+0000", 2024-09-21T14:59:08.316 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-21T14:59:08.316 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.316 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.316 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:08.316 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:55.596252+0000", 2024-09-21T14:59:08.316 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3169e-05 2024-09-21T14:59:08.316 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.316 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.316 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:08.316 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:55.596286+0000", 2024-09-21T14:59:08.316 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.311e-05 2024-09-21T14:59:08.316 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.316 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.316 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:08.317 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:55.596300+0000", 2024-09-21T14:59:08.317 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4369000000000001e-05 2024-09-21T14:59:08.317 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.317 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.317 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:08.317 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:55.596397+0000", 2024-09-21T14:59:08.317 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6924999999999998e-05 2024-09-21T14:59:08.317 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.317 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.317 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:08.317 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:55.596626+0000", 2024-09-21T14:59:08.317 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00022904499999999999 2024-09-21T14:59:08.317 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.317 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.317 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T14:59:08.318 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:55.602376+0000", 2024-09-21T14:59:08.318 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.005750196 2024-09-21T14:59:08.318 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.318 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.318 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:59:08.318 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:55.620263+0000", 2024-09-21T14:59:08.318 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.017886868 2024-09-21T14:59:08.318 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.318 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.318 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:08.318 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:55.620353+0000", 2024-09-21T14:59:08.318 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0105999999999994e-05 2024-09-21T14:59:08.318 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.318 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.319 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:59:08.319 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:55.620449+0000", 2024-09-21T14:59:08.319 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5514999999999996e-05 2024-09-21T14:59:08.319 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.319 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.319 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:08.319 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:55.620513+0000", 2024-09-21T14:59:08.319 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4451999999999999e-05 2024-09-21T14:59:08.319 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:08.319 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:08.319 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:08.319 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:08.319 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:08.319 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:59:08.321 INFO:teuthology.orchestra.run.smithi032.stdout:1365 write 0xb7eb8fc thru 0xb7f110e (0x5813 bytes) 2024-09-21T14:59:08.321 INFO:teuthology.orchestra.run.smithi032.stdout:1366 trunc from 0xe6a3092 to 0x23924ea 2024-09-21T14:59:08.321 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:08.321 INFO:teuthology.orchestra.run.smithi032.stdout:1368 trunc from 0x23924ea to 0x92af314 2024-09-21T14:59:08.321 INFO:teuthology.orchestra.run.smithi032.stdout:1369 trunc from 0x92af314 to 0x97db42e 2024-09-21T14:59:08.321 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T14:59:08.322 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:59:08.321+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:59:08.324 INFO:teuthology.orchestra.run.smithi032.stderr:nodeep-scrub is set 2024-09-21T14:59:08.328 INFO:teuthology.orchestra.run.smithi032.stdout:1370 read 0x782373 thru 0x7877cd (0x545b bytes) 2024-09-21T14:59:08.329 INFO:teuthology.orchestra.run.smithi032.stdout:1371 read 0x2545649 thru 0x2545657 (0xf bytes) 2024-09-21T14:59:08.329 INFO:teuthology.orchestra.run.smithi032.stdout:1372 write 0x254c9b5 thru 0x255539c (0x89e8 bytes) 2024-09-21T14:59:08.340 INFO:teuthology.orchestra.run.smithi032.stdout:1373 writesame 0x4f7238 thru 0x4fc734 data_size 0x1(0x54fd bytes) 2024-09-21T14:59:08.355 INFO:teuthology.orchestra.run.smithi032.stdout:1374 trunc from 0x97db42e to 0x8d2fa5b 2024-09-21T14:59:08.365 INFO:teuthology.orchestra.run.smithi032.stdout:1375 write 0x45dd72b thru 0x45e991a (0xc1f0 bytes) 2024-09-21T14:59:08.382 INFO:teuthology.orchestra.run.smithi032.stdout:1376 punch from 0x27ba21e to 0x27bc190, (0x1f72 bytes) 2024-09-21T14:59:08.382 INFO:teuthology.orchestra.run.smithi032.stdout:1377 punch from 0x234d982 to 0x2358133, (0xa7b1 bytes) 2024-09-21T14:59:08.383 INFO:teuthology.orchestra.run.smithi032.stdout:1380 trunc from 0x8d2fa5b to 0x95f00c3 2024-09-21T14:59:08.395 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:08.395 INFO:teuthology.orchestra.run.smithi032.stdout:1382 write 0x896fe67 thru 0x897890e (0x8aa8 bytes) 2024-09-21T14:59:08.406 INFO:teuthology.orchestra.run.smithi032.stdout:1383 writesame 0x1cc4676 thru 0x1cd16c1 data_size 0x2(0xd04c bytes) 2024-09-21T14:59:08.422 INFO:teuthology.orchestra.run.smithi032.stdout:1386 writesame 0x9009ea3 thru 0x90155a8 data_size 0x39(0xb706 bytes) 2024-09-21T14:59:08.423 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:59:08.421+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:59:08.433 INFO:teuthology.orchestra.run.smithi032.stdout:1387 read 0x27bad53 thru 0x27c1a9f (0x6d4d bytes) 2024-09-21T14:59:08.433 INFO:teuthology.orchestra.run.smithi032.stdout:1388 writesame 0x8f54403 thru 0x8f5d070 data_size 0x19(0x8c6e bytes) 2024-09-21T14:59:08.448 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T14:59:08.449 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T14:59:08.449 INFO:tasks.thrashosds.thrasher:Reweighting osd 2 to 0.351468631756916 2024-09-21T14:59:08.449 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 0.351468631756916 2024-09-21T14:59:08.456 INFO:teuthology.orchestra.run.smithi032.stdout:1389 writesame 0xa5ac7cf thru 0xa5bbaf0 data_size 0x1(0xf322 bytes) 2024-09-21T14:59:08.456 INFO:teuthology.orchestra.run.smithi032.stdout:1390 punch from 0x27b7190 to 0x27c68a8, (0xf718 bytes) 2024-09-21T14:59:08.456 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:08.474 INFO:teuthology.orchestra.run.smithi032.stdout:1393 trunc from 0xa5bbaf1 to 0x1eedec9 2024-09-21T14:59:08.486 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:59:08.487 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:59:08.487 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T14:59:08.487 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:59:08.497 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T14:59:08.499 INFO:teuthology.orchestra.run.smithi032.stdout:1396 read 0xfaaae4 thru 0xfb99e3 (0xef00 bytes) 2024-09-21T14:59:08.499 INFO:teuthology.orchestra.run.smithi032.stdout:1398 clone 13 order 18 su 1024 sc 3 2024-09-21T14:59:08.523 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:59:08.520+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:59:08.593 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:59:08.593 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:59:08.593 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T14:59:08.593 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T14:59:08.593 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:59:08.603 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T14:59:08.624 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:59:08.621+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:59:08.725 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:59:08.725+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:59:08.759 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:59:08.759 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T14:59:08.760 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T14:59:08.760 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T14:59:08.760 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.760 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:10651 3.12 3:4e9d23af:::rbd_data.1136f5ed5dd6.0000000000000020:head [write 0~81920 [fadvise_sequential+fadvise_nocache],write 90112~8192 [fadvise_sequential+fadvise_nocache],write 122880~122880 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:08.760 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.292476+0000", 2024-09-21T14:59:08.760 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 29.464371318000001, 2024-09-21T14:59:08.760 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012765669, 2024-09-21T14:59:08.760 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:08.760 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:08.760 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:08.761 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:08.761 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:08.761 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 10651 2024-09-21T14:59:08.761 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.761 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:08.761 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.761 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:08.761 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292476+0000", 2024-09-21T14:59:08.761 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:08.761 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.761 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.761 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:08.761 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292476+0000", 2024-09-21T14:59:08.761 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:08.762 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.762 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.762 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:08.762 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292475+0000", 2024-09-21T14:59:08.762 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:08.762 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.762 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.762 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:08.762 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292696+0000", 2024-09-21T14:59:08.762 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000220201 2024-09-21T14:59:08.762 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.762 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.763 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:08.763 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292711+0000", 2024-09-21T14:59:08.763 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5803000000000001e-05 2024-09-21T14:59:08.763 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.763 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.763 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:08.763 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292721+0000", 2024-09-21T14:59:08.763 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6160000000000008e-06 2024-09-21T14:59:08.763 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.763 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.763 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:08.763 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292757+0000", 2024-09-21T14:59:08.763 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6211999999999998e-05 2024-09-21T14:59:08.764 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.764 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.764 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:08.764 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292878+0000", 2024-09-21T14:59:08.764 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000120306 2024-09-21T14:59:08.764 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.764 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.764 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 3", 2024-09-21T14:59:08.764 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.293019+0000", 2024-09-21T14:59:08.764 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014186400000000001 2024-09-21T14:59:08.764 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.764 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.764 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:59:08.764 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.294898+0000", 2024-09-21T14:59:08.765 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001878434 2024-09-21T14:59:08.765 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.765 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.765 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:08.765 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.305162+0000", 2024-09-21T14:59:08.765 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.01026373 2024-09-21T14:59:08.765 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.765 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.765 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:59:08.765 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.305209+0000", 2024-09-21T14:59:08.765 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7615999999999999e-05 2024-09-21T14:59:08.765 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.765 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.766 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:08.766 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.305242+0000", 2024-09-21T14:59:08.766 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2979999999999999e-05 2024-09-21T14:59:08.766 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:08.766 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:08.766 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:08.766 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.766 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.766 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:11285 3.12 3:4edf73d2:::rbd_data.1136f5ed5dd6.0000000000000039:head [write 90112~32768 [fadvise_sequential+fadvise_nocache],write 139264~24576 [fadvise_sequential+fadvise_nocache],write 180224~24576 [fadvise_sequential+fadvise_nocache],write 221184~40960 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:08.766 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.539470+0000", 2024-09-21T14:59:08.766 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 29.217377566, 2024-09-21T14:59:08.766 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0083360099999999996, 2024-09-21T14:59:08.766 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:08.767 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:08.767 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:08.767 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:08.767 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:08.767 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 11285 2024-09-21T14:59:08.767 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.767 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:08.767 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.767 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:08.767 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.539470+0000", 2024-09-21T14:59:08.767 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:08.767 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.767 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.767 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:08.768 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.539470+0000", 2024-09-21T14:59:08.768 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:08.768 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.768 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.768 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:08.768 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.539469+0000", 2024-09-21T14:59:08.768 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:08.768 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.768 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.768 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:08.768 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.539652+0000", 2024-09-21T14:59:08.768 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000183084 2024-09-21T14:59:08.768 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.769 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.769 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:08.769 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.539666+0000", 2024-09-21T14:59:08.769 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3947e-05 2024-09-21T14:59:08.769 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.769 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.769 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:08.769 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.539673+0000", 2024-09-21T14:59:08.769 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7309999999999996e-06 2024-09-21T14:59:08.769 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.769 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.769 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:08.769 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.539714+0000", 2024-09-21T14:59:08.769 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1539000000000003e-05 2024-09-21T14:59:08.770 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.770 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.770 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:08.770 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.539832+0000", 2024-09-21T14:59:08.770 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011753500000000001 2024-09-21T14:59:08.770 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.770 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.770 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 3", 2024-09-21T14:59:08.770 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.540005+0000", 2024-09-21T14:59:08.770 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000173031 2024-09-21T14:59:08.770 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.770 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.770 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:59:08.771 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.541606+0000", 2024-09-21T14:59:08.771 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001600899 2024-09-21T14:59:08.771 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.771 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.771 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:08.771 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.547709+0000", 2024-09-21T14:59:08.771 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0061027110000000002 2024-09-21T14:59:08.771 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.771 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.771 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:59:08.771 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.547767+0000", 2024-09-21T14:59:08.771 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8248999999999997e-05 2024-09-21T14:59:08.772 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.772 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.772 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:08.772 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.547806+0000", 2024-09-21T14:59:08.772 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9341000000000002e-05 2024-09-21T14:59:08.772 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:08.772 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:08.772 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:08.772 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.772 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.772 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12189 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 40960~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:08.772 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.618667+0000", 2024-09-21T14:59:08.773 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 29.138181115999998, 2024-09-21T14:59:08.773 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0083014549999999993, 2024-09-21T14:59:08.773 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:08.773 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:08.773 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:08.773 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:08.773 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:08.773 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12189 2024-09-21T14:59:08.773 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.773 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:08.773 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.773 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:08.773 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618667+0000", 2024-09-21T14:59:08.774 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:08.774 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.774 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.774 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:08.774 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618667+0000", 2024-09-21T14:59:08.774 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:08.774 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.774 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.774 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:08.774 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618666+0000", 2024-09-21T14:59:08.774 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:08.774 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.775 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.775 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:08.775 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618669+0000", 2024-09-21T14:59:08.775 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.554e-06 2024-09-21T14:59:08.775 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.775 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.775 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:08.775 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618675+0000", 2024-09-21T14:59:08.775 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2570000000000004e-06 2024-09-21T14:59:08.775 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.775 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.775 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:08.776 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618680+0000", 2024-09-21T14:59:08.776 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6539999999999998e-06 2024-09-21T14:59:08.776 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.776 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.776 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:08.776 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.626683+0000", 2024-09-21T14:59:08.776 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0080035880000000007 2024-09-21T14:59:08.776 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.776 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.776 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:08.776 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.626764+0000", 2024-09-21T14:59:08.776 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0672000000000003e-05 2024-09-21T14:59:08.777 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.777 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.777 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:08.777 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.626968+0000", 2024-09-21T14:59:08.777 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00020407499999999999 2024-09-21T14:59:08.777 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:08.777 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:08.777 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:08.777 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.777 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.777 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12193 3.4 3:25437d51:::rbd_data.11363a21d2be.0000000000000025:10 [read 425984~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:08.777 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.618755+0000", 2024-09-21T14:59:08.777 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 29.138093035000001, 2024-09-21T14:59:08.778 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086461379999999994, 2024-09-21T14:59:08.778 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:08.778 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:08.778 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:08.778 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:08.778 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:08.778 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12193 2024-09-21T14:59:08.778 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.778 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:08.778 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.778 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:08.778 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618755+0000", 2024-09-21T14:59:08.778 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:08.779 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.779 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.779 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:08.779 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618755+0000", 2024-09-21T14:59:08.779 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:08.779 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.779 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.779 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:08.779 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618754+0000", 2024-09-21T14:59:08.779 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:08.779 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.779 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.779 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:08.780 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618759+0000", 2024-09-21T14:59:08.780 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4020000000000004e-06 2024-09-21T14:59:08.780 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.780 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.780 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:08.780 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618767+0000", 2024-09-21T14:59:08.780 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7490000000000004e-06 2024-09-21T14:59:08.780 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.780 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.780 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:08.780 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618773+0000", 2024-09-21T14:59:08.780 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0939999999999996e-06 2024-09-21T14:59:08.780 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.781 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.781 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:08.781 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.627115+0000", 2024-09-21T14:59:08.781 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0083416840000000003 2024-09-21T14:59:08.781 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.781 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.781 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:08.781 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.627214+0000", 2024-09-21T14:59:08.781 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9042000000000005e-05 2024-09-21T14:59:08.781 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.781 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.781 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:08.781 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.627401+0000", 2024-09-21T14:59:08.782 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018677200000000001 2024-09-21T14:59:08.782 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:08.782 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:08.782 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:08.782 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.782 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.782 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12212 3.4 3:21b43b45:::rbd_data.11363a21d2be.000000000000002b:10 [read 237568~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:08.782 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.619922+0000", 2024-09-21T14:59:08.782 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 29.136926051, 2024-09-21T14:59:08.782 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.008324372, 2024-09-21T14:59:08.782 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:08.782 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:08.782 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:08.783 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:08.783 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:08.783 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12212 2024-09-21T14:59:08.783 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.783 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:08.783 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.783 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:08.783 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619922+0000", 2024-09-21T14:59:08.783 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:08.783 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.783 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.783 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:08.784 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619922+0000", 2024-09-21T14:59:08.784 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:08.784 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.784 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.784 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:08.784 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619921+0000", 2024-09-21T14:59:08.784 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:08.784 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.784 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.784 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:08.784 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619924+0000", 2024-09-21T14:59:08.784 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.999e-06 2024-09-21T14:59:08.784 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.785 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.785 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:08.785 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619934+0000", 2024-09-21T14:59:08.785 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.8120000000000002e-06 2024-09-21T14:59:08.785 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.785 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.785 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:08.785 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619939+0000", 2024-09-21T14:59:08.785 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7729999999999999e-06 2024-09-21T14:59:08.785 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.785 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.785 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:08.786 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628005+0000", 2024-09-21T14:59:08.786 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0080663330000000002 2024-09-21T14:59:08.786 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.786 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.786 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:08.786 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628073+0000", 2024-09-21T14:59:08.786 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7479999999999998e-05 2024-09-21T14:59:08.786 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.786 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.786 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:08.786 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628246+0000", 2024-09-21T14:59:08.786 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017337400000000001 2024-09-21T14:59:08.787 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:08.787 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:08.787 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:08.787 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.787 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.787 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12213 3.4 3:21b43b45:::rbd_data.11363a21d2be.000000000000002b:10 [read 270336~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:08.787 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.619945+0000", 2024-09-21T14:59:08.787 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 29.136902768999999, 2024-09-21T14:59:08.787 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086195049999999995, 2024-09-21T14:59:08.787 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:08.787 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:08.788 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:08.788 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:08.788 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:08.788 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12213 2024-09-21T14:59:08.788 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.788 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:08.788 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.788 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:08.788 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619945+0000", 2024-09-21T14:59:08.788 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:08.788 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.788 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.788 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:08.789 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619945+0000", 2024-09-21T14:59:08.789 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:08.789 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.789 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.789 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:08.789 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619945+0000", 2024-09-21T14:59:08.789 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:08.789 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.789 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.789 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:08.789 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619947+0000", 2024-09-21T14:59:08.789 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5780000000000001e-06 2024-09-21T14:59:08.789 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.790 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.790 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:08.790 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619957+0000", 2024-09-21T14:59:08.790 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3500000000000003e-06 2024-09-21T14:59:08.790 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.790 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.790 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:08.790 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619964+0000", 2024-09-21T14:59:08.790 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.199e-06 2024-09-21T14:59:08.790 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.790 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.791 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:08.791 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628285+0000", 2024-09-21T14:59:08.791 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0083210939999999994 2024-09-21T14:59:08.791 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.791 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.791 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:08.791 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628361+0000", 2024-09-21T14:59:08.791 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5873999999999999e-05 2024-09-21T14:59:08.791 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.791 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.792 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:08.792 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628565+0000", 2024-09-21T14:59:08.792 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000203799 2024-09-21T14:59:08.792 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:08.792 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:08.792 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:08.792 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.792 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.792 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12214 3.4 3:21b43b45:::rbd_data.11363a21d2be.000000000000002b:10 [read 376832~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:08.792 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.619969+0000", 2024-09-21T14:59:08.792 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 29.136878767999999, 2024-09-21T14:59:08.792 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0090292089999999998, 2024-09-21T14:59:08.792 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:08.792 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:08.793 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:08.793 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:08.793 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:08.793 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12214 2024-09-21T14:59:08.793 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.793 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:08.793 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.793 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:08.793 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619969+0000", 2024-09-21T14:59:08.793 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:08.793 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.793 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.793 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:08.794 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619969+0000", 2024-09-21T14:59:08.794 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:08.794 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.794 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.794 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:08.794 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619969+0000", 2024-09-21T14:59:08.794 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:08.794 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.794 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.794 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:08.794 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619971+0000", 2024-09-21T14:59:08.794 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1830000000000001e-06 2024-09-21T14:59:08.795 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.795 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.795 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:08.795 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619976+0000", 2024-09-21T14:59:08.795 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7129999999999996e-06 2024-09-21T14:59:08.795 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.795 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.795 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:08.795 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619981+0000", 2024-09-21T14:59:08.795 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2460000000000003e-06 2024-09-21T14:59:08.795 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.795 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.796 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:08.796 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628602+0000", 2024-09-21T14:59:08.796 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086209519999999994 2024-09-21T14:59:08.796 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.796 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.796 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:08.796 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628734+0000", 2024-09-21T14:59:08.796 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000132242 2024-09-21T14:59:08.796 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.796 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.796 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:08.796 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628998+0000", 2024-09-21T14:59:08.796 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00026419799999999999 2024-09-21T14:59:08.796 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:08.797 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:08.797 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:08.797 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.797 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.797 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12215 3.4 3:21b43b45:::rbd_data.11363a21d2be.000000000000002b:10 [read 409600~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:08.797 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.619985+0000", 2024-09-21T14:59:08.797 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 29.136862701999998, 2024-09-21T14:59:08.797 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0095457319999999995, 2024-09-21T14:59:08.797 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:08.797 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:08.797 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:08.797 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:08.797 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:08.798 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12215 2024-09-21T14:59:08.798 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.798 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:08.798 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.798 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:08.798 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619985+0000", 2024-09-21T14:59:08.798 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:08.798 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.798 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.798 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:08.798 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619985+0000", 2024-09-21T14:59:08.798 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:08.798 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.799 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.799 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:08.799 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619985+0000", 2024-09-21T14:59:08.799 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:08.799 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.799 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.799 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:08.799 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619987+0000", 2024-09-21T14:59:08.799 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6129999999999998e-06 2024-09-21T14:59:08.799 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.799 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.799 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:08.800 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619993+0000", 2024-09-21T14:59:08.800 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0820000000000004e-06 2024-09-21T14:59:08.800 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.800 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.800 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:08.800 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619998+0000", 2024-09-21T14:59:08.800 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7759999999999997e-06 2024-09-21T14:59:08.800 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.800 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.800 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:08.800 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629022+0000", 2024-09-21T14:59:08.800 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0090237489999999993 2024-09-21T14:59:08.800 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.800 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.801 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:08.801 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629098+0000", 2024-09-21T14:59:08.801 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5735999999999995e-05 2024-09-21T14:59:08.801 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.801 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.801 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:08.801 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629531+0000", 2024-09-21T14:59:08.801 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00043310100000000003 2024-09-21T14:59:08.801 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:08.801 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:08.801 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:08.801 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.801 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.802 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12216 3.4 3:21b43b45:::rbd_data.11363a21d2be.000000000000002b:10 [read 516096~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:08.802 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.620003+0000", 2024-09-21T14:59:08.802 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 29.136844706000002, 2024-09-21T14:59:08.802 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0098850169999999994, 2024-09-21T14:59:08.802 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:08.802 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:08.802 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:08.802 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:08.802 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:08.802 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12216 2024-09-21T14:59:08.802 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.802 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:08.802 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.803 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:08.803 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620003+0000", 2024-09-21T14:59:08.803 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:08.803 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.803 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.803 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:08.803 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620003+0000", 2024-09-21T14:59:08.803 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:08.803 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.803 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.803 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:08.803 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620003+0000", 2024-09-21T14:59:08.803 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:08.804 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.804 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.804 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:08.804 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620006+0000", 2024-09-21T14:59:08.804 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4350000000000001e-06 2024-09-21T14:59:08.804 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.804 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.804 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:08.804 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620013+0000", 2024-09-21T14:59:08.804 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4219999999999997e-06 2024-09-21T14:59:08.804 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.804 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.805 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:08.805 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620017+0000", 2024-09-21T14:59:08.805 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.865e-06 2024-09-21T14:59:08.805 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.805 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.805 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:08.805 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629581+0000", 2024-09-21T14:59:08.805 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0095631989999999997 2024-09-21T14:59:08.805 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.805 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.805 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:08.805 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629691+0000", 2024-09-21T14:59:08.805 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000110586 2024-09-21T14:59:08.806 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.806 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.806 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:08.806 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629888+0000", 2024-09-21T14:59:08.806 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019691199999999999 2024-09-21T14:59:08.806 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:08.806 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:08.806 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:08.806 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.806 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.806 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12222 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 49152~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:08.807 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.620027+0000", 2024-09-21T14:59:08.807 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 29.136820360000002, 2024-09-21T14:59:08.807 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010228131999999999, 2024-09-21T14:59:08.807 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:08.807 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:08.807 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:08.807 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:08.807 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:08.807 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12222 2024-09-21T14:59:08.807 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.807 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:08.807 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.807 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:08.808 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620027+0000", 2024-09-21T14:59:08.808 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:08.808 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.808 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.808 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:08.808 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620027+0000", 2024-09-21T14:59:08.808 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:08.808 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.808 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.808 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:08.808 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620027+0000", 2024-09-21T14:59:08.809 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:08.809 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.809 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.809 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:08.809 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620030+0000", 2024-09-21T14:59:08.809 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0400000000000001e-06 2024-09-21T14:59:08.809 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.809 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.809 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:08.809 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620037+0000", 2024-09-21T14:59:08.809 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8220000000000001e-06 2024-09-21T14:59:08.809 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.810 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.810 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:08.810 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620042+0000", 2024-09-21T14:59:08.810 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.843e-06 2024-09-21T14:59:08.810 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.810 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.810 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:08.810 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629922+0000", 2024-09-21T14:59:08.810 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0098805100000000003 2024-09-21T14:59:08.810 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.810 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.811 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:08.811 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.630020+0000", 2024-09-21T14:59:08.811 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7596999999999994e-05 2024-09-21T14:59:08.811 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.811 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.811 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:08.811 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.630256+0000", 2024-09-21T14:59:08.811 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00023573299999999999 2024-09-21T14:59:08.811 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:08.811 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:08.811 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:08.812 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.812 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.812 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12253 3.4 3:21b43b45:::rbd_data.11363a21d2be.000000000000002b:10 [read 417792~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:08.812 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.624578+0000", 2024-09-21T14:59:08.812 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 29.132269992000001, 2024-09-21T14:59:08.812 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0085112079999999993, 2024-09-21T14:59:08.812 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:08.812 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:08.812 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:08.812 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:08.812 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:08.812 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12253 2024-09-21T14:59:08.813 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.813 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:08.813 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.813 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:08.813 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624578+0000", 2024-09-21T14:59:08.813 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:08.813 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.813 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.813 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:08.813 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624578+0000", 2024-09-21T14:59:08.814 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:08.814 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.814 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.814 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:08.814 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624578+0000", 2024-09-21T14:59:08.814 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:08.814 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.814 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.814 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:08.814 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624579+0000", 2024-09-21T14:59:08.814 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.688e-06 2024-09-21T14:59:08.814 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.815 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.815 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:08.815 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624582+0000", 2024-09-21T14:59:08.815 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2260000000000001e-06 2024-09-21T14:59:08.815 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.815 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.815 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:08.815 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624585+0000", 2024-09-21T14:59:08.815 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4820000000000001e-06 2024-09-21T14:59:08.815 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.815 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.816 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:08.816 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.632580+0000", 2024-09-21T14:59:08.816 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0079948959999999996 2024-09-21T14:59:08.816 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.816 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.816 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:08.816 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.632683+0000", 2024-09-21T14:59:08.816 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010308 2024-09-21T14:59:08.816 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.816 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.816 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:08.816 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.633089+0000", 2024-09-21T14:59:08.816 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00040612599999999998 2024-09-21T14:59:08.817 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:08.817 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:08.817 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:08.817 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.817 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.817 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12260 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 57344~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:08.817 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.624587+0000", 2024-09-21T14:59:08.817 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 29.132260505000001, 2024-09-21T14:59:08.817 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0088282069999999994, 2024-09-21T14:59:08.817 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:08.817 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:08.817 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:08.817 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:08.818 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:08.818 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12260 2024-09-21T14:59:08.818 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.818 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:08.818 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.818 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:08.818 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624587+0000", 2024-09-21T14:59:08.818 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:08.818 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.818 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.818 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:08.818 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624587+0000", 2024-09-21T14:59:08.818 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:08.819 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.819 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.819 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:08.819 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624587+0000", 2024-09-21T14:59:08.819 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-21T14:59:08.819 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.819 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.819 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:08.819 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624589+0000", 2024-09-21T14:59:08.819 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4339999999999999e-06 2024-09-21T14:59:08.819 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.819 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.819 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:08.819 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624593+0000", 2024-09-21T14:59:08.820 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5859999999999998e-06 2024-09-21T14:59:08.820 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.820 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.820 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:08.820 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624596+0000", 2024-09-21T14:59:08.820 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5830000000000001e-06 2024-09-21T14:59:08.820 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.820 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.820 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:08.820 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.633132+0000", 2024-09-21T14:59:08.820 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.008536666 2024-09-21T14:59:08.820 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.821 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.821 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:08.821 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.633216+0000", 2024-09-21T14:59:08.821 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3387999999999995e-05 2024-09-21T14:59:08.821 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.821 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.821 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:08.821 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.633415+0000", 2024-09-21T14:59:08.821 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019976799999999999 2024-09-21T14:59:08.821 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:08.821 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:08.821 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:08.821 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.822 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.822 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12516 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 245760~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:08.822 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645194+0000", 2024-09-21T14:59:08.822 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 29.111653967999999, 2024-09-21T14:59:08.822 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0084284990000000008, 2024-09-21T14:59:08.822 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:08.822 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:08.822 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:08.822 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:08.822 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:08.822 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12516 2024-09-21T14:59:08.822 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.823 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:08.823 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.823 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:08.823 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645194+0000", 2024-09-21T14:59:08.823 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:08.823 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.823 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.823 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:08.823 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645194+0000", 2024-09-21T14:59:08.823 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:08.823 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.823 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.824 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:08.824 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645193+0000", 2024-09-21T14:59:08.824 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:08.824 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.824 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.824 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:08.824 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645196+0000", 2024-09-21T14:59:08.824 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1030000000000002e-06 2024-09-21T14:59:08.824 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.824 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.824 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:08.825 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645205+0000", 2024-09-21T14:59:08.825 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6200000000000005e-06 2024-09-21T14:59:08.825 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.825 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.825 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:08.825 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645210+0000", 2024-09-21T14:59:08.825 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6280000000000004e-06 2024-09-21T14:59:08.825 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.825 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.825 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:08.825 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.653305+0000", 2024-09-21T14:59:08.825 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0080957879999999996 2024-09-21T14:59:08.825 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.826 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.826 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:08.826 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.653389+0000", 2024-09-21T14:59:08.826 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3242e-05 2024-09-21T14:59:08.826 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.826 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.826 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:08.826 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.653622+0000", 2024-09-21T14:59:08.826 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00023361599999999999 2024-09-21T14:59:08.826 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:08.826 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:08.826 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:08.827 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.827 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.827 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12517 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 278528~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:08.827 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645214+0000", 2024-09-21T14:59:08.827 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 29.111633855000001, 2024-09-21T14:59:08.827 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086832899999999998, 2024-09-21T14:59:08.827 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:08.827 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:08.827 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:08.827 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:08.828 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:08.828 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12517 2024-09-21T14:59:08.828 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.828 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:08.828 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.828 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:08.828 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645214+0000", 2024-09-21T14:59:08.828 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:08.828 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.828 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.828 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:08.828 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645214+0000", 2024-09-21T14:59:08.828 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:08.829 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.829 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.829 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:08.829 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645214+0000", 2024-09-21T14:59:08.829 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:08.829 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.829 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.829 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:08.829 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645216+0000", 2024-09-21T14:59:08.829 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5579999999999999e-06 2024-09-21T14:59:08.829 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.829 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.829 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:08.830 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645224+0000", 2024-09-21T14:59:08.830 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2439999999999993e-06 2024-09-21T14:59:08.830 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.830 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.830 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:08.830 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645231+0000", 2024-09-21T14:59:08.830 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1859999999999997e-06 2024-09-21T14:59:08.830 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.830 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.830 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:08.830 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.653651+0000", 2024-09-21T14:59:08.830 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0084204220000000003 2024-09-21T14:59:08.831 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.831 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.831 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:08.831 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.653726+0000", 2024-09-21T14:59:08.831 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5356000000000002e-05 2024-09-21T14:59:08.831 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.831 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.831 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:08.831 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.653897+0000", 2024-09-21T14:59:08.831 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000170848 2024-09-21T14:59:08.831 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:08.832 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:08.832 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:08.832 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.832 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.832 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12518 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 385024~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:08.832 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645235+0000", 2024-09-21T14:59:08.832 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 29.111612629, 2024-09-21T14:59:08.832 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0089841020000000008, 2024-09-21T14:59:08.832 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:08.832 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:08.832 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:08.832 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:08.833 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:08.833 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12518 2024-09-21T14:59:08.833 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.833 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:08.833 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.833 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:08.833 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645235+0000", 2024-09-21T14:59:08.833 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:08.833 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.833 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.833 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:08.833 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645235+0000", 2024-09-21T14:59:08.834 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:08.834 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.834 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.834 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:08.834 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645235+0000", 2024-09-21T14:59:08.834 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:08.834 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.834 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.834 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:08.834 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645238+0000", 2024-09-21T14:59:08.834 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.869e-06 2024-09-21T14:59:08.834 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.835 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.835 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:08.835 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645244+0000", 2024-09-21T14:59:08.835 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8930000000000002e-06 2024-09-21T14:59:08.835 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.835 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.835 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:08.835 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645249+0000", 2024-09-21T14:59:08.835 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2859999999999999e-06 2024-09-21T14:59:08.835 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.835 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.835 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:08.835 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.653930+0000", 2024-09-21T14:59:08.836 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086815320000000005 2024-09-21T14:59:08.836 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.836 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.836 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:08.836 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654028+0000", 2024-09-21T14:59:08.836 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7578000000000007e-05 2024-09-21T14:59:08.836 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.836 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.836 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:08.836 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654219+0000", 2024-09-21T14:59:08.836 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019132099999999999 2024-09-21T14:59:08.836 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:08.836 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:08.837 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:08.837 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.837 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.837 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12538 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 147456~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:08.837 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645253+0000", 2024-09-21T14:59:08.837 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 29.111595005000002, 2024-09-21T14:59:08.837 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0092095360000000008, 2024-09-21T14:59:08.837 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:08.837 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:08.837 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:08.837 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:08.837 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:08.837 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12538 2024-09-21T14:59:08.837 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.838 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:08.838 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.838 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:08.838 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645253+0000", 2024-09-21T14:59:08.838 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:08.838 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.838 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.838 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:08.838 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645253+0000", 2024-09-21T14:59:08.838 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:08.838 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.838 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.839 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:08.839 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645252+0000", 2024-09-21T14:59:08.839 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:08.839 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.839 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.839 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:08.839 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645255+0000", 2024-09-21T14:59:08.839 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2290000000000002e-06 2024-09-21T14:59:08.839 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.839 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.839 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:08.839 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645267+0000", 2024-09-21T14:59:08.839 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2631e-05 2024-09-21T14:59:08.839 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.840 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.840 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:08.840 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645278+0000", 2024-09-21T14:59:08.840 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0847e-05 2024-09-21T14:59:08.840 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.840 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.840 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:08.840 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654251+0000", 2024-09-21T14:59:08.840 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0089731340000000007 2024-09-21T14:59:08.840 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.840 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.840 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:08.840 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654296+0000", 2024-09-21T14:59:08.841 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4353e-05 2024-09-21T14:59:08.841 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.841 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.841 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:08.841 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654462+0000", 2024-09-21T14:59:08.841 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016668499999999999 2024-09-21T14:59:08.841 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:08.841 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:08.841 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:08.841 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.841 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.841 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12539 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 253952~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:08.841 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645284+0000", 2024-09-21T14:59:08.842 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 29.111563784000001, 2024-09-21T14:59:08.842 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0094524029999999998, 2024-09-21T14:59:08.842 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:08.842 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:08.842 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:08.842 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:08.842 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:08.842 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12539 2024-09-21T14:59:08.842 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.842 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:08.842 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.842 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:08.842 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645284+0000", 2024-09-21T14:59:08.843 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:08.843 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.843 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.843 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:08.843 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645284+0000", 2024-09-21T14:59:08.843 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:08.843 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.843 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.843 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:08.843 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645284+0000", 2024-09-21T14:59:08.843 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:08.843 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.844 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.844 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:08.844 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645286+0000", 2024-09-21T14:59:08.844 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7159999999999999e-06 2024-09-21T14:59:08.844 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.844 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.844 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:08.844 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645292+0000", 2024-09-21T14:59:08.844 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7400000000000001e-06 2024-09-21T14:59:08.844 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.844 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.844 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:08.845 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645297+0000", 2024-09-21T14:59:08.845 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9019999999999998e-06 2024-09-21T14:59:08.845 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.845 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.845 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:08.845 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654490+0000", 2024-09-21T14:59:08.845 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0091932360000000005 2024-09-21T14:59:08.845 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.845 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.845 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:08.845 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654565+0000", 2024-09-21T14:59:08.845 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4657e-05 2024-09-21T14:59:08.846 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.846 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.846 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:08.846 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654736+0000", 2024-09-21T14:59:08.846 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017156199999999999 2024-09-21T14:59:08.846 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:08.846 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:08.846 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:08.846 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.846 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.846 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12540 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 286720~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:08.846 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645302+0000", 2024-09-21T14:59:08.847 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 29.111545515, 2024-09-21T14:59:08.847 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0096603090000000006, 2024-09-21T14:59:08.847 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:08.847 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:08.847 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:08.847 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:08.847 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:08.847 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12540 2024-09-21T14:59:08.847 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.847 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:08.847 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.848 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:08.848 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645302+0000", 2024-09-21T14:59:08.848 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:08.848 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.848 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.848 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:08.848 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645302+0000", 2024-09-21T14:59:08.848 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:08.848 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.848 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.848 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:08.849 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645302+0000", 2024-09-21T14:59:08.849 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:08.849 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.849 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.849 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:08.849 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645305+0000", 2024-09-21T14:59:08.849 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7329999999999999e-06 2024-09-21T14:59:08.849 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.849 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.849 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:08.849 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645316+0000", 2024-09-21T14:59:08.849 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1832e-05 2024-09-21T14:59:08.850 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.850 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.850 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:08.850 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645322+0000", 2024-09-21T14:59:08.850 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.169e-06 2024-09-21T14:59:08.850 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.850 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.850 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:08.850 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654778+0000", 2024-09-21T14:59:08.850 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0094559090000000002 2024-09-21T14:59:08.850 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.850 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.851 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:08.851 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654828+0000", 2024-09-21T14:59:08.851 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0461999999999999e-05 2024-09-21T14:59:08.851 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.851 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.851 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:08.851 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654963+0000", 2024-09-21T14:59:08.851 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000134583 2024-09-21T14:59:08.851 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:08.851 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:08.851 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:08.851 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.852 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.852 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12561 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:08.852 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645742+0000", 2024-09-21T14:59:08.852 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 29.111106248999999, 2024-09-21T14:59:08.852 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0093583810000000007, 2024-09-21T14:59:08.852 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:08.852 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:08.852 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:08.852 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:08.852 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:08.852 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12561 2024-09-21T14:59:08.853 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.853 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:08.853 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.853 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:08.853 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645742+0000", 2024-09-21T14:59:08.853 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:08.853 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.853 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.853 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:08.853 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645742+0000", 2024-09-21T14:59:08.853 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:08.853 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.854 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.854 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:08.854 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645741+0000", 2024-09-21T14:59:08.854 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:08.854 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.854 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.854 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:08.854 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645746+0000", 2024-09-21T14:59:08.854 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8379999999999998e-06 2024-09-21T14:59:08.854 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.854 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.855 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:08.855 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645757+0000", 2024-09-21T14:59:08.855 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0705e-05 2024-09-21T14:59:08.855 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.855 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.855 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:08.855 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645765+0000", 2024-09-21T14:59:08.855 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6729999999999992e-06 2024-09-21T14:59:08.855 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.855 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.855 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:08.855 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654993+0000", 2024-09-21T14:59:08.855 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0092287480000000002 2024-09-21T14:59:08.856 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.856 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.856 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:08.856 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.655050+0000", 2024-09-21T14:59:08.856 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6394999999999998e-05 2024-09-21T14:59:08.856 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.856 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.856 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:08.856 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.655100+0000", 2024-09-21T14:59:08.856 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0049000000000001e-05 2024-09-21T14:59:08.856 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:08.856 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:08.857 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:08.857 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.857 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.857 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:15431 3.12 3:4bc7d206:::rbd_data.1136f5ed5dd6.0000000000000051:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:59:08.857 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:41.673445+0000", 2024-09-21T14:59:08.857 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 27.083402915000001, 2024-09-21T14:59:08.857 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086103610000000004, 2024-09-21T14:59:08.857 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:08.857 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:08.857 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:08.857 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:08.857 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:08.857 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 15431 2024-09-21T14:59:08.858 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.858 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:08.858 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.858 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:08.858 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.673445+0000", 2024-09-21T14:59:08.858 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:08.858 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.858 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.858 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:08.858 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.673445+0000", 2024-09-21T14:59:08.858 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:08.858 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.859 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.859 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:08.859 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.673444+0000", 2024-09-21T14:59:08.859 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:59:08.859 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.859 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.859 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:08.859 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.673451+0000", 2024-09-21T14:59:08.859 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4739999999999999e-06 2024-09-21T14:59:08.859 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.859 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.859 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:08.860 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.673472+0000", 2024-09-21T14:59:08.860 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0687e-05 2024-09-21T14:59:08.860 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.860 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.860 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:08.860 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.673481+0000", 2024-09-21T14:59:08.860 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3349999999999996e-06 2024-09-21T14:59:08.860 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.860 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.860 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:08.860 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.673565+0000", 2024-09-21T14:59:08.860 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4011000000000006e-05 2024-09-21T14:59:08.860 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.861 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.861 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:08.861 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.673650+0000", 2024-09-21T14:59:08.861 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4949000000000003e-05 2024-09-21T14:59:08.861 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.861 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.861 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 3", 2024-09-21T14:59:08.861 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.675014+0000", 2024-09-21T14:59:08.861 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0013636620000000001 2024-09-21T14:59:08.861 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.861 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.861 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:08.861 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.681866+0000", 2024-09-21T14:59:08.861 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0068525640000000002 2024-09-21T14:59:08.862 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.862 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.862 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:59:08.862 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.681885+0000", 2024-09-21T14:59:08.862 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8519999999999999e-05 2024-09-21T14:59:08.862 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.862 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.862 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:59:08.862 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.681961+0000", 2024-09-21T14:59:08.862 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6248000000000006e-05 2024-09-21T14:59:08.862 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:08.862 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:08.863 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:08.863 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.682055+0000", 2024-09-21T14:59:08.863 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4184000000000005e-05 2024-09-21T14:59:08.863 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:08.863 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:08.863 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:08.863 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:08.863 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:08.863 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:59:08.865 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T14:59:08.865 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:59:08.865+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:59:08.965 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:59:08.965+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:59:08.977 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:59:08.977 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:59:08.977 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T14:59:08.977 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:59:08.986 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T14:59:09.066 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:59:09.064+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:59:09.094 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:59:09.094 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:59:09.094 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T14:59:09.094 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T14:59:09.095 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:59:09.104 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T14:59:09.167 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:59:09.165+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:59:09.262 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:59:09.262 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T14:59:09.262 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T14:59:09.262 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T14:59:09.262 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.262 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:8034 3.b 3:d1b12ef9:::rbd_data.11361bed1f78.0000000000000062:d [read 32768~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:59:09.262 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:35.227265+0000", 2024-09-21T14:59:09.262 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 34.031800961000002, 2024-09-21T14:59:09.262 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.006525968, 2024-09-21T14:59:09.262 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:09.262 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:09.263 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:09.263 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:09.263 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:09.263 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 8034 2024-09-21T14:59:09.263 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.263 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:09.263 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.263 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:09.263 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227265+0000", 2024-09-21T14:59:09.263 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:09.263 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.263 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.263 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:09.264 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227265+0000", 2024-09-21T14:59:09.264 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:09.264 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.264 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.264 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:09.264 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227265+0000", 2024-09-21T14:59:09.264 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:09.264 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.264 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.264 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:09.264 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227268+0000", 2024-09-21T14:59:09.264 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8990000000000001e-06 2024-09-21T14:59:09.264 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.265 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.265 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:09.265 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227274+0000", 2024-09-21T14:59:09.265 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4470000000000003e-06 2024-09-21T14:59:09.265 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.265 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.265 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:09.265 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227288+0000", 2024-09-21T14:59:09.265 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3550000000000001e-05 2024-09-21T14:59:09.265 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.265 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.265 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:09.265 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.233418+0000", 2024-09-21T14:59:09.266 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0061307870000000004 2024-09-21T14:59:09.266 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.266 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.266 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:09.266 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.233553+0000", 2024-09-21T14:59:09.266 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013475600000000001 2024-09-21T14:59:09.266 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.266 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.266 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:09.266 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.233791+0000", 2024-09-21T14:59:09.266 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00023788499999999999 2024-09-21T14:59:09.266 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:09.266 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:09.266 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:09.267 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.267 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.267 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:8035 3.b 3:d1b12ef9:::rbd_data.11361bed1f78.0000000000000062:d [read 98304~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:59:09.267 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:35.227295+0000", 2024-09-21T14:59:09.267 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 34.031770756, 2024-09-21T14:59:09.267 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0069673820000000003, 2024-09-21T14:59:09.267 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:09.267 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:09.267 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:09.267 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:09.267 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:09.267 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 8035 2024-09-21T14:59:09.267 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.267 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:09.267 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.267 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:09.268 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227295+0000", 2024-09-21T14:59:09.268 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:09.268 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.268 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.268 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:09.268 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227295+0000", 2024-09-21T14:59:09.268 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:09.268 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.268 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.268 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:09.268 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227295+0000", 2024-09-21T14:59:09.268 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:09.268 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.268 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.269 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:09.269 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227298+0000", 2024-09-21T14:59:09.269 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2339999999999999e-06 2024-09-21T14:59:09.269 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.269 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.269 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:09.269 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227305+0000", 2024-09-21T14:59:09.269 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0389999999999999e-06 2024-09-21T14:59:09.269 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.269 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.269 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:09.269 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227311+0000", 2024-09-21T14:59:09.269 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8470000000000001e-06 2024-09-21T14:59:09.269 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.269 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.270 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:09.270 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.233824+0000", 2024-09-21T14:59:09.270 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0065132920000000004 2024-09-21T14:59:09.270 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.270 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.270 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:09.270 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.233922+0000", 2024-09-21T14:59:09.270 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.8073999999999996e-05 2024-09-21T14:59:09.270 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.270 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.270 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:09.270 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.234263+0000", 2024-09-21T14:59:09.270 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00034027999999999999 2024-09-21T14:59:09.270 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:09.271 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:09.271 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:09.271 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.271 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.271 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:8048 3.b 3:d1b12ef9:::rbd_data.11361bed1f78.0000000000000062:d [read 49152~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:59:09.271 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:35.227316+0000", 2024-09-21T14:59:09.271 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 34.031749963999999, 2024-09-21T14:59:09.271 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0076197799999999996, 2024-09-21T14:59:09.271 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:09.271 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:09.271 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:09.271 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:09.271 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:09.271 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 8048 2024-09-21T14:59:09.271 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.272 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:09.272 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.272 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:09.272 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227316+0000", 2024-09-21T14:59:09.272 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:09.272 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.272 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.272 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:09.272 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227316+0000", 2024-09-21T14:59:09.272 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:09.272 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.272 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.272 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:09.272 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227316+0000", 2024-09-21T14:59:09.273 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:09.273 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.273 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.273 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:09.273 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227324+0000", 2024-09-21T14:59:09.273 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1009999999999992e-06 2024-09-21T14:59:09.273 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.273 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.273 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:09.273 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227346+0000", 2024-09-21T14:59:09.273 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2347000000000001e-05 2024-09-21T14:59:09.273 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.273 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.273 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:09.273 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227360+0000", 2024-09-21T14:59:09.274 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.403e-05 2024-09-21T14:59:09.274 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.274 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.274 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:09.274 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.234320+0000", 2024-09-21T14:59:09.274 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0069597019999999999 2024-09-21T14:59:09.274 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.274 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.274 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:09.274 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.234730+0000", 2024-09-21T14:59:09.274 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00041006300000000001 2024-09-21T14:59:09.274 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.274 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.275 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:09.275 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.234936+0000", 2024-09-21T14:59:09.275 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00020586700000000001 2024-09-21T14:59:09.275 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:09.275 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:09.275 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:09.275 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.275 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.275 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:8049 3.b 3:d1b12ef9:::rbd_data.11361bed1f78.0000000000000062:d [read 114688~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:59:09.275 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:35.227371+0000", 2024-09-21T14:59:09.275 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 34.031694631000001, 2024-09-21T14:59:09.275 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0078533609999999997, 2024-09-21T14:59:09.275 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:09.275 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:09.275 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:09.276 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:09.276 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:09.276 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 8049 2024-09-21T14:59:09.276 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.276 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:09.276 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.276 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:09.276 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227371+0000", 2024-09-21T14:59:09.276 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:09.276 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.276 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.276 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:09.276 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227371+0000", 2024-09-21T14:59:09.276 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:09.277 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.277 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.277 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:09.277 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227370+0000", 2024-09-21T14:59:09.277 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:59:09.277 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.277 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.277 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:09.277 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227378+0000", 2024-09-21T14:59:09.277 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.379e-06 2024-09-21T14:59:09.277 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.277 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.277 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:09.277 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227388+0000", 2024-09-21T14:59:09.278 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0175e-05 2024-09-21T14:59:09.278 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.278 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.278 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:09.278 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227393+0000", 2024-09-21T14:59:09.278 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6690000000000003e-06 2024-09-21T14:59:09.278 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.278 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.278 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:09.278 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.234964+0000", 2024-09-21T14:59:09.278 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0075705319999999996 2024-09-21T14:59:09.278 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.278 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.278 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:09.278 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.235041+0000", 2024-09-21T14:59:09.279 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7170999999999997e-05 2024-09-21T14:59:09.279 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.279 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.279 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:09.279 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.235225+0000", 2024-09-21T14:59:09.279 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000183735 2024-09-21T14:59:09.279 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:09.279 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:09.279 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:09.279 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.279 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.279 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:19869 3.b 3:d7ca7d68:::rbd_data.11363a21d2be.000000000000002d:head [sparse-read 0~393216] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T14:59:09.279 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:44.340896+0000", 2024-09-21T14:59:09.279 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 24.918170234000002, 2024-09-21T14:59:09.279 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0071749170000000003, 2024-09-21T14:59:09.280 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:09.280 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:09.280 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:09.280 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:09.280 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:09.280 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 19869 2024-09-21T14:59:09.280 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.280 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:09.280 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.280 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:09.280 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340896+0000", 2024-09-21T14:59:09.280 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:09.280 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.280 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.281 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:09.281 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340896+0000", 2024-09-21T14:59:09.281 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:09.281 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.281 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.281 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:09.281 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340894+0000", 2024-09-21T14:59:09.281 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:59:09.281 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.281 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.281 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:09.281 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340903+0000", 2024-09-21T14:59:09.281 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0259999999999996e-06 2024-09-21T14:59:09.281 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.281 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.282 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:09.282 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340923+0000", 2024-09-21T14:59:09.282 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0013999999999998e-05 2024-09-21T14:59:09.282 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.282 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.282 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:09.282 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340938+0000", 2024-09-21T14:59:09.282 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4569000000000001e-05 2024-09-21T14:59:09.282 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.282 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.282 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:09.282 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342092+0000", 2024-09-21T14:59:09.282 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00115354 2024-09-21T14:59:09.282 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.282 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.283 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:09.283 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342290+0000", 2024-09-21T14:59:09.283 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019834299999999999 2024-09-21T14:59:09.283 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.283 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.283 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:09.283 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.348071+0000", 2024-09-21T14:59:09.283 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.005780861 2024-09-21T14:59:09.283 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:09.283 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:09.283 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:09.283 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.283 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.283 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:19872 3.b 3:d38ac3f6:::rbd_data.11363a21d2be.0000000000000030:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T14:59:09.283 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:44.340951+0000", 2024-09-21T14:59:09.284 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 24.918115074999999, 2024-09-21T14:59:09.284 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0069228680000000004, 2024-09-21T14:59:09.284 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:09.284 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:09.284 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:09.284 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:09.284 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:09.284 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 19872 2024-09-21T14:59:09.284 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.284 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:09.284 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.284 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:09.284 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340951+0000", 2024-09-21T14:59:09.284 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:09.285 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.285 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.285 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:09.285 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340951+0000", 2024-09-21T14:59:09.285 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:09.285 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.285 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.285 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:09.285 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340950+0000", 2024-09-21T14:59:09.285 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:09.285 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.285 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.285 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:09.285 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340957+0000", 2024-09-21T14:59:09.285 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8430000000000004e-06 2024-09-21T14:59:09.286 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.286 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.286 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:09.286 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340974+0000", 2024-09-21T14:59:09.286 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7082999999999999e-05 2024-09-21T14:59:09.286 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.286 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.286 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:09.286 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340987+0000", 2024-09-21T14:59:09.286 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2763e-05 2024-09-21T14:59:09.286 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.286 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.286 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:09.286 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.347149+0000", 2024-09-21T14:59:09.286 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0061626600000000004 2024-09-21T14:59:09.287 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.287 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.287 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:09.287 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.347354+0000", 2024-09-21T14:59:09.287 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00020441799999999999 2024-09-21T14:59:09.287 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.287 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.287 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:09.287 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.347874+0000", 2024-09-21T14:59:09.287 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00052019399999999997 2024-09-21T14:59:09.287 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:09.287 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:09.287 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:09.287 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.288 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.288 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:19877 3.b 3:d5fc5a5a:::rbd_data.11363a21d2be.0000000000000035:head [sparse-read 0~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T14:59:09.288 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:44.340998+0000", 2024-09-21T14:59:09.288 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 24.918068239, 2024-09-21T14:59:09.288 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0073322129999999998, 2024-09-21T14:59:09.288 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:09.288 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:09.288 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:09.288 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:09.288 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:09.288 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 19877 2024-09-21T14:59:09.288 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.288 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:09.288 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.288 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:09.289 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340998+0000", 2024-09-21T14:59:09.289 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:09.289 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.289 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.289 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:09.289 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340998+0000", 2024-09-21T14:59:09.289 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:09.289 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.289 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.289 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:09.289 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340997+0000", 2024-09-21T14:59:09.289 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:09.289 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.289 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.289 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:09.290 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.341003+0000", 2024-09-21T14:59:09.290 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0070000000000003e-06 2024-09-21T14:59:09.290 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.290 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.290 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:09.290 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.341018+0000", 2024-09-21T14:59:09.290 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5472999999999999e-05 2024-09-21T14:59:09.290 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.290 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.290 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:09.290 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.341040+0000", 2024-09-21T14:59:09.290 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1568000000000001e-05 2024-09-21T14:59:09.290 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.290 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.290 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:09.291 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.347901+0000", 2024-09-21T14:59:09.291 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0068608300000000001 2024-09-21T14:59:09.291 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.291 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.291 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:09.291 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.347973+0000", 2024-09-21T14:59:09.291 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2327000000000001e-05 2024-09-21T14:59:09.291 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.291 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.291 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:09.291 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.348330+0000", 2024-09-21T14:59:09.291 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00035695799999999999 2024-09-21T14:59:09.291 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:09.291 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:09.292 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:09.292 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.292 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.292 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:19941 3.b 3:d7b21c30:::rbd_data.11363a21d2be.0000000000000075:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T14:59:09.292 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:44.342141+0000", 2024-09-21T14:59:09.292 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 24.916924672, 2024-09-21T14:59:09.292 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0063087439999999998, 2024-09-21T14:59:09.292 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:09.292 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:09.292 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:09.292 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:09.292 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:09.292 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 19941 2024-09-21T14:59:09.292 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.292 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:09.292 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.293 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:09.293 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342141+0000", 2024-09-21T14:59:09.293 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:09.293 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.293 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.293 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:09.293 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342141+0000", 2024-09-21T14:59:09.293 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:09.293 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.293 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.293 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:09.293 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342139+0000", 2024-09-21T14:59:09.293 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T14:59:09.293 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.293 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.294 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:09.294 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342151+0000", 2024-09-21T14:59:09.294 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1242e-05 2024-09-21T14:59:09.294 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.294 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.294 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:09.294 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342175+0000", 2024-09-21T14:59:09.294 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4491e-05 2024-09-21T14:59:09.294 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.294 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.294 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:09.294 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342191+0000", 2024-09-21T14:59:09.294 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6288000000000001e-05 2024-09-21T14:59:09.294 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.295 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.295 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:09.295 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.348358+0000", 2024-09-21T14:59:09.295 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0061665449999999998 2024-09-21T14:59:09.295 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.295 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.295 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:09.295 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.348450+0000", 2024-09-21T14:59:09.295 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2245999999999997e-05 2024-09-21T14:59:09.295 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:09.295 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:09.295 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:09.295 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.295 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.295 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:19998 3.b 3:d4c6ec57:::rbd_data.11363a21d2be.00000000000000ae:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T14:59:09.296 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:44.342204+0000", 2024-09-21T14:59:09.296 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 24.916862468000001, 2024-09-21T14:59:09.296 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0063531860000000002, 2024-09-21T14:59:09.296 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:09.296 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:09.296 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:09.296 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:09.296 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:09.296 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 19998 2024-09-21T14:59:09.296 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.296 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:09.296 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.296 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:09.296 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342204+0000", 2024-09-21T14:59:09.297 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:09.297 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.297 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.297 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:09.297 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342204+0000", 2024-09-21T14:59:09.297 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:09.297 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.297 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.297 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:09.297 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342203+0000", 2024-09-21T14:59:09.297 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:09.297 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.297 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.297 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:09.297 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342209+0000", 2024-09-21T14:59:09.298 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1739999999999997e-06 2024-09-21T14:59:09.298 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.298 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.298 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:09.298 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342223+0000", 2024-09-21T14:59:09.298 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4065e-05 2024-09-21T14:59:09.298 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.298 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.298 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:09.298 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342236+0000", 2024-09-21T14:59:09.298 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2537e-05 2024-09-21T14:59:09.298 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.298 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.298 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:09.298 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.348476+0000", 2024-09-21T14:59:09.299 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0062405760000000003 2024-09-21T14:59:09.299 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.299 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.299 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:09.299 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.348557+0000", 2024-09-21T14:59:09.299 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0716000000000004e-05 2024-09-21T14:59:09.299 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:09.299 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:09.299 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:09.299 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.299 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.299 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:19999 3.b 3:d4069488:::rbd_data.11363a21d2be.00000000000000af:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T14:59:09.299 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:44.342246+0000", 2024-09-21T14:59:09.299 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 24.916820303000002, 2024-09-21T14:59:09.299 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0064139210000000004, 2024-09-21T14:59:09.299 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:09.300 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:09.300 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:09.300 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:09.300 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:09.300 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 19999 2024-09-21T14:59:09.300 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.300 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:09.300 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.300 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:09.300 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342246+0000", 2024-09-21T14:59:09.300 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:09.300 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.300 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.300 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:09.300 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342246+0000", 2024-09-21T14:59:09.300 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:09.301 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.301 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.301 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:09.301 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342245+0000", 2024-09-21T14:59:09.301 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:09.301 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.301 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.301 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:09.301 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342251+0000", 2024-09-21T14:59:09.301 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8660000000000002e-06 2024-09-21T14:59:09.301 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.301 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.301 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:09.301 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342264+0000", 2024-09-21T14:59:09.301 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3716999999999999e-05 2024-09-21T14:59:09.302 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.302 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.302 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:09.302 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342275+0000", 2024-09-21T14:59:09.302 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1071e-05 2024-09-21T14:59:09.302 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.302 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.302 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:09.302 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.348581+0000", 2024-09-21T14:59:09.302 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0063053730000000004 2024-09-21T14:59:09.302 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.302 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.302 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:09.302 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.348660+0000", 2024-09-21T14:59:09.302 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8854000000000001e-05 2024-09-21T14:59:09.302 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:09.303 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:09.303 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:09.303 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.303 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.303 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:20440 3.b 3:d6a6f68e:::rbd_object_map.1136c84d9a13:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-21T14:59:09.303 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:45.106428+0000", 2024-09-21T14:59:09.303 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 24.152638456999998, 2024-09-21T14:59:09.303 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0076349679999999998, 2024-09-21T14:59:09.303 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:09.303 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:09.303 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:09.303 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:09.303 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:09.303 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 20440 2024-09-21T14:59:09.303 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.304 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:09.304 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.304 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:09.304 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.106428+0000", 2024-09-21T14:59:09.304 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:09.304 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.304 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.304 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:09.304 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.106428+0000", 2024-09-21T14:59:09.304 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:09.304 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.304 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.304 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:09.304 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.106425+0000", 2024-09-21T14:59:09.305 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-21T14:59:09.305 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.305 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.305 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:09.305 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.106442+0000", 2024-09-21T14:59:09.305 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7237000000000001e-05 2024-09-21T14:59:09.305 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.305 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.305 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:09.305 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.106476+0000", 2024-09-21T14:59:09.305 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4004999999999997e-05 2024-09-21T14:59:09.305 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.305 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.305 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:09.305 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.106491+0000", 2024-09-21T14:59:09.306 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4865000000000001e-05 2024-09-21T14:59:09.306 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.306 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.306 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:09.306 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.106602+0000", 2024-09-21T14:59:09.306 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011113200000000001 2024-09-21T14:59:09.306 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.306 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.306 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op must wait for map", 2024-09-21T14:59:09.306 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.106611+0000", 2024-09-21T14:59:09.306 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9199999999999993e-06 2024-09-21T14:59:09.306 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.306 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.306 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:09.306 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.110475+0000", 2024-09-21T14:59:09.307 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0038633259999999998 2024-09-21T14:59:09.307 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.307 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.307 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:09.307 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.110622+0000", 2024-09-21T14:59:09.307 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014734200000000001 2024-09-21T14:59:09.307 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.307 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.307 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T14:59:09.307 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.111136+0000", 2024-09-21T14:59:09.307 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00051382299999999995 2024-09-21T14:59:09.307 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.307 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.307 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:59:09.308 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.112514+0000", 2024-09-21T14:59:09.308 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001378287 2024-09-21T14:59:09.308 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.308 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.308 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:09.308 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.113830+0000", 2024-09-21T14:59:09.308 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0013157939999999999 2024-09-21T14:59:09.308 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.308 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.308 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:59:09.308 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.113969+0000", 2024-09-21T14:59:09.308 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000138849 2024-09-21T14:59:09.308 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.308 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.309 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:09.309 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.114063+0000", 2024-09-21T14:59:09.309 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3999999999999994e-05 2024-09-21T14:59:09.309 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:09.309 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:09.309 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:09.309 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.309 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.309 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:20460 3.b 3:d6a6f68e:::rbd_object_map.1136c84d9a13:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-21T14:59:09.309 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:45.150183+0000", 2024-09-21T14:59:09.309 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 24.108882639000001, 2024-09-21T14:59:09.309 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0063818800000000004, 2024-09-21T14:59:09.309 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:09.309 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:09.309 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:09.310 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:09.310 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:09.310 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 20460 2024-09-21T14:59:09.310 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.310 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:09.310 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.310 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:09.310 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150183+0000", 2024-09-21T14:59:09.310 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:09.310 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.310 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.310 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:09.310 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150183+0000", 2024-09-21T14:59:09.311 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:09.311 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.311 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.311 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:09.311 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150180+0000", 2024-09-21T14:59:09.311 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999971 2024-09-21T14:59:09.311 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.311 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.311 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:09.311 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150211+0000", 2024-09-21T14:59:09.311 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0993e-05 2024-09-21T14:59:09.311 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.311 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.311 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:09.312 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150241+0000", 2024-09-21T14:59:09.312 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9589000000000001e-05 2024-09-21T14:59:09.312 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.312 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.312 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:09.312 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150261+0000", 2024-09-21T14:59:09.312 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.967e-05 2024-09-21T14:59:09.312 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.312 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.312 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:09.312 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150411+0000", 2024-09-21T14:59:09.312 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015004300000000001 2024-09-21T14:59:09.312 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.312 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.313 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:09.313 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150598+0000", 2024-09-21T14:59:09.313 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018712399999999999 2024-09-21T14:59:09.313 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.313 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.313 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T14:59:09.313 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.151390+0000", 2024-09-21T14:59:09.313 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00079194599999999997 2024-09-21T14:59:09.313 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.313 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.313 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:09.313 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.156302+0000", 2024-09-21T14:59:09.313 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0049123960000000003 2024-09-21T14:59:09.313 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.314 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.314 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:59:09.314 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.156435+0000", 2024-09-21T14:59:09.314 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013287300000000001 2024-09-21T14:59:09.314 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.314 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.314 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:59:09.314 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.156508+0000", 2024-09-21T14:59:09.314 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2504999999999998e-05 2024-09-21T14:59:09.314 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.314 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.314 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:09.315 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.156565+0000", 2024-09-21T14:59:09.315 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7773999999999999e-05 2024-09-21T14:59:09.315 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:09.315 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:09.315 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:09.315 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.315 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.315 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:20461 3.b 3:d6a6f68e:::rbd_object_map.1136c84d9a13:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-21T14:59:09.315 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:45.150284+0000", 2024-09-21T14:59:09.315 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 24.108781818000001, 2024-09-21T14:59:09.315 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0064029040000000001, 2024-09-21T14:59:09.315 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:09.315 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:09.315 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:09.315 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:09.316 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:09.316 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 20461 2024-09-21T14:59:09.316 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.316 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:09.316 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.316 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:09.316 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150284+0000", 2024-09-21T14:59:09.316 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:09.316 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.316 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.316 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:09.316 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150284+0000", 2024-09-21T14:59:09.316 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:09.316 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.317 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.317 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:09.317 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150283+0000", 2024-09-21T14:59:09.317 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:09.317 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.317 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.317 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:09.317 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150289+0000", 2024-09-21T14:59:09.317 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8799999999999996e-06 2024-09-21T14:59:09.317 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.317 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.317 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:09.317 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150319+0000", 2024-09-21T14:59:09.317 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9828999999999999e-05 2024-09-21T14:59:09.317 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.317 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.318 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:09.318 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150328+0000", 2024-09-21T14:59:09.318 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4770000000000003e-06 2024-09-21T14:59:09.318 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.318 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.318 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:09.318 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.152355+0000", 2024-09-21T14:59:09.318 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0020279870000000002 2024-09-21T14:59:09.318 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.318 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.318 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:09.318 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.152652+0000", 2024-09-21T14:59:09.318 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00029681 2024-09-21T14:59:09.318 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.319 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.319 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T14:59:09.319 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.153318+0000", 2024-09-21T14:59:09.319 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000666101 2024-09-21T14:59:09.319 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.319 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.319 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:09.319 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.156416+0000", 2024-09-21T14:59:09.319 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0030980729999999998 2024-09-21T14:59:09.319 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.319 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.319 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:59:09.319 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.156577+0000", 2024-09-21T14:59:09.319 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016049700000000001 2024-09-21T14:59:09.320 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.320 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.320 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:59:09.320 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.156628+0000", 2024-09-21T14:59:09.320 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0642000000000001e-05 2024-09-21T14:59:09.320 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.320 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.320 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:09.320 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.156687+0000", 2024-09-21T14:59:09.320 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9571000000000003e-05 2024-09-21T14:59:09.320 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:09.320 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:09.320 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:09.320 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.320 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.321 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:20462 3.b 3:d6a6f68e:::rbd_object_map.1136c84d9a13:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-21T14:59:09.321 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:45.150372+0000", 2024-09-21T14:59:09.321 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 24.108693735999999, 2024-09-21T14:59:09.321 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0075518110000000003, 2024-09-21T14:59:09.321 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:09.321 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:09.321 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:09.321 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:09.321 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:09.321 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 20462 2024-09-21T14:59:09.321 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.321 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:09.321 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.321 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:09.322 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150372+0000", 2024-09-21T14:59:09.322 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:09.322 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.322 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.322 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:09.322 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150372+0000", 2024-09-21T14:59:09.322 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:09.322 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.322 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.322 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:09.322 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150372+0000", 2024-09-21T14:59:09.322 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:09.322 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.322 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.323 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:09.323 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150378+0000", 2024-09-21T14:59:09.323 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3330000000000002e-06 2024-09-21T14:59:09.323 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.323 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.323 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:09.323 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150413+0000", 2024-09-21T14:59:09.323 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4798999999999997e-05 2024-09-21T14:59:09.323 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.323 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.323 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:09.323 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150423+0000", 2024-09-21T14:59:09.323 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.8099999999999992e-06 2024-09-21T14:59:09.323 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.324 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.324 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:09.324 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.154550+0000", 2024-09-21T14:59:09.324 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0041275959999999999 2024-09-21T14:59:09.324 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.324 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.324 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:09.324 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.154758+0000", 2024-09-21T14:59:09.324 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00020815499999999999 2024-09-21T14:59:09.324 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.324 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.324 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T14:59:09.324 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.155323+0000", 2024-09-21T14:59:09.325 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00056425400000000004 2024-09-21T14:59:09.325 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.325 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.325 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:59:09.325 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.156702+0000", 2024-09-21T14:59:09.325 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001379676 2024-09-21T14:59:09.325 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.325 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.325 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:09.325 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.157662+0000", 2024-09-21T14:59:09.325 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00095948099999999998 2024-09-21T14:59:09.325 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.325 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.325 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:59:09.325 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.157816+0000", 2024-09-21T14:59:09.326 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000154666 2024-09-21T14:59:09.326 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.326 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.326 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:09.326 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.157924+0000", 2024-09-21T14:59:09.326 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000107798 2024-09-21T14:59:09.326 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:09.326 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:09.326 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:09.326 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.326 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.326 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31476 3.c 3:35718b6b:::rbd_data.11367721c19.0000000000000014:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T14:59:09.326 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.505975+0000", 2024-09-21T14:59:09.326 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 10.753091099000001, 2024-09-21T14:59:09.327 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99515995899999998, 2024-09-21T14:59:09.327 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:09.327 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:09.327 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:09.327 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:09.327 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:09.327 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31476 2024-09-21T14:59:09.327 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.327 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:09.327 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.327 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:09.327 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.505975+0000", 2024-09-21T14:59:09.327 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:09.328 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.328 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.328 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:09.328 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.505975+0000", 2024-09-21T14:59:09.328 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:09.328 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.328 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.328 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:09.328 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.505971+0000", 2024-09-21T14:59:09.328 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999967 2024-09-21T14:59:09.328 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.328 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.328 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:09.328 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.505989+0000", 2024-09-21T14:59:09.328 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7985000000000001e-05 2024-09-21T14:59:09.329 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.329 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.329 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:09.329 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506050+0000", 2024-09-21T14:59:09.329 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0659000000000003e-05 2024-09-21T14:59:09.329 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.329 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.329 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:09.329 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506074+0000", 2024-09-21T14:59:09.329 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4156e-05 2024-09-21T14:59:09.329 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.329 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.329 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:09.329 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506225+0000", 2024-09-21T14:59:09.329 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015097 2024-09-21T14:59:09.330 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.330 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.330 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T14:59:09.330 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506256+0000", 2024-09-21T14:59:09.330 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0505999999999999e-05 2024-09-21T14:59:09.330 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.330 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.330 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:09.330 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.497974+0000", 2024-09-21T14:59:09.330 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99171825700000005 2024-09-21T14:59:09.330 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.330 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.330 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for missing object", 2024-09-21T14:59:09.331 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.498449+0000", 2024-09-21T14:59:09.331 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00047506399999999998 2024-09-21T14:59:09.331 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.331 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.331 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:09.331 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.501000+0000", 2024-09-21T14:59:09.331 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0025506919999999998 2024-09-21T14:59:09.331 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.331 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.331 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:09.331 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.501135+0000", 2024-09-21T14:59:09.331 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013524000000000001 2024-09-21T14:59:09.331 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:09.331 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:09.332 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:09.332 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.332 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.332 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31480 3.c 3:36ccd808:::rbd_data.11367721c19.0000000000000018:head [sparse-read 0~14821] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T14:59:09.332 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.506116+0000", 2024-09-21T14:59:09.332 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 10.752949756, 2024-09-21T14:59:09.332 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99316033599999998, 2024-09-21T14:59:09.332 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:09.332 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:09.332 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:09.332 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:09.332 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:09.332 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31480 2024-09-21T14:59:09.332 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.333 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:09.333 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.333 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:09.333 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506116+0000", 2024-09-21T14:59:09.333 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:09.333 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.333 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.333 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:09.333 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506116+0000", 2024-09-21T14:59:09.333 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:09.333 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.333 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.333 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:09.333 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506115+0000", 2024-09-21T14:59:09.334 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:59:09.334 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.334 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.334 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:09.334 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506124+0000", 2024-09-21T14:59:09.334 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0659999999999993e-06 2024-09-21T14:59:09.334 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.334 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.334 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:09.334 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506145+0000", 2024-09-21T14:59:09.334 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1447e-05 2024-09-21T14:59:09.334 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.334 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.334 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:09.335 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506161+0000", 2024-09-21T14:59:09.335 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5209e-05 2024-09-21T14:59:09.335 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.335 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.335 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:09.335 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506337+0000", 2024-09-21T14:59:09.335 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017642999999999999 2024-09-21T14:59:09.335 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.335 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.335 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T14:59:09.335 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506375+0000", 2024-09-21T14:59:09.335 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7536000000000001e-05 2024-09-21T14:59:09.335 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.336 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.336 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:09.336 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.498815+0000", 2024-09-21T14:59:09.336 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99244067999999996 2024-09-21T14:59:09.336 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.336 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.336 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:09.336 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.498950+0000", 2024-09-21T14:59:09.336 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013436700000000001 2024-09-21T14:59:09.336 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.336 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.336 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:09.336 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499277+0000", 2024-09-21T14:59:09.336 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00032696800000000002 2024-09-21T14:59:09.337 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:09.337 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:09.337 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:09.337 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.337 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.337 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31513 3.c 3:36de8523:::rbd_data.11367721c19.0000000000000039:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T14:59:09.337 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.509071+0000", 2024-09-21T14:59:09.337 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 10.749995556, 2024-09-21T14:59:09.337 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99038225099999999, 2024-09-21T14:59:09.337 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:09.337 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:09.337 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:09.338 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:09.338 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:09.338 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31513 2024-09-21T14:59:09.338 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.338 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:09.338 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.338 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:09.338 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509071+0000", 2024-09-21T14:59:09.338 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:09.338 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.338 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.338 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:09.338 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509071+0000", 2024-09-21T14:59:09.339 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:09.339 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.339 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.339 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:09.339 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509068+0000", 2024-09-21T14:59:09.339 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-21T14:59:09.339 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.339 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.339 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:09.339 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509082+0000", 2024-09-21T14:59:09.339 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3545e-05 2024-09-21T14:59:09.339 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.339 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.339 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:09.340 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509116+0000", 2024-09-21T14:59:09.340 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3942000000000001e-05 2024-09-21T14:59:09.340 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.340 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.340 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:09.340 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509134+0000", 2024-09-21T14:59:09.340 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8839999999999999e-05 2024-09-21T14:59:09.340 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.340 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.340 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:09.340 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509318+0000", 2024-09-21T14:59:09.340 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018326799999999999 2024-09-21T14:59:09.340 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.340 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.341 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T14:59:09.341 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509346+0000", 2024-09-21T14:59:09.341 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8019000000000001e-05 2024-09-21T14:59:09.341 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.341 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.341 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:09.341 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499330+0000", 2024-09-21T14:59:09.341 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.98998399299999995 2024-09-21T14:59:09.341 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.341 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.341 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:09.341 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499453+0000", 2024-09-21T14:59:09.341 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000123115 2024-09-21T14:59:09.341 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:09.342 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:09.342 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:09.342 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.342 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.342 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31515 3.c 3:35b36ddd:::rbd_data.11367721c19.000000000000003b:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T14:59:09.342 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.509177+0000", 2024-09-21T14:59:09.342 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 10.74988913, 2024-09-21T14:59:09.342 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99044870600000001, 2024-09-21T14:59:09.342 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:09.342 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:09.342 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:09.342 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:09.342 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:09.342 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31515 2024-09-21T14:59:09.343 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.343 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:09.343 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.343 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:09.343 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509177+0000", 2024-09-21T14:59:09.343 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:09.343 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.343 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.343 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:09.343 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509177+0000", 2024-09-21T14:59:09.343 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:09.343 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.343 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.343 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:09.344 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509176+0000", 2024-09-21T14:59:09.344 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:09.344 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.344 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.344 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:09.344 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509183+0000", 2024-09-21T14:59:09.344 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6720000000000004e-06 2024-09-21T14:59:09.344 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.344 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.344 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:09.344 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509269+0000", 2024-09-21T14:59:09.344 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5881e-05 2024-09-21T14:59:09.344 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.345 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.345 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:09.345 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509303+0000", 2024-09-21T14:59:09.345 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3367000000000001e-05 2024-09-21T14:59:09.345 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.345 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.345 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:09.345 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509425+0000", 2024-09-21T14:59:09.345 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000122199 2024-09-21T14:59:09.345 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.345 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.345 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T14:59:09.345 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509448+0000", 2024-09-21T14:59:09.345 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2688999999999999e-05 2024-09-21T14:59:09.345 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.346 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.346 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:09.346 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499501+0000", 2024-09-21T14:59:09.346 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99005306999999998 2024-09-21T14:59:09.346 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.346 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.346 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:09.346 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499626+0000", 2024-09-21T14:59:09.346 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012501400000000001 2024-09-21T14:59:09.346 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:09.346 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:09.346 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:09.346 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.346 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.347 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31531 3.c 3:314f0287:::rbd_data.11367721c19.000000000000004b:head [sparse-read 0~7106] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T14:59:09.347 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.509329+0000", 2024-09-21T14:59:09.347 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 10.74973711, 2024-09-21T14:59:09.347 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99103085300000004, 2024-09-21T14:59:09.347 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:09.347 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:09.347 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:09.347 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:09.347 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:09.347 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31531 2024-09-21T14:59:09.347 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.347 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:09.347 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.347 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:09.348 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509329+0000", 2024-09-21T14:59:09.348 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:09.348 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.348 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.348 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:09.348 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509329+0000", 2024-09-21T14:59:09.348 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:09.348 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.348 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.348 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:09.348 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509328+0000", 2024-09-21T14:59:09.348 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:59:09.348 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.348 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.349 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:09.349 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509336+0000", 2024-09-21T14:59:09.349 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3170000000000002e-06 2024-09-21T14:59:09.349 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.349 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.349 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:09.349 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509353+0000", 2024-09-21T14:59:09.349 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7214000000000002e-05 2024-09-21T14:59:09.349 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.349 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.349 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:09.349 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509366+0000", 2024-09-21T14:59:09.349 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.26e-05 2024-09-21T14:59:09.350 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.350 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.350 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:09.350 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509523+0000", 2024-09-21T14:59:09.350 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015708000000000001 2024-09-21T14:59:09.350 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.350 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.350 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T14:59:09.350 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509544+0000", 2024-09-21T14:59:09.350 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0783000000000001e-05 2024-09-21T14:59:09.350 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.350 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.350 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:09.350 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499727+0000", 2024-09-21T14:59:09.351 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99018345200000002 2024-09-21T14:59:09.351 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.351 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.351 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:09.351 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499949+0000", 2024-09-21T14:59:09.351 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000221358 2024-09-21T14:59:09.351 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.351 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.351 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:09.351 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.500360+0000", 2024-09-21T14:59:09.351 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00041130700000000002 2024-09-21T14:59:09.351 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:09.351 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:09.351 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:09.351 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.352 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.352 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31605 3.c 3:34f43f7f:::rbd_data.113661d6691d.0000000000000034:head [delete] snapc 20=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-21T14:59:09.352 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:59.850318+0000", 2024-09-21T14:59:09.352 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.4087483479999996, 2024-09-21T14:59:09.352 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0069284979999999999, 2024-09-21T14:59:09.352 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:09.352 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:09.352 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:09.352 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:09.352 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:09.352 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31605 2024-09-21T14:59:09.352 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.352 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:09.353 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.353 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:09.353 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.850318+0000", 2024-09-21T14:59:09.353 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:09.353 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.353 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.353 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:09.353 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.850318+0000", 2024-09-21T14:59:09.353 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:09.353 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.353 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.353 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:09.353 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.850314+0000", 2024-09-21T14:59:09.353 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999962 2024-09-21T14:59:09.353 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.354 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.354 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:09.354 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.850332+0000", 2024-09-21T14:59:09.354 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8414e-05 2024-09-21T14:59:09.354 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.354 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.354 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:09.354 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.850376+0000", 2024-09-21T14:59:09.354 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3989000000000002e-05 2024-09-21T14:59:09.354 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.354 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.354 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:09.354 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.850399+0000", 2024-09-21T14:59:09.355 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2455000000000001e-05 2024-09-21T14:59:09.355 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.355 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.355 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:09.355 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.850537+0000", 2024-09-21T14:59:09.355 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013858100000000001 2024-09-21T14:59:09.355 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.355 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.355 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:09.355 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.850756+0000", 2024-09-21T14:59:09.355 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021833099999999999 2024-09-21T14:59:09.355 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.355 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.355 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T14:59:09.356 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.850939+0000", 2024-09-21T14:59:09.356 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018285600000000001 2024-09-21T14:59:09.356 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.356 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.356 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:59:09.356 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.852297+0000", 2024-09-21T14:59:09.356 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0013588599999999999 2024-09-21T14:59:09.356 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.356 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.356 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:09.356 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.857067+0000", 2024-09-21T14:59:09.356 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0047691349999999999 2024-09-21T14:59:09.356 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.356 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.357 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:59:09.357 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.857143+0000", 2024-09-21T14:59:09.357 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6983000000000005e-05 2024-09-21T14:59:09.357 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:09.357 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:09.357 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:09.357 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.857246+0000", 2024-09-21T14:59:09.357 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010275799999999999 2024-09-21T14:59:09.357 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:09.357 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:09.357 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:09.357 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:09.357 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:09.357 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:59:09.359 INFO:teuthology.orchestra.run.smithi032.stderr:reweighted osd.2 to 0.351469 (59f9) 2024-09-21T14:59:09.359 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:59:09.356+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:59:09.360 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T14:59:09.459 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:59:09.457+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:59:09.505 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:09.505 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:59:09.505 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T14:59:09.505 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:09.515 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T14:59:09.561 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:59:09.557+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:59:09.623 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:09.623 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:59:09.624 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T14:59:09.624 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T14:59:09.624 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:09.633 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T14:59:09.677 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:59:09.677+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:59:09.761 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:59:09.761+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:59:09.794 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:09.794 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T14:59:09.794 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T14:59:09.794 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T14:59:09.794 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.794 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:1473 3.15 3:af2afa6a:::rbd_data.1136515ecb6b.000000000000001b:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T14:59:09.794 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:30.421730+0000", 2024-09-21T14:59:09.794 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 39.369930003999997, 2024-09-21T14:59:09.794 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021478937, 2024-09-21T14:59:09.794 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:09.794 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:09.795 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:09.795 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:09.795 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:09.795 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1473 2024-09-21T14:59:09.795 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.795 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:09.795 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.795 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:09.795 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421730+0000", 2024-09-21T14:59:09.795 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:09.795 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.795 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.795 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:09.795 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421730+0000", 2024-09-21T14:59:09.795 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:09.796 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.796 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.796 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:09.796 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421728+0000", 2024-09-21T14:59:09.796 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-21T14:59:09.796 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.796 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.796 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:09.796 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421739+0000", 2024-09-21T14:59:09.796 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0502e-05 2024-09-21T14:59:09.796 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.796 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.796 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:09.797 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421762+0000", 2024-09-21T14:59:09.797 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3405999999999999e-05 2024-09-21T14:59:09.797 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.797 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.797 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:09.797 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421773+0000", 2024-09-21T14:59:09.797 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0744000000000001e-05 2024-09-21T14:59:09.797 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.797 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.797 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:09.797 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421838+0000", 2024-09-21T14:59:09.797 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5486000000000003e-05 2024-09-21T14:59:09.797 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.797 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.797 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:09.798 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421955+0000", 2024-09-21T14:59:09.798 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000116765 2024-09-21T14:59:09.798 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.798 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.798 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 6", 2024-09-21T14:59:09.798 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.425572+0000", 2024-09-21T14:59:09.798 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0036171380000000002 2024-09-21T14:59:09.798 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.798 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.798 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:09.798 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.442973+0000", 2024-09-21T14:59:09.798 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.017400268 2024-09-21T14:59:09.798 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.798 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.798 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:09.799 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.442996+0000", 2024-09-21T14:59:09.799 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3139e-05 2024-09-21T14:59:09.799 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.799 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.799 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:09.799 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.443079+0000", 2024-09-21T14:59:09.799 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2832999999999998e-05 2024-09-21T14:59:09.799 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.799 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.799 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:09.799 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.443209+0000", 2024-09-21T14:59:09.799 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013052499999999999 2024-09-21T14:59:09.799 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:09.800 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:09.800 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:09.800 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.800 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.800 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:1678 3.c 3:377cc7e1:::rbd_data.1136515ecb6b.000000000000000f:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T14:59:09.800 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:30.988085+0000", 2024-09-21T14:59:09.800 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 38.803575410000001, 2024-09-21T14:59:09.800 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.032974802999999997, 2024-09-21T14:59:09.800 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:09.800 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:09.800 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:09.800 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:09.800 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:09.800 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1678 2024-09-21T14:59:09.801 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.801 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:09.801 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.801 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:09.801 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988085+0000", 2024-09-21T14:59:09.801 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:09.801 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.801 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.801 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:09.801 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988085+0000", 2024-09-21T14:59:09.801 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:09.801 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.801 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.801 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:09.802 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988083+0000", 2024-09-21T14:59:09.802 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-21T14:59:09.802 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.802 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.802 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:09.802 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988093+0000", 2024-09-21T14:59:09.802 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8800000000000003e-06 2024-09-21T14:59:09.802 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.802 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.802 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:09.802 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988113+0000", 2024-09-21T14:59:09.802 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0217999999999999e-05 2024-09-21T14:59:09.802 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.802 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.803 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:09.803 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988123+0000", 2024-09-21T14:59:09.803 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5859999999999994e-06 2024-09-21T14:59:09.803 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.803 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.803 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:09.803 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988203+0000", 2024-09-21T14:59:09.803 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0834000000000006e-05 2024-09-21T14:59:09.803 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.803 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.803 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:09.803 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988376+0000", 2024-09-21T14:59:09.803 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017301299999999999 2024-09-21T14:59:09.803 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.804 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.804 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 7", 2024-09-21T14:59:09.804 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.996338+0000", 2024-09-21T14:59:09.804 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0079618520000000002 2024-09-21T14:59:09.804 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.804 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.804 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:09.804 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.020881+0000", 2024-09-21T14:59:09.804 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024542731000000002 2024-09-21T14:59:09.804 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.804 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.804 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:09.804 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.020904+0000", 2024-09-21T14:59:09.805 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2861e-05 2024-09-21T14:59:09.805 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.805 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.805 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:09.805 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.020987+0000", 2024-09-21T14:59:09.805 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2806000000000006e-05 2024-09-21T14:59:09.805 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.805 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.805 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:09.805 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.021060+0000", 2024-09-21T14:59:09.805 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2871000000000001e-05 2024-09-21T14:59:09.805 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:09.805 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:09.805 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:09.806 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.806 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.806 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9369 3.3 3:c725132c:::rbd_data.11363a21d2be.0000000000000007:10 [read 24576~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:09.806 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157241+0000", 2024-09-21T14:59:09.806 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 30.634418963999998, 2024-09-21T14:59:09.806 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018289156000000001, 2024-09-21T14:59:09.806 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:09.806 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:09.806 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:09.806 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:09.806 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:09.806 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9369 2024-09-21T14:59:09.806 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.806 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:09.807 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.807 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:09.807 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157241+0000", 2024-09-21T14:59:09.807 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:09.807 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.807 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.807 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:09.807 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157241+0000", 2024-09-21T14:59:09.807 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:09.807 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.807 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.807 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:09.808 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157241+0000", 2024-09-21T14:59:09.808 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:09.808 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.808 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.808 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:09.808 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157244+0000", 2024-09-21T14:59:09.808 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5049999999999999e-06 2024-09-21T14:59:09.808 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.808 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.808 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:09.808 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157255+0000", 2024-09-21T14:59:09.808 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0797e-05 2024-09-21T14:59:09.808 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.809 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.809 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:09.809 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157260+0000", 2024-09-21T14:59:09.809 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1749999999999996e-06 2024-09-21T14:59:09.809 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.809 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.809 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:09.809 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175237+0000", 2024-09-21T14:59:09.809 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.017976405000000001 2024-09-21T14:59:09.809 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.809 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.809 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:09.810 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175331+0000", 2024-09-21T14:59:09.810 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4508999999999999e-05 2024-09-21T14:59:09.810 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.810 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.810 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:09.810 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175530+0000", 2024-09-21T14:59:09.810 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00019928200000000001 2024-09-21T14:59:09.810 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:09.810 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:09.810 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:09.810 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.810 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.810 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9370 3.3 3:c725132c:::rbd_data.11363a21d2be.0000000000000007:10 [read 163840~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:09.811 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157279+0000", 2024-09-21T14:59:09.811 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 30.634380786000001, 2024-09-21T14:59:09.811 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018472379000000001, 2024-09-21T14:59:09.811 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:09.811 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:09.811 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:09.811 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:09.811 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:09.811 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9370 2024-09-21T14:59:09.811 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.811 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:09.811 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.811 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:09.811 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157279+0000", 2024-09-21T14:59:09.812 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:09.812 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.812 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.812 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:09.812 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157279+0000", 2024-09-21T14:59:09.812 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:09.812 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.812 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.812 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:09.812 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157279+0000", 2024-09-21T14:59:09.812 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:09.812 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.812 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.813 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:09.813 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157282+0000", 2024-09-21T14:59:09.813 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2229999999999999e-06 2024-09-21T14:59:09.813 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.813 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.813 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:09.813 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157294+0000", 2024-09-21T14:59:09.813 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2173e-05 2024-09-21T14:59:09.813 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.813 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.813 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:09.813 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157300+0000", 2024-09-21T14:59:09.813 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.93e-06 2024-09-21T14:59:09.813 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.814 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.814 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:09.814 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175569+0000", 2024-09-21T14:59:09.814 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018268376999999999 2024-09-21T14:59:09.814 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.814 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.814 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:09.814 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175622+0000", 2024-09-21T14:59:09.814 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3808999999999999e-05 2024-09-21T14:59:09.814 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.814 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.814 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:09.815 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175752+0000", 2024-09-21T14:59:09.815 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012938700000000001 2024-09-21T14:59:09.815 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:09.815 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:09.815 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:09.815 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.815 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.815 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9371 3.3 3:c0bf6a9b:::rbd_data.11363a21d2be.0000000000000000:10 [read 73728~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:09.815 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157306+0000", 2024-09-21T14:59:09.815 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 30.634354675000001, 2024-09-21T14:59:09.815 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018755473000000002, 2024-09-21T14:59:09.815 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:09.815 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:09.815 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:09.816 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:09.816 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:09.816 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9371 2024-09-21T14:59:09.816 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.816 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:09.816 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.816 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:09.816 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157306+0000", 2024-09-21T14:59:09.816 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:09.816 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.816 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.816 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:09.816 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157306+0000", 2024-09-21T14:59:09.816 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:09.816 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.817 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.817 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:09.817 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157305+0000", 2024-09-21T14:59:09.817 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:09.817 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.817 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.817 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:09.817 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157312+0000", 2024-09-21T14:59:09.817 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3289999999999996e-06 2024-09-21T14:59:09.817 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.817 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.817 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:09.817 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157324+0000", 2024-09-21T14:59:09.817 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1725e-05 2024-09-21T14:59:09.817 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.817 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.817 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:09.818 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157339+0000", 2024-09-21T14:59:09.818 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5211999999999999e-05 2024-09-21T14:59:09.818 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.818 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.818 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:09.818 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175798+0000", 2024-09-21T14:59:09.818 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018458781 2024-09-21T14:59:09.818 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.818 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.818 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:09.818 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175899+0000", 2024-09-21T14:59:09.818 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000100784 2024-09-21T14:59:09.818 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.819 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.819 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:09.819 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176061+0000", 2024-09-21T14:59:09.819 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00016216099999999999 2024-09-21T14:59:09.819 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:09.819 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:09.819 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:09.819 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.819 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.819 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9372 3.3 3:c0bf6a9b:::rbd_data.11363a21d2be.0000000000000000:10 [read 212992~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:09.819 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157345+0000", 2024-09-21T14:59:09.819 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 30.634315329, 2024-09-21T14:59:09.819 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018926493999999999, 2024-09-21T14:59:09.819 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:09.819 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:09.820 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:09.820 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:09.820 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:09.820 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9372 2024-09-21T14:59:09.820 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.820 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:09.820 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.820 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:09.820 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157345+0000", 2024-09-21T14:59:09.820 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:09.820 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.820 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.820 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:09.821 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157345+0000", 2024-09-21T14:59:09.821 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:09.821 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.821 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.821 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:09.821 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157344+0000", 2024-09-21T14:59:09.821 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:09.821 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.821 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.821 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:09.821 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157348+0000", 2024-09-21T14:59:09.821 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1389999999999999e-06 2024-09-21T14:59:09.821 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.821 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.821 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:09.822 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157372+0000", 2024-09-21T14:59:09.822 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4711000000000001e-05 2024-09-21T14:59:09.822 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.822 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.822 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:09.822 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157378+0000", 2024-09-21T14:59:09.822 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9909999999999999e-06 2024-09-21T14:59:09.822 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.822 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.822 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:09.822 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176090+0000", 2024-09-21T14:59:09.822 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018712111999999999 2024-09-21T14:59:09.822 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.822 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.823 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:09.823 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176134+0000", 2024-09-21T14:59:09.823 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3377999999999997e-05 2024-09-21T14:59:09.823 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.823 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.823 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:09.823 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176271+0000", 2024-09-21T14:59:09.823 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013763499999999999 2024-09-21T14:59:09.823 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:09.823 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:09.823 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:09.823 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.823 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.823 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9373 3.3 3:c0bf6a9b:::rbd_data.11363a21d2be.0000000000000000:10 [read 352256~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:09.824 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157384+0000", 2024-09-21T14:59:09.824 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 30.634276557, 2024-09-21T14:59:09.824 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019154937, 2024-09-21T14:59:09.824 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:09.824 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:09.824 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:09.824 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:09.824 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:09.824 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9373 2024-09-21T14:59:09.824 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.824 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:09.824 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.824 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:09.825 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157384+0000", 2024-09-21T14:59:09.825 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:09.825 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.825 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.825 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:09.825 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157384+0000", 2024-09-21T14:59:09.825 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:09.825 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.825 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.825 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:09.825 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157383+0000", 2024-09-21T14:59:09.825 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:09.825 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.825 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.825 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:09.826 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157386+0000", 2024-09-21T14:59:09.826 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.191e-06 2024-09-21T14:59:09.826 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.826 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.826 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:09.826 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157393+0000", 2024-09-21T14:59:09.826 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5339999999999996e-06 2024-09-21T14:59:09.826 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.826 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.826 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:09.826 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157397+0000", 2024-09-21T14:59:09.826 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5220000000000001e-06 2024-09-21T14:59:09.826 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.826 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.827 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:09.827 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176302+0000", 2024-09-21T14:59:09.827 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018905113000000001 2024-09-21T14:59:09.827 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.827 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.827 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:09.827 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176379+0000", 2024-09-21T14:59:09.827 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6372999999999995e-05 2024-09-21T14:59:09.827 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.827 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.827 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:09.827 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176539+0000", 2024-09-21T14:59:09.827 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015973800000000001 2024-09-21T14:59:09.827 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:09.828 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:09.828 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:09.828 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.828 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.828 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9374 3.3 3:c0bf6a9b:::rbd_data.11363a21d2be.0000000000000000:10 [read 491520~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:09.828 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157401+0000", 2024-09-21T14:59:09.828 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 30.634258772999999, 2024-09-21T14:59:09.828 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019387755, 2024-09-21T14:59:09.828 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:09.828 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:09.828 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:09.828 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:09.828 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:09.828 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9374 2024-09-21T14:59:09.828 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.828 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:09.829 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.829 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:09.829 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157401+0000", 2024-09-21T14:59:09.829 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:09.829 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.829 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.829 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:09.829 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157401+0000", 2024-09-21T14:59:09.829 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:09.829 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.829 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.829 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:09.829 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157401+0000", 2024-09-21T14:59:09.829 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:09.829 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.830 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.830 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:09.830 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157414+0000", 2024-09-21T14:59:09.830 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2417e-05 2024-09-21T14:59:09.830 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.830 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.830 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:09.830 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157424+0000", 2024-09-21T14:59:09.830 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0861e-05 2024-09-21T14:59:09.830 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.830 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.830 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:09.830 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157429+0000", 2024-09-21T14:59:09.830 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0499999999999999e-06 2024-09-21T14:59:09.831 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.831 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.831 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:09.831 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176596+0000", 2024-09-21T14:59:09.831 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019166552 2024-09-21T14:59:09.831 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.831 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.831 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:09.831 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176647+0000", 2024-09-21T14:59:09.831 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1112000000000001e-05 2024-09-21T14:59:09.831 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.831 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.831 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:09.831 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176789+0000", 2024-09-21T14:59:09.831 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014209300000000001 2024-09-21T14:59:09.832 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:09.832 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:09.832 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:09.832 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.832 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.832 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9401 3.3 3:c725132c:::rbd_data.11363a21d2be.0000000000000007:10 [read 32768~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:09.832 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157621+0000", 2024-09-21T14:59:09.832 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 30.634038963999998, 2024-09-21T14:59:09.832 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019381711999999999, 2024-09-21T14:59:09.832 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:09.832 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:09.832 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:09.832 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:09.832 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:09.832 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9401 2024-09-21T14:59:09.833 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.833 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:09.833 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.833 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:09.833 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157621+0000", 2024-09-21T14:59:09.833 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:09.833 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.833 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.833 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:09.833 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157621+0000", 2024-09-21T14:59:09.833 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:09.833 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.833 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.833 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:09.834 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157620+0000", 2024-09-21T14:59:09.834 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:59:09.834 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.834 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.834 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:09.834 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157626+0000", 2024-09-21T14:59:09.834 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7570000000000001e-06 2024-09-21T14:59:09.834 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.834 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.834 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:09.834 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157636+0000", 2024-09-21T14:59:09.834 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8339999999999994e-06 2024-09-21T14:59:09.834 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.835 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.835 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:09.835 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157643+0000", 2024-09-21T14:59:09.835 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9439999999999999e-06 2024-09-21T14:59:09.835 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.835 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.835 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:09.835 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176816+0000", 2024-09-21T14:59:09.835 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019172872000000001 2024-09-21T14:59:09.835 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.835 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.835 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:09.835 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176878+0000", 2024-09-21T14:59:09.835 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1734000000000002e-05 2024-09-21T14:59:09.835 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.836 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.836 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:09.836 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177003+0000", 2024-09-21T14:59:09.836 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012537400000000001 2024-09-21T14:59:09.836 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:09.836 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:09.836 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:09.836 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.836 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.836 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9402 3.3 3:c725132c:::rbd_data.11363a21d2be.0000000000000007:10 [read 172032~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:09.836 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157649+0000", 2024-09-21T14:59:09.836 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 30.634011092000001, 2024-09-21T14:59:09.836 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019567404, 2024-09-21T14:59:09.836 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:09.836 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:09.836 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:09.837 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:09.837 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:09.837 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9402 2024-09-21T14:59:09.837 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.837 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:09.837 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.837 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:09.837 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157649+0000", 2024-09-21T14:59:09.837 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:09.837 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.837 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.837 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:09.837 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157649+0000", 2024-09-21T14:59:09.837 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:09.838 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.838 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.838 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:09.838 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157649+0000", 2024-09-21T14:59:09.838 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:09.838 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.838 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.838 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:09.838 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157652+0000", 2024-09-21T14:59:09.838 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8779999999999998e-06 2024-09-21T14:59:09.838 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.838 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.838 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:09.838 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157657+0000", 2024-09-21T14:59:09.839 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6659999999999996e-06 2024-09-21T14:59:09.839 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.839 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.839 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:09.839 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157662+0000", 2024-09-21T14:59:09.839 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5519999999999998e-06 2024-09-21T14:59:09.839 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.839 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.839 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:09.839 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177040+0000", 2024-09-21T14:59:09.839 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019377807 2024-09-21T14:59:09.839 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.839 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.839 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:09.839 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177085+0000", 2024-09-21T14:59:09.839 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5195999999999999e-05 2024-09-21T14:59:09.840 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.840 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.840 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:09.840 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177217+0000", 2024-09-21T14:59:09.840 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013172600000000001 2024-09-21T14:59:09.840 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:09.840 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:09.840 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:09.840 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.840 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.840 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9403 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:09.840 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157949+0000", 2024-09-21T14:59:09.840 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 30.633710763, 2024-09-21T14:59:09.840 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023540373999999999, 2024-09-21T14:59:09.840 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:09.840 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:09.841 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:09.841 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:09.841 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:09.841 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9403 2024-09-21T14:59:09.841 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.841 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:09.841 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.841 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:09.841 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157949+0000", 2024-09-21T14:59:09.841 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:09.841 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.841 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.841 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:09.841 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157949+0000", 2024-09-21T14:59:09.841 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:09.842 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.842 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.842 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:09.842 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157948+0000", 2024-09-21T14:59:09.842 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T14:59:09.842 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.842 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.842 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:09.842 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157956+0000", 2024-09-21T14:59:09.842 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7999999999999999e-06 2024-09-21T14:59:09.842 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.842 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.842 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:09.843 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157969+0000", 2024-09-21T14:59:09.843 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2792000000000001e-05 2024-09-21T14:59:09.843 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.843 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.843 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:09.843 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157977+0000", 2024-09-21T14:59:09.843 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.704e-06 2024-09-21T14:59:09.843 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.843 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.843 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:09.843 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177246+0000", 2024-09-21T14:59:09.843 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019268264 2024-09-21T14:59:09.843 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.843 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.843 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:09.843 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177315+0000", 2024-09-21T14:59:09.844 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9480999999999994e-05 2024-09-21T14:59:09.844 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.844 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.844 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:59:09.844 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177615+0000", 2024-09-21T14:59:09.844 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00029929499999999999 2024-09-21T14:59:09.844 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.844 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.844 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:09.844 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181333+0000", 2024-09-21T14:59:09.844 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0037183149999999998 2024-09-21T14:59:09.844 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.844 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.844 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:09.844 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181374+0000", 2024-09-21T14:59:09.845 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.0723999999999998e-05 2024-09-21T14:59:09.845 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.845 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.845 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:09.845 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181434+0000", 2024-09-21T14:59:09.845 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0024e-05 2024-09-21T14:59:09.845 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.845 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.845 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:09.845 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181490+0000", 2024-09-21T14:59:09.845 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6245000000000001e-05 2024-09-21T14:59:09.845 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:09.845 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:09.845 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:09.845 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.845 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.846 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9404 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:09.846 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.160480+0000", 2024-09-21T14:59:09.846 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 30.631180412999999, 2024-09-21T14:59:09.846 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021137686999999999, 2024-09-21T14:59:09.846 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:09.846 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:09.846 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:09.846 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:09.846 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:09.846 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9404 2024-09-21T14:59:09.846 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.846 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:09.846 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.846 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:09.846 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160480+0000", 2024-09-21T14:59:09.847 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:09.847 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.847 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.847 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:09.847 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160480+0000", 2024-09-21T14:59:09.847 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:09.847 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.847 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.847 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:09.847 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160478+0000", 2024-09-21T14:59:09.847 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T14:59:09.847 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.847 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.847 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:09.847 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160486+0000", 2024-09-21T14:59:09.847 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9969999999999998e-06 2024-09-21T14:59:09.848 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.848 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.848 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:09.848 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160502+0000", 2024-09-21T14:59:09.848 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5650000000000001e-05 2024-09-21T14:59:09.848 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.848 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.848 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:09.848 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160511+0000", 2024-09-21T14:59:09.848 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.3610000000000007e-06 2024-09-21T14:59:09.848 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.848 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.848 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:09.848 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.178035+0000", 2024-09-21T14:59:09.848 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.017523520000000001 2024-09-21T14:59:09.848 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.849 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.849 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:09.849 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.178151+0000", 2024-09-21T14:59:09.849 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000116013 2024-09-21T14:59:09.849 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.849 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.849 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:59:09.849 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.178408+0000", 2024-09-21T14:59:09.849 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00025743400000000002 2024-09-21T14:59:09.849 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.849 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.849 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:09.849 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181501+0000", 2024-09-21T14:59:09.849 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.003092891 2024-09-21T14:59:09.849 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.849 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.850 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:09.850 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181538+0000", 2024-09-21T14:59:09.850 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6977999999999997e-05 2024-09-21T14:59:09.850 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.850 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.850 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:09.850 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181582+0000", 2024-09-21T14:59:09.850 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3849e-05 2024-09-21T14:59:09.850 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.850 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.850 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:09.850 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181617+0000", 2024-09-21T14:59:09.850 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5423000000000003e-05 2024-09-21T14:59:09.850 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:09.850 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:09.851 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:09.851 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.851 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.851 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9406 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:09.851 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.166040+0000", 2024-09-21T14:59:09.851 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 30.625620533999999, 2024-09-21T14:59:09.851 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018094524000000001, 2024-09-21T14:59:09.851 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:09.851 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:09.851 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:09.851 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:09.851 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:09.851 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9406 2024-09-21T14:59:09.851 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.851 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:09.851 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.852 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:09.852 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.166040+0000", 2024-09-21T14:59:09.852 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:09.852 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.852 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.852 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:09.852 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.166040+0000", 2024-09-21T14:59:09.852 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:09.852 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.852 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.852 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:09.852 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.166038+0000", 2024-09-21T14:59:09.852 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T14:59:09.852 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.852 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.852 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:09.853 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.166051+0000", 2024-09-21T14:59:09.853 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2638e-05 2024-09-21T14:59:09.853 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.853 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.853 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:09.853 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.166070+0000", 2024-09-21T14:59:09.853 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8691000000000001e-05 2024-09-21T14:59:09.853 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.853 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.853 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:09.853 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.166089+0000", 2024-09-21T14:59:09.853 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9286000000000001e-05 2024-09-21T14:59:09.853 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.853 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.853 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:09.854 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.180194+0000", 2024-09-21T14:59:09.854 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014105039999999999 2024-09-21T14:59:09.854 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.854 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.854 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:09.854 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.180340+0000", 2024-09-21T14:59:09.854 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014623199999999999 2024-09-21T14:59:09.854 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.854 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.854 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:59:09.854 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.180818+0000", 2024-09-21T14:59:09.854 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000477489 2024-09-21T14:59:09.854 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.854 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.855 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:09.855 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.183808+0000", 2024-09-21T14:59:09.855 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0029897610000000001 2024-09-21T14:59:09.855 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.855 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.855 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:09.855 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.183821+0000", 2024-09-21T14:59:09.855 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3251999999999999e-05 2024-09-21T14:59:09.855 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.855 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.855 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:09.855 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.183872+0000", 2024-09-21T14:59:09.855 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0900999999999999e-05 2024-09-21T14:59:09.855 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.855 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.856 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:09.856 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.184134+0000", 2024-09-21T14:59:09.856 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000262484 2024-09-21T14:59:09.856 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:09.856 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:09.856 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:09.856 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.856 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.856 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9407 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:09.856 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.168094+0000", 2024-09-21T14:59:09.856 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 30.623565914, 2024-09-21T14:59:09.856 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019444860000000001, 2024-09-21T14:59:09.856 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:09.856 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:09.856 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:09.857 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:09.857 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:09.857 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9407 2024-09-21T14:59:09.857 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.857 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:09.857 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.857 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:09.857 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.168094+0000", 2024-09-21T14:59:09.857 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:09.857 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.857 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.857 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:09.857 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.168094+0000", 2024-09-21T14:59:09.857 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:09.857 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.858 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.858 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:09.858 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.168093+0000", 2024-09-21T14:59:09.858 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:59:09.858 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.858 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.858 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:09.858 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.168119+0000", 2024-09-21T14:59:09.858 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5803999999999999e-05 2024-09-21T14:59:09.858 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.858 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.858 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:09.858 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.168135+0000", 2024-09-21T14:59:09.858 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6359999999999999e-05 2024-09-21T14:59:09.858 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.859 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.859 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:09.859 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.168144+0000", 2024-09-21T14:59:09.859 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4270000000000008e-06 2024-09-21T14:59:09.859 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.859 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.859 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:09.859 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181943+0000", 2024-09-21T14:59:09.859 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013799617 2024-09-21T14:59:09.859 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.859 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.859 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:09.859 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.182052+0000", 2024-09-21T14:59:09.859 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000108924 2024-09-21T14:59:09.859 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.860 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.860 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:59:09.860 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.183431+0000", 2024-09-21T14:59:09.860 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0013783619999999999 2024-09-21T14:59:09.860 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.860 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.860 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:09.860 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.187276+0000", 2024-09-21T14:59:09.860 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0038450469999999999 2024-09-21T14:59:09.860 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.860 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.860 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:09.860 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.187288+0000", 2024-09-21T14:59:09.860 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2414e-05 2024-09-21T14:59:09.860 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.860 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.861 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:09.861 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.187414+0000", 2024-09-21T14:59:09.861 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012614399999999999 2024-09-21T14:59:09.861 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.861 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.861 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:09.861 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.187539+0000", 2024-09-21T14:59:09.861 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012481800000000001 2024-09-21T14:59:09.861 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:09.861 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:09.861 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:09.861 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.861 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.861 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9410 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:09.861 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.174103+0000", 2024-09-21T14:59:09.862 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 30.617557004999998, 2024-09-21T14:59:09.862 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025493703, 2024-09-21T14:59:09.862 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:09.862 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:09.862 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:09.862 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:09.862 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:09.862 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9410 2024-09-21T14:59:09.862 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.862 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:09.862 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.862 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:09.862 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.174103+0000", 2024-09-21T14:59:09.862 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:09.862 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.863 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.863 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:09.863 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.174103+0000", 2024-09-21T14:59:09.863 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:09.863 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.863 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.863 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:09.863 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.174102+0000", 2024-09-21T14:59:09.863 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:59:09.863 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.863 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.863 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:09.863 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.174108+0000", 2024-09-21T14:59:09.863 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1149999999999999e-06 2024-09-21T14:59:09.863 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.864 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.864 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:09.864 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.174117+0000", 2024-09-21T14:59:09.864 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4870000000000002e-06 2024-09-21T14:59:09.864 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.864 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.864 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:09.864 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.174122+0000", 2024-09-21T14:59:09.864 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9790000000000001e-06 2024-09-21T14:59:09.864 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.864 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.864 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:09.864 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.187830+0000", 2024-09-21T14:59:09.864 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013708048 2024-09-21T14:59:09.864 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.865 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.865 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:09.865 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.187913+0000", 2024-09-21T14:59:09.865 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2763999999999996e-05 2024-09-21T14:59:09.865 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.865 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.865 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:59:09.865 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.188230+0000", 2024-09-21T14:59:09.865 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00031745100000000001 2024-09-21T14:59:09.865 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.865 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.865 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:09.865 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.192708+0000", 2024-09-21T14:59:09.865 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0044783890000000002 2024-09-21T14:59:09.865 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.865 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.866 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:09.866 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199487+0000", 2024-09-21T14:59:09.866 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0067782149999999998 2024-09-21T14:59:09.866 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.866 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.866 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:09.866 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199560+0000", 2024-09-21T14:59:09.866 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3752999999999997e-05 2024-09-21T14:59:09.866 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.866 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.866 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:09.866 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199597+0000", 2024-09-21T14:59:09.866 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6526999999999998e-05 2024-09-21T14:59:09.866 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:09.866 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:09.867 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:09.867 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.867 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.867 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9411 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:09.867 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.175943+0000", 2024-09-21T14:59:09.867 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 30.615716985999999, 2024-09-21T14:59:09.867 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023773181000000001, 2024-09-21T14:59:09.867 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:09.867 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:09.867 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:09.867 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:09.867 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:09.867 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9411 2024-09-21T14:59:09.867 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.867 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:09.867 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.867 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:09.868 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175943+0000", 2024-09-21T14:59:09.868 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:09.868 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.868 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.868 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:09.868 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175943+0000", 2024-09-21T14:59:09.868 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:09.868 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.868 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.868 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:09.868 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175942+0000", 2024-09-21T14:59:09.868 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:59:09.868 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.868 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.868 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:09.868 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175948+0000", 2024-09-21T14:59:09.869 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9689999999999999e-06 2024-09-21T14:59:09.869 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.869 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.869 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:09.869 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175960+0000", 2024-09-21T14:59:09.869 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1745e-05 2024-09-21T14:59:09.869 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.869 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.869 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:09.869 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175965+0000", 2024-09-21T14:59:09.869 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.203e-06 2024-09-21T14:59:09.869 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.869 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.869 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:09.869 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.189900+0000", 2024-09-21T14:59:09.870 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013934940999999999 2024-09-21T14:59:09.870 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.870 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.870 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:09.870 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.189988+0000", 2024-09-21T14:59:09.870 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8419999999999997e-05 2024-09-21T14:59:09.870 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.870 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.870 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:59:09.870 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.190321+0000", 2024-09-21T14:59:09.870 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00033231800000000001 2024-09-21T14:59:09.870 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.870 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.870 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:09.871 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.192746+0000", 2024-09-21T14:59:09.871 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0024254400000000001 2024-09-21T14:59:09.871 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.871 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.871 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:09.871 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199605+0000", 2024-09-21T14:59:09.871 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0068582809999999999 2024-09-21T14:59:09.871 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.871 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.871 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:09.871 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199680+0000", 2024-09-21T14:59:09.871 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5630000000000006e-05 2024-09-21T14:59:09.871 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.871 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.871 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:09.871 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199716+0000", 2024-09-21T14:59:09.872 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6248999999999998e-05 2024-09-21T14:59:09.872 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:09.872 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:09.872 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:09.872 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.872 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.872 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9412 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:09.872 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.177402+0000", 2024-09-21T14:59:09.872 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 30.614257889000001, 2024-09-21T14:59:09.872 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022445590000000001, 2024-09-21T14:59:09.872 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:09.872 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:09.872 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:09.872 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:09.872 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:09.873 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9412 2024-09-21T14:59:09.873 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.873 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:09.873 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.873 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:09.873 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177402+0000", 2024-09-21T14:59:09.873 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:09.873 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.873 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.873 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:09.873 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177402+0000", 2024-09-21T14:59:09.873 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:09.873 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.873 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.873 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:09.874 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177401+0000", 2024-09-21T14:59:09.874 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:59:09.874 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.874 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.874 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:09.874 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177409+0000", 2024-09-21T14:59:09.874 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4710000000000001e-06 2024-09-21T14:59:09.874 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.874 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.874 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:09.874 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177419+0000", 2024-09-21T14:59:09.874 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0351e-05 2024-09-21T14:59:09.874 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.875 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.875 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:09.875 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177424+0000", 2024-09-21T14:59:09.875 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1919999999999996e-06 2024-09-21T14:59:09.875 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.875 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.875 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:09.875 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.191343+0000", 2024-09-21T14:59:09.875 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.01391878 2024-09-21T14:59:09.875 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.875 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.875 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:09.875 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.191446+0000", 2024-09-21T14:59:09.875 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000102453 2024-09-21T14:59:09.875 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.876 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.876 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:59:09.876 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.192200+0000", 2024-09-21T14:59:09.876 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00075476899999999997 2024-09-21T14:59:09.876 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.876 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.876 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:09.876 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.193866+0000", 2024-09-21T14:59:09.876 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0016657519999999999 2024-09-21T14:59:09.876 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.876 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.876 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:09.876 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199724+0000", 2024-09-21T14:59:09.876 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0058579529999999999 2024-09-21T14:59:09.876 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.876 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.877 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:09.877 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199803+0000", 2024-09-21T14:59:09.877 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8662e-05 2024-09-21T14:59:09.877 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.877 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.877 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:09.877 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199848+0000", 2024-09-21T14:59:09.877 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5216000000000002e-05 2024-09-21T14:59:09.877 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:09.877 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:09.877 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:09.877 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.877 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.877 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:10445 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:09.877 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.271980+0000", 2024-09-21T14:59:09.877 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 30.519680524000002, 2024-09-21T14:59:09.877 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018664256000000001, 2024-09-21T14:59:09.878 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:09.878 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:09.878 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:09.878 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:09.878 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:09.878 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 10445 2024-09-21T14:59:09.878 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.878 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:09.878 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.878 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:09.878 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.271980+0000", 2024-09-21T14:59:09.878 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:09.878 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.878 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.878 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:09.879 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.271980+0000", 2024-09-21T14:59:09.879 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:09.879 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.879 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.879 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:09.879 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.271979+0000", 2024-09-21T14:59:09.879 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:09.879 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.879 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.879 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:09.879 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.271982+0000", 2024-09-21T14:59:09.879 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7240000000000001e-06 2024-09-21T14:59:09.879 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.879 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.879 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:09.880 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.271987+0000", 2024-09-21T14:59:09.880 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3639999999999998e-06 2024-09-21T14:59:09.880 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.880 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.880 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:09.880 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.271992+0000", 2024-09-21T14:59:09.880 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7110000000000003e-06 2024-09-21T14:59:09.880 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.880 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.880 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:09.880 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.277567+0000", 2024-09-21T14:59:09.880 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0055745009999999999 2024-09-21T14:59:09.880 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.880 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.880 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:09.880 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.277655+0000", 2024-09-21T14:59:09.880 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8465000000000001e-05 2024-09-21T14:59:09.881 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.881 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.881 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:59:09.881 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.278720+0000", 2024-09-21T14:59:09.881 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0010650569999999999 2024-09-21T14:59:09.881 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.881 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.881 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:09.881 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.281153+0000", 2024-09-21T14:59:09.881 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0024326389999999999 2024-09-21T14:59:09.881 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.881 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.881 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:09.881 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.290574+0000", 2024-09-21T14:59:09.881 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0094209489999999996 2024-09-21T14:59:09.881 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.882 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.882 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:09.882 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.290613+0000", 2024-09-21T14:59:09.882 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.8803000000000001e-05 2024-09-21T14:59:09.882 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.882 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.882 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:09.882 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.290644+0000", 2024-09-21T14:59:09.882 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1404e-05 2024-09-21T14:59:09.882 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:09.882 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:09.882 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:09.882 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.882 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.882 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22769 3.f 3:f39bd56f:::rbd_data.11366acc894.0000000000000004:19 [read 1032192~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:09.883 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.140207+0000", 2024-09-21T14:59:09.883 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 19.651452784, 2024-09-21T14:59:09.883 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018031170999999999, 2024-09-21T14:59:09.883 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:09.883 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:09.883 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:09.883 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:09.883 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:09.883 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22769 2024-09-21T14:59:09.883 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.883 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:09.883 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.883 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:09.883 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.140207+0000", 2024-09-21T14:59:09.883 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:09.883 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.884 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.884 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:09.884 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.140207+0000", 2024-09-21T14:59:09.884 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:09.884 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.884 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.884 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:09.884 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.140207+0000", 2024-09-21T14:59:09.884 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:09.884 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.884 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.884 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:09.884 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.140209+0000", 2024-09-21T14:59:09.884 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.305e-06 2024-09-21T14:59:09.884 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.885 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.885 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:09.885 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.140214+0000", 2024-09-21T14:59:09.885 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9570000000000001e-06 2024-09-21T14:59:09.885 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.885 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.885 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:09.885 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.140219+0000", 2024-09-21T14:59:09.885 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2250000000000002e-06 2024-09-21T14:59:09.885 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.885 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.885 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:09.885 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.157990+0000", 2024-09-21T14:59:09.885 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.017770939999999999 2024-09-21T14:59:09.885 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.886 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.886 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:09.886 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.158062+0000", 2024-09-21T14:59:09.886 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2652999999999997e-05 2024-09-21T14:59:09.886 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.886 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.886 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:09.886 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.158239+0000", 2024-09-21T14:59:09.886 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017640599999999999 2024-09-21T14:59:09.886 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:09.886 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:09.886 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:09.886 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.886 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.887 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:23084 3.3 3:c55e343c:::rbd_data.11366acc894.0000000000000006:19 [read 1032192~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:09.887 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.150724+0000", 2024-09-21T14:59:09.887 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 19.640935948999999, 2024-09-21T14:59:09.887 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018916460999999999, 2024-09-21T14:59:09.887 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:09.887 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:09.887 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:09.887 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:09.887 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:09.887 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 23084 2024-09-21T14:59:09.887 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.887 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:09.887 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.887 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:09.887 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.150724+0000", 2024-09-21T14:59:09.888 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:09.888 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.888 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.888 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:09.888 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.150724+0000", 2024-09-21T14:59:09.888 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:09.888 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.888 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.888 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:09.888 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.150724+0000", 2024-09-21T14:59:09.888 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:09.888 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.888 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.888 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:09.888 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.150727+0000", 2024-09-21T14:59:09.889 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7250000000000002e-06 2024-09-21T14:59:09.889 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.889 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.889 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:09.889 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.150732+0000", 2024-09-21T14:59:09.889 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5809999999999996e-06 2024-09-21T14:59:09.889 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.889 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.889 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:09.889 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.150737+0000", 2024-09-21T14:59:09.889 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3590000000000001e-06 2024-09-21T14:59:09.889 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.889 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.889 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:09.889 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.166694+0000", 2024-09-21T14:59:09.890 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.015957720000000002 2024-09-21T14:59:09.890 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.890 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.890 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:09.890 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.166769+0000", 2024-09-21T14:59:09.890 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5143999999999997e-05 2024-09-21T14:59:09.890 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:09.890 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:09.890 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:09.890 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.169641+0000", 2024-09-21T14:59:09.890 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0028713089999999998 2024-09-21T14:59:09.890 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:09.890 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:09.890 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:09.891 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:09.891 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:09.891 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:09.891 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T14:59:09.893 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T14:59:09.893 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:59:09.892+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:59:09.993 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:59:09.993+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:59:10.060 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:10.060 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:59:10.060 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T14:59:10.060 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:10.071 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T14:59:10.087 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T14:59:10.094 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:59:10.093+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:59:10.140 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T14:59:10.185 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T14:59:10.195 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:59:10.193+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:59:10.217 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:10.217 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:10.224 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T14:59:10.232 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T14:59:10.262 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-21T14:59:10.294 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T14:59:10.296 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:59:10.293+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:59:10.329 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T14:59:10.338 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T14:59:10.364 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T14:59:10.367 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:59:10.367 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:59:10.367 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:59:10.367 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:59:10.368 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:59:10.368 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:59:10.368 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:59:10.399 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:10.399 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:10.399 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:59:10.397+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:59:10.409 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T14:59:10.498 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:59:10.496+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:59:10.565 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:10.565 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:10.575 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T14:59:10.598 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:59:10.597+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:59:10.700 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:59:10.697+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:59:10.705 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:10.705 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:10.716 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T14:59:10.801 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:59:10.801+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:59:10.873 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:10.873 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:10.883 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T14:59:10.902 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:59:10.900+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:59:10.990 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:10.991 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:11.000 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T14:59:11.003 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:59:11.001+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:59:11.103 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:59:11.100+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:59:11.108 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:11.108 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:11.119 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T14:59:11.124 INFO:teuthology.orchestra.run.smithi032.stderr:noscrub is unset 2024-09-21T14:59:11.204 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:59:11.200+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:59:11.267 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:11.267 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:11.278 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T14:59:11.305 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:59:11.305+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:59:11.402 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:11.402 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:11.406 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:59:11.405+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:59:11.415 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T14:59:11.507 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:59:11.505+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:59:11.553 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:11.553 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:11.565 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T14:59:11.608 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:59:11.604+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:59:11.669 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:11.670 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:11.685 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T14:59:11.729 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:59:11.728+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:59:11.811 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:59:11.809+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:59:11.814 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:11.814 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:11.823 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T14:59:11.912 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:59:11.908+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:59:11.948 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:11.948 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:11.958 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T14:59:12.013 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:59:12.012+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:59:12.094 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:12.094 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:12.109 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T14:59:12.114 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:59:12.113+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:59:12.215 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:59:12.213+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:59:12.254 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:12.254 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:12.264 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T14:59:12.315 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:59:12.312+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:59:12.367 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:12.367 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:12.377 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T14:59:12.384 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T14:59:12.429 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:59:12.429+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:59:12.517 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:59:12.516+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:59:12.539 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:12.539 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:12.540 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T14:59:12.548 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T14:59:12.569 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T14:59:12.589 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T14:59:12.617 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T14:59:12.617 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:59:12.616+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:59:12.641 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-21T14:59:12.658 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:12.658 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:12.663 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T14:59:12.668 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T14:59:12.687 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T14:59:12.709 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T14:59:12.711 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:59:12.711 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:59:12.712 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:59:12.712 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:59:12.712 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:59:12.712 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:59:12.712 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:59:12.712 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:59:12.718 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:59:12.717+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:59:12.777 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:59:12.777 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:59:12.778 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T14:59:12.778 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T14:59:12.778 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:59:12.787 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T14:59:12.819 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:59:12.817+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:59:12.901 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:59:12.901 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T14:59:12.901 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T14:59:12.901 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T14:59:12.901 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.901 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:8034 3.b 3:d1b12ef9:::rbd_data.11361bed1f78.0000000000000062:d [read 32768~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:59:12.902 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:35.227265+0000", 2024-09-21T14:59:12.902 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 37.670805776000002, 2024-09-21T14:59:12.902 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.006525968, 2024-09-21T14:59:12.902 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:12.902 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:12.902 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:12.902 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:12.902 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:12.902 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 8034 2024-09-21T14:59:12.902 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.902 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:12.902 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.902 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:12.903 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227265+0000", 2024-09-21T14:59:12.903 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:12.903 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.903 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.903 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:12.903 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227265+0000", 2024-09-21T14:59:12.903 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:12.903 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.903 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.903 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:12.903 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227265+0000", 2024-09-21T14:59:12.903 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:12.903 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.904 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.904 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:12.904 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227268+0000", 2024-09-21T14:59:12.904 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8990000000000001e-06 2024-09-21T14:59:12.904 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.904 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.904 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:12.904 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227274+0000", 2024-09-21T14:59:12.904 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4470000000000003e-06 2024-09-21T14:59:12.904 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.904 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.904 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:12.904 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227288+0000", 2024-09-21T14:59:12.904 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3550000000000001e-05 2024-09-21T14:59:12.905 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.905 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.905 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:12.905 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.233418+0000", 2024-09-21T14:59:12.905 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0061307870000000004 2024-09-21T14:59:12.905 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.905 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.905 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:12.905 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.233553+0000", 2024-09-21T14:59:12.905 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013475600000000001 2024-09-21T14:59:12.905 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.905 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.905 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:12.905 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.233791+0000", 2024-09-21T14:59:12.905 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00023788499999999999 2024-09-21T14:59:12.906 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:12.906 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:12.906 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:12.906 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.906 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.906 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:8035 3.b 3:d1b12ef9:::rbd_data.11361bed1f78.0000000000000062:d [read 98304~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:59:12.906 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:35.227295+0000", 2024-09-21T14:59:12.906 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 37.670775571, 2024-09-21T14:59:12.906 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0069673820000000003, 2024-09-21T14:59:12.906 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:12.906 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:12.906 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:12.907 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:12.907 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:12.907 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 8035 2024-09-21T14:59:12.907 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.907 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:12.907 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.907 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:12.907 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227295+0000", 2024-09-21T14:59:12.907 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:12.907 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.907 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.907 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:12.907 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227295+0000", 2024-09-21T14:59:12.907 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:12.908 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.908 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.908 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:12.908 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227295+0000", 2024-09-21T14:59:12.908 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:12.908 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.908 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.908 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:12.908 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227298+0000", 2024-09-21T14:59:12.908 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2339999999999999e-06 2024-09-21T14:59:12.908 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.908 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.908 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:12.908 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227305+0000", 2024-09-21T14:59:12.908 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0389999999999999e-06 2024-09-21T14:59:12.909 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.909 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.909 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:12.909 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227311+0000", 2024-09-21T14:59:12.909 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8470000000000001e-06 2024-09-21T14:59:12.909 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.909 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.909 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:12.909 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.233824+0000", 2024-09-21T14:59:12.909 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0065132920000000004 2024-09-21T14:59:12.909 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.909 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.909 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:12.909 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.233922+0000", 2024-09-21T14:59:12.910 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.8073999999999996e-05 2024-09-21T14:59:12.910 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.910 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.910 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:12.910 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.234263+0000", 2024-09-21T14:59:12.910 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00034027999999999999 2024-09-21T14:59:12.910 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:12.910 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:12.910 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:12.910 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.910 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.910 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:8048 3.b 3:d1b12ef9:::rbd_data.11361bed1f78.0000000000000062:d [read 49152~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:59:12.910 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:35.227316+0000", 2024-09-21T14:59:12.911 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 37.670754778999999, 2024-09-21T14:59:12.911 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0076197799999999996, 2024-09-21T14:59:12.911 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:12.911 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:12.911 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:12.911 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:12.911 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:12.911 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 8048 2024-09-21T14:59:12.911 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.911 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:12.911 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.911 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:12.911 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227316+0000", 2024-09-21T14:59:12.911 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:12.912 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.912 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.912 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:12.912 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227316+0000", 2024-09-21T14:59:12.912 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:12.912 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.912 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.912 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:12.912 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227316+0000", 2024-09-21T14:59:12.912 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:12.912 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.912 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.912 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:12.912 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227324+0000", 2024-09-21T14:59:12.913 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1009999999999992e-06 2024-09-21T14:59:12.913 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.913 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.913 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:12.913 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227346+0000", 2024-09-21T14:59:12.913 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2347000000000001e-05 2024-09-21T14:59:12.913 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.913 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.913 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:12.913 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227360+0000", 2024-09-21T14:59:12.913 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.403e-05 2024-09-21T14:59:12.913 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.913 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.913 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:12.914 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.234320+0000", 2024-09-21T14:59:12.914 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0069597019999999999 2024-09-21T14:59:12.914 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.914 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.914 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:12.914 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.234730+0000", 2024-09-21T14:59:12.914 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00041006300000000001 2024-09-21T14:59:12.914 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.914 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.914 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:12.914 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.234936+0000", 2024-09-21T14:59:12.915 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00020586700000000001 2024-09-21T14:59:12.915 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:12.915 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:12.915 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:12.915 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.915 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.915 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:8049 3.b 3:d1b12ef9:::rbd_data.11361bed1f78.0000000000000062:d [read 114688~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:59:12.915 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:35.227371+0000", 2024-09-21T14:59:12.915 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 37.670699446, 2024-09-21T14:59:12.915 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0078533609999999997, 2024-09-21T14:59:12.915 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:12.916 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:12.916 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:12.916 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:12.916 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:12.916 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 8049 2024-09-21T14:59:12.916 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.917 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:12.917 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.917 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:12.917 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227371+0000", 2024-09-21T14:59:12.917 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:12.917 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.917 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.917 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:12.918 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227371+0000", 2024-09-21T14:59:12.918 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:12.918 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.918 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.918 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:12.918 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227370+0000", 2024-09-21T14:59:12.918 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:59:12.918 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.919 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.919 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:12.919 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227378+0000", 2024-09-21T14:59:12.919 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.379e-06 2024-09-21T14:59:12.919 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.920 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.920 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:12.920 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227388+0000", 2024-09-21T14:59:12.920 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0175e-05 2024-09-21T14:59:12.920 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.920 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.920 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:12.920 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227393+0000", 2024-09-21T14:59:12.920 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6690000000000003e-06 2024-09-21T14:59:12.921 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.921 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.921 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:12.921 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.234964+0000", 2024-09-21T14:59:12.921 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0075705319999999996 2024-09-21T14:59:12.921 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.921 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.921 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:12.921 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.235041+0000", 2024-09-21T14:59:12.922 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7170999999999997e-05 2024-09-21T14:59:12.922 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.922 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.922 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:12.922 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.235225+0000", 2024-09-21T14:59:12.922 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000183735 2024-09-21T14:59:12.922 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:12.922 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:12.923 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:12.923 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.923 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.923 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:19869 3.b 3:d7ca7d68:::rbd_data.11363a21d2be.000000000000002d:head [sparse-read 0~393216] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T14:59:12.923 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:44.340896+0000", 2024-09-21T14:59:12.923 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 28.557175049000001, 2024-09-21T14:59:12.923 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0071749170000000003, 2024-09-21T14:59:12.923 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:12.923 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:12.924 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:12.924 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:12.924 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:12.924 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 19869 2024-09-21T14:59:12.924 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.924 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:12.924 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.924 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:12.925 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340896+0000", 2024-09-21T14:59:12.925 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:12.925 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.925 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.925 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:12.925 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340896+0000", 2024-09-21T14:59:12.925 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:12.925 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.925 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.926 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:12.926 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340894+0000", 2024-09-21T14:59:12.926 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:59:12.926 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.926 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.926 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:12.926 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340903+0000", 2024-09-21T14:59:12.926 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0259999999999996e-06 2024-09-21T14:59:12.926 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.927 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.927 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:12.927 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340923+0000", 2024-09-21T14:59:12.927 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0013999999999998e-05 2024-09-21T14:59:12.927 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.927 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.927 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:12.927 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340938+0000", 2024-09-21T14:59:12.928 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4569000000000001e-05 2024-09-21T14:59:12.928 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.928 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.928 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:12.928 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342092+0000", 2024-09-21T14:59:12.928 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00115354 2024-09-21T14:59:12.928 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.928 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.929 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:12.929 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342290+0000", 2024-09-21T14:59:12.929 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019834299999999999 2024-09-21T14:59:12.929 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.929 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.929 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:12.929 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.348071+0000", 2024-09-21T14:59:12.929 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.005780861 2024-09-21T14:59:12.929 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:12.930 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:12.930 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:12.930 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.930 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.930 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:19872 3.b 3:d38ac3f6:::rbd_data.11363a21d2be.0000000000000030:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T14:59:12.930 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:44.340951+0000", 2024-09-21T14:59:12.930 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 28.557119889999999, 2024-09-21T14:59:12.930 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0069228680000000004, 2024-09-21T14:59:12.930 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:12.931 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:12.931 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:12.931 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:12.931 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:12.931 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 19872 2024-09-21T14:59:12.931 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.931 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:12.931 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.931 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:12.931 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340951+0000", 2024-09-21T14:59:12.931 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:12.931 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.931 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.932 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:12.932 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340951+0000", 2024-09-21T14:59:12.932 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:12.932 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.932 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.932 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:12.932 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340950+0000", 2024-09-21T14:59:12.932 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:12.932 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.932 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.932 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:12.932 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340957+0000", 2024-09-21T14:59:12.932 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8430000000000004e-06 2024-09-21T14:59:12.932 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.933 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.933 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:12.933 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340974+0000", 2024-09-21T14:59:12.933 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7082999999999999e-05 2024-09-21T14:59:12.933 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.933 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.933 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:12.933 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340987+0000", 2024-09-21T14:59:12.933 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2763e-05 2024-09-21T14:59:12.933 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.933 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.933 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:12.933 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.347149+0000", 2024-09-21T14:59:12.934 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0061626600000000004 2024-09-21T14:59:12.934 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.934 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.934 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:12.934 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.347354+0000", 2024-09-21T14:59:12.934 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00020441799999999999 2024-09-21T14:59:12.934 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.934 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.934 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:12.934 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.347874+0000", 2024-09-21T14:59:12.934 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00052019399999999997 2024-09-21T14:59:12.934 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:12.935 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:12.935 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:12.935 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.935 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.935 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:19877 3.b 3:d5fc5a5a:::rbd_data.11363a21d2be.0000000000000035:head [sparse-read 0~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T14:59:12.935 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:44.340998+0000", 2024-09-21T14:59:12.935 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 28.557073054, 2024-09-21T14:59:12.935 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0073322129999999998, 2024-09-21T14:59:12.935 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:12.935 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:12.935 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:12.935 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:12.935 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:12.936 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 19877 2024-09-21T14:59:12.936 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.936 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:12.936 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.936 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:12.936 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340998+0000", 2024-09-21T14:59:12.936 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:12.936 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.936 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.936 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:12.936 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340998+0000", 2024-09-21T14:59:12.936 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:12.936 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.937 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.937 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:12.937 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340997+0000", 2024-09-21T14:59:12.937 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:12.937 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.937 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.937 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:12.937 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.341003+0000", 2024-09-21T14:59:12.937 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0070000000000003e-06 2024-09-21T14:59:12.937 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.937 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.937 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:12.937 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.341018+0000", 2024-09-21T14:59:12.937 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5472999999999999e-05 2024-09-21T14:59:12.938 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.938 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.938 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:12.938 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.341040+0000", 2024-09-21T14:59:12.938 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1568000000000001e-05 2024-09-21T14:59:12.938 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.938 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.938 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:12.938 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.347901+0000", 2024-09-21T14:59:12.938 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0068608300000000001 2024-09-21T14:59:12.938 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.938 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.939 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:12.939 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.347973+0000", 2024-09-21T14:59:12.939 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2327000000000001e-05 2024-09-21T14:59:12.939 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.939 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.939 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:12.939 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.348330+0000", 2024-09-21T14:59:12.939 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00035695799999999999 2024-09-21T14:59:12.939 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:12.939 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:12.939 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:12.939 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.939 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.939 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:19941 3.b 3:d7b21c30:::rbd_data.11363a21d2be.0000000000000075:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T14:59:12.940 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:44.342141+0000", 2024-09-21T14:59:12.940 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 28.555929487, 2024-09-21T14:59:12.940 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0063087439999999998, 2024-09-21T14:59:12.940 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:12.940 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:12.940 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:12.941 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:12.941 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:12.941 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 19941 2024-09-21T14:59:12.941 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.941 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:12.941 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.941 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:12.941 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342141+0000", 2024-09-21T14:59:12.941 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:12.941 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.941 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.941 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:12.941 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342141+0000", 2024-09-21T14:59:12.941 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:12.942 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.942 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.942 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:12.942 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342139+0000", 2024-09-21T14:59:12.942 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T14:59:12.942 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.942 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.942 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:12.942 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342151+0000", 2024-09-21T14:59:12.942 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1242e-05 2024-09-21T14:59:12.942 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.942 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.942 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:12.942 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342175+0000", 2024-09-21T14:59:12.943 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4491e-05 2024-09-21T14:59:12.943 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.943 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.943 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:12.943 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342191+0000", 2024-09-21T14:59:12.943 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6288000000000001e-05 2024-09-21T14:59:12.943 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.943 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.943 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:12.943 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.348358+0000", 2024-09-21T14:59:12.943 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0061665449999999998 2024-09-21T14:59:12.943 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.943 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.944 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:12.944 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.348450+0000", 2024-09-21T14:59:12.944 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2245999999999997e-05 2024-09-21T14:59:12.944 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:12.944 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:12.944 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:12.944 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.944 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.944 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:19998 3.b 3:d4c6ec57:::rbd_data.11363a21d2be.00000000000000ae:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T14:59:12.944 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:44.342204+0000", 2024-09-21T14:59:12.944 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 28.555867283000001, 2024-09-21T14:59:12.944 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0063531860000000002, 2024-09-21T14:59:12.944 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:12.944 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:12.945 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:12.945 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:12.945 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:12.945 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 19998 2024-09-21T14:59:12.945 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.945 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:12.945 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.945 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:12.945 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342204+0000", 2024-09-21T14:59:12.945 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:12.945 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.945 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.945 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:12.945 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342204+0000", 2024-09-21T14:59:12.946 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:12.946 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.946 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.946 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:12.946 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342203+0000", 2024-09-21T14:59:12.946 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:12.946 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.946 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.946 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:12.946 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342209+0000", 2024-09-21T14:59:12.946 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1739999999999997e-06 2024-09-21T14:59:12.946 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.946 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.947 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:12.947 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342223+0000", 2024-09-21T14:59:12.947 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4065e-05 2024-09-21T14:59:12.947 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.947 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.947 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:12.947 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342236+0000", 2024-09-21T14:59:12.947 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2537e-05 2024-09-21T14:59:12.947 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.947 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.947 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:12.947 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.348476+0000", 2024-09-21T14:59:12.947 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0062405760000000003 2024-09-21T14:59:12.947 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.948 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.948 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:12.948 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.348557+0000", 2024-09-21T14:59:12.948 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0716000000000004e-05 2024-09-21T14:59:12.948 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:12.948 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:12.948 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:12.948 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.948 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.948 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:19999 3.b 3:d4069488:::rbd_data.11363a21d2be.00000000000000af:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T14:59:12.948 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:44.342246+0000", 2024-09-21T14:59:12.948 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 28.555825118000001, 2024-09-21T14:59:12.948 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0064139210000000004, 2024-09-21T14:59:12.948 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:12.949 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:12.949 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:12.949 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:12.949 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:12.949 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 19999 2024-09-21T14:59:12.949 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.949 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:12.949 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.949 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:12.949 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342246+0000", 2024-09-21T14:59:12.949 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:12.949 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.949 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.949 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:12.950 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342246+0000", 2024-09-21T14:59:12.950 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:12.950 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.950 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.950 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:12.950 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342245+0000", 2024-09-21T14:59:12.950 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:12.950 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.950 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.950 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:12.950 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342251+0000", 2024-09-21T14:59:12.950 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8660000000000002e-06 2024-09-21T14:59:12.950 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.950 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.951 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:12.951 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342264+0000", 2024-09-21T14:59:12.951 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3716999999999999e-05 2024-09-21T14:59:12.951 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.951 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.951 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:12.951 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342275+0000", 2024-09-21T14:59:12.951 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1071e-05 2024-09-21T14:59:12.951 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.951 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.951 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:12.951 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.348581+0000", 2024-09-21T14:59:12.951 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0063053730000000004 2024-09-21T14:59:12.952 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.952 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.952 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:12.952 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.348660+0000", 2024-09-21T14:59:12.952 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8854000000000001e-05 2024-09-21T14:59:12.952 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:12.952 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:12.952 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:12.952 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.952 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.952 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:20440 3.b 3:d6a6f68e:::rbd_object_map.1136c84d9a13:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-21T14:59:12.952 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:45.106428+0000", 2024-09-21T14:59:12.952 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 27.791643272000002, 2024-09-21T14:59:12.952 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0076349679999999998, 2024-09-21T14:59:12.953 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:12.953 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:12.953 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:12.953 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:12.953 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:12.953 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 20440 2024-09-21T14:59:12.953 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.953 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:12.953 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.953 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:12.953 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.106428+0000", 2024-09-21T14:59:12.953 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:12.953 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.954 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.954 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:12.954 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.106428+0000", 2024-09-21T14:59:12.954 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:12.954 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.954 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.954 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:12.954 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.106425+0000", 2024-09-21T14:59:12.954 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-21T14:59:12.954 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.954 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.954 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:12.954 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.106442+0000", 2024-09-21T14:59:12.955 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7237000000000001e-05 2024-09-21T14:59:12.955 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.955 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.955 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:12.955 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.106476+0000", 2024-09-21T14:59:12.955 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4004999999999997e-05 2024-09-21T14:59:12.955 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.955 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.955 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:12.955 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.106491+0000", 2024-09-21T14:59:12.955 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4865000000000001e-05 2024-09-21T14:59:12.955 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.955 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.956 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:12.956 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.106602+0000", 2024-09-21T14:59:12.956 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011113200000000001 2024-09-21T14:59:12.956 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.956 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.956 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op must wait for map", 2024-09-21T14:59:12.956 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.106611+0000", 2024-09-21T14:59:12.956 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9199999999999993e-06 2024-09-21T14:59:12.956 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.956 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.956 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:12.956 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.110475+0000", 2024-09-21T14:59:12.956 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0038633259999999998 2024-09-21T14:59:12.956 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.956 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.957 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:12.957 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.110622+0000", 2024-09-21T14:59:12.957 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014734200000000001 2024-09-21T14:59:12.957 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.957 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.957 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T14:59:12.957 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.111136+0000", 2024-09-21T14:59:12.957 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00051382299999999995 2024-09-21T14:59:12.957 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.957 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.957 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:59:12.957 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.112514+0000", 2024-09-21T14:59:12.957 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001378287 2024-09-21T14:59:12.957 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.958 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.958 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:12.958 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.113830+0000", 2024-09-21T14:59:12.958 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0013157939999999999 2024-09-21T14:59:12.958 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.958 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.958 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:59:12.958 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.113969+0000", 2024-09-21T14:59:12.958 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000138849 2024-09-21T14:59:12.958 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.958 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.958 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:12.958 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.114063+0000", 2024-09-21T14:59:12.959 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3999999999999994e-05 2024-09-21T14:59:12.959 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:12.959 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:12.959 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:12.959 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.959 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.959 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:20460 3.b 3:d6a6f68e:::rbd_object_map.1136c84d9a13:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-21T14:59:12.959 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:45.150183+0000", 2024-09-21T14:59:12.959 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 27.747887454000001, 2024-09-21T14:59:12.959 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0063818800000000004, 2024-09-21T14:59:12.959 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:12.959 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:12.959 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:12.959 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:12.960 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:12.960 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 20460 2024-09-21T14:59:12.960 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.960 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:12.960 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.960 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:12.960 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150183+0000", 2024-09-21T14:59:12.960 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:12.960 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.960 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.960 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:12.960 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150183+0000", 2024-09-21T14:59:12.960 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:12.960 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.961 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.961 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:12.961 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150180+0000", 2024-09-21T14:59:12.961 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999971 2024-09-21T14:59:12.961 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.961 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.961 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:12.961 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150211+0000", 2024-09-21T14:59:12.961 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0993e-05 2024-09-21T14:59:12.961 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.961 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.961 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:12.961 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150241+0000", 2024-09-21T14:59:12.961 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9589000000000001e-05 2024-09-21T14:59:12.962 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.962 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.962 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:12.962 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150261+0000", 2024-09-21T14:59:12.962 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.967e-05 2024-09-21T14:59:12.962 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.962 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.962 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:12.962 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150411+0000", 2024-09-21T14:59:12.962 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015004300000000001 2024-09-21T14:59:12.962 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.962 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.962 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:12.962 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150598+0000", 2024-09-21T14:59:12.963 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018712399999999999 2024-09-21T14:59:12.963 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.963 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.963 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T14:59:12.963 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.151390+0000", 2024-09-21T14:59:12.963 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00079194599999999997 2024-09-21T14:59:12.963 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.963 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.963 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:12.963 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.156302+0000", 2024-09-21T14:59:12.963 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0049123960000000003 2024-09-21T14:59:12.963 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.963 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.964 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:59:12.964 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.156435+0000", 2024-09-21T14:59:12.964 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013287300000000001 2024-09-21T14:59:12.964 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.964 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.964 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:59:12.964 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.156508+0000", 2024-09-21T14:59:12.964 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2504999999999998e-05 2024-09-21T14:59:12.964 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.964 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.964 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:12.964 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.156565+0000", 2024-09-21T14:59:12.964 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7773999999999999e-05 2024-09-21T14:59:12.964 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:12.965 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:12.965 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:12.965 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.965 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.965 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:20461 3.b 3:d6a6f68e:::rbd_object_map.1136c84d9a13:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-21T14:59:12.965 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:45.150284+0000", 2024-09-21T14:59:12.965 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 27.747786633, 2024-09-21T14:59:12.965 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0064029040000000001, 2024-09-21T14:59:12.965 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:12.965 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:12.965 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:12.965 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:12.965 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:12.966 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 20461 2024-09-21T14:59:12.966 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.966 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:12.966 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.966 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:12.966 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150284+0000", 2024-09-21T14:59:12.966 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:12.966 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.966 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.966 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:12.966 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150284+0000", 2024-09-21T14:59:12.966 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:12.966 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.967 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.967 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:12.967 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150283+0000", 2024-09-21T14:59:12.967 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:12.967 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.967 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.967 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:12.967 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150289+0000", 2024-09-21T14:59:12.967 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8799999999999996e-06 2024-09-21T14:59:12.967 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.967 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.967 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:12.967 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150319+0000", 2024-09-21T14:59:12.967 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9828999999999999e-05 2024-09-21T14:59:12.968 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.968 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.968 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:12.968 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150328+0000", 2024-09-21T14:59:12.968 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4770000000000003e-06 2024-09-21T14:59:12.968 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.968 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.968 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:12.968 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.152355+0000", 2024-09-21T14:59:12.968 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0020279870000000002 2024-09-21T14:59:12.968 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.968 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.968 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:12.968 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.152652+0000", 2024-09-21T14:59:12.969 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00029681 2024-09-21T14:59:12.969 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.969 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.969 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T14:59:12.969 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.153318+0000", 2024-09-21T14:59:12.969 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000666101 2024-09-21T14:59:12.969 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.969 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.969 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:12.969 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.156416+0000", 2024-09-21T14:59:12.969 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0030980729999999998 2024-09-21T14:59:12.969 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.969 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.969 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:59:12.970 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.156577+0000", 2024-09-21T14:59:12.970 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016049700000000001 2024-09-21T14:59:12.970 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.970 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.970 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:59:12.970 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.156628+0000", 2024-09-21T14:59:12.970 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0642000000000001e-05 2024-09-21T14:59:12.970 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.970 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.970 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:12.970 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.156687+0000", 2024-09-21T14:59:12.970 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9571000000000003e-05 2024-09-21T14:59:12.970 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:12.971 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:12.971 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:12.971 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.971 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.971 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:20462 3.b 3:d6a6f68e:::rbd_object_map.1136c84d9a13:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-21T14:59:12.971 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:45.150372+0000", 2024-09-21T14:59:12.971 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 27.747698550999999, 2024-09-21T14:59:12.971 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0075518110000000003, 2024-09-21T14:59:12.971 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:12.971 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:12.971 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:12.971 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:12.971 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:12.971 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 20462 2024-09-21T14:59:12.971 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.972 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:12.972 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.972 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:12.972 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150372+0000", 2024-09-21T14:59:12.972 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:12.972 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.972 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.972 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:12.972 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150372+0000", 2024-09-21T14:59:12.972 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:12.972 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.972 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.972 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:12.972 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150372+0000", 2024-09-21T14:59:12.972 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:12.973 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.973 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.973 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:12.973 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150378+0000", 2024-09-21T14:59:12.973 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3330000000000002e-06 2024-09-21T14:59:12.973 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.973 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.973 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:12.973 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150413+0000", 2024-09-21T14:59:12.973 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4798999999999997e-05 2024-09-21T14:59:12.973 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.973 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.973 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:12.974 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150423+0000", 2024-09-21T14:59:12.974 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.8099999999999992e-06 2024-09-21T14:59:12.974 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.974 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.974 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:12.974 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.154550+0000", 2024-09-21T14:59:12.974 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0041275959999999999 2024-09-21T14:59:12.974 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.974 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.974 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:12.974 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.154758+0000", 2024-09-21T14:59:12.974 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00020815499999999999 2024-09-21T14:59:12.974 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.974 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.975 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T14:59:12.975 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.155323+0000", 2024-09-21T14:59:12.975 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00056425400000000004 2024-09-21T14:59:12.975 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.975 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.975 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:59:12.975 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.156702+0000", 2024-09-21T14:59:12.975 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001379676 2024-09-21T14:59:12.975 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.975 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.975 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:12.975 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.157662+0000", 2024-09-21T14:59:12.975 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00095948099999999998 2024-09-21T14:59:12.975 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.975 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.976 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:59:12.976 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.157816+0000", 2024-09-21T14:59:12.976 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000154666 2024-09-21T14:59:12.976 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.976 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.976 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:12.976 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.157924+0000", 2024-09-21T14:59:12.976 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000107798 2024-09-21T14:59:12.976 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:12.976 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:12.976 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:12.976 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.976 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.976 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31476 3.c 3:35718b6b:::rbd_data.11367721c19.0000000000000014:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T14:59:12.977 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.505975+0000", 2024-09-21T14:59:12.977 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.392095914, 2024-09-21T14:59:12.977 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99515995899999998, 2024-09-21T14:59:12.977 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:12.977 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:12.977 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:12.977 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:12.977 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:12.977 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31476 2024-09-21T14:59:12.977 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.977 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:12.977 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.977 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:12.977 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.505975+0000", 2024-09-21T14:59:12.978 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:12.978 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.978 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.978 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:12.978 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.505975+0000", 2024-09-21T14:59:12.978 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:12.978 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.978 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.978 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:12.978 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.505971+0000", 2024-09-21T14:59:12.978 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999967 2024-09-21T14:59:12.978 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.978 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.978 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:12.979 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.505989+0000", 2024-09-21T14:59:12.979 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7985000000000001e-05 2024-09-21T14:59:12.979 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.979 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.979 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:12.979 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506050+0000", 2024-09-21T14:59:12.979 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0659000000000003e-05 2024-09-21T14:59:12.979 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.979 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.979 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:12.979 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506074+0000", 2024-09-21T14:59:12.979 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4156e-05 2024-09-21T14:59:12.979 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.980 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.980 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:12.980 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506225+0000", 2024-09-21T14:59:12.980 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015097 2024-09-21T14:59:12.980 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.980 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.980 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T14:59:12.980 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506256+0000", 2024-09-21T14:59:12.980 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0505999999999999e-05 2024-09-21T14:59:12.980 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.980 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.980 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:12.980 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.497974+0000", 2024-09-21T14:59:12.980 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99171825700000005 2024-09-21T14:59:12.981 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.981 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.981 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for missing object", 2024-09-21T14:59:12.981 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.498449+0000", 2024-09-21T14:59:12.981 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00047506399999999998 2024-09-21T14:59:12.981 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.981 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.981 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:12.981 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.501000+0000", 2024-09-21T14:59:12.981 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0025506919999999998 2024-09-21T14:59:12.981 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.981 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.981 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:12.981 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.501135+0000", 2024-09-21T14:59:12.981 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013524000000000001 2024-09-21T14:59:12.982 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:12.982 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:12.982 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:12.982 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.982 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.982 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31480 3.c 3:36ccd808:::rbd_data.11367721c19.0000000000000018:head [sparse-read 0~14821] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T14:59:12.982 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.506116+0000", 2024-09-21T14:59:12.982 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.391954570999999, 2024-09-21T14:59:12.982 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99316033599999998, 2024-09-21T14:59:12.982 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:12.982 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:12.982 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:12.982 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:12.982 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:12.983 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31480 2024-09-21T14:59:12.983 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.983 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:12.983 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.983 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:12.983 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506116+0000", 2024-09-21T14:59:12.983 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:12.983 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.983 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.983 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:12.983 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506116+0000", 2024-09-21T14:59:12.983 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:12.983 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.984 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.984 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:12.984 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506115+0000", 2024-09-21T14:59:12.984 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:59:12.984 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.984 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.984 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:12.984 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506124+0000", 2024-09-21T14:59:12.984 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0659999999999993e-06 2024-09-21T14:59:12.984 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.984 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.984 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:12.984 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506145+0000", 2024-09-21T14:59:12.984 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1447e-05 2024-09-21T14:59:12.985 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.985 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.985 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:12.985 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506161+0000", 2024-09-21T14:59:12.985 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5209e-05 2024-09-21T14:59:12.985 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.985 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.985 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:12.985 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506337+0000", 2024-09-21T14:59:12.985 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017642999999999999 2024-09-21T14:59:12.985 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.985 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.985 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T14:59:12.985 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506375+0000", 2024-09-21T14:59:12.986 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7536000000000001e-05 2024-09-21T14:59:12.986 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.986 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.986 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:12.986 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.498815+0000", 2024-09-21T14:59:12.986 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99244067999999996 2024-09-21T14:59:12.986 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.986 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.986 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:12.986 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.498950+0000", 2024-09-21T14:59:12.986 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013436700000000001 2024-09-21T14:59:12.986 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.986 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.986 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:12.987 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499277+0000", 2024-09-21T14:59:12.987 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00032696800000000002 2024-09-21T14:59:12.987 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:12.987 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:12.987 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:12.987 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.987 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.987 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31513 3.c 3:36de8523:::rbd_data.11367721c19.0000000000000039:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T14:59:12.987 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.509071+0000", 2024-09-21T14:59:12.987 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.389000371, 2024-09-21T14:59:12.987 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99038225099999999, 2024-09-21T14:59:12.987 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:12.987 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:12.987 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:12.988 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:12.988 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:12.988 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31513 2024-09-21T14:59:12.988 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.988 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:12.988 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.988 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:12.988 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509071+0000", 2024-09-21T14:59:12.988 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:12.988 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.988 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.988 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:12.988 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509071+0000", 2024-09-21T14:59:12.988 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:12.989 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.989 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.989 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:12.989 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509068+0000", 2024-09-21T14:59:12.989 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-21T14:59:12.989 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.989 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.989 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:12.989 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509082+0000", 2024-09-21T14:59:12.989 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3545e-05 2024-09-21T14:59:12.989 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.989 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.989 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:12.989 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509116+0000", 2024-09-21T14:59:12.990 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3942000000000001e-05 2024-09-21T14:59:12.990 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.990 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.990 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:12.990 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509134+0000", 2024-09-21T14:59:12.990 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8839999999999999e-05 2024-09-21T14:59:12.990 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.990 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.990 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:12.990 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509318+0000", 2024-09-21T14:59:12.990 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018326799999999999 2024-09-21T14:59:12.990 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.990 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.990 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T14:59:12.991 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509346+0000", 2024-09-21T14:59:12.991 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8019000000000001e-05 2024-09-21T14:59:12.991 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.991 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.991 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:12.991 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499330+0000", 2024-09-21T14:59:12.991 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.98998399299999995 2024-09-21T14:59:12.991 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.991 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.991 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:12.991 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499453+0000", 2024-09-21T14:59:12.991 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000123115 2024-09-21T14:59:12.991 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:12.991 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:12.992 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:12.992 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.992 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.992 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31515 3.c 3:35b36ddd:::rbd_data.11367721c19.000000000000003b:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T14:59:12.992 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.509177+0000", 2024-09-21T14:59:12.992 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.388893945, 2024-09-21T14:59:12.992 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99044870600000001, 2024-09-21T14:59:12.992 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:12.992 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:12.992 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:12.992 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:12.992 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:12.992 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31515 2024-09-21T14:59:12.992 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.993 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:12.993 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.993 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:12.993 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509177+0000", 2024-09-21T14:59:12.993 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:12.993 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.993 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.993 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:12.993 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509177+0000", 2024-09-21T14:59:12.993 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:12.993 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.993 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.993 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:12.993 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509176+0000", 2024-09-21T14:59:12.994 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:12.994 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.994 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.994 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:12.994 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509183+0000", 2024-09-21T14:59:12.994 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6720000000000004e-06 2024-09-21T14:59:12.994 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.994 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.994 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:12.994 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509269+0000", 2024-09-21T14:59:12.994 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5881e-05 2024-09-21T14:59:12.994 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.994 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.995 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:12.995 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509303+0000", 2024-09-21T14:59:12.995 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3367000000000001e-05 2024-09-21T14:59:12.995 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.995 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.995 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:12.995 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509425+0000", 2024-09-21T14:59:12.995 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000122199 2024-09-21T14:59:12.995 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.995 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.995 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T14:59:12.995 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509448+0000", 2024-09-21T14:59:12.995 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2688999999999999e-05 2024-09-21T14:59:12.995 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.996 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.996 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:12.996 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499501+0000", 2024-09-21T14:59:12.996 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99005306999999998 2024-09-21T14:59:12.996 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.996 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.996 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:12.996 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499626+0000", 2024-09-21T14:59:12.996 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012501400000000001 2024-09-21T14:59:12.996 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:12.996 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:12.996 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:12.996 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.996 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.996 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31531 3.c 3:314f0287:::rbd_data.11367721c19.000000000000004b:head [sparse-read 0~7106] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T14:59:12.997 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.509329+0000", 2024-09-21T14:59:12.997 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.388741925, 2024-09-21T14:59:12.997 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99103085300000004, 2024-09-21T14:59:12.997 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:12.997 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:12.997 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:12.997 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:12.997 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:12.997 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31531 2024-09-21T14:59:12.997 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.997 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:12.997 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.997 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:12.998 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509329+0000", 2024-09-21T14:59:12.998 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:12.998 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.998 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.998 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:12.998 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509329+0000", 2024-09-21T14:59:12.998 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:12.998 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.998 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.998 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:12.998 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509328+0000", 2024-09-21T14:59:12.998 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:59:12.998 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.998 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.999 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:12.999 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509336+0000", 2024-09-21T14:59:12.999 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3170000000000002e-06 2024-09-21T14:59:12.999 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.999 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.999 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:12.999 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509353+0000", 2024-09-21T14:59:12.999 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7214000000000002e-05 2024-09-21T14:59:12.999 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:12.999 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:12.999 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:12.999 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509366+0000", 2024-09-21T14:59:12.999 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.26e-05 2024-09-21T14:59:12.999 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:13.000 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:13.000 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:13.000 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509523+0000", 2024-09-21T14:59:13.000 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015708000000000001 2024-09-21T14:59:13.000 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:13.000 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:13.000 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T14:59:13.000 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509544+0000", 2024-09-21T14:59:13.000 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0783000000000001e-05 2024-09-21T14:59:13.000 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:13.000 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:13.000 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:13.000 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499727+0000", 2024-09-21T14:59:13.000 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99018345200000002 2024-09-21T14:59:13.001 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:13.001 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:13.001 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:13.001 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499949+0000", 2024-09-21T14:59:13.001 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000221358 2024-09-21T14:59:13.001 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:13.001 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:13.001 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:13.001 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.500360+0000", 2024-09-21T14:59:13.001 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00041130700000000002 2024-09-21T14:59:13.001 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:13.001 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:13.001 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:13.001 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:13.002 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:13.002 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31605 3.c 3:34f43f7f:::rbd_data.113661d6691d.0000000000000034:head [delete] snapc 20=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-21T14:59:13.002 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:59.850318+0000", 2024-09-21T14:59:13.002 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 13.047753162999999, 2024-09-21T14:59:13.002 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0069284979999999999, 2024-09-21T14:59:13.002 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:13.002 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:13.002 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:13.002 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:13.002 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:13.002 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31605 2024-09-21T14:59:13.002 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:13.002 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:13.002 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:13.003 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:13.003 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.850318+0000", 2024-09-21T14:59:13.003 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:13.003 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:13.003 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:13.003 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:13.003 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.850318+0000", 2024-09-21T14:59:13.003 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:13.003 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:13.003 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:13.003 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:13.003 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.850314+0000", 2024-09-21T14:59:13.003 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999962 2024-09-21T14:59:13.003 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:13.004 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:13.004 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:13.004 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.850332+0000", 2024-09-21T14:59:13.004 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8414e-05 2024-09-21T14:59:13.004 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:13.004 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:13.004 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:13.004 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.850376+0000", 2024-09-21T14:59:13.004 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3989000000000002e-05 2024-09-21T14:59:13.004 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:13.004 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:13.004 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:13.004 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.850399+0000", 2024-09-21T14:59:13.004 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2455000000000001e-05 2024-09-21T14:59:13.004 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:13.005 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:13.005 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:13.005 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.850537+0000", 2024-09-21T14:59:13.005 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013858100000000001 2024-09-21T14:59:13.005 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:13.005 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:13.005 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:13.005 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.850756+0000", 2024-09-21T14:59:13.005 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021833099999999999 2024-09-21T14:59:13.005 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:13.005 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:13.005 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T14:59:13.005 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.850939+0000", 2024-09-21T14:59:13.006 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018285600000000001 2024-09-21T14:59:13.006 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:13.006 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:13.006 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:59:13.006 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.852297+0000", 2024-09-21T14:59:13.006 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0013588599999999999 2024-09-21T14:59:13.006 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:13.006 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:13.006 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:13.006 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.857067+0000", 2024-09-21T14:59:13.006 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0047691349999999999 2024-09-21T14:59:13.006 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:13.006 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:13.006 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:59:13.007 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.857143+0000", 2024-09-21T14:59:13.007 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6983000000000005e-05 2024-09-21T14:59:13.007 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:13.007 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:13.007 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:13.007 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.857246+0000", 2024-09-21T14:59:13.007 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010275799999999999 2024-09-21T14:59:13.007 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:13.007 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:13.007 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:13.007 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:13.007 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:13.007 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:59:13.009 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:59:13.008+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:59:13.009 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T14:59:13.110 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:59:13.108+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:59:13.136 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:13.136 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:59:13.136 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T14:59:13.136 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:13.136 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T14:59:13.149 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T14:59:13.211 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:59:13.208+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:59:13.251 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:13.251 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:59:13.251 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T14:59:13.251 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T14:59:13.251 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:13.261 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T14:59:13.311 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:59:13.309+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:59:13.363 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:13.363 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T14:59:13.363 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T14:59:13.363 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T14:59:13.363 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.363 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:1473 3.15 3:af2afa6a:::rbd_data.1136515ecb6b.000000000000001b:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T14:59:13.363 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:30.421730+0000", 2024-09-21T14:59:13.363 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 42.940721007999997, 2024-09-21T14:59:13.363 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021478937, 2024-09-21T14:59:13.363 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:13.363 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:13.364 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:13.364 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:13.364 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:13.364 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1473 2024-09-21T14:59:13.364 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.364 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:13.364 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.364 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:13.364 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421730+0000", 2024-09-21T14:59:13.364 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:13.364 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.364 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.364 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:13.364 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421730+0000", 2024-09-21T14:59:13.365 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:13.365 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.365 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.365 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:13.365 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421728+0000", 2024-09-21T14:59:13.365 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-21T14:59:13.365 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.365 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.365 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:13.365 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421739+0000", 2024-09-21T14:59:13.365 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0502e-05 2024-09-21T14:59:13.365 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.365 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.366 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:13.366 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421762+0000", 2024-09-21T14:59:13.366 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3405999999999999e-05 2024-09-21T14:59:13.366 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.366 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.366 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:13.366 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421773+0000", 2024-09-21T14:59:13.366 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0744000000000001e-05 2024-09-21T14:59:13.366 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.366 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.366 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:13.366 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421838+0000", 2024-09-21T14:59:13.366 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5486000000000003e-05 2024-09-21T14:59:13.367 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.367 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.367 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:13.367 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421955+0000", 2024-09-21T14:59:13.367 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000116765 2024-09-21T14:59:13.367 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.367 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.367 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 6", 2024-09-21T14:59:13.367 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.425572+0000", 2024-09-21T14:59:13.367 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0036171380000000002 2024-09-21T14:59:13.367 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.367 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.367 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:13.367 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.442973+0000", 2024-09-21T14:59:13.368 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.017400268 2024-09-21T14:59:13.368 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.368 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.368 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:13.368 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.442996+0000", 2024-09-21T14:59:13.368 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3139e-05 2024-09-21T14:59:13.368 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.368 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.368 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:13.368 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.443079+0000", 2024-09-21T14:59:13.368 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2832999999999998e-05 2024-09-21T14:59:13.368 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.368 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.369 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:13.369 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.443209+0000", 2024-09-21T14:59:13.369 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013052499999999999 2024-09-21T14:59:13.369 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:13.369 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:13.369 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:13.369 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.369 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.369 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:1678 3.c 3:377cc7e1:::rbd_data.1136515ecb6b.000000000000000f:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T14:59:13.369 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:30.988085+0000", 2024-09-21T14:59:13.369 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 42.374366414000001, 2024-09-21T14:59:13.369 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.032974802999999997, 2024-09-21T14:59:13.369 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:13.369 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:13.370 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:13.370 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:13.370 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:13.370 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1678 2024-09-21T14:59:13.370 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.370 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:13.370 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.370 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:13.370 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988085+0000", 2024-09-21T14:59:13.370 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:13.370 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.370 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.370 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:13.371 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988085+0000", 2024-09-21T14:59:13.371 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:13.371 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.371 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.371 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:13.371 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988083+0000", 2024-09-21T14:59:13.371 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-21T14:59:13.371 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.371 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.371 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:13.371 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988093+0000", 2024-09-21T14:59:13.371 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8800000000000003e-06 2024-09-21T14:59:13.371 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.371 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.372 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:13.372 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988113+0000", 2024-09-21T14:59:13.372 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0217999999999999e-05 2024-09-21T14:59:13.372 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.372 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.372 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:13.372 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988123+0000", 2024-09-21T14:59:13.372 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5859999999999994e-06 2024-09-21T14:59:13.372 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.372 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.372 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:13.372 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988203+0000", 2024-09-21T14:59:13.372 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0834000000000006e-05 2024-09-21T14:59:13.372 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.373 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.373 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:13.373 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988376+0000", 2024-09-21T14:59:13.373 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017301299999999999 2024-09-21T14:59:13.373 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.373 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.373 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 7", 2024-09-21T14:59:13.373 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.996338+0000", 2024-09-21T14:59:13.373 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0079618520000000002 2024-09-21T14:59:13.373 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.373 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.373 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:13.373 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.020881+0000", 2024-09-21T14:59:13.374 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024542731000000002 2024-09-21T14:59:13.374 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.374 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.374 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:13.374 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.020904+0000", 2024-09-21T14:59:13.374 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2861e-05 2024-09-21T14:59:13.374 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.374 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.374 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:13.374 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.020987+0000", 2024-09-21T14:59:13.374 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2806000000000006e-05 2024-09-21T14:59:13.374 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.374 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.374 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:13.375 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.021060+0000", 2024-09-21T14:59:13.375 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2871000000000001e-05 2024-09-21T14:59:13.375 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:13.375 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:13.375 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:13.375 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.375 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.375 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9369 3.3 3:c725132c:::rbd_data.11363a21d2be.0000000000000007:10 [read 24576~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:13.375 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157241+0000", 2024-09-21T14:59:13.375 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 34.205209967999998, 2024-09-21T14:59:13.375 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018289156000000001, 2024-09-21T14:59:13.375 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:13.376 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:13.376 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:13.376 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:13.376 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:13.376 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9369 2024-09-21T14:59:13.376 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.376 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:13.376 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.376 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:13.376 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157241+0000", 2024-09-21T14:59:13.376 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:13.376 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.376 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.376 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:13.376 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157241+0000", 2024-09-21T14:59:13.377 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:13.377 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.377 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.377 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:13.377 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157241+0000", 2024-09-21T14:59:13.377 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:13.377 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.377 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.377 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:13.377 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157244+0000", 2024-09-21T14:59:13.377 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5049999999999999e-06 2024-09-21T14:59:13.377 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.377 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.377 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:13.378 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157255+0000", 2024-09-21T14:59:13.378 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0797e-05 2024-09-21T14:59:13.378 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.378 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.378 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:13.378 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157260+0000", 2024-09-21T14:59:13.378 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1749999999999996e-06 2024-09-21T14:59:13.378 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.378 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.378 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:13.378 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175237+0000", 2024-09-21T14:59:13.378 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.017976405000000001 2024-09-21T14:59:13.379 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.379 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.379 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:13.379 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175331+0000", 2024-09-21T14:59:13.379 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4508999999999999e-05 2024-09-21T14:59:13.379 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.379 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.379 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:13.379 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175530+0000", 2024-09-21T14:59:13.380 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00019928200000000001 2024-09-21T14:59:13.380 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:13.380 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:13.380 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:13.380 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.380 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.380 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9370 3.3 3:c725132c:::rbd_data.11363a21d2be.0000000000000007:10 [read 163840~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:13.380 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157279+0000", 2024-09-21T14:59:13.380 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 34.205171790000001, 2024-09-21T14:59:13.380 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018472379000000001, 2024-09-21T14:59:13.380 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:13.381 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:13.381 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:13.381 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:13.381 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:13.381 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9370 2024-09-21T14:59:13.381 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.381 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:13.381 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.381 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:13.381 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157279+0000", 2024-09-21T14:59:13.381 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:13.381 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.381 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.381 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:13.382 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157279+0000", 2024-09-21T14:59:13.382 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:13.382 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.382 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.382 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:13.382 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157279+0000", 2024-09-21T14:59:13.382 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:13.382 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.382 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.382 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:13.382 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157282+0000", 2024-09-21T14:59:13.382 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2229999999999999e-06 2024-09-21T14:59:13.382 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.383 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.383 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:13.383 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157294+0000", 2024-09-21T14:59:13.383 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2173e-05 2024-09-21T14:59:13.383 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.383 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.383 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:13.383 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157300+0000", 2024-09-21T14:59:13.383 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.93e-06 2024-09-21T14:59:13.383 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.383 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.383 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:13.383 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175569+0000", 2024-09-21T14:59:13.384 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018268376999999999 2024-09-21T14:59:13.384 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.384 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.384 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:13.384 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175622+0000", 2024-09-21T14:59:13.384 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3808999999999999e-05 2024-09-21T14:59:13.384 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.384 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.384 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:13.384 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175752+0000", 2024-09-21T14:59:13.384 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012938700000000001 2024-09-21T14:59:13.384 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:13.384 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:13.384 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:13.385 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.385 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.385 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9371 3.3 3:c0bf6a9b:::rbd_data.11363a21d2be.0000000000000000:10 [read 73728~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:13.385 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157306+0000", 2024-09-21T14:59:13.385 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 34.205145678999997, 2024-09-21T14:59:13.385 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018755473000000002, 2024-09-21T14:59:13.385 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:13.385 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:13.385 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:13.385 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:13.385 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:13.385 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9371 2024-09-21T14:59:13.385 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.386 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:13.386 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.386 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:13.386 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157306+0000", 2024-09-21T14:59:13.386 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:13.386 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.386 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.386 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:13.386 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157306+0000", 2024-09-21T14:59:13.386 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:13.386 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.386 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.386 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:13.386 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157305+0000", 2024-09-21T14:59:13.387 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:13.387 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.387 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.387 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:13.387 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157312+0000", 2024-09-21T14:59:13.387 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3289999999999996e-06 2024-09-21T14:59:13.387 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.387 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.387 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:13.387 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157324+0000", 2024-09-21T14:59:13.387 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1725e-05 2024-09-21T14:59:13.387 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.387 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.388 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:13.388 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157339+0000", 2024-09-21T14:59:13.388 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5211999999999999e-05 2024-09-21T14:59:13.388 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.388 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.388 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:13.388 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175798+0000", 2024-09-21T14:59:13.388 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018458781 2024-09-21T14:59:13.388 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.388 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.388 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:13.388 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175899+0000", 2024-09-21T14:59:13.388 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000100784 2024-09-21T14:59:13.389 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.389 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.389 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:13.389 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176061+0000", 2024-09-21T14:59:13.389 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00016216099999999999 2024-09-21T14:59:13.389 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:13.389 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:13.389 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:13.389 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.389 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.389 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9372 3.3 3:c0bf6a9b:::rbd_data.11363a21d2be.0000000000000000:10 [read 212992~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:13.389 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157345+0000", 2024-09-21T14:59:13.389 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 34.205106333000003, 2024-09-21T14:59:13.390 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018926493999999999, 2024-09-21T14:59:13.390 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:13.390 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:13.390 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:13.390 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:13.390 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:13.390 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9372 2024-09-21T14:59:13.390 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.390 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:13.390 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.390 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:13.390 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157345+0000", 2024-09-21T14:59:13.390 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:13.391 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.391 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.391 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:13.391 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157345+0000", 2024-09-21T14:59:13.391 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:13.391 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.391 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.391 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:13.391 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157344+0000", 2024-09-21T14:59:13.391 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:13.391 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.391 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.391 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:13.392 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157348+0000", 2024-09-21T14:59:13.392 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1389999999999999e-06 2024-09-21T14:59:13.392 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.392 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.392 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:13.392 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157372+0000", 2024-09-21T14:59:13.392 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4711000000000001e-05 2024-09-21T14:59:13.392 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.392 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.392 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:13.392 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157378+0000", 2024-09-21T14:59:13.392 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9909999999999999e-06 2024-09-21T14:59:13.392 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.392 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.393 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:13.393 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176090+0000", 2024-09-21T14:59:13.393 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018712111999999999 2024-09-21T14:59:13.393 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.393 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.393 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:13.393 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176134+0000", 2024-09-21T14:59:13.393 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3377999999999997e-05 2024-09-21T14:59:13.393 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.393 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.393 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:13.393 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176271+0000", 2024-09-21T14:59:13.393 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013763499999999999 2024-09-21T14:59:13.393 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:13.394 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:13.394 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:13.394 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.394 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.394 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9373 3.3 3:c0bf6a9b:::rbd_data.11363a21d2be.0000000000000000:10 [read 352256~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:13.394 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157384+0000", 2024-09-21T14:59:13.394 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 34.205067561, 2024-09-21T14:59:13.394 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019154937, 2024-09-21T14:59:13.394 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:13.394 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:13.394 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:13.394 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:13.394 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:13.395 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9373 2024-09-21T14:59:13.395 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.395 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:13.395 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.395 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:13.395 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157384+0000", 2024-09-21T14:59:13.395 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:13.395 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.395 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.395 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:13.395 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157384+0000", 2024-09-21T14:59:13.395 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:13.395 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.396 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.396 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:13.396 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157383+0000", 2024-09-21T14:59:13.396 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:13.396 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.396 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.396 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:13.396 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157386+0000", 2024-09-21T14:59:13.396 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.191e-06 2024-09-21T14:59:13.396 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.396 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.396 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:13.396 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157393+0000", 2024-09-21T14:59:13.397 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5339999999999996e-06 2024-09-21T14:59:13.397 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.397 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.397 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:13.397 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157397+0000", 2024-09-21T14:59:13.397 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5220000000000001e-06 2024-09-21T14:59:13.397 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.397 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.397 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:13.397 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176302+0000", 2024-09-21T14:59:13.397 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018905113000000001 2024-09-21T14:59:13.397 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.397 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.397 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:13.398 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176379+0000", 2024-09-21T14:59:13.398 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6372999999999995e-05 2024-09-21T14:59:13.398 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.398 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.398 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:13.398 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176539+0000", 2024-09-21T14:59:13.398 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015973800000000001 2024-09-21T14:59:13.398 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:13.398 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:13.398 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:13.398 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.398 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.398 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9374 3.3 3:c0bf6a9b:::rbd_data.11363a21d2be.0000000000000000:10 [read 491520~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:13.399 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157401+0000", 2024-09-21T14:59:13.399 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 34.205049776999999, 2024-09-21T14:59:13.399 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019387755, 2024-09-21T14:59:13.399 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:13.399 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:13.399 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:13.399 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:13.399 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:13.399 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9374 2024-09-21T14:59:13.399 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.399 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:13.399 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.399 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:13.399 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157401+0000", 2024-09-21T14:59:13.399 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:13.400 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.400 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.400 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:13.400 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157401+0000", 2024-09-21T14:59:13.400 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:13.400 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.400 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.400 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:13.400 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157401+0000", 2024-09-21T14:59:13.400 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:13.400 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.400 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.400 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:13.400 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157414+0000", 2024-09-21T14:59:13.401 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2417e-05 2024-09-21T14:59:13.401 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.401 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.401 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:13.401 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157424+0000", 2024-09-21T14:59:13.401 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0861e-05 2024-09-21T14:59:13.401 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.401 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.401 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:13.401 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157429+0000", 2024-09-21T14:59:13.401 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0499999999999999e-06 2024-09-21T14:59:13.401 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.401 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.401 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:13.402 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176596+0000", 2024-09-21T14:59:13.402 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019166552 2024-09-21T14:59:13.402 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.402 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.402 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:13.402 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176647+0000", 2024-09-21T14:59:13.402 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1112000000000001e-05 2024-09-21T14:59:13.402 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.402 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.402 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:13.402 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176789+0000", 2024-09-21T14:59:13.402 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014209300000000001 2024-09-21T14:59:13.403 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:13.403 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:13.403 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:13.403 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.403 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.403 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9401 3.3 3:c725132c:::rbd_data.11363a21d2be.0000000000000007:10 [read 32768~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:13.403 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157621+0000", 2024-09-21T14:59:13.403 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 34.204829967999999, 2024-09-21T14:59:13.403 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019381711999999999, 2024-09-21T14:59:13.403 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:13.403 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:13.403 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:13.403 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:13.403 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:13.403 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9401 2024-09-21T14:59:13.404 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.404 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:13.404 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.404 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:13.404 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157621+0000", 2024-09-21T14:59:13.404 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:13.404 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.404 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.404 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:13.404 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157621+0000", 2024-09-21T14:59:13.404 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:13.404 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.404 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.405 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:13.405 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157620+0000", 2024-09-21T14:59:13.405 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:59:13.405 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.405 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.405 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:13.405 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157626+0000", 2024-09-21T14:59:13.405 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7570000000000001e-06 2024-09-21T14:59:13.405 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.405 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.405 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:13.405 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157636+0000", 2024-09-21T14:59:13.405 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8339999999999994e-06 2024-09-21T14:59:13.405 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.405 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.406 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:13.406 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157643+0000", 2024-09-21T14:59:13.406 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9439999999999999e-06 2024-09-21T14:59:13.406 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.406 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.406 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:13.406 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176816+0000", 2024-09-21T14:59:13.406 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019172872000000001 2024-09-21T14:59:13.406 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.406 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.406 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:13.406 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176878+0000", 2024-09-21T14:59:13.406 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1734000000000002e-05 2024-09-21T14:59:13.407 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.407 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.407 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:13.407 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177003+0000", 2024-09-21T14:59:13.407 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012537400000000001 2024-09-21T14:59:13.407 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:13.407 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:13.407 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:13.407 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.407 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.407 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9402 3.3 3:c725132c:::rbd_data.11363a21d2be.0000000000000007:10 [read 172032~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:13.407 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157649+0000", 2024-09-21T14:59:13.407 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 34.204802096000002, 2024-09-21T14:59:13.407 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019567404, 2024-09-21T14:59:13.408 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:13.408 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:13.408 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:13.408 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:13.408 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:13.408 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9402 2024-09-21T14:59:13.408 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.408 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:13.408 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.408 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:13.408 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157649+0000", 2024-09-21T14:59:13.408 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:13.408 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.408 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.409 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:13.409 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157649+0000", 2024-09-21T14:59:13.409 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:13.409 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.409 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.409 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:13.409 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157649+0000", 2024-09-21T14:59:13.409 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:13.409 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.409 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.409 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:13.409 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157652+0000", 2024-09-21T14:59:13.409 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8779999999999998e-06 2024-09-21T14:59:13.410 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.410 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.410 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:13.410 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157657+0000", 2024-09-21T14:59:13.410 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6659999999999996e-06 2024-09-21T14:59:13.410 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.410 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.410 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:13.410 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157662+0000", 2024-09-21T14:59:13.410 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5519999999999998e-06 2024-09-21T14:59:13.410 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.410 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.410 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:13.410 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177040+0000", 2024-09-21T14:59:13.411 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019377807 2024-09-21T14:59:13.411 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.411 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.411 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:13.411 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177085+0000", 2024-09-21T14:59:13.411 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5195999999999999e-05 2024-09-21T14:59:13.411 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.411 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.411 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:13.411 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177217+0000", 2024-09-21T14:59:13.411 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013172600000000001 2024-09-21T14:59:13.411 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:13.411 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:13.412 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:13.412 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.412 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.412 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9403 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:13.412 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157949+0000", 2024-09-21T14:59:13.412 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 34.204501766999996, 2024-09-21T14:59:13.412 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023540373999999999, 2024-09-21T14:59:13.412 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:13.412 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:13.413 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:13.413 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:13.413 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:13.413 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9403 2024-09-21T14:59:13.413 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.413 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:13.413 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.414 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:13.414 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157949+0000", 2024-09-21T14:59:13.414 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:13.414 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.414 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.414 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:13.414 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157949+0000", 2024-09-21T14:59:13.414 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:13.414 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.414 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.414 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:13.414 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157948+0000", 2024-09-21T14:59:13.414 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T14:59:13.414 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.415 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.415 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:13.415 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157956+0000", 2024-09-21T14:59:13.415 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7999999999999999e-06 2024-09-21T14:59:13.415 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.415 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.415 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:13.415 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157969+0000", 2024-09-21T14:59:13.415 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2792000000000001e-05 2024-09-21T14:59:13.415 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.415 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.415 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:13.415 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157977+0000", 2024-09-21T14:59:13.416 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.704e-06 2024-09-21T14:59:13.416 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.416 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.416 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:13.416 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177246+0000", 2024-09-21T14:59:13.416 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019268264 2024-09-21T14:59:13.416 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.416 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.416 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:13.416 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177315+0000", 2024-09-21T14:59:13.416 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9480999999999994e-05 2024-09-21T14:59:13.416 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.416 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.416 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:59:13.417 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177615+0000", 2024-09-21T14:59:13.417 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00029929499999999999 2024-09-21T14:59:13.417 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.417 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.417 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:13.417 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181333+0000", 2024-09-21T14:59:13.417 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0037183149999999998 2024-09-21T14:59:13.417 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.417 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.417 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:13.417 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181374+0000", 2024-09-21T14:59:13.417 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.0723999999999998e-05 2024-09-21T14:59:13.417 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.417 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.418 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:13.418 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181434+0000", 2024-09-21T14:59:13.418 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0024e-05 2024-09-21T14:59:13.418 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.418 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.418 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:13.418 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181490+0000", 2024-09-21T14:59:13.418 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6245000000000001e-05 2024-09-21T14:59:13.418 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:13.418 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:13.418 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:13.418 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.418 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.419 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9404 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:13.419 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.160480+0000", 2024-09-21T14:59:13.419 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 34.201971417000003, 2024-09-21T14:59:13.419 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021137686999999999, 2024-09-21T14:59:13.419 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:13.419 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:13.419 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:13.419 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:13.419 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:13.419 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9404 2024-09-21T14:59:13.419 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.419 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:13.419 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.420 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:13.420 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160480+0000", 2024-09-21T14:59:13.420 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:13.420 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.420 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.420 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:13.420 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160480+0000", 2024-09-21T14:59:13.420 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:13.420 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.420 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.420 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:13.420 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160478+0000", 2024-09-21T14:59:13.421 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T14:59:13.421 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.421 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.421 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:13.421 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160486+0000", 2024-09-21T14:59:13.421 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9969999999999998e-06 2024-09-21T14:59:13.421 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.421 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.421 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:13.421 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160502+0000", 2024-09-21T14:59:13.421 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5650000000000001e-05 2024-09-21T14:59:13.421 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.421 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.421 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:13.422 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160511+0000", 2024-09-21T14:59:13.422 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.3610000000000007e-06 2024-09-21T14:59:13.422 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.422 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.422 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:13.422 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.178035+0000", 2024-09-21T14:59:13.423 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.017523520000000001 2024-09-21T14:59:13.423 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.423 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.423 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:13.423 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.178151+0000", 2024-09-21T14:59:13.423 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000116013 2024-09-21T14:59:13.424 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.424 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.424 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:59:13.424 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.178408+0000", 2024-09-21T14:59:13.424 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00025743400000000002 2024-09-21T14:59:13.424 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.424 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.424 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:13.425 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181501+0000", 2024-09-21T14:59:13.425 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.003092891 2024-09-21T14:59:13.425 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.425 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.425 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:13.425 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181538+0000", 2024-09-21T14:59:13.425 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6977999999999997e-05 2024-09-21T14:59:13.425 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.426 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.426 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:13.426 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181582+0000", 2024-09-21T14:59:13.426 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3849e-05 2024-09-21T14:59:13.426 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.426 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.426 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:13.427 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181617+0000", 2024-09-21T14:59:13.427 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5423000000000003e-05 2024-09-21T14:59:13.427 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:13.427 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:13.427 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:13.427 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.427 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.428 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9406 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:13.428 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.166040+0000", 2024-09-21T14:59:13.428 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 34.196411538, 2024-09-21T14:59:13.428 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018094524000000001, 2024-09-21T14:59:13.428 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:13.428 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:13.428 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:13.429 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:13.429 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:13.429 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9406 2024-09-21T14:59:13.429 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.429 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:13.429 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.429 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:13.429 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.166040+0000", 2024-09-21T14:59:13.430 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:13.430 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.430 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.430 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:13.430 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.166040+0000", 2024-09-21T14:59:13.430 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:13.430 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.430 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.430 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:13.430 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.166038+0000", 2024-09-21T14:59:13.430 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T14:59:13.430 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.431 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.431 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:13.431 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.166051+0000", 2024-09-21T14:59:13.431 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2638e-05 2024-09-21T14:59:13.431 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.431 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.431 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:13.431 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.166070+0000", 2024-09-21T14:59:13.431 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8691000000000001e-05 2024-09-21T14:59:13.431 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.431 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.431 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:13.431 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.166089+0000", 2024-09-21T14:59:13.431 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9286000000000001e-05 2024-09-21T14:59:13.432 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.432 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.432 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:13.432 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.180194+0000", 2024-09-21T14:59:13.432 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014105039999999999 2024-09-21T14:59:13.432 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.432 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.432 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:13.432 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.180340+0000", 2024-09-21T14:59:13.432 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014623199999999999 2024-09-21T14:59:13.432 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.432 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.432 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:59:13.432 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.180818+0000", 2024-09-21T14:59:13.433 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000477489 2024-09-21T14:59:13.433 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.433 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.433 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:13.433 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.183808+0000", 2024-09-21T14:59:13.433 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0029897610000000001 2024-09-21T14:59:13.433 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.433 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.433 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:13.433 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.183821+0000", 2024-09-21T14:59:13.433 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3251999999999999e-05 2024-09-21T14:59:13.433 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.433 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.433 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:13.434 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.183872+0000", 2024-09-21T14:59:13.434 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0900999999999999e-05 2024-09-21T14:59:13.434 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.434 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.434 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:13.434 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.184134+0000", 2024-09-21T14:59:13.434 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000262484 2024-09-21T14:59:13.434 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:13.434 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:13.434 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:13.434 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.434 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.434 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9407 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:13.434 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.168094+0000", 2024-09-21T14:59:13.435 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 34.194356917999997, 2024-09-21T14:59:13.435 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019444860000000001, 2024-09-21T14:59:13.435 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:13.435 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:13.435 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:13.435 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:13.435 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:13.435 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9407 2024-09-21T14:59:13.435 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.435 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:13.435 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.435 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:13.435 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.168094+0000", 2024-09-21T14:59:13.436 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:13.436 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.436 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.436 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:13.436 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.168094+0000", 2024-09-21T14:59:13.436 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:13.436 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.436 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.436 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:13.436 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.168093+0000", 2024-09-21T14:59:13.436 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:59:13.436 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.437 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.437 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:13.437 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.168119+0000", 2024-09-21T14:59:13.437 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5803999999999999e-05 2024-09-21T14:59:13.437 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.437 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.437 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:13.437 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.168135+0000", 2024-09-21T14:59:13.437 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6359999999999999e-05 2024-09-21T14:59:13.437 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.437 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.437 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:13.438 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.168144+0000", 2024-09-21T14:59:13.438 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4270000000000008e-06 2024-09-21T14:59:13.438 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.438 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.438 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:13.438 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181943+0000", 2024-09-21T14:59:13.438 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013799617 2024-09-21T14:59:13.438 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.438 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.438 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:13.438 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.182052+0000", 2024-09-21T14:59:13.438 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000108924 2024-09-21T14:59:13.438 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.439 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.439 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:59:13.439 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.183431+0000", 2024-09-21T14:59:13.439 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0013783619999999999 2024-09-21T14:59:13.439 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.439 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.439 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:13.439 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.187276+0000", 2024-09-21T14:59:13.439 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0038450469999999999 2024-09-21T14:59:13.439 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.439 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.439 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:13.439 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.187288+0000", 2024-09-21T14:59:13.440 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2414e-05 2024-09-21T14:59:13.440 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.440 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.440 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:13.440 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.187414+0000", 2024-09-21T14:59:13.440 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012614399999999999 2024-09-21T14:59:13.440 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.440 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.440 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:13.440 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.187539+0000", 2024-09-21T14:59:13.440 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012481800000000001 2024-09-21T14:59:13.440 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:13.440 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:13.441 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:13.441 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.441 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.441 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9410 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:13.441 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.174103+0000", 2024-09-21T14:59:13.441 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 34.188348009000002, 2024-09-21T14:59:13.441 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025493703, 2024-09-21T14:59:13.441 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:13.441 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:13.441 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:13.441 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:13.441 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:13.442 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9410 2024-09-21T14:59:13.442 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.442 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:13.442 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.442 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:13.442 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.174103+0000", 2024-09-21T14:59:13.442 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:13.442 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.442 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.442 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:13.442 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.174103+0000", 2024-09-21T14:59:13.442 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:13.443 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.443 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.443 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:13.443 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.174102+0000", 2024-09-21T14:59:13.443 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:59:13.443 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.443 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.443 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:13.443 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.174108+0000", 2024-09-21T14:59:13.443 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1149999999999999e-06 2024-09-21T14:59:13.443 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.443 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.443 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:13.444 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.174117+0000", 2024-09-21T14:59:13.444 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4870000000000002e-06 2024-09-21T14:59:13.444 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.445 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.445 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:13.445 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.174122+0000", 2024-09-21T14:59:13.446 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9790000000000001e-06 2024-09-21T14:59:13.446 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.446 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.446 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:13.446 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.187830+0000", 2024-09-21T14:59:13.446 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013708048 2024-09-21T14:59:13.447 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.447 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.447 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:13.447 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.187913+0000", 2024-09-21T14:59:13.447 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2763999999999996e-05 2024-09-21T14:59:13.447 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.447 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.447 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:59:13.447 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.188230+0000", 2024-09-21T14:59:13.448 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00031745100000000001 2024-09-21T14:59:13.448 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.448 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.448 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:13.448 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.192708+0000", 2024-09-21T14:59:13.448 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0044783890000000002 2024-09-21T14:59:13.448 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.448 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.448 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:13.448 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199487+0000", 2024-09-21T14:59:13.448 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0067782149999999998 2024-09-21T14:59:13.448 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.448 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.448 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:13.449 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199560+0000", 2024-09-21T14:59:13.449 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3752999999999997e-05 2024-09-21T14:59:13.449 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.449 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.449 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:13.449 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199597+0000", 2024-09-21T14:59:13.449 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6526999999999998e-05 2024-09-21T14:59:13.449 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:13.449 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:13.449 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:13.449 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.449 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.450 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9411 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:13.450 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.175943+0000", 2024-09-21T14:59:13.450 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 34.186507990000003, 2024-09-21T14:59:13.450 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023773181000000001, 2024-09-21T14:59:13.450 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:13.450 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:13.450 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:13.450 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:13.450 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:13.450 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9411 2024-09-21T14:59:13.450 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.450 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:13.450 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.451 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:13.451 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175943+0000", 2024-09-21T14:59:13.451 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:13.451 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.451 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.451 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:13.451 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175943+0000", 2024-09-21T14:59:13.451 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:13.451 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.451 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.451 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:13.451 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175942+0000", 2024-09-21T14:59:13.451 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:59:13.452 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.452 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.452 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:13.452 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175948+0000", 2024-09-21T14:59:13.452 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9689999999999999e-06 2024-09-21T14:59:13.452 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.452 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.452 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:13.452 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175960+0000", 2024-09-21T14:59:13.452 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1745e-05 2024-09-21T14:59:13.452 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.452 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.452 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:13.452 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175965+0000", 2024-09-21T14:59:13.453 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.203e-06 2024-09-21T14:59:13.453 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.453 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.453 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:13.453 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.189900+0000", 2024-09-21T14:59:13.453 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013934940999999999 2024-09-21T14:59:13.453 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.453 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.453 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:13.453 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.189988+0000", 2024-09-21T14:59:13.453 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8419999999999997e-05 2024-09-21T14:59:13.453 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.453 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.453 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:59:13.454 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.190321+0000", 2024-09-21T14:59:13.454 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00033231800000000001 2024-09-21T14:59:13.454 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.454 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.454 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:13.454 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.192746+0000", 2024-09-21T14:59:13.454 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0024254400000000001 2024-09-21T14:59:13.454 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.454 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.454 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:13.454 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199605+0000", 2024-09-21T14:59:13.454 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0068582809999999999 2024-09-21T14:59:13.455 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.455 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.455 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:13.455 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199680+0000", 2024-09-21T14:59:13.455 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5630000000000006e-05 2024-09-21T14:59:13.455 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.455 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.455 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:13.455 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199716+0000", 2024-09-21T14:59:13.455 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6248999999999998e-05 2024-09-21T14:59:13.455 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:13.455 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:13.455 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:13.455 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.456 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.456 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9412 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:13.456 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.177402+0000", 2024-09-21T14:59:13.456 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 34.185048893000001, 2024-09-21T14:59:13.456 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022445590000000001, 2024-09-21T14:59:13.456 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:13.456 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:13.456 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:13.456 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:13.456 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:13.456 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9412 2024-09-21T14:59:13.456 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.456 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:13.456 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.457 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:13.457 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177402+0000", 2024-09-21T14:59:13.457 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:13.457 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.457 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.457 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:13.457 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177402+0000", 2024-09-21T14:59:13.457 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:13.457 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.457 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.457 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:13.457 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177401+0000", 2024-09-21T14:59:13.457 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:59:13.457 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.458 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.458 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:13.458 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177409+0000", 2024-09-21T14:59:13.458 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4710000000000001e-06 2024-09-21T14:59:13.458 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.458 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.458 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:13.458 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177419+0000", 2024-09-21T14:59:13.458 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0351e-05 2024-09-21T14:59:13.458 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.458 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.458 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:13.458 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177424+0000", 2024-09-21T14:59:13.459 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1919999999999996e-06 2024-09-21T14:59:13.459 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.459 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.459 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:13.459 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.191343+0000", 2024-09-21T14:59:13.459 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.01391878 2024-09-21T14:59:13.459 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.459 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.459 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:13.459 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.191446+0000", 2024-09-21T14:59:13.459 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000102453 2024-09-21T14:59:13.459 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.459 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.459 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:59:13.460 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.192200+0000", 2024-09-21T14:59:13.460 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00075476899999999997 2024-09-21T14:59:13.460 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.460 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.460 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:13.460 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.193866+0000", 2024-09-21T14:59:13.460 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0016657519999999999 2024-09-21T14:59:13.460 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.460 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.460 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:13.460 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199724+0000", 2024-09-21T14:59:13.460 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0058579529999999999 2024-09-21T14:59:13.460 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.460 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.461 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:13.461 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199803+0000", 2024-09-21T14:59:13.461 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8662e-05 2024-09-21T14:59:13.461 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.461 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.461 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:13.461 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199848+0000", 2024-09-21T14:59:13.461 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5216000000000002e-05 2024-09-21T14:59:13.461 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:13.461 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:13.461 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:13.461 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.461 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.461 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:10445 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:13.462 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.271980+0000", 2024-09-21T14:59:13.462 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 34.090471528000002, 2024-09-21T14:59:13.462 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018664256000000001, 2024-09-21T14:59:13.462 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:13.462 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:13.462 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:13.462 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:13.462 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:13.462 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 10445 2024-09-21T14:59:13.462 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.462 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:13.462 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.462 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:13.463 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.271980+0000", 2024-09-21T14:59:13.463 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:13.463 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.463 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.463 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:13.463 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.271980+0000", 2024-09-21T14:59:13.463 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:13.463 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.463 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.463 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:13.463 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.271979+0000", 2024-09-21T14:59:13.463 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:13.463 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.463 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.464 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:13.464 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.271982+0000", 2024-09-21T14:59:13.464 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7240000000000001e-06 2024-09-21T14:59:13.464 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.464 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.464 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:13.464 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.271987+0000", 2024-09-21T14:59:13.464 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3639999999999998e-06 2024-09-21T14:59:13.464 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.464 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.464 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:13.464 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.271992+0000", 2024-09-21T14:59:13.464 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7110000000000003e-06 2024-09-21T14:59:13.465 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.465 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.465 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:13.465 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.277567+0000", 2024-09-21T14:59:13.465 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0055745009999999999 2024-09-21T14:59:13.465 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.465 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.465 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:13.465 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.277655+0000", 2024-09-21T14:59:13.465 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8465000000000001e-05 2024-09-21T14:59:13.465 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.465 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.465 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:59:13.466 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.278720+0000", 2024-09-21T14:59:13.466 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0010650569999999999 2024-09-21T14:59:13.466 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.466 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.466 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:13.466 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.281153+0000", 2024-09-21T14:59:13.466 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0024326389999999999 2024-09-21T14:59:13.466 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.466 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.466 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:13.466 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.290574+0000", 2024-09-21T14:59:13.466 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0094209489999999996 2024-09-21T14:59:13.466 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.466 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.467 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:13.467 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.290613+0000", 2024-09-21T14:59:13.467 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.8803000000000001e-05 2024-09-21T14:59:13.467 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.467 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.467 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:13.467 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.290644+0000", 2024-09-21T14:59:13.467 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1404e-05 2024-09-21T14:59:13.467 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:13.467 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:13.467 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:13.467 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.467 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.467 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22769 3.f 3:f39bd56f:::rbd_data.11366acc894.0000000000000004:19 [read 1032192~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:13.468 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.140207+0000", 2024-09-21T14:59:13.468 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 23.222243788, 2024-09-21T14:59:13.468 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018031170999999999, 2024-09-21T14:59:13.468 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:13.468 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:13.468 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:13.468 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:13.468 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:13.468 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22769 2024-09-21T14:59:13.468 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.468 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:13.468 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.468 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:13.468 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.140207+0000", 2024-09-21T14:59:13.469 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:13.469 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.469 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.469 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:13.469 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.140207+0000", 2024-09-21T14:59:13.469 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:13.469 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.469 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.469 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:13.469 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.140207+0000", 2024-09-21T14:59:13.469 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:13.469 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.469 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.469 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:13.470 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.140209+0000", 2024-09-21T14:59:13.470 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.305e-06 2024-09-21T14:59:13.470 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.470 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.470 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:13.470 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.140214+0000", 2024-09-21T14:59:13.470 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9570000000000001e-06 2024-09-21T14:59:13.470 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.470 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.470 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:13.470 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.140219+0000", 2024-09-21T14:59:13.470 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2250000000000002e-06 2024-09-21T14:59:13.471 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.471 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.471 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:13.471 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.157990+0000", 2024-09-21T14:59:13.471 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.017770939999999999 2024-09-21T14:59:13.471 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.471 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.471 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:13.471 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.158062+0000", 2024-09-21T14:59:13.471 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2652999999999997e-05 2024-09-21T14:59:13.471 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.471 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.471 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:13.472 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.158239+0000", 2024-09-21T14:59:13.472 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017640599999999999 2024-09-21T14:59:13.472 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:13.472 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:13.472 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:13.472 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.472 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.472 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:23084 3.3 3:c55e343c:::rbd_data.11366acc894.0000000000000006:19 [read 1032192~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:13.472 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.150724+0000", 2024-09-21T14:59:13.472 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 23.211726952999999, 2024-09-21T14:59:13.472 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018916460999999999, 2024-09-21T14:59:13.472 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:13.472 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:13.473 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:13.473 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:13.473 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:13.473 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 23084 2024-09-21T14:59:13.473 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.473 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:13.473 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.473 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:13.473 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.150724+0000", 2024-09-21T14:59:13.473 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:13.473 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.473 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.473 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:13.473 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.150724+0000", 2024-09-21T14:59:13.474 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:13.474 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.474 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.474 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:13.474 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.150724+0000", 2024-09-21T14:59:13.474 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:13.474 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.474 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.474 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:13.474 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.150727+0000", 2024-09-21T14:59:13.474 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7250000000000002e-06 2024-09-21T14:59:13.474 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.474 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.475 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:13.475 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.150732+0000", 2024-09-21T14:59:13.475 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5809999999999996e-06 2024-09-21T14:59:13.475 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.475 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.475 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:13.475 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.150737+0000", 2024-09-21T14:59:13.475 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3590000000000001e-06 2024-09-21T14:59:13.475 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.475 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.475 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:13.475 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.166694+0000", 2024-09-21T14:59:13.475 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.015957720000000002 2024-09-21T14:59:13.475 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.476 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.476 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:13.476 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.166769+0000", 2024-09-21T14:59:13.476 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5143999999999997e-05 2024-09-21T14:59:13.476 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.476 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.476 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:13.476 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.169641+0000", 2024-09-21T14:59:13.476 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0028713089999999998 2024-09-21T14:59:13.476 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:13.476 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:13.476 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:13.476 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:13.477 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:13.477 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:13.478 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T14:59:13.478 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:59:13.477+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:59:13.578 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:59:13.576+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:59:13.631 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:13.631 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:59:13.631 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T14:59:13.631 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:13.641 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T14:59:13.679 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:59:13.677+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:59:13.745 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:13.745 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:59:13.745 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T14:59:13.745 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T14:59:13.745 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:13.755 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T14:59:13.804 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:59:13.805+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:59:13.866 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:13.866 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T14:59:13.867 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T14:59:13.867 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T14:59:13.867 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.867 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T14:59:13.867 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.205754+0000", 2024-09-21T14:59:13.867 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 70.659177303000007, 2024-09-21T14:59:13.867 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.41117094199999998, 2024-09-21T14:59:13.867 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:13.867 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:13.867 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:13.867 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:59:13.867 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:59:13.867 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1 2024-09-21T14:59:13.868 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.868 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:13.868 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.868 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:13.868 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205754+0000", 2024-09-21T14:59:13.868 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:13.868 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.868 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.868 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:13.868 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205754+0000", 2024-09-21T14:59:13.868 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:13.868 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.868 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.868 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:13.869 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205752+0000", 2024-09-21T14:59:13.869 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T14:59:13.869 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.869 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.869 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:13.869 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205765+0000", 2024-09-21T14:59:13.869 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3230999999999999e-05 2024-09-21T14:59:13.869 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.869 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.869 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:13.869 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205782+0000", 2024-09-21T14:59:13.869 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6634e-05 2024-09-21T14:59:13.869 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.869 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.870 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:13.870 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205793+0000", 2024-09-21T14:59:13.870 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1297e-05 2024-09-21T14:59:13.870 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.870 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.870 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:13.870 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205867+0000", 2024-09-21T14:59:13.870 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4269000000000003e-05 2024-09-21T14:59:13.870 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.870 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.870 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T14:59:13.870 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205877+0000", 2024-09-21T14:59:13.870 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2690000000000005e-06 2024-09-21T14:59:13.870 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.871 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.871 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:13.871 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.613700+0000", 2024-09-21T14:59:13.871 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.407823455 2024-09-21T14:59:13.871 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.871 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.871 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:13.871 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.614203+0000", 2024-09-21T14:59:13.871 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00050311500000000005 2024-09-21T14:59:13.871 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.871 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.871 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:59:13.871 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.614750+0000", 2024-09-21T14:59:13.871 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00054738699999999998 2024-09-21T14:59:13.872 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.872 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.872 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:13.872 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616367+0000", 2024-09-21T14:59:13.872 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001616634 2024-09-21T14:59:13.872 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.872 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.872 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:13.872 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616574+0000", 2024-09-21T14:59:13.872 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00020668499999999999 2024-09-21T14:59:13.872 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.872 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.872 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:13.872 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616802+0000", 2024-09-21T14:59:13.872 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00022817900000000001 2024-09-21T14:59:13.873 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.873 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.873 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:13.873 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616925+0000", 2024-09-21T14:59:13.873 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012309699999999999 2024-09-21T14:59:13.873 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:13.873 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:13.873 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:13.873 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.873 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.873 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T14:59:13.873 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.955142+0000", 2024-09-21T14:59:13.873 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 69.909789656000001, 2024-09-21T14:59:13.873 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012388923, 2024-09-21T14:59:13.874 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:13.874 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:13.874 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:13.874 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:59:13.874 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:59:13.874 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 113 2024-09-21T14:59:13.874 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.874 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:13.874 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.874 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:13.874 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955142+0000", 2024-09-21T14:59:13.874 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:13.874 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.874 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.875 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:13.875 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955142+0000", 2024-09-21T14:59:13.875 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:13.875 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.875 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.875 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:13.875 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955141+0000", 2024-09-21T14:59:13.875 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:13.875 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.875 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.875 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:13.875 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955145+0000", 2024-09-21T14:59:13.875 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2109999999999998e-06 2024-09-21T14:59:13.875 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.876 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.876 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:13.876 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955150+0000", 2024-09-21T14:59:13.876 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4090000000000002e-06 2024-09-21T14:59:13.876 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.876 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.876 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:13.876 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955153+0000", 2024-09-21T14:59:13.876 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1240000000000001e-06 2024-09-21T14:59:13.876 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.876 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.876 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:13.876 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.963026+0000", 2024-09-21T14:59:13.876 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0078724060000000002 2024-09-21T14:59:13.877 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.877 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.877 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:13.877 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.963126+0000", 2024-09-21T14:59:13.877 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000100206 2024-09-21T14:59:13.877 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.877 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.877 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:59:13.877 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.963277+0000", 2024-09-21T14:59:13.877 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015159200000000001 2024-09-21T14:59:13.877 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.877 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.877 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:13.877 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.965519+0000", 2024-09-21T14:59:13.877 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0022418440000000002 2024-09-21T14:59:13.878 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.878 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.878 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:13.878 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967305+0000", 2024-09-21T14:59:13.878 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001785525 2024-09-21T14:59:13.878 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.878 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.878 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:13.878 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967438+0000", 2024-09-21T14:59:13.878 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013284399999999999 2024-09-21T14:59:13.878 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.878 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.878 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:13.878 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967531+0000", 2024-09-21T14:59:13.878 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.3140000000000006e-05 2024-09-21T14:59:13.879 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:13.879 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:13.879 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:13.879 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.879 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.879 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T14:59:13.879 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.955387+0000", 2024-09-21T14:59:13.879 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 69.909544722000007, 2024-09-21T14:59:13.879 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012319478999999999, 2024-09-21T14:59:13.879 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:13.879 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:13.879 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:13.879 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:59:13.879 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:59:13.880 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 114 2024-09-21T14:59:13.880 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.880 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:13.880 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.880 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:13.880 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955387+0000", 2024-09-21T14:59:13.880 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:13.880 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.880 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.880 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:13.880 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955387+0000", 2024-09-21T14:59:13.880 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:13.880 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.880 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.880 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:13.880 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955386+0000", 2024-09-21T14:59:13.881 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:59:13.881 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.881 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.881 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:13.881 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955392+0000", 2024-09-21T14:59:13.881 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4740000000000002e-06 2024-09-21T14:59:13.881 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.881 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.881 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:13.881 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955400+0000", 2024-09-21T14:59:13.881 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2139999999999996e-06 2024-09-21T14:59:13.881 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.881 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.881 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:13.881 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955407+0000", 2024-09-21T14:59:13.882 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2299999999999996e-06 2024-09-21T14:59:13.882 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.882 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.882 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:13.882 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.964840+0000", 2024-09-21T14:59:13.882 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0094330079999999997 2024-09-21T14:59:13.882 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.882 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.882 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:13.882 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.964917+0000", 2024-09-21T14:59:13.882 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7071999999999997e-05 2024-09-21T14:59:13.882 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.882 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.882 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:59:13.883 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.965026+0000", 2024-09-21T14:59:13.883 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000109646 2024-09-21T14:59:13.883 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.883 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.883 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:13.883 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967609+0000", 2024-09-21T14:59:13.883 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0025824929999999999 2024-09-21T14:59:13.883 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.883 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.883 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:13.883 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967629+0000", 2024-09-21T14:59:13.883 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0486999999999999e-05 2024-09-21T14:59:13.883 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.883 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.883 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:13.884 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967672+0000", 2024-09-21T14:59:13.884 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2314999999999997e-05 2024-09-21T14:59:13.884 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.884 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.884 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:13.884 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967706+0000", 2024-09-21T14:59:13.884 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4483999999999997e-05 2024-09-21T14:59:13.884 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:13.884 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:13.884 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:13.884 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.884 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.884 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T14:59:13.884 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.955609+0000", 2024-09-21T14:59:13.884 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 69.909322915999994, 2024-09-21T14:59:13.885 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013624153999999999, 2024-09-21T14:59:13.885 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:13.885 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:13.885 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:13.885 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:59:13.885 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:59:13.885 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 115 2024-09-21T14:59:13.885 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.885 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:13.885 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.885 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:13.885 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955609+0000", 2024-09-21T14:59:13.885 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:13.885 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.886 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.886 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:13.886 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955609+0000", 2024-09-21T14:59:13.886 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:13.886 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.886 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.886 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:13.886 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955607+0000", 2024-09-21T14:59:13.886 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-21T14:59:13.886 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.886 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.886 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:13.886 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955732+0000", 2024-09-21T14:59:13.886 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000125566 2024-09-21T14:59:13.886 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.887 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.887 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:13.887 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955752+0000", 2024-09-21T14:59:13.887 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9916999999999999e-05 2024-09-21T14:59:13.887 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.887 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.887 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:13.887 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955767+0000", 2024-09-21T14:59:13.887 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4687e-05 2024-09-21T14:59:13.887 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.887 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.887 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:13.887 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.965574+0000", 2024-09-21T14:59:13.887 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0098072850000000007 2024-09-21T14:59:13.888 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.888 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.888 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:13.888 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.965642+0000", 2024-09-21T14:59:13.888 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7784e-05 2024-09-21T14:59:13.888 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.888 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.888 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:59:13.888 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.965753+0000", 2024-09-21T14:59:13.888 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000111111 2024-09-21T14:59:13.888 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.888 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.888 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:13.888 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967717+0000", 2024-09-21T14:59:13.888 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0019635339999999999 2024-09-21T14:59:13.888 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.889 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.889 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:13.889 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.969014+0000", 2024-09-21T14:59:13.889 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001297093 2024-09-21T14:59:13.889 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.889 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.889 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:13.889 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.969130+0000", 2024-09-21T14:59:13.889 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000116011 2024-09-21T14:59:13.889 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.889 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.889 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:13.889 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.969233+0000", 2024-09-21T14:59:13.889 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000102982 2024-09-21T14:59:13.889 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:13.890 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:13.890 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:13.890 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.890 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.890 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T14:59:13.890 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.955783+0000", 2024-09-21T14:59:13.890 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 69.909148247000005, 2024-09-21T14:59:13.890 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014630241, 2024-09-21T14:59:13.890 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:13.890 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:13.890 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:13.890 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:59:13.890 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:59:13.890 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 116 2024-09-21T14:59:13.891 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.891 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:13.891 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.891 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:13.891 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955783+0000", 2024-09-21T14:59:13.891 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:13.891 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.891 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.891 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:13.891 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955783+0000", 2024-09-21T14:59:13.891 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:13.891 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.891 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.891 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:13.892 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955782+0000", 2024-09-21T14:59:13.892 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:59:13.892 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.892 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.892 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:13.892 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955791+0000", 2024-09-21T14:59:13.892 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8089999999999999e-06 2024-09-21T14:59:13.892 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.892 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.892 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:13.892 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955804+0000", 2024-09-21T14:59:13.892 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2548000000000001e-05 2024-09-21T14:59:13.892 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.893 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.893 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:13.893 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955814+0000", 2024-09-21T14:59:13.893 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0158e-05 2024-09-21T14:59:13.893 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.893 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.893 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:13.893 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.966515+0000", 2024-09-21T14:59:13.893 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.010701756 2024-09-21T14:59:13.893 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.893 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.893 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:13.893 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.966606+0000", 2024-09-21T14:59:13.893 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0761999999999996e-05 2024-09-21T14:59:13.893 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.893 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.894 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:59:13.894 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.966740+0000", 2024-09-21T14:59:13.894 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000133693 2024-09-21T14:59:13.894 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.894 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.894 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:13.894 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.968920+0000", 2024-09-21T14:59:13.894 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0021799290000000002 2024-09-21T14:59:13.894 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.894 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.894 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:13.894 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.970320+0000", 2024-09-21T14:59:13.894 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0014001759999999999 2024-09-21T14:59:13.894 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.894 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.895 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:13.895 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.970376+0000", 2024-09-21T14:59:13.895 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5696999999999999e-05 2024-09-21T14:59:13.895 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.895 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.895 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:13.895 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.970413+0000", 2024-09-21T14:59:13.895 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7750999999999998e-05 2024-09-21T14:59:13.895 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:13.895 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:13.895 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:13.895 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.895 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.895 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T14:59:13.896 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.955829+0000", 2024-09-21T14:59:13.896 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 69.909102777000001, 2024-09-21T14:59:13.896 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014919597999999999, 2024-09-21T14:59:13.896 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:13.896 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:13.896 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:13.896 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:59:13.896 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:59:13.896 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 117 2024-09-21T14:59:13.896 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.896 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:13.896 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.896 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:13.896 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955829+0000", 2024-09-21T14:59:13.897 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:13.897 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.897 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.897 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:13.897 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955829+0000", 2024-09-21T14:59:13.897 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:13.897 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.897 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.897 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:13.897 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955827+0000", 2024-09-21T14:59:13.897 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T14:59:13.897 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.897 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.897 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:13.897 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955838+0000", 2024-09-21T14:59:13.898 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.111e-05 2024-09-21T14:59:13.898 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.898 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.898 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:13.898 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955875+0000", 2024-09-21T14:59:13.898 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6211999999999998e-05 2024-09-21T14:59:13.898 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.898 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.898 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:13.898 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955899+0000", 2024-09-21T14:59:13.898 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4128999999999999e-05 2024-09-21T14:59:13.898 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.898 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.898 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:13.899 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967767+0000", 2024-09-21T14:59:13.899 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.011868251999999999 2024-09-21T14:59:13.899 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.899 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.899 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:13.899 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967866+0000", 2024-09-21T14:59:13.899 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8664000000000003e-05 2024-09-21T14:59:13.899 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.899 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.899 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:59:13.899 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.968128+0000", 2024-09-21T14:59:13.899 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00026243600000000001 2024-09-21T14:59:13.899 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.899 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.899 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:13.900 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.969393+0000", 2024-09-21T14:59:13.900 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0012650350000000001 2024-09-21T14:59:13.900 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.900 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.900 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:13.900 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.970523+0000", 2024-09-21T14:59:13.900 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001129557 2024-09-21T14:59:13.900 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.900 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.900 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:13.900 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.970603+0000", 2024-09-21T14:59:13.900 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0090000000000001e-05 2024-09-21T14:59:13.900 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.900 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.901 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:13.901 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.970748+0000", 2024-09-21T14:59:13.901 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014552799999999999 2024-09-21T14:59:13.901 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:13.901 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:13.901 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:13.901 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.901 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.901 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T14:59:13.901 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:04.493315+0000", 2024-09-21T14:59:13.901 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 69.371615943999998, 2024-09-21T14:59:13.901 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014055766000000001, 2024-09-21T14:59:13.901 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:13.901 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:13.902 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:13.902 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:59:13.902 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:59:13.902 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 137 2024-09-21T14:59:13.902 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.902 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:13.902 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.902 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:13.902 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493315+0000", 2024-09-21T14:59:13.902 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:13.902 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.902 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.902 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:13.902 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493315+0000", 2024-09-21T14:59:13.902 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:13.903 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.903 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.903 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:13.903 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493314+0000", 2024-09-21T14:59:13.903 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:59:13.903 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.903 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.903 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:13.903 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493507+0000", 2024-09-21T14:59:13.903 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000192444 2024-09-21T14:59:13.903 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.903 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.903 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:13.903 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493528+0000", 2024-09-21T14:59:13.903 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0845000000000001e-05 2024-09-21T14:59:13.904 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.904 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.904 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:13.904 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493543+0000", 2024-09-21T14:59:13.904 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5800000000000001e-05 2024-09-21T14:59:13.904 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.904 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.904 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:13.904 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.502575+0000", 2024-09-21T14:59:13.904 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0090312389999999999 2024-09-21T14:59:13.904 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.904 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.904 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:13.904 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.502677+0000", 2024-09-21T14:59:13.904 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000102271 2024-09-21T14:59:13.905 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.905 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.905 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:59:13.905 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.502825+0000", 2024-09-21T14:59:13.905 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000148171 2024-09-21T14:59:13.905 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.905 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.905 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:13.905 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507251+0000", 2024-09-21T14:59:13.905 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0044253879999999997 2024-09-21T14:59:13.905 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.905 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.905 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:13.905 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507313+0000", 2024-09-21T14:59:13.905 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2170999999999998e-05 2024-09-21T14:59:13.906 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.906 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.906 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:13.906 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507346+0000", 2024-09-21T14:59:13.906 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3079e-05 2024-09-21T14:59:13.906 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.906 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.906 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:13.906 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507371+0000", 2024-09-21T14:59:13.906 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5477999999999999e-05 2024-09-21T14:59:13.906 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:13.906 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:13.906 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:13.907 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.907 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.907 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T14:59:13.907 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:04.493570+0000", 2024-09-21T14:59:13.907 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 69.371361539000006, 2024-09-21T14:59:13.907 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014061686, 2024-09-21T14:59:13.907 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:13.907 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:13.907 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:13.907 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:59:13.907 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:59:13.907 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 138 2024-09-21T14:59:13.907 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.907 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:13.907 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.908 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:13.908 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493570+0000", 2024-09-21T14:59:13.908 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:13.908 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.908 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.908 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:13.908 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493570+0000", 2024-09-21T14:59:13.908 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:13.908 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.908 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.908 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:13.908 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493568+0000", 2024-09-21T14:59:13.908 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T14:59:13.908 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.908 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.908 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:13.909 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493579+0000", 2024-09-21T14:59:13.909 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0574000000000001e-05 2024-09-21T14:59:13.909 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.909 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.909 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:13.909 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493595+0000", 2024-09-21T14:59:13.909 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6504999999999999e-05 2024-09-21T14:59:13.909 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.909 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.909 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:13.909 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493608+0000", 2024-09-21T14:59:13.909 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2836999999999999e-05 2024-09-21T14:59:13.909 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.909 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.909 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:13.910 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.503909+0000", 2024-09-21T14:59:13.910 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.01030118 2024-09-21T14:59:13.910 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.910 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.910 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:13.910 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.504064+0000", 2024-09-21T14:59:13.910 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015493000000000001 2024-09-21T14:59:13.910 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.910 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.910 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:59:13.910 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.504299+0000", 2024-09-21T14:59:13.910 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00023416100000000001 2024-09-21T14:59:13.910 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.910 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.910 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:13.911 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507466+0000", 2024-09-21T14:59:13.911 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0031678380000000001 2024-09-21T14:59:13.911 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.911 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.911 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:13.911 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507483+0000", 2024-09-21T14:59:13.911 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6606999999999999e-05 2024-09-21T14:59:13.911 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.911 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.911 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:13.911 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507566+0000", 2024-09-21T14:59:13.911 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3110999999999998e-05 2024-09-21T14:59:13.911 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.911 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.911 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:13.912 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507632+0000", 2024-09-21T14:59:13.912 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5480000000000003e-05 2024-09-21T14:59:13.912 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:13.912 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:13.912 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:13.912 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.912 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.912 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T14:59:13.912 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:04.493625+0000", 2024-09-21T14:59:13.912 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 69.371306851, 2024-09-21T14:59:13.912 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014665378999999999, 2024-09-21T14:59:13.912 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:13.912 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:13.912 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:13.913 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:59:13.913 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:59:13.913 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139 2024-09-21T14:59:13.913 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.913 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:13.913 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.913 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:13.913 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493625+0000", 2024-09-21T14:59:13.913 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:13.913 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.913 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.913 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:13.913 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493625+0000", 2024-09-21T14:59:13.913 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:13.914 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.914 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.914 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:13.914 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493624+0000", 2024-09-21T14:59:13.914 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:59:13.914 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.914 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.914 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:13.914 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493638+0000", 2024-09-21T14:59:13.914 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4282e-05 2024-09-21T14:59:13.914 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.914 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.914 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:13.914 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493654+0000", 2024-09-21T14:59:13.915 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5739999999999998e-05 2024-09-21T14:59:13.915 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.915 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.915 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:13.915 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493666+0000", 2024-09-21T14:59:13.915 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2865e-05 2024-09-21T14:59:13.915 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.915 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.915 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:13.915 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.505329+0000", 2024-09-21T14:59:13.915 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.011662237000000001 2024-09-21T14:59:13.915 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.915 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.915 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:13.915 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.505377+0000", 2024-09-21T14:59:13.916 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7862999999999999e-05 2024-09-21T14:59:13.916 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.916 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.916 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:59:13.916 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.505455+0000", 2024-09-21T14:59:13.916 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8918000000000006e-05 2024-09-21T14:59:13.916 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.916 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.916 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:13.916 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507650+0000", 2024-09-21T14:59:13.916 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.002195027 2024-09-21T14:59:13.916 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.916 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.916 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:13.916 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508149+0000", 2024-09-21T14:59:13.917 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00049828399999999999 2024-09-21T14:59:13.917 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.917 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.917 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:13.917 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508230+0000", 2024-09-21T14:59:13.917 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1376000000000002e-05 2024-09-21T14:59:13.917 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.917 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.917 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:13.917 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508290+0000", 2024-09-21T14:59:13.917 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9852999999999998e-05 2024-09-21T14:59:13.917 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:13.917 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:13.917 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:13.918 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.918 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.918 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T14:59:13.918 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:04.493679+0000", 2024-09-21T14:59:13.918 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 69.371251952999998, 2024-09-21T14:59:13.918 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014869869000000001, 2024-09-21T14:59:13.918 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:13.918 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:13.918 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:13.918 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:59:13.918 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:59:13.918 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140 2024-09-21T14:59:13.918 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.918 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:13.919 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.919 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:13.919 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493679+0000", 2024-09-21T14:59:13.919 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:13.919 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.919 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.919 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:13.919 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493679+0000", 2024-09-21T14:59:13.919 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:13.919 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.919 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.919 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:13.919 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493679+0000", 2024-09-21T14:59:13.919 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:59:13.920 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.920 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.920 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:13.920 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493889+0000", 2024-09-21T14:59:13.920 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00021032700000000001 2024-09-21T14:59:13.920 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.920 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.920 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:13.920 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493931+0000", 2024-09-21T14:59:13.920 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1971000000000002e-05 2024-09-21T14:59:13.920 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.920 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.920 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:13.920 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493947+0000", 2024-09-21T14:59:13.920 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6385e-05 2024-09-21T14:59:13.921 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.921 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.921 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:13.921 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.505804+0000", 2024-09-21T14:59:13.921 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.011857136000000001 2024-09-21T14:59:13.921 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.921 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.921 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:13.921 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.505992+0000", 2024-09-21T14:59:13.921 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000187665 2024-09-21T14:59:13.921 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.921 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.921 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:59:13.921 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.506281+0000", 2024-09-21T14:59:13.921 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000288712 2024-09-21T14:59:13.922 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.922 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.922 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:13.922 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508349+0000", 2024-09-21T14:59:13.922 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0020685790000000001 2024-09-21T14:59:13.922 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.922 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.922 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:13.922 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508371+0000", 2024-09-21T14:59:13.922 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1747e-05 2024-09-21T14:59:13.922 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.922 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.922 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:13.922 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508450+0000", 2024-09-21T14:59:13.922 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9332000000000005e-05 2024-09-21T14:59:13.923 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.923 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.923 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:13.923 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508549+0000", 2024-09-21T14:59:13.923 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8918000000000004e-05 2024-09-21T14:59:13.923 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:13.923 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:13.923 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:13.923 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.923 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.923 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T14:59:13.923 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:04.493967+0000", 2024-09-21T14:59:13.923 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 69.370964916000005, 2024-09-21T14:59:13.923 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.015281075999999999, 2024-09-21T14:59:13.924 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:13.924 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:13.924 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:13.924 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:59:13.924 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:59:13.924 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 141 2024-09-21T14:59:13.924 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.924 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:13.924 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.924 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:13.924 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493967+0000", 2024-09-21T14:59:13.924 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:13.924 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.924 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.924 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:13.924 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493967+0000", 2024-09-21T14:59:13.925 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:13.925 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.925 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.925 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:13.925 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493965+0000", 2024-09-21T14:59:13.925 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T14:59:13.925 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.925 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.925 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:13.925 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493987+0000", 2024-09-21T14:59:13.925 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1662999999999999e-05 2024-09-21T14:59:13.925 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.925 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.925 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:13.926 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.494004+0000", 2024-09-21T14:59:13.926 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7311000000000001e-05 2024-09-21T14:59:13.926 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.926 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.926 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:13.926 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.494017+0000", 2024-09-21T14:59:13.926 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3202e-05 2024-09-21T14:59:13.926 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.926 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.926 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:13.926 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507678+0000", 2024-09-21T14:59:13.926 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013660432 2024-09-21T14:59:13.926 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.927 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.927 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:13.927 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507726+0000", 2024-09-21T14:59:13.927 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8154e-05 2024-09-21T14:59:13.927 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.927 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.927 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:59:13.927 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507797+0000", 2024-09-21T14:59:13.927 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.1110000000000002e-05 2024-09-21T14:59:13.927 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.927 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.927 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:13.927 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508710+0000", 2024-09-21T14:59:13.927 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00091339299999999997 2024-09-21T14:59:13.927 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.928 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.928 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:13.928 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.509108+0000", 2024-09-21T14:59:13.928 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00039718300000000001 2024-09-21T14:59:13.928 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.928 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.928 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:13.928 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.509186+0000", 2024-09-21T14:59:13.928 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8592999999999998e-05 2024-09-21T14:59:13.928 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.928 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.928 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:13.928 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.509248+0000", 2024-09-21T14:59:13.928 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1450000000000003e-05 2024-09-21T14:59:13.928 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:13.929 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:13.929 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:13.929 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.929 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.929 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6626 3.1a 3:5adee3e5:::rbd_data.11361bed1f78.0000000000000001:d [read 229376~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:59:13.929 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131593+0000", 2024-09-21T14:59:13.929 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 38.733338338000003, 2024-09-21T14:59:13.929 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012576725, 2024-09-21T14:59:13.929 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:13.929 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:13.929 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:13.929 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:13.929 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:13.929 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6626 2024-09-21T14:59:13.930 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.930 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:13.930 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.930 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:13.930 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131593+0000", 2024-09-21T14:59:13.930 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:13.930 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.930 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.930 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:13.930 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131593+0000", 2024-09-21T14:59:13.930 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:13.930 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.930 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.930 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:13.931 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131593+0000", 2024-09-21T14:59:13.931 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T14:59:13.931 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.931 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.931 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:13.931 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131598+0000", 2024-09-21T14:59:13.931 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7849999999999999e-06 2024-09-21T14:59:13.931 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.931 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.931 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:13.931 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131601+0000", 2024-09-21T14:59:13.931 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1949999999999999e-06 2024-09-21T14:59:13.931 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.931 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.931 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:13.932 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131604+0000", 2024-09-21T14:59:13.932 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6350000000000002e-06 2024-09-21T14:59:13.932 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.932 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.932 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:13.932 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.143902+0000", 2024-09-21T14:59:13.932 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012298216000000001 2024-09-21T14:59:13.932 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.932 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.932 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:13.932 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.143993+0000", 2024-09-21T14:59:13.932 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1073e-05 2024-09-21T14:59:13.932 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.932 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.933 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:13.933 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.144170+0000", 2024-09-21T14:59:13.933 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017703 2024-09-21T14:59:13.933 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:13.933 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:13.933 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:13.933 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.933 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.933 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6631 3.1a 3:5d547f54:::rbd_data.11361bed1f78.0000000000000003:d [read 0~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:59:13.933 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131606+0000", 2024-09-21T14:59:13.933 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 38.733325637999997, 2024-09-21T14:59:13.933 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013903148000000001, 2024-09-21T14:59:13.933 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:13.933 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:13.933 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:13.934 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:13.934 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:13.934 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6631 2024-09-21T14:59:13.934 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.934 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:13.934 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.934 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:13.934 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131606+0000", 2024-09-21T14:59:13.934 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:13.934 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.934 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.934 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:13.934 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131606+0000", 2024-09-21T14:59:13.934 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:13.935 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.935 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.935 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:13.935 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131606+0000", 2024-09-21T14:59:13.935 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T14:59:13.935 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.935 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.935 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:13.935 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131607+0000", 2024-09-21T14:59:13.935 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2699999999999999e-06 2024-09-21T14:59:13.935 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.935 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.935 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:13.936 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131610+0000", 2024-09-21T14:59:13.936 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.88e-06 2024-09-21T14:59:13.936 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.936 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.936 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:13.936 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131612+0000", 2024-09-21T14:59:13.936 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5170000000000002e-06 2024-09-21T14:59:13.936 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.936 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.936 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:13.936 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.144205+0000", 2024-09-21T14:59:13.936 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012592763 2024-09-21T14:59:13.936 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.936 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.937 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:13.937 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.144332+0000", 2024-09-21T14:59:13.937 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012694200000000001 2024-09-21T14:59:13.937 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.937 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.937 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:13.937 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.145509+0000", 2024-09-21T14:59:13.937 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001176992 2024-09-21T14:59:13.937 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:13.937 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:13.937 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:13.937 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.937 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.937 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6632 3.1a 3:5d547f54:::rbd_data.11361bed1f78.0000000000000003:d [read 81920~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:59:13.937 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131614+0000", 2024-09-21T14:59:13.938 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 38.733317030000002, 2024-09-21T14:59:13.938 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014093553999999999, 2024-09-21T14:59:13.938 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:13.938 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:13.938 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:13.938 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:13.938 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:13.938 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6632 2024-09-21T14:59:13.938 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.938 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:13.938 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.938 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:13.938 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131614+0000", 2024-09-21T14:59:13.938 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:13.938 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.939 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.939 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:13.939 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131614+0000", 2024-09-21T14:59:13.939 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:13.939 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.939 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.939 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:13.939 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131614+0000", 2024-09-21T14:59:13.939 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T14:59:13.939 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.939 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.939 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:13.939 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131615+0000", 2024-09-21T14:59:13.939 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.218e-06 2024-09-21T14:59:13.939 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.940 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.940 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:13.940 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131618+0000", 2024-09-21T14:59:13.940 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9050000000000001e-06 2024-09-21T14:59:13.940 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.940 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.940 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:13.940 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131621+0000", 2024-09-21T14:59:13.940 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7439999999999999e-06 2024-09-21T14:59:13.940 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.940 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.940 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:13.940 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.145538+0000", 2024-09-21T14:59:13.940 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013916618 2024-09-21T14:59:13.940 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.941 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.941 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:13.941 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.145619+0000", 2024-09-21T14:59:13.941 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1812999999999997e-05 2024-09-21T14:59:13.941 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.941 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.941 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:13.941 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.145708+0000", 2024-09-21T14:59:13.941 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8460000000000003e-05 2024-09-21T14:59:13.941 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:13.941 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:13.941 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:13.941 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.941 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.941 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6633 3.1a 3:5d547f54:::rbd_data.11361bed1f78.0000000000000003:d [read 147456~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:59:13.941 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131623+0000", 2024-09-21T14:59:13.942 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 38.733308063999999, 2024-09-21T14:59:13.942 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014395845000000001, 2024-09-21T14:59:13.942 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:13.942 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:13.942 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:13.942 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:13.942 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:13.942 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6633 2024-09-21T14:59:13.942 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.942 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:13.942 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.942 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:13.942 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131623+0000", 2024-09-21T14:59:13.942 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:13.942 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.943 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.943 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:13.943 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131623+0000", 2024-09-21T14:59:13.943 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:13.943 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.943 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.943 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:13.943 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131623+0000", 2024-09-21T14:59:13.943 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T14:59:13.943 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.943 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.943 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:13.943 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131628+0000", 2024-09-21T14:59:13.943 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0370000000000002e-06 2024-09-21T14:59:13.943 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.944 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.944 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:13.944 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131632+0000", 2024-09-21T14:59:13.944 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3519999999999998e-06 2024-09-21T14:59:13.944 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.944 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.944 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:13.944 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131634+0000", 2024-09-21T14:59:13.944 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5019999999999999e-06 2024-09-21T14:59:13.944 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.944 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.944 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:13.944 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.145734+0000", 2024-09-21T14:59:13.944 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014100118 2024-09-21T14:59:13.944 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.945 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.945 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:13.945 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.145804+0000", 2024-09-21T14:59:13.945 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0229999999999997e-05 2024-09-21T14:59:13.945 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.945 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.945 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:13.945 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146019+0000", 2024-09-21T14:59:13.945 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00021481699999999999 2024-09-21T14:59:13.945 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:13.945 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:13.945 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:13.945 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.945 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.946 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6634 3.1a 3:5d547f54:::rbd_data.11361bed1f78.0000000000000003:d [read 229376~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:59:13.946 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131636+0000", 2024-09-21T14:59:13.946 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 38.733294954000002, 2024-09-21T14:59:13.946 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014645843, 2024-09-21T14:59:13.946 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:13.946 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:13.946 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:13.946 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:13.946 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:13.946 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6634 2024-09-21T14:59:13.946 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.946 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:13.946 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.946 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:13.947 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131636+0000", 2024-09-21T14:59:13.947 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:13.947 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.947 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.947 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:13.947 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131636+0000", 2024-09-21T14:59:13.947 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:13.947 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.947 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.947 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:13.947 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131636+0000", 2024-09-21T14:59:13.947 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:13.947 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.947 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.947 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:13.948 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131641+0000", 2024-09-21T14:59:13.948 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8489999999999997e-06 2024-09-21T14:59:13.948 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.948 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.948 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:13.948 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131644+0000", 2024-09-21T14:59:13.948 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.179e-06 2024-09-21T14:59:13.948 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.948 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.948 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:13.948 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131647+0000", 2024-09-21T14:59:13.948 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8009999999999999e-06 2024-09-21T14:59:13.948 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.948 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.949 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:13.949 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146046+0000", 2024-09-21T14:59:13.949 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014399018 2024-09-21T14:59:13.949 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.949 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.949 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:13.949 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146118+0000", 2024-09-21T14:59:13.949 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2044000000000003e-05 2024-09-21T14:59:13.949 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.949 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.949 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:13.949 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146282+0000", 2024-09-21T14:59:13.949 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000164196 2024-09-21T14:59:13.949 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:13.949 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:13.950 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:13.950 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.950 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.950 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6639 3.1a 3:592d80ed:::rbd_data.11361bed1f78.0000000000000005:d [read 0~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:59:13.950 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131650+0000", 2024-09-21T14:59:13.950 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 38.733281755, 2024-09-21T14:59:13.950 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016002832000000002, 2024-09-21T14:59:13.950 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:13.950 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:13.950 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:13.950 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:13.950 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:13.950 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6639 2024-09-21T14:59:13.950 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.951 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:13.951 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.951 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:13.951 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131650+0000", 2024-09-21T14:59:13.951 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:13.951 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.951 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.951 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:13.951 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131650+0000", 2024-09-21T14:59:13.951 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:13.951 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.951 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.951 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:13.952 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131649+0000", 2024-09-21T14:59:13.952 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:13.952 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.952 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.952 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:13.952 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131651+0000", 2024-09-21T14:59:13.952 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9199999999999998e-06 2024-09-21T14:59:13.952 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.952 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.952 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:13.952 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131656+0000", 2024-09-21T14:59:13.952 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9899999999999997e-06 2024-09-21T14:59:13.952 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.952 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.952 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:13.953 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131665+0000", 2024-09-21T14:59:13.953 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0999999999999993e-06 2024-09-21T14:59:13.953 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.953 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.953 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:13.953 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146309+0000", 2024-09-21T14:59:13.953 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014643893999999999 2024-09-21T14:59:13.953 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.953 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.953 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:13.953 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146392+0000", 2024-09-21T14:59:13.953 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2531e-05 2024-09-21T14:59:13.953 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.953 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.953 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:13.954 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147653+0000", 2024-09-21T14:59:13.954 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0012606710000000001 2024-09-21T14:59:13.954 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:13.954 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:13.954 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:13.954 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.954 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.954 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6640 3.1a 3:592d80ed:::rbd_data.11361bed1f78.0000000000000005:d [read 65536~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:59:13.954 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131670+0000", 2024-09-21T14:59:13.954 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 38.733261456000001, 2024-09-21T14:59:13.954 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016188859, 2024-09-21T14:59:13.954 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:13.954 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:13.954 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:13.955 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:13.955 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:13.955 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6640 2024-09-21T14:59:13.955 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.955 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:13.955 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.955 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:13.955 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131670+0000", 2024-09-21T14:59:13.955 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:13.955 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.955 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.955 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:13.955 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131670+0000", 2024-09-21T14:59:13.955 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:13.955 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.956 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.956 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:13.956 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131670+0000", 2024-09-21T14:59:13.956 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T14:59:13.956 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.956 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.956 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:13.956 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131672+0000", 2024-09-21T14:59:13.956 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.818e-06 2024-09-21T14:59:13.956 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.956 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.956 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:13.956 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131675+0000", 2024-09-21T14:59:13.956 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5930000000000002e-06 2024-09-21T14:59:13.956 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.957 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.957 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:13.957 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131678+0000", 2024-09-21T14:59:13.957 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6790000000000001e-06 2024-09-21T14:59:13.957 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.957 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.957 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:13.957 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147681+0000", 2024-09-21T14:59:13.957 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016002755 2024-09-21T14:59:13.957 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.957 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.957 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:13.957 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147762+0000", 2024-09-21T14:59:13.957 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1052999999999998e-05 2024-09-21T14:59:13.957 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.958 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.958 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:13.958 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147859+0000", 2024-09-21T14:59:13.958 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.7199000000000002e-05 2024-09-21T14:59:13.958 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:13.958 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:13.958 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:13.958 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.958 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.958 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6641 3.1a 3:592d80ed:::rbd_data.11361bed1f78.0000000000000005:d [read 147456~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:59:13.958 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131680+0000", 2024-09-21T14:59:13.958 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 38.733251168000002, 2024-09-21T14:59:13.958 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016431296000000001, 2024-09-21T14:59:13.958 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:13.959 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:13.959 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:13.959 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:13.959 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:13.959 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6641 2024-09-21T14:59:13.959 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.959 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:13.959 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.959 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:13.959 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131680+0000", 2024-09-21T14:59:13.959 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:13.959 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.959 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.959 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:13.960 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131680+0000", 2024-09-21T14:59:13.960 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:13.960 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.960 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.960 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:13.960 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131680+0000", 2024-09-21T14:59:13.960 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T14:59:13.960 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.960 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.960 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:13.960 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131681+0000", 2024-09-21T14:59:13.960 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2610000000000001e-06 2024-09-21T14:59:13.960 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.960 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.960 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:13.960 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131685+0000", 2024-09-21T14:59:13.961 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.8299999999999998e-06 2024-09-21T14:59:13.961 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.961 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.961 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:13.961 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131688+0000", 2024-09-21T14:59:13.961 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6189999999999998e-06 2024-09-21T14:59:13.961 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.961 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.961 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:13.961 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147884+0000", 2024-09-21T14:59:13.961 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016196465 2024-09-21T14:59:13.961 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.961 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.961 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:13.961 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147949+0000", 2024-09-21T14:59:13.962 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4867000000000001e-05 2024-09-21T14:59:13.962 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.962 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.962 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:13.962 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.148112+0000", 2024-09-21T14:59:13.962 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00016247700000000001 2024-09-21T14:59:13.962 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:13.962 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:13.962 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:13.962 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.962 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.962 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6642 3.1a 3:592d80ed:::rbd_data.11361bed1f78.0000000000000005:d [read 212992~32768 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:59:13.962 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131690+0000", 2024-09-21T14:59:13.962 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 38.733241290000002, 2024-09-21T14:59:13.963 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016751335999999999, 2024-09-21T14:59:13.963 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:13.963 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:13.963 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:13.963 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:13.963 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:13.963 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6642 2024-09-21T14:59:13.963 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.963 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:13.963 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.963 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:13.963 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131690+0000", 2024-09-21T14:59:13.963 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:13.963 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.963 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.964 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:13.964 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131690+0000", 2024-09-21T14:59:13.964 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:13.964 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.964 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.964 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:13.964 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131690+0000", 2024-09-21T14:59:13.964 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T14:59:13.964 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.964 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.964 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:13.964 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131691+0000", 2024-09-21T14:59:13.964 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3120000000000001e-06 2024-09-21T14:59:13.964 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.965 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.965 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:13.965 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131694+0000", 2024-09-21T14:59:13.965 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0070000000000002e-06 2024-09-21T14:59:13.965 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.965 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.965 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:13.965 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131700+0000", 2024-09-21T14:59:13.965 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0800000000000002e-06 2024-09-21T14:59:13.965 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.965 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.965 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:13.965 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.148135+0000", 2024-09-21T14:59:13.965 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016434250000000001 2024-09-21T14:59:13.965 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.966 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.966 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:13.966 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.148204+0000", 2024-09-21T14:59:13.966 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9466000000000002e-05 2024-09-21T14:59:13.966 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:13.966 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:13.966 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:13.966 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.148441+0000", 2024-09-21T14:59:13.966 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00023743700000000001 2024-09-21T14:59:13.966 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:13.966 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:13.966 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:13.966 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:13.966 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:13.966 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:13.968 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T14:59:13.968 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:59:13.965+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:59:14.069 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:59:14.069+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:59:14.082 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:14.082 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:59:14.082 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T14:59:14.083 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:14.092 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T14:59:14.143 INFO:teuthology.orchestra.run.smithi032.stderr:nodeep-scrub is unset 2024-09-21T14:59:14.169 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:59:14.169+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:59:14.217 INFO:teuthology.orchestra.run.smithi032.stdout:checking clone #11, image image_client.0-clone11 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent12 2024-09-21T14:59:14.219 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:14.219 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:59:14.220 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T14:59:14.220 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T14:59:14.220 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:14.230 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T14:59:14.270 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:59:14.268+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:59:14.360 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T14:59:14.360 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T14:59:14.360 INFO:tasks.thrashosds.thrasher:Removing osd 4, in_osds are: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T14:59:14.361 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 4 2024-09-21T14:59:14.370 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:59:14.368+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:59:14.408 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:14.408 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T14:59:14.408 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T14:59:14.408 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T14:59:14.409 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.409 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:15130 3.2 3:47f1bbdb:::rbd_data.11361bed1f78.0000000000000067:head [sparse-read 0~147456] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-21T14:59:14.409 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:40.288284+0000", 2024-09-21T14:59:14.409 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 34.119160356999998, 2024-09-21T14:59:14.409 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1142935039999999, 2024-09-21T14:59:14.409 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:14.409 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:14.409 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:14.409 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:14.409 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:14.409 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 15130 2024-09-21T14:59:14.409 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.409 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:14.409 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.410 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:14.410 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288284+0000", 2024-09-21T14:59:14.410 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:14.410 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.410 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.410 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:14.410 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288284+0000", 2024-09-21T14:59:14.410 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:14.410 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.410 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.410 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:14.410 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288284+0000", 2024-09-21T14:59:14.410 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:14.411 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.411 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.411 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:14.411 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288288+0000", 2024-09-21T14:59:14.411 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5829999999999998e-06 2024-09-21T14:59:14.411 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.411 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.411 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:14.411 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288297+0000", 2024-09-21T14:59:14.411 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0669999999999998e-06 2024-09-21T14:59:14.411 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.411 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.411 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:14.412 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288305+0000", 2024-09-21T14:59:14.412 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8860000000000004e-06 2024-09-21T14:59:14.412 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.412 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.412 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:14.412 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288417+0000", 2024-09-21T14:59:14.412 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000112952 2024-09-21T14:59:14.412 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.412 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.412 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T14:59:14.412 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288428+0000", 2024-09-21T14:59:14.412 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0631e-05 2024-09-21T14:59:14.412 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.412 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.413 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:14.413 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.401876+0000", 2024-09-21T14:59:14.413 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1134479610000001 2024-09-21T14:59:14.413 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.413 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.413 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:14.413 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.402076+0000", 2024-09-21T14:59:14.413 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00019944200000000001 2024-09-21T14:59:14.413 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.413 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.413 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:14.413 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.402578+0000", 2024-09-21T14:59:14.414 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00050242899999999998 2024-09-21T14:59:14.414 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:14.414 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:14.414 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:14.414 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.414 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.414 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45708 3.2 3:415db91a:::rbd_data.1136ecbe5879.0000000000000000:28 [read 140288~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:14.414 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141801+0000", 2024-09-21T14:59:14.414 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.2656441630000002, 2024-09-21T14:59:14.414 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034274200999999997, 2024-09-21T14:59:14.414 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:14.414 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:14.414 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:14.415 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:14.415 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:14.415 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45708 2024-09-21T14:59:14.415 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.415 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:14.415 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.415 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:14.415 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141801+0000", 2024-09-21T14:59:14.415 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:14.415 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.415 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.415 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:14.415 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141801+0000", 2024-09-21T14:59:14.415 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:14.416 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.416 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.416 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:14.416 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141800+0000", 2024-09-21T14:59:14.416 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:14.416 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.416 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.416 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:14.416 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141803+0000", 2024-09-21T14:59:14.416 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4210000000000002e-06 2024-09-21T14:59:14.416 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.416 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.416 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:14.417 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141812+0000", 2024-09-21T14:59:14.417 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.9800000000000004e-06 2024-09-21T14:59:14.417 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.417 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.417 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:14.417 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141816+0000", 2024-09-21T14:59:14.417 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4769999999999997e-06 2024-09-21T14:59:14.417 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.417 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.417 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:14.417 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.175865+0000", 2024-09-21T14:59:14.417 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034048875999999999 2024-09-21T14:59:14.417 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.417 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.418 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:14.418 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.175943+0000", 2024-09-21T14:59:14.418 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7464000000000002e-05 2024-09-21T14:59:14.418 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.418 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.418 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:14.418 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176075+0000", 2024-09-21T14:59:14.418 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013230800000000001 2024-09-21T14:59:14.418 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:14.418 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:14.418 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:14.418 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.418 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.418 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45709 3.2 3:415db91a:::rbd_data.1136ecbe5879.0000000000000000:28 [read 143360~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:14.419 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141820+0000", 2024-09-21T14:59:14.419 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.2656243160000002, 2024-09-21T14:59:14.419 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034537536000000001, 2024-09-21T14:59:14.419 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:14.419 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:14.419 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:14.419 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:14.419 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:14.419 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45709 2024-09-21T14:59:14.419 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.419 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:14.419 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.419 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:14.420 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141820+0000", 2024-09-21T14:59:14.420 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:14.420 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.420 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.420 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:14.420 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141820+0000", 2024-09-21T14:59:14.420 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:14.420 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.420 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.420 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:14.420 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141820+0000", 2024-09-21T14:59:14.420 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:14.420 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.420 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.421 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:14.421 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141823+0000", 2024-09-21T14:59:14.421 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5110000000000002e-06 2024-09-21T14:59:14.421 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.421 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.421 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:14.421 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141829+0000", 2024-09-21T14:59:14.421 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0850000000000002e-06 2024-09-21T14:59:14.421 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.421 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.421 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:14.421 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141833+0000", 2024-09-21T14:59:14.421 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7620000000000003e-06 2024-09-21T14:59:14.422 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.422 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.422 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:14.422 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176103+0000", 2024-09-21T14:59:14.422 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034269933000000002 2024-09-21T14:59:14.422 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.422 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.422 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:14.422 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176161+0000", 2024-09-21T14:59:14.422 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7658999999999998e-05 2024-09-21T14:59:14.422 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.422 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.422 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:14.423 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176358+0000", 2024-09-21T14:59:14.423 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000196937 2024-09-21T14:59:14.423 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:14.423 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:14.423 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:14.423 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.423 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.423 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45710 3.2 3:415db91a:::rbd_data.1136ecbe5879.0000000000000000:28 [read 146432~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:14.423 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141838+0000", 2024-09-21T14:59:14.423 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.2656071530000004, 2024-09-21T14:59:14.423 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034953674999999997, 2024-09-21T14:59:14.423 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:14.424 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:14.424 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:14.424 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:14.424 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:14.424 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45710 2024-09-21T14:59:14.424 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.424 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:14.424 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.424 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:14.424 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141838+0000", 2024-09-21T14:59:14.424 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:14.424 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.424 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.424 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:14.425 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141838+0000", 2024-09-21T14:59:14.425 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:14.425 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.425 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.425 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:14.425 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141837+0000", 2024-09-21T14:59:14.425 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:14.425 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.425 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.425 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:14.425 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141843+0000", 2024-09-21T14:59:14.425 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1619999999999996e-06 2024-09-21T14:59:14.425 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.425 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.426 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:14.426 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141850+0000", 2024-09-21T14:59:14.426 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4799999999999998e-06 2024-09-21T14:59:14.426 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.426 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.426 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:14.426 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141853+0000", 2024-09-21T14:59:14.426 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3050000000000001e-06 2024-09-21T14:59:14.426 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.426 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.426 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:14.426 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176490+0000", 2024-09-21T14:59:14.426 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034636413999999997 2024-09-21T14:59:14.427 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.427 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.427 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:14.427 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176606+0000", 2024-09-21T14:59:14.427 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000115841 2024-09-21T14:59:14.427 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.427 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.427 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:14.427 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176791+0000", 2024-09-21T14:59:14.427 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00018581500000000001 2024-09-21T14:59:14.427 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:14.427 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:14.427 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:14.428 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.428 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.428 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45711 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:14.428 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141856+0000", 2024-09-21T14:59:14.428 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.2655888160000002, 2024-09-21T14:59:14.428 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034986308000000001, 2024-09-21T14:59:14.428 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:14.428 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:14.428 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:14.428 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:14.428 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:14.428 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45711 2024-09-21T14:59:14.428 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.428 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:14.429 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.429 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:14.429 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141856+0000", 2024-09-21T14:59:14.429 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:14.429 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.429 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.429 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:14.429 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141856+0000", 2024-09-21T14:59:14.429 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:14.429 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.429 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.429 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:14.429 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141856+0000", 2024-09-21T14:59:14.429 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:14.430 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.430 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.430 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:14.430 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141862+0000", 2024-09-21T14:59:14.430 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.567e-06 2024-09-21T14:59:14.430 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.430 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.430 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:14.430 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141868+0000", 2024-09-21T14:59:14.430 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9569999999999999e-06 2024-09-21T14:59:14.430 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.430 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.430 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:14.431 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141871+0000", 2024-09-21T14:59:14.431 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.146e-06 2024-09-21T14:59:14.431 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.431 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.431 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:14.431 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176831+0000", 2024-09-21T14:59:14.431 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034959935999999997 2024-09-21T14:59:14.431 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.431 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.431 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:14.431 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176842+0000", 2024-09-21T14:59:14.431 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1008e-05 2024-09-21T14:59:14.431 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:14.431 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:14.431 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:14.432 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.432 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.432 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45712 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:14.432 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141874+0000", 2024-09-21T14:59:14.432 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.2655709210000001, 2024-09-21T14:59:14.432 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034993511999999997, 2024-09-21T14:59:14.432 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:14.432 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:14.432 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:14.432 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:14.432 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:14.432 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45712 2024-09-21T14:59:14.432 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.433 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:14.433 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.433 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:14.433 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141874+0000", 2024-09-21T14:59:14.433 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:14.433 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.433 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.433 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:14.433 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141874+0000", 2024-09-21T14:59:14.433 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:14.433 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.433 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.433 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:14.433 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141874+0000", 2024-09-21T14:59:14.434 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:14.434 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.434 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.434 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:14.434 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141875+0000", 2024-09-21T14:59:14.434 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5430000000000001e-06 2024-09-21T14:59:14.434 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.434 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.434 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:14.434 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141878+0000", 2024-09-21T14:59:14.434 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1420000000000002e-06 2024-09-21T14:59:14.434 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.434 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.434 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:14.435 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141881+0000", 2024-09-21T14:59:14.435 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.807e-06 2024-09-21T14:59:14.435 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.435 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.435 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:14.435 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176860+0000", 2024-09-21T14:59:14.435 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034979243 2024-09-21T14:59:14.435 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.435 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.435 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:14.435 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176867+0000", 2024-09-21T14:59:14.435 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0169999999999999e-06 2024-09-21T14:59:14.435 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:14.435 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:14.436 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:14.436 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.436 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.436 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45713 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:14.436 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141883+0000", 2024-09-21T14:59:14.436 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.2655613460000001, 2024-09-21T14:59:14.436 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035010516999999998, 2024-09-21T14:59:14.436 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:14.436 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:14.436 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:14.436 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:14.436 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:14.436 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45713 2024-09-21T14:59:14.437 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.437 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:14.437 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.437 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:14.437 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141883+0000", 2024-09-21T14:59:14.437 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:14.437 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.437 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.437 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:14.437 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141883+0000", 2024-09-21T14:59:14.437 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:14.437 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.437 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.437 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:14.438 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141883+0000", 2024-09-21T14:59:14.438 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:14.438 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.438 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.438 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:14.438 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141885+0000", 2024-09-21T14:59:14.438 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4649999999999999e-06 2024-09-21T14:59:14.438 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.438 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.438 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:14.438 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141888+0000", 2024-09-21T14:59:14.438 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3890000000000001e-06 2024-09-21T14:59:14.439 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.439 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.439 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:14.439 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141891+0000", 2024-09-21T14:59:14.439 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9239999999999999e-06 2024-09-21T14:59:14.439 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.439 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.439 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:14.439 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176887+0000", 2024-09-21T14:59:14.439 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034996370999999998 2024-09-21T14:59:14.439 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.439 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.439 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:14.439 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176894+0000", 2024-09-21T14:59:14.440 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6069999999999996e-06 2024-09-21T14:59:14.440 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:14.440 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:14.440 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:14.440 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.440 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.440 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45714 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:14.440 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141893+0000", 2024-09-21T14:59:14.440 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.2655514310000004, 2024-09-21T14:59:14.440 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035048215000000001, 2024-09-21T14:59:14.440 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:14.440 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:14.440 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:14.441 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:14.441 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:14.441 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45714 2024-09-21T14:59:14.441 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.441 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:14.441 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.441 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:14.441 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141893+0000", 2024-09-21T14:59:14.441 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:14.441 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.441 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.441 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:14.441 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141893+0000", 2024-09-21T14:59:14.442 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:14.442 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.442 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.442 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:14.442 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141893+0000", 2024-09-21T14:59:14.442 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:14.442 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.442 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.442 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:14.442 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141895+0000", 2024-09-21T14:59:14.442 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.703e-06 2024-09-21T14:59:14.442 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.442 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.442 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:14.443 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141900+0000", 2024-09-21T14:59:14.443 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.242e-06 2024-09-21T14:59:14.443 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.443 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.443 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:14.443 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141903+0000", 2024-09-21T14:59:14.443 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.117e-06 2024-09-21T14:59:14.443 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.443 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.443 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:14.443 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176930+0000", 2024-09-21T14:59:14.443 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035026750000000002 2024-09-21T14:59:14.443 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.444 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.444 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:14.444 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176942+0000", 2024-09-21T14:59:14.444 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1668000000000001e-05 2024-09-21T14:59:14.444 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:14.444 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:14.444 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:14.444 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.444 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.444 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45715 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:14.444 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141906+0000", 2024-09-21T14:59:14.444 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.2655392729999999, 2024-09-21T14:59:14.444 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035066057999999997, 2024-09-21T14:59:14.444 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:14.445 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:14.445 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:14.445 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:14.445 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:14.445 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45715 2024-09-21T14:59:14.445 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.445 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:14.445 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.445 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:14.445 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141906+0000", 2024-09-21T14:59:14.445 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:14.445 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.445 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.445 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:14.446 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141906+0000", 2024-09-21T14:59:14.446 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:14.446 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.446 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.446 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:14.446 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141905+0000", 2024-09-21T14:59:14.446 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:14.446 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.446 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.446 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:14.446 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141907+0000", 2024-09-21T14:59:14.446 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5400000000000001e-06 2024-09-21T14:59:14.446 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.447 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.447 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:14.447 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141912+0000", 2024-09-21T14:59:14.447 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.502e-06 2024-09-21T14:59:14.447 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.447 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.447 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:14.447 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141915+0000", 2024-09-21T14:59:14.447 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0170000000000001e-06 2024-09-21T14:59:14.447 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.447 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.447 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:14.447 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176964+0000", 2024-09-21T14:59:14.447 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035048587999999999 2024-09-21T14:59:14.448 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.448 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.448 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:14.448 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176972+0000", 2024-09-21T14:59:14.448 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6580000000000002e-06 2024-09-21T14:59:14.448 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:14.448 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:14.448 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:14.448 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.448 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.448 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45716 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:14.448 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141918+0000", 2024-09-21T14:59:14.448 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.2655270969999997, 2024-09-21T14:59:14.448 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035086223, 2024-09-21T14:59:14.449 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:14.449 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:14.449 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:14.449 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:14.449 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:14.449 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45716 2024-09-21T14:59:14.449 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.449 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:14.449 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.449 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:14.449 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141918+0000", 2024-09-21T14:59:14.449 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:14.449 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.449 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.450 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:14.450 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141918+0000", 2024-09-21T14:59:14.450 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:14.450 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.450 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.450 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:14.450 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141917+0000", 2024-09-21T14:59:14.450 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:14.450 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.450 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.450 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:14.450 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141919+0000", 2024-09-21T14:59:14.450 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5680000000000001e-06 2024-09-21T14:59:14.450 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.451 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.451 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:14.451 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141922+0000", 2024-09-21T14:59:14.451 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2789999999999999e-06 2024-09-21T14:59:14.451 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.451 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.451 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:14.451 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141925+0000", 2024-09-21T14:59:14.451 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9469999999999999e-06 2024-09-21T14:59:14.451 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.451 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.451 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:14.451 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176997+0000", 2024-09-21T14:59:14.452 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035071366999999999 2024-09-21T14:59:14.452 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.452 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.452 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:14.452 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177004+0000", 2024-09-21T14:59:14.452 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3309999999999998e-06 2024-09-21T14:59:14.452 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:14.452 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:14.452 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:14.452 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.452 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.452 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45717 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:14.452 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141928+0000", 2024-09-21T14:59:14.452 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.2655171510000001, 2024-09-21T14:59:14.453 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035102573999999998, 2024-09-21T14:59:14.453 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:14.453 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:14.453 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:14.453 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:14.453 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:14.453 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45717 2024-09-21T14:59:14.453 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.453 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:14.453 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.453 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:14.453 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141928+0000", 2024-09-21T14:59:14.454 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:14.454 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.454 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.454 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:14.454 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141928+0000", 2024-09-21T14:59:14.454 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:14.454 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.454 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.454 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:14.454 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141927+0000", 2024-09-21T14:59:14.454 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T14:59:14.454 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.454 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.455 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:14.455 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141929+0000", 2024-09-21T14:59:14.455 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.533e-06 2024-09-21T14:59:14.455 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.455 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.455 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:14.455 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141934+0000", 2024-09-21T14:59:14.455 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1660000000000002e-06 2024-09-21T14:59:14.455 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.455 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.455 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:14.455 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141937+0000", 2024-09-21T14:59:14.455 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0309999999999999e-06 2024-09-21T14:59:14.455 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.456 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.456 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:14.456 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177023+0000", 2024-09-21T14:59:14.456 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035085595999999997 2024-09-21T14:59:14.456 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.456 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.456 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:14.456 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177030+0000", 2024-09-21T14:59:14.456 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.486e-06 2024-09-21T14:59:14.456 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:14.456 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:14.456 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:14.456 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.457 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.457 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45718 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:14.457 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141940+0000", 2024-09-21T14:59:14.457 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.2655052930000004, 2024-09-21T14:59:14.457 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035124956999999998, 2024-09-21T14:59:14.457 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:14.457 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:14.457 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:14.457 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:14.457 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:14.457 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45718 2024-09-21T14:59:14.457 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.457 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:14.457 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.458 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:14.458 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141940+0000", 2024-09-21T14:59:14.458 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:14.458 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.458 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.458 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:14.458 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141940+0000", 2024-09-21T14:59:14.458 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:14.458 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.458 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.458 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:14.458 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141939+0000", 2024-09-21T14:59:14.458 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:14.459 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.459 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.459 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:14.459 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141941+0000", 2024-09-21T14:59:14.459 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.502e-06 2024-09-21T14:59:14.459 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.459 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.459 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:14.459 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141946+0000", 2024-09-21T14:59:14.459 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3449999999999996e-06 2024-09-21T14:59:14.459 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.459 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.459 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:14.460 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141949+0000", 2024-09-21T14:59:14.460 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9749999999999999e-06 2024-09-21T14:59:14.460 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.460 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.460 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:14.460 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177057+0000", 2024-09-21T14:59:14.460 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035108385999999998 2024-09-21T14:59:14.460 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.460 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.460 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:14.460 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177064+0000", 2024-09-21T14:59:14.460 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0099999999999998e-06 2024-09-21T14:59:14.460 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:14.460 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:14.460 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:14.461 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.461 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.461 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45719 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:14.461 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141952+0000", 2024-09-21T14:59:14.461 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.2654930200000001, 2024-09-21T14:59:14.461 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035139108000000002, 2024-09-21T14:59:14.461 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:14.461 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:14.461 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:14.461 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:14.461 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:14.461 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45719 2024-09-21T14:59:14.461 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.461 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:14.462 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.462 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:14.462 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141952+0000", 2024-09-21T14:59:14.462 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:14.462 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.462 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.462 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:14.462 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141952+0000", 2024-09-21T14:59:14.462 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:14.462 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.462 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.462 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:14.462 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141952+0000", 2024-09-21T14:59:14.462 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:14.463 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.463 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.463 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:14.463 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141953+0000", 2024-09-21T14:59:14.463 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.486e-06 2024-09-21T14:59:14.463 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.463 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.463 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:14.463 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141959+0000", 2024-09-21T14:59:14.463 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4940000000000003e-06 2024-09-21T14:59:14.463 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.463 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.463 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:14.463 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141963+0000", 2024-09-21T14:59:14.463 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3540000000000002e-06 2024-09-21T14:59:14.464 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.464 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.464 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:14.464 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177084+0000", 2024-09-21T14:59:14.464 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035120983000000001 2024-09-21T14:59:14.464 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.464 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.464 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:14.464 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177091+0000", 2024-09-21T14:59:14.464 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0350000000000004e-06 2024-09-21T14:59:14.464 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:14.464 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:14.464 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:14.465 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.465 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.465 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45720 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:14.465 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141965+0000", 2024-09-21T14:59:14.465 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.2654795529999996, 2024-09-21T14:59:14.465 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035155268000000003, 2024-09-21T14:59:14.465 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:14.465 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:14.465 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:14.465 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:14.465 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:14.465 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45720 2024-09-21T14:59:14.465 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.465 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:14.466 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.466 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:14.466 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141965+0000", 2024-09-21T14:59:14.466 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:14.466 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.466 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.466 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:14.466 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141965+0000", 2024-09-21T14:59:14.466 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:14.466 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.466 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.466 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:14.466 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141965+0000", 2024-09-21T14:59:14.466 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:14.467 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.467 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.467 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:14.467 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141966+0000", 2024-09-21T14:59:14.467 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.46e-06 2024-09-21T14:59:14.467 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.467 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.467 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:14.467 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141972+0000", 2024-09-21T14:59:14.467 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4210000000000003e-06 2024-09-21T14:59:14.467 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.467 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.467 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:14.467 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141975+0000", 2024-09-21T14:59:14.468 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.948e-06 2024-09-21T14:59:14.468 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.468 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.468 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:14.468 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177113+0000", 2024-09-21T14:59:14.468 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035137855000000003 2024-09-21T14:59:14.468 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.468 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.468 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:14.468 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177121+0000", 2024-09-21T14:59:14.468 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8269999999999998e-06 2024-09-21T14:59:14.468 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:14.468 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:14.468 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:14.469 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.469 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.469 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45721 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:14.469 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141977+0000", 2024-09-21T14:59:14.469 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.2654675300000005, 2024-09-21T14:59:14.469 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035169984000000001, 2024-09-21T14:59:14.469 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:14.469 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:14.469 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:14.469 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:14.469 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:14.469 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45721 2024-09-21T14:59:14.469 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.469 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:14.469 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.470 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:14.470 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141977+0000", 2024-09-21T14:59:14.470 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:14.470 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.470 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.470 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:14.470 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141977+0000", 2024-09-21T14:59:14.470 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:14.470 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.470 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.470 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:14.470 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141977+0000", 2024-09-21T14:59:14.470 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:14.470 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.471 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.471 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:14.471 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141979+0000", 2024-09-21T14:59:14.471 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.511e-06 2024-09-21T14:59:14.471 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.471 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.471 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:14.471 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141982+0000", 2024-09-21T14:59:14.471 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4180000000000001e-06 2024-09-21T14:59:14.471 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.471 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.471 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:14.471 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141985+0000", 2024-09-21T14:59:14.472 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9450000000000002e-06 2024-09-21T14:59:14.472 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.472 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.472 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:14.472 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177139+0000", 2024-09-21T14:59:14.472 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035154409999999997 2024-09-21T14:59:14.472 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.472 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.472 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:14.472 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177147+0000", 2024-09-21T14:59:14.472 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9570000000000002e-06 2024-09-21T14:59:14.472 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:14.472 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:14.472 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:14.472 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.473 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.473 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45722 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:14.473 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141987+0000", 2024-09-21T14:59:14.473 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.2654575589999997, 2024-09-21T14:59:14.473 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035188492000000002, 2024-09-21T14:59:14.473 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:14.473 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:14.473 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:14.473 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:14.473 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:14.473 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45722 2024-09-21T14:59:14.473 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.473 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:14.473 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.473 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:14.474 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141987+0000", 2024-09-21T14:59:14.474 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:14.474 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.474 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.474 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:14.474 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141987+0000", 2024-09-21T14:59:14.474 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:14.474 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.474 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.474 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:14.474 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141987+0000", 2024-09-21T14:59:14.474 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:14.474 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.475 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.475 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:14.475 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141988+0000", 2024-09-21T14:59:14.475 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.463e-06 2024-09-21T14:59:14.475 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.475 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.475 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:14.475 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141994+0000", 2024-09-21T14:59:14.475 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.361e-06 2024-09-21T14:59:14.475 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.475 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.475 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:14.475 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141997+0000", 2024-09-21T14:59:14.476 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0359999999999998e-06 2024-09-21T14:59:14.476 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.476 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.476 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:14.476 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177169+0000", 2024-09-21T14:59:14.476 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035171651999999998 2024-09-21T14:59:14.476 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.476 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.476 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:14.476 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177176+0000", 2024-09-21T14:59:14.476 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2189999999999998e-06 2024-09-21T14:59:14.476 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:14.476 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:14.476 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:14.476 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.476 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.477 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45723 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:14.477 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141999+0000", 2024-09-21T14:59:14.477 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.2654453920000002, 2024-09-21T14:59:14.477 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035197701999999997, 2024-09-21T14:59:14.477 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:14.477 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:14.477 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:14.477 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:14.477 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:14.477 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45723 2024-09-21T14:59:14.477 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.477 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:14.477 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.477 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:14.477 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141999+0000", 2024-09-21T14:59:14.478 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:14.478 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.478 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.478 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:14.478 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141999+0000", 2024-09-21T14:59:14.478 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:14.478 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.478 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.478 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:14.478 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141999+0000", 2024-09-21T14:59:14.478 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:14.478 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.478 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.478 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:14.479 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142003+0000", 2024-09-21T14:59:14.479 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5049999999999999e-06 2024-09-21T14:59:14.479 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.479 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.479 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:14.479 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142008+0000", 2024-09-21T14:59:14.479 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6579999999999999e-06 2024-09-21T14:59:14.479 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.479 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.479 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:14.479 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142011+0000", 2024-09-21T14:59:14.479 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0529999999999998e-06 2024-09-21T14:59:14.479 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.479 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.479 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:14.480 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177194+0000", 2024-09-21T14:59:14.480 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035182221999999999 2024-09-21T14:59:14.480 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.480 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.480 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:14.480 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177197+0000", 2024-09-21T14:59:14.480 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5030000000000002e-06 2024-09-21T14:59:14.480 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:14.480 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:14.480 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:14.480 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.480 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.480 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45724 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:14.480 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.142015+0000", 2024-09-21T14:59:14.480 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.2654294249999998, 2024-09-21T14:59:14.481 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035195345000000003, 2024-09-21T14:59:14.481 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:14.481 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:14.481 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:14.481 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:14.481 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:14.481 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45724 2024-09-21T14:59:14.481 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.481 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:14.481 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.481 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:14.481 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142015+0000", 2024-09-21T14:59:14.481 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:14.481 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.482 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.482 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:14.482 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142015+0000", 2024-09-21T14:59:14.482 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:14.482 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.482 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.482 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:14.482 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142015+0000", 2024-09-21T14:59:14.482 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:14.482 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.482 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.482 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:14.482 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142017+0000", 2024-09-21T14:59:14.482 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5540000000000001e-06 2024-09-21T14:59:14.483 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.483 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.483 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:14.483 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142022+0000", 2024-09-21T14:59:14.483 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4229999999999996e-06 2024-09-21T14:59:14.483 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.483 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.483 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:14.483 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142025+0000", 2024-09-21T14:59:14.483 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.056e-06 2024-09-21T14:59:14.483 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.483 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.483 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:14.483 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177207+0000", 2024-09-21T14:59:14.484 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035182089 2024-09-21T14:59:14.484 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.484 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.484 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:14.484 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177211+0000", 2024-09-21T14:59:14.484 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4649999999999999e-06 2024-09-21T14:59:14.484 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:14.484 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:14.484 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:14.484 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.484 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.484 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45725 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:14.484 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.142028+0000", 2024-09-21T14:59:14.484 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.2654171459999999, 2024-09-21T14:59:14.485 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035195731000000001, 2024-09-21T14:59:14.485 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:14.485 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:14.485 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:14.485 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:14.485 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:14.485 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45725 2024-09-21T14:59:14.485 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.485 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:14.485 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.485 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:14.485 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142028+0000", 2024-09-21T14:59:14.485 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:14.485 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.486 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.486 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:14.486 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142028+0000", 2024-09-21T14:59:14.486 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:14.486 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.486 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.486 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:14.486 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142027+0000", 2024-09-21T14:59:14.486 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:14.486 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.486 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.486 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:14.486 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142029+0000", 2024-09-21T14:59:14.486 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5269999999999999e-06 2024-09-21T14:59:14.487 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.487 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.487 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:14.487 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142032+0000", 2024-09-21T14:59:14.487 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3249999999999999e-06 2024-09-21T14:59:14.487 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.487 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.487 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:14.487 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142035+0000", 2024-09-21T14:59:14.487 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.897e-06 2024-09-21T14:59:14.487 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.487 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.487 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:14.487 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177220+0000", 2024-09-21T14:59:14.487 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035184871 2024-09-21T14:59:14.488 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.488 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.488 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:14.488 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177223+0000", 2024-09-21T14:59:14.488 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3680000000000002e-06 2024-09-21T14:59:14.488 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:14.488 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:14.488 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:14.488 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.488 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.488 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45726 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:14.488 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.142037+0000", 2024-09-21T14:59:14.488 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.2654074309999999, 2024-09-21T14:59:14.488 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035199273000000003, 2024-09-21T14:59:14.489 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:14.489 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:14.489 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:14.489 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:14.489 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:14.489 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45726 2024-09-21T14:59:14.489 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.489 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:14.489 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.489 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:14.489 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142037+0000", 2024-09-21T14:59:14.489 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:14.489 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.489 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.490 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:14.490 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142037+0000", 2024-09-21T14:59:14.490 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:14.490 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.490 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.490 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:14.490 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142037+0000", 2024-09-21T14:59:14.490 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:14.490 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.490 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.490 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:14.490 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142038+0000", 2024-09-21T14:59:14.490 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.344e-06 2024-09-21T14:59:14.490 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.491 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.491 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:14.491 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142044+0000", 2024-09-21T14:59:14.491 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1930000000000001e-06 2024-09-21T14:59:14.491 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.491 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.491 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:14.491 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142047+0000", 2024-09-21T14:59:14.491 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0280000000000001e-06 2024-09-21T14:59:14.491 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.491 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.491 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:14.491 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177233+0000", 2024-09-21T14:59:14.491 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035186394000000003 2024-09-21T14:59:14.492 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.492 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.492 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:14.492 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177237+0000", 2024-09-21T14:59:14.492 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5530000000000001e-06 2024-09-21T14:59:14.492 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:14.492 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:14.492 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:14.492 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:14.492 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:14.492 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:14.493 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T14:59:14.494 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:59:14.492+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:59:14.594 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:59:14.593+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:59:14.608 INFO:teuthology.orchestra.run.smithi032.stdout:1399 writesame 0x9b1068 thru 0x9b3734 data_size 0x2b(0x26cd bytes) 2024-09-21T14:59:14.624 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:14.624 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T14:59:14.624 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.624 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:60976 3.16 3:6a9608e8:::rbd_data.11366ec8a7a7.0000000000000026:head [set-alloc-hint object_size 262144 write_size 262144,write 241664~3072 in=3072b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-21T14:59:14.624 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:14.624168+0000", 2024-09-21T14:59:14.625 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.001597467, 2024-09-21T14:59:14.625 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001665492, 2024-09-21T14:59:14.625 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:14.625 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "waiting for sub ops", 2024-09-21T14:59:14.625 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:14.625 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:14.625 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:14.625 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 60976 2024-09-21T14:59:14.625 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.625 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:14.625 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.625 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:14.625 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:14.624168+0000", 2024-09-21T14:59:14.625 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:14.625 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.626 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.626 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:14.626 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:14.624168+0000", 2024-09-21T14:59:14.626 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:14.626 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.626 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.626 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:14.626 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:14.624167+0000", 2024-09-21T14:59:14.626 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:59:14.626 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.626 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.626 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:14.626 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:14.624175+0000", 2024-09-21T14:59:14.627 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2150000000000001e-06 2024-09-21T14:59:14.627 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.627 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.627 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:14.627 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:14.624191+0000", 2024-09-21T14:59:14.627 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5814000000000001e-05 2024-09-21T14:59:14.627 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.627 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.627 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:14.627 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:14.624199+0000", 2024-09-21T14:59:14.627 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.7749999999999998e-06 2024-09-21T14:59:14.627 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.627 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.627 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:14.628 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:14.624323+0000", 2024-09-21T14:59:14.628 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000123862 2024-09-21T14:59:14.628 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.628 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.628 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:14.628 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:14.624593+0000", 2024-09-21T14:59:14.628 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00026975399999999998 2024-09-21T14:59:14.628 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.628 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.628 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 4", 2024-09-21T14:59:14.628 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:14.624869+0000", 2024-09-21T14:59:14.628 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00027575400000000002 2024-09-21T14:59:14.628 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:14.628 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:14.628 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:14.629 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:14.629 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-21T14:59:14.629 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 1 2024-09-21T14:59:14.629 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:14.638 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T14:59:14.644 INFO:teuthology.orchestra.run.smithi032.stdout:1400 writesame 0x789ff46 thru 0x78adcfc data_size 0x1(0xddb7 bytes) 2024-09-21T14:59:14.650 INFO:teuthology.orchestra.run.smithi032.stdout:1401 trunc from 0x78adcfd to 0x3a936a2 2024-09-21T14:59:14.682 INFO:teuthology.orchestra.run.smithi032.stdout:1402 write 0xb52ad60 thru 0xb535d04 (0xafa5 bytes) 2024-09-21T14:59:14.693 INFO:teuthology.orchestra.run.smithi032.stdout:1403 read 0x557fa6d thru 0x558e3fe (0xe992 bytes) 2024-09-21T14:59:14.693 INFO:teuthology.orchestra.run.smithi032.stdout:1406 writesame 0xb4c105d thru 0xb4c205e data_size 0x1(0x1002 bytes) 2024-09-21T14:59:14.695 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:59:14.693+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:59:14.697 INFO:teuthology.orchestra.run.smithi032.stdout:1407 read 0x7fdb201 thru 0x7fddd46 (0x2b46 bytes) 2024-09-21T14:59:14.697 INFO:teuthology.orchestra.run.smithi032.stdout:1409 read 0x597586c thru 0x597970a (0x3e9f bytes) 2024-09-21T14:59:14.712 INFO:teuthology.orchestra.run.smithi032.stdout:1410 trunc from 0xb535d05 to 0x19af653 2024-09-21T14:59:14.727 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T14:59:14.739 INFO:tasks.ceph.mon.a.smithi032.stderr:2024-09-21T14:59:14.737+0000 7fdb7159c700 -1 mon.a@0(leader).osd e77 definitely_dead 0 2024-09-21T14:59:14.744 INFO:teuthology.orchestra.run.smithi032.stdout:1411 trunc from 0x19af653 to 0x81a4bde 2024-09-21T14:59:14.751 INFO:teuthology.orchestra.run.smithi032.stdout:1413 read 0x1c345a8 thru 0x1c35f07 (0x1960 bytes) 2024-09-21T14:59:14.751 INFO:teuthology.orchestra.run.smithi032.stdout:1414 writesame 0x73044da thru 0x730527e data_size 0x1(0xda5 bytes) 2024-09-21T14:59:14.753 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:14.753 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:59:14.754 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T14:59:14.754 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T14:59:14.754 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:14.762 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:14.766 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T14:59:14.767 INFO:teuthology.orchestra.run.smithi032.stdout:1416 trunc from 0x81a4bde to 0x4d42a3a 2024-09-21T14:59:14.804 INFO:teuthology.orchestra.run.smithi032.stdout:1420 write 0xb7e97d3 thru 0xb7f7635 (0xde63 bytes) 2024-09-21T14:59:14.805 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:59:14.805+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:59:14.815 INFO:teuthology.orchestra.run.smithi032.stdout:1421 read 0x2fb3f38 thru 0x2fb9846 (0x590f bytes) 2024-09-21T14:59:14.816 INFO:teuthology.orchestra.run.smithi032.stdout:1423 write 0x6dfc818 thru 0x6e0c512 (0xfcfb bytes) 2024-09-21T14:59:14.826 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:14.837 INFO:teuthology.orchestra.run.smithi032.stdout:1427 trunc from 0xb7f7636 to 0x4f2475e 2024-09-21T14:59:14.880 INFO:teuthology.orchestra.run.smithi032.stdout:1428 writesame 0xcd24ce6 thru 0xcd2ffc2 data_size 0x1(0xb2dd bytes) 2024-09-21T14:59:14.882 INFO:teuthology.orchestra.run.smithi032.stdout:1429 trunc from 0xcd2ffc3 to 0xb005ef5 2024-09-21T14:59:14.891 INFO:teuthology.orchestra.run.smithi032.stdout:1430 read 0x1157e99 thru 0x1167412 (0xf57a bytes) 2024-09-21T14:59:14.892 INFO:teuthology.orchestra.run.smithi032.stdout:1432 write 0x33e7792 thru 0x33f69b6 (0xf225 bytes) 2024-09-21T14:59:14.895 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T14:59:14.897 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:59:14.897+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:59:14.903 INFO:teuthology.orchestra.run.smithi032.stdout:1433 writesame 0x3a7472c thru 0x3a7fb88 data_size 0x1(0xb45d bytes) 2024-09-21T14:59:14.904 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:14.904 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T14:59:14.904 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T14:59:14.904 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T14:59:14.904 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.905 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4281.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-21T14:59:14.905 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:09.324416+0000", 2024-09-21T14:59:14.905 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 65.580077830999997, 2024-09-21T14:59:14.905 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.117837947, 2024-09-21T14:59:14.905 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:14.905 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:14.905 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:14.905 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4281", 2024-09-21T14:59:14.905 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/1518520303", 2024-09-21T14:59:14.905 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1 2024-09-21T14:59:14.905 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.905 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:14.905 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.905 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:14.905 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324416+0000", 2024-09-21T14:59:14.906 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:14.906 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.906 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.906 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:14.906 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324416+0000", 2024-09-21T14:59:14.906 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:14.906 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.906 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.906 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:14.906 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324414+0000", 2024-09-21T14:59:14.906 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T14:59:14.906 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.906 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.906 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:14.907 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324429+0000", 2024-09-21T14:59:14.907 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5637e-05 2024-09-21T14:59:14.907 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.907 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.907 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:14.907 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324448+0000", 2024-09-21T14:59:14.907 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9042000000000002e-05 2024-09-21T14:59:14.907 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.907 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.907 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:14.907 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324465+0000", 2024-09-21T14:59:14.907 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6767999999999999e-05 2024-09-21T14:59:14.907 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.907 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.908 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:14.908 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324571+0000", 2024-09-21T14:59:14.908 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000105714 2024-09-21T14:59:14.908 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.908 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.908 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op must wait for map", 2024-09-21T14:59:14.908 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324587+0000", 2024-09-21T14:59:14.908 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5753999999999999e-05 2024-09-21T14:59:14.908 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.908 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.909 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:14.909 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.325864+0000", 2024-09-21T14:59:14.909 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0012773350000000001 2024-09-21T14:59:14.909 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.909 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.909 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T14:59:14.909 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.325878+0000", 2024-09-21T14:59:14.909 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4498000000000001e-05 2024-09-21T14:59:14.909 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.909 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.909 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:14.909 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.439461+0000", 2024-09-21T14:59:14.909 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.11358212500000001 2024-09-21T14:59:14.909 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.909 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.910 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:14.910 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.439848+0000", 2024-09-21T14:59:14.910 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00038747800000000002 2024-09-21T14:59:14.910 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.910 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.910 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:59:14.910 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.440175+0000", 2024-09-21T14:59:14.910 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00032721900000000001 2024-09-21T14:59:14.910 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.910 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.910 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:14.910 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.441770+0000", 2024-09-21T14:59:14.910 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0015949409999999999 2024-09-21T14:59:14.911 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.911 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.911 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:14.911 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.442086+0000", 2024-09-21T14:59:14.911 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00031586299999999999 2024-09-21T14:59:14.911 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.911 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.911 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:14.911 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.442195+0000", 2024-09-21T14:59:14.911 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000108791 2024-09-21T14:59:14.911 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.911 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.911 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:14.911 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.442254+0000", 2024-09-21T14:59:14.911 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9038e-05 2024-09-21T14:59:14.912 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:14.912 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:14.912 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:14.912 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.912 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.912 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4281.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-21T14:59:14.912 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:09.447828+0000", 2024-09-21T14:59:14.912 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 65.456665830000006, 2024-09-21T14:59:14.912 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.32904393700000001, 2024-09-21T14:59:14.912 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:14.912 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:14.912 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:14.912 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4281", 2024-09-21T14:59:14.912 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/1518520303", 2024-09-21T14:59:14.912 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 2 2024-09-21T14:59:14.913 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.913 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:14.913 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.913 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:14.913 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447828+0000", 2024-09-21T14:59:14.913 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:14.913 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.913 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.913 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:14.913 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447828+0000", 2024-09-21T14:59:14.913 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:14.913 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.913 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.913 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:14.913 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447825+0000", 2024-09-21T14:59:14.914 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T14:59:14.914 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.914 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.914 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:14.914 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447841+0000", 2024-09-21T14:59:14.914 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5359999999999999e-05 2024-09-21T14:59:14.914 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.914 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.914 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:14.914 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447875+0000", 2024-09-21T14:59:14.914 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4388999999999999e-05 2024-09-21T14:59:14.914 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.914 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.915 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:14.915 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447903+0000", 2024-09-21T14:59:14.915 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7373e-05 2024-09-21T14:59:14.915 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.915 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.915 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:14.915 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447996+0000", 2024-09-21T14:59:14.915 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2954999999999993e-05 2024-09-21T14:59:14.915 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.915 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.915 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T14:59:14.915 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.448011+0000", 2024-09-21T14:59:14.915 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5038999999999999e-05 2024-09-21T14:59:14.916 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.916 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.916 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:14.916 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.776501+0000", 2024-09-21T14:59:14.916 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.32849012500000002 2024-09-21T14:59:14.916 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.916 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.916 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:14.916 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.776872+0000", 2024-09-21T14:59:14.916 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00037119600000000002 2024-09-21T14:59:14.916 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:14.916 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:14.916 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:14.916 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.916 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.917 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:752 3.14 3:28fc02e5:::rbd_data.113693316f36.0000000000000010:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-21T14:59:14.917 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:26.625579+0000", 2024-09-21T14:59:14.917 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 48.278914974999999, 2024-09-21T14:59:14.917 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042965426000000001, 2024-09-21T14:59:14.917 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:14.917 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:14.917 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:14.917 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:14.917 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:14.917 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 752 2024-09-21T14:59:14.917 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.917 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:14.917 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.917 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:14.917 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625579+0000", 2024-09-21T14:59:14.918 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:14.918 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.918 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.918 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:14.918 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625579+0000", 2024-09-21T14:59:14.918 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:14.918 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.918 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.918 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:14.918 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625577+0000", 2024-09-21T14:59:14.918 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T14:59:14.918 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.918 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.918 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:14.919 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625588+0000", 2024-09-21T14:59:14.919 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1555e-05 2024-09-21T14:59:14.919 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.919 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.919 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:14.919 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625610+0000", 2024-09-21T14:59:14.919 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2107999999999999e-05 2024-09-21T14:59:14.919 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.919 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.919 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:14.919 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625621+0000", 2024-09-21T14:59:14.919 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0762e-05 2024-09-21T14:59:14.919 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.919 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.919 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:14.920 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625692+0000", 2024-09-21T14:59:14.920 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.1203000000000003e-05 2024-09-21T14:59:14.920 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.920 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.920 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:14.920 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625872+0000", 2024-09-21T14:59:14.920 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017962800000000001 2024-09-21T14:59:14.920 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.920 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.920 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:59:14.920 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.633188+0000", 2024-09-21T14:59:14.920 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.007316014 2024-09-21T14:59:14.920 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.920 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.921 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:14.921 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.668357+0000", 2024-09-21T14:59:14.921 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035168933999999999 2024-09-21T14:59:14.921 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.921 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.921 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:14.921 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.668418+0000", 2024-09-21T14:59:14.921 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1282e-05 2024-09-21T14:59:14.921 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.921 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.921 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:14.921 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.668486+0000", 2024-09-21T14:59:14.921 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8468999999999998e-05 2024-09-21T14:59:14.921 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.922 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.922 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:14.922 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.668544+0000", 2024-09-21T14:59:14.922 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7785000000000003e-05 2024-09-21T14:59:14.922 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:14.922 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:14.922 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:14.922 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.922 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.922 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22737 3.a 3:574c7a60:::rbd_data.11366acc894.0000000000000002:19 [read 1032192~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:14.922 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141028+0000", 2024-09-21T14:59:14.922 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 24.763466009999998, 2024-09-21T14:59:14.923 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035206281999999998, 2024-09-21T14:59:14.923 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:14.923 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:14.923 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:14.923 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:14.923 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:14.923 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22737 2024-09-21T14:59:14.923 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.923 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:14.923 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.923 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:14.923 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141028+0000", 2024-09-21T14:59:14.923 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:14.923 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.923 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.924 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:14.924 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141028+0000", 2024-09-21T14:59:14.924 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:14.924 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.924 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.924 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:14.924 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141027+0000", 2024-09-21T14:59:14.924 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:14.924 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.924 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.924 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:14.924 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141030+0000", 2024-09-21T14:59:14.924 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6189999999999998e-06 2024-09-21T14:59:14.924 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.925 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.925 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:14.925 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141036+0000", 2024-09-21T14:59:14.925 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5110000000000003e-06 2024-09-21T14:59:14.925 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.925 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.925 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:14.925 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141040+0000", 2024-09-21T14:59:14.925 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.34e-06 2024-09-21T14:59:14.925 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.925 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.925 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:14.925 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.175948+0000", 2024-09-21T14:59:14.925 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034908540000000002 2024-09-21T14:59:14.925 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.926 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.926 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:14.926 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176025+0000", 2024-09-21T14:59:14.926 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6412e-05 2024-09-21T14:59:14.926 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.926 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.926 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:14.926 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176234+0000", 2024-09-21T14:59:14.926 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000209247 2024-09-21T14:59:14.926 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:14.926 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:14.926 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:14.926 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.926 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.927 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22738 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 49152~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:14.927 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141049+0000", 2024-09-21T14:59:14.927 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 24.763444832000001, 2024-09-21T14:59:14.927 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035428754999999999, 2024-09-21T14:59:14.927 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:14.927 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:14.927 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:14.927 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:14.927 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:14.927 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22738 2024-09-21T14:59:14.927 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.927 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:14.927 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.927 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:14.928 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141049+0000", 2024-09-21T14:59:14.928 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:14.928 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.928 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.928 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:14.928 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141049+0000", 2024-09-21T14:59:14.928 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:14.928 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.928 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.928 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:14.928 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141049+0000", 2024-09-21T14:59:14.928 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:14.928 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.928 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.929 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:14.929 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141051+0000", 2024-09-21T14:59:14.929 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4980000000000001e-06 2024-09-21T14:59:14.929 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.929 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.929 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:14.929 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141057+0000", 2024-09-21T14:59:14.929 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8030000000000002e-06 2024-09-21T14:59:14.929 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.929 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.929 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:14.929 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141066+0000", 2024-09-21T14:59:14.929 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1160000000000005e-06 2024-09-21T14:59:14.929 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.930 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.930 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:14.930 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176264+0000", 2024-09-21T14:59:14.930 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035197580999999999 2024-09-21T14:59:14.930 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.930 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.930 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:14.930 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176327+0000", 2024-09-21T14:59:14.930 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3237000000000005e-05 2024-09-21T14:59:14.930 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.930 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.930 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:14.930 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176478+0000", 2024-09-21T14:59:14.930 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000150851 2024-09-21T14:59:14.930 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:14.931 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:14.931 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:14.931 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.931 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.931 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22739 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 114688~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:14.931 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141070+0000", 2024-09-21T14:59:14.931 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 24.763423589999999, 2024-09-21T14:59:14.931 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035679094000000001, 2024-09-21T14:59:14.931 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:14.931 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:14.931 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:14.931 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:14.931 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:14.931 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22739 2024-09-21T14:59:14.932 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.932 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:14.932 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.932 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:14.932 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141070+0000", 2024-09-21T14:59:14.932 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:14.932 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.932 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.932 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:14.932 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141070+0000", 2024-09-21T14:59:14.932 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:14.932 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.932 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.932 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:14.933 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141070+0000", 2024-09-21T14:59:14.933 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:14.933 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.933 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.933 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:14.933 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141072+0000", 2024-09-21T14:59:14.933 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.379e-06 2024-09-21T14:59:14.933 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.933 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.933 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:14.933 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141078+0000", 2024-09-21T14:59:14.933 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6710000000000004e-06 2024-09-21T14:59:14.933 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.933 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.933 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:14.934 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141083+0000", 2024-09-21T14:59:14.934 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7249999999999997e-06 2024-09-21T14:59:14.934 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.934 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.934 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:14.934 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176506+0000", 2024-09-21T14:59:14.934 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035423076999999997 2024-09-21T14:59:14.934 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.934 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.934 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:14.934 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176582+0000", 2024-09-21T14:59:14.934 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6305999999999998e-05 2024-09-21T14:59:14.934 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.934 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.935 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:14.935 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176749+0000", 2024-09-21T14:59:14.935 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000167262 2024-09-21T14:59:14.935 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:14.935 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:14.935 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:14.935 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.935 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.935 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22740 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 180224~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:14.935 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141090+0000", 2024-09-21T14:59:14.935 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 24.763404185999999, 2024-09-21T14:59:14.935 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036129535999999997, 2024-09-21T14:59:14.935 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:14.935 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:14.936 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:14.936 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:14.936 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:14.936 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22740 2024-09-21T14:59:14.936 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.936 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:14.936 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.936 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:14.936 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141090+0000", 2024-09-21T14:59:14.936 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:14.936 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.936 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.936 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:14.937 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141090+0000", 2024-09-21T14:59:14.937 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:14.937 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.937 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.937 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:14.937 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141089+0000", 2024-09-21T14:59:14.937 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:14.937 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.937 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.937 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:14.937 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141092+0000", 2024-09-21T14:59:14.937 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7709999999999998e-06 2024-09-21T14:59:14.937 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.937 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.937 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:14.938 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141098+0000", 2024-09-21T14:59:14.938 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.015e-06 2024-09-21T14:59:14.938 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.938 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.938 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:14.938 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141103+0000", 2024-09-21T14:59:14.938 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8629999999999999e-06 2024-09-21T14:59:14.938 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.938 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.938 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:14.938 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176778+0000", 2024-09-21T14:59:14.938 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035674984 2024-09-21T14:59:14.938 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.938 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.939 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:14.939 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176851+0000", 2024-09-21T14:59:14.939 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2861000000000006e-05 2024-09-21T14:59:14.939 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.939 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.939 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:14.939 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177219+0000", 2024-09-21T14:59:14.939 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00036845999999999998 2024-09-21T14:59:14.939 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:14.939 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:14.939 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:14.939 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.939 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.939 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22741 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 245760~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:14.940 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141107+0000", 2024-09-21T14:59:14.940 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 24.763386615000002, 2024-09-21T14:59:14.940 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036407729, 2024-09-21T14:59:14.940 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:14.940 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:14.940 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:14.940 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:14.940 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:14.940 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22741 2024-09-21T14:59:14.940 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.940 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:14.940 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.940 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:14.940 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141107+0000", 2024-09-21T14:59:14.941 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:14.941 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.941 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.941 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:14.941 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141107+0000", 2024-09-21T14:59:14.941 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:14.941 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.941 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.941 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:14.941 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141107+0000", 2024-09-21T14:59:14.941 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:14.941 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.941 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.941 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:14.942 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141109+0000", 2024-09-21T14:59:14.942 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6529999999999998e-06 2024-09-21T14:59:14.942 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.942 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.942 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:14.942 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141115+0000", 2024-09-21T14:59:14.942 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9649999999999996e-06 2024-09-21T14:59:14.942 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.942 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.942 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:14.942 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141121+0000", 2024-09-21T14:59:14.942 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1209999999999998e-06 2024-09-21T14:59:14.942 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.943 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.943 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:14.943 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177242+0000", 2024-09-21T14:59:14.943 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036121706000000003 2024-09-21T14:59:14.943 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.943 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.943 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:14.943 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177301+0000", 2024-09-21T14:59:14.943 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9008000000000002e-05 2024-09-21T14:59:14.943 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.943 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.943 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:14.943 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177515+0000", 2024-09-21T14:59:14.943 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00021362 2024-09-21T14:59:14.943 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:14.944 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:14.944 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:14.944 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.944 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.944 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22742 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 311296~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:14.944 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141126+0000", 2024-09-21T14:59:14.944 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 24.763368261, 2024-09-21T14:59:14.944 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036653537, 2024-09-21T14:59:14.944 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:14.944 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:14.944 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:14.944 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:14.944 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:14.944 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22742 2024-09-21T14:59:14.944 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.945 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:14.945 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.945 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:14.945 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141126+0000", 2024-09-21T14:59:14.945 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:14.945 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.945 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.945 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:14.945 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141126+0000", 2024-09-21T14:59:14.945 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:14.945 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.945 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.945 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:14.945 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141125+0000", 2024-09-21T14:59:14.946 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:14.946 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.946 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.946 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:14.946 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141128+0000", 2024-09-21T14:59:14.946 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.886e-06 2024-09-21T14:59:14.946 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.946 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.946 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:14.946 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141137+0000", 2024-09-21T14:59:14.946 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4399999999999994e-06 2024-09-21T14:59:14.946 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.946 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.946 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:14.947 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141143+0000", 2024-09-21T14:59:14.947 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1410000000000004e-06 2024-09-21T14:59:14.947 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.947 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.947 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:14.947 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177543+0000", 2024-09-21T14:59:14.947 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036400544999999999 2024-09-21T14:59:14.947 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.947 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.947 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:14.947 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177605+0000", 2024-09-21T14:59:14.947 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2206999999999996e-05 2024-09-21T14:59:14.947 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.947 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.947 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:14.948 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177779+0000", 2024-09-21T14:59:14.948 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017368099999999999 2024-09-21T14:59:14.948 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:14.948 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:14.948 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:14.948 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.948 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.948 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22743 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 376832~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:14.948 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141147+0000", 2024-09-21T14:59:14.948 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 24.763346776999999, 2024-09-21T14:59:14.948 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036915158000000003, 2024-09-21T14:59:14.948 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:14.948 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:14.948 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:14.949 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:14.949 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:14.949 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22743 2024-09-21T14:59:14.949 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.949 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:14.949 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.949 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:14.949 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141147+0000", 2024-09-21T14:59:14.949 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:14.949 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.949 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.949 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:14.950 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141147+0000", 2024-09-21T14:59:14.950 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:14.950 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.950 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.950 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:14.950 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141147+0000", 2024-09-21T14:59:14.950 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:14.950 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.950 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.950 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:14.950 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141149+0000", 2024-09-21T14:59:14.950 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.79e-06 2024-09-21T14:59:14.950 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.950 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.951 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:14.951 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141155+0000", 2024-09-21T14:59:14.951 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6699999999999999e-06 2024-09-21T14:59:14.951 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.951 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.951 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:14.951 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141160+0000", 2024-09-21T14:59:14.951 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5759999999999999e-06 2024-09-21T14:59:14.951 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.951 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.951 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:14.951 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177817+0000", 2024-09-21T14:59:14.951 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036657404999999997 2024-09-21T14:59:14.951 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.952 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.952 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:14.952 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177892+0000", 2024-09-21T14:59:14.952 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4602000000000002e-05 2024-09-21T14:59:14.952 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.952 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.952 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:14.952 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178062+0000", 2024-09-21T14:59:14.952 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000170464 2024-09-21T14:59:14.952 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:14.952 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:14.952 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:14.953 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.953 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.953 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22744 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 442368~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:14.953 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141164+0000", 2024-09-21T14:59:14.953 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 24.763329643999999, 2024-09-21T14:59:14.953 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037120806999999999, 2024-09-21T14:59:14.953 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:14.953 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:14.953 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:14.953 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:14.953 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:14.953 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22744 2024-09-21T14:59:14.953 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.953 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:14.954 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.954 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:14.954 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141164+0000", 2024-09-21T14:59:14.954 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:14.954 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.954 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.954 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:14.954 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141164+0000", 2024-09-21T14:59:14.954 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:14.954 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.954 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.954 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:14.954 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141164+0000", 2024-09-21T14:59:14.954 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:14.955 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.955 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.955 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:14.955 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141167+0000", 2024-09-21T14:59:14.955 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7159999999999999e-06 2024-09-21T14:59:14.955 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.955 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.955 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:14.955 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141173+0000", 2024-09-21T14:59:14.955 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.071e-06 2024-09-21T14:59:14.955 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.955 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.955 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:14.956 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141177+0000", 2024-09-21T14:59:14.956 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3610000000000003e-06 2024-09-21T14:59:14.956 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.956 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.956 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:14.956 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178083+0000", 2024-09-21T14:59:14.956 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036906405000000003 2024-09-21T14:59:14.956 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.956 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.956 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:14.956 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178143+0000", 2024-09-21T14:59:14.956 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0056999999999998e-05 2024-09-21T14:59:14.956 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.956 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.957 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:14.957 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178285+0000", 2024-09-21T14:59:14.957 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000141555 2024-09-21T14:59:14.957 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:14.957 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:14.957 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:14.957 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.957 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.957 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22745 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 507904~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:14.957 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141182+0000", 2024-09-21T14:59:14.957 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 24.763311324, 2024-09-21T14:59:14.957 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037332331000000003, 2024-09-21T14:59:14.957 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:14.957 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:14.957 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:14.958 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:14.958 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:14.958 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22745 2024-09-21T14:59:14.958 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.958 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:14.958 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.958 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:14.958 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141182+0000", 2024-09-21T14:59:14.958 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:14.958 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.958 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.958 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:14.958 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141182+0000", 2024-09-21T14:59:14.958 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:14.959 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.959 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.959 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:14.959 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141182+0000", 2024-09-21T14:59:14.959 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:14.959 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.959 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.959 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:14.959 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141185+0000", 2024-09-21T14:59:14.959 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.464e-06 2024-09-21T14:59:14.959 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.959 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.959 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:14.959 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141200+0000", 2024-09-21T14:59:14.959 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5886999999999999e-05 2024-09-21T14:59:14.960 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.960 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.960 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:14.960 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141571+0000", 2024-09-21T14:59:14.960 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000370909 2024-09-21T14:59:14.960 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.960 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.960 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:14.960 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178318+0000", 2024-09-21T14:59:14.960 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036746244999999997 2024-09-21T14:59:14.960 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.960 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.960 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:14.960 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178367+0000", 2024-09-21T14:59:14.961 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9317000000000002e-05 2024-09-21T14:59:14.961 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.961 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.961 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:14.961 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178515+0000", 2024-09-21T14:59:14.961 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014784600000000001 2024-09-21T14:59:14.961 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:14.961 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:14.961 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:14.961 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.961 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.961 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22746 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 573440~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:14.961 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141599+0000", 2024-09-21T14:59:14.962 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 24.762894692, 2024-09-21T14:59:14.962 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037402249999999998, 2024-09-21T14:59:14.962 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:14.962 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:14.962 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:14.962 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:14.962 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:14.962 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22746 2024-09-21T14:59:14.962 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.962 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:14.962 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.963 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:14.963 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141599+0000", 2024-09-21T14:59:14.963 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:14.963 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.963 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.963 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:14.963 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141599+0000", 2024-09-21T14:59:14.963 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:14.963 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.963 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.963 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:14.963 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141598+0000", 2024-09-21T14:59:14.963 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:59:14.963 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.964 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.964 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:14.964 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141637+0000", 2024-09-21T14:59:14.964 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.9119999999999998e-05 2024-09-21T14:59:14.964 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.964 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.964 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:14.964 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141647+0000", 2024-09-21T14:59:14.964 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5570000000000001e-06 2024-09-21T14:59:14.964 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.964 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.964 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:14.964 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141654+0000", 2024-09-21T14:59:14.964 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2049999999999996e-06 2024-09-21T14:59:14.965 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.965 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.965 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:14.965 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178553+0000", 2024-09-21T14:59:14.965 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036898528 2024-09-21T14:59:14.965 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.965 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.965 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:14.965 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178625+0000", 2024-09-21T14:59:14.965 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2669999999999994e-05 2024-09-21T14:59:14.965 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.965 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.965 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:14.965 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179001+0000", 2024-09-21T14:59:14.966 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000375977 2024-09-21T14:59:14.966 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:14.966 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:14.966 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:14.966 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.966 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.966 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22747 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 638976~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:14.966 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141660+0000", 2024-09-21T14:59:14.966 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 24.76283355, 2024-09-21T14:59:14.966 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037617352999999999, 2024-09-21T14:59:14.966 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:14.966 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:14.966 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:14.966 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:14.966 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:14.967 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22747 2024-09-21T14:59:14.967 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.967 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:14.967 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.967 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:14.967 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141660+0000", 2024-09-21T14:59:14.967 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:14.967 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.967 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.968 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:14.968 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141660+0000", 2024-09-21T14:59:14.968 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:14.968 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.968 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.968 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:14.968 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141660+0000", 2024-09-21T14:59:14.968 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:14.968 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.968 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.968 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:14.968 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141663+0000", 2024-09-21T14:59:14.968 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.569e-06 2024-09-21T14:59:14.969 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.969 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.969 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:14.969 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141670+0000", 2024-09-21T14:59:14.969 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5719999999999999e-06 2024-09-21T14:59:14.969 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.969 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.969 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:14.969 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141675+0000", 2024-09-21T14:59:14.969 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2889999999999997e-06 2024-09-21T14:59:14.969 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.969 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.969 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:14.969 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179028+0000", 2024-09-21T14:59:14.970 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037353381999999997 2024-09-21T14:59:14.970 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.970 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.970 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:14.970 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179099+0000", 2024-09-21T14:59:14.970 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0259000000000006e-05 2024-09-21T14:59:14.970 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.970 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.970 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:14.970 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179278+0000", 2024-09-21T14:59:14.970 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000178908 2024-09-21T14:59:14.970 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:14.970 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:14.970 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:14.971 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.971 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.971 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22748 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 704512~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:14.971 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141680+0000", 2024-09-21T14:59:14.971 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 24.762813911999999, 2024-09-21T14:59:14.971 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037806166000000002, 2024-09-21T14:59:14.971 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:14.971 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:14.971 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:14.971 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:14.971 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:14.971 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22748 2024-09-21T14:59:14.971 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.971 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:14.972 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.972 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:14.972 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141680+0000", 2024-09-21T14:59:14.972 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:14.972 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.972 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.972 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:14.972 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141680+0000", 2024-09-21T14:59:14.972 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:14.972 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.972 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.972 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:14.972 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141679+0000", 2024-09-21T14:59:14.972 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:14.972 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.973 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.973 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:14.973 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141683+0000", 2024-09-21T14:59:14.973 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3230000000000002e-06 2024-09-21T14:59:14.973 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.973 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.973 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:14.973 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141690+0000", 2024-09-21T14:59:14.973 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8680000000000001e-06 2024-09-21T14:59:14.973 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.973 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.973 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:14.973 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141694+0000", 2024-09-21T14:59:14.974 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6770000000000003e-06 2024-09-21T14:59:14.974 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.974 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.974 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:14.974 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179300+0000", 2024-09-21T14:59:14.974 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037605567999999999 2024-09-21T14:59:14.974 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.974 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.974 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:14.974 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179355+0000", 2024-09-21T14:59:14.974 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4771000000000001e-05 2024-09-21T14:59:14.974 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.974 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.975 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:14.975 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179486+0000", 2024-09-21T14:59:14.975 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013141699999999999 2024-09-21T14:59:14.975 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:14.975 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:14.975 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:14.975 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.975 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.975 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22749 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 770048~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:14.975 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141698+0000", 2024-09-21T14:59:14.975 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 24.762795539999999, 2024-09-21T14:59:14.975 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037967813000000003, 2024-09-21T14:59:14.975 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:14.975 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:14.976 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:14.976 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:14.976 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:14.976 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22749 2024-09-21T14:59:14.976 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.976 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:14.976 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.976 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:14.976 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141698+0000", 2024-09-21T14:59:14.976 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:14.976 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.976 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.976 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:14.976 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141698+0000", 2024-09-21T14:59:14.977 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:14.977 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.977 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.977 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:14.977 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141698+0000", 2024-09-21T14:59:14.977 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:14.977 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.977 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.977 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:14.977 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141701+0000", 2024-09-21T14:59:14.977 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8399999999999999e-06 2024-09-21T14:59:14.977 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.977 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.977 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:14.977 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141706+0000", 2024-09-21T14:59:14.978 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4979999999999997e-06 2024-09-21T14:59:14.978 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.978 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.978 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:14.978 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141711+0000", 2024-09-21T14:59:14.978 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3850000000000004e-06 2024-09-21T14:59:14.978 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.978 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.978 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:14.978 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179504+0000", 2024-09-21T14:59:14.978 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037793514 2024-09-21T14:59:14.978 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.978 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.978 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:14.979 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179546+0000", 2024-09-21T14:59:14.979 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1467999999999997e-05 2024-09-21T14:59:14.979 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.979 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.979 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:14.979 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179666+0000", 2024-09-21T14:59:14.979 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000120499 2024-09-21T14:59:14.979 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:14.979 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:14.979 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:14.979 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.979 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.979 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22750 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 835584~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:14.979 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141715+0000", 2024-09-21T14:59:14.980 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 24.762779239, 2024-09-21T14:59:14.980 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038173179000000002, 2024-09-21T14:59:14.980 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:14.980 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:14.980 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:14.980 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:14.980 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:14.980 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22750 2024-09-21T14:59:14.980 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.980 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:14.980 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.980 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:14.981 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141715+0000", 2024-09-21T14:59:14.981 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:14.981 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.981 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.981 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:14.981 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141715+0000", 2024-09-21T14:59:14.981 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:14.981 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.981 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.981 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:14.981 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141714+0000", 2024-09-21T14:59:14.981 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:14.981 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.981 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.982 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:14.982 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141716+0000", 2024-09-21T14:59:14.982 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1730000000000002e-06 2024-09-21T14:59:14.982 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.982 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.982 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:14.982 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141722+0000", 2024-09-21T14:59:14.982 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.417e-06 2024-09-21T14:59:14.982 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.982 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.982 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:14.982 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141726+0000", 2024-09-21T14:59:14.982 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4830000000000001e-06 2024-09-21T14:59:14.982 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.982 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.983 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:14.983 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179683+0000", 2024-09-21T14:59:14.983 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037956977000000003 2024-09-21T14:59:14.983 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.983 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.983 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:14.983 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179751+0000", 2024-09-21T14:59:14.983 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7762000000000006e-05 2024-09-21T14:59:14.983 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.983 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.983 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:14.983 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179888+0000", 2024-09-21T14:59:14.983 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000136635 2024-09-21T14:59:14.983 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:14.984 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:14.984 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:14.984 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.984 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.984 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22751 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 901120~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:14.984 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141730+0000", 2024-09-21T14:59:14.984 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 24.762763440000001, 2024-09-21T14:59:14.984 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038357011000000003, 2024-09-21T14:59:14.984 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:14.984 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:14.984 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:14.984 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:14.984 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:14.984 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22751 2024-09-21T14:59:14.984 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.985 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:14.985 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.985 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:14.985 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141730+0000", 2024-09-21T14:59:14.985 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:14.985 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.985 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.985 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:14.985 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141730+0000", 2024-09-21T14:59:14.985 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:14.985 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.985 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.985 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:14.985 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141730+0000", 2024-09-21T14:59:14.986 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:14.986 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.986 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.986 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:14.986 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141737+0000", 2024-09-21T14:59:14.986 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.64e-06 2024-09-21T14:59:14.986 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.986 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.986 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:14.986 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141743+0000", 2024-09-21T14:59:14.986 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8130000000000001e-06 2024-09-21T14:59:14.986 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.986 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.987 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:14.987 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141747+0000", 2024-09-21T14:59:14.987 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4920000000000004e-06 2024-09-21T14:59:14.987 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.987 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.987 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:14.987 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179918+0000", 2024-09-21T14:59:14.987 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038170822 2024-09-21T14:59:14.987 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.987 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.987 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:14.987 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179960+0000", 2024-09-21T14:59:14.987 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1999000000000002e-05 2024-09-21T14:59:14.987 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.987 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.988 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:14.988 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180087+0000", 2024-09-21T14:59:14.988 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012752799999999999 2024-09-21T14:59:14.988 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:14.988 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:14.988 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:14.988 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.988 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.988 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22752 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 966656~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:14.988 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141751+0000", 2024-09-21T14:59:14.988 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 24.76274312, 2024-09-21T14:59:14.988 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038553084000000001, 2024-09-21T14:59:14.988 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:14.988 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:14.989 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:14.989 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:14.989 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:14.989 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22752 2024-09-21T14:59:14.989 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.989 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:14.989 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.989 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:14.989 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141751+0000", 2024-09-21T14:59:14.989 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:14.989 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.989 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.989 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:14.989 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141751+0000", 2024-09-21T14:59:14.989 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:14.990 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.990 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.990 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:14.990 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141750+0000", 2024-09-21T14:59:14.990 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:14.990 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.990 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.990 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:14.990 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141753+0000", 2024-09-21T14:59:14.990 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5900000000000002e-06 2024-09-21T14:59:14.990 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.990 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.990 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:14.990 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141759+0000", 2024-09-21T14:59:14.991 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.947e-06 2024-09-21T14:59:14.991 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.991 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.991 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:14.991 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141764+0000", 2024-09-21T14:59:14.991 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8319999999999996e-06 2024-09-21T14:59:14.991 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.991 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.991 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:14.991 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180108+0000", 2024-09-21T14:59:14.991 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038344095000000002 2024-09-21T14:59:14.991 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.991 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.991 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:14.992 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180168+0000", 2024-09-21T14:59:14.992 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0047000000000003e-05 2024-09-21T14:59:14.992 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.992 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.992 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:14.992 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180304+0000", 2024-09-21T14:59:14.992 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013592500000000001 2024-09-21T14:59:14.992 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:14.992 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:14.993 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:14.993 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.993 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.993 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22753 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 1032192~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:14.993 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141768+0000", 2024-09-21T14:59:14.993 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 24.762726246, 2024-09-21T14:59:14.993 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038818810000000002, 2024-09-21T14:59:14.993 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:14.993 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:14.993 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:14.993 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:14.993 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:14.993 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22753 2024-09-21T14:59:14.993 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.993 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:14.994 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.994 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:14.994 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141768+0000", 2024-09-21T14:59:14.994 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:14.994 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.994 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.994 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:14.994 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141768+0000", 2024-09-21T14:59:14.994 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:14.994 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.994 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.994 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:14.994 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141767+0000", 2024-09-21T14:59:14.994 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:14.995 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.995 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.995 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:14.995 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141770+0000", 2024-09-21T14:59:14.995 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3329999999999999e-06 2024-09-21T14:59:14.995 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.995 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.995 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:14.995 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141775+0000", 2024-09-21T14:59:14.995 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2669999999999997e-06 2024-09-21T14:59:14.995 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.995 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.995 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:14.995 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141779+0000", 2024-09-21T14:59:14.996 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2259999999999999e-06 2024-09-21T14:59:14.996 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.996 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.996 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:14.996 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180325+0000", 2024-09-21T14:59:14.996 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038546428000000001 2024-09-21T14:59:14.996 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.996 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.996 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:14.996 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180401+0000", 2024-09-21T14:59:14.996 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5437000000000003e-05 2024-09-21T14:59:14.996 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:14.996 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:14.996 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:14.997 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180586+0000", 2024-09-21T14:59:14.997 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000185429 2024-09-21T14:59:14.997 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:14.997 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:14.997 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:14.997 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:14.997 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:14.997 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:14.999 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T14:59:14.999 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T14:59:14.999 INFO:teuthology.orchestra.run.smithi032.stdout:1434 punch from 0x4c4dbe9 to 0x4c570c3, (0x94da bytes) 2024-09-21T14:59:14.999 INFO:teuthology.orchestra.run.smithi032.stdout:1437 writesame 0x96867d7 thru 0x9691dca data_size 0xa(0xb5f4 bytes) 2024-09-21T14:59:15.000 INFO:teuthology.orchestra.run.smithi032.stdout:1441 writesame 0x2f5fe35 thru 0x2f686a6 data_size 0x23(0x8872 bytes) 2024-09-21T14:59:15.000 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T14:59:15.000 INFO:teuthology.orchestra.run.smithi032.stdout:1442 read 0x4f9b651 thru 0x4fa4ef0 (0x98a0 bytes) 2024-09-21T14:59:15.000 INFO:teuthology.orchestra.run.smithi032.stdout:1443 read 0x2a7984f thru 0x2a7df69 (0x471b bytes) 2024-09-21T14:59:15.000 INFO:teuthology.orchestra.run.smithi032.stdout:1446 clone 14 order 20 su 524288 sc 13 2024-09-21T14:59:15.001 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T14:59:15.001 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-21T14:59:15.001 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:59:14.996+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:59:15.003 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T14:59:15.028 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T14:59:15.052 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T14:59:15.055 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:59:15.055 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:59:15.055 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:59:15.055 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:59:15.055 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:59:15.055 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:59:15.055 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:59:15.055 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:59:15.099 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:59:15.096+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:59:15.121 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:15.122 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:15.130 INFO:teuthology.orchestra.run.smithi032.stderr:marked out osd.4. 2024-09-21T14:59:15.134 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T14:59:15.200 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:59:15.197+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:59:15.233 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:15.233 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:15.243 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T14:59:15.301 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:59:15.296+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:59:15.355 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:15.355 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:15.365 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T14:59:15.409 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:59:15.408+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:59:15.476 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:15.476 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:15.486 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T14:59:15.503 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:59:15.499+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:59:15.589 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:15.590 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:15.599 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T14:59:15.604 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:59:15.601+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:59:15.703 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:15.704 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:15.704 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:59:15.699+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:59:15.713 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T14:59:15.805 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:59:15.799+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:59:15.825 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:15.826 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:15.835 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T14:59:15.906 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:59:15.903+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:59:15.947 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:15.947 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:15.957 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T14:59:16.007 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:59:16.005+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:59:16.066 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:16.067 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:16.076 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T14:59:16.108 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:59:16.103+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:59:16.155 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T14:59:16.192 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:16.192 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:16.202 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T14:59:16.208 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:59:16.203+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:59:16.305 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:16.305 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:16.309 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:59:16.307+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:59:16.315 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T14:59:16.410 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:59:16.407+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:59:16.471 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:16.471 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:16.481 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T14:59:16.511 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:59:16.509+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:59:16.591 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:16.591 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:16.601 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T14:59:16.613 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:59:16.607+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:59:16.710 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:16.710 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:16.714 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:59:16.711+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:59:16.720 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T14:59:16.815 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:59:16.811+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:59:16.830 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:16.830 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:16.840 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T14:59:16.916 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:59:16.911+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:59:16.951 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:16.951 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:16.962 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T14:59:17.017 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:59:17.017+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:59:17.068 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T14:59:17.069 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:17.069 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:17.079 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T14:59:17.118 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:59:17.115+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:59:17.131 INFO:teuthology.orchestra.run.smithi032.stderr:noscrub is set 2024-09-21T14:59:17.219 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:59:17.217+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:59:17.232 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:17.232 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:17.245 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T14:59:17.246 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T14:59:17.274 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T14:59:17.295 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T14:59:17.320 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:59:17.317+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:59:17.330 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T14:59:17.366 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-21T14:59:17.383 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:17.383 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:59:17.383 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T14:59:17.383 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:17.393 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T14:59:17.399 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T14:59:17.421 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:59:17.415+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:59:17.426 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T14:59:17.450 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T14:59:17.453 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:59:17.453 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:59:17.453 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:59:17.453 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:59:17.453 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:59:17.453 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:59:17.453 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:59:17.453 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:59:17.506 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:17.506 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:59:17.506 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T14:59:17.506 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T14:59:17.507 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:17.517 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T14:59:17.521 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:59:17.519+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:59:17.622 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:59:17.621+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:59:17.635 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:17.635 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T14:59:17.635 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T14:59:17.635 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T14:59:17.635 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.635 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:15130 3.2 3:47f1bbdb:::rbd_data.11361bed1f78.0000000000000067:head [sparse-read 0~147456] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-21T14:59:17.635 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:40.288284+0000", 2024-09-21T14:59:17.635 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 37.345827941000003, 2024-09-21T14:59:17.636 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1142935039999999, 2024-09-21T14:59:17.636 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:17.636 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:17.636 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:17.636 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:17.636 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:17.636 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 15130 2024-09-21T14:59:17.636 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.636 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:17.636 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.636 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:17.636 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288284+0000", 2024-09-21T14:59:17.636 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:17.636 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.636 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.636 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:17.637 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288284+0000", 2024-09-21T14:59:17.637 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:17.637 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.637 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.637 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:17.637 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288284+0000", 2024-09-21T14:59:17.637 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:17.637 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.637 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.637 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:17.637 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288288+0000", 2024-09-21T14:59:17.637 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5829999999999998e-06 2024-09-21T14:59:17.637 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.637 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.637 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:17.638 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288297+0000", 2024-09-21T14:59:17.638 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0669999999999998e-06 2024-09-21T14:59:17.638 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.638 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.638 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:17.638 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288305+0000", 2024-09-21T14:59:17.638 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8860000000000004e-06 2024-09-21T14:59:17.638 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.638 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.638 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:17.638 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288417+0000", 2024-09-21T14:59:17.638 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000112952 2024-09-21T14:59:17.638 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.638 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.638 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T14:59:17.638 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288428+0000", 2024-09-21T14:59:17.639 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0631e-05 2024-09-21T14:59:17.639 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.639 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.639 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:17.639 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.401876+0000", 2024-09-21T14:59:17.639 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1134479610000001 2024-09-21T14:59:17.639 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.639 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.639 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:17.639 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.402076+0000", 2024-09-21T14:59:17.639 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00019944200000000001 2024-09-21T14:59:17.639 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.639 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.639 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:17.639 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.402578+0000", 2024-09-21T14:59:17.639 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00050242899999999998 2024-09-21T14:59:17.640 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:17.640 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:17.640 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:17.640 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.640 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.640 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45708 3.2 3:415db91a:::rbd_data.1136ecbe5879.0000000000000000:28 [read 140288~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:17.640 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141801+0000", 2024-09-21T14:59:17.640 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 7.4923117470000005, 2024-09-21T14:59:17.640 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034274200999999997, 2024-09-21T14:59:17.640 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:17.640 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:17.640 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:17.640 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:17.640 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:17.640 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45708 2024-09-21T14:59:17.641 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.641 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:17.641 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.641 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:17.641 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141801+0000", 2024-09-21T14:59:17.641 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:17.641 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.641 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.641 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:17.641 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141801+0000", 2024-09-21T14:59:17.641 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:17.641 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.641 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.642 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:17.642 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141800+0000", 2024-09-21T14:59:17.642 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:17.642 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.642 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.642 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:17.642 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141803+0000", 2024-09-21T14:59:17.642 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4210000000000002e-06 2024-09-21T14:59:17.642 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.642 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.642 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:17.642 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141812+0000", 2024-09-21T14:59:17.642 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.9800000000000004e-06 2024-09-21T14:59:17.642 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.642 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.643 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:17.643 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141816+0000", 2024-09-21T14:59:17.643 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4769999999999997e-06 2024-09-21T14:59:17.643 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.643 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.643 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:17.643 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.175865+0000", 2024-09-21T14:59:17.643 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034048875999999999 2024-09-21T14:59:17.643 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.643 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.643 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:17.643 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.175943+0000", 2024-09-21T14:59:17.643 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7464000000000002e-05 2024-09-21T14:59:17.643 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.643 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.644 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:17.644 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176075+0000", 2024-09-21T14:59:17.644 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013230800000000001 2024-09-21T14:59:17.644 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:17.644 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:17.644 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:17.644 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.644 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.644 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45709 3.2 3:415db91a:::rbd_data.1136ecbe5879.0000000000000000:28 [read 143360~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:17.644 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141820+0000", 2024-09-21T14:59:17.644 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 7.4922918999999997, 2024-09-21T14:59:17.644 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034537536000000001, 2024-09-21T14:59:17.644 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:17.644 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:17.644 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:17.644 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:17.645 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:17.645 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45709 2024-09-21T14:59:17.645 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.645 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:17.645 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.645 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:17.645 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141820+0000", 2024-09-21T14:59:17.645 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:17.645 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.645 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.645 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:17.645 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141820+0000", 2024-09-21T14:59:17.645 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:17.645 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.645 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.645 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:17.645 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141820+0000", 2024-09-21T14:59:17.646 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:17.646 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.646 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.646 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:17.646 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141823+0000", 2024-09-21T14:59:17.646 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5110000000000002e-06 2024-09-21T14:59:17.646 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.646 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.646 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:17.646 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141829+0000", 2024-09-21T14:59:17.646 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0850000000000002e-06 2024-09-21T14:59:17.646 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.646 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.646 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:17.647 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141833+0000", 2024-09-21T14:59:17.647 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7620000000000003e-06 2024-09-21T14:59:17.647 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.647 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.647 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:17.647 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176103+0000", 2024-09-21T14:59:17.647 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034269933000000002 2024-09-21T14:59:17.647 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.647 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.647 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:17.647 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176161+0000", 2024-09-21T14:59:17.647 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7658999999999998e-05 2024-09-21T14:59:17.647 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.647 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.647 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:17.648 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176358+0000", 2024-09-21T14:59:17.648 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000196937 2024-09-21T14:59:17.648 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:17.648 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:17.648 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:17.648 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.648 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.648 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45710 3.2 3:415db91a:::rbd_data.1136ecbe5879.0000000000000000:28 [read 146432~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:17.648 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141838+0000", 2024-09-21T14:59:17.648 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 7.4922747369999998, 2024-09-21T14:59:17.648 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034953674999999997, 2024-09-21T14:59:17.648 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:17.648 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:17.648 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:17.648 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:17.649 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:17.649 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45710 2024-09-21T14:59:17.649 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.649 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:17.649 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.649 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:17.649 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141838+0000", 2024-09-21T14:59:17.649 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:17.649 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.649 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.649 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:17.649 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141838+0000", 2024-09-21T14:59:17.649 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:17.649 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.649 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.649 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:17.650 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141837+0000", 2024-09-21T14:59:17.650 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:17.650 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.650 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.650 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:17.650 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141843+0000", 2024-09-21T14:59:17.650 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1619999999999996e-06 2024-09-21T14:59:17.650 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.650 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.650 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:17.650 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141850+0000", 2024-09-21T14:59:17.650 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4799999999999998e-06 2024-09-21T14:59:17.650 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.650 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.650 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:17.650 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141853+0000", 2024-09-21T14:59:17.651 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3050000000000001e-06 2024-09-21T14:59:17.651 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.651 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.651 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:17.651 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176490+0000", 2024-09-21T14:59:17.651 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034636413999999997 2024-09-21T14:59:17.651 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.651 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.651 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:17.651 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176606+0000", 2024-09-21T14:59:17.651 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000115841 2024-09-21T14:59:17.651 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.651 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.651 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:17.651 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176791+0000", 2024-09-21T14:59:17.652 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00018581500000000001 2024-09-21T14:59:17.652 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:17.652 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:17.652 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:17.652 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.652 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.652 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45711 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:17.652 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141856+0000", 2024-09-21T14:59:17.652 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 7.4922563999999996, 2024-09-21T14:59:17.652 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034986308000000001, 2024-09-21T14:59:17.652 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:17.652 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:17.652 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:17.652 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:17.652 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:17.653 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45711 2024-09-21T14:59:17.653 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.653 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:17.653 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.653 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:17.653 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141856+0000", 2024-09-21T14:59:17.653 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:17.653 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.653 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.653 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:17.653 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141856+0000", 2024-09-21T14:59:17.653 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:17.653 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.653 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.654 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:17.654 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141856+0000", 2024-09-21T14:59:17.654 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:17.654 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.654 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.654 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:17.654 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141862+0000", 2024-09-21T14:59:17.654 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.567e-06 2024-09-21T14:59:17.654 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.654 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.654 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:17.654 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141868+0000", 2024-09-21T14:59:17.654 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9569999999999999e-06 2024-09-21T14:59:17.654 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.654 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.654 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:17.655 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141871+0000", 2024-09-21T14:59:17.655 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.146e-06 2024-09-21T14:59:17.655 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.655 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.655 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:17.655 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176831+0000", 2024-09-21T14:59:17.655 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034959935999999997 2024-09-21T14:59:17.655 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.655 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.655 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:17.655 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176842+0000", 2024-09-21T14:59:17.655 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1008e-05 2024-09-21T14:59:17.655 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:17.655 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:17.655 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:17.655 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.655 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.656 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45712 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:17.656 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141874+0000", 2024-09-21T14:59:17.656 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 7.4922385049999995, 2024-09-21T14:59:17.656 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034993511999999997, 2024-09-21T14:59:17.656 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:17.656 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:17.656 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:17.656 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:17.656 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:17.656 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45712 2024-09-21T14:59:17.656 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.656 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:17.656 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.656 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:17.656 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141874+0000", 2024-09-21T14:59:17.656 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:17.657 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.657 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.657 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:17.657 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141874+0000", 2024-09-21T14:59:17.657 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:17.657 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.657 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.657 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:17.657 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141874+0000", 2024-09-21T14:59:17.657 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:17.657 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.657 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.657 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:17.657 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141875+0000", 2024-09-21T14:59:17.657 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5430000000000001e-06 2024-09-21T14:59:17.657 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.658 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.658 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:17.658 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141878+0000", 2024-09-21T14:59:17.658 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1420000000000002e-06 2024-09-21T14:59:17.658 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.658 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.658 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:17.658 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141881+0000", 2024-09-21T14:59:17.658 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.807e-06 2024-09-21T14:59:17.658 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.658 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.658 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:17.658 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176860+0000", 2024-09-21T14:59:17.658 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034979243 2024-09-21T14:59:17.658 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.659 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.659 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:17.659 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176867+0000", 2024-09-21T14:59:17.659 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0169999999999999e-06 2024-09-21T14:59:17.659 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:17.659 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:17.659 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:17.659 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.659 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.659 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45713 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:17.659 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141883+0000", 2024-09-21T14:59:17.659 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 7.4922289299999996, 2024-09-21T14:59:17.659 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035010516999999998, 2024-09-21T14:59:17.659 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:17.659 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:17.659 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:17.659 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:17.660 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:17.660 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45713 2024-09-21T14:59:17.660 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.660 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:17.660 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.660 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:17.660 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141883+0000", 2024-09-21T14:59:17.660 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:17.660 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.660 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.660 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:17.660 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141883+0000", 2024-09-21T14:59:17.660 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:17.660 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.660 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.660 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:17.661 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141883+0000", 2024-09-21T14:59:17.661 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:17.661 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.661 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.661 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:17.661 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141885+0000", 2024-09-21T14:59:17.661 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4649999999999999e-06 2024-09-21T14:59:17.661 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.661 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.661 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:17.661 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141888+0000", 2024-09-21T14:59:17.661 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3890000000000001e-06 2024-09-21T14:59:17.661 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.661 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.661 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:17.662 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141891+0000", 2024-09-21T14:59:17.662 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9239999999999999e-06 2024-09-21T14:59:17.662 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.662 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.662 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:17.662 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176887+0000", 2024-09-21T14:59:17.662 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034996370999999998 2024-09-21T14:59:17.662 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.662 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.662 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:17.662 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176894+0000", 2024-09-21T14:59:17.662 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6069999999999996e-06 2024-09-21T14:59:17.662 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:17.662 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:17.663 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:17.663 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.663 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.663 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45714 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:17.663 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141893+0000", 2024-09-21T14:59:17.663 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 7.4922190149999999, 2024-09-21T14:59:17.663 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035048215000000001, 2024-09-21T14:59:17.663 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:17.663 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:17.663 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:17.663 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:17.663 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:17.663 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45714 2024-09-21T14:59:17.663 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.663 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:17.664 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.664 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:17.664 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141893+0000", 2024-09-21T14:59:17.664 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:17.664 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.664 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.664 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:17.664 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141893+0000", 2024-09-21T14:59:17.664 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:17.664 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.664 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.664 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:17.664 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141893+0000", 2024-09-21T14:59:17.664 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:17.664 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.665 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.665 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:17.665 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141895+0000", 2024-09-21T14:59:17.665 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.703e-06 2024-09-21T14:59:17.665 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.665 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.665 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:17.665 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141900+0000", 2024-09-21T14:59:17.665 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.242e-06 2024-09-21T14:59:17.665 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.665 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.665 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:17.665 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141903+0000", 2024-09-21T14:59:17.665 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.117e-06 2024-09-21T14:59:17.665 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.665 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.666 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:17.666 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176930+0000", 2024-09-21T14:59:17.666 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035026750000000002 2024-09-21T14:59:17.666 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.666 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.666 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:17.666 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176942+0000", 2024-09-21T14:59:17.666 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1668000000000001e-05 2024-09-21T14:59:17.666 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:17.666 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:17.666 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:17.666 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.666 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.666 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45715 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:17.666 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141906+0000", 2024-09-21T14:59:17.667 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 7.4922068570000002, 2024-09-21T14:59:17.667 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035066057999999997, 2024-09-21T14:59:17.667 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:17.667 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:17.667 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:17.667 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:17.667 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:17.667 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45715 2024-09-21T14:59:17.667 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.667 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:17.667 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.667 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:17.667 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141906+0000", 2024-09-21T14:59:17.667 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:17.668 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.668 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.668 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:17.668 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141906+0000", 2024-09-21T14:59:17.668 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:17.668 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.668 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.668 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:17.668 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141905+0000", 2024-09-21T14:59:17.668 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:17.668 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.668 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.668 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:17.668 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141907+0000", 2024-09-21T14:59:17.668 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5400000000000001e-06 2024-09-21T14:59:17.668 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.669 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.669 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:17.669 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141912+0000", 2024-09-21T14:59:17.669 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.502e-06 2024-09-21T14:59:17.669 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.669 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.669 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:17.669 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141915+0000", 2024-09-21T14:59:17.669 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0170000000000001e-06 2024-09-21T14:59:17.669 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.669 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.669 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:17.669 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176964+0000", 2024-09-21T14:59:17.669 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035048587999999999 2024-09-21T14:59:17.669 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.669 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.670 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:17.670 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176972+0000", 2024-09-21T14:59:17.670 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6580000000000002e-06 2024-09-21T14:59:17.670 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:17.670 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:17.670 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:17.670 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.670 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.670 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45716 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:17.670 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141918+0000", 2024-09-21T14:59:17.670 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 7.492194681, 2024-09-21T14:59:17.670 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035086223, 2024-09-21T14:59:17.670 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:17.670 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:17.671 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:17.671 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:17.671 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:17.671 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45716 2024-09-21T14:59:17.671 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.671 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:17.671 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.671 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:17.671 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141918+0000", 2024-09-21T14:59:17.671 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:17.671 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.671 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.671 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:17.671 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141918+0000", 2024-09-21T14:59:17.671 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:17.671 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.672 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.672 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:17.672 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141917+0000", 2024-09-21T14:59:17.672 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:17.672 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.672 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.672 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:17.672 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141919+0000", 2024-09-21T14:59:17.672 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5680000000000001e-06 2024-09-21T14:59:17.672 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.672 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.672 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:17.672 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141922+0000", 2024-09-21T14:59:17.672 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2789999999999999e-06 2024-09-21T14:59:17.672 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.673 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.673 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:17.673 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141925+0000", 2024-09-21T14:59:17.673 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9469999999999999e-06 2024-09-21T14:59:17.673 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.673 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.673 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:17.673 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176997+0000", 2024-09-21T14:59:17.673 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035071366999999999 2024-09-21T14:59:17.673 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.673 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.673 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:17.673 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177004+0000", 2024-09-21T14:59:17.673 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3309999999999998e-06 2024-09-21T14:59:17.673 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:17.673 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:17.674 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:17.674 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.674 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.674 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45717 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:17.674 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141928+0000", 2024-09-21T14:59:17.674 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 7.4921847350000004, 2024-09-21T14:59:17.674 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035102573999999998, 2024-09-21T14:59:17.674 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:17.674 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:17.674 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:17.674 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:17.674 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:17.674 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45717 2024-09-21T14:59:17.674 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.674 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:17.674 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.675 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:17.675 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141928+0000", 2024-09-21T14:59:17.675 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:17.675 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.675 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.675 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:17.675 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141928+0000", 2024-09-21T14:59:17.675 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:17.675 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.675 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.675 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:17.675 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141927+0000", 2024-09-21T14:59:17.675 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T14:59:17.675 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.675 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.676 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:17.676 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141929+0000", 2024-09-21T14:59:17.676 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.533e-06 2024-09-21T14:59:17.676 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.676 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.676 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:17.676 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141934+0000", 2024-09-21T14:59:17.676 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1660000000000002e-06 2024-09-21T14:59:17.676 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.676 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.676 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:17.676 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141937+0000", 2024-09-21T14:59:17.676 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0309999999999999e-06 2024-09-21T14:59:17.676 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.676 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.676 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:17.676 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177023+0000", 2024-09-21T14:59:17.677 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035085595999999997 2024-09-21T14:59:17.677 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.677 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.677 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:17.677 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177030+0000", 2024-09-21T14:59:17.677 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.486e-06 2024-09-21T14:59:17.677 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:17.677 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:17.677 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:17.677 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.677 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.677 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45718 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:17.677 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141940+0000", 2024-09-21T14:59:17.677 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 7.4921728769999998, 2024-09-21T14:59:17.677 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035124956999999998, 2024-09-21T14:59:17.677 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:17.678 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:17.678 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:17.678 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:17.678 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:17.678 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45718 2024-09-21T14:59:17.678 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.678 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:17.678 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.678 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:17.678 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141940+0000", 2024-09-21T14:59:17.678 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:17.678 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.678 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.678 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:17.678 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141940+0000", 2024-09-21T14:59:17.679 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:17.679 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.679 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.679 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:17.679 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141939+0000", 2024-09-21T14:59:17.679 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:17.679 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.679 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.679 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:17.679 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141941+0000", 2024-09-21T14:59:17.679 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.502e-06 2024-09-21T14:59:17.679 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.679 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.679 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:17.679 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141946+0000", 2024-09-21T14:59:17.679 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3449999999999996e-06 2024-09-21T14:59:17.680 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.680 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.680 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:17.680 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141949+0000", 2024-09-21T14:59:17.680 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9749999999999999e-06 2024-09-21T14:59:17.680 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.680 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.680 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:17.680 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177057+0000", 2024-09-21T14:59:17.680 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035108385999999998 2024-09-21T14:59:17.680 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.680 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.680 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:17.680 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177064+0000", 2024-09-21T14:59:17.680 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0099999999999998e-06 2024-09-21T14:59:17.680 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:17.680 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:17.681 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:17.681 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.681 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.681 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45719 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:17.681 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141952+0000", 2024-09-21T14:59:17.681 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 7.4921606040000004, 2024-09-21T14:59:17.681 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035139108000000002, 2024-09-21T14:59:17.681 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:17.681 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:17.681 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:17.681 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:17.681 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:17.681 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45719 2024-09-21T14:59:17.681 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.681 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:17.681 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.682 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:17.682 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141952+0000", 2024-09-21T14:59:17.682 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:17.682 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.682 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.682 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:17.682 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141952+0000", 2024-09-21T14:59:17.682 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:17.682 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.682 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.682 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:17.682 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141952+0000", 2024-09-21T14:59:17.682 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:17.682 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.682 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.683 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:17.683 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141953+0000", 2024-09-21T14:59:17.683 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.486e-06 2024-09-21T14:59:17.683 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.683 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.683 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:17.683 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141959+0000", 2024-09-21T14:59:17.683 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4940000000000003e-06 2024-09-21T14:59:17.683 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.683 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.683 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:17.683 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141963+0000", 2024-09-21T14:59:17.683 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3540000000000002e-06 2024-09-21T14:59:17.683 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.683 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.683 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:17.684 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177084+0000", 2024-09-21T14:59:17.684 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035120983000000001 2024-09-21T14:59:17.684 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.684 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.684 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:17.684 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177091+0000", 2024-09-21T14:59:17.684 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0350000000000004e-06 2024-09-21T14:59:17.684 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:17.684 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:17.684 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:17.684 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.684 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.684 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45720 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:17.684 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141965+0000", 2024-09-21T14:59:17.684 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 7.4921471369999999, 2024-09-21T14:59:17.684 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035155268000000003, 2024-09-21T14:59:17.685 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:17.685 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:17.685 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:17.685 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:17.685 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:17.685 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45720 2024-09-21T14:59:17.685 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.685 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:17.685 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.685 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:17.685 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141965+0000", 2024-09-21T14:59:17.685 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:17.685 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.685 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.685 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:17.685 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141965+0000", 2024-09-21T14:59:17.685 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:17.686 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.686 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.686 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:17.686 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141965+0000", 2024-09-21T14:59:17.686 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:17.686 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.686 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.686 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:17.686 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141966+0000", 2024-09-21T14:59:17.686 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.46e-06 2024-09-21T14:59:17.686 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.686 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.686 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:17.686 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141972+0000", 2024-09-21T14:59:17.687 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4210000000000003e-06 2024-09-21T14:59:17.687 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.687 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.687 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:17.687 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141975+0000", 2024-09-21T14:59:17.687 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.948e-06 2024-09-21T14:59:17.687 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.687 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.687 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:17.687 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177113+0000", 2024-09-21T14:59:17.687 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035137855000000003 2024-09-21T14:59:17.687 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.687 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.687 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:17.688 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177121+0000", 2024-09-21T14:59:17.688 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8269999999999998e-06 2024-09-21T14:59:17.688 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:17.688 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:17.688 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:17.688 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.688 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.688 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45721 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:17.688 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141977+0000", 2024-09-21T14:59:17.688 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 7.4921351139999999, 2024-09-21T14:59:17.688 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035169984000000001, 2024-09-21T14:59:17.688 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:17.688 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:17.688 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:17.688 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:17.689 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:17.689 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45721 2024-09-21T14:59:17.689 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.689 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:17.689 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.689 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:17.689 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141977+0000", 2024-09-21T14:59:17.689 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:17.689 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.689 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.689 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:17.689 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141977+0000", 2024-09-21T14:59:17.689 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:17.689 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.690 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.690 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:17.690 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141977+0000", 2024-09-21T14:59:17.690 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:17.690 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.690 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.690 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:17.690 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141979+0000", 2024-09-21T14:59:17.690 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.511e-06 2024-09-21T14:59:17.690 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.690 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.690 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:17.690 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141982+0000", 2024-09-21T14:59:17.691 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4180000000000001e-06 2024-09-21T14:59:17.691 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.691 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.691 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:17.691 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141985+0000", 2024-09-21T14:59:17.691 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9450000000000002e-06 2024-09-21T14:59:17.691 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.691 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.691 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:17.691 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177139+0000", 2024-09-21T14:59:17.691 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035154409999999997 2024-09-21T14:59:17.691 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.691 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.691 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:17.691 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177147+0000", 2024-09-21T14:59:17.691 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9570000000000002e-06 2024-09-21T14:59:17.692 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:17.692 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:17.692 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:17.692 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.692 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.692 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45722 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:17.692 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141987+0000", 2024-09-21T14:59:17.692 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 7.492125143, 2024-09-21T14:59:17.692 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035188492000000002, 2024-09-21T14:59:17.692 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:17.692 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:17.692 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:17.692 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:17.692 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:17.692 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45722 2024-09-21T14:59:17.692 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.693 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:17.693 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.693 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:17.693 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141987+0000", 2024-09-21T14:59:17.693 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:17.693 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.693 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.693 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:17.693 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141987+0000", 2024-09-21T14:59:17.693 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:17.693 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.693 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.693 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:17.693 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141987+0000", 2024-09-21T14:59:17.693 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:17.694 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.694 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.694 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:17.694 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141988+0000", 2024-09-21T14:59:17.694 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.463e-06 2024-09-21T14:59:17.694 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.694 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.694 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:17.694 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141994+0000", 2024-09-21T14:59:17.694 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.361e-06 2024-09-21T14:59:17.694 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.695 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.695 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:17.695 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141997+0000", 2024-09-21T14:59:17.695 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0359999999999998e-06 2024-09-21T14:59:17.695 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.695 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.695 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:17.695 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177169+0000", 2024-09-21T14:59:17.695 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035171651999999998 2024-09-21T14:59:17.695 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.695 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.695 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:17.695 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177176+0000", 2024-09-21T14:59:17.695 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2189999999999998e-06 2024-09-21T14:59:17.695 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:17.695 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:17.696 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:17.696 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.696 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.696 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45723 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:17.696 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141999+0000", 2024-09-21T14:59:17.696 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 7.4921129759999996, 2024-09-21T14:59:17.696 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035197701999999997, 2024-09-21T14:59:17.696 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:17.696 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:17.696 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:17.696 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:17.696 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:17.696 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45723 2024-09-21T14:59:17.696 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.696 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:17.696 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.697 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:17.697 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141999+0000", 2024-09-21T14:59:17.697 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:17.697 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.697 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.697 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:17.697 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141999+0000", 2024-09-21T14:59:17.697 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:17.697 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.697 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.697 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:17.697 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141999+0000", 2024-09-21T14:59:17.697 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:17.697 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.697 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.698 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:17.698 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142003+0000", 2024-09-21T14:59:17.698 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5049999999999999e-06 2024-09-21T14:59:17.698 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.698 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.698 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:17.698 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142008+0000", 2024-09-21T14:59:17.698 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6579999999999999e-06 2024-09-21T14:59:17.698 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.698 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.698 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:17.698 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142011+0000", 2024-09-21T14:59:17.698 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0529999999999998e-06 2024-09-21T14:59:17.698 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.698 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.698 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:17.699 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177194+0000", 2024-09-21T14:59:17.699 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035182221999999999 2024-09-21T14:59:17.699 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.699 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.699 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:17.699 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177197+0000", 2024-09-21T14:59:17.699 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5030000000000002e-06 2024-09-21T14:59:17.699 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:17.699 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:17.699 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:17.699 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.699 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.699 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45724 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:17.699 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.142015+0000", 2024-09-21T14:59:17.699 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 7.4920970090000001, 2024-09-21T14:59:17.700 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035195345000000003, 2024-09-21T14:59:17.700 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:17.700 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:17.700 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:17.700 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:17.700 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:17.700 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45724 2024-09-21T14:59:17.700 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.700 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:17.700 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.700 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:17.700 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142015+0000", 2024-09-21T14:59:17.700 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:17.700 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.700 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.700 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:17.701 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142015+0000", 2024-09-21T14:59:17.701 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:17.701 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.701 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.701 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:17.701 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142015+0000", 2024-09-21T14:59:17.701 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:17.701 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.701 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.701 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:17.701 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142017+0000", 2024-09-21T14:59:17.701 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5540000000000001e-06 2024-09-21T14:59:17.701 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.701 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.701 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:17.701 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142022+0000", 2024-09-21T14:59:17.702 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4229999999999996e-06 2024-09-21T14:59:17.702 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.702 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.702 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:17.702 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142025+0000", 2024-09-21T14:59:17.702 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.056e-06 2024-09-21T14:59:17.702 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.702 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.702 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:17.702 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177207+0000", 2024-09-21T14:59:17.702 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035182089 2024-09-21T14:59:17.702 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.702 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.702 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:17.702 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177211+0000", 2024-09-21T14:59:17.702 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4649999999999999e-06 2024-09-21T14:59:17.703 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:17.703 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:17.703 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:17.703 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.703 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.703 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45725 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:17.703 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.142028+0000", 2024-09-21T14:59:17.703 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 7.4920847300000002, 2024-09-21T14:59:17.703 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035195731000000001, 2024-09-21T14:59:17.703 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:17.703 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:17.703 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:17.703 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:17.703 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:17.703 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45725 2024-09-21T14:59:17.704 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.704 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:17.704 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.704 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:17.704 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142028+0000", 2024-09-21T14:59:17.704 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:17.704 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.704 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.704 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:17.704 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142028+0000", 2024-09-21T14:59:17.704 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:17.704 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.704 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.704 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:17.704 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142027+0000", 2024-09-21T14:59:17.704 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:17.705 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.705 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.705 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:17.705 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142029+0000", 2024-09-21T14:59:17.705 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5269999999999999e-06 2024-09-21T14:59:17.705 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.705 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.705 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:17.705 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142032+0000", 2024-09-21T14:59:17.705 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3249999999999999e-06 2024-09-21T14:59:17.705 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.705 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.705 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:17.705 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142035+0000", 2024-09-21T14:59:17.705 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.897e-06 2024-09-21T14:59:17.705 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.706 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.706 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:17.706 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177220+0000", 2024-09-21T14:59:17.706 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035184871 2024-09-21T14:59:17.706 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.706 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.706 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:17.706 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177223+0000", 2024-09-21T14:59:17.706 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3680000000000002e-06 2024-09-21T14:59:17.706 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:17.706 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:17.706 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:17.706 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.706 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.706 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45726 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:17.707 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.142037+0000", 2024-09-21T14:59:17.707 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 7.4920750150000002, 2024-09-21T14:59:17.707 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035199273000000003, 2024-09-21T14:59:17.707 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:17.707 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:17.707 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:17.707 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:17.707 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:17.707 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45726 2024-09-21T14:59:17.707 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.707 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:17.707 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.707 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:17.707 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142037+0000", 2024-09-21T14:59:17.707 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:17.707 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.708 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.708 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:17.708 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142037+0000", 2024-09-21T14:59:17.708 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:17.708 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.708 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.708 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:17.708 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142037+0000", 2024-09-21T14:59:17.708 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:17.708 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.708 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.708 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:17.708 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142038+0000", 2024-09-21T14:59:17.708 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.344e-06 2024-09-21T14:59:17.708 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.708 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.709 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:17.709 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142044+0000", 2024-09-21T14:59:17.709 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1930000000000001e-06 2024-09-21T14:59:17.709 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.709 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.709 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:17.709 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142047+0000", 2024-09-21T14:59:17.709 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0280000000000001e-06 2024-09-21T14:59:17.709 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.709 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.709 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:17.709 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177233+0000", 2024-09-21T14:59:17.709 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035186394000000003 2024-09-21T14:59:17.709 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:17.709 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:17.709 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:17.709 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177237+0000", 2024-09-21T14:59:17.710 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5530000000000001e-06 2024-09-21T14:59:17.710 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:17.710 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:17.710 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:17.710 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:17.710 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:17.710 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:17.711 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T14:59:17.723 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:59:17.719+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:59:17.824 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:59:17.819+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:59:17.867 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:17.867 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:59:17.867 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T14:59:17.867 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:17.884 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T14:59:17.925 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:59:17.925+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:59:17.994 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:17.994 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:59:17.994 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T14:59:17.994 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T14:59:17.994 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:18.004 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T14:59:18.049 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:59:18.049+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:59:18.127 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:59:18.125+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:59:18.165 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:18.166 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T14:59:18.166 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T14:59:18.166 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T14:59:18.166 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.166 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4281.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-21T14:59:18.166 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:09.324416+0000", 2024-09-21T14:59:18.166 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 68.841641916, 2024-09-21T14:59:18.166 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.117837947, 2024-09-21T14:59:18.166 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:18.166 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:18.166 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:18.167 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4281", 2024-09-21T14:59:18.167 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/1518520303", 2024-09-21T14:59:18.167 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1 2024-09-21T14:59:18.167 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.167 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:18.167 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.167 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:18.167 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324416+0000", 2024-09-21T14:59:18.167 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:18.167 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.167 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.167 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:18.167 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324416+0000", 2024-09-21T14:59:18.168 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:18.168 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.168 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.168 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:18.168 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324414+0000", 2024-09-21T14:59:18.168 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T14:59:18.168 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.168 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.168 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:18.168 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324429+0000", 2024-09-21T14:59:18.168 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5637e-05 2024-09-21T14:59:18.168 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.168 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.168 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:18.168 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324448+0000", 2024-09-21T14:59:18.168 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9042000000000002e-05 2024-09-21T14:59:18.169 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.169 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.169 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:18.169 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324465+0000", 2024-09-21T14:59:18.169 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6767999999999999e-05 2024-09-21T14:59:18.169 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.169 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.169 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:18.169 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324571+0000", 2024-09-21T14:59:18.169 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000105714 2024-09-21T14:59:18.169 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.169 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.169 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op must wait for map", 2024-09-21T14:59:18.169 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324587+0000", 2024-09-21T14:59:18.170 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5753999999999999e-05 2024-09-21T14:59:18.170 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.170 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.170 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:18.170 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.325864+0000", 2024-09-21T14:59:18.170 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0012773350000000001 2024-09-21T14:59:18.170 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.170 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.170 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T14:59:18.170 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.325878+0000", 2024-09-21T14:59:18.170 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4498000000000001e-05 2024-09-21T14:59:18.170 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.170 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.170 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:18.171 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.439461+0000", 2024-09-21T14:59:18.171 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.11358212500000001 2024-09-21T14:59:18.171 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.171 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.171 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:18.171 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.439848+0000", 2024-09-21T14:59:18.171 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00038747800000000002 2024-09-21T14:59:18.171 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.171 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.171 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:59:18.171 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.440175+0000", 2024-09-21T14:59:18.171 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00032721900000000001 2024-09-21T14:59:18.171 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.171 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.171 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:18.172 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.441770+0000", 2024-09-21T14:59:18.172 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0015949409999999999 2024-09-21T14:59:18.172 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.172 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.172 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:18.172 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.442086+0000", 2024-09-21T14:59:18.172 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00031586299999999999 2024-09-21T14:59:18.172 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.172 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.172 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:18.172 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.442195+0000", 2024-09-21T14:59:18.172 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000108791 2024-09-21T14:59:18.172 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.172 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.172 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:18.172 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.442254+0000", 2024-09-21T14:59:18.173 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9038e-05 2024-09-21T14:59:18.173 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:18.173 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:18.173 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:18.173 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.173 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.173 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4281.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-21T14:59:18.173 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:09.447828+0000", 2024-09-21T14:59:18.173 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 68.718229914999995, 2024-09-21T14:59:18.173 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.32904393700000001, 2024-09-21T14:59:18.173 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:18.173 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:18.173 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:18.173 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4281", 2024-09-21T14:59:18.173 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/1518520303", 2024-09-21T14:59:18.174 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 2 2024-09-21T14:59:18.174 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.174 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:18.174 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.174 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:18.174 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447828+0000", 2024-09-21T14:59:18.174 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:18.174 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.174 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.174 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:18.174 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447828+0000", 2024-09-21T14:59:18.174 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:18.174 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.174 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.174 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:18.175 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447825+0000", 2024-09-21T14:59:18.175 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T14:59:18.175 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.175 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.175 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:18.175 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447841+0000", 2024-09-21T14:59:18.175 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5359999999999999e-05 2024-09-21T14:59:18.175 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.175 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.175 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:18.175 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447875+0000", 2024-09-21T14:59:18.175 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4388999999999999e-05 2024-09-21T14:59:18.175 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.175 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.175 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:18.176 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447903+0000", 2024-09-21T14:59:18.176 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7373e-05 2024-09-21T14:59:18.176 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.176 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.176 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:18.176 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447996+0000", 2024-09-21T14:59:18.176 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2954999999999993e-05 2024-09-21T14:59:18.176 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.176 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.176 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T14:59:18.176 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.448011+0000", 2024-09-21T14:59:18.176 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5038999999999999e-05 2024-09-21T14:59:18.176 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.176 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.176 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:18.176 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.776501+0000", 2024-09-21T14:59:18.177 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.32849012500000002 2024-09-21T14:59:18.177 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.177 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.177 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:18.177 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.776872+0000", 2024-09-21T14:59:18.177 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00037119600000000002 2024-09-21T14:59:18.177 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:18.177 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:18.177 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:18.177 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.177 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.177 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:752 3.14 3:28fc02e5:::rbd_data.113693316f36.0000000000000010:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-21T14:59:18.177 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:26.625579+0000", 2024-09-21T14:59:18.177 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 51.540479060000003, 2024-09-21T14:59:18.177 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042965426000000001, 2024-09-21T14:59:18.177 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:18.178 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:18.178 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:18.178 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:18.178 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:18.178 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 752 2024-09-21T14:59:18.178 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.178 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:18.178 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.178 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:18.178 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625579+0000", 2024-09-21T14:59:18.178 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:18.178 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.178 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.178 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:18.178 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625579+0000", 2024-09-21T14:59:18.179 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:18.179 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.179 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.179 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:18.179 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625577+0000", 2024-09-21T14:59:18.179 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T14:59:18.179 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.179 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.179 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:18.179 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625588+0000", 2024-09-21T14:59:18.179 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1555e-05 2024-09-21T14:59:18.179 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.179 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.179 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:18.179 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625610+0000", 2024-09-21T14:59:18.180 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2107999999999999e-05 2024-09-21T14:59:18.180 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.180 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.180 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:18.180 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625621+0000", 2024-09-21T14:59:18.180 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0762e-05 2024-09-21T14:59:18.180 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.180 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.180 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:18.180 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625692+0000", 2024-09-21T14:59:18.180 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.1203000000000003e-05 2024-09-21T14:59:18.180 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.180 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.180 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:18.180 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625872+0000", 2024-09-21T14:59:18.180 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017962800000000001 2024-09-21T14:59:18.181 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.181 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.181 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:59:18.181 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.633188+0000", 2024-09-21T14:59:18.181 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.007316014 2024-09-21T14:59:18.181 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.181 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.181 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:18.181 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.668357+0000", 2024-09-21T14:59:18.181 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035168933999999999 2024-09-21T14:59:18.181 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.181 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.181 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:18.181 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.668418+0000", 2024-09-21T14:59:18.181 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1282e-05 2024-09-21T14:59:18.182 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.182 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.182 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:18.182 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.668486+0000", 2024-09-21T14:59:18.182 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8468999999999998e-05 2024-09-21T14:59:18.182 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.182 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.182 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:18.182 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.668544+0000", 2024-09-21T14:59:18.182 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7785000000000003e-05 2024-09-21T14:59:18.182 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:18.182 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:18.182 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:18.182 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.183 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.183 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22737 3.a 3:574c7a60:::rbd_data.11366acc894.0000000000000002:19 [read 1032192~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:18.183 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141028+0000", 2024-09-21T14:59:18.183 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 28.025030095000002, 2024-09-21T14:59:18.183 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035206281999999998, 2024-09-21T14:59:18.183 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:18.183 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:18.183 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:18.183 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:18.183 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:18.183 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22737 2024-09-21T14:59:18.183 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.183 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:18.183 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.183 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:18.184 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141028+0000", 2024-09-21T14:59:18.184 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:18.184 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.184 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.184 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:18.184 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141028+0000", 2024-09-21T14:59:18.184 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:18.184 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.184 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.184 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:18.184 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141027+0000", 2024-09-21T14:59:18.184 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:18.184 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.184 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.184 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:18.184 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141030+0000", 2024-09-21T14:59:18.185 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6189999999999998e-06 2024-09-21T14:59:18.185 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.185 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.185 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:18.185 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141036+0000", 2024-09-21T14:59:18.185 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5110000000000003e-06 2024-09-21T14:59:18.185 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.185 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.185 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:18.185 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141040+0000", 2024-09-21T14:59:18.185 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.34e-06 2024-09-21T14:59:18.185 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.185 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.185 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:18.185 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.175948+0000", 2024-09-21T14:59:18.185 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034908540000000002 2024-09-21T14:59:18.185 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.186 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.186 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:18.186 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176025+0000", 2024-09-21T14:59:18.186 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6412e-05 2024-09-21T14:59:18.186 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.186 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.186 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:18.186 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176234+0000", 2024-09-21T14:59:18.186 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000209247 2024-09-21T14:59:18.186 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:18.186 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:18.186 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:18.186 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.186 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.186 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22738 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 49152~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:18.187 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141049+0000", 2024-09-21T14:59:18.187 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 28.025008917000001, 2024-09-21T14:59:18.187 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035428754999999999, 2024-09-21T14:59:18.187 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:18.187 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:18.187 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:18.187 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:18.187 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:18.187 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22738 2024-09-21T14:59:18.187 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.187 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:18.187 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.187 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:18.187 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141049+0000", 2024-09-21T14:59:18.187 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:18.188 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.188 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.188 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:18.188 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141049+0000", 2024-09-21T14:59:18.188 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:18.188 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.188 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.188 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:18.188 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141049+0000", 2024-09-21T14:59:18.188 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:18.188 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.188 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.188 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:18.188 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141051+0000", 2024-09-21T14:59:18.188 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4980000000000001e-06 2024-09-21T14:59:18.188 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.189 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.189 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:18.189 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141057+0000", 2024-09-21T14:59:18.189 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8030000000000002e-06 2024-09-21T14:59:18.189 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.189 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.189 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:18.189 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141066+0000", 2024-09-21T14:59:18.189 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1160000000000005e-06 2024-09-21T14:59:18.189 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.189 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.189 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:18.189 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176264+0000", 2024-09-21T14:59:18.189 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035197580999999999 2024-09-21T14:59:18.189 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.189 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.190 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:18.190 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176327+0000", 2024-09-21T14:59:18.190 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3237000000000005e-05 2024-09-21T14:59:18.190 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.190 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.190 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:18.190 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176478+0000", 2024-09-21T14:59:18.190 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000150851 2024-09-21T14:59:18.190 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:18.190 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:18.190 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:18.190 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.190 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.190 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22739 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 114688~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:18.191 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141070+0000", 2024-09-21T14:59:18.191 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 28.024987674999998, 2024-09-21T14:59:18.191 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035679094000000001, 2024-09-21T14:59:18.191 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:18.191 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:18.191 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:18.191 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:18.191 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:18.191 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22739 2024-09-21T14:59:18.191 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.191 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:18.191 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.191 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:18.191 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141070+0000", 2024-09-21T14:59:18.191 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:18.191 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.192 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.192 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:18.192 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141070+0000", 2024-09-21T14:59:18.192 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:18.192 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.192 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.192 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:18.192 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141070+0000", 2024-09-21T14:59:18.192 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:18.192 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.192 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.192 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:18.192 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141072+0000", 2024-09-21T14:59:18.192 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.379e-06 2024-09-21T14:59:18.192 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.192 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.193 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:18.193 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141078+0000", 2024-09-21T14:59:18.193 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6710000000000004e-06 2024-09-21T14:59:18.193 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.193 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.193 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:18.193 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141083+0000", 2024-09-21T14:59:18.193 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7249999999999997e-06 2024-09-21T14:59:18.193 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.193 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.193 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:18.193 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176506+0000", 2024-09-21T14:59:18.193 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035423076999999997 2024-09-21T14:59:18.193 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.193 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.194 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:18.194 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176582+0000", 2024-09-21T14:59:18.194 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6305999999999998e-05 2024-09-21T14:59:18.194 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.194 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.194 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:18.194 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176749+0000", 2024-09-21T14:59:18.194 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000167262 2024-09-21T14:59:18.194 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:18.194 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:18.194 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:18.194 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.194 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.194 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22740 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 180224~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:18.194 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141090+0000", 2024-09-21T14:59:18.195 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 28.024968270999999, 2024-09-21T14:59:18.195 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036129535999999997, 2024-09-21T14:59:18.195 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:18.195 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:18.195 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:18.195 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:18.195 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:18.195 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22740 2024-09-21T14:59:18.195 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.195 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:18.195 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.195 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:18.195 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141090+0000", 2024-09-21T14:59:18.195 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:18.195 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.196 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.196 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:18.196 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141090+0000", 2024-09-21T14:59:18.196 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:18.196 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.196 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.196 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:18.196 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141089+0000", 2024-09-21T14:59:18.196 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:18.196 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.196 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.196 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:18.196 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141092+0000", 2024-09-21T14:59:18.196 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7709999999999998e-06 2024-09-21T14:59:18.196 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.196 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.197 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:18.197 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141098+0000", 2024-09-21T14:59:18.197 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.015e-06 2024-09-21T14:59:18.197 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.197 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.197 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:18.197 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141103+0000", 2024-09-21T14:59:18.197 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8629999999999999e-06 2024-09-21T14:59:18.197 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.197 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.197 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:18.197 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176778+0000", 2024-09-21T14:59:18.197 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035674984 2024-09-21T14:59:18.197 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.197 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.198 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:18.198 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176851+0000", 2024-09-21T14:59:18.198 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2861000000000006e-05 2024-09-21T14:59:18.198 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.198 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.198 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:18.198 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177219+0000", 2024-09-21T14:59:18.198 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00036845999999999998 2024-09-21T14:59:18.198 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:18.198 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:18.198 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:18.198 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.198 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.198 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22741 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 245760~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:18.198 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141107+0000", 2024-09-21T14:59:18.199 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 28.024950700000002, 2024-09-21T14:59:18.199 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036407729, 2024-09-21T14:59:18.199 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:18.199 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:18.199 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:18.199 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:18.199 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:18.199 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22741 2024-09-21T14:59:18.199 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.199 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:18.199 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.199 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:18.199 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141107+0000", 2024-09-21T14:59:18.199 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:18.199 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.199 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.200 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:18.200 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141107+0000", 2024-09-21T14:59:18.200 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:18.200 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.200 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.200 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:18.200 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141107+0000", 2024-09-21T14:59:18.200 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:18.200 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.200 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.200 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:18.200 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141109+0000", 2024-09-21T14:59:18.200 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6529999999999998e-06 2024-09-21T14:59:18.200 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.200 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.200 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:18.201 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141115+0000", 2024-09-21T14:59:18.201 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9649999999999996e-06 2024-09-21T14:59:18.201 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.201 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.201 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:18.201 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141121+0000", 2024-09-21T14:59:18.201 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1209999999999998e-06 2024-09-21T14:59:18.201 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.201 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.201 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:18.201 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177242+0000", 2024-09-21T14:59:18.201 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036121706000000003 2024-09-21T14:59:18.201 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.201 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.201 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:18.201 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177301+0000", 2024-09-21T14:59:18.202 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9008000000000002e-05 2024-09-21T14:59:18.202 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.202 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.202 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:18.202 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177515+0000", 2024-09-21T14:59:18.202 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00021362 2024-09-21T14:59:18.202 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:18.202 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:18.202 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:18.202 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.202 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.202 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22742 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 311296~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:18.202 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141126+0000", 2024-09-21T14:59:18.202 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 28.024932346, 2024-09-21T14:59:18.202 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036653537, 2024-09-21T14:59:18.203 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:18.203 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:18.203 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:18.203 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:18.203 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:18.203 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22742 2024-09-21T14:59:18.203 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.203 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:18.203 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.203 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:18.203 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141126+0000", 2024-09-21T14:59:18.203 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:18.203 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.203 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.203 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:18.203 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141126+0000", 2024-09-21T14:59:18.204 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:18.204 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.204 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.204 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:18.204 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141125+0000", 2024-09-21T14:59:18.204 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:18.204 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.204 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.204 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:18.204 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141128+0000", 2024-09-21T14:59:18.204 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.886e-06 2024-09-21T14:59:18.204 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.204 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.204 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:18.204 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141137+0000", 2024-09-21T14:59:18.204 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4399999999999994e-06 2024-09-21T14:59:18.205 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.205 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.205 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:18.205 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141143+0000", 2024-09-21T14:59:18.205 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1410000000000004e-06 2024-09-21T14:59:18.205 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.205 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.205 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:18.205 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177543+0000", 2024-09-21T14:59:18.205 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036400544999999999 2024-09-21T14:59:18.205 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.205 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.205 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:18.205 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177605+0000", 2024-09-21T14:59:18.205 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2206999999999996e-05 2024-09-21T14:59:18.206 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.206 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.206 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:18.206 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177779+0000", 2024-09-21T14:59:18.206 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017368099999999999 2024-09-21T14:59:18.206 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:18.206 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:18.206 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:18.206 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.206 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.206 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22743 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 376832~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:18.206 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141147+0000", 2024-09-21T14:59:18.206 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 28.024910861999999, 2024-09-21T14:59:18.206 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036915158000000003, 2024-09-21T14:59:18.206 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:18.207 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:18.207 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:18.207 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:18.207 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:18.207 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22743 2024-09-21T14:59:18.207 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.207 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:18.207 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.207 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:18.207 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141147+0000", 2024-09-21T14:59:18.207 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:18.207 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.207 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.207 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:18.207 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141147+0000", 2024-09-21T14:59:18.207 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:18.208 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.208 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.208 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:18.208 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141147+0000", 2024-09-21T14:59:18.208 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:18.208 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.208 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.208 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:18.208 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141149+0000", 2024-09-21T14:59:18.208 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.79e-06 2024-09-21T14:59:18.208 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.208 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.208 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:18.208 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141155+0000", 2024-09-21T14:59:18.208 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6699999999999999e-06 2024-09-21T14:59:18.208 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.209 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.209 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:18.209 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141160+0000", 2024-09-21T14:59:18.209 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5759999999999999e-06 2024-09-21T14:59:18.209 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.209 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.209 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:18.209 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177817+0000", 2024-09-21T14:59:18.209 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036657404999999997 2024-09-21T14:59:18.209 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.209 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.209 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:18.209 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177892+0000", 2024-09-21T14:59:18.209 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4602000000000002e-05 2024-09-21T14:59:18.209 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.210 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.210 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:18.210 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178062+0000", 2024-09-21T14:59:18.210 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000170464 2024-09-21T14:59:18.210 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:18.210 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:18.210 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:18.210 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.210 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.210 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22744 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 442368~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:18.210 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141164+0000", 2024-09-21T14:59:18.210 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 28.024893728999999, 2024-09-21T14:59:18.210 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037120806999999999, 2024-09-21T14:59:18.210 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:18.210 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:18.211 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:18.211 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:18.211 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:18.211 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22744 2024-09-21T14:59:18.211 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.211 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:18.211 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.211 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:18.211 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141164+0000", 2024-09-21T14:59:18.211 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:18.211 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.211 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.211 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:18.211 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141164+0000", 2024-09-21T14:59:18.211 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:18.211 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.212 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.212 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:18.212 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141164+0000", 2024-09-21T14:59:18.212 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:18.212 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.212 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.212 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:18.212 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141167+0000", 2024-09-21T14:59:18.212 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7159999999999999e-06 2024-09-21T14:59:18.212 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.212 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.212 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:18.212 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141173+0000", 2024-09-21T14:59:18.212 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.071e-06 2024-09-21T14:59:18.212 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.212 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.212 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:18.213 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141177+0000", 2024-09-21T14:59:18.213 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3610000000000003e-06 2024-09-21T14:59:18.213 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.213 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.213 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:18.213 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178083+0000", 2024-09-21T14:59:18.213 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036906405000000003 2024-09-21T14:59:18.213 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.213 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.213 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:18.213 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178143+0000", 2024-09-21T14:59:18.213 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0056999999999998e-05 2024-09-21T14:59:18.213 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.213 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.213 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:18.213 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178285+0000", 2024-09-21T14:59:18.214 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000141555 2024-09-21T14:59:18.214 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:18.214 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:18.214 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:18.214 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.214 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.214 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22745 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 507904~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:18.214 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141182+0000", 2024-09-21T14:59:18.214 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 28.024875409, 2024-09-21T14:59:18.214 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037332331000000003, 2024-09-21T14:59:18.214 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:18.214 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:18.214 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:18.214 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:18.214 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:18.215 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22745 2024-09-21T14:59:18.215 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.215 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:18.215 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.215 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:18.215 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141182+0000", 2024-09-21T14:59:18.215 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:18.215 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.215 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.215 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:18.215 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141182+0000", 2024-09-21T14:59:18.215 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:18.215 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.215 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.215 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:18.216 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141182+0000", 2024-09-21T14:59:18.216 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:18.216 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.216 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.216 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:18.216 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141185+0000", 2024-09-21T14:59:18.216 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.464e-06 2024-09-21T14:59:18.216 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.216 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.216 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:18.216 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141200+0000", 2024-09-21T14:59:18.216 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5886999999999999e-05 2024-09-21T14:59:18.216 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.216 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.216 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:18.216 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141571+0000", 2024-09-21T14:59:18.217 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000370909 2024-09-21T14:59:18.217 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.217 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.217 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:18.217 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178318+0000", 2024-09-21T14:59:18.217 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036746244999999997 2024-09-21T14:59:18.217 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.217 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.217 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:18.217 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178367+0000", 2024-09-21T14:59:18.217 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9317000000000002e-05 2024-09-21T14:59:18.217 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.217 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.217 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:18.217 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178515+0000", 2024-09-21T14:59:18.218 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014784600000000001 2024-09-21T14:59:18.218 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:18.218 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:18.218 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:18.218 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.218 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.218 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22746 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 573440~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:18.218 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141599+0000", 2024-09-21T14:59:18.218 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 28.024458777, 2024-09-21T14:59:18.218 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037402249999999998, 2024-09-21T14:59:18.218 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:18.218 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:18.218 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:18.218 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:18.218 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:18.219 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22746 2024-09-21T14:59:18.219 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.219 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:18.219 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.219 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:18.219 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141599+0000", 2024-09-21T14:59:18.219 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:18.219 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.219 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.219 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:18.219 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141599+0000", 2024-09-21T14:59:18.219 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:18.219 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.219 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.219 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:18.219 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141598+0000", 2024-09-21T14:59:18.220 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:59:18.220 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.220 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.220 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:18.220 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141637+0000", 2024-09-21T14:59:18.220 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.9119999999999998e-05 2024-09-21T14:59:18.220 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.220 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.220 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:18.220 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141647+0000", 2024-09-21T14:59:18.220 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5570000000000001e-06 2024-09-21T14:59:18.220 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.220 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.220 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:18.220 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141654+0000", 2024-09-21T14:59:18.220 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2049999999999996e-06 2024-09-21T14:59:18.221 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.221 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.221 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:18.221 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178553+0000", 2024-09-21T14:59:18.221 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036898528 2024-09-21T14:59:18.221 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.221 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.221 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:18.221 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178625+0000", 2024-09-21T14:59:18.221 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2669999999999994e-05 2024-09-21T14:59:18.221 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.221 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.221 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:18.221 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179001+0000", 2024-09-21T14:59:18.221 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000375977 2024-09-21T14:59:18.222 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:18.222 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:18.222 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:18.222 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.222 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.222 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22747 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 638976~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:18.222 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141660+0000", 2024-09-21T14:59:18.222 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 28.024397635, 2024-09-21T14:59:18.222 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037617352999999999, 2024-09-21T14:59:18.222 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:18.222 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:18.222 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:18.222 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:18.222 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:18.222 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22747 2024-09-21T14:59:18.222 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.223 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:18.223 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.223 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:18.223 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141660+0000", 2024-09-21T14:59:18.223 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:18.223 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.223 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.223 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:18.223 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141660+0000", 2024-09-21T14:59:18.223 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:18.223 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.223 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.223 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:18.223 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141660+0000", 2024-09-21T14:59:18.223 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:18.224 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.224 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.224 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:18.224 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141663+0000", 2024-09-21T14:59:18.224 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.569e-06 2024-09-21T14:59:18.224 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.224 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.224 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:18.224 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141670+0000", 2024-09-21T14:59:18.224 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5719999999999999e-06 2024-09-21T14:59:18.224 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.224 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.224 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:18.224 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141675+0000", 2024-09-21T14:59:18.224 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2889999999999997e-06 2024-09-21T14:59:18.225 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.225 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.225 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:18.225 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179028+0000", 2024-09-21T14:59:18.225 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037353381999999997 2024-09-21T14:59:18.225 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.225 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.225 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:18.225 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179099+0000", 2024-09-21T14:59:18.225 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0259000000000006e-05 2024-09-21T14:59:18.225 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.225 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.225 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:18.225 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179278+0000", 2024-09-21T14:59:18.225 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000178908 2024-09-21T14:59:18.225 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:18.226 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:18.226 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:18.226 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.226 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.226 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22748 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 704512~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:18.226 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141680+0000", 2024-09-21T14:59:18.226 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 28.024377996999998, 2024-09-21T14:59:18.226 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037806166000000002, 2024-09-21T14:59:18.226 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:18.226 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:18.226 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:18.226 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:18.226 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:18.226 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22748 2024-09-21T14:59:18.226 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.226 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:18.227 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.227 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:18.227 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141680+0000", 2024-09-21T14:59:18.227 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:18.227 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.227 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.227 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:18.227 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141680+0000", 2024-09-21T14:59:18.227 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:18.227 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.227 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.227 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:18.227 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141679+0000", 2024-09-21T14:59:18.227 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:18.227 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.227 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.228 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:18.228 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141683+0000", 2024-09-21T14:59:18.228 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3230000000000002e-06 2024-09-21T14:59:18.228 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.228 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.228 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:18.228 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141690+0000", 2024-09-21T14:59:18.228 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8680000000000001e-06 2024-09-21T14:59:18.228 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.228 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.228 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:18.228 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141694+0000", 2024-09-21T14:59:18.228 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6770000000000003e-06 2024-09-21T14:59:18.228 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.228 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.229 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:18.229 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179300+0000", 2024-09-21T14:59:18.229 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037605567999999999 2024-09-21T14:59:18.229 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.229 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.229 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:18.229 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179355+0000", 2024-09-21T14:59:18.229 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4771000000000001e-05 2024-09-21T14:59:18.229 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.229 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.229 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:18.229 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179486+0000", 2024-09-21T14:59:18.229 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013141699999999999 2024-09-21T14:59:18.229 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:18.229 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:18.230 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:18.230 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.230 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.230 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22749 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 770048~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:18.230 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141698+0000", 2024-09-21T14:59:18.230 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 28.024359624999999, 2024-09-21T14:59:18.230 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037967813000000003, 2024-09-21T14:59:18.230 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:18.230 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:18.230 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:18.230 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:18.230 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:18.230 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22749 2024-09-21T14:59:18.230 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.231 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:18.231 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.231 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:18.231 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141698+0000", 2024-09-21T14:59:18.231 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:18.231 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.231 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.231 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:18.232 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141698+0000", 2024-09-21T14:59:18.232 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:18.232 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.232 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.232 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:18.232 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141698+0000", 2024-09-21T14:59:18.232 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:18.232 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.232 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.233 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:18.233 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141701+0000", 2024-09-21T14:59:18.233 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8399999999999999e-06 2024-09-21T14:59:18.233 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.233 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.233 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:18.233 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141706+0000", 2024-09-21T14:59:18.233 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4979999999999997e-06 2024-09-21T14:59:18.233 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.233 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.233 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:18.233 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141711+0000", 2024-09-21T14:59:18.233 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3850000000000004e-06 2024-09-21T14:59:18.233 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.233 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.234 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:18.234 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179504+0000", 2024-09-21T14:59:18.234 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037793514 2024-09-21T14:59:18.234 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.234 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.234 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:18.234 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179546+0000", 2024-09-21T14:59:18.234 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1467999999999997e-05 2024-09-21T14:59:18.234 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.234 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.234 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:18.234 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179666+0000", 2024-09-21T14:59:18.234 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000120499 2024-09-21T14:59:18.234 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:18.234 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:18.235 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:18.235 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.235 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.235 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22750 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 835584~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:18.235 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141715+0000", 2024-09-21T14:59:18.235 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 28.024343324, 2024-09-21T14:59:18.235 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038173179000000002, 2024-09-21T14:59:18.235 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:18.235 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:18.235 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:18.235 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:18.235 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:18.235 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22750 2024-09-21T14:59:18.235 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.236 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:18.236 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.236 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:18.236 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141715+0000", 2024-09-21T14:59:18.236 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:18.236 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.236 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.236 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:18.236 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141715+0000", 2024-09-21T14:59:18.236 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:18.236 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.236 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.236 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:18.236 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141714+0000", 2024-09-21T14:59:18.236 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:18.237 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.237 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.237 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:18.237 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141716+0000", 2024-09-21T14:59:18.237 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1730000000000002e-06 2024-09-21T14:59:18.237 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.237 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.237 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:18.237 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141722+0000", 2024-09-21T14:59:18.237 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.417e-06 2024-09-21T14:59:18.237 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.237 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.237 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:18.237 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141726+0000", 2024-09-21T14:59:18.237 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4830000000000001e-06 2024-09-21T14:59:18.237 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.238 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.238 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:18.238 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179683+0000", 2024-09-21T14:59:18.238 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037956977000000003 2024-09-21T14:59:18.238 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.238 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.238 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:18.238 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179751+0000", 2024-09-21T14:59:18.238 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7762000000000006e-05 2024-09-21T14:59:18.238 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.238 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.238 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:18.238 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179888+0000", 2024-09-21T14:59:18.238 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000136635 2024-09-21T14:59:18.238 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:18.239 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:18.239 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:18.239 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.239 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.239 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22751 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 901120~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:18.239 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141730+0000", 2024-09-21T14:59:18.239 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 28.024327525, 2024-09-21T14:59:18.239 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038357011000000003, 2024-09-21T14:59:18.239 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:18.239 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:18.239 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:18.239 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:18.239 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:18.239 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22751 2024-09-21T14:59:18.239 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.239 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:18.240 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.240 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:18.240 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141730+0000", 2024-09-21T14:59:18.240 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:18.240 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.240 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.240 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:18.240 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141730+0000", 2024-09-21T14:59:18.240 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:18.240 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.240 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.240 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:18.240 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141730+0000", 2024-09-21T14:59:18.240 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:18.240 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.240 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.241 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:18.241 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141737+0000", 2024-09-21T14:59:18.241 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.64e-06 2024-09-21T14:59:18.241 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.241 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.241 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:18.241 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141743+0000", 2024-09-21T14:59:18.241 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8130000000000001e-06 2024-09-21T14:59:18.241 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.241 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.241 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:18.241 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141747+0000", 2024-09-21T14:59:18.241 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4920000000000004e-06 2024-09-21T14:59:18.241 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.241 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.241 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:18.242 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179918+0000", 2024-09-21T14:59:18.242 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038170822 2024-09-21T14:59:18.242 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.242 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.242 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:18.242 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179960+0000", 2024-09-21T14:59:18.242 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1999000000000002e-05 2024-09-21T14:59:18.242 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.242 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.242 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:18.242 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180087+0000", 2024-09-21T14:59:18.242 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012752799999999999 2024-09-21T14:59:18.242 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:18.243 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:18.243 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:18.243 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.243 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.243 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22752 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 966656~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:18.243 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141751+0000", 2024-09-21T14:59:18.243 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 28.024307204999999, 2024-09-21T14:59:18.243 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038553084000000001, 2024-09-21T14:59:18.243 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:18.243 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:18.243 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:18.243 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:18.243 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:18.243 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22752 2024-09-21T14:59:18.244 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.244 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:18.244 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.244 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:18.244 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141751+0000", 2024-09-21T14:59:18.244 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:18.244 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.244 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.244 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:18.244 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141751+0000", 2024-09-21T14:59:18.244 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:18.244 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.244 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.244 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:18.245 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141750+0000", 2024-09-21T14:59:18.245 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:18.245 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.245 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.245 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:18.245 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141753+0000", 2024-09-21T14:59:18.245 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5900000000000002e-06 2024-09-21T14:59:18.245 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.245 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.245 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:18.245 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141759+0000", 2024-09-21T14:59:18.245 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.947e-06 2024-09-21T14:59:18.245 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.245 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.245 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:18.245 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141764+0000", 2024-09-21T14:59:18.246 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8319999999999996e-06 2024-09-21T14:59:18.246 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.246 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.246 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:18.246 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180108+0000", 2024-09-21T14:59:18.246 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038344095000000002 2024-09-21T14:59:18.246 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.246 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.246 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:18.246 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180168+0000", 2024-09-21T14:59:18.246 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0047000000000003e-05 2024-09-21T14:59:18.246 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.246 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.246 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:18.246 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180304+0000", 2024-09-21T14:59:18.247 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013592500000000001 2024-09-21T14:59:18.247 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:18.247 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:18.247 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:18.247 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.247 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.247 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22753 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 1032192~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:18.247 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141768+0000", 2024-09-21T14:59:18.247 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 28.024290331, 2024-09-21T14:59:18.247 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038818810000000002, 2024-09-21T14:59:18.247 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:18.247 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:18.247 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:18.247 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:18.247 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:18.247 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22753 2024-09-21T14:59:18.248 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.248 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:18.248 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.248 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:18.248 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141768+0000", 2024-09-21T14:59:18.248 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:18.248 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.248 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.248 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:18.248 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141768+0000", 2024-09-21T14:59:18.248 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:18.248 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.248 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.248 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:18.248 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141767+0000", 2024-09-21T14:59:18.248 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:18.249 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.249 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.249 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:18.249 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141770+0000", 2024-09-21T14:59:18.249 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3329999999999999e-06 2024-09-21T14:59:18.249 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.249 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.249 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:18.249 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141775+0000", 2024-09-21T14:59:18.249 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2669999999999997e-06 2024-09-21T14:59:18.249 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.249 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.249 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:18.249 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141779+0000", 2024-09-21T14:59:18.249 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2259999999999999e-06 2024-09-21T14:59:18.249 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.250 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.250 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:18.250 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180325+0000", 2024-09-21T14:59:18.250 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038546428000000001 2024-09-21T14:59:18.250 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.250 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.250 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:18.250 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180401+0000", 2024-09-21T14:59:18.250 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5437000000000003e-05 2024-09-21T14:59:18.250 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:18.250 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:18.250 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:18.250 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180586+0000", 2024-09-21T14:59:18.250 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000185429 2024-09-21T14:59:18.250 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:18.251 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:18.251 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:18.251 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:18.251 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:18.251 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:18.252 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T14:59:18.252 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:59:18.247+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:59:18.352 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:59:18.353+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:59:18.365 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:59:18.366 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:59:18.366 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T14:59:18.366 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:59:18.375 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T14:59:18.453 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:59:18.453+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:59:18.480 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:59:18.480 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:59:18.480 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T14:59:18.480 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T14:59:18.480 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:59:18.490 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T14:59:18.554 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:59:18.553+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:59:18.599 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:59:18.599 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T14:59:18.599 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T14:59:18.599 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T14:59:18.599 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.599 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:1452 3.19 3:9d40fe72:::rbd_data.1136515ecb6b.0000000000000013:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T14:59:18.600 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:30.292755+0000", 2024-09-21T14:59:18.600 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 48.305088386999998, 2024-09-21T14:59:18.600 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043852191999999998, 2024-09-21T14:59:18.600 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:18.600 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:18.600 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:18.600 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:18.600 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:18.600 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1452 2024-09-21T14:59:18.600 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.600 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:18.600 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.600 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:18.600 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292755+0000", 2024-09-21T14:59:18.600 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:18.600 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.601 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.601 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:18.601 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292755+0000", 2024-09-21T14:59:18.601 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:18.601 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.601 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.601 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:18.601 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292754+0000", 2024-09-21T14:59:18.601 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:18.601 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.601 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.601 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:18.601 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292760+0000", 2024-09-21T14:59:18.601 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5029999999999997e-06 2024-09-21T14:59:18.601 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.602 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.602 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:18.602 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292771+0000", 2024-09-21T14:59:18.602 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1551e-05 2024-09-21T14:59:18.602 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.602 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.602 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:18.602 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292779+0000", 2024-09-21T14:59:18.602 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1559999999999998e-06 2024-09-21T14:59:18.602 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.602 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.602 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:18.602 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292896+0000", 2024-09-21T14:59:18.602 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000117304 2024-09-21T14:59:18.603 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.603 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.603 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:18.603 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.293124+0000", 2024-09-21T14:59:18.603 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00022819999999999999 2024-09-21T14:59:18.603 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.603 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.603 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 1", 2024-09-21T14:59:18.603 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.299967+0000", 2024-09-21T14:59:18.603 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0068427009999999996 2024-09-21T14:59:18.603 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.603 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.603 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:18.603 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336463+0000", 2024-09-21T14:59:18.603 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.036495842000000001 2024-09-21T14:59:18.604 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.604 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.604 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:59:18.604 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336480+0000", 2024-09-21T14:59:18.604 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7419999999999999e-05 2024-09-21T14:59:18.604 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.604 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.604 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:59:18.604 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336550+0000", 2024-09-21T14:59:18.604 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9951999999999997e-05 2024-09-21T14:59:18.604 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.604 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.604 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:18.604 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336607+0000", 2024-09-21T14:59:18.604 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7423000000000002e-05 2024-09-21T14:59:18.605 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:18.605 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:18.605 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:18.605 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.605 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.605 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop_reply(client.4406.0:1452 3.19 e36/24 ondisk, result = 0)", 2024-09-21T14:59:18.605 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:30.322854+0000", 2024-09-21T14:59:18.605 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 48.274989711000003, 2024-09-21T14:59:18.605 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013613079, 2024-09-21T14:59:18.605 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:18.605 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:18.605 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:18.605 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.605 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:18.606 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.322854+0000", 2024-09-21T14:59:18.606 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:18.606 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.606 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.606 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:18.606 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.322854+0000", 2024-09-21T14:59:18.606 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:18.606 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.606 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.606 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:18.606 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.322852+0000", 2024-09-21T14:59:18.606 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:59:18.606 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.607 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.607 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:18.607 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.322860+0000", 2024-09-21T14:59:18.607 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.605e-06 2024-09-21T14:59:18.607 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.607 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.607 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:18.607 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.322878+0000", 2024-09-21T14:59:18.607 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8204000000000001e-05 2024-09-21T14:59:18.607 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.607 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.607 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:18.607 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.322888+0000", 2024-09-21T14:59:18.608 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4099999999999997e-06 2024-09-21T14:59:18.608 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.608 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.608 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:18.608 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336440+0000", 2024-09-21T14:59:18.608 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013551848 2024-09-21T14:59:18.608 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.608 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.608 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:18.608 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336453+0000", 2024-09-21T14:59:18.608 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3465000000000001e-05 2024-09-21T14:59:18.608 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.608 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.608 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:18.608 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336467+0000", 2024-09-21T14:59:18.609 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3913999999999999e-05 2024-09-21T14:59:18.609 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:18.609 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:18.609 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:18.609 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.609 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.609 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:1567 3.8 3:139d3d55:::rbd_data.1136515ecb6b.0000000000000000:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T14:59:18.609 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:30.681087+0000", 2024-09-21T14:59:18.609 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 47.916756171999999, 2024-09-21T14:59:18.609 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.039009224000000002, 2024-09-21T14:59:18.609 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:18.609 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:18.609 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:18.609 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:18.609 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:18.609 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1567 2024-09-21T14:59:18.610 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.610 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:18.610 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.610 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:18.610 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681087+0000", 2024-09-21T14:59:18.610 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:18.610 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.610 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.610 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:18.610 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681087+0000", 2024-09-21T14:59:18.610 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:18.610 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.610 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.610 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:18.610 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681086+0000", 2024-09-21T14:59:18.611 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:59:18.611 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.611 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.611 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:18.611 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681097+0000", 2024-09-21T14:59:18.611 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.134e-05 2024-09-21T14:59:18.611 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.611 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.611 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:18.611 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681122+0000", 2024-09-21T14:59:18.611 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5313e-05 2024-09-21T14:59:18.611 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.611 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.611 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:18.611 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681138+0000", 2024-09-21T14:59:18.612 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5916000000000001e-05 2024-09-21T14:59:18.612 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.612 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.612 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:18.612 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681270+0000", 2024-09-21T14:59:18.612 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000131474 2024-09-21T14:59:18.612 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.612 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.612 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:18.612 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681416+0000", 2024-09-21T14:59:18.612 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014637 2024-09-21T14:59:18.612 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.612 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.612 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 7", 2024-09-21T14:59:18.612 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.693163+0000", 2024-09-21T14:59:18.613 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011747241 2024-09-21T14:59:18.613 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.613 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.613 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:59:18.613 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.719750+0000", 2024-09-21T14:59:18.613 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.026586852000000001 2024-09-21T14:59:18.613 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.613 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.613 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:18.613 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.719859+0000", 2024-09-21T14:59:18.613 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010925999999999999 2024-09-21T14:59:18.613 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.613 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.613 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:59:18.613 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.719951+0000", 2024-09-21T14:59:18.613 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1294999999999999e-05 2024-09-21T14:59:18.614 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.614 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.614 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:18.614 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.720097+0000", 2024-09-21T14:59:18.614 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014583 2024-09-21T14:59:18.614 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:18.614 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:18.614 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:18.614 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.614 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.614 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9347 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 16384~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:18.614 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158291+0000", 2024-09-21T14:59:18.614 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 39.439552671000001, 2024-09-21T14:59:18.614 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013600958, 2024-09-21T14:59:18.614 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:18.615 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:18.615 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:18.615 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:18.615 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:18.615 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9347 2024-09-21T14:59:18.615 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.615 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:18.615 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.615 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:18.615 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158291+0000", 2024-09-21T14:59:18.615 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:18.615 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.615 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.615 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:18.615 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158291+0000", 2024-09-21T14:59:18.616 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:18.616 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.616 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.616 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:18.616 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158290+0000", 2024-09-21T14:59:18.616 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:18.616 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.616 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.616 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:18.616 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158295+0000", 2024-09-21T14:59:18.616 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2399999999999998e-06 2024-09-21T14:59:18.616 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.616 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.616 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:18.617 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158312+0000", 2024-09-21T14:59:18.617 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6929999999999999e-05 2024-09-21T14:59:18.617 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.617 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.617 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:18.617 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158353+0000", 2024-09-21T14:59:18.617 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0805e-05 2024-09-21T14:59:18.617 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.617 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.617 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:18.617 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.171634+0000", 2024-09-21T14:59:18.617 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.01328085 2024-09-21T14:59:18.617 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.617 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.617 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:18.617 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.171722+0000", 2024-09-21T14:59:18.618 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8115000000000006e-05 2024-09-21T14:59:18.618 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.618 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.618 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:18.618 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.171892+0000", 2024-09-21T14:59:18.618 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000169722 2024-09-21T14:59:18.618 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:18.618 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:18.618 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:18.618 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.618 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.618 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9348 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 122880~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:18.618 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158369+0000", 2024-09-21T14:59:18.618 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 39.439474503, 2024-09-21T14:59:18.618 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.01388847, 2024-09-21T14:59:18.619 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:18.619 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:18.619 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:18.619 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:18.619 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:18.619 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9348 2024-09-21T14:59:18.619 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.619 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:18.619 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.619 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:18.619 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158369+0000", 2024-09-21T14:59:18.619 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:18.619 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.619 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.620 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:18.620 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158369+0000", 2024-09-21T14:59:18.620 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:18.620 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.620 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.620 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:18.620 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158368+0000", 2024-09-21T14:59:18.620 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:18.620 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.620 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.620 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:18.620 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158374+0000", 2024-09-21T14:59:18.620 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0499999999999997e-06 2024-09-21T14:59:18.620 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.620 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.620 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:18.621 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158386+0000", 2024-09-21T14:59:18.621 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1579999999999999e-05 2024-09-21T14:59:18.621 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.621 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.621 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:18.621 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158396+0000", 2024-09-21T14:59:18.621 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.8260000000000005e-06 2024-09-21T14:59:18.621 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.621 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.621 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:18.621 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.171967+0000", 2024-09-21T14:59:18.621 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013571318000000001 2024-09-21T14:59:18.621 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.621 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.621 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:18.621 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172034+0000", 2024-09-21T14:59:18.622 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.656e-05 2024-09-21T14:59:18.622 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.622 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.622 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:18.622 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172258+0000", 2024-09-21T14:59:18.622 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00022395999999999999 2024-09-21T14:59:18.622 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:18.622 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:18.622 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:18.622 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.622 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.622 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9349 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 155648~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:18.622 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158403+0000", 2024-09-21T14:59:18.622 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 39.439440757, 2024-09-21T14:59:18.622 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014229851, 2024-09-21T14:59:18.623 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:18.623 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:18.623 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:18.623 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:18.623 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:18.623 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9349 2024-09-21T14:59:18.623 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.623 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:18.623 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.623 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:18.623 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158403+0000", 2024-09-21T14:59:18.623 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:18.623 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.623 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.623 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:18.623 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158403+0000", 2024-09-21T14:59:18.624 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:18.624 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.624 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.624 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:18.624 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158402+0000", 2024-09-21T14:59:18.624 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:18.624 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.624 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.624 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:18.624 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158406+0000", 2024-09-21T14:59:18.624 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1339999999999997e-06 2024-09-21T14:59:18.624 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.624 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.624 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:18.624 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158416+0000", 2024-09-21T14:59:18.625 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3149999999999998e-06 2024-09-21T14:59:18.625 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.625 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.625 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:18.625 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158439+0000", 2024-09-21T14:59:18.625 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3255999999999999e-05 2024-09-21T14:59:18.625 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.625 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.625 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:18.625 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172305+0000", 2024-09-21T14:59:18.625 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013866603999999999 2024-09-21T14:59:18.625 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.625 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.625 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:18.625 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172413+0000", 2024-09-21T14:59:18.625 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000107664 2024-09-21T14:59:18.626 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.626 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.626 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:18.626 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172633+0000", 2024-09-21T14:59:18.626 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021950800000000001 2024-09-21T14:59:18.626 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:18.626 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:18.626 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:18.626 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.626 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.626 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9350 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 294912~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:18.626 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158448+0000", 2024-09-21T14:59:18.626 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 39.439395886, 2024-09-21T14:59:18.627 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014471525000000001, 2024-09-21T14:59:18.627 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:18.627 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:18.627 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:18.627 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:18.627 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:18.627 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9350 2024-09-21T14:59:18.627 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.627 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:18.627 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.627 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:18.627 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158448+0000", 2024-09-21T14:59:18.627 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:18.627 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.627 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.627 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:18.628 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158448+0000", 2024-09-21T14:59:18.628 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:18.628 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.628 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.628 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:18.628 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158447+0000", 2024-09-21T14:59:18.628 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:18.628 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.628 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.628 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:18.628 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158452+0000", 2024-09-21T14:59:18.628 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8400000000000002e-06 2024-09-21T14:59:18.628 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.628 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.628 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:18.628 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158475+0000", 2024-09-21T14:59:18.629 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3091e-05 2024-09-21T14:59:18.629 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.629 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.629 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:18.629 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158515+0000", 2024-09-21T14:59:18.629 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0265000000000002e-05 2024-09-21T14:59:18.629 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.629 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.629 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:18.629 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172682+0000", 2024-09-21T14:59:18.629 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014166761999999999 2024-09-21T14:59:18.629 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.629 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.630 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:18.630 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172752+0000", 2024-09-21T14:59:18.630 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9698999999999998e-05 2024-09-21T14:59:18.630 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.630 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.630 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:18.630 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172919+0000", 2024-09-21T14:59:18.630 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016759199999999999 2024-09-21T14:59:18.630 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:18.630 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:18.630 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:18.630 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.630 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.630 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9351 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 434176~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:18.630 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158526+0000", 2024-09-21T14:59:18.631 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 39.439317168000002, 2024-09-21T14:59:18.631 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014662107000000001, 2024-09-21T14:59:18.631 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:18.631 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:18.631 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:18.631 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:18.631 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:18.631 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9351 2024-09-21T14:59:18.631 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.631 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:18.631 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.631 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:18.631 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158526+0000", 2024-09-21T14:59:18.631 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:18.632 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.632 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.632 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:18.632 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158526+0000", 2024-09-21T14:59:18.632 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:18.632 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.632 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.632 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:18.632 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158525+0000", 2024-09-21T14:59:18.632 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:18.632 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.632 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.632 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:18.632 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158531+0000", 2024-09-21T14:59:18.633 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5840000000000002e-06 2024-09-21T14:59:18.633 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.633 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.633 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:18.633 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158564+0000", 2024-09-21T14:59:18.633 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3195000000000003e-05 2024-09-21T14:59:18.633 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.633 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.633 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:18.633 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158577+0000", 2024-09-21T14:59:18.633 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2330000000000001e-05 2024-09-21T14:59:18.633 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.633 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.633 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:18.634 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172956+0000", 2024-09-21T14:59:18.634 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014379438 2024-09-21T14:59:18.634 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.634 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.634 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:18.634 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173020+0000", 2024-09-21T14:59:18.634 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4401999999999998e-05 2024-09-21T14:59:18.634 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.634 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.634 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:18.634 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173188+0000", 2024-09-21T14:59:18.634 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016804 2024-09-21T14:59:18.634 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:18.634 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:18.634 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:18.635 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.635 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.635 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9379 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 24576~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:18.635 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158586+0000", 2024-09-21T14:59:18.635 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 39.439257908000002, 2024-09-21T14:59:18.635 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014865173000000001, 2024-09-21T14:59:18.635 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:18.635 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:18.635 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:18.635 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:18.635 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:18.635 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9379 2024-09-21T14:59:18.635 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.635 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:18.635 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.636 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:18.636 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158586+0000", 2024-09-21T14:59:18.636 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:18.636 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.636 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.636 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:18.636 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158586+0000", 2024-09-21T14:59:18.636 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:18.636 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.636 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.636 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:18.636 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158585+0000", 2024-09-21T14:59:18.636 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:18.636 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.636 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.637 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:18.637 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158590+0000", 2024-09-21T14:59:18.637 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5820000000000001e-06 2024-09-21T14:59:18.637 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.637 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.637 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:18.637 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158676+0000", 2024-09-21T14:59:18.637 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5075999999999997e-05 2024-09-21T14:59:18.637 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.637 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.637 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:18.637 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158695+0000", 2024-09-21T14:59:18.637 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9137999999999999e-05 2024-09-21T14:59:18.637 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.637 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.637 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:18.638 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173219+0000", 2024-09-21T14:59:18.638 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014524034999999999 2024-09-21T14:59:18.638 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.638 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.638 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:18.638 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173276+0000", 2024-09-21T14:59:18.638 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7442000000000003e-05 2024-09-21T14:59:18.638 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.638 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.638 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:18.638 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173451+0000", 2024-09-21T14:59:18.638 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017464800000000001 2024-09-21T14:59:18.638 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:18.639 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:18.639 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:18.639 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.639 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.639 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9380 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 163840~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:18.639 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158706+0000", 2024-09-21T14:59:18.639 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 39.439137398, 2024-09-21T14:59:18.639 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015096381000000001, 2024-09-21T14:59:18.639 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:18.639 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:18.639 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:18.639 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:18.639 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:18.639 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9380 2024-09-21T14:59:18.639 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.639 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:18.640 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.640 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:18.640 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158706+0000", 2024-09-21T14:59:18.640 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:18.640 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.640 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.640 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:18.640 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158706+0000", 2024-09-21T14:59:18.640 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:18.640 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.640 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.640 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:18.640 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158705+0000", 2024-09-21T14:59:18.640 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:18.640 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.640 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.641 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:18.641 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158712+0000", 2024-09-21T14:59:18.641 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7240000000000003e-06 2024-09-21T14:59:18.641 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.641 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.641 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:18.641 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158755+0000", 2024-09-21T14:59:18.641 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2953e-05 2024-09-21T14:59:18.641 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.641 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.641 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:18.641 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158765+0000", 2024-09-21T14:59:18.641 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0149000000000001e-05 2024-09-21T14:59:18.641 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.641 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.641 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:18.642 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173492+0000", 2024-09-21T14:59:18.642 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014727105000000001 2024-09-21T14:59:18.642 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.642 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.642 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:18.642 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173564+0000", 2024-09-21T14:59:18.642 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1789e-05 2024-09-21T14:59:18.642 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.642 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.642 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:18.642 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173803+0000", 2024-09-21T14:59:18.642 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00023867100000000001 2024-09-21T14:59:18.642 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:18.642 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:18.642 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:18.643 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.643 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.643 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9381 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 303104~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:18.643 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158774+0000", 2024-09-21T14:59:18.643 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 39.439069318000001, 2024-09-21T14:59:18.643 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015309095999999999, 2024-09-21T14:59:18.643 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:18.643 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:18.643 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:18.643 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:18.643 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:18.643 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9381 2024-09-21T14:59:18.643 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.644 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:18.644 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.644 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:18.644 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158774+0000", 2024-09-21T14:59:18.644 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:18.644 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.644 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.644 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:18.644 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158774+0000", 2024-09-21T14:59:18.644 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:18.644 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.644 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.644 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:18.644 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158773+0000", 2024-09-21T14:59:18.644 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:18.645 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.645 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.645 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:18.645 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158779+0000", 2024-09-21T14:59:18.645 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6999999999999996e-06 2024-09-21T14:59:18.645 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.645 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.645 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:18.645 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158790+0000", 2024-09-21T14:59:18.645 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0594000000000001e-05 2024-09-21T14:59:18.645 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.645 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.645 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:18.645 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158798+0000", 2024-09-21T14:59:18.645 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5860000000000005e-06 2024-09-21T14:59:18.645 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.646 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.646 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:18.646 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173829+0000", 2024-09-21T14:59:18.646 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015030349 2024-09-21T14:59:18.646 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.646 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.646 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:18.646 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173918+0000", 2024-09-21T14:59:18.646 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9375000000000004e-05 2024-09-21T14:59:18.646 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.646 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.646 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:18.646 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.174083+0000", 2024-09-21T14:59:18.646 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016536200000000001 2024-09-21T14:59:18.646 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:18.647 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:18.647 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:18.647 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.647 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.647 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9382 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 442368~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:18.647 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158805+0000", 2024-09-21T14:59:18.647 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 39.439038412999999, 2024-09-21T14:59:18.647 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015592623, 2024-09-21T14:59:18.647 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:18.647 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:18.647 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:18.647 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:18.647 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:18.647 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9382 2024-09-21T14:59:18.648 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.648 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:18.648 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.648 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:18.648 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158805+0000", 2024-09-21T14:59:18.648 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:18.648 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.648 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.648 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:18.648 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158805+0000", 2024-09-21T14:59:18.648 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:18.648 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.648 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.648 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:18.648 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158805+0000", 2024-09-21T14:59:18.648 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:18.649 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.649 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.649 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:18.649 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158808+0000", 2024-09-21T14:59:18.649 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9179999999999996e-06 2024-09-21T14:59:18.649 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.649 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.649 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:18.649 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158817+0000", 2024-09-21T14:59:18.649 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9299999999999992e-06 2024-09-21T14:59:18.649 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.649 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.649 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:18.649 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158825+0000", 2024-09-21T14:59:18.649 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6739999999999997e-06 2024-09-21T14:59:18.649 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.650 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.650 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:18.650 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.174122+0000", 2024-09-21T14:59:18.650 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.01529694 2024-09-21T14:59:18.650 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.650 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.650 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:18.650 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.174185+0000", 2024-09-21T14:59:18.650 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3015000000000006e-05 2024-09-21T14:59:18.650 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.650 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.650 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:18.650 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.174398+0000", 2024-09-21T14:59:18.650 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021274799999999999 2024-09-21T14:59:18.650 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:18.651 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:18.651 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:18.651 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.651 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.651 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22699 3.19 3:98de45b8:::rbd_data.11366acc894.0000000000000000:19 [read 638976~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:18.651 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.140173+0000", 2024-09-21T14:59:18.651 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 28.457670316000002, 2024-09-21T14:59:18.651 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013511366, 2024-09-21T14:59:18.651 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:18.651 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:18.651 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:18.651 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:18.651 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:18.651 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22699 2024-09-21T14:59:18.651 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.651 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:18.652 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.652 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:18.652 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140173+0000", 2024-09-21T14:59:18.652 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:18.652 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.652 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.652 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:18.652 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140173+0000", 2024-09-21T14:59:18.652 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:18.652 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.652 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.652 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:18.652 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140173+0000", 2024-09-21T14:59:18.652 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:18.652 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.653 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.653 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:18.653 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140175+0000", 2024-09-21T14:59:18.653 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5859999999999999e-06 2024-09-21T14:59:18.653 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.653 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.653 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:18.653 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140181+0000", 2024-09-21T14:59:18.653 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0340000000000002e-06 2024-09-21T14:59:18.653 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.653 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.653 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:18.653 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140186+0000", 2024-09-21T14:59:18.653 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.04e-06 2024-09-21T14:59:18.653 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.653 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.654 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:18.654 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.153426+0000", 2024-09-21T14:59:18.654 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013240001 2024-09-21T14:59:18.654 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.654 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.654 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:18.654 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.153513+0000", 2024-09-21T14:59:18.654 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6038999999999994e-05 2024-09-21T14:59:18.654 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.654 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.654 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:18.654 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.153685+0000", 2024-09-21T14:59:18.654 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000172067 2024-09-21T14:59:18.654 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:18.654 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:18.655 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:18.655 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.655 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.655 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22700 3.19 3:98de45b8:::rbd_data.11366acc894.0000000000000000:19 [read 704512~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:18.655 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.140191+0000", 2024-09-21T14:59:18.655 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 28.457652799000002, 2024-09-21T14:59:18.655 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013770486, 2024-09-21T14:59:18.655 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:18.655 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:18.655 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:18.655 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:18.655 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:18.655 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22700 2024-09-21T14:59:18.656 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.656 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:18.656 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.656 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:18.656 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140191+0000", 2024-09-21T14:59:18.656 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:18.656 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.656 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.656 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:18.656 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140191+0000", 2024-09-21T14:59:18.656 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:18.656 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.656 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.656 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:18.656 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140190+0000", 2024-09-21T14:59:18.656 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:18.657 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.657 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.657 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:18.657 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140199+0000", 2024-09-21T14:59:18.657 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3340000000000002e-06 2024-09-21T14:59:18.657 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.657 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.657 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:18.657 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140205+0000", 2024-09-21T14:59:18.657 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3820000000000001e-06 2024-09-21T14:59:18.657 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.657 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.657 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:18.657 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140210+0000", 2024-09-21T14:59:18.657 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0969999999999996e-06 2024-09-21T14:59:18.657 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.658 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.658 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:18.658 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.153713+0000", 2024-09-21T14:59:18.658 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013502713 2024-09-21T14:59:18.658 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.658 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.658 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:18.658 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.153786+0000", 2024-09-21T14:59:18.658 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3022999999999995e-05 2024-09-21T14:59:18.658 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.658 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.658 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:18.658 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.153961+0000", 2024-09-21T14:59:18.658 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000175274 2024-09-21T14:59:18.658 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:18.659 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:18.659 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:18.659 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.659 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.659 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22701 3.19 3:98de45b8:::rbd_data.11366acc894.0000000000000000:19 [read 770048~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:18.659 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.140215+0000", 2024-09-21T14:59:18.659 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 28.457628981999999, 2024-09-21T14:59:18.659 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014072497, 2024-09-21T14:59:18.659 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:18.659 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:18.659 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:18.659 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:18.659 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:18.659 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22701 2024-09-21T14:59:18.659 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.659 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:18.660 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.660 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:18.660 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140215+0000", 2024-09-21T14:59:18.660 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:18.660 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.660 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.660 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:18.660 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140215+0000", 2024-09-21T14:59:18.660 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:18.660 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.660 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.660 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:18.660 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140214+0000", 2024-09-21T14:59:18.660 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:18.660 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.661 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.661 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:18.661 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140217+0000", 2024-09-21T14:59:18.661 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.74e-06 2024-09-21T14:59:18.661 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.661 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.661 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:18.661 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140223+0000", 2024-09-21T14:59:18.661 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0929999999999999e-06 2024-09-21T14:59:18.661 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.661 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.661 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:18.661 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140228+0000", 2024-09-21T14:59:18.661 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2279999999999998e-06 2024-09-21T14:59:18.661 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.661 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.662 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:18.662 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.153991+0000", 2024-09-21T14:59:18.662 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013762329 2024-09-21T14:59:18.662 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.662 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.662 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:18.662 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.154098+0000", 2024-09-21T14:59:18.662 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000107117 2024-09-21T14:59:18.662 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.662 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.662 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:18.662 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.154287+0000", 2024-09-21T14:59:18.662 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000189431 2024-09-21T14:59:18.662 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:18.662 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:18.662 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:18.663 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.663 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.663 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22702 3.19 3:98de45b8:::rbd_data.11366acc894.0000000000000000:19 [read 835584~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:18.663 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.140232+0000", 2024-09-21T14:59:18.663 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 28.45761113, 2024-09-21T14:59:18.663 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015338742000000001, 2024-09-21T14:59:18.663 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:18.663 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:18.663 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:18.663 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:18.663 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:18.663 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22702 2024-09-21T14:59:18.663 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.663 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:18.663 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.664 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:18.664 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140232+0000", 2024-09-21T14:59:18.664 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:18.664 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.664 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.664 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:18.664 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140232+0000", 2024-09-21T14:59:18.664 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:18.664 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.664 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.664 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:18.664 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140232+0000", 2024-09-21T14:59:18.664 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:18.664 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.664 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.664 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:18.665 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140234+0000", 2024-09-21T14:59:18.665 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.402e-06 2024-09-21T14:59:18.665 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.665 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.665 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:18.665 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140241+0000", 2024-09-21T14:59:18.665 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.054e-06 2024-09-21T14:59:18.665 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.665 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.665 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:18.665 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140246+0000", 2024-09-21T14:59:18.665 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0050000000000004e-06 2024-09-21T14:59:18.665 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.666 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.666 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:18.666 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.154315+0000", 2024-09-21T14:59:18.666 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014069440000000001 2024-09-21T14:59:18.666 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.666 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.666 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:18.666 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.154383+0000", 2024-09-21T14:59:18.666 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8151000000000005e-05 2024-09-21T14:59:18.666 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.666 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.666 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:18.666 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155571+0000", 2024-09-21T14:59:18.666 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001188029 2024-09-21T14:59:18.667 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:18.667 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:18.667 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:18.667 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.667 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.667 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22703 3.19 3:98de45b8:::rbd_data.11366acc894.0000000000000000:19 [read 901120~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:18.667 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.140250+0000", 2024-09-21T14:59:18.667 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 28.457593972000002, 2024-09-21T14:59:18.667 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014992169, 2024-09-21T14:59:18.667 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:18.667 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:18.667 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:18.667 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:18.667 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:18.667 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22703 2024-09-21T14:59:18.668 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.668 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:18.668 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.668 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:18.668 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140250+0000", 2024-09-21T14:59:18.668 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:18.668 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.668 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.668 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:18.668 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140250+0000", 2024-09-21T14:59:18.668 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:18.668 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.668 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.668 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:18.668 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140249+0000", 2024-09-21T14:59:18.668 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:18.669 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.669 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.669 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:18.669 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140251+0000", 2024-09-21T14:59:18.669 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.136e-06 2024-09-21T14:59:18.669 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.669 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.669 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:18.669 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140257+0000", 2024-09-21T14:59:18.669 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9610000000000002e-06 2024-09-21T14:59:18.669 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.669 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.669 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:18.669 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140262+0000", 2024-09-21T14:59:18.669 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4549999999999997e-06 2024-09-21T14:59:18.670 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.670 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.670 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:18.670 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155006+0000", 2024-09-21T14:59:18.670 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014744189 2024-09-21T14:59:18.670 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.670 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.670 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:18.670 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155081+0000", 2024-09-21T14:59:18.670 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4548000000000005e-05 2024-09-21T14:59:18.670 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.670 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.670 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:18.671 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155242+0000", 2024-09-21T14:59:18.671 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000161204 2024-09-21T14:59:18.671 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:18.671 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:18.671 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:18.671 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.671 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.671 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22704 3.19 3:98de45b8:::rbd_data.11366acc894.0000000000000000:19 [read 966656~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:18.671 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.140266+0000", 2024-09-21T14:59:18.671 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 28.457577655000001, 2024-09-21T14:59:18.671 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015223443999999999, 2024-09-21T14:59:18.671 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:18.671 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:18.671 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:18.671 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:18.671 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:18.672 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22704 2024-09-21T14:59:18.672 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.672 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:18.672 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.672 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:18.672 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140266+0000", 2024-09-21T14:59:18.672 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:18.672 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.672 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.672 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:18.672 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140266+0000", 2024-09-21T14:59:18.672 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:18.672 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.672 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.672 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:18.672 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140266+0000", 2024-09-21T14:59:18.673 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:18.673 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.673 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.673 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:18.673 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140268+0000", 2024-09-21T14:59:18.673 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.475e-06 2024-09-21T14:59:18.673 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.673 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.673 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:18.673 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140274+0000", 2024-09-21T14:59:18.673 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.319e-06 2024-09-21T14:59:18.673 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.673 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.673 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:18.673 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140279+0000", 2024-09-21T14:59:18.674 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.882e-06 2024-09-21T14:59:18.674 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.674 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.674 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:18.674 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155264+0000", 2024-09-21T14:59:18.674 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014984655 2024-09-21T14:59:18.674 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.674 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.674 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:18.674 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155338+0000", 2024-09-21T14:59:18.674 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4091999999999995e-05 2024-09-21T14:59:18.674 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.674 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.674 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:18.675 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155489+0000", 2024-09-21T14:59:18.675 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000151349 2024-09-21T14:59:18.675 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:18.675 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:18.675 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:18.675 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.675 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.675 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22705 3.19 3:98de45b8:::rbd_data.11366acc894.0000000000000000:19 [read 1032192~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:18.675 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.140284+0000", 2024-09-21T14:59:18.675 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 28.457559797999998, 2024-09-21T14:59:18.675 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015467896, 2024-09-21T14:59:18.675 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:18.675 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:18.675 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:18.675 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:18.676 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:18.676 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22705 2024-09-21T14:59:18.676 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.676 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:18.676 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.676 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:18.676 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140284+0000", 2024-09-21T14:59:18.676 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:18.676 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.676 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.676 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:18.676 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140284+0000", 2024-09-21T14:59:18.676 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:18.676 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.676 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.676 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:18.677 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140283+0000", 2024-09-21T14:59:18.677 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:18.677 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.677 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.677 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:18.677 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140286+0000", 2024-09-21T14:59:18.677 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7130000000000001e-06 2024-09-21T14:59:18.677 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.677 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.677 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:18.677 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140292+0000", 2024-09-21T14:59:18.677 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3430000000000001e-06 2024-09-21T14:59:18.677 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.677 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.677 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:18.677 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140298+0000", 2024-09-21T14:59:18.678 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9730000000000002e-06 2024-09-21T14:59:18.678 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.678 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.678 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:18.678 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155510+0000", 2024-09-21T14:59:18.678 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015211738000000001 2024-09-21T14:59:18.678 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.678 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.678 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:18.678 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155598+0000", 2024-09-21T14:59:18.678 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7822e-05 2024-09-21T14:59:18.678 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.678 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.679 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:18.679 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155752+0000", 2024-09-21T14:59:18.679 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0001537 2024-09-21T14:59:18.679 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:18.679 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:18.679 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:18.679 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.679 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.679 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:37892 3.1b 3:d963c97f:::rbd_data.11365d19aa81.00000000000000be:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-21T14:59:18.679 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:04.029946+0000", 2024-09-21T14:59:18.679 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.567898031, 2024-09-21T14:59:18.679 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.017686021999999999, 2024-09-21T14:59:18.679 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:18.679 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:18.680 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:18.680 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:18.680 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:18.680 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 37892 2024-09-21T14:59:18.680 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.680 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:18.680 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.680 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:18.680 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.029946+0000", 2024-09-21T14:59:18.680 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:18.680 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.680 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.680 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:18.680 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.029946+0000", 2024-09-21T14:59:18.680 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:18.681 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.681 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.681 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:18.681 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.029944+0000", 2024-09-21T14:59:18.681 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T14:59:18.681 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.681 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.681 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:18.681 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.029953+0000", 2024-09-21T14:59:18.681 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7589999999999992e-06 2024-09-21T14:59:18.681 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.681 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.681 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:18.681 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.029971+0000", 2024-09-21T14:59:18.681 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7795000000000002e-05 2024-09-21T14:59:18.682 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.682 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.682 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:18.682 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.029984+0000", 2024-09-21T14:59:18.682 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.27e-05 2024-09-21T14:59:18.682 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.682 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.682 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:18.682 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.030072+0000", 2024-09-21T14:59:18.682 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8082000000000001e-05 2024-09-21T14:59:18.682 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.682 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.682 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:18.682 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.030276+0000", 2024-09-21T14:59:18.682 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00020375699999999999 2024-09-21T14:59:18.683 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.683 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.683 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-21T14:59:18.683 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.031634+0000", 2024-09-21T14:59:18.683 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0013586049999999999 2024-09-21T14:59:18.683 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.683 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.683 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:59:18.683 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.037554+0000", 2024-09-21T14:59:18.683 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0059197449999999997 2024-09-21T14:59:18.683 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.683 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.683 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:18.683 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.047490+0000", 2024-09-21T14:59:18.683 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0099361680000000004 2024-09-21T14:59:18.683 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.684 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.684 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:59:18.684 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.047569+0000", 2024-09-21T14:59:18.684 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8298000000000001e-05 2024-09-21T14:59:18.684 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:18.684 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:18.684 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:18.684 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.047632+0000", 2024-09-21T14:59:18.684 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3006e-05 2024-09-21T14:59:18.684 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:18.684 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:18.684 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:18.684 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:18.684 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:18.685 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:59:18.686 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T14:59:18.686 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:59:18.685+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:59:18.787 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:59:18.785+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:59:18.800 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:59:18.800 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:59:18.800 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T14:59:18.800 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:59:18.809 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T14:59:18.888 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:59:18.883+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:59:18.919 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:59:18.919 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:59:18.919 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T14:59:18.919 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T14:59:18.919 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:59:18.929 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T14:59:18.988 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:59:18.985+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:59:19.085 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:59:19.085 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T14:59:19.085 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T14:59:19.086 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T14:59:19.086 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.086 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45834 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:19.086 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159133+0000", 2024-09-21T14:59:19.086 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 8.9235707949999998, 2024-09-21T14:59:19.086 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068236884999999997, 2024-09-21T14:59:19.086 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:19.086 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:19.086 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:19.086 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:19.086 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:19.086 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45834 2024-09-21T14:59:19.087 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.087 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:19.087 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.087 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:19.087 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159133+0000", 2024-09-21T14:59:19.087 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:19.087 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.087 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.087 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:19.087 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159133+0000", 2024-09-21T14:59:19.087 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:19.087 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.087 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.088 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:19.088 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159133+0000", 2024-09-21T14:59:19.088 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:19.088 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.088 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.088 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:19.088 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159135+0000", 2024-09-21T14:59:19.088 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4500000000000001e-06 2024-09-21T14:59:19.088 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.088 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.088 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:19.088 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159138+0000", 2024-09-21T14:59:19.089 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2770000000000001e-06 2024-09-21T14:59:19.089 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.089 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.089 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:19.089 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159141+0000", 2024-09-21T14:59:19.089 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9210000000000001e-06 2024-09-21T14:59:19.089 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.089 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.089 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:19.089 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227361+0000", 2024-09-21T14:59:19.089 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068220154000000005 2024-09-21T14:59:19.090 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.090 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.090 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:19.090 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227370+0000", 2024-09-21T14:59:19.090 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3239999999999992e-06 2024-09-21T14:59:19.090 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:19.090 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:19.090 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:19.090 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.090 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.090 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45839 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:19.090 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159313+0000", 2024-09-21T14:59:19.090 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 8.9233908290000006, 2024-09-21T14:59:19.090 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068238699, 2024-09-21T14:59:19.091 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:19.091 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:19.091 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:19.091 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:19.091 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:19.091 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45839 2024-09-21T14:59:19.091 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.091 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:19.091 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.091 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:19.091 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159313+0000", 2024-09-21T14:59:19.091 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:19.091 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.092 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.092 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:19.092 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159313+0000", 2024-09-21T14:59:19.092 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:19.092 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.092 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.092 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:19.092 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159313+0000", 2024-09-21T14:59:19.092 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:19.092 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.092 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.092 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:19.092 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159316+0000", 2024-09-21T14:59:19.092 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.033e-06 2024-09-21T14:59:19.093 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.093 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.093 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:19.093 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159322+0000", 2024-09-21T14:59:19.093 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1820000000000003e-06 2024-09-21T14:59:19.093 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.093 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.093 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:19.093 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159327+0000", 2024-09-21T14:59:19.093 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.062e-06 2024-09-21T14:59:19.093 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.093 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.093 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:19.093 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227544+0000", 2024-09-21T14:59:19.094 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068217142999999994 2024-09-21T14:59:19.094 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.094 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.094 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:19.094 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227552+0000", 2024-09-21T14:59:19.094 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6260000000000003e-06 2024-09-21T14:59:19.094 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:19.094 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:19.094 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:19.094 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.094 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.094 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45840 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:19.094 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159332+0000", 2024-09-21T14:59:19.095 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 8.9233725320000001, 2024-09-21T14:59:19.095 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068262046000000007, 2024-09-21T14:59:19.095 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:19.095 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:19.095 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:19.095 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:19.095 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:19.095 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45840 2024-09-21T14:59:19.095 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.095 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:19.095 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.095 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:19.095 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159332+0000", 2024-09-21T14:59:19.096 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:19.096 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.096 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.096 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:19.096 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159332+0000", 2024-09-21T14:59:19.096 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:19.096 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.096 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.096 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:19.096 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159331+0000", 2024-09-21T14:59:19.096 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:19.096 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.096 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.097 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:19.097 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159351+0000", 2024-09-21T14:59:19.097 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0064999999999998e-05 2024-09-21T14:59:19.097 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.097 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.097 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:19.097 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159367+0000", 2024-09-21T14:59:19.097 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5308e-05 2024-09-21T14:59:19.097 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.097 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.097 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:19.097 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159373+0000", 2024-09-21T14:59:19.097 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1840000000000004e-06 2024-09-21T14:59:19.098 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.098 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.098 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:19.098 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227584+0000", 2024-09-21T14:59:19.098 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068211401000000005 2024-09-21T14:59:19.098 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.098 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.098 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:19.098 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227594+0000", 2024-09-21T14:59:19.098 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4120000000000007e-06 2024-09-21T14:59:19.098 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:19.098 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:19.099 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:19.099 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.099 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.099 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45841 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:19.099 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159379+0000", 2024-09-21T14:59:19.099 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 8.9233253759999993, 2024-09-21T14:59:19.099 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068249039999999997, 2024-09-21T14:59:19.099 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:19.099 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:19.099 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:19.099 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:19.099 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:19.099 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45841 2024-09-21T14:59:19.099 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.099 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:19.100 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.100 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:19.100 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159379+0000", 2024-09-21T14:59:19.100 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:19.100 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.100 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.100 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:19.100 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159379+0000", 2024-09-21T14:59:19.100 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:19.100 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.100 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.100 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:19.100 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159378+0000", 2024-09-21T14:59:19.100 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:19.100 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.101 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.101 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:19.101 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159382+0000", 2024-09-21T14:59:19.101 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.641e-06 2024-09-21T14:59:19.101 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.101 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.101 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:19.101 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159421+0000", 2024-09-21T14:59:19.101 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9439e-05 2024-09-21T14:59:19.101 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.101 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.101 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:19.101 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159430+0000", 2024-09-21T14:59:19.101 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8429999999999999e-06 2024-09-21T14:59:19.102 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.102 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.102 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:19.102 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227617+0000", 2024-09-21T14:59:19.102 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068187084999999995 2024-09-21T14:59:19.102 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.102 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.102 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:19.102 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227628+0000", 2024-09-21T14:59:19.102 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0507e-05 2024-09-21T14:59:19.102 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:19.102 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:19.102 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:19.102 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.103 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.103 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45844 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:19.103 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159499+0000", 2024-09-21T14:59:19.103 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 8.9232049900000003, 2024-09-21T14:59:19.103 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068255415, 2024-09-21T14:59:19.103 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:19.103 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:19.103 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:19.103 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:19.103 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:19.103 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45844 2024-09-21T14:59:19.103 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.103 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:19.103 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.104 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:19.104 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159499+0000", 2024-09-21T14:59:19.104 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:19.104 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.104 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.104 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:19.104 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159499+0000", 2024-09-21T14:59:19.104 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:19.104 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.104 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.104 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:19.104 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159499+0000", 2024-09-21T14:59:19.104 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:19.104 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.105 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.105 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:19.105 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159503+0000", 2024-09-21T14:59:19.105 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0999999999999997e-06 2024-09-21T14:59:19.105 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.105 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.105 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:19.105 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159512+0000", 2024-09-21T14:59:19.105 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5850000000000006e-06 2024-09-21T14:59:19.105 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.105 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.105 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:19.105 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159518+0000", 2024-09-21T14:59:19.105 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0909999999999998e-06 2024-09-21T14:59:19.105 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.106 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.106 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:19.106 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227743+0000", 2024-09-21T14:59:19.106 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068224536000000002 2024-09-21T14:59:19.106 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.106 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.106 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:19.106 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227755+0000", 2024-09-21T14:59:19.106 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1659e-05 2024-09-21T14:59:19.106 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:19.106 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:19.106 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:19.106 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.107 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.107 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45845 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:19.107 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159524+0000", 2024-09-21T14:59:19.107 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 8.9231806299999992, 2024-09-21T14:59:19.107 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068267194000000003, 2024-09-21T14:59:19.107 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:19.107 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:19.107 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:19.107 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:19.107 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:19.107 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45845 2024-09-21T14:59:19.107 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.107 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:19.107 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.108 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:19.108 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159524+0000", 2024-09-21T14:59:19.108 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:19.108 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.108 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.108 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:19.108 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159524+0000", 2024-09-21T14:59:19.108 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:19.108 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.108 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.108 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:19.108 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159523+0000", 2024-09-21T14:59:19.108 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:19.108 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.108 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.109 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:19.109 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159526+0000", 2024-09-21T14:59:19.109 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1980000000000001e-06 2024-09-21T14:59:19.109 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.109 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.109 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:19.109 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159533+0000", 2024-09-21T14:59:19.109 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.348e-06 2024-09-21T14:59:19.109 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.109 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.109 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:19.109 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159546+0000", 2024-09-21T14:59:19.109 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3464e-05 2024-09-21T14:59:19.109 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.109 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.110 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:19.110 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227783+0000", 2024-09-21T14:59:19.110 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068236926000000003 2024-09-21T14:59:19.110 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.110 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.110 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:19.110 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227791+0000", 2024-09-21T14:59:19.110 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6669999999999996e-06 2024-09-21T14:59:19.110 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:19.110 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:19.110 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:19.110 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.110 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.110 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45846 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:19.110 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159584+0000", 2024-09-21T14:59:19.111 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 8.9231199879999998, 2024-09-21T14:59:19.111 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068254767999999993, 2024-09-21T14:59:19.111 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:19.111 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:19.111 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:19.111 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:19.111 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:19.111 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45846 2024-09-21T14:59:19.111 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.111 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:19.111 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.111 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:19.111 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159584+0000", 2024-09-21T14:59:19.112 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:19.112 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.112 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.112 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:19.112 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159584+0000", 2024-09-21T14:59:19.112 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:19.112 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.112 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.112 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:19.112 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159583+0000", 2024-09-21T14:59:19.112 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:19.112 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.112 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.112 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:19.113 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159588+0000", 2024-09-21T14:59:19.113 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1330000000000001e-06 2024-09-21T14:59:19.113 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.113 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.113 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:19.113 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159596+0000", 2024-09-21T14:59:19.113 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5469999999999997e-06 2024-09-21T14:59:19.113 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.113 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.113 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:19.113 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159609+0000", 2024-09-21T14:59:19.113 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2785999999999999e-05 2024-09-21T14:59:19.113 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.113 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.114 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:19.114 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227830+0000", 2024-09-21T14:59:19.114 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068220879999999998 2024-09-21T14:59:19.114 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.114 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.114 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:19.114 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227839+0000", 2024-09-21T14:59:19.114 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1239999999999994e-06 2024-09-21T14:59:19.114 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:19.114 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:19.114 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:19.114 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.114 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.115 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45847 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:19.115 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159614+0000", 2024-09-21T14:59:19.115 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 8.923089912, 2024-09-21T14:59:19.115 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068260639999999997, 2024-09-21T14:59:19.115 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:19.115 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:19.115 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:19.115 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:19.115 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:19.115 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45847 2024-09-21T14:59:19.115 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.115 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:19.115 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.115 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:19.116 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159614+0000", 2024-09-21T14:59:19.116 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:19.116 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.116 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.116 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:19.116 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159614+0000", 2024-09-21T14:59:19.116 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:19.116 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.116 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.116 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:19.116 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159614+0000", 2024-09-21T14:59:19.116 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:19.116 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.116 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.117 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:19.117 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159617+0000", 2024-09-21T14:59:19.117 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2909999999999999e-06 2024-09-21T14:59:19.117 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.117 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.117 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:19.117 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159623+0000", 2024-09-21T14:59:19.117 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.511e-06 2024-09-21T14:59:19.117 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.117 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.117 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:19.117 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159631+0000", 2024-09-21T14:59:19.117 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.571e-06 2024-09-21T14:59:19.117 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.117 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.118 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:19.118 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227866+0000", 2024-09-21T14:59:19.118 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068235136000000002 2024-09-21T14:59:19.118 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.118 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.118 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:19.118 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227875+0000", 2024-09-21T14:59:19.118 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7199999999999995e-06 2024-09-21T14:59:19.118 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:19.118 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:19.118 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:19.118 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.118 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.119 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45848 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:19.119 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159636+0000", 2024-09-21T14:59:19.119 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 8.9230681320000009, 2024-09-21T14:59:19.119 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068267283999999998, 2024-09-21T14:59:19.119 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:19.119 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:19.119 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:19.119 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:19.119 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:19.119 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45848 2024-09-21T14:59:19.119 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.119 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:19.119 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.119 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:19.120 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159636+0000", 2024-09-21T14:59:19.120 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:19.120 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.120 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.120 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:19.120 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159636+0000", 2024-09-21T14:59:19.120 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:19.120 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.120 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.120 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:19.120 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159636+0000", 2024-09-21T14:59:19.120 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:19.120 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.120 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.120 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:19.120 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159638+0000", 2024-09-21T14:59:19.121 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6560000000000001e-06 2024-09-21T14:59:19.121 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.121 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.121 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:19.121 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159644+0000", 2024-09-21T14:59:19.121 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0839999999999997e-06 2024-09-21T14:59:19.121 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.121 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.121 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:19.121 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159649+0000", 2024-09-21T14:59:19.121 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7500000000000003e-06 2024-09-21T14:59:19.121 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.121 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.121 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:19.122 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227892+0000", 2024-09-21T14:59:19.122 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068243315999999998 2024-09-21T14:59:19.122 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.122 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.122 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:19.122 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227903+0000", 2024-09-21T14:59:19.122 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0835999999999999e-05 2024-09-21T14:59:19.122 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:19.122 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:19.122 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:19.123 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.123 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.123 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45849 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:19.123 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159667+0000", 2024-09-21T14:59:19.123 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 8.9230373180000004, 2024-09-21T14:59:19.123 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068291051000000005, 2024-09-21T14:59:19.123 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:19.123 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:19.123 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:19.123 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:19.123 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:19.123 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45849 2024-09-21T14:59:19.123 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.123 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:19.124 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.124 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:19.124 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159667+0000", 2024-09-21T14:59:19.124 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:19.124 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.124 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.124 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:19.124 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159667+0000", 2024-09-21T14:59:19.124 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:19.124 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.124 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.124 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:19.124 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159666+0000", 2024-09-21T14:59:19.124 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:19.125 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.125 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.125 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:19.125 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159670+0000", 2024-09-21T14:59:19.125 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.427e-06 2024-09-21T14:59:19.125 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.125 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.125 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:19.125 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159678+0000", 2024-09-21T14:59:19.125 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9869999999999999e-06 2024-09-21T14:59:19.125 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.125 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.125 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:19.125 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159688+0000", 2024-09-21T14:59:19.126 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0631999999999999e-05 2024-09-21T14:59:19.126 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.126 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.126 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:19.126 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227950+0000", 2024-09-21T14:59:19.126 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068261798999999998 2024-09-21T14:59:19.126 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.126 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.126 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:19.126 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227958+0000", 2024-09-21T14:59:19.126 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7510000000000008e-06 2024-09-21T14:59:19.126 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:19.126 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:19.127 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:19.127 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.127 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.127 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45850 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:19.127 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159695+0000", 2024-09-21T14:59:19.127 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 8.9230096070000009, 2024-09-21T14:59:19.127 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068302999000000003, 2024-09-21T14:59:19.127 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:19.127 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:19.127 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:19.127 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:19.127 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:19.127 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45850 2024-09-21T14:59:19.127 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.128 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:19.128 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.128 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:19.128 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159695+0000", 2024-09-21T14:59:19.128 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:19.128 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.128 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.128 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:19.128 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159695+0000", 2024-09-21T14:59:19.128 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:19.128 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.128 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.128 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:19.129 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159694+0000", 2024-09-21T14:59:19.129 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:19.129 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.129 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.129 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:19.129 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159697+0000", 2024-09-21T14:59:19.129 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.258e-06 2024-09-21T14:59:19.129 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.129 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.129 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:19.129 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159704+0000", 2024-09-21T14:59:19.129 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7660000000000001e-06 2024-09-21T14:59:19.129 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.129 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.130 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:19.130 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159713+0000", 2024-09-21T14:59:19.130 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1779999999999992e-06 2024-09-21T14:59:19.130 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.130 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.130 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:19.130 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227989+0000", 2024-09-21T14:59:19.130 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068276015999999995 2024-09-21T14:59:19.130 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.130 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.130 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:19.130 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227998+0000", 2024-09-21T14:59:19.130 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2050000000000002e-06 2024-09-21T14:59:19.130 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:19.131 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:19.131 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:19.131 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.131 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.131 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45851 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:19.131 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159719+0000", 2024-09-21T14:59:19.131 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 8.9229854740000007, 2024-09-21T14:59:19.131 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068320327, 2024-09-21T14:59:19.131 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:19.131 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:19.131 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:19.131 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:19.131 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:19.131 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45851 2024-09-21T14:59:19.131 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.132 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:19.132 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.132 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:19.132 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159719+0000", 2024-09-21T14:59:19.132 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:19.132 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.132 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.132 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:19.132 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159719+0000", 2024-09-21T14:59:19.132 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:19.132 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.132 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.132 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:19.132 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159718+0000", 2024-09-21T14:59:19.132 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:19.132 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.133 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.133 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:19.133 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159721+0000", 2024-09-21T14:59:19.133 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2169999999999999e-06 2024-09-21T14:59:19.133 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.133 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.133 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:19.133 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159730+0000", 2024-09-21T14:59:19.133 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5720000000000003e-06 2024-09-21T14:59:19.133 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.133 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.133 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:19.133 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159735+0000", 2024-09-21T14:59:19.133 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1100000000000002e-06 2024-09-21T14:59:19.134 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.134 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.134 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:19.134 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228029+0000", 2024-09-21T14:59:19.134 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068293624999999997 2024-09-21T14:59:19.134 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.134 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.134 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:19.134 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228039+0000", 2024-09-21T14:59:19.134 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0254e-05 2024-09-21T14:59:19.134 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:19.134 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:19.135 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:19.135 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.135 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.135 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45852 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:19.135 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159740+0000", 2024-09-21T14:59:19.135 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 8.9229643060000008, 2024-09-21T14:59:19.135 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068340260999999999, 2024-09-21T14:59:19.135 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:19.135 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:19.135 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:19.135 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:19.135 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:19.135 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45852 2024-09-21T14:59:19.135 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.135 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:19.135 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.136 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:19.136 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159740+0000", 2024-09-21T14:59:19.136 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:19.136 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.136 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.136 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:19.136 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159740+0000", 2024-09-21T14:59:19.136 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:19.136 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.136 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.136 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:19.136 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159739+0000", 2024-09-21T14:59:19.136 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:19.136 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.137 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.137 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:19.137 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159757+0000", 2024-09-21T14:59:19.137 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7314999999999999e-05 2024-09-21T14:59:19.137 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.137 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.137 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:19.137 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159769+0000", 2024-09-21T14:59:19.137 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2376e-05 2024-09-21T14:59:19.137 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.137 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.137 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:19.137 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159775+0000", 2024-09-21T14:59:19.137 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2179999999999996e-06 2024-09-21T14:59:19.137 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.138 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.138 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:19.138 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228071+0000", 2024-09-21T14:59:19.138 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068295344999999993 2024-09-21T14:59:19.138 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.138 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.138 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:19.138 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228080+0000", 2024-09-21T14:59:19.138 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3959999999999995e-06 2024-09-21T14:59:19.138 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:19.138 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:19.138 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:19.138 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.139 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.139 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45853 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:19.139 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159781+0000", 2024-09-21T14:59:19.139 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 8.9229230420000007, 2024-09-21T14:59:19.139 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068337093000000002, 2024-09-21T14:59:19.139 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:19.139 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:19.139 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:19.139 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:19.139 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:19.139 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45853 2024-09-21T14:59:19.139 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.139 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:19.140 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.140 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:19.140 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159781+0000", 2024-09-21T14:59:19.140 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:19.140 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.140 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.140 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:19.140 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159781+0000", 2024-09-21T14:59:19.140 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:19.140 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.140 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.140 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:19.140 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159781+0000", 2024-09-21T14:59:19.140 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:19.141 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.141 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.141 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:19.141 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159784+0000", 2024-09-21T14:59:19.141 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4759999999999998e-06 2024-09-21T14:59:19.141 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.141 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.141 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:19.141 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159792+0000", 2024-09-21T14:59:19.141 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0569999999999993e-06 2024-09-21T14:59:19.141 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.141 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.141 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:19.142 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159797+0000", 2024-09-21T14:59:19.142 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9640000000000002e-06 2024-09-21T14:59:19.142 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.142 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.142 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:19.142 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228111+0000", 2024-09-21T14:59:19.142 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068313737999999999 2024-09-21T14:59:19.142 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.142 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.142 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:19.142 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228118+0000", 2024-09-21T14:59:19.142 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.43e-06 2024-09-21T14:59:19.143 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:19.143 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:19.143 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:19.143 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.143 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.143 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45854 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:19.143 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159802+0000", 2024-09-21T14:59:19.143 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 8.9229023919999992, 2024-09-21T14:59:19.143 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068358938999999994, 2024-09-21T14:59:19.143 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:19.143 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:19.143 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:19.143 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:19.144 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:19.144 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45854 2024-09-21T14:59:19.144 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.144 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:19.144 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.144 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:19.144 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159802+0000", 2024-09-21T14:59:19.144 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:19.144 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.144 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.144 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:19.144 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159802+0000", 2024-09-21T14:59:19.144 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:19.144 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.144 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.145 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:19.145 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159801+0000", 2024-09-21T14:59:19.145 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:19.145 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.145 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.145 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:19.145 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159804+0000", 2024-09-21T14:59:19.145 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0970000000000002e-06 2024-09-21T14:59:19.145 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.145 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.145 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:19.145 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159823+0000", 2024-09-21T14:59:19.145 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8895000000000001e-05 2024-09-21T14:59:19.145 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.145 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.145 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:19.146 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159837+0000", 2024-09-21T14:59:19.146 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3247999999999999e-05 2024-09-21T14:59:19.146 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.146 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.146 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:19.146 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228151+0000", 2024-09-21T14:59:19.146 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068314642999999994 2024-09-21T14:59:19.146 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.146 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.146 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:19.146 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228161+0000", 2024-09-21T14:59:19.146 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4250000000000004e-06 2024-09-21T14:59:19.146 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:19.146 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:19.146 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:19.147 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.147 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.147 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45983 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:19.147 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159848+0000", 2024-09-21T14:59:19.147 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 8.9228561329999998, 2024-09-21T14:59:19.147 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068333801, 2024-09-21T14:59:19.147 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:19.147 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:19.147 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:19.147 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:19.147 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:19.147 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45983 2024-09-21T14:59:19.147 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.147 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:19.147 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.148 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:19.148 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159848+0000", 2024-09-21T14:59:19.148 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:19.148 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.148 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.148 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:19.148 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159848+0000", 2024-09-21T14:59:19.148 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:19.148 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.148 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.148 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:19.148 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159847+0000", 2024-09-21T14:59:19.148 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:19.148 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.148 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.148 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:19.149 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159856+0000", 2024-09-21T14:59:19.149 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4349999999999997e-06 2024-09-21T14:59:19.149 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.149 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.149 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:19.149 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159866+0000", 2024-09-21T14:59:19.149 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0076e-05 2024-09-21T14:59:19.149 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.149 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.149 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:19.149 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159872+0000", 2024-09-21T14:59:19.149 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8309999999999997e-06 2024-09-21T14:59:19.149 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.149 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.149 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:19.150 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228175+0000", 2024-09-21T14:59:19.150 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068302680000000005 2024-09-21T14:59:19.150 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.150 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.150 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:19.150 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228182+0000", 2024-09-21T14:59:19.150 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3150000000000003e-06 2024-09-21T14:59:19.150 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:19.150 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:19.150 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:19.150 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.150 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.150 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45984 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:19.150 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159877+0000", 2024-09-21T14:59:19.150 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 8.9228267129999992, 2024-09-21T14:59:19.150 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068328448, 2024-09-21T14:59:19.151 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:19.151 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:19.151 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:19.151 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:19.151 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:19.151 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45984 2024-09-21T14:59:19.151 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.151 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:19.151 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.151 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:19.151 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159877+0000", 2024-09-21T14:59:19.151 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:19.151 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.151 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.151 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:19.152 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159877+0000", 2024-09-21T14:59:19.152 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:19.152 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.152 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.152 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:19.152 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159877+0000", 2024-09-21T14:59:19.152 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:19.152 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.152 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.152 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:19.152 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159880+0000", 2024-09-21T14:59:19.152 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7610000000000001e-06 2024-09-21T14:59:19.152 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.152 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.152 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:19.153 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159888+0000", 2024-09-21T14:59:19.153 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0330000000000003e-06 2024-09-21T14:59:19.153 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.153 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.153 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:19.153 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159893+0000", 2024-09-21T14:59:19.153 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3550000000000004e-06 2024-09-21T14:59:19.153 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.153 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.153 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:19.153 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228199+0000", 2024-09-21T14:59:19.153 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068306438999999997 2024-09-21T14:59:19.153 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.153 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.153 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:19.154 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228206+0000", 2024-09-21T14:59:19.154 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5599999999999999e-06 2024-09-21T14:59:19.154 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:19.154 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:19.154 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:19.154 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.154 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.154 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45985 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:19.154 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159897+0000", 2024-09-21T14:59:19.154 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 8.9228068900000004, 2024-09-21T14:59:19.154 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068345664, 2024-09-21T14:59:19.154 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:19.154 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:19.154 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:19.154 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:19.155 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:19.155 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45985 2024-09-21T14:59:19.155 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.155 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:19.155 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.155 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:19.155 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159897+0000", 2024-09-21T14:59:19.155 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:19.155 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.155 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.155 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:19.155 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159897+0000", 2024-09-21T14:59:19.155 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:19.155 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.155 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.155 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:19.156 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159897+0000", 2024-09-21T14:59:19.156 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:19.156 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.156 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.156 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:19.156 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159900+0000", 2024-09-21T14:59:19.156 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.954e-06 2024-09-21T14:59:19.156 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.156 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.156 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:19.156 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159913+0000", 2024-09-21T14:59:19.156 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3106e-05 2024-09-21T14:59:19.156 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.156 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.156 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:19.156 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159918+0000", 2024-09-21T14:59:19.157 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5199999999999997e-06 2024-09-21T14:59:19.157 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.157 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.157 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:19.157 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228236+0000", 2024-09-21T14:59:19.157 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068317040999999995 2024-09-21T14:59:19.157 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.157 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.157 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:19.157 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228243+0000", 2024-09-21T14:59:19.157 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3880000000000002e-06 2024-09-21T14:59:19.157 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:19.157 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:19.157 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:19.157 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.157 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.158 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45986 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:19.158 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159924+0000", 2024-09-21T14:59:19.158 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 8.9227805950000008, 2024-09-21T14:59:19.158 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068364362999999997, 2024-09-21T14:59:19.158 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:19.158 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:19.158 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:19.158 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:19.158 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:19.158 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45986 2024-09-21T14:59:19.158 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.158 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:19.158 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.158 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:19.159 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159924+0000", 2024-09-21T14:59:19.159 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:19.159 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.159 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.159 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:19.159 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159924+0000", 2024-09-21T14:59:19.159 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:19.159 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.159 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.159 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:19.159 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159923+0000", 2024-09-21T14:59:19.159 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:19.159 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.159 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.159 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:19.159 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159926+0000", 2024-09-21T14:59:19.160 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0510000000000001e-06 2024-09-21T14:59:19.160 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.160 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.160 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:19.160 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159932+0000", 2024-09-21T14:59:19.160 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4160000000000001e-06 2024-09-21T14:59:19.160 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.160 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.160 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:19.160 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159938+0000", 2024-09-21T14:59:19.160 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.383e-06 2024-09-21T14:59:19.160 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.160 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.160 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:19.161 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228275+0000", 2024-09-21T14:59:19.161 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068337007000000005 2024-09-21T14:59:19.161 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.161 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.161 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:19.161 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228288+0000", 2024-09-21T14:59:19.161 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3052e-05 2024-09-21T14:59:19.161 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:19.161 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:19.161 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:19.161 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.161 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.161 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45987 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:19.161 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159944+0000", 2024-09-21T14:59:19.161 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 8.9227603710000007, 2024-09-21T14:59:19.162 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068387232000000006, 2024-09-21T14:59:19.162 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:19.162 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:19.162 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:19.162 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:19.162 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:19.162 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45987 2024-09-21T14:59:19.162 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.162 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:19.162 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.162 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:19.162 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159944+0000", 2024-09-21T14:59:19.162 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:19.162 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.162 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.163 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:19.163 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159944+0000", 2024-09-21T14:59:19.163 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:19.163 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.163 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.163 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:19.163 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159943+0000", 2024-09-21T14:59:19.163 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:19.163 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.163 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.163 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:19.163 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159946+0000", 2024-09-21T14:59:19.163 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6309999999999999e-06 2024-09-21T14:59:19.163 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.163 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.164 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:19.164 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159952+0000", 2024-09-21T14:59:19.164 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6540000000000004e-06 2024-09-21T14:59:19.164 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.164 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.164 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:19.164 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159957+0000", 2024-09-21T14:59:19.164 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9280000000000001e-06 2024-09-21T14:59:19.164 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.164 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.164 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:19.164 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228322+0000", 2024-09-21T14:59:19.164 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068365067000000002 2024-09-21T14:59:19.164 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.164 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.165 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:19.165 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228331+0000", 2024-09-21T14:59:19.165 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3270000000000007e-06 2024-09-21T14:59:19.165 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:19.165 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:19.165 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:19.165 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:19.165 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:19.165 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:59:19.165 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T14:59:19.167 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T14:59:19.167 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:59:19.163+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:59:19.266 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:59:19.265+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:59:19.276 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:59:19.277 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:59:19.277 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T14:59:19.277 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:59:19.286 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T14:59:19.367 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:59:19.365+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:59:19.389 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:59:19.389 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:59:19.389 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T14:59:19.389 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T14:59:19.389 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:59:19.399 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T14:59:19.465 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T14:59:19.468 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:59:19.469+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:59:19.507 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:59:19.507 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T14:59:19.507 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T14:59:19.507 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T14:59:19.507 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.507 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:10651 3.12 3:4e9d23af:::rbd_data.1136f5ed5dd6.0000000000000020:head [write 0~81920 [fadvise_sequential+fadvise_nocache],write 90112~8192 [fadvise_sequential+fadvise_nocache],write 122880~122880 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:19.507 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.292476+0000", 2024-09-21T14:59:19.507 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 40.211791066000004, 2024-09-21T14:59:19.507 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012765669, 2024-09-21T14:59:19.507 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:19.508 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:19.508 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:19.508 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:19.508 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:19.508 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 10651 2024-09-21T14:59:19.508 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.508 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:19.508 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.508 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:19.508 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292476+0000", 2024-09-21T14:59:19.508 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:19.508 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.508 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.508 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:19.508 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292476+0000", 2024-09-21T14:59:19.508 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:19.509 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.509 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.509 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:19.509 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292475+0000", 2024-09-21T14:59:19.509 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:19.509 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.509 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.509 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:19.509 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292696+0000", 2024-09-21T14:59:19.509 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000220201 2024-09-21T14:59:19.509 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.509 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.509 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:19.509 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292711+0000", 2024-09-21T14:59:19.509 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5803000000000001e-05 2024-09-21T14:59:19.510 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.510 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.510 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:19.510 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292721+0000", 2024-09-21T14:59:19.510 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6160000000000008e-06 2024-09-21T14:59:19.510 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.510 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.510 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:19.510 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292757+0000", 2024-09-21T14:59:19.510 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6211999999999998e-05 2024-09-21T14:59:19.510 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.510 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.510 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:19.510 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292878+0000", 2024-09-21T14:59:19.510 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000120306 2024-09-21T14:59:19.511 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.511 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.511 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 3", 2024-09-21T14:59:19.511 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.293019+0000", 2024-09-21T14:59:19.511 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014186400000000001 2024-09-21T14:59:19.511 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.511 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.511 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:59:19.511 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.294898+0000", 2024-09-21T14:59:19.511 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001878434 2024-09-21T14:59:19.511 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.511 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.511 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:19.511 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.305162+0000", 2024-09-21T14:59:19.511 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.01026373 2024-09-21T14:59:19.511 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.512 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.512 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:59:19.512 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.305209+0000", 2024-09-21T14:59:19.512 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7615999999999999e-05 2024-09-21T14:59:19.512 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.512 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.512 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:19.512 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.305242+0000", 2024-09-21T14:59:19.512 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2979999999999999e-05 2024-09-21T14:59:19.512 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:19.512 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:19.512 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:19.512 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.512 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.513 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:11285 3.12 3:4edf73d2:::rbd_data.1136f5ed5dd6.0000000000000039:head [write 90112~32768 [fadvise_sequential+fadvise_nocache],write 139264~24576 [fadvise_sequential+fadvise_nocache],write 180224~24576 [fadvise_sequential+fadvise_nocache],write 221184~40960 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:19.513 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.539470+0000", 2024-09-21T14:59:19.513 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 39.964797314000002, 2024-09-21T14:59:19.513 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0083360099999999996, 2024-09-21T14:59:19.513 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:19.513 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:19.513 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:19.513 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:19.513 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:19.513 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 11285 2024-09-21T14:59:19.513 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.513 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:19.513 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.513 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:19.513 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.539470+0000", 2024-09-21T14:59:19.513 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:19.514 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.514 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.514 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:19.514 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.539470+0000", 2024-09-21T14:59:19.514 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:19.514 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.514 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.514 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:19.514 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.539469+0000", 2024-09-21T14:59:19.514 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:19.514 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.514 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.514 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:19.514 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.539652+0000", 2024-09-21T14:59:19.514 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000183084 2024-09-21T14:59:19.514 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.515 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.515 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:19.515 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.539666+0000", 2024-09-21T14:59:19.515 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3947e-05 2024-09-21T14:59:19.515 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.515 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.515 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:19.515 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.539673+0000", 2024-09-21T14:59:19.515 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7309999999999996e-06 2024-09-21T14:59:19.515 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.515 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.515 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:19.515 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.539714+0000", 2024-09-21T14:59:19.515 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1539000000000003e-05 2024-09-21T14:59:19.515 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.515 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.516 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:19.516 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.539832+0000", 2024-09-21T14:59:19.516 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011753500000000001 2024-09-21T14:59:19.516 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.516 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.516 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 3", 2024-09-21T14:59:19.516 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.540005+0000", 2024-09-21T14:59:19.516 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000173031 2024-09-21T14:59:19.516 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.516 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.516 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:59:19.516 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.541606+0000", 2024-09-21T14:59:19.516 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001600899 2024-09-21T14:59:19.516 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.516 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.516 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:19.517 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.547709+0000", 2024-09-21T14:59:19.517 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0061027110000000002 2024-09-21T14:59:19.517 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.517 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.517 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:59:19.517 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.547767+0000", 2024-09-21T14:59:19.517 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8248999999999997e-05 2024-09-21T14:59:19.517 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.517 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.517 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:19.517 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.547806+0000", 2024-09-21T14:59:19.517 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9341000000000002e-05 2024-09-21T14:59:19.517 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:19.517 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:19.517 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:19.517 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.518 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.518 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12189 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 40960~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:19.518 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.618667+0000", 2024-09-21T14:59:19.518 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 39.885600863999997, 2024-09-21T14:59:19.518 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0083014549999999993, 2024-09-21T14:59:19.518 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:19.518 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:19.518 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:19.518 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:19.518 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:19.518 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12189 2024-09-21T14:59:19.518 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.518 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:19.518 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.518 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:19.518 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618667+0000", 2024-09-21T14:59:19.519 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:19.519 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.519 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.519 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:19.519 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618667+0000", 2024-09-21T14:59:19.519 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:19.519 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.519 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.519 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:19.519 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618666+0000", 2024-09-21T14:59:19.519 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:19.519 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.519 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.519 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:19.519 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618669+0000", 2024-09-21T14:59:19.520 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.554e-06 2024-09-21T14:59:19.520 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.520 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.520 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:19.520 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618675+0000", 2024-09-21T14:59:19.520 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2570000000000004e-06 2024-09-21T14:59:19.520 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.520 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.520 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:19.520 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618680+0000", 2024-09-21T14:59:19.520 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6539999999999998e-06 2024-09-21T14:59:19.520 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.520 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.520 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:19.520 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.626683+0000", 2024-09-21T14:59:19.520 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0080035880000000007 2024-09-21T14:59:19.521 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.521 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.521 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:19.521 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.626764+0000", 2024-09-21T14:59:19.521 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0672000000000003e-05 2024-09-21T14:59:19.521 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.521 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.521 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:19.521 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.626968+0000", 2024-09-21T14:59:19.521 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00020407499999999999 2024-09-21T14:59:19.521 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:19.521 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:19.521 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:19.521 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.521 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.521 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12193 3.4 3:25437d51:::rbd_data.11363a21d2be.0000000000000025:10 [read 425984~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:19.522 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.618755+0000", 2024-09-21T14:59:19.522 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 39.885512783000003, 2024-09-21T14:59:19.522 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086461379999999994, 2024-09-21T14:59:19.522 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:19.522 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:19.522 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:19.522 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:19.522 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:19.522 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12193 2024-09-21T14:59:19.522 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.522 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:19.522 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.522 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:19.522 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618755+0000", 2024-09-21T14:59:19.522 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:19.523 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.523 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.523 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:19.523 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618755+0000", 2024-09-21T14:59:19.523 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:19.523 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.523 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.523 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:19.523 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618754+0000", 2024-09-21T14:59:19.523 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:19.523 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.523 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.523 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:19.523 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618759+0000", 2024-09-21T14:59:19.523 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4020000000000004e-06 2024-09-21T14:59:19.523 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.524 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.524 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:19.524 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618767+0000", 2024-09-21T14:59:19.524 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7490000000000004e-06 2024-09-21T14:59:19.524 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.524 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.524 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:19.524 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618773+0000", 2024-09-21T14:59:19.524 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0939999999999996e-06 2024-09-21T14:59:19.524 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.524 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.524 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:19.524 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.627115+0000", 2024-09-21T14:59:19.524 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0083416840000000003 2024-09-21T14:59:19.524 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.524 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.525 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:19.525 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.627214+0000", 2024-09-21T14:59:19.525 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9042000000000005e-05 2024-09-21T14:59:19.525 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.525 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.525 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:19.525 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.627401+0000", 2024-09-21T14:59:19.525 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018677200000000001 2024-09-21T14:59:19.525 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:19.525 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:19.525 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:19.525 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.525 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.525 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12212 3.4 3:21b43b45:::rbd_data.11363a21d2be.000000000000002b:10 [read 237568~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:19.525 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.619922+0000", 2024-09-21T14:59:19.525 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 39.884345799000002, 2024-09-21T14:59:19.526 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.008324372, 2024-09-21T14:59:19.526 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:19.526 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:19.526 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:19.526 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:19.526 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:19.526 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12212 2024-09-21T14:59:19.526 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.526 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:19.526 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.526 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:19.526 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619922+0000", 2024-09-21T14:59:19.526 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:19.526 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.526 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.527 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:19.527 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619922+0000", 2024-09-21T14:59:19.527 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:19.527 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.527 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.527 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:19.527 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619921+0000", 2024-09-21T14:59:19.527 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:19.527 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.527 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.527 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:19.527 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619924+0000", 2024-09-21T14:59:19.527 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.999e-06 2024-09-21T14:59:19.527 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.527 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.528 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:19.528 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619934+0000", 2024-09-21T14:59:19.528 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.8120000000000002e-06 2024-09-21T14:59:19.528 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.528 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.528 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:19.528 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619939+0000", 2024-09-21T14:59:19.528 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7729999999999999e-06 2024-09-21T14:59:19.528 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.528 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.528 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:19.528 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628005+0000", 2024-09-21T14:59:19.528 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0080663330000000002 2024-09-21T14:59:19.528 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.529 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.529 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:19.529 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628073+0000", 2024-09-21T14:59:19.529 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7479999999999998e-05 2024-09-21T14:59:19.529 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.529 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.529 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:19.529 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628246+0000", 2024-09-21T14:59:19.529 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017337400000000001 2024-09-21T14:59:19.529 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:19.529 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:19.529 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:19.529 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.529 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.529 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12213 3.4 3:21b43b45:::rbd_data.11363a21d2be.000000000000002b:10 [read 270336~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:19.529 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.619945+0000", 2024-09-21T14:59:19.530 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 39.884322517000001, 2024-09-21T14:59:19.530 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086195049999999995, 2024-09-21T14:59:19.530 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:19.530 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:19.530 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:19.530 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:19.530 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:19.530 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12213 2024-09-21T14:59:19.530 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.530 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:19.530 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.530 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:19.530 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619945+0000", 2024-09-21T14:59:19.530 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:19.530 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.531 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.531 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:19.531 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619945+0000", 2024-09-21T14:59:19.531 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:19.531 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.531 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.531 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:19.531 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619945+0000", 2024-09-21T14:59:19.531 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:19.531 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.531 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.531 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:19.531 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619947+0000", 2024-09-21T14:59:19.531 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5780000000000001e-06 2024-09-21T14:59:19.531 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.532 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.532 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:19.532 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619957+0000", 2024-09-21T14:59:19.532 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3500000000000003e-06 2024-09-21T14:59:19.532 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.532 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.532 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:19.532 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619964+0000", 2024-09-21T14:59:19.532 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.199e-06 2024-09-21T14:59:19.532 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.532 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.532 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:19.532 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628285+0000", 2024-09-21T14:59:19.532 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0083210939999999994 2024-09-21T14:59:19.533 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.533 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.533 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:19.533 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628361+0000", 2024-09-21T14:59:19.533 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5873999999999999e-05 2024-09-21T14:59:19.533 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.533 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.533 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:19.533 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628565+0000", 2024-09-21T14:59:19.533 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000203799 2024-09-21T14:59:19.533 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:19.533 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:19.533 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:19.533 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.533 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.534 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12214 3.4 3:21b43b45:::rbd_data.11363a21d2be.000000000000002b:10 [read 376832~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:19.534 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.619969+0000", 2024-09-21T14:59:19.534 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 39.884298516000001, 2024-09-21T14:59:19.534 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0090292089999999998, 2024-09-21T14:59:19.534 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:19.534 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:19.534 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:19.534 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:19.534 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:19.534 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12214 2024-09-21T14:59:19.534 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.534 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:19.534 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.534 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:19.534 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619969+0000", 2024-09-21T14:59:19.535 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:19.535 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.535 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.535 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:19.535 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619969+0000", 2024-09-21T14:59:19.535 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:19.535 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.535 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.535 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:19.535 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619969+0000", 2024-09-21T14:59:19.535 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:19.535 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.535 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.535 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:19.535 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619971+0000", 2024-09-21T14:59:19.535 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1830000000000001e-06 2024-09-21T14:59:19.536 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.536 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.536 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:19.536 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619976+0000", 2024-09-21T14:59:19.536 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7129999999999996e-06 2024-09-21T14:59:19.536 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.536 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.536 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:19.536 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619981+0000", 2024-09-21T14:59:19.536 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2460000000000003e-06 2024-09-21T14:59:19.536 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.536 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.536 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:19.536 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628602+0000", 2024-09-21T14:59:19.536 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086209519999999994 2024-09-21T14:59:19.537 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.537 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.537 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:19.537 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628734+0000", 2024-09-21T14:59:19.537 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000132242 2024-09-21T14:59:19.537 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.537 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.537 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:19.537 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628998+0000", 2024-09-21T14:59:19.537 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00026419799999999999 2024-09-21T14:59:19.537 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:19.537 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:19.537 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:19.537 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.537 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.537 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12215 3.4 3:21b43b45:::rbd_data.11363a21d2be.000000000000002b:10 [read 409600~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:19.538 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.619985+0000", 2024-09-21T14:59:19.538 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 39.884282450000001, 2024-09-21T14:59:19.538 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0095457319999999995, 2024-09-21T14:59:19.538 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:19.538 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:19.538 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:19.538 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:19.538 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:19.538 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12215 2024-09-21T14:59:19.538 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.538 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:19.538 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.538 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:19.538 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619985+0000", 2024-09-21T14:59:19.539 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:19.539 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.539 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.539 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:19.539 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619985+0000", 2024-09-21T14:59:19.539 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:19.539 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.539 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.539 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:19.539 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619985+0000", 2024-09-21T14:59:19.539 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:19.539 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.539 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.539 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:19.539 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619987+0000", 2024-09-21T14:59:19.540 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6129999999999998e-06 2024-09-21T14:59:19.540 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.540 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.540 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:19.540 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619993+0000", 2024-09-21T14:59:19.540 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0820000000000004e-06 2024-09-21T14:59:19.540 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.540 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.540 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:19.540 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619998+0000", 2024-09-21T14:59:19.540 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7759999999999997e-06 2024-09-21T14:59:19.540 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.540 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.540 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:19.540 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629022+0000", 2024-09-21T14:59:19.540 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0090237489999999993 2024-09-21T14:59:19.541 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.541 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.541 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:19.541 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629098+0000", 2024-09-21T14:59:19.541 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5735999999999995e-05 2024-09-21T14:59:19.541 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.541 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.541 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:19.541 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629531+0000", 2024-09-21T14:59:19.541 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00043310100000000003 2024-09-21T14:59:19.541 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:19.541 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:19.541 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:19.541 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.541 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.541 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12216 3.4 3:21b43b45:::rbd_data.11363a21d2be.000000000000002b:10 [read 516096~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:19.542 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.620003+0000", 2024-09-21T14:59:19.542 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 39.884264453999997, 2024-09-21T14:59:19.542 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0098850169999999994, 2024-09-21T14:59:19.542 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:19.542 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:19.542 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:19.542 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:19.542 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:19.542 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12216 2024-09-21T14:59:19.542 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.542 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:19.542 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.542 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:19.542 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620003+0000", 2024-09-21T14:59:19.542 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:19.543 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.543 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.543 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:19.543 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620003+0000", 2024-09-21T14:59:19.543 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:19.543 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.543 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.543 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:19.543 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620003+0000", 2024-09-21T14:59:19.543 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:19.543 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.543 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.543 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:19.543 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620006+0000", 2024-09-21T14:59:19.543 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4350000000000001e-06 2024-09-21T14:59:19.544 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.544 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.544 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:19.544 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620013+0000", 2024-09-21T14:59:19.544 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4219999999999997e-06 2024-09-21T14:59:19.544 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.544 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.544 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:19.544 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620017+0000", 2024-09-21T14:59:19.544 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.865e-06 2024-09-21T14:59:19.544 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.544 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.544 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:19.544 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629581+0000", 2024-09-21T14:59:19.545 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0095631989999999997 2024-09-21T14:59:19.545 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.545 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.545 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:19.545 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629691+0000", 2024-09-21T14:59:19.545 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000110586 2024-09-21T14:59:19.545 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.545 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.545 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:19.545 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629888+0000", 2024-09-21T14:59:19.545 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019691199999999999 2024-09-21T14:59:19.545 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:19.545 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:19.545 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:19.545 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.545 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.546 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12222 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 49152~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:19.546 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.620027+0000", 2024-09-21T14:59:19.546 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 39.884240108, 2024-09-21T14:59:19.546 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010228131999999999, 2024-09-21T14:59:19.546 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:19.546 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:19.546 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:19.546 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:19.546 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:19.546 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12222 2024-09-21T14:59:19.546 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.546 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:19.546 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.546 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:19.546 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620027+0000", 2024-09-21T14:59:19.547 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:19.547 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.547 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.547 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:19.547 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620027+0000", 2024-09-21T14:59:19.547 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:19.547 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.547 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.547 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:19.547 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620027+0000", 2024-09-21T14:59:19.547 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:19.547 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.547 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.547 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:19.547 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620030+0000", 2024-09-21T14:59:19.548 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0400000000000001e-06 2024-09-21T14:59:19.548 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.548 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.548 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:19.548 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620037+0000", 2024-09-21T14:59:19.548 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8220000000000001e-06 2024-09-21T14:59:19.548 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.548 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.548 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:19.548 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620042+0000", 2024-09-21T14:59:19.548 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.843e-06 2024-09-21T14:59:19.548 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.548 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.548 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:19.548 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629922+0000", 2024-09-21T14:59:19.548 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0098805100000000003 2024-09-21T14:59:19.549 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.549 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.549 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:19.549 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.630020+0000", 2024-09-21T14:59:19.549 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7596999999999994e-05 2024-09-21T14:59:19.549 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.549 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.549 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:19.549 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.630256+0000", 2024-09-21T14:59:19.549 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00023573299999999999 2024-09-21T14:59:19.549 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:19.549 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:19.549 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:19.549 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.549 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.549 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12253 3.4 3:21b43b45:::rbd_data.11363a21d2be.000000000000002b:10 [read 417792~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:19.550 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.624578+0000", 2024-09-21T14:59:19.550 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 39.879689740000003, 2024-09-21T14:59:19.550 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0085112079999999993, 2024-09-21T14:59:19.550 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:19.550 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:19.550 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:19.550 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:19.550 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:19.550 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12253 2024-09-21T14:59:19.550 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.550 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:19.550 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.550 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:19.550 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624578+0000", 2024-09-21T14:59:19.550 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:19.551 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.551 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.551 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:19.551 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624578+0000", 2024-09-21T14:59:19.551 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:19.551 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.551 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.551 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:19.551 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624578+0000", 2024-09-21T14:59:19.551 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:19.551 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.551 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.551 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:19.551 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624579+0000", 2024-09-21T14:59:19.551 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.688e-06 2024-09-21T14:59:19.551 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.552 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.552 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:19.552 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624582+0000", 2024-09-21T14:59:19.552 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2260000000000001e-06 2024-09-21T14:59:19.552 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.552 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.552 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:19.552 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624585+0000", 2024-09-21T14:59:19.552 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4820000000000001e-06 2024-09-21T14:59:19.552 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.552 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.552 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:19.552 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.632580+0000", 2024-09-21T14:59:19.552 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0079948959999999996 2024-09-21T14:59:19.552 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.552 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.553 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:19.553 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.632683+0000", 2024-09-21T14:59:19.553 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010308 2024-09-21T14:59:19.553 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.553 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.553 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:19.553 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.633089+0000", 2024-09-21T14:59:19.553 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00040612599999999998 2024-09-21T14:59:19.553 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:19.553 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:19.553 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:19.553 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.553 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.553 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12260 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 57344~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:19.553 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.624587+0000", 2024-09-21T14:59:19.554 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 39.879680252999997, 2024-09-21T14:59:19.554 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0088282069999999994, 2024-09-21T14:59:19.554 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:19.554 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:19.554 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:19.554 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:19.554 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:19.554 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12260 2024-09-21T14:59:19.554 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.554 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:19.554 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.554 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:19.554 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624587+0000", 2024-09-21T14:59:19.554 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:19.554 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.554 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.555 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:19.555 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624587+0000", 2024-09-21T14:59:19.555 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:19.555 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.555 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.555 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:19.555 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624587+0000", 2024-09-21T14:59:19.555 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-21T14:59:19.555 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.555 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.555 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:19.555 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624589+0000", 2024-09-21T14:59:19.555 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4339999999999999e-06 2024-09-21T14:59:19.555 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.555 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.556 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:19.556 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624593+0000", 2024-09-21T14:59:19.556 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5859999999999998e-06 2024-09-21T14:59:19.556 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.556 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.556 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:19.556 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624596+0000", 2024-09-21T14:59:19.556 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5830000000000001e-06 2024-09-21T14:59:19.556 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.556 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.556 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:19.556 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.633132+0000", 2024-09-21T14:59:19.556 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.008536666 2024-09-21T14:59:19.556 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.556 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.556 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:19.557 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.633216+0000", 2024-09-21T14:59:19.557 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3387999999999995e-05 2024-09-21T14:59:19.557 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.557 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.557 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:19.557 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.633415+0000", 2024-09-21T14:59:19.557 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019976799999999999 2024-09-21T14:59:19.557 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:19.557 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:19.557 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:19.557 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.557 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.557 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12516 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 245760~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:19.557 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645194+0000", 2024-09-21T14:59:19.557 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 39.859073715999997, 2024-09-21T14:59:19.558 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0084284990000000008, 2024-09-21T14:59:19.558 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:19.558 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:19.558 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:19.558 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:19.558 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:19.558 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12516 2024-09-21T14:59:19.558 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.558 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:19.558 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.558 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:19.558 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645194+0000", 2024-09-21T14:59:19.558 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:19.558 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.558 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.558 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:19.559 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645194+0000", 2024-09-21T14:59:19.559 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:19.559 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.559 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.559 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:19.559 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645193+0000", 2024-09-21T14:59:19.559 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:19.559 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.559 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.559 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:19.559 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645196+0000", 2024-09-21T14:59:19.559 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1030000000000002e-06 2024-09-21T14:59:19.559 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.559 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.559 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:19.560 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645205+0000", 2024-09-21T14:59:19.560 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6200000000000005e-06 2024-09-21T14:59:19.560 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.560 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.560 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:19.560 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645210+0000", 2024-09-21T14:59:19.560 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6280000000000004e-06 2024-09-21T14:59:19.560 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.560 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.560 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:19.560 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.653305+0000", 2024-09-21T14:59:19.560 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0080957879999999996 2024-09-21T14:59:19.560 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.560 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.560 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:19.560 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.653389+0000", 2024-09-21T14:59:19.561 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3242e-05 2024-09-21T14:59:19.561 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.561 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.561 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:19.561 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.653622+0000", 2024-09-21T14:59:19.561 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00023361599999999999 2024-09-21T14:59:19.561 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:19.561 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:19.561 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:19.561 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.561 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.561 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12517 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 278528~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:19.561 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645214+0000", 2024-09-21T14:59:19.561 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 39.859053603, 2024-09-21T14:59:19.561 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086832899999999998, 2024-09-21T14:59:19.561 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:19.562 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:19.562 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:19.562 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:19.562 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:19.562 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12517 2024-09-21T14:59:19.562 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.562 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:19.562 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.562 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:19.562 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645214+0000", 2024-09-21T14:59:19.562 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:19.562 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.562 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.562 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:19.562 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645214+0000", 2024-09-21T14:59:19.563 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:19.563 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.563 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.563 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:19.563 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645214+0000", 2024-09-21T14:59:19.563 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:19.563 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.563 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.563 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:19.563 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645216+0000", 2024-09-21T14:59:19.563 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5579999999999999e-06 2024-09-21T14:59:19.563 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.563 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.563 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:19.563 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645224+0000", 2024-09-21T14:59:19.564 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2439999999999993e-06 2024-09-21T14:59:19.564 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.564 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.564 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:19.564 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645231+0000", 2024-09-21T14:59:19.564 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1859999999999997e-06 2024-09-21T14:59:19.564 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.564 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.564 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:19.564 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.653651+0000", 2024-09-21T14:59:19.564 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0084204220000000003 2024-09-21T14:59:19.564 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.564 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.564 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:19.564 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.653726+0000", 2024-09-21T14:59:19.564 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5356000000000002e-05 2024-09-21T14:59:19.565 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.565 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.565 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:19.565 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.653897+0000", 2024-09-21T14:59:19.565 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000170848 2024-09-21T14:59:19.565 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:19.565 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:19.565 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:19.565 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.565 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.565 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12518 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 385024~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:19.565 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645235+0000", 2024-09-21T14:59:19.565 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 39.859032376999998, 2024-09-21T14:59:19.565 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0089841020000000008, 2024-09-21T14:59:19.566 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:19.566 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:19.566 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:19.566 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:19.566 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:19.566 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12518 2024-09-21T14:59:19.566 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.566 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:19.566 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.566 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:19.566 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645235+0000", 2024-09-21T14:59:19.566 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:19.566 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.566 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.566 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:19.566 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645235+0000", 2024-09-21T14:59:19.567 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:19.567 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.567 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.567 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:19.567 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645235+0000", 2024-09-21T14:59:19.567 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:19.567 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.567 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.567 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:19.567 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645238+0000", 2024-09-21T14:59:19.567 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.869e-06 2024-09-21T14:59:19.567 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.567 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.567 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:19.567 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645244+0000", 2024-09-21T14:59:19.568 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8930000000000002e-06 2024-09-21T14:59:19.568 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.568 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.568 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:19.569 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645249+0000", 2024-09-21T14:59:19.569 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2859999999999999e-06 2024-09-21T14:59:19.569 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.569 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.569 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:19.569 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.653930+0000", 2024-09-21T14:59:19.569 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086815320000000005 2024-09-21T14:59:19.569 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.570 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.570 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:19.570 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654028+0000", 2024-09-21T14:59:19.570 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7578000000000007e-05 2024-09-21T14:59:19.570 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.570 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.570 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:19.570 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654219+0000", 2024-09-21T14:59:19.570 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019132099999999999 2024-09-21T14:59:19.570 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:19.570 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:19.570 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:19.570 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.570 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.571 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12538 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 147456~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:19.571 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645253+0000", 2024-09-21T14:59:19.571 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 39.859014752999997, 2024-09-21T14:59:19.571 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0092095360000000008, 2024-09-21T14:59:19.571 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:19.571 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:19.571 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:19.571 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:19.571 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:19.571 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12538 2024-09-21T14:59:19.571 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.571 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:19.571 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.571 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:19.571 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645253+0000", 2024-09-21T14:59:19.571 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:19.572 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.572 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.572 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:19.572 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645253+0000", 2024-09-21T14:59:19.572 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:19.572 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.572 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.572 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:19.572 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645252+0000", 2024-09-21T14:59:19.572 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:19.572 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.572 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.572 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:19.572 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645255+0000", 2024-09-21T14:59:19.572 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2290000000000002e-06 2024-09-21T14:59:19.573 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.573 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.573 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:19.573 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645267+0000", 2024-09-21T14:59:19.573 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2631e-05 2024-09-21T14:59:19.573 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.573 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.573 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:19.573 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645278+0000", 2024-09-21T14:59:19.573 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0847e-05 2024-09-21T14:59:19.573 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.573 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.573 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:19.573 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654251+0000", 2024-09-21T14:59:19.573 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0089731340000000007 2024-09-21T14:59:19.573 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.574 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.574 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:19.574 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654296+0000", 2024-09-21T14:59:19.574 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4353e-05 2024-09-21T14:59:19.574 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.574 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.574 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:19.574 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654462+0000", 2024-09-21T14:59:19.574 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016668499999999999 2024-09-21T14:59:19.574 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:19.574 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:19.574 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:19.574 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.574 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.574 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12539 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 253952~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:19.575 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645284+0000", 2024-09-21T14:59:19.575 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 39.858983532000003, 2024-09-21T14:59:19.575 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0094524029999999998, 2024-09-21T14:59:19.575 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:19.575 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:19.575 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:19.575 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:19.575 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:19.575 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12539 2024-09-21T14:59:19.575 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.575 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:19.575 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.575 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:19.575 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645284+0000", 2024-09-21T14:59:19.575 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:19.576 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.576 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.576 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:19.576 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645284+0000", 2024-09-21T14:59:19.576 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:19.576 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.576 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.576 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:19.576 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645284+0000", 2024-09-21T14:59:19.576 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:19.576 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.576 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.576 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:19.576 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645286+0000", 2024-09-21T14:59:19.576 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7159999999999999e-06 2024-09-21T14:59:19.576 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.577 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.577 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:19.577 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645292+0000", 2024-09-21T14:59:19.577 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7400000000000001e-06 2024-09-21T14:59:19.577 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.577 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.577 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:19.577 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645297+0000", 2024-09-21T14:59:19.577 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9019999999999998e-06 2024-09-21T14:59:19.577 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.577 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.577 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:19.577 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654490+0000", 2024-09-21T14:59:19.577 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0091932360000000005 2024-09-21T14:59:19.577 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.578 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.578 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:19.578 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654565+0000", 2024-09-21T14:59:19.578 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4657e-05 2024-09-21T14:59:19.578 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.578 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.578 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:19.578 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654736+0000", 2024-09-21T14:59:19.578 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017156199999999999 2024-09-21T14:59:19.578 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:19.578 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:19.578 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:19.578 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.579 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.579 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12540 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 286720~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:19.579 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645302+0000", 2024-09-21T14:59:19.579 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 39.858965263000002, 2024-09-21T14:59:19.579 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0096603090000000006, 2024-09-21T14:59:19.579 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:19.579 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:19.579 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:19.579 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:19.579 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:19.579 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12540 2024-09-21T14:59:19.579 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.579 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:19.579 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.579 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:19.579 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645302+0000", 2024-09-21T14:59:19.580 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:19.580 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.580 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.580 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:19.580 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645302+0000", 2024-09-21T14:59:19.580 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:19.580 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.580 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.580 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:19.580 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645302+0000", 2024-09-21T14:59:19.580 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:19.580 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.580 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.580 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:19.580 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645305+0000", 2024-09-21T14:59:19.581 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7329999999999999e-06 2024-09-21T14:59:19.581 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.581 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.581 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:19.581 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645316+0000", 2024-09-21T14:59:19.581 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1832e-05 2024-09-21T14:59:19.581 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.581 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.581 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:19.581 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645322+0000", 2024-09-21T14:59:19.581 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.169e-06 2024-09-21T14:59:19.581 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.581 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.581 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:19.581 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654778+0000", 2024-09-21T14:59:19.581 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0094559090000000002 2024-09-21T14:59:19.582 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.582 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.582 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:19.582 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654828+0000", 2024-09-21T14:59:19.582 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0461999999999999e-05 2024-09-21T14:59:19.582 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.582 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.582 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:19.582 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654963+0000", 2024-09-21T14:59:19.582 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000134583 2024-09-21T14:59:19.582 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:19.582 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:19.582 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:19.582 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.582 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.582 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12561 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:19.583 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645742+0000", 2024-09-21T14:59:19.583 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 39.858525997000001, 2024-09-21T14:59:19.583 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0093583810000000007, 2024-09-21T14:59:19.583 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:19.583 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:19.583 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:19.583 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:19.583 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:19.583 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12561 2024-09-21T14:59:19.583 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.583 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:19.583 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.583 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:19.583 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645742+0000", 2024-09-21T14:59:19.583 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:19.584 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.584 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.584 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:19.584 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645742+0000", 2024-09-21T14:59:19.584 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:19.584 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.584 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.584 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:19.584 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645741+0000", 2024-09-21T14:59:19.584 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:19.584 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.584 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.584 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:19.584 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645746+0000", 2024-09-21T14:59:19.584 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8379999999999998e-06 2024-09-21T14:59:19.585 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.585 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.585 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:19.585 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645757+0000", 2024-09-21T14:59:19.585 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0705e-05 2024-09-21T14:59:19.585 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.585 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.585 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:19.585 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645765+0000", 2024-09-21T14:59:19.585 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6729999999999992e-06 2024-09-21T14:59:19.585 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.585 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.585 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:19.585 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654993+0000", 2024-09-21T14:59:19.585 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0092287480000000002 2024-09-21T14:59:19.586 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.586 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.586 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:19.586 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.655050+0000", 2024-09-21T14:59:19.586 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6394999999999998e-05 2024-09-21T14:59:19.586 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.586 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.586 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:19.586 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.655100+0000", 2024-09-21T14:59:19.586 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0049000000000001e-05 2024-09-21T14:59:19.586 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:19.586 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:19.586 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:19.586 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.586 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.587 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:15431 3.12 3:4bc7d206:::rbd_data.1136f5ed5dd6.0000000000000051:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:59:19.587 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:41.673445+0000", 2024-09-21T14:59:19.587 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 37.830822662999999, 2024-09-21T14:59:19.587 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086103610000000004, 2024-09-21T14:59:19.587 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:19.587 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:19.587 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:19.587 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:19.587 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:19.587 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 15431 2024-09-21T14:59:19.587 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.587 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:19.587 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.587 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:19.588 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.673445+0000", 2024-09-21T14:59:19.588 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:19.588 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.588 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.588 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:19.588 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.673445+0000", 2024-09-21T14:59:19.588 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:19.588 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.588 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.588 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:19.588 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.673444+0000", 2024-09-21T14:59:19.588 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:59:19.588 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.589 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.589 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:19.589 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.673451+0000", 2024-09-21T14:59:19.589 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4739999999999999e-06 2024-09-21T14:59:19.589 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.589 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.589 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:19.589 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.673472+0000", 2024-09-21T14:59:19.589 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0687e-05 2024-09-21T14:59:19.589 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.589 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.589 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:19.589 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.673481+0000", 2024-09-21T14:59:19.589 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3349999999999996e-06 2024-09-21T14:59:19.589 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.590 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.590 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:19.590 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.673565+0000", 2024-09-21T14:59:19.590 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4011000000000006e-05 2024-09-21T14:59:19.590 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.590 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.590 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:19.590 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.673650+0000", 2024-09-21T14:59:19.590 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4949000000000003e-05 2024-09-21T14:59:19.590 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.590 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.590 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 3", 2024-09-21T14:59:19.590 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.675014+0000", 2024-09-21T14:59:19.591 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0013636620000000001 2024-09-21T14:59:19.591 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.591 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.591 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:19.591 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.681866+0000", 2024-09-21T14:59:19.591 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0068525640000000002 2024-09-21T14:59:19.591 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.591 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.591 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:59:19.591 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.681885+0000", 2024-09-21T14:59:19.591 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8519999999999999e-05 2024-09-21T14:59:19.591 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.591 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.591 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:59:19.591 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.681961+0000", 2024-09-21T14:59:19.591 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6248000000000006e-05 2024-09-21T14:59:19.592 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:19.592 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:19.592 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:19.592 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.682055+0000", 2024-09-21T14:59:19.592 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4184000000000005e-05 2024-09-21T14:59:19.592 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:19.592 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:19.592 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:19.592 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:19.592 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:19.592 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:59:19.593 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T14:59:19.594 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:59:19.591+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:59:19.611 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T14:59:19.632 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T14:59:19.658 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T14:59:19.679 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T14:59:19.695 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:59:19.693+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:59:19.703 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-21T14:59:19.729 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T14:59:19.749 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T14:59:19.755 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:59:19.755 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:59:19.755 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T14:59:19.756 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:59:19.766 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T14:59:19.773 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T14:59:19.775 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:59:19.775 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:59:19.775 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:59:19.775 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:59:19.775 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:59:19.775 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:59:19.775 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:59:19.775 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:59:19.796 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:59:19.793+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:59:19.876 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:19.876 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:19.886 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T14:59:19.897 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:59:19.897+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:59:19.995 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:19.995 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:19.998 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:59:19.997+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:59:20.006 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T14:59:20.099 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:59:20.097+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:59:20.123 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:20.123 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:20.134 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T14:59:20.144 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 5, 6, 7] out_osds: [4] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T14:59:20.144 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T14:59:20.144 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-21T14:59:20.144 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-21T14:59:20.179 INFO:teuthology.orchestra.run.smithi032.stderr:nodeep-scrub is set 2024-09-21T14:59:20.199 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:59:20.197+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:59:20.271 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:20.271 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:20.283 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T14:59:20.300 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:59:20.299+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:59:20.340 INFO:teuthology.orchestra.run.smithi032.stdout:checking clone #12, image image_client.0-clone12 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent13 2024-09-21T14:59:20.401 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:59:20.401+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:59:20.423 INFO:teuthology.orchestra.run.smithi032.stdout:1448 trunc from 0xb005ef5 to 0xde3b503 2024-09-21T14:59:20.438 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:20.438 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:20.450 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T14:59:20.450 INFO:teuthology.orchestra.run.smithi032.stdout:1449 write 0xeb48ce0 thru 0xeb4eee6 (0x6207 bytes) 2024-09-21T14:59:20.457 INFO:teuthology.orchestra.run.smithi032.stdout:1450 punch from 0xbd46953 to 0xbd5479e, (0xde4b bytes) 2024-09-21T14:59:20.457 INFO:teuthology.orchestra.run.smithi032.stdout:1451 read 0x7d7ebfd thru 0x7d88503 (0x9907 bytes) 2024-09-21T14:59:20.457 INFO:teuthology.orchestra.run.smithi032.stdout:1452 write 0x487ba44 thru 0x488100e (0x55cb bytes) 2024-09-21T14:59:20.464 INFO:teuthology.orchestra.run.smithi032.stdout:1453 writesame 0xce97248 thru 0xce9cde9 data_size 0x25(0x5ba2 bytes) 2024-09-21T14:59:20.469 INFO:teuthology.orchestra.run.smithi032.stdout:1454 read 0x46de540 thru 0x46e34bf (0x4f80 bytes) 2024-09-21T14:59:20.469 INFO:teuthology.orchestra.run.smithi032.stdout:1455 writesame 0xd72539c thru 0xd72b4ce data_size 0x1(0x6133 bytes) 2024-09-21T14:59:20.472 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-21T14:59:20.472 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:20.473 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 83, 2024-09-21T14:59:20.473 INFO:teuthology.orchestra.run.smithi032.stdout: "fsid": "caf51e55-9234-4a26-bcd2-d60811962560", 2024-09-21T14:59:20.473 INFO:teuthology.orchestra.run.smithi032.stdout: "created": "2024-09-21T14:57:50.338822+0000", 2024-09-21T14:59:20.473 INFO:teuthology.orchestra.run.smithi032.stdout: "modified": "2024-09-21T14:59:20.161748+0000", 2024-09-21T14:59:20.473 INFO:teuthology.orchestra.run.smithi032.stdout: "last_up_change": "2024-09-21T14:58:01.177141+0000", 2024-09-21T14:59:20.473 INFO:teuthology.orchestra.run.smithi032.stdout: "last_in_change": "2024-09-21T14:59:15.110268+0000", 2024-09-21T14:59:20.473 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-21T14:59:20.473 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_num": 5806080, 2024-09-21T14:59:20.473 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_set": [ 2024-09-21T14:59:20.473 INFO:teuthology.orchestra.run.smithi032.stdout: "nodeep-scrub", 2024-09-21T14:59:20.473 INFO:teuthology.orchestra.run.smithi032.stdout: "noscrub", 2024-09-21T14:59:20.473 INFO:teuthology.orchestra.run.smithi032.stdout: "pglog_hardlimit", 2024-09-21T14:59:20.473 INFO:teuthology.orchestra.run.smithi032.stdout: "purged_snapdirs", 2024-09-21T14:59:20.473 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_deletes", 2024-09-21T14:59:20.474 INFO:teuthology.orchestra.run.smithi032.stdout: "sortbitwise" 2024-09-21T14:59:20.474 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-21T14:59:20.474 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_version": 5, 2024-09-21T14:59:20.474 INFO:teuthology.orchestra.run.smithi032.stdout: "full_ratio": 0.94999998807907104, 2024-09-21T14:59:20.474 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-21T14:59:20.474 INFO:teuthology.orchestra.run.smithi032.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-21T14:59:20.474 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_snapshot": "", 2024-09-21T14:59:20.474 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_max": 3, 2024-09-21T14:59:20.474 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd": 8, 2024-09-21T14:59:20.474 INFO:teuthology.orchestra.run.smithi032.stdout: "require_min_compat_client": "luminous", 2024-09-21T14:59:20.474 INFO:teuthology.orchestra.run.smithi032.stdout: "min_compat_client": "luminous", 2024-09-21T14:59:20.474 INFO:teuthology.orchestra.run.smithi032.stdout: "require_osd_release": "reef", 2024-09-21T14:59:20.474 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_crimson": false, 2024-09-21T14:59:20.475 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2024-09-21T14:59:20.475 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.475 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 1, 2024-09-21T14:59:20.475 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": ".mgr", 2024-09-21T14:59:20.475 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-09-21T14:58:01.842810+0000", 2024-09-21T14:59:20.475 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-09-21T14:59:20.475 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2024-09-21T14:59:20.475 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-09-21T14:59:20.475 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-09-21T14:59:20.475 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-09-21T14:59:20.475 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-09-21T14:59:20.475 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-09-21T14:59:20.475 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-09-21T14:59:20.475 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T14:59:20.476 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T14:59:20.476 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-09-21T14:59:20.476 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-09-21T14:59:20.476 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 1, 2024-09-21T14:59:20.476 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 1, 2024-09-21T14:59:20.476 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 1, 2024-09-21T14:59:20.476 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 1, 2024-09-21T14:59:20.476 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 1, 2024-09-21T14:59:20.476 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-09-21T14:59:20.476 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-09-21T14:59:20.476 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-09-21T14:59:20.476 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-09-21T14:59:20.476 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-09-21T14:59:20.476 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-09-21T14:59:20.476 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-09-21T14:59:20.477 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.477 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "18", 2024-09-21T14:59:20.477 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-09-21T14:59:20.477 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T14:59:20.477 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T14:59:20.477 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-09-21T14:59:20.477 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-09-21T14:59:20.477 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-09-21T14:59:20.477 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-09-21T14:59:20.477 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-09-21T14:59:20.477 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-09-21T14:59:20.477 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-09-21T14:59:20.477 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-09-21T14:59:20.477 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-09-21T14:59:20.478 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-09-21T14:59:20.478 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-09-21T14:59:20.478 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-09-21T14:59:20.478 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-09-21T14:59:20.478 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-09-21T14:59:20.478 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-09-21T14:59:20.478 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T14:59:20.478 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T14:59:20.478 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T14:59:20.478 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-09-21T14:59:20.478 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-09-21T14:59:20.478 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-09-21T14:59:20.479 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-09-21T14:59:20.479 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-09-21T14:59:20.479 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.479 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-09-21T14:59:20.479 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-09-21T14:59:20.479 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-09-21T14:59:20.479 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-09-21T14:59:20.479 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-09-21T14:59:20.479 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T14:59:20.479 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-09-21T14:59:20.479 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-09-21T14:59:20.479 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-09-21T14:59:20.479 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-09-21T14:59:20.479 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-09-21T14:59:20.480 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-09-21T14:59:20.480 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_max": 32, 2024-09-21T14:59:20.480 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 1 2024-09-21T14:59:20.480 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.480 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-09-21T14:59:20.480 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": {} 2024-09-21T14:59:20.480 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.480 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-09-21T14:59:20.480 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 8, 2024-09-21T14:59:20.480 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 8, 2024-09-21T14:59:20.480 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 0.25, 2024-09-21T14:59:20.480 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 2, 2024-09-21T14:59:20.480 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 2, 2024-09-21T14:59:20.480 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 0.88126564025878906, 2024-09-21T14:59:20.480 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 0.87999999523162842, 2024-09-21T14:59:20.481 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-21T14:59:20.481 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:20.481 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.481 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.481 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 2, 2024-09-21T14:59:20.481 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "rbd", 2024-09-21T14:59:20.481 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-09-21T14:58:08.284925+0000", 2024-09-21T14:59:20.481 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 8193, 2024-09-21T14:59:20.481 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T14:59:20.481 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-09-21T14:59:20.481 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-09-21T14:59:20.481 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-09-21T14:59:20.481 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-09-21T14:59:20.481 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-09-21T14:59:20.481 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-09-21T14:59:20.482 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T14:59:20.482 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T14:59:20.482 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-09-21T14:59:20.482 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-09-21T14:59:20.482 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2024-09-21T14:59:20.482 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2024-09-21T14:59:20.482 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2024-09-21T14:59:20.482 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2024-09-21T14:59:20.482 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2024-09-21T14:59:20.482 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-09-21T14:59:20.482 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-09-21T14:59:20.482 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-09-21T14:59:20.482 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-09-21T14:59:20.482 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-09-21T14:59:20.482 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-09-21T14:59:20.483 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-09-21T14:59:20.483 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.483 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "22", 2024-09-21T14:59:20.483 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-09-21T14:59:20.483 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T14:59:20.483 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T14:59:20.483 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-09-21T14:59:20.483 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-09-21T14:59:20.483 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 3, 2024-09-21T14:59:20.483 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 22, 2024-09-21T14:59:20.483 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-09-21T14:59:20.483 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-09-21T14:59:20.483 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-09-21T14:59:20.483 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-09-21T14:59:20.484 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-09-21T14:59:20.484 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-09-21T14:59:20.484 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-09-21T14:59:20.484 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-09-21T14:59:20.484 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-09-21T14:59:20.484 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-09-21T14:59:20.484 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-09-21T14:59:20.484 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T14:59:20.484 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T14:59:20.484 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T14:59:20.484 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-09-21T14:59:20.485 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-09-21T14:59:20.485 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-09-21T14:59:20.485 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-09-21T14:59:20.485 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-09-21T14:59:20.485 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.485 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-09-21T14:59:20.485 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-09-21T14:59:20.485 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-09-21T14:59:20.485 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-09-21T14:59:20.485 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-09-21T14:59:20.485 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T14:59:20.485 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-09-21T14:59:20.485 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-09-21T14:59:20.485 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-09-21T14:59:20.486 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-09-21T14:59:20.486 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-09-21T14:59:20.486 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2024-09-21T14:59:20.486 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-09-21T14:59:20.486 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd": {} 2024-09-21T14:59:20.486 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.486 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-09-21T14:59:20.486 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 5, 2024-09-21T14:59:20.486 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 5, 2024-09-21T14:59:20.486 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 0.75, 2024-09-21T14:59:20.486 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 3.75, 2024-09-21T14:59:20.486 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 3.75, 2024-09-21T14:59:20.486 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 0.88126564025878906, 2024-09-21T14:59:20.487 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 0.87999999523162842, 2024-09-21T14:59:20.487 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-21T14:59:20.487 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:20.487 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.487 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.487 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 3, 2024-09-21T14:59:20.487 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "pool_client.0", 2024-09-21T14:59:20.487 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-09-21T14:58:17.510709+0000", 2024-09-21T14:59:20.487 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 8193, 2024-09-21T14:59:20.487 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T14:59:20.487 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-09-21T14:59:20.487 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-09-21T14:59:20.487 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-09-21T14:59:20.487 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-09-21T14:59:20.487 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-09-21T14:59:20.488 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-09-21T14:59:20.488 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T14:59:20.488 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T14:59:20.488 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-09-21T14:59:20.488 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-09-21T14:59:20.488 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 32, 2024-09-21T14:59:20.488 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 32, 2024-09-21T14:59:20.488 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 32, 2024-09-21T14:59:20.488 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 32, 2024-09-21T14:59:20.488 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 32, 2024-09-21T14:59:20.488 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-09-21T14:59:20.488 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-09-21T14:59:20.488 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-09-21T14:59:20.488 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-09-21T14:59:20.488 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-09-21T14:59:20.489 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-09-21T14:59:20.489 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-09-21T14:59:20.489 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.489 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "83", 2024-09-21T14:59:20.489 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-09-21T14:59:20.489 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T14:59:20.489 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T14:59:20.489 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-09-21T14:59:20.489 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-09-21T14:59:20.489 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 45, 2024-09-21T14:59:20.489 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 83, 2024-09-21T14:59:20.489 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-09-21T14:59:20.489 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-09-21T14:59:20.489 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-09-21T14:59:20.490 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-09-21T14:59:20.490 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-09-21T14:59:20.490 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-09-21T14:59:20.490 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-09-21T14:59:20.490 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-09-21T14:59:20.490 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-09-21T14:59:20.490 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-09-21T14:59:20.490 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-09-21T14:59:20.490 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T14:59:20.490 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T14:59:20.490 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T14:59:20.490 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-09-21T14:59:20.490 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-09-21T14:59:20.490 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-09-21T14:59:20.491 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-09-21T14:59:20.491 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-09-21T14:59:20.491 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.491 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-09-21T14:59:20.491 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-09-21T14:59:20.491 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-09-21T14:59:20.491 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-09-21T14:59:20.491 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-09-21T14:59:20.491 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T14:59:20.491 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-09-21T14:59:20.491 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-09-21T14:59:20.491 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-09-21T14:59:20.491 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-09-21T14:59:20.491 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-09-21T14:59:20.491 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2024-09-21T14:59:20.492 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-09-21T14:59:20.492 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd": {} 2024-09-21T14:59:20.492 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.492 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-09-21T14:59:20.492 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 2.4900000095367432, 2024-09-21T14:59:20.492 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 2.4900000095367432, 2024-09-21T14:59:20.492 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 0.87999999523162842, 2024-09-21T14:59:20.492 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 2.190000057220459, 2024-09-21T14:59:20.492 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 2.190000057220459, 2024-09-21T14:59:20.492 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 0.88126564025878906, 2024-09-21T14:59:20.492 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 0.87999999523162842, 2024-09-21T14:59:20.492 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-21T14:59:20.492 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:20.492 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:20.493 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-21T14:59:20.493 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2024-09-21T14:59:20.493 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.493 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-09-21T14:59:20.493 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "a4a73316-3a94-42e9-9a51-deb46a66bb06", 2024-09-21T14:59:20.493 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-21T14:59:20.493 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-21T14:59:20.493 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-21T14:59:20.493 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-21T14:59:20.493 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-21T14:59:20.493 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-21T14:59:20.493 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2024-09-21T14:59:20.493 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 81, 2024-09-21T14:59:20.493 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-21T14:59:20.494 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-21T14:59:20.494 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-21T14:59:20.494 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:59:20.494 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.494 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:59:20.494 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6808", 2024-09-21T14:59:20.494 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4016672243 2024-09-21T14:59:20.494 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.494 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.494 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:59:20.494 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6809", 2024-09-21T14:59:20.494 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4016672243 2024-09-21T14:59:20.494 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:20.495 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:20.495 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.495 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-21T14:59:20.495 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:59:20.495 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.495 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:59:20.495 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6810", 2024-09-21T14:59:20.495 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4016672243 2024-09-21T14:59:20.495 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.495 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.495 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:59:20.495 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6811", 2024-09-21T14:59:20.495 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4016672243 2024-09-21T14:59:20.496 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:20.496 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:20.496 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.496 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-21T14:59:20.496 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:59:20.496 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.496 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:59:20.496 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6814", 2024-09-21T14:59:20.496 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4016672243 2024-09-21T14:59:20.496 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.496 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.496 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:59:20.496 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6815", 2024-09-21T14:59:20.496 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4016672243 2024-09-21T14:59:20.496 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:20.497 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:20.497 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.497 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-21T14:59:20.497 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:59:20.497 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.497 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:59:20.497 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6812", 2024-09-21T14:59:20.497 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4016672243 2024-09-21T14:59:20.497 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.497 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.497 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:59:20.497 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6813", 2024-09-21T14:59:20.497 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4016672243 2024-09-21T14:59:20.497 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:20.497 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:20.498 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.498 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6809/4016672243", 2024-09-21T14:59:20.498 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6811/4016672243", 2024-09-21T14:59:20.498 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6815/4016672243", 2024-09-21T14:59:20.498 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6813/4016672243", 2024-09-21T14:59:20.498 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-21T14:59:20.498 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-21T14:59:20.498 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-21T14:59:20.498 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:20.498 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.498 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.498 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-09-21T14:59:20.498 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "006757a0-94d1-4a9b-aef8-73634cc5828e", 2024-09-21T14:59:20.499 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-21T14:59:20.499 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-21T14:59:20.499 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-21T14:59:20.499 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-21T14:59:20.499 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-21T14:59:20.499 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-21T14:59:20.499 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2024-09-21T14:59:20.499 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 81, 2024-09-21T14:59:20.499 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-21T14:59:20.499 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-21T14:59:20.499 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-21T14:59:20.499 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:59:20.499 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.499 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:59:20.500 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6824", 2024-09-21T14:59:20.500 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 641090577 2024-09-21T14:59:20.500 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.500 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.500 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:59:20.500 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6825", 2024-09-21T14:59:20.500 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 641090577 2024-09-21T14:59:20.500 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:20.500 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:20.500 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.500 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-21T14:59:20.500 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:59:20.500 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.500 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:59:20.500 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6826", 2024-09-21T14:59:20.501 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 641090577 2024-09-21T14:59:20.501 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.501 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.501 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:59:20.501 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6827", 2024-09-21T14:59:20.501 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 641090577 2024-09-21T14:59:20.501 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:20.501 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:20.501 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.501 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-21T14:59:20.501 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:59:20.501 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.501 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:59:20.501 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6830", 2024-09-21T14:59:20.501 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 641090577 2024-09-21T14:59:20.501 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.502 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.502 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:59:20.502 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6831", 2024-09-21T14:59:20.502 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 641090577 2024-09-21T14:59:20.502 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:20.502 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:20.502 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.502 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-21T14:59:20.502 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:59:20.502 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.502 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:59:20.502 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6828", 2024-09-21T14:59:20.502 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 641090577 2024-09-21T14:59:20.502 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.503 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.503 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:59:20.503 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6829", 2024-09-21T14:59:20.503 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 641090577 2024-09-21T14:59:20.503 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:20.503 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:20.503 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.503 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6825/641090577", 2024-09-21T14:59:20.503 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6827/641090577", 2024-09-21T14:59:20.503 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6831/641090577", 2024-09-21T14:59:20.503 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6829/641090577", 2024-09-21T14:59:20.503 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-21T14:59:20.503 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-21T14:59:20.503 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-21T14:59:20.504 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:20.504 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.504 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.504 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-09-21T14:59:20.504 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "0b499a23-820e-4e6c-a558-adbf88b7014f", 2024-09-21T14:59:20.504 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-21T14:59:20.504 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-21T14:59:20.504 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 0.3514556884765625, 2024-09-21T14:59:20.504 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-21T14:59:20.504 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-21T14:59:20.504 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-21T14:59:20.504 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2024-09-21T14:59:20.504 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 51, 2024-09-21T14:59:20.504 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-21T14:59:20.505 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-21T14:59:20.505 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-21T14:59:20.505 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:59:20.505 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.505 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:59:20.505 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6800", 2024-09-21T14:59:20.505 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2609154414 2024-09-21T14:59:20.505 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.505 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.505 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:59:20.505 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6801", 2024-09-21T14:59:20.505 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2609154414 2024-09-21T14:59:20.505 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:20.505 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:20.506 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.506 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-21T14:59:20.506 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:59:20.506 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.506 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:59:20.506 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6802", 2024-09-21T14:59:20.506 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2609154414 2024-09-21T14:59:20.506 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.506 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.506 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:59:20.506 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6803", 2024-09-21T14:59:20.506 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2609154414 2024-09-21T14:59:20.506 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:20.506 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:20.506 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.507 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-21T14:59:20.507 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:59:20.507 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.507 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:59:20.507 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6806", 2024-09-21T14:59:20.507 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2609154414 2024-09-21T14:59:20.507 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.507 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.507 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:59:20.507 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6807", 2024-09-21T14:59:20.507 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2609154414 2024-09-21T14:59:20.507 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:20.507 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:20.507 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.508 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-21T14:59:20.508 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:59:20.508 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.508 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:59:20.508 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6804", 2024-09-21T14:59:20.508 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2609154414 2024-09-21T14:59:20.508 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.508 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.508 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:59:20.508 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6805", 2024-09-21T14:59:20.508 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2609154414 2024-09-21T14:59:20.508 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:20.508 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:20.509 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.509 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6801/2609154414", 2024-09-21T14:59:20.509 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6803/2609154414", 2024-09-21T14:59:20.509 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6807/2609154414", 2024-09-21T14:59:20.509 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6805/2609154414", 2024-09-21T14:59:20.509 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-21T14:59:20.509 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-21T14:59:20.509 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-21T14:59:20.509 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:20.509 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.509 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.509 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-09-21T14:59:20.509 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "5a57b3dd-4a07-45b7-b159-3cac4c7eb262", 2024-09-21T14:59:20.509 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-21T14:59:20.510 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-21T14:59:20.510 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-21T14:59:20.510 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-21T14:59:20.510 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-21T14:59:20.510 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-21T14:59:20.510 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2024-09-21T14:59:20.510 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 81, 2024-09-21T14:59:20.510 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-21T14:59:20.510 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-21T14:59:20.510 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-21T14:59:20.510 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:59:20.510 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.510 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:59:20.510 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6816", 2024-09-21T14:59:20.510 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1462563376 2024-09-21T14:59:20.511 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.511 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.511 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:59:20.511 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6817", 2024-09-21T14:59:20.511 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1462563376 2024-09-21T14:59:20.511 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:20.511 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:20.511 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.511 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-21T14:59:20.511 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:59:20.511 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.511 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:59:20.511 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6818", 2024-09-21T14:59:20.511 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1462563376 2024-09-21T14:59:20.511 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.512 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.512 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:59:20.512 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6819", 2024-09-21T14:59:20.512 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1462563376 2024-09-21T14:59:20.512 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:20.512 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:20.512 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.512 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-21T14:59:20.512 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:59:20.512 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.512 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:59:20.512 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6822", 2024-09-21T14:59:20.512 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1462563376 2024-09-21T14:59:20.512 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.512 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.512 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:59:20.513 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6823", 2024-09-21T14:59:20.513 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1462563376 2024-09-21T14:59:20.513 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:20.513 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:20.513 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.513 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-21T14:59:20.513 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:59:20.513 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.513 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:59:20.513 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6820", 2024-09-21T14:59:20.513 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1462563376 2024-09-21T14:59:20.513 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.513 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.514 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:59:20.514 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6821", 2024-09-21T14:59:20.514 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1462563376 2024-09-21T14:59:20.514 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:20.514 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:20.514 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.514 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6817/1462563376", 2024-09-21T14:59:20.514 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6819/1462563376", 2024-09-21T14:59:20.514 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6823/1462563376", 2024-09-21T14:59:20.514 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6821/1462563376", 2024-09-21T14:59:20.514 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-21T14:59:20.514 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-21T14:59:20.514 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-21T14:59:20.514 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:20.515 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.515 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.515 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-09-21T14:59:20.515 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "39a00807-7517-474e-8842-623dd7093cc5", 2024-09-21T14:59:20.515 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-21T14:59:20.515 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 0, 2024-09-21T14:59:20.515 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 0, 2024-09-21T14:59:20.515 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-21T14:59:20.515 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-21T14:59:20.515 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-21T14:59:20.515 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2024-09-21T14:59:20.515 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 80, 2024-09-21T14:59:20.515 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-21T14:59:20.515 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-21T14:59:20.515 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-21T14:59:20.516 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:59:20.516 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.516 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:59:20.516 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6801", 2024-09-21T14:59:20.516 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 144412042 2024-09-21T14:59:20.516 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.516 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.516 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:59:20.516 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6803", 2024-09-21T14:59:20.516 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 144412042 2024-09-21T14:59:20.516 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:20.516 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:20.516 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.516 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-21T14:59:20.517 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:59:20.517 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.517 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:59:20.517 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6805", 2024-09-21T14:59:20.517 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 144412042 2024-09-21T14:59:20.517 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.517 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.517 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:59:20.517 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6807", 2024-09-21T14:59:20.517 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 144412042 2024-09-21T14:59:20.517 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:20.517 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:20.517 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.517 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-21T14:59:20.517 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:59:20.518 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.518 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:59:20.518 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6812", 2024-09-21T14:59:20.518 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 144412042 2024-09-21T14:59:20.518 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.518 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.518 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:59:20.518 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6814", 2024-09-21T14:59:20.518 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 144412042 2024-09-21T14:59:20.518 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:20.518 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:20.518 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.518 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-21T14:59:20.519 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:59:20.519 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.519 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:59:20.519 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6809", 2024-09-21T14:59:20.519 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 144412042 2024-09-21T14:59:20.519 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.519 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.519 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:59:20.519 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6811", 2024-09-21T14:59:20.519 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 144412042 2024-09-21T14:59:20.519 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:20.519 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:20.519 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.519 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.46:6803/144412042", 2024-09-21T14:59:20.519 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.46:6807/144412042", 2024-09-21T14:59:20.520 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.46:6814/144412042", 2024-09-21T14:59:20.520 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.46:6811/144412042", 2024-09-21T14:59:20.520 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-21T14:59:20.520 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-21T14:59:20.520 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-21T14:59:20.520 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:20.520 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.520 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.520 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-09-21T14:59:20.520 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "edf9f045-ddeb-4504-a0fc-1a0e4e853550", 2024-09-21T14:59:20.520 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-21T14:59:20.520 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-21T14:59:20.520 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-21T14:59:20.520 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-21T14:59:20.521 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-21T14:59:20.521 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-21T14:59:20.521 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2024-09-21T14:59:20.521 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 75, 2024-09-21T14:59:20.521 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-21T14:59:20.521 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-21T14:59:20.521 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-21T14:59:20.521 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:59:20.521 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.521 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:59:20.521 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6800", 2024-09-21T14:59:20.521 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 849101442 2024-09-21T14:59:20.521 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.521 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.521 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:59:20.521 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6802", 2024-09-21T14:59:20.522 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 849101442 2024-09-21T14:59:20.522 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:20.522 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:20.522 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.522 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-21T14:59:20.522 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:59:20.522 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.522 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:59:20.522 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6804", 2024-09-21T14:59:20.522 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 849101442 2024-09-21T14:59:20.522 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.522 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.522 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:59:20.522 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6806", 2024-09-21T14:59:20.522 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 849101442 2024-09-21T14:59:20.523 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:20.523 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:20.523 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.523 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-21T14:59:20.523 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:59:20.523 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.523 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:59:20.523 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6813", 2024-09-21T14:59:20.523 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 849101442 2024-09-21T14:59:20.523 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.523 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.523 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:59:20.523 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6815", 2024-09-21T14:59:20.524 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 849101442 2024-09-21T14:59:20.524 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:20.524 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:20.524 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.524 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-21T14:59:20.524 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:59:20.524 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.524 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:59:20.524 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6808", 2024-09-21T14:59:20.524 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 849101442 2024-09-21T14:59:20.524 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.524 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.524 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:59:20.525 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6810", 2024-09-21T14:59:20.525 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 849101442 2024-09-21T14:59:20.525 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:20.525 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:20.525 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.525 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.46:6802/849101442", 2024-09-21T14:59:20.525 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.46:6806/849101442", 2024-09-21T14:59:20.525 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.46:6815/849101442", 2024-09-21T14:59:20.525 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.46:6810/849101442", 2024-09-21T14:59:20.525 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-21T14:59:20.525 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-21T14:59:20.525 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-21T14:59:20.525 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:20.526 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.526 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.526 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-09-21T14:59:20.526 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "c820ea6b-88fe-4421-83ec-17c466691fea", 2024-09-21T14:59:20.526 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-21T14:59:20.526 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-21T14:59:20.526 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-21T14:59:20.526 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 0.0501251220703125, 2024-09-21T14:59:20.526 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-21T14:59:20.526 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-21T14:59:20.526 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2024-09-21T14:59:20.526 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 24, 2024-09-21T14:59:20.526 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-21T14:59:20.527 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-21T14:59:20.527 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-21T14:59:20.527 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:59:20.527 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.527 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:59:20.527 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6817", 2024-09-21T14:59:20.527 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 847189172 2024-09-21T14:59:20.527 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.527 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.527 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:59:20.527 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6818", 2024-09-21T14:59:20.527 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 847189172 2024-09-21T14:59:20.527 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:20.528 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:20.528 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.528 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-21T14:59:20.528 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:59:20.528 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.528 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:59:20.528 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6820", 2024-09-21T14:59:20.528 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 847189172 2024-09-21T14:59:20.528 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.528 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.528 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:59:20.528 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6822", 2024-09-21T14:59:20.528 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 847189172 2024-09-21T14:59:20.529 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:20.529 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:20.529 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.529 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-21T14:59:20.529 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:59:20.529 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.529 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:59:20.529 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6827", 2024-09-21T14:59:20.529 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 847189172 2024-09-21T14:59:20.529 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.529 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.529 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:59:20.529 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6828", 2024-09-21T14:59:20.529 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 847189172 2024-09-21T14:59:20.529 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:20.530 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:20.530 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.530 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-21T14:59:20.530 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:59:20.530 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.530 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:59:20.530 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6823", 2024-09-21T14:59:20.530 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 847189172 2024-09-21T14:59:20.530 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.530 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.530 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:59:20.530 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6825", 2024-09-21T14:59:20.530 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 847189172 2024-09-21T14:59:20.531 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:20.531 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:20.531 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.531 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.46:6818/847189172", 2024-09-21T14:59:20.531 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.46:6822/847189172", 2024-09-21T14:59:20.531 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.46:6828/847189172", 2024-09-21T14:59:20.531 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.46:6825/847189172", 2024-09-21T14:59:20.531 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-21T14:59:20.531 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-21T14:59:20.531 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-21T14:59:20.531 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:20.531 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.531 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.532 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-09-21T14:59:20.532 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "3a2f82a4-28a0-4fcb-9d6b-2285fa4d07ac", 2024-09-21T14:59:20.532 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-21T14:59:20.532 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-21T14:59:20.532 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 0.95001220703125, 2024-09-21T14:59:20.532 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-21T14:59:20.532 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-21T14:59:20.532 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-21T14:59:20.532 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 15, 2024-09-21T14:59:20.532 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 81, 2024-09-21T14:59:20.532 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-21T14:59:20.532 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-21T14:59:20.532 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-21T14:59:20.532 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:59:20.533 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.533 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:59:20.533 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6816", 2024-09-21T14:59:20.533 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3894963553 2024-09-21T14:59:20.533 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.533 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.533 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:59:20.533 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6819", 2024-09-21T14:59:20.533 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3894963553 2024-09-21T14:59:20.533 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:20.533 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:20.533 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.533 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-21T14:59:20.533 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:59:20.534 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.534 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:59:20.534 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6821", 2024-09-21T14:59:20.534 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3894963553 2024-09-21T14:59:20.534 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.534 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.534 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:59:20.534 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6824", 2024-09-21T14:59:20.534 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3894963553 2024-09-21T14:59:20.534 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:20.534 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:20.534 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.535 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-21T14:59:20.535 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:59:20.535 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.535 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:59:20.535 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6830", 2024-09-21T14:59:20.535 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3894963553 2024-09-21T14:59:20.535 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.535 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.535 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:59:20.535 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6831", 2024-09-21T14:59:20.535 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3894963553 2024-09-21T14:59:20.535 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:20.535 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:20.535 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.536 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-21T14:59:20.536 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:59:20.536 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.536 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:59:20.536 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6826", 2024-09-21T14:59:20.536 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3894963553 2024-09-21T14:59:20.536 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.536 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.536 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:59:20.536 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6829", 2024-09-21T14:59:20.536 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3894963553 2024-09-21T14:59:20.536 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:20.536 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:20.536 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.536 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.46:6819/3894963553", 2024-09-21T14:59:20.537 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.46:6824/3894963553", 2024-09-21T14:59:20.537 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.46:6831/3894963553", 2024-09-21T14:59:20.537 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.46:6829/3894963553", 2024-09-21T14:59:20.537 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-21T14:59:20.537 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-21T14:59:20.537 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-21T14:59:20.537 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:20.537 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:20.537 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-21T14:59:20.537 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_xinfo": [ 2024-09-21T14:59:20.537 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.537 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-09-21T14:59:20.537 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-21T14:59:20.537 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-21T14:59:20.537 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-21T14:59:20.538 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-21T14:59:20.538 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-21T14:59:20.538 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-21T14:57:57.458317+0000", 2024-09-21T14:59:20.538 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-21T14:59:20.538 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.538 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.538 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-09-21T14:59:20.538 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-21T14:59:20.538 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-21T14:59:20.538 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-21T14:59:20.538 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-21T14:59:20.538 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-21T14:59:20.538 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-21T14:57:57.907624+0000", 2024-09-21T14:59:20.538 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-21T14:59:20.539 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.539 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.539 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-09-21T14:59:20.539 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-21T14:59:20.539 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-21T14:59:20.539 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-21T14:59:20.539 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-21T14:59:20.539 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-21T14:59:20.539 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-21T14:57:57.278817+0000", 2024-09-21T14:59:20.539 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-21T14:59:20.539 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.539 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.539 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-09-21T14:59:20.539 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-21T14:59:20.539 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-21T14:59:20.540 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-21T14:59:20.540 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-21T14:59:20.540 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-21T14:59:20.540 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-21T14:57:57.500449+0000", 2024-09-21T14:59:20.540 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-21T14:59:20.540 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.540 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.540 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-09-21T14:59:20.540 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-21T14:59:20.540 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-21T14:59:20.540 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-21T14:59:20.540 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-21T14:59:20.540 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 65536, 2024-09-21T14:59:20.540 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-21T14:57:57.504325+0000", 2024-09-21T14:59:20.540 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-21T14:59:20.540 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.541 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.541 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-09-21T14:59:20.541 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-21T14:59:20.541 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-21T14:59:20.541 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-21T14:59:20.541 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-21T14:59:20.541 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-21T14:59:20.541 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-21T14:57:57.777525+0000", 2024-09-21T14:59:20.541 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-21T14:59:20.541 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.541 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.541 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-09-21T14:59:20.541 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-21T14:59:20.541 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-21T14:59:20.541 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-21T14:59:20.541 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-21T14:59:20.542 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-21T14:59:20.542 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-21T14:57:57.698961+0000", 2024-09-21T14:59:20.542 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-21T14:59:20.542 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.542 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.542 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-09-21T14:59:20.542 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-21T14:59:20.542 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-21T14:59:20.542 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-21T14:59:20.542 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-21T14:59:20.542 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-21T14:59:20.542 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-21T14:57:57.932625+0000", 2024-09-21T14:59:20.542 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-21T14:59:20.542 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:20.543 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-21T14:59:20.543 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap": [ 2024-09-21T14:59:20.543 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.543 INFO:teuthology.orchestra.run.smithi032.stdout: "pgid": "3.2", 2024-09-21T14:59:20.543 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2024-09-21T14:59:20.543 INFO:teuthology.orchestra.run.smithi032.stdout: 2, 2024-09-21T14:59:20.543 INFO:teuthology.orchestra.run.smithi032.stdout: 3 2024-09-21T14:59:20.543 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:20.543 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:20.543 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-21T14:59:20.543 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_items": [ 2024-09-21T14:59:20.543 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.543 INFO:teuthology.orchestra.run.smithi032.stdout: "pgid": "3.c", 2024-09-21T14:59:20.543 INFO:teuthology.orchestra.run.smithi032.stdout: "mappings": [ 2024-09-21T14:59:20.544 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.544 INFO:teuthology.orchestra.run.smithi032.stdout: "from": 0, 2024-09-21T14:59:20.544 INFO:teuthology.orchestra.run.smithi032.stdout: "to": 6 2024-09-21T14:59:20.544 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:20.544 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:20.544 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:20.544 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-21T14:59:20.544 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_primaries": [], 2024-09-21T14:59:20.544 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_temp": [], 2024-09-21T14:59:20.544 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_temp": [], 2024-09-21T14:59:20.544 INFO:teuthology.orchestra.run.smithi032.stdout: "blocklist": {}, 2024-09-21T14:59:20.544 INFO:teuthology.orchestra.run.smithi032.stdout: "range_blocklist": {}, 2024-09-21T14:59:20.544 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profiles": { 2024-09-21T14:59:20.545 INFO:teuthology.orchestra.run.smithi032.stdout: "default": { 2024-09-21T14:59:20.545 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2024-09-21T14:59:20.545 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2024-09-21T14:59:20.545 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "1", 2024-09-21T14:59:20.545 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2024-09-21T14:59:20.545 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van" 2024-09-21T14:59:20.545 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:20.545 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.545 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps_queue": [ 2024-09-21T14:59:20.545 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.545 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 3, 2024-09-21T14:59:20.545 INFO:teuthology.orchestra.run.smithi032.stdout: "snaps": [ 2024-09-21T14:59:20.545 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.545 INFO:teuthology.orchestra.run.smithi032.stdout: "begin": 41, 2024-09-21T14:59:20.545 INFO:teuthology.orchestra.run.smithi032.stdout: "length": 1 2024-09-21T14:59:20.546 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:20.546 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.546 INFO:teuthology.orchestra.run.smithi032.stdout: "begin": 44, 2024-09-21T14:59:20.546 INFO:teuthology.orchestra.run.smithi032.stdout: "length": 1 2024-09-21T14:59:20.546 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:20.546 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:20.546 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:20.546 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-21T14:59:20.546 INFO:teuthology.orchestra.run.smithi032.stdout: "new_removed_snaps": [ 2024-09-21T14:59:20.546 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.546 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 3, 2024-09-21T14:59:20.546 INFO:teuthology.orchestra.run.smithi032.stdout: "snaps": [ 2024-09-21T14:59:20.546 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:20.546 INFO:teuthology.orchestra.run.smithi032.stdout: "begin": 44, 2024-09-21T14:59:20.547 INFO:teuthology.orchestra.run.smithi032.stdout: "length": 1 2024-09-21T14:59:20.547 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:20.547 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:20.547 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:20.547 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-21T14:59:20.547 INFO:teuthology.orchestra.run.smithi032.stdout: "new_purged_snaps": [], 2024-09-21T14:59:20.547 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_node_flags": {}, 2024-09-21T14:59:20.547 INFO:teuthology.orchestra.run.smithi032.stdout: "device_class_flags": {}, 2024-09-21T14:59:20.547 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode": { 2024-09-21T14:59:20.547 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_enabled": false, 2024-09-21T14:59:20.547 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_bucket_count": 0, 2024-09-21T14:59:20.547 INFO:teuthology.orchestra.run.smithi032.stdout: "degraded_stretch_mode": 0, 2024-09-21T14:59:20.547 INFO:teuthology.orchestra.run.smithi032.stdout: "recovering_stretch_mode": 0, 2024-09-21T14:59:20.548 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_bucket": 0 2024-09-21T14:59:20.548 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:20.548 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:20.550 INFO:tasks.thrashosds.thrasher:j is {'epoch': 83, 'fsid': 'caf51e55-9234-4a26-bcd2-d60811962560', 'created': '2024-09-21T14:57:50.338822+0000', 'modified': '2024-09-21T14:59:20.161748+0000', 'last_up_change': '2024-09-21T14:58:01.177141+0000', 'last_in_change': '2024-09-21T14:59:15.110268+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T14:58:01.842810+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.8812656402587891, 'average_primary_affinity': 0.8799999952316284, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-21T14:58:08.284925+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 22, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 5, 'score_stable': 5, 'optimal_score': 0.75, 'raw_score_acting': 3.75, 'raw_score_stable': 3.75, 'primary_affinity_weighted': 0.8812656402587891, 'average_primary_affinity': 0.8799999952316284, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-21T14:58:17.510709+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 45, 'snap_epoch': 83, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2.490000009536743, 'score_stable': 2.490000009536743, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 2.190000057220459, 'raw_score_stable': 2.190000057220459, 'primary_affinity_weighted': 0.8812656402587891, 'average_primary_affinity': 0.8799999952316284, 'average_primary_affinity_weighted': 0.8799999952316284}}], 'osds': [{'osd': 0, 'uuid': 'a4a73316-3a94-42e9-9a51-deb46a66bb06', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 81, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6808', 'nonce': 4016672243}, {'type': 'v1', 'addr': '172.21.15.32:6809', 'nonce': 4016672243}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6810', 'nonce': 4016672243}, {'type': 'v1', 'addr': '172.21.15.32:6811', 'nonce': 4016672243}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6814', 'nonce': 4016672243}, {'type': 'v1', 'addr': '172.21.15.32:6815', 'nonce': 4016672243}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6812', 'nonce': 4016672243}, {'type': 'v1', 'addr': '172.21.15.32:6813', 'nonce': 4016672243}]}, 'public_addr': '172.21.15.32:6809/4016672243', 'cluster_addr': '172.21.15.32:6811/4016672243', 'heartbeat_back_addr': '172.21.15.32:6815/4016672243', 'heartbeat_front_addr': '172.21.15.32:6813/4016672243', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '006757a0-94d1-4a9b-aef8-73634cc5828e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 81, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6824', 'nonce': 641090577}, {'type': 'v1', 'addr': '172.21.15.32:6825', 'nonce': 641090577}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6826', 'nonce': 641090577}, {'type': 'v1', 'addr': '172.21.15.32:6827', 'nonce': 641090577}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6830', 'nonce': 641090577}, {'type': 'v1', 'addr': '172.21.15.32:6831', 'nonce': 641090577}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6828', 'nonce': 641090577}, {'type': 'v1', 'addr': '172.21.15.32:6829', 'nonce': 641090577}]}, 'public_addr': '172.21.15.32:6825/641090577', 'cluster_addr': '172.21.15.32:6827/641090577', 'heartbeat_back_addr': '172.21.15.32:6831/641090577', 'heartbeat_front_addr': '172.21.15.32:6829/641090577', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '0b499a23-820e-4e6c-a558-adbf88b7014f', 'up': 1, 'in': 1, 'weight': 0.3514556884765625, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 51, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6800', 'nonce': 2609154414}, {'type': 'v1', 'addr': '172.21.15.32:6801', 'nonce': 2609154414}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6802', 'nonce': 2609154414}, {'type': 'v1', 'addr': '172.21.15.32:6803', 'nonce': 2609154414}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6806', 'nonce': 2609154414}, {'type': 'v1', 'addr': '172.21.15.32:6807', 'nonce': 2609154414}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6804', 'nonce': 2609154414}, {'type': 'v1', 'addr': '172.21.15.32:6805', 'nonce': 2609154414}]}, 'public_addr': '172.21.15.32:6801/2609154414', 'cluster_addr': '172.21.15.32:6803/2609154414', 'heartbeat_back_addr': '172.21.15.32:6807/2609154414', 'heartbeat_front_addr': '172.21.15.32:6805/2609154414', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '5a57b3dd-4a07-45b7-b159-3cac4c7eb262', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 81, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6816', 'nonce': 1462563376}, {'type': 'v1', 'addr': '172.21.15.32:6817', 'nonce': 1462563376}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6818', 'nonce': 1462563376}, {'type': 'v1', 'addr': '172.21.15.32:6819', 'nonce': 1462563376}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6822', 'nonce': 1462563376}, {'type': 'v1', 'addr': '172.21.15.32:6823', 'nonce': 1462563376}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6820', 'nonce': 1462563376}, {'type': 'v1', 'addr': '172.21.15.32:6821', 'nonce': 1462563376}]}, 'public_addr': '172.21.15.32:6817/1462563376', 'cluster_addr': '172.21.15.32:6819/1462563376', 'heartbeat_back_addr': '172.21.15.32:6823/1462563376', 'heartbeat_front_addr': '172.21.15.32:6821/1462563376', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '39a00807-7517-474e-8842-623dd7093cc5', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 80, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6801', 'nonce': 144412042}, {'type': 'v1', 'addr': '172.21.15.46:6803', 'nonce': 144412042}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6805', 'nonce': 144412042}, {'type': 'v1', 'addr': '172.21.15.46:6807', 'nonce': 144412042}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6812', 'nonce': 144412042}, {'type': 'v1', 'addr': '172.21.15.46:6814', 'nonce': 144412042}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6809', 'nonce': 144412042}, {'type': 'v1', 'addr': '172.21.15.46:6811', 'nonce': 144412042}]}, 'public_addr': '172.21.15.46:6803/144412042', 'cluster_addr': '172.21.15.46:6807/144412042', 'heartbeat_back_addr': '172.21.15.46:6814/144412042', 'heartbeat_front_addr': '172.21.15.46:6811/144412042', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'edf9f045-ddeb-4504-a0fc-1a0e4e853550', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 75, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6800', 'nonce': 849101442}, {'type': 'v1', 'addr': '172.21.15.46:6802', 'nonce': 849101442}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6804', 'nonce': 849101442}, {'type': 'v1', 'addr': '172.21.15.46:6806', 'nonce': 849101442}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6813', 'nonce': 849101442}, {'type': 'v1', 'addr': '172.21.15.46:6815', 'nonce': 849101442}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6808', 'nonce': 849101442}, {'type': 'v1', 'addr': '172.21.15.46:6810', 'nonce': 849101442}]}, 'public_addr': '172.21.15.46:6802/849101442', 'cluster_addr': '172.21.15.46:6806/849101442', 'heartbeat_back_addr': '172.21.15.46:6815/849101442', 'heartbeat_front_addr': '172.21.15.46:6810/849101442', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'c820ea6b-88fe-4421-83ec-17c466691fea', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.0501251220703125, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6817', 'nonce': 847189172}, {'type': 'v1', 'addr': '172.21.15.46:6818', 'nonce': 847189172}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6820', 'nonce': 847189172}, {'type': 'v1', 'addr': '172.21.15.46:6822', 'nonce': 847189172}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6827', 'nonce': 847189172}, {'type': 'v1', 'addr': '172.21.15.46:6828', 'nonce': 847189172}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6823', 'nonce': 847189172}, {'type': 'v1', 'addr': '172.21.15.46:6825', 'nonce': 847189172}]}, 'public_addr': '172.21.15.46:6818/847189172', 'cluster_addr': '172.21.15.46:6822/847189172', 'heartbeat_back_addr': '172.21.15.46:6828/847189172', 'heartbeat_front_addr': '172.21.15.46:6825/847189172', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '3a2f82a4-28a0-4fcb-9d6b-2285fa4d07ac', 'up': 1, 'in': 1, 'weight': 0.95001220703125, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 81, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6816', 'nonce': 3894963553}, {'type': 'v1', 'addr': '172.21.15.46:6819', 'nonce': 3894963553}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6821', 'nonce': 3894963553}, {'type': 'v1', 'addr': '172.21.15.46:6824', 'nonce': 3894963553}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6830', 'nonce': 3894963553}, {'type': 'v1', 'addr': '172.21.15.46:6831', 'nonce': 3894963553}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6826', 'nonce': 3894963553}, {'type': 'v1', 'addr': '172.21.15.46:6829', 'nonce': 3894963553}]}, 'public_addr': '172.21.15.46:6819/3894963553', 'cluster_addr': '172.21.15.46:6824/3894963553', 'heartbeat_back_addr': '172.21.15.46:6831/3894963553', 'heartbeat_front_addr': '172.21.15.46:6829/3894963553', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:57:57.458317+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:57:57.907624+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:57:57.278817+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:57:57.500449+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-21T14:57:57.504325+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:57:57.777525+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:57:57.698961+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:57:57.932625+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.2', 'osds': [2, 3]}], 'pg_upmap_items': [{'pgid': '3.c', 'mappings': [{'from': 0, 'to': 6}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 41, 'length': 1}, {'begin': 44, 'length': 1}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 44, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-21T14:59:20.550 INFO:tasks.thrashosds.thrasher:Clearing pg_upmap on 3.c 2024-09-21T14:59:20.550 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd rm-pg-upmap-items 3.c 2024-09-21T14:59:20.551 INFO:teuthology.orchestra.run.smithi032.stdout:1456 read 0xcff0b53 thru 0xcff3cdc (0x318a bytes) 2024-09-21T14:59:20.551 INFO:teuthology.orchestra.run.smithi032.stdout:1457 write 0xeb25e20 thru 0xeb2bf9c (0x617d bytes) 2024-09-21T14:59:20.551 INFO:teuthology.orchestra.run.smithi032.stdout:1458 read 0xacc227f thru 0xaccb05a (0x8ddc bytes) 2024-09-21T14:59:20.551 INFO:teuthology.orchestra.run.smithi032.stdout:1460 punch from 0x6729457 to 0x67382db, (0xee84 bytes) 2024-09-21T14:59:20.551 INFO:teuthology.orchestra.run.smithi032.stdout:1462 trunc from 0xeb4eee7 to 0x82d832a 2024-09-21T14:59:20.551 INFO:teuthology.orchestra.run.smithi032.stdout:1465 write 0xc0f0573 thru 0xc0f8d88 (0x8816 bytes) 2024-09-21T14:59:20.551 INFO:teuthology.orchestra.run.smithi032.stdout:1466 read 0x2d58de7 thru 0x2d5f35a (0x6574 bytes) 2024-09-21T14:59:20.551 INFO:teuthology.orchestra.run.smithi032.stdout:1467 read 0x2d195e4 thru 0x2d2307a (0x9a97 bytes) 2024-09-21T14:59:20.551 INFO:teuthology.orchestra.run.smithi032.stdout:1469 writesame 0x7d58f0 thru 0x7db176 data_size 0x1(0x5887 bytes) 2024-09-21T14:59:20.552 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:59:20.543+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:59:20.563 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:20.563 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:20.566 INFO:teuthology.orchestra.run.smithi032.stdout:1470 writesame 0x15c8b83 thru 0x15d573e data_size 0x76(0xcbbc bytes) 2024-09-21T14:59:20.573 INFO:teuthology.orchestra.run.smithi032.stdout:1472 trunc from 0xc0f8d89 to 0xab4a6f5 2024-09-21T14:59:20.575 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T14:59:20.606 INFO:teuthology.orchestra.run.smithi032.stdout:1473 write 0xe576a64 thru 0xe5850cb (0xe668 bytes) 2024-09-21T14:59:20.615 INFO:teuthology.orchestra.run.smithi032.stdout:1474 write 0x8d11486 thru 0x8d13f9d (0x2b18 bytes) 2024-09-21T14:59:20.621 INFO:teuthology.orchestra.run.smithi032.stdout:1475 writesame 0x2e641b3 thru 0x2e69d75 data_size 0xd(0x5bc3 bytes) 2024-09-21T14:59:20.629 INFO:teuthology.orchestra.run.smithi032.stdout:1476 punch from 0x1cc0e32 to 0x1cc8b79, (0x7d47 bytes) 2024-09-21T14:59:20.629 INFO:teuthology.orchestra.run.smithi032.stdout:1477 read 0x946799b thru 0x9477985 (0xffeb bytes) 2024-09-21T14:59:20.629 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:20.630 INFO:teuthology.orchestra.run.smithi032.stdout:1480 writesame 0xd110dc2 thru 0xd117ec0 data_size 0x1(0x70ff bytes) 2024-09-21T14:59:20.650 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:59:20.647+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:59:20.660 INFO:teuthology.orchestra.run.smithi032.stdout:1481 read 0xba1c08e thru 0xba27759 (0xb6cc bytes) 2024-09-21T14:59:20.660 INFO:teuthology.orchestra.run.smithi032.stdout:1482 read 0x5e5d937 thru 0x5e5e5c8 (0xc92 bytes) 2024-09-21T14:59:20.661 INFO:teuthology.orchestra.run.smithi032.stdout:1483 read 0xc676a02 thru 0xc6840ff (0xd6fe bytes) 2024-09-21T14:59:20.661 INFO:teuthology.orchestra.run.smithi032.stdout:1484 write 0x8bc2ea7 thru 0x8bd0cd2 (0xde2c bytes) 2024-09-21T14:59:20.667 INFO:teuthology.orchestra.run.smithi032.stdout:1485 writesame 0x296fc73 thru 0x2972f96 data_size 0x6(0x3324 bytes) 2024-09-21T14:59:20.674 INFO:teuthology.orchestra.run.smithi032.stdout:1487 writesame 0x1cf30b5 thru 0x1cf8e3e data_size 0x27(0x5d8a bytes) 2024-09-21T14:59:20.680 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:20.680 INFO:teuthology.orchestra.run.smithi032.stdout:1489 read 0xe9b3a6 thru 0xe9ffbd (0x4c18 bytes) 2024-09-21T14:59:20.681 INFO:teuthology.orchestra.run.smithi032.stdout:1490 write 0x4c34c0a thru 0x4c36961 (0x1d58 bytes) 2024-09-21T14:59:20.686 INFO:teuthology.orchestra.run.smithi032.stdout:1491 write 0x8f8c9fa thru 0x8f8e280 (0x1887 bytes) 2024-09-21T14:59:20.690 INFO:teuthology.orchestra.run.smithi032.stdout:1493 writesame 0x1e01e3c thru 0x1e0675e data_size 0x3(0x4923 bytes) 2024-09-21T14:59:20.700 INFO:teuthology.orchestra.run.smithi032.stdout:1494 punch from 0xacaa3d4 to 0xacabd18, (0x1944 bytes) 2024-09-21T14:59:20.701 INFO:teuthology.orchestra.run.smithi032.stdout:1495 punch from 0x43a7069 to 0x43b455c, (0xd4f3 bytes) 2024-09-21T14:59:20.701 INFO:teuthology.orchestra.run.smithi032.stdout:1496 write 0x54904dc thru 0x5495252 (0x4d77 bytes) 2024-09-21T14:59:20.706 INFO:teuthology.orchestra.run.smithi032.stdout:1497 writesame 0x795144d thru 0x795fccc data_size 0xa0(0xe880 bytes) 2024-09-21T14:59:20.712 INFO:teuthology.orchestra.run.smithi032.stdout:1498 read 0x67eb7a5 thru 0x67f1d94 (0x65f0 bytes) 2024-09-21T14:59:20.712 INFO:teuthology.orchestra.run.smithi032.stdout:1499 write 0x25dcb60 thru 0x25e328d (0x672e bytes) 2024-09-21T14:59:20.717 INFO:teuthology.orchestra.run.smithi032.stdout:1501 punch from 0x71afec5 to 0x71ba416, (0xa551 bytes) 2024-09-21T14:59:20.718 INFO:teuthology.orchestra.run.smithi032.stdout:1503 write 0x91c9f08 thru 0x91d4d7d (0xae76 bytes) 2024-09-21T14:59:20.720 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:20.720 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:20.724 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:20.724 INFO:teuthology.orchestra.run.smithi032.stdout:1505 punch from 0x2d669b7 to 0x2d75168, (0xe7b1 bytes) 2024-09-21T14:59:20.724 INFO:teuthology.orchestra.run.smithi032.stdout:1507 read 0x94f660c thru 0x9501955 (0xb34a bytes) 2024-09-21T14:59:20.724 INFO:teuthology.orchestra.run.smithi032.stdout:1510 read 0x9a8ccfe thru 0x9a99e91 (0xd194 bytes) 2024-09-21T14:59:20.725 INFO:teuthology.orchestra.run.smithi032.stdout:1512 write 0x82ccf80 thru 0x82d4ebb (0x7f3c bytes) 2024-09-21T14:59:20.732 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T14:59:20.743 INFO:teuthology.orchestra.run.smithi032.stdout:1514 trunc from 0xe5850cc to 0x6d3185b 2024-09-21T14:59:20.761 INFO:teuthology.orchestra.run.smithi032.stdout:1517 write 0x2e76cad thru 0x2e7db3b (0x6e8f bytes) 2024-09-21T14:59:20.761 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:59:20.761+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:59:20.784 INFO:teuthology.orchestra.run.smithi032.stdout:1519 write 0xdae2c8e thru 0xdae3725 (0xa98 bytes) 2024-09-21T14:59:20.790 INFO:teuthology.orchestra.run.smithi032.stdout:1520 write 0x2c24253 thru 0x2c2a7fe (0x65ac bytes) 2024-09-21T14:59:20.807 INFO:teuthology.orchestra.run.smithi032.stdout:1521 write 0xe7b18bc thru 0xe7bff56 (0xe69b bytes) 2024-09-21T14:59:20.814 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:20.815 INFO:teuthology.orchestra.run.smithi032.stdout:1523 write 0x8a6700e thru 0x8a6a0d0 (0x30c3 bytes) 2024-09-21T14:59:20.820 INFO:teuthology.orchestra.run.smithi032.stdout:1524 write 0x9e51933 thru 0x9e55b5b (0x4229 bytes) 2024-09-21T14:59:20.826 INFO:teuthology.orchestra.run.smithi032.stdout:1525 punch from 0x404ca74 to 0x4056b0c, (0xa098 bytes) 2024-09-21T14:59:20.826 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:20.826 INFO:teuthology.orchestra.run.smithi032.stdout:1528 read 0xdfe84b0 thru 0xdfec375 (0x3ec6 bytes) 2024-09-21T14:59:20.826 INFO:teuthology.orchestra.run.smithi032.stdout:1530 punch from 0xb6813b4 to 0xb683a2f, (0x267b bytes) 2024-09-21T14:59:20.826 INFO:teuthology.orchestra.run.smithi032.stdout:1532 read 0x2fbc3d7 thru 0x2fc8c1f (0xc849 bytes) 2024-09-21T14:59:20.826 INFO:teuthology.orchestra.run.smithi032.stdout:1533 write 0x7488bc6 thru 0x7489448 (0x883 bytes) 2024-09-21T14:59:20.831 INFO:teuthology.orchestra.run.smithi032.stdout:1534 punch from 0x4e1859e to 0x4e2436a, (0xbdcc bytes) 2024-09-21T14:59:20.831 INFO:teuthology.orchestra.run.smithi032.stdout:1535 read 0x4745411 thru 0x474a52b (0x511b bytes) 2024-09-21T14:59:20.831 INFO:teuthology.orchestra.run.smithi032.stdout:1536 read 0x6311875 thru 0x6315dfa (0x4586 bytes) 2024-09-21T14:59:20.831 INFO:teuthology.orchestra.run.smithi032.stdout:1537 write 0x48d3981 thru 0x48e321a (0xf89a bytes) 2024-09-21T14:59:20.836 INFO:teuthology.orchestra.run.smithi032.stdout:1538 punch from 0x78bc5cc to 0x78beb74, (0x25a8 bytes) 2024-09-21T14:59:20.836 INFO:teuthology.orchestra.run.smithi032.stdout:1540 read 0x1b537b8 thru 0x1b56733 (0x2f7c bytes) 2024-09-21T14:59:20.836 INFO:teuthology.orchestra.run.smithi032.stdout:1541 write 0x62cbf0e thru 0x62d4dc6 (0x8eb9 bytes) 2024-09-21T14:59:20.843 INFO:teuthology.orchestra.run.smithi032.stdout:1542 read 0xa19a341 thru 0xa19e1c0 (0x3e80 bytes) 2024-09-21T14:59:20.843 INFO:teuthology.orchestra.run.smithi032.stdout:1543 write 0x2e10c0a thru 0x2e179fd (0x6df4 bytes) 2024-09-21T14:59:20.847 INFO:teuthology.orchestra.run.smithi032.stdout:1544 writesame 0x53bb09e thru 0x53c81ac data_size 0x1(0xd10f bytes) 2024-09-21T14:59:20.852 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:59:20.847+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:59:20.858 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:20.858 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:20.869 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T14:59:20.888 INFO:teuthology.orchestra.run.smithi032.stdout:1545 write 0xabc1d2a thru 0xabcfa57 (0xdd2e bytes) 2024-09-21T14:59:20.895 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:20.895 INFO:teuthology.orchestra.run.smithi032.stdout:1548 read 0x3e57a62 thru 0x3e5a99f (0x2f3e bytes) 2024-09-21T14:59:20.895 INFO:teuthology.orchestra.run.smithi032.stdout:1549 read 0x14d99c7 thru 0x14da2b3 (0x8ed bytes) 2024-09-21T14:59:20.896 INFO:teuthology.orchestra.run.smithi032.stdout:1551 writesame 0x6453085 thru 0x645aa51 data_size 0x1(0x79cd bytes) 2024-09-21T14:59:20.924 INFO:teuthology.orchestra.run.smithi032.stdout:1552 read 0xd502aff thru 0xd50f154 (0xc656 bytes) 2024-09-21T14:59:20.924 INFO:teuthology.orchestra.run.smithi032.stdout:1553 punch from 0xcffee9 to 0xd05266, (0x537d bytes) 2024-09-21T14:59:20.930 INFO:teuthology.orchestra.run.smithi032.stdout:1554 trunc from 0xe7bff57 to 0xa44725d 2024-09-21T14:59:20.952 INFO:teuthology.orchestra.run.smithi032.stdout:1557 trunc from 0xa44725d to 0xb1b8f29 2024-09-21T14:59:20.952 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:59:20.953+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:59:20.962 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:20.968 INFO:teuthology.orchestra.run.smithi032.stdout:1559 trunc from 0xb1b8f29 to 0x8433add 2024-09-21T14:59:20.989 INFO:teuthology.orchestra.run.smithi032.stdout:1560 trunc from 0x8433add to 0x6d69fba 2024-09-21T14:59:20.997 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:20.997 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:21.009 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T14:59:21.040 INFO:teuthology.orchestra.run.smithi032.stdout:1561 write 0xedcff38 thru 0xedde080 (0xe149 bytes) 2024-09-21T14:59:21.045 INFO:teuthology.orchestra.run.smithi032.stdout:1562 punch from 0x8a71d8 to 0x8b2e67, (0xbc8f bytes) 2024-09-21T14:59:21.045 INFO:teuthology.orchestra.run.smithi032.stdout:1563 read 0x9d3f169 thru 0x9d437a6 (0x463e bytes) 2024-09-21T14:59:21.046 INFO:teuthology.orchestra.run.smithi032.stdout:1565 write 0x93499c9 thru 0x934a455 (0xa8d bytes) 2024-09-21T14:59:21.051 INFO:teuthology.orchestra.run.smithi032.stdout:1566 write 0xa0cb1bc thru 0xa0d1742 (0x6587 bytes) 2024-09-21T14:59:21.053 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:59:21.053+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:59:21.059 INFO:teuthology.orchestra.run.smithi032.stdout:1567 read 0xd4cd486 thru 0xd4d9e40 (0xc9bb bytes) 2024-09-21T14:59:21.059 INFO:teuthology.orchestra.run.smithi032.stdout:1568 write 0x9acbe08 thru 0x9ad836d (0xc566 bytes) 2024-09-21T14:59:21.063 INFO:teuthology.orchestra.run.smithi032.stdout:1569 writesame 0x38f0a53 thru 0x38f4e8e data_size 0x16(0x443c bytes) 2024-09-21T14:59:21.070 INFO:teuthology.orchestra.run.smithi032.stdout:1570 read 0x6cca4ac thru 0x6cd0eb8 (0x6a0d bytes) 2024-09-21T14:59:21.079 INFO:teuthology.orchestra.run.smithi032.stdout:1571 trunc from 0xedde081 to 0x9dc0386 2024-09-21T14:59:21.105 INFO:teuthology.orchestra.run.smithi032.stdout:1573 punch from 0x33155c7 to 0x331cc83, (0x76bc bytes) 2024-09-21T14:59:21.123 INFO:teuthology.orchestra.run.smithi032.stdout:1574 write 0xe5dd9df thru 0xe5e5e59 (0x847b bytes) 2024-09-21T14:59:21.130 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:21.130 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:21.130 INFO:teuthology.orchestra.run.smithi032.stdout:1577 write 0x5c434ff thru 0x5c47f2a (0x4a2c bytes) 2024-09-21T14:59:21.135 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:21.135 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:21.154 INFO:teuthology.orchestra.run.smithi032.stdout:1583 trunc from 0xe5e5e5a to 0x2a9f3b0 2024-09-21T14:59:21.154 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:21.154 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:21.154 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:59:21.153+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:59:21.166 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T14:59:21.176 INFO:teuthology.orchestra.run.smithi032.stderr:clear 3.c pg_upmap_items mapping 2024-09-21T14:59:21.188 INFO:teuthology.orchestra.run.smithi032.stdout:1584 punch from 0x119a9c3 to 0x11a65fa, (0xbc37 bytes) 2024-09-21T14:59:21.188 INFO:teuthology.orchestra.run.smithi032.stdout:1585 write 0x1560d77 thru 0x156f74a (0xe9d4 bytes) 2024-09-21T14:59:21.218 INFO:teuthology.orchestra.run.smithi032.stdout:1586 writesame 0xb17e058 thru 0xb184061 data_size 0x13(0x600a bytes) 2024-09-21T14:59:21.218 INFO:teuthology.orchestra.run.smithi032.stdout:1589 read 0x4177c7c thru 0x417e027 (0x63ac bytes) 2024-09-21T14:59:21.218 INFO:teuthology.orchestra.run.smithi032.stdout:1592 write 0x8ff9644 thru 0x8ffd5ad (0x3f6a bytes) 2024-09-21T14:59:21.237 INFO:teuthology.orchestra.run.smithi032.stdout:1593 writesame 0xd217cef thru 0xd21908a data_size 0xa(0x139c bytes) 2024-09-21T14:59:21.237 INFO:teuthology.orchestra.run.smithi032.stdout:1594 punch from 0xfbf778 to 0xfc936c, (0x9bf4 bytes) 2024-09-21T14:59:21.237 INFO:teuthology.orchestra.run.smithi032.stdout:1595 punch from 0xb6c2e67 to 0xb6c8615, (0x57ae bytes) 2024-09-21T14:59:21.237 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:21.252 INFO:teuthology.orchestra.run.smithi032.stdout:1598 trunc from 0xd21908b to 0x42f54a3 2024-09-21T14:59:21.255 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:59:21.253+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:59:21.271 INFO:teuthology.orchestra.run.smithi032.stdout:1599 read 0x2f37235 thru 0x2f3b368 (0x4134 bytes) 2024-09-21T14:59:21.272 INFO:teuthology.orchestra.run.smithi032.stdout:1600 read 0x1023868 thru 0x1032b05 (0xf29e bytes) 2024-09-21T14:59:21.294 INFO:teuthology.orchestra.run.smithi032.stdout:1601 write 0x9de555a thru 0x9df0108 (0xabaf bytes) 2024-09-21T14:59:21.303 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:21.303 INFO:teuthology.orchestra.run.smithi032.stdout:1604 write 0x543cb64 thru 0x543fd6c (0x3209 bytes) 2024-09-21T14:59:21.310 INFO:teuthology.orchestra.run.smithi032.stdout:1605 read 0x6c00e22 thru 0x6c048de (0x3abd bytes) 2024-09-21T14:59:21.320 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:21.320 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:21.326 INFO:teuthology.orchestra.run.smithi032.stdout:1606 write 0xd036d2a thru 0xd039f52 (0x3229 bytes) 2024-09-21T14:59:21.331 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T14:59:21.332 INFO:teuthology.orchestra.run.smithi032.stdout:1607 writesame 0x9c1af5f thru 0x9c1e04b data_size 0x19(0x30ed bytes) 2024-09-21T14:59:21.349 INFO:teuthology.orchestra.run.smithi032.stdout:1608 writesame 0xd719baa thru 0xd722efd data_size 0x1c(0x9354 bytes) 2024-09-21T14:59:21.349 INFO:teuthology.orchestra.run.smithi032.stdout:1609 write 0xe89ca thru 0xf748f (0xeac6 bytes) 2024-09-21T14:59:21.354 INFO:teuthology.orchestra.run.smithi032.stdout:1610 read 0x80a45c2 thru 0x80aa271 (0x5cb0 bytes) 2024-09-21T14:59:21.354 INFO:teuthology.orchestra.run.smithi032.stdout:1612 read 0x3b5c1c7 thru 0x3b5e858 (0x2692 bytes) 2024-09-21T14:59:21.354 INFO:teuthology.orchestra.run.smithi032.stdout:1613 write 0x83151e4 thru 0x8316543 (0x1360 bytes) 2024-09-21T14:59:21.355 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:59:21.353+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:59:21.361 INFO:teuthology.orchestra.run.smithi032.stdout:1614 read 0x3e9e2ad thru 0x3eaa4fb (0xc24f bytes) 2024-09-21T14:59:21.361 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:21.361 INFO:teuthology.orchestra.run.smithi032.stdout:1616 punch from 0x50ac28a to 0x50b9161, (0xced7 bytes) 2024-09-21T14:59:21.361 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:21.362 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:21.362 INFO:teuthology.orchestra.run.smithi032.stdout:1620 read 0x6b2a23a thru 0x6b32e72 (0x8c39 bytes) 2024-09-21T14:59:21.362 INFO:teuthology.orchestra.run.smithi032.stdout:1621 write 0xce68bd thru 0xcf2331 (0xba75 bytes) 2024-09-21T14:59:21.365 INFO:teuthology.orchestra.run.smithi032.stdout:1623 punch from 0x4e0c76b to 0x4e1961e, (0xceb3 bytes) 2024-09-21T14:59:21.365 INFO:teuthology.orchestra.run.smithi032.stdout:1624 clone 15 order 21 su 1048576 sc 12 2024-09-21T14:59:21.433 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:21.433 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:21.443 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T14:59:21.457 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:59:21.455+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:59:21.557 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:59:21.555+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:59:21.595 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:21.596 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:21.606 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T14:59:21.658 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:59:21.657+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:59:21.709 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:21.709 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:21.719 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T14:59:21.766 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:59:21.763+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:59:21.790 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T14:59:21.837 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:21.837 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:21.848 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T14:59:21.860 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:59:21.857+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:59:21.938 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T14:59:21.960 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T14:59:21.960 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:59:21.961+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:59:21.963 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:21.963 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:21.973 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T14:59:21.981 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T14:59:22.001 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T14:59:22.022 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-21T14:59:22.044 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T14:59:22.061 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:59:22.061+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:59:22.067 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T14:59:22.084 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:59:22.084 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T14:59:22.084 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T14:59:22.084 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T14:59:22.085 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.085 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:1452 3.19 3:9d40fe72:::rbd_data.1136515ecb6b.0000000000000013:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T14:59:22.085 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:30.292755+0000", 2024-09-21T14:59:22.085 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 51.788967022000001, 2024-09-21T14:59:22.085 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043852191999999998, 2024-09-21T14:59:22.085 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:22.085 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:22.085 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:22.085 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:22.085 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:22.085 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1452 2024-09-21T14:59:22.085 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.086 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:22.086 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.086 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:22.086 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292755+0000", 2024-09-21T14:59:22.086 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:22.086 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.086 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.086 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:22.086 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292755+0000", 2024-09-21T14:59:22.086 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:22.086 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.086 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.086 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:22.087 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292754+0000", 2024-09-21T14:59:22.087 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:22.087 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.087 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.087 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:22.087 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292760+0000", 2024-09-21T14:59:22.087 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5029999999999997e-06 2024-09-21T14:59:22.087 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.087 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.087 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:22.087 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292771+0000", 2024-09-21T14:59:22.087 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1551e-05 2024-09-21T14:59:22.087 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.088 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.088 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:22.088 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292779+0000", 2024-09-21T14:59:22.088 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1559999999999998e-06 2024-09-21T14:59:22.088 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.088 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.088 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:22.088 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292896+0000", 2024-09-21T14:59:22.088 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000117304 2024-09-21T14:59:22.088 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.088 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.088 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:22.088 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.293124+0000", 2024-09-21T14:59:22.088 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00022819999999999999 2024-09-21T14:59:22.088 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.088 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.089 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 1", 2024-09-21T14:59:22.089 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.299967+0000", 2024-09-21T14:59:22.089 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0068427009999999996 2024-09-21T14:59:22.089 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.089 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.089 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:22.089 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336463+0000", 2024-09-21T14:59:22.089 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.036495842000000001 2024-09-21T14:59:22.089 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.089 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.089 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:59:22.089 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336480+0000", 2024-09-21T14:59:22.089 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7419999999999999e-05 2024-09-21T14:59:22.089 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.090 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.090 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:59:22.090 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336550+0000", 2024-09-21T14:59:22.090 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9951999999999997e-05 2024-09-21T14:59:22.090 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.090 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.090 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:22.090 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336607+0000", 2024-09-21T14:59:22.090 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7423000000000002e-05 2024-09-21T14:59:22.090 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:22.090 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:22.090 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:22.090 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.090 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.091 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop_reply(client.4406.0:1452 3.19 e36/24 ondisk, result = 0)", 2024-09-21T14:59:22.091 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:30.322854+0000", 2024-09-21T14:59:22.091 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 51.758868346, 2024-09-21T14:59:22.091 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013613079, 2024-09-21T14:59:22.091 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:22.091 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:22.091 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:22.091 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.091 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:22.091 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.322854+0000", 2024-09-21T14:59:22.091 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:22.091 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.091 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.091 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:22.091 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.322854+0000", 2024-09-21T14:59:22.092 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:22.092 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.092 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.092 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:22.092 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.322852+0000", 2024-09-21T14:59:22.092 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:59:22.092 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.092 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.092 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:22.092 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.322860+0000", 2024-09-21T14:59:22.092 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.605e-06 2024-09-21T14:59:22.092 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.092 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.092 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:22.093 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.322878+0000", 2024-09-21T14:59:22.093 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8204000000000001e-05 2024-09-21T14:59:22.093 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.093 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.093 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:22.093 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.322888+0000", 2024-09-21T14:59:22.093 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4099999999999997e-06 2024-09-21T14:59:22.093 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.093 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.093 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:22.093 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336440+0000", 2024-09-21T14:59:22.093 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013551848 2024-09-21T14:59:22.093 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.093 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.093 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:22.094 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336453+0000", 2024-09-21T14:59:22.094 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3465000000000001e-05 2024-09-21T14:59:22.094 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.094 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.094 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:22.094 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336467+0000", 2024-09-21T14:59:22.094 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3913999999999999e-05 2024-09-21T14:59:22.094 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:22.094 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:22.094 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:22.094 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.094 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.095 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:1567 3.8 3:139d3d55:::rbd_data.1136515ecb6b.0000000000000000:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T14:59:22.095 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:30.681087+0000", 2024-09-21T14:59:22.095 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 51.400634807000003, 2024-09-21T14:59:22.095 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.039009224000000002, 2024-09-21T14:59:22.095 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:22.095 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:22.095 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:22.095 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:22.095 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:22.095 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1567 2024-09-21T14:59:22.095 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.095 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:22.095 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.096 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:22.096 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681087+0000", 2024-09-21T14:59:22.096 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:22.096 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.096 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.096 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:22.096 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681087+0000", 2024-09-21T14:59:22.096 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:22.096 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.096 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.096 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:22.096 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681086+0000", 2024-09-21T14:59:22.096 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:59:22.096 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.096 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.097 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:22.097 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681097+0000", 2024-09-21T14:59:22.097 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.134e-05 2024-09-21T14:59:22.097 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.097 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.097 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:22.097 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681122+0000", 2024-09-21T14:59:22.097 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5313e-05 2024-09-21T14:59:22.097 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.097 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.097 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:22.097 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681138+0000", 2024-09-21T14:59:22.097 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5916000000000001e-05 2024-09-21T14:59:22.097 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.097 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.097 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:22.098 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681270+0000", 2024-09-21T14:59:22.098 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000131474 2024-09-21T14:59:22.098 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.098 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.098 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:22.098 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681416+0000", 2024-09-21T14:59:22.098 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014637 2024-09-21T14:59:22.098 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.098 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.098 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 7", 2024-09-21T14:59:22.098 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.693163+0000", 2024-09-21T14:59:22.098 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011747241 2024-09-21T14:59:22.098 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.098 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.098 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:59:22.098 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.719750+0000", 2024-09-21T14:59:22.099 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.026586852000000001 2024-09-21T14:59:22.099 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.099 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.099 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:22.099 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.719859+0000", 2024-09-21T14:59:22.099 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010925999999999999 2024-09-21T14:59:22.099 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.099 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.099 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:59:22.099 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.719951+0000", 2024-09-21T14:59:22.099 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1294999999999999e-05 2024-09-21T14:59:22.099 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.099 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.099 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:22.099 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.720097+0000", 2024-09-21T14:59:22.099 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014583 2024-09-21T14:59:22.100 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:22.100 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:22.100 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:22.100 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.100 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.100 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9347 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 16384~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:22.100 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158291+0000", 2024-09-21T14:59:22.100 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.923431305999998, 2024-09-21T14:59:22.100 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013600958, 2024-09-21T14:59:22.100 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:22.100 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:22.100 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:22.100 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:22.100 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:22.100 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9347 2024-09-21T14:59:22.100 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.101 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:22.101 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.101 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:22.101 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158291+0000", 2024-09-21T14:59:22.101 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:22.101 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.101 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.101 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:22.101 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158291+0000", 2024-09-21T14:59:22.101 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:22.101 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.101 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.101 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:22.101 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158290+0000", 2024-09-21T14:59:22.101 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:22.101 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.102 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.102 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:22.102 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158295+0000", 2024-09-21T14:59:22.102 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2399999999999998e-06 2024-09-21T14:59:22.102 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.102 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.102 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:22.102 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158312+0000", 2024-09-21T14:59:22.102 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6929999999999999e-05 2024-09-21T14:59:22.102 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.102 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.102 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:22.102 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158353+0000", 2024-09-21T14:59:22.102 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0805e-05 2024-09-21T14:59:22.102 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.103 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.103 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:22.103 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.171634+0000", 2024-09-21T14:59:22.103 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.01328085 2024-09-21T14:59:22.103 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.103 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.103 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:22.103 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.171722+0000", 2024-09-21T14:59:22.103 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8115000000000006e-05 2024-09-21T14:59:22.103 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.103 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.103 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:22.103 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.171892+0000", 2024-09-21T14:59:22.103 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000169722 2024-09-21T14:59:22.103 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:22.103 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:22.104 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:22.104 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.104 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.104 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9348 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 122880~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:22.104 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158369+0000", 2024-09-21T14:59:22.104 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.923353138000003, 2024-09-21T14:59:22.104 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.01388847, 2024-09-21T14:59:22.104 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:22.104 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:22.104 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:22.104 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:22.104 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:22.104 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9348 2024-09-21T14:59:22.104 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.104 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:22.105 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.105 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:22.105 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158369+0000", 2024-09-21T14:59:22.105 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:22.105 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.105 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.105 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:22.105 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158369+0000", 2024-09-21T14:59:22.105 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:22.105 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.105 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.105 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:22.105 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158368+0000", 2024-09-21T14:59:22.105 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:22.105 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.105 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.105 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:22.106 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158374+0000", 2024-09-21T14:59:22.106 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0499999999999997e-06 2024-09-21T14:59:22.106 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.106 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.106 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:22.106 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158386+0000", 2024-09-21T14:59:22.106 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1579999999999999e-05 2024-09-21T14:59:22.106 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.106 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.106 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:22.106 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158396+0000", 2024-09-21T14:59:22.106 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.8260000000000005e-06 2024-09-21T14:59:22.106 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.106 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.106 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:22.107 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.171967+0000", 2024-09-21T14:59:22.107 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013571318000000001 2024-09-21T14:59:22.107 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.107 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.107 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:22.107 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172034+0000", 2024-09-21T14:59:22.107 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.656e-05 2024-09-21T14:59:22.107 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.107 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.107 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:22.107 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172258+0000", 2024-09-21T14:59:22.107 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00022395999999999999 2024-09-21T14:59:22.107 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:22.107 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:22.107 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:22.107 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.107 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.108 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9349 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 155648~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:22.108 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158403+0000", 2024-09-21T14:59:22.108 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.923319392000003, 2024-09-21T14:59:22.108 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014229851, 2024-09-21T14:59:22.108 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:22.108 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:22.108 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:22.108 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:22.108 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:22.108 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9349 2024-09-21T14:59:22.108 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.108 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:22.109 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.109 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:22.109 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158403+0000", 2024-09-21T14:59:22.109 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:22.109 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.109 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.109 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:22.109 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158403+0000", 2024-09-21T14:59:22.109 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:22.109 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.109 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.109 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:22.109 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158402+0000", 2024-09-21T14:59:22.109 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:22.109 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.110 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.110 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:22.110 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158406+0000", 2024-09-21T14:59:22.110 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1339999999999997e-06 2024-09-21T14:59:22.110 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.110 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.110 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:22.110 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158416+0000", 2024-09-21T14:59:22.110 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3149999999999998e-06 2024-09-21T14:59:22.110 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.110 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.110 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:22.110 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158439+0000", 2024-09-21T14:59:22.110 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3255999999999999e-05 2024-09-21T14:59:22.111 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.111 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.111 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:22.111 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172305+0000", 2024-09-21T14:59:22.111 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013866603999999999 2024-09-21T14:59:22.111 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.111 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.111 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:22.111 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172413+0000", 2024-09-21T14:59:22.111 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000107664 2024-09-21T14:59:22.111 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.111 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.111 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:22.111 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172633+0000", 2024-09-21T14:59:22.112 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021950800000000001 2024-09-21T14:59:22.112 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:22.112 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:22.112 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:22.112 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.112 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.112 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9350 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 294912~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:22.112 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158448+0000", 2024-09-21T14:59:22.112 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.923274521000003, 2024-09-21T14:59:22.112 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014471525000000001, 2024-09-21T14:59:22.112 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:22.112 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:22.112 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:22.112 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:22.113 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:22.113 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9350 2024-09-21T14:59:22.113 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.113 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:22.113 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.113 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:22.113 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158448+0000", 2024-09-21T14:59:22.113 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:22.113 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.113 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.113 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:22.113 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158448+0000", 2024-09-21T14:59:22.113 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:22.113 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.113 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.114 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:22.114 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158447+0000", 2024-09-21T14:59:22.114 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:22.114 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.114 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.114 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:22.114 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158452+0000", 2024-09-21T14:59:22.114 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8400000000000002e-06 2024-09-21T14:59:22.114 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.114 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.114 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:22.114 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158475+0000", 2024-09-21T14:59:22.114 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3091e-05 2024-09-21T14:59:22.114 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.115 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.115 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:22.115 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158515+0000", 2024-09-21T14:59:22.115 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0265000000000002e-05 2024-09-21T14:59:22.115 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.115 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.115 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:22.115 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172682+0000", 2024-09-21T14:59:22.115 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014166761999999999 2024-09-21T14:59:22.115 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.115 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.115 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:22.115 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172752+0000", 2024-09-21T14:59:22.115 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9698999999999998e-05 2024-09-21T14:59:22.116 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.116 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.116 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:22.116 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172919+0000", 2024-09-21T14:59:22.116 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016759199999999999 2024-09-21T14:59:22.116 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:22.116 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:22.116 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:22.116 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.116 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.116 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9351 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 434176~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:22.116 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158526+0000", 2024-09-21T14:59:22.116 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.923195802999999, 2024-09-21T14:59:22.117 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014662107000000001, 2024-09-21T14:59:22.117 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:22.117 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:22.117 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:22.117 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:22.117 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:22.117 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9351 2024-09-21T14:59:22.117 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.117 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:22.117 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.117 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:22.117 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158526+0000", 2024-09-21T14:59:22.117 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:22.118 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.118 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.118 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:22.118 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158526+0000", 2024-09-21T14:59:22.118 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:22.118 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.118 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.118 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:22.118 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158525+0000", 2024-09-21T14:59:22.118 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:22.118 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.118 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.118 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:22.118 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158531+0000", 2024-09-21T14:59:22.118 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5840000000000002e-06 2024-09-21T14:59:22.119 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.119 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.119 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:22.119 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158564+0000", 2024-09-21T14:59:22.119 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3195000000000003e-05 2024-09-21T14:59:22.119 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.119 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.119 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:22.119 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158577+0000", 2024-09-21T14:59:22.119 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2330000000000001e-05 2024-09-21T14:59:22.119 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.119 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.119 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:22.119 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172956+0000", 2024-09-21T14:59:22.120 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014379438 2024-09-21T14:59:22.120 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.120 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.120 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:22.120 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173020+0000", 2024-09-21T14:59:22.120 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4401999999999998e-05 2024-09-21T14:59:22.120 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.120 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.120 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:22.120 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173188+0000", 2024-09-21T14:59:22.120 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016804 2024-09-21T14:59:22.120 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:22.120 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:22.120 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:22.121 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.121 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.121 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9379 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 24576~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:22.121 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158586+0000", 2024-09-21T14:59:22.121 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.923136542999998, 2024-09-21T14:59:22.121 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014865173000000001, 2024-09-21T14:59:22.121 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:22.121 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:22.121 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:22.121 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:22.121 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:22.121 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9379 2024-09-21T14:59:22.121 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.121 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:22.121 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.121 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:22.122 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158586+0000", 2024-09-21T14:59:22.122 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:22.122 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.122 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.122 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:22.122 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158586+0000", 2024-09-21T14:59:22.122 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:22.122 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.122 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.122 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:22.122 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158585+0000", 2024-09-21T14:59:22.122 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:22.122 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.123 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.123 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:22.123 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158590+0000", 2024-09-21T14:59:22.123 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5820000000000001e-06 2024-09-21T14:59:22.123 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.123 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.123 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:22.123 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158676+0000", 2024-09-21T14:59:22.123 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5075999999999997e-05 2024-09-21T14:59:22.123 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.123 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.123 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:22.123 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158695+0000", 2024-09-21T14:59:22.123 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9137999999999999e-05 2024-09-21T14:59:22.124 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.124 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.124 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:22.124 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173219+0000", 2024-09-21T14:59:22.124 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014524034999999999 2024-09-21T14:59:22.124 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.124 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.124 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:22.124 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173276+0000", 2024-09-21T14:59:22.124 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7442000000000003e-05 2024-09-21T14:59:22.124 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.124 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.124 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:22.124 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173451+0000", 2024-09-21T14:59:22.124 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017464800000000001 2024-09-21T14:59:22.125 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:22.125 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:22.125 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:22.125 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.125 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.125 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9380 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 163840~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:22.125 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158706+0000", 2024-09-21T14:59:22.125 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.923016033000003, 2024-09-21T14:59:22.125 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015096381000000001, 2024-09-21T14:59:22.125 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:22.125 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:22.125 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:22.125 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:22.125 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:22.126 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9380 2024-09-21T14:59:22.126 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.126 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:22.126 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.126 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:22.126 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158706+0000", 2024-09-21T14:59:22.126 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:22.126 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.126 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.126 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:22.126 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158706+0000", 2024-09-21T14:59:22.126 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:22.126 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.126 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.126 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:22.127 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158705+0000", 2024-09-21T14:59:22.127 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:22.127 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.127 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.127 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:22.127 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158712+0000", 2024-09-21T14:59:22.127 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7240000000000003e-06 2024-09-21T14:59:22.127 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.127 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.127 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:22.127 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158755+0000", 2024-09-21T14:59:22.127 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2953e-05 2024-09-21T14:59:22.127 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.127 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.127 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:22.128 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158765+0000", 2024-09-21T14:59:22.128 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0149000000000001e-05 2024-09-21T14:59:22.128 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.128 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.128 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:22.128 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173492+0000", 2024-09-21T14:59:22.128 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014727105000000001 2024-09-21T14:59:22.128 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.128 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.128 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:22.128 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173564+0000", 2024-09-21T14:59:22.128 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1789e-05 2024-09-21T14:59:22.128 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.128 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.128 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:22.129 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173803+0000", 2024-09-21T14:59:22.129 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00023867100000000001 2024-09-21T14:59:22.129 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:22.129 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:22.129 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:22.129 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.129 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.129 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9381 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 303104~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:22.129 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158774+0000", 2024-09-21T14:59:22.129 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.922947952999998, 2024-09-21T14:59:22.129 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015309095999999999, 2024-09-21T14:59:22.129 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:22.129 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:22.129 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:22.129 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:22.129 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:22.130 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9381 2024-09-21T14:59:22.130 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.130 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:22.130 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.130 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:22.130 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158774+0000", 2024-09-21T14:59:22.130 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:22.130 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.130 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.130 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:22.130 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158774+0000", 2024-09-21T14:59:22.130 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:22.130 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.130 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.130 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:22.131 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158773+0000", 2024-09-21T14:59:22.131 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:22.131 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.131 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.131 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:22.131 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158779+0000", 2024-09-21T14:59:22.131 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6999999999999996e-06 2024-09-21T14:59:22.131 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.131 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.131 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:22.131 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158790+0000", 2024-09-21T14:59:22.131 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0594000000000001e-05 2024-09-21T14:59:22.131 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.132 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.132 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:22.132 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158798+0000", 2024-09-21T14:59:22.132 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5860000000000005e-06 2024-09-21T14:59:22.132 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.132 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.132 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:22.132 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173829+0000", 2024-09-21T14:59:22.132 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015030349 2024-09-21T14:59:22.132 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.132 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.132 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:22.132 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173918+0000", 2024-09-21T14:59:22.132 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9375000000000004e-05 2024-09-21T14:59:22.132 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.132 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.133 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:22.133 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.174083+0000", 2024-09-21T14:59:22.133 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016536200000000001 2024-09-21T14:59:22.133 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:22.133 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:22.133 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:22.133 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.133 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.133 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9382 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 442368~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:22.133 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158805+0000", 2024-09-21T14:59:22.133 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.922917048000002, 2024-09-21T14:59:22.133 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015592623, 2024-09-21T14:59:22.133 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:22.133 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:22.133 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:22.134 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:22.134 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:22.134 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9382 2024-09-21T14:59:22.134 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.134 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:22.134 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.134 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:22.134 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158805+0000", 2024-09-21T14:59:22.134 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:22.134 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.134 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.134 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:22.135 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158805+0000", 2024-09-21T14:59:22.135 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:22.135 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.135 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.135 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:22.135 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158805+0000", 2024-09-21T14:59:22.135 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:22.135 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.135 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.135 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:22.135 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158808+0000", 2024-09-21T14:59:22.135 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9179999999999996e-06 2024-09-21T14:59:22.135 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.135 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.136 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:22.136 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158817+0000", 2024-09-21T14:59:22.136 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9299999999999992e-06 2024-09-21T14:59:22.136 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.136 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.136 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:22.136 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158825+0000", 2024-09-21T14:59:22.136 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6739999999999997e-06 2024-09-21T14:59:22.136 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.136 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.136 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:22.136 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.174122+0000", 2024-09-21T14:59:22.136 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.01529694 2024-09-21T14:59:22.136 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.137 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.137 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:22.137 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.174185+0000", 2024-09-21T14:59:22.137 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3015000000000006e-05 2024-09-21T14:59:22.137 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.137 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.137 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:22.137 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.174398+0000", 2024-09-21T14:59:22.137 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021274799999999999 2024-09-21T14:59:22.137 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:22.137 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:22.137 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:22.137 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.137 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.137 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22699 3.19 3:98de45b8:::rbd_data.11366acc894.0000000000000000:19 [read 638976~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:22.138 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.140173+0000", 2024-09-21T14:59:22.138 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 31.941548951000001, 2024-09-21T14:59:22.138 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013511366, 2024-09-21T14:59:22.138 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:22.138 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:22.138 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:22.138 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:22.138 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:22.138 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22699 2024-09-21T14:59:22.138 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.138 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:22.138 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.138 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:22.138 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140173+0000", 2024-09-21T14:59:22.139 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:22.139 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.139 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.139 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:22.139 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140173+0000", 2024-09-21T14:59:22.139 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:22.139 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.139 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.139 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:22.139 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140173+0000", 2024-09-21T14:59:22.139 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:22.139 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.139 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.140 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:22.140 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140175+0000", 2024-09-21T14:59:22.140 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5859999999999999e-06 2024-09-21T14:59:22.140 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.140 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.140 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:22.140 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140181+0000", 2024-09-21T14:59:22.140 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0340000000000002e-06 2024-09-21T14:59:22.140 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.140 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.140 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:22.140 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140186+0000", 2024-09-21T14:59:22.140 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.04e-06 2024-09-21T14:59:22.140 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.141 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.141 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:22.141 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.153426+0000", 2024-09-21T14:59:22.141 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013240001 2024-09-21T14:59:22.141 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.141 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.141 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:22.141 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.153513+0000", 2024-09-21T14:59:22.141 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6038999999999994e-05 2024-09-21T14:59:22.141 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.141 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.141 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:22.141 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.153685+0000", 2024-09-21T14:59:22.141 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000172067 2024-09-21T14:59:22.141 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:22.142 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:22.142 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:22.142 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.142 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.142 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22700 3.19 3:98de45b8:::rbd_data.11366acc894.0000000000000000:19 [read 704512~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:22.142 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.140191+0000", 2024-09-21T14:59:22.142 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 31.941531434000002, 2024-09-21T14:59:22.142 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013770486, 2024-09-21T14:59:22.142 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:22.142 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:22.142 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:22.142 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:22.142 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:22.142 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22700 2024-09-21T14:59:22.143 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.143 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:22.143 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.143 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:22.143 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140191+0000", 2024-09-21T14:59:22.143 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:22.143 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.143 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.143 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:22.143 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140191+0000", 2024-09-21T14:59:22.143 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:22.143 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.143 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.143 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:22.143 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140190+0000", 2024-09-21T14:59:22.144 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:22.144 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.144 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.144 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:22.144 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140199+0000", 2024-09-21T14:59:22.144 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3340000000000002e-06 2024-09-21T14:59:22.144 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.144 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.144 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:22.144 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140205+0000", 2024-09-21T14:59:22.144 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3820000000000001e-06 2024-09-21T14:59:22.144 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.144 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.144 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:22.144 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140210+0000", 2024-09-21T14:59:22.144 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0969999999999996e-06 2024-09-21T14:59:22.145 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.145 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.145 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:22.145 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.153713+0000", 2024-09-21T14:59:22.145 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013502713 2024-09-21T14:59:22.145 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.145 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.145 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:22.145 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.153786+0000", 2024-09-21T14:59:22.145 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3022999999999995e-05 2024-09-21T14:59:22.145 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.145 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.145 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:22.145 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.153961+0000", 2024-09-21T14:59:22.145 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000175274 2024-09-21T14:59:22.145 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:22.146 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:22.146 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:22.146 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.146 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.146 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22701 3.19 3:98de45b8:::rbd_data.11366acc894.0000000000000000:19 [read 770048~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:22.146 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.140215+0000", 2024-09-21T14:59:22.146 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 31.941507616999999, 2024-09-21T14:59:22.146 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014072497, 2024-09-21T14:59:22.146 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:22.146 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:22.146 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:22.146 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:22.146 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:22.146 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22701 2024-09-21T14:59:22.146 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.147 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:22.147 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.147 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:22.147 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140215+0000", 2024-09-21T14:59:22.147 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:22.147 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.147 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.147 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:22.147 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140215+0000", 2024-09-21T14:59:22.147 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:22.147 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.147 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.147 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:22.147 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140214+0000", 2024-09-21T14:59:22.147 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:22.147 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.147 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.148 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:22.148 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140217+0000", 2024-09-21T14:59:22.148 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.74e-06 2024-09-21T14:59:22.148 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.148 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.148 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:22.148 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140223+0000", 2024-09-21T14:59:22.148 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0929999999999999e-06 2024-09-21T14:59:22.148 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.148 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.148 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:22.148 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140228+0000", 2024-09-21T14:59:22.148 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2279999999999998e-06 2024-09-21T14:59:22.148 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.148 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.148 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:22.149 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.153991+0000", 2024-09-21T14:59:22.149 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013762329 2024-09-21T14:59:22.149 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.149 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.149 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:22.149 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.154098+0000", 2024-09-21T14:59:22.149 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000107117 2024-09-21T14:59:22.149 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.149 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.149 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:22.149 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.154287+0000", 2024-09-21T14:59:22.149 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000189431 2024-09-21T14:59:22.149 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:22.149 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:22.149 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:22.149 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.149 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.150 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22702 3.19 3:98de45b8:::rbd_data.11366acc894.0000000000000000:19 [read 835584~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:22.150 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.140232+0000", 2024-09-21T14:59:22.150 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 31.941489765, 2024-09-21T14:59:22.150 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015338742000000001, 2024-09-21T14:59:22.150 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:22.150 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:22.150 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:22.150 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:22.150 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:22.150 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22702 2024-09-21T14:59:22.150 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.150 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:22.150 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.150 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:22.150 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140232+0000", 2024-09-21T14:59:22.151 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:22.151 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.151 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.151 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:22.151 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140232+0000", 2024-09-21T14:59:22.151 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:22.151 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.151 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.151 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:22.151 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140232+0000", 2024-09-21T14:59:22.151 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:22.151 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.151 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.151 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:22.151 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140234+0000", 2024-09-21T14:59:22.152 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.402e-06 2024-09-21T14:59:22.152 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.152 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.152 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:22.152 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140241+0000", 2024-09-21T14:59:22.152 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.054e-06 2024-09-21T14:59:22.152 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.152 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.152 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:22.152 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140246+0000", 2024-09-21T14:59:22.152 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0050000000000004e-06 2024-09-21T14:59:22.152 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.152 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.152 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:22.152 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.154315+0000", 2024-09-21T14:59:22.152 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014069440000000001 2024-09-21T14:59:22.152 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.153 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.153 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:22.153 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.154383+0000", 2024-09-21T14:59:22.153 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8151000000000005e-05 2024-09-21T14:59:22.153 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.153 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.153 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:22.153 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155571+0000", 2024-09-21T14:59:22.153 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001188029 2024-09-21T14:59:22.153 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:22.153 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:22.153 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:22.153 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.153 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.153 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22703 3.19 3:98de45b8:::rbd_data.11366acc894.0000000000000000:19 [read 901120~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:22.153 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.140250+0000", 2024-09-21T14:59:22.154 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 31.941472607000001, 2024-09-21T14:59:22.154 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014992169, 2024-09-21T14:59:22.154 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:22.154 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:22.154 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:22.154 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:22.154 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:22.154 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22703 2024-09-21T14:59:22.154 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.154 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:22.154 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.154 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:22.154 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140250+0000", 2024-09-21T14:59:22.154 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:22.155 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.155 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.155 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:22.155 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140250+0000", 2024-09-21T14:59:22.155 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:22.155 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.155 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.155 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:22.155 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140249+0000", 2024-09-21T14:59:22.155 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:22.155 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.155 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.155 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:22.155 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140251+0000", 2024-09-21T14:59:22.155 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.136e-06 2024-09-21T14:59:22.155 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.156 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.156 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:22.156 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140257+0000", 2024-09-21T14:59:22.156 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9610000000000002e-06 2024-09-21T14:59:22.156 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.156 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.156 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:22.156 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140262+0000", 2024-09-21T14:59:22.156 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4549999999999997e-06 2024-09-21T14:59:22.156 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.156 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.156 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:22.156 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155006+0000", 2024-09-21T14:59:22.156 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014744189 2024-09-21T14:59:22.156 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.156 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.156 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:22.157 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155081+0000", 2024-09-21T14:59:22.157 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4548000000000005e-05 2024-09-21T14:59:22.157 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.157 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.157 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:22.157 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155242+0000", 2024-09-21T14:59:22.157 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000161204 2024-09-21T14:59:22.157 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:22.157 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:22.157 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:22.157 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.157 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.157 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22704 3.19 3:98de45b8:::rbd_data.11366acc894.0000000000000000:19 [read 966656~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:22.157 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.140266+0000", 2024-09-21T14:59:22.157 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 31.941456290000001, 2024-09-21T14:59:22.157 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015223443999999999, 2024-09-21T14:59:22.158 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:22.158 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:22.158 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:22.158 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:22.158 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:22.158 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22704 2024-09-21T14:59:22.158 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.158 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:22.158 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.158 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:22.158 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140266+0000", 2024-09-21T14:59:22.158 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:22.158 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.158 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.158 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:22.159 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140266+0000", 2024-09-21T14:59:22.159 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:22.159 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.159 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.159 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:22.159 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140266+0000", 2024-09-21T14:59:22.159 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:22.159 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.159 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.159 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:22.159 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140268+0000", 2024-09-21T14:59:22.159 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.475e-06 2024-09-21T14:59:22.159 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.159 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.159 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:22.159 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140274+0000", 2024-09-21T14:59:22.160 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.319e-06 2024-09-21T14:59:22.160 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.160 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.160 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:22.160 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140279+0000", 2024-09-21T14:59:22.160 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.882e-06 2024-09-21T14:59:22.160 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.160 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.160 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:22.160 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155264+0000", 2024-09-21T14:59:22.160 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014984655 2024-09-21T14:59:22.160 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.160 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.160 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:22.160 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155338+0000", 2024-09-21T14:59:22.160 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4091999999999995e-05 2024-09-21T14:59:22.161 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.161 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.161 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:22.161 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155489+0000", 2024-09-21T14:59:22.161 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000151349 2024-09-21T14:59:22.161 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:22.161 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:22.161 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:22.161 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.161 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.161 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22705 3.19 3:98de45b8:::rbd_data.11366acc894.0000000000000000:19 [read 1032192~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:22.161 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.140284+0000", 2024-09-21T14:59:22.161 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 31.941438432999998, 2024-09-21T14:59:22.161 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015467896, 2024-09-21T14:59:22.161 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:22.161 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:22.162 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:22.162 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:22.162 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:22.162 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22705 2024-09-21T14:59:22.162 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.162 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:22.162 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.162 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:22.162 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140284+0000", 2024-09-21T14:59:22.162 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:22.162 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.162 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.162 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:22.162 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140284+0000", 2024-09-21T14:59:22.162 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:22.163 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.163 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.163 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:22.163 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140283+0000", 2024-09-21T14:59:22.163 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:22.163 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.163 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.163 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:22.163 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140286+0000", 2024-09-21T14:59:22.163 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7130000000000001e-06 2024-09-21T14:59:22.163 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.163 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.163 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:22.163 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140292+0000", 2024-09-21T14:59:22.163 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3430000000000001e-06 2024-09-21T14:59:22.164 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.164 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.164 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:22.164 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140298+0000", 2024-09-21T14:59:22.164 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9730000000000002e-06 2024-09-21T14:59:22.164 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.164 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.164 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:22.164 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155510+0000", 2024-09-21T14:59:22.164 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015211738000000001 2024-09-21T14:59:22.164 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.164 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.164 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:22.164 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155598+0000", 2024-09-21T14:59:22.165 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7822e-05 2024-09-21T14:59:22.165 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.165 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.165 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:22.165 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155752+0000", 2024-09-21T14:59:22.165 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0001537 2024-09-21T14:59:22.165 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:22.165 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:22.165 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:22.165 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.165 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.165 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:37892 3.1b 3:d963c97f:::rbd_data.11365d19aa81.00000000000000be:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-21T14:59:22.165 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:04.029946+0000", 2024-09-21T14:59:22.165 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 18.051776665999999, 2024-09-21T14:59:22.166 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.017686021999999999, 2024-09-21T14:59:22.166 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:22.166 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:22.166 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:22.166 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:22.166 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:22.166 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 37892 2024-09-21T14:59:22.166 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.166 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:22.166 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.166 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:22.166 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.029946+0000", 2024-09-21T14:59:22.166 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:22.166 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.166 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.166 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:22.167 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.029946+0000", 2024-09-21T14:59:22.167 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:22.167 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.167 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.167 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:22.167 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.029944+0000", 2024-09-21T14:59:22.167 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T14:59:22.167 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.167 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.167 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:22.167 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.029953+0000", 2024-09-21T14:59:22.167 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7589999999999992e-06 2024-09-21T14:59:22.167 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.167 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.167 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:22.167 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.029971+0000", 2024-09-21T14:59:22.168 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7795000000000002e-05 2024-09-21T14:59:22.168 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.168 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.168 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:22.168 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.029984+0000", 2024-09-21T14:59:22.168 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.27e-05 2024-09-21T14:59:22.168 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.168 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.168 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:22.168 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.030072+0000", 2024-09-21T14:59:22.168 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8082000000000001e-05 2024-09-21T14:59:22.168 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.168 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.168 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:22.168 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.030276+0000", 2024-09-21T14:59:22.168 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00020375699999999999 2024-09-21T14:59:22.168 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.169 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.169 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-21T14:59:22.169 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.031634+0000", 2024-09-21T14:59:22.169 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0013586049999999999 2024-09-21T14:59:22.169 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.169 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.169 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:59:22.169 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.037554+0000", 2024-09-21T14:59:22.169 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0059197449999999997 2024-09-21T14:59:22.169 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.169 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.169 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:22.169 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.047490+0000", 2024-09-21T14:59:22.169 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0099361680000000004 2024-09-21T14:59:22.169 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.169 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.170 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:59:22.170 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.047569+0000", 2024-09-21T14:59:22.170 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8298000000000001e-05 2024-09-21T14:59:22.170 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.170 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.170 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:22.170 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.047632+0000", 2024-09-21T14:59:22.170 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3006e-05 2024-09-21T14:59:22.170 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:22.170 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:22.170 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:22.170 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:22.170 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:22.170 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:59:22.171 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T14:59:22.172 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T14:59:22.173 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:59:22.173 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:59:22.173 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:59:22.173 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:59:22.173 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:59:22.173 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:59:22.173 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:59:22.173 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:59:22.173 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:59:22.169+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:59:22.195 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T14:59:22.272 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:59:22.267+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:59:22.282 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:59:22.282 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:59:22.283 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T14:59:22.283 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:59:22.292 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T14:59:22.373 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:59:22.373+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:59:22.442 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:59:22.443 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:59:22.443 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T14:59:22.443 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T14:59:22.443 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:59:22.452 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T14:59:22.474 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:59:22.471+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:59:22.563 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:59:22.563 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T14:59:22.563 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T14:59:22.563 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T14:59:22.563 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.563 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45834 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:22.563 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159133+0000", 2024-09-21T14:59:22.563 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 12.401183424999999, 2024-09-21T14:59:22.563 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068236884999999997, 2024-09-21T14:59:22.564 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:22.564 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:22.564 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:22.564 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:22.564 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:22.564 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45834 2024-09-21T14:59:22.564 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.564 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:22.564 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.564 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:22.564 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159133+0000", 2024-09-21T14:59:22.564 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:22.564 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.564 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.565 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:22.565 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159133+0000", 2024-09-21T14:59:22.565 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:22.565 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.565 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.565 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:22.565 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159133+0000", 2024-09-21T14:59:22.565 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:22.565 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.565 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.565 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:22.565 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159135+0000", 2024-09-21T14:59:22.565 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4500000000000001e-06 2024-09-21T14:59:22.565 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.565 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.565 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:22.566 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159138+0000", 2024-09-21T14:59:22.566 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2770000000000001e-06 2024-09-21T14:59:22.566 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.566 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.566 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:22.566 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159141+0000", 2024-09-21T14:59:22.566 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9210000000000001e-06 2024-09-21T14:59:22.566 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.566 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.566 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:22.566 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227361+0000", 2024-09-21T14:59:22.566 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068220154000000005 2024-09-21T14:59:22.566 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.566 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.566 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:22.567 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227370+0000", 2024-09-21T14:59:22.567 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3239999999999992e-06 2024-09-21T14:59:22.567 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:22.567 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:22.567 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:22.567 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.567 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.567 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45839 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:22.567 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159313+0000", 2024-09-21T14:59:22.567 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 12.401003459, 2024-09-21T14:59:22.567 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068238699, 2024-09-21T14:59:22.567 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:22.567 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:22.567 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:22.567 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:22.567 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:22.568 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45839 2024-09-21T14:59:22.568 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.568 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:22.568 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.568 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:22.568 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159313+0000", 2024-09-21T14:59:22.568 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:22.568 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.568 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.568 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:22.568 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159313+0000", 2024-09-21T14:59:22.568 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:22.568 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.568 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.568 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:22.568 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159313+0000", 2024-09-21T14:59:22.569 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:22.569 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.569 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.569 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:22.569 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159316+0000", 2024-09-21T14:59:22.569 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.033e-06 2024-09-21T14:59:22.569 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.569 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.569 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:22.569 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159322+0000", 2024-09-21T14:59:22.569 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1820000000000003e-06 2024-09-21T14:59:22.569 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.569 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.569 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:22.569 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159327+0000", 2024-09-21T14:59:22.569 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.062e-06 2024-09-21T14:59:22.570 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.570 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.570 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:22.570 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227544+0000", 2024-09-21T14:59:22.570 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068217142999999994 2024-09-21T14:59:22.570 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.570 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.570 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:22.570 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227552+0000", 2024-09-21T14:59:22.570 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6260000000000003e-06 2024-09-21T14:59:22.570 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:22.570 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:22.570 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:22.570 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.571 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.571 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45840 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:22.571 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159332+0000", 2024-09-21T14:59:22.571 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 12.400985162, 2024-09-21T14:59:22.571 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068262046000000007, 2024-09-21T14:59:22.571 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:22.571 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:22.571 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:22.571 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:22.571 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:22.571 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45840 2024-09-21T14:59:22.571 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.571 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:22.571 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.572 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:22.572 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159332+0000", 2024-09-21T14:59:22.572 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:22.572 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.572 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.572 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:22.572 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159332+0000", 2024-09-21T14:59:22.572 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:22.572 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.572 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.572 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:22.572 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159331+0000", 2024-09-21T14:59:22.572 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:22.572 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.572 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.572 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:22.572 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159351+0000", 2024-09-21T14:59:22.573 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0064999999999998e-05 2024-09-21T14:59:22.573 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.573 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.573 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:22.573 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159367+0000", 2024-09-21T14:59:22.573 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5308e-05 2024-09-21T14:59:22.573 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.573 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.573 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:22.573 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159373+0000", 2024-09-21T14:59:22.573 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1840000000000004e-06 2024-09-21T14:59:22.573 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.573 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.573 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:22.573 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227584+0000", 2024-09-21T14:59:22.574 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068211401000000005 2024-09-21T14:59:22.574 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.574 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.574 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:22.574 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227594+0000", 2024-09-21T14:59:22.574 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4120000000000007e-06 2024-09-21T14:59:22.574 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:22.574 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:22.574 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:22.574 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.574 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.574 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45841 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:22.574 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159379+0000", 2024-09-21T14:59:22.574 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 12.400938006000001, 2024-09-21T14:59:22.575 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068249039999999997, 2024-09-21T14:59:22.575 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:22.575 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:22.575 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:22.575 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:22.575 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:22.575 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45841 2024-09-21T14:59:22.575 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.575 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:22.575 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.575 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:22.575 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159379+0000", 2024-09-21T14:59:22.575 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:22.575 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.575 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.576 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:22.576 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159379+0000", 2024-09-21T14:59:22.576 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:22.576 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.576 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.576 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:22.576 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159378+0000", 2024-09-21T14:59:22.576 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:22.576 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.576 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.576 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:22.576 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159382+0000", 2024-09-21T14:59:22.576 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.641e-06 2024-09-21T14:59:22.576 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.576 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.576 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:22.577 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159421+0000", 2024-09-21T14:59:22.577 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9439e-05 2024-09-21T14:59:22.577 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.577 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.577 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:22.577 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159430+0000", 2024-09-21T14:59:22.577 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8429999999999999e-06 2024-09-21T14:59:22.577 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.577 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.577 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:22.577 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227617+0000", 2024-09-21T14:59:22.577 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068187084999999995 2024-09-21T14:59:22.577 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.577 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.577 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:22.577 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227628+0000", 2024-09-21T14:59:22.578 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0507e-05 2024-09-21T14:59:22.578 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:22.578 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:22.578 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:22.578 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.578 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.578 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45844 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:22.578 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159499+0000", 2024-09-21T14:59:22.578 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 12.40081762, 2024-09-21T14:59:22.578 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068255415, 2024-09-21T14:59:22.578 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:22.578 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:22.578 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:22.578 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:22.578 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:22.578 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45844 2024-09-21T14:59:22.579 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.579 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:22.579 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.579 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:22.579 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159499+0000", 2024-09-21T14:59:22.579 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:22.579 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.579 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.579 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:22.579 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159499+0000", 2024-09-21T14:59:22.579 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:22.579 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.579 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.579 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:22.579 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159499+0000", 2024-09-21T14:59:22.579 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:22.580 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.580 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.580 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:22.580 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159503+0000", 2024-09-21T14:59:22.580 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0999999999999997e-06 2024-09-21T14:59:22.580 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.580 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.580 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:22.580 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159512+0000", 2024-09-21T14:59:22.580 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5850000000000006e-06 2024-09-21T14:59:22.580 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.580 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.580 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:22.580 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159518+0000", 2024-09-21T14:59:22.580 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0909999999999998e-06 2024-09-21T14:59:22.580 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.581 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.581 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:22.581 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227743+0000", 2024-09-21T14:59:22.581 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068224536000000002 2024-09-21T14:59:22.581 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.581 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.581 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:22.581 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227755+0000", 2024-09-21T14:59:22.581 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1659e-05 2024-09-21T14:59:22.581 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:22.581 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:22.581 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:22.581 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.581 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.581 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45845 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:22.581 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159524+0000", 2024-09-21T14:59:22.581 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 12.40079326, 2024-09-21T14:59:22.582 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068267194000000003, 2024-09-21T14:59:22.582 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:22.582 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:22.582 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:22.582 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:22.582 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:22.582 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45845 2024-09-21T14:59:22.582 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.582 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:22.582 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.582 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:22.582 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159524+0000", 2024-09-21T14:59:22.582 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:22.582 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.582 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.583 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:22.583 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159524+0000", 2024-09-21T14:59:22.583 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:22.583 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.583 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.583 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:22.583 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159523+0000", 2024-09-21T14:59:22.583 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:22.583 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.583 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.583 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:22.583 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159526+0000", 2024-09-21T14:59:22.583 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1980000000000001e-06 2024-09-21T14:59:22.583 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.583 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.584 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:22.584 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159533+0000", 2024-09-21T14:59:22.584 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.348e-06 2024-09-21T14:59:22.584 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.584 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.584 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:22.584 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159546+0000", 2024-09-21T14:59:22.584 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3464e-05 2024-09-21T14:59:22.584 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.584 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.584 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:22.584 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227783+0000", 2024-09-21T14:59:22.584 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068236926000000003 2024-09-21T14:59:22.585 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.585 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.585 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:22.585 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227791+0000", 2024-09-21T14:59:22.585 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6669999999999996e-06 2024-09-21T14:59:22.585 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:22.585 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:22.585 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:22.585 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.585 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.585 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45846 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:22.585 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159584+0000", 2024-09-21T14:59:22.585 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 12.400732617999999, 2024-09-21T14:59:22.585 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068254767999999993, 2024-09-21T14:59:22.585 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:22.586 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:22.586 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:22.586 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:22.586 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:22.586 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45846 2024-09-21T14:59:22.586 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.586 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:22.586 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.586 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:22.586 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159584+0000", 2024-09-21T14:59:22.586 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:22.586 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.586 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.586 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:22.587 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159584+0000", 2024-09-21T14:59:22.587 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:22.587 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.587 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.587 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:22.587 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159583+0000", 2024-09-21T14:59:22.587 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:22.587 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.587 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.587 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:22.587 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159588+0000", 2024-09-21T14:59:22.587 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1330000000000001e-06 2024-09-21T14:59:22.587 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.587 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.588 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:22.588 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159596+0000", 2024-09-21T14:59:22.588 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5469999999999997e-06 2024-09-21T14:59:22.588 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.588 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.588 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:22.588 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159609+0000", 2024-09-21T14:59:22.588 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2785999999999999e-05 2024-09-21T14:59:22.588 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.588 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.588 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:22.588 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227830+0000", 2024-09-21T14:59:22.588 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068220879999999998 2024-09-21T14:59:22.588 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.589 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.589 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:22.589 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227839+0000", 2024-09-21T14:59:22.589 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1239999999999994e-06 2024-09-21T14:59:22.589 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:22.589 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:22.589 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:22.589 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.589 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.589 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45847 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:22.589 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159614+0000", 2024-09-21T14:59:22.589 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 12.400702541999999, 2024-09-21T14:59:22.589 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068260639999999997, 2024-09-21T14:59:22.589 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:22.590 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:22.590 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:22.590 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:22.590 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:22.590 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45847 2024-09-21T14:59:22.590 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.590 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:22.590 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.590 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:22.590 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159614+0000", 2024-09-21T14:59:22.590 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:22.590 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.590 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.590 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:22.591 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159614+0000", 2024-09-21T14:59:22.591 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:22.591 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.591 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.591 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:22.591 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159614+0000", 2024-09-21T14:59:22.591 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:22.591 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.591 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.591 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:22.591 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159617+0000", 2024-09-21T14:59:22.591 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2909999999999999e-06 2024-09-21T14:59:22.591 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.592 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.592 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:22.592 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159623+0000", 2024-09-21T14:59:22.592 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.511e-06 2024-09-21T14:59:22.592 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.592 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.592 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:22.592 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159631+0000", 2024-09-21T14:59:22.592 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.571e-06 2024-09-21T14:59:22.592 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.592 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.592 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:22.592 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227866+0000", 2024-09-21T14:59:22.592 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068235136000000002 2024-09-21T14:59:22.593 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.593 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.593 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:22.593 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227875+0000", 2024-09-21T14:59:22.593 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7199999999999995e-06 2024-09-21T14:59:22.593 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:22.593 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:22.593 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:22.593 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.593 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.593 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45848 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:22.593 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159636+0000", 2024-09-21T14:59:22.593 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 12.400680762, 2024-09-21T14:59:22.593 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068267283999999998, 2024-09-21T14:59:22.594 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:22.594 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:22.594 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:22.594 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:22.594 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:22.594 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45848 2024-09-21T14:59:22.594 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.594 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:22.594 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.594 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:22.594 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159636+0000", 2024-09-21T14:59:22.594 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:22.594 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.594 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.594 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:22.595 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159636+0000", 2024-09-21T14:59:22.595 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:22.595 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.595 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.595 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:22.595 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159636+0000", 2024-09-21T14:59:22.595 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:22.595 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.595 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.595 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:22.595 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159638+0000", 2024-09-21T14:59:22.595 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6560000000000001e-06 2024-09-21T14:59:22.595 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.595 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.595 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:22.596 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159644+0000", 2024-09-21T14:59:22.596 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0839999999999997e-06 2024-09-21T14:59:22.596 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.596 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.596 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:22.596 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159649+0000", 2024-09-21T14:59:22.596 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7500000000000003e-06 2024-09-21T14:59:22.596 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.596 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.596 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:22.596 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227892+0000", 2024-09-21T14:59:22.596 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068243315999999998 2024-09-21T14:59:22.596 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.596 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.596 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:22.597 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227903+0000", 2024-09-21T14:59:22.597 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0835999999999999e-05 2024-09-21T14:59:22.597 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:22.597 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:22.597 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:22.597 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.597 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.597 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45849 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:22.597 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159667+0000", 2024-09-21T14:59:22.597 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 12.400649948, 2024-09-21T14:59:22.597 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068291051000000005, 2024-09-21T14:59:22.597 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:22.597 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:22.597 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:22.597 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:22.597 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:22.598 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45849 2024-09-21T14:59:22.598 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.598 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:22.598 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.598 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:22.598 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159667+0000", 2024-09-21T14:59:22.598 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:22.598 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.598 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.598 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:22.598 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159667+0000", 2024-09-21T14:59:22.598 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:22.598 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.598 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.598 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:22.598 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159666+0000", 2024-09-21T14:59:22.599 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:22.599 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.599 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.599 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:22.599 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159670+0000", 2024-09-21T14:59:22.599 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.427e-06 2024-09-21T14:59:22.599 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.599 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.599 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:22.599 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159678+0000", 2024-09-21T14:59:22.599 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9869999999999999e-06 2024-09-21T14:59:22.599 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.599 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.599 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:22.599 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159688+0000", 2024-09-21T14:59:22.599 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0631999999999999e-05 2024-09-21T14:59:22.600 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.600 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.600 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:22.600 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227950+0000", 2024-09-21T14:59:22.600 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068261798999999998 2024-09-21T14:59:22.600 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.600 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.600 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:22.600 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227958+0000", 2024-09-21T14:59:22.600 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7510000000000008e-06 2024-09-21T14:59:22.600 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:22.600 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:22.600 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:22.600 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.600 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.600 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45850 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:22.600 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159695+0000", 2024-09-21T14:59:22.601 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 12.400622237, 2024-09-21T14:59:22.601 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068302999000000003, 2024-09-21T14:59:22.601 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:22.601 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:22.601 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:22.601 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:22.601 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:22.601 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45850 2024-09-21T14:59:22.601 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.601 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:22.601 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.601 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:22.601 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159695+0000", 2024-09-21T14:59:22.601 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:22.601 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.601 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.602 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:22.602 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159695+0000", 2024-09-21T14:59:22.602 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:22.602 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.602 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.602 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:22.602 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159694+0000", 2024-09-21T14:59:22.602 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:22.602 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.602 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.602 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:22.602 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159697+0000", 2024-09-21T14:59:22.602 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.258e-06 2024-09-21T14:59:22.602 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.602 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.602 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:22.603 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159704+0000", 2024-09-21T14:59:22.603 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7660000000000001e-06 2024-09-21T14:59:22.603 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.603 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.603 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:22.603 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159713+0000", 2024-09-21T14:59:22.603 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1779999999999992e-06 2024-09-21T14:59:22.603 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.603 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.603 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:22.603 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227989+0000", 2024-09-21T14:59:22.603 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068276015999999995 2024-09-21T14:59:22.603 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.603 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.603 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:22.603 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227998+0000", 2024-09-21T14:59:22.604 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2050000000000002e-06 2024-09-21T14:59:22.604 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:22.604 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:22.604 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:22.604 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.604 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.604 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45851 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:22.604 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159719+0000", 2024-09-21T14:59:22.604 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 12.400598104, 2024-09-21T14:59:22.604 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068320327, 2024-09-21T14:59:22.604 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:22.604 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:22.604 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:22.604 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:22.604 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:22.604 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45851 2024-09-21T14:59:22.605 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.605 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:22.605 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.605 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:22.605 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159719+0000", 2024-09-21T14:59:22.605 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:22.605 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.605 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.605 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:22.605 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159719+0000", 2024-09-21T14:59:22.605 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:22.605 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.605 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.605 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:22.605 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159718+0000", 2024-09-21T14:59:22.605 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:22.606 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.606 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.606 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:22.606 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159721+0000", 2024-09-21T14:59:22.606 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2169999999999999e-06 2024-09-21T14:59:22.606 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.606 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.606 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:22.606 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159730+0000", 2024-09-21T14:59:22.606 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5720000000000003e-06 2024-09-21T14:59:22.606 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.606 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.606 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:22.606 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159735+0000", 2024-09-21T14:59:22.606 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1100000000000002e-06 2024-09-21T14:59:22.607 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.607 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.607 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:22.607 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228029+0000", 2024-09-21T14:59:22.607 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068293624999999997 2024-09-21T14:59:22.607 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.607 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.607 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:22.607 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228039+0000", 2024-09-21T14:59:22.607 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0254e-05 2024-09-21T14:59:22.607 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:22.607 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:22.607 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:22.608 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.608 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.608 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45852 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:22.608 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159740+0000", 2024-09-21T14:59:22.608 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 12.400576936, 2024-09-21T14:59:22.608 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068340260999999999, 2024-09-21T14:59:22.608 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:22.608 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:22.608 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:22.608 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:22.608 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:22.608 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45852 2024-09-21T14:59:22.608 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.608 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:22.608 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.609 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:22.609 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159740+0000", 2024-09-21T14:59:22.609 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:22.609 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.609 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.609 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:22.609 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159740+0000", 2024-09-21T14:59:22.609 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:22.609 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.609 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.609 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:22.609 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159739+0000", 2024-09-21T14:59:22.609 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:22.609 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.609 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.610 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:22.610 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159757+0000", 2024-09-21T14:59:22.610 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7314999999999999e-05 2024-09-21T14:59:22.610 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.610 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.610 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:22.610 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159769+0000", 2024-09-21T14:59:22.610 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2376e-05 2024-09-21T14:59:22.610 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.610 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.610 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:22.610 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159775+0000", 2024-09-21T14:59:22.610 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2179999999999996e-06 2024-09-21T14:59:22.611 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.611 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.611 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:22.611 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228071+0000", 2024-09-21T14:59:22.611 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068295344999999993 2024-09-21T14:59:22.611 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.611 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.611 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:22.611 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228080+0000", 2024-09-21T14:59:22.611 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3959999999999995e-06 2024-09-21T14:59:22.611 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:22.611 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:22.611 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:22.611 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.611 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.612 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45853 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:22.612 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159781+0000", 2024-09-21T14:59:22.612 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 12.400535672, 2024-09-21T14:59:22.612 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068337093000000002, 2024-09-21T14:59:22.612 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:22.612 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:22.612 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:22.612 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:22.612 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:22.612 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45853 2024-09-21T14:59:22.612 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.612 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:22.612 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.612 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:22.612 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159781+0000", 2024-09-21T14:59:22.612 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:22.613 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.613 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.613 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:22.613 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159781+0000", 2024-09-21T14:59:22.613 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:22.613 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.613 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.613 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:22.613 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159781+0000", 2024-09-21T14:59:22.613 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:22.613 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.613 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.613 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:22.613 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159784+0000", 2024-09-21T14:59:22.613 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4759999999999998e-06 2024-09-21T14:59:22.613 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.613 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.614 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:22.614 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159792+0000", 2024-09-21T14:59:22.614 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0569999999999993e-06 2024-09-21T14:59:22.614 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.614 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.614 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:22.614 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159797+0000", 2024-09-21T14:59:22.614 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9640000000000002e-06 2024-09-21T14:59:22.614 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.614 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.614 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:22.614 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228111+0000", 2024-09-21T14:59:22.614 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068313737999999999 2024-09-21T14:59:22.614 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.614 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.615 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:22.615 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228118+0000", 2024-09-21T14:59:22.615 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.43e-06 2024-09-21T14:59:22.615 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:22.615 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:22.615 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:22.615 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.615 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.615 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45854 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:22.615 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159802+0000", 2024-09-21T14:59:22.615 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 12.400515022, 2024-09-21T14:59:22.615 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068358938999999994, 2024-09-21T14:59:22.615 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:22.615 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:22.615 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:22.615 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:22.616 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:22.616 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45854 2024-09-21T14:59:22.616 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.616 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:22.616 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.616 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:22.616 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159802+0000", 2024-09-21T14:59:22.616 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:22.616 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.616 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.616 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:22.616 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159802+0000", 2024-09-21T14:59:22.616 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:22.616 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.616 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.616 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:22.617 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159801+0000", 2024-09-21T14:59:22.617 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:22.617 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.617 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.617 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:22.617 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159804+0000", 2024-09-21T14:59:22.617 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0970000000000002e-06 2024-09-21T14:59:22.617 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.617 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.617 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:22.617 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159823+0000", 2024-09-21T14:59:22.617 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8895000000000001e-05 2024-09-21T14:59:22.617 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.617 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.617 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:22.617 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159837+0000", 2024-09-21T14:59:22.618 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3247999999999999e-05 2024-09-21T14:59:22.618 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.618 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.618 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:22.618 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228151+0000", 2024-09-21T14:59:22.618 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068314642999999994 2024-09-21T14:59:22.618 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.618 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.618 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:22.618 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228161+0000", 2024-09-21T14:59:22.618 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4250000000000004e-06 2024-09-21T14:59:22.618 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:22.618 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:22.618 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:22.618 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.618 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.619 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45983 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:22.619 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159848+0000", 2024-09-21T14:59:22.619 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 12.400468762999999, 2024-09-21T14:59:22.619 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068333801, 2024-09-21T14:59:22.619 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:22.619 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:22.619 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:22.619 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:22.619 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:22.619 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45983 2024-09-21T14:59:22.619 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.619 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:22.619 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.619 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:22.619 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159848+0000", 2024-09-21T14:59:22.619 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:22.620 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.620 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.620 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:22.620 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159848+0000", 2024-09-21T14:59:22.620 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:22.620 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.620 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.620 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:22.620 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159847+0000", 2024-09-21T14:59:22.620 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:22.620 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.620 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.620 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:22.620 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159856+0000", 2024-09-21T14:59:22.620 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4349999999999997e-06 2024-09-21T14:59:22.620 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.621 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.621 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:22.621 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159866+0000", 2024-09-21T14:59:22.621 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0076e-05 2024-09-21T14:59:22.621 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.621 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.621 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:22.621 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159872+0000", 2024-09-21T14:59:22.621 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8309999999999997e-06 2024-09-21T14:59:22.621 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.621 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.621 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:22.621 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228175+0000", 2024-09-21T14:59:22.621 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068302680000000005 2024-09-21T14:59:22.621 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.621 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.621 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:22.622 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228182+0000", 2024-09-21T14:59:22.622 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3150000000000003e-06 2024-09-21T14:59:22.622 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:22.622 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:22.622 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:22.622 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.622 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.622 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45984 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:22.622 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159877+0000", 2024-09-21T14:59:22.622 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 12.400439343, 2024-09-21T14:59:22.622 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068328448, 2024-09-21T14:59:22.622 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:22.622 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:22.622 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:22.622 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:22.622 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:22.623 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45984 2024-09-21T14:59:22.623 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.623 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:22.623 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.623 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:22.623 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159877+0000", 2024-09-21T14:59:22.623 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:22.623 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.623 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.623 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:22.623 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159877+0000", 2024-09-21T14:59:22.623 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:22.623 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.623 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.623 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:22.623 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159877+0000", 2024-09-21T14:59:22.624 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:22.624 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.624 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.624 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:22.624 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159880+0000", 2024-09-21T14:59:22.624 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7610000000000001e-06 2024-09-21T14:59:22.624 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.624 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.624 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:22.624 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159888+0000", 2024-09-21T14:59:22.624 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0330000000000003e-06 2024-09-21T14:59:22.624 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.624 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.624 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:22.624 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159893+0000", 2024-09-21T14:59:22.625 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3550000000000004e-06 2024-09-21T14:59:22.625 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.625 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.625 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:22.625 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228199+0000", 2024-09-21T14:59:22.625 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068306438999999997 2024-09-21T14:59:22.625 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.625 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.625 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:22.625 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228206+0000", 2024-09-21T14:59:22.625 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5599999999999999e-06 2024-09-21T14:59:22.625 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:22.625 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:22.625 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:22.625 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.625 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.626 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45985 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:22.626 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159897+0000", 2024-09-21T14:59:22.626 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 12.40041952, 2024-09-21T14:59:22.626 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068345664, 2024-09-21T14:59:22.626 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:22.626 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:22.626 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:22.626 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:22.626 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:22.626 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45985 2024-09-21T14:59:22.626 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.626 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:22.626 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.626 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:22.626 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159897+0000", 2024-09-21T14:59:22.627 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:22.627 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.627 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.627 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:22.627 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159897+0000", 2024-09-21T14:59:22.627 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:22.627 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.627 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.627 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:22.627 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159897+0000", 2024-09-21T14:59:22.627 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:22.627 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.627 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.627 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:22.627 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159900+0000", 2024-09-21T14:59:22.627 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.954e-06 2024-09-21T14:59:22.627 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.628 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.628 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:22.628 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159913+0000", 2024-09-21T14:59:22.628 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3106e-05 2024-09-21T14:59:22.628 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.628 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.628 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:22.628 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159918+0000", 2024-09-21T14:59:22.628 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5199999999999997e-06 2024-09-21T14:59:22.628 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.628 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.628 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:22.628 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228236+0000", 2024-09-21T14:59:22.628 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068317040999999995 2024-09-21T14:59:22.628 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.628 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.629 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:22.629 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228243+0000", 2024-09-21T14:59:22.629 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3880000000000002e-06 2024-09-21T14:59:22.629 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:22.629 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:22.629 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:22.629 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.629 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.629 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45986 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:22.629 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159924+0000", 2024-09-21T14:59:22.629 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 12.400393225, 2024-09-21T14:59:22.629 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068364362999999997, 2024-09-21T14:59:22.629 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:22.629 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:22.629 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:22.629 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:22.630 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:22.630 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45986 2024-09-21T14:59:22.630 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.630 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:22.630 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.630 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:22.630 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159924+0000", 2024-09-21T14:59:22.630 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:22.630 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.630 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.630 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:22.630 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159924+0000", 2024-09-21T14:59:22.630 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:22.630 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.630 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.631 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:22.631 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159923+0000", 2024-09-21T14:59:22.631 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:22.631 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.631 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.631 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:22.631 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159926+0000", 2024-09-21T14:59:22.631 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0510000000000001e-06 2024-09-21T14:59:22.631 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.631 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.631 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:22.631 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159932+0000", 2024-09-21T14:59:22.631 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4160000000000001e-06 2024-09-21T14:59:22.631 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.632 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.632 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:22.632 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159938+0000", 2024-09-21T14:59:22.632 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.383e-06 2024-09-21T14:59:22.632 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.632 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.632 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:22.632 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228275+0000", 2024-09-21T14:59:22.632 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068337007000000005 2024-09-21T14:59:22.632 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.632 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.632 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:22.632 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228288+0000", 2024-09-21T14:59:22.632 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3052e-05 2024-09-21T14:59:22.632 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:22.632 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:22.633 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:22.633 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.633 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.633 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45987 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:22.633 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159944+0000", 2024-09-21T14:59:22.633 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 12.400373001, 2024-09-21T14:59:22.633 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068387232000000006, 2024-09-21T14:59:22.633 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:22.633 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:22.633 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:22.633 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:22.633 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:22.633 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45987 2024-09-21T14:59:22.633 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.633 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:22.634 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.634 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:22.634 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159944+0000", 2024-09-21T14:59:22.634 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:22.634 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.634 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.634 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:22.634 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159944+0000", 2024-09-21T14:59:22.634 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:22.634 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.634 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.634 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:22.634 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159943+0000", 2024-09-21T14:59:22.634 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:22.635 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.635 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.635 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:22.635 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159946+0000", 2024-09-21T14:59:22.635 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6309999999999999e-06 2024-09-21T14:59:22.635 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.635 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.635 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:22.635 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159952+0000", 2024-09-21T14:59:22.635 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6540000000000004e-06 2024-09-21T14:59:22.635 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.635 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.635 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:22.635 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159957+0000", 2024-09-21T14:59:22.636 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9280000000000001e-06 2024-09-21T14:59:22.636 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.636 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.636 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:22.636 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228322+0000", 2024-09-21T14:59:22.636 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068365067000000002 2024-09-21T14:59:22.636 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:22.636 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:22.636 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:22.636 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228331+0000", 2024-09-21T14:59:22.636 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3270000000000007e-06 2024-09-21T14:59:22.636 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:22.636 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:22.636 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:22.636 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:22.636 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:22.636 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:59:22.638 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T14:59:22.638 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:59:22.635+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:59:22.738 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:59:22.735+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:59:22.794 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:59:22.795 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:59:22.795 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T14:59:22.795 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:59:22.805 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T14:59:22.839 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:59:22.837+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:59:22.909 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:59:22.909 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:59:22.909 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T14:59:22.909 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T14:59:22.909 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:59:22.919 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T14:59:22.940 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:59:22.941+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:59:23.029 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:59:23.029 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T14:59:23.029 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T14:59:23.029 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T14:59:23.029 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.029 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:10651 3.12 3:4e9d23af:::rbd_data.1136f5ed5dd6.0000000000000020:head [write 0~81920 [fadvise_sequential+fadvise_nocache],write 90112~8192 [fadvise_sequential+fadvise_nocache],write 122880~122880 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:23.029 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.292476+0000", 2024-09-21T14:59:23.030 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 43.735675974000003, 2024-09-21T14:59:23.030 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012765669, 2024-09-21T14:59:23.030 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:23.030 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:23.030 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:23.030 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:23.030 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:23.030 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 10651 2024-09-21T14:59:23.030 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.030 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:23.030 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.030 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:23.030 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292476+0000", 2024-09-21T14:59:23.030 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:23.030 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.031 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.031 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:23.031 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292476+0000", 2024-09-21T14:59:23.031 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:23.031 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.031 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.031 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:23.031 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292475+0000", 2024-09-21T14:59:23.031 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:23.031 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.031 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.031 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:23.031 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292696+0000", 2024-09-21T14:59:23.031 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000220201 2024-09-21T14:59:23.032 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.032 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.032 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:23.032 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292711+0000", 2024-09-21T14:59:23.032 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5803000000000001e-05 2024-09-21T14:59:23.032 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.032 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.032 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:23.032 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292721+0000", 2024-09-21T14:59:23.032 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6160000000000008e-06 2024-09-21T14:59:23.032 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.032 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.033 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:23.033 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292757+0000", 2024-09-21T14:59:23.033 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6211999999999998e-05 2024-09-21T14:59:23.033 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.033 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.033 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:23.033 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292878+0000", 2024-09-21T14:59:23.033 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000120306 2024-09-21T14:59:23.033 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.033 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.033 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 3", 2024-09-21T14:59:23.033 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.293019+0000", 2024-09-21T14:59:23.033 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014186400000000001 2024-09-21T14:59:23.033 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.033 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.034 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:59:23.034 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.294898+0000", 2024-09-21T14:59:23.034 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001878434 2024-09-21T14:59:23.034 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.034 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.034 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:23.034 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.305162+0000", 2024-09-21T14:59:23.034 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.01026373 2024-09-21T14:59:23.034 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.034 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.034 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:59:23.034 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.305209+0000", 2024-09-21T14:59:23.034 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7615999999999999e-05 2024-09-21T14:59:23.035 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.035 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.035 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:23.035 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.305242+0000", 2024-09-21T14:59:23.035 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2979999999999999e-05 2024-09-21T14:59:23.035 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:23.035 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:23.035 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:23.035 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.035 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.035 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:11285 3.12 3:4edf73d2:::rbd_data.1136f5ed5dd6.0000000000000039:head [write 90112~32768 [fadvise_sequential+fadvise_nocache],write 139264~24576 [fadvise_sequential+fadvise_nocache],write 180224~24576 [fadvise_sequential+fadvise_nocache],write 221184~40960 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:23.035 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.539470+0000", 2024-09-21T14:59:23.035 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 43.488682222000001, 2024-09-21T14:59:23.035 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0083360099999999996, 2024-09-21T14:59:23.036 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:23.036 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:23.036 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:23.036 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:23.036 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:23.036 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 11285 2024-09-21T14:59:23.036 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.036 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:23.036 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.036 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:23.036 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.539470+0000", 2024-09-21T14:59:23.036 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:23.036 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.036 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.036 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:23.037 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.539470+0000", 2024-09-21T14:59:23.037 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:23.037 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.037 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.037 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:23.037 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.539469+0000", 2024-09-21T14:59:23.037 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:23.037 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.037 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.037 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:23.037 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.539652+0000", 2024-09-21T14:59:23.037 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000183084 2024-09-21T14:59:23.037 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.037 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.037 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:23.037 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.539666+0000", 2024-09-21T14:59:23.038 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3947e-05 2024-09-21T14:59:23.038 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.038 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.038 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:23.038 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.539673+0000", 2024-09-21T14:59:23.038 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7309999999999996e-06 2024-09-21T14:59:23.038 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.038 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.038 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:23.038 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.539714+0000", 2024-09-21T14:59:23.038 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1539000000000003e-05 2024-09-21T14:59:23.038 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.038 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.038 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:23.038 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.539832+0000", 2024-09-21T14:59:23.039 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011753500000000001 2024-09-21T14:59:23.039 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.039 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.039 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 3", 2024-09-21T14:59:23.039 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.540005+0000", 2024-09-21T14:59:23.039 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000173031 2024-09-21T14:59:23.039 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.039 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.039 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:59:23.039 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.541606+0000", 2024-09-21T14:59:23.039 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001600899 2024-09-21T14:59:23.039 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.039 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.039 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:23.039 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.547709+0000", 2024-09-21T14:59:23.039 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0061027110000000002 2024-09-21T14:59:23.040 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.040 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.040 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:59:23.040 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.547767+0000", 2024-09-21T14:59:23.040 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8248999999999997e-05 2024-09-21T14:59:23.040 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.040 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.040 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:23.040 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.547806+0000", 2024-09-21T14:59:23.040 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9341000000000002e-05 2024-09-21T14:59:23.040 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:23.040 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:23.040 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:23.040 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.040 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.040 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12189 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 40960~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:23.041 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.618667+0000", 2024-09-21T14:59:23.041 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 43.409485771999996, 2024-09-21T14:59:23.041 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0083014549999999993, 2024-09-21T14:59:23.041 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:23.041 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:23.041 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:23.041 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:23.041 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:23.041 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12189 2024-09-21T14:59:23.041 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.041 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:23.041 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.041 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:23.041 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618667+0000", 2024-09-21T14:59:23.041 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:23.041 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.042 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.042 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:23.042 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618667+0000", 2024-09-21T14:59:23.042 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:23.042 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.042 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.042 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:23.042 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618666+0000", 2024-09-21T14:59:23.042 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:23.042 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.042 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.042 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:23.042 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618669+0000", 2024-09-21T14:59:23.042 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.554e-06 2024-09-21T14:59:23.043 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.043 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.043 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:23.043 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618675+0000", 2024-09-21T14:59:23.043 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2570000000000004e-06 2024-09-21T14:59:23.043 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.043 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.043 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:23.043 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618680+0000", 2024-09-21T14:59:23.043 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6539999999999998e-06 2024-09-21T14:59:23.043 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.043 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.043 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:23.043 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.626683+0000", 2024-09-21T14:59:23.043 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0080035880000000007 2024-09-21T14:59:23.044 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.044 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.044 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:23.045 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.626764+0000", 2024-09-21T14:59:23.045 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0672000000000003e-05 2024-09-21T14:59:23.045 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.045 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.045 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:23.045 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.626968+0000", 2024-09-21T14:59:23.046 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00020407499999999999 2024-09-21T14:59:23.046 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:23.046 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:23.046 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:23.046 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.046 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.046 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12193 3.4 3:25437d51:::rbd_data.11363a21d2be.0000000000000025:10 [read 425984~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:23.046 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.618755+0000", 2024-09-21T14:59:23.046 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 43.409397691000002, 2024-09-21T14:59:23.046 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086461379999999994, 2024-09-21T14:59:23.046 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:23.046 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:23.046 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:23.046 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:23.046 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:23.047 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12193 2024-09-21T14:59:23.047 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.047 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:23.047 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.047 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:23.047 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618755+0000", 2024-09-21T14:59:23.047 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:23.047 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.047 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.047 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:23.047 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618755+0000", 2024-09-21T14:59:23.047 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:23.047 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.048 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.048 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:23.048 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618754+0000", 2024-09-21T14:59:23.048 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:23.048 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.048 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.048 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:23.048 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618759+0000", 2024-09-21T14:59:23.048 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4020000000000004e-06 2024-09-21T14:59:23.048 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.048 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.048 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:23.048 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618767+0000", 2024-09-21T14:59:23.048 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7490000000000004e-06 2024-09-21T14:59:23.049 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.049 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.049 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:23.049 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618773+0000", 2024-09-21T14:59:23.049 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0939999999999996e-06 2024-09-21T14:59:23.049 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.049 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.049 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:23.049 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.627115+0000", 2024-09-21T14:59:23.049 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0083416840000000003 2024-09-21T14:59:23.049 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.049 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.049 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:23.049 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.627214+0000", 2024-09-21T14:59:23.049 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9042000000000005e-05 2024-09-21T14:59:23.049 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.050 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.050 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:23.050 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.627401+0000", 2024-09-21T14:59:23.050 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018677200000000001 2024-09-21T14:59:23.050 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:23.050 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:23.050 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:23.050 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.050 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.050 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12212 3.4 3:21b43b45:::rbd_data.11363a21d2be.000000000000002b:10 [read 237568~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:23.050 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.619922+0000", 2024-09-21T14:59:23.050 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 43.408230707000001, 2024-09-21T14:59:23.050 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.008324372, 2024-09-21T14:59:23.050 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:23.051 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:23.051 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:23.051 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:23.051 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:23.051 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12212 2024-09-21T14:59:23.051 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.051 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:23.051 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.051 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:23.051 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619922+0000", 2024-09-21T14:59:23.051 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:23.051 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.051 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.051 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:23.052 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619922+0000", 2024-09-21T14:59:23.052 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:23.052 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.052 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.052 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:23.052 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619921+0000", 2024-09-21T14:59:23.052 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:23.052 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.052 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.052 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:23.052 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619924+0000", 2024-09-21T14:59:23.052 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.999e-06 2024-09-21T14:59:23.052 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.052 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.053 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:23.053 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619934+0000", 2024-09-21T14:59:23.053 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.8120000000000002e-06 2024-09-21T14:59:23.053 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.053 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.053 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:23.053 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619939+0000", 2024-09-21T14:59:23.053 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7729999999999999e-06 2024-09-21T14:59:23.053 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.053 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.053 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:23.053 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628005+0000", 2024-09-21T14:59:23.053 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0080663330000000002 2024-09-21T14:59:23.054 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.054 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.054 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:23.054 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628073+0000", 2024-09-21T14:59:23.054 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7479999999999998e-05 2024-09-21T14:59:23.054 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.054 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.054 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:23.054 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628246+0000", 2024-09-21T14:59:23.054 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017337400000000001 2024-09-21T14:59:23.054 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:23.054 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:23.054 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:23.054 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.055 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.055 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12213 3.4 3:21b43b45:::rbd_data.11363a21d2be.000000000000002b:10 [read 270336~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:23.055 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.619945+0000", 2024-09-21T14:59:23.055 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 43.408207425000001, 2024-09-21T14:59:23.055 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086195049999999995, 2024-09-21T14:59:23.055 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:23.055 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:23.055 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:23.055 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:23.055 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:23.055 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12213 2024-09-21T14:59:23.055 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.055 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:23.056 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.056 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:23.056 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619945+0000", 2024-09-21T14:59:23.056 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:23.056 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.056 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.056 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:23.056 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619945+0000", 2024-09-21T14:59:23.056 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:23.056 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.056 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.056 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:23.056 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619945+0000", 2024-09-21T14:59:23.057 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:23.057 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.057 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.057 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:23.057 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619947+0000", 2024-09-21T14:59:23.057 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5780000000000001e-06 2024-09-21T14:59:23.057 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.057 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.057 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:23.057 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619957+0000", 2024-09-21T14:59:23.057 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3500000000000003e-06 2024-09-21T14:59:23.057 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.057 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.057 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:23.057 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619964+0000", 2024-09-21T14:59:23.058 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.199e-06 2024-09-21T14:59:23.058 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.058 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.058 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:23.058 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628285+0000", 2024-09-21T14:59:23.058 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0083210939999999994 2024-09-21T14:59:23.058 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.058 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.058 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:23.058 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628361+0000", 2024-09-21T14:59:23.058 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5873999999999999e-05 2024-09-21T14:59:23.058 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.058 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.058 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:23.059 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628565+0000", 2024-09-21T14:59:23.059 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000203799 2024-09-21T14:59:23.059 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:23.059 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:23.059 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:23.059 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.059 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.059 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12214 3.4 3:21b43b45:::rbd_data.11363a21d2be.000000000000002b:10 [read 376832~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:23.059 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.619969+0000", 2024-09-21T14:59:23.059 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 43.408183424000001, 2024-09-21T14:59:23.059 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0090292089999999998, 2024-09-21T14:59:23.060 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:23.060 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:23.060 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:23.060 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:23.060 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:23.060 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12214 2024-09-21T14:59:23.060 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.060 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:23.060 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.060 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:23.060 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619969+0000", 2024-09-21T14:59:23.060 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:23.060 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.060 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.061 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:23.061 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619969+0000", 2024-09-21T14:59:23.061 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:23.061 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.061 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.061 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:23.061 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619969+0000", 2024-09-21T14:59:23.061 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:23.061 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.061 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.061 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:23.061 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619971+0000", 2024-09-21T14:59:23.061 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1830000000000001e-06 2024-09-21T14:59:23.061 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.062 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.062 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:23.062 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619976+0000", 2024-09-21T14:59:23.062 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7129999999999996e-06 2024-09-21T14:59:23.062 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.062 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.062 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:23.062 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619981+0000", 2024-09-21T14:59:23.062 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2460000000000003e-06 2024-09-21T14:59:23.062 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.062 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.062 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:23.063 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628602+0000", 2024-09-21T14:59:23.063 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086209519999999994 2024-09-21T14:59:23.063 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.063 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.063 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:23.063 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628734+0000", 2024-09-21T14:59:23.063 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000132242 2024-09-21T14:59:23.063 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.063 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.063 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:23.063 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628998+0000", 2024-09-21T14:59:23.063 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00026419799999999999 2024-09-21T14:59:23.064 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:23.064 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:23.064 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:23.064 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.064 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.064 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12215 3.4 3:21b43b45:::rbd_data.11363a21d2be.000000000000002b:10 [read 409600~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:23.064 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.619985+0000", 2024-09-21T14:59:23.064 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 43.408167358, 2024-09-21T14:59:23.064 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0095457319999999995, 2024-09-21T14:59:23.064 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:23.064 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:23.064 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:23.064 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:23.064 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:23.065 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12215 2024-09-21T14:59:23.065 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.065 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:23.065 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.065 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:23.065 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619985+0000", 2024-09-21T14:59:23.065 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:23.065 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.065 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.065 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:23.065 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619985+0000", 2024-09-21T14:59:23.065 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:23.066 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.066 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.066 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:23.066 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619985+0000", 2024-09-21T14:59:23.066 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:23.066 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.066 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.066 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:23.066 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619987+0000", 2024-09-21T14:59:23.066 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6129999999999998e-06 2024-09-21T14:59:23.066 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.066 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.066 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:23.067 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619993+0000", 2024-09-21T14:59:23.067 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0820000000000004e-06 2024-09-21T14:59:23.067 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.067 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.067 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:23.067 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619998+0000", 2024-09-21T14:59:23.067 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7759999999999997e-06 2024-09-21T14:59:23.067 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.067 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.067 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:23.067 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629022+0000", 2024-09-21T14:59:23.067 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0090237489999999993 2024-09-21T14:59:23.067 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.068 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.068 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:23.068 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629098+0000", 2024-09-21T14:59:23.068 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5735999999999995e-05 2024-09-21T14:59:23.068 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.068 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.068 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:23.068 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629531+0000", 2024-09-21T14:59:23.068 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00043310100000000003 2024-09-21T14:59:23.068 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:23.068 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:23.068 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:23.068 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.068 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.069 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12216 3.4 3:21b43b45:::rbd_data.11363a21d2be.000000000000002b:10 [read 516096~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:23.069 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.620003+0000", 2024-09-21T14:59:23.069 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 43.408149362000003, 2024-09-21T14:59:23.069 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0098850169999999994, 2024-09-21T14:59:23.069 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:23.069 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:23.069 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:23.069 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:23.069 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:23.069 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12216 2024-09-21T14:59:23.069 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.069 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:23.069 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.069 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:23.070 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620003+0000", 2024-09-21T14:59:23.070 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:23.070 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.070 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.070 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:23.070 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620003+0000", 2024-09-21T14:59:23.070 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:23.070 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.070 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.070 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:23.070 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620003+0000", 2024-09-21T14:59:23.070 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:23.070 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.071 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.071 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:23.071 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620006+0000", 2024-09-21T14:59:23.071 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4350000000000001e-06 2024-09-21T14:59:23.071 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.071 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.071 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:23.071 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620013+0000", 2024-09-21T14:59:23.071 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4219999999999997e-06 2024-09-21T14:59:23.071 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.071 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.071 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:23.071 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620017+0000", 2024-09-21T14:59:23.072 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.865e-06 2024-09-21T14:59:23.072 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.072 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.072 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:23.072 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629581+0000", 2024-09-21T14:59:23.072 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0095631989999999997 2024-09-21T14:59:23.072 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.072 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.072 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:23.072 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629691+0000", 2024-09-21T14:59:23.072 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000110586 2024-09-21T14:59:23.072 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.072 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.072 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:23.072 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629888+0000", 2024-09-21T14:59:23.072 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019691199999999999 2024-09-21T14:59:23.073 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:23.073 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:23.073 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:23.073 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.073 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.073 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12222 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 49152~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:23.073 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.620027+0000", 2024-09-21T14:59:23.073 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 43.408125016, 2024-09-21T14:59:23.073 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010228131999999999, 2024-09-21T14:59:23.073 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:23.073 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:23.073 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:23.073 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:23.073 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:23.074 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12222 2024-09-21T14:59:23.074 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.074 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:23.074 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.074 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:23.074 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620027+0000", 2024-09-21T14:59:23.074 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:23.074 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.074 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.074 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:23.074 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620027+0000", 2024-09-21T14:59:23.074 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:23.074 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.075 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.075 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:23.075 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620027+0000", 2024-09-21T14:59:23.075 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:23.075 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.075 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.075 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:23.075 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620030+0000", 2024-09-21T14:59:23.075 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0400000000000001e-06 2024-09-21T14:59:23.075 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.075 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.075 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:23.075 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620037+0000", 2024-09-21T14:59:23.075 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8220000000000001e-06 2024-09-21T14:59:23.075 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.076 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.076 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:23.076 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620042+0000", 2024-09-21T14:59:23.076 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.843e-06 2024-09-21T14:59:23.076 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.076 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.076 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:23.076 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629922+0000", 2024-09-21T14:59:23.076 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0098805100000000003 2024-09-21T14:59:23.076 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.076 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.076 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:23.076 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.630020+0000", 2024-09-21T14:59:23.076 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7596999999999994e-05 2024-09-21T14:59:23.077 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.077 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.077 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:23.077 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.630256+0000", 2024-09-21T14:59:23.077 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00023573299999999999 2024-09-21T14:59:23.077 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:23.077 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:23.077 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:23.077 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.077 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.077 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12253 3.4 3:21b43b45:::rbd_data.11363a21d2be.000000000000002b:10 [read 417792~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:23.077 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.624578+0000", 2024-09-21T14:59:23.077 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 43.403574648000003, 2024-09-21T14:59:23.077 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0085112079999999993, 2024-09-21T14:59:23.077 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:23.077 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:23.078 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:23.078 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:23.078 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:23.078 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12253 2024-09-21T14:59:23.078 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.078 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:23.078 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.078 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:23.078 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624578+0000", 2024-09-21T14:59:23.078 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:23.078 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.078 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.078 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:23.078 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624578+0000", 2024-09-21T14:59:23.078 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:23.078 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.079 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.079 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:23.079 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624578+0000", 2024-09-21T14:59:23.079 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:23.079 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.079 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.079 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:23.079 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624579+0000", 2024-09-21T14:59:23.079 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.688e-06 2024-09-21T14:59:23.079 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.079 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.079 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:23.079 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624582+0000", 2024-09-21T14:59:23.079 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2260000000000001e-06 2024-09-21T14:59:23.079 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.079 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.080 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:23.080 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624585+0000", 2024-09-21T14:59:23.080 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4820000000000001e-06 2024-09-21T14:59:23.080 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.080 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.080 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:23.080 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.632580+0000", 2024-09-21T14:59:23.080 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0079948959999999996 2024-09-21T14:59:23.080 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.080 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.080 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:23.080 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.632683+0000", 2024-09-21T14:59:23.080 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010308 2024-09-21T14:59:23.080 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.081 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.081 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:23.081 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.633089+0000", 2024-09-21T14:59:23.081 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00040612599999999998 2024-09-21T14:59:23.081 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:23.081 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:23.081 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:23.081 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.081 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.081 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12260 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 57344~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:23.081 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.624587+0000", 2024-09-21T14:59:23.081 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 43.403565161000003, 2024-09-21T14:59:23.081 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0088282069999999994, 2024-09-21T14:59:23.081 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:23.082 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:23.082 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:23.082 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:23.082 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:23.082 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12260 2024-09-21T14:59:23.082 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.082 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:23.082 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.082 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:23.082 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624587+0000", 2024-09-21T14:59:23.082 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:23.082 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.082 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.082 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:23.083 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624587+0000", 2024-09-21T14:59:23.083 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:23.083 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.083 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.083 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:23.083 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624587+0000", 2024-09-21T14:59:23.083 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-21T14:59:23.083 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.083 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.083 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:23.083 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624589+0000", 2024-09-21T14:59:23.083 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4339999999999999e-06 2024-09-21T14:59:23.083 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.083 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.084 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:23.084 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624593+0000", 2024-09-21T14:59:23.084 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5859999999999998e-06 2024-09-21T14:59:23.084 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.084 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.084 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:23.084 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624596+0000", 2024-09-21T14:59:23.084 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5830000000000001e-06 2024-09-21T14:59:23.084 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.084 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.084 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:23.084 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.633132+0000", 2024-09-21T14:59:23.084 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.008536666 2024-09-21T14:59:23.084 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.085 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.085 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:23.085 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.633216+0000", 2024-09-21T14:59:23.085 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3387999999999995e-05 2024-09-21T14:59:23.085 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.085 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.085 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:23.085 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.633415+0000", 2024-09-21T14:59:23.085 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019976799999999999 2024-09-21T14:59:23.085 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:23.085 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:23.085 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:23.085 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.085 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.085 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12516 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 245760~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:23.086 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645194+0000", 2024-09-21T14:59:23.086 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 43.382958623999997, 2024-09-21T14:59:23.086 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0084284990000000008, 2024-09-21T14:59:23.086 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:23.086 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:23.086 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:23.086 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:23.086 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:23.086 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12516 2024-09-21T14:59:23.086 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.086 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:23.086 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.086 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:23.087 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645194+0000", 2024-09-21T14:59:23.087 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:23.087 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.087 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.087 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:23.087 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645194+0000", 2024-09-21T14:59:23.087 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:23.087 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.087 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.087 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:23.087 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645193+0000", 2024-09-21T14:59:23.087 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:23.087 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.087 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.087 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:23.087 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645196+0000", 2024-09-21T14:59:23.088 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1030000000000002e-06 2024-09-21T14:59:23.088 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.088 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.088 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:23.088 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645205+0000", 2024-09-21T14:59:23.088 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6200000000000005e-06 2024-09-21T14:59:23.088 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.088 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.088 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:23.088 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645210+0000", 2024-09-21T14:59:23.088 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6280000000000004e-06 2024-09-21T14:59:23.088 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.088 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.088 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:23.088 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.653305+0000", 2024-09-21T14:59:23.088 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0080957879999999996 2024-09-21T14:59:23.089 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.089 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.089 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:23.089 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.653389+0000", 2024-09-21T14:59:23.089 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3242e-05 2024-09-21T14:59:23.089 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.089 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.089 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:23.089 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.653622+0000", 2024-09-21T14:59:23.089 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00023361599999999999 2024-09-21T14:59:23.089 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:23.089 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:23.089 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:23.089 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.089 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.090 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12517 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 278528~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:23.090 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645214+0000", 2024-09-21T14:59:23.090 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 43.382938510999999, 2024-09-21T14:59:23.090 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086832899999999998, 2024-09-21T14:59:23.090 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:23.090 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:23.090 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:23.090 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:23.090 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:23.090 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12517 2024-09-21T14:59:23.090 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.090 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:23.090 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.090 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:23.091 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645214+0000", 2024-09-21T14:59:23.091 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:23.091 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.091 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.091 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:23.091 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645214+0000", 2024-09-21T14:59:23.091 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:23.091 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.091 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.091 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:23.091 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645214+0000", 2024-09-21T14:59:23.091 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:23.091 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.091 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.091 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:23.092 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645216+0000", 2024-09-21T14:59:23.092 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5579999999999999e-06 2024-09-21T14:59:23.092 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.092 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.092 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:23.092 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645224+0000", 2024-09-21T14:59:23.092 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2439999999999993e-06 2024-09-21T14:59:23.092 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.092 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.092 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:23.092 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645231+0000", 2024-09-21T14:59:23.092 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1859999999999997e-06 2024-09-21T14:59:23.092 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.092 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.092 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:23.093 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.653651+0000", 2024-09-21T14:59:23.093 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0084204220000000003 2024-09-21T14:59:23.093 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.093 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.093 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:23.093 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.653726+0000", 2024-09-21T14:59:23.093 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5356000000000002e-05 2024-09-21T14:59:23.093 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.093 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.093 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:23.093 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.653897+0000", 2024-09-21T14:59:23.093 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000170848 2024-09-21T14:59:23.093 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:23.093 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:23.094 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:23.094 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.094 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.094 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12518 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 385024~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:23.094 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645235+0000", 2024-09-21T14:59:23.094 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 43.382917284999998, 2024-09-21T14:59:23.094 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0089841020000000008, 2024-09-21T14:59:23.094 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:23.094 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:23.094 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:23.094 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:23.094 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:23.094 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12518 2024-09-21T14:59:23.095 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.095 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:23.095 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.095 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:23.095 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645235+0000", 2024-09-21T14:59:23.095 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:23.095 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.095 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.095 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:23.095 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645235+0000", 2024-09-21T14:59:23.095 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:23.095 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.095 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.095 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:23.096 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645235+0000", 2024-09-21T14:59:23.096 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:23.096 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.096 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.096 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:23.096 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645238+0000", 2024-09-21T14:59:23.096 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.869e-06 2024-09-21T14:59:23.096 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.096 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.096 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:23.096 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645244+0000", 2024-09-21T14:59:23.096 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8930000000000002e-06 2024-09-21T14:59:23.096 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.096 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.097 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:23.097 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645249+0000", 2024-09-21T14:59:23.097 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2859999999999999e-06 2024-09-21T14:59:23.097 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.097 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.097 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:23.097 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.653930+0000", 2024-09-21T14:59:23.097 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086815320000000005 2024-09-21T14:59:23.097 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.097 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.097 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:23.097 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654028+0000", 2024-09-21T14:59:23.097 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7578000000000007e-05 2024-09-21T14:59:23.097 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.097 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.097 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:23.098 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654219+0000", 2024-09-21T14:59:23.098 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019132099999999999 2024-09-21T14:59:23.098 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:23.098 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:23.098 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:23.098 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.098 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.098 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12538 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 147456~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:23.098 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645253+0000", 2024-09-21T14:59:23.098 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 43.382899661000003, 2024-09-21T14:59:23.098 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0092095360000000008, 2024-09-21T14:59:23.098 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:23.098 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:23.099 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:23.099 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:23.099 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:23.099 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12538 2024-09-21T14:59:23.099 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.099 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:23.099 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.099 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:23.099 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645253+0000", 2024-09-21T14:59:23.099 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:23.099 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.099 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.099 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:23.099 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645253+0000", 2024-09-21T14:59:23.099 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:23.099 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.100 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.100 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:23.100 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645252+0000", 2024-09-21T14:59:23.100 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:23.100 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.100 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.100 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:23.100 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645255+0000", 2024-09-21T14:59:23.100 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2290000000000002e-06 2024-09-21T14:59:23.100 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.100 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.101 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:23.101 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645267+0000", 2024-09-21T14:59:23.101 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2631e-05 2024-09-21T14:59:23.101 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.101 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.101 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:23.101 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645278+0000", 2024-09-21T14:59:23.101 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0847e-05 2024-09-21T14:59:23.101 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.101 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.101 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:23.101 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654251+0000", 2024-09-21T14:59:23.101 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0089731340000000007 2024-09-21T14:59:23.102 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.102 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.102 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:23.102 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654296+0000", 2024-09-21T14:59:23.102 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4353e-05 2024-09-21T14:59:23.102 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.102 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.102 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:23.102 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654462+0000", 2024-09-21T14:59:23.102 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016668499999999999 2024-09-21T14:59:23.102 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:23.102 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:23.102 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:23.102 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.103 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.103 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12539 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 253952~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:23.103 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645284+0000", 2024-09-21T14:59:23.103 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 43.382868440000003, 2024-09-21T14:59:23.103 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0094524029999999998, 2024-09-21T14:59:23.103 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:23.103 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:23.103 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:23.103 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:23.103 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:23.103 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12539 2024-09-21T14:59:23.103 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.103 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:23.103 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.103 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:23.104 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645284+0000", 2024-09-21T14:59:23.104 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:23.104 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.104 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.104 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:23.104 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645284+0000", 2024-09-21T14:59:23.104 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:23.104 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.104 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.104 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:23.104 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645284+0000", 2024-09-21T14:59:23.104 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:23.104 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.105 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.105 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:23.105 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645286+0000", 2024-09-21T14:59:23.105 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7159999999999999e-06 2024-09-21T14:59:23.105 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.105 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.105 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:23.105 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645292+0000", 2024-09-21T14:59:23.105 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7400000000000001e-06 2024-09-21T14:59:23.105 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.105 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.105 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:23.105 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645297+0000", 2024-09-21T14:59:23.105 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9019999999999998e-06 2024-09-21T14:59:23.105 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.105 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.106 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:23.106 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654490+0000", 2024-09-21T14:59:23.106 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0091932360000000005 2024-09-21T14:59:23.106 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.106 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.106 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:23.106 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654565+0000", 2024-09-21T14:59:23.106 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4657e-05 2024-09-21T14:59:23.106 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.106 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.106 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:23.106 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654736+0000", 2024-09-21T14:59:23.106 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017156199999999999 2024-09-21T14:59:23.106 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:23.107 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:23.107 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:23.107 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.107 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.107 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12540 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 286720~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:23.107 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645302+0000", 2024-09-21T14:59:23.107 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 43.382850171000001, 2024-09-21T14:59:23.107 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0096603090000000006, 2024-09-21T14:59:23.107 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:23.107 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:23.107 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:23.107 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:23.107 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:23.107 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12540 2024-09-21T14:59:23.107 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.108 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:23.108 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.108 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:23.108 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645302+0000", 2024-09-21T14:59:23.108 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:23.108 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.108 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.108 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:23.108 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645302+0000", 2024-09-21T14:59:23.108 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:23.108 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.108 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.108 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:23.108 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645302+0000", 2024-09-21T14:59:23.108 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:23.109 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.109 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.109 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:23.109 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645305+0000", 2024-09-21T14:59:23.109 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7329999999999999e-06 2024-09-21T14:59:23.109 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.109 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.109 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:23.109 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645316+0000", 2024-09-21T14:59:23.109 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1832e-05 2024-09-21T14:59:23.109 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.109 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.109 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:23.109 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645322+0000", 2024-09-21T14:59:23.109 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.169e-06 2024-09-21T14:59:23.110 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.110 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.110 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:23.110 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654778+0000", 2024-09-21T14:59:23.110 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0094559090000000002 2024-09-21T14:59:23.110 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.110 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.110 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:23.110 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654828+0000", 2024-09-21T14:59:23.110 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0461999999999999e-05 2024-09-21T14:59:23.110 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.110 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.110 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:23.111 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654963+0000", 2024-09-21T14:59:23.111 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000134583 2024-09-21T14:59:23.111 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:23.111 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:23.111 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:23.111 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.111 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.111 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12561 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:23.111 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645742+0000", 2024-09-21T14:59:23.111 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 43.382410905, 2024-09-21T14:59:23.111 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0093583810000000007, 2024-09-21T14:59:23.111 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:23.111 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:23.111 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:23.112 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:23.112 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:23.112 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12561 2024-09-21T14:59:23.112 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.112 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:23.112 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.112 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:23.112 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645742+0000", 2024-09-21T14:59:23.112 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:23.112 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.112 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.112 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:23.112 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645742+0000", 2024-09-21T14:59:23.112 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:23.112 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.113 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.113 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:23.113 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645741+0000", 2024-09-21T14:59:23.113 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:23.113 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.113 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.113 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:23.113 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645746+0000", 2024-09-21T14:59:23.113 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8379999999999998e-06 2024-09-21T14:59:23.113 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.113 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.113 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:23.113 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645757+0000", 2024-09-21T14:59:23.114 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0705e-05 2024-09-21T14:59:23.114 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.114 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.114 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:23.114 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645765+0000", 2024-09-21T14:59:23.114 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6729999999999992e-06 2024-09-21T14:59:23.114 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.114 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.114 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:23.114 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654993+0000", 2024-09-21T14:59:23.114 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0092287480000000002 2024-09-21T14:59:23.115 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.115 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.115 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:23.115 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.655050+0000", 2024-09-21T14:59:23.115 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6394999999999998e-05 2024-09-21T14:59:23.115 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.115 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.115 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:23.115 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.655100+0000", 2024-09-21T14:59:23.115 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0049000000000001e-05 2024-09-21T14:59:23.115 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:23.115 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:23.115 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:23.115 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.115 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.116 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:15431 3.12 3:4bc7d206:::rbd_data.1136f5ed5dd6.0000000000000051:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:59:23.116 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:41.673445+0000", 2024-09-21T14:59:23.116 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 41.354707570999999, 2024-09-21T14:59:23.116 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086103610000000004, 2024-09-21T14:59:23.116 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:23.116 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:23.116 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:23.116 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:23.116 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:23.116 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 15431 2024-09-21T14:59:23.116 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.116 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:23.116 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.116 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:23.116 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.673445+0000", 2024-09-21T14:59:23.116 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:23.116 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.117 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.117 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:23.117 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.673445+0000", 2024-09-21T14:59:23.117 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:23.117 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.117 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.117 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:23.117 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.673444+0000", 2024-09-21T14:59:23.117 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:59:23.117 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.117 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.117 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:23.117 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.673451+0000", 2024-09-21T14:59:23.117 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4739999999999999e-06 2024-09-21T14:59:23.117 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.118 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.118 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:23.118 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.673472+0000", 2024-09-21T14:59:23.118 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0687e-05 2024-09-21T14:59:23.118 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.118 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.118 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:23.118 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.673481+0000", 2024-09-21T14:59:23.118 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3349999999999996e-06 2024-09-21T14:59:23.118 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.118 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.118 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:23.118 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.673565+0000", 2024-09-21T14:59:23.118 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4011000000000006e-05 2024-09-21T14:59:23.118 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.119 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.119 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:23.119 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.673650+0000", 2024-09-21T14:59:23.119 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4949000000000003e-05 2024-09-21T14:59:23.119 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.119 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.119 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 3", 2024-09-21T14:59:23.119 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.675014+0000", 2024-09-21T14:59:23.119 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0013636620000000001 2024-09-21T14:59:23.119 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.119 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.119 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:23.119 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.681866+0000", 2024-09-21T14:59:23.119 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0068525640000000002 2024-09-21T14:59:23.119 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.120 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.120 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:59:23.120 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.681885+0000", 2024-09-21T14:59:23.120 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8519999999999999e-05 2024-09-21T14:59:23.120 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.120 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.120 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:59:23.120 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.681961+0000", 2024-09-21T14:59:23.120 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6248000000000006e-05 2024-09-21T14:59:23.120 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.120 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.120 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:23.120 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.682055+0000", 2024-09-21T14:59:23.120 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4184000000000005e-05 2024-09-21T14:59:23.120 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:23.121 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:23.121 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:23.121 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:23.121 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:23.121 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:59:23.122 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T14:59:23.123 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:59:23.119+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:59:23.196 INFO:teuthology.orchestra.run.smithi032.stderr:noscrub is unset 2024-09-21T14:59:23.223 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:59:23.219+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:59:23.244 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:59:23.244 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:59:23.244 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T14:59:23.244 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:59:23.256 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T14:59:23.324 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:59:23.319+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:59:23.359 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:59:23.359 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:59:23.359 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T14:59:23.359 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T14:59:23.360 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:59:23.370 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T14:59:23.425 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:59:23.419+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:59:23.475 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:59:23.476 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T14:59:23.476 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T14:59:23.476 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T14:59:23.476 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.476 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:8034 3.b 3:d1b12ef9:::rbd_data.11361bed1f78.0000000000000062:d [read 32768~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:59:23.476 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:35.227265+0000", 2024-09-21T14:59:23.476 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 48.246907079000003, 2024-09-21T14:59:23.476 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.006525968, 2024-09-21T14:59:23.476 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:23.476 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:23.476 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:23.476 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:23.476 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:23.476 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 8034 2024-09-21T14:59:23.476 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.477 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:23.477 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.477 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:23.477 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227265+0000", 2024-09-21T14:59:23.477 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:23.477 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.477 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.477 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:23.477 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227265+0000", 2024-09-21T14:59:23.477 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:23.477 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.477 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.477 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:23.477 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227265+0000", 2024-09-21T14:59:23.477 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:23.477 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.478 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.478 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:23.478 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227268+0000", 2024-09-21T14:59:23.478 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8990000000000001e-06 2024-09-21T14:59:23.478 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.478 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.478 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:23.478 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227274+0000", 2024-09-21T14:59:23.478 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4470000000000003e-06 2024-09-21T14:59:23.478 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.478 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.478 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:23.478 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227288+0000", 2024-09-21T14:59:23.478 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3550000000000001e-05 2024-09-21T14:59:23.479 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.479 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.479 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:23.479 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.233418+0000", 2024-09-21T14:59:23.479 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0061307870000000004 2024-09-21T14:59:23.479 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.479 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.479 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:23.479 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.233553+0000", 2024-09-21T14:59:23.479 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013475600000000001 2024-09-21T14:59:23.479 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.479 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.479 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:23.479 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.233791+0000", 2024-09-21T14:59:23.479 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00023788499999999999 2024-09-21T14:59:23.479 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:23.480 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:23.480 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:23.480 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.480 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.480 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:8035 3.b 3:d1b12ef9:::rbd_data.11361bed1f78.0000000000000062:d [read 98304~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:59:23.480 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:35.227295+0000", 2024-09-21T14:59:23.480 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 48.246876874000002, 2024-09-21T14:59:23.480 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0069673820000000003, 2024-09-21T14:59:23.480 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:23.480 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:23.480 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:23.480 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:23.480 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:23.480 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 8035 2024-09-21T14:59:23.480 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.480 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:23.481 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.481 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:23.481 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227295+0000", 2024-09-21T14:59:23.481 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:23.481 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.481 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.481 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:23.481 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227295+0000", 2024-09-21T14:59:23.481 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:23.481 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.481 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.481 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:23.481 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227295+0000", 2024-09-21T14:59:23.481 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:23.481 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.481 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.482 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:23.482 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227298+0000", 2024-09-21T14:59:23.482 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2339999999999999e-06 2024-09-21T14:59:23.482 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.482 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.482 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:23.482 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227305+0000", 2024-09-21T14:59:23.482 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0389999999999999e-06 2024-09-21T14:59:23.482 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.482 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.482 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:23.482 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227311+0000", 2024-09-21T14:59:23.482 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8470000000000001e-06 2024-09-21T14:59:23.482 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.482 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.483 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:23.483 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.233824+0000", 2024-09-21T14:59:23.483 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0065132920000000004 2024-09-21T14:59:23.483 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.483 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.483 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:23.483 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.233922+0000", 2024-09-21T14:59:23.483 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.8073999999999996e-05 2024-09-21T14:59:23.483 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.483 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.483 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:23.483 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.234263+0000", 2024-09-21T14:59:23.483 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00034027999999999999 2024-09-21T14:59:23.484 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:23.484 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:23.484 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:23.484 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.484 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.484 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:8048 3.b 3:d1b12ef9:::rbd_data.11361bed1f78.0000000000000062:d [read 49152~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:59:23.484 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:35.227316+0000", 2024-09-21T14:59:23.484 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 48.246856082000001, 2024-09-21T14:59:23.484 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0076197799999999996, 2024-09-21T14:59:23.484 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:23.484 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:23.484 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:23.484 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:23.484 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:23.484 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 8048 2024-09-21T14:59:23.485 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.485 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:23.485 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.485 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:23.485 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227316+0000", 2024-09-21T14:59:23.485 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:23.485 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.485 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.485 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:23.485 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227316+0000", 2024-09-21T14:59:23.485 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:23.485 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.485 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.485 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:23.485 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227316+0000", 2024-09-21T14:59:23.485 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:23.486 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.486 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.486 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:23.486 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227324+0000", 2024-09-21T14:59:23.486 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1009999999999992e-06 2024-09-21T14:59:23.486 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.486 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.486 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:23.486 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227346+0000", 2024-09-21T14:59:23.486 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2347000000000001e-05 2024-09-21T14:59:23.486 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.486 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.486 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:23.487 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227360+0000", 2024-09-21T14:59:23.487 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.403e-05 2024-09-21T14:59:23.487 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.487 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.487 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:23.487 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.234320+0000", 2024-09-21T14:59:23.487 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0069597019999999999 2024-09-21T14:59:23.487 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.487 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.487 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:23.487 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.234730+0000", 2024-09-21T14:59:23.487 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00041006300000000001 2024-09-21T14:59:23.487 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.487 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.487 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:23.487 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.234936+0000", 2024-09-21T14:59:23.488 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00020586700000000001 2024-09-21T14:59:23.488 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:23.488 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:23.488 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:23.488 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.488 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.488 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:8049 3.b 3:d1b12ef9:::rbd_data.11361bed1f78.0000000000000062:d [read 114688~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:59:23.488 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:35.227371+0000", 2024-09-21T14:59:23.488 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 48.246800749000002, 2024-09-21T14:59:23.488 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0078533609999999997, 2024-09-21T14:59:23.488 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:23.488 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:23.488 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:23.488 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:23.488 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:23.489 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 8049 2024-09-21T14:59:23.489 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.489 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:23.489 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.489 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:23.489 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227371+0000", 2024-09-21T14:59:23.489 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:23.489 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.489 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.489 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:23.489 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227371+0000", 2024-09-21T14:59:23.489 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:23.489 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.489 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.489 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:23.489 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227370+0000", 2024-09-21T14:59:23.490 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:59:23.490 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.490 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.490 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:23.490 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227378+0000", 2024-09-21T14:59:23.490 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.379e-06 2024-09-21T14:59:23.490 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.490 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.490 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:23.490 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227388+0000", 2024-09-21T14:59:23.490 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0175e-05 2024-09-21T14:59:23.490 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.490 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.490 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:23.491 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227393+0000", 2024-09-21T14:59:23.491 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6690000000000003e-06 2024-09-21T14:59:23.491 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.491 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.491 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:23.491 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.234964+0000", 2024-09-21T14:59:23.491 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0075705319999999996 2024-09-21T14:59:23.491 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.491 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.491 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:23.491 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.235041+0000", 2024-09-21T14:59:23.491 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7170999999999997e-05 2024-09-21T14:59:23.491 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.492 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.492 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:23.492 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.235225+0000", 2024-09-21T14:59:23.492 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000183735 2024-09-21T14:59:23.492 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:23.492 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:23.492 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:23.492 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.492 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.492 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:19869 3.b 3:d7ca7d68:::rbd_data.11363a21d2be.000000000000002d:head [sparse-read 0~393216] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T14:59:23.492 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:44.340896+0000", 2024-09-21T14:59:23.492 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 39.133276352000003, 2024-09-21T14:59:23.492 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0071749170000000003, 2024-09-21T14:59:23.492 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:23.492 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:23.493 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:23.493 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:23.493 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:23.493 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 19869 2024-09-21T14:59:23.493 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.493 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:23.493 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.493 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:23.493 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340896+0000", 2024-09-21T14:59:23.493 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:23.493 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.493 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.493 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:23.493 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340896+0000", 2024-09-21T14:59:23.493 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:23.493 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.494 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.494 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:23.494 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340894+0000", 2024-09-21T14:59:23.494 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:59:23.494 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.494 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.494 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:23.494 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340903+0000", 2024-09-21T14:59:23.494 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0259999999999996e-06 2024-09-21T14:59:23.494 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.494 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.494 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:23.494 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340923+0000", 2024-09-21T14:59:23.494 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0013999999999998e-05 2024-09-21T14:59:23.495 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.495 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.495 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:23.495 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340938+0000", 2024-09-21T14:59:23.495 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4569000000000001e-05 2024-09-21T14:59:23.495 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.495 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.495 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:23.495 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342092+0000", 2024-09-21T14:59:23.495 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00115354 2024-09-21T14:59:23.495 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.495 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.495 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:23.495 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342290+0000", 2024-09-21T14:59:23.495 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019834299999999999 2024-09-21T14:59:23.495 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.496 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.496 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:23.496 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.348071+0000", 2024-09-21T14:59:23.496 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.005780861 2024-09-21T14:59:23.496 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:23.496 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:23.496 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:23.496 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.496 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.496 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:19872 3.b 3:d38ac3f6:::rbd_data.11363a21d2be.0000000000000030:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T14:59:23.496 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:44.340951+0000", 2024-09-21T14:59:23.496 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 39.133221192999997, 2024-09-21T14:59:23.496 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0069228680000000004, 2024-09-21T14:59:23.496 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:23.496 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:23.496 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:23.497 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:23.497 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:23.497 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 19872 2024-09-21T14:59:23.497 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.497 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:23.497 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.497 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:23.497 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340951+0000", 2024-09-21T14:59:23.497 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:23.497 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.497 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.497 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:23.497 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340951+0000", 2024-09-21T14:59:23.497 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:23.497 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.497 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.497 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:23.498 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340950+0000", 2024-09-21T14:59:23.498 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:23.498 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.498 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.498 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:23.498 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340957+0000", 2024-09-21T14:59:23.498 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8430000000000004e-06 2024-09-21T14:59:23.498 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.498 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.498 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:23.498 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340974+0000", 2024-09-21T14:59:23.498 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7082999999999999e-05 2024-09-21T14:59:23.498 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.498 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.499 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:23.499 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340987+0000", 2024-09-21T14:59:23.499 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2763e-05 2024-09-21T14:59:23.499 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.499 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.499 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:23.499 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.347149+0000", 2024-09-21T14:59:23.499 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0061626600000000004 2024-09-21T14:59:23.499 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.499 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.499 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:23.499 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.347354+0000", 2024-09-21T14:59:23.499 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00020441799999999999 2024-09-21T14:59:23.499 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.499 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.500 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:23.500 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.347874+0000", 2024-09-21T14:59:23.500 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00052019399999999997 2024-09-21T14:59:23.500 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:23.500 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:23.500 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:23.500 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.500 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.500 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:19877 3.b 3:d5fc5a5a:::rbd_data.11363a21d2be.0000000000000035:head [sparse-read 0~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T14:59:23.500 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:44.340998+0000", 2024-09-21T14:59:23.500 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 39.133174357000001, 2024-09-21T14:59:23.500 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0073322129999999998, 2024-09-21T14:59:23.500 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:23.500 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:23.500 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:23.500 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:23.501 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:23.501 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 19877 2024-09-21T14:59:23.501 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.501 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:23.501 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.501 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:23.501 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340998+0000", 2024-09-21T14:59:23.501 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:23.501 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.501 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.501 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:23.501 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340998+0000", 2024-09-21T14:59:23.501 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:23.501 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.501 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.502 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:23.502 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340997+0000", 2024-09-21T14:59:23.502 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:23.502 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.502 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.502 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:23.502 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.341003+0000", 2024-09-21T14:59:23.502 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0070000000000003e-06 2024-09-21T14:59:23.502 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.502 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.503 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:23.503 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.341018+0000", 2024-09-21T14:59:23.503 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5472999999999999e-05 2024-09-21T14:59:23.503 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.503 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.503 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:23.503 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.341040+0000", 2024-09-21T14:59:23.503 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1568000000000001e-05 2024-09-21T14:59:23.503 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.503 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.503 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:23.503 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.347901+0000", 2024-09-21T14:59:23.503 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0068608300000000001 2024-09-21T14:59:23.503 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.504 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.504 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:23.504 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.347973+0000", 2024-09-21T14:59:23.504 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2327000000000001e-05 2024-09-21T14:59:23.504 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.504 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.504 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:23.504 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.348330+0000", 2024-09-21T14:59:23.504 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00035695799999999999 2024-09-21T14:59:23.504 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:23.504 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:23.504 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:23.504 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.504 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.504 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:19998 3.b 3:d4c6ec57:::rbd_data.11363a21d2be.00000000000000ae:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T14:59:23.504 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:44.342204+0000", 2024-09-21T14:59:23.505 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 39.131968585999999, 2024-09-21T14:59:23.505 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0063531860000000002, 2024-09-21T14:59:23.505 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:23.505 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:23.505 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:23.505 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:23.505 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:23.505 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 19998 2024-09-21T14:59:23.505 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.505 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:23.505 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.505 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:23.505 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342204+0000", 2024-09-21T14:59:23.505 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:23.505 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.505 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.506 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:23.506 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342204+0000", 2024-09-21T14:59:23.506 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:23.506 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.506 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.506 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:23.506 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342203+0000", 2024-09-21T14:59:23.506 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:23.506 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.506 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.506 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:23.506 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342209+0000", 2024-09-21T14:59:23.506 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1739999999999997e-06 2024-09-21T14:59:23.506 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.506 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.507 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:23.507 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342223+0000", 2024-09-21T14:59:23.507 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4065e-05 2024-09-21T14:59:23.507 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.507 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.507 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:23.507 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342236+0000", 2024-09-21T14:59:23.507 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2537e-05 2024-09-21T14:59:23.507 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.507 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.507 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:23.507 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.348476+0000", 2024-09-21T14:59:23.507 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0062405760000000003 2024-09-21T14:59:23.507 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.507 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.507 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:23.508 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.348557+0000", 2024-09-21T14:59:23.508 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0716000000000004e-05 2024-09-21T14:59:23.508 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:23.508 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:23.508 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:23.508 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.508 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.508 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:19999 3.b 3:d4069488:::rbd_data.11363a21d2be.00000000000000af:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T14:59:23.508 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:44.342246+0000", 2024-09-21T14:59:23.508 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 39.131926421000003, 2024-09-21T14:59:23.508 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0064139210000000004, 2024-09-21T14:59:23.508 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:23.508 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:23.508 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:23.508 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:23.508 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:23.509 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 19999 2024-09-21T14:59:23.509 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.509 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:23.509 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.509 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:23.509 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342246+0000", 2024-09-21T14:59:23.509 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:23.509 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.509 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.509 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:23.509 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342246+0000", 2024-09-21T14:59:23.509 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:23.509 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.509 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.509 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:23.509 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342245+0000", 2024-09-21T14:59:23.510 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:23.510 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.510 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.510 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:23.510 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342251+0000", 2024-09-21T14:59:23.510 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8660000000000002e-06 2024-09-21T14:59:23.510 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.510 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.510 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:23.510 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342264+0000", 2024-09-21T14:59:23.510 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3716999999999999e-05 2024-09-21T14:59:23.510 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.510 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.510 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:23.511 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342275+0000", 2024-09-21T14:59:23.511 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1071e-05 2024-09-21T14:59:23.511 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.511 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.511 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:23.511 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.348581+0000", 2024-09-21T14:59:23.511 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0063053730000000004 2024-09-21T14:59:23.511 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.511 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.511 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:23.511 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.348660+0000", 2024-09-21T14:59:23.511 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8854000000000001e-05 2024-09-21T14:59:23.511 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:23.511 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:23.511 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:23.511 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.512 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.512 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:20440 3.b 3:d6a6f68e:::rbd_object_map.1136c84d9a13:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-21T14:59:23.512 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:45.106428+0000", 2024-09-21T14:59:23.512 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 38.367744575000003, 2024-09-21T14:59:23.512 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0076349679999999998, 2024-09-21T14:59:23.512 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:23.512 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:23.512 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:23.512 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:23.512 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:23.512 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 20440 2024-09-21T14:59:23.512 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.512 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:23.512 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.512 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:23.512 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.106428+0000", 2024-09-21T14:59:23.513 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:23.513 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.513 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.513 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:23.513 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.106428+0000", 2024-09-21T14:59:23.513 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:23.513 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.513 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.513 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:23.513 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.106425+0000", 2024-09-21T14:59:23.513 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-21T14:59:23.513 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.513 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.513 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:23.514 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.106442+0000", 2024-09-21T14:59:23.514 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7237000000000001e-05 2024-09-21T14:59:23.514 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.514 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.514 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:23.514 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.106476+0000", 2024-09-21T14:59:23.514 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4004999999999997e-05 2024-09-21T14:59:23.514 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.514 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.514 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:23.514 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.106491+0000", 2024-09-21T14:59:23.514 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4865000000000001e-05 2024-09-21T14:59:23.514 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.514 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.515 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:23.515 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.106602+0000", 2024-09-21T14:59:23.515 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011113200000000001 2024-09-21T14:59:23.515 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.515 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.515 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op must wait for map", 2024-09-21T14:59:23.515 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.106611+0000", 2024-09-21T14:59:23.515 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9199999999999993e-06 2024-09-21T14:59:23.515 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.515 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.515 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:23.515 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.110475+0000", 2024-09-21T14:59:23.515 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0038633259999999998 2024-09-21T14:59:23.515 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.515 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.515 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:23.516 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.110622+0000", 2024-09-21T14:59:23.516 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014734200000000001 2024-09-21T14:59:23.516 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.516 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.516 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T14:59:23.516 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.111136+0000", 2024-09-21T14:59:23.516 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00051382299999999995 2024-09-21T14:59:23.516 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.516 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.516 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:59:23.516 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.112514+0000", 2024-09-21T14:59:23.516 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001378287 2024-09-21T14:59:23.516 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.516 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.517 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:23.517 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.113830+0000", 2024-09-21T14:59:23.517 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0013157939999999999 2024-09-21T14:59:23.517 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.517 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.517 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:59:23.517 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.113969+0000", 2024-09-21T14:59:23.517 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000138849 2024-09-21T14:59:23.517 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.517 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.517 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:23.517 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.114063+0000", 2024-09-21T14:59:23.517 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3999999999999994e-05 2024-09-21T14:59:23.517 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:23.517 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:23.517 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:23.518 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.518 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.518 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:20460 3.b 3:d6a6f68e:::rbd_object_map.1136c84d9a13:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-21T14:59:23.518 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:45.150183+0000", 2024-09-21T14:59:23.518 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 38.323988757000002, 2024-09-21T14:59:23.518 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0063818800000000004, 2024-09-21T14:59:23.518 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:23.518 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:23.518 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:23.518 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:23.518 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:23.518 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 20460 2024-09-21T14:59:23.518 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.518 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:23.518 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.519 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:23.519 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150183+0000", 2024-09-21T14:59:23.519 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:23.519 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.519 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.519 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:23.519 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150183+0000", 2024-09-21T14:59:23.519 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:23.519 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.519 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.519 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:23.519 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150180+0000", 2024-09-21T14:59:23.519 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999971 2024-09-21T14:59:23.519 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.519 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.519 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:23.520 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150211+0000", 2024-09-21T14:59:23.520 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0993e-05 2024-09-21T14:59:23.520 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.520 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.520 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:23.520 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150241+0000", 2024-09-21T14:59:23.520 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9589000000000001e-05 2024-09-21T14:59:23.520 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.520 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.520 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:23.520 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150261+0000", 2024-09-21T14:59:23.520 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.967e-05 2024-09-21T14:59:23.520 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.520 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.520 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:23.520 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150411+0000", 2024-09-21T14:59:23.521 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015004300000000001 2024-09-21T14:59:23.521 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.521 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.521 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:23.521 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150598+0000", 2024-09-21T14:59:23.521 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018712399999999999 2024-09-21T14:59:23.521 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.521 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.521 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T14:59:23.521 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.151390+0000", 2024-09-21T14:59:23.521 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00079194599999999997 2024-09-21T14:59:23.521 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.521 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.521 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:23.521 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.156302+0000", 2024-09-21T14:59:23.521 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0049123960000000003 2024-09-21T14:59:23.521 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.522 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.522 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:59:23.522 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.156435+0000", 2024-09-21T14:59:23.522 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013287300000000001 2024-09-21T14:59:23.522 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.522 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.522 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:59:23.522 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.156508+0000", 2024-09-21T14:59:23.522 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2504999999999998e-05 2024-09-21T14:59:23.522 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.522 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.522 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:23.522 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.156565+0000", 2024-09-21T14:59:23.522 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7773999999999999e-05 2024-09-21T14:59:23.523 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:23.523 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:23.523 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:23.523 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.523 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.523 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:20461 3.b 3:d6a6f68e:::rbd_object_map.1136c84d9a13:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-21T14:59:23.523 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:45.150284+0000", 2024-09-21T14:59:23.523 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 38.323887935999998, 2024-09-21T14:59:23.523 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0064029040000000001, 2024-09-21T14:59:23.523 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:23.523 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:23.523 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:23.523 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:23.523 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:23.523 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 20461 2024-09-21T14:59:23.524 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.524 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:23.524 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.524 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:23.524 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150284+0000", 2024-09-21T14:59:23.524 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:23.524 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.524 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.524 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:23.524 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150284+0000", 2024-09-21T14:59:23.524 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:23.524 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.524 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.524 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:23.524 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150283+0000", 2024-09-21T14:59:23.525 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:23.525 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.525 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.525 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:23.525 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150289+0000", 2024-09-21T14:59:23.525 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8799999999999996e-06 2024-09-21T14:59:23.525 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.525 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.525 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:23.525 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150319+0000", 2024-09-21T14:59:23.525 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9828999999999999e-05 2024-09-21T14:59:23.525 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.525 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.525 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:23.525 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150328+0000", 2024-09-21T14:59:23.526 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4770000000000003e-06 2024-09-21T14:59:23.526 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.526 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.526 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:23.526 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.152355+0000", 2024-09-21T14:59:23.526 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0020279870000000002 2024-09-21T14:59:23.526 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.526 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.526 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:23.526 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.152652+0000", 2024-09-21T14:59:23.526 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00029681 2024-09-21T14:59:23.526 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.526 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.526 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T14:59:23.527 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.153318+0000", 2024-09-21T14:59:23.527 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000666101 2024-09-21T14:59:23.527 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.527 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.527 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:23.527 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.156416+0000", 2024-09-21T14:59:23.527 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0030980729999999998 2024-09-21T14:59:23.527 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.527 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.527 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:59:23.527 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.156577+0000", 2024-09-21T14:59:23.527 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016049700000000001 2024-09-21T14:59:23.527 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.527 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.527 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:59:23.527 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.156628+0000", 2024-09-21T14:59:23.528 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0642000000000001e-05 2024-09-21T14:59:23.528 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.528 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.528 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:23.528 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.156687+0000", 2024-09-21T14:59:23.528 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9571000000000003e-05 2024-09-21T14:59:23.528 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:23.528 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:23.528 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:23.528 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.528 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.528 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:20462 3.b 3:d6a6f68e:::rbd_object_map.1136c84d9a13:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-21T14:59:23.528 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:45.150372+0000", 2024-09-21T14:59:23.528 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 38.323799854000001, 2024-09-21T14:59:23.528 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0075518110000000003, 2024-09-21T14:59:23.528 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:23.529 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:23.529 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:23.529 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:23.529 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:23.529 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 20462 2024-09-21T14:59:23.529 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.529 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:23.529 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.529 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:23.529 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150372+0000", 2024-09-21T14:59:23.529 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:23.529 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.529 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.529 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:23.529 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150372+0000", 2024-09-21T14:59:23.529 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:23.530 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.530 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.530 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:23.530 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150372+0000", 2024-09-21T14:59:23.530 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:23.530 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.530 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.530 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:23.530 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150378+0000", 2024-09-21T14:59:23.530 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3330000000000002e-06 2024-09-21T14:59:23.530 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.530 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.530 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:23.530 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150413+0000", 2024-09-21T14:59:23.530 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4798999999999997e-05 2024-09-21T14:59:23.531 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.531 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.531 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:23.531 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150423+0000", 2024-09-21T14:59:23.531 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.8099999999999992e-06 2024-09-21T14:59:23.531 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.531 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.531 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:23.531 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.154550+0000", 2024-09-21T14:59:23.531 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0041275959999999999 2024-09-21T14:59:23.531 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.531 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.531 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:23.531 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.154758+0000", 2024-09-21T14:59:23.531 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00020815499999999999 2024-09-21T14:59:23.531 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.532 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.532 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T14:59:23.532 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.155323+0000", 2024-09-21T14:59:23.532 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00056425400000000004 2024-09-21T14:59:23.532 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.532 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.532 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:59:23.532 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.156702+0000", 2024-09-21T14:59:23.532 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001379676 2024-09-21T14:59:23.532 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.532 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.532 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:23.532 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.157662+0000", 2024-09-21T14:59:23.532 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00095948099999999998 2024-09-21T14:59:23.533 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.533 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.533 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:59:23.533 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.157816+0000", 2024-09-21T14:59:23.533 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000154666 2024-09-21T14:59:23.533 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.533 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.533 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:23.533 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.157924+0000", 2024-09-21T14:59:23.533 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000107798 2024-09-21T14:59:23.533 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:23.533 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:23.533 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:23.533 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.533 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.534 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31476 3.c 3:35718b6b:::rbd_data.11367721c19.0000000000000014:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T14:59:23.534 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.505975+0000", 2024-09-21T14:59:23.534 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 24.968197217, 2024-09-21T14:59:23.534 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99515995899999998, 2024-09-21T14:59:23.534 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:23.534 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:23.534 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:23.534 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:23.534 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:23.534 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31476 2024-09-21T14:59:23.534 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.534 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:23.534 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.535 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:23.535 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.505975+0000", 2024-09-21T14:59:23.535 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:23.535 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.535 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.535 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:23.535 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.505975+0000", 2024-09-21T14:59:23.535 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:23.535 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.535 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.535 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:23.535 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.505971+0000", 2024-09-21T14:59:23.535 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999967 2024-09-21T14:59:23.535 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.535 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.535 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:23.536 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.505989+0000", 2024-09-21T14:59:23.536 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7985000000000001e-05 2024-09-21T14:59:23.536 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.536 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.536 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:23.536 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506050+0000", 2024-09-21T14:59:23.536 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0659000000000003e-05 2024-09-21T14:59:23.536 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.536 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.536 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:23.536 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506074+0000", 2024-09-21T14:59:23.536 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4156e-05 2024-09-21T14:59:23.536 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.536 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.536 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:23.536 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506225+0000", 2024-09-21T14:59:23.537 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015097 2024-09-21T14:59:23.537 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.537 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.537 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T14:59:23.537 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506256+0000", 2024-09-21T14:59:23.537 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0505999999999999e-05 2024-09-21T14:59:23.537 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.537 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.537 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:23.537 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.497974+0000", 2024-09-21T14:59:23.537 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99171825700000005 2024-09-21T14:59:23.537 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.537 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.537 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for missing object", 2024-09-21T14:59:23.537 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.498449+0000", 2024-09-21T14:59:23.537 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00047506399999999998 2024-09-21T14:59:23.537 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.538 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.538 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:23.538 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.501000+0000", 2024-09-21T14:59:23.538 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0025506919999999998 2024-09-21T14:59:23.538 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.538 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.538 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:23.538 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.501135+0000", 2024-09-21T14:59:23.538 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013524000000000001 2024-09-21T14:59:23.538 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:23.538 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:23.538 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:23.538 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.538 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.539 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31480 3.c 3:36ccd808:::rbd_data.11367721c19.0000000000000018:head [sparse-read 0~14821] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T14:59:23.539 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.506116+0000", 2024-09-21T14:59:23.539 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 24.968055874000001, 2024-09-21T14:59:23.539 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99316033599999998, 2024-09-21T14:59:23.539 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:23.539 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:23.539 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:23.539 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:23.539 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:23.539 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31480 2024-09-21T14:59:23.539 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.539 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:23.539 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.539 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:23.539 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506116+0000", 2024-09-21T14:59:23.540 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:23.540 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.540 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.540 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:23.540 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506116+0000", 2024-09-21T14:59:23.540 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:23.540 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.540 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.540 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:23.540 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506115+0000", 2024-09-21T14:59:23.540 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:59:23.540 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.540 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.540 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:23.540 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506124+0000", 2024-09-21T14:59:23.540 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0659999999999993e-06 2024-09-21T14:59:23.541 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.541 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.541 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:23.541 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506145+0000", 2024-09-21T14:59:23.541 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1447e-05 2024-09-21T14:59:23.541 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.541 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.541 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:23.541 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506161+0000", 2024-09-21T14:59:23.541 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5209e-05 2024-09-21T14:59:23.541 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.541 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.541 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:23.541 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506337+0000", 2024-09-21T14:59:23.541 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017642999999999999 2024-09-21T14:59:23.541 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.542 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.542 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T14:59:23.542 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506375+0000", 2024-09-21T14:59:23.542 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7536000000000001e-05 2024-09-21T14:59:23.542 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.542 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.542 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:23.542 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.498815+0000", 2024-09-21T14:59:23.542 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99244067999999996 2024-09-21T14:59:23.542 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.542 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.542 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:23.542 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.498950+0000", 2024-09-21T14:59:23.542 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013436700000000001 2024-09-21T14:59:23.542 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.543 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.543 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:23.543 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499277+0000", 2024-09-21T14:59:23.543 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00032696800000000002 2024-09-21T14:59:23.543 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:23.543 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:23.543 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:23.543 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.543 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.543 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31513 3.c 3:36de8523:::rbd_data.11367721c19.0000000000000039:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T14:59:23.543 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.509071+0000", 2024-09-21T14:59:23.543 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 24.965101674, 2024-09-21T14:59:23.543 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99038225099999999, 2024-09-21T14:59:23.543 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:23.543 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:23.544 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:23.544 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:23.544 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:23.544 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31513 2024-09-21T14:59:23.544 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.544 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:23.544 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.544 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:23.544 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509071+0000", 2024-09-21T14:59:23.544 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:23.544 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.544 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.544 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:23.544 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509071+0000", 2024-09-21T14:59:23.545 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:23.545 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.545 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.545 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:23.545 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509068+0000", 2024-09-21T14:59:23.545 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-21T14:59:23.545 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.545 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.545 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:23.545 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509082+0000", 2024-09-21T14:59:23.545 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3545e-05 2024-09-21T14:59:23.545 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.545 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.545 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:23.545 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509116+0000", 2024-09-21T14:59:23.545 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3942000000000001e-05 2024-09-21T14:59:23.546 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.546 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.546 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:23.546 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509134+0000", 2024-09-21T14:59:23.546 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8839999999999999e-05 2024-09-21T14:59:23.546 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.546 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.546 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:23.546 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509318+0000", 2024-09-21T14:59:23.546 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018326799999999999 2024-09-21T14:59:23.546 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.546 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.546 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T14:59:23.546 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509346+0000", 2024-09-21T14:59:23.546 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8019000000000001e-05 2024-09-21T14:59:23.547 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.547 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.547 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:23.547 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499330+0000", 2024-09-21T14:59:23.547 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.98998399299999995 2024-09-21T14:59:23.547 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.547 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.547 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:23.547 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499453+0000", 2024-09-21T14:59:23.547 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000123115 2024-09-21T14:59:23.547 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:23.547 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:23.547 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:23.547 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.547 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.548 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31515 3.c 3:35b36ddd:::rbd_data.11367721c19.000000000000003b:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T14:59:23.548 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.509177+0000", 2024-09-21T14:59:23.548 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 24.964995248000001, 2024-09-21T14:59:23.548 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99044870600000001, 2024-09-21T14:59:23.548 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:23.548 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:23.548 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:23.548 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:23.548 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:23.548 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31515 2024-09-21T14:59:23.548 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.548 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:23.548 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.548 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:23.548 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509177+0000", 2024-09-21T14:59:23.549 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:23.549 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.549 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.549 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:23.549 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509177+0000", 2024-09-21T14:59:23.549 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:23.549 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.549 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.549 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:23.549 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509176+0000", 2024-09-21T14:59:23.549 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:23.549 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.549 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.549 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:23.550 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509183+0000", 2024-09-21T14:59:23.550 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6720000000000004e-06 2024-09-21T14:59:23.550 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.550 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.550 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:23.550 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509269+0000", 2024-09-21T14:59:23.550 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5881e-05 2024-09-21T14:59:23.550 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.550 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.550 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:23.550 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509303+0000", 2024-09-21T14:59:23.550 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3367000000000001e-05 2024-09-21T14:59:23.550 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.550 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.551 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:23.551 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509425+0000", 2024-09-21T14:59:23.551 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000122199 2024-09-21T14:59:23.551 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.551 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.551 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T14:59:23.551 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509448+0000", 2024-09-21T14:59:23.551 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2688999999999999e-05 2024-09-21T14:59:23.551 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.551 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.551 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:23.551 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499501+0000", 2024-09-21T14:59:23.551 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99005306999999998 2024-09-21T14:59:23.551 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.551 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.551 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:23.552 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499626+0000", 2024-09-21T14:59:23.552 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012501400000000001 2024-09-21T14:59:23.552 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:23.552 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:23.552 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:23.552 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.552 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.552 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31531 3.c 3:314f0287:::rbd_data.11367721c19.000000000000004b:head [sparse-read 0~7106] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T14:59:23.552 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.509329+0000", 2024-09-21T14:59:23.552 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 24.964843227999999, 2024-09-21T14:59:23.552 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99103085300000004, 2024-09-21T14:59:23.552 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:23.552 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:23.552 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:23.553 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:23.553 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:23.553 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31531 2024-09-21T14:59:23.553 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.553 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:23.553 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.553 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:23.553 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509329+0000", 2024-09-21T14:59:23.553 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:23.553 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.553 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.553 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:23.553 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509329+0000", 2024-09-21T14:59:23.554 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:23.554 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.554 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.554 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:23.554 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509328+0000", 2024-09-21T14:59:23.554 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:59:23.554 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.554 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.554 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:23.554 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509336+0000", 2024-09-21T14:59:23.554 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3170000000000002e-06 2024-09-21T14:59:23.554 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.554 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.555 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:23.555 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509353+0000", 2024-09-21T14:59:23.555 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7214000000000002e-05 2024-09-21T14:59:23.555 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.555 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.555 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:23.555 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509366+0000", 2024-09-21T14:59:23.555 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.26e-05 2024-09-21T14:59:23.555 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.555 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.555 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:23.555 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509523+0000", 2024-09-21T14:59:23.555 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015708000000000001 2024-09-21T14:59:23.555 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.555 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.556 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T14:59:23.556 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509544+0000", 2024-09-21T14:59:23.556 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0783000000000001e-05 2024-09-21T14:59:23.556 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.556 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.556 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:23.556 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499727+0000", 2024-09-21T14:59:23.556 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99018345200000002 2024-09-21T14:59:23.556 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.556 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.556 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:23.556 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499949+0000", 2024-09-21T14:59:23.556 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000221358 2024-09-21T14:59:23.556 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.556 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.556 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:23.557 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.500360+0000", 2024-09-21T14:59:23.557 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00041130700000000002 2024-09-21T14:59:23.557 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:23.557 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:23.557 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:23.557 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.557 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.557 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31605 3.c 3:34f43f7f:::rbd_data.113661d6691d.0000000000000034:head [delete] snapc 20=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-21T14:59:23.557 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:59.850318+0000", 2024-09-21T14:59:23.557 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.623854466000001, 2024-09-21T14:59:23.557 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0069284979999999999, 2024-09-21T14:59:23.557 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:23.557 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:23.557 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:23.557 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:23.557 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:23.558 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31605 2024-09-21T14:59:23.558 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.558 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:23.558 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.558 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:23.558 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.850318+0000", 2024-09-21T14:59:23.558 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:23.558 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.558 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.558 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:23.558 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.850318+0000", 2024-09-21T14:59:23.558 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:23.558 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.558 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.558 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:23.558 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.850314+0000", 2024-09-21T14:59:23.559 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999962 2024-09-21T14:59:23.559 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.559 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.559 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:23.559 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.850332+0000", 2024-09-21T14:59:23.559 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8414e-05 2024-09-21T14:59:23.559 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.559 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.559 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:23.559 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.850376+0000", 2024-09-21T14:59:23.559 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3989000000000002e-05 2024-09-21T14:59:23.559 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.559 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.559 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:23.559 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.850399+0000", 2024-09-21T14:59:23.559 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2455000000000001e-05 2024-09-21T14:59:23.560 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.560 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.560 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:23.560 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.850537+0000", 2024-09-21T14:59:23.560 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013858100000000001 2024-09-21T14:59:23.560 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.560 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.560 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:23.560 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.850756+0000", 2024-09-21T14:59:23.560 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021833099999999999 2024-09-21T14:59:23.560 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.560 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.560 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T14:59:23.560 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.850939+0000", 2024-09-21T14:59:23.560 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018285600000000001 2024-09-21T14:59:23.560 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.561 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.561 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:59:23.561 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.852297+0000", 2024-09-21T14:59:23.561 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0013588599999999999 2024-09-21T14:59:23.561 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.561 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.561 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:23.561 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.857067+0000", 2024-09-21T14:59:23.561 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0047691349999999999 2024-09-21T14:59:23.561 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.561 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.561 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:59:23.561 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.857143+0000", 2024-09-21T14:59:23.561 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6983000000000005e-05 2024-09-21T14:59:23.561 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.561 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.562 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:23.562 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.857246+0000", 2024-09-21T14:59:23.562 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010275799999999999 2024-09-21T14:59:23.562 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:23.562 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:23.562 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:23.562 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.562 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.562 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:61580 3.b 3:d3369e4f:::rbd_data.1136862f872f.0000000000000010:head [write 0~557056 [fadvise_sequential+fadvise_nocache],zero 557056~16384,write 573440~49152 [fadvise_sequential+fadvise_nocache],zero 622592~16384,write 638976~49152 [fadvise_sequential+fadvise_nocache],zero 688128~16384,write 704512~49152 [fadvise_sequential+fadvise_nocache],zero 753664~16384,write 770048~49152 [fadvise_sequential+fadvise_nocache],zero 819200~16384,write 835584~49152 [fadvise_sequential+fadvise_nocache],zero 884736~16384,write 901120~49152 [fadvise_sequential+fadvise_nocache],zero 950272~16384,write 966656~49152 [fadvise_sequential+fadvise_nocache],zero 1015808~16384,write 1032192~16384 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-21T14:59:23.562 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:19.231716+0000", 2024-09-21T14:59:23.562 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.2424560659999999, 2024-09-21T14:59:23.562 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0096097690000000006, 2024-09-21T14:59:23.562 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:23.562 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:23.562 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:23.563 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:23.563 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:23.563 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 61580 2024-09-21T14:59:23.563 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.563 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:23.563 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.563 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:23.563 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.231716+0000", 2024-09-21T14:59:23.563 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:23.563 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.563 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.563 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:23.563 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.231716+0000", 2024-09-21T14:59:23.563 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:23.563 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.563 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.564 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:23.564 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.231715+0000", 2024-09-21T14:59:23.564 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:59:23.564 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.564 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.564 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:23.564 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.235290+0000", 2024-09-21T14:59:23.564 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0035750399999999998 2024-09-21T14:59:23.564 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.564 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.564 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:23.564 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.235310+0000", 2024-09-21T14:59:23.564 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0120999999999999e-05 2024-09-21T14:59:23.564 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.564 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.565 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:23.565 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.235319+0000", 2024-09-21T14:59:23.565 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2420000000000006e-06 2024-09-21T14:59:23.565 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.565 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.565 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:23.565 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.235416+0000", 2024-09-21T14:59:23.565 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7204e-05 2024-09-21T14:59:23.565 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.565 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.565 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:23.565 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.235775+0000", 2024-09-21T14:59:23.565 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000358883 2024-09-21T14:59:23.565 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.565 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.566 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T14:59:23.566 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.236560+0000", 2024-09-21T14:59:23.566 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000784463 2024-09-21T14:59:23.566 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.566 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.566 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:59:23.566 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.241119+0000", 2024-09-21T14:59:23.566 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0045590589999999999 2024-09-21T14:59:23.566 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.566 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.566 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:23.566 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.241210+0000", 2024-09-21T14:59:23.566 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1617000000000001e-05 2024-09-21T14:59:23.566 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.566 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.567 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:59:23.567 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.241285+0000", 2024-09-21T14:59:23.567 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4470999999999999e-05 2024-09-21T14:59:23.567 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:23.567 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:23.567 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:23.567 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.241326+0000", 2024-09-21T14:59:23.567 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0967000000000003e-05 2024-09-21T14:59:23.567 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:23.567 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:23.567 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:23.567 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:23.567 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:23.567 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:59:23.569 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T14:59:23.569 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:59:23.569+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:59:23.670 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:59:23.669+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:59:23.686 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:23.687 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:59:23.687 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T14:59:23.687 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:23.697 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T14:59:23.770 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:59:23.769+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:59:23.802 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:23.802 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:59:23.802 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T14:59:23.802 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T14:59:23.802 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:23.812 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T14:59:23.871 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:59:23.867+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:59:23.970 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:23.970 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T14:59:23.970 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T14:59:23.970 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T14:59:23.970 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:23.970 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:1473 3.15 3:af2afa6a:::rbd_data.1136515ecb6b.000000000000001b:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T14:59:23.970 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:30.421730+0000", 2024-09-21T14:59:23.970 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 53.546518028000001, 2024-09-21T14:59:23.970 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021478937, 2024-09-21T14:59:23.970 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:23.970 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:23.970 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:23.971 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:23.971 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:23.971 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1473 2024-09-21T14:59:23.971 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:23.971 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:23.971 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:23.971 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:23.971 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421730+0000", 2024-09-21T14:59:23.971 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:23.971 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:23.971 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:23.971 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:23.971 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421730+0000", 2024-09-21T14:59:23.972 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:23.972 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:23.972 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:23.972 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:23.972 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421728+0000", 2024-09-21T14:59:23.972 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-21T14:59:23.972 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:23.972 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:23.972 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:23.972 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421739+0000", 2024-09-21T14:59:23.972 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0502e-05 2024-09-21T14:59:23.972 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:23.972 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:23.972 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:23.973 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421762+0000", 2024-09-21T14:59:23.973 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3405999999999999e-05 2024-09-21T14:59:23.973 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:23.973 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:23.973 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:23.973 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421773+0000", 2024-09-21T14:59:23.973 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0744000000000001e-05 2024-09-21T14:59:23.973 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:23.973 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:23.973 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:23.973 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421838+0000", 2024-09-21T14:59:23.973 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5486000000000003e-05 2024-09-21T14:59:23.973 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:23.973 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:23.974 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:23.974 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421955+0000", 2024-09-21T14:59:23.974 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000116765 2024-09-21T14:59:23.974 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:23.974 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:23.974 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 6", 2024-09-21T14:59:23.974 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.425572+0000", 2024-09-21T14:59:23.974 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0036171380000000002 2024-09-21T14:59:23.974 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:23.974 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:23.974 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:23.974 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.442973+0000", 2024-09-21T14:59:23.974 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.017400268 2024-09-21T14:59:23.974 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:23.975 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:23.975 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:23.975 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.442996+0000", 2024-09-21T14:59:23.975 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3139e-05 2024-09-21T14:59:23.975 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:23.975 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:23.975 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:23.975 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.443079+0000", 2024-09-21T14:59:23.975 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2832999999999998e-05 2024-09-21T14:59:23.975 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:23.975 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:23.975 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:23.975 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.443209+0000", 2024-09-21T14:59:23.975 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013052499999999999 2024-09-21T14:59:23.976 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:23.976 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:23.976 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:23.976 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:23.976 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:23.976 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:1678 3.c 3:377cc7e1:::rbd_data.1136515ecb6b.000000000000000f:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T14:59:23.976 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:30.988085+0000", 2024-09-21T14:59:23.976 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 52.980163433999998, 2024-09-21T14:59:23.976 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.032974802999999997, 2024-09-21T14:59:23.976 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:23.976 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:23.976 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:23.976 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:23.976 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:23.976 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1678 2024-09-21T14:59:23.976 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:23.977 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:23.977 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:23.977 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:23.977 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988085+0000", 2024-09-21T14:59:23.977 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:23.977 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:23.977 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:23.977 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:23.977 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988085+0000", 2024-09-21T14:59:23.977 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:23.977 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:23.977 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:23.977 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:23.977 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988083+0000", 2024-09-21T14:59:23.977 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-21T14:59:23.977 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:23.978 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:23.978 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:23.978 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988093+0000", 2024-09-21T14:59:23.978 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8800000000000003e-06 2024-09-21T14:59:23.978 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:23.978 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:23.978 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:23.978 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988113+0000", 2024-09-21T14:59:23.978 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0217999999999999e-05 2024-09-21T14:59:23.978 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:23.978 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:23.978 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:23.978 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988123+0000", 2024-09-21T14:59:23.978 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5859999999999994e-06 2024-09-21T14:59:23.978 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:23.979 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:23.979 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:23.979 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988203+0000", 2024-09-21T14:59:23.979 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0834000000000006e-05 2024-09-21T14:59:23.979 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:23.979 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:23.979 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:23.979 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988376+0000", 2024-09-21T14:59:23.979 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017301299999999999 2024-09-21T14:59:23.979 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:23.979 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:23.979 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 7", 2024-09-21T14:59:23.979 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.996338+0000", 2024-09-21T14:59:23.979 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0079618520000000002 2024-09-21T14:59:23.979 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:23.979 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:23.980 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:23.980 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.020881+0000", 2024-09-21T14:59:23.980 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024542731000000002 2024-09-21T14:59:23.980 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:23.980 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:23.980 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:23.980 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.020904+0000", 2024-09-21T14:59:23.980 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2861e-05 2024-09-21T14:59:23.980 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:23.980 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:23.980 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:23.980 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.020987+0000", 2024-09-21T14:59:23.980 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2806000000000006e-05 2024-09-21T14:59:23.980 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:23.980 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:23.981 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:23.981 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.021060+0000", 2024-09-21T14:59:23.981 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2871000000000001e-05 2024-09-21T14:59:23.981 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:23.981 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:23.981 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:23.981 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:23.981 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:23.981 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9369 3.3 3:c725132c:::rbd_data.11363a21d2be.0000000000000007:10 [read 24576~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:23.981 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157241+0000", 2024-09-21T14:59:23.981 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 44.811006988000003, 2024-09-21T14:59:23.981 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018289156000000001, 2024-09-21T14:59:23.981 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:23.982 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:23.982 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:23.982 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:23.982 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:23.982 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9369 2024-09-21T14:59:23.982 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:23.982 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:23.982 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:23.982 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:23.982 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157241+0000", 2024-09-21T14:59:23.982 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:23.982 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:23.982 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:23.982 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:23.983 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157241+0000", 2024-09-21T14:59:23.983 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:23.983 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:23.983 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:23.983 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:23.983 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157241+0000", 2024-09-21T14:59:23.983 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:23.983 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:23.983 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:23.983 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:23.983 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157244+0000", 2024-09-21T14:59:23.983 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5049999999999999e-06 2024-09-21T14:59:23.983 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:23.984 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:23.984 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:23.984 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157255+0000", 2024-09-21T14:59:23.984 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0797e-05 2024-09-21T14:59:23.984 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:23.984 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:23.984 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:23.984 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157260+0000", 2024-09-21T14:59:23.984 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1749999999999996e-06 2024-09-21T14:59:23.984 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:23.984 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:23.984 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:23.984 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175237+0000", 2024-09-21T14:59:23.984 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.017976405000000001 2024-09-21T14:59:23.985 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:23.985 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:23.985 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:23.985 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175331+0000", 2024-09-21T14:59:23.985 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4508999999999999e-05 2024-09-21T14:59:23.985 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:23.985 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:23.985 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:23.985 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175530+0000", 2024-09-21T14:59:23.985 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00019928200000000001 2024-09-21T14:59:23.985 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:23.985 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:23.985 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:23.985 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:23.986 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:23.986 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9370 3.3 3:c725132c:::rbd_data.11363a21d2be.0000000000000007:10 [read 163840~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:23.986 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157279+0000", 2024-09-21T14:59:23.986 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 44.810968809999999, 2024-09-21T14:59:23.986 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018472379000000001, 2024-09-21T14:59:23.986 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:23.986 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:23.986 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:23.986 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:23.986 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:23.986 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9370 2024-09-21T14:59:23.986 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:23.986 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:23.986 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:23.987 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:23.987 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157279+0000", 2024-09-21T14:59:23.987 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:23.987 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:23.987 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:23.987 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:23.987 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157279+0000", 2024-09-21T14:59:23.987 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:23.987 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:23.987 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:23.987 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:23.987 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157279+0000", 2024-09-21T14:59:23.987 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:23.987 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:23.987 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:23.988 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:23.988 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157282+0000", 2024-09-21T14:59:23.988 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2229999999999999e-06 2024-09-21T14:59:23.988 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:23.988 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:23.988 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:23.988 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157294+0000", 2024-09-21T14:59:23.988 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2173e-05 2024-09-21T14:59:23.988 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:23.988 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:23.988 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:23.988 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157300+0000", 2024-09-21T14:59:23.988 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.93e-06 2024-09-21T14:59:23.988 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:23.988 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:23.988 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:23.989 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175569+0000", 2024-09-21T14:59:23.989 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018268376999999999 2024-09-21T14:59:23.989 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:23.989 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:23.989 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:23.989 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175622+0000", 2024-09-21T14:59:23.989 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3808999999999999e-05 2024-09-21T14:59:23.989 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:23.989 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:23.989 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:23.989 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175752+0000", 2024-09-21T14:59:23.989 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012938700000000001 2024-09-21T14:59:23.989 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:23.989 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:23.990 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:23.990 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:23.990 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:23.990 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9371 3.3 3:c0bf6a9b:::rbd_data.11363a21d2be.0000000000000000:10 [read 73728~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:23.990 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157306+0000", 2024-09-21T14:59:23.990 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 44.810942699000002, 2024-09-21T14:59:23.990 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018755473000000002, 2024-09-21T14:59:23.990 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:23.990 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:23.990 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:23.990 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:23.990 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:23.990 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9371 2024-09-21T14:59:23.990 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:23.991 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:23.991 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:23.991 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:23.991 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157306+0000", 2024-09-21T14:59:23.991 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:23.991 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:23.991 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:23.991 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:23.991 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157306+0000", 2024-09-21T14:59:23.991 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:23.991 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:23.991 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:23.991 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:23.991 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157305+0000", 2024-09-21T14:59:23.991 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:23.991 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:23.992 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:23.992 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:23.992 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157312+0000", 2024-09-21T14:59:23.992 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3289999999999996e-06 2024-09-21T14:59:23.992 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:23.992 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:23.992 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:23.992 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157324+0000", 2024-09-21T14:59:23.992 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1725e-05 2024-09-21T14:59:23.992 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:23.992 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:23.992 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:23.992 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157339+0000", 2024-09-21T14:59:23.992 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5211999999999999e-05 2024-09-21T14:59:23.992 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:23.993 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:23.993 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:23.993 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175798+0000", 2024-09-21T14:59:23.993 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018458781 2024-09-21T14:59:23.993 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:23.993 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:23.993 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:23.993 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175899+0000", 2024-09-21T14:59:23.993 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000100784 2024-09-21T14:59:23.993 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:23.993 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:23.993 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:23.993 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176061+0000", 2024-09-21T14:59:23.993 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00016216099999999999 2024-09-21T14:59:23.993 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:23.994 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:23.994 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:23.994 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:23.994 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:23.994 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9372 3.3 3:c0bf6a9b:::rbd_data.11363a21d2be.0000000000000000:10 [read 212992~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:23.994 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157345+0000", 2024-09-21T14:59:23.994 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 44.810903353, 2024-09-21T14:59:23.994 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018926493999999999, 2024-09-21T14:59:23.994 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:23.994 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:23.994 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:23.994 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:23.994 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:23.994 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9372 2024-09-21T14:59:23.995 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:23.995 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:23.995 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:23.995 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:23.995 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157345+0000", 2024-09-21T14:59:23.995 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:23.995 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:23.995 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:23.995 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:23.995 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157345+0000", 2024-09-21T14:59:23.995 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:23.995 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:23.995 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:23.995 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:23.996 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157344+0000", 2024-09-21T14:59:23.996 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:23.996 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:23.996 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:23.996 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:23.996 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157348+0000", 2024-09-21T14:59:23.996 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1389999999999999e-06 2024-09-21T14:59:23.996 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:23.996 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:23.996 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:23.996 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157372+0000", 2024-09-21T14:59:23.996 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4711000000000001e-05 2024-09-21T14:59:23.996 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:23.996 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:23.997 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:23.997 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157378+0000", 2024-09-21T14:59:23.997 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9909999999999999e-06 2024-09-21T14:59:23.997 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:23.997 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:23.997 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:23.997 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176090+0000", 2024-09-21T14:59:23.997 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018712111999999999 2024-09-21T14:59:23.997 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:23.997 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:23.997 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:23.997 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176134+0000", 2024-09-21T14:59:23.997 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3377999999999997e-05 2024-09-21T14:59:23.997 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:23.997 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:23.998 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:23.998 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176271+0000", 2024-09-21T14:59:23.998 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013763499999999999 2024-09-21T14:59:23.998 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:23.998 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:23.998 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:23.998 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:23.998 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:23.998 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9373 3.3 3:c0bf6a9b:::rbd_data.11363a21d2be.0000000000000000:10 [read 352256~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:23.998 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157384+0000", 2024-09-21T14:59:23.998 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 44.810864580999997, 2024-09-21T14:59:23.998 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019154937, 2024-09-21T14:59:23.998 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:23.998 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:23.999 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:23.999 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:23.999 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:23.999 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9373 2024-09-21T14:59:23.999 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:23.999 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:23.999 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:23.999 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:23.999 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157384+0000", 2024-09-21T14:59:23.999 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:23.999 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:23.999 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:23.999 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:24.000 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157384+0000", 2024-09-21T14:59:24.000 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:24.000 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.000 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.000 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:24.000 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157383+0000", 2024-09-21T14:59:24.000 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:24.000 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.000 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.000 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:24.000 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157386+0000", 2024-09-21T14:59:24.000 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.191e-06 2024-09-21T14:59:24.000 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.000 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.001 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:24.001 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157393+0000", 2024-09-21T14:59:24.001 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5339999999999996e-06 2024-09-21T14:59:24.001 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.001 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.001 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:24.001 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157397+0000", 2024-09-21T14:59:24.001 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5220000000000001e-06 2024-09-21T14:59:24.001 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.001 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.001 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:24.001 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176302+0000", 2024-09-21T14:59:24.001 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018905113000000001 2024-09-21T14:59:24.001 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.002 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.002 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:24.002 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176379+0000", 2024-09-21T14:59:24.002 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6372999999999995e-05 2024-09-21T14:59:24.002 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.002 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.002 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:24.002 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176539+0000", 2024-09-21T14:59:24.002 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015973800000000001 2024-09-21T14:59:24.002 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:24.002 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:24.002 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:24.002 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.002 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.003 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9374 3.3 3:c0bf6a9b:::rbd_data.11363a21d2be.0000000000000000:10 [read 491520~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:24.003 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157401+0000", 2024-09-21T14:59:24.003 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 44.810846796999996, 2024-09-21T14:59:24.003 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019387755, 2024-09-21T14:59:24.003 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:24.003 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:24.003 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:24.003 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:24.003 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:24.003 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9374 2024-09-21T14:59:24.003 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.003 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:24.003 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.003 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:24.004 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157401+0000", 2024-09-21T14:59:24.004 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:24.004 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.004 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.004 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:24.004 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157401+0000", 2024-09-21T14:59:24.004 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:24.004 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.004 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.004 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:24.004 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157401+0000", 2024-09-21T14:59:24.004 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:24.004 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.005 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.005 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:24.005 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157414+0000", 2024-09-21T14:59:24.005 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2417e-05 2024-09-21T14:59:24.005 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.005 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.005 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:24.005 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157424+0000", 2024-09-21T14:59:24.005 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0861e-05 2024-09-21T14:59:24.005 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.005 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.005 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:24.005 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157429+0000", 2024-09-21T14:59:24.005 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0499999999999999e-06 2024-09-21T14:59:24.006 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.006 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.006 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:24.006 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176596+0000", 2024-09-21T14:59:24.006 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019166552 2024-09-21T14:59:24.006 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.006 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.006 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:24.006 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176647+0000", 2024-09-21T14:59:24.006 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1112000000000001e-05 2024-09-21T14:59:24.006 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.006 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.006 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:24.006 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176789+0000", 2024-09-21T14:59:24.007 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014209300000000001 2024-09-21T14:59:24.007 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:24.007 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:24.007 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:24.007 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.007 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.007 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9401 3.3 3:c725132c:::rbd_data.11363a21d2be.0000000000000007:10 [read 32768~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:24.007 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157621+0000", 2024-09-21T14:59:24.007 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 44.810626988000003, 2024-09-21T14:59:24.007 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019381711999999999, 2024-09-21T14:59:24.007 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:24.007 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:24.007 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:24.007 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:24.007 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:24.008 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9401 2024-09-21T14:59:24.008 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.008 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:24.008 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.008 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:24.008 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157621+0000", 2024-09-21T14:59:24.008 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:24.008 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.008 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.008 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:24.008 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157621+0000", 2024-09-21T14:59:24.008 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:24.008 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.009 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.009 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:24.009 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157620+0000", 2024-09-21T14:59:24.009 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:59:24.009 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.009 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.009 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:24.009 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157626+0000", 2024-09-21T14:59:24.009 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7570000000000001e-06 2024-09-21T14:59:24.009 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.009 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.009 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:24.009 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157636+0000", 2024-09-21T14:59:24.009 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8339999999999994e-06 2024-09-21T14:59:24.009 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.009 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.010 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:24.010 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157643+0000", 2024-09-21T14:59:24.010 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9439999999999999e-06 2024-09-21T14:59:24.010 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.010 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.010 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:24.010 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176816+0000", 2024-09-21T14:59:24.010 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019172872000000001 2024-09-21T14:59:24.010 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.010 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.010 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:24.010 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176878+0000", 2024-09-21T14:59:24.010 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1734000000000002e-05 2024-09-21T14:59:24.010 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.010 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.010 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:24.011 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177003+0000", 2024-09-21T14:59:24.011 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012537400000000001 2024-09-21T14:59:24.011 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:24.011 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:24.011 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:24.011 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.011 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.011 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9402 3.3 3:c725132c:::rbd_data.11363a21d2be.0000000000000007:10 [read 172032~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:24.011 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157649+0000", 2024-09-21T14:59:24.011 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 44.810599115999999, 2024-09-21T14:59:24.011 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019567404, 2024-09-21T14:59:24.011 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:24.011 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:24.011 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:24.011 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:24.012 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:24.012 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9402 2024-09-21T14:59:24.012 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.012 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:24.012 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.012 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:24.012 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157649+0000", 2024-09-21T14:59:24.012 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:24.012 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.012 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.012 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:24.012 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157649+0000", 2024-09-21T14:59:24.012 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:24.012 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.012 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.012 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:24.013 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157649+0000", 2024-09-21T14:59:24.013 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:24.013 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.013 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.013 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:24.013 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157652+0000", 2024-09-21T14:59:24.013 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8779999999999998e-06 2024-09-21T14:59:24.013 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.013 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.013 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:24.013 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157657+0000", 2024-09-21T14:59:24.013 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6659999999999996e-06 2024-09-21T14:59:24.013 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.013 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.013 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:24.013 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157662+0000", 2024-09-21T14:59:24.014 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5519999999999998e-06 2024-09-21T14:59:24.014 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.014 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.014 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:24.014 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177040+0000", 2024-09-21T14:59:24.014 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019377807 2024-09-21T14:59:24.014 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.014 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.014 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:24.014 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177085+0000", 2024-09-21T14:59:24.014 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5195999999999999e-05 2024-09-21T14:59:24.014 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.014 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.015 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:24.015 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177217+0000", 2024-09-21T14:59:24.015 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013172600000000001 2024-09-21T14:59:24.015 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:24.015 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:24.015 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:24.015 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.015 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.015 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9403 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:24.015 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157949+0000", 2024-09-21T14:59:24.015 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 44.810298787000001, 2024-09-21T14:59:24.015 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023540373999999999, 2024-09-21T14:59:24.015 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:24.015 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:24.015 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:24.015 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:24.016 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:24.016 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9403 2024-09-21T14:59:24.016 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.016 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:24.016 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.016 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:24.016 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157949+0000", 2024-09-21T14:59:24.016 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:24.016 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.016 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.016 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:24.016 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157949+0000", 2024-09-21T14:59:24.016 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:24.016 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.016 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.017 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:24.017 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157948+0000", 2024-09-21T14:59:24.017 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T14:59:24.017 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.017 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.017 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:24.017 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157956+0000", 2024-09-21T14:59:24.017 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7999999999999999e-06 2024-09-21T14:59:24.017 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.017 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.017 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:24.017 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157969+0000", 2024-09-21T14:59:24.017 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2792000000000001e-05 2024-09-21T14:59:24.017 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.017 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.017 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:24.017 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157977+0000", 2024-09-21T14:59:24.018 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.704e-06 2024-09-21T14:59:24.018 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.018 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.018 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:24.018 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177246+0000", 2024-09-21T14:59:24.018 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019268264 2024-09-21T14:59:24.018 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.018 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.018 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:24.018 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177315+0000", 2024-09-21T14:59:24.018 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9480999999999994e-05 2024-09-21T14:59:24.018 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.018 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.018 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:59:24.018 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177615+0000", 2024-09-21T14:59:24.019 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00029929499999999999 2024-09-21T14:59:24.019 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.019 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.019 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:24.019 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181333+0000", 2024-09-21T14:59:24.019 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0037183149999999998 2024-09-21T14:59:24.019 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.019 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.019 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:24.019 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181374+0000", 2024-09-21T14:59:24.019 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.0723999999999998e-05 2024-09-21T14:59:24.019 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.019 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.019 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:24.019 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181434+0000", 2024-09-21T14:59:24.020 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0024e-05 2024-09-21T14:59:24.020 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.020 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.020 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:24.020 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181490+0000", 2024-09-21T14:59:24.020 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6245000000000001e-05 2024-09-21T14:59:24.020 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:24.020 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:24.020 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:24.020 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.020 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.020 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9404 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:24.020 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.160480+0000", 2024-09-21T14:59:24.020 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 44.807768437, 2024-09-21T14:59:24.020 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021137686999999999, 2024-09-21T14:59:24.020 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:24.020 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:24.021 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:24.021 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:24.021 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:24.021 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9404 2024-09-21T14:59:24.021 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.021 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:24.021 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.021 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:24.021 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160480+0000", 2024-09-21T14:59:24.021 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:24.021 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.021 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.021 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:24.021 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160480+0000", 2024-09-21T14:59:24.021 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:24.021 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.022 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.022 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:24.022 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160478+0000", 2024-09-21T14:59:24.022 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T14:59:24.022 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.022 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.022 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:24.022 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160486+0000", 2024-09-21T14:59:24.022 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9969999999999998e-06 2024-09-21T14:59:24.022 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.022 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.022 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:24.022 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160502+0000", 2024-09-21T14:59:24.022 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5650000000000001e-05 2024-09-21T14:59:24.022 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.023 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.023 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:24.023 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160511+0000", 2024-09-21T14:59:24.023 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.3610000000000007e-06 2024-09-21T14:59:24.023 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.023 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.023 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:24.023 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.178035+0000", 2024-09-21T14:59:24.023 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.017523520000000001 2024-09-21T14:59:24.023 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.023 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.023 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:24.023 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.178151+0000", 2024-09-21T14:59:24.023 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000116013 2024-09-21T14:59:24.023 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.023 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.024 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:59:24.024 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.178408+0000", 2024-09-21T14:59:24.024 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00025743400000000002 2024-09-21T14:59:24.024 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.024 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.024 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:24.024 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181501+0000", 2024-09-21T14:59:24.024 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.003092891 2024-09-21T14:59:24.024 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.024 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.024 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:24.024 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181538+0000", 2024-09-21T14:59:24.024 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6977999999999997e-05 2024-09-21T14:59:24.024 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.024 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.024 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:24.025 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181582+0000", 2024-09-21T14:59:24.025 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3849e-05 2024-09-21T14:59:24.025 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.025 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.025 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:24.025 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181617+0000", 2024-09-21T14:59:24.025 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5423000000000003e-05 2024-09-21T14:59:24.025 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:24.025 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:24.025 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:24.025 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.025 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.025 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9406 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:24.025 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.166040+0000", 2024-09-21T14:59:24.025 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 44.802208557999997, 2024-09-21T14:59:24.025 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018094524000000001, 2024-09-21T14:59:24.026 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:24.026 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:24.026 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:24.026 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:24.026 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:24.026 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9406 2024-09-21T14:59:24.026 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.026 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:24.026 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.026 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:24.026 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.166040+0000", 2024-09-21T14:59:24.026 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:24.026 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.026 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.027 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:24.027 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.166040+0000", 2024-09-21T14:59:24.027 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:24.027 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.027 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.027 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:24.027 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.166038+0000", 2024-09-21T14:59:24.027 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T14:59:24.027 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.027 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.027 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:24.027 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.166051+0000", 2024-09-21T14:59:24.027 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2638e-05 2024-09-21T14:59:24.027 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.027 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.027 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:24.028 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.166070+0000", 2024-09-21T14:59:24.028 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8691000000000001e-05 2024-09-21T14:59:24.028 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.028 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.028 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:24.028 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.166089+0000", 2024-09-21T14:59:24.028 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9286000000000001e-05 2024-09-21T14:59:24.028 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.028 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.028 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:24.028 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.180194+0000", 2024-09-21T14:59:24.028 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014105039999999999 2024-09-21T14:59:24.028 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.028 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.028 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:24.028 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.180340+0000", 2024-09-21T14:59:24.029 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014623199999999999 2024-09-21T14:59:24.029 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.029 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.029 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:59:24.029 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.180818+0000", 2024-09-21T14:59:24.029 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000477489 2024-09-21T14:59:24.029 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.029 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.029 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:24.029 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.183808+0000", 2024-09-21T14:59:24.029 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0029897610000000001 2024-09-21T14:59:24.029 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.029 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.029 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:24.029 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.183821+0000", 2024-09-21T14:59:24.029 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3251999999999999e-05 2024-09-21T14:59:24.030 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.030 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.030 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:24.030 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.183872+0000", 2024-09-21T14:59:24.030 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0900999999999999e-05 2024-09-21T14:59:24.030 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.030 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.030 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:24.030 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.184134+0000", 2024-09-21T14:59:24.030 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000262484 2024-09-21T14:59:24.030 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:24.030 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:24.030 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:24.030 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.030 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.030 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9407 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:24.031 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.168094+0000", 2024-09-21T14:59:24.031 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 44.800153938000001, 2024-09-21T14:59:24.031 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019444860000000001, 2024-09-21T14:59:24.031 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:24.031 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:24.031 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:24.031 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:24.031 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:24.031 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9407 2024-09-21T14:59:24.031 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.031 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:24.031 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.031 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:24.031 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.168094+0000", 2024-09-21T14:59:24.031 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:24.032 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.032 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.032 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:24.032 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.168094+0000", 2024-09-21T14:59:24.032 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:24.032 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.032 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.032 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:24.032 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.168093+0000", 2024-09-21T14:59:24.032 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:59:24.032 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.032 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.032 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:24.032 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.168119+0000", 2024-09-21T14:59:24.032 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5803999999999999e-05 2024-09-21T14:59:24.033 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.033 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.033 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:24.033 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.168135+0000", 2024-09-21T14:59:24.033 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6359999999999999e-05 2024-09-21T14:59:24.033 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.033 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.033 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:24.033 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.168144+0000", 2024-09-21T14:59:24.033 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4270000000000008e-06 2024-09-21T14:59:24.033 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.033 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.033 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:24.033 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181943+0000", 2024-09-21T14:59:24.033 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013799617 2024-09-21T14:59:24.034 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.034 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.034 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:24.034 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.182052+0000", 2024-09-21T14:59:24.034 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000108924 2024-09-21T14:59:24.034 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.034 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.034 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:59:24.034 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.183431+0000", 2024-09-21T14:59:24.034 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0013783619999999999 2024-09-21T14:59:24.034 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.034 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.034 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:24.035 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.187276+0000", 2024-09-21T14:59:24.035 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0038450469999999999 2024-09-21T14:59:24.035 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.035 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.035 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:24.035 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.187288+0000", 2024-09-21T14:59:24.035 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2414e-05 2024-09-21T14:59:24.035 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.035 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.035 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:24.035 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.187414+0000", 2024-09-21T14:59:24.035 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012614399999999999 2024-09-21T14:59:24.035 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.035 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.035 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:24.036 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.187539+0000", 2024-09-21T14:59:24.036 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012481800000000001 2024-09-21T14:59:24.036 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:24.036 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:24.036 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:24.036 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.036 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.036 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9410 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:24.036 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.174103+0000", 2024-09-21T14:59:24.036 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 44.794145028999999, 2024-09-21T14:59:24.036 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025493703, 2024-09-21T14:59:24.036 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:24.036 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:24.036 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:24.036 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:24.037 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:24.037 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9410 2024-09-21T14:59:24.037 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.037 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:24.037 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.037 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:24.037 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.174103+0000", 2024-09-21T14:59:24.037 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:24.037 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.037 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.037 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:24.037 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.174103+0000", 2024-09-21T14:59:24.037 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:24.037 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.037 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.037 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:24.038 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.174102+0000", 2024-09-21T14:59:24.038 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:59:24.038 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.038 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.038 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:24.038 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.174108+0000", 2024-09-21T14:59:24.038 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1149999999999999e-06 2024-09-21T14:59:24.038 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.038 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.038 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:24.038 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.174117+0000", 2024-09-21T14:59:24.038 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4870000000000002e-06 2024-09-21T14:59:24.038 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.038 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.038 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:24.039 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.174122+0000", 2024-09-21T14:59:24.039 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9790000000000001e-06 2024-09-21T14:59:24.039 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.039 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.039 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:24.039 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.187830+0000", 2024-09-21T14:59:24.039 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013708048 2024-09-21T14:59:24.039 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.039 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.039 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:24.039 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.187913+0000", 2024-09-21T14:59:24.039 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2763999999999996e-05 2024-09-21T14:59:24.039 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.039 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.039 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:59:24.040 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.188230+0000", 2024-09-21T14:59:24.040 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00031745100000000001 2024-09-21T14:59:24.040 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.040 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.040 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:24.040 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.192708+0000", 2024-09-21T14:59:24.040 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0044783890000000002 2024-09-21T14:59:24.040 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.040 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.040 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:24.040 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199487+0000", 2024-09-21T14:59:24.040 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0067782149999999998 2024-09-21T14:59:24.040 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.040 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.040 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:24.040 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199560+0000", 2024-09-21T14:59:24.041 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3752999999999997e-05 2024-09-21T14:59:24.041 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.041 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.041 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:24.041 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199597+0000", 2024-09-21T14:59:24.041 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6526999999999998e-05 2024-09-21T14:59:24.041 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:24.041 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:24.041 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:24.041 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.041 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.041 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9411 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:24.041 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.175943+0000", 2024-09-21T14:59:24.041 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 44.79230501, 2024-09-21T14:59:24.041 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023773181000000001, 2024-09-21T14:59:24.042 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:24.042 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:24.042 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:24.042 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:24.042 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:24.042 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9411 2024-09-21T14:59:24.042 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.042 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:24.042 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.042 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:24.042 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175943+0000", 2024-09-21T14:59:24.042 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:24.042 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.042 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.042 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:24.043 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175943+0000", 2024-09-21T14:59:24.043 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:24.043 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.043 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.043 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:24.043 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175942+0000", 2024-09-21T14:59:24.043 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:59:24.043 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.043 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.043 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:24.043 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175948+0000", 2024-09-21T14:59:24.043 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9689999999999999e-06 2024-09-21T14:59:24.043 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.043 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.044 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:24.044 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175960+0000", 2024-09-21T14:59:24.044 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1745e-05 2024-09-21T14:59:24.044 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.044 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.044 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:24.044 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175965+0000", 2024-09-21T14:59:24.044 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.203e-06 2024-09-21T14:59:24.044 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.044 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.044 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:24.044 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.189900+0000", 2024-09-21T14:59:24.044 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013934940999999999 2024-09-21T14:59:24.045 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.045 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.045 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:24.045 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.189988+0000", 2024-09-21T14:59:24.045 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8419999999999997e-05 2024-09-21T14:59:24.045 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.045 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.045 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:59:24.045 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.190321+0000", 2024-09-21T14:59:24.045 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00033231800000000001 2024-09-21T14:59:24.045 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.045 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.045 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:24.045 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.192746+0000", 2024-09-21T14:59:24.046 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0024254400000000001 2024-09-21T14:59:24.046 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.046 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.046 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:24.046 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199605+0000", 2024-09-21T14:59:24.046 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0068582809999999999 2024-09-21T14:59:24.046 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.046 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.046 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:24.046 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199680+0000", 2024-09-21T14:59:24.046 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5630000000000006e-05 2024-09-21T14:59:24.046 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.046 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.046 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:24.046 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199716+0000", 2024-09-21T14:59:24.047 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6248999999999998e-05 2024-09-21T14:59:24.047 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:24.047 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:24.047 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:24.047 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.047 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.047 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9412 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:24.047 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.177402+0000", 2024-09-21T14:59:24.047 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 44.790845912999998, 2024-09-21T14:59:24.047 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022445590000000001, 2024-09-21T14:59:24.047 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:24.047 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:24.047 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:24.047 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:24.047 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:24.048 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9412 2024-09-21T14:59:24.048 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.048 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:24.048 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.048 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:24.048 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177402+0000", 2024-09-21T14:59:24.048 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:24.048 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.048 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.048 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:24.048 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177402+0000", 2024-09-21T14:59:24.048 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:24.048 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.048 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.048 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:24.049 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177401+0000", 2024-09-21T14:59:24.049 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:59:24.049 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.049 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.049 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:24.049 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177409+0000", 2024-09-21T14:59:24.049 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4710000000000001e-06 2024-09-21T14:59:24.049 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.049 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.049 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:24.049 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177419+0000", 2024-09-21T14:59:24.049 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0351e-05 2024-09-21T14:59:24.049 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.049 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.049 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:24.050 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177424+0000", 2024-09-21T14:59:24.050 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1919999999999996e-06 2024-09-21T14:59:24.050 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.050 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.050 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:24.050 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.191343+0000", 2024-09-21T14:59:24.050 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.01391878 2024-09-21T14:59:24.050 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.050 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.050 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:24.050 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.191446+0000", 2024-09-21T14:59:24.050 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000102453 2024-09-21T14:59:24.050 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.050 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.051 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:59:24.051 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.192200+0000", 2024-09-21T14:59:24.051 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00075476899999999997 2024-09-21T14:59:24.051 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.051 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.051 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:24.051 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.193866+0000", 2024-09-21T14:59:24.051 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0016657519999999999 2024-09-21T14:59:24.051 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.051 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.051 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:24.051 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199724+0000", 2024-09-21T14:59:24.051 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0058579529999999999 2024-09-21T14:59:24.051 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.051 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.052 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:24.052 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199803+0000", 2024-09-21T14:59:24.052 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8662e-05 2024-09-21T14:59:24.052 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.052 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.052 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:24.052 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199848+0000", 2024-09-21T14:59:24.052 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5216000000000002e-05 2024-09-21T14:59:24.052 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:24.052 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:24.052 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:24.052 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.052 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.052 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:10445 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:24.052 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.271980+0000", 2024-09-21T14:59:24.052 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 44.696268547999999, 2024-09-21T14:59:24.052 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018664256000000001, 2024-09-21T14:59:24.053 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:24.053 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:24.053 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:24.053 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:24.053 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:24.053 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 10445 2024-09-21T14:59:24.053 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.053 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:24.053 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.053 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:24.053 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.271980+0000", 2024-09-21T14:59:24.053 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:24.053 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.054 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.054 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:24.054 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.271980+0000", 2024-09-21T14:59:24.054 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:24.054 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.054 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.054 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:24.054 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.271979+0000", 2024-09-21T14:59:24.054 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:24.054 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.054 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.054 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:24.054 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.271982+0000", 2024-09-21T14:59:24.054 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7240000000000001e-06 2024-09-21T14:59:24.055 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.055 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.055 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:24.055 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.271987+0000", 2024-09-21T14:59:24.055 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3639999999999998e-06 2024-09-21T14:59:24.055 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.055 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.055 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:24.055 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.271992+0000", 2024-09-21T14:59:24.055 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7110000000000003e-06 2024-09-21T14:59:24.055 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.055 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.055 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:24.055 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.277567+0000", 2024-09-21T14:59:24.055 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0055745009999999999 2024-09-21T14:59:24.055 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.056 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.056 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:24.056 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.277655+0000", 2024-09-21T14:59:24.056 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8465000000000001e-05 2024-09-21T14:59:24.056 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.056 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.056 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:59:24.056 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.278720+0000", 2024-09-21T14:59:24.056 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0010650569999999999 2024-09-21T14:59:24.056 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.056 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.056 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:24.056 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.281153+0000", 2024-09-21T14:59:24.056 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0024326389999999999 2024-09-21T14:59:24.056 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.056 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.057 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:24.057 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.290574+0000", 2024-09-21T14:59:24.057 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0094209489999999996 2024-09-21T14:59:24.057 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.057 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.057 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:24.057 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.290613+0000", 2024-09-21T14:59:24.057 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.8803000000000001e-05 2024-09-21T14:59:24.057 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.057 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.057 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:24.057 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.290644+0000", 2024-09-21T14:59:24.057 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1404e-05 2024-09-21T14:59:24.057 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:24.057 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:24.058 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:24.058 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.058 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.058 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22769 3.f 3:f39bd56f:::rbd_data.11366acc894.0000000000000004:19 [read 1032192~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:24.058 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.140207+0000", 2024-09-21T14:59:24.058 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 33.828040807999997, 2024-09-21T14:59:24.058 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018031170999999999, 2024-09-21T14:59:24.058 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:24.058 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:24.058 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:24.058 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:24.058 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:24.058 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22769 2024-09-21T14:59:24.058 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.059 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:24.059 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.059 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:24.059 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.140207+0000", 2024-09-21T14:59:24.059 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:24.059 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.059 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.059 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:24.059 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.140207+0000", 2024-09-21T14:59:24.059 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:24.059 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.059 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.059 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:24.059 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.140207+0000", 2024-09-21T14:59:24.059 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:24.059 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.059 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.060 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:24.060 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.140209+0000", 2024-09-21T14:59:24.060 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.305e-06 2024-09-21T14:59:24.060 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.060 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.060 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:24.060 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.140214+0000", 2024-09-21T14:59:24.060 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9570000000000001e-06 2024-09-21T14:59:24.060 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.060 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.060 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:24.060 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.140219+0000", 2024-09-21T14:59:24.060 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2250000000000002e-06 2024-09-21T14:59:24.060 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.060 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.060 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:24.061 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.157990+0000", 2024-09-21T14:59:24.061 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.017770939999999999 2024-09-21T14:59:24.061 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.061 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.061 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:24.061 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.158062+0000", 2024-09-21T14:59:24.061 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2652999999999997e-05 2024-09-21T14:59:24.061 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.061 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.061 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:24.061 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.158239+0000", 2024-09-21T14:59:24.061 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017640599999999999 2024-09-21T14:59:24.061 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:24.061 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:24.061 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:24.061 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.062 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.062 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:23084 3.3 3:c55e343c:::rbd_data.11366acc894.0000000000000006:19 [read 1032192~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:24.062 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.150724+0000", 2024-09-21T14:59:24.062 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 33.817523973, 2024-09-21T14:59:24.062 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018916460999999999, 2024-09-21T14:59:24.062 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:24.062 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:24.062 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:24.062 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:24.062 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:24.062 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 23084 2024-09-21T14:59:24.062 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.062 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:24.062 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.062 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:24.063 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.150724+0000", 2024-09-21T14:59:24.063 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:24.063 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.063 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.063 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:24.063 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.150724+0000", 2024-09-21T14:59:24.063 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:24.063 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.063 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.063 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:24.063 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.150724+0000", 2024-09-21T14:59:24.063 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:24.063 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.063 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.063 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:24.063 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.150727+0000", 2024-09-21T14:59:24.064 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7250000000000002e-06 2024-09-21T14:59:24.064 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.064 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.064 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:24.064 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.150732+0000", 2024-09-21T14:59:24.064 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5809999999999996e-06 2024-09-21T14:59:24.064 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.064 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.064 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:24.064 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.150737+0000", 2024-09-21T14:59:24.064 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3590000000000001e-06 2024-09-21T14:59:24.064 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.064 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.064 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:24.064 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.166694+0000", 2024-09-21T14:59:24.065 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.015957720000000002 2024-09-21T14:59:24.065 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.065 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.065 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:24.065 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.166769+0000", 2024-09-21T14:59:24.065 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5143999999999997e-05 2024-09-21T14:59:24.065 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:24.065 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:24.065 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:24.065 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.169641+0000", 2024-09-21T14:59:24.065 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0028713089999999998 2024-09-21T14:59:24.065 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:24.065 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:24.065 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:24.066 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:24.066 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:24.066 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:24.067 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T14:59:24.067 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:59:24.065+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:59:24.168 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:59:24.165+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:59:24.174 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T14:59:24.182 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:24.182 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:59:24.182 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T14:59:24.182 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:24.193 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T14:59:24.269 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:59:24.269+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:59:24.320 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:24.321 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:59:24.321 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T14:59:24.321 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T14:59:24.321 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:24.333 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T14:59:24.349 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T14:59:24.365 INFO:teuthology.orchestra.run.smithi032.stdout:checking clone #13, image image_client.0-clone13 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent14 2024-09-21T14:59:24.370 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:59:24.367+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:59:24.382 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T14:59:24.405 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T14:59:24.425 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T14:59:24.447 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-21T14:59:24.462 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:24.462 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:24.470 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:59:24.467+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:59:24.480 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T14:59:24.482 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T14:59:24.506 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T14:59:24.527 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T14:59:24.529 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:59:24.529 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:59:24.529 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:59:24.529 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:59:24.530 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:59:24.530 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:59:24.530 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:59:24.530 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:59:24.571 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:59:24.567+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:59:24.579 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:24.579 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:24.588 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T14:59:24.672 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:59:24.667+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:59:24.746 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:24.746 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:24.756 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T14:59:24.772 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:59:24.773+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:59:24.865 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:24.865 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:24.873 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:59:24.873+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:59:24.874 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T14:59:24.974 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:59:24.971+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:59:25.029 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:25.029 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:25.039 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T14:59:25.075 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:59:25.071+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:59:25.154 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:25.154 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:25.166 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T14:59:25.176 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:59:25.173+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:59:25.212 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T14:59:25.273 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:25.273 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:25.277 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:59:25.271+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:59:25.284 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T14:59:25.370 INFO:teuthology.orchestra.run.smithi032.stdout:1627 punch from 0xb8ed697 to 0xb8fcdf6, (0xf75f bytes) 2024-09-21T14:59:25.378 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:59:25.375+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:59:25.380 INFO:teuthology.orchestra.run.smithi032.stdout:1628 write 0xc53b3fa thru 0xc5423b5 (0x6fbc bytes) 2024-09-21T14:59:25.399 INFO:teuthology.orchestra.run.smithi032.stdout:1629 write 0xee01202 thru 0xee10165 (0xef64 bytes) 2024-09-21T14:59:25.406 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:25.406 INFO:teuthology.orchestra.run.smithi032.stdout:1632 read 0x9fded00 thru 0x9fe55a4 (0x68a5 bytes) 2024-09-21T14:59:25.406 INFO:teuthology.orchestra.run.smithi032.stdout:1633 read 0x346ed51 thru 0x3478da0 (0xa050 bytes) 2024-09-21T14:59:25.406 INFO:teuthology.orchestra.run.smithi032.stdout:1635 punch from 0x25ed108 to 0x25f3163, (0x605b bytes) 2024-09-21T14:59:25.406 INFO:teuthology.orchestra.run.smithi032.stdout:1636 punch from 0x1f30151 to 0x1f30947, (0x7f6 bytes) 2024-09-21T14:59:25.407 INFO:teuthology.orchestra.run.smithi032.stdout:1638 read 0xc30be41 thru 0xc30e37e (0x253e bytes) 2024-09-21T14:59:25.407 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:25.407 INFO:teuthology.orchestra.run.smithi032.stdout:1644 punch from 0x23fd8c7 to 0x24072a9, (0x99e2 bytes) 2024-09-21T14:59:25.407 INFO:teuthology.orchestra.run.smithi032.stdout:1646 read 0xafb3e2b thru 0xafb95c4 (0x579a bytes) 2024-09-21T14:59:25.407 INFO:teuthology.orchestra.run.smithi032.stdout:1647 write 0x8eb2991 thru 0x8ebae7a (0x84ea bytes) 2024-09-21T14:59:25.409 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:25.409 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:25.413 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:25.413 INFO:teuthology.orchestra.run.smithi032.stdout:1649 read 0xc0472ab thru 0xc05585b (0xe5b1 bytes) 2024-09-21T14:59:25.413 INFO:teuthology.orchestra.run.smithi032.stdout:1650 write 0x5f8d3a1 thru 0x5f91f10 (0x4b70 bytes) 2024-09-21T14:59:25.419 INFO:teuthology.orchestra.run.smithi032.stdout:1653 read 0xdc5fba3 thru 0xdc6714c (0x75aa bytes) 2024-09-21T14:59:25.420 INFO:teuthology.orchestra.run.smithi032.stdout:1655 read 0x53c5098 thru 0x53d0811 (0xb77a bytes) 2024-09-21T14:59:25.421 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T14:59:25.421 INFO:teuthology.orchestra.run.smithi032.stdout:1656 read 0x1d7fdfc thru 0x1d89df9 (0x9ffe bytes) 2024-09-21T14:59:25.422 INFO:teuthology.orchestra.run.smithi032.stdout:1657 read 0x4b36250 thru 0x4b36f6e (0xd1f bytes) 2024-09-21T14:59:25.422 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:25.422 INFO:teuthology.orchestra.run.smithi032.stdout:1660 read 0xa9eec03 thru 0xa9f3f75 (0x5373 bytes) 2024-09-21T14:59:25.433 INFO:teuthology.orchestra.run.smithi032.stdout:1661 trunc from 0xee10166 to 0x875d8c4 2024-09-21T14:59:25.459 INFO:teuthology.orchestra.run.smithi032.stdout:1662 trunc from 0x875d8c4 to 0xacabafd 2024-09-21T14:59:25.468 INFO:teuthology.orchestra.run.smithi032.stdout:1663 write 0x255002e thru 0x2552d18 (0x2ceb bytes) 2024-09-21T14:59:25.471 INFO:teuthology.orchestra.run.smithi032.stdout:1664 write 0x33abb8b thru 0x33abff7 (0x46d bytes) 2024-09-21T14:59:25.476 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:25.476 INFO:teuthology.orchestra.run.smithi032.stdout:1666 read 0x454898f thru 0x45513d7 (0x8a49 bytes) 2024-09-21T14:59:25.476 INFO:teuthology.orchestra.run.smithi032.stdout:1667 write 0x1f02f92 thru 0x1f0e097 (0xb106 bytes) 2024-09-21T14:59:25.479 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:59:25.475+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:59:25.484 INFO:teuthology.orchestra.run.smithi032.stdout:1668 read 0x1400399 thru 0x1408364 (0x7fcc bytes) 2024-09-21T14:59:25.485 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:25.485 INFO:teuthology.orchestra.run.smithi032.stdout:1671 punch from 0x50eccaa to 0x50f1bca, (0x4f20 bytes) 2024-09-21T14:59:25.485 INFO:teuthology.orchestra.run.smithi032.stdout:1672 punch from 0x7cfe1d2 to 0x7d04080, (0x5eae bytes) 2024-09-21T14:59:25.488 INFO:teuthology.orchestra.run.smithi032.stdout:1673 trunc from 0xacabafd to 0x8fd4335 2024-09-21T14:59:25.502 INFO:teuthology.orchestra.run.smithi032.stdout:1674 read 0x641947e thru 0x6425eca (0xca4d bytes) 2024-09-21T14:59:25.512 INFO:teuthology.orchestra.run.smithi032.stdout:1677 trunc from 0x8fd4335 to 0x39d1ef8 2024-09-21T14:59:25.528 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:25.529 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:25.535 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:25.535 INFO:teuthology.orchestra.run.smithi032.stdout:1679 punch from 0x23eceaa to 0x23f6a59, (0x9baf bytes) 2024-09-21T14:59:25.535 INFO:teuthology.orchestra.run.smithi032.stdout:1681 read 0x5f9439 thru 0x603695 (0xa25d bytes) 2024-09-21T14:59:25.538 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T14:59:25.555 INFO:teuthology.orchestra.run.smithi032.stdout:1682 writesame 0x8c21d9b thru 0x8c2a0ce data_size 0x6(0x8334 bytes) 2024-09-21T14:59:25.555 INFO:teuthology.orchestra.run.smithi032.stdout:1683 write 0x2f96e60 thru 0x2fa2f0d (0xc0ae bytes) 2024-09-21T14:59:25.562 INFO:teuthology.orchestra.run.smithi032.stdout:1685 read 0x3ceb3f1 thru 0x3cec89b (0x14ab bytes) 2024-09-21T14:59:25.563 INFO:teuthology.orchestra.run.smithi032.stdout:1686 read 0x4a66089 thru 0x4a66f92 (0xf0a bytes) 2024-09-21T14:59:25.563 INFO:teuthology.orchestra.run.smithi032.stdout:1687 read 0x73737a6 thru 0x737bf96 (0x87f1 bytes) 2024-09-21T14:59:25.563 INFO:teuthology.orchestra.run.smithi032.stdout:1690 punch from 0x72c9174 to 0x72cefb2, (0x5e3e bytes) 2024-09-21T14:59:25.563 INFO:teuthology.orchestra.run.smithi032.stdout:1691 punch from 0x3308e9c to 0x3318723, (0xf887 bytes) 2024-09-21T14:59:25.565 INFO:teuthology.orchestra.run.smithi032.stdout:1692 trunc from 0x8c2a0cf to 0x7193728 2024-09-21T14:59:25.579 INFO:teuthology.orchestra.run.smithi032.stdout:1693 write 0x3d20ab2 thru 0x3d2bb2d (0xb07c bytes) 2024-09-21T14:59:25.579 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:59:25.577+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:59:25.612 INFO:teuthology.orchestra.run.smithi032.stdout:1694 write 0xcefda05 thru 0xcf00790 (0x2d8c bytes) 2024-09-21T14:59:25.640 INFO:teuthology.orchestra.run.smithi032.stdout:1695 trunc from 0xcf00791 to 0x3f0113d 2024-09-21T14:59:25.665 INFO:teuthology.orchestra.run.smithi032.stdout:1696 read 0x38ae811 thru 0x38af177 (0x967 bytes) 2024-09-21T14:59:25.670 INFO:teuthology.orchestra.run.smithi032.stdout:1697 trunc from 0x3f0113d to 0x218de38 2024-09-21T14:59:25.680 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:59:25.681+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:59:25.701 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:25.701 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:25.703 INFO:teuthology.orchestra.run.smithi032.stdout:1698 read 0x111bea7 thru 0x11243fd (0x8557 bytes) 2024-09-21T14:59:25.704 INFO:teuthology.orchestra.run.smithi032.stdout:1700 read 0x1dd273b thru 0x1de023b (0xdb01 bytes) 2024-09-21T14:59:25.705 INFO:teuthology.orchestra.run.smithi032.stdout:1701 read 0x1fa6cee thru 0x1fac691 (0x59a4 bytes) 2024-09-21T14:59:25.705 INFO:teuthology.orchestra.run.smithi032.stdout:1702 read 0xf682f9 thru 0xf7409e (0xbda6 bytes) 2024-09-21T14:59:25.711 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T14:59:25.746 INFO:teuthology.orchestra.run.smithi032.stdout:1703 write 0xe58bc52 thru 0xe58dc0a (0x1fb9 bytes) 2024-09-21T14:59:25.753 INFO:teuthology.orchestra.run.smithi032.stdout:1704 write 0x4ab8fc6 thru 0x4ac3c7b (0xacb6 bytes) 2024-09-21T14:59:25.760 INFO:teuthology.orchestra.run.smithi032.stdout:1706 write 0x6027bae thru 0x602cfb9 (0x540c bytes) 2024-09-21T14:59:25.767 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:25.767 INFO:teuthology.orchestra.run.smithi032.stdout:1709 write 0x5cf6a0d thru 0x5d06313 (0xf907 bytes) 2024-09-21T14:59:25.781 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:59:25.781+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:59:25.804 INFO:teuthology.orchestra.run.smithi032.stdout:1710 trunc from 0xe58dc0b to 0x1b0914d 2024-09-21T14:59:25.819 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:25.819 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:25.831 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T14:59:25.836 INFO:teuthology.orchestra.run.smithi032.stdout:1712 punch from 0xc2e555 to 0xc2f820, (0x12cb bytes) 2024-09-21T14:59:25.846 INFO:teuthology.orchestra.run.smithi032.stdout:1714 trunc from 0x1b0914d to 0x5bf76a0 2024-09-21T14:59:25.856 INFO:teuthology.orchestra.run.smithi032.stdout:1715 read 0x539c77d thru 0x53a9fa9 (0xd82d bytes) 2024-09-21T14:59:25.856 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:25.856 INFO:teuthology.orchestra.run.smithi032.stdout:1718 punch from 0x46f0fa4 to 0x46f8bc3, (0x7c1f bytes) 2024-09-21T14:59:25.857 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:25.857 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:25.857 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:25.871 INFO:teuthology.orchestra.run.smithi032.stdout:1724 writesame 0x85aba1d thru 0x85b0c29 data_size 0x1(0x520d bytes) 2024-09-21T14:59:25.872 INFO:teuthology.orchestra.run.smithi032.stdout:1725 punch from 0x3e444f4 to 0x3e48d40, (0x484c bytes) 2024-09-21T14:59:25.872 INFO:teuthology.orchestra.run.smithi032.stdout:1727 read 0x2ced8b4 thru 0x2cf8b46 (0xb293 bytes) 2024-09-21T14:59:25.872 INFO:teuthology.orchestra.run.smithi032.stdout:1728 read 0x3080f4c thru 0x3086537 (0x55ec bytes) 2024-09-21T14:59:25.882 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:59:25.879+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:59:25.894 INFO:teuthology.orchestra.run.smithi032.stdout:1729 write 0xe0b695d thru 0xe0c5416 (0xeaba bytes) 2024-09-21T14:59:25.901 INFO:teuthology.orchestra.run.smithi032.stdout:1730 write 0x9ead3d3 thru 0x9eafd00 (0x292e bytes) 2024-09-21T14:59:25.908 INFO:teuthology.orchestra.run.smithi032.stdout:1731 read 0xdb4a6c5 thru 0xdb4deb8 (0x37f4 bytes) 2024-09-21T14:59:25.931 INFO:teuthology.orchestra.run.smithi032.stdout:1732 trunc from 0xe0c5417 to 0x30686a9 2024-09-21T14:59:25.932 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:25.932 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:25.943 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T14:59:25.978 INFO:teuthology.orchestra.run.smithi032.stdout:1733 trunc from 0x30686a9 to 0xe46a741 2024-09-21T14:59:25.983 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:59:25.979+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:59:25.990 INFO:teuthology.orchestra.run.smithi032.stdout:1734 write 0x3ac4ddf thru 0x3ac9156 (0x4378 bytes) 2024-09-21T14:59:26.002 INFO:teuthology.orchestra.run.smithi032.stdout:1735 trunc from 0xe46a741 to 0xbaf5576 2024-09-21T14:59:26.015 INFO:teuthology.orchestra.run.smithi032.stdout:1736 write 0xc7791 thru 0xd21d6 (0xaa46 bytes) 2024-09-21T14:59:26.019 INFO:teuthology.orchestra.run.smithi032.stdout:1737 read 0x9a1ff84 thru 0x9a22fe4 (0x3061 bytes) 2024-09-21T14:59:26.019 INFO:teuthology.orchestra.run.smithi032.stdout:1738 read 0x3b5435e thru 0x3b604b2 (0xc155 bytes) 2024-09-21T14:59:26.020 INFO:teuthology.orchestra.run.smithi032.stdout:1739 write 0xa41c95c thru 0xa42a0ab (0xd750 bytes) 2024-09-21T14:59:26.026 INFO:teuthology.orchestra.run.smithi032.stdout:1740 read 0xcca4ab thru 0xcda0da (0xfc30 bytes) 2024-09-21T14:59:26.028 INFO:teuthology.orchestra.run.smithi032.stdout:1741 write 0xb43d070 thru 0xb44a082 (0xd013 bytes) 2024-09-21T14:59:26.052 INFO:teuthology.orchestra.run.smithi032.stdout:1744 write 0xdee3fbe thru 0xdee9592 (0x55d5 bytes) 2024-09-21T14:59:26.058 INFO:teuthology.orchestra.run.smithi032.stdout:1745 punch from 0x781b086 to 0x7826ce7, (0xbc61 bytes) 2024-09-21T14:59:26.059 INFO:teuthology.orchestra.run.smithi032.stdout:1746 write 0x23732fc thru 0x237b309 (0x800e bytes) 2024-09-21T14:59:26.060 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:26.061 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:26.065 INFO:teuthology.orchestra.run.smithi032.stdout:1747 read 0x46ed09e thru 0x46efef8 (0x2e5b bytes) 2024-09-21T14:59:26.066 INFO:teuthology.orchestra.run.smithi032.stdout:1748 read 0x21f691e thru 0x21f7273 (0x956 bytes) 2024-09-21T14:59:26.066 INFO:teuthology.orchestra.run.smithi032.stdout:1749 read 0x71a6d74 thru 0x71b0d3b (0x9fc8 bytes) 2024-09-21T14:59:26.066 INFO:teuthology.orchestra.run.smithi032.stdout:1750 punch from 0xc617292 to 0xc62242a, (0xb198 bytes) 2024-09-21T14:59:26.066 INFO:teuthology.orchestra.run.smithi032.stdout:1752 write 0x832b1b1 thru 0x8332332 (0x7182 bytes) 2024-09-21T14:59:26.071 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:26.071 INFO:teuthology.orchestra.run.smithi032.stdout:1754 write 0x38d2881 thru 0x38de295 (0xba15 bytes) 2024-09-21T14:59:26.072 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T14:59:26.078 INFO:teuthology.orchestra.run.smithi032.stdout:1757 read 0x3c47490 thru 0x3c4ef35 (0x7aa6 bytes) 2024-09-21T14:59:26.078 INFO:teuthology.orchestra.run.smithi032.stdout:1758 read 0x6efea85 thru 0x6f052a3 (0x681f bytes) 2024-09-21T14:59:26.078 INFO:teuthology.orchestra.run.smithi032.stdout:1759 punch from 0x7d27a03 to 0x7d31e3b, (0xa438 bytes) 2024-09-21T14:59:26.078 INFO:teuthology.orchestra.run.smithi032.stdout:1760 write 0x7f8e543 thru 0x7f94ba6 (0x6664 bytes) 2024-09-21T14:59:26.084 INFO:teuthology.orchestra.run.smithi032.stdout:1761 punch from 0xa66c02c to 0xa670a9e, (0x4a72 bytes) 2024-09-21T14:59:26.084 INFO:teuthology.orchestra.run.smithi032.stdout:1763 read 0x33de709 thru 0x33df72c (0x1024 bytes) 2024-09-21T14:59:26.085 INFO:teuthology.orchestra.run.smithi032.stdout:1764 write 0xc91ca59 thru 0xc921afc (0x50a4 bytes) 2024-09-21T14:59:26.103 INFO:teuthology.orchestra.run.smithi032.stdout:1765 trunc from 0xdee9593 to 0x8cf2736 2024-09-21T14:59:26.118 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:59:26.115+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:59:26.129 INFO:teuthology.orchestra.run.smithi032.stdout:1767 read 0x360bf8f thru 0x360f5e9 (0x365b bytes) 2024-09-21T14:59:26.129 INFO:teuthology.orchestra.run.smithi032.stdout:1768 read 0x6c6fd2b thru 0x6c7be77 (0xc14d bytes) 2024-09-21T14:59:26.129 INFO:teuthology.orchestra.run.smithi032.stdout:1770 write 0x2ee6384 thru 0x2ef5103 (0xed80 bytes) 2024-09-21T14:59:26.136 INFO:teuthology.orchestra.run.smithi032.stdout:1771 punch from 0x41c71d to 0x41d51b, (0xdfe bytes) 2024-09-21T14:59:26.136 INFO:teuthology.orchestra.run.smithi032.stdout:1772 punch from 0x31291e2 to 0x312fc3e, (0x6a5c bytes) 2024-09-21T14:59:26.136 INFO:teuthology.orchestra.run.smithi032.stdout:1773 punch from 0x7d5f1e0 to 0x7d6c8e6, (0xd706 bytes) 2024-09-21T14:59:26.136 INFO:teuthology.orchestra.run.smithi032.stdout:1774 punch from 0x2742059 to 0x27452e6, (0x328d bytes) 2024-09-21T14:59:26.149 INFO:teuthology.orchestra.run.smithi032.stdout:1776 trunc from 0x8cf2736 to 0x34c4a88 2024-09-21T14:59:26.182 INFO:teuthology.orchestra.run.smithi032.stdout:1777 read 0x1deb8af thru 0x1df51b1 (0x9903 bytes) 2024-09-21T14:59:26.184 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:59:26.179+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:59:26.189 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 5, 6, 7] out_osds: [4] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T14:59:26.190 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T14:59:26.190 INFO:tasks.thrashosds.thrasher:Adding osd 4 2024-09-21T14:59:26.190 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 4 2024-09-21T14:59:26.214 INFO:teuthology.orchestra.run.smithi032.stdout:1778 writesame 0xae2d6ae thru 0xae2f89f data_size 0x16(0x21f2 bytes) 2024-09-21T14:59:26.214 INFO:teuthology.orchestra.run.smithi032.stdout:1780 writesame 0x79ceb24 thru 0x79d179e data_size 0x1(0x2c7b bytes) 2024-09-21T14:59:26.214 INFO:teuthology.orchestra.run.smithi032.stderr:nodeep-scrub is unset 2024-09-21T14:59:26.239 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:26.239 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:26.240 INFO:teuthology.orchestra.run.smithi032.stdout:1781 writesame 0xba8fcae thru 0xba9ce61 data_size 0x2(0xd1b4 bytes) 2024-09-21T14:59:26.241 INFO:teuthology.orchestra.run.smithi032.stdout:1782 punch from 0x5014756 to 0x50225a8, (0xde52 bytes) 2024-09-21T14:59:26.241 INFO:teuthology.orchestra.run.smithi032.stdout:1783 read 0x2083c2e thru 0x2087049 (0x341c bytes) 2024-09-21T14:59:26.241 INFO:teuthology.orchestra.run.smithi032.stdout:1784 read 0x4d5b4e5 thru 0x4d653b5 (0x9ed1 bytes) 2024-09-21T14:59:26.241 INFO:teuthology.orchestra.run.smithi032.stdout:1785 write 0x571df79 thru 0x572a688 (0xc710 bytes) 2024-09-21T14:59:26.246 INFO:teuthology.orchestra.run.smithi032.stdout:1789 read 0x61e653f thru 0x61f60ba (0xfb7c bytes) 2024-09-21T14:59:26.246 INFO:teuthology.orchestra.run.smithi032.stdout:1791 read 0xb839bfd thru 0xb847758 (0xdb5c bytes) 2024-09-21T14:59:26.246 INFO:teuthology.orchestra.run.smithi032.stdout:1792 punch from 0x6ef8d88 to 0x6f06709, (0xd981 bytes) 2024-09-21T14:59:26.249 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T14:59:26.264 INFO:teuthology.orchestra.run.smithi032.stdout:1793 trunc from 0xba9ce62 to 0xe56dfc 2024-09-21T14:59:26.285 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:59:26.285+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:59:26.291 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:26.291 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:26.292 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:26.292 INFO:teuthology.orchestra.run.smithi032.stdout:1798 read 0x5dc864 thru 0x5e207d (0x581a bytes) 2024-09-21T14:59:26.311 INFO:teuthology.orchestra.run.smithi032.stdout:1799 write 0x6dda6fc thru 0x6de84d9 (0xddde bytes) 2024-09-21T14:59:26.318 INFO:teuthology.orchestra.run.smithi032.stdout:1800 writesame 0x1884cd4 thru 0x1889900 data_size 0x1(0x4c2d bytes) 2024-09-21T14:59:26.336 INFO:teuthology.orchestra.run.smithi032.stdout:1801 read 0x50db946 thru 0x50e0e3e (0x54f9 bytes) 2024-09-21T14:59:26.337 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:26.337 INFO:teuthology.orchestra.run.smithi032.stdout:1805 trunc from 0x6de84da to 0x73d6c0f 2024-09-21T14:59:26.348 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:26.348 INFO:teuthology.orchestra.run.smithi032.stdout:1809 write 0x4a359e7 thru 0x4a38ac6 (0x30e0 bytes) 2024-09-21T14:59:26.355 INFO:teuthology.orchestra.run.smithi032.stdout:1810 read 0xe21332 thru 0xe2b191 (0x9e60 bytes) 2024-09-21T14:59:26.356 INFO:teuthology.orchestra.run.smithi032.stdout:1811 punch from 0x2aeca6d to 0x2af4c6b, (0x81fe bytes) 2024-09-21T14:59:26.359 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:26.359 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:26.370 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T14:59:26.377 INFO:teuthology.orchestra.run.smithi032.stdout:1812 write 0xd277081 thru 0xd2814cf (0xa44f bytes) 2024-09-21T14:59:26.384 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:26.384 INFO:teuthology.orchestra.run.smithi032.stdout:1815 read 0xb60b233 thru 0xb61b0f5 (0xfec3 bytes) 2024-09-21T14:59:26.384 INFO:teuthology.orchestra.run.smithi032.stdout:1816 write 0xbe24f72 thru 0xbe2b382 (0x6411 bytes) 2024-09-21T14:59:26.385 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:59:26.385+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:59:26.389 INFO:teuthology.orchestra.run.smithi032.stdout:1817 read 0x441e6ed thru 0x4426119 (0x7a2d bytes) 2024-09-21T14:59:26.390 INFO:teuthology.orchestra.run.smithi032.stdout:1818 trunc from 0xd2814d0 to 0xd6c0931 2024-09-21T14:59:26.400 INFO:teuthology.orchestra.run.smithi032.stdout:1819 punch from 0x6eff667 to 0x6f0d8a3, (0xe23c bytes) 2024-09-21T14:59:26.400 INFO:teuthology.orchestra.run.smithi032.stdout:1820 punch from 0x932578b to 0x9325e8c, (0x701 bytes) 2024-09-21T14:59:26.400 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:26.400 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:26.400 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:26.400 INFO:teuthology.orchestra.run.smithi032.stdout:1826 read 0xcc1aa25 thru 0xcc28d9d (0xe379 bytes) 2024-09-21T14:59:26.400 INFO:teuthology.orchestra.run.smithi032.stdout:1828 write 0x19cdf5c thru 0x19dd36c (0xf411 bytes) 2024-09-21T14:59:26.406 INFO:teuthology.orchestra.run.smithi032.stdout:1829 writesame 0x5d11adb thru 0x5d1b7d7 data_size 0x1(0x9cfd bytes) 2024-09-21T14:59:26.432 INFO:teuthology.orchestra.run.smithi032.stdout:1831 punch from 0x4013161 to 0x401fe07, (0xcca6 bytes) 2024-09-21T14:59:26.432 INFO:teuthology.orchestra.run.smithi032.stdout:1832 punch from 0x9d88cf8 to 0x9d8b0c9, (0x23d1 bytes) 2024-09-21T14:59:26.432 INFO:teuthology.orchestra.run.smithi032.stdout:1834 write 0xcd5227 thru 0xcdd6e3 (0x84bd bytes) 2024-09-21T14:59:26.435 INFO:teuthology.orchestra.run.smithi032.stdout:1835 read 0x534627b thru 0x535054a (0xa2d0 bytes) 2024-09-21T14:59:26.435 INFO:teuthology.orchestra.run.smithi032.stdout:1837 read 0x5adc1f9 thru 0x5ae3afa (0x7902 bytes) 2024-09-21T14:59:26.436 INFO:teuthology.orchestra.run.smithi032.stdout:1839 read 0xc8a2157 thru 0xc8a33fb (0x12a5 bytes) 2024-09-21T14:59:26.436 INFO:teuthology.orchestra.run.smithi032.stdout:1840 clone 16 order 23 su 1048576 sc 10 2024-09-21T14:59:26.478 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:26.478 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:26.486 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:59:26.483+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:59:26.487 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T14:59:26.520 INFO:tasks.ceph.mon.a.smithi032.stderr:2024-09-21T14:59:26.521+0000 7fdb7159c700 -1 mon.a@0(leader).osd e89 definitely_dead 0 2024-09-21T14:59:26.542 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T14:59:26.588 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:59:26.589+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:59:26.593 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:26.594 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:26.604 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T14:59:26.688 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:59:26.689+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:59:26.690 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T14:59:26.712 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:26.712 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:26.712 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T14:59:26.722 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T14:59:26.734 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T14:59:26.753 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T14:59:26.773 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-21T14:59:26.789 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:59:26.789+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:59:26.797 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T14:59:26.821 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T14:59:26.830 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:26.830 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:26.840 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T14:59:26.843 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T14:59:26.846 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:59:26.846 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:59:26.846 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:59:26.846 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:59:26.846 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:59:26.846 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:59:26.846 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:59:26.846 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:59:26.890 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:59:26.887+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:59:26.949 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:26.949 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:59:26.949 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T14:59:26.949 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:26.959 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T14:59:26.991 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:59:26.989+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:59:27.073 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:27.073 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:59:27.073 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T14:59:27.073 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T14:59:27.073 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:27.083 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T14:59:27.132 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:59:27.133+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:59:27.193 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:27.193 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T14:59:27.193 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T14:59:27.193 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T14:59:27.193 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.194 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:1473 3.15 3:af2afa6a:::rbd_data.1136515ecb6b.000000000000001b:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T14:59:27.194 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:30.421730+0000", 2024-09-21T14:59:27.194 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 56.769303063999999, 2024-09-21T14:59:27.194 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021478937, 2024-09-21T14:59:27.194 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:27.194 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:27.194 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:27.195 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:27.195 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:27.195 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1473 2024-09-21T14:59:27.195 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.195 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:27.195 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.196 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:27.196 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421730+0000", 2024-09-21T14:59:27.196 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:27.196 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.196 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.196 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:27.197 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421730+0000", 2024-09-21T14:59:27.197 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:27.197 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.197 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.197 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:27.197 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421728+0000", 2024-09-21T14:59:27.198 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-21T14:59:27.198 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.198 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.198 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:27.198 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421739+0000", 2024-09-21T14:59:27.198 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0502e-05 2024-09-21T14:59:27.198 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.198 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.199 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:27.199 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421762+0000", 2024-09-21T14:59:27.199 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3405999999999999e-05 2024-09-21T14:59:27.199 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.199 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.199 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:27.199 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421773+0000", 2024-09-21T14:59:27.199 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0744000000000001e-05 2024-09-21T14:59:27.200 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.200 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.200 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:27.200 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421838+0000", 2024-09-21T14:59:27.200 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5486000000000003e-05 2024-09-21T14:59:27.200 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.200 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.200 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:27.200 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421955+0000", 2024-09-21T14:59:27.201 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000116765 2024-09-21T14:59:27.201 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.201 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.201 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 6", 2024-09-21T14:59:27.201 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.425572+0000", 2024-09-21T14:59:27.201 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0036171380000000002 2024-09-21T14:59:27.201 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.201 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.201 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:27.202 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.442973+0000", 2024-09-21T14:59:27.202 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.017400268 2024-09-21T14:59:27.202 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.202 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.202 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:27.202 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.442996+0000", 2024-09-21T14:59:27.203 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3139e-05 2024-09-21T14:59:27.203 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.203 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.203 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:27.203 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.443079+0000", 2024-09-21T14:59:27.203 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2832999999999998e-05 2024-09-21T14:59:27.203 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.203 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.203 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:27.203 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.443209+0000", 2024-09-21T14:59:27.204 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013052499999999999 2024-09-21T14:59:27.204 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:27.204 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:27.204 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:27.204 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.204 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.204 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:1678 3.c 3:377cc7e1:::rbd_data.1136515ecb6b.000000000000000f:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T14:59:27.204 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:30.988085+0000", 2024-09-21T14:59:27.204 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 56.202948470000003, 2024-09-21T14:59:27.204 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.032974802999999997, 2024-09-21T14:59:27.204 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:27.204 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:27.204 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:27.204 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:27.205 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:27.205 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1678 2024-09-21T14:59:27.205 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.205 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:27.205 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.205 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:27.205 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988085+0000", 2024-09-21T14:59:27.205 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:27.205 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.205 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.205 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:27.205 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988085+0000", 2024-09-21T14:59:27.205 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:27.205 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.206 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.206 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:27.206 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988083+0000", 2024-09-21T14:59:27.206 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-21T14:59:27.206 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.206 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.206 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:27.206 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988093+0000", 2024-09-21T14:59:27.206 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8800000000000003e-06 2024-09-21T14:59:27.206 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.206 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.206 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:27.206 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988113+0000", 2024-09-21T14:59:27.207 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0217999999999999e-05 2024-09-21T14:59:27.207 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.207 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.207 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:27.207 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988123+0000", 2024-09-21T14:59:27.207 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5859999999999994e-06 2024-09-21T14:59:27.207 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.207 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.207 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:27.207 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988203+0000", 2024-09-21T14:59:27.207 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0834000000000006e-05 2024-09-21T14:59:27.207 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.207 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.208 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:27.208 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988376+0000", 2024-09-21T14:59:27.208 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017301299999999999 2024-09-21T14:59:27.208 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.208 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.208 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 7", 2024-09-21T14:59:27.208 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.996338+0000", 2024-09-21T14:59:27.208 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0079618520000000002 2024-09-21T14:59:27.208 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.208 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.208 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:27.208 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.020881+0000", 2024-09-21T14:59:27.208 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024542731000000002 2024-09-21T14:59:27.209 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.209 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.209 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:27.209 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.020904+0000", 2024-09-21T14:59:27.209 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2861e-05 2024-09-21T14:59:27.209 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.209 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.209 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:27.209 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.020987+0000", 2024-09-21T14:59:27.209 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2806000000000006e-05 2024-09-21T14:59:27.209 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.209 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.209 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:27.209 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.021060+0000", 2024-09-21T14:59:27.209 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2871000000000001e-05 2024-09-21T14:59:27.210 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:27.210 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:27.210 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:27.210 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.210 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.210 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9369 3.3 3:c725132c:::rbd_data.11363a21d2be.0000000000000007:10 [read 24576~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:27.210 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157241+0000", 2024-09-21T14:59:27.210 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 48.033792024, 2024-09-21T14:59:27.210 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018289156000000001, 2024-09-21T14:59:27.210 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:27.210 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:27.210 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:27.210 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:27.211 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:27.211 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9369 2024-09-21T14:59:27.211 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.211 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:27.211 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.211 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:27.211 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157241+0000", 2024-09-21T14:59:27.211 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:27.211 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.211 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.211 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:27.211 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157241+0000", 2024-09-21T14:59:27.211 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:27.211 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.211 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.212 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:27.212 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157241+0000", 2024-09-21T14:59:27.212 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:27.212 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.212 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.212 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:27.212 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157244+0000", 2024-09-21T14:59:27.212 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5049999999999999e-06 2024-09-21T14:59:27.212 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.212 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.212 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:27.212 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157255+0000", 2024-09-21T14:59:27.212 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0797e-05 2024-09-21T14:59:27.213 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.213 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.213 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:27.213 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157260+0000", 2024-09-21T14:59:27.213 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1749999999999996e-06 2024-09-21T14:59:27.213 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.213 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.213 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:27.213 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175237+0000", 2024-09-21T14:59:27.213 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.017976405000000001 2024-09-21T14:59:27.213 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.214 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.214 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:27.214 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175331+0000", 2024-09-21T14:59:27.214 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4508999999999999e-05 2024-09-21T14:59:27.214 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.214 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.214 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:27.214 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175530+0000", 2024-09-21T14:59:27.214 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00019928200000000001 2024-09-21T14:59:27.214 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:27.215 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:27.215 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:27.215 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.215 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.215 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9370 3.3 3:c725132c:::rbd_data.11363a21d2be.0000000000000007:10 [read 163840~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:27.215 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157279+0000", 2024-09-21T14:59:27.215 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 48.033753846000003, 2024-09-21T14:59:27.215 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018472379000000001, 2024-09-21T14:59:27.215 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:27.215 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:27.215 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:27.215 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:27.215 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:27.216 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9370 2024-09-21T14:59:27.216 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.216 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:27.216 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.216 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:27.216 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157279+0000", 2024-09-21T14:59:27.216 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:27.216 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.216 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.216 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:27.216 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157279+0000", 2024-09-21T14:59:27.216 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:27.216 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.216 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.217 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:27.217 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157279+0000", 2024-09-21T14:59:27.217 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:27.217 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.217 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.217 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:27.217 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157282+0000", 2024-09-21T14:59:27.217 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2229999999999999e-06 2024-09-21T14:59:27.217 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.217 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.217 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:27.217 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157294+0000", 2024-09-21T14:59:27.217 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2173e-05 2024-09-21T14:59:27.217 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.218 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.218 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:27.218 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157300+0000", 2024-09-21T14:59:27.218 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.93e-06 2024-09-21T14:59:27.218 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.218 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.218 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:27.218 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175569+0000", 2024-09-21T14:59:27.218 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018268376999999999 2024-09-21T14:59:27.218 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.218 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.218 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:27.218 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175622+0000", 2024-09-21T14:59:27.219 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3808999999999999e-05 2024-09-21T14:59:27.219 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.219 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.219 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:27.219 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175752+0000", 2024-09-21T14:59:27.219 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012938700000000001 2024-09-21T14:59:27.219 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:27.219 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:27.219 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:27.219 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.219 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.219 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9371 3.3 3:c0bf6a9b:::rbd_data.11363a21d2be.0000000000000000:10 [read 73728~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:27.219 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157306+0000", 2024-09-21T14:59:27.219 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 48.033727734999999, 2024-09-21T14:59:27.220 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018755473000000002, 2024-09-21T14:59:27.220 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:27.220 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:27.220 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:27.220 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:27.220 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:27.220 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9371 2024-09-21T14:59:27.220 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.220 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:27.220 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.220 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:27.220 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157306+0000", 2024-09-21T14:59:27.220 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:27.220 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.221 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.221 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:27.221 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157306+0000", 2024-09-21T14:59:27.221 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:27.221 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.221 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.221 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:27.221 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157305+0000", 2024-09-21T14:59:27.221 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:27.221 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.221 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.221 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:27.221 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157312+0000", 2024-09-21T14:59:27.221 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3289999999999996e-06 2024-09-21T14:59:27.222 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.222 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.222 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:27.222 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157324+0000", 2024-09-21T14:59:27.222 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1725e-05 2024-09-21T14:59:27.222 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.222 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.222 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:27.222 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157339+0000", 2024-09-21T14:59:27.222 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5211999999999999e-05 2024-09-21T14:59:27.222 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.222 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.222 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:27.222 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175798+0000", 2024-09-21T14:59:27.223 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018458781 2024-09-21T14:59:27.223 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.223 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.223 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:27.223 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175899+0000", 2024-09-21T14:59:27.223 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000100784 2024-09-21T14:59:27.223 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.223 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.223 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:27.223 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176061+0000", 2024-09-21T14:59:27.223 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00016216099999999999 2024-09-21T14:59:27.223 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:27.223 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:27.223 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:27.224 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.224 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.224 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9372 3.3 3:c0bf6a9b:::rbd_data.11363a21d2be.0000000000000000:10 [read 212992~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:27.224 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157345+0000", 2024-09-21T14:59:27.224 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 48.033688388999998, 2024-09-21T14:59:27.224 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018926493999999999, 2024-09-21T14:59:27.224 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:27.224 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:27.224 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:27.225 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:27.225 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:27.225 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9372 2024-09-21T14:59:27.225 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.225 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:27.225 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.225 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:27.225 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157345+0000", 2024-09-21T14:59:27.225 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:27.225 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.226 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.226 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:27.226 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157345+0000", 2024-09-21T14:59:27.226 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:27.226 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.226 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.226 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:27.226 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157344+0000", 2024-09-21T14:59:27.226 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:27.226 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.226 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.226 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:27.227 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157348+0000", 2024-09-21T14:59:27.227 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1389999999999999e-06 2024-09-21T14:59:27.227 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.227 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.227 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:27.227 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157372+0000", 2024-09-21T14:59:27.227 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4711000000000001e-05 2024-09-21T14:59:27.227 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.227 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.227 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:27.227 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157378+0000", 2024-09-21T14:59:27.227 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9909999999999999e-06 2024-09-21T14:59:27.227 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.228 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.228 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:27.228 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176090+0000", 2024-09-21T14:59:27.228 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018712111999999999 2024-09-21T14:59:27.228 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.228 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.228 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:27.228 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176134+0000", 2024-09-21T14:59:27.228 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3377999999999997e-05 2024-09-21T14:59:27.228 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.228 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.228 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:27.228 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176271+0000", 2024-09-21T14:59:27.229 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013763499999999999 2024-09-21T14:59:27.229 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:27.229 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:27.229 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:27.229 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.229 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.229 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9373 3.3 3:c0bf6a9b:::rbd_data.11363a21d2be.0000000000000000:10 [read 352256~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:27.229 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157384+0000", 2024-09-21T14:59:27.229 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 48.033649617000002, 2024-09-21T14:59:27.229 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019154937, 2024-09-21T14:59:27.229 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:27.229 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:27.229 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:27.230 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:27.230 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:27.230 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9373 2024-09-21T14:59:27.230 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.230 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:27.230 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.230 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:27.230 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157384+0000", 2024-09-21T14:59:27.230 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:27.230 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.230 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.230 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:27.230 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157384+0000", 2024-09-21T14:59:27.231 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:27.231 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.231 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.231 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:27.231 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157383+0000", 2024-09-21T14:59:27.231 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:27.231 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.231 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.231 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:27.231 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157386+0000", 2024-09-21T14:59:27.231 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.191e-06 2024-09-21T14:59:27.231 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.231 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.232 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:27.232 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157393+0000", 2024-09-21T14:59:27.232 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5339999999999996e-06 2024-09-21T14:59:27.232 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.232 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.232 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:27.232 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157397+0000", 2024-09-21T14:59:27.232 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5220000000000001e-06 2024-09-21T14:59:27.232 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.232 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.232 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:27.232 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176302+0000", 2024-09-21T14:59:27.232 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018905113000000001 2024-09-21T14:59:27.232 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.233 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.233 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:27.233 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176379+0000", 2024-09-21T14:59:27.233 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6372999999999995e-05 2024-09-21T14:59:27.233 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.233 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.233 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:27.233 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176539+0000", 2024-09-21T14:59:27.233 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015973800000000001 2024-09-21T14:59:27.233 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:27.233 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:27.233 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:27.233 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.233 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.234 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9374 3.3 3:c0bf6a9b:::rbd_data.11363a21d2be.0000000000000000:10 [read 491520~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:27.234 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157401+0000", 2024-09-21T14:59:27.234 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 48.033631833000001, 2024-09-21T14:59:27.234 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019387755, 2024-09-21T14:59:27.234 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:27.234 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:27.234 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:27.234 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:27.234 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:27.234 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9374 2024-09-21T14:59:27.234 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.234 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:27.234 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.235 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:27.235 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157401+0000", 2024-09-21T14:59:27.235 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:27.235 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.235 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.235 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:27.235 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157401+0000", 2024-09-21T14:59:27.235 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:27.235 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.235 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.235 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:27.236 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157401+0000", 2024-09-21T14:59:27.236 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:27.236 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.236 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.236 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:27.236 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157414+0000", 2024-09-21T14:59:27.236 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2417e-05 2024-09-21T14:59:27.237 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.237 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.237 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:27.237 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157424+0000", 2024-09-21T14:59:27.237 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0861e-05 2024-09-21T14:59:27.237 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.237 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.238 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:27.238 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157429+0000", 2024-09-21T14:59:27.238 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0499999999999999e-06 2024-09-21T14:59:27.238 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.238 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.238 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:27.239 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176596+0000", 2024-09-21T14:59:27.239 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019166552 2024-09-21T14:59:27.239 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.239 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.239 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:27.239 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176647+0000", 2024-09-21T14:59:27.239 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1112000000000001e-05 2024-09-21T14:59:27.239 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.240 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.240 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:27.240 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176789+0000", 2024-09-21T14:59:27.240 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014209300000000001 2024-09-21T14:59:27.240 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:27.240 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:27.240 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:27.240 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.241 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.241 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9401 3.3 3:c725132c:::rbd_data.11363a21d2be.0000000000000007:10 [read 32768~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:27.241 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157621+0000", 2024-09-21T14:59:27.241 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 48.033412024, 2024-09-21T14:59:27.241 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019381711999999999, 2024-09-21T14:59:27.241 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:27.241 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:27.242 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:27.242 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:27.242 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:27.242 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9401 2024-09-21T14:59:27.242 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.242 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:27.243 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.243 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:27.243 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157621+0000", 2024-09-21T14:59:27.243 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:27.243 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.243 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.243 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:27.244 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157621+0000", 2024-09-21T14:59:27.244 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:27.244 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.244 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.244 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:27.244 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157620+0000", 2024-09-21T14:59:27.244 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:59:27.245 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.245 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.245 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:27.245 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157626+0000", 2024-09-21T14:59:27.245 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7570000000000001e-06 2024-09-21T14:59:27.245 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.245 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.245 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:27.246 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157636+0000", 2024-09-21T14:59:27.246 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8339999999999994e-06 2024-09-21T14:59:27.246 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.246 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.246 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:27.246 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157643+0000", 2024-09-21T14:59:27.246 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9439999999999999e-06 2024-09-21T14:59:27.247 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.247 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.247 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:27.247 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176816+0000", 2024-09-21T14:59:27.247 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019172872000000001 2024-09-21T14:59:27.247 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.248 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.248 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:27.248 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176878+0000", 2024-09-21T14:59:27.248 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1734000000000002e-05 2024-09-21T14:59:27.248 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.248 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.248 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:27.249 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177003+0000", 2024-09-21T14:59:27.249 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012537400000000001 2024-09-21T14:59:27.249 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:27.249 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:27.249 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:27.250 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.250 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.250 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9402 3.3 3:c725132c:::rbd_data.11363a21d2be.0000000000000007:10 [read 172032~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:27.250 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157649+0000", 2024-09-21T14:59:27.250 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 48.033384151999996, 2024-09-21T14:59:27.250 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019567404, 2024-09-21T14:59:27.251 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:27.251 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:27.251 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:27.251 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:27.251 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:27.251 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9402 2024-09-21T14:59:27.251 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.252 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:27.252 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.252 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:27.252 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157649+0000", 2024-09-21T14:59:27.252 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:27.252 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.252 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.252 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:27.253 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157649+0000", 2024-09-21T14:59:27.253 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:27.253 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.253 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.253 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:27.253 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157649+0000", 2024-09-21T14:59:27.253 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:27.253 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.253 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.254 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:27.254 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157652+0000", 2024-09-21T14:59:27.254 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8779999999999998e-06 2024-09-21T14:59:27.254 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.254 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.254 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:27.254 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157657+0000", 2024-09-21T14:59:27.255 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6659999999999996e-06 2024-09-21T14:59:27.255 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.255 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.255 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:27.255 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157662+0000", 2024-09-21T14:59:27.255 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5519999999999998e-06 2024-09-21T14:59:27.255 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.256 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.256 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:27.256 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177040+0000", 2024-09-21T14:59:27.256 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019377807 2024-09-21T14:59:27.256 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.256 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.256 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:27.256 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177085+0000", 2024-09-21T14:59:27.256 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5195999999999999e-05 2024-09-21T14:59:27.256 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.256 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.257 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:27.257 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177217+0000", 2024-09-21T14:59:27.257 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013172600000000001 2024-09-21T14:59:27.257 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:27.257 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:27.257 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:27.257 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.257 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.257 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9403 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:27.257 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157949+0000", 2024-09-21T14:59:27.257 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 48.033083822999998, 2024-09-21T14:59:27.257 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023540373999999999, 2024-09-21T14:59:27.257 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:27.258 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:27.258 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:27.258 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:27.258 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:27.258 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9403 2024-09-21T14:59:27.258 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.258 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:27.258 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.258 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:27.258 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157949+0000", 2024-09-21T14:59:27.258 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:27.258 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.258 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.259 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:27.259 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157949+0000", 2024-09-21T14:59:27.259 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:27.259 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.259 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.259 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:27.259 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157948+0000", 2024-09-21T14:59:27.259 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T14:59:27.259 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.259 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.259 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:27.259 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157956+0000", 2024-09-21T14:59:27.259 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7999999999999999e-06 2024-09-21T14:59:27.259 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.260 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.260 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:27.260 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157969+0000", 2024-09-21T14:59:27.260 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2792000000000001e-05 2024-09-21T14:59:27.260 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.260 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.260 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:27.260 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157977+0000", 2024-09-21T14:59:27.260 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.704e-06 2024-09-21T14:59:27.260 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.260 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.260 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:27.260 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177246+0000", 2024-09-21T14:59:27.261 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019268264 2024-09-21T14:59:27.261 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.261 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.261 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:27.261 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177315+0000", 2024-09-21T14:59:27.261 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9480999999999994e-05 2024-09-21T14:59:27.261 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.261 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.261 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:59:27.261 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177615+0000", 2024-09-21T14:59:27.261 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00029929499999999999 2024-09-21T14:59:27.261 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.261 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.261 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:27.262 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181333+0000", 2024-09-21T14:59:27.262 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0037183149999999998 2024-09-21T14:59:27.262 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.262 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.262 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:27.262 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181374+0000", 2024-09-21T14:59:27.262 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.0723999999999998e-05 2024-09-21T14:59:27.262 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.262 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.262 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:27.262 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181434+0000", 2024-09-21T14:59:27.262 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0024e-05 2024-09-21T14:59:27.262 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.263 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.263 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:27.263 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181490+0000", 2024-09-21T14:59:27.263 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6245000000000001e-05 2024-09-21T14:59:27.263 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:27.263 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:27.263 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:27.263 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.263 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.263 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9404 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:27.263 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.160480+0000", 2024-09-21T14:59:27.263 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 48.030553472999998, 2024-09-21T14:59:27.263 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021137686999999999, 2024-09-21T14:59:27.263 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:27.264 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:27.264 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:27.264 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:27.264 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:27.264 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9404 2024-09-21T14:59:27.264 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.264 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:27.264 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.264 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:27.264 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160480+0000", 2024-09-21T14:59:27.264 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:27.264 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.264 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.264 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:27.265 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160480+0000", 2024-09-21T14:59:27.265 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:27.265 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.265 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.265 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:27.265 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160478+0000", 2024-09-21T14:59:27.265 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T14:59:27.265 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.265 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.265 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:27.265 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160486+0000", 2024-09-21T14:59:27.265 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9969999999999998e-06 2024-09-21T14:59:27.265 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.265 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.266 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:27.266 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160502+0000", 2024-09-21T14:59:27.266 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5650000000000001e-05 2024-09-21T14:59:27.266 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.266 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.266 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:27.266 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160511+0000", 2024-09-21T14:59:27.266 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.3610000000000007e-06 2024-09-21T14:59:27.267 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.267 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.267 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:27.267 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.178035+0000", 2024-09-21T14:59:27.267 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.017523520000000001 2024-09-21T14:59:27.267 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.267 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.267 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:27.267 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.178151+0000", 2024-09-21T14:59:27.268 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000116013 2024-09-21T14:59:27.268 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.268 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.268 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:59:27.268 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.178408+0000", 2024-09-21T14:59:27.268 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00025743400000000002 2024-09-21T14:59:27.268 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.268 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.268 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:27.268 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181501+0000", 2024-09-21T14:59:27.268 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.003092891 2024-09-21T14:59:27.268 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.268 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.268 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:27.269 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181538+0000", 2024-09-21T14:59:27.269 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6977999999999997e-05 2024-09-21T14:59:27.269 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.269 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.269 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:27.269 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181582+0000", 2024-09-21T14:59:27.269 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3849e-05 2024-09-21T14:59:27.269 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.269 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.269 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:27.269 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181617+0000", 2024-09-21T14:59:27.269 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5423000000000003e-05 2024-09-21T14:59:27.269 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:27.270 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:27.270 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:27.270 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.270 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.270 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9406 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:27.270 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.166040+0000", 2024-09-21T14:59:27.270 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 48.024993594000001, 2024-09-21T14:59:27.270 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018094524000000001, 2024-09-21T14:59:27.270 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:27.270 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:27.270 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:27.270 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:27.270 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:27.271 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9406 2024-09-21T14:59:27.271 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.271 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:27.271 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.271 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:27.271 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.166040+0000", 2024-09-21T14:59:27.271 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:27.271 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.271 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.271 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:27.271 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.166040+0000", 2024-09-21T14:59:27.271 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:27.272 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.272 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.272 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:27.272 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.166038+0000", 2024-09-21T14:59:27.272 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T14:59:27.272 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.272 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.272 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:27.272 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.166051+0000", 2024-09-21T14:59:27.272 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2638e-05 2024-09-21T14:59:27.272 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.272 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.272 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:27.273 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.166070+0000", 2024-09-21T14:59:27.273 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8691000000000001e-05 2024-09-21T14:59:27.273 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.273 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.273 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:27.273 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.166089+0000", 2024-09-21T14:59:27.273 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9286000000000001e-05 2024-09-21T14:59:27.273 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.273 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.273 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:27.273 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.180194+0000", 2024-09-21T14:59:27.273 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014105039999999999 2024-09-21T14:59:27.273 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.273 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.274 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:27.274 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.180340+0000", 2024-09-21T14:59:27.274 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014623199999999999 2024-09-21T14:59:27.274 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.274 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.274 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:59:27.274 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.180818+0000", 2024-09-21T14:59:27.274 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000477489 2024-09-21T14:59:27.274 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.274 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.274 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:27.274 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.183808+0000", 2024-09-21T14:59:27.274 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0029897610000000001 2024-09-21T14:59:27.275 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.275 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.275 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:27.275 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.183821+0000", 2024-09-21T14:59:27.275 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3251999999999999e-05 2024-09-21T14:59:27.275 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.275 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.275 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:27.275 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.183872+0000", 2024-09-21T14:59:27.275 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0900999999999999e-05 2024-09-21T14:59:27.275 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.275 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.275 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:27.275 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.184134+0000", 2024-09-21T14:59:27.276 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000262484 2024-09-21T14:59:27.276 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:27.276 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:27.276 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:27.276 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.276 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.276 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9407 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:27.276 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.168094+0000", 2024-09-21T14:59:27.276 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 48.022938973999999, 2024-09-21T14:59:27.276 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019444860000000001, 2024-09-21T14:59:27.276 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:27.276 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:27.276 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:27.276 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:27.277 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:27.277 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9407 2024-09-21T14:59:27.277 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.277 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:27.277 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.277 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:27.277 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.168094+0000", 2024-09-21T14:59:27.277 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:27.277 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.277 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.277 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:27.278 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.168094+0000", 2024-09-21T14:59:27.278 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:27.278 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.278 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.278 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:27.278 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.168093+0000", 2024-09-21T14:59:27.278 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:59:27.278 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.278 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.278 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:27.278 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.168119+0000", 2024-09-21T14:59:27.278 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5803999999999999e-05 2024-09-21T14:59:27.279 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.279 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.279 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:27.279 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.168135+0000", 2024-09-21T14:59:27.279 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6359999999999999e-05 2024-09-21T14:59:27.279 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.279 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.279 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:27.279 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.168144+0000", 2024-09-21T14:59:27.279 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4270000000000008e-06 2024-09-21T14:59:27.279 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.279 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.279 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:27.279 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181943+0000", 2024-09-21T14:59:27.280 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013799617 2024-09-21T14:59:27.280 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.280 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.280 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:27.280 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.182052+0000", 2024-09-21T14:59:27.280 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000108924 2024-09-21T14:59:27.280 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.280 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.280 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:59:27.280 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.183431+0000", 2024-09-21T14:59:27.280 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0013783619999999999 2024-09-21T14:59:27.280 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.280 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.281 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:27.281 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.187276+0000", 2024-09-21T14:59:27.281 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0038450469999999999 2024-09-21T14:59:27.281 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.281 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.281 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:27.281 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.187288+0000", 2024-09-21T14:59:27.281 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2414e-05 2024-09-21T14:59:27.281 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.281 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.281 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:27.281 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.187414+0000", 2024-09-21T14:59:27.282 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012614399999999999 2024-09-21T14:59:27.282 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.282 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.282 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:27.282 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.187539+0000", 2024-09-21T14:59:27.282 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012481800000000001 2024-09-21T14:59:27.282 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:27.282 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:27.282 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:27.282 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.282 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.282 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9410 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:27.282 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.174103+0000", 2024-09-21T14:59:27.283 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 48.016930064999997, 2024-09-21T14:59:27.283 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025493703, 2024-09-21T14:59:27.283 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:27.283 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:27.283 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:27.283 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:27.283 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:27.283 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9410 2024-09-21T14:59:27.283 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.283 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:27.283 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.283 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:27.283 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.174103+0000", 2024-09-21T14:59:27.283 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:27.284 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.284 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.284 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:27.284 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.174103+0000", 2024-09-21T14:59:27.284 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:27.284 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.284 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.284 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:27.284 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.174102+0000", 2024-09-21T14:59:27.284 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:59:27.284 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.284 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.284 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:27.285 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.174108+0000", 2024-09-21T14:59:27.285 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1149999999999999e-06 2024-09-21T14:59:27.285 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.285 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.285 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:27.285 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.174117+0000", 2024-09-21T14:59:27.285 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4870000000000002e-06 2024-09-21T14:59:27.285 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.285 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.285 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:27.285 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.174122+0000", 2024-09-21T14:59:27.285 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9790000000000001e-06 2024-09-21T14:59:27.285 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.285 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.285 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:27.286 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.187830+0000", 2024-09-21T14:59:27.286 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013708048 2024-09-21T14:59:27.286 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.286 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.286 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:27.286 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.187913+0000", 2024-09-21T14:59:27.286 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2763999999999996e-05 2024-09-21T14:59:27.286 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.286 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.286 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:59:27.286 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.188230+0000", 2024-09-21T14:59:27.286 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00031745100000000001 2024-09-21T14:59:27.286 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.287 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.287 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:27.287 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.192708+0000", 2024-09-21T14:59:27.287 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0044783890000000002 2024-09-21T14:59:27.287 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.287 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.287 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:27.287 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199487+0000", 2024-09-21T14:59:27.287 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0067782149999999998 2024-09-21T14:59:27.287 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.287 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.288 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:27.288 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199560+0000", 2024-09-21T14:59:27.288 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3752999999999997e-05 2024-09-21T14:59:27.288 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.288 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.288 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:27.288 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199597+0000", 2024-09-21T14:59:27.288 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6526999999999998e-05 2024-09-21T14:59:27.288 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:27.288 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:27.288 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:27.288 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.288 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.288 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9411 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:27.289 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.175943+0000", 2024-09-21T14:59:27.289 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 48.015090045999997, 2024-09-21T14:59:27.289 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023773181000000001, 2024-09-21T14:59:27.289 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:27.289 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:27.289 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:27.289 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:27.289 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:27.289 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9411 2024-09-21T14:59:27.289 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.289 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:27.289 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.289 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:27.289 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175943+0000", 2024-09-21T14:59:27.289 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:27.289 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.290 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.290 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:27.290 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175943+0000", 2024-09-21T14:59:27.290 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:27.290 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.290 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.290 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:27.290 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175942+0000", 2024-09-21T14:59:27.290 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:59:27.290 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.290 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.290 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:27.290 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175948+0000", 2024-09-21T14:59:27.291 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9689999999999999e-06 2024-09-21T14:59:27.291 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.291 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.291 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:27.291 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175960+0000", 2024-09-21T14:59:27.291 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1745e-05 2024-09-21T14:59:27.291 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.291 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.291 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:27.291 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175965+0000", 2024-09-21T14:59:27.291 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.203e-06 2024-09-21T14:59:27.291 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.291 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.292 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:27.292 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.189900+0000", 2024-09-21T14:59:27.292 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013934940999999999 2024-09-21T14:59:27.292 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.292 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.292 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:27.292 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.189988+0000", 2024-09-21T14:59:27.292 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8419999999999997e-05 2024-09-21T14:59:27.292 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.292 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.292 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:59:27.292 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.190321+0000", 2024-09-21T14:59:27.292 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00033231800000000001 2024-09-21T14:59:27.293 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.293 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.293 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:27.293 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.192746+0000", 2024-09-21T14:59:27.293 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0024254400000000001 2024-09-21T14:59:27.293 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.293 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.293 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:27.293 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199605+0000", 2024-09-21T14:59:27.293 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0068582809999999999 2024-09-21T14:59:27.293 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.293 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.293 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:27.293 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199680+0000", 2024-09-21T14:59:27.294 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5630000000000006e-05 2024-09-21T14:59:27.294 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.294 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.294 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:27.294 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199716+0000", 2024-09-21T14:59:27.294 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6248999999999998e-05 2024-09-21T14:59:27.294 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:27.294 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:27.294 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:27.294 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.294 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.294 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9412 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:27.294 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.177402+0000", 2024-09-21T14:59:27.295 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 48.013630949000003, 2024-09-21T14:59:27.295 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022445590000000001, 2024-09-21T14:59:27.295 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:27.295 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:27.295 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:27.295 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:27.295 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:27.295 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9412 2024-09-21T14:59:27.295 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.295 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:27.295 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.295 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:27.295 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177402+0000", 2024-09-21T14:59:27.295 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:27.296 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.296 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.296 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:27.296 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177402+0000", 2024-09-21T14:59:27.296 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:27.296 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.296 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.296 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:27.296 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177401+0000", 2024-09-21T14:59:27.296 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:59:27.296 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.296 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.296 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:27.296 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177409+0000", 2024-09-21T14:59:27.297 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4710000000000001e-06 2024-09-21T14:59:27.297 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.297 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.297 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:27.297 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177419+0000", 2024-09-21T14:59:27.297 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0351e-05 2024-09-21T14:59:27.297 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.297 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.297 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:27.297 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177424+0000", 2024-09-21T14:59:27.297 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1919999999999996e-06 2024-09-21T14:59:27.297 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.297 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.297 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:27.298 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.191343+0000", 2024-09-21T14:59:27.298 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.01391878 2024-09-21T14:59:27.298 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.298 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.298 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:27.298 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.191446+0000", 2024-09-21T14:59:27.298 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000102453 2024-09-21T14:59:27.298 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.298 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.298 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:59:27.299 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.192200+0000", 2024-09-21T14:59:27.299 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00075476899999999997 2024-09-21T14:59:27.299 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.299 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.299 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:27.299 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.193866+0000", 2024-09-21T14:59:27.299 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0016657519999999999 2024-09-21T14:59:27.299 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.299 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.299 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:27.299 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199724+0000", 2024-09-21T14:59:27.299 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0058579529999999999 2024-09-21T14:59:27.299 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.299 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.300 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:27.300 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199803+0000", 2024-09-21T14:59:27.300 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8662e-05 2024-09-21T14:59:27.300 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.300 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.300 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:27.300 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199848+0000", 2024-09-21T14:59:27.300 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5216000000000002e-05 2024-09-21T14:59:27.300 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:27.300 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:27.300 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:27.300 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.300 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.300 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:10445 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:27.301 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.271980+0000", 2024-09-21T14:59:27.301 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 47.919053583999997, 2024-09-21T14:59:27.301 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018664256000000001, 2024-09-21T14:59:27.301 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:27.301 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:27.301 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:27.301 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:27.301 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:27.301 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 10445 2024-09-21T14:59:27.301 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.301 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:27.301 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.301 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:27.302 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.271980+0000", 2024-09-21T14:59:27.302 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:27.302 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.302 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.302 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:27.302 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.271980+0000", 2024-09-21T14:59:27.302 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:27.302 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.302 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.302 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:27.302 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.271979+0000", 2024-09-21T14:59:27.302 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:27.302 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.302 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.303 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:27.303 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.271982+0000", 2024-09-21T14:59:27.303 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7240000000000001e-06 2024-09-21T14:59:27.303 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.303 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.303 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:27.303 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.271987+0000", 2024-09-21T14:59:27.303 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3639999999999998e-06 2024-09-21T14:59:27.303 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.303 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.303 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:27.303 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.271992+0000", 2024-09-21T14:59:27.303 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7110000000000003e-06 2024-09-21T14:59:27.303 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.303 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.304 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:27.304 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.277567+0000", 2024-09-21T14:59:27.304 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0055745009999999999 2024-09-21T14:59:27.304 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.304 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.304 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:27.304 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.277655+0000", 2024-09-21T14:59:27.304 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8465000000000001e-05 2024-09-21T14:59:27.304 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.304 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.304 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:59:27.304 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.278720+0000", 2024-09-21T14:59:27.304 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0010650569999999999 2024-09-21T14:59:27.304 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.305 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.305 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:27.305 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.281153+0000", 2024-09-21T14:59:27.305 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0024326389999999999 2024-09-21T14:59:27.305 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.305 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.305 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:27.305 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.290574+0000", 2024-09-21T14:59:27.305 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0094209489999999996 2024-09-21T14:59:27.305 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.305 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.305 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:27.305 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.290613+0000", 2024-09-21T14:59:27.305 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.8803000000000001e-05 2024-09-21T14:59:27.306 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.306 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.306 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:27.306 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.290644+0000", 2024-09-21T14:59:27.306 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1404e-05 2024-09-21T14:59:27.306 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:27.306 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:27.306 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:27.306 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.306 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.306 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22769 3.f 3:f39bd56f:::rbd_data.11366acc894.0000000000000004:19 [read 1032192~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:27.306 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.140207+0000", 2024-09-21T14:59:27.306 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 37.050825844000002, 2024-09-21T14:59:27.307 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018031170999999999, 2024-09-21T14:59:27.307 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:27.307 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:27.307 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:27.307 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:27.307 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:27.307 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22769 2024-09-21T14:59:27.307 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.307 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:27.307 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.307 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:27.307 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.140207+0000", 2024-09-21T14:59:27.307 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:27.307 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.308 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.308 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:27.308 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.140207+0000", 2024-09-21T14:59:27.308 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:27.308 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.308 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.308 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:27.308 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.140207+0000", 2024-09-21T14:59:27.308 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:27.308 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.308 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.309 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:27.309 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.140209+0000", 2024-09-21T14:59:27.309 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.305e-06 2024-09-21T14:59:27.309 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.309 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.309 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:27.309 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.140214+0000", 2024-09-21T14:59:27.309 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9570000000000001e-06 2024-09-21T14:59:27.309 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.309 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.309 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:27.309 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.140219+0000", 2024-09-21T14:59:27.309 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2250000000000002e-06 2024-09-21T14:59:27.309 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.310 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.310 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:27.310 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.157990+0000", 2024-09-21T14:59:27.310 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.017770939999999999 2024-09-21T14:59:27.310 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.310 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.310 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:27.310 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.158062+0000", 2024-09-21T14:59:27.310 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2652999999999997e-05 2024-09-21T14:59:27.310 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.310 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.310 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:27.310 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.158239+0000", 2024-09-21T14:59:27.310 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017640599999999999 2024-09-21T14:59:27.311 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:27.311 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:27.311 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:27.311 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.311 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.311 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:23084 3.3 3:c55e343c:::rbd_data.11366acc894.0000000000000006:19 [read 1032192~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:27.311 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.150724+0000", 2024-09-21T14:59:27.311 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 37.040309008999998, 2024-09-21T14:59:27.311 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018916460999999999, 2024-09-21T14:59:27.311 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:27.311 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:27.311 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:27.311 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:27.311 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:27.312 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 23084 2024-09-21T14:59:27.312 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.312 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:27.312 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.312 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:27.312 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.150724+0000", 2024-09-21T14:59:27.312 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:27.312 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.312 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.312 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:27.312 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.150724+0000", 2024-09-21T14:59:27.312 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:27.312 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.312 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.312 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:27.313 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.150724+0000", 2024-09-21T14:59:27.313 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:27.313 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.313 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.313 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:27.313 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.150727+0000", 2024-09-21T14:59:27.313 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7250000000000002e-06 2024-09-21T14:59:27.313 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.313 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.313 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:27.313 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.150732+0000", 2024-09-21T14:59:27.313 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5809999999999996e-06 2024-09-21T14:59:27.313 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.313 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.314 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:27.314 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.150737+0000", 2024-09-21T14:59:27.314 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3590000000000001e-06 2024-09-21T14:59:27.314 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.314 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.314 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:27.314 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.166694+0000", 2024-09-21T14:59:27.314 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.015957720000000002 2024-09-21T14:59:27.314 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.314 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.314 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:27.314 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.166769+0000", 2024-09-21T14:59:27.314 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5143999999999997e-05 2024-09-21T14:59:27.315 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.315 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.315 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:27.315 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.169641+0000", 2024-09-21T14:59:27.315 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0028713089999999998 2024-09-21T14:59:27.315 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:27.315 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:27.315 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:27.315 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:27.315 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:27.315 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:27.316 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:59:27.187+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:59:27.317 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T14:59:27.318 INFO:teuthology.orchestra.run.smithi032.stderr:marked in osd.4. 2024-09-21T14:59:27.318 INFO:tasks.thrashosds.thrasher:Added osd 4 2024-09-21T14:59:27.319 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:59:27.317+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:59:27.418 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:59:27.417+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:59:27.437 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:27.437 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:59:27.437 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T14:59:27.437 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:27.447 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T14:59:27.519 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:59:27.515+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:59:27.604 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:27.605 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:59:27.605 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T14:59:27.605 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T14:59:27.605 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:27.615 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T14:59:27.620 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:59:27.621+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:59:27.722 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:59:27.719+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:59:27.723 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:27.723 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T14:59:27.723 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T14:59:27.723 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T14:59:27.723 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.723 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T14:59:27.723 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.205754+0000", 2024-09-21T14:59:27.723 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 84.517482228000006, 2024-09-21T14:59:27.723 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.41117094199999998, 2024-09-21T14:59:27.724 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:27.724 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:27.724 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:27.724 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:59:27.724 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:59:27.724 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1 2024-09-21T14:59:27.724 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.724 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:27.724 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.724 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:27.724 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205754+0000", 2024-09-21T14:59:27.724 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:27.724 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.725 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.725 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:27.725 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205754+0000", 2024-09-21T14:59:27.725 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:27.725 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.725 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.725 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:27.725 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205752+0000", 2024-09-21T14:59:27.725 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T14:59:27.725 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.725 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.725 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:27.725 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205765+0000", 2024-09-21T14:59:27.725 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3230999999999999e-05 2024-09-21T14:59:27.726 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.726 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.726 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:27.726 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205782+0000", 2024-09-21T14:59:27.726 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6634e-05 2024-09-21T14:59:27.726 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.726 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.726 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:27.726 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205793+0000", 2024-09-21T14:59:27.726 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1297e-05 2024-09-21T14:59:27.726 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.726 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.726 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:27.727 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205867+0000", 2024-09-21T14:59:27.727 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4269000000000003e-05 2024-09-21T14:59:27.727 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.727 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.727 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T14:59:27.727 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205877+0000", 2024-09-21T14:59:27.727 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2690000000000005e-06 2024-09-21T14:59:27.727 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.727 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.727 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:27.728 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.613700+0000", 2024-09-21T14:59:27.728 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.407823455 2024-09-21T14:59:27.728 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.728 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.728 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:27.728 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.614203+0000", 2024-09-21T14:59:27.728 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00050311500000000005 2024-09-21T14:59:27.728 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.728 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.728 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:59:27.728 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.614750+0000", 2024-09-21T14:59:27.728 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00054738699999999998 2024-09-21T14:59:27.728 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.728 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.729 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:27.729 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616367+0000", 2024-09-21T14:59:27.729 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001616634 2024-09-21T14:59:27.729 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.729 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.729 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:27.729 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616574+0000", 2024-09-21T14:59:27.729 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00020668499999999999 2024-09-21T14:59:27.729 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.729 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.729 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:27.729 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616802+0000", 2024-09-21T14:59:27.729 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00022817900000000001 2024-09-21T14:59:27.730 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.730 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.730 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:27.730 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616925+0000", 2024-09-21T14:59:27.730 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012309699999999999 2024-09-21T14:59:27.730 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:27.730 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:27.730 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:27.730 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.730 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.730 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T14:59:27.731 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.955609+0000", 2024-09-21T14:59:27.731 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 83.767627841000007, 2024-09-21T14:59:27.731 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013624153999999999, 2024-09-21T14:59:27.731 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:27.731 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:27.731 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:27.731 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:59:27.731 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:59:27.731 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 115 2024-09-21T14:59:27.731 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.731 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:27.731 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.731 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:27.732 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955609+0000", 2024-09-21T14:59:27.732 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:27.732 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.732 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.732 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:27.732 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955609+0000", 2024-09-21T14:59:27.732 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:27.732 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.732 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.732 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:27.732 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955607+0000", 2024-09-21T14:59:27.732 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-21T14:59:27.732 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.733 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.733 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:27.733 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955732+0000", 2024-09-21T14:59:27.733 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000125566 2024-09-21T14:59:27.733 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.733 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.733 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:27.733 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955752+0000", 2024-09-21T14:59:27.733 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9916999999999999e-05 2024-09-21T14:59:27.733 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.733 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.733 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:27.733 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955767+0000", 2024-09-21T14:59:27.733 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4687e-05 2024-09-21T14:59:27.734 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.734 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.734 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:27.734 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.965574+0000", 2024-09-21T14:59:27.734 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0098072850000000007 2024-09-21T14:59:27.734 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.734 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.734 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:27.734 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.965642+0000", 2024-09-21T14:59:27.734 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7784e-05 2024-09-21T14:59:27.734 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.734 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.734 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:59:27.734 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.965753+0000", 2024-09-21T14:59:27.735 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000111111 2024-09-21T14:59:27.735 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.735 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.735 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:27.735 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967717+0000", 2024-09-21T14:59:27.735 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0019635339999999999 2024-09-21T14:59:27.735 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.735 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.735 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:27.735 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.969014+0000", 2024-09-21T14:59:27.735 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001297093 2024-09-21T14:59:27.735 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.735 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.736 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:27.736 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.969130+0000", 2024-09-21T14:59:27.736 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000116011 2024-09-21T14:59:27.736 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.736 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.736 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:27.736 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.969233+0000", 2024-09-21T14:59:27.736 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000102982 2024-09-21T14:59:27.736 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:27.736 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:27.736 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:27.736 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.736 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.736 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T14:59:27.736 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.955783+0000", 2024-09-21T14:59:27.736 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 83.767453172000003, 2024-09-21T14:59:27.737 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014630241, 2024-09-21T14:59:27.737 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:27.737 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:27.737 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:27.737 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:59:27.737 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:59:27.737 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 116 2024-09-21T14:59:27.737 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.737 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:27.737 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.737 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:27.737 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955783+0000", 2024-09-21T14:59:27.737 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:27.737 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.738 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.738 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:27.738 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955783+0000", 2024-09-21T14:59:27.738 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:27.738 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.738 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.738 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:27.738 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955782+0000", 2024-09-21T14:59:27.738 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:59:27.738 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.738 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.738 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:27.738 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955791+0000", 2024-09-21T14:59:27.739 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8089999999999999e-06 2024-09-21T14:59:27.739 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.739 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.739 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:27.739 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955804+0000", 2024-09-21T14:59:27.739 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2548000000000001e-05 2024-09-21T14:59:27.739 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.739 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.739 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:27.739 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955814+0000", 2024-09-21T14:59:27.739 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0158e-05 2024-09-21T14:59:27.739 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.739 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.739 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:27.740 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.966515+0000", 2024-09-21T14:59:27.740 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.010701756 2024-09-21T14:59:27.740 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.740 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.740 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:27.740 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.966606+0000", 2024-09-21T14:59:27.740 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0761999999999996e-05 2024-09-21T14:59:27.740 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.740 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.740 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:59:27.740 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.966740+0000", 2024-09-21T14:59:27.740 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000133693 2024-09-21T14:59:27.740 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.740 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.740 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:27.741 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.968920+0000", 2024-09-21T14:59:27.741 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0021799290000000002 2024-09-21T14:59:27.741 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.741 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.741 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:27.741 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.970320+0000", 2024-09-21T14:59:27.741 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0014001759999999999 2024-09-21T14:59:27.741 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.741 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.741 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:27.741 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.970376+0000", 2024-09-21T14:59:27.741 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5696999999999999e-05 2024-09-21T14:59:27.741 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.741 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.742 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:27.742 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.970413+0000", 2024-09-21T14:59:27.742 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7750999999999998e-05 2024-09-21T14:59:27.742 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:27.742 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:27.742 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:27.742 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.742 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.742 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T14:59:27.742 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.955829+0000", 2024-09-21T14:59:27.742 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 83.767407702, 2024-09-21T14:59:27.743 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014919597999999999, 2024-09-21T14:59:27.743 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:27.743 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:27.743 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:27.743 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:59:27.743 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:59:27.743 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 117 2024-09-21T14:59:27.743 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.743 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:27.743 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.743 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:27.743 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955829+0000", 2024-09-21T14:59:27.744 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:27.744 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.744 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.744 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:27.744 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955829+0000", 2024-09-21T14:59:27.744 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:27.744 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.744 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.744 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:27.744 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955827+0000", 2024-09-21T14:59:27.744 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T14:59:27.744 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.744 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.744 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:27.745 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955838+0000", 2024-09-21T14:59:27.745 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.111e-05 2024-09-21T14:59:27.745 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.745 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.745 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:27.745 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955875+0000", 2024-09-21T14:59:27.745 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6211999999999998e-05 2024-09-21T14:59:27.745 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.745 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.745 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:27.745 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955899+0000", 2024-09-21T14:59:27.745 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4128999999999999e-05 2024-09-21T14:59:27.745 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.745 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.746 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:27.746 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967767+0000", 2024-09-21T14:59:27.746 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.011868251999999999 2024-09-21T14:59:27.746 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.746 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.746 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:27.746 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967866+0000", 2024-09-21T14:59:27.746 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8664000000000003e-05 2024-09-21T14:59:27.746 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.746 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.746 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:59:27.746 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.968128+0000", 2024-09-21T14:59:27.746 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00026243600000000001 2024-09-21T14:59:27.747 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.747 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.747 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:27.747 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.969393+0000", 2024-09-21T14:59:27.747 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0012650350000000001 2024-09-21T14:59:27.747 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.747 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.747 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:27.747 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.970523+0000", 2024-09-21T14:59:27.747 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001129557 2024-09-21T14:59:27.747 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.747 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.747 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:27.747 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.970603+0000", 2024-09-21T14:59:27.747 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0090000000000001e-05 2024-09-21T14:59:27.748 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.748 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.749 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:27.749 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.970748+0000", 2024-09-21T14:59:27.749 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014552799999999999 2024-09-21T14:59:27.749 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:27.749 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:27.750 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:27.750 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.750 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.750 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T14:59:27.750 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:04.493315+0000", 2024-09-21T14:59:27.750 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 83.229920868999997, 2024-09-21T14:59:27.750 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014055766000000001, 2024-09-21T14:59:27.750 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:27.750 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:27.750 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:27.750 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:59:27.750 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:59:27.750 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 137 2024-09-21T14:59:27.751 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.751 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:27.751 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.751 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:27.751 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493315+0000", 2024-09-21T14:59:27.751 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:27.751 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.751 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.751 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:27.751 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493315+0000", 2024-09-21T14:59:27.751 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:27.752 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.752 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.752 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:27.752 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493314+0000", 2024-09-21T14:59:27.752 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:59:27.752 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.752 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.752 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:27.752 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493507+0000", 2024-09-21T14:59:27.752 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000192444 2024-09-21T14:59:27.752 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.752 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.752 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:27.753 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493528+0000", 2024-09-21T14:59:27.753 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0845000000000001e-05 2024-09-21T14:59:27.753 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.753 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.753 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:27.753 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493543+0000", 2024-09-21T14:59:27.753 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5800000000000001e-05 2024-09-21T14:59:27.753 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.753 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.753 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:27.753 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.502575+0000", 2024-09-21T14:59:27.753 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0090312389999999999 2024-09-21T14:59:27.753 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.754 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.754 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:27.754 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.502677+0000", 2024-09-21T14:59:27.754 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000102271 2024-09-21T14:59:27.754 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.754 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.754 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:59:27.754 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.502825+0000", 2024-09-21T14:59:27.754 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000148171 2024-09-21T14:59:27.754 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.754 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.754 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:27.755 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507251+0000", 2024-09-21T14:59:27.755 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0044253879999999997 2024-09-21T14:59:27.755 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.755 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.755 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:27.755 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507313+0000", 2024-09-21T14:59:27.755 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2170999999999998e-05 2024-09-21T14:59:27.755 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.755 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.755 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:27.755 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507346+0000", 2024-09-21T14:59:27.755 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3079e-05 2024-09-21T14:59:27.755 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.756 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.756 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:27.756 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507371+0000", 2024-09-21T14:59:27.756 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5477999999999999e-05 2024-09-21T14:59:27.756 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:27.756 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:27.756 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:27.756 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.756 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.756 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T14:59:27.756 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:04.493570+0000", 2024-09-21T14:59:27.756 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 83.229666464000005, 2024-09-21T14:59:27.756 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014061686, 2024-09-21T14:59:27.756 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:27.757 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:27.757 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:27.757 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:59:27.757 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:59:27.757 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 138 2024-09-21T14:59:27.757 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.757 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:27.757 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.757 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:27.757 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493570+0000", 2024-09-21T14:59:27.757 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:27.757 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.757 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.757 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:27.758 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493570+0000", 2024-09-21T14:59:27.758 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:27.758 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.758 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.758 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:27.758 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493568+0000", 2024-09-21T14:59:27.758 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T14:59:27.758 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.758 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.758 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:27.758 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493579+0000", 2024-09-21T14:59:27.758 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0574000000000001e-05 2024-09-21T14:59:27.759 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.759 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.759 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:27.759 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493595+0000", 2024-09-21T14:59:27.759 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6504999999999999e-05 2024-09-21T14:59:27.759 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.759 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.759 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:27.759 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493608+0000", 2024-09-21T14:59:27.759 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2836999999999999e-05 2024-09-21T14:59:27.759 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.760 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.760 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:27.760 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.503909+0000", 2024-09-21T14:59:27.760 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.01030118 2024-09-21T14:59:27.760 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.760 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.760 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:27.760 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.504064+0000", 2024-09-21T14:59:27.760 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015493000000000001 2024-09-21T14:59:27.760 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.760 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.760 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:59:27.760 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.504299+0000", 2024-09-21T14:59:27.761 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00023416100000000001 2024-09-21T14:59:27.761 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.761 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.761 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:27.761 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507466+0000", 2024-09-21T14:59:27.761 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0031678380000000001 2024-09-21T14:59:27.761 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.761 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.761 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:27.761 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507483+0000", 2024-09-21T14:59:27.761 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6606999999999999e-05 2024-09-21T14:59:27.761 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.762 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.762 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:27.762 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507566+0000", 2024-09-21T14:59:27.762 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3110999999999998e-05 2024-09-21T14:59:27.762 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.762 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.762 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:27.762 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507632+0000", 2024-09-21T14:59:27.762 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5480000000000003e-05 2024-09-21T14:59:27.762 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:27.762 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:27.763 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:27.763 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.763 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.763 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T14:59:27.763 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:04.493625+0000", 2024-09-21T14:59:27.763 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 83.229611775999999, 2024-09-21T14:59:27.763 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014665378999999999, 2024-09-21T14:59:27.763 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:27.763 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:27.763 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:27.763 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:59:27.764 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:59:27.764 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139 2024-09-21T14:59:27.764 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.764 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:27.764 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.764 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:27.764 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493625+0000", 2024-09-21T14:59:27.764 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:27.764 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.764 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.764 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:27.764 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493625+0000", 2024-09-21T14:59:27.764 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:27.765 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.765 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.765 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:27.765 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493624+0000", 2024-09-21T14:59:27.765 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:59:27.765 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.765 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.765 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:27.765 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493638+0000", 2024-09-21T14:59:27.765 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4282e-05 2024-09-21T14:59:27.765 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.765 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.765 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:27.766 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493654+0000", 2024-09-21T14:59:27.766 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5739999999999998e-05 2024-09-21T14:59:27.766 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.766 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.766 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:27.766 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493666+0000", 2024-09-21T14:59:27.766 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2865e-05 2024-09-21T14:59:27.766 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.766 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.766 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:27.766 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.505329+0000", 2024-09-21T14:59:27.767 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.011662237000000001 2024-09-21T14:59:27.767 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.767 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.767 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:27.767 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.505377+0000", 2024-09-21T14:59:27.767 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7862999999999999e-05 2024-09-21T14:59:27.767 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.767 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.767 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:59:27.767 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.505455+0000", 2024-09-21T14:59:27.767 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8918000000000006e-05 2024-09-21T14:59:27.767 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.767 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.768 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:27.768 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507650+0000", 2024-09-21T14:59:27.768 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.002195027 2024-09-21T14:59:27.768 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.768 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.768 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:27.768 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508149+0000", 2024-09-21T14:59:27.768 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00049828399999999999 2024-09-21T14:59:27.768 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.768 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.768 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:27.769 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508230+0000", 2024-09-21T14:59:27.769 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1376000000000002e-05 2024-09-21T14:59:27.769 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.769 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.769 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:27.769 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508290+0000", 2024-09-21T14:59:27.770 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9852999999999998e-05 2024-09-21T14:59:27.770 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:27.770 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:27.770 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:27.770 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.770 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.771 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T14:59:27.771 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:04.493679+0000", 2024-09-21T14:59:27.771 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 83.229556877999997, 2024-09-21T14:59:27.771 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014869869000000001, 2024-09-21T14:59:27.771 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:27.772 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:27.772 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:27.772 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:59:27.772 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:59:27.772 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140 2024-09-21T14:59:27.772 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.772 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:27.773 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.773 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:27.773 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493679+0000", 2024-09-21T14:59:27.773 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:27.773 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.773 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.773 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:27.773 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493679+0000", 2024-09-21T14:59:27.774 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:27.774 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.774 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.774 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:27.774 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493679+0000", 2024-09-21T14:59:27.774 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:59:27.774 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.774 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.774 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:27.775 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493889+0000", 2024-09-21T14:59:27.775 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00021032700000000001 2024-09-21T14:59:27.775 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.775 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.775 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:27.775 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493931+0000", 2024-09-21T14:59:27.775 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1971000000000002e-05 2024-09-21T14:59:27.775 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.775 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.776 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:27.776 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493947+0000", 2024-09-21T14:59:27.776 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6385e-05 2024-09-21T14:59:27.776 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.776 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.776 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:27.776 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.505804+0000", 2024-09-21T14:59:27.776 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.011857136000000001 2024-09-21T14:59:27.776 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.776 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.776 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:27.776 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.505992+0000", 2024-09-21T14:59:27.777 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000187665 2024-09-21T14:59:27.777 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.777 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.777 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:59:27.777 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.506281+0000", 2024-09-21T14:59:27.777 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000288712 2024-09-21T14:59:27.777 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.777 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.777 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:27.777 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508349+0000", 2024-09-21T14:59:27.777 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0020685790000000001 2024-09-21T14:59:27.777 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.777 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.777 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:27.778 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508371+0000", 2024-09-21T14:59:27.778 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1747e-05 2024-09-21T14:59:27.778 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.778 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.778 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:27.778 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508450+0000", 2024-09-21T14:59:27.778 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9332000000000005e-05 2024-09-21T14:59:27.778 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.778 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.778 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:27.779 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508549+0000", 2024-09-21T14:59:27.779 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8918000000000004e-05 2024-09-21T14:59:27.779 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:27.779 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:27.779 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:27.779 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.779 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.779 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T14:59:27.779 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:04.493967+0000", 2024-09-21T14:59:27.779 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 83.229269841000004, 2024-09-21T14:59:27.779 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.015281075999999999, 2024-09-21T14:59:27.779 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:27.779 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:27.780 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:27.780 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:59:27.780 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:59:27.780 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 141 2024-09-21T14:59:27.780 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.780 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:27.780 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.780 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:27.780 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493967+0000", 2024-09-21T14:59:27.780 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:27.780 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.780 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.780 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:27.781 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493967+0000", 2024-09-21T14:59:27.781 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:27.781 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.781 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.781 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:27.781 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493965+0000", 2024-09-21T14:59:27.781 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T14:59:27.781 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.781 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.781 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:27.781 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493987+0000", 2024-09-21T14:59:27.781 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1662999999999999e-05 2024-09-21T14:59:27.781 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.782 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.782 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:27.782 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.494004+0000", 2024-09-21T14:59:27.782 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7311000000000001e-05 2024-09-21T14:59:27.782 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.782 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.782 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:27.782 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.494017+0000", 2024-09-21T14:59:27.782 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3202e-05 2024-09-21T14:59:27.782 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.782 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.782 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:27.783 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507678+0000", 2024-09-21T14:59:27.783 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013660432 2024-09-21T14:59:27.783 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.783 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.783 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:27.783 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507726+0000", 2024-09-21T14:59:27.783 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8154e-05 2024-09-21T14:59:27.783 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.783 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.783 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:59:27.783 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507797+0000", 2024-09-21T14:59:27.783 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.1110000000000002e-05 2024-09-21T14:59:27.783 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.784 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.784 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:27.784 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508710+0000", 2024-09-21T14:59:27.784 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00091339299999999997 2024-09-21T14:59:27.784 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.784 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.784 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:27.784 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.509108+0000", 2024-09-21T14:59:27.784 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00039718300000000001 2024-09-21T14:59:27.784 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.784 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.784 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:27.784 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.509186+0000", 2024-09-21T14:59:27.784 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8592999999999998e-05 2024-09-21T14:59:27.784 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.785 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.785 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:27.785 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.509248+0000", 2024-09-21T14:59:27.785 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1450000000000003e-05 2024-09-21T14:59:27.785 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:27.785 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:27.785 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:27.785 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.785 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.785 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6626 3.1a 3:5adee3e5:::rbd_data.11361bed1f78.0000000000000001:d [read 229376~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:59:27.785 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131593+0000", 2024-09-21T14:59:27.785 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 52.591643263000002, 2024-09-21T14:59:27.785 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012576725, 2024-09-21T14:59:27.785 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:27.786 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:27.786 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:27.786 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:27.786 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:27.786 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6626 2024-09-21T14:59:27.786 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.786 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:27.786 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.786 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:27.786 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131593+0000", 2024-09-21T14:59:27.786 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:27.787 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.787 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.787 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:27.787 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131593+0000", 2024-09-21T14:59:27.787 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:27.787 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.787 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.787 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:27.787 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131593+0000", 2024-09-21T14:59:27.787 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T14:59:27.787 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.787 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.787 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:27.787 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131598+0000", 2024-09-21T14:59:27.787 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7849999999999999e-06 2024-09-21T14:59:27.788 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.788 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.788 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:27.788 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131601+0000", 2024-09-21T14:59:27.788 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1949999999999999e-06 2024-09-21T14:59:27.788 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.788 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.788 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:27.788 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131604+0000", 2024-09-21T14:59:27.788 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6350000000000002e-06 2024-09-21T14:59:27.788 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.788 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.788 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:27.788 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.143902+0000", 2024-09-21T14:59:27.788 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012298216000000001 2024-09-21T14:59:27.789 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.789 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.789 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:27.789 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.143993+0000", 2024-09-21T14:59:27.789 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1073e-05 2024-09-21T14:59:27.789 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.789 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.789 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:27.789 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.144170+0000", 2024-09-21T14:59:27.789 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017703 2024-09-21T14:59:27.789 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:27.789 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:27.789 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:27.789 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.789 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.790 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6631 3.1a 3:5d547f54:::rbd_data.11361bed1f78.0000000000000003:d [read 0~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:59:27.790 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131606+0000", 2024-09-21T14:59:27.790 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 52.591630563000002, 2024-09-21T14:59:27.790 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013903148000000001, 2024-09-21T14:59:27.790 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:27.790 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:27.790 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:27.790 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:27.790 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:27.790 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6631 2024-09-21T14:59:27.790 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.790 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:27.790 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.791 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:27.791 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131606+0000", 2024-09-21T14:59:27.791 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:27.791 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.791 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.791 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:27.791 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131606+0000", 2024-09-21T14:59:27.791 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:27.791 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.791 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.791 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:27.791 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131606+0000", 2024-09-21T14:59:27.791 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T14:59:27.791 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.792 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.792 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:27.792 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131607+0000", 2024-09-21T14:59:27.792 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2699999999999999e-06 2024-09-21T14:59:27.792 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.792 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.792 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:27.792 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131610+0000", 2024-09-21T14:59:27.792 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.88e-06 2024-09-21T14:59:27.792 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.792 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.792 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:27.792 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131612+0000", 2024-09-21T14:59:27.792 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5170000000000002e-06 2024-09-21T14:59:27.792 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.793 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.793 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:27.793 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.144205+0000", 2024-09-21T14:59:27.793 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012592763 2024-09-21T14:59:27.793 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.793 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.793 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:27.793 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.144332+0000", 2024-09-21T14:59:27.793 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012694200000000001 2024-09-21T14:59:27.793 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.793 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.793 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:27.793 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.145509+0000", 2024-09-21T14:59:27.793 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001176992 2024-09-21T14:59:27.794 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:27.794 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:27.794 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:27.794 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.794 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.794 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6632 3.1a 3:5d547f54:::rbd_data.11361bed1f78.0000000000000003:d [read 81920~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:59:27.794 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131614+0000", 2024-09-21T14:59:27.794 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 52.591621955000001, 2024-09-21T14:59:27.794 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014093553999999999, 2024-09-21T14:59:27.794 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:27.794 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:27.795 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:27.795 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:27.795 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:27.795 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6632 2024-09-21T14:59:27.795 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.795 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:27.795 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.795 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:27.795 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131614+0000", 2024-09-21T14:59:27.795 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:27.795 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.795 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.795 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:27.796 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131614+0000", 2024-09-21T14:59:27.796 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:27.796 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.796 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.796 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:27.796 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131614+0000", 2024-09-21T14:59:27.796 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T14:59:27.796 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.796 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.796 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:27.796 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131615+0000", 2024-09-21T14:59:27.796 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.218e-06 2024-09-21T14:59:27.796 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.796 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.797 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:27.797 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131618+0000", 2024-09-21T14:59:27.797 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9050000000000001e-06 2024-09-21T14:59:27.797 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.797 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.797 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:27.797 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131621+0000", 2024-09-21T14:59:27.797 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7439999999999999e-06 2024-09-21T14:59:27.797 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.797 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.797 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:27.797 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.145538+0000", 2024-09-21T14:59:27.797 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013916618 2024-09-21T14:59:27.798 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.798 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.798 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:27.798 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.145619+0000", 2024-09-21T14:59:27.798 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1812999999999997e-05 2024-09-21T14:59:27.798 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.798 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.798 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:27.798 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.145708+0000", 2024-09-21T14:59:27.798 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8460000000000003e-05 2024-09-21T14:59:27.798 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:27.798 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:27.798 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:27.799 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.799 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.799 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6633 3.1a 3:5d547f54:::rbd_data.11361bed1f78.0000000000000003:d [read 147456~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:59:27.799 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131623+0000", 2024-09-21T14:59:27.799 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 52.591612988999998, 2024-09-21T14:59:27.799 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014395845000000001, 2024-09-21T14:59:27.799 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:27.799 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:27.799 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:27.799 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:27.799 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:27.799 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6633 2024-09-21T14:59:27.799 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.800 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:27.800 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.800 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:27.800 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131623+0000", 2024-09-21T14:59:27.800 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:27.800 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.800 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.800 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:27.800 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131623+0000", 2024-09-21T14:59:27.800 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:27.800 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.800 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.800 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:27.801 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131623+0000", 2024-09-21T14:59:27.801 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T14:59:27.801 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.801 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.801 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:27.801 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131628+0000", 2024-09-21T14:59:27.801 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0370000000000002e-06 2024-09-21T14:59:27.801 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.801 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.801 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:27.802 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131632+0000", 2024-09-21T14:59:27.802 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3519999999999998e-06 2024-09-21T14:59:27.802 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.802 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.802 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:27.802 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131634+0000", 2024-09-21T14:59:27.802 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5019999999999999e-06 2024-09-21T14:59:27.802 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.802 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.802 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:27.802 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.145734+0000", 2024-09-21T14:59:27.802 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014100118 2024-09-21T14:59:27.802 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.803 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.803 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:27.803 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.145804+0000", 2024-09-21T14:59:27.803 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0229999999999997e-05 2024-09-21T14:59:27.803 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.803 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.803 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:27.803 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146019+0000", 2024-09-21T14:59:27.803 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00021481699999999999 2024-09-21T14:59:27.803 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:27.803 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:27.803 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:27.803 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.804 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.804 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6634 3.1a 3:5d547f54:::rbd_data.11361bed1f78.0000000000000003:d [read 229376~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:59:27.804 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131636+0000", 2024-09-21T14:59:27.804 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 52.591599879, 2024-09-21T14:59:27.804 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014645843, 2024-09-21T14:59:27.804 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:27.804 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:27.804 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:27.804 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:27.804 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:27.804 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6634 2024-09-21T14:59:27.804 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.804 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:27.805 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.805 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:27.805 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131636+0000", 2024-09-21T14:59:27.805 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:27.805 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.805 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.805 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:27.805 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131636+0000", 2024-09-21T14:59:27.805 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:27.805 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.805 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.805 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:27.805 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131636+0000", 2024-09-21T14:59:27.805 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:27.806 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.806 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.806 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:27.806 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131641+0000", 2024-09-21T14:59:27.806 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8489999999999997e-06 2024-09-21T14:59:27.806 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.806 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.806 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:27.806 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131644+0000", 2024-09-21T14:59:27.806 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.179e-06 2024-09-21T14:59:27.806 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.806 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.807 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:27.807 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131647+0000", 2024-09-21T14:59:27.807 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8009999999999999e-06 2024-09-21T14:59:27.807 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.807 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.807 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:27.807 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146046+0000", 2024-09-21T14:59:27.807 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014399018 2024-09-21T14:59:27.807 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.807 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.807 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:27.807 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146118+0000", 2024-09-21T14:59:27.807 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2044000000000003e-05 2024-09-21T14:59:27.807 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.808 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.808 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:27.808 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146282+0000", 2024-09-21T14:59:27.808 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000164196 2024-09-21T14:59:27.808 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:27.808 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:27.808 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:27.808 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.808 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.808 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6639 3.1a 3:592d80ed:::rbd_data.11361bed1f78.0000000000000005:d [read 0~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:59:27.808 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131650+0000", 2024-09-21T14:59:27.808 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 52.591586679999999, 2024-09-21T14:59:27.808 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016002832000000002, 2024-09-21T14:59:27.809 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:27.809 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:27.809 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:27.809 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:27.809 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:27.809 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6639 2024-09-21T14:59:27.809 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.809 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:27.809 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.809 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:27.809 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131650+0000", 2024-09-21T14:59:27.809 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:27.809 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.809 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.810 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:27.810 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131650+0000", 2024-09-21T14:59:27.810 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:27.810 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.810 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.810 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:27.810 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131649+0000", 2024-09-21T14:59:27.810 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:27.810 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.810 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.810 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:27.810 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131651+0000", 2024-09-21T14:59:27.811 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9199999999999998e-06 2024-09-21T14:59:27.811 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.811 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.811 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:27.811 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131656+0000", 2024-09-21T14:59:27.811 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9899999999999997e-06 2024-09-21T14:59:27.811 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.811 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.811 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:27.811 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131665+0000", 2024-09-21T14:59:27.811 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0999999999999993e-06 2024-09-21T14:59:27.811 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.811 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.811 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:27.812 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146309+0000", 2024-09-21T14:59:27.812 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014643893999999999 2024-09-21T14:59:27.812 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.812 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.812 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:27.812 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146392+0000", 2024-09-21T14:59:27.812 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2531e-05 2024-09-21T14:59:27.812 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.812 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.812 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:27.812 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147653+0000", 2024-09-21T14:59:27.812 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0012606710000000001 2024-09-21T14:59:27.812 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:27.812 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:27.813 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:27.813 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.813 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.813 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6640 3.1a 3:592d80ed:::rbd_data.11361bed1f78.0000000000000005:d [read 65536~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:59:27.813 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131670+0000", 2024-09-21T14:59:27.813 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 52.591566381, 2024-09-21T14:59:27.813 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016188859, 2024-09-21T14:59:27.813 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:27.813 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:27.813 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:27.813 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:27.813 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:27.813 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6640 2024-09-21T14:59:27.813 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.813 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:27.814 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.814 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:27.814 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131670+0000", 2024-09-21T14:59:27.814 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:27.814 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.814 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.814 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:27.814 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131670+0000", 2024-09-21T14:59:27.814 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:27.814 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.814 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.814 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:27.815 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131670+0000", 2024-09-21T14:59:27.815 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T14:59:27.815 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.815 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.815 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:27.815 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131672+0000", 2024-09-21T14:59:27.815 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.818e-06 2024-09-21T14:59:27.815 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.815 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.815 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:27.815 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131675+0000", 2024-09-21T14:59:27.815 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5930000000000002e-06 2024-09-21T14:59:27.815 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.815 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.816 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:27.816 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131678+0000", 2024-09-21T14:59:27.816 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6790000000000001e-06 2024-09-21T14:59:27.816 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.816 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.816 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:27.816 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147681+0000", 2024-09-21T14:59:27.816 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016002755 2024-09-21T14:59:27.816 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.816 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.816 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:27.816 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147762+0000", 2024-09-21T14:59:27.816 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1052999999999998e-05 2024-09-21T14:59:27.816 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.817 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.817 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:27.817 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147859+0000", 2024-09-21T14:59:27.817 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.7199000000000002e-05 2024-09-21T14:59:27.817 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:27.817 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:27.817 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:27.817 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.817 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.817 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6641 3.1a 3:592d80ed:::rbd_data.11361bed1f78.0000000000000005:d [read 147456~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:59:27.817 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131680+0000", 2024-09-21T14:59:27.817 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 52.591556093000001, 2024-09-21T14:59:27.817 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016431296000000001, 2024-09-21T14:59:27.817 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:27.818 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:27.818 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:27.818 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:27.818 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:27.818 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6641 2024-09-21T14:59:27.818 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.818 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:27.818 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.818 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:27.818 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131680+0000", 2024-09-21T14:59:27.818 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:27.818 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.818 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.819 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:27.819 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131680+0000", 2024-09-21T14:59:27.819 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:27.819 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.819 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.819 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:27.819 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131680+0000", 2024-09-21T14:59:27.819 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T14:59:27.819 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.819 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.819 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:27.819 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131681+0000", 2024-09-21T14:59:27.819 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2610000000000001e-06 2024-09-21T14:59:27.819 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.819 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.820 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:27.820 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131685+0000", 2024-09-21T14:59:27.820 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.8299999999999998e-06 2024-09-21T14:59:27.820 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.820 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.820 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:27.820 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131688+0000", 2024-09-21T14:59:27.820 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6189999999999998e-06 2024-09-21T14:59:27.820 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.820 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.820 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:27.820 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147884+0000", 2024-09-21T14:59:27.820 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016196465 2024-09-21T14:59:27.820 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.820 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.821 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:27.821 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147949+0000", 2024-09-21T14:59:27.821 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4867000000000001e-05 2024-09-21T14:59:27.821 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.821 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.821 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:27.821 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.148112+0000", 2024-09-21T14:59:27.821 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00016247700000000001 2024-09-21T14:59:27.821 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:27.821 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:27.821 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:27.821 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.821 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.821 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6642 3.1a 3:592d80ed:::rbd_data.11361bed1f78.0000000000000005:d [read 212992~32768 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:59:27.822 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131690+0000", 2024-09-21T14:59:27.822 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 52.591546215000001, 2024-09-21T14:59:27.822 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016751335999999999, 2024-09-21T14:59:27.822 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:27.822 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:27.822 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:27.822 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:27.822 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:27.822 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6642 2024-09-21T14:59:27.822 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.822 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:27.822 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.822 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:27.822 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131690+0000", 2024-09-21T14:59:27.823 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:27.823 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.823 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.823 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:27.823 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131690+0000", 2024-09-21T14:59:27.823 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:27.823 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.823 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.823 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:27.823 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131690+0000", 2024-09-21T14:59:27.823 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T14:59:27.823 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.823 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.823 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:27.823 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131691+0000", 2024-09-21T14:59:27.824 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3120000000000001e-06 2024-09-21T14:59:27.824 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.824 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.824 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:27.824 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131694+0000", 2024-09-21T14:59:27.824 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0070000000000002e-06 2024-09-21T14:59:27.824 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.824 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.824 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:27.824 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131700+0000", 2024-09-21T14:59:27.824 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0800000000000002e-06 2024-09-21T14:59:27.824 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.824 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.824 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:27.824 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.148135+0000", 2024-09-21T14:59:27.825 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016434250000000001 2024-09-21T14:59:27.825 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.825 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.825 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:27.825 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.148204+0000", 2024-09-21T14:59:27.825 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9466000000000002e-05 2024-09-21T14:59:27.825 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.825 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.825 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:27.825 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.148441+0000", 2024-09-21T14:59:27.825 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00023743700000000001 2024-09-21T14:59:27.825 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:27.825 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:27.825 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:27.826 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.826 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.826 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:61214 3.7 3:e28aec10:::rbd_header.1136862f872f:head [watch reconnect cookie 139864671932080 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-21T14:59:27.826 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:18.147172+0000", 2024-09-21T14:59:27.826 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 9.5760645849999992, 2024-09-21T14:59:27.826 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.019264956, 2024-09-21T14:59:27.826 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:27.826 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:27.826 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:27.826 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:27.826 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:27.826 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 61214 2024-09-21T14:59:27.826 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.826 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:27.826 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.827 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:27.827 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147172+0000", 2024-09-21T14:59:27.827 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:27.827 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.827 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.827 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:27.827 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147172+0000", 2024-09-21T14:59:27.827 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:27.827 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.827 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.827 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:27.827 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147168+0000", 2024-09-21T14:59:27.827 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999962 2024-09-21T14:59:27.828 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.828 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.828 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:27.828 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147188+0000", 2024-09-21T14:59:27.828 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9717000000000001e-05 2024-09-21T14:59:27.828 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.828 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.828 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:27.828 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147239+0000", 2024-09-21T14:59:27.828 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1493999999999999e-05 2024-09-21T14:59:27.828 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.828 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.828 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:27.828 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147265+0000", 2024-09-21T14:59:27.829 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5746999999999999e-05 2024-09-21T14:59:27.829 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.829 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.829 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:27.829 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147372+0000", 2024-09-21T14:59:27.829 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010681200000000001 2024-09-21T14:59:27.829 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.829 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.829 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op must wait for map", 2024-09-21T14:59:27.829 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147380+0000", 2024-09-21T14:59:27.829 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7919999999999993e-06 2024-09-21T14:59:27.829 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.829 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.830 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:27.830 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.154025+0000", 2024-09-21T14:59:27.830 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0066454909999999999 2024-09-21T14:59:27.830 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.830 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.830 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T14:59:27.831 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.154041+0000", 2024-09-21T14:59:27.831 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5894000000000001e-05 2024-09-21T14:59:27.831 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.831 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.831 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:27.832 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.164345+0000", 2024-09-21T14:59:27.832 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0103040459999999 2024-09-21T14:59:27.832 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.832 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.832 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:27.832 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.164590+0000", 2024-09-21T14:59:27.833 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00024492699999999998 2024-09-21T14:59:27.833 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.833 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.833 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:27.833 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.166437+0000", 2024-09-21T14:59:27.833 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0018466330000000001 2024-09-21T14:59:27.833 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:27.834 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:27.834 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:27.834 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.834 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.834 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:61215 3.7 3:e28aec10:::rbd_header.1136862f872f:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-21T14:59:27.834 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:18.152008+0000", 2024-09-21T14:59:27.834 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 9.571228133, 2024-09-21T14:59:27.834 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.015644958, 2024-09-21T14:59:27.834 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:27.834 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:27.834 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:27.834 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:27.835 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:27.835 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 61215 2024-09-21T14:59:27.835 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.835 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:27.835 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.835 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:27.835 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152008+0000", 2024-09-21T14:59:27.836 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:27.836 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.836 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.836 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:27.836 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152008+0000", 2024-09-21T14:59:27.836 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:27.836 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.836 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.836 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:27.836 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152005+0000", 2024-09-21T14:59:27.836 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-21T14:59:27.836 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.836 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.836 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:27.836 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152025+0000", 2024-09-21T14:59:27.837 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9306000000000001e-05 2024-09-21T14:59:27.837 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.837 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.837 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:27.837 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152054+0000", 2024-09-21T14:59:27.837 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9189000000000001e-05 2024-09-21T14:59:27.837 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.837 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.837 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:27.837 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152078+0000", 2024-09-21T14:59:27.837 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4317000000000001e-05 2024-09-21T14:59:27.837 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.837 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.837 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:27.837 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152243+0000", 2024-09-21T14:59:27.838 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000164802 2024-09-21T14:59:27.838 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.838 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.838 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting_for_map not empty", 2024-09-21T14:59:27.838 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152251+0000", 2024-09-21T14:59:27.838 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0600000000000008e-06 2024-09-21T14:59:27.838 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.838 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.838 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:27.838 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.154741+0000", 2024-09-21T14:59:27.838 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0024900069999999998 2024-09-21T14:59:27.838 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.838 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.838 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T14:59:27.839 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.154753+0000", 2024-09-21T14:59:27.839 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1588e-05 2024-09-21T14:59:27.839 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.839 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.839 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:27.839 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.165380+0000", 2024-09-21T14:59:27.839 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0106272249999999 2024-09-21T14:59:27.839 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.839 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.839 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:27.839 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.165517+0000", 2024-09-21T14:59:27.839 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013737999999999999 2024-09-21T14:59:27.839 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.839 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.839 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 6", 2024-09-21T14:59:27.840 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.165938+0000", 2024-09-21T14:59:27.840 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00042082399999999998 2024-09-21T14:59:27.840 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.840 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.840 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:27.840 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.166795+0000", 2024-09-21T14:59:27.840 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00085674899999999997 2024-09-21T14:59:27.840 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.840 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.840 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:27.840 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.167598+0000", 2024-09-21T14:59:27.840 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00080335599999999997 2024-09-21T14:59:27.840 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.841 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.841 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:27.841 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.167632+0000", 2024-09-21T14:59:27.841 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4122000000000003e-05 2024-09-21T14:59:27.841 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:27.841 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:27.841 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:27.841 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.167653+0000", 2024-09-21T14:59:27.841 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0862000000000001e-05 2024-09-21T14:59:27.841 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:27.841 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:27.841 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:27.841 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:27.841 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:27.841 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:27.843 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T14:59:27.843 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:59:27.841+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:59:27.944 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:59:27.939+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:59:28.003 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:28.003 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:59:28.003 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T14:59:28.003 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:28.015 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T14:59:28.044 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:59:28.045+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:59:28.128 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:28.128 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:59:28.128 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T14:59:28.128 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T14:59:28.128 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:28.138 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T14:59:28.146 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:59:28.143+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:59:28.227 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T14:59:28.247 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:59:28.245+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:59:28.324 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:28.325 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T14:59:28.325 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T14:59:28.325 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T14:59:28.325 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.325 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:15130 3.2 3:47f1bbdb:::rbd_data.11361bed1f78.0000000000000067:head [sparse-read 0~147456] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-21T14:59:28.325 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:40.288284+0000", 2024-09-21T14:59:28.325 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 48.035536065000002, 2024-09-21T14:59:28.325 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1142935039999999, 2024-09-21T14:59:28.325 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:28.325 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:28.325 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:28.326 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:28.326 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:28.326 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 15130 2024-09-21T14:59:28.326 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.326 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:28.326 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.326 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:28.326 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288284+0000", 2024-09-21T14:59:28.326 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:28.326 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.326 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.326 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:28.326 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288284+0000", 2024-09-21T14:59:28.327 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:28.327 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.327 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.327 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:28.327 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288284+0000", 2024-09-21T14:59:28.327 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:28.327 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.327 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.327 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:28.327 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288288+0000", 2024-09-21T14:59:28.327 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5829999999999998e-06 2024-09-21T14:59:28.327 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.327 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.327 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:28.327 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288297+0000", 2024-09-21T14:59:28.328 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0669999999999998e-06 2024-09-21T14:59:28.328 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.328 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.328 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:28.328 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288305+0000", 2024-09-21T14:59:28.328 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8860000000000004e-06 2024-09-21T14:59:28.328 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.328 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.328 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:28.328 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288417+0000", 2024-09-21T14:59:28.328 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000112952 2024-09-21T14:59:28.328 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.328 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.328 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T14:59:28.328 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288428+0000", 2024-09-21T14:59:28.329 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0631e-05 2024-09-21T14:59:28.329 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.329 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.329 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:28.329 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.401876+0000", 2024-09-21T14:59:28.329 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1134479610000001 2024-09-21T14:59:28.329 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.329 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.329 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:28.329 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.402076+0000", 2024-09-21T14:59:28.329 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00019944200000000001 2024-09-21T14:59:28.329 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.329 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.329 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:28.329 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.402578+0000", 2024-09-21T14:59:28.330 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00050242899999999998 2024-09-21T14:59:28.330 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:28.330 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:28.330 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:28.330 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.330 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.330 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45708 3.2 3:415db91a:::rbd_data.1136ecbe5879.0000000000000000:28 [read 140288~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:28.330 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141801+0000", 2024-09-21T14:59:28.330 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.182019871000001, 2024-09-21T14:59:28.330 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034274200999999997, 2024-09-21T14:59:28.330 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:28.330 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:28.330 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:28.331 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:28.331 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:28.331 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45708 2024-09-21T14:59:28.331 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.331 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:28.331 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.331 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:28.331 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141801+0000", 2024-09-21T14:59:28.331 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:28.331 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.331 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.331 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:28.331 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141801+0000", 2024-09-21T14:59:28.332 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:28.332 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.332 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.332 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:28.332 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141800+0000", 2024-09-21T14:59:28.332 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:28.332 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.332 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.332 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:28.332 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141803+0000", 2024-09-21T14:59:28.332 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4210000000000002e-06 2024-09-21T14:59:28.332 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.333 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.333 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:28.333 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141812+0000", 2024-09-21T14:59:28.333 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.9800000000000004e-06 2024-09-21T14:59:28.333 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.333 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.333 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:28.333 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141816+0000", 2024-09-21T14:59:28.333 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4769999999999997e-06 2024-09-21T14:59:28.333 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.333 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.333 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:28.333 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.175865+0000", 2024-09-21T14:59:28.333 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034048875999999999 2024-09-21T14:59:28.334 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.334 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.334 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:28.334 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.175943+0000", 2024-09-21T14:59:28.334 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7464000000000002e-05 2024-09-21T14:59:28.334 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.334 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.334 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:28.334 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176075+0000", 2024-09-21T14:59:28.334 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013230800000000001 2024-09-21T14:59:28.334 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:28.334 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:28.335 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:28.335 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.335 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.335 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45709 3.2 3:415db91a:::rbd_data.1136ecbe5879.0000000000000000:28 [read 143360~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:28.335 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141820+0000", 2024-09-21T14:59:28.335 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.182000024000001, 2024-09-21T14:59:28.335 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034537536000000001, 2024-09-21T14:59:28.335 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:28.335 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:28.335 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:28.335 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:28.335 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:28.335 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45709 2024-09-21T14:59:28.335 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.336 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:28.336 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.336 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:28.336 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141820+0000", 2024-09-21T14:59:28.336 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:28.336 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.336 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.336 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:28.336 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141820+0000", 2024-09-21T14:59:28.336 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:28.336 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.336 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.336 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:28.336 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141820+0000", 2024-09-21T14:59:28.336 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:28.337 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.337 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.337 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:28.337 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141823+0000", 2024-09-21T14:59:28.337 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5110000000000002e-06 2024-09-21T14:59:28.337 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.337 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.337 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:28.337 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141829+0000", 2024-09-21T14:59:28.337 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0850000000000002e-06 2024-09-21T14:59:28.337 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.337 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.337 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:28.337 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141833+0000", 2024-09-21T14:59:28.337 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7620000000000003e-06 2024-09-21T14:59:28.338 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.338 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.338 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:28.338 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176103+0000", 2024-09-21T14:59:28.338 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034269933000000002 2024-09-21T14:59:28.338 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.338 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.338 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:28.338 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176161+0000", 2024-09-21T14:59:28.338 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7658999999999998e-05 2024-09-21T14:59:28.338 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.338 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.339 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:28.339 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176358+0000", 2024-09-21T14:59:28.339 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000196937 2024-09-21T14:59:28.339 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:28.339 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:28.339 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:28.339 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.339 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.339 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45710 3.2 3:415db91a:::rbd_data.1136ecbe5879.0000000000000000:28 [read 146432~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:28.339 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141838+0000", 2024-09-21T14:59:28.339 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.181982861000002, 2024-09-21T14:59:28.339 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034953674999999997, 2024-09-21T14:59:28.339 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:28.340 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:28.340 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:28.340 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:28.340 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:28.340 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45710 2024-09-21T14:59:28.340 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.340 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:28.340 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.340 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:28.340 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141838+0000", 2024-09-21T14:59:28.340 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:28.340 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.340 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.340 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:28.341 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141838+0000", 2024-09-21T14:59:28.341 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:28.341 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.341 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.341 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:28.341 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141837+0000", 2024-09-21T14:59:28.341 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:28.341 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.341 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.341 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:28.341 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141843+0000", 2024-09-21T14:59:28.341 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1619999999999996e-06 2024-09-21T14:59:28.341 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.341 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.342 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:28.342 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141850+0000", 2024-09-21T14:59:28.342 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4799999999999998e-06 2024-09-21T14:59:28.342 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.342 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.342 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:28.342 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141853+0000", 2024-09-21T14:59:28.342 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3050000000000001e-06 2024-09-21T14:59:28.342 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.342 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.342 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:28.342 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176490+0000", 2024-09-21T14:59:28.342 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034636413999999997 2024-09-21T14:59:28.343 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.343 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.343 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:28.343 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176606+0000", 2024-09-21T14:59:28.343 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000115841 2024-09-21T14:59:28.343 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.343 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.343 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:28.343 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176791+0000", 2024-09-21T14:59:28.343 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00018581500000000001 2024-09-21T14:59:28.343 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:28.343 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:28.343 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:28.343 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.343 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.344 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45711 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:28.344 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141856+0000", 2024-09-21T14:59:28.344 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.181964524000001, 2024-09-21T14:59:28.344 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034986308000000001, 2024-09-21T14:59:28.344 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:28.344 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:28.344 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:28.344 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:28.344 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:28.344 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45711 2024-09-21T14:59:28.344 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.344 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:28.344 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.345 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:28.345 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141856+0000", 2024-09-21T14:59:28.345 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:28.345 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.345 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.345 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:28.345 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141856+0000", 2024-09-21T14:59:28.345 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:28.345 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.345 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.345 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:28.345 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141856+0000", 2024-09-21T14:59:28.345 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:28.346 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.346 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.346 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:28.346 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141862+0000", 2024-09-21T14:59:28.346 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.567e-06 2024-09-21T14:59:28.346 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.346 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.346 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:28.346 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141868+0000", 2024-09-21T14:59:28.346 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9569999999999999e-06 2024-09-21T14:59:28.346 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.346 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.346 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:28.346 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141871+0000", 2024-09-21T14:59:28.347 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.146e-06 2024-09-21T14:59:28.347 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.347 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.347 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:28.347 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176831+0000", 2024-09-21T14:59:28.347 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034959935999999997 2024-09-21T14:59:28.347 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.347 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.347 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:28.347 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176842+0000", 2024-09-21T14:59:28.347 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1008e-05 2024-09-21T14:59:28.347 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:28.348 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:28.348 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:28.348 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.348 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.348 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45712 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:28.348 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141874+0000", 2024-09-21T14:59:28.348 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.181946628999999, 2024-09-21T14:59:28.348 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034993511999999997, 2024-09-21T14:59:28.348 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:28.348 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:28.348 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:28.348 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:28.348 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:28.349 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45712 2024-09-21T14:59:28.349 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.349 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:28.349 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.349 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:28.349 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141874+0000", 2024-09-21T14:59:28.349 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:28.349 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.349 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.349 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:28.349 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141874+0000", 2024-09-21T14:59:28.349 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:28.349 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.349 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.350 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:28.350 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141874+0000", 2024-09-21T14:59:28.350 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:28.350 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.350 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.350 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:28.350 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141875+0000", 2024-09-21T14:59:28.350 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5430000000000001e-06 2024-09-21T14:59:28.350 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.350 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.350 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:28.351 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141878+0000", 2024-09-21T14:59:28.351 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1420000000000002e-06 2024-09-21T14:59:28.351 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.351 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.351 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:28.351 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141881+0000", 2024-09-21T14:59:28.351 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.807e-06 2024-09-21T14:59:28.351 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.351 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.351 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:28.351 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176860+0000", 2024-09-21T14:59:28.351 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034979243 2024-09-21T14:59:28.351 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.352 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.352 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:28.352 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176867+0000", 2024-09-21T14:59:28.352 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0169999999999999e-06 2024-09-21T14:59:28.352 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:28.352 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:28.352 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:28.353 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.353 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.353 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45713 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:28.353 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141883+0000", 2024-09-21T14:59:28.353 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.181937053999999, 2024-09-21T14:59:28.353 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035010516999999998, 2024-09-21T14:59:28.353 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:28.353 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:28.353 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:28.353 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:28.353 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:28.354 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45713 2024-09-21T14:59:28.354 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.354 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:28.354 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.354 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:28.354 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141883+0000", 2024-09-21T14:59:28.354 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:28.354 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.354 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.354 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:28.354 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141883+0000", 2024-09-21T14:59:28.354 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:28.354 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.355 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.355 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:28.355 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141883+0000", 2024-09-21T14:59:28.355 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:28.355 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.355 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.355 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:28.355 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141885+0000", 2024-09-21T14:59:28.355 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4649999999999999e-06 2024-09-21T14:59:28.355 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.355 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.355 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:28.355 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141888+0000", 2024-09-21T14:59:28.355 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3890000000000001e-06 2024-09-21T14:59:28.356 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.356 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.356 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:28.356 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141891+0000", 2024-09-21T14:59:28.356 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9239999999999999e-06 2024-09-21T14:59:28.356 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.356 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.356 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:28.356 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176887+0000", 2024-09-21T14:59:28.356 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034996370999999998 2024-09-21T14:59:28.356 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.356 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.356 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:28.356 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176894+0000", 2024-09-21T14:59:28.357 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6069999999999996e-06 2024-09-21T14:59:28.357 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:28.357 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:28.357 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:28.357 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.357 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.357 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45714 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:28.357 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141893+0000", 2024-09-21T14:59:28.357 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.181927138999999, 2024-09-21T14:59:28.357 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035048215000000001, 2024-09-21T14:59:28.357 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:28.357 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:28.357 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:28.357 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:28.358 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:28.358 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45714 2024-09-21T14:59:28.358 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.358 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:28.358 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.358 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:28.358 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141893+0000", 2024-09-21T14:59:28.358 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:28.358 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.358 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.358 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:28.358 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141893+0000", 2024-09-21T14:59:28.358 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:28.359 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.359 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.359 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:28.359 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141893+0000", 2024-09-21T14:59:28.359 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:28.359 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.359 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.359 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:28.359 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141895+0000", 2024-09-21T14:59:28.359 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.703e-06 2024-09-21T14:59:28.359 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.359 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.359 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:28.360 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141900+0000", 2024-09-21T14:59:28.360 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.242e-06 2024-09-21T14:59:28.360 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.360 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.360 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:28.360 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141903+0000", 2024-09-21T14:59:28.360 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.117e-06 2024-09-21T14:59:28.360 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.360 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.360 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:28.360 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176930+0000", 2024-09-21T14:59:28.360 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035026750000000002 2024-09-21T14:59:28.360 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.360 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.361 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:28.361 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176942+0000", 2024-09-21T14:59:28.361 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1668000000000001e-05 2024-09-21T14:59:28.361 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:28.361 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:28.361 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:28.361 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.361 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.361 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45715 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:28.361 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141906+0000", 2024-09-21T14:59:28.361 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.181914980999998, 2024-09-21T14:59:28.362 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035066057999999997, 2024-09-21T14:59:28.362 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:28.362 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:28.362 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:28.362 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:28.362 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:28.362 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45715 2024-09-21T14:59:28.362 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.362 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:28.362 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.362 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:28.362 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141906+0000", 2024-09-21T14:59:28.363 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:28.363 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.363 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.363 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:28.363 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141906+0000", 2024-09-21T14:59:28.363 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:28.363 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.363 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.363 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:28.363 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141905+0000", 2024-09-21T14:59:28.363 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:28.363 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.363 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.363 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:28.364 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141907+0000", 2024-09-21T14:59:28.364 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5400000000000001e-06 2024-09-21T14:59:28.364 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.364 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.364 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:28.364 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141912+0000", 2024-09-21T14:59:28.364 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.502e-06 2024-09-21T14:59:28.364 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.364 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.364 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:28.364 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141915+0000", 2024-09-21T14:59:28.364 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0170000000000001e-06 2024-09-21T14:59:28.364 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.364 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.365 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:28.365 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176964+0000", 2024-09-21T14:59:28.365 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035048587999999999 2024-09-21T14:59:28.365 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.365 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.365 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:28.365 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176972+0000", 2024-09-21T14:59:28.365 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6580000000000002e-06 2024-09-21T14:59:28.365 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:28.365 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:28.365 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:28.365 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.365 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.365 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45716 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:28.365 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141918+0000", 2024-09-21T14:59:28.366 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.181902805, 2024-09-21T14:59:28.366 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035086223, 2024-09-21T14:59:28.366 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:28.366 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:28.366 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:28.366 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:28.366 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:28.366 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45716 2024-09-21T14:59:28.366 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.366 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:28.366 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.366 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:28.366 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141918+0000", 2024-09-21T14:59:28.366 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:28.367 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.367 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.367 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:28.367 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141918+0000", 2024-09-21T14:59:28.367 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:28.367 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.367 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.367 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:28.367 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141917+0000", 2024-09-21T14:59:28.367 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:28.367 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.367 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.368 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:28.368 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141919+0000", 2024-09-21T14:59:28.368 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5680000000000001e-06 2024-09-21T14:59:28.368 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.368 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.368 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:28.368 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141922+0000", 2024-09-21T14:59:28.368 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2789999999999999e-06 2024-09-21T14:59:28.368 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.368 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.368 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:28.369 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141925+0000", 2024-09-21T14:59:28.369 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9469999999999999e-06 2024-09-21T14:59:28.369 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.369 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.369 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:28.369 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176997+0000", 2024-09-21T14:59:28.369 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035071366999999999 2024-09-21T14:59:28.369 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.369 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.369 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:28.369 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177004+0000", 2024-09-21T14:59:28.369 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3309999999999998e-06 2024-09-21T14:59:28.369 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:28.369 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:28.369 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:28.370 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.370 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.370 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45717 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:28.370 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141928+0000", 2024-09-21T14:59:28.370 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.181892859000001, 2024-09-21T14:59:28.370 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035102573999999998, 2024-09-21T14:59:28.370 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:28.370 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:28.370 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:28.370 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:28.370 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:28.370 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45717 2024-09-21T14:59:28.370 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.370 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:28.371 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.371 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:28.371 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141928+0000", 2024-09-21T14:59:28.371 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:28.371 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.371 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.371 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:28.371 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141928+0000", 2024-09-21T14:59:28.371 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:28.371 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.371 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.371 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:28.371 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141927+0000", 2024-09-21T14:59:28.371 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T14:59:28.371 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.372 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.372 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:28.372 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141929+0000", 2024-09-21T14:59:28.372 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.533e-06 2024-09-21T14:59:28.372 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.372 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.372 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:28.372 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141934+0000", 2024-09-21T14:59:28.372 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1660000000000002e-06 2024-09-21T14:59:28.372 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.372 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.372 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:28.372 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141937+0000", 2024-09-21T14:59:28.372 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0309999999999999e-06 2024-09-21T14:59:28.372 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.372 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.373 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:28.373 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177023+0000", 2024-09-21T14:59:28.373 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035085595999999997 2024-09-21T14:59:28.373 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.373 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.373 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:28.373 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177030+0000", 2024-09-21T14:59:28.373 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.486e-06 2024-09-21T14:59:28.373 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:28.373 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:28.373 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:28.373 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.374 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.374 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45718 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:28.374 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141940+0000", 2024-09-21T14:59:28.374 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.181881001000001, 2024-09-21T14:59:28.374 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035124956999999998, 2024-09-21T14:59:28.374 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:28.374 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:28.374 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:28.374 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:28.374 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:28.374 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45718 2024-09-21T14:59:28.374 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.374 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:28.375 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.375 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:28.375 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141940+0000", 2024-09-21T14:59:28.375 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:28.375 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.375 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.375 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:28.375 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141940+0000", 2024-09-21T14:59:28.375 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:28.375 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.375 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.375 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:28.375 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141939+0000", 2024-09-21T14:59:28.376 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:28.376 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.376 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.376 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:28.376 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141941+0000", 2024-09-21T14:59:28.376 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.502e-06 2024-09-21T14:59:28.376 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.376 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.376 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:28.376 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141946+0000", 2024-09-21T14:59:28.376 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3449999999999996e-06 2024-09-21T14:59:28.376 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.376 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.376 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:28.377 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141949+0000", 2024-09-21T14:59:28.377 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9749999999999999e-06 2024-09-21T14:59:28.377 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.377 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.377 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:28.377 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177057+0000", 2024-09-21T14:59:28.377 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035108385999999998 2024-09-21T14:59:28.377 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.377 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.377 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:28.377 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177064+0000", 2024-09-21T14:59:28.377 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0099999999999998e-06 2024-09-21T14:59:28.377 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:28.377 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:28.378 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:28.378 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.378 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.378 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45719 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:28.378 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141952+0000", 2024-09-21T14:59:28.378 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.181868728000001, 2024-09-21T14:59:28.378 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035139108000000002, 2024-09-21T14:59:28.378 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:28.378 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:28.378 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:28.378 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:28.378 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:28.378 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45719 2024-09-21T14:59:28.378 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.378 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:28.379 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.379 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:28.379 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141952+0000", 2024-09-21T14:59:28.379 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:28.379 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.379 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.379 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:28.379 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141952+0000", 2024-09-21T14:59:28.379 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:28.379 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.379 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.379 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:28.379 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141952+0000", 2024-09-21T14:59:28.379 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:28.379 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.380 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.380 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:28.380 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141953+0000", 2024-09-21T14:59:28.380 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.486e-06 2024-09-21T14:59:28.380 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.380 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.380 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:28.380 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141959+0000", 2024-09-21T14:59:28.380 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4940000000000003e-06 2024-09-21T14:59:28.380 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.380 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.380 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:28.380 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141963+0000", 2024-09-21T14:59:28.380 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3540000000000002e-06 2024-09-21T14:59:28.380 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.381 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.381 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:28.381 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177084+0000", 2024-09-21T14:59:28.381 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035120983000000001 2024-09-21T14:59:28.381 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.381 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.381 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:28.381 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177091+0000", 2024-09-21T14:59:28.381 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0350000000000004e-06 2024-09-21T14:59:28.381 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:28.381 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:28.381 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:28.381 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.381 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.381 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45720 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:28.382 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141965+0000", 2024-09-21T14:59:28.382 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.181855260999999, 2024-09-21T14:59:28.382 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035155268000000003, 2024-09-21T14:59:28.382 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:28.382 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:28.382 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:28.382 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:28.382 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:28.382 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45720 2024-09-21T14:59:28.382 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.382 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:28.382 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.382 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:28.382 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141965+0000", 2024-09-21T14:59:28.383 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:28.383 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.383 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.383 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:28.383 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141965+0000", 2024-09-21T14:59:28.383 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:28.383 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.383 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.383 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:28.383 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141965+0000", 2024-09-21T14:59:28.383 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:28.383 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.383 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.383 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:28.384 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141966+0000", 2024-09-21T14:59:28.384 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.46e-06 2024-09-21T14:59:28.384 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.384 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.384 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:28.384 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141972+0000", 2024-09-21T14:59:28.384 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4210000000000003e-06 2024-09-21T14:59:28.384 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.384 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.384 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:28.384 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141975+0000", 2024-09-21T14:59:28.384 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.948e-06 2024-09-21T14:59:28.384 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.384 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.385 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:28.385 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177113+0000", 2024-09-21T14:59:28.385 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035137855000000003 2024-09-21T14:59:28.385 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.385 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.385 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:28.385 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177121+0000", 2024-09-21T14:59:28.385 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8269999999999998e-06 2024-09-21T14:59:28.385 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:28.385 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:28.385 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:28.385 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.385 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.385 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45721 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:28.386 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141977+0000", 2024-09-21T14:59:28.386 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.181843237999999, 2024-09-21T14:59:28.386 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035169984000000001, 2024-09-21T14:59:28.386 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:28.386 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:28.386 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:28.386 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:28.386 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:28.386 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45721 2024-09-21T14:59:28.386 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.386 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:28.386 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.386 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:28.387 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141977+0000", 2024-09-21T14:59:28.387 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:28.387 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.387 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.387 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:28.387 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141977+0000", 2024-09-21T14:59:28.387 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:28.387 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.387 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.387 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:28.387 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141977+0000", 2024-09-21T14:59:28.387 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:28.387 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.388 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.388 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:28.388 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141979+0000", 2024-09-21T14:59:28.388 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.511e-06 2024-09-21T14:59:28.388 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.388 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.388 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:28.388 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141982+0000", 2024-09-21T14:59:28.388 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4180000000000001e-06 2024-09-21T14:59:28.388 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.388 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.388 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:28.388 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141985+0000", 2024-09-21T14:59:28.389 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9450000000000002e-06 2024-09-21T14:59:28.389 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.389 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.389 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:28.389 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177139+0000", 2024-09-21T14:59:28.389 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035154409999999997 2024-09-21T14:59:28.389 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.389 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.389 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:28.389 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177147+0000", 2024-09-21T14:59:28.389 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9570000000000002e-06 2024-09-21T14:59:28.389 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:28.389 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:28.389 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:28.390 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.390 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.390 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45722 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:28.390 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141987+0000", 2024-09-21T14:59:28.390 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.181833266999998, 2024-09-21T14:59:28.390 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035188492000000002, 2024-09-21T14:59:28.390 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:28.390 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:28.390 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:28.390 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:28.390 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:28.390 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45722 2024-09-21T14:59:28.391 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.391 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:28.391 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.391 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:28.391 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141987+0000", 2024-09-21T14:59:28.391 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:28.391 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.391 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.391 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:28.391 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141987+0000", 2024-09-21T14:59:28.391 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:28.391 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.391 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.391 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:28.391 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141987+0000", 2024-09-21T14:59:28.392 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:28.392 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.392 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.392 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:28.392 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141988+0000", 2024-09-21T14:59:28.392 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.463e-06 2024-09-21T14:59:28.392 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.392 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.392 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:28.392 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141994+0000", 2024-09-21T14:59:28.392 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.361e-06 2024-09-21T14:59:28.392 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.392 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.392 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:28.393 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141997+0000", 2024-09-21T14:59:28.393 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0359999999999998e-06 2024-09-21T14:59:28.393 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.393 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.393 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:28.393 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177169+0000", 2024-09-21T14:59:28.393 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035171651999999998 2024-09-21T14:59:28.393 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.393 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.393 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:28.393 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177176+0000", 2024-09-21T14:59:28.393 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2189999999999998e-06 2024-09-21T14:59:28.393 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:28.393 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:28.394 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:28.394 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.394 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.394 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45723 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:28.394 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141999+0000", 2024-09-21T14:59:28.394 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.181821100000001, 2024-09-21T14:59:28.394 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035197701999999997, 2024-09-21T14:59:28.394 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:28.394 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:28.394 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:28.394 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:28.394 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:28.394 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45723 2024-09-21T14:59:28.394 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.394 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:28.395 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.395 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:28.395 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141999+0000", 2024-09-21T14:59:28.395 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:28.395 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.395 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.395 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:28.395 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141999+0000", 2024-09-21T14:59:28.395 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:28.395 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.395 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.395 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:28.395 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141999+0000", 2024-09-21T14:59:28.395 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:28.396 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.396 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.396 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:28.396 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142003+0000", 2024-09-21T14:59:28.396 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5049999999999999e-06 2024-09-21T14:59:28.396 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.396 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.396 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:28.396 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142008+0000", 2024-09-21T14:59:28.396 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6579999999999999e-06 2024-09-21T14:59:28.396 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.396 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.396 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:28.396 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142011+0000", 2024-09-21T14:59:28.396 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0529999999999998e-06 2024-09-21T14:59:28.397 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.397 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.397 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:28.397 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177194+0000", 2024-09-21T14:59:28.397 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035182221999999999 2024-09-21T14:59:28.397 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.397 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.397 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:28.397 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177197+0000", 2024-09-21T14:59:28.397 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5030000000000002e-06 2024-09-21T14:59:28.397 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:28.397 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:28.397 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:28.397 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.398 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.398 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45724 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:28.398 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.142015+0000", 2024-09-21T14:59:28.398 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.181805133000001, 2024-09-21T14:59:28.398 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035195345000000003, 2024-09-21T14:59:28.398 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:28.398 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:28.398 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:28.398 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:28.399 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:28.399 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45724 2024-09-21T14:59:28.399 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.399 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:28.399 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.399 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:28.399 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142015+0000", 2024-09-21T14:59:28.399 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:28.399 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.399 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.399 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:28.399 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142015+0000", 2024-09-21T14:59:28.399 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:28.399 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.400 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.400 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:28.400 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142015+0000", 2024-09-21T14:59:28.400 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:28.400 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.400 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.400 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:28.400 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142017+0000", 2024-09-21T14:59:28.400 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5540000000000001e-06 2024-09-21T14:59:28.400 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.400 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.400 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:28.400 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142022+0000", 2024-09-21T14:59:28.400 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4229999999999996e-06 2024-09-21T14:59:28.400 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.401 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.401 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:28.401 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142025+0000", 2024-09-21T14:59:28.401 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.056e-06 2024-09-21T14:59:28.401 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.401 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.401 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:28.401 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177207+0000", 2024-09-21T14:59:28.401 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035182089 2024-09-21T14:59:28.401 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.401 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.401 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:28.401 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177211+0000", 2024-09-21T14:59:28.401 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4649999999999999e-06 2024-09-21T14:59:28.401 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:28.402 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:28.402 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:28.402 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.402 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.402 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45725 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:28.402 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.142028+0000", 2024-09-21T14:59:28.402 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.181792854000001, 2024-09-21T14:59:28.402 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035195731000000001, 2024-09-21T14:59:28.402 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:28.402 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:28.402 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:28.402 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:28.402 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:28.402 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45725 2024-09-21T14:59:28.403 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.403 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:28.403 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.403 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:28.403 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142028+0000", 2024-09-21T14:59:28.403 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:28.403 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.403 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.403 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:28.403 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142028+0000", 2024-09-21T14:59:28.403 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:28.404 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.404 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.404 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:28.404 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142027+0000", 2024-09-21T14:59:28.404 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:28.404 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.404 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.404 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:28.404 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142029+0000", 2024-09-21T14:59:28.404 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5269999999999999e-06 2024-09-21T14:59:28.404 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.404 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.404 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:28.404 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142032+0000", 2024-09-21T14:59:28.404 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3249999999999999e-06 2024-09-21T14:59:28.405 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.405 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.405 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:28.405 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142035+0000", 2024-09-21T14:59:28.405 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.897e-06 2024-09-21T14:59:28.405 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.405 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.405 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:28.405 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177220+0000", 2024-09-21T14:59:28.405 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035184871 2024-09-21T14:59:28.405 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.405 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.406 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:28.406 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177223+0000", 2024-09-21T14:59:28.406 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3680000000000002e-06 2024-09-21T14:59:28.406 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:28.406 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:28.406 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:28.406 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.406 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.406 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45726 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:28.406 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.142037+0000", 2024-09-21T14:59:28.406 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.181783139, 2024-09-21T14:59:28.406 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035199273000000003, 2024-09-21T14:59:28.406 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:28.407 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:28.407 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:28.407 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:28.407 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:28.407 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45726 2024-09-21T14:59:28.407 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.407 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:28.407 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.407 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:28.407 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142037+0000", 2024-09-21T14:59:28.407 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:28.407 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.407 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.407 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:28.407 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142037+0000", 2024-09-21T14:59:28.408 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:28.408 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.408 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.408 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:28.408 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142037+0000", 2024-09-21T14:59:28.408 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:28.408 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.408 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.408 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:28.408 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142038+0000", 2024-09-21T14:59:28.408 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.344e-06 2024-09-21T14:59:28.408 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.408 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.408 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:28.408 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142044+0000", 2024-09-21T14:59:28.409 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1930000000000001e-06 2024-09-21T14:59:28.409 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.409 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.409 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:28.409 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142047+0000", 2024-09-21T14:59:28.409 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0280000000000001e-06 2024-09-21T14:59:28.409 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.409 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.409 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:28.409 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177233+0000", 2024-09-21T14:59:28.409 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035186394000000003 2024-09-21T14:59:28.409 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.409 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.409 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:28.410 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177237+0000", 2024-09-21T14:59:28.410 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5530000000000001e-06 2024-09-21T14:59:28.410 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:28.410 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:28.410 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:28.410 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:28.410 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:28.410 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:28.411 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T14:59:28.412 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:59:28.407+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:59:28.512 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:59:28.513+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:59:28.562 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:28.562 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:59:28.562 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T14:59:28.562 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:28.573 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T14:59:28.613 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:59:28.613+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:59:28.677 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:28.677 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:59:28.677 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T14:59:28.677 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T14:59:28.677 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:28.687 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T14:59:28.714 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:59:28.711+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:59:28.816 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:59:28.811+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:59:28.842 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:28.842 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T14:59:28.842 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T14:59:28.842 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T14:59:28.843 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.843 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4281.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-21T14:59:28.843 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:09.324416+0000", 2024-09-21T14:59:28.843 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 79.516536970000004, 2024-09-21T14:59:28.843 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.117837947, 2024-09-21T14:59:28.843 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:28.843 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:28.843 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:28.843 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4281", 2024-09-21T14:59:28.843 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/1518520303", 2024-09-21T14:59:28.843 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1 2024-09-21T14:59:28.843 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.843 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:28.844 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.844 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:28.844 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324416+0000", 2024-09-21T14:59:28.844 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:28.844 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.844 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.844 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:28.844 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324416+0000", 2024-09-21T14:59:28.844 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:28.844 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.844 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.844 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:28.844 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324414+0000", 2024-09-21T14:59:28.844 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T14:59:28.845 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.845 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.845 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:28.845 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324429+0000", 2024-09-21T14:59:28.845 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5637e-05 2024-09-21T14:59:28.845 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.845 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.845 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:28.845 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324448+0000", 2024-09-21T14:59:28.845 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9042000000000002e-05 2024-09-21T14:59:28.845 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.845 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.845 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:28.845 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324465+0000", 2024-09-21T14:59:28.845 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6767999999999999e-05 2024-09-21T14:59:28.845 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.846 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.846 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:28.846 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324571+0000", 2024-09-21T14:59:28.846 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000105714 2024-09-21T14:59:28.846 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.846 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.846 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op must wait for map", 2024-09-21T14:59:28.846 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324587+0000", 2024-09-21T14:59:28.846 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5753999999999999e-05 2024-09-21T14:59:28.846 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.846 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.847 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:28.847 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.325864+0000", 2024-09-21T14:59:28.847 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0012773350000000001 2024-09-21T14:59:28.847 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.847 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.847 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T14:59:28.847 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.325878+0000", 2024-09-21T14:59:28.847 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4498000000000001e-05 2024-09-21T14:59:28.847 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.847 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.847 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:28.847 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.439461+0000", 2024-09-21T14:59:28.847 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.11358212500000001 2024-09-21T14:59:28.847 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.847 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.847 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:28.848 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.439848+0000", 2024-09-21T14:59:28.848 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00038747800000000002 2024-09-21T14:59:28.848 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.848 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.848 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:59:28.848 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.440175+0000", 2024-09-21T14:59:28.848 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00032721900000000001 2024-09-21T14:59:28.848 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.848 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.848 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:28.848 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.441770+0000", 2024-09-21T14:59:28.848 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0015949409999999999 2024-09-21T14:59:28.848 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.848 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.848 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:28.849 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.442086+0000", 2024-09-21T14:59:28.849 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00031586299999999999 2024-09-21T14:59:28.849 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.849 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.849 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:28.849 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.442195+0000", 2024-09-21T14:59:28.849 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000108791 2024-09-21T14:59:28.849 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.849 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.849 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:28.849 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.442254+0000", 2024-09-21T14:59:28.849 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9038e-05 2024-09-21T14:59:28.849 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:28.849 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:28.850 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:28.850 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.850 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.850 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4281.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-21T14:59:28.850 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:09.447828+0000", 2024-09-21T14:59:28.850 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 79.393124968999999, 2024-09-21T14:59:28.850 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.32904393700000001, 2024-09-21T14:59:28.850 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:28.850 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:28.850 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:28.850 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4281", 2024-09-21T14:59:28.850 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/1518520303", 2024-09-21T14:59:28.850 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 2 2024-09-21T14:59:28.850 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.851 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:28.851 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.851 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:28.851 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447828+0000", 2024-09-21T14:59:28.851 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:28.851 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.851 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.851 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:28.851 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447828+0000", 2024-09-21T14:59:28.851 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:28.851 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.851 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.852 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:28.852 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447825+0000", 2024-09-21T14:59:28.852 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T14:59:28.852 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.852 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.852 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:28.852 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447841+0000", 2024-09-21T14:59:28.852 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5359999999999999e-05 2024-09-21T14:59:28.852 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.852 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.852 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:28.852 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447875+0000", 2024-09-21T14:59:28.852 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4388999999999999e-05 2024-09-21T14:59:28.852 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.853 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.853 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:28.853 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447903+0000", 2024-09-21T14:59:28.853 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7373e-05 2024-09-21T14:59:28.853 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.853 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.853 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:28.853 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447996+0000", 2024-09-21T14:59:28.853 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2954999999999993e-05 2024-09-21T14:59:28.853 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.853 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.853 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T14:59:28.853 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.448011+0000", 2024-09-21T14:59:28.853 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5038999999999999e-05 2024-09-21T14:59:28.853 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.854 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.854 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:28.854 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.776501+0000", 2024-09-21T14:59:28.854 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.32849012500000002 2024-09-21T14:59:28.854 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.854 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.854 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:28.854 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.776872+0000", 2024-09-21T14:59:28.854 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00037119600000000002 2024-09-21T14:59:28.854 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:28.854 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:28.855 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:28.855 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.855 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.855 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:752 3.14 3:28fc02e5:::rbd_data.113693316f36.0000000000000010:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-21T14:59:28.855 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:26.625579+0000", 2024-09-21T14:59:28.855 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 62.215374113999999, 2024-09-21T14:59:28.855 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042965426000000001, 2024-09-21T14:59:28.855 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:28.855 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:28.855 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:28.855 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:28.855 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:28.856 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 752 2024-09-21T14:59:28.856 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.856 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:28.856 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.856 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:28.856 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625579+0000", 2024-09-21T14:59:28.856 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:28.856 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.856 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.856 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:28.856 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625579+0000", 2024-09-21T14:59:28.856 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:28.856 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.856 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.857 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:28.857 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625577+0000", 2024-09-21T14:59:28.857 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T14:59:28.857 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.857 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.857 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:28.857 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625588+0000", 2024-09-21T14:59:28.857 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1555e-05 2024-09-21T14:59:28.857 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.857 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.857 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:28.857 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625610+0000", 2024-09-21T14:59:28.858 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2107999999999999e-05 2024-09-21T14:59:28.858 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.858 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.858 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:28.858 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625621+0000", 2024-09-21T14:59:28.858 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0762e-05 2024-09-21T14:59:28.858 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.858 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.858 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:28.858 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625692+0000", 2024-09-21T14:59:28.858 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.1203000000000003e-05 2024-09-21T14:59:28.858 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.858 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.859 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:28.859 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625872+0000", 2024-09-21T14:59:28.859 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017962800000000001 2024-09-21T14:59:28.859 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.859 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.859 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:59:28.859 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.633188+0000", 2024-09-21T14:59:28.859 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.007316014 2024-09-21T14:59:28.859 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.859 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.859 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:28.859 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.668357+0000", 2024-09-21T14:59:28.859 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035168933999999999 2024-09-21T14:59:28.860 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.860 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.860 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:28.860 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.668418+0000", 2024-09-21T14:59:28.860 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1282e-05 2024-09-21T14:59:28.860 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.860 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.860 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:28.860 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.668486+0000", 2024-09-21T14:59:28.860 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8468999999999998e-05 2024-09-21T14:59:28.860 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.860 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.861 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:28.861 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.668544+0000", 2024-09-21T14:59:28.861 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7785000000000003e-05 2024-09-21T14:59:28.861 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:28.861 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:28.861 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:28.861 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.861 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.861 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22737 3.a 3:574c7a60:::rbd_data.11366acc894.0000000000000002:19 [read 1032192~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:28.861 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141028+0000", 2024-09-21T14:59:28.861 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 38.699925149000002, 2024-09-21T14:59:28.861 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035206281999999998, 2024-09-21T14:59:28.861 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:28.862 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:28.862 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:28.862 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:28.862 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:28.862 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22737 2024-09-21T14:59:28.862 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.862 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:28.862 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.862 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:28.862 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141028+0000", 2024-09-21T14:59:28.862 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:28.862 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.862 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.863 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:28.863 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141028+0000", 2024-09-21T14:59:28.863 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:28.863 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.863 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.863 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:28.863 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141027+0000", 2024-09-21T14:59:28.863 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:28.863 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.863 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.863 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:28.863 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141030+0000", 2024-09-21T14:59:28.864 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6189999999999998e-06 2024-09-21T14:59:28.864 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.864 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.864 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:28.864 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141036+0000", 2024-09-21T14:59:28.864 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5110000000000003e-06 2024-09-21T14:59:28.864 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.864 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.864 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:28.864 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141040+0000", 2024-09-21T14:59:28.864 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.34e-06 2024-09-21T14:59:28.864 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.864 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.865 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:28.865 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.175948+0000", 2024-09-21T14:59:28.865 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034908540000000002 2024-09-21T14:59:28.865 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.865 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.865 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:28.865 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176025+0000", 2024-09-21T14:59:28.865 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6412e-05 2024-09-21T14:59:28.865 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.865 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.865 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:28.865 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176234+0000", 2024-09-21T14:59:28.865 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000209247 2024-09-21T14:59:28.865 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:28.866 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:28.866 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:28.866 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.866 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.866 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22738 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 49152~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:28.866 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141049+0000", 2024-09-21T14:59:28.866 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 38.699903970999998, 2024-09-21T14:59:28.866 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035428754999999999, 2024-09-21T14:59:28.866 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:28.866 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:28.866 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:28.866 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:28.867 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:28.867 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22738 2024-09-21T14:59:28.867 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.867 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:28.867 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.867 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:28.867 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141049+0000", 2024-09-21T14:59:28.867 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:28.867 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.867 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.867 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:28.867 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141049+0000", 2024-09-21T14:59:28.867 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:28.867 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.867 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.868 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:28.868 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141049+0000", 2024-09-21T14:59:28.868 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:28.868 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.868 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.868 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:28.868 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141051+0000", 2024-09-21T14:59:28.868 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4980000000000001e-06 2024-09-21T14:59:28.868 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.868 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.868 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:28.868 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141057+0000", 2024-09-21T14:59:28.868 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8030000000000002e-06 2024-09-21T14:59:28.868 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.869 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.869 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:28.869 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141066+0000", 2024-09-21T14:59:28.869 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1160000000000005e-06 2024-09-21T14:59:28.869 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.869 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.869 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:28.869 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176264+0000", 2024-09-21T14:59:28.869 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035197580999999999 2024-09-21T14:59:28.869 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.869 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.869 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:28.869 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176327+0000", 2024-09-21T14:59:28.869 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3237000000000005e-05 2024-09-21T14:59:28.869 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.870 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.870 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:28.870 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176478+0000", 2024-09-21T14:59:28.870 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000150851 2024-09-21T14:59:28.870 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:28.870 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:28.870 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:28.870 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.870 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.870 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22739 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 114688~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:28.870 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141070+0000", 2024-09-21T14:59:28.870 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 38.699882729000002, 2024-09-21T14:59:28.870 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035679094000000001, 2024-09-21T14:59:28.871 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:28.871 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:28.871 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:28.871 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:28.871 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:28.871 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22739 2024-09-21T14:59:28.871 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.871 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:28.871 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.871 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:28.871 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141070+0000", 2024-09-21T14:59:28.872 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:28.872 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.872 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.872 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:28.872 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141070+0000", 2024-09-21T14:59:28.872 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:28.872 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.872 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.872 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:28.872 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141070+0000", 2024-09-21T14:59:28.872 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:28.872 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.872 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.872 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:28.873 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141072+0000", 2024-09-21T14:59:28.873 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.379e-06 2024-09-21T14:59:28.873 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.873 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.873 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:28.873 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141078+0000", 2024-09-21T14:59:28.873 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6710000000000004e-06 2024-09-21T14:59:28.873 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.873 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.873 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:28.873 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141083+0000", 2024-09-21T14:59:28.873 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7249999999999997e-06 2024-09-21T14:59:28.873 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.873 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.873 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:28.874 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176506+0000", 2024-09-21T14:59:28.874 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035423076999999997 2024-09-21T14:59:28.874 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.874 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.874 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:28.874 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176582+0000", 2024-09-21T14:59:28.874 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6305999999999998e-05 2024-09-21T14:59:28.874 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.874 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.874 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:28.874 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176749+0000", 2024-09-21T14:59:28.874 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000167262 2024-09-21T14:59:28.874 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:28.874 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:28.875 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:28.875 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.875 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.875 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22740 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 180224~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:28.875 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141090+0000", 2024-09-21T14:59:28.875 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 38.699863325000003, 2024-09-21T14:59:28.875 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036129535999999997, 2024-09-21T14:59:28.875 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:28.875 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:28.875 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:28.875 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:28.875 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:28.875 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22740 2024-09-21T14:59:28.875 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.876 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:28.876 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.876 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:28.876 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141090+0000", 2024-09-21T14:59:28.876 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:28.876 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.876 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.876 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:28.876 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141090+0000", 2024-09-21T14:59:28.876 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:28.876 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.876 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.876 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:28.877 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141089+0000", 2024-09-21T14:59:28.877 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:28.877 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.877 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.877 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:28.877 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141092+0000", 2024-09-21T14:59:28.877 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7709999999999998e-06 2024-09-21T14:59:28.877 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.877 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.877 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:28.877 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141098+0000", 2024-09-21T14:59:28.877 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.015e-06 2024-09-21T14:59:28.877 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.878 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.878 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:28.878 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141103+0000", 2024-09-21T14:59:28.878 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8629999999999999e-06 2024-09-21T14:59:28.878 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.878 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.878 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:28.878 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176778+0000", 2024-09-21T14:59:28.878 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035674984 2024-09-21T14:59:28.878 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.878 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.878 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:28.878 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176851+0000", 2024-09-21T14:59:28.878 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2861000000000006e-05 2024-09-21T14:59:28.879 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.879 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.879 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:28.879 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177219+0000", 2024-09-21T14:59:28.879 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00036845999999999998 2024-09-21T14:59:28.879 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:28.879 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:28.879 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:28.879 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.879 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.879 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22741 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 245760~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:28.879 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141107+0000", 2024-09-21T14:59:28.879 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 38.699845754000002, 2024-09-21T14:59:28.880 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036407729, 2024-09-21T14:59:28.880 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:28.880 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:28.880 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:28.880 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:28.880 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:28.880 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22741 2024-09-21T14:59:28.880 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.880 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:28.880 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.880 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:28.880 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141107+0000", 2024-09-21T14:59:28.880 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:28.880 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.880 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.881 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:28.881 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141107+0000", 2024-09-21T14:59:28.881 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:28.881 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.881 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.881 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:28.881 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141107+0000", 2024-09-21T14:59:28.881 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:28.881 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.881 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.881 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:28.881 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141109+0000", 2024-09-21T14:59:28.881 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6529999999999998e-06 2024-09-21T14:59:28.881 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.881 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.882 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:28.882 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141115+0000", 2024-09-21T14:59:28.882 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9649999999999996e-06 2024-09-21T14:59:28.882 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.882 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.882 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:28.882 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141121+0000", 2024-09-21T14:59:28.882 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1209999999999998e-06 2024-09-21T14:59:28.882 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.882 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.882 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:28.882 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177242+0000", 2024-09-21T14:59:28.882 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036121706000000003 2024-09-21T14:59:28.882 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.882 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.883 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:28.883 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177301+0000", 2024-09-21T14:59:28.883 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9008000000000002e-05 2024-09-21T14:59:28.883 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.883 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.883 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:28.883 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177515+0000", 2024-09-21T14:59:28.883 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00021362 2024-09-21T14:59:28.883 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:28.883 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:28.883 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:28.883 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.883 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.883 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22742 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 311296~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:28.883 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141126+0000", 2024-09-21T14:59:28.884 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 38.699827399999997, 2024-09-21T14:59:28.884 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036653537, 2024-09-21T14:59:28.884 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:28.884 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:28.884 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:28.884 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:28.884 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:28.884 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22742 2024-09-21T14:59:28.884 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.884 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:28.884 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.884 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:28.884 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141126+0000", 2024-09-21T14:59:28.885 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:28.885 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.885 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.885 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:28.885 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141126+0000", 2024-09-21T14:59:28.885 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:28.885 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.885 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.885 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:28.885 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141125+0000", 2024-09-21T14:59:28.885 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:28.885 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.885 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.885 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:28.885 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141128+0000", 2024-09-21T14:59:28.886 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.886e-06 2024-09-21T14:59:28.886 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.886 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.886 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:28.886 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141137+0000", 2024-09-21T14:59:28.886 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4399999999999994e-06 2024-09-21T14:59:28.886 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.886 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.886 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:28.886 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141143+0000", 2024-09-21T14:59:28.886 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1410000000000004e-06 2024-09-21T14:59:28.886 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.886 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.886 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:28.887 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177543+0000", 2024-09-21T14:59:28.887 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036400544999999999 2024-09-21T14:59:28.887 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.887 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.887 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:28.887 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177605+0000", 2024-09-21T14:59:28.887 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2206999999999996e-05 2024-09-21T14:59:28.887 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.887 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.887 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:28.887 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177779+0000", 2024-09-21T14:59:28.887 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017368099999999999 2024-09-21T14:59:28.887 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:28.887 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:28.888 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:28.888 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.888 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.888 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22743 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 376832~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:28.888 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141147+0000", 2024-09-21T14:59:28.888 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 38.699805916000003, 2024-09-21T14:59:28.888 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036915158000000003, 2024-09-21T14:59:28.888 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:28.888 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:28.888 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:28.888 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:28.888 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:28.888 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22743 2024-09-21T14:59:28.889 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.889 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:28.889 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.889 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:28.889 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141147+0000", 2024-09-21T14:59:28.889 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:28.889 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.889 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.889 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:28.889 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141147+0000", 2024-09-21T14:59:28.889 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:28.889 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.889 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.889 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:28.889 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141147+0000", 2024-09-21T14:59:28.890 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:28.890 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.890 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.890 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:28.890 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141149+0000", 2024-09-21T14:59:28.890 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.79e-06 2024-09-21T14:59:28.890 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.890 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.890 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:28.890 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141155+0000", 2024-09-21T14:59:28.890 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6699999999999999e-06 2024-09-21T14:59:28.890 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.890 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.890 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:28.891 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141160+0000", 2024-09-21T14:59:28.891 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5759999999999999e-06 2024-09-21T14:59:28.891 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.891 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.891 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:28.891 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177817+0000", 2024-09-21T14:59:28.891 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036657404999999997 2024-09-21T14:59:28.891 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.891 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.891 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:28.891 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177892+0000", 2024-09-21T14:59:28.891 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4602000000000002e-05 2024-09-21T14:59:28.891 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.891 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.892 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:28.892 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178062+0000", 2024-09-21T14:59:28.892 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000170464 2024-09-21T14:59:28.892 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:28.892 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:28.892 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:28.892 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.892 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.892 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22744 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 442368~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:28.892 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141164+0000", 2024-09-21T14:59:28.892 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 38.699788783000002, 2024-09-21T14:59:28.892 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037120806999999999, 2024-09-21T14:59:28.892 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:28.892 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:28.892 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:28.893 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:28.893 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:28.893 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22744 2024-09-21T14:59:28.893 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.893 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:28.893 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.893 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:28.893 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141164+0000", 2024-09-21T14:59:28.893 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:28.893 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.893 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.893 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:28.893 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141164+0000", 2024-09-21T14:59:28.893 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:28.893 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.893 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.894 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:28.894 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141164+0000", 2024-09-21T14:59:28.894 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:28.894 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.894 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.894 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:28.894 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141167+0000", 2024-09-21T14:59:28.894 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7159999999999999e-06 2024-09-21T14:59:28.894 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.894 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.894 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:28.894 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141173+0000", 2024-09-21T14:59:28.894 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.071e-06 2024-09-21T14:59:28.895 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.895 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.895 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:28.895 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141177+0000", 2024-09-21T14:59:28.895 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3610000000000003e-06 2024-09-21T14:59:28.895 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.895 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.895 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:28.895 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178083+0000", 2024-09-21T14:59:28.895 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036906405000000003 2024-09-21T14:59:28.895 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.895 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.895 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:28.895 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178143+0000", 2024-09-21T14:59:28.896 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0056999999999998e-05 2024-09-21T14:59:28.896 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.896 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.896 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:28.896 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178285+0000", 2024-09-21T14:59:28.896 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000141555 2024-09-21T14:59:28.896 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:28.896 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:28.896 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:28.896 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.896 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.896 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22745 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 507904~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:28.896 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141182+0000", 2024-09-21T14:59:28.896 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 38.699770463, 2024-09-21T14:59:28.897 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037332331000000003, 2024-09-21T14:59:28.897 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:28.897 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:28.897 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:28.897 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:28.897 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:28.897 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22745 2024-09-21T14:59:28.897 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.897 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:28.897 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.897 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:28.897 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141182+0000", 2024-09-21T14:59:28.897 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:28.897 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.897 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.898 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:28.898 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141182+0000", 2024-09-21T14:59:28.898 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:28.898 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.898 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.898 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:28.898 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141182+0000", 2024-09-21T14:59:28.898 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:28.898 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.898 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.898 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:28.898 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141185+0000", 2024-09-21T14:59:28.898 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.464e-06 2024-09-21T14:59:28.898 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.899 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.899 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:28.899 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141200+0000", 2024-09-21T14:59:28.899 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5886999999999999e-05 2024-09-21T14:59:28.899 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.899 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.899 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:28.899 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141571+0000", 2024-09-21T14:59:28.899 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000370909 2024-09-21T14:59:28.899 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.899 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.899 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:28.899 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178318+0000", 2024-09-21T14:59:28.899 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036746244999999997 2024-09-21T14:59:28.900 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.900 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.900 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:28.900 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178367+0000", 2024-09-21T14:59:28.900 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9317000000000002e-05 2024-09-21T14:59:28.900 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.900 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.900 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:28.900 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178515+0000", 2024-09-21T14:59:28.900 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014784600000000001 2024-09-21T14:59:28.900 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:28.900 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:28.901 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:28.901 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.901 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.901 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22746 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 573440~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:28.901 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141599+0000", 2024-09-21T14:59:28.901 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 38.699353831000003, 2024-09-21T14:59:28.901 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037402249999999998, 2024-09-21T14:59:28.901 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:28.901 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:28.901 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:28.901 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:28.901 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:28.901 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22746 2024-09-21T14:59:28.901 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.902 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:28.902 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.902 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:28.902 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141599+0000", 2024-09-21T14:59:28.902 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:28.902 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.902 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.902 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:28.902 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141599+0000", 2024-09-21T14:59:28.902 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:28.902 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.902 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.902 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:28.902 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141598+0000", 2024-09-21T14:59:28.903 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:59:28.903 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.903 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.903 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:28.903 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141637+0000", 2024-09-21T14:59:28.903 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.9119999999999998e-05 2024-09-21T14:59:28.903 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.903 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.903 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:28.903 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141647+0000", 2024-09-21T14:59:28.903 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5570000000000001e-06 2024-09-21T14:59:28.903 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.903 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.903 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:28.903 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141654+0000", 2024-09-21T14:59:28.904 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2049999999999996e-06 2024-09-21T14:59:28.904 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.904 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.904 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:28.904 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178553+0000", 2024-09-21T14:59:28.904 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036898528 2024-09-21T14:59:28.904 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.904 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.904 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:28.904 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178625+0000", 2024-09-21T14:59:28.904 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2669999999999994e-05 2024-09-21T14:59:28.904 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.905 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.905 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:28.905 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179001+0000", 2024-09-21T14:59:28.905 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000375977 2024-09-21T14:59:28.905 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:28.905 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:28.905 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:28.905 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.905 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.905 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22747 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 638976~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:28.905 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141660+0000", 2024-09-21T14:59:28.905 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 38.699292689000004, 2024-09-21T14:59:28.905 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037617352999999999, 2024-09-21T14:59:28.906 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:28.906 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:28.906 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:28.906 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:28.906 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:28.906 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22747 2024-09-21T14:59:28.906 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.906 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:28.906 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.906 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:28.906 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141660+0000", 2024-09-21T14:59:28.906 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:28.906 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.906 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.907 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:28.907 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141660+0000", 2024-09-21T14:59:28.907 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:28.907 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.907 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.907 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:28.907 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141660+0000", 2024-09-21T14:59:28.907 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:28.907 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.907 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.907 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:28.907 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141663+0000", 2024-09-21T14:59:28.907 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.569e-06 2024-09-21T14:59:28.907 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.908 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.908 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:28.908 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141670+0000", 2024-09-21T14:59:28.908 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5719999999999999e-06 2024-09-21T14:59:28.908 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.908 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.908 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:28.908 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141675+0000", 2024-09-21T14:59:28.908 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2889999999999997e-06 2024-09-21T14:59:28.908 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.908 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.908 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:28.908 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179028+0000", 2024-09-21T14:59:28.908 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037353381999999997 2024-09-21T14:59:28.909 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.909 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.909 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:28.909 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179099+0000", 2024-09-21T14:59:28.909 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0259000000000006e-05 2024-09-21T14:59:28.909 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.909 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.909 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:28.909 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179278+0000", 2024-09-21T14:59:28.909 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000178908 2024-09-21T14:59:28.909 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:28.909 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:28.910 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:28.910 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.910 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.910 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22748 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 704512~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:28.910 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141680+0000", 2024-09-21T14:59:28.910 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 38.699273050999999, 2024-09-21T14:59:28.910 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037806166000000002, 2024-09-21T14:59:28.910 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:28.910 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:28.910 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:28.910 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:28.910 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:28.910 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22748 2024-09-21T14:59:28.910 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.911 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:28.911 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.911 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:28.911 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141680+0000", 2024-09-21T14:59:28.911 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:28.911 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.911 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.911 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:28.911 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141680+0000", 2024-09-21T14:59:28.911 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:28.911 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.911 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.911 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:28.911 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141679+0000", 2024-09-21T14:59:28.912 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:28.912 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.912 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.912 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:28.912 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141683+0000", 2024-09-21T14:59:28.912 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3230000000000002e-06 2024-09-21T14:59:28.912 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.912 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.912 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:28.912 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141690+0000", 2024-09-21T14:59:28.912 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8680000000000001e-06 2024-09-21T14:59:28.912 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.912 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.913 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:28.913 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141694+0000", 2024-09-21T14:59:28.913 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6770000000000003e-06 2024-09-21T14:59:28.913 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.913 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.913 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:28.913 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179300+0000", 2024-09-21T14:59:28.913 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037605567999999999 2024-09-21T14:59:28.913 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.913 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.913 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:28.913 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179355+0000", 2024-09-21T14:59:28.913 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4771000000000001e-05 2024-09-21T14:59:28.913 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.913 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.914 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:28.914 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179486+0000", 2024-09-21T14:59:28.914 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013141699999999999 2024-09-21T14:59:28.914 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:28.914 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:28.914 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:28.914 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.914 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.914 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22749 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 770048~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:28.914 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141698+0000", 2024-09-21T14:59:28.914 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 38.699254678999999, 2024-09-21T14:59:28.914 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037967813000000003, 2024-09-21T14:59:28.914 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:28.915 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:28.915 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:28.915 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:28.915 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:28.915 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22749 2024-09-21T14:59:28.915 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.915 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:28.915 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.915 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:28.915 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141698+0000", 2024-09-21T14:59:28.915 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:28.915 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.915 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.915 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:28.916 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141698+0000", 2024-09-21T14:59:28.916 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:28.916 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.916 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.916 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:28.916 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141698+0000", 2024-09-21T14:59:28.916 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:28.916 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.916 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.916 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:28.916 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141701+0000", 2024-09-21T14:59:28.916 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8399999999999999e-06 2024-09-21T14:59:28.916 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.916 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.917 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:28.917 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141706+0000", 2024-09-21T14:59:28.917 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4979999999999997e-06 2024-09-21T14:59:28.917 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.917 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.917 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:28.917 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141711+0000", 2024-09-21T14:59:28.917 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3850000000000004e-06 2024-09-21T14:59:28.917 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.917 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.917 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:28.917 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179504+0000", 2024-09-21T14:59:28.917 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037793514 2024-09-21T14:59:28.917 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.918 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.918 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:28.918 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179546+0000", 2024-09-21T14:59:28.918 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1467999999999997e-05 2024-09-21T14:59:28.918 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.918 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.918 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:28.918 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179666+0000", 2024-09-21T14:59:28.918 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000120499 2024-09-21T14:59:28.918 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:28.918 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:28.918 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:28.918 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.918 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.919 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22750 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 835584~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:28.919 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141715+0000", 2024-09-21T14:59:28.919 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 38.699238377999997, 2024-09-21T14:59:28.919 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038173179000000002, 2024-09-21T14:59:28.919 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:28.919 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:28.919 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:28.919 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:28.919 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:28.919 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22750 2024-09-21T14:59:28.919 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.919 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:28.919 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.919 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:28.920 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141715+0000", 2024-09-21T14:59:28.920 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:28.920 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.920 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.920 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:28.920 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141715+0000", 2024-09-21T14:59:28.920 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:28.920 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.920 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.920 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:28.920 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141714+0000", 2024-09-21T14:59:28.920 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:28.920 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.920 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.920 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:28.921 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141716+0000", 2024-09-21T14:59:28.921 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1730000000000002e-06 2024-09-21T14:59:28.921 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.921 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.921 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:28.921 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141722+0000", 2024-09-21T14:59:28.921 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.417e-06 2024-09-21T14:59:28.921 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.921 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.921 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:28.921 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141726+0000", 2024-09-21T14:59:28.921 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4830000000000001e-06 2024-09-21T14:59:28.921 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.922 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.922 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:28.922 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179683+0000", 2024-09-21T14:59:28.922 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037956977000000003 2024-09-21T14:59:28.922 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.922 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.922 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:28.922 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179751+0000", 2024-09-21T14:59:28.922 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7762000000000006e-05 2024-09-21T14:59:28.922 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.922 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.922 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:28.922 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179888+0000", 2024-09-21T14:59:28.922 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000136635 2024-09-21T14:59:28.923 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:28.923 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:28.923 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:28.923 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.923 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.923 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22751 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 901120~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:28.923 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141730+0000", 2024-09-21T14:59:28.923 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 38.699222579000001, 2024-09-21T14:59:28.923 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038357011000000003, 2024-09-21T14:59:28.923 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:28.923 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:28.923 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:28.923 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:28.923 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:28.924 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22751 2024-09-21T14:59:28.924 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.924 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:28.924 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.924 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:28.924 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141730+0000", 2024-09-21T14:59:28.924 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:28.924 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.924 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.924 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:28.924 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141730+0000", 2024-09-21T14:59:28.924 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:28.924 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.924 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.925 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:28.925 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141730+0000", 2024-09-21T14:59:28.925 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:28.925 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.925 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.925 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:28.925 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141737+0000", 2024-09-21T14:59:28.925 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.64e-06 2024-09-21T14:59:28.925 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.925 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.925 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:28.925 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141743+0000", 2024-09-21T14:59:28.925 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8130000000000001e-06 2024-09-21T14:59:28.925 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.925 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.926 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:28.926 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141747+0000", 2024-09-21T14:59:28.926 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4920000000000004e-06 2024-09-21T14:59:28.926 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.926 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.926 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:28.926 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179918+0000", 2024-09-21T14:59:28.926 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038170822 2024-09-21T14:59:28.926 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.926 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.926 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:28.926 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179960+0000", 2024-09-21T14:59:28.926 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1999000000000002e-05 2024-09-21T14:59:28.926 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.926 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.927 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:28.927 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180087+0000", 2024-09-21T14:59:28.927 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012752799999999999 2024-09-21T14:59:28.927 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:28.927 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:28.927 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:28.927 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.927 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.927 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22752 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 966656~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:28.927 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141751+0000", 2024-09-21T14:59:28.927 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 38.699202259000003, 2024-09-21T14:59:28.927 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038553084000000001, 2024-09-21T14:59:28.927 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:28.927 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:28.928 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:28.928 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:28.928 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:28.928 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22752 2024-09-21T14:59:28.928 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.928 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:28.928 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.928 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:28.928 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141751+0000", 2024-09-21T14:59:28.928 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:28.928 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.928 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.928 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:28.928 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141751+0000", 2024-09-21T14:59:28.928 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:28.928 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.929 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.929 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:28.929 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141750+0000", 2024-09-21T14:59:28.929 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:28.929 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.929 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.929 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:28.929 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141753+0000", 2024-09-21T14:59:28.929 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5900000000000002e-06 2024-09-21T14:59:28.929 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.929 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.929 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:28.929 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141759+0000", 2024-09-21T14:59:28.929 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.947e-06 2024-09-21T14:59:28.930 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.930 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.930 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:28.930 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141764+0000", 2024-09-21T14:59:28.930 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8319999999999996e-06 2024-09-21T14:59:28.930 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.930 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.930 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:28.930 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180108+0000", 2024-09-21T14:59:28.930 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038344095000000002 2024-09-21T14:59:28.930 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.930 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.930 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:28.930 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180168+0000", 2024-09-21T14:59:28.931 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0047000000000003e-05 2024-09-21T14:59:28.931 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.931 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.931 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:28.931 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180304+0000", 2024-09-21T14:59:28.931 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013592500000000001 2024-09-21T14:59:28.931 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:28.931 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:28.931 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:28.931 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.931 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.931 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22753 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 1032192~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:28.931 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141768+0000", 2024-09-21T14:59:28.931 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 38.699185385, 2024-09-21T14:59:28.932 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038818810000000002, 2024-09-21T14:59:28.932 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:28.932 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:28.932 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:28.932 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:28.932 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:28.932 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22753 2024-09-21T14:59:28.932 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.932 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:28.932 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.932 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:28.932 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141768+0000", 2024-09-21T14:59:28.932 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:28.932 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.933 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.933 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:28.933 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141768+0000", 2024-09-21T14:59:28.933 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:28.933 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.933 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.933 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:28.933 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141767+0000", 2024-09-21T14:59:28.933 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:28.933 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.933 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.933 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:28.933 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141770+0000", 2024-09-21T14:59:28.933 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3329999999999999e-06 2024-09-21T14:59:28.934 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.934 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.934 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:28.934 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141775+0000", 2024-09-21T14:59:28.934 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2669999999999997e-06 2024-09-21T14:59:28.934 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.934 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.934 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:28.934 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141779+0000", 2024-09-21T14:59:28.934 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2259999999999999e-06 2024-09-21T14:59:28.935 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.935 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.935 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:28.935 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180325+0000", 2024-09-21T14:59:28.935 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038546428000000001 2024-09-21T14:59:28.935 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.935 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.935 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:28.935 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180401+0000", 2024-09-21T14:59:28.935 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5437000000000003e-05 2024-09-21T14:59:28.935 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:28.935 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:28.935 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:28.936 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180586+0000", 2024-09-21T14:59:28.936 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000185429 2024-09-21T14:59:28.936 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:28.936 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:28.936 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:28.936 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:28.936 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:28.936 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:28.936 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T14:59:28.938 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T14:59:28.939 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:59:28.935+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:59:29.038 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:59:29.037+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:59:29.056 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:59:29.057 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:59:29.057 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T14:59:29.057 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:59:29.068 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T14:59:29.091 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T14:59:29.112 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T14:59:29.134 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T14:59:29.139 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:59:29.135+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:59:29.154 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T14:59:29.176 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-21T14:59:29.179 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:29.179 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:29.189 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T14:59:29.199 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T14:59:29.231 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T14:59:29.233 INFO:teuthology.orchestra.run.smithi032.stderr:noscrub is set 2024-09-21T14:59:29.240 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:59:29.235+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:59:29.250 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T14:59:29.252 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:59:29.252 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:59:29.252 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:59:29.253 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:59:29.253 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:59:29.253 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:59:29.253 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:59:29.253 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:59:29.297 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:29.297 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:29.307 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T14:59:29.341 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:59:29.341+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:59:29.442 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:59:29.441+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:59:29.456 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:29.456 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:29.466 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T14:59:29.543 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:59:29.541+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:59:29.573 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:29.573 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:29.584 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T14:59:29.644 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:59:29.645+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:59:29.704 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:29.705 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:29.715 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T14:59:29.744 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:59:29.739+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:59:29.831 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:29.831 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:29.842 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T14:59:29.845 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:59:29.839+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:59:29.946 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:59:29.945+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:59:29.950 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:29.950 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:29.960 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T14:59:30.047 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:59:30.043+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:59:30.072 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:30.073 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:30.083 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T14:59:30.148 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:59:30.145+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:59:30.200 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:30.200 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:30.210 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T14:59:30.248 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:59:30.243+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:59:30.322 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:30.322 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:30.333 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T14:59:30.378 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:59:30.375+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:59:30.444 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:30.445 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:30.450 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:59:30.449+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:59:30.455 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T14:59:30.551 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:59:30.547+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:59:30.577 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:30.578 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:30.588 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T14:59:30.652 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:59:30.649+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:59:30.703 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:30.703 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:30.714 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T14:59:30.752 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:59:30.747+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:59:30.819 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:30.819 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:30.829 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T14:59:30.853 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:59:30.853+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:59:30.941 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:30.941 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:30.951 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T14:59:30.953 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:59:30.947+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:59:31.054 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:59:31.051+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:59:31.104 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:31.104 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:31.114 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T14:59:31.154 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:59:31.153+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:59:31.226 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:31.226 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:31.236 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T14:59:31.244 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T14:59:31.264 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T14:59:31.280 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:59:31.281+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:59:31.356 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:31.356 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:31.356 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:59:31.351+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:59:31.367 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T14:59:31.423 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T14:59:31.447 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T14:59:31.457 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:59:31.457+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:59:31.466 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T14:59:31.490 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T14:59:31.513 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-21T14:59:31.513 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:31.513 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:59:31.514 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T14:59:31.514 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T14:59:31.514 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:31.530 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T14:59:31.535 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T14:59:31.558 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:59:31.555+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:59:31.559 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T14:59:31.583 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T14:59:31.586 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:59:31.586 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:59:31.586 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:59:31.586 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:59:31.586 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:59:31.586 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:59:31.586 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:59:31.586 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:59:31.659 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:59:31.657+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:59:31.672 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:31.672 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T14:59:31.672 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T14:59:31.672 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T14:59:31.672 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.672 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:15130 3.2 3:47f1bbdb:::rbd_data.11361bed1f78.0000000000000067:head [sparse-read 0~147456] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-21T14:59:31.672 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:40.288284+0000", 2024-09-21T14:59:31.673 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 51.384473827999997, 2024-09-21T14:59:31.673 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1142935039999999, 2024-09-21T14:59:31.673 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:31.673 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:31.673 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:31.673 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:31.673 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:31.673 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 15130 2024-09-21T14:59:31.673 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.673 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:31.673 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.673 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:31.674 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288284+0000", 2024-09-21T14:59:31.674 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:31.674 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.674 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.674 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:31.674 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288284+0000", 2024-09-21T14:59:31.674 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:31.674 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.674 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.674 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:31.674 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288284+0000", 2024-09-21T14:59:31.674 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:31.674 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.675 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.675 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:31.675 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288288+0000", 2024-09-21T14:59:31.675 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5829999999999998e-06 2024-09-21T14:59:31.675 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.675 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.675 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:31.675 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288297+0000", 2024-09-21T14:59:31.675 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0669999999999998e-06 2024-09-21T14:59:31.675 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.675 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.675 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:31.675 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288305+0000", 2024-09-21T14:59:31.676 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8860000000000004e-06 2024-09-21T14:59:31.676 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.676 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.676 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:31.676 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288417+0000", 2024-09-21T14:59:31.676 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000112952 2024-09-21T14:59:31.676 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.676 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.676 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T14:59:31.676 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288428+0000", 2024-09-21T14:59:31.676 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0631e-05 2024-09-21T14:59:31.676 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.676 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.676 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:31.677 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.401876+0000", 2024-09-21T14:59:31.677 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1134479610000001 2024-09-21T14:59:31.677 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.677 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.677 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:31.677 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.402076+0000", 2024-09-21T14:59:31.677 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00019944200000000001 2024-09-21T14:59:31.677 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.677 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.677 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:31.677 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.402578+0000", 2024-09-21T14:59:31.677 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00050242899999999998 2024-09-21T14:59:31.677 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:31.677 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:31.677 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:31.678 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.678 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.678 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45708 3.2 3:415db91a:::rbd_data.1136ecbe5879.0000000000000000:28 [read 140288~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:31.678 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141801+0000", 2024-09-21T14:59:31.678 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 21.530957634, 2024-09-21T14:59:31.678 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034274200999999997, 2024-09-21T14:59:31.678 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:31.678 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:31.678 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:31.678 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:31.678 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:31.678 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45708 2024-09-21T14:59:31.678 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.678 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:31.679 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.679 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:31.679 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141801+0000", 2024-09-21T14:59:31.679 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:31.679 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.679 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.679 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:31.679 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141801+0000", 2024-09-21T14:59:31.679 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:31.679 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.679 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.679 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:31.679 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141800+0000", 2024-09-21T14:59:31.680 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:31.680 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.680 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.680 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:31.680 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141803+0000", 2024-09-21T14:59:31.680 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4210000000000002e-06 2024-09-21T14:59:31.680 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.680 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.680 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:31.680 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141812+0000", 2024-09-21T14:59:31.680 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.9800000000000004e-06 2024-09-21T14:59:31.680 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.680 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.681 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:31.681 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141816+0000", 2024-09-21T14:59:31.681 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4769999999999997e-06 2024-09-21T14:59:31.681 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.681 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.681 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:31.681 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.175865+0000", 2024-09-21T14:59:31.681 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034048875999999999 2024-09-21T14:59:31.681 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.681 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.681 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:31.681 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.175943+0000", 2024-09-21T14:59:31.681 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7464000000000002e-05 2024-09-21T14:59:31.681 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.682 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.682 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:31.682 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176075+0000", 2024-09-21T14:59:31.682 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013230800000000001 2024-09-21T14:59:31.682 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:31.682 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:31.682 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:31.682 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.682 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.682 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45709 3.2 3:415db91a:::rbd_data.1136ecbe5879.0000000000000000:28 [read 143360~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:31.682 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141820+0000", 2024-09-21T14:59:31.682 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 21.530937786999999, 2024-09-21T14:59:31.682 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034537536000000001, 2024-09-21T14:59:31.682 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:31.682 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:31.683 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:31.683 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:31.683 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:31.683 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45709 2024-09-21T14:59:31.683 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.683 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:31.683 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.683 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:31.683 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141820+0000", 2024-09-21T14:59:31.683 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:31.683 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.683 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.683 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:31.683 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141820+0000", 2024-09-21T14:59:31.683 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:31.684 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.684 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.684 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:31.684 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141820+0000", 2024-09-21T14:59:31.684 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:31.684 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.684 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.684 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:31.684 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141823+0000", 2024-09-21T14:59:31.684 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5110000000000002e-06 2024-09-21T14:59:31.684 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.684 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.684 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:31.684 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141829+0000", 2024-09-21T14:59:31.685 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0850000000000002e-06 2024-09-21T14:59:31.685 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.685 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.685 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:31.685 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141833+0000", 2024-09-21T14:59:31.685 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7620000000000003e-06 2024-09-21T14:59:31.685 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.685 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.685 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:31.685 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176103+0000", 2024-09-21T14:59:31.685 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034269933000000002 2024-09-21T14:59:31.685 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.685 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.685 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:31.686 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176161+0000", 2024-09-21T14:59:31.686 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7658999999999998e-05 2024-09-21T14:59:31.686 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.686 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.686 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:31.686 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176358+0000", 2024-09-21T14:59:31.686 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000196937 2024-09-21T14:59:31.686 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:31.686 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:31.686 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:31.686 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.686 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.686 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45710 3.2 3:415db91a:::rbd_data.1136ecbe5879.0000000000000000:28 [read 146432~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:31.687 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141838+0000", 2024-09-21T14:59:31.687 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 21.530920624, 2024-09-21T14:59:31.687 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034953674999999997, 2024-09-21T14:59:31.687 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:31.687 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:31.687 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:31.687 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:31.687 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:31.687 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45710 2024-09-21T14:59:31.687 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.687 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:31.687 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.687 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:31.687 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141838+0000", 2024-09-21T14:59:31.688 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:31.688 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.688 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.688 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:31.688 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141838+0000", 2024-09-21T14:59:31.688 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:31.688 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.688 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.688 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:31.688 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141837+0000", 2024-09-21T14:59:31.688 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:31.688 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.688 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.689 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:31.689 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141843+0000", 2024-09-21T14:59:31.689 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1619999999999996e-06 2024-09-21T14:59:31.689 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.689 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.689 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:31.689 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141850+0000", 2024-09-21T14:59:31.689 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4799999999999998e-06 2024-09-21T14:59:31.689 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.689 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.689 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:31.689 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141853+0000", 2024-09-21T14:59:31.689 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3050000000000001e-06 2024-09-21T14:59:31.689 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.689 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.690 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:31.690 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176490+0000", 2024-09-21T14:59:31.690 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034636413999999997 2024-09-21T14:59:31.690 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.690 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.690 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:31.690 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176606+0000", 2024-09-21T14:59:31.690 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000115841 2024-09-21T14:59:31.690 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.690 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.690 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:31.690 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176791+0000", 2024-09-21T14:59:31.690 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00018581500000000001 2024-09-21T14:59:31.690 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:31.691 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:31.691 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:31.691 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.691 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.691 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45711 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:31.691 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141856+0000", 2024-09-21T14:59:31.691 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 21.530902287, 2024-09-21T14:59:31.691 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034986308000000001, 2024-09-21T14:59:31.691 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:31.691 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:31.691 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:31.691 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:31.691 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:31.692 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45711 2024-09-21T14:59:31.692 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.692 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:31.692 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.692 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:31.692 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141856+0000", 2024-09-21T14:59:31.692 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:31.692 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.692 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.692 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:31.692 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141856+0000", 2024-09-21T14:59:31.692 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:31.692 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.693 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.693 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:31.693 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141856+0000", 2024-09-21T14:59:31.693 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:31.693 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.693 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.693 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:31.693 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141862+0000", 2024-09-21T14:59:31.693 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.567e-06 2024-09-21T14:59:31.693 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.693 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.693 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:31.693 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141868+0000", 2024-09-21T14:59:31.694 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9569999999999999e-06 2024-09-21T14:59:31.694 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.694 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.694 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:31.694 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141871+0000", 2024-09-21T14:59:31.694 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.146e-06 2024-09-21T14:59:31.694 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.694 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.694 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:31.694 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176831+0000", 2024-09-21T14:59:31.694 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034959935999999997 2024-09-21T14:59:31.694 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.694 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.694 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:31.695 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176842+0000", 2024-09-21T14:59:31.695 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1008e-05 2024-09-21T14:59:31.695 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:31.695 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:31.695 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:31.695 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.695 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.695 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45712 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:31.695 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141874+0000", 2024-09-21T14:59:31.695 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 21.530884392000001, 2024-09-21T14:59:31.695 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034993511999999997, 2024-09-21T14:59:31.695 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:31.695 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:31.695 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:31.695 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:31.696 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:31.696 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45712 2024-09-21T14:59:31.696 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.696 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:31.696 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.696 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:31.696 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141874+0000", 2024-09-21T14:59:31.696 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:31.696 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.696 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.696 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:31.696 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141874+0000", 2024-09-21T14:59:31.696 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:31.696 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.696 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.697 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:31.697 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141874+0000", 2024-09-21T14:59:31.697 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:31.697 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.697 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.697 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:31.697 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141875+0000", 2024-09-21T14:59:31.697 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5430000000000001e-06 2024-09-21T14:59:31.697 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.697 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.697 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:31.697 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141878+0000", 2024-09-21T14:59:31.697 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1420000000000002e-06 2024-09-21T14:59:31.697 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.697 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.697 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:31.698 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141881+0000", 2024-09-21T14:59:31.698 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.807e-06 2024-09-21T14:59:31.698 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.698 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.698 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:31.698 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176860+0000", 2024-09-21T14:59:31.698 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034979243 2024-09-21T14:59:31.698 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.698 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.698 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:31.698 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176867+0000", 2024-09-21T14:59:31.698 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0169999999999999e-06 2024-09-21T14:59:31.698 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:31.698 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:31.699 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:31.699 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.699 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.699 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45713 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:31.699 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141883+0000", 2024-09-21T14:59:31.699 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 21.530874817000001, 2024-09-21T14:59:31.699 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035010516999999998, 2024-09-21T14:59:31.699 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:31.699 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:31.699 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:31.699 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:31.700 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:31.700 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45713 2024-09-21T14:59:31.700 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.700 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:31.700 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.700 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:31.700 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141883+0000", 2024-09-21T14:59:31.700 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:31.700 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.700 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.700 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:31.700 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141883+0000", 2024-09-21T14:59:31.700 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:31.700 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.700 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.701 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:31.701 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141883+0000", 2024-09-21T14:59:31.701 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:31.701 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.701 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.701 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:31.701 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141885+0000", 2024-09-21T14:59:31.701 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4649999999999999e-06 2024-09-21T14:59:31.701 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.701 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.701 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:31.701 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141888+0000", 2024-09-21T14:59:31.701 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3890000000000001e-06 2024-09-21T14:59:31.701 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.701 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.701 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:31.702 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141891+0000", 2024-09-21T14:59:31.702 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9239999999999999e-06 2024-09-21T14:59:31.702 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.702 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.702 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:31.702 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176887+0000", 2024-09-21T14:59:31.702 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034996370999999998 2024-09-21T14:59:31.702 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.702 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.702 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:31.702 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176894+0000", 2024-09-21T14:59:31.702 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6069999999999996e-06 2024-09-21T14:59:31.702 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:31.702 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:31.703 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:31.703 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.703 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.703 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45714 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:31.703 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141893+0000", 2024-09-21T14:59:31.703 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 21.530864902000001, 2024-09-21T14:59:31.703 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035048215000000001, 2024-09-21T14:59:31.703 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:31.703 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:31.703 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:31.703 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:31.703 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:31.703 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45714 2024-09-21T14:59:31.704 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.704 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:31.704 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.704 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:31.704 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141893+0000", 2024-09-21T14:59:31.704 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:31.704 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.704 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.704 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:31.704 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141893+0000", 2024-09-21T14:59:31.704 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:31.704 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.704 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.704 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:31.705 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141893+0000", 2024-09-21T14:59:31.705 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:31.705 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.705 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.705 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:31.705 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141895+0000", 2024-09-21T14:59:31.705 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.703e-06 2024-09-21T14:59:31.705 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.705 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.705 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:31.705 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141900+0000", 2024-09-21T14:59:31.705 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.242e-06 2024-09-21T14:59:31.705 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.705 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.705 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:31.705 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141903+0000", 2024-09-21T14:59:31.706 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.117e-06 2024-09-21T14:59:31.706 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.706 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.706 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:31.706 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176930+0000", 2024-09-21T14:59:31.706 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035026750000000002 2024-09-21T14:59:31.706 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.706 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.706 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:31.706 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176942+0000", 2024-09-21T14:59:31.706 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1668000000000001e-05 2024-09-21T14:59:31.706 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:31.706 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:31.706 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:31.707 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.707 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.707 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45715 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:31.707 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141906+0000", 2024-09-21T14:59:31.707 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 21.530852744000001, 2024-09-21T14:59:31.707 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035066057999999997, 2024-09-21T14:59:31.707 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:31.707 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:31.707 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:31.707 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:31.707 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:31.707 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45715 2024-09-21T14:59:31.707 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.707 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:31.708 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.708 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:31.708 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141906+0000", 2024-09-21T14:59:31.708 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:31.708 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.708 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.708 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:31.708 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141906+0000", 2024-09-21T14:59:31.708 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:31.708 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.708 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.708 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:31.708 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141905+0000", 2024-09-21T14:59:31.708 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:31.708 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.708 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.709 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:31.709 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141907+0000", 2024-09-21T14:59:31.709 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5400000000000001e-06 2024-09-21T14:59:31.709 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.709 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.709 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:31.709 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141912+0000", 2024-09-21T14:59:31.709 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.502e-06 2024-09-21T14:59:31.709 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.709 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.709 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:31.709 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141915+0000", 2024-09-21T14:59:31.709 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0170000000000001e-06 2024-09-21T14:59:31.709 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.709 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.710 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:31.710 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176964+0000", 2024-09-21T14:59:31.710 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035048587999999999 2024-09-21T14:59:31.710 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.710 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.710 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:31.710 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176972+0000", 2024-09-21T14:59:31.710 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6580000000000002e-06 2024-09-21T14:59:31.710 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:31.710 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:31.710 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:31.710 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.710 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.710 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45716 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:31.711 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141918+0000", 2024-09-21T14:59:31.711 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 21.530840567999999, 2024-09-21T14:59:31.711 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035086223, 2024-09-21T14:59:31.711 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:31.711 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:31.711 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:31.711 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:31.711 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:31.711 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45716 2024-09-21T14:59:31.711 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.711 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:31.711 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.711 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:31.711 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141918+0000", 2024-09-21T14:59:31.711 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:31.711 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.712 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.712 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:31.712 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141918+0000", 2024-09-21T14:59:31.712 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:31.712 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.712 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.712 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:31.712 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141917+0000", 2024-09-21T14:59:31.712 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:31.712 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.712 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.712 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:31.712 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141919+0000", 2024-09-21T14:59:31.712 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5680000000000001e-06 2024-09-21T14:59:31.712 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.712 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.713 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:31.713 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141922+0000", 2024-09-21T14:59:31.713 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2789999999999999e-06 2024-09-21T14:59:31.713 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.713 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.713 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:31.713 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141925+0000", 2024-09-21T14:59:31.713 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9469999999999999e-06 2024-09-21T14:59:31.713 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.713 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.713 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:31.713 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176997+0000", 2024-09-21T14:59:31.713 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035071366999999999 2024-09-21T14:59:31.713 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.713 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.713 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:31.713 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177004+0000", 2024-09-21T14:59:31.714 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3309999999999998e-06 2024-09-21T14:59:31.714 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:31.714 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:31.714 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:31.714 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.714 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.714 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45717 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:31.714 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141928+0000", 2024-09-21T14:59:31.714 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 21.530830622, 2024-09-21T14:59:31.714 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035102573999999998, 2024-09-21T14:59:31.714 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:31.714 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:31.714 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:31.714 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:31.715 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:31.715 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45717 2024-09-21T14:59:31.715 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.715 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:31.715 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.715 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:31.715 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141928+0000", 2024-09-21T14:59:31.715 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:31.715 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.715 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.715 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:31.715 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141928+0000", 2024-09-21T14:59:31.715 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:31.715 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.715 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.715 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:31.716 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141927+0000", 2024-09-21T14:59:31.716 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T14:59:31.716 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.716 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.716 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:31.716 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141929+0000", 2024-09-21T14:59:31.716 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.533e-06 2024-09-21T14:59:31.716 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.716 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.716 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:31.716 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141934+0000", 2024-09-21T14:59:31.716 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1660000000000002e-06 2024-09-21T14:59:31.716 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.716 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.716 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:31.716 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141937+0000", 2024-09-21T14:59:31.717 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0309999999999999e-06 2024-09-21T14:59:31.717 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.717 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.717 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:31.717 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177023+0000", 2024-09-21T14:59:31.717 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035085595999999997 2024-09-21T14:59:31.717 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.717 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.717 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:31.717 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177030+0000", 2024-09-21T14:59:31.717 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.486e-06 2024-09-21T14:59:31.717 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:31.717 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:31.717 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:31.717 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.717 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.718 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45718 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:31.718 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141940+0000", 2024-09-21T14:59:31.718 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 21.530818763999999, 2024-09-21T14:59:31.718 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035124956999999998, 2024-09-21T14:59:31.718 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:31.718 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:31.718 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:31.718 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:31.718 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:31.718 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45718 2024-09-21T14:59:31.718 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.718 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:31.718 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.719 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:31.719 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141940+0000", 2024-09-21T14:59:31.719 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:31.719 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.719 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.719 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:31.719 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141940+0000", 2024-09-21T14:59:31.719 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:31.719 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.719 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.719 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:31.719 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141939+0000", 2024-09-21T14:59:31.719 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:31.719 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.719 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.720 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:31.720 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141941+0000", 2024-09-21T14:59:31.720 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.502e-06 2024-09-21T14:59:31.720 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.720 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.720 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:31.720 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141946+0000", 2024-09-21T14:59:31.720 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3449999999999996e-06 2024-09-21T14:59:31.720 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.720 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.720 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:31.720 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141949+0000", 2024-09-21T14:59:31.720 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9749999999999999e-06 2024-09-21T14:59:31.720 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.720 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.721 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:31.721 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177057+0000", 2024-09-21T14:59:31.721 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035108385999999998 2024-09-21T14:59:31.721 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.721 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.721 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:31.721 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177064+0000", 2024-09-21T14:59:31.721 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0099999999999998e-06 2024-09-21T14:59:31.721 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:31.721 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:31.721 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:31.721 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.721 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.721 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45719 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:31.722 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141952+0000", 2024-09-21T14:59:31.722 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 21.530806491, 2024-09-21T14:59:31.722 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035139108000000002, 2024-09-21T14:59:31.722 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:31.722 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:31.722 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:31.722 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:31.722 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:31.722 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45719 2024-09-21T14:59:31.722 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.722 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:31.722 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.722 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:31.722 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141952+0000", 2024-09-21T14:59:31.723 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:31.723 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.723 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.723 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:31.723 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141952+0000", 2024-09-21T14:59:31.723 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:31.723 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.723 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.723 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:31.723 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141952+0000", 2024-09-21T14:59:31.723 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:31.723 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.723 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.723 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:31.724 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141953+0000", 2024-09-21T14:59:31.724 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.486e-06 2024-09-21T14:59:31.724 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.724 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.724 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:31.724 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141959+0000", 2024-09-21T14:59:31.724 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4940000000000003e-06 2024-09-21T14:59:31.724 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.724 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.724 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:31.724 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141963+0000", 2024-09-21T14:59:31.724 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3540000000000002e-06 2024-09-21T14:59:31.724 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.724 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.724 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:31.725 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177084+0000", 2024-09-21T14:59:31.725 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035120983000000001 2024-09-21T14:59:31.725 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.725 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.725 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:31.725 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177091+0000", 2024-09-21T14:59:31.725 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0350000000000004e-06 2024-09-21T14:59:31.725 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:31.725 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:31.725 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:31.725 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.725 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.725 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45720 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:31.725 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141965+0000", 2024-09-21T14:59:31.725 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 21.530793024000001, 2024-09-21T14:59:31.725 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035155268000000003, 2024-09-21T14:59:31.726 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:31.726 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:31.726 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:31.726 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:31.726 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:31.726 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45720 2024-09-21T14:59:31.726 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.726 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:31.726 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.726 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:31.726 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141965+0000", 2024-09-21T14:59:31.726 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:31.726 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.726 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.726 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:31.726 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141965+0000", 2024-09-21T14:59:31.727 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:31.727 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.727 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.727 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:31.727 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141965+0000", 2024-09-21T14:59:31.727 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:31.727 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.727 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.727 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:31.727 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141966+0000", 2024-09-21T14:59:31.727 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.46e-06 2024-09-21T14:59:31.727 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.727 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.727 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:31.727 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141972+0000", 2024-09-21T14:59:31.727 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4210000000000003e-06 2024-09-21T14:59:31.728 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.728 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.728 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:31.728 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141975+0000", 2024-09-21T14:59:31.728 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.948e-06 2024-09-21T14:59:31.728 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.728 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.728 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:31.728 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177113+0000", 2024-09-21T14:59:31.728 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035137855000000003 2024-09-21T14:59:31.728 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.728 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.728 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:31.728 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177121+0000", 2024-09-21T14:59:31.728 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8269999999999998e-06 2024-09-21T14:59:31.728 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:31.729 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:31.729 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:31.729 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.729 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.729 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45721 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:31.729 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141977+0000", 2024-09-21T14:59:31.729 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 21.530781001000001, 2024-09-21T14:59:31.729 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035169984000000001, 2024-09-21T14:59:31.729 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:31.729 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:31.729 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:31.729 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:31.729 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:31.729 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45721 2024-09-21T14:59:31.729 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.729 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:31.730 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.730 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:31.730 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141977+0000", 2024-09-21T14:59:31.730 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:31.730 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.730 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.730 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:31.730 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141977+0000", 2024-09-21T14:59:31.730 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:31.730 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.730 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.730 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:31.730 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141977+0000", 2024-09-21T14:59:31.730 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:31.731 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.731 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.731 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:31.731 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141979+0000", 2024-09-21T14:59:31.731 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.511e-06 2024-09-21T14:59:31.731 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.731 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.731 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:31.731 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141982+0000", 2024-09-21T14:59:31.731 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4180000000000001e-06 2024-09-21T14:59:31.731 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.731 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.731 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:31.731 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141985+0000", 2024-09-21T14:59:31.731 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9450000000000002e-06 2024-09-21T14:59:31.731 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.732 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.732 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:31.732 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177139+0000", 2024-09-21T14:59:31.732 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035154409999999997 2024-09-21T14:59:31.732 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.732 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.732 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:31.732 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177147+0000", 2024-09-21T14:59:31.732 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9570000000000002e-06 2024-09-21T14:59:31.732 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:31.732 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:31.732 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:31.732 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.732 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.732 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45722 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:31.732 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141987+0000", 2024-09-21T14:59:31.733 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 21.53077103, 2024-09-21T14:59:31.733 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035188492000000002, 2024-09-21T14:59:31.733 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:31.733 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:31.733 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:31.733 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:31.733 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:31.733 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45722 2024-09-21T14:59:31.733 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.733 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:31.733 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.733 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:31.733 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141987+0000", 2024-09-21T14:59:31.733 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:31.733 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.734 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.734 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:31.734 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141987+0000", 2024-09-21T14:59:31.734 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:31.734 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.734 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.734 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:31.734 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141987+0000", 2024-09-21T14:59:31.734 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:31.734 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.734 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.734 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:31.734 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141988+0000", 2024-09-21T14:59:31.734 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.463e-06 2024-09-21T14:59:31.734 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.735 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.735 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:31.735 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141994+0000", 2024-09-21T14:59:31.735 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.361e-06 2024-09-21T14:59:31.735 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.735 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.735 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:31.735 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141997+0000", 2024-09-21T14:59:31.735 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0359999999999998e-06 2024-09-21T14:59:31.735 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.735 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.735 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:31.735 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177169+0000", 2024-09-21T14:59:31.735 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035171651999999998 2024-09-21T14:59:31.735 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.735 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.736 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:31.736 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177176+0000", 2024-09-21T14:59:31.736 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2189999999999998e-06 2024-09-21T14:59:31.736 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:31.736 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:31.736 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:31.736 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.736 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.736 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45723 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:31.736 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141999+0000", 2024-09-21T14:59:31.736 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 21.530758862999999, 2024-09-21T14:59:31.736 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035197701999999997, 2024-09-21T14:59:31.736 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:31.736 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:31.736 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:31.736 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:31.737 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:31.737 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45723 2024-09-21T14:59:31.737 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.737 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:31.737 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.737 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:31.737 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141999+0000", 2024-09-21T14:59:31.737 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:31.737 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.737 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.737 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:31.737 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141999+0000", 2024-09-21T14:59:31.737 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:31.737 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.737 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.737 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:31.738 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141999+0000", 2024-09-21T14:59:31.738 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:31.738 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.738 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.738 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:31.738 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142003+0000", 2024-09-21T14:59:31.738 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5049999999999999e-06 2024-09-21T14:59:31.738 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.738 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.738 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:31.738 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142008+0000", 2024-09-21T14:59:31.738 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6579999999999999e-06 2024-09-21T14:59:31.738 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.738 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.738 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:31.739 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142011+0000", 2024-09-21T14:59:31.739 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0529999999999998e-06 2024-09-21T14:59:31.739 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.739 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.739 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:31.739 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177194+0000", 2024-09-21T14:59:31.739 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035182221999999999 2024-09-21T14:59:31.739 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.739 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.739 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:31.739 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177197+0000", 2024-09-21T14:59:31.739 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5030000000000002e-06 2024-09-21T14:59:31.739 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:31.739 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:31.739 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:31.739 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.739 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.740 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45724 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:31.740 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.142015+0000", 2024-09-21T14:59:31.740 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 21.530742896, 2024-09-21T14:59:31.740 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035195345000000003, 2024-09-21T14:59:31.740 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:31.740 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:31.740 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:31.740 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:31.740 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:31.740 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45724 2024-09-21T14:59:31.740 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.740 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:31.740 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.740 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:31.740 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142015+0000", 2024-09-21T14:59:31.741 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:31.741 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.741 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.741 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:31.741 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142015+0000", 2024-09-21T14:59:31.741 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:31.741 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.741 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.741 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:31.741 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142015+0000", 2024-09-21T14:59:31.741 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:31.741 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.741 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.741 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:31.741 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142017+0000", 2024-09-21T14:59:31.741 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5540000000000001e-06 2024-09-21T14:59:31.742 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.742 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.742 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:31.742 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142022+0000", 2024-09-21T14:59:31.742 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4229999999999996e-06 2024-09-21T14:59:31.742 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.742 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.742 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:31.742 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142025+0000", 2024-09-21T14:59:31.742 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.056e-06 2024-09-21T14:59:31.742 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.742 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.742 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:31.742 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177207+0000", 2024-09-21T14:59:31.742 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035182089 2024-09-21T14:59:31.743 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.743 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.743 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:31.743 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177211+0000", 2024-09-21T14:59:31.743 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4649999999999999e-06 2024-09-21T14:59:31.743 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:31.743 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:31.743 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:31.743 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.743 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.743 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45725 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:31.743 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.142028+0000", 2024-09-21T14:59:31.743 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 21.530730617, 2024-09-21T14:59:31.743 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035195731000000001, 2024-09-21T14:59:31.743 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:31.743 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:31.744 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:31.744 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:31.744 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:31.744 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45725 2024-09-21T14:59:31.744 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.744 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:31.744 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.744 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:31.744 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142028+0000", 2024-09-21T14:59:31.744 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:31.744 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.744 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.744 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:31.744 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142028+0000", 2024-09-21T14:59:31.744 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:31.744 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.745 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.745 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:31.745 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142027+0000", 2024-09-21T14:59:31.745 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:31.745 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.745 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.745 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:31.745 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142029+0000", 2024-09-21T14:59:31.745 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5269999999999999e-06 2024-09-21T14:59:31.745 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.745 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.745 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:31.745 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142032+0000", 2024-09-21T14:59:31.745 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3249999999999999e-06 2024-09-21T14:59:31.745 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.745 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.746 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:31.746 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142035+0000", 2024-09-21T14:59:31.746 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.897e-06 2024-09-21T14:59:31.746 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.746 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.746 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:31.746 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177220+0000", 2024-09-21T14:59:31.746 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035184871 2024-09-21T14:59:31.746 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.746 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.746 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:31.746 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177223+0000", 2024-09-21T14:59:31.746 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3680000000000002e-06 2024-09-21T14:59:31.746 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:31.746 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:31.746 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:31.747 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.747 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.747 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45726 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:31.747 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.142037+0000", 2024-09-21T14:59:31.747 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 21.530720901999999, 2024-09-21T14:59:31.747 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035199273000000003, 2024-09-21T14:59:31.747 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:31.747 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:31.747 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:31.747 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:31.747 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:31.747 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45726 2024-09-21T14:59:31.747 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.747 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:31.747 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.747 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:31.748 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142037+0000", 2024-09-21T14:59:31.748 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:31.748 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.748 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.748 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:31.748 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142037+0000", 2024-09-21T14:59:31.748 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:31.748 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.748 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.748 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:31.748 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142037+0000", 2024-09-21T14:59:31.748 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:31.748 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.748 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.748 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:31.748 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142038+0000", 2024-09-21T14:59:31.748 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.344e-06 2024-09-21T14:59:31.749 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.749 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.749 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:31.749 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142044+0000", 2024-09-21T14:59:31.749 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1930000000000001e-06 2024-09-21T14:59:31.749 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.749 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.749 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:31.749 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142047+0000", 2024-09-21T14:59:31.749 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0280000000000001e-06 2024-09-21T14:59:31.749 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.749 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.749 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:31.749 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177233+0000", 2024-09-21T14:59:31.749 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035186394000000003 2024-09-21T14:59:31.749 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:31.750 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:31.750 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:31.750 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177237+0000", 2024-09-21T14:59:31.750 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5530000000000001e-06 2024-09-21T14:59:31.750 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:31.750 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:31.750 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:31.750 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:31.750 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:31.750 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:31.751 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T14:59:31.760 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:59:31.755+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:59:31.860 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:59:31.859+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:59:31.865 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:31.865 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:59:31.865 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T14:59:31.865 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:31.875 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T14:59:31.961 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:59:31.955+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:59:31.981 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:31.981 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:59:31.981 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T14:59:31.981 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T14:59:31.981 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:31.991 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T14:59:32.062 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:59:32.059+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:59:32.101 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:32.101 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T14:59:32.101 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T14:59:32.101 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T14:59:32.102 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.102 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4281.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-21T14:59:32.102 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:09.324416+0000", 2024-09-21T14:59:32.102 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 82.776151244999994, 2024-09-21T14:59:32.102 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.117837947, 2024-09-21T14:59:32.102 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:32.102 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:32.102 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:32.102 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4281", 2024-09-21T14:59:32.102 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/1518520303", 2024-09-21T14:59:32.102 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1 2024-09-21T14:59:32.102 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.102 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:32.103 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.103 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:32.103 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324416+0000", 2024-09-21T14:59:32.103 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:32.103 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.103 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.103 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:32.103 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324416+0000", 2024-09-21T14:59:32.103 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:32.103 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.103 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.103 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:32.103 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324414+0000", 2024-09-21T14:59:32.103 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T14:59:32.104 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.104 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.104 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:32.104 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324429+0000", 2024-09-21T14:59:32.104 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5637e-05 2024-09-21T14:59:32.104 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.104 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.104 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:32.104 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324448+0000", 2024-09-21T14:59:32.104 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9042000000000002e-05 2024-09-21T14:59:32.104 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.104 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.104 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:32.104 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324465+0000", 2024-09-21T14:59:32.104 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6767999999999999e-05 2024-09-21T14:59:32.105 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.105 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.105 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:32.105 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324571+0000", 2024-09-21T14:59:32.105 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000105714 2024-09-21T14:59:32.105 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.105 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.105 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op must wait for map", 2024-09-21T14:59:32.105 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324587+0000", 2024-09-21T14:59:32.105 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5753999999999999e-05 2024-09-21T14:59:32.105 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.105 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.105 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:32.105 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.325864+0000", 2024-09-21T14:59:32.106 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0012773350000000001 2024-09-21T14:59:32.106 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.106 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.106 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T14:59:32.106 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.325878+0000", 2024-09-21T14:59:32.106 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4498000000000001e-05 2024-09-21T14:59:32.106 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.106 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.106 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:32.106 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.439461+0000", 2024-09-21T14:59:32.106 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.11358212500000001 2024-09-21T14:59:32.106 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.106 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.106 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:32.106 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.439848+0000", 2024-09-21T14:59:32.107 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00038747800000000002 2024-09-21T14:59:32.107 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.107 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.107 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:59:32.107 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.440175+0000", 2024-09-21T14:59:32.107 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00032721900000000001 2024-09-21T14:59:32.107 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.107 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.107 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:32.107 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.441770+0000", 2024-09-21T14:59:32.107 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0015949409999999999 2024-09-21T14:59:32.107 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.107 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.107 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:32.107 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.442086+0000", 2024-09-21T14:59:32.108 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00031586299999999999 2024-09-21T14:59:32.108 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.108 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.108 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:32.108 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.442195+0000", 2024-09-21T14:59:32.108 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000108791 2024-09-21T14:59:32.108 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.108 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.108 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:32.108 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.442254+0000", 2024-09-21T14:59:32.108 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9038e-05 2024-09-21T14:59:32.108 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:32.108 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:32.108 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:32.108 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.108 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.109 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4281.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-21T14:59:32.109 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:09.447828+0000", 2024-09-21T14:59:32.109 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 82.652739244000003, 2024-09-21T14:59:32.109 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.32904393700000001, 2024-09-21T14:59:32.109 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:32.109 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:32.109 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:32.109 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4281", 2024-09-21T14:59:32.109 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/1518520303", 2024-09-21T14:59:32.109 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 2 2024-09-21T14:59:32.109 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.109 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:32.109 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.109 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:32.109 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447828+0000", 2024-09-21T14:59:32.110 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:32.110 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.110 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.110 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:32.110 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447828+0000", 2024-09-21T14:59:32.110 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:32.110 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.110 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.110 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:32.110 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447825+0000", 2024-09-21T14:59:32.110 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T14:59:32.110 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.110 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.110 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:32.110 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447841+0000", 2024-09-21T14:59:32.111 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5359999999999999e-05 2024-09-21T14:59:32.111 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.111 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.111 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:32.111 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447875+0000", 2024-09-21T14:59:32.111 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4388999999999999e-05 2024-09-21T14:59:32.111 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.111 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.111 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:32.111 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447903+0000", 2024-09-21T14:59:32.111 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7373e-05 2024-09-21T14:59:32.111 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.111 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.111 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:32.111 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447996+0000", 2024-09-21T14:59:32.111 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2954999999999993e-05 2024-09-21T14:59:32.112 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.112 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.112 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T14:59:32.112 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.448011+0000", 2024-09-21T14:59:32.112 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5038999999999999e-05 2024-09-21T14:59:32.112 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.112 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.112 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:32.112 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.776501+0000", 2024-09-21T14:59:32.112 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.32849012500000002 2024-09-21T14:59:32.112 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.112 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.112 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:32.112 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.776872+0000", 2024-09-21T14:59:32.112 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00037119600000000002 2024-09-21T14:59:32.112 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:32.112 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:32.113 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:32.113 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.113 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.113 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:752 3.14 3:28fc02e5:::rbd_data.113693316f36.0000000000000010:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-21T14:59:32.113 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:26.625579+0000", 2024-09-21T14:59:32.113 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 65.474988389000004, 2024-09-21T14:59:32.113 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042965426000000001, 2024-09-21T14:59:32.113 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:32.113 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:32.113 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:32.113 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:32.113 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:32.113 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 752 2024-09-21T14:59:32.113 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.113 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:32.114 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.114 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:32.114 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625579+0000", 2024-09-21T14:59:32.114 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:32.114 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.114 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.114 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:32.114 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625579+0000", 2024-09-21T14:59:32.114 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:32.114 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.114 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.114 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:32.114 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625577+0000", 2024-09-21T14:59:32.114 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T14:59:32.114 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.115 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.115 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:32.115 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625588+0000", 2024-09-21T14:59:32.115 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1555e-05 2024-09-21T14:59:32.115 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.115 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.115 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:32.115 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625610+0000", 2024-09-21T14:59:32.115 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2107999999999999e-05 2024-09-21T14:59:32.115 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.115 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.115 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:32.115 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625621+0000", 2024-09-21T14:59:32.115 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0762e-05 2024-09-21T14:59:32.115 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.115 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.116 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:32.116 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625692+0000", 2024-09-21T14:59:32.116 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.1203000000000003e-05 2024-09-21T14:59:32.116 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.116 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.116 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:32.116 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625872+0000", 2024-09-21T14:59:32.116 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017962800000000001 2024-09-21T14:59:32.116 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.116 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.116 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:59:32.116 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.633188+0000", 2024-09-21T14:59:32.116 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.007316014 2024-09-21T14:59:32.116 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.116 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.116 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:32.117 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.668357+0000", 2024-09-21T14:59:32.117 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035168933999999999 2024-09-21T14:59:32.117 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.117 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.117 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:32.117 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.668418+0000", 2024-09-21T14:59:32.117 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1282e-05 2024-09-21T14:59:32.117 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.117 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.117 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:32.117 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.668486+0000", 2024-09-21T14:59:32.117 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8468999999999998e-05 2024-09-21T14:59:32.117 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.117 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.117 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:32.117 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.668544+0000", 2024-09-21T14:59:32.118 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7785000000000003e-05 2024-09-21T14:59:32.118 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:32.118 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:32.118 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:32.118 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.118 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.118 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22737 3.a 3:574c7a60:::rbd_data.11366acc894.0000000000000002:19 [read 1032192~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:32.118 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141028+0000", 2024-09-21T14:59:32.118 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 41.959539423999999, 2024-09-21T14:59:32.118 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035206281999999998, 2024-09-21T14:59:32.118 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:32.118 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:32.118 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:32.118 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:32.118 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:32.118 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22737 2024-09-21T14:59:32.119 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.119 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:32.119 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.119 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:32.119 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141028+0000", 2024-09-21T14:59:32.119 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:32.119 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.119 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.119 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:32.119 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141028+0000", 2024-09-21T14:59:32.119 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:32.119 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.119 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.120 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:32.120 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141027+0000", 2024-09-21T14:59:32.120 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:32.120 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.120 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.120 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:32.120 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141030+0000", 2024-09-21T14:59:32.120 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6189999999999998e-06 2024-09-21T14:59:32.120 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.120 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.120 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:32.120 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141036+0000", 2024-09-21T14:59:32.120 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5110000000000003e-06 2024-09-21T14:59:32.120 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.120 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.120 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:32.121 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141040+0000", 2024-09-21T14:59:32.121 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.34e-06 2024-09-21T14:59:32.121 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.121 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.121 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:32.121 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.175948+0000", 2024-09-21T14:59:32.121 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034908540000000002 2024-09-21T14:59:32.121 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.121 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.121 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:32.121 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176025+0000", 2024-09-21T14:59:32.121 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6412e-05 2024-09-21T14:59:32.121 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.121 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.121 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:32.121 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176234+0000", 2024-09-21T14:59:32.122 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000209247 2024-09-21T14:59:32.122 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:32.122 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:32.122 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:32.122 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.122 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.122 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22738 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 49152~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:32.122 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141049+0000", 2024-09-21T14:59:32.122 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 41.959518246000002, 2024-09-21T14:59:32.122 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035428754999999999, 2024-09-21T14:59:32.122 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:32.122 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:32.122 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:32.122 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:32.122 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:32.122 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22738 2024-09-21T14:59:32.123 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.123 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:32.123 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.123 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:32.123 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141049+0000", 2024-09-21T14:59:32.123 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:32.123 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.123 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.123 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:32.123 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141049+0000", 2024-09-21T14:59:32.123 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:32.123 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.123 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.123 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:32.123 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141049+0000", 2024-09-21T14:59:32.123 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:32.124 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.124 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.124 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:32.124 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141051+0000", 2024-09-21T14:59:32.124 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4980000000000001e-06 2024-09-21T14:59:32.124 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.124 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.124 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:32.124 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141057+0000", 2024-09-21T14:59:32.124 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8030000000000002e-06 2024-09-21T14:59:32.124 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.124 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.124 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:32.124 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141066+0000", 2024-09-21T14:59:32.124 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1160000000000005e-06 2024-09-21T14:59:32.124 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.125 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.125 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:32.125 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176264+0000", 2024-09-21T14:59:32.125 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035197580999999999 2024-09-21T14:59:32.125 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.125 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.125 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:32.125 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176327+0000", 2024-09-21T14:59:32.125 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3237000000000005e-05 2024-09-21T14:59:32.125 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.125 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.125 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:32.125 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176478+0000", 2024-09-21T14:59:32.125 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000150851 2024-09-21T14:59:32.125 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:32.125 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:32.126 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:32.126 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.126 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.126 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22739 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 114688~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:32.126 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141070+0000", 2024-09-21T14:59:32.126 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 41.959497003999999, 2024-09-21T14:59:32.126 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035679094000000001, 2024-09-21T14:59:32.126 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:32.126 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:32.126 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:32.126 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:32.126 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:32.126 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22739 2024-09-21T14:59:32.126 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.127 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:32.127 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.127 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:32.127 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141070+0000", 2024-09-21T14:59:32.127 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:32.127 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.127 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.127 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:32.127 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141070+0000", 2024-09-21T14:59:32.127 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:32.127 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.127 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.127 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:32.127 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141070+0000", 2024-09-21T14:59:32.127 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:32.127 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.128 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.128 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:32.128 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141072+0000", 2024-09-21T14:59:32.128 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.379e-06 2024-09-21T14:59:32.128 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.128 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.128 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:32.128 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141078+0000", 2024-09-21T14:59:32.128 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6710000000000004e-06 2024-09-21T14:59:32.128 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.128 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.128 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:32.128 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141083+0000", 2024-09-21T14:59:32.128 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7249999999999997e-06 2024-09-21T14:59:32.128 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.128 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.129 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:32.129 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176506+0000", 2024-09-21T14:59:32.129 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035423076999999997 2024-09-21T14:59:32.129 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.129 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.129 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:32.129 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176582+0000", 2024-09-21T14:59:32.129 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6305999999999998e-05 2024-09-21T14:59:32.129 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.129 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.129 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:32.129 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176749+0000", 2024-09-21T14:59:32.129 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000167262 2024-09-21T14:59:32.129 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:32.129 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:32.129 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:32.130 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.130 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.130 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22740 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 180224~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:32.130 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141090+0000", 2024-09-21T14:59:32.130 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 41.9594776, 2024-09-21T14:59:32.130 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036129535999999997, 2024-09-21T14:59:32.130 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:32.130 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:32.130 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:32.130 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:32.130 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:32.130 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22740 2024-09-21T14:59:32.130 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.130 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:32.130 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.130 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:32.131 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141090+0000", 2024-09-21T14:59:32.131 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:32.131 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.131 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.131 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:32.131 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141090+0000", 2024-09-21T14:59:32.131 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:32.131 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.131 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.131 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:32.131 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141089+0000", 2024-09-21T14:59:32.131 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:32.131 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.131 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.131 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:32.131 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141092+0000", 2024-09-21T14:59:32.132 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7709999999999998e-06 2024-09-21T14:59:32.132 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.132 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.132 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:32.132 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141098+0000", 2024-09-21T14:59:32.132 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.015e-06 2024-09-21T14:59:32.132 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.132 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.132 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:32.132 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141103+0000", 2024-09-21T14:59:32.132 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8629999999999999e-06 2024-09-21T14:59:32.132 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.132 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.132 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:32.132 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176778+0000", 2024-09-21T14:59:32.132 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035674984 2024-09-21T14:59:32.133 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.133 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.133 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:32.133 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176851+0000", 2024-09-21T14:59:32.133 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2861000000000006e-05 2024-09-21T14:59:32.133 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.133 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.133 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:32.133 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177219+0000", 2024-09-21T14:59:32.133 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00036845999999999998 2024-09-21T14:59:32.133 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:32.133 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:32.133 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:32.133 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.133 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.133 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22741 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 245760~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:32.133 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141107+0000", 2024-09-21T14:59:32.134 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 41.959460028999999, 2024-09-21T14:59:32.134 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036407729, 2024-09-21T14:59:32.134 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:32.134 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:32.134 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:32.134 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:32.134 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:32.134 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22741 2024-09-21T14:59:32.134 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.134 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:32.134 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.134 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:32.134 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141107+0000", 2024-09-21T14:59:32.134 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:32.134 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.135 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.135 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:32.135 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141107+0000", 2024-09-21T14:59:32.135 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:32.135 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.135 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.135 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:32.135 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141107+0000", 2024-09-21T14:59:32.135 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:32.135 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.135 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.135 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:32.135 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141109+0000", 2024-09-21T14:59:32.135 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6529999999999998e-06 2024-09-21T14:59:32.135 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.135 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.136 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:32.136 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141115+0000", 2024-09-21T14:59:32.136 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9649999999999996e-06 2024-09-21T14:59:32.136 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.136 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.136 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:32.136 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141121+0000", 2024-09-21T14:59:32.136 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1209999999999998e-06 2024-09-21T14:59:32.136 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.136 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.136 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:32.136 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177242+0000", 2024-09-21T14:59:32.136 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036121706000000003 2024-09-21T14:59:32.136 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.136 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.137 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:32.137 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177301+0000", 2024-09-21T14:59:32.137 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9008000000000002e-05 2024-09-21T14:59:32.137 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.137 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.137 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:32.137 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177515+0000", 2024-09-21T14:59:32.137 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00021362 2024-09-21T14:59:32.137 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:32.137 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:32.137 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:32.137 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.137 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.137 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22742 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 311296~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:32.137 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141126+0000", 2024-09-21T14:59:32.137 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 41.959441675000001, 2024-09-21T14:59:32.138 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036653537, 2024-09-21T14:59:32.138 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:32.138 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:32.138 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:32.138 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:32.138 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:32.138 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22742 2024-09-21T14:59:32.138 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.138 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:32.138 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.138 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:32.138 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141126+0000", 2024-09-21T14:59:32.138 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:32.138 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.138 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.138 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:32.139 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141126+0000", 2024-09-21T14:59:32.139 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:32.139 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.139 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.139 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:32.139 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141125+0000", 2024-09-21T14:59:32.139 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:32.139 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.139 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.139 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:32.139 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141128+0000", 2024-09-21T14:59:32.139 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.886e-06 2024-09-21T14:59:32.139 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.139 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.139 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:32.140 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141137+0000", 2024-09-21T14:59:32.140 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4399999999999994e-06 2024-09-21T14:59:32.140 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.140 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.140 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:32.140 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141143+0000", 2024-09-21T14:59:32.140 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1410000000000004e-06 2024-09-21T14:59:32.140 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.140 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.140 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:32.140 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177543+0000", 2024-09-21T14:59:32.140 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036400544999999999 2024-09-21T14:59:32.140 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.140 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.140 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:32.141 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177605+0000", 2024-09-21T14:59:32.141 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2206999999999996e-05 2024-09-21T14:59:32.141 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.141 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.141 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:32.141 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177779+0000", 2024-09-21T14:59:32.141 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017368099999999999 2024-09-21T14:59:32.141 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:32.141 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:32.141 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:32.141 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.141 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.141 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22743 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 376832~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:32.141 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141147+0000", 2024-09-21T14:59:32.141 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 41.959420191, 2024-09-21T14:59:32.142 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036915158000000003, 2024-09-21T14:59:32.142 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:32.142 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:32.142 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:32.142 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:32.142 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:32.142 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22743 2024-09-21T14:59:32.142 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.142 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:32.142 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.142 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:32.142 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141147+0000", 2024-09-21T14:59:32.142 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:32.142 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.142 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.143 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:32.143 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141147+0000", 2024-09-21T14:59:32.143 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:32.143 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.143 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.143 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:32.143 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141147+0000", 2024-09-21T14:59:32.143 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:32.143 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.143 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.143 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:32.143 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141149+0000", 2024-09-21T14:59:32.143 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.79e-06 2024-09-21T14:59:32.144 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.144 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.144 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:32.144 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141155+0000", 2024-09-21T14:59:32.144 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6699999999999999e-06 2024-09-21T14:59:32.144 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.144 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.144 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:32.144 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141160+0000", 2024-09-21T14:59:32.144 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5759999999999999e-06 2024-09-21T14:59:32.144 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.144 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.144 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:32.144 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177817+0000", 2024-09-21T14:59:32.145 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036657404999999997 2024-09-21T14:59:32.145 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.145 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.145 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:32.145 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177892+0000", 2024-09-21T14:59:32.145 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4602000000000002e-05 2024-09-21T14:59:32.145 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.145 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.145 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:32.145 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178062+0000", 2024-09-21T14:59:32.145 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000170464 2024-09-21T14:59:32.145 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:32.145 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:32.145 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:32.145 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.145 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.146 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22744 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 442368~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:32.146 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141164+0000", 2024-09-21T14:59:32.146 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 41.959403057999999, 2024-09-21T14:59:32.146 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037120806999999999, 2024-09-21T14:59:32.146 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:32.146 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:32.146 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:32.146 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:32.146 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:32.146 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22744 2024-09-21T14:59:32.146 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.146 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:32.146 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.146 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:32.146 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141164+0000", 2024-09-21T14:59:32.147 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:32.147 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.147 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.147 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:32.147 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141164+0000", 2024-09-21T14:59:32.147 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:32.147 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.147 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.147 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:32.147 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141164+0000", 2024-09-21T14:59:32.147 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:32.147 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.147 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.147 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:32.147 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141167+0000", 2024-09-21T14:59:32.147 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7159999999999999e-06 2024-09-21T14:59:32.148 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.148 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.148 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:32.148 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141173+0000", 2024-09-21T14:59:32.148 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.071e-06 2024-09-21T14:59:32.148 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.148 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.148 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:32.148 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141177+0000", 2024-09-21T14:59:32.148 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3610000000000003e-06 2024-09-21T14:59:32.148 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.148 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.148 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:32.148 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178083+0000", 2024-09-21T14:59:32.148 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036906405000000003 2024-09-21T14:59:32.148 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.149 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.149 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:32.149 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178143+0000", 2024-09-21T14:59:32.149 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0056999999999998e-05 2024-09-21T14:59:32.149 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.149 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.149 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:32.149 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178285+0000", 2024-09-21T14:59:32.149 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000141555 2024-09-21T14:59:32.149 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:32.149 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:32.149 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:32.149 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.149 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.149 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22745 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 507904~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:32.150 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141182+0000", 2024-09-21T14:59:32.150 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 41.959384737999997, 2024-09-21T14:59:32.150 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037332331000000003, 2024-09-21T14:59:32.150 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:32.150 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:32.150 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:32.150 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:32.150 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:32.150 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22745 2024-09-21T14:59:32.150 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.150 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:32.150 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.150 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:32.150 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141182+0000", 2024-09-21T14:59:32.151 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:32.151 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.151 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.151 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:32.151 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141182+0000", 2024-09-21T14:59:32.151 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:32.151 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.151 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.151 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:32.151 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141182+0000", 2024-09-21T14:59:32.151 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:32.151 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.151 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.151 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:32.151 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141185+0000", 2024-09-21T14:59:32.152 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.464e-06 2024-09-21T14:59:32.152 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.152 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.152 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:32.152 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141200+0000", 2024-09-21T14:59:32.152 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5886999999999999e-05 2024-09-21T14:59:32.152 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.152 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.152 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:32.152 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141571+0000", 2024-09-21T14:59:32.152 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000370909 2024-09-21T14:59:32.152 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.152 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.152 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:32.152 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178318+0000", 2024-09-21T14:59:32.152 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036746244999999997 2024-09-21T14:59:32.153 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.153 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.153 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:32.153 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178367+0000", 2024-09-21T14:59:32.153 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9317000000000002e-05 2024-09-21T14:59:32.153 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.153 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.153 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:32.153 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178515+0000", 2024-09-21T14:59:32.153 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014784600000000001 2024-09-21T14:59:32.153 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:32.153 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:32.153 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:32.153 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.153 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.154 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22746 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 573440~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:32.154 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141599+0000", 2024-09-21T14:59:32.154 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 41.958968106, 2024-09-21T14:59:32.154 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037402249999999998, 2024-09-21T14:59:32.154 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:32.154 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:32.154 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:32.154 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:32.154 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:32.154 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22746 2024-09-21T14:59:32.154 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.154 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:32.154 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.154 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:32.155 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141599+0000", 2024-09-21T14:59:32.155 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:32.155 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.155 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.155 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:32.155 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141599+0000", 2024-09-21T14:59:32.155 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:32.155 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.155 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.155 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:32.155 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141598+0000", 2024-09-21T14:59:32.155 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:59:32.155 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.155 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.155 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:32.156 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141637+0000", 2024-09-21T14:59:32.156 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.9119999999999998e-05 2024-09-21T14:59:32.156 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.156 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.156 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:32.156 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141647+0000", 2024-09-21T14:59:32.156 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5570000000000001e-06 2024-09-21T14:59:32.156 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.156 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.156 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:32.156 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141654+0000", 2024-09-21T14:59:32.156 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2049999999999996e-06 2024-09-21T14:59:32.156 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.156 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.156 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:32.156 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178553+0000", 2024-09-21T14:59:32.156 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036898528 2024-09-21T14:59:32.157 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.157 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.157 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:32.157 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178625+0000", 2024-09-21T14:59:32.157 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2669999999999994e-05 2024-09-21T14:59:32.157 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.157 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.157 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:32.157 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179001+0000", 2024-09-21T14:59:32.157 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000375977 2024-09-21T14:59:32.157 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:32.157 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:32.157 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:32.157 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.157 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.157 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22747 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 638976~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:32.158 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141660+0000", 2024-09-21T14:59:32.158 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 41.958906964000001, 2024-09-21T14:59:32.158 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037617352999999999, 2024-09-21T14:59:32.158 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:32.158 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:32.158 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:32.158 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:32.158 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:32.158 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22747 2024-09-21T14:59:32.158 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.158 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:32.158 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.158 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:32.158 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141660+0000", 2024-09-21T14:59:32.158 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:32.159 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.159 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.159 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:32.159 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141660+0000", 2024-09-21T14:59:32.159 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:32.159 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.159 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.159 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:32.159 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141660+0000", 2024-09-21T14:59:32.159 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:32.159 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.159 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.159 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:32.159 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141663+0000", 2024-09-21T14:59:32.159 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.569e-06 2024-09-21T14:59:32.160 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.160 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.160 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:32.160 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141670+0000", 2024-09-21T14:59:32.160 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5719999999999999e-06 2024-09-21T14:59:32.160 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.160 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.160 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:32.160 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141675+0000", 2024-09-21T14:59:32.160 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2889999999999997e-06 2024-09-21T14:59:32.160 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.160 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.160 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:32.160 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179028+0000", 2024-09-21T14:59:32.160 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037353381999999997 2024-09-21T14:59:32.161 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.161 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.161 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:32.161 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179099+0000", 2024-09-21T14:59:32.161 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0259000000000006e-05 2024-09-21T14:59:32.161 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.161 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.161 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:32.161 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179278+0000", 2024-09-21T14:59:32.161 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000178908 2024-09-21T14:59:32.161 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:32.161 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:32.161 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:32.161 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.161 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.161 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22748 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 704512~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:32.162 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141680+0000", 2024-09-21T14:59:32.162 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 41.958887326000003, 2024-09-21T14:59:32.162 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037806166000000002, 2024-09-21T14:59:32.162 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:32.162 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:32.162 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:32.162 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:32.162 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:32.162 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22748 2024-09-21T14:59:32.162 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.162 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:32.162 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.162 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:32.162 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141680+0000", 2024-09-21T14:59:32.162 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:32.162 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.163 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.163 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:32.163 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141680+0000", 2024-09-21T14:59:32.163 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:32.163 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.163 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.163 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:32.163 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141679+0000", 2024-09-21T14:59:32.163 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:32.163 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.163 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.163 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:32.163 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141683+0000", 2024-09-21T14:59:32.163 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3230000000000002e-06 2024-09-21T14:59:32.163 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.163 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.164 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:32.164 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141690+0000", 2024-09-21T14:59:32.164 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8680000000000001e-06 2024-09-21T14:59:32.164 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.164 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.164 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:32.164 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141694+0000", 2024-09-21T14:59:32.164 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6770000000000003e-06 2024-09-21T14:59:32.164 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.164 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.164 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:32.164 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179300+0000", 2024-09-21T14:59:32.164 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037605567999999999 2024-09-21T14:59:32.164 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.165 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.165 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:32.165 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179355+0000", 2024-09-21T14:59:32.165 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4771000000000001e-05 2024-09-21T14:59:32.165 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.165 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.165 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:32.165 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179486+0000", 2024-09-21T14:59:32.165 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013141699999999999 2024-09-21T14:59:32.165 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:32.165 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:32.165 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:32.165 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.165 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.165 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22749 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 770048~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:32.166 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141698+0000", 2024-09-21T14:59:32.166 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 41.958868954000003, 2024-09-21T14:59:32.166 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037967813000000003, 2024-09-21T14:59:32.166 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:32.166 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:32.166 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:32.166 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:32.166 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:32.166 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22749 2024-09-21T14:59:32.166 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.166 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:32.166 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.166 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:32.166 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141698+0000", 2024-09-21T14:59:32.166 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:32.166 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.167 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.167 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:32.167 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141698+0000", 2024-09-21T14:59:32.167 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:32.167 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.167 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.167 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:32.167 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141698+0000", 2024-09-21T14:59:32.167 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:32.167 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.167 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.167 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:32.167 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141701+0000", 2024-09-21T14:59:32.167 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8399999999999999e-06 2024-09-21T14:59:32.167 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.167 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.167 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:32.168 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141706+0000", 2024-09-21T14:59:32.168 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4979999999999997e-06 2024-09-21T14:59:32.168 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.168 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.168 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:32.168 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141711+0000", 2024-09-21T14:59:32.168 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3850000000000004e-06 2024-09-21T14:59:32.168 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.168 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.168 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:32.168 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179504+0000", 2024-09-21T14:59:32.168 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037793514 2024-09-21T14:59:32.168 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.168 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.168 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:32.168 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179546+0000", 2024-09-21T14:59:32.169 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1467999999999997e-05 2024-09-21T14:59:32.169 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.169 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.169 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:32.169 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179666+0000", 2024-09-21T14:59:32.169 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000120499 2024-09-21T14:59:32.169 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:32.169 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:32.169 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:32.169 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.169 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.169 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22750 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 835584~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:32.169 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141715+0000", 2024-09-21T14:59:32.169 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 41.958852653000001, 2024-09-21T14:59:32.169 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038173179000000002, 2024-09-21T14:59:32.170 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:32.170 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:32.170 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:32.170 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:32.170 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:32.170 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22750 2024-09-21T14:59:32.170 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.170 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:32.170 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.170 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:32.170 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141715+0000", 2024-09-21T14:59:32.170 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:32.170 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.170 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.170 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:32.170 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141715+0000", 2024-09-21T14:59:32.171 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:32.171 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.171 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.171 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:32.171 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141714+0000", 2024-09-21T14:59:32.171 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:32.171 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.171 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.171 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:32.171 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141716+0000", 2024-09-21T14:59:32.171 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1730000000000002e-06 2024-09-21T14:59:32.171 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.171 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.171 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:32.172 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141722+0000", 2024-09-21T14:59:32.172 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.417e-06 2024-09-21T14:59:32.172 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.172 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.172 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:32.172 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141726+0000", 2024-09-21T14:59:32.172 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4830000000000001e-06 2024-09-21T14:59:32.172 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.172 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.172 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:32.172 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179683+0000", 2024-09-21T14:59:32.172 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037956977000000003 2024-09-21T14:59:32.172 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.172 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.172 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:32.172 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179751+0000", 2024-09-21T14:59:32.172 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7762000000000006e-05 2024-09-21T14:59:32.173 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.173 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.173 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:32.173 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179888+0000", 2024-09-21T14:59:32.173 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000136635 2024-09-21T14:59:32.173 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:32.173 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:32.173 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:32.173 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.173 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.173 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22751 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 901120~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:32.173 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141730+0000", 2024-09-21T14:59:32.173 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 41.958836853999998, 2024-09-21T14:59:32.173 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038357011000000003, 2024-09-21T14:59:32.173 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:32.174 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:32.174 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:32.174 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:32.174 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:32.174 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22751 2024-09-21T14:59:32.174 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.174 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:32.174 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.174 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:32.174 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141730+0000", 2024-09-21T14:59:32.174 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:32.174 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.174 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.174 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:32.174 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141730+0000", 2024-09-21T14:59:32.175 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:32.175 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.175 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.175 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:32.175 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141730+0000", 2024-09-21T14:59:32.175 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:32.175 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.175 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.175 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:32.175 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141737+0000", 2024-09-21T14:59:32.175 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.64e-06 2024-09-21T14:59:32.175 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.175 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.175 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:32.175 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141743+0000", 2024-09-21T14:59:32.175 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8130000000000001e-06 2024-09-21T14:59:32.176 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.176 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.176 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:32.176 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141747+0000", 2024-09-21T14:59:32.176 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4920000000000004e-06 2024-09-21T14:59:32.176 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.176 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.176 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:32.176 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179918+0000", 2024-09-21T14:59:32.176 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038170822 2024-09-21T14:59:32.176 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.176 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.176 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:32.176 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179960+0000", 2024-09-21T14:59:32.176 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1999000000000002e-05 2024-09-21T14:59:32.177 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.177 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.177 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:32.177 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180087+0000", 2024-09-21T14:59:32.177 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012752799999999999 2024-09-21T14:59:32.177 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:32.177 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:32.177 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:32.177 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.177 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.177 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22752 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 966656~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:32.177 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141751+0000", 2024-09-21T14:59:32.177 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 41.958816534, 2024-09-21T14:59:32.177 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038553084000000001, 2024-09-21T14:59:32.177 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:32.177 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:32.178 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:32.178 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:32.178 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:32.178 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22752 2024-09-21T14:59:32.178 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.178 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:32.178 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.178 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:32.178 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141751+0000", 2024-09-21T14:59:32.178 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:32.178 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.178 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.178 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:32.178 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141751+0000", 2024-09-21T14:59:32.178 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:32.179 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.179 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.179 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:32.179 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141750+0000", 2024-09-21T14:59:32.179 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:32.179 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.179 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.179 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:32.179 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141753+0000", 2024-09-21T14:59:32.179 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5900000000000002e-06 2024-09-21T14:59:32.179 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.179 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.179 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:32.179 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141759+0000", 2024-09-21T14:59:32.179 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.947e-06 2024-09-21T14:59:32.179 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.180 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.180 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:32.180 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141764+0000", 2024-09-21T14:59:32.180 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8319999999999996e-06 2024-09-21T14:59:32.180 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.180 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.180 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:32.180 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180108+0000", 2024-09-21T14:59:32.180 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038344095000000002 2024-09-21T14:59:32.180 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.180 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.180 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:32.180 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180168+0000", 2024-09-21T14:59:32.180 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0047000000000003e-05 2024-09-21T14:59:32.180 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.180 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.180 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:32.181 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180304+0000", 2024-09-21T14:59:32.181 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013592500000000001 2024-09-21T14:59:32.181 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:32.181 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:32.181 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:32.181 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.181 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.181 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22753 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 1032192~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:32.181 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141768+0000", 2024-09-21T14:59:32.181 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 41.958799659999997, 2024-09-21T14:59:32.181 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038818810000000002, 2024-09-21T14:59:32.181 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:32.181 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:32.181 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:32.181 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:32.182 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:32.182 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22753 2024-09-21T14:59:32.182 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.182 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:32.182 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.182 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:32.182 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141768+0000", 2024-09-21T14:59:32.182 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:32.182 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.182 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.182 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:32.182 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141768+0000", 2024-09-21T14:59:32.182 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:32.182 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.182 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.182 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:32.183 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141767+0000", 2024-09-21T14:59:32.183 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:32.183 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.183 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.183 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:32.183 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141770+0000", 2024-09-21T14:59:32.183 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3329999999999999e-06 2024-09-21T14:59:32.183 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.183 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.183 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:32.183 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141775+0000", 2024-09-21T14:59:32.183 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2669999999999997e-06 2024-09-21T14:59:32.183 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.183 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.183 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:32.184 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141779+0000", 2024-09-21T14:59:32.184 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2259999999999999e-06 2024-09-21T14:59:32.184 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.184 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.184 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:32.184 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180325+0000", 2024-09-21T14:59:32.184 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038546428000000001 2024-09-21T14:59:32.184 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.184 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.184 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:32.184 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180401+0000", 2024-09-21T14:59:32.184 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5437000000000003e-05 2024-09-21T14:59:32.184 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:32.184 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:32.184 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:32.184 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180586+0000", 2024-09-21T14:59:32.185 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000185429 2024-09-21T14:59:32.185 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:32.185 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:32.185 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:32.185 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:32.185 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:32.185 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:32.186 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T14:59:32.187 INFO:teuthology.orchestra.run.smithi032.stderr:nodeep-scrub is set 2024-09-21T14:59:32.187 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:59:32.183+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:59:32.287 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:59:32.283+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:59:32.304 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:59:32.304 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:59:32.304 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T14:59:32.304 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:59:32.314 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T14:59:32.319 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 5, 6, 7, 4] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T14:59:32.319 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T14:59:32.320 INFO:tasks.thrashosds.thrasher:inject_pause on osd.2 2024-09-21T14:59:32.320 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-21T14:59:32.320 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-21T14:59:32.320 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok config set filestore_inject_stall 3 2024-09-21T14:59:32.388 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:59:32.383+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:59:32.419 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:59:32.419 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:59:32.419 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T14:59:32.420 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T14:59:32.420 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:59:32.429 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T14:59:32.449 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:32.449 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-21T14:59:32.449 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:32.488 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:59:32.483+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:59:32.587 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:59:32.587 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T14:59:32.588 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T14:59:32.588 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T14:59:32.588 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.588 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:1452 3.19 3:9d40fe72:::rbd_data.1136515ecb6b.0000000000000013:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T14:59:32.588 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:30.292755+0000", 2024-09-21T14:59:32.588 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 62.291945814999998, 2024-09-21T14:59:32.588 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043852191999999998, 2024-09-21T14:59:32.588 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:32.588 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:32.588 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:32.588 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:32.588 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:32.588 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1452 2024-09-21T14:59:32.589 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.589 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:32.589 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.589 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:32.589 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292755+0000", 2024-09-21T14:59:32.589 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:32.589 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.589 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.589 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:32.589 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292755+0000", 2024-09-21T14:59:32.589 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:32.589 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.589 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.589 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:32.590 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292754+0000", 2024-09-21T14:59:32.590 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:32.590 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.590 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.590 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:32.590 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292760+0000", 2024-09-21T14:59:32.590 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5029999999999997e-06 2024-09-21T14:59:32.590 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.590 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.590 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:32.590 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292771+0000", 2024-09-21T14:59:32.590 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1551e-05 2024-09-21T14:59:32.590 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.591 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.591 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:32.591 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292779+0000", 2024-09-21T14:59:32.591 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1559999999999998e-06 2024-09-21T14:59:32.591 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.591 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.591 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:32.591 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292896+0000", 2024-09-21T14:59:32.591 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000117304 2024-09-21T14:59:32.591 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.591 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.591 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:32.591 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.293124+0000", 2024-09-21T14:59:32.592 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00022819999999999999 2024-09-21T14:59:32.592 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.592 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.592 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 1", 2024-09-21T14:59:32.592 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.299967+0000", 2024-09-21T14:59:32.592 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0068427009999999996 2024-09-21T14:59:32.592 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.592 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.592 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:32.592 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336463+0000", 2024-09-21T14:59:32.592 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.036495842000000001 2024-09-21T14:59:32.592 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.592 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.592 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:59:32.593 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336480+0000", 2024-09-21T14:59:32.593 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7419999999999999e-05 2024-09-21T14:59:32.593 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.593 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.593 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:59:32.593 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336550+0000", 2024-09-21T14:59:32.593 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9951999999999997e-05 2024-09-21T14:59:32.593 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.593 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.593 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:32.593 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336607+0000", 2024-09-21T14:59:32.593 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7423000000000002e-05 2024-09-21T14:59:32.594 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:32.594 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:32.594 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:32.594 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.594 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.594 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop_reply(client.4406.0:1452 3.19 e36/24 ondisk, result = 0)", 2024-09-21T14:59:32.594 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:30.322854+0000", 2024-09-21T14:59:32.594 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 62.261847138999997, 2024-09-21T14:59:32.594 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013613079, 2024-09-21T14:59:32.594 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:32.594 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:32.594 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:32.594 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.595 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:32.595 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.322854+0000", 2024-09-21T14:59:32.595 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:32.595 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.595 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.595 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:32.595 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.322854+0000", 2024-09-21T14:59:32.595 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:32.595 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.595 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.595 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:32.595 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.322852+0000", 2024-09-21T14:59:32.595 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:59:32.595 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.596 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.596 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:32.596 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.322860+0000", 2024-09-21T14:59:32.596 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.605e-06 2024-09-21T14:59:32.596 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.596 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.596 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:32.596 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.322878+0000", 2024-09-21T14:59:32.596 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8204000000000001e-05 2024-09-21T14:59:32.596 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.596 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.596 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:32.596 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.322888+0000", 2024-09-21T14:59:32.597 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4099999999999997e-06 2024-09-21T14:59:32.597 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.597 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.597 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:32.597 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336440+0000", 2024-09-21T14:59:32.597 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013551848 2024-09-21T14:59:32.597 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.597 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.597 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:32.597 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336453+0000", 2024-09-21T14:59:32.597 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3465000000000001e-05 2024-09-21T14:59:32.597 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.597 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.597 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:32.598 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336467+0000", 2024-09-21T14:59:32.598 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3913999999999999e-05 2024-09-21T14:59:32.598 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:32.598 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:32.598 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:32.598 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.598 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.598 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:1567 3.8 3:139d3d55:::rbd_data.1136515ecb6b.0000000000000000:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T14:59:32.598 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:30.681087+0000", 2024-09-21T14:59:32.598 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 61.9036136, 2024-09-21T14:59:32.598 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.039009224000000002, 2024-09-21T14:59:32.598 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:32.598 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:32.598 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:32.599 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:32.599 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:32.599 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1567 2024-09-21T14:59:32.599 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.599 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:32.599 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.599 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:32.599 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681087+0000", 2024-09-21T14:59:32.599 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:32.599 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.599 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.599 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:32.599 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681087+0000", 2024-09-21T14:59:32.599 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:32.600 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.600 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.600 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:32.600 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681086+0000", 2024-09-21T14:59:32.600 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:59:32.600 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.600 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.600 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:32.600 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681097+0000", 2024-09-21T14:59:32.600 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.134e-05 2024-09-21T14:59:32.600 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.600 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.600 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:32.600 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681122+0000", 2024-09-21T14:59:32.601 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5313e-05 2024-09-21T14:59:32.601 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.601 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.601 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:32.601 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681138+0000", 2024-09-21T14:59:32.601 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5916000000000001e-05 2024-09-21T14:59:32.601 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.601 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.601 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:32.601 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681270+0000", 2024-09-21T14:59:32.601 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000131474 2024-09-21T14:59:32.601 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.601 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.601 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:32.601 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681416+0000", 2024-09-21T14:59:32.602 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014637 2024-09-21T14:59:32.602 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.602 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.602 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 7", 2024-09-21T14:59:32.602 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.693163+0000", 2024-09-21T14:59:32.602 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011747241 2024-09-21T14:59:32.602 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.602 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.602 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:59:32.602 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.719750+0000", 2024-09-21T14:59:32.602 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.026586852000000001 2024-09-21T14:59:32.602 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.602 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.602 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:32.603 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.719859+0000", 2024-09-21T14:59:32.603 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010925999999999999 2024-09-21T14:59:32.603 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.603 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.603 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:59:32.603 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.719951+0000", 2024-09-21T14:59:32.603 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1294999999999999e-05 2024-09-21T14:59:32.603 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.603 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.603 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:32.603 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.720097+0000", 2024-09-21T14:59:32.603 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014583 2024-09-21T14:59:32.603 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:32.603 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:32.604 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:32.604 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.604 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.604 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9347 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 16384~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:32.604 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158291+0000", 2024-09-21T14:59:32.604 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 53.426410099000002, 2024-09-21T14:59:32.604 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013600958, 2024-09-21T14:59:32.604 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:32.604 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:32.604 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:32.604 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:32.604 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:32.604 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9347 2024-09-21T14:59:32.604 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.605 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:32.605 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.605 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:32.605 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158291+0000", 2024-09-21T14:59:32.605 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:32.605 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.605 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.605 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:32.605 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158291+0000", 2024-09-21T14:59:32.605 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:32.605 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.605 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.605 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:32.605 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158290+0000", 2024-09-21T14:59:32.605 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:32.605 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.606 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.606 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:32.606 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158295+0000", 2024-09-21T14:59:32.606 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2399999999999998e-06 2024-09-21T14:59:32.606 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.606 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.606 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:32.606 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158312+0000", 2024-09-21T14:59:32.606 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6929999999999999e-05 2024-09-21T14:59:32.606 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.606 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.606 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:32.606 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158353+0000", 2024-09-21T14:59:32.606 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0805e-05 2024-09-21T14:59:32.606 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.607 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.607 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:32.607 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.171634+0000", 2024-09-21T14:59:32.607 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.01328085 2024-09-21T14:59:32.607 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.607 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.607 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:32.607 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.171722+0000", 2024-09-21T14:59:32.607 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8115000000000006e-05 2024-09-21T14:59:32.607 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.607 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.607 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:32.607 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.171892+0000", 2024-09-21T14:59:32.607 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000169722 2024-09-21T14:59:32.608 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:32.608 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:32.608 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:32.608 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.608 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.608 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9348 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 122880~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:32.608 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158369+0000", 2024-09-21T14:59:32.608 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 53.426331931, 2024-09-21T14:59:32.608 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.01388847, 2024-09-21T14:59:32.608 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:32.608 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:32.608 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:32.608 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:32.608 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:32.608 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9348 2024-09-21T14:59:32.609 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.609 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:32.609 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.609 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:32.609 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158369+0000", 2024-09-21T14:59:32.609 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:32.609 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.609 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.609 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:32.609 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158369+0000", 2024-09-21T14:59:32.609 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:32.609 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.609 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.609 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:32.609 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158368+0000", 2024-09-21T14:59:32.610 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:32.610 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.610 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.610 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:32.610 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158374+0000", 2024-09-21T14:59:32.610 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0499999999999997e-06 2024-09-21T14:59:32.610 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.610 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.610 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:32.610 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158386+0000", 2024-09-21T14:59:32.610 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1579999999999999e-05 2024-09-21T14:59:32.610 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.610 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.610 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:32.610 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158396+0000", 2024-09-21T14:59:32.611 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.8260000000000005e-06 2024-09-21T14:59:32.611 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.611 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.611 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:32.611 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.171967+0000", 2024-09-21T14:59:32.611 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013571318000000001 2024-09-21T14:59:32.611 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.611 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.611 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:32.611 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172034+0000", 2024-09-21T14:59:32.611 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.656e-05 2024-09-21T14:59:32.611 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.611 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.611 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:32.612 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172258+0000", 2024-09-21T14:59:32.612 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00022395999999999999 2024-09-21T14:59:32.612 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:32.612 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:32.612 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:32.612 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.612 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.612 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9349 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 155648~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:32.612 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158403+0000", 2024-09-21T14:59:32.612 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 53.426298185, 2024-09-21T14:59:32.612 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014229851, 2024-09-21T14:59:32.612 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:32.612 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:32.612 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:32.612 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:32.612 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:32.613 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9349 2024-09-21T14:59:32.613 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.613 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:32.613 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.613 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:32.613 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158403+0000", 2024-09-21T14:59:32.613 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:32.613 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.613 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.613 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:32.613 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158403+0000", 2024-09-21T14:59:32.613 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:32.613 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.613 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.614 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:32.614 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158402+0000", 2024-09-21T14:59:32.614 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:32.614 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.614 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.614 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:32.614 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158406+0000", 2024-09-21T14:59:32.614 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1339999999999997e-06 2024-09-21T14:59:32.614 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.614 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.614 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:32.614 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158416+0000", 2024-09-21T14:59:32.614 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3149999999999998e-06 2024-09-21T14:59:32.614 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.615 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.615 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:32.615 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158439+0000", 2024-09-21T14:59:32.615 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3255999999999999e-05 2024-09-21T14:59:32.615 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.615 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.615 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:32.615 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172305+0000", 2024-09-21T14:59:32.615 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013866603999999999 2024-09-21T14:59:32.615 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.615 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.615 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:32.615 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172413+0000", 2024-09-21T14:59:32.615 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000107664 2024-09-21T14:59:32.615 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.616 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.616 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:32.616 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172633+0000", 2024-09-21T14:59:32.616 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021950800000000001 2024-09-21T14:59:32.616 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:32.616 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:32.616 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:32.616 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.616 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.616 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9350 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 294912~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:32.616 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158448+0000", 2024-09-21T14:59:32.616 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 53.426253314, 2024-09-21T14:59:32.616 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014471525000000001, 2024-09-21T14:59:32.616 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:32.616 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:32.616 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:32.617 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:32.617 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:32.617 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9350 2024-09-21T14:59:32.617 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.617 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:32.617 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.617 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:32.617 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158448+0000", 2024-09-21T14:59:32.617 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:32.617 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.617 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.617 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:32.617 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158448+0000", 2024-09-21T14:59:32.617 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:32.617 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.618 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.618 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:32.618 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158447+0000", 2024-09-21T14:59:32.618 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:32.618 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.618 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.618 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:32.618 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158452+0000", 2024-09-21T14:59:32.618 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8400000000000002e-06 2024-09-21T14:59:32.618 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.618 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.618 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:32.618 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158475+0000", 2024-09-21T14:59:32.618 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3091e-05 2024-09-21T14:59:32.618 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.619 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.619 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:32.619 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158515+0000", 2024-09-21T14:59:32.619 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0265000000000002e-05 2024-09-21T14:59:32.619 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.619 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.619 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:32.619 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172682+0000", 2024-09-21T14:59:32.619 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014166761999999999 2024-09-21T14:59:32.619 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.619 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.619 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:32.619 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172752+0000", 2024-09-21T14:59:32.619 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9698999999999998e-05 2024-09-21T14:59:32.620 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.620 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.620 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:32.620 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172919+0000", 2024-09-21T14:59:32.620 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016759199999999999 2024-09-21T14:59:32.620 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:32.620 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:32.620 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:32.620 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.620 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.620 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9351 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 434176~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:32.620 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158526+0000", 2024-09-21T14:59:32.620 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 53.426174596000003, 2024-09-21T14:59:32.620 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014662107000000001, 2024-09-21T14:59:32.620 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:32.620 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:32.621 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:32.621 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:32.621 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:32.621 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9351 2024-09-21T14:59:32.621 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.621 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:32.621 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.621 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:32.621 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158526+0000", 2024-09-21T14:59:32.621 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:32.621 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.621 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.621 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:32.621 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158526+0000", 2024-09-21T14:59:32.621 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:32.622 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.622 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.622 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:32.622 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158525+0000", 2024-09-21T14:59:32.622 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:32.622 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.622 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.622 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:32.622 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158531+0000", 2024-09-21T14:59:32.622 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5840000000000002e-06 2024-09-21T14:59:32.622 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.622 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.622 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:32.622 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158564+0000", 2024-09-21T14:59:32.623 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3195000000000003e-05 2024-09-21T14:59:32.623 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.623 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.623 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:32.623 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158577+0000", 2024-09-21T14:59:32.623 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2330000000000001e-05 2024-09-21T14:59:32.623 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.623 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.623 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:32.623 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172956+0000", 2024-09-21T14:59:32.623 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014379438 2024-09-21T14:59:32.623 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.623 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.623 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:32.624 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173020+0000", 2024-09-21T14:59:32.624 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4401999999999998e-05 2024-09-21T14:59:32.624 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.624 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.624 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:32.624 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173188+0000", 2024-09-21T14:59:32.624 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016804 2024-09-21T14:59:32.624 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:32.624 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:32.624 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:32.624 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.624 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.624 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9379 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 24576~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:32.624 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158586+0000", 2024-09-21T14:59:32.625 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 53.426115336000002, 2024-09-21T14:59:32.625 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014865173000000001, 2024-09-21T14:59:32.625 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:32.625 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:32.625 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:32.625 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:32.625 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:32.625 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9379 2024-09-21T14:59:32.625 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.625 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:32.625 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.625 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:32.625 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158586+0000", 2024-09-21T14:59:32.625 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:32.625 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.625 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.626 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:32.626 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158586+0000", 2024-09-21T14:59:32.626 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:32.626 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.626 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.626 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:32.626 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158585+0000", 2024-09-21T14:59:32.626 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:32.626 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.626 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.626 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:32.626 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158590+0000", 2024-09-21T14:59:32.626 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5820000000000001e-06 2024-09-21T14:59:32.626 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.627 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.627 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:32.627 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158676+0000", 2024-09-21T14:59:32.627 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5075999999999997e-05 2024-09-21T14:59:32.627 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.627 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.627 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:32.627 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158695+0000", 2024-09-21T14:59:32.627 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9137999999999999e-05 2024-09-21T14:59:32.627 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.627 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.627 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:32.627 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173219+0000", 2024-09-21T14:59:32.627 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014524034999999999 2024-09-21T14:59:32.627 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.628 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.628 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:32.628 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173276+0000", 2024-09-21T14:59:32.628 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7442000000000003e-05 2024-09-21T14:59:32.628 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.628 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.628 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:32.628 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173451+0000", 2024-09-21T14:59:32.628 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017464800000000001 2024-09-21T14:59:32.628 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:32.628 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:32.628 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:32.628 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.628 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.629 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9380 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 163840~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:32.629 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158706+0000", 2024-09-21T14:59:32.629 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 53.425994826, 2024-09-21T14:59:32.629 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015096381000000001, 2024-09-21T14:59:32.629 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:32.629 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:32.629 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:32.629 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:32.629 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:32.629 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9380 2024-09-21T14:59:32.629 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.629 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:32.629 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.629 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:32.629 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158706+0000", 2024-09-21T14:59:32.629 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:32.630 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.630 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.630 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:32.630 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158706+0000", 2024-09-21T14:59:32.630 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:32.630 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.630 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.630 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:32.630 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158705+0000", 2024-09-21T14:59:32.630 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:32.630 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.630 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.630 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:32.631 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158712+0000", 2024-09-21T14:59:32.631 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7240000000000003e-06 2024-09-21T14:59:32.631 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.631 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.631 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:32.631 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158755+0000", 2024-09-21T14:59:32.631 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2953e-05 2024-09-21T14:59:32.631 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.631 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.631 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:32.631 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158765+0000", 2024-09-21T14:59:32.631 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0149000000000001e-05 2024-09-21T14:59:32.631 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.631 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.631 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:32.632 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173492+0000", 2024-09-21T14:59:32.632 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014727105000000001 2024-09-21T14:59:32.632 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.632 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.632 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:32.632 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173564+0000", 2024-09-21T14:59:32.632 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1789e-05 2024-09-21T14:59:32.632 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.632 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.632 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:32.632 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173803+0000", 2024-09-21T14:59:32.632 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00023867100000000001 2024-09-21T14:59:32.632 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:32.632 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:32.632 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:32.633 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.633 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.633 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9381 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 303104~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:32.633 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158774+0000", 2024-09-21T14:59:32.633 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 53.425926746000002, 2024-09-21T14:59:32.633 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015309095999999999, 2024-09-21T14:59:32.633 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:32.633 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:32.633 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:32.633 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:32.633 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:32.633 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9381 2024-09-21T14:59:32.633 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.633 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:32.633 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.634 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:32.634 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158774+0000", 2024-09-21T14:59:32.634 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:32.634 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.634 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.634 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:32.634 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158774+0000", 2024-09-21T14:59:32.634 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:32.634 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.634 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.634 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:32.634 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158773+0000", 2024-09-21T14:59:32.634 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:32.634 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.635 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.635 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:32.635 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158779+0000", 2024-09-21T14:59:32.635 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6999999999999996e-06 2024-09-21T14:59:32.635 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.635 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.635 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:32.635 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158790+0000", 2024-09-21T14:59:32.635 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0594000000000001e-05 2024-09-21T14:59:32.635 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.635 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.635 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:32.635 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158798+0000", 2024-09-21T14:59:32.635 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5860000000000005e-06 2024-09-21T14:59:32.635 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.636 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.636 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:32.636 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173829+0000", 2024-09-21T14:59:32.636 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015030349 2024-09-21T14:59:32.636 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.636 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.636 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:32.636 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173918+0000", 2024-09-21T14:59:32.636 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9375000000000004e-05 2024-09-21T14:59:32.636 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.636 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.636 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:32.636 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.174083+0000", 2024-09-21T14:59:32.636 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016536200000000001 2024-09-21T14:59:32.636 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:32.637 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:32.637 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:32.637 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.637 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.637 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9382 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 442368~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:32.637 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158805+0000", 2024-09-21T14:59:32.637 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 53.425895840999999, 2024-09-21T14:59:32.637 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015592623, 2024-09-21T14:59:32.637 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:32.637 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:32.637 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:32.637 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:32.637 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:32.637 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9382 2024-09-21T14:59:32.637 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.637 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:32.638 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.638 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:32.638 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158805+0000", 2024-09-21T14:59:32.638 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:32.638 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.638 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.638 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:32.638 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158805+0000", 2024-09-21T14:59:32.638 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:32.638 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.638 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.638 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:32.638 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158805+0000", 2024-09-21T14:59:32.638 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:32.639 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.639 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.639 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:32.639 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158808+0000", 2024-09-21T14:59:32.639 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9179999999999996e-06 2024-09-21T14:59:32.639 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.639 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.639 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:32.639 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158817+0000", 2024-09-21T14:59:32.639 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9299999999999992e-06 2024-09-21T14:59:32.639 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.639 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.639 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:32.639 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158825+0000", 2024-09-21T14:59:32.639 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6739999999999997e-06 2024-09-21T14:59:32.640 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.640 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.640 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:32.640 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.174122+0000", 2024-09-21T14:59:32.640 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.01529694 2024-09-21T14:59:32.640 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.640 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.640 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:32.640 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.174185+0000", 2024-09-21T14:59:32.640 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3015000000000006e-05 2024-09-21T14:59:32.640 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.640 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.640 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:32.640 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.174398+0000", 2024-09-21T14:59:32.640 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021274799999999999 2024-09-21T14:59:32.640 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:32.641 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:32.641 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:32.641 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.641 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.641 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22700 3.19 3:98de45b8:::rbd_data.11366acc894.0000000000000000:19 [read 704512~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:32.641 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.140191+0000", 2024-09-21T14:59:32.641 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.444510227000002, 2024-09-21T14:59:32.641 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013770486, 2024-09-21T14:59:32.641 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:32.641 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:32.641 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:32.641 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:32.641 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:32.641 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22700 2024-09-21T14:59:32.641 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.641 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:32.642 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.642 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:32.642 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140191+0000", 2024-09-21T14:59:32.642 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:32.642 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.642 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.642 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:32.642 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140191+0000", 2024-09-21T14:59:32.642 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:32.642 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.642 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.642 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:32.642 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140190+0000", 2024-09-21T14:59:32.642 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:32.642 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.643 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.643 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:32.643 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140199+0000", 2024-09-21T14:59:32.643 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3340000000000002e-06 2024-09-21T14:59:32.643 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.643 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.643 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:32.643 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140205+0000", 2024-09-21T14:59:32.643 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3820000000000001e-06 2024-09-21T14:59:32.643 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.643 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.643 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:32.643 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140210+0000", 2024-09-21T14:59:32.643 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0969999999999996e-06 2024-09-21T14:59:32.643 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.644 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.644 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:32.644 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.153713+0000", 2024-09-21T14:59:32.644 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013502713 2024-09-21T14:59:32.644 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.644 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.644 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:32.644 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.153786+0000", 2024-09-21T14:59:32.644 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3022999999999995e-05 2024-09-21T14:59:32.644 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.644 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.644 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:32.644 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.153961+0000", 2024-09-21T14:59:32.644 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000175274 2024-09-21T14:59:32.644 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:32.645 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:32.645 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:32.645 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.645 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.645 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22701 3.19 3:98de45b8:::rbd_data.11366acc894.0000000000000000:19 [read 770048~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:32.645 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.140215+0000", 2024-09-21T14:59:32.645 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.444486410000003, 2024-09-21T14:59:32.645 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014072497, 2024-09-21T14:59:32.645 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:32.645 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:32.645 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:32.645 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:32.645 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:32.645 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22701 2024-09-21T14:59:32.645 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.645 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:32.646 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.646 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:32.646 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140215+0000", 2024-09-21T14:59:32.646 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:32.646 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.646 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.646 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:32.646 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140215+0000", 2024-09-21T14:59:32.646 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:32.646 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.646 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.646 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:32.646 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140214+0000", 2024-09-21T14:59:32.646 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:32.646 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.647 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.647 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:32.647 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140217+0000", 2024-09-21T14:59:32.647 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.74e-06 2024-09-21T14:59:32.647 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.647 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.647 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:32.647 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140223+0000", 2024-09-21T14:59:32.647 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0929999999999999e-06 2024-09-21T14:59:32.647 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.647 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.647 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:32.647 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140228+0000", 2024-09-21T14:59:32.647 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2279999999999998e-06 2024-09-21T14:59:32.647 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.648 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.648 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:32.648 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.153991+0000", 2024-09-21T14:59:32.648 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013762329 2024-09-21T14:59:32.648 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.648 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.648 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:32.648 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.154098+0000", 2024-09-21T14:59:32.648 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000107117 2024-09-21T14:59:32.648 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.648 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.648 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:32.648 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.154287+0000", 2024-09-21T14:59:32.648 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000189431 2024-09-21T14:59:32.648 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:32.649 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:32.649 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:32.649 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.649 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.649 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22702 3.19 3:98de45b8:::rbd_data.11366acc894.0000000000000000:19 [read 835584~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:32.649 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.140232+0000", 2024-09-21T14:59:32.649 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.444468557999997, 2024-09-21T14:59:32.649 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015338742000000001, 2024-09-21T14:59:32.649 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:32.649 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:32.649 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:32.649 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:32.649 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:32.649 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22702 2024-09-21T14:59:32.649 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.649 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:32.650 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.650 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:32.650 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140232+0000", 2024-09-21T14:59:32.650 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:32.650 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.650 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.650 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:32.650 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140232+0000", 2024-09-21T14:59:32.650 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:32.650 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.650 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.650 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:32.650 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140232+0000", 2024-09-21T14:59:32.650 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:32.651 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.651 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.651 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:32.651 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140234+0000", 2024-09-21T14:59:32.651 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.402e-06 2024-09-21T14:59:32.651 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.651 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.651 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:32.651 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140241+0000", 2024-09-21T14:59:32.651 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.054e-06 2024-09-21T14:59:32.651 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.651 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.651 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:32.651 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140246+0000", 2024-09-21T14:59:32.651 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0050000000000004e-06 2024-09-21T14:59:32.651 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.652 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.652 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:32.652 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.154315+0000", 2024-09-21T14:59:32.652 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014069440000000001 2024-09-21T14:59:32.652 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.652 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.652 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:32.652 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.154383+0000", 2024-09-21T14:59:32.652 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8151000000000005e-05 2024-09-21T14:59:32.652 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.652 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.652 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:32.652 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155571+0000", 2024-09-21T14:59:32.652 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001188029 2024-09-21T14:59:32.652 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:32.652 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:32.653 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:32.653 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.653 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.653 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22703 3.19 3:98de45b8:::rbd_data.11366acc894.0000000000000000:19 [read 901120~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:32.653 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.140250+0000", 2024-09-21T14:59:32.653 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.444451399999998, 2024-09-21T14:59:32.653 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014992169, 2024-09-21T14:59:32.653 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:32.653 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:32.653 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:32.653 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:32.653 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:32.653 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22703 2024-09-21T14:59:32.653 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.653 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:32.653 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.654 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:32.654 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140250+0000", 2024-09-21T14:59:32.654 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:32.654 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.654 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.654 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:32.654 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140250+0000", 2024-09-21T14:59:32.654 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:32.654 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.654 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.654 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:32.654 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140249+0000", 2024-09-21T14:59:32.655 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:32.655 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.655 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.655 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:32.655 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140251+0000", 2024-09-21T14:59:32.655 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.136e-06 2024-09-21T14:59:32.655 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.655 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.655 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:32.655 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140257+0000", 2024-09-21T14:59:32.655 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9610000000000002e-06 2024-09-21T14:59:32.655 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.656 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.656 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:32.656 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140262+0000", 2024-09-21T14:59:32.656 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4549999999999997e-06 2024-09-21T14:59:32.656 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.656 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.656 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:32.656 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155006+0000", 2024-09-21T14:59:32.656 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014744189 2024-09-21T14:59:32.656 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.656 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.656 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:32.656 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155081+0000", 2024-09-21T14:59:32.657 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4548000000000005e-05 2024-09-21T14:59:32.657 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.657 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.657 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:32.657 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155242+0000", 2024-09-21T14:59:32.657 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000161204 2024-09-21T14:59:32.657 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:32.657 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:32.657 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:32.657 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.657 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.657 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22704 3.19 3:98de45b8:::rbd_data.11366acc894.0000000000000000:19 [read 966656~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:32.657 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.140266+0000", 2024-09-21T14:59:32.658 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.444435083000002, 2024-09-21T14:59:32.658 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015223443999999999, 2024-09-21T14:59:32.658 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:32.658 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:32.658 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:32.658 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:32.658 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:32.658 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22704 2024-09-21T14:59:32.658 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.658 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:32.658 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.658 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:32.658 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140266+0000", 2024-09-21T14:59:32.659 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:32.659 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.659 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.659 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:32.659 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140266+0000", 2024-09-21T14:59:32.659 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:32.659 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.659 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.659 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:32.659 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140266+0000", 2024-09-21T14:59:32.659 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:32.659 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.659 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.659 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:32.660 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140268+0000", 2024-09-21T14:59:32.660 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.475e-06 2024-09-21T14:59:32.660 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.660 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.660 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:32.660 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140274+0000", 2024-09-21T14:59:32.660 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.319e-06 2024-09-21T14:59:32.660 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.660 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.660 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:32.660 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140279+0000", 2024-09-21T14:59:32.660 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.882e-06 2024-09-21T14:59:32.660 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.660 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.660 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:32.661 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155264+0000", 2024-09-21T14:59:32.661 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014984655 2024-09-21T14:59:32.661 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.661 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.661 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:32.661 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155338+0000", 2024-09-21T14:59:32.661 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4091999999999995e-05 2024-09-21T14:59:32.661 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.661 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.661 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:32.661 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155489+0000", 2024-09-21T14:59:32.661 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000151349 2024-09-21T14:59:32.661 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:32.661 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:32.662 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:32.662 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.662 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.662 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22705 3.19 3:98de45b8:::rbd_data.11366acc894.0000000000000000:19 [read 1032192~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:32.662 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.140284+0000", 2024-09-21T14:59:32.662 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.444417225999999, 2024-09-21T14:59:32.662 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015467896, 2024-09-21T14:59:32.662 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:32.662 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:32.662 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:32.662 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:32.662 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:32.662 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22705 2024-09-21T14:59:32.662 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.663 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:32.663 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.663 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:32.663 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140284+0000", 2024-09-21T14:59:32.663 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:32.663 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.663 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.663 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:32.663 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140284+0000", 2024-09-21T14:59:32.663 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:32.663 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.663 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.663 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:32.663 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140283+0000", 2024-09-21T14:59:32.664 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:32.664 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.664 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.664 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:32.664 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140286+0000", 2024-09-21T14:59:32.664 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7130000000000001e-06 2024-09-21T14:59:32.664 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.664 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.664 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:32.664 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140292+0000", 2024-09-21T14:59:32.664 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3430000000000001e-06 2024-09-21T14:59:32.664 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.664 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.664 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:32.664 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140298+0000", 2024-09-21T14:59:32.665 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9730000000000002e-06 2024-09-21T14:59:32.665 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.665 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.665 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:32.665 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155510+0000", 2024-09-21T14:59:32.665 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015211738000000001 2024-09-21T14:59:32.665 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.665 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.665 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:32.665 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155598+0000", 2024-09-21T14:59:32.665 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7822e-05 2024-09-21T14:59:32.665 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.665 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.665 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:32.666 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155752+0000", 2024-09-21T14:59:32.666 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0001537 2024-09-21T14:59:32.666 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:32.666 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:32.666 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:32.666 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.666 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.666 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:37892 3.1b 3:d963c97f:::rbd_data.11365d19aa81.00000000000000be:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-21T14:59:32.666 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:04.029946+0000", 2024-09-21T14:59:32.666 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 28.554755458999999, 2024-09-21T14:59:32.666 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.017686021999999999, 2024-09-21T14:59:32.666 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:32.666 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:32.666 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:32.666 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:32.667 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:32.667 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 37892 2024-09-21T14:59:32.667 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.667 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:32.667 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.667 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:32.667 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.029946+0000", 2024-09-21T14:59:32.667 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:32.667 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.667 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.667 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:32.667 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.029946+0000", 2024-09-21T14:59:32.667 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:32.667 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.667 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.668 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:32.668 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.029944+0000", 2024-09-21T14:59:32.668 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T14:59:32.668 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.668 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.668 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:32.668 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.029953+0000", 2024-09-21T14:59:32.668 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7589999999999992e-06 2024-09-21T14:59:32.668 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.668 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.668 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:32.668 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.029971+0000", 2024-09-21T14:59:32.668 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7795000000000002e-05 2024-09-21T14:59:32.668 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.669 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.669 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:32.669 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.029984+0000", 2024-09-21T14:59:32.669 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.27e-05 2024-09-21T14:59:32.669 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.669 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.669 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:32.669 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.030072+0000", 2024-09-21T14:59:32.669 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8082000000000001e-05 2024-09-21T14:59:32.669 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.669 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.669 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:32.669 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.030276+0000", 2024-09-21T14:59:32.669 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00020375699999999999 2024-09-21T14:59:32.670 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.670 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.670 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-21T14:59:32.670 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.031634+0000", 2024-09-21T14:59:32.670 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0013586049999999999 2024-09-21T14:59:32.670 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.670 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.670 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:59:32.670 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.037554+0000", 2024-09-21T14:59:32.670 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0059197449999999997 2024-09-21T14:59:32.670 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.671 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.671 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:32.671 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.047490+0000", 2024-09-21T14:59:32.671 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0099361680000000004 2024-09-21T14:59:32.671 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.671 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.671 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:59:32.671 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.047569+0000", 2024-09-21T14:59:32.671 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8298000000000001e-05 2024-09-21T14:59:32.671 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.671 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.671 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:32.671 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.047632+0000", 2024-09-21T14:59:32.672 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3006e-05 2024-09-21T14:59:32.672 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:32.672 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:32.672 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:32.672 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.672 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.672 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:63820 3.19 3.626bc079 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-21T14:59:32.672 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:27.241293+0000", 2024-09-21T14:59:32.672 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.3434083660000002, 2024-09-21T14:59:32.672 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.98723808199999996, 2024-09-21T14:59:32.672 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:32.672 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:32.672 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:32.673 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:32.673 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:32.673 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 63820 2024-09-21T14:59:32.673 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.673 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:32.673 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.673 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:32.673 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241293+0000", 2024-09-21T14:59:32.673 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:32.673 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.673 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.673 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:32.673 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241293+0000", 2024-09-21T14:59:32.673 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:32.674 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.674 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.674 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:32.674 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241291+0000", 2024-09-21T14:59:32.674 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T14:59:32.674 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.674 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.674 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:32.674 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241299+0000", 2024-09-21T14:59:32.674 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1529999999999997e-06 2024-09-21T14:59:32.674 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.674 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.674 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:32.675 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241317+0000", 2024-09-21T14:59:32.675 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7764000000000001e-05 2024-09-21T14:59:32.675 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.675 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.675 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:32.675 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241328+0000", 2024-09-21T14:59:32.675 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1245e-05 2024-09-21T14:59:32.675 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.675 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.675 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:32.675 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241407+0000", 2024-09-21T14:59:32.675 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8762000000000002e-05 2024-09-21T14:59:32.675 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.675 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.675 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T14:59:32.676 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241418+0000", 2024-09-21T14:59:32.676 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1695000000000001e-05 2024-09-21T14:59:32.676 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.676 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.676 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:32.676 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:28.228510+0000", 2024-09-21T14:59:32.676 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.98709166100000001 2024-09-21T14:59:32.676 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:32.676 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:32.676 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:32.676 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:28.228531+0000", 2024-09-21T14:59:32.676 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0696e-05 2024-09-21T14:59:32.676 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:32.676 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:32.677 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:32.677 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:32.677 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:32.677 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:59:32.678 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T14:59:32.679 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:59:32.677+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:59:32.779 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:59:32.777+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:59:32.790 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:59:32.790 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:59:32.790 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T14:59:32.790 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:59:32.800 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T14:59:32.880 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:59:32.875+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:59:32.911 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:59:32.911 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:59:32.911 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T14:59:32.911 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T14:59:32.911 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:59:32.921 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T14:59:32.980 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:59:32.975+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:59:33.033 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:59:33.033 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T14:59:33.033 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T14:59:33.033 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T14:59:33.033 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.033 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45834 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:33.033 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159133+0000", 2024-09-21T14:59:33.033 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.871438435000002, 2024-09-21T14:59:33.033 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068236884999999997, 2024-09-21T14:59:33.033 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:33.033 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:33.034 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:33.034 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:33.034 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:33.034 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45834 2024-09-21T14:59:33.034 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.034 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:33.034 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.034 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:33.034 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159133+0000", 2024-09-21T14:59:33.034 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:33.034 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.034 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.034 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:33.034 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159133+0000", 2024-09-21T14:59:33.035 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:33.035 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.035 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.035 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:33.035 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159133+0000", 2024-09-21T14:59:33.035 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:33.035 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.035 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.035 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:33.035 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159135+0000", 2024-09-21T14:59:33.035 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4500000000000001e-06 2024-09-21T14:59:33.035 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.035 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.035 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:33.035 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159138+0000", 2024-09-21T14:59:33.036 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2770000000000001e-06 2024-09-21T14:59:33.036 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.036 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.036 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:33.036 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159141+0000", 2024-09-21T14:59:33.036 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9210000000000001e-06 2024-09-21T14:59:33.036 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.036 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.036 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:33.036 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227361+0000", 2024-09-21T14:59:33.036 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068220154000000005 2024-09-21T14:59:33.036 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.036 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.036 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:33.036 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227370+0000", 2024-09-21T14:59:33.036 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3239999999999992e-06 2024-09-21T14:59:33.037 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:33.037 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:33.037 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:33.037 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.037 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.037 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45839 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:33.037 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159313+0000", 2024-09-21T14:59:33.037 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.871258469000001, 2024-09-21T14:59:33.037 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068238699, 2024-09-21T14:59:33.037 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:33.037 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:33.037 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:33.037 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:33.037 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:33.037 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45839 2024-09-21T14:59:33.038 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.038 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:33.038 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.038 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:33.038 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159313+0000", 2024-09-21T14:59:33.038 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:33.038 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.038 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.038 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:33.038 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159313+0000", 2024-09-21T14:59:33.038 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:33.038 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.038 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.038 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:33.038 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159313+0000", 2024-09-21T14:59:33.039 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:33.039 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.039 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.039 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:33.039 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159316+0000", 2024-09-21T14:59:33.039 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.033e-06 2024-09-21T14:59:33.039 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.039 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.039 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:33.039 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159322+0000", 2024-09-21T14:59:33.039 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1820000000000003e-06 2024-09-21T14:59:33.039 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.039 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.039 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:33.039 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159327+0000", 2024-09-21T14:59:33.040 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.062e-06 2024-09-21T14:59:33.040 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.040 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.040 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:33.040 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227544+0000", 2024-09-21T14:59:33.040 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068217142999999994 2024-09-21T14:59:33.040 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.040 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.040 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:33.040 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227552+0000", 2024-09-21T14:59:33.040 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6260000000000003e-06 2024-09-21T14:59:33.040 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:33.040 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:33.040 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:33.040 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.040 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.041 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45840 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:33.041 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159332+0000", 2024-09-21T14:59:33.041 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.871240172, 2024-09-21T14:59:33.041 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068262046000000007, 2024-09-21T14:59:33.041 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:33.041 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:33.041 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:33.041 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:33.041 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:33.041 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45840 2024-09-21T14:59:33.041 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.041 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:33.041 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.041 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:33.041 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159332+0000", 2024-09-21T14:59:33.042 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:33.042 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.042 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.042 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:33.042 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159332+0000", 2024-09-21T14:59:33.042 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:33.042 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.042 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.042 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:33.042 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159331+0000", 2024-09-21T14:59:33.042 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:33.042 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.042 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.042 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:33.042 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159351+0000", 2024-09-21T14:59:33.043 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0064999999999998e-05 2024-09-21T14:59:33.043 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.043 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.043 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:33.043 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159367+0000", 2024-09-21T14:59:33.043 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5308e-05 2024-09-21T14:59:33.043 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.043 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.043 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:33.043 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159373+0000", 2024-09-21T14:59:33.043 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1840000000000004e-06 2024-09-21T14:59:33.043 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.043 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.043 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:33.043 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227584+0000", 2024-09-21T14:59:33.044 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068211401000000005 2024-09-21T14:59:33.044 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.044 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.044 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:33.044 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227594+0000", 2024-09-21T14:59:33.044 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4120000000000007e-06 2024-09-21T14:59:33.044 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:33.044 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:33.044 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:33.044 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.044 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.044 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45841 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:33.044 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159379+0000", 2024-09-21T14:59:33.044 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.871193015999999, 2024-09-21T14:59:33.045 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068249039999999997, 2024-09-21T14:59:33.045 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:33.045 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:33.045 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:33.045 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:33.045 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:33.045 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45841 2024-09-21T14:59:33.045 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.045 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:33.045 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.045 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:33.045 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159379+0000", 2024-09-21T14:59:33.045 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:33.045 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.045 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.045 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:33.046 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159379+0000", 2024-09-21T14:59:33.046 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:33.046 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.046 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.046 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:33.046 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159378+0000", 2024-09-21T14:59:33.046 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:33.046 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.046 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.046 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:33.046 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159382+0000", 2024-09-21T14:59:33.046 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.641e-06 2024-09-21T14:59:33.046 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.046 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.046 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:33.047 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159421+0000", 2024-09-21T14:59:33.047 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9439e-05 2024-09-21T14:59:33.047 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.047 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.047 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:33.047 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159430+0000", 2024-09-21T14:59:33.047 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8429999999999999e-06 2024-09-21T14:59:33.047 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.047 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.047 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:33.047 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227617+0000", 2024-09-21T14:59:33.047 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068187084999999995 2024-09-21T14:59:33.047 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.047 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.047 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:33.047 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227628+0000", 2024-09-21T14:59:33.048 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0507e-05 2024-09-21T14:59:33.048 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:33.048 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:33.048 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:33.048 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.048 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.048 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45844 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:33.048 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159499+0000", 2024-09-21T14:59:33.048 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.87107263, 2024-09-21T14:59:33.048 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068255415, 2024-09-21T14:59:33.048 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:33.048 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:33.048 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:33.048 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:33.048 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:33.049 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45844 2024-09-21T14:59:33.049 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.049 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:33.049 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.049 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:33.049 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159499+0000", 2024-09-21T14:59:33.049 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:33.049 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.049 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.049 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:33.049 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159499+0000", 2024-09-21T14:59:33.049 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:33.049 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.049 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.049 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:33.049 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159499+0000", 2024-09-21T14:59:33.050 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:33.050 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.050 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.050 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:33.050 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159503+0000", 2024-09-21T14:59:33.050 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0999999999999997e-06 2024-09-21T14:59:33.050 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.050 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.050 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:33.050 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159512+0000", 2024-09-21T14:59:33.050 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5850000000000006e-06 2024-09-21T14:59:33.050 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.050 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.050 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:33.050 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159518+0000", 2024-09-21T14:59:33.051 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0909999999999998e-06 2024-09-21T14:59:33.051 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.051 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.051 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:33.051 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227743+0000", 2024-09-21T14:59:33.051 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068224536000000002 2024-09-21T14:59:33.051 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.051 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.051 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:33.051 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227755+0000", 2024-09-21T14:59:33.051 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1659e-05 2024-09-21T14:59:33.051 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:33.051 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:33.051 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:33.051 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.051 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.052 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45845 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:33.052 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159524+0000", 2024-09-21T14:59:33.052 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.871048269999999, 2024-09-21T14:59:33.052 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068267194000000003, 2024-09-21T14:59:33.052 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:33.052 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:33.052 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:33.052 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:33.052 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:33.052 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45845 2024-09-21T14:59:33.052 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.052 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:33.052 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.052 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:33.052 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159524+0000", 2024-09-21T14:59:33.053 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:33.053 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.053 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.053 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:33.053 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159524+0000", 2024-09-21T14:59:33.053 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:33.053 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.053 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.053 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:33.053 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159523+0000", 2024-09-21T14:59:33.053 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:33.053 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.053 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.053 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:33.053 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159526+0000", 2024-09-21T14:59:33.053 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1980000000000001e-06 2024-09-21T14:59:33.054 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.054 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.054 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:33.054 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159533+0000", 2024-09-21T14:59:33.054 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.348e-06 2024-09-21T14:59:33.054 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.054 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.054 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:33.054 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159546+0000", 2024-09-21T14:59:33.054 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3464e-05 2024-09-21T14:59:33.054 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.054 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.054 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:33.054 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227783+0000", 2024-09-21T14:59:33.055 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068236926000000003 2024-09-21T14:59:33.055 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.055 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.055 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:33.055 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227791+0000", 2024-09-21T14:59:33.055 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6669999999999996e-06 2024-09-21T14:59:33.055 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:33.055 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:33.055 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:33.055 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.055 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.055 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45846 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:33.055 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159584+0000", 2024-09-21T14:59:33.055 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.870987628000002, 2024-09-21T14:59:33.055 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068254767999999993, 2024-09-21T14:59:33.055 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:33.056 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:33.056 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:33.056 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:33.056 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:33.056 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45846 2024-09-21T14:59:33.056 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.056 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:33.056 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.056 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:33.056 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159584+0000", 2024-09-21T14:59:33.056 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:33.056 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.056 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.056 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:33.056 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159584+0000", 2024-09-21T14:59:33.056 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:33.057 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.057 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.057 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:33.057 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159583+0000", 2024-09-21T14:59:33.057 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:33.057 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.057 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.057 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:33.057 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159588+0000", 2024-09-21T14:59:33.057 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1330000000000001e-06 2024-09-21T14:59:33.057 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.057 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.057 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:33.057 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159596+0000", 2024-09-21T14:59:33.057 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5469999999999997e-06 2024-09-21T14:59:33.057 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.058 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.058 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:33.058 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159609+0000", 2024-09-21T14:59:33.058 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2785999999999999e-05 2024-09-21T14:59:33.058 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.058 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.058 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:33.058 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227830+0000", 2024-09-21T14:59:33.058 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068220879999999998 2024-09-21T14:59:33.058 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.058 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.058 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:33.058 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227839+0000", 2024-09-21T14:59:33.058 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1239999999999994e-06 2024-09-21T14:59:33.058 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:33.059 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:33.059 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:33.059 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.059 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.059 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45847 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:33.059 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159614+0000", 2024-09-21T14:59:33.059 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.870957552, 2024-09-21T14:59:33.059 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068260639999999997, 2024-09-21T14:59:33.059 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:33.059 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:33.059 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:33.059 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:33.059 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:33.059 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45847 2024-09-21T14:59:33.059 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.059 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:33.060 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.060 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:33.060 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159614+0000", 2024-09-21T14:59:33.060 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:33.060 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.060 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.060 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:33.060 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159614+0000", 2024-09-21T14:59:33.060 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:33.060 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.060 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.060 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:33.060 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159614+0000", 2024-09-21T14:59:33.060 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:33.060 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.060 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.061 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:33.061 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159617+0000", 2024-09-21T14:59:33.061 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2909999999999999e-06 2024-09-21T14:59:33.061 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.061 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.061 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:33.061 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159623+0000", 2024-09-21T14:59:33.061 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.511e-06 2024-09-21T14:59:33.061 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.061 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.061 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:33.061 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159631+0000", 2024-09-21T14:59:33.061 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.571e-06 2024-09-21T14:59:33.061 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.061 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.061 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:33.062 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227866+0000", 2024-09-21T14:59:33.062 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068235136000000002 2024-09-21T14:59:33.062 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.062 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.062 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:33.062 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227875+0000", 2024-09-21T14:59:33.062 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7199999999999995e-06 2024-09-21T14:59:33.062 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:33.062 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:33.062 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:33.062 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.062 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.062 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45848 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:33.062 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159636+0000", 2024-09-21T14:59:33.062 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.870935771999999, 2024-09-21T14:59:33.063 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068267283999999998, 2024-09-21T14:59:33.063 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:33.063 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:33.063 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:33.063 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:33.063 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:33.063 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45848 2024-09-21T14:59:33.063 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.063 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:33.063 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.063 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:33.063 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159636+0000", 2024-09-21T14:59:33.063 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:33.063 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.063 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.063 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:33.064 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159636+0000", 2024-09-21T14:59:33.064 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:33.064 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.064 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.064 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:33.064 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159636+0000", 2024-09-21T14:59:33.064 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:33.064 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.064 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.064 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:33.064 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159638+0000", 2024-09-21T14:59:33.064 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6560000000000001e-06 2024-09-21T14:59:33.064 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.064 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.065 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:33.065 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159644+0000", 2024-09-21T14:59:33.065 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0839999999999997e-06 2024-09-21T14:59:33.065 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.065 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.065 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:33.065 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159649+0000", 2024-09-21T14:59:33.065 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7500000000000003e-06 2024-09-21T14:59:33.065 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.065 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.065 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:33.065 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227892+0000", 2024-09-21T14:59:33.065 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068243315999999998 2024-09-21T14:59:33.065 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.065 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.066 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:33.066 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227903+0000", 2024-09-21T14:59:33.066 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0835999999999999e-05 2024-09-21T14:59:33.066 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:33.066 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:33.066 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:33.066 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.066 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.066 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45849 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:33.066 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159667+0000", 2024-09-21T14:59:33.066 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.870904958000001, 2024-09-21T14:59:33.066 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068291051000000005, 2024-09-21T14:59:33.066 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:33.066 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:33.066 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:33.067 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:33.067 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:33.067 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45849 2024-09-21T14:59:33.067 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.067 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:33.067 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.067 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:33.067 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159667+0000", 2024-09-21T14:59:33.067 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:33.067 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.067 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.067 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:33.067 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159667+0000", 2024-09-21T14:59:33.067 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:33.067 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.067 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.068 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:33.068 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159666+0000", 2024-09-21T14:59:33.068 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:33.068 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.068 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.068 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:33.068 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159670+0000", 2024-09-21T14:59:33.068 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.427e-06 2024-09-21T14:59:33.068 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.068 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.068 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:33.068 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159678+0000", 2024-09-21T14:59:33.068 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9869999999999999e-06 2024-09-21T14:59:33.068 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.068 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.068 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:33.069 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159688+0000", 2024-09-21T14:59:33.069 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0631999999999999e-05 2024-09-21T14:59:33.069 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.069 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.069 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:33.069 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227950+0000", 2024-09-21T14:59:33.069 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068261798999999998 2024-09-21T14:59:33.069 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.069 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.069 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:33.069 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227958+0000", 2024-09-21T14:59:33.069 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7510000000000008e-06 2024-09-21T14:59:33.069 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:33.069 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:33.069 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:33.069 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.070 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.070 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45850 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:33.070 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159695+0000", 2024-09-21T14:59:33.070 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.870877246999999, 2024-09-21T14:59:33.070 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068302999000000003, 2024-09-21T14:59:33.070 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:33.070 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:33.070 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:33.070 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:33.070 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:33.070 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45850 2024-09-21T14:59:33.070 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.070 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:33.070 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.070 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:33.071 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159695+0000", 2024-09-21T14:59:33.071 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:33.071 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.071 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.071 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:33.071 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159695+0000", 2024-09-21T14:59:33.071 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:33.071 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.071 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.071 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:33.071 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159694+0000", 2024-09-21T14:59:33.071 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:33.071 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.071 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.071 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:33.071 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159697+0000", 2024-09-21T14:59:33.072 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.258e-06 2024-09-21T14:59:33.072 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.072 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.072 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:33.072 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159704+0000", 2024-09-21T14:59:33.072 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7660000000000001e-06 2024-09-21T14:59:33.072 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.072 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.072 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:33.072 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159713+0000", 2024-09-21T14:59:33.072 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1779999999999992e-06 2024-09-21T14:59:33.072 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.072 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.072 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:33.072 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227989+0000", 2024-09-21T14:59:33.072 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068276015999999995 2024-09-21T14:59:33.073 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.073 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.073 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:33.073 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227998+0000", 2024-09-21T14:59:33.073 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2050000000000002e-06 2024-09-21T14:59:33.073 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:33.073 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:33.073 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:33.073 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.073 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.073 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45851 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:33.073 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159719+0000", 2024-09-21T14:59:33.073 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.870853113999999, 2024-09-21T14:59:33.073 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068320327, 2024-09-21T14:59:33.073 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:33.074 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:33.074 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:33.074 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:33.074 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:33.074 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45851 2024-09-21T14:59:33.074 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.074 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:33.074 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.074 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:33.074 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159719+0000", 2024-09-21T14:59:33.074 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:33.074 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.074 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.074 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:33.074 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159719+0000", 2024-09-21T14:59:33.075 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:33.075 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.075 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.075 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:33.075 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159718+0000", 2024-09-21T14:59:33.075 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:33.075 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.075 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.075 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:33.075 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159721+0000", 2024-09-21T14:59:33.075 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2169999999999999e-06 2024-09-21T14:59:33.075 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.075 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.075 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:33.075 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159730+0000", 2024-09-21T14:59:33.075 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5720000000000003e-06 2024-09-21T14:59:33.076 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.076 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.076 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:33.076 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159735+0000", 2024-09-21T14:59:33.076 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1100000000000002e-06 2024-09-21T14:59:33.076 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.076 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.076 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:33.076 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228029+0000", 2024-09-21T14:59:33.076 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068293624999999997 2024-09-21T14:59:33.076 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.076 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.076 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:33.076 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228039+0000", 2024-09-21T14:59:33.076 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0254e-05 2024-09-21T14:59:33.077 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:33.077 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:33.077 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:33.077 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.077 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.077 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45852 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:33.077 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159740+0000", 2024-09-21T14:59:33.077 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.870831945999999, 2024-09-21T14:59:33.077 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068340260999999999, 2024-09-21T14:59:33.077 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:33.077 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:33.077 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:33.077 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:33.077 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:33.077 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45852 2024-09-21T14:59:33.077 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.078 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:33.078 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.078 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:33.078 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159740+0000", 2024-09-21T14:59:33.078 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:33.078 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.078 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.078 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:33.078 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159740+0000", 2024-09-21T14:59:33.078 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:33.078 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.078 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.078 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:33.078 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159739+0000", 2024-09-21T14:59:33.078 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:33.078 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.079 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.079 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:33.079 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159757+0000", 2024-09-21T14:59:33.079 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7314999999999999e-05 2024-09-21T14:59:33.079 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.079 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.079 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:33.079 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159769+0000", 2024-09-21T14:59:33.079 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2376e-05 2024-09-21T14:59:33.079 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.079 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.079 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:33.079 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159775+0000", 2024-09-21T14:59:33.079 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2179999999999996e-06 2024-09-21T14:59:33.079 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.079 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.080 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:33.080 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228071+0000", 2024-09-21T14:59:33.080 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068295344999999993 2024-09-21T14:59:33.080 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.080 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.080 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:33.080 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228080+0000", 2024-09-21T14:59:33.080 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3959999999999995e-06 2024-09-21T14:59:33.080 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:33.080 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:33.080 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:33.080 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.080 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.080 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45853 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:33.080 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159781+0000", 2024-09-21T14:59:33.081 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.870790681999999, 2024-09-21T14:59:33.081 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068337093000000002, 2024-09-21T14:59:33.081 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:33.081 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:33.081 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:33.081 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:33.081 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:33.081 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45853 2024-09-21T14:59:33.081 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.081 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:33.081 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.081 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:33.081 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159781+0000", 2024-09-21T14:59:33.081 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:33.081 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.081 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.082 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:33.082 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159781+0000", 2024-09-21T14:59:33.082 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:33.082 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.082 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.082 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:33.082 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159781+0000", 2024-09-21T14:59:33.082 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:33.082 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.082 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.082 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:33.082 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159784+0000", 2024-09-21T14:59:33.082 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4759999999999998e-06 2024-09-21T14:59:33.082 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.082 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.083 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:33.083 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159792+0000", 2024-09-21T14:59:33.083 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0569999999999993e-06 2024-09-21T14:59:33.083 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.083 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.083 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:33.083 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159797+0000", 2024-09-21T14:59:33.083 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9640000000000002e-06 2024-09-21T14:59:33.083 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.083 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.083 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:33.083 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228111+0000", 2024-09-21T14:59:33.083 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068313737999999999 2024-09-21T14:59:33.083 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.083 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.083 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:33.084 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228118+0000", 2024-09-21T14:59:33.084 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.43e-06 2024-09-21T14:59:33.084 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:33.084 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:33.084 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:33.084 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.084 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.084 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45854 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:33.084 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159802+0000", 2024-09-21T14:59:33.084 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.870770031999999, 2024-09-21T14:59:33.084 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068358938999999994, 2024-09-21T14:59:33.084 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:33.084 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:33.085 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:33.085 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:33.085 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:33.085 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45854 2024-09-21T14:59:33.085 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.085 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:33.085 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.085 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:33.085 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159802+0000", 2024-09-21T14:59:33.085 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:33.085 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.085 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.085 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:33.085 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159802+0000", 2024-09-21T14:59:33.085 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:33.086 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.086 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.086 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:33.086 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159801+0000", 2024-09-21T14:59:33.086 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:33.086 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.086 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.086 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:33.086 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159804+0000", 2024-09-21T14:59:33.086 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0970000000000002e-06 2024-09-21T14:59:33.086 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.086 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.086 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:33.086 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159823+0000", 2024-09-21T14:59:33.086 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8895000000000001e-05 2024-09-21T14:59:33.086 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.087 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.087 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:33.087 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159837+0000", 2024-09-21T14:59:33.087 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3247999999999999e-05 2024-09-21T14:59:33.087 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.087 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.087 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:33.087 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228151+0000", 2024-09-21T14:59:33.087 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068314642999999994 2024-09-21T14:59:33.087 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.087 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.087 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:33.087 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228161+0000", 2024-09-21T14:59:33.087 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4250000000000004e-06 2024-09-21T14:59:33.087 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:33.088 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:33.088 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:33.088 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.088 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.088 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45983 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:33.088 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159848+0000", 2024-09-21T14:59:33.088 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.870723773000002, 2024-09-21T14:59:33.088 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068333801, 2024-09-21T14:59:33.088 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:33.088 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:33.088 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:33.088 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:33.088 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:33.088 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45983 2024-09-21T14:59:33.088 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.089 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:33.089 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.089 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:33.089 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159848+0000", 2024-09-21T14:59:33.089 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:33.089 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.089 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.089 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:33.089 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159848+0000", 2024-09-21T14:59:33.089 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:33.089 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.089 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.089 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:33.089 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159847+0000", 2024-09-21T14:59:33.089 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:33.089 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.090 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.090 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:33.090 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159856+0000", 2024-09-21T14:59:33.090 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4349999999999997e-06 2024-09-21T14:59:33.090 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.090 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.090 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:33.090 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159866+0000", 2024-09-21T14:59:33.090 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0076e-05 2024-09-21T14:59:33.090 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.090 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.090 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:33.090 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159872+0000", 2024-09-21T14:59:33.090 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8309999999999997e-06 2024-09-21T14:59:33.090 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.090 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.091 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:33.091 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228175+0000", 2024-09-21T14:59:33.091 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068302680000000005 2024-09-21T14:59:33.091 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.091 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.091 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:33.091 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228182+0000", 2024-09-21T14:59:33.091 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3150000000000003e-06 2024-09-21T14:59:33.091 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:33.091 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:33.091 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:33.091 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.091 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.091 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45984 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:33.091 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159877+0000", 2024-09-21T14:59:33.091 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.870694353000001, 2024-09-21T14:59:33.092 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068328448, 2024-09-21T14:59:33.092 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:33.092 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:33.092 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:33.092 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:33.092 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:33.092 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45984 2024-09-21T14:59:33.092 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.092 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:33.092 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.092 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:33.092 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159877+0000", 2024-09-21T14:59:33.092 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:33.092 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.092 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.092 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:33.093 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159877+0000", 2024-09-21T14:59:33.093 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:33.093 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.093 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.093 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:33.093 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159877+0000", 2024-09-21T14:59:33.093 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:33.093 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.093 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.093 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:33.093 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159880+0000", 2024-09-21T14:59:33.093 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7610000000000001e-06 2024-09-21T14:59:33.093 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.093 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.093 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:33.094 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159888+0000", 2024-09-21T14:59:33.094 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0330000000000003e-06 2024-09-21T14:59:33.094 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.094 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.094 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:33.094 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159893+0000", 2024-09-21T14:59:33.094 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3550000000000004e-06 2024-09-21T14:59:33.094 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.094 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.094 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:33.094 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228199+0000", 2024-09-21T14:59:33.094 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068306438999999997 2024-09-21T14:59:33.094 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.094 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.094 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:33.095 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228206+0000", 2024-09-21T14:59:33.095 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5599999999999999e-06 2024-09-21T14:59:33.095 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:33.095 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:33.095 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:33.095 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.095 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.095 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45985 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:33.095 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159897+0000", 2024-09-21T14:59:33.095 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.870674529999999, 2024-09-21T14:59:33.095 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068345664, 2024-09-21T14:59:33.095 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:33.095 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:33.095 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:33.095 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:33.095 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:33.096 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45985 2024-09-21T14:59:33.096 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.096 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:33.096 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.096 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:33.096 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159897+0000", 2024-09-21T14:59:33.096 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:33.096 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.096 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.096 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:33.096 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159897+0000", 2024-09-21T14:59:33.096 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:33.096 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.096 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.096 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:33.097 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159897+0000", 2024-09-21T14:59:33.097 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:33.097 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.097 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.097 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:33.097 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159900+0000", 2024-09-21T14:59:33.097 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.954e-06 2024-09-21T14:59:33.097 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.097 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.097 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:33.097 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159913+0000", 2024-09-21T14:59:33.097 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3106e-05 2024-09-21T14:59:33.097 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.098 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.098 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:33.098 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159918+0000", 2024-09-21T14:59:33.098 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5199999999999997e-06 2024-09-21T14:59:33.098 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.098 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.098 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:33.098 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228236+0000", 2024-09-21T14:59:33.098 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068317040999999995 2024-09-21T14:59:33.098 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.098 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.098 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:33.098 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228243+0000", 2024-09-21T14:59:33.098 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3880000000000002e-06 2024-09-21T14:59:33.098 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:33.099 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:33.099 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:33.099 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.099 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.099 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45986 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:33.099 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159924+0000", 2024-09-21T14:59:33.099 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.870648235000001, 2024-09-21T14:59:33.099 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068364362999999997, 2024-09-21T14:59:33.099 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:33.099 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:33.099 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:33.099 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:33.099 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:33.099 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45986 2024-09-21T14:59:33.099 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.099 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:33.100 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.100 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:33.100 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159924+0000", 2024-09-21T14:59:33.100 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:33.100 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.100 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.100 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:33.100 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159924+0000", 2024-09-21T14:59:33.100 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:33.100 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.100 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.100 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:33.100 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159923+0000", 2024-09-21T14:59:33.100 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:33.100 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.100 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.101 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:33.101 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159926+0000", 2024-09-21T14:59:33.101 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0510000000000001e-06 2024-09-21T14:59:33.101 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.101 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.101 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:33.101 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159932+0000", 2024-09-21T14:59:33.101 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4160000000000001e-06 2024-09-21T14:59:33.101 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.101 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.101 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:33.101 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159938+0000", 2024-09-21T14:59:33.101 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.383e-06 2024-09-21T14:59:33.101 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.101 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.102 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:33.102 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228275+0000", 2024-09-21T14:59:33.102 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068337007000000005 2024-09-21T14:59:33.102 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.102 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.102 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:33.102 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228288+0000", 2024-09-21T14:59:33.102 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3052e-05 2024-09-21T14:59:33.102 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:33.102 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:33.102 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:33.102 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.102 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.102 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45987 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:33.102 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159944+0000", 2024-09-21T14:59:33.102 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.870628011000001, 2024-09-21T14:59:33.103 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068387232000000006, 2024-09-21T14:59:33.103 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:33.103 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:33.103 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:33.103 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:33.103 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:33.103 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45987 2024-09-21T14:59:33.103 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.103 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:33.103 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.103 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:33.103 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159944+0000", 2024-09-21T14:59:33.103 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:33.103 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.103 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.103 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:33.104 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159944+0000", 2024-09-21T14:59:33.104 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:33.104 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.104 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.104 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:33.104 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159943+0000", 2024-09-21T14:59:33.104 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:33.104 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.104 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.104 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:33.104 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159946+0000", 2024-09-21T14:59:33.104 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6309999999999999e-06 2024-09-21T14:59:33.104 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.104 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.104 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:33.105 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159952+0000", 2024-09-21T14:59:33.105 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6540000000000004e-06 2024-09-21T14:59:33.105 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.105 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.105 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:33.105 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159957+0000", 2024-09-21T14:59:33.105 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9280000000000001e-06 2024-09-21T14:59:33.105 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.105 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.105 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:33.105 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228322+0000", 2024-09-21T14:59:33.105 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068365067000000002 2024-09-21T14:59:33.105 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.105 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.105 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:33.105 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228331+0000", 2024-09-21T14:59:33.106 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3270000000000007e-06 2024-09-21T14:59:33.106 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:33.106 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:33.106 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:33.106 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:33.106 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:33.106 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:59:33.107 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T14:59:33.107 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:59:33.103+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:59:33.208 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:59:33.203+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:59:33.264 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:59:33.264 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:59:33.264 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T14:59:33.264 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:59:33.274 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T14:59:33.309 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:59:33.303+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:59:33.319 INFO:teuthology.orchestra.run.smithi032.stdout:checking clone #14, image image_client.0-clone14 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent15 2024-09-21T14:59:33.376 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:59:33.377 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:59:33.377 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T14:59:33.377 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T14:59:33.377 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:59:33.387 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T14:59:33.430 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:59:33.427+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:59:33.511 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:59:33.507+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:59:33.535 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:59:33.535 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T14:59:33.535 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T14:59:33.535 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T14:59:33.535 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.535 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:10651 3.12 3:4e9d23af:::rbd_data.1136f5ed5dd6.0000000000000020:head [write 0~81920 [fadvise_sequential+fadvise_nocache],write 90112~8192 [fadvise_sequential+fadvise_nocache],write 122880~122880 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:33.535 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.292476+0000", 2024-09-21T14:59:33.536 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 54.238846494000001, 2024-09-21T14:59:33.536 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012765669, 2024-09-21T14:59:33.536 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:33.536 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:33.536 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:33.536 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:33.536 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:33.536 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 10651 2024-09-21T14:59:33.536 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.536 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:33.536 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.536 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:33.536 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292476+0000", 2024-09-21T14:59:33.536 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:33.537 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.537 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.537 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:33.537 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292476+0000", 2024-09-21T14:59:33.537 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:33.537 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.537 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.537 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:33.537 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292475+0000", 2024-09-21T14:59:33.537 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:33.537 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.537 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.537 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:33.537 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292696+0000", 2024-09-21T14:59:33.538 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000220201 2024-09-21T14:59:33.538 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.538 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.538 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:33.538 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292711+0000", 2024-09-21T14:59:33.538 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5803000000000001e-05 2024-09-21T14:59:33.538 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.538 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.538 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:33.538 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292721+0000", 2024-09-21T14:59:33.538 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6160000000000008e-06 2024-09-21T14:59:33.538 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.538 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.538 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:33.538 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292757+0000", 2024-09-21T14:59:33.539 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6211999999999998e-05 2024-09-21T14:59:33.539 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.539 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.539 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:33.539 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292878+0000", 2024-09-21T14:59:33.539 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000120306 2024-09-21T14:59:33.539 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.539 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.539 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 3", 2024-09-21T14:59:33.539 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.293019+0000", 2024-09-21T14:59:33.539 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014186400000000001 2024-09-21T14:59:33.539 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.540 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.540 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:59:33.540 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.294898+0000", 2024-09-21T14:59:33.540 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001878434 2024-09-21T14:59:33.540 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.540 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.540 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:33.540 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.305162+0000", 2024-09-21T14:59:33.540 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.01026373 2024-09-21T14:59:33.540 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.540 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.540 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:59:33.540 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.305209+0000", 2024-09-21T14:59:33.540 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7615999999999999e-05 2024-09-21T14:59:33.541 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.541 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.541 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:33.541 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.305242+0000", 2024-09-21T14:59:33.541 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2979999999999999e-05 2024-09-21T14:59:33.541 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:33.541 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:33.541 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:33.541 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.541 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.541 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:11285 3.12 3:4edf73d2:::rbd_data.1136f5ed5dd6.0000000000000039:head [write 90112~32768 [fadvise_sequential+fadvise_nocache],write 139264~24576 [fadvise_sequential+fadvise_nocache],write 180224~24576 [fadvise_sequential+fadvise_nocache],write 221184~40960 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:33.541 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.539470+0000", 2024-09-21T14:59:33.541 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 53.991852741999999, 2024-09-21T14:59:33.542 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0083360099999999996, 2024-09-21T14:59:33.542 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:33.542 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:33.542 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:33.542 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:33.542 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:33.542 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 11285 2024-09-21T14:59:33.542 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.542 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:33.542 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.542 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:33.542 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.539470+0000", 2024-09-21T14:59:33.542 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:33.542 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.542 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.543 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:33.543 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.539470+0000", 2024-09-21T14:59:33.543 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:33.543 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.543 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.543 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:33.543 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.539469+0000", 2024-09-21T14:59:33.543 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:33.543 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.543 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.543 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:33.543 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.539652+0000", 2024-09-21T14:59:33.543 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000183084 2024-09-21T14:59:33.543 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.543 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.544 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:33.544 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.539666+0000", 2024-09-21T14:59:33.544 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3947e-05 2024-09-21T14:59:33.544 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.544 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.544 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:33.544 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.539673+0000", 2024-09-21T14:59:33.544 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7309999999999996e-06 2024-09-21T14:59:33.544 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.544 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.544 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:33.544 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.539714+0000", 2024-09-21T14:59:33.544 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1539000000000003e-05 2024-09-21T14:59:33.545 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.545 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.545 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:33.545 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.539832+0000", 2024-09-21T14:59:33.545 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011753500000000001 2024-09-21T14:59:33.545 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.545 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.545 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 3", 2024-09-21T14:59:33.545 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.540005+0000", 2024-09-21T14:59:33.545 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000173031 2024-09-21T14:59:33.545 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.545 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.545 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:59:33.545 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.541606+0000", 2024-09-21T14:59:33.545 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001600899 2024-09-21T14:59:33.545 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.546 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.546 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:33.546 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.547709+0000", 2024-09-21T14:59:33.546 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0061027110000000002 2024-09-21T14:59:33.546 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.546 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.546 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:59:33.546 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.547767+0000", 2024-09-21T14:59:33.546 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8248999999999997e-05 2024-09-21T14:59:33.546 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.546 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.546 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:33.546 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.547806+0000", 2024-09-21T14:59:33.546 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9341000000000002e-05 2024-09-21T14:59:33.547 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:33.547 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:33.547 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:33.547 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.547 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.547 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12189 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 40960~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:33.547 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.618667+0000", 2024-09-21T14:59:33.547 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 53.912656292000001, 2024-09-21T14:59:33.547 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0083014549999999993, 2024-09-21T14:59:33.547 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:33.547 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:33.547 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:33.547 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:33.547 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:33.547 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12189 2024-09-21T14:59:33.548 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.548 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:33.548 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.548 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:33.548 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618667+0000", 2024-09-21T14:59:33.548 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:33.548 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.548 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.548 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:33.548 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618667+0000", 2024-09-21T14:59:33.548 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:33.548 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.548 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.548 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:33.549 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618666+0000", 2024-09-21T14:59:33.549 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:33.549 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.549 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.549 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:33.549 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618669+0000", 2024-09-21T14:59:33.549 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.554e-06 2024-09-21T14:59:33.549 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.549 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.549 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:33.549 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618675+0000", 2024-09-21T14:59:33.549 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2570000000000004e-06 2024-09-21T14:59:33.549 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.549 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.550 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:33.550 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618680+0000", 2024-09-21T14:59:33.550 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6539999999999998e-06 2024-09-21T14:59:33.550 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.550 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.550 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:33.550 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.626683+0000", 2024-09-21T14:59:33.550 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0080035880000000007 2024-09-21T14:59:33.550 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.550 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.550 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:33.550 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.626764+0000", 2024-09-21T14:59:33.550 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0672000000000003e-05 2024-09-21T14:59:33.550 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.551 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.551 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:33.551 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.626968+0000", 2024-09-21T14:59:33.551 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00020407499999999999 2024-09-21T14:59:33.551 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:33.551 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:33.551 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:33.551 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.551 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.551 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12193 3.4 3:25437d51:::rbd_data.11363a21d2be.0000000000000025:10 [read 425984~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:33.551 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.618755+0000", 2024-09-21T14:59:33.551 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 53.912568211, 2024-09-21T14:59:33.551 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086461379999999994, 2024-09-21T14:59:33.551 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:33.552 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:33.552 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:33.552 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:33.552 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:33.552 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12193 2024-09-21T14:59:33.552 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.552 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:33.552 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.552 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:33.552 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618755+0000", 2024-09-21T14:59:33.552 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:33.552 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.552 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.552 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:33.552 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618755+0000", 2024-09-21T14:59:33.552 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:33.553 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.553 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.553 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:33.553 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618754+0000", 2024-09-21T14:59:33.553 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:33.553 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.553 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.553 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:33.553 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618759+0000", 2024-09-21T14:59:33.553 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4020000000000004e-06 2024-09-21T14:59:33.553 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.553 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.553 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:33.553 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618767+0000", 2024-09-21T14:59:33.554 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7490000000000004e-06 2024-09-21T14:59:33.554 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.554 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.554 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:33.554 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618773+0000", 2024-09-21T14:59:33.554 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0939999999999996e-06 2024-09-21T14:59:33.554 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.554 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.554 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:33.554 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.627115+0000", 2024-09-21T14:59:33.554 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0083416840000000003 2024-09-21T14:59:33.554 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.554 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.555 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:33.555 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.627214+0000", 2024-09-21T14:59:33.555 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9042000000000005e-05 2024-09-21T14:59:33.555 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.555 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.555 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:33.555 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.627401+0000", 2024-09-21T14:59:33.555 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018677200000000001 2024-09-21T14:59:33.555 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:33.555 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:33.555 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:33.555 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.555 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.555 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12212 3.4 3:21b43b45:::rbd_data.11363a21d2be.000000000000002b:10 [read 237568~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:33.555 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.619922+0000", 2024-09-21T14:59:33.555 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 53.911401226999999, 2024-09-21T14:59:33.556 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.008324372, 2024-09-21T14:59:33.556 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:33.556 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:33.556 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:33.556 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:33.556 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:33.556 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12212 2024-09-21T14:59:33.556 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.556 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:33.556 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.556 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:33.556 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619922+0000", 2024-09-21T14:59:33.556 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:33.556 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.556 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.557 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:33.557 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619922+0000", 2024-09-21T14:59:33.557 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:33.557 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.557 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.557 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:33.557 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619921+0000", 2024-09-21T14:59:33.557 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:33.557 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.557 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.557 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:33.557 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619924+0000", 2024-09-21T14:59:33.557 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.999e-06 2024-09-21T14:59:33.557 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.558 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.558 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:33.558 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619934+0000", 2024-09-21T14:59:33.558 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.8120000000000002e-06 2024-09-21T14:59:33.558 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.558 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.558 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:33.558 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619939+0000", 2024-09-21T14:59:33.558 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7729999999999999e-06 2024-09-21T14:59:33.558 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.558 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.558 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:33.558 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628005+0000", 2024-09-21T14:59:33.558 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0080663330000000002 2024-09-21T14:59:33.558 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.559 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.559 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:33.559 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628073+0000", 2024-09-21T14:59:33.559 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7479999999999998e-05 2024-09-21T14:59:33.559 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.559 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.559 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:33.559 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628246+0000", 2024-09-21T14:59:33.559 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017337400000000001 2024-09-21T14:59:33.559 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:33.559 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:33.559 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:33.559 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.559 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.560 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12213 3.4 3:21b43b45:::rbd_data.11363a21d2be.000000000000002b:10 [read 270336~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:33.560 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.619945+0000", 2024-09-21T14:59:33.560 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 53.911377944999998, 2024-09-21T14:59:33.560 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086195049999999995, 2024-09-21T14:59:33.560 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:33.560 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:33.560 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:33.560 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:33.560 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:33.560 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12213 2024-09-21T14:59:33.560 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.560 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:33.560 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.560 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:33.560 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619945+0000", 2024-09-21T14:59:33.561 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:33.561 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.561 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.561 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:33.561 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619945+0000", 2024-09-21T14:59:33.561 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:33.561 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.561 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.561 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:33.561 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619945+0000", 2024-09-21T14:59:33.561 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:33.561 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.561 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.561 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:33.561 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619947+0000", 2024-09-21T14:59:33.562 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5780000000000001e-06 2024-09-21T14:59:33.562 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.562 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.562 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:33.562 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619957+0000", 2024-09-21T14:59:33.562 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3500000000000003e-06 2024-09-21T14:59:33.562 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.562 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.562 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:33.562 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619964+0000", 2024-09-21T14:59:33.562 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.199e-06 2024-09-21T14:59:33.562 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.562 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.562 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:33.562 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628285+0000", 2024-09-21T14:59:33.563 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0083210939999999994 2024-09-21T14:59:33.563 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.563 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.563 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:33.563 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628361+0000", 2024-09-21T14:59:33.563 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5873999999999999e-05 2024-09-21T14:59:33.563 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.563 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.563 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:33.563 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628565+0000", 2024-09-21T14:59:33.563 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000203799 2024-09-21T14:59:33.563 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:33.563 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:33.564 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:33.564 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.564 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.564 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12214 3.4 3:21b43b45:::rbd_data.11363a21d2be.000000000000002b:10 [read 376832~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:33.564 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.619969+0000", 2024-09-21T14:59:33.564 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 53.911353943999998, 2024-09-21T14:59:33.564 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0090292089999999998, 2024-09-21T14:59:33.564 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:33.564 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:33.564 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:33.564 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:33.564 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:33.564 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12214 2024-09-21T14:59:33.565 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.565 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:33.565 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.565 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:33.565 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619969+0000", 2024-09-21T14:59:33.565 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:33.565 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.565 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.565 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:33.565 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619969+0000", 2024-09-21T14:59:33.565 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:33.565 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.565 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.565 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:33.566 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619969+0000", 2024-09-21T14:59:33.566 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:33.566 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.566 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.566 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:33.566 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619971+0000", 2024-09-21T14:59:33.566 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1830000000000001e-06 2024-09-21T14:59:33.566 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.566 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.566 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:33.566 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619976+0000", 2024-09-21T14:59:33.566 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7129999999999996e-06 2024-09-21T14:59:33.566 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.566 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.566 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:33.567 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619981+0000", 2024-09-21T14:59:33.567 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2460000000000003e-06 2024-09-21T14:59:33.567 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.567 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.567 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:33.567 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628602+0000", 2024-09-21T14:59:33.567 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086209519999999994 2024-09-21T14:59:33.567 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.567 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.567 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:33.567 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628734+0000", 2024-09-21T14:59:33.567 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000132242 2024-09-21T14:59:33.567 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.567 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.568 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:33.568 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628998+0000", 2024-09-21T14:59:33.568 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00026419799999999999 2024-09-21T14:59:33.568 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:33.568 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:33.568 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:33.568 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.568 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.568 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12215 3.4 3:21b43b45:::rbd_data.11363a21d2be.000000000000002b:10 [read 409600~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:33.568 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.619985+0000", 2024-09-21T14:59:33.568 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 53.911337877999998, 2024-09-21T14:59:33.568 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0095457319999999995, 2024-09-21T14:59:33.568 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:33.568 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:33.568 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:33.569 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:33.569 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:33.569 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12215 2024-09-21T14:59:33.569 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.569 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:33.569 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.569 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:33.569 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619985+0000", 2024-09-21T14:59:33.569 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:33.569 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.569 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.569 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:33.569 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619985+0000", 2024-09-21T14:59:33.569 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:33.570 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.570 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.570 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:33.570 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619985+0000", 2024-09-21T14:59:33.570 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:33.570 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.570 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.570 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:33.570 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619987+0000", 2024-09-21T14:59:33.570 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6129999999999998e-06 2024-09-21T14:59:33.570 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.570 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.570 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:33.570 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619993+0000", 2024-09-21T14:59:33.570 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0820000000000004e-06 2024-09-21T14:59:33.571 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.571 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.571 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:33.571 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619998+0000", 2024-09-21T14:59:33.571 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7759999999999997e-06 2024-09-21T14:59:33.571 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.571 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.571 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:33.571 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629022+0000", 2024-09-21T14:59:33.571 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0090237489999999993 2024-09-21T14:59:33.571 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.571 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.571 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:33.571 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629098+0000", 2024-09-21T14:59:33.571 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5735999999999995e-05 2024-09-21T14:59:33.572 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.572 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.572 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:33.572 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629531+0000", 2024-09-21T14:59:33.572 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00043310100000000003 2024-09-21T14:59:33.572 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:33.572 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:33.572 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:33.572 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.572 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.572 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12216 3.4 3:21b43b45:::rbd_data.11363a21d2be.000000000000002b:10 [read 516096~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:33.572 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.620003+0000", 2024-09-21T14:59:33.572 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 53.911319882000001, 2024-09-21T14:59:33.572 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0098850169999999994, 2024-09-21T14:59:33.573 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:33.573 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:33.573 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:33.573 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:33.573 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:33.573 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12216 2024-09-21T14:59:33.573 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.573 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:33.573 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.573 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:33.573 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620003+0000", 2024-09-21T14:59:33.573 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:33.573 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.573 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.573 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:33.574 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620003+0000", 2024-09-21T14:59:33.574 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:33.574 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.574 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.574 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:33.574 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620003+0000", 2024-09-21T14:59:33.574 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:33.574 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.574 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.574 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:33.574 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620006+0000", 2024-09-21T14:59:33.574 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4350000000000001e-06 2024-09-21T14:59:33.574 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.574 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.575 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:33.575 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620013+0000", 2024-09-21T14:59:33.575 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4219999999999997e-06 2024-09-21T14:59:33.575 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.575 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.575 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:33.575 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620017+0000", 2024-09-21T14:59:33.575 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.865e-06 2024-09-21T14:59:33.575 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.575 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.575 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:33.575 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629581+0000", 2024-09-21T14:59:33.575 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0095631989999999997 2024-09-21T14:59:33.575 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.575 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.576 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:33.576 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629691+0000", 2024-09-21T14:59:33.576 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000110586 2024-09-21T14:59:33.576 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.576 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.576 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:33.576 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629888+0000", 2024-09-21T14:59:33.576 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019691199999999999 2024-09-21T14:59:33.576 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:33.576 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:33.576 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:33.576 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.576 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.576 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12222 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 49152~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:33.577 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.620027+0000", 2024-09-21T14:59:33.577 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 53.911295535999997, 2024-09-21T14:59:33.577 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010228131999999999, 2024-09-21T14:59:33.577 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:33.577 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:33.577 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:33.577 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:33.577 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:33.577 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12222 2024-09-21T14:59:33.577 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.577 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:33.577 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.577 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:33.577 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620027+0000", 2024-09-21T14:59:33.578 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:33.578 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.578 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.578 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:33.578 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620027+0000", 2024-09-21T14:59:33.578 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:33.578 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.578 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.578 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:33.578 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620027+0000", 2024-09-21T14:59:33.578 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:33.578 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.578 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.578 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:33.579 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620030+0000", 2024-09-21T14:59:33.579 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0400000000000001e-06 2024-09-21T14:59:33.579 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.579 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.579 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:33.579 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620037+0000", 2024-09-21T14:59:33.579 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8220000000000001e-06 2024-09-21T14:59:33.579 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.579 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.579 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:33.579 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620042+0000", 2024-09-21T14:59:33.579 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.843e-06 2024-09-21T14:59:33.579 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.579 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.580 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:33.580 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629922+0000", 2024-09-21T14:59:33.580 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0098805100000000003 2024-09-21T14:59:33.580 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.580 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.580 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:33.580 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.630020+0000", 2024-09-21T14:59:33.580 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7596999999999994e-05 2024-09-21T14:59:33.580 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.580 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.580 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:33.580 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.630256+0000", 2024-09-21T14:59:33.580 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00023573299999999999 2024-09-21T14:59:33.580 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:33.581 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:33.581 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:33.581 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.581 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.581 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12253 3.4 3:21b43b45:::rbd_data.11363a21d2be.000000000000002b:10 [read 417792~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:33.581 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.624578+0000", 2024-09-21T14:59:33.581 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 53.906745168, 2024-09-21T14:59:33.581 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0085112079999999993, 2024-09-21T14:59:33.581 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:33.581 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:33.581 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:33.581 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:33.581 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:33.581 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12253 2024-09-21T14:59:33.582 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.582 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:33.582 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.582 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:33.582 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624578+0000", 2024-09-21T14:59:33.582 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:33.582 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.582 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.582 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:33.582 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624578+0000", 2024-09-21T14:59:33.582 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:33.582 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.582 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.582 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:33.583 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624578+0000", 2024-09-21T14:59:33.583 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:33.583 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.583 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.583 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:33.583 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624579+0000", 2024-09-21T14:59:33.583 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.688e-06 2024-09-21T14:59:33.583 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.583 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.583 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:33.583 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624582+0000", 2024-09-21T14:59:33.583 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2260000000000001e-06 2024-09-21T14:59:33.583 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.583 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.584 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:33.584 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624585+0000", 2024-09-21T14:59:33.584 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4820000000000001e-06 2024-09-21T14:59:33.584 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.584 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.584 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:33.584 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.632580+0000", 2024-09-21T14:59:33.584 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0079948959999999996 2024-09-21T14:59:33.584 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.584 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.584 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:33.584 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.632683+0000", 2024-09-21T14:59:33.584 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010308 2024-09-21T14:59:33.584 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.585 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.585 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:33.585 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.633089+0000", 2024-09-21T14:59:33.585 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00040612599999999998 2024-09-21T14:59:33.585 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:33.585 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:33.585 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:33.585 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.585 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.585 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12260 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 57344~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:33.585 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.624587+0000", 2024-09-21T14:59:33.585 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 53.906735681000001, 2024-09-21T14:59:33.585 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0088282069999999994, 2024-09-21T14:59:33.586 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:33.586 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:33.586 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:33.586 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:33.586 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:33.586 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12260 2024-09-21T14:59:33.586 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.586 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:33.586 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.586 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:33.586 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624587+0000", 2024-09-21T14:59:33.586 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:33.586 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.586 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.587 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:33.587 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624587+0000", 2024-09-21T14:59:33.587 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:33.587 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.587 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.587 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:33.587 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624587+0000", 2024-09-21T14:59:33.587 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-21T14:59:33.587 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.587 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.587 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:33.587 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624589+0000", 2024-09-21T14:59:33.587 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4339999999999999e-06 2024-09-21T14:59:33.587 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.588 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.588 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:33.588 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624593+0000", 2024-09-21T14:59:33.588 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5859999999999998e-06 2024-09-21T14:59:33.588 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.588 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.588 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:33.588 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624596+0000", 2024-09-21T14:59:33.588 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5830000000000001e-06 2024-09-21T14:59:33.588 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.588 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.588 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:33.588 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.633132+0000", 2024-09-21T14:59:33.588 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.008536666 2024-09-21T14:59:33.589 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.589 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.589 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:33.589 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.633216+0000", 2024-09-21T14:59:33.589 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3387999999999995e-05 2024-09-21T14:59:33.589 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.589 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.589 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:33.589 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.633415+0000", 2024-09-21T14:59:33.589 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019976799999999999 2024-09-21T14:59:33.589 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:33.589 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:33.589 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:33.589 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.590 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.590 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12516 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 245760~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:33.590 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645194+0000", 2024-09-21T14:59:33.590 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 53.886129144000002, 2024-09-21T14:59:33.590 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0084284990000000008, 2024-09-21T14:59:33.590 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:33.590 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:33.590 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:33.590 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:33.590 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:33.590 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12516 2024-09-21T14:59:33.590 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.590 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:33.591 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.591 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:33.591 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645194+0000", 2024-09-21T14:59:33.591 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:33.591 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.591 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.591 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:33.591 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645194+0000", 2024-09-21T14:59:33.591 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:33.591 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.591 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.591 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:33.591 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645193+0000", 2024-09-21T14:59:33.592 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:33.592 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.592 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.592 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:33.592 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645196+0000", 2024-09-21T14:59:33.592 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1030000000000002e-06 2024-09-21T14:59:33.592 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.592 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.592 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:33.592 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645205+0000", 2024-09-21T14:59:33.592 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6200000000000005e-06 2024-09-21T14:59:33.592 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.592 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.592 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:33.593 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645210+0000", 2024-09-21T14:59:33.593 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6280000000000004e-06 2024-09-21T14:59:33.593 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.593 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.593 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:33.593 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.653305+0000", 2024-09-21T14:59:33.593 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0080957879999999996 2024-09-21T14:59:33.593 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.593 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.593 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:33.593 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.653389+0000", 2024-09-21T14:59:33.593 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3242e-05 2024-09-21T14:59:33.594 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.594 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.594 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:33.594 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.653622+0000", 2024-09-21T14:59:33.594 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00023361599999999999 2024-09-21T14:59:33.594 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:33.594 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:33.594 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:33.594 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.594 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.594 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12517 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 278528~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:33.594 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645214+0000", 2024-09-21T14:59:33.594 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 53.886109030999997, 2024-09-21T14:59:33.595 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086832899999999998, 2024-09-21T14:59:33.595 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:33.595 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:33.595 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:33.595 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:33.595 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:33.595 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12517 2024-09-21T14:59:33.595 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.595 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:33.595 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.595 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:33.595 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645214+0000", 2024-09-21T14:59:33.596 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:33.596 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.596 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.596 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:33.596 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645214+0000", 2024-09-21T14:59:33.596 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:33.596 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.596 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.596 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:33.596 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645214+0000", 2024-09-21T14:59:33.596 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:33.596 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.596 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.596 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:33.596 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645216+0000", 2024-09-21T14:59:33.597 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5579999999999999e-06 2024-09-21T14:59:33.597 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.597 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.597 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:33.597 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645224+0000", 2024-09-21T14:59:33.597 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2439999999999993e-06 2024-09-21T14:59:33.597 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.597 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.597 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:33.597 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645231+0000", 2024-09-21T14:59:33.597 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1859999999999997e-06 2024-09-21T14:59:33.597 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.597 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.597 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:33.597 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.653651+0000", 2024-09-21T14:59:33.598 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0084204220000000003 2024-09-21T14:59:33.598 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.598 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.598 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:33.598 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.653726+0000", 2024-09-21T14:59:33.598 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5356000000000002e-05 2024-09-21T14:59:33.598 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.598 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.598 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:33.598 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.653897+0000", 2024-09-21T14:59:33.598 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000170848 2024-09-21T14:59:33.598 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:33.598 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:33.599 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:33.599 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.599 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.599 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12518 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 385024~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:33.599 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645235+0000", 2024-09-21T14:59:33.599 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 53.886087805000003, 2024-09-21T14:59:33.599 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0089841020000000008, 2024-09-21T14:59:33.599 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:33.599 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:33.599 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:33.599 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:33.599 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:33.599 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12518 2024-09-21T14:59:33.599 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.600 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:33.600 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.600 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:33.600 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645235+0000", 2024-09-21T14:59:33.600 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:33.600 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.600 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.600 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:33.600 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645235+0000", 2024-09-21T14:59:33.600 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:33.600 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.600 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.600 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:33.600 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645235+0000", 2024-09-21T14:59:33.601 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:33.601 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.601 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.601 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:33.601 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645238+0000", 2024-09-21T14:59:33.601 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.869e-06 2024-09-21T14:59:33.601 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.601 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.601 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:33.601 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645244+0000", 2024-09-21T14:59:33.601 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8930000000000002e-06 2024-09-21T14:59:33.601 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.601 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.601 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:33.601 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645249+0000", 2024-09-21T14:59:33.601 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2859999999999999e-06 2024-09-21T14:59:33.602 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.602 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.602 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:33.602 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.653930+0000", 2024-09-21T14:59:33.602 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086815320000000005 2024-09-21T14:59:33.602 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.602 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.602 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:33.602 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654028+0000", 2024-09-21T14:59:33.602 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7578000000000007e-05 2024-09-21T14:59:33.602 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.602 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.602 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:33.602 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654219+0000", 2024-09-21T14:59:33.603 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019132099999999999 2024-09-21T14:59:33.603 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:33.603 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:33.603 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:33.603 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.603 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.603 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12538 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 147456~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:33.603 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645253+0000", 2024-09-21T14:59:33.603 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 53.886070181000001, 2024-09-21T14:59:33.603 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0092095360000000008, 2024-09-21T14:59:33.603 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:33.603 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:33.603 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:33.603 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:33.604 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:33.604 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12538 2024-09-21T14:59:33.604 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.604 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:33.604 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.604 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:33.604 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645253+0000", 2024-09-21T14:59:33.604 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:33.604 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.604 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.604 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:33.604 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645253+0000", 2024-09-21T14:59:33.604 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:33.604 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.605 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.605 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:33.605 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645252+0000", 2024-09-21T14:59:33.605 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:33.605 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.605 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.605 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:33.605 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645255+0000", 2024-09-21T14:59:33.605 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2290000000000002e-06 2024-09-21T14:59:33.605 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.605 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.605 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:33.605 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645267+0000", 2024-09-21T14:59:33.605 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2631e-05 2024-09-21T14:59:33.606 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.606 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.606 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:33.606 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645278+0000", 2024-09-21T14:59:33.606 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0847e-05 2024-09-21T14:59:33.606 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.606 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.606 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:33.606 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654251+0000", 2024-09-21T14:59:33.606 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0089731340000000007 2024-09-21T14:59:33.606 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.606 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.606 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:33.606 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654296+0000", 2024-09-21T14:59:33.606 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4353e-05 2024-09-21T14:59:33.607 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.607 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.607 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:33.607 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654462+0000", 2024-09-21T14:59:33.607 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016668499999999999 2024-09-21T14:59:33.607 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:33.607 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:33.607 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:33.607 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.607 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.607 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12539 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 253952~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:33.607 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645284+0000", 2024-09-21T14:59:33.607 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 53.88603896, 2024-09-21T14:59:33.607 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0094524029999999998, 2024-09-21T14:59:33.607 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:33.607 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:33.608 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:33.608 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:33.608 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:33.608 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12539 2024-09-21T14:59:33.608 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.608 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:33.608 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.608 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:33.608 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645284+0000", 2024-09-21T14:59:33.608 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:33.608 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.608 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.608 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:33.608 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645284+0000", 2024-09-21T14:59:33.609 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:33.609 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.609 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.609 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:33.609 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645284+0000", 2024-09-21T14:59:33.609 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:33.609 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.609 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.609 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:33.609 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645286+0000", 2024-09-21T14:59:33.609 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7159999999999999e-06 2024-09-21T14:59:33.609 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.609 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.609 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:33.610 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645292+0000", 2024-09-21T14:59:33.610 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7400000000000001e-06 2024-09-21T14:59:33.610 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.610 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.610 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:33.610 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645297+0000", 2024-09-21T14:59:33.610 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9019999999999998e-06 2024-09-21T14:59:33.610 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.610 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.610 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:33.610 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654490+0000", 2024-09-21T14:59:33.610 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0091932360000000005 2024-09-21T14:59:33.610 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.610 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.611 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:33.611 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654565+0000", 2024-09-21T14:59:33.611 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4657e-05 2024-09-21T14:59:33.611 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.611 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.611 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:33.611 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654736+0000", 2024-09-21T14:59:33.611 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017156199999999999 2024-09-21T14:59:33.611 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:33.611 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:33.611 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:33.611 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.611 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.611 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12540 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 286720~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:33.611 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645302+0000", 2024-09-21T14:59:33.612 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 53.886020690999999, 2024-09-21T14:59:33.612 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0096603090000000006, 2024-09-21T14:59:33.612 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:33.612 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:33.612 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:33.612 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:33.612 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:33.612 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12540 2024-09-21T14:59:33.612 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.612 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:33.612 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.612 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:33.612 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645302+0000", 2024-09-21T14:59:33.612 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:33.612 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.613 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.613 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:33.613 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645302+0000", 2024-09-21T14:59:33.613 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:33.613 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.613 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.613 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:33.613 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645302+0000", 2024-09-21T14:59:33.613 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:33.613 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.613 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.613 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:33.613 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645305+0000", 2024-09-21T14:59:33.614 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7329999999999999e-06 2024-09-21T14:59:33.614 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.614 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.614 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:33.614 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645316+0000", 2024-09-21T14:59:33.614 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1832e-05 2024-09-21T14:59:33.614 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.614 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.614 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:33.614 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645322+0000", 2024-09-21T14:59:33.614 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.169e-06 2024-09-21T14:59:33.614 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.614 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.614 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:33.615 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654778+0000", 2024-09-21T14:59:33.615 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0094559090000000002 2024-09-21T14:59:33.615 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.615 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.615 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:33.615 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654828+0000", 2024-09-21T14:59:33.615 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0461999999999999e-05 2024-09-21T14:59:33.615 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.615 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.615 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:33.615 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654963+0000", 2024-09-21T14:59:33.615 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000134583 2024-09-21T14:59:33.615 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:33.616 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:33.616 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:33.616 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.616 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.616 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12561 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:33.616 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645742+0000", 2024-09-21T14:59:33.616 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 53.885581424999998, 2024-09-21T14:59:33.616 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0093583810000000007, 2024-09-21T14:59:33.616 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:33.616 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:33.616 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:33.616 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:33.616 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:33.616 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12561 2024-09-21T14:59:33.617 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.617 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:33.617 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.617 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:33.617 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645742+0000", 2024-09-21T14:59:33.617 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:33.617 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.617 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.617 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:33.617 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645742+0000", 2024-09-21T14:59:33.617 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:33.617 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.617 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.617 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:33.617 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645741+0000", 2024-09-21T14:59:33.618 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:33.618 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.618 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.618 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:33.618 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645746+0000", 2024-09-21T14:59:33.618 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8379999999999998e-06 2024-09-21T14:59:33.618 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.618 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.618 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:33.618 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645757+0000", 2024-09-21T14:59:33.618 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0705e-05 2024-09-21T14:59:33.618 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.618 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.618 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:33.619 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645765+0000", 2024-09-21T14:59:33.619 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6729999999999992e-06 2024-09-21T14:59:33.619 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.619 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.619 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:33.619 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654993+0000", 2024-09-21T14:59:33.619 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0092287480000000002 2024-09-21T14:59:33.619 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.619 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.619 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:33.619 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.655050+0000", 2024-09-21T14:59:33.619 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6394999999999998e-05 2024-09-21T14:59:33.619 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.620 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.620 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:33.620 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.655100+0000", 2024-09-21T14:59:33.620 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0049000000000001e-05 2024-09-21T14:59:33.620 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:33.620 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:33.620 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:33.620 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.620 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.620 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:15431 3.12 3:4bc7d206:::rbd_data.1136f5ed5dd6.0000000000000051:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:59:33.620 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:41.673445+0000", 2024-09-21T14:59:33.620 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 51.857878091000003, 2024-09-21T14:59:33.620 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086103610000000004, 2024-09-21T14:59:33.620 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:33.621 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:33.621 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:33.621 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:33.621 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:33.621 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 15431 2024-09-21T14:59:33.621 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.621 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:33.621 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.621 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:33.621 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.673445+0000", 2024-09-21T14:59:33.621 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:33.621 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.621 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.621 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:33.622 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.673445+0000", 2024-09-21T14:59:33.622 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:33.622 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.622 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.622 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:33.622 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.673444+0000", 2024-09-21T14:59:33.622 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:59:33.622 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.622 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.622 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:33.622 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.673451+0000", 2024-09-21T14:59:33.622 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4739999999999999e-06 2024-09-21T14:59:33.622 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.622 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.622 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:33.623 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.673472+0000", 2024-09-21T14:59:33.623 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0687e-05 2024-09-21T14:59:33.623 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.623 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.623 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:33.623 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.673481+0000", 2024-09-21T14:59:33.623 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3349999999999996e-06 2024-09-21T14:59:33.623 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.623 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.623 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:33.623 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.673565+0000", 2024-09-21T14:59:33.623 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4011000000000006e-05 2024-09-21T14:59:33.623 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.623 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.623 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:33.624 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.673650+0000", 2024-09-21T14:59:33.624 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4949000000000003e-05 2024-09-21T14:59:33.624 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.624 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.624 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 3", 2024-09-21T14:59:33.624 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.675014+0000", 2024-09-21T14:59:33.624 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0013636620000000001 2024-09-21T14:59:33.624 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.624 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.624 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:33.624 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.681866+0000", 2024-09-21T14:59:33.624 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0068525640000000002 2024-09-21T14:59:33.624 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.624 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.624 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:59:33.625 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.681885+0000", 2024-09-21T14:59:33.625 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8519999999999999e-05 2024-09-21T14:59:33.625 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.625 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.625 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:59:33.625 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.681961+0000", 2024-09-21T14:59:33.625 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6248000000000006e-05 2024-09-21T14:59:33.625 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:33.625 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:33.625 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:33.625 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.682055+0000", 2024-09-21T14:59:33.625 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4184000000000005e-05 2024-09-21T14:59:33.625 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:33.625 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:33.626 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:33.626 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:33.626 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:33.626 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:59:33.626 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T14:59:33.628 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T14:59:33.628 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:59:33.625+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:59:33.713 INFO:teuthology.orchestra.run.smithi032.stdout:1841 read 0xb82530f thru 0xb82bba0 (0x6892 bytes) 2024-09-21T14:59:33.714 INFO:teuthology.orchestra.run.smithi032.stdout:1842 write 0x4dc5b2d thru 0x4dc8a3c (0x2f10 bytes) 2024-09-21T14:59:33.724 INFO:teuthology.orchestra.run.smithi032.stdout:1843 read 0x3f84c54 thru 0x3f92fa8 (0xe355 bytes) 2024-09-21T14:59:33.724 INFO:teuthology.orchestra.run.smithi032.stdout:1844 write 0x5f83668 thru 0x5f88de9 (0x5782 bytes) 2024-09-21T14:59:33.728 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:59:33.723+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:59:33.731 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:33.731 INFO:teuthology.orchestra.run.smithi032.stdout:1846 writesame 0x5c6c81a thru 0x5c7716e data_size 0x1(0xa955 bytes) 2024-09-21T14:59:33.742 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:59:33.742 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:59:33.743 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T14:59:33.743 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:59:33.753 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T14:59:33.759 INFO:teuthology.orchestra.run.smithi032.stdout:1848 clone 17 order 19 su 8192 sc 8 2024-09-21T14:59:33.775 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T14:59:33.796 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T14:59:33.816 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T14:59:33.829 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:59:33.823+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:59:33.838 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T14:59:33.857 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:59:33.857 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:59:33.857 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T14:59:33.857 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T14:59:33.857 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:59:33.861 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-21T14:59:33.867 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T14:59:33.883 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T14:59:33.903 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T14:59:33.926 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T14:59:33.929 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:59:33.929 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:59:33.929 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:59:33.929 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:59:33.929 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:59:33.929 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:59:33.929 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:59:33.929 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:59:33.929 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:59:33.929+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:59:34.019 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:34.019 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:34.029 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T14:59:34.030 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:59:34.029+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:59:34.131 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:59:34.129+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:59:34.150 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:34.150 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:34.160 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T14:59:34.187 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T14:59:34.231 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:59:34.227+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:59:34.276 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:34.276 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:34.287 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T14:59:34.332 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:59:34.327+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:59:34.391 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:34.391 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:34.401 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T14:59:34.433 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:59:34.427+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:59:34.501 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:34.501 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:34.511 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T14:59:34.533 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:59:34.531+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:59:34.616 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:34.616 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:34.625 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T14:59:34.634 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:59:34.633+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:59:34.735 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:34.735 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:34.735 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:59:34.731+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:59:34.744 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T14:59:34.836 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:59:34.835+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:59:34.901 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:34.902 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:34.911 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T14:59:34.936 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:59:34.931+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:59:35.021 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:35.021 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:35.032 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T14:59:35.037 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:59:35.031+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:59:35.138 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:59:35.135+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:59:35.141 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:35.141 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:35.151 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T14:59:35.239 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:59:35.235+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:59:35.266 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:35.266 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:35.277 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T14:59:35.278 INFO:teuthology.orchestra.run.smithi032.stderr:noscrub is unset 2024-09-21T14:59:35.340 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:59:35.335+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:59:35.411 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:35.411 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:35.426 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T14:59:35.440 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:59:35.435+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:59:35.541 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:59:35.535+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:59:35.580 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:35.580 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:35.594 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T14:59:35.642 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:59:35.641+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:59:35.703 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:35.703 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:35.713 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T14:59:35.743 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:59:35.741+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:59:35.823 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:35.823 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:35.834 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T14:59:35.843 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:59:35.841+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:59:35.941 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T14:59:35.944 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:59:35.939+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:59:35.951 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:35.951 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:35.963 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T14:59:36.044 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:59:36.045+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:59:36.080 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:36.080 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:36.091 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T14:59:36.107 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T14:59:36.131 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T14:59:36.145 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:59:36.139+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:59:36.156 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T14:59:36.180 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T14:59:36.201 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-21T14:59:36.221 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:36.221 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:36.231 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T14:59:36.235 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T14:59:36.246 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:59:36.245+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:59:36.257 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T14:59:36.279 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T14:59:36.281 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:59:36.282 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:59:36.282 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:59:36.282 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:59:36.282 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:59:36.282 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:59:36.282 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:59:36.282 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:59:36.347 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:59:36.345+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:59:36.391 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:59:36.392 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T14:59:36.392 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T14:59:36.392 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T14:59:36.392 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.392 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45834 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:36.392 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159133+0000", 2024-09-21T14:59:36.392 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 26.229733675999999, 2024-09-21T14:59:36.392 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068236884999999997, 2024-09-21T14:59:36.392 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:36.392 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:36.392 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:36.393 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:36.393 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:36.393 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45834 2024-09-21T14:59:36.393 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.393 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:36.393 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.393 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:36.393 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159133+0000", 2024-09-21T14:59:36.393 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:36.393 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.393 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.393 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:36.393 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159133+0000", 2024-09-21T14:59:36.394 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:36.394 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.394 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.394 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:36.394 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159133+0000", 2024-09-21T14:59:36.394 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:36.394 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.394 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.394 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:36.394 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159135+0000", 2024-09-21T14:59:36.394 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4500000000000001e-06 2024-09-21T14:59:36.395 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.395 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.395 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:36.395 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159138+0000", 2024-09-21T14:59:36.395 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2770000000000001e-06 2024-09-21T14:59:36.395 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.395 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.395 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:36.395 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159141+0000", 2024-09-21T14:59:36.395 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9210000000000001e-06 2024-09-21T14:59:36.395 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.395 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.396 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:36.396 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227361+0000", 2024-09-21T14:59:36.396 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068220154000000005 2024-09-21T14:59:36.396 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.396 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.396 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:36.396 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227370+0000", 2024-09-21T14:59:36.396 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3239999999999992e-06 2024-09-21T14:59:36.396 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:36.396 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:36.396 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:36.396 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.396 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.397 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45839 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:36.397 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159313+0000", 2024-09-21T14:59:36.397 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 26.229553710000001, 2024-09-21T14:59:36.397 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068238699, 2024-09-21T14:59:36.397 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:36.397 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:36.397 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:36.397 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:36.397 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:36.397 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45839 2024-09-21T14:59:36.397 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.397 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:36.398 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.398 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:36.398 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159313+0000", 2024-09-21T14:59:36.398 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:36.398 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.398 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.398 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:36.398 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159313+0000", 2024-09-21T14:59:36.398 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:36.398 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.398 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.398 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:36.399 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159313+0000", 2024-09-21T14:59:36.399 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:36.399 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.399 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.399 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:36.399 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159316+0000", 2024-09-21T14:59:36.399 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.033e-06 2024-09-21T14:59:36.399 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.399 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.399 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:36.399 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159322+0000", 2024-09-21T14:59:36.399 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1820000000000003e-06 2024-09-21T14:59:36.400 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.400 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.400 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:36.400 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159327+0000", 2024-09-21T14:59:36.400 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.062e-06 2024-09-21T14:59:36.400 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.400 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.400 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:36.400 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227544+0000", 2024-09-21T14:59:36.400 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068217142999999994 2024-09-21T14:59:36.400 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.400 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.400 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:36.401 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227552+0000", 2024-09-21T14:59:36.401 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6260000000000003e-06 2024-09-21T14:59:36.401 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:36.401 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:36.401 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:36.401 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.401 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.401 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45840 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:36.401 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159332+0000", 2024-09-21T14:59:36.401 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 26.229535413000001, 2024-09-21T14:59:36.401 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068262046000000007, 2024-09-21T14:59:36.401 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:36.402 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:36.402 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:36.402 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:36.402 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:36.402 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45840 2024-09-21T14:59:36.402 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.402 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:36.402 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.402 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:36.402 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159332+0000", 2024-09-21T14:59:36.402 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:36.402 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.403 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.403 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:36.403 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159332+0000", 2024-09-21T14:59:36.403 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:36.403 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.403 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.403 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:36.403 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159331+0000", 2024-09-21T14:59:36.403 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:36.403 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.403 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.403 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:36.403 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159351+0000", 2024-09-21T14:59:36.404 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0064999999999998e-05 2024-09-21T14:59:36.404 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.404 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.404 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:36.404 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159367+0000", 2024-09-21T14:59:36.404 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5308e-05 2024-09-21T14:59:36.404 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.404 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.404 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:36.404 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159373+0000", 2024-09-21T14:59:36.404 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1840000000000004e-06 2024-09-21T14:59:36.404 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.405 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.405 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:36.405 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227584+0000", 2024-09-21T14:59:36.405 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068211401000000005 2024-09-21T14:59:36.405 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.405 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.405 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:36.405 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227594+0000", 2024-09-21T14:59:36.405 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4120000000000007e-06 2024-09-21T14:59:36.405 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:36.405 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:36.405 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:36.405 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.405 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.406 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45841 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:36.406 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159379+0000", 2024-09-21T14:59:36.406 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 26.229488257, 2024-09-21T14:59:36.406 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068249039999999997, 2024-09-21T14:59:36.406 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:36.406 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:36.406 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:36.406 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:36.406 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:36.406 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45841 2024-09-21T14:59:36.406 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.406 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:36.407 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.407 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:36.407 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159379+0000", 2024-09-21T14:59:36.407 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:36.407 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.407 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.407 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:36.407 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159379+0000", 2024-09-21T14:59:36.407 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:36.407 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.407 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.407 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:36.407 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159378+0000", 2024-09-21T14:59:36.408 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:36.408 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.408 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.408 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:36.408 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159382+0000", 2024-09-21T14:59:36.408 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.641e-06 2024-09-21T14:59:36.408 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.408 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.408 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:36.408 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159421+0000", 2024-09-21T14:59:36.408 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9439e-05 2024-09-21T14:59:36.408 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.408 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.409 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:36.409 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159430+0000", 2024-09-21T14:59:36.409 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8429999999999999e-06 2024-09-21T14:59:36.409 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.409 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.409 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:36.409 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227617+0000", 2024-09-21T14:59:36.409 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068187084999999995 2024-09-21T14:59:36.409 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.409 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.409 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:36.409 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227628+0000", 2024-09-21T14:59:36.409 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0507e-05 2024-09-21T14:59:36.409 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:36.410 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:36.410 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:36.410 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.410 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.410 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45844 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:36.410 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159499+0000", 2024-09-21T14:59:36.410 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 26.229367871000001, 2024-09-21T14:59:36.410 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068255415, 2024-09-21T14:59:36.410 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:36.410 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:36.410 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:36.410 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:36.411 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:36.411 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45844 2024-09-21T14:59:36.411 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.411 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:36.411 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.411 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:36.411 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159499+0000", 2024-09-21T14:59:36.411 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:36.411 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.411 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.411 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:36.411 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159499+0000", 2024-09-21T14:59:36.411 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:36.411 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.412 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.412 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:36.412 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159499+0000", 2024-09-21T14:59:36.412 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:36.412 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.412 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.412 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:36.412 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159503+0000", 2024-09-21T14:59:36.412 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0999999999999997e-06 2024-09-21T14:59:36.412 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.412 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.412 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:36.412 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159512+0000", 2024-09-21T14:59:36.413 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5850000000000006e-06 2024-09-21T14:59:36.413 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.413 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.413 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:36.413 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159518+0000", 2024-09-21T14:59:36.413 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0909999999999998e-06 2024-09-21T14:59:36.413 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.413 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.413 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:36.413 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227743+0000", 2024-09-21T14:59:36.413 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068224536000000002 2024-09-21T14:59:36.413 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.413 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.413 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:36.414 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227755+0000", 2024-09-21T14:59:36.414 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1659e-05 2024-09-21T14:59:36.414 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:36.414 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:36.414 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:36.414 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.414 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.414 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45845 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:36.414 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159524+0000", 2024-09-21T14:59:36.414 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 26.229343511, 2024-09-21T14:59:36.414 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068267194000000003, 2024-09-21T14:59:36.414 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:36.415 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:36.415 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:36.415 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:36.415 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:36.415 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45845 2024-09-21T14:59:36.415 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.415 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:36.415 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.415 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:36.415 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159524+0000", 2024-09-21T14:59:36.415 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:36.415 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.415 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.415 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:36.416 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159524+0000", 2024-09-21T14:59:36.416 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:36.416 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.416 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.416 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:36.416 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159523+0000", 2024-09-21T14:59:36.416 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:36.416 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.416 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.416 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:36.416 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159526+0000", 2024-09-21T14:59:36.416 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1980000000000001e-06 2024-09-21T14:59:36.416 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.417 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.417 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:36.417 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159533+0000", 2024-09-21T14:59:36.417 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.348e-06 2024-09-21T14:59:36.417 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.417 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.417 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:36.417 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159546+0000", 2024-09-21T14:59:36.417 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3464e-05 2024-09-21T14:59:36.417 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.417 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.417 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:36.417 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227783+0000", 2024-09-21T14:59:36.417 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068236926000000003 2024-09-21T14:59:36.418 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.418 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.418 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:36.418 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227791+0000", 2024-09-21T14:59:36.418 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6669999999999996e-06 2024-09-21T14:59:36.418 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:36.418 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:36.418 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:36.418 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.418 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.418 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45846 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:36.418 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159584+0000", 2024-09-21T14:59:36.418 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 26.229282868999999, 2024-09-21T14:59:36.419 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068254767999999993, 2024-09-21T14:59:36.419 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:36.419 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:36.419 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:36.419 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:36.419 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:36.419 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45846 2024-09-21T14:59:36.419 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.419 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:36.419 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.419 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:36.419 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159584+0000", 2024-09-21T14:59:36.420 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:36.420 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.420 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.420 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:36.420 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159584+0000", 2024-09-21T14:59:36.420 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:36.420 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.420 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.420 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:36.420 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159583+0000", 2024-09-21T14:59:36.420 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:36.420 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.421 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.421 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:36.421 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159588+0000", 2024-09-21T14:59:36.421 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1330000000000001e-06 2024-09-21T14:59:36.421 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.421 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.421 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:36.421 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159596+0000", 2024-09-21T14:59:36.421 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5469999999999997e-06 2024-09-21T14:59:36.421 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.421 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.421 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:36.421 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159609+0000", 2024-09-21T14:59:36.421 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2785999999999999e-05 2024-09-21T14:59:36.422 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.422 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.422 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:36.422 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227830+0000", 2024-09-21T14:59:36.422 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068220879999999998 2024-09-21T14:59:36.422 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.422 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.422 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:36.422 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227839+0000", 2024-09-21T14:59:36.422 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1239999999999994e-06 2024-09-21T14:59:36.422 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:36.422 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:36.423 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:36.423 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.423 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.423 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45847 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:36.423 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159614+0000", 2024-09-21T14:59:36.423 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 26.229252793000001, 2024-09-21T14:59:36.423 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068260639999999997, 2024-09-21T14:59:36.423 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:36.423 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:36.423 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:36.423 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:36.423 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:36.423 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45847 2024-09-21T14:59:36.424 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.424 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:36.424 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.424 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:36.424 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159614+0000", 2024-09-21T14:59:36.424 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:36.424 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.424 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.424 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:36.424 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159614+0000", 2024-09-21T14:59:36.424 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:36.424 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.424 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.425 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:36.425 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159614+0000", 2024-09-21T14:59:36.425 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:36.425 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.425 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.425 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:36.425 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159617+0000", 2024-09-21T14:59:36.425 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2909999999999999e-06 2024-09-21T14:59:36.425 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.425 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.425 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:36.425 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159623+0000", 2024-09-21T14:59:36.425 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.511e-06 2024-09-21T14:59:36.426 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.426 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.426 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:36.426 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159631+0000", 2024-09-21T14:59:36.426 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.571e-06 2024-09-21T14:59:36.426 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.426 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.426 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:36.426 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227866+0000", 2024-09-21T14:59:36.426 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068235136000000002 2024-09-21T14:59:36.426 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.426 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.426 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:36.427 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227875+0000", 2024-09-21T14:59:36.427 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7199999999999995e-06 2024-09-21T14:59:36.427 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:36.427 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:36.427 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:36.427 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.427 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.427 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45848 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:36.427 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159636+0000", 2024-09-21T14:59:36.427 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 26.229231013, 2024-09-21T14:59:36.427 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068267283999999998, 2024-09-21T14:59:36.427 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:36.427 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:36.428 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:36.428 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:36.428 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:36.428 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45848 2024-09-21T14:59:36.428 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.428 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:36.428 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.428 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:36.428 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159636+0000", 2024-09-21T14:59:36.428 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:36.428 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.428 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.428 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:36.428 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159636+0000", 2024-09-21T14:59:36.429 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:36.429 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.429 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.429 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:36.429 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159636+0000", 2024-09-21T14:59:36.429 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:36.429 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.429 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.429 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:36.429 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159638+0000", 2024-09-21T14:59:36.429 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6560000000000001e-06 2024-09-21T14:59:36.429 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.430 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.430 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:36.430 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159644+0000", 2024-09-21T14:59:36.430 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0839999999999997e-06 2024-09-21T14:59:36.430 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.430 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.430 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:36.430 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159649+0000", 2024-09-21T14:59:36.430 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7500000000000003e-06 2024-09-21T14:59:36.430 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.430 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.430 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:36.431 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227892+0000", 2024-09-21T14:59:36.431 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068243315999999998 2024-09-21T14:59:36.431 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.431 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.431 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:36.431 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227903+0000", 2024-09-21T14:59:36.431 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0835999999999999e-05 2024-09-21T14:59:36.431 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:36.431 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:36.431 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:36.431 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.431 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.431 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45849 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:36.432 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159667+0000", 2024-09-21T14:59:36.432 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 26.229200199000001, 2024-09-21T14:59:36.432 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068291051000000005, 2024-09-21T14:59:36.432 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:36.432 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:36.432 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:36.432 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:36.432 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:36.432 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45849 2024-09-21T14:59:36.432 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.432 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:36.432 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.432 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:36.432 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159667+0000", 2024-09-21T14:59:36.433 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:36.433 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.433 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.433 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:36.433 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159667+0000", 2024-09-21T14:59:36.433 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:36.433 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.433 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.433 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:36.433 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159666+0000", 2024-09-21T14:59:36.433 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:36.433 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.433 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.434 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:36.434 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159670+0000", 2024-09-21T14:59:36.434 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.427e-06 2024-09-21T14:59:36.434 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.434 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.434 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:36.434 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159678+0000", 2024-09-21T14:59:36.434 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9869999999999999e-06 2024-09-21T14:59:36.434 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.434 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.434 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:36.434 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159688+0000", 2024-09-21T14:59:36.434 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0631999999999999e-05 2024-09-21T14:59:36.435 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.435 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.435 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:36.435 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227950+0000", 2024-09-21T14:59:36.435 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068261798999999998 2024-09-21T14:59:36.435 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.435 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.435 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:36.435 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227958+0000", 2024-09-21T14:59:36.435 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7510000000000008e-06 2024-09-21T14:59:36.435 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:36.435 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:36.435 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:36.435 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.436 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.436 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45850 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:36.436 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159695+0000", 2024-09-21T14:59:36.436 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 26.229172488, 2024-09-21T14:59:36.436 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068302999000000003, 2024-09-21T14:59:36.436 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:36.436 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:36.436 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:36.436 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:36.436 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:36.436 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45850 2024-09-21T14:59:36.436 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.436 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:36.437 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.437 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:36.437 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159695+0000", 2024-09-21T14:59:36.437 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:36.437 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.437 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.437 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:36.437 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159695+0000", 2024-09-21T14:59:36.437 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:36.437 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.437 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.437 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:36.437 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159694+0000", 2024-09-21T14:59:36.438 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:36.438 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.438 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.438 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:36.438 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159697+0000", 2024-09-21T14:59:36.438 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.258e-06 2024-09-21T14:59:36.438 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.438 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.438 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:36.438 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159704+0000", 2024-09-21T14:59:36.438 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7660000000000001e-06 2024-09-21T14:59:36.438 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.438 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.439 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:36.439 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159713+0000", 2024-09-21T14:59:36.439 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1779999999999992e-06 2024-09-21T14:59:36.439 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.439 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.439 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:36.439 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227989+0000", 2024-09-21T14:59:36.439 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068276015999999995 2024-09-21T14:59:36.439 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.439 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.439 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:36.439 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227998+0000", 2024-09-21T14:59:36.439 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2050000000000002e-06 2024-09-21T14:59:36.440 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:36.440 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:36.440 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:36.440 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.440 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.440 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45851 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:36.440 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159719+0000", 2024-09-21T14:59:36.440 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 26.229148355, 2024-09-21T14:59:36.440 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068320327, 2024-09-21T14:59:36.440 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:36.440 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:36.440 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:36.440 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:36.440 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:36.441 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45851 2024-09-21T14:59:36.441 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.441 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:36.441 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.441 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:36.441 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159719+0000", 2024-09-21T14:59:36.441 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:36.441 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.441 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.441 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:36.441 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159719+0000", 2024-09-21T14:59:36.442 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:36.442 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.442 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.442 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:36.442 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159718+0000", 2024-09-21T14:59:36.442 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:36.442 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.442 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.442 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:36.442 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159721+0000", 2024-09-21T14:59:36.442 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2169999999999999e-06 2024-09-21T14:59:36.442 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.443 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.443 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:36.443 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159730+0000", 2024-09-21T14:59:36.443 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5720000000000003e-06 2024-09-21T14:59:36.443 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.443 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.443 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:36.443 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159735+0000", 2024-09-21T14:59:36.443 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1100000000000002e-06 2024-09-21T14:59:36.443 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.443 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.443 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:36.443 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228029+0000", 2024-09-21T14:59:36.444 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068293624999999997 2024-09-21T14:59:36.444 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.444 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.444 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:36.444 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228039+0000", 2024-09-21T14:59:36.444 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0254e-05 2024-09-21T14:59:36.444 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:36.444 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:36.444 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:36.444 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.444 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.444 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45852 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:36.444 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159740+0000", 2024-09-21T14:59:36.445 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 26.229127187, 2024-09-21T14:59:36.445 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068340260999999999, 2024-09-21T14:59:36.445 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:36.445 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:36.445 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:36.445 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:36.445 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:36.445 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45852 2024-09-21T14:59:36.445 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.445 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:36.445 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.445 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:36.445 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159740+0000", 2024-09-21T14:59:36.446 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:36.446 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.446 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.446 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:36.446 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159740+0000", 2024-09-21T14:59:36.446 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:36.446 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.446 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.446 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:36.446 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159739+0000", 2024-09-21T14:59:36.446 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:36.446 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.447 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.447 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:36.447 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159757+0000", 2024-09-21T14:59:36.447 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7314999999999999e-05 2024-09-21T14:59:36.447 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.447 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.447 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:36.447 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159769+0000", 2024-09-21T14:59:36.447 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2376e-05 2024-09-21T14:59:36.447 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.447 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.447 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:36.447 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159775+0000", 2024-09-21T14:59:36.448 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2179999999999996e-06 2024-09-21T14:59:36.448 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.448 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.448 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:36.448 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228071+0000", 2024-09-21T14:59:36.448 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068295344999999993 2024-09-21T14:59:36.448 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.448 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.448 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:36.448 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228080+0000", 2024-09-21T14:59:36.448 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3959999999999995e-06 2024-09-21T14:59:36.448 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:36.448 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:36.449 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:36.449 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.449 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.449 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45853 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:36.449 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159781+0000", 2024-09-21T14:59:36.449 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 26.229085923, 2024-09-21T14:59:36.449 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068337093000000002, 2024-09-21T14:59:36.449 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:36.449 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:36.449 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:36.449 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:36.449 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:36.449 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45853 2024-09-21T14:59:36.450 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.450 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:36.450 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.450 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:36.450 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159781+0000", 2024-09-21T14:59:36.450 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:36.450 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.450 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.450 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:36.450 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159781+0000", 2024-09-21T14:59:36.450 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:36.450 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.451 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.451 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:36.451 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159781+0000", 2024-09-21T14:59:36.451 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:36.451 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.451 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.451 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:36.451 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159784+0000", 2024-09-21T14:59:36.451 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4759999999999998e-06 2024-09-21T14:59:36.451 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.451 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.451 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:36.452 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159792+0000", 2024-09-21T14:59:36.452 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0569999999999993e-06 2024-09-21T14:59:36.452 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.452 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.452 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:36.452 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159797+0000", 2024-09-21T14:59:36.452 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9640000000000002e-06 2024-09-21T14:59:36.452 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.452 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.452 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:36.452 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228111+0000", 2024-09-21T14:59:36.452 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068313737999999999 2024-09-21T14:59:36.453 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.453 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.453 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:36.453 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228118+0000", 2024-09-21T14:59:36.453 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.43e-06 2024-09-21T14:59:36.453 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:36.453 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:36.453 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:36.453 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.453 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.453 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45854 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:36.453 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159802+0000", 2024-09-21T14:59:36.454 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 26.229065273, 2024-09-21T14:59:36.454 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068358938999999994, 2024-09-21T14:59:36.454 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:36.454 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:36.454 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:36.454 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:36.454 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:36.454 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45854 2024-09-21T14:59:36.454 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.454 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:36.454 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.455 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:36.455 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159802+0000", 2024-09-21T14:59:36.455 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:36.455 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.455 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.455 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:36.455 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159802+0000", 2024-09-21T14:59:36.455 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:36.455 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.455 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.455 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:36.456 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159801+0000", 2024-09-21T14:59:36.456 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:36.456 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.456 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.456 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:36.456 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159804+0000", 2024-09-21T14:59:36.456 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0970000000000002e-06 2024-09-21T14:59:36.456 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.456 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.456 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:36.456 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159823+0000", 2024-09-21T14:59:36.456 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8895000000000001e-05 2024-09-21T14:59:36.457 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.457 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.457 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:36.457 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159837+0000", 2024-09-21T14:59:36.457 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3247999999999999e-05 2024-09-21T14:59:36.457 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.457 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.457 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:36.457 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228151+0000", 2024-09-21T14:59:36.457 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068314642999999994 2024-09-21T14:59:36.457 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.457 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.457 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:36.458 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228161+0000", 2024-09-21T14:59:36.458 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4250000000000004e-06 2024-09-21T14:59:36.458 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:36.458 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:36.458 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:36.458 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.458 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.458 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45983 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:36.458 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159848+0000", 2024-09-21T14:59:36.458 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 26.229019013999999, 2024-09-21T14:59:36.459 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068333801, 2024-09-21T14:59:36.459 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:36.459 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:36.459 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:36.459 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:36.459 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:36.459 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45983 2024-09-21T14:59:36.459 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.459 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:36.459 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.459 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:36.459 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159848+0000", 2024-09-21T14:59:36.459 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:36.460 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.460 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.460 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:36.460 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159848+0000", 2024-09-21T14:59:36.460 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:36.460 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.460 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.460 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:36.460 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159847+0000", 2024-09-21T14:59:36.460 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:36.460 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.460 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.460 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:36.460 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159856+0000", 2024-09-21T14:59:36.461 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4349999999999997e-06 2024-09-21T14:59:36.461 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.461 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.461 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:36.461 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159866+0000", 2024-09-21T14:59:36.461 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0076e-05 2024-09-21T14:59:36.461 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.461 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.461 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:36.461 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159872+0000", 2024-09-21T14:59:36.461 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8309999999999997e-06 2024-09-21T14:59:36.462 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.462 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.462 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:36.462 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228175+0000", 2024-09-21T14:59:36.462 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068302680000000005 2024-09-21T14:59:36.462 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.462 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.462 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:36.462 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228182+0000", 2024-09-21T14:59:36.462 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3150000000000003e-06 2024-09-21T14:59:36.462 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:36.462 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:36.463 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:36.463 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.463 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.463 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45984 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:36.463 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159877+0000", 2024-09-21T14:59:36.463 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 26.228989594000002, 2024-09-21T14:59:36.463 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068328448, 2024-09-21T14:59:36.463 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:36.463 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:36.463 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:36.463 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:36.463 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:36.464 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45984 2024-09-21T14:59:36.464 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.464 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:36.464 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.464 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:36.464 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159877+0000", 2024-09-21T14:59:36.464 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:36.464 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.464 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.464 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:36.464 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159877+0000", 2024-09-21T14:59:36.465 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:36.465 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.465 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.465 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:36.465 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159877+0000", 2024-09-21T14:59:36.465 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:36.465 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.465 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.465 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:36.465 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159880+0000", 2024-09-21T14:59:36.465 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7610000000000001e-06 2024-09-21T14:59:36.465 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.465 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.466 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:36.466 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159888+0000", 2024-09-21T14:59:36.466 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0330000000000003e-06 2024-09-21T14:59:36.466 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.466 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.466 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:36.466 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159893+0000", 2024-09-21T14:59:36.466 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3550000000000004e-06 2024-09-21T14:59:36.466 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.466 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.466 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:36.466 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228199+0000", 2024-09-21T14:59:36.466 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068306438999999997 2024-09-21T14:59:36.467 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.467 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.467 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:36.467 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228206+0000", 2024-09-21T14:59:36.467 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5599999999999999e-06 2024-09-21T14:59:36.467 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:36.467 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:36.467 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:36.467 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.467 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.467 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45985 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:36.468 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159897+0000", 2024-09-21T14:59:36.468 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 26.228969770999999, 2024-09-21T14:59:36.468 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068345664, 2024-09-21T14:59:36.468 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:36.468 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:36.468 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:36.468 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:36.468 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:36.468 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45985 2024-09-21T14:59:36.468 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.468 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:36.468 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.468 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:36.469 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159897+0000", 2024-09-21T14:59:36.469 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:36.469 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.469 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.469 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:36.469 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159897+0000", 2024-09-21T14:59:36.469 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:36.469 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.469 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.469 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:36.469 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159897+0000", 2024-09-21T14:59:36.469 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:36.470 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.470 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.470 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:36.470 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159900+0000", 2024-09-21T14:59:36.470 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.954e-06 2024-09-21T14:59:36.470 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.470 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.470 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:36.470 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159913+0000", 2024-09-21T14:59:36.470 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3106e-05 2024-09-21T14:59:36.470 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.470 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.471 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:36.471 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159918+0000", 2024-09-21T14:59:36.471 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5199999999999997e-06 2024-09-21T14:59:36.471 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.471 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.471 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:36.471 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228236+0000", 2024-09-21T14:59:36.471 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068317040999999995 2024-09-21T14:59:36.471 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.471 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.471 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:36.471 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228243+0000", 2024-09-21T14:59:36.472 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3880000000000002e-06 2024-09-21T14:59:36.472 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:36.472 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:36.472 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:36.472 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.472 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.472 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45986 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:36.472 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159924+0000", 2024-09-21T14:59:36.472 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 26.228943476000001, 2024-09-21T14:59:36.472 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068364362999999997, 2024-09-21T14:59:36.472 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:36.472 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:36.473 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:36.473 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:36.473 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:36.473 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45986 2024-09-21T14:59:36.473 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.473 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:36.473 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.473 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:36.473 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159924+0000", 2024-09-21T14:59:36.473 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:36.473 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.474 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.474 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:36.474 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159924+0000", 2024-09-21T14:59:36.474 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:36.474 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.474 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.474 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:36.474 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159923+0000", 2024-09-21T14:59:36.474 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:36.474 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.474 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.475 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:36.475 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159926+0000", 2024-09-21T14:59:36.475 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0510000000000001e-06 2024-09-21T14:59:36.475 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.475 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.475 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:36.475 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159932+0000", 2024-09-21T14:59:36.475 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4160000000000001e-06 2024-09-21T14:59:36.475 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.475 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.475 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:36.475 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159938+0000", 2024-09-21T14:59:36.475 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.383e-06 2024-09-21T14:59:36.476 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.476 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.476 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:36.476 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228275+0000", 2024-09-21T14:59:36.476 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068337007000000005 2024-09-21T14:59:36.476 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.476 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.476 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:36.476 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228288+0000", 2024-09-21T14:59:36.476 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3052e-05 2024-09-21T14:59:36.476 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:36.476 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:36.476 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:36.477 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.477 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.477 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45987 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:36.477 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159944+0000", 2024-09-21T14:59:36.477 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 26.228923252000001, 2024-09-21T14:59:36.477 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068387232000000006, 2024-09-21T14:59:36.477 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:36.477 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:36.477 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:36.477 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:36.477 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:36.478 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45987 2024-09-21T14:59:36.478 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.478 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:36.478 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.478 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:36.478 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159944+0000", 2024-09-21T14:59:36.478 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:36.478 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.478 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.478 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:36.478 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159944+0000", 2024-09-21T14:59:36.479 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:36.479 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.479 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.479 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:36.479 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159943+0000", 2024-09-21T14:59:36.479 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:36.479 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.479 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.479 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:36.479 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159946+0000", 2024-09-21T14:59:36.479 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6309999999999999e-06 2024-09-21T14:59:36.480 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.480 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.480 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:36.480 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159952+0000", 2024-09-21T14:59:36.480 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6540000000000004e-06 2024-09-21T14:59:36.480 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.480 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.480 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:36.480 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159957+0000", 2024-09-21T14:59:36.480 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9280000000000001e-06 2024-09-21T14:59:36.480 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.480 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.480 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:36.481 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228322+0000", 2024-09-21T14:59:36.481 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068365067000000002 2024-09-21T14:59:36.481 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.481 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.481 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:36.481 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228331+0000", 2024-09-21T14:59:36.481 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3270000000000007e-06 2024-09-21T14:59:36.481 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:36.481 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:36.481 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:36.481 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:36.481 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:36.481 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:59:36.483 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T14:59:36.483 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:59:36.481+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:59:36.583 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:59:36.579+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:59:36.601 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:59:36.601 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:59:36.601 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T14:59:36.601 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:59:36.612 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T14:59:36.684 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:59:36.685+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:59:36.768 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:59:36.768 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:59:36.769 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T14:59:36.769 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T14:59:36.769 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:59:36.778 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T14:59:36.785 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:59:36.785+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:59:36.886 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:59:36.883+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:59:36.889 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:59:36.889 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T14:59:36.890 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T14:59:36.890 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T14:59:36.890 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.890 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:10651 3.12 3:4e9d23af:::rbd_data.1136f5ed5dd6.0000000000000020:head [write 0~81920 [fadvise_sequential+fadvise_nocache],write 90112~8192 [fadvise_sequential+fadvise_nocache],write 122880~122880 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:36.890 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.292476+0000", 2024-09-21T14:59:36.890 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 57.594379072000002, 2024-09-21T14:59:36.890 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012765669, 2024-09-21T14:59:36.890 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:36.890 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:36.890 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:36.890 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:36.890 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:36.890 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 10651 2024-09-21T14:59:36.891 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.891 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:36.891 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.891 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:36.891 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292476+0000", 2024-09-21T14:59:36.891 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:36.891 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.891 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.891 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:36.891 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292476+0000", 2024-09-21T14:59:36.891 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:36.891 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.891 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.892 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:36.892 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292475+0000", 2024-09-21T14:59:36.892 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:36.892 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.892 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.892 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:36.892 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292696+0000", 2024-09-21T14:59:36.892 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000220201 2024-09-21T14:59:36.892 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.892 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.892 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:36.892 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292711+0000", 2024-09-21T14:59:36.892 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5803000000000001e-05 2024-09-21T14:59:36.892 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.893 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.893 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:36.893 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292721+0000", 2024-09-21T14:59:36.893 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6160000000000008e-06 2024-09-21T14:59:36.893 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.893 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.893 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:36.893 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292757+0000", 2024-09-21T14:59:36.893 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6211999999999998e-05 2024-09-21T14:59:36.893 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.893 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.893 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:36.893 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292878+0000", 2024-09-21T14:59:36.893 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000120306 2024-09-21T14:59:36.893 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.894 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.894 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 3", 2024-09-21T14:59:36.894 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.293019+0000", 2024-09-21T14:59:36.894 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014186400000000001 2024-09-21T14:59:36.894 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.894 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.894 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:59:36.894 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.294898+0000", 2024-09-21T14:59:36.894 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001878434 2024-09-21T14:59:36.894 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.894 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.894 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:36.894 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.305162+0000", 2024-09-21T14:59:36.895 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.01026373 2024-09-21T14:59:36.895 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.895 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.895 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:59:36.895 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.305209+0000", 2024-09-21T14:59:36.895 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7615999999999999e-05 2024-09-21T14:59:36.895 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.895 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.895 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:36.895 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.305242+0000", 2024-09-21T14:59:36.895 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2979999999999999e-05 2024-09-21T14:59:36.895 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:36.895 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:36.896 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:36.896 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.896 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.896 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:11285 3.12 3:4edf73d2:::rbd_data.1136f5ed5dd6.0000000000000039:head [write 90112~32768 [fadvise_sequential+fadvise_nocache],write 139264~24576 [fadvise_sequential+fadvise_nocache],write 180224~24576 [fadvise_sequential+fadvise_nocache],write 221184~40960 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:36.896 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.539470+0000", 2024-09-21T14:59:36.896 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 57.347385320000001, 2024-09-21T14:59:36.896 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0083360099999999996, 2024-09-21T14:59:36.896 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:36.896 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:36.896 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:36.896 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:36.896 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:36.896 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 11285 2024-09-21T14:59:36.896 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.896 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:36.897 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.897 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:36.897 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.539470+0000", 2024-09-21T14:59:36.897 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:36.897 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.897 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.897 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:36.897 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.539470+0000", 2024-09-21T14:59:36.897 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:36.897 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.897 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.897 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:36.897 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.539469+0000", 2024-09-21T14:59:36.897 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:36.898 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.898 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.898 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:36.898 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.539652+0000", 2024-09-21T14:59:36.898 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000183084 2024-09-21T14:59:36.898 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.898 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.898 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:36.898 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.539666+0000", 2024-09-21T14:59:36.898 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3947e-05 2024-09-21T14:59:36.898 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.898 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.898 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:36.898 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.539673+0000", 2024-09-21T14:59:36.899 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7309999999999996e-06 2024-09-21T14:59:36.899 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.899 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.899 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:36.899 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.539714+0000", 2024-09-21T14:59:36.899 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1539000000000003e-05 2024-09-21T14:59:36.899 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.899 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.899 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:36.899 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.539832+0000", 2024-09-21T14:59:36.899 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011753500000000001 2024-09-21T14:59:36.899 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.899 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.899 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 3", 2024-09-21T14:59:36.900 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.540005+0000", 2024-09-21T14:59:36.900 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000173031 2024-09-21T14:59:36.900 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.900 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.900 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:59:36.900 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.541606+0000", 2024-09-21T14:59:36.900 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001600899 2024-09-21T14:59:36.900 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.900 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.900 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:36.900 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.547709+0000", 2024-09-21T14:59:36.900 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0061027110000000002 2024-09-21T14:59:36.900 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.900 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.900 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:59:36.900 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.547767+0000", 2024-09-21T14:59:36.901 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8248999999999997e-05 2024-09-21T14:59:36.901 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.901 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.901 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:36.901 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.547806+0000", 2024-09-21T14:59:36.901 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9341000000000002e-05 2024-09-21T14:59:36.901 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:36.901 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:36.901 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:36.901 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.901 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.901 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12189 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 40960~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:36.901 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.618667+0000", 2024-09-21T14:59:36.901 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 57.268188870000003, 2024-09-21T14:59:36.902 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0083014549999999993, 2024-09-21T14:59:36.902 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:36.902 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:36.902 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:36.902 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:36.902 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:36.902 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12189 2024-09-21T14:59:36.902 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.902 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:36.902 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.902 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:36.902 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618667+0000", 2024-09-21T14:59:36.902 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:36.903 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.903 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.903 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:36.903 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618667+0000", 2024-09-21T14:59:36.903 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:36.903 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.903 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.903 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:36.903 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618666+0000", 2024-09-21T14:59:36.903 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:36.903 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.903 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.903 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:36.904 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618669+0000", 2024-09-21T14:59:36.904 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.554e-06 2024-09-21T14:59:36.904 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.904 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.904 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:36.904 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618675+0000", 2024-09-21T14:59:36.904 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2570000000000004e-06 2024-09-21T14:59:36.904 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.904 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.904 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:36.904 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618680+0000", 2024-09-21T14:59:36.904 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6539999999999998e-06 2024-09-21T14:59:36.904 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.904 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.904 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:36.905 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.626683+0000", 2024-09-21T14:59:36.905 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0080035880000000007 2024-09-21T14:59:36.905 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.905 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.905 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:36.905 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.626764+0000", 2024-09-21T14:59:36.905 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0672000000000003e-05 2024-09-21T14:59:36.905 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.905 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.905 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:36.905 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.626968+0000", 2024-09-21T14:59:36.905 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00020407499999999999 2024-09-21T14:59:36.905 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:36.905 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:36.905 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:36.906 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.906 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.906 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12193 3.4 3:25437d51:::rbd_data.11363a21d2be.0000000000000025:10 [read 425984~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:36.906 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.618755+0000", 2024-09-21T14:59:36.906 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 57.268100789000002, 2024-09-21T14:59:36.906 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086461379999999994, 2024-09-21T14:59:36.906 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:36.906 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:36.906 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:36.906 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:36.906 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:36.906 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12193 2024-09-21T14:59:36.906 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.906 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:36.906 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.907 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:36.907 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618755+0000", 2024-09-21T14:59:36.907 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:36.907 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.907 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.907 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:36.907 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618755+0000", 2024-09-21T14:59:36.907 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:36.907 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.907 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.907 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:36.907 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618754+0000", 2024-09-21T14:59:36.907 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:36.907 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.908 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.908 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:36.908 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618759+0000", 2024-09-21T14:59:36.908 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4020000000000004e-06 2024-09-21T14:59:36.908 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.908 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.908 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:36.908 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618767+0000", 2024-09-21T14:59:36.908 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7490000000000004e-06 2024-09-21T14:59:36.908 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.908 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.908 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:36.908 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618773+0000", 2024-09-21T14:59:36.908 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0939999999999996e-06 2024-09-21T14:59:36.908 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.908 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.909 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:36.909 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.627115+0000", 2024-09-21T14:59:36.909 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0083416840000000003 2024-09-21T14:59:36.909 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.909 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.909 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:36.909 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.627214+0000", 2024-09-21T14:59:36.909 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9042000000000005e-05 2024-09-21T14:59:36.909 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.909 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.909 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:36.909 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.627401+0000", 2024-09-21T14:59:36.909 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018677200000000001 2024-09-21T14:59:36.909 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:36.910 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:36.910 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:36.910 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.910 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.910 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12212 3.4 3:21b43b45:::rbd_data.11363a21d2be.000000000000002b:10 [read 237568~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:36.910 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.619922+0000", 2024-09-21T14:59:36.910 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 57.266933805000001, 2024-09-21T14:59:36.910 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.008324372, 2024-09-21T14:59:36.910 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:36.910 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:36.910 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:36.910 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:36.910 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:36.910 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12212 2024-09-21T14:59:36.911 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.911 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:36.911 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.911 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:36.911 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619922+0000", 2024-09-21T14:59:36.911 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:36.911 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.911 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.911 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:36.911 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619922+0000", 2024-09-21T14:59:36.911 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:36.911 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.911 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.911 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:36.912 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619921+0000", 2024-09-21T14:59:36.912 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:36.912 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.912 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.912 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:36.912 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619924+0000", 2024-09-21T14:59:36.912 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.999e-06 2024-09-21T14:59:36.912 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.912 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.912 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:36.912 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619934+0000", 2024-09-21T14:59:36.912 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.8120000000000002e-06 2024-09-21T14:59:36.912 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.912 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.912 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:36.913 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619939+0000", 2024-09-21T14:59:36.913 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7729999999999999e-06 2024-09-21T14:59:36.913 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.913 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.913 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:36.913 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628005+0000", 2024-09-21T14:59:36.913 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0080663330000000002 2024-09-21T14:59:36.913 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.913 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.913 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:36.913 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628073+0000", 2024-09-21T14:59:36.913 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7479999999999998e-05 2024-09-21T14:59:36.913 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.913 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.913 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:36.913 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628246+0000", 2024-09-21T14:59:36.914 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017337400000000001 2024-09-21T14:59:36.914 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:36.914 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:36.914 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:36.914 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.914 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.914 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12213 3.4 3:21b43b45:::rbd_data.11363a21d2be.000000000000002b:10 [read 270336~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:36.914 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.619945+0000", 2024-09-21T14:59:36.914 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 57.266910523, 2024-09-21T14:59:36.914 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086195049999999995, 2024-09-21T14:59:36.914 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:36.914 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:36.914 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:36.914 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:36.914 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:36.915 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12213 2024-09-21T14:59:36.915 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.915 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:36.915 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.915 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:36.915 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619945+0000", 2024-09-21T14:59:36.915 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:36.915 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.915 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.915 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:36.915 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619945+0000", 2024-09-21T14:59:36.915 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:36.915 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.915 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.915 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:36.916 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619945+0000", 2024-09-21T14:59:36.916 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:36.916 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.916 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.916 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:36.916 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619947+0000", 2024-09-21T14:59:36.916 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5780000000000001e-06 2024-09-21T14:59:36.916 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.916 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.916 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:36.916 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619957+0000", 2024-09-21T14:59:36.916 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3500000000000003e-06 2024-09-21T14:59:36.916 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.916 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.916 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:36.917 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619964+0000", 2024-09-21T14:59:36.917 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.199e-06 2024-09-21T14:59:36.917 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.917 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.917 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:36.917 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628285+0000", 2024-09-21T14:59:36.917 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0083210939999999994 2024-09-21T14:59:36.917 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.917 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.917 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:36.917 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628361+0000", 2024-09-21T14:59:36.917 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5873999999999999e-05 2024-09-21T14:59:36.917 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.917 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.917 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:36.917 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628565+0000", 2024-09-21T14:59:36.918 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000203799 2024-09-21T14:59:36.918 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:36.918 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:36.918 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:36.918 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.918 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.918 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12214 3.4 3:21b43b45:::rbd_data.11363a21d2be.000000000000002b:10 [read 376832~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:36.918 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.619969+0000", 2024-09-21T14:59:36.918 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 57.266886522, 2024-09-21T14:59:36.918 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0090292089999999998, 2024-09-21T14:59:36.918 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:36.918 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:36.918 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:36.918 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:36.919 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:36.919 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12214 2024-09-21T14:59:36.919 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.919 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:36.919 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.919 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:36.919 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619969+0000", 2024-09-21T14:59:36.919 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:36.919 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.919 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.919 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:36.919 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619969+0000", 2024-09-21T14:59:36.919 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:36.919 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.919 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.919 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:36.920 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619969+0000", 2024-09-21T14:59:36.920 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:36.920 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.920 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.920 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:36.920 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619971+0000", 2024-09-21T14:59:36.920 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1830000000000001e-06 2024-09-21T14:59:36.920 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.920 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.920 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:36.920 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619976+0000", 2024-09-21T14:59:36.920 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7129999999999996e-06 2024-09-21T14:59:36.920 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.920 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.920 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:36.920 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619981+0000", 2024-09-21T14:59:36.921 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2460000000000003e-06 2024-09-21T14:59:36.921 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.921 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.921 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:36.921 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628602+0000", 2024-09-21T14:59:36.921 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086209519999999994 2024-09-21T14:59:36.921 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.921 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.921 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:36.921 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628734+0000", 2024-09-21T14:59:36.921 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000132242 2024-09-21T14:59:36.921 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.921 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.921 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:36.922 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628998+0000", 2024-09-21T14:59:36.922 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00026419799999999999 2024-09-21T14:59:36.922 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:36.922 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:36.922 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:36.922 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.922 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.922 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12215 3.4 3:21b43b45:::rbd_data.11363a21d2be.000000000000002b:10 [read 409600~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:36.922 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.619985+0000", 2024-09-21T14:59:36.922 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 57.266870455999999, 2024-09-21T14:59:36.922 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0095457319999999995, 2024-09-21T14:59:36.922 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:36.922 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:36.922 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:36.922 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:36.923 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:36.923 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12215 2024-09-21T14:59:36.923 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.923 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:36.923 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.923 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:36.923 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619985+0000", 2024-09-21T14:59:36.923 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:36.923 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.923 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.923 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:36.923 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619985+0000", 2024-09-21T14:59:36.923 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:36.923 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.923 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.924 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:36.924 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619985+0000", 2024-09-21T14:59:36.924 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:36.924 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.924 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.924 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:36.924 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619987+0000", 2024-09-21T14:59:36.924 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6129999999999998e-06 2024-09-21T14:59:36.924 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.924 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.924 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:36.924 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619993+0000", 2024-09-21T14:59:36.924 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0820000000000004e-06 2024-09-21T14:59:36.924 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.924 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.924 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:36.925 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619998+0000", 2024-09-21T14:59:36.925 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7759999999999997e-06 2024-09-21T14:59:36.925 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.925 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.925 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:36.925 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629022+0000", 2024-09-21T14:59:36.925 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0090237489999999993 2024-09-21T14:59:36.925 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.925 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.925 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:36.925 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629098+0000", 2024-09-21T14:59:36.925 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5735999999999995e-05 2024-09-21T14:59:36.925 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.925 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.925 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:36.926 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629531+0000", 2024-09-21T14:59:36.926 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00043310100000000003 2024-09-21T14:59:36.926 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:36.926 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:36.926 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:36.926 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.926 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.926 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12216 3.4 3:21b43b45:::rbd_data.11363a21d2be.000000000000002b:10 [read 516096~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:36.926 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.620003+0000", 2024-09-21T14:59:36.926 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 57.266852460000003, 2024-09-21T14:59:36.926 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0098850169999999994, 2024-09-21T14:59:36.926 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:36.926 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:36.926 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:36.926 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:36.927 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:36.927 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12216 2024-09-21T14:59:36.927 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.927 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:36.927 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.927 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:36.927 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620003+0000", 2024-09-21T14:59:36.927 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:36.927 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.927 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.927 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:36.927 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620003+0000", 2024-09-21T14:59:36.927 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:36.927 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.927 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.928 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:36.928 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620003+0000", 2024-09-21T14:59:36.928 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:36.928 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.928 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.928 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:36.928 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620006+0000", 2024-09-21T14:59:36.928 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4350000000000001e-06 2024-09-21T14:59:36.928 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.928 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.928 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:36.928 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620013+0000", 2024-09-21T14:59:36.928 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4219999999999997e-06 2024-09-21T14:59:36.928 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.928 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.929 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:36.929 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620017+0000", 2024-09-21T14:59:36.929 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.865e-06 2024-09-21T14:59:36.929 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.929 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.929 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:36.929 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629581+0000", 2024-09-21T14:59:36.929 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0095631989999999997 2024-09-21T14:59:36.929 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.929 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.929 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:36.929 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629691+0000", 2024-09-21T14:59:36.929 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000110586 2024-09-21T14:59:36.929 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.929 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.929 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:36.930 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629888+0000", 2024-09-21T14:59:36.930 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019691199999999999 2024-09-21T14:59:36.930 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:36.930 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:36.930 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:36.930 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.930 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.930 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12222 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 49152~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:36.930 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.620027+0000", 2024-09-21T14:59:36.930 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 57.266828113999999, 2024-09-21T14:59:36.930 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010228131999999999, 2024-09-21T14:59:36.930 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:36.930 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:36.930 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:36.930 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:36.931 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:36.931 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12222 2024-09-21T14:59:36.931 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.931 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:36.931 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.931 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:36.931 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620027+0000", 2024-09-21T14:59:36.931 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:36.931 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.931 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.931 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:36.931 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620027+0000", 2024-09-21T14:59:36.931 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:36.931 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.931 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.932 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:36.932 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620027+0000", 2024-09-21T14:59:36.932 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:36.932 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.932 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.932 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:36.932 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620030+0000", 2024-09-21T14:59:36.932 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0400000000000001e-06 2024-09-21T14:59:36.932 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.932 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.932 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:36.932 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620037+0000", 2024-09-21T14:59:36.932 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8220000000000001e-06 2024-09-21T14:59:36.932 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.932 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.933 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:36.933 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620042+0000", 2024-09-21T14:59:36.933 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.843e-06 2024-09-21T14:59:36.933 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.933 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.933 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:36.933 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629922+0000", 2024-09-21T14:59:36.933 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0098805100000000003 2024-09-21T14:59:36.933 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.933 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.933 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:36.933 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.630020+0000", 2024-09-21T14:59:36.933 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7596999999999994e-05 2024-09-21T14:59:36.933 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.934 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.934 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:36.934 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.630256+0000", 2024-09-21T14:59:36.934 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00023573299999999999 2024-09-21T14:59:36.934 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:36.934 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:36.934 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:36.934 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.934 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.934 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12253 3.4 3:21b43b45:::rbd_data.11363a21d2be.000000000000002b:10 [read 417792~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:36.934 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.624578+0000", 2024-09-21T14:59:36.934 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 57.262277746000002, 2024-09-21T14:59:36.934 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0085112079999999993, 2024-09-21T14:59:36.934 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:36.935 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:36.935 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:36.935 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:36.935 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:36.935 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12253 2024-09-21T14:59:36.935 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.935 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:36.935 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.935 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:36.935 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624578+0000", 2024-09-21T14:59:36.935 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:36.935 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.935 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.935 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:36.935 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624578+0000", 2024-09-21T14:59:36.936 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:36.936 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.936 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.936 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:36.936 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624578+0000", 2024-09-21T14:59:36.936 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:36.936 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.936 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.936 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:36.936 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624579+0000", 2024-09-21T14:59:36.936 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.688e-06 2024-09-21T14:59:36.936 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.936 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.936 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:36.936 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624582+0000", 2024-09-21T14:59:36.937 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2260000000000001e-06 2024-09-21T14:59:36.937 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.937 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.937 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:36.937 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624585+0000", 2024-09-21T14:59:36.937 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4820000000000001e-06 2024-09-21T14:59:36.937 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.937 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.937 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:36.937 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.632580+0000", 2024-09-21T14:59:36.937 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0079948959999999996 2024-09-21T14:59:36.937 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.937 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.937 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:36.937 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.632683+0000", 2024-09-21T14:59:36.937 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010308 2024-09-21T14:59:36.938 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.938 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.938 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:36.938 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.633089+0000", 2024-09-21T14:59:36.938 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00040612599999999998 2024-09-21T14:59:36.938 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:36.938 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:36.938 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:36.938 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.938 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.938 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12260 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 57344~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:36.938 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.624587+0000", 2024-09-21T14:59:36.938 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 57.262268259000003, 2024-09-21T14:59:36.938 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0088282069999999994, 2024-09-21T14:59:36.939 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:36.939 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:36.939 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:36.939 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:36.939 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:36.939 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12260 2024-09-21T14:59:36.939 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.939 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:36.939 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.939 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:36.939 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624587+0000", 2024-09-21T14:59:36.939 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:36.939 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.939 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.939 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:36.939 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624587+0000", 2024-09-21T14:59:36.940 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:36.940 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.940 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.940 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:36.940 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624587+0000", 2024-09-21T14:59:36.940 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-21T14:59:36.940 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.940 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.940 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:36.940 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624589+0000", 2024-09-21T14:59:36.940 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4339999999999999e-06 2024-09-21T14:59:36.940 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.940 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.940 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:36.941 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624593+0000", 2024-09-21T14:59:36.941 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5859999999999998e-06 2024-09-21T14:59:36.941 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.941 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.941 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:36.941 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624596+0000", 2024-09-21T14:59:36.941 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5830000000000001e-06 2024-09-21T14:59:36.941 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.941 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.941 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:36.941 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.633132+0000", 2024-09-21T14:59:36.941 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.008536666 2024-09-21T14:59:36.941 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.941 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.941 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:36.942 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.633216+0000", 2024-09-21T14:59:36.942 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3387999999999995e-05 2024-09-21T14:59:36.942 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.942 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.942 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:36.942 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.633415+0000", 2024-09-21T14:59:36.942 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019976799999999999 2024-09-21T14:59:36.942 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:36.942 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:36.942 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:36.942 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.942 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.942 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12516 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 245760~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:36.942 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645194+0000", 2024-09-21T14:59:36.943 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 57.241661722000003, 2024-09-21T14:59:36.943 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0084284990000000008, 2024-09-21T14:59:36.943 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:36.943 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:36.943 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:36.943 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:36.943 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:36.943 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12516 2024-09-21T14:59:36.943 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.943 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:36.943 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.943 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:36.943 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645194+0000", 2024-09-21T14:59:36.943 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:36.944 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.944 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.944 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:36.944 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645194+0000", 2024-09-21T14:59:36.944 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:36.944 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.944 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.944 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:36.944 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645193+0000", 2024-09-21T14:59:36.944 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:36.944 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.944 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.944 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:36.944 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645196+0000", 2024-09-21T14:59:36.944 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1030000000000002e-06 2024-09-21T14:59:36.945 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.945 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.945 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:36.945 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645205+0000", 2024-09-21T14:59:36.945 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6200000000000005e-06 2024-09-21T14:59:36.945 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.945 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.945 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:36.945 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645210+0000", 2024-09-21T14:59:36.945 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6280000000000004e-06 2024-09-21T14:59:36.945 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.945 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.945 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:36.945 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.653305+0000", 2024-09-21T14:59:36.945 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0080957879999999996 2024-09-21T14:59:36.945 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.946 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.946 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:36.946 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.653389+0000", 2024-09-21T14:59:36.946 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3242e-05 2024-09-21T14:59:36.946 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.946 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.946 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:36.946 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.653622+0000", 2024-09-21T14:59:36.946 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00023361599999999999 2024-09-21T14:59:36.946 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:36.946 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:36.946 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:36.946 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.946 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.946 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12517 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 278528~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:36.947 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645214+0000", 2024-09-21T14:59:36.947 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 57.241641608999998, 2024-09-21T14:59:36.947 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086832899999999998, 2024-09-21T14:59:36.947 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:36.947 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:36.947 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:36.947 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:36.947 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:36.947 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12517 2024-09-21T14:59:36.947 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.947 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:36.947 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.947 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:36.947 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645214+0000", 2024-09-21T14:59:36.947 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:36.948 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.948 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.948 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:36.948 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645214+0000", 2024-09-21T14:59:36.948 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:36.948 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.948 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.948 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:36.948 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645214+0000", 2024-09-21T14:59:36.948 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:36.948 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.948 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.948 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:36.948 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645216+0000", 2024-09-21T14:59:36.948 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5579999999999999e-06 2024-09-21T14:59:36.949 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.949 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.949 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:36.949 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645224+0000", 2024-09-21T14:59:36.949 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2439999999999993e-06 2024-09-21T14:59:36.949 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.949 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.949 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:36.949 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645231+0000", 2024-09-21T14:59:36.949 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1859999999999997e-06 2024-09-21T14:59:36.949 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.949 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.949 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:36.949 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.653651+0000", 2024-09-21T14:59:36.950 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0084204220000000003 2024-09-21T14:59:36.950 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.950 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.950 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:36.950 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.653726+0000", 2024-09-21T14:59:36.950 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5356000000000002e-05 2024-09-21T14:59:36.950 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.950 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.950 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:36.950 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.653897+0000", 2024-09-21T14:59:36.950 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000170848 2024-09-21T14:59:36.950 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:36.950 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:36.950 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:36.950 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.951 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.951 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12518 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 385024~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:36.951 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645235+0000", 2024-09-21T14:59:36.951 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 57.241620382999997, 2024-09-21T14:59:36.951 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0089841020000000008, 2024-09-21T14:59:36.951 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:36.951 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:36.951 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:36.951 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:36.951 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:36.951 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12518 2024-09-21T14:59:36.951 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.951 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:36.951 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.951 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:36.951 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645235+0000", 2024-09-21T14:59:36.952 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:36.952 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.952 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.952 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:36.952 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645235+0000", 2024-09-21T14:59:36.952 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:36.952 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.952 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.952 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:36.952 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645235+0000", 2024-09-21T14:59:36.952 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:36.952 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.952 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.952 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:36.952 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645238+0000", 2024-09-21T14:59:36.953 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.869e-06 2024-09-21T14:59:36.953 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.953 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.953 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:36.953 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645244+0000", 2024-09-21T14:59:36.953 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8930000000000002e-06 2024-09-21T14:59:36.953 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.953 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.953 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:36.953 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645249+0000", 2024-09-21T14:59:36.953 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2859999999999999e-06 2024-09-21T14:59:36.953 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.953 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.953 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:36.953 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.653930+0000", 2024-09-21T14:59:36.953 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086815320000000005 2024-09-21T14:59:36.954 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.954 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.954 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:36.954 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654028+0000", 2024-09-21T14:59:36.954 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7578000000000007e-05 2024-09-21T14:59:36.954 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.954 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.954 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:36.954 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654219+0000", 2024-09-21T14:59:36.954 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019132099999999999 2024-09-21T14:59:36.954 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:36.954 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:36.954 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:36.954 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.954 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.955 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12538 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 147456~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:36.955 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645253+0000", 2024-09-21T14:59:36.955 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 57.241602759000003, 2024-09-21T14:59:36.955 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0092095360000000008, 2024-09-21T14:59:36.955 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:36.955 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:36.955 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:36.955 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:36.955 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:36.955 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12538 2024-09-21T14:59:36.955 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.955 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:36.955 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.955 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:36.955 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645253+0000", 2024-09-21T14:59:36.956 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:36.956 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.956 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.956 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:36.956 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645253+0000", 2024-09-21T14:59:36.956 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:36.956 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.956 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.956 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:36.956 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645252+0000", 2024-09-21T14:59:36.956 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:36.956 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.956 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.956 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:36.957 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645255+0000", 2024-09-21T14:59:36.957 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2290000000000002e-06 2024-09-21T14:59:36.957 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.957 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.957 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:36.957 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645267+0000", 2024-09-21T14:59:36.957 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2631e-05 2024-09-21T14:59:36.957 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.957 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.957 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:36.957 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645278+0000", 2024-09-21T14:59:36.957 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0847e-05 2024-09-21T14:59:36.957 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.957 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.958 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:36.958 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654251+0000", 2024-09-21T14:59:36.958 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0089731340000000007 2024-09-21T14:59:36.958 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.958 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.958 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:36.958 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654296+0000", 2024-09-21T14:59:36.958 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4353e-05 2024-09-21T14:59:36.958 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.958 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.958 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:36.958 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654462+0000", 2024-09-21T14:59:36.958 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016668499999999999 2024-09-21T14:59:36.958 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:36.959 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:36.959 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:36.959 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.959 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.959 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12539 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 253952~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:36.959 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645284+0000", 2024-09-21T14:59:36.959 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 57.241571538000002, 2024-09-21T14:59:36.959 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0094524029999999998, 2024-09-21T14:59:36.959 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:36.959 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:36.959 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:36.959 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:36.959 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:36.959 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12539 2024-09-21T14:59:36.960 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.960 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:36.960 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.960 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:36.960 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645284+0000", 2024-09-21T14:59:36.960 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:36.960 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.960 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.960 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:36.960 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645284+0000", 2024-09-21T14:59:36.960 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:36.960 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.960 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.960 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:36.961 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645284+0000", 2024-09-21T14:59:36.961 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:36.961 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.961 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.961 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:36.961 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645286+0000", 2024-09-21T14:59:36.961 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7159999999999999e-06 2024-09-21T14:59:36.961 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.961 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.961 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:36.961 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645292+0000", 2024-09-21T14:59:36.961 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7400000000000001e-06 2024-09-21T14:59:36.961 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.961 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.961 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:36.962 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645297+0000", 2024-09-21T14:59:36.962 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9019999999999998e-06 2024-09-21T14:59:36.962 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.962 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.962 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:36.962 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654490+0000", 2024-09-21T14:59:36.962 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0091932360000000005 2024-09-21T14:59:36.962 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.962 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.962 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:36.962 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654565+0000", 2024-09-21T14:59:36.962 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4657e-05 2024-09-21T14:59:36.962 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.962 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.963 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:36.963 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654736+0000", 2024-09-21T14:59:36.963 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017156199999999999 2024-09-21T14:59:36.963 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:36.963 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:36.963 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:36.963 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.963 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.963 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12540 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 286720~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:36.963 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645302+0000", 2024-09-21T14:59:36.963 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 57.241553269000001, 2024-09-21T14:59:36.963 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0096603090000000006, 2024-09-21T14:59:36.963 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:36.963 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:36.963 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:36.964 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:36.964 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:36.964 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12540 2024-09-21T14:59:36.964 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.964 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:36.964 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.964 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:36.964 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645302+0000", 2024-09-21T14:59:36.964 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:36.964 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.964 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.964 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:36.964 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645302+0000", 2024-09-21T14:59:36.964 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:36.964 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.965 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.965 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:36.965 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645302+0000", 2024-09-21T14:59:36.965 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:36.965 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.965 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.965 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:36.965 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645305+0000", 2024-09-21T14:59:36.965 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7329999999999999e-06 2024-09-21T14:59:36.965 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.965 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.965 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:36.965 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645316+0000", 2024-09-21T14:59:36.965 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1832e-05 2024-09-21T14:59:36.965 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.966 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.966 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:36.966 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645322+0000", 2024-09-21T14:59:36.966 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.169e-06 2024-09-21T14:59:36.966 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.966 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.966 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:36.966 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654778+0000", 2024-09-21T14:59:36.966 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0094559090000000002 2024-09-21T14:59:36.966 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.966 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.966 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:36.966 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654828+0000", 2024-09-21T14:59:36.966 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0461999999999999e-05 2024-09-21T14:59:36.966 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.967 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.967 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:36.967 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654963+0000", 2024-09-21T14:59:36.967 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000134583 2024-09-21T14:59:36.967 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:36.967 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:36.967 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:36.967 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.967 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.967 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12561 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:36.967 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645742+0000", 2024-09-21T14:59:36.967 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 57.241114003, 2024-09-21T14:59:36.967 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0093583810000000007, 2024-09-21T14:59:36.967 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:36.967 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:36.968 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:36.968 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:36.968 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:36.968 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12561 2024-09-21T14:59:36.968 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.968 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:36.968 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.968 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:36.968 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645742+0000", 2024-09-21T14:59:36.968 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:36.968 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.968 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.968 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:36.968 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645742+0000", 2024-09-21T14:59:36.968 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:36.969 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.969 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.969 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:36.969 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645741+0000", 2024-09-21T14:59:36.969 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:36.969 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.969 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.969 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:36.969 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645746+0000", 2024-09-21T14:59:36.969 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8379999999999998e-06 2024-09-21T14:59:36.969 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.969 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.969 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:36.969 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645757+0000", 2024-09-21T14:59:36.969 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0705e-05 2024-09-21T14:59:36.970 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.970 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.970 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:36.970 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645765+0000", 2024-09-21T14:59:36.970 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6729999999999992e-06 2024-09-21T14:59:36.970 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.970 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.970 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:36.970 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654993+0000", 2024-09-21T14:59:36.970 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0092287480000000002 2024-09-21T14:59:36.970 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.970 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.970 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:36.970 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.655050+0000", 2024-09-21T14:59:36.971 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6394999999999998e-05 2024-09-21T14:59:36.971 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.971 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.971 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:36.971 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.655100+0000", 2024-09-21T14:59:36.971 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0049000000000001e-05 2024-09-21T14:59:36.971 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:36.971 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:36.971 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:36.971 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.971 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.971 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:15431 3.12 3:4bc7d206:::rbd_data.1136f5ed5dd6.0000000000000051:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:59:36.971 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:41.673445+0000", 2024-09-21T14:59:36.971 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 55.213410668999998, 2024-09-21T14:59:36.971 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086103610000000004, 2024-09-21T14:59:36.972 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:36.972 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:36.972 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:36.972 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:36.972 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:36.972 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 15431 2024-09-21T14:59:36.972 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.972 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:36.972 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.972 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:36.972 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.673445+0000", 2024-09-21T14:59:36.972 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:36.972 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.972 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.972 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:36.973 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.673445+0000", 2024-09-21T14:59:36.973 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:36.973 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.973 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.973 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:36.973 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.673444+0000", 2024-09-21T14:59:36.973 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:59:36.973 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.973 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.973 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:36.973 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.673451+0000", 2024-09-21T14:59:36.973 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4739999999999999e-06 2024-09-21T14:59:36.973 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.973 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.973 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:36.973 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.673472+0000", 2024-09-21T14:59:36.974 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0687e-05 2024-09-21T14:59:36.974 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.974 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.974 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:36.974 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.673481+0000", 2024-09-21T14:59:36.974 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3349999999999996e-06 2024-09-21T14:59:36.974 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.974 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.974 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:36.974 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.673565+0000", 2024-09-21T14:59:36.974 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4011000000000006e-05 2024-09-21T14:59:36.974 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.974 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.974 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:36.975 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.673650+0000", 2024-09-21T14:59:36.975 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4949000000000003e-05 2024-09-21T14:59:36.975 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.975 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.975 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 3", 2024-09-21T14:59:36.975 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.675014+0000", 2024-09-21T14:59:36.975 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0013636620000000001 2024-09-21T14:59:36.975 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.975 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.975 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:36.975 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.681866+0000", 2024-09-21T14:59:36.975 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0068525640000000002 2024-09-21T14:59:36.975 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.975 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.975 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:59:36.976 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.681885+0000", 2024-09-21T14:59:36.976 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8519999999999999e-05 2024-09-21T14:59:36.976 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.976 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.976 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:59:36.976 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.681961+0000", 2024-09-21T14:59:36.976 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6248000000000006e-05 2024-09-21T14:59:36.976 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:36.976 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:36.976 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:36.976 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.682055+0000", 2024-09-21T14:59:36.976 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4184000000000005e-05 2024-09-21T14:59:36.976 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:36.976 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:36.976 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:36.977 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:36.977 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:36.977 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:59:36.978 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T14:59:36.986 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:59:36.985+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:59:37.087 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:59:37.085+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:59:37.094 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:59:37.094 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:59:37.094 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T14:59:37.094 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:59:37.104 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T14:59:37.187 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:59:37.183+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:59:37.209 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:59:37.209 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:59:37.209 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T14:59:37.210 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T14:59:37.210 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:59:37.219 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T14:59:37.288 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:59:37.289+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:59:37.294 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T14:59:37.329 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:59:37.329 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T14:59:37.329 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T14:59:37.329 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T14:59:37.330 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.330 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:8034 3.b 3:d1b12ef9:::rbd_data.11361bed1f78.0000000000000062:d [read 32768~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:59:37.330 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:35.227265+0000", 2024-09-21T14:59:37.330 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 62.099315930000003, 2024-09-21T14:59:37.330 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.006525968, 2024-09-21T14:59:37.330 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:37.330 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:37.330 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:37.330 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:37.330 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:37.330 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 8034 2024-09-21T14:59:37.330 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.330 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:37.330 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.331 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:37.331 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227265+0000", 2024-09-21T14:59:37.331 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:37.331 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.331 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.331 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:37.331 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227265+0000", 2024-09-21T14:59:37.331 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:37.331 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.331 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.331 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:37.331 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227265+0000", 2024-09-21T14:59:37.331 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:37.331 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.331 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.332 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:37.332 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227268+0000", 2024-09-21T14:59:37.332 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8990000000000001e-06 2024-09-21T14:59:37.332 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.332 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.332 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:37.332 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227274+0000", 2024-09-21T14:59:37.332 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4470000000000003e-06 2024-09-21T14:59:37.332 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.332 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.332 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:37.332 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227288+0000", 2024-09-21T14:59:37.332 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3550000000000001e-05 2024-09-21T14:59:37.332 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.332 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.333 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:37.333 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.233418+0000", 2024-09-21T14:59:37.333 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0061307870000000004 2024-09-21T14:59:37.333 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.333 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.333 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:37.333 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.233553+0000", 2024-09-21T14:59:37.333 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013475600000000001 2024-09-21T14:59:37.333 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.333 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.333 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:37.333 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.233791+0000", 2024-09-21T14:59:37.333 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00023788499999999999 2024-09-21T14:59:37.333 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:37.334 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:37.334 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:37.334 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.334 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.334 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:8035 3.b 3:d1b12ef9:::rbd_data.11361bed1f78.0000000000000062:d [read 98304~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:59:37.334 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:35.227295+0000", 2024-09-21T14:59:37.334 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 62.099285725000001, 2024-09-21T14:59:37.334 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0069673820000000003, 2024-09-21T14:59:37.334 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:37.334 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:37.334 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:37.334 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:37.334 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:37.334 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 8035 2024-09-21T14:59:37.335 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.335 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:37.335 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.335 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:37.335 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227295+0000", 2024-09-21T14:59:37.335 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:37.335 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.335 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.335 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:37.335 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227295+0000", 2024-09-21T14:59:37.335 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:37.335 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.335 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.335 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:37.336 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227295+0000", 2024-09-21T14:59:37.336 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:37.336 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.336 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.336 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:37.336 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227298+0000", 2024-09-21T14:59:37.336 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2339999999999999e-06 2024-09-21T14:59:37.336 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.336 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.336 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:37.336 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227305+0000", 2024-09-21T14:59:37.336 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0389999999999999e-06 2024-09-21T14:59:37.336 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.336 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.336 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:37.337 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227311+0000", 2024-09-21T14:59:37.337 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8470000000000001e-06 2024-09-21T14:59:37.337 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.337 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.337 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:37.337 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.233824+0000", 2024-09-21T14:59:37.337 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0065132920000000004 2024-09-21T14:59:37.337 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.337 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.337 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:37.337 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.233922+0000", 2024-09-21T14:59:37.337 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.8073999999999996e-05 2024-09-21T14:59:37.337 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.337 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.337 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:37.338 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.234263+0000", 2024-09-21T14:59:37.338 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00034027999999999999 2024-09-21T14:59:37.338 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:37.338 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:37.338 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:37.338 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.338 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.338 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:8048 3.b 3:d1b12ef9:::rbd_data.11361bed1f78.0000000000000062:d [read 49152~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:59:37.338 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:35.227316+0000", 2024-09-21T14:59:37.338 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 62.099264933000001, 2024-09-21T14:59:37.338 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0076197799999999996, 2024-09-21T14:59:37.338 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:37.338 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:37.339 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:37.339 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:37.339 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:37.339 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 8048 2024-09-21T14:59:37.339 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.339 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:37.339 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.339 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:37.339 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227316+0000", 2024-09-21T14:59:37.339 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:37.339 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.339 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.339 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:37.339 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227316+0000", 2024-09-21T14:59:37.339 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:37.340 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.340 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.340 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:37.340 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227316+0000", 2024-09-21T14:59:37.340 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:37.340 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.340 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.340 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:37.340 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227324+0000", 2024-09-21T14:59:37.340 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1009999999999992e-06 2024-09-21T14:59:37.340 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.340 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.340 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:37.340 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227346+0000", 2024-09-21T14:59:37.340 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2347000000000001e-05 2024-09-21T14:59:37.340 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.341 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.341 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:37.341 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227360+0000", 2024-09-21T14:59:37.341 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.403e-05 2024-09-21T14:59:37.341 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.341 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.341 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:37.341 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.234320+0000", 2024-09-21T14:59:37.341 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0069597019999999999 2024-09-21T14:59:37.341 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.341 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.341 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:37.341 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.234730+0000", 2024-09-21T14:59:37.341 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00041006300000000001 2024-09-21T14:59:37.341 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.342 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.342 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:37.342 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.234936+0000", 2024-09-21T14:59:37.342 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00020586700000000001 2024-09-21T14:59:37.342 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:37.342 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:37.342 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:37.342 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.342 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.342 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:8049 3.b 3:d1b12ef9:::rbd_data.11361bed1f78.0000000000000062:d [read 114688~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:59:37.342 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:35.227371+0000", 2024-09-21T14:59:37.342 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 62.099209600000002, 2024-09-21T14:59:37.342 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0078533609999999997, 2024-09-21T14:59:37.342 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:37.342 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:37.343 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:37.343 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:37.343 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:37.343 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 8049 2024-09-21T14:59:37.343 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.343 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:37.343 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.343 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:37.343 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227371+0000", 2024-09-21T14:59:37.343 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:37.343 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.343 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.343 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:37.343 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227371+0000", 2024-09-21T14:59:37.343 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:37.343 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.344 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.344 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:37.344 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227370+0000", 2024-09-21T14:59:37.344 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:59:37.344 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.344 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.344 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:37.344 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227378+0000", 2024-09-21T14:59:37.344 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.379e-06 2024-09-21T14:59:37.344 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.344 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.344 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:37.344 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227388+0000", 2024-09-21T14:59:37.344 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0175e-05 2024-09-21T14:59:37.344 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.345 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.345 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:37.345 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227393+0000", 2024-09-21T14:59:37.345 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6690000000000003e-06 2024-09-21T14:59:37.345 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.345 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.345 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:37.345 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.234964+0000", 2024-09-21T14:59:37.345 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0075705319999999996 2024-09-21T14:59:37.345 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.345 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.345 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:37.345 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.235041+0000", 2024-09-21T14:59:37.345 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7170999999999997e-05 2024-09-21T14:59:37.345 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.345 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.346 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:37.346 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.235225+0000", 2024-09-21T14:59:37.346 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000183735 2024-09-21T14:59:37.346 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:37.346 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:37.346 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:37.346 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.346 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.346 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:19869 3.b 3:d7ca7d68:::rbd_data.11363a21d2be.000000000000002d:head [sparse-read 0~393216] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T14:59:37.346 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:44.340896+0000", 2024-09-21T14:59:37.346 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 52.985685203000003, 2024-09-21T14:59:37.346 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0071749170000000003, 2024-09-21T14:59:37.346 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:37.347 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:37.347 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:37.347 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:37.347 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:37.347 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 19869 2024-09-21T14:59:37.347 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.347 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:37.347 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.347 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:37.347 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340896+0000", 2024-09-21T14:59:37.347 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:37.347 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.347 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.347 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:37.347 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340896+0000", 2024-09-21T14:59:37.348 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:37.348 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.348 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.348 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:37.348 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340894+0000", 2024-09-21T14:59:37.348 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:59:37.348 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.348 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.348 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:37.348 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340903+0000", 2024-09-21T14:59:37.348 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0259999999999996e-06 2024-09-21T14:59:37.348 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.348 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.348 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:37.348 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340923+0000", 2024-09-21T14:59:37.349 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0013999999999998e-05 2024-09-21T14:59:37.349 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.349 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.349 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:37.349 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340938+0000", 2024-09-21T14:59:37.349 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4569000000000001e-05 2024-09-21T14:59:37.349 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.349 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.349 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:37.349 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342092+0000", 2024-09-21T14:59:37.349 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00115354 2024-09-21T14:59:37.349 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.349 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.349 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:37.349 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342290+0000", 2024-09-21T14:59:37.349 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019834299999999999 2024-09-21T14:59:37.350 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.350 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.350 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:37.350 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.348071+0000", 2024-09-21T14:59:37.350 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.005780861 2024-09-21T14:59:37.350 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:37.350 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:37.350 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:37.350 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.350 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.350 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:19872 3.b 3:d38ac3f6:::rbd_data.11363a21d2be.0000000000000030:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T14:59:37.350 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:44.340951+0000", 2024-09-21T14:59:37.350 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 52.985630043999997, 2024-09-21T14:59:37.350 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0069228680000000004, 2024-09-21T14:59:37.351 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:37.351 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:37.351 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:37.351 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:37.351 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:37.351 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 19872 2024-09-21T14:59:37.351 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.351 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:37.351 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.351 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:37.351 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340951+0000", 2024-09-21T14:59:37.351 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:37.351 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.351 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.351 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:37.352 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340951+0000", 2024-09-21T14:59:37.352 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:37.352 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.352 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.352 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:37.352 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340950+0000", 2024-09-21T14:59:37.352 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:37.352 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.352 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.352 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:37.352 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340957+0000", 2024-09-21T14:59:37.352 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8430000000000004e-06 2024-09-21T14:59:37.352 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.352 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.352 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:37.353 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340974+0000", 2024-09-21T14:59:37.353 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7082999999999999e-05 2024-09-21T14:59:37.353 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.353 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.353 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:37.353 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340987+0000", 2024-09-21T14:59:37.353 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2763e-05 2024-09-21T14:59:37.353 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.353 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.353 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:37.353 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.347149+0000", 2024-09-21T14:59:37.353 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0061626600000000004 2024-09-21T14:59:37.353 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.353 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.353 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:37.354 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.347354+0000", 2024-09-21T14:59:37.354 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00020441799999999999 2024-09-21T14:59:37.354 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.354 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.354 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:37.354 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.347874+0000", 2024-09-21T14:59:37.354 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00052019399999999997 2024-09-21T14:59:37.354 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:37.354 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:37.354 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:37.354 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.354 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.354 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:19877 3.b 3:d5fc5a5a:::rbd_data.11363a21d2be.0000000000000035:head [sparse-read 0~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T14:59:37.354 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:44.340998+0000", 2024-09-21T14:59:37.354 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 52.985583208000001, 2024-09-21T14:59:37.354 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0073322129999999998, 2024-09-21T14:59:37.355 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:37.355 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:37.355 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:37.355 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:37.355 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:37.355 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 19877 2024-09-21T14:59:37.355 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.355 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:37.355 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.355 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:37.355 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340998+0000", 2024-09-21T14:59:37.355 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:37.355 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.356 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.356 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:37.356 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340998+0000", 2024-09-21T14:59:37.356 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:37.356 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.356 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.356 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:37.356 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.340997+0000", 2024-09-21T14:59:37.356 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:37.356 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.356 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.356 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:37.356 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.341003+0000", 2024-09-21T14:59:37.356 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0070000000000003e-06 2024-09-21T14:59:37.356 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.357 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.357 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:37.357 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.341018+0000", 2024-09-21T14:59:37.357 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5472999999999999e-05 2024-09-21T14:59:37.357 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.357 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.357 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:37.357 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.341040+0000", 2024-09-21T14:59:37.357 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1568000000000001e-05 2024-09-21T14:59:37.357 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.357 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.357 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:37.357 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.347901+0000", 2024-09-21T14:59:37.357 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0068608300000000001 2024-09-21T14:59:37.357 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.358 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.358 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:37.358 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.347973+0000", 2024-09-21T14:59:37.358 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2327000000000001e-05 2024-09-21T14:59:37.358 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.358 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.358 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:37.358 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.348330+0000", 2024-09-21T14:59:37.358 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00035695799999999999 2024-09-21T14:59:37.358 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:37.358 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:37.358 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:37.358 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.358 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.358 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:19999 3.b 3:d4069488:::rbd_data.11363a21d2be.00000000000000af:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-21T14:59:37.359 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:44.342246+0000", 2024-09-21T14:59:37.359 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 52.984335272000003, 2024-09-21T14:59:37.359 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0064139210000000004, 2024-09-21T14:59:37.359 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:37.359 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:37.359 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:37.359 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:37.359 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:37.359 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 19999 2024-09-21T14:59:37.359 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.359 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:37.359 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.359 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:37.359 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342246+0000", 2024-09-21T14:59:37.359 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:37.359 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.360 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.360 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:37.360 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342246+0000", 2024-09-21T14:59:37.360 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:37.360 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.360 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.360 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:37.360 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342245+0000", 2024-09-21T14:59:37.360 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:37.360 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.360 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.360 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:37.360 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342251+0000", 2024-09-21T14:59:37.360 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8660000000000002e-06 2024-09-21T14:59:37.360 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.361 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.361 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:37.361 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342264+0000", 2024-09-21T14:59:37.361 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3716999999999999e-05 2024-09-21T14:59:37.361 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.361 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.361 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:37.361 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.342275+0000", 2024-09-21T14:59:37.361 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1071e-05 2024-09-21T14:59:37.361 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.361 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.361 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:37.361 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.348581+0000", 2024-09-21T14:59:37.361 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0063053730000000004 2024-09-21T14:59:37.361 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.361 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.362 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:37.362 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:44.348660+0000", 2024-09-21T14:59:37.362 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8854000000000001e-05 2024-09-21T14:59:37.362 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:37.362 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:37.362 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:37.362 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.362 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.362 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:20440 3.b 3:d6a6f68e:::rbd_object_map.1136c84d9a13:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-21T14:59:37.362 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:45.106428+0000", 2024-09-21T14:59:37.362 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 52.220153426000003, 2024-09-21T14:59:37.362 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0076349679999999998, 2024-09-21T14:59:37.362 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:37.362 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:37.362 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:37.363 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:37.363 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:37.363 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 20440 2024-09-21T14:59:37.363 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.363 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:37.363 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.363 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:37.363 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.106428+0000", 2024-09-21T14:59:37.363 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:37.363 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.363 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.363 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:37.363 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.106428+0000", 2024-09-21T14:59:37.363 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:37.364 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.364 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.364 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:37.364 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.106425+0000", 2024-09-21T14:59:37.364 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-21T14:59:37.364 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.364 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.364 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:37.364 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.106442+0000", 2024-09-21T14:59:37.364 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7237000000000001e-05 2024-09-21T14:59:37.364 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.364 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.364 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:37.364 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.106476+0000", 2024-09-21T14:59:37.364 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4004999999999997e-05 2024-09-21T14:59:37.365 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.365 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.365 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:37.365 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.106491+0000", 2024-09-21T14:59:37.365 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4865000000000001e-05 2024-09-21T14:59:37.365 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.365 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.365 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:37.365 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.106602+0000", 2024-09-21T14:59:37.365 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011113200000000001 2024-09-21T14:59:37.365 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.365 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.365 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op must wait for map", 2024-09-21T14:59:37.365 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.106611+0000", 2024-09-21T14:59:37.365 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9199999999999993e-06 2024-09-21T14:59:37.365 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.366 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.366 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:37.366 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.110475+0000", 2024-09-21T14:59:37.366 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0038633259999999998 2024-09-21T14:59:37.366 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.366 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.366 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:37.366 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.110622+0000", 2024-09-21T14:59:37.366 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014734200000000001 2024-09-21T14:59:37.366 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.366 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.366 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T14:59:37.366 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.111136+0000", 2024-09-21T14:59:37.366 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00051382299999999995 2024-09-21T14:59:37.366 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.367 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.367 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:59:37.367 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.112514+0000", 2024-09-21T14:59:37.367 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001378287 2024-09-21T14:59:37.367 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.367 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.367 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:37.367 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.113830+0000", 2024-09-21T14:59:37.367 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0013157939999999999 2024-09-21T14:59:37.367 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.367 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.367 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:59:37.367 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.113969+0000", 2024-09-21T14:59:37.367 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000138849 2024-09-21T14:59:37.367 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.367 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.368 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:37.368 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.114063+0000", 2024-09-21T14:59:37.368 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3999999999999994e-05 2024-09-21T14:59:37.368 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:37.368 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:37.368 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:37.368 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.368 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.368 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:20460 3.b 3:d6a6f68e:::rbd_object_map.1136c84d9a13:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-21T14:59:37.368 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:45.150183+0000", 2024-09-21T14:59:37.368 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 52.176397608000002, 2024-09-21T14:59:37.368 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0063818800000000004, 2024-09-21T14:59:37.368 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:37.368 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:37.368 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:37.369 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:37.369 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:37.369 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 20460 2024-09-21T14:59:37.369 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.369 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:37.369 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.369 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:37.369 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150183+0000", 2024-09-21T14:59:37.369 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:37.369 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.369 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.369 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:37.369 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150183+0000", 2024-09-21T14:59:37.369 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:37.370 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.370 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.370 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:37.370 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150180+0000", 2024-09-21T14:59:37.370 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999971 2024-09-21T14:59:37.370 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.370 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.370 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:37.370 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150211+0000", 2024-09-21T14:59:37.370 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0993e-05 2024-09-21T14:59:37.370 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.370 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.370 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:37.370 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150241+0000", 2024-09-21T14:59:37.371 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9589000000000001e-05 2024-09-21T14:59:37.371 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.371 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.371 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:37.371 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150261+0000", 2024-09-21T14:59:37.371 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.967e-05 2024-09-21T14:59:37.371 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.371 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.371 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:37.371 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150411+0000", 2024-09-21T14:59:37.371 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015004300000000001 2024-09-21T14:59:37.371 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.371 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.371 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:37.371 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150598+0000", 2024-09-21T14:59:37.372 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018712399999999999 2024-09-21T14:59:37.372 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.372 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.372 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T14:59:37.372 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.151390+0000", 2024-09-21T14:59:37.372 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00079194599999999997 2024-09-21T14:59:37.372 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.372 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.372 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:37.372 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.156302+0000", 2024-09-21T14:59:37.372 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0049123960000000003 2024-09-21T14:59:37.372 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.372 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.372 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:59:37.372 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.156435+0000", 2024-09-21T14:59:37.373 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013287300000000001 2024-09-21T14:59:37.373 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.373 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.373 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:59:37.373 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.156508+0000", 2024-09-21T14:59:37.373 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2504999999999998e-05 2024-09-21T14:59:37.373 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.373 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.373 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:37.373 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.156565+0000", 2024-09-21T14:59:37.373 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7773999999999999e-05 2024-09-21T14:59:37.373 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:37.373 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:37.373 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:37.374 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.374 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.374 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:20461 3.b 3:d6a6f68e:::rbd_object_map.1136c84d9a13:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-21T14:59:37.374 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:45.150284+0000", 2024-09-21T14:59:37.374 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 52.176296786999998, 2024-09-21T14:59:37.374 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0064029040000000001, 2024-09-21T14:59:37.374 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:37.374 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:37.374 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:37.374 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:37.374 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:37.374 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 20461 2024-09-21T14:59:37.374 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.375 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:37.375 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.375 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:37.375 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150284+0000", 2024-09-21T14:59:37.375 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:37.375 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.375 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.375 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:37.375 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150284+0000", 2024-09-21T14:59:37.375 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:37.375 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.375 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.375 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:37.376 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150283+0000", 2024-09-21T14:59:37.376 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:37.376 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.376 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.376 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:37.376 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150289+0000", 2024-09-21T14:59:37.376 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8799999999999996e-06 2024-09-21T14:59:37.376 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.376 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.376 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:37.376 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150319+0000", 2024-09-21T14:59:37.376 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9828999999999999e-05 2024-09-21T14:59:37.376 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.377 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.377 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:37.377 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150328+0000", 2024-09-21T14:59:37.377 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4770000000000003e-06 2024-09-21T14:59:37.377 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.377 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.377 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:37.377 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.152355+0000", 2024-09-21T14:59:37.377 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0020279870000000002 2024-09-21T14:59:37.377 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.377 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.377 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:37.377 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.152652+0000", 2024-09-21T14:59:37.377 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00029681 2024-09-21T14:59:37.377 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.377 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.378 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T14:59:37.378 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.153318+0000", 2024-09-21T14:59:37.378 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000666101 2024-09-21T14:59:37.378 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.378 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.378 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:37.378 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.156416+0000", 2024-09-21T14:59:37.378 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0030980729999999998 2024-09-21T14:59:37.378 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.378 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.378 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:59:37.378 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.156577+0000", 2024-09-21T14:59:37.378 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016049700000000001 2024-09-21T14:59:37.378 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.379 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.379 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:59:37.379 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.156628+0000", 2024-09-21T14:59:37.379 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0642000000000001e-05 2024-09-21T14:59:37.379 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.379 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.379 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:37.379 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.156687+0000", 2024-09-21T14:59:37.379 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9571000000000003e-05 2024-09-21T14:59:37.379 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:37.379 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:37.379 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:37.379 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.380 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.380 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:20462 3.b 3:d6a6f68e:::rbd_object_map.1136c84d9a13:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-21T14:59:37.380 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:45.150372+0000", 2024-09-21T14:59:37.380 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 52.176208705000001, 2024-09-21T14:59:37.380 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0075518110000000003, 2024-09-21T14:59:37.380 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:37.380 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:37.380 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:37.380 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:37.380 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:37.380 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 20462 2024-09-21T14:59:37.380 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.380 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:37.381 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.381 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:37.381 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150372+0000", 2024-09-21T14:59:37.381 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:37.381 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.381 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.381 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:37.381 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150372+0000", 2024-09-21T14:59:37.381 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:37.381 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.381 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.381 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:37.381 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150372+0000", 2024-09-21T14:59:37.381 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:37.381 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.382 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.382 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:37.382 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150378+0000", 2024-09-21T14:59:37.382 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3330000000000002e-06 2024-09-21T14:59:37.382 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.382 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.382 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:37.382 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150413+0000", 2024-09-21T14:59:37.382 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4798999999999997e-05 2024-09-21T14:59:37.382 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.382 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.382 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:37.382 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.150423+0000", 2024-09-21T14:59:37.382 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.8099999999999992e-06 2024-09-21T14:59:37.383 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.383 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.383 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:37.383 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.154550+0000", 2024-09-21T14:59:37.383 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0041275959999999999 2024-09-21T14:59:37.383 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.383 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.383 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:37.383 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.154758+0000", 2024-09-21T14:59:37.383 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00020815499999999999 2024-09-21T14:59:37.383 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.383 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.383 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T14:59:37.383 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.155323+0000", 2024-09-21T14:59:37.383 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00056425400000000004 2024-09-21T14:59:37.384 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.384 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.384 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:59:37.384 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.156702+0000", 2024-09-21T14:59:37.384 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001379676 2024-09-21T14:59:37.384 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.384 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.384 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:37.384 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.157662+0000", 2024-09-21T14:59:37.384 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00095948099999999998 2024-09-21T14:59:37.384 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.384 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.384 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:59:37.384 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.157816+0000", 2024-09-21T14:59:37.384 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000154666 2024-09-21T14:59:37.384 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.385 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.385 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:37.385 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:45.157924+0000", 2024-09-21T14:59:37.385 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000107798 2024-09-21T14:59:37.385 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:37.385 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:37.385 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:37.385 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.385 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.385 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31476 3.c 3:35718b6b:::rbd_data.11367721c19.0000000000000014:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T14:59:37.385 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.505975+0000", 2024-09-21T14:59:37.385 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 38.820606068000004, 2024-09-21T14:59:37.385 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99515995899999998, 2024-09-21T14:59:37.385 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:37.385 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:37.386 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:37.386 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:37.386 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:37.386 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31476 2024-09-21T14:59:37.386 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.386 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:37.386 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.386 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:37.386 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.505975+0000", 2024-09-21T14:59:37.386 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:37.386 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.386 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.386 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:37.387 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.505975+0000", 2024-09-21T14:59:37.387 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:37.387 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.387 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.387 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:37.387 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.505971+0000", 2024-09-21T14:59:37.387 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999967 2024-09-21T14:59:37.387 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.387 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.387 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:37.387 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.505989+0000", 2024-09-21T14:59:37.387 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7985000000000001e-05 2024-09-21T14:59:37.387 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.387 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.387 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:37.388 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506050+0000", 2024-09-21T14:59:37.388 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0659000000000003e-05 2024-09-21T14:59:37.388 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.388 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.388 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:37.388 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506074+0000", 2024-09-21T14:59:37.388 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4156e-05 2024-09-21T14:59:37.388 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.388 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.388 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:37.388 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506225+0000", 2024-09-21T14:59:37.388 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015097 2024-09-21T14:59:37.388 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.388 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.389 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T14:59:37.389 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506256+0000", 2024-09-21T14:59:37.389 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0505999999999999e-05 2024-09-21T14:59:37.389 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.389 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.389 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:37.389 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.497974+0000", 2024-09-21T14:59:37.389 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99171825700000005 2024-09-21T14:59:37.389 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.389 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.389 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for missing object", 2024-09-21T14:59:37.389 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.498449+0000", 2024-09-21T14:59:37.389 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00047506399999999998 2024-09-21T14:59:37.389 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.389 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.390 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:37.390 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.501000+0000", 2024-09-21T14:59:37.390 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0025506919999999998 2024-09-21T14:59:37.390 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.390 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.390 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:37.390 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.501135+0000", 2024-09-21T14:59:37.390 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013524000000000001 2024-09-21T14:59:37.390 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:37.390 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:37.390 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:37.390 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.390 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.390 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31480 3.c 3:36ccd808:::rbd_data.11367721c19.0000000000000018:head [sparse-read 0~14821] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T14:59:37.391 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.506116+0000", 2024-09-21T14:59:37.391 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 38.820464725000001, 2024-09-21T14:59:37.391 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99316033599999998, 2024-09-21T14:59:37.391 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:37.391 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:37.391 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:37.391 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:37.391 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:37.391 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31480 2024-09-21T14:59:37.391 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.391 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:37.391 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.391 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:37.391 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506116+0000", 2024-09-21T14:59:37.391 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:37.392 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.392 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.392 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:37.392 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506116+0000", 2024-09-21T14:59:37.392 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:37.392 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.392 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.392 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:37.392 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506115+0000", 2024-09-21T14:59:37.392 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:59:37.392 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.392 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.392 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:37.392 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506124+0000", 2024-09-21T14:59:37.392 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0659999999999993e-06 2024-09-21T14:59:37.393 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.393 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.393 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:37.393 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506145+0000", 2024-09-21T14:59:37.393 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1447e-05 2024-09-21T14:59:37.393 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.393 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.393 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:37.393 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506161+0000", 2024-09-21T14:59:37.393 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5209e-05 2024-09-21T14:59:37.393 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.393 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.393 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:37.393 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506337+0000", 2024-09-21T14:59:37.393 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017642999999999999 2024-09-21T14:59:37.394 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.394 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.394 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T14:59:37.394 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506375+0000", 2024-09-21T14:59:37.394 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7536000000000001e-05 2024-09-21T14:59:37.394 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.394 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.394 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:37.394 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.498815+0000", 2024-09-21T14:59:37.394 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99244067999999996 2024-09-21T14:59:37.394 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.394 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.394 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:37.395 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.498950+0000", 2024-09-21T14:59:37.395 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013436700000000001 2024-09-21T14:59:37.395 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.395 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.395 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:37.395 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499277+0000", 2024-09-21T14:59:37.395 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00032696800000000002 2024-09-21T14:59:37.395 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:37.395 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:37.395 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:37.395 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.395 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.395 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31513 3.c 3:36de8523:::rbd_data.11367721c19.0000000000000039:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T14:59:37.395 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.509071+0000", 2024-09-21T14:59:37.395 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 38.817510525000003, 2024-09-21T14:59:37.396 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99038225099999999, 2024-09-21T14:59:37.396 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:37.396 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:37.396 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:37.396 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:37.396 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:37.396 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31513 2024-09-21T14:59:37.396 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.396 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:37.396 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.396 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:37.396 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509071+0000", 2024-09-21T14:59:37.396 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:37.396 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.396 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.396 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:37.397 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509071+0000", 2024-09-21T14:59:37.397 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:37.397 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.397 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.397 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:37.397 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509068+0000", 2024-09-21T14:59:37.397 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-21T14:59:37.397 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.397 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.397 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:37.397 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509082+0000", 2024-09-21T14:59:37.397 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3545e-05 2024-09-21T14:59:37.397 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.397 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.398 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:37.398 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509116+0000", 2024-09-21T14:59:37.398 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3942000000000001e-05 2024-09-21T14:59:37.398 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.398 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.398 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:37.398 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509134+0000", 2024-09-21T14:59:37.398 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8839999999999999e-05 2024-09-21T14:59:37.398 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.398 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.398 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:37.398 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509318+0000", 2024-09-21T14:59:37.398 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018326799999999999 2024-09-21T14:59:37.398 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.398 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.399 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T14:59:37.399 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509346+0000", 2024-09-21T14:59:37.399 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8019000000000001e-05 2024-09-21T14:59:37.399 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.399 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.399 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:37.399 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499330+0000", 2024-09-21T14:59:37.399 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.98998399299999995 2024-09-21T14:59:37.399 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.399 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.399 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:37.399 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499453+0000", 2024-09-21T14:59:37.399 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000123115 2024-09-21T14:59:37.399 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:37.399 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:37.399 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:37.400 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.400 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.400 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31515 3.c 3:35b36ddd:::rbd_data.11367721c19.000000000000003b:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T14:59:37.400 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.509177+0000", 2024-09-21T14:59:37.400 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 38.817404099000001, 2024-09-21T14:59:37.400 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99044870600000001, 2024-09-21T14:59:37.400 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:37.400 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:37.400 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:37.400 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:37.400 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:37.400 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31515 2024-09-21T14:59:37.400 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.400 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:37.400 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.400 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:37.401 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509177+0000", 2024-09-21T14:59:37.401 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:37.401 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.401 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.401 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:37.401 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509177+0000", 2024-09-21T14:59:37.401 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:37.401 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.401 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.401 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:37.401 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509176+0000", 2024-09-21T14:59:37.401 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:37.401 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.401 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.401 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:37.402 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509183+0000", 2024-09-21T14:59:37.402 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6720000000000004e-06 2024-09-21T14:59:37.402 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.402 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.402 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:37.402 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509269+0000", 2024-09-21T14:59:37.402 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5881e-05 2024-09-21T14:59:37.402 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.402 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.402 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:37.402 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509303+0000", 2024-09-21T14:59:37.402 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3367000000000001e-05 2024-09-21T14:59:37.402 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.402 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.402 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:37.403 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509425+0000", 2024-09-21T14:59:37.403 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000122199 2024-09-21T14:59:37.403 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.403 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.403 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T14:59:37.403 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509448+0000", 2024-09-21T14:59:37.403 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2688999999999999e-05 2024-09-21T14:59:37.403 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.403 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.403 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:37.403 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499501+0000", 2024-09-21T14:59:37.403 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99005306999999998 2024-09-21T14:59:37.403 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.403 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.403 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:37.404 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499626+0000", 2024-09-21T14:59:37.404 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012501400000000001 2024-09-21T14:59:37.404 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:37.404 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:37.404 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:37.404 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.404 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.404 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31531 3.c 3:314f0287:::rbd_data.11367721c19.000000000000004b:head [sparse-read 0~7106] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T14:59:37.404 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.509329+0000", 2024-09-21T14:59:37.404 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 38.817252078999999, 2024-09-21T14:59:37.404 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99103085300000004, 2024-09-21T14:59:37.404 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:37.404 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:37.404 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:37.405 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:37.405 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:37.405 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31531 2024-09-21T14:59:37.405 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.405 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:37.405 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.405 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:37.405 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509329+0000", 2024-09-21T14:59:37.405 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:37.405 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.405 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.405 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:37.405 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509329+0000", 2024-09-21T14:59:37.405 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:37.405 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.405 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.406 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:37.406 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509328+0000", 2024-09-21T14:59:37.406 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:59:37.406 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.406 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.406 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:37.406 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509336+0000", 2024-09-21T14:59:37.406 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3170000000000002e-06 2024-09-21T14:59:37.406 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.406 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.406 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:37.406 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509353+0000", 2024-09-21T14:59:37.406 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7214000000000002e-05 2024-09-21T14:59:37.406 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.407 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.407 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:37.407 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509366+0000", 2024-09-21T14:59:37.407 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.26e-05 2024-09-21T14:59:37.407 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.407 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.407 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:37.407 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509523+0000", 2024-09-21T14:59:37.407 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015708000000000001 2024-09-21T14:59:37.407 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.407 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.407 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T14:59:37.407 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509544+0000", 2024-09-21T14:59:37.407 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0783000000000001e-05 2024-09-21T14:59:37.407 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.408 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.408 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:37.408 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499727+0000", 2024-09-21T14:59:37.408 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99018345200000002 2024-09-21T14:59:37.408 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.408 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.408 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:37.408 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499949+0000", 2024-09-21T14:59:37.408 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000221358 2024-09-21T14:59:37.408 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.408 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.408 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:37.408 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.500360+0000", 2024-09-21T14:59:37.408 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00041130700000000002 2024-09-21T14:59:37.408 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:37.409 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:37.409 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:37.409 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.409 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.409 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31605 3.c 3:34f43f7f:::rbd_data.113661d6691d.0000000000000034:head [delete] snapc 20=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-21T14:59:37.409 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:59.850318+0000", 2024-09-21T14:59:37.409 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 37.476263316999997, 2024-09-21T14:59:37.409 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0069284979999999999, 2024-09-21T14:59:37.409 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:37.409 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:37.409 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:37.409 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:37.409 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:37.409 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31605 2024-09-21T14:59:37.409 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.409 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:37.410 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.410 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:37.410 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.850318+0000", 2024-09-21T14:59:37.410 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:37.410 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.410 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.410 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:37.410 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.850318+0000", 2024-09-21T14:59:37.410 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:37.410 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.410 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.410 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:37.410 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.850314+0000", 2024-09-21T14:59:37.410 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999962 2024-09-21T14:59:37.410 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.411 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.411 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:37.411 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.850332+0000", 2024-09-21T14:59:37.411 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8414e-05 2024-09-21T14:59:37.411 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.411 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.411 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:37.411 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.850376+0000", 2024-09-21T14:59:37.411 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3989000000000002e-05 2024-09-21T14:59:37.411 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.411 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.411 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:37.411 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.850399+0000", 2024-09-21T14:59:37.411 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2455000000000001e-05 2024-09-21T14:59:37.411 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.412 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.412 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:37.412 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.850537+0000", 2024-09-21T14:59:37.412 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013858100000000001 2024-09-21T14:59:37.412 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.412 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.412 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:37.412 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.850756+0000", 2024-09-21T14:59:37.412 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021833099999999999 2024-09-21T14:59:37.412 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.412 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.412 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T14:59:37.412 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.850939+0000", 2024-09-21T14:59:37.412 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018285600000000001 2024-09-21T14:59:37.412 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.413 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.413 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:59:37.413 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.852297+0000", 2024-09-21T14:59:37.413 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0013588599999999999 2024-09-21T14:59:37.413 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.413 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.413 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:37.413 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.857067+0000", 2024-09-21T14:59:37.413 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0047691349999999999 2024-09-21T14:59:37.413 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.413 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.413 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:59:37.413 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.857143+0000", 2024-09-21T14:59:37.413 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6983000000000005e-05 2024-09-21T14:59:37.413 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.414 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.414 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:37.414 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.857246+0000", 2024-09-21T14:59:37.414 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010275799999999999 2024-09-21T14:59:37.414 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:37.414 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:37.414 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:37.414 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.414 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.414 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:61580 3.b 3:d3369e4f:::rbd_data.1136862f872f.0000000000000010:head [write 0~557056 [fadvise_sequential+fadvise_nocache],zero 557056~16384,write 573440~49152 [fadvise_sequential+fadvise_nocache],zero 622592~16384,write 638976~49152 [fadvise_sequential+fadvise_nocache],zero 688128~16384,write 704512~49152 [fadvise_sequential+fadvise_nocache],zero 753664~16384,write 770048~49152 [fadvise_sequential+fadvise_nocache],zero 819200~16384,write 835584~49152 [fadvise_sequential+fadvise_nocache],zero 884736~16384,write 901120~49152 [fadvise_sequential+fadvise_nocache],zero 950272~16384,write 966656~49152 [fadvise_sequential+fadvise_nocache],zero 1015808~16384,write 1032192~16384 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-21T14:59:37.414 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:19.231716+0000", 2024-09-21T14:59:37.414 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 18.094864916999999, 2024-09-21T14:59:37.414 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0096097690000000006, 2024-09-21T14:59:37.414 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:37.414 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:37.415 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:37.415 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:37.415 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:37.415 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 61580 2024-09-21T14:59:37.415 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.415 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:37.415 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.415 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:37.415 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.231716+0000", 2024-09-21T14:59:37.415 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:37.415 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.415 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.415 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:37.415 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.231716+0000", 2024-09-21T14:59:37.415 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:37.416 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.416 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.416 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:37.416 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.231715+0000", 2024-09-21T14:59:37.416 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:59:37.416 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.416 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.416 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:37.416 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.235290+0000", 2024-09-21T14:59:37.416 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0035750399999999998 2024-09-21T14:59:37.416 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.416 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.416 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:37.416 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.235310+0000", 2024-09-21T14:59:37.416 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0120999999999999e-05 2024-09-21T14:59:37.416 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.417 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.417 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:37.417 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.235319+0000", 2024-09-21T14:59:37.417 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2420000000000006e-06 2024-09-21T14:59:37.417 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.417 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.417 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:37.417 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.235416+0000", 2024-09-21T14:59:37.417 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7204e-05 2024-09-21T14:59:37.417 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.417 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.417 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:37.417 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.235775+0000", 2024-09-21T14:59:37.417 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000358883 2024-09-21T14:59:37.417 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.417 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.418 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T14:59:37.418 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.236560+0000", 2024-09-21T14:59:37.418 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000784463 2024-09-21T14:59:37.418 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.418 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.418 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:59:37.418 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.241119+0000", 2024-09-21T14:59:37.418 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0045590589999999999 2024-09-21T14:59:37.418 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.418 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.418 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:37.418 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.241210+0000", 2024-09-21T14:59:37.418 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1617000000000001e-05 2024-09-21T14:59:37.418 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.418 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.419 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:59:37.419 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.241285+0000", 2024-09-21T14:59:37.419 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4470999999999999e-05 2024-09-21T14:59:37.419 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.419 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.419 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:37.419 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.241326+0000", 2024-09-21T14:59:37.419 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0967000000000003e-05 2024-09-21T14:59:37.419 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:37.419 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:37.419 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:37.419 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.419 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.419 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:62659 3.b 3:d350b07f:::rbd_data.11366ec8a7a7.0000000000000016:head [sparse-read 0~10240] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-21T14:59:37.419 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:24.459995+0000", 2024-09-21T14:59:37.419 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 12.866586214, 2024-09-21T14:59:37.420 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.006552942, 2024-09-21T14:59:37.420 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:37.420 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:37.420 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:37.420 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:37.420 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:37.420 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 62659 2024-09-21T14:59:37.420 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.420 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:37.420 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.420 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:37.420 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:24.459995+0000", 2024-09-21T14:59:37.420 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:37.420 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.420 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.420 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:37.421 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:24.459995+0000", 2024-09-21T14:59:37.421 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:37.421 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.421 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.421 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:37.421 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:24.459994+0000", 2024-09-21T14:59:37.421 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:37.421 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.421 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.421 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:37.421 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:24.460000+0000", 2024-09-21T14:59:37.421 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5989999999999998e-06 2024-09-21T14:59:37.421 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.421 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.421 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:37.422 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:24.460017+0000", 2024-09-21T14:59:37.422 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6617e-05 2024-09-21T14:59:37.422 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.422 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.422 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:37.422 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:24.460031+0000", 2024-09-21T14:59:37.422 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3716e-05 2024-09-21T14:59:37.422 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.422 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.422 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:37.422 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:24.465400+0000", 2024-09-21T14:59:37.422 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0053697249999999997 2024-09-21T14:59:37.422 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.422 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.422 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:37.423 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:24.465592+0000", 2024-09-21T14:59:37.423 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019164299999999999 2024-09-21T14:59:37.423 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:37.423 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:37.423 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:37.423 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:24.466548+0000", 2024-09-21T14:59:37.423 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00095565799999999996 2024-09-21T14:59:37.423 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:37.423 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:37.423 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:37.423 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:37.423 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:37.423 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:59:37.424 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T14:59:37.425 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:59:37.419+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:59:37.460 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 5, 6, 7, 4] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T14:59:37.460 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T14:59:37.460 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-21T14:59:37.460 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-21T14:59:37.464 INFO:teuthology.orchestra.run.smithi032.stdout:checking clone #15, image image_client.0-clone15 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent16 2024-09-21T14:59:37.526 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:59:37.525+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:59:37.540 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:37.541 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:59:37.541 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T14:59:37.541 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:37.552 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T14:59:37.627 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:59:37.623+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:59:37.714 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:37.714 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:59:37.714 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T14:59:37.714 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T14:59:37.714 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:37.724 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T14:59:37.727 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:59:37.727+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:59:37.804 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-21T14:59:37.804 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:37.804 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 99, 2024-09-21T14:59:37.804 INFO:teuthology.orchestra.run.smithi032.stdout: "fsid": "caf51e55-9234-4a26-bcd2-d60811962560", 2024-09-21T14:59:37.804 INFO:teuthology.orchestra.run.smithi032.stdout: "created": "2024-09-21T14:57:50.338822+0000", 2024-09-21T14:59:37.804 INFO:teuthology.orchestra.run.smithi032.stdout: "modified": "2024-09-21T14:59:37.275493+0000", 2024-09-21T14:59:37.804 INFO:teuthology.orchestra.run.smithi032.stdout: "last_up_change": "2024-09-21T14:58:01.177141+0000", 2024-09-21T14:59:37.804 INFO:teuthology.orchestra.run.smithi032.stdout: "last_in_change": "2024-09-21T14:59:27.207073+0000", 2024-09-21T14:59:37.804 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-21T14:59:37.804 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_num": 5804032, 2024-09-21T14:59:37.804 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_set": [ 2024-09-21T14:59:37.804 INFO:teuthology.orchestra.run.smithi032.stdout: "nodeep-scrub", 2024-09-21T14:59:37.804 INFO:teuthology.orchestra.run.smithi032.stdout: "pglog_hardlimit", 2024-09-21T14:59:37.805 INFO:teuthology.orchestra.run.smithi032.stdout: "purged_snapdirs", 2024-09-21T14:59:37.805 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_deletes", 2024-09-21T14:59:37.805 INFO:teuthology.orchestra.run.smithi032.stdout: "sortbitwise" 2024-09-21T14:59:37.805 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-21T14:59:37.805 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_version": 5, 2024-09-21T14:59:37.805 INFO:teuthology.orchestra.run.smithi032.stdout: "full_ratio": 0.94999998807907104, 2024-09-21T14:59:37.805 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-21T14:59:37.805 INFO:teuthology.orchestra.run.smithi032.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-21T14:59:37.805 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_snapshot": "", 2024-09-21T14:59:37.805 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_max": 3, 2024-09-21T14:59:37.805 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd": 8, 2024-09-21T14:59:37.805 INFO:teuthology.orchestra.run.smithi032.stdout: "require_min_compat_client": "luminous", 2024-09-21T14:59:37.805 INFO:teuthology.orchestra.run.smithi032.stdout: "min_compat_client": "luminous", 2024-09-21T14:59:37.805 INFO:teuthology.orchestra.run.smithi032.stdout: "require_osd_release": "reef", 2024-09-21T14:59:37.805 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_crimson": false, 2024-09-21T14:59:37.806 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2024-09-21T14:59:37.806 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.806 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 1, 2024-09-21T14:59:37.806 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": ".mgr", 2024-09-21T14:59:37.806 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-09-21T14:58:01.842810+0000", 2024-09-21T14:59:37.806 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-09-21T14:59:37.806 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2024-09-21T14:59:37.806 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-09-21T14:59:37.806 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-09-21T14:59:37.806 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-09-21T14:59:37.806 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-09-21T14:59:37.806 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-09-21T14:59:37.806 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-09-21T14:59:37.806 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T14:59:37.807 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T14:59:37.807 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-09-21T14:59:37.807 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-09-21T14:59:37.807 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 1, 2024-09-21T14:59:37.807 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 1, 2024-09-21T14:59:37.807 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 1, 2024-09-21T14:59:37.807 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 1, 2024-09-21T14:59:37.807 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 1, 2024-09-21T14:59:37.807 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-09-21T14:59:37.807 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-09-21T14:59:37.807 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-09-21T14:59:37.807 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-09-21T14:59:37.807 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-09-21T14:59:37.807 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-09-21T14:59:37.807 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-09-21T14:59:37.807 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.808 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "18", 2024-09-21T14:59:37.808 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-09-21T14:59:37.808 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T14:59:37.808 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T14:59:37.808 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-09-21T14:59:37.808 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-09-21T14:59:37.808 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-09-21T14:59:37.808 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-09-21T14:59:37.808 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-09-21T14:59:37.808 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-09-21T14:59:37.808 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-09-21T14:59:37.808 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-09-21T14:59:37.808 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-09-21T14:59:37.808 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-09-21T14:59:37.808 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-09-21T14:59:37.809 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-09-21T14:59:37.809 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-09-21T14:59:37.809 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-09-21T14:59:37.809 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-09-21T14:59:37.809 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T14:59:37.809 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T14:59:37.809 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T14:59:37.809 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-09-21T14:59:37.809 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-09-21T14:59:37.809 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-09-21T14:59:37.809 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-09-21T14:59:37.809 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-09-21T14:59:37.809 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.809 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-09-21T14:59:37.809 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-09-21T14:59:37.810 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-09-21T14:59:37.810 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-09-21T14:59:37.810 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-09-21T14:59:37.810 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T14:59:37.810 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-09-21T14:59:37.810 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-09-21T14:59:37.810 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-09-21T14:59:37.810 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-09-21T14:59:37.810 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-09-21T14:59:37.810 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-09-21T14:59:37.810 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_max": 32, 2024-09-21T14:59:37.810 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 1 2024-09-21T14:59:37.810 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.810 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-09-21T14:59:37.811 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": {} 2024-09-21T14:59:37.811 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.811 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-09-21T14:59:37.811 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 8, 2024-09-21T14:59:37.811 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 8, 2024-09-21T14:59:37.811 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 0.25, 2024-09-21T14:59:37.811 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 2, 2024-09-21T14:59:37.811 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 2, 2024-09-21T14:59:37.811 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 0.88126564025878906, 2024-09-21T14:59:37.811 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 0.87999999523162842, 2024-09-21T14:59:37.811 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-21T14:59:37.811 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:37.811 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.811 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.811 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 2, 2024-09-21T14:59:37.812 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "rbd", 2024-09-21T14:59:37.812 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-09-21T14:58:08.284925+0000", 2024-09-21T14:59:37.812 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 8193, 2024-09-21T14:59:37.812 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T14:59:37.812 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-09-21T14:59:37.812 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-09-21T14:59:37.812 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-09-21T14:59:37.812 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-09-21T14:59:37.812 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-09-21T14:59:37.812 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-09-21T14:59:37.812 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T14:59:37.812 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T14:59:37.812 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-09-21T14:59:37.812 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-09-21T14:59:37.812 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2024-09-21T14:59:37.813 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2024-09-21T14:59:37.813 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2024-09-21T14:59:37.813 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2024-09-21T14:59:37.813 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2024-09-21T14:59:37.813 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-09-21T14:59:37.813 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-09-21T14:59:37.813 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-09-21T14:59:37.813 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-09-21T14:59:37.813 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-09-21T14:59:37.813 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-09-21T14:59:37.813 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-09-21T14:59:37.813 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.813 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "22", 2024-09-21T14:59:37.813 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-09-21T14:59:37.813 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T14:59:37.814 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T14:59:37.814 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-09-21T14:59:37.814 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-09-21T14:59:37.814 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 3, 2024-09-21T14:59:37.814 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 22, 2024-09-21T14:59:37.814 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-09-21T14:59:37.814 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-09-21T14:59:37.814 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-09-21T14:59:37.814 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-09-21T14:59:37.814 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-09-21T14:59:37.814 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-09-21T14:59:37.814 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-09-21T14:59:37.814 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-09-21T14:59:37.815 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-09-21T14:59:37.815 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-09-21T14:59:37.815 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-09-21T14:59:37.815 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T14:59:37.815 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T14:59:37.815 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T14:59:37.815 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-09-21T14:59:37.815 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-09-21T14:59:37.815 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-09-21T14:59:37.815 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-09-21T14:59:37.815 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-09-21T14:59:37.815 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.815 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-09-21T14:59:37.815 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-09-21T14:59:37.816 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-09-21T14:59:37.816 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-09-21T14:59:37.816 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-09-21T14:59:37.816 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T14:59:37.816 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-09-21T14:59:37.816 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-09-21T14:59:37.816 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-09-21T14:59:37.816 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-09-21T14:59:37.816 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-09-21T14:59:37.816 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2024-09-21T14:59:37.816 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-09-21T14:59:37.816 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd": {} 2024-09-21T14:59:37.816 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.816 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-09-21T14:59:37.816 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 4.9800000190734863, 2024-09-21T14:59:37.817 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 4.9800000190734863, 2024-09-21T14:59:37.817 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 0.87999999523162842, 2024-09-21T14:59:37.817 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 4.380000114440918, 2024-09-21T14:59:37.817 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 4.380000114440918, 2024-09-21T14:59:37.817 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 0.88126564025878906, 2024-09-21T14:59:37.817 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 0.87999999523162842, 2024-09-21T14:59:37.817 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-21T14:59:37.817 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:37.817 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.817 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.817 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 3, 2024-09-21T14:59:37.817 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "pool_client.0", 2024-09-21T14:59:37.817 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-09-21T14:58:17.510709+0000", 2024-09-21T14:59:37.817 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 8193, 2024-09-21T14:59:37.817 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T14:59:37.817 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-09-21T14:59:37.818 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-09-21T14:59:37.818 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-09-21T14:59:37.818 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-09-21T14:59:37.818 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-09-21T14:59:37.818 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-09-21T14:59:37.818 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T14:59:37.818 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T14:59:37.818 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-09-21T14:59:37.818 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-09-21T14:59:37.818 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 32, 2024-09-21T14:59:37.818 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 32, 2024-09-21T14:59:37.818 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 32, 2024-09-21T14:59:37.818 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 32, 2024-09-21T14:59:37.818 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 32, 2024-09-21T14:59:37.818 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-09-21T14:59:37.819 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-09-21T14:59:37.819 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-09-21T14:59:37.819 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-09-21T14:59:37.819 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-09-21T14:59:37.819 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-09-21T14:59:37.819 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-09-21T14:59:37.819 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.819 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "99", 2024-09-21T14:59:37.819 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-09-21T14:59:37.819 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T14:59:37.819 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T14:59:37.819 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-09-21T14:59:37.819 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-09-21T14:59:37.819 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 54, 2024-09-21T14:59:37.820 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 99, 2024-09-21T14:59:37.820 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-09-21T14:59:37.820 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-09-21T14:59:37.820 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-09-21T14:59:37.820 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-09-21T14:59:37.820 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-09-21T14:59:37.820 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-09-21T14:59:37.820 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-09-21T14:59:37.820 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-09-21T14:59:37.820 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-09-21T14:59:37.820 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-09-21T14:59:37.820 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-09-21T14:59:37.820 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T14:59:37.820 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T14:59:37.820 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T14:59:37.820 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-09-21T14:59:37.821 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-09-21T14:59:37.821 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-09-21T14:59:37.821 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-09-21T14:59:37.821 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-09-21T14:59:37.821 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.821 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-09-21T14:59:37.821 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-09-21T14:59:37.821 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-09-21T14:59:37.821 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-09-21T14:59:37.821 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-09-21T14:59:37.821 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T14:59:37.821 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-09-21T14:59:37.821 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-09-21T14:59:37.821 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-09-21T14:59:37.821 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-09-21T14:59:37.822 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-09-21T14:59:37.822 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2024-09-21T14:59:37.822 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-09-21T14:59:37.822 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd": {} 2024-09-21T14:59:37.822 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.822 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-09-21T14:59:37.822 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 2.25, 2024-09-21T14:59:37.822 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 2.25, 2024-09-21T14:59:37.822 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 1, 2024-09-21T14:59:37.823 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 2.25, 2024-09-21T14:59:37.823 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 2.25, 2024-09-21T14:59:37.823 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 0.88126564025878906, 2024-09-21T14:59:37.823 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 0.87999999523162842, 2024-09-21T14:59:37.823 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-21T14:59:37.823 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:37.823 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:37.823 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-21T14:59:37.823 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2024-09-21T14:59:37.823 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.823 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-09-21T14:59:37.823 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "a4a73316-3a94-42e9-9a51-deb46a66bb06", 2024-09-21T14:59:37.823 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-21T14:59:37.823 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-21T14:59:37.823 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-21T14:59:37.824 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-21T14:59:37.824 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-21T14:59:37.824 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-21T14:59:37.824 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2024-09-21T14:59:37.824 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 93, 2024-09-21T14:59:37.824 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-21T14:59:37.824 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-21T14:59:37.824 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-21T14:59:37.824 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:59:37.824 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.824 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:59:37.824 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6808", 2024-09-21T14:59:37.824 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4016672243 2024-09-21T14:59:37.824 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.824 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.825 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:59:37.825 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6809", 2024-09-21T14:59:37.825 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4016672243 2024-09-21T14:59:37.825 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:37.825 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:37.825 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.825 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-21T14:59:37.825 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:59:37.825 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.825 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:59:37.825 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6810", 2024-09-21T14:59:37.825 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4016672243 2024-09-21T14:59:37.825 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.825 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.826 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:59:37.826 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6811", 2024-09-21T14:59:37.826 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4016672243 2024-09-21T14:59:37.826 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:37.826 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:37.826 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.826 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-21T14:59:37.826 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:59:37.826 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.826 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:59:37.826 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6814", 2024-09-21T14:59:37.826 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4016672243 2024-09-21T14:59:37.827 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.827 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.827 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:59:37.827 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6815", 2024-09-21T14:59:37.827 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4016672243 2024-09-21T14:59:37.827 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:37.827 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:37.827 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.827 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-21T14:59:37.827 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:59:37.827 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.827 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:59:37.827 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6812", 2024-09-21T14:59:37.828 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4016672243 2024-09-21T14:59:37.828 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.828 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.828 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:59:37.828 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6813", 2024-09-21T14:59:37.828 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4016672243 2024-09-21T14:59:37.828 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:37.828 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:37.828 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.828 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6809/4016672243", 2024-09-21T14:59:37.828 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6811/4016672243", 2024-09-21T14:59:37.828 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6815/4016672243", 2024-09-21T14:59:37.828 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6813/4016672243", 2024-09-21T14:59:37.829 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-21T14:59:37.829 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-21T14:59:37.829 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-21T14:59:37.829 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:37.829 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.829 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.829 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-09-21T14:59:37.829 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "006757a0-94d1-4a9b-aef8-73634cc5828e", 2024-09-21T14:59:37.829 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-21T14:59:37.829 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-21T14:59:37.829 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-21T14:59:37.829 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-21T14:59:37.829 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-21T14:59:37.829 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-21T14:59:37.830 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2024-09-21T14:59:37.830 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 93, 2024-09-21T14:59:37.830 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-21T14:59:37.830 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-21T14:59:37.830 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-21T14:59:37.830 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:59:37.830 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.830 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:59:37.830 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6824", 2024-09-21T14:59:37.830 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 641090577 2024-09-21T14:59:37.830 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.830 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.830 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:59:37.830 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6825", 2024-09-21T14:59:37.831 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 641090577 2024-09-21T14:59:37.831 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:37.831 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:37.831 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.831 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-21T14:59:37.831 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:59:37.831 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.831 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:59:37.831 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6826", 2024-09-21T14:59:37.831 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 641090577 2024-09-21T14:59:37.831 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.831 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.831 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:59:37.831 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6827", 2024-09-21T14:59:37.831 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 641090577 2024-09-21T14:59:37.832 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:37.832 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:37.832 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.832 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-21T14:59:37.832 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:59:37.832 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.832 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:59:37.832 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6830", 2024-09-21T14:59:37.832 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 641090577 2024-09-21T14:59:37.832 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.832 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.832 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:59:37.832 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6831", 2024-09-21T14:59:37.832 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 641090577 2024-09-21T14:59:37.832 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:37.832 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:37.833 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.833 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-21T14:59:37.833 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:59:37.833 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.833 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:59:37.833 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6828", 2024-09-21T14:59:37.833 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 641090577 2024-09-21T14:59:37.833 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.833 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.833 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:59:37.833 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6829", 2024-09-21T14:59:37.833 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 641090577 2024-09-21T14:59:37.833 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:37.833 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:37.833 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.834 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6825/641090577", 2024-09-21T14:59:37.834 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6827/641090577", 2024-09-21T14:59:37.834 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6831/641090577", 2024-09-21T14:59:37.834 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6829/641090577", 2024-09-21T14:59:37.834 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-21T14:59:37.834 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-21T14:59:37.834 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-21T14:59:37.834 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:37.834 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.834 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.834 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-09-21T14:59:37.834 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "0b499a23-820e-4e6c-a558-adbf88b7014f", 2024-09-21T14:59:37.834 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-21T14:59:37.834 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-21T14:59:37.835 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 0.3514556884765625, 2024-09-21T14:59:37.835 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-21T14:59:37.835 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-21T14:59:37.835 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-21T14:59:37.835 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2024-09-21T14:59:37.835 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 51, 2024-09-21T14:59:37.835 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-21T14:59:37.835 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-21T14:59:37.835 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-21T14:59:37.835 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:59:37.835 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.835 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:59:37.835 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6800", 2024-09-21T14:59:37.835 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2609154414 2024-09-21T14:59:37.836 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.836 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.836 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:59:37.836 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6801", 2024-09-21T14:59:37.836 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2609154414 2024-09-21T14:59:37.836 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:37.836 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:37.836 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.836 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-21T14:59:37.836 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:59:37.836 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.836 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:59:37.836 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6802", 2024-09-21T14:59:37.836 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2609154414 2024-09-21T14:59:37.836 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.836 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.837 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:59:37.837 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6803", 2024-09-21T14:59:37.837 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2609154414 2024-09-21T14:59:37.837 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:37.837 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:37.837 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.837 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-21T14:59:37.837 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:59:37.837 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.837 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:59:37.837 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6806", 2024-09-21T14:59:37.837 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2609154414 2024-09-21T14:59:37.837 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.838 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.838 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:59:37.838 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6807", 2024-09-21T14:59:37.838 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2609154414 2024-09-21T14:59:37.838 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:37.838 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:37.838 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.838 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-21T14:59:37.838 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:59:37.838 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.838 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:59:37.838 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6804", 2024-09-21T14:59:37.838 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2609154414 2024-09-21T14:59:37.838 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.838 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.839 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:59:37.839 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6805", 2024-09-21T14:59:37.839 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2609154414 2024-09-21T14:59:37.839 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:37.839 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:37.839 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.839 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6801/2609154414", 2024-09-21T14:59:37.839 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6803/2609154414", 2024-09-21T14:59:37.839 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6807/2609154414", 2024-09-21T14:59:37.839 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6805/2609154414", 2024-09-21T14:59:37.839 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-21T14:59:37.839 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-21T14:59:37.839 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-21T14:59:37.840 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:37.840 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.840 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.840 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-09-21T14:59:37.840 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "5a57b3dd-4a07-45b7-b159-3cac4c7eb262", 2024-09-21T14:59:37.840 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-21T14:59:37.840 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-21T14:59:37.840 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-21T14:59:37.840 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-21T14:59:37.840 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-21T14:59:37.840 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-21T14:59:37.840 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2024-09-21T14:59:37.840 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 94, 2024-09-21T14:59:37.840 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-21T14:59:37.840 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-21T14:59:37.841 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-21T14:59:37.841 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:59:37.841 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.841 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:59:37.841 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6816", 2024-09-21T14:59:37.841 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1462563376 2024-09-21T14:59:37.841 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.841 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.841 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:59:37.841 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6817", 2024-09-21T14:59:37.841 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1462563376 2024-09-21T14:59:37.841 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:37.841 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:37.841 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.841 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-21T14:59:37.841 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:59:37.842 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.842 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:59:37.842 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6818", 2024-09-21T14:59:37.842 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1462563376 2024-09-21T14:59:37.842 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.842 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.842 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:59:37.842 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6819", 2024-09-21T14:59:37.842 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1462563376 2024-09-21T14:59:37.842 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:37.842 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:37.842 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.842 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-21T14:59:37.842 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:59:37.843 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.843 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:59:37.843 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6822", 2024-09-21T14:59:37.843 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1462563376 2024-09-21T14:59:37.843 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.843 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.843 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:59:37.843 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6823", 2024-09-21T14:59:37.843 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1462563376 2024-09-21T14:59:37.843 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:37.843 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:37.843 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.843 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-21T14:59:37.844 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:59:37.844 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.844 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:59:37.844 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6820", 2024-09-21T14:59:37.844 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1462563376 2024-09-21T14:59:37.844 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.844 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.844 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:59:37.844 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6821", 2024-09-21T14:59:37.844 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1462563376 2024-09-21T14:59:37.844 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:37.844 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:37.844 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.845 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6817/1462563376", 2024-09-21T14:59:37.845 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6819/1462563376", 2024-09-21T14:59:37.845 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6823/1462563376", 2024-09-21T14:59:37.845 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6821/1462563376", 2024-09-21T14:59:37.845 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-21T14:59:37.845 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-21T14:59:37.845 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-21T14:59:37.845 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:37.845 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.845 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.845 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-09-21T14:59:37.845 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "39a00807-7517-474e-8842-623dd7093cc5", 2024-09-21T14:59:37.845 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-21T14:59:37.846 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-21T14:59:37.846 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-21T14:59:37.846 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-21T14:59:37.846 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-21T14:59:37.846 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-21T14:59:37.846 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2024-09-21T14:59:37.846 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 94, 2024-09-21T14:59:37.846 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-21T14:59:37.846 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-21T14:59:37.846 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-21T14:59:37.846 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:59:37.846 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.846 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:59:37.846 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6801", 2024-09-21T14:59:37.846 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 144412042 2024-09-21T14:59:37.847 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.847 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.847 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:59:37.847 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6803", 2024-09-21T14:59:37.847 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 144412042 2024-09-21T14:59:37.847 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:37.847 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:37.847 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.847 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-21T14:59:37.847 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:59:37.847 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.847 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:59:37.847 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6805", 2024-09-21T14:59:37.847 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 144412042 2024-09-21T14:59:37.847 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.847 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.848 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:59:37.848 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6807", 2024-09-21T14:59:37.848 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 144412042 2024-09-21T14:59:37.848 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:37.848 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:37.848 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.848 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-21T14:59:37.848 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:59:37.848 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.848 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:59:37.848 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6812", 2024-09-21T14:59:37.848 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 144412042 2024-09-21T14:59:37.848 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.848 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.848 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:59:37.848 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6814", 2024-09-21T14:59:37.849 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 144412042 2024-09-21T14:59:37.849 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:37.849 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:37.849 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.849 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-21T14:59:37.849 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:59:37.849 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.849 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:59:37.849 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6809", 2024-09-21T14:59:37.849 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 144412042 2024-09-21T14:59:37.849 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.849 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.849 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:59:37.849 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6811", 2024-09-21T14:59:37.850 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 144412042 2024-09-21T14:59:37.850 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:37.850 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:37.850 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.850 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.46:6803/144412042", 2024-09-21T14:59:37.850 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.46:6807/144412042", 2024-09-21T14:59:37.850 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.46:6814/144412042", 2024-09-21T14:59:37.850 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.46:6811/144412042", 2024-09-21T14:59:37.850 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-21T14:59:37.850 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-21T14:59:37.850 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-21T14:59:37.850 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:37.850 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.850 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.851 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-09-21T14:59:37.851 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "edf9f045-ddeb-4504-a0fc-1a0e4e853550", 2024-09-21T14:59:37.851 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-21T14:59:37.851 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-21T14:59:37.851 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-21T14:59:37.851 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-21T14:59:37.851 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-21T14:59:37.851 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-21T14:59:37.851 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2024-09-21T14:59:37.851 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 75, 2024-09-21T14:59:37.851 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-21T14:59:37.851 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-21T14:59:37.851 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-21T14:59:37.851 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:59:37.851 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.852 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:59:37.852 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6800", 2024-09-21T14:59:37.852 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 849101442 2024-09-21T14:59:37.852 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.852 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.852 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:59:37.852 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6802", 2024-09-21T14:59:37.852 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 849101442 2024-09-21T14:59:37.852 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:37.852 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:37.852 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.852 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-21T14:59:37.852 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:59:37.852 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.853 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:59:37.853 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6804", 2024-09-21T14:59:37.853 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 849101442 2024-09-21T14:59:37.853 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.853 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.853 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:59:37.853 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6806", 2024-09-21T14:59:37.853 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 849101442 2024-09-21T14:59:37.853 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:37.853 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:37.853 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.853 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-21T14:59:37.853 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:59:37.853 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.853 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:59:37.854 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6813", 2024-09-21T14:59:37.854 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 849101442 2024-09-21T14:59:37.854 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.854 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.854 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:59:37.854 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6815", 2024-09-21T14:59:37.854 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 849101442 2024-09-21T14:59:37.854 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:37.854 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:37.854 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.854 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-21T14:59:37.854 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:59:37.854 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.854 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:59:37.855 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6808", 2024-09-21T14:59:37.855 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 849101442 2024-09-21T14:59:37.855 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.855 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.855 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:59:37.855 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6810", 2024-09-21T14:59:37.855 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 849101442 2024-09-21T14:59:37.855 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:37.855 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:37.855 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.855 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.46:6802/849101442", 2024-09-21T14:59:37.855 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.46:6806/849101442", 2024-09-21T14:59:37.855 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.46:6815/849101442", 2024-09-21T14:59:37.855 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.46:6810/849101442", 2024-09-21T14:59:37.856 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-21T14:59:37.856 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-21T14:59:37.856 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-21T14:59:37.856 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:37.856 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.856 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.856 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-09-21T14:59:37.856 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "c820ea6b-88fe-4421-83ec-17c466691fea", 2024-09-21T14:59:37.856 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-21T14:59:37.856 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-21T14:59:37.856 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-21T14:59:37.856 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 0.0501251220703125, 2024-09-21T14:59:37.856 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-21T14:59:37.856 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-21T14:59:37.856 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2024-09-21T14:59:37.857 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 86, 2024-09-21T14:59:37.857 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-21T14:59:37.857 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-21T14:59:37.857 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-21T14:59:37.857 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:59:37.857 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.857 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:59:37.857 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6817", 2024-09-21T14:59:37.857 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 847189172 2024-09-21T14:59:37.857 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.857 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.857 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:59:37.857 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6818", 2024-09-21T14:59:37.857 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 847189172 2024-09-21T14:59:37.857 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:37.858 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:37.858 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.858 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-21T14:59:37.858 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:59:37.858 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.858 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:59:37.858 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6820", 2024-09-21T14:59:37.858 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 847189172 2024-09-21T14:59:37.858 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.858 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.858 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:59:37.858 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6822", 2024-09-21T14:59:37.858 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 847189172 2024-09-21T14:59:37.858 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:37.859 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:37.859 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.859 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-21T14:59:37.859 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:59:37.859 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.859 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:59:37.859 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6827", 2024-09-21T14:59:37.859 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 847189172 2024-09-21T14:59:37.859 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.859 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.859 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:59:37.859 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6828", 2024-09-21T14:59:37.859 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 847189172 2024-09-21T14:59:37.859 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:37.860 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:37.860 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.860 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-21T14:59:37.860 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:59:37.860 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.860 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:59:37.860 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6823", 2024-09-21T14:59:37.860 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 847189172 2024-09-21T14:59:37.860 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.860 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.860 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:59:37.860 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6825", 2024-09-21T14:59:37.860 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 847189172 2024-09-21T14:59:37.860 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:37.860 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:37.861 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.861 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.46:6818/847189172", 2024-09-21T14:59:37.861 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.46:6822/847189172", 2024-09-21T14:59:37.861 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.46:6828/847189172", 2024-09-21T14:59:37.861 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.46:6825/847189172", 2024-09-21T14:59:37.861 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-21T14:59:37.861 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-21T14:59:37.861 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-21T14:59:37.861 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:37.861 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.861 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.861 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-09-21T14:59:37.861 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "3a2f82a4-28a0-4fcb-9d6b-2285fa4d07ac", 2024-09-21T14:59:37.861 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-21T14:59:37.861 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-21T14:59:37.862 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 0.95001220703125, 2024-09-21T14:59:37.862 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-21T14:59:37.862 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-21T14:59:37.862 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-21T14:59:37.862 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 15, 2024-09-21T14:59:37.862 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 81, 2024-09-21T14:59:37.862 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-21T14:59:37.862 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-21T14:59:37.862 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-21T14:59:37.862 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:59:37.862 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.862 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:59:37.862 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6816", 2024-09-21T14:59:37.862 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3894963553 2024-09-21T14:59:37.863 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.863 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.863 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:59:37.863 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6819", 2024-09-21T14:59:37.863 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3894963553 2024-09-21T14:59:37.863 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:37.863 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:37.863 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.863 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-21T14:59:37.863 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:59:37.863 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.863 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:59:37.863 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6821", 2024-09-21T14:59:37.863 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3894963553 2024-09-21T14:59:37.864 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.864 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.864 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:59:37.864 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6824", 2024-09-21T14:59:37.864 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3894963553 2024-09-21T14:59:37.864 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:37.864 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:37.864 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.864 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-21T14:59:37.864 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:59:37.864 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.864 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:59:37.864 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6830", 2024-09-21T14:59:37.864 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3894963553 2024-09-21T14:59:37.864 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.865 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.865 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:59:37.865 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6831", 2024-09-21T14:59:37.865 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3894963553 2024-09-21T14:59:37.865 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:37.865 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:37.865 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.865 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-21T14:59:37.865 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T14:59:37.865 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.865 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T14:59:37.865 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6826", 2024-09-21T14:59:37.865 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3894963553 2024-09-21T14:59:37.865 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.865 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.866 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T14:59:37.866 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6829", 2024-09-21T14:59:37.866 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3894963553 2024-09-21T14:59:37.866 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:37.866 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:37.866 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.866 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.46:6819/3894963553", 2024-09-21T14:59:37.866 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.46:6824/3894963553", 2024-09-21T14:59:37.866 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.46:6831/3894963553", 2024-09-21T14:59:37.866 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.46:6829/3894963553", 2024-09-21T14:59:37.866 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-21T14:59:37.866 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-21T14:59:37.866 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-21T14:59:37.867 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:37.867 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:37.867 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-21T14:59:37.867 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_xinfo": [ 2024-09-21T14:59:37.867 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.867 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-09-21T14:59:37.867 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-21T14:59:37.867 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-21T14:59:37.867 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-21T14:59:37.867 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-21T14:59:37.867 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-21T14:59:37.867 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-21T14:57:57.458317+0000", 2024-09-21T14:59:37.867 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-21T14:59:37.867 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.867 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.868 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-09-21T14:59:37.868 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-21T14:59:37.868 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-21T14:59:37.868 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-21T14:59:37.868 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-21T14:59:37.868 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-21T14:59:37.868 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-21T14:57:57.907624+0000", 2024-09-21T14:59:37.868 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-21T14:59:37.868 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.868 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.868 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-09-21T14:59:37.868 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-21T14:59:37.868 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-21T14:59:37.868 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-21T14:59:37.868 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-21T14:59:37.869 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-21T14:59:37.869 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-21T14:57:57.278817+0000", 2024-09-21T14:59:37.869 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-21T14:59:37.869 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.869 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.869 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-09-21T14:59:37.869 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-21T14:59:37.869 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-21T14:59:37.869 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-21T14:59:37.869 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-21T14:59:37.869 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-21T14:59:37.869 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-21T14:57:57.500449+0000", 2024-09-21T14:59:37.869 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-21T14:59:37.869 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.869 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.869 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-09-21T14:59:37.870 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-21T14:59:37.870 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-21T14:59:37.870 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-21T14:59:37.870 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-21T14:59:37.870 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-21T14:59:37.870 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-21T14:57:57.504325+0000", 2024-09-21T14:59:37.870 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-21T14:59:37.870 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.870 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.870 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-09-21T14:59:37.870 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-21T14:59:37.870 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-21T14:59:37.870 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-21T14:59:37.870 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-21T14:59:37.870 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-21T14:59:37.871 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-21T14:57:57.777525+0000", 2024-09-21T14:59:37.871 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-21T14:59:37.871 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.871 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.871 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-09-21T14:59:37.871 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-21T14:59:37.871 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-21T14:59:37.871 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-21T14:59:37.871 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-21T14:59:37.871 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-21T14:59:37.871 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-21T14:57:57.698961+0000", 2024-09-21T14:59:37.871 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-21T14:59:37.871 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.872 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.872 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-09-21T14:59:37.872 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-21T14:59:37.872 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-21T14:59:37.872 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-21T14:59:37.872 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-21T14:59:37.872 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-21T14:59:37.872 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-21T14:57:57.932625+0000", 2024-09-21T14:59:37.872 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-21T14:59:37.872 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:37.872 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-21T14:59:37.872 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap": [ 2024-09-21T14:59:37.872 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.872 INFO:teuthology.orchestra.run.smithi032.stdout: "pgid": "3.2", 2024-09-21T14:59:37.872 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2024-09-21T14:59:37.873 INFO:teuthology.orchestra.run.smithi032.stdout: 2, 2024-09-21T14:59:37.873 INFO:teuthology.orchestra.run.smithi032.stdout: 3 2024-09-21T14:59:37.873 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:37.873 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:37.873 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-21T14:59:37.873 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_items": [], 2024-09-21T14:59:37.873 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_primaries": [], 2024-09-21T14:59:37.873 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_temp": [], 2024-09-21T14:59:37.873 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_temp": [], 2024-09-21T14:59:37.873 INFO:teuthology.orchestra.run.smithi032.stdout: "blocklist": {}, 2024-09-21T14:59:37.873 INFO:teuthology.orchestra.run.smithi032.stdout: "range_blocklist": {}, 2024-09-21T14:59:37.873 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profiles": { 2024-09-21T14:59:37.873 INFO:teuthology.orchestra.run.smithi032.stdout: "default": { 2024-09-21T14:59:37.873 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2024-09-21T14:59:37.873 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2024-09-21T14:59:37.873 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "1", 2024-09-21T14:59:37.874 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2024-09-21T14:59:37.874 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van" 2024-09-21T14:59:37.874 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:37.874 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.874 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps_queue": [ 2024-09-21T14:59:37.874 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.874 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 3, 2024-09-21T14:59:37.874 INFO:teuthology.orchestra.run.smithi032.stdout: "snaps": [ 2024-09-21T14:59:37.874 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.874 INFO:teuthology.orchestra.run.smithi032.stdout: "begin": 50, 2024-09-21T14:59:37.874 INFO:teuthology.orchestra.run.smithi032.stdout: "length": 1 2024-09-21T14:59:37.874 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.874 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.874 INFO:teuthology.orchestra.run.smithi032.stdout: "begin": 53, 2024-09-21T14:59:37.875 INFO:teuthology.orchestra.run.smithi032.stdout: "length": 1 2024-09-21T14:59:37.875 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:37.875 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:37.875 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:37.875 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-21T14:59:37.875 INFO:teuthology.orchestra.run.smithi032.stdout: "new_removed_snaps": [ 2024-09-21T14:59:37.875 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.875 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 3, 2024-09-21T14:59:37.875 INFO:teuthology.orchestra.run.smithi032.stdout: "snaps": [ 2024-09-21T14:59:37.875 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.875 INFO:teuthology.orchestra.run.smithi032.stdout: "begin": 53, 2024-09-21T14:59:37.875 INFO:teuthology.orchestra.run.smithi032.stdout: "length": 1 2024-09-21T14:59:37.875 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:37.875 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:37.875 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:37.875 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-21T14:59:37.876 INFO:teuthology.orchestra.run.smithi032.stdout: "new_purged_snaps": [], 2024-09-21T14:59:37.876 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_node_flags": {}, 2024-09-21T14:59:37.876 INFO:teuthology.orchestra.run.smithi032.stdout: "device_class_flags": {}, 2024-09-21T14:59:37.876 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode": { 2024-09-21T14:59:37.876 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_enabled": false, 2024-09-21T14:59:37.876 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_bucket_count": 0, 2024-09-21T14:59:37.876 INFO:teuthology.orchestra.run.smithi032.stdout: "degraded_stretch_mode": 0, 2024-09-21T14:59:37.876 INFO:teuthology.orchestra.run.smithi032.stdout: "recovering_stretch_mode": 0, 2024-09-21T14:59:37.876 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_bucket": 0 2024-09-21T14:59:37.876 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:37.876 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:37.878 INFO:tasks.thrashosds.thrasher:j is {'epoch': 99, 'fsid': 'caf51e55-9234-4a26-bcd2-d60811962560', 'created': '2024-09-21T14:57:50.338822+0000', 'modified': '2024-09-21T14:59:37.275493+0000', 'last_up_change': '2024-09-21T14:58:01.177141+0000', 'last_in_change': '2024-09-21T14:59:27.207073+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T14:58:01.842810+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.8812656402587891, 'average_primary_affinity': 0.8799999952316284, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-21T14:58:08.284925+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 22, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 0.8812656402587891, 'average_primary_affinity': 0.8799999952316284, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-21T14:58:17.510709+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '99', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 54, 'snap_epoch': 99, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.8812656402587891, 'average_primary_affinity': 0.8799999952316284, 'average_primary_affinity_weighted': 0.8799999952316284}}], 'osds': [{'osd': 0, 'uuid': 'a4a73316-3a94-42e9-9a51-deb46a66bb06', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 93, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6808', 'nonce': 4016672243}, {'type': 'v1', 'addr': '172.21.15.32:6809', 'nonce': 4016672243}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6810', 'nonce': 4016672243}, {'type': 'v1', 'addr': '172.21.15.32:6811', 'nonce': 4016672243}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6814', 'nonce': 4016672243}, {'type': 'v1', 'addr': '172.21.15.32:6815', 'nonce': 4016672243}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6812', 'nonce': 4016672243}, {'type': 'v1', 'addr': '172.21.15.32:6813', 'nonce': 4016672243}]}, 'public_addr': '172.21.15.32:6809/4016672243', 'cluster_addr': '172.21.15.32:6811/4016672243', 'heartbeat_back_addr': '172.21.15.32:6815/4016672243', 'heartbeat_front_addr': '172.21.15.32:6813/4016672243', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '006757a0-94d1-4a9b-aef8-73634cc5828e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 93, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6824', 'nonce': 641090577}, {'type': 'v1', 'addr': '172.21.15.32:6825', 'nonce': 641090577}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6826', 'nonce': 641090577}, {'type': 'v1', 'addr': '172.21.15.32:6827', 'nonce': 641090577}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6830', 'nonce': 641090577}, {'type': 'v1', 'addr': '172.21.15.32:6831', 'nonce': 641090577}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6828', 'nonce': 641090577}, {'type': 'v1', 'addr': '172.21.15.32:6829', 'nonce': 641090577}]}, 'public_addr': '172.21.15.32:6825/641090577', 'cluster_addr': '172.21.15.32:6827/641090577', 'heartbeat_back_addr': '172.21.15.32:6831/641090577', 'heartbeat_front_addr': '172.21.15.32:6829/641090577', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '0b499a23-820e-4e6c-a558-adbf88b7014f', 'up': 1, 'in': 1, 'weight': 0.3514556884765625, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 51, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6800', 'nonce': 2609154414}, {'type': 'v1', 'addr': '172.21.15.32:6801', 'nonce': 2609154414}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6802', 'nonce': 2609154414}, {'type': 'v1', 'addr': '172.21.15.32:6803', 'nonce': 2609154414}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6806', 'nonce': 2609154414}, {'type': 'v1', 'addr': '172.21.15.32:6807', 'nonce': 2609154414}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6804', 'nonce': 2609154414}, {'type': 'v1', 'addr': '172.21.15.32:6805', 'nonce': 2609154414}]}, 'public_addr': '172.21.15.32:6801/2609154414', 'cluster_addr': '172.21.15.32:6803/2609154414', 'heartbeat_back_addr': '172.21.15.32:6807/2609154414', 'heartbeat_front_addr': '172.21.15.32:6805/2609154414', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '5a57b3dd-4a07-45b7-b159-3cac4c7eb262', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 94, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6816', 'nonce': 1462563376}, {'type': 'v1', 'addr': '172.21.15.32:6817', 'nonce': 1462563376}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6818', 'nonce': 1462563376}, {'type': 'v1', 'addr': '172.21.15.32:6819', 'nonce': 1462563376}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6822', 'nonce': 1462563376}, {'type': 'v1', 'addr': '172.21.15.32:6823', 'nonce': 1462563376}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6820', 'nonce': 1462563376}, {'type': 'v1', 'addr': '172.21.15.32:6821', 'nonce': 1462563376}]}, 'public_addr': '172.21.15.32:6817/1462563376', 'cluster_addr': '172.21.15.32:6819/1462563376', 'heartbeat_back_addr': '172.21.15.32:6823/1462563376', 'heartbeat_front_addr': '172.21.15.32:6821/1462563376', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '39a00807-7517-474e-8842-623dd7093cc5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 94, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6801', 'nonce': 144412042}, {'type': 'v1', 'addr': '172.21.15.46:6803', 'nonce': 144412042}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6805', 'nonce': 144412042}, {'type': 'v1', 'addr': '172.21.15.46:6807', 'nonce': 144412042}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6812', 'nonce': 144412042}, {'type': 'v1', 'addr': '172.21.15.46:6814', 'nonce': 144412042}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6809', 'nonce': 144412042}, {'type': 'v1', 'addr': '172.21.15.46:6811', 'nonce': 144412042}]}, 'public_addr': '172.21.15.46:6803/144412042', 'cluster_addr': '172.21.15.46:6807/144412042', 'heartbeat_back_addr': '172.21.15.46:6814/144412042', 'heartbeat_front_addr': '172.21.15.46:6811/144412042', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'edf9f045-ddeb-4504-a0fc-1a0e4e853550', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 75, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6800', 'nonce': 849101442}, {'type': 'v1', 'addr': '172.21.15.46:6802', 'nonce': 849101442}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6804', 'nonce': 849101442}, {'type': 'v1', 'addr': '172.21.15.46:6806', 'nonce': 849101442}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6813', 'nonce': 849101442}, {'type': 'v1', 'addr': '172.21.15.46:6815', 'nonce': 849101442}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6808', 'nonce': 849101442}, {'type': 'v1', 'addr': '172.21.15.46:6810', 'nonce': 849101442}]}, 'public_addr': '172.21.15.46:6802/849101442', 'cluster_addr': '172.21.15.46:6806/849101442', 'heartbeat_back_addr': '172.21.15.46:6815/849101442', 'heartbeat_front_addr': '172.21.15.46:6810/849101442', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'c820ea6b-88fe-4421-83ec-17c466691fea', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.0501251220703125, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 86, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6817', 'nonce': 847189172}, {'type': 'v1', 'addr': '172.21.15.46:6818', 'nonce': 847189172}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6820', 'nonce': 847189172}, {'type': 'v1', 'addr': '172.21.15.46:6822', 'nonce': 847189172}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6827', 'nonce': 847189172}, {'type': 'v1', 'addr': '172.21.15.46:6828', 'nonce': 847189172}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6823', 'nonce': 847189172}, {'type': 'v1', 'addr': '172.21.15.46:6825', 'nonce': 847189172}]}, 'public_addr': '172.21.15.46:6818/847189172', 'cluster_addr': '172.21.15.46:6822/847189172', 'heartbeat_back_addr': '172.21.15.46:6828/847189172', 'heartbeat_front_addr': '172.21.15.46:6825/847189172', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '3a2f82a4-28a0-4fcb-9d6b-2285fa4d07ac', 'up': 1, 'in': 1, 'weight': 0.95001220703125, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 81, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6816', 'nonce': 3894963553}, {'type': 'v1', 'addr': '172.21.15.46:6819', 'nonce': 3894963553}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6821', 'nonce': 3894963553}, {'type': 'v1', 'addr': '172.21.15.46:6824', 'nonce': 3894963553}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6830', 'nonce': 3894963553}, {'type': 'v1', 'addr': '172.21.15.46:6831', 'nonce': 3894963553}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6826', 'nonce': 3894963553}, {'type': 'v1', 'addr': '172.21.15.46:6829', 'nonce': 3894963553}]}, 'public_addr': '172.21.15.46:6819/3894963553', 'cluster_addr': '172.21.15.46:6824/3894963553', 'heartbeat_back_addr': '172.21.15.46:6831/3894963553', 'heartbeat_front_addr': '172.21.15.46:6829/3894963553', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:57:57.458317+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:57:57.907624+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:57:57.278817+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:57:57.500449+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:57:57.504325+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:57:57.777525+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:57:57.698961+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:57:57.932625+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.2', 'osds': [2, 3]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 50, 'length': 1}, {'begin': 53, 'length': 1}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 53, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-21T14:59:37.878 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T14:59:37.880 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:37.880 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T14:59:37.880 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T14:59:37.880 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T14:59:37.880 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.880 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:1473 3.15 3:af2afa6a:::rbd_data.1136515ecb6b.000000000000001b:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T14:59:37.880 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:30.421730+0000", 2024-09-21T14:59:37.880 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 67.408593154000002, 2024-09-21T14:59:37.880 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021478937, 2024-09-21T14:59:37.880 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:37.880 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:37.881 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:37.881 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:37.881 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:37.881 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1473 2024-09-21T14:59:37.881 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.881 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:37.881 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.881 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:37.881 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421730+0000", 2024-09-21T14:59:37.881 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:37.881 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.881 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.881 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:37.882 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421730+0000", 2024-09-21T14:59:37.882 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:37.882 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.882 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.882 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:37.882 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421728+0000", 2024-09-21T14:59:37.882 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-21T14:59:37.882 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.882 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.882 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:37.882 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421739+0000", 2024-09-21T14:59:37.882 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0502e-05 2024-09-21T14:59:37.882 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.882 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.883 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:37.883 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421762+0000", 2024-09-21T14:59:37.883 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3405999999999999e-05 2024-09-21T14:59:37.883 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.883 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.883 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:37.883 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421773+0000", 2024-09-21T14:59:37.883 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0744000000000001e-05 2024-09-21T14:59:37.883 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.883 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.883 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:37.883 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421838+0000", 2024-09-21T14:59:37.883 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5486000000000003e-05 2024-09-21T14:59:37.883 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.884 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.884 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:37.884 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421955+0000", 2024-09-21T14:59:37.884 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000116765 2024-09-21T14:59:37.884 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.884 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.884 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 6", 2024-09-21T14:59:37.884 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.425572+0000", 2024-09-21T14:59:37.884 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0036171380000000002 2024-09-21T14:59:37.884 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.884 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.884 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:37.884 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.442973+0000", 2024-09-21T14:59:37.885 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.017400268 2024-09-21T14:59:37.885 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.885 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.885 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:37.885 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.442996+0000", 2024-09-21T14:59:37.885 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3139e-05 2024-09-21T14:59:37.885 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.885 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.885 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:37.885 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.443079+0000", 2024-09-21T14:59:37.885 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2832999999999998e-05 2024-09-21T14:59:37.885 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.885 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.885 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:37.886 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.443209+0000", 2024-09-21T14:59:37.886 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013052499999999999 2024-09-21T14:59:37.886 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:37.886 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:37.886 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:37.886 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.886 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.886 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:1678 3.c 3:377cc7e1:::rbd_data.1136515ecb6b.000000000000000f:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T14:59:37.886 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:30.988085+0000", 2024-09-21T14:59:37.886 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 66.842238559999998, 2024-09-21T14:59:37.886 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.032974802999999997, 2024-09-21T14:59:37.886 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:37.886 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:37.887 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:37.887 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:37.887 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:37.887 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1678 2024-09-21T14:59:37.887 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.887 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:37.887 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.887 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:37.887 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988085+0000", 2024-09-21T14:59:37.887 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:37.887 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.887 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.887 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:37.887 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988085+0000", 2024-09-21T14:59:37.888 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:37.888 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.888 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.888 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:37.888 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988083+0000", 2024-09-21T14:59:37.888 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-21T14:59:37.888 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.888 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.888 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:37.888 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988093+0000", 2024-09-21T14:59:37.888 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8800000000000003e-06 2024-09-21T14:59:37.888 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.888 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.889 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:37.889 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988113+0000", 2024-09-21T14:59:37.889 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0217999999999999e-05 2024-09-21T14:59:37.889 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.889 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.889 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:37.889 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988123+0000", 2024-09-21T14:59:37.889 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5859999999999994e-06 2024-09-21T14:59:37.889 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.889 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.889 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:37.889 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988203+0000", 2024-09-21T14:59:37.889 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0834000000000006e-05 2024-09-21T14:59:37.890 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.890 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.890 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:37.890 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988376+0000", 2024-09-21T14:59:37.890 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017301299999999999 2024-09-21T14:59:37.890 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.890 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.890 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 7", 2024-09-21T14:59:37.890 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.996338+0000", 2024-09-21T14:59:37.890 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0079618520000000002 2024-09-21T14:59:37.890 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.890 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.890 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:37.891 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.020881+0000", 2024-09-21T14:59:37.891 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024542731000000002 2024-09-21T14:59:37.891 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.891 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.891 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:37.891 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.020904+0000", 2024-09-21T14:59:37.891 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2861e-05 2024-09-21T14:59:37.891 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.891 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.891 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:37.891 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.020987+0000", 2024-09-21T14:59:37.891 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2806000000000006e-05 2024-09-21T14:59:37.891 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.891 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.892 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:37.892 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.021060+0000", 2024-09-21T14:59:37.892 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2871000000000001e-05 2024-09-21T14:59:37.892 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:37.892 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:37.892 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:37.892 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.892 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.892 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9369 3.3 3:c725132c:::rbd_data.11363a21d2be.0000000000000007:10 [read 24576~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:37.892 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157241+0000", 2024-09-21T14:59:37.892 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 58.673082114000003, 2024-09-21T14:59:37.892 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018289156000000001, 2024-09-21T14:59:37.892 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:37.892 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:37.893 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:37.893 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:37.893 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:37.893 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9369 2024-09-21T14:59:37.893 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.893 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:37.893 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.893 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:37.893 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157241+0000", 2024-09-21T14:59:37.893 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:37.893 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.893 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.893 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:37.893 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157241+0000", 2024-09-21T14:59:37.894 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:37.894 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.894 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.894 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:37.894 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157241+0000", 2024-09-21T14:59:37.894 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:37.894 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.894 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.894 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:37.894 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157244+0000", 2024-09-21T14:59:37.894 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5049999999999999e-06 2024-09-21T14:59:37.894 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.895 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.895 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:37.895 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157255+0000", 2024-09-21T14:59:37.895 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0797e-05 2024-09-21T14:59:37.895 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.895 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.895 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:37.895 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157260+0000", 2024-09-21T14:59:37.895 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1749999999999996e-06 2024-09-21T14:59:37.895 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.895 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.895 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:37.895 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175237+0000", 2024-09-21T14:59:37.895 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.017976405000000001 2024-09-21T14:59:37.895 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.896 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.896 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:37.896 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175331+0000", 2024-09-21T14:59:37.896 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4508999999999999e-05 2024-09-21T14:59:37.896 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.896 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.896 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:37.896 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175530+0000", 2024-09-21T14:59:37.896 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00019928200000000001 2024-09-21T14:59:37.896 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:37.896 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:37.896 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:37.896 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.896 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.896 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9370 3.3 3:c725132c:::rbd_data.11363a21d2be.0000000000000007:10 [read 163840~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:37.897 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157279+0000", 2024-09-21T14:59:37.897 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 58.673043935999999, 2024-09-21T14:59:37.897 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018472379000000001, 2024-09-21T14:59:37.897 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:37.897 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:37.897 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:37.897 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:37.897 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:37.897 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9370 2024-09-21T14:59:37.897 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.897 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:37.897 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.897 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:37.897 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157279+0000", 2024-09-21T14:59:37.898 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:37.898 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.898 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.898 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:37.898 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157279+0000", 2024-09-21T14:59:37.898 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:37.898 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.898 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.898 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:37.898 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157279+0000", 2024-09-21T14:59:37.898 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:37.898 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.898 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.898 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:37.899 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157282+0000", 2024-09-21T14:59:37.899 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2229999999999999e-06 2024-09-21T14:59:37.899 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.899 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.899 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:37.899 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157294+0000", 2024-09-21T14:59:37.899 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2173e-05 2024-09-21T14:59:37.899 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.899 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.899 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:37.899 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157300+0000", 2024-09-21T14:59:37.899 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.93e-06 2024-09-21T14:59:37.899 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.899 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.899 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:37.900 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175569+0000", 2024-09-21T14:59:37.900 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018268376999999999 2024-09-21T14:59:37.900 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.900 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.900 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:37.900 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175622+0000", 2024-09-21T14:59:37.900 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3808999999999999e-05 2024-09-21T14:59:37.900 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.900 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.900 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:37.900 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175752+0000", 2024-09-21T14:59:37.900 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012938700000000001 2024-09-21T14:59:37.900 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:37.900 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:37.900 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:37.901 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.901 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.901 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9371 3.3 3:c0bf6a9b:::rbd_data.11363a21d2be.0000000000000000:10 [read 73728~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:37.901 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157306+0000", 2024-09-21T14:59:37.901 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 58.673017825000002, 2024-09-21T14:59:37.901 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018755473000000002, 2024-09-21T14:59:37.901 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:37.901 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:37.901 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:37.901 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:37.901 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:37.901 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9371 2024-09-21T14:59:37.901 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.901 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:37.901 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.902 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:37.902 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157306+0000", 2024-09-21T14:59:37.902 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:37.902 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.902 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.902 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:37.902 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157306+0000", 2024-09-21T14:59:37.902 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:37.902 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.902 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.902 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:37.902 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157305+0000", 2024-09-21T14:59:37.902 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:37.903 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.903 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.903 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:37.903 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157312+0000", 2024-09-21T14:59:37.903 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3289999999999996e-06 2024-09-21T14:59:37.903 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.903 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.903 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:37.903 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157324+0000", 2024-09-21T14:59:37.903 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1725e-05 2024-09-21T14:59:37.903 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.903 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.903 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:37.903 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157339+0000", 2024-09-21T14:59:37.904 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5211999999999999e-05 2024-09-21T14:59:37.904 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.904 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.904 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:37.904 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175798+0000", 2024-09-21T14:59:37.904 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018458781 2024-09-21T14:59:37.904 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.904 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.904 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:37.904 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175899+0000", 2024-09-21T14:59:37.904 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000100784 2024-09-21T14:59:37.904 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.904 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.904 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:37.904 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176061+0000", 2024-09-21T14:59:37.905 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00016216099999999999 2024-09-21T14:59:37.905 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:37.905 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:37.905 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:37.905 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.905 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.905 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9372 3.3 3:c0bf6a9b:::rbd_data.11363a21d2be.0000000000000000:10 [read 212992~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:37.905 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157345+0000", 2024-09-21T14:59:37.905 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 58.672978479000001, 2024-09-21T14:59:37.905 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018926493999999999, 2024-09-21T14:59:37.905 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:37.905 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:37.905 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:37.905 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:37.905 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:37.906 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9372 2024-09-21T14:59:37.906 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.906 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:37.906 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.906 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:37.906 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157345+0000", 2024-09-21T14:59:37.906 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:37.906 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.906 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.906 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:37.906 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157345+0000", 2024-09-21T14:59:37.906 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:37.906 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.906 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.907 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:37.907 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157344+0000", 2024-09-21T14:59:37.907 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:37.907 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.907 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.907 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:37.907 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157348+0000", 2024-09-21T14:59:37.907 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1389999999999999e-06 2024-09-21T14:59:37.907 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.907 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.907 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:37.907 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157372+0000", 2024-09-21T14:59:37.907 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4711000000000001e-05 2024-09-21T14:59:37.907 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.908 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.908 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:37.908 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157378+0000", 2024-09-21T14:59:37.908 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9909999999999999e-06 2024-09-21T14:59:37.908 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.908 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.908 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:37.908 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176090+0000", 2024-09-21T14:59:37.908 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018712111999999999 2024-09-21T14:59:37.908 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.908 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.908 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:37.908 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176134+0000", 2024-09-21T14:59:37.908 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3377999999999997e-05 2024-09-21T14:59:37.908 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.909 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.909 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:37.909 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176271+0000", 2024-09-21T14:59:37.909 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013763499999999999 2024-09-21T14:59:37.909 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:37.909 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:37.909 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:37.909 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.909 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.909 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9373 3.3 3:c0bf6a9b:::rbd_data.11363a21d2be.0000000000000000:10 [read 352256~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:37.909 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157384+0000", 2024-09-21T14:59:37.909 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 58.672939706999998, 2024-09-21T14:59:37.909 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019154937, 2024-09-21T14:59:37.909 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:37.909 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:37.910 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:37.910 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:37.910 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:37.910 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9373 2024-09-21T14:59:37.910 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.910 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:37.910 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.910 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:37.910 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157384+0000", 2024-09-21T14:59:37.910 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:37.910 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.910 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.910 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:37.910 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157384+0000", 2024-09-21T14:59:37.911 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:37.911 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.911 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.911 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:37.911 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157383+0000", 2024-09-21T14:59:37.911 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:37.911 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.911 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.911 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:37.911 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157386+0000", 2024-09-21T14:59:37.911 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.191e-06 2024-09-21T14:59:37.911 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.911 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.911 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:37.912 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157393+0000", 2024-09-21T14:59:37.912 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5339999999999996e-06 2024-09-21T14:59:37.912 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.912 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.912 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:37.912 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157397+0000", 2024-09-21T14:59:37.912 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5220000000000001e-06 2024-09-21T14:59:37.912 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.912 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.912 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:37.912 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176302+0000", 2024-09-21T14:59:37.912 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018905113000000001 2024-09-21T14:59:37.912 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.912 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.912 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:37.913 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176379+0000", 2024-09-21T14:59:37.913 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6372999999999995e-05 2024-09-21T14:59:37.913 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.913 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.913 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:37.913 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176539+0000", 2024-09-21T14:59:37.913 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015973800000000001 2024-09-21T14:59:37.913 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:37.913 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:37.913 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:37.913 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.913 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.913 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9374 3.3 3:c0bf6a9b:::rbd_data.11363a21d2be.0000000000000000:10 [read 491520~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:37.913 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157401+0000", 2024-09-21T14:59:37.913 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 58.672921922999997, 2024-09-21T14:59:37.914 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019387755, 2024-09-21T14:59:37.914 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:37.914 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:37.914 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:37.914 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:37.914 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:37.914 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9374 2024-09-21T14:59:37.914 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.914 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:37.914 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.914 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:37.914 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157401+0000", 2024-09-21T14:59:37.914 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:37.914 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.914 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.915 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:37.915 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157401+0000", 2024-09-21T14:59:37.915 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:37.915 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.915 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.915 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:37.915 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157401+0000", 2024-09-21T14:59:37.915 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:37.915 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.915 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.915 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:37.915 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157414+0000", 2024-09-21T14:59:37.915 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2417e-05 2024-09-21T14:59:37.916 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.916 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.916 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:37.916 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157424+0000", 2024-09-21T14:59:37.916 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0861e-05 2024-09-21T14:59:37.916 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.916 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.916 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:37.916 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157429+0000", 2024-09-21T14:59:37.916 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0499999999999999e-06 2024-09-21T14:59:37.916 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.916 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.916 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:37.916 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176596+0000", 2024-09-21T14:59:37.916 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019166552 2024-09-21T14:59:37.917 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.917 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.917 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:37.917 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176647+0000", 2024-09-21T14:59:37.917 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1112000000000001e-05 2024-09-21T14:59:37.917 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.917 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.917 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:37.917 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176789+0000", 2024-09-21T14:59:37.917 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014209300000000001 2024-09-21T14:59:37.917 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:37.917 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:37.917 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:37.917 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.917 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.917 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9401 3.3 3:c725132c:::rbd_data.11363a21d2be.0000000000000007:10 [read 32768~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:37.918 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157621+0000", 2024-09-21T14:59:37.918 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 58.672702114000003, 2024-09-21T14:59:37.918 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019381711999999999, 2024-09-21T14:59:37.918 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:37.918 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:37.918 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:37.918 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:37.918 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:37.918 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9401 2024-09-21T14:59:37.918 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.918 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:37.918 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.918 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:37.918 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157621+0000", 2024-09-21T14:59:37.918 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:37.919 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.919 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.919 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:37.919 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157621+0000", 2024-09-21T14:59:37.919 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:37.919 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.919 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.919 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:37.919 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157620+0000", 2024-09-21T14:59:37.919 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:59:37.919 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.919 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.919 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:37.919 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157626+0000", 2024-09-21T14:59:37.919 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7570000000000001e-06 2024-09-21T14:59:37.920 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.920 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.920 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:37.920 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157636+0000", 2024-09-21T14:59:37.920 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8339999999999994e-06 2024-09-21T14:59:37.920 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.920 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.920 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:37.920 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157643+0000", 2024-09-21T14:59:37.920 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9439999999999999e-06 2024-09-21T14:59:37.920 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.920 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.920 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:37.920 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176816+0000", 2024-09-21T14:59:37.920 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019172872000000001 2024-09-21T14:59:37.920 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.921 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.921 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:37.921 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176878+0000", 2024-09-21T14:59:37.921 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1734000000000002e-05 2024-09-21T14:59:37.921 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.921 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.921 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:37.921 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177003+0000", 2024-09-21T14:59:37.921 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012537400000000001 2024-09-21T14:59:37.921 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:37.921 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:37.921 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:37.921 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.921 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.921 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9402 3.3 3:c725132c:::rbd_data.11363a21d2be.0000000000000007:10 [read 172032~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:37.922 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157649+0000", 2024-09-21T14:59:37.922 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 58.672674241999999, 2024-09-21T14:59:37.922 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019567404, 2024-09-21T14:59:37.922 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:37.922 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:37.922 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:37.922 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:37.922 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:37.922 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9402 2024-09-21T14:59:37.922 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.922 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:37.922 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.922 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:37.922 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157649+0000", 2024-09-21T14:59:37.923 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:37.923 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.923 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.923 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:37.923 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157649+0000", 2024-09-21T14:59:37.923 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:37.923 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.923 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.923 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:37.923 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157649+0000", 2024-09-21T14:59:37.923 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:37.923 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.923 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.923 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:37.923 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157652+0000", 2024-09-21T14:59:37.924 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8779999999999998e-06 2024-09-21T14:59:37.924 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.924 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.924 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:37.924 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157657+0000", 2024-09-21T14:59:37.924 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6659999999999996e-06 2024-09-21T14:59:37.924 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.924 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.924 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:37.924 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157662+0000", 2024-09-21T14:59:37.924 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5519999999999998e-06 2024-09-21T14:59:37.924 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.924 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.924 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:37.924 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177040+0000", 2024-09-21T14:59:37.925 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019377807 2024-09-21T14:59:37.925 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.925 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.925 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:37.925 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177085+0000", 2024-09-21T14:59:37.925 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5195999999999999e-05 2024-09-21T14:59:37.925 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.925 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.925 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:37.925 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177217+0000", 2024-09-21T14:59:37.925 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013172600000000001 2024-09-21T14:59:37.925 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:37.925 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:37.925 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:37.925 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.925 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.926 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9403 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:37.926 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157949+0000", 2024-09-21T14:59:37.926 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 58.672373913000001, 2024-09-21T14:59:37.926 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023540373999999999, 2024-09-21T14:59:37.926 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:37.926 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:37.926 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:37.926 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:37.926 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:37.926 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9403 2024-09-21T14:59:37.926 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.926 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:37.926 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.926 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:37.927 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157949+0000", 2024-09-21T14:59:37.927 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:37.927 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.927 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.927 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:37.927 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157949+0000", 2024-09-21T14:59:37.927 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:37.927 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.927 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.927 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:37.927 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157948+0000", 2024-09-21T14:59:37.927 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T14:59:37.927 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.927 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.927 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:37.928 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157956+0000", 2024-09-21T14:59:37.928 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7999999999999999e-06 2024-09-21T14:59:37.928 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.928 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.928 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:37.928 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157969+0000", 2024-09-21T14:59:37.928 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2792000000000001e-05 2024-09-21T14:59:37.928 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.928 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.928 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:37.928 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157977+0000", 2024-09-21T14:59:37.928 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.704e-06 2024-09-21T14:59:37.928 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.928 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.928 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:37.928 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177246+0000", 2024-09-21T14:59:37.929 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019268264 2024-09-21T14:59:37.929 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.929 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.929 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:37.929 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177315+0000", 2024-09-21T14:59:37.929 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9480999999999994e-05 2024-09-21T14:59:37.929 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.929 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.929 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:59:37.929 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177615+0000", 2024-09-21T14:59:37.929 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00029929499999999999 2024-09-21T14:59:37.929 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.929 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.929 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:37.929 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181333+0000", 2024-09-21T14:59:37.930 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0037183149999999998 2024-09-21T14:59:37.930 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.930 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.930 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:37.930 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181374+0000", 2024-09-21T14:59:37.930 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.0723999999999998e-05 2024-09-21T14:59:37.930 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.930 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.930 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:37.930 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181434+0000", 2024-09-21T14:59:37.930 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0024e-05 2024-09-21T14:59:37.930 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.930 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.930 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:37.931 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181490+0000", 2024-09-21T14:59:37.931 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6245000000000001e-05 2024-09-21T14:59:37.931 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:37.931 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:37.931 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:37.931 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.931 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.931 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9404 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:37.931 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.160480+0000", 2024-09-21T14:59:37.931 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 58.669843563000001, 2024-09-21T14:59:37.931 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021137686999999999, 2024-09-21T14:59:37.931 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:37.931 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:37.931 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:37.932 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:37.932 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:37.932 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9404 2024-09-21T14:59:37.932 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.932 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:37.932 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.932 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:37.932 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160480+0000", 2024-09-21T14:59:37.932 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:37.932 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.932 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.932 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:37.932 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160480+0000", 2024-09-21T14:59:37.932 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:37.933 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.933 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.933 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:37.933 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160478+0000", 2024-09-21T14:59:37.933 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T14:59:37.933 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.933 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.933 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:37.933 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160486+0000", 2024-09-21T14:59:37.933 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9969999999999998e-06 2024-09-21T14:59:37.933 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.933 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.933 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:37.933 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160502+0000", 2024-09-21T14:59:37.934 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5650000000000001e-05 2024-09-21T14:59:37.934 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.934 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.934 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:37.934 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160511+0000", 2024-09-21T14:59:37.934 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.3610000000000007e-06 2024-09-21T14:59:37.934 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.934 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.934 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:37.934 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.178035+0000", 2024-09-21T14:59:37.934 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.017523520000000001 2024-09-21T14:59:37.934 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.934 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.934 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:37.935 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.178151+0000", 2024-09-21T14:59:37.935 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000116013 2024-09-21T14:59:37.935 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.935 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.935 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:59:37.935 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.178408+0000", 2024-09-21T14:59:37.935 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00025743400000000002 2024-09-21T14:59:37.935 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.935 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.935 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:37.935 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181501+0000", 2024-09-21T14:59:37.935 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.003092891 2024-09-21T14:59:37.935 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.935 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.936 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:37.936 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181538+0000", 2024-09-21T14:59:37.936 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6977999999999997e-05 2024-09-21T14:59:37.936 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.936 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.936 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:37.936 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181582+0000", 2024-09-21T14:59:37.936 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3849e-05 2024-09-21T14:59:37.936 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.936 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.936 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:37.936 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181617+0000", 2024-09-21T14:59:37.936 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5423000000000003e-05 2024-09-21T14:59:37.937 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:37.937 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:37.937 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:37.937 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.937 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.937 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9406 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:37.937 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.166040+0000", 2024-09-21T14:59:37.937 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 58.664283683999997, 2024-09-21T14:59:37.937 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018094524000000001, 2024-09-21T14:59:37.937 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:37.937 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:37.937 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:37.937 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:37.937 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:37.937 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9406 2024-09-21T14:59:37.938 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.938 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:37.938 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.938 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:37.938 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.166040+0000", 2024-09-21T14:59:37.938 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:37.938 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.938 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.938 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:37.938 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.166040+0000", 2024-09-21T14:59:37.938 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:37.938 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.938 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.938 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:37.938 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.166038+0000", 2024-09-21T14:59:37.939 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T14:59:37.939 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.939 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.939 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:37.939 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.166051+0000", 2024-09-21T14:59:37.939 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2638e-05 2024-09-21T14:59:37.939 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.939 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.939 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:37.939 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.166070+0000", 2024-09-21T14:59:37.939 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8691000000000001e-05 2024-09-21T14:59:37.939 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.939 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.939 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:37.939 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.166089+0000", 2024-09-21T14:59:37.940 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9286000000000001e-05 2024-09-21T14:59:37.940 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.940 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.940 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:37.940 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.180194+0000", 2024-09-21T14:59:37.940 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014105039999999999 2024-09-21T14:59:37.940 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.940 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.940 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:37.940 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.180340+0000", 2024-09-21T14:59:37.940 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014623199999999999 2024-09-21T14:59:37.940 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.940 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.940 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:59:37.940 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.180818+0000", 2024-09-21T14:59:37.941 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000477489 2024-09-21T14:59:37.941 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.941 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.941 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:37.941 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.183808+0000", 2024-09-21T14:59:37.941 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0029897610000000001 2024-09-21T14:59:37.941 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.941 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.941 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:37.941 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.183821+0000", 2024-09-21T14:59:37.941 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3251999999999999e-05 2024-09-21T14:59:37.941 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.941 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.941 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:37.942 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.183872+0000", 2024-09-21T14:59:37.942 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0900999999999999e-05 2024-09-21T14:59:37.942 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.942 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.942 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:37.942 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.184134+0000", 2024-09-21T14:59:37.942 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000262484 2024-09-21T14:59:37.942 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:37.942 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:37.942 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:37.942 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.942 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.942 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9407 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:37.942 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.168094+0000", 2024-09-21T14:59:37.942 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 58.662229064000002, 2024-09-21T14:59:37.943 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019444860000000001, 2024-09-21T14:59:37.943 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:37.943 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:37.943 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:37.943 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:37.943 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:37.943 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9407 2024-09-21T14:59:37.943 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.943 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:37.943 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.943 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:37.943 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.168094+0000", 2024-09-21T14:59:37.943 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:37.943 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.943 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.944 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:37.944 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.168094+0000", 2024-09-21T14:59:37.944 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:37.944 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.944 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.944 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:37.944 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.168093+0000", 2024-09-21T14:59:37.944 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:59:37.944 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.944 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.944 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:37.944 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.168119+0000", 2024-09-21T14:59:37.944 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5803999999999999e-05 2024-09-21T14:59:37.944 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.944 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.944 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:37.945 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.168135+0000", 2024-09-21T14:59:37.945 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6359999999999999e-05 2024-09-21T14:59:37.945 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.945 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.945 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:37.945 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.168144+0000", 2024-09-21T14:59:37.945 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4270000000000008e-06 2024-09-21T14:59:37.945 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.945 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.945 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:37.945 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181943+0000", 2024-09-21T14:59:37.945 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013799617 2024-09-21T14:59:37.945 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.945 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.945 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:37.945 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.182052+0000", 2024-09-21T14:59:37.946 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000108924 2024-09-21T14:59:37.946 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.946 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.946 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:59:37.946 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.183431+0000", 2024-09-21T14:59:37.946 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0013783619999999999 2024-09-21T14:59:37.946 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.946 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.946 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:37.946 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.187276+0000", 2024-09-21T14:59:37.946 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0038450469999999999 2024-09-21T14:59:37.946 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.946 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.947 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:37.947 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.187288+0000", 2024-09-21T14:59:37.947 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2414e-05 2024-09-21T14:59:37.947 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.947 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.947 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:37.947 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.187414+0000", 2024-09-21T14:59:37.947 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012614399999999999 2024-09-21T14:59:37.947 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.947 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.947 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:37.947 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.187539+0000", 2024-09-21T14:59:37.947 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012481800000000001 2024-09-21T14:59:37.947 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:37.947 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:37.947 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:37.948 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.948 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.948 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9410 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:37.948 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.174103+0000", 2024-09-21T14:59:37.948 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 58.656220155, 2024-09-21T14:59:37.948 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025493703, 2024-09-21T14:59:37.948 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:37.948 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:37.948 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:37.948 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:37.948 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:37.948 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9410 2024-09-21T14:59:37.948 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.948 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:37.948 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.949 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:37.949 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.174103+0000", 2024-09-21T14:59:37.949 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:37.949 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.949 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.949 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:37.949 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.174103+0000", 2024-09-21T14:59:37.949 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:37.949 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.949 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.949 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:37.949 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.174102+0000", 2024-09-21T14:59:37.949 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:59:37.949 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.949 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.950 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:37.950 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.174108+0000", 2024-09-21T14:59:37.950 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1149999999999999e-06 2024-09-21T14:59:37.950 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.950 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.950 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:37.950 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.174117+0000", 2024-09-21T14:59:37.950 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4870000000000002e-06 2024-09-21T14:59:37.950 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.950 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.950 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:37.950 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.174122+0000", 2024-09-21T14:59:37.950 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9790000000000001e-06 2024-09-21T14:59:37.951 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.951 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.951 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:37.951 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.187830+0000", 2024-09-21T14:59:37.951 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013708048 2024-09-21T14:59:37.951 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.951 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.951 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:37.951 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.187913+0000", 2024-09-21T14:59:37.951 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2763999999999996e-05 2024-09-21T14:59:37.951 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.951 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.951 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:59:37.951 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.188230+0000", 2024-09-21T14:59:37.951 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00031745100000000001 2024-09-21T14:59:37.952 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.952 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.952 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:37.952 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.192708+0000", 2024-09-21T14:59:37.952 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0044783890000000002 2024-09-21T14:59:37.952 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.952 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.952 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:37.952 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199487+0000", 2024-09-21T14:59:37.952 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0067782149999999998 2024-09-21T14:59:37.952 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.952 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.952 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:37.953 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199560+0000", 2024-09-21T14:59:37.953 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3752999999999997e-05 2024-09-21T14:59:37.953 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.953 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.953 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:37.953 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199597+0000", 2024-09-21T14:59:37.953 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6526999999999998e-05 2024-09-21T14:59:37.953 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:37.953 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:37.953 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:37.953 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.953 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.953 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9411 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:37.954 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.175943+0000", 2024-09-21T14:59:37.954 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 58.654380136, 2024-09-21T14:59:37.954 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023773181000000001, 2024-09-21T14:59:37.954 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:37.954 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:37.954 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:37.954 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:37.954 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:37.954 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9411 2024-09-21T14:59:37.954 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.954 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:37.954 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.954 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:37.955 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175943+0000", 2024-09-21T14:59:37.955 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:37.955 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.955 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.955 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:37.955 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175943+0000", 2024-09-21T14:59:37.955 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:37.955 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.955 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.955 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:37.955 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175942+0000", 2024-09-21T14:59:37.955 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:59:37.955 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.956 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.957 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:37.957 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175948+0000", 2024-09-21T14:59:37.958 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9689999999999999e-06 2024-09-21T14:59:37.958 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.958 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.958 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:37.958 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175960+0000", 2024-09-21T14:59:37.958 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1745e-05 2024-09-21T14:59:37.958 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.958 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.958 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:37.958 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175965+0000", 2024-09-21T14:59:37.958 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.203e-06 2024-09-21T14:59:37.958 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.959 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.959 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:37.959 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.189900+0000", 2024-09-21T14:59:37.959 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013934940999999999 2024-09-21T14:59:37.959 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.959 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.959 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:37.959 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.189988+0000", 2024-09-21T14:59:37.959 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8419999999999997e-05 2024-09-21T14:59:37.959 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.959 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.959 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:59:37.959 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.190321+0000", 2024-09-21T14:59:37.959 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00033231800000000001 2024-09-21T14:59:37.959 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.960 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.960 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:37.960 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.192746+0000", 2024-09-21T14:59:37.960 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0024254400000000001 2024-09-21T14:59:37.960 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.960 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.960 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:37.960 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199605+0000", 2024-09-21T14:59:37.960 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0068582809999999999 2024-09-21T14:59:37.960 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.960 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.960 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:37.960 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199680+0000", 2024-09-21T14:59:37.960 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5630000000000006e-05 2024-09-21T14:59:37.960 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.960 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.961 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:37.961 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199716+0000", 2024-09-21T14:59:37.961 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6248999999999998e-05 2024-09-21T14:59:37.961 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:37.961 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:37.961 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:37.961 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.961 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.961 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9412 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:37.961 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.177402+0000", 2024-09-21T14:59:37.962 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 58.652921038999999, 2024-09-21T14:59:37.962 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022445590000000001, 2024-09-21T14:59:37.962 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:37.962 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:37.962 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:37.962 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:37.962 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:37.962 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9412 2024-09-21T14:59:37.962 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.962 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:37.962 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.963 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:37.963 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177402+0000", 2024-09-21T14:59:37.963 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:37.963 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.963 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.963 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:37.963 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177402+0000", 2024-09-21T14:59:37.963 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:37.963 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.963 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.963 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:37.963 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177401+0000", 2024-09-21T14:59:37.963 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:59:37.963 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.963 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.964 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:37.964 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177409+0000", 2024-09-21T14:59:37.964 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4710000000000001e-06 2024-09-21T14:59:37.964 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.964 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.964 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:37.964 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177419+0000", 2024-09-21T14:59:37.964 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0351e-05 2024-09-21T14:59:37.964 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.964 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.964 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:37.964 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177424+0000", 2024-09-21T14:59:37.964 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1919999999999996e-06 2024-09-21T14:59:37.964 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.964 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.965 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:37.965 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.191343+0000", 2024-09-21T14:59:37.965 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.01391878 2024-09-21T14:59:37.965 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.965 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.965 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:37.965 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.191446+0000", 2024-09-21T14:59:37.965 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000102453 2024-09-21T14:59:37.965 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.965 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.965 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:59:37.965 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.192200+0000", 2024-09-21T14:59:37.965 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00075476899999999997 2024-09-21T14:59:37.965 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.966 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.966 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:37.966 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.193866+0000", 2024-09-21T14:59:37.966 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0016657519999999999 2024-09-21T14:59:37.966 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.966 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.966 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:37.966 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199724+0000", 2024-09-21T14:59:37.966 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0058579529999999999 2024-09-21T14:59:37.966 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.966 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.966 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:37.966 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199803+0000", 2024-09-21T14:59:37.967 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8662e-05 2024-09-21T14:59:37.967 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.967 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.967 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:37.967 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199848+0000", 2024-09-21T14:59:37.967 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5216000000000002e-05 2024-09-21T14:59:37.967 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:37.967 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:37.967 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:37.967 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.967 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.967 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:10445 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:37.967 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.271980+0000", 2024-09-21T14:59:37.967 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 58.558343674, 2024-09-21T14:59:37.967 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018664256000000001, 2024-09-21T14:59:37.968 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:37.968 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:37.968 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:37.968 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:37.968 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:37.968 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 10445 2024-09-21T14:59:37.968 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.968 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:37.968 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.968 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:37.968 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.271980+0000", 2024-09-21T14:59:37.968 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:37.968 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.968 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.968 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:37.968 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.271980+0000", 2024-09-21T14:59:37.969 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:37.969 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.969 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.969 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:37.969 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.271979+0000", 2024-09-21T14:59:37.969 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:37.969 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.969 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.969 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:37.969 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.271982+0000", 2024-09-21T14:59:37.969 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7240000000000001e-06 2024-09-21T14:59:37.969 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.969 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.969 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:37.969 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.271987+0000", 2024-09-21T14:59:37.970 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3639999999999998e-06 2024-09-21T14:59:37.970 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.970 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.970 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:37.970 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.271992+0000", 2024-09-21T14:59:37.970 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7110000000000003e-06 2024-09-21T14:59:37.970 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.970 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.970 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:37.970 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.277567+0000", 2024-09-21T14:59:37.970 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0055745009999999999 2024-09-21T14:59:37.970 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.970 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.970 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:37.970 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.277655+0000", 2024-09-21T14:59:37.971 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8465000000000001e-05 2024-09-21T14:59:37.971 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.971 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.971 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:59:37.971 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.278720+0000", 2024-09-21T14:59:37.971 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0010650569999999999 2024-09-21T14:59:37.971 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.971 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.971 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:37.971 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.281153+0000", 2024-09-21T14:59:37.971 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0024326389999999999 2024-09-21T14:59:37.971 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.971 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.971 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:37.971 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.290574+0000", 2024-09-21T14:59:37.972 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0094209489999999996 2024-09-21T14:59:37.972 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.972 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.972 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:37.972 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.290613+0000", 2024-09-21T14:59:37.972 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.8803000000000001e-05 2024-09-21T14:59:37.972 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.972 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.972 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:37.972 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.290644+0000", 2024-09-21T14:59:37.972 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1404e-05 2024-09-21T14:59:37.972 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:37.972 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:37.972 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:37.972 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.973 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.973 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22769 3.f 3:f39bd56f:::rbd_data.11366acc894.0000000000000004:19 [read 1032192~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:37.973 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.140207+0000", 2024-09-21T14:59:37.973 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 47.690115933999998, 2024-09-21T14:59:37.973 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018031170999999999, 2024-09-21T14:59:37.973 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:37.973 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:37.973 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:37.973 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:37.973 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:37.973 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22769 2024-09-21T14:59:37.973 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.973 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:37.973 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.973 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:37.973 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.140207+0000", 2024-09-21T14:59:37.974 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:37.974 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.974 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.974 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:37.974 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.140207+0000", 2024-09-21T14:59:37.974 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:37.974 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.974 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.974 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:37.974 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.140207+0000", 2024-09-21T14:59:37.974 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:37.974 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.974 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.974 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:37.975 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.140209+0000", 2024-09-21T14:59:37.975 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.305e-06 2024-09-21T14:59:37.975 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.975 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.975 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:37.975 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.140214+0000", 2024-09-21T14:59:37.975 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9570000000000001e-06 2024-09-21T14:59:37.975 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.975 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.975 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:37.975 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.140219+0000", 2024-09-21T14:59:37.975 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2250000000000002e-06 2024-09-21T14:59:37.975 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.975 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.975 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:37.976 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.157990+0000", 2024-09-21T14:59:37.976 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.017770939999999999 2024-09-21T14:59:37.976 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.976 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.976 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:37.976 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.158062+0000", 2024-09-21T14:59:37.976 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2652999999999997e-05 2024-09-21T14:59:37.976 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.976 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.976 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:37.976 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.158239+0000", 2024-09-21T14:59:37.976 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017640599999999999 2024-09-21T14:59:37.976 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:37.976 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:37.977 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:37.977 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.977 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.977 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:23084 3.3 3:c55e343c:::rbd_data.11366acc894.0000000000000006:19 [read 1032192~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:37.977 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.150724+0000", 2024-09-21T14:59:37.977 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 47.679599099000001, 2024-09-21T14:59:37.977 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018916460999999999, 2024-09-21T14:59:37.977 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:37.977 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:37.977 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:37.977 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:37.977 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:37.977 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 23084 2024-09-21T14:59:37.977 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.977 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:37.978 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.978 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:37.978 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.150724+0000", 2024-09-21T14:59:37.978 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:37.978 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.978 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.978 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:37.978 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.150724+0000", 2024-09-21T14:59:37.978 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:37.978 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.978 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.978 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:37.978 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.150724+0000", 2024-09-21T14:59:37.978 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:37.978 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.979 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.979 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:37.979 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.150727+0000", 2024-09-21T14:59:37.979 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7250000000000002e-06 2024-09-21T14:59:37.979 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.979 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.979 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:37.979 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.150732+0000", 2024-09-21T14:59:37.979 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5809999999999996e-06 2024-09-21T14:59:37.979 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.979 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.979 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:37.979 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.150737+0000", 2024-09-21T14:59:37.979 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3590000000000001e-06 2024-09-21T14:59:37.980 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.980 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.980 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:37.980 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.166694+0000", 2024-09-21T14:59:37.980 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.015957720000000002 2024-09-21T14:59:37.980 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.980 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.980 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:37.980 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.166769+0000", 2024-09-21T14:59:37.980 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5143999999999997e-05 2024-09-21T14:59:37.980 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:37.980 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:37.980 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:37.980 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.169641+0000", 2024-09-21T14:59:37.980 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0028713089999999998 2024-09-21T14:59:37.981 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:37.981 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:37.981 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:37.981 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:37.981 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:37.981 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:37.981 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T14:59:37.982 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:59:37.877+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:59:37.984 INFO:teuthology.orchestra.run.smithi032.stdout:1849 read 0x6e6031f thru 0x6e6f823 (0xf505 bytes) 2024-09-21T14:59:37.984 INFO:teuthology.orchestra.run.smithi032.stdout:1850 read 0xd1bde0b thru 0xd1c6766 (0x895c bytes) 2024-09-21T14:59:37.984 INFO:teuthology.orchestra.run.smithi032.stdout:1852 punch from 0xc710ddd to 0xc710ebd, (0xe0 bytes) 2024-09-21T14:59:37.984 INFO:teuthology.orchestra.run.smithi032.stdout:1853 writesame 0x70a4f80 thru 0x70a95af data_size 0x8(0x4630 bytes) 2024-09-21T14:59:37.984 INFO:teuthology.orchestra.run.smithi032.stdout:1855 write 0x36c99e6 thru 0x36cccd1 (0x32ec bytes) 2024-09-21T14:59:37.984 INFO:teuthology.orchestra.run.smithi032.stdout:1856 read 0xc15520a thru 0xc15ca71 (0x7868 bytes) 2024-09-21T14:59:37.984 INFO:teuthology.orchestra.run.smithi032.stdout:1857 write 0x931f195 thru 0x9329386 (0xa1f2 bytes) 2024-09-21T14:59:37.984 INFO:teuthology.orchestra.run.smithi032.stdout:1858 write 0x137d2d2 thru 0x137dbe5 (0x914 bytes) 2024-09-21T14:59:37.985 INFO:teuthology.orchestra.run.smithi032.stdout:1859 read 0x920091 thru 0x923486 (0x33f6 bytes) 2024-09-21T14:59:37.985 INFO:teuthology.orchestra.run.smithi032.stdout:1861 write 0x448fcd6 thru 0x4494ef1 (0x521c bytes) 2024-09-21T14:59:37.985 INFO:teuthology.orchestra.run.smithi032.stdout:1862 writesame 0x131116 thru 0x13b42b data_size 0xa(0xa316 bytes) 2024-09-21T14:59:37.985 INFO:teuthology.orchestra.run.smithi032.stdout:1864 writesame 0x10d28e4 thru 0x10d38a7 data_size 0x2(0xfc4 bytes) 2024-09-21T14:59:37.985 INFO:teuthology.orchestra.run.smithi032.stdout:1865 writesame 0x937fb3c thru 0x9385605 data_size 0x1(0x5aca bytes) 2024-09-21T14:59:37.985 INFO:teuthology.orchestra.run.smithi032.stdout:1866 write 0xd64cb1a thru 0xd658012 (0xb4f9 bytes) 2024-09-21T14:59:37.985 INFO:teuthology.orchestra.run.smithi032.stdout:1867 punch from 0x22c1ebf to 0x22cd2c3, (0xb404 bytes) 2024-09-21T14:59:37.985 INFO:teuthology.orchestra.run.smithi032.stdout:1868 write 0x819c5b0 thru 0x819cb41 (0x592 bytes) 2024-09-21T14:59:37.985 INFO:teuthology.orchestra.run.smithi032.stdout:1869 trunc from 0xd6c0931 to 0xe854abd 2024-09-21T14:59:37.986 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:59:37.979+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:59:37.988 INFO:teuthology.orchestra.run.smithi032.stdout:1870 write 0xbf84d11 thru 0xbf87e31 (0x3121 bytes) 2024-09-21T14:59:37.992 INFO:teuthology.orchestra.run.smithi032.stdout:1871 writesame 0xafdc471 thru 0xafe9b6a data_size 0x7(0xd6fa bytes) 2024-09-21T14:59:38.008 INFO:teuthology.orchestra.run.smithi032.stdout:1873 read 0x5b49506 thru 0x5b4b389 (0x1e84 bytes) 2024-09-21T14:59:38.018 INFO:teuthology.orchestra.run.smithi032.stdout:1874 writesame 0xedb8c80 thru 0xedc57bb data_size 0x2(0xcb3c bytes) 2024-09-21T14:59:38.037 INFO:teuthology.orchestra.run.smithi032.stdout:1875 trunc from 0xedc57bc to 0x362a68c 2024-09-21T14:59:38.084 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:59:38.079+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:59:38.099 INFO:teuthology.orchestra.run.smithi032.stdout:1876 read 0x26a6269 thru 0x26a8fea (0x2d82 bytes) 2024-09-21T14:59:38.099 INFO:teuthology.orchestra.run.smithi032.stdout:1881 punch from 0xdc28a8 to 0xdc929b, (0x69f3 bytes) 2024-09-21T14:59:38.107 INFO:teuthology.orchestra.run.smithi032.stdout:1884 write 0x48914ef thru 0x4891a88 (0x59a bytes) 2024-09-21T14:59:38.122 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:38.122 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:59:38.122 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T14:59:38.122 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:38.130 INFO:teuthology.orchestra.run.smithi032.stdout:1885 write 0x97705bd thru 0x9780391 (0xfdd5 bytes) 2024-09-21T14:59:38.135 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T14:59:38.155 INFO:teuthology.orchestra.run.smithi032.stdout:1887 trunc from 0x9780392 to 0x4b15437 2024-09-21T14:59:38.185 INFO:teuthology.orchestra.run.smithi032.stdout:1888 write 0x52a364b thru 0x52a4519 (0xecf bytes) 2024-09-21T14:59:38.185 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:59:38.179+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:59:38.213 INFO:teuthology.orchestra.run.smithi032.stdout:1889 write 0xb3cdfe7 thru 0xb3d8273 (0xa28d bytes) 2024-09-21T14:59:38.227 INFO:teuthology.orchestra.run.smithi032.stdout:1890 read 0x6680651 thru 0x6683c9b (0x364b bytes) 2024-09-21T14:59:38.227 INFO:teuthology.orchestra.run.smithi032.stdout:1891 read 0x204eafc thru 0x205c7fb (0xdd00 bytes) 2024-09-21T14:59:38.239 INFO:teuthology.orchestra.run.smithi032.stdout:1892 write 0xc50b692 thru 0xc50cd7f (0x16ee bytes) 2024-09-21T14:59:38.245 INFO:teuthology.orchestra.run.smithi032.stdout:1893 write 0x3a55f61 thru 0x3a6023e (0xa2de bytes) 2024-09-21T14:59:38.259 INFO:teuthology.orchestra.run.smithi032.stdout:1895 write 0x936f40b thru 0x9373ea0 (0x4a96 bytes) 2024-09-21T14:59:38.265 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:38.265 INFO:teuthology.orchestra.run.smithi032.stdout:1897 read 0x58f5222 thru 0x5901a61 (0xc840 bytes) 2024-09-21T14:59:38.277 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:38.277 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:59:38.277 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T14:59:38.277 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T14:59:38.277 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:38.283 INFO:teuthology.orchestra.run.smithi032.stdout:1898 trunc from 0xc50cd80 to 0xe1cb69 2024-09-21T14:59:38.286 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:59:38.283+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:59:38.292 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T14:59:38.294 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T14:59:38.317 INFO:teuthology.orchestra.run.smithi032.stderr:nodeep-scrub is unset 2024-09-21T14:59:38.349 INFO:teuthology.orchestra.run.smithi032.stdout:1899 writesame 0x827b6ed thru 0x8281f91 data_size 0x2b(0x68a5 bytes) 2024-09-21T14:59:38.356 INFO:teuthology.orchestra.run.smithi032.stdout:1900 trunc from 0x8281f92 to 0xcca7bfd 2024-09-21T14:59:38.365 INFO:teuthology.orchestra.run.smithi032.stdout:1901 writesame 0x1e80a2a thru 0x1e80c20 data_size 0x1(0x1f7 bytes) 2024-09-21T14:59:38.373 INFO:teuthology.orchestra.run.smithi032.stdout:1902 read 0x3b559e4 thru 0x3b5c2b7 (0x68d4 bytes) 2024-09-21T14:59:38.373 INFO:teuthology.orchestra.run.smithi032.stdout:1903 writesame 0xc3d0ddd thru 0xc3dfe2d data_size 0x1(0xf051 bytes) 2024-09-21T14:59:38.387 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:59:38.383+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:59:38.390 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-21T14:59:38.394 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-09-21T14:59:38.406 INFO:teuthology.orchestra.run.smithi032.stdout:1904 writesame 0xc2bf35f thru 0xc2cb5f9 data_size 0xb(0xc29b bytes) 2024-09-21T14:59:38.408 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":142,"stamp":"2024-09-21T14:59:38.299664+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":406234620,"num_objects":862,"num_object_clones":0,"num_object_copies":1724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":862,"num_whiteouts":0,"num_read":70152,"num_read_kb":425308,"num_write":10469,"num_write_kb":300902,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1206,"num_bytes_recovered":80408212,"num_keys_recovered":119,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7234,"ondisk_log_size":7234,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":353456,"kb_used_data":43048,"kb_used_omap":12,"kb_used_meta":310259,"kb_avail":754621264,"statfs":{"total":773094113280,"available":772732174336,"internally_reserved":0,"allocated":44081152,"data_stored":313156090,"data_compressed":5642743,"data_compressed_allocated":27013120,"data_compressed_original":304416213,"omap_allocated":12744,"internal_metadata":317705784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":66395675,"num_objects":107,"num_object_clones":0,"num_object_copies":214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":107,"num_whiteouts":0,"num_read":4212,"num_read_kb":39520,"num_write":375,"num_write_kb":27650,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.500150"},"pg_stats":[{"pgid":"3.1f","version":"98'219","reported_seq":1950,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T14:59:35.273021+0000","last_change":"2024-09-21T14:59:33.129559+0000","last_active":"2024-09-21T14:59:35.273021+0000","last_peered":"2024-09-21T14:59:35.273021+0000","last_clean":"2024-09-21T14:59:35.273021+0000","last_became_active":"2024-09-21T14:58:19.410055+0000","last_became_peered":"2024-09-21T14:58:19.410055+0000","last_unstale":"2024-09-21T14:59:35.273021+0000","last_undegraded":"2024-09-21T14:59:35.273021+0000","last_fullsized":"2024-09-21T14:59:35.273021+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":219,"log_dups_size":0,"ondisk_log_size":219,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:49:13.944219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001260768,"stat_sum":{"num_bytes":6895335,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":2101,"num_read_kb":8918,"num_write":319,"num_write_kb":9198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"}]},{"pgid":"3.1e","version":"98'150","reported_seq":1595,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T14:59:35.273020+0000","last_change":"2024-09-21T14:59:33.129453+0000","last_active":"2024-09-21T14:59:35.273020+0000","last_peered":"2024-09-21T14:59:35.273020+0000","last_clean":"2024-09-21T14:59:35.273020+0000","last_became_active":"2024-09-21T14:59:13.130091+0000","last_became_peered":"2024-09-21T14:59:13.130091+0000","last_unstale":"2024-09-21T14:59:35.273020+0000","last_undegraded":"2024-09-21T14:59:35.273020+0000","last_fullsized":"2024-09-21T14:59:35.273020+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":150,"log_dups_size":0,"ondisk_log_size":150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:46:00.199136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001044987,"stat_sum":{"num_bytes":11322194,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":1419,"num_read_kb":11439,"num_write":205,"num_write_kb":7018,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":3844512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"}]},{"pgid":"3.1d","version":"98'101","reported_seq":2829,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T14:59:35.478545+0000","last_change":"2024-09-21T14:59:33.133014+0000","last_active":"2024-09-21T14:59:35.478545+0000","last_peered":"2024-09-21T14:59:35.478545+0000","last_clean":"2024-09-21T14:59:35.478545+0000","last_became_active":"2024-09-21T14:58:19.411387+0000","last_became_peered":"2024-09-21T14:58:19.411387+0000","last_unstale":"2024-09-21T14:59:35.478545+0000","last_undegraded":"2024-09-21T14:59:35.478545+0000","last_fullsized":"2024-09-21T14:59:35.478545+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:37:57.597069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019847340000000002,"stat_sum":{"num_bytes":10257463,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":2567,"num_read_kb":12776,"num_write":135,"num_write_kb":7143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"32","length":"1"}]},{"pgid":"3.1c","version":"98'173","reported_seq":1257,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T14:59:35.280616+0000","last_change":"2024-09-21T14:59:33.144497+0000","last_active":"2024-09-21T14:59:35.280616+0000","last_peered":"2024-09-21T14:59:35.280616+0000","last_clean":"2024-09-21T14:59:35.280616+0000","last_became_active":"2024-09-21T14:58:19.411306+0000","last_became_peered":"2024-09-21T14:58:19.411306+0000","last_unstale":"2024-09-21T14:59:35.280616+0000","last_undegraded":"2024-09-21T14:59:35.280616+0000","last_fullsized":"2024-09-21T14:59:35.280616+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":173,"log_dups_size":0,"ondisk_log_size":173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:20:27.079795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012611040000000001,"stat_sum":{"num_bytes":6764293,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":1185,"num_read_kb":7442,"num_write":208,"num_write_kb":10817,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"32","length":"1"}]},{"pgid":"3.1b","version":"98'237","reported_seq":1457,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T14:59:35.504019+0000","last_change":"2024-09-21T14:59:33.286305+0000","last_active":"2024-09-21T14:59:35.504019+0000","last_peered":"2024-09-21T14:59:35.504019+0000","last_clean":"2024-09-21T14:59:35.504019+0000","last_became_active":"2024-09-21T14:59:31.131051+0000","last_became_peered":"2024-09-21T14:59:31.131051+0000","last_unstale":"2024-09-21T14:59:35.504019+0000","last_undegraded":"2024-09-21T14:59:35.504019+0000","last_fullsized":"2024-09-21T14:59:35.504019+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":237,"log_dups_size":0,"ondisk_log_size":237,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:21:37.320774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00110776,"stat_sum":{"num_bytes":5568723,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":1631,"num_read_kb":10100,"num_write":293,"num_write_kb":8886,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":117,"num_bytes_recovered":7757116,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"}]},{"pgid":"3.1a","version":"98'197","reported_seq":1648,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T14:59:35.384083+0000","last_change":"2024-09-21T14:59:33.144299+0000","last_active":"2024-09-21T14:59:35.384083+0000","last_peered":"2024-09-21T14:59:35.384083+0000","last_clean":"2024-09-21T14:59:35.384083+0000","last_became_active":"2024-09-21T14:58:19.409415+0000","last_became_peered":"2024-09-21T14:58:19.409415+0000","last_unstale":"2024-09-21T14:59:35.384083+0000","last_undegraded":"2024-09-21T14:59:35.384083+0000","last_fullsized":"2024-09-21T14:59:35.384083+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:28:46.336301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011529089999999999,"stat_sum":{"num_bytes":9697369,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":1754,"num_read_kb":10242,"num_write":256,"num_write_kb":10516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"32","length":"1"}]},{"pgid":"3.19","version":"98'209","reported_seq":2510,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T14:59:35.844810+0000","last_change":"2024-09-21T14:59:33.285938+0000","last_active":"2024-09-21T14:59:35.844810+0000","last_peered":"2024-09-21T14:59:35.844810+0000","last_clean":"2024-09-21T14:59:35.844810+0000","last_became_active":"2024-09-21T14:59:32.130624+0000","last_became_peered":"2024-09-21T14:59:32.130624+0000","last_unstale":"2024-09-21T14:59:35.844810+0000","last_undegraded":"2024-09-21T14:59:35.844810+0000","last_fullsized":"2024-09-21T14:59:35.844810+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":209,"log_dups_size":0,"ondisk_log_size":209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:37:53.312733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083872199999999997,"stat_sum":{"num_bytes":21989245,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":2688,"num_read_kb":27765,"num_write":228,"num_write_kb":9760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":128,"num_bytes_recovered":11896992,"num_keys_recovered":44,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"}]},{"pgid":"3.18","version":"98'366","reported_seq":5003,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T14:59:35.526266+0000","last_change":"2024-09-21T14:59:33.128445+0000","last_active":"2024-09-21T14:59:35.526266+0000","last_peered":"2024-09-21T14:59:35.526266+0000","last_clean":"2024-09-21T14:59:35.526266+0000","last_became_active":"2024-09-21T14:58:19.409371+0000","last_became_peered":"2024-09-21T14:58:19.409371+0000","last_unstale":"2024-09-21T14:59:35.526266+0000","last_undegraded":"2024-09-21T14:59:35.526266+0000","last_fullsized":"2024-09-21T14:59:35.526266+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":366,"log_dups_size":0,"ondisk_log_size":366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:35:27.143445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092585900000000001,"stat_sum":{"num_bytes":11516101,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":5940,"num_read_kb":15577,"num_write":525,"num_write_kb":9695,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"}]},{"pgid":"3.17","version":"98'116","reported_seq":2054,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T14:59:35.273315+0000","last_change":"2024-09-21T14:59:33.130419+0000","last_active":"2024-09-21T14:59:35.273315+0000","last_peered":"2024-09-21T14:59:35.273315+0000","last_clean":"2024-09-21T14:59:35.273315+0000","last_became_active":"2024-09-21T14:58:19.410306+0000","last_became_peered":"2024-09-21T14:58:19.410306+0000","last_unstale":"2024-09-21T14:59:35.273315+0000","last_undegraded":"2024-09-21T14:59:35.273315+0000","last_fullsized":"2024-09-21T14:59:35.273315+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":116,"log_dups_size":0,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:33:50.089165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001718804,"stat_sum":{"num_bytes":5667019,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":1888,"num_read_kb":7788,"num_write":171,"num_write_kb":8878,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"}]},{"pgid":"3.16","version":"98'405","reported_seq":2173,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T14:59:35.873142+0000","last_change":"2024-09-21T14:59:33.133526+0000","last_active":"2024-09-21T14:59:35.873142+0000","last_peered":"2024-09-21T14:59:35.873142+0000","last_clean":"2024-09-21T14:59:35.873142+0000","last_became_active":"2024-09-21T14:59:32.132898+0000","last_became_peered":"2024-09-21T14:59:32.132898+0000","last_unstale":"2024-09-21T14:59:35.873142+0000","last_undegraded":"2024-09-21T14:59:35.873142+0000","last_fullsized":"2024-09-21T14:59:35.873142+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":405,"log_dups_size":0,"ondisk_log_size":405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:09:47.566958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021390620000000002,"stat_sum":{"num_bytes":9623540,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":2902,"num_read_kb":9943,"num_write":635,"num_write_kb":8219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":153,"num_bytes_recovered":9621881,"num_keys_recovered":33,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"32","length":"1"}]},{"pgid":"3.15","version":"98'369","reported_seq":1921,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T14:59:35.851383+0000","last_change":"2024-09-21T14:59:33.435269+0000","last_active":"2024-09-21T14:59:35.851383+0000","last_peered":"2024-09-21T14:59:35.851383+0000","last_clean":"2024-09-21T14:59:35.851383+0000","last_became_active":"2024-09-21T14:58:19.410687+0000","last_became_peered":"2024-09-21T14:58:19.410687+0000","last_unstale":"2024-09-21T14:59:35.851383+0000","last_undegraded":"2024-09-21T14:59:35.851383+0000","last_fullsized":"2024-09-21T14:59:35.851383+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":369,"log_dups_size":0,"ondisk_log_size":369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:21:42.226982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00213598,"stat_sum":{"num_bytes":11424210,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":2580,"num_read_kb":16722,"num_write":573,"num_write_kb":10691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"32","length":"1"}]},{"pgid":"3.14","version":"98'211","reported_seq":1468,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T14:59:35.561948+0000","last_change":"2024-09-21T14:59:33.133530+0000","last_active":"2024-09-21T14:59:35.561948+0000","last_peered":"2024-09-21T14:59:35.561948+0000","last_clean":"2024-09-21T14:59:35.561948+0000","last_became_active":"2024-09-21T14:58:19.407310+0000","last_became_peered":"2024-09-21T14:58:19.407310+0000","last_unstale":"2024-09-21T14:59:35.561948+0000","last_undegraded":"2024-09-21T14:59:35.561948+0000","last_fullsized":"2024-09-21T14:59:35.561948+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":211,"log_dups_size":0,"ondisk_log_size":211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:47:46.079115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024542029999999999,"stat_sum":{"num_bytes":19135517,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":1547,"num_read_kb":16997,"num_write":286,"num_write_kb":5847,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"32","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":91,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T14:59:34.265275+0000","last_change":"2024-09-21T14:58:11.348177+0000","last_active":"2024-09-21T14:59:34.265275+0000","last_peered":"2024-09-21T14:59:34.265275+0000","last_clean":"2024-09-21T14:59:34.265275+0000","last_became_active":"2024-09-21T14:58:09.328417+0000","last_became_peered":"2024-09-21T14:58:09.328417+0000","last_unstale":"2024-09-21T14:59:34.265275+0000","last_undegraded":"2024-09-21T14:59:34.265275+0000","last_fullsized":"2024-09-21T14:59:34.265275+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:52:22.671185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018917490000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"98'184","reported_seq":1877,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T14:59:35.834918+0000","last_change":"2024-09-21T14:59:33.286806+0000","last_active":"2024-09-21T14:59:35.834918+0000","last_peered":"2024-09-21T14:59:35.834918+0000","last_clean":"2024-09-21T14:59:35.834918+0000","last_became_active":"2024-09-21T14:59:32.131232+0000","last_became_peered":"2024-09-21T14:59:32.131232+0000","last_unstale":"2024-09-21T14:59:35.834918+0000","last_undegraded":"2024-09-21T14:59:35.834918+0000","last_fullsized":"2024-09-21T14:59:35.834918+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":184,"log_dups_size":0,"ondisk_log_size":184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:31:04.421782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014658060000000001,"stat_sum":{"num_bytes":13194102,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":2018,"num_read_kb":8748,"num_write":237,"num_write_kb":8440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":3585713,"num_keys_recovered":9,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":17,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T14:59:34.300346+0000","last_change":"2024-09-21T14:59:28.228883+0000","last_active":"2024-09-21T14:59:34.300346+0000","last_peered":"2024-09-21T14:59:34.300346+0000","last_clean":"2024-09-21T14:59:34.300346+0000","last_became_active":"2024-09-21T14:59:28.228464+0000","last_became_peered":"2024-09-21T14:59:28.228464+0000","last_unstale":"2024-09-21T14:59:34.300346+0000","last_undegraded":"2024-09-21T14:59:34.300346+0000","last_fullsized":"2024-09-21T14:59:34.300346+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:00:26.310864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"98'230","reported_seq":1571,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T14:59:35.465179+0000","last_change":"2024-09-21T14:59:33.286901+0000","last_active":"2024-09-21T14:59:35.465179+0000","last_peered":"2024-09-21T14:59:35.465179+0000","last_clean":"2024-09-21T14:59:35.465179+0000","last_became_active":"2024-09-21T14:59:31.132581+0000","last_became_peered":"2024-09-21T14:59:31.132581+0000","last_unstale":"2024-09-21T14:59:35.465179+0000","last_undegraded":"2024-09-21T14:59:35.465179+0000","last_fullsized":"2024-09-21T14:59:35.465179+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":230,"log_dups_size":0,"ondisk_log_size":230,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:36:45.467515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014921069999999999,"stat_sum":{"num_bytes":14532111,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":1777,"num_read_kb":12032,"num_write":303,"num_write_kb":9604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":119,"num_bytes_recovered":6871924,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":107,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T14:59:34.265446+0000","last_change":"2024-09-21T14:59:28.231502+0000","last_active":"2024-09-21T14:59:34.265446+0000","last_peered":"2024-09-21T14:59:34.265446+0000","last_clean":"2024-09-21T14:59:34.265446+0000","last_became_active":"2024-09-21T14:59:28.231043+0000","last_became_peered":"2024-09-21T14:59:28.231043+0000","last_unstale":"2024-09-21T14:59:34.265446+0000","last_undegraded":"2024-09-21T14:59:34.265446+0000","last_fullsized":"2024-09-21T14:59:34.265446+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:09:46.773690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019338770000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"98'209","reported_seq":2595,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T14:59:35.272405+0000","last_change":"2024-09-21T14:59:33.128175+0000","last_active":"2024-09-21T14:59:35.272405+0000","last_peered":"2024-09-21T14:59:35.272405+0000","last_clean":"2024-09-21T14:59:35.272405+0000","last_became_active":"2024-09-21T14:58:47.169989+0000","last_became_peered":"2024-09-21T14:58:47.169989+0000","last_unstale":"2024-09-21T14:59:35.272405+0000","last_undegraded":"2024-09-21T14:59:35.272405+0000","last_fullsized":"2024-09-21T14:59:35.272405+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":209,"log_dups_size":0,"ondisk_log_size":209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:52:16.238600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053533200000000002,"stat_sum":{"num_bytes":14612821,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":2734,"num_read_kb":10759,"num_write":311,"num_write_kb":8941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":92,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T14:59:34.263912+0000","last_change":"2024-09-21T14:58:11.406176+0000","last_active":"2024-09-21T14:59:34.263912+0000","last_peered":"2024-09-21T14:59:34.263912+0000","last_clean":"2024-09-21T14:59:34.263912+0000","last_became_active":"2024-09-21T14:58:09.777465+0000","last_became_peered":"2024-09-21T14:58:09.777465+0000","last_unstale":"2024-09-21T14:59:34.263912+0000","last_undegraded":"2024-09-21T14:59:34.263912+0000","last_fullsized":"2024-09-21T14:59:34.263912+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:55:04.667816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095704000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"98'384","reported_seq":1744,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T14:59:35.552633+0000","last_change":"2024-09-21T14:59:33.306239+0000","last_active":"2024-09-21T14:59:35.552633+0000","last_peered":"2024-09-21T14:59:35.552633+0000","last_clean":"2024-09-21T14:59:35.552633+0000","last_became_active":"2024-09-21T14:58:19.408971+0000","last_became_peered":"2024-09-21T14:58:19.408971+0000","last_unstale":"2024-09-21T14:59:35.552633+0000","last_undegraded":"2024-09-21T14:59:35.552633+0000","last_fullsized":"2024-09-21T14:59:35.552633+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":384,"log_dups_size":0,"ondisk_log_size":384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:29:52.642121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053352900000000003,"stat_sum":{"num_bytes":9946275,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":2589,"num_read_kb":10510,"num_write":683,"num_write_kb":9714,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"}]},{"pgid":"3.3","version":"98'311","reported_seq":1593,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T14:59:35.838394+0000","last_change":"2024-09-21T14:59:33.437039+0000","last_active":"2024-09-21T14:59:35.838394+0000","last_peered":"2024-09-21T14:59:35.838394+0000","last_clean":"2024-09-21T14:59:35.838394+0000","last_became_active":"2024-09-21T14:58:19.417453+0000","last_became_peered":"2024-09-21T14:58:19.417453+0000","last_unstale":"2024-09-21T14:59:35.838394+0000","last_undegraded":"2024-09-21T14:59:35.838394+0000","last_fullsized":"2024-09-21T14:59:35.838394+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":311,"log_dups_size":0,"ondisk_log_size":311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:19:27.087272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0053334189999999998,"stat_sum":{"num_bytes":12625204,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":2102,"num_read_kb":16217,"num_write":576,"num_write_kb":10262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"32","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":93,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T14:59:34.265075+0000","last_change":"2024-09-21T14:58:11.347504+0000","last_active":"2024-09-21T14:59:34.265075+0000","last_peered":"2024-09-21T14:59:34.265075+0000","last_clean":"2024-09-21T14:59:34.265075+0000","last_became_active":"2024-09-21T14:58:09.775981+0000","last_became_peered":"2024-09-21T14:58:09.775981+0000","last_unstale":"2024-09-21T14:59:34.265075+0000","last_undegraded":"2024-09-21T14:59:34.265075+0000","last_fullsized":"2024-09-21T14:59:34.265075+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:47:16.286652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012568880000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"98'194","reported_seq":1053,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T14:59:35.485383+0000","last_change":"2024-09-21T14:59:33.127882+0000","last_active":"2024-09-21T14:59:35.485383+0000","last_peered":"2024-09-21T14:59:35.485383+0000","last_clean":"2024-09-21T14:59:35.485383+0000","last_became_active":"2024-09-21T14:58:19.405811+0000","last_became_peered":"2024-09-21T14:58:19.405811+0000","last_unstale":"2024-09-21T14:59:35.485383+0000","last_undegraded":"2024-09-21T14:59:35.485383+0000","last_fullsized":"2024-09-21T14:59:35.485383+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":194,"log_dups_size":0,"ondisk_log_size":194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:36:25.551607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067526100000000003,"stat_sum":{"num_bytes":8205864,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":1254,"num_read_kb":7957,"num_write":285,"num_write_kb":8361,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":92,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T14:59:34.266664+0000","last_change":"2024-09-21T14:58:11.342210+0000","last_active":"2024-09-21T14:59:34.266664+0000","last_peered":"2024-09-21T14:59:34.266664+0000","last_clean":"2024-09-21T14:59:34.266664+0000","last_became_active":"2024-09-21T14:58:09.330624+0000","last_became_peered":"2024-09-21T14:58:09.330624+0000","last_unstale":"2024-09-21T14:59:34.266664+0000","last_undegraded":"2024-09-21T14:59:34.266664+0000","last_fullsized":"2024-09-21T14:59:34.266664+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:29:43.229538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001051151,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"98'105","reported_seq":174,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T14:59:35.272123+0000","last_change":"2024-09-21T14:59:33.286673+0000","last_active":"2024-09-21T14:59:35.272123+0000","last_peered":"2024-09-21T14:59:35.272123+0000","last_clean":"2024-09-21T14:59:35.272123+0000","last_became_active":"2024-09-21T14:59:31.132388+0000","last_became_peered":"2024-09-21T14:59:31.132388+0000","last_unstale":"2024-09-21T14:59:35.272123+0000","last_undegraded":"2024-09-21T14:59:35.272123+0000","last_fullsized":"2024-09-21T14:59:35.272123+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:29:39.955121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013608750000000001,"stat_sum":{"num_bytes":14389944,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":1206,"num_read_kb":14803,"num_write":126,"num_write_kb":8352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":150,"num_bytes_recovered":8136552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":91,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T14:59:34.264698+0000","last_change":"2024-09-21T14:58:11.346766+0000","last_active":"2024-09-21T14:59:34.264698+0000","last_peered":"2024-09-21T14:59:34.264698+0000","last_clean":"2024-09-21T14:59:34.264698+0000","last_became_active":"2024-09-21T14:58:09.329748+0000","last_became_peered":"2024-09-21T14:58:09.329748+0000","last_unstale":"2024-09-21T14:59:34.264698+0000","last_undegraded":"2024-09-21T14:59:34.264698+0000","last_fullsized":"2024-09-21T14:59:34.264698+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:31:20.267675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052698000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"98'98","reported_seq":2915,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T14:59:35.853546+0000","last_change":"2024-09-21T14:59:33.306829+0000","last_active":"2024-09-21T14:59:35.853546+0000","last_peered":"2024-09-21T14:59:35.853546+0000","last_clean":"2024-09-21T14:59:35.853546+0000","last_became_active":"2024-09-21T14:58:41.401003+0000","last_became_peered":"2024-09-21T14:58:41.401003+0000","last_unstale":"2024-09-21T14:59:35.853546+0000","last_undegraded":"2024-09-21T14:59:35.853546+0000","last_fullsized":"2024-09-21T14:59:35.853546+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:05:37.644835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001080115,"stat_sum":{"num_bytes":20978843,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":2592,"num_read_kb":13653,"num_write":107,"num_write_kb":6530,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1403348,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":92,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T14:59:34.265202+0000","last_change":"2024-09-21T14:58:11.347700+0000","last_active":"2024-09-21T14:59:34.265202+0000","last_peered":"2024-09-21T14:59:34.265202+0000","last_clean":"2024-09-21T14:59:34.265202+0000","last_became_active":"2024-09-21T14:58:09.438842+0000","last_became_peered":"2024-09-21T14:58:09.438842+0000","last_unstale":"2024-09-21T14:59:34.265202+0000","last_undegraded":"2024-09-21T14:59:34.265202+0000","last_fullsized":"2024-09-21T14:59:34.265202+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:41:34.626249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001426194,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":240,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T14:59:34.273721+0000","last_change":"2024-09-21T14:58:03.616055+0000","last_active":"2024-09-21T14:59:34.273721+0000","last_peered":"2024-09-21T14:59:34.273721+0000","last_clean":"2024-09-21T14:59:34.273721+0000","last_became_active":"2024-09-21T14:58:03.613214+0000","last_became_peered":"2024-09-21T14:58:03.613214+0000","last_unstale":"2024-09-21T14:59:34.273721+0000","last_undegraded":"2024-09-21T14:59:34.273721+0000","last_fullsized":"2024-09-21T14:59:34.273721+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:02.180162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:02.180162+0000","last_clean_scrub_stamp":"2024-09-21T14:58:02.180162+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:29:05.860361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"98'128","reported_seq":2018,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T14:59:35.858193+0000","last_change":"2024-09-21T14:59:33.285927+0000","last_active":"2024-09-21T14:59:35.858193+0000","last_peered":"2024-09-21T14:59:35.858193+0000","last_clean":"2024-09-21T14:59:35.858193+0000","last_became_active":"2024-09-21T14:59:31.131460+0000","last_became_peered":"2024-09-21T14:59:31.131460+0000","last_unstale":"2024-09-21T14:59:35.858193+0000","last_undegraded":"2024-09-21T14:59:35.858193+0000","last_fullsized":"2024-09-21T14:59:35.858193+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:01:38.353476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080642500000000005,"stat_sum":{"num_bytes":17228412,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":1600,"num_read_kb":16220,"num_write":140,"num_write_kb":12540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":119,"num_bytes_recovered":6097618,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"}]},{"pgid":"3.9","version":"98'87","reported_seq":1101,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T14:59:35.824962+0000","last_change":"2024-09-21T14:59:33.131842+0000","last_active":"2024-09-21T14:59:35.824962+0000","last_peered":"2024-09-21T14:59:35.824962+0000","last_clean":"2024-09-21T14:59:35.824962+0000","last_became_active":"2024-09-21T14:59:28.233956+0000","last_became_peered":"2024-09-21T14:59:28.233956+0000","last_unstale":"2024-09-21T14:59:35.824962+0000","last_undegraded":"2024-09-21T14:59:35.824962+0000","last_fullsized":"2024-09-21T14:59:35.824962+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:29:09.682792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001538034,"stat_sum":{"num_bytes":7894122,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":651,"num_read_kb":7371,"num_write":100,"num_write_kb":7481,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":91,"num_bytes_recovered":6925016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"32","length":"1"}]},{"pgid":"3.a","version":"98'417","reported_seq":2104,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T14:59:35.278011+0000","last_change":"2024-09-21T14:59:33.132860+0000","last_active":"2024-09-21T14:59:35.278011+0000","last_peered":"2024-09-21T14:59:35.278011+0000","last_clean":"2024-09-21T14:59:35.278011+0000","last_became_active":"2024-09-21T14:58:19.407847+0000","last_became_peered":"2024-09-21T14:58:19.407847+0000","last_unstale":"2024-09-21T14:59:35.278011+0000","last_undegraded":"2024-09-21T14:59:35.278011+0000","last_fullsized":"2024-09-21T14:59:35.278011+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":417,"log_dups_size":0,"ondisk_log_size":417,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:45:44.979236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020610229999999999,"stat_sum":{"num_bytes":14954814,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":3158,"num_read_kb":20370,"num_write":586,"num_write_kb":13147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"32","length":"1"}]},{"pgid":"3.b","version":"98'247","reported_seq":1615,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T14:59:36.011059+0000","last_change":"2024-09-21T14:59:33.136074+0000","last_active":"2024-09-21T14:59:36.011059+0000","last_peered":"2024-09-21T14:59:36.011059+0000","last_clean":"2024-09-21T14:59:36.011059+0000","last_became_active":"2024-09-21T14:58:19.410679+0000","last_became_peered":"2024-09-21T14:58:19.410679+0000","last_unstale":"2024-09-21T14:59:36.011059+0000","last_undegraded":"2024-09-21T14:59:36.011059+0000","last_fullsized":"2024-09-21T14:59:36.011059+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":247,"log_dups_size":0,"ondisk_log_size":247,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:07:40.758932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015194169999999999,"stat_sum":{"num_bytes":13450744,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":1875,"num_read_kb":13693,"num_write":359,"num_write_kb":12521,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"}]},{"pgid":"3.c","version":"98'221","reported_seq":3042,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T14:59:35.844873+0000","last_change":"2024-09-21T14:59:33.433669+0000","last_active":"2024-09-21T14:59:35.844873+0000","last_peered":"2024-09-21T14:59:35.844873+0000","last_clean":"2024-09-21T14:59:35.844873+0000","last_became_active":"2024-09-21T14:59:25.130057+0000","last_became_peered":"2024-09-21T14:59:25.130057+0000","last_unstale":"2024-09-21T14:59:35.844873+0000","last_undegraded":"2024-09-21T14:59:35.844873+0000","last_fullsized":"2024-09-21T14:59:35.844873+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":221,"log_dups_size":0,"ondisk_log_size":221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:53:33.752550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019260480000000001,"stat_sum":{"num_bytes":14874742,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":3007,"num_read_kb":15433,"num_write":349,"num_write_kb":11076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":9059600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"32","length":"1"}]},{"pgid":"3.d","version":"98'162","reported_seq":2281,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T14:59:35.277962+0000","last_change":"2024-09-21T14:59:33.133928+0000","last_active":"2024-09-21T14:59:35.277962+0000","last_peered":"2024-09-21T14:59:35.277962+0000","last_clean":"2024-09-21T14:59:35.277962+0000","last_became_active":"2024-09-21T14:58:19.407884+0000","last_became_peered":"2024-09-21T14:58:19.407884+0000","last_unstale":"2024-09-21T14:59:35.277962+0000","last_undegraded":"2024-09-21T14:59:35.277962+0000","last_fullsized":"2024-09-21T14:59:35.277962+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:28:57.111615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002157703,"stat_sum":{"num_bytes":24293494,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":2105,"num_read_kb":25083,"num_write":221,"num_write_kb":8576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"32","length":"1"}]},{"pgid":"3.e","version":"98'362","reported_seq":1850,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T14:59:35.484815+0000","last_change":"2024-09-21T14:59:33.435583+0000","last_active":"2024-09-21T14:59:35.484815+0000","last_peered":"2024-09-21T14:59:35.484815+0000","last_clean":"2024-09-21T14:59:35.484815+0000","last_became_active":"2024-09-21T14:59:13.131825+0000","last_became_peered":"2024-09-21T14:59:13.131825+0000","last_unstale":"2024-09-21T14:59:35.484815+0000","last_undegraded":"2024-09-21T14:59:35.484815+0000","last_fullsized":"2024-09-21T14:59:35.484815+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":362,"log_dups_size":0,"ondisk_log_size":362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:20:52.475349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024523320000000002,"stat_sum":{"num_bytes":9119565,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":2245,"num_read_kb":9566,"num_write":647,"num_write_kb":7420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":2113894,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"32","length":"1"}]},{"pgid":"3.f","version":"96'252","reported_seq":2296,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T14:59:35.861719+0000","last_change":"2024-09-21T14:59:33.436017+0000","last_active":"2024-09-21T14:59:35.861719+0000","last_peered":"2024-09-21T14:59:35.861719+0000","last_clean":"2024-09-21T14:59:35.861719+0000","last_became_active":"2024-09-21T14:58:19.410406+0000","last_became_peered":"2024-09-21T14:58:19.410406+0000","last_unstale":"2024-09-21T14:59:35.861719+0000","last_undegraded":"2024-09-21T14:59:35.861719+0000","last_fullsized":"2024-09-21T14:59:35.861719+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":252,"log_dups_size":0,"ondisk_log_size":252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:59:16.026939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024498279999999998,"stat_sum":{"num_bytes":18194313,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":2538,"num_read_kb":18015,"num_write":399,"num_write_kb":11502,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"32","length":"1"}]},{"pgid":"3.10","version":"98'224","reported_seq":2199,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T14:59:35.863536+0000","last_change":"2024-09-21T14:59:33.143378+0000","last_active":"2024-09-21T14:59:35.863536+0000","last_peered":"2024-09-21T14:59:35.863536+0000","last_clean":"2024-09-21T14:59:35.863536+0000","last_became_active":"2024-09-21T14:58:19.409935+0000","last_became_peered":"2024-09-21T14:58:19.409935+0000","last_unstale":"2024-09-21T14:59:35.863536+0000","last_undegraded":"2024-09-21T14:59:35.863536+0000","last_fullsized":"2024-09-21T14:59:35.863536+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":224,"log_dups_size":0,"ondisk_log_size":224,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:10:36.252629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037195000000000001,"stat_sum":{"num_bytes":22393122,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":2194,"num_read_kb":24025,"num_write":336,"num_write_kb":13321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"32","length":"1"}]},{"pgid":"3.11","version":"98'111","reported_seq":1523,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T14:59:35.533779+0000","last_change":"2024-09-21T14:59:33.131263+0000","last_active":"2024-09-21T14:59:35.533779+0000","last_peered":"2024-09-21T14:59:35.533779+0000","last_clean":"2024-09-21T14:59:35.533779+0000","last_became_active":"2024-09-21T14:59:10.128001+0000","last_became_peered":"2024-09-21T14:59:10.128001+0000","last_unstale":"2024-09-21T14:59:35.533779+0000","last_undegraded":"2024-09-21T14:59:35.533779+0000","last_fullsized":"2024-09-21T14:59:35.533779+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:44:22.770961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001020408,"stat_sum":{"num_bytes":7087982,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":1167,"num_read_kb":9525,"num_write":138,"num_write_kb":9077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":3094046,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"32","length":"1"}]},{"pgid":"3.12","version":"98'169","reported_seq":2867,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T14:59:35.427365+0000","last_change":"2024-09-21T14:59:33.132244+0000","last_active":"2024-09-21T14:59:35.427365+0000","last_peered":"2024-09-21T14:59:35.427365+0000","last_clean":"2024-09-21T14:59:35.427365+0000","last_became_active":"2024-09-21T14:58:47.175860+0000","last_became_peered":"2024-09-21T14:58:47.175860+0000","last_unstale":"2024-09-21T14:59:35.427365+0000","last_undegraded":"2024-09-21T14:59:35.427365+0000","last_fullsized":"2024-09-21T14:59:35.427365+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:04:46.485874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001750887,"stat_sum":{"num_bytes":8718921,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":2988,"num_read_kb":6570,"num_write":238,"num_write_kb":7363,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"32","length":"1"}]},{"pgid":"3.13","version":"98'266","reported_seq":1390,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-21T14:59:35.387741+0000","last_change":"2024-09-21T14:59:33.133146+0000","last_active":"2024-09-21T14:59:35.387741+0000","last_peered":"2024-09-21T14:59:35.387741+0000","last_clean":"2024-09-21T14:59:35.387741+0000","last_became_active":"2024-09-21T14:58:19.412030+0000","last_became_peered":"2024-09-21T14:58:19.412030+0000","last_unstale":"2024-09-21T14:59:35.387741+0000","last_undegraded":"2024-09-21T14:59:35.387741+0000","last_fullsized":"2024-09-21T14:59:35.387741+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":266,"log_dups_size":0,"ondisk_log_size":266,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:44:29.282391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021046760000000002,"stat_sum":{"num_bytes":9087829,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":2034,"num_read_kb":8949,"num_write":333,"num_write_kb":7588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"32","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":405644233,"num_objects":858,"num_object_clones":0,"num_object_copies":1716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":858,"num_whiteouts":0,"num_read":70036,"num_read_kb":425208,"num_write":10308,"num_write_kb":298484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1206,"num_bytes_recovered":80408212,"num_keys_recovered":119,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":36556800,"data_stored":333420610,"data_compressed":5889036,"data_compressed_allocated":28758016,"data_compressed_original":329049557,"omap_allocated":0,"internal_metadata":0},"log_size":7114,"ondisk_log_size":7114,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":10404,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509463,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43844,"kb_used_data":4532,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94327996,"statfs":{"total":96636764160,"available":96591867904,"internally_reserved":0,"allocated":4640768,"data_stored":33743582,"data_compressed":525681,"data_compressed_allocated":2777088,"data_compressed_original":32819934,"omap_allocated":1589,"internal_metadata":40237515},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:59:02 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.917,"5min":1.917,"15min":1.917},"last":0.68999999999999995}]},{"osd":1,"last update":"Sat Sep 21 14:59:02 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":2.2149999999999999,"5min":2.2149999999999999,"15min":2.2149999999999999},"last":0.64500000000000002}]},{"osd":2,"last update":"Sat Sep 21 14:59:02 2024","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.9570000000000001,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.466,"5min":1.466,"15min":1.466},"last":1.036}]},{"osd":3,"last update":"Sat Sep 21 14:59:02 2024","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":2.0870000000000002,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":1.0589999999999999},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.5569999999999999,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":1.0820000000000001}]},{"osd":4,"last update":"Sat Sep 21 14:59:02 2024","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":2.7130000000000001,"5min":2.7130000000000001,"15min":2.7130000000000001},"last":0.95299999999999996},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":2.125,"5min":2.125,"15min":2.125},"last":0.78500000000000003}]},{"osd":5,"last update":"Sat Sep 21 14:59:02 2024","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.7929999999999999,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":3.1880000000000002,"5min":3.1880000000000002,"15min":3.1880000000000002},"last":1.022}]},{"osd":6,"last update":"Sat Sep 21 14:59:02 2024","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":3.2730000000000001,"5min":3.2730000000000001,"15min":3.2730000000000001},"last":0.92900000000000005},{"interface":"front","average":{"1min":1.014,"5min":1.014,"15min":1.014},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":2.2879999999999998,"5min":2.2879999999999998,"15min":2.2879999999999998},"last":1.115}]}]},{"osd":6,"up_from":14,"seq":60129542166,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44692,"kb_used_data":5380,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94327148,"statfs":{"total":96636764160,"available":96590999552,"internally_reserved":0,"allocated":5509120,"data_stored":36275380,"data_compressed":812555,"data_compressed_allocated":3317760,"data_compressed_original":35147134,"omap_allocated":1593,"internal_metadata":40237511},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:59:03 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.593,"5min":1.593,"15min":1.593},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.587,"5min":1.587,"15min":1.587},"last":0.93600000000000005}]},{"osd":1,"last update":"Sat Sep 21 14:59:03 2024","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":1.1120000000000001},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.67100000000000004}]},{"osd":2,"last update":"Sat Sep 21 14:59:03 2024","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.758,"5min":1.758,"15min":1.758},"last":1.1819999999999999},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.7949999999999999,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.97999999999999998}]},{"osd":3,"last update":"Sat Sep 21 14:59:03 2024","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.835,"5min":1.835,"15min":1.835},"last":0.71699999999999997},{"interface":"front","average":{"1min":1.145,"5min":1.145,"15min":1.145},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":1.258}]},{"osd":4,"last update":"Sat Sep 21 14:59:03 2024","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.82399999999999995},{"interface":"front","average":{"1min":1.04,"5min":1.04,"15min":1.04},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.897,"5min":1.897,"15min":1.897},"last":1.1319999999999999}]},{"osd":5,"last update":"Sat Sep 21 14:59:03 2024","interfaces":[{"interface":"back","average":{"1min":1.143,"5min":1.143,"15min":1.143},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.7310000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":1.161},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":1.2869999999999999}]},{"osd":7,"last update":"Sat Sep 21 14:59:03 2024","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"min":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"max":{"1min":1.8660000000000001,"5min":1.8660000000000001,"15min":1.8660000000000001},"last":1.389},{"interface":"front","average":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"min":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":2.004,"5min":2.004,"15min":2.004},"last":1.0880000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542167,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45392,"kb_used_data":6076,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94326448,"statfs":{"total":96636764160,"available":96590282752,"internally_reserved":0,"allocated":6221824,"data_stored":45612648,"data_compressed":663859,"data_compressed_allocated":3842048,"data_compressed_original":44381923,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:59:00 2024","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":4.4800000000000004,"5min":4.4800000000000004,"15min":4.4800000000000004},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.48399999999999999}]},{"osd":1,"last update":"Sat Sep 21 14:59:00 2024","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.75,"5min":1.75,"15min":1.75},"last":1.042},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":2.2400000000000002,"5min":2.2400000000000002,"15min":2.2400000000000002},"last":0.97199999999999998}]},{"osd":2,"last update":"Sat Sep 21 14:59:00 2024","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":3.419,"5min":3.419,"15min":3.419},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.423,"5min":1.423,"15min":1.423},"last":0.92400000000000004}]},{"osd":3,"last update":"Sat Sep 21 14:59:00 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.70399999999999996},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.867,"5min":1.867,"15min":1.867},"last":0.89000000000000001}]},{"osd":4,"last update":"Sat Sep 21 14:59:00 2024","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.03,"15min":1.03},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.383,"5min":1.383,"15min":1.383},"last":1.0720000000000001},{"interface":"front","average":{"1min":1.109,"5min":1.109,"15min":1.109},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.623,"5min":1.623,"15min":1.623},"last":0.82599999999999996}]},{"osd":6,"last update":"Sat Sep 21 14:59:00 2024","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.244,"5min":1.244,"15min":1.244},"last":0.78900000000000003},{"interface":"front","average":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":4.2469999999999999,"5min":4.2469999999999999,"15min":4.2469999999999999},"last":1.0089999999999999}]},{"osd":7,"last update":"Sat Sep 21 14:59:04 2024","interfaces":[{"interface":"back","average":{"1min":1.385,"5min":1.385,"15min":1.385},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":3.3010000000000002,"5min":3.3010000000000002,"15min":3.3010000000000002},"last":0.95199999999999996},{"interface":"front","average":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":2.1240000000000001,"5min":2.1240000000000001,"15min":2.1240000000000001},"last":0.86099999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542166,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43988,"kb_used_data":4676,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94327852,"statfs":{"total":96636764160,"available":96591720448,"internally_reserved":0,"allocated":4788224,"data_stored":36642415,"data_compressed":665405,"data_compressed_allocated":3043328,"data_compressed_original":35766272,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:59:02 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.94099999999999995},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.873}]},{"osd":1,"last update":"Sat Sep 21 14:59:02 2024","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.99199999999999999}]},{"osd":2,"last update":"Sat Sep 21 14:59:02 2024","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":1.032},{"interface":"front","average":{"1min":1.038,"5min":1.038,"15min":1.038},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.63500000000000001}]},{"osd":3,"last update":"Sat Sep 21 14:59:02 2024","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":1.268}]},{"osd":5,"last update":"Sat Sep 21 14:59:02 2024","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":1.1220000000000001},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":1.1000000000000001}]},{"osd":6,"last update":"Sat Sep 21 14:59:02 2024","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":1.1719999999999999},{"interface":"front","average":{"1min":1.095,"5min":1.095,"15min":1.095},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.6140000000000001,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":1.1970000000000001}]},{"osd":7,"last update":"Sat Sep 21 14:59:02 2024","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.04,"15min":1.04},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.7010000000000001,"5min":1.7010000000000001,"15min":1.7010000000000001},"last":1.1519999999999999},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.53,"5min":1.53,"15min":1.53},"last":1.3129999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542167,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50324,"kb_used_data":6912,"kb_used_omap":1,"kb_used_meta":43390,"kb_avail":94321516,"statfs":{"total":96636764160,"available":96585232384,"internally_reserved":0,"allocated":7077888,"data_stored":51650221,"data_compressed":905468,"data_compressed_allocated":4476928,"data_compressed_original":50272516,"omap_allocated":1592,"internal_metadata":44431816},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:59:04 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.919,"5min":1.919,"15min":1.919},"last":0.65300000000000002}]},{"osd":1,"last update":"Sat Sep 21 14:59:04 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.67900000000000005}]},{"osd":2,"last update":"Sat Sep 21 14:59:04 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":1.0049999999999999}]},{"osd":4,"last update":"Sat Sep 21 14:59:04 2024","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.7609999999999999,"5min":1.7609999999999999,"15min":1.7609999999999999},"last":0.879},{"interface":"front","average":{"1min":1.046,"5min":1.046,"15min":1.046},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":2.028,"5min":2.028,"15min":2.028},"last":0.82899999999999996}]},{"osd":5,"last update":"Sat Sep 21 14:59:04 2024","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":2.0899999999999999,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":0.77900000000000003},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.431,"5min":1.431,"15min":1.431},"last":0.83799999999999997}]},{"osd":6,"last update":"Sat Sep 21 14:59:04 2024","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.105,"15min":1.105},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":2.1219999999999999,"5min":2.1219999999999999,"15min":2.1219999999999999},"last":0.72799999999999998},{"interface":"front","average":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.8740000000000001,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":0.76300000000000001}]},{"osd":7,"last update":"Sat Sep 21 14:59:04 2024","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.145,"15min":1.145},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.706,"5min":1.706,"15min":1.706},"last":0.93500000000000005},{"interface":"front","average":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":2.3380000000000001,"5min":2.3380000000000001,"15min":2.3380000000000001},"last":1.024}]}]},{"osd":2,"up_from":14,"seq":60129542165,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42592,"kb_used_data":3284,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94329248,"statfs":{"total":96636764160,"available":96593149952,"internally_reserved":0,"allocated":3362816,"data_stored":18049465,"data_compressed":447652,"data_compressed_allocated":1687552,"data_compressed_original":17215542,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:59:00 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.244,"5min":1.244,"15min":1.244},"last":0.81499999999999995}]},{"osd":1,"last update":"Sat Sep 21 14:59:00 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":0.62},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.89300000000000002}]},{"osd":3,"last update":"Sat Sep 21 14:59:00 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":2.1259999999999999,"5min":2.1259999999999999,"15min":2.1259999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.653,"5min":1.653,"15min":1.653},"last":1.0960000000000001}]},{"osd":4,"last update":"Sat Sep 21 14:59:00 2024","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.46,"5min":1.46,"15min":1.46},"last":0.752},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.675,"5min":1.675,"15min":1.675},"last":1.1679999999999999}]},{"osd":5,"last update":"Sat Sep 21 14:59:00 2024","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":1.008,"15min":1.008},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.655,"5min":1.655,"15min":1.655},"last":0.86099999999999999},{"interface":"front","average":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.546,"5min":1.546,"15min":1.546},"last":1.127}]},{"osd":6,"last update":"Sat Sep 21 14:59:00 2024","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.7889999999999999,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":0.98999999999999999}]},{"osd":7,"last update":"Sat Sep 21 14:59:06 2024","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.395,"5min":1.395,"15min":1.395},"last":1.3220000000000001},{"interface":"front","average":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.5609999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":1.2170000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542166,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33272,"kb_used_data":6244,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94338568,"statfs":{"total":96636764160,"available":96602693632,"internally_reserved":0,"allocated":6393856,"data_stored":49164555,"data_compressed":720505,"data_compressed_allocated":4063232,"data_compressed_original":47954109,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 14:59:02 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.501},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.42599999999999999}]},{"osd":2,"last update":"Sat Sep 21 14:59:02 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":2.5529999999999999,"5min":2.5529999999999999,"15min":2.5529999999999999},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.58099999999999996}]},{"osd":3,"last update":"Sat Sep 21 14:59:02 2024","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":3.081,"5min":3.081,"15min":3.081},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":3.2320000000000002,"5min":3.2320000000000002,"15min":3.2320000000000002},"last":0.66500000000000004}]},{"osd":4,"last update":"Sat Sep 21 14:59:02 2024","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":1.7809999999999999,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":0.86099999999999999}]},{"osd":5,"last update":"Sat Sep 21 14:59:02 2024","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.474,"5min":1.474,"15min":1.474},"last":0.92000000000000004}]},{"osd":6,"last update":"Sat Sep 21 14:59:02 2024","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":2.3519999999999999,"5min":2.3519999999999999,"15min":2.3519999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.69599999999999995}]},{"osd":7,"last update":"Sat Sep 21 14:59:02 2024","interfaces":[{"interface":"back","average":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":2.2360000000000002,"5min":2.2360000000000002,"15min":2.2360000000000002},"last":0.79100000000000004},{"interface":"front","average":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.5229999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.81699999999999995}]}]},{"osd":0,"up_from":14,"seq":60129542165,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49352,"kb_used_data":5944,"kb_used_omap":1,"kb_used_meta":43390,"kb_avail":94322488,"statfs":{"total":96636764160,"available":96586227712,"internally_reserved":0,"allocated":6086656,"data_stored":42017824,"data_compressed":901618,"data_compressed_allocated":3805184,"data_compressed_original":40858783,"omap_allocated":1593,"internal_metadata":44431815},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 21 14:59:04 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.6000000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.64000000000000001}]},{"osd":2,"last update":"Sat Sep 21 14:59:04 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.748},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.58399999999999996}]},{"osd":3,"last update":"Sat Sep 21 14:59:04 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.67700000000000005}]},{"osd":4,"last update":"Sat Sep 21 14:59:04 2024","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.7629999999999999,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":0.82499999999999996},{"interface":"front","average":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.665,"5min":1.665,"15min":1.665},"last":0.91600000000000004}]},{"osd":5,"last update":"Sat Sep 21 14:59:04 2024","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":1.036,"15min":1.036},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.97399999999999998},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":2.1520000000000001,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":1.0900000000000001}]},{"osd":6,"last update":"Sat Sep 21 14:59:04 2024","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":1.6950000000000001,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":0.93799999999999994},{"interface":"front","average":{"1min":1.071,"5min":1.071,"15min":1.071},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.5569999999999999,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":1.0640000000000001}]},{"osd":7,"last update":"Sat Sep 21 14:59:04 2024","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":1.1399999999999999},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":1.0209999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5202,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5202,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5328896,"data_stored":46738863,"data_compressed":968294,"data_compressed_allocated":4190208,"data_compressed_original":46118047,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":5464064,"data_stored":52041708,"data_compressed":746871,"data_compressed_allocated":4300800,"data_compressed_original":51386557,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2220032,"data_stored":17767042,"data_compressed":447796,"data_compressed_allocated":1703936,"data_compressed_original":17477686,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":6103040,"data_stored":54120291,"data_compressed":907124,"data_compressed_allocated":4665344,"data_compressed_original":53287172,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4063232,"data_stored":42001061,"data_compressed":744822,"data_compressed_allocated":3473408,"data_compressed_original":41668608,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":5070848,"data_stored":45526289,"data_compressed":659233,"data_compressed_allocated":3870720,"data_compressed_original":44857059,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4825088,"data_stored":42026341,"data_compressed":889215,"data_compressed_allocated":3776512,"data_compressed_original":41434494,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":3481600,"data_stored":33199015,"data_compressed":525681,"data_compressed_allocated":2777088,"data_compressed_original":32819934,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T14:59:38.411 INFO:tasks.thrashosds.thrasher:Setting 3.3 to [5, 2, 1, 0] 2024-09-21T14:59:38.411 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.3', '5', '2', '1', '0'] 2024-09-21T14:59:38.411 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.3 5 2 1 0 2024-09-21T14:59:38.413 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:38.425 INFO:teuthology.orchestra.run.smithi032.stdout:1906 trunc from 0xcca7bfd to 0x4df3ed0 2024-09-21T14:59:38.452 INFO:teuthology.orchestra.run.smithi032.stdout:1907 writesame 0x52f7c71 thru 0x52f9dad data_size 0x1(0x213d bytes) 2024-09-21T14:59:38.453 INFO:teuthology.orchestra.run.smithi032.stdout:1908 trunc from 0x52f9dae to 0x49ece53 2024-09-21T14:59:38.460 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:38.460 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T14:59:38.460 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T14:59:38.460 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T14:59:38.461 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.461 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T14:59:38.461 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.205754+0000", 2024-09-21T14:59:38.461 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 95.253703842999997, 2024-09-21T14:59:38.461 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.41117094199999998, 2024-09-21T14:59:38.461 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:38.461 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:38.461 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:38.461 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:59:38.461 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:59:38.461 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1 2024-09-21T14:59:38.461 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.461 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:38.461 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.461 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:38.462 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205754+0000", 2024-09-21T14:59:38.462 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:38.462 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.462 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.462 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:38.462 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205754+0000", 2024-09-21T14:59:38.462 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:38.462 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.462 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.462 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:38.462 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205752+0000", 2024-09-21T14:59:38.462 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T14:59:38.462 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.463 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.463 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:38.463 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205765+0000", 2024-09-21T14:59:38.463 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3230999999999999e-05 2024-09-21T14:59:38.463 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.463 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.463 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:38.463 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205782+0000", 2024-09-21T14:59:38.463 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6634e-05 2024-09-21T14:59:38.463 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.463 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.463 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:38.463 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205793+0000", 2024-09-21T14:59:38.463 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1297e-05 2024-09-21T14:59:38.464 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.464 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.464 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:38.464 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205867+0000", 2024-09-21T14:59:38.464 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4269000000000003e-05 2024-09-21T14:59:38.464 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.464 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.464 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T14:59:38.464 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205877+0000", 2024-09-21T14:59:38.464 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2690000000000005e-06 2024-09-21T14:59:38.464 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.464 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.464 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:38.464 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.613700+0000", 2024-09-21T14:59:38.465 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.407823455 2024-09-21T14:59:38.465 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.465 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.465 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:38.465 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.614203+0000", 2024-09-21T14:59:38.465 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00050311500000000005 2024-09-21T14:59:38.465 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.465 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.465 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:59:38.465 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.614750+0000", 2024-09-21T14:59:38.465 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00054738699999999998 2024-09-21T14:59:38.465 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.465 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.465 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:38.465 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616367+0000", 2024-09-21T14:59:38.466 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001616634 2024-09-21T14:59:38.466 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.466 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.466 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:38.466 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616574+0000", 2024-09-21T14:59:38.466 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00020668499999999999 2024-09-21T14:59:38.466 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.466 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.466 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:38.466 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616802+0000", 2024-09-21T14:59:38.466 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00022817900000000001 2024-09-21T14:59:38.466 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.466 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.467 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:38.467 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616925+0000", 2024-09-21T14:59:38.467 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012309699999999999 2024-09-21T14:59:38.467 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:38.467 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:38.467 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:38.467 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.467 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.467 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T14:59:38.467 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.955609+0000", 2024-09-21T14:59:38.467 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 94.503849455999998, 2024-09-21T14:59:38.467 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013624153999999999, 2024-09-21T14:59:38.467 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:38.467 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:38.467 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:38.467 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:59:38.468 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:59:38.468 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 115 2024-09-21T14:59:38.468 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.468 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:38.468 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.468 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:38.468 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955609+0000", 2024-09-21T14:59:38.468 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:38.468 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.468 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.468 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:38.468 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955609+0000", 2024-09-21T14:59:38.468 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:38.468 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.468 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.469 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:38.469 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955607+0000", 2024-09-21T14:59:38.469 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-21T14:59:38.469 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.469 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.469 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:38.469 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955732+0000", 2024-09-21T14:59:38.469 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000125566 2024-09-21T14:59:38.469 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.469 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.469 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:38.469 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955752+0000", 2024-09-21T14:59:38.469 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9916999999999999e-05 2024-09-21T14:59:38.469 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.470 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.470 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:38.470 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955767+0000", 2024-09-21T14:59:38.470 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4687e-05 2024-09-21T14:59:38.470 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.470 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.470 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:38.470 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.965574+0000", 2024-09-21T14:59:38.470 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0098072850000000007 2024-09-21T14:59:38.470 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.470 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.470 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:38.470 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.965642+0000", 2024-09-21T14:59:38.470 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7784e-05 2024-09-21T14:59:38.470 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.471 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.471 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:59:38.471 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.965753+0000", 2024-09-21T14:59:38.471 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000111111 2024-09-21T14:59:38.471 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.471 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.471 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:38.471 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967717+0000", 2024-09-21T14:59:38.471 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0019635339999999999 2024-09-21T14:59:38.471 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.471 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.471 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:38.471 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.969014+0000", 2024-09-21T14:59:38.471 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001297093 2024-09-21T14:59:38.472 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.472 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.472 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:38.472 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.969130+0000", 2024-09-21T14:59:38.472 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000116011 2024-09-21T14:59:38.472 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.472 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.472 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:38.472 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.969233+0000", 2024-09-21T14:59:38.472 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000102982 2024-09-21T14:59:38.472 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:38.472 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:38.472 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:38.472 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.472 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.473 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T14:59:38.473 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.955783+0000", 2024-09-21T14:59:38.473 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 94.503674786999994, 2024-09-21T14:59:38.473 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014630241, 2024-09-21T14:59:38.473 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:38.473 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:38.473 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:38.473 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:59:38.473 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:59:38.473 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 116 2024-09-21T14:59:38.473 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.473 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:38.473 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.473 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:38.474 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955783+0000", 2024-09-21T14:59:38.474 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:38.474 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.474 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.474 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:38.474 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955783+0000", 2024-09-21T14:59:38.474 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:38.474 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.474 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.474 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:38.474 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955782+0000", 2024-09-21T14:59:38.474 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:59:38.474 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.474 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.475 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:38.475 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955791+0000", 2024-09-21T14:59:38.475 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8089999999999999e-06 2024-09-21T14:59:38.475 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.475 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.475 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:38.475 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955804+0000", 2024-09-21T14:59:38.475 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2548000000000001e-05 2024-09-21T14:59:38.475 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.475 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.475 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:38.475 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955814+0000", 2024-09-21T14:59:38.475 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0158e-05 2024-09-21T14:59:38.475 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.476 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.476 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:38.476 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.966515+0000", 2024-09-21T14:59:38.476 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.010701756 2024-09-21T14:59:38.476 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.476 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.476 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:38.476 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.966606+0000", 2024-09-21T14:59:38.476 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0761999999999996e-05 2024-09-21T14:59:38.476 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.476 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.476 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:59:38.476 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.966740+0000", 2024-09-21T14:59:38.476 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000133693 2024-09-21T14:59:38.476 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.477 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.477 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:38.477 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.968920+0000", 2024-09-21T14:59:38.477 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0021799290000000002 2024-09-21T14:59:38.477 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.477 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.477 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:38.477 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.970320+0000", 2024-09-21T14:59:38.477 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0014001759999999999 2024-09-21T14:59:38.477 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.477 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.477 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:38.477 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.970376+0000", 2024-09-21T14:59:38.477 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5696999999999999e-05 2024-09-21T14:59:38.477 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.477 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.478 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:38.478 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.970413+0000", 2024-09-21T14:59:38.478 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7750999999999998e-05 2024-09-21T14:59:38.478 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:38.478 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:38.478 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:38.478 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.478 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.478 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T14:59:38.478 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.955829+0000", 2024-09-21T14:59:38.478 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 94.503629317000005, 2024-09-21T14:59:38.478 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014919597999999999, 2024-09-21T14:59:38.478 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:38.478 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:38.479 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:38.479 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:59:38.479 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:59:38.479 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 117 2024-09-21T14:59:38.479 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.479 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:38.479 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.479 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:38.479 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955829+0000", 2024-09-21T14:59:38.479 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:38.479 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.479 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.479 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:38.479 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955829+0000", 2024-09-21T14:59:38.479 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:38.479 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.480 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.480 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:38.480 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955827+0000", 2024-09-21T14:59:38.480 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T14:59:38.480 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.480 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.480 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:38.480 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955838+0000", 2024-09-21T14:59:38.480 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.111e-05 2024-09-21T14:59:38.480 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.480 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.480 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:38.480 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955875+0000", 2024-09-21T14:59:38.480 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6211999999999998e-05 2024-09-21T14:59:38.480 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.480 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.481 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:38.481 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955899+0000", 2024-09-21T14:59:38.481 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4128999999999999e-05 2024-09-21T14:59:38.481 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.481 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.481 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:38.481 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967767+0000", 2024-09-21T14:59:38.481 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.011868251999999999 2024-09-21T14:59:38.481 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.481 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.481 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:38.481 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967866+0000", 2024-09-21T14:59:38.481 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8664000000000003e-05 2024-09-21T14:59:38.481 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.481 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.482 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:59:38.482 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.968128+0000", 2024-09-21T14:59:38.482 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00026243600000000001 2024-09-21T14:59:38.482 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.482 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.482 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:38.482 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.969393+0000", 2024-09-21T14:59:38.482 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0012650350000000001 2024-09-21T14:59:38.482 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.482 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.482 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:38.482 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.970523+0000", 2024-09-21T14:59:38.482 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001129557 2024-09-21T14:59:38.483 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.483 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.483 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:38.483 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.970603+0000", 2024-09-21T14:59:38.483 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0090000000000001e-05 2024-09-21T14:59:38.483 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.483 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.483 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:38.483 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.970748+0000", 2024-09-21T14:59:38.483 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014552799999999999 2024-09-21T14:59:38.483 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:38.483 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:38.483 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:38.483 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.483 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.484 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T14:59:38.484 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:04.493315+0000", 2024-09-21T14:59:38.484 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 93.966142484000002, 2024-09-21T14:59:38.484 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014055766000000001, 2024-09-21T14:59:38.484 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:38.484 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:38.484 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:38.484 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:59:38.484 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:59:38.484 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 137 2024-09-21T14:59:38.484 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.484 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:38.484 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.484 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:38.485 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493315+0000", 2024-09-21T14:59:38.485 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:38.485 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.485 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.485 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:38.485 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493315+0000", 2024-09-21T14:59:38.485 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:38.485 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.485 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.485 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:38.485 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493314+0000", 2024-09-21T14:59:38.485 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:59:38.485 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.485 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.486 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:38.486 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493507+0000", 2024-09-21T14:59:38.486 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000192444 2024-09-21T14:59:38.486 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.486 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.486 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:38.486 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493528+0000", 2024-09-21T14:59:38.486 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0845000000000001e-05 2024-09-21T14:59:38.486 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.486 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.486 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:38.486 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493543+0000", 2024-09-21T14:59:38.486 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5800000000000001e-05 2024-09-21T14:59:38.486 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.487 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.487 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:38.487 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.502575+0000", 2024-09-21T14:59:38.487 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0090312389999999999 2024-09-21T14:59:38.487 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.487 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.487 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:38.487 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.502677+0000", 2024-09-21T14:59:38.487 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000102271 2024-09-21T14:59:38.487 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.487 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.487 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:59:38.487 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.502825+0000", 2024-09-21T14:59:38.487 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000148171 2024-09-21T14:59:38.488 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.488 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.488 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:38.488 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507251+0000", 2024-09-21T14:59:38.488 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0044253879999999997 2024-09-21T14:59:38.488 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.488 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.488 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:38.488 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507313+0000", 2024-09-21T14:59:38.488 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2170999999999998e-05 2024-09-21T14:59:38.488 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.488 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.488 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:38.488 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507346+0000", 2024-09-21T14:59:38.488 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3079e-05 2024-09-21T14:59:38.489 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.489 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.489 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:38.489 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507371+0000", 2024-09-21T14:59:38.489 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5477999999999999e-05 2024-09-21T14:59:38.489 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:38.489 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:38.489 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:38.489 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.489 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.489 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T14:59:38.489 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:04.493570+0000", 2024-09-21T14:59:38.489 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 93.965888078999996, 2024-09-21T14:59:38.489 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014061686, 2024-09-21T14:59:38.490 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:38.490 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:38.490 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:38.490 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:59:38.490 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:59:38.490 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 138 2024-09-21T14:59:38.490 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.490 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:38.490 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.490 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:38.490 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493570+0000", 2024-09-21T14:59:38.490 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:38.490 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.490 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.491 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:38.491 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493570+0000", 2024-09-21T14:59:38.491 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:38.491 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.491 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.491 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:38.491 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493568+0000", 2024-09-21T14:59:38.491 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T14:59:38.491 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.491 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.491 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:38.491 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493579+0000", 2024-09-21T14:59:38.491 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0574000000000001e-05 2024-09-21T14:59:38.492 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.492 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.492 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:38.492 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493595+0000", 2024-09-21T14:59:38.492 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6504999999999999e-05 2024-09-21T14:59:38.492 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.492 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.492 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:38.492 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493608+0000", 2024-09-21T14:59:38.492 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2836999999999999e-05 2024-09-21T14:59:38.492 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.492 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.492 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:38.492 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.503909+0000", 2024-09-21T14:59:38.492 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.01030118 2024-09-21T14:59:38.492 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.493 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.493 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:38.493 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.504064+0000", 2024-09-21T14:59:38.493 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015493000000000001 2024-09-21T14:59:38.493 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.493 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.493 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:59:38.493 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.504299+0000", 2024-09-21T14:59:38.493 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00023416100000000001 2024-09-21T14:59:38.493 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.493 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.493 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:38.493 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507466+0000", 2024-09-21T14:59:38.493 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0031678380000000001 2024-09-21T14:59:38.493 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.494 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.494 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:38.494 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507483+0000", 2024-09-21T14:59:38.494 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6606999999999999e-05 2024-09-21T14:59:38.494 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.494 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.494 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:38.494 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507566+0000", 2024-09-21T14:59:38.494 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3110999999999998e-05 2024-09-21T14:59:38.494 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.494 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.494 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:38.494 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507632+0000", 2024-09-21T14:59:38.495 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5480000000000003e-05 2024-09-21T14:59:38.495 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:38.495 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:38.495 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:38.495 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.495 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.495 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T14:59:38.495 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:04.493625+0000", 2024-09-21T14:59:38.495 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 93.965833391000004, 2024-09-21T14:59:38.495 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014665378999999999, 2024-09-21T14:59:38.495 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:38.495 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:38.495 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:38.495 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:59:38.495 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:59:38.496 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139 2024-09-21T14:59:38.496 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.496 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:38.496 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.496 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:38.496 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493625+0000", 2024-09-21T14:59:38.496 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:38.496 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.496 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.496 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:38.496 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493625+0000", 2024-09-21T14:59:38.496 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:38.496 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.496 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.496 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:38.496 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493624+0000", 2024-09-21T14:59:38.497 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:59:38.497 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.497 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.497 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:38.497 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493638+0000", 2024-09-21T14:59:38.497 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4282e-05 2024-09-21T14:59:38.497 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.497 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.497 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:38.497 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493654+0000", 2024-09-21T14:59:38.497 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5739999999999998e-05 2024-09-21T14:59:38.497 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.497 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.497 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:38.497 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493666+0000", 2024-09-21T14:59:38.497 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2865e-05 2024-09-21T14:59:38.498 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.498 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.498 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:38.498 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.505329+0000", 2024-09-21T14:59:38.498 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.011662237000000001 2024-09-21T14:59:38.498 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.498 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.498 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:38.498 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.505377+0000", 2024-09-21T14:59:38.498 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7862999999999999e-05 2024-09-21T14:59:38.498 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.498 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.498 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:59:38.498 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.505455+0000", 2024-09-21T14:59:38.499 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8918000000000006e-05 2024-09-21T14:59:38.499 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.499 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.499 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:38.499 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507650+0000", 2024-09-21T14:59:38.499 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.002195027 2024-09-21T14:59:38.499 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.499 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.499 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:38.499 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508149+0000", 2024-09-21T14:59:38.499 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00049828399999999999 2024-09-21T14:59:38.499 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.499 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.499 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:38.500 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508230+0000", 2024-09-21T14:59:38.500 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1376000000000002e-05 2024-09-21T14:59:38.500 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.500 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.500 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:38.500 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508290+0000", 2024-09-21T14:59:38.500 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9852999999999998e-05 2024-09-21T14:59:38.500 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:38.500 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:38.500 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:38.500 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.500 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.500 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T14:59:38.500 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:04.493679+0000", 2024-09-21T14:59:38.500 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 93.965778493000002, 2024-09-21T14:59:38.500 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014869869000000001, 2024-09-21T14:59:38.501 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:38.501 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:38.501 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:38.501 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:59:38.501 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:59:38.501 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140 2024-09-21T14:59:38.501 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.501 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:38.501 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.501 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:38.501 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493679+0000", 2024-09-21T14:59:38.501 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:38.501 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.501 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.501 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:38.502 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493679+0000", 2024-09-21T14:59:38.502 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:38.502 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.502 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.502 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:38.502 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493679+0000", 2024-09-21T14:59:38.502 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:59:38.502 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.502 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.502 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:38.502 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493889+0000", 2024-09-21T14:59:38.502 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00021032700000000001 2024-09-21T14:59:38.502 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.503 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.503 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:38.503 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493931+0000", 2024-09-21T14:59:38.503 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1971000000000002e-05 2024-09-21T14:59:38.503 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.503 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.503 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:38.503 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493947+0000", 2024-09-21T14:59:38.503 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6385e-05 2024-09-21T14:59:38.503 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.503 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.503 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:38.503 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.505804+0000", 2024-09-21T14:59:38.503 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.011857136000000001 2024-09-21T14:59:38.504 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.504 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.504 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:38.504 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.505992+0000", 2024-09-21T14:59:38.504 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000187665 2024-09-21T14:59:38.504 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.504 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.504 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:59:38.504 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.506281+0000", 2024-09-21T14:59:38.504 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000288712 2024-09-21T14:59:38.504 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.504 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.504 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:38.504 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508349+0000", 2024-09-21T14:59:38.505 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0020685790000000001 2024-09-21T14:59:38.505 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.505 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.505 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:38.505 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508371+0000", 2024-09-21T14:59:38.505 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1747e-05 2024-09-21T14:59:38.505 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.505 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.505 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:38.505 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508450+0000", 2024-09-21T14:59:38.505 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9332000000000005e-05 2024-09-21T14:59:38.505 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.505 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.505 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:38.505 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508549+0000", 2024-09-21T14:59:38.505 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8918000000000004e-05 2024-09-21T14:59:38.506 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:38.506 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:38.506 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:38.506 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.506 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.506 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T14:59:38.506 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:04.493967+0000", 2024-09-21T14:59:38.506 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 93.965491455999995, 2024-09-21T14:59:38.506 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.015281075999999999, 2024-09-21T14:59:38.506 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:38.506 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:38.506 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:38.506 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:59:38.506 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:59:38.507 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 141 2024-09-21T14:59:38.507 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.507 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:38.507 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.507 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:38.507 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493967+0000", 2024-09-21T14:59:38.507 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:38.507 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.507 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.507 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:38.507 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493967+0000", 2024-09-21T14:59:38.507 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:38.507 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.507 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.507 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:38.508 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493965+0000", 2024-09-21T14:59:38.508 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T14:59:38.508 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.508 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.508 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:38.508 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493987+0000", 2024-09-21T14:59:38.508 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1662999999999999e-05 2024-09-21T14:59:38.508 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.508 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.508 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:38.508 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.494004+0000", 2024-09-21T14:59:38.508 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7311000000000001e-05 2024-09-21T14:59:38.508 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.508 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.508 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:38.509 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.494017+0000", 2024-09-21T14:59:38.509 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3202e-05 2024-09-21T14:59:38.509 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.509 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.509 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:38.509 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507678+0000", 2024-09-21T14:59:38.509 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013660432 2024-09-21T14:59:38.509 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.509 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.509 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:38.509 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507726+0000", 2024-09-21T14:59:38.509 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8154e-05 2024-09-21T14:59:38.509 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.509 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.510 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:59:38.510 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507797+0000", 2024-09-21T14:59:38.510 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.1110000000000002e-05 2024-09-21T14:59:38.510 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.510 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.510 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:38.510 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508710+0000", 2024-09-21T14:59:38.510 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00091339299999999997 2024-09-21T14:59:38.510 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.510 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.510 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:38.510 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.509108+0000", 2024-09-21T14:59:38.510 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00039718300000000001 2024-09-21T14:59:38.510 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.511 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.511 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:38.511 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.509186+0000", 2024-09-21T14:59:38.511 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8592999999999998e-05 2024-09-21T14:59:38.511 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.511 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.511 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:38.511 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.509248+0000", 2024-09-21T14:59:38.511 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1450000000000003e-05 2024-09-21T14:59:38.511 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:38.511 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:38.511 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:38.511 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.511 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.511 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6626 3.1a 3:5adee3e5:::rbd_data.11361bed1f78.0000000000000001:d [read 229376~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:59:38.512 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131593+0000", 2024-09-21T14:59:38.512 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 63.327864878, 2024-09-21T14:59:38.512 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012576725, 2024-09-21T14:59:38.512 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:38.512 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:38.512 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:38.512 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:38.512 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:38.512 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6626 2024-09-21T14:59:38.512 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.512 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:38.512 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.512 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:38.512 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131593+0000", 2024-09-21T14:59:38.512 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:38.512 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.513 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.513 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:38.513 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131593+0000", 2024-09-21T14:59:38.513 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:38.513 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.513 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.513 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:38.513 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131593+0000", 2024-09-21T14:59:38.513 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T14:59:38.513 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.513 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.513 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:38.513 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131598+0000", 2024-09-21T14:59:38.513 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7849999999999999e-06 2024-09-21T14:59:38.513 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.514 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.514 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:38.514 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131601+0000", 2024-09-21T14:59:38.514 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1949999999999999e-06 2024-09-21T14:59:38.514 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.514 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.514 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:38.514 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131604+0000", 2024-09-21T14:59:38.514 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6350000000000002e-06 2024-09-21T14:59:38.514 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.514 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.514 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:38.514 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.143902+0000", 2024-09-21T14:59:38.514 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012298216000000001 2024-09-21T14:59:38.515 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.515 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.515 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:38.515 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.143993+0000", 2024-09-21T14:59:38.515 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1073e-05 2024-09-21T14:59:38.515 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.515 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.515 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:38.515 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.144170+0000", 2024-09-21T14:59:38.515 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017703 2024-09-21T14:59:38.515 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:38.515 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:38.515 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:38.515 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.515 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.516 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6631 3.1a 3:5d547f54:::rbd_data.11361bed1f78.0000000000000003:d [read 0~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:59:38.516 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131606+0000", 2024-09-21T14:59:38.516 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 63.327852178000001, 2024-09-21T14:59:38.516 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013903148000000001, 2024-09-21T14:59:38.516 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:38.516 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:38.516 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:38.516 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:38.516 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:38.516 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6631 2024-09-21T14:59:38.516 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.516 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:38.516 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.516 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:38.517 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131606+0000", 2024-09-21T14:59:38.517 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:38.517 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.517 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.517 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:38.517 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131606+0000", 2024-09-21T14:59:38.517 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:38.517 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.517 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.517 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:38.517 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131606+0000", 2024-09-21T14:59:38.517 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T14:59:38.517 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.517 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.517 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:38.517 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131607+0000", 2024-09-21T14:59:38.518 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2699999999999999e-06 2024-09-21T14:59:38.518 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.518 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.518 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:38.518 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131610+0000", 2024-09-21T14:59:38.518 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.88e-06 2024-09-21T14:59:38.518 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.518 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.518 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:38.518 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131612+0000", 2024-09-21T14:59:38.518 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5170000000000002e-06 2024-09-21T14:59:38.518 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.518 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.518 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:38.519 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.144205+0000", 2024-09-21T14:59:38.519 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012592763 2024-09-21T14:59:38.519 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.519 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.519 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:38.519 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.144332+0000", 2024-09-21T14:59:38.519 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012694200000000001 2024-09-21T14:59:38.519 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.519 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.519 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:38.519 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.145509+0000", 2024-09-21T14:59:38.519 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001176992 2024-09-21T14:59:38.519 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:38.519 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:38.519 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:38.519 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.520 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.520 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6632 3.1a 3:5d547f54:::rbd_data.11361bed1f78.0000000000000003:d [read 81920~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:59:38.520 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131614+0000", 2024-09-21T14:59:38.520 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 63.327843569999999, 2024-09-21T14:59:38.520 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014093553999999999, 2024-09-21T14:59:38.520 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:38.520 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:38.520 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:38.520 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:38.520 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:38.520 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6632 2024-09-21T14:59:38.520 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.520 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:38.520 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.520 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:38.521 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131614+0000", 2024-09-21T14:59:38.521 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:38.521 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.521 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.521 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:38.521 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131614+0000", 2024-09-21T14:59:38.521 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:38.521 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.521 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.521 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:38.521 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131614+0000", 2024-09-21T14:59:38.521 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T14:59:38.521 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.521 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.522 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:38.522 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131615+0000", 2024-09-21T14:59:38.522 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.218e-06 2024-09-21T14:59:38.522 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.522 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.522 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:38.522 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131618+0000", 2024-09-21T14:59:38.522 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9050000000000001e-06 2024-09-21T14:59:38.522 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.522 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.522 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:38.522 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131621+0000", 2024-09-21T14:59:38.522 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7439999999999999e-06 2024-09-21T14:59:38.522 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.523 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.523 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:38.523 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.145538+0000", 2024-09-21T14:59:38.523 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013916618 2024-09-21T14:59:38.523 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.523 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.523 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:38.523 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.145619+0000", 2024-09-21T14:59:38.523 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1812999999999997e-05 2024-09-21T14:59:38.523 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.523 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.523 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:38.523 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.145708+0000", 2024-09-21T14:59:38.523 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8460000000000003e-05 2024-09-21T14:59:38.523 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:38.524 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:38.524 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:38.524 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.524 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.524 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6633 3.1a 3:5d547f54:::rbd_data.11361bed1f78.0000000000000003:d [read 147456~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:59:38.524 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131623+0000", 2024-09-21T14:59:38.524 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 63.327834604000003, 2024-09-21T14:59:38.524 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014395845000000001, 2024-09-21T14:59:38.524 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:38.524 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:38.524 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:38.524 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:38.524 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:38.524 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6633 2024-09-21T14:59:38.524 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.524 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:38.525 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.525 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:38.525 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131623+0000", 2024-09-21T14:59:38.525 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:38.525 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.525 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.525 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:38.525 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131623+0000", 2024-09-21T14:59:38.525 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:38.525 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.525 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.525 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:38.525 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131623+0000", 2024-09-21T14:59:38.526 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T14:59:38.526 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.526 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.526 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:38.526 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131628+0000", 2024-09-21T14:59:38.526 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0370000000000002e-06 2024-09-21T14:59:38.526 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.526 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.526 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:38.526 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131632+0000", 2024-09-21T14:59:38.526 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3519999999999998e-06 2024-09-21T14:59:38.526 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.526 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.526 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:38.526 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131634+0000", 2024-09-21T14:59:38.527 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5019999999999999e-06 2024-09-21T14:59:38.527 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.527 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.527 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:38.527 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.145734+0000", 2024-09-21T14:59:38.527 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014100118 2024-09-21T14:59:38.527 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.527 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.527 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:38.527 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.145804+0000", 2024-09-21T14:59:38.527 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0229999999999997e-05 2024-09-21T14:59:38.527 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.527 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.527 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:38.528 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146019+0000", 2024-09-21T14:59:38.528 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00021481699999999999 2024-09-21T14:59:38.528 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:38.528 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:38.528 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:38.528 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.528 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.528 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6634 3.1a 3:5d547f54:::rbd_data.11361bed1f78.0000000000000003:d [read 229376~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:59:38.528 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131636+0000", 2024-09-21T14:59:38.528 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 63.327821493999998, 2024-09-21T14:59:38.528 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014645843, 2024-09-21T14:59:38.528 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:38.528 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:38.528 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:38.528 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:38.529 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:38.529 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6634 2024-09-21T14:59:38.529 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.529 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:38.529 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.529 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:38.529 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131636+0000", 2024-09-21T14:59:38.529 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:38.529 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.529 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.529 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:38.529 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131636+0000", 2024-09-21T14:59:38.529 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:38.529 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.529 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.530 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:38.530 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131636+0000", 2024-09-21T14:59:38.530 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:38.530 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.530 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.530 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:38.530 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131641+0000", 2024-09-21T14:59:38.530 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8489999999999997e-06 2024-09-21T14:59:38.530 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.530 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.530 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:38.530 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131644+0000", 2024-09-21T14:59:38.530 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.179e-06 2024-09-21T14:59:38.530 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.530 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.531 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:38.531 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131647+0000", 2024-09-21T14:59:38.531 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8009999999999999e-06 2024-09-21T14:59:38.531 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.531 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.531 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:38.531 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146046+0000", 2024-09-21T14:59:38.531 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014399018 2024-09-21T14:59:38.531 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.531 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.531 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:38.531 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146118+0000", 2024-09-21T14:59:38.531 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2044000000000003e-05 2024-09-21T14:59:38.531 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.532 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.532 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:38.532 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146282+0000", 2024-09-21T14:59:38.532 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000164196 2024-09-21T14:59:38.532 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:38.532 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:38.532 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:38.532 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.532 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.532 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6639 3.1a 3:592d80ed:::rbd_data.11361bed1f78.0000000000000005:d [read 0~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:59:38.532 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131650+0000", 2024-09-21T14:59:38.532 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 63.327808294999997, 2024-09-21T14:59:38.532 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016002832000000002, 2024-09-21T14:59:38.532 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:38.532 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:38.533 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:38.533 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:38.533 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:38.533 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6639 2024-09-21T14:59:38.533 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.533 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:38.533 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.533 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:38.533 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131650+0000", 2024-09-21T14:59:38.533 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:38.533 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.533 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.533 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:38.533 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131650+0000", 2024-09-21T14:59:38.533 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:38.533 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.534 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.534 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:38.534 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131649+0000", 2024-09-21T14:59:38.534 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:38.534 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.534 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.534 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:38.534 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131651+0000", 2024-09-21T14:59:38.534 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9199999999999998e-06 2024-09-21T14:59:38.534 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.534 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.534 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:38.534 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131656+0000", 2024-09-21T14:59:38.534 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9899999999999997e-06 2024-09-21T14:59:38.535 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.535 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.535 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:38.535 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131665+0000", 2024-09-21T14:59:38.535 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0999999999999993e-06 2024-09-21T14:59:38.535 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.535 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.535 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:38.535 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146309+0000", 2024-09-21T14:59:38.535 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014643893999999999 2024-09-21T14:59:38.535 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.535 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.535 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:38.535 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146392+0000", 2024-09-21T14:59:38.535 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2531e-05 2024-09-21T14:59:38.536 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.536 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.536 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:38.536 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147653+0000", 2024-09-21T14:59:38.536 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0012606710000000001 2024-09-21T14:59:38.536 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:38.536 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:38.536 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:38.536 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.536 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.536 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6640 3.1a 3:592d80ed:::rbd_data.11361bed1f78.0000000000000005:d [read 65536~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:59:38.536 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131670+0000", 2024-09-21T14:59:38.536 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 63.327787995999998, 2024-09-21T14:59:38.536 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016188859, 2024-09-21T14:59:38.536 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:38.537 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:38.537 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:38.537 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:38.537 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:38.537 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6640 2024-09-21T14:59:38.537 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.537 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:38.537 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.537 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:38.537 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131670+0000", 2024-09-21T14:59:38.537 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:38.537 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.537 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.537 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:38.537 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131670+0000", 2024-09-21T14:59:38.538 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:38.538 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.538 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.538 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:38.538 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131670+0000", 2024-09-21T14:59:38.538 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T14:59:38.538 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.538 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.538 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:38.538 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131672+0000", 2024-09-21T14:59:38.538 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.818e-06 2024-09-21T14:59:38.538 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.538 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.538 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:38.539 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131675+0000", 2024-09-21T14:59:38.539 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5930000000000002e-06 2024-09-21T14:59:38.539 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.539 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.539 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:38.539 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131678+0000", 2024-09-21T14:59:38.539 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6790000000000001e-06 2024-09-21T14:59:38.539 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.539 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.539 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:38.539 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147681+0000", 2024-09-21T14:59:38.539 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016002755 2024-09-21T14:59:38.539 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.540 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.540 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:38.540 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147762+0000", 2024-09-21T14:59:38.540 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1052999999999998e-05 2024-09-21T14:59:38.540 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.540 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.540 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:38.540 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147859+0000", 2024-09-21T14:59:38.540 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.7199000000000002e-05 2024-09-21T14:59:38.540 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:38.540 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:38.540 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:38.540 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.540 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.541 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6641 3.1a 3:592d80ed:::rbd_data.11361bed1f78.0000000000000005:d [read 147456~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:59:38.541 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131680+0000", 2024-09-21T14:59:38.541 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 63.327777707999999, 2024-09-21T14:59:38.541 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016431296000000001, 2024-09-21T14:59:38.541 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:38.541 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:38.541 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:38.541 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:38.541 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:38.541 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6641 2024-09-21T14:59:38.541 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.541 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:38.541 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.541 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:38.542 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131680+0000", 2024-09-21T14:59:38.542 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:38.542 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.542 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.542 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:38.542 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131680+0000", 2024-09-21T14:59:38.542 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:38.542 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.542 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.542 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:38.542 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131680+0000", 2024-09-21T14:59:38.542 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T14:59:38.542 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.543 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.543 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:38.543 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131681+0000", 2024-09-21T14:59:38.543 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2610000000000001e-06 2024-09-21T14:59:38.543 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.543 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.543 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:38.543 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131685+0000", 2024-09-21T14:59:38.543 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.8299999999999998e-06 2024-09-21T14:59:38.543 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.543 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.543 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:38.543 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131688+0000", 2024-09-21T14:59:38.544 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6189999999999998e-06 2024-09-21T14:59:38.544 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.544 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.544 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:38.544 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147884+0000", 2024-09-21T14:59:38.544 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016196465 2024-09-21T14:59:38.544 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.544 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.544 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:38.544 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147949+0000", 2024-09-21T14:59:38.544 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4867000000000001e-05 2024-09-21T14:59:38.544 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.544 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.544 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:38.544 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.148112+0000", 2024-09-21T14:59:38.545 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00016247700000000001 2024-09-21T14:59:38.545 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:38.545 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:38.545 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:38.545 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.545 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.545 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6642 3.1a 3:592d80ed:::rbd_data.11361bed1f78.0000000000000005:d [read 212992~32768 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:59:38.545 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131690+0000", 2024-09-21T14:59:38.545 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 63.327767829999999, 2024-09-21T14:59:38.545 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016751335999999999, 2024-09-21T14:59:38.545 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:38.545 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:38.545 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:38.545 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:38.545 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:38.545 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6642 2024-09-21T14:59:38.546 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.546 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:38.546 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.546 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:38.546 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131690+0000", 2024-09-21T14:59:38.546 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:38.546 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.546 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.546 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:38.546 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131690+0000", 2024-09-21T14:59:38.546 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:38.546 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.546 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.546 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:38.546 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131690+0000", 2024-09-21T14:59:38.547 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T14:59:38.547 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.547 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.547 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:38.547 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131691+0000", 2024-09-21T14:59:38.547 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3120000000000001e-06 2024-09-21T14:59:38.547 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.547 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.547 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:38.547 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131694+0000", 2024-09-21T14:59:38.547 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0070000000000002e-06 2024-09-21T14:59:38.547 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.547 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.547 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:38.547 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131700+0000", 2024-09-21T14:59:38.547 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0800000000000002e-06 2024-09-21T14:59:38.548 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.548 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.548 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:38.548 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.148135+0000", 2024-09-21T14:59:38.548 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016434250000000001 2024-09-21T14:59:38.548 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.548 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.548 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:38.548 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.148204+0000", 2024-09-21T14:59:38.548 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9466000000000002e-05 2024-09-21T14:59:38.548 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.548 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.548 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:38.548 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.148441+0000", 2024-09-21T14:59:38.548 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00023743700000000001 2024-09-21T14:59:38.549 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:38.549 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:38.549 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:38.549 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.549 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.549 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:61214 3.7 3:e28aec10:::rbd_header.1136862f872f:head [watch reconnect cookie 139864671932080 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-21T14:59:38.549 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:18.147172+0000", 2024-09-21T14:59:38.549 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 20.312286199999999, 2024-09-21T14:59:38.549 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.019264956, 2024-09-21T14:59:38.549 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:38.549 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:38.549 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:38.549 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:38.549 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:38.549 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 61214 2024-09-21T14:59:38.549 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.550 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:38.550 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.550 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:38.550 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147172+0000", 2024-09-21T14:59:38.550 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:38.550 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.550 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.550 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:38.550 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147172+0000", 2024-09-21T14:59:38.550 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:38.550 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.550 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.550 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:38.550 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147168+0000", 2024-09-21T14:59:38.551 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999962 2024-09-21T14:59:38.551 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.551 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.551 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:38.551 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147188+0000", 2024-09-21T14:59:38.551 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9717000000000001e-05 2024-09-21T14:59:38.551 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.551 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.551 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:38.551 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147239+0000", 2024-09-21T14:59:38.551 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1493999999999999e-05 2024-09-21T14:59:38.551 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.551 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.551 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:38.551 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147265+0000", 2024-09-21T14:59:38.552 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5746999999999999e-05 2024-09-21T14:59:38.552 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.552 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.552 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:38.552 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147372+0000", 2024-09-21T14:59:38.552 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010681200000000001 2024-09-21T14:59:38.552 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.552 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.552 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op must wait for map", 2024-09-21T14:59:38.552 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147380+0000", 2024-09-21T14:59:38.552 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7919999999999993e-06 2024-09-21T14:59:38.552 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.552 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.552 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:38.552 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.154025+0000", 2024-09-21T14:59:38.553 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0066454909999999999 2024-09-21T14:59:38.553 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.553 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.553 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T14:59:38.553 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.154041+0000", 2024-09-21T14:59:38.553 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5894000000000001e-05 2024-09-21T14:59:38.553 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.553 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.553 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:38.553 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.164345+0000", 2024-09-21T14:59:38.553 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0103040459999999 2024-09-21T14:59:38.553 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.553 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.554 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:38.554 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.164590+0000", 2024-09-21T14:59:38.554 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00024492699999999998 2024-09-21T14:59:38.554 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.554 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.554 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:38.554 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.166437+0000", 2024-09-21T14:59:38.554 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0018466330000000001 2024-09-21T14:59:38.554 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:38.554 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:38.554 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:38.554 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.554 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.554 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:61215 3.7 3:e28aec10:::rbd_header.1136862f872f:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-21T14:59:38.555 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:18.152008+0000", 2024-09-21T14:59:38.555 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 20.307449748, 2024-09-21T14:59:38.555 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.015644958, 2024-09-21T14:59:38.555 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:38.555 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:38.555 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:38.555 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:38.555 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:38.555 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 61215 2024-09-21T14:59:38.555 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.555 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:38.555 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.555 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:38.555 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152008+0000", 2024-09-21T14:59:38.556 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:38.556 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.556 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.556 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:38.556 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152008+0000", 2024-09-21T14:59:38.556 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:38.556 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.556 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.556 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:38.556 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152005+0000", 2024-09-21T14:59:38.556 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-21T14:59:38.556 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.556 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.556 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:38.557 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152025+0000", 2024-09-21T14:59:38.557 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9306000000000001e-05 2024-09-21T14:59:38.557 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.557 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.557 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:38.557 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152054+0000", 2024-09-21T14:59:38.557 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9189000000000001e-05 2024-09-21T14:59:38.557 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.557 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.557 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:38.557 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152078+0000", 2024-09-21T14:59:38.557 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4317000000000001e-05 2024-09-21T14:59:38.557 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.557 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.558 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:38.558 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152243+0000", 2024-09-21T14:59:38.558 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000164802 2024-09-21T14:59:38.558 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.558 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.558 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting_for_map not empty", 2024-09-21T14:59:38.558 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152251+0000", 2024-09-21T14:59:38.558 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0600000000000008e-06 2024-09-21T14:59:38.558 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.558 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.558 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:38.558 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.154741+0000", 2024-09-21T14:59:38.558 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0024900069999999998 2024-09-21T14:59:38.558 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.559 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.559 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T14:59:38.559 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.154753+0000", 2024-09-21T14:59:38.559 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1588e-05 2024-09-21T14:59:38.559 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.559 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.559 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:38.559 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.165380+0000", 2024-09-21T14:59:38.559 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0106272249999999 2024-09-21T14:59:38.559 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.559 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.559 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:38.559 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.165517+0000", 2024-09-21T14:59:38.560 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013737999999999999 2024-09-21T14:59:38.560 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.560 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.560 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 6", 2024-09-21T14:59:38.560 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.165938+0000", 2024-09-21T14:59:38.560 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00042082399999999998 2024-09-21T14:59:38.560 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.560 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.560 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:38.560 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.166795+0000", 2024-09-21T14:59:38.560 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00085674899999999997 2024-09-21T14:59:38.560 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.560 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.560 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:38.561 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.167598+0000", 2024-09-21T14:59:38.561 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00080335599999999997 2024-09-21T14:59:38.561 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.561 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.561 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:38.561 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.167632+0000", 2024-09-21T14:59:38.561 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4122000000000003e-05 2024-09-21T14:59:38.561 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:38.561 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:38.561 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:38.561 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.167653+0000", 2024-09-21T14:59:38.561 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0862000000000001e-05 2024-09-21T14:59:38.561 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:38.562 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:38.562 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:38.562 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:38.562 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:38.562 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:38.563 INFO:teuthology.orchestra.run.smithi032.stdout:1909 read 0x434c903 thru 0x4356c50 (0xa34e bytes) 2024-09-21T14:59:38.563 INFO:teuthology.orchestra.run.smithi032.stdout:1912 clone 18 order 22 su 524288 sc 10 2024-09-21T14:59:38.564 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T14:59:38.564 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T14:59:38.564 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T14:59:38.565 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T14:59:38.565 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T14:59:38.565 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:59:38.561+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:59:38.582 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-21T14:59:38.608 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T14:59:38.628 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T14:59:38.658 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T14:59:38.660 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:59:38.660 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:59:38.660 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:59:38.660 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:59:38.660 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:59:38.660 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:59:38.660 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:59:38.660 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:59:38.663 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:59:38.661+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:59:38.688 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:38.688 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:38.698 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T14:59:38.765 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:59:38.765+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:59:38.849 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:38.849 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:38.860 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T14:59:38.866 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:59:38.863+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:59:38.965 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:38.965 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:38.966 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:59:38.965+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:59:38.975 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T14:59:39.066 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:59:39.065+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:59:39.087 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:39.087 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:39.097 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T14:59:39.167 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:59:39.165+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:59:39.207 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:39.208 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:39.217 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T14:59:39.267 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:59:39.265+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:59:39.314 INFO:teuthology.orchestra.run.smithi032.stderr:set 3.3 pg_upmap_items mapping to [5->2,1->0] 2024-09-21T14:59:39.368 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:59:39.369+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:59:39.379 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:39.379 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:39.390 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T14:59:39.470 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:59:39.469+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:59:39.506 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:39.506 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:39.515 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T14:59:39.571 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:59:39.569+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:59:39.627 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:39.627 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:39.637 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T14:59:39.672 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:59:39.667+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:59:39.749 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:39.749 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:39.758 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T14:59:39.773 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:59:39.767+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:59:39.868 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:39.868 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:39.873 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:59:39.871+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:59:39.878 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T14:59:39.975 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:59:39.973+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:59:40.033 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:40.033 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:40.043 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T14:59:40.075 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:59:40.071+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:59:40.165 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:40.165 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:40.176 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:59:40.177+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:59:40.181 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T14:59:40.277 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:59:40.271+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:59:40.285 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:40.285 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:40.294 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T14:59:40.332 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T14:59:40.378 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:59:40.377+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:59:40.399 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:40.399 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:40.409 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T14:59:40.479 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:59:40.475+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:59:40.520 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:40.520 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:40.530 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T14:59:40.579 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:59:40.575+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:59:40.672 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T14:59:40.680 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:59:40.675+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:59:40.681 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:40.682 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:40.691 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T14:59:40.781 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:59:40.781+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:59:40.822 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T14:59:40.844 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:40.844 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:40.844 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T14:59:40.853 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T14:59:40.870 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T14:59:40.881 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:59:40.881+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:59:40.890 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T14:59:40.911 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-21T14:59:40.934 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T14:59:40.955 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T14:59:40.963 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:40.963 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:40.973 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T14:59:40.978 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T14:59:40.981 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:59:40.981 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:59:40.981 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:59:40.981 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:59:40.981 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:59:40.981 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:59:40.981 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:59:40.981 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:59:40.982 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:59:40.975+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:59:41.082 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:41.082 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:59:41.082 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T14:59:41.082 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:41.082 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:59:41.081+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:59:41.091 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T14:59:41.132 INFO:teuthology.orchestra.run.smithi032.stderr:noscrub is set 2024-09-21T14:59:41.183 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:59:41.181+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:59:41.205 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:41.205 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:59:41.205 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T14:59:41.205 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T14:59:41.206 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:41.215 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T14:59:41.223 INFO:teuthology.orchestra.run.smithi032.stdout:checking clone #16, image image_client.0-clone16 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent17 2024-09-21T14:59:41.284 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:59:41.285+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:59:41.338 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:41.338 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T14:59:41.338 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T14:59:41.338 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T14:59:41.338 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.339 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:1473 3.15 3:af2afa6a:::rbd_data.1136515ecb6b.000000000000001b:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T14:59:41.339 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:30.421730+0000", 2024-09-21T14:59:41.339 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 70.915944804999995, 2024-09-21T14:59:41.339 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021478937, 2024-09-21T14:59:41.339 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:41.339 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:41.339 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:41.339 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:41.339 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:41.339 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1473 2024-09-21T14:59:41.339 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.339 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:41.339 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.339 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:41.340 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421730+0000", 2024-09-21T14:59:41.340 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:41.340 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.340 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.340 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:41.340 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421730+0000", 2024-09-21T14:59:41.340 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:41.340 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.340 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.340 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:41.340 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421728+0000", 2024-09-21T14:59:41.340 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-21T14:59:41.340 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.340 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.341 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:41.341 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421739+0000", 2024-09-21T14:59:41.341 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0502e-05 2024-09-21T14:59:41.341 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.341 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.341 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:41.341 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421762+0000", 2024-09-21T14:59:41.341 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3405999999999999e-05 2024-09-21T14:59:41.341 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.341 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.341 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:41.341 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421773+0000", 2024-09-21T14:59:41.341 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0744000000000001e-05 2024-09-21T14:59:41.342 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.342 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.342 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:41.342 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421838+0000", 2024-09-21T14:59:41.342 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5486000000000003e-05 2024-09-21T14:59:41.342 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.342 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.342 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:41.342 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421955+0000", 2024-09-21T14:59:41.342 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000116765 2024-09-21T14:59:41.342 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.342 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.342 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 6", 2024-09-21T14:59:41.342 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.425572+0000", 2024-09-21T14:59:41.343 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0036171380000000002 2024-09-21T14:59:41.343 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.343 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.343 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:41.343 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.442973+0000", 2024-09-21T14:59:41.343 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.017400268 2024-09-21T14:59:41.343 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.343 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.343 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:41.343 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.442996+0000", 2024-09-21T14:59:41.343 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3139e-05 2024-09-21T14:59:41.343 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.343 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.344 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:41.344 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.443079+0000", 2024-09-21T14:59:41.344 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2832999999999998e-05 2024-09-21T14:59:41.344 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.344 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.344 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:41.344 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.443209+0000", 2024-09-21T14:59:41.344 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013052499999999999 2024-09-21T14:59:41.344 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:41.344 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:41.344 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:41.344 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.344 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.344 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:1678 3.c 3:377cc7e1:::rbd_data.1136515ecb6b.000000000000000f:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T14:59:41.345 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:30.988085+0000", 2024-09-21T14:59:41.345 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 70.349590211000006, 2024-09-21T14:59:41.345 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.032974802999999997, 2024-09-21T14:59:41.345 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:41.345 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:41.345 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:41.345 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:41.345 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:41.345 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1678 2024-09-21T14:59:41.345 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.345 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:41.345 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.345 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:41.345 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988085+0000", 2024-09-21T14:59:41.346 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:41.346 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.346 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.346 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:41.346 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988085+0000", 2024-09-21T14:59:41.346 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:41.346 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.346 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.346 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:41.346 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988083+0000", 2024-09-21T14:59:41.346 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-21T14:59:41.346 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.346 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.347 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:41.347 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988093+0000", 2024-09-21T14:59:41.347 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8800000000000003e-06 2024-09-21T14:59:41.347 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.347 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.347 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:41.347 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988113+0000", 2024-09-21T14:59:41.347 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0217999999999999e-05 2024-09-21T14:59:41.347 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.347 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.347 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:41.347 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988123+0000", 2024-09-21T14:59:41.347 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5859999999999994e-06 2024-09-21T14:59:41.348 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.348 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.348 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:41.348 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988203+0000", 2024-09-21T14:59:41.348 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0834000000000006e-05 2024-09-21T14:59:41.348 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.348 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.348 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:41.348 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988376+0000", 2024-09-21T14:59:41.348 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017301299999999999 2024-09-21T14:59:41.348 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.348 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.348 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 7", 2024-09-21T14:59:41.348 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.996338+0000", 2024-09-21T14:59:41.349 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0079618520000000002 2024-09-21T14:59:41.349 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.349 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.349 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:41.349 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.020881+0000", 2024-09-21T14:59:41.349 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024542731000000002 2024-09-21T14:59:41.349 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.349 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.349 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:41.349 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.020904+0000", 2024-09-21T14:59:41.349 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2861e-05 2024-09-21T14:59:41.349 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.349 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.349 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:41.349 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.020987+0000", 2024-09-21T14:59:41.350 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2806000000000006e-05 2024-09-21T14:59:41.350 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.350 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.350 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:41.350 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.021060+0000", 2024-09-21T14:59:41.350 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2871000000000001e-05 2024-09-21T14:59:41.350 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:41.350 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:41.350 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:41.350 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.350 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.350 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9370 3.3 3:c725132c:::rbd_data.11363a21d2be.0000000000000007:10 [read 163840~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:41.350 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157279+0000", 2024-09-21T14:59:41.350 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 62.180395587, 2024-09-21T14:59:41.351 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018472379000000001, 2024-09-21T14:59:41.351 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:41.351 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:41.351 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:41.351 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:41.351 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:41.351 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9370 2024-09-21T14:59:41.351 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.351 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:41.351 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.351 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:41.351 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157279+0000", 2024-09-21T14:59:41.351 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:41.351 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.352 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.352 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:41.352 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157279+0000", 2024-09-21T14:59:41.352 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:41.352 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.352 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.352 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:41.352 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157279+0000", 2024-09-21T14:59:41.352 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:41.352 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.352 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.352 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:41.352 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157282+0000", 2024-09-21T14:59:41.352 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2229999999999999e-06 2024-09-21T14:59:41.353 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.353 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.353 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:41.353 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157294+0000", 2024-09-21T14:59:41.353 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2173e-05 2024-09-21T14:59:41.353 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.353 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.353 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:41.353 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157300+0000", 2024-09-21T14:59:41.353 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.93e-06 2024-09-21T14:59:41.353 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.353 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.353 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:41.353 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175569+0000", 2024-09-21T14:59:41.353 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018268376999999999 2024-09-21T14:59:41.354 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.354 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.354 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:41.354 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175622+0000", 2024-09-21T14:59:41.354 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3808999999999999e-05 2024-09-21T14:59:41.354 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.354 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.354 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:41.354 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175752+0000", 2024-09-21T14:59:41.354 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012938700000000001 2024-09-21T14:59:41.354 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:41.354 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:41.354 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:41.355 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.355 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.355 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9371 3.3 3:c0bf6a9b:::rbd_data.11363a21d2be.0000000000000000:10 [read 73728~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:41.355 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157306+0000", 2024-09-21T14:59:41.355 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 62.180369476000003, 2024-09-21T14:59:41.355 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018755473000000002, 2024-09-21T14:59:41.355 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:41.355 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:41.355 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:41.355 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:41.355 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:41.355 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9371 2024-09-21T14:59:41.355 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.355 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:41.356 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.356 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:41.356 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157306+0000", 2024-09-21T14:59:41.356 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:41.356 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.356 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.356 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:41.356 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157306+0000", 2024-09-21T14:59:41.356 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:41.356 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.357 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.357 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:41.357 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157305+0000", 2024-09-21T14:59:41.357 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:41.357 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.357 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.357 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:41.357 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157312+0000", 2024-09-21T14:59:41.357 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3289999999999996e-06 2024-09-21T14:59:41.357 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.357 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.358 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:41.358 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157324+0000", 2024-09-21T14:59:41.358 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1725e-05 2024-09-21T14:59:41.358 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.358 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.358 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:41.358 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157339+0000", 2024-09-21T14:59:41.358 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5211999999999999e-05 2024-09-21T14:59:41.358 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.358 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.358 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:41.358 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175798+0000", 2024-09-21T14:59:41.358 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018458781 2024-09-21T14:59:41.358 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.359 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.359 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:41.359 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175899+0000", 2024-09-21T14:59:41.359 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000100784 2024-09-21T14:59:41.359 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.359 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.359 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:41.359 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176061+0000", 2024-09-21T14:59:41.359 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00016216099999999999 2024-09-21T14:59:41.359 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:41.359 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:41.359 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:41.359 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.359 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.359 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9372 3.3 3:c0bf6a9b:::rbd_data.11363a21d2be.0000000000000000:10 [read 212992~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:41.360 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157345+0000", 2024-09-21T14:59:41.360 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 62.180330130000002, 2024-09-21T14:59:41.360 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018926493999999999, 2024-09-21T14:59:41.360 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:41.360 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:41.360 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:41.360 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:41.360 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:41.360 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9372 2024-09-21T14:59:41.360 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.360 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:41.360 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.360 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:41.360 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157345+0000", 2024-09-21T14:59:41.361 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:41.361 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.361 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.361 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:41.361 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157345+0000", 2024-09-21T14:59:41.361 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:41.361 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.361 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.361 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:41.361 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157344+0000", 2024-09-21T14:59:41.361 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:41.362 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.362 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.362 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:41.362 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157348+0000", 2024-09-21T14:59:41.362 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1389999999999999e-06 2024-09-21T14:59:41.362 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.362 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.362 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:41.362 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157372+0000", 2024-09-21T14:59:41.362 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4711000000000001e-05 2024-09-21T14:59:41.362 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.362 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.362 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:41.362 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157378+0000", 2024-09-21T14:59:41.363 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9909999999999999e-06 2024-09-21T14:59:41.363 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.363 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.363 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:41.363 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176090+0000", 2024-09-21T14:59:41.363 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018712111999999999 2024-09-21T14:59:41.363 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.363 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.363 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:41.363 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176134+0000", 2024-09-21T14:59:41.363 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3377999999999997e-05 2024-09-21T14:59:41.363 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.363 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.363 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:41.363 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176271+0000", 2024-09-21T14:59:41.364 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013763499999999999 2024-09-21T14:59:41.364 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:41.364 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:41.364 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:41.364 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.364 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.364 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9373 3.3 3:c0bf6a9b:::rbd_data.11363a21d2be.0000000000000000:10 [read 352256~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:41.364 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157384+0000", 2024-09-21T14:59:41.364 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 62.180291357999998, 2024-09-21T14:59:41.364 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019154937, 2024-09-21T14:59:41.364 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:41.364 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:41.364 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:41.364 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:41.365 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:41.365 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9373 2024-09-21T14:59:41.365 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.365 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:41.365 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.365 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:41.365 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157384+0000", 2024-09-21T14:59:41.365 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:41.365 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.365 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.365 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:41.365 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157384+0000", 2024-09-21T14:59:41.365 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:41.365 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.366 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.366 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:41.366 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157383+0000", 2024-09-21T14:59:41.366 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:41.366 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.366 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.366 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:41.366 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157386+0000", 2024-09-21T14:59:41.366 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.191e-06 2024-09-21T14:59:41.366 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.366 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.366 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:41.366 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157393+0000", 2024-09-21T14:59:41.366 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5339999999999996e-06 2024-09-21T14:59:41.366 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.367 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.367 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:41.367 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157397+0000", 2024-09-21T14:59:41.367 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5220000000000001e-06 2024-09-21T14:59:41.367 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.367 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.367 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:41.367 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176302+0000", 2024-09-21T14:59:41.367 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018905113000000001 2024-09-21T14:59:41.367 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.367 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.367 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:41.367 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176379+0000", 2024-09-21T14:59:41.368 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6372999999999995e-05 2024-09-21T14:59:41.368 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.368 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.368 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:41.368 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176539+0000", 2024-09-21T14:59:41.368 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015973800000000001 2024-09-21T14:59:41.368 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:41.368 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:41.368 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:41.368 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.368 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.368 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9374 3.3 3:c0bf6a9b:::rbd_data.11363a21d2be.0000000000000000:10 [read 491520~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:41.368 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157401+0000", 2024-09-21T14:59:41.368 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 62.180273573999997, 2024-09-21T14:59:41.368 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019387755, 2024-09-21T14:59:41.369 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:41.369 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:41.369 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:41.369 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:41.369 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:41.369 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9374 2024-09-21T14:59:41.369 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.369 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:41.369 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.369 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:41.369 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157401+0000", 2024-09-21T14:59:41.369 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:41.369 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.369 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.369 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:41.370 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157401+0000", 2024-09-21T14:59:41.370 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:41.370 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.370 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.370 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:41.370 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157401+0000", 2024-09-21T14:59:41.370 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:41.370 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.370 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.370 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:41.370 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157414+0000", 2024-09-21T14:59:41.370 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2417e-05 2024-09-21T14:59:41.370 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.370 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.371 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:41.371 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157424+0000", 2024-09-21T14:59:41.371 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0861e-05 2024-09-21T14:59:41.371 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.371 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.371 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:41.371 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157429+0000", 2024-09-21T14:59:41.371 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0499999999999999e-06 2024-09-21T14:59:41.371 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.371 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.371 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:41.371 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176596+0000", 2024-09-21T14:59:41.371 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019166552 2024-09-21T14:59:41.371 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.371 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.372 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:41.372 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176647+0000", 2024-09-21T14:59:41.372 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1112000000000001e-05 2024-09-21T14:59:41.372 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.372 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.372 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:41.372 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176789+0000", 2024-09-21T14:59:41.372 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014209300000000001 2024-09-21T14:59:41.372 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:41.372 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:41.372 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:41.372 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.372 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.372 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9401 3.3 3:c725132c:::rbd_data.11363a21d2be.0000000000000007:10 [read 32768~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:41.372 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157621+0000", 2024-09-21T14:59:41.372 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 62.180053764999997, 2024-09-21T14:59:41.373 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019381711999999999, 2024-09-21T14:59:41.373 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:41.373 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:41.373 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:41.373 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:41.373 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:41.373 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9401 2024-09-21T14:59:41.373 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.373 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:41.373 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.373 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:41.373 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157621+0000", 2024-09-21T14:59:41.373 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:41.373 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.374 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.374 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:41.374 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157621+0000", 2024-09-21T14:59:41.374 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:41.374 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.374 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.374 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:41.374 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157620+0000", 2024-09-21T14:59:41.374 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:59:41.374 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.374 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.374 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:41.374 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157626+0000", 2024-09-21T14:59:41.374 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7570000000000001e-06 2024-09-21T14:59:41.375 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.375 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.375 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:41.375 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157636+0000", 2024-09-21T14:59:41.375 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8339999999999994e-06 2024-09-21T14:59:41.375 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.375 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.375 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:41.375 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157643+0000", 2024-09-21T14:59:41.375 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9439999999999999e-06 2024-09-21T14:59:41.375 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.375 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.375 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:41.375 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176816+0000", 2024-09-21T14:59:41.375 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019172872000000001 2024-09-21T14:59:41.375 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.376 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.376 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:41.376 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176878+0000", 2024-09-21T14:59:41.376 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1734000000000002e-05 2024-09-21T14:59:41.376 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.376 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.376 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:41.376 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177003+0000", 2024-09-21T14:59:41.376 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012537400000000001 2024-09-21T14:59:41.376 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:41.376 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:41.376 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:41.376 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.376 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.376 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9402 3.3 3:c725132c:::rbd_data.11363a21d2be.0000000000000007:10 [read 172032~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:41.377 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157649+0000", 2024-09-21T14:59:41.377 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 62.180025893, 2024-09-21T14:59:41.377 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019567404, 2024-09-21T14:59:41.377 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:41.377 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:41.377 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:41.377 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:41.377 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:41.377 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9402 2024-09-21T14:59:41.377 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.377 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:41.377 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.377 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:41.377 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157649+0000", 2024-09-21T14:59:41.377 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:41.378 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.378 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.378 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:41.378 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157649+0000", 2024-09-21T14:59:41.378 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:41.378 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.378 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.378 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:41.378 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157649+0000", 2024-09-21T14:59:41.378 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:41.378 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.378 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.378 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:41.378 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157652+0000", 2024-09-21T14:59:41.378 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8779999999999998e-06 2024-09-21T14:59:41.379 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.379 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.379 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:41.379 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157657+0000", 2024-09-21T14:59:41.379 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6659999999999996e-06 2024-09-21T14:59:41.379 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.379 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.379 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:41.379 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157662+0000", 2024-09-21T14:59:41.379 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5519999999999998e-06 2024-09-21T14:59:41.379 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.379 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.379 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:41.379 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177040+0000", 2024-09-21T14:59:41.379 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019377807 2024-09-21T14:59:41.380 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.380 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.380 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:41.380 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177085+0000", 2024-09-21T14:59:41.380 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5195999999999999e-05 2024-09-21T14:59:41.380 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.380 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.380 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:41.380 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177217+0000", 2024-09-21T14:59:41.380 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013172600000000001 2024-09-21T14:59:41.380 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:41.380 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:41.380 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:41.380 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.380 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.380 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9403 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:41.381 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157949+0000", 2024-09-21T14:59:41.381 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 62.179725564000002, 2024-09-21T14:59:41.381 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023540373999999999, 2024-09-21T14:59:41.381 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:41.381 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:41.381 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:41.381 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:41.381 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:41.381 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9403 2024-09-21T14:59:41.381 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.381 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:41.381 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.381 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:41.381 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157949+0000", 2024-09-21T14:59:41.381 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:41.382 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.382 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.382 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:41.382 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157949+0000", 2024-09-21T14:59:41.382 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:41.382 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.382 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.382 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:41.382 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157948+0000", 2024-09-21T14:59:41.382 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T14:59:41.382 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.382 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.382 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:41.383 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157956+0000", 2024-09-21T14:59:41.383 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7999999999999999e-06 2024-09-21T14:59:41.383 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.383 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.383 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:41.383 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157969+0000", 2024-09-21T14:59:41.383 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2792000000000001e-05 2024-09-21T14:59:41.383 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.383 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.383 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:41.383 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157977+0000", 2024-09-21T14:59:41.383 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.704e-06 2024-09-21T14:59:41.383 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.383 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.384 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:41.384 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177246+0000", 2024-09-21T14:59:41.384 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019268264 2024-09-21T14:59:41.384 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.384 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.384 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:41.384 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177315+0000", 2024-09-21T14:59:41.384 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9480999999999994e-05 2024-09-21T14:59:41.384 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.384 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.384 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:59:41.384 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177615+0000", 2024-09-21T14:59:41.384 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00029929499999999999 2024-09-21T14:59:41.384 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.385 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.385 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:41.385 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181333+0000", 2024-09-21T14:59:41.385 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0037183149999999998 2024-09-21T14:59:41.385 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.385 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.385 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:41.385 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181374+0000", 2024-09-21T14:59:41.385 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.0723999999999998e-05 2024-09-21T14:59:41.385 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.385 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.385 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:41.385 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181434+0000", 2024-09-21T14:59:41.385 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0024e-05 2024-09-21T14:59:41.386 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.386 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.386 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:41.386 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181490+0000", 2024-09-21T14:59:41.386 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6245000000000001e-05 2024-09-21T14:59:41.386 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:41.386 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:41.386 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:41.386 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.386 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.386 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9404 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:41.386 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.160480+0000", 2024-09-21T14:59:41.386 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 62.177195214000001, 2024-09-21T14:59:41.386 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021137686999999999, 2024-09-21T14:59:41.387 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:41.387 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:41.387 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:41.387 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:41.387 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:41.387 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9404 2024-09-21T14:59:41.387 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.387 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:41.387 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.387 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:41.387 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160480+0000", 2024-09-21T14:59:41.387 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:41.387 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.387 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.388 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:41.388 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160480+0000", 2024-09-21T14:59:41.388 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:41.388 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.388 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.388 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:41.388 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160478+0000", 2024-09-21T14:59:41.388 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T14:59:41.388 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.388 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.388 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:41.388 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160486+0000", 2024-09-21T14:59:41.388 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9969999999999998e-06 2024-09-21T14:59:41.388 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.389 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.389 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:41.389 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160502+0000", 2024-09-21T14:59:41.389 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5650000000000001e-05 2024-09-21T14:59:41.389 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.389 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.389 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:41.389 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160511+0000", 2024-09-21T14:59:41.389 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.3610000000000007e-06 2024-09-21T14:59:41.389 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.389 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.389 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:41.389 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.178035+0000", 2024-09-21T14:59:41.389 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.017523520000000001 2024-09-21T14:59:41.390 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.390 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.390 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:41.390 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.178151+0000", 2024-09-21T14:59:41.390 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000116013 2024-09-21T14:59:41.390 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.390 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.390 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:59:41.390 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.178408+0000", 2024-09-21T14:59:41.390 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00025743400000000002 2024-09-21T14:59:41.390 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.390 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.390 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:41.391 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181501+0000", 2024-09-21T14:59:41.391 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.003092891 2024-09-21T14:59:41.391 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.391 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.391 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:41.391 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181538+0000", 2024-09-21T14:59:41.391 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6977999999999997e-05 2024-09-21T14:59:41.391 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.391 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.391 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:41.391 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181582+0000", 2024-09-21T14:59:41.391 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3849e-05 2024-09-21T14:59:41.391 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.391 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.391 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:41.392 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181617+0000", 2024-09-21T14:59:41.392 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5423000000000003e-05 2024-09-21T14:59:41.392 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:41.392 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:41.392 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:41.392 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.392 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.392 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9407 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:41.392 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.168094+0000", 2024-09-21T14:59:41.392 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 62.169580715000002, 2024-09-21T14:59:41.392 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019444860000000001, 2024-09-21T14:59:41.392 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:41.392 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:41.392 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:41.393 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:41.393 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:41.393 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9407 2024-09-21T14:59:41.393 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.393 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:41.393 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.393 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:41.393 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.168094+0000", 2024-09-21T14:59:41.393 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:41.393 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.393 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.393 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:41.393 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.168094+0000", 2024-09-21T14:59:41.393 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:41.393 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.394 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.394 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:41.394 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.168093+0000", 2024-09-21T14:59:41.394 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:59:41.394 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.394 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.394 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:41.394 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.168119+0000", 2024-09-21T14:59:41.394 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5803999999999999e-05 2024-09-21T14:59:41.394 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.394 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.394 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:41.394 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.168135+0000", 2024-09-21T14:59:41.395 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6359999999999999e-05 2024-09-21T14:59:41.395 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.395 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.395 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:41.395 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.168144+0000", 2024-09-21T14:59:41.395 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4270000000000008e-06 2024-09-21T14:59:41.395 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.395 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.395 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:41.395 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181943+0000", 2024-09-21T14:59:41.395 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013799617 2024-09-21T14:59:41.395 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.395 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.396 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:41.396 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.182052+0000", 2024-09-21T14:59:41.396 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000108924 2024-09-21T14:59:41.396 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.396 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.396 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:59:41.396 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.183431+0000", 2024-09-21T14:59:41.396 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0013783619999999999 2024-09-21T14:59:41.396 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.396 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.396 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:41.396 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.187276+0000", 2024-09-21T14:59:41.396 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0038450469999999999 2024-09-21T14:59:41.397 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.397 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.397 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:41.397 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.187288+0000", 2024-09-21T14:59:41.397 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2414e-05 2024-09-21T14:59:41.397 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.397 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.397 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:41.397 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.187414+0000", 2024-09-21T14:59:41.397 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012614399999999999 2024-09-21T14:59:41.397 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.397 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.397 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:41.397 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.187539+0000", 2024-09-21T14:59:41.398 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012481800000000001 2024-09-21T14:59:41.398 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:41.398 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:41.398 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:41.398 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.398 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.398 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9410 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:41.398 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.174103+0000", 2024-09-21T14:59:41.398 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 62.163571806, 2024-09-21T14:59:41.398 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025493703, 2024-09-21T14:59:41.398 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:41.398 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:41.399 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:41.399 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:41.399 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:41.399 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9410 2024-09-21T14:59:41.399 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.399 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:41.399 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.399 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:41.399 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.174103+0000", 2024-09-21T14:59:41.399 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:41.399 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.399 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.399 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:41.400 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.174103+0000", 2024-09-21T14:59:41.400 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:41.400 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.400 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.400 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:41.400 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.174102+0000", 2024-09-21T14:59:41.400 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:59:41.400 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.400 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.400 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:41.400 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.174108+0000", 2024-09-21T14:59:41.400 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1149999999999999e-06 2024-09-21T14:59:41.400 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.400 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.400 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:41.401 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.174117+0000", 2024-09-21T14:59:41.401 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4870000000000002e-06 2024-09-21T14:59:41.401 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.401 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.401 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:41.401 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.174122+0000", 2024-09-21T14:59:41.401 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9790000000000001e-06 2024-09-21T14:59:41.401 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.401 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.401 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:41.401 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.187830+0000", 2024-09-21T14:59:41.401 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013708048 2024-09-21T14:59:41.401 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.401 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.401 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:41.402 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.187913+0000", 2024-09-21T14:59:41.402 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2763999999999996e-05 2024-09-21T14:59:41.402 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.402 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.402 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:59:41.402 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.188230+0000", 2024-09-21T14:59:41.402 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00031745100000000001 2024-09-21T14:59:41.402 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.402 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.402 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:41.402 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.192708+0000", 2024-09-21T14:59:41.402 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0044783890000000002 2024-09-21T14:59:41.402 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.402 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.403 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:41.403 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199487+0000", 2024-09-21T14:59:41.403 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0067782149999999998 2024-09-21T14:59:41.403 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.403 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.403 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:41.403 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199560+0000", 2024-09-21T14:59:41.403 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3752999999999997e-05 2024-09-21T14:59:41.403 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.403 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.403 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:41.403 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199597+0000", 2024-09-21T14:59:41.403 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6526999999999998e-05 2024-09-21T14:59:41.403 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:41.404 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:41.404 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:41.404 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.404 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.404 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9411 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:41.404 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.175943+0000", 2024-09-21T14:59:41.404 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 62.161731787000001, 2024-09-21T14:59:41.404 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023773181000000001, 2024-09-21T14:59:41.404 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:41.404 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:41.404 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:41.404 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:41.404 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:41.404 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9411 2024-09-21T14:59:41.405 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.405 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:41.405 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.405 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:41.405 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175943+0000", 2024-09-21T14:59:41.405 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:41.405 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.405 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.405 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:41.405 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175943+0000", 2024-09-21T14:59:41.405 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:41.405 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.405 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.405 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:41.405 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175942+0000", 2024-09-21T14:59:41.405 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:59:41.406 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.406 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.406 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:41.406 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175948+0000", 2024-09-21T14:59:41.406 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9689999999999999e-06 2024-09-21T14:59:41.406 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.406 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.406 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:41.406 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175960+0000", 2024-09-21T14:59:41.406 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1745e-05 2024-09-21T14:59:41.406 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.406 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.406 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:41.406 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175965+0000", 2024-09-21T14:59:41.407 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.203e-06 2024-09-21T14:59:41.407 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.407 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.407 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:41.407 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.189900+0000", 2024-09-21T14:59:41.407 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013934940999999999 2024-09-21T14:59:41.407 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.407 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.407 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:41.407 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.189988+0000", 2024-09-21T14:59:41.407 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8419999999999997e-05 2024-09-21T14:59:41.407 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.407 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.407 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:59:41.407 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.190321+0000", 2024-09-21T14:59:41.407 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00033231800000000001 2024-09-21T14:59:41.408 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.408 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.408 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:41.408 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.192746+0000", 2024-09-21T14:59:41.408 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0024254400000000001 2024-09-21T14:59:41.408 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.408 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.408 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:41.408 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199605+0000", 2024-09-21T14:59:41.408 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0068582809999999999 2024-09-21T14:59:41.408 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.408 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.408 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:41.408 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199680+0000", 2024-09-21T14:59:41.408 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5630000000000006e-05 2024-09-21T14:59:41.408 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.409 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.409 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:41.409 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199716+0000", 2024-09-21T14:59:41.409 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6248999999999998e-05 2024-09-21T14:59:41.409 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:41.409 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:41.409 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:41.409 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.409 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.409 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9412 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:41.409 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.177402+0000", 2024-09-21T14:59:41.409 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 62.160272689999999, 2024-09-21T14:59:41.409 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022445590000000001, 2024-09-21T14:59:41.409 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:41.409 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:41.409 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:41.410 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:41.410 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:41.410 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9412 2024-09-21T14:59:41.410 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.410 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:41.410 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.410 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:41.410 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177402+0000", 2024-09-21T14:59:41.410 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:41.410 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.410 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.410 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:41.410 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177402+0000", 2024-09-21T14:59:41.410 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:41.410 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.410 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.411 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:41.411 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177401+0000", 2024-09-21T14:59:41.411 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:59:41.411 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.411 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.411 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:41.411 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177409+0000", 2024-09-21T14:59:41.411 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4710000000000001e-06 2024-09-21T14:59:41.411 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.411 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.411 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:41.411 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177419+0000", 2024-09-21T14:59:41.411 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0351e-05 2024-09-21T14:59:41.411 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.411 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.412 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:41.412 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177424+0000", 2024-09-21T14:59:41.412 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1919999999999996e-06 2024-09-21T14:59:41.412 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.412 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.412 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:41.412 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.191343+0000", 2024-09-21T14:59:41.412 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.01391878 2024-09-21T14:59:41.412 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.412 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.412 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:41.412 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.191446+0000", 2024-09-21T14:59:41.412 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000102453 2024-09-21T14:59:41.412 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.412 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.412 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:59:41.413 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.192200+0000", 2024-09-21T14:59:41.413 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00075476899999999997 2024-09-21T14:59:41.413 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.413 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.413 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:41.413 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.193866+0000", 2024-09-21T14:59:41.413 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0016657519999999999 2024-09-21T14:59:41.413 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.413 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.413 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:41.413 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199724+0000", 2024-09-21T14:59:41.413 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0058579529999999999 2024-09-21T14:59:41.413 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.413 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.413 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:41.413 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199803+0000", 2024-09-21T14:59:41.414 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8662e-05 2024-09-21T14:59:41.414 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.414 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.414 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:41.414 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199848+0000", 2024-09-21T14:59:41.414 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5216000000000002e-05 2024-09-21T14:59:41.414 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:41.414 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:41.414 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:41.414 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.414 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.414 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:10445 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:41.414 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.271980+0000", 2024-09-21T14:59:41.414 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 62.065695325, 2024-09-21T14:59:41.415 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018664256000000001, 2024-09-21T14:59:41.415 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:41.415 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:41.415 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:41.415 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:41.415 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:41.415 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 10445 2024-09-21T14:59:41.415 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.415 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:41.415 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.415 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:41.415 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.271980+0000", 2024-09-21T14:59:41.415 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:41.415 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.415 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.415 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:41.416 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.271980+0000", 2024-09-21T14:59:41.416 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:41.416 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.416 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.416 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:41.416 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.271979+0000", 2024-09-21T14:59:41.416 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:41.416 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.416 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.416 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:41.416 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.271982+0000", 2024-09-21T14:59:41.416 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7240000000000001e-06 2024-09-21T14:59:41.416 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.416 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.416 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:41.416 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.271987+0000", 2024-09-21T14:59:41.417 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3639999999999998e-06 2024-09-21T14:59:41.417 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.417 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.417 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:41.417 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.271992+0000", 2024-09-21T14:59:41.417 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7110000000000003e-06 2024-09-21T14:59:41.417 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.417 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.417 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:41.417 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.277567+0000", 2024-09-21T14:59:41.417 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0055745009999999999 2024-09-21T14:59:41.417 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.417 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.417 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:41.417 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.277655+0000", 2024-09-21T14:59:41.417 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8465000000000001e-05 2024-09-21T14:59:41.418 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.418 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.418 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:59:41.418 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.278720+0000", 2024-09-21T14:59:41.418 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0010650569999999999 2024-09-21T14:59:41.418 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.418 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.418 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:41.418 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.281153+0000", 2024-09-21T14:59:41.418 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0024326389999999999 2024-09-21T14:59:41.418 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.418 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.418 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:41.418 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.290574+0000", 2024-09-21T14:59:41.418 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0094209489999999996 2024-09-21T14:59:41.418 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.419 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.419 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:41.419 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.290613+0000", 2024-09-21T14:59:41.419 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.8803000000000001e-05 2024-09-21T14:59:41.419 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.419 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.419 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:41.419 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.290644+0000", 2024-09-21T14:59:41.419 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1404e-05 2024-09-21T14:59:41.419 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:41.419 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:41.419 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:41.419 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.419 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.419 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:23084 3.3 3:c55e343c:::rbd_data.11366acc894.0000000000000006:19 [read 1032192~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:41.419 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.150724+0000", 2024-09-21T14:59:41.420 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 51.186950750000001, 2024-09-21T14:59:41.420 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018916460999999999, 2024-09-21T14:59:41.420 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:41.420 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:41.420 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:41.420 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:41.420 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:41.420 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 23084 2024-09-21T14:59:41.420 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.420 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:41.420 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.420 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:41.420 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.150724+0000", 2024-09-21T14:59:41.420 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:41.420 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.420 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.421 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:41.421 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.150724+0000", 2024-09-21T14:59:41.421 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:41.421 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.421 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.421 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:41.421 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.150724+0000", 2024-09-21T14:59:41.421 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:41.421 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.421 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.421 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:41.421 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.150727+0000", 2024-09-21T14:59:41.421 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7250000000000002e-06 2024-09-21T14:59:41.421 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.421 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.421 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:41.422 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.150732+0000", 2024-09-21T14:59:41.422 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5809999999999996e-06 2024-09-21T14:59:41.422 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.422 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.422 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:41.422 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.150737+0000", 2024-09-21T14:59:41.422 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3590000000000001e-06 2024-09-21T14:59:41.422 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.422 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.422 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:41.422 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.166694+0000", 2024-09-21T14:59:41.422 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.015957720000000002 2024-09-21T14:59:41.422 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.422 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.422 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:41.422 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.166769+0000", 2024-09-21T14:59:41.423 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5143999999999997e-05 2024-09-21T14:59:41.423 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.423 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.423 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:41.423 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.169641+0000", 2024-09-21T14:59:41.423 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0028713089999999998 2024-09-21T14:59:41.423 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:41.423 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:41.423 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:41.423 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.423 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.423 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:71939 3.e 3:75b0de84:::rbd_object_map.11368900df38:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100)", 2024-09-21T14:59:41.423 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:38.376206+0000", 2024-09-21T14:59:41.424 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 2.9614695210000002, 2024-09-21T14:59:41.424 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019353591999999999, 2024-09-21T14:59:41.424 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:41.424 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:41.424 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:41.424 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:41.424 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:41.424 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 71939 2024-09-21T14:59:41.424 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.424 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:41.424 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.424 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:41.424 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.376206+0000", 2024-09-21T14:59:41.424 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:41.424 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.424 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.425 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:41.425 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.376206+0000", 2024-09-21T14:59:41.425 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:41.425 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.425 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.425 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:41.425 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.376205+0000", 2024-09-21T14:59:41.425 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:41.425 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.425 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.425 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:41.425 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.376209+0000", 2024-09-21T14:59:41.425 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.8820000000000003e-06 2024-09-21T14:59:41.425 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.425 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.425 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:41.426 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.376215+0000", 2024-09-21T14:59:41.426 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5729999999999998e-06 2024-09-21T14:59:41.426 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.426 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.426 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:41.426 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.376221+0000", 2024-09-21T14:59:41.426 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2040000000000002e-06 2024-09-21T14:59:41.426 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.426 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.426 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:41.426 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.383959+0000", 2024-09-21T14:59:41.426 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0077380089999999997 2024-09-21T14:59:41.426 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.426 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.426 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:41.426 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.384131+0000", 2024-09-21T14:59:41.427 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017252899999999999 2024-09-21T14:59:41.427 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.427 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.427 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:59:41.427 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.384666+0000", 2024-09-21T14:59:41.427 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00053445399999999996 2024-09-21T14:59:41.427 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.427 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.427 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:41.427 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.388174+0000", 2024-09-21T14:59:41.427 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.003508639 2024-09-21T14:59:41.427 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.427 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.427 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:41.427 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.395460+0000", 2024-09-21T14:59:41.428 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.007285489 2024-09-21T14:59:41.428 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.428 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.428 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:41.428 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.395503+0000", 2024-09-21T14:59:41.428 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3080000000000001e-05 2024-09-21T14:59:41.428 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.428 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.428 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:41.429 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.395559+0000", 2024-09-21T14:59:41.429 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6118e-05 2024-09-21T14:59:41.429 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:41.429 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:41.429 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:41.429 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.430 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.430 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:71940 3.e 3:75b0de84:::rbd_object_map.11368900df38:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100)", 2024-09-21T14:59:41.430 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:38.376225+0000", 2024-09-21T14:59:41.430 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 2.961450283, 2024-09-21T14:59:41.430 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019578849999999998, 2024-09-21T14:59:41.430 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:41.430 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:41.430 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:41.430 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:41.430 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:41.430 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 71940 2024-09-21T14:59:41.430 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.430 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:41.430 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.431 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:41.431 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.376225+0000", 2024-09-21T14:59:41.431 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:41.431 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.431 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.431 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:41.431 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.376225+0000", 2024-09-21T14:59:41.431 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:41.431 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.431 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.431 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:41.431 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.376224+0000", 2024-09-21T14:59:41.431 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:41.431 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.431 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.432 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:41.432 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.376228+0000", 2024-09-21T14:59:41.432 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5889999999999999e-06 2024-09-21T14:59:41.432 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.432 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.432 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:41.432 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.376233+0000", 2024-09-21T14:59:41.432 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.322e-06 2024-09-21T14:59:41.432 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.432 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.432 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:41.432 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.376238+0000", 2024-09-21T14:59:41.432 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7960000000000003e-06 2024-09-21T14:59:41.432 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.433 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.433 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:41.433 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.385506+0000", 2024-09-21T14:59:41.433 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0092679279999999999 2024-09-21T14:59:41.433 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.433 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.433 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:41.433 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.385598+0000", 2024-09-21T14:59:41.433 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1939000000000002e-05 2024-09-21T14:59:41.433 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.433 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.433 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:59:41.433 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.386013+0000", 2024-09-21T14:59:41.434 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000414793 2024-09-21T14:59:41.434 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.434 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.434 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:41.434 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.395634+0000", 2024-09-21T14:59:41.434 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0096215720000000001 2024-09-21T14:59:41.434 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.434 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.434 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:41.434 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.395754+0000", 2024-09-21T14:59:41.434 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000120097 2024-09-21T14:59:41.434 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.434 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.435 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:41.435 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.395782+0000", 2024-09-21T14:59:41.435 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7583e-05 2024-09-21T14:59:41.435 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.435 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.435 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:41.435 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.395804+0000", 2024-09-21T14:59:41.435 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1594999999999999e-05 2024-09-21T14:59:41.435 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:41.435 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:41.435 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:41.435 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.435 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.435 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:71941 3.e 3:75b0de84:::rbd_object_map.11368900df38:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100)", 2024-09-21T14:59:41.436 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:38.376242+0000", 2024-09-21T14:59:41.436 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 2.961433231, 2024-09-21T14:59:41.436 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019640456000000001, 2024-09-21T14:59:41.436 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:41.436 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:41.436 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:41.436 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:41.436 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:41.436 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 71941 2024-09-21T14:59:41.436 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.436 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:41.436 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.436 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:41.436 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.376242+0000", 2024-09-21T14:59:41.436 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:41.437 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.437 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.437 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:41.437 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.376242+0000", 2024-09-21T14:59:41.437 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:41.437 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.437 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.437 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:41.437 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.376241+0000", 2024-09-21T14:59:41.437 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:41.437 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.437 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.437 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:41.437 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.376244+0000", 2024-09-21T14:59:41.437 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0340000000000001e-06 2024-09-21T14:59:41.438 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.438 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.438 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:41.438 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.376250+0000", 2024-09-21T14:59:41.438 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3499999999999996e-06 2024-09-21T14:59:41.438 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.438 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.438 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:41.438 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.376255+0000", 2024-09-21T14:59:41.438 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1320000000000002e-06 2024-09-21T14:59:41.438 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.438 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.438 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:41.438 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.386791+0000", 2024-09-21T14:59:41.438 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.010536085000000001 2024-09-21T14:59:41.439 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.439 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.439 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:41.439 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.386868+0000", 2024-09-21T14:59:41.439 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6674999999999993e-05 2024-09-21T14:59:41.439 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.439 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.439 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:59:41.439 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.387258+0000", 2024-09-21T14:59:41.439 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00038990999999999999 2024-09-21T14:59:41.439 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.439 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.439 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:41.439 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.395685+0000", 2024-09-21T14:59:41.439 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0084273590000000006 2024-09-21T14:59:41.439 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.440 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.440 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:41.440 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.395812+0000", 2024-09-21T14:59:41.440 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012719700000000001 2024-09-21T14:59:41.440 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.440 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.440 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:41.440 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.395849+0000", 2024-09-21T14:59:41.440 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6748999999999997e-05 2024-09-21T14:59:41.440 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.440 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.440 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:41.440 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.395882+0000", 2024-09-21T14:59:41.440 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3318999999999997e-05 2024-09-21T14:59:41.440 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:41.440 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:41.441 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:41.441 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:41.441 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:41.441 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:41.442 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T14:59:41.443 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:59:41.439+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:59:41.543 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:59:41.539+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:59:41.597 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:41.597 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:59:41.597 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T14:59:41.597 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:41.607 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T14:59:41.616 INFO:teuthology.orchestra.run.smithi032.stdout:1914 punch from 0x1460de1 to 0x146b601, (0xa820 bytes) 2024-09-21T14:59:41.625 INFO:teuthology.orchestra.run.smithi032.stdout:1915 read 0x3a78486 thru 0x3a853f4 (0xcf6f bytes) 2024-09-21T14:59:41.625 INFO:teuthology.orchestra.run.smithi032.stdout:1916 read 0x2864e0b thru 0x2871964 (0xcb5a bytes) 2024-09-21T14:59:41.625 INFO:teuthology.orchestra.run.smithi032.stdout:1919 writesame 0x21a37bd thru 0x21b1d36 data_size 0x3(0xe57a bytes) 2024-09-21T14:59:41.642 INFO:teuthology.orchestra.run.smithi032.stdout:1920 trunc from 0x49ece53 to 0x64d9c5f 2024-09-21T14:59:41.644 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:59:41.645+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:59:41.653 INFO:teuthology.orchestra.run.smithi032.stdout:1921 read 0x410eda9 thru 0x4111428 (0x2680 bytes) 2024-09-21T14:59:41.654 INFO:teuthology.orchestra.run.smithi032.stdout:1923 writesame 0x5213073 thru 0x5217310 data_size 0x1(0x429e bytes) 2024-09-21T14:59:41.671 INFO:teuthology.orchestra.run.smithi032.stdout:1925 read 0x3fcb545 thru 0x3fd2598 (0x7054 bytes) 2024-09-21T14:59:41.671 INFO:teuthology.orchestra.run.smithi032.stdout:1926 read 0x139d7de thru 0x13a85ce (0xadf1 bytes) 2024-09-21T14:59:41.672 INFO:teuthology.orchestra.run.smithi032.stdout:1927 read 0x5224d98 thru 0x5232b04 (0xdd6d bytes) 2024-09-21T14:59:41.672 INFO:teuthology.orchestra.run.smithi032.stdout:1929 read 0x7729a4 thru 0x7734ab (0xb08 bytes) 2024-09-21T14:59:41.686 INFO:teuthology.orchestra.run.smithi032.stdout:1930 trunc from 0x64d9c5f to 0x5eec9f 2024-09-21T14:59:41.709 INFO:teuthology.orchestra.run.smithi032.stdout:1931 read 0x44b6d6 thru 0x44b82e (0x159 bytes) 2024-09-21T14:59:41.718 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:41.718 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:59:41.718 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T14:59:41.718 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T14:59:41.718 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:41.728 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T14:59:41.728 INFO:teuthology.orchestra.run.smithi032.stdout:1932 write 0x47f1798 thru 0x47f71ab (0x5a14 bytes) 2024-09-21T14:59:41.734 INFO:teuthology.orchestra.run.smithi032.stdout:1934 read 0x46d786c thru 0x46e1a0e (0xa1a3 bytes) 2024-09-21T14:59:41.742 INFO:teuthology.orchestra.run.smithi032.stdout:1935 trunc from 0x47f71ac to 0xe78f9b 2024-09-21T14:59:41.745 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:59:41.745+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:59:41.765 INFO:teuthology.orchestra.run.smithi032.stdout:1936 punch from 0x54af44 to 0x557f5b, (0xd017 bytes) 2024-09-21T14:59:41.766 INFO:teuthology.orchestra.run.smithi032.stdout:1937 trunc from 0xe78f9b to 0x120e00a 2024-09-21T14:59:41.776 INFO:teuthology.orchestra.run.smithi032.stdout:1939 write 0xccdbb1 thru 0xcd40fb (0x654b bytes) 2024-09-21T14:59:41.779 INFO:teuthology.orchestra.run.smithi032.stdout:1940 punch from 0x578fe3 to 0x5793a3, (0x3c0 bytes) 2024-09-21T14:59:41.781 INFO:teuthology.orchestra.run.smithi032.stdout:1941 trunc from 0x120e00a to 0x246bcb 2024-09-21T14:59:41.837 INFO:teuthology.orchestra.run.smithi032.stdout:1942 writesame 0xaa792d5 thru 0xaa832f3 data_size 0x1(0xa01f bytes) 2024-09-21T14:59:41.838 INFO:teuthology.orchestra.run.smithi032.stdout:1944 writesame 0xe63f0c thru 0xe6bcc4 data_size 0x29(0x7db9 bytes) 2024-09-21T14:59:41.845 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:41.845 INFO:teuthology.orchestra.run.smithi032.stdout:1947 writesame 0x4062ecd thru 0x406eb36 data_size 0x3(0xbc6a bytes) 2024-09-21T14:59:41.845 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:59:41.845+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:59:41.859 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:41.860 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T14:59:41.860 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T14:59:41.860 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T14:59:41.860 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.860 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T14:59:41.860 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.205754+0000", 2024-09-21T14:59:41.860 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 98.652979791000007, 2024-09-21T14:59:41.860 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.41117094199999998, 2024-09-21T14:59:41.860 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:41.860 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:41.860 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:41.860 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:59:41.860 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:59:41.860 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1 2024-09-21T14:59:41.860 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.860 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:41.861 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.861 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:41.861 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205754+0000", 2024-09-21T14:59:41.861 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:41.861 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.861 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.861 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:41.861 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205754+0000", 2024-09-21T14:59:41.861 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:41.861 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.861 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.861 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:41.861 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205752+0000", 2024-09-21T14:59:41.861 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T14:59:41.862 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.862 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.862 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:41.862 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205765+0000", 2024-09-21T14:59:41.862 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3230999999999999e-05 2024-09-21T14:59:41.862 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.862 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.862 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:41.862 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205782+0000", 2024-09-21T14:59:41.862 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6634e-05 2024-09-21T14:59:41.862 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.862 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.862 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:41.862 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205793+0000", 2024-09-21T14:59:41.862 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1297e-05 2024-09-21T14:59:41.863 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.863 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.863 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:41.863 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205867+0000", 2024-09-21T14:59:41.863 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4269000000000003e-05 2024-09-21T14:59:41.863 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.863 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.863 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T14:59:41.863 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205877+0000", 2024-09-21T14:59:41.863 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2690000000000005e-06 2024-09-21T14:59:41.863 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.863 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.863 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:41.863 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.613700+0000", 2024-09-21T14:59:41.864 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.407823455 2024-09-21T14:59:41.864 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.864 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.864 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:41.864 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.614203+0000", 2024-09-21T14:59:41.864 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00050311500000000005 2024-09-21T14:59:41.864 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.864 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.864 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:59:41.864 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.614750+0000", 2024-09-21T14:59:41.864 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00054738699999999998 2024-09-21T14:59:41.864 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.864 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.864 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:41.864 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616367+0000", 2024-09-21T14:59:41.865 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001616634 2024-09-21T14:59:41.865 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.865 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.865 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:41.865 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616574+0000", 2024-09-21T14:59:41.865 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00020668499999999999 2024-09-21T14:59:41.865 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.865 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.865 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:41.865 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616802+0000", 2024-09-21T14:59:41.865 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00022817900000000001 2024-09-21T14:59:41.865 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.865 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.865 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:41.865 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616925+0000", 2024-09-21T14:59:41.866 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012309699999999999 2024-09-21T14:59:41.866 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:41.866 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:41.866 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:41.866 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.866 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.866 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T14:59:41.866 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.955609+0000", 2024-09-21T14:59:41.866 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 97.903125403999994, 2024-09-21T14:59:41.866 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013624153999999999, 2024-09-21T14:59:41.866 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:41.866 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:41.866 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:41.866 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:59:41.867 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:59:41.867 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 115 2024-09-21T14:59:41.867 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.867 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:41.867 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.867 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:41.867 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955609+0000", 2024-09-21T14:59:41.867 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:41.867 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.867 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.867 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:41.867 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955609+0000", 2024-09-21T14:59:41.867 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:41.867 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.867 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.868 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:41.868 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955607+0000", 2024-09-21T14:59:41.868 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-21T14:59:41.868 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.868 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.868 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:41.868 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955732+0000", 2024-09-21T14:59:41.868 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000125566 2024-09-21T14:59:41.868 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.868 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.868 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:41.868 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955752+0000", 2024-09-21T14:59:41.868 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9916999999999999e-05 2024-09-21T14:59:41.868 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.869 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.869 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:41.869 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955767+0000", 2024-09-21T14:59:41.869 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4687e-05 2024-09-21T14:59:41.869 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.869 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.869 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:41.869 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.965574+0000", 2024-09-21T14:59:41.869 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0098072850000000007 2024-09-21T14:59:41.869 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.869 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.869 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:41.869 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.965642+0000", 2024-09-21T14:59:41.869 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7784e-05 2024-09-21T14:59:41.870 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.870 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.870 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:59:41.870 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.965753+0000", 2024-09-21T14:59:41.870 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000111111 2024-09-21T14:59:41.870 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.870 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.870 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:41.870 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967717+0000", 2024-09-21T14:59:41.870 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0019635339999999999 2024-09-21T14:59:41.870 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.870 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.870 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:41.870 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.969014+0000", 2024-09-21T14:59:41.870 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001297093 2024-09-21T14:59:41.871 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.871 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.871 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:41.871 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.969130+0000", 2024-09-21T14:59:41.871 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000116011 2024-09-21T14:59:41.871 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.871 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.871 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:41.871 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.969233+0000", 2024-09-21T14:59:41.871 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000102982 2024-09-21T14:59:41.871 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:41.871 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:41.871 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:41.871 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.872 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.872 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T14:59:41.872 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.955783+0000", 2024-09-21T14:59:41.872 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 97.902950735000005, 2024-09-21T14:59:41.872 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014630241, 2024-09-21T14:59:41.872 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:41.872 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:41.872 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:41.872 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:59:41.872 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:59:41.872 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 116 2024-09-21T14:59:41.872 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.872 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:41.872 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.872 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:41.873 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955783+0000", 2024-09-21T14:59:41.873 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:41.873 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.873 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.873 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:41.873 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955783+0000", 2024-09-21T14:59:41.873 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:41.873 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.873 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.873 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:41.873 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955782+0000", 2024-09-21T14:59:41.873 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:59:41.873 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.873 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.874 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:41.874 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955791+0000", 2024-09-21T14:59:41.874 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8089999999999999e-06 2024-09-21T14:59:41.874 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.874 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.874 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:41.874 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955804+0000", 2024-09-21T14:59:41.874 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2548000000000001e-05 2024-09-21T14:59:41.874 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.874 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.874 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:41.874 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955814+0000", 2024-09-21T14:59:41.874 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0158e-05 2024-09-21T14:59:41.874 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.874 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.875 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:41.875 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.966515+0000", 2024-09-21T14:59:41.875 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.010701756 2024-09-21T14:59:41.875 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.875 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.875 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:41.875 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.966606+0000", 2024-09-21T14:59:41.875 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0761999999999996e-05 2024-09-21T14:59:41.875 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.875 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.875 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:59:41.875 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.966740+0000", 2024-09-21T14:59:41.875 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000133693 2024-09-21T14:59:41.875 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.876 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.876 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:41.876 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.968920+0000", 2024-09-21T14:59:41.876 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0021799290000000002 2024-09-21T14:59:41.876 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.876 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.876 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:41.876 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.970320+0000", 2024-09-21T14:59:41.876 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0014001759999999999 2024-09-21T14:59:41.876 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.876 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.876 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:41.876 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.970376+0000", 2024-09-21T14:59:41.876 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5696999999999999e-05 2024-09-21T14:59:41.876 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.877 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.877 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:41.877 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.970413+0000", 2024-09-21T14:59:41.877 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7750999999999998e-05 2024-09-21T14:59:41.877 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:41.877 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:41.877 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:41.877 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.877 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.877 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T14:59:41.877 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.955829+0000", 2024-09-21T14:59:41.877 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 97.902905265000001, 2024-09-21T14:59:41.877 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014919597999999999, 2024-09-21T14:59:41.877 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:41.878 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:41.878 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:41.878 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:59:41.878 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:59:41.878 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 117 2024-09-21T14:59:41.878 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.878 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:41.878 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.878 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:41.878 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955829+0000", 2024-09-21T14:59:41.878 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:41.878 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.878 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.878 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:41.879 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955829+0000", 2024-09-21T14:59:41.879 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:41.879 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.879 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.879 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:41.879 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955827+0000", 2024-09-21T14:59:41.879 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T14:59:41.879 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.879 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.879 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:41.879 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955838+0000", 2024-09-21T14:59:41.879 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.111e-05 2024-09-21T14:59:41.879 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.879 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.879 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:41.880 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955875+0000", 2024-09-21T14:59:41.880 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6211999999999998e-05 2024-09-21T14:59:41.880 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.880 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.880 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:41.880 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955899+0000", 2024-09-21T14:59:41.880 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4128999999999999e-05 2024-09-21T14:59:41.880 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.880 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.880 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:41.880 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967767+0000", 2024-09-21T14:59:41.880 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.011868251999999999 2024-09-21T14:59:41.880 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.881 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.881 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:41.881 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967866+0000", 2024-09-21T14:59:41.881 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8664000000000003e-05 2024-09-21T14:59:41.881 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.881 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.881 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:59:41.881 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.968128+0000", 2024-09-21T14:59:41.881 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00026243600000000001 2024-09-21T14:59:41.881 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.881 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.881 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:41.881 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.969393+0000", 2024-09-21T14:59:41.881 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0012650350000000001 2024-09-21T14:59:41.881 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.881 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.882 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:41.882 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.970523+0000", 2024-09-21T14:59:41.882 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001129557 2024-09-21T14:59:41.882 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.882 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.882 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:41.882 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.970603+0000", 2024-09-21T14:59:41.882 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0090000000000001e-05 2024-09-21T14:59:41.882 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.882 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.882 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:41.882 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.970748+0000", 2024-09-21T14:59:41.882 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014552799999999999 2024-09-21T14:59:41.882 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:41.882 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:41.883 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:41.883 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.883 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.883 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T14:59:41.883 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:04.493315+0000", 2024-09-21T14:59:41.883 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 97.365418431999998, 2024-09-21T14:59:41.883 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014055766000000001, 2024-09-21T14:59:41.883 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:41.883 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:41.883 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:41.883 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:59:41.883 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:59:41.883 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 137 2024-09-21T14:59:41.883 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.883 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:41.883 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.884 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:41.884 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493315+0000", 2024-09-21T14:59:41.884 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:41.884 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.884 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.884 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:41.884 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493315+0000", 2024-09-21T14:59:41.884 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:41.884 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.884 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.884 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:41.884 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493314+0000", 2024-09-21T14:59:41.884 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:59:41.884 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.884 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.884 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:41.885 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493507+0000", 2024-09-21T14:59:41.885 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000192444 2024-09-21T14:59:41.885 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.885 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.885 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:41.885 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493528+0000", 2024-09-21T14:59:41.885 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0845000000000001e-05 2024-09-21T14:59:41.885 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.885 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.885 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:41.885 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493543+0000", 2024-09-21T14:59:41.885 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5800000000000001e-05 2024-09-21T14:59:41.885 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.885 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.885 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:41.885 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.502575+0000", 2024-09-21T14:59:41.886 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0090312389999999999 2024-09-21T14:59:41.886 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.886 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.886 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:41.886 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.502677+0000", 2024-09-21T14:59:41.886 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000102271 2024-09-21T14:59:41.886 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.886 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.886 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:59:41.886 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.502825+0000", 2024-09-21T14:59:41.886 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000148171 2024-09-21T14:59:41.886 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.886 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.886 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:41.887 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507251+0000", 2024-09-21T14:59:41.887 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0044253879999999997 2024-09-21T14:59:41.887 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.887 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.887 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:41.887 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507313+0000", 2024-09-21T14:59:41.887 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2170999999999998e-05 2024-09-21T14:59:41.887 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.887 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.887 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:41.887 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507346+0000", 2024-09-21T14:59:41.887 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3079e-05 2024-09-21T14:59:41.887 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.887 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.887 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:41.888 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507371+0000", 2024-09-21T14:59:41.888 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5477999999999999e-05 2024-09-21T14:59:41.888 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:41.888 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:41.888 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:41.888 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.888 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.888 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T14:59:41.888 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:04.493570+0000", 2024-09-21T14:59:41.888 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 97.365164027000006, 2024-09-21T14:59:41.888 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014061686, 2024-09-21T14:59:41.888 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:41.888 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:41.888 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:41.888 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:59:41.889 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:59:41.889 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 138 2024-09-21T14:59:41.889 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.889 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:41.889 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.889 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:41.889 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493570+0000", 2024-09-21T14:59:41.889 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:41.889 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.889 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.889 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:41.889 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493570+0000", 2024-09-21T14:59:41.889 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:41.889 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.889 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.890 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:41.890 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493568+0000", 2024-09-21T14:59:41.890 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T14:59:41.890 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.890 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.890 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:41.890 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493579+0000", 2024-09-21T14:59:41.890 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0574000000000001e-05 2024-09-21T14:59:41.890 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.890 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.890 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:41.890 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493595+0000", 2024-09-21T14:59:41.890 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6504999999999999e-05 2024-09-21T14:59:41.890 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.891 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.891 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:41.891 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493608+0000", 2024-09-21T14:59:41.891 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2836999999999999e-05 2024-09-21T14:59:41.891 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.891 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.891 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:41.891 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.503909+0000", 2024-09-21T14:59:41.891 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.01030118 2024-09-21T14:59:41.891 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.891 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.891 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:41.891 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.504064+0000", 2024-09-21T14:59:41.891 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015493000000000001 2024-09-21T14:59:41.891 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.892 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.892 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:59:41.892 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.504299+0000", 2024-09-21T14:59:41.892 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00023416100000000001 2024-09-21T14:59:41.892 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.892 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.892 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:41.892 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507466+0000", 2024-09-21T14:59:41.892 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0031678380000000001 2024-09-21T14:59:41.892 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.892 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.892 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:41.892 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507483+0000", 2024-09-21T14:59:41.892 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6606999999999999e-05 2024-09-21T14:59:41.893 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.893 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.893 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:41.893 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507566+0000", 2024-09-21T14:59:41.893 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3110999999999998e-05 2024-09-21T14:59:41.893 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.893 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.893 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:41.893 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507632+0000", 2024-09-21T14:59:41.893 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5480000000000003e-05 2024-09-21T14:59:41.893 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:41.893 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:41.893 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:41.893 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.894 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.894 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T14:59:41.894 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:04.493625+0000", 2024-09-21T14:59:41.894 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 97.365109339, 2024-09-21T14:59:41.894 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014665378999999999, 2024-09-21T14:59:41.894 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:41.894 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:41.894 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:41.894 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:59:41.894 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:59:41.894 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139 2024-09-21T14:59:41.894 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.894 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:41.894 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.895 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:41.895 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493625+0000", 2024-09-21T14:59:41.895 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:41.895 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.895 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.895 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:41.895 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493625+0000", 2024-09-21T14:59:41.895 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:41.895 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.895 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.895 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:41.895 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493624+0000", 2024-09-21T14:59:41.895 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:59:41.895 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.896 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.896 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:41.896 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493638+0000", 2024-09-21T14:59:41.896 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4282e-05 2024-09-21T14:59:41.896 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.896 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.896 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:41.896 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493654+0000", 2024-09-21T14:59:41.896 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5739999999999998e-05 2024-09-21T14:59:41.896 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.896 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.896 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:41.896 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493666+0000", 2024-09-21T14:59:41.896 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2865e-05 2024-09-21T14:59:41.897 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.897 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.897 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:41.897 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.505329+0000", 2024-09-21T14:59:41.897 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.011662237000000001 2024-09-21T14:59:41.897 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.897 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.897 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:41.897 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.505377+0000", 2024-09-21T14:59:41.897 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7862999999999999e-05 2024-09-21T14:59:41.897 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.897 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.897 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:59:41.897 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.505455+0000", 2024-09-21T14:59:41.898 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8918000000000006e-05 2024-09-21T14:59:41.898 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.898 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.898 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:41.898 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507650+0000", 2024-09-21T14:59:41.898 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.002195027 2024-09-21T14:59:41.898 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.898 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.898 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:41.898 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508149+0000", 2024-09-21T14:59:41.898 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00049828399999999999 2024-09-21T14:59:41.898 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.898 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.898 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:41.899 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508230+0000", 2024-09-21T14:59:41.899 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1376000000000002e-05 2024-09-21T14:59:41.899 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.899 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.899 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:41.899 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508290+0000", 2024-09-21T14:59:41.899 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9852999999999998e-05 2024-09-21T14:59:41.899 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:41.899 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:41.899 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:41.899 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.899 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.899 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T14:59:41.899 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:04.493679+0000", 2024-09-21T14:59:41.900 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 97.365054440999998, 2024-09-21T14:59:41.900 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014869869000000001, 2024-09-21T14:59:41.900 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:41.900 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:41.900 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:41.900 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:59:41.900 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:59:41.900 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140 2024-09-21T14:59:41.900 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.900 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:41.900 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.900 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:41.900 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493679+0000", 2024-09-21T14:59:41.900 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:41.901 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.901 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.901 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:41.901 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493679+0000", 2024-09-21T14:59:41.901 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:41.901 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.901 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.901 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:41.901 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493679+0000", 2024-09-21T14:59:41.901 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:59:41.901 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.901 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.901 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:41.901 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493889+0000", 2024-09-21T14:59:41.901 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00021032700000000001 2024-09-21T14:59:41.901 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.902 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.902 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:41.902 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493931+0000", 2024-09-21T14:59:41.902 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1971000000000002e-05 2024-09-21T14:59:41.902 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.902 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.902 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:41.902 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493947+0000", 2024-09-21T14:59:41.902 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6385e-05 2024-09-21T14:59:41.902 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.902 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.902 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:41.902 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.505804+0000", 2024-09-21T14:59:41.902 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.011857136000000001 2024-09-21T14:59:41.902 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.903 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.903 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:41.903 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.505992+0000", 2024-09-21T14:59:41.903 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000187665 2024-09-21T14:59:41.903 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.903 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.903 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:59:41.903 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.506281+0000", 2024-09-21T14:59:41.903 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000288712 2024-09-21T14:59:41.903 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.903 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.903 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:41.903 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508349+0000", 2024-09-21T14:59:41.903 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0020685790000000001 2024-09-21T14:59:41.903 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.904 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.904 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:41.904 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508371+0000", 2024-09-21T14:59:41.904 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1747e-05 2024-09-21T14:59:41.904 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.904 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.904 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:41.904 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508450+0000", 2024-09-21T14:59:41.904 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9332000000000005e-05 2024-09-21T14:59:41.904 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.904 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.904 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:41.904 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508549+0000", 2024-09-21T14:59:41.904 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8918000000000004e-05 2024-09-21T14:59:41.904 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:41.905 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:41.905 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:41.905 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.905 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.905 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T14:59:41.905 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:04.493967+0000", 2024-09-21T14:59:41.905 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 97.364767404000006, 2024-09-21T14:59:41.905 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.015281075999999999, 2024-09-21T14:59:41.905 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:41.905 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:41.905 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:41.905 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:59:41.905 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:59:41.905 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 141 2024-09-21T14:59:41.906 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.906 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:41.906 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.906 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:41.906 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493967+0000", 2024-09-21T14:59:41.906 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:41.906 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.906 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.906 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:41.906 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493967+0000", 2024-09-21T14:59:41.906 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:41.906 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.906 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.906 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:41.907 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493965+0000", 2024-09-21T14:59:41.907 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T14:59:41.907 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.907 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.907 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:41.907 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493987+0000", 2024-09-21T14:59:41.907 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1662999999999999e-05 2024-09-21T14:59:41.907 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.907 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.907 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:41.907 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.494004+0000", 2024-09-21T14:59:41.907 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7311000000000001e-05 2024-09-21T14:59:41.907 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.907 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.907 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:41.907 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.494017+0000", 2024-09-21T14:59:41.908 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3202e-05 2024-09-21T14:59:41.908 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.908 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.908 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:41.908 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507678+0000", 2024-09-21T14:59:41.908 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013660432 2024-09-21T14:59:41.908 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.908 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.908 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:41.908 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507726+0000", 2024-09-21T14:59:41.908 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8154e-05 2024-09-21T14:59:41.908 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.908 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.908 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:59:41.908 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507797+0000", 2024-09-21T14:59:41.909 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.1110000000000002e-05 2024-09-21T14:59:41.909 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.909 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.909 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:41.909 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508710+0000", 2024-09-21T14:59:41.909 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00091339299999999997 2024-09-21T14:59:41.909 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.909 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.909 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:41.909 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.509108+0000", 2024-09-21T14:59:41.909 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00039718300000000001 2024-09-21T14:59:41.909 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.909 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.909 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:41.909 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.509186+0000", 2024-09-21T14:59:41.909 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8592999999999998e-05 2024-09-21T14:59:41.910 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.910 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.910 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:41.910 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.509248+0000", 2024-09-21T14:59:41.910 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1450000000000003e-05 2024-09-21T14:59:41.910 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:41.910 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:41.910 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:41.910 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.910 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.910 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6631 3.1a 3:5d547f54:::rbd_data.11361bed1f78.0000000000000003:d [read 0~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:59:41.910 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131606+0000", 2024-09-21T14:59:41.910 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 66.727128125999997, 2024-09-21T14:59:41.910 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013903148000000001, 2024-09-21T14:59:41.910 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:41.911 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:41.911 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:41.911 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:41.911 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:41.911 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6631 2024-09-21T14:59:41.911 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.911 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:41.911 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.911 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:41.911 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131606+0000", 2024-09-21T14:59:41.911 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:41.911 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.911 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.911 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:41.912 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131606+0000", 2024-09-21T14:59:41.912 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:41.912 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.912 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.912 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:41.912 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131606+0000", 2024-09-21T14:59:41.912 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T14:59:41.912 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.912 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.912 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:41.912 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131607+0000", 2024-09-21T14:59:41.912 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2699999999999999e-06 2024-09-21T14:59:41.912 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.912 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.913 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:41.913 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131610+0000", 2024-09-21T14:59:41.913 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.88e-06 2024-09-21T14:59:41.913 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.913 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.913 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:41.913 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131612+0000", 2024-09-21T14:59:41.913 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5170000000000002e-06 2024-09-21T14:59:41.913 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.913 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.913 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:41.913 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.144205+0000", 2024-09-21T14:59:41.913 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012592763 2024-09-21T14:59:41.913 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.913 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.914 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:41.914 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.144332+0000", 2024-09-21T14:59:41.914 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012694200000000001 2024-09-21T14:59:41.914 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.914 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.914 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:41.914 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.145509+0000", 2024-09-21T14:59:41.914 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001176992 2024-09-21T14:59:41.914 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:41.914 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:41.914 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:41.914 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.914 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.914 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6632 3.1a 3:5d547f54:::rbd_data.11361bed1f78.0000000000000003:d [read 81920~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:59:41.915 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131614+0000", 2024-09-21T14:59:41.915 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 66.727119517999995, 2024-09-21T14:59:41.915 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014093553999999999, 2024-09-21T14:59:41.915 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:41.915 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:41.915 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:41.915 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:41.915 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:41.915 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6632 2024-09-21T14:59:41.915 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.915 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:41.915 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.915 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:41.915 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131614+0000", 2024-09-21T14:59:41.916 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:41.916 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.916 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.916 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:41.916 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131614+0000", 2024-09-21T14:59:41.916 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:41.916 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.916 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.916 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:41.916 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131614+0000", 2024-09-21T14:59:41.916 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T14:59:41.916 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.916 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.916 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:41.917 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131615+0000", 2024-09-21T14:59:41.917 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.218e-06 2024-09-21T14:59:41.917 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.917 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.917 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:41.917 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131618+0000", 2024-09-21T14:59:41.917 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9050000000000001e-06 2024-09-21T14:59:41.917 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.917 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.917 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:41.917 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131621+0000", 2024-09-21T14:59:41.917 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7439999999999999e-06 2024-09-21T14:59:41.917 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.917 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.918 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:41.918 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.145538+0000", 2024-09-21T14:59:41.918 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013916618 2024-09-21T14:59:41.918 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.918 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.918 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:41.918 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.145619+0000", 2024-09-21T14:59:41.918 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1812999999999997e-05 2024-09-21T14:59:41.918 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.918 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.918 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:41.918 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.145708+0000", 2024-09-21T14:59:41.918 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8460000000000003e-05 2024-09-21T14:59:41.919 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:41.919 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:41.919 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:41.919 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.919 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.919 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6633 3.1a 3:5d547f54:::rbd_data.11361bed1f78.0000000000000003:d [read 147456~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:59:41.919 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131623+0000", 2024-09-21T14:59:41.919 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 66.727110551999999, 2024-09-21T14:59:41.919 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014395845000000001, 2024-09-21T14:59:41.919 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:41.919 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:41.919 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:41.919 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:41.919 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:41.920 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6633 2024-09-21T14:59:41.920 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.920 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:41.920 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.920 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:41.920 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131623+0000", 2024-09-21T14:59:41.920 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:41.920 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.920 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.920 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:41.920 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131623+0000", 2024-09-21T14:59:41.920 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:41.920 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.920 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.920 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:41.921 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131623+0000", 2024-09-21T14:59:41.921 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T14:59:41.921 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.921 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.921 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:41.921 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131628+0000", 2024-09-21T14:59:41.921 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0370000000000002e-06 2024-09-21T14:59:41.921 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.921 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.921 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:41.921 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131632+0000", 2024-09-21T14:59:41.921 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3519999999999998e-06 2024-09-21T14:59:41.921 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.921 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.922 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:41.922 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131634+0000", 2024-09-21T14:59:41.922 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5019999999999999e-06 2024-09-21T14:59:41.922 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.922 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.922 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:41.922 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.145734+0000", 2024-09-21T14:59:41.922 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014100118 2024-09-21T14:59:41.922 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.922 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.922 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:41.922 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.145804+0000", 2024-09-21T14:59:41.922 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0229999999999997e-05 2024-09-21T14:59:41.922 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.922 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.923 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:41.923 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146019+0000", 2024-09-21T14:59:41.923 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00021481699999999999 2024-09-21T14:59:41.923 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:41.923 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:41.923 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:41.923 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.923 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.923 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6634 3.1a 3:5d547f54:::rbd_data.11361bed1f78.0000000000000003:d [read 229376~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:59:41.923 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131636+0000", 2024-09-21T14:59:41.923 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 66.727097442000002, 2024-09-21T14:59:41.923 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014645843, 2024-09-21T14:59:41.923 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:41.923 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:41.923 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:41.924 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:41.924 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:41.924 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6634 2024-09-21T14:59:41.924 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.924 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:41.924 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.924 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:41.924 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131636+0000", 2024-09-21T14:59:41.924 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:41.924 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.924 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.924 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:41.924 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131636+0000", 2024-09-21T14:59:41.924 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:41.924 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.924 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.925 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:41.925 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131636+0000", 2024-09-21T14:59:41.925 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:41.925 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.925 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.925 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:41.925 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131641+0000", 2024-09-21T14:59:41.925 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8489999999999997e-06 2024-09-21T14:59:41.925 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.925 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.925 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:41.925 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131644+0000", 2024-09-21T14:59:41.925 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.179e-06 2024-09-21T14:59:41.925 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.926 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.926 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:41.926 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131647+0000", 2024-09-21T14:59:41.926 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8009999999999999e-06 2024-09-21T14:59:41.926 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.926 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.926 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:41.926 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146046+0000", 2024-09-21T14:59:41.926 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014399018 2024-09-21T14:59:41.926 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.926 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.926 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:41.926 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146118+0000", 2024-09-21T14:59:41.926 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2044000000000003e-05 2024-09-21T14:59:41.926 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.927 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.927 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:41.927 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146282+0000", 2024-09-21T14:59:41.927 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000164196 2024-09-21T14:59:41.927 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:41.927 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:41.927 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:41.927 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.927 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.927 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6639 3.1a 3:592d80ed:::rbd_data.11361bed1f78.0000000000000005:d [read 0~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:59:41.927 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131650+0000", 2024-09-21T14:59:41.927 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 66.727084242999993, 2024-09-21T14:59:41.927 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016002832000000002, 2024-09-21T14:59:41.927 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:41.927 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:41.928 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:41.928 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:41.928 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:41.928 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6639 2024-09-21T14:59:41.928 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.928 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:41.928 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.928 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:41.928 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131650+0000", 2024-09-21T14:59:41.928 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:41.928 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.928 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.928 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:41.928 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131650+0000", 2024-09-21T14:59:41.929 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:41.929 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.929 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.929 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:41.929 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131649+0000", 2024-09-21T14:59:41.929 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:41.929 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.929 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.929 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:41.929 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131651+0000", 2024-09-21T14:59:41.929 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9199999999999998e-06 2024-09-21T14:59:41.929 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.929 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.929 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:41.930 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131656+0000", 2024-09-21T14:59:41.930 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9899999999999997e-06 2024-09-21T14:59:41.930 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.930 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.930 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:41.930 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131665+0000", 2024-09-21T14:59:41.930 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0999999999999993e-06 2024-09-21T14:59:41.930 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.930 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.930 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:41.930 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146309+0000", 2024-09-21T14:59:41.930 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014643893999999999 2024-09-21T14:59:41.930 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.930 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.930 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:41.931 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146392+0000", 2024-09-21T14:59:41.931 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2531e-05 2024-09-21T14:59:41.931 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.931 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.931 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:41.931 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147653+0000", 2024-09-21T14:59:41.931 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0012606710000000001 2024-09-21T14:59:41.931 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:41.931 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:41.931 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:41.931 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.931 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.931 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6640 3.1a 3:592d80ed:::rbd_data.11361bed1f78.0000000000000005:d [read 65536~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:59:41.931 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131670+0000", 2024-09-21T14:59:41.932 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 66.727063943999994, 2024-09-21T14:59:41.932 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016188859, 2024-09-21T14:59:41.932 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:41.932 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:41.932 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:41.932 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:41.932 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:41.932 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6640 2024-09-21T14:59:41.932 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.932 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:41.932 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.932 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:41.932 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131670+0000", 2024-09-21T14:59:41.932 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:41.932 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.933 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.933 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:41.933 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131670+0000", 2024-09-21T14:59:41.933 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:41.933 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.933 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.933 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:41.933 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131670+0000", 2024-09-21T14:59:41.933 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T14:59:41.933 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.933 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.933 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:41.933 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131672+0000", 2024-09-21T14:59:41.933 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.818e-06 2024-09-21T14:59:41.933 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.934 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.934 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:41.934 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131675+0000", 2024-09-21T14:59:41.934 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5930000000000002e-06 2024-09-21T14:59:41.934 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.934 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.934 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:41.934 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131678+0000", 2024-09-21T14:59:41.934 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6790000000000001e-06 2024-09-21T14:59:41.934 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.934 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.934 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:41.934 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147681+0000", 2024-09-21T14:59:41.935 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016002755 2024-09-21T14:59:41.935 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.935 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.935 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:41.935 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147762+0000", 2024-09-21T14:59:41.935 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1052999999999998e-05 2024-09-21T14:59:41.935 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.935 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.935 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:41.935 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147859+0000", 2024-09-21T14:59:41.935 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.7199000000000002e-05 2024-09-21T14:59:41.935 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:41.935 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:41.935 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:41.936 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.936 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.936 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6641 3.1a 3:592d80ed:::rbd_data.11361bed1f78.0000000000000005:d [read 147456~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:59:41.936 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131680+0000", 2024-09-21T14:59:41.936 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 66.727053655999995, 2024-09-21T14:59:41.936 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016431296000000001, 2024-09-21T14:59:41.936 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:41.936 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:41.936 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:41.936 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:41.936 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:41.936 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6641 2024-09-21T14:59:41.936 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.937 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:41.937 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.937 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:41.937 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131680+0000", 2024-09-21T14:59:41.937 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:41.937 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.937 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.937 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:41.937 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131680+0000", 2024-09-21T14:59:41.937 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:41.937 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.937 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.937 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:41.938 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131680+0000", 2024-09-21T14:59:41.938 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T14:59:41.938 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.938 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.938 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:41.938 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131681+0000", 2024-09-21T14:59:41.938 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2610000000000001e-06 2024-09-21T14:59:41.938 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.938 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.938 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:41.938 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131685+0000", 2024-09-21T14:59:41.938 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.8299999999999998e-06 2024-09-21T14:59:41.938 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.938 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.939 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:41.939 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131688+0000", 2024-09-21T14:59:41.939 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6189999999999998e-06 2024-09-21T14:59:41.939 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.939 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.939 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:41.939 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147884+0000", 2024-09-21T14:59:41.939 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016196465 2024-09-21T14:59:41.939 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.939 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.939 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:41.939 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147949+0000", 2024-09-21T14:59:41.939 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4867000000000001e-05 2024-09-21T14:59:41.939 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.940 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.940 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:41.940 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.148112+0000", 2024-09-21T14:59:41.940 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00016247700000000001 2024-09-21T14:59:41.940 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:41.940 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:41.940 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:41.940 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.940 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.940 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6642 3.1a 3:592d80ed:::rbd_data.11361bed1f78.0000000000000005:d [read 212992~32768 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:59:41.940 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131690+0000", 2024-09-21T14:59:41.940 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 66.727043777999995, 2024-09-21T14:59:41.940 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016751335999999999, 2024-09-21T14:59:41.941 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:41.941 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:41.941 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:41.941 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:41.941 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:41.941 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6642 2024-09-21T14:59:41.941 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.941 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:41.941 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.941 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:41.941 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131690+0000", 2024-09-21T14:59:41.941 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:41.941 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.941 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.941 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:41.942 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131690+0000", 2024-09-21T14:59:41.942 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:41.942 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.942 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.942 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:41.942 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131690+0000", 2024-09-21T14:59:41.942 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T14:59:41.942 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.942 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.942 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:41.942 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131691+0000", 2024-09-21T14:59:41.942 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3120000000000001e-06 2024-09-21T14:59:41.942 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.942 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.943 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:41.943 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131694+0000", 2024-09-21T14:59:41.943 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0070000000000002e-06 2024-09-21T14:59:41.943 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.943 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.943 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:41.943 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131700+0000", 2024-09-21T14:59:41.943 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0800000000000002e-06 2024-09-21T14:59:41.943 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.943 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.943 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:41.943 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.148135+0000", 2024-09-21T14:59:41.943 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016434250000000001 2024-09-21T14:59:41.943 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.943 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.944 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:41.944 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.148204+0000", 2024-09-21T14:59:41.944 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9466000000000002e-05 2024-09-21T14:59:41.944 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.944 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.944 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:41.944 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.148441+0000", 2024-09-21T14:59:41.944 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00023743700000000001 2024-09-21T14:59:41.944 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:41.944 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:41.944 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:41.944 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.944 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.944 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:61214 3.7 3:e28aec10:::rbd_header.1136862f872f:head [watch reconnect cookie 139864671932080 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-21T14:59:41.945 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:18.147172+0000", 2024-09-21T14:59:41.945 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 23.711562147999999, 2024-09-21T14:59:41.945 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.019264956, 2024-09-21T14:59:41.945 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:41.945 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:41.945 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:41.945 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:41.945 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:41.945 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 61214 2024-09-21T14:59:41.945 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.945 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:41.945 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.945 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:41.945 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147172+0000", 2024-09-21T14:59:41.945 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:41.945 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.946 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.946 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:41.946 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147172+0000", 2024-09-21T14:59:41.946 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:41.946 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.946 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.946 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:41.946 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147168+0000", 2024-09-21T14:59:41.946 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999962 2024-09-21T14:59:41.946 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.946 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.946 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:41.946 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147188+0000", 2024-09-21T14:59:41.946 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9717000000000001e-05 2024-09-21T14:59:41.947 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.947 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.947 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:41.947 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147239+0000", 2024-09-21T14:59:41.947 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1493999999999999e-05 2024-09-21T14:59:41.947 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.947 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.947 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:41.947 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147265+0000", 2024-09-21T14:59:41.947 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5746999999999999e-05 2024-09-21T14:59:41.947 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.947 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.947 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:41.947 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147372+0000", 2024-09-21T14:59:41.947 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010681200000000001 2024-09-21T14:59:41.948 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.948 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.948 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op must wait for map", 2024-09-21T14:59:41.948 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147380+0000", 2024-09-21T14:59:41.948 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7919999999999993e-06 2024-09-21T14:59:41.948 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.948 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.948 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:41.948 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.154025+0000", 2024-09-21T14:59:41.948 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0066454909999999999 2024-09-21T14:59:41.948 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.948 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.948 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T14:59:41.948 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.154041+0000", 2024-09-21T14:59:41.948 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5894000000000001e-05 2024-09-21T14:59:41.948 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.949 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.949 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:41.949 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.164345+0000", 2024-09-21T14:59:41.949 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0103040459999999 2024-09-21T14:59:41.949 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.949 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.949 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:41.949 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.164590+0000", 2024-09-21T14:59:41.949 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00024492699999999998 2024-09-21T14:59:41.949 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.949 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.949 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:41.949 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.166437+0000", 2024-09-21T14:59:41.949 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0018466330000000001 2024-09-21T14:59:41.949 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:41.949 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:41.950 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:41.950 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.950 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.950 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:61215 3.7 3:e28aec10:::rbd_header.1136862f872f:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-21T14:59:41.950 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:18.152008+0000", 2024-09-21T14:59:41.950 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 23.706725695999999, 2024-09-21T14:59:41.950 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.015644958, 2024-09-21T14:59:41.950 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:41.950 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:41.950 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:41.950 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:41.950 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:41.950 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 61215 2024-09-21T14:59:41.950 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.951 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:41.951 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.951 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:41.951 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152008+0000", 2024-09-21T14:59:41.951 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:41.951 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.951 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.951 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:41.951 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152008+0000", 2024-09-21T14:59:41.951 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:41.951 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.951 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.951 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:41.951 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152005+0000", 2024-09-21T14:59:41.952 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-21T14:59:41.952 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.952 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.952 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:41.952 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152025+0000", 2024-09-21T14:59:41.952 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9306000000000001e-05 2024-09-21T14:59:41.952 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.952 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.952 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:41.952 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152054+0000", 2024-09-21T14:59:41.952 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9189000000000001e-05 2024-09-21T14:59:41.952 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.952 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.953 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:41.953 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152078+0000", 2024-09-21T14:59:41.953 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4317000000000001e-05 2024-09-21T14:59:41.953 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.953 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.953 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:41.953 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152243+0000", 2024-09-21T14:59:41.953 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000164802 2024-09-21T14:59:41.953 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.953 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.953 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting_for_map not empty", 2024-09-21T14:59:41.953 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152251+0000", 2024-09-21T14:59:41.953 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0600000000000008e-06 2024-09-21T14:59:41.953 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.954 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.954 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:41.954 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.154741+0000", 2024-09-21T14:59:41.954 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0024900069999999998 2024-09-21T14:59:41.954 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.954 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.954 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T14:59:41.954 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.154753+0000", 2024-09-21T14:59:41.954 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1588e-05 2024-09-21T14:59:41.954 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.954 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.954 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:41.954 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.165380+0000", 2024-09-21T14:59:41.954 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0106272249999999 2024-09-21T14:59:41.955 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.955 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.955 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:41.955 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.165517+0000", 2024-09-21T14:59:41.955 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013737999999999999 2024-09-21T14:59:41.955 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.955 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.955 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 6", 2024-09-21T14:59:41.955 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.165938+0000", 2024-09-21T14:59:41.955 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00042082399999999998 2024-09-21T14:59:41.955 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.955 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.955 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:41.955 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.166795+0000", 2024-09-21T14:59:41.956 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00085674899999999997 2024-09-21T14:59:41.956 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.956 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.956 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:41.956 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.167598+0000", 2024-09-21T14:59:41.956 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00080335599999999997 2024-09-21T14:59:41.956 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.956 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.956 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:41.956 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.167632+0000", 2024-09-21T14:59:41.956 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4122000000000003e-05 2024-09-21T14:59:41.956 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.956 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.956 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:41.957 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.167653+0000", 2024-09-21T14:59:41.957 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0862000000000001e-05 2024-09-21T14:59:41.957 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:41.957 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:41.957 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:41.957 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.957 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.957 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:73576 3.1c 3:3fc65d4e:::rbd_data.1136f4939602.0000000000000018:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-21T14:59:41.957 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:41.660752+0000", 2024-09-21T14:59:41.957 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.19798220499999999, 2024-09-21T14:59:41.957 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012812616000000001, 2024-09-21T14:59:41.957 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:41.957 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:41.957 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:41.958 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:41.958 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:41.958 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 73576 2024-09-21T14:59:41.958 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.958 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:41.958 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.958 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:41.958 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.660752+0000", 2024-09-21T14:59:41.958 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:41.958 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.958 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.958 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:41.958 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.660752+0000", 2024-09-21T14:59:41.959 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:41.959 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.959 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.959 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:41.959 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.660751+0000", 2024-09-21T14:59:41.959 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:59:41.959 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.959 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.959 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:41.959 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.660760+0000", 2024-09-21T14:59:41.959 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2299999999999997e-06 2024-09-21T14:59:41.959 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.959 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.959 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:41.960 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.660778+0000", 2024-09-21T14:59:41.960 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7555999999999999e-05 2024-09-21T14:59:41.960 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.960 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.960 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:41.960 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.660787+0000", 2024-09-21T14:59:41.960 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6369999999999994e-06 2024-09-21T14:59:41.960 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.960 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.960 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:41.960 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.660885+0000", 2024-09-21T14:59:41.960 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.7876999999999998e-05 2024-09-21T14:59:41.960 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.960 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.960 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:41.960 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.661156+0000", 2024-09-21T14:59:41.961 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00027094400000000001 2024-09-21T14:59:41.961 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.961 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.961 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 0", 2024-09-21T14:59:41.961 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.664772+0000", 2024-09-21T14:59:41.961 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0036163380000000002 2024-09-21T14:59:41.961 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.961 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.961 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:41.961 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.673494+0000", 2024-09-21T14:59:41.961 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0087212339999999996 2024-09-21T14:59:41.961 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.961 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.961 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:41.961 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.673505+0000", 2024-09-21T14:59:41.962 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1868000000000001e-05 2024-09-21T14:59:41.962 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.962 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.962 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:41.962 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.673537+0000", 2024-09-21T14:59:41.962 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1247000000000001e-05 2024-09-21T14:59:41.962 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:41.962 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:41.962 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:41.962 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.673564+0000", 2024-09-21T14:59:41.962 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7636000000000001e-05 2024-09-21T14:59:41.962 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:41.962 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:41.962 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:41.963 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:41.963 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:41.963 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:41.964 INFO:teuthology.orchestra.run.smithi032.stdout:1948 punch from 0xb576c5 to 0xb67011, (0xf94c bytes) 2024-09-21T14:59:41.964 INFO:teuthology.orchestra.run.smithi032.stdout:1949 punch from 0x73ab8d8 to 0x73b9a47, (0xe16f bytes) 2024-09-21T14:59:41.964 INFO:teuthology.orchestra.run.smithi032.stdout:1950 read 0x8e5caf9 thru 0x8e635e6 (0x6aee bytes) 2024-09-21T14:59:41.964 INFO:teuthology.orchestra.run.smithi032.stdout:1951 punch from 0x2c4cc3b to 0x2c558e3, (0x8ca8 bytes) 2024-09-21T14:59:41.964 INFO:teuthology.orchestra.run.smithi032.stdout:1952 writesame 0x9660630 thru 0x966ea3c data_size 0x1(0xe40d bytes) 2024-09-21T14:59:41.965 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T14:59:41.967 INFO:teuthology.orchestra.run.smithi032.stdout:1954 punch from 0x124ee39 to 0x124ee5c, (0x23 bytes) 2024-09-21T14:59:41.967 INFO:teuthology.orchestra.run.smithi032.stdout:1956 writesame 0x1ab4e63 thru 0x1abac01 data_size 0x3(0x5d9f bytes) 2024-09-21T14:59:41.967 INFO:teuthology.orchestra.run.smithi032.stdout:1959 punch from 0x87ecaad to 0x87f4c2c, (0x817f bytes) 2024-09-21T14:59:41.967 INFO:teuthology.orchestra.run.smithi032.stdout:1961 punch from 0xa17cd1d to 0xa18676c, (0x9a4f bytes) 2024-09-21T14:59:41.967 INFO:teuthology.orchestra.run.smithi032.stdout:1963 punch from 0x582f405 to 0x58342e3, (0x4ede bytes) 2024-09-21T14:59:41.967 INFO:teuthology.orchestra.run.smithi032.stdout:1964 writesame 0x7b2bd60 thru 0x7b37462 data_size 0x17(0xb703 bytes) 2024-09-21T14:59:41.967 INFO:teuthology.orchestra.run.smithi032.stdout:1965 punch from 0x13ead3f to 0x13f10ba, (0x637b bytes) 2024-09-21T14:59:41.967 INFO:teuthology.orchestra.run.smithi032.stdout:1966 writesame 0x4bec261 thru 0x4bfbc50 data_size 0x8(0xf9f0 bytes) 2024-09-21T14:59:41.967 INFO:teuthology.orchestra.run.smithi032.stdout:1967 punch from 0x1ac4aba to 0x1acf6d2, (0xac18 bytes) 2024-09-21T14:59:41.967 INFO:teuthology.orchestra.run.smithi032.stdout:1968 writesame 0x11f9229 thru 0x11ffb83 data_size 0x1(0x695b bytes) 2024-09-21T14:59:41.967 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:41.967 INFO:teuthology.orchestra.run.smithi032.stdout:1970 writesame 0x327e88d thru 0x32875eb data_size 0x1(0x8d5f bytes) 2024-09-21T14:59:41.968 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:59:41.959+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:59:41.985 INFO:teuthology.orchestra.run.smithi032.stdout:1971 punch from 0x4a8c68 to 0x4b0399, (0x7731 bytes) 2024-09-21T14:59:41.985 INFO:teuthology.orchestra.run.smithi032.stdout:1973 writesame 0x57c71a0 thru 0x57d263a data_size 0x7(0xb49b bytes) 2024-09-21T14:59:41.995 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:41.995 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:41.995 INFO:teuthology.orchestra.run.smithi032.stdout:1976 write 0x3d4dc41 thru 0x3d5a680 (0xca40 bytes) 2024-09-21T14:59:42.002 INFO:teuthology.orchestra.run.smithi032.stdout:1977 punch from 0x2e10f37 to 0x2e1d05e, (0xc127 bytes) 2024-09-21T14:59:42.003 INFO:teuthology.orchestra.run.smithi032.stdout:1979 write 0x76514c7 thru 0x765e290 (0xcdca bytes) 2024-09-21T14:59:42.009 INFO:teuthology.orchestra.run.smithi032.stdout:1980 write 0x801f2f6 thru 0x801fb85 (0x890 bytes) 2024-09-21T14:59:42.012 INFO:teuthology.orchestra.run.smithi032.stdout:1982 write 0x7458fdf thru 0x7459cbf (0xce1 bytes) 2024-09-21T14:59:42.018 INFO:teuthology.orchestra.run.smithi032.stdout:1983 write 0x250cf6c thru 0x2510256 (0x32eb bytes) 2024-09-21T14:59:42.022 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:42.022 INFO:teuthology.orchestra.run.smithi032.stdout:1985 read 0x8902ed0 thru 0x890acfe (0x7e2f bytes) 2024-09-21T14:59:42.022 INFO:teuthology.orchestra.run.smithi032.stdout:1986 punch from 0x46ad56d to 0x46adf92, (0xa25 bytes) 2024-09-21T14:59:42.035 INFO:teuthology.orchestra.run.smithi032.stdout:1987 trunc from 0xaa832f4 to 0x4238f6c 2024-09-21T14:59:42.060 INFO:teuthology.orchestra.run.smithi032.stdout:1989 punch from 0x3889381 to 0x3892a70, (0x96ef bytes) 2024-09-21T14:59:42.065 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:59:42.059+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:59:42.077 INFO:teuthology.orchestra.run.smithi032.stdout:1990 write 0x754df54 thru 0x7557f27 (0x9fd4 bytes) 2024-09-21T14:59:42.078 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:42.078 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:59:42.078 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T14:59:42.078 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:42.089 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T14:59:42.093 INFO:teuthology.orchestra.run.smithi032.stdout:1991 trunc from 0x7557f28 to 0x283ef02 2024-09-21T14:59:42.125 INFO:teuthology.orchestra.run.smithi032.stdout:1992 read 0x16b98bf thru 0x16bba74 (0x21b6 bytes) 2024-09-21T14:59:42.156 INFO:teuthology.orchestra.run.smithi032.stdout:1993 writesame 0xbbcef43 thru 0xbbd398d data_size 0x13(0x4a4b bytes) 2024-09-21T14:59:42.156 INFO:teuthology.orchestra.run.smithi032.stdout:1994 read 0xd5039f thru 0xd52d85 (0x29e7 bytes) 2024-09-21T14:59:42.156 INFO:teuthology.orchestra.run.smithi032.stdout:1995 write 0x57b441a thru 0x57bcc33 (0x881a bytes) 2024-09-21T14:59:42.163 INFO:teuthology.orchestra.run.smithi032.stdout:1997 read 0x4caf9df thru 0x4cb731b (0x793d bytes) 2024-09-21T14:59:42.164 INFO:teuthology.orchestra.run.smithi032.stdout:1998 write 0x3b02268 thru 0x3b06202 (0x3f9b bytes) 2024-09-21T14:59:42.166 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:59:42.165+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:59:42.170 INFO:teuthology.orchestra.run.smithi032.stdout:2001 punch from 0x170e72c to 0x17128e2, (0x41b6 bytes) 2024-09-21T14:59:42.170 INFO:teuthology.orchestra.run.smithi032.stdout:2002 punch from 0x360dc41 to 0x361d083, (0xf442 bytes) 2024-09-21T14:59:42.170 INFO:teuthology.orchestra.run.smithi032.stdout:2003 read 0xb1a24b5 thru 0xb1a2615 (0x161 bytes) 2024-09-21T14:59:42.170 INFO:teuthology.orchestra.run.smithi032.stdout:2005 read 0x71e0f50 thru 0x71ea7f5 (0x98a6 bytes) 2024-09-21T14:59:42.170 INFO:teuthology.orchestra.run.smithi032.stdout:2006 write 0x3466a55 thru 0x346fad0 (0x907c bytes) 2024-09-21T14:59:42.178 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:42.178 INFO:teuthology.orchestra.run.smithi032.stdout:2009 read 0x75c28c9 thru 0x75cca21 (0xa159 bytes) 2024-09-21T14:59:42.179 INFO:teuthology.orchestra.run.smithi032.stdout:2011 punch from 0xa4f8a09 to 0xa4fc615, (0x3c0c bytes) 2024-09-21T14:59:42.179 INFO:teuthology.orchestra.run.smithi032.stdout:2012 writesame 0x9caafbb thru 0x9cb142f data_size 0x1(0x6475 bytes) 2024-09-21T14:59:42.201 INFO:teuthology.orchestra.run.smithi032.stdout:2013 read 0xb829373 thru 0xb82d29b (0x3f29 bytes) 2024-09-21T14:59:42.207 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:42.207 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:59:42.208 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T14:59:42.208 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T14:59:42.208 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:42.216 INFO:teuthology.orchestra.run.smithi032.stdout:2014 write 0xcd9b8b2 thru 0xcd9bd28 (0x477 bytes) 2024-09-21T14:59:42.217 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T14:59:42.223 INFO:teuthology.orchestra.run.smithi032.stdout:2017 writesame 0x1fb2c87 thru 0x1fc1c27 data_size 0x1(0xefa1 bytes) 2024-09-21T14:59:42.266 INFO:teuthology.orchestra.run.smithi032.stdout:2018 trunc from 0xcd9bd29 to 0x97d92f3 2024-09-21T14:59:42.267 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:59:42.265+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:59:42.291 INFO:teuthology.orchestra.run.smithi032.stdout:2019 read 0x4edd6ff thru 0x4ee7cf7 (0xa5f9 bytes) 2024-09-21T14:59:42.298 INFO:teuthology.orchestra.run.smithi032.stdout:2020 trunc from 0x97d92f3 to 0x671f0c4 2024-09-21T14:59:42.340 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:42.340 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T14:59:42.340 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T14:59:42.340 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T14:59:42.340 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.341 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:15130 3.2 3:47f1bbdb:::rbd_data.11361bed1f78.0000000000000067:head [sparse-read 0~147456] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-21T14:59:42.341 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:40.288284+0000", 2024-09-21T14:59:42.341 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 62.051218568000003, 2024-09-21T14:59:42.341 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1142935039999999, 2024-09-21T14:59:42.341 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:42.341 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:42.341 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:42.341 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:42.341 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:42.341 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 15130 2024-09-21T14:59:42.341 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.341 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:42.341 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.341 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:42.342 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288284+0000", 2024-09-21T14:59:42.342 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:42.342 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.342 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.342 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:42.342 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288284+0000", 2024-09-21T14:59:42.342 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:42.342 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.342 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.342 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:42.342 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288284+0000", 2024-09-21T14:59:42.342 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:42.342 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.342 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.343 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:42.343 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288288+0000", 2024-09-21T14:59:42.343 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5829999999999998e-06 2024-09-21T14:59:42.343 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.343 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.343 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:42.343 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288297+0000", 2024-09-21T14:59:42.343 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0669999999999998e-06 2024-09-21T14:59:42.343 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.343 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.343 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:42.343 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288305+0000", 2024-09-21T14:59:42.343 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8860000000000004e-06 2024-09-21T14:59:42.344 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.344 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.344 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:42.344 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288417+0000", 2024-09-21T14:59:42.344 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000112952 2024-09-21T14:59:42.344 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.344 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.344 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T14:59:42.344 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288428+0000", 2024-09-21T14:59:42.344 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0631e-05 2024-09-21T14:59:42.344 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.344 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.344 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:42.345 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.401876+0000", 2024-09-21T14:59:42.345 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1134479610000001 2024-09-21T14:59:42.345 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.345 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.345 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:42.345 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.402076+0000", 2024-09-21T14:59:42.345 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00019944200000000001 2024-09-21T14:59:42.345 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.345 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.345 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:42.345 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.402578+0000", 2024-09-21T14:59:42.345 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00050242899999999998 2024-09-21T14:59:42.345 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:42.346 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:42.346 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:42.346 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.346 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.346 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45708 3.2 3:415db91a:::rbd_data.1136ecbe5879.0000000000000000:28 [read 140288~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:42.346 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141801+0000", 2024-09-21T14:59:42.346 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 32.197702374000002, 2024-09-21T14:59:42.346 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034274200999999997, 2024-09-21T14:59:42.346 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:42.346 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:42.346 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:42.346 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:42.346 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:42.346 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45708 2024-09-21T14:59:42.347 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.347 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:42.347 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.347 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:42.347 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141801+0000", 2024-09-21T14:59:42.347 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:42.347 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.347 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.347 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:42.347 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141801+0000", 2024-09-21T14:59:42.347 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:42.347 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.347 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.348 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:42.348 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141800+0000", 2024-09-21T14:59:42.348 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:42.348 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.348 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.348 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:42.348 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141803+0000", 2024-09-21T14:59:42.348 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4210000000000002e-06 2024-09-21T14:59:42.348 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.348 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.348 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:42.348 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141812+0000", 2024-09-21T14:59:42.348 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.9800000000000004e-06 2024-09-21T14:59:42.348 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.348 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.349 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:42.349 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141816+0000", 2024-09-21T14:59:42.349 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4769999999999997e-06 2024-09-21T14:59:42.349 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.349 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.349 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:42.349 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.175865+0000", 2024-09-21T14:59:42.349 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034048875999999999 2024-09-21T14:59:42.349 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.349 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.349 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:42.349 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.175943+0000", 2024-09-21T14:59:42.349 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7464000000000002e-05 2024-09-21T14:59:42.349 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.349 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.350 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:42.350 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176075+0000", 2024-09-21T14:59:42.350 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013230800000000001 2024-09-21T14:59:42.350 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:42.350 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:42.350 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:42.350 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.350 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.350 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45709 3.2 3:415db91a:::rbd_data.1136ecbe5879.0000000000000000:28 [read 143360~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:42.350 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141820+0000", 2024-09-21T14:59:42.350 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 32.197682526999998, 2024-09-21T14:59:42.350 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034537536000000001, 2024-09-21T14:59:42.350 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:42.350 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:42.350 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:42.351 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:42.351 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:42.351 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45709 2024-09-21T14:59:42.351 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.351 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:42.351 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.351 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:42.351 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141820+0000", 2024-09-21T14:59:42.351 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:42.351 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.351 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.351 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:42.351 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141820+0000", 2024-09-21T14:59:42.351 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:42.352 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.352 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.352 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:42.352 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141820+0000", 2024-09-21T14:59:42.352 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:42.352 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.352 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.352 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:42.352 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141823+0000", 2024-09-21T14:59:42.352 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5110000000000002e-06 2024-09-21T14:59:42.352 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.352 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.352 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:42.352 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141829+0000", 2024-09-21T14:59:42.352 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0850000000000002e-06 2024-09-21T14:59:42.353 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.353 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.353 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:42.353 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141833+0000", 2024-09-21T14:59:42.353 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7620000000000003e-06 2024-09-21T14:59:42.353 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.353 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.353 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:42.353 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176103+0000", 2024-09-21T14:59:42.353 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034269933000000002 2024-09-21T14:59:42.353 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.353 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.353 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:42.353 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176161+0000", 2024-09-21T14:59:42.354 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7658999999999998e-05 2024-09-21T14:59:42.354 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.354 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.354 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:42.354 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176358+0000", 2024-09-21T14:59:42.354 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000196937 2024-09-21T14:59:42.354 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:42.354 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:42.354 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:42.354 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.354 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.354 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45710 3.2 3:415db91a:::rbd_data.1136ecbe5879.0000000000000000:28 [read 146432~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:42.354 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141838+0000", 2024-09-21T14:59:42.355 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 32.197665364000002, 2024-09-21T14:59:42.355 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034953674999999997, 2024-09-21T14:59:42.355 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:42.355 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:42.355 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:42.355 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:42.355 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:42.355 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45710 2024-09-21T14:59:42.355 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.355 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:42.355 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.355 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:42.355 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141838+0000", 2024-09-21T14:59:42.355 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:42.356 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.356 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.356 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:42.356 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141838+0000", 2024-09-21T14:59:42.356 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:42.356 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.356 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.356 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:42.356 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141837+0000", 2024-09-21T14:59:42.356 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:42.356 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.356 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.356 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:42.357 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141843+0000", 2024-09-21T14:59:42.357 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1619999999999996e-06 2024-09-21T14:59:42.357 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.357 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.357 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:42.357 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141850+0000", 2024-09-21T14:59:42.357 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4799999999999998e-06 2024-09-21T14:59:42.357 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.357 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.357 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:42.357 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141853+0000", 2024-09-21T14:59:42.357 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3050000000000001e-06 2024-09-21T14:59:42.357 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.357 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.358 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:42.358 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176490+0000", 2024-09-21T14:59:42.358 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034636413999999997 2024-09-21T14:59:42.358 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.358 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.358 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:42.358 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176606+0000", 2024-09-21T14:59:42.358 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000115841 2024-09-21T14:59:42.358 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.358 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.358 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:42.358 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176791+0000", 2024-09-21T14:59:42.358 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00018581500000000001 2024-09-21T14:59:42.358 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:42.359 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:42.359 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:42.359 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.359 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.359 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45711 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:42.359 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141856+0000", 2024-09-21T14:59:42.359 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 32.197647027000002, 2024-09-21T14:59:42.359 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034986308000000001, 2024-09-21T14:59:42.359 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:42.359 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:42.359 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:42.359 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:42.359 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:42.359 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45711 2024-09-21T14:59:42.360 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.360 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:42.360 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.360 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:42.360 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141856+0000", 2024-09-21T14:59:42.360 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:42.360 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.360 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.360 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:42.360 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141856+0000", 2024-09-21T14:59:42.360 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:42.360 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.360 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.360 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:42.360 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141856+0000", 2024-09-21T14:59:42.361 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:42.361 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.361 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.361 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:42.361 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141862+0000", 2024-09-21T14:59:42.361 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.567e-06 2024-09-21T14:59:42.361 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.361 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.361 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:42.361 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141868+0000", 2024-09-21T14:59:42.361 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9569999999999999e-06 2024-09-21T14:59:42.361 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.361 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.361 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:42.361 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141871+0000", 2024-09-21T14:59:42.362 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.146e-06 2024-09-21T14:59:42.362 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.362 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.362 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:42.362 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176831+0000", 2024-09-21T14:59:42.362 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034959935999999997 2024-09-21T14:59:42.362 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.362 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.362 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:42.362 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176842+0000", 2024-09-21T14:59:42.362 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1008e-05 2024-09-21T14:59:42.362 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:42.362 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:42.362 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:42.363 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.363 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.363 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45712 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:42.363 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141874+0000", 2024-09-21T14:59:42.363 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 32.197629132000003, 2024-09-21T14:59:42.363 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034993511999999997, 2024-09-21T14:59:42.363 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:42.363 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:42.363 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:42.363 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:42.363 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:42.363 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45712 2024-09-21T14:59:42.363 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.363 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:42.363 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.364 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:42.364 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141874+0000", 2024-09-21T14:59:42.364 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:42.364 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.364 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.364 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:42.364 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141874+0000", 2024-09-21T14:59:42.364 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:42.364 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.364 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.364 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:42.364 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141874+0000", 2024-09-21T14:59:42.364 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:42.364 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.364 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.365 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:42.365 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141875+0000", 2024-09-21T14:59:42.365 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5430000000000001e-06 2024-09-21T14:59:42.365 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.365 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.365 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:42.365 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141878+0000", 2024-09-21T14:59:42.365 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1420000000000002e-06 2024-09-21T14:59:42.365 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.365 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.365 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:42.365 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141881+0000", 2024-09-21T14:59:42.365 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.807e-06 2024-09-21T14:59:42.365 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.365 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.366 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:42.366 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176860+0000", 2024-09-21T14:59:42.366 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034979243 2024-09-21T14:59:42.366 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.366 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.366 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:42.366 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176867+0000", 2024-09-21T14:59:42.366 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0169999999999999e-06 2024-09-21T14:59:42.366 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:42.366 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:42.366 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:42.366 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.366 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.366 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45713 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:42.367 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141883+0000", 2024-09-21T14:59:42.367 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 32.197619557000003, 2024-09-21T14:59:42.367 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035010516999999998, 2024-09-21T14:59:42.367 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:42.367 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:42.367 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:42.367 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:42.367 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:42.367 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45713 2024-09-21T14:59:42.367 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.367 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:42.367 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.367 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:42.367 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141883+0000", 2024-09-21T14:59:42.367 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:42.368 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.368 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.368 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:42.368 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141883+0000", 2024-09-21T14:59:42.368 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:42.368 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.368 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.368 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:42.368 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141883+0000", 2024-09-21T14:59:42.368 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:42.368 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.368 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.368 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:42.368 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141885+0000", 2024-09-21T14:59:42.368 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4649999999999999e-06 2024-09-21T14:59:42.368 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.369 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.369 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:42.369 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141888+0000", 2024-09-21T14:59:42.369 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3890000000000001e-06 2024-09-21T14:59:42.369 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.369 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.369 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:42.369 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141891+0000", 2024-09-21T14:59:42.369 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9239999999999999e-06 2024-09-21T14:59:42.369 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.369 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.369 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:42.369 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176887+0000", 2024-09-21T14:59:42.369 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034996370999999998 2024-09-21T14:59:42.369 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.370 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.370 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:42.370 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176894+0000", 2024-09-21T14:59:42.370 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6069999999999996e-06 2024-09-21T14:59:42.370 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:42.370 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:42.370 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:42.370 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.370 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.370 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45714 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:42.370 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141893+0000", 2024-09-21T14:59:42.370 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 32.197609642000003, 2024-09-21T14:59:42.370 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035048215000000001, 2024-09-21T14:59:42.371 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:42.371 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:42.371 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:42.371 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:42.371 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:42.371 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45714 2024-09-21T14:59:42.371 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.371 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:42.371 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.371 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:42.371 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141893+0000", 2024-09-21T14:59:42.371 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:42.371 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.371 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.371 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:42.372 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141893+0000", 2024-09-21T14:59:42.372 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:42.372 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.372 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.372 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:42.372 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141893+0000", 2024-09-21T14:59:42.372 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:42.372 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.372 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.372 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:42.372 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141895+0000", 2024-09-21T14:59:42.372 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.703e-06 2024-09-21T14:59:42.372 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.372 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.372 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:42.372 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141900+0000", 2024-09-21T14:59:42.373 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.242e-06 2024-09-21T14:59:42.373 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.373 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.373 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:42.373 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141903+0000", 2024-09-21T14:59:42.373 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.117e-06 2024-09-21T14:59:42.373 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.373 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.373 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:42.373 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176930+0000", 2024-09-21T14:59:42.373 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035026750000000002 2024-09-21T14:59:42.373 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.373 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.373 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:42.374 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176942+0000", 2024-09-21T14:59:42.374 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1668000000000001e-05 2024-09-21T14:59:42.374 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:42.374 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:42.374 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:42.374 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.374 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.374 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45715 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:42.374 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141906+0000", 2024-09-21T14:59:42.374 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 32.197597483999999, 2024-09-21T14:59:42.374 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035066057999999997, 2024-09-21T14:59:42.374 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:42.374 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:42.374 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:42.374 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:42.375 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:42.375 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45715 2024-09-21T14:59:42.375 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.375 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:42.375 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.375 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:42.375 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141906+0000", 2024-09-21T14:59:42.375 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:42.375 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.375 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.375 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:42.375 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141906+0000", 2024-09-21T14:59:42.375 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:42.375 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.376 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.376 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:42.376 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141905+0000", 2024-09-21T14:59:42.376 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:42.376 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.376 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.376 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:42.376 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141907+0000", 2024-09-21T14:59:42.376 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5400000000000001e-06 2024-09-21T14:59:42.376 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.376 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.376 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:42.376 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141912+0000", 2024-09-21T14:59:42.376 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.502e-06 2024-09-21T14:59:42.376 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.377 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.377 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:42.377 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141915+0000", 2024-09-21T14:59:42.377 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0170000000000001e-06 2024-09-21T14:59:42.377 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.377 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.377 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:42.377 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176964+0000", 2024-09-21T14:59:42.377 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035048587999999999 2024-09-21T14:59:42.377 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.377 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.377 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:42.377 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176972+0000", 2024-09-21T14:59:42.377 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6580000000000002e-06 2024-09-21T14:59:42.377 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:42.378 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:42.378 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:42.378 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.378 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.378 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45716 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:42.378 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141918+0000", 2024-09-21T14:59:42.378 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 32.197585308000001, 2024-09-21T14:59:42.378 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035086223, 2024-09-21T14:59:42.378 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:42.378 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:42.378 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:42.378 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:42.378 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:42.378 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45716 2024-09-21T14:59:42.379 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.379 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:42.379 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.379 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:42.379 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141918+0000", 2024-09-21T14:59:42.379 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:42.379 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.379 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.379 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:42.379 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141918+0000", 2024-09-21T14:59:42.379 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:42.379 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.379 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.379 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:42.380 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141917+0000", 2024-09-21T14:59:42.380 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:42.380 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.380 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.380 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:42.380 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141919+0000", 2024-09-21T14:59:42.380 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5680000000000001e-06 2024-09-21T14:59:42.380 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.380 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.380 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:42.380 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141922+0000", 2024-09-21T14:59:42.380 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2789999999999999e-06 2024-09-21T14:59:42.380 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.380 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.380 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:42.381 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141925+0000", 2024-09-21T14:59:42.381 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9469999999999999e-06 2024-09-21T14:59:42.381 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.381 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.381 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:42.381 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176997+0000", 2024-09-21T14:59:42.381 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035071366999999999 2024-09-21T14:59:42.381 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.381 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.381 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:42.381 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177004+0000", 2024-09-21T14:59:42.381 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3309999999999998e-06 2024-09-21T14:59:42.381 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:42.381 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:42.382 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:42.382 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.382 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.382 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45717 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:42.382 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141928+0000", 2024-09-21T14:59:42.382 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 32.197575362000002, 2024-09-21T14:59:42.382 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035102573999999998, 2024-09-21T14:59:42.382 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:42.382 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:42.382 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:42.382 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:42.382 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:42.382 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45717 2024-09-21T14:59:42.382 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.383 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:42.383 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.383 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:42.383 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141928+0000", 2024-09-21T14:59:42.383 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:42.383 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.383 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.383 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:42.383 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141928+0000", 2024-09-21T14:59:42.383 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:42.383 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.383 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.383 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:42.383 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141927+0000", 2024-09-21T14:59:42.383 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T14:59:42.384 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.384 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.384 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:42.384 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141929+0000", 2024-09-21T14:59:42.384 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.533e-06 2024-09-21T14:59:42.384 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.384 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.384 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:42.384 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141934+0000", 2024-09-21T14:59:42.384 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1660000000000002e-06 2024-09-21T14:59:42.384 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.384 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.384 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:42.384 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141937+0000", 2024-09-21T14:59:42.384 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0309999999999999e-06 2024-09-21T14:59:42.385 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.385 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.385 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:42.385 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177023+0000", 2024-09-21T14:59:42.385 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035085595999999997 2024-09-21T14:59:42.385 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.385 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.385 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:42.385 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177030+0000", 2024-09-21T14:59:42.385 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.486e-06 2024-09-21T14:59:42.385 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:42.385 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:42.385 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:42.385 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.385 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.385 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45718 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:42.385 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141940+0000", 2024-09-21T14:59:42.386 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 32.197563504000001, 2024-09-21T14:59:42.386 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035124956999999998, 2024-09-21T14:59:42.386 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:42.386 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:42.386 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:42.386 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:42.386 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:42.386 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45718 2024-09-21T14:59:42.386 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.386 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:42.386 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.386 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:42.386 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141940+0000", 2024-09-21T14:59:42.386 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:42.386 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.387 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.387 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:42.387 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141940+0000", 2024-09-21T14:59:42.387 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:42.387 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.387 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.387 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:42.387 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141939+0000", 2024-09-21T14:59:42.387 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:42.387 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.387 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.387 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:42.387 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141941+0000", 2024-09-21T14:59:42.387 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.502e-06 2024-09-21T14:59:42.387 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.387 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.388 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:42.388 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141946+0000", 2024-09-21T14:59:42.388 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3449999999999996e-06 2024-09-21T14:59:42.388 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.388 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.388 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:42.388 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141949+0000", 2024-09-21T14:59:42.388 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9749999999999999e-06 2024-09-21T14:59:42.388 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.388 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.388 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:42.388 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177057+0000", 2024-09-21T14:59:42.388 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035108385999999998 2024-09-21T14:59:42.388 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.388 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.388 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:42.389 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177064+0000", 2024-09-21T14:59:42.389 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0099999999999998e-06 2024-09-21T14:59:42.389 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:42.389 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:42.389 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:42.389 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.389 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.389 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45719 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:42.389 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141952+0000", 2024-09-21T14:59:42.389 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 32.197551230999998, 2024-09-21T14:59:42.389 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035139108000000002, 2024-09-21T14:59:42.389 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:42.389 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:42.389 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:42.389 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:42.389 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:42.390 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45719 2024-09-21T14:59:42.390 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.390 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:42.390 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.390 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:42.390 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141952+0000", 2024-09-21T14:59:42.390 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:42.390 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.390 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.390 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:42.390 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141952+0000", 2024-09-21T14:59:42.390 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:42.390 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.390 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.390 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:42.391 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141952+0000", 2024-09-21T14:59:42.391 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:42.391 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.391 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.391 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:42.391 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141953+0000", 2024-09-21T14:59:42.391 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.486e-06 2024-09-21T14:59:42.391 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.391 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.391 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:42.391 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141959+0000", 2024-09-21T14:59:42.391 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4940000000000003e-06 2024-09-21T14:59:42.391 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.391 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.391 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:42.391 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141963+0000", 2024-09-21T14:59:42.392 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3540000000000002e-06 2024-09-21T14:59:42.392 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.392 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.392 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:42.392 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177084+0000", 2024-09-21T14:59:42.392 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035120983000000001 2024-09-21T14:59:42.392 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.392 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.392 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:42.392 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177091+0000", 2024-09-21T14:59:42.392 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0350000000000004e-06 2024-09-21T14:59:42.392 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:42.392 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:42.392 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:42.392 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.393 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.393 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45720 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:42.393 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141965+0000", 2024-09-21T14:59:42.393 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 32.197537764000003, 2024-09-21T14:59:42.393 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035155268000000003, 2024-09-21T14:59:42.393 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:42.393 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:42.393 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:42.393 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:42.393 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:42.393 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45720 2024-09-21T14:59:42.393 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.393 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:42.393 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.393 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:42.394 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141965+0000", 2024-09-21T14:59:42.394 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:42.394 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.394 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.394 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:42.394 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141965+0000", 2024-09-21T14:59:42.394 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:42.394 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.394 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.394 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:42.394 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141965+0000", 2024-09-21T14:59:42.394 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:42.394 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.395 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.395 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:42.395 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141966+0000", 2024-09-21T14:59:42.395 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.46e-06 2024-09-21T14:59:42.395 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.395 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.395 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:42.395 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141972+0000", 2024-09-21T14:59:42.395 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4210000000000003e-06 2024-09-21T14:59:42.395 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.395 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.395 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:42.395 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141975+0000", 2024-09-21T14:59:42.395 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.948e-06 2024-09-21T14:59:42.396 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.396 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.396 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:42.396 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177113+0000", 2024-09-21T14:59:42.396 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035137855000000003 2024-09-21T14:59:42.396 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.396 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.396 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:42.396 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177121+0000", 2024-09-21T14:59:42.396 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8269999999999998e-06 2024-09-21T14:59:42.396 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:42.396 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:42.396 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:42.396 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.397 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.397 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45721 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:42.397 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141977+0000", 2024-09-21T14:59:42.397 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 32.197525741, 2024-09-21T14:59:42.397 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035169984000000001, 2024-09-21T14:59:42.397 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:42.397 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:42.397 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:42.397 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:42.397 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:42.397 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45721 2024-09-21T14:59:42.397 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.397 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:42.397 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.397 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:42.398 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141977+0000", 2024-09-21T14:59:42.398 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:42.398 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.398 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.398 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:42.398 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141977+0000", 2024-09-21T14:59:42.398 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:42.398 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.398 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.398 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:42.398 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141977+0000", 2024-09-21T14:59:42.398 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:42.398 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.398 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.398 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:42.399 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141979+0000", 2024-09-21T14:59:42.399 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.511e-06 2024-09-21T14:59:42.399 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.399 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.399 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:42.399 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141982+0000", 2024-09-21T14:59:42.399 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4180000000000001e-06 2024-09-21T14:59:42.399 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.399 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.399 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:42.399 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141985+0000", 2024-09-21T14:59:42.399 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9450000000000002e-06 2024-09-21T14:59:42.399 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.399 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.399 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:42.400 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177139+0000", 2024-09-21T14:59:42.400 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035154409999999997 2024-09-21T14:59:42.400 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.400 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.400 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:42.400 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177147+0000", 2024-09-21T14:59:42.400 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9570000000000002e-06 2024-09-21T14:59:42.400 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:42.400 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:42.400 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:42.400 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.400 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.400 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45722 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:42.400 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141987+0000", 2024-09-21T14:59:42.400 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 32.197515770000003, 2024-09-21T14:59:42.401 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035188492000000002, 2024-09-21T14:59:42.401 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:42.401 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:42.401 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:42.401 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:42.401 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:42.401 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45722 2024-09-21T14:59:42.401 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.401 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:42.401 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.401 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:42.401 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141987+0000", 2024-09-21T14:59:42.401 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:42.401 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.401 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.401 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:42.402 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141987+0000", 2024-09-21T14:59:42.402 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:42.402 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.402 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.402 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:42.402 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141987+0000", 2024-09-21T14:59:42.402 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:42.402 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.402 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.402 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:42.402 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141988+0000", 2024-09-21T14:59:42.402 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.463e-06 2024-09-21T14:59:42.402 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.402 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.403 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:42.403 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141994+0000", 2024-09-21T14:59:42.403 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.361e-06 2024-09-21T14:59:42.403 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.403 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.403 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:42.403 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141997+0000", 2024-09-21T14:59:42.403 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0359999999999998e-06 2024-09-21T14:59:42.403 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.403 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.403 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:42.403 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177169+0000", 2024-09-21T14:59:42.403 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035171651999999998 2024-09-21T14:59:42.404 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.404 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.404 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:42.404 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177176+0000", 2024-09-21T14:59:42.404 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2189999999999998e-06 2024-09-21T14:59:42.404 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:42.404 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:42.404 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:42.404 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.404 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.404 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45723 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:42.404 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141999+0000", 2024-09-21T14:59:42.404 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 32.197503603000001, 2024-09-21T14:59:42.404 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035197701999999997, 2024-09-21T14:59:42.404 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:42.405 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:42.405 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:42.405 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:42.405 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:42.405 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45723 2024-09-21T14:59:42.405 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.405 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:42.405 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.405 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:42.405 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141999+0000", 2024-09-21T14:59:42.405 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:42.405 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.405 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.405 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:42.405 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141999+0000", 2024-09-21T14:59:42.405 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:42.406 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.406 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.406 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:42.406 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141999+0000", 2024-09-21T14:59:42.406 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:42.406 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.406 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.406 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:42.406 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142003+0000", 2024-09-21T14:59:42.406 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5049999999999999e-06 2024-09-21T14:59:42.406 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.406 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.406 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:42.406 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142008+0000", 2024-09-21T14:59:42.407 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6579999999999999e-06 2024-09-21T14:59:42.407 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.407 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.407 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:42.407 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142011+0000", 2024-09-21T14:59:42.407 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0529999999999998e-06 2024-09-21T14:59:42.407 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.407 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.407 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:42.407 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177194+0000", 2024-09-21T14:59:42.407 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035182221999999999 2024-09-21T14:59:42.407 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.407 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.407 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:42.407 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177197+0000", 2024-09-21T14:59:42.407 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5030000000000002e-06 2024-09-21T14:59:42.408 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:42.408 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:42.408 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:42.408 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.408 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.408 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45724 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:42.408 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.142015+0000", 2024-09-21T14:59:42.408 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 32.197487635999998, 2024-09-21T14:59:42.408 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035195345000000003, 2024-09-21T14:59:42.408 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:42.408 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:42.408 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:42.408 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:42.408 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:42.408 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45724 2024-09-21T14:59:42.409 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.409 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:42.409 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.409 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:42.409 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142015+0000", 2024-09-21T14:59:42.409 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:42.409 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.409 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.409 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:42.409 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142015+0000", 2024-09-21T14:59:42.409 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:42.409 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.409 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.409 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:42.409 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142015+0000", 2024-09-21T14:59:42.409 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:42.410 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.410 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.410 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:42.410 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142017+0000", 2024-09-21T14:59:42.410 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5540000000000001e-06 2024-09-21T14:59:42.410 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.410 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.410 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:42.410 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142022+0000", 2024-09-21T14:59:42.410 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4229999999999996e-06 2024-09-21T14:59:42.410 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.410 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.410 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:42.410 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142025+0000", 2024-09-21T14:59:42.410 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.056e-06 2024-09-21T14:59:42.410 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.411 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.411 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:42.411 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177207+0000", 2024-09-21T14:59:42.411 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035182089 2024-09-21T14:59:42.411 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.411 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.411 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:42.411 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177211+0000", 2024-09-21T14:59:42.411 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4649999999999999e-06 2024-09-21T14:59:42.411 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:42.411 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:42.411 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:42.411 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.411 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.411 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45725 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:42.412 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.142028+0000", 2024-09-21T14:59:42.412 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 32.197475357000002, 2024-09-21T14:59:42.412 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035195731000000001, 2024-09-21T14:59:42.412 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:42.412 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:42.412 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:42.412 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:42.412 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:42.412 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45725 2024-09-21T14:59:42.412 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.412 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:42.412 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.412 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:42.412 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142028+0000", 2024-09-21T14:59:42.412 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:42.412 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.413 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.413 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:42.413 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142028+0000", 2024-09-21T14:59:42.413 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:42.413 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.413 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.413 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:42.413 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142027+0000", 2024-09-21T14:59:42.413 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:42.413 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.413 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.413 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:42.413 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142029+0000", 2024-09-21T14:59:42.413 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5269999999999999e-06 2024-09-21T14:59:42.413 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.413 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.414 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:42.414 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142032+0000", 2024-09-21T14:59:42.414 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3249999999999999e-06 2024-09-21T14:59:42.414 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.414 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.414 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:42.414 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142035+0000", 2024-09-21T14:59:42.414 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.897e-06 2024-09-21T14:59:42.414 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.414 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.414 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:42.414 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177220+0000", 2024-09-21T14:59:42.414 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035184871 2024-09-21T14:59:42.414 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.415 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.415 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:42.415 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177223+0000", 2024-09-21T14:59:42.415 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3680000000000002e-06 2024-09-21T14:59:42.415 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:42.415 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:42.415 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:42.415 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.415 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.415 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45726 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:42.415 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.142037+0000", 2024-09-21T14:59:42.415 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 32.197465641999997, 2024-09-21T14:59:42.415 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035199273000000003, 2024-09-21T14:59:42.415 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:42.416 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:42.416 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:42.416 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:42.416 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:42.416 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45726 2024-09-21T14:59:42.416 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.416 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:42.416 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.416 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:42.416 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142037+0000", 2024-09-21T14:59:42.416 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:42.416 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.416 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.416 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:42.416 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142037+0000", 2024-09-21T14:59:42.416 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:42.417 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.417 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.417 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:42.417 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142037+0000", 2024-09-21T14:59:42.417 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:42.417 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.417 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.417 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:42.417 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142038+0000", 2024-09-21T14:59:42.417 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.344e-06 2024-09-21T14:59:42.417 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.417 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.417 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:42.417 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142044+0000", 2024-09-21T14:59:42.417 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1930000000000001e-06 2024-09-21T14:59:42.417 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.418 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.418 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:42.418 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142047+0000", 2024-09-21T14:59:42.418 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0280000000000001e-06 2024-09-21T14:59:42.418 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.418 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.418 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:42.418 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177233+0000", 2024-09-21T14:59:42.418 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035186394000000003 2024-09-21T14:59:42.418 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.418 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.418 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:42.418 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177237+0000", 2024-09-21T14:59:42.418 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5530000000000001e-06 2024-09-21T14:59:42.418 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:42.418 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:42.418 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:42.419 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:42.419 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:42.419 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:42.420 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T14:59:42.421 INFO:teuthology.orchestra.run.smithi032.stdout:2021 writesame 0xe8bc90a thru 0xe8c222f data_size 0x1(0x5926 bytes) 2024-09-21T14:59:42.421 INFO:teuthology.orchestra.run.smithi032.stdout:2022 write 0x9d28665 thru 0x9d367cb (0xe167 bytes) 2024-09-21T14:59:42.421 INFO:teuthology.orchestra.run.smithi032.stdout:2023 trunc from 0xe8c2230 to 0xeaf2ec5 2024-09-21T14:59:42.421 INFO:teuthology.orchestra.run.smithi032.stdout:2024 write 0xce2fbc0 thru 0xce32a3f (0x2e80 bytes) 2024-09-21T14:59:42.421 INFO:teuthology.orchestra.run.smithi032.stdout:2026 write 0x570890b thru 0x5712b69 (0xa25f bytes) 2024-09-21T14:59:42.421 INFO:teuthology.orchestra.run.smithi032.stdout:2027 writesame 0x55d6a75 thru 0x55e071a data_size 0x1(0x9ca6 bytes) 2024-09-21T14:59:42.421 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:59:42.421+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:59:42.424 INFO:teuthology.orchestra.run.smithi032.stdout:2028 write 0xacf42bc thru 0xacfa7d3 (0x6518 bytes) 2024-09-21T14:59:42.430 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:42.430 INFO:teuthology.orchestra.run.smithi032.stdout:2030 read 0x757cec4 thru 0x7580df9 (0x3f36 bytes) 2024-09-21T14:59:42.431 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:42.431 INFO:teuthology.orchestra.run.smithi032.stdout:2032 read 0x4c1cff0 thru 0x4c1fa3f (0x2a50 bytes) 2024-09-21T14:59:42.431 INFO:teuthology.orchestra.run.smithi032.stdout:2033 read 0x34d5030 thru 0x34e0479 (0xb44a bytes) 2024-09-21T14:59:42.444 INFO:teuthology.orchestra.run.smithi032.stdout:2035 trunc from 0xeaf2ec5 to 0x98ac082 2024-09-21T14:59:42.469 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:42.469 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:42.469 INFO:teuthology.orchestra.run.smithi032.stdout:2040 read 0x752288b thru 0x7523834 (0xfaa bytes) 2024-09-21T14:59:42.470 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:42.470 INFO:teuthology.orchestra.run.smithi032.stdout:2042 write 0x7b3991f thru 0x7b42329 (0x8a0b bytes) 2024-09-21T14:59:42.477 INFO:teuthology.orchestra.run.smithi032.stdout:2044 read 0x46a297a thru 0x46ad803 (0xae8a bytes) 2024-09-21T14:59:42.477 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:42.478 INFO:teuthology.orchestra.run.smithi032.stdout:2046 trunc from 0x98ac082 to 0x9ba971b 2024-09-21T14:59:42.490 INFO:teuthology.orchestra.run.smithi032.stdout:2047 read 0x420af82 thru 0x42194b5 (0xe534 bytes) 2024-09-21T14:59:42.491 INFO:teuthology.orchestra.run.smithi032.stdout:2048 read 0x9aacecd thru 0x9ab1428 (0x455c bytes) 2024-09-21T14:59:42.491 INFO:teuthology.orchestra.run.smithi032.stdout:2049 write 0x39d7d0b thru 0x39e776a (0xfa60 bytes) 2024-09-21T14:59:42.498 INFO:teuthology.orchestra.run.smithi032.stdout:2051 read 0x36af071 thru 0x36be55f (0xf4ef bytes) 2024-09-21T14:59:42.498 INFO:teuthology.orchestra.run.smithi032.stdout:2052 read 0x56dde40 thru 0x56ecf1a (0xf0db bytes) 2024-09-21T14:59:42.499 INFO:teuthology.orchestra.run.smithi032.stdout:2053 read 0xfedbb4 thru 0xfee300 (0x74d bytes) 2024-09-21T14:59:42.499 INFO:teuthology.orchestra.run.smithi032.stdout:2054 write 0x657708e thru 0x6578a39 (0x19ac bytes) 2024-09-21T14:59:42.505 INFO:teuthology.orchestra.run.smithi032.stdout:2055 read 0x2759a68 thru 0x2763d15 (0xa2ae bytes) 2024-09-21T14:59:42.521 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:59:42.515+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:59:42.525 INFO:teuthology.orchestra.run.smithi032.stdout:2056 write 0xce56fd0 thru 0xce612ba (0xa2eb bytes) 2024-09-21T14:59:42.532 INFO:teuthology.orchestra.run.smithi032.stdout:2057 write 0xc826ff3 thru 0xc834fbf (0xdfcd bytes) 2024-09-21T14:59:42.539 INFO:teuthology.orchestra.run.smithi032.stdout:2058 punch from 0x8b8ec32 to 0x8b90ef5, (0x22c3 bytes) 2024-09-21T14:59:42.539 INFO:teuthology.orchestra.run.smithi032.stdout:2059 read 0x5fb244d thru 0x5fb9bee (0x77a2 bytes) 2024-09-21T14:59:42.540 INFO:teuthology.orchestra.run.smithi032.stdout:2062 read 0xcd087a3 thru 0xcd1720b (0xea69 bytes) 2024-09-21T14:59:42.541 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:42.541 INFO:teuthology.orchestra.run.smithi032.stdout:2065 clone 19 order 20 su 8192 sc 12 2024-09-21T14:59:42.545 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:42.545 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:59:42.545 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T14:59:42.545 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:42.555 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T14:59:42.622 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:59:42.619+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:59:42.706 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:42.707 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:59:42.707 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T14:59:42.707 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T14:59:42.707 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:42.717 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T14:59:42.723 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:59:42.721+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:59:42.823 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:42.823 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T14:59:42.823 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T14:59:42.823 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T14:59:42.823 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.823 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4281.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-21T14:59:42.823 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:09.324416+0000", 2024-09-21T14:59:42.823 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 93.498958900999995, 2024-09-21T14:59:42.823 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.117837947, 2024-09-21T14:59:42.823 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:42.823 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:42.824 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:42.824 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4281", 2024-09-21T14:59:42.824 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/1518520303", 2024-09-21T14:59:42.824 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1 2024-09-21T14:59:42.824 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.824 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:42.824 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.824 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:42.824 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324416+0000", 2024-09-21T14:59:42.824 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:42.824 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.824 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.824 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:42.824 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324416+0000", 2024-09-21T14:59:42.825 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:42.825 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.825 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.825 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:42.825 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324414+0000", 2024-09-21T14:59:42.825 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T14:59:42.825 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.825 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.825 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:42.825 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324429+0000", 2024-09-21T14:59:42.825 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5637e-05 2024-09-21T14:59:42.825 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.825 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.825 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:42.826 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324448+0000", 2024-09-21T14:59:42.826 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9042000000000002e-05 2024-09-21T14:59:42.826 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.826 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.826 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:42.826 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324465+0000", 2024-09-21T14:59:42.826 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6767999999999999e-05 2024-09-21T14:59:42.826 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.826 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.826 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:42.826 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324571+0000", 2024-09-21T14:59:42.826 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000105714 2024-09-21T14:59:42.826 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.827 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.827 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op must wait for map", 2024-09-21T14:59:42.827 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324587+0000", 2024-09-21T14:59:42.827 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5753999999999999e-05 2024-09-21T14:59:42.827 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.827 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.827 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:42.827 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.325864+0000", 2024-09-21T14:59:42.827 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0012773350000000001 2024-09-21T14:59:42.827 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.827 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.827 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T14:59:42.827 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.325878+0000", 2024-09-21T14:59:42.828 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4498000000000001e-05 2024-09-21T14:59:42.828 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.828 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.828 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:42.828 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.439461+0000", 2024-09-21T14:59:42.828 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.11358212500000001 2024-09-21T14:59:42.828 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.828 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.828 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:42.828 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.439848+0000", 2024-09-21T14:59:42.828 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00038747800000000002 2024-09-21T14:59:42.828 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.828 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.828 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:59:42.829 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.440175+0000", 2024-09-21T14:59:42.829 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00032721900000000001 2024-09-21T14:59:42.829 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.829 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.829 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:42.829 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.441770+0000", 2024-09-21T14:59:42.829 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0015949409999999999 2024-09-21T14:59:42.829 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.829 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.829 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:42.829 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.442086+0000", 2024-09-21T14:59:42.829 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00031586299999999999 2024-09-21T14:59:42.829 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.829 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.829 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:42.830 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.442195+0000", 2024-09-21T14:59:42.830 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000108791 2024-09-21T14:59:42.830 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.830 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.830 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:42.830 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.442254+0000", 2024-09-21T14:59:42.830 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9038e-05 2024-09-21T14:59:42.830 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:42.830 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:42.830 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:42.830 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.830 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.830 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4281.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-21T14:59:42.830 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:09.447828+0000", 2024-09-21T14:59:42.830 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 93.375546900000003, 2024-09-21T14:59:42.831 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.32904393700000001, 2024-09-21T14:59:42.831 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:42.831 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:42.831 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:42.831 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4281", 2024-09-21T14:59:42.831 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/1518520303", 2024-09-21T14:59:42.831 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 2 2024-09-21T14:59:42.831 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.831 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:42.831 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.831 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:42.831 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447828+0000", 2024-09-21T14:59:42.831 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:42.832 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.832 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.832 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:42.832 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447828+0000", 2024-09-21T14:59:42.832 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:42.832 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.832 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.832 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:42.832 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447825+0000", 2024-09-21T14:59:42.832 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T14:59:42.832 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.832 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.832 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:42.832 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447841+0000", 2024-09-21T14:59:42.832 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5359999999999999e-05 2024-09-21T14:59:42.832 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.833 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.833 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:42.833 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447875+0000", 2024-09-21T14:59:42.833 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4388999999999999e-05 2024-09-21T14:59:42.833 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.833 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.833 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:42.833 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447903+0000", 2024-09-21T14:59:42.833 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7373e-05 2024-09-21T14:59:42.833 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.833 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.833 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:42.833 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447996+0000", 2024-09-21T14:59:42.833 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2954999999999993e-05 2024-09-21T14:59:42.834 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.834 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.834 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T14:59:42.834 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.448011+0000", 2024-09-21T14:59:42.834 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5038999999999999e-05 2024-09-21T14:59:42.834 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.834 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.834 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:42.834 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.776501+0000", 2024-09-21T14:59:42.834 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.32849012500000002 2024-09-21T14:59:42.834 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.834 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.834 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:42.835 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.776872+0000", 2024-09-21T14:59:42.835 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00037119600000000002 2024-09-21T14:59:42.835 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:42.835 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:42.835 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:42.835 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.835 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.835 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:752 3.14 3:28fc02e5:::rbd_data.113693316f36.0000000000000010:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-21T14:59:42.835 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:26.625579+0000", 2024-09-21T14:59:42.835 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 76.197796045000004, 2024-09-21T14:59:42.835 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042965426000000001, 2024-09-21T14:59:42.835 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:42.835 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:42.835 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:42.836 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:42.836 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:42.836 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 752 2024-09-21T14:59:42.836 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.836 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:42.836 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.836 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:42.836 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625579+0000", 2024-09-21T14:59:42.836 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:42.836 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.836 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.836 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:42.836 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625579+0000", 2024-09-21T14:59:42.836 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:42.836 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.837 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.837 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:42.837 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625577+0000", 2024-09-21T14:59:42.837 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T14:59:42.837 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.837 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.837 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:42.837 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625588+0000", 2024-09-21T14:59:42.837 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1555e-05 2024-09-21T14:59:42.837 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.837 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.837 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:42.837 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625610+0000", 2024-09-21T14:59:42.837 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2107999999999999e-05 2024-09-21T14:59:42.837 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.837 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.838 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:42.838 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625621+0000", 2024-09-21T14:59:42.838 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0762e-05 2024-09-21T14:59:42.838 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.838 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.838 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:42.838 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625692+0000", 2024-09-21T14:59:42.838 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.1203000000000003e-05 2024-09-21T14:59:42.838 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.838 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.838 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:42.838 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625872+0000", 2024-09-21T14:59:42.838 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017962800000000001 2024-09-21T14:59:42.838 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.838 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.839 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:59:42.839 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.633188+0000", 2024-09-21T14:59:42.839 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.007316014 2024-09-21T14:59:42.839 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.839 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.839 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:42.839 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.668357+0000", 2024-09-21T14:59:42.839 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035168933999999999 2024-09-21T14:59:42.839 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.839 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.839 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:42.839 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.668418+0000", 2024-09-21T14:59:42.839 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1282e-05 2024-09-21T14:59:42.840 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.840 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.840 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:42.840 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.668486+0000", 2024-09-21T14:59:42.840 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8468999999999998e-05 2024-09-21T14:59:42.840 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.840 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.840 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:42.840 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.668544+0000", 2024-09-21T14:59:42.840 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7785000000000003e-05 2024-09-21T14:59:42.840 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:42.840 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:42.840 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:42.840 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.840 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.841 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22738 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 49152~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:42.841 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141049+0000", 2024-09-21T14:59:42.841 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 52.682325902000002, 2024-09-21T14:59:42.841 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035428754999999999, 2024-09-21T14:59:42.841 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:42.841 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:42.841 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:42.841 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:42.841 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:42.841 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22738 2024-09-21T14:59:42.841 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.841 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:42.841 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.841 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:42.841 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141049+0000", 2024-09-21T14:59:42.842 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:42.842 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.842 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.842 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:42.842 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141049+0000", 2024-09-21T14:59:42.842 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:42.842 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.842 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.842 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:42.842 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141049+0000", 2024-09-21T14:59:42.842 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:42.842 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.843 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.843 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:42.843 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141051+0000", 2024-09-21T14:59:42.843 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4980000000000001e-06 2024-09-21T14:59:42.843 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.843 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.843 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:42.843 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141057+0000", 2024-09-21T14:59:42.843 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8030000000000002e-06 2024-09-21T14:59:42.843 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.843 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.843 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:42.843 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141066+0000", 2024-09-21T14:59:42.843 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1160000000000005e-06 2024-09-21T14:59:42.844 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.844 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.844 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:42.844 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176264+0000", 2024-09-21T14:59:42.844 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035197580999999999 2024-09-21T14:59:42.844 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.844 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.844 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:42.844 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176327+0000", 2024-09-21T14:59:42.844 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3237000000000005e-05 2024-09-21T14:59:42.844 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.844 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.844 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:42.844 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176478+0000", 2024-09-21T14:59:42.844 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000150851 2024-09-21T14:59:42.845 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:42.845 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:42.845 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:42.845 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.845 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.845 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22739 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 114688~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:42.845 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141070+0000", 2024-09-21T14:59:42.845 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 52.68230466, 2024-09-21T14:59:42.845 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035679094000000001, 2024-09-21T14:59:42.845 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:42.845 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:42.845 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:42.845 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:42.845 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:42.845 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22739 2024-09-21T14:59:42.846 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.846 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:42.846 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.846 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:42.846 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141070+0000", 2024-09-21T14:59:42.846 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:42.846 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.846 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.846 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:42.846 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141070+0000", 2024-09-21T14:59:42.846 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:42.846 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.846 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.846 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:42.846 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141070+0000", 2024-09-21T14:59:42.847 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:42.847 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.847 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.847 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:42.847 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141072+0000", 2024-09-21T14:59:42.847 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.379e-06 2024-09-21T14:59:42.847 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.847 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.847 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:42.847 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141078+0000", 2024-09-21T14:59:42.847 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6710000000000004e-06 2024-09-21T14:59:42.847 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.847 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.847 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:42.848 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141083+0000", 2024-09-21T14:59:42.848 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7249999999999997e-06 2024-09-21T14:59:42.848 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.848 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.848 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:42.848 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176506+0000", 2024-09-21T14:59:42.848 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035423076999999997 2024-09-21T14:59:42.848 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.848 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.848 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:42.848 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176582+0000", 2024-09-21T14:59:42.848 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6305999999999998e-05 2024-09-21T14:59:42.848 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.848 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.848 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:42.848 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176749+0000", 2024-09-21T14:59:42.849 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000167262 2024-09-21T14:59:42.849 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:42.849 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:42.849 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:42.849 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.849 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.849 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22740 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 180224~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:42.849 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141090+0000", 2024-09-21T14:59:42.849 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 52.682285256, 2024-09-21T14:59:42.849 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036129535999999997, 2024-09-21T14:59:42.849 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:42.849 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:42.849 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:42.849 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:42.849 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:42.849 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22740 2024-09-21T14:59:42.850 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.850 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:42.850 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.850 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:42.850 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141090+0000", 2024-09-21T14:59:42.850 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:42.850 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.850 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.850 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:42.850 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141090+0000", 2024-09-21T14:59:42.850 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:42.850 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.850 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.850 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:42.850 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141089+0000", 2024-09-21T14:59:42.851 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:42.851 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.851 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.851 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:42.851 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141092+0000", 2024-09-21T14:59:42.851 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7709999999999998e-06 2024-09-21T14:59:42.851 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.851 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.851 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:42.851 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141098+0000", 2024-09-21T14:59:42.851 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.015e-06 2024-09-21T14:59:42.851 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.851 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.851 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:42.851 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141103+0000", 2024-09-21T14:59:42.852 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8629999999999999e-06 2024-09-21T14:59:42.852 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.852 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.852 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:42.852 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176778+0000", 2024-09-21T14:59:42.852 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035674984 2024-09-21T14:59:42.852 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.852 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.852 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:42.852 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176851+0000", 2024-09-21T14:59:42.852 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2861000000000006e-05 2024-09-21T14:59:42.852 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.852 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.852 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:42.852 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177219+0000", 2024-09-21T14:59:42.853 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00036845999999999998 2024-09-21T14:59:42.853 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:42.853 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:42.853 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:42.853 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.853 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.853 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22741 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 245760~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:42.853 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141107+0000", 2024-09-21T14:59:42.853 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 52.682267684999999, 2024-09-21T14:59:42.853 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036407729, 2024-09-21T14:59:42.853 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:42.853 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:42.853 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:42.854 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:42.854 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:42.854 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22741 2024-09-21T14:59:42.854 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.854 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:42.854 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.854 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:42.854 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141107+0000", 2024-09-21T14:59:42.854 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:42.854 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.854 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.854 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:42.854 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141107+0000", 2024-09-21T14:59:42.855 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:42.855 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.855 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.855 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:42.855 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141107+0000", 2024-09-21T14:59:42.855 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:42.855 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.855 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.855 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:42.855 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141109+0000", 2024-09-21T14:59:42.855 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6529999999999998e-06 2024-09-21T14:59:42.855 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.855 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.855 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:42.855 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141115+0000", 2024-09-21T14:59:42.856 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9649999999999996e-06 2024-09-21T14:59:42.856 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.856 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.856 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:42.856 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141121+0000", 2024-09-21T14:59:42.856 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1209999999999998e-06 2024-09-21T14:59:42.856 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.856 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.856 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:42.856 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177242+0000", 2024-09-21T14:59:42.856 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036121706000000003 2024-09-21T14:59:42.856 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.856 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.856 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:42.856 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177301+0000", 2024-09-21T14:59:42.857 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9008000000000002e-05 2024-09-21T14:59:42.857 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.857 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.857 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:42.857 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177515+0000", 2024-09-21T14:59:42.857 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00021362 2024-09-21T14:59:42.857 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:42.857 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:42.857 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:42.857 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.857 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.857 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22742 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 311296~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:42.857 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141126+0000", 2024-09-21T14:59:42.857 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 52.682249331000001, 2024-09-21T14:59:42.858 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036653537, 2024-09-21T14:59:42.858 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:42.858 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:42.858 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:42.858 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:42.858 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:42.858 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22742 2024-09-21T14:59:42.858 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.858 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:42.858 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.858 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:42.858 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141126+0000", 2024-09-21T14:59:42.858 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:42.858 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.859 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.859 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:42.859 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141126+0000", 2024-09-21T14:59:42.859 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:42.859 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.859 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.859 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:42.859 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141125+0000", 2024-09-21T14:59:42.859 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:42.859 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.859 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.859 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:42.859 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141128+0000", 2024-09-21T14:59:42.859 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.886e-06 2024-09-21T14:59:42.860 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.860 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.860 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:42.860 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141137+0000", 2024-09-21T14:59:42.860 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4399999999999994e-06 2024-09-21T14:59:42.860 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.860 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.860 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:42.860 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141143+0000", 2024-09-21T14:59:42.860 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1410000000000004e-06 2024-09-21T14:59:42.860 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.860 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.860 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:42.860 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177543+0000", 2024-09-21T14:59:42.860 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036400544999999999 2024-09-21T14:59:42.860 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.861 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.861 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:42.861 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177605+0000", 2024-09-21T14:59:42.861 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2206999999999996e-05 2024-09-21T14:59:42.861 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.861 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.861 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:42.861 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177779+0000", 2024-09-21T14:59:42.861 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017368099999999999 2024-09-21T14:59:42.861 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:42.861 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:42.861 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:42.861 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.861 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.861 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22743 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 376832~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:42.862 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141147+0000", 2024-09-21T14:59:42.862 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 52.682227847, 2024-09-21T14:59:42.862 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036915158000000003, 2024-09-21T14:59:42.862 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:42.862 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:42.862 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:42.862 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:42.862 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:42.862 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22743 2024-09-21T14:59:42.862 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.862 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:42.862 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.862 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:42.862 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141147+0000", 2024-09-21T14:59:42.862 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:42.863 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.863 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.863 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:42.863 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141147+0000", 2024-09-21T14:59:42.863 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:42.863 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.863 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.863 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:42.863 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141147+0000", 2024-09-21T14:59:42.863 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:42.863 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.863 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.864 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:42.864 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141149+0000", 2024-09-21T14:59:42.864 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.79e-06 2024-09-21T14:59:42.864 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.864 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.864 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:42.864 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141155+0000", 2024-09-21T14:59:42.864 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6699999999999999e-06 2024-09-21T14:59:42.864 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.864 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.864 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:42.864 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141160+0000", 2024-09-21T14:59:42.864 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5759999999999999e-06 2024-09-21T14:59:42.864 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.865 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.865 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:42.865 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177817+0000", 2024-09-21T14:59:42.865 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036657404999999997 2024-09-21T14:59:42.865 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.865 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.865 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:42.865 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177892+0000", 2024-09-21T14:59:42.865 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4602000000000002e-05 2024-09-21T14:59:42.865 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.865 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.865 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:42.865 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178062+0000", 2024-09-21T14:59:42.866 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000170464 2024-09-21T14:59:42.866 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:42.866 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:42.866 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:42.866 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.866 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.866 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22744 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 442368~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:42.866 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141164+0000", 2024-09-21T14:59:42.866 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 52.682210714, 2024-09-21T14:59:42.866 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037120806999999999, 2024-09-21T14:59:42.866 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:42.866 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:42.866 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:42.866 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:42.867 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:42.867 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22744 2024-09-21T14:59:42.867 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.867 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:42.867 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.867 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:42.867 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141164+0000", 2024-09-21T14:59:42.867 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:42.867 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.867 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.867 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:42.867 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141164+0000", 2024-09-21T14:59:42.867 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:42.867 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.867 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.868 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:42.868 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141164+0000", 2024-09-21T14:59:42.868 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:42.868 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.868 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.868 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:42.868 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141167+0000", 2024-09-21T14:59:42.868 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7159999999999999e-06 2024-09-21T14:59:42.868 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.868 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.868 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:42.868 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141173+0000", 2024-09-21T14:59:42.868 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.071e-06 2024-09-21T14:59:42.868 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.868 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.869 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:42.869 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141177+0000", 2024-09-21T14:59:42.869 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3610000000000003e-06 2024-09-21T14:59:42.869 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.869 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.869 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:42.869 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178083+0000", 2024-09-21T14:59:42.869 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036906405000000003 2024-09-21T14:59:42.869 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.869 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.869 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:42.869 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178143+0000", 2024-09-21T14:59:42.869 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0056999999999998e-05 2024-09-21T14:59:42.869 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.869 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.869 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:42.870 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178285+0000", 2024-09-21T14:59:42.870 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000141555 2024-09-21T14:59:42.870 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:42.870 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:42.870 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:42.870 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.870 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.870 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22745 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 507904~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:42.870 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141182+0000", 2024-09-21T14:59:42.870 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 52.682192393999998, 2024-09-21T14:59:42.870 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037332331000000003, 2024-09-21T14:59:42.870 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:42.870 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:42.870 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:42.870 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:42.871 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:42.871 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22745 2024-09-21T14:59:42.871 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.871 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:42.871 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.871 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:42.871 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141182+0000", 2024-09-21T14:59:42.871 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:42.871 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.871 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.871 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:42.871 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141182+0000", 2024-09-21T14:59:42.871 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:42.871 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.872 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.872 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:42.872 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141182+0000", 2024-09-21T14:59:42.872 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:42.872 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.872 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.872 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:42.872 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141185+0000", 2024-09-21T14:59:42.872 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.464e-06 2024-09-21T14:59:42.872 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.872 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.872 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:42.872 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141200+0000", 2024-09-21T14:59:42.872 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5886999999999999e-05 2024-09-21T14:59:42.872 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.873 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.873 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:42.873 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141571+0000", 2024-09-21T14:59:42.873 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000370909 2024-09-21T14:59:42.873 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.873 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.873 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:42.873 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178318+0000", 2024-09-21T14:59:42.873 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036746244999999997 2024-09-21T14:59:42.873 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.873 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.873 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:42.873 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178367+0000", 2024-09-21T14:59:42.873 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9317000000000002e-05 2024-09-21T14:59:42.873 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.874 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.874 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:42.874 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178515+0000", 2024-09-21T14:59:42.874 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014784600000000001 2024-09-21T14:59:42.874 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:42.874 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:42.874 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:42.874 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.874 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.874 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22746 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 573440~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:42.874 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141599+0000", 2024-09-21T14:59:42.874 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 52.681775762000001, 2024-09-21T14:59:42.874 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037402249999999998, 2024-09-21T14:59:42.874 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:42.874 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:42.875 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:42.875 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:42.875 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:42.875 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22746 2024-09-21T14:59:42.875 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.875 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:42.875 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.875 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:42.875 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141599+0000", 2024-09-21T14:59:42.875 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:42.875 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.875 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.875 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:42.875 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141599+0000", 2024-09-21T14:59:42.876 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:42.876 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.876 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.876 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:42.876 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141598+0000", 2024-09-21T14:59:42.876 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:59:42.876 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.876 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.876 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:42.876 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141637+0000", 2024-09-21T14:59:42.876 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.9119999999999998e-05 2024-09-21T14:59:42.876 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.876 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.876 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:42.876 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141647+0000", 2024-09-21T14:59:42.877 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5570000000000001e-06 2024-09-21T14:59:42.877 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.877 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.877 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:42.877 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141654+0000", 2024-09-21T14:59:42.877 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2049999999999996e-06 2024-09-21T14:59:42.877 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.877 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.877 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:42.877 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178553+0000", 2024-09-21T14:59:42.877 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036898528 2024-09-21T14:59:42.877 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.877 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.877 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:42.877 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178625+0000", 2024-09-21T14:59:42.878 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2669999999999994e-05 2024-09-21T14:59:42.878 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.878 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.878 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:42.878 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179001+0000", 2024-09-21T14:59:42.878 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000375977 2024-09-21T14:59:42.878 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:42.878 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:42.878 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:42.878 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.878 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.878 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22747 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 638976~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:42.878 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141660+0000", 2024-09-21T14:59:42.878 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 52.681714620000001, 2024-09-21T14:59:42.879 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037617352999999999, 2024-09-21T14:59:42.879 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:42.879 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:42.879 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:42.879 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:42.879 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:42.879 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22747 2024-09-21T14:59:42.879 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.879 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:42.879 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.879 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:42.879 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141660+0000", 2024-09-21T14:59:42.879 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:42.879 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.879 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.880 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:42.880 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141660+0000", 2024-09-21T14:59:42.880 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:42.880 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.880 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.880 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:42.880 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141660+0000", 2024-09-21T14:59:42.880 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:42.880 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.880 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.880 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:42.880 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141663+0000", 2024-09-21T14:59:42.880 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.569e-06 2024-09-21T14:59:42.880 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.880 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.880 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:42.881 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141670+0000", 2024-09-21T14:59:42.881 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5719999999999999e-06 2024-09-21T14:59:42.881 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.881 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.881 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:42.881 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141675+0000", 2024-09-21T14:59:42.881 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2889999999999997e-06 2024-09-21T14:59:42.881 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.881 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.881 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:42.881 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179028+0000", 2024-09-21T14:59:42.881 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037353381999999997 2024-09-21T14:59:42.881 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.881 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.881 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:42.882 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179099+0000", 2024-09-21T14:59:42.882 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0259000000000006e-05 2024-09-21T14:59:42.882 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.882 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.882 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:42.882 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179278+0000", 2024-09-21T14:59:42.882 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000178908 2024-09-21T14:59:42.882 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:42.882 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:42.882 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:42.882 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.882 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.882 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22748 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 704512~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:42.882 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141680+0000", 2024-09-21T14:59:42.883 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 52.681694982000003, 2024-09-21T14:59:42.883 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037806166000000002, 2024-09-21T14:59:42.883 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:42.883 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:42.883 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:42.883 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:42.883 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:42.883 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22748 2024-09-21T14:59:42.883 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.883 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:42.883 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.883 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:42.883 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141680+0000", 2024-09-21T14:59:42.883 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:42.883 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.884 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.884 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:42.884 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141680+0000", 2024-09-21T14:59:42.884 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:42.884 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.884 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.884 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:42.884 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141679+0000", 2024-09-21T14:59:42.884 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:42.884 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.884 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.884 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:42.884 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141683+0000", 2024-09-21T14:59:42.884 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3230000000000002e-06 2024-09-21T14:59:42.885 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.885 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.885 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:42.885 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141690+0000", 2024-09-21T14:59:42.885 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8680000000000001e-06 2024-09-21T14:59:42.885 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.885 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.885 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:42.885 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141694+0000", 2024-09-21T14:59:42.885 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6770000000000003e-06 2024-09-21T14:59:42.885 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.885 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.885 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:42.885 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179300+0000", 2024-09-21T14:59:42.885 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037605567999999999 2024-09-21T14:59:42.886 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.886 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.886 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:42.886 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179355+0000", 2024-09-21T14:59:42.886 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4771000000000001e-05 2024-09-21T14:59:42.886 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.886 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.886 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:42.886 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179486+0000", 2024-09-21T14:59:42.886 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013141699999999999 2024-09-21T14:59:42.886 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:42.886 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:42.886 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:42.886 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.887 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.887 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22749 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 770048~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:42.887 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141698+0000", 2024-09-21T14:59:42.887 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 52.681676609999997, 2024-09-21T14:59:42.887 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037967813000000003, 2024-09-21T14:59:42.887 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:42.887 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:42.887 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:42.887 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:42.887 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:42.887 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22749 2024-09-21T14:59:42.887 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.887 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:42.887 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.888 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:42.888 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141698+0000", 2024-09-21T14:59:42.888 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:42.888 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.888 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.888 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:42.888 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141698+0000", 2024-09-21T14:59:42.888 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:42.888 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.888 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.888 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:42.888 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141698+0000", 2024-09-21T14:59:42.888 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:42.889 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.889 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.889 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:42.889 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141701+0000", 2024-09-21T14:59:42.889 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8399999999999999e-06 2024-09-21T14:59:42.889 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.889 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.889 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:42.889 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141706+0000", 2024-09-21T14:59:42.889 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4979999999999997e-06 2024-09-21T14:59:42.889 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.889 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.889 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:42.890 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141711+0000", 2024-09-21T14:59:42.890 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3850000000000004e-06 2024-09-21T14:59:42.890 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.890 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.890 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:42.890 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179504+0000", 2024-09-21T14:59:42.890 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037793514 2024-09-21T14:59:42.890 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.890 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.890 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:42.890 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179546+0000", 2024-09-21T14:59:42.890 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1467999999999997e-05 2024-09-21T14:59:42.890 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.891 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.891 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:42.891 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179666+0000", 2024-09-21T14:59:42.891 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000120499 2024-09-21T14:59:42.891 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:42.891 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:42.891 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:42.891 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.891 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.891 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22750 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 835584~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:42.891 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141715+0000", 2024-09-21T14:59:42.891 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 52.681660309000002, 2024-09-21T14:59:42.891 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038173179000000002, 2024-09-21T14:59:42.892 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:42.892 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:42.892 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:42.892 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:42.892 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:42.892 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22750 2024-09-21T14:59:42.892 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.892 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:42.892 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.892 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:42.892 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141715+0000", 2024-09-21T14:59:42.892 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:42.892 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.893 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.893 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:42.893 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141715+0000", 2024-09-21T14:59:42.893 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:42.893 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.893 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.893 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:42.893 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141714+0000", 2024-09-21T14:59:42.893 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:42.893 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.893 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.893 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:42.893 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141716+0000", 2024-09-21T14:59:42.894 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1730000000000002e-06 2024-09-21T14:59:42.894 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.894 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.894 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:42.894 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141722+0000", 2024-09-21T14:59:42.894 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.417e-06 2024-09-21T14:59:42.894 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.894 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.894 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:42.894 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141726+0000", 2024-09-21T14:59:42.894 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4830000000000001e-06 2024-09-21T14:59:42.894 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.895 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.895 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:42.895 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179683+0000", 2024-09-21T14:59:42.895 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037956977000000003 2024-09-21T14:59:42.895 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.895 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.895 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:42.895 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179751+0000", 2024-09-21T14:59:42.895 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7762000000000006e-05 2024-09-21T14:59:42.895 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.895 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.895 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:42.895 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179888+0000", 2024-09-21T14:59:42.895 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000136635 2024-09-21T14:59:42.895 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:42.896 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:42.896 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:42.896 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.896 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.896 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22751 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 901120~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:42.896 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141730+0000", 2024-09-21T14:59:42.896 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 52.681644509999998, 2024-09-21T14:59:42.896 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038357011000000003, 2024-09-21T14:59:42.896 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:42.896 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:42.896 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:42.896 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:42.896 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:42.896 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22751 2024-09-21T14:59:42.896 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.896 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:42.897 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.897 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:42.897 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141730+0000", 2024-09-21T14:59:42.897 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:42.897 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.897 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.897 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:42.897 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141730+0000", 2024-09-21T14:59:42.897 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:42.897 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.897 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.897 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:42.897 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141730+0000", 2024-09-21T14:59:42.897 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:42.897 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.897 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.898 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:42.898 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141737+0000", 2024-09-21T14:59:42.898 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.64e-06 2024-09-21T14:59:42.898 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.898 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.898 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:42.898 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141743+0000", 2024-09-21T14:59:42.898 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8130000000000001e-06 2024-09-21T14:59:42.898 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.898 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.898 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:42.898 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141747+0000", 2024-09-21T14:59:42.898 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4920000000000004e-06 2024-09-21T14:59:42.898 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.898 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.899 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:42.899 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179918+0000", 2024-09-21T14:59:42.899 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038170822 2024-09-21T14:59:42.899 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.899 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.899 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:42.899 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179960+0000", 2024-09-21T14:59:42.899 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1999000000000002e-05 2024-09-21T14:59:42.899 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.899 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.899 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:42.899 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180087+0000", 2024-09-21T14:59:42.899 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012752799999999999 2024-09-21T14:59:42.899 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:42.899 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:42.900 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:42.900 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.900 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.900 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22752 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 966656~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:42.900 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141751+0000", 2024-09-21T14:59:42.900 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 52.681624190000001, 2024-09-21T14:59:42.900 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038553084000000001, 2024-09-21T14:59:42.900 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:42.900 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:42.900 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:42.900 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:42.900 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:42.900 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22752 2024-09-21T14:59:42.900 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.900 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:42.901 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.901 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:42.901 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141751+0000", 2024-09-21T14:59:42.901 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:42.901 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.901 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.901 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:42.901 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141751+0000", 2024-09-21T14:59:42.901 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:42.901 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.901 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.901 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:42.901 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141750+0000", 2024-09-21T14:59:42.901 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:42.901 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.901 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.902 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:42.902 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141753+0000", 2024-09-21T14:59:42.902 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5900000000000002e-06 2024-09-21T14:59:42.902 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.902 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.902 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:42.902 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141759+0000", 2024-09-21T14:59:42.902 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.947e-06 2024-09-21T14:59:42.902 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.902 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.902 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:42.902 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141764+0000", 2024-09-21T14:59:42.902 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8319999999999996e-06 2024-09-21T14:59:42.902 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.902 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.903 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:42.903 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180108+0000", 2024-09-21T14:59:42.903 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038344095000000002 2024-09-21T14:59:42.903 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.903 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.903 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:42.903 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180168+0000", 2024-09-21T14:59:42.903 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0047000000000003e-05 2024-09-21T14:59:42.903 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.903 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.903 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:42.903 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180304+0000", 2024-09-21T14:59:42.903 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013592500000000001 2024-09-21T14:59:42.903 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:42.904 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:42.904 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:42.904 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.904 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.904 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22753 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 1032192~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:42.904 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141768+0000", 2024-09-21T14:59:42.904 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 52.681607315999997, 2024-09-21T14:59:42.904 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038818810000000002, 2024-09-21T14:59:42.904 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:42.904 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:42.904 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:42.904 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:42.904 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:42.904 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22753 2024-09-21T14:59:42.904 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.905 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:42.905 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.905 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:42.905 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141768+0000", 2024-09-21T14:59:42.905 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:42.905 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.905 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.905 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:42.905 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141768+0000", 2024-09-21T14:59:42.905 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:42.905 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.905 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.905 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:42.905 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141767+0000", 2024-09-21T14:59:42.906 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:42.906 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.906 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.906 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:42.906 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141770+0000", 2024-09-21T14:59:42.906 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3329999999999999e-06 2024-09-21T14:59:42.906 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.906 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.906 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:42.906 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141775+0000", 2024-09-21T14:59:42.906 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2669999999999997e-06 2024-09-21T14:59:42.906 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.906 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.906 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:42.906 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141779+0000", 2024-09-21T14:59:42.907 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2259999999999999e-06 2024-09-21T14:59:42.907 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.907 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.907 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:42.907 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180325+0000", 2024-09-21T14:59:42.907 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038546428000000001 2024-09-21T14:59:42.907 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.907 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.907 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:42.907 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180401+0000", 2024-09-21T14:59:42.907 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5437000000000003e-05 2024-09-21T14:59:42.907 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.907 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.907 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:42.907 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180586+0000", 2024-09-21T14:59:42.908 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000185429 2024-09-21T14:59:42.908 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:42.908 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:42.908 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:42.908 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.908 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.908 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:73793 3.11 3:8c8a112d:::rbd_data.1136f4939602.0000000000000003:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-21T14:59:42.908 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:42.225633+0000", 2024-09-21T14:59:42.908 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.59774140200000003, 2024-09-21T14:59:42.908 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036337102000000003, 2024-09-21T14:59:42.908 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:42.908 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:42.908 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:42.908 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:42.908 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:42.909 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 73793 2024-09-21T14:59:42.909 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.909 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:42.909 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.909 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:42.909 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.225633+0000", 2024-09-21T14:59:42.909 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:42.909 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.909 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.909 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:42.909 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.225633+0000", 2024-09-21T14:59:42.909 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:42.909 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.909 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.909 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:42.909 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.225630+0000", 2024-09-21T14:59:42.910 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999967 2024-09-21T14:59:42.910 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.910 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.910 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:42.910 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.225649+0000", 2024-09-21T14:59:42.910 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9377000000000001e-05 2024-09-21T14:59:42.910 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.910 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.910 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:42.910 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.225689+0000", 2024-09-21T14:59:42.910 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.9879999999999998e-05 2024-09-21T14:59:42.910 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.910 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.910 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:42.910 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.225706+0000", 2024-09-21T14:59:42.911 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6537000000000001e-05 2024-09-21T14:59:42.911 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.911 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.911 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:42.911 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.225772+0000", 2024-09-21T14:59:42.911 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5894000000000003e-05 2024-09-21T14:59:42.911 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.911 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.911 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:42.911 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.225846+0000", 2024-09-21T14:59:42.911 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4165000000000006e-05 2024-09-21T14:59:42.911 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.911 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.911 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 1", 2024-09-21T14:59:42.911 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.231400+0000", 2024-09-21T14:59:42.912 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0055538269999999999 2024-09-21T14:59:42.912 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.912 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.912 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:42.912 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.261709+0000", 2024-09-21T14:59:42.912 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.030309750999999999 2024-09-21T14:59:42.912 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.912 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.912 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:42.912 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.261738+0000", 2024-09-21T14:59:42.912 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8569999999999999e-05 2024-09-21T14:59:42.912 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.912 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.912 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:42.912 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.261839+0000", 2024-09-21T14:59:42.912 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010118 2024-09-21T14:59:42.913 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:42.913 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:42.913 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:42.913 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.261971+0000", 2024-09-21T14:59:42.913 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000131414 2024-09-21T14:59:42.913 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:42.913 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:42.913 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:42.913 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:42.913 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:42.913 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:42.914 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T14:59:42.915 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:59:42.913+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:59:42.998 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T14:59:43.033 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:59:43.033 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:59:43.033 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T14:59:43.033 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:59:43.043 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T14:59:43.044 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:59:43.045+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:59:43.116 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:59:43.117+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:59:43.144 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:59:43.144 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:59:43.145 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T14:59:43.145 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T14:59:43.145 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:59:43.148 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T14:59:43.154 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T14:59:43.155 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T14:59:43.177 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T14:59:43.197 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T14:59:43.217 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:59:43.211+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:59:43.218 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T14:59:43.240 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-21T14:59:43.259 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:43.259 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:43.267 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T14:59:43.270 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T14:59:43.293 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T14:59:43.314 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T14:59:43.316 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:59:43.316 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:59:43.316 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:59:43.316 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:59:43.316 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:59:43.316 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:59:43.316 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:59:43.316 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:59:43.318 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:59:43.317+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:59:43.377 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:43.377 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:43.388 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T14:59:43.420 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:59:43.415+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:59:43.491 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:43.491 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:43.501 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T14:59:43.521 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:59:43.521+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:59:43.622 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:59:43.621+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:59:43.656 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:43.656 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:43.666 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T14:59:43.722 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:59:43.719+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:59:43.775 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:43.775 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:43.785 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T14:59:43.826 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:59:43.823+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:59:43.924 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:59:43.925+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:59:43.936 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:43.936 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:43.946 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T14:59:44.024 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:59:44.025+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:59:44.058 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:44.058 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:44.068 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T14:59:44.125 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:59:44.125+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:59:44.178 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:44.178 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:44.188 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T14:59:44.225 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:59:44.219+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:59:44.299 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:44.299 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:44.310 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T14:59:44.326 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:59:44.325+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:59:44.328 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 5, 6, 7, 4] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T14:59:44.328 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T14:59:44.328 INFO:tasks.thrashosds.thrasher:inject_pause on osd.2 2024-09-21T14:59:44.328 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-21T14:59:44.329 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-21T14:59:44.329 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok config set filestore_inject_stall 3 2024-09-21T14:59:44.345 INFO:teuthology.orchestra.run.smithi032.stderr:nodeep-scrub is set 2024-09-21T14:59:44.427 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:59:44.423+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:59:44.478 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:44.478 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "" 2024-09-21T14:59:44.478 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:44.504 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:44.504 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:44.517 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T14:59:44.529 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:59:44.529+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:59:44.628 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:59:44.623+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:59:44.665 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:44.665 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:44.676 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T14:59:44.729 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:59:44.723+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:59:44.819 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:44.819 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:44.829 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:59:44.829+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:59:44.832 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T14:59:44.930 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:59:44.929+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:59:44.993 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:44.993 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:45.003 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T14:59:45.048 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:59:45.049+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:59:45.106 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:45.106 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:45.116 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T14:59:45.132 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:59:45.133+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:59:45.223 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:45.223 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:45.233 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:59:45.233+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:59:45.234 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T14:59:45.293 INFO:teuthology.orchestra.run.smithi032.stdout:checking clone #17, image image_client.0-clone17 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent18 2024-09-21T14:59:45.335 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T14:59:45.338 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:59:45.333+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:59:45.395 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:45.396 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:45.406 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T14:59:45.435 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:59:45.431+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:59:45.445 INFO:teuthology.orchestra.run.smithi032.stdout:2066 read 0x57d5131 thru 0x57e491d (0xf7ed bytes) 2024-09-21T14:59:45.447 INFO:teuthology.orchestra.run.smithi032.stdout:2067 write 0x6efa654 thru 0x6f097de (0xf18b bytes) 2024-09-21T14:59:45.471 INFO:teuthology.orchestra.run.smithi032.stdout:2069 write 0xa75cf1d thru 0xa761b37 (0x4c1b bytes) 2024-09-21T14:59:45.478 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T14:59:45.479 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:45.479 INFO:teuthology.orchestra.run.smithi032.stdout:2071 writesame 0x911c5a5 thru 0x9121913 data_size 0x1f(0x536f bytes) 2024-09-21T14:59:45.483 INFO:teuthology.orchestra.run.smithi032.stdout:2072 punch from 0xe2e525 to 0xe31772, (0x324d bytes) 2024-09-21T14:59:45.494 INFO:teuthology.orchestra.run.smithi032.stdout:2077 trunc from 0xce612bb to 0x5924d69 2024-09-21T14:59:45.503 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T14:59:45.532 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:45.532 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:45.534 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T14:59:45.536 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:59:45.531+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:59:45.538 INFO:teuthology.orchestra.run.smithi032.stdout:2079 read 0x3fa76b9 thru 0x3fab9c7 (0x430f bytes) 2024-09-21T14:59:45.538 INFO:teuthology.orchestra.run.smithi032.stdout:2080 punch from 0x5203772 to 0x520f0c1, (0xb94f bytes) 2024-09-21T14:59:45.538 INFO:teuthology.orchestra.run.smithi032.stdout:2082 read 0x207de9d thru 0x208685f (0x89c3 bytes) 2024-09-21T14:59:45.543 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T14:59:45.546 INFO:teuthology.orchestra.run.smithi032.stdout:2084 trunc from 0x5924d69 to 0x229dc0 2024-09-21T14:59:45.559 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T14:59:45.587 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-21T14:59:45.604 INFO:teuthology.orchestra.run.smithi032.stdout:2086 write 0x6686189 thru 0x668c46d (0x62e5 bytes) 2024-09-21T14:59:45.612 INFO:teuthology.orchestra.run.smithi032.stdout:2087 write 0x60dbd61 thru 0x60e6a9f (0xad3f bytes) 2024-09-21T14:59:45.612 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T14:59:45.624 INFO:teuthology.orchestra.run.smithi032.stdout:2088 read 0x4bbf9d2 thru 0x4bcb265 (0xb894 bytes) 2024-09-21T14:59:45.634 INFO:teuthology.orchestra.run.smithi032.stdout:2090 writesame 0x84aaf48 thru 0x84b71d2 data_size 0xd(0xc28b bytes) 2024-09-21T14:59:45.634 INFO:teuthology.orchestra.run.smithi032.stdout:2092 read 0x2611b6a thru 0x261922d (0x76c4 bytes) 2024-09-21T14:59:45.635 INFO:teuthology.orchestra.run.smithi032.stdout:2093 punch from 0x3082d06 to 0x308da8a, (0xad84 bytes) 2024-09-21T14:59:45.637 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:59:45.637+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:59:45.643 INFO:teuthology.orchestra.run.smithi032.stdout:2094 trunc from 0x84b71d3 to 0x2403670 2024-09-21T14:59:45.649 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T14:59:45.664 INFO:teuthology.orchestra.run.smithi032.stdout:2095 punch from 0x123160a to 0x12325f6, (0xfec bytes) 2024-09-21T14:59:45.664 INFO:teuthology.orchestra.run.smithi032.stdout:2097 punch from 0xc1f09f to 0xc284fa, (0x945b bytes) 2024-09-21T14:59:45.664 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:45.664 INFO:teuthology.orchestra.run.smithi032.stdout:2099 punch from 0x177be69 to 0x178b4db, (0xf672 bytes) 2024-09-21T14:59:45.675 INFO:teuthology.orchestra.run.smithi032.stdout:2100 trunc from 0x2403670 to 0x914200e 2024-09-21T14:59:45.685 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T14:59:45.686 INFO:teuthology.orchestra.run.smithi032.stdout:2101 read 0x2b0b325 thru 0x2b0d1ce (0x1eaa bytes) 2024-09-21T14:59:45.686 INFO:teuthology.orchestra.run.smithi032.stdout:2103 writesame 0x59e34db thru 0x59ed34e data_size 0x2(0x9e74 bytes) 2024-09-21T14:59:45.686 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:45.687 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:59:45.687 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T14:59:45.687 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T14:59:45.687 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:45.687 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:59:45.688 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:59:45.688 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:59:45.688 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:59:45.688 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:59:45.688 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:59:45.688 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:59:45.688 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:59:45.700 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T14:59:45.705 INFO:teuthology.orchestra.run.smithi032.stdout:2105 writesame 0x4879c0c thru 0x487bab7 data_size 0x1a(0x1eac bytes) 2024-09-21T14:59:45.711 INFO:teuthology.orchestra.run.smithi032.stdout:2106 write 0x4a5183 thru 0x4aabd4 (0x5a52 bytes) 2024-09-21T14:59:45.725 INFO:teuthology.orchestra.run.smithi032.stdout:2107 write 0x92507fa thru 0x925bff0 (0xb7f7 bytes) 2024-09-21T14:59:45.737 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:59:45.731+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:59:45.739 INFO:teuthology.orchestra.run.smithi032.stdout:2108 read 0x8b650a2 thru 0x8b6f6fe (0xa65d bytes) 2024-09-21T14:59:45.739 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:45.739 INFO:teuthology.orchestra.run.smithi032.stdout:2110 read 0x25cbfb6 thru 0x25cc2e3 (0x32e bytes) 2024-09-21T14:59:45.739 INFO:teuthology.orchestra.run.smithi032.stdout:2112 write 0x2b108c8 thru 0x2b1341e (0x2b57 bytes) 2024-09-21T14:59:45.757 INFO:teuthology.orchestra.run.smithi032.stdout:2114 trunc from 0x925bff1 to 0x25be997 2024-09-21T14:59:45.781 INFO:teuthology.orchestra.run.smithi032.stdout:2115 read 0x43f2f4 thru 0x445de2 (0x6aef bytes) 2024-09-21T14:59:45.781 INFO:teuthology.orchestra.run.smithi032.stdout:2116 read 0x238bba thru 0x243f1b (0xb362 bytes) 2024-09-21T14:59:45.793 INFO:teuthology.orchestra.run.smithi032.stdout:2120 trunc from 0x25be997 to 0x901c72d 2024-09-21T14:59:45.803 INFO:teuthology.orchestra.run.smithi032.stdout:2122 punch from 0x2b522f5 to 0x2b53322, (0x102d bytes) 2024-09-21T14:59:45.803 INFO:teuthology.orchestra.run.smithi032.stdout:2123 write 0xb240d3 thru 0xb3357b (0xf4a9 bytes) 2024-09-21T14:59:45.808 INFO:teuthology.orchestra.run.smithi032.stdout:2125 punch from 0x6dbddd4 to 0x6dc42b2, (0x64de bytes) 2024-09-21T14:59:45.808 INFO:teuthology.orchestra.run.smithi032.stdout:2126 read 0xf1f6ec thru 0xf22a23 (0x3338 bytes) 2024-09-21T14:59:45.808 INFO:teuthology.orchestra.run.smithi032.stdout:2129 writesame 0x2403daa thru 0x24112fa data_size 0x3(0xd551 bytes) 2024-09-21T14:59:45.834 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:45.834 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T14:59:45.834 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T14:59:45.834 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T14:59:45.834 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.834 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:15130 3.2 3:47f1bbdb:::rbd_data.11361bed1f78.0000000000000067:head [sparse-read 0~147456] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-21T14:59:45.834 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:40.288284+0000", 2024-09-21T14:59:45.834 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 65.546372634999997, 2024-09-21T14:59:45.834 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1142935039999999, 2024-09-21T14:59:45.834 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:45.834 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:45.835 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:45.835 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:45.835 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:45.835 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 15130 2024-09-21T14:59:45.835 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.835 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:45.835 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.835 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:45.835 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288284+0000", 2024-09-21T14:59:45.835 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:45.835 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.835 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.835 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:45.835 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288284+0000", 2024-09-21T14:59:45.836 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:45.836 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.836 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.836 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:45.836 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288284+0000", 2024-09-21T14:59:45.836 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:45.836 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.836 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.836 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:45.836 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288288+0000", 2024-09-21T14:59:45.836 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5829999999999998e-06 2024-09-21T14:59:45.836 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.836 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.836 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:45.837 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288297+0000", 2024-09-21T14:59:45.837 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0669999999999998e-06 2024-09-21T14:59:45.837 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.837 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.837 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:45.837 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288305+0000", 2024-09-21T14:59:45.837 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8860000000000004e-06 2024-09-21T14:59:45.837 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.837 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.837 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:45.837 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288417+0000", 2024-09-21T14:59:45.837 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000112952 2024-09-21T14:59:45.837 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.837 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.838 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T14:59:45.838 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288428+0000", 2024-09-21T14:59:45.838 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0631e-05 2024-09-21T14:59:45.838 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.838 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.838 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:45.838 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.401876+0000", 2024-09-21T14:59:45.838 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1134479610000001 2024-09-21T14:59:45.838 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.838 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.838 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:45.838 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.402076+0000", 2024-09-21T14:59:45.838 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00019944200000000001 2024-09-21T14:59:45.838 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.839 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.839 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:45.839 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.402578+0000", 2024-09-21T14:59:45.839 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00050242899999999998 2024-09-21T14:59:45.839 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:45.839 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:45.839 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:45.839 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.839 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.839 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45708 3.2 3:415db91a:::rbd_data.1136ecbe5879.0000000000000000:28 [read 140288~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:45.840 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141801+0000", 2024-09-21T14:59:45.840 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 35.692856441000004, 2024-09-21T14:59:45.840 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034274200999999997, 2024-09-21T14:59:45.840 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:45.840 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:45.840 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:45.840 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:45.840 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:45.840 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45708 2024-09-21T14:59:45.840 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.840 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:45.840 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.840 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:45.840 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141801+0000", 2024-09-21T14:59:45.841 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:45.841 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.841 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.841 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:45.841 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141801+0000", 2024-09-21T14:59:45.841 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:45.841 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.841 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.841 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:45.841 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141800+0000", 2024-09-21T14:59:45.841 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:45.841 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.841 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.841 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:45.841 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141803+0000", 2024-09-21T14:59:45.842 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4210000000000002e-06 2024-09-21T14:59:45.842 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.842 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.842 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:45.842 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141812+0000", 2024-09-21T14:59:45.842 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.9800000000000004e-06 2024-09-21T14:59:45.842 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.842 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.842 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:45.842 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141816+0000", 2024-09-21T14:59:45.842 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4769999999999997e-06 2024-09-21T14:59:45.842 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.842 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.842 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:45.843 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.175865+0000", 2024-09-21T14:59:45.843 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034048875999999999 2024-09-21T14:59:45.843 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.843 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.843 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:45.843 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.175943+0000", 2024-09-21T14:59:45.843 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7464000000000002e-05 2024-09-21T14:59:45.843 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.843 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.843 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:45.843 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176075+0000", 2024-09-21T14:59:45.843 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013230800000000001 2024-09-21T14:59:45.843 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:45.843 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:45.843 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:45.844 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.844 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.844 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45709 3.2 3:415db91a:::rbd_data.1136ecbe5879.0000000000000000:28 [read 143360~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:45.844 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141820+0000", 2024-09-21T14:59:45.844 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 35.692836593999999, 2024-09-21T14:59:45.844 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034537536000000001, 2024-09-21T14:59:45.844 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:45.844 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:45.844 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:45.844 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:45.844 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:45.844 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45709 2024-09-21T14:59:45.844 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.844 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:45.845 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.845 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:45.845 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141820+0000", 2024-09-21T14:59:45.845 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:45.845 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.845 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.845 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:45.845 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141820+0000", 2024-09-21T14:59:45.845 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:45.845 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.845 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.845 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:45.845 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141820+0000", 2024-09-21T14:59:45.845 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:45.845 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.846 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.846 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:45.846 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141823+0000", 2024-09-21T14:59:45.846 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5110000000000002e-06 2024-09-21T14:59:45.846 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.846 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.846 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:45.846 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141829+0000", 2024-09-21T14:59:45.846 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0850000000000002e-06 2024-09-21T14:59:45.846 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.846 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.846 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:45.846 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141833+0000", 2024-09-21T14:59:45.846 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7620000000000003e-06 2024-09-21T14:59:45.847 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.847 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.847 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:45.847 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176103+0000", 2024-09-21T14:59:45.847 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034269933000000002 2024-09-21T14:59:45.847 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.847 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.847 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:45.847 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176161+0000", 2024-09-21T14:59:45.847 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7658999999999998e-05 2024-09-21T14:59:45.847 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.847 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.847 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:45.847 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176358+0000", 2024-09-21T14:59:45.848 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000196937 2024-09-21T14:59:45.848 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:45.848 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:45.848 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:45.848 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.848 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.848 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45710 3.2 3:415db91a:::rbd_data.1136ecbe5879.0000000000000000:28 [read 146432~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:45.848 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141838+0000", 2024-09-21T14:59:45.848 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 35.692819430999997, 2024-09-21T14:59:45.848 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034953674999999997, 2024-09-21T14:59:45.848 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:45.848 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:45.848 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:45.848 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:45.849 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:45.849 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45710 2024-09-21T14:59:45.849 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.849 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:45.849 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.849 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:45.849 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141838+0000", 2024-09-21T14:59:45.849 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:45.849 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.849 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.849 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:45.849 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141838+0000", 2024-09-21T14:59:45.849 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:45.849 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.849 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.850 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:45.850 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141837+0000", 2024-09-21T14:59:45.850 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:45.850 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.850 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.850 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:45.850 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141843+0000", 2024-09-21T14:59:45.850 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1619999999999996e-06 2024-09-21T14:59:45.850 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.850 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.850 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:45.850 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141850+0000", 2024-09-21T14:59:45.850 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4799999999999998e-06 2024-09-21T14:59:45.850 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.850 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.851 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:45.851 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141853+0000", 2024-09-21T14:59:45.851 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3050000000000001e-06 2024-09-21T14:59:45.851 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.851 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.851 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:45.851 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176490+0000", 2024-09-21T14:59:45.851 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034636413999999997 2024-09-21T14:59:45.851 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.851 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.851 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:45.851 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176606+0000", 2024-09-21T14:59:45.851 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000115841 2024-09-21T14:59:45.851 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.851 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.852 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:45.852 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176791+0000", 2024-09-21T14:59:45.852 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00018581500000000001 2024-09-21T14:59:45.852 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:45.852 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:45.852 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:45.852 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.852 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.852 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45711 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:45.852 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141856+0000", 2024-09-21T14:59:45.852 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 35.692801094000004, 2024-09-21T14:59:45.852 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034986308000000001, 2024-09-21T14:59:45.852 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:45.852 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:45.852 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:45.853 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:45.853 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:45.853 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45711 2024-09-21T14:59:45.853 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.853 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:45.853 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.853 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:45.853 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141856+0000", 2024-09-21T14:59:45.853 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:45.853 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.853 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.853 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:45.853 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141856+0000", 2024-09-21T14:59:45.853 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:45.853 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.854 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.854 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:45.854 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141856+0000", 2024-09-21T14:59:45.854 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:45.854 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.854 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.854 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:45.854 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141862+0000", 2024-09-21T14:59:45.854 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.567e-06 2024-09-21T14:59:45.854 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.854 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.854 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:45.855 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141868+0000", 2024-09-21T14:59:45.855 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9569999999999999e-06 2024-09-21T14:59:45.855 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.855 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.855 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:45.855 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141871+0000", 2024-09-21T14:59:45.855 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.146e-06 2024-09-21T14:59:45.855 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.855 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.855 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:45.855 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176831+0000", 2024-09-21T14:59:45.855 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034959935999999997 2024-09-21T14:59:45.855 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.856 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.856 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:45.856 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176842+0000", 2024-09-21T14:59:45.856 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1008e-05 2024-09-21T14:59:45.856 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:45.856 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:45.856 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:45.856 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.856 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.856 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45712 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:45.856 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141874+0000", 2024-09-21T14:59:45.856 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 35.692783198999997, 2024-09-21T14:59:45.856 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034993511999999997, 2024-09-21T14:59:45.856 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:45.856 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:45.857 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:45.857 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:45.857 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:45.857 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45712 2024-09-21T14:59:45.857 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.857 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:45.857 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.857 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:45.857 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141874+0000", 2024-09-21T14:59:45.857 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:45.857 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.857 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.857 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:45.857 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141874+0000", 2024-09-21T14:59:45.857 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:45.858 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.858 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.858 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:45.858 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141874+0000", 2024-09-21T14:59:45.858 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:45.858 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.858 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.858 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:45.858 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141875+0000", 2024-09-21T14:59:45.858 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5430000000000001e-06 2024-09-21T14:59:45.858 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.858 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.858 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:45.858 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141878+0000", 2024-09-21T14:59:45.859 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1420000000000002e-06 2024-09-21T14:59:45.859 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.859 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.859 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:45.859 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141881+0000", 2024-09-21T14:59:45.859 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.807e-06 2024-09-21T14:59:45.859 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.859 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.859 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:45.859 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176860+0000", 2024-09-21T14:59:45.859 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034979243 2024-09-21T14:59:45.859 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.859 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.859 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:45.859 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176867+0000", 2024-09-21T14:59:45.859 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0169999999999999e-06 2024-09-21T14:59:45.860 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:45.860 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:45.860 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:45.860 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.860 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.860 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45713 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:45.860 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141883+0000", 2024-09-21T14:59:45.860 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 35.692773623999997, 2024-09-21T14:59:45.860 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035010516999999998, 2024-09-21T14:59:45.860 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:45.860 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:45.860 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:45.860 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:45.860 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:45.860 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45713 2024-09-21T14:59:45.860 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.861 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:45.861 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.861 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:45.861 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141883+0000", 2024-09-21T14:59:45.861 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:45.861 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.861 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.861 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:45.861 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141883+0000", 2024-09-21T14:59:45.861 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:45.861 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.861 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.861 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:45.861 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141883+0000", 2024-09-21T14:59:45.862 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:45.862 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.862 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.862 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:45.862 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141885+0000", 2024-09-21T14:59:45.862 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4649999999999999e-06 2024-09-21T14:59:45.862 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.862 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.862 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:45.862 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141888+0000", 2024-09-21T14:59:45.862 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3890000000000001e-06 2024-09-21T14:59:45.862 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.862 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.862 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:45.862 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141891+0000", 2024-09-21T14:59:45.863 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9239999999999999e-06 2024-09-21T14:59:45.863 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.863 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.863 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:45.863 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176887+0000", 2024-09-21T14:59:45.863 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034996370999999998 2024-09-21T14:59:45.863 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.863 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.863 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:45.863 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176894+0000", 2024-09-21T14:59:45.863 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6069999999999996e-06 2024-09-21T14:59:45.863 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:45.863 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:45.864 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:45.864 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.864 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.864 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45714 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:45.864 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141893+0000", 2024-09-21T14:59:45.864 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 35.692763708999998, 2024-09-21T14:59:45.864 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035048215000000001, 2024-09-21T14:59:45.864 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:45.864 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:45.864 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:45.864 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:45.864 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:45.864 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45714 2024-09-21T14:59:45.864 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.864 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:45.865 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.865 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:45.865 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141893+0000", 2024-09-21T14:59:45.865 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:45.865 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.865 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.865 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:45.865 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141893+0000", 2024-09-21T14:59:45.865 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:45.865 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.865 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.865 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:45.865 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141893+0000", 2024-09-21T14:59:45.865 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:45.865 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.865 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.866 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:45.866 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141895+0000", 2024-09-21T14:59:45.866 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.703e-06 2024-09-21T14:59:45.866 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.866 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.866 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:45.866 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141900+0000", 2024-09-21T14:59:45.866 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.242e-06 2024-09-21T14:59:45.866 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.866 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.866 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:45.866 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141903+0000", 2024-09-21T14:59:45.866 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.117e-06 2024-09-21T14:59:45.866 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.867 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.867 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:45.867 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176930+0000", 2024-09-21T14:59:45.867 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035026750000000002 2024-09-21T14:59:45.867 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.867 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.867 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:45.867 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176942+0000", 2024-09-21T14:59:45.867 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1668000000000001e-05 2024-09-21T14:59:45.867 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:45.867 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:45.867 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:45.867 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.867 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.867 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45715 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:45.868 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141906+0000", 2024-09-21T14:59:45.868 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 35.692751551000001, 2024-09-21T14:59:45.868 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035066057999999997, 2024-09-21T14:59:45.868 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:45.868 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:45.868 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:45.868 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:45.868 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:45.868 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45715 2024-09-21T14:59:45.868 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.868 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:45.868 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.868 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:45.868 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141906+0000", 2024-09-21T14:59:45.868 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:45.869 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.869 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.869 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:45.869 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141906+0000", 2024-09-21T14:59:45.869 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:45.869 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.869 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.869 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:45.869 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141905+0000", 2024-09-21T14:59:45.869 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:45.869 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.869 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.869 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:45.869 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141907+0000", 2024-09-21T14:59:45.869 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5400000000000001e-06 2024-09-21T14:59:45.870 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.870 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.870 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:45.870 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141912+0000", 2024-09-21T14:59:45.870 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.502e-06 2024-09-21T14:59:45.870 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.870 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.870 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:45.870 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141915+0000", 2024-09-21T14:59:45.870 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0170000000000001e-06 2024-09-21T14:59:45.870 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.870 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.870 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:45.870 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176964+0000", 2024-09-21T14:59:45.871 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035048587999999999 2024-09-21T14:59:45.871 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.871 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.871 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:45.871 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176972+0000", 2024-09-21T14:59:45.871 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6580000000000002e-06 2024-09-21T14:59:45.871 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:45.871 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:45.871 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:45.871 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.871 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.871 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45716 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:45.871 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141918+0000", 2024-09-21T14:59:45.872 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 35.692739375000002, 2024-09-21T14:59:45.872 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035086223, 2024-09-21T14:59:45.872 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:45.872 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:45.872 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:45.872 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:45.872 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:45.872 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45716 2024-09-21T14:59:45.872 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.872 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:45.872 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.872 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:45.872 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141918+0000", 2024-09-21T14:59:45.872 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:45.872 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.873 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.873 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:45.873 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141918+0000", 2024-09-21T14:59:45.873 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:45.873 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.873 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.873 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:45.873 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141917+0000", 2024-09-21T14:59:45.873 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:45.873 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.873 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.873 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:45.873 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141919+0000", 2024-09-21T14:59:45.873 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5680000000000001e-06 2024-09-21T14:59:45.873 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.874 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.874 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:45.874 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141922+0000", 2024-09-21T14:59:45.874 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2789999999999999e-06 2024-09-21T14:59:45.874 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.874 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.874 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:45.874 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141925+0000", 2024-09-21T14:59:45.874 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9469999999999999e-06 2024-09-21T14:59:45.874 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.874 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.874 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:45.874 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176997+0000", 2024-09-21T14:59:45.875 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035071366999999999 2024-09-21T14:59:45.875 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.875 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.875 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:45.875 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177004+0000", 2024-09-21T14:59:45.875 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3309999999999998e-06 2024-09-21T14:59:45.875 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:45.875 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:45.875 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:45.875 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.875 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.875 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45717 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:45.875 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141928+0000", 2024-09-21T14:59:45.875 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 35.692729429000003, 2024-09-21T14:59:45.875 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035102573999999998, 2024-09-21T14:59:45.876 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:45.876 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:45.876 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:45.876 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:45.876 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:45.876 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45717 2024-09-21T14:59:45.876 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.876 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:45.876 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.876 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:45.876 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141928+0000", 2024-09-21T14:59:45.876 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:45.876 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.876 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.876 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:45.877 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141928+0000", 2024-09-21T14:59:45.877 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:45.877 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.877 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.877 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:45.877 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141927+0000", 2024-09-21T14:59:45.877 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T14:59:45.877 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.877 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.877 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:45.877 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141929+0000", 2024-09-21T14:59:45.877 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.533e-06 2024-09-21T14:59:45.877 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.877 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.878 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:45.878 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141934+0000", 2024-09-21T14:59:45.878 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1660000000000002e-06 2024-09-21T14:59:45.878 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.878 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.878 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:45.878 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141937+0000", 2024-09-21T14:59:45.878 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0309999999999999e-06 2024-09-21T14:59:45.878 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.878 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.878 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:45.878 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177023+0000", 2024-09-21T14:59:45.878 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035085595999999997 2024-09-21T14:59:45.878 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.878 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.879 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:45.879 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177030+0000", 2024-09-21T14:59:45.879 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.486e-06 2024-09-21T14:59:45.879 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:45.879 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:45.879 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:45.879 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.879 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.879 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45718 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:45.879 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141940+0000", 2024-09-21T14:59:45.879 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 35.692717571000003, 2024-09-21T14:59:45.879 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035124956999999998, 2024-09-21T14:59:45.879 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:45.879 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:45.879 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:45.880 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:45.880 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:45.880 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45718 2024-09-21T14:59:45.880 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.880 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:45.880 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.880 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:45.880 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141940+0000", 2024-09-21T14:59:45.880 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:45.880 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.880 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.880 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:45.880 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141940+0000", 2024-09-21T14:59:45.880 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:45.880 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.880 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.881 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:45.881 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141939+0000", 2024-09-21T14:59:45.881 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:45.881 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.881 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.881 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:45.881 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141941+0000", 2024-09-21T14:59:45.881 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.502e-06 2024-09-21T14:59:45.881 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.881 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.881 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:45.881 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141946+0000", 2024-09-21T14:59:45.881 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3449999999999996e-06 2024-09-21T14:59:45.881 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.882 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.882 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:45.882 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141949+0000", 2024-09-21T14:59:45.882 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9749999999999999e-06 2024-09-21T14:59:45.882 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.882 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.882 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:45.882 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177057+0000", 2024-09-21T14:59:45.882 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035108385999999998 2024-09-21T14:59:45.882 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.882 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.882 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:45.882 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177064+0000", 2024-09-21T14:59:45.883 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0099999999999998e-06 2024-09-21T14:59:45.883 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:45.883 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:45.883 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:45.883 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.883 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.883 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45719 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:45.883 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141952+0000", 2024-09-21T14:59:45.883 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 35.692705298, 2024-09-21T14:59:45.883 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035139108000000002, 2024-09-21T14:59:45.883 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:45.883 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:45.883 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:45.883 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:45.884 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:45.884 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45719 2024-09-21T14:59:45.884 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.884 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:45.884 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.884 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:45.884 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141952+0000", 2024-09-21T14:59:45.884 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:45.884 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.884 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.884 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:45.884 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141952+0000", 2024-09-21T14:59:45.884 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:45.884 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.884 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.885 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:45.885 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141952+0000", 2024-09-21T14:59:45.885 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:45.885 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.885 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.885 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:45.885 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141953+0000", 2024-09-21T14:59:45.885 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.486e-06 2024-09-21T14:59:45.885 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.885 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.885 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:45.885 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141959+0000", 2024-09-21T14:59:45.885 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4940000000000003e-06 2024-09-21T14:59:45.886 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.886 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.886 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:45.886 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141963+0000", 2024-09-21T14:59:45.886 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3540000000000002e-06 2024-09-21T14:59:45.886 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.886 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.886 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:45.886 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177084+0000", 2024-09-21T14:59:45.886 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035120983000000001 2024-09-21T14:59:45.886 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.886 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.886 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:45.886 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177091+0000", 2024-09-21T14:59:45.887 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0350000000000004e-06 2024-09-21T14:59:45.887 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:45.887 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:45.887 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:45.887 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.887 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.887 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45720 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:45.887 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141965+0000", 2024-09-21T14:59:45.887 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 35.692691830999998, 2024-09-21T14:59:45.887 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035155268000000003, 2024-09-21T14:59:45.887 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:45.887 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:45.887 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:45.887 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:45.888 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:45.888 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45720 2024-09-21T14:59:45.888 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.888 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:45.888 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.888 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:45.888 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141965+0000", 2024-09-21T14:59:45.888 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:45.888 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.888 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.888 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:45.888 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141965+0000", 2024-09-21T14:59:45.888 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:45.888 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.889 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.889 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:45.889 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141965+0000", 2024-09-21T14:59:45.889 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:45.889 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.889 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.889 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:45.889 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141966+0000", 2024-09-21T14:59:45.889 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.46e-06 2024-09-21T14:59:45.889 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.889 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.889 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:45.889 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141972+0000", 2024-09-21T14:59:45.890 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4210000000000003e-06 2024-09-21T14:59:45.890 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.890 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.890 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:45.890 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141975+0000", 2024-09-21T14:59:45.890 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.948e-06 2024-09-21T14:59:45.890 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.890 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.890 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:45.890 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177113+0000", 2024-09-21T14:59:45.890 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035137855000000003 2024-09-21T14:59:45.890 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.890 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.890 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:45.891 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177121+0000", 2024-09-21T14:59:45.891 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8269999999999998e-06 2024-09-21T14:59:45.891 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:45.891 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:45.891 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:45.891 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.891 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.891 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45721 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:45.891 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141977+0000", 2024-09-21T14:59:45.891 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 35.692679808000001, 2024-09-21T14:59:45.891 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035169984000000001, 2024-09-21T14:59:45.891 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:45.891 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:45.891 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:45.892 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:45.892 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:45.892 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45721 2024-09-21T14:59:45.892 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.892 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:45.892 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.892 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:45.892 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141977+0000", 2024-09-21T14:59:45.892 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:45.892 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.892 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.892 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:45.893 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141977+0000", 2024-09-21T14:59:45.893 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:45.893 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.893 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.893 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:45.893 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141977+0000", 2024-09-21T14:59:45.893 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:45.893 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.893 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.893 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:45.893 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141979+0000", 2024-09-21T14:59:45.893 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.511e-06 2024-09-21T14:59:45.893 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.893 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.894 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:45.894 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141982+0000", 2024-09-21T14:59:45.894 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4180000000000001e-06 2024-09-21T14:59:45.894 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.894 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.894 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:45.894 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141985+0000", 2024-09-21T14:59:45.894 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9450000000000002e-06 2024-09-21T14:59:45.894 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.894 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.894 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:45.894 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177139+0000", 2024-09-21T14:59:45.894 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035154409999999997 2024-09-21T14:59:45.895 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.895 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.895 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:45.895 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177147+0000", 2024-09-21T14:59:45.895 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9570000000000002e-06 2024-09-21T14:59:45.895 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:45.895 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:45.895 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:45.895 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.895 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.895 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45722 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:45.895 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141987+0000", 2024-09-21T14:59:45.895 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 35.692669836999997, 2024-09-21T14:59:45.895 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035188492000000002, 2024-09-21T14:59:45.895 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:45.896 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:45.896 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:45.896 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:45.896 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:45.896 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45722 2024-09-21T14:59:45.896 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.896 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:45.896 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.896 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:45.896 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141987+0000", 2024-09-21T14:59:45.896 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:45.896 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.896 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.896 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:45.896 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141987+0000", 2024-09-21T14:59:45.896 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:45.897 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.897 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.897 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:45.897 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141987+0000", 2024-09-21T14:59:45.897 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:45.897 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.897 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.897 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:45.897 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141988+0000", 2024-09-21T14:59:45.897 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.463e-06 2024-09-21T14:59:45.897 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.897 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.897 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:45.897 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141994+0000", 2024-09-21T14:59:45.897 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.361e-06 2024-09-21T14:59:45.897 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.898 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.898 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:45.898 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141997+0000", 2024-09-21T14:59:45.898 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0359999999999998e-06 2024-09-21T14:59:45.898 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.898 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.898 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:45.898 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177169+0000", 2024-09-21T14:59:45.898 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035171651999999998 2024-09-21T14:59:45.898 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.898 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.898 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:45.898 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177176+0000", 2024-09-21T14:59:45.898 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2189999999999998e-06 2024-09-21T14:59:45.898 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:45.899 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:45.899 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:45.899 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.899 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.899 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45723 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:45.899 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141999+0000", 2024-09-21T14:59:45.899 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 35.692657670000003, 2024-09-21T14:59:45.899 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035197701999999997, 2024-09-21T14:59:45.899 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:45.899 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:45.899 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:45.899 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:45.899 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:45.899 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45723 2024-09-21T14:59:45.900 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.900 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:45.900 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.900 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:45.900 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141999+0000", 2024-09-21T14:59:45.900 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:45.900 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.900 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.900 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:45.900 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141999+0000", 2024-09-21T14:59:45.900 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:45.900 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.900 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.900 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:45.900 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141999+0000", 2024-09-21T14:59:45.900 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:45.901 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.901 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.901 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:45.901 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142003+0000", 2024-09-21T14:59:45.901 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5049999999999999e-06 2024-09-21T14:59:45.901 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.901 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.901 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:45.901 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142008+0000", 2024-09-21T14:59:45.901 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6579999999999999e-06 2024-09-21T14:59:45.901 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.901 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.901 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:45.901 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142011+0000", 2024-09-21T14:59:45.901 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0529999999999998e-06 2024-09-21T14:59:45.902 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.902 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.902 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:45.902 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177194+0000", 2024-09-21T14:59:45.902 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035182221999999999 2024-09-21T14:59:45.902 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.902 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.902 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:45.902 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177197+0000", 2024-09-21T14:59:45.902 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5030000000000002e-06 2024-09-21T14:59:45.902 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:45.902 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:45.902 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:45.902 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.903 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.903 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45724 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:45.903 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.142015+0000", 2024-09-21T14:59:45.903 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 35.692641703, 2024-09-21T14:59:45.903 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035195345000000003, 2024-09-21T14:59:45.903 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:45.903 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:45.903 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:45.903 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:45.903 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:45.903 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45724 2024-09-21T14:59:45.903 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.903 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:45.903 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.903 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:45.904 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142015+0000", 2024-09-21T14:59:45.904 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:45.904 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.904 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.904 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:45.904 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142015+0000", 2024-09-21T14:59:45.904 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:45.904 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.904 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.904 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:45.904 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142015+0000", 2024-09-21T14:59:45.904 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:45.904 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.904 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.904 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:45.905 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142017+0000", 2024-09-21T14:59:45.905 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5540000000000001e-06 2024-09-21T14:59:45.905 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.905 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.905 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:45.905 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142022+0000", 2024-09-21T14:59:45.905 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4229999999999996e-06 2024-09-21T14:59:45.905 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.905 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.905 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:45.905 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142025+0000", 2024-09-21T14:59:45.905 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.056e-06 2024-09-21T14:59:45.905 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.905 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.905 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:45.906 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177207+0000", 2024-09-21T14:59:45.906 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035182089 2024-09-21T14:59:45.906 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.906 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.906 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:45.906 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177211+0000", 2024-09-21T14:59:45.906 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4649999999999999e-06 2024-09-21T14:59:45.906 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:45.906 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:45.906 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:45.906 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.906 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.906 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45725 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:45.907 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.142028+0000", 2024-09-21T14:59:45.907 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 35.692629424000003, 2024-09-21T14:59:45.907 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035195731000000001, 2024-09-21T14:59:45.907 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:45.907 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:45.907 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:45.907 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:45.907 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:45.907 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45725 2024-09-21T14:59:45.907 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.907 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:45.907 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.907 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:45.907 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142028+0000", 2024-09-21T14:59:45.908 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:45.908 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.908 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.908 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:45.908 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142028+0000", 2024-09-21T14:59:45.908 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:45.908 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.908 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.908 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:45.908 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142027+0000", 2024-09-21T14:59:45.908 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:45.908 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.908 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.908 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:45.909 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142029+0000", 2024-09-21T14:59:45.909 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5269999999999999e-06 2024-09-21T14:59:45.909 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.909 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.909 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:45.909 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142032+0000", 2024-09-21T14:59:45.909 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3249999999999999e-06 2024-09-21T14:59:45.909 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.909 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.909 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:45.909 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142035+0000", 2024-09-21T14:59:45.909 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.897e-06 2024-09-21T14:59:45.909 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.909 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.910 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:45.910 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177220+0000", 2024-09-21T14:59:45.910 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035184871 2024-09-21T14:59:45.910 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.910 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.910 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:45.910 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177223+0000", 2024-09-21T14:59:45.910 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3680000000000002e-06 2024-09-21T14:59:45.910 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:45.910 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:45.910 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:45.910 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.910 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.910 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45726 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:45.911 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.142037+0000", 2024-09-21T14:59:45.911 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 35.692619708999999, 2024-09-21T14:59:45.911 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035199273000000003, 2024-09-21T14:59:45.911 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:45.911 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:45.911 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:45.911 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:45.911 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:45.911 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45726 2024-09-21T14:59:45.911 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.911 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:45.911 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.911 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:45.911 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142037+0000", 2024-09-21T14:59:45.912 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:45.912 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.912 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.912 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:45.912 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142037+0000", 2024-09-21T14:59:45.912 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:45.912 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.912 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.912 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:45.912 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142037+0000", 2024-09-21T14:59:45.912 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:45.912 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.912 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.912 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:45.912 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142038+0000", 2024-09-21T14:59:45.913 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.344e-06 2024-09-21T14:59:45.913 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.913 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.913 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:45.913 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142044+0000", 2024-09-21T14:59:45.913 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1930000000000001e-06 2024-09-21T14:59:45.913 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.913 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.913 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:45.913 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142047+0000", 2024-09-21T14:59:45.913 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0280000000000001e-06 2024-09-21T14:59:45.913 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.913 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.913 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:45.914 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177233+0000", 2024-09-21T14:59:45.914 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035186394000000003 2024-09-21T14:59:45.914 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:45.914 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:45.914 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:45.914 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177237+0000", 2024-09-21T14:59:45.914 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5530000000000001e-06 2024-09-21T14:59:45.914 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:45.914 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:45.914 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:45.914 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:45.914 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:45.914 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:45.916 INFO:teuthology.orchestra.run.smithi032.stdout:2130 writesame 0xd3ba832 thru 0xd3c06b2 data_size 0x1(0x5e81 bytes) 2024-09-21T14:59:45.916 INFO:teuthology.orchestra.run.smithi032.stdout:2131 punch from 0x7be304b to 0x7be7395, (0x434a bytes) 2024-09-21T14:59:45.916 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:45.917 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T14:59:45.917 INFO:teuthology.orchestra.run.smithi032.stdout:2133 trunc from 0xd3c06b3 to 0x6868e39 2024-09-21T14:59:45.917 INFO:teuthology.orchestra.run.smithi032.stdout:2134 writesame 0xb86b1d9 thru 0xb876b6b data_size 0x1(0xb993 bytes) 2024-09-21T14:59:45.917 INFO:teuthology.orchestra.run.smithi032.stdout:2135 write 0x9c9396d thru 0x9c9a59a (0x6c2e bytes) 2024-09-21T14:59:45.917 INFO:teuthology.orchestra.run.smithi032.stdout:2136 writesame 0x1167250 thru 0x1170fe0 data_size 0x13(0x9d91 bytes) 2024-09-21T14:59:45.918 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:59:45.917+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:59:45.923 INFO:teuthology.orchestra.run.smithi032.stdout:2137 writesame 0xe27b32e thru 0xe285bae data_size 0x3(0xa881 bytes) 2024-09-21T14:59:45.923 INFO:teuthology.orchestra.run.smithi032.stdout:2138 read 0x9c8b589 thru 0x9c99ef3 (0xe96b bytes) 2024-09-21T14:59:45.924 INFO:teuthology.orchestra.run.smithi032.stdout:2139 read 0x382e7d4 thru 0x3838001 (0x982e bytes) 2024-09-21T14:59:45.925 INFO:teuthology.orchestra.run.smithi032.stdout:2140 write 0xd623a1d thru 0xd62a927 (0x6f0b bytes) 2024-09-21T14:59:45.937 INFO:teuthology.orchestra.run.smithi032.stdout:2141 write 0x2d4ac2b thru 0x2d4f271 (0x4647 bytes) 2024-09-21T14:59:45.940 INFO:teuthology.orchestra.run.smithi032.stdout:2144 write 0x6b0cf4a thru 0x6b12704 (0x57bb bytes) 2024-09-21T14:59:45.971 INFO:teuthology.orchestra.run.smithi032.stdout:2146 trunc from 0xe285baf to 0x3310fc3 2024-09-21T14:59:46.011 INFO:teuthology.orchestra.run.smithi032.stdout:2147 write 0x8969871 thru 0x89710c8 (0x7858 bytes) 2024-09-21T14:59:46.017 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:59:46.017+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:59:46.024 INFO:teuthology.orchestra.run.smithi032.stdout:2148 punch from 0x7b3b5a2 to 0x7b47fd2, (0xca30 bytes) 2024-09-21T14:59:46.025 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:46.025 INFO:teuthology.orchestra.run.smithi032.stdout:2150 punch from 0x6f3cb4c to 0x6f42dc3, (0x6277 bytes) 2024-09-21T14:59:46.038 INFO:teuthology.orchestra.run.smithi032.stdout:2151 writesame 0xb206910 thru 0xb20828b data_size 0xe(0x197c bytes) 2024-09-21T14:59:46.040 INFO:teuthology.orchestra.run.smithi032.stdout:2153 trunc from 0xb20828c to 0xca8d9f0 2024-09-21T14:59:46.042 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:46.042 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T14:59:46.042 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.042 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:80997 3.9 3:93317751:::rbd_object_map.11369d879d59:head [call lock.assert_locked in=44b,call rbd.object_map_resize in=29b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-21T14:59:46.042 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:46.043753+0000", 2024-09-21T14:59:46.042 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.00077623299999999998, 2024-09-21T14:59:46.043 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00082131499999999998, 2024-09-21T14:59:46.043 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:46.043 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "waiting for sub ops", 2024-09-21T14:59:46.043 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:46.043 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:46.043 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:46.043 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 80997 2024-09-21T14:59:46.043 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.043 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:46.043 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.043 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:46.043 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:46.043753+0000", 2024-09-21T14:59:46.043 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:46.043 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.043 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.044 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:46.044 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:46.043753+0000", 2024-09-21T14:59:46.044 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:46.044 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.044 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.044 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:46.044 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:46.043750+0000", 2024-09-21T14:59:46.044 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T14:59:46.044 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.044 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.044 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:46.044 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:46.043767+0000", 2024-09-21T14:59:46.044 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7110000000000001e-05 2024-09-21T14:59:46.044 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.044 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.045 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:46.045 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:46.043798+0000", 2024-09-21T14:59:46.045 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0981e-05 2024-09-21T14:59:46.045 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.045 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.045 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:46.045 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:46.043815+0000", 2024-09-21T14:59:46.045 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6841999999999999e-05 2024-09-21T14:59:46.045 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.045 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.045 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:46.045 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:46.043907+0000", 2024-09-21T14:59:46.045 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1846000000000001e-05 2024-09-21T14:59:46.045 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.045 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.046 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:46.046 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:46.044062+0000", 2024-09-21T14:59:46.046 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015510400000000001 2024-09-21T14:59:46.046 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.046 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.046 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 4", 2024-09-21T14:59:46.046 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:46.044461+0000", 2024-09-21T14:59:46.046 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00039860500000000001 2024-09-21T14:59:46.046 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:46.046 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:46.046 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:46.046 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:46.046 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-21T14:59:46.046 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 1 2024-09-21T14:59:46.046 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:46.052 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T14:59:46.053 INFO:teuthology.orchestra.run.smithi032.stdout:2155 writesame 0x3471991 thru 0x347e751 data_size 0x1(0xcdc1 bytes) 2024-09-21T14:59:46.080 INFO:teuthology.orchestra.run.smithi032.stdout:2159 read 0x6af1c1c thru 0x6afaa6c (0x8e51 bytes) 2024-09-21T14:59:46.081 INFO:teuthology.orchestra.run.smithi032.stdout:2160 read 0x2d840fe thru 0x2d8b932 (0x7835 bytes) 2024-09-21T14:59:46.081 INFO:teuthology.orchestra.run.smithi032.stdout:2161 write 0xa9c230e thru 0xa9c7b09 (0x57fc bytes) 2024-09-21T14:59:46.089 INFO:teuthology.orchestra.run.smithi032.stdout:2162 write 0xbcd7184 thru 0xbcdc130 (0x4fad bytes) 2024-09-21T14:59:46.098 INFO:teuthology.orchestra.run.smithi032.stdout:2163 read 0x82692b6 thru 0x8277d7e (0xeac9 bytes) 2024-09-21T14:59:46.098 INFO:teuthology.orchestra.run.smithi032.stdout:2164 write 0x39eab25 thru 0x39f7e98 (0xd374 bytes) 2024-09-21T14:59:46.118 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:59:46.115+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:59:46.119 INFO:teuthology.orchestra.run.smithi032.stdout:2165 write 0xe63b6d6 thru 0xe646210 (0xab3b bytes) 2024-09-21T14:59:46.134 INFO:teuthology.orchestra.run.smithi032.stdout:2166 read 0x35cfa5c thru 0x35d9d03 (0xa2a8 bytes) 2024-09-21T14:59:46.147 INFO:teuthology.orchestra.run.smithi032.stdout:2167 trunc from 0xe646211 to 0x6fcc17d 2024-09-21T14:59:46.166 INFO:teuthology.orchestra.run.smithi032.stdout:2168 read 0x1ff6d1a thru 0x1ffb776 (0x4a5d bytes) 2024-09-21T14:59:46.166 INFO:teuthology.orchestra.run.smithi032.stdout:2170 punch from 0x13704a4 to 0x1373627, (0x3183 bytes) 2024-09-21T14:59:46.166 INFO:teuthology.orchestra.run.smithi032.stdout:2171 write 0x66c41e6 thru 0x66cba30 (0x784b bytes) 2024-09-21T14:59:46.194 INFO:teuthology.orchestra.run.smithi032.stdout:2173 writesame 0xe5a77c8 thru 0xe5aa478 data_size 0x1(0x2cb1 bytes) 2024-09-21T14:59:46.194 INFO:teuthology.orchestra.run.smithi032.stdout:2174 write 0xd8b2044 thru 0xd8b75ce (0x558b bytes) 2024-09-21T14:59:46.201 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:46.201 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:59:46.201 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T14:59:46.201 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T14:59:46.201 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:46.214 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T14:59:46.218 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:59:46.215+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:59:46.224 INFO:teuthology.orchestra.run.smithi032.stdout:2175 trunc from 0xe5aa479 to 0xe7986f 2024-09-21T14:59:46.253 INFO:teuthology.orchestra.run.smithi032.stdout:2176 trunc from 0xe7986f to 0x1ed9a06 2024-09-21T14:59:46.261 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:46.261 INFO:teuthology.orchestra.run.smithi032.stdout:2178 read 0x190ece6 thru 0x1919944 (0xac5f bytes) 2024-09-21T14:59:46.261 INFO:teuthology.orchestra.run.smithi032.stdout:2179 punch from 0x1dc5088 to 0x1dd0d0a, (0xbc82 bytes) 2024-09-21T14:59:46.261 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:46.281 INFO:teuthology.orchestra.run.smithi032.stdout:2181 writesame 0x89af258 thru 0x89bd084 data_size 0x1(0xde2d bytes) 2024-09-21T14:59:46.281 INFO:teuthology.orchestra.run.smithi032.stdout:2182 clone 20 order 22 su 131072 sc 8 2024-09-21T14:59:46.320 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:59:46.315+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:59:46.332 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:46.332 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T14:59:46.332 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T14:59:46.332 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T14:59:46.333 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.333 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4281.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-21T14:59:46.333 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:09.324416+0000", 2024-09-21T14:59:46.333 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 97.007244689999993, 2024-09-21T14:59:46.333 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.117837947, 2024-09-21T14:59:46.333 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:46.333 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:46.333 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:46.333 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4281", 2024-09-21T14:59:46.333 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/1518520303", 2024-09-21T14:59:46.333 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1 2024-09-21T14:59:46.333 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.333 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:46.334 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.334 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:46.334 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324416+0000", 2024-09-21T14:59:46.334 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:46.334 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.334 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.334 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:46.334 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324416+0000", 2024-09-21T14:59:46.334 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:46.334 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.334 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.334 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:46.334 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324414+0000", 2024-09-21T14:59:46.335 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T14:59:46.335 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.335 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.335 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:46.335 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324429+0000", 2024-09-21T14:59:46.335 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5637e-05 2024-09-21T14:59:46.335 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.335 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.335 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:46.335 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324448+0000", 2024-09-21T14:59:46.335 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9042000000000002e-05 2024-09-21T14:59:46.335 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.335 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.335 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:46.336 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324465+0000", 2024-09-21T14:59:46.336 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6767999999999999e-05 2024-09-21T14:59:46.336 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.336 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.336 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:46.336 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324571+0000", 2024-09-21T14:59:46.336 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000105714 2024-09-21T14:59:46.336 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.336 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.336 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op must wait for map", 2024-09-21T14:59:46.336 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324587+0000", 2024-09-21T14:59:46.336 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5753999999999999e-05 2024-09-21T14:59:46.336 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.336 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.336 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:46.336 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.325864+0000", 2024-09-21T14:59:46.337 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0012773350000000001 2024-09-21T14:59:46.337 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.337 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.337 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T14:59:46.337 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.325878+0000", 2024-09-21T14:59:46.337 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4498000000000001e-05 2024-09-21T14:59:46.337 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.337 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.337 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:46.337 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.439461+0000", 2024-09-21T14:59:46.337 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.11358212500000001 2024-09-21T14:59:46.337 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.337 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.337 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:46.337 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.439848+0000", 2024-09-21T14:59:46.338 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00038747800000000002 2024-09-21T14:59:46.338 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.338 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.338 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:59:46.338 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.440175+0000", 2024-09-21T14:59:46.338 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00032721900000000001 2024-09-21T14:59:46.338 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.338 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.338 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:46.338 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.441770+0000", 2024-09-21T14:59:46.338 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0015949409999999999 2024-09-21T14:59:46.338 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.338 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.338 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:46.339 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.442086+0000", 2024-09-21T14:59:46.339 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00031586299999999999 2024-09-21T14:59:46.339 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.339 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.339 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:46.339 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.442195+0000", 2024-09-21T14:59:46.339 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000108791 2024-09-21T14:59:46.339 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.339 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.339 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:46.339 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.442254+0000", 2024-09-21T14:59:46.339 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9038e-05 2024-09-21T14:59:46.339 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:46.339 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:46.339 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:46.340 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.340 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.340 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4281.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-21T14:59:46.340 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:09.447828+0000", 2024-09-21T14:59:46.341 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 96.883832689000002, 2024-09-21T14:59:46.341 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.32904393700000001, 2024-09-21T14:59:46.341 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:46.341 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:46.341 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:46.341 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4281", 2024-09-21T14:59:46.342 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/1518520303", 2024-09-21T14:59:46.342 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 2 2024-09-21T14:59:46.342 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.342 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:46.342 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.342 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:46.342 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447828+0000", 2024-09-21T14:59:46.342 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:46.342 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.342 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.342 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:46.342 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447828+0000", 2024-09-21T14:59:46.342 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:46.342 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.343 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.343 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:46.343 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447825+0000", 2024-09-21T14:59:46.343 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T14:59:46.343 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.343 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.343 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:46.343 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447841+0000", 2024-09-21T14:59:46.343 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5359999999999999e-05 2024-09-21T14:59:46.343 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.343 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.343 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:46.343 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447875+0000", 2024-09-21T14:59:46.344 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4388999999999999e-05 2024-09-21T14:59:46.344 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.344 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.344 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:46.344 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447903+0000", 2024-09-21T14:59:46.344 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7373e-05 2024-09-21T14:59:46.344 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.344 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.344 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:46.344 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447996+0000", 2024-09-21T14:59:46.344 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2954999999999993e-05 2024-09-21T14:59:46.344 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.344 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.344 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T14:59:46.345 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.448011+0000", 2024-09-21T14:59:46.345 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5038999999999999e-05 2024-09-21T14:59:46.345 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.345 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.345 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:46.345 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.776501+0000", 2024-09-21T14:59:46.345 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.32849012500000002 2024-09-21T14:59:46.345 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.345 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.345 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:46.345 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.776872+0000", 2024-09-21T14:59:46.345 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00037119600000000002 2024-09-21T14:59:46.345 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:46.345 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:46.345 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:46.346 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.346 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.346 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:752 3.14 3:28fc02e5:::rbd_data.113693316f36.0000000000000010:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-21T14:59:46.346 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:26.625579+0000", 2024-09-21T14:59:46.346 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 79.706081834000003, 2024-09-21T14:59:46.346 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042965426000000001, 2024-09-21T14:59:46.346 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:46.346 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:46.346 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:46.346 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:46.346 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:46.346 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 752 2024-09-21T14:59:46.346 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.346 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:46.347 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.347 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:46.347 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625579+0000", 2024-09-21T14:59:46.347 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:46.347 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.347 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.347 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:46.347 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625579+0000", 2024-09-21T14:59:46.347 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:46.347 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.347 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.347 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:46.347 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625577+0000", 2024-09-21T14:59:46.347 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T14:59:46.348 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.348 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.348 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:46.348 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625588+0000", 2024-09-21T14:59:46.348 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1555e-05 2024-09-21T14:59:46.348 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.348 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.348 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:46.348 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625610+0000", 2024-09-21T14:59:46.348 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2107999999999999e-05 2024-09-21T14:59:46.348 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.348 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.348 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:46.349 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625621+0000", 2024-09-21T14:59:46.349 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0762e-05 2024-09-21T14:59:46.349 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.349 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.349 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:46.349 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625692+0000", 2024-09-21T14:59:46.349 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.1203000000000003e-05 2024-09-21T14:59:46.349 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.349 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.349 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:46.349 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625872+0000", 2024-09-21T14:59:46.349 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017962800000000001 2024-09-21T14:59:46.349 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.349 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.350 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:59:46.350 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.633188+0000", 2024-09-21T14:59:46.350 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.007316014 2024-09-21T14:59:46.350 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.350 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.350 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:46.350 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.668357+0000", 2024-09-21T14:59:46.350 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035168933999999999 2024-09-21T14:59:46.350 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.350 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.350 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:46.350 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.668418+0000", 2024-09-21T14:59:46.350 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1282e-05 2024-09-21T14:59:46.350 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.351 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.351 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:46.351 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.668486+0000", 2024-09-21T14:59:46.351 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8468999999999998e-05 2024-09-21T14:59:46.351 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.351 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.351 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:46.351 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.668544+0000", 2024-09-21T14:59:46.351 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7785000000000003e-05 2024-09-21T14:59:46.351 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:46.351 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:46.351 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:46.351 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.351 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.351 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22738 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 49152~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:46.352 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141049+0000", 2024-09-21T14:59:46.352 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 56.190611691000001, 2024-09-21T14:59:46.352 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035428754999999999, 2024-09-21T14:59:46.352 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:46.352 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:46.352 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:46.352 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:46.352 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:46.352 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22738 2024-09-21T14:59:46.352 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.352 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:46.352 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.352 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:46.352 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141049+0000", 2024-09-21T14:59:46.353 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:46.353 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.353 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.353 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:46.353 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141049+0000", 2024-09-21T14:59:46.353 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:46.353 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.353 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.353 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:46.353 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141049+0000", 2024-09-21T14:59:46.353 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:46.353 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.353 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.353 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:46.353 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141051+0000", 2024-09-21T14:59:46.354 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4980000000000001e-06 2024-09-21T14:59:46.354 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.354 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.354 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:46.354 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141057+0000", 2024-09-21T14:59:46.354 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8030000000000002e-06 2024-09-21T14:59:46.354 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.354 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.354 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:46.354 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141066+0000", 2024-09-21T14:59:46.354 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1160000000000005e-06 2024-09-21T14:59:46.354 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.354 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.355 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:46.355 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176264+0000", 2024-09-21T14:59:46.355 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035197580999999999 2024-09-21T14:59:46.355 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.355 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.355 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:46.355 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176327+0000", 2024-09-21T14:59:46.355 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3237000000000005e-05 2024-09-21T14:59:46.355 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.355 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.355 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:46.355 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176478+0000", 2024-09-21T14:59:46.355 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000150851 2024-09-21T14:59:46.355 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:46.355 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:46.356 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:46.356 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.356 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.356 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22739 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 114688~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:46.356 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141070+0000", 2024-09-21T14:59:46.356 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 56.190590448999998, 2024-09-21T14:59:46.356 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035679094000000001, 2024-09-21T14:59:46.356 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:46.356 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:46.356 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:46.356 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:46.356 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:46.356 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22739 2024-09-21T14:59:46.356 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.357 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:46.357 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.357 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:46.357 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141070+0000", 2024-09-21T14:59:46.357 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:46.357 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.357 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.357 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:46.357 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141070+0000", 2024-09-21T14:59:46.357 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:46.357 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.357 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.357 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:46.357 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141070+0000", 2024-09-21T14:59:46.357 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:46.358 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.358 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.358 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:46.358 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141072+0000", 2024-09-21T14:59:46.358 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.379e-06 2024-09-21T14:59:46.358 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.358 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.358 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:46.358 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141078+0000", 2024-09-21T14:59:46.358 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6710000000000004e-06 2024-09-21T14:59:46.358 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.358 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.358 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:46.358 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141083+0000", 2024-09-21T14:59:46.359 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7249999999999997e-06 2024-09-21T14:59:46.359 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.359 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.359 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:46.359 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176506+0000", 2024-09-21T14:59:46.359 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035423076999999997 2024-09-21T14:59:46.359 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.359 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.359 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:46.359 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176582+0000", 2024-09-21T14:59:46.359 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6305999999999998e-05 2024-09-21T14:59:46.359 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.359 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.359 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:46.360 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176749+0000", 2024-09-21T14:59:46.360 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000167262 2024-09-21T14:59:46.360 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:46.360 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:46.360 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:46.360 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.360 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.360 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22740 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 180224~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:46.360 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141090+0000", 2024-09-21T14:59:46.360 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 56.190571044999999, 2024-09-21T14:59:46.360 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036129535999999997, 2024-09-21T14:59:46.360 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:46.360 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:46.360 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:46.361 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:46.361 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:46.361 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22740 2024-09-21T14:59:46.361 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.361 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:46.361 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.361 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:46.361 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141090+0000", 2024-09-21T14:59:46.361 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:46.361 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.361 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.361 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:46.361 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141090+0000", 2024-09-21T14:59:46.361 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:46.362 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.362 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.362 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:46.362 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141089+0000", 2024-09-21T14:59:46.362 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:46.362 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.362 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.362 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:46.362 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141092+0000", 2024-09-21T14:59:46.362 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7709999999999998e-06 2024-09-21T14:59:46.362 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.362 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.362 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:46.362 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141098+0000", 2024-09-21T14:59:46.362 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.015e-06 2024-09-21T14:59:46.363 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.363 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.363 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:46.363 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141103+0000", 2024-09-21T14:59:46.363 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8629999999999999e-06 2024-09-21T14:59:46.363 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.363 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.363 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:46.363 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176778+0000", 2024-09-21T14:59:46.363 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035674984 2024-09-21T14:59:46.363 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.363 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.363 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:46.363 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176851+0000", 2024-09-21T14:59:46.364 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2861000000000006e-05 2024-09-21T14:59:46.364 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.364 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.364 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:46.364 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177219+0000", 2024-09-21T14:59:46.364 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00036845999999999998 2024-09-21T14:59:46.364 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:46.364 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:46.364 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:46.364 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.364 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.364 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22741 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 245760~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:46.365 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141107+0000", 2024-09-21T14:59:46.365 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 56.190553473999998, 2024-09-21T14:59:46.365 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036407729, 2024-09-21T14:59:46.365 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:46.365 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:46.365 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:46.365 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:46.365 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:46.365 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22741 2024-09-21T14:59:46.365 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.365 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:46.365 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.365 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:46.365 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141107+0000", 2024-09-21T14:59:46.366 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:46.366 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.366 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.366 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:46.366 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141107+0000", 2024-09-21T14:59:46.366 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:46.366 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.366 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.366 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:46.366 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141107+0000", 2024-09-21T14:59:46.366 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:46.366 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.366 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.366 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:46.367 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141109+0000", 2024-09-21T14:59:46.367 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6529999999999998e-06 2024-09-21T14:59:46.367 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.367 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.367 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:46.367 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141115+0000", 2024-09-21T14:59:46.367 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9649999999999996e-06 2024-09-21T14:59:46.367 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.367 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.367 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:46.367 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141121+0000", 2024-09-21T14:59:46.367 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1209999999999998e-06 2024-09-21T14:59:46.367 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.367 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.367 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:46.368 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177242+0000", 2024-09-21T14:59:46.368 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036121706000000003 2024-09-21T14:59:46.368 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.368 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.368 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:46.368 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177301+0000", 2024-09-21T14:59:46.368 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9008000000000002e-05 2024-09-21T14:59:46.368 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.368 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.368 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:46.368 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177515+0000", 2024-09-21T14:59:46.368 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00021362 2024-09-21T14:59:46.368 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:46.368 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:46.368 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:46.368 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.369 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.369 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22742 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 311296~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:46.369 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141126+0000", 2024-09-21T14:59:46.369 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 56.19053512, 2024-09-21T14:59:46.369 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036653537, 2024-09-21T14:59:46.369 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:46.369 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:46.369 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:46.369 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:46.369 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:46.369 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22742 2024-09-21T14:59:46.369 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.369 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:46.369 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.369 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:46.370 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141126+0000", 2024-09-21T14:59:46.370 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:46.370 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.370 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.370 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:46.370 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141126+0000", 2024-09-21T14:59:46.370 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:46.370 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.370 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.370 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:46.370 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141125+0000", 2024-09-21T14:59:46.370 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:46.370 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.370 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.370 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:46.371 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141128+0000", 2024-09-21T14:59:46.371 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.886e-06 2024-09-21T14:59:46.371 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.371 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.371 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:46.371 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141137+0000", 2024-09-21T14:59:46.371 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4399999999999994e-06 2024-09-21T14:59:46.371 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.371 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.371 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:46.371 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141143+0000", 2024-09-21T14:59:46.371 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1410000000000004e-06 2024-09-21T14:59:46.371 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.371 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.371 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:46.371 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177543+0000", 2024-09-21T14:59:46.372 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036400544999999999 2024-09-21T14:59:46.372 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.372 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.372 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:46.372 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177605+0000", 2024-09-21T14:59:46.372 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2206999999999996e-05 2024-09-21T14:59:46.372 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.372 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.372 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:46.372 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177779+0000", 2024-09-21T14:59:46.372 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017368099999999999 2024-09-21T14:59:46.372 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:46.372 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:46.372 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:46.372 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.373 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.373 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22743 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 376832~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:46.373 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141147+0000", 2024-09-21T14:59:46.373 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 56.190513635999999, 2024-09-21T14:59:46.373 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036915158000000003, 2024-09-21T14:59:46.373 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:46.373 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:46.373 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:46.373 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:46.373 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:46.373 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22743 2024-09-21T14:59:46.373 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.373 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:46.373 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.374 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:46.374 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141147+0000", 2024-09-21T14:59:46.374 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:46.374 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.374 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.374 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:46.374 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141147+0000", 2024-09-21T14:59:46.374 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:46.374 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.374 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.374 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:46.374 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141147+0000", 2024-09-21T14:59:46.374 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:46.375 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.375 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.375 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:46.375 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141149+0000", 2024-09-21T14:59:46.375 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.79e-06 2024-09-21T14:59:46.375 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.375 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.375 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:46.375 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141155+0000", 2024-09-21T14:59:46.375 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6699999999999999e-06 2024-09-21T14:59:46.375 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.375 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.375 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:46.375 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141160+0000", 2024-09-21T14:59:46.375 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5759999999999999e-06 2024-09-21T14:59:46.376 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.376 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.376 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:46.376 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177817+0000", 2024-09-21T14:59:46.376 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036657404999999997 2024-09-21T14:59:46.376 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.376 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.376 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:46.376 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177892+0000", 2024-09-21T14:59:46.376 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4602000000000002e-05 2024-09-21T14:59:46.376 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.376 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.376 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:46.376 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178062+0000", 2024-09-21T14:59:46.377 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000170464 2024-09-21T14:59:46.377 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:46.377 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:46.377 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:46.377 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.377 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.377 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22744 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 442368~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:46.377 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141164+0000", 2024-09-21T14:59:46.377 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 56.190496502999999, 2024-09-21T14:59:46.377 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037120806999999999, 2024-09-21T14:59:46.377 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:46.377 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:46.377 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:46.377 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:46.378 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:46.378 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22744 2024-09-21T14:59:46.378 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.378 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:46.378 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.378 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:46.378 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141164+0000", 2024-09-21T14:59:46.378 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:46.378 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.378 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.378 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:46.378 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141164+0000", 2024-09-21T14:59:46.378 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:46.378 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.379 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.379 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:46.379 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141164+0000", 2024-09-21T14:59:46.379 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:46.379 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.379 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.379 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:46.379 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141167+0000", 2024-09-21T14:59:46.379 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7159999999999999e-06 2024-09-21T14:59:46.379 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.379 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.379 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:46.379 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141173+0000", 2024-09-21T14:59:46.379 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.071e-06 2024-09-21T14:59:46.380 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.380 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.380 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:46.380 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141177+0000", 2024-09-21T14:59:46.380 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3610000000000003e-06 2024-09-21T14:59:46.380 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.380 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.380 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:46.380 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178083+0000", 2024-09-21T14:59:46.380 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036906405000000003 2024-09-21T14:59:46.380 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.380 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.380 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:46.380 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178143+0000", 2024-09-21T14:59:46.380 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0056999999999998e-05 2024-09-21T14:59:46.381 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.381 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.381 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:46.381 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178285+0000", 2024-09-21T14:59:46.381 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000141555 2024-09-21T14:59:46.381 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:46.381 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:46.381 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:46.381 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.381 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.381 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22745 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 507904~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:46.381 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141182+0000", 2024-09-21T14:59:46.381 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 56.190478183000003, 2024-09-21T14:59:46.381 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037332331000000003, 2024-09-21T14:59:46.382 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:46.382 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:46.382 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:46.382 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:46.382 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:46.382 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22745 2024-09-21T14:59:46.382 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.382 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:46.382 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.382 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:46.382 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141182+0000", 2024-09-21T14:59:46.382 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:46.382 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.382 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.382 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:46.383 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141182+0000", 2024-09-21T14:59:46.383 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:46.383 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.383 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.383 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:46.383 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141182+0000", 2024-09-21T14:59:46.383 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:46.383 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.383 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.383 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:46.383 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141185+0000", 2024-09-21T14:59:46.383 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.464e-06 2024-09-21T14:59:46.383 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.383 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.384 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:46.384 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141200+0000", 2024-09-21T14:59:46.384 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5886999999999999e-05 2024-09-21T14:59:46.384 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.384 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.384 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:46.384 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141571+0000", 2024-09-21T14:59:46.384 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000370909 2024-09-21T14:59:46.384 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.384 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.384 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:46.384 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178318+0000", 2024-09-21T14:59:46.384 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036746244999999997 2024-09-21T14:59:46.384 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.385 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.385 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:46.385 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178367+0000", 2024-09-21T14:59:46.385 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9317000000000002e-05 2024-09-21T14:59:46.385 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.385 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.385 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:46.385 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178515+0000", 2024-09-21T14:59:46.385 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014784600000000001 2024-09-21T14:59:46.385 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:46.385 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:46.385 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:46.385 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.385 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.385 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22746 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 573440~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:46.386 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141599+0000", 2024-09-21T14:59:46.386 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 56.190061550999999, 2024-09-21T14:59:46.386 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037402249999999998, 2024-09-21T14:59:46.386 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:46.386 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:46.386 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:46.386 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:46.386 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:46.386 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22746 2024-09-21T14:59:46.386 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.386 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:46.386 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.386 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:46.386 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141599+0000", 2024-09-21T14:59:46.387 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:46.387 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.387 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.387 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:46.387 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141599+0000", 2024-09-21T14:59:46.387 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:46.387 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.387 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.387 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:46.387 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141598+0000", 2024-09-21T14:59:46.387 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:59:46.387 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.387 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.387 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:46.387 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141637+0000", 2024-09-21T14:59:46.388 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.9119999999999998e-05 2024-09-21T14:59:46.388 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.388 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.388 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:46.388 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141647+0000", 2024-09-21T14:59:46.388 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5570000000000001e-06 2024-09-21T14:59:46.388 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.388 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.388 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:46.388 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141654+0000", 2024-09-21T14:59:46.388 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2049999999999996e-06 2024-09-21T14:59:46.388 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.388 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.388 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:46.388 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178553+0000", 2024-09-21T14:59:46.389 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036898528 2024-09-21T14:59:46.389 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.389 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.389 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:46.389 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178625+0000", 2024-09-21T14:59:46.389 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2669999999999994e-05 2024-09-21T14:59:46.389 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.389 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.389 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:46.389 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179001+0000", 2024-09-21T14:59:46.389 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000375977 2024-09-21T14:59:46.389 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:46.389 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:46.389 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:46.389 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.390 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.390 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22747 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 638976~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:46.390 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141660+0000", 2024-09-21T14:59:46.390 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 56.190000409, 2024-09-21T14:59:46.390 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037617352999999999, 2024-09-21T14:59:46.390 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:46.390 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:46.390 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:46.390 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:46.390 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:46.390 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22747 2024-09-21T14:59:46.390 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.390 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:46.390 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.391 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:46.391 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141660+0000", 2024-09-21T14:59:46.391 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:46.391 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.391 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.391 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:46.391 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141660+0000", 2024-09-21T14:59:46.391 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:46.391 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.391 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.391 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:46.391 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141660+0000", 2024-09-21T14:59:46.391 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:46.391 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.391 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.392 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:46.392 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141663+0000", 2024-09-21T14:59:46.392 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.569e-06 2024-09-21T14:59:46.392 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.392 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.392 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:46.392 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141670+0000", 2024-09-21T14:59:46.392 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5719999999999999e-06 2024-09-21T14:59:46.392 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.392 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.392 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:46.392 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141675+0000", 2024-09-21T14:59:46.392 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2889999999999997e-06 2024-09-21T14:59:46.392 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.392 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.393 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:46.393 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179028+0000", 2024-09-21T14:59:46.393 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037353381999999997 2024-09-21T14:59:46.393 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.393 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.393 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:46.393 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179099+0000", 2024-09-21T14:59:46.393 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0259000000000006e-05 2024-09-21T14:59:46.393 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.393 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.393 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:46.393 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179278+0000", 2024-09-21T14:59:46.393 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000178908 2024-09-21T14:59:46.393 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:46.393 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:46.394 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:46.394 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.394 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.394 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22748 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 704512~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:46.394 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141680+0000", 2024-09-21T14:59:46.394 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 56.189980771000002, 2024-09-21T14:59:46.394 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037806166000000002, 2024-09-21T14:59:46.394 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:46.394 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:46.394 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:46.394 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:46.394 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:46.394 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22748 2024-09-21T14:59:46.394 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.394 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:46.395 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.395 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:46.395 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141680+0000", 2024-09-21T14:59:46.395 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:46.395 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.395 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.395 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:46.395 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141680+0000", 2024-09-21T14:59:46.395 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:46.395 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.395 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.395 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:46.395 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141679+0000", 2024-09-21T14:59:46.395 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:46.396 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.396 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.396 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:46.396 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141683+0000", 2024-09-21T14:59:46.396 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3230000000000002e-06 2024-09-21T14:59:46.396 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.396 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.396 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:46.396 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141690+0000", 2024-09-21T14:59:46.396 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8680000000000001e-06 2024-09-21T14:59:46.396 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.396 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.396 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:46.396 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141694+0000", 2024-09-21T14:59:46.397 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6770000000000003e-06 2024-09-21T14:59:46.397 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.397 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.397 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:46.397 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179300+0000", 2024-09-21T14:59:46.397 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037605567999999999 2024-09-21T14:59:46.397 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.397 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.397 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:46.397 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179355+0000", 2024-09-21T14:59:46.397 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4771000000000001e-05 2024-09-21T14:59:46.397 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.397 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.397 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:46.397 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179486+0000", 2024-09-21T14:59:46.398 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013141699999999999 2024-09-21T14:59:46.398 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:46.398 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:46.398 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:46.398 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.398 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.398 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22749 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 770048~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:46.398 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141698+0000", 2024-09-21T14:59:46.398 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 56.189962399000002, 2024-09-21T14:59:46.398 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037967813000000003, 2024-09-21T14:59:46.398 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:46.398 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:46.398 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:46.398 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:46.398 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:46.399 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22749 2024-09-21T14:59:46.399 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.399 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:46.399 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.399 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:46.399 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141698+0000", 2024-09-21T14:59:46.399 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:46.399 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.399 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.399 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:46.399 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141698+0000", 2024-09-21T14:59:46.399 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:46.399 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.399 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.399 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:46.399 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141698+0000", 2024-09-21T14:59:46.400 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:46.400 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.400 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.400 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:46.400 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141701+0000", 2024-09-21T14:59:46.400 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8399999999999999e-06 2024-09-21T14:59:46.400 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.400 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.400 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:46.400 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141706+0000", 2024-09-21T14:59:46.400 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4979999999999997e-06 2024-09-21T14:59:46.400 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.400 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.400 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:46.400 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141711+0000", 2024-09-21T14:59:46.401 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3850000000000004e-06 2024-09-21T14:59:46.401 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.401 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.401 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:46.401 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179504+0000", 2024-09-21T14:59:46.401 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037793514 2024-09-21T14:59:46.401 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.401 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.401 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:46.401 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179546+0000", 2024-09-21T14:59:46.401 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1467999999999997e-05 2024-09-21T14:59:46.401 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.401 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.401 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:46.401 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179666+0000", 2024-09-21T14:59:46.402 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000120499 2024-09-21T14:59:46.402 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:46.402 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:46.402 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:46.402 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.402 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.402 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22750 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 835584~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:46.402 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141715+0000", 2024-09-21T14:59:46.402 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 56.189946098, 2024-09-21T14:59:46.402 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038173179000000002, 2024-09-21T14:59:46.402 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:46.402 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:46.402 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:46.402 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:46.402 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:46.403 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22750 2024-09-21T14:59:46.403 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.403 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:46.403 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.403 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:46.403 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141715+0000", 2024-09-21T14:59:46.403 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:46.403 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.403 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.403 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:46.403 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141715+0000", 2024-09-21T14:59:46.403 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:46.403 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.404 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.404 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:46.404 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141714+0000", 2024-09-21T14:59:46.404 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:46.404 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.404 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.404 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:46.404 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141716+0000", 2024-09-21T14:59:46.404 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1730000000000002e-06 2024-09-21T14:59:46.404 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.404 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.404 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:46.404 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141722+0000", 2024-09-21T14:59:46.404 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.417e-06 2024-09-21T14:59:46.405 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.405 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.405 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:46.405 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141726+0000", 2024-09-21T14:59:46.405 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4830000000000001e-06 2024-09-21T14:59:46.405 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.405 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.405 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:46.405 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179683+0000", 2024-09-21T14:59:46.405 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037956977000000003 2024-09-21T14:59:46.405 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.405 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.405 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:46.405 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179751+0000", 2024-09-21T14:59:46.405 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7762000000000006e-05 2024-09-21T14:59:46.406 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.406 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.406 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:46.406 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179888+0000", 2024-09-21T14:59:46.406 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000136635 2024-09-21T14:59:46.406 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:46.406 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:46.406 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:46.406 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.406 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.406 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22751 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 901120~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:46.406 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141730+0000", 2024-09-21T14:59:46.406 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 56.189930298999997, 2024-09-21T14:59:46.406 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038357011000000003, 2024-09-21T14:59:46.407 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:46.407 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:46.407 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:46.407 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:46.407 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:46.407 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22751 2024-09-21T14:59:46.407 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.407 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:46.407 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.407 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:46.407 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141730+0000", 2024-09-21T14:59:46.407 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:46.407 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.407 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.407 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:46.407 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141730+0000", 2024-09-21T14:59:46.408 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:46.408 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.408 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.408 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:46.408 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141730+0000", 2024-09-21T14:59:46.408 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:46.408 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.408 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.408 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:46.408 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141737+0000", 2024-09-21T14:59:46.408 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.64e-06 2024-09-21T14:59:46.408 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.408 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.408 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:46.408 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141743+0000", 2024-09-21T14:59:46.409 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8130000000000001e-06 2024-09-21T14:59:46.409 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.409 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.409 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:46.409 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141747+0000", 2024-09-21T14:59:46.409 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4920000000000004e-06 2024-09-21T14:59:46.409 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.409 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.409 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:46.409 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179918+0000", 2024-09-21T14:59:46.409 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038170822 2024-09-21T14:59:46.409 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.409 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.409 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:46.409 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179960+0000", 2024-09-21T14:59:46.410 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1999000000000002e-05 2024-09-21T14:59:46.410 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.410 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.410 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:46.410 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180087+0000", 2024-09-21T14:59:46.410 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012752799999999999 2024-09-21T14:59:46.410 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:46.410 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:46.410 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:46.410 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.410 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.410 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22752 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 966656~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:46.410 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141751+0000", 2024-09-21T14:59:46.410 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 56.189909978999999, 2024-09-21T14:59:46.410 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038553084000000001, 2024-09-21T14:59:46.411 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:46.411 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:46.411 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:46.411 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:46.411 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:46.411 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22752 2024-09-21T14:59:46.411 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.411 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:46.411 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.411 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:46.411 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141751+0000", 2024-09-21T14:59:46.411 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:46.411 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.411 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.412 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:46.412 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141751+0000", 2024-09-21T14:59:46.412 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:46.412 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.412 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.412 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:46.412 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141750+0000", 2024-09-21T14:59:46.412 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:46.412 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.412 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.412 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:46.412 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141753+0000", 2024-09-21T14:59:46.412 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5900000000000002e-06 2024-09-21T14:59:46.412 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.412 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.413 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:46.413 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141759+0000", 2024-09-21T14:59:46.413 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.947e-06 2024-09-21T14:59:46.413 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.413 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.413 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:46.413 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141764+0000", 2024-09-21T14:59:46.413 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8319999999999996e-06 2024-09-21T14:59:46.413 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.413 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.413 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:46.413 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180108+0000", 2024-09-21T14:59:46.413 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038344095000000002 2024-09-21T14:59:46.413 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.413 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.413 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:46.414 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180168+0000", 2024-09-21T14:59:46.414 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0047000000000003e-05 2024-09-21T14:59:46.414 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.414 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.414 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:46.414 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180304+0000", 2024-09-21T14:59:46.414 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013592500000000001 2024-09-21T14:59:46.414 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:46.414 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:46.414 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:46.414 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.414 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.414 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22753 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 1032192~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:46.415 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141768+0000", 2024-09-21T14:59:46.415 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 56.189893105000003, 2024-09-21T14:59:46.415 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038818810000000002, 2024-09-21T14:59:46.415 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:46.415 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:46.415 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:46.415 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:46.415 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:46.415 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22753 2024-09-21T14:59:46.415 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.415 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:46.415 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.415 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:46.415 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141768+0000", 2024-09-21T14:59:46.415 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:46.416 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.416 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.416 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:46.416 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141768+0000", 2024-09-21T14:59:46.416 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:46.416 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.416 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.416 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:46.416 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141767+0000", 2024-09-21T14:59:46.416 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:46.416 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.416 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.416 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:46.416 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141770+0000", 2024-09-21T14:59:46.416 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3329999999999999e-06 2024-09-21T14:59:46.416 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.417 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.417 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:46.417 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141775+0000", 2024-09-21T14:59:46.417 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2669999999999997e-06 2024-09-21T14:59:46.417 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.417 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.417 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:46.417 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141779+0000", 2024-09-21T14:59:46.417 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2259999999999999e-06 2024-09-21T14:59:46.417 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.417 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.417 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:46.417 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180325+0000", 2024-09-21T14:59:46.417 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038546428000000001 2024-09-21T14:59:46.417 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.417 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.417 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:46.418 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180401+0000", 2024-09-21T14:59:46.418 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5437000000000003e-05 2024-09-21T14:59:46.418 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.418 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.418 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:46.418 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180586+0000", 2024-09-21T14:59:46.418 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000185429 2024-09-21T14:59:46.418 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:46.418 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:46.418 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:46.418 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.418 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.418 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:73793 3.11 3:8c8a112d:::rbd_data.1136f4939602.0000000000000003:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-21T14:59:46.418 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:42.225633+0000", 2024-09-21T14:59:46.418 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.1060271909999999, 2024-09-21T14:59:46.418 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036337102000000003, 2024-09-21T14:59:46.419 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:46.419 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:46.419 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:46.419 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:46.419 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:46.419 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 73793 2024-09-21T14:59:46.419 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.419 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:46.419 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.419 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:46.419 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.225633+0000", 2024-09-21T14:59:46.419 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:46.419 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.419 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.419 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:46.419 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.225633+0000", 2024-09-21T14:59:46.420 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:46.420 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.420 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.420 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:46.420 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.225630+0000", 2024-09-21T14:59:46.420 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999967 2024-09-21T14:59:46.420 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.420 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.420 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:46.420 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.225649+0000", 2024-09-21T14:59:46.420 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9377000000000001e-05 2024-09-21T14:59:46.420 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.420 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.420 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:46.420 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.225689+0000", 2024-09-21T14:59:46.420 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.9879999999999998e-05 2024-09-21T14:59:46.421 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.421 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.421 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:46.421 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.225706+0000", 2024-09-21T14:59:46.421 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6537000000000001e-05 2024-09-21T14:59:46.421 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.421 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.421 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:46.421 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.225772+0000", 2024-09-21T14:59:46.421 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5894000000000003e-05 2024-09-21T14:59:46.421 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.421 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.421 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:46.421 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.225846+0000", 2024-09-21T14:59:46.421 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4165000000000006e-05 2024-09-21T14:59:46.421 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.421 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.422 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 1", 2024-09-21T14:59:46.422 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.231400+0000", 2024-09-21T14:59:46.422 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0055538269999999999 2024-09-21T14:59:46.422 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.422 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.422 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:46.422 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.261709+0000", 2024-09-21T14:59:46.422 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.030309750999999999 2024-09-21T14:59:46.422 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.422 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.422 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:46.422 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.261738+0000", 2024-09-21T14:59:46.422 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8569999999999999e-05 2024-09-21T14:59:46.422 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.422 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.423 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:46.423 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.261839+0000", 2024-09-21T14:59:46.423 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010118 2024-09-21T14:59:46.423 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:46.423 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:46.423 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:46.423 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.261971+0000", 2024-09-21T14:59:46.423 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000131414 2024-09-21T14:59:46.423 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:46.423 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:46.423 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:46.423 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:46.423 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:46.423 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:46.424 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T14:59:46.425 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T14:59:46.426 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:59:46.419+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:59:46.526 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:59:46.523+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:59:46.545 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:59:46.545 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:59:46.545 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T14:59:46.545 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:59:46.555 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T14:59:46.626 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:59:46.625+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:59:46.667 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:59:46.667 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:59:46.667 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T14:59:46.667 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T14:59:46.667 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:59:46.677 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T14:59:46.727 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:59:46.723+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:59:46.791 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:59:46.791 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T14:59:46.791 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T14:59:46.791 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T14:59:46.791 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.791 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:1452 3.19 3:9d40fe72:::rbd_data.1136515ecb6b.0000000000000013:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T14:59:46.791 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:30.292755+0000", 2024-09-21T14:59:46.791 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 76.494754220999994, 2024-09-21T14:59:46.791 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043852191999999998, 2024-09-21T14:59:46.791 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:46.791 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:46.791 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:46.792 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:46.792 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:46.792 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1452 2024-09-21T14:59:46.792 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.792 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:46.792 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.792 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:46.792 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292755+0000", 2024-09-21T14:59:46.792 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:46.792 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.792 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.792 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:46.792 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292755+0000", 2024-09-21T14:59:46.792 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:46.792 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.793 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.793 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:46.793 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292754+0000", 2024-09-21T14:59:46.793 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:46.793 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.793 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.793 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:46.793 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292760+0000", 2024-09-21T14:59:46.793 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5029999999999997e-06 2024-09-21T14:59:46.793 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.793 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.793 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:46.793 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292771+0000", 2024-09-21T14:59:46.793 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1551e-05 2024-09-21T14:59:46.793 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.794 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.794 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:46.794 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292779+0000", 2024-09-21T14:59:46.794 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1559999999999998e-06 2024-09-21T14:59:46.794 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.794 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.794 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:46.794 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292896+0000", 2024-09-21T14:59:46.794 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000117304 2024-09-21T14:59:46.794 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.794 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.794 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:46.794 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.293124+0000", 2024-09-21T14:59:46.794 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00022819999999999999 2024-09-21T14:59:46.795 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.795 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.795 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 1", 2024-09-21T14:59:46.795 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.299967+0000", 2024-09-21T14:59:46.795 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0068427009999999996 2024-09-21T14:59:46.795 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.795 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.795 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:46.795 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336463+0000", 2024-09-21T14:59:46.795 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.036495842000000001 2024-09-21T14:59:46.795 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.795 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.795 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:59:46.795 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336480+0000", 2024-09-21T14:59:46.795 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7419999999999999e-05 2024-09-21T14:59:46.795 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.796 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.796 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:59:46.796 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336550+0000", 2024-09-21T14:59:46.796 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9951999999999997e-05 2024-09-21T14:59:46.796 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.796 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.796 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:46.796 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336607+0000", 2024-09-21T14:59:46.796 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7423000000000002e-05 2024-09-21T14:59:46.796 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:46.796 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:46.796 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:46.796 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.796 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.796 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop_reply(client.4406.0:1452 3.19 e36/24 ondisk, result = 0)", 2024-09-21T14:59:46.797 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:30.322854+0000", 2024-09-21T14:59:46.797 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 76.464655544999999, 2024-09-21T14:59:46.797 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013613079, 2024-09-21T14:59:46.797 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:46.797 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:46.797 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:46.797 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.797 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:46.797 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.322854+0000", 2024-09-21T14:59:46.797 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:46.797 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.797 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.797 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:46.797 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.322854+0000", 2024-09-21T14:59:46.798 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:46.798 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.798 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.798 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:46.798 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.322852+0000", 2024-09-21T14:59:46.798 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:59:46.798 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.798 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.798 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:46.798 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.322860+0000", 2024-09-21T14:59:46.798 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.605e-06 2024-09-21T14:59:46.798 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.798 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.798 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:46.799 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.322878+0000", 2024-09-21T14:59:46.799 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8204000000000001e-05 2024-09-21T14:59:46.799 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.799 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.799 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:46.799 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.322888+0000", 2024-09-21T14:59:46.799 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4099999999999997e-06 2024-09-21T14:59:46.799 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.799 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.799 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:46.799 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336440+0000", 2024-09-21T14:59:46.799 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013551848 2024-09-21T14:59:46.799 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.799 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.799 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:46.800 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336453+0000", 2024-09-21T14:59:46.800 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3465000000000001e-05 2024-09-21T14:59:46.800 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.800 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.800 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:46.800 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336467+0000", 2024-09-21T14:59:46.800 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3913999999999999e-05 2024-09-21T14:59:46.800 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:46.800 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:46.800 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:46.800 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.800 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.800 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:1567 3.8 3:139d3d55:::rbd_data.1136515ecb6b.0000000000000000:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T14:59:46.800 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:30.681087+0000", 2024-09-21T14:59:46.800 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 76.106422006000003, 2024-09-21T14:59:46.800 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.039009224000000002, 2024-09-21T14:59:46.801 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:46.801 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:46.801 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:46.801 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:46.801 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:46.801 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1567 2024-09-21T14:59:46.801 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.801 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:46.801 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.801 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:46.801 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681087+0000", 2024-09-21T14:59:46.801 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:46.801 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.801 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.801 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:46.801 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681087+0000", 2024-09-21T14:59:46.802 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:46.802 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.802 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.802 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:46.802 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681086+0000", 2024-09-21T14:59:46.802 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:59:46.802 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.802 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.802 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:46.802 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681097+0000", 2024-09-21T14:59:46.802 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.134e-05 2024-09-21T14:59:46.802 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.802 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.802 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:46.802 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681122+0000", 2024-09-21T14:59:46.802 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5313e-05 2024-09-21T14:59:46.803 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.803 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.803 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:46.803 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681138+0000", 2024-09-21T14:59:46.803 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5916000000000001e-05 2024-09-21T14:59:46.803 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.803 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.803 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:46.803 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681270+0000", 2024-09-21T14:59:46.803 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000131474 2024-09-21T14:59:46.803 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.803 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.803 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:46.803 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681416+0000", 2024-09-21T14:59:46.804 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014637 2024-09-21T14:59:46.804 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.804 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.804 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 7", 2024-09-21T14:59:46.804 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.693163+0000", 2024-09-21T14:59:46.804 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011747241 2024-09-21T14:59:46.804 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.804 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.804 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:59:46.804 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.719750+0000", 2024-09-21T14:59:46.804 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.026586852000000001 2024-09-21T14:59:46.804 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.804 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.804 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:46.805 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.719859+0000", 2024-09-21T14:59:46.805 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010925999999999999 2024-09-21T14:59:46.805 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.805 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.805 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:59:46.805 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.719951+0000", 2024-09-21T14:59:46.805 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1294999999999999e-05 2024-09-21T14:59:46.805 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.805 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.805 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:46.805 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.720097+0000", 2024-09-21T14:59:46.805 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014583 2024-09-21T14:59:46.805 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:46.805 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:46.806 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:46.806 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.806 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.806 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9348 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 122880~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:46.806 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158369+0000", 2024-09-21T14:59:46.806 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 67.629140336999995, 2024-09-21T14:59:46.806 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.01388847, 2024-09-21T14:59:46.806 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:46.806 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:46.806 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:46.806 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:46.806 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:46.806 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9348 2024-09-21T14:59:46.806 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.807 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:46.807 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.807 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:46.807 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158369+0000", 2024-09-21T14:59:46.807 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:46.807 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.807 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.807 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:46.807 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158369+0000", 2024-09-21T14:59:46.807 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:46.807 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.807 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.807 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:46.807 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158368+0000", 2024-09-21T14:59:46.808 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:46.808 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.808 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.808 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:46.808 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158374+0000", 2024-09-21T14:59:46.808 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0499999999999997e-06 2024-09-21T14:59:46.808 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.808 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.808 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:46.808 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158386+0000", 2024-09-21T14:59:46.808 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1579999999999999e-05 2024-09-21T14:59:46.808 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.808 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.808 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:46.808 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158396+0000", 2024-09-21T14:59:46.808 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.8260000000000005e-06 2024-09-21T14:59:46.809 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.809 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.809 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:46.809 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.171967+0000", 2024-09-21T14:59:46.809 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013571318000000001 2024-09-21T14:59:46.809 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.809 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.809 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:46.809 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172034+0000", 2024-09-21T14:59:46.809 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.656e-05 2024-09-21T14:59:46.809 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.809 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.809 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:46.809 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172258+0000", 2024-09-21T14:59:46.809 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00022395999999999999 2024-09-21T14:59:46.809 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:46.810 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:46.810 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:46.810 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.810 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.810 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9349 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 155648~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:46.810 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158403+0000", 2024-09-21T14:59:46.810 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 67.629106590999996, 2024-09-21T14:59:46.810 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014229851, 2024-09-21T14:59:46.810 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:46.810 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:46.810 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:46.810 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:46.810 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:46.810 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9349 2024-09-21T14:59:46.810 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.811 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:46.811 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.811 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:46.811 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158403+0000", 2024-09-21T14:59:46.811 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:46.811 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.811 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.811 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:46.811 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158403+0000", 2024-09-21T14:59:46.811 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:46.811 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.811 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.811 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:46.811 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158402+0000", 2024-09-21T14:59:46.811 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:46.811 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.812 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.812 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:46.812 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158406+0000", 2024-09-21T14:59:46.812 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1339999999999997e-06 2024-09-21T14:59:46.812 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.812 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.812 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:46.812 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158416+0000", 2024-09-21T14:59:46.812 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3149999999999998e-06 2024-09-21T14:59:46.812 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.812 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.812 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:46.812 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158439+0000", 2024-09-21T14:59:46.812 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3255999999999999e-05 2024-09-21T14:59:46.812 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.812 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.812 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:46.813 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172305+0000", 2024-09-21T14:59:46.813 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013866603999999999 2024-09-21T14:59:46.813 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.813 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.813 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:46.813 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172413+0000", 2024-09-21T14:59:46.813 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000107664 2024-09-21T14:59:46.813 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.813 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.813 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:46.813 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172633+0000", 2024-09-21T14:59:46.813 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021950800000000001 2024-09-21T14:59:46.813 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:46.813 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:46.813 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:46.814 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.814 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.814 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9350 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 294912~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:46.814 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158448+0000", 2024-09-21T14:59:46.814 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 67.629061719999996, 2024-09-21T14:59:46.814 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014471525000000001, 2024-09-21T14:59:46.814 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:46.814 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:46.814 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:46.814 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:46.814 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:46.814 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9350 2024-09-21T14:59:46.814 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.814 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:46.814 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.815 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:46.815 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158448+0000", 2024-09-21T14:59:46.815 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:46.815 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.815 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.815 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:46.815 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158448+0000", 2024-09-21T14:59:46.815 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:46.815 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.815 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.815 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:46.815 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158447+0000", 2024-09-21T14:59:46.815 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:46.815 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.816 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.816 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:46.816 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158452+0000", 2024-09-21T14:59:46.816 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8400000000000002e-06 2024-09-21T14:59:46.816 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.816 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.816 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:46.816 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158475+0000", 2024-09-21T14:59:46.816 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3091e-05 2024-09-21T14:59:46.816 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.816 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.816 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:46.816 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158515+0000", 2024-09-21T14:59:46.816 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0265000000000002e-05 2024-09-21T14:59:46.817 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.817 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.817 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:46.817 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172682+0000", 2024-09-21T14:59:46.817 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014166761999999999 2024-09-21T14:59:46.817 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.817 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.817 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:46.817 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172752+0000", 2024-09-21T14:59:46.817 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9698999999999998e-05 2024-09-21T14:59:46.817 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.817 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.818 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:46.818 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172919+0000", 2024-09-21T14:59:46.818 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016759199999999999 2024-09-21T14:59:46.818 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:46.818 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:46.818 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:46.818 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.818 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.818 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9351 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 434176~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:46.818 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158526+0000", 2024-09-21T14:59:46.818 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 67.628983001999998, 2024-09-21T14:59:46.818 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014662107000000001, 2024-09-21T14:59:46.818 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:46.819 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:46.819 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:46.819 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:46.819 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:46.819 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9351 2024-09-21T14:59:46.819 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.819 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:46.819 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.819 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:46.819 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158526+0000", 2024-09-21T14:59:46.819 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:46.819 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.819 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.819 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:46.819 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158526+0000", 2024-09-21T14:59:46.819 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:46.820 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.820 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.820 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:46.820 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158525+0000", 2024-09-21T14:59:46.820 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:46.820 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.820 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.820 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:46.820 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158531+0000", 2024-09-21T14:59:46.820 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5840000000000002e-06 2024-09-21T14:59:46.820 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.820 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.820 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:46.820 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158564+0000", 2024-09-21T14:59:46.820 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3195000000000003e-05 2024-09-21T14:59:46.821 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.821 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.821 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:46.821 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158577+0000", 2024-09-21T14:59:46.821 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2330000000000001e-05 2024-09-21T14:59:46.821 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.821 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.821 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:46.821 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172956+0000", 2024-09-21T14:59:46.821 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014379438 2024-09-21T14:59:46.821 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.821 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.821 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:46.821 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173020+0000", 2024-09-21T14:59:46.822 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4401999999999998e-05 2024-09-21T14:59:46.822 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.822 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.822 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:46.822 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173188+0000", 2024-09-21T14:59:46.822 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016804 2024-09-21T14:59:46.822 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:46.822 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:46.822 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:46.822 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.822 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.822 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9379 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 24576~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:46.822 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158586+0000", 2024-09-21T14:59:46.823 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 67.628923741999998, 2024-09-21T14:59:46.823 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014865173000000001, 2024-09-21T14:59:46.823 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:46.823 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:46.823 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:46.823 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:46.823 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:46.823 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9379 2024-09-21T14:59:46.823 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.823 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:46.823 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.823 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:46.823 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158586+0000", 2024-09-21T14:59:46.823 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:46.824 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.824 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.824 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:46.824 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158586+0000", 2024-09-21T14:59:46.824 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:46.824 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.824 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.824 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:46.824 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158585+0000", 2024-09-21T14:59:46.824 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:46.824 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.824 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.824 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:46.824 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158590+0000", 2024-09-21T14:59:46.825 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5820000000000001e-06 2024-09-21T14:59:46.825 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.825 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.825 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:46.825 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158676+0000", 2024-09-21T14:59:46.825 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5075999999999997e-05 2024-09-21T14:59:46.825 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.825 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.825 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:46.825 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158695+0000", 2024-09-21T14:59:46.825 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9137999999999999e-05 2024-09-21T14:59:46.825 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.825 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.825 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:46.826 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173219+0000", 2024-09-21T14:59:46.826 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014524034999999999 2024-09-21T14:59:46.826 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.826 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.826 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:46.826 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173276+0000", 2024-09-21T14:59:46.826 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7442000000000003e-05 2024-09-21T14:59:46.826 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.826 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.826 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:46.826 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173451+0000", 2024-09-21T14:59:46.826 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017464800000000001 2024-09-21T14:59:46.826 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:46.826 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:46.827 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:46.827 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.827 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.827 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9380 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 163840~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:46.827 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158706+0000", 2024-09-21T14:59:46.827 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 67.628803231999996, 2024-09-21T14:59:46.827 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015096381000000001, 2024-09-21T14:59:46.827 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:46.827 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:46.827 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:46.827 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:46.827 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:46.827 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9380 2024-09-21T14:59:46.827 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.827 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:46.827 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.828 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:46.828 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158706+0000", 2024-09-21T14:59:46.828 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:46.828 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.828 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.828 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:46.828 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158706+0000", 2024-09-21T14:59:46.828 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:46.828 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.828 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.828 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:46.828 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158705+0000", 2024-09-21T14:59:46.828 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:46.828 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.828 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.829 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:46.829 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158712+0000", 2024-09-21T14:59:46.829 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7240000000000003e-06 2024-09-21T14:59:46.829 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.829 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.829 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:46.829 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158755+0000", 2024-09-21T14:59:46.829 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2953e-05 2024-09-21T14:59:46.829 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.829 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.829 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:46.829 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158765+0000", 2024-09-21T14:59:46.829 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0149000000000001e-05 2024-09-21T14:59:46.829 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.829 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.829 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:46.830 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173492+0000", 2024-09-21T14:59:46.830 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014727105000000001 2024-09-21T14:59:46.830 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.830 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.830 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:46.830 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173564+0000", 2024-09-21T14:59:46.830 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1789e-05 2024-09-21T14:59:46.830 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.830 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.830 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:46.830 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173803+0000", 2024-09-21T14:59:46.830 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00023867100000000001 2024-09-21T14:59:46.830 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:46.830 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:46.830 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:46.831 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.831 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.831 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9381 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 303104~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:46.831 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158774+0000", 2024-09-21T14:59:46.831 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 67.628735152000004, 2024-09-21T14:59:46.831 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015309095999999999, 2024-09-21T14:59:46.831 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:46.831 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:46.831 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:46.831 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:46.831 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:46.831 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9381 2024-09-21T14:59:46.831 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.831 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:46.832 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.832 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:46.832 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158774+0000", 2024-09-21T14:59:46.832 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:46.832 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.832 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.832 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:46.832 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158774+0000", 2024-09-21T14:59:46.832 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:46.832 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.832 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.832 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:46.832 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158773+0000", 2024-09-21T14:59:46.832 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:46.832 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.833 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.833 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:46.833 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158779+0000", 2024-09-21T14:59:46.833 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6999999999999996e-06 2024-09-21T14:59:46.833 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.833 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.833 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:46.833 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158790+0000", 2024-09-21T14:59:46.833 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0594000000000001e-05 2024-09-21T14:59:46.833 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.833 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.833 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:46.833 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158798+0000", 2024-09-21T14:59:46.834 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5860000000000005e-06 2024-09-21T14:59:46.834 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.834 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.834 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:46.834 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173829+0000", 2024-09-21T14:59:46.834 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015030349 2024-09-21T14:59:46.834 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.834 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.834 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:46.834 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173918+0000", 2024-09-21T14:59:46.834 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9375000000000004e-05 2024-09-21T14:59:46.834 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.834 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.835 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:46.835 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.174083+0000", 2024-09-21T14:59:46.835 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016536200000000001 2024-09-21T14:59:46.835 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:46.835 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:46.835 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:46.835 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.835 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.835 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9382 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 442368~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:46.835 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158805+0000", 2024-09-21T14:59:46.835 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 67.628704247000002, 2024-09-21T14:59:46.835 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015592623, 2024-09-21T14:59:46.835 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:46.835 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:46.836 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:46.836 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:46.836 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:46.836 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9382 2024-09-21T14:59:46.836 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.836 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:46.836 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.836 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:46.836 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158805+0000", 2024-09-21T14:59:46.836 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:46.836 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.836 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.836 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:46.836 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158805+0000", 2024-09-21T14:59:46.837 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:46.837 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.837 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.837 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:46.837 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158805+0000", 2024-09-21T14:59:46.837 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:46.837 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.837 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.837 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:46.837 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158808+0000", 2024-09-21T14:59:46.837 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9179999999999996e-06 2024-09-21T14:59:46.837 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.837 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.837 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:46.837 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158817+0000", 2024-09-21T14:59:46.838 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9299999999999992e-06 2024-09-21T14:59:46.838 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.838 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.838 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:46.838 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158825+0000", 2024-09-21T14:59:46.838 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6739999999999997e-06 2024-09-21T14:59:46.838 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.838 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.838 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:46.838 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.174122+0000", 2024-09-21T14:59:46.838 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.01529694 2024-09-21T14:59:46.838 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.838 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.838 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:46.839 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.174185+0000", 2024-09-21T14:59:46.839 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3015000000000006e-05 2024-09-21T14:59:46.839 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.839 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.839 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:46.839 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.174398+0000", 2024-09-21T14:59:46.839 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021274799999999999 2024-09-21T14:59:46.839 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:46.839 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:46.839 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:46.839 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.839 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.839 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22700 3.19 3:98de45b8:::rbd_data.11366acc894.0000000000000000:19 [read 704512~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:46.839 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.140191+0000", 2024-09-21T14:59:46.839 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 56.647318632999998, 2024-09-21T14:59:46.840 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013770486, 2024-09-21T14:59:46.840 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:46.840 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:46.840 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:46.840 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:46.840 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:46.840 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22700 2024-09-21T14:59:46.840 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.840 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:46.840 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.840 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:46.840 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140191+0000", 2024-09-21T14:59:46.840 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:46.841 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.841 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.841 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:46.841 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140191+0000", 2024-09-21T14:59:46.841 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:46.841 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.841 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.841 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:46.841 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140190+0000", 2024-09-21T14:59:46.841 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:46.841 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.841 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.841 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:46.841 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140199+0000", 2024-09-21T14:59:46.841 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3340000000000002e-06 2024-09-21T14:59:46.842 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.842 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.842 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:46.842 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140205+0000", 2024-09-21T14:59:46.842 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3820000000000001e-06 2024-09-21T14:59:46.842 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.842 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.842 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:46.842 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140210+0000", 2024-09-21T14:59:46.842 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0969999999999996e-06 2024-09-21T14:59:46.842 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.842 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.842 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:46.842 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.153713+0000", 2024-09-21T14:59:46.843 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013502713 2024-09-21T14:59:46.843 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.843 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.843 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:46.843 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.153786+0000", 2024-09-21T14:59:46.843 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3022999999999995e-05 2024-09-21T14:59:46.843 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.843 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.843 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:46.843 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.153961+0000", 2024-09-21T14:59:46.843 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000175274 2024-09-21T14:59:46.843 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:46.843 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:46.844 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:46.844 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.844 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.844 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22701 3.19 3:98de45b8:::rbd_data.11366acc894.0000000000000000:19 [read 770048~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:46.844 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.140215+0000", 2024-09-21T14:59:46.844 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 56.647294815999999, 2024-09-21T14:59:46.844 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014072497, 2024-09-21T14:59:46.844 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:46.844 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:46.844 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:46.844 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:46.844 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:46.844 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22701 2024-09-21T14:59:46.844 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.845 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:46.845 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.845 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:46.845 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140215+0000", 2024-09-21T14:59:46.845 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:46.845 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.845 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.845 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:46.845 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140215+0000", 2024-09-21T14:59:46.845 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:46.845 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.845 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.845 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:46.845 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140214+0000", 2024-09-21T14:59:46.845 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:46.846 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.846 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.846 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:46.846 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140217+0000", 2024-09-21T14:59:46.846 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.74e-06 2024-09-21T14:59:46.846 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.846 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.846 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:46.846 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140223+0000", 2024-09-21T14:59:46.846 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0929999999999999e-06 2024-09-21T14:59:46.846 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.846 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.846 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:46.846 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140228+0000", 2024-09-21T14:59:46.847 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2279999999999998e-06 2024-09-21T14:59:46.847 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.847 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.847 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:46.847 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.153991+0000", 2024-09-21T14:59:46.847 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013762329 2024-09-21T14:59:46.847 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.847 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.847 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:46.847 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.154098+0000", 2024-09-21T14:59:46.847 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000107117 2024-09-21T14:59:46.847 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.847 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.847 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:46.848 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.154287+0000", 2024-09-21T14:59:46.848 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000189431 2024-09-21T14:59:46.848 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:46.848 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:46.848 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:46.848 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.848 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.848 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22702 3.19 3:98de45b8:::rbd_data.11366acc894.0000000000000000:19 [read 835584~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:46.848 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.140232+0000", 2024-09-21T14:59:46.848 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 56.647276964, 2024-09-21T14:59:46.848 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015338742000000001, 2024-09-21T14:59:46.848 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:46.848 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:46.848 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:46.848 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:46.848 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:46.849 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22702 2024-09-21T14:59:46.849 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.849 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:46.849 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.849 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:46.849 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140232+0000", 2024-09-21T14:59:46.849 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:46.849 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.849 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.849 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:46.849 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140232+0000", 2024-09-21T14:59:46.849 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:46.849 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.849 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.849 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:46.849 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140232+0000", 2024-09-21T14:59:46.850 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:46.850 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.850 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.850 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:46.850 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140234+0000", 2024-09-21T14:59:46.850 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.402e-06 2024-09-21T14:59:46.850 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.850 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.850 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:46.850 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140241+0000", 2024-09-21T14:59:46.850 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.054e-06 2024-09-21T14:59:46.850 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.850 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.850 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:46.851 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140246+0000", 2024-09-21T14:59:46.851 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0050000000000004e-06 2024-09-21T14:59:46.851 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.851 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.851 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:46.851 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.154315+0000", 2024-09-21T14:59:46.851 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014069440000000001 2024-09-21T14:59:46.851 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.851 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.851 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:46.851 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.154383+0000", 2024-09-21T14:59:46.851 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8151000000000005e-05 2024-09-21T14:59:46.851 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.851 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.851 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:46.851 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155571+0000", 2024-09-21T14:59:46.852 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001188029 2024-09-21T14:59:46.852 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:46.852 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:46.852 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:46.852 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.852 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.852 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22703 3.19 3:98de45b8:::rbd_data.11366acc894.0000000000000000:19 [read 901120~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:46.852 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.140250+0000", 2024-09-21T14:59:46.852 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 56.647259806000001, 2024-09-21T14:59:46.852 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014992169, 2024-09-21T14:59:46.852 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:46.852 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:46.852 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:46.852 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:46.852 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:46.852 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22703 2024-09-21T14:59:46.853 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.853 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:46.853 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.853 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:46.853 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140250+0000", 2024-09-21T14:59:46.853 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:46.853 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.853 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.853 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:46.853 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140250+0000", 2024-09-21T14:59:46.853 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:46.853 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.853 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.853 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:46.853 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140249+0000", 2024-09-21T14:59:46.854 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:46.854 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.854 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.854 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:46.854 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140251+0000", 2024-09-21T14:59:46.854 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.136e-06 2024-09-21T14:59:46.854 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.854 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.854 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:46.854 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140257+0000", 2024-09-21T14:59:46.854 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9610000000000002e-06 2024-09-21T14:59:46.854 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.854 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.854 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:46.854 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140262+0000", 2024-09-21T14:59:46.855 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4549999999999997e-06 2024-09-21T14:59:46.855 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.855 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.855 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:46.855 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155006+0000", 2024-09-21T14:59:46.855 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014744189 2024-09-21T14:59:46.855 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.855 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.855 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:46.855 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155081+0000", 2024-09-21T14:59:46.855 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4548000000000005e-05 2024-09-21T14:59:46.855 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.855 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.855 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:46.855 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155242+0000", 2024-09-21T14:59:46.855 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000161204 2024-09-21T14:59:46.856 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:46.856 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:46.856 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:46.856 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.856 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.856 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22704 3.19 3:98de45b8:::rbd_data.11366acc894.0000000000000000:19 [read 966656~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:46.856 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.140266+0000", 2024-09-21T14:59:46.856 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 56.647243488999997, 2024-09-21T14:59:46.856 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015223443999999999, 2024-09-21T14:59:46.856 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:46.856 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:46.856 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:46.856 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:46.856 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:46.856 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22704 2024-09-21T14:59:46.856 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.857 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:46.857 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.857 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:46.857 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140266+0000", 2024-09-21T14:59:46.857 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:46.857 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.857 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.857 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:46.857 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140266+0000", 2024-09-21T14:59:46.857 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:46.857 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.857 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.857 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:46.857 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140266+0000", 2024-09-21T14:59:46.857 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:46.857 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.858 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.858 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:46.858 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140268+0000", 2024-09-21T14:59:46.858 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.475e-06 2024-09-21T14:59:46.858 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.858 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.858 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:46.858 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140274+0000", 2024-09-21T14:59:46.858 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.319e-06 2024-09-21T14:59:46.858 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.858 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.858 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:46.858 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140279+0000", 2024-09-21T14:59:46.858 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.882e-06 2024-09-21T14:59:46.858 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.858 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.859 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:46.859 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155264+0000", 2024-09-21T14:59:46.859 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014984655 2024-09-21T14:59:46.859 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.859 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.859 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:46.859 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155338+0000", 2024-09-21T14:59:46.859 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4091999999999995e-05 2024-09-21T14:59:46.859 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.859 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.859 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:46.859 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155489+0000", 2024-09-21T14:59:46.859 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000151349 2024-09-21T14:59:46.859 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:46.859 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:46.859 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:46.860 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.860 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.860 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22705 3.19 3:98de45b8:::rbd_data.11366acc894.0000000000000000:19 [read 1032192~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:46.860 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.140284+0000", 2024-09-21T14:59:46.860 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 56.647225632000001, 2024-09-21T14:59:46.860 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015467896, 2024-09-21T14:59:46.860 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:46.860 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:46.860 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:46.860 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:46.860 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:46.860 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22705 2024-09-21T14:59:46.860 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.860 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:46.860 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.860 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:46.860 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140284+0000", 2024-09-21T14:59:46.861 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:46.861 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.861 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.861 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:46.861 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140284+0000", 2024-09-21T14:59:46.861 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:46.861 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.861 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.861 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:46.861 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140283+0000", 2024-09-21T14:59:46.861 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:46.861 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.861 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.861 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:46.861 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140286+0000", 2024-09-21T14:59:46.861 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7130000000000001e-06 2024-09-21T14:59:46.862 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.862 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.862 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:46.862 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140292+0000", 2024-09-21T14:59:46.862 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3430000000000001e-06 2024-09-21T14:59:46.862 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.862 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.862 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:46.862 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140298+0000", 2024-09-21T14:59:46.862 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9730000000000002e-06 2024-09-21T14:59:46.862 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.862 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.862 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:46.862 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155510+0000", 2024-09-21T14:59:46.862 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015211738000000001 2024-09-21T14:59:46.862 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.863 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.863 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:46.863 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155598+0000", 2024-09-21T14:59:46.863 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7822e-05 2024-09-21T14:59:46.863 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.863 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.863 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:46.863 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155752+0000", 2024-09-21T14:59:46.863 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0001537 2024-09-21T14:59:46.863 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:46.863 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:46.863 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:46.863 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.863 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.863 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:37892 3.1b 3:d963c97f:::rbd_data.11365d19aa81.00000000000000be:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-21T14:59:46.864 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:04.029946+0000", 2024-09-21T14:59:46.864 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.757563865000002, 2024-09-21T14:59:46.864 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.017686021999999999, 2024-09-21T14:59:46.864 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:46.864 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:46.864 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:46.864 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:46.864 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:46.864 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 37892 2024-09-21T14:59:46.864 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.864 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:46.864 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.864 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:46.864 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.029946+0000", 2024-09-21T14:59:46.864 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:46.865 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.865 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.865 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:46.865 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.029946+0000", 2024-09-21T14:59:46.865 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:46.865 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.865 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.865 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:46.865 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.029944+0000", 2024-09-21T14:59:46.865 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T14:59:46.865 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.865 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.865 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:46.865 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.029953+0000", 2024-09-21T14:59:46.865 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7589999999999992e-06 2024-09-21T14:59:46.866 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.866 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.866 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:46.866 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.029971+0000", 2024-09-21T14:59:46.866 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7795000000000002e-05 2024-09-21T14:59:46.866 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.866 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.866 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:46.866 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.029984+0000", 2024-09-21T14:59:46.866 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.27e-05 2024-09-21T14:59:46.866 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.866 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.866 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:46.866 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.030072+0000", 2024-09-21T14:59:46.866 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8082000000000001e-05 2024-09-21T14:59:46.867 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.867 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.867 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:46.867 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.030276+0000", 2024-09-21T14:59:46.867 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00020375699999999999 2024-09-21T14:59:46.867 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.867 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.867 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-21T14:59:46.867 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.031634+0000", 2024-09-21T14:59:46.867 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0013586049999999999 2024-09-21T14:59:46.867 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.867 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.867 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:59:46.868 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.037554+0000", 2024-09-21T14:59:46.868 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0059197449999999997 2024-09-21T14:59:46.868 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.868 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.868 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:46.868 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.047490+0000", 2024-09-21T14:59:46.868 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0099361680000000004 2024-09-21T14:59:46.868 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.868 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.868 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:59:46.868 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.047569+0000", 2024-09-21T14:59:46.868 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8298000000000001e-05 2024-09-21T14:59:46.868 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.868 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.868 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:46.868 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.047632+0000", 2024-09-21T14:59:46.868 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3006e-05 2024-09-21T14:59:46.869 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:46.869 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:46.869 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:46.869 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.869 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.869 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:63820 3.19 3.626bc079 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-21T14:59:46.869 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:27.241293+0000", 2024-09-21T14:59:46.869 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 19.546216772000001, 2024-09-21T14:59:46.869 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.98723808199999996, 2024-09-21T14:59:46.869 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:46.869 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:46.869 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:46.869 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:46.869 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:46.869 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 63820 2024-09-21T14:59:46.870 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.870 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:46.870 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.870 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:46.870 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241293+0000", 2024-09-21T14:59:46.870 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:46.870 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.870 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.870 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:46.870 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241293+0000", 2024-09-21T14:59:46.870 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:46.870 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.870 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.871 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:46.871 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241291+0000", 2024-09-21T14:59:46.871 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T14:59:46.871 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.871 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.871 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:46.871 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241299+0000", 2024-09-21T14:59:46.871 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1529999999999997e-06 2024-09-21T14:59:46.871 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.871 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.871 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:46.871 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241317+0000", 2024-09-21T14:59:46.871 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7764000000000001e-05 2024-09-21T14:59:46.871 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.872 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.872 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:46.872 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241328+0000", 2024-09-21T14:59:46.872 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1245e-05 2024-09-21T14:59:46.872 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.872 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.872 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:46.872 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241407+0000", 2024-09-21T14:59:46.872 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8762000000000002e-05 2024-09-21T14:59:46.872 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.872 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.872 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T14:59:46.872 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241418+0000", 2024-09-21T14:59:46.872 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1695000000000001e-05 2024-09-21T14:59:46.872 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.873 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.873 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:46.873 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:28.228510+0000", 2024-09-21T14:59:46.873 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.98709166100000001 2024-09-21T14:59:46.873 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.873 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.873 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:46.873 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:28.228531+0000", 2024-09-21T14:59:46.873 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0696e-05 2024-09-21T14:59:46.873 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:46.873 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:46.873 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:46.873 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.873 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.873 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:73845 3.6 3:601bd01e:::rbd_data.1136f4939602.0000000000000015:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-21T14:59:46.874 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:42.393435+0000", 2024-09-21T14:59:46.874 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.3940746610000003, 2024-09-21T14:59:46.874 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.030713345, 2024-09-21T14:59:46.874 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:46.874 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:46.874 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:46.874 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:46.874 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:46.874 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 73845 2024-09-21T14:59:46.874 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.874 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:46.874 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.874 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:46.874 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.393435+0000", 2024-09-21T14:59:46.875 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:46.875 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.875 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.875 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:46.875 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.393435+0000", 2024-09-21T14:59:46.875 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:46.875 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.875 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.875 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:46.875 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.393433+0000", 2024-09-21T14:59:46.875 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T14:59:46.875 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.875 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.875 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:46.875 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.393446+0000", 2024-09-21T14:59:46.875 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3013000000000001e-05 2024-09-21T14:59:46.876 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.876 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.876 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:46.876 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.393475+0000", 2024-09-21T14:59:46.876 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9615999999999999e-05 2024-09-21T14:59:46.876 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.876 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.876 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:46.876 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.393489+0000", 2024-09-21T14:59:46.876 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4103e-05 2024-09-21T14:59:46.876 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.876 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.876 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:46.876 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.393565+0000", 2024-09-21T14:59:46.876 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5195000000000001e-05 2024-09-21T14:59:46.876 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.877 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.877 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:46.877 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.393734+0000", 2024-09-21T14:59:46.877 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016909999999999999 2024-09-21T14:59:46.877 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.877 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.877 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 7", 2024-09-21T14:59:46.877 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.400243+0000", 2024-09-21T14:59:46.877 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0065093629999999998 2024-09-21T14:59:46.877 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.877 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.877 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:59:46.877 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.423721+0000", 2024-09-21T14:59:46.877 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.023477984 2024-09-21T14:59:46.877 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.878 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.878 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:46.878 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.423852+0000", 2024-09-21T14:59:46.878 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000131369 2024-09-21T14:59:46.878 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.878 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.878 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:59:46.878 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.424011+0000", 2024-09-21T14:59:46.878 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000158231 2024-09-21T14:59:46.878 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:46.878 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:46.878 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:46.878 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.424148+0000", 2024-09-21T14:59:46.878 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013732699999999999 2024-09-21T14:59:46.878 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:46.878 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:46.879 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:46.879 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:46.879 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:46.879 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:59:46.880 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T14:59:46.881 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:59:46.881+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:59:46.981 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:59:46.975+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:59:47.039 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:59:47.039 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:59:47.039 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T14:59:47.039 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:59:47.049 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T14:59:47.094 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:59:47.091+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:59:47.162 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:59:47.162 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:59:47.162 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T14:59:47.162 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T14:59:47.162 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:59:47.171 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T14:59:47.183 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:59:47.179+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:59:47.284 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:59:47.281+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:59:47.289 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:59:47.289 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T14:59:47.289 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T14:59:47.289 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T14:59:47.289 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.289 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45834 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:47.289 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159133+0000", 2024-09-21T14:59:47.289 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 37.127086937999998, 2024-09-21T14:59:47.289 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068236884999999997, 2024-09-21T14:59:47.289 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:47.289 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:47.289 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:47.290 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:47.290 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:47.290 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45834 2024-09-21T14:59:47.290 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.290 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:47.290 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.290 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:47.290 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159133+0000", 2024-09-21T14:59:47.290 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:47.290 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.290 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.290 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:47.290 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159133+0000", 2024-09-21T14:59:47.290 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:47.291 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.291 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.291 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:47.291 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159133+0000", 2024-09-21T14:59:47.291 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:47.291 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.291 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.291 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:47.291 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159135+0000", 2024-09-21T14:59:47.291 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4500000000000001e-06 2024-09-21T14:59:47.291 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.291 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.291 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:47.291 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159138+0000", 2024-09-21T14:59:47.291 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2770000000000001e-06 2024-09-21T14:59:47.292 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.292 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.292 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:47.292 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159141+0000", 2024-09-21T14:59:47.292 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9210000000000001e-06 2024-09-21T14:59:47.292 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.292 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.292 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:47.292 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227361+0000", 2024-09-21T14:59:47.292 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068220154000000005 2024-09-21T14:59:47.292 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.292 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.292 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:47.292 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227370+0000", 2024-09-21T14:59:47.293 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3239999999999992e-06 2024-09-21T14:59:47.293 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:47.293 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:47.293 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:47.293 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.293 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.293 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45839 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:47.293 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159313+0000", 2024-09-21T14:59:47.293 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 37.126906972, 2024-09-21T14:59:47.293 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068238699, 2024-09-21T14:59:47.293 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:47.293 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:47.293 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:47.293 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:47.294 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:47.294 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45839 2024-09-21T14:59:47.294 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.294 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:47.294 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.294 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:47.294 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159313+0000", 2024-09-21T14:59:47.294 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:47.294 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.294 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.294 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:47.294 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159313+0000", 2024-09-21T14:59:47.294 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:47.294 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.294 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.295 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:47.295 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159313+0000", 2024-09-21T14:59:47.295 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:47.295 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.295 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.295 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:47.295 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159316+0000", 2024-09-21T14:59:47.295 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.033e-06 2024-09-21T14:59:47.295 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.295 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.295 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:47.295 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159322+0000", 2024-09-21T14:59:47.295 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1820000000000003e-06 2024-09-21T14:59:47.295 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.295 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.296 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:47.296 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159327+0000", 2024-09-21T14:59:47.296 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.062e-06 2024-09-21T14:59:47.296 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.296 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.296 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:47.296 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227544+0000", 2024-09-21T14:59:47.296 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068217142999999994 2024-09-21T14:59:47.296 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.296 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.296 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:47.296 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227552+0000", 2024-09-21T14:59:47.296 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6260000000000003e-06 2024-09-21T14:59:47.296 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:47.296 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:47.297 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:47.297 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.297 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.297 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45840 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:47.297 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159332+0000", 2024-09-21T14:59:47.297 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 37.126888675000004, 2024-09-21T14:59:47.297 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068262046000000007, 2024-09-21T14:59:47.297 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:47.297 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:47.297 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:47.297 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:47.297 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:47.297 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45840 2024-09-21T14:59:47.297 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.297 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:47.297 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.298 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:47.298 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159332+0000", 2024-09-21T14:59:47.298 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:47.298 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.298 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.298 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:47.298 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159332+0000", 2024-09-21T14:59:47.298 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:47.298 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.298 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.298 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:47.298 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159331+0000", 2024-09-21T14:59:47.298 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:47.298 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.299 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.299 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:47.299 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159351+0000", 2024-09-21T14:59:47.299 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0064999999999998e-05 2024-09-21T14:59:47.299 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.299 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.299 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:47.299 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159367+0000", 2024-09-21T14:59:47.299 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5308e-05 2024-09-21T14:59:47.299 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.299 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.299 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:47.299 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159373+0000", 2024-09-21T14:59:47.299 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1840000000000004e-06 2024-09-21T14:59:47.300 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.300 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.300 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:47.300 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227584+0000", 2024-09-21T14:59:47.300 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068211401000000005 2024-09-21T14:59:47.300 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.300 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.300 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:47.300 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227594+0000", 2024-09-21T14:59:47.300 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4120000000000007e-06 2024-09-21T14:59:47.300 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:47.300 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:47.300 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:47.300 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.301 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.301 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45841 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:47.301 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159379+0000", 2024-09-21T14:59:47.301 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 37.126841519000003, 2024-09-21T14:59:47.301 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068249039999999997, 2024-09-21T14:59:47.301 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:47.301 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:47.301 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:47.301 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:47.301 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:47.301 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45841 2024-09-21T14:59:47.301 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.301 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:47.302 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.302 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:47.302 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159379+0000", 2024-09-21T14:59:47.302 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:47.302 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.302 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.302 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:47.302 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159379+0000", 2024-09-21T14:59:47.302 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:47.302 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.302 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.302 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:47.302 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159378+0000", 2024-09-21T14:59:47.302 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:47.303 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.303 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.303 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:47.303 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159382+0000", 2024-09-21T14:59:47.303 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.641e-06 2024-09-21T14:59:47.303 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.303 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.303 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:47.303 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159421+0000", 2024-09-21T14:59:47.303 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9439e-05 2024-09-21T14:59:47.303 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.304 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.304 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:47.304 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159430+0000", 2024-09-21T14:59:47.304 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8429999999999999e-06 2024-09-21T14:59:47.304 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.304 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.304 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:47.304 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227617+0000", 2024-09-21T14:59:47.304 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068187084999999995 2024-09-21T14:59:47.304 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.304 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.304 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:47.304 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227628+0000", 2024-09-21T14:59:47.304 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0507e-05 2024-09-21T14:59:47.305 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:47.305 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:47.305 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:47.305 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.305 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.305 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45844 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:47.305 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159499+0000", 2024-09-21T14:59:47.305 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 37.126721132999997, 2024-09-21T14:59:47.305 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068255415, 2024-09-21T14:59:47.305 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:47.305 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:47.305 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:47.305 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:47.306 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:47.306 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45844 2024-09-21T14:59:47.306 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.306 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:47.306 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.306 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:47.306 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159499+0000", 2024-09-21T14:59:47.306 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:47.306 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.306 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.306 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:47.306 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159499+0000", 2024-09-21T14:59:47.307 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:47.307 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.307 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.307 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:47.307 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159499+0000", 2024-09-21T14:59:47.307 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:47.307 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.307 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.307 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:47.307 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159503+0000", 2024-09-21T14:59:47.307 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0999999999999997e-06 2024-09-21T14:59:47.307 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.307 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.307 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:47.308 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159512+0000", 2024-09-21T14:59:47.308 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5850000000000006e-06 2024-09-21T14:59:47.308 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.308 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.308 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:47.308 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159518+0000", 2024-09-21T14:59:47.308 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0909999999999998e-06 2024-09-21T14:59:47.308 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.308 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.308 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:47.308 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227743+0000", 2024-09-21T14:59:47.308 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068224536000000002 2024-09-21T14:59:47.308 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.309 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.309 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:47.309 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227755+0000", 2024-09-21T14:59:47.309 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1659e-05 2024-09-21T14:59:47.309 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:47.309 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:47.309 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:47.309 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.309 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.309 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45845 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:47.309 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159524+0000", 2024-09-21T14:59:47.309 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 37.126696772999999, 2024-09-21T14:59:47.309 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068267194000000003, 2024-09-21T14:59:47.309 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:47.309 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:47.310 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:47.310 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:47.310 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:47.310 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45845 2024-09-21T14:59:47.310 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.310 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:47.310 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.310 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:47.310 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159524+0000", 2024-09-21T14:59:47.310 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:47.310 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.310 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.310 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:47.310 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159524+0000", 2024-09-21T14:59:47.311 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:47.311 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.311 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.311 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:47.311 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159523+0000", 2024-09-21T14:59:47.311 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:47.311 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.311 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.311 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:47.311 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159526+0000", 2024-09-21T14:59:47.311 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1980000000000001e-06 2024-09-21T14:59:47.311 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.311 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.311 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:47.312 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159533+0000", 2024-09-21T14:59:47.312 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.348e-06 2024-09-21T14:59:47.312 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.312 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.312 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:47.312 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159546+0000", 2024-09-21T14:59:47.312 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3464e-05 2024-09-21T14:59:47.312 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.312 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.312 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:47.312 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227783+0000", 2024-09-21T14:59:47.312 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068236926000000003 2024-09-21T14:59:47.312 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.312 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.313 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:47.313 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227791+0000", 2024-09-21T14:59:47.313 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6669999999999996e-06 2024-09-21T14:59:47.313 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:47.313 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:47.313 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:47.313 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.313 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.313 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45846 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:47.313 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159584+0000", 2024-09-21T14:59:47.313 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 37.126636130999998, 2024-09-21T14:59:47.313 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068254767999999993, 2024-09-21T14:59:47.313 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:47.313 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:47.313 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:47.314 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:47.314 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:47.314 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45846 2024-09-21T14:59:47.314 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.314 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:47.314 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.314 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:47.314 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159584+0000", 2024-09-21T14:59:47.314 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:47.314 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.314 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.314 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:47.314 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159584+0000", 2024-09-21T14:59:47.315 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:47.315 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.315 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.315 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:47.315 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159583+0000", 2024-09-21T14:59:47.315 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:47.315 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.315 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.315 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:47.315 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159588+0000", 2024-09-21T14:59:47.315 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1330000000000001e-06 2024-09-21T14:59:47.315 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.315 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.315 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:47.315 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159596+0000", 2024-09-21T14:59:47.316 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5469999999999997e-06 2024-09-21T14:59:47.316 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.316 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.316 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:47.316 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159609+0000", 2024-09-21T14:59:47.316 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2785999999999999e-05 2024-09-21T14:59:47.316 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.316 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.316 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:47.316 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227830+0000", 2024-09-21T14:59:47.316 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068220879999999998 2024-09-21T14:59:47.316 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.316 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.316 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:47.316 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227839+0000", 2024-09-21T14:59:47.317 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1239999999999994e-06 2024-09-21T14:59:47.317 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:47.317 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:47.317 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:47.317 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.317 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.317 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45847 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:47.317 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159614+0000", 2024-09-21T14:59:47.317 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 37.126606055000003, 2024-09-21T14:59:47.317 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068260639999999997, 2024-09-21T14:59:47.317 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:47.317 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:47.317 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:47.317 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:47.317 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:47.317 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45847 2024-09-21T14:59:47.318 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.318 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:47.318 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.318 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:47.318 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159614+0000", 2024-09-21T14:59:47.318 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:47.318 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.318 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.318 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:47.318 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159614+0000", 2024-09-21T14:59:47.318 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:47.318 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.318 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.319 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:47.319 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159614+0000", 2024-09-21T14:59:47.319 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:47.319 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.319 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.319 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:47.319 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159617+0000", 2024-09-21T14:59:47.319 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2909999999999999e-06 2024-09-21T14:59:47.319 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.319 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.319 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:47.319 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159623+0000", 2024-09-21T14:59:47.319 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.511e-06 2024-09-21T14:59:47.319 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.320 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.320 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:47.320 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159631+0000", 2024-09-21T14:59:47.320 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.571e-06 2024-09-21T14:59:47.320 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.320 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.320 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:47.320 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227866+0000", 2024-09-21T14:59:47.320 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068235136000000002 2024-09-21T14:59:47.320 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.320 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.320 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:47.320 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227875+0000", 2024-09-21T14:59:47.320 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7199999999999995e-06 2024-09-21T14:59:47.320 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:47.320 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:47.321 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:47.321 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.321 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.321 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45848 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:47.321 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159636+0000", 2024-09-21T14:59:47.321 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 37.126584274999999, 2024-09-21T14:59:47.321 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068267283999999998, 2024-09-21T14:59:47.321 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:47.321 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:47.321 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:47.321 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:47.321 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:47.321 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45848 2024-09-21T14:59:47.321 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.321 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:47.322 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.322 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:47.322 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159636+0000", 2024-09-21T14:59:47.322 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:47.322 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.322 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.322 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:47.322 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159636+0000", 2024-09-21T14:59:47.322 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:47.322 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.322 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.322 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:47.322 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159636+0000", 2024-09-21T14:59:47.323 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:47.323 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.323 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.323 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:47.323 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159638+0000", 2024-09-21T14:59:47.323 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6560000000000001e-06 2024-09-21T14:59:47.323 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.323 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.323 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:47.323 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159644+0000", 2024-09-21T14:59:47.323 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0839999999999997e-06 2024-09-21T14:59:47.323 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.323 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.323 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:47.323 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159649+0000", 2024-09-21T14:59:47.324 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7500000000000003e-06 2024-09-21T14:59:47.324 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.324 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.324 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:47.324 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227892+0000", 2024-09-21T14:59:47.324 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068243315999999998 2024-09-21T14:59:47.324 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.324 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.324 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:47.324 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227903+0000", 2024-09-21T14:59:47.324 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0835999999999999e-05 2024-09-21T14:59:47.324 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:47.324 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:47.324 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:47.324 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.324 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.325 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45849 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:47.325 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159667+0000", 2024-09-21T14:59:47.325 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 37.126553461, 2024-09-21T14:59:47.325 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068291051000000005, 2024-09-21T14:59:47.325 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:47.325 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:47.325 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:47.325 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:47.325 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:47.325 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45849 2024-09-21T14:59:47.325 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.325 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:47.325 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.325 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:47.325 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159667+0000", 2024-09-21T14:59:47.326 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:47.326 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.326 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.326 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:47.326 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159667+0000", 2024-09-21T14:59:47.326 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:47.326 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.326 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.326 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:47.326 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159666+0000", 2024-09-21T14:59:47.326 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:47.326 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.326 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.326 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:47.326 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159670+0000", 2024-09-21T14:59:47.327 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.427e-06 2024-09-21T14:59:47.327 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.327 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.327 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:47.327 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159678+0000", 2024-09-21T14:59:47.327 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9869999999999999e-06 2024-09-21T14:59:47.327 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.327 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.327 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:47.327 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159688+0000", 2024-09-21T14:59:47.327 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0631999999999999e-05 2024-09-21T14:59:47.327 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.327 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.327 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:47.327 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227950+0000", 2024-09-21T14:59:47.327 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068261798999999998 2024-09-21T14:59:47.328 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.328 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.328 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:47.328 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227958+0000", 2024-09-21T14:59:47.328 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7510000000000008e-06 2024-09-21T14:59:47.328 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:47.328 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:47.328 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:47.328 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.328 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.328 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45850 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:47.328 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159695+0000", 2024-09-21T14:59:47.328 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 37.126525749999999, 2024-09-21T14:59:47.328 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068302999000000003, 2024-09-21T14:59:47.328 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:47.328 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:47.329 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:47.329 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:47.329 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:47.329 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45850 2024-09-21T14:59:47.329 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.329 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:47.329 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.329 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:47.329 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159695+0000", 2024-09-21T14:59:47.329 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:47.329 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.329 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.329 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:47.329 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159695+0000", 2024-09-21T14:59:47.329 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:47.330 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.330 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.330 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:47.330 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159694+0000", 2024-09-21T14:59:47.330 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:47.330 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.330 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.330 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:47.330 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159697+0000", 2024-09-21T14:59:47.330 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.258e-06 2024-09-21T14:59:47.330 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.330 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.330 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:47.330 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159704+0000", 2024-09-21T14:59:47.330 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7660000000000001e-06 2024-09-21T14:59:47.331 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.331 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.331 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:47.331 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159713+0000", 2024-09-21T14:59:47.331 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1779999999999992e-06 2024-09-21T14:59:47.331 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.331 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.331 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:47.331 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227989+0000", 2024-09-21T14:59:47.331 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068276015999999995 2024-09-21T14:59:47.331 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.331 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.331 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:47.331 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227998+0000", 2024-09-21T14:59:47.331 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2050000000000002e-06 2024-09-21T14:59:47.332 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:47.332 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:47.332 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:47.332 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.332 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.332 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45851 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:47.332 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159719+0000", 2024-09-21T14:59:47.332 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 37.126501617000002, 2024-09-21T14:59:47.332 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068320327, 2024-09-21T14:59:47.332 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:47.332 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:47.332 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:47.332 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:47.333 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:47.333 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45851 2024-09-21T14:59:47.333 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.333 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:47.333 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.333 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:47.333 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159719+0000", 2024-09-21T14:59:47.333 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:47.333 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.333 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.333 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:47.333 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159719+0000", 2024-09-21T14:59:47.333 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:47.333 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.334 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.334 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:47.334 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159718+0000", 2024-09-21T14:59:47.334 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:47.334 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.334 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.334 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:47.334 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159721+0000", 2024-09-21T14:59:47.334 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2169999999999999e-06 2024-09-21T14:59:47.334 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.334 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.334 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:47.335 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159730+0000", 2024-09-21T14:59:47.335 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5720000000000003e-06 2024-09-21T14:59:47.335 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.335 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.335 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:47.335 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159735+0000", 2024-09-21T14:59:47.335 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1100000000000002e-06 2024-09-21T14:59:47.335 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.335 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.335 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:47.335 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228029+0000", 2024-09-21T14:59:47.335 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068293624999999997 2024-09-21T14:59:47.335 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.335 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.336 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:47.336 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228039+0000", 2024-09-21T14:59:47.336 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0254e-05 2024-09-21T14:59:47.336 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:47.336 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:47.336 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:47.336 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.336 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.336 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45852 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:47.336 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159740+0000", 2024-09-21T14:59:47.336 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 37.126480448999999, 2024-09-21T14:59:47.336 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068340260999999999, 2024-09-21T14:59:47.336 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:47.336 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:47.336 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:47.336 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:47.337 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:47.337 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45852 2024-09-21T14:59:47.337 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.337 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:47.337 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.337 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:47.337 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159740+0000", 2024-09-21T14:59:47.337 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:47.337 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.337 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.337 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:47.337 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159740+0000", 2024-09-21T14:59:47.337 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:47.337 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.338 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.338 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:47.338 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159739+0000", 2024-09-21T14:59:47.338 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:47.338 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.338 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.338 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:47.338 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159757+0000", 2024-09-21T14:59:47.338 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7314999999999999e-05 2024-09-21T14:59:47.338 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.338 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.338 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:47.338 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159769+0000", 2024-09-21T14:59:47.338 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2376e-05 2024-09-21T14:59:47.339 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.339 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.339 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:47.339 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159775+0000", 2024-09-21T14:59:47.339 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2179999999999996e-06 2024-09-21T14:59:47.339 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.339 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.339 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:47.339 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228071+0000", 2024-09-21T14:59:47.339 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068295344999999993 2024-09-21T14:59:47.339 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.339 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.339 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:47.339 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228080+0000", 2024-09-21T14:59:47.339 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3959999999999995e-06 2024-09-21T14:59:47.340 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:47.340 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:47.340 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:47.340 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.340 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.340 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45853 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:47.340 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159781+0000", 2024-09-21T14:59:47.340 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 37.126439185000002, 2024-09-21T14:59:47.340 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068337093000000002, 2024-09-21T14:59:47.340 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:47.340 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:47.340 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:47.340 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:47.340 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:47.340 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45853 2024-09-21T14:59:47.340 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.341 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:47.341 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.341 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:47.341 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159781+0000", 2024-09-21T14:59:47.341 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:47.341 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.341 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.341 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:47.341 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159781+0000", 2024-09-21T14:59:47.341 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:47.341 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.341 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.341 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:47.341 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159781+0000", 2024-09-21T14:59:47.342 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:47.342 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.342 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.342 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:47.342 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159784+0000", 2024-09-21T14:59:47.342 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4759999999999998e-06 2024-09-21T14:59:47.342 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.342 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.342 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:47.342 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159792+0000", 2024-09-21T14:59:47.342 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0569999999999993e-06 2024-09-21T14:59:47.342 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.342 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.342 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:47.343 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159797+0000", 2024-09-21T14:59:47.343 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9640000000000002e-06 2024-09-21T14:59:47.343 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.343 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.343 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:47.343 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228111+0000", 2024-09-21T14:59:47.343 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068313737999999999 2024-09-21T14:59:47.343 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.343 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.343 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:47.343 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228118+0000", 2024-09-21T14:59:47.343 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.43e-06 2024-09-21T14:59:47.343 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:47.343 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:47.343 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:47.344 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.344 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.344 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45854 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:47.344 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159802+0000", 2024-09-21T14:59:47.344 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 37.126418534999999, 2024-09-21T14:59:47.344 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068358938999999994, 2024-09-21T14:59:47.344 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:47.344 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:47.344 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:47.344 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:47.344 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:47.344 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45854 2024-09-21T14:59:47.344 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.344 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:47.344 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.345 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:47.345 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159802+0000", 2024-09-21T14:59:47.345 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:47.345 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.345 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.345 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:47.345 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159802+0000", 2024-09-21T14:59:47.345 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:47.345 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.345 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.345 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:47.345 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159801+0000", 2024-09-21T14:59:47.345 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:47.345 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.345 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.345 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:47.346 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159804+0000", 2024-09-21T14:59:47.346 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0970000000000002e-06 2024-09-21T14:59:47.346 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.346 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.346 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:47.346 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159823+0000", 2024-09-21T14:59:47.346 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8895000000000001e-05 2024-09-21T14:59:47.346 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.346 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.346 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:47.346 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159837+0000", 2024-09-21T14:59:47.346 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3247999999999999e-05 2024-09-21T14:59:47.346 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.346 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.347 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:47.347 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228151+0000", 2024-09-21T14:59:47.347 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068314642999999994 2024-09-21T14:59:47.347 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.347 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.347 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:47.347 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228161+0000", 2024-09-21T14:59:47.347 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4250000000000004e-06 2024-09-21T14:59:47.347 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:47.347 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:47.347 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:47.347 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.347 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.347 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45983 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:47.348 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159848+0000", 2024-09-21T14:59:47.348 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 37.126372275999998, 2024-09-21T14:59:47.348 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068333801, 2024-09-21T14:59:47.348 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:47.348 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:47.348 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:47.348 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:47.348 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:47.348 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45983 2024-09-21T14:59:47.348 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.348 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:47.348 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.348 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:47.348 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159848+0000", 2024-09-21T14:59:47.348 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:47.349 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.349 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.349 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:47.349 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159848+0000", 2024-09-21T14:59:47.349 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:47.349 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.349 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.349 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:47.349 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159847+0000", 2024-09-21T14:59:47.349 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:47.349 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.349 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.349 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:47.349 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159856+0000", 2024-09-21T14:59:47.349 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4349999999999997e-06 2024-09-21T14:59:47.350 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.350 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.350 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:47.350 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159866+0000", 2024-09-21T14:59:47.350 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0076e-05 2024-09-21T14:59:47.350 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.350 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.350 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:47.350 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159872+0000", 2024-09-21T14:59:47.350 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8309999999999997e-06 2024-09-21T14:59:47.350 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.350 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.350 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:47.350 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228175+0000", 2024-09-21T14:59:47.351 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068302680000000005 2024-09-21T14:59:47.351 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.351 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.351 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:47.351 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228182+0000", 2024-09-21T14:59:47.351 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3150000000000003e-06 2024-09-21T14:59:47.351 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:47.351 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:47.351 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:47.351 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.351 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.351 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45984 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:47.351 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159877+0000", 2024-09-21T14:59:47.351 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 37.126342856000001, 2024-09-21T14:59:47.351 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068328448, 2024-09-21T14:59:47.352 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:47.352 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:47.352 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:47.352 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:47.352 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:47.352 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45984 2024-09-21T14:59:47.352 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.352 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:47.352 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.352 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:47.352 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159877+0000", 2024-09-21T14:59:47.352 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:47.352 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.352 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.353 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:47.353 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159877+0000", 2024-09-21T14:59:47.353 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:47.353 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.353 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.353 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:47.353 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159877+0000", 2024-09-21T14:59:47.353 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:47.353 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.353 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.353 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:47.353 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159880+0000", 2024-09-21T14:59:47.353 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7610000000000001e-06 2024-09-21T14:59:47.353 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.353 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.353 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:47.354 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159888+0000", 2024-09-21T14:59:47.354 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0330000000000003e-06 2024-09-21T14:59:47.354 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.354 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.354 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:47.354 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159893+0000", 2024-09-21T14:59:47.354 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3550000000000004e-06 2024-09-21T14:59:47.354 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.354 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.354 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:47.354 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228199+0000", 2024-09-21T14:59:47.354 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068306438999999997 2024-09-21T14:59:47.354 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.354 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.354 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:47.355 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228206+0000", 2024-09-21T14:59:47.355 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5599999999999999e-06 2024-09-21T14:59:47.355 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:47.355 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:47.355 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:47.355 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.355 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.355 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45985 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:47.355 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159897+0000", 2024-09-21T14:59:47.355 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 37.126323032999998, 2024-09-21T14:59:47.355 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068345664, 2024-09-21T14:59:47.355 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:47.355 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:47.356 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:47.356 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:47.356 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:47.356 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45985 2024-09-21T14:59:47.356 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.356 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:47.356 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.356 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:47.356 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159897+0000", 2024-09-21T14:59:47.356 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:47.356 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.356 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.357 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:47.357 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159897+0000", 2024-09-21T14:59:47.357 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:47.357 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.357 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.357 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:47.357 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159897+0000", 2024-09-21T14:59:47.357 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:47.357 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.357 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.357 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:47.357 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159900+0000", 2024-09-21T14:59:47.357 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.954e-06 2024-09-21T14:59:47.357 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.357 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.357 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:47.358 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159913+0000", 2024-09-21T14:59:47.358 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3106e-05 2024-09-21T14:59:47.358 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.358 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.358 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:47.358 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159918+0000", 2024-09-21T14:59:47.358 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5199999999999997e-06 2024-09-21T14:59:47.358 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.358 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.358 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:47.358 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228236+0000", 2024-09-21T14:59:47.358 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068317040999999995 2024-09-21T14:59:47.358 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.358 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.359 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:47.359 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228243+0000", 2024-09-21T14:59:47.359 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3880000000000002e-06 2024-09-21T14:59:47.359 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:47.361 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:47.361 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:47.361 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.361 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.361 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45986 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:47.361 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159924+0000", 2024-09-21T14:59:47.362 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 37.126296738000001, 2024-09-21T14:59:47.362 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068364362999999997, 2024-09-21T14:59:47.362 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:47.362 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:47.362 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:47.362 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:47.362 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:47.362 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45986 2024-09-21T14:59:47.362 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.362 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:47.362 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.362 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:47.362 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159924+0000", 2024-09-21T14:59:47.363 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:47.363 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.363 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.363 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:47.363 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159924+0000", 2024-09-21T14:59:47.363 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:47.363 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.363 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.363 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:47.363 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159923+0000", 2024-09-21T14:59:47.363 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:47.363 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.363 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.363 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:47.364 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159926+0000", 2024-09-21T14:59:47.364 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0510000000000001e-06 2024-09-21T14:59:47.364 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.364 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.364 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:47.364 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159932+0000", 2024-09-21T14:59:47.364 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4160000000000001e-06 2024-09-21T14:59:47.364 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.364 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.364 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:47.364 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159938+0000", 2024-09-21T14:59:47.364 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.383e-06 2024-09-21T14:59:47.365 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.365 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.365 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:47.365 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228275+0000", 2024-09-21T14:59:47.365 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068337007000000005 2024-09-21T14:59:47.365 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.365 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.365 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:47.365 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228288+0000", 2024-09-21T14:59:47.365 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3052e-05 2024-09-21T14:59:47.365 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:47.365 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:47.365 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:47.365 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.366 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.366 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45987 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:47.366 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159944+0000", 2024-09-21T14:59:47.366 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 37.126276513999997, 2024-09-21T14:59:47.366 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068387232000000006, 2024-09-21T14:59:47.366 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:47.366 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:47.366 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:47.366 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:47.366 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:47.366 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45987 2024-09-21T14:59:47.366 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.366 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:47.367 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.367 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:47.367 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159944+0000", 2024-09-21T14:59:47.367 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:47.367 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.367 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.367 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:47.367 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159944+0000", 2024-09-21T14:59:47.367 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:47.367 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.367 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.367 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:47.367 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159943+0000", 2024-09-21T14:59:47.368 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:47.368 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.368 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.368 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:47.368 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159946+0000", 2024-09-21T14:59:47.368 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6309999999999999e-06 2024-09-21T14:59:47.368 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.368 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.368 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:47.368 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159952+0000", 2024-09-21T14:59:47.368 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6540000000000004e-06 2024-09-21T14:59:47.368 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.368 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.369 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:47.369 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159957+0000", 2024-09-21T14:59:47.369 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9280000000000001e-06 2024-09-21T14:59:47.369 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.369 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.369 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:47.369 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228322+0000", 2024-09-21T14:59:47.369 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068365067000000002 2024-09-21T14:59:47.369 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.369 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.369 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:47.369 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228331+0000", 2024-09-21T14:59:47.369 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3270000000000007e-06 2024-09-21T14:59:47.369 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:47.370 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:47.370 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:47.370 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:47.370 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:47.370 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:59:47.371 INFO:teuthology.orchestra.run.smithi032.stderr:noscrub is unset 2024-09-21T14:59:47.371 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T14:59:47.430 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:59:47.429+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:59:47.485 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:59:47.485+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:59:47.493 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:59:47.493 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:59:47.493 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T14:59:47.493 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:59:47.503 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T14:59:47.587 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:59:47.583+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:59:47.606 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:59:47.606 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:59:47.606 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T14:59:47.607 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T14:59:47.607 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:59:47.617 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T14:59:47.688 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:59:47.683+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:59:47.706 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T14:59:47.730 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:59:47.730 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T14:59:47.731 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T14:59:47.731 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T14:59:47.731 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.731 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:10651 3.12 3:4e9d23af:::rbd_data.1136f5ed5dd6.0000000000000020:head [write 0~81920 [fadvise_sequential+fadvise_nocache],write 90112~8192 [fadvise_sequential+fadvise_nocache],write 122880~122880 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:47.731 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.292476+0000", 2024-09-21T14:59:47.731 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 68.434933635999997, 2024-09-21T14:59:47.731 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012765669, 2024-09-21T14:59:47.731 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:47.731 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:47.731 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:47.731 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:47.731 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:47.731 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 10651 2024-09-21T14:59:47.731 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.732 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:47.732 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.732 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:47.732 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292476+0000", 2024-09-21T14:59:47.732 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:47.732 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.732 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.732 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:47.732 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292476+0000", 2024-09-21T14:59:47.732 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:47.732 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.732 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.732 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:47.732 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292475+0000", 2024-09-21T14:59:47.733 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:47.733 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.733 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.733 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:47.733 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292696+0000", 2024-09-21T14:59:47.733 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000220201 2024-09-21T14:59:47.733 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.733 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.733 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:47.733 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292711+0000", 2024-09-21T14:59:47.733 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5803000000000001e-05 2024-09-21T14:59:47.733 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.733 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.733 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:47.734 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292721+0000", 2024-09-21T14:59:47.734 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6160000000000008e-06 2024-09-21T14:59:47.734 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.734 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.734 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:47.734 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292757+0000", 2024-09-21T14:59:47.734 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6211999999999998e-05 2024-09-21T14:59:47.734 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.734 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.734 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:47.734 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292878+0000", 2024-09-21T14:59:47.734 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000120306 2024-09-21T14:59:47.734 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.734 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.735 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 3", 2024-09-21T14:59:47.735 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.293019+0000", 2024-09-21T14:59:47.735 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014186400000000001 2024-09-21T14:59:47.735 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.735 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.735 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:59:47.735 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.294898+0000", 2024-09-21T14:59:47.735 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001878434 2024-09-21T14:59:47.735 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.735 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.735 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:47.735 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.305162+0000", 2024-09-21T14:59:47.735 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.01026373 2024-09-21T14:59:47.735 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.736 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.736 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:59:47.736 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.305209+0000", 2024-09-21T14:59:47.736 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7615999999999999e-05 2024-09-21T14:59:47.736 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.736 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.736 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:47.736 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.305242+0000", 2024-09-21T14:59:47.736 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2979999999999999e-05 2024-09-21T14:59:47.736 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:47.737 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:47.737 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:47.737 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.737 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.737 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:11285 3.12 3:4edf73d2:::rbd_data.1136f5ed5dd6.0000000000000039:head [write 90112~32768 [fadvise_sequential+fadvise_nocache],write 139264~24576 [fadvise_sequential+fadvise_nocache],write 180224~24576 [fadvise_sequential+fadvise_nocache],write 221184~40960 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:47.737 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.539470+0000", 2024-09-21T14:59:47.737 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 68.187939884000002, 2024-09-21T14:59:47.737 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0083360099999999996, 2024-09-21T14:59:47.737 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:47.737 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:47.737 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:47.737 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:47.737 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:47.737 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 11285 2024-09-21T14:59:47.738 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.738 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:47.738 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.738 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:47.738 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.539470+0000", 2024-09-21T14:59:47.738 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:47.738 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.738 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.738 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:47.738 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.539470+0000", 2024-09-21T14:59:47.738 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:47.738 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.738 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.738 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:47.739 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.539469+0000", 2024-09-21T14:59:47.739 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:47.739 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.739 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.739 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:47.739 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.539652+0000", 2024-09-21T14:59:47.739 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000183084 2024-09-21T14:59:47.739 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.739 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.739 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:47.739 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.539666+0000", 2024-09-21T14:59:47.739 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3947e-05 2024-09-21T14:59:47.739 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.739 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.740 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:47.740 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.539673+0000", 2024-09-21T14:59:47.740 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7309999999999996e-06 2024-09-21T14:59:47.740 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.740 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.740 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:47.740 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.539714+0000", 2024-09-21T14:59:47.740 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1539000000000003e-05 2024-09-21T14:59:47.740 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.740 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.740 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:47.740 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.539832+0000", 2024-09-21T14:59:47.740 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011753500000000001 2024-09-21T14:59:47.740 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.741 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.741 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 3", 2024-09-21T14:59:47.741 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.540005+0000", 2024-09-21T14:59:47.741 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000173031 2024-09-21T14:59:47.741 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.741 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.741 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:59:47.741 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.541606+0000", 2024-09-21T14:59:47.741 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001600899 2024-09-21T14:59:47.741 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.741 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.741 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:47.741 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.547709+0000", 2024-09-21T14:59:47.741 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0061027110000000002 2024-09-21T14:59:47.742 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.742 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.742 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:59:47.742 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.547767+0000", 2024-09-21T14:59:47.742 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8248999999999997e-05 2024-09-21T14:59:47.742 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.742 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.742 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:47.742 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.547806+0000", 2024-09-21T14:59:47.742 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9341000000000002e-05 2024-09-21T14:59:47.742 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:47.742 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:47.742 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:47.742 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.742 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.743 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12189 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 40960~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:47.743 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.618667+0000", 2024-09-21T14:59:47.743 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 68.108743434000004, 2024-09-21T14:59:47.743 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0083014549999999993, 2024-09-21T14:59:47.743 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:47.743 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:47.743 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:47.743 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:47.743 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:47.743 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12189 2024-09-21T14:59:47.743 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.743 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:47.743 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.743 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:47.744 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618667+0000", 2024-09-21T14:59:47.744 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:47.744 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.744 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.744 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:47.744 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618667+0000", 2024-09-21T14:59:47.744 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:47.744 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.744 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.744 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:47.744 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618666+0000", 2024-09-21T14:59:47.744 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:47.744 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.744 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.745 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:47.745 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618669+0000", 2024-09-21T14:59:47.745 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.554e-06 2024-09-21T14:59:47.745 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.745 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.745 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:47.745 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618675+0000", 2024-09-21T14:59:47.745 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2570000000000004e-06 2024-09-21T14:59:47.745 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.745 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.745 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:47.745 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618680+0000", 2024-09-21T14:59:47.745 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6539999999999998e-06 2024-09-21T14:59:47.745 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.746 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.746 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:47.746 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.626683+0000", 2024-09-21T14:59:47.746 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0080035880000000007 2024-09-21T14:59:47.746 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.746 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.746 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:47.746 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.626764+0000", 2024-09-21T14:59:47.746 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0672000000000003e-05 2024-09-21T14:59:47.746 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.746 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.746 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:47.746 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.626968+0000", 2024-09-21T14:59:47.746 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00020407499999999999 2024-09-21T14:59:47.746 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:47.747 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:47.747 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:47.747 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.747 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.747 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12193 3.4 3:25437d51:::rbd_data.11363a21d2be.0000000000000025:10 [read 425984~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:47.747 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.618755+0000", 2024-09-21T14:59:47.747 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 68.108655353000003, 2024-09-21T14:59:47.747 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086461379999999994, 2024-09-21T14:59:47.747 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:47.747 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:47.747 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:47.747 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:47.747 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:47.747 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12193 2024-09-21T14:59:47.748 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.748 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:47.748 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.748 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:47.748 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618755+0000", 2024-09-21T14:59:47.748 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:47.748 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.748 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.748 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:47.748 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618755+0000", 2024-09-21T14:59:47.748 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:47.748 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.748 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.748 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:47.749 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618754+0000", 2024-09-21T14:59:47.749 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:47.749 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.749 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.749 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:47.749 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618759+0000", 2024-09-21T14:59:47.749 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4020000000000004e-06 2024-09-21T14:59:47.749 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.749 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.749 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:47.749 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618767+0000", 2024-09-21T14:59:47.749 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7490000000000004e-06 2024-09-21T14:59:47.749 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.749 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.749 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:47.750 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618773+0000", 2024-09-21T14:59:47.750 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0939999999999996e-06 2024-09-21T14:59:47.750 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.750 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.750 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:47.750 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.627115+0000", 2024-09-21T14:59:47.750 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0083416840000000003 2024-09-21T14:59:47.750 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.750 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.750 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:47.750 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.627214+0000", 2024-09-21T14:59:47.750 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9042000000000005e-05 2024-09-21T14:59:47.750 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.750 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.751 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:47.751 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.627401+0000", 2024-09-21T14:59:47.751 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018677200000000001 2024-09-21T14:59:47.751 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:47.751 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:47.751 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:47.751 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.751 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.751 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12212 3.4 3:21b43b45:::rbd_data.11363a21d2be.000000000000002b:10 [read 237568~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:47.751 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.619922+0000", 2024-09-21T14:59:47.751 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 68.107488368999995, 2024-09-21T14:59:47.751 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.008324372, 2024-09-21T14:59:47.751 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:47.751 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:47.752 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:47.752 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:47.752 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:47.752 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12212 2024-09-21T14:59:47.752 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.752 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:47.752 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.752 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:47.752 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619922+0000", 2024-09-21T14:59:47.752 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:47.752 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.752 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.752 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:47.752 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619922+0000", 2024-09-21T14:59:47.752 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:47.753 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.753 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.753 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:47.753 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619921+0000", 2024-09-21T14:59:47.753 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:47.753 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.753 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.753 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:47.753 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619924+0000", 2024-09-21T14:59:47.753 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.999e-06 2024-09-21T14:59:47.753 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.753 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.753 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:47.753 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619934+0000", 2024-09-21T14:59:47.754 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.8120000000000002e-06 2024-09-21T14:59:47.754 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.754 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.754 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:47.754 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619939+0000", 2024-09-21T14:59:47.754 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7729999999999999e-06 2024-09-21T14:59:47.754 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.754 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.754 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:47.754 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628005+0000", 2024-09-21T14:59:47.754 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0080663330000000002 2024-09-21T14:59:47.754 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.754 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.754 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:47.755 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628073+0000", 2024-09-21T14:59:47.755 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7479999999999998e-05 2024-09-21T14:59:47.755 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.755 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.755 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:47.755 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628246+0000", 2024-09-21T14:59:47.755 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017337400000000001 2024-09-21T14:59:47.755 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:47.755 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:47.755 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:47.755 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.755 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.755 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12213 3.4 3:21b43b45:::rbd_data.11363a21d2be.000000000000002b:10 [read 270336~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:47.755 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.619945+0000", 2024-09-21T14:59:47.755 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 68.107465086999994, 2024-09-21T14:59:47.756 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086195049999999995, 2024-09-21T14:59:47.756 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:47.756 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:47.756 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:47.756 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:47.756 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:47.756 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12213 2024-09-21T14:59:47.756 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.756 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:47.756 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.756 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:47.756 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619945+0000", 2024-09-21T14:59:47.756 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:47.756 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.756 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.757 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:47.757 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619945+0000", 2024-09-21T14:59:47.757 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:47.757 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.757 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.757 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:47.757 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619945+0000", 2024-09-21T14:59:47.757 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:47.757 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.757 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.757 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:47.757 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619947+0000", 2024-09-21T14:59:47.757 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5780000000000001e-06 2024-09-21T14:59:47.757 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.757 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.758 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:47.758 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619957+0000", 2024-09-21T14:59:47.758 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3500000000000003e-06 2024-09-21T14:59:47.758 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.758 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.758 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:47.758 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619964+0000", 2024-09-21T14:59:47.758 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.199e-06 2024-09-21T14:59:47.758 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.758 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.758 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:47.758 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628285+0000", 2024-09-21T14:59:47.758 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0083210939999999994 2024-09-21T14:59:47.758 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.759 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.759 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:47.759 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628361+0000", 2024-09-21T14:59:47.759 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5873999999999999e-05 2024-09-21T14:59:47.759 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.759 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.759 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:47.759 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628565+0000", 2024-09-21T14:59:47.759 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000203799 2024-09-21T14:59:47.759 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:47.759 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:47.759 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:47.759 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.759 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.759 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12214 3.4 3:21b43b45:::rbd_data.11363a21d2be.000000000000002b:10 [read 376832~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:47.760 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.619969+0000", 2024-09-21T14:59:47.760 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 68.107441085999994, 2024-09-21T14:59:47.760 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0090292089999999998, 2024-09-21T14:59:47.760 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:47.760 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:47.760 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:47.760 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:47.760 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:47.760 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12214 2024-09-21T14:59:47.760 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.760 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:47.760 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.760 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:47.760 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619969+0000", 2024-09-21T14:59:47.760 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:47.761 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.761 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.761 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:47.761 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619969+0000", 2024-09-21T14:59:47.761 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:47.761 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.761 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.761 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:47.761 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619969+0000", 2024-09-21T14:59:47.761 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:47.761 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.761 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.761 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:47.761 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619971+0000", 2024-09-21T14:59:47.762 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1830000000000001e-06 2024-09-21T14:59:47.762 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.762 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.762 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:47.762 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619976+0000", 2024-09-21T14:59:47.762 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7129999999999996e-06 2024-09-21T14:59:47.762 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.762 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.762 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:47.762 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619981+0000", 2024-09-21T14:59:47.762 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2460000000000003e-06 2024-09-21T14:59:47.762 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.762 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.762 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:47.762 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628602+0000", 2024-09-21T14:59:47.763 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086209519999999994 2024-09-21T14:59:47.763 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.763 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.763 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:47.763 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628734+0000", 2024-09-21T14:59:47.763 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000132242 2024-09-21T14:59:47.763 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.763 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.763 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:47.763 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628998+0000", 2024-09-21T14:59:47.763 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00026419799999999999 2024-09-21T14:59:47.763 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:47.763 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:47.763 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:47.764 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.764 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.764 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12215 3.4 3:21b43b45:::rbd_data.11363a21d2be.000000000000002b:10 [read 409600~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:47.764 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.619985+0000", 2024-09-21T14:59:47.764 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 68.107425019999994, 2024-09-21T14:59:47.764 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0095457319999999995, 2024-09-21T14:59:47.764 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:47.764 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:47.764 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:47.764 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:47.764 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:47.764 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12215 2024-09-21T14:59:47.764 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.764 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:47.765 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.765 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:47.765 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619985+0000", 2024-09-21T14:59:47.765 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:47.765 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.765 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.765 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:47.765 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619985+0000", 2024-09-21T14:59:47.765 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:47.765 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.765 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.765 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:47.765 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619985+0000", 2024-09-21T14:59:47.765 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:47.765 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.766 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.766 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:47.766 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619987+0000", 2024-09-21T14:59:47.766 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6129999999999998e-06 2024-09-21T14:59:47.766 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.766 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.766 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:47.766 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619993+0000", 2024-09-21T14:59:47.766 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0820000000000004e-06 2024-09-21T14:59:47.766 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.766 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.766 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:47.766 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619998+0000", 2024-09-21T14:59:47.766 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7759999999999997e-06 2024-09-21T14:59:47.767 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.767 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.767 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:47.767 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629022+0000", 2024-09-21T14:59:47.767 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0090237489999999993 2024-09-21T14:59:47.767 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.767 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.767 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:47.767 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629098+0000", 2024-09-21T14:59:47.767 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5735999999999995e-05 2024-09-21T14:59:47.767 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.767 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.767 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:47.767 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629531+0000", 2024-09-21T14:59:47.767 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00043310100000000003 2024-09-21T14:59:47.768 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:47.768 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:47.768 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:47.768 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.768 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.768 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12216 3.4 3:21b43b45:::rbd_data.11363a21d2be.000000000000002b:10 [read 516096~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:47.768 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.620003+0000", 2024-09-21T14:59:47.768 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 68.107407023999997, 2024-09-21T14:59:47.768 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0098850169999999994, 2024-09-21T14:59:47.768 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:47.768 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:47.768 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:47.768 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:47.768 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:47.769 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12216 2024-09-21T14:59:47.769 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.769 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:47.769 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.769 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:47.769 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620003+0000", 2024-09-21T14:59:47.769 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:47.769 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.769 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.769 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:47.769 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620003+0000", 2024-09-21T14:59:47.769 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:47.769 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.770 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.770 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:47.770 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620003+0000", 2024-09-21T14:59:47.770 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:47.770 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.770 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.770 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:47.770 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620006+0000", 2024-09-21T14:59:47.770 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4350000000000001e-06 2024-09-21T14:59:47.770 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.770 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.770 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:47.770 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620013+0000", 2024-09-21T14:59:47.771 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4219999999999997e-06 2024-09-21T14:59:47.771 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.771 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.771 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:47.771 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620017+0000", 2024-09-21T14:59:47.771 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.865e-06 2024-09-21T14:59:47.771 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.771 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.771 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:47.771 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629581+0000", 2024-09-21T14:59:47.771 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0095631989999999997 2024-09-21T14:59:47.771 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.771 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.771 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:47.771 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629691+0000", 2024-09-21T14:59:47.772 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000110586 2024-09-21T14:59:47.772 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.772 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.772 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:47.772 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629888+0000", 2024-09-21T14:59:47.772 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019691199999999999 2024-09-21T14:59:47.772 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:47.772 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:47.772 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:47.772 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.772 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.772 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12222 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 49152~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:47.772 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.620027+0000", 2024-09-21T14:59:47.772 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 68.107382677999993, 2024-09-21T14:59:47.773 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010228131999999999, 2024-09-21T14:59:47.773 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:47.773 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:47.773 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:47.773 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:47.773 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:47.773 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12222 2024-09-21T14:59:47.773 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.773 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:47.773 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.773 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:47.773 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620027+0000", 2024-09-21T14:59:47.773 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:47.773 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.773 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.774 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:47.774 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620027+0000", 2024-09-21T14:59:47.774 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:47.774 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.774 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.774 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:47.774 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620027+0000", 2024-09-21T14:59:47.774 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:47.774 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.774 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.774 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:47.774 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620030+0000", 2024-09-21T14:59:47.774 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0400000000000001e-06 2024-09-21T14:59:47.774 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.775 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.775 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:47.775 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620037+0000", 2024-09-21T14:59:47.775 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8220000000000001e-06 2024-09-21T14:59:47.775 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.775 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.775 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:47.775 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620042+0000", 2024-09-21T14:59:47.775 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.843e-06 2024-09-21T14:59:47.775 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.775 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.775 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:47.775 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629922+0000", 2024-09-21T14:59:47.775 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0098805100000000003 2024-09-21T14:59:47.775 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.776 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.776 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:47.776 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.630020+0000", 2024-09-21T14:59:47.776 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7596999999999994e-05 2024-09-21T14:59:47.776 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.776 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.776 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:47.776 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.630256+0000", 2024-09-21T14:59:47.776 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00023573299999999999 2024-09-21T14:59:47.776 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:47.776 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:47.776 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:47.776 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.776 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.776 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12253 3.4 3:21b43b45:::rbd_data.11363a21d2be.000000000000002b:10 [read 417792~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:47.777 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.624578+0000", 2024-09-21T14:59:47.777 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 68.102832309999997, 2024-09-21T14:59:47.777 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0085112079999999993, 2024-09-21T14:59:47.777 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:47.777 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:47.777 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:47.777 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:47.777 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:47.777 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12253 2024-09-21T14:59:47.777 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.777 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:47.777 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.777 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:47.777 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624578+0000", 2024-09-21T14:59:47.777 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:47.778 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.778 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.778 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:47.778 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624578+0000", 2024-09-21T14:59:47.778 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:47.778 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.778 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.778 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:47.778 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624578+0000", 2024-09-21T14:59:47.778 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:47.778 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.778 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.778 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:47.778 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624579+0000", 2024-09-21T14:59:47.779 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.688e-06 2024-09-21T14:59:47.779 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.779 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.779 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:47.779 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624582+0000", 2024-09-21T14:59:47.779 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2260000000000001e-06 2024-09-21T14:59:47.779 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.779 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.779 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:47.779 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624585+0000", 2024-09-21T14:59:47.779 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4820000000000001e-06 2024-09-21T14:59:47.779 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.779 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.779 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:47.779 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.632580+0000", 2024-09-21T14:59:47.780 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0079948959999999996 2024-09-21T14:59:47.780 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.780 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.780 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:47.780 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.632683+0000", 2024-09-21T14:59:47.780 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010308 2024-09-21T14:59:47.780 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.780 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.780 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:47.780 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.633089+0000", 2024-09-21T14:59:47.780 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00040612599999999998 2024-09-21T14:59:47.780 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:47.780 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:47.780 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:47.781 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.781 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.781 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12260 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 57344~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:47.781 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.624587+0000", 2024-09-21T14:59:47.781 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 68.102822822999997, 2024-09-21T14:59:47.781 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0088282069999999994, 2024-09-21T14:59:47.781 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:47.781 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:47.781 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:47.781 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:47.781 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:47.781 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12260 2024-09-21T14:59:47.781 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.781 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:47.781 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.782 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:47.782 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624587+0000", 2024-09-21T14:59:47.782 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:47.782 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.782 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.782 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:47.782 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624587+0000", 2024-09-21T14:59:47.782 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:47.782 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.782 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.782 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:47.782 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624587+0000", 2024-09-21T14:59:47.782 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-21T14:59:47.783 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.783 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.783 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:47.783 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624589+0000", 2024-09-21T14:59:47.783 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4339999999999999e-06 2024-09-21T14:59:47.783 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.783 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.783 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:47.783 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624593+0000", 2024-09-21T14:59:47.783 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5859999999999998e-06 2024-09-21T14:59:47.783 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.783 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.783 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:47.783 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624596+0000", 2024-09-21T14:59:47.784 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5830000000000001e-06 2024-09-21T14:59:47.784 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.784 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.784 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:47.784 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.633132+0000", 2024-09-21T14:59:47.784 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.008536666 2024-09-21T14:59:47.784 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.784 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.784 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:47.784 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.633216+0000", 2024-09-21T14:59:47.784 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3387999999999995e-05 2024-09-21T14:59:47.784 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.784 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.784 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:47.785 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.633415+0000", 2024-09-21T14:59:47.785 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019976799999999999 2024-09-21T14:59:47.785 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:47.785 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:47.785 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:47.785 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.785 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.785 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12516 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 245760~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:47.785 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645194+0000", 2024-09-21T14:59:47.785 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 68.082216286000005, 2024-09-21T14:59:47.785 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0084284990000000008, 2024-09-21T14:59:47.785 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:47.785 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:47.785 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:47.785 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:47.786 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:47.786 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12516 2024-09-21T14:59:47.786 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.786 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:47.786 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.786 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:47.786 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645194+0000", 2024-09-21T14:59:47.786 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:47.786 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.786 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.786 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:47.786 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645194+0000", 2024-09-21T14:59:47.786 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:47.786 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.787 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.787 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:47.787 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645193+0000", 2024-09-21T14:59:47.787 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:47.787 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.787 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.787 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:47.787 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645196+0000", 2024-09-21T14:59:47.787 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1030000000000002e-06 2024-09-21T14:59:47.787 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.787 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.787 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:47.787 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645205+0000", 2024-09-21T14:59:47.787 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6200000000000005e-06 2024-09-21T14:59:47.787 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.788 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.788 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:47.788 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645210+0000", 2024-09-21T14:59:47.788 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6280000000000004e-06 2024-09-21T14:59:47.788 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.788 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.788 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:47.788 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.653305+0000", 2024-09-21T14:59:47.788 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0080957879999999996 2024-09-21T14:59:47.788 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.788 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.788 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:47.788 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.653389+0000", 2024-09-21T14:59:47.788 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3242e-05 2024-09-21T14:59:47.788 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.789 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.789 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:47.789 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.653622+0000", 2024-09-21T14:59:47.789 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00023361599999999999 2024-09-21T14:59:47.789 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:47.789 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:47.789 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:47.789 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.789 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.789 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12517 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 278528~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:47.789 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645214+0000", 2024-09-21T14:59:47.789 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 68.082196173, 2024-09-21T14:59:47.789 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086832899999999998, 2024-09-21T14:59:47.789 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:47.790 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:47.790 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:47.790 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:47.790 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:47.790 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12517 2024-09-21T14:59:47.790 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.790 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:47.790 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.790 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:47.790 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645214+0000", 2024-09-21T14:59:47.790 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:47.790 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.790 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.790 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:47.791 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645214+0000", 2024-09-21T14:59:47.791 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:47.791 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.791 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.791 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:47.791 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645214+0000", 2024-09-21T14:59:47.791 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:47.791 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.791 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.791 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:47.791 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645216+0000", 2024-09-21T14:59:47.791 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5579999999999999e-06 2024-09-21T14:59:47.791 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.791 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.791 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:47.792 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645224+0000", 2024-09-21T14:59:47.792 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2439999999999993e-06 2024-09-21T14:59:47.792 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.792 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.792 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:47.792 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645231+0000", 2024-09-21T14:59:47.792 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1859999999999997e-06 2024-09-21T14:59:47.792 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.792 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.792 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:47.792 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.653651+0000", 2024-09-21T14:59:47.792 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0084204220000000003 2024-09-21T14:59:47.792 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.792 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.792 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:47.793 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.653726+0000", 2024-09-21T14:59:47.793 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5356000000000002e-05 2024-09-21T14:59:47.793 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.793 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.793 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:47.793 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.653897+0000", 2024-09-21T14:59:47.793 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000170848 2024-09-21T14:59:47.793 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:47.793 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:47.793 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:47.793 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.793 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.793 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12518 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 385024~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:47.794 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645235+0000", 2024-09-21T14:59:47.794 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 68.082174946999999, 2024-09-21T14:59:47.794 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0089841020000000008, 2024-09-21T14:59:47.794 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:47.794 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:47.794 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:47.794 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:47.794 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:47.794 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12518 2024-09-21T14:59:47.794 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.794 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:47.794 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.794 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:47.794 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645235+0000", 2024-09-21T14:59:47.795 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:47.795 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.795 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.795 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:47.795 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645235+0000", 2024-09-21T14:59:47.795 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:47.795 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.795 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.795 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:47.795 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645235+0000", 2024-09-21T14:59:47.795 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:47.795 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.795 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.796 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:47.796 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645238+0000", 2024-09-21T14:59:47.796 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.869e-06 2024-09-21T14:59:47.796 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.796 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.796 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:47.796 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645244+0000", 2024-09-21T14:59:47.796 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8930000000000002e-06 2024-09-21T14:59:47.796 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.796 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.796 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:47.796 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645249+0000", 2024-09-21T14:59:47.796 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2859999999999999e-06 2024-09-21T14:59:47.796 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.796 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.797 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:47.797 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.653930+0000", 2024-09-21T14:59:47.797 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086815320000000005 2024-09-21T14:59:47.797 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.797 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.797 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:47.797 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654028+0000", 2024-09-21T14:59:47.797 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7578000000000007e-05 2024-09-21T14:59:47.797 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.797 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.797 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:47.797 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654219+0000", 2024-09-21T14:59:47.797 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019132099999999999 2024-09-21T14:59:47.797 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:47.797 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:47.798 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:47.798 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.798 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.798 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12538 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 147456~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:47.798 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645253+0000", 2024-09-21T14:59:47.798 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 68.082157323000004, 2024-09-21T14:59:47.798 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0092095360000000008, 2024-09-21T14:59:47.798 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:47.798 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:47.798 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:47.798 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:47.798 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:47.798 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12538 2024-09-21T14:59:47.798 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.798 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:47.799 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.799 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:47.799 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645253+0000", 2024-09-21T14:59:47.799 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:47.799 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.799 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.799 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:47.799 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645253+0000", 2024-09-21T14:59:47.799 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:47.799 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.799 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.799 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:47.799 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645252+0000", 2024-09-21T14:59:47.799 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:47.799 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.800 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.800 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:47.800 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645255+0000", 2024-09-21T14:59:47.800 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2290000000000002e-06 2024-09-21T14:59:47.800 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.800 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.800 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:47.800 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645267+0000", 2024-09-21T14:59:47.800 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2631e-05 2024-09-21T14:59:47.800 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.800 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.800 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:47.800 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645278+0000", 2024-09-21T14:59:47.800 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0847e-05 2024-09-21T14:59:47.800 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.801 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.801 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:47.801 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654251+0000", 2024-09-21T14:59:47.801 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0089731340000000007 2024-09-21T14:59:47.801 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.801 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.801 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:47.801 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654296+0000", 2024-09-21T14:59:47.801 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4353e-05 2024-09-21T14:59:47.801 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.801 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.801 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:47.801 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654462+0000", 2024-09-21T14:59:47.801 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016668499999999999 2024-09-21T14:59:47.801 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:47.801 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:47.802 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:47.802 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.802 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.802 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12539 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 253952~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:47.802 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645284+0000", 2024-09-21T14:59:47.802 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 68.082126102000004, 2024-09-21T14:59:47.802 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0094524029999999998, 2024-09-21T14:59:47.802 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:47.802 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:47.802 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:47.802 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:47.802 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:47.802 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12539 2024-09-21T14:59:47.802 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.803 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:47.803 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.803 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:47.803 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645284+0000", 2024-09-21T14:59:47.803 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:47.803 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.803 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.803 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:47.803 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645284+0000", 2024-09-21T14:59:47.803 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:47.803 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.803 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.803 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:47.803 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645284+0000", 2024-09-21T14:59:47.803 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:47.804 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.804 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.804 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:47.804 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645286+0000", 2024-09-21T14:59:47.804 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7159999999999999e-06 2024-09-21T14:59:47.804 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.804 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.804 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:47.804 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645292+0000", 2024-09-21T14:59:47.804 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7400000000000001e-06 2024-09-21T14:59:47.804 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.804 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.804 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:47.804 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645297+0000", 2024-09-21T14:59:47.805 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9019999999999998e-06 2024-09-21T14:59:47.805 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.805 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.805 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:47.805 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654490+0000", 2024-09-21T14:59:47.805 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0091932360000000005 2024-09-21T14:59:47.805 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.805 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.805 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:47.805 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654565+0000", 2024-09-21T14:59:47.805 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4657e-05 2024-09-21T14:59:47.805 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.805 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.805 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:47.805 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654736+0000", 2024-09-21T14:59:47.806 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017156199999999999 2024-09-21T14:59:47.806 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:47.806 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:47.806 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:47.806 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.806 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.806 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12540 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 286720~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:47.806 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645302+0000", 2024-09-21T14:59:47.806 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 68.082107832999995, 2024-09-21T14:59:47.806 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0096603090000000006, 2024-09-21T14:59:47.806 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:47.806 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:47.806 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:47.806 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:47.806 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:47.807 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12540 2024-09-21T14:59:47.807 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.807 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:47.807 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.807 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:47.807 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645302+0000", 2024-09-21T14:59:47.807 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:47.807 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.807 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.807 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:47.807 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645302+0000", 2024-09-21T14:59:47.807 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:47.807 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.807 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.807 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:47.807 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645302+0000", 2024-09-21T14:59:47.808 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:47.808 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.808 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.808 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:47.808 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645305+0000", 2024-09-21T14:59:47.808 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7329999999999999e-06 2024-09-21T14:59:47.808 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.808 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.808 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:47.808 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645316+0000", 2024-09-21T14:59:47.808 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1832e-05 2024-09-21T14:59:47.808 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.808 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.808 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:47.808 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645322+0000", 2024-09-21T14:59:47.809 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.169e-06 2024-09-21T14:59:47.809 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.809 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.809 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:47.809 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654778+0000", 2024-09-21T14:59:47.809 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0094559090000000002 2024-09-21T14:59:47.809 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.809 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.809 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:47.809 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654828+0000", 2024-09-21T14:59:47.809 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0461999999999999e-05 2024-09-21T14:59:47.809 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.809 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.809 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:47.809 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654963+0000", 2024-09-21T14:59:47.809 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000134583 2024-09-21T14:59:47.810 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:47.810 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:47.810 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:47.810 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.810 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.810 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12561 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:47.810 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645742+0000", 2024-09-21T14:59:47.810 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 68.081668566999994, 2024-09-21T14:59:47.810 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0093583810000000007, 2024-09-21T14:59:47.810 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:47.810 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:47.810 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:47.810 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:47.810 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:47.810 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12561 2024-09-21T14:59:47.811 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.811 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:47.811 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.811 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:47.811 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645742+0000", 2024-09-21T14:59:47.811 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:47.811 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.811 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.811 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:47.811 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645742+0000", 2024-09-21T14:59:47.811 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:47.811 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.811 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.811 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:47.811 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645741+0000", 2024-09-21T14:59:47.811 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:47.812 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.812 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.812 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:47.812 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645746+0000", 2024-09-21T14:59:47.812 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8379999999999998e-06 2024-09-21T14:59:47.812 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.812 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.812 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:47.812 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645757+0000", 2024-09-21T14:59:47.812 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0705e-05 2024-09-21T14:59:47.812 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.812 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.812 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:47.812 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645765+0000", 2024-09-21T14:59:47.812 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6729999999999992e-06 2024-09-21T14:59:47.813 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.813 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.813 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:47.813 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654993+0000", 2024-09-21T14:59:47.813 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0092287480000000002 2024-09-21T14:59:47.813 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.813 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.813 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:47.813 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.655050+0000", 2024-09-21T14:59:47.813 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6394999999999998e-05 2024-09-21T14:59:47.813 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.813 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.813 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:47.813 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.655100+0000", 2024-09-21T14:59:47.813 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0049000000000001e-05 2024-09-21T14:59:47.813 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:47.814 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:47.814 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:47.814 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.814 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.814 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:15431 3.12 3:4bc7d206:::rbd_data.1136f5ed5dd6.0000000000000051:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:59:47.814 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:41.673445+0000", 2024-09-21T14:59:47.814 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 66.053965233, 2024-09-21T14:59:47.814 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086103610000000004, 2024-09-21T14:59:47.814 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:47.814 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:47.814 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:47.814 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:47.814 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:47.814 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 15431 2024-09-21T14:59:47.815 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.815 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:47.815 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.815 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:47.815 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.673445+0000", 2024-09-21T14:59:47.815 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:47.815 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.815 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.815 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:47.815 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.673445+0000", 2024-09-21T14:59:47.815 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:47.815 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.815 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.815 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:47.815 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.673444+0000", 2024-09-21T14:59:47.816 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:59:47.816 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.816 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.816 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:47.816 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.673451+0000", 2024-09-21T14:59:47.816 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4739999999999999e-06 2024-09-21T14:59:47.816 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.816 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.816 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:47.816 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.673472+0000", 2024-09-21T14:59:47.816 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0687e-05 2024-09-21T14:59:47.816 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.816 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.816 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:47.816 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.673481+0000", 2024-09-21T14:59:47.817 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3349999999999996e-06 2024-09-21T14:59:47.817 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.817 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.817 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:47.817 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.673565+0000", 2024-09-21T14:59:47.817 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4011000000000006e-05 2024-09-21T14:59:47.817 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.817 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.817 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:47.817 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.673650+0000", 2024-09-21T14:59:47.817 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4949000000000003e-05 2024-09-21T14:59:47.817 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.817 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.817 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 3", 2024-09-21T14:59:47.817 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.675014+0000", 2024-09-21T14:59:47.817 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0013636620000000001 2024-09-21T14:59:47.818 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.818 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.818 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:47.818 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.681866+0000", 2024-09-21T14:59:47.818 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0068525640000000002 2024-09-21T14:59:47.818 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.818 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.818 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:59:47.818 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.681885+0000", 2024-09-21T14:59:47.818 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8519999999999999e-05 2024-09-21T14:59:47.818 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.818 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.818 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:59:47.818 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.681961+0000", 2024-09-21T14:59:47.819 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6248000000000006e-05 2024-09-21T14:59:47.819 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:47.819 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:47.819 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:47.819 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.682055+0000", 2024-09-21T14:59:47.819 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4184000000000005e-05 2024-09-21T14:59:47.819 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:47.819 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:47.819 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:47.819 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:47.819 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:47.819 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:59:47.821 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T14:59:47.821 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:59:47.821+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:59:47.864 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T14:59:47.886 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T14:59:47.905 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T14:59:47.921 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:59:47.919+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:59:47.926 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T14:59:47.947 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-21T14:59:47.969 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T14:59:47.978 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:59:47.978 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:59:47.978 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T14:59:47.978 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:59:47.988 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T14:59:47.992 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T14:59:48.014 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T14:59:48.016 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:59:48.016 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:59:48.016 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:59:48.016 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:59:48.016 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:59:48.016 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:59:48.016 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:59:48.017 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:59:48.023 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:59:48.019+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:59:48.096 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:48.096 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:48.105 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T14:59:48.124 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:59:48.125+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:59:48.225 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:59:48.219+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:59:48.260 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:48.261 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:48.271 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T14:59:48.326 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:59:48.323+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:59:48.396 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:48.397 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:48.408 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T14:59:48.427 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:59:48.425+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:59:48.519 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:48.520 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:48.521 INFO:teuthology.orchestra.run.smithi032.stdout:checking clone #18, image image_client.0-clone18 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent19 2024-09-21T14:59:48.528 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:59:48.529+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:59:48.530 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T14:59:48.628 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:59:48.623+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:59:48.693 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:48.694 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:48.704 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T14:59:48.729 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:59:48.729+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:59:48.811 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:48.811 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:48.821 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T14:59:48.830 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:59:48.827+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:59:48.899 INFO:teuthology.orchestra.run.smithi032.stdout:2183 trunc from 0x89bd085 to 0x8e8f4d 2024-09-21T14:59:48.920 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:48.930 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:59:48.929+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:59:48.942 INFO:teuthology.orchestra.run.smithi032.stdout:2188 writesame 0x943e5f1 thru 0x9442b24 data_size 0x1(0x4534 bytes) 2024-09-21T14:59:48.942 INFO:teuthology.orchestra.run.smithi032.stdout:2189 read 0xaa4ff2 thru 0xaa5bac (0xbbb bytes) 2024-09-21T14:59:48.943 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:48.943 INFO:teuthology.orchestra.run.smithi032.stdout:2191 write 0x5d7e3cc thru 0x5d83bcd (0x5802 bytes) 2024-09-21T14:59:48.952 INFO:teuthology.orchestra.run.smithi032.stdout:2193 write 0x28d6a91 thru 0x28e2ee1 (0xc451 bytes) 2024-09-21T14:59:48.960 INFO:teuthology.orchestra.run.smithi032.stdout:2194 read 0x641fc05 thru 0x6420a69 (0xe65 bytes) 2024-09-21T14:59:48.971 INFO:teuthology.orchestra.run.smithi032.stdout:2195 writesame 0xb228e4b thru 0xb236752 data_size 0x3c(0xd908 bytes) 2024-09-21T14:59:48.982 INFO:teuthology.orchestra.run.smithi032.stdout:2196 trunc from 0xb236753 to 0x443a3b8 2024-09-21T14:59:48.986 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:48.986 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:48.994 INFO:teuthology.orchestra.run.smithi032.stdout:2198 punch from 0x2d8cb55 to 0x2d9a4c7, (0xd972 bytes) 2024-09-21T14:59:48.994 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:48.994 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:48.994 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:49.003 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T14:59:49.011 INFO:teuthology.orchestra.run.smithi032.stdout:2203 write 0xa2351c5 thru 0xa23ddb0 (0x8bec bytes) 2024-09-21T14:59:49.016 INFO:teuthology.orchestra.run.smithi032.stdout:2204 read 0x70aec8e thru 0x70b575b (0x6ace bytes) 2024-09-21T14:59:49.029 INFO:teuthology.orchestra.run.smithi032.stdout:2205 write 0xc65ea17 thru 0xc663894 (0x4e7e bytes) 2024-09-21T14:59:49.031 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:59:49.029+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:59:49.036 INFO:teuthology.orchestra.run.smithi032.stdout:2206 write 0x3e22b52 thru 0x3e32586 (0xfa35 bytes) 2024-09-21T14:59:49.042 INFO:teuthology.orchestra.run.smithi032.stdout:2207 read 0xcb9d9e thru 0xcba9dc (0xc3f bytes) 2024-09-21T14:59:49.043 INFO:teuthology.orchestra.run.smithi032.stdout:2208 writesame 0xa1b91fa thru 0xa1c8e41 data_size 0x1b(0xfc48 bytes) 2024-09-21T14:59:49.051 INFO:teuthology.orchestra.run.smithi032.stdout:2209 write 0x36bdc7e thru 0x36c9244 (0xb5c7 bytes) 2024-09-21T14:59:49.058 INFO:teuthology.orchestra.run.smithi032.stdout:2211 read 0x4395230 thru 0x43a0387 (0xb158 bytes) 2024-09-21T14:59:49.059 INFO:teuthology.orchestra.run.smithi032.stdout:2212 read 0x7b5c833 thru 0x7b69792 (0xcf60 bytes) 2024-09-21T14:59:49.059 INFO:teuthology.orchestra.run.smithi032.stdout:2213 punch from 0xc53b3f4 to 0xc5460f6, (0xad02 bytes) 2024-09-21T14:59:49.059 INFO:teuthology.orchestra.run.smithi032.stdout:2214 write 0x6952604 thru 0x695e14d (0xbb4a bytes) 2024-09-21T14:59:49.067 INFO:teuthology.orchestra.run.smithi032.stdout:2215 read 0x93aee47 thru 0x93bc509 (0xd6c3 bytes) 2024-09-21T14:59:49.067 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:49.067 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:49.067 INFO:teuthology.orchestra.run.smithi032.stdout:2220 read 0x960ec10 thru 0x96153bb (0x67ac bytes) 2024-09-21T14:59:49.067 INFO:teuthology.orchestra.run.smithi032.stdout:2222 writesame 0x28e76c4 thru 0x28efda7 data_size 0x4(0x86e4 bytes) 2024-09-21T14:59:49.077 INFO:teuthology.orchestra.run.smithi032.stdout:2224 write 0x586b4ab thru 0x5871c30 (0x6786 bytes) 2024-09-21T14:59:49.086 INFO:teuthology.orchestra.run.smithi032.stdout:2226 trunc from 0xc663895 to 0x8b9b54a 2024-09-21T14:59:49.102 INFO:teuthology.orchestra.run.smithi032.stdout:2227 write 0x7f04c27 thru 0x7f04f69 (0x343 bytes) 2024-09-21T14:59:49.111 INFO:teuthology.orchestra.run.smithi032.stdout:2228 trunc from 0x8b9b54a to 0x7191122 2024-09-21T14:59:49.113 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:49.113 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:49.123 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T14:59:49.124 INFO:teuthology.orchestra.run.smithi032.stdout:2229 read 0xdae7b0 thru 0xdb0d8b (0x25dc bytes) 2024-09-21T14:59:49.124 INFO:teuthology.orchestra.run.smithi032.stdout:2230 write 0x714e5ba thru 0x7152a4d (0x4494 bytes) 2024-09-21T14:59:49.128 INFO:teuthology.orchestra.run.smithi032.stdout:2231 write 0x691bf8d thru 0x69278ad (0xb921 bytes) 2024-09-21T14:59:49.132 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:59:49.133+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:59:49.136 INFO:teuthology.orchestra.run.smithi032.stdout:2232 writesame 0x50e5e3a thru 0x50f34ab data_size 0x1(0xd672 bytes) 2024-09-21T14:59:49.182 INFO:teuthology.orchestra.run.smithi032.stdout:2234 punch from 0x123eecc to 0x1240ca5, (0x1dd9 bytes) 2024-09-21T14:59:49.183 INFO:teuthology.orchestra.run.smithi032.stdout:2237 read 0x4d7897a thru 0x4d84d92 (0xc419 bytes) 2024-09-21T14:59:49.183 INFO:teuthology.orchestra.run.smithi032.stdout:2240 writesame 0x64cf015 thru 0x64da4cc data_size 0x4(0xb4b8 bytes) 2024-09-21T14:59:49.198 INFO:teuthology.orchestra.run.smithi032.stdout:2241 read 0x5d1690c thru 0x5d213f1 (0xaae6 bytes) 2024-09-21T14:59:49.198 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:49.198 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:49.211 INFO:teuthology.orchestra.run.smithi032.stdout:2244 writesame 0x920e70c thru 0x9218462 data_size 0x1(0x9d57 bytes) 2024-09-21T14:59:49.212 INFO:teuthology.orchestra.run.smithi032.stdout:2245 punch from 0x5645f4 to 0x567c7d, (0x3689 bytes) 2024-09-21T14:59:49.212 INFO:teuthology.orchestra.run.smithi032.stdout:2246 write 0x87436c3 thru 0x87525a7 (0xeee5 bytes) 2024-09-21T14:59:49.220 INFO:teuthology.orchestra.run.smithi032.stdout:2249 read 0x6ec4745 thru 0x6ed2b77 (0xe433 bytes) 2024-09-21T14:59:49.233 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:59:49.227+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:59:49.238 INFO:teuthology.orchestra.run.smithi032.stdout:2250 write 0xd3cc2ee thru 0xd3d7c76 (0xb989 bytes) 2024-09-21T14:59:49.247 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:49.247 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:49.248 INFO:teuthology.orchestra.run.smithi032.stdout:2251 trunc from 0xd3d7c77 to 0xaaf31ab 2024-09-21T14:59:49.259 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T14:59:49.276 INFO:teuthology.orchestra.run.smithi032.stdout:2252 write 0xb57f7eb thru 0xb581458 (0x1c6e bytes) 2024-09-21T14:59:49.300 INFO:teuthology.orchestra.run.smithi032.stdout:2254 trunc from 0xb581459 to 0x235eea8 2024-09-21T14:59:49.323 INFO:teuthology.orchestra.run.smithi032.stdout:2255 read 0x1f05944 thru 0x1f12e37 (0xd4f4 bytes) 2024-09-21T14:59:49.334 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:59:49.331+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:59:49.343 INFO:teuthology.orchestra.run.smithi032.stdout:2256 write 0x9237e88 thru 0x923ea59 (0x6bd2 bytes) 2024-09-21T14:59:49.349 INFO:teuthology.orchestra.run.smithi032.stdout:2258 read 0x402119a thru 0x402a8ed (0x9754 bytes) 2024-09-21T14:59:49.349 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:49.349 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:49.349 INFO:teuthology.orchestra.run.smithi032.stdout:2263 write 0x671f56c thru 0x67258c4 (0x6359 bytes) 2024-09-21T14:59:49.358 INFO:teuthology.orchestra.run.smithi032.stdout:2267 trunc from 0x923ea5a to 0xa4fc5b9 2024-09-21T14:59:49.370 INFO:teuthology.orchestra.run.smithi032.stdout:2268 writesame 0xf1e5d thru 0xfe7cb data_size 0x1(0xc96f bytes) 2024-09-21T14:59:49.375 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:49.376 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:49.382 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T14:59:49.389 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T14:59:49.399 INFO:teuthology.orchestra.run.smithi032.stdout:2269 punch from 0xceb0f3 to 0xcf4b4c, (0x9a59 bytes) 2024-09-21T14:59:49.408 INFO:teuthology.orchestra.run.smithi032.stdout:2272 write 0xb9ea8bf thru 0xb9f5692 (0xadd4 bytes) 2024-09-21T14:59:49.415 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:49.429 INFO:teuthology.orchestra.run.smithi032.stdout:2274 writesame 0xea0bb27 thru 0xea11d0a data_size 0x46(0x61e4 bytes) 2024-09-21T14:59:49.429 INFO:teuthology.orchestra.run.smithi032.stdout:2275 writesame 0xadff34e thru 0xae0867c data_size 0x1(0x932f bytes) 2024-09-21T14:59:49.435 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:59:49.431+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:59:49.458 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:49.458 INFO:teuthology.orchestra.run.smithi032.stdout:2277 read 0x63fe8db thru 0x6403eec (0x5612 bytes) 2024-09-21T14:59:49.459 INFO:teuthology.orchestra.run.smithi032.stdout:2279 read 0x5fd09f thru 0x608bc4 (0xbb26 bytes) 2024-09-21T14:59:49.460 INFO:teuthology.orchestra.run.smithi032.stdout:2281 read 0x10337b5 thru 0x1036a58 (0x32a4 bytes) 2024-09-21T14:59:49.461 INFO:teuthology.orchestra.run.smithi032.stdout:2282 read 0xd5ad43d thru 0xd5aee28 (0x19ec bytes) 2024-09-21T14:59:49.461 INFO:teuthology.orchestra.run.smithi032.stdout:2283 read 0xc699c3 thru 0xc75d53 (0xc391 bytes) 2024-09-21T14:59:49.462 INFO:teuthology.orchestra.run.smithi032.stdout:2284 writesame 0x241274e thru 0x2414a32 data_size 0x1(0x22e5 bytes) 2024-09-21T14:59:49.477 INFO:teuthology.orchestra.run.smithi032.stdout:2285 punch from 0x8ffc6dd to 0x8ffd4cb, (0xdee bytes) 2024-09-21T14:59:49.478 INFO:teuthology.orchestra.run.smithi032.stdout:2286 punch from 0xc48e0be to 0xc49b3be, (0xd300 bytes) 2024-09-21T14:59:49.478 INFO:teuthology.orchestra.run.smithi032.stdout:2287 writesame 0x5de8710 thru 0x5de8bc1 data_size 0x1(0x4b2 bytes) 2024-09-21T14:59:49.485 INFO:teuthology.orchestra.run.smithi032.stdout:2290 read 0xd168565 thru 0xd1687ca (0x266 bytes) 2024-09-21T14:59:49.490 INFO:teuthology.orchestra.run.smithi032.stdout:2291 trunc from 0xea11d0b to 0xc002dd8 2024-09-21T14:59:49.493 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 5, 6, 7, 4] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T14:59:49.493 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T14:59:49.493 INFO:tasks.thrashosds.thrasher:inject_pause on osd.3 2024-09-21T14:59:49.493 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-21T14:59:49.493 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-21T14:59:49.494 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok config set filestore_inject_stall 3 2024-09-21T14:59:49.499 INFO:teuthology.orchestra.run.smithi032.stdout:2292 punch from 0x6521e09 to 0x652c2b3, (0xa4aa bytes) 2024-09-21T14:59:49.508 INFO:teuthology.orchestra.run.smithi032.stdout:2293 write 0xc19c719 thru 0xc1a5381 (0x8c69 bytes) 2024-09-21T14:59:49.515 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:49.515 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:49.516 INFO:teuthology.orchestra.run.smithi032.stdout:2294 write 0x2675f0b thru 0x2685e53 (0xff49 bytes) 2024-09-21T14:59:49.524 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:49.524 INFO:teuthology.orchestra.run.smithi032.stdout:2296 read 0xb29a7ea thru 0xb2a0244 (0x5a5b bytes) 2024-09-21T14:59:49.528 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T14:59:49.533 INFO:teuthology.orchestra.run.smithi032.stdout:2297 write 0xcccd912 thru 0xccd5317 (0x7a06 bytes) 2024-09-21T14:59:49.535 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:59:49.533+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:59:49.546 INFO:teuthology.orchestra.run.smithi032.stdout:2299 trunc from 0xccd5318 to 0x7fc3084 2024-09-21T14:59:49.562 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:49.562 INFO:teuthology.orchestra.run.smithi032.stdout:2302 write 0x6a34939 thru 0x6a3c189 (0x7851 bytes) 2024-09-21T14:59:49.577 INFO:teuthology.orchestra.run.smithi032.stdout:2303 trunc from 0x7fc3084 to 0x14b8fd7 2024-09-21T14:59:49.598 INFO:teuthology.orchestra.run.smithi032.stdout:2304 read 0x92b847 thru 0x9340ad (0x8867 bytes) 2024-09-21T14:59:49.610 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:49.611 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-21T14:59:49.611 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:49.625 INFO:teuthology.orchestra.run.smithi032.stdout:2305 write 0xd05fbe2 thru 0xd06a0f1 (0xa510 bytes) 2024-09-21T14:59:49.633 INFO:teuthology.orchestra.run.smithi032.stdout:2306 read 0x5632608 thru 0x563fe1b (0xd814 bytes) 2024-09-21T14:59:49.633 INFO:teuthology.orchestra.run.smithi032.stdout:2308 read 0x29354ba thru 0x2940df9 (0xb940 bytes) 2024-09-21T14:59:49.633 INFO:teuthology.orchestra.run.smithi032.stdout:2309 write 0x18d4b7e thru 0x18e3b8f (0xf012 bytes) 2024-09-21T14:59:49.636 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:59:49.637+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:59:49.637 INFO:teuthology.orchestra.run.smithi032.stdout:2310 read 0x15e9f69 thru 0x15f3df7 (0x9e8f bytes) 2024-09-21T14:59:49.638 INFO:teuthology.orchestra.run.smithi032.stdout:2312 clone 21 order 21 su 524288 sc 11 2024-09-21T14:59:49.685 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:49.685 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:49.695 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T14:59:49.738 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:59:49.731+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:59:49.838 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:59:49.837+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:59:49.858 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:49.858 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:49.868 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T14:59:49.939 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:59:49.937+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:59:49.976 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:49.976 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:49.986 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T14:59:50.028 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T14:59:50.041 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:59:50.035+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:59:50.108 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:50.108 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:50.119 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T14:59:50.138 INFO:teuthology.orchestra.run.smithi032.stderr:nodeep-scrub is unset 2024-09-21T14:59:50.142 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:59:50.141+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:59:50.181 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T14:59:50.206 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T14:59:50.221 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:50.222 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:50.223 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T14:59:50.231 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T14:59:50.241 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T14:59:50.243 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:59:50.241+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:59:50.259 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-21T14:59:50.280 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T14:59:50.301 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T14:59:50.323 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T14:59:50.325 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:59:50.325 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:59:50.325 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:59:50.325 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:59:50.326 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:59:50.326 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:59:50.326 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:59:50.326 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:59:50.343 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:59:50.343 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T14:59:50.343 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T14:59:50.343 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T14:59:50.344 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.344 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:1452 3.19 3:9d40fe72:::rbd_data.1136515ecb6b.0000000000000013:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T14:59:50.344 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:30.292755+0000", 2024-09-21T14:59:50.344 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 80.047341707000001, 2024-09-21T14:59:50.344 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043852191999999998, 2024-09-21T14:59:50.344 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:50.344 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:50.344 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:50.344 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:50.344 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:50.344 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1452 2024-09-21T14:59:50.344 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.344 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:50.344 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.344 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:50.345 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292755+0000", 2024-09-21T14:59:50.345 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:50.345 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.345 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.345 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:50.345 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292755+0000", 2024-09-21T14:59:50.345 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:50.345 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.345 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.345 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:50.345 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292754+0000", 2024-09-21T14:59:50.345 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:50.345 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.345 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.345 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:50.345 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292760+0000", 2024-09-21T14:59:50.346 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5029999999999997e-06 2024-09-21T14:59:50.346 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.346 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.346 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:50.346 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292771+0000", 2024-09-21T14:59:50.346 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1551e-05 2024-09-21T14:59:50.346 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.346 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.346 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:50.346 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292779+0000", 2024-09-21T14:59:50.346 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1559999999999998e-06 2024-09-21T14:59:50.346 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.346 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.347 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:50.347 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292896+0000", 2024-09-21T14:59:50.347 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000117304 2024-09-21T14:59:50.347 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.347 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.347 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:50.347 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.293124+0000", 2024-09-21T14:59:50.347 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00022819999999999999 2024-09-21T14:59:50.347 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.347 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.347 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 1", 2024-09-21T14:59:50.347 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.299967+0000", 2024-09-21T14:59:50.347 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0068427009999999996 2024-09-21T14:59:50.347 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.348 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.348 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:50.348 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336463+0000", 2024-09-21T14:59:50.348 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.036495842000000001 2024-09-21T14:59:50.348 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.348 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.348 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:59:50.348 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336480+0000", 2024-09-21T14:59:50.348 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7419999999999999e-05 2024-09-21T14:59:50.348 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.348 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.348 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:59:50.348 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336550+0000", 2024-09-21T14:59:50.348 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9951999999999997e-05 2024-09-21T14:59:50.348 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.349 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.349 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:50.349 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336607+0000", 2024-09-21T14:59:50.349 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7423000000000002e-05 2024-09-21T14:59:50.349 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:50.349 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:50.349 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:50.349 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.349 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.349 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop_reply(client.4406.0:1452 3.19 e36/24 ondisk, result = 0)", 2024-09-21T14:59:50.349 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:30.322854+0000", 2024-09-21T14:59:50.349 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 80.017243031000007, 2024-09-21T14:59:50.349 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013613079, 2024-09-21T14:59:50.349 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:50.349 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:50.350 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:50.350 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.350 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:50.350 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.322854+0000", 2024-09-21T14:59:50.350 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:50.350 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.350 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.350 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:50.350 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.322854+0000", 2024-09-21T14:59:50.350 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:50.350 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.350 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.350 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:50.351 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.322852+0000", 2024-09-21T14:59:50.351 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:59:50.351 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.351 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.351 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:50.351 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.322860+0000", 2024-09-21T14:59:50.351 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.605e-06 2024-09-21T14:59:50.351 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.351 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.351 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:50.351 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.322878+0000", 2024-09-21T14:59:50.351 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8204000000000001e-05 2024-09-21T14:59:50.351 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.351 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.352 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:50.352 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.322888+0000", 2024-09-21T14:59:50.352 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4099999999999997e-06 2024-09-21T14:59:50.352 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.352 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.352 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:50.352 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336440+0000", 2024-09-21T14:59:50.352 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013551848 2024-09-21T14:59:50.352 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.352 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.352 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:50.352 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336453+0000", 2024-09-21T14:59:50.352 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3465000000000001e-05 2024-09-21T14:59:50.352 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.353 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.353 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:50.353 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336467+0000", 2024-09-21T14:59:50.353 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3913999999999999e-05 2024-09-21T14:59:50.353 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:50.353 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:50.353 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:50.353 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.353 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.353 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:1567 3.8 3:139d3d55:::rbd_data.1136515ecb6b.0000000000000000:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T14:59:50.353 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:30.681087+0000", 2024-09-21T14:59:50.353 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 79.659009491999996, 2024-09-21T14:59:50.353 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.039009224000000002, 2024-09-21T14:59:50.353 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:50.353 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:50.354 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:50.354 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:50.354 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:50.354 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1567 2024-09-21T14:59:50.354 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.354 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:50.354 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.354 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:50.354 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681087+0000", 2024-09-21T14:59:50.354 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:50.354 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.354 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.354 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:50.354 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681087+0000", 2024-09-21T14:59:50.354 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:50.355 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.355 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.355 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:50.355 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681086+0000", 2024-09-21T14:59:50.355 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:59:50.355 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.355 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.355 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:50.355 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681097+0000", 2024-09-21T14:59:50.355 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.134e-05 2024-09-21T14:59:50.355 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.355 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.355 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:50.355 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681122+0000", 2024-09-21T14:59:50.355 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5313e-05 2024-09-21T14:59:50.356 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.356 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.356 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:50.356 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681138+0000", 2024-09-21T14:59:50.356 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5916000000000001e-05 2024-09-21T14:59:50.356 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.356 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.356 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:50.356 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681270+0000", 2024-09-21T14:59:50.356 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000131474 2024-09-21T14:59:50.356 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.356 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.356 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:50.356 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681416+0000", 2024-09-21T14:59:50.356 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014637 2024-09-21T14:59:50.357 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.357 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.357 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 7", 2024-09-21T14:59:50.357 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.693163+0000", 2024-09-21T14:59:50.357 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011747241 2024-09-21T14:59:50.357 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.357 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.357 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:59:50.357 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.719750+0000", 2024-09-21T14:59:50.357 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.026586852000000001 2024-09-21T14:59:50.357 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.357 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.357 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:50.357 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.719859+0000", 2024-09-21T14:59:50.358 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010925999999999999 2024-09-21T14:59:50.358 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.358 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.358 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:59:50.358 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.719951+0000", 2024-09-21T14:59:50.358 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1294999999999999e-05 2024-09-21T14:59:50.358 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.358 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.358 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:50.358 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.720097+0000", 2024-09-21T14:59:50.358 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014583 2024-09-21T14:59:50.358 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:50.358 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:50.358 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:50.359 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.359 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.359 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9348 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 122880~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:50.359 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158369+0000", 2024-09-21T14:59:50.359 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 71.181727823000003, 2024-09-21T14:59:50.359 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.01388847, 2024-09-21T14:59:50.359 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:50.359 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:50.359 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:50.359 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:50.359 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:50.359 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9348 2024-09-21T14:59:50.359 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.359 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:50.360 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.360 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:50.360 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158369+0000", 2024-09-21T14:59:50.360 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:50.360 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.360 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.360 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:50.360 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158369+0000", 2024-09-21T14:59:50.360 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:50.360 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.360 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.360 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:50.360 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158368+0000", 2024-09-21T14:59:50.360 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:50.361 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.361 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.361 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:50.361 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158374+0000", 2024-09-21T14:59:50.361 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0499999999999997e-06 2024-09-21T14:59:50.361 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.361 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.361 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:50.361 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158386+0000", 2024-09-21T14:59:50.361 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1579999999999999e-05 2024-09-21T14:59:50.361 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.361 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.361 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:50.361 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158396+0000", 2024-09-21T14:59:50.361 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.8260000000000005e-06 2024-09-21T14:59:50.362 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.362 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.362 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:50.362 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.171967+0000", 2024-09-21T14:59:50.362 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013571318000000001 2024-09-21T14:59:50.362 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.362 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.362 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:50.362 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172034+0000", 2024-09-21T14:59:50.362 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.656e-05 2024-09-21T14:59:50.362 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.362 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.362 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:50.362 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172258+0000", 2024-09-21T14:59:50.363 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00022395999999999999 2024-09-21T14:59:50.363 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:50.363 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:50.363 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:50.363 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.363 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.363 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9349 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 155648~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:50.363 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158403+0000", 2024-09-21T14:59:50.363 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 71.181694077000003, 2024-09-21T14:59:50.363 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014229851, 2024-09-21T14:59:50.363 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:50.363 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:50.363 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:50.363 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:50.364 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:50.364 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9349 2024-09-21T14:59:50.364 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.364 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:50.364 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.364 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:50.364 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158403+0000", 2024-09-21T14:59:50.364 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:50.364 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.364 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.364 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:50.364 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158403+0000", 2024-09-21T14:59:50.364 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:50.364 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.364 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.365 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:50.365 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158402+0000", 2024-09-21T14:59:50.365 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:50.365 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.365 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.365 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:50.365 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158406+0000", 2024-09-21T14:59:50.365 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1339999999999997e-06 2024-09-21T14:59:50.365 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.365 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.365 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:50.365 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158416+0000", 2024-09-21T14:59:50.365 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3149999999999998e-06 2024-09-21T14:59:50.365 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.366 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.366 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:50.366 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158439+0000", 2024-09-21T14:59:50.366 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3255999999999999e-05 2024-09-21T14:59:50.366 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.366 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.366 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:50.366 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172305+0000", 2024-09-21T14:59:50.366 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013866603999999999 2024-09-21T14:59:50.366 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.366 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.366 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:50.366 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172413+0000", 2024-09-21T14:59:50.366 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000107664 2024-09-21T14:59:50.367 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.367 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.367 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:50.367 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172633+0000", 2024-09-21T14:59:50.367 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021950800000000001 2024-09-21T14:59:50.367 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:50.367 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:50.367 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:50.367 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.367 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.367 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9350 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 294912~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:50.367 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158448+0000", 2024-09-21T14:59:50.367 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 71.181649206000003, 2024-09-21T14:59:50.367 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014471525000000001, 2024-09-21T14:59:50.368 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:50.368 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:50.368 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:50.368 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:50.368 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:50.368 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9350 2024-09-21T14:59:50.368 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.368 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:50.368 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.368 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:50.368 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158448+0000", 2024-09-21T14:59:50.368 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:50.368 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.368 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.368 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:50.369 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158448+0000", 2024-09-21T14:59:50.369 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:50.369 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.369 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.369 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:50.369 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158447+0000", 2024-09-21T14:59:50.369 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:50.369 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.369 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.369 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:50.369 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158452+0000", 2024-09-21T14:59:50.369 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8400000000000002e-06 2024-09-21T14:59:50.369 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.369 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.370 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:50.370 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158475+0000", 2024-09-21T14:59:50.370 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3091e-05 2024-09-21T14:59:50.370 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.370 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.370 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:50.370 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158515+0000", 2024-09-21T14:59:50.370 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0265000000000002e-05 2024-09-21T14:59:50.370 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.370 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.370 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:50.370 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172682+0000", 2024-09-21T14:59:50.370 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014166761999999999 2024-09-21T14:59:50.370 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.371 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.371 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:50.371 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172752+0000", 2024-09-21T14:59:50.371 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9698999999999998e-05 2024-09-21T14:59:50.371 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.371 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.371 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:50.371 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172919+0000", 2024-09-21T14:59:50.371 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016759199999999999 2024-09-21T14:59:50.371 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:50.371 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:50.371 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:50.371 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.371 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.371 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9351 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 434176~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:50.372 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158526+0000", 2024-09-21T14:59:50.372 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 71.181570488000006, 2024-09-21T14:59:50.372 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014662107000000001, 2024-09-21T14:59:50.372 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:50.372 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:50.372 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:50.372 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:50.372 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:50.372 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9351 2024-09-21T14:59:50.372 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.372 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:50.372 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.372 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:50.372 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158526+0000", 2024-09-21T14:59:50.373 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:50.373 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.373 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.373 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:50.373 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158526+0000", 2024-09-21T14:59:50.373 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:50.373 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.373 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.373 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:50.373 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158525+0000", 2024-09-21T14:59:50.373 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:50.373 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.373 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.373 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:50.374 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158531+0000", 2024-09-21T14:59:50.374 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5840000000000002e-06 2024-09-21T14:59:50.374 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.374 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.374 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:50.374 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158564+0000", 2024-09-21T14:59:50.374 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3195000000000003e-05 2024-09-21T14:59:50.374 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.374 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.374 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:50.374 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158577+0000", 2024-09-21T14:59:50.374 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2330000000000001e-05 2024-09-21T14:59:50.374 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.374 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.375 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:50.375 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172956+0000", 2024-09-21T14:59:50.375 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014379438 2024-09-21T14:59:50.375 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.375 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.375 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:50.375 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173020+0000", 2024-09-21T14:59:50.375 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4401999999999998e-05 2024-09-21T14:59:50.375 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.375 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.375 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:50.375 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173188+0000", 2024-09-21T14:59:50.375 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016804 2024-09-21T14:59:50.375 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:50.375 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:50.376 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:50.376 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.376 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.376 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9379 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 24576~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:50.376 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158586+0000", 2024-09-21T14:59:50.376 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 71.181511228000005, 2024-09-21T14:59:50.376 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014865173000000001, 2024-09-21T14:59:50.376 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:50.376 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:50.376 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:50.376 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:50.376 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:50.376 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9379 2024-09-21T14:59:50.376 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.377 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:50.377 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.377 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:50.377 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158586+0000", 2024-09-21T14:59:50.377 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:50.377 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.377 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.377 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:50.377 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158586+0000", 2024-09-21T14:59:50.377 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:50.377 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.377 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.377 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:50.377 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158585+0000", 2024-09-21T14:59:50.377 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:50.378 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.378 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.378 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:50.378 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158590+0000", 2024-09-21T14:59:50.378 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5820000000000001e-06 2024-09-21T14:59:50.378 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.378 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.378 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:50.378 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158676+0000", 2024-09-21T14:59:50.378 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5075999999999997e-05 2024-09-21T14:59:50.378 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.378 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.378 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:50.379 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158695+0000", 2024-09-21T14:59:50.379 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9137999999999999e-05 2024-09-21T14:59:50.379 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.379 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.379 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:50.379 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173219+0000", 2024-09-21T14:59:50.379 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014524034999999999 2024-09-21T14:59:50.379 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.379 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.379 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:50.379 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173276+0000", 2024-09-21T14:59:50.379 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7442000000000003e-05 2024-09-21T14:59:50.379 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.379 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.380 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:50.380 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173451+0000", 2024-09-21T14:59:50.380 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017464800000000001 2024-09-21T14:59:50.380 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:50.380 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:50.380 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:50.380 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.380 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.380 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9380 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 163840~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:50.380 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158706+0000", 2024-09-21T14:59:50.380 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 71.181390718000003, 2024-09-21T14:59:50.380 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015096381000000001, 2024-09-21T14:59:50.380 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:50.380 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:50.381 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:50.381 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:50.381 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:50.381 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9380 2024-09-21T14:59:50.381 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.381 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:50.381 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.381 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:50.381 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158706+0000", 2024-09-21T14:59:50.381 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:50.381 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.381 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.381 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:50.381 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158706+0000", 2024-09-21T14:59:50.382 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:50.382 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.382 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.382 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:50.382 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158705+0000", 2024-09-21T14:59:50.382 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:50.382 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.382 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.382 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:50.382 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158712+0000", 2024-09-21T14:59:50.382 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7240000000000003e-06 2024-09-21T14:59:50.382 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.382 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.382 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:50.383 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158755+0000", 2024-09-21T14:59:50.383 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2953e-05 2024-09-21T14:59:50.383 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.383 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.383 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:50.383 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158765+0000", 2024-09-21T14:59:50.383 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0149000000000001e-05 2024-09-21T14:59:50.383 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.383 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.383 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:50.383 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173492+0000", 2024-09-21T14:59:50.383 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014727105000000001 2024-09-21T14:59:50.383 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.384 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.384 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:50.384 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173564+0000", 2024-09-21T14:59:50.384 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1789e-05 2024-09-21T14:59:50.384 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.384 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.384 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:50.384 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173803+0000", 2024-09-21T14:59:50.384 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00023867100000000001 2024-09-21T14:59:50.384 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:50.384 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:50.384 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:50.384 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.384 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.385 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9381 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 303104~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:50.385 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158774+0000", 2024-09-21T14:59:50.385 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 71.181322637999997, 2024-09-21T14:59:50.385 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015309095999999999, 2024-09-21T14:59:50.385 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:50.385 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:50.385 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:50.385 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:50.385 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:50.385 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9381 2024-09-21T14:59:50.385 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.385 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:50.385 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.385 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:50.385 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158774+0000", 2024-09-21T14:59:50.386 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:50.386 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.386 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.386 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:50.386 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158774+0000", 2024-09-21T14:59:50.386 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:50.386 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.386 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.386 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:50.386 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158773+0000", 2024-09-21T14:59:50.386 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:50.386 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.386 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.386 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:50.387 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158779+0000", 2024-09-21T14:59:50.387 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6999999999999996e-06 2024-09-21T14:59:50.387 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.387 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.387 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:50.387 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158790+0000", 2024-09-21T14:59:50.387 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0594000000000001e-05 2024-09-21T14:59:50.387 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.387 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.387 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:50.387 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158798+0000", 2024-09-21T14:59:50.387 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5860000000000005e-06 2024-09-21T14:59:50.387 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.387 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.388 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:50.388 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173829+0000", 2024-09-21T14:59:50.388 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015030349 2024-09-21T14:59:50.388 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.388 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.388 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:50.388 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173918+0000", 2024-09-21T14:59:50.388 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9375000000000004e-05 2024-09-21T14:59:50.388 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.388 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.388 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:50.388 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.174083+0000", 2024-09-21T14:59:50.388 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016536200000000001 2024-09-21T14:59:50.388 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:50.388 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:50.389 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:50.389 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.389 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.389 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9382 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 442368~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:50.389 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158805+0000", 2024-09-21T14:59:50.389 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 71.181291732999995, 2024-09-21T14:59:50.389 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015592623, 2024-09-21T14:59:50.389 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:50.389 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:50.389 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:50.389 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:50.389 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:50.389 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9382 2024-09-21T14:59:50.390 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.390 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:50.390 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.390 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:50.390 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158805+0000", 2024-09-21T14:59:50.390 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:50.390 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.390 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.390 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:50.390 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158805+0000", 2024-09-21T14:59:50.390 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:50.390 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.390 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.390 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:50.391 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158805+0000", 2024-09-21T14:59:50.391 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:50.391 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.391 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.391 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:50.391 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158808+0000", 2024-09-21T14:59:50.391 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9179999999999996e-06 2024-09-21T14:59:50.391 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.391 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.391 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:50.391 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158817+0000", 2024-09-21T14:59:50.391 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9299999999999992e-06 2024-09-21T14:59:50.391 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.391 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.392 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:50.392 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158825+0000", 2024-09-21T14:59:50.392 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6739999999999997e-06 2024-09-21T14:59:50.392 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.392 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.392 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:50.392 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.174122+0000", 2024-09-21T14:59:50.392 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.01529694 2024-09-21T14:59:50.392 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.392 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.392 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:50.392 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.174185+0000", 2024-09-21T14:59:50.392 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3015000000000006e-05 2024-09-21T14:59:50.392 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.393 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.393 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:50.393 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.174398+0000", 2024-09-21T14:59:50.393 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021274799999999999 2024-09-21T14:59:50.393 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:50.393 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:50.393 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:50.393 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.393 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.393 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22700 3.19 3:98de45b8:::rbd_data.11366acc894.0000000000000000:19 [read 704512~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:50.393 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.140191+0000", 2024-09-21T14:59:50.393 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 60.199906118999998, 2024-09-21T14:59:50.393 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013770486, 2024-09-21T14:59:50.393 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:50.394 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:50.394 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:50.394 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:50.394 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:50.394 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22700 2024-09-21T14:59:50.394 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.394 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:50.394 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.394 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:50.394 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140191+0000", 2024-09-21T14:59:50.394 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:50.394 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.394 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.394 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:50.395 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140191+0000", 2024-09-21T14:59:50.395 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:50.395 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.395 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.395 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:50.395 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140190+0000", 2024-09-21T14:59:50.395 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:50.395 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.395 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.395 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:50.395 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140199+0000", 2024-09-21T14:59:50.395 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3340000000000002e-06 2024-09-21T14:59:50.395 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.396 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.396 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:50.396 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140205+0000", 2024-09-21T14:59:50.396 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3820000000000001e-06 2024-09-21T14:59:50.396 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.396 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.396 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:50.396 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140210+0000", 2024-09-21T14:59:50.396 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0969999999999996e-06 2024-09-21T14:59:50.396 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.396 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.396 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:50.396 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.153713+0000", 2024-09-21T14:59:50.396 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013502713 2024-09-21T14:59:50.397 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.397 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.397 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:50.397 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.153786+0000", 2024-09-21T14:59:50.397 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3022999999999995e-05 2024-09-21T14:59:50.397 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.397 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.397 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:50.397 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.153961+0000", 2024-09-21T14:59:50.397 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000175274 2024-09-21T14:59:50.397 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:50.397 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:50.397 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:50.397 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.398 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.398 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22701 3.19 3:98de45b8:::rbd_data.11366acc894.0000000000000000:19 [read 770048~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:50.398 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.140215+0000", 2024-09-21T14:59:50.398 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 60.199882301999999, 2024-09-21T14:59:50.398 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014072497, 2024-09-21T14:59:50.398 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:50.398 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:50.398 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:50.398 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:50.398 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:50.398 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22701 2024-09-21T14:59:50.398 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.398 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:50.398 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.399 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:50.399 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140215+0000", 2024-09-21T14:59:50.399 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:50.399 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.399 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.399 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:50.399 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140215+0000", 2024-09-21T14:59:50.399 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:50.399 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.399 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.399 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:50.399 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140214+0000", 2024-09-21T14:59:50.399 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:50.400 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.400 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.400 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:50.400 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140217+0000", 2024-09-21T14:59:50.400 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.74e-06 2024-09-21T14:59:50.400 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.400 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.400 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:50.400 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140223+0000", 2024-09-21T14:59:50.400 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0929999999999999e-06 2024-09-21T14:59:50.400 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.400 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.400 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:50.401 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140228+0000", 2024-09-21T14:59:50.401 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2279999999999998e-06 2024-09-21T14:59:50.401 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.401 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.401 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:50.401 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.153991+0000", 2024-09-21T14:59:50.401 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013762329 2024-09-21T14:59:50.401 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.401 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.401 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:50.401 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.154098+0000", 2024-09-21T14:59:50.401 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000107117 2024-09-21T14:59:50.401 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.401 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.402 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:50.402 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.154287+0000", 2024-09-21T14:59:50.402 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000189431 2024-09-21T14:59:50.402 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:50.402 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:50.402 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:50.402 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.402 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.402 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22702 3.19 3:98de45b8:::rbd_data.11366acc894.0000000000000000:19 [read 835584~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:50.402 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.140232+0000", 2024-09-21T14:59:50.402 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 60.19986445, 2024-09-21T14:59:50.402 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015338742000000001, 2024-09-21T14:59:50.402 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:50.402 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:50.403 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:50.403 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:50.403 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:50.403 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22702 2024-09-21T14:59:50.403 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.403 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:50.403 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.403 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:50.403 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140232+0000", 2024-09-21T14:59:50.403 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:50.403 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.403 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.403 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:50.403 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140232+0000", 2024-09-21T14:59:50.404 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:50.404 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.404 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.404 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:50.404 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140232+0000", 2024-09-21T14:59:50.404 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:50.404 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.404 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.404 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:50.404 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140234+0000", 2024-09-21T14:59:50.404 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.402e-06 2024-09-21T14:59:50.404 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.404 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.404 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:50.404 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140241+0000", 2024-09-21T14:59:50.405 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.054e-06 2024-09-21T14:59:50.405 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.405 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.405 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:50.405 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140246+0000", 2024-09-21T14:59:50.405 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0050000000000004e-06 2024-09-21T14:59:50.405 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.405 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.405 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:50.405 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.154315+0000", 2024-09-21T14:59:50.405 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014069440000000001 2024-09-21T14:59:50.405 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.405 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.405 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:50.406 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.154383+0000", 2024-09-21T14:59:50.406 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8151000000000005e-05 2024-09-21T14:59:50.406 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.406 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.406 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:50.406 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155571+0000", 2024-09-21T14:59:50.406 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001188029 2024-09-21T14:59:50.406 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:50.406 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:50.406 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:50.406 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.406 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.406 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22703 3.19 3:98de45b8:::rbd_data.11366acc894.0000000000000000:19 [read 901120~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:50.406 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.140250+0000", 2024-09-21T14:59:50.407 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 60.199847292000001, 2024-09-21T14:59:50.407 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014992169, 2024-09-21T14:59:50.407 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:50.407 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:50.407 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:50.407 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:50.407 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:50.407 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22703 2024-09-21T14:59:50.407 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.407 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:50.407 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.407 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:50.407 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140250+0000", 2024-09-21T14:59:50.407 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:50.408 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.408 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.408 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:50.408 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140250+0000", 2024-09-21T14:59:50.408 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:50.408 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.408 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.408 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:50.408 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140249+0000", 2024-09-21T14:59:50.408 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:50.408 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.408 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.408 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:50.408 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140251+0000", 2024-09-21T14:59:50.409 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.136e-06 2024-09-21T14:59:50.409 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.409 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.409 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:50.409 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140257+0000", 2024-09-21T14:59:50.409 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9610000000000002e-06 2024-09-21T14:59:50.409 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.409 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.409 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:50.409 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140262+0000", 2024-09-21T14:59:50.409 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4549999999999997e-06 2024-09-21T14:59:50.409 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.409 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.409 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:50.410 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155006+0000", 2024-09-21T14:59:50.410 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014744189 2024-09-21T14:59:50.410 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.410 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.410 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:50.410 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155081+0000", 2024-09-21T14:59:50.410 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4548000000000005e-05 2024-09-21T14:59:50.410 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.410 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.410 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:50.410 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155242+0000", 2024-09-21T14:59:50.410 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000161204 2024-09-21T14:59:50.410 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:50.410 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:50.411 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:50.411 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.411 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.411 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22704 3.19 3:98de45b8:::rbd_data.11366acc894.0000000000000000:19 [read 966656~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:50.411 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.140266+0000", 2024-09-21T14:59:50.411 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 60.199830974999998, 2024-09-21T14:59:50.411 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015223443999999999, 2024-09-21T14:59:50.411 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:50.411 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:50.411 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:50.411 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:50.411 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:50.411 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22704 2024-09-21T14:59:50.411 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.412 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:50.412 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.412 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:50.412 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140266+0000", 2024-09-21T14:59:50.412 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:50.412 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.412 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.412 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:50.412 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140266+0000", 2024-09-21T14:59:50.412 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:50.412 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.412 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.412 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:50.412 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140266+0000", 2024-09-21T14:59:50.413 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:50.413 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.413 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.413 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:50.413 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140268+0000", 2024-09-21T14:59:50.413 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.475e-06 2024-09-21T14:59:50.413 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.413 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.413 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:50.413 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140274+0000", 2024-09-21T14:59:50.413 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.319e-06 2024-09-21T14:59:50.413 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.413 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.413 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:50.414 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140279+0000", 2024-09-21T14:59:50.414 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.882e-06 2024-09-21T14:59:50.414 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.414 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.414 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:50.414 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155264+0000", 2024-09-21T14:59:50.414 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014984655 2024-09-21T14:59:50.414 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.414 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.414 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:50.414 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155338+0000", 2024-09-21T14:59:50.414 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4091999999999995e-05 2024-09-21T14:59:50.414 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.414 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.415 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:50.415 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155489+0000", 2024-09-21T14:59:50.415 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000151349 2024-09-21T14:59:50.415 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:50.415 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:50.415 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:50.415 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.415 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.415 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22705 3.19 3:98de45b8:::rbd_data.11366acc894.0000000000000000:19 [read 1032192~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:50.415 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.140284+0000", 2024-09-21T14:59:50.415 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 60.199813118000002, 2024-09-21T14:59:50.415 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015467896, 2024-09-21T14:59:50.415 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:50.415 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:50.416 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:50.416 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:50.416 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:50.416 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22705 2024-09-21T14:59:50.416 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.416 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:50.416 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.416 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:50.416 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140284+0000", 2024-09-21T14:59:50.416 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:50.416 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.416 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.416 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:50.416 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140284+0000", 2024-09-21T14:59:50.417 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:50.417 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.417 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.417 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:50.417 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140283+0000", 2024-09-21T14:59:50.417 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:50.417 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.417 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.417 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:50.417 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140286+0000", 2024-09-21T14:59:50.417 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7130000000000001e-06 2024-09-21T14:59:50.417 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.417 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.417 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:50.417 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140292+0000", 2024-09-21T14:59:50.418 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3430000000000001e-06 2024-09-21T14:59:50.418 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.418 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.418 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:50.418 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140298+0000", 2024-09-21T14:59:50.418 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9730000000000002e-06 2024-09-21T14:59:50.418 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.418 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.418 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:50.418 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155510+0000", 2024-09-21T14:59:50.418 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015211738000000001 2024-09-21T14:59:50.418 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.418 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.419 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:50.419 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155598+0000", 2024-09-21T14:59:50.419 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7822e-05 2024-09-21T14:59:50.419 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.419 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.419 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:50.419 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155752+0000", 2024-09-21T14:59:50.419 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0001537 2024-09-21T14:59:50.419 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:50.419 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:50.419 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:50.419 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.419 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.419 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:37892 3.1b 3:d963c97f:::rbd_data.11365d19aa81.00000000000000be:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-21T14:59:50.419 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:04.029946+0000", 2024-09-21T14:59:50.420 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 46.310151351000002, 2024-09-21T14:59:50.420 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.017686021999999999, 2024-09-21T14:59:50.420 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:50.420 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:50.420 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:50.420 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:50.420 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:50.420 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 37892 2024-09-21T14:59:50.420 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.420 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:50.420 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.420 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:50.420 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.029946+0000", 2024-09-21T14:59:50.420 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:50.421 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.421 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.421 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:50.421 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.029946+0000", 2024-09-21T14:59:50.421 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:50.421 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.421 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.421 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:50.421 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.029944+0000", 2024-09-21T14:59:50.421 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T14:59:50.421 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.421 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.421 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:50.421 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.029953+0000", 2024-09-21T14:59:50.422 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7589999999999992e-06 2024-09-21T14:59:50.422 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.422 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.422 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:50.422 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.029971+0000", 2024-09-21T14:59:50.422 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7795000000000002e-05 2024-09-21T14:59:50.422 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.422 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.422 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:50.422 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.029984+0000", 2024-09-21T14:59:50.422 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.27e-05 2024-09-21T14:59:50.422 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.422 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.422 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:50.422 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.030072+0000", 2024-09-21T14:59:50.423 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8082000000000001e-05 2024-09-21T14:59:50.423 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.423 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.423 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:50.423 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.030276+0000", 2024-09-21T14:59:50.423 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00020375699999999999 2024-09-21T14:59:50.423 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.423 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.423 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-21T14:59:50.423 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.031634+0000", 2024-09-21T14:59:50.423 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0013586049999999999 2024-09-21T14:59:50.423 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.423 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.423 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:59:50.424 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.037554+0000", 2024-09-21T14:59:50.424 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0059197449999999997 2024-09-21T14:59:50.424 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.424 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.424 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:50.424 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.047490+0000", 2024-09-21T14:59:50.424 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0099361680000000004 2024-09-21T14:59:50.424 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.424 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.424 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:59:50.424 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.047569+0000", 2024-09-21T14:59:50.424 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8298000000000001e-05 2024-09-21T14:59:50.424 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.424 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.424 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:50.425 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.047632+0000", 2024-09-21T14:59:50.425 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3006e-05 2024-09-21T14:59:50.425 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:50.425 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:50.425 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:50.425 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.425 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.425 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:63820 3.19 3.626bc079 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-21T14:59:50.425 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:27.241293+0000", 2024-09-21T14:59:50.425 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.098804258000001, 2024-09-21T14:59:50.425 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.98723808199999996, 2024-09-21T14:59:50.426 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:50.426 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:50.426 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:50.426 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:50.426 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:50.426 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 63820 2024-09-21T14:59:50.426 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.426 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:50.426 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.426 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:50.426 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241293+0000", 2024-09-21T14:59:50.426 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:50.426 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.427 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.427 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:50.427 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241293+0000", 2024-09-21T14:59:50.427 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:50.427 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.427 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.427 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:50.427 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241291+0000", 2024-09-21T14:59:50.427 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T14:59:50.427 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.427 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.427 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:50.427 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241299+0000", 2024-09-21T14:59:50.427 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1529999999999997e-06 2024-09-21T14:59:50.428 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.428 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.428 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:50.428 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241317+0000", 2024-09-21T14:59:50.428 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7764000000000001e-05 2024-09-21T14:59:50.428 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.428 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.428 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:50.428 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241328+0000", 2024-09-21T14:59:50.428 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1245e-05 2024-09-21T14:59:50.428 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.428 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.428 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:50.428 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241407+0000", 2024-09-21T14:59:50.429 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8762000000000002e-05 2024-09-21T14:59:50.429 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.429 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.429 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T14:59:50.429 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241418+0000", 2024-09-21T14:59:50.429 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1695000000000001e-05 2024-09-21T14:59:50.429 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.429 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.429 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:50.429 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:28.228510+0000", 2024-09-21T14:59:50.429 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.98709166100000001 2024-09-21T14:59:50.429 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.429 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.429 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:50.430 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:28.228531+0000", 2024-09-21T14:59:50.430 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0696e-05 2024-09-21T14:59:50.430 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:50.430 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:50.430 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:50.430 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.430 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.430 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:73845 3.6 3:601bd01e:::rbd_data.1136f4939602.0000000000000015:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-21T14:59:50.430 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:42.393435+0000", 2024-09-21T14:59:50.430 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 7.9466621469999996, 2024-09-21T14:59:50.430 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.030713345, 2024-09-21T14:59:50.430 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:50.430 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:50.430 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:50.431 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:50.431 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:50.431 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 73845 2024-09-21T14:59:50.431 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.431 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:50.431 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.431 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:50.431 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.393435+0000", 2024-09-21T14:59:50.431 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:50.431 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.431 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.431 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:50.431 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.393435+0000", 2024-09-21T14:59:50.431 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:50.432 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.432 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.432 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:50.432 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.393433+0000", 2024-09-21T14:59:50.432 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T14:59:50.432 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.432 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.432 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:50.432 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.393446+0000", 2024-09-21T14:59:50.432 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3013000000000001e-05 2024-09-21T14:59:50.432 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.432 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.432 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:50.432 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.393475+0000", 2024-09-21T14:59:50.433 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9615999999999999e-05 2024-09-21T14:59:50.433 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.433 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.433 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:50.433 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.393489+0000", 2024-09-21T14:59:50.433 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4103e-05 2024-09-21T14:59:50.433 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.433 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.433 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:50.433 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.393565+0000", 2024-09-21T14:59:50.433 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5195000000000001e-05 2024-09-21T14:59:50.433 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.433 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.433 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:50.433 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.393734+0000", 2024-09-21T14:59:50.433 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016909999999999999 2024-09-21T14:59:50.434 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.434 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.434 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 7", 2024-09-21T14:59:50.434 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.400243+0000", 2024-09-21T14:59:50.434 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0065093629999999998 2024-09-21T14:59:50.434 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.434 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.434 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:59:50.434 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.423721+0000", 2024-09-21T14:59:50.434 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.023477984 2024-09-21T14:59:50.434 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.434 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.434 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:50.435 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.423852+0000", 2024-09-21T14:59:50.435 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000131369 2024-09-21T14:59:50.435 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.435 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.435 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:59:50.435 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.424011+0000", 2024-09-21T14:59:50.435 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000158231 2024-09-21T14:59:50.435 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.435 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.435 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:50.435 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.424148+0000", 2024-09-21T14:59:50.435 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013732699999999999 2024-09-21T14:59:50.435 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:50.435 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:50.436 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:50.436 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:50.436 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:50.436 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:59:50.436 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:59:50.381+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:59:50.437 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T14:59:50.444 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:59:50.441+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:59:50.544 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:59:50.539+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:59:50.551 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:59:50.551 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:59:50.551 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T14:59:50.551 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:59:50.561 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T14:59:50.645 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:59:50.639+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:59:50.714 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:59:50.714 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:59:50.714 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T14:59:50.714 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T14:59:50.715 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:59:50.724 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T14:59:50.746 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:59:50.745+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:59:50.838 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:59:50.838 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T14:59:50.838 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T14:59:50.838 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T14:59:50.839 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.839 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45834 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:50.839 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159133+0000", 2024-09-21T14:59:50.839 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 40.676414147999999, 2024-09-21T14:59:50.839 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068236884999999997, 2024-09-21T14:59:50.839 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:50.839 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:50.839 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:50.839 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:50.839 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:50.839 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45834 2024-09-21T14:59:50.839 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.839 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:50.840 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.840 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:50.840 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159133+0000", 2024-09-21T14:59:50.840 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:50.840 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.840 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.840 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:50.840 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159133+0000", 2024-09-21T14:59:50.840 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:50.840 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.840 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.840 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:50.840 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159133+0000", 2024-09-21T14:59:50.840 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:50.841 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.841 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.841 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:50.841 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159135+0000", 2024-09-21T14:59:50.841 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4500000000000001e-06 2024-09-21T14:59:50.841 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.841 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.841 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:50.841 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159138+0000", 2024-09-21T14:59:50.841 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2770000000000001e-06 2024-09-21T14:59:50.841 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.841 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.842 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:50.842 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159141+0000", 2024-09-21T14:59:50.842 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9210000000000001e-06 2024-09-21T14:59:50.842 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.842 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.842 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:50.842 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227361+0000", 2024-09-21T14:59:50.842 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068220154000000005 2024-09-21T14:59:50.842 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.842 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.842 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:50.842 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227370+0000", 2024-09-21T14:59:50.843 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3239999999999992e-06 2024-09-21T14:59:50.843 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:50.843 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:50.843 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:50.843 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.843 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.843 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45839 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:50.843 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159313+0000", 2024-09-21T14:59:50.843 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 40.676234182000002, 2024-09-21T14:59:50.843 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068238699, 2024-09-21T14:59:50.843 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:50.843 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:50.843 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:50.843 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:50.844 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:50.844 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45839 2024-09-21T14:59:50.844 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.844 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:50.844 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.844 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:50.844 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159313+0000", 2024-09-21T14:59:50.844 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:50.844 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.844 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.844 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:50.844 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159313+0000", 2024-09-21T14:59:50.844 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:50.844 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.845 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.845 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:50.845 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159313+0000", 2024-09-21T14:59:50.845 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:50.845 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.845 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.845 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:50.845 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159316+0000", 2024-09-21T14:59:50.845 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.033e-06 2024-09-21T14:59:50.845 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.845 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.845 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:50.845 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159322+0000", 2024-09-21T14:59:50.846 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1820000000000003e-06 2024-09-21T14:59:50.846 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.846 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.846 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:50.846 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159327+0000", 2024-09-21T14:59:50.846 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.062e-06 2024-09-21T14:59:50.846 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.846 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.846 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:50.846 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227544+0000", 2024-09-21T14:59:50.846 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068217142999999994 2024-09-21T14:59:50.846 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.846 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.847 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:50.847 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227552+0000", 2024-09-21T14:59:50.847 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6260000000000003e-06 2024-09-21T14:59:50.847 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:50.847 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:50.847 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:50.847 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.848 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.848 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45840 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:50.848 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159332+0000", 2024-09-21T14:59:50.848 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 40.676215884999998, 2024-09-21T14:59:50.848 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068262046000000007, 2024-09-21T14:59:50.848 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:50.848 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:50.848 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:50.849 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:50.849 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:50.849 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45840 2024-09-21T14:59:50.849 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.849 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:50.849 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.849 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:50.849 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159332+0000", 2024-09-21T14:59:50.849 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:50.849 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.849 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.849 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:50.849 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159332+0000", 2024-09-21T14:59:50.849 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:50.850 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.850 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.850 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:50.850 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159331+0000", 2024-09-21T14:59:50.850 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:50.850 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.850 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.850 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:50.850 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159351+0000", 2024-09-21T14:59:50.850 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0064999999999998e-05 2024-09-21T14:59:50.851 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.851 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.851 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:50.851 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159367+0000", 2024-09-21T14:59:50.851 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5308e-05 2024-09-21T14:59:50.851 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.851 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.851 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:50.851 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159373+0000", 2024-09-21T14:59:50.851 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1840000000000004e-06 2024-09-21T14:59:50.851 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.851 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.851 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:50.851 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227584+0000", 2024-09-21T14:59:50.852 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068211401000000005 2024-09-21T14:59:50.852 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.852 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.852 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:50.852 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227594+0000", 2024-09-21T14:59:50.852 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4120000000000007e-06 2024-09-21T14:59:50.852 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:50.852 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:50.852 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:50.852 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.852 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.852 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45841 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:50.852 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159379+0000", 2024-09-21T14:59:50.852 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 40.676168728999997, 2024-09-21T14:59:50.852 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068249039999999997, 2024-09-21T14:59:50.853 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:50.853 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:50.853 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:50.853 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:50.853 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:50.853 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45841 2024-09-21T14:59:50.853 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.853 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:50.853 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.853 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:50.853 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159379+0000", 2024-09-21T14:59:50.853 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:50.853 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.853 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.854 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:50.854 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159379+0000", 2024-09-21T14:59:50.854 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:50.854 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.854 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.854 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:50.854 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159378+0000", 2024-09-21T14:59:50.854 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:50.854 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.854 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.854 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:50.854 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159382+0000", 2024-09-21T14:59:50.854 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.641e-06 2024-09-21T14:59:50.854 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.855 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.855 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:50.855 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159421+0000", 2024-09-21T14:59:50.855 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9439e-05 2024-09-21T14:59:50.855 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.855 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.855 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:50.855 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159430+0000", 2024-09-21T14:59:50.855 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8429999999999999e-06 2024-09-21T14:59:50.855 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.855 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.855 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:50.855 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227617+0000", 2024-09-21T14:59:50.855 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068187084999999995 2024-09-21T14:59:50.856 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.856 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.856 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:50.856 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227628+0000", 2024-09-21T14:59:50.856 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0507e-05 2024-09-21T14:59:50.856 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:50.856 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:50.856 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:50.856 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.856 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.856 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45844 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:50.856 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159499+0000", 2024-09-21T14:59:50.856 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 40.676048342999998, 2024-09-21T14:59:50.856 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068255415, 2024-09-21T14:59:50.857 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:50.857 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:50.857 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:50.857 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:50.857 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:50.857 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45844 2024-09-21T14:59:50.857 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.858 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:50.858 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.858 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:50.858 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159499+0000", 2024-09-21T14:59:50.858 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:50.858 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.858 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.858 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:50.858 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159499+0000", 2024-09-21T14:59:50.858 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:50.858 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.859 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.859 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:50.859 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159499+0000", 2024-09-21T14:59:50.859 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:50.859 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.859 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.859 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:50.859 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159503+0000", 2024-09-21T14:59:50.860 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0999999999999997e-06 2024-09-21T14:59:50.860 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.860 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.860 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:50.860 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159512+0000", 2024-09-21T14:59:50.860 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5850000000000006e-06 2024-09-21T14:59:50.860 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.860 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.860 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:50.860 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159518+0000", 2024-09-21T14:59:50.860 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0909999999999998e-06 2024-09-21T14:59:50.860 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.860 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.860 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:50.861 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227743+0000", 2024-09-21T14:59:50.861 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068224536000000002 2024-09-21T14:59:50.861 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.861 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.861 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:50.861 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227755+0000", 2024-09-21T14:59:50.861 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1659e-05 2024-09-21T14:59:50.861 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:50.861 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:50.861 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:50.861 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.861 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.861 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45845 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:50.861 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159524+0000", 2024-09-21T14:59:50.861 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 40.676023983, 2024-09-21T14:59:50.861 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068267194000000003, 2024-09-21T14:59:50.862 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:50.862 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:50.862 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:50.862 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:50.862 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:50.862 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45845 2024-09-21T14:59:50.862 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.862 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:50.862 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.862 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:50.862 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159524+0000", 2024-09-21T14:59:50.862 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:50.862 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.863 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.863 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:50.863 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159524+0000", 2024-09-21T14:59:50.863 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:50.863 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.863 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.863 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:50.863 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159523+0000", 2024-09-21T14:59:50.863 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:50.863 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.863 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.863 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:50.863 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159526+0000", 2024-09-21T14:59:50.863 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1980000000000001e-06 2024-09-21T14:59:50.863 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.864 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.864 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:50.864 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159533+0000", 2024-09-21T14:59:50.864 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.348e-06 2024-09-21T14:59:50.864 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.864 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.864 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:50.864 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159546+0000", 2024-09-21T14:59:50.864 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3464e-05 2024-09-21T14:59:50.864 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.864 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.864 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:50.864 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227783+0000", 2024-09-21T14:59:50.864 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068236926000000003 2024-09-21T14:59:50.865 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.865 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.865 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:50.865 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227791+0000", 2024-09-21T14:59:50.865 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6669999999999996e-06 2024-09-21T14:59:50.865 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:50.865 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:50.865 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:50.865 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.865 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.865 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45846 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:50.865 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159584+0000", 2024-09-21T14:59:50.865 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 40.675963340999999, 2024-09-21T14:59:50.866 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068254767999999993, 2024-09-21T14:59:50.866 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:50.866 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:50.866 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:50.866 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:50.866 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:50.866 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45846 2024-09-21T14:59:50.866 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.866 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:50.866 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.866 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:50.866 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159584+0000", 2024-09-21T14:59:50.866 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:50.867 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.867 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.867 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:50.867 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159584+0000", 2024-09-21T14:59:50.867 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:50.867 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.867 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.867 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:50.867 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159583+0000", 2024-09-21T14:59:50.867 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:50.867 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.867 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.867 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:50.868 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159588+0000", 2024-09-21T14:59:50.868 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1330000000000001e-06 2024-09-21T14:59:50.868 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.868 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.868 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:50.868 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159596+0000", 2024-09-21T14:59:50.868 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5469999999999997e-06 2024-09-21T14:59:50.868 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.868 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.868 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:50.868 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159609+0000", 2024-09-21T14:59:50.868 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2785999999999999e-05 2024-09-21T14:59:50.868 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.868 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.868 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:50.869 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227830+0000", 2024-09-21T14:59:50.869 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068220879999999998 2024-09-21T14:59:50.869 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.869 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.869 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:50.869 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227839+0000", 2024-09-21T14:59:50.869 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1239999999999994e-06 2024-09-21T14:59:50.869 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:50.869 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:50.869 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:50.869 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.869 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.869 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45847 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:50.869 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159614+0000", 2024-09-21T14:59:50.869 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 40.675933264999998, 2024-09-21T14:59:50.870 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068260639999999997, 2024-09-21T14:59:50.870 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:50.870 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:50.870 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:50.870 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:50.870 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:50.870 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45847 2024-09-21T14:59:50.870 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.870 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:50.870 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.870 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:50.870 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159614+0000", 2024-09-21T14:59:50.870 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:50.870 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.871 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.871 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:50.871 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159614+0000", 2024-09-21T14:59:50.871 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:50.871 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.871 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.871 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:50.871 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159614+0000", 2024-09-21T14:59:50.871 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:50.871 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.871 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.871 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:50.871 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159617+0000", 2024-09-21T14:59:50.872 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2909999999999999e-06 2024-09-21T14:59:50.872 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.872 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.872 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:50.872 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159623+0000", 2024-09-21T14:59:50.872 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.511e-06 2024-09-21T14:59:50.872 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.872 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.872 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:50.872 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159631+0000", 2024-09-21T14:59:50.872 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.571e-06 2024-09-21T14:59:50.872 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.872 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.872 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:50.873 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227866+0000", 2024-09-21T14:59:50.873 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068235136000000002 2024-09-21T14:59:50.873 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.873 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.873 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:50.873 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227875+0000", 2024-09-21T14:59:50.873 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7199999999999995e-06 2024-09-21T14:59:50.873 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:50.873 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:50.873 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:50.873 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.873 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.873 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45848 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:50.873 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159636+0000", 2024-09-21T14:59:50.873 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 40.675911485, 2024-09-21T14:59:50.874 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068267283999999998, 2024-09-21T14:59:50.874 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:50.874 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:50.874 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:50.874 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:50.874 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:50.874 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45848 2024-09-21T14:59:50.874 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.874 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:50.874 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.874 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:50.874 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159636+0000", 2024-09-21T14:59:50.874 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:50.875 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.875 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.875 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:50.875 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159636+0000", 2024-09-21T14:59:50.875 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:50.875 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.875 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.875 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:50.875 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159636+0000", 2024-09-21T14:59:50.875 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:50.875 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.875 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.875 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:50.875 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159638+0000", 2024-09-21T14:59:50.876 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6560000000000001e-06 2024-09-21T14:59:50.876 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.876 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.876 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:50.876 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159644+0000", 2024-09-21T14:59:50.876 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0839999999999997e-06 2024-09-21T14:59:50.876 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.876 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.876 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:50.876 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159649+0000", 2024-09-21T14:59:50.876 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7500000000000003e-06 2024-09-21T14:59:50.876 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.876 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.876 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:50.876 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227892+0000", 2024-09-21T14:59:50.877 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068243315999999998 2024-09-21T14:59:50.877 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.877 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.877 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:50.877 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227903+0000", 2024-09-21T14:59:50.877 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0835999999999999e-05 2024-09-21T14:59:50.877 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:50.877 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:50.877 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:50.877 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.877 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.877 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45849 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:50.877 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159667+0000", 2024-09-21T14:59:50.877 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 40.675880671000002, 2024-09-21T14:59:50.878 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068291051000000005, 2024-09-21T14:59:50.878 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:50.878 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:50.878 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:50.878 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:50.878 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:50.878 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45849 2024-09-21T14:59:50.878 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.878 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:50.878 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.878 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:50.878 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159667+0000", 2024-09-21T14:59:50.878 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:50.879 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.879 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.879 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:50.879 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159667+0000", 2024-09-21T14:59:50.879 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:50.879 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.879 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.879 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:50.879 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159666+0000", 2024-09-21T14:59:50.879 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:50.879 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.879 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.879 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:50.879 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159670+0000", 2024-09-21T14:59:50.880 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.427e-06 2024-09-21T14:59:50.880 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.880 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.880 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:50.880 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159678+0000", 2024-09-21T14:59:50.880 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9869999999999999e-06 2024-09-21T14:59:50.880 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.880 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.880 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:50.880 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159688+0000", 2024-09-21T14:59:50.880 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0631999999999999e-05 2024-09-21T14:59:50.880 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.880 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.880 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:50.881 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227950+0000", 2024-09-21T14:59:50.881 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068261798999999998 2024-09-21T14:59:50.881 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.881 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.881 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:50.881 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227958+0000", 2024-09-21T14:59:50.881 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7510000000000008e-06 2024-09-21T14:59:50.881 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:50.881 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:50.881 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:50.881 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.881 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.881 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45850 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:50.881 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159695+0000", 2024-09-21T14:59:50.881 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 40.67585296, 2024-09-21T14:59:50.882 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068302999000000003, 2024-09-21T14:59:50.882 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:50.882 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:50.882 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:50.882 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:50.882 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:50.882 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45850 2024-09-21T14:59:50.882 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.882 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:50.882 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.882 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:50.882 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159695+0000", 2024-09-21T14:59:50.882 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:50.883 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.883 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.883 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:50.883 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159695+0000", 2024-09-21T14:59:50.883 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:50.883 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.883 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.883 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:50.883 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159694+0000", 2024-09-21T14:59:50.883 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:50.883 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.883 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.883 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:50.883 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159697+0000", 2024-09-21T14:59:50.884 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.258e-06 2024-09-21T14:59:50.884 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.884 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.884 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:50.884 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159704+0000", 2024-09-21T14:59:50.884 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7660000000000001e-06 2024-09-21T14:59:50.884 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.884 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.884 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:50.884 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159713+0000", 2024-09-21T14:59:50.884 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1779999999999992e-06 2024-09-21T14:59:50.884 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.885 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.885 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:50.885 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227989+0000", 2024-09-21T14:59:50.885 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068276015999999995 2024-09-21T14:59:50.885 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.885 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.885 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:50.885 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227998+0000", 2024-09-21T14:59:50.885 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2050000000000002e-06 2024-09-21T14:59:50.885 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:50.885 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:50.885 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:50.885 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.885 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.886 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45851 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:50.886 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159719+0000", 2024-09-21T14:59:50.886 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 40.675828826999997, 2024-09-21T14:59:50.886 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068320327, 2024-09-21T14:59:50.886 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:50.886 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:50.886 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:50.886 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:50.886 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:50.886 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45851 2024-09-21T14:59:50.886 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.886 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:50.886 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.887 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:50.887 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159719+0000", 2024-09-21T14:59:50.887 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:50.887 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.887 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.887 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:50.887 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159719+0000", 2024-09-21T14:59:50.887 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:50.887 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.887 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.887 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:50.887 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159718+0000", 2024-09-21T14:59:50.887 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:50.887 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.888 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.888 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:50.888 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159721+0000", 2024-09-21T14:59:50.888 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2169999999999999e-06 2024-09-21T14:59:50.888 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.888 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.888 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:50.888 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159730+0000", 2024-09-21T14:59:50.888 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5720000000000003e-06 2024-09-21T14:59:50.888 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.888 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.888 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:50.888 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159735+0000", 2024-09-21T14:59:50.888 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1100000000000002e-06 2024-09-21T14:59:50.888 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.889 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.889 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:50.889 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228029+0000", 2024-09-21T14:59:50.889 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068293624999999997 2024-09-21T14:59:50.889 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.889 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.889 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:50.889 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228039+0000", 2024-09-21T14:59:50.889 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0254e-05 2024-09-21T14:59:50.889 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:50.889 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:50.889 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:50.889 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.889 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.889 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45852 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:50.890 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159740+0000", 2024-09-21T14:59:50.890 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 40.675807659, 2024-09-21T14:59:50.890 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068340260999999999, 2024-09-21T14:59:50.890 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:50.890 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:50.890 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:50.890 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:50.890 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:50.890 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45852 2024-09-21T14:59:50.890 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.890 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:50.891 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.891 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:50.891 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159740+0000", 2024-09-21T14:59:50.891 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:50.891 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.891 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.891 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:50.891 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159740+0000", 2024-09-21T14:59:50.891 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:50.891 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.891 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.891 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:50.892 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159739+0000", 2024-09-21T14:59:50.892 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:50.892 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.892 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.892 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:50.892 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159757+0000", 2024-09-21T14:59:50.892 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7314999999999999e-05 2024-09-21T14:59:50.892 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.892 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.892 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:50.892 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159769+0000", 2024-09-21T14:59:50.892 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2376e-05 2024-09-21T14:59:50.892 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.892 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.893 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:50.893 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159775+0000", 2024-09-21T14:59:50.893 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2179999999999996e-06 2024-09-21T14:59:50.893 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.893 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.893 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:50.893 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228071+0000", 2024-09-21T14:59:50.893 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068295344999999993 2024-09-21T14:59:50.893 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.893 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.893 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:50.893 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228080+0000", 2024-09-21T14:59:50.893 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3959999999999995e-06 2024-09-21T14:59:50.893 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:50.894 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:50.894 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:50.894 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.894 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.894 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45853 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:50.894 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159781+0000", 2024-09-21T14:59:50.894 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 40.675766394999997, 2024-09-21T14:59:50.894 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068337093000000002, 2024-09-21T14:59:50.894 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:50.894 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:50.894 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:50.894 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:50.894 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:50.894 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45853 2024-09-21T14:59:50.895 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.895 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:50.895 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.895 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:50.895 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159781+0000", 2024-09-21T14:59:50.895 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:50.895 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.895 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.895 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:50.895 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159781+0000", 2024-09-21T14:59:50.895 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:50.895 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.895 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.895 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:50.895 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159781+0000", 2024-09-21T14:59:50.896 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:50.896 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.896 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.896 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:50.896 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159784+0000", 2024-09-21T14:59:50.896 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4759999999999998e-06 2024-09-21T14:59:50.896 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.896 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.896 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:50.896 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159792+0000", 2024-09-21T14:59:50.896 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0569999999999993e-06 2024-09-21T14:59:50.896 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.896 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.896 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:50.897 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159797+0000", 2024-09-21T14:59:50.897 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9640000000000002e-06 2024-09-21T14:59:50.897 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.897 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.897 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:50.897 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228111+0000", 2024-09-21T14:59:50.897 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068313737999999999 2024-09-21T14:59:50.897 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.897 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.897 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:50.897 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228118+0000", 2024-09-21T14:59:50.897 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.43e-06 2024-09-21T14:59:50.898 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:50.898 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:50.898 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:50.898 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.898 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.898 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45854 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:50.898 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159802+0000", 2024-09-21T14:59:50.898 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 40.675745745, 2024-09-21T14:59:50.898 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068358938999999994, 2024-09-21T14:59:50.898 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:50.898 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:50.898 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:50.898 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:50.899 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:50.899 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45854 2024-09-21T14:59:50.899 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.899 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:50.899 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.899 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:50.899 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159802+0000", 2024-09-21T14:59:50.899 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:50.899 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.899 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.899 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:50.899 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159802+0000", 2024-09-21T14:59:50.899 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:50.900 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.900 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.900 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:50.900 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159801+0000", 2024-09-21T14:59:50.900 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:50.900 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.900 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.900 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:50.900 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159804+0000", 2024-09-21T14:59:50.900 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0970000000000002e-06 2024-09-21T14:59:50.900 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.900 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.900 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:50.900 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159823+0000", 2024-09-21T14:59:50.900 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8895000000000001e-05 2024-09-21T14:59:50.901 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.901 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.901 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:50.901 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159837+0000", 2024-09-21T14:59:50.901 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3247999999999999e-05 2024-09-21T14:59:50.901 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.901 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.901 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:50.901 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228151+0000", 2024-09-21T14:59:50.901 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068314642999999994 2024-09-21T14:59:50.901 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.901 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.901 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:50.902 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228161+0000", 2024-09-21T14:59:50.902 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4250000000000004e-06 2024-09-21T14:59:50.902 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:50.902 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:50.902 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:50.902 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.902 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.902 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45983 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:50.902 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159848+0000", 2024-09-21T14:59:50.902 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 40.675699485999999, 2024-09-21T14:59:50.902 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068333801, 2024-09-21T14:59:50.902 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:50.902 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:50.903 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:50.903 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:50.903 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:50.903 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45983 2024-09-21T14:59:50.903 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.903 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:50.903 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.903 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:50.903 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159848+0000", 2024-09-21T14:59:50.903 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:50.903 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.903 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.903 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:50.903 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159848+0000", 2024-09-21T14:59:50.904 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:50.904 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.904 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.904 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:50.904 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159847+0000", 2024-09-21T14:59:50.904 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:50.904 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.904 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.904 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:50.904 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159856+0000", 2024-09-21T14:59:50.904 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4349999999999997e-06 2024-09-21T14:59:50.904 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.904 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.904 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:50.904 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159866+0000", 2024-09-21T14:59:50.905 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0076e-05 2024-09-21T14:59:50.905 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.905 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.905 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:50.905 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159872+0000", 2024-09-21T14:59:50.905 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8309999999999997e-06 2024-09-21T14:59:50.905 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.905 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.905 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:50.905 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228175+0000", 2024-09-21T14:59:50.905 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068302680000000005 2024-09-21T14:59:50.905 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.906 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.906 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:50.906 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228182+0000", 2024-09-21T14:59:50.906 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3150000000000003e-06 2024-09-21T14:59:50.906 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:50.906 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:50.906 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:50.906 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.906 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.906 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45984 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:50.906 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159877+0000", 2024-09-21T14:59:50.906 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 40.675670066000002, 2024-09-21T14:59:50.906 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068328448, 2024-09-21T14:59:50.906 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:50.906 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:50.907 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:50.907 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:50.907 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:50.907 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45984 2024-09-21T14:59:50.907 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.907 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:50.907 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.907 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:50.907 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159877+0000", 2024-09-21T14:59:50.907 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:50.907 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.907 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.907 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:50.907 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159877+0000", 2024-09-21T14:59:50.908 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:50.908 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.908 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.908 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:50.908 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159877+0000", 2024-09-21T14:59:50.908 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:50.908 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.908 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.908 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:50.908 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159880+0000", 2024-09-21T14:59:50.908 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7610000000000001e-06 2024-09-21T14:59:50.908 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.908 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.908 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:50.908 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159888+0000", 2024-09-21T14:59:50.909 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0330000000000003e-06 2024-09-21T14:59:50.909 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.909 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.909 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:50.909 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159893+0000", 2024-09-21T14:59:50.909 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3550000000000004e-06 2024-09-21T14:59:50.909 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.909 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.909 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:50.909 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228199+0000", 2024-09-21T14:59:50.909 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068306438999999997 2024-09-21T14:59:50.909 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.909 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.909 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:50.909 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228206+0000", 2024-09-21T14:59:50.910 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5599999999999999e-06 2024-09-21T14:59:50.910 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:50.910 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:50.910 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:50.910 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.910 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.910 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45985 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:50.910 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159897+0000", 2024-09-21T14:59:50.910 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 40.675650243, 2024-09-21T14:59:50.910 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068345664, 2024-09-21T14:59:50.910 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:50.910 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:50.910 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:50.910 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:50.911 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:50.911 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45985 2024-09-21T14:59:50.911 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.911 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:50.911 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.911 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:50.911 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159897+0000", 2024-09-21T14:59:50.911 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:50.911 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.911 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.911 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:50.911 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159897+0000", 2024-09-21T14:59:50.911 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:50.911 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.912 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.912 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:50.912 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159897+0000", 2024-09-21T14:59:50.912 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:50.912 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.912 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.912 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:50.912 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159900+0000", 2024-09-21T14:59:50.912 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.954e-06 2024-09-21T14:59:50.912 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.912 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.912 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:50.912 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159913+0000", 2024-09-21T14:59:50.912 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3106e-05 2024-09-21T14:59:50.913 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.913 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.913 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:50.913 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159918+0000", 2024-09-21T14:59:50.913 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5199999999999997e-06 2024-09-21T14:59:50.913 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.913 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.913 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:50.913 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228236+0000", 2024-09-21T14:59:50.914 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068317040999999995 2024-09-21T14:59:50.914 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.914 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.914 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:50.914 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228243+0000", 2024-09-21T14:59:50.914 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3880000000000002e-06 2024-09-21T14:59:50.914 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:50.914 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:50.914 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:50.914 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.914 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.914 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45986 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:50.914 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159924+0000", 2024-09-21T14:59:50.914 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 40.675623948000002, 2024-09-21T14:59:50.915 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068364362999999997, 2024-09-21T14:59:50.915 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:50.915 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:50.915 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:50.915 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:50.915 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:50.915 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45986 2024-09-21T14:59:50.915 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.915 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:50.915 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.915 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:50.915 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159924+0000", 2024-09-21T14:59:50.915 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:50.915 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.916 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.916 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:50.916 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159924+0000", 2024-09-21T14:59:50.916 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:50.916 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.916 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.916 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:50.916 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159923+0000", 2024-09-21T14:59:50.916 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:50.916 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.916 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.916 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:50.916 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159926+0000", 2024-09-21T14:59:50.916 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0510000000000001e-06 2024-09-21T14:59:50.916 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.917 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.917 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:50.917 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159932+0000", 2024-09-21T14:59:50.917 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4160000000000001e-06 2024-09-21T14:59:50.917 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.917 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.917 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:50.917 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159938+0000", 2024-09-21T14:59:50.917 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.383e-06 2024-09-21T14:59:50.917 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.917 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.917 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:50.918 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228275+0000", 2024-09-21T14:59:50.918 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068337007000000005 2024-09-21T14:59:50.918 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.918 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.918 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:50.918 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228288+0000", 2024-09-21T14:59:50.918 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3052e-05 2024-09-21T14:59:50.918 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:50.918 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:50.918 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:50.918 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.918 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.918 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45987 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:50.918 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159944+0000", 2024-09-21T14:59:50.918 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 40.675603723999998, 2024-09-21T14:59:50.919 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068387232000000006, 2024-09-21T14:59:50.919 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:50.919 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:50.919 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:50.919 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:50.919 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:50.919 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45987 2024-09-21T14:59:50.919 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.919 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:50.919 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.919 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:50.919 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159944+0000", 2024-09-21T14:59:50.919 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:50.920 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.920 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.920 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:50.920 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159944+0000", 2024-09-21T14:59:50.920 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:50.920 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.920 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.920 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:50.920 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159943+0000", 2024-09-21T14:59:50.920 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:50.920 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.920 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.920 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:50.920 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159946+0000", 2024-09-21T14:59:50.920 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6309999999999999e-06 2024-09-21T14:59:50.921 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.921 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.921 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:50.921 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159952+0000", 2024-09-21T14:59:50.921 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6540000000000004e-06 2024-09-21T14:59:50.921 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.921 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.921 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:50.921 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159957+0000", 2024-09-21T14:59:50.921 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9280000000000001e-06 2024-09-21T14:59:50.921 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.922 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.922 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:50.922 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228322+0000", 2024-09-21T14:59:50.922 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068365067000000002 2024-09-21T14:59:50.922 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:50.922 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:50.922 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:50.922 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228331+0000", 2024-09-21T14:59:50.922 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3270000000000007e-06 2024-09-21T14:59:50.922 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:50.922 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:50.922 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:50.922 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:50.922 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:50.923 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:59:50.924 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T14:59:50.924 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:59:50.925+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:59:51.025 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:59:51.025+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:59:51.040 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:59:51.040 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:59:51.040 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T14:59:51.040 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:59:51.049 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T14:59:51.125 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:59:51.125+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:59:51.166 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:59:51.166 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:59:51.166 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T14:59:51.166 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T14:59:51.166 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:59:51.180 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T14:59:51.226 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:59:51.225+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:59:51.290 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:59:51.290 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T14:59:51.290 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T14:59:51.290 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T14:59:51.290 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.290 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:10651 3.12 3:4e9d23af:::rbd_data.1136f5ed5dd6.0000000000000020:head [write 0~81920 [fadvise_sequential+fadvise_nocache],write 90112~8192 [fadvise_sequential+fadvise_nocache],write 122880~122880 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:51.290 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.292476+0000", 2024-09-21T14:59:51.290 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 71.994061071999994, 2024-09-21T14:59:51.290 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012765669, 2024-09-21T14:59:51.291 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:51.291 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:51.291 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:51.291 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:51.291 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:51.291 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 10651 2024-09-21T14:59:51.291 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.291 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:51.291 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.291 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:51.291 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292476+0000", 2024-09-21T14:59:51.291 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:51.291 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.292 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.292 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:51.292 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292476+0000", 2024-09-21T14:59:51.292 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:51.292 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.292 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.292 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:51.292 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292475+0000", 2024-09-21T14:59:51.292 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:51.292 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.292 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.292 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:51.292 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292696+0000", 2024-09-21T14:59:51.292 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000220201 2024-09-21T14:59:51.293 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.293 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.293 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:51.293 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292711+0000", 2024-09-21T14:59:51.293 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5803000000000001e-05 2024-09-21T14:59:51.293 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.293 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.293 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:51.293 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292721+0000", 2024-09-21T14:59:51.293 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6160000000000008e-06 2024-09-21T14:59:51.293 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.293 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.293 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:51.294 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292757+0000", 2024-09-21T14:59:51.294 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6211999999999998e-05 2024-09-21T14:59:51.294 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.294 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.294 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:51.294 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292878+0000", 2024-09-21T14:59:51.294 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000120306 2024-09-21T14:59:51.294 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.294 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.294 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 3", 2024-09-21T14:59:51.294 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.293019+0000", 2024-09-21T14:59:51.294 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014186400000000001 2024-09-21T14:59:51.294 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.295 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.295 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:59:51.295 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.294898+0000", 2024-09-21T14:59:51.295 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001878434 2024-09-21T14:59:51.295 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.295 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.295 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:51.295 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.305162+0000", 2024-09-21T14:59:51.295 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.01026373 2024-09-21T14:59:51.295 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.295 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.295 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:59:51.295 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.305209+0000", 2024-09-21T14:59:51.295 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7615999999999999e-05 2024-09-21T14:59:51.296 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.296 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.296 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:51.296 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.305242+0000", 2024-09-21T14:59:51.296 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2979999999999999e-05 2024-09-21T14:59:51.296 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:51.296 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:51.296 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:51.296 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.296 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.296 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:11285 3.12 3:4edf73d2:::rbd_data.1136f5ed5dd6.0000000000000039:head [write 90112~32768 [fadvise_sequential+fadvise_nocache],write 139264~24576 [fadvise_sequential+fadvise_nocache],write 180224~24576 [fadvise_sequential+fadvise_nocache],write 221184~40960 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:51.296 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.539470+0000", 2024-09-21T14:59:51.296 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 71.747067319999999, 2024-09-21T14:59:51.297 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0083360099999999996, 2024-09-21T14:59:51.297 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:51.297 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:51.297 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:51.297 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:51.297 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:51.297 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 11285 2024-09-21T14:59:51.297 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.297 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:51.297 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.297 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:51.297 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.539470+0000", 2024-09-21T14:59:51.297 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:51.297 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.298 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.298 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:51.298 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.539470+0000", 2024-09-21T14:59:51.298 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:51.298 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.298 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.298 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:51.298 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.539469+0000", 2024-09-21T14:59:51.298 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:51.298 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.298 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.299 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:51.299 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.539652+0000", 2024-09-21T14:59:51.299 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000183084 2024-09-21T14:59:51.299 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.299 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.299 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:51.299 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.539666+0000", 2024-09-21T14:59:51.299 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3947e-05 2024-09-21T14:59:51.299 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.299 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.299 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:51.300 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.539673+0000", 2024-09-21T14:59:51.300 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7309999999999996e-06 2024-09-21T14:59:51.300 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.300 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.300 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:51.300 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.539714+0000", 2024-09-21T14:59:51.300 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1539000000000003e-05 2024-09-21T14:59:51.300 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.300 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.300 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:51.300 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.539832+0000", 2024-09-21T14:59:51.300 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011753500000000001 2024-09-21T14:59:51.300 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.300 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.300 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 3", 2024-09-21T14:59:51.301 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.540005+0000", 2024-09-21T14:59:51.301 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000173031 2024-09-21T14:59:51.301 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.301 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.301 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:59:51.301 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.541606+0000", 2024-09-21T14:59:51.301 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001600899 2024-09-21T14:59:51.301 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.301 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.301 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:51.301 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.547709+0000", 2024-09-21T14:59:51.301 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0061027110000000002 2024-09-21T14:59:51.301 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.302 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.302 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:59:51.302 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.547767+0000", 2024-09-21T14:59:51.302 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8248999999999997e-05 2024-09-21T14:59:51.302 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.302 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.302 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:51.302 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.547806+0000", 2024-09-21T14:59:51.302 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9341000000000002e-05 2024-09-21T14:59:51.302 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:51.302 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:51.302 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:51.303 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.303 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.303 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12189 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 40960~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:51.303 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.618667+0000", 2024-09-21T14:59:51.303 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 71.667870870000002, 2024-09-21T14:59:51.303 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0083014549999999993, 2024-09-21T14:59:51.303 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:51.303 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:51.303 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:51.303 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:51.303 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:51.303 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12189 2024-09-21T14:59:51.303 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.303 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:51.303 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.304 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:51.304 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618667+0000", 2024-09-21T14:59:51.304 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:51.304 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.304 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.304 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:51.304 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618667+0000", 2024-09-21T14:59:51.304 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:51.304 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.304 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.304 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:51.304 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618666+0000", 2024-09-21T14:59:51.304 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:51.304 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.305 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.305 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:51.305 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618669+0000", 2024-09-21T14:59:51.305 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.554e-06 2024-09-21T14:59:51.305 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.305 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.305 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:51.305 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618675+0000", 2024-09-21T14:59:51.305 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2570000000000004e-06 2024-09-21T14:59:51.305 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.305 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.305 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:51.305 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618680+0000", 2024-09-21T14:59:51.305 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6539999999999998e-06 2024-09-21T14:59:51.305 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.306 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.306 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:51.306 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.626683+0000", 2024-09-21T14:59:51.306 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0080035880000000007 2024-09-21T14:59:51.306 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.306 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.306 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:51.306 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.626764+0000", 2024-09-21T14:59:51.306 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0672000000000003e-05 2024-09-21T14:59:51.306 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.306 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.306 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:51.306 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.626968+0000", 2024-09-21T14:59:51.306 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00020407499999999999 2024-09-21T14:59:51.307 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:51.307 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:51.307 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:51.307 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.307 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.307 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12193 3.4 3:25437d51:::rbd_data.11363a21d2be.0000000000000025:10 [read 425984~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:51.307 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.618755+0000", 2024-09-21T14:59:51.307 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 71.667782789, 2024-09-21T14:59:51.307 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086461379999999994, 2024-09-21T14:59:51.307 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:51.307 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:51.307 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:51.307 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:51.307 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:51.308 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12193 2024-09-21T14:59:51.308 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.308 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:51.308 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.308 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:51.308 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618755+0000", 2024-09-21T14:59:51.308 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:51.308 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.308 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.308 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:51.308 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618755+0000", 2024-09-21T14:59:51.308 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:51.308 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.309 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.309 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:51.309 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618754+0000", 2024-09-21T14:59:51.309 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:51.309 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.309 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.309 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:51.309 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618759+0000", 2024-09-21T14:59:51.309 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4020000000000004e-06 2024-09-21T14:59:51.309 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.309 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.309 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:51.309 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618767+0000", 2024-09-21T14:59:51.309 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7490000000000004e-06 2024-09-21T14:59:51.310 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.310 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.310 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:51.310 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618773+0000", 2024-09-21T14:59:51.310 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0939999999999996e-06 2024-09-21T14:59:51.310 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.310 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.310 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:51.310 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.627115+0000", 2024-09-21T14:59:51.310 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0083416840000000003 2024-09-21T14:59:51.310 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.310 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.310 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:51.311 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.627214+0000", 2024-09-21T14:59:51.311 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9042000000000005e-05 2024-09-21T14:59:51.311 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.311 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.311 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:51.311 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.627401+0000", 2024-09-21T14:59:51.311 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018677200000000001 2024-09-21T14:59:51.311 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:51.311 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:51.311 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:51.311 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.311 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.311 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12212 3.4 3:21b43b45:::rbd_data.11363a21d2be.000000000000002b:10 [read 237568~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:51.312 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.619922+0000", 2024-09-21T14:59:51.312 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 71.666615805000006, 2024-09-21T14:59:51.312 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.008324372, 2024-09-21T14:59:51.312 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:51.312 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:51.312 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:51.312 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:51.312 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:51.312 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12212 2024-09-21T14:59:51.312 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.312 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:51.312 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.312 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:51.312 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619922+0000", 2024-09-21T14:59:51.313 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:51.313 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.313 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.313 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:51.313 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619922+0000", 2024-09-21T14:59:51.313 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:51.313 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.313 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.313 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:51.313 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619921+0000", 2024-09-21T14:59:51.313 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:51.313 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.313 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.313 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:51.314 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619924+0000", 2024-09-21T14:59:51.314 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.999e-06 2024-09-21T14:59:51.314 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.314 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.314 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:51.314 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619934+0000", 2024-09-21T14:59:51.314 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.8120000000000002e-06 2024-09-21T14:59:51.314 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.314 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.314 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:51.314 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619939+0000", 2024-09-21T14:59:51.314 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7729999999999999e-06 2024-09-21T14:59:51.314 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.315 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.315 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:51.315 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628005+0000", 2024-09-21T14:59:51.315 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0080663330000000002 2024-09-21T14:59:51.315 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.315 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.315 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:51.315 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628073+0000", 2024-09-21T14:59:51.315 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7479999999999998e-05 2024-09-21T14:59:51.315 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.315 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.315 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:51.315 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628246+0000", 2024-09-21T14:59:51.315 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017337400000000001 2024-09-21T14:59:51.316 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:51.316 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:51.316 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:51.316 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.316 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.316 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12213 3.4 3:21b43b45:::rbd_data.11363a21d2be.000000000000002b:10 [read 270336~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:51.316 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.619945+0000", 2024-09-21T14:59:51.316 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 71.666592523000006, 2024-09-21T14:59:51.316 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086195049999999995, 2024-09-21T14:59:51.316 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:51.316 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:51.316 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:51.316 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:51.316 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:51.317 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12213 2024-09-21T14:59:51.317 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.317 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:51.317 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.317 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:51.317 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619945+0000", 2024-09-21T14:59:51.317 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:51.317 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.317 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.317 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:51.317 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619945+0000", 2024-09-21T14:59:51.317 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:51.317 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.317 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.318 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:51.318 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619945+0000", 2024-09-21T14:59:51.318 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:51.318 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.318 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.318 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:51.318 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619947+0000", 2024-09-21T14:59:51.319 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5780000000000001e-06 2024-09-21T14:59:51.319 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.319 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.319 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:51.319 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619957+0000", 2024-09-21T14:59:51.319 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3500000000000003e-06 2024-09-21T14:59:51.319 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.320 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.320 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:51.320 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619964+0000", 2024-09-21T14:59:51.320 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.199e-06 2024-09-21T14:59:51.320 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.320 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.321 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:51.321 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628285+0000", 2024-09-21T14:59:51.321 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0083210939999999994 2024-09-21T14:59:51.321 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.321 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.321 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:51.321 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628361+0000", 2024-09-21T14:59:51.322 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5873999999999999e-05 2024-09-21T14:59:51.322 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.322 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.322 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:51.322 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628565+0000", 2024-09-21T14:59:51.322 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000203799 2024-09-21T14:59:51.323 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:51.323 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:51.323 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:51.323 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.323 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.323 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12214 3.4 3:21b43b45:::rbd_data.11363a21d2be.000000000000002b:10 [read 376832~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:51.323 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.619969+0000", 2024-09-21T14:59:51.324 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 71.666568522000006, 2024-09-21T14:59:51.324 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0090292089999999998, 2024-09-21T14:59:51.324 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:51.324 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:51.324 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:51.324 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:51.325 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:51.325 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12214 2024-09-21T14:59:51.325 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.325 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:51.325 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.325 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:51.326 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619969+0000", 2024-09-21T14:59:51.326 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:51.326 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.326 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.326 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:51.326 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619969+0000", 2024-09-21T14:59:51.327 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:51.327 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.327 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.327 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:51.327 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619969+0000", 2024-09-21T14:59:51.327 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:51.328 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.328 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.328 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:51.328 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619971+0000", 2024-09-21T14:59:51.328 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1830000000000001e-06 2024-09-21T14:59:51.328 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.329 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.329 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:51.329 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619976+0000", 2024-09-21T14:59:51.329 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7129999999999996e-06 2024-09-21T14:59:51.329 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.329 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.329 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:51.330 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619981+0000", 2024-09-21T14:59:51.330 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2460000000000003e-06 2024-09-21T14:59:51.330 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.330 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.330 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:51.330 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628602+0000", 2024-09-21T14:59:51.330 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086209519999999994 2024-09-21T14:59:51.331 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.331 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.331 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:51.331 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628734+0000", 2024-09-21T14:59:51.331 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000132242 2024-09-21T14:59:51.331 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.332 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.332 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:51.332 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628998+0000", 2024-09-21T14:59:51.332 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00026419799999999999 2024-09-21T14:59:51.332 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:51.332 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:51.332 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:51.333 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.333 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.333 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12215 3.4 3:21b43b45:::rbd_data.11363a21d2be.000000000000002b:10 [read 409600~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:51.333 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.619985+0000", 2024-09-21T14:59:51.333 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 71.666552456000005, 2024-09-21T14:59:51.334 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0095457319999999995, 2024-09-21T14:59:51.334 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:51.334 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:51.334 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:51.334 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:51.334 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:51.335 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12215 2024-09-21T14:59:51.335 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.335 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:51.335 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.335 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:51.335 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619985+0000", 2024-09-21T14:59:51.336 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:51.336 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.336 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.336 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:51.336 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619985+0000", 2024-09-21T14:59:51.336 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:51.337 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.337 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.337 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:51.337 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619985+0000", 2024-09-21T14:59:51.337 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:51.337 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.338 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.338 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:51.338 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619987+0000", 2024-09-21T14:59:51.338 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6129999999999998e-06 2024-09-21T14:59:51.338 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.338 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.339 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:51.339 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619993+0000", 2024-09-21T14:59:51.339 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0820000000000004e-06 2024-09-21T14:59:51.339 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.339 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.339 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:51.340 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619998+0000", 2024-09-21T14:59:51.340 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7759999999999997e-06 2024-09-21T14:59:51.340 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.340 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.340 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:51.340 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629022+0000", 2024-09-21T14:59:51.341 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0090237489999999993 2024-09-21T14:59:51.341 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.341 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.341 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:51.341 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629098+0000", 2024-09-21T14:59:51.341 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5735999999999995e-05 2024-09-21T14:59:51.342 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.342 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.342 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:51.342 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629531+0000", 2024-09-21T14:59:51.342 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00043310100000000003 2024-09-21T14:59:51.342 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:51.342 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:51.342 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:51.342 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.343 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.343 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12216 3.4 3:21b43b45:::rbd_data.11363a21d2be.000000000000002b:10 [read 516096~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:51.343 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.620003+0000", 2024-09-21T14:59:51.343 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 71.666534459999994, 2024-09-21T14:59:51.343 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0098850169999999994, 2024-09-21T14:59:51.343 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:51.343 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:51.343 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:51.344 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:51.344 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:51.344 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12216 2024-09-21T14:59:51.344 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.344 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:51.344 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.344 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:51.344 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620003+0000", 2024-09-21T14:59:51.344 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:51.345 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.345 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.345 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:51.345 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620003+0000", 2024-09-21T14:59:51.345 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:51.345 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.345 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.345 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:51.346 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620003+0000", 2024-09-21T14:59:51.346 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:51.346 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.346 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.346 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:51.346 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620006+0000", 2024-09-21T14:59:51.346 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4350000000000001e-06 2024-09-21T14:59:51.346 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.346 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.346 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:51.347 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620013+0000", 2024-09-21T14:59:51.347 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4219999999999997e-06 2024-09-21T14:59:51.347 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.347 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.347 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:51.347 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620017+0000", 2024-09-21T14:59:51.347 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.865e-06 2024-09-21T14:59:51.347 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.347 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.347 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:51.347 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629581+0000", 2024-09-21T14:59:51.347 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0095631989999999997 2024-09-21T14:59:51.347 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.348 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.348 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:51.348 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629691+0000", 2024-09-21T14:59:51.348 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000110586 2024-09-21T14:59:51.348 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.348 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.348 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:51.348 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629888+0000", 2024-09-21T14:59:51.348 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019691199999999999 2024-09-21T14:59:51.348 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:51.348 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:51.348 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:51.348 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.348 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.349 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12222 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 49152~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:51.349 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.620027+0000", 2024-09-21T14:59:51.349 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 71.666510114000005, 2024-09-21T14:59:51.349 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010228131999999999, 2024-09-21T14:59:51.349 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:51.349 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:51.349 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:51.349 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:51.349 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:51.349 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12222 2024-09-21T14:59:51.349 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.349 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:51.349 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.349 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:51.350 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620027+0000", 2024-09-21T14:59:51.350 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:51.350 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.350 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.350 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:51.350 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620027+0000", 2024-09-21T14:59:51.350 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:51.350 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.350 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.350 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:51.350 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620027+0000", 2024-09-21T14:59:51.350 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:51.350 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.351 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.351 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:51.351 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620030+0000", 2024-09-21T14:59:51.351 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0400000000000001e-06 2024-09-21T14:59:51.351 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.351 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.351 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:51.351 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620037+0000", 2024-09-21T14:59:51.351 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8220000000000001e-06 2024-09-21T14:59:51.351 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.351 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.351 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:51.351 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620042+0000", 2024-09-21T14:59:51.352 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.843e-06 2024-09-21T14:59:51.352 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.352 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.352 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:51.352 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629922+0000", 2024-09-21T14:59:51.352 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0098805100000000003 2024-09-21T14:59:51.352 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.352 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.352 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:51.352 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.630020+0000", 2024-09-21T14:59:51.352 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7596999999999994e-05 2024-09-21T14:59:51.352 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.352 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.353 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:51.353 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.630256+0000", 2024-09-21T14:59:51.353 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00023573299999999999 2024-09-21T14:59:51.353 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:51.353 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:51.353 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:51.353 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.353 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.353 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12253 3.4 3:21b43b45:::rbd_data.11363a21d2be.000000000000002b:10 [read 417792~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:51.353 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.624578+0000", 2024-09-21T14:59:51.353 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 71.661959745999994, 2024-09-21T14:59:51.353 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0085112079999999993, 2024-09-21T14:59:51.353 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:51.353 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:51.354 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:51.354 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:51.354 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:51.354 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12253 2024-09-21T14:59:51.354 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.354 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:51.354 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.354 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:51.354 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624578+0000", 2024-09-21T14:59:51.354 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:51.354 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.354 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.354 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:51.355 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624578+0000", 2024-09-21T14:59:51.355 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:51.355 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.355 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.355 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:51.355 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624578+0000", 2024-09-21T14:59:51.355 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:51.355 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.355 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.355 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:51.355 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624579+0000", 2024-09-21T14:59:51.356 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.688e-06 2024-09-21T14:59:51.356 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.356 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.356 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:51.356 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624582+0000", 2024-09-21T14:59:51.356 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2260000000000001e-06 2024-09-21T14:59:51.356 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.356 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.356 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:51.356 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624585+0000", 2024-09-21T14:59:51.356 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4820000000000001e-06 2024-09-21T14:59:51.356 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.356 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.356 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:51.357 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.632580+0000", 2024-09-21T14:59:51.357 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0079948959999999996 2024-09-21T14:59:51.357 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.357 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.357 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:51.357 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.632683+0000", 2024-09-21T14:59:51.357 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010308 2024-09-21T14:59:51.357 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.357 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.357 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:51.357 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.633089+0000", 2024-09-21T14:59:51.357 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00040612599999999998 2024-09-21T14:59:51.357 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:51.357 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:51.358 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:51.358 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.358 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.358 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12260 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 57344~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:51.358 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.624587+0000", 2024-09-21T14:59:51.358 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 71.661950258999994, 2024-09-21T14:59:51.358 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0088282069999999994, 2024-09-21T14:59:51.358 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:51.358 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:51.358 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:51.358 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:51.358 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:51.358 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12260 2024-09-21T14:59:51.359 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.359 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:51.359 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.359 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:51.359 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624587+0000", 2024-09-21T14:59:51.359 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:51.359 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.359 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.359 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:51.359 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624587+0000", 2024-09-21T14:59:51.359 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:51.359 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.359 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.359 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:51.360 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624587+0000", 2024-09-21T14:59:51.360 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-21T14:59:51.360 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.360 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.360 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:51.360 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624589+0000", 2024-09-21T14:59:51.360 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4339999999999999e-06 2024-09-21T14:59:51.360 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.360 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.360 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:51.360 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624593+0000", 2024-09-21T14:59:51.360 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5859999999999998e-06 2024-09-21T14:59:51.360 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.361 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.361 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:51.361 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624596+0000", 2024-09-21T14:59:51.361 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5830000000000001e-06 2024-09-21T14:59:51.361 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.361 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.361 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:51.361 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.633132+0000", 2024-09-21T14:59:51.361 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.008536666 2024-09-21T14:59:51.361 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.361 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.361 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:51.361 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.633216+0000", 2024-09-21T14:59:51.361 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3387999999999995e-05 2024-09-21T14:59:51.361 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.362 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.362 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:51.362 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.633415+0000", 2024-09-21T14:59:51.362 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019976799999999999 2024-09-21T14:59:51.362 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:51.362 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:51.362 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:51.362 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.362 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.362 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12516 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 245760~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:51.362 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645194+0000", 2024-09-21T14:59:51.362 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 71.641343722000002, 2024-09-21T14:59:51.362 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0084284990000000008, 2024-09-21T14:59:51.363 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:51.363 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:51.363 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:51.363 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:51.363 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:51.363 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12516 2024-09-21T14:59:51.363 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.363 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:51.363 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.363 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:51.363 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645194+0000", 2024-09-21T14:59:51.363 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:51.363 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.363 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.364 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:51.364 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645194+0000", 2024-09-21T14:59:51.364 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:51.364 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.364 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.364 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:51.364 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645193+0000", 2024-09-21T14:59:51.364 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:51.364 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.364 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.364 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:51.364 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645196+0000", 2024-09-21T14:59:51.364 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1030000000000002e-06 2024-09-21T14:59:51.364 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.365 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.365 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:51.365 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645205+0000", 2024-09-21T14:59:51.365 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6200000000000005e-06 2024-09-21T14:59:51.365 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.365 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.365 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:51.365 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645210+0000", 2024-09-21T14:59:51.365 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6280000000000004e-06 2024-09-21T14:59:51.365 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.365 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.365 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:51.365 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.653305+0000", 2024-09-21T14:59:51.365 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0080957879999999996 2024-09-21T14:59:51.366 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.366 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.366 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:51.366 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.653389+0000", 2024-09-21T14:59:51.366 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3242e-05 2024-09-21T14:59:51.366 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.366 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.366 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:51.366 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.653622+0000", 2024-09-21T14:59:51.366 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00023361599999999999 2024-09-21T14:59:51.366 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:51.366 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:51.367 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:51.367 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.367 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.367 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12517 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 278528~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:51.367 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645214+0000", 2024-09-21T14:59:51.367 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 71.641323608999997, 2024-09-21T14:59:51.367 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086832899999999998, 2024-09-21T14:59:51.367 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:51.367 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:51.367 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:51.367 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:51.367 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:51.367 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12517 2024-09-21T14:59:51.367 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.368 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:51.368 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.368 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:51.368 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645214+0000", 2024-09-21T14:59:51.368 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:51.368 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.368 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.368 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:51.368 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645214+0000", 2024-09-21T14:59:51.368 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:51.368 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.368 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.368 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:51.368 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645214+0000", 2024-09-21T14:59:51.369 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:51.369 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.369 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.369 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:51.369 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645216+0000", 2024-09-21T14:59:51.369 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5579999999999999e-06 2024-09-21T14:59:51.369 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.369 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.369 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:51.369 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645224+0000", 2024-09-21T14:59:51.369 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2439999999999993e-06 2024-09-21T14:59:51.369 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.369 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.370 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:51.370 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645231+0000", 2024-09-21T14:59:51.370 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1859999999999997e-06 2024-09-21T14:59:51.370 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.370 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.370 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:51.370 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.653651+0000", 2024-09-21T14:59:51.370 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0084204220000000003 2024-09-21T14:59:51.370 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.370 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.370 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:51.370 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.653726+0000", 2024-09-21T14:59:51.370 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5356000000000002e-05 2024-09-21T14:59:51.371 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.371 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.371 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:51.371 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.653897+0000", 2024-09-21T14:59:51.371 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000170848 2024-09-21T14:59:51.371 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:51.371 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:51.371 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:51.371 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.372 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.372 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12518 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 385024~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:51.372 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645235+0000", 2024-09-21T14:59:51.372 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 71.641302382999996, 2024-09-21T14:59:51.372 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0089841020000000008, 2024-09-21T14:59:51.372 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:51.372 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:51.373 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:51.373 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:51.373 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:51.373 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12518 2024-09-21T14:59:51.373 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.373 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:51.373 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.374 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:51.374 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645235+0000", 2024-09-21T14:59:51.374 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:51.374 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.374 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.374 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:51.375 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645235+0000", 2024-09-21T14:59:51.375 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:51.375 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.375 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.375 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:51.375 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645235+0000", 2024-09-21T14:59:51.375 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:51.376 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.376 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.376 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:51.376 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645238+0000", 2024-09-21T14:59:51.376 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.869e-06 2024-09-21T14:59:51.376 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.376 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.377 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:51.377 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645244+0000", 2024-09-21T14:59:51.377 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8930000000000002e-06 2024-09-21T14:59:51.377 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.377 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.377 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:51.377 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645249+0000", 2024-09-21T14:59:51.378 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2859999999999999e-06 2024-09-21T14:59:51.378 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.378 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.378 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:51.378 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.653930+0000", 2024-09-21T14:59:51.378 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086815320000000005 2024-09-21T14:59:51.379 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.379 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.379 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:51.379 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654028+0000", 2024-09-21T14:59:51.379 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7578000000000007e-05 2024-09-21T14:59:51.379 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.380 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.380 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:51.380 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654219+0000", 2024-09-21T14:59:51.380 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019132099999999999 2024-09-21T14:59:51.380 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:51.380 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:51.381 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:51.381 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.381 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.381 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12538 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 147456~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:51.381 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645253+0000", 2024-09-21T14:59:51.381 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 71.641284759000001, 2024-09-21T14:59:51.382 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0092095360000000008, 2024-09-21T14:59:51.382 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:51.382 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:51.382 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:51.382 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:51.382 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:51.382 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12538 2024-09-21T14:59:51.383 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.383 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:51.383 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.383 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:51.383 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645253+0000", 2024-09-21T14:59:51.383 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:51.384 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.384 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.384 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:51.384 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645253+0000", 2024-09-21T14:59:51.384 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:51.384 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.385 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.385 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:51.385 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645252+0000", 2024-09-21T14:59:51.385 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:51.385 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.385 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.386 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:51.386 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645255+0000", 2024-09-21T14:59:51.386 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2290000000000002e-06 2024-09-21T14:59:51.386 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.386 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.386 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:51.387 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645267+0000", 2024-09-21T14:59:51.387 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2631e-05 2024-09-21T14:59:51.387 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.387 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.387 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:51.387 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645278+0000", 2024-09-21T14:59:51.388 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0847e-05 2024-09-21T14:59:51.388 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.388 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.388 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:51.388 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654251+0000", 2024-09-21T14:59:51.388 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0089731340000000007 2024-09-21T14:59:51.388 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.389 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.389 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:51.389 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654296+0000", 2024-09-21T14:59:51.389 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4353e-05 2024-09-21T14:59:51.389 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.389 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.390 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:51.390 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654462+0000", 2024-09-21T14:59:51.390 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016668499999999999 2024-09-21T14:59:51.390 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:51.390 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:51.390 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:51.391 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.391 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.391 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12539 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 253952~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:51.391 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645284+0000", 2024-09-21T14:59:51.391 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 71.641253538000001, 2024-09-21T14:59:51.391 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0094524029999999998, 2024-09-21T14:59:51.392 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:51.392 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:51.392 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:51.392 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:51.392 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:51.392 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12539 2024-09-21T14:59:51.393 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.393 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:51.393 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.393 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:51.393 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645284+0000", 2024-09-21T14:59:51.393 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:51.393 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.393 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.393 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:51.393 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645284+0000", 2024-09-21T14:59:51.394 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:51.394 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.394 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.394 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:51.394 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645284+0000", 2024-09-21T14:59:51.394 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:51.394 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.394 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.394 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:51.394 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645286+0000", 2024-09-21T14:59:51.394 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7159999999999999e-06 2024-09-21T14:59:51.394 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.394 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.395 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:51.395 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645292+0000", 2024-09-21T14:59:51.395 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7400000000000001e-06 2024-09-21T14:59:51.395 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.395 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.395 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:51.395 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645297+0000", 2024-09-21T14:59:51.395 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9019999999999998e-06 2024-09-21T14:59:51.395 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.395 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.395 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:51.395 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654490+0000", 2024-09-21T14:59:51.395 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0091932360000000005 2024-09-21T14:59:51.396 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.396 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.396 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:51.396 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654565+0000", 2024-09-21T14:59:51.396 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4657e-05 2024-09-21T14:59:51.396 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.396 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.396 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:51.396 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654736+0000", 2024-09-21T14:59:51.396 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017156199999999999 2024-09-21T14:59:51.396 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:51.396 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:51.396 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:51.396 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.397 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.397 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12540 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 286720~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:51.397 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645302+0000", 2024-09-21T14:59:51.397 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 71.641235269000006, 2024-09-21T14:59:51.397 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0096603090000000006, 2024-09-21T14:59:51.397 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:51.397 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:51.397 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:51.397 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:51.397 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:51.397 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12540 2024-09-21T14:59:51.397 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.397 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:51.397 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.398 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:51.398 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645302+0000", 2024-09-21T14:59:51.398 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:51.398 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.398 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.398 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:51.398 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645302+0000", 2024-09-21T14:59:51.398 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:51.398 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.398 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.398 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:51.398 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645302+0000", 2024-09-21T14:59:51.398 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:51.399 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.399 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.399 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:51.399 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645305+0000", 2024-09-21T14:59:51.399 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7329999999999999e-06 2024-09-21T14:59:51.399 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.399 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.399 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:51.399 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645316+0000", 2024-09-21T14:59:51.399 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1832e-05 2024-09-21T14:59:51.399 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.399 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.399 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:51.399 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645322+0000", 2024-09-21T14:59:51.400 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.169e-06 2024-09-21T14:59:51.400 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.400 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.400 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:51.400 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654778+0000", 2024-09-21T14:59:51.400 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0094559090000000002 2024-09-21T14:59:51.400 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.400 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.400 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:51.400 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654828+0000", 2024-09-21T14:59:51.400 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0461999999999999e-05 2024-09-21T14:59:51.400 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.400 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.401 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:51.401 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654963+0000", 2024-09-21T14:59:51.401 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000134583 2024-09-21T14:59:51.401 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:51.401 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:51.401 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:51.401 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.401 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.401 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12561 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:51.401 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645742+0000", 2024-09-21T14:59:51.401 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 71.640796003000005, 2024-09-21T14:59:51.401 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0093583810000000007, 2024-09-21T14:59:51.401 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:51.401 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:51.402 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:51.402 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:51.402 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:51.402 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12561 2024-09-21T14:59:51.402 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.402 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:51.402 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.402 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:51.402 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645742+0000", 2024-09-21T14:59:51.402 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:51.402 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.402 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.402 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:51.403 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645742+0000", 2024-09-21T14:59:51.403 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:51.403 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.403 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.403 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:51.403 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645741+0000", 2024-09-21T14:59:51.403 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:51.403 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.403 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.403 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:51.403 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645746+0000", 2024-09-21T14:59:51.403 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8379999999999998e-06 2024-09-21T14:59:51.403 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.404 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.404 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:51.404 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645757+0000", 2024-09-21T14:59:51.404 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0705e-05 2024-09-21T14:59:51.404 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.404 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.404 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:51.404 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645765+0000", 2024-09-21T14:59:51.404 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6729999999999992e-06 2024-09-21T14:59:51.404 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.404 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.404 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:51.404 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654993+0000", 2024-09-21T14:59:51.404 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0092287480000000002 2024-09-21T14:59:51.405 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.405 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.405 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:51.405 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.655050+0000", 2024-09-21T14:59:51.405 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6394999999999998e-05 2024-09-21T14:59:51.405 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.405 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.405 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:51.405 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.655100+0000", 2024-09-21T14:59:51.405 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0049000000000001e-05 2024-09-21T14:59:51.405 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:51.405 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:51.405 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:51.405 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.406 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.406 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:15431 3.12 3:4bc7d206:::rbd_data.1136f5ed5dd6.0000000000000051:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-21T14:59:51.406 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:41.673445+0000", 2024-09-21T14:59:51.406 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 69.613092668999997, 2024-09-21T14:59:51.406 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086103610000000004, 2024-09-21T14:59:51.406 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:51.406 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:51.406 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:51.406 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:51.406 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:51.406 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 15431 2024-09-21T14:59:51.406 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.406 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:51.407 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.407 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:51.407 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.673445+0000", 2024-09-21T14:59:51.407 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:51.407 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.407 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.407 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:51.407 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.673445+0000", 2024-09-21T14:59:51.407 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:51.407 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.407 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.407 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:51.407 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.673444+0000", 2024-09-21T14:59:51.407 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:59:51.408 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.408 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.408 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:51.408 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.673451+0000", 2024-09-21T14:59:51.408 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4739999999999999e-06 2024-09-21T14:59:51.408 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.408 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.408 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:51.408 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.673472+0000", 2024-09-21T14:59:51.408 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0687e-05 2024-09-21T14:59:51.408 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.408 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.408 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:51.408 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.673481+0000", 2024-09-21T14:59:51.409 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3349999999999996e-06 2024-09-21T14:59:51.409 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.409 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.409 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:51.409 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.673565+0000", 2024-09-21T14:59:51.409 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4011000000000006e-05 2024-09-21T14:59:51.409 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.409 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.409 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:51.409 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.673650+0000", 2024-09-21T14:59:51.409 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4949000000000003e-05 2024-09-21T14:59:51.409 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.409 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.409 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 3", 2024-09-21T14:59:51.409 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.675014+0000", 2024-09-21T14:59:51.410 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0013636620000000001 2024-09-21T14:59:51.410 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.410 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.410 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:51.410 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.681866+0000", 2024-09-21T14:59:51.410 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0068525640000000002 2024-09-21T14:59:51.410 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.410 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.410 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:59:51.410 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.681885+0000", 2024-09-21T14:59:51.410 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8519999999999999e-05 2024-09-21T14:59:51.410 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.410 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.411 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:59:51.411 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.681961+0000", 2024-09-21T14:59:51.411 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6248000000000006e-05 2024-09-21T14:59:51.411 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.411 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.411 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:51.411 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.682055+0000", 2024-09-21T14:59:51.411 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4184000000000005e-05 2024-09-21T14:59:51.411 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:51.411 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:51.411 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:51.411 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:51.411 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:51.411 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:59:51.413 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T14:59:51.413 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:59:51.407+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:59:51.514 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:59:51.513+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:59:51.529 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:59:51.529 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:59:51.529 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T14:59:51.530 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:59:51.539 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T14:59:51.615 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:59:51.611+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:59:51.645 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:59:51.645 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:59:51.645 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T14:59:51.645 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T14:59:51.645 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:59:51.655 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T14:59:51.715 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:59:51.711+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:59:51.815 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:59:51.815 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T14:59:51.815 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T14:59:51.815 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T14:59:51.815 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.815 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:8049 3.b 3:d1b12ef9:::rbd_data.11361bed1f78.0000000000000062:d [read 114688~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:59:51.815 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:35.227371+0000", 2024-09-21T14:59:51.815 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 76.584028450000005, 2024-09-21T14:59:51.815 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0078533609999999997, 2024-09-21T14:59:51.815 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:51.816 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:51.816 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:51.816 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:51.816 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:51.816 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 8049 2024-09-21T14:59:51.816 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.816 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:51.816 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.816 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:51.816 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227371+0000", 2024-09-21T14:59:51.816 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:51.816 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.816 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.816 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:51.817 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227371+0000", 2024-09-21T14:59:51.817 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:51.817 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.817 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.817 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:51.817 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227370+0000", 2024-09-21T14:59:51.817 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:59:51.817 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.817 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.817 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:51.817 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227378+0000", 2024-09-21T14:59:51.817 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.379e-06 2024-09-21T14:59:51.817 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.817 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.818 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:51.818 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227388+0000", 2024-09-21T14:59:51.818 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0175e-05 2024-09-21T14:59:51.818 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.818 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.818 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:51.818 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227393+0000", 2024-09-21T14:59:51.818 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6690000000000003e-06 2024-09-21T14:59:51.818 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.818 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.818 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:51.818 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.234964+0000", 2024-09-21T14:59:51.818 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0075705319999999996 2024-09-21T14:59:51.818 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.819 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.819 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:51.819 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.235041+0000", 2024-09-21T14:59:51.819 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7170999999999997e-05 2024-09-21T14:59:51.819 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.819 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.819 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:51.819 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.235225+0000", 2024-09-21T14:59:51.819 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000183735 2024-09-21T14:59:51.819 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:51.819 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:51.819 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:51.819 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.819 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.819 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31476 3.c 3:35718b6b:::rbd_data.11367721c19.0000000000000014:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T14:59:51.820 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.505975+0000", 2024-09-21T14:59:51.820 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 53.305424918, 2024-09-21T14:59:51.820 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99515995899999998, 2024-09-21T14:59:51.820 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:51.820 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:51.820 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:51.820 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:51.820 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:51.820 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31476 2024-09-21T14:59:51.820 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.820 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:51.820 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.821 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:51.821 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.505975+0000", 2024-09-21T14:59:51.821 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:51.821 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.821 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.821 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:51.821 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.505975+0000", 2024-09-21T14:59:51.821 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:51.821 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.821 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.821 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:51.821 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.505971+0000", 2024-09-21T14:59:51.821 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999967 2024-09-21T14:59:51.821 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.821 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.822 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:51.822 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.505989+0000", 2024-09-21T14:59:51.822 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7985000000000001e-05 2024-09-21T14:59:51.822 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.822 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.822 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:51.822 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506050+0000", 2024-09-21T14:59:51.822 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0659000000000003e-05 2024-09-21T14:59:51.822 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.822 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.822 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:51.822 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506074+0000", 2024-09-21T14:59:51.822 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4156e-05 2024-09-21T14:59:51.822 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.823 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.823 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:51.823 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506225+0000", 2024-09-21T14:59:51.823 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015097 2024-09-21T14:59:51.823 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.823 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.823 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T14:59:51.823 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506256+0000", 2024-09-21T14:59:51.823 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0505999999999999e-05 2024-09-21T14:59:51.823 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.823 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.823 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:51.823 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.497974+0000", 2024-09-21T14:59:51.823 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99171825700000005 2024-09-21T14:59:51.824 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.824 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.824 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for missing object", 2024-09-21T14:59:51.824 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.498449+0000", 2024-09-21T14:59:51.824 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00047506399999999998 2024-09-21T14:59:51.824 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.824 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.824 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:51.824 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.501000+0000", 2024-09-21T14:59:51.824 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0025506919999999998 2024-09-21T14:59:51.824 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.824 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.824 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:51.824 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.501135+0000", 2024-09-21T14:59:51.825 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013524000000000001 2024-09-21T14:59:51.825 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:51.825 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:51.825 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:51.825 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.825 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.825 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31480 3.c 3:36ccd808:::rbd_data.11367721c19.0000000000000018:head [sparse-read 0~14821] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T14:59:51.825 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.506116+0000", 2024-09-21T14:59:51.825 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 53.305283574999997, 2024-09-21T14:59:51.825 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99316033599999998, 2024-09-21T14:59:51.825 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:51.825 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:51.825 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:51.825 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:51.825 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:51.826 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31480 2024-09-21T14:59:51.826 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.826 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:51.826 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.826 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:51.826 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506116+0000", 2024-09-21T14:59:51.826 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:51.826 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.827 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.827 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:51.828 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506116+0000", 2024-09-21T14:59:51.828 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:51.828 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.828 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.828 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:51.828 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506115+0000", 2024-09-21T14:59:51.828 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:59:51.828 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.828 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.828 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:51.828 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506124+0000", 2024-09-21T14:59:51.828 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0659999999999993e-06 2024-09-21T14:59:51.829 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.829 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.829 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:51.829 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506145+0000", 2024-09-21T14:59:51.829 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1447e-05 2024-09-21T14:59:51.829 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.829 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.829 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:51.829 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506161+0000", 2024-09-21T14:59:51.829 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5209e-05 2024-09-21T14:59:51.829 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.829 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.829 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:51.829 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506337+0000", 2024-09-21T14:59:51.830 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017642999999999999 2024-09-21T14:59:51.830 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.830 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.830 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T14:59:51.830 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506375+0000", 2024-09-21T14:59:51.830 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7536000000000001e-05 2024-09-21T14:59:51.830 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.830 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.830 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:51.830 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.498815+0000", 2024-09-21T14:59:51.830 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99244067999999996 2024-09-21T14:59:51.830 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.830 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.830 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:51.831 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.498950+0000", 2024-09-21T14:59:51.831 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013436700000000001 2024-09-21T14:59:51.831 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.831 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.831 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:51.831 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499277+0000", 2024-09-21T14:59:51.831 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00032696800000000002 2024-09-21T14:59:51.831 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:51.831 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:51.831 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:51.831 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.831 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.831 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31513 3.c 3:36de8523:::rbd_data.11367721c19.0000000000000039:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T14:59:51.831 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.509071+0000", 2024-09-21T14:59:51.831 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 53.302329374999999, 2024-09-21T14:59:51.832 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99038225099999999, 2024-09-21T14:59:51.832 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:51.832 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:51.832 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:51.832 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:51.832 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:51.832 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31513 2024-09-21T14:59:51.832 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.832 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:51.832 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.832 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:51.832 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509071+0000", 2024-09-21T14:59:51.832 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:51.832 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.832 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.833 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:51.833 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509071+0000", 2024-09-21T14:59:51.833 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:51.833 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.833 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.833 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:51.833 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509068+0000", 2024-09-21T14:59:51.833 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-21T14:59:51.833 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.833 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.833 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:51.833 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509082+0000", 2024-09-21T14:59:51.833 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3545e-05 2024-09-21T14:59:51.833 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.833 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.834 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:51.834 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509116+0000", 2024-09-21T14:59:51.834 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3942000000000001e-05 2024-09-21T14:59:51.834 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.834 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.834 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:51.834 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509134+0000", 2024-09-21T14:59:51.834 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8839999999999999e-05 2024-09-21T14:59:51.834 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.834 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.834 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:51.834 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509318+0000", 2024-09-21T14:59:51.834 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018326799999999999 2024-09-21T14:59:51.834 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.835 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.835 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T14:59:51.835 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509346+0000", 2024-09-21T14:59:51.835 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8019000000000001e-05 2024-09-21T14:59:51.835 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.835 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.835 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:51.835 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499330+0000", 2024-09-21T14:59:51.835 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.98998399299999995 2024-09-21T14:59:51.835 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.835 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.835 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:51.835 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499453+0000", 2024-09-21T14:59:51.835 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000123115 2024-09-21T14:59:51.836 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:51.836 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:51.836 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:51.836 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.836 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.836 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31515 3.c 3:35b36ddd:::rbd_data.11367721c19.000000000000003b:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T14:59:51.836 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.509177+0000", 2024-09-21T14:59:51.836 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 53.302222948999997, 2024-09-21T14:59:51.836 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99044870600000001, 2024-09-21T14:59:51.836 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:51.836 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:51.836 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:51.836 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:51.836 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:51.836 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31515 2024-09-21T14:59:51.837 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.837 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:51.837 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.837 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:51.837 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509177+0000", 2024-09-21T14:59:51.837 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:51.837 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.837 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.837 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:51.837 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509177+0000", 2024-09-21T14:59:51.837 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:51.837 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.837 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.837 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:51.838 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509176+0000", 2024-09-21T14:59:51.838 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:51.838 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.838 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.838 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:51.838 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509183+0000", 2024-09-21T14:59:51.838 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6720000000000004e-06 2024-09-21T14:59:51.838 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.838 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.838 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:51.838 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509269+0000", 2024-09-21T14:59:51.838 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5881e-05 2024-09-21T14:59:51.838 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.838 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.839 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:51.839 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509303+0000", 2024-09-21T14:59:51.839 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3367000000000001e-05 2024-09-21T14:59:51.839 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.839 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.839 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:51.839 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509425+0000", 2024-09-21T14:59:51.839 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000122199 2024-09-21T14:59:51.839 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.839 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.839 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T14:59:51.839 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509448+0000", 2024-09-21T14:59:51.839 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2688999999999999e-05 2024-09-21T14:59:51.839 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.839 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.840 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:51.840 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499501+0000", 2024-09-21T14:59:51.840 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99005306999999998 2024-09-21T14:59:51.840 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.840 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.840 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:51.840 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499626+0000", 2024-09-21T14:59:51.840 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012501400000000001 2024-09-21T14:59:51.840 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:51.840 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:51.840 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:51.840 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.840 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.840 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31531 3.c 3:314f0287:::rbd_data.11367721c19.000000000000004b:head [sparse-read 0~7106] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T14:59:51.840 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.509329+0000", 2024-09-21T14:59:51.841 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 53.302070929000003, 2024-09-21T14:59:51.841 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99103085300000004, 2024-09-21T14:59:51.841 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:51.841 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:51.841 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:51.841 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:51.841 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:51.841 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31531 2024-09-21T14:59:51.841 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.841 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:51.841 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.841 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:51.841 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509329+0000", 2024-09-21T14:59:51.841 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:51.842 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.842 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.842 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:51.842 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509329+0000", 2024-09-21T14:59:51.842 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:51.842 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.842 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.842 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:51.842 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509328+0000", 2024-09-21T14:59:51.842 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:59:51.842 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.842 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.842 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:51.842 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509336+0000", 2024-09-21T14:59:51.843 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3170000000000002e-06 2024-09-21T14:59:51.843 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.843 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.843 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:51.843 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509353+0000", 2024-09-21T14:59:51.843 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7214000000000002e-05 2024-09-21T14:59:51.843 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.843 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.843 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:51.843 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509366+0000", 2024-09-21T14:59:51.843 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.26e-05 2024-09-21T14:59:51.843 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.843 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.843 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:51.843 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509523+0000", 2024-09-21T14:59:51.844 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015708000000000001 2024-09-21T14:59:51.844 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.844 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.844 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T14:59:51.844 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509544+0000", 2024-09-21T14:59:51.844 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0783000000000001e-05 2024-09-21T14:59:51.844 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.844 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.844 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:51.844 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499727+0000", 2024-09-21T14:59:51.844 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99018345200000002 2024-09-21T14:59:51.844 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.844 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.845 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:51.845 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499949+0000", 2024-09-21T14:59:51.845 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000221358 2024-09-21T14:59:51.845 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.845 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.845 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:51.845 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.500360+0000", 2024-09-21T14:59:51.845 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00041130700000000002 2024-09-21T14:59:51.845 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:51.845 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:51.845 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:51.845 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.845 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.845 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:61580 3.b 3:d3369e4f:::rbd_data.1136862f872f.0000000000000010:head [write 0~557056 [fadvise_sequential+fadvise_nocache],zero 557056~16384,write 573440~49152 [fadvise_sequential+fadvise_nocache],zero 622592~16384,write 638976~49152 [fadvise_sequential+fadvise_nocache],zero 688128~16384,write 704512~49152 [fadvise_sequential+fadvise_nocache],zero 753664~16384,write 770048~49152 [fadvise_sequential+fadvise_nocache],zero 819200~16384,write 835584~49152 [fadvise_sequential+fadvise_nocache],zero 884736~16384,write 901120~49152 [fadvise_sequential+fadvise_nocache],zero 950272~16384,write 966656~49152 [fadvise_sequential+fadvise_nocache],zero 1015808~16384,write 1032192~16384 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-21T14:59:51.845 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:19.231716+0000", 2024-09-21T14:59:51.846 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 32.579683766999999, 2024-09-21T14:59:51.846 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0096097690000000006, 2024-09-21T14:59:51.846 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:51.846 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:51.846 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:51.846 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:51.846 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:51.846 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 61580 2024-09-21T14:59:51.846 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.846 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:51.846 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.846 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:51.846 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.231716+0000", 2024-09-21T14:59:51.846 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:51.846 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.847 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.847 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:51.847 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.231716+0000", 2024-09-21T14:59:51.847 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:51.847 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.847 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.847 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:51.847 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.231715+0000", 2024-09-21T14:59:51.847 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:59:51.847 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.847 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.847 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:51.847 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.235290+0000", 2024-09-21T14:59:51.847 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0035750399999999998 2024-09-21T14:59:51.847 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.848 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.848 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:51.848 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.235310+0000", 2024-09-21T14:59:51.848 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0120999999999999e-05 2024-09-21T14:59:51.848 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.848 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.848 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:51.848 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.235319+0000", 2024-09-21T14:59:51.848 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2420000000000006e-06 2024-09-21T14:59:51.848 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.848 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.848 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:51.848 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.235416+0000", 2024-09-21T14:59:51.848 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7204e-05 2024-09-21T14:59:51.848 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.848 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.849 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:51.849 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.235775+0000", 2024-09-21T14:59:51.849 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000358883 2024-09-21T14:59:51.849 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.849 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.849 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T14:59:51.849 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.236560+0000", 2024-09-21T14:59:51.849 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000784463 2024-09-21T14:59:51.849 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.849 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.849 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:59:51.849 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.241119+0000", 2024-09-21T14:59:51.849 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0045590589999999999 2024-09-21T14:59:51.849 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.850 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.850 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:51.850 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.241210+0000", 2024-09-21T14:59:51.850 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1617000000000001e-05 2024-09-21T14:59:51.850 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.850 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.850 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:59:51.850 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.241285+0000", 2024-09-21T14:59:51.850 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4470999999999999e-05 2024-09-21T14:59:51.850 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.850 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.850 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:51.850 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.241326+0000", 2024-09-21T14:59:51.850 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0967000000000003e-05 2024-09-21T14:59:51.851 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:51.851 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:51.851 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:51.851 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.851 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.851 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:73699 3.b 3:d293b54a:::rbd_data.1136f4939602.000000000000000a:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-21T14:59:51.851 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:41.971359+0000", 2024-09-21T14:59:51.851 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.8400412109999991, 2024-09-21T14:59:51.851 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0082840980000000002, 2024-09-21T14:59:51.851 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:51.851 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:51.851 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:51.851 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:51.851 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:51.851 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 73699 2024-09-21T14:59:51.852 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.852 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:51.852 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.852 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:51.852 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.971359+0000", 2024-09-21T14:59:51.852 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:51.852 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.852 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.852 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:51.852 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.971359+0000", 2024-09-21T14:59:51.852 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:51.852 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.852 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.852 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:51.852 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.971356+0000", 2024-09-21T14:59:51.852 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-21T14:59:51.853 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.853 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.853 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:51.853 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.971370+0000", 2024-09-21T14:59:51.853 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3349999999999999e-05 2024-09-21T14:59:51.853 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.853 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.853 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:51.853 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.971393+0000", 2024-09-21T14:59:51.853 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3799999999999999e-05 2024-09-21T14:59:51.853 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.853 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.853 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:51.853 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.971405+0000", 2024-09-21T14:59:51.853 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1569000000000001e-05 2024-09-21T14:59:51.853 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.854 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.854 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:51.854 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.971510+0000", 2024-09-21T14:59:51.854 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010456899999999999 2024-09-21T14:59:51.854 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.854 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.854 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:51.854 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.971752+0000", 2024-09-21T14:59:51.854 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00024250300000000001 2024-09-21T14:59:51.854 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.854 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.854 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T14:59:51.854 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.973012+0000", 2024-09-21T14:59:51.854 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0012595429999999999 2024-09-21T14:59:51.854 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.855 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.855 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:59:51.855 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.979357+0000", 2024-09-21T14:59:51.855 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0063454469999999997 2024-09-21T14:59:51.855 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.855 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.855 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:51.855 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.979467+0000", 2024-09-21T14:59:51.855 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000109623 2024-09-21T14:59:51.855 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.855 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.855 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:59:51.855 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.979585+0000", 2024-09-21T14:59:51.855 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000118044 2024-09-21T14:59:51.855 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.856 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.856 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:51.856 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.979643+0000", 2024-09-21T14:59:51.857 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8069999999999998e-05 2024-09-21T14:59:51.857 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:51.857 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:51.858 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:51.858 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.858 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.858 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:82545 3.b 3:d4eb1ee5:::rbd_data.11368296f791.0000000000000000:head [write 0~671744 [fadvise_sequential+fadvise_nocache],write 704512~65536 [fadvise_sequential+fadvise_nocache],write 786432~49152 [fadvise_sequential+fadvise_nocache],write 868352~49152 [fadvise_sequential+fadvise_nocache],write 933888~65536 [fadvise_sequential+fadvise_nocache],write 1032192~32768 [fadvise_sequential+fadvise_nocache],write 1097728~65536 [fadvise_sequential+fadvise_nocache],write 1179648~49152 [fadvise_sequential+fadvise_nocache],write 1261568~49152 [fadvise_sequential+fadvise_nocache],write 1327104~65536 [fadvise_sequential+fadvise_nocache],write 1425408~32768 [fadvise_sequential+fadvise_nocache],write 1490944~65536 [fadvise_sequential+fadvise_nocache],write 1572864~8192 [fadvise_sequential+fadvise_nocache],write 1630208~49152 [fadvise_sequential+fadvise_nocache],write 1703936~40960 [fadvise_sequential+fadvise_nocache],write 1794048~40960 [fadvise_sequential+fadvise_nocache],write 1859584~49152 [fadvise_sequential+fadvise_nocache],write 1957888~8192 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108)", 2024-09-21T14:59:51.858 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:47.416089+0000", 2024-09-21T14:59:51.858 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.3953112040000004, 2024-09-21T14:59:51.858 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011896455, 2024-09-21T14:59:51.858 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:51.858 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:51.858 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:51.858 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:51.858 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:51.858 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82545 2024-09-21T14:59:51.858 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.859 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:51.859 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.859 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:51.859 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.416089+0000", 2024-09-21T14:59:51.859 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:51.859 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.859 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.859 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:51.859 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.416089+0000", 2024-09-21T14:59:51.859 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:51.859 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.859 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.859 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:51.859 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.416087+0000", 2024-09-21T14:59:51.859 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T14:59:51.860 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.860 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.860 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:51.860 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.419643+0000", 2024-09-21T14:59:51.860 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0035555550000000002 2024-09-21T14:59:51.860 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.860 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.860 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:51.860 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.419663+0000", 2024-09-21T14:59:51.860 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0202000000000001e-05 2024-09-21T14:59:51.860 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.860 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.860 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:51.860 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.419676+0000", 2024-09-21T14:59:51.860 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3207000000000001e-05 2024-09-21T14:59:51.860 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.861 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.861 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:51.861 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.419759+0000", 2024-09-21T14:59:51.861 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3521000000000002e-05 2024-09-21T14:59:51.861 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.861 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.861 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op must wait for map", 2024-09-21T14:59:51.861 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.419768+0000", 2024-09-21T14:59:51.861 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4479999999999994e-06 2024-09-21T14:59:51.861 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.861 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.861 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:51.861 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.422171+0000", 2024-09-21T14:59:51.861 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0024035469999999998 2024-09-21T14:59:51.861 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.861 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.862 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:51.862 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.422350+0000", 2024-09-21T14:59:51.862 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000179001 2024-09-21T14:59:51.862 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.862 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.862 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T14:59:51.862 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.422710+0000", 2024-09-21T14:59:51.862 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000359878 2024-09-21T14:59:51.862 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.862 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.862 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:59:51.862 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.426676+0000", 2024-09-21T14:59:51.862 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0039655790000000003 2024-09-21T14:59:51.862 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.863 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.863 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:51.863 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.427884+0000", 2024-09-21T14:59:51.863 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001208207 2024-09-21T14:59:51.863 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.863 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.863 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:59:51.863 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.427953+0000", 2024-09-21T14:59:51.863 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8533000000000003e-05 2024-09-21T14:59:51.863 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.863 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.863 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:51.863 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.427985+0000", 2024-09-21T14:59:51.864 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2481999999999998e-05 2024-09-21T14:59:51.864 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:51.864 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:51.864 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:51.864 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.864 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.864 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:83723 3.b 3:d4eb1ee5:::rbd_data.11368296f791.0000000000000000:40 [sparse-read 1179648~0 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-21T14:59:51.864 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:51.156687+0000", 2024-09-21T14:59:51.864 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.65471299000000005, 2024-09-21T14:59:51.864 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0078343370000000002, 2024-09-21T14:59:51.864 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:51.864 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:51.864 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:51.864 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:51.865 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:51.865 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 83723 2024-09-21T14:59:51.865 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.865 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:51.865 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.865 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:51.865 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156687+0000", 2024-09-21T14:59:51.865 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:51.865 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.865 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.865 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:51.865 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156687+0000", 2024-09-21T14:59:51.865 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:51.865 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.866 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.866 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:51.866 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156686+0000", 2024-09-21T14:59:51.866 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:51.866 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.866 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.866 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:51.866 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156711+0000", 2024-09-21T14:59:51.866 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4776000000000001e-05 2024-09-21T14:59:51.866 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.866 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.866 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:51.866 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156723+0000", 2024-09-21T14:59:51.866 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1734e-05 2024-09-21T14:59:51.867 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.867 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.867 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:51.867 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156732+0000", 2024-09-21T14:59:51.867 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4390000000000006e-06 2024-09-21T14:59:51.867 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.867 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.867 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:51.867 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.164314+0000", 2024-09-21T14:59:51.867 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0075816579999999998 2024-09-21T14:59:51.867 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.867 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.867 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:51.867 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.164405+0000", 2024-09-21T14:59:51.868 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1247000000000002e-05 2024-09-21T14:59:51.868 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.868 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.868 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:51.868 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.164521+0000", 2024-09-21T14:59:51.868 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011599099999999999 2024-09-21T14:59:51.868 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:51.868 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:51.868 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:51.868 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.868 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.868 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:83724 3.b 3:d4eb1ee5:::rbd_data.11368296f791.0000000000000000:40 [sparse-read 2621440~0 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-21T14:59:51.868 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:51.156740+0000", 2024-09-21T14:59:51.869 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.65465949099999998, 2024-09-21T14:59:51.869 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0079578040000000006, 2024-09-21T14:59:51.869 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:51.869 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:51.869 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:51.869 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:51.869 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:51.869 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 83724 2024-09-21T14:59:51.869 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.869 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:51.869 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.869 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:51.869 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156740+0000", 2024-09-21T14:59:51.869 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:51.870 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.870 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.870 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:51.870 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156740+0000", 2024-09-21T14:59:51.870 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:51.870 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.870 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.870 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:51.870 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156740+0000", 2024-09-21T14:59:51.870 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:51.870 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.870 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.870 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:51.870 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156744+0000", 2024-09-21T14:59:51.871 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0300000000000004e-06 2024-09-21T14:59:51.871 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.871 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.871 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:51.871 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156753+0000", 2024-09-21T14:59:51.871 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5759999999999995e-06 2024-09-21T14:59:51.871 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.871 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.871 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:51.871 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156761+0000", 2024-09-21T14:59:51.871 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.17e-06 2024-09-21T14:59:51.871 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.871 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.871 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:51.871 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.164559+0000", 2024-09-21T14:59:51.872 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0077988739999999999 2024-09-21T14:59:51.872 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.872 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.872 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:51.872 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.164608+0000", 2024-09-21T14:59:51.872 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8906999999999998e-05 2024-09-21T14:59:51.872 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.872 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.872 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:51.872 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.164698+0000", 2024-09-21T14:59:51.872 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9854999999999999e-05 2024-09-21T14:59:51.872 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:51.872 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:51.872 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:51.872 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.873 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.873 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:83735 3.b 3:d3640016:::rbd_data.11368296f791.0000000000000006:40 [sparse-read 393216~131072 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-21T14:59:51.873 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:51.156767+0000", 2024-09-21T14:59:51.873 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.65463258099999999, 2024-09-21T14:59:51.873 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0082826410000000003, 2024-09-21T14:59:51.873 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:51.873 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:51.873 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:51.873 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:51.873 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:51.873 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 83735 2024-09-21T14:59:51.873 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.873 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:51.873 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.874 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:51.874 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156767+0000", 2024-09-21T14:59:51.874 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:51.874 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.874 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.874 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:51.874 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156767+0000", 2024-09-21T14:59:51.874 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:51.874 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.874 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.874 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:51.874 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156767+0000", 2024-09-21T14:59:51.875 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:51.875 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.875 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.875 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:51.875 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156771+0000", 2024-09-21T14:59:51.875 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0940000000000001e-06 2024-09-21T14:59:51.875 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.875 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.875 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:51.875 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156781+0000", 2024-09-21T14:59:51.875 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0267e-05 2024-09-21T14:59:51.875 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.875 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.876 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:51.876 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156789+0000", 2024-09-21T14:59:51.876 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7659999999999998e-06 2024-09-21T14:59:51.876 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.876 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.876 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:51.876 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.164742+0000", 2024-09-21T14:59:51.876 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0079534519999999997 2024-09-21T14:59:51.876 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.876 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.876 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:51.876 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.164820+0000", 2024-09-21T14:59:51.876 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7927000000000001e-05 2024-09-21T14:59:51.876 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.877 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.877 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:51.877 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165050+0000", 2024-09-21T14:59:51.877 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000229731 2024-09-21T14:59:51.877 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:51.877 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:51.877 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:51.877 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.877 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.877 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:83736 3.b 3:d3640016:::rbd_data.11368296f791.0000000000000006:40 [sparse-read 1835008~0 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-21T14:59:51.877 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:51.156796+0000", 2024-09-21T14:59:51.877 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.65460357800000002, 2024-09-21T14:59:51.877 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0085139110000000007, 2024-09-21T14:59:51.877 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:51.877 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:51.878 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:51.878 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:51.878 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:51.878 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 83736 2024-09-21T14:59:51.878 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.878 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:51.878 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.878 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:51.878 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156796+0000", 2024-09-21T14:59:51.878 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:51.878 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.878 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.878 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:51.879 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156796+0000", 2024-09-21T14:59:51.879 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:51.879 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.879 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.879 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:51.879 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156796+0000", 2024-09-21T14:59:51.879 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:51.879 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.879 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.879 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:51.879 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156800+0000", 2024-09-21T14:59:51.879 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.34e-06 2024-09-21T14:59:51.879 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.879 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.880 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:51.880 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156813+0000", 2024-09-21T14:59:51.880 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2532e-05 2024-09-21T14:59:51.880 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.880 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.880 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:51.880 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156821+0000", 2024-09-21T14:59:51.880 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2069999999999995e-06 2024-09-21T14:59:51.880 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.880 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.880 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:51.880 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165100+0000", 2024-09-21T14:59:51.880 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0082788730000000008 2024-09-21T14:59:51.880 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.881 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.881 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:51.881 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165192+0000", 2024-09-21T14:59:51.881 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2095999999999994e-05 2024-09-21T14:59:51.881 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.881 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.881 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:51.881 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165310+0000", 2024-09-21T14:59:51.881 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000118461 2024-09-21T14:59:51.881 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:51.881 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:51.881 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:51.881 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.881 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.882 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:83739 3.b 3:d4eb1ee5:::rbd_data.11368296f791.0000000000000000:40 [sparse-read 524288~131072 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-21T14:59:51.882 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:51.156827+0000", 2024-09-21T14:59:51.882 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.65457249500000003, 2024-09-21T14:59:51.882 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0087319900000000002, 2024-09-21T14:59:51.882 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:51.882 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:51.882 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:51.882 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:51.882 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:51.882 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 83739 2024-09-21T14:59:51.882 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.882 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:51.882 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.883 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:51.883 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156827+0000", 2024-09-21T14:59:51.883 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:51.883 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.883 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.883 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:51.883 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156827+0000", 2024-09-21T14:59:51.883 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:51.883 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.883 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.883 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:51.883 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156827+0000", 2024-09-21T14:59:51.883 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:51.883 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.884 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.884 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:51.884 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156831+0000", 2024-09-21T14:59:51.884 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9940000000000002e-06 2024-09-21T14:59:51.884 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.884 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.884 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:51.884 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156841+0000", 2024-09-21T14:59:51.884 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6760000000000002e-06 2024-09-21T14:59:51.884 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.884 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.884 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:51.884 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156853+0000", 2024-09-21T14:59:51.884 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1972e-05 2024-09-21T14:59:51.885 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.885 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.885 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:51.885 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165334+0000", 2024-09-21T14:59:51.885 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0084816939999999997 2024-09-21T14:59:51.885 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.885 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.885 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:51.885 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165375+0000", 2024-09-21T14:59:51.885 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0911000000000002e-05 2024-09-21T14:59:51.885 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.885 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.885 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:51.885 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165559+0000", 2024-09-21T14:59:51.885 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018429300000000001 2024-09-21T14:59:51.886 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:51.886 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:51.886 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:51.886 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.886 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.886 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:83740 3.b 3:d4eb1ee5:::rbd_data.11368296f791.0000000000000000:40 [sparse-read 1966080~0 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-21T14:59:51.886 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:51.156859+0000", 2024-09-21T14:59:51.886 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.65454050500000005, 2024-09-21T14:59:51.886 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0088540089999999995, 2024-09-21T14:59:51.886 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:51.886 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:51.886 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:51.886 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:51.887 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:51.887 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 83740 2024-09-21T14:59:51.887 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.887 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:51.887 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.887 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:51.887 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156859+0000", 2024-09-21T14:59:51.887 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:51.887 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.887 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.887 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:51.887 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156859+0000", 2024-09-21T14:59:51.887 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:51.887 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.888 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.888 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:51.888 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156859+0000", 2024-09-21T14:59:51.888 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:51.888 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.888 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.888 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:51.888 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156863+0000", 2024-09-21T14:59:51.888 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.844e-06 2024-09-21T14:59:51.888 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.888 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.888 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:51.888 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156873+0000", 2024-09-21T14:59:51.888 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0069e-05 2024-09-21T14:59:51.889 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.889 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.889 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:51.889 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156880+0000", 2024-09-21T14:59:51.889 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4240000000000004e-06 2024-09-21T14:59:51.889 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.889 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.889 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:51.889 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165590+0000", 2024-09-21T14:59:51.889 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0087098100000000001 2024-09-21T14:59:51.889 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.889 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.889 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:51.889 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165656+0000", 2024-09-21T14:59:51.889 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5541000000000001e-05 2024-09-21T14:59:51.890 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.890 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.890 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:51.890 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165713+0000", 2024-09-21T14:59:51.890 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7865000000000002e-05 2024-09-21T14:59:51.890 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:51.890 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:51.890 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:51.890 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.890 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.890 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:83751 3.b 3:d3640016:::rbd_data.11368296f791.0000000000000006:40 [sparse-read 1179648~0 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-21T14:59:51.890 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:51.156888+0000", 2024-09-21T14:59:51.890 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.65451224399999997, 2024-09-21T14:59:51.890 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0090039539999999998, 2024-09-21T14:59:51.891 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:51.891 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:51.891 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:51.891 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:51.891 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:51.891 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 83751 2024-09-21T14:59:51.891 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.891 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:51.891 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.891 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:51.891 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156888+0000", 2024-09-21T14:59:51.891 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:51.891 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.891 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.891 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:51.891 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156888+0000", 2024-09-21T14:59:51.892 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:51.892 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.892 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.892 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:51.892 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156887+0000", 2024-09-21T14:59:51.892 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:51.892 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.892 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.892 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:51.892 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156891+0000", 2024-09-21T14:59:51.892 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.518e-06 2024-09-21T14:59:51.892 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.892 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.892 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:51.893 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156900+0000", 2024-09-21T14:59:51.893 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1759999999999999e-06 2024-09-21T14:59:51.893 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.893 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.893 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:51.893 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156908+0000", 2024-09-21T14:59:51.893 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8439999999999998e-06 2024-09-21T14:59:51.893 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.893 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.893 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:51.893 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165738+0000", 2024-09-21T14:59:51.893 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0088305689999999999 2024-09-21T14:59:51.893 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.893 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.893 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:51.894 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165804+0000", 2024-09-21T14:59:51.894 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5535000000000002e-05 2024-09-21T14:59:51.894 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.894 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.894 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:51.894 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165892+0000", 2024-09-21T14:59:51.894 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7860000000000002e-05 2024-09-21T14:59:51.894 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:51.894 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:51.894 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:51.894 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.894 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.894 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:83752 3.b 3:d3640016:::rbd_data.11368296f791.0000000000000006:40 [sparse-read 2621440~0 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-21T14:59:51.895 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:51.156915+0000", 2024-09-21T14:59:51.895 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.654484697, 2024-09-21T14:59:51.895 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0090768180000000004, 2024-09-21T14:59:51.895 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:51.895 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:51.895 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:51.895 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:51.895 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:51.895 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 83752 2024-09-21T14:59:51.895 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.895 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:51.895 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.895 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:51.895 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156915+0000", 2024-09-21T14:59:51.896 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:51.896 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.896 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.896 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:51.896 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156915+0000", 2024-09-21T14:59:51.896 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:51.896 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.896 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.896 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:51.896 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156915+0000", 2024-09-21T14:59:51.897 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:51.897 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.897 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.897 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:51.897 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156919+0000", 2024-09-21T14:59:51.897 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2679999999999996e-06 2024-09-21T14:59:51.897 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.897 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.897 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:51.897 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156929+0000", 2024-09-21T14:59:51.897 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.031e-05 2024-09-21T14:59:51.897 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.897 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.897 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:51.897 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156937+0000", 2024-09-21T14:59:51.897 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8820000000000001e-06 2024-09-21T14:59:51.898 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.898 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.898 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:51.898 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165909+0000", 2024-09-21T14:59:51.898 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0089722989999999996 2024-09-21T14:59:51.898 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.898 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.898 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:51.898 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165948+0000", 2024-09-21T14:59:51.898 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8114999999999997e-05 2024-09-21T14:59:51.898 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.898 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.898 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:51.898 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165992+0000", 2024-09-21T14:59:51.898 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4503999999999999e-05 2024-09-21T14:59:51.899 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:51.899 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:51.899 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:51.899 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.899 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.899 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:83755 3.b 3:d4eb1ee5:::rbd_data.11368296f791.0000000000000000:40 [sparse-read 1310720~0 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-21T14:59:51.899 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:51.156944+0000", 2024-09-21T14:59:51.899 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.65445562300000004, 2024-09-21T14:59:51.899 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.009159146, 2024-09-21T14:59:51.899 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:51.899 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:51.899 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:51.899 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:51.899 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:51.899 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 83755 2024-09-21T14:59:51.899 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.900 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:51.900 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.900 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:51.900 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156944+0000", 2024-09-21T14:59:51.900 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:51.900 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.900 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.900 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:51.900 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156944+0000", 2024-09-21T14:59:51.900 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:51.900 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.900 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.900 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:51.900 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156944+0000", 2024-09-21T14:59:51.900 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:51.901 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.901 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.901 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:51.901 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156948+0000", 2024-09-21T14:59:51.901 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5279999999999997e-06 2024-09-21T14:59:51.901 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.901 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.901 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:51.901 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156959+0000", 2024-09-21T14:59:51.901 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0215e-05 2024-09-21T14:59:51.901 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.901 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.901 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:51.901 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156967+0000", 2024-09-21T14:59:51.901 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6119999999999999e-06 2024-09-21T14:59:51.902 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.902 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.902 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:51.902 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.166007+0000", 2024-09-21T14:59:51.902 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0090395260000000009 2024-09-21T14:59:51.902 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.902 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.902 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:51.902 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.166053+0000", 2024-09-21T14:59:51.902 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6281e-05 2024-09-21T14:59:51.902 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.902 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.902 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:51.902 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.166103+0000", 2024-09-21T14:59:51.902 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0510000000000003e-05 2024-09-21T14:59:51.903 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:51.903 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:51.903 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:51.903 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.903 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.903 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:83763 3.b 3:d3640016:::rbd_data.11368296f791.0000000000000006:40 [sparse-read 524288~81920 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-21T14:59:51.903 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:51.157975+0000", 2024-09-21T14:59:51.903 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.65342451499999998, 2024-09-21T14:59:51.903 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.008327348, 2024-09-21T14:59:51.903 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:51.903 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:51.903 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:51.903 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:51.903 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:51.903 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 83763 2024-09-21T14:59:51.903 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.904 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:51.904 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.904 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:51.904 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.157975+0000", 2024-09-21T14:59:51.904 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:51.904 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.904 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.904 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:51.904 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.157975+0000", 2024-09-21T14:59:51.904 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:51.904 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.904 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.904 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:51.904 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.157970+0000", 2024-09-21T14:59:51.904 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999943 2024-09-21T14:59:51.904 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.905 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.905 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:51.905 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.157985+0000", 2024-09-21T14:59:51.905 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4807000000000001e-05 2024-09-21T14:59:51.905 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.905 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.905 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:51.905 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.158012+0000", 2024-09-21T14:59:51.905 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7600999999999999e-05 2024-09-21T14:59:51.905 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.905 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.905 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:51.905 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.158019+0000", 2024-09-21T14:59:51.905 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9319999999999999e-06 2024-09-21T14:59:51.905 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.905 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.906 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:51.906 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.166118+0000", 2024-09-21T14:59:51.906 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0080992489999999993 2024-09-21T14:59:51.906 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.906 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.906 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:51.906 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.166156+0000", 2024-09-21T14:59:51.906 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7639999999999999e-05 2024-09-21T14:59:51.906 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.906 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.906 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:51.906 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.166303+0000", 2024-09-21T14:59:51.906 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000146799 2024-09-21T14:59:51.906 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:51.907 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:51.907 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:51.907 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.907 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.907 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:83764 3.b 3:d3640016:::rbd_data.11368296f791.0000000000000006:40 [sparse-read 1966080~0 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-21T14:59:51.907 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:51.158026+0000", 2024-09-21T14:59:51.907 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.65337408200000002, 2024-09-21T14:59:51.907 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0083785839999999997, 2024-09-21T14:59:51.907 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:51.907 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:51.907 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:51.907 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:51.907 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:51.907 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 83764 2024-09-21T14:59:51.908 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.908 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:51.908 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.908 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:51.908 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.158026+0000", 2024-09-21T14:59:51.908 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:51.908 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.908 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.908 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:51.908 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.158026+0000", 2024-09-21T14:59:51.908 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:51.908 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.908 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.908 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:51.908 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.158025+0000", 2024-09-21T14:59:51.908 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:51.908 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.909 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.909 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:51.909 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.158053+0000", 2024-09-21T14:59:51.909 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7594e-05 2024-09-21T14:59:51.909 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.909 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.909 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:51.909 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.158061+0000", 2024-09-21T14:59:51.909 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2069999999999995e-06 2024-09-21T14:59:51.909 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.909 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.909 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:51.909 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.158067+0000", 2024-09-21T14:59:51.909 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3799999999999999e-06 2024-09-21T14:59:51.909 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.910 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.910 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:51.910 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.166321+0000", 2024-09-21T14:59:51.910 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0082539870000000008 2024-09-21T14:59:51.910 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.910 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.910 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:51.910 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.166360+0000", 2024-09-21T14:59:51.910 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8597999999999999e-05 2024-09-21T14:59:51.910 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:51.910 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:51.910 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:51.910 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.166404+0000", 2024-09-21T14:59:51.910 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4378000000000001e-05 2024-09-21T14:59:51.910 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:51.911 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:51.911 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:51.911 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:51.911 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:51.911 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:59:51.912 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T14:59:51.913 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:59:51.913+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:59:52.013 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:59:52.007+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:59:52.033 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:52.033 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:59:52.033 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T14:59:52.033 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:52.043 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T14:59:52.115 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:59:52.113+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:59:52.151 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T14:59:52.152 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:52.152 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:59:52.152 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T14:59:52.152 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T14:59:52.152 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:52.164 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T14:59:52.216 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:59:52.211+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:59:52.295 INFO:teuthology.orchestra.run.smithi032.stdout:checking clone #19, image image_client.0-clone19 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent20 2024-09-21T14:59:52.316 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:59:52.317+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:59:52.334 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:52.335 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T14:59:52.335 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T14:59:52.335 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T14:59:52.335 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.335 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:1473 3.15 3:af2afa6a:::rbd_data.1136515ecb6b.000000000000001b:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T14:59:52.335 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:30.421730+0000", 2024-09-21T14:59:52.335 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 81.911937116000004, 2024-09-21T14:59:52.335 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021478937, 2024-09-21T14:59:52.335 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:52.335 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:52.335 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:52.335 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:52.336 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:52.336 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1473 2024-09-21T14:59:52.336 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.336 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:52.336 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.336 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:52.336 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421730+0000", 2024-09-21T14:59:52.336 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:52.336 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.336 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.336 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:52.336 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421730+0000", 2024-09-21T14:59:52.336 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:52.337 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.337 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.337 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:52.337 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421728+0000", 2024-09-21T14:59:52.337 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-21T14:59:52.337 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.337 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.337 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:52.337 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421739+0000", 2024-09-21T14:59:52.337 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0502e-05 2024-09-21T14:59:52.337 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.337 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.337 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:52.337 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421762+0000", 2024-09-21T14:59:52.337 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3405999999999999e-05 2024-09-21T14:59:52.338 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.338 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.338 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:52.338 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421773+0000", 2024-09-21T14:59:52.338 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0744000000000001e-05 2024-09-21T14:59:52.338 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.338 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.338 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:52.338 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421838+0000", 2024-09-21T14:59:52.338 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5486000000000003e-05 2024-09-21T14:59:52.339 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.339 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.339 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:52.339 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421955+0000", 2024-09-21T14:59:52.339 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000116765 2024-09-21T14:59:52.339 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.339 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.339 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 6", 2024-09-21T14:59:52.339 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.425572+0000", 2024-09-21T14:59:52.339 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0036171380000000002 2024-09-21T14:59:52.339 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.339 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.339 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:52.339 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.442973+0000", 2024-09-21T14:59:52.340 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.017400268 2024-09-21T14:59:52.340 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.340 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.340 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:52.340 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.442996+0000", 2024-09-21T14:59:52.340 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3139e-05 2024-09-21T14:59:52.340 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.340 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.340 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:52.340 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.443079+0000", 2024-09-21T14:59:52.340 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2832999999999998e-05 2024-09-21T14:59:52.340 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.340 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.340 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:52.341 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.443209+0000", 2024-09-21T14:59:52.341 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013052499999999999 2024-09-21T14:59:52.341 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:52.341 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:52.341 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:52.341 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.341 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.341 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:1678 3.c 3:377cc7e1:::rbd_data.1136515ecb6b.000000000000000f:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T14:59:52.341 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:30.988085+0000", 2024-09-21T14:59:52.341 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 81.345582522000001, 2024-09-21T14:59:52.341 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.032974802999999997, 2024-09-21T14:59:52.341 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:52.341 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:52.341 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:52.342 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:52.342 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:52.342 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1678 2024-09-21T14:59:52.342 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.342 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:52.342 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.342 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:52.342 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988085+0000", 2024-09-21T14:59:52.343 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:52.343 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.343 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.343 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:52.343 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988085+0000", 2024-09-21T14:59:52.343 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:52.343 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.343 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.343 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:52.343 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988083+0000", 2024-09-21T14:59:52.343 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-21T14:59:52.343 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.343 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.343 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:52.344 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988093+0000", 2024-09-21T14:59:52.344 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8800000000000003e-06 2024-09-21T14:59:52.344 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.344 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.344 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:52.344 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988113+0000", 2024-09-21T14:59:52.344 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0217999999999999e-05 2024-09-21T14:59:52.344 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.344 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.344 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:52.344 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988123+0000", 2024-09-21T14:59:52.344 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5859999999999994e-06 2024-09-21T14:59:52.344 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.344 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.344 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:52.345 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988203+0000", 2024-09-21T14:59:52.345 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0834000000000006e-05 2024-09-21T14:59:52.345 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.345 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.345 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:52.345 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988376+0000", 2024-09-21T14:59:52.345 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017301299999999999 2024-09-21T14:59:52.345 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.345 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.345 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 7", 2024-09-21T14:59:52.345 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.996338+0000", 2024-09-21T14:59:52.345 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0079618520000000002 2024-09-21T14:59:52.345 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.345 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.345 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:52.345 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.020881+0000", 2024-09-21T14:59:52.346 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024542731000000002 2024-09-21T14:59:52.346 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.346 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.346 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:52.346 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.020904+0000", 2024-09-21T14:59:52.346 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2861e-05 2024-09-21T14:59:52.346 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.346 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.346 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:52.346 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.020987+0000", 2024-09-21T14:59:52.346 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2806000000000006e-05 2024-09-21T14:59:52.346 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.346 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.346 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:52.347 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.021060+0000", 2024-09-21T14:59:52.347 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2871000000000001e-05 2024-09-21T14:59:52.347 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:52.347 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:52.347 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:52.347 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.347 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.347 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9370 3.3 3:c725132c:::rbd_data.11363a21d2be.0000000000000007:10 [read 163840~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:52.347 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157279+0000", 2024-09-21T14:59:52.347 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 73.176387898000002, 2024-09-21T14:59:52.347 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018472379000000001, 2024-09-21T14:59:52.347 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:52.348 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:52.348 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:52.348 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:52.348 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:52.348 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9370 2024-09-21T14:59:52.348 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.348 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:52.348 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.348 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:52.348 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157279+0000", 2024-09-21T14:59:52.348 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:52.348 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.348 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.348 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:52.349 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157279+0000", 2024-09-21T14:59:52.349 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:52.349 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.349 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.349 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:52.349 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157279+0000", 2024-09-21T14:59:52.349 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:52.349 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.349 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.349 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:52.349 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157282+0000", 2024-09-21T14:59:52.349 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2229999999999999e-06 2024-09-21T14:59:52.349 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.349 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.350 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:52.350 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157294+0000", 2024-09-21T14:59:52.350 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2173e-05 2024-09-21T14:59:52.350 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.350 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.350 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:52.350 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157300+0000", 2024-09-21T14:59:52.350 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.93e-06 2024-09-21T14:59:52.350 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.350 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.350 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:52.350 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175569+0000", 2024-09-21T14:59:52.350 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018268376999999999 2024-09-21T14:59:52.351 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.351 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.351 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:52.351 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175622+0000", 2024-09-21T14:59:52.351 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3808999999999999e-05 2024-09-21T14:59:52.351 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.351 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.351 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:52.351 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175752+0000", 2024-09-21T14:59:52.351 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012938700000000001 2024-09-21T14:59:52.351 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:52.351 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:52.351 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:52.351 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.352 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.352 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9371 3.3 3:c0bf6a9b:::rbd_data.11363a21d2be.0000000000000000:10 [read 73728~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:52.352 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157306+0000", 2024-09-21T14:59:52.352 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 73.176361787000005, 2024-09-21T14:59:52.352 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018755473000000002, 2024-09-21T14:59:52.352 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:52.352 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:52.352 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:52.352 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:52.352 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:52.352 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9371 2024-09-21T14:59:52.352 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.352 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:52.353 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.353 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:52.353 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157306+0000", 2024-09-21T14:59:52.353 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:52.353 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.353 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.353 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:52.353 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157306+0000", 2024-09-21T14:59:52.353 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:52.353 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.353 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.353 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:52.353 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157305+0000", 2024-09-21T14:59:52.354 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:52.354 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.354 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.354 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:52.354 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157312+0000", 2024-09-21T14:59:52.354 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3289999999999996e-06 2024-09-21T14:59:52.354 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.354 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.354 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:52.354 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157324+0000", 2024-09-21T14:59:52.354 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1725e-05 2024-09-21T14:59:52.354 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.354 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.354 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:52.355 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157339+0000", 2024-09-21T14:59:52.355 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5211999999999999e-05 2024-09-21T14:59:52.355 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.355 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.355 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:52.355 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175798+0000", 2024-09-21T14:59:52.355 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018458781 2024-09-21T14:59:52.355 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.355 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.355 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:52.355 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175899+0000", 2024-09-21T14:59:52.355 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000100784 2024-09-21T14:59:52.355 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.355 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.356 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:52.356 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176061+0000", 2024-09-21T14:59:52.356 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00016216099999999999 2024-09-21T14:59:52.356 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:52.356 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:52.356 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:52.356 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.356 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.356 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9372 3.3 3:c0bf6a9b:::rbd_data.11363a21d2be.0000000000000000:10 [read 212992~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:52.356 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157345+0000", 2024-09-21T14:59:52.356 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 73.176322440999996, 2024-09-21T14:59:52.356 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018926493999999999, 2024-09-21T14:59:52.356 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:52.356 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:52.356 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:52.357 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:52.357 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:52.357 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9372 2024-09-21T14:59:52.357 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.357 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:52.357 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.357 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:52.357 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157345+0000", 2024-09-21T14:59:52.357 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:52.357 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.357 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.357 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:52.357 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157345+0000", 2024-09-21T14:59:52.358 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:52.358 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.358 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.358 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:52.358 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157344+0000", 2024-09-21T14:59:52.358 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:52.358 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.358 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.358 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:52.358 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157348+0000", 2024-09-21T14:59:52.358 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1389999999999999e-06 2024-09-21T14:59:52.358 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.358 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.359 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:52.359 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157372+0000", 2024-09-21T14:59:52.359 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4711000000000001e-05 2024-09-21T14:59:52.359 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.359 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.359 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:52.359 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157378+0000", 2024-09-21T14:59:52.359 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9909999999999999e-06 2024-09-21T14:59:52.359 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.359 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.359 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:52.359 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176090+0000", 2024-09-21T14:59:52.359 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018712111999999999 2024-09-21T14:59:52.359 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.359 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.360 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:52.360 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176134+0000", 2024-09-21T14:59:52.360 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3377999999999997e-05 2024-09-21T14:59:52.360 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.360 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.360 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:52.360 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176271+0000", 2024-09-21T14:59:52.360 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013763499999999999 2024-09-21T14:59:52.360 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:52.360 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:52.360 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:52.360 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.360 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.360 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9373 3.3 3:c0bf6a9b:::rbd_data.11363a21d2be.0000000000000000:10 [read 352256~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:52.361 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157384+0000", 2024-09-21T14:59:52.361 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 73.176283669, 2024-09-21T14:59:52.361 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019154937, 2024-09-21T14:59:52.361 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:52.361 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:52.361 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:52.361 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:52.361 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:52.361 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9373 2024-09-21T14:59:52.361 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.361 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:52.361 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.361 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:52.362 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157384+0000", 2024-09-21T14:59:52.362 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:52.362 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.362 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.362 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:52.362 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157384+0000", 2024-09-21T14:59:52.362 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:52.362 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.362 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.362 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:52.362 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157383+0000", 2024-09-21T14:59:52.362 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:52.362 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.362 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.363 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:52.363 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157386+0000", 2024-09-21T14:59:52.363 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.191e-06 2024-09-21T14:59:52.363 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.363 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.363 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:52.363 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157393+0000", 2024-09-21T14:59:52.363 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5339999999999996e-06 2024-09-21T14:59:52.363 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.363 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.363 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:52.363 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157397+0000", 2024-09-21T14:59:52.363 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5220000000000001e-06 2024-09-21T14:59:52.363 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.363 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.364 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:52.364 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176302+0000", 2024-09-21T14:59:52.364 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018905113000000001 2024-09-21T14:59:52.364 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.364 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.364 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:52.364 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176379+0000", 2024-09-21T14:59:52.364 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6372999999999995e-05 2024-09-21T14:59:52.364 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.364 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.364 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:52.364 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176539+0000", 2024-09-21T14:59:52.364 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015973800000000001 2024-09-21T14:59:52.364 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:52.365 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:52.365 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:52.365 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.365 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.365 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9374 3.3 3:c0bf6a9b:::rbd_data.11363a21d2be.0000000000000000:10 [read 491520~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:52.365 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157401+0000", 2024-09-21T14:59:52.365 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 73.176265885000007, 2024-09-21T14:59:52.365 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019387755, 2024-09-21T14:59:52.365 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:52.365 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:52.365 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:52.365 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:52.365 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:52.365 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9374 2024-09-21T14:59:52.366 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.366 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:52.366 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.366 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:52.366 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157401+0000", 2024-09-21T14:59:52.366 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:52.366 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.366 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.366 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:52.366 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157401+0000", 2024-09-21T14:59:52.366 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:52.366 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.366 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.367 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:52.367 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157401+0000", 2024-09-21T14:59:52.367 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:52.367 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.367 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.367 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:52.367 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157414+0000", 2024-09-21T14:59:52.367 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2417e-05 2024-09-21T14:59:52.367 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.367 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.367 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:52.367 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157424+0000", 2024-09-21T14:59:52.367 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0861e-05 2024-09-21T14:59:52.367 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.368 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.368 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:52.368 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157429+0000", 2024-09-21T14:59:52.368 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0499999999999999e-06 2024-09-21T14:59:52.368 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.368 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.368 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:52.368 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176596+0000", 2024-09-21T14:59:52.368 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019166552 2024-09-21T14:59:52.368 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.368 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.368 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:52.368 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176647+0000", 2024-09-21T14:59:52.369 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1112000000000001e-05 2024-09-21T14:59:52.369 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.369 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.369 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:52.369 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176789+0000", 2024-09-21T14:59:52.369 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014209300000000001 2024-09-21T14:59:52.369 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:52.369 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:52.369 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:52.369 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.369 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.369 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9401 3.3 3:c725132c:::rbd_data.11363a21d2be.0000000000000007:10 [read 32768~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:52.369 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157621+0000", 2024-09-21T14:59:52.369 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 73.176046076000006, 2024-09-21T14:59:52.370 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019381711999999999, 2024-09-21T14:59:52.370 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:52.370 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:52.370 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:52.370 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:52.370 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:52.370 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9401 2024-09-21T14:59:52.370 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.370 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:52.370 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.370 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:52.370 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157621+0000", 2024-09-21T14:59:52.370 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:52.370 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.371 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.371 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:52.371 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157621+0000", 2024-09-21T14:59:52.371 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:52.371 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.371 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.371 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:52.371 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157620+0000", 2024-09-21T14:59:52.371 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:59:52.371 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.371 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.371 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:52.371 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157626+0000", 2024-09-21T14:59:52.372 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7570000000000001e-06 2024-09-21T14:59:52.372 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.372 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.372 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:52.372 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157636+0000", 2024-09-21T14:59:52.372 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8339999999999994e-06 2024-09-21T14:59:52.372 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.372 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.372 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:52.372 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157643+0000", 2024-09-21T14:59:52.372 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9439999999999999e-06 2024-09-21T14:59:52.372 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.372 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.372 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:52.373 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176816+0000", 2024-09-21T14:59:52.373 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019172872000000001 2024-09-21T14:59:52.373 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.373 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.373 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:52.373 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176878+0000", 2024-09-21T14:59:52.373 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1734000000000002e-05 2024-09-21T14:59:52.373 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.373 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.373 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:52.373 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177003+0000", 2024-09-21T14:59:52.373 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012537400000000001 2024-09-21T14:59:52.373 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:52.373 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:52.374 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:52.374 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.374 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.374 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9402 3.3 3:c725132c:::rbd_data.11363a21d2be.0000000000000007:10 [read 172032~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:52.374 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157649+0000", 2024-09-21T14:59:52.374 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 73.176018204000002, 2024-09-21T14:59:52.374 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019567404, 2024-09-21T14:59:52.374 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:52.374 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:52.374 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:52.374 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:52.374 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:52.375 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9402 2024-09-21T14:59:52.375 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.375 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:52.375 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.375 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:52.375 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157649+0000", 2024-09-21T14:59:52.375 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:52.375 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.375 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.375 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:52.375 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157649+0000", 2024-09-21T14:59:52.375 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:52.375 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.376 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.376 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:52.376 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157649+0000", 2024-09-21T14:59:52.376 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:52.376 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.376 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.376 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:52.376 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157652+0000", 2024-09-21T14:59:52.376 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8779999999999998e-06 2024-09-21T14:59:52.376 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.376 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.376 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:52.376 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157657+0000", 2024-09-21T14:59:52.376 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6659999999999996e-06 2024-09-21T14:59:52.376 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.377 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.377 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:52.377 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157662+0000", 2024-09-21T14:59:52.377 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5519999999999998e-06 2024-09-21T14:59:52.377 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.377 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.377 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:52.377 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177040+0000", 2024-09-21T14:59:52.377 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019377807 2024-09-21T14:59:52.377 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.377 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.377 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:52.377 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177085+0000", 2024-09-21T14:59:52.377 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5195999999999999e-05 2024-09-21T14:59:52.378 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.378 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.378 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:52.378 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177217+0000", 2024-09-21T14:59:52.378 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013172600000000001 2024-09-21T14:59:52.378 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:52.378 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:52.378 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:52.378 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.378 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.378 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9403 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:52.378 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157949+0000", 2024-09-21T14:59:52.378 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 73.175717875000004, 2024-09-21T14:59:52.378 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023540373999999999, 2024-09-21T14:59:52.379 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:52.379 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:52.379 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:52.379 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:52.379 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:52.379 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9403 2024-09-21T14:59:52.379 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.379 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:52.379 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.379 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:52.379 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157949+0000", 2024-09-21T14:59:52.379 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:52.379 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.380 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.380 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:52.380 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157949+0000", 2024-09-21T14:59:52.380 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:52.380 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.380 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.380 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:52.380 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157948+0000", 2024-09-21T14:59:52.380 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T14:59:52.380 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.380 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.380 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:52.380 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157956+0000", 2024-09-21T14:59:52.380 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7999999999999999e-06 2024-09-21T14:59:52.381 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.381 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.381 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:52.381 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157969+0000", 2024-09-21T14:59:52.381 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2792000000000001e-05 2024-09-21T14:59:52.381 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.381 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.381 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:52.381 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157977+0000", 2024-09-21T14:59:52.381 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.704e-06 2024-09-21T14:59:52.381 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.381 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.381 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:52.381 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177246+0000", 2024-09-21T14:59:52.382 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019268264 2024-09-21T14:59:52.382 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.382 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.382 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:52.382 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177315+0000", 2024-09-21T14:59:52.382 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9480999999999994e-05 2024-09-21T14:59:52.382 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.382 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.382 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:59:52.382 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177615+0000", 2024-09-21T14:59:52.382 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00029929499999999999 2024-09-21T14:59:52.382 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.382 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.382 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:52.383 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181333+0000", 2024-09-21T14:59:52.383 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0037183149999999998 2024-09-21T14:59:52.383 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.383 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.383 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:52.383 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181374+0000", 2024-09-21T14:59:52.383 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.0723999999999998e-05 2024-09-21T14:59:52.383 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.383 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.383 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:52.383 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181434+0000", 2024-09-21T14:59:52.383 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0024e-05 2024-09-21T14:59:52.383 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.383 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.384 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:52.384 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181490+0000", 2024-09-21T14:59:52.384 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6245000000000001e-05 2024-09-21T14:59:52.384 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:52.384 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:52.384 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:52.384 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.384 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.384 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9404 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:52.384 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.160480+0000", 2024-09-21T14:59:52.384 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 73.173187525000003, 2024-09-21T14:59:52.384 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021137686999999999, 2024-09-21T14:59:52.384 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:52.385 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:52.385 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:52.385 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:52.385 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:52.385 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9404 2024-09-21T14:59:52.385 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.385 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:52.385 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.385 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:52.385 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160480+0000", 2024-09-21T14:59:52.385 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:52.385 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.385 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.385 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:52.385 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160480+0000", 2024-09-21T14:59:52.386 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:52.386 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.386 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.386 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:52.386 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160478+0000", 2024-09-21T14:59:52.386 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T14:59:52.386 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.386 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.386 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:52.386 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160486+0000", 2024-09-21T14:59:52.386 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9969999999999998e-06 2024-09-21T14:59:52.386 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.386 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.387 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:52.387 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160502+0000", 2024-09-21T14:59:52.387 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5650000000000001e-05 2024-09-21T14:59:52.387 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.387 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.387 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:52.387 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160511+0000", 2024-09-21T14:59:52.387 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.3610000000000007e-06 2024-09-21T14:59:52.387 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.387 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.387 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:52.387 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.178035+0000", 2024-09-21T14:59:52.387 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.017523520000000001 2024-09-21T14:59:52.387 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.388 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.388 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:52.388 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.178151+0000", 2024-09-21T14:59:52.388 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000116013 2024-09-21T14:59:52.388 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.388 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.388 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:59:52.388 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.178408+0000", 2024-09-21T14:59:52.388 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00025743400000000002 2024-09-21T14:59:52.388 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.388 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.388 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:52.388 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181501+0000", 2024-09-21T14:59:52.388 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.003092891 2024-09-21T14:59:52.389 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.389 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.389 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:52.389 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181538+0000", 2024-09-21T14:59:52.389 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6977999999999997e-05 2024-09-21T14:59:52.389 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.389 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.389 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:52.389 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181582+0000", 2024-09-21T14:59:52.389 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3849e-05 2024-09-21T14:59:52.389 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.389 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.389 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:52.389 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181617+0000", 2024-09-21T14:59:52.389 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5423000000000003e-05 2024-09-21T14:59:52.390 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:52.390 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:52.390 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:52.390 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.390 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.390 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9407 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:52.390 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.168094+0000", 2024-09-21T14:59:52.390 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 73.165573026000004, 2024-09-21T14:59:52.390 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019444860000000001, 2024-09-21T14:59:52.390 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:52.390 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:52.390 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:52.390 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:52.391 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:52.391 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9407 2024-09-21T14:59:52.391 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.391 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:52.391 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.391 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:52.391 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.168094+0000", 2024-09-21T14:59:52.391 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:52.391 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.391 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.391 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:52.391 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.168094+0000", 2024-09-21T14:59:52.391 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:52.391 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.392 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.392 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:52.392 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.168093+0000", 2024-09-21T14:59:52.392 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:59:52.392 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.392 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.392 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:52.392 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.168119+0000", 2024-09-21T14:59:52.392 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5803999999999999e-05 2024-09-21T14:59:52.392 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.392 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.392 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:52.392 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.168135+0000", 2024-09-21T14:59:52.392 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6359999999999999e-05 2024-09-21T14:59:52.393 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.393 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.393 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:52.393 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.168144+0000", 2024-09-21T14:59:52.393 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4270000000000008e-06 2024-09-21T14:59:52.393 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.393 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.393 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:52.393 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181943+0000", 2024-09-21T14:59:52.393 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013799617 2024-09-21T14:59:52.393 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.393 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.393 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:52.393 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.182052+0000", 2024-09-21T14:59:52.394 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000108924 2024-09-21T14:59:52.394 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.394 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.394 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:59:52.394 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.183431+0000", 2024-09-21T14:59:52.394 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0013783619999999999 2024-09-21T14:59:52.394 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.394 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.394 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:52.394 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.187276+0000", 2024-09-21T14:59:52.394 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0038450469999999999 2024-09-21T14:59:52.394 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.394 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.394 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:52.395 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.187288+0000", 2024-09-21T14:59:52.395 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2414e-05 2024-09-21T14:59:52.395 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.395 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.395 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:52.395 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.187414+0000", 2024-09-21T14:59:52.395 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012614399999999999 2024-09-21T14:59:52.395 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.395 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.395 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:52.395 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.187539+0000", 2024-09-21T14:59:52.395 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012481800000000001 2024-09-21T14:59:52.395 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:52.395 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:52.396 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:52.396 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.396 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.396 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9410 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:52.396 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.174103+0000", 2024-09-21T14:59:52.396 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 73.159564117000002, 2024-09-21T14:59:52.396 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025493703, 2024-09-21T14:59:52.396 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:52.396 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:52.396 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:52.396 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:52.396 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:52.396 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9410 2024-09-21T14:59:52.397 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.397 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:52.397 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.397 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:52.397 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.174103+0000", 2024-09-21T14:59:52.397 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:52.397 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.397 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.397 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:52.397 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.174103+0000", 2024-09-21T14:59:52.397 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:52.397 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.397 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.397 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:52.398 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.174102+0000", 2024-09-21T14:59:52.398 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:59:52.398 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.398 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.398 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:52.398 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.174108+0000", 2024-09-21T14:59:52.398 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1149999999999999e-06 2024-09-21T14:59:52.398 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.398 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.398 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:52.398 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.174117+0000", 2024-09-21T14:59:52.398 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4870000000000002e-06 2024-09-21T14:59:52.398 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.398 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.399 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:52.399 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.174122+0000", 2024-09-21T14:59:52.399 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9790000000000001e-06 2024-09-21T14:59:52.399 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.399 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.399 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:52.399 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.187830+0000", 2024-09-21T14:59:52.399 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013708048 2024-09-21T14:59:52.399 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.399 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.399 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:52.399 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.187913+0000", 2024-09-21T14:59:52.399 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2763999999999996e-05 2024-09-21T14:59:52.400 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.400 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.400 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:59:52.400 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.188230+0000", 2024-09-21T14:59:52.400 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00031745100000000001 2024-09-21T14:59:52.400 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.400 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.400 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:52.400 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.192708+0000", 2024-09-21T14:59:52.400 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0044783890000000002 2024-09-21T14:59:52.400 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.400 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.400 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:52.400 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199487+0000", 2024-09-21T14:59:52.400 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0067782149999999998 2024-09-21T14:59:52.401 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.401 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.401 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:52.401 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199560+0000", 2024-09-21T14:59:52.401 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3752999999999997e-05 2024-09-21T14:59:52.401 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.401 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.401 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:52.401 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199597+0000", 2024-09-21T14:59:52.401 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6526999999999998e-05 2024-09-21T14:59:52.401 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:52.401 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:52.401 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:52.402 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.402 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.402 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9411 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:52.402 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.175943+0000", 2024-09-21T14:59:52.402 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 73.157724098000003, 2024-09-21T14:59:52.402 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023773181000000001, 2024-09-21T14:59:52.402 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:52.402 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:52.402 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:52.402 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:52.402 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:52.402 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9411 2024-09-21T14:59:52.402 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.403 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:52.403 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.403 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:52.403 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175943+0000", 2024-09-21T14:59:52.403 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:52.403 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.403 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.403 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:52.403 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175943+0000", 2024-09-21T14:59:52.403 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:52.403 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.403 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.403 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:52.404 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175942+0000", 2024-09-21T14:59:52.404 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:59:52.404 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.404 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.404 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:52.404 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175948+0000", 2024-09-21T14:59:52.404 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9689999999999999e-06 2024-09-21T14:59:52.404 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.404 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.404 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:52.404 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175960+0000", 2024-09-21T14:59:52.404 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1745e-05 2024-09-21T14:59:52.405 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.405 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.405 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:52.405 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175965+0000", 2024-09-21T14:59:52.405 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.203e-06 2024-09-21T14:59:52.405 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.406 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.406 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:52.406 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.189900+0000", 2024-09-21T14:59:52.406 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013934940999999999 2024-09-21T14:59:52.406 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.406 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.406 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:52.406 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.189988+0000", 2024-09-21T14:59:52.406 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8419999999999997e-05 2024-09-21T14:59:52.406 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.406 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.407 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:59:52.407 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.190321+0000", 2024-09-21T14:59:52.407 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00033231800000000001 2024-09-21T14:59:52.407 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.407 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.407 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:52.407 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.192746+0000", 2024-09-21T14:59:52.407 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0024254400000000001 2024-09-21T14:59:52.407 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.407 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.407 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:52.407 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199605+0000", 2024-09-21T14:59:52.407 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0068582809999999999 2024-09-21T14:59:52.408 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.408 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.408 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:52.408 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199680+0000", 2024-09-21T14:59:52.408 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5630000000000006e-05 2024-09-21T14:59:52.408 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.408 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.408 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:52.408 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199716+0000", 2024-09-21T14:59:52.408 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6248999999999998e-05 2024-09-21T14:59:52.408 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:52.408 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:52.408 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:52.408 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.409 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.409 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9412 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:52.409 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.177402+0000", 2024-09-21T14:59:52.409 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 73.156265000999994, 2024-09-21T14:59:52.409 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022445590000000001, 2024-09-21T14:59:52.409 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:52.409 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:52.409 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:52.409 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:52.409 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:52.409 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9412 2024-09-21T14:59:52.409 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.409 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:52.409 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.410 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:52.410 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177402+0000", 2024-09-21T14:59:52.410 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:52.410 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.410 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.410 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:52.410 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177402+0000", 2024-09-21T14:59:52.410 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:52.410 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.410 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.410 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:52.410 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177401+0000", 2024-09-21T14:59:52.411 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:59:52.411 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.411 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.411 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:52.411 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177409+0000", 2024-09-21T14:59:52.411 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4710000000000001e-06 2024-09-21T14:59:52.411 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.411 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.411 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:52.411 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177419+0000", 2024-09-21T14:59:52.411 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0351e-05 2024-09-21T14:59:52.411 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.411 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.412 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:52.412 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177424+0000", 2024-09-21T14:59:52.412 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1919999999999996e-06 2024-09-21T14:59:52.412 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.412 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.412 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:52.412 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.191343+0000", 2024-09-21T14:59:52.412 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.01391878 2024-09-21T14:59:52.412 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.412 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.412 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:52.412 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.191446+0000", 2024-09-21T14:59:52.412 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000102453 2024-09-21T14:59:52.412 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.412 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.413 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:59:52.413 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.192200+0000", 2024-09-21T14:59:52.413 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00075476899999999997 2024-09-21T14:59:52.413 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.413 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.413 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:52.413 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.193866+0000", 2024-09-21T14:59:52.413 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0016657519999999999 2024-09-21T14:59:52.413 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.413 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.413 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:52.413 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199724+0000", 2024-09-21T14:59:52.413 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0058579529999999999 2024-09-21T14:59:52.413 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.413 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.414 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:52.414 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199803+0000", 2024-09-21T14:59:52.414 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8662e-05 2024-09-21T14:59:52.414 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.414 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.414 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:52.414 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199848+0000", 2024-09-21T14:59:52.414 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5216000000000002e-05 2024-09-21T14:59:52.414 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:52.414 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:52.414 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:52.414 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.414 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.414 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:10445 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:52.415 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.271980+0000", 2024-09-21T14:59:52.415 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 73.061687636000002, 2024-09-21T14:59:52.415 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018664256000000001, 2024-09-21T14:59:52.415 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:52.415 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:52.415 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:52.415 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:52.415 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:52.415 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 10445 2024-09-21T14:59:52.415 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.415 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:52.415 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.415 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:52.415 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.271980+0000", 2024-09-21T14:59:52.415 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:52.416 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.416 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.416 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:52.416 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.271980+0000", 2024-09-21T14:59:52.416 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:52.416 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.416 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.416 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:52.416 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.271979+0000", 2024-09-21T14:59:52.416 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:52.416 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.416 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.416 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:52.416 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.271982+0000", 2024-09-21T14:59:52.417 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7240000000000001e-06 2024-09-21T14:59:52.417 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.417 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.417 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:52.417 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.271987+0000", 2024-09-21T14:59:52.417 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3639999999999998e-06 2024-09-21T14:59:52.417 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.417 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.417 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:52.417 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.271992+0000", 2024-09-21T14:59:52.417 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7110000000000003e-06 2024-09-21T14:59:52.417 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.417 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.417 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:52.417 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.277567+0000", 2024-09-21T14:59:52.418 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0055745009999999999 2024-09-21T14:59:52.418 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.418 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.418 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:52.418 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.277655+0000", 2024-09-21T14:59:52.418 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8465000000000001e-05 2024-09-21T14:59:52.418 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.418 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.418 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:59:52.418 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.278720+0000", 2024-09-21T14:59:52.418 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0010650569999999999 2024-09-21T14:59:52.418 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.418 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.418 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:52.418 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.281153+0000", 2024-09-21T14:59:52.419 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0024326389999999999 2024-09-21T14:59:52.419 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.419 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.419 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:52.419 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.290574+0000", 2024-09-21T14:59:52.419 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0094209489999999996 2024-09-21T14:59:52.419 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.419 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.419 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:52.419 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.290613+0000", 2024-09-21T14:59:52.419 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.8803000000000001e-05 2024-09-21T14:59:52.419 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.419 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.419 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:52.420 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.290644+0000", 2024-09-21T14:59:52.420 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1404e-05 2024-09-21T14:59:52.420 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:52.420 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:52.420 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:52.420 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.420 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.420 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:23084 3.3 3:c55e343c:::rbd_data.11366acc894.0000000000000006:19 [read 1032192~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:52.420 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.150724+0000", 2024-09-21T14:59:52.420 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 62.182943061000003, 2024-09-21T14:59:52.420 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018916460999999999, 2024-09-21T14:59:52.420 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:52.420 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:52.421 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:52.421 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:52.421 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:52.421 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 23084 2024-09-21T14:59:52.421 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.421 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:52.421 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.421 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:52.421 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.150724+0000", 2024-09-21T14:59:52.421 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:52.421 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.421 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.421 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:52.422 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.150724+0000", 2024-09-21T14:59:52.422 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:52.422 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.422 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.422 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:52.422 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.150724+0000", 2024-09-21T14:59:52.422 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:52.422 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.422 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.422 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:52.422 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.150727+0000", 2024-09-21T14:59:52.422 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7250000000000002e-06 2024-09-21T14:59:52.422 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.422 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.423 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:52.423 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.150732+0000", 2024-09-21T14:59:52.423 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5809999999999996e-06 2024-09-21T14:59:52.423 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.423 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.423 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:52.423 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.150737+0000", 2024-09-21T14:59:52.423 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3590000000000001e-06 2024-09-21T14:59:52.423 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.423 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.423 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:52.423 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.166694+0000", 2024-09-21T14:59:52.423 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.015957720000000002 2024-09-21T14:59:52.423 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.423 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.424 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:52.424 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.166769+0000", 2024-09-21T14:59:52.424 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5143999999999997e-05 2024-09-21T14:59:52.424 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.424 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.424 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:52.424 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.169641+0000", 2024-09-21T14:59:52.424 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0028713089999999998 2024-09-21T14:59:52.424 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:52.424 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:52.424 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:52.424 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.424 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.424 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:71939 3.e 3:75b0de84:::rbd_object_map.11368900df38:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100)", 2024-09-21T14:59:52.425 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:38.376206+0000", 2024-09-21T14:59:52.425 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 13.957461832, 2024-09-21T14:59:52.425 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019353591999999999, 2024-09-21T14:59:52.425 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:52.425 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:52.425 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:52.425 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:52.425 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:52.425 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 71939 2024-09-21T14:59:52.425 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.425 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:52.425 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.425 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:52.425 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.376206+0000", 2024-09-21T14:59:52.425 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:52.425 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.426 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.426 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:52.426 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.376206+0000", 2024-09-21T14:59:52.426 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:52.426 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.426 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.426 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:52.426 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.376205+0000", 2024-09-21T14:59:52.426 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:52.426 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.426 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.426 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:52.426 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.376209+0000", 2024-09-21T14:59:52.427 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.8820000000000003e-06 2024-09-21T14:59:52.427 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.427 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.427 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:52.427 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.376215+0000", 2024-09-21T14:59:52.427 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5729999999999998e-06 2024-09-21T14:59:52.427 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.427 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.427 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:52.427 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.376221+0000", 2024-09-21T14:59:52.427 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2040000000000002e-06 2024-09-21T14:59:52.427 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.427 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.427 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:52.428 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.383959+0000", 2024-09-21T14:59:52.428 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0077380089999999997 2024-09-21T14:59:52.428 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.428 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.428 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:52.428 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.384131+0000", 2024-09-21T14:59:52.428 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017252899999999999 2024-09-21T14:59:52.428 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.428 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.428 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:59:52.428 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.384666+0000", 2024-09-21T14:59:52.428 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00053445399999999996 2024-09-21T14:59:52.428 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.428 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.429 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:52.429 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.388174+0000", 2024-09-21T14:59:52.429 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.003508639 2024-09-21T14:59:52.429 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.429 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.429 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:52.429 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.395460+0000", 2024-09-21T14:59:52.429 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.007285489 2024-09-21T14:59:52.429 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.429 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.429 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:52.429 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.395503+0000", 2024-09-21T14:59:52.429 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3080000000000001e-05 2024-09-21T14:59:52.429 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.429 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.430 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:52.430 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.395559+0000", 2024-09-21T14:59:52.430 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6118e-05 2024-09-21T14:59:52.430 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:52.430 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:52.430 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:52.430 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.430 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.430 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:71940 3.e 3:75b0de84:::rbd_object_map.11368900df38:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100)", 2024-09-21T14:59:52.430 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:38.376225+0000", 2024-09-21T14:59:52.430 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 13.957442594, 2024-09-21T14:59:52.430 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019578849999999998, 2024-09-21T14:59:52.430 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:52.430 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:52.430 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:52.430 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:52.431 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:52.431 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 71940 2024-09-21T14:59:52.431 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.431 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:52.431 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.431 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:52.431 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.376225+0000", 2024-09-21T14:59:52.431 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:52.431 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.431 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.431 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:52.431 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.376225+0000", 2024-09-21T14:59:52.431 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:52.431 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.431 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.432 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:52.432 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.376224+0000", 2024-09-21T14:59:52.432 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:52.432 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.432 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.432 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:52.432 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.376228+0000", 2024-09-21T14:59:52.432 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5889999999999999e-06 2024-09-21T14:59:52.432 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.432 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.432 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:52.432 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.376233+0000", 2024-09-21T14:59:52.432 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.322e-06 2024-09-21T14:59:52.432 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.432 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.432 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:52.433 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.376238+0000", 2024-09-21T14:59:52.433 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7960000000000003e-06 2024-09-21T14:59:52.433 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.433 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.433 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:52.433 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.385506+0000", 2024-09-21T14:59:52.433 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0092679279999999999 2024-09-21T14:59:52.433 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.433 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.433 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:52.433 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.385598+0000", 2024-09-21T14:59:52.433 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1939000000000002e-05 2024-09-21T14:59:52.433 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.433 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.433 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:59:52.433 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.386013+0000", 2024-09-21T14:59:52.434 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000414793 2024-09-21T14:59:52.434 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.434 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.434 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:52.434 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.395634+0000", 2024-09-21T14:59:52.434 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0096215720000000001 2024-09-21T14:59:52.434 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.434 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.434 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:52.434 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.395754+0000", 2024-09-21T14:59:52.434 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000120097 2024-09-21T14:59:52.434 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.434 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.434 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:52.435 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.395782+0000", 2024-09-21T14:59:52.435 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7583e-05 2024-09-21T14:59:52.435 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.435 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.435 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:52.435 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.395804+0000", 2024-09-21T14:59:52.435 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1594999999999999e-05 2024-09-21T14:59:52.435 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:52.435 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:52.435 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:52.435 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.435 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.435 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:71941 3.e 3:75b0de84:::rbd_object_map.11368900df38:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100)", 2024-09-21T14:59:52.435 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:38.376242+0000", 2024-09-21T14:59:52.435 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 13.957425541999999, 2024-09-21T14:59:52.436 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019640456000000001, 2024-09-21T14:59:52.436 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:52.436 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:52.436 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:52.436 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:52.436 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:52.436 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 71941 2024-09-21T14:59:52.436 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.436 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:52.436 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.436 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:52.436 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.376242+0000", 2024-09-21T14:59:52.436 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:52.436 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.437 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.437 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:52.437 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.376242+0000", 2024-09-21T14:59:52.437 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:52.437 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.437 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.437 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:52.437 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.376241+0000", 2024-09-21T14:59:52.437 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:52.437 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.437 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.437 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:52.437 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.376244+0000", 2024-09-21T14:59:52.437 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0340000000000001e-06 2024-09-21T14:59:52.438 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.438 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.438 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:52.438 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.376250+0000", 2024-09-21T14:59:52.438 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3499999999999996e-06 2024-09-21T14:59:52.438 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.438 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.438 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:52.438 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.376255+0000", 2024-09-21T14:59:52.438 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1320000000000002e-06 2024-09-21T14:59:52.438 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.438 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.438 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:52.438 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.386791+0000", 2024-09-21T14:59:52.439 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.010536085000000001 2024-09-21T14:59:52.439 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.439 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.439 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:52.439 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.386868+0000", 2024-09-21T14:59:52.439 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6674999999999993e-05 2024-09-21T14:59:52.439 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.439 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.439 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:59:52.439 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.387258+0000", 2024-09-21T14:59:52.439 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00038990999999999999 2024-09-21T14:59:52.439 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.439 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.439 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:52.439 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.395685+0000", 2024-09-21T14:59:52.440 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0084273590000000006 2024-09-21T14:59:52.440 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.440 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.440 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:52.440 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.395812+0000", 2024-09-21T14:59:52.440 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012719700000000001 2024-09-21T14:59:52.440 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.440 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.440 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:52.440 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.395849+0000", 2024-09-21T14:59:52.440 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6748999999999997e-05 2024-09-21T14:59:52.440 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:52.440 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:52.440 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:52.441 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.395882+0000", 2024-09-21T14:59:52.441 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3318999999999997e-05 2024-09-21T14:59:52.441 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:52.441 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:52.441 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:52.441 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:52.441 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:52.441 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:52.441 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T14:59:52.443 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T14:59:52.443 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:59:52.441+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:59:52.544 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:59:52.539+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:59:52.578 INFO:teuthology.orchestra.run.smithi032.stdout:2313 trunc from 0xd06a0f2 to 0x2ecab56 2024-09-21T14:59:52.593 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:52.594 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:59:52.594 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T14:59:52.594 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:52.605 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T14:59:52.616 INFO:teuthology.orchestra.run.smithi032.stdout:2315 trunc from 0x2ecab56 to 0x7b07181 2024-09-21T14:59:52.630 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T14:59:52.644 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:59:52.645+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:59:52.645 INFO:teuthology.orchestra.run.smithi032.stdout:2316 write 0xe46174c thru 0xe46ee6e (0xd723 bytes) 2024-09-21T14:59:52.651 INFO:teuthology.orchestra.run.smithi032.stdout:2317 read 0x52f4980 thru 0x53043da (0xfa5b bytes) 2024-09-21T14:59:52.651 INFO:teuthology.orchestra.run.smithi032.stdout:2318 write 0x347e200 thru 0x348e0dc (0xfedd bytes) 2024-09-21T14:59:52.652 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T14:59:52.672 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T14:59:52.682 INFO:teuthology.orchestra.run.smithi032.stdout:2320 trunc from 0xe46ee6f to 0xb9cadd 2024-09-21T14:59:52.696 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T14:59:52.720 INFO:teuthology.orchestra.run.smithi032.stdout:2321 trunc from 0xb9cadd to 0xb69009a 2024-09-21T14:59:52.720 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-21T14:59:52.722 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:52.722 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:52.734 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T14:59:52.738 INFO:teuthology.orchestra.run.smithi032.stdout:2325 writesame 0xbc099b8 thru 0xbc0e029 data_size 0x47(0x4672 bytes) 2024-09-21T14:59:52.738 INFO:teuthology.orchestra.run.smithi032.stdout:2326 write 0x5da6da0 thru 0x5da7578 (0x7d9 bytes) 2024-09-21T14:59:52.745 INFO:teuthology.orchestra.run.smithi032.stdout:2327 read 0x8a723fa thru 0x8a73070 (0xc77 bytes) 2024-09-21T14:59:52.745 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:59:52.739+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:59:52.751 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T14:59:52.754 INFO:teuthology.orchestra.run.smithi032.stdout:2328 writesame 0xbf6b57f thru 0xbf79ecf data_size 0x1(0xe951 bytes) 2024-09-21T14:59:52.754 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:52.754 INFO:teuthology.orchestra.run.smithi032.stdout:2330 trunc from 0xbf79ed0 to 0xbdd5957 2024-09-21T14:59:52.775 INFO:teuthology.orchestra.run.smithi032.stdout:2333 writesame 0xde9256e thru 0xde94aee data_size 0x1(0x2581 bytes) 2024-09-21T14:59:52.775 INFO:teuthology.orchestra.run.smithi032.stdout:2334 read 0x5357f69 thru 0x535c200 (0x4298 bytes) 2024-09-21T14:59:52.775 INFO:teuthology.orchestra.run.smithi032.stdout:2335 write 0x65521dd thru 0x655ef16 (0xcd3a bytes) 2024-09-21T14:59:52.778 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T14:59:52.782 INFO:teuthology.orchestra.run.smithi032.stdout:2338 write 0xcb5e530 thru 0xcb5ea33 (0x504 bytes) 2024-09-21T14:59:52.802 INFO:teuthology.orchestra.run.smithi032.stdout:2339 trunc from 0xde94aef to 0x71bec50 2024-09-21T14:59:52.802 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T14:59:52.804 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:59:52.804 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:59:52.804 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:59:52.804 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:59:52.804 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:59:52.804 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:59:52.804 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:59:52.804 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:59:52.822 INFO:teuthology.orchestra.run.smithi032.stdout:2340 punch from 0x234cbac to 0x234d27b, (0x6cf bytes) 2024-09-21T14:59:52.845 INFO:teuthology.orchestra.run.smithi032.stdout:2341 writesame 0xe8166a8 thru 0xe81cc04 data_size 0xb(0x655d bytes) 2024-09-21T14:59:52.845 INFO:teuthology.orchestra.run.smithi032.stdout:2344 write 0x46f02ae thru 0x46fc454 (0xc1a7 bytes) 2024-09-21T14:59:52.846 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:59:52.845+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:59:52.852 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:52.852 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:52.853 INFO:teuthology.orchestra.run.smithi032.stdout:2345 read 0x6038e9 thru 0x607470 (0x3b88 bytes) 2024-09-21T14:59:52.855 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:52.855 INFO:teuthology.orchestra.run.smithi032.stdout:2348 write 0x117db3c thru 0x117f879 (0x1d3e bytes) 2024-09-21T14:59:52.858 INFO:teuthology.orchestra.run.smithi032.stdout:2349 writesame 0x8f92679 thru 0x8f96a9f data_size 0x1(0x4427 bytes) 2024-09-21T14:59:52.863 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T14:59:52.874 INFO:teuthology.orchestra.run.smithi032.stdout:2351 read 0xaa9fdc8 thru 0xaaa4d2d (0x4f66 bytes) 2024-09-21T14:59:52.874 INFO:teuthology.orchestra.run.smithi032.stdout:2353 read 0xabe6c92 thru 0xabf35d4 (0xc943 bytes) 2024-09-21T14:59:52.874 INFO:teuthology.orchestra.run.smithi032.stdout:2354 write 0x8c10b30 thru 0x8c1ac76 (0xa147 bytes) 2024-09-21T14:59:52.882 INFO:teuthology.orchestra.run.smithi032.stdout:2355 punch from 0x574fb5f to 0x575d89b, (0xdd3c bytes) 2024-09-21T14:59:52.882 INFO:teuthology.orchestra.run.smithi032.stdout:2356 writesame 0x49e6f37 thru 0x49ed8ad data_size 0x31(0x6977 bytes) 2024-09-21T14:59:52.888 INFO:teuthology.orchestra.run.smithi032.stdout:2359 read 0x923a8c0 thru 0x9242086 (0x77c7 bytes) 2024-09-21T14:59:52.896 INFO:teuthology.orchestra.run.smithi032.stdout:2360 trunc from 0xe81cc05 to 0x9a3fbf8 2024-09-21T14:59:52.906 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:52.906 INFO:teuthology.orchestra.run.smithi032.stdout:2364 write 0x42d8cc0 thru 0x42dd4c5 (0x4806 bytes) 2024-09-21T14:59:52.914 INFO:teuthology.orchestra.run.smithi032.stdout:2365 writesame 0x407e029 thru 0x4085f5e data_size 0x13(0x7f36 bytes) 2024-09-21T14:59:52.923 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:52.930 INFO:teuthology.orchestra.run.smithi032.stdout:2367 trunc from 0x9a3fbf8 to 0xded68ba 2024-09-21T14:59:52.946 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:59:52.945+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:59:52.952 INFO:teuthology.orchestra.run.smithi032.stdout:2368 write 0xe83392b thru 0xe842f24 (0xf5fa bytes) 2024-09-21T14:59:52.967 INFO:teuthology.orchestra.run.smithi032.stdout:2371 trunc from 0xe842f25 to 0xa2f31cf 2024-09-21T14:59:52.989 INFO:teuthology.orchestra.run.smithi032.stdout:2372 read 0xa03d42d thru 0xa040da6 (0x397a bytes) 2024-09-21T14:59:52.989 INFO:teuthology.orchestra.run.smithi032.stdout:2373 read 0x4f58724 thru 0x4f5fe95 (0x7772 bytes) 2024-09-21T14:59:52.990 INFO:teuthology.orchestra.run.smithi032.stdout:2374 read 0x80994ec thru 0x80a6e2e (0xd943 bytes) 2024-09-21T14:59:52.990 INFO:teuthology.orchestra.run.smithi032.stdout:2375 writesame 0x5ef42c7 thru 0x5f007f8 data_size 0x2b(0xc532 bytes) 2024-09-21T14:59:52.995 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:52.995 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:53.007 INFO:teuthology.orchestra.run.smithi032.stdout:2376 trunc from 0xa2f31cf to 0xec5c905 2024-09-21T14:59:53.008 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T14:59:53.016 INFO:teuthology.orchestra.run.smithi032.stdout:2379 read 0xbbb900a thru 0xbbbf2f1 (0x62e8 bytes) 2024-09-21T14:59:53.017 INFO:teuthology.orchestra.run.smithi032.stdout:2380 read 0x716eaab thru 0x7170572 (0x1ac8 bytes) 2024-09-21T14:59:53.017 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:53.017 INFO:teuthology.orchestra.run.smithi032.stdout:2382 punch from 0x3980bca to 0x398e34c, (0xd782 bytes) 2024-09-21T14:59:53.021 INFO:teuthology.orchestra.run.smithi032.stdout:2383 trunc from 0xec5c905 to 0xb9da000 2024-09-21T14:59:53.031 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:53.031 INFO:teuthology.orchestra.run.smithi032.stdout:2386 write 0x4b4dd7a thru 0x4b5036a (0x25f1 bytes) 2024-09-21T14:59:53.039 INFO:teuthology.orchestra.run.smithi032.stdout:2387 writesame 0xa7d99b2 thru 0xa7e4b73 data_size 0x3d(0xb1c2 bytes) 2024-09-21T14:59:53.047 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:53.047 INFO:teuthology.orchestra.run.smithi032.stdout:2389 writesame 0x1e7b6cd thru 0x1e88dfe data_size 0x23(0xd732 bytes) 2024-09-21T14:59:53.048 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:59:53.043+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:59:53.058 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:53.058 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:53.058 INFO:teuthology.orchestra.run.smithi032.stdout:2392 punch from 0x40c05dd to 0x40c3a71, (0x3494 bytes) 2024-09-21T14:59:53.058 INFO:teuthology.orchestra.run.smithi032.stdout:2393 writesame 0xa645da0 thru 0xa655289 data_size 0x3a(0xf4ea bytes) 2024-09-21T14:59:53.081 INFO:teuthology.orchestra.run.smithi032.stdout:2394 trunc from 0xb9da000 to 0x2e0a147 2024-09-21T14:59:53.113 INFO:teuthology.orchestra.run.smithi032.stdout:2396 punch from 0xefa833 to 0xf00fb7, (0x6784 bytes) 2024-09-21T14:59:53.113 INFO:teuthology.orchestra.run.smithi032.stdout:2397 read 0x1d45671 thru 0x1d54f4d (0xf8dd bytes) 2024-09-21T14:59:53.125 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:53.125 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:53.127 INFO:teuthology.orchestra.run.smithi032.stdout:2398 write 0x5adc772 thru 0x5aec721 (0xffb0 bytes) 2024-09-21T14:59:53.135 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T14:59:53.136 INFO:teuthology.orchestra.run.smithi032.stdout:2402 writesame 0x1d68566 thru 0x1d76bdf data_size 0x1(0xe67a bytes) 2024-09-21T14:59:53.180 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:59:53.181+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:59:53.185 INFO:teuthology.orchestra.run.smithi032.stdout:2403 punch from 0x3ef3cf1 to 0x3effd8c, (0xc09b bytes) 2024-09-21T14:59:53.206 INFO:teuthology.orchestra.run.smithi032.stdout:2405 write 0xaaaf6d6 thru 0xaab3eaf (0x47da bytes) 2024-09-21T14:59:53.213 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:53.214 INFO:teuthology.orchestra.run.smithi032.stdout:2407 trunc from 0xaab3eb0 to 0xbadfc00 2024-09-21T14:59:53.238 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:53.238 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:53.239 INFO:teuthology.orchestra.run.smithi032.stdout:2408 write 0xe8b339d thru 0xe8bca6b (0x96cf bytes) 2024-09-21T14:59:53.248 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T14:59:53.250 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:59:53.249+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:59:53.252 INFO:teuthology.orchestra.run.smithi032.stdout:2409 trunc from 0xe8bca6c to 0xab4f2d5 2024-09-21T14:59:53.272 INFO:teuthology.orchestra.run.smithi032.stdout:2410 write 0x5bb7006 thru 0x5bc6f1b (0xff16 bytes) 2024-09-21T14:59:53.280 INFO:teuthology.orchestra.run.smithi032.stdout:2412 trunc from 0xab4f2d5 to 0xa929a73 2024-09-21T14:59:53.295 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:53.295 INFO:teuthology.orchestra.run.smithi032.stdout:2415 read 0x15d4efe thru 0x15d8352 (0x3455 bytes) 2024-09-21T14:59:53.312 INFO:teuthology.orchestra.run.smithi032.stdout:2416 write 0xd43d912 thru 0xd4403b7 (0x2aa6 bytes) 2024-09-21T14:59:53.319 INFO:teuthology.orchestra.run.smithi032.stdout:2417 read 0xa3dee1c thru 0xa3eaa35 (0xbc1a bytes) 2024-09-21T14:59:53.319 INFO:teuthology.orchestra.run.smithi032.stdout:2418 read 0xd3bf4ab thru 0xd3c475f (0x52b5 bytes) 2024-09-21T14:59:53.319 INFO:teuthology.orchestra.run.smithi032.stdout:2419 write 0x2ac7936 thru 0x2ad0ca0 (0x936b bytes) 2024-09-21T14:59:53.328 INFO:teuthology.orchestra.run.smithi032.stdout:2421 read 0x411b7c4 thru 0x412618f (0xa9cc bytes) 2024-09-21T14:59:53.328 INFO:teuthology.orchestra.run.smithi032.stdout:2422 writesame 0x2be81c4 thru 0x2beddab data_size 0x44(0x5be8 bytes) 2024-09-21T14:59:53.336 INFO:teuthology.orchestra.run.smithi032.stdout:2423 punch from 0x888d38a to 0x8893690, (0x6306 bytes) 2024-09-21T14:59:53.350 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:59:53.347+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:59:53.352 INFO:teuthology.orchestra.run.smithi032.stdout:2424 trunc from 0xd4403b8 to 0x3b1738e 2024-09-21T14:59:53.374 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:53.374 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:53.375 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:53.375 INFO:teuthology.orchestra.run.smithi032.stdout:2429 read 0x2547cd4 thru 0x254f3cb (0x76f8 bytes) 2024-09-21T14:59:53.375 INFO:teuthology.orchestra.run.smithi032.stdout:2430 punch from 0x4a2652 to 0x4a3695, (0x1043 bytes) 2024-09-21T14:59:53.375 INFO:teuthology.orchestra.run.smithi032.stdout:2431 write 0x21eade1 thru 0x21ee91a (0x3b3a bytes) 2024-09-21T14:59:53.383 INFO:teuthology.orchestra.run.smithi032.stdout:2432 read 0x209ae57 thru 0x20a6658 (0xb802 bytes) 2024-09-21T14:59:53.383 INFO:teuthology.orchestra.run.smithi032.stdout:2433 read 0x32cc7e7 thru 0x32d8167 (0xb981 bytes) 2024-09-21T14:59:53.384 INFO:teuthology.orchestra.run.smithi032.stdout:2435 punch from 0x2ccae8b to 0x2cd2b89, (0x7cfe bytes) 2024-09-21T14:59:53.384 INFO:teuthology.orchestra.run.smithi032.stdout:2437 punch from 0x21f1532 to 0x21f1c2d, (0x6fb bytes) 2024-09-21T14:59:53.385 INFO:teuthology.orchestra.run.smithi032.stderr:noscrub is set 2024-09-21T14:59:53.388 INFO:teuthology.orchestra.run.smithi032.stdout:2438 trunc from 0x3b1738e to 0x6e8918e 2024-09-21T14:59:53.391 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T14:59:53.398 INFO:teuthology.orchestra.run.smithi032.stdout:2439 writesame 0x1107edf thru 0x1110848 data_size 0x1a(0x896a bytes) 2024-09-21T14:59:53.422 INFO:teuthology.orchestra.run.smithi032.stdout:2440 writesame 0xcf12a5f thru 0xcf1352e data_size 0x1(0xad0 bytes) 2024-09-21T14:59:53.422 INFO:teuthology.orchestra.run.smithi032.stdout:2441 punch from 0xbabc93a to 0xbac7c68, (0xb32e bytes) 2024-09-21T14:59:53.422 INFO:teuthology.orchestra.run.smithi032.stdout:2442 punch from 0xca6e938 to 0xca74d60, (0x6428 bytes) 2024-09-21T14:59:53.422 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:53.426 INFO:teuthology.orchestra.run.smithi032.stdout:2445 trunc from 0xcf1352f to 0x9e6f9c4 2024-09-21T14:59:53.437 INFO:teuthology.orchestra.run.smithi032.stdout:2446 writesame 0x54865ee thru 0x5496445 data_size 0xc(0xfe58 bytes) 2024-09-21T14:59:53.447 INFO:teuthology.orchestra.run.smithi032.stdout:2448 writesame 0x19aadac thru 0x19b0546 data_size 0x1(0x579b bytes) 2024-09-21T14:59:53.451 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:59:53.447+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:59:53.485 INFO:teuthology.orchestra.run.smithi032.stdout:2449 write 0xe5a3708 thru 0xe5a6249 (0x2b42 bytes) 2024-09-21T14:59:53.491 INFO:teuthology.orchestra.run.smithi032.stdout:2450 writesame 0x5c8dd6e thru 0x5c9ac98 data_size 0x1(0xcf2b bytes) 2024-09-21T14:59:53.498 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:53.498 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:53.508 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T14:59:53.527 INFO:teuthology.orchestra.run.smithi032.stdout:2451 read 0x5e59b77 thru 0x5e61b01 (0x7f8b bytes) 2024-09-21T14:59:53.527 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:53.527 INFO:teuthology.orchestra.run.smithi032.stdout:2453 write 0x771f76d thru 0x77290df (0x9973 bytes) 2024-09-21T14:59:53.533 INFO:teuthology.orchestra.run.smithi032.stdout:2454 write 0x33da56a thru 0x33dada0 (0x837 bytes) 2024-09-21T14:59:53.540 INFO:teuthology.orchestra.run.smithi032.stdout:2455 write 0x9757716 thru 0x9764606 (0xcef1 bytes) 2024-09-21T14:59:53.550 INFO:teuthology.orchestra.run.smithi032.stdout:2456 writesame 0xa4bbbb7 thru 0xa4c0206 data_size 0x64(0x4650 bytes) 2024-09-21T14:59:53.552 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:59:53.553+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:59:53.576 INFO:teuthology.orchestra.run.smithi032.stdout:2457 trunc from 0xe5a624a to 0x1e36007 2024-09-21T14:59:53.618 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:53.619 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:53.629 INFO:teuthology.orchestra.run.smithi032.stdout:2458 writesame 0xe5b8e88 thru 0xe5c20af data_size 0xc(0x9228 bytes) 2024-09-21T14:59:53.629 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:53.629 INFO:teuthology.orchestra.run.smithi032.stdout:2460 writesame 0x11efdd2 thru 0x11f0044 data_size 0x3(0x273 bytes) 2024-09-21T14:59:53.630 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T14:59:53.633 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:53.633 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:53.633 INFO:teuthology.orchestra.run.smithi032.stdout:2463 read 0x314ba24 thru 0x315ac29 (0xf206 bytes) 2024-09-21T14:59:53.633 INFO:teuthology.orchestra.run.smithi032.stdout:2464 punch from 0x1c91091 to 0x1c92ffa, (0x1f69 bytes) 2024-09-21T14:59:53.633 INFO:teuthology.orchestra.run.smithi032.stdout:2465 writesame 0x38dabb6 thru 0x38e5249 data_size 0xe(0xa694 bytes) 2024-09-21T14:59:53.641 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:53.652 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:59:53.653+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:59:53.664 INFO:teuthology.orchestra.run.smithi032.stdout:2467 trunc from 0xe5c20b0 to 0x2125f24 2024-09-21T14:59:53.704 INFO:teuthology.orchestra.run.smithi032.stdout:2468 write 0x3c4241a thru 0x3c48c4e (0x6835 bytes) 2024-09-21T14:59:53.730 INFO:teuthology.orchestra.run.smithi032.stdout:2470 write 0x6ce7d79 thru 0x6cea5bb (0x2843 bytes) 2024-09-21T14:59:53.752 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:53.752 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:53.752 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:59:53.753+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:59:53.757 INFO:teuthology.orchestra.run.smithi032.stdout:2471 write 0x8c2012a thru 0x8c2110d (0xfe4 bytes) 2024-09-21T14:59:53.762 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T14:59:53.763 INFO:teuthology.orchestra.run.smithi032.stdout:2472 punch from 0x7e65729 to 0x7e669ca, (0x12a1 bytes) 2024-09-21T14:59:53.783 INFO:teuthology.orchestra.run.smithi032.stdout:2473 write 0xc876a7f thru 0xc8858ff (0xee81 bytes) 2024-09-21T14:59:53.792 INFO:teuthology.orchestra.run.smithi032.stdout:2474 write 0x56fde20 thru 0x56ff0bf (0x12a0 bytes) 2024-09-21T14:59:53.799 INFO:teuthology.orchestra.run.smithi032.stdout:2475 punch from 0x12cbf52 to 0x12da8c0, (0xe96e bytes) 2024-09-21T14:59:53.799 INFO:teuthology.orchestra.run.smithi032.stdout:2476 read 0x17063d3 thru 0x170efca (0x8bf8 bytes) 2024-09-21T14:59:53.799 INFO:teuthology.orchestra.run.smithi032.stdout:2477 write 0x1980c60 thru 0x1988b78 (0x7f19 bytes) 2024-09-21T14:59:53.803 INFO:teuthology.orchestra.run.smithi032.stdout:2479 read 0x44f6907 thru 0x4505362 (0xea5c bytes) 2024-09-21T14:59:53.806 INFO:teuthology.orchestra.run.smithi032.stdout:2480 trunc from 0xc885900 to 0xda09346 2024-09-21T14:59:53.843 INFO:teuthology.orchestra.run.smithi032.stdout:2481 trunc from 0xda09346 to 0x2111b26 2024-09-21T14:59:53.854 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:59:53.851+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:59:53.868 INFO:teuthology.orchestra.run.smithi032.stdout:2484 punch from 0x99d17 to 0xa516c, (0xb455 bytes) 2024-09-21T14:59:53.871 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:53.871 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:53.881 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T14:59:53.891 INFO:teuthology.orchestra.run.smithi032.stdout:2485 trunc from 0x2111b26 to 0xced1c01 2024-09-21T14:59:53.903 INFO:teuthology.orchestra.run.smithi032.stdout:2487 write 0xa7261f4 thru 0xa734a7e (0xe88b bytes) 2024-09-21T14:59:53.915 INFO:teuthology.orchestra.run.smithi032.stdout:2489 trunc from 0xced1c01 to 0xa4d82c8 2024-09-21T14:59:53.940 INFO:teuthology.orchestra.run.smithi032.stdout:2490 trunc from 0xa4d82c8 to 0x62b0c94 2024-09-21T14:59:53.955 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:59:53.951+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:59:53.958 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:53.958 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:53.985 INFO:teuthology.orchestra.run.smithi032.stdout:2494 write 0xe7fd607 thru 0xe80cff0 (0xf9ea bytes) 2024-09-21T14:59:53.986 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:53.986 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:53.994 INFO:teuthology.orchestra.run.smithi032.stdout:2495 write 0x86ce8bb thru 0x86dc08d (0xd7d3 bytes) 2024-09-21T14:59:53.997 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T14:59:54.018 INFO:teuthology.orchestra.run.smithi032.stdout:2496 trunc from 0xe80cff1 to 0x517f274 2024-09-21T14:59:54.040 INFO:teuthology.orchestra.run.smithi032.stdout:2497 trunc from 0x517f274 to 0x4af221e 2024-09-21T14:59:54.056 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:59:54.051+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:59:54.074 INFO:teuthology.orchestra.run.smithi032.stdout:2498 write 0x98ae763 thru 0x98b19be (0x325c bytes) 2024-09-21T14:59:54.082 INFO:teuthology.orchestra.run.smithi032.stdout:2499 writesame 0x4d2fba0 thru 0x4d35fa4 data_size 0xf(0x6405 bytes) 2024-09-21T14:59:54.110 INFO:teuthology.orchestra.run.smithi032.stdout:2500 write 0xca4e265 thru 0xca57c09 (0x99a5 bytes) 2024-09-21T14:59:54.118 INFO:teuthology.orchestra.run.smithi032.stdout:2502 punch from 0x446a512 to 0x4470b47, (0x6635 bytes) 2024-09-21T14:59:54.118 INFO:teuthology.orchestra.run.smithi032.stdout:2503 writesame 0x1183102 thru 0x118a727 data_size 0x47(0x7626 bytes) 2024-09-21T14:59:54.121 INFO:teuthology.orchestra.run.smithi032.stdout:2504 read 0xa3d2ff3 thru 0xa3dfbc6 (0xcbd4 bytes) 2024-09-21T14:59:54.140 INFO:teuthology.orchestra.run.smithi032.stdout:2505 writesame 0xe405d9c thru 0xe407550 data_size 0x15(0x17b5 bytes) 2024-09-21T14:59:54.140 INFO:teuthology.orchestra.run.smithi032.stdout:2506 read 0xbd6f40d thru 0xbd7edd3 (0xf9c7 bytes) 2024-09-21T14:59:54.140 INFO:teuthology.orchestra.run.smithi032.stdout:2507 write 0x17e98cb thru 0x17ed6ff (0x3e35 bytes) 2024-09-21T14:59:54.144 INFO:teuthology.orchestra.run.smithi032.stdout:2511 punch from 0xddead40 to 0xdded2c4, (0x2584 bytes) 2024-09-21T14:59:54.145 INFO:teuthology.orchestra.run.smithi032.stdout:2512 punch from 0xbf84d86 to 0xbf92be1, (0xde5b bytes) 2024-09-21T14:59:54.145 INFO:teuthology.orchestra.run.smithi032.stdout:2513 punch from 0x7f27127 to 0x7f2cabf, (0x5998 bytes) 2024-09-21T14:59:54.145 INFO:teuthology.orchestra.run.smithi032.stdout:2514 punch from 0xcb0b229 to 0xcb19af8, (0xe8cf bytes) 2024-09-21T14:59:54.145 INFO:teuthology.orchestra.run.smithi032.stdout:2515 write 0xd9618d6 thru 0xd967269 (0x5994 bytes) 2024-09-21T14:59:54.152 INFO:teuthology.orchestra.run.smithi032.stdout:2516 write 0x2c3d0d5 thru 0x2c3eef4 (0x1e20 bytes) 2024-09-21T14:59:54.157 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:59:54.157+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:59:54.160 INFO:teuthology.orchestra.run.smithi032.stdout:2517 write 0x219956f thru 0x21a14a1 (0x7f33 bytes) 2024-09-21T14:59:54.161 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:54.161 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:54.164 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:54.164 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:54.164 INFO:teuthology.orchestra.run.smithi032.stdout:2523 read 0x264a907 thru 0x265a248 (0xf942 bytes) 2024-09-21T14:59:54.165 INFO:teuthology.orchestra.run.smithi032.stdout:2524 read 0x2ea1740 thru 0x2eaca4a (0xb30b bytes) 2024-09-21T14:59:54.165 INFO:teuthology.orchestra.run.smithi032.stdout:2525 write 0x6d65058 thru 0x6d6c499 (0x7442 bytes) 2024-09-21T14:59:54.171 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T14:59:54.172 INFO:teuthology.orchestra.run.smithi032.stdout:2526 read 0xbf9267f thru 0xbf92df3 (0x775 bytes) 2024-09-21T14:59:54.173 INFO:teuthology.orchestra.run.smithi032.stdout:2527 writesame 0x65080ac thru 0x650da74 data_size 0x1(0x59c9 bytes) 2024-09-21T14:59:54.196 INFO:teuthology.orchestra.run.smithi032.stdout:2528 read 0x6604dbc thru 0x660739b (0x25e0 bytes) 2024-09-21T14:59:54.197 INFO:teuthology.orchestra.run.smithi032.stdout:2532 punch from 0x551057 to 0x55d97c, (0xc925 bytes) 2024-09-21T14:59:54.211 INFO:teuthology.orchestra.run.smithi032.stdout:2534 write 0xe61afdf thru 0xe6267e7 (0xb809 bytes) 2024-09-21T14:59:54.218 INFO:teuthology.orchestra.run.smithi032.stdout:2535 punch from 0x3c6d71d to 0x3c777ab, (0xa08e bytes) 2024-09-21T14:59:54.218 INFO:teuthology.orchestra.run.smithi032.stdout:2536 punch from 0xd70b3fc to 0xd717fad, (0xcbb1 bytes) 2024-09-21T14:59:54.218 INFO:teuthology.orchestra.run.smithi032.stdout:2537 read 0x8e87c56 thru 0x8e890b2 (0x145d bytes) 2024-09-21T14:59:54.218 INFO:teuthology.orchestra.run.smithi032.stdout:2538 read 0x98ed09 thru 0x9991e1 (0xa4d9 bytes) 2024-09-21T14:59:54.219 INFO:teuthology.orchestra.run.smithi032.stdout:2539 write 0x228eb4b thru 0x2291a8e (0x2f44 bytes) 2024-09-21T14:59:54.222 INFO:teuthology.orchestra.run.smithi032.stdout:2541 write 0x4b30512 thru 0x4b349ca (0x44b9 bytes) 2024-09-21T14:59:54.229 INFO:teuthology.orchestra.run.smithi032.stdout:2542 read 0x148f51 thru 0x14c0c0 (0x3170 bytes) 2024-09-21T14:59:54.255 INFO:teuthology.orchestra.run.smithi032.stdout:2544 trunc from 0xe6267e8 to 0x1291b1c 2024-09-21T14:59:54.258 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:59:54.255+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:59:54.281 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:54.285 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:54.285 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:54.293 INFO:teuthology.orchestra.run.smithi032.stdout:2546 trunc from 0x1291b1c to 0x7ce3115 2024-09-21T14:59:54.295 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T14:59:54.302 INFO:teuthology.orchestra.run.smithi032.stdout:2548 punch from 0x5a09e0d to 0x5a0f779, (0x596c bytes) 2024-09-21T14:59:54.302 INFO:teuthology.orchestra.run.smithi032.stdout:2549 punch from 0x52cf79a to 0x52d6647, (0x6ead bytes) 2024-09-21T14:59:54.302 INFO:teuthology.orchestra.run.smithi032.stdout:2550 read 0x3205fe1 thru 0x32141dd (0xe1fd bytes) 2024-09-21T14:59:54.302 INFO:teuthology.orchestra.run.smithi032.stdout:2551 read 0x2c1224a thru 0x2c20ed0 (0xec87 bytes) 2024-09-21T14:59:54.302 INFO:teuthology.orchestra.run.smithi032.stdout:2552 punch from 0x1d73fe9 to 0x1d766ec, (0x2703 bytes) 2024-09-21T14:59:54.303 INFO:teuthology.orchestra.run.smithi032.stdout:2553 read 0x32568bd thru 0x3259833 (0x2f77 bytes) 2024-09-21T14:59:54.303 INFO:teuthology.orchestra.run.smithi032.stdout:2554 writesame 0x367f804 thru 0x3689a62 data_size 0x1(0xa25f bytes) 2024-09-21T14:59:54.351 INFO:teuthology.orchestra.run.smithi032.stdout:2555 trunc from 0x7ce3115 to 0xe59ebf0 2024-09-21T14:59:54.358 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:59:54.357+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:59:54.361 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:54.370 INFO:teuthology.orchestra.run.smithi032.stdout:2558 trunc from 0xe59ebf0 to 0xafcf704 2024-09-21T14:59:54.383 INFO:teuthology.orchestra.run.smithi032.stdout:2561 punch from 0x9e05853 to 0x9e09378, (0x3b25 bytes) 2024-09-21T14:59:54.384 INFO:teuthology.orchestra.run.smithi032.stdout:2562 trunc from 0xafcf704 to 0xb97ea60 2024-09-21T14:59:54.396 INFO:teuthology.orchestra.run.smithi032.stdout:2563 punch from 0x81cd4cd to 0x81cf54a, (0x207d bytes) 2024-09-21T14:59:54.408 INFO:teuthology.orchestra.run.smithi032.stdout:2565 write 0xbb3a46e thru 0xbb42435 (0x7fc8 bytes) 2024-09-21T14:59:54.415 INFO:teuthology.orchestra.run.smithi032.stdout:2567 punch from 0x35e21d1 to 0x35e3247, (0x1076 bytes) 2024-09-21T14:59:54.424 INFO:teuthology.orchestra.run.smithi032.stdout:2568 trunc from 0xbb42436 to 0x7ead5db 2024-09-21T14:59:54.447 INFO:teuthology.orchestra.run.smithi032.stdout:2569 read 0x2386714 thru 0x2386ce8 (0x5d5 bytes) 2024-09-21T14:59:54.451 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:54.451 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:54.459 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:59:54.457+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:59:54.461 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T14:59:54.472 INFO:teuthology.orchestra.run.smithi032.stdout:2570 write 0xc822e0e thru 0xc82f375 (0xc568 bytes) 2024-09-21T14:59:54.479 INFO:teuthology.orchestra.run.smithi032.stdout:2571 read 0x4407fb thru 0x44ad8c (0xa592 bytes) 2024-09-21T14:59:54.480 INFO:teuthology.orchestra.run.smithi032.stdout:2572 writesame 0x8180050 thru 0x818293d data_size 0x3e(0x28ee bytes) 2024-09-21T14:59:54.487 INFO:teuthology.orchestra.run.smithi032.stdout:2573 read 0xc62450a thru 0xc626930 (0x2427 bytes) 2024-09-21T14:59:54.488 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:54.488 INFO:teuthology.orchestra.run.smithi032.stdout:2575 read 0x433b351 thru 0x434ad1b (0xf9cb bytes) 2024-09-21T14:59:54.488 INFO:teuthology.orchestra.run.smithi032.stdout:2577 writesame 0x78c521a thru 0x78c5963 data_size 0x3(0x74a bytes) 2024-09-21T14:59:54.494 INFO:teuthology.orchestra.run.smithi032.stdout:2578 writesame 0x404b44c thru 0x4052edd data_size 0x1d(0x7a92 bytes) 2024-09-21T14:59:54.501 INFO:teuthology.orchestra.run.smithi032.stdout:2579 read 0x2746f64 thru 0x274b279 (0x4316 bytes) 2024-09-21T14:59:54.502 INFO:teuthology.orchestra.run.smithi032.stdout:2580 write 0xa0d14df thru 0xa0da4ed (0x900f bytes) 2024-09-21T14:59:54.509 INFO:teuthology.orchestra.run.smithi032.stdout:2581 read 0x341d4f4 thru 0x34227f0 (0x52fd bytes) 2024-09-21T14:59:54.509 INFO:teuthology.orchestra.run.smithi032.stdout:2584 write 0x67bead thru 0x689d43 (0xde97 bytes) 2024-09-21T14:59:54.514 INFO:teuthology.orchestra.run.smithi032.stdout:2586 read 0x5ca26b1 thru 0x5caf71e (0xd06e bytes) 2024-09-21T14:59:54.514 INFO:teuthology.orchestra.run.smithi032.stdout:2587 read 0x62c66db thru 0x62c896d (0x2293 bytes) 2024-09-21T14:59:54.528 INFO:teuthology.orchestra.run.smithi032.stdout:2588 write 0xd47fc8a thru 0xd48c580 (0xc8f7 bytes) 2024-09-21T14:59:54.537 INFO:teuthology.orchestra.run.smithi032.stdout:2589 writesame 0x4668612 thru 0x4669d2b data_size 0x1(0x171a bytes) 2024-09-21T14:59:54.547 INFO:teuthology.orchestra.run.smithi032.stdout:2590 punch from 0x87651e1 to 0x876f579, (0xa398 bytes) 2024-09-21T14:59:54.547 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:54.547 INFO:teuthology.orchestra.run.smithi032.stdout:2596 read 0x3d61005 thru 0x3d67675 (0x6671 bytes) 2024-09-21T14:59:54.547 INFO:teuthology.orchestra.run.smithi032.stdout:2597 write 0x184a38 thru 0x1875c9 (0x2b92 bytes) 2024-09-21T14:59:54.550 INFO:teuthology.orchestra.run.smithi032.stdout:2598 write 0x89325a6 thru 0x893a8ad (0x8308 bytes) 2024-09-21T14:59:54.558 INFO:teuthology.orchestra.run.smithi032.stdout:2599 writesame 0x3dba146 thru 0x3dc0d52 data_size 0x1(0x6c0d bytes) 2024-09-21T14:59:54.560 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:59:54.561+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:59:54.572 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:54.572 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:54.582 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T14:59:54.583 INFO:teuthology.orchestra.run.smithi032.stdout:2600 read 0x374ac48 thru 0x3758b27 (0xdee0 bytes) 2024-09-21T14:59:54.583 INFO:teuthology.orchestra.run.smithi032.stdout:2601 read 0xf1f51b thru 0xf246c7 (0x51ad bytes) 2024-09-21T14:59:54.606 INFO:teuthology.orchestra.run.smithi032.stdout:2602 trunc from 0xd48c581 to 0x18834b1 2024-09-21T14:59:54.625 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 5, 6, 7, 4] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T14:59:54.626 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T14:59:54.626 INFO:tasks.thrashosds.thrasher:inject_pause on osd.4 2024-09-21T14:59:54.626 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-21T14:59:54.626 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-21T14:59:54.626 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok config set filestore_inject_stall 3 2024-09-21T14:59:54.627 INFO:teuthology.orchestra.run.smithi032.stdout:2603 read 0x15c8c64 thru 0x15cfcb1 (0x704e bytes) 2024-09-21T14:59:54.627 INFO:teuthology.orchestra.run.smithi032.stdout:2605 read 0x15f017c thru 0x15fda6f (0xd8f4 bytes) 2024-09-21T14:59:54.627 INFO:teuthology.orchestra.run.smithi032.stdout:2606 trunc from 0x18834b1 to 0x17040c2 2024-09-21T14:59:54.637 INFO:teuthology.orchestra.run.smithi032.stdout:2607 punch from 0x159735e to 0x15a0c16, (0x98b8 bytes) 2024-09-21T14:59:54.661 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:59:54.661+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:59:54.667 INFO:teuthology.orchestra.run.smithi032.stdout:2610 write 0xb55f55d thru 0xb56840d (0x8eb1 bytes) 2024-09-21T14:59:54.674 INFO:teuthology.orchestra.run.smithi032.stdout:2611 write 0x9ed6f08 thru 0x9ed9acf (0x2bc8 bytes) 2024-09-21T14:59:54.681 INFO:teuthology.orchestra.run.smithi032.stdout:2612 write 0x7fec8 thru 0x817c8 (0x1901 bytes) 2024-09-21T14:59:54.685 INFO:teuthology.orchestra.run.smithi032.stdout:2613 writesame 0x2cdc8e7 thru 0x2cec50e data_size 0x4(0xfc28 bytes) 2024-09-21T14:59:54.690 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:54.690 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:54.701 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T14:59:54.703 INFO:teuthology.orchestra.run.smithi032.stdout:2614 read 0x189e722 thru 0x18a8ea8 (0xa787 bytes) 2024-09-21T14:59:54.715 INFO:teuthology.orchestra.run.smithi032.stdout:2615 write 0xbb828c4 thru 0xbb8d835 (0xaf72 bytes) 2024-09-21T14:59:54.722 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:54.722 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:54.723 INFO:teuthology.orchestra.run.smithi032.stdout:2618 read 0x7c7ead2 thru 0x7c7fd42 (0x1271 bytes) 2024-09-21T14:59:54.723 INFO:teuthology.orchestra.run.smithi032.stdout:2619 writesame 0x1aa5461 thru 0x1ab3e88 data_size 0x7f(0xea28 bytes) 2024-09-21T14:59:54.729 INFO:teuthology.orchestra.run.smithi032.stdout:2620 write 0x5536fce thru 0x55412f4 (0xa327 bytes) 2024-09-21T14:59:54.734 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:59:54.734 INFO:teuthology.orchestra.run.smithi046.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-21T14:59:54.734 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:59:54.745 INFO:teuthology.orchestra.run.smithi032.stdout:2621 trunc from 0xbb8d836 to 0x7310e7e 2024-09-21T14:59:54.761 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:59:54.761+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:59:54.766 INFO:teuthology.orchestra.run.smithi032.stdout:2622 punch from 0x5adc19b to 0x5ae0138, (0x3f9d bytes) 2024-09-21T14:59:54.781 INFO:teuthology.orchestra.run.smithi032.stdout:2623 writesame 0x893a513 thru 0x89421a6 data_size 0x77(0x7c94 bytes) 2024-09-21T14:59:54.781 INFO:teuthology.orchestra.run.smithi032.stdout:2626 read 0x4514697 thru 0x45213fd (0xcd67 bytes) 2024-09-21T14:59:54.789 INFO:teuthology.orchestra.run.smithi032.stdout:2627 trunc from 0x89421a7 to 0xc7a832c 2024-09-21T14:59:54.800 INFO:teuthology.orchestra.run.smithi032.stdout:2628 read 0x3e6af0f thru 0x3e76714 (0xb806 bytes) 2024-09-21T14:59:54.800 INFO:teuthology.orchestra.run.smithi032.stdout:2629 write 0x4ce0e87 thru 0x4ced314 (0xc48e bytes) 2024-09-21T14:59:54.808 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:54.808 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:54.810 INFO:teuthology.orchestra.run.smithi032.stdout:2630 trunc from 0xc7a832c to 0xbcc293e 2024-09-21T14:59:54.816 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T14:59:54.818 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T14:59:54.829 INFO:teuthology.orchestra.run.smithi032.stdout:2631 trunc from 0xbcc293e to 0x9699b55 2024-09-21T14:59:54.840 INFO:teuthology.orchestra.run.smithi032.stdout:2632 read 0x5e8dc93 thru 0x5e9b178 (0xd4e6 bytes) 2024-09-21T14:59:54.840 INFO:teuthology.orchestra.run.smithi032.stdout:2634 punch from 0x359c553 to 0x35ab87d, (0xf32a bytes) 2024-09-21T14:59:54.841 INFO:teuthology.orchestra.run.smithi032.stdout:2635 read 0x292c49c thru 0x2937e5f (0xb9c4 bytes) 2024-09-21T14:59:54.854 INFO:teuthology.orchestra.run.smithi032.stdout:2636 trunc from 0x9699b55 to 0x1c90c98 2024-09-21T14:59:54.862 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:59:54.861+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:59:54.898 INFO:teuthology.orchestra.run.smithi032.stdout:2639 writesame 0x7676cd8 thru 0x7685bed data_size 0x65(0xef16 bytes) 2024-09-21T14:59:54.913 INFO:teuthology.orchestra.run.smithi032.stdout:2641 writesame 0x89d1293 thru 0x89e0c39 data_size 0x1(0xf9a7 bytes) 2024-09-21T14:59:54.913 INFO:teuthology.orchestra.run.smithi032.stdout:2642 read 0x774dfd3 thru 0x775b7b0 (0xd7de bytes) 2024-09-21T14:59:54.913 INFO:teuthology.orchestra.run.smithi032.stdout:2643 punch from 0x5c589cd to 0x5c685e3, (0xfc16 bytes) 2024-09-21T14:59:54.913 INFO:teuthology.orchestra.run.smithi032.stdout:2645 punch from 0x5720548 to 0x572abcd, (0xa685 bytes) 2024-09-21T14:59:54.920 INFO:teuthology.orchestra.run.smithi032.stdout:2647 trunc from 0x89e0c3a to 0x4484a01 2024-09-21T14:59:54.925 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:54.925 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:54.935 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T14:59:54.950 INFO:teuthology.orchestra.run.smithi032.stdout:2648 writesame 0xbe86018 thru 0xbe8d4c7 data_size 0x8(0x74b0 bytes) 2024-09-21T14:59:54.951 INFO:teuthology.orchestra.run.smithi032.stdout:2649 clone 22 order 19 su 131072 sc 6 2024-09-21T14:59:54.963 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:59:54.959+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:59:54.968 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T14:59:54.989 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T14:59:55.010 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T14:59:55.032 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T14:59:55.039 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:55.039 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:55.049 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T14:59:55.055 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-21T14:59:55.063 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:59:55.061+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:59:55.079 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T14:59:55.160 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:55.161 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:55.164 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:59:55.161+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:59:55.171 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T14:59:55.265 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:59:55.265+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:59:55.287 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:55.287 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:59:55.287 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T14:59:55.287 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:55.300 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T14:59:55.309 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T14:59:55.331 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T14:59:55.334 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:59:55.334 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:59:55.334 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:59:55.334 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:59:55.334 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:59:55.334 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:59:55.334 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:59:55.334 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:59:55.365 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:59:55.359+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:59:55.400 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T14:59:55.414 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:55.414 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:59:55.415 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T14:59:55.415 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T14:59:55.415 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:55.425 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T14:59:55.466 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:59:55.463+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:59:55.567 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:59:55.565+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:59:55.582 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:55.583 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T14:59:55.583 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T14:59:55.583 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T14:59:55.583 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.583 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:1473 3.15 3:af2afa6a:::rbd_data.1136515ecb6b.000000000000001b:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T14:59:55.583 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:30.421730+0000", 2024-09-21T14:59:55.583 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 85.160021839999999, 2024-09-21T14:59:55.583 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021478937, 2024-09-21T14:59:55.583 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:55.583 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:55.583 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:55.583 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:55.583 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:55.583 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1473 2024-09-21T14:59:55.584 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.584 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:55.584 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.584 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:55.584 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421730+0000", 2024-09-21T14:59:55.584 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:55.584 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.584 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.584 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:55.584 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421730+0000", 2024-09-21T14:59:55.584 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:55.584 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.584 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.584 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:55.584 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421728+0000", 2024-09-21T14:59:55.585 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-21T14:59:55.585 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.585 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.585 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:55.585 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421739+0000", 2024-09-21T14:59:55.585 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0502e-05 2024-09-21T14:59:55.585 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.585 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.585 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:55.585 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421762+0000", 2024-09-21T14:59:55.585 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3405999999999999e-05 2024-09-21T14:59:55.585 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.585 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.585 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:55.585 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421773+0000", 2024-09-21T14:59:55.585 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0744000000000001e-05 2024-09-21T14:59:55.586 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.586 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.586 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:55.586 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421838+0000", 2024-09-21T14:59:55.586 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5486000000000003e-05 2024-09-21T14:59:55.586 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.586 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.586 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:55.586 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.421955+0000", 2024-09-21T14:59:55.586 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000116765 2024-09-21T14:59:55.586 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.586 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.586 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 6", 2024-09-21T14:59:55.586 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.425572+0000", 2024-09-21T14:59:55.586 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0036171380000000002 2024-09-21T14:59:55.587 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.587 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.587 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:55.587 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.442973+0000", 2024-09-21T14:59:55.587 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.017400268 2024-09-21T14:59:55.587 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.587 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.587 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:55.587 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.442996+0000", 2024-09-21T14:59:55.587 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3139e-05 2024-09-21T14:59:55.587 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.587 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.587 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:55.587 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.443079+0000", 2024-09-21T14:59:55.587 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2832999999999998e-05 2024-09-21T14:59:55.588 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.588 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.588 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:55.588 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.443209+0000", 2024-09-21T14:59:55.588 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013052499999999999 2024-09-21T14:59:55.588 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:55.588 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:55.588 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:55.588 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.588 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.588 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:1678 3.c 3:377cc7e1:::rbd_data.1136515ecb6b.000000000000000f:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T14:59:55.588 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:30.988085+0000", 2024-09-21T14:59:55.588 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 84.593667245999995, 2024-09-21T14:59:55.588 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.032974802999999997, 2024-09-21T14:59:55.588 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:55.588 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:55.589 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:55.589 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:55.589 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:55.589 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1678 2024-09-21T14:59:55.589 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.589 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:55.589 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.589 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:55.589 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988085+0000", 2024-09-21T14:59:55.589 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:55.589 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.589 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.589 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:55.589 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988085+0000", 2024-09-21T14:59:55.589 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:55.590 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.590 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.590 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:55.590 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988083+0000", 2024-09-21T14:59:55.590 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-21T14:59:55.590 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.590 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.590 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:55.590 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988093+0000", 2024-09-21T14:59:55.590 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8800000000000003e-06 2024-09-21T14:59:55.590 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.590 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.590 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:55.590 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988113+0000", 2024-09-21T14:59:55.590 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0217999999999999e-05 2024-09-21T14:59:55.591 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.591 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.591 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:55.591 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988123+0000", 2024-09-21T14:59:55.591 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5859999999999994e-06 2024-09-21T14:59:55.591 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.591 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.591 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:55.591 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988203+0000", 2024-09-21T14:59:55.591 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0834000000000006e-05 2024-09-21T14:59:55.591 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.591 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.591 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:55.591 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988376+0000", 2024-09-21T14:59:55.592 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017301299999999999 2024-09-21T14:59:55.592 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.592 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.592 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 7", 2024-09-21T14:59:55.592 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.996338+0000", 2024-09-21T14:59:55.592 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0079618520000000002 2024-09-21T14:59:55.592 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.592 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.592 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:55.592 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.020881+0000", 2024-09-21T14:59:55.592 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024542731000000002 2024-09-21T14:59:55.592 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.592 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.592 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:55.592 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.020904+0000", 2024-09-21T14:59:55.592 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2861e-05 2024-09-21T14:59:55.593 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.593 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.593 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:55.593 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.020987+0000", 2024-09-21T14:59:55.593 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2806000000000006e-05 2024-09-21T14:59:55.593 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.593 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.593 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:55.593 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.021060+0000", 2024-09-21T14:59:55.593 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2871000000000001e-05 2024-09-21T14:59:55.593 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:55.593 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:55.593 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:55.593 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.593 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.593 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9370 3.3 3:c725132c:::rbd_data.11363a21d2be.0000000000000007:10 [read 163840~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:55.594 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157279+0000", 2024-09-21T14:59:55.594 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 76.424472621999996, 2024-09-21T14:59:55.594 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018472379000000001, 2024-09-21T14:59:55.594 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:55.594 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:55.594 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:55.594 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:55.594 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:55.594 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9370 2024-09-21T14:59:55.594 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.594 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:55.594 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.594 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:55.594 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157279+0000", 2024-09-21T14:59:55.595 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:55.595 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.595 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.595 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:55.595 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157279+0000", 2024-09-21T14:59:55.595 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:55.595 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.595 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.595 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:55.595 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157279+0000", 2024-09-21T14:59:55.595 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:55.595 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.595 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.596 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:55.596 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157282+0000", 2024-09-21T14:59:55.596 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2229999999999999e-06 2024-09-21T14:59:55.596 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.596 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.596 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:55.596 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157294+0000", 2024-09-21T14:59:55.596 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2173e-05 2024-09-21T14:59:55.596 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.596 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.596 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:55.596 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157300+0000", 2024-09-21T14:59:55.596 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.93e-06 2024-09-21T14:59:55.596 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.597 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.597 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:55.597 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175569+0000", 2024-09-21T14:59:55.597 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018268376999999999 2024-09-21T14:59:55.597 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.597 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.597 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:55.597 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175622+0000", 2024-09-21T14:59:55.597 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3808999999999999e-05 2024-09-21T14:59:55.597 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.597 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.597 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:55.597 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175752+0000", 2024-09-21T14:59:55.597 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012938700000000001 2024-09-21T14:59:55.598 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:55.598 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:55.598 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:55.598 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.598 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.598 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9371 3.3 3:c0bf6a9b:::rbd_data.11363a21d2be.0000000000000000:10 [read 73728~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:55.598 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157306+0000", 2024-09-21T14:59:55.598 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 76.424446510999999, 2024-09-21T14:59:55.598 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018755473000000002, 2024-09-21T14:59:55.598 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:55.598 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:55.598 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:55.598 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:55.598 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:55.599 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9371 2024-09-21T14:59:55.599 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.599 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:55.599 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.599 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:55.599 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157306+0000", 2024-09-21T14:59:55.599 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:55.599 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.599 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.599 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:55.599 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157306+0000", 2024-09-21T14:59:55.599 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:55.599 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.599 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.599 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:55.600 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157305+0000", 2024-09-21T14:59:55.600 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:55.600 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.600 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.600 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:55.600 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157312+0000", 2024-09-21T14:59:55.600 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3289999999999996e-06 2024-09-21T14:59:55.600 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.600 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.600 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:55.600 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157324+0000", 2024-09-21T14:59:55.600 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1725e-05 2024-09-21T14:59:55.600 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.600 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.600 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:55.601 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157339+0000", 2024-09-21T14:59:55.601 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5211999999999999e-05 2024-09-21T14:59:55.601 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.601 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.601 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:55.601 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175798+0000", 2024-09-21T14:59:55.601 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018458781 2024-09-21T14:59:55.601 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.601 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.601 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:55.601 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175899+0000", 2024-09-21T14:59:55.601 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000100784 2024-09-21T14:59:55.601 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.601 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.601 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:55.601 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176061+0000", 2024-09-21T14:59:55.602 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00016216099999999999 2024-09-21T14:59:55.602 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:55.602 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:55.602 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:55.602 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.602 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.602 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9372 3.3 3:c0bf6a9b:::rbd_data.11363a21d2be.0000000000000000:10 [read 212992~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:55.602 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157345+0000", 2024-09-21T14:59:55.602 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 76.424407165000005, 2024-09-21T14:59:55.602 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018926493999999999, 2024-09-21T14:59:55.602 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:55.602 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:55.602 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:55.602 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:55.602 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:55.603 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9372 2024-09-21T14:59:55.603 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.603 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:55.603 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.603 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:55.603 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157345+0000", 2024-09-21T14:59:55.603 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:55.603 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.603 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.603 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:55.603 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157345+0000", 2024-09-21T14:59:55.603 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:55.603 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.603 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.603 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:55.603 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157344+0000", 2024-09-21T14:59:55.604 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:55.604 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.604 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.604 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:55.604 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157348+0000", 2024-09-21T14:59:55.604 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1389999999999999e-06 2024-09-21T14:59:55.604 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.604 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.604 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:55.604 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157372+0000", 2024-09-21T14:59:55.604 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4711000000000001e-05 2024-09-21T14:59:55.604 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.604 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.604 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:55.604 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157378+0000", 2024-09-21T14:59:55.605 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9909999999999999e-06 2024-09-21T14:59:55.605 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.605 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.605 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:55.605 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176090+0000", 2024-09-21T14:59:55.605 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018712111999999999 2024-09-21T14:59:55.605 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.605 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.605 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:55.605 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176134+0000", 2024-09-21T14:59:55.605 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3377999999999997e-05 2024-09-21T14:59:55.605 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.605 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.605 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:55.605 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176271+0000", 2024-09-21T14:59:55.606 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013763499999999999 2024-09-21T14:59:55.606 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:55.606 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:55.606 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:55.606 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.606 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.606 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9373 3.3 3:c0bf6a9b:::rbd_data.11363a21d2be.0000000000000000:10 [read 352256~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:55.606 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157384+0000", 2024-09-21T14:59:55.606 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 76.424368392999995, 2024-09-21T14:59:55.606 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019154937, 2024-09-21T14:59:55.606 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:55.606 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:55.606 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:55.606 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:55.606 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:55.607 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9373 2024-09-21T14:59:55.607 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.607 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:55.607 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.607 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:55.607 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157384+0000", 2024-09-21T14:59:55.607 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:55.607 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.607 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.607 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:55.607 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157384+0000", 2024-09-21T14:59:55.607 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:55.607 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.608 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.608 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:55.608 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157383+0000", 2024-09-21T14:59:55.608 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:55.608 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.608 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.608 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:55.608 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157386+0000", 2024-09-21T14:59:55.608 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.191e-06 2024-09-21T14:59:55.608 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.608 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.608 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:55.608 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157393+0000", 2024-09-21T14:59:55.608 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5339999999999996e-06 2024-09-21T14:59:55.609 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.609 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.609 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:55.609 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157397+0000", 2024-09-21T14:59:55.609 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5220000000000001e-06 2024-09-21T14:59:55.609 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.609 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.609 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:55.609 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176302+0000", 2024-09-21T14:59:55.609 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018905113000000001 2024-09-21T14:59:55.609 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.609 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.609 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:55.609 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176379+0000", 2024-09-21T14:59:55.610 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6372999999999995e-05 2024-09-21T14:59:55.610 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.610 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.610 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:55.610 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176539+0000", 2024-09-21T14:59:55.610 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015973800000000001 2024-09-21T14:59:55.610 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:55.610 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:55.610 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:55.610 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.610 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.610 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9374 3.3 3:c0bf6a9b:::rbd_data.11363a21d2be.0000000000000000:10 [read 491520~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:55.610 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157401+0000", 2024-09-21T14:59:55.611 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 76.424350609000001, 2024-09-21T14:59:55.611 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019387755, 2024-09-21T14:59:55.611 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:55.611 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:55.611 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:55.611 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:55.611 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:55.611 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9374 2024-09-21T14:59:55.611 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.611 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:55.611 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.611 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:55.611 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157401+0000", 2024-09-21T14:59:55.611 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:55.611 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.612 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.612 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:55.612 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157401+0000", 2024-09-21T14:59:55.612 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:55.612 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.612 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.612 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:55.612 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157401+0000", 2024-09-21T14:59:55.612 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:55.612 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.612 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.612 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:55.612 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157414+0000", 2024-09-21T14:59:55.612 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2417e-05 2024-09-21T14:59:55.613 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.613 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.613 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:55.613 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157424+0000", 2024-09-21T14:59:55.613 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0861e-05 2024-09-21T14:59:55.613 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.613 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.613 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:55.613 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157429+0000", 2024-09-21T14:59:55.613 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0499999999999999e-06 2024-09-21T14:59:55.613 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.613 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.613 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:55.613 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176596+0000", 2024-09-21T14:59:55.614 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019166552 2024-09-21T14:59:55.614 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.614 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.614 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:55.614 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176647+0000", 2024-09-21T14:59:55.614 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1112000000000001e-05 2024-09-21T14:59:55.614 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.614 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.614 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:55.614 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176789+0000", 2024-09-21T14:59:55.614 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014209300000000001 2024-09-21T14:59:55.614 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:55.614 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:55.614 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:55.615 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.615 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.615 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9401 3.3 3:c725132c:::rbd_data.11363a21d2be.0000000000000007:10 [read 32768~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:55.615 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157621+0000", 2024-09-21T14:59:55.615 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 76.4241308, 2024-09-21T14:59:55.615 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019381711999999999, 2024-09-21T14:59:55.615 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:55.615 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:55.615 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:55.615 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:55.615 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:55.615 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9401 2024-09-21T14:59:55.615 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.615 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:55.616 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.616 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:55.616 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157621+0000", 2024-09-21T14:59:55.616 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:55.616 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.616 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.616 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:55.616 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157621+0000", 2024-09-21T14:59:55.616 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:55.616 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.616 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.616 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:55.616 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157620+0000", 2024-09-21T14:59:55.616 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:59:55.616 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.617 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.617 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:55.617 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157626+0000", 2024-09-21T14:59:55.617 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7570000000000001e-06 2024-09-21T14:59:55.617 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.617 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.617 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:55.617 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157636+0000", 2024-09-21T14:59:55.617 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8339999999999994e-06 2024-09-21T14:59:55.617 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.617 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.617 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:55.617 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157643+0000", 2024-09-21T14:59:55.617 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9439999999999999e-06 2024-09-21T14:59:55.617 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.617 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.618 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:55.618 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176816+0000", 2024-09-21T14:59:55.618 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019172872000000001 2024-09-21T14:59:55.618 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.618 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.618 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:55.618 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.176878+0000", 2024-09-21T14:59:55.618 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1734000000000002e-05 2024-09-21T14:59:55.618 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.618 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.618 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:55.618 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177003+0000", 2024-09-21T14:59:55.618 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012537400000000001 2024-09-21T14:59:55.618 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:55.619 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:55.619 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:55.619 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.619 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.619 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9402 3.3 3:c725132c:::rbd_data.11363a21d2be.0000000000000007:10 [read 172032~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:55.619 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157649+0000", 2024-09-21T14:59:55.619 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 76.424102927999996, 2024-09-21T14:59:55.619 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019567404, 2024-09-21T14:59:55.619 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:55.619 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:55.619 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:55.619 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:55.619 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:55.619 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9402 2024-09-21T14:59:55.619 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.620 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:55.620 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.620 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:55.620 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157649+0000", 2024-09-21T14:59:55.620 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:55.620 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.620 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.620 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:55.620 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157649+0000", 2024-09-21T14:59:55.620 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:55.620 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.620 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.620 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:55.620 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157649+0000", 2024-09-21T14:59:55.620 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:55.621 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.621 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.621 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:55.621 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157652+0000", 2024-09-21T14:59:55.621 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8779999999999998e-06 2024-09-21T14:59:55.621 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.621 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.621 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:55.621 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157657+0000", 2024-09-21T14:59:55.621 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6659999999999996e-06 2024-09-21T14:59:55.621 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.621 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.621 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:55.621 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157662+0000", 2024-09-21T14:59:55.621 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5519999999999998e-06 2024-09-21T14:59:55.621 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.622 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.622 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:55.622 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177040+0000", 2024-09-21T14:59:55.622 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019377807 2024-09-21T14:59:55.622 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.622 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.622 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:55.622 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177085+0000", 2024-09-21T14:59:55.622 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5195999999999999e-05 2024-09-21T14:59:55.622 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.622 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.622 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:55.622 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177217+0000", 2024-09-21T14:59:55.622 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013172600000000001 2024-09-21T14:59:55.622 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:55.623 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:55.623 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:55.623 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.623 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.623 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9403 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:55.623 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157949+0000", 2024-09-21T14:59:55.623 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 76.423802598999998, 2024-09-21T14:59:55.623 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023540373999999999, 2024-09-21T14:59:55.623 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:55.623 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:55.623 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:55.623 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:55.623 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:55.623 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9403 2024-09-21T14:59:55.623 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.624 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:55.624 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.624 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:55.624 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157949+0000", 2024-09-21T14:59:55.624 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:55.624 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.624 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.624 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:55.624 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157949+0000", 2024-09-21T14:59:55.624 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:55.624 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.624 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.624 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:55.624 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157948+0000", 2024-09-21T14:59:55.625 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T14:59:55.625 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.625 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.625 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:55.625 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157956+0000", 2024-09-21T14:59:55.625 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7999999999999999e-06 2024-09-21T14:59:55.625 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.625 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.625 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:55.625 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157969+0000", 2024-09-21T14:59:55.625 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2792000000000001e-05 2024-09-21T14:59:55.625 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.625 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.625 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:55.625 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157977+0000", 2024-09-21T14:59:55.625 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.704e-06 2024-09-21T14:59:55.626 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.626 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.626 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:55.626 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177246+0000", 2024-09-21T14:59:55.626 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019268264 2024-09-21T14:59:55.626 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.626 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.626 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:55.626 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177315+0000", 2024-09-21T14:59:55.626 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9480999999999994e-05 2024-09-21T14:59:55.626 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.626 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.626 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:59:55.626 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177615+0000", 2024-09-21T14:59:55.626 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00029929499999999999 2024-09-21T14:59:55.627 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.627 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.627 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:55.627 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181333+0000", 2024-09-21T14:59:55.627 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0037183149999999998 2024-09-21T14:59:55.627 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.627 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.627 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:55.627 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181374+0000", 2024-09-21T14:59:55.627 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.0723999999999998e-05 2024-09-21T14:59:55.627 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.627 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.627 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:55.627 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181434+0000", 2024-09-21T14:59:55.627 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0024e-05 2024-09-21T14:59:55.628 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.628 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.628 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:55.628 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181490+0000", 2024-09-21T14:59:55.628 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6245000000000001e-05 2024-09-21T14:59:55.628 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:55.628 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:55.628 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:55.628 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.628 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.628 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9404 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:55.628 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.160480+0000", 2024-09-21T14:59:55.628 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 76.421272248999998, 2024-09-21T14:59:55.628 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021137686999999999, 2024-09-21T14:59:55.628 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:55.628 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:55.629 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:55.629 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:55.629 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:55.629 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9404 2024-09-21T14:59:55.629 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.629 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:55.629 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.629 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:55.629 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160480+0000", 2024-09-21T14:59:55.629 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:55.629 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.629 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.629 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:55.629 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160480+0000", 2024-09-21T14:59:55.629 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:55.630 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.630 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.630 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:55.630 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160478+0000", 2024-09-21T14:59:55.630 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T14:59:55.630 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.630 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.630 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:55.630 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160486+0000", 2024-09-21T14:59:55.630 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9969999999999998e-06 2024-09-21T14:59:55.630 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.630 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.630 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:55.630 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160502+0000", 2024-09-21T14:59:55.631 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5650000000000001e-05 2024-09-21T14:59:55.631 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.631 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.631 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:55.631 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.160511+0000", 2024-09-21T14:59:55.631 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.3610000000000007e-06 2024-09-21T14:59:55.631 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.631 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.631 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:55.631 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.178035+0000", 2024-09-21T14:59:55.631 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.017523520000000001 2024-09-21T14:59:55.631 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.631 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.631 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:55.632 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.178151+0000", 2024-09-21T14:59:55.632 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000116013 2024-09-21T14:59:55.632 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.632 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.632 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:59:55.632 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.178408+0000", 2024-09-21T14:59:55.632 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00025743400000000002 2024-09-21T14:59:55.632 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.632 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.632 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:55.632 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181501+0000", 2024-09-21T14:59:55.632 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.003092891 2024-09-21T14:59:55.632 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.632 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.632 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:55.633 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181538+0000", 2024-09-21T14:59:55.633 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6977999999999997e-05 2024-09-21T14:59:55.633 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.633 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.633 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:55.633 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181582+0000", 2024-09-21T14:59:55.633 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3849e-05 2024-09-21T14:59:55.633 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.633 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.633 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:55.633 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181617+0000", 2024-09-21T14:59:55.633 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5423000000000003e-05 2024-09-21T14:59:55.633 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:55.633 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:55.634 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:55.634 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.634 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.634 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9407 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:55.634 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.168094+0000", 2024-09-21T14:59:55.634 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 76.413657749999999, 2024-09-21T14:59:55.634 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019444860000000001, 2024-09-21T14:59:55.634 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:55.634 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:55.634 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:55.634 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:55.634 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:55.635 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9407 2024-09-21T14:59:55.635 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.635 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:55.635 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.635 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:55.635 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.168094+0000", 2024-09-21T14:59:55.635 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:55.635 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.635 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.635 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:55.635 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.168094+0000", 2024-09-21T14:59:55.635 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:55.635 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.635 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.636 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:55.636 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.168093+0000", 2024-09-21T14:59:55.636 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:59:55.636 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.636 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.636 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:55.636 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.168119+0000", 2024-09-21T14:59:55.636 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5803999999999999e-05 2024-09-21T14:59:55.636 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.636 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.636 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:55.636 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.168135+0000", 2024-09-21T14:59:55.636 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6359999999999999e-05 2024-09-21T14:59:55.637 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.637 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.637 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:55.637 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.168144+0000", 2024-09-21T14:59:55.637 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4270000000000008e-06 2024-09-21T14:59:55.637 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.637 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.637 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:55.637 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181943+0000", 2024-09-21T14:59:55.637 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013799617 2024-09-21T14:59:55.637 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.637 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.637 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:55.637 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.182052+0000", 2024-09-21T14:59:55.637 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000108924 2024-09-21T14:59:55.638 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.638 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.638 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:59:55.638 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.183431+0000", 2024-09-21T14:59:55.638 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0013783619999999999 2024-09-21T14:59:55.638 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.638 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.638 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:55.638 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.187276+0000", 2024-09-21T14:59:55.638 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0038450469999999999 2024-09-21T14:59:55.638 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.638 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.638 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:55.639 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.187288+0000", 2024-09-21T14:59:55.639 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2414e-05 2024-09-21T14:59:55.639 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.639 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.639 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:55.639 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.187414+0000", 2024-09-21T14:59:55.639 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012614399999999999 2024-09-21T14:59:55.639 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.639 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.639 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:55.639 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.187539+0000", 2024-09-21T14:59:55.639 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012481800000000001 2024-09-21T14:59:55.639 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:55.639 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:55.640 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:55.640 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.640 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.640 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9410 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:55.640 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.174103+0000", 2024-09-21T14:59:55.640 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 76.407648840999997, 2024-09-21T14:59:55.640 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025493703, 2024-09-21T14:59:55.640 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:55.640 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:55.640 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:55.640 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:55.640 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:55.640 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9410 2024-09-21T14:59:55.640 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.640 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:55.641 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.641 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:55.641 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.174103+0000", 2024-09-21T14:59:55.641 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:55.641 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.641 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.641 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:55.641 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.174103+0000", 2024-09-21T14:59:55.641 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:55.641 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.641 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.641 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:55.641 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.174102+0000", 2024-09-21T14:59:55.641 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:59:55.641 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.642 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.642 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:55.642 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.174108+0000", 2024-09-21T14:59:55.642 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1149999999999999e-06 2024-09-21T14:59:55.642 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.642 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.642 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:55.642 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.174117+0000", 2024-09-21T14:59:55.642 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4870000000000002e-06 2024-09-21T14:59:55.642 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.642 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.642 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:55.642 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.174122+0000", 2024-09-21T14:59:55.642 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9790000000000001e-06 2024-09-21T14:59:55.643 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.643 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.643 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:55.643 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.187830+0000", 2024-09-21T14:59:55.643 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013708048 2024-09-21T14:59:55.643 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.643 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.643 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:55.643 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.187913+0000", 2024-09-21T14:59:55.643 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2763999999999996e-05 2024-09-21T14:59:55.643 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.643 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.643 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:59:55.643 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.188230+0000", 2024-09-21T14:59:55.644 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00031745100000000001 2024-09-21T14:59:55.644 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.644 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.644 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:55.644 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.192708+0000", 2024-09-21T14:59:55.644 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0044783890000000002 2024-09-21T14:59:55.644 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.644 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.644 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:55.644 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199487+0000", 2024-09-21T14:59:55.644 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0067782149999999998 2024-09-21T14:59:55.644 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.644 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.644 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:55.645 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199560+0000", 2024-09-21T14:59:55.645 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3752999999999997e-05 2024-09-21T14:59:55.645 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.645 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.645 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:55.645 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199597+0000", 2024-09-21T14:59:55.645 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6526999999999998e-05 2024-09-21T14:59:55.645 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:55.645 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:55.645 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:55.645 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.645 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.645 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9411 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:55.645 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.175943+0000", 2024-09-21T14:59:55.645 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 76.405808821999997, 2024-09-21T14:59:55.646 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023773181000000001, 2024-09-21T14:59:55.646 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:55.646 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:55.646 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:55.646 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:55.646 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:55.646 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9411 2024-09-21T14:59:55.646 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.646 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:55.646 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.646 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:55.646 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175943+0000", 2024-09-21T14:59:55.646 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:55.646 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.646 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.647 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:55.647 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175943+0000", 2024-09-21T14:59:55.647 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:55.647 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.647 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.647 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:55.647 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175942+0000", 2024-09-21T14:59:55.647 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:59:55.647 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.647 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.647 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:55.647 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175948+0000", 2024-09-21T14:59:55.647 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9689999999999999e-06 2024-09-21T14:59:55.648 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.648 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.648 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:55.648 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175960+0000", 2024-09-21T14:59:55.648 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1745e-05 2024-09-21T14:59:55.648 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.648 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.648 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:55.648 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175965+0000", 2024-09-21T14:59:55.648 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.203e-06 2024-09-21T14:59:55.648 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.648 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.648 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:55.648 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.189900+0000", 2024-09-21T14:59:55.648 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013934940999999999 2024-09-21T14:59:55.648 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.649 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.649 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:55.649 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.189988+0000", 2024-09-21T14:59:55.649 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8419999999999997e-05 2024-09-21T14:59:55.649 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.649 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.649 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:59:55.649 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.190321+0000", 2024-09-21T14:59:55.649 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00033231800000000001 2024-09-21T14:59:55.649 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.649 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.649 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:55.649 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.192746+0000", 2024-09-21T14:59:55.649 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0024254400000000001 2024-09-21T14:59:55.649 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.650 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.650 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:55.650 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199605+0000", 2024-09-21T14:59:55.650 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0068582809999999999 2024-09-21T14:59:55.650 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.650 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.650 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:55.650 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199680+0000", 2024-09-21T14:59:55.650 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5630000000000006e-05 2024-09-21T14:59:55.650 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.650 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.650 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:55.650 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199716+0000", 2024-09-21T14:59:55.650 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6248999999999998e-05 2024-09-21T14:59:55.651 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:55.651 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:55.651 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:55.651 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.651 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.651 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9412 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:55.651 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.177402+0000", 2024-09-21T14:59:55.651 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 76.404349725000003, 2024-09-21T14:59:55.651 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022445590000000001, 2024-09-21T14:59:55.651 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:55.651 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:55.651 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:55.651 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:55.652 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:55.652 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9412 2024-09-21T14:59:55.652 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.652 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:55.652 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.652 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:55.652 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177402+0000", 2024-09-21T14:59:55.652 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:55.652 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.652 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.652 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:55.652 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177402+0000", 2024-09-21T14:59:55.652 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:55.652 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.652 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.653 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:55.653 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177401+0000", 2024-09-21T14:59:55.653 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:59:55.653 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.653 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.653 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:55.653 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177409+0000", 2024-09-21T14:59:55.653 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4710000000000001e-06 2024-09-21T14:59:55.653 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.653 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.653 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:55.653 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177419+0000", 2024-09-21T14:59:55.653 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0351e-05 2024-09-21T14:59:55.653 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.653 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.654 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:55.654 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177424+0000", 2024-09-21T14:59:55.654 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1919999999999996e-06 2024-09-21T14:59:55.654 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.654 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.654 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:55.654 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.191343+0000", 2024-09-21T14:59:55.654 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.01391878 2024-09-21T14:59:55.654 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.654 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.654 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:55.654 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.191446+0000", 2024-09-21T14:59:55.654 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000102453 2024-09-21T14:59:55.654 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.655 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.655 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:59:55.655 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.192200+0000", 2024-09-21T14:59:55.655 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00075476899999999997 2024-09-21T14:59:55.655 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.655 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.655 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:55.655 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.193866+0000", 2024-09-21T14:59:55.655 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0016657519999999999 2024-09-21T14:59:55.655 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.655 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.655 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:55.655 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199724+0000", 2024-09-21T14:59:55.655 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0058579529999999999 2024-09-21T14:59:55.655 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.656 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.656 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:55.656 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199803+0000", 2024-09-21T14:59:55.656 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8662e-05 2024-09-21T14:59:55.656 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.656 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.656 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:55.656 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199848+0000", 2024-09-21T14:59:55.656 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5216000000000002e-05 2024-09-21T14:59:55.656 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:55.656 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:55.656 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:55.656 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.656 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.656 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:10445 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:55.657 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.271980+0000", 2024-09-21T14:59:55.657 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 76.309772359999997, 2024-09-21T14:59:55.657 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018664256000000001, 2024-09-21T14:59:55.657 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:55.657 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:55.657 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:55.657 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:55.657 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:55.657 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 10445 2024-09-21T14:59:55.657 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.657 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:55.657 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.657 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:55.657 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.271980+0000", 2024-09-21T14:59:55.657 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:55.657 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.658 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.658 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:55.658 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.271980+0000", 2024-09-21T14:59:55.658 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:55.658 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.658 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.658 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:55.658 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.271979+0000", 2024-09-21T14:59:55.658 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:55.658 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.658 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.658 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:55.658 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.271982+0000", 2024-09-21T14:59:55.658 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7240000000000001e-06 2024-09-21T14:59:55.659 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.659 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.659 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:55.659 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.271987+0000", 2024-09-21T14:59:55.659 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3639999999999998e-06 2024-09-21T14:59:55.659 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.659 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.659 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:55.659 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.271992+0000", 2024-09-21T14:59:55.659 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7110000000000003e-06 2024-09-21T14:59:55.659 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.659 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.659 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:55.659 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.277567+0000", 2024-09-21T14:59:55.659 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0055745009999999999 2024-09-21T14:59:55.660 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.660 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.660 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:55.660 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.277655+0000", 2024-09-21T14:59:55.660 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8465000000000001e-05 2024-09-21T14:59:55.660 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.660 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.660 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:59:55.660 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.278720+0000", 2024-09-21T14:59:55.660 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0010650569999999999 2024-09-21T14:59:55.660 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.660 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.660 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:55.660 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.281153+0000", 2024-09-21T14:59:55.660 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0024326389999999999 2024-09-21T14:59:55.661 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.661 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.661 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:55.661 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.290574+0000", 2024-09-21T14:59:55.661 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0094209489999999996 2024-09-21T14:59:55.661 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.661 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.661 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:55.661 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.290613+0000", 2024-09-21T14:59:55.661 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.8803000000000001e-05 2024-09-21T14:59:55.661 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.661 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.661 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:55.661 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.290644+0000", 2024-09-21T14:59:55.661 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1404e-05 2024-09-21T14:59:55.662 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:55.662 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:55.662 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:55.662 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.662 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.662 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:23084 3.3 3:c55e343c:::rbd_data.11366acc894.0000000000000006:19 [read 1032192~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:55.662 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.150724+0000", 2024-09-21T14:59:55.662 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 65.431027784999998, 2024-09-21T14:59:55.662 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018916460999999999, 2024-09-21T14:59:55.662 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:55.662 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:55.662 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:55.662 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:55.662 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:55.663 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 23084 2024-09-21T14:59:55.663 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.663 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:55.663 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.663 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:55.663 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.150724+0000", 2024-09-21T14:59:55.663 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:55.663 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.663 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.663 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:55.663 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.150724+0000", 2024-09-21T14:59:55.663 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:55.663 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.663 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.664 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:55.664 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.150724+0000", 2024-09-21T14:59:55.664 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:55.664 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.664 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.664 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:55.664 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.150727+0000", 2024-09-21T14:59:55.664 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7250000000000002e-06 2024-09-21T14:59:55.664 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.664 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.664 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:55.664 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.150732+0000", 2024-09-21T14:59:55.664 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5809999999999996e-06 2024-09-21T14:59:55.665 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.665 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.665 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:55.665 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.150737+0000", 2024-09-21T14:59:55.665 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3590000000000001e-06 2024-09-21T14:59:55.665 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.665 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.665 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:55.665 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.166694+0000", 2024-09-21T14:59:55.665 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.015957720000000002 2024-09-21T14:59:55.665 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.665 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.665 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:55.665 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.166769+0000", 2024-09-21T14:59:55.666 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5143999999999997e-05 2024-09-21T14:59:55.666 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.666 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.666 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:55.666 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.169641+0000", 2024-09-21T14:59:55.666 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0028713089999999998 2024-09-21T14:59:55.666 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:55.666 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:55.666 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:55.666 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.666 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.666 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:71939 3.e 3:75b0de84:::rbd_object_map.11368900df38:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100)", 2024-09-21T14:59:55.666 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:38.376206+0000", 2024-09-21T14:59:55.667 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 17.205546556000002, 2024-09-21T14:59:55.667 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019353591999999999, 2024-09-21T14:59:55.667 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:55.667 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:55.667 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:55.667 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:55.667 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:55.667 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 71939 2024-09-21T14:59:55.667 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.667 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:55.667 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.667 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:55.667 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.376206+0000", 2024-09-21T14:59:55.668 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:55.668 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.668 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.668 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:55.668 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.376206+0000", 2024-09-21T14:59:55.668 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:55.668 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.668 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.668 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:55.668 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.376205+0000", 2024-09-21T14:59:55.668 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:55.668 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.668 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.668 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:55.669 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.376209+0000", 2024-09-21T14:59:55.669 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.8820000000000003e-06 2024-09-21T14:59:55.669 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.669 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.669 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:55.669 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.376215+0000", 2024-09-21T14:59:55.669 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5729999999999998e-06 2024-09-21T14:59:55.669 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.669 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.669 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:55.669 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.376221+0000", 2024-09-21T14:59:55.669 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2040000000000002e-06 2024-09-21T14:59:55.669 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.669 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.669 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:55.670 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.383959+0000", 2024-09-21T14:59:55.670 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0077380089999999997 2024-09-21T14:59:55.670 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.670 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.670 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:55.670 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.384131+0000", 2024-09-21T14:59:55.670 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017252899999999999 2024-09-21T14:59:55.670 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.670 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.670 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:59:55.670 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.384666+0000", 2024-09-21T14:59:55.670 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00053445399999999996 2024-09-21T14:59:55.670 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.670 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.671 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:55.671 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.388174+0000", 2024-09-21T14:59:55.671 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.003508639 2024-09-21T14:59:55.671 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.671 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.671 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:55.671 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.395460+0000", 2024-09-21T14:59:55.671 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.007285489 2024-09-21T14:59:55.671 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.671 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.671 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:55.671 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.395503+0000", 2024-09-21T14:59:55.671 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3080000000000001e-05 2024-09-21T14:59:55.671 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.672 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.672 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:55.672 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.395559+0000", 2024-09-21T14:59:55.672 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6118e-05 2024-09-21T14:59:55.672 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:55.672 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:55.672 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:55.672 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.672 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.672 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:71940 3.e 3:75b0de84:::rbd_object_map.11368900df38:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100)", 2024-09-21T14:59:55.672 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:38.376225+0000", 2024-09-21T14:59:55.672 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 17.205527318000001, 2024-09-21T14:59:55.672 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019578849999999998, 2024-09-21T14:59:55.672 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:55.673 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:55.673 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:55.673 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:55.673 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:55.673 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 71940 2024-09-21T14:59:55.673 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.673 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:55.673 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.673 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:55.673 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.376225+0000", 2024-09-21T14:59:55.673 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:55.673 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.673 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.673 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:55.674 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.376225+0000", 2024-09-21T14:59:55.674 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:55.674 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.674 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.674 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:55.674 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.376224+0000", 2024-09-21T14:59:55.674 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:55.674 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.674 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.674 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:55.674 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.376228+0000", 2024-09-21T14:59:55.674 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5889999999999999e-06 2024-09-21T14:59:55.674 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.675 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.675 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:55.675 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.376233+0000", 2024-09-21T14:59:55.675 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.322e-06 2024-09-21T14:59:55.675 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.675 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.675 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:55.675 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.376238+0000", 2024-09-21T14:59:55.675 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7960000000000003e-06 2024-09-21T14:59:55.675 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.675 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.675 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:55.675 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.385506+0000", 2024-09-21T14:59:55.675 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0092679279999999999 2024-09-21T14:59:55.676 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.676 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.676 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:55.676 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.385598+0000", 2024-09-21T14:59:55.676 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1939000000000002e-05 2024-09-21T14:59:55.676 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.676 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.676 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:59:55.676 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.386013+0000", 2024-09-21T14:59:55.676 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000414793 2024-09-21T14:59:55.676 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.676 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.676 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:55.677 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.395634+0000", 2024-09-21T14:59:55.677 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0096215720000000001 2024-09-21T14:59:55.677 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.677 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.677 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:55.677 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.395754+0000", 2024-09-21T14:59:55.677 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000120097 2024-09-21T14:59:55.677 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.677 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.677 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:55.677 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.395782+0000", 2024-09-21T14:59:55.677 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7583e-05 2024-09-21T14:59:55.678 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.678 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.678 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:55.678 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.395804+0000", 2024-09-21T14:59:55.678 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1594999999999999e-05 2024-09-21T14:59:55.678 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:55.678 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:55.678 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:55.678 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.678 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.678 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:71941 3.e 3:75b0de84:::rbd_object_map.11368900df38:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100)", 2024-09-21T14:59:55.678 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:38.376242+0000", 2024-09-21T14:59:55.678 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 17.205510266000001, 2024-09-21T14:59:55.678 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019640456000000001, 2024-09-21T14:59:55.679 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:55.679 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:55.679 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:55.679 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:55.679 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:55.679 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 71941 2024-09-21T14:59:55.679 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.679 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:55.679 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.679 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:55.679 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.376242+0000", 2024-09-21T14:59:55.679 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:55.679 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.679 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.680 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:55.680 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.376242+0000", 2024-09-21T14:59:55.680 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:55.680 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.680 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.680 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:55.680 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.376241+0000", 2024-09-21T14:59:55.680 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:55.680 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.680 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.680 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:55.680 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.376244+0000", 2024-09-21T14:59:55.680 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0340000000000001e-06 2024-09-21T14:59:55.680 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.681 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.681 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:55.681 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.376250+0000", 2024-09-21T14:59:55.681 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3499999999999996e-06 2024-09-21T14:59:55.681 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.681 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.681 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:55.681 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.376255+0000", 2024-09-21T14:59:55.681 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1320000000000002e-06 2024-09-21T14:59:55.681 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.681 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.681 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:55.681 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.386791+0000", 2024-09-21T14:59:55.681 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.010536085000000001 2024-09-21T14:59:55.682 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.682 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.682 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:55.682 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.386868+0000", 2024-09-21T14:59:55.682 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6674999999999993e-05 2024-09-21T14:59:55.682 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.682 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.682 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:59:55.682 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.387258+0000", 2024-09-21T14:59:55.682 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00038990999999999999 2024-09-21T14:59:55.682 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.682 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.682 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:55.682 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.395685+0000", 2024-09-21T14:59:55.683 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0084273590000000006 2024-09-21T14:59:55.683 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.683 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.683 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:55.683 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.395812+0000", 2024-09-21T14:59:55.683 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012719700000000001 2024-09-21T14:59:55.683 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.683 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.683 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:55.683 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.395849+0000", 2024-09-21T14:59:55.683 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6748999999999997e-05 2024-09-21T14:59:55.683 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:55.683 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:55.683 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:55.683 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:38.395882+0000", 2024-09-21T14:59:55.684 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3318999999999997e-05 2024-09-21T14:59:55.684 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:55.684 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:55.684 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:55.684 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:55.684 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:55.684 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:55.685 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T14:59:55.686 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:59:55.679+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:59:55.786 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:59:55.785+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:59:55.797 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:55.797 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:59:55.797 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T14:59:55.797 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:55.807 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T14:59:55.886 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:59:55.885+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:59:55.913 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:55.913 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:59:55.913 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T14:59:55.913 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T14:59:55.913 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:55.922 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T14:59:55.987 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:59:55.985+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:59:56.037 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:56.037 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T14:59:56.037 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T14:59:56.037 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T14:59:56.037 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.037 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T14:59:56.038 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.205754+0000", 2024-09-21T14:59:56.038 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 112.830061899, 2024-09-21T14:59:56.038 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.41117094199999998, 2024-09-21T14:59:56.038 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:56.038 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:56.038 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:56.038 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:59:56.038 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:59:56.038 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1 2024-09-21T14:59:56.038 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.038 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:56.038 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.038 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:56.038 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205754+0000", 2024-09-21T14:59:56.039 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.039 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.039 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.039 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:56.039 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205754+0000", 2024-09-21T14:59:56.039 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.039 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.039 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.039 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:56.039 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205752+0000", 2024-09-21T14:59:56.039 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T14:59:56.039 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.039 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.039 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:56.039 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205765+0000", 2024-09-21T14:59:56.040 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3230999999999999e-05 2024-09-21T14:59:56.040 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.040 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.040 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:56.040 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205782+0000", 2024-09-21T14:59:56.040 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6634e-05 2024-09-21T14:59:56.040 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.040 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.040 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:56.040 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205793+0000", 2024-09-21T14:59:56.040 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1297e-05 2024-09-21T14:59:56.040 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.040 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.040 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:56.040 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205867+0000", 2024-09-21T14:59:56.040 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4269000000000003e-05 2024-09-21T14:59:56.041 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.041 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.041 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T14:59:56.041 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205877+0000", 2024-09-21T14:59:56.041 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2690000000000005e-06 2024-09-21T14:59:56.041 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.041 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.041 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:56.041 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.613700+0000", 2024-09-21T14:59:56.041 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.407823455 2024-09-21T14:59:56.041 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.041 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.041 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:56.041 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.614203+0000", 2024-09-21T14:59:56.041 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00050311500000000005 2024-09-21T14:59:56.042 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.042 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.042 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:59:56.042 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.614750+0000", 2024-09-21T14:59:56.042 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00054738699999999998 2024-09-21T14:59:56.042 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.042 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.042 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:56.042 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616367+0000", 2024-09-21T14:59:56.042 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001616634 2024-09-21T14:59:56.042 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.042 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.042 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:56.042 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616574+0000", 2024-09-21T14:59:56.042 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00020668499999999999 2024-09-21T14:59:56.043 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.043 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.043 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:56.043 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616802+0000", 2024-09-21T14:59:56.043 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00022817900000000001 2024-09-21T14:59:56.043 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.043 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.043 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:56.043 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616925+0000", 2024-09-21T14:59:56.043 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012309699999999999 2024-09-21T14:59:56.043 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.043 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:56.043 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.043 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.043 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.044 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T14:59:56.044 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.955609+0000", 2024-09-21T14:59:56.044 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 112.080207512, 2024-09-21T14:59:56.044 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013624153999999999, 2024-09-21T14:59:56.044 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:56.044 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:56.044 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:56.044 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:59:56.044 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:59:56.044 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 115 2024-09-21T14:59:56.044 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.044 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:56.044 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.044 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:56.045 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955609+0000", 2024-09-21T14:59:56.045 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.045 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.045 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.045 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:56.045 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955609+0000", 2024-09-21T14:59:56.045 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.045 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.045 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.045 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:56.045 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955607+0000", 2024-09-21T14:59:56.045 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-21T14:59:56.045 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.045 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.045 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:56.046 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955732+0000", 2024-09-21T14:59:56.046 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000125566 2024-09-21T14:59:56.046 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.046 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.046 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:56.046 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955752+0000", 2024-09-21T14:59:56.046 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9916999999999999e-05 2024-09-21T14:59:56.046 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.046 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.046 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:56.046 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955767+0000", 2024-09-21T14:59:56.046 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4687e-05 2024-09-21T14:59:56.046 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.046 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.047 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:56.047 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.965574+0000", 2024-09-21T14:59:56.047 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0098072850000000007 2024-09-21T14:59:56.047 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.047 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.047 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:56.047 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.965642+0000", 2024-09-21T14:59:56.047 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7784e-05 2024-09-21T14:59:56.047 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.047 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.047 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:59:56.047 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.965753+0000", 2024-09-21T14:59:56.047 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000111111 2024-09-21T14:59:56.047 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.047 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.048 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:56.048 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967717+0000", 2024-09-21T14:59:56.048 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0019635339999999999 2024-09-21T14:59:56.048 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.048 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.048 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:56.048 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.969014+0000", 2024-09-21T14:59:56.048 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001297093 2024-09-21T14:59:56.048 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.048 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.048 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:56.048 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.969130+0000", 2024-09-21T14:59:56.048 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000116011 2024-09-21T14:59:56.048 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.048 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.048 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:56.049 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.969233+0000", 2024-09-21T14:59:56.049 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000102982 2024-09-21T14:59:56.049 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.049 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:56.049 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.049 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.049 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.049 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T14:59:56.049 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.955783+0000", 2024-09-21T14:59:56.049 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 112.080032843, 2024-09-21T14:59:56.049 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014630241, 2024-09-21T14:59:56.049 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:56.049 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:56.049 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:56.050 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:59:56.050 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:59:56.050 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 116 2024-09-21T14:59:56.050 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.050 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:56.050 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.050 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:56.050 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955783+0000", 2024-09-21T14:59:56.050 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.050 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.050 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.050 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:56.050 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955783+0000", 2024-09-21T14:59:56.050 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.050 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.051 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.051 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:56.051 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955782+0000", 2024-09-21T14:59:56.051 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:59:56.051 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.051 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.051 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:56.051 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955791+0000", 2024-09-21T14:59:56.051 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8089999999999999e-06 2024-09-21T14:59:56.051 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.051 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.051 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:56.051 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955804+0000", 2024-09-21T14:59:56.051 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2548000000000001e-05 2024-09-21T14:59:56.051 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.052 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.052 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:56.052 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955814+0000", 2024-09-21T14:59:56.052 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0158e-05 2024-09-21T14:59:56.052 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.052 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.052 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:56.052 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.966515+0000", 2024-09-21T14:59:56.052 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.010701756 2024-09-21T14:59:56.052 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.052 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.052 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:56.052 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.966606+0000", 2024-09-21T14:59:56.052 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0761999999999996e-05 2024-09-21T14:59:56.052 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.052 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.053 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:59:56.053 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.966740+0000", 2024-09-21T14:59:56.053 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000133693 2024-09-21T14:59:56.053 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.053 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.053 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:56.053 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.968920+0000", 2024-09-21T14:59:56.053 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0021799290000000002 2024-09-21T14:59:56.053 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.053 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.053 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:56.053 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.970320+0000", 2024-09-21T14:59:56.053 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0014001759999999999 2024-09-21T14:59:56.053 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.053 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.054 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:56.054 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.970376+0000", 2024-09-21T14:59:56.054 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5696999999999999e-05 2024-09-21T14:59:56.054 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.054 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.054 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:56.054 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.970413+0000", 2024-09-21T14:59:56.054 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7750999999999998e-05 2024-09-21T14:59:56.054 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.054 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:56.054 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.054 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.054 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.054 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T14:59:56.055 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.955829+0000", 2024-09-21T14:59:56.055 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 112.07998737299999, 2024-09-21T14:59:56.055 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014919597999999999, 2024-09-21T14:59:56.055 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:56.055 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:56.055 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:56.055 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:59:56.055 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:59:56.055 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 117 2024-09-21T14:59:56.055 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.055 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:56.055 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.055 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:56.055 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955829+0000", 2024-09-21T14:59:56.055 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.055 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.056 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.056 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:56.056 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955829+0000", 2024-09-21T14:59:56.056 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.056 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.056 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.056 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:56.056 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955827+0000", 2024-09-21T14:59:56.056 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T14:59:56.056 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.056 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.056 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:56.056 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955838+0000", 2024-09-21T14:59:56.056 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.111e-05 2024-09-21T14:59:56.056 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.057 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.057 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:56.057 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955875+0000", 2024-09-21T14:59:56.057 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6211999999999998e-05 2024-09-21T14:59:56.057 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.057 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.057 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:56.057 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955899+0000", 2024-09-21T14:59:56.057 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4128999999999999e-05 2024-09-21T14:59:56.057 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.057 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.057 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:56.057 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967767+0000", 2024-09-21T14:59:56.057 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.011868251999999999 2024-09-21T14:59:56.057 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.058 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.058 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:56.058 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967866+0000", 2024-09-21T14:59:56.058 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8664000000000003e-05 2024-09-21T14:59:56.058 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.058 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.058 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:59:56.058 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.968128+0000", 2024-09-21T14:59:56.058 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00026243600000000001 2024-09-21T14:59:56.058 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.058 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.058 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:56.058 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.969393+0000", 2024-09-21T14:59:56.058 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0012650350000000001 2024-09-21T14:59:56.058 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.058 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.059 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:56.059 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.970523+0000", 2024-09-21T14:59:56.059 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001129557 2024-09-21T14:59:56.059 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.059 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.059 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:56.059 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.970603+0000", 2024-09-21T14:59:56.059 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0090000000000001e-05 2024-09-21T14:59:56.059 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.059 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.059 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:56.059 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.970748+0000", 2024-09-21T14:59:56.059 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014552799999999999 2024-09-21T14:59:56.059 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.060 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:56.060 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.060 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.060 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.060 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T14:59:56.060 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:04.493315+0000", 2024-09-21T14:59:56.060 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 111.54250054000001, 2024-09-21T14:59:56.060 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014055766000000001, 2024-09-21T14:59:56.060 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:56.060 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:56.060 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:56.060 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:59:56.060 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:59:56.060 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 137 2024-09-21T14:59:56.060 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.061 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:56.061 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.061 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:56.061 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493315+0000", 2024-09-21T14:59:56.061 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.061 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.061 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.061 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:56.061 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493315+0000", 2024-09-21T14:59:56.061 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.061 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.061 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.061 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:56.062 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493314+0000", 2024-09-21T14:59:56.062 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:59:56.062 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.062 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.062 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:56.062 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493507+0000", 2024-09-21T14:59:56.062 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000192444 2024-09-21T14:59:56.062 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.062 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.062 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:56.062 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493528+0000", 2024-09-21T14:59:56.062 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0845000000000001e-05 2024-09-21T14:59:56.062 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.062 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.063 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:56.063 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493543+0000", 2024-09-21T14:59:56.063 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5800000000000001e-05 2024-09-21T14:59:56.063 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.063 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.063 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:56.063 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.502575+0000", 2024-09-21T14:59:56.063 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0090312389999999999 2024-09-21T14:59:56.063 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.063 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.063 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:56.063 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.502677+0000", 2024-09-21T14:59:56.063 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000102271 2024-09-21T14:59:56.063 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.064 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.064 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:59:56.064 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.502825+0000", 2024-09-21T14:59:56.064 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000148171 2024-09-21T14:59:56.064 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.064 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.064 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:56.064 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507251+0000", 2024-09-21T14:59:56.064 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0044253879999999997 2024-09-21T14:59:56.064 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.064 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.064 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:56.064 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507313+0000", 2024-09-21T14:59:56.064 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2170999999999998e-05 2024-09-21T14:59:56.065 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.065 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.065 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:56.065 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507346+0000", 2024-09-21T14:59:56.065 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3079e-05 2024-09-21T14:59:56.065 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.065 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.065 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:56.065 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507371+0000", 2024-09-21T14:59:56.065 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5477999999999999e-05 2024-09-21T14:59:56.065 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.065 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:56.065 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.065 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.065 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.066 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T14:59:56.066 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:04.493570+0000", 2024-09-21T14:59:56.066 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 111.542246135, 2024-09-21T14:59:56.066 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014061686, 2024-09-21T14:59:56.066 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:56.066 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:56.066 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:56.066 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:59:56.066 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:59:56.066 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 138 2024-09-21T14:59:56.066 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.066 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:56.066 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.066 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:56.066 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493570+0000", 2024-09-21T14:59:56.067 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.067 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.067 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.067 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:56.067 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493570+0000", 2024-09-21T14:59:56.067 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.067 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.067 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.067 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:56.067 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493568+0000", 2024-09-21T14:59:56.067 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T14:59:56.067 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.067 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.067 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:56.068 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493579+0000", 2024-09-21T14:59:56.068 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0574000000000001e-05 2024-09-21T14:59:56.068 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.068 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.068 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:56.068 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493595+0000", 2024-09-21T14:59:56.068 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6504999999999999e-05 2024-09-21T14:59:56.068 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.068 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.068 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:56.068 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493608+0000", 2024-09-21T14:59:56.068 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2836999999999999e-05 2024-09-21T14:59:56.068 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.068 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.068 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:56.069 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.503909+0000", 2024-09-21T14:59:56.069 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.01030118 2024-09-21T14:59:56.069 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.069 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.069 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:56.069 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.504064+0000", 2024-09-21T14:59:56.069 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015493000000000001 2024-09-21T14:59:56.069 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.069 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.069 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:59:56.069 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.504299+0000", 2024-09-21T14:59:56.069 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00023416100000000001 2024-09-21T14:59:56.069 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.069 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.069 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:56.070 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507466+0000", 2024-09-21T14:59:56.070 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0031678380000000001 2024-09-21T14:59:56.070 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.070 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.070 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:56.070 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507483+0000", 2024-09-21T14:59:56.070 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6606999999999999e-05 2024-09-21T14:59:56.070 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.070 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.070 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:56.070 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507566+0000", 2024-09-21T14:59:56.070 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3110999999999998e-05 2024-09-21T14:59:56.070 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.070 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.071 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:56.071 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507632+0000", 2024-09-21T14:59:56.071 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5480000000000003e-05 2024-09-21T14:59:56.071 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.071 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:56.071 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.071 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.071 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.071 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T14:59:56.071 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:04.493625+0000", 2024-09-21T14:59:56.071 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 111.54219144699999, 2024-09-21T14:59:56.071 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014665378999999999, 2024-09-21T14:59:56.071 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:56.071 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:56.072 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:56.072 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:59:56.072 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:59:56.072 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139 2024-09-21T14:59:56.072 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.072 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:56.072 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.072 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:56.072 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493625+0000", 2024-09-21T14:59:56.072 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.072 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.072 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.072 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:56.072 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493625+0000", 2024-09-21T14:59:56.072 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.072 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.073 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.073 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:56.073 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493624+0000", 2024-09-21T14:59:56.073 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:59:56.073 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.073 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.073 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:56.073 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493638+0000", 2024-09-21T14:59:56.073 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4282e-05 2024-09-21T14:59:56.073 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.073 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.073 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:56.073 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493654+0000", 2024-09-21T14:59:56.073 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5739999999999998e-05 2024-09-21T14:59:56.073 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.074 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.074 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:56.074 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493666+0000", 2024-09-21T14:59:56.074 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2865e-05 2024-09-21T14:59:56.074 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.074 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.074 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:56.074 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.505329+0000", 2024-09-21T14:59:56.074 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.011662237000000001 2024-09-21T14:59:56.074 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.074 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.074 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:56.074 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.505377+0000", 2024-09-21T14:59:56.074 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7862999999999999e-05 2024-09-21T14:59:56.075 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.075 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.075 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:59:56.075 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.505455+0000", 2024-09-21T14:59:56.075 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8918000000000006e-05 2024-09-21T14:59:56.075 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.075 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.075 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:56.075 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507650+0000", 2024-09-21T14:59:56.075 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.002195027 2024-09-21T14:59:56.075 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.075 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.075 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:56.076 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508149+0000", 2024-09-21T14:59:56.076 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00049828399999999999 2024-09-21T14:59:56.076 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.076 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.076 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:56.076 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508230+0000", 2024-09-21T14:59:56.076 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1376000000000002e-05 2024-09-21T14:59:56.076 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.076 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.076 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:56.076 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508290+0000", 2024-09-21T14:59:56.076 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9852999999999998e-05 2024-09-21T14:59:56.076 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.076 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:56.077 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.077 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.077 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.077 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T14:59:56.077 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:04.493679+0000", 2024-09-21T14:59:56.077 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 111.54213654900001, 2024-09-21T14:59:56.077 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014869869000000001, 2024-09-21T14:59:56.077 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:56.077 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:56.077 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:56.077 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:59:56.077 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:59:56.077 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140 2024-09-21T14:59:56.077 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.078 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:56.078 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.078 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:56.078 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493679+0000", 2024-09-21T14:59:56.078 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.078 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.078 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.078 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:56.078 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493679+0000", 2024-09-21T14:59:56.078 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.078 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.078 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.078 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:56.078 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493679+0000", 2024-09-21T14:59:56.079 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:59:56.079 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.079 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.079 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:56.079 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493889+0000", 2024-09-21T14:59:56.079 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00021032700000000001 2024-09-21T14:59:56.079 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.079 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.079 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:56.079 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493931+0000", 2024-09-21T14:59:56.079 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1971000000000002e-05 2024-09-21T14:59:56.079 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.079 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.079 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:56.079 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493947+0000", 2024-09-21T14:59:56.079 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6385e-05 2024-09-21T14:59:56.080 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.080 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.080 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:56.080 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.505804+0000", 2024-09-21T14:59:56.080 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.011857136000000001 2024-09-21T14:59:56.080 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.080 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.080 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:56.080 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.505992+0000", 2024-09-21T14:59:56.080 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000187665 2024-09-21T14:59:56.080 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.080 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.080 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:59:56.080 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.506281+0000", 2024-09-21T14:59:56.080 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000288712 2024-09-21T14:59:56.081 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.081 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.081 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:56.081 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508349+0000", 2024-09-21T14:59:56.081 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0020685790000000001 2024-09-21T14:59:56.081 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.081 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.081 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:56.081 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508371+0000", 2024-09-21T14:59:56.081 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1747e-05 2024-09-21T14:59:56.081 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.081 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.081 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:56.081 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508450+0000", 2024-09-21T14:59:56.081 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9332000000000005e-05 2024-09-21T14:59:56.082 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.082 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.082 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:56.082 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508549+0000", 2024-09-21T14:59:56.082 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8918000000000004e-05 2024-09-21T14:59:56.082 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.082 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:56.082 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.082 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.082 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.082 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T14:59:56.082 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:04.493967+0000", 2024-09-21T14:59:56.082 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 111.541849512, 2024-09-21T14:59:56.082 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.015281075999999999, 2024-09-21T14:59:56.082 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:56.083 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:56.083 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:56.083 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T14:59:56.083 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T14:59:56.083 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 141 2024-09-21T14:59:56.083 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.083 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:56.083 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.083 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:56.083 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493967+0000", 2024-09-21T14:59:56.083 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.083 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.083 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.084 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:56.084 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493967+0000", 2024-09-21T14:59:56.084 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.084 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.084 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.084 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:56.084 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493965+0000", 2024-09-21T14:59:56.084 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T14:59:56.084 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.084 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.084 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:56.084 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493987+0000", 2024-09-21T14:59:56.084 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1662999999999999e-05 2024-09-21T14:59:56.085 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.085 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.085 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:56.085 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.494004+0000", 2024-09-21T14:59:56.085 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7311000000000001e-05 2024-09-21T14:59:56.085 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.085 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.085 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:56.085 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.494017+0000", 2024-09-21T14:59:56.085 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3202e-05 2024-09-21T14:59:56.085 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.085 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.085 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:56.085 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507678+0000", 2024-09-21T14:59:56.085 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013660432 2024-09-21T14:59:56.086 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.086 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.086 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:56.086 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507726+0000", 2024-09-21T14:59:56.086 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8154e-05 2024-09-21T14:59:56.086 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.086 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.086 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:59:56.086 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507797+0000", 2024-09-21T14:59:56.086 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.1110000000000002e-05 2024-09-21T14:59:56.086 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.086 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.086 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:56.086 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508710+0000", 2024-09-21T14:59:56.087 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00091339299999999997 2024-09-21T14:59:56.087 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.087 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.087 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:56.087 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.509108+0000", 2024-09-21T14:59:56.087 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00039718300000000001 2024-09-21T14:59:56.087 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.087 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.087 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:56.087 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.509186+0000", 2024-09-21T14:59:56.087 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8592999999999998e-05 2024-09-21T14:59:56.087 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.087 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.087 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:56.088 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.509248+0000", 2024-09-21T14:59:56.088 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1450000000000003e-05 2024-09-21T14:59:56.088 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.088 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:56.088 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.088 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.088 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.088 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6631 3.1a 3:5d547f54:::rbd_data.11361bed1f78.0000000000000003:d [read 0~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:59:56.088 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131606+0000", 2024-09-21T14:59:56.088 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 80.904210234000004, 2024-09-21T14:59:56.088 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013903148000000001, 2024-09-21T14:59:56.088 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:56.088 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:56.088 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:56.088 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:56.089 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:56.089 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6631 2024-09-21T14:59:56.089 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.089 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:56.089 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.089 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:56.089 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131606+0000", 2024-09-21T14:59:56.089 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.089 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.089 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.089 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:56.089 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131606+0000", 2024-09-21T14:59:56.089 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.089 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.089 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.090 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:56.090 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131606+0000", 2024-09-21T14:59:56.090 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T14:59:56.090 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.090 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.090 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:56.090 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131607+0000", 2024-09-21T14:59:56.090 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2699999999999999e-06 2024-09-21T14:59:56.090 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.090 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.090 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:56.090 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131610+0000", 2024-09-21T14:59:56.090 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.88e-06 2024-09-21T14:59:56.090 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.091 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.091 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:56.091 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131612+0000", 2024-09-21T14:59:56.091 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5170000000000002e-06 2024-09-21T14:59:56.091 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.091 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.091 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:56.091 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.144205+0000", 2024-09-21T14:59:56.091 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012592763 2024-09-21T14:59:56.091 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.091 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.091 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:56.091 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.144332+0000", 2024-09-21T14:59:56.091 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012694200000000001 2024-09-21T14:59:56.091 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.092 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.092 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:56.092 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.145509+0000", 2024-09-21T14:59:56.092 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001176992 2024-09-21T14:59:56.092 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.092 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:56.092 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.092 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.092 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.092 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6632 3.1a 3:5d547f54:::rbd_data.11361bed1f78.0000000000000003:d [read 81920~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:59:56.092 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131614+0000", 2024-09-21T14:59:56.092 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 80.904201626000003, 2024-09-21T14:59:56.092 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014093553999999999, 2024-09-21T14:59:56.092 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:56.092 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:56.093 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:56.093 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:56.093 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:56.093 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6632 2024-09-21T14:59:56.093 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.093 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:56.093 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.093 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:56.093 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131614+0000", 2024-09-21T14:59:56.093 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.093 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.093 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.093 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:56.093 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131614+0000", 2024-09-21T14:59:56.093 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.094 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.094 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.094 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:56.094 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131614+0000", 2024-09-21T14:59:56.094 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T14:59:56.094 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.094 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.094 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:56.094 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131615+0000", 2024-09-21T14:59:56.094 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.218e-06 2024-09-21T14:59:56.094 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.094 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.094 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:56.094 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131618+0000", 2024-09-21T14:59:56.095 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9050000000000001e-06 2024-09-21T14:59:56.095 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.095 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.095 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:56.095 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131621+0000", 2024-09-21T14:59:56.095 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7439999999999999e-06 2024-09-21T14:59:56.095 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.095 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.095 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:56.095 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.145538+0000", 2024-09-21T14:59:56.095 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013916618 2024-09-21T14:59:56.095 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.095 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.095 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:56.095 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.145619+0000", 2024-09-21T14:59:56.096 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1812999999999997e-05 2024-09-21T14:59:56.096 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.096 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.096 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:56.096 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.145708+0000", 2024-09-21T14:59:56.096 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8460000000000003e-05 2024-09-21T14:59:56.096 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.096 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:56.096 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.096 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.096 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.096 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6633 3.1a 3:5d547f54:::rbd_data.11361bed1f78.0000000000000003:d [read 147456~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:59:56.096 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131623+0000", 2024-09-21T14:59:56.096 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 80.904192660000007, 2024-09-21T14:59:56.096 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014395845000000001, 2024-09-21T14:59:56.097 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:56.097 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:56.097 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:56.097 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:56.097 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:56.097 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6633 2024-09-21T14:59:56.097 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.097 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:56.097 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.097 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:56.097 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131623+0000", 2024-09-21T14:59:56.097 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.097 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.097 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.097 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:56.098 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131623+0000", 2024-09-21T14:59:56.098 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.098 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.098 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.098 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:56.098 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131623+0000", 2024-09-21T14:59:56.098 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T14:59:56.098 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.098 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.098 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:56.098 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131628+0000", 2024-09-21T14:59:56.098 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0370000000000002e-06 2024-09-21T14:59:56.098 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.099 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.099 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:56.099 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131632+0000", 2024-09-21T14:59:56.099 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3519999999999998e-06 2024-09-21T14:59:56.099 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.099 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.099 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:56.099 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131634+0000", 2024-09-21T14:59:56.099 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5019999999999999e-06 2024-09-21T14:59:56.099 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.099 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.099 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:56.099 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.145734+0000", 2024-09-21T14:59:56.099 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014100118 2024-09-21T14:59:56.099 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.099 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.100 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:56.100 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.145804+0000", 2024-09-21T14:59:56.100 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0229999999999997e-05 2024-09-21T14:59:56.100 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.100 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.100 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:56.100 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146019+0000", 2024-09-21T14:59:56.100 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00021481699999999999 2024-09-21T14:59:56.100 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.100 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:56.100 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.100 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.100 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.100 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6634 3.1a 3:5d547f54:::rbd_data.11361bed1f78.0000000000000003:d [read 229376~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:59:56.100 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131636+0000", 2024-09-21T14:59:56.100 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 80.904179549999995, 2024-09-21T14:59:56.101 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014645843, 2024-09-21T14:59:56.101 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:56.101 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:56.101 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:56.101 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:56.101 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:56.101 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6634 2024-09-21T14:59:56.101 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.101 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:56.101 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.101 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:56.101 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131636+0000", 2024-09-21T14:59:56.101 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.101 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.101 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.101 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:56.102 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131636+0000", 2024-09-21T14:59:56.102 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.102 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.102 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.102 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:56.102 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131636+0000", 2024-09-21T14:59:56.102 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:56.102 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.102 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.102 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:56.102 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131641+0000", 2024-09-21T14:59:56.102 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8489999999999997e-06 2024-09-21T14:59:56.102 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.103 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.103 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:56.103 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131644+0000", 2024-09-21T14:59:56.103 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.179e-06 2024-09-21T14:59:56.103 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.103 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.103 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:56.103 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131647+0000", 2024-09-21T14:59:56.103 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8009999999999999e-06 2024-09-21T14:59:56.103 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.103 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.103 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:56.103 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146046+0000", 2024-09-21T14:59:56.103 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014399018 2024-09-21T14:59:56.103 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.104 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.104 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:56.104 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146118+0000", 2024-09-21T14:59:56.104 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2044000000000003e-05 2024-09-21T14:59:56.104 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.104 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.104 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:56.104 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146282+0000", 2024-09-21T14:59:56.104 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000164196 2024-09-21T14:59:56.104 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.104 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:56.104 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.104 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.104 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.105 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6639 3.1a 3:592d80ed:::rbd_data.11361bed1f78.0000000000000005:d [read 0~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:59:56.105 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131650+0000", 2024-09-21T14:59:56.105 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 80.904166351000001, 2024-09-21T14:59:56.105 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016002832000000002, 2024-09-21T14:59:56.105 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:56.105 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:56.105 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:56.105 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:56.105 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:56.105 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6639 2024-09-21T14:59:56.105 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.105 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:56.105 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.105 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:56.106 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131650+0000", 2024-09-21T14:59:56.106 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.106 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.106 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.106 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:56.106 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131650+0000", 2024-09-21T14:59:56.106 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.106 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.106 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.106 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:56.106 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131649+0000", 2024-09-21T14:59:56.106 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:56.106 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.106 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.107 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:56.107 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131651+0000", 2024-09-21T14:59:56.107 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9199999999999998e-06 2024-09-21T14:59:56.107 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.107 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.107 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:56.107 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131656+0000", 2024-09-21T14:59:56.107 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9899999999999997e-06 2024-09-21T14:59:56.107 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.107 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.107 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:56.107 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131665+0000", 2024-09-21T14:59:56.107 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0999999999999993e-06 2024-09-21T14:59:56.107 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.108 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.108 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:56.108 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146309+0000", 2024-09-21T14:59:56.108 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014643893999999999 2024-09-21T14:59:56.108 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.108 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.108 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:56.108 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146392+0000", 2024-09-21T14:59:56.108 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2531e-05 2024-09-21T14:59:56.108 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.108 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.108 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:56.108 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147653+0000", 2024-09-21T14:59:56.108 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0012606710000000001 2024-09-21T14:59:56.108 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.108 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:56.109 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.109 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.109 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.109 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6640 3.1a 3:592d80ed:::rbd_data.11361bed1f78.0000000000000005:d [read 65536~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:59:56.109 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131670+0000", 2024-09-21T14:59:56.109 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 80.904146052000002, 2024-09-21T14:59:56.109 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016188859, 2024-09-21T14:59:56.109 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:56.109 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:56.109 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:56.109 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:56.109 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:56.109 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6640 2024-09-21T14:59:56.109 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.109 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:56.109 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.110 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:56.110 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131670+0000", 2024-09-21T14:59:56.110 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.110 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.110 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.110 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:56.110 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131670+0000", 2024-09-21T14:59:56.110 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.110 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.110 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.110 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:56.110 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131670+0000", 2024-09-21T14:59:56.110 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T14:59:56.110 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.110 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.111 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:56.111 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131672+0000", 2024-09-21T14:59:56.111 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.818e-06 2024-09-21T14:59:56.111 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.111 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.111 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:56.111 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131675+0000", 2024-09-21T14:59:56.111 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5930000000000002e-06 2024-09-21T14:59:56.111 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.111 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.111 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:56.111 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131678+0000", 2024-09-21T14:59:56.111 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6790000000000001e-06 2024-09-21T14:59:56.111 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.111 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.112 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:56.112 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147681+0000", 2024-09-21T14:59:56.112 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016002755 2024-09-21T14:59:56.112 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.112 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.112 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:56.112 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147762+0000", 2024-09-21T14:59:56.112 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1052999999999998e-05 2024-09-21T14:59:56.112 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.112 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.112 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:56.112 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147859+0000", 2024-09-21T14:59:56.112 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.7199000000000002e-05 2024-09-21T14:59:56.112 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.112 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:56.112 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.113 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.113 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.113 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6641 3.1a 3:592d80ed:::rbd_data.11361bed1f78.0000000000000005:d [read 147456~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:59:56.113 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131680+0000", 2024-09-21T14:59:56.113 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 80.904135764000003, 2024-09-21T14:59:56.113 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016431296000000001, 2024-09-21T14:59:56.113 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:56.113 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:56.113 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:56.113 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:56.113 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:56.113 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6641 2024-09-21T14:59:56.113 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.113 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:56.113 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.114 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:56.114 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131680+0000", 2024-09-21T14:59:56.114 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.114 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.114 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.114 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:56.114 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131680+0000", 2024-09-21T14:59:56.114 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.114 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.114 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.114 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:56.114 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131680+0000", 2024-09-21T14:59:56.114 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T14:59:56.114 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.115 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.115 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:56.115 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131681+0000", 2024-09-21T14:59:56.115 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2610000000000001e-06 2024-09-21T14:59:56.115 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.115 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.115 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:56.115 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131685+0000", 2024-09-21T14:59:56.115 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.8299999999999998e-06 2024-09-21T14:59:56.115 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.115 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.115 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:56.115 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131688+0000", 2024-09-21T14:59:56.115 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6189999999999998e-06 2024-09-21T14:59:56.116 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.116 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.116 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:56.116 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147884+0000", 2024-09-21T14:59:56.116 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016196465 2024-09-21T14:59:56.116 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.116 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.116 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:56.116 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147949+0000", 2024-09-21T14:59:56.116 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4867000000000001e-05 2024-09-21T14:59:56.116 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.116 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.116 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:56.116 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.148112+0000", 2024-09-21T14:59:56.116 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00016247700000000001 2024-09-21T14:59:56.117 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.117 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:56.117 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.117 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.117 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.117 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6642 3.1a 3:592d80ed:::rbd_data.11361bed1f78.0000000000000005:d [read 212992~32768 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T14:59:56.117 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131690+0000", 2024-09-21T14:59:56.117 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 80.904125886000003, 2024-09-21T14:59:56.117 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016751335999999999, 2024-09-21T14:59:56.117 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:56.117 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:56.117 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:56.117 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:56.117 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:56.117 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6642 2024-09-21T14:59:56.117 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.118 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:56.118 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.118 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:56.118 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131690+0000", 2024-09-21T14:59:56.118 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.118 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.118 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.118 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:56.118 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131690+0000", 2024-09-21T14:59:56.118 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.118 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.118 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.118 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:56.118 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131690+0000", 2024-09-21T14:59:56.119 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T14:59:56.119 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.119 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.119 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:56.119 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131691+0000", 2024-09-21T14:59:56.119 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3120000000000001e-06 2024-09-21T14:59:56.119 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.119 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.119 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:56.119 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131694+0000", 2024-09-21T14:59:56.119 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0070000000000002e-06 2024-09-21T14:59:56.119 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.119 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.119 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:56.120 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131700+0000", 2024-09-21T14:59:56.120 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0800000000000002e-06 2024-09-21T14:59:56.120 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.120 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.120 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:56.120 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.148135+0000", 2024-09-21T14:59:56.120 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016434250000000001 2024-09-21T14:59:56.120 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.120 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.120 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:56.120 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.148204+0000", 2024-09-21T14:59:56.120 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9466000000000002e-05 2024-09-21T14:59:56.120 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.120 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.120 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:56.120 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.148441+0000", 2024-09-21T14:59:56.121 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00023743700000000001 2024-09-21T14:59:56.121 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.121 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:56.121 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.121 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.121 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.121 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:61214 3.7 3:e28aec10:::rbd_header.1136862f872f:head [watch reconnect cookie 139864671932080 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-21T14:59:56.121 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:18.147172+0000", 2024-09-21T14:59:56.121 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 37.888644255999999, 2024-09-21T14:59:56.121 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.019264956, 2024-09-21T14:59:56.121 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:56.121 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:56.121 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:56.121 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:56.121 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:56.122 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 61214 2024-09-21T14:59:56.122 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.122 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:56.122 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.122 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:56.122 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147172+0000", 2024-09-21T14:59:56.122 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.122 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.122 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.122 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:56.122 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147172+0000", 2024-09-21T14:59:56.122 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.122 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.122 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.122 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:56.123 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147168+0000", 2024-09-21T14:59:56.123 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999962 2024-09-21T14:59:56.123 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.123 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.123 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:56.123 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147188+0000", 2024-09-21T14:59:56.123 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9717000000000001e-05 2024-09-21T14:59:56.123 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.123 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.123 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:56.123 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147239+0000", 2024-09-21T14:59:56.123 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1493999999999999e-05 2024-09-21T14:59:56.123 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.123 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.123 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:56.124 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147265+0000", 2024-09-21T14:59:56.124 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5746999999999999e-05 2024-09-21T14:59:56.124 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.124 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.124 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:56.124 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147372+0000", 2024-09-21T14:59:56.124 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010681200000000001 2024-09-21T14:59:56.124 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.124 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.124 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op must wait for map", 2024-09-21T14:59:56.124 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147380+0000", 2024-09-21T14:59:56.124 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7919999999999993e-06 2024-09-21T14:59:56.124 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.125 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.125 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:56.125 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.154025+0000", 2024-09-21T14:59:56.125 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0066454909999999999 2024-09-21T14:59:56.125 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.125 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.125 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T14:59:56.125 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.154041+0000", 2024-09-21T14:59:56.125 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5894000000000001e-05 2024-09-21T14:59:56.125 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.125 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.125 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:56.125 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.164345+0000", 2024-09-21T14:59:56.125 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0103040459999999 2024-09-21T14:59:56.125 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.126 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.126 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:56.126 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.164590+0000", 2024-09-21T14:59:56.126 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00024492699999999998 2024-09-21T14:59:56.126 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.126 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.126 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:56.126 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.166437+0000", 2024-09-21T14:59:56.126 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0018466330000000001 2024-09-21T14:59:56.126 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.126 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:56.126 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.126 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.126 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.126 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:61215 3.7 3:e28aec10:::rbd_header.1136862f872f:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-21T14:59:56.127 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:18.152008+0000", 2024-09-21T14:59:56.127 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 37.883807804, 2024-09-21T14:59:56.127 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.015644958, 2024-09-21T14:59:56.127 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:56.127 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:56.127 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:56.127 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:56.127 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:56.127 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 61215 2024-09-21T14:59:56.127 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.127 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:56.127 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.127 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:56.127 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152008+0000", 2024-09-21T14:59:56.127 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.128 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.128 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.128 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:56.128 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152008+0000", 2024-09-21T14:59:56.128 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.128 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.128 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.128 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:56.128 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152005+0000", 2024-09-21T14:59:56.128 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-21T14:59:56.128 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.128 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.128 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:56.128 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152025+0000", 2024-09-21T14:59:56.128 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9306000000000001e-05 2024-09-21T14:59:56.129 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.129 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.129 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:56.129 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152054+0000", 2024-09-21T14:59:56.129 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9189000000000001e-05 2024-09-21T14:59:56.129 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.129 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.129 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:56.129 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152078+0000", 2024-09-21T14:59:56.129 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4317000000000001e-05 2024-09-21T14:59:56.129 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.129 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.129 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:56.129 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152243+0000", 2024-09-21T14:59:56.129 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000164802 2024-09-21T14:59:56.129 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.130 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.130 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting_for_map not empty", 2024-09-21T14:59:56.130 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152251+0000", 2024-09-21T14:59:56.130 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0600000000000008e-06 2024-09-21T14:59:56.130 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.130 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.130 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:56.130 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.154741+0000", 2024-09-21T14:59:56.130 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0024900069999999998 2024-09-21T14:59:56.130 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.130 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.130 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T14:59:56.130 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.154753+0000", 2024-09-21T14:59:56.130 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1588e-05 2024-09-21T14:59:56.130 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.130 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.131 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:56.131 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.165380+0000", 2024-09-21T14:59:56.131 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0106272249999999 2024-09-21T14:59:56.131 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.131 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.131 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:56.131 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.165517+0000", 2024-09-21T14:59:56.131 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013737999999999999 2024-09-21T14:59:56.131 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.131 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.131 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 6", 2024-09-21T14:59:56.131 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.165938+0000", 2024-09-21T14:59:56.131 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00042082399999999998 2024-09-21T14:59:56.131 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.131 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.132 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:56.132 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.166795+0000", 2024-09-21T14:59:56.132 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00085674899999999997 2024-09-21T14:59:56.132 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.132 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.132 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:56.132 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.167598+0000", 2024-09-21T14:59:56.132 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00080335599999999997 2024-09-21T14:59:56.132 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.132 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.132 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:56.132 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.167632+0000", 2024-09-21T14:59:56.132 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4122000000000003e-05 2024-09-21T14:59:56.132 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.132 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.132 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:56.133 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.167653+0000", 2024-09-21T14:59:56.133 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0862000000000001e-05 2024-09-21T14:59:56.133 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.133 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:56.133 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.133 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.133 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.133 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:73698 3.10 3:0a4d95f6:::rbd_data.1136f4939602.000000000000000b:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-21T14:59:56.133 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:41.969870+0000", 2024-09-21T14:59:56.133 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.065945665999999, 2024-09-21T14:59:56.133 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.017502461, 2024-09-21T14:59:56.133 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:56.133 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:56.133 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:56.134 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:56.134 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:56.134 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 73698 2024-09-21T14:59:56.134 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.134 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:56.134 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.134 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:56.134 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.969870+0000", 2024-09-21T14:59:56.134 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.134 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.134 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.134 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:56.135 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.969870+0000", 2024-09-21T14:59:56.135 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.135 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.135 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.135 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:56.135 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.969868+0000", 2024-09-21T14:59:56.135 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T14:59:56.135 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.135 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.135 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:56.135 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.969885+0000", 2024-09-21T14:59:56.135 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7416999999999999e-05 2024-09-21T14:59:56.135 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.135 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.135 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:56.136 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.969918+0000", 2024-09-21T14:59:56.136 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3133000000000003e-05 2024-09-21T14:59:56.136 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.136 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.136 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:56.136 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.969934+0000", 2024-09-21T14:59:56.136 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6027000000000001e-05 2024-09-21T14:59:56.136 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.136 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.136 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:56.136 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.970021+0000", 2024-09-21T14:59:56.136 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6907e-05 2024-09-21T14:59:56.136 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.136 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.136 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:56.136 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.970148+0000", 2024-09-21T14:59:56.137 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012701499999999999 2024-09-21T14:59:56.137 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.137 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.137 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T14:59:56.137 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.972971+0000", 2024-09-21T14:59:56.137 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.002822873 2024-09-21T14:59:56.137 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.137 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.137 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:56.137 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.987124+0000", 2024-09-21T14:59:56.137 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0141529 2024-09-21T14:59:56.137 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.137 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.137 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:56.137 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.987146+0000", 2024-09-21T14:59:56.137 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1783000000000001e-05 2024-09-21T14:59:56.138 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.138 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.138 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:56.138 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.987258+0000", 2024-09-21T14:59:56.138 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000112555 2024-09-21T14:59:56.138 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.138 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.138 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:56.138 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.987373+0000", 2024-09-21T14:59:56.138 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000114434 2024-09-21T14:59:56.138 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.138 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:56.138 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.138 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.138 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:56.138 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:56.140 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T14:59:56.140 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:59:56.137+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:59:56.140 INFO:teuthology.orchestra.run.smithi032.stderr:nodeep-scrub is set 2024-09-21T14:59:56.241 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:59:56.235+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:59:56.262 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:56.263 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:59:56.263 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T14:59:56.263 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:56.273 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T14:59:56.343 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:59:56.338+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:59:56.377 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:56.377 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:59:56.377 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T14:59:56.377 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T14:59:56.377 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:56.388 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T14:59:56.443 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:59:56.441+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:59:56.492 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:56.492 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T14:59:56.492 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T14:59:56.492 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T14:59:56.492 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.492 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:15130 3.2 3:47f1bbdb:::rbd_data.11361bed1f78.0000000000000067:head [sparse-read 0~147456] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-21T14:59:56.492 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:40.288284+0000", 2024-09-21T14:59:56.492 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 76.203204811999996, 2024-09-21T14:59:56.493 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1142935039999999, 2024-09-21T14:59:56.493 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:56.493 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:56.493 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:56.493 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:56.493 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:56.493 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 15130 2024-09-21T14:59:56.493 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.493 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:56.493 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.493 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:56.493 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288284+0000", 2024-09-21T14:59:56.493 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.494 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.494 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.494 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:56.494 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288284+0000", 2024-09-21T14:59:56.494 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.494 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.494 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.494 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:56.494 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288284+0000", 2024-09-21T14:59:56.494 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:56.494 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.494 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.494 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:56.495 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288288+0000", 2024-09-21T14:59:56.495 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5829999999999998e-06 2024-09-21T14:59:56.495 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.495 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.495 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:56.495 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288297+0000", 2024-09-21T14:59:56.495 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0669999999999998e-06 2024-09-21T14:59:56.495 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.495 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.495 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:56.495 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288305+0000", 2024-09-21T14:59:56.495 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8860000000000004e-06 2024-09-21T14:59:56.495 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.496 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.496 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:56.496 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288417+0000", 2024-09-21T14:59:56.496 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000112952 2024-09-21T14:59:56.496 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.496 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.496 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T14:59:56.496 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288428+0000", 2024-09-21T14:59:56.496 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0631e-05 2024-09-21T14:59:56.496 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.496 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.496 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:56.496 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.401876+0000", 2024-09-21T14:59:56.497 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1134479610000001 2024-09-21T14:59:56.497 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.497 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.497 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:56.497 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.402076+0000", 2024-09-21T14:59:56.497 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00019944200000000001 2024-09-21T14:59:56.497 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.497 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.497 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:56.497 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.402578+0000", 2024-09-21T14:59:56.497 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00050242899999999998 2024-09-21T14:59:56.497 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.497 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:56.498 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.498 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.498 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.498 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45708 3.2 3:415db91a:::rbd_data.1136ecbe5879.0000000000000000:28 [read 140288~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:56.498 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141801+0000", 2024-09-21T14:59:56.498 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 46.349688618000002, 2024-09-21T14:59:56.498 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034274200999999997, 2024-09-21T14:59:56.498 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:56.498 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:56.498 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:56.498 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:56.498 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:56.499 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45708 2024-09-21T14:59:56.499 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.499 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:56.499 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.499 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:56.499 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141801+0000", 2024-09-21T14:59:56.499 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.499 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.499 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.499 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:56.499 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141801+0000", 2024-09-21T14:59:56.499 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.499 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.500 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.500 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:56.500 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141800+0000", 2024-09-21T14:59:56.500 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:56.500 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.500 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.500 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:56.500 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141803+0000", 2024-09-21T14:59:56.500 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4210000000000002e-06 2024-09-21T14:59:56.500 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.500 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.500 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:56.500 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141812+0000", 2024-09-21T14:59:56.501 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.9800000000000004e-06 2024-09-21T14:59:56.501 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.501 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.501 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:56.501 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141816+0000", 2024-09-21T14:59:56.501 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4769999999999997e-06 2024-09-21T14:59:56.501 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.501 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.501 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:56.501 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.175865+0000", 2024-09-21T14:59:56.501 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034048875999999999 2024-09-21T14:59:56.501 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.501 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.501 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:56.502 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.175943+0000", 2024-09-21T14:59:56.502 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7464000000000002e-05 2024-09-21T14:59:56.502 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.502 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.502 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:56.502 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176075+0000", 2024-09-21T14:59:56.502 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013230800000000001 2024-09-21T14:59:56.502 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.502 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:56.502 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.502 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.502 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.503 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45709 3.2 3:415db91a:::rbd_data.1136ecbe5879.0000000000000000:28 [read 143360~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:56.503 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141820+0000", 2024-09-21T14:59:56.503 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 46.349668770999997, 2024-09-21T14:59:56.503 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034537536000000001, 2024-09-21T14:59:56.503 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:56.503 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:56.503 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:56.503 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:56.503 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:56.503 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45709 2024-09-21T14:59:56.503 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.503 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:56.503 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.503 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:56.503 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141820+0000", 2024-09-21T14:59:56.504 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.504 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.504 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.504 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:56.504 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141820+0000", 2024-09-21T14:59:56.504 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.504 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.504 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.504 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:56.504 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141820+0000", 2024-09-21T14:59:56.504 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:56.504 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.504 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.504 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:56.505 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141823+0000", 2024-09-21T14:59:56.505 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5110000000000002e-06 2024-09-21T14:59:56.505 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.505 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.505 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:56.505 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141829+0000", 2024-09-21T14:59:56.505 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0850000000000002e-06 2024-09-21T14:59:56.505 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.505 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.505 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:56.505 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141833+0000", 2024-09-21T14:59:56.505 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7620000000000003e-06 2024-09-21T14:59:56.505 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.505 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.505 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:56.505 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176103+0000", 2024-09-21T14:59:56.506 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034269933000000002 2024-09-21T14:59:56.506 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.506 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.506 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:56.506 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176161+0000", 2024-09-21T14:59:56.506 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7658999999999998e-05 2024-09-21T14:59:56.506 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.506 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.506 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:56.506 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176358+0000", 2024-09-21T14:59:56.506 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000196937 2024-09-21T14:59:56.506 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.506 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:56.506 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.507 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.507 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.507 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45710 3.2 3:415db91a:::rbd_data.1136ecbe5879.0000000000000000:28 [read 146432~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:56.507 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141838+0000", 2024-09-21T14:59:56.507 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 46.349651608000002, 2024-09-21T14:59:56.507 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034953674999999997, 2024-09-21T14:59:56.507 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:56.507 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:56.507 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:56.507 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:56.507 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:56.507 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45710 2024-09-21T14:59:56.507 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.507 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:56.507 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.507 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:56.508 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141838+0000", 2024-09-21T14:59:56.508 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.508 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.508 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.508 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:56.508 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141838+0000", 2024-09-21T14:59:56.508 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.508 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.508 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.508 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:56.508 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141837+0000", 2024-09-21T14:59:56.508 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:56.508 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.508 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.508 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:56.509 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141843+0000", 2024-09-21T14:59:56.509 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1619999999999996e-06 2024-09-21T14:59:56.509 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.509 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.509 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:56.509 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141850+0000", 2024-09-21T14:59:56.509 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4799999999999998e-06 2024-09-21T14:59:56.509 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.509 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.509 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:56.509 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141853+0000", 2024-09-21T14:59:56.509 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3050000000000001e-06 2024-09-21T14:59:56.509 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.509 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.509 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:56.509 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176490+0000", 2024-09-21T14:59:56.510 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034636413999999997 2024-09-21T14:59:56.510 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.510 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.510 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:56.510 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176606+0000", 2024-09-21T14:59:56.510 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000115841 2024-09-21T14:59:56.510 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.510 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.510 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:56.510 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176791+0000", 2024-09-21T14:59:56.510 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00018581500000000001 2024-09-21T14:59:56.510 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.510 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:56.510 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.510 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.511 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.511 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45711 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:56.511 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141856+0000", 2024-09-21T14:59:56.511 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 46.349633271000002, 2024-09-21T14:59:56.511 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034986308000000001, 2024-09-21T14:59:56.511 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:56.511 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:56.511 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:56.511 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:56.511 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:56.511 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45711 2024-09-21T14:59:56.511 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.511 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:56.511 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.511 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:56.512 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141856+0000", 2024-09-21T14:59:56.512 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.512 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.512 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.512 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:56.512 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141856+0000", 2024-09-21T14:59:56.512 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.512 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.512 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.512 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:56.512 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141856+0000", 2024-09-21T14:59:56.512 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:56.512 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.512 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.512 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:56.512 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141862+0000", 2024-09-21T14:59:56.513 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.567e-06 2024-09-21T14:59:56.513 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.513 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.513 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:56.513 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141868+0000", 2024-09-21T14:59:56.513 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9569999999999999e-06 2024-09-21T14:59:56.513 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.513 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.513 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:56.513 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141871+0000", 2024-09-21T14:59:56.513 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.146e-06 2024-09-21T14:59:56.513 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.513 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.513 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:56.514 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176831+0000", 2024-09-21T14:59:56.514 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034959935999999997 2024-09-21T14:59:56.514 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.514 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.514 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:56.514 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176842+0000", 2024-09-21T14:59:56.514 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1008e-05 2024-09-21T14:59:56.514 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.514 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:56.514 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.514 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.514 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.514 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45712 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:56.514 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141874+0000", 2024-09-21T14:59:56.515 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 46.349615376000003, 2024-09-21T14:59:56.515 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034993511999999997, 2024-09-21T14:59:56.515 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:56.515 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:56.515 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:56.515 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:56.515 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:56.515 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45712 2024-09-21T14:59:56.515 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.515 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:56.515 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.515 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:56.515 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141874+0000", 2024-09-21T14:59:56.515 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.515 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.516 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.516 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:56.516 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141874+0000", 2024-09-21T14:59:56.516 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.516 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.516 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.516 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:56.516 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141874+0000", 2024-09-21T14:59:56.516 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:56.516 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.516 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.516 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:56.516 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141875+0000", 2024-09-21T14:59:56.516 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5430000000000001e-06 2024-09-21T14:59:56.516 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.516 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.517 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:56.517 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141878+0000", 2024-09-21T14:59:56.517 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1420000000000002e-06 2024-09-21T14:59:56.517 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.517 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.517 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:56.517 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141881+0000", 2024-09-21T14:59:56.517 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.807e-06 2024-09-21T14:59:56.517 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.517 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.517 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:56.517 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176860+0000", 2024-09-21T14:59:56.517 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034979243 2024-09-21T14:59:56.517 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.517 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.518 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:56.518 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176867+0000", 2024-09-21T14:59:56.518 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0169999999999999e-06 2024-09-21T14:59:56.518 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.518 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:56.518 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.518 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.518 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.518 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45713 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:56.518 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141883+0000", 2024-09-21T14:59:56.518 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 46.349605801000003, 2024-09-21T14:59:56.518 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035010516999999998, 2024-09-21T14:59:56.518 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:56.518 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:56.518 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:56.519 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:56.519 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:56.519 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45713 2024-09-21T14:59:56.519 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.519 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:56.519 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.519 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:56.519 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141883+0000", 2024-09-21T14:59:56.519 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.519 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.519 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.519 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:56.519 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141883+0000", 2024-09-21T14:59:56.519 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.519 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.520 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.520 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:56.520 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141883+0000", 2024-09-21T14:59:56.520 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:56.520 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.520 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.520 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:56.520 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141885+0000", 2024-09-21T14:59:56.520 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4649999999999999e-06 2024-09-21T14:59:56.520 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.520 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.520 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:56.520 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141888+0000", 2024-09-21T14:59:56.520 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3890000000000001e-06 2024-09-21T14:59:56.520 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.521 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.521 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:56.521 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141891+0000", 2024-09-21T14:59:56.521 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9239999999999999e-06 2024-09-21T14:59:56.521 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.521 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.521 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:56.521 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176887+0000", 2024-09-21T14:59:56.521 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034996370999999998 2024-09-21T14:59:56.521 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.521 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.521 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:56.521 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176894+0000", 2024-09-21T14:59:56.521 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6069999999999996e-06 2024-09-21T14:59:56.521 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.521 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:56.522 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.522 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.522 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.522 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45714 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:56.522 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141893+0000", 2024-09-21T14:59:56.522 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 46.349595886000003, 2024-09-21T14:59:56.522 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035048215000000001, 2024-09-21T14:59:56.522 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:56.522 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:56.522 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:56.522 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:56.522 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:56.522 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45714 2024-09-21T14:59:56.522 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.523 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:56.523 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.523 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:56.523 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141893+0000", 2024-09-21T14:59:56.523 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.523 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.523 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.523 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:56.523 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141893+0000", 2024-09-21T14:59:56.523 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.523 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.523 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.523 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:56.523 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141893+0000", 2024-09-21T14:59:56.523 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:56.524 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.524 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.524 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:56.524 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141895+0000", 2024-09-21T14:59:56.524 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.703e-06 2024-09-21T14:59:56.524 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.524 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.524 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:56.524 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141900+0000", 2024-09-21T14:59:56.524 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.242e-06 2024-09-21T14:59:56.524 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.524 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.524 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:56.524 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141903+0000", 2024-09-21T14:59:56.525 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.117e-06 2024-09-21T14:59:56.525 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.525 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.525 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:56.525 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176930+0000", 2024-09-21T14:59:56.525 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035026750000000002 2024-09-21T14:59:56.525 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.525 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.525 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:56.525 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176942+0000", 2024-09-21T14:59:56.525 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1668000000000001e-05 2024-09-21T14:59:56.525 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.525 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:56.525 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.525 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.526 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.526 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45715 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:56.526 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141906+0000", 2024-09-21T14:59:56.526 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 46.349583727999999, 2024-09-21T14:59:56.526 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035066057999999997, 2024-09-21T14:59:56.526 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:56.526 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:56.526 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:56.526 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:56.526 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:56.526 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45715 2024-09-21T14:59:56.526 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.526 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:56.526 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.527 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:56.527 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141906+0000", 2024-09-21T14:59:56.527 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.527 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.527 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.527 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:56.527 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141906+0000", 2024-09-21T14:59:56.527 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.527 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.527 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.527 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:56.527 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141905+0000", 2024-09-21T14:59:56.527 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:56.528 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.528 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.528 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:56.528 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141907+0000", 2024-09-21T14:59:56.528 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5400000000000001e-06 2024-09-21T14:59:56.528 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.528 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.528 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:56.528 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141912+0000", 2024-09-21T14:59:56.528 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.502e-06 2024-09-21T14:59:56.528 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.528 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.528 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:56.528 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141915+0000", 2024-09-21T14:59:56.529 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0170000000000001e-06 2024-09-21T14:59:56.529 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.529 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.529 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:56.529 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176964+0000", 2024-09-21T14:59:56.529 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035048587999999999 2024-09-21T14:59:56.529 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.529 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.529 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:56.529 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176972+0000", 2024-09-21T14:59:56.529 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6580000000000002e-06 2024-09-21T14:59:56.529 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.529 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:56.529 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.529 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.530 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.530 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45716 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:56.530 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141918+0000", 2024-09-21T14:59:56.530 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 46.349571552, 2024-09-21T14:59:56.530 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035086223, 2024-09-21T14:59:56.530 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:56.530 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:56.530 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:56.530 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:56.530 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:56.530 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45716 2024-09-21T14:59:56.530 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.530 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:56.530 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.531 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:56.531 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141918+0000", 2024-09-21T14:59:56.531 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.531 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.531 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.531 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:56.531 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141918+0000", 2024-09-21T14:59:56.531 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.531 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.531 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.531 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:56.531 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141917+0000", 2024-09-21T14:59:56.531 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:56.531 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.532 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.532 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:56.532 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141919+0000", 2024-09-21T14:59:56.532 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5680000000000001e-06 2024-09-21T14:59:56.532 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.532 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.532 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:56.532 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141922+0000", 2024-09-21T14:59:56.532 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2789999999999999e-06 2024-09-21T14:59:56.532 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.532 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.532 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:56.532 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141925+0000", 2024-09-21T14:59:56.532 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9469999999999999e-06 2024-09-21T14:59:56.532 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.533 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.533 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:56.533 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176997+0000", 2024-09-21T14:59:56.533 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035071366999999999 2024-09-21T14:59:56.533 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.533 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.533 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:56.533 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177004+0000", 2024-09-21T14:59:56.533 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3309999999999998e-06 2024-09-21T14:59:56.533 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.533 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:56.533 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.533 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.534 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.534 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45717 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:56.534 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141928+0000", 2024-09-21T14:59:56.534 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 46.349561606000002, 2024-09-21T14:59:56.534 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035102573999999998, 2024-09-21T14:59:56.534 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:56.534 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:56.534 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:56.534 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:56.534 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:56.534 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45717 2024-09-21T14:59:56.534 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.534 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:56.535 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.535 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:56.535 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141928+0000", 2024-09-21T14:59:56.535 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.535 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.535 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.535 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:56.535 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141928+0000", 2024-09-21T14:59:56.535 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.535 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.535 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.535 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:56.535 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141927+0000", 2024-09-21T14:59:56.535 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T14:59:56.536 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.536 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.536 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:56.536 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141929+0000", 2024-09-21T14:59:56.536 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.533e-06 2024-09-21T14:59:56.536 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.536 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.536 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:56.536 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141934+0000", 2024-09-21T14:59:56.536 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1660000000000002e-06 2024-09-21T14:59:56.536 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.536 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.536 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:56.536 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141937+0000", 2024-09-21T14:59:56.537 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0309999999999999e-06 2024-09-21T14:59:56.537 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.537 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.537 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:56.537 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177023+0000", 2024-09-21T14:59:56.537 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035085595999999997 2024-09-21T14:59:56.537 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.537 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.537 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:56.537 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177030+0000", 2024-09-21T14:59:56.537 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.486e-06 2024-09-21T14:59:56.537 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.537 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:56.537 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.537 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.538 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.538 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45718 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:56.538 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141940+0000", 2024-09-21T14:59:56.538 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 46.349549748000001, 2024-09-21T14:59:56.538 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035124956999999998, 2024-09-21T14:59:56.538 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:56.538 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:56.538 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:56.538 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:56.538 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:56.538 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45718 2024-09-21T14:59:56.538 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.538 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:56.538 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.539 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:56.539 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141940+0000", 2024-09-21T14:59:56.539 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.539 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.539 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.539 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:56.539 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141940+0000", 2024-09-21T14:59:56.539 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.539 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.539 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.539 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:56.539 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141939+0000", 2024-09-21T14:59:56.539 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:56.539 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.539 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.540 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:56.540 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141941+0000", 2024-09-21T14:59:56.540 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.502e-06 2024-09-21T14:59:56.540 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.540 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.540 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:56.540 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141946+0000", 2024-09-21T14:59:56.540 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3449999999999996e-06 2024-09-21T14:59:56.540 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.540 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.540 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:56.540 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141949+0000", 2024-09-21T14:59:56.540 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9749999999999999e-06 2024-09-21T14:59:56.540 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.541 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.541 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:56.541 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177057+0000", 2024-09-21T14:59:56.541 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035108385999999998 2024-09-21T14:59:56.541 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.541 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.541 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:56.541 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177064+0000", 2024-09-21T14:59:56.541 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0099999999999998e-06 2024-09-21T14:59:56.541 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.541 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:56.541 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.541 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.541 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.541 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45719 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:56.542 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141952+0000", 2024-09-21T14:59:56.542 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 46.349537474999998, 2024-09-21T14:59:56.542 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035139108000000002, 2024-09-21T14:59:56.542 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:56.542 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:56.542 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:56.542 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:56.542 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:56.542 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45719 2024-09-21T14:59:56.542 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.542 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:56.542 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.542 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:56.542 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141952+0000", 2024-09-21T14:59:56.543 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.543 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.543 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.543 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:56.543 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141952+0000", 2024-09-21T14:59:56.543 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.543 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.543 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.543 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:56.543 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141952+0000", 2024-09-21T14:59:56.543 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:56.543 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.543 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.543 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:56.544 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141953+0000", 2024-09-21T14:59:56.544 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.486e-06 2024-09-21T14:59:56.544 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.544 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.544 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:56.544 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141959+0000", 2024-09-21T14:59:56.544 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4940000000000003e-06 2024-09-21T14:59:56.544 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.544 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.544 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:56.544 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141963+0000", 2024-09-21T14:59:56.544 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3540000000000002e-06 2024-09-21T14:59:56.544 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.545 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.545 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:56.545 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177084+0000", 2024-09-21T14:59:56.545 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035120983000000001 2024-09-21T14:59:56.545 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.545 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.545 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:56.545 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177091+0000", 2024-09-21T14:59:56.545 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0350000000000004e-06 2024-09-21T14:59:56.545 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.545 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:56.545 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.545 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.545 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.546 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45720 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:56.546 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141965+0000", 2024-09-21T14:59:56.546 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 46.349524008000003, 2024-09-21T14:59:56.546 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035155268000000003, 2024-09-21T14:59:56.546 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:56.546 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:56.546 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:56.546 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:56.546 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:56.546 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45720 2024-09-21T14:59:56.546 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.546 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:56.546 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.546 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:56.546 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141965+0000", 2024-09-21T14:59:56.547 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.547 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.547 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.547 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:56.547 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141965+0000", 2024-09-21T14:59:56.547 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.547 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.547 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.547 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:56.547 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141965+0000", 2024-09-21T14:59:56.547 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:56.547 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.547 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.548 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:56.548 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141966+0000", 2024-09-21T14:59:56.548 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.46e-06 2024-09-21T14:59:56.548 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.548 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.548 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:56.548 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141972+0000", 2024-09-21T14:59:56.548 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4210000000000003e-06 2024-09-21T14:59:56.548 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.548 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.548 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:56.548 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141975+0000", 2024-09-21T14:59:56.548 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.948e-06 2024-09-21T14:59:56.548 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.549 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.549 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:56.549 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177113+0000", 2024-09-21T14:59:56.549 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035137855000000003 2024-09-21T14:59:56.549 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.549 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.549 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:56.549 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177121+0000", 2024-09-21T14:59:56.549 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8269999999999998e-06 2024-09-21T14:59:56.549 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.549 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:56.549 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.549 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.549 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.550 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45721 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:56.550 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141977+0000", 2024-09-21T14:59:56.550 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 46.349511984999999, 2024-09-21T14:59:56.550 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035169984000000001, 2024-09-21T14:59:56.550 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:56.550 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:56.550 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:56.550 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:56.550 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:56.550 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45721 2024-09-21T14:59:56.550 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.550 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:56.550 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.550 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:56.551 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141977+0000", 2024-09-21T14:59:56.551 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.551 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.551 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.551 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:56.551 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141977+0000", 2024-09-21T14:59:56.551 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.551 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.551 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.551 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:56.551 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141977+0000", 2024-09-21T14:59:56.551 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:56.551 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.551 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.552 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:56.552 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141979+0000", 2024-09-21T14:59:56.552 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.511e-06 2024-09-21T14:59:56.552 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.552 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.552 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:56.552 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141982+0000", 2024-09-21T14:59:56.552 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4180000000000001e-06 2024-09-21T14:59:56.552 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.552 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.552 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:56.552 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141985+0000", 2024-09-21T14:59:56.552 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9450000000000002e-06 2024-09-21T14:59:56.552 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.552 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.553 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:56.553 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177139+0000", 2024-09-21T14:59:56.553 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035154409999999997 2024-09-21T14:59:56.553 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.553 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.553 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:56.553 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177147+0000", 2024-09-21T14:59:56.553 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9570000000000002e-06 2024-09-21T14:59:56.553 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.553 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:56.553 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.553 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.553 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.553 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45722 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:56.554 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141987+0000", 2024-09-21T14:59:56.554 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 46.349502014000002, 2024-09-21T14:59:56.554 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035188492000000002, 2024-09-21T14:59:56.554 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:56.554 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:56.554 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:56.554 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:56.554 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:56.554 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45722 2024-09-21T14:59:56.554 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.554 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:56.554 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.554 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:56.555 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141987+0000", 2024-09-21T14:59:56.555 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.555 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.555 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.555 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:56.555 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141987+0000", 2024-09-21T14:59:56.555 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.555 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.555 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.555 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:56.555 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141987+0000", 2024-09-21T14:59:56.555 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:56.555 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.555 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.556 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:56.556 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141988+0000", 2024-09-21T14:59:56.556 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.463e-06 2024-09-21T14:59:56.556 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.556 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.556 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:56.556 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141994+0000", 2024-09-21T14:59:56.556 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.361e-06 2024-09-21T14:59:56.556 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.556 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.556 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:56.556 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141997+0000", 2024-09-21T14:59:56.556 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0359999999999998e-06 2024-09-21T14:59:56.556 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.557 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.557 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:56.557 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177169+0000", 2024-09-21T14:59:56.557 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035171651999999998 2024-09-21T14:59:56.557 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.557 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.557 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:56.557 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177176+0000", 2024-09-21T14:59:56.557 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2189999999999998e-06 2024-09-21T14:59:56.557 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.557 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:56.557 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.558 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.558 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.558 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45723 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:56.558 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141999+0000", 2024-09-21T14:59:56.558 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 46.349489847000001, 2024-09-21T14:59:56.558 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035197701999999997, 2024-09-21T14:59:56.558 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:56.558 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:56.558 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:56.558 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:56.558 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:56.558 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45723 2024-09-21T14:59:56.558 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.558 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:56.559 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.559 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:56.559 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141999+0000", 2024-09-21T14:59:56.559 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.559 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.559 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.559 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:56.559 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141999+0000", 2024-09-21T14:59:56.559 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.559 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.559 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.559 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:56.559 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141999+0000", 2024-09-21T14:59:56.559 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:56.560 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.560 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.560 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:56.560 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142003+0000", 2024-09-21T14:59:56.560 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5049999999999999e-06 2024-09-21T14:59:56.561 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.561 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.561 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:56.561 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142008+0000", 2024-09-21T14:59:56.561 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6579999999999999e-06 2024-09-21T14:59:56.561 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.562 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.562 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:56.562 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142011+0000", 2024-09-21T14:59:56.562 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0529999999999998e-06 2024-09-21T14:59:56.562 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.562 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.562 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:56.562 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177194+0000", 2024-09-21T14:59:56.562 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035182221999999999 2024-09-21T14:59:56.562 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.562 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.562 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:56.562 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177197+0000", 2024-09-21T14:59:56.562 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5030000000000002e-06 2024-09-21T14:59:56.563 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.563 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:56.563 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.563 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.563 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.563 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45724 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:56.563 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.142015+0000", 2024-09-21T14:59:56.563 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 46.349473879999998, 2024-09-21T14:59:56.563 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035195345000000003, 2024-09-21T14:59:56.563 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:56.563 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:56.563 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:56.564 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:56.564 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:56.564 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45724 2024-09-21T14:59:56.564 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.564 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:56.564 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.564 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:56.564 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142015+0000", 2024-09-21T14:59:56.564 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.564 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.564 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.564 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:56.564 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142015+0000", 2024-09-21T14:59:56.564 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.565 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.565 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.565 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:56.565 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142015+0000", 2024-09-21T14:59:56.565 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:56.565 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.565 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.565 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:56.565 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142017+0000", 2024-09-21T14:59:56.565 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5540000000000001e-06 2024-09-21T14:59:56.565 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.565 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.565 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:56.565 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142022+0000", 2024-09-21T14:59:56.565 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4229999999999996e-06 2024-09-21T14:59:56.566 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.566 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.566 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:56.566 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142025+0000", 2024-09-21T14:59:56.566 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.056e-06 2024-09-21T14:59:56.566 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.566 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.566 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:56.566 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177207+0000", 2024-09-21T14:59:56.566 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035182089 2024-09-21T14:59:56.566 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.566 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.566 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:56.566 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177211+0000", 2024-09-21T14:59:56.566 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4649999999999999e-06 2024-09-21T14:59:56.567 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.567 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:56.567 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.567 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.567 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.567 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45725 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:56.567 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.142028+0000", 2024-09-21T14:59:56.567 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 46.349461601000002, 2024-09-21T14:59:56.567 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035195731000000001, 2024-09-21T14:59:56.567 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:56.567 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:56.567 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:56.567 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:56.567 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:56.567 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45725 2024-09-21T14:59:56.568 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.568 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:56.568 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.568 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:56.568 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142028+0000", 2024-09-21T14:59:56.568 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.568 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.568 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.568 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:56.568 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142028+0000", 2024-09-21T14:59:56.568 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.568 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.568 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.568 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:56.568 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142027+0000", 2024-09-21T14:59:56.569 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:56.569 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.569 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.569 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:56.569 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142029+0000", 2024-09-21T14:59:56.569 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5269999999999999e-06 2024-09-21T14:59:56.569 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.569 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.569 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:56.569 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142032+0000", 2024-09-21T14:59:56.569 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3249999999999999e-06 2024-09-21T14:59:56.569 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.569 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.569 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:56.570 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142035+0000", 2024-09-21T14:59:56.570 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.897e-06 2024-09-21T14:59:56.570 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.570 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.570 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:56.570 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177220+0000", 2024-09-21T14:59:56.570 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035184871 2024-09-21T14:59:56.570 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.570 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.570 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:56.570 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177223+0000", 2024-09-21T14:59:56.570 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3680000000000002e-06 2024-09-21T14:59:56.570 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.570 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:56.570 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.571 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.571 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.571 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45726 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T14:59:56.571 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.142037+0000", 2024-09-21T14:59:56.571 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 46.349451885999997, 2024-09-21T14:59:56.571 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035199273000000003, 2024-09-21T14:59:56.571 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:56.571 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:56.571 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:56.571 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:56.571 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:56.571 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45726 2024-09-21T14:59:56.571 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.571 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:56.571 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.572 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:56.572 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142037+0000", 2024-09-21T14:59:56.572 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.572 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.572 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.572 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:56.572 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142037+0000", 2024-09-21T14:59:56.572 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.572 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.572 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.572 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:56.572 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142037+0000", 2024-09-21T14:59:56.572 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:56.572 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.572 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.572 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:56.573 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142038+0000", 2024-09-21T14:59:56.573 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.344e-06 2024-09-21T14:59:56.573 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.573 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.573 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:56.573 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142044+0000", 2024-09-21T14:59:56.573 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1930000000000001e-06 2024-09-21T14:59:56.573 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.573 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.573 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:56.573 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142047+0000", 2024-09-21T14:59:56.573 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0280000000000001e-06 2024-09-21T14:59:56.573 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.573 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.573 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:56.574 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177233+0000", 2024-09-21T14:59:56.574 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035186394000000003 2024-09-21T14:59:56.574 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.574 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.574 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:56.574 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177237+0000", 2024-09-21T14:59:56.574 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5530000000000001e-06 2024-09-21T14:59:56.574 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.574 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:56.574 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.574 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.574 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:56.574 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:56.575 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T14:59:56.576 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:59:56.574+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:59:56.676 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:59:56.670+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:59:56.688 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:56.688 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:59:56.688 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T14:59:56.688 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:56.698 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T14:59:56.777 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:59:56.770+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:59:56.810 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:56.811 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:59:56.811 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T14:59:56.811 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T14:59:56.811 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:56.821 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T14:59:56.878 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:59:56.877+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:59:56.928 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:56.928 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T14:59:56.928 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T14:59:56.928 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T14:59:56.928 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.928 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4281.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-21T14:59:56.928 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:09.324416+0000", 2024-09-21T14:59:56.928 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 107.602071528, 2024-09-21T14:59:56.928 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.117837947, 2024-09-21T14:59:56.928 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:56.928 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:56.928 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:56.929 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4281", 2024-09-21T14:59:56.929 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/1518520303", 2024-09-21T14:59:56.929 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1 2024-09-21T14:59:56.929 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.929 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:56.929 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.929 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:56.929 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324416+0000", 2024-09-21T14:59:56.929 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.929 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.929 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.929 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:56.929 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324416+0000", 2024-09-21T14:59:56.930 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.930 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.930 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.930 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:56.930 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324414+0000", 2024-09-21T14:59:56.930 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T14:59:56.930 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.930 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.930 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:56.930 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324429+0000", 2024-09-21T14:59:56.930 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5637e-05 2024-09-21T14:59:56.930 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.930 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.931 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:56.931 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324448+0000", 2024-09-21T14:59:56.931 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9042000000000002e-05 2024-09-21T14:59:56.931 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.931 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.931 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:56.931 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324465+0000", 2024-09-21T14:59:56.931 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6767999999999999e-05 2024-09-21T14:59:56.931 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.931 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.931 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:56.931 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324571+0000", 2024-09-21T14:59:56.931 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000105714 2024-09-21T14:59:56.932 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.932 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.932 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op must wait for map", 2024-09-21T14:59:56.932 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324587+0000", 2024-09-21T14:59:56.932 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5753999999999999e-05 2024-09-21T14:59:56.932 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.932 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.932 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:56.932 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.325864+0000", 2024-09-21T14:59:56.932 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0012773350000000001 2024-09-21T14:59:56.932 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.932 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.932 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T14:59:56.932 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.325878+0000", 2024-09-21T14:59:56.933 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4498000000000001e-05 2024-09-21T14:59:56.933 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.933 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.933 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:56.933 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.439461+0000", 2024-09-21T14:59:56.933 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.11358212500000001 2024-09-21T14:59:56.933 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.933 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.933 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:56.933 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.439848+0000", 2024-09-21T14:59:56.933 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00038747800000000002 2024-09-21T14:59:56.933 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.933 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.933 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:59:56.934 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.440175+0000", 2024-09-21T14:59:56.934 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00032721900000000001 2024-09-21T14:59:56.934 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.934 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.934 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:56.934 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.441770+0000", 2024-09-21T14:59:56.934 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0015949409999999999 2024-09-21T14:59:56.934 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.934 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.934 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:56.934 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.442086+0000", 2024-09-21T14:59:56.934 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00031586299999999999 2024-09-21T14:59:56.934 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.935 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.935 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:56.935 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.442195+0000", 2024-09-21T14:59:56.935 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000108791 2024-09-21T14:59:56.935 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.935 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.935 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:56.935 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.442254+0000", 2024-09-21T14:59:56.935 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9038e-05 2024-09-21T14:59:56.935 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.935 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:56.935 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.935 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.935 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.935 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4281.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-21T14:59:56.936 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:09.447828+0000", 2024-09-21T14:59:56.936 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 107.478659527, 2024-09-21T14:59:56.936 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.32904393700000001, 2024-09-21T14:59:56.936 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:56.936 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T14:59:56.936 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:56.936 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4281", 2024-09-21T14:59:56.936 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/1518520303", 2024-09-21T14:59:56.936 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 2 2024-09-21T14:59:56.936 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.936 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:56.936 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.936 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:56.936 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447828+0000", 2024-09-21T14:59:56.936 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.937 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.937 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.937 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:56.937 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447828+0000", 2024-09-21T14:59:56.937 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.937 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.937 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.937 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:56.937 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447825+0000", 2024-09-21T14:59:56.937 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T14:59:56.937 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.937 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.937 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:56.937 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447841+0000", 2024-09-21T14:59:56.937 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5359999999999999e-05 2024-09-21T14:59:56.938 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.938 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.938 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:56.938 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447875+0000", 2024-09-21T14:59:56.938 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4388999999999999e-05 2024-09-21T14:59:56.938 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.938 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.938 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:56.938 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447903+0000", 2024-09-21T14:59:56.938 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7373e-05 2024-09-21T14:59:56.938 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.938 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.938 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:56.938 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447996+0000", 2024-09-21T14:59:56.938 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2954999999999993e-05 2024-09-21T14:59:56.939 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.939 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.939 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T14:59:56.939 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.448011+0000", 2024-09-21T14:59:56.939 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5038999999999999e-05 2024-09-21T14:59:56.939 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.939 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.939 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:56.939 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.776501+0000", 2024-09-21T14:59:56.939 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.32849012500000002 2024-09-21T14:59:56.939 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.939 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.939 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:56.939 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.776872+0000", 2024-09-21T14:59:56.939 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00037119600000000002 2024-09-21T14:59:56.940 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.940 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:56.940 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.940 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.940 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.940 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:752 3.14 3:28fc02e5:::rbd_data.113693316f36.0000000000000010:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-21T14:59:56.940 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:26.625579+0000", 2024-09-21T14:59:56.940 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 90.300908672000006, 2024-09-21T14:59:56.940 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042965426000000001, 2024-09-21T14:59:56.940 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:56.940 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:56.940 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:56.940 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:56.940 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:56.940 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 752 2024-09-21T14:59:56.941 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.941 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:56.941 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.941 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:56.941 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625579+0000", 2024-09-21T14:59:56.941 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.941 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.941 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.941 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:56.941 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625579+0000", 2024-09-21T14:59:56.941 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.941 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.941 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.941 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:56.941 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625577+0000", 2024-09-21T14:59:56.941 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T14:59:56.942 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.942 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.942 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:56.942 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625588+0000", 2024-09-21T14:59:56.942 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1555e-05 2024-09-21T14:59:56.942 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.942 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.942 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:56.942 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625610+0000", 2024-09-21T14:59:56.942 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2107999999999999e-05 2024-09-21T14:59:56.942 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.942 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.942 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:56.942 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625621+0000", 2024-09-21T14:59:56.942 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0762e-05 2024-09-21T14:59:56.942 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.943 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.943 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:56.943 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625692+0000", 2024-09-21T14:59:56.943 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.1203000000000003e-05 2024-09-21T14:59:56.943 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.943 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.943 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:56.943 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625872+0000", 2024-09-21T14:59:56.943 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017962800000000001 2024-09-21T14:59:56.943 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.943 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.943 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T14:59:56.943 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.633188+0000", 2024-09-21T14:59:56.943 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.007316014 2024-09-21T14:59:56.944 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.944 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.944 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:56.944 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.668357+0000", 2024-09-21T14:59:56.944 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035168933999999999 2024-09-21T14:59:56.944 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.944 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.944 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:56.944 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.668418+0000", 2024-09-21T14:59:56.944 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1282e-05 2024-09-21T14:59:56.944 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.944 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.944 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:56.944 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.668486+0000", 2024-09-21T14:59:56.944 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8468999999999998e-05 2024-09-21T14:59:56.945 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.945 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.945 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:56.945 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.668544+0000", 2024-09-21T14:59:56.945 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7785000000000003e-05 2024-09-21T14:59:56.945 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.945 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:56.945 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.945 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.945 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.945 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22738 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 49152~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:56.945 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141049+0000", 2024-09-21T14:59:56.945 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 66.785438529000004, 2024-09-21T14:59:56.945 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035428754999999999, 2024-09-21T14:59:56.946 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:56.946 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:56.946 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:56.946 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:56.946 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:56.946 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22738 2024-09-21T14:59:56.946 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.946 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:56.946 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.946 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:56.946 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141049+0000", 2024-09-21T14:59:56.946 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.946 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.947 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.947 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:56.947 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141049+0000", 2024-09-21T14:59:56.947 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.947 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.947 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.947 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:56.947 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141049+0000", 2024-09-21T14:59:56.947 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:56.947 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.947 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.947 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:56.947 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141051+0000", 2024-09-21T14:59:56.947 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4980000000000001e-06 2024-09-21T14:59:56.947 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.948 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.948 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:56.948 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141057+0000", 2024-09-21T14:59:56.948 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8030000000000002e-06 2024-09-21T14:59:56.948 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.948 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.948 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:56.948 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141066+0000", 2024-09-21T14:59:56.948 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1160000000000005e-06 2024-09-21T14:59:56.948 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.948 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.948 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:56.948 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176264+0000", 2024-09-21T14:59:56.948 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035197580999999999 2024-09-21T14:59:56.949 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.949 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.949 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:56.949 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176327+0000", 2024-09-21T14:59:56.949 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3237000000000005e-05 2024-09-21T14:59:56.949 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.949 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.949 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:56.949 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176478+0000", 2024-09-21T14:59:56.949 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000150851 2024-09-21T14:59:56.949 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.949 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:56.949 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.949 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.950 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.950 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22739 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 114688~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:56.950 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141070+0000", 2024-09-21T14:59:56.950 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 66.785417287000001, 2024-09-21T14:59:56.950 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035679094000000001, 2024-09-21T14:59:56.950 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:56.950 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:56.950 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:56.950 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:56.950 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:56.950 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22739 2024-09-21T14:59:56.950 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.950 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:56.950 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.951 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:56.951 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141070+0000", 2024-09-21T14:59:56.951 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.951 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.951 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.951 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:56.951 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141070+0000", 2024-09-21T14:59:56.951 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.951 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.951 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.951 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:56.951 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141070+0000", 2024-09-21T14:59:56.951 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:56.951 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.952 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.952 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:56.952 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141072+0000", 2024-09-21T14:59:56.952 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.379e-06 2024-09-21T14:59:56.952 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.952 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.952 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:56.952 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141078+0000", 2024-09-21T14:59:56.952 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6710000000000004e-06 2024-09-21T14:59:56.952 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.952 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.952 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:56.952 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141083+0000", 2024-09-21T14:59:56.952 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7249999999999997e-06 2024-09-21T14:59:56.953 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.953 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.953 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:56.953 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176506+0000", 2024-09-21T14:59:56.953 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035423076999999997 2024-09-21T14:59:56.953 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.953 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.953 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:56.953 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176582+0000", 2024-09-21T14:59:56.953 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6305999999999998e-05 2024-09-21T14:59:56.953 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.953 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.953 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:56.954 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176749+0000", 2024-09-21T14:59:56.954 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000167262 2024-09-21T14:59:56.954 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.954 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:56.954 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.954 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.954 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.954 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22740 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 180224~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:56.954 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141090+0000", 2024-09-21T14:59:56.954 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 66.785397883000002, 2024-09-21T14:59:56.954 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036129535999999997, 2024-09-21T14:59:56.954 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:56.954 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:56.954 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:56.955 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:56.955 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:56.955 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22740 2024-09-21T14:59:56.955 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.955 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:56.955 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.955 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:56.955 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141090+0000", 2024-09-21T14:59:56.955 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.955 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.955 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.955 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:56.955 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141090+0000", 2024-09-21T14:59:56.955 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.956 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.956 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.956 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:56.956 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141089+0000", 2024-09-21T14:59:56.956 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:56.956 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.956 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.956 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:56.956 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141092+0000", 2024-09-21T14:59:56.956 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7709999999999998e-06 2024-09-21T14:59:56.956 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.956 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.956 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:56.956 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141098+0000", 2024-09-21T14:59:56.957 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.015e-06 2024-09-21T14:59:56.957 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.957 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.957 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:56.957 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141103+0000", 2024-09-21T14:59:56.957 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8629999999999999e-06 2024-09-21T14:59:56.957 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.957 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.957 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:56.957 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176778+0000", 2024-09-21T14:59:56.957 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035674984 2024-09-21T14:59:56.957 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.957 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.957 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:56.958 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176851+0000", 2024-09-21T14:59:56.958 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2861000000000006e-05 2024-09-21T14:59:56.958 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.958 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.958 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:56.958 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177219+0000", 2024-09-21T14:59:56.958 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00036845999999999998 2024-09-21T14:59:56.958 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.958 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:56.958 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.958 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.958 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.958 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22741 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 245760~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:56.958 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141107+0000", 2024-09-21T14:59:56.958 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 66.785380312000001, 2024-09-21T14:59:56.959 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036407729, 2024-09-21T14:59:56.959 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:56.959 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:56.959 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:56.959 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:56.959 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:56.959 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22741 2024-09-21T14:59:56.959 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.959 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:56.959 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.959 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:56.959 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141107+0000", 2024-09-21T14:59:56.959 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.959 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.959 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.959 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:56.960 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141107+0000", 2024-09-21T14:59:56.960 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.960 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.960 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.960 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:56.960 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141107+0000", 2024-09-21T14:59:56.960 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:56.960 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.960 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.960 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:56.960 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141109+0000", 2024-09-21T14:59:56.960 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6529999999999998e-06 2024-09-21T14:59:56.960 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.960 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.960 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:56.961 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141115+0000", 2024-09-21T14:59:56.961 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9649999999999996e-06 2024-09-21T14:59:56.961 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.961 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.961 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:56.961 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141121+0000", 2024-09-21T14:59:56.961 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1209999999999998e-06 2024-09-21T14:59:56.961 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.961 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.961 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:56.961 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177242+0000", 2024-09-21T14:59:56.961 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036121706000000003 2024-09-21T14:59:56.961 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.961 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.961 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:56.962 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177301+0000", 2024-09-21T14:59:56.962 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9008000000000002e-05 2024-09-21T14:59:56.962 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.962 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.962 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:56.962 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177515+0000", 2024-09-21T14:59:56.962 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00021362 2024-09-21T14:59:56.962 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.962 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:56.962 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.962 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.962 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.962 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22742 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 311296~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:56.962 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141126+0000", 2024-09-21T14:59:56.962 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 66.785361957999996, 2024-09-21T14:59:56.962 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036653537, 2024-09-21T14:59:56.963 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:56.963 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:56.963 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:56.963 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:56.963 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:56.963 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22742 2024-09-21T14:59:56.963 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.963 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:56.963 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.963 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:56.963 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141126+0000", 2024-09-21T14:59:56.963 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.963 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.963 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.964 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:56.964 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141126+0000", 2024-09-21T14:59:56.964 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.964 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.964 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.964 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:56.964 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141125+0000", 2024-09-21T14:59:56.964 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:56.964 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.964 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.964 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:56.964 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141128+0000", 2024-09-21T14:59:56.964 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.886e-06 2024-09-21T14:59:56.964 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.964 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.965 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:56.965 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141137+0000", 2024-09-21T14:59:56.965 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4399999999999994e-06 2024-09-21T14:59:56.965 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.965 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.965 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:56.965 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141143+0000", 2024-09-21T14:59:56.965 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1410000000000004e-06 2024-09-21T14:59:56.965 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.965 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.965 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:56.965 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177543+0000", 2024-09-21T14:59:56.965 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036400544999999999 2024-09-21T14:59:56.965 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.965 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.965 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:56.966 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177605+0000", 2024-09-21T14:59:56.966 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2206999999999996e-05 2024-09-21T14:59:56.966 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.966 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.966 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:56.966 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177779+0000", 2024-09-21T14:59:56.966 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017368099999999999 2024-09-21T14:59:56.966 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.966 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:56.966 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.966 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.966 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.966 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22743 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 376832~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:56.966 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141147+0000", 2024-09-21T14:59:56.966 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 66.785340473999995, 2024-09-21T14:59:56.967 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036915158000000003, 2024-09-21T14:59:56.967 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:56.967 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:56.967 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:56.967 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:56.967 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:56.967 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22743 2024-09-21T14:59:56.967 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.967 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:56.967 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.967 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:56.967 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141147+0000", 2024-09-21T14:59:56.967 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.967 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.967 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.967 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:56.968 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141147+0000", 2024-09-21T14:59:56.968 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.968 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.968 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.968 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:56.968 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141147+0000", 2024-09-21T14:59:56.968 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:56.968 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.968 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.968 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:56.968 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141149+0000", 2024-09-21T14:59:56.968 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.79e-06 2024-09-21T14:59:56.968 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.968 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.968 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:56.969 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141155+0000", 2024-09-21T14:59:56.969 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6699999999999999e-06 2024-09-21T14:59:56.969 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.969 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.969 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:56.969 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141160+0000", 2024-09-21T14:59:56.969 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5759999999999999e-06 2024-09-21T14:59:56.969 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.969 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.969 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:56.969 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177817+0000", 2024-09-21T14:59:56.969 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036657404999999997 2024-09-21T14:59:56.969 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.969 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.969 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:56.969 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177892+0000", 2024-09-21T14:59:56.970 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4602000000000002e-05 2024-09-21T14:59:56.970 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.970 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.970 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:56.970 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178062+0000", 2024-09-21T14:59:56.970 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000170464 2024-09-21T14:59:56.970 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.970 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:56.970 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.970 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.970 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.970 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22744 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 442368~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:56.970 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141164+0000", 2024-09-21T14:59:56.970 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 66.785323340999994, 2024-09-21T14:59:56.971 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037120806999999999, 2024-09-21T14:59:56.971 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:56.971 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:56.971 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:56.971 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:56.971 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:56.971 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22744 2024-09-21T14:59:56.971 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.971 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:56.971 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.971 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:56.971 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141164+0000", 2024-09-21T14:59:56.971 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.971 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.971 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.972 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:56.972 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141164+0000", 2024-09-21T14:59:56.972 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.972 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.972 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.972 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:56.972 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141164+0000", 2024-09-21T14:59:56.972 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:56.972 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.972 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.972 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:56.972 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141167+0000", 2024-09-21T14:59:56.972 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7159999999999999e-06 2024-09-21T14:59:56.972 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.972 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.972 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:56.972 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141173+0000", 2024-09-21T14:59:56.973 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.071e-06 2024-09-21T14:59:56.973 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.973 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.973 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:56.973 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141177+0000", 2024-09-21T14:59:56.973 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3610000000000003e-06 2024-09-21T14:59:56.973 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.973 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.973 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:56.973 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178083+0000", 2024-09-21T14:59:56.973 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036906405000000003 2024-09-21T14:59:56.973 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.973 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.973 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:56.974 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178143+0000", 2024-09-21T14:59:56.974 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0056999999999998e-05 2024-09-21T14:59:56.974 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.974 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.974 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:56.974 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178285+0000", 2024-09-21T14:59:56.974 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000141555 2024-09-21T14:59:56.974 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.974 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:56.974 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.974 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.974 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.974 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22745 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 507904~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:56.974 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141182+0000", 2024-09-21T14:59:56.974 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 66.785305020999999, 2024-09-21T14:59:56.975 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037332331000000003, 2024-09-21T14:59:56.975 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:56.975 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:56.975 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:56.975 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:56.975 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:56.975 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22745 2024-09-21T14:59:56.975 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.975 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:56.975 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.975 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:56.975 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141182+0000", 2024-09-21T14:59:56.975 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.975 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.975 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.975 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:56.976 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141182+0000", 2024-09-21T14:59:56.976 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.976 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.976 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.976 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:56.976 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141182+0000", 2024-09-21T14:59:56.976 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:56.976 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.976 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.976 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:56.976 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141185+0000", 2024-09-21T14:59:56.976 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.464e-06 2024-09-21T14:59:56.976 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.976 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.976 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:56.976 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141200+0000", 2024-09-21T14:59:56.977 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5886999999999999e-05 2024-09-21T14:59:56.977 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.977 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.977 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:56.977 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141571+0000", 2024-09-21T14:59:56.977 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000370909 2024-09-21T14:59:56.977 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.977 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.977 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:56.977 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178318+0000", 2024-09-21T14:59:56.977 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036746244999999997 2024-09-21T14:59:56.977 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.977 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.977 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:56.977 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178367+0000", 2024-09-21T14:59:56.977 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9317000000000002e-05 2024-09-21T14:59:56.978 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.978 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.978 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:56.978 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178515+0000", 2024-09-21T14:59:56.978 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014784600000000001 2024-09-21T14:59:56.978 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.978 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:56.978 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.978 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.978 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.978 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22746 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 573440~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:56.978 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141599+0000", 2024-09-21T14:59:56.978 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 66.784888389000002, 2024-09-21T14:59:56.978 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037402249999999998, 2024-09-21T14:59:56.979 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:56.979 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:56.979 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:56.979 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:56.979 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:56.979 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22746 2024-09-21T14:59:56.979 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.979 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:56.979 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.979 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:56.979 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141599+0000", 2024-09-21T14:59:56.979 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.979 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.979 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.979 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:56.979 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141599+0000", 2024-09-21T14:59:56.980 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.980 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.980 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.980 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:56.980 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141598+0000", 2024-09-21T14:59:56.980 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T14:59:56.980 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.980 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.980 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:56.980 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141637+0000", 2024-09-21T14:59:56.980 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.9119999999999998e-05 2024-09-21T14:59:56.980 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.980 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.980 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:56.980 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141647+0000", 2024-09-21T14:59:56.981 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5570000000000001e-06 2024-09-21T14:59:56.981 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.981 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.981 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:56.981 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141654+0000", 2024-09-21T14:59:56.981 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2049999999999996e-06 2024-09-21T14:59:56.981 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.981 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.981 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:56.981 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178553+0000", 2024-09-21T14:59:56.981 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036898528 2024-09-21T14:59:56.981 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.981 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.981 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:56.981 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178625+0000", 2024-09-21T14:59:56.982 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2669999999999994e-05 2024-09-21T14:59:56.982 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.982 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.982 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:56.982 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179001+0000", 2024-09-21T14:59:56.982 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000375977 2024-09-21T14:59:56.982 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.982 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:56.982 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.982 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.982 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.982 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22747 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 638976~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:56.982 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141660+0000", 2024-09-21T14:59:56.982 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 66.784827246999996, 2024-09-21T14:59:56.982 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037617352999999999, 2024-09-21T14:59:56.983 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:56.983 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:56.983 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:56.983 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:56.983 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:56.983 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22747 2024-09-21T14:59:56.983 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.983 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:56.983 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.983 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:56.983 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141660+0000", 2024-09-21T14:59:56.983 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.983 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.983 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.984 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:56.984 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141660+0000", 2024-09-21T14:59:56.984 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.984 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.984 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.984 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:56.984 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141660+0000", 2024-09-21T14:59:56.984 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:56.984 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.984 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.984 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:56.984 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141663+0000", 2024-09-21T14:59:56.984 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.569e-06 2024-09-21T14:59:56.984 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.985 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.985 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:56.985 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141670+0000", 2024-09-21T14:59:56.985 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5719999999999999e-06 2024-09-21T14:59:56.985 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.985 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.985 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:56.985 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141675+0000", 2024-09-21T14:59:56.985 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2889999999999997e-06 2024-09-21T14:59:56.985 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.985 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.985 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:56.985 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179028+0000", 2024-09-21T14:59:56.985 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037353381999999997 2024-09-21T14:59:56.985 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.986 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.986 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:56.986 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179099+0000", 2024-09-21T14:59:56.986 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0259000000000006e-05 2024-09-21T14:59:56.986 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.986 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.986 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:56.986 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179278+0000", 2024-09-21T14:59:56.986 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000178908 2024-09-21T14:59:56.986 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.986 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:56.986 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.986 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.986 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.986 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22748 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 704512~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:56.987 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141680+0000", 2024-09-21T14:59:56.987 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 66.784807608999998, 2024-09-21T14:59:56.987 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037806166000000002, 2024-09-21T14:59:56.987 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:56.987 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:56.987 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:56.987 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:56.987 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:56.987 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22748 2024-09-21T14:59:56.987 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.987 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:56.987 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.987 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:56.987 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141680+0000", 2024-09-21T14:59:56.987 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.987 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.988 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.988 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:56.988 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141680+0000", 2024-09-21T14:59:56.988 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.988 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.988 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.988 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:56.988 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141679+0000", 2024-09-21T14:59:56.988 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:56.988 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.988 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.988 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:56.988 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141683+0000", 2024-09-21T14:59:56.988 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3230000000000002e-06 2024-09-21T14:59:56.988 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.988 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.989 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:56.989 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141690+0000", 2024-09-21T14:59:56.989 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8680000000000001e-06 2024-09-21T14:59:56.989 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.989 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.989 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:56.989 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141694+0000", 2024-09-21T14:59:56.989 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6770000000000003e-06 2024-09-21T14:59:56.989 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.989 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.989 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:56.989 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179300+0000", 2024-09-21T14:59:56.989 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037605567999999999 2024-09-21T14:59:56.989 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.989 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.989 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:56.990 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179355+0000", 2024-09-21T14:59:56.990 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4771000000000001e-05 2024-09-21T14:59:56.990 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.990 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.990 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:56.990 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179486+0000", 2024-09-21T14:59:56.990 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013141699999999999 2024-09-21T14:59:56.990 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.990 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:56.990 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.990 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.990 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.990 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22749 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 770048~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:56.990 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141698+0000", 2024-09-21T14:59:56.990 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 66.784789236999998, 2024-09-21T14:59:56.991 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037967813000000003, 2024-09-21T14:59:56.991 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:56.991 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:56.991 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:56.991 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:56.991 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:56.991 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22749 2024-09-21T14:59:56.991 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.991 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:56.991 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.991 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:56.991 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141698+0000", 2024-09-21T14:59:56.991 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.991 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.991 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.991 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:56.992 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141698+0000", 2024-09-21T14:59:56.992 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.992 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.992 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.992 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:56.992 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141698+0000", 2024-09-21T14:59:56.992 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:56.992 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.992 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.992 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:56.992 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141701+0000", 2024-09-21T14:59:56.992 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8399999999999999e-06 2024-09-21T14:59:56.992 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.992 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.992 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:56.992 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141706+0000", 2024-09-21T14:59:56.992 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4979999999999997e-06 2024-09-21T14:59:56.993 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.993 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.993 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:56.993 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141711+0000", 2024-09-21T14:59:56.993 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3850000000000004e-06 2024-09-21T14:59:56.993 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.993 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.993 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:56.993 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179504+0000", 2024-09-21T14:59:56.993 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037793514 2024-09-21T14:59:56.993 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.993 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.993 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:56.993 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179546+0000", 2024-09-21T14:59:56.994 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1467999999999997e-05 2024-09-21T14:59:56.994 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.994 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.994 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:56.994 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179666+0000", 2024-09-21T14:59:56.994 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000120499 2024-09-21T14:59:56.994 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.994 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:56.994 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.994 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.994 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.994 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22750 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 835584~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:56.994 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141715+0000", 2024-09-21T14:59:56.994 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 66.784772935999996, 2024-09-21T14:59:56.995 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038173179000000002, 2024-09-21T14:59:56.995 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:56.995 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:56.995 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:56.995 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:56.995 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:56.995 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22750 2024-09-21T14:59:56.995 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.995 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:56.995 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.995 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:56.995 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141715+0000", 2024-09-21T14:59:56.995 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.995 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.995 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.995 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:56.996 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141715+0000", 2024-09-21T14:59:56.996 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.996 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.996 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.996 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:56.996 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141714+0000", 2024-09-21T14:59:56.996 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:56.996 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.996 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.996 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:56.996 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141716+0000", 2024-09-21T14:59:56.996 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1730000000000002e-06 2024-09-21T14:59:56.996 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.996 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.996 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:56.997 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141722+0000", 2024-09-21T14:59:56.997 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.417e-06 2024-09-21T14:59:56.997 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.997 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.997 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:56.997 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141726+0000", 2024-09-21T14:59:56.997 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4830000000000001e-06 2024-09-21T14:59:56.997 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.997 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.997 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:56.997 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179683+0000", 2024-09-21T14:59:56.997 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037956977000000003 2024-09-21T14:59:56.997 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.997 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.997 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:56.997 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179751+0000", 2024-09-21T14:59:56.998 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7762000000000006e-05 2024-09-21T14:59:56.998 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.998 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.998 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:56.998 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179888+0000", 2024-09-21T14:59:56.998 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000136635 2024-09-21T14:59:56.998 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.998 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:56.998 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:56.998 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.998 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.998 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22751 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 901120~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:56.998 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141730+0000", 2024-09-21T14:59:56.998 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 66.784757137, 2024-09-21T14:59:56.998 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038357011000000003, 2024-09-21T14:59:56.999 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:56.999 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:56.999 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:56.999 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:56.999 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:56.999 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22751 2024-09-21T14:59:56.999 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.999 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:56.999 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.999 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:56.999 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141730+0000", 2024-09-21T14:59:56.999 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:56.999 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:56.999 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:56.999 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:56.999 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141730+0000", 2024-09-21T14:59:56.999 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:57.000 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:57.000 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:57.000 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:57.000 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141730+0000", 2024-09-21T14:59:57.000 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:57.000 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:57.000 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:57.000 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:57.000 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141737+0000", 2024-09-21T14:59:57.000 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.64e-06 2024-09-21T14:59:57.000 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:57.000 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:57.000 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:57.000 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141743+0000", 2024-09-21T14:59:57.000 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8130000000000001e-06 2024-09-21T14:59:57.001 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:57.001 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:57.001 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:57.001 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141747+0000", 2024-09-21T14:59:57.001 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4920000000000004e-06 2024-09-21T14:59:57.001 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:57.001 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:57.001 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:57.001 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179918+0000", 2024-09-21T14:59:57.001 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038170822 2024-09-21T14:59:57.001 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:57.001 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:57.001 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:57.001 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179960+0000", 2024-09-21T14:59:57.001 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1999000000000002e-05 2024-09-21T14:59:57.001 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:57.002 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:57.002 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:57.002 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180087+0000", 2024-09-21T14:59:57.002 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012752799999999999 2024-09-21T14:59:57.002 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:57.002 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:57.002 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:57.002 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:57.002 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:57.002 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22752 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 966656~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:57.002 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141751+0000", 2024-09-21T14:59:57.002 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 66.784736816999995, 2024-09-21T14:59:57.002 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038553084000000001, 2024-09-21T14:59:57.002 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:57.002 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:57.002 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:57.003 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:57.003 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:57.003 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22752 2024-09-21T14:59:57.003 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:57.003 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:57.003 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:57.003 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:57.003 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141751+0000", 2024-09-21T14:59:57.003 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:57.003 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:57.003 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:57.003 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:57.003 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141751+0000", 2024-09-21T14:59:57.003 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:57.003 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:57.004 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:57.004 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:57.004 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141750+0000", 2024-09-21T14:59:57.004 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:57.004 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:57.004 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:57.004 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:57.004 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141753+0000", 2024-09-21T14:59:57.004 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5900000000000002e-06 2024-09-21T14:59:57.004 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:57.004 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:57.004 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:57.004 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141759+0000", 2024-09-21T14:59:57.004 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.947e-06 2024-09-21T14:59:57.004 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:57.005 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:57.005 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:57.005 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141764+0000", 2024-09-21T14:59:57.005 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8319999999999996e-06 2024-09-21T14:59:57.005 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:57.005 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:57.005 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:57.005 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180108+0000", 2024-09-21T14:59:57.005 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038344095000000002 2024-09-21T14:59:57.005 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:57.005 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:57.005 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:57.005 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180168+0000", 2024-09-21T14:59:57.005 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0047000000000003e-05 2024-09-21T14:59:57.005 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:57.005 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:57.006 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:57.006 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180304+0000", 2024-09-21T14:59:57.006 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013592500000000001 2024-09-21T14:59:57.006 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:57.006 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:57.006 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:57.006 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:57.006 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:57.006 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22753 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 1032192~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:57.006 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141768+0000", 2024-09-21T14:59:57.006 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 66.784719942999999, 2024-09-21T14:59:57.006 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038818810000000002, 2024-09-21T14:59:57.006 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:57.006 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T14:59:57.006 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:57.006 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:57.007 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:57.007 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22753 2024-09-21T14:59:57.007 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:57.007 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:57.007 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:57.007 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:57.007 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141768+0000", 2024-09-21T14:59:57.007 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:57.007 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:57.007 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:57.007 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:57.007 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141768+0000", 2024-09-21T14:59:57.007 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:57.007 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:57.007 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:57.008 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:57.008 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141767+0000", 2024-09-21T14:59:57.008 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:57.008 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:57.008 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:57.008 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:57.008 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141770+0000", 2024-09-21T14:59:57.008 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3329999999999999e-06 2024-09-21T14:59:57.008 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:57.008 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:57.008 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:57.008 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141775+0000", 2024-09-21T14:59:57.008 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2669999999999997e-06 2024-09-21T14:59:57.008 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:57.008 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:57.008 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:57.008 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141779+0000", 2024-09-21T14:59:57.009 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2259999999999999e-06 2024-09-21T14:59:57.009 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:57.009 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:57.009 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:57.009 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180325+0000", 2024-09-21T14:59:57.009 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038546428000000001 2024-09-21T14:59:57.009 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:57.009 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:57.009 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:57.009 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180401+0000", 2024-09-21T14:59:57.009 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5437000000000003e-05 2024-09-21T14:59:57.009 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:57.009 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:57.009 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:57.009 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180586+0000", 2024-09-21T14:59:57.009 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000185429 2024-09-21T14:59:57.010 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:57.010 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:57.010 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:57.010 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:57.010 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:57.010 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:73793 3.11 3:8c8a112d:::rbd_data.1136f4939602.0000000000000003:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-21T14:59:57.010 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:42.225633+0000", 2024-09-21T14:59:57.010 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.700854029, 2024-09-21T14:59:57.010 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036337102000000003, 2024-09-21T14:59:57.010 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T14:59:57.010 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:57.010 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T14:59:57.010 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T14:59:57.010 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:57.010 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 73793 2024-09-21T14:59:57.011 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:57.011 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T14:59:57.011 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:57.011 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T14:59:57.011 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.225633+0000", 2024-09-21T14:59:57.011 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:57.011 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:57.011 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:57.011 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T14:59:57.011 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.225633+0000", 2024-09-21T14:59:57.011 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T14:59:57.011 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:57.011 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:57.011 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T14:59:57.011 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.225630+0000", 2024-09-21T14:59:57.011 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999967 2024-09-21T14:59:57.011 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:57.012 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:57.012 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T14:59:57.012 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.225649+0000", 2024-09-21T14:59:57.012 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9377000000000001e-05 2024-09-21T14:59:57.012 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:57.012 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:57.012 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T14:59:57.012 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.225689+0000", 2024-09-21T14:59:57.012 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.9879999999999998e-05 2024-09-21T14:59:57.012 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:57.012 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:57.012 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T14:59:57.012 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.225706+0000", 2024-09-21T14:59:57.012 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6537000000000001e-05 2024-09-21T14:59:57.012 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:57.012 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:57.013 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T14:59:57.013 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.225772+0000", 2024-09-21T14:59:57.013 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5894000000000003e-05 2024-09-21T14:59:57.013 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:57.013 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:57.013 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T14:59:57.013 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.225846+0000", 2024-09-21T14:59:57.013 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4165000000000006e-05 2024-09-21T14:59:57.013 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:57.013 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:57.013 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 1", 2024-09-21T14:59:57.013 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.231400+0000", 2024-09-21T14:59:57.013 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0055538269999999999 2024-09-21T14:59:57.013 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:57.013 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:57.013 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:57.014 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.261709+0000", 2024-09-21T14:59:57.014 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.030309750999999999 2024-09-21T14:59:57.014 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:57.014 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:57.014 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T14:59:57.014 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.261738+0000", 2024-09-21T14:59:57.014 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8569999999999999e-05 2024-09-21T14:59:57.014 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:57.014 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:57.014 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T14:59:57.014 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.261839+0000", 2024-09-21T14:59:57.014 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010118 2024-09-21T14:59:57.014 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T14:59:57.014 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T14:59:57.015 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T14:59:57.015 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.261971+0000", 2024-09-21T14:59:57.015 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000131414 2024-09-21T14:59:57.015 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:57.015 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:57.015 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:57.015 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T14:59:57.015 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T14:59:57.015 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:57.016 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T14:59:57.017 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:59:57.017+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:59:57.117 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:59:57.117+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:59:57.135 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:59:57.135 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:59:57.135 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T14:59:57.135 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:59:57.147 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T14:59:57.218 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:59:57.217+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:59:57.247 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:59:57.248 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T14:59:57.248 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T14:59:57.248 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T14:59:57.248 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:59:57.257 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T14:59:57.302 INFO:teuthology.orchestra.run.smithi032.stdout:checking clone #20, image image_client.0-clone20 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent21 2024-09-21T14:59:57.318 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:59:57.314+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:59:57.345 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T14:59:57.364 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:59:57.364 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T14:59:57.365 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T14:59:57.365 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T14:59:57.365 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.365 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:1452 3.19 3:9d40fe72:::rbd_data.1136515ecb6b.0000000000000013:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T14:59:57.365 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:30.292755+0000", 2024-09-21T14:59:57.365 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 87.068343963000004, 2024-09-21T14:59:57.365 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043852191999999998, 2024-09-21T14:59:57.365 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:57.365 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:57.365 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:57.365 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:57.365 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:57.365 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1452 2024-09-21T14:59:57.365 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.366 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:57.366 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.366 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:57.366 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292755+0000", 2024-09-21T14:59:57.366 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:57.366 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.366 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.366 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:57.366 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292755+0000", 2024-09-21T14:59:57.366 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:57.366 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.366 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.366 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:57.366 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292754+0000", 2024-09-21T14:59:57.366 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:57.367 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.367 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.367 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:57.367 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292760+0000", 2024-09-21T14:59:57.367 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5029999999999997e-06 2024-09-21T14:59:57.367 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.367 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.367 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:57.367 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292771+0000", 2024-09-21T14:59:57.367 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1551e-05 2024-09-21T14:59:57.367 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.367 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.367 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:57.367 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292779+0000", 2024-09-21T14:59:57.367 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1559999999999998e-06 2024-09-21T14:59:57.368 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.368 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.368 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:57.368 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292896+0000", 2024-09-21T14:59:57.368 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000117304 2024-09-21T14:59:57.368 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.368 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.368 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:57.368 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.293124+0000", 2024-09-21T14:59:57.368 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00022819999999999999 2024-09-21T14:59:57.368 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.368 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.368 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 1", 2024-09-21T14:59:57.368 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.299967+0000", 2024-09-21T14:59:57.368 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0068427009999999996 2024-09-21T14:59:57.368 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.368 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.369 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:57.369 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336463+0000", 2024-09-21T14:59:57.369 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.036495842000000001 2024-09-21T14:59:57.369 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.369 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.369 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:59:57.369 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336480+0000", 2024-09-21T14:59:57.369 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7419999999999999e-05 2024-09-21T14:59:57.369 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.369 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.369 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:59:57.369 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336550+0000", 2024-09-21T14:59:57.369 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9951999999999997e-05 2024-09-21T14:59:57.369 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.370 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.370 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:57.370 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336607+0000", 2024-09-21T14:59:57.370 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7423000000000002e-05 2024-09-21T14:59:57.370 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:57.370 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:57.370 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:57.370 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.370 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.370 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:1567 3.8 3:139d3d55:::rbd_data.1136515ecb6b.0000000000000000:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T14:59:57.370 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:30.681087+0000", 2024-09-21T14:59:57.370 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 86.680011747999998, 2024-09-21T14:59:57.370 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.039009224000000002, 2024-09-21T14:59:57.370 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:57.370 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:57.370 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:57.371 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:57.371 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:57.371 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1567 2024-09-21T14:59:57.371 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.371 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:57.371 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.371 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:57.371 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681087+0000", 2024-09-21T14:59:57.371 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:57.371 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.371 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.371 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:57.371 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681087+0000", 2024-09-21T14:59:57.371 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:57.371 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.371 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.372 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:57.372 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681086+0000", 2024-09-21T14:59:57.372 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T14:59:57.372 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.372 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.372 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:57.372 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681097+0000", 2024-09-21T14:59:57.372 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.134e-05 2024-09-21T14:59:57.372 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.372 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.372 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:57.372 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681122+0000", 2024-09-21T14:59:57.372 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5313e-05 2024-09-21T14:59:57.372 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.372 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.373 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:57.373 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681138+0000", 2024-09-21T14:59:57.373 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5916000000000001e-05 2024-09-21T14:59:57.373 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.373 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.373 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:57.373 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681270+0000", 2024-09-21T14:59:57.373 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000131474 2024-09-21T14:59:57.373 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.373 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.373 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:57.373 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681416+0000", 2024-09-21T14:59:57.373 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014637 2024-09-21T14:59:57.373 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.373 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.373 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 7", 2024-09-21T14:59:57.374 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.693163+0000", 2024-09-21T14:59:57.374 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011747241 2024-09-21T14:59:57.374 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.374 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.374 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:59:57.374 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.719750+0000", 2024-09-21T14:59:57.374 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.026586852000000001 2024-09-21T14:59:57.374 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.374 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.374 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:57.374 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.719859+0000", 2024-09-21T14:59:57.374 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010925999999999999 2024-09-21T14:59:57.374 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.374 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.375 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:59:57.375 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.719951+0000", 2024-09-21T14:59:57.375 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1294999999999999e-05 2024-09-21T14:59:57.375 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.375 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.375 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:57.375 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.720097+0000", 2024-09-21T14:59:57.375 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014583 2024-09-21T14:59:57.375 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:57.375 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:57.375 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:57.375 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.375 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.375 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9349 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 155648~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:57.375 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158403+0000", 2024-09-21T14:59:57.375 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 78.202696333000006, 2024-09-21T14:59:57.376 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014229851, 2024-09-21T14:59:57.376 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:57.376 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:57.376 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:57.376 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:57.376 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:57.376 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9349 2024-09-21T14:59:57.376 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.376 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:57.376 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.376 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:57.376 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158403+0000", 2024-09-21T14:59:57.376 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:57.376 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.376 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.377 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:57.377 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158403+0000", 2024-09-21T14:59:57.377 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:57.377 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.377 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.377 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:57.377 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158402+0000", 2024-09-21T14:59:57.377 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:57.377 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.377 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.377 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:57.377 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158406+0000", 2024-09-21T14:59:57.377 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1339999999999997e-06 2024-09-21T14:59:57.377 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.377 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.377 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:57.378 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158416+0000", 2024-09-21T14:59:57.378 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3149999999999998e-06 2024-09-21T14:59:57.378 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.378 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.378 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:57.378 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158439+0000", 2024-09-21T14:59:57.378 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3255999999999999e-05 2024-09-21T14:59:57.378 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.378 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.378 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:57.378 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172305+0000", 2024-09-21T14:59:57.378 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013866603999999999 2024-09-21T14:59:57.378 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.378 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.378 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:57.378 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172413+0000", 2024-09-21T14:59:57.379 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000107664 2024-09-21T14:59:57.379 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.379 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.379 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:57.379 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172633+0000", 2024-09-21T14:59:57.379 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021950800000000001 2024-09-21T14:59:57.379 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:57.379 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:57.379 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:57.379 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.379 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.379 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9350 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 294912~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:57.379 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158448+0000", 2024-09-21T14:59:57.379 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 78.202651462000006, 2024-09-21T14:59:57.379 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014471525000000001, 2024-09-21T14:59:57.380 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:57.380 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:57.380 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:57.380 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:57.380 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:57.380 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9350 2024-09-21T14:59:57.380 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.380 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:57.380 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.380 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:57.380 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158448+0000", 2024-09-21T14:59:57.380 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:57.380 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.380 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.380 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:57.380 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158448+0000", 2024-09-21T14:59:57.381 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:57.381 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.381 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.381 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:57.381 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158447+0000", 2024-09-21T14:59:57.381 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:57.381 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.381 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.381 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:57.381 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158452+0000", 2024-09-21T14:59:57.381 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8400000000000002e-06 2024-09-21T14:59:57.381 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.381 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.381 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:57.382 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158475+0000", 2024-09-21T14:59:57.382 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3091e-05 2024-09-21T14:59:57.382 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.382 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.382 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:57.382 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158515+0000", 2024-09-21T14:59:57.382 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0265000000000002e-05 2024-09-21T14:59:57.382 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.382 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.382 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:57.382 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172682+0000", 2024-09-21T14:59:57.382 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014166761999999999 2024-09-21T14:59:57.382 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.382 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.382 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:57.383 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172752+0000", 2024-09-21T14:59:57.383 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9698999999999998e-05 2024-09-21T14:59:57.383 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.383 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.383 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:57.383 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172919+0000", 2024-09-21T14:59:57.383 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016759199999999999 2024-09-21T14:59:57.383 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:57.383 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:57.383 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:57.383 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.383 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.383 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9351 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 434176~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:57.383 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158526+0000", 2024-09-21T14:59:57.383 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 78.202572743999994, 2024-09-21T14:59:57.384 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014662107000000001, 2024-09-21T14:59:57.384 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:57.384 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:57.384 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:57.384 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:57.384 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:57.384 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9351 2024-09-21T14:59:57.384 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.384 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:57.384 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.384 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:57.384 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158526+0000", 2024-09-21T14:59:57.384 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:57.385 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.385 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.385 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:57.385 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158526+0000", 2024-09-21T14:59:57.385 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:57.385 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.385 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.385 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:57.385 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158525+0000", 2024-09-21T14:59:57.385 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:57.385 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.385 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.385 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:57.385 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158531+0000", 2024-09-21T14:59:57.385 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5840000000000002e-06 2024-09-21T14:59:57.385 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.386 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.386 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:57.386 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158564+0000", 2024-09-21T14:59:57.386 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3195000000000003e-05 2024-09-21T14:59:57.386 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.386 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.386 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:57.386 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158577+0000", 2024-09-21T14:59:57.386 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2330000000000001e-05 2024-09-21T14:59:57.386 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.386 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.386 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:57.386 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172956+0000", 2024-09-21T14:59:57.386 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014379438 2024-09-21T14:59:57.387 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.387 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.387 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:57.387 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173020+0000", 2024-09-21T14:59:57.387 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4401999999999998e-05 2024-09-21T14:59:57.387 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.387 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.387 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:57.387 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173188+0000", 2024-09-21T14:59:57.387 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016804 2024-09-21T14:59:57.387 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:57.387 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:57.387 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:57.387 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.387 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.388 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9379 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 24576~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:57.388 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158586+0000", 2024-09-21T14:59:57.388 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 78.202513483999994, 2024-09-21T14:59:57.388 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014865173000000001, 2024-09-21T14:59:57.388 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:57.388 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:57.388 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:57.388 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:57.388 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:57.388 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9379 2024-09-21T14:59:57.388 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.388 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:57.388 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.388 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:57.388 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158586+0000", 2024-09-21T14:59:57.389 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:57.389 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.389 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.389 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:57.389 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158586+0000", 2024-09-21T14:59:57.389 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:57.389 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.389 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.389 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:57.389 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158585+0000", 2024-09-21T14:59:57.389 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:57.389 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.389 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.389 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:57.389 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158590+0000", 2024-09-21T14:59:57.389 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5820000000000001e-06 2024-09-21T14:59:57.390 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.390 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.390 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:57.390 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158676+0000", 2024-09-21T14:59:57.390 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5075999999999997e-05 2024-09-21T14:59:57.390 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.390 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.390 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:57.390 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158695+0000", 2024-09-21T14:59:57.390 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9137999999999999e-05 2024-09-21T14:59:57.390 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.390 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.390 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:57.390 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173219+0000", 2024-09-21T14:59:57.391 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014524034999999999 2024-09-21T14:59:57.391 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.391 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.391 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:57.391 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173276+0000", 2024-09-21T14:59:57.391 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7442000000000003e-05 2024-09-21T14:59:57.391 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.391 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.391 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:57.391 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173451+0000", 2024-09-21T14:59:57.391 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017464800000000001 2024-09-21T14:59:57.391 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:57.391 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:57.391 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:57.391 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.391 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.391 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9380 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 163840~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:57.392 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158706+0000", 2024-09-21T14:59:57.392 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 78.202392974000006, 2024-09-21T14:59:57.392 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015096381000000001, 2024-09-21T14:59:57.392 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:57.392 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:57.392 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:57.392 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:57.392 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:57.392 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9380 2024-09-21T14:59:57.392 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.392 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:57.392 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.392 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:57.392 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158706+0000", 2024-09-21T14:59:57.393 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:57.393 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.393 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.393 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:57.393 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158706+0000", 2024-09-21T14:59:57.393 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:57.393 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.393 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.393 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:57.393 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158705+0000", 2024-09-21T14:59:57.393 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:57.393 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.393 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.393 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:57.393 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158712+0000", 2024-09-21T14:59:57.394 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7240000000000003e-06 2024-09-21T14:59:57.394 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.394 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.394 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:57.394 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158755+0000", 2024-09-21T14:59:57.394 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2953e-05 2024-09-21T14:59:57.394 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.394 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.394 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:57.394 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158765+0000", 2024-09-21T14:59:57.394 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0149000000000001e-05 2024-09-21T14:59:57.394 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.394 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.394 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:57.395 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173492+0000", 2024-09-21T14:59:57.395 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014727105000000001 2024-09-21T14:59:57.395 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.395 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.395 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:57.395 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173564+0000", 2024-09-21T14:59:57.395 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1789e-05 2024-09-21T14:59:57.395 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.395 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.395 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:57.395 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173803+0000", 2024-09-21T14:59:57.395 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00023867100000000001 2024-09-21T14:59:57.395 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:57.395 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:57.395 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:57.396 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.396 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.396 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9381 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 303104~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:57.396 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158774+0000", 2024-09-21T14:59:57.396 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 78.202324894, 2024-09-21T14:59:57.396 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015309095999999999, 2024-09-21T14:59:57.396 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:57.396 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:57.396 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:57.396 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:57.396 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:57.396 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9381 2024-09-21T14:59:57.396 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.396 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:57.397 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.397 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:57.397 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158774+0000", 2024-09-21T14:59:57.397 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:57.397 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.397 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.397 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:57.397 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158774+0000", 2024-09-21T14:59:57.397 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:57.397 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.397 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.397 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:57.397 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158773+0000", 2024-09-21T14:59:57.397 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T14:59:57.397 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.397 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.398 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:57.398 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158779+0000", 2024-09-21T14:59:57.398 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6999999999999996e-06 2024-09-21T14:59:57.398 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.398 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.398 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:57.398 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158790+0000", 2024-09-21T14:59:57.398 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0594000000000001e-05 2024-09-21T14:59:57.398 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.398 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.398 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:57.398 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158798+0000", 2024-09-21T14:59:57.398 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5860000000000005e-06 2024-09-21T14:59:57.398 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.398 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.398 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:57.399 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173829+0000", 2024-09-21T14:59:57.399 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015030349 2024-09-21T14:59:57.399 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.399 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.399 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:57.399 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173918+0000", 2024-09-21T14:59:57.399 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9375000000000004e-05 2024-09-21T14:59:57.399 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.399 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.399 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:57.399 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.174083+0000", 2024-09-21T14:59:57.399 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016536200000000001 2024-09-21T14:59:57.399 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:57.399 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:57.399 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:57.400 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.400 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.400 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9382 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 442368~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T14:59:57.400 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158805+0000", 2024-09-21T14:59:57.400 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 78.202293988999998, 2024-09-21T14:59:57.400 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015592623, 2024-09-21T14:59:57.400 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:57.400 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:57.400 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:57.400 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:57.400 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:57.400 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9382 2024-09-21T14:59:57.400 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.400 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:57.400 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.400 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:57.401 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158805+0000", 2024-09-21T14:59:57.401 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:57.401 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.401 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.401 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:57.401 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158805+0000", 2024-09-21T14:59:57.401 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:57.401 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.401 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.401 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:57.401 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158805+0000", 2024-09-21T14:59:57.401 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:57.401 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.401 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.401 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:57.402 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158808+0000", 2024-09-21T14:59:57.402 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9179999999999996e-06 2024-09-21T14:59:57.402 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.402 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.402 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:57.402 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158817+0000", 2024-09-21T14:59:57.402 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9299999999999992e-06 2024-09-21T14:59:57.402 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.402 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.402 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:57.402 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158825+0000", 2024-09-21T14:59:57.402 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6739999999999997e-06 2024-09-21T14:59:57.402 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.402 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.402 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:57.403 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.174122+0000", 2024-09-21T14:59:57.403 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.01529694 2024-09-21T14:59:57.403 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.403 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.403 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:57.403 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.174185+0000", 2024-09-21T14:59:57.403 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3015000000000006e-05 2024-09-21T14:59:57.403 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.403 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.403 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:57.403 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.174398+0000", 2024-09-21T14:59:57.403 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021274799999999999 2024-09-21T14:59:57.403 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:57.403 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:57.404 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:57.404 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.404 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.404 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22701 3.19 3:98de45b8:::rbd_data.11366acc894.0000000000000000:19 [read 770048~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:57.404 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.140215+0000", 2024-09-21T14:59:57.404 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 67.220884557999995, 2024-09-21T14:59:57.404 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014072497, 2024-09-21T14:59:57.404 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:57.404 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:57.404 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:57.404 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:57.405 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:57.405 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22701 2024-09-21T14:59:57.405 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.405 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:57.405 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.405 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:57.405 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140215+0000", 2024-09-21T14:59:57.405 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:57.405 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.405 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.405 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:57.405 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140215+0000", 2024-09-21T14:59:57.405 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:57.405 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.405 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.406 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:57.406 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140214+0000", 2024-09-21T14:59:57.406 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:57.406 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.406 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.406 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:57.406 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140217+0000", 2024-09-21T14:59:57.406 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.74e-06 2024-09-21T14:59:57.406 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.406 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.406 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:57.406 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140223+0000", 2024-09-21T14:59:57.406 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0929999999999999e-06 2024-09-21T14:59:57.406 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.406 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.407 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:57.407 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140228+0000", 2024-09-21T14:59:57.407 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2279999999999998e-06 2024-09-21T14:59:57.407 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.407 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.407 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:57.407 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.153991+0000", 2024-09-21T14:59:57.407 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013762329 2024-09-21T14:59:57.407 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.407 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.407 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:57.407 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.154098+0000", 2024-09-21T14:59:57.407 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000107117 2024-09-21T14:59:57.407 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.407 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.408 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:57.408 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.154287+0000", 2024-09-21T14:59:57.408 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000189431 2024-09-21T14:59:57.408 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:57.408 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:57.408 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:57.408 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.408 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.408 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22702 3.19 3:98de45b8:::rbd_data.11366acc894.0000000000000000:19 [read 835584~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:57.408 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.140232+0000", 2024-09-21T14:59:57.408 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 67.220866705999995, 2024-09-21T14:59:57.408 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015338742000000001, 2024-09-21T14:59:57.408 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:57.408 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:57.408 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:57.408 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:57.409 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:57.409 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22702 2024-09-21T14:59:57.409 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.409 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:57.409 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.409 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:57.409 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140232+0000", 2024-09-21T14:59:57.409 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:57.409 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.409 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.409 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:57.409 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140232+0000", 2024-09-21T14:59:57.409 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:57.409 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.409 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.410 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:57.410 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140232+0000", 2024-09-21T14:59:57.410 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:57.410 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.410 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.410 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:57.410 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140234+0000", 2024-09-21T14:59:57.410 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.402e-06 2024-09-21T14:59:57.410 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.410 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.410 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:57.410 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140241+0000", 2024-09-21T14:59:57.410 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.054e-06 2024-09-21T14:59:57.410 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.410 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.410 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:57.411 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140246+0000", 2024-09-21T14:59:57.411 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0050000000000004e-06 2024-09-21T14:59:57.411 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.411 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.411 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:57.411 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.154315+0000", 2024-09-21T14:59:57.411 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014069440000000001 2024-09-21T14:59:57.411 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.411 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.411 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:57.411 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.154383+0000", 2024-09-21T14:59:57.411 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8151000000000005e-05 2024-09-21T14:59:57.411 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.411 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.411 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:57.412 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155571+0000", 2024-09-21T14:59:57.412 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001188029 2024-09-21T14:59:57.412 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:57.412 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:57.412 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:57.412 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.412 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.412 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22703 3.19 3:98de45b8:::rbd_data.11366acc894.0000000000000000:19 [read 901120~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:57.412 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.140250+0000", 2024-09-21T14:59:57.412 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 67.220849548000004, 2024-09-21T14:59:57.412 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014992169, 2024-09-21T14:59:57.412 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:57.412 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:57.412 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:57.412 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:57.412 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:57.413 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22703 2024-09-21T14:59:57.413 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.413 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:57.413 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.413 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:57.413 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140250+0000", 2024-09-21T14:59:57.413 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:57.413 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.413 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.413 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:57.413 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140250+0000", 2024-09-21T14:59:57.413 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:57.413 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.413 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.413 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:57.414 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140249+0000", 2024-09-21T14:59:57.414 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:57.414 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.414 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.414 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:57.414 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140251+0000", 2024-09-21T14:59:57.414 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.136e-06 2024-09-21T14:59:57.414 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.414 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.414 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:57.414 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140257+0000", 2024-09-21T14:59:57.414 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9610000000000002e-06 2024-09-21T14:59:57.414 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.414 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.415 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:57.415 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140262+0000", 2024-09-21T14:59:57.415 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4549999999999997e-06 2024-09-21T14:59:57.415 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.415 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.415 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:57.415 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155006+0000", 2024-09-21T14:59:57.415 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014744189 2024-09-21T14:59:57.415 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.415 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.415 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:57.415 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155081+0000", 2024-09-21T14:59:57.415 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4548000000000005e-05 2024-09-21T14:59:57.415 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.415 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.416 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:57.416 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155242+0000", 2024-09-21T14:59:57.416 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000161204 2024-09-21T14:59:57.416 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:57.416 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:57.416 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:57.416 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.416 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.416 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22704 3.19 3:98de45b8:::rbd_data.11366acc894.0000000000000000:19 [read 966656~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:57.416 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.140266+0000", 2024-09-21T14:59:57.416 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 67.220833231, 2024-09-21T14:59:57.416 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015223443999999999, 2024-09-21T14:59:57.416 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:57.416 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:57.417 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:57.417 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:57.417 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:57.417 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22704 2024-09-21T14:59:57.417 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.417 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:57.417 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.417 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:57.417 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140266+0000", 2024-09-21T14:59:57.417 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:57.417 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.417 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.417 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:57.417 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140266+0000", 2024-09-21T14:59:57.417 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:57.417 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.418 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.418 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:57.418 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140266+0000", 2024-09-21T14:59:57.418 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:57.418 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.418 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.418 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:57.418 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140268+0000", 2024-09-21T14:59:57.418 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.475e-06 2024-09-21T14:59:57.418 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.418 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.418 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:57.418 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140274+0000", 2024-09-21T14:59:57.419 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.319e-06 2024-09-21T14:59:57.419 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.419 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.419 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:57.419 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140279+0000", 2024-09-21T14:59:57.419 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.882e-06 2024-09-21T14:59:57.419 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.419 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.419 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:57.419 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155264+0000", 2024-09-21T14:59:57.419 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014984655 2024-09-21T14:59:57.419 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.419 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.419 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:57.419 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155338+0000", 2024-09-21T14:59:57.420 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4091999999999995e-05 2024-09-21T14:59:57.420 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.420 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.420 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:57.420 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155489+0000", 2024-09-21T14:59:57.420 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000151349 2024-09-21T14:59:57.420 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:57.420 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:57.420 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:57.420 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.420 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.420 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22705 3.19 3:98de45b8:::rbd_data.11366acc894.0000000000000000:19 [read 1032192~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T14:59:57.420 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.140284+0000", 2024-09-21T14:59:57.420 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 67.220815373999997, 2024-09-21T14:59:57.420 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015467896, 2024-09-21T14:59:57.420 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:57.421 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T14:59:57.421 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:57.421 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:57.421 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:57.421 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22705 2024-09-21T14:59:57.421 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.421 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:57.421 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.421 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:57.421 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140284+0000", 2024-09-21T14:59:57.421 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:57.421 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.421 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.421 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:57.421 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140284+0000", 2024-09-21T14:59:57.421 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:57.422 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.422 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.422 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:57.422 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140283+0000", 2024-09-21T14:59:57.422 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:57.422 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.422 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.422 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:57.422 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140286+0000", 2024-09-21T14:59:57.422 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7130000000000001e-06 2024-09-21T14:59:57.422 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.422 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.423 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:57.423 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140292+0000", 2024-09-21T14:59:57.423 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3430000000000001e-06 2024-09-21T14:59:57.423 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.423 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.423 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:57.423 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140298+0000", 2024-09-21T14:59:57.423 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9730000000000002e-06 2024-09-21T14:59:57.423 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.423 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.423 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:57.423 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155510+0000", 2024-09-21T14:59:57.423 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015211738000000001 2024-09-21T14:59:57.423 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.423 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.424 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:57.424 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155598+0000", 2024-09-21T14:59:57.424 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7822e-05 2024-09-21T14:59:57.424 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.424 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.424 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:57.424 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155752+0000", 2024-09-21T14:59:57.424 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0001537 2024-09-21T14:59:57.424 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:57.424 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:57.424 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:57.424 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.424 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.424 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:37892 3.1b 3:d963c97f:::rbd_data.11365d19aa81.00000000000000be:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-21T14:59:57.424 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:04.029946+0000", 2024-09-21T14:59:57.425 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 53.331153606999997, 2024-09-21T14:59:57.425 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.017686021999999999, 2024-09-21T14:59:57.425 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:57.425 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:57.425 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:57.425 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:57.425 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:57.425 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 37892 2024-09-21T14:59:57.425 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.425 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:57.425 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.425 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:57.426 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.029946+0000", 2024-09-21T14:59:57.426 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:57.426 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.426 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.426 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:57.426 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.029946+0000", 2024-09-21T14:59:57.426 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:57.426 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.426 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.426 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:57.426 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.029944+0000", 2024-09-21T14:59:57.426 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T14:59:57.426 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.426 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.427 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:57.427 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.029953+0000", 2024-09-21T14:59:57.427 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7589999999999992e-06 2024-09-21T14:59:57.427 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.427 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.427 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:57.427 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.029971+0000", 2024-09-21T14:59:57.427 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7795000000000002e-05 2024-09-21T14:59:57.427 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.427 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.427 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:57.427 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.029984+0000", 2024-09-21T14:59:57.427 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.27e-05 2024-09-21T14:59:57.427 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.427 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.427 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:57.428 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.030072+0000", 2024-09-21T14:59:57.428 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8082000000000001e-05 2024-09-21T14:59:57.428 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.428 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.428 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:57.428 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.030276+0000", 2024-09-21T14:59:57.428 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00020375699999999999 2024-09-21T14:59:57.428 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.428 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.428 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-21T14:59:57.428 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.031634+0000", 2024-09-21T14:59:57.428 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0013586049999999999 2024-09-21T14:59:57.428 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.429 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.429 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:59:57.429 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.037554+0000", 2024-09-21T14:59:57.429 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0059197449999999997 2024-09-21T14:59:57.429 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.429 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.429 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:57.429 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.047490+0000", 2024-09-21T14:59:57.429 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0099361680000000004 2024-09-21T14:59:57.429 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.429 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.429 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:59:57.429 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.047569+0000", 2024-09-21T14:59:57.429 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8298000000000001e-05 2024-09-21T14:59:57.429 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.430 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.430 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:57.430 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.047632+0000", 2024-09-21T14:59:57.430 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3006e-05 2024-09-21T14:59:57.430 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:57.430 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:57.430 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:57.430 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.430 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.430 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:63820 3.19 3.626bc079 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-21T14:59:57.430 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:27.241293+0000", 2024-09-21T14:59:57.430 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 30.119806514, 2024-09-21T14:59:57.430 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.98723808199999996, 2024-09-21T14:59:57.430 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:57.430 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T14:59:57.431 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:57.431 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:57.431 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:57.431 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 63820 2024-09-21T14:59:57.431 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.431 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:57.431 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.431 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:57.431 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241293+0000", 2024-09-21T14:59:57.431 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:57.431 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.431 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.431 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:57.431 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241293+0000", 2024-09-21T14:59:57.431 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:57.431 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.432 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.432 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:57.432 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241291+0000", 2024-09-21T14:59:57.432 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T14:59:57.432 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.432 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.432 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:57.432 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241299+0000", 2024-09-21T14:59:57.432 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1529999999999997e-06 2024-09-21T14:59:57.432 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.432 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.432 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:57.432 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241317+0000", 2024-09-21T14:59:57.432 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7764000000000001e-05 2024-09-21T14:59:57.432 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.433 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.433 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:57.433 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241328+0000", 2024-09-21T14:59:57.433 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1245e-05 2024-09-21T14:59:57.433 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.433 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.433 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:57.433 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241407+0000", 2024-09-21T14:59:57.433 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8762000000000002e-05 2024-09-21T14:59:57.433 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.433 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.433 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T14:59:57.433 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241418+0000", 2024-09-21T14:59:57.433 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1695000000000001e-05 2024-09-21T14:59:57.433 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.433 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.434 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:57.434 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:28.228510+0000", 2024-09-21T14:59:57.434 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.98709166100000001 2024-09-21T14:59:57.434 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.434 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.434 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:57.434 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:28.228531+0000", 2024-09-21T14:59:57.434 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0696e-05 2024-09-21T14:59:57.434 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:57.434 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:57.434 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:57.434 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.435 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.435 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:73845 3.6 3:601bd01e:::rbd_data.1136f4939602.0000000000000015:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-21T14:59:57.435 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:42.393435+0000", 2024-09-21T14:59:57.435 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.967664403000001, 2024-09-21T14:59:57.435 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.030713345, 2024-09-21T14:59:57.435 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:57.435 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:57.435 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:57.435 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:57.435 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:57.435 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 73845 2024-09-21T14:59:57.435 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.435 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:57.435 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.436 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:57.436 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.393435+0000", 2024-09-21T14:59:57.436 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:57.436 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.436 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.436 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:57.436 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.393435+0000", 2024-09-21T14:59:57.436 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:57.436 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.436 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.436 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:57.436 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.393433+0000", 2024-09-21T14:59:57.436 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T14:59:57.436 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.436 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.436 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:57.437 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.393446+0000", 2024-09-21T14:59:57.437 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3013000000000001e-05 2024-09-21T14:59:57.437 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.437 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.437 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:57.437 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.393475+0000", 2024-09-21T14:59:57.437 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9615999999999999e-05 2024-09-21T14:59:57.437 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.437 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.437 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:57.437 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.393489+0000", 2024-09-21T14:59:57.437 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4103e-05 2024-09-21T14:59:57.437 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.437 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.437 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:57.437 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.393565+0000", 2024-09-21T14:59:57.438 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5195000000000001e-05 2024-09-21T14:59:57.438 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.438 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.438 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:57.438 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.393734+0000", 2024-09-21T14:59:57.438 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016909999999999999 2024-09-21T14:59:57.438 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.438 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.438 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 7", 2024-09-21T14:59:57.438 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.400243+0000", 2024-09-21T14:59:57.438 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0065093629999999998 2024-09-21T14:59:57.438 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.438 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.438 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:59:57.439 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.423721+0000", 2024-09-21T14:59:57.439 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.023477984 2024-09-21T14:59:57.439 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.439 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.439 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:57.439 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.423852+0000", 2024-09-21T14:59:57.439 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000131369 2024-09-21T14:59:57.439 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.439 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.439 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:59:57.439 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.424011+0000", 2024-09-21T14:59:57.439 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000158231 2024-09-21T14:59:57.439 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.439 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.440 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:57.440 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.424148+0000", 2024-09-21T14:59:57.440 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013732699999999999 2024-09-21T14:59:57.440 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:57.440 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:57.440 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:57.440 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.440 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.440 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:85603 3.7 3:e5bd5c60:::rbd_object_map.113637a2ddc2.0000000000000044:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115)", 2024-09-21T14:59:57.440 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:56.162443+0000", 2024-09-21T14:59:57.440 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.198656231, 2024-09-21T14:59:57.440 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013898304, 2024-09-21T14:59:57.440 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:57.440 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:57.440 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:57.440 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:57.441 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:57.441 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 85603 2024-09-21T14:59:57.441 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.441 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:57.441 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.441 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:57.441 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.162443+0000", 2024-09-21T14:59:57.441 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:57.441 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.441 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.441 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:57.441 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.162443+0000", 2024-09-21T14:59:57.441 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:57.441 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.441 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.441 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:57.442 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.162443+0000", 2024-09-21T14:59:57.442 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:57.442 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.442 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.442 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:57.442 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.162445+0000", 2024-09-21T14:59:57.442 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.88e-06 2024-09-21T14:59:57.442 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.442 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.442 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:57.442 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.162450+0000", 2024-09-21T14:59:57.442 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4020000000000004e-06 2024-09-21T14:59:57.442 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.442 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.442 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:57.443 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.162454+0000", 2024-09-21T14:59:57.443 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9770000000000002e-06 2024-09-21T14:59:57.443 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.443 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.443 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:57.443 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.170929+0000", 2024-09-21T14:59:57.443 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0084749170000000002 2024-09-21T14:59:57.443 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.443 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.443 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:57.443 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.171055+0000", 2024-09-21T14:59:57.443 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000126511 2024-09-21T14:59:57.443 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.443 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.443 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T14:59:57.443 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.171537+0000", 2024-09-21T14:59:57.444 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00048164399999999998 2024-09-21T14:59:57.444 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.444 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.444 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:59:57.444 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.175973+0000", 2024-09-21T14:59:57.444 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0044358089999999998 2024-09-21T14:59:57.444 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.444 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.444 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:57.444 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.176128+0000", 2024-09-21T14:59:57.444 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015554699999999999 2024-09-21T14:59:57.444 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.444 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.444 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:59:57.445 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.176299+0000", 2024-09-21T14:59:57.445 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017067699999999999 2024-09-21T14:59:57.445 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.445 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.445 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:57.445 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.176341+0000", 2024-09-21T14:59:57.445 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2265999999999998e-05 2024-09-21T14:59:57.445 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:57.445 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:57.445 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:57.445 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.445 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.445 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:85604 3.7 3:e5bd5c60:::rbd_object_map.113637a2ddc2.0000000000000044:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115)", 2024-09-21T14:59:57.445 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:56.162457+0000", 2024-09-21T14:59:57.445 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.1986417739999999, 2024-09-21T14:59:57.446 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014870204, 2024-09-21T14:59:57.446 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:57.446 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:57.446 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:57.446 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:57.446 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:57.446 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 85604 2024-09-21T14:59:57.446 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.446 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:57.446 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.446 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:57.446 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.162457+0000", 2024-09-21T14:59:57.446 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:57.446 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.447 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.447 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:57.447 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.162457+0000", 2024-09-21T14:59:57.447 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:57.447 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.447 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.447 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:57.447 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.162457+0000", 2024-09-21T14:59:57.447 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:57.447 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.447 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.447 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:57.447 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.162460+0000", 2024-09-21T14:59:57.447 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6220000000000001e-06 2024-09-21T14:59:57.447 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.447 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.448 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:57.448 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.162473+0000", 2024-09-21T14:59:57.448 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3045e-05 2024-09-21T14:59:57.448 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.448 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.448 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:57.448 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.162484+0000", 2024-09-21T14:59:57.448 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0941000000000001e-05 2024-09-21T14:59:57.448 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.448 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.448 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:57.448 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.172414+0000", 2024-09-21T14:59:57.448 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0099308350000000007 2024-09-21T14:59:57.449 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.449 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.449 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:57.449 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.172692+0000", 2024-09-21T14:59:57.449 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00027786 2024-09-21T14:59:57.449 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.449 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.449 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T14:59:57.449 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.173552+0000", 2024-09-21T14:59:57.449 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00085935999999999998 2024-09-21T14:59:57.449 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.449 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.449 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:59:57.449 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.176079+0000", 2024-09-21T14:59:57.449 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0025274189999999999 2024-09-21T14:59:57.449 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.450 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.450 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:57.450 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.177234+0000", 2024-09-21T14:59:57.450 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001154569 2024-09-21T14:59:57.450 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.450 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.450 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:59:57.450 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.177286+0000", 2024-09-21T14:59:57.450 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1969999999999999e-05 2024-09-21T14:59:57.450 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.450 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.450 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:57.450 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.177328+0000", 2024-09-21T14:59:57.450 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1891999999999998e-05 2024-09-21T14:59:57.450 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:57.451 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:57.451 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:57.451 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.451 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.451 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:85605 3.7 3:e5bd5c60:::rbd_object_map.113637a2ddc2.0000000000000044:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115)", 2024-09-21T14:59:57.451 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:56.162490+0000", 2024-09-21T14:59:57.451 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.1986086149999999, 2024-09-21T14:59:57.451 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015121122000000001, 2024-09-21T14:59:57.451 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T14:59:57.451 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T14:59:57.451 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T14:59:57.451 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T14:59:57.451 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T14:59:57.451 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 85605 2024-09-21T14:59:57.451 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.452 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T14:59:57.452 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.452 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T14:59:57.452 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.162490+0000", 2024-09-21T14:59:57.452 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:57.452 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.452 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.452 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T14:59:57.452 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.162490+0000", 2024-09-21T14:59:57.452 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T14:59:57.452 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.452 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.452 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T14:59:57.452 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.162490+0000", 2024-09-21T14:59:57.453 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T14:59:57.453 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.453 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.453 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T14:59:57.453 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.162494+0000", 2024-09-21T14:59:57.453 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.5020000000000001e-06 2024-09-21T14:59:57.453 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.453 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.453 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T14:59:57.453 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.162498+0000", 2024-09-21T14:59:57.453 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6920000000000001e-06 2024-09-21T14:59:57.453 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.453 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.453 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T14:59:57.454 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.162503+0000", 2024-09-21T14:59:57.454 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.617e-06 2024-09-21T14:59:57.454 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.454 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.454 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T14:59:57.454 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.174549+0000", 2024-09-21T14:59:57.454 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012046238000000001 2024-09-21T14:59:57.454 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.454 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.454 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T14:59:57.454 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.174640+0000", 2024-09-21T14:59:57.454 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0810999999999995e-05 2024-09-21T14:59:57.454 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.454 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.455 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T14:59:57.455 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.175156+0000", 2024-09-21T14:59:57.455 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00051634700000000001 2024-09-21T14:59:57.455 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.455 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.455 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T14:59:57.455 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.177347+0000", 2024-09-21T14:59:57.455 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0021907319999999999 2024-09-21T14:59:57.455 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.455 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.455 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T14:59:57.455 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.177407+0000", 2024-09-21T14:59:57.455 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9657999999999997e-05 2024-09-21T14:59:57.455 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.456 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.456 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T14:59:57.456 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.177498+0000", 2024-09-21T14:59:57.456 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1225999999999997e-05 2024-09-21T14:59:57.456 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T14:59:57.456 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T14:59:57.456 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T14:59:57.456 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.177612+0000", 2024-09-21T14:59:57.456 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011362999999999999 2024-09-21T14:59:57.456 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:57.456 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:57.456 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:57.456 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T14:59:57.456 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T14:59:57.457 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:59:57.458 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T14:59:57.458 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:59:57.454+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:59:57.496 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T14:59:57.517 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T14:59:57.538 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T14:59:57.558 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T14:59:57.557+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T14:59:57.560 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T14:59:57.583 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-21T14:59:57.608 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T14:59:57.619 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:57.619 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:57.629 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T14:59:57.631 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T14:59:57.653 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T14:59:57.656 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:59:57.656 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:59:57.656 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:59:57.656 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:59:57.656 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:59:57.656 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:59:57.656 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:59:57.656 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T14:59:57.659 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:59:57.657+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:59:57.668 INFO:teuthology.orchestra.run.smithi032.stdout:2650 writesame 0xa510a43 thru 0xa51f471 data_size 0x1(0xea2f bytes) 2024-09-21T14:59:57.730 INFO:teuthology.orchestra.run.smithi032.stdout:2651 trunc from 0xbe8d4c8 to 0x4c2e893 2024-09-21T14:59:57.741 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:57.741 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:57.752 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T14:59:57.761 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:59:57.761+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:59:57.774 INFO:teuthology.orchestra.run.smithi032.stdout:2652 write 0xb4c4a11 thru 0xb4d053f (0xbb2f bytes) 2024-09-21T14:59:57.787 INFO:teuthology.orchestra.run.smithi032.stdout:2653 trunc from 0xb4d0540 to 0xe76732c 2024-09-21T14:59:57.801 INFO:teuthology.orchestra.run.smithi032.stdout:2654 punch from 0xaab9a42 to 0xaabe923, (0x4ee1 bytes) 2024-09-21T14:59:57.801 INFO:teuthology.orchestra.run.smithi032.stdout:2656 writesame 0x5b92dbe thru 0x5b9f761 data_size 0x1d(0xc9a4 bytes) 2024-09-21T14:59:57.835 INFO:teuthology.orchestra.run.smithi032.stdout:2658 trunc from 0xe76732c to 0xcd3520 2024-09-21T14:59:57.861 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:57.861 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:57.862 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:59:57.861+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:59:57.869 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:57.872 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T14:59:57.878 INFO:teuthology.orchestra.run.smithi032.stdout:2660 trunc from 0xcd3520 to 0x49447b2 2024-09-21T14:59:57.890 INFO:teuthology.orchestra.run.smithi032.stdout:2661 write 0x3536a0a thru 0x3540057 (0x964e bytes) 2024-09-21T14:59:57.904 INFO:teuthology.orchestra.run.smithi032.stdout:2663 trunc from 0x49447b2 to 0x1be034d 2024-09-21T14:59:57.925 INFO:teuthology.orchestra.run.smithi032.stdout:2664 read 0x1a9d774 thru 0x1aa654b (0x8dd8 bytes) 2024-09-21T14:59:57.925 INFO:teuthology.orchestra.run.smithi032.stdout:2665 read 0xe79d68 thru 0xe7c720 (0x29b9 bytes) 2024-09-21T14:59:57.926 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:57.944 INFO:teuthology.orchestra.run.smithi032.stdout:2667 write 0x3116450 thru 0x3124514 (0xe0c5 bytes) 2024-09-21T14:59:57.953 INFO:teuthology.orchestra.run.smithi032.stdout:2670 read 0x1b72a87 thru 0x1b77034 (0x45ae bytes) 2024-09-21T14:59:57.953 INFO:teuthology.orchestra.run.smithi032.stdout:2671 punch from 0x1325e31 to 0x1328f09, (0x30d8 bytes) 2024-09-21T14:59:57.963 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:59:57.958+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:59:57.978 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:57.978 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:57.989 INFO:teuthology.orchestra.run.smithi032.stdout:2672 write 0xd5be35f thru 0xd5c6dfd (0x8a9f bytes) 2024-09-21T14:59:57.990 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T14:59:57.996 INFO:teuthology.orchestra.run.smithi032.stdout:2673 read 0x84254df thru 0x84286fe (0x3220 bytes) 2024-09-21T14:59:58.006 INFO:teuthology.orchestra.run.smithi032.stdout:2674 trunc from 0xd5c6dfe to 0x9205fb7 2024-09-21T14:59:58.027 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:58.050 INFO:teuthology.orchestra.run.smithi032.stdout:2677 write 0xbbbfe35 thru 0xbbc8580 (0x874c bytes) 2024-09-21T14:59:58.064 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:59:58.058+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:59:58.072 INFO:teuthology.orchestra.run.smithi032.stdout:2678 trunc from 0xbbc8581 to 0x5f4b9c4 2024-09-21T14:59:58.093 INFO:teuthology.orchestra.run.smithi032.stdout:2679 read 0x4f4d70c thru 0x4f525bf (0x4eb4 bytes) 2024-09-21T14:59:58.093 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:58.094 INFO:teuthology.orchestra.run.smithi032.stdout:2684 write 0x49cce58 thru 0x49d9b8a (0xcd33 bytes) 2024-09-21T14:59:58.129 INFO:teuthology.orchestra.run.smithi032.stdout:2685 write 0xc39507d thru 0xc3a49f1 (0xf975 bytes) 2024-09-21T14:59:58.137 INFO:teuthology.orchestra.run.smithi032.stdout:2686 write 0x8747bf6 thru 0x874d549 (0x5954 bytes) 2024-09-21T14:59:58.143 INFO:teuthology.orchestra.run.smithi032.stdout:2687 write 0x4a3843a thru 0x4a3fa65 (0x762c bytes) 2024-09-21T14:59:58.150 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:58.150 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:58.154 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T14:59:58.161 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T14:59:58.168 INFO:teuthology.orchestra.run.smithi032.stdout:2688 write 0xe4a45ed thru 0xe4b1f39 (0xd94d bytes) 2024-09-21T14:59:58.174 INFO:teuthology.orchestra.run.smithi032.stdout:2691 read 0x9d14515 thru 0x9d16b7d (0x2669 bytes) 2024-09-21T14:59:58.174 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:58.174 INFO:teuthology.orchestra.run.smithi032.stdout:2694 trunc from 0xe4b1f3a to 0xe787e87 2024-09-21T14:59:58.185 INFO:teuthology.orchestra.run.smithi032.stdout:2695 write 0x893aa33 thru 0x8949d5e (0xf32c bytes) 2024-09-21T14:59:58.193 INFO:teuthology.orchestra.run.smithi032.stdout:2696 writesame 0xbbb3686 thru 0xbbb4d5d data_size 0x8(0x16d8 bytes) 2024-09-21T14:59:58.200 INFO:teuthology.orchestra.run.smithi032.stdout:2697 punch from 0xa521044 to 0xa527a43, (0x69ff bytes) 2024-09-21T14:59:58.200 INFO:teuthology.orchestra.run.smithi032.stdout:2702 read 0x1f948c3 thru 0x1f9b878 (0x6fb6 bytes) 2024-09-21T14:59:58.200 INFO:teuthology.orchestra.run.smithi032.stdout:2703 read 0xad359b2 thru 0xad41b1a (0xc169 bytes) 2024-09-21T14:59:58.200 INFO:teuthology.orchestra.run.smithi032.stdout:2704 write 0x9de9239 thru 0x9df270f (0x94d7 bytes) 2024-09-21T14:59:58.206 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:59:58.202+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:59:58.208 INFO:teuthology.orchestra.run.smithi032.stdout:2706 write 0x585f96 thru 0x58ff33 (0x9f9e bytes) 2024-09-21T14:59:58.212 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:58.212 INFO:teuthology.orchestra.run.smithi032.stdout:2708 read 0x4e9eec8 thru 0x4ea6d56 (0x7e8f bytes) 2024-09-21T14:59:58.212 INFO:teuthology.orchestra.run.smithi032.stdout:2709 read 0x1d005a2 thru 0x1d071d8 (0x6c37 bytes) 2024-09-21T14:59:58.212 INFO:teuthology.orchestra.run.smithi032.stdout:2710 punch from 0x1f84ae5 to 0x1f8f2e2, (0xa7fd bytes) 2024-09-21T14:59:58.212 INFO:teuthology.orchestra.run.smithi032.stdout:2711 punch from 0x1dc311b to 0x1dce152, (0xb037 bytes) 2024-09-21T14:59:58.233 INFO:teuthology.orchestra.run.smithi032.stdout:2712 trunc from 0xe787e87 to 0x3608340 2024-09-21T14:59:58.253 INFO:teuthology.orchestra.run.smithi032.stdout:2713 punch from 0x1b8ebd6 to 0x1b9d3c0, (0xe7ea bytes) 2024-09-21T14:59:58.266 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:59:58.265+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:59:58.272 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:58.273 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:58.275 INFO:teuthology.orchestra.run.smithi032.stdout:2714 write 0x9717d7a thru 0x971ff1f (0x81a6 bytes) 2024-09-21T14:59:58.282 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T14:59:58.301 INFO:teuthology.orchestra.run.smithi032.stdout:2715 writesame 0xc42f7bf thru 0xc439e91 data_size 0x1(0xa6d3 bytes) 2024-09-21T14:59:58.302 INFO:teuthology.orchestra.run.smithi032.stdout:2716 writesame 0x47bba0 thru 0x47ccc4 data_size 0x21(0x1125 bytes) 2024-09-21T14:59:58.307 INFO:teuthology.orchestra.run.smithi032.stdout:2717 trunc from 0xc439e92 to 0xdd73d3d 2024-09-21T14:59:58.330 INFO:teuthology.orchestra.run.smithi032.stdout:2718 writesame 0xe3083a8 thru 0xe318124 data_size 0x3(0xfd7d bytes) 2024-09-21T14:59:58.344 INFO:teuthology.orchestra.run.smithi032.stdout:2719 write 0xeb11186 thru 0xeb1ee5b (0xdcd6 bytes) 2024-09-21T14:59:58.352 INFO:teuthology.orchestra.run.smithi032.stdout:2720 read 0xad6bf7 thru 0xadf088 (0x8492 bytes) 2024-09-21T14:59:58.365 INFO:teuthology.orchestra.run.smithi032.stdout:2721 trunc from 0xeb1ee5c to 0x8a69b72 2024-09-21T14:59:58.367 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:59:58.365+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:59:58.387 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:58.387 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:58.398 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T14:59:58.402 INFO:teuthology.orchestra.run.smithi032.stdout:2722 write 0xa20f587 thru 0xa21555e (0x5fd8 bytes) 2024-09-21T14:59:58.408 INFO:teuthology.orchestra.run.smithi032.stdout:2723 write 0x6a689f7 thru 0x6a6bd1f (0x3329 bytes) 2024-09-21T14:59:58.415 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:58.415 INFO:teuthology.orchestra.run.smithi032.stdout:2728 writesame 0x5805dea thru 0x58095d9 data_size 0x28(0x37f0 bytes) 2024-09-21T14:59:58.421 INFO:teuthology.orchestra.run.smithi032.stdout:2731 read 0x3899c61 thru 0x389cf06 (0x32a6 bytes) 2024-09-21T14:59:58.421 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:58.421 INFO:teuthology.orchestra.run.smithi032.stdout:2734 punch from 0x86ee19b to 0x86f0cc0, (0x2b25 bytes) 2024-09-21T14:59:58.421 INFO:teuthology.orchestra.run.smithi032.stdout:2736 read 0xa1718b thru 0xa1faca (0x8940 bytes) 2024-09-21T14:59:58.421 INFO:teuthology.orchestra.run.smithi032.stdout:2737 read 0x2cd85dc thru 0x2cdc733 (0x4158 bytes) 2024-09-21T14:59:58.423 INFO:teuthology.orchestra.run.smithi032.stdout:2739 trunc from 0xa21555f to 0xac6ad8a 2024-09-21T14:59:58.435 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:58.454 INFO:teuthology.orchestra.run.smithi032.stdout:2741 write 0xd1c5b14 thru 0xd1d2177 (0xc664 bytes) 2024-09-21T14:59:58.461 INFO:teuthology.orchestra.run.smithi032.stdout:2742 read 0x6acd3be thru 0x6ad9520 (0xc163 bytes) 2024-09-21T14:59:58.461 INFO:teuthology.orchestra.run.smithi032.stdout:2743 read 0xc7d2580 thru 0xc7d3f16 (0x1997 bytes) 2024-09-21T14:59:58.468 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:59:58.469+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:59:58.472 INFO:teuthology.orchestra.run.smithi032.stdout:2746 trunc from 0xd1d2178 to 0x7995e41 2024-09-21T14:59:58.494 INFO:teuthology.orchestra.run.smithi032.stdout:2748 punch from 0x458e32b to 0x459b935, (0xd60a bytes) 2024-09-21T14:59:58.494 INFO:teuthology.orchestra.run.smithi032.stdout:2749 read 0x564bc1f thru 0x5651d09 (0x60eb bytes) 2024-09-21T14:59:58.494 INFO:teuthology.orchestra.run.smithi032.stdout:2751 punch from 0x8f85c2 to 0x905874, (0xd2b2 bytes) 2024-09-21T14:59:58.494 INFO:teuthology.orchestra.run.smithi032.stdout:2752 punch from 0x1c83dc4 to 0x1c8e117, (0xa353 bytes) 2024-09-21T14:59:58.494 INFO:teuthology.orchestra.run.smithi032.stdout:2753 write 0x3942725 thru 0x394ea64 (0xc340 bytes) 2024-09-21T14:59:58.500 INFO:teuthology.orchestra.run.smithi032.stdout:2754 write 0x2387047 thru 0x23891e5 (0x219f bytes) 2024-09-21T14:59:58.502 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:58.502 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:58.508 INFO:teuthology.orchestra.run.smithi032.stdout:2756 trunc from 0x7995e41 to 0x8e2fa32 2024-09-21T14:59:58.512 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T14:59:58.521 INFO:teuthology.orchestra.run.smithi032.stdout:2757 write 0x7cdd60a thru 0x7ce9b32 (0xc529 bytes) 2024-09-21T14:59:58.545 INFO:teuthology.orchestra.run.smithi032.stdout:2758 write 0xa2a99a2 thru 0xa2add4f (0x43ae bytes) 2024-09-21T14:59:58.552 INFO:teuthology.orchestra.run.smithi032.stdout:2759 punch from 0x300d21 to 0x3090a2, (0x8381 bytes) 2024-09-21T14:59:58.553 INFO:teuthology.orchestra.run.smithi032.stdout:2761 read 0x9d0f162 thru 0x9d1800c (0x8eab bytes) 2024-09-21T14:59:58.562 INFO:teuthology.orchestra.run.smithi032.stdout:2763 trunc from 0xa2add50 to 0x59e6721 2024-09-21T14:59:58.569 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:59:58.569+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:59:58.613 INFO:teuthology.orchestra.run.smithi032.stdout:2765 write 0xb6b7c43 thru 0xb6bce40 (0x51fe bytes) 2024-09-21T14:59:58.614 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:58.614 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:58.621 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T14:59:58.621 INFO:teuthology.orchestra.run.smithi032.stdout:2769 write 0x8fb29c3 thru 0x8fbcdb8 (0xa3f6 bytes) 2024-09-21T14:59:58.623 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T14:59:58.629 INFO:teuthology.orchestra.run.smithi032.stdout:2770 read 0x8c40871 thru 0x8c415d0 (0xd60 bytes) 2024-09-21T14:59:58.629 INFO:teuthology.orchestra.run.smithi032.stdout:2773 clone 23 order 22 su 16384 sc 13 2024-09-21T14:59:58.670 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T14:59:58.669+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T14:59:58.740 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:58.740 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:58.750 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T14:59:58.793 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:59:58.793+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:59:58.871 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:59:58.869+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:59:58.908 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:58.908 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:58.918 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T14:59:58.972 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T14:59:58.973+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T14:59:59.026 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:59.027 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:59.036 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T14:59:59.073 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T14:59:59.070+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T14:59:59.174 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T14:59:59.173+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T14:59:59.188 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:59.188 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:59.198 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T14:59:59.275 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:59:59.274+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:59:59.302 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:59.302 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:59.312 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T14:59:59.376 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:59:59.370+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:59:59.412 INFO:teuthology.orchestra.run.smithi032.stderr:noscrub is unset 2024-09-21T14:59:59.428 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:59.428 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:59.438 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T14:59:59.476 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:59:59.470+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:59:59.543 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:59.543 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:59.553 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T14:59:59.577 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:59:59.570+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:59:59.657 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:59.657 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:59.667 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T14:59:59.668 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T14:59:59.678 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:59:59.670+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:59:59.745 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 5, 6, 7, 4] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T14:59:59.745 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T14:59:59.745 INFO:tasks.thrashosds.thrasher:inject_pause on osd.5 2024-09-21T14:59:59.745 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-21T14:59:59.745 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-21T14:59:59.746 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok config set filestore_inject_stall 3 2024-09-21T14:59:59.774 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T14:59:59.774 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T14:59:59.788 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T14:59:59.794 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T14:59:59.790+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T14:59:59.806 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T14:59:59.827 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T14:59:59.846 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T14:59:59.864 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T14:59:59.864 INFO:teuthology.orchestra.run.smithi046.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-21T14:59:59.864 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T14:59:59.868 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T14:59:59.879 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T14:59:59.874+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T14:59:59.891 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T14:59:59.891 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T14:59:59.892 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T14:59:59.892 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T14:59:59.892 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-21T14:59:59.903 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T14:59:59.917 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T14:59:59.949 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T14:59:59.980 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T14:59:59.978+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T14:59:59.986 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T14:59:59.990 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T14:59:59.990 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T14:59:59.990 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T14:59:59.990 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T14:59:59.990 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T14:59:59.990 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T14:59:59.990 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T14:59:59.990 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:00:00.038 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:00.038 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:00:00.038 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T15:00:00.038 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T15:00:00.038 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:00.048 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T15:00:00.081 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:00:00.074+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:00:00.183 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:00:00.181+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:00:00.259 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:00.259 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T15:00:00.259 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T15:00:00.259 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T15:00:00.259 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.259 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4281.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-21T15:00:00.259 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:09.324416+0000", 2024-09-21T15:00:00.259 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 110.934098902, 2024-09-21T15:00:00.259 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.117837947, 2024-09-21T15:00:00.259 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:00.259 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:00.260 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:00.260 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4281", 2024-09-21T15:00:00.260 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/1518520303", 2024-09-21T15:00:00.260 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1 2024-09-21T15:00:00.260 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.260 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:00.260 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.260 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:00.260 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324416+0000", 2024-09-21T15:00:00.260 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:00.260 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.260 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.260 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:00.261 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324416+0000", 2024-09-21T15:00:00.261 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:00.261 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.261 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.261 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:00.261 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324414+0000", 2024-09-21T15:00:00.261 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T15:00:00.261 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.261 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.261 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:00.261 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324429+0000", 2024-09-21T15:00:00.262 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5637e-05 2024-09-21T15:00:00.262 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.262 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.262 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:00.262 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324448+0000", 2024-09-21T15:00:00.262 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9042000000000002e-05 2024-09-21T15:00:00.262 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.262 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.262 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:00.262 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324465+0000", 2024-09-21T15:00:00.262 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6767999999999999e-05 2024-09-21T15:00:00.262 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.262 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.263 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:00.263 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324571+0000", 2024-09-21T15:00:00.263 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000105714 2024-09-21T15:00:00.263 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.263 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.263 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op must wait for map", 2024-09-21T15:00:00.263 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324587+0000", 2024-09-21T15:00:00.263 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5753999999999999e-05 2024-09-21T15:00:00.263 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.263 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.263 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:00.263 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.325864+0000", 2024-09-21T15:00:00.263 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0012773350000000001 2024-09-21T15:00:00.264 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.264 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.264 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:00:00.264 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.325878+0000", 2024-09-21T15:00:00.264 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4498000000000001e-05 2024-09-21T15:00:00.264 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.264 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.264 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:00.264 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.439461+0000", 2024-09-21T15:00:00.264 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.11358212500000001 2024-09-21T15:00:00.264 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.264 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.264 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:00.264 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.439848+0000", 2024-09-21T15:00:00.265 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00038747800000000002 2024-09-21T15:00:00.265 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.265 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.265 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T15:00:00.265 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.440175+0000", 2024-09-21T15:00:00.265 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00032721900000000001 2024-09-21T15:00:00.265 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.265 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.265 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:00.265 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.441770+0000", 2024-09-21T15:00:00.265 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0015949409999999999 2024-09-21T15:00:00.265 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.265 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.265 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:00.266 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.442086+0000", 2024-09-21T15:00:00.266 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00031586299999999999 2024-09-21T15:00:00.266 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.266 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.266 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:00.266 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.442195+0000", 2024-09-21T15:00:00.266 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000108791 2024-09-21T15:00:00.266 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.266 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.266 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:00.266 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.442254+0000", 2024-09-21T15:00:00.267 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9038e-05 2024-09-21T15:00:00.267 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:00.267 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:00.267 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:00.267 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.267 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.267 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4281.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-21T15:00:00.267 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:09.447828+0000", 2024-09-21T15:00:00.267 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 110.810686901, 2024-09-21T15:00:00.267 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.32904393700000001, 2024-09-21T15:00:00.267 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:00.267 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:00.267 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:00.267 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4281", 2024-09-21T15:00:00.267 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/1518520303", 2024-09-21T15:00:00.268 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 2 2024-09-21T15:00:00.268 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.268 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:00.268 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.268 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:00.268 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447828+0000", 2024-09-21T15:00:00.268 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:00.268 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.268 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.268 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:00.268 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447828+0000", 2024-09-21T15:00:00.268 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:00.268 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.268 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.269 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:00.269 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447825+0000", 2024-09-21T15:00:00.269 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T15:00:00.269 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.269 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.269 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:00.269 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447841+0000", 2024-09-21T15:00:00.269 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5359999999999999e-05 2024-09-21T15:00:00.269 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.269 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.269 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:00.269 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447875+0000", 2024-09-21T15:00:00.269 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4388999999999999e-05 2024-09-21T15:00:00.269 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.269 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.270 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:00.270 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447903+0000", 2024-09-21T15:00:00.270 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7373e-05 2024-09-21T15:00:00.270 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.270 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.270 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:00.270 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447996+0000", 2024-09-21T15:00:00.270 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2954999999999993e-05 2024-09-21T15:00:00.270 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.270 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.270 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:00:00.270 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.448011+0000", 2024-09-21T15:00:00.270 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5038999999999999e-05 2024-09-21T15:00:00.270 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.271 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.271 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:00.271 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.776501+0000", 2024-09-21T15:00:00.271 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.32849012500000002 2024-09-21T15:00:00.271 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.271 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.271 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:00.271 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.776872+0000", 2024-09-21T15:00:00.271 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00037119600000000002 2024-09-21T15:00:00.271 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:00.271 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:00.271 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:00.271 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.271 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.271 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:752 3.14 3:28fc02e5:::rbd_data.113693316f36.0000000000000010:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-21T15:00:00.272 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:26.625579+0000", 2024-09-21T15:00:00.272 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 93.632936045999998, 2024-09-21T15:00:00.272 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042965426000000001, 2024-09-21T15:00:00.272 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:00.272 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:00.272 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:00.272 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:00.272 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:00.272 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 752 2024-09-21T15:00:00.272 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.272 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:00.272 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.272 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:00.272 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625579+0000", 2024-09-21T15:00:00.273 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:00.273 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.273 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.273 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:00.273 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625579+0000", 2024-09-21T15:00:00.273 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:00.273 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.273 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.273 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:00.273 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625577+0000", 2024-09-21T15:00:00.273 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T15:00:00.273 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.274 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.274 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:00.274 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625588+0000", 2024-09-21T15:00:00.274 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1555e-05 2024-09-21T15:00:00.274 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.274 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.274 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:00.274 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625610+0000", 2024-09-21T15:00:00.274 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2107999999999999e-05 2024-09-21T15:00:00.274 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.274 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.274 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:00.274 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625621+0000", 2024-09-21T15:00:00.275 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0762e-05 2024-09-21T15:00:00.275 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.275 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.275 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:00.275 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625692+0000", 2024-09-21T15:00:00.275 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.1203000000000003e-05 2024-09-21T15:00:00.275 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.275 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.275 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:00.275 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625872+0000", 2024-09-21T15:00:00.275 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017962800000000001 2024-09-21T15:00:00.275 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.275 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.275 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T15:00:00.275 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.633188+0000", 2024-09-21T15:00:00.276 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.007316014 2024-09-21T15:00:00.276 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.276 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.276 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:00.276 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.668357+0000", 2024-09-21T15:00:00.276 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035168933999999999 2024-09-21T15:00:00.276 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.276 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.276 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:00.276 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.668418+0000", 2024-09-21T15:00:00.276 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1282e-05 2024-09-21T15:00:00.276 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.276 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.276 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:00.277 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.668486+0000", 2024-09-21T15:00:00.277 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8468999999999998e-05 2024-09-21T15:00:00.277 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.277 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.277 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:00.277 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.668544+0000", 2024-09-21T15:00:00.277 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7785000000000003e-05 2024-09-21T15:00:00.277 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:00.277 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:00.277 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:00.277 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.277 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.277 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22738 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 49152~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T15:00:00.278 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141049+0000", 2024-09-21T15:00:00.278 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 70.117465902999996, 2024-09-21T15:00:00.278 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035428754999999999, 2024-09-21T15:00:00.278 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:00.278 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:00.278 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:00.278 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:00.278 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:00.278 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22738 2024-09-21T15:00:00.278 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.278 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:00.278 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.279 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:00.279 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141049+0000", 2024-09-21T15:00:00.279 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:00.279 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.279 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.279 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:00.279 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141049+0000", 2024-09-21T15:00:00.279 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:00.279 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.279 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.279 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:00.279 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141049+0000", 2024-09-21T15:00:00.279 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:00.280 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.280 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.280 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:00.280 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141051+0000", 2024-09-21T15:00:00.280 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4980000000000001e-06 2024-09-21T15:00:00.280 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.280 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.280 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:00.280 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141057+0000", 2024-09-21T15:00:00.280 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8030000000000002e-06 2024-09-21T15:00:00.280 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.280 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.280 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:00.281 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141066+0000", 2024-09-21T15:00:00.281 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1160000000000005e-06 2024-09-21T15:00:00.281 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.281 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.281 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:00.281 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176264+0000", 2024-09-21T15:00:00.281 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035197580999999999 2024-09-21T15:00:00.281 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.281 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.281 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:00.281 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176327+0000", 2024-09-21T15:00:00.281 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3237000000000005e-05 2024-09-21T15:00:00.281 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.281 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.282 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:00.282 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176478+0000", 2024-09-21T15:00:00.282 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000150851 2024-09-21T15:00:00.282 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:00.282 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:00.282 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:00.282 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.282 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.282 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22739 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 114688~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T15:00:00.282 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141070+0000", 2024-09-21T15:00:00.282 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 70.117444660999993, 2024-09-21T15:00:00.282 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035679094000000001, 2024-09-21T15:00:00.282 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:00.283 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:00.283 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:00.283 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:00.283 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:00.283 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22739 2024-09-21T15:00:00.283 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.283 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:00.283 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.283 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:00.283 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141070+0000", 2024-09-21T15:00:00.283 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:00.283 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.283 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.284 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:00.284 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141070+0000", 2024-09-21T15:00:00.284 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:00.284 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.284 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.284 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:00.284 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141070+0000", 2024-09-21T15:00:00.284 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:00.284 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.284 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.284 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:00.284 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141072+0000", 2024-09-21T15:00:00.284 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.379e-06 2024-09-21T15:00:00.285 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.285 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.285 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:00.285 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141078+0000", 2024-09-21T15:00:00.285 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6710000000000004e-06 2024-09-21T15:00:00.285 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.285 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.285 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:00.285 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141083+0000", 2024-09-21T15:00:00.285 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7249999999999997e-06 2024-09-21T15:00:00.285 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.285 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.285 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:00.285 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176506+0000", 2024-09-21T15:00:00.286 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035423076999999997 2024-09-21T15:00:00.286 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.286 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.286 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:00.286 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176582+0000", 2024-09-21T15:00:00.286 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6305999999999998e-05 2024-09-21T15:00:00.286 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.286 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.286 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:00.286 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176749+0000", 2024-09-21T15:00:00.286 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000167262 2024-09-21T15:00:00.286 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:00.286 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:00.286 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:00.287 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.287 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.287 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22740 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 180224~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T15:00:00.287 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141090+0000", 2024-09-21T15:00:00.287 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 70.117425256999994, 2024-09-21T15:00:00.287 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036129535999999997, 2024-09-21T15:00:00.287 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:00.287 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:00.287 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:00.287 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:00.287 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:00.287 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22740 2024-09-21T15:00:00.287 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.287 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:00.287 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.288 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:00.288 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141090+0000", 2024-09-21T15:00:00.288 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:00.288 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.288 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.288 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:00.288 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141090+0000", 2024-09-21T15:00:00.288 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:00.288 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.288 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.288 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:00.288 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141089+0000", 2024-09-21T15:00:00.288 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:00.288 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.289 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.289 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:00.289 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141092+0000", 2024-09-21T15:00:00.289 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7709999999999998e-06 2024-09-21T15:00:00.289 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.289 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.289 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:00.289 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141098+0000", 2024-09-21T15:00:00.289 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.015e-06 2024-09-21T15:00:00.289 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.289 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.289 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:00.289 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141103+0000", 2024-09-21T15:00:00.290 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8629999999999999e-06 2024-09-21T15:00:00.290 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.290 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.290 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:00.290 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176778+0000", 2024-09-21T15:00:00.290 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035674984 2024-09-21T15:00:00.290 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.290 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.290 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:00.290 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.176851+0000", 2024-09-21T15:00:00.290 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2861000000000006e-05 2024-09-21T15:00:00.290 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.290 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.290 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:00.290 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177219+0000", 2024-09-21T15:00:00.291 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00036845999999999998 2024-09-21T15:00:00.291 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:00.291 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:00.291 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:00.291 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.291 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.291 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22741 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 245760~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T15:00:00.291 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141107+0000", 2024-09-21T15:00:00.291 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 70.117407686000007, 2024-09-21T15:00:00.291 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036407729, 2024-09-21T15:00:00.291 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:00.291 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:00.291 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:00.291 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:00.292 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:00.292 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22741 2024-09-21T15:00:00.292 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.292 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:00.292 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.292 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:00.292 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141107+0000", 2024-09-21T15:00:00.292 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:00.292 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.292 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.292 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:00.292 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141107+0000", 2024-09-21T15:00:00.292 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:00.292 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.292 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.292 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:00.293 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141107+0000", 2024-09-21T15:00:00.293 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:00.293 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.293 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.293 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:00.293 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141109+0000", 2024-09-21T15:00:00.293 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6529999999999998e-06 2024-09-21T15:00:00.293 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.293 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.293 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:00.293 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141115+0000", 2024-09-21T15:00:00.293 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9649999999999996e-06 2024-09-21T15:00:00.293 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.293 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.293 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:00.294 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141121+0000", 2024-09-21T15:00:00.294 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1209999999999998e-06 2024-09-21T15:00:00.294 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.294 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.294 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:00.294 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177242+0000", 2024-09-21T15:00:00.294 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036121706000000003 2024-09-21T15:00:00.294 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.294 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.294 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:00.294 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177301+0000", 2024-09-21T15:00:00.294 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9008000000000002e-05 2024-09-21T15:00:00.294 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.294 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.295 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:00.295 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177515+0000", 2024-09-21T15:00:00.295 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00021362 2024-09-21T15:00:00.295 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:00.295 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:00.295 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:00.295 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.295 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.295 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22742 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 311296~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T15:00:00.295 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141126+0000", 2024-09-21T15:00:00.295 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 70.117389332000002, 2024-09-21T15:00:00.295 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036653537, 2024-09-21T15:00:00.295 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:00.295 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:00.296 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:00.296 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:00.296 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:00.296 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22742 2024-09-21T15:00:00.296 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.296 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:00.296 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.296 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:00.296 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141126+0000", 2024-09-21T15:00:00.296 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:00.296 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.296 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.296 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:00.296 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141126+0000", 2024-09-21T15:00:00.297 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:00.297 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.297 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.297 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:00.297 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141125+0000", 2024-09-21T15:00:00.297 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:00.297 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.297 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.297 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:00.297 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141128+0000", 2024-09-21T15:00:00.297 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.886e-06 2024-09-21T15:00:00.297 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.297 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.297 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:00.298 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141137+0000", 2024-09-21T15:00:00.298 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4399999999999994e-06 2024-09-21T15:00:00.298 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.298 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.298 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:00.298 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141143+0000", 2024-09-21T15:00:00.298 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1410000000000004e-06 2024-09-21T15:00:00.298 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.298 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.298 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:00.298 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177543+0000", 2024-09-21T15:00:00.298 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036400544999999999 2024-09-21T15:00:00.298 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.298 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.299 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:00.299 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177605+0000", 2024-09-21T15:00:00.299 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2206999999999996e-05 2024-09-21T15:00:00.299 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.299 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.299 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:00.299 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177779+0000", 2024-09-21T15:00:00.299 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017368099999999999 2024-09-21T15:00:00.299 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:00.299 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:00.299 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:00.300 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.300 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.300 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22743 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 376832~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T15:00:00.300 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141147+0000", 2024-09-21T15:00:00.300 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 70.117367848000001, 2024-09-21T15:00:00.300 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036915158000000003, 2024-09-21T15:00:00.300 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:00.300 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:00.300 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:00.300 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:00.300 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:00.300 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22743 2024-09-21T15:00:00.300 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.300 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:00.300 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.301 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:00.301 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141147+0000", 2024-09-21T15:00:00.301 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:00.301 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.301 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.301 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:00.301 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141147+0000", 2024-09-21T15:00:00.301 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:00.301 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.301 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.301 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:00.301 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141147+0000", 2024-09-21T15:00:00.301 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:00.301 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.301 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.302 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:00.302 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141149+0000", 2024-09-21T15:00:00.302 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.79e-06 2024-09-21T15:00:00.302 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.302 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.302 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:00.302 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141155+0000", 2024-09-21T15:00:00.302 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6699999999999999e-06 2024-09-21T15:00:00.302 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.302 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.302 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:00.302 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141160+0000", 2024-09-21T15:00:00.302 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5759999999999999e-06 2024-09-21T15:00:00.302 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.302 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.303 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:00.303 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177817+0000", 2024-09-21T15:00:00.303 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036657404999999997 2024-09-21T15:00:00.303 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.303 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.303 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:00.303 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177892+0000", 2024-09-21T15:00:00.303 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4602000000000002e-05 2024-09-21T15:00:00.303 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.303 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.303 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:00.303 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178062+0000", 2024-09-21T15:00:00.303 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000170464 2024-09-21T15:00:00.303 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:00.303 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:00.304 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:00.304 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.304 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.304 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22744 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 442368~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T15:00:00.304 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141164+0000", 2024-09-21T15:00:00.304 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 70.117350715000001, 2024-09-21T15:00:00.304 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037120806999999999, 2024-09-21T15:00:00.304 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:00.304 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:00.304 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:00.304 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:00.304 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:00.304 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22744 2024-09-21T15:00:00.305 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.305 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:00.305 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.305 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:00.305 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141164+0000", 2024-09-21T15:00:00.305 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:00.305 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.305 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.305 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:00.305 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141164+0000", 2024-09-21T15:00:00.305 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:00.305 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.305 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.305 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:00.305 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141164+0000", 2024-09-21T15:00:00.306 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:00.306 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.306 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.306 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:00.306 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141167+0000", 2024-09-21T15:00:00.306 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7159999999999999e-06 2024-09-21T15:00:00.306 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.306 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.306 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:00.306 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141173+0000", 2024-09-21T15:00:00.306 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.071e-06 2024-09-21T15:00:00.306 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.306 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.306 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:00.307 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141177+0000", 2024-09-21T15:00:00.307 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3610000000000003e-06 2024-09-21T15:00:00.307 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.307 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.307 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:00.307 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178083+0000", 2024-09-21T15:00:00.307 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036906405000000003 2024-09-21T15:00:00.307 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.307 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.307 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:00.307 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178143+0000", 2024-09-21T15:00:00.307 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0056999999999998e-05 2024-09-21T15:00:00.307 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.307 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.307 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:00.308 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178285+0000", 2024-09-21T15:00:00.308 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000141555 2024-09-21T15:00:00.308 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:00.308 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:00.308 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:00.308 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.308 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.308 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22745 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 507904~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T15:00:00.308 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141182+0000", 2024-09-21T15:00:00.308 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 70.117332395000005, 2024-09-21T15:00:00.308 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037332331000000003, 2024-09-21T15:00:00.308 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:00.308 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:00.308 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:00.308 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:00.309 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:00.309 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22745 2024-09-21T15:00:00.309 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.309 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:00.309 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.309 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:00.309 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141182+0000", 2024-09-21T15:00:00.309 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:00.309 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.309 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.309 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:00.309 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141182+0000", 2024-09-21T15:00:00.309 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:00.309 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.309 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.310 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:00.310 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141182+0000", 2024-09-21T15:00:00.310 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:00.310 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.310 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.310 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:00.310 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141185+0000", 2024-09-21T15:00:00.310 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.464e-06 2024-09-21T15:00:00.310 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.310 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.310 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:00.310 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141200+0000", 2024-09-21T15:00:00.310 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5886999999999999e-05 2024-09-21T15:00:00.310 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.311 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.311 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:00.311 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141571+0000", 2024-09-21T15:00:00.311 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000370909 2024-09-21T15:00:00.311 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.311 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.311 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:00.311 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178318+0000", 2024-09-21T15:00:00.311 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036746244999999997 2024-09-21T15:00:00.311 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.311 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.311 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:00.311 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178367+0000", 2024-09-21T15:00:00.312 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9317000000000002e-05 2024-09-21T15:00:00.312 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.312 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.312 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:00.312 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178515+0000", 2024-09-21T15:00:00.312 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014784600000000001 2024-09-21T15:00:00.312 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:00.312 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:00.312 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:00.312 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.312 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.312 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22746 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 573440~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T15:00:00.312 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141599+0000", 2024-09-21T15:00:00.312 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 70.116915762999994, 2024-09-21T15:00:00.312 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037402249999999998, 2024-09-21T15:00:00.312 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:00.313 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:00.313 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:00.313 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:00.313 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:00.313 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22746 2024-09-21T15:00:00.313 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.313 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:00.313 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.313 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:00.313 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141599+0000", 2024-09-21T15:00:00.313 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:00.313 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.313 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.313 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:00.313 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141599+0000", 2024-09-21T15:00:00.313 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:00.314 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.314 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.314 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:00.314 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141598+0000", 2024-09-21T15:00:00.314 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:00:00.314 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.314 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.314 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:00.314 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141637+0000", 2024-09-21T15:00:00.314 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.9119999999999998e-05 2024-09-21T15:00:00.314 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.314 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.314 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:00.315 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141647+0000", 2024-09-21T15:00:00.315 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5570000000000001e-06 2024-09-21T15:00:00.315 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.315 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.315 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:00.315 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141654+0000", 2024-09-21T15:00:00.315 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2049999999999996e-06 2024-09-21T15:00:00.315 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.315 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.315 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:00.315 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178553+0000", 2024-09-21T15:00:00.315 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036898528 2024-09-21T15:00:00.315 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.315 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.316 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:00.316 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178625+0000", 2024-09-21T15:00:00.316 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2669999999999994e-05 2024-09-21T15:00:00.316 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.316 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.316 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:00.316 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179001+0000", 2024-09-21T15:00:00.316 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000375977 2024-09-21T15:00:00.316 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:00.316 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:00.316 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:00.316 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.316 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.316 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22747 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 638976~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T15:00:00.317 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141660+0000", 2024-09-21T15:00:00.317 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 70.116854621000002, 2024-09-21T15:00:00.317 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037617352999999999, 2024-09-21T15:00:00.317 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:00.317 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:00.317 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:00.317 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:00.317 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:00.317 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22747 2024-09-21T15:00:00.317 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.317 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:00.317 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.317 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:00.317 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141660+0000", 2024-09-21T15:00:00.317 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:00.318 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.318 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.318 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:00.318 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141660+0000", 2024-09-21T15:00:00.318 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:00.318 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.318 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.318 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:00.318 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141660+0000", 2024-09-21T15:00:00.318 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:00.318 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.318 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.318 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:00.318 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141663+0000", 2024-09-21T15:00:00.318 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.569e-06 2024-09-21T15:00:00.319 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.319 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.319 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:00.319 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141670+0000", 2024-09-21T15:00:00.319 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5719999999999999e-06 2024-09-21T15:00:00.319 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.319 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.319 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:00.319 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141675+0000", 2024-09-21T15:00:00.319 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2889999999999997e-06 2024-09-21T15:00:00.319 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.319 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.319 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:00.320 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179028+0000", 2024-09-21T15:00:00.320 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037353381999999997 2024-09-21T15:00:00.320 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.320 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.320 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:00.320 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179099+0000", 2024-09-21T15:00:00.320 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0259000000000006e-05 2024-09-21T15:00:00.320 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.320 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.320 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:00.320 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179278+0000", 2024-09-21T15:00:00.320 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000178908 2024-09-21T15:00:00.320 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:00.320 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:00.321 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:00.321 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.321 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.321 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22748 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 704512~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T15:00:00.321 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141680+0000", 2024-09-21T15:00:00.321 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 70.116834983000004, 2024-09-21T15:00:00.321 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037806166000000002, 2024-09-21T15:00:00.321 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:00.321 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:00.321 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:00.321 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:00.321 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:00.321 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22748 2024-09-21T15:00:00.322 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.322 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:00.322 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.322 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:00.322 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141680+0000", 2024-09-21T15:00:00.322 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:00.322 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.322 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.322 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:00.322 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141680+0000", 2024-09-21T15:00:00.322 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:00.322 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.322 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.323 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:00.323 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141679+0000", 2024-09-21T15:00:00.323 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:00.323 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.323 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.323 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:00.323 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141683+0000", 2024-09-21T15:00:00.323 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3230000000000002e-06 2024-09-21T15:00:00.323 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.323 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.323 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:00.323 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141690+0000", 2024-09-21T15:00:00.323 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8680000000000001e-06 2024-09-21T15:00:00.323 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.324 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.324 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:00.324 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141694+0000", 2024-09-21T15:00:00.324 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6770000000000003e-06 2024-09-21T15:00:00.324 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.324 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.324 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:00.324 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179300+0000", 2024-09-21T15:00:00.324 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037605567999999999 2024-09-21T15:00:00.324 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.324 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.324 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:00.324 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179355+0000", 2024-09-21T15:00:00.324 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4771000000000001e-05 2024-09-21T15:00:00.325 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.325 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.325 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:00.325 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179486+0000", 2024-09-21T15:00:00.325 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013141699999999999 2024-09-21T15:00:00.325 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:00.325 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:00.325 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:00.325 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.325 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.325 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22749 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 770048~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T15:00:00.325 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141698+0000", 2024-09-21T15:00:00.326 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 70.116816611000004, 2024-09-21T15:00:00.326 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037967813000000003, 2024-09-21T15:00:00.326 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:00.326 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:00.326 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:00.326 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:00.326 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:00.326 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22749 2024-09-21T15:00:00.326 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.326 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:00.326 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.326 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:00.326 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141698+0000", 2024-09-21T15:00:00.326 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:00.327 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.327 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.327 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:00.327 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141698+0000", 2024-09-21T15:00:00.327 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:00.327 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.327 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.327 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:00.327 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141698+0000", 2024-09-21T15:00:00.327 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:00.327 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.327 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.327 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:00.328 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141701+0000", 2024-09-21T15:00:00.328 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8399999999999999e-06 2024-09-21T15:00:00.328 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.328 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.328 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:00.328 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141706+0000", 2024-09-21T15:00:00.328 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4979999999999997e-06 2024-09-21T15:00:00.328 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.328 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.328 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:00.328 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141711+0000", 2024-09-21T15:00:00.328 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3850000000000004e-06 2024-09-21T15:00:00.328 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.328 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.328 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:00.329 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179504+0000", 2024-09-21T15:00:00.329 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037793514 2024-09-21T15:00:00.329 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.329 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.329 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:00.329 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179546+0000", 2024-09-21T15:00:00.329 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1467999999999997e-05 2024-09-21T15:00:00.329 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.329 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.329 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:00.329 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179666+0000", 2024-09-21T15:00:00.329 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000120499 2024-09-21T15:00:00.329 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:00.329 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:00.329 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:00.330 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.330 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.330 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22750 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 835584~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T15:00:00.330 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141715+0000", 2024-09-21T15:00:00.330 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 70.116800310000002, 2024-09-21T15:00:00.330 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038173179000000002, 2024-09-21T15:00:00.330 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:00.330 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:00.330 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:00.330 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:00.330 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:00.330 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22750 2024-09-21T15:00:00.330 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.330 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:00.331 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.331 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:00.331 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141715+0000", 2024-09-21T15:00:00.331 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:00.331 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.331 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.331 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:00.331 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141715+0000", 2024-09-21T15:00:00.331 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:00.331 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.331 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.331 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:00.331 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141714+0000", 2024-09-21T15:00:00.331 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:00.332 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.332 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.332 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:00.332 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141716+0000", 2024-09-21T15:00:00.332 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1730000000000002e-06 2024-09-21T15:00:00.332 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.332 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.332 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:00.332 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141722+0000", 2024-09-21T15:00:00.332 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.417e-06 2024-09-21T15:00:00.332 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.332 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.332 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:00.333 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141726+0000", 2024-09-21T15:00:00.333 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4830000000000001e-06 2024-09-21T15:00:00.333 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.333 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.333 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:00.333 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179683+0000", 2024-09-21T15:00:00.333 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037956977000000003 2024-09-21T15:00:00.333 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.333 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.333 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:00.333 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179751+0000", 2024-09-21T15:00:00.333 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7762000000000006e-05 2024-09-21T15:00:00.333 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.333 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.334 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:00.334 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179888+0000", 2024-09-21T15:00:00.334 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000136635 2024-09-21T15:00:00.334 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:00.334 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:00.334 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:00.334 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.334 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.334 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22751 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 901120~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T15:00:00.334 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141730+0000", 2024-09-21T15:00:00.334 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 70.116784511000006, 2024-09-21T15:00:00.334 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038357011000000003, 2024-09-21T15:00:00.334 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:00.334 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:00.335 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:00.335 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:00.335 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:00.335 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22751 2024-09-21T15:00:00.335 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.335 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:00.335 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.335 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:00.335 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141730+0000", 2024-09-21T15:00:00.335 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:00.335 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.335 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.335 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:00.336 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141730+0000", 2024-09-21T15:00:00.336 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:00.336 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.336 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.336 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:00.336 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141730+0000", 2024-09-21T15:00:00.336 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:00.336 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.336 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.336 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:00.336 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141737+0000", 2024-09-21T15:00:00.336 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.64e-06 2024-09-21T15:00:00.336 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.336 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.337 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:00.337 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141743+0000", 2024-09-21T15:00:00.337 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8130000000000001e-06 2024-09-21T15:00:00.337 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.337 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.337 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:00.337 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141747+0000", 2024-09-21T15:00:00.337 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4920000000000004e-06 2024-09-21T15:00:00.337 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.337 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.337 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:00.337 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179918+0000", 2024-09-21T15:00:00.337 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038170822 2024-09-21T15:00:00.337 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.337 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.338 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:00.338 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179960+0000", 2024-09-21T15:00:00.338 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1999000000000002e-05 2024-09-21T15:00:00.338 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.338 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.338 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:00.338 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180087+0000", 2024-09-21T15:00:00.338 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012752799999999999 2024-09-21T15:00:00.338 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:00.338 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:00.338 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:00.338 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.338 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.338 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22752 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 966656~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T15:00:00.338 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141751+0000", 2024-09-21T15:00:00.339 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 70.116764191000001, 2024-09-21T15:00:00.339 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038553084000000001, 2024-09-21T15:00:00.339 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:00.339 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:00.339 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:00.339 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:00.339 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:00.339 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22752 2024-09-21T15:00:00.339 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.339 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:00.340 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.340 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:00.340 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141751+0000", 2024-09-21T15:00:00.340 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:00.340 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.340 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.340 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:00.340 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141751+0000", 2024-09-21T15:00:00.340 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:00.340 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.341 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.341 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:00.341 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141750+0000", 2024-09-21T15:00:00.341 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:00.341 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.341 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.342 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:00.342 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141753+0000", 2024-09-21T15:00:00.342 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5900000000000002e-06 2024-09-21T15:00:00.342 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.342 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.342 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:00.342 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141759+0000", 2024-09-21T15:00:00.342 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.947e-06 2024-09-21T15:00:00.342 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.342 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.342 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:00.343 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141764+0000", 2024-09-21T15:00:00.343 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8319999999999996e-06 2024-09-21T15:00:00.343 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.343 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.343 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:00.343 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180108+0000", 2024-09-21T15:00:00.343 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038344095000000002 2024-09-21T15:00:00.343 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.343 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.343 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:00.343 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180168+0000", 2024-09-21T15:00:00.343 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0047000000000003e-05 2024-09-21T15:00:00.343 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.343 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.344 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:00.344 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180304+0000", 2024-09-21T15:00:00.344 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013592500000000001 2024-09-21T15:00:00.344 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:00.344 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:00.344 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:00.344 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.344 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.344 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22753 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 1032192~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T15:00:00.344 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141768+0000", 2024-09-21T15:00:00.344 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 70.116747317000005, 2024-09-21T15:00:00.344 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038818810000000002, 2024-09-21T15:00:00.344 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:00.344 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:00.344 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:00.345 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:00.345 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:00.345 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22753 2024-09-21T15:00:00.345 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.345 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:00.345 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.345 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:00.345 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141768+0000", 2024-09-21T15:00:00.345 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:00.345 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.345 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.345 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:00.345 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141768+0000", 2024-09-21T15:00:00.345 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:00.346 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.346 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.346 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:00.346 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141767+0000", 2024-09-21T15:00:00.346 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:00.346 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.346 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.346 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:00.346 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141770+0000", 2024-09-21T15:00:00.346 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3329999999999999e-06 2024-09-21T15:00:00.346 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.346 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.346 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:00.346 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141775+0000", 2024-09-21T15:00:00.347 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2669999999999997e-06 2024-09-21T15:00:00.347 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.347 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.347 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:00.347 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141779+0000", 2024-09-21T15:00:00.347 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2259999999999999e-06 2024-09-21T15:00:00.347 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.347 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.347 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:00.347 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180325+0000", 2024-09-21T15:00:00.347 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038546428000000001 2024-09-21T15:00:00.347 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.347 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.347 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:00.348 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180401+0000", 2024-09-21T15:00:00.348 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5437000000000003e-05 2024-09-21T15:00:00.348 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.348 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.348 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:00.348 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180586+0000", 2024-09-21T15:00:00.348 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000185429 2024-09-21T15:00:00.348 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:00.348 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:00.348 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:00.348 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.348 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.348 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:73793 3.11 3:8c8a112d:::rbd_data.1136f4939602.0000000000000003:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-21T15:00:00.348 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:42.225633+0000", 2024-09-21T15:00:00.348 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.032881403000001, 2024-09-21T15:00:00.349 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036337102000000003, 2024-09-21T15:00:00.349 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:00.349 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:00.349 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:00.349 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:00.349 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:00.349 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 73793 2024-09-21T15:00:00.349 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.349 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:00.349 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.349 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:00.349 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.225633+0000", 2024-09-21T15:00:00.349 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:00.349 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.349 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.350 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:00.350 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.225633+0000", 2024-09-21T15:00:00.350 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:00.350 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.350 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.350 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:00.350 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.225630+0000", 2024-09-21T15:00:00.350 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999967 2024-09-21T15:00:00.350 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.350 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.350 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:00.350 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.225649+0000", 2024-09-21T15:00:00.350 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9377000000000001e-05 2024-09-21T15:00:00.351 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.351 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.351 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:00.351 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.225689+0000", 2024-09-21T15:00:00.351 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.9879999999999998e-05 2024-09-21T15:00:00.351 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.351 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.351 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:00.351 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.225706+0000", 2024-09-21T15:00:00.351 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6537000000000001e-05 2024-09-21T15:00:00.351 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.351 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.351 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:00.351 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.225772+0000", 2024-09-21T15:00:00.351 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5894000000000003e-05 2024-09-21T15:00:00.352 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.352 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.352 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:00.352 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.225846+0000", 2024-09-21T15:00:00.352 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4165000000000006e-05 2024-09-21T15:00:00.352 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.352 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.352 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 1", 2024-09-21T15:00:00.352 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.231400+0000", 2024-09-21T15:00:00.352 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0055538269999999999 2024-09-21T15:00:00.352 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.352 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.352 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:00.352 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.261709+0000", 2024-09-21T15:00:00.353 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.030309750999999999 2024-09-21T15:00:00.353 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.353 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.353 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:00.353 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.261738+0000", 2024-09-21T15:00:00.353 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8569999999999999e-05 2024-09-21T15:00:00.353 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.353 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.353 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:00.353 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.261839+0000", 2024-09-21T15:00:00.353 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010118 2024-09-21T15:00:00.353 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:00.353 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:00.353 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:00.353 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.261971+0000", 2024-09-21T15:00:00.353 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000131414 2024-09-21T15:00:00.354 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:00.354 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:00.354 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:00.354 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:00.354 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:00.354 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:00.355 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T15:00:00.356 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:00:00.353+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:00:00.456 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:00:00.453+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:00:00.461 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:00.461 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:00:00.461 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T15:00:00.461 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:00.471 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T15:00:00.556 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:00:00.557+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:00:00.574 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:00.574 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:00:00.575 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T15:00:00.575 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T15:00:00.575 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:00.584 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T15:00:00.658 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:00:00.654+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:00:00.695 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:00.695 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T15:00:00.695 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T15:00:00.695 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T15:00:00.695 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.695 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:1452 3.19 3:9d40fe72:::rbd_data.1136515ecb6b.0000000000000013:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T15:00:00.695 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:30.292755+0000", 2024-09-21T15:00:00.695 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 90.400308175999996, 2024-09-21T15:00:00.695 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043852191999999998, 2024-09-21T15:00:00.695 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:00.695 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:00.696 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:00.696 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:00.696 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:00.696 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1452 2024-09-21T15:00:00.696 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.696 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:00.696 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.696 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:00.696 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292755+0000", 2024-09-21T15:00:00.696 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:00.696 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.696 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.696 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:00.696 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292755+0000", 2024-09-21T15:00:00.697 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:00.697 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.697 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.697 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:00.697 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292754+0000", 2024-09-21T15:00:00.697 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:00:00.697 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.697 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.697 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:00.697 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292760+0000", 2024-09-21T15:00:00.697 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5029999999999997e-06 2024-09-21T15:00:00.697 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.697 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.697 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:00.697 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292771+0000", 2024-09-21T15:00:00.698 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1551e-05 2024-09-21T15:00:00.698 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.698 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.698 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:00.698 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292779+0000", 2024-09-21T15:00:00.698 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1559999999999998e-06 2024-09-21T15:00:00.698 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.698 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.698 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:00.698 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292896+0000", 2024-09-21T15:00:00.698 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000117304 2024-09-21T15:00:00.698 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.698 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.698 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:00.699 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.293124+0000", 2024-09-21T15:00:00.699 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00022819999999999999 2024-09-21T15:00:00.699 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.699 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.699 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 1", 2024-09-21T15:00:00.699 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.299967+0000", 2024-09-21T15:00:00.699 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0068427009999999996 2024-09-21T15:00:00.699 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.699 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.699 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:00.699 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336463+0000", 2024-09-21T15:00:00.699 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.036495842000000001 2024-09-21T15:00:00.699 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.699 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.699 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:00.700 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336480+0000", 2024-09-21T15:00:00.700 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7419999999999999e-05 2024-09-21T15:00:00.700 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.700 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.700 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:00.700 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336550+0000", 2024-09-21T15:00:00.700 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9951999999999997e-05 2024-09-21T15:00:00.700 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.700 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.700 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:00.700 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336607+0000", 2024-09-21T15:00:00.700 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7423000000000002e-05 2024-09-21T15:00:00.700 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:00.701 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:00.701 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:00.701 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.701 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.701 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:1567 3.8 3:139d3d55:::rbd_data.1136515ecb6b.0000000000000000:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T15:00:00.701 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:30.681087+0000", 2024-09-21T15:00:00.701 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 90.011975961000005, 2024-09-21T15:00:00.701 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.039009224000000002, 2024-09-21T15:00:00.701 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:00.701 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:00.701 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:00.701 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:00.702 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:00.702 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1567 2024-09-21T15:00:00.702 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.702 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:00.702 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.702 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:00.702 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681087+0000", 2024-09-21T15:00:00.702 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:00.702 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.702 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.702 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:00.702 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681087+0000", 2024-09-21T15:00:00.702 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:00.703 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.703 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.703 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:00.703 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681086+0000", 2024-09-21T15:00:00.703 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:00:00.703 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.703 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.703 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:00.703 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681097+0000", 2024-09-21T15:00:00.703 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.134e-05 2024-09-21T15:00:00.703 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.703 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.703 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:00.703 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681122+0000", 2024-09-21T15:00:00.703 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5313e-05 2024-09-21T15:00:00.704 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.704 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.704 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:00.704 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681138+0000", 2024-09-21T15:00:00.704 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5916000000000001e-05 2024-09-21T15:00:00.704 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.704 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.704 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:00.704 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681270+0000", 2024-09-21T15:00:00.704 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000131474 2024-09-21T15:00:00.704 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.704 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.704 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:00.704 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681416+0000", 2024-09-21T15:00:00.704 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014637 2024-09-21T15:00:00.705 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.705 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.705 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 7", 2024-09-21T15:00:00.705 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.693163+0000", 2024-09-21T15:00:00.705 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011747241 2024-09-21T15:00:00.705 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.705 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.705 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:00.705 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.719750+0000", 2024-09-21T15:00:00.705 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.026586852000000001 2024-09-21T15:00:00.705 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.705 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.705 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:00.705 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.719859+0000", 2024-09-21T15:00:00.705 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010925999999999999 2024-09-21T15:00:00.705 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.706 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.706 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:00.706 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.719951+0000", 2024-09-21T15:00:00.706 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1294999999999999e-05 2024-09-21T15:00:00.706 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.706 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.706 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:00.706 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.720097+0000", 2024-09-21T15:00:00.706 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014583 2024-09-21T15:00:00.706 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:00.706 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:00.706 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:00.706 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.706 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.707 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9349 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 155648~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:00.707 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158403+0000", 2024-09-21T15:00:00.707 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 81.534660545999998, 2024-09-21T15:00:00.707 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014229851, 2024-09-21T15:00:00.707 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:00.707 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:00.707 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:00.708 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:00.708 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:00.708 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9349 2024-09-21T15:00:00.708 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.708 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:00.708 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.708 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:00.708 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158403+0000", 2024-09-21T15:00:00.708 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:00.708 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.708 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.708 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:00.708 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158403+0000", 2024-09-21T15:00:00.709 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:00.709 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.709 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.709 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:00.709 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158402+0000", 2024-09-21T15:00:00.709 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:00.709 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.709 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.709 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:00.709 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158406+0000", 2024-09-21T15:00:00.709 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1339999999999997e-06 2024-09-21T15:00:00.709 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.709 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.709 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:00.709 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158416+0000", 2024-09-21T15:00:00.710 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3149999999999998e-06 2024-09-21T15:00:00.710 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.710 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.710 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:00.710 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158439+0000", 2024-09-21T15:00:00.710 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3255999999999999e-05 2024-09-21T15:00:00.710 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.710 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.710 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:00.710 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172305+0000", 2024-09-21T15:00:00.710 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013866603999999999 2024-09-21T15:00:00.710 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.710 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.710 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:00.710 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172413+0000", 2024-09-21T15:00:00.711 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000107664 2024-09-21T15:00:00.711 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.711 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.711 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:00.711 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172633+0000", 2024-09-21T15:00:00.711 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021950800000000001 2024-09-21T15:00:00.711 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:00.711 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:00.711 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:00.711 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.711 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.711 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9350 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 294912~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:00.711 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158448+0000", 2024-09-21T15:00:00.711 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 81.534615674999998, 2024-09-21T15:00:00.711 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014471525000000001, 2024-09-21T15:00:00.711 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:00.712 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:00.712 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:00.712 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:00.712 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:00.712 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9350 2024-09-21T15:00:00.712 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.712 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:00.712 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.712 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:00.712 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158448+0000", 2024-09-21T15:00:00.712 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:00.712 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.712 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.712 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:00.713 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158448+0000", 2024-09-21T15:00:00.713 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:00.713 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.713 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.713 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:00.713 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158447+0000", 2024-09-21T15:00:00.713 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:00:00.713 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.713 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.713 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:00.713 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158452+0000", 2024-09-21T15:00:00.713 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8400000000000002e-06 2024-09-21T15:00:00.713 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.713 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.713 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:00.714 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158475+0000", 2024-09-21T15:00:00.714 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3091e-05 2024-09-21T15:00:00.714 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.714 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.714 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:00.714 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158515+0000", 2024-09-21T15:00:00.714 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0265000000000002e-05 2024-09-21T15:00:00.714 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.714 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.714 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:00.714 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172682+0000", 2024-09-21T15:00:00.714 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014166761999999999 2024-09-21T15:00:00.714 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.714 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.714 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:00.715 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172752+0000", 2024-09-21T15:00:00.715 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9698999999999998e-05 2024-09-21T15:00:00.715 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.715 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.715 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:00.715 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172919+0000", 2024-09-21T15:00:00.715 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016759199999999999 2024-09-21T15:00:00.715 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:00.715 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:00.715 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:00.715 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.715 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.715 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9351 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 434176~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:00.715 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158526+0000", 2024-09-21T15:00:00.716 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 81.534536957, 2024-09-21T15:00:00.716 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014662107000000001, 2024-09-21T15:00:00.716 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:00.716 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:00.716 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:00.716 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:00.716 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:00.716 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9351 2024-09-21T15:00:00.716 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.716 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:00.716 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.716 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:00.716 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158526+0000", 2024-09-21T15:00:00.716 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:00.716 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.716 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.717 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:00.717 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158526+0000", 2024-09-21T15:00:00.717 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:00.717 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.717 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.717 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:00.717 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158525+0000", 2024-09-21T15:00:00.717 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:00:00.717 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.717 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.717 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:00.717 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158531+0000", 2024-09-21T15:00:00.717 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5840000000000002e-06 2024-09-21T15:00:00.717 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.717 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.718 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:00.718 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158564+0000", 2024-09-21T15:00:00.718 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3195000000000003e-05 2024-09-21T15:00:00.718 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.718 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.718 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:00.718 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158577+0000", 2024-09-21T15:00:00.718 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2330000000000001e-05 2024-09-21T15:00:00.718 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.718 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.718 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:00.718 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.172956+0000", 2024-09-21T15:00:00.719 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014379438 2024-09-21T15:00:00.719 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.719 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.719 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:00.719 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173020+0000", 2024-09-21T15:00:00.719 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4401999999999998e-05 2024-09-21T15:00:00.719 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.719 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.719 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:00.719 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173188+0000", 2024-09-21T15:00:00.719 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016804 2024-09-21T15:00:00.719 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:00.719 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:00.719 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:00.719 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.720 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.720 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9379 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 24576~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:00.720 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158586+0000", 2024-09-21T15:00:00.720 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 81.534477697, 2024-09-21T15:00:00.720 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014865173000000001, 2024-09-21T15:00:00.720 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:00.720 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:00.720 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:00.720 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:00.720 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:00.720 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9379 2024-09-21T15:00:00.720 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.720 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:00.721 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.721 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:00.721 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158586+0000", 2024-09-21T15:00:00.721 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:00.721 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.721 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.721 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:00.721 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158586+0000", 2024-09-21T15:00:00.721 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:00.721 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.721 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.721 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:00.721 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158585+0000", 2024-09-21T15:00:00.721 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:00:00.722 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.722 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.722 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:00.722 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158590+0000", 2024-09-21T15:00:00.722 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5820000000000001e-06 2024-09-21T15:00:00.722 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.722 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.722 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:00.722 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158676+0000", 2024-09-21T15:00:00.722 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5075999999999997e-05 2024-09-21T15:00:00.722 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.722 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.722 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:00.723 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158695+0000", 2024-09-21T15:00:00.723 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9137999999999999e-05 2024-09-21T15:00:00.723 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.723 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.723 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:00.723 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173219+0000", 2024-09-21T15:00:00.723 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014524034999999999 2024-09-21T15:00:00.723 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.723 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.723 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:00.723 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173276+0000", 2024-09-21T15:00:00.723 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7442000000000003e-05 2024-09-21T15:00:00.723 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.723 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.724 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:00.724 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173451+0000", 2024-09-21T15:00:00.724 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017464800000000001 2024-09-21T15:00:00.724 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:00.724 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:00.724 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:00.724 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.724 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.724 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9380 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 163840~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:00.724 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158706+0000", 2024-09-21T15:00:00.724 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 81.534357186999998, 2024-09-21T15:00:00.724 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015096381000000001, 2024-09-21T15:00:00.724 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:00.724 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:00.725 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:00.725 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:00.725 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:00.725 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9380 2024-09-21T15:00:00.725 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.725 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:00.725 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.725 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:00.725 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158706+0000", 2024-09-21T15:00:00.725 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:00.725 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.725 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.725 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:00.725 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158706+0000", 2024-09-21T15:00:00.725 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:00.726 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.726 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.726 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:00.726 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158705+0000", 2024-09-21T15:00:00.726 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:00:00.726 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.726 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.726 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:00.726 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158712+0000", 2024-09-21T15:00:00.726 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7240000000000003e-06 2024-09-21T15:00:00.726 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.726 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.727 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:00.727 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158755+0000", 2024-09-21T15:00:00.727 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2953e-05 2024-09-21T15:00:00.727 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.727 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.727 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:00.727 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158765+0000", 2024-09-21T15:00:00.727 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0149000000000001e-05 2024-09-21T15:00:00.727 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.727 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.727 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:00.727 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173492+0000", 2024-09-21T15:00:00.727 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014727105000000001 2024-09-21T15:00:00.727 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.727 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.727 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:00.728 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173564+0000", 2024-09-21T15:00:00.728 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1789e-05 2024-09-21T15:00:00.728 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.728 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.728 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:00.728 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173803+0000", 2024-09-21T15:00:00.728 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00023867100000000001 2024-09-21T15:00:00.728 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:00.728 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:00.728 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:00.728 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.728 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.728 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9381 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 303104~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:00.729 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158774+0000", 2024-09-21T15:00:00.729 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 81.534289107000006, 2024-09-21T15:00:00.729 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015309095999999999, 2024-09-21T15:00:00.729 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:00.729 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:00.729 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:00.729 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:00.729 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:00.729 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9381 2024-09-21T15:00:00.729 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.729 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:00.729 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.729 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:00.729 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158774+0000", 2024-09-21T15:00:00.729 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:00.730 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.730 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.730 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:00.730 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158774+0000", 2024-09-21T15:00:00.730 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:00.730 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.730 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.730 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:00.730 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158773+0000", 2024-09-21T15:00:00.730 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:00:00.730 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.730 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.730 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:00.730 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158779+0000", 2024-09-21T15:00:00.731 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6999999999999996e-06 2024-09-21T15:00:00.731 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.731 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.731 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:00.731 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158790+0000", 2024-09-21T15:00:00.731 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0594000000000001e-05 2024-09-21T15:00:00.731 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.731 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.731 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:00.731 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158798+0000", 2024-09-21T15:00:00.731 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5860000000000005e-06 2024-09-21T15:00:00.731 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.731 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.731 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:00.732 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173829+0000", 2024-09-21T15:00:00.732 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015030349 2024-09-21T15:00:00.732 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.732 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.732 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:00.732 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173918+0000", 2024-09-21T15:00:00.732 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9375000000000004e-05 2024-09-21T15:00:00.732 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.732 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.732 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:00.732 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.174083+0000", 2024-09-21T15:00:00.732 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016536200000000001 2024-09-21T15:00:00.732 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:00.732 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:00.733 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:00.733 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.733 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.733 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9382 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 442368~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:00.733 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158805+0000", 2024-09-21T15:00:00.733 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 81.534258202000004, 2024-09-21T15:00:00.733 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015592623, 2024-09-21T15:00:00.733 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:00.733 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:00.733 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:00.733 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:00.733 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:00.733 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9382 2024-09-21T15:00:00.733 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.734 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:00.734 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.734 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:00.734 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158805+0000", 2024-09-21T15:00:00.734 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:00.734 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.734 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.734 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:00.734 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158805+0000", 2024-09-21T15:00:00.734 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:00.734 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.734 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.734 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:00.734 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158805+0000", 2024-09-21T15:00:00.735 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:00.735 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.735 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.735 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:00.735 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158808+0000", 2024-09-21T15:00:00.735 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9179999999999996e-06 2024-09-21T15:00:00.735 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.735 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.735 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:00.735 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158817+0000", 2024-09-21T15:00:00.735 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9299999999999992e-06 2024-09-21T15:00:00.735 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.735 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.735 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:00.735 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158825+0000", 2024-09-21T15:00:00.736 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6739999999999997e-06 2024-09-21T15:00:00.736 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.736 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.736 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:00.736 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.174122+0000", 2024-09-21T15:00:00.736 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.01529694 2024-09-21T15:00:00.736 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.736 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.736 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:00.736 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.174185+0000", 2024-09-21T15:00:00.736 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3015000000000006e-05 2024-09-21T15:00:00.736 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.736 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.736 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:00.737 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.174398+0000", 2024-09-21T15:00:00.737 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021274799999999999 2024-09-21T15:00:00.737 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:00.737 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:00.737 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:00.737 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.737 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.737 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22701 3.19 3:98de45b8:::rbd_data.11366acc894.0000000000000000:19 [read 770048~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T15:00:00.737 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.140215+0000", 2024-09-21T15:00:00.737 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 70.552848771000001, 2024-09-21T15:00:00.737 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014072497, 2024-09-21T15:00:00.738 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:00.738 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:00.738 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:00.738 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:00.738 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:00.738 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22701 2024-09-21T15:00:00.738 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.738 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:00.738 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.738 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:00.739 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140215+0000", 2024-09-21T15:00:00.739 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:00.739 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.739 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.739 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:00.739 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140215+0000", 2024-09-21T15:00:00.739 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:00.739 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.739 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.739 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:00.739 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140214+0000", 2024-09-21T15:00:00.739 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:00.739 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.739 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.739 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:00.739 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140217+0000", 2024-09-21T15:00:00.740 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.74e-06 2024-09-21T15:00:00.740 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.740 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.740 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:00.740 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140223+0000", 2024-09-21T15:00:00.740 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0929999999999999e-06 2024-09-21T15:00:00.740 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.740 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.740 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:00.740 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140228+0000", 2024-09-21T15:00:00.741 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2279999999999998e-06 2024-09-21T15:00:00.741 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.741 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.741 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:00.741 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.153991+0000", 2024-09-21T15:00:00.741 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013762329 2024-09-21T15:00:00.741 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.741 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.741 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:00.742 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.154098+0000", 2024-09-21T15:00:00.742 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000107117 2024-09-21T15:00:00.742 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.742 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.742 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:00.742 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.154287+0000", 2024-09-21T15:00:00.742 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000189431 2024-09-21T15:00:00.742 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:00.742 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:00.742 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:00.742 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.742 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.742 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22702 3.19 3:98de45b8:::rbd_data.11366acc894.0000000000000000:19 [read 835584~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T15:00:00.742 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.140232+0000", 2024-09-21T15:00:00.742 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 70.552830919000002, 2024-09-21T15:00:00.743 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015338742000000001, 2024-09-21T15:00:00.743 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:00.743 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:00.743 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:00.743 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:00.743 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:00.743 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22702 2024-09-21T15:00:00.743 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.743 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:00.743 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.743 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:00.743 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140232+0000", 2024-09-21T15:00:00.743 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:00.743 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.743 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.744 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:00.744 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140232+0000", 2024-09-21T15:00:00.744 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:00.744 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.744 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.744 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:00.744 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140232+0000", 2024-09-21T15:00:00.744 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:00.744 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.744 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.744 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:00.744 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140234+0000", 2024-09-21T15:00:00.745 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.402e-06 2024-09-21T15:00:00.745 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.745 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.745 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:00.745 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140241+0000", 2024-09-21T15:00:00.745 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.054e-06 2024-09-21T15:00:00.745 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.745 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.745 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:00.745 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140246+0000", 2024-09-21T15:00:00.745 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0050000000000004e-06 2024-09-21T15:00:00.745 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.745 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.746 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:00.746 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.154315+0000", 2024-09-21T15:00:00.746 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014069440000000001 2024-09-21T15:00:00.746 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.746 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.746 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:00.746 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.154383+0000", 2024-09-21T15:00:00.746 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8151000000000005e-05 2024-09-21T15:00:00.746 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.747 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.747 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:00.747 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155571+0000", 2024-09-21T15:00:00.747 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001188029 2024-09-21T15:00:00.747 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:00.747 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:00.748 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:00.748 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.748 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.748 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22703 3.19 3:98de45b8:::rbd_data.11366acc894.0000000000000000:19 [read 901120~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T15:00:00.748 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.140250+0000", 2024-09-21T15:00:00.748 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 70.552813760999996, 2024-09-21T15:00:00.748 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014992169, 2024-09-21T15:00:00.748 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:00.748 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:00.748 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:00.748 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:00.749 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:00.749 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22703 2024-09-21T15:00:00.749 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.749 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:00.749 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.749 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:00.749 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140250+0000", 2024-09-21T15:00:00.749 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:00.749 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.749 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.749 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:00.749 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140250+0000", 2024-09-21T15:00:00.749 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:00.750 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.750 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.750 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:00.750 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140249+0000", 2024-09-21T15:00:00.750 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:00.750 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.750 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.750 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:00.750 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140251+0000", 2024-09-21T15:00:00.750 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.136e-06 2024-09-21T15:00:00.750 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.750 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.750 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:00.751 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140257+0000", 2024-09-21T15:00:00.751 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9610000000000002e-06 2024-09-21T15:00:00.751 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.751 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.751 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:00.751 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140262+0000", 2024-09-21T15:00:00.751 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4549999999999997e-06 2024-09-21T15:00:00.751 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.751 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.751 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:00.751 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155006+0000", 2024-09-21T15:00:00.751 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014744189 2024-09-21T15:00:00.751 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.752 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.752 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:00.752 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155081+0000", 2024-09-21T15:00:00.752 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4548000000000005e-05 2024-09-21T15:00:00.752 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.752 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.752 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:00.752 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155242+0000", 2024-09-21T15:00:00.752 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000161204 2024-09-21T15:00:00.752 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:00.752 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:00.752 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:00.752 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.752 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.752 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22704 3.19 3:98de45b8:::rbd_data.11366acc894.0000000000000000:19 [read 966656~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T15:00:00.753 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.140266+0000", 2024-09-21T15:00:00.753 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 70.552797444000007, 2024-09-21T15:00:00.753 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015223443999999999, 2024-09-21T15:00:00.753 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:00.753 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:00.753 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:00.753 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:00.753 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:00.753 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22704 2024-09-21T15:00:00.753 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.753 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:00.753 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.753 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:00.753 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140266+0000", 2024-09-21T15:00:00.754 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:00.754 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.754 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.754 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:00.754 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140266+0000", 2024-09-21T15:00:00.754 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:00.754 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.754 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.754 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:00.754 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140266+0000", 2024-09-21T15:00:00.754 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:00.754 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.754 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.754 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:00.755 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140268+0000", 2024-09-21T15:00:00.755 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.475e-06 2024-09-21T15:00:00.755 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.755 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.755 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:00.755 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140274+0000", 2024-09-21T15:00:00.755 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.319e-06 2024-09-21T15:00:00.755 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.755 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.755 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:00.755 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140279+0000", 2024-09-21T15:00:00.755 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.882e-06 2024-09-21T15:00:00.755 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.755 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.756 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:00.756 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155264+0000", 2024-09-21T15:00:00.756 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014984655 2024-09-21T15:00:00.756 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.756 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.756 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:00.756 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155338+0000", 2024-09-21T15:00:00.756 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4091999999999995e-05 2024-09-21T15:00:00.756 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.756 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.756 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:00.756 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155489+0000", 2024-09-21T15:00:00.756 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000151349 2024-09-21T15:00:00.756 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:00.757 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:00.757 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:00.757 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.757 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.757 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22705 3.19 3:98de45b8:::rbd_data.11366acc894.0000000000000000:19 [read 1032192~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T15:00:00.757 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.140284+0000", 2024-09-21T15:00:00.757 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 70.552779587000003, 2024-09-21T15:00:00.757 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015467896, 2024-09-21T15:00:00.757 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:00.757 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:00.757 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:00.757 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:00.757 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:00.758 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22705 2024-09-21T15:00:00.758 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.758 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:00.758 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.758 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:00.758 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140284+0000", 2024-09-21T15:00:00.758 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:00.758 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.758 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.758 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:00.758 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140284+0000", 2024-09-21T15:00:00.758 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:00.759 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.759 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.759 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:00.759 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140283+0000", 2024-09-21T15:00:00.759 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:00.759 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.759 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.759 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:00.759 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140286+0000", 2024-09-21T15:00:00.759 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7130000000000001e-06 2024-09-21T15:00:00.759 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.759 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.759 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:00.760 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140292+0000", 2024-09-21T15:00:00.760 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3430000000000001e-06 2024-09-21T15:00:00.760 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.760 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.760 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:00.760 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140298+0000", 2024-09-21T15:00:00.760 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9730000000000002e-06 2024-09-21T15:00:00.760 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.760 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.760 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:00.760 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155510+0000", 2024-09-21T15:00:00.760 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015211738000000001 2024-09-21T15:00:00.760 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.760 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.761 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:00.761 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155598+0000", 2024-09-21T15:00:00.761 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7822e-05 2024-09-21T15:00:00.761 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.761 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.761 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:00.761 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155752+0000", 2024-09-21T15:00:00.761 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0001537 2024-09-21T15:00:00.761 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:00.761 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:00.761 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:00.761 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.761 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.761 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:37892 3.1b 3:d963c97f:::rbd_data.11365d19aa81.00000000000000be:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-21T15:00:00.762 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:04.029946+0000", 2024-09-21T15:00:00.762 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 56.663117819999997, 2024-09-21T15:00:00.762 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.017686021999999999, 2024-09-21T15:00:00.762 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:00.762 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:00.762 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:00.762 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:00.762 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:00.762 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 37892 2024-09-21T15:00:00.762 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.762 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:00.762 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.762 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:00.763 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.029946+0000", 2024-09-21T15:00:00.763 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:00.763 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.763 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.763 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:00.763 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.029946+0000", 2024-09-21T15:00:00.763 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:00.763 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.763 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.763 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:00.763 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.029944+0000", 2024-09-21T15:00:00.763 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:00:00.763 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.764 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.764 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:00.764 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.029953+0000", 2024-09-21T15:00:00.764 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7589999999999992e-06 2024-09-21T15:00:00.764 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.764 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.764 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:00.764 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.029971+0000", 2024-09-21T15:00:00.764 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7795000000000002e-05 2024-09-21T15:00:00.764 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.764 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.764 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:00.765 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.029984+0000", 2024-09-21T15:00:00.765 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.27e-05 2024-09-21T15:00:00.765 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.765 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.765 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:00.765 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.030072+0000", 2024-09-21T15:00:00.765 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8082000000000001e-05 2024-09-21T15:00:00.765 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.765 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.765 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:00.765 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.030276+0000", 2024-09-21T15:00:00.765 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00020375699999999999 2024-09-21T15:00:00.765 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.765 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.765 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-21T15:00:00.766 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.031634+0000", 2024-09-21T15:00:00.766 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0013586049999999999 2024-09-21T15:00:00.766 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.766 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.766 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:00.766 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.037554+0000", 2024-09-21T15:00:00.766 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0059197449999999997 2024-09-21T15:00:00.766 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.766 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.766 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:00.766 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.047490+0000", 2024-09-21T15:00:00.766 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0099361680000000004 2024-09-21T15:00:00.766 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.766 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.767 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:00.767 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.047569+0000", 2024-09-21T15:00:00.767 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8298000000000001e-05 2024-09-21T15:00:00.767 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.767 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.767 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:00.767 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.047632+0000", 2024-09-21T15:00:00.767 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3006e-05 2024-09-21T15:00:00.767 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:00.767 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:00.767 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:00.767 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.768 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.768 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:63820 3.19 3.626bc079 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-21T15:00:00.768 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:27.241293+0000", 2024-09-21T15:00:00.768 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 33.451770727000003, 2024-09-21T15:00:00.768 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.98723808199999996, 2024-09-21T15:00:00.768 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:00.768 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:00.768 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:00.768 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:00.768 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:00.768 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 63820 2024-09-21T15:00:00.768 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.768 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:00.768 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.769 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:00.769 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241293+0000", 2024-09-21T15:00:00.769 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:00.769 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.769 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.769 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:00.769 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241293+0000", 2024-09-21T15:00:00.769 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:00.769 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.769 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.769 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:00.769 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241291+0000", 2024-09-21T15:00:00.769 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:00:00.769 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.770 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.770 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:00.770 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241299+0000", 2024-09-21T15:00:00.770 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1529999999999997e-06 2024-09-21T15:00:00.770 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.770 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.770 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:00.770 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241317+0000", 2024-09-21T15:00:00.770 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7764000000000001e-05 2024-09-21T15:00:00.770 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.770 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.770 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:00.770 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241328+0000", 2024-09-21T15:00:00.770 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1245e-05 2024-09-21T15:00:00.771 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.771 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.771 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:00.771 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241407+0000", 2024-09-21T15:00:00.771 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8762000000000002e-05 2024-09-21T15:00:00.771 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.771 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.771 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:00:00.771 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241418+0000", 2024-09-21T15:00:00.771 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1695000000000001e-05 2024-09-21T15:00:00.771 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.771 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.771 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:00.771 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:28.228510+0000", 2024-09-21T15:00:00.772 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.98709166100000001 2024-09-21T15:00:00.772 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.772 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.772 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:00.772 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:28.228531+0000", 2024-09-21T15:00:00.772 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0696e-05 2024-09-21T15:00:00.772 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:00.772 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:00.772 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:00.772 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.772 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.772 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:73845 3.6 3:601bd01e:::rbd_data.1136f4939602.0000000000000015:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-21T15:00:00.772 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:42.393435+0000", 2024-09-21T15:00:00.772 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 18.299628616, 2024-09-21T15:00:00.772 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.030713345, 2024-09-21T15:00:00.773 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:00.773 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:00.773 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:00.773 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:00.773 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:00.773 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 73845 2024-09-21T15:00:00.773 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.773 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:00.773 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.773 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:00.773 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.393435+0000", 2024-09-21T15:00:00.773 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:00.773 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.773 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.774 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:00.774 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.393435+0000", 2024-09-21T15:00:00.774 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:00.774 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.774 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.774 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:00.774 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.393433+0000", 2024-09-21T15:00:00.774 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:00:00.774 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.774 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.774 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:00.774 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.393446+0000", 2024-09-21T15:00:00.774 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3013000000000001e-05 2024-09-21T15:00:00.775 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.775 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.775 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:00.775 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.393475+0000", 2024-09-21T15:00:00.775 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9615999999999999e-05 2024-09-21T15:00:00.775 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.775 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.775 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:00.775 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.393489+0000", 2024-09-21T15:00:00.775 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4103e-05 2024-09-21T15:00:00.775 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.775 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.775 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:00.775 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.393565+0000", 2024-09-21T15:00:00.775 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5195000000000001e-05 2024-09-21T15:00:00.776 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.776 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.776 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:00.776 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.393734+0000", 2024-09-21T15:00:00.776 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016909999999999999 2024-09-21T15:00:00.776 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.776 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.776 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 7", 2024-09-21T15:00:00.776 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.400243+0000", 2024-09-21T15:00:00.776 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0065093629999999998 2024-09-21T15:00:00.776 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.777 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.777 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:00.777 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.423721+0000", 2024-09-21T15:00:00.777 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.023477984 2024-09-21T15:00:00.777 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.777 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.778 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:00.778 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.423852+0000", 2024-09-21T15:00:00.778 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000131369 2024-09-21T15:00:00.778 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.778 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.778 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:00.778 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.424011+0000", 2024-09-21T15:00:00.778 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000158231 2024-09-21T15:00:00.778 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.778 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.778 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:00.779 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.424148+0000", 2024-09-21T15:00:00.779 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013732699999999999 2024-09-21T15:00:00.779 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:00.779 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:00.779 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:00.779 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.779 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.779 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:85603 3.7 3:e5bd5c60:::rbd_object_map.113637a2ddc2.0000000000000044:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115)", 2024-09-21T15:00:00.779 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:56.162443+0000", 2024-09-21T15:00:00.779 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.5306204440000002, 2024-09-21T15:00:00.779 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013898304, 2024-09-21T15:00:00.779 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:00.779 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:00.779 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:00.780 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:00.780 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:00.780 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 85603 2024-09-21T15:00:00.780 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.780 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:00.780 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.780 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:00.780 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.162443+0000", 2024-09-21T15:00:00.780 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:00.780 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.780 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.780 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:00.780 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.162443+0000", 2024-09-21T15:00:00.780 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:00.780 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.781 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.781 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:00.781 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.162443+0000", 2024-09-21T15:00:00.781 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:00.781 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.781 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.781 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:00.781 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.162445+0000", 2024-09-21T15:00:00.781 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.88e-06 2024-09-21T15:00:00.781 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.781 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.781 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:00.781 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.162450+0000", 2024-09-21T15:00:00.781 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4020000000000004e-06 2024-09-21T15:00:00.782 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.782 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.782 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:00.782 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.162454+0000", 2024-09-21T15:00:00.782 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9770000000000002e-06 2024-09-21T15:00:00.782 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.782 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.782 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:00.782 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.170929+0000", 2024-09-21T15:00:00.782 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0084749170000000002 2024-09-21T15:00:00.782 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.782 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.782 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:00.785 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.171055+0000", 2024-09-21T15:00:00.786 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000126511 2024-09-21T15:00:00.786 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.786 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.786 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T15:00:00.786 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.171537+0000", 2024-09-21T15:00:00.786 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00048164399999999998 2024-09-21T15:00:00.786 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.786 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.786 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:00.786 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.175973+0000", 2024-09-21T15:00:00.786 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0044358089999999998 2024-09-21T15:00:00.786 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.787 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.787 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:00.787 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.176128+0000", 2024-09-21T15:00:00.787 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015554699999999999 2024-09-21T15:00:00.787 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.787 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.787 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:00.787 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.176299+0000", 2024-09-21T15:00:00.787 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017067699999999999 2024-09-21T15:00:00.787 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.787 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.787 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:00.787 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.176341+0000", 2024-09-21T15:00:00.788 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2265999999999998e-05 2024-09-21T15:00:00.788 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:00.788 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:00.788 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:00.788 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.788 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.788 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:85604 3.7 3:e5bd5c60:::rbd_object_map.113637a2ddc2.0000000000000044:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115)", 2024-09-21T15:00:00.788 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:56.162457+0000", 2024-09-21T15:00:00.788 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.5306059869999995, 2024-09-21T15:00:00.788 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014870204, 2024-09-21T15:00:00.788 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:00.788 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:00.788 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:00.788 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:00.789 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:00.789 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 85604 2024-09-21T15:00:00.789 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.789 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:00.789 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.789 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:00.789 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.162457+0000", 2024-09-21T15:00:00.789 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:00.789 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.789 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.789 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:00.789 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.162457+0000", 2024-09-21T15:00:00.789 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:00.789 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.790 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.790 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:00.790 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.162457+0000", 2024-09-21T15:00:00.790 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:00.790 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.790 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.790 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:00.790 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.162460+0000", 2024-09-21T15:00:00.790 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6220000000000001e-06 2024-09-21T15:00:00.790 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.790 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.790 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:00.790 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.162473+0000", 2024-09-21T15:00:00.790 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3045e-05 2024-09-21T15:00:00.791 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.791 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.791 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:00.791 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.162484+0000", 2024-09-21T15:00:00.791 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0941000000000001e-05 2024-09-21T15:00:00.791 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.791 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.791 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:00.791 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.172414+0000", 2024-09-21T15:00:00.791 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0099308350000000007 2024-09-21T15:00:00.791 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.791 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.791 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:00.791 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.172692+0000", 2024-09-21T15:00:00.792 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00027786 2024-09-21T15:00:00.792 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.792 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.792 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T15:00:00.792 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.173552+0000", 2024-09-21T15:00:00.792 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00085935999999999998 2024-09-21T15:00:00.792 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.792 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.792 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:00.792 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.176079+0000", 2024-09-21T15:00:00.792 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0025274189999999999 2024-09-21T15:00:00.792 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.792 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.792 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:00.793 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.177234+0000", 2024-09-21T15:00:00.793 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001154569 2024-09-21T15:00:00.793 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.793 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.793 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:00.793 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.177286+0000", 2024-09-21T15:00:00.793 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1969999999999999e-05 2024-09-21T15:00:00.793 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.793 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.793 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:00.793 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.177328+0000", 2024-09-21T15:00:00.793 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1891999999999998e-05 2024-09-21T15:00:00.793 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:00.793 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:00.793 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:00.794 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.794 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.794 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:85605 3.7 3:e5bd5c60:::rbd_object_map.113637a2ddc2.0000000000000044:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115)", 2024-09-21T15:00:00.794 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:56.162490+0000", 2024-09-21T15:00:00.794 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.5305728280000004, 2024-09-21T15:00:00.794 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015121122000000001, 2024-09-21T15:00:00.794 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:00.794 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:00.794 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:00.794 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:00.794 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:00.794 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 85605 2024-09-21T15:00:00.794 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.794 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:00.795 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.795 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:00.795 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.162490+0000", 2024-09-21T15:00:00.795 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:00.795 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.795 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.795 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:00.795 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.162490+0000", 2024-09-21T15:00:00.795 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:00.795 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.795 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.795 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:00.795 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.162490+0000", 2024-09-21T15:00:00.796 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:00.796 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.796 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.796 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:00.796 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.162494+0000", 2024-09-21T15:00:00.796 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.5020000000000001e-06 2024-09-21T15:00:00.796 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.796 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.796 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:00.796 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.162498+0000", 2024-09-21T15:00:00.796 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6920000000000001e-06 2024-09-21T15:00:00.796 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.796 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.796 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:00.797 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.162503+0000", 2024-09-21T15:00:00.797 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.617e-06 2024-09-21T15:00:00.797 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.797 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.797 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:00.797 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.174549+0000", 2024-09-21T15:00:00.797 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012046238000000001 2024-09-21T15:00:00.797 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.797 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.797 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:00.797 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.174640+0000", 2024-09-21T15:00:00.797 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0810999999999995e-05 2024-09-21T15:00:00.797 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.797 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.798 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T15:00:00.798 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.175156+0000", 2024-09-21T15:00:00.798 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00051634700000000001 2024-09-21T15:00:00.798 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.798 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.798 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:00.798 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.177347+0000", 2024-09-21T15:00:00.798 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0021907319999999999 2024-09-21T15:00:00.798 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.798 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.798 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:00.798 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.177407+0000", 2024-09-21T15:00:00.798 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9657999999999997e-05 2024-09-21T15:00:00.798 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.799 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.799 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:00.799 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.177498+0000", 2024-09-21T15:00:00.799 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1225999999999997e-05 2024-09-21T15:00:00.799 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:00.799 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:00.799 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:00.799 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.177612+0000", 2024-09-21T15:00:00.799 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011362999999999999 2024-09-21T15:00:00.799 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:00.799 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:00.799 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:00.799 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:00.800 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:00.800 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:00.801 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T15:00:00.801 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:00:00.794+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:00:00.902 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:00:00.901+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:00:00.918 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:00.918 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:00:00.918 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T15:00:00.918 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:00.928 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T15:00:01.003 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:00:00.998+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:00:01.043 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:01.043 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:00:01.043 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T15:00:01.043 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T15:00:01.043 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:01.053 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T15:00:01.104 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:00:01.098+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:00:01.165 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:01.165 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T15:00:01.165 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T15:00:01.165 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T15:00:01.165 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.166 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45834 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:01.166 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159133+0000", 2024-09-21T15:00:01.166 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 51.003597165999999, 2024-09-21T15:00:01.166 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068236884999999997, 2024-09-21T15:00:01.166 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:01.166 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:01.166 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:01.166 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:01.166 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:01.166 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45834 2024-09-21T15:00:01.166 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.167 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:01.167 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.167 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:01.167 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159133+0000", 2024-09-21T15:00:01.167 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:01.167 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.167 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.167 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:01.167 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159133+0000", 2024-09-21T15:00:01.167 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:01.167 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.167 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.167 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:01.167 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159133+0000", 2024-09-21T15:00:01.168 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:01.168 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.168 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.168 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:01.168 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159135+0000", 2024-09-21T15:00:01.168 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4500000000000001e-06 2024-09-21T15:00:01.168 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.168 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.168 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:01.168 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159138+0000", 2024-09-21T15:00:01.168 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2770000000000001e-06 2024-09-21T15:00:01.168 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.168 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.168 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:01.168 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159141+0000", 2024-09-21T15:00:01.169 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9210000000000001e-06 2024-09-21T15:00:01.169 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.169 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.169 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:01.169 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227361+0000", 2024-09-21T15:00:01.169 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068220154000000005 2024-09-21T15:00:01.169 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.169 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.169 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:01.169 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227370+0000", 2024-09-21T15:00:01.169 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3239999999999992e-06 2024-09-21T15:00:01.169 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:01.169 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:01.170 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:01.170 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.170 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.170 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45839 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:01.170 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159313+0000", 2024-09-21T15:00:01.170 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 51.003417200000001, 2024-09-21T15:00:01.170 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068238699, 2024-09-21T15:00:01.170 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:01.170 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:01.170 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:01.170 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:01.170 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:01.170 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45839 2024-09-21T15:00:01.171 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.171 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:01.171 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.171 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:01.171 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159313+0000", 2024-09-21T15:00:01.171 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:01.171 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.171 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.171 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:01.171 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159313+0000", 2024-09-21T15:00:01.171 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:01.171 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.171 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.171 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:01.172 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159313+0000", 2024-09-21T15:00:01.172 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:01.172 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.172 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.172 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:01.172 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159316+0000", 2024-09-21T15:00:01.172 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.033e-06 2024-09-21T15:00:01.172 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.172 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.172 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:01.172 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159322+0000", 2024-09-21T15:00:01.172 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1820000000000003e-06 2024-09-21T15:00:01.172 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.172 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.173 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:01.173 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159327+0000", 2024-09-21T15:00:01.173 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.062e-06 2024-09-21T15:00:01.173 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.173 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.173 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:01.173 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227544+0000", 2024-09-21T15:00:01.173 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068217142999999994 2024-09-21T15:00:01.173 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.173 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.173 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:01.173 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227552+0000", 2024-09-21T15:00:01.173 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6260000000000003e-06 2024-09-21T15:00:01.173 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:01.173 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:01.174 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:01.174 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.174 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.174 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45840 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:01.174 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159332+0000", 2024-09-21T15:00:01.174 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 51.003398902999997, 2024-09-21T15:00:01.174 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068262046000000007, 2024-09-21T15:00:01.174 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:01.174 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:01.174 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:01.174 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:01.174 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:01.174 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45840 2024-09-21T15:00:01.174 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.175 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:01.175 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.175 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:01.175 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159332+0000", 2024-09-21T15:00:01.175 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:01.175 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.175 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.175 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:01.175 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159332+0000", 2024-09-21T15:00:01.175 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:01.175 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.175 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.175 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:01.175 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159331+0000", 2024-09-21T15:00:01.175 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:01.176 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.176 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.176 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:01.176 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159351+0000", 2024-09-21T15:00:01.176 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0064999999999998e-05 2024-09-21T15:00:01.176 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.176 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.176 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:01.176 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159367+0000", 2024-09-21T15:00:01.176 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5308e-05 2024-09-21T15:00:01.176 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.176 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.176 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:01.176 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159373+0000", 2024-09-21T15:00:01.177 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1840000000000004e-06 2024-09-21T15:00:01.177 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.177 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.177 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:01.177 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227584+0000", 2024-09-21T15:00:01.177 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068211401000000005 2024-09-21T15:00:01.177 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.177 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.177 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:01.177 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227594+0000", 2024-09-21T15:00:01.177 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4120000000000007e-06 2024-09-21T15:00:01.177 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:01.177 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:01.177 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:01.178 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.178 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.178 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45841 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:01.178 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159379+0000", 2024-09-21T15:00:01.178 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 51.003351747000004, 2024-09-21T15:00:01.178 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068249039999999997, 2024-09-21T15:00:01.178 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:01.178 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:01.178 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:01.178 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:01.178 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:01.178 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45841 2024-09-21T15:00:01.178 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.178 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:01.179 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.179 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:01.179 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159379+0000", 2024-09-21T15:00:01.179 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:01.179 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.179 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.179 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:01.179 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159379+0000", 2024-09-21T15:00:01.179 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:01.179 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.179 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.179 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:01.179 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159378+0000", 2024-09-21T15:00:01.179 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:01.180 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.180 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.180 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:01.180 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159382+0000", 2024-09-21T15:00:01.180 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.641e-06 2024-09-21T15:00:01.180 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.180 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.180 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:01.180 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159421+0000", 2024-09-21T15:00:01.180 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9439e-05 2024-09-21T15:00:01.180 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.180 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.180 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:01.180 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159430+0000", 2024-09-21T15:00:01.181 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8429999999999999e-06 2024-09-21T15:00:01.181 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.181 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.181 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:01.181 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227617+0000", 2024-09-21T15:00:01.181 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068187084999999995 2024-09-21T15:00:01.181 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.181 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.181 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:01.181 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227628+0000", 2024-09-21T15:00:01.181 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0507e-05 2024-09-21T15:00:01.181 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:01.181 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:01.181 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:01.181 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.182 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.182 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45844 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:01.182 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159499+0000", 2024-09-21T15:00:01.182 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 51.003231360999997, 2024-09-21T15:00:01.182 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068255415, 2024-09-21T15:00:01.182 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:01.182 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:01.182 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:01.182 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:01.182 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:01.182 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45844 2024-09-21T15:00:01.182 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.182 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:01.182 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.183 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:01.183 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159499+0000", 2024-09-21T15:00:01.183 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:01.183 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.183 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.183 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:01.183 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159499+0000", 2024-09-21T15:00:01.183 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:01.183 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.183 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.183 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:01.183 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159499+0000", 2024-09-21T15:00:01.183 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:01.184 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.184 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.184 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:01.184 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159503+0000", 2024-09-21T15:00:01.184 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0999999999999997e-06 2024-09-21T15:00:01.184 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.184 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.184 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:01.184 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159512+0000", 2024-09-21T15:00:01.184 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5850000000000006e-06 2024-09-21T15:00:01.184 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.184 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.184 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:01.184 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159518+0000", 2024-09-21T15:00:01.185 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0909999999999998e-06 2024-09-21T15:00:01.185 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.185 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.185 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:01.185 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227743+0000", 2024-09-21T15:00:01.185 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068224536000000002 2024-09-21T15:00:01.185 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.185 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.185 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:01.185 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227755+0000", 2024-09-21T15:00:01.185 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1659e-05 2024-09-21T15:00:01.185 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:01.185 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:01.186 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:01.186 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.186 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.186 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45845 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:01.186 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159524+0000", 2024-09-21T15:00:01.186 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 51.003207001, 2024-09-21T15:00:01.186 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068267194000000003, 2024-09-21T15:00:01.186 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:01.186 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:01.186 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:01.186 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:01.186 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:01.186 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45845 2024-09-21T15:00:01.186 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.187 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:01.187 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.187 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:01.187 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159524+0000", 2024-09-21T15:00:01.187 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:01.187 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.187 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.187 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:01.187 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159524+0000", 2024-09-21T15:00:01.187 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:01.187 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.188 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.188 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:01.188 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159523+0000", 2024-09-21T15:00:01.188 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:01.188 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.188 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.188 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:01.188 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159526+0000", 2024-09-21T15:00:01.188 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1980000000000001e-06 2024-09-21T15:00:01.188 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.188 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.188 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:01.188 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159533+0000", 2024-09-21T15:00:01.188 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.348e-06 2024-09-21T15:00:01.189 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.189 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.189 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:01.189 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159546+0000", 2024-09-21T15:00:01.189 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3464e-05 2024-09-21T15:00:01.189 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.189 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.189 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:01.189 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227783+0000", 2024-09-21T15:00:01.189 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068236926000000003 2024-09-21T15:00:01.189 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.189 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.189 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:01.189 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227791+0000", 2024-09-21T15:00:01.190 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6669999999999996e-06 2024-09-21T15:00:01.190 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:01.190 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:01.190 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:01.190 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.190 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.190 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45846 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:01.190 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159584+0000", 2024-09-21T15:00:01.190 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 51.003146358999999, 2024-09-21T15:00:01.190 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068254767999999993, 2024-09-21T15:00:01.190 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:01.190 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:01.190 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:01.191 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:01.191 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:01.191 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45846 2024-09-21T15:00:01.191 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.191 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:01.191 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.191 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:01.191 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159584+0000", 2024-09-21T15:00:01.191 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:01.191 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.191 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.191 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:01.191 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159584+0000", 2024-09-21T15:00:01.192 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:01.192 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.192 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.192 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:01.192 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159583+0000", 2024-09-21T15:00:01.192 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:01.192 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.192 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.192 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:01.192 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159588+0000", 2024-09-21T15:00:01.192 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1330000000000001e-06 2024-09-21T15:00:01.192 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.192 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.192 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:01.192 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159596+0000", 2024-09-21T15:00:01.193 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5469999999999997e-06 2024-09-21T15:00:01.193 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.193 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.193 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:01.193 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159609+0000", 2024-09-21T15:00:01.193 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2785999999999999e-05 2024-09-21T15:00:01.193 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.193 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.193 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:01.193 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227830+0000", 2024-09-21T15:00:01.193 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068220879999999998 2024-09-21T15:00:01.193 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.193 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.193 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:01.194 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227839+0000", 2024-09-21T15:00:01.194 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1239999999999994e-06 2024-09-21T15:00:01.194 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:01.194 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:01.194 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:01.194 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.194 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.194 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45847 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:01.194 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159614+0000", 2024-09-21T15:00:01.194 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 51.003116282999997, 2024-09-21T15:00:01.194 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068260639999999997, 2024-09-21T15:00:01.194 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:01.194 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:01.194 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:01.195 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:01.195 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:01.195 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45847 2024-09-21T15:00:01.195 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.195 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:01.195 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.195 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:01.195 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159614+0000", 2024-09-21T15:00:01.195 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:01.195 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.195 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.195 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:01.195 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159614+0000", 2024-09-21T15:00:01.195 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:01.196 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.196 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.196 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:01.196 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159614+0000", 2024-09-21T15:00:01.196 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:01.196 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.196 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.196 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:01.196 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159617+0000", 2024-09-21T15:00:01.196 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2909999999999999e-06 2024-09-21T15:00:01.196 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.196 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.196 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:01.196 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159623+0000", 2024-09-21T15:00:01.197 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.511e-06 2024-09-21T15:00:01.197 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.197 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.197 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:01.197 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159631+0000", 2024-09-21T15:00:01.197 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.571e-06 2024-09-21T15:00:01.197 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.197 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.197 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:01.197 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227866+0000", 2024-09-21T15:00:01.197 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068235136000000002 2024-09-21T15:00:01.197 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.197 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.198 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:01.198 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227875+0000", 2024-09-21T15:00:01.198 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7199999999999995e-06 2024-09-21T15:00:01.198 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:01.198 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:01.198 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:01.198 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.198 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.198 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45848 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:01.198 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159636+0000", 2024-09-21T15:00:01.198 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 51.003094503, 2024-09-21T15:00:01.198 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068267283999999998, 2024-09-21T15:00:01.198 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:01.199 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:01.199 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:01.199 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:01.199 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:01.199 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45848 2024-09-21T15:00:01.199 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.199 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:01.199 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.199 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:01.199 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159636+0000", 2024-09-21T15:00:01.199 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:01.199 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.199 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.199 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:01.200 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159636+0000", 2024-09-21T15:00:01.200 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:01.200 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.200 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.200 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:01.200 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159636+0000", 2024-09-21T15:00:01.200 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:01.200 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.200 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.200 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:01.200 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159638+0000", 2024-09-21T15:00:01.200 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6560000000000001e-06 2024-09-21T15:00:01.200 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.200 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.201 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:01.201 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159644+0000", 2024-09-21T15:00:01.201 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0839999999999997e-06 2024-09-21T15:00:01.201 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.201 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.201 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:01.201 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159649+0000", 2024-09-21T15:00:01.201 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7500000000000003e-06 2024-09-21T15:00:01.201 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.201 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.201 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:01.201 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227892+0000", 2024-09-21T15:00:01.201 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068243315999999998 2024-09-21T15:00:01.201 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.202 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.202 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:01.202 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227903+0000", 2024-09-21T15:00:01.202 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0835999999999999e-05 2024-09-21T15:00:01.202 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:01.202 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:01.202 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:01.202 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.202 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.202 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45849 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:01.202 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159667+0000", 2024-09-21T15:00:01.202 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 51.003063689000001, 2024-09-21T15:00:01.202 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068291051000000005, 2024-09-21T15:00:01.203 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:01.203 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:01.203 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:01.203 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:01.203 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:01.203 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45849 2024-09-21T15:00:01.203 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.203 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:01.203 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.203 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:01.203 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159667+0000", 2024-09-21T15:00:01.203 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:01.203 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.203 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.204 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:01.204 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159667+0000", 2024-09-21T15:00:01.204 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:01.204 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.204 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.204 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:01.204 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159666+0000", 2024-09-21T15:00:01.204 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:01.204 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.204 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.204 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:01.204 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159670+0000", 2024-09-21T15:00:01.204 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.427e-06 2024-09-21T15:00:01.204 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.205 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.205 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:01.205 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159678+0000", 2024-09-21T15:00:01.205 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9869999999999999e-06 2024-09-21T15:00:01.205 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.205 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.205 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:01.205 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159688+0000", 2024-09-21T15:00:01.205 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0631999999999999e-05 2024-09-21T15:00:01.205 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.205 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.205 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:01.205 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227950+0000", 2024-09-21T15:00:01.205 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068261798999999998 2024-09-21T15:00:01.206 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.206 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.206 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:01.206 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227958+0000", 2024-09-21T15:00:01.206 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7510000000000008e-06 2024-09-21T15:00:01.206 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:01.206 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:01.206 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:01.206 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.206 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.206 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45850 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:01.206 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159695+0000", 2024-09-21T15:00:01.206 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 51.003035978, 2024-09-21T15:00:01.206 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068302999000000003, 2024-09-21T15:00:01.207 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:01.207 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:01.207 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:01.207 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:01.207 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:01.207 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45850 2024-09-21T15:00:01.207 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.207 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:01.207 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.207 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:01.207 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159695+0000", 2024-09-21T15:00:01.207 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:01.207 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.207 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.207 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:01.208 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159695+0000", 2024-09-21T15:00:01.208 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:01.208 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.208 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.208 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:01.208 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159694+0000", 2024-09-21T15:00:01.208 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:01.208 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.208 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.208 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:01.208 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159697+0000", 2024-09-21T15:00:01.208 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.258e-06 2024-09-21T15:00:01.208 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.208 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.208 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:01.209 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159704+0000", 2024-09-21T15:00:01.209 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7660000000000001e-06 2024-09-21T15:00:01.209 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.209 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.209 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:01.209 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159713+0000", 2024-09-21T15:00:01.209 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1779999999999992e-06 2024-09-21T15:00:01.209 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.209 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.209 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:01.209 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227989+0000", 2024-09-21T15:00:01.209 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068276015999999995 2024-09-21T15:00:01.209 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.209 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.209 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:01.210 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227998+0000", 2024-09-21T15:00:01.210 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2050000000000002e-06 2024-09-21T15:00:01.210 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:01.210 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:01.210 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:01.210 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.210 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.210 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45851 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:01.210 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159719+0000", 2024-09-21T15:00:01.210 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 51.003011845000003, 2024-09-21T15:00:01.210 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068320327, 2024-09-21T15:00:01.210 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:01.210 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:01.211 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:01.211 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:01.211 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:01.211 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45851 2024-09-21T15:00:01.211 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.211 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:01.211 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.211 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:01.211 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159719+0000", 2024-09-21T15:00:01.211 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:01.211 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.211 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.212 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:01.212 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159719+0000", 2024-09-21T15:00:01.212 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:01.212 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.212 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.212 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:01.212 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159718+0000", 2024-09-21T15:00:01.212 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:01.212 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.212 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.212 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:01.212 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159721+0000", 2024-09-21T15:00:01.212 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2169999999999999e-06 2024-09-21T15:00:01.213 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.213 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.213 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:01.213 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159730+0000", 2024-09-21T15:00:01.213 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5720000000000003e-06 2024-09-21T15:00:01.213 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.213 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.213 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:01.213 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159735+0000", 2024-09-21T15:00:01.213 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1100000000000002e-06 2024-09-21T15:00:01.213 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.213 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.213 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:01.213 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228029+0000", 2024-09-21T15:00:01.214 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068293624999999997 2024-09-21T15:00:01.214 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.214 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.214 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:01.214 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228039+0000", 2024-09-21T15:00:01.214 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0254e-05 2024-09-21T15:00:01.214 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:01.214 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:01.214 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:01.214 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.214 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.214 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45852 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:01.214 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159740+0000", 2024-09-21T15:00:01.214 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 51.002990677, 2024-09-21T15:00:01.215 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068340260999999999, 2024-09-21T15:00:01.215 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:01.215 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:01.215 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:01.215 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:01.215 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:01.215 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45852 2024-09-21T15:00:01.215 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.215 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:01.215 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.215 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:01.215 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159740+0000", 2024-09-21T15:00:01.215 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:01.215 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.215 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.216 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:01.216 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159740+0000", 2024-09-21T15:00:01.216 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:01.216 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.216 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.216 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:01.216 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159739+0000", 2024-09-21T15:00:01.216 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:01.216 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.216 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.216 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:01.216 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159757+0000", 2024-09-21T15:00:01.216 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7314999999999999e-05 2024-09-21T15:00:01.216 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.217 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.217 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:01.217 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159769+0000", 2024-09-21T15:00:01.217 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2376e-05 2024-09-21T15:00:01.217 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.217 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.217 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:01.217 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159775+0000", 2024-09-21T15:00:01.217 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2179999999999996e-06 2024-09-21T15:00:01.217 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.217 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.217 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:01.217 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228071+0000", 2024-09-21T15:00:01.217 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068295344999999993 2024-09-21T15:00:01.218 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.218 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.218 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:01.218 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228080+0000", 2024-09-21T15:00:01.218 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3959999999999995e-06 2024-09-21T15:00:01.218 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:01.218 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:01.218 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:01.218 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.218 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.218 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45853 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:01.218 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159781+0000", 2024-09-21T15:00:01.218 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 51.002949413000003, 2024-09-21T15:00:01.218 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068337093000000002, 2024-09-21T15:00:01.219 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:01.219 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:01.219 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:01.219 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:01.219 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:01.219 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45853 2024-09-21T15:00:01.219 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.219 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:01.219 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.219 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:01.219 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159781+0000", 2024-09-21T15:00:01.219 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:01.219 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.220 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.220 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:01.220 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159781+0000", 2024-09-21T15:00:01.220 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:01.220 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.220 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.220 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:01.220 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159781+0000", 2024-09-21T15:00:01.220 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:01.220 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.220 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.220 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:01.220 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159784+0000", 2024-09-21T15:00:01.220 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4759999999999998e-06 2024-09-21T15:00:01.220 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.221 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.221 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:01.221 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159792+0000", 2024-09-21T15:00:01.221 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0569999999999993e-06 2024-09-21T15:00:01.221 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.221 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.221 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:01.221 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159797+0000", 2024-09-21T15:00:01.221 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9640000000000002e-06 2024-09-21T15:00:01.221 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.221 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.221 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:01.221 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228111+0000", 2024-09-21T15:00:01.222 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068313737999999999 2024-09-21T15:00:01.222 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.222 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.222 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:01.222 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228118+0000", 2024-09-21T15:00:01.222 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.43e-06 2024-09-21T15:00:01.222 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:01.222 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:01.222 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:01.222 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.222 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.222 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45854 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:01.223 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159802+0000", 2024-09-21T15:00:01.223 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 51.002928763, 2024-09-21T15:00:01.223 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068358938999999994, 2024-09-21T15:00:01.223 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:01.223 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:01.223 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:01.223 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:01.223 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:01.223 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45854 2024-09-21T15:00:01.223 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.223 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:01.223 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.223 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:01.224 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159802+0000", 2024-09-21T15:00:01.224 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:01.224 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.224 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.224 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:01.224 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159802+0000", 2024-09-21T15:00:01.224 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:01.224 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.224 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.224 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:01.224 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159801+0000", 2024-09-21T15:00:01.224 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:01.224 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.225 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.225 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:01.225 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159804+0000", 2024-09-21T15:00:01.225 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0970000000000002e-06 2024-09-21T15:00:01.225 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.225 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.225 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:01.225 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159823+0000", 2024-09-21T15:00:01.225 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8895000000000001e-05 2024-09-21T15:00:01.225 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.225 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.225 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:01.225 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159837+0000", 2024-09-21T15:00:01.225 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3247999999999999e-05 2024-09-21T15:00:01.226 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.226 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.226 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:01.226 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228151+0000", 2024-09-21T15:00:01.226 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068314642999999994 2024-09-21T15:00:01.226 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.226 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.226 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:01.226 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228161+0000", 2024-09-21T15:00:01.226 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4250000000000004e-06 2024-09-21T15:00:01.227 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:01.227 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:01.227 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:01.227 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.227 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.227 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45983 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:01.227 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159848+0000", 2024-09-21T15:00:01.227 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 51.002882503999999, 2024-09-21T15:00:01.227 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068333801, 2024-09-21T15:00:01.227 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:01.227 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:01.227 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:01.227 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:01.228 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:01.228 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45983 2024-09-21T15:00:01.228 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.228 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:01.228 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.228 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:01.228 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159848+0000", 2024-09-21T15:00:01.228 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:01.228 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.228 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.228 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:01.228 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159848+0000", 2024-09-21T15:00:01.229 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:01.229 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.229 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.229 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:01.229 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159847+0000", 2024-09-21T15:00:01.229 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:01.229 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.229 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.229 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:01.229 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159856+0000", 2024-09-21T15:00:01.229 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4349999999999997e-06 2024-09-21T15:00:01.229 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.229 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.229 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:01.230 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159866+0000", 2024-09-21T15:00:01.230 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0076e-05 2024-09-21T15:00:01.230 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.230 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.230 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:01.230 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159872+0000", 2024-09-21T15:00:01.230 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8309999999999997e-06 2024-09-21T15:00:01.230 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.230 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.230 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:01.230 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228175+0000", 2024-09-21T15:00:01.230 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068302680000000005 2024-09-21T15:00:01.230 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.231 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.231 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:01.231 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228182+0000", 2024-09-21T15:00:01.231 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3150000000000003e-06 2024-09-21T15:00:01.231 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:01.231 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:01.231 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:01.231 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.231 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.231 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45984 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:01.231 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159877+0000", 2024-09-21T15:00:01.231 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 51.002853084000002, 2024-09-21T15:00:01.231 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068328448, 2024-09-21T15:00:01.231 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:01.232 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:01.232 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:01.232 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:01.232 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:01.232 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45984 2024-09-21T15:00:01.232 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.232 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:01.232 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.232 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:01.232 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159877+0000", 2024-09-21T15:00:01.232 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:01.232 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.232 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.233 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:01.233 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159877+0000", 2024-09-21T15:00:01.233 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:01.233 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.233 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.233 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:01.233 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159877+0000", 2024-09-21T15:00:01.233 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:01.233 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.233 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.233 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:01.233 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159880+0000", 2024-09-21T15:00:01.233 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7610000000000001e-06 2024-09-21T15:00:01.233 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.234 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.234 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:01.234 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159888+0000", 2024-09-21T15:00:01.234 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0330000000000003e-06 2024-09-21T15:00:01.234 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.234 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.234 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:01.234 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159893+0000", 2024-09-21T15:00:01.234 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3550000000000004e-06 2024-09-21T15:00:01.234 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.234 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.234 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:01.234 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228199+0000", 2024-09-21T15:00:01.235 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068306438999999997 2024-09-21T15:00:01.235 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.235 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.235 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:01.235 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228206+0000", 2024-09-21T15:00:01.235 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5599999999999999e-06 2024-09-21T15:00:01.235 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:01.235 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:01.235 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:01.235 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.235 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.235 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45985 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:01.235 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159897+0000", 2024-09-21T15:00:01.236 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 51.002833260999999, 2024-09-21T15:00:01.236 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068345664, 2024-09-21T15:00:01.236 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:01.236 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:01.236 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:01.236 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:01.236 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:01.236 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45985 2024-09-21T15:00:01.236 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.236 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:01.236 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.236 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:01.236 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159897+0000", 2024-09-21T15:00:01.236 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:01.237 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.237 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.237 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:01.237 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159897+0000", 2024-09-21T15:00:01.237 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:01.237 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.237 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.237 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:01.237 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159897+0000", 2024-09-21T15:00:01.237 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:01.237 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.237 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.237 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:01.237 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159900+0000", 2024-09-21T15:00:01.238 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.954e-06 2024-09-21T15:00:01.238 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.238 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.238 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:01.238 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159913+0000", 2024-09-21T15:00:01.238 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3106e-05 2024-09-21T15:00:01.238 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.238 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.238 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:01.238 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159918+0000", 2024-09-21T15:00:01.238 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5199999999999997e-06 2024-09-21T15:00:01.238 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.238 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.239 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:01.239 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228236+0000", 2024-09-21T15:00:01.239 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068317040999999995 2024-09-21T15:00:01.239 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.239 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.239 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:01.239 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228243+0000", 2024-09-21T15:00:01.239 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3880000000000002e-06 2024-09-21T15:00:01.239 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:01.239 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:01.239 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:01.239 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.239 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.239 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45986 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:01.240 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159924+0000", 2024-09-21T15:00:01.240 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 51.002806966000001, 2024-09-21T15:00:01.240 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068364362999999997, 2024-09-21T15:00:01.240 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:01.240 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:01.240 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:01.240 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:01.240 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:01.240 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45986 2024-09-21T15:00:01.240 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.240 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:01.240 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.240 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:01.241 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159924+0000", 2024-09-21T15:00:01.241 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:01.241 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.241 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.241 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:01.241 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159924+0000", 2024-09-21T15:00:01.241 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:01.241 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.241 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.241 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:01.241 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159923+0000", 2024-09-21T15:00:01.241 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:01.241 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.241 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.242 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:01.242 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159926+0000", 2024-09-21T15:00:01.242 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0510000000000001e-06 2024-09-21T15:00:01.242 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.242 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.242 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:01.242 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159932+0000", 2024-09-21T15:00:01.242 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4160000000000001e-06 2024-09-21T15:00:01.242 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.242 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.242 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:01.242 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159938+0000", 2024-09-21T15:00:01.242 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.383e-06 2024-09-21T15:00:01.243 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.243 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.243 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:01.243 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228275+0000", 2024-09-21T15:00:01.243 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068337007000000005 2024-09-21T15:00:01.243 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.243 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.243 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:01.243 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228288+0000", 2024-09-21T15:00:01.243 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3052e-05 2024-09-21T15:00:01.243 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:01.243 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:01.243 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:01.243 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.244 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.244 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45987 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:01.244 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159944+0000", 2024-09-21T15:00:01.244 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 51.002786741999998, 2024-09-21T15:00:01.244 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068387232000000006, 2024-09-21T15:00:01.244 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:01.244 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:01.244 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:01.244 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:01.244 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:01.244 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45987 2024-09-21T15:00:01.244 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.244 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:01.244 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.245 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:01.245 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159944+0000", 2024-09-21T15:00:01.245 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:01.245 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.245 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.245 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:01.245 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159944+0000", 2024-09-21T15:00:01.245 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:01.245 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.245 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.245 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:01.245 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159943+0000", 2024-09-21T15:00:01.245 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:01.245 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.245 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.246 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:01.246 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159946+0000", 2024-09-21T15:00:01.246 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6309999999999999e-06 2024-09-21T15:00:01.246 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.246 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.246 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:01.246 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159952+0000", 2024-09-21T15:00:01.246 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6540000000000004e-06 2024-09-21T15:00:01.246 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.246 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.246 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:01.246 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159957+0000", 2024-09-21T15:00:01.246 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9280000000000001e-06 2024-09-21T15:00:01.246 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.247 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.247 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:01.247 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228322+0000", 2024-09-21T15:00:01.247 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068365067000000002 2024-09-21T15:00:01.247 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.247 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.247 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:01.247 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228331+0000", 2024-09-21T15:00:01.247 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3270000000000007e-06 2024-09-21T15:00:01.247 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:01.247 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:01.247 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:01.247 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:01.247 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:01.248 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:01.249 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T15:00:01.249 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:00:01.242+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:00:01.350 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:00:01.342+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:00:01.408 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:01.408 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:00:01.408 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T15:00:01.408 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:01.420 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T15:00:01.425 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T15:00:01.451 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:00:01.446+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:00:01.520 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:01.520 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:00:01.520 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T15:00:01.520 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T15:00:01.520 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:01.529 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T15:00:01.569 INFO:teuthology.orchestra.run.smithi032.stdout:checking clone #21, image image_client.0-clone21 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent22 2024-09-21T15:00:01.578 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:00:01.574+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:00:01.653 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:00:01.646+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:00:01.694 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:01.694 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T15:00:01.694 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T15:00:01.695 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T15:00:01.695 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.695 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:10651 3.12 3:4e9d23af:::rbd_data.1136f5ed5dd6.0000000000000020:head [write 0~81920 [fadvise_sequential+fadvise_nocache],write 90112~8192 [fadvise_sequential+fadvise_nocache],write 122880~122880 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:01.695 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.292476+0000", 2024-09-21T15:00:01.695 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 82.399047611, 2024-09-21T15:00:01.695 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012765669, 2024-09-21T15:00:01.695 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:01.695 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:01.695 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:01.695 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:01.695 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:01.695 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 10651 2024-09-21T15:00:01.695 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.695 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:01.696 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.696 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:01.696 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292476+0000", 2024-09-21T15:00:01.696 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:01.696 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.696 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.696 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:01.696 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292476+0000", 2024-09-21T15:00:01.696 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:01.696 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.696 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.696 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:01.696 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292475+0000", 2024-09-21T15:00:01.696 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:00:01.696 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.697 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.697 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:01.697 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292696+0000", 2024-09-21T15:00:01.697 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000220201 2024-09-21T15:00:01.697 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.697 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.697 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:01.697 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292711+0000", 2024-09-21T15:00:01.697 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5803000000000001e-05 2024-09-21T15:00:01.697 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.697 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.697 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:01.697 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292721+0000", 2024-09-21T15:00:01.697 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6160000000000008e-06 2024-09-21T15:00:01.697 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.697 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.698 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:01.698 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292757+0000", 2024-09-21T15:00:01.698 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6211999999999998e-05 2024-09-21T15:00:01.698 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.698 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.698 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:01.698 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292878+0000", 2024-09-21T15:00:01.698 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000120306 2024-09-21T15:00:01.698 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.698 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.698 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 3", 2024-09-21T15:00:01.698 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.293019+0000", 2024-09-21T15:00:01.698 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014186400000000001 2024-09-21T15:00:01.698 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.699 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.699 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:01.699 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.294898+0000", 2024-09-21T15:00:01.699 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001878434 2024-09-21T15:00:01.699 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.699 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.699 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:01.699 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.305162+0000", 2024-09-21T15:00:01.699 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.01026373 2024-09-21T15:00:01.699 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.699 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.699 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:01.699 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.305209+0000", 2024-09-21T15:00:01.699 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7615999999999999e-05 2024-09-21T15:00:01.699 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.699 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.700 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:01.700 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.305242+0000", 2024-09-21T15:00:01.700 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2979999999999999e-05 2024-09-21T15:00:01.700 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:01.700 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:01.700 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:01.700 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.700 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.700 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:11285 3.12 3:4edf73d2:::rbd_data.1136f5ed5dd6.0000000000000039:head [write 90112~32768 [fadvise_sequential+fadvise_nocache],write 139264~24576 [fadvise_sequential+fadvise_nocache],write 180224~24576 [fadvise_sequential+fadvise_nocache],write 221184~40960 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:01.700 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.539470+0000", 2024-09-21T15:00:01.700 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 82.152053859000006, 2024-09-21T15:00:01.700 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0083360099999999996, 2024-09-21T15:00:01.700 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:01.700 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:01.700 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:01.701 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:01.701 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:01.701 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 11285 2024-09-21T15:00:01.701 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.701 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:01.701 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.701 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:01.701 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.539470+0000", 2024-09-21T15:00:01.701 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:01.701 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.701 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.701 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:01.701 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.539470+0000", 2024-09-21T15:00:01.701 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:01.701 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.701 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.702 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:01.702 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.539469+0000", 2024-09-21T15:00:01.702 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:00:01.702 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.702 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.702 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:01.702 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.539652+0000", 2024-09-21T15:00:01.702 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000183084 2024-09-21T15:00:01.702 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.702 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.702 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:01.702 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.539666+0000", 2024-09-21T15:00:01.702 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3947e-05 2024-09-21T15:00:01.702 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.702 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.703 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:01.703 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.539673+0000", 2024-09-21T15:00:01.703 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7309999999999996e-06 2024-09-21T15:00:01.703 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.703 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.703 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:01.703 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.539714+0000", 2024-09-21T15:00:01.703 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1539000000000003e-05 2024-09-21T15:00:01.703 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.703 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.703 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:01.703 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.539832+0000", 2024-09-21T15:00:01.703 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011753500000000001 2024-09-21T15:00:01.703 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.704 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.704 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 3", 2024-09-21T15:00:01.704 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.540005+0000", 2024-09-21T15:00:01.704 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000173031 2024-09-21T15:00:01.704 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.704 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.704 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:01.704 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.541606+0000", 2024-09-21T15:00:01.704 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001600899 2024-09-21T15:00:01.704 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.704 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.704 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:01.704 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.547709+0000", 2024-09-21T15:00:01.704 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0061027110000000002 2024-09-21T15:00:01.704 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.705 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.705 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:01.705 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.547767+0000", 2024-09-21T15:00:01.705 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8248999999999997e-05 2024-09-21T15:00:01.705 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.705 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.705 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:01.705 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.547806+0000", 2024-09-21T15:00:01.705 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9341000000000002e-05 2024-09-21T15:00:01.705 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:01.705 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:01.705 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:01.705 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.705 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.705 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12189 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 40960~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:01.706 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.618667+0000", 2024-09-21T15:00:01.706 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 82.072857408999994, 2024-09-21T15:00:01.706 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0083014549999999993, 2024-09-21T15:00:01.706 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:01.706 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:01.706 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:01.706 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:01.706 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:01.706 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12189 2024-09-21T15:00:01.706 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.706 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:01.706 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.706 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:01.706 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618667+0000", 2024-09-21T15:00:01.706 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:01.707 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.707 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.707 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:01.707 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618667+0000", 2024-09-21T15:00:01.707 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:01.707 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.707 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.707 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:01.707 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618666+0000", 2024-09-21T15:00:01.707 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:01.707 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.707 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.707 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:01.707 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618669+0000", 2024-09-21T15:00:01.707 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.554e-06 2024-09-21T15:00:01.708 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.708 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.708 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:01.708 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618675+0000", 2024-09-21T15:00:01.708 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2570000000000004e-06 2024-09-21T15:00:01.708 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.708 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.708 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:01.708 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618680+0000", 2024-09-21T15:00:01.708 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6539999999999998e-06 2024-09-21T15:00:01.708 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.708 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.708 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:01.708 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.626683+0000", 2024-09-21T15:00:01.708 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0080035880000000007 2024-09-21T15:00:01.708 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.709 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.709 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:01.709 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.626764+0000", 2024-09-21T15:00:01.709 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0672000000000003e-05 2024-09-21T15:00:01.709 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.709 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.709 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:01.709 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.626968+0000", 2024-09-21T15:00:01.709 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00020407499999999999 2024-09-21T15:00:01.709 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:01.709 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:01.709 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:01.709 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.709 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.709 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12193 3.4 3:25437d51:::rbd_data.11363a21d2be.0000000000000025:10 [read 425984~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:01.710 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.618755+0000", 2024-09-21T15:00:01.710 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 82.072769328000007, 2024-09-21T15:00:01.710 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086461379999999994, 2024-09-21T15:00:01.710 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:01.710 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:01.710 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:01.710 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:01.710 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:01.710 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12193 2024-09-21T15:00:01.710 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.710 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:01.710 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.710 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:01.710 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618755+0000", 2024-09-21T15:00:01.710 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:01.710 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.711 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.711 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:01.711 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618755+0000", 2024-09-21T15:00:01.711 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:01.711 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.711 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.711 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:01.711 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618754+0000", 2024-09-21T15:00:01.711 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:01.711 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.711 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.711 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:01.711 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618759+0000", 2024-09-21T15:00:01.711 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4020000000000004e-06 2024-09-21T15:00:01.711 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.712 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.712 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:01.712 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618767+0000", 2024-09-21T15:00:01.712 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7490000000000004e-06 2024-09-21T15:00:01.712 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.712 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.712 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:01.712 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618773+0000", 2024-09-21T15:00:01.712 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0939999999999996e-06 2024-09-21T15:00:01.712 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.712 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.712 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:01.712 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.627115+0000", 2024-09-21T15:00:01.712 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0083416840000000003 2024-09-21T15:00:01.713 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.713 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.713 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:01.713 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.627214+0000", 2024-09-21T15:00:01.713 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9042000000000005e-05 2024-09-21T15:00:01.713 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.713 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.713 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:01.713 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.627401+0000", 2024-09-21T15:00:01.713 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018677200000000001 2024-09-21T15:00:01.713 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:01.713 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:01.713 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:01.713 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.713 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.713 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12212 3.4 3:21b43b45:::rbd_data.11363a21d2be.000000000000002b:10 [read 237568~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:01.714 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.619922+0000", 2024-09-21T15:00:01.714 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 82.071602343999999, 2024-09-21T15:00:01.714 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.008324372, 2024-09-21T15:00:01.714 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:01.714 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:01.714 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:01.714 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:01.714 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:01.714 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12212 2024-09-21T15:00:01.714 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.714 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:01.714 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.714 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:01.714 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619922+0000", 2024-09-21T15:00:01.714 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:01.714 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.715 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.715 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:01.715 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619922+0000", 2024-09-21T15:00:01.715 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:01.715 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.715 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.715 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:01.715 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619921+0000", 2024-09-21T15:00:01.715 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:01.715 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.715 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.715 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:01.715 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619924+0000", 2024-09-21T15:00:01.716 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.999e-06 2024-09-21T15:00:01.716 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.716 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.716 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:01.716 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619934+0000", 2024-09-21T15:00:01.716 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.8120000000000002e-06 2024-09-21T15:00:01.716 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.716 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.716 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:01.716 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619939+0000", 2024-09-21T15:00:01.716 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7729999999999999e-06 2024-09-21T15:00:01.716 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.716 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.716 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:01.717 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628005+0000", 2024-09-21T15:00:01.717 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0080663330000000002 2024-09-21T15:00:01.717 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.717 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.717 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:01.717 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628073+0000", 2024-09-21T15:00:01.717 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7479999999999998e-05 2024-09-21T15:00:01.717 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.717 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.717 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:01.717 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628246+0000", 2024-09-21T15:00:01.717 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017337400000000001 2024-09-21T15:00:01.717 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:01.717 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:01.717 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:01.717 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.718 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.718 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12213 3.4 3:21b43b45:::rbd_data.11363a21d2be.000000000000002b:10 [read 270336~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:01.718 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.619945+0000", 2024-09-21T15:00:01.718 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 82.071579061999998, 2024-09-21T15:00:01.718 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086195049999999995, 2024-09-21T15:00:01.718 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:01.718 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:01.718 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:01.718 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:01.718 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:01.718 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12213 2024-09-21T15:00:01.718 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.718 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:01.718 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.718 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:01.719 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619945+0000", 2024-09-21T15:00:01.719 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:01.719 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.719 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.719 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:01.719 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619945+0000", 2024-09-21T15:00:01.719 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:01.719 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.719 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.719 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:01.719 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619945+0000", 2024-09-21T15:00:01.719 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:01.719 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.719 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.719 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:01.720 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619947+0000", 2024-09-21T15:00:01.720 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5780000000000001e-06 2024-09-21T15:00:01.720 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.720 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.720 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:01.720 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619957+0000", 2024-09-21T15:00:01.720 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3500000000000003e-06 2024-09-21T15:00:01.720 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.720 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.720 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:01.720 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619964+0000", 2024-09-21T15:00:01.720 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.199e-06 2024-09-21T15:00:01.720 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.720 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.720 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:01.721 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628285+0000", 2024-09-21T15:00:01.721 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0083210939999999994 2024-09-21T15:00:01.721 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.721 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.721 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:01.721 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628361+0000", 2024-09-21T15:00:01.721 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5873999999999999e-05 2024-09-21T15:00:01.721 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.721 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.721 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:01.721 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628565+0000", 2024-09-21T15:00:01.721 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000203799 2024-09-21T15:00:01.721 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:01.721 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:01.721 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:01.721 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.722 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.722 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12214 3.4 3:21b43b45:::rbd_data.11363a21d2be.000000000000002b:10 [read 376832~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:01.722 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.619969+0000", 2024-09-21T15:00:01.722 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 82.071555060999998, 2024-09-21T15:00:01.722 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0090292089999999998, 2024-09-21T15:00:01.722 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:01.722 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:01.722 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:01.722 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:01.722 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:01.722 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12214 2024-09-21T15:00:01.722 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.722 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:01.723 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.723 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:01.723 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619969+0000", 2024-09-21T15:00:01.723 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:01.723 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.723 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.723 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:01.723 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619969+0000", 2024-09-21T15:00:01.723 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:01.723 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.723 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.723 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:01.723 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619969+0000", 2024-09-21T15:00:01.723 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:01.723 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.724 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.724 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:01.724 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619971+0000", 2024-09-21T15:00:01.724 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1830000000000001e-06 2024-09-21T15:00:01.724 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.724 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.724 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:01.724 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619976+0000", 2024-09-21T15:00:01.724 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7129999999999996e-06 2024-09-21T15:00:01.724 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.724 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.724 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:01.724 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619981+0000", 2024-09-21T15:00:01.724 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2460000000000003e-06 2024-09-21T15:00:01.724 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.725 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.725 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:01.725 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628602+0000", 2024-09-21T15:00:01.725 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086209519999999994 2024-09-21T15:00:01.725 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.725 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.725 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:01.725 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628734+0000", 2024-09-21T15:00:01.725 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000132242 2024-09-21T15:00:01.725 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.725 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.725 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:01.725 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628998+0000", 2024-09-21T15:00:01.725 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00026419799999999999 2024-09-21T15:00:01.725 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:01.726 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:01.726 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:01.726 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.726 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.726 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12215 3.4 3:21b43b45:::rbd_data.11363a21d2be.000000000000002b:10 [read 409600~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:01.726 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.619985+0000", 2024-09-21T15:00:01.726 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 82.071538994999997, 2024-09-21T15:00:01.726 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0095457319999999995, 2024-09-21T15:00:01.726 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:01.726 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:01.726 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:01.726 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:01.726 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:01.726 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12215 2024-09-21T15:00:01.726 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.727 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:01.727 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.727 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:01.727 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619985+0000", 2024-09-21T15:00:01.727 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:01.727 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.727 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.727 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:01.727 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619985+0000", 2024-09-21T15:00:01.727 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:01.727 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.727 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.727 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:01.727 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619985+0000", 2024-09-21T15:00:01.727 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:01.727 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.728 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.728 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:01.728 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619987+0000", 2024-09-21T15:00:01.728 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6129999999999998e-06 2024-09-21T15:00:01.728 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.728 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.728 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:01.728 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619993+0000", 2024-09-21T15:00:01.728 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0820000000000004e-06 2024-09-21T15:00:01.728 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.728 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.728 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:01.728 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619998+0000", 2024-09-21T15:00:01.728 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7759999999999997e-06 2024-09-21T15:00:01.728 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.728 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.729 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:01.729 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629022+0000", 2024-09-21T15:00:01.729 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0090237489999999993 2024-09-21T15:00:01.729 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.729 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.729 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:01.729 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629098+0000", 2024-09-21T15:00:01.729 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5735999999999995e-05 2024-09-21T15:00:01.729 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.729 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.729 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:01.729 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629531+0000", 2024-09-21T15:00:01.729 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00043310100000000003 2024-09-21T15:00:01.729 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:01.729 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:01.730 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:01.730 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.730 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.730 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12216 3.4 3:21b43b45:::rbd_data.11363a21d2be.000000000000002b:10 [read 516096~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:01.730 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.620003+0000", 2024-09-21T15:00:01.730 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 82.071520999000001, 2024-09-21T15:00:01.730 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0098850169999999994, 2024-09-21T15:00:01.730 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:01.730 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:01.730 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:01.730 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:01.730 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:01.730 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12216 2024-09-21T15:00:01.730 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.730 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:01.731 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.731 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:01.731 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620003+0000", 2024-09-21T15:00:01.731 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:01.731 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.731 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.731 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:01.731 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620003+0000", 2024-09-21T15:00:01.731 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:01.731 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.731 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.731 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:01.731 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620003+0000", 2024-09-21T15:00:01.731 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:01.731 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.731 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.732 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:01.732 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620006+0000", 2024-09-21T15:00:01.732 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4350000000000001e-06 2024-09-21T15:00:01.732 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.732 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.732 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:01.732 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620013+0000", 2024-09-21T15:00:01.732 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4219999999999997e-06 2024-09-21T15:00:01.732 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.732 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.732 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:01.732 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620017+0000", 2024-09-21T15:00:01.732 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.865e-06 2024-09-21T15:00:01.732 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.732 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.733 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:01.733 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629581+0000", 2024-09-21T15:00:01.733 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0095631989999999997 2024-09-21T15:00:01.733 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.733 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.733 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:01.733 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629691+0000", 2024-09-21T15:00:01.733 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000110586 2024-09-21T15:00:01.733 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.733 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.733 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:01.733 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629888+0000", 2024-09-21T15:00:01.733 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019691199999999999 2024-09-21T15:00:01.733 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:01.734 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:01.734 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:01.734 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.734 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.734 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12222 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 49152~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:01.734 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.620027+0000", 2024-09-21T15:00:01.734 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 82.071496652999997, 2024-09-21T15:00:01.734 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010228131999999999, 2024-09-21T15:00:01.734 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:01.734 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:01.734 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:01.734 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:01.734 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:01.734 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12222 2024-09-21T15:00:01.735 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.735 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:01.735 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.735 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:01.735 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620027+0000", 2024-09-21T15:00:01.735 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:01.735 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.735 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.735 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:01.735 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620027+0000", 2024-09-21T15:00:01.735 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:01.735 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.735 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.735 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:01.736 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620027+0000", 2024-09-21T15:00:01.736 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:01.736 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.736 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.736 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:01.736 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620030+0000", 2024-09-21T15:00:01.736 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0400000000000001e-06 2024-09-21T15:00:01.736 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.736 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.736 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:01.736 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620037+0000", 2024-09-21T15:00:01.736 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8220000000000001e-06 2024-09-21T15:00:01.736 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.736 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.736 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:01.736 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620042+0000", 2024-09-21T15:00:01.737 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.843e-06 2024-09-21T15:00:01.737 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.737 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.737 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:01.737 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629922+0000", 2024-09-21T15:00:01.737 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0098805100000000003 2024-09-21T15:00:01.737 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.737 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.737 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:01.737 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.630020+0000", 2024-09-21T15:00:01.737 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7596999999999994e-05 2024-09-21T15:00:01.737 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.737 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.737 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:01.737 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.630256+0000", 2024-09-21T15:00:01.738 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00023573299999999999 2024-09-21T15:00:01.738 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:01.738 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:01.738 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:01.738 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.738 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.738 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12253 3.4 3:21b43b45:::rbd_data.11363a21d2be.000000000000002b:10 [read 417792~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:01.738 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.624578+0000", 2024-09-21T15:00:01.738 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 82.066946285, 2024-09-21T15:00:01.738 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0085112079999999993, 2024-09-21T15:00:01.738 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:01.738 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:01.738 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:01.738 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:01.738 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:01.738 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12253 2024-09-21T15:00:01.739 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.739 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:01.739 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.739 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:01.739 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624578+0000", 2024-09-21T15:00:01.739 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:01.739 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.739 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.739 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:01.739 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624578+0000", 2024-09-21T15:00:01.739 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:01.739 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.739 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.739 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:01.739 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624578+0000", 2024-09-21T15:00:01.740 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:01.740 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.740 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.740 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:01.740 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624579+0000", 2024-09-21T15:00:01.740 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.688e-06 2024-09-21T15:00:01.740 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.740 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.740 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:01.740 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624582+0000", 2024-09-21T15:00:01.740 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2260000000000001e-06 2024-09-21T15:00:01.740 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.740 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.740 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:01.740 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624585+0000", 2024-09-21T15:00:01.741 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4820000000000001e-06 2024-09-21T15:00:01.741 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.741 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.741 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:01.741 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.632580+0000", 2024-09-21T15:00:01.741 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0079948959999999996 2024-09-21T15:00:01.741 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.741 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.741 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:01.741 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.632683+0000", 2024-09-21T15:00:01.741 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010308 2024-09-21T15:00:01.741 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.741 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.741 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:01.741 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.633089+0000", 2024-09-21T15:00:01.742 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00040612599999999998 2024-09-21T15:00:01.742 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:01.742 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:01.742 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:01.742 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.742 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.742 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12260 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 57344~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:01.742 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.624587+0000", 2024-09-21T15:00:01.742 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 82.066936798, 2024-09-21T15:00:01.742 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0088282069999999994, 2024-09-21T15:00:01.742 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:01.742 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:01.742 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:01.742 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:01.743 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:01.743 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12260 2024-09-21T15:00:01.743 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.743 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:01.743 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.743 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:01.743 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624587+0000", 2024-09-21T15:00:01.743 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:01.743 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.743 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.743 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:01.743 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624587+0000", 2024-09-21T15:00:01.743 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:01.743 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.743 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.743 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:01.744 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624587+0000", 2024-09-21T15:00:01.744 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-21T15:00:01.744 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.744 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.744 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:01.744 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624589+0000", 2024-09-21T15:00:01.744 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4339999999999999e-06 2024-09-21T15:00:01.744 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.744 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.744 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:01.744 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624593+0000", 2024-09-21T15:00:01.744 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5859999999999998e-06 2024-09-21T15:00:01.744 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.744 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.744 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:01.744 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624596+0000", 2024-09-21T15:00:01.745 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5830000000000001e-06 2024-09-21T15:00:01.745 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.745 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.745 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:01.745 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.633132+0000", 2024-09-21T15:00:01.745 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.008536666 2024-09-21T15:00:01.745 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.745 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.745 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:01.745 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.633216+0000", 2024-09-21T15:00:01.745 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3387999999999995e-05 2024-09-21T15:00:01.745 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.745 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.745 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:01.745 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.633415+0000", 2024-09-21T15:00:01.746 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019976799999999999 2024-09-21T15:00:01.746 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:01.746 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:01.746 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:01.746 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.746 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.746 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12516 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 245760~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:01.746 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645194+0000", 2024-09-21T15:00:01.746 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 82.046330260999994, 2024-09-21T15:00:01.746 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0084284990000000008, 2024-09-21T15:00:01.746 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:01.746 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:01.746 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:01.746 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:01.747 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:01.747 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12516 2024-09-21T15:00:01.747 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.747 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:01.747 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.747 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:01.747 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645194+0000", 2024-09-21T15:00:01.747 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:01.747 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.747 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.747 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:01.747 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645194+0000", 2024-09-21T15:00:01.747 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:01.747 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.747 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.748 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:01.748 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645193+0000", 2024-09-21T15:00:01.748 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:01.748 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.748 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.748 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:01.748 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645196+0000", 2024-09-21T15:00:01.748 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1030000000000002e-06 2024-09-21T15:00:01.748 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.748 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.748 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:01.748 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645205+0000", 2024-09-21T15:00:01.748 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6200000000000005e-06 2024-09-21T15:00:01.748 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.748 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.748 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:01.749 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645210+0000", 2024-09-21T15:00:01.749 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6280000000000004e-06 2024-09-21T15:00:01.749 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.749 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.749 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:01.749 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.653305+0000", 2024-09-21T15:00:01.749 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0080957879999999996 2024-09-21T15:00:01.749 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.749 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.749 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:01.749 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.653389+0000", 2024-09-21T15:00:01.749 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3242e-05 2024-09-21T15:00:01.749 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.749 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.749 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:01.750 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.653622+0000", 2024-09-21T15:00:01.750 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00023361599999999999 2024-09-21T15:00:01.750 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:01.750 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:01.750 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:01.750 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.750 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.750 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12517 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 278528~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:01.750 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645214+0000", 2024-09-21T15:00:01.750 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 82.046310148000003, 2024-09-21T15:00:01.750 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086832899999999998, 2024-09-21T15:00:01.750 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:01.750 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:01.750 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:01.750 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:01.750 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:01.751 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12517 2024-09-21T15:00:01.751 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.751 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:01.751 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.751 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:01.751 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645214+0000", 2024-09-21T15:00:01.751 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:01.751 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.751 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.751 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:01.751 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645214+0000", 2024-09-21T15:00:01.751 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:01.751 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.751 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.751 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:01.751 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645214+0000", 2024-09-21T15:00:01.752 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:01.752 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.752 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.752 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:01.752 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645216+0000", 2024-09-21T15:00:01.752 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5579999999999999e-06 2024-09-21T15:00:01.752 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.752 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.752 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:01.752 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645224+0000", 2024-09-21T15:00:01.752 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2439999999999993e-06 2024-09-21T15:00:01.752 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.752 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.752 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:01.752 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645231+0000", 2024-09-21T15:00:01.753 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1859999999999997e-06 2024-09-21T15:00:01.753 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.753 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.753 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:01.753 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.653651+0000", 2024-09-21T15:00:01.753 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0084204220000000003 2024-09-21T15:00:01.753 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.753 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.753 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:01.753 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.653726+0000", 2024-09-21T15:00:01.753 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5356000000000002e-05 2024-09-21T15:00:01.753 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.753 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.753 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:01.753 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.653897+0000", 2024-09-21T15:00:01.754 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000170848 2024-09-21T15:00:01.754 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:01.754 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:01.754 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:01.754 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.754 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.754 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12518 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 385024~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:01.754 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645235+0000", 2024-09-21T15:00:01.754 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 82.046288922000002, 2024-09-21T15:00:01.754 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0089841020000000008, 2024-09-21T15:00:01.754 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:01.754 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:01.754 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:01.754 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:01.754 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:01.755 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12518 2024-09-21T15:00:01.755 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.755 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:01.755 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.755 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:01.755 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645235+0000", 2024-09-21T15:00:01.755 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:01.755 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.755 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.755 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:01.755 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645235+0000", 2024-09-21T15:00:01.755 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:01.755 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.755 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.755 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:01.755 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645235+0000", 2024-09-21T15:00:01.756 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:01.756 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.756 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.756 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:01.756 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645238+0000", 2024-09-21T15:00:01.756 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.869e-06 2024-09-21T15:00:01.756 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.756 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.756 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:01.756 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645244+0000", 2024-09-21T15:00:01.756 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8930000000000002e-06 2024-09-21T15:00:01.756 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.756 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.756 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:01.756 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645249+0000", 2024-09-21T15:00:01.756 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2859999999999999e-06 2024-09-21T15:00:01.757 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.757 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.757 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:01.757 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.653930+0000", 2024-09-21T15:00:01.757 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086815320000000005 2024-09-21T15:00:01.757 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.757 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.757 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:01.757 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654028+0000", 2024-09-21T15:00:01.757 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7578000000000007e-05 2024-09-21T15:00:01.757 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.757 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.757 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:01.758 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654219+0000", 2024-09-21T15:00:01.758 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019132099999999999 2024-09-21T15:00:01.758 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:01.758 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:01.758 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:01.758 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.758 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.758 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12538 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 147456~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:01.758 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645253+0000", 2024-09-21T15:00:01.758 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 82.046271297999994, 2024-09-21T15:00:01.758 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0092095360000000008, 2024-09-21T15:00:01.758 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:01.758 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:01.758 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:01.758 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:01.759 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:01.759 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12538 2024-09-21T15:00:01.759 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.759 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:01.759 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.759 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:01.759 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645253+0000", 2024-09-21T15:00:01.759 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:01.759 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.759 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.759 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:01.759 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645253+0000", 2024-09-21T15:00:01.759 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:01.759 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.759 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.759 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:01.760 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645252+0000", 2024-09-21T15:00:01.760 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:01.760 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.760 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.760 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:01.760 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645255+0000", 2024-09-21T15:00:01.760 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2290000000000002e-06 2024-09-21T15:00:01.760 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.760 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.760 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:01.760 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645267+0000", 2024-09-21T15:00:01.760 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2631e-05 2024-09-21T15:00:01.760 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.760 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.760 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:01.760 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645278+0000", 2024-09-21T15:00:01.761 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0847e-05 2024-09-21T15:00:01.761 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.761 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.761 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:01.761 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654251+0000", 2024-09-21T15:00:01.761 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0089731340000000007 2024-09-21T15:00:01.761 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.761 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.761 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:01.761 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654296+0000", 2024-09-21T15:00:01.761 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4353e-05 2024-09-21T15:00:01.761 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.761 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.761 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:01.761 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654462+0000", 2024-09-21T15:00:01.761 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016668499999999999 2024-09-21T15:00:01.762 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:01.762 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:01.762 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:01.762 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.762 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.762 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12539 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 253952~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:01.762 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645284+0000", 2024-09-21T15:00:01.762 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 82.046240076999993, 2024-09-21T15:00:01.762 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0094524029999999998, 2024-09-21T15:00:01.762 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:01.762 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:01.762 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:01.762 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:01.762 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:01.762 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12539 2024-09-21T15:00:01.763 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.763 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:01.763 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.763 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:01.763 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645284+0000", 2024-09-21T15:00:01.763 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:01.763 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.763 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.763 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:01.763 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645284+0000", 2024-09-21T15:00:01.763 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:01.763 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.763 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.763 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:01.763 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645284+0000", 2024-09-21T15:00:01.764 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:01.764 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.764 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.764 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:01.764 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645286+0000", 2024-09-21T15:00:01.764 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7159999999999999e-06 2024-09-21T15:00:01.764 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.764 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.764 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:01.764 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645292+0000", 2024-09-21T15:00:01.764 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7400000000000001e-06 2024-09-21T15:00:01.764 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.764 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.764 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:01.764 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645297+0000", 2024-09-21T15:00:01.765 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9019999999999998e-06 2024-09-21T15:00:01.765 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.765 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.765 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:01.765 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654490+0000", 2024-09-21T15:00:01.765 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0091932360000000005 2024-09-21T15:00:01.765 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.765 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.765 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:01.765 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654565+0000", 2024-09-21T15:00:01.765 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4657e-05 2024-09-21T15:00:01.765 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.765 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.765 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:01.766 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654736+0000", 2024-09-21T15:00:01.766 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017156199999999999 2024-09-21T15:00:01.766 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:01.766 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:01.766 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:01.766 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.766 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.766 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12540 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 286720~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:01.766 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645302+0000", 2024-09-21T15:00:01.766 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 82.046221807999999, 2024-09-21T15:00:01.766 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0096603090000000006, 2024-09-21T15:00:01.766 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:01.766 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:01.766 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:01.766 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:01.767 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:01.767 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12540 2024-09-21T15:00:01.767 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.767 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:01.767 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.767 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:01.767 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645302+0000", 2024-09-21T15:00:01.767 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:01.767 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.767 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.767 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:01.767 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645302+0000", 2024-09-21T15:00:01.767 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:01.767 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.767 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.768 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:01.768 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645302+0000", 2024-09-21T15:00:01.768 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:01.768 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.768 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.768 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:01.768 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645305+0000", 2024-09-21T15:00:01.768 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7329999999999999e-06 2024-09-21T15:00:01.768 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.768 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.768 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:01.768 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645316+0000", 2024-09-21T15:00:01.768 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1832e-05 2024-09-21T15:00:01.768 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.768 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.768 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:01.769 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645322+0000", 2024-09-21T15:00:01.769 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.169e-06 2024-09-21T15:00:01.769 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.769 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.769 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:01.769 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654778+0000", 2024-09-21T15:00:01.769 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0094559090000000002 2024-09-21T15:00:01.769 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.769 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.769 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:01.769 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654828+0000", 2024-09-21T15:00:01.769 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0461999999999999e-05 2024-09-21T15:00:01.769 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.769 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.769 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:01.770 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654963+0000", 2024-09-21T15:00:01.770 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000134583 2024-09-21T15:00:01.770 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:01.770 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:01.770 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:01.770 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.770 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.770 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12561 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:01.770 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645742+0000", 2024-09-21T15:00:01.770 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 82.045782541999998, 2024-09-21T15:00:01.770 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0093583810000000007, 2024-09-21T15:00:01.770 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:01.770 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:01.770 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:01.771 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:01.771 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:01.771 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12561 2024-09-21T15:00:01.771 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.771 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:01.771 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.771 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:01.771 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645742+0000", 2024-09-21T15:00:01.771 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:01.771 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.771 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.771 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:01.771 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645742+0000", 2024-09-21T15:00:01.771 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:01.771 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.772 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.772 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:01.772 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645741+0000", 2024-09-21T15:00:01.772 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:00:01.772 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.772 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.772 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:01.772 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645746+0000", 2024-09-21T15:00:01.772 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8379999999999998e-06 2024-09-21T15:00:01.772 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.772 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.772 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:01.772 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645757+0000", 2024-09-21T15:00:01.772 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0705e-05 2024-09-21T15:00:01.772 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.773 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.773 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:01.773 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645765+0000", 2024-09-21T15:00:01.773 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6729999999999992e-06 2024-09-21T15:00:01.773 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.773 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.773 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:01.773 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654993+0000", 2024-09-21T15:00:01.773 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0092287480000000002 2024-09-21T15:00:01.773 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.773 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.773 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:01.773 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.655050+0000", 2024-09-21T15:00:01.773 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6394999999999998e-05 2024-09-21T15:00:01.773 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.773 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.774 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:01.774 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.655100+0000", 2024-09-21T15:00:01.774 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0049000000000001e-05 2024-09-21T15:00:01.774 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:01.774 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:01.774 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:01.774 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.774 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.774 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:15431 3.12 3:4bc7d206:::rbd_data.1136f5ed5dd6.0000000000000051:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-21T15:00:01.774 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:41.673445+0000", 2024-09-21T15:00:01.774 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 80.018079208000003, 2024-09-21T15:00:01.774 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086103610000000004, 2024-09-21T15:00:01.774 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:01.774 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:01.774 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:01.775 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:01.775 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:01.775 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 15431 2024-09-21T15:00:01.775 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.775 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:01.775 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.775 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:01.775 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.673445+0000", 2024-09-21T15:00:01.775 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:01.775 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.775 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.775 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:01.775 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.673445+0000", 2024-09-21T15:00:01.775 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:01.775 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.776 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.776 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:01.776 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.673444+0000", 2024-09-21T15:00:01.776 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:00:01.776 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.776 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.776 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:01.776 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.673451+0000", 2024-09-21T15:00:01.776 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4739999999999999e-06 2024-09-21T15:00:01.776 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.776 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.776 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:01.776 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.673472+0000", 2024-09-21T15:00:01.776 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0687e-05 2024-09-21T15:00:01.776 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.776 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.777 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:01.777 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.673481+0000", 2024-09-21T15:00:01.777 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3349999999999996e-06 2024-09-21T15:00:01.777 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.777 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.777 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:01.777 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.673565+0000", 2024-09-21T15:00:01.777 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4011000000000006e-05 2024-09-21T15:00:01.777 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.777 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.777 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:01.777 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.673650+0000", 2024-09-21T15:00:01.777 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4949000000000003e-05 2024-09-21T15:00:01.777 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.777 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.777 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 3", 2024-09-21T15:00:01.778 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.675014+0000", 2024-09-21T15:00:01.778 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0013636620000000001 2024-09-21T15:00:01.778 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.778 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.778 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:01.778 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.681866+0000", 2024-09-21T15:00:01.778 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0068525640000000002 2024-09-21T15:00:01.778 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.778 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.778 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:01.778 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.681885+0000", 2024-09-21T15:00:01.778 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8519999999999999e-05 2024-09-21T15:00:01.778 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.778 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.778 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:01.778 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.681961+0000", 2024-09-21T15:00:01.779 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6248000000000006e-05 2024-09-21T15:00:01.779 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:01.779 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:01.779 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:01.779 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.682055+0000", 2024-09-21T15:00:01.779 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4184000000000005e-05 2024-09-21T15:00:01.779 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:01.779 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:01.779 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:01.779 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:01.779 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:01.779 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:01.780 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T15:00:01.781 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:00:01.774+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:00:01.882 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:00:01.874+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:00:01.906 INFO:teuthology.orchestra.run.smithi032.stdout:2774 read 0x16744dc thru 0x167ccac (0x87d1 bytes) 2024-09-21T15:00:01.907 INFO:teuthology.orchestra.run.smithi032.stdout:2777 writesame 0x79d2cce thru 0x79dd491 data_size 0x12(0xa7c4 bytes) 2024-09-21T15:00:01.919 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:01.919 INFO:teuthology.orchestra.run.smithi032.stdout:2779 read 0x85f07d thru 0x86c3ff (0xd383 bytes) 2024-09-21T15:00:01.920 INFO:teuthology.orchestra.run.smithi032.stdout:2782 read 0x3c4c662 thru 0x3c51c4a (0x55e9 bytes) 2024-09-21T15:00:01.932 INFO:teuthology.orchestra.run.smithi032.stdout:2783 write 0xbfdd940 thru 0xbfe5c1c (0x82dd bytes) 2024-09-21T15:00:01.935 INFO:teuthology.orchestra.run.smithi032.stdout:2785 read 0x99b814 thru 0x9a16c8 (0x5eb5 bytes) 2024-09-21T15:00:01.936 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:01.937 INFO:teuthology.orchestra.run.smithi032.stdout:2788 writesame 0x68fce92 thru 0x69022a8 data_size 0x1(0x5417 bytes) 2024-09-21T15:00:01.939 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:01.939 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:00:01.939 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T15:00:01.939 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:01.949 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T15:00:01.960 INFO:teuthology.orchestra.run.smithi032.stdout:2791 write 0xd5a984e thru 0xd5aaec8 (0x167b bytes) 2024-09-21T15:00:01.964 INFO:teuthology.orchestra.run.smithi032.stdout:2792 read 0x9e6562c thru 0x9e7157f (0xbf54 bytes) 2024-09-21T15:00:01.966 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:01.966 INFO:teuthology.orchestra.run.smithi032.stdout:2794 read 0x2b852d2 thru 0x2b8ba4a (0x6779 bytes) 2024-09-21T15:00:01.987 INFO:teuthology.orchestra.run.smithi032.stdout:2795 trunc from 0xd5aaec9 to 0xe6cd71 2024-09-21T15:00:01.994 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:00:01.990+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:00:02.005 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T15:00:02.046 INFO:teuthology.orchestra.run.smithi032.stdout:2796 write 0xc85c13a thru 0xc85d4d1 (0x1398 bytes) 2024-09-21T15:00:02.051 INFO:teuthology.orchestra.run.smithi032.stdout:2799 trunc from 0xc85d4d2 to 0xc128265 2024-09-21T15:00:02.060 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:02.060 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:00:02.060 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T15:00:02.060 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T15:00:02.060 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:02.061 INFO:teuthology.orchestra.run.smithi032.stdout:2800 punch from 0x10d0c0d to 0x10d5ffe, (0x53f1 bytes) 2024-09-21T15:00:02.061 INFO:teuthology.orchestra.run.smithi032.stdout:2801 read 0x70dbc15 thru 0x70dce12 (0x11fe bytes) 2024-09-21T15:00:02.061 INFO:teuthology.orchestra.run.smithi032.stdout:2802 read 0x513b491 thru 0x513cfdc (0x1b4c bytes) 2024-09-21T15:00:02.070 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T15:00:02.076 INFO:teuthology.orchestra.run.smithi032.stdout:2804 trunc from 0xc128265 to 0x3804881 2024-09-21T15:00:02.083 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:00:02.078+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:00:02.092 INFO:teuthology.orchestra.run.smithi032.stdout:2806 read 0x22e0e66 thru 0x22e9072 (0x820d bytes) 2024-09-21T15:00:02.109 INFO:teuthology.orchestra.run.smithi032.stdout:2808 trunc from 0x3804881 to 0xdaf2ce5 2024-09-21T15:00:02.138 INFO:teuthology.orchestra.run.smithi032.stdout:2811 trunc from 0xdaf2ce5 to 0x125e6ee 2024-09-21T15:00:02.152 INFO:teuthology.orchestra.run.smithi032.stdout:2813 write 0x44fdc9 thru 0x450d0f (0xf47 bytes) 2024-09-21T15:00:02.156 INFO:teuthology.orchestra.run.smithi032.stdout:2815 punch from 0x82c1d0 to 0x8322f9, (0x6129 bytes) 2024-09-21T15:00:02.156 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T15:00:02.159 INFO:teuthology.orchestra.run.smithi032.stdout:2817 trunc from 0x125e6ee to 0x3232015 2024-09-21T15:00:02.169 INFO:teuthology.orchestra.run.smithi032.stdout:2818 punch from 0x1b8abae to 0x1b8e373, (0x37c5 bytes) 2024-09-21T15:00:02.169 INFO:teuthology.orchestra.run.smithi032.stdout:2820 writesame 0x17c5f52 thru 0x17c7498 data_size 0x1(0x1547 bytes) 2024-09-21T15:00:02.180 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T15:00:02.184 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:00:02.181+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:00:02.187 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:02.187 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T15:00:02.187 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T15:00:02.187 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T15:00:02.188 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.188 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:8049 3.b 3:d1b12ef9:::rbd_data.11361bed1f78.0000000000000062:d [read 114688~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T15:00:02.188 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:35.227371+0000", 2024-09-21T15:00:02.188 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 86.956831993999998, 2024-09-21T15:00:02.188 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0078533609999999997, 2024-09-21T15:00:02.188 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:02.188 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:02.188 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:02.188 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:02.188 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:02.188 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 8049 2024-09-21T15:00:02.188 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.188 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:02.189 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.189 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:02.189 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227371+0000", 2024-09-21T15:00:02.189 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:02.189 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.189 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.189 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:02.189 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227371+0000", 2024-09-21T15:00:02.189 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:02.189 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.189 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.189 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:02.189 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227370+0000", 2024-09-21T15:00:02.189 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:00:02.189 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.190 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.190 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:02.190 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227378+0000", 2024-09-21T15:00:02.190 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.379e-06 2024-09-21T15:00:02.190 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.190 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.190 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:02.190 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227388+0000", 2024-09-21T15:00:02.190 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0175e-05 2024-09-21T15:00:02.190 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.190 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.190 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:02.190 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.227393+0000", 2024-09-21T15:00:02.190 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6690000000000003e-06 2024-09-21T15:00:02.190 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.191 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.191 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:02.191 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.234964+0000", 2024-09-21T15:00:02.191 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0075705319999999996 2024-09-21T15:00:02.191 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.191 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.191 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:02.191 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.235041+0000", 2024-09-21T15:00:02.191 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7170999999999997e-05 2024-09-21T15:00:02.191 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.191 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.191 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:02.191 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:35.235225+0000", 2024-09-21T15:00:02.191 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000183735 2024-09-21T15:00:02.192 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:02.192 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:02.192 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:02.192 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.192 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.192 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31476 3.c 3:35718b6b:::rbd_data.11367721c19.0000000000000014:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T15:00:02.192 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.505975+0000", 2024-09-21T15:00:02.192 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 63.678228462, 2024-09-21T15:00:02.192 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99515995899999998, 2024-09-21T15:00:02.192 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:02.192 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:02.192 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:02.192 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:02.192 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:02.192 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31476 2024-09-21T15:00:02.193 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.193 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:02.193 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.193 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:02.193 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.505975+0000", 2024-09-21T15:00:02.193 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:02.193 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.193 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.193 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:02.193 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.505975+0000", 2024-09-21T15:00:02.193 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:02.193 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.193 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.193 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:02.193 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.505971+0000", 2024-09-21T15:00:02.194 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999967 2024-09-21T15:00:02.194 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.194 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.194 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:02.194 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.505989+0000", 2024-09-21T15:00:02.194 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7985000000000001e-05 2024-09-21T15:00:02.194 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.194 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.194 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:02.194 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506050+0000", 2024-09-21T15:00:02.194 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0659000000000003e-05 2024-09-21T15:00:02.194 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.194 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.195 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:02.195 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506074+0000", 2024-09-21T15:00:02.195 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4156e-05 2024-09-21T15:00:02.195 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.195 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.195 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:02.195 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506225+0000", 2024-09-21T15:00:02.195 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015097 2024-09-21T15:00:02.195 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.195 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.195 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:00:02.195 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506256+0000", 2024-09-21T15:00:02.195 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0505999999999999e-05 2024-09-21T15:00:02.196 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.196 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.196 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:02.196 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.497974+0000", 2024-09-21T15:00:02.196 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99171825700000005 2024-09-21T15:00:02.196 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.196 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.196 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for missing object", 2024-09-21T15:00:02.196 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.498449+0000", 2024-09-21T15:00:02.196 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00047506399999999998 2024-09-21T15:00:02.196 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.196 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.196 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:02.197 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.501000+0000", 2024-09-21T15:00:02.197 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0025506919999999998 2024-09-21T15:00:02.197 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.197 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.197 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:02.197 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.501135+0000", 2024-09-21T15:00:02.197 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013524000000000001 2024-09-21T15:00:02.197 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:02.197 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:02.197 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:02.197 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.197 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.197 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31480 3.c 3:36ccd808:::rbd_data.11367721c19.0000000000000018:head [sparse-read 0~14821] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T15:00:02.197 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.506116+0000", 2024-09-21T15:00:02.197 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 63.678087118999997, 2024-09-21T15:00:02.198 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99316033599999998, 2024-09-21T15:00:02.198 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:02.198 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:02.198 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:02.198 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:02.198 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:02.198 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31480 2024-09-21T15:00:02.198 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.198 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:02.198 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.198 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:02.198 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506116+0000", 2024-09-21T15:00:02.198 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:02.199 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.199 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.199 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:02.199 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506116+0000", 2024-09-21T15:00:02.199 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:02.199 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.199 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.199 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:02.199 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506115+0000", 2024-09-21T15:00:02.199 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:00:02.199 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.199 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.199 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:02.199 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506124+0000", 2024-09-21T15:00:02.200 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0659999999999993e-06 2024-09-21T15:00:02.200 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.200 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.200 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:02.200 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506145+0000", 2024-09-21T15:00:02.200 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1447e-05 2024-09-21T15:00:02.200 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.200 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.200 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:02.200 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506161+0000", 2024-09-21T15:00:02.200 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5209e-05 2024-09-21T15:00:02.200 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.200 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.200 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:02.201 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506337+0000", 2024-09-21T15:00:02.201 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017642999999999999 2024-09-21T15:00:02.201 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.201 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.201 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:00:02.201 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506375+0000", 2024-09-21T15:00:02.201 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7536000000000001e-05 2024-09-21T15:00:02.201 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.201 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.201 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:02.201 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.498815+0000", 2024-09-21T15:00:02.201 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99244067999999996 2024-09-21T15:00:02.201 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.201 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.202 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:02.202 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.498950+0000", 2024-09-21T15:00:02.202 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013436700000000001 2024-09-21T15:00:02.202 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.202 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.202 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:02.202 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499277+0000", 2024-09-21T15:00:02.202 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00032696800000000002 2024-09-21T15:00:02.202 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:02.202 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:02.202 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:02.202 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.202 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.203 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31513 3.c 3:36de8523:::rbd_data.11367721c19.0000000000000039:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T15:00:02.203 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.509071+0000", 2024-09-21T15:00:02.203 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 63.675132918999999, 2024-09-21T15:00:02.203 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99038225099999999, 2024-09-21T15:00:02.203 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:02.203 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:02.203 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:02.203 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:02.203 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:02.203 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31513 2024-09-21T15:00:02.203 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.203 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:02.203 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.204 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:02.204 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509071+0000", 2024-09-21T15:00:02.204 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:02.204 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.204 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.204 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:02.204 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509071+0000", 2024-09-21T15:00:02.204 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:02.204 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.204 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.204 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:02.204 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509068+0000", 2024-09-21T15:00:02.204 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-21T15:00:02.205 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.205 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.205 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:02.205 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509082+0000", 2024-09-21T15:00:02.205 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3545e-05 2024-09-21T15:00:02.205 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.205 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.205 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:02.205 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509116+0000", 2024-09-21T15:00:02.205 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3942000000000001e-05 2024-09-21T15:00:02.205 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.205 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.205 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:02.205 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509134+0000", 2024-09-21T15:00:02.205 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8839999999999999e-05 2024-09-21T15:00:02.206 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.206 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.206 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:02.206 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509318+0000", 2024-09-21T15:00:02.206 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018326799999999999 2024-09-21T15:00:02.206 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.206 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.206 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:00:02.206 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509346+0000", 2024-09-21T15:00:02.206 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8019000000000001e-05 2024-09-21T15:00:02.206 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.206 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.206 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:02.207 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499330+0000", 2024-09-21T15:00:02.207 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.98998399299999995 2024-09-21T15:00:02.207 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.207 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.207 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:02.207 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499453+0000", 2024-09-21T15:00:02.207 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000123115 2024-09-21T15:00:02.207 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:02.207 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:02.207 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:02.207 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.207 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.207 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31515 3.c 3:35b36ddd:::rbd_data.11367721c19.000000000000003b:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T15:00:02.208 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.509177+0000", 2024-09-21T15:00:02.208 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 63.675026492999997, 2024-09-21T15:00:02.208 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99044870600000001, 2024-09-21T15:00:02.208 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:02.208 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:02.208 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:02.208 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:02.208 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:02.208 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31515 2024-09-21T15:00:02.208 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.208 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:02.208 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.208 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:02.208 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509177+0000", 2024-09-21T15:00:02.209 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:02.209 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.209 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.209 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:02.209 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509177+0000", 2024-09-21T15:00:02.209 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:02.209 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.209 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.209 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:02.209 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509176+0000", 2024-09-21T15:00:02.209 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:00:02.209 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.209 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.210 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:02.210 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509183+0000", 2024-09-21T15:00:02.210 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6720000000000004e-06 2024-09-21T15:00:02.210 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.210 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.210 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:02.210 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509269+0000", 2024-09-21T15:00:02.210 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5881e-05 2024-09-21T15:00:02.210 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.210 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.210 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:02.210 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509303+0000", 2024-09-21T15:00:02.210 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3367000000000001e-05 2024-09-21T15:00:02.211 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.211 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.211 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:02.211 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509425+0000", 2024-09-21T15:00:02.211 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000122199 2024-09-21T15:00:02.211 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.211 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.211 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:00:02.211 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509448+0000", 2024-09-21T15:00:02.211 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2688999999999999e-05 2024-09-21T15:00:02.211 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.211 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.211 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:02.211 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499501+0000", 2024-09-21T15:00:02.211 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99005306999999998 2024-09-21T15:00:02.212 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.212 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.212 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:02.212 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499626+0000", 2024-09-21T15:00:02.212 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012501400000000001 2024-09-21T15:00:02.212 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:02.212 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:02.212 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:02.212 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.212 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.212 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31531 3.c 3:314f0287:::rbd_data.11367721c19.000000000000004b:head [sparse-read 0~7106] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T15:00:02.212 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.509329+0000", 2024-09-21T15:00:02.212 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 63.674874473000003, 2024-09-21T15:00:02.213 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99103085300000004, 2024-09-21T15:00:02.213 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:02.213 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:02.213 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:02.213 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:02.213 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:02.213 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31531 2024-09-21T15:00:02.213 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.213 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:02.213 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.213 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:02.213 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509329+0000", 2024-09-21T15:00:02.213 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:02.214 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.214 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.214 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:02.214 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509329+0000", 2024-09-21T15:00:02.214 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:02.214 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.214 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.214 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:02.214 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509328+0000", 2024-09-21T15:00:02.214 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:00:02.214 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.214 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.214 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:02.214 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509336+0000", 2024-09-21T15:00:02.215 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3170000000000002e-06 2024-09-21T15:00:02.215 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.215 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.215 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:02.215 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509353+0000", 2024-09-21T15:00:02.215 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7214000000000002e-05 2024-09-21T15:00:02.215 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.215 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.215 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:02.215 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509366+0000", 2024-09-21T15:00:02.215 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.26e-05 2024-09-21T15:00:02.215 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.215 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.215 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:02.216 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509523+0000", 2024-09-21T15:00:02.216 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015708000000000001 2024-09-21T15:00:02.216 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.216 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.216 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:00:02.216 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509544+0000", 2024-09-21T15:00:02.216 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0783000000000001e-05 2024-09-21T15:00:02.216 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.216 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.216 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:02.216 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499727+0000", 2024-09-21T15:00:02.216 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99018345200000002 2024-09-21T15:00:02.216 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.216 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.216 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:02.217 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499949+0000", 2024-09-21T15:00:02.217 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000221358 2024-09-21T15:00:02.217 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.217 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.217 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:02.217 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.500360+0000", 2024-09-21T15:00:02.217 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00041130700000000002 2024-09-21T15:00:02.217 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:02.217 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:02.217 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:02.217 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.217 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.217 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:61580 3.b 3:d3369e4f:::rbd_data.1136862f872f.0000000000000010:head [write 0~557056 [fadvise_sequential+fadvise_nocache],zero 557056~16384,write 573440~49152 [fadvise_sequential+fadvise_nocache],zero 622592~16384,write 638976~49152 [fadvise_sequential+fadvise_nocache],zero 688128~16384,write 704512~49152 [fadvise_sequential+fadvise_nocache],zero 753664~16384,write 770048~49152 [fadvise_sequential+fadvise_nocache],zero 819200~16384,write 835584~49152 [fadvise_sequential+fadvise_nocache],zero 884736~16384,write 901120~49152 [fadvise_sequential+fadvise_nocache],zero 950272~16384,write 966656~49152 [fadvise_sequential+fadvise_nocache],zero 1015808~16384,write 1032192~16384 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-21T15:00:02.218 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:19.231716+0000", 2024-09-21T15:00:02.218 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.952487310999999, 2024-09-21T15:00:02.218 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0096097690000000006, 2024-09-21T15:00:02.218 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:02.218 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:02.218 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:02.218 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:02.218 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:02.218 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 61580 2024-09-21T15:00:02.218 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.218 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:02.218 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.218 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:02.219 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.231716+0000", 2024-09-21T15:00:02.219 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:02.219 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.219 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.219 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:02.219 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.231716+0000", 2024-09-21T15:00:02.219 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:02.219 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.219 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.219 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:02.219 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.231715+0000", 2024-09-21T15:00:02.219 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:00:02.219 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.219 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.220 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:02.220 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.235290+0000", 2024-09-21T15:00:02.220 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0035750399999999998 2024-09-21T15:00:02.220 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.220 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.220 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:02.220 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.235310+0000", 2024-09-21T15:00:02.220 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0120999999999999e-05 2024-09-21T15:00:02.220 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.220 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.220 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:02.220 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.235319+0000", 2024-09-21T15:00:02.220 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2420000000000006e-06 2024-09-21T15:00:02.220 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.220 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.221 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:02.221 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.235416+0000", 2024-09-21T15:00:02.221 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7204e-05 2024-09-21T15:00:02.221 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.221 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.221 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:02.221 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.235775+0000", 2024-09-21T15:00:02.221 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000358883 2024-09-21T15:00:02.221 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.221 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.221 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T15:00:02.221 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.236560+0000", 2024-09-21T15:00:02.221 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000784463 2024-09-21T15:00:02.221 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.221 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.221 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:02.222 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.241119+0000", 2024-09-21T15:00:02.222 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0045590589999999999 2024-09-21T15:00:02.222 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.222 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.222 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:02.222 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.241210+0000", 2024-09-21T15:00:02.222 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1617000000000001e-05 2024-09-21T15:00:02.222 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.222 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.222 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:02.222 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.241285+0000", 2024-09-21T15:00:02.222 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4470999999999999e-05 2024-09-21T15:00:02.222 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.222 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.223 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:02.223 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.241326+0000", 2024-09-21T15:00:02.223 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0967000000000003e-05 2024-09-21T15:00:02.223 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:02.223 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:02.223 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:02.223 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.223 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.223 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:73699 3.b 3:d293b54a:::rbd_data.1136f4939602.000000000000000a:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-21T15:00:02.223 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:41.971359+0000", 2024-09-21T15:00:02.223 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 20.212844754999999, 2024-09-21T15:00:02.223 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0082840980000000002, 2024-09-21T15:00:02.223 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:02.223 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:02.224 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:02.224 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:02.224 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:02.224 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 73699 2024-09-21T15:00:02.224 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.224 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:02.224 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.224 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:02.224 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.971359+0000", 2024-09-21T15:00:02.224 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:02.224 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.224 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.224 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:02.224 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.971359+0000", 2024-09-21T15:00:02.225 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:02.225 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.225 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.225 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:02.225 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.971356+0000", 2024-09-21T15:00:02.225 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-21T15:00:02.225 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.225 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.225 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:02.225 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.971370+0000", 2024-09-21T15:00:02.225 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3349999999999999e-05 2024-09-21T15:00:02.225 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.226 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.226 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:02.226 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.971393+0000", 2024-09-21T15:00:02.226 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3799999999999999e-05 2024-09-21T15:00:02.226 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.226 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.226 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:02.226 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.971405+0000", 2024-09-21T15:00:02.226 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1569000000000001e-05 2024-09-21T15:00:02.226 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.226 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.226 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:02.226 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.971510+0000", 2024-09-21T15:00:02.226 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010456899999999999 2024-09-21T15:00:02.226 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.227 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.227 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:02.227 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.971752+0000", 2024-09-21T15:00:02.227 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00024250300000000001 2024-09-21T15:00:02.227 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.227 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.227 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T15:00:02.227 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.973012+0000", 2024-09-21T15:00:02.227 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0012595429999999999 2024-09-21T15:00:02.227 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.227 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.227 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:02.227 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.979357+0000", 2024-09-21T15:00:02.227 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0063454469999999997 2024-09-21T15:00:02.227 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.228 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.228 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:02.228 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.979467+0000", 2024-09-21T15:00:02.228 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000109623 2024-09-21T15:00:02.228 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.228 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.228 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:02.228 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.979585+0000", 2024-09-21T15:00:02.228 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000118044 2024-09-21T15:00:02.228 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.228 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.228 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:02.228 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.979643+0000", 2024-09-21T15:00:02.229 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8069999999999998e-05 2024-09-21T15:00:02.229 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:02.229 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:02.229 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:02.229 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.229 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.229 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:82545 3.b 3:d4eb1ee5:::rbd_data.11368296f791.0000000000000000:head [write 0~671744 [fadvise_sequential+fadvise_nocache],write 704512~65536 [fadvise_sequential+fadvise_nocache],write 786432~49152 [fadvise_sequential+fadvise_nocache],write 868352~49152 [fadvise_sequential+fadvise_nocache],write 933888~65536 [fadvise_sequential+fadvise_nocache],write 1032192~32768 [fadvise_sequential+fadvise_nocache],write 1097728~65536 [fadvise_sequential+fadvise_nocache],write 1179648~49152 [fadvise_sequential+fadvise_nocache],write 1261568~49152 [fadvise_sequential+fadvise_nocache],write 1327104~65536 [fadvise_sequential+fadvise_nocache],write 1425408~32768 [fadvise_sequential+fadvise_nocache],write 1490944~65536 [fadvise_sequential+fadvise_nocache],write 1572864~8192 [fadvise_sequential+fadvise_nocache],write 1630208~49152 [fadvise_sequential+fadvise_nocache],write 1703936~40960 [fadvise_sequential+fadvise_nocache],write 1794048~40960 [fadvise_sequential+fadvise_nocache],write 1859584~49152 [fadvise_sequential+fadvise_nocache],write 1957888~8192 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:00:02.229 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:47.416089+0000", 2024-09-21T15:00:02.229 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.768114748, 2024-09-21T15:00:02.229 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011896455, 2024-09-21T15:00:02.229 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:02.229 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:02.229 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:02.230 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:02.230 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:02.230 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82545 2024-09-21T15:00:02.230 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.230 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:02.230 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.230 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:02.230 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.416089+0000", 2024-09-21T15:00:02.230 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:02.230 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.230 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.230 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:02.230 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.416089+0000", 2024-09-21T15:00:02.230 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:02.230 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.231 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.231 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:02.231 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.416087+0000", 2024-09-21T15:00:02.231 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:00:02.231 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.231 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.231 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:02.231 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.419643+0000", 2024-09-21T15:00:02.231 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0035555550000000002 2024-09-21T15:00:02.231 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.231 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.231 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:02.231 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.419663+0000", 2024-09-21T15:00:02.231 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0202000000000001e-05 2024-09-21T15:00:02.231 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.232 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.232 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:02.232 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.419676+0000", 2024-09-21T15:00:02.232 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3207000000000001e-05 2024-09-21T15:00:02.232 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.232 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.232 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:02.232 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.419759+0000", 2024-09-21T15:00:02.232 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3521000000000002e-05 2024-09-21T15:00:02.232 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.232 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.232 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op must wait for map", 2024-09-21T15:00:02.232 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.419768+0000", 2024-09-21T15:00:02.232 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4479999999999994e-06 2024-09-21T15:00:02.232 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.232 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.233 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:02.233 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.422171+0000", 2024-09-21T15:00:02.233 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0024035469999999998 2024-09-21T15:00:02.233 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.233 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.233 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:02.233 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.422350+0000", 2024-09-21T15:00:02.233 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000179001 2024-09-21T15:00:02.233 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.233 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.233 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T15:00:02.233 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.422710+0000", 2024-09-21T15:00:02.233 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000359878 2024-09-21T15:00:02.233 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.234 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.234 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:02.234 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.426676+0000", 2024-09-21T15:00:02.234 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0039655790000000003 2024-09-21T15:00:02.234 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.234 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.234 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:02.234 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.427884+0000", 2024-09-21T15:00:02.234 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001208207 2024-09-21T15:00:02.234 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.234 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.234 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:02.234 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.427953+0000", 2024-09-21T15:00:02.234 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8533000000000003e-05 2024-09-21T15:00:02.234 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.235 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.235 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:02.235 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.427985+0000", 2024-09-21T15:00:02.235 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2481999999999998e-05 2024-09-21T15:00:02.235 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:02.235 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:02.235 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:02.235 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.235 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.235 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:83723 3.b 3:d4eb1ee5:::rbd_data.11368296f791.0000000000000000:40 [sparse-read 1179648~0 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-21T15:00:02.235 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:51.156687+0000", 2024-09-21T15:00:02.235 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 11.027516534, 2024-09-21T15:00:02.235 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0078343370000000002, 2024-09-21T15:00:02.235 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:02.236 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:02.236 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:02.236 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:02.236 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:02.236 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 83723 2024-09-21T15:00:02.236 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.236 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:02.236 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.236 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:02.236 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156687+0000", 2024-09-21T15:00:02.236 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:02.236 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.236 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.236 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:02.237 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156687+0000", 2024-09-21T15:00:02.237 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:02.237 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.237 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.237 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:02.237 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156686+0000", 2024-09-21T15:00:02.237 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:02.237 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.237 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.237 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:02.237 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156711+0000", 2024-09-21T15:00:02.237 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4776000000000001e-05 2024-09-21T15:00:02.237 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.237 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.237 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:02.238 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156723+0000", 2024-09-21T15:00:02.238 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1734e-05 2024-09-21T15:00:02.238 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.238 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.238 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:02.238 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156732+0000", 2024-09-21T15:00:02.238 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4390000000000006e-06 2024-09-21T15:00:02.238 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.238 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.238 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:02.238 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.164314+0000", 2024-09-21T15:00:02.238 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0075816579999999998 2024-09-21T15:00:02.238 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.239 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.239 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:02.239 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.164405+0000", 2024-09-21T15:00:02.239 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1247000000000002e-05 2024-09-21T15:00:02.239 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.239 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.239 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:02.239 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.164521+0000", 2024-09-21T15:00:02.239 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011599099999999999 2024-09-21T15:00:02.239 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:02.239 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:02.239 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:02.239 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.239 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.240 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:83724 3.b 3:d4eb1ee5:::rbd_data.11368296f791.0000000000000000:40 [sparse-read 2621440~0 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-21T15:00:02.240 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:51.156740+0000", 2024-09-21T15:00:02.240 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 11.027463035, 2024-09-21T15:00:02.240 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0079578040000000006, 2024-09-21T15:00:02.240 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:02.240 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:02.240 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:02.240 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:02.240 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:02.240 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 83724 2024-09-21T15:00:02.240 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.240 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:02.240 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.240 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:02.241 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156740+0000", 2024-09-21T15:00:02.241 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:02.241 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.241 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.241 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:02.241 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156740+0000", 2024-09-21T15:00:02.241 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:02.241 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.241 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.241 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:02.241 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156740+0000", 2024-09-21T15:00:02.241 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:02.241 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.241 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.241 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:02.242 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156744+0000", 2024-09-21T15:00:02.242 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0300000000000004e-06 2024-09-21T15:00:02.242 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.242 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.242 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:02.242 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156753+0000", 2024-09-21T15:00:02.242 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5759999999999995e-06 2024-09-21T15:00:02.242 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.242 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.242 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:02.242 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156761+0000", 2024-09-21T15:00:02.242 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.17e-06 2024-09-21T15:00:02.242 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.242 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.242 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:02.243 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.164559+0000", 2024-09-21T15:00:02.243 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0077988739999999999 2024-09-21T15:00:02.243 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.243 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.243 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:02.243 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.164608+0000", 2024-09-21T15:00:02.243 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8906999999999998e-05 2024-09-21T15:00:02.243 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.243 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.243 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:02.243 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.164698+0000", 2024-09-21T15:00:02.243 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9854999999999999e-05 2024-09-21T15:00:02.243 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:02.243 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:02.244 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:02.244 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.244 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.244 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:83735 3.b 3:d3640016:::rbd_data.11368296f791.0000000000000006:40 [sparse-read 393216~131072 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-21T15:00:02.244 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:51.156767+0000", 2024-09-21T15:00:02.244 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 11.027436124999999, 2024-09-21T15:00:02.244 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0082826410000000003, 2024-09-21T15:00:02.244 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:02.244 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:02.244 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:02.244 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:02.244 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:02.244 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 83735 2024-09-21T15:00:02.245 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.245 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:02.245 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.245 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:02.245 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156767+0000", 2024-09-21T15:00:02.245 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:02.245 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.245 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.245 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:02.245 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156767+0000", 2024-09-21T15:00:02.245 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:02.245 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.245 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.245 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:02.246 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156767+0000", 2024-09-21T15:00:02.246 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:02.246 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.246 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.246 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:02.246 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156771+0000", 2024-09-21T15:00:02.246 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0940000000000001e-06 2024-09-21T15:00:02.246 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.246 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.246 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:02.246 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156781+0000", 2024-09-21T15:00:02.246 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0267e-05 2024-09-21T15:00:02.246 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.246 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.246 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:02.247 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156789+0000", 2024-09-21T15:00:02.247 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7659999999999998e-06 2024-09-21T15:00:02.247 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.247 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.247 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:02.247 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.164742+0000", 2024-09-21T15:00:02.247 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0079534519999999997 2024-09-21T15:00:02.247 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.247 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.247 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:02.247 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.164820+0000", 2024-09-21T15:00:02.247 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7927000000000001e-05 2024-09-21T15:00:02.247 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.247 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.248 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:02.248 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165050+0000", 2024-09-21T15:00:02.248 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000229731 2024-09-21T15:00:02.248 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:02.248 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:02.248 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:02.248 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.248 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.248 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:83736 3.b 3:d3640016:::rbd_data.11368296f791.0000000000000006:40 [sparse-read 1835008~0 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-21T15:00:02.248 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:51.156796+0000", 2024-09-21T15:00:02.248 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 11.027407122, 2024-09-21T15:00:02.248 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0085139110000000007, 2024-09-21T15:00:02.248 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:02.248 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:02.248 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:02.249 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:02.249 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:02.249 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 83736 2024-09-21T15:00:02.249 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.249 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:02.249 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.249 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:02.249 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156796+0000", 2024-09-21T15:00:02.249 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:02.249 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.249 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.249 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:02.249 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156796+0000", 2024-09-21T15:00:02.249 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:02.250 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.250 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.250 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:02.250 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156796+0000", 2024-09-21T15:00:02.250 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:02.250 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.250 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.250 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:02.250 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156800+0000", 2024-09-21T15:00:02.250 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.34e-06 2024-09-21T15:00:02.250 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.250 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.250 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:02.250 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156813+0000", 2024-09-21T15:00:02.251 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2532e-05 2024-09-21T15:00:02.251 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.251 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.251 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:02.251 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156821+0000", 2024-09-21T15:00:02.251 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2069999999999995e-06 2024-09-21T15:00:02.251 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.251 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.251 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:02.251 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165100+0000", 2024-09-21T15:00:02.251 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0082788730000000008 2024-09-21T15:00:02.251 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.251 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.251 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:02.251 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165192+0000", 2024-09-21T15:00:02.252 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2095999999999994e-05 2024-09-21T15:00:02.252 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.252 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.252 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:02.252 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165310+0000", 2024-09-21T15:00:02.252 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000118461 2024-09-21T15:00:02.252 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:02.252 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:02.252 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:02.252 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.252 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.252 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:83739 3.b 3:d4eb1ee5:::rbd_data.11368296f791.0000000000000000:40 [sparse-read 524288~131072 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-21T15:00:02.252 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:51.156827+0000", 2024-09-21T15:00:02.252 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 11.027376039, 2024-09-21T15:00:02.252 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0087319900000000002, 2024-09-21T15:00:02.253 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:02.253 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:02.253 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:02.253 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:02.253 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:02.253 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 83739 2024-09-21T15:00:02.253 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.253 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:02.253 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.253 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:02.253 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156827+0000", 2024-09-21T15:00:02.253 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:02.253 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.253 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.254 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:02.254 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156827+0000", 2024-09-21T15:00:02.254 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:02.254 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.254 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.254 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:02.254 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156827+0000", 2024-09-21T15:00:02.254 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:02.254 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.254 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.254 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:02.254 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156831+0000", 2024-09-21T15:00:02.254 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9940000000000002e-06 2024-09-21T15:00:02.254 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.255 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.255 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:02.255 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156841+0000", 2024-09-21T15:00:02.255 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6760000000000002e-06 2024-09-21T15:00:02.255 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.255 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.255 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:02.255 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156853+0000", 2024-09-21T15:00:02.255 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1972e-05 2024-09-21T15:00:02.255 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.255 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.255 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:02.255 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165334+0000", 2024-09-21T15:00:02.255 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0084816939999999997 2024-09-21T15:00:02.255 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.255 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.256 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:02.256 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165375+0000", 2024-09-21T15:00:02.256 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0911000000000002e-05 2024-09-21T15:00:02.256 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.256 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.256 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:02.256 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165559+0000", 2024-09-21T15:00:02.256 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018429300000000001 2024-09-21T15:00:02.256 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:02.256 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:02.256 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:02.256 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.256 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.256 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:83740 3.b 3:d4eb1ee5:::rbd_data.11368296f791.0000000000000000:40 [sparse-read 1966080~0 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-21T15:00:02.257 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:51.156859+0000", 2024-09-21T15:00:02.257 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 11.027344049, 2024-09-21T15:00:02.257 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0088540089999999995, 2024-09-21T15:00:02.257 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:02.257 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:02.257 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:02.257 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:02.257 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:02.257 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 83740 2024-09-21T15:00:02.257 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.257 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:02.257 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.257 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:02.257 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156859+0000", 2024-09-21T15:00:02.258 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:02.258 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.258 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.258 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:02.258 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156859+0000", 2024-09-21T15:00:02.258 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:02.258 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.258 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.258 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:02.258 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156859+0000", 2024-09-21T15:00:02.258 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:02.258 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.258 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.259 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:02.259 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156863+0000", 2024-09-21T15:00:02.259 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.844e-06 2024-09-21T15:00:02.259 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.259 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.259 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:02.259 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156873+0000", 2024-09-21T15:00:02.259 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0069e-05 2024-09-21T15:00:02.259 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.259 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.259 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:02.259 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156880+0000", 2024-09-21T15:00:02.259 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4240000000000004e-06 2024-09-21T15:00:02.259 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.260 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.260 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:02.260 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165590+0000", 2024-09-21T15:00:02.260 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0087098100000000001 2024-09-21T15:00:02.260 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.260 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.260 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:02.260 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165656+0000", 2024-09-21T15:00:02.260 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5541000000000001e-05 2024-09-21T15:00:02.260 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.260 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.260 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:02.260 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165713+0000", 2024-09-21T15:00:02.261 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7865000000000002e-05 2024-09-21T15:00:02.261 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:02.261 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:02.261 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:02.261 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.261 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.261 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:83751 3.b 3:d3640016:::rbd_data.11368296f791.0000000000000006:40 [sparse-read 1179648~0 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-21T15:00:02.261 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:51.156888+0000", 2024-09-21T15:00:02.261 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 11.027315787999999, 2024-09-21T15:00:02.261 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0090039539999999998, 2024-09-21T15:00:02.261 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:02.261 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:02.261 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:02.261 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:02.261 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:02.261 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 83751 2024-09-21T15:00:02.262 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.262 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:02.262 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.262 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:02.262 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156888+0000", 2024-09-21T15:00:02.262 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:02.262 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.262 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.262 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:02.262 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156888+0000", 2024-09-21T15:00:02.262 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:02.262 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.262 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.262 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:02.263 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156887+0000", 2024-09-21T15:00:02.263 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:02.263 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.263 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.263 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:02.263 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156891+0000", 2024-09-21T15:00:02.263 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.518e-06 2024-09-21T15:00:02.263 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.263 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.263 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:02.263 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156900+0000", 2024-09-21T15:00:02.263 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1759999999999999e-06 2024-09-21T15:00:02.263 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.263 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.263 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:02.263 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156908+0000", 2024-09-21T15:00:02.264 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8439999999999998e-06 2024-09-21T15:00:02.264 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.264 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.264 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:02.264 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165738+0000", 2024-09-21T15:00:02.264 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0088305689999999999 2024-09-21T15:00:02.264 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.264 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.264 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:02.264 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165804+0000", 2024-09-21T15:00:02.264 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5535000000000002e-05 2024-09-21T15:00:02.264 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.264 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.264 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:02.264 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165892+0000", 2024-09-21T15:00:02.265 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7860000000000002e-05 2024-09-21T15:00:02.265 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:02.265 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:02.265 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:02.265 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.265 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.265 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:83752 3.b 3:d3640016:::rbd_data.11368296f791.0000000000000006:40 [sparse-read 2621440~0 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-21T15:00:02.265 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:51.156915+0000", 2024-09-21T15:00:02.265 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 11.027288241000001, 2024-09-21T15:00:02.265 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0090768180000000004, 2024-09-21T15:00:02.265 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:02.265 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:02.265 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:02.265 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:02.265 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:02.266 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 83752 2024-09-21T15:00:02.266 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.266 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:02.266 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.266 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:02.266 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156915+0000", 2024-09-21T15:00:02.266 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:02.266 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.266 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.266 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:02.266 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156915+0000", 2024-09-21T15:00:02.266 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:02.266 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.267 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.267 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:02.267 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156915+0000", 2024-09-21T15:00:02.267 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:02.267 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.267 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.267 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:02.267 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156919+0000", 2024-09-21T15:00:02.267 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2679999999999996e-06 2024-09-21T15:00:02.267 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.267 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.267 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:02.267 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156929+0000", 2024-09-21T15:00:02.268 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.031e-05 2024-09-21T15:00:02.268 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.268 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.268 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:02.268 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156937+0000", 2024-09-21T15:00:02.268 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8820000000000001e-06 2024-09-21T15:00:02.268 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.268 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.268 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:02.268 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165909+0000", 2024-09-21T15:00:02.268 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0089722989999999996 2024-09-21T15:00:02.268 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.268 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.269 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:02.269 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165948+0000", 2024-09-21T15:00:02.269 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8114999999999997e-05 2024-09-21T15:00:02.269 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.269 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.269 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:02.269 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165992+0000", 2024-09-21T15:00:02.269 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4503999999999999e-05 2024-09-21T15:00:02.269 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:02.269 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:02.269 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:02.269 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.269 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.269 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:83755 3.b 3:d4eb1ee5:::rbd_data.11368296f791.0000000000000000:40 [sparse-read 1310720~0 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-21T15:00:02.270 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:51.156944+0000", 2024-09-21T15:00:02.270 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 11.027259167, 2024-09-21T15:00:02.270 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.009159146, 2024-09-21T15:00:02.270 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:02.270 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:02.270 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:02.270 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:02.270 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:02.270 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 83755 2024-09-21T15:00:02.270 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.270 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:02.270 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.270 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:02.270 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156944+0000", 2024-09-21T15:00:02.270 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:02.271 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.271 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.271 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:02.271 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156944+0000", 2024-09-21T15:00:02.271 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:02.271 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.271 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.271 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:02.271 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156944+0000", 2024-09-21T15:00:02.271 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:02.271 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.271 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.271 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:02.271 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156948+0000", 2024-09-21T15:00:02.271 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5279999999999997e-06 2024-09-21T15:00:02.272 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.272 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.272 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:02.272 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156959+0000", 2024-09-21T15:00:02.272 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0215e-05 2024-09-21T15:00:02.272 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.272 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.272 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:02.272 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156967+0000", 2024-09-21T15:00:02.272 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6119999999999999e-06 2024-09-21T15:00:02.272 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.272 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.272 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:02.272 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.166007+0000", 2024-09-21T15:00:02.273 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0090395260000000009 2024-09-21T15:00:02.273 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.273 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.273 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:02.273 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.166053+0000", 2024-09-21T15:00:02.273 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6281e-05 2024-09-21T15:00:02.273 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.273 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.273 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:02.273 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.166103+0000", 2024-09-21T15:00:02.273 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0510000000000003e-05 2024-09-21T15:00:02.273 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:02.273 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:02.273 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:02.273 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.274 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.274 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:83763 3.b 3:d3640016:::rbd_data.11368296f791.0000000000000006:40 [sparse-read 524288~81920 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-21T15:00:02.274 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:51.157975+0000", 2024-09-21T15:00:02.274 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 11.026228058999999, 2024-09-21T15:00:02.274 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.008327348, 2024-09-21T15:00:02.274 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:02.274 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:02.274 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:02.274 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:02.274 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:02.274 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 83763 2024-09-21T15:00:02.274 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.274 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:02.274 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.274 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:02.275 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.157975+0000", 2024-09-21T15:00:02.275 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:02.275 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.275 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.275 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:02.275 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.157975+0000", 2024-09-21T15:00:02.275 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:02.275 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.275 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.275 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:02.275 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.157970+0000", 2024-09-21T15:00:02.275 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999943 2024-09-21T15:00:02.275 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.275 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.276 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:02.276 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.157985+0000", 2024-09-21T15:00:02.276 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4807000000000001e-05 2024-09-21T15:00:02.276 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.276 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.276 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:02.276 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.158012+0000", 2024-09-21T15:00:02.276 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7600999999999999e-05 2024-09-21T15:00:02.276 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.276 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.276 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:02.276 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.158019+0000", 2024-09-21T15:00:02.276 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9319999999999999e-06 2024-09-21T15:00:02.276 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.276 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.277 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:02.277 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.166118+0000", 2024-09-21T15:00:02.277 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0080992489999999993 2024-09-21T15:00:02.277 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.277 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.277 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:02.277 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.166156+0000", 2024-09-21T15:00:02.277 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7639999999999999e-05 2024-09-21T15:00:02.277 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.277 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.277 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:02.277 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.166303+0000", 2024-09-21T15:00:02.277 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000146799 2024-09-21T15:00:02.277 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:02.277 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:02.278 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:02.278 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.278 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.278 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:83764 3.b 3:d3640016:::rbd_data.11368296f791.0000000000000006:40 [sparse-read 1966080~0 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-21T15:00:02.278 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:51.158026+0000", 2024-09-21T15:00:02.278 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 11.026177626000001, 2024-09-21T15:00:02.278 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0083785839999999997, 2024-09-21T15:00:02.278 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:02.278 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:02.278 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:02.278 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:02.278 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:02.278 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 83764 2024-09-21T15:00:02.278 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.278 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:02.279 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.279 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:02.279 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.158026+0000", 2024-09-21T15:00:02.279 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:02.279 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.279 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.279 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:02.279 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.158026+0000", 2024-09-21T15:00:02.279 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:02.279 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.279 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.279 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:02.279 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.158025+0000", 2024-09-21T15:00:02.279 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:02.280 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.280 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.280 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:02.280 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.158053+0000", 2024-09-21T15:00:02.280 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7594e-05 2024-09-21T15:00:02.280 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.280 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.280 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:02.280 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.158061+0000", 2024-09-21T15:00:02.280 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2069999999999995e-06 2024-09-21T15:00:02.280 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.280 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.280 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:02.280 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.158067+0000", 2024-09-21T15:00:02.280 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3799999999999999e-06 2024-09-21T15:00:02.281 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.281 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.281 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:02.281 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.166321+0000", 2024-09-21T15:00:02.281 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0082539870000000008 2024-09-21T15:00:02.281 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.281 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.281 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:02.281 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.166360+0000", 2024-09-21T15:00:02.281 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8597999999999999e-05 2024-09-21T15:00:02.281 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:02.281 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:02.281 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:02.281 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.166404+0000", 2024-09-21T15:00:02.281 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4378000000000001e-05 2024-09-21T15:00:02.282 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:02.282 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:02.282 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:02.282 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:02.282 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:02.282 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:02.283 INFO:teuthology.orchestra.run.smithi032.stdout:2821 trunc from 0x3232015 to 0xe6c8f3c 2024-09-21T15:00:02.283 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T15:00:02.284 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T15:00:02.285 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:02.285 INFO:teuthology.orchestra.run.smithi032.stdout:2823 read 0x4f12552 thru 0x4f1da5d (0xb50c bytes) 2024-09-21T15:00:02.285 INFO:teuthology.orchestra.run.smithi032.stdout:2825 read 0x9c07998 thru 0x9c167b1 (0xee1a bytes) 2024-09-21T15:00:02.285 INFO:teuthology.orchestra.run.smithi032.stdout:2826 read 0x23f8d0f thru 0x23fe083 (0x5375 bytes) 2024-09-21T15:00:02.285 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:02.285 INFO:teuthology.orchestra.run.smithi032.stdout:2828 punch from 0xd0e91f to 0xd189f0, (0xa0d1 bytes) 2024-09-21T15:00:02.285 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:02.285 INFO:teuthology.orchestra.run.smithi032.stdout:2830 write 0x77945fb thru 0x779ebb7 (0xa5bd bytes) 2024-09-21T15:00:02.285 INFO:teuthology.orchestra.run.smithi032.stdout:2831 read 0x38fc1f7 thru 0x39076e1 (0xb4eb bytes) 2024-09-21T15:00:02.285 INFO:teuthology.orchestra.run.smithi032.stdout:2833 writesame 0x985ddc7 thru 0x986b7a1 data_size 0x1(0xd9db bytes) 2024-09-21T15:00:02.285 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T15:00:02.286 INFO:teuthology.orchestra.run.smithi032.stdout:2835 read 0xdbe96c8 thru 0xdbed007 (0x3940 bytes) 2024-09-21T15:00:02.286 INFO:teuthology.orchestra.run.smithi032.stdout:2836 punch from 0xeb685e to 0xec064b, (0x9ded bytes) 2024-09-21T15:00:02.286 INFO:teuthology.orchestra.run.smithi032.stdout:2837 writesame 0xea6ff0 thru 0xea74f6 data_size 0x1(0x507 bytes) 2024-09-21T15:00:02.286 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-21T15:00:02.287 INFO:teuthology.orchestra.run.smithi032.stdout:2838 writesame 0x75c91d1 thru 0x75d70c5 data_size 0x1(0xdef5 bytes) 2024-09-21T15:00:02.287 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T15:00:02.287 INFO:teuthology.orchestra.run.smithi032.stdout:2839 write 0x4f00716 thru 0x4f04fee (0x48d9 bytes) 2024-09-21T15:00:02.287 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:02.287 INFO:teuthology.orchestra.run.smithi032.stdout:2841 writesame 0x832ec02 thru 0x8335f93 data_size 0x3(0x7392 bytes) 2024-09-21T15:00:02.289 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:00:02.289+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:00:02.290 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T15:00:02.297 INFO:teuthology.orchestra.run.smithi032.stdout:2843 trunc from 0xe6c8f3c to 0x574e31d 2024-09-21T15:00:02.318 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T15:00:02.320 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:00:02.320 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:00:02.321 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:00:02.321 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:00:02.321 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:00:02.321 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:00:02.321 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:00:02.321 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:00:02.327 INFO:teuthology.orchestra.run.smithi032.stdout:2844 trunc from 0x574e31d to 0x90e4b0a 2024-09-21T15:00:02.336 INFO:teuthology.orchestra.run.smithi032.stdout:2845 write 0x509e835 thru 0x50a5513 (0x6cdf bytes) 2024-09-21T15:00:02.343 INFO:teuthology.orchestra.run.smithi032.stdout:2846 punch from 0xb1fd8b to 0xb23009, (0x327e bytes) 2024-09-21T15:00:02.343 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:02.358 INFO:teuthology.orchestra.run.smithi032.stdout:2848 writesame 0xb8c47f4 thru 0xb8c67bc data_size 0x1(0x1fc9 bytes) 2024-09-21T15:00:02.358 INFO:teuthology.orchestra.run.smithi032.stdout:2849 read 0x2c7471c thru 0x2c7c12e (0x7a13 bytes) 2024-09-21T15:00:02.370 INFO:teuthology.orchestra.run.smithi032.stdout:2852 trunc from 0xb8c67bd to 0x53ba02e 2024-09-21T15:00:02.386 INFO:teuthology.orchestra.run.smithi032.stdout:2853 punch from 0x29154d8 to 0x2924395, (0xeebd bytes) 2024-09-21T15:00:02.390 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:00:02.386+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:00:02.396 INFO:teuthology.orchestra.run.smithi032.stdout:2855 trunc from 0x53ba02e to 0x112244b 2024-09-21T15:00:02.420 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:02.420 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:02.421 INFO:teuthology.orchestra.run.smithi032.stderr:nodeep-scrub is unset 2024-09-21T15:00:02.425 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:02.425 INFO:teuthology.orchestra.run.smithi032.stdout:2858 read 0x683bc5 thru 0x685f19 (0x2355 bytes) 2024-09-21T15:00:02.432 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T15:00:02.439 INFO:teuthology.orchestra.run.smithi032.stdout:2859 writesame 0x3fdc00b thru 0x3fe28d8 data_size 0x5(0x68ce bytes) 2024-09-21T15:00:02.439 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:02.465 INFO:teuthology.orchestra.run.smithi032.stdout:2861 write 0xde807a3 thru 0xde8376b (0x2fc9 bytes) 2024-09-21T15:00:02.475 INFO:teuthology.orchestra.run.smithi032.stdout:2862 read 0x4c7478c thru 0x4c7cb76 (0x83eb bytes) 2024-09-21T15:00:02.475 INFO:teuthology.orchestra.run.smithi032.stdout:2863 write 0x33c9318 thru 0x33d2ba0 (0x9889 bytes) 2024-09-21T15:00:02.482 INFO:teuthology.orchestra.run.smithi032.stdout:2865 read 0xeefd13 thru 0xefccf8 (0xcfe6 bytes) 2024-09-21T15:00:02.491 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:00:02.486+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:00:02.503 INFO:teuthology.orchestra.run.smithi032.stdout:2866 trunc from 0xde8376c to 0x4548c8d 2024-09-21T15:00:02.524 INFO:teuthology.orchestra.run.smithi032.stdout:2868 write 0x356ede thru 0x3659b5 (0xead8 bytes) 2024-09-21T15:00:02.529 INFO:teuthology.orchestra.run.smithi032.stdout:2869 punch from 0x1b451a2 to 0x1b47613, (0x2471 bytes) 2024-09-21T15:00:02.550 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:02.550 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:02.554 INFO:teuthology.orchestra.run.smithi032.stdout:2870 write 0xcb9e76f thru 0xcbac85c (0xe0ee bytes) 2024-09-21T15:00:02.560 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T15:00:02.567 INFO:teuthology.orchestra.run.smithi032.stdout:2871 write 0x15bbad8 thru 0x15c96f6 (0xdc1f bytes) 2024-09-21T15:00:02.572 INFO:teuthology.orchestra.run.smithi032.stdout:2872 punch from 0x975b496 to 0x97613fb, (0x5f65 bytes) 2024-09-21T15:00:02.572 INFO:teuthology.orchestra.run.smithi032.stdout:2873 read 0x7ce6d0b thru 0x7cea25d (0x3553 bytes) 2024-09-21T15:00:02.572 INFO:teuthology.orchestra.run.smithi032.stdout:2874 read 0xdbf378 thru 0xdcaab1 (0xb73a bytes) 2024-09-21T15:00:02.572 INFO:teuthology.orchestra.run.smithi032.stdout:2875 read 0x25e0594 thru 0x25ee213 (0xdc80 bytes) 2024-09-21T15:00:02.584 INFO:teuthology.orchestra.run.smithi032.stdout:2876 trunc from 0xcbac85d to 0x6c5df6f 2024-09-21T15:00:02.592 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:00:02.589+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:00:02.604 INFO:teuthology.orchestra.run.smithi032.stdout:2877 read 0x3930484 thru 0x393938d (0x8f0a bytes) 2024-09-21T15:00:02.618 INFO:teuthology.orchestra.run.smithi032.stdout:2879 writesame 0x8e40c7c thru 0x8e49901 data_size 0x1(0x8c86 bytes) 2024-09-21T15:00:02.618 INFO:teuthology.orchestra.run.smithi032.stdout:2880 writesame 0x48cb6df thru 0x48d6c72 data_size 0x2(0xb594 bytes) 2024-09-21T15:00:02.653 INFO:teuthology.orchestra.run.smithi032.stdout:2881 trunc from 0x8e49902 to 0x14f2cf9 2024-09-21T15:00:02.682 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:02.682 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:02.689 INFO:teuthology.orchestra.run.smithi032.stdout:2884 trunc from 0x14f2cf9 to 0x76fe980 2024-09-21T15:00:02.693 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:00:02.693+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:00:02.695 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T15:00:02.698 INFO:teuthology.orchestra.run.smithi032.stdout:2886 read 0x2ea6110 thru 0x2ea67c0 (0x6b1 bytes) 2024-09-21T15:00:02.698 INFO:teuthology.orchestra.run.smithi032.stdout:2889 read 0x5ae4514 thru 0x5af40e0 (0xfbcd bytes) 2024-09-21T15:00:02.705 INFO:teuthology.orchestra.run.smithi032.stdout:2890 trunc from 0x76fe980 to 0xbe1c44d 2024-09-21T15:00:02.716 INFO:teuthology.orchestra.run.smithi032.stdout:2891 read 0xb36fe80 thru 0xb37ac23 (0xada4 bytes) 2024-09-21T15:00:02.716 INFO:teuthology.orchestra.run.smithi032.stdout:2892 read 0x282c5f3 thru 0x282ef63 (0x2971 bytes) 2024-09-21T15:00:02.717 INFO:teuthology.orchestra.run.smithi032.stdout:2893 write 0x3942149 thru 0x3948f59 (0x6e11 bytes) 2024-09-21T15:00:02.738 INFO:teuthology.orchestra.run.smithi032.stdout:2894 writesame 0xcb8ca30 thru 0xcb91c90 data_size 0x1(0x5261 bytes) 2024-09-21T15:00:02.739 INFO:teuthology.orchestra.run.smithi032.stdout:2895 writesame 0x68626a8 thru 0x68713e4 data_size 0x1(0xed3d bytes) 2024-09-21T15:00:02.764 INFO:teuthology.orchestra.run.smithi032.stdout:2896 write 0xbe8291e thru 0xbe880f7 (0x57da bytes) 2024-09-21T15:00:02.772 INFO:teuthology.orchestra.run.smithi032.stdout:2897 writesame 0x33da6d5 thru 0x33dcd58 data_size 0x44(0x2684 bytes) 2024-09-21T15:00:02.788 INFO:teuthology.orchestra.run.smithi032.stdout:2898 write 0xe2f675a thru 0xe30380f (0xd0b6 bytes) 2024-09-21T15:00:02.793 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:00:02.793+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:00:02.805 INFO:teuthology.orchestra.run.smithi032.stdout:2905 trunc from 0xe303810 to 0xbf658ee 2024-09-21T15:00:02.819 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:02.819 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:02.829 INFO:teuthology.orchestra.run.smithi032.stdout:2907 trunc from 0xbf658ee to 0xd6ba5dd 2024-09-21T15:00:02.830 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T15:00:02.839 INFO:teuthology.orchestra.run.smithi032.stdout:2908 writesame 0xc029209 thru 0xc02ef02 data_size 0x3(0x5cfa bytes) 2024-09-21T15:00:02.850 INFO:teuthology.orchestra.run.smithi032.stdout:2911 write 0xce61202 thru 0xce69ec2 (0x8cc1 bytes) 2024-09-21T15:00:02.871 INFO:teuthology.orchestra.run.smithi032.stdout:2912 trunc from 0xd6ba5dd to 0x57b2f88 2024-09-21T15:00:02.894 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:00:02.893+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:00:02.897 INFO:teuthology.orchestra.run.smithi032.stdout:2913 trunc from 0x57b2f88 to 0x3b81980 2024-09-21T15:00:02.909 INFO:teuthology.orchestra.run.smithi032.stdout:2914 read 0x3b3561d thru 0x3b3687e (0x1262 bytes) 2024-09-21T15:00:02.923 INFO:teuthology.orchestra.run.smithi032.stdout:2915 writesame 0x5f12803 thru 0x5f12a07 data_size 0x1(0x205 bytes) 2024-09-21T15:00:02.926 INFO:teuthology.orchestra.run.smithi032.stdout:2917 write 0x1c09250 thru 0x1c13059 (0x9e0a bytes) 2024-09-21T15:00:02.931 INFO:teuthology.orchestra.run.smithi032.stdout:2919 punch from 0x332db28 to 0x33374cf, (0x99a7 bytes) 2024-09-21T15:00:02.931 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:02.949 INFO:teuthology.orchestra.run.smithi032.stdout:2921 write 0xa391166 thru 0xa39b690 (0xa52b bytes) 2024-09-21T15:00:02.959 INFO:teuthology.orchestra.run.smithi032.stdout:2922 punch from 0x2cf48cf to 0x2d00d7e, (0xc4af bytes) 2024-09-21T15:00:02.959 INFO:teuthology.orchestra.run.smithi032.stdout:2923 punch from 0x7226f57 to 0x7233fc6, (0xd06f bytes) 2024-09-21T15:00:02.959 INFO:teuthology.orchestra.run.smithi032.stdout:2924 read 0x167fc1d thru 0x168bd6c (0xc150 bytes) 2024-09-21T15:00:02.960 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:02.961 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:02.961 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:02.975 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T15:00:02.975 INFO:teuthology.orchestra.run.smithi032.stdout:2927 trunc from 0xa39b691 to 0xd75411 2024-09-21T15:00:03.016 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:00:03.017+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:00:03.022 INFO:teuthology.orchestra.run.smithi032.stdout:2928 write 0x6b8588d thru 0x6b8d7fd (0x7f71 bytes) 2024-09-21T15:00:03.033 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:03.033 INFO:teuthology.orchestra.run.smithi032.stdout:2930 read 0x1448bdc thru 0x144f682 (0x6aa7 bytes) 2024-09-21T15:00:03.034 INFO:teuthology.orchestra.run.smithi032.stdout:2931 read 0x1530318 thru 0x153073a (0x423 bytes) 2024-09-21T15:00:03.060 INFO:teuthology.orchestra.run.smithi032.stdout:2932 write 0xee680fc thru 0xee6b27f (0x3184 bytes) 2024-09-21T15:00:03.067 INFO:teuthology.orchestra.run.smithi032.stdout:2933 read 0x2c701d5 thru 0x2c7dd4d (0xdb79 bytes) 2024-09-21T15:00:03.069 INFO:teuthology.orchestra.run.smithi032.stdout:2935 punch from 0x1802539 to 0x1802ecf, (0x996 bytes) 2024-09-21T15:00:03.083 INFO:teuthology.orchestra.run.smithi032.stdout:2936 trunc from 0xee6b280 to 0x5aa7110 2024-09-21T15:00:03.097 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:00:03.090+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:00:03.123 INFO:teuthology.orchestra.run.smithi032.stdout:2937 write 0x978a82a thru 0x979395a (0x9131 bytes) 2024-09-21T15:00:03.133 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:03.140 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:03.141 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:03.151 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T15:00:03.151 INFO:teuthology.orchestra.run.smithi032.stdout:2939 write 0xcf4b736 thru 0xcf54888 (0x9153 bytes) 2024-09-21T15:00:03.161 INFO:teuthology.orchestra.run.smithi032.stdout:2940 read 0x36e6e24 thru 0x36f5453 (0xe630 bytes) 2024-09-21T15:00:03.174 INFO:teuthology.orchestra.run.smithi032.stdout:2941 trunc from 0xcf54889 to 0x516f4ef 2024-09-21T15:00:03.197 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:00:03.190+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:00:03.205 INFO:teuthology.orchestra.run.smithi032.stdout:2942 write 0x65c6481 thru 0x65cb3bc (0x4f3c bytes) 2024-09-21T15:00:03.215 INFO:teuthology.orchestra.run.smithi032.stdout:2943 read 0x10d74a4 thru 0x10e5638 (0xe195 bytes) 2024-09-21T15:00:03.216 INFO:teuthology.orchestra.run.smithi032.stdout:2944 read 0x1b48b9a thru 0x1b527f8 (0x9c5f bytes) 2024-09-21T15:00:03.226 INFO:teuthology.orchestra.run.smithi032.stdout:2947 trunc from 0x65cb3bd to 0xab6422e 2024-09-21T15:00:03.238 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:03.253 INFO:teuthology.orchestra.run.smithi032.stdout:2949 write 0xd4ef8fd thru 0xd4f1610 (0x1d14 bytes) 2024-09-21T15:00:03.261 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:03.261 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:03.274 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T15:00:03.281 INFO:teuthology.orchestra.run.smithi032.stdout:2951 trunc from 0xd4f1611 to 0x2ebbfdb 2024-09-21T15:00:03.298 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:00:03.297+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:00:03.306 INFO:teuthology.orchestra.run.smithi032.stdout:2952 read 0x66822d thru 0x66fa93 (0x7867 bytes) 2024-09-21T15:00:03.307 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:03.307 INFO:teuthology.orchestra.run.smithi032.stdout:2954 write 0x248a5eb thru 0x248b848 (0x125e bytes) 2024-09-21T15:00:03.309 INFO:teuthology.orchestra.run.smithi032.stdout:2957 punch from 0x18b2c57 to 0x18be6cb, (0xba74 bytes) 2024-09-21T15:00:03.320 INFO:teuthology.orchestra.run.smithi032.stdout:2958 trunc from 0x2ebbfdb to 0x887c7f0 2024-09-21T15:00:03.330 INFO:teuthology.orchestra.run.smithi032.stdout:2960 read 0x1fefd6e thru 0x1ffc1d5 (0xc468 bytes) 2024-09-21T15:00:03.331 INFO:teuthology.orchestra.run.smithi032.stdout:2961 read 0x5da1ba4 thru 0x5da6cf6 (0x5153 bytes) 2024-09-21T15:00:03.350 INFO:teuthology.orchestra.run.smithi032.stdout:2963 write 0xd804600 thru 0xd80f6d4 (0xb0d5 bytes) 2024-09-21T15:00:03.359 INFO:teuthology.orchestra.run.smithi032.stdout:2964 punch from 0x26f97b9 to 0x2706aa7, (0xd2ee bytes) 2024-09-21T15:00:03.371 INFO:teuthology.orchestra.run.smithi032.stdout:2965 trunc from 0xd80f6d5 to 0x6388c12 2024-09-21T15:00:03.390 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:03.391 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:03.399 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:00:03.394+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:00:03.403 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T15:00:03.406 INFO:teuthology.orchestra.run.smithi032.stdout:2966 trunc from 0x6388c12 to 0xe4e363f 2024-09-21T15:00:03.417 INFO:teuthology.orchestra.run.smithi032.stdout:2969 read 0x6df483 thru 0x6e0bab (0x1729 bytes) 2024-09-21T15:00:03.418 INFO:teuthology.orchestra.run.smithi032.stdout:2970 read 0x1c100ab thru 0x1c1df1f (0xde75 bytes) 2024-09-21T15:00:03.421 INFO:teuthology.orchestra.run.smithi032.stdout:2972 punch from 0x5cf6e81 to 0x5cf999d, (0x2b1c bytes) 2024-09-21T15:00:03.421 INFO:teuthology.orchestra.run.smithi032.stdout:2973 write 0x7696dfb thru 0x769d664 (0x686a bytes) 2024-09-21T15:00:03.430 INFO:teuthology.orchestra.run.smithi032.stdout:2974 write 0xa6e1ba thru 0xa7330d (0x5154 bytes) 2024-09-21T15:00:03.433 INFO:teuthology.orchestra.run.smithi032.stdout:2977 writesame 0xc46e6a2 thru 0xc46e99a data_size 0x1(0x2f9 bytes) 2024-09-21T15:00:03.439 INFO:teuthology.orchestra.run.smithi032.stdout:2978 read 0xb323a56 thru 0xb32edb8 (0xb363 bytes) 2024-09-21T15:00:03.440 INFO:teuthology.orchestra.run.smithi032.stdout:2980 write 0xa3bb145 thru 0xa3c5725 (0xa5e1 bytes) 2024-09-21T15:00:03.450 INFO:teuthology.orchestra.run.smithi032.stdout:2981 read 0x80105ee thru 0x80121cb (0x1bde bytes) 2024-09-21T15:00:03.450 INFO:teuthology.orchestra.run.smithi032.stdout:2983 read 0x2011cd5 thru 0x2019adf (0x7e0b bytes) 2024-09-21T15:00:03.451 INFO:teuthology.orchestra.run.smithi032.stdout:2985 writesame 0xa9102c9 thru 0xa91944c data_size 0x86(0x9184 bytes) 2024-09-21T15:00:03.460 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:03.460 INFO:teuthology.orchestra.run.smithi032.stdout:2988 read 0x6caf524 thru 0x6cb3ab1 (0x458e bytes) 2024-09-21T15:00:03.460 INFO:teuthology.orchestra.run.smithi032.stdout:2989 writesame 0xbf18275 thru 0xbf22764 data_size 0x1d(0xa4f0 bytes) 2024-09-21T15:00:03.469 INFO:teuthology.orchestra.run.smithi032.stdout:2990 write 0x3d84ea thru 0x3dc0b3 (0x3bca bytes) 2024-09-21T15:00:03.473 INFO:teuthology.orchestra.run.smithi032.stdout:2991 read 0x673403b thru 0x6742650 (0xe616 bytes) 2024-09-21T15:00:03.473 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:03.473 INFO:teuthology.orchestra.run.smithi032.stdout:2993 read 0x56ded22 thru 0x56ec686 (0xd965 bytes) 2024-09-21T15:00:03.473 INFO:teuthology.orchestra.run.smithi032.stdout:2995 read 0x4490197 thru 0x449d454 (0xd2be bytes) 2024-09-21T15:00:03.473 INFO:teuthology.orchestra.run.smithi032.stdout:2998 write 0xa1b312d thru 0xa1b73a4 (0x4278 bytes) 2024-09-21T15:00:03.476 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:03.476 INFO:teuthology.orchestra.run.smithi032.stdout:3000 read 0x17d36c4 thru 0x17d9513 (0x5e50 bytes) 2024-09-21T15:00:03.477 INFO:teuthology.orchestra.run.smithi032.stdout:3002 read 0x9bd6818 thru 0x9be31c8 (0xc9b1 bytes) 2024-09-21T15:00:03.477 INFO:teuthology.orchestra.run.smithi032.stdout:3004 writesame 0x347fa1e thru 0x348dc22 data_size 0x9(0xe205 bytes) 2024-09-21T15:00:03.500 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:00:03.500+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:00:03.507 INFO:teuthology.orchestra.run.smithi032.stdout:3005 trunc from 0xe4e363f to 0x526d447 2024-09-21T15:00:03.526 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:03.526 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:03.532 INFO:teuthology.orchestra.run.smithi032.stdout:3006 punch from 0x50c1161 to 0x50cc636, (0xb4d5 bytes) 2024-09-21T15:00:03.532 INFO:teuthology.orchestra.run.smithi032.stdout:3007 punch from 0x29f84ee to 0x29fd671, (0x5183 bytes) 2024-09-21T15:00:03.532 INFO:teuthology.orchestra.run.smithi032.stdout:3008 read 0x3311a24 thru 0x331242a (0xa07 bytes) 2024-09-21T15:00:03.537 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T15:00:03.544 INFO:teuthology.orchestra.run.smithi032.stdout:3009 writesame 0x6193e1d thru 0x619d7e5 data_size 0xb(0x99c9 bytes) 2024-09-21T15:00:03.559 INFO:teuthology.orchestra.run.smithi032.stdout:3010 write 0x712712b thru 0x712e2d0 (0x71a6 bytes) 2024-09-21T15:00:03.569 INFO:teuthology.orchestra.run.smithi032.stdout:3011 read 0x5980863 thru 0x59886f9 (0x7e97 bytes) 2024-09-21T15:00:03.570 INFO:teuthology.orchestra.run.smithi032.stdout:3012 write 0x4427848 thru 0x442ab5c (0x3315 bytes) 2024-09-21T15:00:03.573 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:03.573 INFO:teuthology.orchestra.run.smithi032.stdout:3015 punch from 0x19a9446 to 0x19b3200, (0x9dba bytes) 2024-09-21T15:00:03.573 INFO:teuthology.orchestra.run.smithi032.stdout:3017 writesame 0x54bd276 thru 0x54c3914 data_size 0x3f(0x669f bytes) 2024-09-21T15:00:03.601 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:00:03.594+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:00:03.603 INFO:teuthology.orchestra.run.smithi032.stdout:3019 write 0xd37067b thru 0xd37ec33 (0xe5b9 bytes) 2024-09-21T15:00:03.615 INFO:teuthology.orchestra.run.smithi032.stdout:3020 writesame 0x21f986d thru 0x2204290 data_size 0x2(0xaa24 bytes) 2024-09-21T15:00:03.625 INFO:teuthology.orchestra.run.smithi032.stdout:3021 write 0x8ccefb thru 0x8cdf57 (0x105d bytes) 2024-09-21T15:00:03.630 INFO:teuthology.orchestra.run.smithi032.stdout:3022 trunc from 0xd37ec34 to 0xc04d022 2024-09-21T15:00:03.650 INFO:teuthology.orchestra.run.smithi032.stdout:3024 read 0x63a2085 thru 0x63ae86e (0xc7ea bytes) 2024-09-21T15:00:03.651 INFO:teuthology.orchestra.run.smithi032.stdout:3027 read 0xafa8ed1 thru 0xafac70d (0x383d bytes) 2024-09-21T15:00:03.651 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:03.651 INFO:teuthology.orchestra.run.smithi032.stdout:3029 read 0x642ecde thru 0x6438024 (0x9347 bytes) 2024-09-21T15:00:03.652 INFO:teuthology.orchestra.run.smithi032.stdout:3031 writesame 0x5d56aee thru 0x5d5d576 data_size 0x1(0x6a89 bytes) 2024-09-21T15:00:03.655 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:03.655 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:03.667 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T15:00:03.670 INFO:teuthology.orchestra.run.smithi032.stdout:3032 writesame 0x414137f thru 0x41416ee data_size 0x14(0x370 bytes) 2024-09-21T15:00:03.674 INFO:teuthology.orchestra.run.smithi032.stdout:3033 read 0x9f9f83f thru 0x9f9ff94 (0x756 bytes) 2024-09-21T15:00:03.674 INFO:teuthology.orchestra.run.smithi032.stdout:3034 write 0xad79f1e thru 0xad81f52 (0x8035 bytes) 2024-09-21T15:00:03.683 INFO:teuthology.orchestra.run.smithi032.stdout:3035 write 0x7ab920 thru 0x7afeee (0x45cf bytes) 2024-09-21T15:00:03.702 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:00:03.698+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:00:03.705 INFO:teuthology.orchestra.run.smithi032.stdout:3036 write 0xeb3e4ee thru 0xeb3ff3a (0x1a4d bytes) 2024-09-21T15:00:03.714 INFO:teuthology.orchestra.run.smithi032.stdout:3039 punch from 0x2bb65dd to 0x2bb6be1, (0x604 bytes) 2024-09-21T15:00:03.714 INFO:teuthology.orchestra.run.smithi032.stdout:3042 read 0x9a2b341 thru 0x9a37425 (0xc0e5 bytes) 2024-09-21T15:00:03.714 INFO:teuthology.orchestra.run.smithi032.stdout:3043 write 0x542c5f6 thru 0x543973a (0xd145 bytes) 2024-09-21T15:00:03.717 INFO:teuthology.orchestra.run.smithi032.stdout:3046 read 0x4787a56 thru 0x4797092 (0xf63d bytes) 2024-09-21T15:00:03.718 INFO:teuthology.orchestra.run.smithi032.stdout:3047 read 0x2cd178a thru 0x2cdc020 (0xa897 bytes) 2024-09-21T15:00:03.719 INFO:teuthology.orchestra.run.smithi032.stdout:3048 write 0xd10a6eb thru 0xd11a085 (0xf99b bytes) 2024-09-21T15:00:03.732 INFO:teuthology.orchestra.run.smithi032.stdout:3051 write 0xeaef98a thru 0xeaf4af7 (0x516e bytes) 2024-09-21T15:00:03.742 INFO:teuthology.orchestra.run.smithi032.stdout:3053 write 0x5c49c91 thru 0x5c517e1 (0x7b51 bytes) 2024-09-21T15:00:03.748 INFO:teuthology.orchestra.run.smithi032.stdout:3054 write 0x94a8721 thru 0x94acd2e (0x460e bytes) 2024-09-21T15:00:03.756 INFO:teuthology.orchestra.run.smithi032.stdout:3055 write 0x76eede6 thru 0x76fbd99 (0xcfb4 bytes) 2024-09-21T15:00:03.768 INFO:teuthology.orchestra.run.smithi032.stdout:3056 write 0xc5173b7 thru 0xc519530 (0x217a bytes) 2024-09-21T15:00:03.775 INFO:teuthology.orchestra.run.smithi032.stdout:3057 read 0x72cefc7 thru 0x72d8114 (0x914e bytes) 2024-09-21T15:00:03.775 INFO:teuthology.orchestra.run.smithi032.stdout:3058 write 0xae14f74 thru 0xae17597 (0x2624 bytes) 2024-09-21T15:00:03.778 INFO:teuthology.orchestra.run.smithi032.stdout:3059 punch from 0x3899688 to 0x389b997, (0x230f bytes) 2024-09-21T15:00:03.778 INFO:teuthology.orchestra.run.smithi032.stdout:3061 writesame 0x77fd803 thru 0x7809cc5 data_size 0x1(0xc4c3 bytes) 2024-09-21T15:00:03.779 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:03.779 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:03.791 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T15:00:03.798 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:03.798 INFO:teuthology.orchestra.run.smithi032.stdout:3063 read 0x57dd3b5 thru 0x57de98b (0x15d7 bytes) 2024-09-21T15:00:03.798 INFO:teuthology.orchestra.run.smithi032.stdout:3064 write 0x949ad26 thru 0x94a7b07 (0xcde2 bytes) 2024-09-21T15:00:03.802 INFO:teuthology.orchestra.run.smithi032.stdout:3065 write 0x820701f thru 0x8212f2e (0xbf10 bytes) 2024-09-21T15:00:03.802 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:00:03.798+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:00:03.809 INFO:teuthology.orchestra.run.smithi032.stdout:3068 read 0xba9e4cf thru 0xba9f0fc (0xc2e bytes) 2024-09-21T15:00:03.809 INFO:teuthology.orchestra.run.smithi032.stdout:3069 writesame 0x55fe207 thru 0x5607cb6 data_size 0x24(0x9ab0 bytes) 2024-09-21T15:00:03.812 INFO:teuthology.orchestra.run.smithi032.stdout:3070 write 0x117a981 thru 0x1180b29 (0x61a9 bytes) 2024-09-21T15:00:03.815 INFO:teuthology.orchestra.run.smithi032.stdout:3071 punch from 0x85f8819 to 0x8606aba, (0xe2a1 bytes) 2024-09-21T15:00:03.815 INFO:teuthology.orchestra.run.smithi032.stdout:3072 punch from 0x9f81bdb to 0x9f8dbdf, (0xc004 bytes) 2024-09-21T15:00:03.815 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:03.815 INFO:teuthology.orchestra.run.smithi032.stdout:3075 punch from 0x2e2fa05 to 0x2e319d4, (0x1fcf bytes) 2024-09-21T15:00:03.815 INFO:teuthology.orchestra.run.smithi032.stdout:3076 write 0x214cbce thru 0x214d21f (0x652 bytes) 2024-09-21T15:00:03.817 INFO:teuthology.orchestra.run.smithi032.stdout:3077 punch from 0x9b4fc3b to 0x9b58a73, (0x8e38 bytes) 2024-09-21T15:00:03.817 INFO:teuthology.orchestra.run.smithi032.stdout:3078 writesame 0x6850e3b thru 0x6851d8a data_size 0x23(0xf50 bytes) 2024-09-21T15:00:03.819 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:03.819 INFO:teuthology.orchestra.run.smithi032.stdout:3080 read 0x8205d35 thru 0x82159f9 (0xfcc5 bytes) 2024-09-21T15:00:03.820 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:03.820 INFO:teuthology.orchestra.run.smithi032.stdout:3083 write 0x37f4c19 thru 0x3802667 (0xda4f bytes) 2024-09-21T15:00:03.823 INFO:teuthology.orchestra.run.smithi032.stdout:3084 read 0x6593ef1 thru 0x6595692 (0x17a2 bytes) 2024-09-21T15:00:03.824 INFO:teuthology.orchestra.run.smithi032.stdout:3085 punch from 0x1cf613e to 0x1d04c61, (0xeb23 bytes) 2024-09-21T15:00:03.824 INFO:teuthology.orchestra.run.smithi032.stdout:3086 punch from 0x2a7de09 to 0x2a8dc46, (0xfe3d bytes) 2024-09-21T15:00:03.830 INFO:teuthology.orchestra.run.smithi032.stdout:3087 trunc from 0xeb3ff3b to 0xb306357 2024-09-21T15:00:03.853 INFO:teuthology.orchestra.run.smithi032.stdout:3089 punch from 0x39cda6d to 0x39d25bc, (0x4b4f bytes) 2024-09-21T15:00:03.853 INFO:teuthology.orchestra.run.smithi032.stdout:3090 read 0x73dddea thru 0x73ea1ef (0xc406 bytes) 2024-09-21T15:00:03.854 INFO:teuthology.orchestra.run.smithi032.stdout:3092 writesame 0x27ebf02 thru 0x27fa687 data_size 0x5(0xe786 bytes) 2024-09-21T15:00:03.865 INFO:teuthology.orchestra.run.smithi032.stdout:3094 read 0x40fa7ec thru 0x40fd3a3 (0x2bb8 bytes) 2024-09-21T15:00:03.865 INFO:teuthology.orchestra.run.smithi032.stdout:3095 writesame 0x92a1920 thru 0x92a322c data_size 0xb(0x190d bytes) 2024-09-21T15:00:03.868 INFO:teuthology.orchestra.run.smithi032.stdout:3096 punch from 0x89dada to 0x8a3d95, (0x62bb bytes) 2024-09-21T15:00:03.868 INFO:teuthology.orchestra.run.smithi032.stdout:3099 write 0x67be799 thru 0x67c0e7b (0x26e3 bytes) 2024-09-21T15:00:03.874 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:03.889 INFO:teuthology.orchestra.run.smithi032.stdout:3101 write 0xea7d8cf thru 0xea83ef7 (0x6629 bytes) 2024-09-21T15:00:03.898 INFO:teuthology.orchestra.run.smithi032.stdout:3102 write 0xd653c1f thru 0xd65ce75 (0x9257 bytes) 2024-09-21T15:00:03.903 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:00:03.898+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:00:03.910 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:03.910 INFO:teuthology.orchestra.run.smithi032.stdout:3104 read 0x4a21a84 thru 0x4a2743e (0x59bb bytes) 2024-09-21T15:00:03.910 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:03.910 INFO:teuthology.orchestra.run.smithi032.stdout:3107 writesame 0x773c8fb thru 0x774b97c data_size 0x5e(0xf082 bytes) 2024-09-21T15:00:03.914 INFO:teuthology.orchestra.run.smithi032.stdout:3110 punch from 0x40b8f52 to 0x40c5628, (0xc6d6 bytes) 2024-09-21T15:00:03.914 INFO:teuthology.orchestra.run.smithi032.stdout:3113 clone 24 order 24 su 524288 sc 11 2024-09-21T15:00:03.914 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:03.915 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:03.925 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T15:00:04.004 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:00:04.004+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:00:04.041 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:04.041 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:04.051 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T15:00:04.105 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:00:04.104+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:00:04.159 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:04.159 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:04.169 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T15:00:04.206 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:00:04.204+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:00:04.275 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:04.275 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:04.284 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T15:00:04.307 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:00:04.302+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:00:04.335 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T15:00:04.396 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:04.396 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:04.406 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T15:00:04.407 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:00:04.404+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:00:04.434 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T15:00:04.498 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T15:00:04.508 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:00:04.502+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:00:04.517 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T15:00:04.535 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T15:00:04.556 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T15:00:04.562 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:04.563 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:04.573 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T15:00:04.576 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-21T15:00:04.598 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T15:00:04.618 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:00:04.614+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:00:04.621 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T15:00:04.650 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T15:00:04.652 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:00:04.652 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:00:04.652 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:00:04.652 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:00:04.652 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:00:04.653 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:00:04.653 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:00:04.653 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:00:04.683 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:04.683 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T15:00:04.683 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T15:00:04.683 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T15:00:04.683 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.683 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45834 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:04.683 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159133+0000", 2024-09-21T15:00:04.683 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 54.521142922000003, 2024-09-21T15:00:04.683 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068236884999999997, 2024-09-21T15:00:04.683 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:04.683 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:04.684 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:04.684 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:04.684 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:04.684 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45834 2024-09-21T15:00:04.684 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.684 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:04.684 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.684 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:04.684 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159133+0000", 2024-09-21T15:00:04.684 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:04.684 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.684 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.684 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:04.684 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159133+0000", 2024-09-21T15:00:04.684 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:04.684 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.685 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.685 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:04.685 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159133+0000", 2024-09-21T15:00:04.685 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:04.685 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.685 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.685 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:04.685 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159135+0000", 2024-09-21T15:00:04.685 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4500000000000001e-06 2024-09-21T15:00:04.685 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.685 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.685 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:04.685 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159138+0000", 2024-09-21T15:00:04.685 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2770000000000001e-06 2024-09-21T15:00:04.685 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.686 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.686 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:04.686 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159141+0000", 2024-09-21T15:00:04.686 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9210000000000001e-06 2024-09-21T15:00:04.686 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.686 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.686 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:04.686 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227361+0000", 2024-09-21T15:00:04.686 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068220154000000005 2024-09-21T15:00:04.686 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.686 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.686 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:04.686 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227370+0000", 2024-09-21T15:00:04.686 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3239999999999992e-06 2024-09-21T15:00:04.686 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:04.687 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:04.687 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:04.687 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.687 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.687 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45839 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:04.687 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159313+0000", 2024-09-21T15:00:04.687 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 54.520962955999998, 2024-09-21T15:00:04.687 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068238699, 2024-09-21T15:00:04.687 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:04.687 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:04.687 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:04.687 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:04.687 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:04.688 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45839 2024-09-21T15:00:04.688 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.688 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:04.688 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.688 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:04.688 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159313+0000", 2024-09-21T15:00:04.688 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:04.688 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.688 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.688 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:04.688 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159313+0000", 2024-09-21T15:00:04.688 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:04.688 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.688 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.688 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:04.689 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159313+0000", 2024-09-21T15:00:04.689 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:04.689 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.689 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.689 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:04.689 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159316+0000", 2024-09-21T15:00:04.689 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.033e-06 2024-09-21T15:00:04.689 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.689 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.689 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:04.689 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159322+0000", 2024-09-21T15:00:04.689 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1820000000000003e-06 2024-09-21T15:00:04.689 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.689 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.689 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:04.689 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159327+0000", 2024-09-21T15:00:04.690 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.062e-06 2024-09-21T15:00:04.690 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.690 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.690 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:04.690 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227544+0000", 2024-09-21T15:00:04.690 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068217142999999994 2024-09-21T15:00:04.690 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.690 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.690 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:04.690 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227552+0000", 2024-09-21T15:00:04.690 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6260000000000003e-06 2024-09-21T15:00:04.690 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:04.690 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:04.690 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:04.691 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.691 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.691 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45840 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:04.691 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159332+0000", 2024-09-21T15:00:04.691 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 54.520944659000001, 2024-09-21T15:00:04.691 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068262046000000007, 2024-09-21T15:00:04.691 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:04.691 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:04.691 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:04.691 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:04.691 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:04.691 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45840 2024-09-21T15:00:04.691 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.691 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:04.691 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.692 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:04.692 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159332+0000", 2024-09-21T15:00:04.692 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:04.692 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.692 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.692 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:04.692 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159332+0000", 2024-09-21T15:00:04.692 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:04.692 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.692 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.692 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:04.692 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159331+0000", 2024-09-21T15:00:04.692 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:04.692 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.692 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.693 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:04.693 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159351+0000", 2024-09-21T15:00:04.693 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0064999999999998e-05 2024-09-21T15:00:04.693 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.693 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.693 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:04.693 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159367+0000", 2024-09-21T15:00:04.693 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5308e-05 2024-09-21T15:00:04.693 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.693 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.693 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:04.693 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159373+0000", 2024-09-21T15:00:04.693 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1840000000000004e-06 2024-09-21T15:00:04.693 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.693 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.693 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:04.694 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227584+0000", 2024-09-21T15:00:04.694 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068211401000000005 2024-09-21T15:00:04.694 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.694 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.694 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:04.694 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227594+0000", 2024-09-21T15:00:04.694 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4120000000000007e-06 2024-09-21T15:00:04.694 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:04.694 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:04.694 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:04.694 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.694 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.694 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45841 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:04.694 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159379+0000", 2024-09-21T15:00:04.694 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 54.520897503, 2024-09-21T15:00:04.694 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068249039999999997, 2024-09-21T15:00:04.695 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:04.695 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:04.695 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:04.695 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:04.695 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:04.695 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45841 2024-09-21T15:00:04.695 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.695 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:04.695 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.695 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:04.695 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159379+0000", 2024-09-21T15:00:04.695 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:04.695 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.695 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.695 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:04.695 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159379+0000", 2024-09-21T15:00:04.696 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:04.696 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.696 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.696 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:04.696 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159378+0000", 2024-09-21T15:00:04.696 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:04.696 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.696 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.696 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:04.696 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159382+0000", 2024-09-21T15:00:04.696 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.641e-06 2024-09-21T15:00:04.696 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.696 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.696 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:04.697 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159421+0000", 2024-09-21T15:00:04.697 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9439e-05 2024-09-21T15:00:04.697 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.697 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.697 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:04.697 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159430+0000", 2024-09-21T15:00:04.697 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8429999999999999e-06 2024-09-21T15:00:04.697 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.697 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.697 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:04.697 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227617+0000", 2024-09-21T15:00:04.697 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068187084999999995 2024-09-21T15:00:04.697 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.697 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.698 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:04.698 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227628+0000", 2024-09-21T15:00:04.698 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0507e-05 2024-09-21T15:00:04.698 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:04.698 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:04.698 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:04.698 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.698 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.698 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45844 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:04.698 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159499+0000", 2024-09-21T15:00:04.698 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 54.520777117000002, 2024-09-21T15:00:04.698 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068255415, 2024-09-21T15:00:04.698 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:04.698 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:04.698 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:04.699 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:04.699 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:04.699 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45844 2024-09-21T15:00:04.699 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.699 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:04.699 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.699 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:04.699 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159499+0000", 2024-09-21T15:00:04.699 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:04.699 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.699 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.699 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:04.699 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159499+0000", 2024-09-21T15:00:04.699 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:04.699 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.699 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.700 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:04.700 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159499+0000", 2024-09-21T15:00:04.700 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:04.700 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.700 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.700 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:04.700 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159503+0000", 2024-09-21T15:00:04.700 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0999999999999997e-06 2024-09-21T15:00:04.700 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.700 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.700 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:04.700 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159512+0000", 2024-09-21T15:00:04.700 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5850000000000006e-06 2024-09-21T15:00:04.700 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.700 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.701 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:04.701 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159518+0000", 2024-09-21T15:00:04.701 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0909999999999998e-06 2024-09-21T15:00:04.701 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.701 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.701 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:04.701 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227743+0000", 2024-09-21T15:00:04.701 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068224536000000002 2024-09-21T15:00:04.701 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.701 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.701 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:04.701 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227755+0000", 2024-09-21T15:00:04.701 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1659e-05 2024-09-21T15:00:04.701 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:04.701 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:04.702 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:04.702 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.702 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.702 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45845 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:04.702 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159524+0000", 2024-09-21T15:00:04.702 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 54.520752756999997, 2024-09-21T15:00:04.702 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068267194000000003, 2024-09-21T15:00:04.702 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:04.702 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:04.702 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:04.702 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:04.702 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:04.702 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45845 2024-09-21T15:00:04.702 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.702 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:04.703 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.703 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:04.703 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159524+0000", 2024-09-21T15:00:04.703 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:04.703 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.703 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.703 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:04.703 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159524+0000", 2024-09-21T15:00:04.703 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:04.703 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.703 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.703 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:04.703 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159523+0000", 2024-09-21T15:00:04.703 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:04.703 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.703 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.704 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:04.704 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159526+0000", 2024-09-21T15:00:04.704 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1980000000000001e-06 2024-09-21T15:00:04.704 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.704 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.704 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:04.704 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159533+0000", 2024-09-21T15:00:04.704 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.348e-06 2024-09-21T15:00:04.704 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.704 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.704 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:04.704 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159546+0000", 2024-09-21T15:00:04.704 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3464e-05 2024-09-21T15:00:04.704 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.705 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.705 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:04.705 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227783+0000", 2024-09-21T15:00:04.705 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068236926000000003 2024-09-21T15:00:04.705 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.705 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.705 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:04.705 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227791+0000", 2024-09-21T15:00:04.705 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6669999999999996e-06 2024-09-21T15:00:04.705 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:04.705 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:04.705 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:04.705 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.705 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.705 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45846 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:04.706 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159584+0000", 2024-09-21T15:00:04.706 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 54.520692115000003, 2024-09-21T15:00:04.706 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068254767999999993, 2024-09-21T15:00:04.706 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:04.706 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:04.706 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:04.706 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:04.706 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:04.706 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45846 2024-09-21T15:00:04.706 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.706 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:04.706 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.706 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:04.706 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159584+0000", 2024-09-21T15:00:04.706 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:04.707 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.707 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.707 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:04.707 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159584+0000", 2024-09-21T15:00:04.707 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:04.707 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.707 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.707 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:04.707 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159583+0000", 2024-09-21T15:00:04.707 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:04.707 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.707 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.707 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:04.707 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159588+0000", 2024-09-21T15:00:04.707 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1330000000000001e-06 2024-09-21T15:00:04.708 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.708 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.708 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:04.708 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159596+0000", 2024-09-21T15:00:04.708 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5469999999999997e-06 2024-09-21T15:00:04.708 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.708 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.708 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:04.708 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159609+0000", 2024-09-21T15:00:04.708 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2785999999999999e-05 2024-09-21T15:00:04.708 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.708 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.708 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:04.708 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227830+0000", 2024-09-21T15:00:04.708 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068220879999999998 2024-09-21T15:00:04.708 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.709 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.709 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:04.709 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227839+0000", 2024-09-21T15:00:04.709 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1239999999999994e-06 2024-09-21T15:00:04.709 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:04.709 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:04.709 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:04.709 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.709 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.709 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45847 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:04.709 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159614+0000", 2024-09-21T15:00:04.709 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 54.520662039000001, 2024-09-21T15:00:04.709 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068260639999999997, 2024-09-21T15:00:04.709 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:04.709 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:04.709 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:04.710 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:04.710 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:04.710 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45847 2024-09-21T15:00:04.710 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.710 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:04.710 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.710 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:04.710 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159614+0000", 2024-09-21T15:00:04.710 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:04.710 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.710 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.710 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:04.710 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159614+0000", 2024-09-21T15:00:04.710 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:04.710 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.710 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.711 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:04.711 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159614+0000", 2024-09-21T15:00:04.711 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:04.711 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.711 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.711 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:04.711 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159617+0000", 2024-09-21T15:00:04.711 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2909999999999999e-06 2024-09-21T15:00:04.711 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.711 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.711 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:04.711 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159623+0000", 2024-09-21T15:00:04.711 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.511e-06 2024-09-21T15:00:04.711 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.711 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.712 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:04.712 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159631+0000", 2024-09-21T15:00:04.712 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.571e-06 2024-09-21T15:00:04.712 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.712 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.712 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:04.712 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227866+0000", 2024-09-21T15:00:04.712 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068235136000000002 2024-09-21T15:00:04.712 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.712 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.712 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:04.712 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227875+0000", 2024-09-21T15:00:04.712 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7199999999999995e-06 2024-09-21T15:00:04.712 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:04.712 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:04.712 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:04.712 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.713 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.713 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45848 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:04.713 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159636+0000", 2024-09-21T15:00:04.713 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 54.520640258999997, 2024-09-21T15:00:04.713 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068267283999999998, 2024-09-21T15:00:04.713 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:04.713 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:04.713 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:04.713 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:04.713 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:04.713 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45848 2024-09-21T15:00:04.713 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.713 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:04.713 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.713 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:04.713 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159636+0000", 2024-09-21T15:00:04.714 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:04.714 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.714 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.714 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:04.714 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159636+0000", 2024-09-21T15:00:04.714 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:04.714 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.714 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.714 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:04.714 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159636+0000", 2024-09-21T15:00:04.714 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:04.714 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.714 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.714 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:04.714 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159638+0000", 2024-09-21T15:00:04.715 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6560000000000001e-06 2024-09-21T15:00:04.715 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.715 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.715 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:04.715 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159644+0000", 2024-09-21T15:00:04.715 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0839999999999997e-06 2024-09-21T15:00:04.715 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.715 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.715 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:04.715 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159649+0000", 2024-09-21T15:00:04.715 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7500000000000003e-06 2024-09-21T15:00:04.715 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.715 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.715 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:04.715 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227892+0000", 2024-09-21T15:00:04.715 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068243315999999998 2024-09-21T15:00:04.716 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.716 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.716 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:04.716 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227903+0000", 2024-09-21T15:00:04.716 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0835999999999999e-05 2024-09-21T15:00:04.716 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:04.716 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:04.716 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:04.716 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.716 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.716 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45849 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:04.716 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159667+0000", 2024-09-21T15:00:04.716 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 54.520609444999998, 2024-09-21T15:00:04.716 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068291051000000005, 2024-09-21T15:00:04.717 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:04.717 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:04.717 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:04.717 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:04.717 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:04.717 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45849 2024-09-21T15:00:04.717 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.717 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:04.717 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.717 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:04.717 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159667+0000", 2024-09-21T15:00:04.717 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:04.717 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.717 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.717 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:04.717 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159667+0000", 2024-09-21T15:00:04.717 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:04.718 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.718 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.718 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:04.718 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159666+0000", 2024-09-21T15:00:04.718 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:04.718 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.718 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.718 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:04.718 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159670+0000", 2024-09-21T15:00:04.718 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.427e-06 2024-09-21T15:00:04.718 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.718 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.718 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:04.718 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159678+0000", 2024-09-21T15:00:04.718 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9869999999999999e-06 2024-09-21T15:00:04.718 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.719 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.719 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:04.719 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159688+0000", 2024-09-21T15:00:04.719 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0631999999999999e-05 2024-09-21T15:00:04.719 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.719 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.719 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:04.719 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227950+0000", 2024-09-21T15:00:04.719 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068261798999999998 2024-09-21T15:00:04.719 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.719 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.719 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:04.719 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227958+0000", 2024-09-21T15:00:04.719 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7510000000000008e-06 2024-09-21T15:00:04.719 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:04.720 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:04.720 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:04.720 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.720 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.720 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45850 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:04.720 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159695+0000", 2024-09-21T15:00:04.720 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 54.520581733999997, 2024-09-21T15:00:04.720 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068302999000000003, 2024-09-21T15:00:04.720 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:04.720 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:04.720 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:04.720 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:04.720 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:04.720 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45850 2024-09-21T15:00:04.720 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.720 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:04.721 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.721 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:04.721 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159695+0000", 2024-09-21T15:00:04.721 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:04.721 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.721 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.721 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:04.721 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159695+0000", 2024-09-21T15:00:04.721 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:04.721 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.721 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.721 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:04.721 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159694+0000", 2024-09-21T15:00:04.721 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:04.721 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.721 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.722 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:04.722 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159697+0000", 2024-09-21T15:00:04.722 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.258e-06 2024-09-21T15:00:04.722 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.722 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.722 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:04.722 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159704+0000", 2024-09-21T15:00:04.722 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7660000000000001e-06 2024-09-21T15:00:04.722 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.722 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.722 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:04.722 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159713+0000", 2024-09-21T15:00:04.722 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1779999999999992e-06 2024-09-21T15:00:04.722 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.723 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.723 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:04.723 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227989+0000", 2024-09-21T15:00:04.723 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068276015999999995 2024-09-21T15:00:04.723 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.723 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.723 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:04.723 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227998+0000", 2024-09-21T15:00:04.723 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2050000000000002e-06 2024-09-21T15:00:04.723 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:04.723 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:04.723 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:04.723 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.724 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.724 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45851 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:04.724 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159719+0000", 2024-09-21T15:00:04.724 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 54.520557601, 2024-09-21T15:00:04.724 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068320327, 2024-09-21T15:00:04.724 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:04.724 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:04.724 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:04.724 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:04.724 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:04.724 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45851 2024-09-21T15:00:04.724 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.724 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:04.724 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.724 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:04.725 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159719+0000", 2024-09-21T15:00:04.725 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:04.725 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.725 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.725 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:04.725 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159719+0000", 2024-09-21T15:00:04.725 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:04.725 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.725 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.725 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:04.725 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159718+0000", 2024-09-21T15:00:04.725 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:04.725 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.726 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.726 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:04.726 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159721+0000", 2024-09-21T15:00:04.726 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2169999999999999e-06 2024-09-21T15:00:04.726 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.726 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.726 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:04.726 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159730+0000", 2024-09-21T15:00:04.726 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5720000000000003e-06 2024-09-21T15:00:04.726 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.726 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.726 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:04.726 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159735+0000", 2024-09-21T15:00:04.726 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1100000000000002e-06 2024-09-21T15:00:04.727 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.727 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.727 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:04.727 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228029+0000", 2024-09-21T15:00:04.727 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068293624999999997 2024-09-21T15:00:04.727 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.727 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.727 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:04.727 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228039+0000", 2024-09-21T15:00:04.727 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0254e-05 2024-09-21T15:00:04.727 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:04.727 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:04.727 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:04.727 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.727 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.728 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45852 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:04.728 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159740+0000", 2024-09-21T15:00:04.728 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 54.520536432999997, 2024-09-21T15:00:04.728 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068340260999999999, 2024-09-21T15:00:04.728 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:04.728 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:04.728 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:04.728 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:04.728 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:04.728 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45852 2024-09-21T15:00:04.728 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.728 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:04.728 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.728 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:04.729 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159740+0000", 2024-09-21T15:00:04.729 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:04.729 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.729 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.729 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:04.729 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159740+0000", 2024-09-21T15:00:04.729 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:04.729 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.729 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.729 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:04.729 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159739+0000", 2024-09-21T15:00:04.729 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:04.729 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.729 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.729 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:04.729 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159757+0000", 2024-09-21T15:00:04.730 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7314999999999999e-05 2024-09-21T15:00:04.730 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.730 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.730 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:04.730 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159769+0000", 2024-09-21T15:00:04.730 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2376e-05 2024-09-21T15:00:04.730 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.730 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.730 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:04.730 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159775+0000", 2024-09-21T15:00:04.730 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2179999999999996e-06 2024-09-21T15:00:04.730 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.730 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.730 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:04.731 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228071+0000", 2024-09-21T15:00:04.731 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068295344999999993 2024-09-21T15:00:04.731 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.731 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.731 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:04.731 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228080+0000", 2024-09-21T15:00:04.731 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3959999999999995e-06 2024-09-21T15:00:04.731 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:04.731 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:04.731 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:04.731 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.731 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.731 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45853 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:04.731 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159781+0000", 2024-09-21T15:00:04.731 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 54.520495169, 2024-09-21T15:00:04.732 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068337093000000002, 2024-09-21T15:00:04.732 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:04.732 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:04.732 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:04.733 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:04.733 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:04.733 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45853 2024-09-21T15:00:04.733 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.733 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:04.734 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.734 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:04.734 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159781+0000", 2024-09-21T15:00:04.734 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:04.734 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.734 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.734 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:04.734 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159781+0000", 2024-09-21T15:00:04.734 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:04.734 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.734 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.734 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:04.734 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159781+0000", 2024-09-21T15:00:04.734 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:04.735 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.735 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.735 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:04.735 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159784+0000", 2024-09-21T15:00:04.735 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4759999999999998e-06 2024-09-21T15:00:04.735 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.735 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.735 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:04.735 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159792+0000", 2024-09-21T15:00:04.735 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0569999999999993e-06 2024-09-21T15:00:04.735 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.735 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.735 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:04.735 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159797+0000", 2024-09-21T15:00:04.735 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9640000000000002e-06 2024-09-21T15:00:04.736 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.736 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.736 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:04.736 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228111+0000", 2024-09-21T15:00:04.736 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068313737999999999 2024-09-21T15:00:04.736 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.736 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.736 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:04.736 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228118+0000", 2024-09-21T15:00:04.736 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.43e-06 2024-09-21T15:00:04.736 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:04.736 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:04.736 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:04.736 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.736 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.737 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45854 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:04.737 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159802+0000", 2024-09-21T15:00:04.737 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 54.520474518999997, 2024-09-21T15:00:04.737 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068358938999999994, 2024-09-21T15:00:04.737 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:04.737 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:04.737 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:04.737 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:04.737 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:04.737 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45854 2024-09-21T15:00:04.737 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.737 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:04.737 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.737 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:04.737 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159802+0000", 2024-09-21T15:00:04.738 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:04.738 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.738 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.738 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:04.738 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159802+0000", 2024-09-21T15:00:04.738 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:04.738 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.738 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.738 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:04.738 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159801+0000", 2024-09-21T15:00:04.738 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:04.738 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.738 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.738 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:04.739 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159804+0000", 2024-09-21T15:00:04.739 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0970000000000002e-06 2024-09-21T15:00:04.739 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.739 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.739 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:04.739 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159823+0000", 2024-09-21T15:00:04.739 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8895000000000001e-05 2024-09-21T15:00:04.739 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.739 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.739 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:04.739 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159837+0000", 2024-09-21T15:00:04.739 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3247999999999999e-05 2024-09-21T15:00:04.739 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.740 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.740 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:04.740 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228151+0000", 2024-09-21T15:00:04.740 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068314642999999994 2024-09-21T15:00:04.740 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.740 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.740 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:04.740 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228161+0000", 2024-09-21T15:00:04.740 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4250000000000004e-06 2024-09-21T15:00:04.740 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:04.740 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:04.740 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:04.740 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.740 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.740 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45983 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:04.741 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159848+0000", 2024-09-21T15:00:04.741 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 54.520428260000003, 2024-09-21T15:00:04.741 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068333801, 2024-09-21T15:00:04.741 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:04.741 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:04.741 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:04.741 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:04.741 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:04.741 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45983 2024-09-21T15:00:04.741 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.741 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:04.741 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.741 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:04.741 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159848+0000", 2024-09-21T15:00:04.741 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:04.742 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.742 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.742 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:04.742 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159848+0000", 2024-09-21T15:00:04.742 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:04.742 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.742 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.742 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:04.742 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159847+0000", 2024-09-21T15:00:04.742 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:04.742 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.742 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.742 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:04.742 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159856+0000", 2024-09-21T15:00:04.743 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4349999999999997e-06 2024-09-21T15:00:04.743 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.743 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.743 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:04.743 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159866+0000", 2024-09-21T15:00:04.743 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0076e-05 2024-09-21T15:00:04.743 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.743 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.743 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:04.743 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159872+0000", 2024-09-21T15:00:04.743 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8309999999999997e-06 2024-09-21T15:00:04.743 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.743 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.743 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:04.743 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228175+0000", 2024-09-21T15:00:04.743 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068302680000000005 2024-09-21T15:00:04.744 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.744 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.744 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:04.744 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228182+0000", 2024-09-21T15:00:04.744 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3150000000000003e-06 2024-09-21T15:00:04.744 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:04.744 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:04.744 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:04.744 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.744 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.744 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45984 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:04.744 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159877+0000", 2024-09-21T15:00:04.744 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 54.520398839999999, 2024-09-21T15:00:04.744 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068328448, 2024-09-21T15:00:04.744 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:04.744 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:04.745 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:04.745 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:04.745 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:04.745 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45984 2024-09-21T15:00:04.745 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.745 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:04.745 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.745 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:04.745 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159877+0000", 2024-09-21T15:00:04.745 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:04.745 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.745 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.745 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:04.745 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159877+0000", 2024-09-21T15:00:04.745 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:04.746 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.746 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.746 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:04.746 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159877+0000", 2024-09-21T15:00:04.746 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:04.746 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.746 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.746 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:04.746 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159880+0000", 2024-09-21T15:00:04.746 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7610000000000001e-06 2024-09-21T15:00:04.746 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.746 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.746 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:04.746 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159888+0000", 2024-09-21T15:00:04.747 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0330000000000003e-06 2024-09-21T15:00:04.747 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.747 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.747 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:04.747 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159893+0000", 2024-09-21T15:00:04.747 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3550000000000004e-06 2024-09-21T15:00:04.747 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.747 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.747 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:04.747 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228199+0000", 2024-09-21T15:00:04.747 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068306438999999997 2024-09-21T15:00:04.747 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.748 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.748 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:04.748 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228206+0000", 2024-09-21T15:00:04.748 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5599999999999999e-06 2024-09-21T15:00:04.748 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:04.748 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:04.748 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:04.748 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.748 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.748 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45985 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:04.748 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159897+0000", 2024-09-21T15:00:04.748 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 54.520379017000003, 2024-09-21T15:00:04.748 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068345664, 2024-09-21T15:00:04.748 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:04.748 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:04.749 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:04.749 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:04.749 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:04.749 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45985 2024-09-21T15:00:04.749 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.749 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:04.749 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.749 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:04.749 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159897+0000", 2024-09-21T15:00:04.749 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:04.749 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.749 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.749 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:04.749 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159897+0000", 2024-09-21T15:00:04.749 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:04.750 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.750 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.750 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:04.750 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159897+0000", 2024-09-21T15:00:04.750 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:04.750 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.750 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.750 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:04.750 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159900+0000", 2024-09-21T15:00:04.750 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.954e-06 2024-09-21T15:00:04.750 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.750 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.750 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:04.750 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159913+0000", 2024-09-21T15:00:04.751 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3106e-05 2024-09-21T15:00:04.751 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.751 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.751 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:04.751 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159918+0000", 2024-09-21T15:00:04.751 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5199999999999997e-06 2024-09-21T15:00:04.751 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.751 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.751 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:04.751 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228236+0000", 2024-09-21T15:00:04.751 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068317040999999995 2024-09-21T15:00:04.751 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.751 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.751 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:04.752 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228243+0000", 2024-09-21T15:00:04.752 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3880000000000002e-06 2024-09-21T15:00:04.752 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:04.752 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:04.752 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:04.752 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.752 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.752 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45986 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:04.752 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159924+0000", 2024-09-21T15:00:04.752 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 54.520352721999998, 2024-09-21T15:00:04.752 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068364362999999997, 2024-09-21T15:00:04.752 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:04.752 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:04.752 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:04.753 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:04.753 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:04.753 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45986 2024-09-21T15:00:04.753 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.753 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:04.753 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.753 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:04.753 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159924+0000", 2024-09-21T15:00:04.753 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:04.753 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.753 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.753 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:04.753 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159924+0000", 2024-09-21T15:00:04.753 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:04.753 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.754 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.754 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:04.754 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159923+0000", 2024-09-21T15:00:04.754 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:04.754 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.754 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.754 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:04.754 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159926+0000", 2024-09-21T15:00:04.754 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0510000000000001e-06 2024-09-21T15:00:04.754 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.754 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.754 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:04.754 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159932+0000", 2024-09-21T15:00:04.754 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4160000000000001e-06 2024-09-21T15:00:04.755 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.755 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.755 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:04.755 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159938+0000", 2024-09-21T15:00:04.755 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.383e-06 2024-09-21T15:00:04.755 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.755 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.755 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:04.755 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228275+0000", 2024-09-21T15:00:04.755 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068337007000000005 2024-09-21T15:00:04.755 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.755 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.755 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:04.755 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228288+0000", 2024-09-21T15:00:04.755 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3052e-05 2024-09-21T15:00:04.756 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:04.756 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:04.756 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:04.756 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.756 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.756 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45987 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:04.756 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159944+0000", 2024-09-21T15:00:04.756 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 54.520332498000002, 2024-09-21T15:00:04.756 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068387232000000006, 2024-09-21T15:00:04.756 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:04.756 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:04.756 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:04.756 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:04.756 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:04.756 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45987 2024-09-21T15:00:04.757 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.757 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:04.757 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.757 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:04.757 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159944+0000", 2024-09-21T15:00:04.757 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:04.757 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.757 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.757 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:04.757 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159944+0000", 2024-09-21T15:00:04.757 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:04.757 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.757 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.757 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:04.758 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159943+0000", 2024-09-21T15:00:04.758 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:04.758 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.758 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.758 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:04.758 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159946+0000", 2024-09-21T15:00:04.758 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6309999999999999e-06 2024-09-21T15:00:04.758 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.758 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.758 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:04.758 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159952+0000", 2024-09-21T15:00:04.758 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6540000000000004e-06 2024-09-21T15:00:04.758 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.758 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.759 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:04.759 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159957+0000", 2024-09-21T15:00:04.759 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9280000000000001e-06 2024-09-21T15:00:04.759 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.759 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.759 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:04.759 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228322+0000", 2024-09-21T15:00:04.759 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068365067000000002 2024-09-21T15:00:04.759 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:04.759 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:04.759 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:04.759 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228331+0000", 2024-09-21T15:00:04.759 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3270000000000007e-06 2024-09-21T15:00:04.759 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:04.759 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:04.760 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:04.760 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:04.760 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:04.760 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:04.761 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T15:00:04.762 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:00:04.760+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:00:04.862 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:00:04.858+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:00:04.874 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 5, 6, 7, 4] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T15:00:04.874 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T15:00:04.874 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-21T15:00:04.874 INFO:tasks.thrashosds.thrasher:Setting osd 6 primary_affinity to 0.834935 2024-09-21T15:00:04.875 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 0.8349353155413833 2024-09-21T15:00:04.876 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:04.876 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:00:04.876 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T15:00:04.876 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:04.886 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T15:00:04.962 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:00:04.960+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:00:05.043 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:05.043 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:00:05.043 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T15:00:05.043 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T15:00:05.043 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:05.053 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T15:00:05.063 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:00:05.060+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:00:05.143 INFO:teuthology.orchestra.run.smithi032.stderr:noscrub is set 2024-09-21T15:00:05.171 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:00:05.164+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:00:05.184 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:05.184 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T15:00:05.184 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T15:00:05.184 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T15:00:05.184 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.184 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:10651 3.12 3:4e9d23af:::rbd_data.1136f5ed5dd6.0000000000000020:head [write 0~81920 [fadvise_sequential+fadvise_nocache],write 90112~8192 [fadvise_sequential+fadvise_nocache],write 122880~122880 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:05.184 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.292476+0000", 2024-09-21T15:00:05.184 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 85.888797959000001, 2024-09-21T15:00:05.185 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012765669, 2024-09-21T15:00:05.185 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:05.185 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:05.185 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:05.185 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:05.185 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:05.185 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 10651 2024-09-21T15:00:05.185 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.185 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:05.185 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.185 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:05.185 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292476+0000", 2024-09-21T15:00:05.185 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:05.185 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.186 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.186 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:05.186 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292476+0000", 2024-09-21T15:00:05.186 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:05.186 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.186 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.186 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:05.186 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292475+0000", 2024-09-21T15:00:05.186 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:00:05.186 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.186 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.186 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:05.186 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292696+0000", 2024-09-21T15:00:05.186 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000220201 2024-09-21T15:00:05.187 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.187 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.187 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:05.187 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292711+0000", 2024-09-21T15:00:05.187 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5803000000000001e-05 2024-09-21T15:00:05.187 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.187 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.187 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:05.187 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292721+0000", 2024-09-21T15:00:05.187 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6160000000000008e-06 2024-09-21T15:00:05.187 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.187 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.187 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:05.187 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292757+0000", 2024-09-21T15:00:05.187 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6211999999999998e-05 2024-09-21T15:00:05.188 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.188 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.188 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:05.188 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292878+0000", 2024-09-21T15:00:05.188 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000120306 2024-09-21T15:00:05.188 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.188 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.188 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 3", 2024-09-21T15:00:05.188 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.293019+0000", 2024-09-21T15:00:05.188 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014186400000000001 2024-09-21T15:00:05.188 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.188 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.188 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:05.188 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.294898+0000", 2024-09-21T15:00:05.189 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001878434 2024-09-21T15:00:05.189 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.189 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.189 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:05.189 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.305162+0000", 2024-09-21T15:00:05.189 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.01026373 2024-09-21T15:00:05.189 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.189 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.189 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:05.189 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.305209+0000", 2024-09-21T15:00:05.189 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7615999999999999e-05 2024-09-21T15:00:05.189 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.189 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.189 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:05.189 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.305242+0000", 2024-09-21T15:00:05.190 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2979999999999999e-05 2024-09-21T15:00:05.190 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:05.190 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:05.190 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:05.190 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.190 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.190 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:11285 3.12 3:4edf73d2:::rbd_data.1136f5ed5dd6.0000000000000039:head [write 90112~32768 [fadvise_sequential+fadvise_nocache],write 139264~24576 [fadvise_sequential+fadvise_nocache],write 180224~24576 [fadvise_sequential+fadvise_nocache],write 221184~40960 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:05.190 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.539470+0000", 2024-09-21T15:00:05.190 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 85.641804207000007, 2024-09-21T15:00:05.190 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0083360099999999996, 2024-09-21T15:00:05.190 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:05.190 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:05.190 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:05.191 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:05.191 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:05.191 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 11285 2024-09-21T15:00:05.191 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.191 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:05.191 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.191 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:05.191 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.539470+0000", 2024-09-21T15:00:05.191 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:05.191 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.191 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.191 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:05.191 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.539470+0000", 2024-09-21T15:00:05.191 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:05.191 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.192 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.192 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:05.192 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.539469+0000", 2024-09-21T15:00:05.192 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:00:05.192 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.192 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.192 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:05.192 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.539652+0000", 2024-09-21T15:00:05.192 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000183084 2024-09-21T15:00:05.192 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.192 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.192 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:05.192 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.539666+0000", 2024-09-21T15:00:05.192 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3947e-05 2024-09-21T15:00:05.193 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.193 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.193 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:05.193 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.539673+0000", 2024-09-21T15:00:05.193 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7309999999999996e-06 2024-09-21T15:00:05.193 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.193 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.193 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:05.193 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.539714+0000", 2024-09-21T15:00:05.193 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1539000000000003e-05 2024-09-21T15:00:05.193 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.193 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.193 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:05.193 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.539832+0000", 2024-09-21T15:00:05.193 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011753500000000001 2024-09-21T15:00:05.194 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.194 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.194 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 3", 2024-09-21T15:00:05.194 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.540005+0000", 2024-09-21T15:00:05.194 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000173031 2024-09-21T15:00:05.194 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.194 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.194 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:05.194 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.541606+0000", 2024-09-21T15:00:05.194 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001600899 2024-09-21T15:00:05.194 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.194 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.194 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:05.194 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.547709+0000", 2024-09-21T15:00:05.195 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0061027110000000002 2024-09-21T15:00:05.195 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.195 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.195 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:05.195 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.547767+0000", 2024-09-21T15:00:05.195 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8248999999999997e-05 2024-09-21T15:00:05.195 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.195 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.195 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:05.195 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.547806+0000", 2024-09-21T15:00:05.195 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9341000000000002e-05 2024-09-21T15:00:05.195 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:05.195 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:05.195 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:05.195 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.196 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.196 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12189 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 40960~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:05.196 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.618667+0000", 2024-09-21T15:00:05.196 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 85.562607756999995, 2024-09-21T15:00:05.196 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0083014549999999993, 2024-09-21T15:00:05.196 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:05.196 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:05.196 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:05.196 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:05.196 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:05.196 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12189 2024-09-21T15:00:05.196 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.196 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:05.196 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.197 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:05.197 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618667+0000", 2024-09-21T15:00:05.197 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:05.197 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.197 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.197 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:05.197 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618667+0000", 2024-09-21T15:00:05.197 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:05.197 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.197 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.197 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:05.197 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618666+0000", 2024-09-21T15:00:05.197 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:05.197 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.198 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.198 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:05.198 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618669+0000", 2024-09-21T15:00:05.198 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.554e-06 2024-09-21T15:00:05.198 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.198 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.198 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:05.198 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618675+0000", 2024-09-21T15:00:05.198 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2570000000000004e-06 2024-09-21T15:00:05.198 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.198 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.198 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:05.198 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618680+0000", 2024-09-21T15:00:05.198 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6539999999999998e-06 2024-09-21T15:00:05.199 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.199 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.199 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:05.199 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.626683+0000", 2024-09-21T15:00:05.199 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0080035880000000007 2024-09-21T15:00:05.199 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.199 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.199 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:05.199 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.626764+0000", 2024-09-21T15:00:05.199 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0672000000000003e-05 2024-09-21T15:00:05.199 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.199 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.199 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:05.199 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.626968+0000", 2024-09-21T15:00:05.200 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00020407499999999999 2024-09-21T15:00:05.200 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:05.200 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:05.200 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:05.200 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.200 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.200 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12193 3.4 3:25437d51:::rbd_data.11363a21d2be.0000000000000025:10 [read 425984~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:05.200 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.618755+0000", 2024-09-21T15:00:05.200 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 85.562519675999994, 2024-09-21T15:00:05.200 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086461379999999994, 2024-09-21T15:00:05.200 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:05.200 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:05.200 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:05.200 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:05.200 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:05.200 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12193 2024-09-21T15:00:05.201 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.201 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:05.201 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.201 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:05.201 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618755+0000", 2024-09-21T15:00:05.201 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:05.201 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.201 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.201 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:05.201 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618755+0000", 2024-09-21T15:00:05.201 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:05.201 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.201 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.201 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:05.201 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618754+0000", 2024-09-21T15:00:05.202 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:05.202 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.202 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.202 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:05.202 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618759+0000", 2024-09-21T15:00:05.202 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4020000000000004e-06 2024-09-21T15:00:05.202 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.202 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.202 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:05.202 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618767+0000", 2024-09-21T15:00:05.202 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7490000000000004e-06 2024-09-21T15:00:05.202 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.202 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.203 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:05.203 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618773+0000", 2024-09-21T15:00:05.203 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0939999999999996e-06 2024-09-21T15:00:05.203 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.203 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.203 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:05.203 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.627115+0000", 2024-09-21T15:00:05.203 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0083416840000000003 2024-09-21T15:00:05.203 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.203 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.203 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:05.203 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.627214+0000", 2024-09-21T15:00:05.203 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9042000000000005e-05 2024-09-21T15:00:05.203 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.203 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.204 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:05.204 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.627401+0000", 2024-09-21T15:00:05.204 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018677200000000001 2024-09-21T15:00:05.204 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:05.204 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:05.204 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:05.204 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.204 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.204 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12212 3.4 3:21b43b45:::rbd_data.11363a21d2be.000000000000002b:10 [read 237568~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:05.204 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.619922+0000", 2024-09-21T15:00:05.204 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 85.561352692, 2024-09-21T15:00:05.204 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.008324372, 2024-09-21T15:00:05.204 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:05.204 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:05.205 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:05.205 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:05.205 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:05.205 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12212 2024-09-21T15:00:05.205 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.205 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:05.205 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.205 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:05.205 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619922+0000", 2024-09-21T15:00:05.205 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:05.205 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.205 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.205 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:05.205 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619922+0000", 2024-09-21T15:00:05.205 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:05.205 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.206 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.206 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:05.206 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619921+0000", 2024-09-21T15:00:05.206 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:05.206 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.206 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.206 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:05.206 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619924+0000", 2024-09-21T15:00:05.206 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.999e-06 2024-09-21T15:00:05.206 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.206 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.206 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:05.206 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619934+0000", 2024-09-21T15:00:05.207 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.8120000000000002e-06 2024-09-21T15:00:05.207 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.207 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.207 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:05.207 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619939+0000", 2024-09-21T15:00:05.207 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7729999999999999e-06 2024-09-21T15:00:05.207 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.207 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.207 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:05.207 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628005+0000", 2024-09-21T15:00:05.207 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0080663330000000002 2024-09-21T15:00:05.207 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.207 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.207 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:05.207 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628073+0000", 2024-09-21T15:00:05.208 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7479999999999998e-05 2024-09-21T15:00:05.208 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.208 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.208 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:05.208 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628246+0000", 2024-09-21T15:00:05.208 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017337400000000001 2024-09-21T15:00:05.208 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:05.208 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:05.208 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:05.208 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.208 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.208 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12213 3.4 3:21b43b45:::rbd_data.11363a21d2be.000000000000002b:10 [read 270336~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:05.208 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.619945+0000", 2024-09-21T15:00:05.208 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 85.561329409999999, 2024-09-21T15:00:05.209 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086195049999999995, 2024-09-21T15:00:05.209 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:05.209 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:05.209 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:05.209 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:05.209 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:05.209 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12213 2024-09-21T15:00:05.209 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.209 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:05.209 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.209 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:05.209 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619945+0000", 2024-09-21T15:00:05.209 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:05.209 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.209 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.210 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:05.210 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619945+0000", 2024-09-21T15:00:05.210 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:05.210 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.210 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.210 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:05.210 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619945+0000", 2024-09-21T15:00:05.210 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:05.210 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.210 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.210 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:05.210 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619947+0000", 2024-09-21T15:00:05.210 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5780000000000001e-06 2024-09-21T15:00:05.211 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.211 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.211 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:05.211 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619957+0000", 2024-09-21T15:00:05.211 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3500000000000003e-06 2024-09-21T15:00:05.211 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.211 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.211 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:05.211 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619964+0000", 2024-09-21T15:00:05.211 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.199e-06 2024-09-21T15:00:05.211 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.211 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.211 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:05.211 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628285+0000", 2024-09-21T15:00:05.211 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0083210939999999994 2024-09-21T15:00:05.212 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.212 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.212 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:05.212 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628361+0000", 2024-09-21T15:00:05.212 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5873999999999999e-05 2024-09-21T15:00:05.212 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.212 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.212 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:05.212 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628565+0000", 2024-09-21T15:00:05.212 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000203799 2024-09-21T15:00:05.212 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:05.212 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:05.212 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:05.212 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.213 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.213 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12214 3.4 3:21b43b45:::rbd_data.11363a21d2be.000000000000002b:10 [read 376832~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:05.213 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.619969+0000", 2024-09-21T15:00:05.213 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 85.561305408999999, 2024-09-21T15:00:05.213 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0090292089999999998, 2024-09-21T15:00:05.213 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:05.213 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:05.213 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:05.213 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:05.213 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:05.213 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12214 2024-09-21T15:00:05.213 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.213 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:05.213 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.213 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:05.214 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619969+0000", 2024-09-21T15:00:05.214 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:05.214 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.214 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.214 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:05.214 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619969+0000", 2024-09-21T15:00:05.214 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:05.214 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.214 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.214 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:05.214 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619969+0000", 2024-09-21T15:00:05.214 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:05.214 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.215 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.215 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:05.215 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619971+0000", 2024-09-21T15:00:05.215 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1830000000000001e-06 2024-09-21T15:00:05.215 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.215 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.215 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:05.215 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619976+0000", 2024-09-21T15:00:05.215 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7129999999999996e-06 2024-09-21T15:00:05.215 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.215 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.215 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:05.215 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619981+0000", 2024-09-21T15:00:05.215 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2460000000000003e-06 2024-09-21T15:00:05.215 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.216 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.216 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:05.216 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628602+0000", 2024-09-21T15:00:05.216 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086209519999999994 2024-09-21T15:00:05.216 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.216 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.216 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:05.216 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628734+0000", 2024-09-21T15:00:05.216 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000132242 2024-09-21T15:00:05.216 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.216 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.216 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:05.216 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628998+0000", 2024-09-21T15:00:05.216 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00026419799999999999 2024-09-21T15:00:05.216 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:05.217 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:05.217 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:05.217 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.217 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.217 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12215 3.4 3:21b43b45:::rbd_data.11363a21d2be.000000000000002b:10 [read 409600~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:05.217 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.619985+0000", 2024-09-21T15:00:05.217 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 85.561289342999999, 2024-09-21T15:00:05.217 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0095457319999999995, 2024-09-21T15:00:05.217 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:05.217 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:05.217 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:05.217 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:05.217 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:05.217 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12215 2024-09-21T15:00:05.217 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.218 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:05.218 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.218 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:05.218 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619985+0000", 2024-09-21T15:00:05.218 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:05.218 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.218 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.218 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:05.218 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619985+0000", 2024-09-21T15:00:05.218 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:05.218 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.218 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.218 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:05.218 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619985+0000", 2024-09-21T15:00:05.219 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:05.219 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.219 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.219 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:05.219 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619987+0000", 2024-09-21T15:00:05.219 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6129999999999998e-06 2024-09-21T15:00:05.219 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.219 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.219 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:05.219 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619993+0000", 2024-09-21T15:00:05.219 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0820000000000004e-06 2024-09-21T15:00:05.219 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.219 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.219 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:05.219 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619998+0000", 2024-09-21T15:00:05.220 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7759999999999997e-06 2024-09-21T15:00:05.220 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.220 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.220 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:05.220 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629022+0000", 2024-09-21T15:00:05.220 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0090237489999999993 2024-09-21T15:00:05.220 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.220 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.220 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:05.220 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629098+0000", 2024-09-21T15:00:05.220 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5735999999999995e-05 2024-09-21T15:00:05.220 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.220 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.220 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:05.220 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629531+0000", 2024-09-21T15:00:05.220 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00043310100000000003 2024-09-21T15:00:05.221 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:05.221 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:05.221 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:05.221 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.221 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.221 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12216 3.4 3:21b43b45:::rbd_data.11363a21d2be.000000000000002b:10 [read 516096~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:05.221 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.620003+0000", 2024-09-21T15:00:05.221 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 85.561271347000002, 2024-09-21T15:00:05.221 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0098850169999999994, 2024-09-21T15:00:05.221 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:05.221 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:05.221 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:05.221 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:05.221 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:05.221 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12216 2024-09-21T15:00:05.221 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.222 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:05.222 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.222 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:05.222 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620003+0000", 2024-09-21T15:00:05.222 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:05.222 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.222 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.222 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:05.222 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620003+0000", 2024-09-21T15:00:05.222 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:05.222 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.222 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.222 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:05.222 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620003+0000", 2024-09-21T15:00:05.222 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:05.223 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.223 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.223 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:05.223 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620006+0000", 2024-09-21T15:00:05.223 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4350000000000001e-06 2024-09-21T15:00:05.223 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.223 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.223 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:05.223 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620013+0000", 2024-09-21T15:00:05.223 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4219999999999997e-06 2024-09-21T15:00:05.223 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.223 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.223 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:05.223 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620017+0000", 2024-09-21T15:00:05.224 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.865e-06 2024-09-21T15:00:05.224 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.224 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.224 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:05.224 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629581+0000", 2024-09-21T15:00:05.224 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0095631989999999997 2024-09-21T15:00:05.224 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.224 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.224 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:05.224 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629691+0000", 2024-09-21T15:00:05.224 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000110586 2024-09-21T15:00:05.224 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.224 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.224 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:05.225 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629888+0000", 2024-09-21T15:00:05.225 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019691199999999999 2024-09-21T15:00:05.225 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:05.225 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:05.225 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:05.225 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.225 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.225 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12222 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 49152~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:05.225 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.620027+0000", 2024-09-21T15:00:05.225 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 85.561247000999998, 2024-09-21T15:00:05.225 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010228131999999999, 2024-09-21T15:00:05.225 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:05.225 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:05.225 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:05.225 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:05.225 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:05.226 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12222 2024-09-21T15:00:05.226 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.226 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:05.226 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.226 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:05.226 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620027+0000", 2024-09-21T15:00:05.226 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:05.226 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.226 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.226 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:05.226 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620027+0000", 2024-09-21T15:00:05.226 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:05.226 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.226 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.227 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:05.227 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620027+0000", 2024-09-21T15:00:05.227 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:05.227 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.227 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.227 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:05.227 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620030+0000", 2024-09-21T15:00:05.227 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0400000000000001e-06 2024-09-21T15:00:05.227 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.227 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.227 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:05.227 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620037+0000", 2024-09-21T15:00:05.227 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8220000000000001e-06 2024-09-21T15:00:05.227 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.227 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.228 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:05.228 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620042+0000", 2024-09-21T15:00:05.228 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.843e-06 2024-09-21T15:00:05.228 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.228 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.228 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:05.228 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629922+0000", 2024-09-21T15:00:05.228 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0098805100000000003 2024-09-21T15:00:05.228 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.228 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.228 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:05.228 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.630020+0000", 2024-09-21T15:00:05.228 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7596999999999994e-05 2024-09-21T15:00:05.228 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.228 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.229 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:05.229 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.630256+0000", 2024-09-21T15:00:05.229 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00023573299999999999 2024-09-21T15:00:05.229 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:05.229 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:05.229 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:05.229 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.229 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.229 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12253 3.4 3:21b43b45:::rbd_data.11363a21d2be.000000000000002b:10 [read 417792~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:05.229 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.624578+0000", 2024-09-21T15:00:05.229 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 85.556696633000001, 2024-09-21T15:00:05.229 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0085112079999999993, 2024-09-21T15:00:05.229 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:05.229 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:05.229 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:05.230 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:05.230 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:05.230 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12253 2024-09-21T15:00:05.230 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.230 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:05.230 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.230 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:05.230 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624578+0000", 2024-09-21T15:00:05.230 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:05.230 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.230 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.230 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:05.230 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624578+0000", 2024-09-21T15:00:05.230 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:05.231 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.231 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.231 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:05.231 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624578+0000", 2024-09-21T15:00:05.231 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:05.231 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.231 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.231 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:05.231 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624579+0000", 2024-09-21T15:00:05.231 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.688e-06 2024-09-21T15:00:05.231 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.231 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.231 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:05.232 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624582+0000", 2024-09-21T15:00:05.232 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2260000000000001e-06 2024-09-21T15:00:05.232 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.232 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.232 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:05.232 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624585+0000", 2024-09-21T15:00:05.232 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4820000000000001e-06 2024-09-21T15:00:05.232 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.232 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.232 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:05.232 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.632580+0000", 2024-09-21T15:00:05.232 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0079948959999999996 2024-09-21T15:00:05.232 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.232 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.232 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:05.233 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.632683+0000", 2024-09-21T15:00:05.233 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010308 2024-09-21T15:00:05.233 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.233 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.233 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:05.233 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.633089+0000", 2024-09-21T15:00:05.233 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00040612599999999998 2024-09-21T15:00:05.233 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:05.233 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:05.233 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:05.233 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.233 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.233 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12260 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 57344~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:05.233 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.624587+0000", 2024-09-21T15:00:05.234 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 85.556687146000002, 2024-09-21T15:00:05.234 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0088282069999999994, 2024-09-21T15:00:05.234 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:05.234 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:05.234 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:05.234 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:05.234 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:05.234 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12260 2024-09-21T15:00:05.234 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.234 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:05.234 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.234 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:05.234 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624587+0000", 2024-09-21T15:00:05.235 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:05.235 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.235 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.235 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:05.235 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624587+0000", 2024-09-21T15:00:05.235 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:05.235 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.235 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.235 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:05.235 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624587+0000", 2024-09-21T15:00:05.235 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-21T15:00:05.236 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.236 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.236 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:05.236 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624589+0000", 2024-09-21T15:00:05.236 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4339999999999999e-06 2024-09-21T15:00:05.236 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.236 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.236 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:05.236 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624593+0000", 2024-09-21T15:00:05.236 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5859999999999998e-06 2024-09-21T15:00:05.236 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.236 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.236 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:05.236 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624596+0000", 2024-09-21T15:00:05.237 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5830000000000001e-06 2024-09-21T15:00:05.237 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.237 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.237 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:05.237 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.633132+0000", 2024-09-21T15:00:05.237 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.008536666 2024-09-21T15:00:05.237 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.237 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.237 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:05.237 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.633216+0000", 2024-09-21T15:00:05.237 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3387999999999995e-05 2024-09-21T15:00:05.237 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.237 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.237 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:05.238 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.633415+0000", 2024-09-21T15:00:05.238 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019976799999999999 2024-09-21T15:00:05.238 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:05.238 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:05.238 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:05.238 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.238 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.238 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12516 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 245760~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:05.238 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645194+0000", 2024-09-21T15:00:05.238 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 85.536080608999995, 2024-09-21T15:00:05.238 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0084284990000000008, 2024-09-21T15:00:05.238 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:05.238 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:05.239 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:05.239 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:05.239 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:05.239 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12516 2024-09-21T15:00:05.239 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.239 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:05.239 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.239 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:05.239 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645194+0000", 2024-09-21T15:00:05.239 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:05.239 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.239 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.239 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:05.239 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645194+0000", 2024-09-21T15:00:05.239 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:05.240 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.240 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.240 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:05.240 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645193+0000", 2024-09-21T15:00:05.240 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:05.240 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.240 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.240 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:05.240 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645196+0000", 2024-09-21T15:00:05.240 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1030000000000002e-06 2024-09-21T15:00:05.240 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.240 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.240 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:05.240 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645205+0000", 2024-09-21T15:00:05.241 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6200000000000005e-06 2024-09-21T15:00:05.241 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.241 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.241 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:05.241 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645210+0000", 2024-09-21T15:00:05.241 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6280000000000004e-06 2024-09-21T15:00:05.241 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.241 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.241 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:05.241 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.653305+0000", 2024-09-21T15:00:05.241 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0080957879999999996 2024-09-21T15:00:05.241 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.241 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.241 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:05.242 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.653389+0000", 2024-09-21T15:00:05.242 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3242e-05 2024-09-21T15:00:05.242 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.242 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.242 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:05.242 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.653622+0000", 2024-09-21T15:00:05.242 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00023361599999999999 2024-09-21T15:00:05.242 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:05.242 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:05.242 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:05.242 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.242 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.242 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12517 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 278528~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:05.242 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645214+0000", 2024-09-21T15:00:05.243 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 85.536060496000005, 2024-09-21T15:00:05.243 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086832899999999998, 2024-09-21T15:00:05.243 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:05.243 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:05.243 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:05.243 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:05.243 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:05.243 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12517 2024-09-21T15:00:05.243 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.243 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:05.243 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.243 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:05.243 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645214+0000", 2024-09-21T15:00:05.243 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:05.243 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.244 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.244 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:05.244 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645214+0000", 2024-09-21T15:00:05.244 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:05.244 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.244 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.244 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:05.244 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645214+0000", 2024-09-21T15:00:05.244 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:05.244 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.244 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.244 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:05.244 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645216+0000", 2024-09-21T15:00:05.244 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5579999999999999e-06 2024-09-21T15:00:05.245 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.245 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.245 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:05.245 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645224+0000", 2024-09-21T15:00:05.245 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2439999999999993e-06 2024-09-21T15:00:05.245 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.245 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.245 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:05.245 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645231+0000", 2024-09-21T15:00:05.245 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1859999999999997e-06 2024-09-21T15:00:05.245 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.245 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.245 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:05.245 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.653651+0000", 2024-09-21T15:00:05.245 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0084204220000000003 2024-09-21T15:00:05.246 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.246 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.246 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:05.246 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.653726+0000", 2024-09-21T15:00:05.246 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5356000000000002e-05 2024-09-21T15:00:05.246 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.246 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.246 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:05.246 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.653897+0000", 2024-09-21T15:00:05.246 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000170848 2024-09-21T15:00:05.246 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:05.246 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:05.246 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:05.246 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.247 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.247 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12518 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 385024~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:05.247 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645235+0000", 2024-09-21T15:00:05.247 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 85.536039270000003, 2024-09-21T15:00:05.247 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0089841020000000008, 2024-09-21T15:00:05.247 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:05.247 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:05.247 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:05.247 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:05.247 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:05.247 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12518 2024-09-21T15:00:05.247 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.247 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:05.247 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.248 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:05.248 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645235+0000", 2024-09-21T15:00:05.248 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:05.248 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.248 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.248 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:05.248 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645235+0000", 2024-09-21T15:00:05.248 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:05.248 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.248 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.248 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:05.248 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645235+0000", 2024-09-21T15:00:05.248 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:05.248 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.249 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.249 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:05.249 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645238+0000", 2024-09-21T15:00:05.249 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.869e-06 2024-09-21T15:00:05.249 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.249 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.249 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:05.249 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645244+0000", 2024-09-21T15:00:05.249 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8930000000000002e-06 2024-09-21T15:00:05.249 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.249 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.249 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:05.249 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645249+0000", 2024-09-21T15:00:05.249 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2859999999999999e-06 2024-09-21T15:00:05.249 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.250 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.250 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:05.250 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.653930+0000", 2024-09-21T15:00:05.250 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086815320000000005 2024-09-21T15:00:05.250 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.250 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.250 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:05.250 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654028+0000", 2024-09-21T15:00:05.250 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7578000000000007e-05 2024-09-21T15:00:05.250 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.250 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.250 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:05.250 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654219+0000", 2024-09-21T15:00:05.250 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019132099999999999 2024-09-21T15:00:05.250 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:05.251 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:05.251 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:05.251 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.251 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.251 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12538 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 147456~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:05.251 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645253+0000", 2024-09-21T15:00:05.251 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 85.536021645999995, 2024-09-21T15:00:05.251 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0092095360000000008, 2024-09-21T15:00:05.251 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:05.251 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:05.251 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:05.251 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:05.251 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:05.252 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12538 2024-09-21T15:00:05.252 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.252 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:05.252 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.252 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:05.252 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645253+0000", 2024-09-21T15:00:05.252 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:05.252 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.252 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.252 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:05.252 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645253+0000", 2024-09-21T15:00:05.252 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:05.252 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.252 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.253 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:05.253 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645252+0000", 2024-09-21T15:00:05.253 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:05.253 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.253 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.253 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:05.253 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645255+0000", 2024-09-21T15:00:05.253 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2290000000000002e-06 2024-09-21T15:00:05.253 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.253 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.253 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:05.253 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645267+0000", 2024-09-21T15:00:05.253 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2631e-05 2024-09-21T15:00:05.253 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.253 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.254 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:05.254 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645278+0000", 2024-09-21T15:00:05.254 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0847e-05 2024-09-21T15:00:05.254 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.254 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.254 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:05.254 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654251+0000", 2024-09-21T15:00:05.254 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0089731340000000007 2024-09-21T15:00:05.254 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.254 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.254 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:05.254 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654296+0000", 2024-09-21T15:00:05.254 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4353e-05 2024-09-21T15:00:05.254 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.255 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.255 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:05.255 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654462+0000", 2024-09-21T15:00:05.255 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016668499999999999 2024-09-21T15:00:05.255 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:05.255 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:05.255 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:05.255 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.255 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.255 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12539 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 253952~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:05.255 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645284+0000", 2024-09-21T15:00:05.255 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 85.535990424999994, 2024-09-21T15:00:05.255 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0094524029999999998, 2024-09-21T15:00:05.256 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:05.256 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:05.256 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:05.256 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:05.256 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:05.256 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12539 2024-09-21T15:00:05.256 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.256 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:05.256 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.256 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:05.256 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645284+0000", 2024-09-21T15:00:05.256 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:05.256 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.256 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.257 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:05.257 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645284+0000", 2024-09-21T15:00:05.257 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:05.257 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.257 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.257 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:05.257 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645284+0000", 2024-09-21T15:00:05.257 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:05.257 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.257 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.257 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:05.257 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645286+0000", 2024-09-21T15:00:05.257 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7159999999999999e-06 2024-09-21T15:00:05.257 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.257 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.258 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:05.258 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645292+0000", 2024-09-21T15:00:05.258 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7400000000000001e-06 2024-09-21T15:00:05.258 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.258 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.258 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:05.258 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645297+0000", 2024-09-21T15:00:05.258 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9019999999999998e-06 2024-09-21T15:00:05.258 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.258 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.258 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:05.258 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654490+0000", 2024-09-21T15:00:05.258 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0091932360000000005 2024-09-21T15:00:05.258 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.259 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.259 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:05.259 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654565+0000", 2024-09-21T15:00:05.259 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4657e-05 2024-09-21T15:00:05.259 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.259 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.259 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:05.259 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654736+0000", 2024-09-21T15:00:05.259 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017156199999999999 2024-09-21T15:00:05.259 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:05.259 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:05.259 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:05.259 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.259 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.260 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12540 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 286720~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:05.260 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645302+0000", 2024-09-21T15:00:05.260 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 85.535972156, 2024-09-21T15:00:05.260 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0096603090000000006, 2024-09-21T15:00:05.260 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:05.260 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:05.260 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:05.260 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:05.260 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:05.260 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12540 2024-09-21T15:00:05.260 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.260 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:05.260 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.260 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:05.260 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645302+0000", 2024-09-21T15:00:05.261 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:05.261 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.261 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.261 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:05.261 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645302+0000", 2024-09-21T15:00:05.261 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:05.261 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.261 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.261 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:05.261 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645302+0000", 2024-09-21T15:00:05.261 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:05.261 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.261 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.261 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:05.262 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645305+0000", 2024-09-21T15:00:05.262 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7329999999999999e-06 2024-09-21T15:00:05.262 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.262 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.262 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:05.262 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645316+0000", 2024-09-21T15:00:05.262 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1832e-05 2024-09-21T15:00:05.262 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.262 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.262 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:05.262 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645322+0000", 2024-09-21T15:00:05.262 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.169e-06 2024-09-21T15:00:05.262 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.262 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.262 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:05.263 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654778+0000", 2024-09-21T15:00:05.263 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0094559090000000002 2024-09-21T15:00:05.263 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.263 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.263 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:05.263 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654828+0000", 2024-09-21T15:00:05.263 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0461999999999999e-05 2024-09-21T15:00:05.263 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.263 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.263 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:05.263 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654963+0000", 2024-09-21T15:00:05.263 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000134583 2024-09-21T15:00:05.263 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:05.263 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:05.263 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:05.264 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.264 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.264 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12561 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:05.264 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645742+0000", 2024-09-21T15:00:05.264 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 85.535532889999999, 2024-09-21T15:00:05.264 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0093583810000000007, 2024-09-21T15:00:05.264 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:05.264 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:05.264 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:05.264 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:05.264 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:05.264 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12561 2024-09-21T15:00:05.264 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.264 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:05.265 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.265 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:05.265 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645742+0000", 2024-09-21T15:00:05.265 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:05.265 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.265 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.265 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:05.265 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645742+0000", 2024-09-21T15:00:05.265 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:05.265 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.265 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.265 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:05.265 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645741+0000", 2024-09-21T15:00:05.265 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:00:05.266 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.266 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.266 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:05.266 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645746+0000", 2024-09-21T15:00:05.266 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8379999999999998e-06 2024-09-21T15:00:05.266 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.266 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.266 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:05.266 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645757+0000", 2024-09-21T15:00:05.266 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0705e-05 2024-09-21T15:00:05.266 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.266 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.266 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:05.266 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645765+0000", 2024-09-21T15:00:05.267 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6729999999999992e-06 2024-09-21T15:00:05.267 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.267 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.267 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:05.267 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654993+0000", 2024-09-21T15:00:05.267 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0092287480000000002 2024-09-21T15:00:05.267 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.267 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.267 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:05.267 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.655050+0000", 2024-09-21T15:00:05.267 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6394999999999998e-05 2024-09-21T15:00:05.267 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.267 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.267 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:05.268 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.655100+0000", 2024-09-21T15:00:05.268 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0049000000000001e-05 2024-09-21T15:00:05.268 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:05.268 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:05.268 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:05.268 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.268 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.268 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:15431 3.12 3:4bc7d206:::rbd_data.1136f5ed5dd6.0000000000000051:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-21T15:00:05.268 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:41.673445+0000", 2024-09-21T15:00:05.268 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 83.507829556000004, 2024-09-21T15:00:05.268 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086103610000000004, 2024-09-21T15:00:05.268 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:05.268 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:05.268 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:05.269 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:05.269 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:05.269 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 15431 2024-09-21T15:00:05.269 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.269 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:05.269 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.269 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:05.269 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.673445+0000", 2024-09-21T15:00:05.269 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:05.269 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.269 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.269 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:05.269 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.673445+0000", 2024-09-21T15:00:05.269 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:05.269 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.270 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.270 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:05.270 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.673444+0000", 2024-09-21T15:00:05.270 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:00:05.270 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.270 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.270 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:05.270 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.673451+0000", 2024-09-21T15:00:05.270 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4739999999999999e-06 2024-09-21T15:00:05.270 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.270 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.270 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:05.270 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.673472+0000", 2024-09-21T15:00:05.270 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0687e-05 2024-09-21T15:00:05.271 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.271 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.271 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:05.271 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.673481+0000", 2024-09-21T15:00:05.271 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3349999999999996e-06 2024-09-21T15:00:05.271 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.271 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.271 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:05.271 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.673565+0000", 2024-09-21T15:00:05.271 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4011000000000006e-05 2024-09-21T15:00:05.271 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.271 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.271 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:05.271 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.673650+0000", 2024-09-21T15:00:05.271 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4949000000000003e-05 2024-09-21T15:00:05.272 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.272 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.272 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 3", 2024-09-21T15:00:05.272 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.675014+0000", 2024-09-21T15:00:05.272 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0013636620000000001 2024-09-21T15:00:05.272 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.272 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.272 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:05.272 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.681866+0000", 2024-09-21T15:00:05.272 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0068525640000000002 2024-09-21T15:00:05.272 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.272 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.272 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:05.272 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.681885+0000", 2024-09-21T15:00:05.273 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8519999999999999e-05 2024-09-21T15:00:05.273 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.273 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.273 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:05.273 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.681961+0000", 2024-09-21T15:00:05.273 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6248000000000006e-05 2024-09-21T15:00:05.273 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.273 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.273 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:05.273 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:41.682055+0000", 2024-09-21T15:00:05.273 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4184000000000005e-05 2024-09-21T15:00:05.273 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:05.273 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:05.273 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:05.273 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:05.273 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:05.274 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:05.275 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T15:00:05.275 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:00:05.270+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:00:05.375 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:00:05.372+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:00:05.462 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:05.462 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:00:05.463 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T15:00:05.463 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:05.472 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T15:00:05.476 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:00:05.472+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:00:05.574 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:05.574 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:00:05.574 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T15:00:05.574 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T15:00:05.574 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:05.576 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:00:05.570+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:00:05.583 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T15:00:05.677 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:00:05.670+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:00:05.697 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:05.697 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T15:00:05.697 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T15:00:05.697 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T15:00:05.697 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.697 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31476 3.c 3:35718b6b:::rbd_data.11367721c19.0000000000000014:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T15:00:05.697 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.505975+0000", 2024-09-21T15:00:05.697 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 67.186829309999993, 2024-09-21T15:00:05.697 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99515995899999998, 2024-09-21T15:00:05.698 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:05.698 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:05.698 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:05.698 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:05.698 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:05.698 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31476 2024-09-21T15:00:05.698 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.698 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:05.698 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.699 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:05.699 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.505975+0000", 2024-09-21T15:00:05.699 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:05.699 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.699 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.699 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:05.699 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.505975+0000", 2024-09-21T15:00:05.699 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:05.699 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.699 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.699 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:05.699 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.505971+0000", 2024-09-21T15:00:05.700 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999967 2024-09-21T15:00:05.700 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.700 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.700 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:05.700 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.505989+0000", 2024-09-21T15:00:05.700 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7985000000000001e-05 2024-09-21T15:00:05.700 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.700 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.700 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:05.700 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506050+0000", 2024-09-21T15:00:05.700 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0659000000000003e-05 2024-09-21T15:00:05.700 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.700 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.700 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:05.701 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506074+0000", 2024-09-21T15:00:05.701 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4156e-05 2024-09-21T15:00:05.701 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.701 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.701 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:05.701 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506225+0000", 2024-09-21T15:00:05.701 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015097 2024-09-21T15:00:05.701 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.701 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.701 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:00:05.701 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506256+0000", 2024-09-21T15:00:05.701 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0505999999999999e-05 2024-09-21T15:00:05.701 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.701 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.702 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:05.702 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.497974+0000", 2024-09-21T15:00:05.702 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99171825700000005 2024-09-21T15:00:05.702 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.702 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.702 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for missing object", 2024-09-21T15:00:05.702 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.498449+0000", 2024-09-21T15:00:05.702 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00047506399999999998 2024-09-21T15:00:05.702 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.702 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.702 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:05.702 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.501000+0000", 2024-09-21T15:00:05.702 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0025506919999999998 2024-09-21T15:00:05.702 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.703 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.703 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:05.703 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.501135+0000", 2024-09-21T15:00:05.703 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013524000000000001 2024-09-21T15:00:05.703 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:05.703 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:05.703 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:05.703 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.703 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.703 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31480 3.c 3:36ccd808:::rbd_data.11367721c19.0000000000000018:head [sparse-read 0~14821] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T15:00:05.703 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.506116+0000", 2024-09-21T15:00:05.703 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 67.186687966999997, 2024-09-21T15:00:05.703 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99316033599999998, 2024-09-21T15:00:05.703 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:05.703 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:05.704 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:05.704 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:05.704 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:05.704 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31480 2024-09-21T15:00:05.704 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.704 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:05.704 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.704 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:05.704 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506116+0000", 2024-09-21T15:00:05.704 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:05.704 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.704 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.704 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:05.704 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506116+0000", 2024-09-21T15:00:05.705 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:05.705 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.705 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.705 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:05.705 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506115+0000", 2024-09-21T15:00:05.705 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:00:05.705 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.705 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.705 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:05.705 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506124+0000", 2024-09-21T15:00:05.705 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0659999999999993e-06 2024-09-21T15:00:05.705 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.705 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.705 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:05.705 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506145+0000", 2024-09-21T15:00:05.705 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1447e-05 2024-09-21T15:00:05.706 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.706 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.706 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:05.706 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506161+0000", 2024-09-21T15:00:05.706 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5209e-05 2024-09-21T15:00:05.706 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.706 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.706 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:05.706 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506337+0000", 2024-09-21T15:00:05.706 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017642999999999999 2024-09-21T15:00:05.706 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.706 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.706 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:00:05.706 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506375+0000", 2024-09-21T15:00:05.707 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7536000000000001e-05 2024-09-21T15:00:05.707 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.707 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.707 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:05.707 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.498815+0000", 2024-09-21T15:00:05.707 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99244067999999996 2024-09-21T15:00:05.707 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.707 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.707 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:05.707 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.498950+0000", 2024-09-21T15:00:05.707 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013436700000000001 2024-09-21T15:00:05.707 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.707 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.707 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:05.707 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499277+0000", 2024-09-21T15:00:05.707 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00032696800000000002 2024-09-21T15:00:05.708 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:05.708 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:05.708 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:05.708 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.708 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.708 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31513 3.c 3:36de8523:::rbd_data.11367721c19.0000000000000039:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T15:00:05.708 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.509071+0000", 2024-09-21T15:00:05.708 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 67.183733767000007, 2024-09-21T15:00:05.708 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99038225099999999, 2024-09-21T15:00:05.708 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:05.708 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:05.708 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:05.708 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:05.708 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:05.708 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31513 2024-09-21T15:00:05.709 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.709 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:05.709 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.709 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:05.709 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509071+0000", 2024-09-21T15:00:05.709 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:05.709 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.709 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.709 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:05.709 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509071+0000", 2024-09-21T15:00:05.709 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:05.709 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.709 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.709 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:05.709 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509068+0000", 2024-09-21T15:00:05.710 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-21T15:00:05.710 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.710 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.710 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:05.710 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509082+0000", 2024-09-21T15:00:05.710 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3545e-05 2024-09-21T15:00:05.710 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.710 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.710 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:05.710 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509116+0000", 2024-09-21T15:00:05.710 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3942000000000001e-05 2024-09-21T15:00:05.710 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.710 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.710 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:05.710 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509134+0000", 2024-09-21T15:00:05.710 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8839999999999999e-05 2024-09-21T15:00:05.711 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.711 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.711 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:05.711 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509318+0000", 2024-09-21T15:00:05.711 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018326799999999999 2024-09-21T15:00:05.711 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.711 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.711 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:00:05.711 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509346+0000", 2024-09-21T15:00:05.711 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8019000000000001e-05 2024-09-21T15:00:05.711 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.711 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.711 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:05.711 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499330+0000", 2024-09-21T15:00:05.711 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.98998399299999995 2024-09-21T15:00:05.711 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.712 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.712 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:05.712 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499453+0000", 2024-09-21T15:00:05.712 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000123115 2024-09-21T15:00:05.712 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:05.712 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:05.712 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:05.712 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.712 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.712 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31515 3.c 3:35b36ddd:::rbd_data.11367721c19.000000000000003b:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T15:00:05.712 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.509177+0000", 2024-09-21T15:00:05.712 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 67.183627341000005, 2024-09-21T15:00:05.712 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99044870600000001, 2024-09-21T15:00:05.712 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:05.712 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:05.713 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:05.713 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:05.713 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:05.713 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31515 2024-09-21T15:00:05.713 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.713 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:05.713 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.713 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:05.713 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509177+0000", 2024-09-21T15:00:05.713 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:05.713 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.713 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.713 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:05.713 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509177+0000", 2024-09-21T15:00:05.713 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:05.713 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.714 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.714 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:05.714 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509176+0000", 2024-09-21T15:00:05.714 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:00:05.714 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.714 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.714 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:05.714 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509183+0000", 2024-09-21T15:00:05.714 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6720000000000004e-06 2024-09-21T15:00:05.714 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.714 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.714 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:05.715 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509269+0000", 2024-09-21T15:00:05.715 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5881e-05 2024-09-21T15:00:05.715 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.715 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.715 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:05.715 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509303+0000", 2024-09-21T15:00:05.715 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3367000000000001e-05 2024-09-21T15:00:05.715 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.715 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.716 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:05.716 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509425+0000", 2024-09-21T15:00:05.716 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000122199 2024-09-21T15:00:05.716 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.716 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.716 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:00:05.716 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509448+0000", 2024-09-21T15:00:05.716 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2688999999999999e-05 2024-09-21T15:00:05.716 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.716 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.717 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:05.717 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499501+0000", 2024-09-21T15:00:05.717 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99005306999999998 2024-09-21T15:00:05.717 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.717 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.717 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:05.717 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499626+0000", 2024-09-21T15:00:05.717 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012501400000000001 2024-09-21T15:00:05.718 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:05.718 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:05.718 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:05.718 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.718 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.718 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31531 3.c 3:314f0287:::rbd_data.11367721c19.000000000000004b:head [sparse-read 0~7106] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T15:00:05.718 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.509329+0000", 2024-09-21T15:00:05.718 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 67.183475321000003, 2024-09-21T15:00:05.718 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99103085300000004, 2024-09-21T15:00:05.718 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:05.718 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:05.718 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:05.718 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:05.718 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:05.719 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31531 2024-09-21T15:00:05.719 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.719 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:05.719 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.719 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:05.719 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509329+0000", 2024-09-21T15:00:05.719 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:05.719 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.719 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.719 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:05.719 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509329+0000", 2024-09-21T15:00:05.719 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:05.719 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.719 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.719 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:05.720 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509328+0000", 2024-09-21T15:00:05.720 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:00:05.720 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.720 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.720 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:05.720 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509336+0000", 2024-09-21T15:00:05.720 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3170000000000002e-06 2024-09-21T15:00:05.720 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.720 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.720 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:05.720 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509353+0000", 2024-09-21T15:00:05.720 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7214000000000002e-05 2024-09-21T15:00:05.720 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.720 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.720 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:05.721 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509366+0000", 2024-09-21T15:00:05.721 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.26e-05 2024-09-21T15:00:05.721 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.721 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.721 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:05.721 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509523+0000", 2024-09-21T15:00:05.721 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015708000000000001 2024-09-21T15:00:05.721 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.721 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.721 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:00:05.721 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509544+0000", 2024-09-21T15:00:05.721 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0783000000000001e-05 2024-09-21T15:00:05.721 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.721 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.721 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:05.722 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499727+0000", 2024-09-21T15:00:05.722 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99018345200000002 2024-09-21T15:00:05.722 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.722 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.722 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:05.722 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499949+0000", 2024-09-21T15:00:05.722 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000221358 2024-09-21T15:00:05.722 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.722 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.722 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:05.722 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.500360+0000", 2024-09-21T15:00:05.722 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00041130700000000002 2024-09-21T15:00:05.722 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:05.722 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:05.723 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:05.723 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.723 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.723 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:61580 3.b 3:d3369e4f:::rbd_data.1136862f872f.0000000000000010:head [write 0~557056 [fadvise_sequential+fadvise_nocache],zero 557056~16384,write 573440~49152 [fadvise_sequential+fadvise_nocache],zero 622592~16384,write 638976~49152 [fadvise_sequential+fadvise_nocache],zero 688128~16384,write 704512~49152 [fadvise_sequential+fadvise_nocache],zero 753664~16384,write 770048~49152 [fadvise_sequential+fadvise_nocache],zero 819200~16384,write 835584~49152 [fadvise_sequential+fadvise_nocache],zero 884736~16384,write 901120~49152 [fadvise_sequential+fadvise_nocache],zero 950272~16384,write 966656~49152 [fadvise_sequential+fadvise_nocache],zero 1015808~16384,write 1032192~16384 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-21T15:00:05.723 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:19.231716+0000", 2024-09-21T15:00:05.723 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 46.461088158999999, 2024-09-21T15:00:05.723 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0096097690000000006, 2024-09-21T15:00:05.723 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:05.723 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:05.723 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:05.723 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:05.723 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:05.723 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 61580 2024-09-21T15:00:05.723 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.724 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:05.724 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.724 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:05.724 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.231716+0000", 2024-09-21T15:00:05.724 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:05.724 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.724 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.724 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:05.724 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.231716+0000", 2024-09-21T15:00:05.724 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:05.724 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.724 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.724 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:05.724 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.231715+0000", 2024-09-21T15:00:05.725 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:00:05.725 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.725 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.725 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:05.725 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.235290+0000", 2024-09-21T15:00:05.725 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0035750399999999998 2024-09-21T15:00:05.725 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.725 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.725 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:05.725 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.235310+0000", 2024-09-21T15:00:05.725 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0120999999999999e-05 2024-09-21T15:00:05.725 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.725 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.725 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:05.726 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.235319+0000", 2024-09-21T15:00:05.726 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2420000000000006e-06 2024-09-21T15:00:05.726 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.726 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.726 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:05.726 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.235416+0000", 2024-09-21T15:00:05.726 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7204e-05 2024-09-21T15:00:05.726 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.726 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.726 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:05.726 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.235775+0000", 2024-09-21T15:00:05.726 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000358883 2024-09-21T15:00:05.726 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.727 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.727 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T15:00:05.727 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.236560+0000", 2024-09-21T15:00:05.727 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000784463 2024-09-21T15:00:05.727 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.727 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.727 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:05.727 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.241119+0000", 2024-09-21T15:00:05.727 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0045590589999999999 2024-09-21T15:00:05.727 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.727 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.727 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:05.727 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.241210+0000", 2024-09-21T15:00:05.727 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1617000000000001e-05 2024-09-21T15:00:05.727 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.728 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.728 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:05.728 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.241285+0000", 2024-09-21T15:00:05.728 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4470999999999999e-05 2024-09-21T15:00:05.728 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.728 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.728 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:05.728 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.241326+0000", 2024-09-21T15:00:05.728 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0967000000000003e-05 2024-09-21T15:00:05.728 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:05.728 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:05.728 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:05.728 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.728 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.729 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:73699 3.b 3:d293b54a:::rbd_data.1136f4939602.000000000000000a:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-21T15:00:05.729 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:41.971359+0000", 2024-09-21T15:00:05.729 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.721445602999999, 2024-09-21T15:00:05.729 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0082840980000000002, 2024-09-21T15:00:05.729 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:05.729 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:05.729 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:05.729 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:05.729 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:05.729 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 73699 2024-09-21T15:00:05.729 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.729 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:05.729 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.729 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:05.730 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.971359+0000", 2024-09-21T15:00:05.730 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:05.730 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.730 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.730 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:05.730 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.971359+0000", 2024-09-21T15:00:05.730 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:05.730 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.730 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.730 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:05.730 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.971356+0000", 2024-09-21T15:00:05.730 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-21T15:00:05.730 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.730 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.730 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:05.731 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.971370+0000", 2024-09-21T15:00:05.731 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3349999999999999e-05 2024-09-21T15:00:05.731 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.731 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.731 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:05.731 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.971393+0000", 2024-09-21T15:00:05.731 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3799999999999999e-05 2024-09-21T15:00:05.731 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.731 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.731 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:05.731 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.971405+0000", 2024-09-21T15:00:05.731 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1569000000000001e-05 2024-09-21T15:00:05.731 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.731 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.731 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:05.732 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.971510+0000", 2024-09-21T15:00:05.732 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010456899999999999 2024-09-21T15:00:05.732 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.732 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.732 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:05.732 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.971752+0000", 2024-09-21T15:00:05.732 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00024250300000000001 2024-09-21T15:00:05.732 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.732 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.732 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T15:00:05.732 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.973012+0000", 2024-09-21T15:00:05.732 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0012595429999999999 2024-09-21T15:00:05.732 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.732 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.732 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:05.733 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.979357+0000", 2024-09-21T15:00:05.733 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0063454469999999997 2024-09-21T15:00:05.733 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.733 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.733 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:05.733 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.979467+0000", 2024-09-21T15:00:05.733 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000109623 2024-09-21T15:00:05.733 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.733 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.733 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:05.733 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.979585+0000", 2024-09-21T15:00:05.733 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000118044 2024-09-21T15:00:05.733 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.733 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.734 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:05.734 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.979643+0000", 2024-09-21T15:00:05.734 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8069999999999998e-05 2024-09-21T15:00:05.734 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:05.734 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:05.734 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:05.734 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.734 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.734 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:82545 3.b 3:d4eb1ee5:::rbd_data.11368296f791.0000000000000000:head [write 0~671744 [fadvise_sequential+fadvise_nocache],write 704512~65536 [fadvise_sequential+fadvise_nocache],write 786432~49152 [fadvise_sequential+fadvise_nocache],write 868352~49152 [fadvise_sequential+fadvise_nocache],write 933888~65536 [fadvise_sequential+fadvise_nocache],write 1032192~32768 [fadvise_sequential+fadvise_nocache],write 1097728~65536 [fadvise_sequential+fadvise_nocache],write 1179648~49152 [fadvise_sequential+fadvise_nocache],write 1261568~49152 [fadvise_sequential+fadvise_nocache],write 1327104~65536 [fadvise_sequential+fadvise_nocache],write 1425408~32768 [fadvise_sequential+fadvise_nocache],write 1490944~65536 [fadvise_sequential+fadvise_nocache],write 1572864~8192 [fadvise_sequential+fadvise_nocache],write 1630208~49152 [fadvise_sequential+fadvise_nocache],write 1703936~40960 [fadvise_sequential+fadvise_nocache],write 1794048~40960 [fadvise_sequential+fadvise_nocache],write 1859584~49152 [fadvise_sequential+fadvise_nocache],write 1957888~8192 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:00:05.734 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:47.416089+0000", 2024-09-21T15:00:05.734 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 18.276715595999999, 2024-09-21T15:00:05.734 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011896455, 2024-09-21T15:00:05.734 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:05.735 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:05.735 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:05.735 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:05.735 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:05.735 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82545 2024-09-21T15:00:05.735 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.735 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:05.735 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.735 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:05.735 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.416089+0000", 2024-09-21T15:00:05.735 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:05.735 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.735 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.735 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:05.735 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.416089+0000", 2024-09-21T15:00:05.736 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:05.736 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.736 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.736 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:05.736 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.416087+0000", 2024-09-21T15:00:05.736 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:00:05.736 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.736 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.736 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:05.736 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.419643+0000", 2024-09-21T15:00:05.736 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0035555550000000002 2024-09-21T15:00:05.736 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.736 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.736 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:05.737 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.419663+0000", 2024-09-21T15:00:05.737 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0202000000000001e-05 2024-09-21T15:00:05.737 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.737 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.737 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:05.737 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.419676+0000", 2024-09-21T15:00:05.737 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3207000000000001e-05 2024-09-21T15:00:05.737 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.737 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.737 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:05.737 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.419759+0000", 2024-09-21T15:00:05.737 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3521000000000002e-05 2024-09-21T15:00:05.737 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.737 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.737 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op must wait for map", 2024-09-21T15:00:05.738 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.419768+0000", 2024-09-21T15:00:05.738 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4479999999999994e-06 2024-09-21T15:00:05.738 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.738 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.738 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:05.738 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.422171+0000", 2024-09-21T15:00:05.738 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0024035469999999998 2024-09-21T15:00:05.738 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.738 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.738 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:05.738 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.422350+0000", 2024-09-21T15:00:05.738 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000179001 2024-09-21T15:00:05.738 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.738 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.739 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T15:00:05.739 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.422710+0000", 2024-09-21T15:00:05.739 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000359878 2024-09-21T15:00:05.739 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.739 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.739 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:05.739 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.426676+0000", 2024-09-21T15:00:05.739 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0039655790000000003 2024-09-21T15:00:05.739 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.739 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.739 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:05.739 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.427884+0000", 2024-09-21T15:00:05.739 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001208207 2024-09-21T15:00:05.739 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.740 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.740 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:05.740 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.427953+0000", 2024-09-21T15:00:05.740 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8533000000000003e-05 2024-09-21T15:00:05.740 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.740 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.740 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:05.740 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.427985+0000", 2024-09-21T15:00:05.740 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2481999999999998e-05 2024-09-21T15:00:05.740 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:05.740 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:05.740 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:05.740 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.741 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.741 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:83735 3.b 3:d3640016:::rbd_data.11368296f791.0000000000000006:40 [sparse-read 393216~131072 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-21T15:00:05.741 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:51.156767+0000", 2024-09-21T15:00:05.741 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.536036973, 2024-09-21T15:00:05.741 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0082826410000000003, 2024-09-21T15:00:05.741 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:05.741 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:05.741 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:05.741 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:05.741 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:05.741 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 83735 2024-09-21T15:00:05.741 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.741 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:05.742 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.742 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:05.742 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156767+0000", 2024-09-21T15:00:05.742 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:05.742 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.742 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.742 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:05.742 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156767+0000", 2024-09-21T15:00:05.742 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:05.742 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.742 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.742 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:05.742 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156767+0000", 2024-09-21T15:00:05.743 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:05.743 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.743 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.743 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:05.743 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156771+0000", 2024-09-21T15:00:05.743 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0940000000000001e-06 2024-09-21T15:00:05.743 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.743 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.743 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:05.743 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156781+0000", 2024-09-21T15:00:05.743 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0267e-05 2024-09-21T15:00:05.743 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.743 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.743 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:05.744 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156789+0000", 2024-09-21T15:00:05.744 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7659999999999998e-06 2024-09-21T15:00:05.744 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.744 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.744 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:05.744 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.164742+0000", 2024-09-21T15:00:05.744 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0079534519999999997 2024-09-21T15:00:05.744 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.744 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.744 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:05.744 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.164820+0000", 2024-09-21T15:00:05.744 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7927000000000001e-05 2024-09-21T15:00:05.744 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.744 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.745 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:05.745 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165050+0000", 2024-09-21T15:00:05.745 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000229731 2024-09-21T15:00:05.745 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:05.745 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:05.745 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:05.745 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.745 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.745 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:83736 3.b 3:d3640016:::rbd_data.11368296f791.0000000000000006:40 [sparse-read 1835008~0 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-21T15:00:05.745 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:51.156796+0000", 2024-09-21T15:00:05.745 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.53600797, 2024-09-21T15:00:05.745 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0085139110000000007, 2024-09-21T15:00:05.745 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:05.745 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:05.746 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:05.746 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:05.746 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:05.746 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 83736 2024-09-21T15:00:05.746 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.746 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:05.746 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.746 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:05.746 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156796+0000", 2024-09-21T15:00:05.746 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:05.746 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.746 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.746 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:05.747 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156796+0000", 2024-09-21T15:00:05.747 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:05.747 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.747 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.747 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:05.747 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156796+0000", 2024-09-21T15:00:05.747 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:05.747 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.747 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.747 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:05.747 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156800+0000", 2024-09-21T15:00:05.747 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.34e-06 2024-09-21T15:00:05.747 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.747 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.747 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:05.748 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156813+0000", 2024-09-21T15:00:05.748 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2532e-05 2024-09-21T15:00:05.748 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.748 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.748 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:05.748 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156821+0000", 2024-09-21T15:00:05.748 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2069999999999995e-06 2024-09-21T15:00:05.748 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.748 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.748 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:05.748 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165100+0000", 2024-09-21T15:00:05.748 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0082788730000000008 2024-09-21T15:00:05.748 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.748 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.748 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:05.748 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165192+0000", 2024-09-21T15:00:05.749 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2095999999999994e-05 2024-09-21T15:00:05.749 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.749 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.749 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:05.749 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165310+0000", 2024-09-21T15:00:05.749 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000118461 2024-09-21T15:00:05.749 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:05.749 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:05.749 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:05.749 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.749 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.749 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:83739 3.b 3:d4eb1ee5:::rbd_data.11368296f791.0000000000000000:40 [sparse-read 524288~131072 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-21T15:00:05.749 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:51.156827+0000", 2024-09-21T15:00:05.749 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.535976887, 2024-09-21T15:00:05.749 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0087319900000000002, 2024-09-21T15:00:05.749 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:05.750 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:05.750 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:05.750 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:05.750 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:05.750 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 83739 2024-09-21T15:00:05.750 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.750 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:05.750 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.750 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:05.750 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156827+0000", 2024-09-21T15:00:05.750 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:05.750 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.750 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.750 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:05.750 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156827+0000", 2024-09-21T15:00:05.751 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:05.751 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.751 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.751 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:05.751 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156827+0000", 2024-09-21T15:00:05.751 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:05.751 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.751 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.751 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:05.751 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156831+0000", 2024-09-21T15:00:05.751 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9940000000000002e-06 2024-09-21T15:00:05.751 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.751 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.751 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:05.751 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156841+0000", 2024-09-21T15:00:05.752 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6760000000000002e-06 2024-09-21T15:00:05.752 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.752 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.752 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:05.752 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156853+0000", 2024-09-21T15:00:05.752 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1972e-05 2024-09-21T15:00:05.752 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.752 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.752 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:05.752 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165334+0000", 2024-09-21T15:00:05.752 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0084816939999999997 2024-09-21T15:00:05.752 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.752 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.752 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:05.752 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165375+0000", 2024-09-21T15:00:05.753 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0911000000000002e-05 2024-09-21T15:00:05.753 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.753 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.753 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:05.753 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165559+0000", 2024-09-21T15:00:05.753 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018429300000000001 2024-09-21T15:00:05.753 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:05.753 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:05.753 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:05.753 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.753 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.753 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:83740 3.b 3:d4eb1ee5:::rbd_data.11368296f791.0000000000000000:40 [sparse-read 1966080~0 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-21T15:00:05.753 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:51.156859+0000", 2024-09-21T15:00:05.753 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.535944897, 2024-09-21T15:00:05.753 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0088540089999999995, 2024-09-21T15:00:05.754 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:05.754 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:05.754 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:05.754 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:05.754 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:05.754 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 83740 2024-09-21T15:00:05.754 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.754 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:05.754 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.754 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:05.754 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156859+0000", 2024-09-21T15:00:05.754 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:05.754 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.754 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.755 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:05.755 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156859+0000", 2024-09-21T15:00:05.755 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:05.755 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.755 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.755 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:05.755 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156859+0000", 2024-09-21T15:00:05.755 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:05.755 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.755 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.755 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:05.755 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156863+0000", 2024-09-21T15:00:05.755 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.844e-06 2024-09-21T15:00:05.755 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.755 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.755 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:05.756 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156873+0000", 2024-09-21T15:00:05.756 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0069e-05 2024-09-21T15:00:05.756 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.756 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.756 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:05.756 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156880+0000", 2024-09-21T15:00:05.756 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4240000000000004e-06 2024-09-21T15:00:05.756 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.756 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.756 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:05.756 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165590+0000", 2024-09-21T15:00:05.756 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0087098100000000001 2024-09-21T15:00:05.756 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.756 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.756 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:05.756 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165656+0000", 2024-09-21T15:00:05.757 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5541000000000001e-05 2024-09-21T15:00:05.757 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.757 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.757 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:05.757 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165713+0000", 2024-09-21T15:00:05.757 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7865000000000002e-05 2024-09-21T15:00:05.757 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:05.757 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:05.757 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:05.757 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.757 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.757 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:83751 3.b 3:d3640016:::rbd_data.11368296f791.0000000000000006:40 [sparse-read 1179648~0 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-21T15:00:05.757 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:51.156888+0000", 2024-09-21T15:00:05.757 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.535916636, 2024-09-21T15:00:05.758 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0090039539999999998, 2024-09-21T15:00:05.758 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:05.758 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:05.758 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:05.758 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:05.758 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:05.758 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 83751 2024-09-21T15:00:05.758 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.758 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:05.758 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.758 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:05.758 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156888+0000", 2024-09-21T15:00:05.758 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:05.758 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.759 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.759 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:05.759 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156888+0000", 2024-09-21T15:00:05.759 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:05.759 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.759 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.759 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:05.759 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156887+0000", 2024-09-21T15:00:05.759 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:05.759 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.759 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.759 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:05.759 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156891+0000", 2024-09-21T15:00:05.759 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.518e-06 2024-09-21T15:00:05.760 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.760 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.760 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:05.760 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156900+0000", 2024-09-21T15:00:05.760 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1759999999999999e-06 2024-09-21T15:00:05.760 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.760 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.760 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:05.760 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156908+0000", 2024-09-21T15:00:05.760 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8439999999999998e-06 2024-09-21T15:00:05.760 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.760 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.760 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:05.760 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165738+0000", 2024-09-21T15:00:05.760 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0088305689999999999 2024-09-21T15:00:05.760 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.761 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.761 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:05.761 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165804+0000", 2024-09-21T15:00:05.761 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5535000000000002e-05 2024-09-21T15:00:05.761 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.761 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.761 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:05.761 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165892+0000", 2024-09-21T15:00:05.761 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7860000000000002e-05 2024-09-21T15:00:05.761 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:05.761 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:05.761 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:05.761 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.761 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.761 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:83752 3.b 3:d3640016:::rbd_data.11368296f791.0000000000000006:40 [sparse-read 2621440~0 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-21T15:00:05.761 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:51.156915+0000", 2024-09-21T15:00:05.762 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.535889088999999, 2024-09-21T15:00:05.762 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0090768180000000004, 2024-09-21T15:00:05.762 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:05.762 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:05.762 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:05.762 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:05.762 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:05.762 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 83752 2024-09-21T15:00:05.762 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.762 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:05.762 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.762 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:05.762 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156915+0000", 2024-09-21T15:00:05.762 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:05.762 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.763 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.763 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:05.763 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156915+0000", 2024-09-21T15:00:05.763 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:05.763 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.763 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.763 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:05.763 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156915+0000", 2024-09-21T15:00:05.763 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:05.763 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.763 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.763 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:05.763 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156919+0000", 2024-09-21T15:00:05.763 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2679999999999996e-06 2024-09-21T15:00:05.764 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.764 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.764 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:05.764 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156929+0000", 2024-09-21T15:00:05.764 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.031e-05 2024-09-21T15:00:05.764 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.764 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.764 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:05.764 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156937+0000", 2024-09-21T15:00:05.764 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8820000000000001e-06 2024-09-21T15:00:05.764 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.764 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.764 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:05.765 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165909+0000", 2024-09-21T15:00:05.765 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0089722989999999996 2024-09-21T15:00:05.765 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.765 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.765 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:05.765 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165948+0000", 2024-09-21T15:00:05.765 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8114999999999997e-05 2024-09-21T15:00:05.765 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.765 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.765 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:05.765 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165992+0000", 2024-09-21T15:00:05.765 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4503999999999999e-05 2024-09-21T15:00:05.765 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:05.766 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:05.766 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:05.766 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.766 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.766 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:83755 3.b 3:d4eb1ee5:::rbd_data.11368296f791.0000000000000000:40 [sparse-read 1310720~0 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-21T15:00:05.766 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:51.156944+0000", 2024-09-21T15:00:05.766 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.535860015000001, 2024-09-21T15:00:05.766 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.009159146, 2024-09-21T15:00:05.766 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:05.766 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:05.766 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:05.766 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:05.766 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:05.766 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 83755 2024-09-21T15:00:05.766 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.767 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:05.767 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.767 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:05.767 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156944+0000", 2024-09-21T15:00:05.767 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:05.767 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.767 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.767 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:05.767 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156944+0000", 2024-09-21T15:00:05.767 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:05.767 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.767 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.767 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:05.767 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156944+0000", 2024-09-21T15:00:05.767 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:05.767 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.768 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.768 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:05.768 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156948+0000", 2024-09-21T15:00:05.768 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5279999999999997e-06 2024-09-21T15:00:05.768 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.768 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.768 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:05.768 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156959+0000", 2024-09-21T15:00:05.768 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0215e-05 2024-09-21T15:00:05.768 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.768 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.768 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:05.768 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156967+0000", 2024-09-21T15:00:05.768 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6119999999999999e-06 2024-09-21T15:00:05.769 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.769 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.769 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:05.769 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.166007+0000", 2024-09-21T15:00:05.769 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0090395260000000009 2024-09-21T15:00:05.769 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.769 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.769 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:05.769 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.166053+0000", 2024-09-21T15:00:05.769 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6281e-05 2024-09-21T15:00:05.769 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.769 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.769 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:05.769 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.166103+0000", 2024-09-21T15:00:05.769 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0510000000000003e-05 2024-09-21T15:00:05.769 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:05.770 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:05.770 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:05.770 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.770 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.770 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:83763 3.b 3:d3640016:::rbd_data.11368296f791.0000000000000006:40 [sparse-read 524288~81920 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-21T15:00:05.770 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:51.157975+0000", 2024-09-21T15:00:05.770 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.534828907, 2024-09-21T15:00:05.770 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.008327348, 2024-09-21T15:00:05.770 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:05.770 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:05.770 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:05.770 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:05.770 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:05.771 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 83763 2024-09-21T15:00:05.771 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.771 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:05.771 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.771 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:05.771 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.157975+0000", 2024-09-21T15:00:05.771 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:05.771 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.771 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.771 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:05.771 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.157975+0000", 2024-09-21T15:00:05.771 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:05.771 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.771 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.771 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:05.772 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.157970+0000", 2024-09-21T15:00:05.772 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999943 2024-09-21T15:00:05.772 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.772 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.772 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:05.772 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.157985+0000", 2024-09-21T15:00:05.772 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4807000000000001e-05 2024-09-21T15:00:05.772 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.772 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.772 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:05.772 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.158012+0000", 2024-09-21T15:00:05.772 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7600999999999999e-05 2024-09-21T15:00:05.772 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.772 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.773 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:05.773 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.158019+0000", 2024-09-21T15:00:05.773 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9319999999999999e-06 2024-09-21T15:00:05.773 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.773 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.773 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:05.773 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.166118+0000", 2024-09-21T15:00:05.773 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0080992489999999993 2024-09-21T15:00:05.773 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.773 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.773 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:05.773 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.166156+0000", 2024-09-21T15:00:05.773 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7639999999999999e-05 2024-09-21T15:00:05.773 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.774 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.774 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:05.774 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.166303+0000", 2024-09-21T15:00:05.774 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000146799 2024-09-21T15:00:05.774 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:05.774 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:05.774 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:05.774 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.774 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.774 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:83764 3.b 3:d3640016:::rbd_data.11368296f791.0000000000000006:40 [sparse-read 1966080~0 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-21T15:00:05.774 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:51.158026+0000", 2024-09-21T15:00:05.774 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.534778473999999, 2024-09-21T15:00:05.774 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0083785839999999997, 2024-09-21T15:00:05.774 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:05.775 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:05.775 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:05.775 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:05.775 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:05.775 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 83764 2024-09-21T15:00:05.775 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.775 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:05.775 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.775 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:05.775 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.158026+0000", 2024-09-21T15:00:05.775 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:05.775 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.775 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.775 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:05.776 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.158026+0000", 2024-09-21T15:00:05.776 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:05.776 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.776 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.776 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:05.776 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.158025+0000", 2024-09-21T15:00:05.776 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:05.776 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.776 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.776 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:05.776 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.158053+0000", 2024-09-21T15:00:05.776 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7594e-05 2024-09-21T15:00:05.776 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.776 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.776 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:05.777 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.158061+0000", 2024-09-21T15:00:05.777 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2069999999999995e-06 2024-09-21T15:00:05.777 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.777 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.777 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:05.777 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.158067+0000", 2024-09-21T15:00:05.777 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3799999999999999e-06 2024-09-21T15:00:05.777 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.777 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.777 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:05.777 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.166321+0000", 2024-09-21T15:00:05.777 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0082539870000000008 2024-09-21T15:00:05.777 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.777 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.778 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:05.778 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.166360+0000", 2024-09-21T15:00:05.778 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8597999999999999e-05 2024-09-21T15:00:05.778 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.778 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.778 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:05.778 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.166404+0000", 2024-09-21T15:00:05.778 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4378000000000001e-05 2024-09-21T15:00:05.778 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:05.778 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:05.778 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:05.778 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.778 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.778 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:97675 3.b 3:d321acb8:::rbd_data.1136bfc79a70.0000000000000003:head [write 0~4718592 [fadvise_sequential+fadvise_nocache],write 5242880~147456 [fadvise_sequential+fadvise_nocache],write 5455872~147456 [fadvise_sequential+fadvise_nocache],write 5668864~229376 [fadvise_sequential+fadvise_nocache],write 5963776~147456 [fadvise_sequential+fadvise_nocache],write 6176768~229376 [fadvise_sequential+fadvise_nocache],write 6471680~147456 [fadvise_sequential+fadvise_nocache],write 6684672~229376 [fadvise_sequential+fadvise_nocache],write 6979584~147456 [fadvise_sequential+fadvise_nocache],write 7192576~229376 [fadvise_sequential+fadvise_nocache],write 7487488~147456 [fadvise_sequential+fadvise_nocache],write 7700480~147456 [fadvise_sequential+fadvise_nocache],write 7864320~65536 [fadvise_sequential+fadvise_nocache],write 7995392~147456 [fadvise_sequential+fadvise_nocache],write 8208384~147456 [fadvise_sequential+fadvise_nocache],write 8388608~49152 [fadvise_sequential+fadvise_nocache],write 8536064~114688 [fadvise_sequential+fadvise_nocache],write 8749056~114688 [fadvise_sequential+fadvise_nocache],write 8912896~32768 [fadvise_sequential+fadvise_nocache],write 9125888~32768 [fadvise_sequential+fadvise_nocache],write 9338880~32768 [fadvise_sequential+fadvise_nocache],write 9437184~16384 [fadvise_sequential+fadvise_nocache],write 9633792~32768 [fadvise_sequential+fadvise_nocache],write 9846784~32768 [fadvise_sequential+fadvise_nocache],write 9961472~114688 [fadvise_sequential+fadvise_nocache],write 10092544~196608 [fadvise_sequential+fadvise_nocache],write 10305536~278528 [fadvise_sequential+fadvise_nocache],write 10600448~196608 [fadvise_sequential+fadvise_nocache],write 10813440~278528 [fadvise_sequential+fadvise_nocache],write 11108352~196608 [fadvise_sequential+fadvise_nocache],write 11321344~196608 [fadvise_sequential+fadvise_nocache],write 11534336~65536 [fadvise_sequential+fadvise_nocache],write 11632640~98304 [fadvise_sequential+fadvise_nocache],write 11747328~65536 [fadvise_sequential+fadvise_nocache],write 11845632~98304 [fadvise_sequential+fadvise_nocache],write 11960320~65536 [fadvise_sequential+fadvise_nocache],write 12058624~49152 [fadvise_sequential+fadvise_nocache],write 12140544~98304 [fadvise_sequential+fadvise_nocache],write 12255232~65536 [fadvise_sequential+fadvise_nocache],write 12353536~98304 [fadvise_sequential+fadvise_nocache],write 12468224~65536 [fadvise_sequential+fadvise_nocache],write 12566528~16384 [fadvise_sequential+fadvise_nocache],write 12648448~98304 [fadvise_sequential+fadvise_nocache],write 12861440~98304 [fadvise_sequential+fadvise_nocache],write 13074432~32768 [fadvise_sequential+fadvise_nocache],write 13156352~98304 [fadvise_sequential+fadvise_nocache],write 13369344~98304 [fadvise_sequential+fadvise_nocache],write 13582336~49152 [fadvise_sequential+fadvise_nocache],write 13664256~98304 [fadvise_sequential+fadvise_nocache],write 13877248~98304 [fadvise_sequential+fadvise_nocache],write 14090240~65536 [fadvise_sequential+fadvise_nocache],write 15286272~81920 [fadvise_sequential+fadvise_nocache],write 15499264~81920 [fadvise_sequential+fadvise_nocache],write 15712256~16384 [fadvise_sequential+fadvise_nocache],write 15794176~81920 [fadvise_sequential+fadvise_nocache],write 16007168~81920 [fadvise_sequential+fadvise_nocache],write 16220160~32768 [fadvise_sequential+fadvise_nocache],write 16302080~81920 [fadvise_sequential+fadvise_nocache],write 16515072~81920 [fadvise_sequential+fadvise_nocache],write 16728064~49152 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:05.779 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.302412+0000", 2024-09-21T15:00:05.779 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.39039280700000001, 2024-09-21T15:00:05.779 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.067380532000000007, 2024-09-21T15:00:05.779 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:05.779 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:05.779 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:05.779 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:05.779 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:05.779 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97675 2024-09-21T15:00:05.779 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.779 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:05.779 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.779 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:05.779 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.302412+0000", 2024-09-21T15:00:05.779 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:05.780 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.780 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.780 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:05.780 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.302412+0000", 2024-09-21T15:00:05.780 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:05.780 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.780 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.780 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:05.780 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.302409+0000", 2024-09-21T15:00:05.780 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-21T15:00:05.780 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.780 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.780 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:05.780 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328550+0000", 2024-09-21T15:00:05.780 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.026140674999999999 2024-09-21T15:00:05.781 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.781 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.781 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:05.781 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328584+0000", 2024-09-21T15:00:05.781 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4165999999999998e-05 2024-09-21T15:00:05.781 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.781 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.781 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:05.781 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328599+0000", 2024-09-21T15:00:05.781 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5505000000000001e-05 2024-09-21T15:00:05.781 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.781 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.781 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:05.781 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328704+0000", 2024-09-21T15:00:05.781 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010487199999999999 2024-09-21T15:00:05.781 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.782 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.782 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:05.782 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.329082+0000", 2024-09-21T15:00:05.782 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00037804899999999998 2024-09-21T15:00:05.782 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.782 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.782 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T15:00:05.782 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.331158+0000", 2024-09-21T15:00:05.782 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0020755449999999998 2024-09-21T15:00:05.782 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.782 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.782 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:05.782 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.360553+0000", 2024-09-21T15:00:05.783 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.029395553000000001 2024-09-21T15:00:05.783 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.783 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.783 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:05.783 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.369496+0000", 2024-09-21T15:00:05.783 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0089428619999999993 2024-09-21T15:00:05.783 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.783 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.783 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:05.783 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.369686+0000", 2024-09-21T15:00:05.783 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019000399999999999 2024-09-21T15:00:05.783 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.783 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.783 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:05.784 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.369792+0000", 2024-09-21T15:00:05.784 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000105775 2024-09-21T15:00:05.784 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:05.784 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:05.784 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:05.784 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.784 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.784 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:97693 3.b 3:d7d7b39b:::rbd_data.1136b9c49b1c.000000000000000c:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:05.784 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.328643+0000", 2024-09-21T15:00:05.784 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.36416102099999997, 2024-09-21T15:00:05.784 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.031062136000000001, 2024-09-21T15:00:05.784 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:05.784 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:05.784 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:05.785 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:05.785 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:05.785 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97693 2024-09-21T15:00:05.785 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.785 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:05.785 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.785 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:05.785 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328643+0000", 2024-09-21T15:00:05.785 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:05.785 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.785 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.785 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:05.785 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328643+0000", 2024-09-21T15:00:05.785 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:05.786 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.786 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.786 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:05.786 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328642+0000", 2024-09-21T15:00:05.786 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:00:05.786 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.786 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.786 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:05.786 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328650+0000", 2024-09-21T15:00:05.786 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7770000000000008e-06 2024-09-21T15:00:05.786 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.786 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.786 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:05.786 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328662+0000", 2024-09-21T15:00:05.786 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2026e-05 2024-09-21T15:00:05.786 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.787 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.787 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:05.787 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328671+0000", 2024-09-21T15:00:05.787 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.585e-06 2024-09-21T15:00:05.787 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.787 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.787 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:05.787 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.359498+0000", 2024-09-21T15:00:05.787 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.030827001 2024-09-21T15:00:05.787 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.787 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.787 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:05.787 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.359603+0000", 2024-09-21T15:00:05.787 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010512400000000001 2024-09-21T15:00:05.788 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.788 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.788 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:05.788 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.359705+0000", 2024-09-21T15:00:05.788 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000102327 2024-09-21T15:00:05.788 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:05.788 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:05.788 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:05.788 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.788 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.788 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:97724 3.b 3:d4bf6bcf:::rbd_data.1136b9c49b1c.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:05.788 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.328704+0000", 2024-09-21T15:00:05.788 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.364100749, 2024-09-21T15:00:05.788 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.031157957, 2024-09-21T15:00:05.788 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:05.789 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:05.789 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:05.789 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:05.789 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:05.789 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97724 2024-09-21T15:00:05.789 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.789 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:05.789 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.789 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:05.789 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328704+0000", 2024-09-21T15:00:05.789 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:05.789 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.789 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.789 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:05.789 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328704+0000", 2024-09-21T15:00:05.790 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:05.790 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.790 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.790 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:05.790 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328703+0000", 2024-09-21T15:00:05.790 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:00:05.790 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.790 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.790 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:05.790 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328708+0000", 2024-09-21T15:00:05.790 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0050000000000004e-06 2024-09-21T15:00:05.790 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.790 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.790 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:05.791 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328719+0000", 2024-09-21T15:00:05.791 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1052e-05 2024-09-21T15:00:05.791 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.791 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.791 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:05.791 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328727+0000", 2024-09-21T15:00:05.791 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4519999999999997e-06 2024-09-21T15:00:05.791 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.791 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.791 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:05.791 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.359737+0000", 2024-09-21T15:00:05.791 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.031009341999999999 2024-09-21T15:00:05.791 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.791 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.791 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:05.792 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.359806+0000", 2024-09-21T15:00:05.792 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9750000000000001e-05 2024-09-21T15:00:05.792 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:05.792 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:05.792 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:05.792 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.359862+0000", 2024-09-21T15:00:05.792 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5084999999999999e-05 2024-09-21T15:00:05.792 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:05.792 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:05.792 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:05.792 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:05.792 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:05.792 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:05.794 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T15:00:05.794 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:00:05.786+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:00:05.895 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:00:05.890+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:00:05.917 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:05.917 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:00:05.917 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T15:00:05.917 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:05.927 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T15:00:05.995 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:00:05.994+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:00:06.041 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:06.046 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:00:06.046 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T15:00:06.046 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T15:00:06.046 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:06.051 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T15:00:06.096 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:00:06.090+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:00:06.143 INFO:teuthology.orchestra.run.smithi032.stderr:set osd.6 primary-affinity to 0.834935 (8547182) 2024-09-21T15:00:06.170 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:06.170 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T15:00:06.170 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T15:00:06.170 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T15:00:06.170 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.170 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:1678 3.c 3:377cc7e1:::rbd_data.1136515ecb6b.000000000000000f:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T15:00:06.170 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:30.988085+0000", 2024-09-21T15:00:06.170 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 95.181009567999993, 2024-09-21T15:00:06.170 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.032974802999999997, 2024-09-21T15:00:06.170 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:06.171 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:06.171 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:06.171 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:06.171 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:06.171 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1678 2024-09-21T15:00:06.171 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.171 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:06.171 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.171 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:06.171 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988085+0000", 2024-09-21T15:00:06.171 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:06.171 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.171 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.171 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:06.171 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988085+0000", 2024-09-21T15:00:06.172 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:06.172 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.172 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.172 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:06.172 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988083+0000", 2024-09-21T15:00:06.172 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-21T15:00:06.172 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.172 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.172 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:06.172 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988093+0000", 2024-09-21T15:00:06.172 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8800000000000003e-06 2024-09-21T15:00:06.172 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.172 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.172 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:06.172 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988113+0000", 2024-09-21T15:00:06.173 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0217999999999999e-05 2024-09-21T15:00:06.173 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.173 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.173 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:06.173 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988123+0000", 2024-09-21T15:00:06.173 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5859999999999994e-06 2024-09-21T15:00:06.173 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.173 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.173 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:06.173 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988203+0000", 2024-09-21T15:00:06.173 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0834000000000006e-05 2024-09-21T15:00:06.173 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.173 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.173 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:06.173 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988376+0000", 2024-09-21T15:00:06.174 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017301299999999999 2024-09-21T15:00:06.174 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.174 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.174 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 7", 2024-09-21T15:00:06.174 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.996338+0000", 2024-09-21T15:00:06.174 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0079618520000000002 2024-09-21T15:00:06.174 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.174 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.174 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:06.174 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.020881+0000", 2024-09-21T15:00:06.174 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024542731000000002 2024-09-21T15:00:06.174 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.174 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.174 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:06.175 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.020904+0000", 2024-09-21T15:00:06.175 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2861e-05 2024-09-21T15:00:06.175 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.175 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.175 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:06.175 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.020987+0000", 2024-09-21T15:00:06.175 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2806000000000006e-05 2024-09-21T15:00:06.175 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.175 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.175 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:06.175 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.021060+0000", 2024-09-21T15:00:06.175 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2871000000000001e-05 2024-09-21T15:00:06.175 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:06.175 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:06.176 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:06.176 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.176 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.176 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9403 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:06.176 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157949+0000", 2024-09-21T15:00:06.176 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 87.011144920999996, 2024-09-21T15:00:06.176 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023540373999999999, 2024-09-21T15:00:06.176 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:06.176 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:06.176 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:06.176 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:06.176 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:06.176 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9403 2024-09-21T15:00:06.176 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.176 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:06.177 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.177 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:06.177 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157949+0000", 2024-09-21T15:00:06.177 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:06.177 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.177 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.177 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:06.177 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157949+0000", 2024-09-21T15:00:06.177 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:06.177 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.177 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.177 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:06.177 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157948+0000", 2024-09-21T15:00:06.177 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T15:00:06.177 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.178 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.178 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:06.178 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157956+0000", 2024-09-21T15:00:06.178 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7999999999999999e-06 2024-09-21T15:00:06.178 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.178 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.178 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:06.178 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157969+0000", 2024-09-21T15:00:06.178 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2792000000000001e-05 2024-09-21T15:00:06.178 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.178 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.178 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:06.178 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157977+0000", 2024-09-21T15:00:06.178 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.704e-06 2024-09-21T15:00:06.178 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.179 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.179 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:06.179 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177246+0000", 2024-09-21T15:00:06.179 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019268264 2024-09-21T15:00:06.179 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.179 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.179 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:06.179 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177315+0000", 2024-09-21T15:00:06.179 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9480999999999994e-05 2024-09-21T15:00:06.179 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.179 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.179 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T15:00:06.179 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177615+0000", 2024-09-21T15:00:06.179 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00029929499999999999 2024-09-21T15:00:06.179 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.180 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.180 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:06.180 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181333+0000", 2024-09-21T15:00:06.180 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0037183149999999998 2024-09-21T15:00:06.180 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.180 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.180 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:06.180 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181374+0000", 2024-09-21T15:00:06.180 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.0723999999999998e-05 2024-09-21T15:00:06.180 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.180 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.180 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:06.180 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181434+0000", 2024-09-21T15:00:06.180 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0024e-05 2024-09-21T15:00:06.181 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.181 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.181 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:06.181 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181490+0000", 2024-09-21T15:00:06.181 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6245000000000001e-05 2024-09-21T15:00:06.181 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:06.181 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:06.181 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:06.181 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.181 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.181 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9410 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:06.181 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.174103+0000", 2024-09-21T15:00:06.181 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 86.994991162999995, 2024-09-21T15:00:06.181 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025493703, 2024-09-21T15:00:06.181 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:06.182 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:06.182 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:06.182 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:06.182 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:06.182 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9410 2024-09-21T15:00:06.182 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.182 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:06.182 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.182 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:06.182 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.174103+0000", 2024-09-21T15:00:06.182 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:06.182 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.182 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.182 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:06.183 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.174103+0000", 2024-09-21T15:00:06.183 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:06.183 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.183 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.183 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:06.183 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.174102+0000", 2024-09-21T15:00:06.183 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:00:06.183 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.183 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.183 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:06.183 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.174108+0000", 2024-09-21T15:00:06.183 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1149999999999999e-06 2024-09-21T15:00:06.183 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.183 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.184 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:06.184 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.174117+0000", 2024-09-21T15:00:06.184 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4870000000000002e-06 2024-09-21T15:00:06.184 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.184 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.184 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:06.184 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.174122+0000", 2024-09-21T15:00:06.184 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9790000000000001e-06 2024-09-21T15:00:06.184 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.184 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.184 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:06.184 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.187830+0000", 2024-09-21T15:00:06.184 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013708048 2024-09-21T15:00:06.184 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.185 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.185 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:06.185 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.187913+0000", 2024-09-21T15:00:06.185 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2763999999999996e-05 2024-09-21T15:00:06.185 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.185 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.185 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T15:00:06.185 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.188230+0000", 2024-09-21T15:00:06.185 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00031745100000000001 2024-09-21T15:00:06.185 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.185 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.185 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:06.185 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.192708+0000", 2024-09-21T15:00:06.185 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0044783890000000002 2024-09-21T15:00:06.185 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.186 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.186 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:06.186 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199487+0000", 2024-09-21T15:00:06.186 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0067782149999999998 2024-09-21T15:00:06.186 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.186 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.186 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:06.186 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199560+0000", 2024-09-21T15:00:06.186 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3752999999999997e-05 2024-09-21T15:00:06.186 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.186 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.186 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:06.186 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199597+0000", 2024-09-21T15:00:06.187 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6526999999999998e-05 2024-09-21T15:00:06.187 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:06.187 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:06.187 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:06.187 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.187 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.187 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9411 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:06.187 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.175943+0000", 2024-09-21T15:00:06.187 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 86.993151143999995, 2024-09-21T15:00:06.187 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023773181000000001, 2024-09-21T15:00:06.187 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:06.187 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:06.187 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:06.187 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:06.187 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:06.188 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9411 2024-09-21T15:00:06.188 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.188 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:06.188 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.188 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:06.188 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175943+0000", 2024-09-21T15:00:06.188 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:06.188 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.188 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.188 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:06.188 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175943+0000", 2024-09-21T15:00:06.188 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:06.188 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.188 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.188 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:06.189 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175942+0000", 2024-09-21T15:00:06.189 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:00:06.189 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.189 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.189 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:06.189 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175948+0000", 2024-09-21T15:00:06.189 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9689999999999999e-06 2024-09-21T15:00:06.189 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.189 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.189 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:06.189 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175960+0000", 2024-09-21T15:00:06.189 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1745e-05 2024-09-21T15:00:06.189 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.189 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.189 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:06.190 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175965+0000", 2024-09-21T15:00:06.190 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.203e-06 2024-09-21T15:00:06.190 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.190 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.190 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:06.190 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.189900+0000", 2024-09-21T15:00:06.190 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013934940999999999 2024-09-21T15:00:06.190 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.190 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.190 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:06.190 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.189988+0000", 2024-09-21T15:00:06.190 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8419999999999997e-05 2024-09-21T15:00:06.190 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.190 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.191 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T15:00:06.191 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.190321+0000", 2024-09-21T15:00:06.191 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00033231800000000001 2024-09-21T15:00:06.191 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.191 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.191 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:06.191 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.192746+0000", 2024-09-21T15:00:06.191 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0024254400000000001 2024-09-21T15:00:06.191 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.191 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.191 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:06.191 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199605+0000", 2024-09-21T15:00:06.191 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0068582809999999999 2024-09-21T15:00:06.191 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.192 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.192 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:06.192 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199680+0000", 2024-09-21T15:00:06.192 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5630000000000006e-05 2024-09-21T15:00:06.192 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.192 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.192 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:06.192 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199716+0000", 2024-09-21T15:00:06.192 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6248999999999998e-05 2024-09-21T15:00:06.192 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:06.192 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:06.192 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:06.192 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.192 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.192 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9412 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:06.192 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.177402+0000", 2024-09-21T15:00:06.193 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 86.991692047000001, 2024-09-21T15:00:06.193 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022445590000000001, 2024-09-21T15:00:06.193 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:06.193 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:06.193 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:06.193 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:06.193 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:06.193 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9412 2024-09-21T15:00:06.193 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.193 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:06.193 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.193 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:06.193 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177402+0000", 2024-09-21T15:00:06.193 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:06.194 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.194 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.194 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:06.194 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177402+0000", 2024-09-21T15:00:06.194 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:06.194 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.194 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.194 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:06.194 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177401+0000", 2024-09-21T15:00:06.194 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:00:06.194 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.194 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.194 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:06.194 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177409+0000", 2024-09-21T15:00:06.195 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4710000000000001e-06 2024-09-21T15:00:06.195 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.195 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.195 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:06.195 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177419+0000", 2024-09-21T15:00:06.195 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0351e-05 2024-09-21T15:00:06.195 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.195 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.195 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:06.195 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177424+0000", 2024-09-21T15:00:06.195 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1919999999999996e-06 2024-09-21T15:00:06.195 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.195 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.195 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:06.196 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.191343+0000", 2024-09-21T15:00:06.196 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.01391878 2024-09-21T15:00:06.196 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.196 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.196 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:06.196 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.191446+0000", 2024-09-21T15:00:06.196 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000102453 2024-09-21T15:00:06.196 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.196 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.196 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T15:00:06.196 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.192200+0000", 2024-09-21T15:00:06.196 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00075476899999999997 2024-09-21T15:00:06.196 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.196 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.197 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:06.197 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.193866+0000", 2024-09-21T15:00:06.197 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0016657519999999999 2024-09-21T15:00:06.197 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.197 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.197 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:06.197 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199724+0000", 2024-09-21T15:00:06.197 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0058579529999999999 2024-09-21T15:00:06.197 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.197 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.197 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:06.197 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199803+0000", 2024-09-21T15:00:06.197 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8662e-05 2024-09-21T15:00:06.197 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.198 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.198 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:06.198 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199848+0000", 2024-09-21T15:00:06.198 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5216000000000002e-05 2024-09-21T15:00:06.198 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:06.198 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:06.198 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:06.198 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.198 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.198 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:96341 3.15 3:aaa1da52:::rbd_data.1136b9c49b1c.0000000000000003:49 [read 212992~32768 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:06.198 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.200375+0000", 2024-09-21T15:00:06.198 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.96871918999999995, 2024-09-21T15:00:06.198 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026255225, 2024-09-21T15:00:06.198 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:06.198 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:06.199 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:06.199 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:06.199 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:06.199 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 96341 2024-09-21T15:00:06.199 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.199 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:06.199 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.199 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:06.199 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200375+0000", 2024-09-21T15:00:06.199 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:06.199 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.199 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.199 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:06.199 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200375+0000", 2024-09-21T15:00:06.199 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:06.200 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.200 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.200 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:06.200 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200374+0000", 2024-09-21T15:00:06.200 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T15:00:06.200 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.200 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.200 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:06.200 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200381+0000", 2024-09-21T15:00:06.200 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6950000000000003e-06 2024-09-21T15:00:06.200 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.200 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.200 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:06.200 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200390+0000", 2024-09-21T15:00:06.200 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2750000000000001e-06 2024-09-21T15:00:06.201 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.201 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.201 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:06.201 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200397+0000", 2024-09-21T15:00:06.201 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.1389999999999998e-06 2024-09-21T15:00:06.201 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.201 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.201 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:06.201 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.216693+0000", 2024-09-21T15:00:06.201 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016296076 2024-09-21T15:00:06.201 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.201 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.201 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:06.201 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.216784+0000", 2024-09-21T15:00:06.201 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0484999999999998e-05 2024-09-21T15:00:06.201 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.202 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.202 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:06.202 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.226630+0000", 2024-09-21T15:00:06.202 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0098468730000000008 2024-09-21T15:00:06.202 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:06.202 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:06.202 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:06.202 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.202 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.202 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:96478 3.15 3:aee50b17:::rbd_data.1136b9c49b1c.0000000000000015:49 [read 1327104~49152 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:06.202 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.200926+0000", 2024-09-21T15:00:06.202 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.96816793000000001, 2024-09-21T15:00:06.202 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021532795, 2024-09-21T15:00:06.202 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:06.202 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:06.202 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:06.203 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:06.203 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:06.203 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 96478 2024-09-21T15:00:06.203 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.203 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:06.203 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.203 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:06.203 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200926+0000", 2024-09-21T15:00:06.203 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:06.203 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.203 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.203 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:06.203 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200926+0000", 2024-09-21T15:00:06.203 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:06.203 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.204 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.204 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:06.204 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200926+0000", 2024-09-21T15:00:06.204 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:06.204 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.204 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.204 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:06.204 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200929+0000", 2024-09-21T15:00:06.204 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4449999999999999e-06 2024-09-21T15:00:06.204 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.204 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.204 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:06.204 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200934+0000", 2024-09-21T15:00:06.204 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2249999999999999e-06 2024-09-21T15:00:06.205 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.205 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.205 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:06.205 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200938+0000", 2024-09-21T15:00:06.205 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6739999999999996e-06 2024-09-21T15:00:06.205 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.205 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.205 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:06.205 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.222046+0000", 2024-09-21T15:00:06.205 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021107225 2024-09-21T15:00:06.205 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.205 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.205 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:06.205 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.222143+0000", 2024-09-21T15:00:06.205 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.7740999999999998e-05 2024-09-21T15:00:06.206 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.206 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.206 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:06.206 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.222459+0000", 2024-09-21T15:00:06.206 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00031578500000000001 2024-09-21T15:00:06.206 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:06.206 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:06.206 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:06.206 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.206 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.206 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:96479 3.15 3:aee50b17:::rbd_data.1136b9c49b1c.0000000000000015:49 [read 1769472~49152 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:06.206 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.200942+0000", 2024-09-21T15:00:06.206 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.96815189599999996, 2024-09-21T15:00:06.206 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021994197, 2024-09-21T15:00:06.206 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:06.207 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:06.207 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:06.207 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:06.207 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:06.207 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 96479 2024-09-21T15:00:06.207 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.207 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:06.207 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.207 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:06.207 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200942+0000", 2024-09-21T15:00:06.207 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:06.207 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.207 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.207 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:06.207 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200942+0000", 2024-09-21T15:00:06.208 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:06.208 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.208 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.208 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:06.208 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200942+0000", 2024-09-21T15:00:06.208 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:06.208 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.208 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.208 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:06.208 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200945+0000", 2024-09-21T15:00:06.208 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3989999999999998e-06 2024-09-21T15:00:06.208 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.208 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.208 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:06.208 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200950+0000", 2024-09-21T15:00:06.208 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.631e-06 2024-09-21T15:00:06.209 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.209 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.209 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:06.209 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200955+0000", 2024-09-21T15:00:06.209 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6319999999999999e-06 2024-09-21T15:00:06.209 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.209 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.209 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:06.209 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.222484+0000", 2024-09-21T15:00:06.209 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021528988999999998 2024-09-21T15:00:06.209 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.209 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.209 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:06.209 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.222558+0000", 2024-09-21T15:00:06.209 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4239000000000005e-05 2024-09-21T15:00:06.209 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.210 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.210 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:06.210 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.222937+0000", 2024-09-21T15:00:06.210 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00037861999999999998 2024-09-21T15:00:06.210 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:06.210 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:06.210 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:06.210 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.210 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.210 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:96480 3.15 3:aee50b17:::rbd_data.1136b9c49b1c.0000000000000015:49 [read 2211840~49152 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:06.210 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.200959+0000", 2024-09-21T15:00:06.210 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.96813560399999998, 2024-09-21T15:00:06.210 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022433293, 2024-09-21T15:00:06.210 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:06.210 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:06.211 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:06.211 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:06.211 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:06.211 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 96480 2024-09-21T15:00:06.211 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.211 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:06.211 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.211 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:06.211 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200959+0000", 2024-09-21T15:00:06.211 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:06.211 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.211 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.211 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:06.211 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200959+0000", 2024-09-21T15:00:06.211 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:06.212 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.212 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.212 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:06.212 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200958+0000", 2024-09-21T15:00:06.212 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:06.212 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.212 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.212 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:06.212 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200961+0000", 2024-09-21T15:00:06.212 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3549999999999999e-06 2024-09-21T15:00:06.212 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.212 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.212 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:06.212 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200966+0000", 2024-09-21T15:00:06.212 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5029999999999997e-06 2024-09-21T15:00:06.213 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.213 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.213 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:06.213 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200971+0000", 2024-09-21T15:00:06.213 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.561e-06 2024-09-21T15:00:06.213 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.213 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.213 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:06.213 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.222963+0000", 2024-09-21T15:00:06.213 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021992088 2024-09-21T15:00:06.213 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.213 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.213 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:06.214 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.223045+0000", 2024-09-21T15:00:06.214 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2464000000000002e-05 2024-09-21T15:00:06.214 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.214 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.214 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:06.214 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.223392+0000", 2024-09-21T15:00:06.214 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00034665299999999999 2024-09-21T15:00:06.214 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:06.214 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:06.214 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:06.214 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.214 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.214 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:96481 3.15 3:aee50b17:::rbd_data.1136b9c49b1c.0000000000000015:49 [read 2654208~49152 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:06.214 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.200975+0000", 2024-09-21T15:00:06.215 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.96811956799999999, 2024-09-21T15:00:06.215 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022639701000000002, 2024-09-21T15:00:06.215 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:06.215 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:06.215 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:06.215 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:06.215 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:06.215 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 96481 2024-09-21T15:00:06.215 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.215 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:06.215 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.215 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:06.215 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200975+0000", 2024-09-21T15:00:06.215 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:06.216 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.216 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.216 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:06.216 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200975+0000", 2024-09-21T15:00:06.216 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:06.216 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.216 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.216 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:06.216 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200974+0000", 2024-09-21T15:00:06.216 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:06.216 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.216 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.216 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:06.216 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200977+0000", 2024-09-21T15:00:06.217 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.136e-06 2024-09-21T15:00:06.217 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.217 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.217 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:06.217 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200982+0000", 2024-09-21T15:00:06.217 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4129999999999997e-06 2024-09-21T15:00:06.217 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.217 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.217 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:06.217 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200987+0000", 2024-09-21T15:00:06.217 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5199999999999999e-06 2024-09-21T15:00:06.217 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.217 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.217 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:06.217 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.223420+0000", 2024-09-21T15:00:06.218 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022433611999999999 2024-09-21T15:00:06.218 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.218 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.218 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:06.218 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.223513+0000", 2024-09-21T15:00:06.218 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2477000000000002e-05 2024-09-21T15:00:06.218 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.218 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.218 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:06.218 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.223615+0000", 2024-09-21T15:00:06.218 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010186199999999999 2024-09-21T15:00:06.218 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:06.219 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:06.219 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:06.219 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.219 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.219 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:96482 3.15 3:aee50b17:::rbd_data.1136b9c49b1c.0000000000000015:49 [read 3096576~49152 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:06.219 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.200991+0000", 2024-09-21T15:00:06.219 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.96810368800000002, 2024-09-21T15:00:06.219 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023020315, 2024-09-21T15:00:06.219 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:06.219 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:06.219 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:06.219 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:06.219 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:06.219 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 96482 2024-09-21T15:00:06.220 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.220 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:06.220 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.220 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:06.220 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200991+0000", 2024-09-21T15:00:06.220 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:06.220 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.220 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.220 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:06.220 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200991+0000", 2024-09-21T15:00:06.220 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:06.220 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.220 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.220 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:06.221 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200990+0000", 2024-09-21T15:00:06.221 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:06.221 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.221 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.221 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:06.221 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200993+0000", 2024-09-21T15:00:06.221 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4119999999999999e-06 2024-09-21T15:00:06.221 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.221 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.221 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:06.221 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200998+0000", 2024-09-21T15:00:06.221 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4639999999999997e-06 2024-09-21T15:00:06.221 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.221 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.221 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:06.222 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.201003+0000", 2024-09-21T15:00:06.222 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8520000000000003e-06 2024-09-21T15:00:06.222 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.222 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.222 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:06.222 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.223660+0000", 2024-09-21T15:00:06.222 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022657295000000001 2024-09-21T15:00:06.222 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.222 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.222 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:06.222 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.223834+0000", 2024-09-21T15:00:06.222 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017377700000000001 2024-09-21T15:00:06.222 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.223 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.223 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:06.223 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.224011+0000", 2024-09-21T15:00:06.223 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017705999999999999 2024-09-21T15:00:06.223 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:06.223 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:06.223 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:06.223 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.223 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.223 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:97677 3.15 3:a8e72133:::rbd_data.1136bfc79a70.0000000000000005:head [write 0~4440027 [fadvise_sequential+fadvise_nocache],write 4751360~147456 [fadvise_sequential+fadvise_nocache],write 4964352~147456 [fadvise_sequential+fadvise_nocache],write 5177344~65536 [fadvise_sequential+fadvise_nocache],write 5259264~147456 [fadvise_sequential+fadvise_nocache],write 5472256~147456 [fadvise_sequential+fadvise_nocache],write 5685248~229376 [fadvise_sequential+fadvise_nocache],write 5980160~147456 [fadvise_sequential+fadvise_nocache],write 6193152~229376 [fadvise_sequential+fadvise_nocache],write 6488064~147456 [fadvise_sequential+fadvise_nocache],write 6701056~229376 [fadvise_sequential+fadvise_nocache],write 6995968~147456 [fadvise_sequential+fadvise_nocache],write 7208960~229376 [fadvise_sequential+fadvise_nocache],write 7503872~147456 [fadvise_sequential+fadvise_nocache],write 7716864~229376 [fadvise_sequential+fadvise_nocache],write 8044544~114688 [fadvise_sequential+fadvise_nocache],write 8257536~114688 [fadvise_sequential+fadvise_nocache],write 8388608~65536 [fadvise_sequential+fadvise_nocache],write 8552448~114688 [fadvise_sequential+fadvise_nocache],write 8765440~114688 [fadvise_sequential+fadvise_nocache],write 8929280~32768 [fadvise_sequential+fadvise_nocache],write 9142272~32768 [fadvise_sequential+fadvise_nocache],write 9355264~32768 [fadvise_sequential+fadvise_nocache],write 9437184~32768 [fadvise_sequential+fadvise_nocache],write 9650176~32768 [fadvise_sequential+fadvise_nocache],write 9863168~32768 [fadvise_sequential+fadvise_nocache],write 9961472~131072 [fadvise_sequential+fadvise_nocache],write 10108928~196608 [fadvise_sequential+fadvise_nocache],write 10321920~278528 [fadvise_sequential+fadvise_nocache],write 10616832~196608 [fadvise_sequential+fadvise_nocache],write 10829824~278528 [fadvise_sequential+fadvise_nocache],write 11124736~196608 [fadvise_sequential+fadvise_nocache],write 11337728~196608 [fadvise_sequential+fadvise_nocache],write 11550720~65536 [fadvise_sequential+fadvise_nocache],write 11649024~98304 [fadvise_sequential+fadvise_nocache],write 11763712~65536 [fadvise_sequential+fadvise_nocache],write 11862016~98304 [fadvise_sequential+fadvise_nocache],write 11976704~65536 [fadvise_sequential+fadvise_nocache],write 12058624~65536 [fadvise_sequential+fadvise_nocache],write 12156928~98304 [fadvise_sequential+fadvise_nocache],write 12271616~65536 [fadvise_sequential+fadvise_nocache],write 12369920~98304 [fadvise_sequential+fadvise_nocache],write 12484608~65536 [fadvise_sequential+fadvise_nocache],write 12664832~98304 [fadvise_sequential+fadvise_nocache],write 12877824~98304 [fadvise_sequential+fadvise_nocache],write 13090816~16384 [fadvise_sequential+fadvise_nocache],write 13172736~98304 [fadvise_sequential+fadvise_nocache],write 13385728~98304 [fadvise_sequential+fadvise_nocache],write 13598720~32768 [fadvise_sequential+fadvise_nocache],write 13680640~98304 [fadvise_sequential+fadvise_nocache],write 13893632~98304 [fadvise_sequential+fadvise_nocache],write 14106624~49152 [fadvise_sequential+fadvise_nocache],write 14794752~81920 [fadvise_sequential+fadvise_nocache],write 15007744~81920 [fadvise_sequential+fadvise_nocache],write 15302656~81920 [fadvise_sequential+fadvise_nocache],write 15515648~81920 [fadvise_sequential+fadvise_nocache],write 15810560~81920 [fadvise_sequential+fadvise_nocache],write 16023552~81920 [fadvise_sequential+fadvise_nocache],write 16236544~16384 [fadvise_sequential+fadvise_nocache],write 16318464~81920 [fadvise_sequential+fadvise_nocache],write 16531456~81920 [fadvise_sequential+fadvise_nocache],write 16744448~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:06.223 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.302946+0000", 2024-09-21T15:00:06.223 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.86614823799999996, 2024-09-21T15:00:06.223 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.091231204999999996, 2024-09-21T15:00:06.224 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:06.224 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:06.224 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:06.224 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:06.224 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:06.224 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 97677 2024-09-21T15:00:06.224 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.224 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:06.224 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.224 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:06.224 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.302946+0000", 2024-09-21T15:00:06.224 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:06.224 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.224 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.225 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:06.225 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.302946+0000", 2024-09-21T15:00:06.225 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:06.225 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.225 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.225 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:06.225 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.302942+0000", 2024-09-21T15:00:06.225 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999962 2024-09-21T15:00:06.225 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.225 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.225 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:06.225 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311253+0000", 2024-09-21T15:00:06.225 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0083105360000000003 2024-09-21T15:00:06.226 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.226 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.226 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:06.226 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311283+0000", 2024-09-21T15:00:06.226 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0065000000000001e-05 2024-09-21T15:00:06.226 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.226 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.226 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:06.226 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311296+0000", 2024-09-21T15:00:06.226 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3206e-05 2024-09-21T15:00:06.226 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.226 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.226 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:06.226 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311404+0000", 2024-09-21T15:00:06.226 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000107514 2024-09-21T15:00:06.227 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.227 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.227 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:06.227 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311795+0000", 2024-09-21T15:00:06.227 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000390913 2024-09-21T15:00:06.227 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.227 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.227 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 6", 2024-09-21T15:00:06.227 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.314686+0000", 2024-09-21T15:00:06.227 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0028915659999999999 2024-09-21T15:00:06.227 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.227 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.227 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:06.227 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.368178+0000", 2024-09-21T15:00:06.227 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.053491706 2024-09-21T15:00:06.228 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.228 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.228 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:06.228 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.393925+0000", 2024-09-21T15:00:06.228 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025747289 2024-09-21T15:00:06.228 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.228 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.228 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:06.228 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.394127+0000", 2024-09-21T15:00:06.228 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000201444 2024-09-21T15:00:06.228 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.228 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.228 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:06.228 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.394177+0000", 2024-09-21T15:00:06.229 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0689000000000002e-05 2024-09-21T15:00:06.229 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:06.229 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:06.229 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:06.229 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.229 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.229 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:97679 3.f 3:f2a362b4:::rbd_data.1136bfc79a70.0000000000000007:head [write 0~4194304 [fadvise_sequential+fadvise_nocache],write 4767744~147456 [fadvise_sequential+fadvise_nocache],write 4980736~147456 [fadvise_sequential+fadvise_nocache],write 5193728~49152 [fadvise_sequential+fadvise_nocache],write 5275648~147456 [fadvise_sequential+fadvise_nocache],write 5488640~147456 [fadvise_sequential+fadvise_nocache],write 5701632~65536 [fadvise_sequential+fadvise_nocache],write 5783552~147456 [fadvise_sequential+fadvise_nocache],write 5996544~147456 [fadvise_sequential+fadvise_nocache],write 6209536~229376 [fadvise_sequential+fadvise_nocache],write 6504448~147456 [fadvise_sequential+fadvise_nocache],write 6717440~229376 [fadvise_sequential+fadvise_nocache],write 7012352~147456 [fadvise_sequential+fadvise_nocache],write 7225344~229376 [fadvise_sequential+fadvise_nocache],write 7520256~147456 [fadvise_sequential+fadvise_nocache],write 7733248~229376 [fadvise_sequential+fadvise_nocache],write 8060928~114688 [fadvise_sequential+fadvise_nocache],write 8273920~196608 [fadvise_sequential+fadvise_nocache],write 8568832~114688 [fadvise_sequential+fadvise_nocache],write 8781824~114688 [fadvise_sequential+fadvise_nocache],write 8945664~32768 [fadvise_sequential+fadvise_nocache],write 9158656~32768 [fadvise_sequential+fadvise_nocache],write 9371648~32768 [fadvise_sequential+fadvise_nocache],write 9453568~32768 [fadvise_sequential+fadvise_nocache],write 9666560~32768 [fadvise_sequential+fadvise_nocache],write 9879552~32768 [fadvise_sequential+fadvise_nocache],write 9961472~147456 [fadvise_sequential+fadvise_nocache],write 10125312~196608 [fadvise_sequential+fadvise_nocache],write 10338304~278528 [fadvise_sequential+fadvise_nocache],write 10633216~196608 [fadvise_sequential+fadvise_nocache],write 10846208~278528 [fadvise_sequential+fadvise_nocache],write 11157504~180224 [fadvise_sequential+fadvise_nocache],write 11370496~98304 [fadvise_sequential+fadvise_nocache],write 11485184~65536 [fadvise_sequential+fadvise_nocache],write 11567104~65536 [fadvise_sequential+fadvise_nocache],write 11665408~98304 [fadvise_sequential+fadvise_nocache],write 11780096~65536 [fadvise_sequential+fadvise_nocache],write 11878400~98304 [fadvise_sequential+fadvise_nocache],write 11993088~65536 [fadvise_sequential+fadvise_nocache],write 12075008~61231 [fadvise_sequential+fadvise_nocache],write 12173312~98304 [fadvise_sequential+fadvise_nocache],write 12386304~98304 [fadvise_sequential+fadvise_nocache],write 12681216~98304 [fadvise_sequential+fadvise_nocache],write 12894208~98304 [fadvise_sequential+fadvise_nocache],write 13189120~98304 [fadvise_sequential+fadvise_nocache],write 13402112~98304 [fadvise_sequential+fadvise_nocache],write 13615104~16384 [fadvise_sequential+fadvise_nocache],write 13697024~98304 [fadvise_sequential+fadvise_nocache],write 13910016~98304 [fadvise_sequential+fadvise_nocache],write 14123008~32768 [fadvise_sequential+fadvise_nocache],write 14811136~81920 [fadvise_sequential+fadvise_nocache],write 15024128~81920 [fadvise_sequential+fadvise_nocache],write 15319040~81920 [fadvise_sequential+fadvise_nocache],write 15532032~81920 [fadvise_sequential+fadvise_nocache],write 15826944~81920 [fadvise_sequential+fadvise_nocache],write 16039936~81920 [fadvise_sequential+fadvise_nocache],write 16334848~81920 [fadvise_sequential+fadvise_nocache],write 16547840~81920 [fadvise_sequential+fadvise_nocache],write 16760832~16384 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:06.229 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.311349+0000", 2024-09-21T15:00:06.229 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.85774584099999995, 2024-09-21T15:00:06.229 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.048644222000000001, 2024-09-21T15:00:06.229 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:06.229 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:06.229 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:06.229 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:06.230 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:06.230 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 97679 2024-09-21T15:00:06.230 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.230 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:06.230 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.230 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:06.230 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311349+0000", 2024-09-21T15:00:06.230 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:06.230 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.230 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.230 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:06.230 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311349+0000", 2024-09-21T15:00:06.230 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:06.230 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.231 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.231 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:06.231 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311347+0000", 2024-09-21T15:00:06.231 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-21T15:00:06.231 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.231 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.231 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:06.231 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.322631+0000", 2024-09-21T15:00:06.231 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.011284307 2024-09-21T15:00:06.231 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.231 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.231 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:06.231 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.322676+0000", 2024-09-21T15:00:06.231 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5120000000000002e-05 2024-09-21T15:00:06.232 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.232 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.232 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:06.232 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.322694+0000", 2024-09-21T15:00:06.232 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8150000000000001e-05 2024-09-21T15:00:06.232 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.232 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.232 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:06.232 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.322781+0000", 2024-09-21T15:00:06.232 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.687e-05 2024-09-21T15:00:06.232 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.232 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.232 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:06.232 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.323108+0000", 2024-09-21T15:00:06.232 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000327363 2024-09-21T15:00:06.232 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.233 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.233 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 1", 2024-09-21T15:00:06.233 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.325174+0000", 2024-09-21T15:00:06.233 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0020656960000000001 2024-09-21T15:00:06.233 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.233 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.233 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:06.233 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.359764+0000", 2024-09-21T15:00:06.233 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034589987000000003 2024-09-21T15:00:06.233 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.233 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.233 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:06.233 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.359774+0000", 2024-09-21T15:00:06.233 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8730000000000002e-06 2024-09-21T15:00:06.233 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.234 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.234 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:06.234 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.359948+0000", 2024-09-21T15:00:06.234 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000174088 2024-09-21T15:00:06.234 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.234 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.234 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:06.234 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.359993+0000", 2024-09-21T15:00:06.234 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4706000000000002e-05 2024-09-21T15:00:06.234 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:06.234 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:06.234 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:06.234 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.234 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.235 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:97673 3.1d e122/24 3:bd91946e:::rbd_data.1136bfc79a70.0000000000000002:head v 122'162, mlcod=120'160)", 2024-09-21T15:00:06.235 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.313730+0000", 2024-09-21T15:00:06.235 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.85536391300000003, 2024-09-21T15:00:06.235 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055373462999999998, 2024-09-21T15:00:06.235 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:06.235 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:06.235 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:06.235 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.235 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:06.235 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.313730+0000", 2024-09-21T15:00:06.235 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:06.235 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.235 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.235 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:06.235 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.313730+0000", 2024-09-21T15:00:06.235 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:06.236 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.236 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.236 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:06.236 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.313730+0000", 2024-09-21T15:00:06.236 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:06.236 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.236 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.236 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:06.236 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.336113+0000", 2024-09-21T15:00:06.236 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022382962999999999 2024-09-21T15:00:06.236 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.236 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.236 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:06.236 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.336140+0000", 2024-09-21T15:00:06.236 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7248999999999999e-05 2024-09-21T15:00:06.237 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.237 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.237 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:06.237 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.336151+0000", 2024-09-21T15:00:06.237 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1015e-05 2024-09-21T15:00:06.237 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.237 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.237 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:06.237 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.355428+0000", 2024-09-21T15:00:06.237 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019276379999999999 2024-09-21T15:00:06.237 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.237 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.237 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:06.237 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.355484+0000", 2024-09-21T15:00:06.238 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6820999999999997e-05 2024-09-21T15:00:06.238 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.238 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.238 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:06.238 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.369040+0000", 2024-09-21T15:00:06.238 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013555466 2024-09-21T15:00:06.238 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.238 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.238 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:06.238 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.369104+0000", 2024-09-21T15:00:06.238 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3986999999999996e-05 2024-09-21T15:00:06.238 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:06.238 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:06.238 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:06.239 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.239 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.239 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:97684 3.15 3:aaa1da52:::rbd_data.1136b9c49b1c.0000000000000003:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:06.239 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.331547+0000", 2024-09-21T15:00:06.239 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.83754780699999998, 2024-09-21T15:00:06.239 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024267581, 2024-09-21T15:00:06.239 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:06.239 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:06.239 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:06.239 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:06.239 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:06.239 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 97684 2024-09-21T15:00:06.239 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.239 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:06.239 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.240 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:06.240 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.331547+0000", 2024-09-21T15:00:06.240 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:06.240 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.240 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.240 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:06.240 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.331547+0000", 2024-09-21T15:00:06.240 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:06.240 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.240 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.240 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:06.240 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.331544+0000", 2024-09-21T15:00:06.240 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T15:00:06.240 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.240 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.241 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:06.241 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.331553+0000", 2024-09-21T15:00:06.241 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8650000000000007e-06 2024-09-21T15:00:06.241 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.241 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.241 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:06.241 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.331571+0000", 2024-09-21T15:00:06.241 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8031000000000001e-05 2024-09-21T15:00:06.241 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.241 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.241 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:06.241 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.331582+0000", 2024-09-21T15:00:06.241 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0811e-05 2024-09-21T15:00:06.241 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.241 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.242 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:06.242 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.355617+0000", 2024-09-21T15:00:06.242 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024035117000000002 2024-09-21T15:00:06.242 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.242 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.242 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:06.242 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.355712+0000", 2024-09-21T15:00:06.242 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5297000000000006e-05 2024-09-21T15:00:06.242 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.242 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.242 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:06.242 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.355814+0000", 2024-09-21T15:00:06.242 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000101668 2024-09-21T15:00:06.243 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:06.243 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:06.243 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:06.243 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.243 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.243 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:97688 3.f 3:f6b1418f:::rbd_data.1136b9c49b1c.0000000000000007:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:06.243 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.331623+0000", 2024-09-21T15:00:06.243 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.83747180600000004, 2024-09-21T15:00:06.243 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026357163999999999, 2024-09-21T15:00:06.243 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:06.243 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:06.243 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:06.243 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:06.243 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:06.244 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 97688 2024-09-21T15:00:06.244 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.244 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:06.244 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.244 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:06.244 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.331623+0000", 2024-09-21T15:00:06.244 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:06.244 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.244 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.244 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:06.244 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.331623+0000", 2024-09-21T15:00:06.244 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:06.244 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.244 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.244 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:06.245 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.331622+0000", 2024-09-21T15:00:06.245 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:00:06.245 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.245 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.245 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:06.245 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.331627+0000", 2024-09-21T15:00:06.245 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3199999999999999e-06 2024-09-21T15:00:06.245 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.245 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.245 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:06.245 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.331636+0000", 2024-09-21T15:00:06.245 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2070000000000001e-06 2024-09-21T15:00:06.245 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.245 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.245 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:06.246 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.331642+0000", 2024-09-21T15:00:06.246 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2539999999999998e-06 2024-09-21T15:00:06.246 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.246 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.246 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:06.246 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.357765+0000", 2024-09-21T15:00:06.246 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026122777999999999 2024-09-21T15:00:06.246 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.246 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.246 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:06.246 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.357872+0000", 2024-09-21T15:00:06.246 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010690400000000001 2024-09-21T15:00:06.246 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.246 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.247 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:06.247 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.357980+0000", 2024-09-21T15:00:06.247 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010765200000000001 2024-09-21T15:00:06.247 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:06.247 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:06.247 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:06.247 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.247 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.247 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:97701 3.f 3:f043bda3:::rbd_data.1136b9c49b1c.0000000000000014:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:06.247 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.331656+0000", 2024-09-21T15:00:06.247 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.83743835, 2024-09-21T15:00:06.247 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.027952106000000001, 2024-09-21T15:00:06.247 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:06.247 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:06.247 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:06.248 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:06.248 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:06.248 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 97701 2024-09-21T15:00:06.248 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.248 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:06.248 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.248 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:06.248 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.331656+0000", 2024-09-21T15:00:06.248 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:06.248 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.248 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.248 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:06.248 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.331656+0000", 2024-09-21T15:00:06.248 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:06.249 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.249 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.249 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:06.249 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.331656+0000", 2024-09-21T15:00:06.249 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:06.249 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.249 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.249 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:06.249 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.331659+0000", 2024-09-21T15:00:06.249 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4640000000000002e-06 2024-09-21T15:00:06.249 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.249 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.249 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:06.249 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.333832+0000", 2024-09-21T15:00:06.249 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0021731820000000001 2024-09-21T15:00:06.250 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.250 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.250 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:06.250 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.333848+0000", 2024-09-21T15:00:06.250 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6107e-05 2024-09-21T15:00:06.250 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.250 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.250 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:06.250 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.358014+0000", 2024-09-21T15:00:06.250 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024165498000000001 2024-09-21T15:00:06.250 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.250 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.250 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:06.250 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.359472+0000", 2024-09-21T15:00:06.250 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001457999 2024-09-21T15:00:06.251 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.251 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.251 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:06.251 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.359608+0000", 2024-09-21T15:00:06.251 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013634899999999999 2024-09-21T15:00:06.251 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:06.251 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:06.251 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:06.251 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.251 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.251 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:97702 3.15 3:aee50b17:::rbd_data.1136b9c49b1c.0000000000000015:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:06.251 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.333871+0000", 2024-09-21T15:00:06.251 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.83522379400000002, 2024-09-21T15:00:06.251 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022283784000000001, 2024-09-21T15:00:06.252 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:06.252 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:06.252 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:06.252 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:06.252 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:06.252 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 97702 2024-09-21T15:00:06.252 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.252 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:06.252 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.252 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:06.252 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.333871+0000", 2024-09-21T15:00:06.252 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:06.252 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.252 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.252 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:06.253 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.333871+0000", 2024-09-21T15:00:06.253 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:06.253 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.253 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.253 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:06.253 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.333868+0000", 2024-09-21T15:00:06.253 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T15:00:06.253 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.253 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.253 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:06.253 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.333881+0000", 2024-09-21T15:00:06.253 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2896e-05 2024-09-21T15:00:06.253 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.253 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.253 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:06.254 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.333890+0000", 2024-09-21T15:00:06.254 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.7900000000000005e-06 2024-09-21T15:00:06.254 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.254 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.254 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:06.254 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.333896+0000", 2024-09-21T15:00:06.254 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7060000000000001e-06 2024-09-21T15:00:06.254 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.254 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.254 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:06.254 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.355845+0000", 2024-09-21T15:00:06.254 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021948834 2024-09-21T15:00:06.254 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.255 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.255 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:06.255 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.355997+0000", 2024-09-21T15:00:06.255 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015249999999999999 2024-09-21T15:00:06.255 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.255 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.255 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:06.255 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.356154+0000", 2024-09-21T15:00:06.255 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000157267 2024-09-21T15:00:06.255 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:06.255 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:06.255 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:06.255 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.255 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.256 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:97712 3.f 3:f31de823:::rbd_data.1136b9c49b1c.000000000000001f:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:06.256 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.333902+0000", 2024-09-21T15:00:06.256 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.83519225799999997, 2024-09-21T15:00:06.256 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025825147999999999, 2024-09-21T15:00:06.256 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:06.256 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:06.256 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:06.256 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:06.256 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:06.256 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 97712 2024-09-21T15:00:06.256 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.256 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:06.256 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.257 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:06.257 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.333902+0000", 2024-09-21T15:00:06.257 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:06.257 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.257 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.257 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:06.257 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.333902+0000", 2024-09-21T15:00:06.257 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:06.257 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.257 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.257 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:06.257 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.333902+0000", 2024-09-21T15:00:06.257 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:06.257 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.257 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.258 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:06.258 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.333905+0000", 2024-09-21T15:00:06.258 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6449999999999998e-06 2024-09-21T15:00:06.258 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.258 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.258 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:06.258 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.333911+0000", 2024-09-21T15:00:06.258 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9399999999999999e-06 2024-09-21T15:00:06.258 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.258 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.258 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:06.258 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.333916+0000", 2024-09-21T15:00:06.258 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8130000000000003e-06 2024-09-21T15:00:06.258 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.259 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.259 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:06.259 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.359634+0000", 2024-09-21T15:00:06.259 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025718039000000002 2024-09-21T15:00:06.259 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.259 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.259 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:06.259 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.359727+0000", 2024-09-21T15:00:06.259 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.3068999999999999e-05 2024-09-21T15:00:06.259 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:06.259 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:06.259 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:06.259 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.259 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.260 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:97718 3.15 3:afbff86b:::rbd_data.1136b9c49b1c.0000000000000025:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:06.260 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.333921+0000", 2024-09-21T15:00:06.260 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.83517327299999999, 2024-09-21T15:00:06.260 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022577750000000001, 2024-09-21T15:00:06.260 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:06.260 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:06.260 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:06.260 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:06.260 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:06.260 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 97718 2024-09-21T15:00:06.260 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.260 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:06.260 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.260 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:06.260 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.333921+0000", 2024-09-21T15:00:06.261 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:06.261 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.261 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.261 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:06.261 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.333921+0000", 2024-09-21T15:00:06.261 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:06.261 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.261 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.261 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:06.261 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.333921+0000", 2024-09-21T15:00:06.261 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:06.261 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.261 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.261 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:06.262 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.333924+0000", 2024-09-21T15:00:06.262 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9110000000000002e-06 2024-09-21T15:00:06.262 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.262 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.262 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:06.262 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.333929+0000", 2024-09-21T15:00:06.262 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5659999999999997e-06 2024-09-21T15:00:06.262 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.262 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.262 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:06.262 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.333935+0000", 2024-09-21T15:00:06.262 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9020000000000004e-06 2024-09-21T15:00:06.262 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.262 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.262 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:06.263 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.356229+0000", 2024-09-21T15:00:06.263 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022294389000000001 2024-09-21T15:00:06.263 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.263 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.263 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:06.263 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.356406+0000", 2024-09-21T15:00:06.263 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017647800000000001 2024-09-21T15:00:06.263 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.263 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.263 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:06.263 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.356499+0000", 2024-09-21T15:00:06.263 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2928000000000002e-05 2024-09-21T15:00:06.263 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:06.263 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:06.264 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:06.264 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:06.264 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:06.264 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:06.265 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T15:00:06.265 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:00:06.264+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:00:06.329 INFO:teuthology.orchestra.run.smithi032.stdout:checking clone #22, image image_client.0-clone22 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent23 2024-09-21T15:00:06.366 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:00:06.358+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:00:06.383 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:06.383 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:00:06.383 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T15:00:06.383 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:06.395 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T15:00:06.466 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:00:06.464+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:00:06.503 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:06.503 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:00:06.503 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T15:00:06.503 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T15:00:06.503 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:06.513 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T15:00:06.567 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:00:06.564+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:00:06.625 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:06.625 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T15:00:06.625 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T15:00:06.626 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T15:00:06.626 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.626 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T15:00:06.626 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.205754+0000", 2024-09-21T15:00:06.626 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 123.418041235, 2024-09-21T15:00:06.626 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.41117094199999998, 2024-09-21T15:00:06.626 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:06.626 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:06.626 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:06.626 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T15:00:06.626 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T15:00:06.626 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1 2024-09-21T15:00:06.626 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.626 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:06.627 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.627 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:06.627 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205754+0000", 2024-09-21T15:00:06.627 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:06.627 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.627 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.627 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:06.627 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205754+0000", 2024-09-21T15:00:06.627 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:06.627 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.627 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.627 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:06.627 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205752+0000", 2024-09-21T15:00:06.627 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T15:00:06.627 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.627 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.628 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:06.628 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205765+0000", 2024-09-21T15:00:06.628 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3230999999999999e-05 2024-09-21T15:00:06.628 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.628 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.628 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:06.628 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205782+0000", 2024-09-21T15:00:06.628 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6634e-05 2024-09-21T15:00:06.628 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.628 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.628 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:06.628 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205793+0000", 2024-09-21T15:00:06.628 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1297e-05 2024-09-21T15:00:06.628 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.629 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.629 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:06.629 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205867+0000", 2024-09-21T15:00:06.629 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4269000000000003e-05 2024-09-21T15:00:06.629 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.629 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.629 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:00:06.629 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205877+0000", 2024-09-21T15:00:06.629 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2690000000000005e-06 2024-09-21T15:00:06.629 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.629 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.629 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:06.629 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.613700+0000", 2024-09-21T15:00:06.630 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.407823455 2024-09-21T15:00:06.630 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.630 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.630 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:06.630 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.614203+0000", 2024-09-21T15:00:06.630 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00050311500000000005 2024-09-21T15:00:06.630 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.630 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.630 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T15:00:06.630 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.614750+0000", 2024-09-21T15:00:06.630 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00054738699999999998 2024-09-21T15:00:06.630 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.630 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.630 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:06.630 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616367+0000", 2024-09-21T15:00:06.631 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001616634 2024-09-21T15:00:06.631 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.631 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.631 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:06.631 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616574+0000", 2024-09-21T15:00:06.631 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00020668499999999999 2024-09-21T15:00:06.631 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.631 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.631 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:06.631 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616802+0000", 2024-09-21T15:00:06.631 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00022817900000000001 2024-09-21T15:00:06.631 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.631 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.631 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:06.631 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616925+0000", 2024-09-21T15:00:06.632 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012309699999999999 2024-09-21T15:00:06.632 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:06.632 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:06.632 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:06.632 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.632 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.632 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T15:00:06.632 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.955829+0000", 2024-09-21T15:00:06.632 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 122.667966709, 2024-09-21T15:00:06.632 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014919597999999999, 2024-09-21T15:00:06.632 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:06.632 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:06.632 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:06.632 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T15:00:06.632 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T15:00:06.633 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 117 2024-09-21T15:00:06.633 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.633 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:06.633 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.633 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:06.633 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955829+0000", 2024-09-21T15:00:06.633 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:06.633 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.633 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.633 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:06.633 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955829+0000", 2024-09-21T15:00:06.633 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:06.633 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.633 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.634 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:06.634 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955827+0000", 2024-09-21T15:00:06.634 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T15:00:06.634 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.634 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.634 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:06.634 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955838+0000", 2024-09-21T15:00:06.634 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.111e-05 2024-09-21T15:00:06.634 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.634 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.634 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:06.634 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955875+0000", 2024-09-21T15:00:06.634 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6211999999999998e-05 2024-09-21T15:00:06.634 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.635 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.635 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:06.635 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955899+0000", 2024-09-21T15:00:06.635 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4128999999999999e-05 2024-09-21T15:00:06.635 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.635 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.635 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:06.635 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967767+0000", 2024-09-21T15:00:06.635 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.011868251999999999 2024-09-21T15:00:06.635 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.635 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.635 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:06.636 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967866+0000", 2024-09-21T15:00:06.636 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8664000000000003e-05 2024-09-21T15:00:06.636 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.636 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.636 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T15:00:06.636 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.968128+0000", 2024-09-21T15:00:06.636 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00026243600000000001 2024-09-21T15:00:06.636 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.636 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.636 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:06.636 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.969393+0000", 2024-09-21T15:00:06.636 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0012650350000000001 2024-09-21T15:00:06.636 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.636 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.636 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:06.637 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.970523+0000", 2024-09-21T15:00:06.637 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001129557 2024-09-21T15:00:06.637 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.637 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.637 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:06.637 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.970603+0000", 2024-09-21T15:00:06.637 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0090000000000001e-05 2024-09-21T15:00:06.637 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.637 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.637 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:06.637 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.970748+0000", 2024-09-21T15:00:06.637 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014552799999999999 2024-09-21T15:00:06.637 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:06.637 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:06.637 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:06.638 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.638 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.638 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T15:00:06.638 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:04.493679+0000", 2024-09-21T15:00:06.638 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 122.130115885, 2024-09-21T15:00:06.638 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014869869000000001, 2024-09-21T15:00:06.638 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:06.638 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:06.638 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:06.638 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T15:00:06.638 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T15:00:06.638 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140 2024-09-21T15:00:06.638 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.638 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:06.638 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.639 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:06.639 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493679+0000", 2024-09-21T15:00:06.639 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:06.639 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.639 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.639 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:06.639 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493679+0000", 2024-09-21T15:00:06.639 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:06.639 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.639 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.639 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:06.639 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493679+0000", 2024-09-21T15:00:06.639 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:00:06.639 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.639 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.640 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:06.640 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493889+0000", 2024-09-21T15:00:06.640 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00021032700000000001 2024-09-21T15:00:06.640 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.640 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.640 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:06.640 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493931+0000", 2024-09-21T15:00:06.640 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1971000000000002e-05 2024-09-21T15:00:06.640 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.640 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.640 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:06.640 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493947+0000", 2024-09-21T15:00:06.640 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6385e-05 2024-09-21T15:00:06.640 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.640 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.640 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:06.641 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.505804+0000", 2024-09-21T15:00:06.641 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.011857136000000001 2024-09-21T15:00:06.641 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.641 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.641 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:06.641 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.505992+0000", 2024-09-21T15:00:06.641 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000187665 2024-09-21T15:00:06.641 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.641 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.641 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T15:00:06.641 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.506281+0000", 2024-09-21T15:00:06.641 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000288712 2024-09-21T15:00:06.641 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.642 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.642 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:06.642 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508349+0000", 2024-09-21T15:00:06.642 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0020685790000000001 2024-09-21T15:00:06.642 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.642 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.642 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:06.642 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508371+0000", 2024-09-21T15:00:06.642 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1747e-05 2024-09-21T15:00:06.642 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.642 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.642 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:06.642 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508450+0000", 2024-09-21T15:00:06.642 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9332000000000005e-05 2024-09-21T15:00:06.643 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.643 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.643 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:06.643 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508549+0000", 2024-09-21T15:00:06.643 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8918000000000004e-05 2024-09-21T15:00:06.643 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:06.643 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:06.643 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:06.643 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.643 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.643 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T15:00:06.643 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:04.493967+0000", 2024-09-21T15:00:06.643 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 122.129828848, 2024-09-21T15:00:06.643 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.015281075999999999, 2024-09-21T15:00:06.644 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:06.644 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:06.644 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:06.644 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T15:00:06.644 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T15:00:06.644 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 141 2024-09-21T15:00:06.644 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.644 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:06.644 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.644 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:06.644 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493967+0000", 2024-09-21T15:00:06.644 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:06.644 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.644 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.644 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:06.645 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493967+0000", 2024-09-21T15:00:06.645 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:06.645 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.645 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.645 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:06.645 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493965+0000", 2024-09-21T15:00:06.645 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T15:00:06.645 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.645 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.645 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:06.645 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493987+0000", 2024-09-21T15:00:06.645 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1662999999999999e-05 2024-09-21T15:00:06.645 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.645 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.645 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:06.646 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.494004+0000", 2024-09-21T15:00:06.646 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7311000000000001e-05 2024-09-21T15:00:06.646 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.646 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.646 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:06.646 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.494017+0000", 2024-09-21T15:00:06.646 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3202e-05 2024-09-21T15:00:06.646 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.646 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.646 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:06.646 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507678+0000", 2024-09-21T15:00:06.646 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013660432 2024-09-21T15:00:06.646 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.646 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.646 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:06.647 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507726+0000", 2024-09-21T15:00:06.647 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8154e-05 2024-09-21T15:00:06.647 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.647 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.647 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T15:00:06.647 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507797+0000", 2024-09-21T15:00:06.647 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.1110000000000002e-05 2024-09-21T15:00:06.647 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.647 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.647 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:06.647 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508710+0000", 2024-09-21T15:00:06.647 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00091339299999999997 2024-09-21T15:00:06.647 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.647 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.647 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:06.647 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.509108+0000", 2024-09-21T15:00:06.648 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00039718300000000001 2024-09-21T15:00:06.648 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.648 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.648 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:06.648 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.509186+0000", 2024-09-21T15:00:06.648 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8592999999999998e-05 2024-09-21T15:00:06.648 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.648 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.648 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:06.648 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.509248+0000", 2024-09-21T15:00:06.648 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1450000000000003e-05 2024-09-21T15:00:06.648 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:06.648 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:06.648 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:06.648 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.648 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.649 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6639 3.1a 3:592d80ed:::rbd_data.11361bed1f78.0000000000000005:d [read 0~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T15:00:06.649 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131650+0000", 2024-09-21T15:00:06.649 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 91.492145687000004, 2024-09-21T15:00:06.649 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016002832000000002, 2024-09-21T15:00:06.649 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:06.649 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:06.649 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:06.649 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:06.649 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:06.649 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6639 2024-09-21T15:00:06.649 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.649 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:06.649 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.649 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:06.649 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131650+0000", 2024-09-21T15:00:06.650 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:06.650 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.650 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.650 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:06.650 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131650+0000", 2024-09-21T15:00:06.650 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:06.650 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.650 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.650 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:06.650 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131649+0000", 2024-09-21T15:00:06.650 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:06.650 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.650 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.651 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:06.651 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131651+0000", 2024-09-21T15:00:06.651 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9199999999999998e-06 2024-09-21T15:00:06.651 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.651 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.651 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:06.651 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131656+0000", 2024-09-21T15:00:06.651 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9899999999999997e-06 2024-09-21T15:00:06.651 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.651 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.651 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:06.651 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131665+0000", 2024-09-21T15:00:06.651 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0999999999999993e-06 2024-09-21T15:00:06.651 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.651 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.651 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:06.652 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146309+0000", 2024-09-21T15:00:06.652 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014643893999999999 2024-09-21T15:00:06.652 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.652 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.652 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:06.652 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146392+0000", 2024-09-21T15:00:06.652 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2531e-05 2024-09-21T15:00:06.652 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.652 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.652 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:06.652 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147653+0000", 2024-09-21T15:00:06.652 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0012606710000000001 2024-09-21T15:00:06.652 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:06.652 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:06.653 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:06.653 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.653 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.653 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6640 3.1a 3:592d80ed:::rbd_data.11361bed1f78.0000000000000005:d [read 65536~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T15:00:06.653 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131670+0000", 2024-09-21T15:00:06.653 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 91.492125388000005, 2024-09-21T15:00:06.653 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016188859, 2024-09-21T15:00:06.653 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:06.653 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:06.653 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:06.653 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:06.653 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:06.653 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6640 2024-09-21T15:00:06.653 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.653 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:06.654 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.654 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:06.654 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131670+0000", 2024-09-21T15:00:06.654 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:06.654 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.654 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.654 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:06.654 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131670+0000", 2024-09-21T15:00:06.654 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:06.654 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.654 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.654 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:06.654 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131670+0000", 2024-09-21T15:00:06.655 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:00:06.655 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.655 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.655 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:06.655 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131672+0000", 2024-09-21T15:00:06.655 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.818e-06 2024-09-21T15:00:06.655 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.655 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.655 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:06.655 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131675+0000", 2024-09-21T15:00:06.655 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5930000000000002e-06 2024-09-21T15:00:06.655 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.655 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.655 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:06.655 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131678+0000", 2024-09-21T15:00:06.655 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6790000000000001e-06 2024-09-21T15:00:06.656 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.656 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.656 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:06.656 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147681+0000", 2024-09-21T15:00:06.656 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016002755 2024-09-21T15:00:06.656 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.656 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.656 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:06.656 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147762+0000", 2024-09-21T15:00:06.656 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1052999999999998e-05 2024-09-21T15:00:06.656 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.656 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.656 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:06.656 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147859+0000", 2024-09-21T15:00:06.657 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.7199000000000002e-05 2024-09-21T15:00:06.657 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:06.657 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:06.657 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:06.657 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.657 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.657 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6641 3.1a 3:592d80ed:::rbd_data.11361bed1f78.0000000000000005:d [read 147456~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T15:00:06.657 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131680+0000", 2024-09-21T15:00:06.657 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 91.492115100000007, 2024-09-21T15:00:06.657 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016431296000000001, 2024-09-21T15:00:06.657 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:06.657 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:06.657 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:06.657 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:06.657 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:06.658 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6641 2024-09-21T15:00:06.658 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.658 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:06.658 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.658 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:06.658 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131680+0000", 2024-09-21T15:00:06.658 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:06.658 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.658 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.658 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:06.658 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131680+0000", 2024-09-21T15:00:06.658 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:06.658 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.659 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.659 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:06.659 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131680+0000", 2024-09-21T15:00:06.659 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:00:06.659 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.659 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.659 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:06.659 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131681+0000", 2024-09-21T15:00:06.659 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2610000000000001e-06 2024-09-21T15:00:06.659 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.659 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.659 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:06.659 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131685+0000", 2024-09-21T15:00:06.659 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.8299999999999998e-06 2024-09-21T15:00:06.659 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.660 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.660 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:06.660 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131688+0000", 2024-09-21T15:00:06.660 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6189999999999998e-06 2024-09-21T15:00:06.660 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.660 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.660 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:06.660 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147884+0000", 2024-09-21T15:00:06.660 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016196465 2024-09-21T15:00:06.660 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.660 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.660 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:06.660 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147949+0000", 2024-09-21T15:00:06.660 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4867000000000001e-05 2024-09-21T15:00:06.660 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.661 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.661 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:06.661 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.148112+0000", 2024-09-21T15:00:06.661 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00016247700000000001 2024-09-21T15:00:06.661 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:06.661 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:06.661 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:06.661 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.661 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.661 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6642 3.1a 3:592d80ed:::rbd_data.11361bed1f78.0000000000000005:d [read 212992~32768 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T15:00:06.661 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131690+0000", 2024-09-21T15:00:06.661 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 91.492105222000006, 2024-09-21T15:00:06.661 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016751335999999999, 2024-09-21T15:00:06.661 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:06.662 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:06.662 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:06.662 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:06.662 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:06.662 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6642 2024-09-21T15:00:06.662 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.662 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:06.662 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.662 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:06.662 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131690+0000", 2024-09-21T15:00:06.662 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:06.662 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.662 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.662 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:06.663 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131690+0000", 2024-09-21T15:00:06.663 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:06.663 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.663 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.663 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:06.663 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131690+0000", 2024-09-21T15:00:06.663 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:00:06.663 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.663 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.663 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:06.663 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131691+0000", 2024-09-21T15:00:06.663 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3120000000000001e-06 2024-09-21T15:00:06.663 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.663 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.664 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:06.664 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131694+0000", 2024-09-21T15:00:06.664 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0070000000000002e-06 2024-09-21T15:00:06.664 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.664 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.664 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:06.664 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131700+0000", 2024-09-21T15:00:06.664 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0800000000000002e-06 2024-09-21T15:00:06.664 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.664 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.664 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:06.664 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.148135+0000", 2024-09-21T15:00:06.664 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016434250000000001 2024-09-21T15:00:06.665 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.665 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.665 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:06.665 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.148204+0000", 2024-09-21T15:00:06.665 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9466000000000002e-05 2024-09-21T15:00:06.665 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.665 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.665 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:06.665 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.148441+0000", 2024-09-21T15:00:06.665 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00023743700000000001 2024-09-21T15:00:06.665 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:06.665 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:06.665 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:06.665 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.665 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.666 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:61214 3.7 3:e28aec10:::rbd_header.1136862f872f:head [watch reconnect cookie 139864671932080 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-21T15:00:06.666 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:18.147172+0000", 2024-09-21T15:00:06.666 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 48.476623592000003, 2024-09-21T15:00:06.666 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.019264956, 2024-09-21T15:00:06.666 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:06.666 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:06.666 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:06.666 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:06.666 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:06.666 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 61214 2024-09-21T15:00:06.666 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.666 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:06.666 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.666 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:06.667 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147172+0000", 2024-09-21T15:00:06.667 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:06.667 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.667 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.667 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:06.667 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147172+0000", 2024-09-21T15:00:06.667 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:06.667 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.667 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.667 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:06.667 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147168+0000", 2024-09-21T15:00:06.667 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999962 2024-09-21T15:00:06.667 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.667 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.667 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:06.668 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147188+0000", 2024-09-21T15:00:06.668 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9717000000000001e-05 2024-09-21T15:00:06.668 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.668 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.668 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:06.668 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147239+0000", 2024-09-21T15:00:06.668 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1493999999999999e-05 2024-09-21T15:00:06.668 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.668 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.668 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:06.668 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147265+0000", 2024-09-21T15:00:06.668 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5746999999999999e-05 2024-09-21T15:00:06.668 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.668 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.668 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:06.668 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147372+0000", 2024-09-21T15:00:06.669 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010681200000000001 2024-09-21T15:00:06.669 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.669 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.669 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op must wait for map", 2024-09-21T15:00:06.669 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147380+0000", 2024-09-21T15:00:06.669 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7919999999999993e-06 2024-09-21T15:00:06.669 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.669 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.669 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:06.669 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.154025+0000", 2024-09-21T15:00:06.669 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0066454909999999999 2024-09-21T15:00:06.669 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.669 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.669 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:00:06.669 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.154041+0000", 2024-09-21T15:00:06.670 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5894000000000001e-05 2024-09-21T15:00:06.670 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.670 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.670 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:06.670 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.164345+0000", 2024-09-21T15:00:06.670 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0103040459999999 2024-09-21T15:00:06.670 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.670 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.670 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:06.670 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.164590+0000", 2024-09-21T15:00:06.670 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00024492699999999998 2024-09-21T15:00:06.670 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.670 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.670 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:06.670 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.166437+0000", 2024-09-21T15:00:06.671 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0018466330000000001 2024-09-21T15:00:06.671 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:06.671 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:06.671 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:06.671 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.671 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.671 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:61215 3.7 3:e28aec10:::rbd_header.1136862f872f:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-21T15:00:06.671 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:18.152008+0000", 2024-09-21T15:00:06.671 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 48.471787140000004, 2024-09-21T15:00:06.671 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.015644958, 2024-09-21T15:00:06.671 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:06.671 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:06.671 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:06.671 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:06.672 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:06.672 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 61215 2024-09-21T15:00:06.672 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.672 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:06.672 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.672 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:06.672 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152008+0000", 2024-09-21T15:00:06.672 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:06.672 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.672 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.672 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:06.672 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152008+0000", 2024-09-21T15:00:06.672 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:06.672 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.672 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.672 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:06.673 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152005+0000", 2024-09-21T15:00:06.673 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-21T15:00:06.673 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.673 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.673 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:06.673 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152025+0000", 2024-09-21T15:00:06.673 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9306000000000001e-05 2024-09-21T15:00:06.673 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.673 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.673 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:06.673 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152054+0000", 2024-09-21T15:00:06.673 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9189000000000001e-05 2024-09-21T15:00:06.673 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.673 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.673 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:06.673 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152078+0000", 2024-09-21T15:00:06.674 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4317000000000001e-05 2024-09-21T15:00:06.674 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.674 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.674 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:06.674 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152243+0000", 2024-09-21T15:00:06.674 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000164802 2024-09-21T15:00:06.674 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.674 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.674 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting_for_map not empty", 2024-09-21T15:00:06.674 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152251+0000", 2024-09-21T15:00:06.674 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0600000000000008e-06 2024-09-21T15:00:06.674 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.674 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.674 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:06.675 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.154741+0000", 2024-09-21T15:00:06.675 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0024900069999999998 2024-09-21T15:00:06.675 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.675 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.675 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:00:06.675 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.154753+0000", 2024-09-21T15:00:06.675 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1588e-05 2024-09-21T15:00:06.675 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.675 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.675 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:06.675 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.165380+0000", 2024-09-21T15:00:06.675 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0106272249999999 2024-09-21T15:00:06.675 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.675 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.675 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:06.676 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.165517+0000", 2024-09-21T15:00:06.676 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013737999999999999 2024-09-21T15:00:06.676 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.676 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.676 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 6", 2024-09-21T15:00:06.676 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.165938+0000", 2024-09-21T15:00:06.676 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00042082399999999998 2024-09-21T15:00:06.676 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.676 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.676 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:06.676 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.166795+0000", 2024-09-21T15:00:06.676 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00085674899999999997 2024-09-21T15:00:06.676 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.676 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.676 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:06.676 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.167598+0000", 2024-09-21T15:00:06.677 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00080335599999999997 2024-09-21T15:00:06.677 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.677 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.677 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:06.677 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.167632+0000", 2024-09-21T15:00:06.677 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4122000000000003e-05 2024-09-21T15:00:06.677 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.677 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.677 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:06.677 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.167653+0000", 2024-09-21T15:00:06.677 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0862000000000001e-05 2024-09-21T15:00:06.677 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:06.677 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:06.677 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:06.677 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.677 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.678 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:73698 3.10 3:0a4d95f6:::rbd_data.1136f4939602.000000000000000b:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-21T15:00:06.678 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:41.969870+0000", 2024-09-21T15:00:06.678 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 24.653925002000001, 2024-09-21T15:00:06.678 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.017502461, 2024-09-21T15:00:06.678 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:06.678 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:06.678 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:06.678 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:06.678 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:06.678 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 73698 2024-09-21T15:00:06.678 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.678 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:06.678 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.678 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:06.679 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.969870+0000", 2024-09-21T15:00:06.679 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:06.679 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.679 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.679 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:06.679 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.969870+0000", 2024-09-21T15:00:06.679 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:06.679 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.679 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.679 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:06.679 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.969868+0000", 2024-09-21T15:00:06.679 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T15:00:06.679 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.679 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.679 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:06.679 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.969885+0000", 2024-09-21T15:00:06.680 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7416999999999999e-05 2024-09-21T15:00:06.680 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.680 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.680 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:06.680 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.969918+0000", 2024-09-21T15:00:06.680 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3133000000000003e-05 2024-09-21T15:00:06.680 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.680 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.680 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:06.680 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.969934+0000", 2024-09-21T15:00:06.680 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6027000000000001e-05 2024-09-21T15:00:06.680 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.680 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.680 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:06.680 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.970021+0000", 2024-09-21T15:00:06.681 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6907e-05 2024-09-21T15:00:06.681 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.681 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.681 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:06.681 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.970148+0000", 2024-09-21T15:00:06.681 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012701499999999999 2024-09-21T15:00:06.681 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.681 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.681 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T15:00:06.681 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.972971+0000", 2024-09-21T15:00:06.681 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.002822873 2024-09-21T15:00:06.681 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.681 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.681 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:06.681 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.987124+0000", 2024-09-21T15:00:06.681 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0141529 2024-09-21T15:00:06.682 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.682 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.682 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:06.682 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.987146+0000", 2024-09-21T15:00:06.682 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1783000000000001e-05 2024-09-21T15:00:06.682 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.682 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.682 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:06.682 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.987258+0000", 2024-09-21T15:00:06.682 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000112555 2024-09-21T15:00:06.682 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.682 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.682 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:06.682 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.987373+0000", 2024-09-21T15:00:06.682 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000114434 2024-09-21T15:00:06.683 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:06.683 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:06.683 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:06.683 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.683 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.683 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:97638 3.1a 3:5e573525:::rbd_data.1136b9c49b1c.0000000000000026:49 [read 1277952~49152 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:06.683 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.276202+0000", 2024-09-21T15:00:06.683 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.347592897, 2024-09-21T15:00:06.683 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.015360357999999999, 2024-09-21T15:00:06.683 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:06.683 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:06.683 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:06.683 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:06.683 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:06.683 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 97638 2024-09-21T15:00:06.684 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.684 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:06.684 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.684 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:06.684 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276202+0000", 2024-09-21T15:00:06.684 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:06.684 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.684 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.684 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:06.684 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276202+0000", 2024-09-21T15:00:06.684 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:06.684 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.684 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.684 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:06.685 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276202+0000", 2024-09-21T15:00:06.685 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:06.685 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.685 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.685 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:06.685 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276204+0000", 2024-09-21T15:00:06.685 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5359999999999999e-06 2024-09-21T15:00:06.685 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.685 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.685 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:06.685 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276210+0000", 2024-09-21T15:00:06.685 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3040000000000004e-06 2024-09-21T15:00:06.685 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.685 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.686 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:06.686 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276214+0000", 2024-09-21T15:00:06.686 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6779999999999999e-06 2024-09-21T15:00:06.686 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.686 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.686 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:06.686 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.290563+0000", 2024-09-21T15:00:06.686 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014348346 2024-09-21T15:00:06.686 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.686 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.686 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:06.686 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.290623+0000", 2024-09-21T15:00:06.686 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0001999999999999e-05 2024-09-21T15:00:06.687 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.687 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.687 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:06.687 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.291563+0000", 2024-09-21T15:00:06.687 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00094003100000000003 2024-09-21T15:00:06.687 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:06.687 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:06.687 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:06.687 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.687 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.687 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:97640 3.1a 3:5e573525:::rbd_data.1136b9c49b1c.0000000000000026:49 [read 2162688~49152 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:06.687 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.276235+0000", 2024-09-21T15:00:06.687 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.3475603789999999, 2024-09-21T15:00:06.687 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014898288000000001, 2024-09-21T15:00:06.687 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:06.688 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:06.688 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:06.688 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:06.688 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:06.688 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 97640 2024-09-21T15:00:06.688 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.688 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:06.688 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.688 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:06.688 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276235+0000", 2024-09-21T15:00:06.688 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:06.688 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.688 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.688 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:06.688 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276235+0000", 2024-09-21T15:00:06.688 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:06.689 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.689 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.689 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:06.689 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276235+0000", 2024-09-21T15:00:06.689 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:06.689 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.689 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.689 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:06.689 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276237+0000", 2024-09-21T15:00:06.689 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4779999999999998e-06 2024-09-21T15:00:06.689 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.689 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.689 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:06.689 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276244+0000", 2024-09-21T15:00:06.689 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6950000000000003e-06 2024-09-21T15:00:06.690 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.690 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.690 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:06.690 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276249+0000", 2024-09-21T15:00:06.690 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0420000000000002e-06 2024-09-21T15:00:06.690 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.690 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.690 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:06.690 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.291013+0000", 2024-09-21T15:00:06.690 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014763705 2024-09-21T15:00:06.690 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.690 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.690 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:06.690 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.291063+0000", 2024-09-21T15:00:06.690 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0702999999999999e-05 2024-09-21T15:00:06.691 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.691 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.691 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:06.691 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.291133+0000", 2024-09-21T15:00:06.691 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9986000000000004e-05 2024-09-21T15:00:06.691 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:06.691 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:06.691 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:06.691 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.691 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.691 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:97641 3.1a 3:5e573525:::rbd_data.1136b9c49b1c.0000000000000026:49 [read 2621440~32768 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:06.691 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.276254+0000", 2024-09-21T15:00:06.691 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.3475412449999999, 2024-09-21T15:00:06.691 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.015023668, 2024-09-21T15:00:06.691 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:06.692 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:06.692 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:06.692 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:06.692 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:06.692 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 97641 2024-09-21T15:00:06.692 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.692 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:06.692 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.692 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:06.692 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276254+0000", 2024-09-21T15:00:06.692 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:06.692 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.692 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.692 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:06.692 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276254+0000", 2024-09-21T15:00:06.692 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:06.693 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.693 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.693 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:06.693 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276254+0000", 2024-09-21T15:00:06.693 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:06.693 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.693 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.693 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:06.693 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276262+0000", 2024-09-21T15:00:06.693 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3820000000000004e-06 2024-09-21T15:00:06.693 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.693 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.693 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:06.693 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276268+0000", 2024-09-21T15:00:06.693 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1240000000000002e-06 2024-09-21T15:00:06.694 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.694 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.694 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:06.694 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276273+0000", 2024-09-21T15:00:06.694 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7990000000000001e-06 2024-09-21T15:00:06.694 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.694 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.694 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:06.694 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.291163+0000", 2024-09-21T15:00:06.694 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014889576999999999 2024-09-21T15:00:06.694 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.694 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.694 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:06.695 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.291210+0000", 2024-09-21T15:00:06.695 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7479999999999999e-05 2024-09-21T15:00:06.695 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.695 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.695 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:06.695 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.291278+0000", 2024-09-21T15:00:06.695 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7735000000000001e-05 2024-09-21T15:00:06.695 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:06.695 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:06.695 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:06.695 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.695 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.695 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:97642 3.1a 3:5e573525:::rbd_data.1136b9c49b1c.0000000000000026:49 [read 3063808~32768 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:06.695 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.276277+0000", 2024-09-21T15:00:06.696 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.347518451, 2024-09-21T15:00:06.696 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.015140887, 2024-09-21T15:00:06.696 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:06.696 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:06.696 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:06.696 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:06.696 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:06.696 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 97642 2024-09-21T15:00:06.696 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.696 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:06.696 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.696 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:06.696 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276277+0000", 2024-09-21T15:00:06.696 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:06.696 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.697 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.697 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:06.697 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276277+0000", 2024-09-21T15:00:06.697 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:06.697 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.697 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.697 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:06.697 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276277+0000", 2024-09-21T15:00:06.697 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:06.697 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.697 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.697 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:06.697 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276279+0000", 2024-09-21T15:00:06.697 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5770000000000001e-06 2024-09-21T15:00:06.697 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.697 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.698 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:06.698 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276284+0000", 2024-09-21T15:00:06.698 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8910000000000003e-06 2024-09-21T15:00:06.698 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.698 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.698 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:06.698 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276288+0000", 2024-09-21T15:00:06.698 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3089999999999997e-06 2024-09-21T15:00:06.698 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.698 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.698 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:06.698 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.291306+0000", 2024-09-21T15:00:06.698 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.015017842 2024-09-21T15:00:06.698 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.698 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.699 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:06.699 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.291352+0000", 2024-09-21T15:00:06.699 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6248999999999997e-05 2024-09-21T15:00:06.699 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.699 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.699 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:06.699 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.291418+0000", 2024-09-21T15:00:06.699 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5333000000000006e-05 2024-09-21T15:00:06.699 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:06.699 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:06.699 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:06.699 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.699 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.699 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:97643 3.1a 3:5e573525:::rbd_data.1136b9c49b1c.0000000000000026:49 [read 3506176~32768 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:06.699 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.276293+0000", 2024-09-21T15:00:06.699 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.3475021890000001, 2024-09-21T15:00:06.700 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.015260792, 2024-09-21T15:00:06.700 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:06.700 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:06.700 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:06.700 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:06.700 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:06.700 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 97643 2024-09-21T15:00:06.700 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.700 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:06.700 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.700 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:06.700 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276293+0000", 2024-09-21T15:00:06.700 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:06.700 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.700 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.700 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:06.701 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276293+0000", 2024-09-21T15:00:06.701 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:06.701 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.701 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.701 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:06.701 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276293+0000", 2024-09-21T15:00:06.701 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:06.701 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.701 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.701 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:06.701 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276296+0000", 2024-09-21T15:00:06.701 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6989999999999999e-06 2024-09-21T15:00:06.701 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.701 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.701 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:06.702 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276302+0000", 2024-09-21T15:00:06.702 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9989999999999997e-06 2024-09-21T15:00:06.702 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.702 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.702 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:06.702 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276306+0000", 2024-09-21T15:00:06.702 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.831e-06 2024-09-21T15:00:06.702 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.702 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.702 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:06.702 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.291447+0000", 2024-09-21T15:00:06.702 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.015141056999999999 2024-09-21T15:00:06.702 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.702 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.702 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:06.702 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.291491+0000", 2024-09-21T15:00:06.703 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3210000000000001e-05 2024-09-21T15:00:06.703 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.703 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.703 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:06.703 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.291554+0000", 2024-09-21T15:00:06.703 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3367000000000005e-05 2024-09-21T15:00:06.703 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:06.703 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:06.703 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:06.703 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.703 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.703 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:97678 3.d e122/24 3:b2f34c6d:::rbd_data.1136bfc79a70.0000000000000006:head v 122'234, mlcod=120'232)", 2024-09-21T15:00:06.703 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.321655+0000", 2024-09-21T15:00:06.703 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.3021402229999999, 2024-09-21T15:00:06.703 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024509388999999999, 2024-09-21T15:00:06.704 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:06.704 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:06.704 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:06.704 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.704 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:06.704 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.321655+0000", 2024-09-21T15:00:06.704 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:06.704 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.704 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.704 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:06.704 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.321655+0000", 2024-09-21T15:00:06.704 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:06.704 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.704 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.704 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:06.705 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.321655+0000", 2024-09-21T15:00:06.705 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:06.705 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.705 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.705 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:06.705 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.329453+0000", 2024-09-21T15:00:06.705 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0077980560000000003 2024-09-21T15:00:06.705 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.705 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.705 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:06.705 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.329482+0000", 2024-09-21T15:00:06.705 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9153999999999999e-05 2024-09-21T15:00:06.705 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.705 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.705 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:06.705 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.329493+0000", 2024-09-21T15:00:06.706 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1103e-05 2024-09-21T15:00:06.706 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.706 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.706 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:06.706 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.329555+0000", 2024-09-21T15:00:06.706 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2345e-05 2024-09-21T15:00:06.706 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.706 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.706 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:06.706 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.329589+0000", 2024-09-21T15:00:06.706 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3908999999999997e-05 2024-09-21T15:00:06.706 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.706 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.706 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:06.706 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.346083+0000", 2024-09-21T15:00:06.707 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016493383 2024-09-21T15:00:06.707 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.707 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.707 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:06.707 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.346165+0000", 2024-09-21T15:00:06.707 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1796000000000001e-05 2024-09-21T15:00:06.707 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:06.707 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:06.707 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:06.707 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.707 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.707 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:97679 3.f e122/24 3:f2a362b4:::rbd_data.1136bfc79a70.0000000000000007:head v 122'331, mlcod=120'329)", 2024-09-21T15:00:06.707 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.337145+0000", 2024-09-21T15:00:06.707 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.286650214, 2024-09-21T15:00:06.707 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021912411, 2024-09-21T15:00:06.708 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:06.708 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:06.708 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:06.708 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.708 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:06.708 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.337145+0000", 2024-09-21T15:00:06.708 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:06.708 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.708 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.708 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:06.708 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.337145+0000", 2024-09-21T15:00:06.708 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:06.708 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.708 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.708 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:06.709 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.337145+0000", 2024-09-21T15:00:06.709 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:06.709 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.709 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.709 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:06.709 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.343899+0000", 2024-09-21T15:00:06.709 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0067541420000000003 2024-09-21T15:00:06.709 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.709 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.709 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:06.709 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.343921+0000", 2024-09-21T15:00:06.709 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2255e-05 2024-09-21T15:00:06.709 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.709 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.709 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:06.709 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.343933+0000", 2024-09-21T15:00:06.710 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1528e-05 2024-09-21T15:00:06.710 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.710 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.710 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:06.710 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.344001+0000", 2024-09-21T15:00:06.710 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8269999999999995e-05 2024-09-21T15:00:06.710 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.710 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.710 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:06.710 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.344029+0000", 2024-09-21T15:00:06.710 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8254000000000001e-05 2024-09-21T15:00:06.710 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.710 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.710 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:06.710 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.358901+0000", 2024-09-21T15:00:06.710 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0148719 2024-09-21T15:00:06.711 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.711 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.711 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:06.711 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.359058+0000", 2024-09-21T15:00:06.711 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000156446 2024-09-21T15:00:06.711 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:06.711 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:06.711 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:06.711 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.711 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.711 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:97676 3.1f e122/24 3:ffaa836a:::rbd_data.1136bfc79a70.0000000000000004:head v 122'312, mlcod=120'310)", 2024-09-21T15:00:06.711 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.348683+0000", 2024-09-21T15:00:06.711 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.2751120490000001, 2024-09-21T15:00:06.711 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.033768802000000001, 2024-09-21T15:00:06.711 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:06.712 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:06.712 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:06.712 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.712 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:06.712 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.348683+0000", 2024-09-21T15:00:06.712 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:06.712 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.712 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.712 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:06.712 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.348683+0000", 2024-09-21T15:00:06.712 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:06.712 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.712 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.712 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:06.712 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.348683+0000", 2024-09-21T15:00:06.712 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:00:06.713 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.713 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.713 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:06.713 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.366937+0000", 2024-09-21T15:00:06.713 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018254415 2024-09-21T15:00:06.713 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.713 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.713 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:06.713 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.366959+0000", 2024-09-21T15:00:06.713 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1634e-05 2024-09-21T15:00:06.713 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.713 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.713 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:06.713 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.366971+0000", 2024-09-21T15:00:06.713 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2305e-05 2024-09-21T15:00:06.713 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.714 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.714 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:06.714 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.367034+0000", 2024-09-21T15:00:06.714 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3211000000000002e-05 2024-09-21T15:00:06.714 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.714 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.714 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:06.714 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.367056+0000", 2024-09-21T15:00:06.714 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2098000000000001e-05 2024-09-21T15:00:06.714 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.714 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.714 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:06.714 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.382400+0000", 2024-09-21T15:00:06.714 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.015343867000000001 2024-09-21T15:00:06.714 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.715 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.715 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:06.715 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.382452+0000", 2024-09-21T15:00:06.715 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1992e-05 2024-09-21T15:00:06.715 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:06.715 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:06.715 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:06.715 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.715 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.715 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:97734 3.19 e122/94 3:9fcf754f:::rbd_data.1136bfc79a70.0000000000000009:head v 122'314, mlcod=120'312)", 2024-09-21T15:00:06.715 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.378547+0000", 2024-09-21T15:00:06.715 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.245248071, 2024-09-21T15:00:06.715 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.063380675999999997, 2024-09-21T15:00:06.715 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:06.715 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:06.716 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:06.716 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.716 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:06.716 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.378547+0000", 2024-09-21T15:00:06.716 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:06.716 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.716 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.716 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:06.716 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.378547+0000", 2024-09-21T15:00:06.716 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:06.716 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.716 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.716 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:06.716 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.378547+0000", 2024-09-21T15:00:06.716 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:00:06.716 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.717 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.717 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:06.717 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.425118+0000", 2024-09-21T15:00:06.717 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046571166999999997 2024-09-21T15:00:06.717 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.717 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.717 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:06.717 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.425150+0000", 2024-09-21T15:00:06.717 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2150999999999998e-05 2024-09-21T15:00:06.717 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.717 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.717 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:06.717 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.425164+0000", 2024-09-21T15:00:06.717 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3302000000000001e-05 2024-09-21T15:00:06.717 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.717 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.718 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:06.718 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.425231+0000", 2024-09-21T15:00:06.718 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7106000000000004e-05 2024-09-21T15:00:06.718 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.718 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.718 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:06.718 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.425259+0000", 2024-09-21T15:00:06.718 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8087000000000001e-05 2024-09-21T15:00:06.718 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.718 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.718 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:06.718 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.441890+0000", 2024-09-21T15:00:06.718 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016631328000000001 2024-09-21T15:00:06.718 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:06.718 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:06.718 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:06.719 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.441928+0000", 2024-09-21T15:00:06.719 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7701999999999999e-05 2024-09-21T15:00:06.719 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:06.719 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:06.719 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:06.719 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:06.719 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:06.719 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:06.719 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T15:00:06.720 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:00:06.718+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:00:06.721 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T15:00:06.821 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:00:06.818+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:00:06.880 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:06.880 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:00:06.880 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T15:00:06.880 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:06.891 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T15:00:06.922 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:00:06.920+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:00:06.930 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T15:00:06.968 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T15:00:07.003 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T15:00:07.020 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:07.020 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:07.023 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:00:07.020+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:00:07.030 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T15:00:07.032 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T15:00:07.065 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-21T15:00:07.103 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T15:00:07.123 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:00:07.120+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:00:07.142 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T15:00:07.162 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T15:00:07.165 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T15:00:07.167 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:07.167 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:07.169 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:00:07.169 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:00:07.169 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:00:07.169 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:00:07.169 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:00:07.169 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:00:07.169 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:00:07.169 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:00:07.179 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T15:00:07.224 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:00:07.218+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:00:07.324 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:00:07.324+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:00:07.333 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:07.333 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:07.343 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T15:00:07.422 INFO:teuthology.orchestra.run.smithi032.stdout:3114 writesame 0x18ec5e1 thru 0x18f1e98 data_size 0x44(0x58b8 bytes) 2024-09-21T15:00:07.425 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:00:07.424+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:00:07.436 INFO:teuthology.orchestra.run.smithi032.stdout:3116 read 0xd72ddf2 thru 0xd7343bc (0x65cb bytes) 2024-09-21T15:00:07.437 INFO:teuthology.orchestra.run.smithi032.stdout:3117 punch from 0x4a0c63c to 0x4a1714f, (0xab13 bytes) 2024-09-21T15:00:07.437 INFO:teuthology.orchestra.run.smithi032.stdout:3118 write 0x9beef4a thru 0x9bf2f4c (0x4003 bytes) 2024-09-21T15:00:07.441 INFO:teuthology.orchestra.run.smithi032.stdout:3119 write 0xa31c656 thru 0xa326d16 (0xa6c1 bytes) 2024-09-21T15:00:07.445 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:07.445 INFO:teuthology.orchestra.run.smithi032.stdout:3121 write 0x7076cdb thru 0x707b553 (0x4879 bytes) 2024-09-21T15:00:07.449 INFO:teuthology.orchestra.run.smithi032.stdout:3123 punch from 0xb5d77ce to 0xb5dfeda, (0x870c bytes) 2024-09-21T15:00:07.449 INFO:teuthology.orchestra.run.smithi032.stdout:3124 punch from 0x533b689 to 0x5342b8b, (0x7502 bytes) 2024-09-21T15:00:07.451 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:07.451 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:07.464 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T15:00:07.466 INFO:teuthology.orchestra.run.smithi032.stdout:3125 trunc from 0xea83ef8 to 0x456f327 2024-09-21T15:00:07.496 INFO:teuthology.orchestra.run.smithi032.stdout:3126 punch from 0x2b8c3bd to 0x2b984da, (0xc11d bytes) 2024-09-21T15:00:07.496 INFO:teuthology.orchestra.run.smithi032.stdout:3127 punch from 0x337c653 to 0x337e6ec, (0x2099 bytes) 2024-09-21T15:00:07.496 INFO:teuthology.orchestra.run.smithi032.stdout:3128 read 0x952f81 thru 0x9581bf (0x523f bytes) 2024-09-21T15:00:07.505 INFO:teuthology.orchestra.run.smithi032.stdout:3130 trunc from 0x456f327 to 0x9dae59b 2024-09-21T15:00:07.515 INFO:teuthology.orchestra.run.smithi032.stdout:3132 write 0x57c762d thru 0x57d7248 (0xfc1c bytes) 2024-09-21T15:00:07.526 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:00:07.524+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:00:07.527 INFO:teuthology.orchestra.run.smithi032.stdout:3133 trunc from 0x9dae59b to 0x455cd9e 2024-09-21T15:00:07.544 INFO:teuthology.orchestra.run.smithi032.stdout:3134 read 0x16bf091 thru 0x16bf43c (0x3ac bytes) 2024-09-21T15:00:07.554 INFO:teuthology.orchestra.run.smithi032.stdout:3136 trunc from 0x455cd9e to 0x8572ae1 2024-09-21T15:00:07.567 INFO:teuthology.orchestra.run.smithi032.stdout:3138 write 0x15b5155 thru 0x15bf6c0 (0xa56c bytes) 2024-09-21T15:00:07.571 INFO:teuthology.orchestra.run.smithi032.stdout:3139 trunc from 0x8572ae1 to 0x7a8f9c5 2024-09-21T15:00:07.575 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:07.575 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:07.586 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T15:00:07.588 INFO:teuthology.orchestra.run.smithi032.stdout:3141 punch from 0x66622d1 to 0x666eee0, (0xcc0f bytes) 2024-09-21T15:00:07.588 INFO:teuthology.orchestra.run.smithi032.stdout:3142 read 0x6b12582 thru 0x6b14057 (0x1ad6 bytes) 2024-09-21T15:00:07.588 INFO:teuthology.orchestra.run.smithi032.stdout:3143 read 0x22f9bb8 thru 0x23038dd (0x9d26 bytes) 2024-09-21T15:00:07.590 INFO:teuthology.orchestra.run.smithi032.stdout:3144 write 0x71bf9fc thru 0x71cdd1d (0xe322 bytes) 2024-09-21T15:00:07.604 INFO:teuthology.orchestra.run.smithi032.stdout:3146 trunc from 0x7a8f9c5 to 0x1d7bd82 2024-09-21T15:00:07.621 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:07.622 INFO:teuthology.orchestra.run.smithi032.stdout:3148 read 0x166b2d2 thru 0x1670a34 (0x5763 bytes) 2024-09-21T15:00:07.630 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:00:07.626+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:00:07.645 INFO:teuthology.orchestra.run.smithi032.stdout:3150 trunc from 0x1d7bd82 to 0xe851201 2024-09-21T15:00:07.658 INFO:teuthology.orchestra.run.smithi032.stdout:3151 writesame 0xe72ae7 thru 0xe7ba4a data_size 0x4c(0x8f64 bytes) 2024-09-21T15:00:07.662 INFO:teuthology.orchestra.run.smithi032.stdout:3152 punch from 0x29dbcd2 to 0x29e3d37, (0x8065 bytes) 2024-09-21T15:00:07.662 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:07.662 INFO:teuthology.orchestra.run.smithi032.stdout:3154 read 0x444c356 thru 0x4459c6c (0xd917 bytes) 2024-09-21T15:00:07.662 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:07.662 INFO:teuthology.orchestra.run.smithi032.stdout:3157 read 0x7e46f9c thru 0x7e53057 (0xc0bc bytes) 2024-09-21T15:00:07.663 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:07.663 INFO:teuthology.orchestra.run.smithi032.stdout:3159 clone 25 order 23 su 262144 sc 10 2024-09-21T15:00:07.703 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:07.703 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:07.713 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T15:00:07.728 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:00:07.722+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:00:07.817 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:07.817 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:07.827 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T15:00:07.829 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:00:07.828+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:00:07.931 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:00:07.928+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:00:07.981 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:07.981 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:07.991 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T15:00:08.032 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:00:08.028+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:00:08.097 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:08.098 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:08.107 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T15:00:08.132 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:00:08.132+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:00:08.163 INFO:teuthology.orchestra.run.smithi032.stderr:nodeep-scrub is set 2024-09-21T15:00:08.214 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:08.214 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:08.225 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T15:00:08.233 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:00:08.226+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:00:08.329 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:08.330 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:08.334 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:00:08.330+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:00:08.340 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T15:00:08.436 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:00:08.430+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:00:08.493 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:08.493 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:08.503 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T15:00:08.537 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:00:08.536+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:00:08.614 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:08.614 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:08.623 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T15:00:08.638 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:00:08.630+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:00:08.732 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:08.732 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:08.738 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:00:08.734+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:00:08.743 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T15:00:08.839 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:00:08.836+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:00:08.898 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:08.898 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:08.909 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T15:00:08.939 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:00:08.934+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:00:09.019 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:09.019 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:09.029 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T15:00:09.040 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:00:09.036+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:00:09.141 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:00:09.140+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:00:09.184 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:09.184 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:09.186 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T15:00:09.197 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T15:00:09.241 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:00:09.240+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:00:09.342 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:00:09.340+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:00:09.348 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T15:00:09.352 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:09.353 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:00:09.353 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T15:00:09.353 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:09.363 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T15:00:09.367 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T15:00:09.394 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T15:00:09.430 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T15:00:09.442 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:00:09.440+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:00:09.462 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-21T15:00:09.484 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:09.484 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:00:09.484 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T15:00:09.484 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T15:00:09.484 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:09.486 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T15:00:09.494 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T15:00:09.508 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T15:00:09.532 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T15:00:09.534 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:00:09.535 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:00:09.535 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:00:09.535 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:00:09.535 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:00:09.535 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:00:09.535 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:00:09.535 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:00:09.543 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:00:09.540+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:00:09.619 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:09.619 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T15:00:09.619 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T15:00:09.619 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T15:00:09.619 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.620 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:1678 3.c 3:377cc7e1:::rbd_data.1136515ecb6b.000000000000000f:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T15:00:09.620 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:30.988085+0000", 2024-09-21T15:00:09.620 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 98.628694138, 2024-09-21T15:00:09.620 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.032974802999999997, 2024-09-21T15:00:09.620 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:09.620 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:09.620 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:09.620 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:09.620 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:09.620 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1678 2024-09-21T15:00:09.621 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.621 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:09.621 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.621 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:09.621 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988085+0000", 2024-09-21T15:00:09.621 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:09.621 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.621 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.622 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:09.622 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988085+0000", 2024-09-21T15:00:09.622 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:09.622 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.622 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.622 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:09.622 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988083+0000", 2024-09-21T15:00:09.622 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-21T15:00:09.622 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.623 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.623 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:09.623 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988093+0000", 2024-09-21T15:00:09.623 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8800000000000003e-06 2024-09-21T15:00:09.623 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.623 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.623 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:09.623 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988113+0000", 2024-09-21T15:00:09.624 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0217999999999999e-05 2024-09-21T15:00:09.624 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.624 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.624 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:09.624 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988123+0000", 2024-09-21T15:00:09.624 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5859999999999994e-06 2024-09-21T15:00:09.624 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.624 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.625 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:09.625 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988203+0000", 2024-09-21T15:00:09.625 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0834000000000006e-05 2024-09-21T15:00:09.625 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.625 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.625 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:09.625 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988376+0000", 2024-09-21T15:00:09.625 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017301299999999999 2024-09-21T15:00:09.625 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.626 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.626 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 7", 2024-09-21T15:00:09.626 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.996338+0000", 2024-09-21T15:00:09.626 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0079618520000000002 2024-09-21T15:00:09.626 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.626 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.626 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:09.626 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.020881+0000", 2024-09-21T15:00:09.626 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024542731000000002 2024-09-21T15:00:09.627 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.627 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.627 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:09.627 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.020904+0000", 2024-09-21T15:00:09.627 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2861e-05 2024-09-21T15:00:09.627 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.627 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.628 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:09.628 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.020987+0000", 2024-09-21T15:00:09.628 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2806000000000006e-05 2024-09-21T15:00:09.628 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.628 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.628 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:09.628 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.021060+0000", 2024-09-21T15:00:09.628 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2871000000000001e-05 2024-09-21T15:00:09.628 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:09.629 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:09.629 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:09.629 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.629 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.629 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9403 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:09.629 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157949+0000", 2024-09-21T15:00:09.629 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 90.458829491000003, 2024-09-21T15:00:09.629 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023540373999999999, 2024-09-21T15:00:09.629 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:09.629 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:09.630 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:09.630 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:09.630 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:09.630 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9403 2024-09-21T15:00:09.630 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.630 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:09.630 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.630 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:09.630 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157949+0000", 2024-09-21T15:00:09.630 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:09.631 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.631 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.631 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:09.631 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157949+0000", 2024-09-21T15:00:09.631 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:09.631 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.631 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.631 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:09.631 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157948+0000", 2024-09-21T15:00:09.631 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T15:00:09.632 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.632 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.632 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:09.632 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157956+0000", 2024-09-21T15:00:09.632 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7999999999999999e-06 2024-09-21T15:00:09.632 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.632 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.632 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:09.632 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157969+0000", 2024-09-21T15:00:09.632 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2792000000000001e-05 2024-09-21T15:00:09.632 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.633 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.633 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:09.633 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157977+0000", 2024-09-21T15:00:09.633 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.704e-06 2024-09-21T15:00:09.633 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.633 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.633 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:09.633 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177246+0000", 2024-09-21T15:00:09.633 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019268264 2024-09-21T15:00:09.634 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.634 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.634 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:09.634 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177315+0000", 2024-09-21T15:00:09.634 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9480999999999994e-05 2024-09-21T15:00:09.634 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.634 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.634 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T15:00:09.634 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177615+0000", 2024-09-21T15:00:09.634 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00029929499999999999 2024-09-21T15:00:09.635 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.635 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.635 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:09.635 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181333+0000", 2024-09-21T15:00:09.635 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0037183149999999998 2024-09-21T15:00:09.635 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.635 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.635 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:09.635 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181374+0000", 2024-09-21T15:00:09.635 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.0723999999999998e-05 2024-09-21T15:00:09.636 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.636 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.636 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:09.636 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181434+0000", 2024-09-21T15:00:09.636 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0024e-05 2024-09-21T15:00:09.636 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.636 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.636 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:09.636 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181490+0000", 2024-09-21T15:00:09.636 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6245000000000001e-05 2024-09-21T15:00:09.637 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:09.637 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:09.637 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:09.637 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.637 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.637 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9410 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:09.637 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.174103+0000", 2024-09-21T15:00:09.637 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 90.442675733000002, 2024-09-21T15:00:09.637 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025493703, 2024-09-21T15:00:09.637 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:09.638 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:09.638 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:09.638 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:09.638 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:09.638 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9410 2024-09-21T15:00:09.638 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.638 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:09.638 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.638 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:09.638 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.174103+0000", 2024-09-21T15:00:09.639 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:09.639 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.639 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.639 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:09.639 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.174103+0000", 2024-09-21T15:00:09.639 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:09.639 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.639 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.639 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:09.639 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.174102+0000", 2024-09-21T15:00:09.640 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:00:09.640 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.640 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.640 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:09.640 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.174108+0000", 2024-09-21T15:00:09.640 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1149999999999999e-06 2024-09-21T15:00:09.640 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.640 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.640 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:09.640 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.174117+0000", 2024-09-21T15:00:09.640 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4870000000000002e-06 2024-09-21T15:00:09.641 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.641 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.641 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:09.641 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.174122+0000", 2024-09-21T15:00:09.641 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9790000000000001e-06 2024-09-21T15:00:09.641 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.641 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.641 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:09.641 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.187830+0000", 2024-09-21T15:00:09.641 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013708048 2024-09-21T15:00:09.642 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.642 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.642 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:09.642 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.187913+0000", 2024-09-21T15:00:09.642 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2763999999999996e-05 2024-09-21T15:00:09.642 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.642 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.642 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T15:00:09.642 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.188230+0000", 2024-09-21T15:00:09.642 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00031745100000000001 2024-09-21T15:00:09.642 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.643 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.643 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:09.643 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.192708+0000", 2024-09-21T15:00:09.643 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0044783890000000002 2024-09-21T15:00:09.643 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.643 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.643 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:09.643 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199487+0000", 2024-09-21T15:00:09.643 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0067782149999999998 2024-09-21T15:00:09.643 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.643 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.644 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:09.644 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199560+0000", 2024-09-21T15:00:09.644 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3752999999999997e-05 2024-09-21T15:00:09.644 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.644 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.644 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:09.644 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199597+0000", 2024-09-21T15:00:09.644 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6526999999999998e-05 2024-09-21T15:00:09.644 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:09.644 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:09.645 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:09.645 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.645 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.645 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9411 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:09.645 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.175943+0000", 2024-09-21T15:00:09.645 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 90.440835714000002, 2024-09-21T15:00:09.645 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023773181000000001, 2024-09-21T15:00:09.645 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:09.645 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:09.645 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:09.646 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:09.646 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:09.646 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9411 2024-09-21T15:00:09.646 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.646 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:09.646 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.646 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:09.646 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175943+0000", 2024-09-21T15:00:09.646 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:09.646 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.647 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.647 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:09.647 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175943+0000", 2024-09-21T15:00:09.647 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:09.647 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.647 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.647 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:09.647 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175942+0000", 2024-09-21T15:00:09.647 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:00:09.647 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.647 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.648 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:09.648 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175948+0000", 2024-09-21T15:00:09.648 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9689999999999999e-06 2024-09-21T15:00:09.648 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.648 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.648 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:09.648 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175960+0000", 2024-09-21T15:00:09.648 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1745e-05 2024-09-21T15:00:09.648 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.648 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.648 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:09.649 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175965+0000", 2024-09-21T15:00:09.649 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.203e-06 2024-09-21T15:00:09.649 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.649 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.649 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:09.649 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.189900+0000", 2024-09-21T15:00:09.649 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013934940999999999 2024-09-21T15:00:09.649 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.649 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.649 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:09.649 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.189988+0000", 2024-09-21T15:00:09.650 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8419999999999997e-05 2024-09-21T15:00:09.650 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.650 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.650 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T15:00:09.650 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.190321+0000", 2024-09-21T15:00:09.650 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00033231800000000001 2024-09-21T15:00:09.650 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.650 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.650 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:09.650 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.192746+0000", 2024-09-21T15:00:09.650 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0024254400000000001 2024-09-21T15:00:09.651 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.651 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.651 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:09.651 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199605+0000", 2024-09-21T15:00:09.651 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0068582809999999999 2024-09-21T15:00:09.651 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.651 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.651 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:09.651 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199680+0000", 2024-09-21T15:00:09.651 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5630000000000006e-05 2024-09-21T15:00:09.651 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.652 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.652 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:09.652 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199716+0000", 2024-09-21T15:00:09.652 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6248999999999998e-05 2024-09-21T15:00:09.652 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:09.652 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:09.652 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:09.652 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.652 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.653 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9412 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:09.653 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.177402+0000", 2024-09-21T15:00:09.653 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 90.439376616999994, 2024-09-21T15:00:09.653 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022445590000000001, 2024-09-21T15:00:09.653 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:09.653 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:09.653 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:09.653 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:09.653 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:09.653 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9412 2024-09-21T15:00:09.654 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.654 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:09.654 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.654 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:09.654 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177402+0000", 2024-09-21T15:00:09.654 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:09.654 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.654 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.654 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:09.655 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177402+0000", 2024-09-21T15:00:09.655 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:09.655 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.655 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.655 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:09.655 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177401+0000", 2024-09-21T15:00:09.655 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:00:09.655 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.655 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.656 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:09.656 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177409+0000", 2024-09-21T15:00:09.656 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4710000000000001e-06 2024-09-21T15:00:09.656 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.656 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.656 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:09.656 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177419+0000", 2024-09-21T15:00:09.656 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0351e-05 2024-09-21T15:00:09.656 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.656 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.656 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:09.656 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177424+0000", 2024-09-21T15:00:09.657 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1919999999999996e-06 2024-09-21T15:00:09.657 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.657 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.657 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:09.657 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.191343+0000", 2024-09-21T15:00:09.657 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.01391878 2024-09-21T15:00:09.657 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.657 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.657 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:09.657 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.191446+0000", 2024-09-21T15:00:09.657 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000102453 2024-09-21T15:00:09.658 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.658 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.658 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T15:00:09.658 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.192200+0000", 2024-09-21T15:00:09.658 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00075476899999999997 2024-09-21T15:00:09.658 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.658 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.658 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:09.658 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.193866+0000", 2024-09-21T15:00:09.659 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0016657519999999999 2024-09-21T15:00:09.659 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.659 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.659 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:09.659 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199724+0000", 2024-09-21T15:00:09.659 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0058579529999999999 2024-09-21T15:00:09.659 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.659 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.659 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:09.659 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199803+0000", 2024-09-21T15:00:09.660 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8662e-05 2024-09-21T15:00:09.660 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.660 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.660 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:09.660 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199848+0000", 2024-09-21T15:00:09.660 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5216000000000002e-05 2024-09-21T15:00:09.660 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:09.660 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:09.660 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:09.660 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.660 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.661 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:96341 3.15 3:aaa1da52:::rbd_data.1136b9c49b1c.0000000000000003:49 [read 212992~32768 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:09.661 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.200375+0000", 2024-09-21T15:00:09.661 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.4164037599999997, 2024-09-21T15:00:09.661 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026255225, 2024-09-21T15:00:09.661 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:09.661 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:09.661 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:09.661 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:09.661 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:09.661 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 96341 2024-09-21T15:00:09.661 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.662 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:09.662 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.662 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:09.662 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200375+0000", 2024-09-21T15:00:09.662 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:09.662 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.662 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.662 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:09.662 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200375+0000", 2024-09-21T15:00:09.662 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:09.663 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.663 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.663 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:09.663 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200374+0000", 2024-09-21T15:00:09.663 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T15:00:09.663 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.663 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.663 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:09.663 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200381+0000", 2024-09-21T15:00:09.663 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6950000000000003e-06 2024-09-21T15:00:09.663 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.664 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.664 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:09.664 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200390+0000", 2024-09-21T15:00:09.664 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2750000000000001e-06 2024-09-21T15:00:09.664 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.664 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.664 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:09.664 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200397+0000", 2024-09-21T15:00:09.664 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.1389999999999998e-06 2024-09-21T15:00:09.664 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.665 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.665 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:09.665 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.216693+0000", 2024-09-21T15:00:09.665 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016296076 2024-09-21T15:00:09.665 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.665 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.665 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:09.665 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.216784+0000", 2024-09-21T15:00:09.665 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0484999999999998e-05 2024-09-21T15:00:09.665 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.665 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.666 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:09.666 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.226630+0000", 2024-09-21T15:00:09.666 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0098468730000000008 2024-09-21T15:00:09.666 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:09.666 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:09.666 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:09.666 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.666 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.666 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:96478 3.15 3:aee50b17:::rbd_data.1136b9c49b1c.0000000000000015:49 [read 1327104~49152 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:09.666 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.200926+0000", 2024-09-21T15:00:09.667 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.4158524999999997, 2024-09-21T15:00:09.667 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021532795, 2024-09-21T15:00:09.667 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:09.667 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:09.667 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:09.667 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:09.667 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:09.667 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 96478 2024-09-21T15:00:09.667 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.667 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:09.667 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.667 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:09.668 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200926+0000", 2024-09-21T15:00:09.668 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:09.668 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.668 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.668 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:09.668 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200926+0000", 2024-09-21T15:00:09.668 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:09.668 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.668 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.668 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:09.668 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200926+0000", 2024-09-21T15:00:09.668 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:09.669 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.669 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.669 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:09.669 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200929+0000", 2024-09-21T15:00:09.669 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4449999999999999e-06 2024-09-21T15:00:09.669 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.669 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.669 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:09.669 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200934+0000", 2024-09-21T15:00:09.669 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2249999999999999e-06 2024-09-21T15:00:09.669 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.669 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.670 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:09.670 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200938+0000", 2024-09-21T15:00:09.670 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6739999999999996e-06 2024-09-21T15:00:09.670 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.670 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.670 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:09.670 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.222046+0000", 2024-09-21T15:00:09.670 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021107225 2024-09-21T15:00:09.670 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.670 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.670 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:09.671 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.222143+0000", 2024-09-21T15:00:09.671 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.7740999999999998e-05 2024-09-21T15:00:09.671 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.671 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.671 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:09.671 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.222459+0000", 2024-09-21T15:00:09.671 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00031578500000000001 2024-09-21T15:00:09.671 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:09.671 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:09.671 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:09.671 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.671 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.672 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:96479 3.15 3:aee50b17:::rbd_data.1136b9c49b1c.0000000000000015:49 [read 1769472~49152 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:09.672 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.200942+0000", 2024-09-21T15:00:09.672 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.415836466, 2024-09-21T15:00:09.672 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021994197, 2024-09-21T15:00:09.672 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:09.672 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:09.672 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:09.672 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:09.672 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:09.672 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 96479 2024-09-21T15:00:09.672 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.672 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:09.673 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.673 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:09.673 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200942+0000", 2024-09-21T15:00:09.673 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:09.673 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.673 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.673 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:09.673 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200942+0000", 2024-09-21T15:00:09.673 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:09.673 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.673 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.674 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:09.674 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200942+0000", 2024-09-21T15:00:09.674 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:09.674 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.674 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.674 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:09.674 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200945+0000", 2024-09-21T15:00:09.674 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3989999999999998e-06 2024-09-21T15:00:09.674 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.674 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.675 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:09.675 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200950+0000", 2024-09-21T15:00:09.675 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.631e-06 2024-09-21T15:00:09.675 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.675 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.675 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:09.675 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200955+0000", 2024-09-21T15:00:09.675 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6319999999999999e-06 2024-09-21T15:00:09.675 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.675 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.676 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:09.676 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.222484+0000", 2024-09-21T15:00:09.676 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021528988999999998 2024-09-21T15:00:09.676 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.676 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.676 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:09.676 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.222558+0000", 2024-09-21T15:00:09.676 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4239000000000005e-05 2024-09-21T15:00:09.676 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.676 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.676 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:09.676 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.222937+0000", 2024-09-21T15:00:09.677 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00037861999999999998 2024-09-21T15:00:09.677 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:09.677 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:09.677 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:09.677 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.677 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.677 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:96480 3.15 3:aee50b17:::rbd_data.1136b9c49b1c.0000000000000015:49 [read 2211840~49152 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:09.677 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.200959+0000", 2024-09-21T15:00:09.677 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.4158201740000003, 2024-09-21T15:00:09.677 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022433293, 2024-09-21T15:00:09.678 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:09.678 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:09.678 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:09.678 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:09.678 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:09.678 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 96480 2024-09-21T15:00:09.678 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.678 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:09.678 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.678 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:09.679 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200959+0000", 2024-09-21T15:00:09.679 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:09.679 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.679 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.679 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:09.679 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200959+0000", 2024-09-21T15:00:09.679 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:09.679 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.679 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.679 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:09.679 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200958+0000", 2024-09-21T15:00:09.680 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:09.680 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.680 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.680 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:09.680 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200961+0000", 2024-09-21T15:00:09.680 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3549999999999999e-06 2024-09-21T15:00:09.680 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.680 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.680 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:09.680 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200966+0000", 2024-09-21T15:00:09.681 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5029999999999997e-06 2024-09-21T15:00:09.681 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.681 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.681 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:09.681 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200971+0000", 2024-09-21T15:00:09.681 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.561e-06 2024-09-21T15:00:09.681 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.681 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.681 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:09.681 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.222963+0000", 2024-09-21T15:00:09.681 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021992088 2024-09-21T15:00:09.682 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.682 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.682 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:09.682 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.223045+0000", 2024-09-21T15:00:09.682 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2464000000000002e-05 2024-09-21T15:00:09.682 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.682 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.682 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:09.682 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.223392+0000", 2024-09-21T15:00:09.682 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00034665299999999999 2024-09-21T15:00:09.683 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:09.683 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:09.683 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:09.683 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.683 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.683 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:96481 3.15 3:aee50b17:::rbd_data.1136b9c49b1c.0000000000000015:49 [read 2654208~49152 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:09.683 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.200975+0000", 2024-09-21T15:00:09.683 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.4158041380000004, 2024-09-21T15:00:09.683 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022639701000000002, 2024-09-21T15:00:09.683 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:09.683 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:09.684 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:09.684 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:09.684 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:09.684 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 96481 2024-09-21T15:00:09.684 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.684 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:09.684 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.684 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:09.684 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200975+0000", 2024-09-21T15:00:09.684 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:09.684 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.684 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.685 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:09.685 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200975+0000", 2024-09-21T15:00:09.685 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:09.685 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.685 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.685 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:09.685 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200974+0000", 2024-09-21T15:00:09.685 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:09.685 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.685 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.685 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:09.685 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200977+0000", 2024-09-21T15:00:09.686 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.136e-06 2024-09-21T15:00:09.686 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.686 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.686 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:09.686 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200982+0000", 2024-09-21T15:00:09.686 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4129999999999997e-06 2024-09-21T15:00:09.686 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.686 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.686 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:09.686 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200987+0000", 2024-09-21T15:00:09.686 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5199999999999999e-06 2024-09-21T15:00:09.687 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.687 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.687 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:09.687 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.223420+0000", 2024-09-21T15:00:09.687 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022433611999999999 2024-09-21T15:00:09.687 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.687 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.687 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:09.687 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.223513+0000", 2024-09-21T15:00:09.687 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2477000000000002e-05 2024-09-21T15:00:09.687 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.687 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.688 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:09.688 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.223615+0000", 2024-09-21T15:00:09.688 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010186199999999999 2024-09-21T15:00:09.688 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:09.688 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:09.688 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:09.688 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.688 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.688 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:96482 3.15 3:aee50b17:::rbd_data.1136b9c49b1c.0000000000000015:49 [read 3096576~49152 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:09.688 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.200991+0000", 2024-09-21T15:00:09.688 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.4157882580000001, 2024-09-21T15:00:09.688 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023020315, 2024-09-21T15:00:09.689 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:09.689 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:09.689 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:09.689 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:09.689 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:09.689 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 96482 2024-09-21T15:00:09.689 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.689 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:09.689 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.689 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:09.689 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200991+0000", 2024-09-21T15:00:09.689 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:09.690 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.690 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.690 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:09.690 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200991+0000", 2024-09-21T15:00:09.690 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:09.690 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.690 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.690 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:09.690 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200990+0000", 2024-09-21T15:00:09.690 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:09.690 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.690 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.690 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:09.691 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200993+0000", 2024-09-21T15:00:09.691 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4119999999999999e-06 2024-09-21T15:00:09.691 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.691 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.691 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:09.691 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200998+0000", 2024-09-21T15:00:09.691 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4639999999999997e-06 2024-09-21T15:00:09.691 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.691 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.691 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:09.691 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.201003+0000", 2024-09-21T15:00:09.691 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8520000000000003e-06 2024-09-21T15:00:09.692 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.692 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.692 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:09.692 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.223660+0000", 2024-09-21T15:00:09.692 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022657295000000001 2024-09-21T15:00:09.692 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.692 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.692 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:09.692 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.223834+0000", 2024-09-21T15:00:09.692 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017377700000000001 2024-09-21T15:00:09.692 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.692 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.692 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:09.693 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.224011+0000", 2024-09-21T15:00:09.693 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017705999999999999 2024-09-21T15:00:09.693 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:09.693 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:09.693 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:09.693 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.693 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.693 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:97677 3.15 3:a8e72133:::rbd_data.1136bfc79a70.0000000000000005:head [write 0~4440027 [fadvise_sequential+fadvise_nocache],write 4751360~147456 [fadvise_sequential+fadvise_nocache],write 4964352~147456 [fadvise_sequential+fadvise_nocache],write 5177344~65536 [fadvise_sequential+fadvise_nocache],write 5259264~147456 [fadvise_sequential+fadvise_nocache],write 5472256~147456 [fadvise_sequential+fadvise_nocache],write 5685248~229376 [fadvise_sequential+fadvise_nocache],write 5980160~147456 [fadvise_sequential+fadvise_nocache],write 6193152~229376 [fadvise_sequential+fadvise_nocache],write 6488064~147456 [fadvise_sequential+fadvise_nocache],write 6701056~229376 [fadvise_sequential+fadvise_nocache],write 6995968~147456 [fadvise_sequential+fadvise_nocache],write 7208960~229376 [fadvise_sequential+fadvise_nocache],write 7503872~147456 [fadvise_sequential+fadvise_nocache],write 7716864~229376 [fadvise_sequential+fadvise_nocache],write 8044544~114688 [fadvise_sequential+fadvise_nocache],write 8257536~114688 [fadvise_sequential+fadvise_nocache],write 8388608~65536 [fadvise_sequential+fadvise_nocache],write 8552448~114688 [fadvise_sequential+fadvise_nocache],write 8765440~114688 [fadvise_sequential+fadvise_nocache],write 8929280~32768 [fadvise_sequential+fadvise_nocache],write 9142272~32768 [fadvise_sequential+fadvise_nocache],write 9355264~32768 [fadvise_sequential+fadvise_nocache],write 9437184~32768 [fadvise_sequential+fadvise_nocache],write 9650176~32768 [fadvise_sequential+fadvise_nocache],write 9863168~32768 [fadvise_sequential+fadvise_nocache],write 9961472~131072 [fadvise_sequential+fadvise_nocache],write 10108928~196608 [fadvise_sequential+fadvise_nocache],write 10321920~278528 [fadvise_sequential+fadvise_nocache],write 10616832~196608 [fadvise_sequential+fadvise_nocache],write 10829824~278528 [fadvise_sequential+fadvise_nocache],write 11124736~196608 [fadvise_sequential+fadvise_nocache],write 11337728~196608 [fadvise_sequential+fadvise_nocache],write 11550720~65536 [fadvise_sequential+fadvise_nocache],write 11649024~98304 [fadvise_sequential+fadvise_nocache],write 11763712~65536 [fadvise_sequential+fadvise_nocache],write 11862016~98304 [fadvise_sequential+fadvise_nocache],write 11976704~65536 [fadvise_sequential+fadvise_nocache],write 12058624~65536 [fadvise_sequential+fadvise_nocache],write 12156928~98304 [fadvise_sequential+fadvise_nocache],write 12271616~65536 [fadvise_sequential+fadvise_nocache],write 12369920~98304 [fadvise_sequential+fadvise_nocache],write 12484608~65536 [fadvise_sequential+fadvise_nocache],write 12664832~98304 [fadvise_sequential+fadvise_nocache],write 12877824~98304 [fadvise_sequential+fadvise_nocache],write 13090816~16384 [fadvise_sequential+fadvise_nocache],write 13172736~98304 [fadvise_sequential+fadvise_nocache],write 13385728~98304 [fadvise_sequential+fadvise_nocache],write 13598720~32768 [fadvise_sequential+fadvise_nocache],write 13680640~98304 [fadvise_sequential+fadvise_nocache],write 13893632~98304 [fadvise_sequential+fadvise_nocache],write 14106624~49152 [fadvise_sequential+fadvise_nocache],write 14794752~81920 [fadvise_sequential+fadvise_nocache],write 15007744~81920 [fadvise_sequential+fadvise_nocache],write 15302656~81920 [fadvise_sequential+fadvise_nocache],write 15515648~81920 [fadvise_sequential+fadvise_nocache],write 15810560~81920 [fadvise_sequential+fadvise_nocache],write 16023552~81920 [fadvise_sequential+fadvise_nocache],write 16236544~16384 [fadvise_sequential+fadvise_nocache],write 16318464~81920 [fadvise_sequential+fadvise_nocache],write 16531456~81920 [fadvise_sequential+fadvise_nocache],write 16744448~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:09.693 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.302946+0000", 2024-09-21T15:00:09.693 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.3138328079999999, 2024-09-21T15:00:09.693 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.091231204999999996, 2024-09-21T15:00:09.693 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:09.694 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:09.694 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:09.694 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:09.694 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:09.694 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 97677 2024-09-21T15:00:09.694 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.694 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:09.694 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.694 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:09.694 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.302946+0000", 2024-09-21T15:00:09.694 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:09.695 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.695 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.695 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:09.695 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.302946+0000", 2024-09-21T15:00:09.695 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:09.695 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.695 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.695 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:09.695 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.302942+0000", 2024-09-21T15:00:09.695 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999962 2024-09-21T15:00:09.695 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.695 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.695 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:09.696 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311253+0000", 2024-09-21T15:00:09.696 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0083105360000000003 2024-09-21T15:00:09.696 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.696 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.696 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:09.696 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311283+0000", 2024-09-21T15:00:09.696 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0065000000000001e-05 2024-09-21T15:00:09.696 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.696 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.696 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:09.696 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311296+0000", 2024-09-21T15:00:09.696 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3206e-05 2024-09-21T15:00:09.696 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.697 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.697 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:09.697 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311404+0000", 2024-09-21T15:00:09.697 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000107514 2024-09-21T15:00:09.697 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.697 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.697 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:09.697 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311795+0000", 2024-09-21T15:00:09.697 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000390913 2024-09-21T15:00:09.697 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.697 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.697 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 6", 2024-09-21T15:00:09.698 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.314686+0000", 2024-09-21T15:00:09.698 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0028915659999999999 2024-09-21T15:00:09.698 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.698 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.698 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:09.698 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.368178+0000", 2024-09-21T15:00:09.698 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.053491706 2024-09-21T15:00:09.698 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.698 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.698 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:09.698 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.393925+0000", 2024-09-21T15:00:09.698 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025747289 2024-09-21T15:00:09.698 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.699 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.699 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:09.699 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.394127+0000", 2024-09-21T15:00:09.699 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000201444 2024-09-21T15:00:09.699 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.699 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.699 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:09.699 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.394177+0000", 2024-09-21T15:00:09.699 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0689000000000002e-05 2024-09-21T15:00:09.699 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:09.699 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:09.699 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:09.699 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.700 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.700 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:97679 3.f 3:f2a362b4:::rbd_data.1136bfc79a70.0000000000000007:head [write 0~4194304 [fadvise_sequential+fadvise_nocache],write 4767744~147456 [fadvise_sequential+fadvise_nocache],write 4980736~147456 [fadvise_sequential+fadvise_nocache],write 5193728~49152 [fadvise_sequential+fadvise_nocache],write 5275648~147456 [fadvise_sequential+fadvise_nocache],write 5488640~147456 [fadvise_sequential+fadvise_nocache],write 5701632~65536 [fadvise_sequential+fadvise_nocache],write 5783552~147456 [fadvise_sequential+fadvise_nocache],write 5996544~147456 [fadvise_sequential+fadvise_nocache],write 6209536~229376 [fadvise_sequential+fadvise_nocache],write 6504448~147456 [fadvise_sequential+fadvise_nocache],write 6717440~229376 [fadvise_sequential+fadvise_nocache],write 7012352~147456 [fadvise_sequential+fadvise_nocache],write 7225344~229376 [fadvise_sequential+fadvise_nocache],write 7520256~147456 [fadvise_sequential+fadvise_nocache],write 7733248~229376 [fadvise_sequential+fadvise_nocache],write 8060928~114688 [fadvise_sequential+fadvise_nocache],write 8273920~196608 [fadvise_sequential+fadvise_nocache],write 8568832~114688 [fadvise_sequential+fadvise_nocache],write 8781824~114688 [fadvise_sequential+fadvise_nocache],write 8945664~32768 [fadvise_sequential+fadvise_nocache],write 9158656~32768 [fadvise_sequential+fadvise_nocache],write 9371648~32768 [fadvise_sequential+fadvise_nocache],write 9453568~32768 [fadvise_sequential+fadvise_nocache],write 9666560~32768 [fadvise_sequential+fadvise_nocache],write 9879552~32768 [fadvise_sequential+fadvise_nocache],write 9961472~147456 [fadvise_sequential+fadvise_nocache],write 10125312~196608 [fadvise_sequential+fadvise_nocache],write 10338304~278528 [fadvise_sequential+fadvise_nocache],write 10633216~196608 [fadvise_sequential+fadvise_nocache],write 10846208~278528 [fadvise_sequential+fadvise_nocache],write 11157504~180224 [fadvise_sequential+fadvise_nocache],write 11370496~98304 [fadvise_sequential+fadvise_nocache],write 11485184~65536 [fadvise_sequential+fadvise_nocache],write 11567104~65536 [fadvise_sequential+fadvise_nocache],write 11665408~98304 [fadvise_sequential+fadvise_nocache],write 11780096~65536 [fadvise_sequential+fadvise_nocache],write 11878400~98304 [fadvise_sequential+fadvise_nocache],write 11993088~65536 [fadvise_sequential+fadvise_nocache],write 12075008~61231 [fadvise_sequential+fadvise_nocache],write 12173312~98304 [fadvise_sequential+fadvise_nocache],write 12386304~98304 [fadvise_sequential+fadvise_nocache],write 12681216~98304 [fadvise_sequential+fadvise_nocache],write 12894208~98304 [fadvise_sequential+fadvise_nocache],write 13189120~98304 [fadvise_sequential+fadvise_nocache],write 13402112~98304 [fadvise_sequential+fadvise_nocache],write 13615104~16384 [fadvise_sequential+fadvise_nocache],write 13697024~98304 [fadvise_sequential+fadvise_nocache],write 13910016~98304 [fadvise_sequential+fadvise_nocache],write 14123008~32768 [fadvise_sequential+fadvise_nocache],write 14811136~81920 [fadvise_sequential+fadvise_nocache],write 15024128~81920 [fadvise_sequential+fadvise_nocache],write 15319040~81920 [fadvise_sequential+fadvise_nocache],write 15532032~81920 [fadvise_sequential+fadvise_nocache],write 15826944~81920 [fadvise_sequential+fadvise_nocache],write 16039936~81920 [fadvise_sequential+fadvise_nocache],write 16334848~81920 [fadvise_sequential+fadvise_nocache],write 16547840~81920 [fadvise_sequential+fadvise_nocache],write 16760832~16384 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:09.700 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.311349+0000", 2024-09-21T15:00:09.700 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.3054304109999997, 2024-09-21T15:00:09.700 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.048644222000000001, 2024-09-21T15:00:09.700 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:09.700 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:09.700 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:09.700 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:09.700 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:09.700 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 97679 2024-09-21T15:00:09.700 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.701 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:09.701 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.701 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:09.701 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311349+0000", 2024-09-21T15:00:09.701 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:09.701 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.701 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.701 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:09.701 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311349+0000", 2024-09-21T15:00:09.701 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:09.701 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.701 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.701 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:09.701 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311347+0000", 2024-09-21T15:00:09.702 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-21T15:00:09.702 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.702 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.702 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:09.702 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.322631+0000", 2024-09-21T15:00:09.702 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.011284307 2024-09-21T15:00:09.702 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.702 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.702 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:09.702 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.322676+0000", 2024-09-21T15:00:09.702 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5120000000000002e-05 2024-09-21T15:00:09.702 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.702 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.703 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:09.703 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.322694+0000", 2024-09-21T15:00:09.703 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8150000000000001e-05 2024-09-21T15:00:09.703 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.703 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.703 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:09.703 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.322781+0000", 2024-09-21T15:00:09.703 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.687e-05 2024-09-21T15:00:09.703 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.703 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.703 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:09.703 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.323108+0000", 2024-09-21T15:00:09.703 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000327363 2024-09-21T15:00:09.704 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.704 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.704 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 1", 2024-09-21T15:00:09.704 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.325174+0000", 2024-09-21T15:00:09.704 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0020656960000000001 2024-09-21T15:00:09.704 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.704 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.704 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:09.704 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.359764+0000", 2024-09-21T15:00:09.704 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034589987000000003 2024-09-21T15:00:09.704 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.704 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.704 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:09.705 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.359774+0000", 2024-09-21T15:00:09.705 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8730000000000002e-06 2024-09-21T15:00:09.705 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.705 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.705 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:09.705 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.359948+0000", 2024-09-21T15:00:09.705 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000174088 2024-09-21T15:00:09.705 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.705 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.705 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:09.705 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.359993+0000", 2024-09-21T15:00:09.705 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4706000000000002e-05 2024-09-21T15:00:09.705 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:09.706 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:09.706 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:09.706 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.706 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.706 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:97673 3.1d e122/24 3:bd91946e:::rbd_data.1136bfc79a70.0000000000000002:head v 122'162, mlcod=120'160)", 2024-09-21T15:00:09.706 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.313730+0000", 2024-09-21T15:00:09.706 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.3030484829999995, 2024-09-21T15:00:09.706 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055373462999999998, 2024-09-21T15:00:09.706 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:09.706 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:09.706 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:09.706 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.706 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:09.707 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.313730+0000", 2024-09-21T15:00:09.707 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:09.707 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.707 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.707 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:09.707 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.313730+0000", 2024-09-21T15:00:09.707 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:09.707 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.707 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.707 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:09.707 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.313730+0000", 2024-09-21T15:00:09.707 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:09.707 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.707 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.708 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:09.708 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.336113+0000", 2024-09-21T15:00:09.708 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022382962999999999 2024-09-21T15:00:09.708 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.708 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.708 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:09.708 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.336140+0000", 2024-09-21T15:00:09.708 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7248999999999999e-05 2024-09-21T15:00:09.708 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.708 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.708 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:09.708 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.336151+0000", 2024-09-21T15:00:09.708 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1015e-05 2024-09-21T15:00:09.709 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.709 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.709 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:09.709 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.355428+0000", 2024-09-21T15:00:09.709 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019276379999999999 2024-09-21T15:00:09.709 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.709 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.709 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:09.709 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.355484+0000", 2024-09-21T15:00:09.709 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6820999999999997e-05 2024-09-21T15:00:09.709 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.709 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.710 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:09.710 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.369040+0000", 2024-09-21T15:00:09.710 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013555466 2024-09-21T15:00:09.710 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.710 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.710 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:09.710 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.369104+0000", 2024-09-21T15:00:09.710 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3986999999999996e-05 2024-09-21T15:00:09.710 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:09.710 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:09.710 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:09.710 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.710 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.711 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:97684 3.15 3:aaa1da52:::rbd_data.1136b9c49b1c.0000000000000003:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:09.711 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.331547+0000", 2024-09-21T15:00:09.711 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.2852323769999998, 2024-09-21T15:00:09.711 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024267581, 2024-09-21T15:00:09.711 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:09.711 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:09.711 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:09.711 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:09.711 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:09.711 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 97684 2024-09-21T15:00:09.711 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.711 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:09.711 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.712 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:09.712 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.331547+0000", 2024-09-21T15:00:09.712 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:09.712 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.712 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.712 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:09.712 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.331547+0000", 2024-09-21T15:00:09.712 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:09.712 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.712 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.712 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:09.712 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.331544+0000", 2024-09-21T15:00:09.712 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T15:00:09.713 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.713 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.713 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:09.713 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.331553+0000", 2024-09-21T15:00:09.713 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8650000000000007e-06 2024-09-21T15:00:09.713 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.713 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.713 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:09.713 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.331571+0000", 2024-09-21T15:00:09.713 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8031000000000001e-05 2024-09-21T15:00:09.713 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.713 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.714 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:09.714 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.331582+0000", 2024-09-21T15:00:09.714 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0811e-05 2024-09-21T15:00:09.714 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.714 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.714 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:09.714 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.355617+0000", 2024-09-21T15:00:09.714 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024035117000000002 2024-09-21T15:00:09.714 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.714 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.714 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:09.715 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.355712+0000", 2024-09-21T15:00:09.715 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5297000000000006e-05 2024-09-21T15:00:09.715 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.715 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.715 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:09.715 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.355814+0000", 2024-09-21T15:00:09.715 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000101668 2024-09-21T15:00:09.715 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:09.715 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:09.715 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:09.715 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.715 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.715 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:97688 3.f 3:f6b1418f:::rbd_data.1136b9c49b1c.0000000000000007:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:09.716 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.331623+0000", 2024-09-21T15:00:09.716 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.2851563759999998, 2024-09-21T15:00:09.716 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026357163999999999, 2024-09-21T15:00:09.716 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:09.716 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:09.716 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:09.716 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:09.716 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:09.716 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 97688 2024-09-21T15:00:09.716 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.716 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:09.716 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.717 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:09.717 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.331623+0000", 2024-09-21T15:00:09.717 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:09.717 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.717 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.717 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:09.717 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.331623+0000", 2024-09-21T15:00:09.717 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:09.717 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.717 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.717 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:09.717 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.331622+0000", 2024-09-21T15:00:09.717 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:00:09.718 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.718 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.718 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:09.718 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.331627+0000", 2024-09-21T15:00:09.718 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3199999999999999e-06 2024-09-21T15:00:09.718 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.718 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.718 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:09.718 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.331636+0000", 2024-09-21T15:00:09.718 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2070000000000001e-06 2024-09-21T15:00:09.718 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.718 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.718 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:09.719 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.331642+0000", 2024-09-21T15:00:09.719 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2539999999999998e-06 2024-09-21T15:00:09.719 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.719 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.719 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:09.719 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.357765+0000", 2024-09-21T15:00:09.719 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026122777999999999 2024-09-21T15:00:09.719 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.719 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.719 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:09.719 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.357872+0000", 2024-09-21T15:00:09.719 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010690400000000001 2024-09-21T15:00:09.719 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.720 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.720 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:09.720 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.357980+0000", 2024-09-21T15:00:09.720 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010765200000000001 2024-09-21T15:00:09.720 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:09.720 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:09.720 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:09.720 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.720 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.720 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:97701 3.f 3:f043bda3:::rbd_data.1136b9c49b1c.0000000000000014:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:09.720 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.331656+0000", 2024-09-21T15:00:09.720 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.2851229200000001, 2024-09-21T15:00:09.720 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.027952106000000001, 2024-09-21T15:00:09.720 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:09.721 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:09.721 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:09.721 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:09.721 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:09.721 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 97701 2024-09-21T15:00:09.721 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.721 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:09.721 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.721 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:09.721 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.331656+0000", 2024-09-21T15:00:09.721 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:09.721 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.721 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.721 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:09.722 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.331656+0000", 2024-09-21T15:00:09.722 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:09.722 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.722 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.722 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:09.722 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.331656+0000", 2024-09-21T15:00:09.722 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:09.722 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.722 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.722 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:09.722 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.331659+0000", 2024-09-21T15:00:09.722 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4640000000000002e-06 2024-09-21T15:00:09.722 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.723 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.723 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:09.723 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.333832+0000", 2024-09-21T15:00:09.723 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0021731820000000001 2024-09-21T15:00:09.723 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.723 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.723 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:09.723 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.333848+0000", 2024-09-21T15:00:09.723 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6107e-05 2024-09-21T15:00:09.723 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.723 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.723 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:09.723 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.358014+0000", 2024-09-21T15:00:09.724 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024165498000000001 2024-09-21T15:00:09.724 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.724 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.724 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:09.724 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.359472+0000", 2024-09-21T15:00:09.724 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001457999 2024-09-21T15:00:09.724 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.724 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.724 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:09.724 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.359608+0000", 2024-09-21T15:00:09.724 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013634899999999999 2024-09-21T15:00:09.724 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:09.724 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:09.725 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:09.725 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.725 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.725 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:97702 3.15 3:aee50b17:::rbd_data.1136b9c49b1c.0000000000000015:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:09.725 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.333871+0000", 2024-09-21T15:00:09.725 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.2829083639999999, 2024-09-21T15:00:09.725 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022283784000000001, 2024-09-21T15:00:09.725 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:09.725 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:09.725 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:09.725 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:09.725 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:09.725 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 97702 2024-09-21T15:00:09.725 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.726 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:09.726 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.726 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:09.726 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.333871+0000", 2024-09-21T15:00:09.726 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:09.726 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.726 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.726 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:09.726 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.333871+0000", 2024-09-21T15:00:09.726 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:09.726 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.726 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.726 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:09.727 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.333868+0000", 2024-09-21T15:00:09.727 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T15:00:09.727 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.727 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.727 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:09.727 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.333881+0000", 2024-09-21T15:00:09.727 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2896e-05 2024-09-21T15:00:09.727 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.727 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.727 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:09.727 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.333890+0000", 2024-09-21T15:00:09.727 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.7900000000000005e-06 2024-09-21T15:00:09.727 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.727 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.728 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:09.728 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.333896+0000", 2024-09-21T15:00:09.728 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7060000000000001e-06 2024-09-21T15:00:09.728 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.728 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.728 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:09.728 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.355845+0000", 2024-09-21T15:00:09.728 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021948834 2024-09-21T15:00:09.728 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.728 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.728 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:09.728 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.355997+0000", 2024-09-21T15:00:09.728 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015249999999999999 2024-09-21T15:00:09.728 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.729 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.729 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:09.729 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.356154+0000", 2024-09-21T15:00:09.729 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000157267 2024-09-21T15:00:09.729 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:09.729 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:09.729 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:09.729 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.729 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.729 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:97712 3.f 3:f31de823:::rbd_data.1136b9c49b1c.000000000000001f:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:09.729 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.333902+0000", 2024-09-21T15:00:09.729 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.282876828, 2024-09-21T15:00:09.729 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025825147999999999, 2024-09-21T15:00:09.730 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:09.730 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:09.730 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:09.730 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:09.730 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:09.730 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 97712 2024-09-21T15:00:09.730 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.730 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:09.730 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.730 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:09.730 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.333902+0000", 2024-09-21T15:00:09.730 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:09.730 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.731 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.731 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:09.731 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.333902+0000", 2024-09-21T15:00:09.731 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:09.731 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.731 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.731 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:09.731 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.333902+0000", 2024-09-21T15:00:09.731 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:09.731 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.731 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.731 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:09.731 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.333905+0000", 2024-09-21T15:00:09.731 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6449999999999998e-06 2024-09-21T15:00:09.732 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.732 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.732 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:09.732 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.333911+0000", 2024-09-21T15:00:09.732 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9399999999999999e-06 2024-09-21T15:00:09.732 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.732 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.732 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:09.732 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.333916+0000", 2024-09-21T15:00:09.732 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8130000000000003e-06 2024-09-21T15:00:09.732 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.732 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.732 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:09.732 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.359634+0000", 2024-09-21T15:00:09.733 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025718039000000002 2024-09-21T15:00:09.733 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.733 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.733 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:09.733 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.359727+0000", 2024-09-21T15:00:09.733 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.3068999999999999e-05 2024-09-21T15:00:09.733 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:09.733 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:09.733 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:09.733 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.733 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.733 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:97718 3.15 3:afbff86b:::rbd_data.1136b9c49b1c.0000000000000025:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:09.733 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.333921+0000", 2024-09-21T15:00:09.733 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.2828578430000004, 2024-09-21T15:00:09.734 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022577750000000001, 2024-09-21T15:00:09.734 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:09.734 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:09.734 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:09.734 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:09.734 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:09.734 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 97718 2024-09-21T15:00:09.734 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.734 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:09.734 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.734 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:09.734 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.333921+0000", 2024-09-21T15:00:09.735 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:09.735 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.735 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.735 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:09.735 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.333921+0000", 2024-09-21T15:00:09.735 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:09.735 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.735 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.735 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:09.735 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.333921+0000", 2024-09-21T15:00:09.735 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:09.735 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.735 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.735 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:09.736 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.333924+0000", 2024-09-21T15:00:09.736 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9110000000000002e-06 2024-09-21T15:00:09.736 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.736 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.736 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:09.736 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.333929+0000", 2024-09-21T15:00:09.736 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5659999999999997e-06 2024-09-21T15:00:09.736 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.736 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.736 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:09.736 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.333935+0000", 2024-09-21T15:00:09.736 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9020000000000004e-06 2024-09-21T15:00:09.736 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.736 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.736 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:09.737 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.356229+0000", 2024-09-21T15:00:09.737 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022294389000000001 2024-09-21T15:00:09.737 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.737 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.737 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:09.737 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.356406+0000", 2024-09-21T15:00:09.737 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017647800000000001 2024-09-21T15:00:09.737 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:09.737 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:09.737 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:09.737 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.356499+0000", 2024-09-21T15:00:09.737 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2928000000000002e-05 2024-09-21T15:00:09.737 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:09.737 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:09.738 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:09.738 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:09.738 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:09.738 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:09.739 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T15:00:09.739 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:00:09.736+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:00:09.840 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:00:09.836+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:00:09.854 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:09.854 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:00:09.854 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T15:00:09.854 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:09.864 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T15:00:09.940 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:00:09.938+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:00:10.015 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:10.016 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:00:10.016 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T15:00:10.016 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T15:00:10.016 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:10.026 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T15:00:10.041 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:00:10.034+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:00:10.140 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:10.141 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T15:00:10.141 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T15:00:10.141 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T15:00:10.141 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.141 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T15:00:10.141 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.205754+0000", 2024-09-21T15:00:10.141 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 126.931709637, 2024-09-21T15:00:10.141 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.41117094199999998, 2024-09-21T15:00:10.141 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:10.141 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:10.141 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:10.141 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T15:00:10.141 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T15:00:10.142 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1 2024-09-21T15:00:10.142 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.142 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:10.142 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.142 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:10.142 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205754+0000", 2024-09-21T15:00:10.142 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:10.142 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.142 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.142 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:10.142 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205754+0000", 2024-09-21T15:00:10.142 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:10.143 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.143 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.143 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:10.143 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205752+0000", 2024-09-21T15:00:10.143 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T15:00:10.143 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.143 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.143 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:10.143 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205765+0000", 2024-09-21T15:00:10.143 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3230999999999999e-05 2024-09-21T15:00:10.143 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.143 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.143 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:10.144 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205782+0000", 2024-09-21T15:00:10.144 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6634e-05 2024-09-21T15:00:10.144 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.144 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.144 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:10.144 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205793+0000", 2024-09-21T15:00:10.144 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1297e-05 2024-09-21T15:00:10.144 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.144 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.144 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:10.144 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205867+0000", 2024-09-21T15:00:10.144 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4269000000000003e-05 2024-09-21T15:00:10.145 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.145 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.145 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:00:10.145 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205877+0000", 2024-09-21T15:00:10.145 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2690000000000005e-06 2024-09-21T15:00:10.145 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.145 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.145 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:10.145 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.613700+0000", 2024-09-21T15:00:10.145 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.407823455 2024-09-21T15:00:10.145 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.145 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.145 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:10.145 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.614203+0000", 2024-09-21T15:00:10.146 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00050311500000000005 2024-09-21T15:00:10.146 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.146 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.146 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T15:00:10.146 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.614750+0000", 2024-09-21T15:00:10.146 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00054738699999999998 2024-09-21T15:00:10.146 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.146 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.146 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:10.146 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616367+0000", 2024-09-21T15:00:10.146 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001616634 2024-09-21T15:00:10.146 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.146 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.147 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:10.147 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616574+0000", 2024-09-21T15:00:10.147 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00020668499999999999 2024-09-21T15:00:10.147 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.147 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.147 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:10.147 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616802+0000", 2024-09-21T15:00:10.147 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00022817900000000001 2024-09-21T15:00:10.147 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.147 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.147 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:10.147 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616925+0000", 2024-09-21T15:00:10.147 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012309699999999999 2024-09-21T15:00:10.148 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:10.148 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:10.148 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:10.148 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.148 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.148 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T15:00:10.148 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.955829+0000", 2024-09-21T15:00:10.148 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 126.18163511100001, 2024-09-21T15:00:10.148 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014919597999999999, 2024-09-21T15:00:10.148 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:10.148 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:10.148 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:10.148 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T15:00:10.149 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T15:00:10.149 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 117 2024-09-21T15:00:10.149 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.149 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:10.149 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.149 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:10.149 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955829+0000", 2024-09-21T15:00:10.149 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:10.149 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.149 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.149 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:10.149 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955829+0000", 2024-09-21T15:00:10.149 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:10.150 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.150 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.150 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:10.150 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955827+0000", 2024-09-21T15:00:10.150 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T15:00:10.150 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.150 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.150 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:10.150 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955838+0000", 2024-09-21T15:00:10.150 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.111e-05 2024-09-21T15:00:10.150 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.150 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.150 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:10.151 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955875+0000", 2024-09-21T15:00:10.151 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6211999999999998e-05 2024-09-21T15:00:10.151 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.151 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.151 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:10.151 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.955899+0000", 2024-09-21T15:00:10.151 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4128999999999999e-05 2024-09-21T15:00:10.151 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.151 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.151 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:10.151 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967767+0000", 2024-09-21T15:00:10.152 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.011868251999999999 2024-09-21T15:00:10.152 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.152 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.152 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:10.152 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.967866+0000", 2024-09-21T15:00:10.152 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8664000000000003e-05 2024-09-21T15:00:10.152 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.152 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.152 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T15:00:10.152 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.968128+0000", 2024-09-21T15:00:10.152 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00026243600000000001 2024-09-21T15:00:10.152 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.153 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.153 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:10.153 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.969393+0000", 2024-09-21T15:00:10.153 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0012650350000000001 2024-09-21T15:00:10.153 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.153 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.153 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:10.153 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.970523+0000", 2024-09-21T15:00:10.153 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001129557 2024-09-21T15:00:10.153 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.153 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.153 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:10.154 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.970603+0000", 2024-09-21T15:00:10.154 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0090000000000001e-05 2024-09-21T15:00:10.154 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.154 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.154 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:10.154 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.970748+0000", 2024-09-21T15:00:10.154 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014552799999999999 2024-09-21T15:00:10.154 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:10.154 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:10.154 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:10.154 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.154 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.155 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T15:00:10.155 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:04.493679+0000", 2024-09-21T15:00:10.155 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 125.643784287, 2024-09-21T15:00:10.155 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014869869000000001, 2024-09-21T15:00:10.155 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:10.155 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:10.155 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:10.155 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T15:00:10.155 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T15:00:10.155 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140 2024-09-21T15:00:10.155 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.155 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:10.156 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.156 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:10.156 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493679+0000", 2024-09-21T15:00:10.156 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:10.156 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.156 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.156 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:10.156 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493679+0000", 2024-09-21T15:00:10.156 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:10.156 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.156 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.156 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:10.156 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493679+0000", 2024-09-21T15:00:10.157 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:00:10.157 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.157 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.157 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:10.157 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493889+0000", 2024-09-21T15:00:10.157 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00021032700000000001 2024-09-21T15:00:10.157 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.157 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.157 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:10.157 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493931+0000", 2024-09-21T15:00:10.157 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1971000000000002e-05 2024-09-21T15:00:10.157 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.157 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.158 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:10.158 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493947+0000", 2024-09-21T15:00:10.158 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6385e-05 2024-09-21T15:00:10.158 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.158 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.158 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:10.158 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.505804+0000", 2024-09-21T15:00:10.158 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.011857136000000001 2024-09-21T15:00:10.158 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.158 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.158 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:10.158 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.505992+0000", 2024-09-21T15:00:10.158 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000187665 2024-09-21T15:00:10.159 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.159 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.159 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T15:00:10.159 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.506281+0000", 2024-09-21T15:00:10.159 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000288712 2024-09-21T15:00:10.159 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.159 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.159 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:10.159 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508349+0000", 2024-09-21T15:00:10.159 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0020685790000000001 2024-09-21T15:00:10.159 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.159 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.159 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:10.159 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508371+0000", 2024-09-21T15:00:10.160 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1747e-05 2024-09-21T15:00:10.160 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.160 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.160 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:10.160 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508450+0000", 2024-09-21T15:00:10.160 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9332000000000005e-05 2024-09-21T15:00:10.160 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.160 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.160 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:10.160 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508549+0000", 2024-09-21T15:00:10.160 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8918000000000004e-05 2024-09-21T15:00:10.160 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:10.161 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:10.161 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:10.161 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.161 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.161 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T15:00:10.161 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:04.493967+0000", 2024-09-21T15:00:10.161 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 125.64349725, 2024-09-21T15:00:10.161 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.015281075999999999, 2024-09-21T15:00:10.161 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:10.161 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:10.161 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:10.161 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T15:00:10.161 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T15:00:10.162 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 141 2024-09-21T15:00:10.162 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.162 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:10.162 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.162 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:10.162 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493967+0000", 2024-09-21T15:00:10.162 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:10.162 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.162 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.162 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:10.162 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493967+0000", 2024-09-21T15:00:10.162 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:10.162 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.162 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.163 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:10.163 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493965+0000", 2024-09-21T15:00:10.163 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T15:00:10.163 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.163 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.163 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:10.163 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493987+0000", 2024-09-21T15:00:10.163 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1662999999999999e-05 2024-09-21T15:00:10.163 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.163 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.163 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:10.163 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.494004+0000", 2024-09-21T15:00:10.163 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7311000000000001e-05 2024-09-21T15:00:10.163 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.164 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.164 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:10.164 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.494017+0000", 2024-09-21T15:00:10.164 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3202e-05 2024-09-21T15:00:10.164 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.164 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.164 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:10.164 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507678+0000", 2024-09-21T15:00:10.164 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013660432 2024-09-21T15:00:10.164 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.164 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.164 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:10.165 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507726+0000", 2024-09-21T15:00:10.165 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8154e-05 2024-09-21T15:00:10.165 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.165 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.165 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T15:00:10.165 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507797+0000", 2024-09-21T15:00:10.165 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.1110000000000002e-05 2024-09-21T15:00:10.165 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.165 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.165 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:10.165 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508710+0000", 2024-09-21T15:00:10.165 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00091339299999999997 2024-09-21T15:00:10.165 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.165 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.166 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:10.166 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.509108+0000", 2024-09-21T15:00:10.166 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00039718300000000001 2024-09-21T15:00:10.166 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.166 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.166 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:10.166 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.509186+0000", 2024-09-21T15:00:10.166 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8592999999999998e-05 2024-09-21T15:00:10.166 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.166 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.166 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:10.166 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.509248+0000", 2024-09-21T15:00:10.166 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1450000000000003e-05 2024-09-21T15:00:10.166 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:10.167 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:10.167 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:10.167 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.167 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.167 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6639 3.1a 3:592d80ed:::rbd_data.11361bed1f78.0000000000000005:d [read 0~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T15:00:10.167 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131650+0000", 2024-09-21T15:00:10.167 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 95.005814088999998, 2024-09-21T15:00:10.167 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016002832000000002, 2024-09-21T15:00:10.167 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:10.167 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:10.167 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:10.167 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:10.167 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:10.167 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6639 2024-09-21T15:00:10.168 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.168 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:10.168 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.168 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:10.168 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131650+0000", 2024-09-21T15:00:10.168 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:10.168 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.168 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.168 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:10.168 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131650+0000", 2024-09-21T15:00:10.168 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:10.168 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.168 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.169 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:10.169 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131649+0000", 2024-09-21T15:00:10.169 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:10.169 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.169 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.169 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:10.169 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131651+0000", 2024-09-21T15:00:10.169 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9199999999999998e-06 2024-09-21T15:00:10.169 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.169 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.169 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:10.169 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131656+0000", 2024-09-21T15:00:10.169 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9899999999999997e-06 2024-09-21T15:00:10.169 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.170 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.170 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:10.170 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131665+0000", 2024-09-21T15:00:10.170 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0999999999999993e-06 2024-09-21T15:00:10.170 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.170 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.170 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:10.170 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146309+0000", 2024-09-21T15:00:10.170 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014643893999999999 2024-09-21T15:00:10.170 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.170 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.170 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:10.170 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146392+0000", 2024-09-21T15:00:10.171 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2531e-05 2024-09-21T15:00:10.171 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.171 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.171 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:10.171 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147653+0000", 2024-09-21T15:00:10.171 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0012606710000000001 2024-09-21T15:00:10.171 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:10.171 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:10.171 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:10.171 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.171 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.171 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6640 3.1a 3:592d80ed:::rbd_data.11361bed1f78.0000000000000005:d [read 65536~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T15:00:10.171 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131670+0000", 2024-09-21T15:00:10.172 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 95.005793789999998, 2024-09-21T15:00:10.172 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016188859, 2024-09-21T15:00:10.172 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:10.172 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:10.172 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:10.172 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:10.172 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:10.172 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6640 2024-09-21T15:00:10.172 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.172 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:10.172 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.172 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:10.172 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131670+0000", 2024-09-21T15:00:10.172 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:10.173 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.173 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.173 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:10.173 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131670+0000", 2024-09-21T15:00:10.173 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:10.173 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.173 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.173 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:10.173 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131670+0000", 2024-09-21T15:00:10.173 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:00:10.173 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.173 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.173 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:10.173 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131672+0000", 2024-09-21T15:00:10.174 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.818e-06 2024-09-21T15:00:10.174 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.174 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.174 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:10.174 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131675+0000", 2024-09-21T15:00:10.174 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5930000000000002e-06 2024-09-21T15:00:10.174 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.174 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.174 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:10.174 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131678+0000", 2024-09-21T15:00:10.174 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6790000000000001e-06 2024-09-21T15:00:10.174 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.175 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.175 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:10.175 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147681+0000", 2024-09-21T15:00:10.175 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016002755 2024-09-21T15:00:10.175 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.175 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.175 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:10.175 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147762+0000", 2024-09-21T15:00:10.175 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1052999999999998e-05 2024-09-21T15:00:10.175 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.175 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.175 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:10.175 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147859+0000", 2024-09-21T15:00:10.175 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.7199000000000002e-05 2024-09-21T15:00:10.176 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:10.176 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:10.176 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:10.176 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.176 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.176 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6641 3.1a 3:592d80ed:::rbd_data.11361bed1f78.0000000000000005:d [read 147456~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T15:00:10.176 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131680+0000", 2024-09-21T15:00:10.176 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 95.005783502, 2024-09-21T15:00:10.176 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016431296000000001, 2024-09-21T15:00:10.176 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:10.176 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:10.176 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:10.176 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:10.176 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:10.177 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6641 2024-09-21T15:00:10.177 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.177 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:10.177 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.177 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:10.177 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131680+0000", 2024-09-21T15:00:10.177 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:10.177 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.177 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.177 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:10.177 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131680+0000", 2024-09-21T15:00:10.177 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:10.177 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.178 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.178 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:10.178 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131680+0000", 2024-09-21T15:00:10.178 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:00:10.178 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.178 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.178 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:10.178 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131681+0000", 2024-09-21T15:00:10.178 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2610000000000001e-06 2024-09-21T15:00:10.178 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.178 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.178 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:10.178 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131685+0000", 2024-09-21T15:00:10.179 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.8299999999999998e-06 2024-09-21T15:00:10.179 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.179 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.179 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:10.179 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131688+0000", 2024-09-21T15:00:10.179 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6189999999999998e-06 2024-09-21T15:00:10.179 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.179 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.179 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:10.179 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147884+0000", 2024-09-21T15:00:10.179 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016196465 2024-09-21T15:00:10.179 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.180 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.180 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:10.180 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147949+0000", 2024-09-21T15:00:10.180 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4867000000000001e-05 2024-09-21T15:00:10.180 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.180 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.180 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:10.180 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.148112+0000", 2024-09-21T15:00:10.180 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00016247700000000001 2024-09-21T15:00:10.180 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:10.180 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:10.180 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:10.180 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.180 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.181 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6642 3.1a 3:592d80ed:::rbd_data.11361bed1f78.0000000000000005:d [read 212992~32768 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T15:00:10.181 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131690+0000", 2024-09-21T15:00:10.181 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 95.005773624, 2024-09-21T15:00:10.181 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016751335999999999, 2024-09-21T15:00:10.181 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:10.181 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:10.181 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:10.181 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:10.181 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:10.181 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6642 2024-09-21T15:00:10.181 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.181 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:10.181 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.182 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:10.182 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131690+0000", 2024-09-21T15:00:10.182 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:10.182 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.182 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.182 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:10.182 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131690+0000", 2024-09-21T15:00:10.182 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:10.182 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.182 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.182 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:10.182 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131690+0000", 2024-09-21T15:00:10.182 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:00:10.183 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.183 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.183 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:10.183 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131691+0000", 2024-09-21T15:00:10.183 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3120000000000001e-06 2024-09-21T15:00:10.183 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.183 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.183 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:10.183 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131694+0000", 2024-09-21T15:00:10.183 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0070000000000002e-06 2024-09-21T15:00:10.183 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.183 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.183 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:10.184 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131700+0000", 2024-09-21T15:00:10.184 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0800000000000002e-06 2024-09-21T15:00:10.184 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.184 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.184 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:10.184 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.148135+0000", 2024-09-21T15:00:10.184 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016434250000000001 2024-09-21T15:00:10.184 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.184 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.184 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:10.184 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.148204+0000", 2024-09-21T15:00:10.184 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9466000000000002e-05 2024-09-21T15:00:10.184 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.184 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.185 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:10.185 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.148441+0000", 2024-09-21T15:00:10.185 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00023743700000000001 2024-09-21T15:00:10.185 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:10.185 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:10.185 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:10.185 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.185 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.185 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:61214 3.7 3:e28aec10:::rbd_header.1136862f872f:head [watch reconnect cookie 139864671932080 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-21T15:00:10.185 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:18.147172+0000", 2024-09-21T15:00:10.185 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 51.990291994000003, 2024-09-21T15:00:10.185 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.019264956, 2024-09-21T15:00:10.185 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:10.185 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:10.186 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:10.186 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:10.186 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:10.186 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 61214 2024-09-21T15:00:10.186 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.186 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:10.186 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.186 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:10.186 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147172+0000", 2024-09-21T15:00:10.186 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:10.186 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.186 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.186 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:10.187 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147172+0000", 2024-09-21T15:00:10.187 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:10.187 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.187 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.187 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:10.187 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147168+0000", 2024-09-21T15:00:10.187 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999962 2024-09-21T15:00:10.187 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.187 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.187 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:10.187 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147188+0000", 2024-09-21T15:00:10.187 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9717000000000001e-05 2024-09-21T15:00:10.187 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.187 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.188 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:10.188 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147239+0000", 2024-09-21T15:00:10.188 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1493999999999999e-05 2024-09-21T15:00:10.188 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.188 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.188 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:10.188 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147265+0000", 2024-09-21T15:00:10.188 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5746999999999999e-05 2024-09-21T15:00:10.188 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.188 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.188 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:10.188 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147372+0000", 2024-09-21T15:00:10.188 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010681200000000001 2024-09-21T15:00:10.188 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.188 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.189 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op must wait for map", 2024-09-21T15:00:10.189 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147380+0000", 2024-09-21T15:00:10.189 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7919999999999993e-06 2024-09-21T15:00:10.189 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.189 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.189 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:10.189 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.154025+0000", 2024-09-21T15:00:10.189 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0066454909999999999 2024-09-21T15:00:10.189 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.189 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.189 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:00:10.189 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.154041+0000", 2024-09-21T15:00:10.189 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5894000000000001e-05 2024-09-21T15:00:10.190 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.190 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.190 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:10.190 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.164345+0000", 2024-09-21T15:00:10.190 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0103040459999999 2024-09-21T15:00:10.190 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.190 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.190 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:10.190 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.164590+0000", 2024-09-21T15:00:10.190 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00024492699999999998 2024-09-21T15:00:10.190 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.190 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.191 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:10.191 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.166437+0000", 2024-09-21T15:00:10.191 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0018466330000000001 2024-09-21T15:00:10.191 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:10.191 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:10.191 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:10.191 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.191 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.191 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:61215 3.7 3:e28aec10:::rbd_header.1136862f872f:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-21T15:00:10.191 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:18.152008+0000", 2024-09-21T15:00:10.191 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 51.985455541999997, 2024-09-21T15:00:10.191 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.015644958, 2024-09-21T15:00:10.191 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:10.191 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:10.191 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:10.192 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:10.192 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:10.192 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 61215 2024-09-21T15:00:10.192 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.192 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:10.192 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.192 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:10.192 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152008+0000", 2024-09-21T15:00:10.192 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:10.192 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.192 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.192 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:10.192 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152008+0000", 2024-09-21T15:00:10.192 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:10.192 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.193 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.193 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:10.193 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152005+0000", 2024-09-21T15:00:10.193 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-21T15:00:10.193 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.193 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.193 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:10.193 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152025+0000", 2024-09-21T15:00:10.193 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9306000000000001e-05 2024-09-21T15:00:10.193 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.193 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.193 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:10.193 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152054+0000", 2024-09-21T15:00:10.193 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9189000000000001e-05 2024-09-21T15:00:10.194 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.194 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.194 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:10.194 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152078+0000", 2024-09-21T15:00:10.194 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4317000000000001e-05 2024-09-21T15:00:10.194 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.194 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.194 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:10.194 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152243+0000", 2024-09-21T15:00:10.194 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000164802 2024-09-21T15:00:10.194 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.194 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.194 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting_for_map not empty", 2024-09-21T15:00:10.195 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152251+0000", 2024-09-21T15:00:10.195 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0600000000000008e-06 2024-09-21T15:00:10.195 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.195 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.195 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:10.195 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.154741+0000", 2024-09-21T15:00:10.195 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0024900069999999998 2024-09-21T15:00:10.195 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.195 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.195 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:00:10.195 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.154753+0000", 2024-09-21T15:00:10.195 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1588e-05 2024-09-21T15:00:10.195 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.195 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.195 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:10.196 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.165380+0000", 2024-09-21T15:00:10.196 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0106272249999999 2024-09-21T15:00:10.196 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.196 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.196 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:10.196 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.165517+0000", 2024-09-21T15:00:10.196 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013737999999999999 2024-09-21T15:00:10.196 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.196 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.196 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 6", 2024-09-21T15:00:10.196 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.165938+0000", 2024-09-21T15:00:10.196 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00042082399999999998 2024-09-21T15:00:10.196 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.196 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.197 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:10.197 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.166795+0000", 2024-09-21T15:00:10.197 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00085674899999999997 2024-09-21T15:00:10.197 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.197 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.197 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:10.197 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.167598+0000", 2024-09-21T15:00:10.197 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00080335599999999997 2024-09-21T15:00:10.197 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.197 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.197 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:10.197 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.167632+0000", 2024-09-21T15:00:10.197 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4122000000000003e-05 2024-09-21T15:00:10.197 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.198 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.198 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:10.198 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T15:00:10.199 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.167653+0000", 2024-09-21T15:00:10.200 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0862000000000001e-05 2024-09-21T15:00:10.200 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:10.200 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:10.200 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:10.200 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.200 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.200 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:73698 3.10 3:0a4d95f6:::rbd_data.1136f4939602.000000000000000b:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-21T15:00:10.200 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:41.969870+0000", 2024-09-21T15:00:10.200 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 28.167593404000002, 2024-09-21T15:00:10.200 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.017502461, 2024-09-21T15:00:10.200 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:10.200 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:10.200 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:10.201 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:10.201 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:10.201 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 73698 2024-09-21T15:00:10.201 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.201 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:10.201 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.201 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:10.201 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.969870+0000", 2024-09-21T15:00:10.201 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:10.201 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.201 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.201 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:10.201 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.969870+0000", 2024-09-21T15:00:10.201 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:10.201 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.202 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.202 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:10.202 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.969868+0000", 2024-09-21T15:00:10.202 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T15:00:10.202 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.202 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.202 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:10.202 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.969885+0000", 2024-09-21T15:00:10.202 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7416999999999999e-05 2024-09-21T15:00:10.202 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.202 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.202 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:10.202 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.969918+0000", 2024-09-21T15:00:10.202 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3133000000000003e-05 2024-09-21T15:00:10.203 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.203 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.203 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:10.203 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.969934+0000", 2024-09-21T15:00:10.203 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6027000000000001e-05 2024-09-21T15:00:10.203 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.203 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.203 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:10.203 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.970021+0000", 2024-09-21T15:00:10.203 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6907e-05 2024-09-21T15:00:10.203 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.203 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.203 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:10.203 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.970148+0000", 2024-09-21T15:00:10.203 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012701499999999999 2024-09-21T15:00:10.204 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.204 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.204 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T15:00:10.204 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.972971+0000", 2024-09-21T15:00:10.204 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.002822873 2024-09-21T15:00:10.204 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.204 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.204 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:10.204 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.987124+0000", 2024-09-21T15:00:10.204 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0141529 2024-09-21T15:00:10.204 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.204 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.204 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:10.205 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.987146+0000", 2024-09-21T15:00:10.205 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1783000000000001e-05 2024-09-21T15:00:10.205 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.205 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.205 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:10.205 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.987258+0000", 2024-09-21T15:00:10.205 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000112555 2024-09-21T15:00:10.205 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.205 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.205 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:10.205 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.987373+0000", 2024-09-21T15:00:10.205 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000114434 2024-09-21T15:00:10.205 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:10.205 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:10.205 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:10.206 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.206 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.206 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:97638 3.1a 3:5e573525:::rbd_data.1136b9c49b1c.0000000000000026:49 [read 1277952~49152 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:10.206 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.276202+0000", 2024-09-21T15:00:10.206 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.8612612989999997, 2024-09-21T15:00:10.206 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.015360357999999999, 2024-09-21T15:00:10.206 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:10.206 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:10.206 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:10.206 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:10.206 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:10.206 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 97638 2024-09-21T15:00:10.206 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.206 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:10.206 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.207 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:10.207 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276202+0000", 2024-09-21T15:00:10.207 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:10.207 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.207 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.207 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:10.207 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276202+0000", 2024-09-21T15:00:10.207 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:10.207 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.207 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.207 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:10.207 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276202+0000", 2024-09-21T15:00:10.207 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:10.207 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.207 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.208 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:10.208 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276204+0000", 2024-09-21T15:00:10.208 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5359999999999999e-06 2024-09-21T15:00:10.208 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.208 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.208 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:10.208 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276210+0000", 2024-09-21T15:00:10.208 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3040000000000004e-06 2024-09-21T15:00:10.208 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.208 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.208 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:10.208 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276214+0000", 2024-09-21T15:00:10.208 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6779999999999999e-06 2024-09-21T15:00:10.208 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.208 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.209 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:10.209 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.290563+0000", 2024-09-21T15:00:10.209 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014348346 2024-09-21T15:00:10.209 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.209 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.209 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:10.209 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.290623+0000", 2024-09-21T15:00:10.209 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0001999999999999e-05 2024-09-21T15:00:10.209 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.209 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.209 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:10.209 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.291563+0000", 2024-09-21T15:00:10.209 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00094003100000000003 2024-09-21T15:00:10.209 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:10.209 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:10.210 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:10.210 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.210 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.210 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:97640 3.1a 3:5e573525:::rbd_data.1136b9c49b1c.0000000000000026:49 [read 2162688~49152 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:10.210 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.276235+0000", 2024-09-21T15:00:10.210 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.8612287810000003, 2024-09-21T15:00:10.210 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014898288000000001, 2024-09-21T15:00:10.210 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:10.210 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:10.210 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:10.210 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:10.210 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:10.210 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 97640 2024-09-21T15:00:10.210 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.211 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:10.211 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.211 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:10.211 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276235+0000", 2024-09-21T15:00:10.211 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:10.211 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.211 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.211 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:10.211 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276235+0000", 2024-09-21T15:00:10.211 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:10.211 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.211 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.211 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:10.211 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276235+0000", 2024-09-21T15:00:10.211 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:10.212 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.212 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.212 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:10.212 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276237+0000", 2024-09-21T15:00:10.212 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4779999999999998e-06 2024-09-21T15:00:10.212 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.212 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.212 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:10.212 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276244+0000", 2024-09-21T15:00:10.212 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6950000000000003e-06 2024-09-21T15:00:10.212 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.212 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.212 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:10.213 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276249+0000", 2024-09-21T15:00:10.213 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0420000000000002e-06 2024-09-21T15:00:10.213 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.213 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.213 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:10.213 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.291013+0000", 2024-09-21T15:00:10.213 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014763705 2024-09-21T15:00:10.213 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.213 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.213 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:10.213 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.291063+0000", 2024-09-21T15:00:10.213 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0702999999999999e-05 2024-09-21T15:00:10.213 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.213 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.214 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:10.214 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.291133+0000", 2024-09-21T15:00:10.214 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9986000000000004e-05 2024-09-21T15:00:10.214 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:10.214 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:10.214 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:10.214 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.214 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.214 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:97641 3.1a 3:5e573525:::rbd_data.1136b9c49b1c.0000000000000026:49 [read 2621440~32768 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:10.214 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.276254+0000", 2024-09-21T15:00:10.214 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.8612096469999999, 2024-09-21T15:00:10.214 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.015023668, 2024-09-21T15:00:10.214 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:10.215 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:10.215 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:10.215 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:10.215 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:10.215 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 97641 2024-09-21T15:00:10.215 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.215 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:10.215 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.215 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:10.215 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276254+0000", 2024-09-21T15:00:10.215 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:10.215 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.215 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.215 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:10.216 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276254+0000", 2024-09-21T15:00:10.216 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:10.216 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.216 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.216 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:10.216 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276254+0000", 2024-09-21T15:00:10.216 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:10.216 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.216 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.216 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:10.216 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276262+0000", 2024-09-21T15:00:10.216 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3820000000000004e-06 2024-09-21T15:00:10.216 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.217 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.217 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:10.217 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276268+0000", 2024-09-21T15:00:10.217 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1240000000000002e-06 2024-09-21T15:00:10.217 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.217 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.217 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:10.217 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276273+0000", 2024-09-21T15:00:10.217 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7990000000000001e-06 2024-09-21T15:00:10.217 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.217 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.217 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:10.217 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.291163+0000", 2024-09-21T15:00:10.217 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014889576999999999 2024-09-21T15:00:10.218 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.218 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.218 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:10.218 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.291210+0000", 2024-09-21T15:00:10.218 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7479999999999999e-05 2024-09-21T15:00:10.218 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.218 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.218 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:10.218 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.291278+0000", 2024-09-21T15:00:10.218 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7735000000000001e-05 2024-09-21T15:00:10.218 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:10.218 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:10.218 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:10.218 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.219 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.219 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:97642 3.1a 3:5e573525:::rbd_data.1136b9c49b1c.0000000000000026:49 [read 3063808~32768 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:10.219 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.276277+0000", 2024-09-21T15:00:10.219 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.8611868529999995, 2024-09-21T15:00:10.219 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.015140887, 2024-09-21T15:00:10.219 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:10.219 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:10.219 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:10.219 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:10.219 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:10.219 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 97642 2024-09-21T15:00:10.219 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.219 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:10.219 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.220 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:10.220 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276277+0000", 2024-09-21T15:00:10.220 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:10.220 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.220 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.220 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:10.220 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276277+0000", 2024-09-21T15:00:10.220 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:10.220 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.220 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.220 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:10.220 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276277+0000", 2024-09-21T15:00:10.220 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:10.220 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.221 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.221 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:10.221 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276279+0000", 2024-09-21T15:00:10.221 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5770000000000001e-06 2024-09-21T15:00:10.221 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.221 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.221 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:10.221 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276284+0000", 2024-09-21T15:00:10.221 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8910000000000003e-06 2024-09-21T15:00:10.221 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.221 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.221 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:10.221 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276288+0000", 2024-09-21T15:00:10.221 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3089999999999997e-06 2024-09-21T15:00:10.222 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.222 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.222 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:10.222 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.291306+0000", 2024-09-21T15:00:10.222 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.015017842 2024-09-21T15:00:10.222 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.222 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.222 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:10.222 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.291352+0000", 2024-09-21T15:00:10.222 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6248999999999997e-05 2024-09-21T15:00:10.222 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.222 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.222 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:10.222 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.291418+0000", 2024-09-21T15:00:10.222 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5333000000000006e-05 2024-09-21T15:00:10.223 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:10.223 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:10.223 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:10.223 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.223 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.223 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:97643 3.1a 3:5e573525:::rbd_data.1136b9c49b1c.0000000000000026:49 [read 3506176~32768 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:10.223 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.276293+0000", 2024-09-21T15:00:10.223 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.8611705910000005, 2024-09-21T15:00:10.223 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.015260792, 2024-09-21T15:00:10.223 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:10.223 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:10.223 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:10.223 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:10.223 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:10.224 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 97643 2024-09-21T15:00:10.224 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.224 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:10.224 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.224 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:10.224 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276293+0000", 2024-09-21T15:00:10.224 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:10.224 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.224 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.224 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:10.224 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276293+0000", 2024-09-21T15:00:10.224 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:10.224 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.224 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.225 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:10.225 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276293+0000", 2024-09-21T15:00:10.225 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:10.225 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.225 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.225 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:10.225 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276296+0000", 2024-09-21T15:00:10.225 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6989999999999999e-06 2024-09-21T15:00:10.225 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.225 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.225 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:10.225 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276302+0000", 2024-09-21T15:00:10.225 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9989999999999997e-06 2024-09-21T15:00:10.225 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.226 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.226 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:10.226 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276306+0000", 2024-09-21T15:00:10.226 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.831e-06 2024-09-21T15:00:10.226 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.226 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.226 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:10.226 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.291447+0000", 2024-09-21T15:00:10.226 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.015141056999999999 2024-09-21T15:00:10.226 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.226 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.226 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:10.226 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.291491+0000", 2024-09-21T15:00:10.227 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3210000000000001e-05 2024-09-21T15:00:10.227 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.227 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.227 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:10.227 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.291554+0000", 2024-09-21T15:00:10.227 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3367000000000005e-05 2024-09-21T15:00:10.227 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:10.227 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:10.227 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:10.227 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.227 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.227 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:97678 3.d e122/24 3:b2f34c6d:::rbd_data.1136bfc79a70.0000000000000006:head v 122'234, mlcod=120'232)", 2024-09-21T15:00:10.227 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.321655+0000", 2024-09-21T15:00:10.227 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.8158086249999998, 2024-09-21T15:00:10.228 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024509388999999999, 2024-09-21T15:00:10.228 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:10.228 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:10.228 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:10.228 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.228 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:10.228 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.321655+0000", 2024-09-21T15:00:10.228 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:10.228 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.228 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.228 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:10.228 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.321655+0000", 2024-09-21T15:00:10.228 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:10.228 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.229 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.229 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:10.229 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.321655+0000", 2024-09-21T15:00:10.229 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:10.229 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.229 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.229 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:10.229 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.329453+0000", 2024-09-21T15:00:10.229 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0077980560000000003 2024-09-21T15:00:10.229 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.229 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.229 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:10.229 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.329482+0000", 2024-09-21T15:00:10.229 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9153999999999999e-05 2024-09-21T15:00:10.230 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.230 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.230 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:10.230 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.329493+0000", 2024-09-21T15:00:10.230 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1103e-05 2024-09-21T15:00:10.230 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.230 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.230 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:10.230 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.329555+0000", 2024-09-21T15:00:10.230 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2345e-05 2024-09-21T15:00:10.230 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.230 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.230 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:10.230 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.329589+0000", 2024-09-21T15:00:10.231 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3908999999999997e-05 2024-09-21T15:00:10.231 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.231 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.231 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:10.231 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.346083+0000", 2024-09-21T15:00:10.231 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016493383 2024-09-21T15:00:10.231 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.231 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.231 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:10.231 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.346165+0000", 2024-09-21T15:00:10.231 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1796000000000001e-05 2024-09-21T15:00:10.231 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:10.231 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:10.231 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:10.231 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.232 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.232 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:97679 3.f e122/24 3:f2a362b4:::rbd_data.1136bfc79a70.0000000000000007:head v 122'331, mlcod=120'329)", 2024-09-21T15:00:10.232 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.337145+0000", 2024-09-21T15:00:10.232 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.8003186160000002, 2024-09-21T15:00:10.232 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021912411, 2024-09-21T15:00:10.232 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:10.232 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:10.232 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:10.232 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.232 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:10.232 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.337145+0000", 2024-09-21T15:00:10.232 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:10.232 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.232 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.233 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:10.233 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.337145+0000", 2024-09-21T15:00:10.233 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:10.233 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.233 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.233 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:10.233 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.337145+0000", 2024-09-21T15:00:10.233 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:10.233 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.233 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.233 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:10.233 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.343899+0000", 2024-09-21T15:00:10.233 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0067541420000000003 2024-09-21T15:00:10.233 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.234 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.234 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:10.234 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.343921+0000", 2024-09-21T15:00:10.234 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2255e-05 2024-09-21T15:00:10.234 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.234 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.234 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:10.234 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.343933+0000", 2024-09-21T15:00:10.234 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1528e-05 2024-09-21T15:00:10.234 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.234 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.234 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:10.234 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.344001+0000", 2024-09-21T15:00:10.235 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8269999999999995e-05 2024-09-21T15:00:10.235 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.235 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.235 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:10.235 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.344029+0000", 2024-09-21T15:00:10.235 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8254000000000001e-05 2024-09-21T15:00:10.235 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.235 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.235 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:10.235 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.358901+0000", 2024-09-21T15:00:10.235 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0148719 2024-09-21T15:00:10.235 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.235 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.235 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:10.235 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.359058+0000", 2024-09-21T15:00:10.236 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000156446 2024-09-21T15:00:10.236 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:10.236 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:10.236 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:10.236 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.236 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.236 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:97676 3.1f e122/24 3:ffaa836a:::rbd_data.1136bfc79a70.0000000000000004:head v 122'312, mlcod=120'310)", 2024-09-21T15:00:10.236 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.348683+0000", 2024-09-21T15:00:10.236 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.788780451, 2024-09-21T15:00:10.236 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.033768802000000001, 2024-09-21T15:00:10.236 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:10.236 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:10.236 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:10.236 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.236 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:10.236 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.348683+0000", 2024-09-21T15:00:10.237 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:10.237 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.237 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.237 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:10.237 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.348683+0000", 2024-09-21T15:00:10.237 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:10.237 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.237 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.237 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:10.237 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.348683+0000", 2024-09-21T15:00:10.237 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:00:10.237 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.237 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.237 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:10.237 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.366937+0000", 2024-09-21T15:00:10.238 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018254415 2024-09-21T15:00:10.238 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.238 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.238 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:10.238 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.366959+0000", 2024-09-21T15:00:10.238 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1634e-05 2024-09-21T15:00:10.238 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.238 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.238 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:10.238 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.366971+0000", 2024-09-21T15:00:10.238 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2305e-05 2024-09-21T15:00:10.238 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.238 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.238 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:10.239 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.367034+0000", 2024-09-21T15:00:10.239 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3211000000000002e-05 2024-09-21T15:00:10.239 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.239 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.239 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:10.239 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.367056+0000", 2024-09-21T15:00:10.239 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2098000000000001e-05 2024-09-21T15:00:10.239 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.239 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.239 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:10.239 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.382400+0000", 2024-09-21T15:00:10.239 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.015343867000000001 2024-09-21T15:00:10.239 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.239 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.239 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:10.240 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.382452+0000", 2024-09-21T15:00:10.240 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1992e-05 2024-09-21T15:00:10.240 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:10.240 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:10.240 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:10.240 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.240 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.240 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:97734 3.19 e122/94 3:9fcf754f:::rbd_data.1136bfc79a70.0000000000000009:head v 122'314, mlcod=120'312)", 2024-09-21T15:00:10.240 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.378547+0000", 2024-09-21T15:00:10.240 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.7589164730000002, 2024-09-21T15:00:10.240 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.063380675999999997, 2024-09-21T15:00:10.240 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:10.240 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:10.240 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:10.240 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.240 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:10.241 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.378547+0000", 2024-09-21T15:00:10.241 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:10.241 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.241 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.241 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:10.241 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.378547+0000", 2024-09-21T15:00:10.241 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:10.241 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.241 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.241 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:10.241 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.378547+0000", 2024-09-21T15:00:10.241 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:00:10.241 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.241 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.241 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:10.242 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.425118+0000", 2024-09-21T15:00:10.242 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046571166999999997 2024-09-21T15:00:10.242 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.242 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.242 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:10.242 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.425150+0000", 2024-09-21T15:00:10.242 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2150999999999998e-05 2024-09-21T15:00:10.242 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.242 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.242 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:10.242 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.425164+0000", 2024-09-21T15:00:10.242 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3302000000000001e-05 2024-09-21T15:00:10.242 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.242 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.242 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:10.243 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.425231+0000", 2024-09-21T15:00:10.243 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7106000000000004e-05 2024-09-21T15:00:10.243 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.243 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.243 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:10.243 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.425259+0000", 2024-09-21T15:00:10.243 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8087000000000001e-05 2024-09-21T15:00:10.243 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.243 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.243 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:10.243 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.441890+0000", 2024-09-21T15:00:10.243 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016631328000000001 2024-09-21T15:00:10.243 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.243 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.244 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:10.244 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.441928+0000", 2024-09-21T15:00:10.244 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7701999999999999e-05 2024-09-21T15:00:10.244 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:10.244 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:10.244 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:10.244 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:10.244 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:10.244 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:10.245 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T15:00:10.245 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:00:10.196+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:00:10.299 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:00:10.296+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:00:10.330 INFO:teuthology.orchestra.run.smithi032.stdout:checking clone #23, image image_client.0-clone23 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent24 2024-09-21T15:00:10.376 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:10.376 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:00:10.377 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T15:00:10.377 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:10.394 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T15:00:10.400 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:00:10.394+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:00:10.501 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:00:10.494+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:00:10.505 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:10.505 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:00:10.505 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T15:00:10.505 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T15:00:10.505 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:10.516 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T15:00:10.601 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:00:10.600+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:00:10.621 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:10.621 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T15:00:10.621 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T15:00:10.621 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T15:00:10.621 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.621 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:15130 3.2 3:47f1bbdb:::rbd_data.11361bed1f78.0000000000000067:head [sparse-read 0~147456] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-21T15:00:10.621 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:40.288284+0000", 2024-09-21T15:00:10.621 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 90.331548303000005, 2024-09-21T15:00:10.621 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1142935039999999, 2024-09-21T15:00:10.621 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:10.621 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:10.621 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:10.622 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:10.622 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:10.622 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 15130 2024-09-21T15:00:10.622 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.622 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:10.622 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.622 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:10.622 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288284+0000", 2024-09-21T15:00:10.622 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:10.622 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.622 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.622 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:10.622 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288284+0000", 2024-09-21T15:00:10.622 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:10.623 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.623 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.623 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:10.623 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288284+0000", 2024-09-21T15:00:10.623 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:10.623 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.623 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.623 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:10.623 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288288+0000", 2024-09-21T15:00:10.623 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5829999999999998e-06 2024-09-21T15:00:10.623 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.623 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.623 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:10.623 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288297+0000", 2024-09-21T15:00:10.624 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0669999999999998e-06 2024-09-21T15:00:10.624 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.624 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.624 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:10.624 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288305+0000", 2024-09-21T15:00:10.624 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8860000000000004e-06 2024-09-21T15:00:10.624 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.624 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.624 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:10.624 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288417+0000", 2024-09-21T15:00:10.624 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000112952 2024-09-21T15:00:10.624 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.624 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.624 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:00:10.625 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288428+0000", 2024-09-21T15:00:10.625 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0631e-05 2024-09-21T15:00:10.625 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.625 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.625 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:10.625 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.401876+0000", 2024-09-21T15:00:10.625 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1134479610000001 2024-09-21T15:00:10.625 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.625 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.625 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:10.625 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.402076+0000", 2024-09-21T15:00:10.625 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00019944200000000001 2024-09-21T15:00:10.625 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.625 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.625 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:10.626 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.402578+0000", 2024-09-21T15:00:10.626 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00050242899999999998 2024-09-21T15:00:10.626 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:10.626 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:10.626 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:10.626 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.626 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.626 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45708 3.2 3:415db91a:::rbd_data.1136ecbe5879.0000000000000000:28 [read 140288~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:10.626 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141801+0000", 2024-09-21T15:00:10.626 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 60.478032108999997, 2024-09-21T15:00:10.626 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034274200999999997, 2024-09-21T15:00:10.626 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:10.626 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:10.626 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:10.626 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:10.627 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:10.627 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45708 2024-09-21T15:00:10.627 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.627 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:10.627 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.627 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:10.627 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141801+0000", 2024-09-21T15:00:10.627 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:10.627 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.627 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.627 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:10.627 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141801+0000", 2024-09-21T15:00:10.627 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:10.627 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.628 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.628 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:10.628 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141800+0000", 2024-09-21T15:00:10.628 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:10.628 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.628 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.628 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:10.628 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141803+0000", 2024-09-21T15:00:10.628 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4210000000000002e-06 2024-09-21T15:00:10.628 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.628 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.628 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:10.628 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141812+0000", 2024-09-21T15:00:10.628 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.9800000000000004e-06 2024-09-21T15:00:10.628 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.629 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.629 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:10.629 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141816+0000", 2024-09-21T15:00:10.629 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4769999999999997e-06 2024-09-21T15:00:10.629 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.629 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.629 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:10.629 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.175865+0000", 2024-09-21T15:00:10.629 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034048875999999999 2024-09-21T15:00:10.629 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.629 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.629 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:10.629 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.175943+0000", 2024-09-21T15:00:10.629 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7464000000000002e-05 2024-09-21T15:00:10.630 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.630 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.630 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:10.630 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176075+0000", 2024-09-21T15:00:10.630 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013230800000000001 2024-09-21T15:00:10.630 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:10.630 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:10.630 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:10.630 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.630 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.630 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45709 3.2 3:415db91a:::rbd_data.1136ecbe5879.0000000000000000:28 [read 143360~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:10.630 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141820+0000", 2024-09-21T15:00:10.630 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 60.478012262, 2024-09-21T15:00:10.630 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034537536000000001, 2024-09-21T15:00:10.630 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:10.631 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:10.631 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:10.631 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:10.631 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:10.631 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45709 2024-09-21T15:00:10.631 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.631 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:10.631 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.631 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:10.631 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141820+0000", 2024-09-21T15:00:10.631 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:10.631 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.631 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.631 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:10.631 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141820+0000", 2024-09-21T15:00:10.631 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:10.632 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.632 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.632 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:10.632 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141820+0000", 2024-09-21T15:00:10.632 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:10.632 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.632 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.632 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:10.632 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141823+0000", 2024-09-21T15:00:10.632 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5110000000000002e-06 2024-09-21T15:00:10.632 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.632 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.632 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:10.632 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141829+0000", 2024-09-21T15:00:10.632 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0850000000000002e-06 2024-09-21T15:00:10.632 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.633 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.633 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:10.633 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141833+0000", 2024-09-21T15:00:10.633 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7620000000000003e-06 2024-09-21T15:00:10.633 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.633 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.633 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:10.633 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176103+0000", 2024-09-21T15:00:10.633 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034269933000000002 2024-09-21T15:00:10.633 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.633 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.633 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:10.633 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176161+0000", 2024-09-21T15:00:10.634 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7658999999999998e-05 2024-09-21T15:00:10.634 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.634 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.634 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:10.634 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176358+0000", 2024-09-21T15:00:10.634 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000196937 2024-09-21T15:00:10.634 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:10.634 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:10.634 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:10.634 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.634 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.634 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45710 3.2 3:415db91a:::rbd_data.1136ecbe5879.0000000000000000:28 [read 146432~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:10.634 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141838+0000", 2024-09-21T15:00:10.634 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 60.477995098999997, 2024-09-21T15:00:10.635 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034953674999999997, 2024-09-21T15:00:10.635 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:10.635 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:10.635 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:10.635 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:10.635 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:10.635 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45710 2024-09-21T15:00:10.635 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.635 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:10.635 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.635 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:10.635 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141838+0000", 2024-09-21T15:00:10.635 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:10.635 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.635 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.635 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:10.636 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141838+0000", 2024-09-21T15:00:10.636 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:10.636 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.636 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.636 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:10.636 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141837+0000", 2024-09-21T15:00:10.636 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:10.636 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.636 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.636 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:10.636 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141843+0000", 2024-09-21T15:00:10.636 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1619999999999996e-06 2024-09-21T15:00:10.636 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.636 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.636 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:10.636 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141850+0000", 2024-09-21T15:00:10.637 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4799999999999998e-06 2024-09-21T15:00:10.637 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.637 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.637 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:10.637 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141853+0000", 2024-09-21T15:00:10.637 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3050000000000001e-06 2024-09-21T15:00:10.637 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.637 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.637 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:10.637 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176490+0000", 2024-09-21T15:00:10.637 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034636413999999997 2024-09-21T15:00:10.637 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.637 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.637 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:10.637 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176606+0000", 2024-09-21T15:00:10.637 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000115841 2024-09-21T15:00:10.638 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.638 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.638 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:10.638 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176791+0000", 2024-09-21T15:00:10.638 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00018581500000000001 2024-09-21T15:00:10.638 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:10.638 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:10.638 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:10.638 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.638 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.638 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45711 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:10.638 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141856+0000", 2024-09-21T15:00:10.638 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 60.477976761999997, 2024-09-21T15:00:10.638 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034986308000000001, 2024-09-21T15:00:10.638 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:10.638 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:10.639 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:10.639 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:10.639 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:10.639 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45711 2024-09-21T15:00:10.639 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.639 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:10.639 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.639 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:10.639 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141856+0000", 2024-09-21T15:00:10.639 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:10.639 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.639 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.639 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:10.639 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141856+0000", 2024-09-21T15:00:10.639 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:10.640 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.640 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.640 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:10.640 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141856+0000", 2024-09-21T15:00:10.640 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:10.640 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.640 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.640 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:10.640 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141862+0000", 2024-09-21T15:00:10.640 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.567e-06 2024-09-21T15:00:10.640 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.640 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.640 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:10.640 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141868+0000", 2024-09-21T15:00:10.640 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9569999999999999e-06 2024-09-21T15:00:10.640 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.641 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.641 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:10.641 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141871+0000", 2024-09-21T15:00:10.641 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.146e-06 2024-09-21T15:00:10.641 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.641 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.641 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:10.641 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176831+0000", 2024-09-21T15:00:10.641 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034959935999999997 2024-09-21T15:00:10.641 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.641 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.641 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:10.641 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176842+0000", 2024-09-21T15:00:10.641 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1008e-05 2024-09-21T15:00:10.641 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:10.641 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:10.642 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:10.642 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.642 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.642 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45712 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:10.642 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141874+0000", 2024-09-21T15:00:10.642 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 60.477958866999998, 2024-09-21T15:00:10.642 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034993511999999997, 2024-09-21T15:00:10.642 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:10.642 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:10.642 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:10.642 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:10.642 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:10.642 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45712 2024-09-21T15:00:10.642 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.642 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:10.643 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.643 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:10.643 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141874+0000", 2024-09-21T15:00:10.643 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:10.643 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.643 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.643 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:10.643 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141874+0000", 2024-09-21T15:00:10.643 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:10.643 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.643 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.643 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:10.643 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141874+0000", 2024-09-21T15:00:10.643 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:10.643 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.644 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.644 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:10.644 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141875+0000", 2024-09-21T15:00:10.644 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5430000000000001e-06 2024-09-21T15:00:10.644 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.644 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.644 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:10.644 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141878+0000", 2024-09-21T15:00:10.644 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1420000000000002e-06 2024-09-21T15:00:10.644 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.644 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.644 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:10.644 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141881+0000", 2024-09-21T15:00:10.644 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.807e-06 2024-09-21T15:00:10.645 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.645 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.645 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:10.645 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176860+0000", 2024-09-21T15:00:10.645 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034979243 2024-09-21T15:00:10.645 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.645 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.645 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:10.645 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176867+0000", 2024-09-21T15:00:10.645 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0169999999999999e-06 2024-09-21T15:00:10.645 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:10.645 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:10.645 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:10.645 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.645 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.645 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45713 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:10.646 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141883+0000", 2024-09-21T15:00:10.646 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 60.477949291999998, 2024-09-21T15:00:10.646 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035010516999999998, 2024-09-21T15:00:10.646 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:10.646 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:10.646 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:10.646 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:10.646 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:10.646 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45713 2024-09-21T15:00:10.646 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.646 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:10.646 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.646 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:10.646 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141883+0000", 2024-09-21T15:00:10.646 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:10.647 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.647 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.647 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:10.647 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141883+0000", 2024-09-21T15:00:10.647 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:10.647 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.647 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.647 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:10.647 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141883+0000", 2024-09-21T15:00:10.647 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:10.647 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.647 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.647 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:10.647 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141885+0000", 2024-09-21T15:00:10.647 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4649999999999999e-06 2024-09-21T15:00:10.647 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.648 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.648 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:10.648 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141888+0000", 2024-09-21T15:00:10.648 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3890000000000001e-06 2024-09-21T15:00:10.648 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.648 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.648 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:10.648 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141891+0000", 2024-09-21T15:00:10.648 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9239999999999999e-06 2024-09-21T15:00:10.648 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.648 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.648 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:10.648 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176887+0000", 2024-09-21T15:00:10.648 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034996370999999998 2024-09-21T15:00:10.648 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.648 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.649 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:10.649 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176894+0000", 2024-09-21T15:00:10.649 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6069999999999996e-06 2024-09-21T15:00:10.649 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:10.649 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:10.649 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:10.649 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.649 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.649 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45714 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:10.649 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141893+0000", 2024-09-21T15:00:10.649 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 60.477939376999998, 2024-09-21T15:00:10.649 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035048215000000001, 2024-09-21T15:00:10.649 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:10.649 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:10.649 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:10.649 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:10.650 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:10.650 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45714 2024-09-21T15:00:10.650 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.650 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:10.650 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.650 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:10.650 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141893+0000", 2024-09-21T15:00:10.650 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:10.650 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.650 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.650 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:10.650 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141893+0000", 2024-09-21T15:00:10.650 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:10.650 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.650 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.651 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:10.651 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141893+0000", 2024-09-21T15:00:10.651 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:10.651 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.651 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.651 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:10.651 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141895+0000", 2024-09-21T15:00:10.651 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.703e-06 2024-09-21T15:00:10.651 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.651 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.651 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:10.651 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141900+0000", 2024-09-21T15:00:10.651 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.242e-06 2024-09-21T15:00:10.651 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.651 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.651 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:10.652 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141903+0000", 2024-09-21T15:00:10.652 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.117e-06 2024-09-21T15:00:10.653 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.653 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.653 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:10.653 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176930+0000", 2024-09-21T15:00:10.653 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035026750000000002 2024-09-21T15:00:10.654 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.654 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.654 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:10.654 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176942+0000", 2024-09-21T15:00:10.654 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1668000000000001e-05 2024-09-21T15:00:10.654 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:10.654 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:10.654 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:10.654 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.654 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.654 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45715 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:10.654 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141906+0000", 2024-09-21T15:00:10.654 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 60.477927219000001, 2024-09-21T15:00:10.654 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035066057999999997, 2024-09-21T15:00:10.655 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:10.655 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:10.655 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:10.655 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:10.655 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:10.655 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45715 2024-09-21T15:00:10.655 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.655 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:10.655 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.655 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:10.655 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141906+0000", 2024-09-21T15:00:10.655 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:10.655 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.655 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.656 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:10.656 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141906+0000", 2024-09-21T15:00:10.656 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:10.656 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.656 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.656 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:10.656 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141905+0000", 2024-09-21T15:00:10.656 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:10.656 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.656 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.656 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:10.656 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141907+0000", 2024-09-21T15:00:10.656 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5400000000000001e-06 2024-09-21T15:00:10.657 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.657 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.657 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:10.657 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141912+0000", 2024-09-21T15:00:10.657 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.502e-06 2024-09-21T15:00:10.657 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.657 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.657 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:10.657 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141915+0000", 2024-09-21T15:00:10.657 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0170000000000001e-06 2024-09-21T15:00:10.657 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.657 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.657 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:10.657 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176964+0000", 2024-09-21T15:00:10.657 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035048587999999999 2024-09-21T15:00:10.658 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.658 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.658 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:10.658 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176972+0000", 2024-09-21T15:00:10.658 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6580000000000002e-06 2024-09-21T15:00:10.658 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:10.658 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:10.658 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:10.658 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.658 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.658 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45716 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:10.658 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141918+0000", 2024-09-21T15:00:10.658 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 60.477915043000003, 2024-09-21T15:00:10.658 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035086223, 2024-09-21T15:00:10.659 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:10.659 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:10.659 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:10.659 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:10.659 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:10.659 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45716 2024-09-21T15:00:10.659 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.659 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:10.659 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.659 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:10.659 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141918+0000", 2024-09-21T15:00:10.659 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:10.659 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.659 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.659 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:10.660 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141918+0000", 2024-09-21T15:00:10.660 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:10.660 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.660 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.660 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:10.660 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141917+0000", 2024-09-21T15:00:10.660 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:10.660 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.660 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.660 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:10.660 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141919+0000", 2024-09-21T15:00:10.660 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5680000000000001e-06 2024-09-21T15:00:10.660 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.660 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.661 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:10.661 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141922+0000", 2024-09-21T15:00:10.661 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2789999999999999e-06 2024-09-21T15:00:10.661 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.661 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.661 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:10.661 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141925+0000", 2024-09-21T15:00:10.661 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9469999999999999e-06 2024-09-21T15:00:10.661 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.661 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.661 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:10.661 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176997+0000", 2024-09-21T15:00:10.661 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035071366999999999 2024-09-21T15:00:10.662 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.662 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.662 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:10.662 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177004+0000", 2024-09-21T15:00:10.662 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3309999999999998e-06 2024-09-21T15:00:10.662 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:10.662 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:10.662 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:10.662 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.662 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.662 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45717 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:10.662 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141928+0000", 2024-09-21T15:00:10.662 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 60.477905096999997, 2024-09-21T15:00:10.663 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035102573999999998, 2024-09-21T15:00:10.663 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:10.663 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:10.663 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:10.663 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:10.663 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:10.663 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45717 2024-09-21T15:00:10.663 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.663 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:10.663 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.663 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:10.663 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141928+0000", 2024-09-21T15:00:10.663 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:10.663 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.663 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.664 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:10.664 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141928+0000", 2024-09-21T15:00:10.664 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:10.664 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.664 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.664 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:10.664 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141927+0000", 2024-09-21T15:00:10.664 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:00:10.664 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.664 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.664 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:10.664 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141929+0000", 2024-09-21T15:00:10.664 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.533e-06 2024-09-21T15:00:10.664 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.664 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.664 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:10.665 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141934+0000", 2024-09-21T15:00:10.665 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1660000000000002e-06 2024-09-21T15:00:10.665 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.665 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.665 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:10.665 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141937+0000", 2024-09-21T15:00:10.665 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0309999999999999e-06 2024-09-21T15:00:10.665 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.665 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.665 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:10.665 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177023+0000", 2024-09-21T15:00:10.665 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035085595999999997 2024-09-21T15:00:10.666 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.666 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.666 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:10.666 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177030+0000", 2024-09-21T15:00:10.666 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.486e-06 2024-09-21T15:00:10.666 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:10.666 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:10.666 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:10.666 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.666 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.666 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45718 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:10.666 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141940+0000", 2024-09-21T15:00:10.667 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 60.477893238999997, 2024-09-21T15:00:10.667 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035124956999999998, 2024-09-21T15:00:10.667 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:10.667 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:10.667 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:10.667 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:10.667 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:10.667 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45718 2024-09-21T15:00:10.667 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.667 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:10.667 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.667 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:10.667 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141940+0000", 2024-09-21T15:00:10.667 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:10.668 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.668 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.668 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:10.668 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141940+0000", 2024-09-21T15:00:10.668 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:10.668 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.668 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.668 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:10.668 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141939+0000", 2024-09-21T15:00:10.668 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:10.668 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.668 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.668 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:10.668 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141941+0000", 2024-09-21T15:00:10.668 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.502e-06 2024-09-21T15:00:10.668 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.669 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.669 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:10.669 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141946+0000", 2024-09-21T15:00:10.669 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3449999999999996e-06 2024-09-21T15:00:10.669 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.669 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.669 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:10.669 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141949+0000", 2024-09-21T15:00:10.669 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9749999999999999e-06 2024-09-21T15:00:10.669 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.669 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.669 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:10.669 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177057+0000", 2024-09-21T15:00:10.669 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035108385999999998 2024-09-21T15:00:10.669 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.670 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.670 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:10.670 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177064+0000", 2024-09-21T15:00:10.670 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0099999999999998e-06 2024-09-21T15:00:10.670 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:10.670 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:10.670 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:10.670 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.670 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.670 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45719 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:10.670 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141952+0000", 2024-09-21T15:00:10.670 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 60.477880966000001, 2024-09-21T15:00:10.670 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035139108000000002, 2024-09-21T15:00:10.670 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:10.671 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:10.671 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:10.671 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:10.671 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:10.671 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45719 2024-09-21T15:00:10.671 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.671 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:10.671 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.671 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:10.671 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141952+0000", 2024-09-21T15:00:10.671 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:10.671 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.671 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.671 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:10.671 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141952+0000", 2024-09-21T15:00:10.672 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:10.672 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.672 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.672 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:10.672 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141952+0000", 2024-09-21T15:00:10.672 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:10.672 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.672 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.672 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:10.672 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141953+0000", 2024-09-21T15:00:10.672 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.486e-06 2024-09-21T15:00:10.672 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.672 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.672 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:10.672 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141959+0000", 2024-09-21T15:00:10.673 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4940000000000003e-06 2024-09-21T15:00:10.673 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.673 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.673 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:10.673 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141963+0000", 2024-09-21T15:00:10.673 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3540000000000002e-06 2024-09-21T15:00:10.673 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.673 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.673 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:10.673 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177084+0000", 2024-09-21T15:00:10.673 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035120983000000001 2024-09-21T15:00:10.673 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.673 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.673 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:10.673 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177091+0000", 2024-09-21T15:00:10.673 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0350000000000004e-06 2024-09-21T15:00:10.674 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:10.674 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:10.674 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:10.674 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.674 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.674 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45720 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:10.674 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141965+0000", 2024-09-21T15:00:10.674 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 60.477867498999998, 2024-09-21T15:00:10.674 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035155268000000003, 2024-09-21T15:00:10.674 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:10.674 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:10.674 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:10.674 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:10.674 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:10.674 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45720 2024-09-21T15:00:10.675 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.675 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:10.675 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.675 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:10.675 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141965+0000", 2024-09-21T15:00:10.675 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:10.675 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.675 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.675 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:10.675 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141965+0000", 2024-09-21T15:00:10.675 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:10.675 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.675 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.675 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:10.675 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141965+0000", 2024-09-21T15:00:10.676 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:10.676 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.676 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.676 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:10.676 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141966+0000", 2024-09-21T15:00:10.676 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.46e-06 2024-09-21T15:00:10.676 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.676 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.676 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:10.676 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141972+0000", 2024-09-21T15:00:10.676 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4210000000000003e-06 2024-09-21T15:00:10.676 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.676 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.676 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:10.676 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141975+0000", 2024-09-21T15:00:10.677 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.948e-06 2024-09-21T15:00:10.677 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.677 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.677 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:10.677 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177113+0000", 2024-09-21T15:00:10.677 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035137855000000003 2024-09-21T15:00:10.677 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.677 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.677 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:10.677 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177121+0000", 2024-09-21T15:00:10.677 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8269999999999998e-06 2024-09-21T15:00:10.677 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:10.677 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:10.677 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:10.678 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.678 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.678 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45721 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:10.678 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141977+0000", 2024-09-21T15:00:10.678 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 60.477855476000002, 2024-09-21T15:00:10.678 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035169984000000001, 2024-09-21T15:00:10.678 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:10.678 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:10.678 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:10.678 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:10.678 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:10.678 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45721 2024-09-21T15:00:10.678 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.678 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:10.679 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.679 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:10.679 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141977+0000", 2024-09-21T15:00:10.679 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:10.679 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.679 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.679 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:10.679 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141977+0000", 2024-09-21T15:00:10.679 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:10.679 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.679 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.679 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:10.679 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141977+0000", 2024-09-21T15:00:10.680 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:10.680 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.680 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.680 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:10.680 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141979+0000", 2024-09-21T15:00:10.680 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.511e-06 2024-09-21T15:00:10.680 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.680 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.680 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:10.680 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141982+0000", 2024-09-21T15:00:10.680 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4180000000000001e-06 2024-09-21T15:00:10.680 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.680 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.680 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:10.680 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141985+0000", 2024-09-21T15:00:10.681 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9450000000000002e-06 2024-09-21T15:00:10.681 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.681 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.681 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:10.681 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177139+0000", 2024-09-21T15:00:10.681 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035154409999999997 2024-09-21T15:00:10.681 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.681 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.681 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:10.681 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177147+0000", 2024-09-21T15:00:10.681 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9570000000000002e-06 2024-09-21T15:00:10.681 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:10.681 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:10.681 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:10.681 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.682 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.682 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45722 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:10.682 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141987+0000", 2024-09-21T15:00:10.682 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 60.477845504999998, 2024-09-21T15:00:10.682 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035188492000000002, 2024-09-21T15:00:10.682 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:10.682 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:10.682 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:10.682 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:10.682 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:10.682 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45722 2024-09-21T15:00:10.682 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.682 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:10.682 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.682 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:10.683 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141987+0000", 2024-09-21T15:00:10.683 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:10.683 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.683 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.683 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:10.683 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141987+0000", 2024-09-21T15:00:10.683 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:10.683 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.683 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.683 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:10.683 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141987+0000", 2024-09-21T15:00:10.683 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:10.683 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.683 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.683 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:10.683 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141988+0000", 2024-09-21T15:00:10.684 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.463e-06 2024-09-21T15:00:10.684 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.684 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.684 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:10.684 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141994+0000", 2024-09-21T15:00:10.684 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.361e-06 2024-09-21T15:00:10.684 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.684 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.684 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:10.684 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141997+0000", 2024-09-21T15:00:10.684 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0359999999999998e-06 2024-09-21T15:00:10.684 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.684 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.684 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:10.684 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177169+0000", 2024-09-21T15:00:10.685 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035171651999999998 2024-09-21T15:00:10.685 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.685 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.685 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:10.685 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177176+0000", 2024-09-21T15:00:10.685 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2189999999999998e-06 2024-09-21T15:00:10.685 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:10.685 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:10.685 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:10.685 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.685 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.685 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45723 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:10.685 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141999+0000", 2024-09-21T15:00:10.685 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 60.477833338000003, 2024-09-21T15:00:10.685 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035197701999999997, 2024-09-21T15:00:10.686 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:10.686 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:10.686 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:10.686 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:10.686 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:10.686 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45723 2024-09-21T15:00:10.686 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.686 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:10.686 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.686 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:10.686 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141999+0000", 2024-09-21T15:00:10.686 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:10.686 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.686 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.686 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:10.687 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141999+0000", 2024-09-21T15:00:10.687 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:10.687 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.687 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.687 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:10.687 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141999+0000", 2024-09-21T15:00:10.687 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:10.687 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.687 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.687 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:10.687 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142003+0000", 2024-09-21T15:00:10.687 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5049999999999999e-06 2024-09-21T15:00:10.687 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.687 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.687 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:10.687 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142008+0000", 2024-09-21T15:00:10.688 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6579999999999999e-06 2024-09-21T15:00:10.688 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.688 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.688 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:10.688 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142011+0000", 2024-09-21T15:00:10.688 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0529999999999998e-06 2024-09-21T15:00:10.688 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.688 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.688 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:10.688 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177194+0000", 2024-09-21T15:00:10.688 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035182221999999999 2024-09-21T15:00:10.688 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.689 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.689 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:10.689 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177197+0000", 2024-09-21T15:00:10.689 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5030000000000002e-06 2024-09-21T15:00:10.689 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:10.689 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:10.689 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:10.689 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.689 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.689 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45724 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:10.689 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.142015+0000", 2024-09-21T15:00:10.689 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 60.477817371, 2024-09-21T15:00:10.689 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035195345000000003, 2024-09-21T15:00:10.690 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:10.690 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:10.690 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:10.690 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:10.690 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:10.690 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45724 2024-09-21T15:00:10.690 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.690 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:10.690 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.690 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:10.690 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142015+0000", 2024-09-21T15:00:10.690 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:10.691 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.691 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.691 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:10.691 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142015+0000", 2024-09-21T15:00:10.691 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:10.691 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.691 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.691 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:10.691 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142015+0000", 2024-09-21T15:00:10.691 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:10.691 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.691 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.691 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:10.691 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142017+0000", 2024-09-21T15:00:10.692 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5540000000000001e-06 2024-09-21T15:00:10.692 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.692 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.692 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:10.692 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142022+0000", 2024-09-21T15:00:10.692 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4229999999999996e-06 2024-09-21T15:00:10.692 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.692 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.692 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:10.692 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142025+0000", 2024-09-21T15:00:10.692 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.056e-06 2024-09-21T15:00:10.692 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.692 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.693 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:10.693 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177207+0000", 2024-09-21T15:00:10.693 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035182089 2024-09-21T15:00:10.693 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.693 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.693 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:10.693 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177211+0000", 2024-09-21T15:00:10.693 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4649999999999999e-06 2024-09-21T15:00:10.693 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:10.693 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:10.693 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:10.693 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.694 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.694 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45725 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:10.694 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.142028+0000", 2024-09-21T15:00:10.694 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 60.477805091999997, 2024-09-21T15:00:10.694 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035195731000000001, 2024-09-21T15:00:10.694 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:10.694 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:10.694 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:10.694 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:10.694 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:10.694 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45725 2024-09-21T15:00:10.694 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.695 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:10.695 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.695 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:10.695 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142028+0000", 2024-09-21T15:00:10.695 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:10.695 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.695 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.695 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:10.695 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142028+0000", 2024-09-21T15:00:10.695 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:10.695 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.695 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.695 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:10.696 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142027+0000", 2024-09-21T15:00:10.696 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:10.696 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.696 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.696 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:10.696 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142029+0000", 2024-09-21T15:00:10.696 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5269999999999999e-06 2024-09-21T15:00:10.696 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.696 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.696 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:10.696 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142032+0000", 2024-09-21T15:00:10.696 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3249999999999999e-06 2024-09-21T15:00:10.696 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.696 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.697 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:10.697 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142035+0000", 2024-09-21T15:00:10.697 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.897e-06 2024-09-21T15:00:10.697 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.697 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.697 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:10.697 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177220+0000", 2024-09-21T15:00:10.697 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035184871 2024-09-21T15:00:10.697 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.697 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.697 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:10.697 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177223+0000", 2024-09-21T15:00:10.698 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3680000000000002e-06 2024-09-21T15:00:10.698 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:10.698 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:10.698 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:10.698 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.698 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.698 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45726 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:10.698 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.142037+0000", 2024-09-21T15:00:10.698 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 60.477795377, 2024-09-21T15:00:10.698 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035199273000000003, 2024-09-21T15:00:10.698 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:10.698 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:10.699 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:10.699 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:10.699 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:10.699 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45726 2024-09-21T15:00:10.699 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.699 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:10.699 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.699 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:10.699 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142037+0000", 2024-09-21T15:00:10.699 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:10.699 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.699 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.699 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:10.700 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142037+0000", 2024-09-21T15:00:10.700 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:10.700 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.700 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.700 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:10.700 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142037+0000", 2024-09-21T15:00:10.700 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:10.700 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.700 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.700 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:10.700 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142038+0000", 2024-09-21T15:00:10.700 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.344e-06 2024-09-21T15:00:10.700 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.700 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.701 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:10.701 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142044+0000", 2024-09-21T15:00:10.701 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1930000000000001e-06 2024-09-21T15:00:10.701 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.701 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.701 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:10.701 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142047+0000", 2024-09-21T15:00:10.701 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0280000000000001e-06 2024-09-21T15:00:10.701 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.701 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.701 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:10.701 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177233+0000", 2024-09-21T15:00:10.701 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035186394000000003 2024-09-21T15:00:10.702 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:10.702 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:10.702 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:10.702 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177237+0000", 2024-09-21T15:00:10.702 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5530000000000001e-06 2024-09-21T15:00:10.702 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:10.702 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:10.702 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:10.702 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:10.702 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:10.702 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:10.703 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T15:00:10.704 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:00:10.702+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:00:10.750 INFO:teuthology.orchestra.run.smithi032.stdout:3161 read 0x8a20523 thru 0x8a25af8 (0x55d6 bytes) 2024-09-21T15:00:10.751 INFO:teuthology.orchestra.run.smithi032.stdout:3162 writesame 0x61ba682 thru 0x61c519e data_size 0x1(0xab1d bytes) 2024-09-21T15:00:10.792 INFO:teuthology.orchestra.run.smithi032.stdout:3163 write 0x94e0492 thru 0x94eb11e (0xac8d bytes) 2024-09-21T15:00:10.799 INFO:teuthology.orchestra.run.smithi032.stdout:3167 read 0x47ca65c thru 0x47d9a88 (0xf42d bytes) 2024-09-21T15:00:10.800 INFO:teuthology.orchestra.run.smithi032.stdout:3169 write 0x6565190 thru 0x6565378 (0x1e9 bytes) 2024-09-21T15:00:10.803 INFO:teuthology.orchestra.run.smithi032.stdout:3171 punch from 0xd11c432 to 0xd121216, (0x4de4 bytes) 2024-09-21T15:00:10.803 INFO:teuthology.orchestra.run.smithi032.stdout:3172 write 0x4e549e4 thru 0x4e5ff7b (0xb598 bytes) 2024-09-21T15:00:10.804 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:00:10.800+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:00:10.807 INFO:teuthology.orchestra.run.smithi032.stdout:3174 write 0xaafe997 thru 0xab0b485 (0xcaef bytes) 2024-09-21T15:00:10.815 INFO:teuthology.orchestra.run.smithi032.stdout:3176 read 0x89df3f8 thru 0x89e18b0 (0x24b9 bytes) 2024-09-21T15:00:10.816 INFO:teuthology.orchestra.run.smithi032.stdout:3177 write 0xa3c169d thru 0xa3cdfef (0xc953 bytes) 2024-09-21T15:00:10.830 INFO:teuthology.orchestra.run.smithi032.stdout:3178 trunc from 0xe851201 to 0x9b4c68c 2024-09-21T15:00:10.852 INFO:teuthology.orchestra.run.smithi032.stdout:3180 read 0x1b21366 thru 0x1b23708 (0x23a3 bytes) 2024-09-21T15:00:10.852 INFO:teuthology.orchestra.run.smithi032.stdout:3181 writesame 0x280e726 thru 0x281404e data_size 0x37(0x5929 bytes) 2024-09-21T15:00:10.856 INFO:teuthology.orchestra.run.smithi032.stdout:3182 write 0x92e0dac thru 0x92eccaf (0xbf04 bytes) 2024-09-21T15:00:10.866 INFO:teuthology.orchestra.run.smithi032.stdout:3183 read 0x522ab65 thru 0x5236bea (0xc086 bytes) 2024-09-21T15:00:10.866 INFO:teuthology.orchestra.run.smithi032.stdout:3185 writesame 0x9486c9d thru 0x9491825 data_size 0x1(0xab89 bytes) 2024-09-21T15:00:10.870 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:10.870 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:00:10.870 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T15:00:10.870 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:10.881 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T15:00:10.897 INFO:teuthology.orchestra.run.smithi032.stdout:3187 read 0x446138 thru 0x44a0ee (0x3fb7 bytes) 2024-09-21T15:00:10.905 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:00:10.898+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:00:10.914 INFO:teuthology.orchestra.run.smithi032.stdout:3188 write 0xe76a694 thru 0xe76e008 (0x3975 bytes) 2024-09-21T15:00:10.920 INFO:teuthology.orchestra.run.smithi032.stdout:3190 read 0x5a93c1 thru 0x5aef3b (0x5b7b bytes) 2024-09-21T15:00:10.921 INFO:teuthology.orchestra.run.smithi032.stdout:3192 read 0x3db8f1 thru 0x3e3dd9 (0x84e9 bytes) 2024-09-21T15:00:10.935 INFO:teuthology.orchestra.run.smithi032.stdout:3193 trunc from 0xe76e009 to 0x5adf65c 2024-09-21T15:00:10.984 INFO:teuthology.orchestra.run.smithi032.stdout:3196 writesame 0xc6d0b60 thru 0xc6d3154 data_size 0x29(0x25f5 bytes) 2024-09-21T15:00:10.984 INFO:teuthology.orchestra.run.smithi032.stdout:3197 read 0x63ff335 thru 0x6402e4a (0x3b16 bytes) 2024-09-21T15:00:10.985 INFO:teuthology.orchestra.run.smithi032.stdout:3199 punch from 0x9a54537 to 0x9a5d0cf, (0x8b98 bytes) 2024-09-21T15:00:10.985 INFO:teuthology.orchestra.run.smithi032.stdout:3200 writesame 0x832dee7 thru 0x8335195 data_size 0x11(0x72af bytes) 2024-09-21T15:00:10.988 INFO:teuthology.orchestra.run.smithi032.stdout:3201 punch from 0x311aad6 to 0x3123a2a, (0x8f54 bytes) 2024-09-21T15:00:10.989 INFO:teuthology.orchestra.run.smithi032.stdout:3202 read 0x7c45390 thru 0x7c52bfb (0xd86c bytes) 2024-09-21T15:00:10.990 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:10.990 INFO:teuthology.orchestra.run.smithi032.stdout:3204 read 0xb83d728 thru 0xb84151c (0x3df5 bytes) 2024-09-21T15:00:10.990 INFO:teuthology.orchestra.run.smithi032.stdout:3206 punch from 0x6e54464 to 0x6e5b8db, (0x7477 bytes) 2024-09-21T15:00:10.990 INFO:teuthology.orchestra.run.smithi032.stdout:3207 write 0x6412c20 thru 0x6415bf3 (0x2fd4 bytes) 2024-09-21T15:00:10.994 INFO:teuthology.orchestra.run.smithi032.stdout:3208 punch from 0x1223991 to 0x122ace7, (0x7356 bytes) 2024-09-21T15:00:10.994 INFO:teuthology.orchestra.run.smithi032.stdout:3209 write 0x4938cf6 thru 0x494845d (0xf768 bytes) 2024-09-21T15:00:11.001 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:11.001 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:00:11.001 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T15:00:11.002 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T15:00:11.002 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:11.005 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:00:10.998+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:00:11.012 INFO:teuthology.orchestra.run.smithi032.stdout:3210 trunc from 0xc6d3155 to 0x508ffaf 2024-09-21T15:00:11.014 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T15:00:11.033 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:11.050 INFO:teuthology.orchestra.run.smithi032.stdout:3213 writesame 0xa218053 thru 0xa21db70 data_size 0x2(0x5b1e bytes) 2024-09-21T15:00:11.050 INFO:teuthology.orchestra.run.smithi032.stdout:3214 writesame 0x13803d6 thru 0x138febc data_size 0x1(0xfae7 bytes) 2024-09-21T15:00:11.093 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:11.093 INFO:teuthology.orchestra.run.smithi032.stdout:3216 writesame 0x825e42a thru 0x82692a9 data_size 0x40(0xae80 bytes) 2024-09-21T15:00:11.097 INFO:teuthology.orchestra.run.smithi032.stdout:3218 write 0x4d01fb6 thru 0x4d10500 (0xe54b bytes) 2024-09-21T15:00:11.101 INFO:teuthology.orchestra.run.smithi032.stdout:3219 read 0x9fe03a4 thru 0x9fe615d (0x5dba bytes) 2024-09-21T15:00:11.102 INFO:teuthology.orchestra.run.smithi032.stdout:3220 read 0xedcd64 thru 0xee9a36 (0xccd3 bytes) 2024-09-21T15:00:11.103 INFO:teuthology.orchestra.run.smithi032.stdout:3221 read 0x159cba5 thru 0x15a2edc (0x6338 bytes) 2024-09-21T15:00:11.106 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:00:11.104+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:00:11.119 INFO:teuthology.orchestra.run.smithi032.stdout:3222 writesame 0xc68ef38 thru 0xc697393 data_size 0x56(0x845c bytes) 2024-09-21T15:00:11.132 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:11.132 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T15:00:11.132 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T15:00:11.132 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T15:00:11.133 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.133 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4281.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-21T15:00:11.133 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:09.324416+0000", 2024-09-21T15:00:11.133 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 121.807092711, 2024-09-21T15:00:11.133 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.117837947, 2024-09-21T15:00:11.133 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:11.133 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:11.133 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:11.133 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4281", 2024-09-21T15:00:11.133 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/1518520303", 2024-09-21T15:00:11.133 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1 2024-09-21T15:00:11.133 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.134 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:11.134 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.134 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:11.134 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324416+0000", 2024-09-21T15:00:11.134 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:11.134 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.134 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.134 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:11.134 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324416+0000", 2024-09-21T15:00:11.134 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:11.134 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.134 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.135 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:11.135 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324414+0000", 2024-09-21T15:00:11.135 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T15:00:11.135 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.135 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.135 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:11.135 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324429+0000", 2024-09-21T15:00:11.135 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5637e-05 2024-09-21T15:00:11.135 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.135 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.135 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:11.135 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324448+0000", 2024-09-21T15:00:11.135 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9042000000000002e-05 2024-09-21T15:00:11.136 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.136 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.136 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:11.136 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324465+0000", 2024-09-21T15:00:11.136 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6767999999999999e-05 2024-09-21T15:00:11.136 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.136 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.136 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:11.136 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324571+0000", 2024-09-21T15:00:11.136 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000105714 2024-09-21T15:00:11.136 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.136 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.137 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op must wait for map", 2024-09-21T15:00:11.137 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324587+0000", 2024-09-21T15:00:11.137 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5753999999999999e-05 2024-09-21T15:00:11.137 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.137 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.137 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:11.137 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.325864+0000", 2024-09-21T15:00:11.137 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0012773350000000001 2024-09-21T15:00:11.137 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.137 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.137 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:00:11.137 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.325878+0000", 2024-09-21T15:00:11.137 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4498000000000001e-05 2024-09-21T15:00:11.138 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.138 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.138 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:11.138 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.439461+0000", 2024-09-21T15:00:11.138 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.11358212500000001 2024-09-21T15:00:11.138 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.138 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.138 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:11.138 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.439848+0000", 2024-09-21T15:00:11.138 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00038747800000000002 2024-09-21T15:00:11.138 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.138 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.138 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T15:00:11.139 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.440175+0000", 2024-09-21T15:00:11.139 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00032721900000000001 2024-09-21T15:00:11.139 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.139 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.139 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:11.139 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.441770+0000", 2024-09-21T15:00:11.139 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0015949409999999999 2024-09-21T15:00:11.139 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.139 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.139 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:11.139 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.442086+0000", 2024-09-21T15:00:11.139 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00031586299999999999 2024-09-21T15:00:11.139 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.140 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.140 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:11.140 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.442195+0000", 2024-09-21T15:00:11.140 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000108791 2024-09-21T15:00:11.140 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.140 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.140 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:11.140 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.442254+0000", 2024-09-21T15:00:11.140 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9038e-05 2024-09-21T15:00:11.140 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:11.140 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:11.140 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:11.140 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.140 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.141 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4281.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-21T15:00:11.141 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:09.447828+0000", 2024-09-21T15:00:11.141 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 121.68368071, 2024-09-21T15:00:11.141 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.32904393700000001, 2024-09-21T15:00:11.141 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:11.141 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:11.141 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:11.141 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4281", 2024-09-21T15:00:11.141 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/1518520303", 2024-09-21T15:00:11.141 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 2 2024-09-21T15:00:11.141 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.141 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:11.141 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.142 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:11.142 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447828+0000", 2024-09-21T15:00:11.142 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:11.142 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.142 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.142 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:11.142 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447828+0000", 2024-09-21T15:00:11.142 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:11.142 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.142 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.142 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:11.142 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447825+0000", 2024-09-21T15:00:11.142 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T15:00:11.143 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.143 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.143 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:11.143 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447841+0000", 2024-09-21T15:00:11.143 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5359999999999999e-05 2024-09-21T15:00:11.143 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.143 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.143 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:11.143 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447875+0000", 2024-09-21T15:00:11.143 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4388999999999999e-05 2024-09-21T15:00:11.143 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.143 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.144 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:11.144 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447903+0000", 2024-09-21T15:00:11.144 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7373e-05 2024-09-21T15:00:11.144 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.144 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.144 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:11.144 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447996+0000", 2024-09-21T15:00:11.144 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2954999999999993e-05 2024-09-21T15:00:11.144 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.144 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.144 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:00:11.144 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.448011+0000", 2024-09-21T15:00:11.144 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5038999999999999e-05 2024-09-21T15:00:11.145 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.145 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.145 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:11.145 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.776501+0000", 2024-09-21T15:00:11.145 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.32849012500000002 2024-09-21T15:00:11.145 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.145 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.145 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:11.145 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.776872+0000", 2024-09-21T15:00:11.145 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00037119600000000002 2024-09-21T15:00:11.145 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:11.145 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:11.146 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:11.146 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.146 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.146 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:752 3.14 3:28fc02e5:::rbd_data.113693316f36.0000000000000010:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-21T15:00:11.146 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:26.625579+0000", 2024-09-21T15:00:11.146 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 104.50592985500001, 2024-09-21T15:00:11.146 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042965426000000001, 2024-09-21T15:00:11.146 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:11.146 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:11.146 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:11.146 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:11.146 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:11.146 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 752 2024-09-21T15:00:11.147 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.147 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:11.147 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.147 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:11.147 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625579+0000", 2024-09-21T15:00:11.147 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:11.147 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.147 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.147 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:11.147 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625579+0000", 2024-09-21T15:00:11.147 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:11.147 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.147 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.148 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:11.148 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625577+0000", 2024-09-21T15:00:11.148 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T15:00:11.148 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.148 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.148 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:11.148 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625588+0000", 2024-09-21T15:00:11.148 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1555e-05 2024-09-21T15:00:11.148 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.148 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.148 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:11.148 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625610+0000", 2024-09-21T15:00:11.148 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2107999999999999e-05 2024-09-21T15:00:11.149 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.149 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.149 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:11.149 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625621+0000", 2024-09-21T15:00:11.149 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0762e-05 2024-09-21T15:00:11.149 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.149 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.149 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:11.149 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625692+0000", 2024-09-21T15:00:11.149 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.1203000000000003e-05 2024-09-21T15:00:11.149 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.149 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.149 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:11.150 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625872+0000", 2024-09-21T15:00:11.150 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017962800000000001 2024-09-21T15:00:11.150 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.150 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.150 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T15:00:11.150 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.633188+0000", 2024-09-21T15:00:11.150 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.007316014 2024-09-21T15:00:11.150 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.150 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.150 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:11.150 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.668357+0000", 2024-09-21T15:00:11.150 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035168933999999999 2024-09-21T15:00:11.150 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.151 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.151 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:11.151 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.668418+0000", 2024-09-21T15:00:11.151 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1282e-05 2024-09-21T15:00:11.151 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.151 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.151 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:11.151 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.668486+0000", 2024-09-21T15:00:11.151 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8468999999999998e-05 2024-09-21T15:00:11.151 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.151 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.151 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:11.151 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.668544+0000", 2024-09-21T15:00:11.152 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7785000000000003e-05 2024-09-21T15:00:11.152 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:11.152 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:11.152 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:11.152 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.152 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.152 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22741 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 245760~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T15:00:11.152 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141107+0000", 2024-09-21T15:00:11.152 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 80.990401495, 2024-09-21T15:00:11.152 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036407729, 2024-09-21T15:00:11.152 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:11.152 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:11.152 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:11.153 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:11.153 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:11.153 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22741 2024-09-21T15:00:11.153 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.153 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:11.153 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.153 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:11.153 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141107+0000", 2024-09-21T15:00:11.153 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:11.153 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.153 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.153 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:11.153 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141107+0000", 2024-09-21T15:00:11.153 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:11.154 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.154 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.154 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:11.154 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141107+0000", 2024-09-21T15:00:11.154 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:11.154 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.154 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.154 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:11.154 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141109+0000", 2024-09-21T15:00:11.154 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6529999999999998e-06 2024-09-21T15:00:11.154 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.154 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.154 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:11.155 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141115+0000", 2024-09-21T15:00:11.155 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9649999999999996e-06 2024-09-21T15:00:11.155 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.155 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.155 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:11.155 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141121+0000", 2024-09-21T15:00:11.155 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1209999999999998e-06 2024-09-21T15:00:11.155 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.155 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.155 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:11.155 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177242+0000", 2024-09-21T15:00:11.155 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036121706000000003 2024-09-21T15:00:11.156 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.156 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.156 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:11.156 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177301+0000", 2024-09-21T15:00:11.156 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9008000000000002e-05 2024-09-21T15:00:11.156 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.156 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.156 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:11.156 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177515+0000", 2024-09-21T15:00:11.156 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00021362 2024-09-21T15:00:11.156 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:11.156 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:11.156 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:11.157 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.157 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.157 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22742 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 311296~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T15:00:11.157 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141126+0000", 2024-09-21T15:00:11.157 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 80.990383140999995, 2024-09-21T15:00:11.157 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036653537, 2024-09-21T15:00:11.157 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:11.157 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:11.157 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:11.157 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:11.157 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:11.157 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22742 2024-09-21T15:00:11.157 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.158 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:11.158 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.158 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:11.158 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141126+0000", 2024-09-21T15:00:11.158 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:11.158 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.158 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.158 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:11.158 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141126+0000", 2024-09-21T15:00:11.158 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:11.158 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.158 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.158 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:11.159 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141125+0000", 2024-09-21T15:00:11.159 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:11.159 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.159 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.159 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:11.159 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141128+0000", 2024-09-21T15:00:11.159 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.886e-06 2024-09-21T15:00:11.159 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.159 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.159 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:11.159 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141137+0000", 2024-09-21T15:00:11.159 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4399999999999994e-06 2024-09-21T15:00:11.159 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.160 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.160 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:11.160 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141143+0000", 2024-09-21T15:00:11.160 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1410000000000004e-06 2024-09-21T15:00:11.160 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.160 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.160 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:11.160 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177543+0000", 2024-09-21T15:00:11.160 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036400544999999999 2024-09-21T15:00:11.160 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.160 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.160 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:11.160 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177605+0000", 2024-09-21T15:00:11.160 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2206999999999996e-05 2024-09-21T15:00:11.161 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.161 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.161 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:11.161 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177779+0000", 2024-09-21T15:00:11.161 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017368099999999999 2024-09-21T15:00:11.161 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:11.161 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:11.161 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:11.161 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.161 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.161 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22743 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 376832~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T15:00:11.161 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141147+0000", 2024-09-21T15:00:11.161 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 80.990361656999994, 2024-09-21T15:00:11.162 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036915158000000003, 2024-09-21T15:00:11.162 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:11.162 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:11.162 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:11.162 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:11.162 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:11.162 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22743 2024-09-21T15:00:11.162 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.162 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:11.162 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.162 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:11.162 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141147+0000", 2024-09-21T15:00:11.162 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:11.163 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.163 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.163 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:11.163 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141147+0000", 2024-09-21T15:00:11.163 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:11.163 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.163 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.163 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:11.163 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141147+0000", 2024-09-21T15:00:11.163 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:11.163 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.163 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.163 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:11.163 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141149+0000", 2024-09-21T15:00:11.164 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.79e-06 2024-09-21T15:00:11.164 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.164 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.164 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:11.164 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141155+0000", 2024-09-21T15:00:11.164 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6699999999999999e-06 2024-09-21T15:00:11.164 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.164 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.164 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:11.164 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141160+0000", 2024-09-21T15:00:11.164 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5759999999999999e-06 2024-09-21T15:00:11.164 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.165 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.165 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:11.165 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177817+0000", 2024-09-21T15:00:11.165 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036657404999999997 2024-09-21T15:00:11.165 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.165 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.165 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:11.165 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177892+0000", 2024-09-21T15:00:11.165 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4602000000000002e-05 2024-09-21T15:00:11.165 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.165 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.165 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:11.165 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178062+0000", 2024-09-21T15:00:11.165 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000170464 2024-09-21T15:00:11.166 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:11.166 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:11.166 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:11.166 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.166 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.166 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22744 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 442368~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T15:00:11.166 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141164+0000", 2024-09-21T15:00:11.166 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 80.990344523999994, 2024-09-21T15:00:11.166 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037120806999999999, 2024-09-21T15:00:11.166 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:11.166 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:11.166 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:11.166 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:11.167 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:11.167 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22744 2024-09-21T15:00:11.167 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.167 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:11.167 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.167 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:11.167 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141164+0000", 2024-09-21T15:00:11.167 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:11.167 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.167 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.167 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:11.167 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141164+0000", 2024-09-21T15:00:11.167 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:11.167 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.168 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.168 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:11.168 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141164+0000", 2024-09-21T15:00:11.168 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:11.168 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.168 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.168 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:11.168 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141167+0000", 2024-09-21T15:00:11.168 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7159999999999999e-06 2024-09-21T15:00:11.168 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.168 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.168 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:11.168 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141173+0000", 2024-09-21T15:00:11.169 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.071e-06 2024-09-21T15:00:11.169 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.169 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.169 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:11.169 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141177+0000", 2024-09-21T15:00:11.169 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3610000000000003e-06 2024-09-21T15:00:11.169 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.169 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.169 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:11.169 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178083+0000", 2024-09-21T15:00:11.169 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036906405000000003 2024-09-21T15:00:11.169 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.169 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.169 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:11.170 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178143+0000", 2024-09-21T15:00:11.170 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0056999999999998e-05 2024-09-21T15:00:11.170 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.170 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.170 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:11.170 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178285+0000", 2024-09-21T15:00:11.170 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000141555 2024-09-21T15:00:11.170 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:11.170 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:11.170 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:11.170 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.170 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.170 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22745 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 507904~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T15:00:11.171 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141182+0000", 2024-09-21T15:00:11.171 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 80.990326203999999, 2024-09-21T15:00:11.171 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037332331000000003, 2024-09-21T15:00:11.171 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:11.171 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:11.171 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:11.171 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:11.171 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:11.171 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22745 2024-09-21T15:00:11.171 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.171 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:11.171 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.172 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:11.172 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141182+0000", 2024-09-21T15:00:11.172 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:11.172 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.172 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.172 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:11.172 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141182+0000", 2024-09-21T15:00:11.172 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:11.172 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.172 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.172 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:11.172 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141182+0000", 2024-09-21T15:00:11.172 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:11.173 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.173 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.173 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:11.173 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141185+0000", 2024-09-21T15:00:11.173 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.464e-06 2024-09-21T15:00:11.173 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.173 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.173 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:11.173 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141200+0000", 2024-09-21T15:00:11.173 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5886999999999999e-05 2024-09-21T15:00:11.173 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.173 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.173 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:11.174 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141571+0000", 2024-09-21T15:00:11.174 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000370909 2024-09-21T15:00:11.174 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.174 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.174 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:11.174 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178318+0000", 2024-09-21T15:00:11.174 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036746244999999997 2024-09-21T15:00:11.174 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.174 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.174 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:11.174 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178367+0000", 2024-09-21T15:00:11.174 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9317000000000002e-05 2024-09-21T15:00:11.174 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.175 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.175 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:11.175 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178515+0000", 2024-09-21T15:00:11.175 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014784600000000001 2024-09-21T15:00:11.175 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:11.175 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:11.175 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:11.175 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.175 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.175 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22746 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 573440~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T15:00:11.175 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141599+0000", 2024-09-21T15:00:11.175 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 80.989909572000002, 2024-09-21T15:00:11.175 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037402249999999998, 2024-09-21T15:00:11.176 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:11.176 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:11.176 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:11.176 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:11.176 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:11.176 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22746 2024-09-21T15:00:11.176 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.176 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:11.176 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.176 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:11.176 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141599+0000", 2024-09-21T15:00:11.176 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:11.176 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.176 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.177 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:11.177 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141599+0000", 2024-09-21T15:00:11.177 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:11.177 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.177 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.177 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:11.177 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141598+0000", 2024-09-21T15:00:11.177 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:00:11.177 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.177 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.177 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:11.177 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141637+0000", 2024-09-21T15:00:11.177 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.9119999999999998e-05 2024-09-21T15:00:11.178 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.178 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.178 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:11.178 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141647+0000", 2024-09-21T15:00:11.178 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5570000000000001e-06 2024-09-21T15:00:11.178 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.178 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.178 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:11.178 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141654+0000", 2024-09-21T15:00:11.178 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2049999999999996e-06 2024-09-21T15:00:11.178 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.179 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.179 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:11.179 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178553+0000", 2024-09-21T15:00:11.179 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036898528 2024-09-21T15:00:11.179 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.179 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.179 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:11.179 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178625+0000", 2024-09-21T15:00:11.179 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2669999999999994e-05 2024-09-21T15:00:11.179 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.179 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.179 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:11.179 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179001+0000", 2024-09-21T15:00:11.180 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000375977 2024-09-21T15:00:11.180 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:11.180 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:11.180 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:11.180 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.180 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.180 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22747 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 638976~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T15:00:11.180 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141660+0000", 2024-09-21T15:00:11.180 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 80.989848429999995, 2024-09-21T15:00:11.180 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037617352999999999, 2024-09-21T15:00:11.180 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:11.180 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:11.180 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:11.181 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:11.181 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:11.181 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22747 2024-09-21T15:00:11.181 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.181 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:11.181 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.181 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:11.181 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141660+0000", 2024-09-21T15:00:11.181 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:11.181 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.181 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.181 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:11.181 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141660+0000", 2024-09-21T15:00:11.182 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:11.182 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.182 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.182 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:11.182 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141660+0000", 2024-09-21T15:00:11.182 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:11.182 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.182 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.182 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:11.182 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141663+0000", 2024-09-21T15:00:11.182 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.569e-06 2024-09-21T15:00:11.182 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.182 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.183 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:11.183 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141670+0000", 2024-09-21T15:00:11.183 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5719999999999999e-06 2024-09-21T15:00:11.183 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.183 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.183 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:11.183 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141675+0000", 2024-09-21T15:00:11.183 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2889999999999997e-06 2024-09-21T15:00:11.183 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.183 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.183 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:11.183 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179028+0000", 2024-09-21T15:00:11.183 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037353381999999997 2024-09-21T15:00:11.183 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.184 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.184 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:11.184 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179099+0000", 2024-09-21T15:00:11.184 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0259000000000006e-05 2024-09-21T15:00:11.184 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.184 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.184 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:11.184 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179278+0000", 2024-09-21T15:00:11.184 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000178908 2024-09-21T15:00:11.184 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:11.184 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:11.184 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:11.185 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.185 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.185 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22748 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 704512~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T15:00:11.185 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141680+0000", 2024-09-21T15:00:11.185 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 80.989828791999997, 2024-09-21T15:00:11.185 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037806166000000002, 2024-09-21T15:00:11.185 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:11.185 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:11.185 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:11.185 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:11.185 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:11.185 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22748 2024-09-21T15:00:11.185 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.186 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:11.186 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.186 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:11.186 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141680+0000", 2024-09-21T15:00:11.186 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:11.186 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.186 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.186 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:11.186 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141680+0000", 2024-09-21T15:00:11.186 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:11.186 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.186 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.186 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:11.187 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141679+0000", 2024-09-21T15:00:11.187 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:11.187 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.187 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.187 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:11.187 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141683+0000", 2024-09-21T15:00:11.187 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3230000000000002e-06 2024-09-21T15:00:11.187 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.187 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.187 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:11.187 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141690+0000", 2024-09-21T15:00:11.187 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8680000000000001e-06 2024-09-21T15:00:11.187 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.188 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.188 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:11.188 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141694+0000", 2024-09-21T15:00:11.188 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6770000000000003e-06 2024-09-21T15:00:11.188 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.188 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.188 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:11.188 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179300+0000", 2024-09-21T15:00:11.188 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037605567999999999 2024-09-21T15:00:11.188 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.188 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.188 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:11.188 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179355+0000", 2024-09-21T15:00:11.188 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4771000000000001e-05 2024-09-21T15:00:11.189 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.189 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.189 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:11.189 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179486+0000", 2024-09-21T15:00:11.189 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013141699999999999 2024-09-21T15:00:11.189 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:11.189 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:11.189 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:11.189 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.189 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.189 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22749 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 770048~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T15:00:11.189 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141698+0000", 2024-09-21T15:00:11.190 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 80.989810419999998, 2024-09-21T15:00:11.190 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037967813000000003, 2024-09-21T15:00:11.190 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:11.190 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:11.190 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:11.190 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:11.190 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:11.190 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22749 2024-09-21T15:00:11.190 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.190 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:11.190 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.190 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:11.191 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141698+0000", 2024-09-21T15:00:11.191 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:11.191 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.191 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.191 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:11.191 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141698+0000", 2024-09-21T15:00:11.191 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:11.191 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.191 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.191 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:11.191 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141698+0000", 2024-09-21T15:00:11.191 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:11.192 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.192 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.192 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:11.192 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141701+0000", 2024-09-21T15:00:11.192 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8399999999999999e-06 2024-09-21T15:00:11.192 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.192 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.192 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:11.192 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141706+0000", 2024-09-21T15:00:11.192 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4979999999999997e-06 2024-09-21T15:00:11.192 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.192 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.192 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:11.193 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141711+0000", 2024-09-21T15:00:11.193 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3850000000000004e-06 2024-09-21T15:00:11.193 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.193 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.193 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:11.193 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179504+0000", 2024-09-21T15:00:11.193 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037793514 2024-09-21T15:00:11.193 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.193 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.193 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:11.193 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179546+0000", 2024-09-21T15:00:11.193 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1467999999999997e-05 2024-09-21T15:00:11.193 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.194 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.194 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:11.194 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179666+0000", 2024-09-21T15:00:11.194 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000120499 2024-09-21T15:00:11.194 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:11.194 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:11.194 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:11.194 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.194 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.194 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22750 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 835584~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T15:00:11.194 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141715+0000", 2024-09-21T15:00:11.194 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 80.989794118999995, 2024-09-21T15:00:11.195 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038173179000000002, 2024-09-21T15:00:11.195 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:11.195 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:11.195 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:11.195 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:11.195 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:11.195 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22750 2024-09-21T15:00:11.195 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.195 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:11.195 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.195 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:11.195 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141715+0000", 2024-09-21T15:00:11.196 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:11.196 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.196 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.196 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:11.196 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141715+0000", 2024-09-21T15:00:11.196 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:11.196 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.196 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.196 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:11.196 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141714+0000", 2024-09-21T15:00:11.196 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:11.196 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.197 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.197 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:11.197 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141716+0000", 2024-09-21T15:00:11.197 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1730000000000002e-06 2024-09-21T15:00:11.197 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.197 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.197 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:11.197 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141722+0000", 2024-09-21T15:00:11.197 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.417e-06 2024-09-21T15:00:11.197 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.197 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.197 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:11.197 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141726+0000", 2024-09-21T15:00:11.198 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4830000000000001e-06 2024-09-21T15:00:11.198 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.198 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.198 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:11.198 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179683+0000", 2024-09-21T15:00:11.198 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037956977000000003 2024-09-21T15:00:11.198 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.198 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.198 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:11.198 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179751+0000", 2024-09-21T15:00:11.198 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7762000000000006e-05 2024-09-21T15:00:11.198 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.198 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.199 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:11.199 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179888+0000", 2024-09-21T15:00:11.199 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000136635 2024-09-21T15:00:11.199 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:11.199 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:11.199 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:11.199 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.199 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.199 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22751 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 901120~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T15:00:11.199 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141730+0000", 2024-09-21T15:00:11.199 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 80.989778319999999, 2024-09-21T15:00:11.199 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038357011000000003, 2024-09-21T15:00:11.199 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:11.200 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:11.200 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:11.200 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:11.200 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:11.200 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22751 2024-09-21T15:00:11.200 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.200 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:11.200 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.200 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:11.200 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141730+0000", 2024-09-21T15:00:11.200 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:11.200 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.200 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.201 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:11.201 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141730+0000", 2024-09-21T15:00:11.201 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:11.201 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.201 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.201 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:11.201 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141730+0000", 2024-09-21T15:00:11.201 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:11.201 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.201 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.201 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:11.201 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141737+0000", 2024-09-21T15:00:11.201 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.64e-06 2024-09-21T15:00:11.202 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.202 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.202 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:11.202 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141743+0000", 2024-09-21T15:00:11.202 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8130000000000001e-06 2024-09-21T15:00:11.202 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.202 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.202 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:11.202 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141747+0000", 2024-09-21T15:00:11.202 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4920000000000004e-06 2024-09-21T15:00:11.202 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.202 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.202 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:11.203 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179918+0000", 2024-09-21T15:00:11.203 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038170822 2024-09-21T15:00:11.203 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.203 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.203 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:11.203 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179960+0000", 2024-09-21T15:00:11.203 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1999000000000002e-05 2024-09-21T15:00:11.203 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.203 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.203 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:11.203 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180087+0000", 2024-09-21T15:00:11.203 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012752799999999999 2024-09-21T15:00:11.203 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:11.204 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:11.204 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:11.204 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.204 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.204 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22752 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 966656~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T15:00:11.204 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141751+0000", 2024-09-21T15:00:11.204 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 80.989757999999995, 2024-09-21T15:00:11.204 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038553084000000001, 2024-09-21T15:00:11.204 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:11.204 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:11.204 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:11.205 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:11.205 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:11.205 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22752 2024-09-21T15:00:11.205 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.205 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:11.205 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.205 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:11.205 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141751+0000", 2024-09-21T15:00:11.205 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:11.205 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.205 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.205 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:11.206 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141751+0000", 2024-09-21T15:00:11.206 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:11.206 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.206 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.206 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:11.206 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141750+0000", 2024-09-21T15:00:11.206 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:11.206 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.206 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.206 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:11.206 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141753+0000", 2024-09-21T15:00:11.206 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5900000000000002e-06 2024-09-21T15:00:11.207 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.207 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.207 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:11.207 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141759+0000", 2024-09-21T15:00:11.207 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.947e-06 2024-09-21T15:00:11.207 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.207 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.207 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:11.207 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141764+0000", 2024-09-21T15:00:11.207 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8319999999999996e-06 2024-09-21T15:00:11.207 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.207 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.207 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:11.208 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180108+0000", 2024-09-21T15:00:11.208 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038344095000000002 2024-09-21T15:00:11.208 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.208 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.208 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:11.208 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180168+0000", 2024-09-21T15:00:11.208 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0047000000000003e-05 2024-09-21T15:00:11.208 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.208 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.208 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:11.208 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180304+0000", 2024-09-21T15:00:11.208 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013592500000000001 2024-09-21T15:00:11.208 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:11.209 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:11.209 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:11.209 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.209 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.209 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22753 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 1032192~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T15:00:11.209 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141768+0000", 2024-09-21T15:00:11.209 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 80.989741125999998, 2024-09-21T15:00:11.209 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038818810000000002, 2024-09-21T15:00:11.209 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:11.209 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:11.209 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:11.209 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:11.209 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:11.210 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22753 2024-09-21T15:00:11.210 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.210 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:11.210 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.210 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:11.210 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141768+0000", 2024-09-21T15:00:11.210 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:11.210 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.210 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.210 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:11.210 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141768+0000", 2024-09-21T15:00:11.210 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:11.211 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.211 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.211 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:11.211 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141767+0000", 2024-09-21T15:00:11.211 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:11.211 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.211 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.211 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:11.211 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141770+0000", 2024-09-21T15:00:11.211 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3329999999999999e-06 2024-09-21T15:00:11.211 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.211 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.211 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:11.211 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141775+0000", 2024-09-21T15:00:11.212 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2669999999999997e-06 2024-09-21T15:00:11.212 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.212 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.212 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:11.212 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141779+0000", 2024-09-21T15:00:11.212 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2259999999999999e-06 2024-09-21T15:00:11.212 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.212 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.212 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:11.212 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180325+0000", 2024-09-21T15:00:11.212 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038546428000000001 2024-09-21T15:00:11.212 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.212 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.213 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:11.213 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180401+0000", 2024-09-21T15:00:11.213 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5437000000000003e-05 2024-09-21T15:00:11.213 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.213 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.213 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:11.213 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180586+0000", 2024-09-21T15:00:11.213 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000185429 2024-09-21T15:00:11.213 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:11.213 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:11.213 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:11.213 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.213 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.214 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:73793 3.11 3:8c8a112d:::rbd_data.1136f4939602.0000000000000003:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-21T15:00:11.214 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:42.225633+0000", 2024-09-21T15:00:11.214 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 28.905875212000002, 2024-09-21T15:00:11.214 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036337102000000003, 2024-09-21T15:00:11.214 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:11.214 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:11.214 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:11.214 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:11.214 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:11.214 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 73793 2024-09-21T15:00:11.214 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.214 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:11.214 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.215 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:11.215 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.225633+0000", 2024-09-21T15:00:11.215 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:11.215 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.215 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.215 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:11.215 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.225633+0000", 2024-09-21T15:00:11.215 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:11.215 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.215 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.215 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:11.215 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.225630+0000", 2024-09-21T15:00:11.215 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999967 2024-09-21T15:00:11.216 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.216 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.216 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:11.216 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.225649+0000", 2024-09-21T15:00:11.216 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9377000000000001e-05 2024-09-21T15:00:11.216 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.216 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.216 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:11.216 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.225689+0000", 2024-09-21T15:00:11.216 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.9879999999999998e-05 2024-09-21T15:00:11.216 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.216 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.216 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:11.217 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.225706+0000", 2024-09-21T15:00:11.217 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6537000000000001e-05 2024-09-21T15:00:11.217 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.217 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.217 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:11.217 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.225772+0000", 2024-09-21T15:00:11.217 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5894000000000003e-05 2024-09-21T15:00:11.217 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.217 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.217 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:11.217 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.225846+0000", 2024-09-21T15:00:11.217 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4165000000000006e-05 2024-09-21T15:00:11.218 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.218 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.218 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 1", 2024-09-21T15:00:11.218 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.231400+0000", 2024-09-21T15:00:11.218 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0055538269999999999 2024-09-21T15:00:11.218 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.218 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.218 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:11.218 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.261709+0000", 2024-09-21T15:00:11.218 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.030309750999999999 2024-09-21T15:00:11.219 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.219 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.219 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:11.219 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.261738+0000", 2024-09-21T15:00:11.219 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8569999999999999e-05 2024-09-21T15:00:11.219 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.219 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.219 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:11.219 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.261839+0000", 2024-09-21T15:00:11.219 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010118 2024-09-21T15:00:11.219 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.219 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.219 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:11.220 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.261971+0000", 2024-09-21T15:00:11.220 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000131414 2024-09-21T15:00:11.220 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:11.220 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:11.220 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:11.220 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.220 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.220 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:97673 3.1d 3:bd91946e:::rbd_data.1136bfc79a70.0000000000000002:head [write 0~4718592 [fadvise_sequential+fadvise_nocache],write 5242880~32768 [fadvise_sequential+fadvise_nocache],write 5341184~147456 [fadvise_sequential+fadvise_nocache],write 5554176~147456 [fadvise_sequential+fadvise_nocache],write 5767168~16384 [fadvise_sequential+fadvise_nocache],write 5849088~147456 [fadvise_sequential+fadvise_nocache],write 6062080~147456 [fadvise_sequential+fadvise_nocache],write 6275072~16384 [fadvise_sequential+fadvise_nocache],write 6356992~147456 [fadvise_sequential+fadvise_nocache],write 6569984~147456 [fadvise_sequential+fadvise_nocache],write 6782976~32768 [fadvise_sequential+fadvise_nocache],write 6864896~147456 [fadvise_sequential+fadvise_nocache],write 7077888~147456 [fadvise_sequential+fadvise_nocache],write 7290880~49152 [fadvise_sequential+fadvise_nocache],write 7372800~147456 [fadvise_sequential+fadvise_nocache],write 7585792~147456 [fadvise_sequential+fadvise_nocache],write 7798784~65536 [fadvise_sequential+fadvise_nocache],write 7880704~147456 [fadvise_sequential+fadvise_nocache],write 8093696~147456 [fadvise_sequential+fadvise_nocache],write 8306688~81920 [fadvise_sequential+fadvise_nocache],write 8421376~114688 [fadvise_sequential+fadvise_nocache],write 8634368~114688 [fadvise_sequential+fadvise_nocache],write 8847360~65536 [fadvise_sequential+fadvise_nocache],write 9011200~32768 [fadvise_sequential+fadvise_nocache],write 9224192~32768 [fadvise_sequential+fadvise_nocache],write 9519104~32768 [fadvise_sequential+fadvise_nocache],write 9732096~32768 [fadvise_sequential+fadvise_nocache],write 9945088~16384 [fadvise_sequential+fadvise_nocache],write 9977856~196608 [fadvise_sequential+fadvise_nocache],write 10190848~196608 [fadvise_sequential+fadvise_nocache],write 10403840~278528 [fadvise_sequential+fadvise_nocache],write 10698752~196608 [fadvise_sequential+fadvise_nocache],write 10911744~278528 [fadvise_sequential+fadvise_nocache],write 11206656~196608 [fadvise_sequential+fadvise_nocache],write 11419648~196608 [fadvise_sequential+fadvise_nocache],write 11632640~65536 [fadvise_sequential+fadvise_nocache],write 11730944~98304 [fadvise_sequential+fadvise_nocache],write 11845632~65536 [fadvise_sequential+fadvise_nocache],write 11943936~98304 [fadvise_sequential+fadvise_nocache],write 12058624~65536 [fadvise_sequential+fadvise_nocache],write 12140544~65536 [fadvise_sequential+fadvise_nocache],write 12238848~98304 [fadvise_sequential+fadvise_nocache],write 12353536~65536 [fadvise_sequential+fadvise_nocache],write 12451840~98304 [fadvise_sequential+fadvise_nocache],write 12566528~65536 [fadvise_sequential+fadvise_nocache],write 12746752~98304 [fadvise_sequential+fadvise_nocache],write 12959744~98304 [fadvise_sequential+fadvise_nocache],write 13107200~32768 [fadvise_sequential+fadvise_nocache],write 13254656~98304 [fadvise_sequential+fadvise_nocache],write 13467648~98304 [fadvise_sequential+fadvise_nocache],write 13631488~16384 [fadvise_sequential+fadvise_nocache],write 13762560~98304 [fadvise_sequential+fadvise_nocache],write 13975552~98304 [fadvise_sequential+fadvise_nocache],write 15204352~49152 [fadvise_sequential+fadvise_nocache],write 15384576~81920 [fadvise_sequential+fadvise_nocache],write 15597568~81920 [fadvise_sequential+fadvise_nocache],write 15728640~32768 [fadvise_sequential+fadvise_nocache],write 15892480~81920 [fadvise_sequential+fadvise_nocache],write 16105472~81920 [fadvise_sequential+fadvise_nocache],write 16252928~16384 [fadvise_sequential+fadvise_nocache],write 16400384~81920 [fadvise_sequential+fadvise_nocache],write 16613376~81920 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:11.220 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.300719+0000", 2024-09-21T15:00:11.220 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 5.8307899379999997, 2024-09-21T15:00:11.220 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.068947114000000004, 2024-09-21T15:00:11.220 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:11.221 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:11.221 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:11.221 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:11.221 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:11.221 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 97673 2024-09-21T15:00:11.221 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.221 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:11.221 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.221 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:11.221 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.300719+0000", 2024-09-21T15:00:11.221 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:11.221 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.222 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.222 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:11.222 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.300719+0000", 2024-09-21T15:00:11.222 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:11.222 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.222 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.222 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:11.222 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.300716+0000", 2024-09-21T15:00:11.222 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-21T15:00:11.222 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.222 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.222 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:11.222 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.308380+0000", 2024-09-21T15:00:11.223 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0076638519999999996 2024-09-21T15:00:11.223 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.223 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.223 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:11.223 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.308418+0000", 2024-09-21T15:00:11.223 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.8380000000000002e-05 2024-09-21T15:00:11.223 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.223 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.223 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:11.223 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.308431+0000", 2024-09-21T15:00:11.223 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3213999999999999e-05 2024-09-21T15:00:11.223 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.224 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.224 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:11.224 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.308562+0000", 2024-09-21T15:00:11.224 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013015999999999999 2024-09-21T15:00:11.224 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.224 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.224 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:11.224 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.309173+0000", 2024-09-21T15:00:11.224 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00061185500000000002 2024-09-21T15:00:11.224 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.224 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.224 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 0", 2024-09-21T15:00:11.224 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311612+0000", 2024-09-21T15:00:11.225 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0024383949999999999 2024-09-21T15:00:11.225 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.225 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.225 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:11.225 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.355482+0000", 2024-09-21T15:00:11.225 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.043869940000000003 2024-09-21T15:00:11.225 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.225 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.225 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:11.225 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.369306+0000", 2024-09-21T15:00:11.225 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013824279 2024-09-21T15:00:11.225 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.225 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.225 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:11.226 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.369575+0000", 2024-09-21T15:00:11.226 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000269404 2024-09-21T15:00:11.226 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.226 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.226 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:11.226 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.369666+0000", 2024-09-21T15:00:11.226 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0426000000000005e-05 2024-09-21T15:00:11.226 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:11.226 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:11.226 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:11.226 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.227 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.227 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:97678 3.d 3:b2f34c6d:::rbd_data.1136bfc79a70.0000000000000006:head [write 0~4194304 [fadvise_sequential+fadvise_nocache],write 4718592~81920 [fadvise_sequential+fadvise_nocache],write 4866048~147456 [fadvise_sequential+fadvise_nocache],write 5079040~147456 [fadvise_sequential+fadvise_nocache],write 5242880~65536 [fadvise_sequential+fadvise_nocache],write 5373952~147456 [fadvise_sequential+fadvise_nocache],write 5586944~147456 [fadvise_sequential+fadvise_nocache],write 5767168~49152 [fadvise_sequential+fadvise_nocache],write 5881856~147456 [fadvise_sequential+fadvise_nocache],write 6094848~147456 [fadvise_sequential+fadvise_nocache],write 6291456~32768 [fadvise_sequential+fadvise_nocache],write 6389760~147456 [fadvise_sequential+fadvise_nocache],write 6602752~147456 [fadvise_sequential+fadvise_nocache],write 6815744~16384 [fadvise_sequential+fadvise_nocache],write 6897664~147456 [fadvise_sequential+fadvise_nocache],write 7110656~147456 [fadvise_sequential+fadvise_nocache],write 7323648~16384 [fadvise_sequential+fadvise_nocache],write 7405568~147456 [fadvise_sequential+fadvise_nocache],write 7618560~147456 [fadvise_sequential+fadvise_nocache],write 7831552~32768 [fadvise_sequential+fadvise_nocache],write 7946240~114688 [fadvise_sequential+fadvise_nocache],write 8159232~114688 [fadvise_sequential+fadvise_nocache],write 8372224~16384 [fadvise_sequential+fadvise_nocache],write 8454144~114688 [fadvise_sequential+fadvise_nocache],write 8667136~114688 [fadvise_sequential+fadvise_nocache],write 8880128~32768 [fadvise_sequential+fadvise_nocache],write 9043968~32768 [fadvise_sequential+fadvise_nocache],write 9256960~32768 [fadvise_sequential+fadvise_nocache],write 9551872~32768 [fadvise_sequential+fadvise_nocache],write 9764864~32768 [fadvise_sequential+fadvise_nocache],write 9961472~32768 [fadvise_sequential+fadvise_nocache],write 10010624~196608 [fadvise_sequential+fadvise_nocache],write 10223616~196608 [fadvise_sequential+fadvise_nocache],write 10436608~65536 [fadvise_sequential+fadvise_nocache],write 10518528~196608 [fadvise_sequential+fadvise_nocache],write 10731520~196608 [fadvise_sequential+fadvise_nocache],write 10944512~65536 [fadvise_sequential+fadvise_nocache],write 11026432~196608 [fadvise_sequential+fadvise_nocache],write 11239424~196608 [fadvise_sequential+fadvise_nocache],write 11452416~81920 [fadvise_sequential+fadvise_nocache],write 11550720~98304 [fadvise_sequential+fadvise_nocache],write 11665408~65536 [fadvise_sequential+fadvise_nocache],write 11763712~98304 [fadvise_sequential+fadvise_nocache],write 11878400~65536 [fadvise_sequential+fadvise_nocache],write 11976704~180224 [fadvise_sequential+fadvise_nocache],write 12173312~65536 [fadvise_sequential+fadvise_nocache],write 12271616~98304 [fadvise_sequential+fadvise_nocache],write 12386304~65536 [fadvise_sequential+fadvise_nocache],write 12484608~180224 [fadvise_sequential+fadvise_nocache],write 12779520~98304 [fadvise_sequential+fadvise_nocache],write 12992512~98304 [fadvise_sequential+fadvise_nocache],write 13107200~65536 [fadvise_sequential+fadvise_nocache],write 13287424~98304 [fadvise_sequential+fadvise_nocache],write 13500416~98304 [fadvise_sequential+fadvise_nocache],write 13631488~49152 [fadvise_sequential+fadvise_nocache],write 13795328~98304 [fadvise_sequential+fadvise_nocache],write 14008320~98304 [fadvise_sequential+fadvise_nocache],write 14696448~81920 [fadvise_sequential+fadvise_nocache],write 14909440~81920 [fadvise_sequential+fadvise_nocache],write 15122432~163840 [fadvise_sequential+fadvise_nocache],write 15417344~81920 [fadvise_sequential+fadvise_nocache],write 15630336~81920 [fadvise_sequential+fadvise_nocache],write 15728640~65536 [fadvise_sequential+fadvise_nocache],write 15925248~81920 [fadvise_sequential+fadvise_nocache],write 16138240~81920 [fadvise_sequential+fadvise_nocache],write 16252928~49152 [fadvise_sequential+fadvise_nocache],write 16433152~81920 [fadvise_sequential+fadvise_nocache],write 16646144~81920 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:11.227 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.308471+0000", 2024-09-21T15:00:11.227 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 5.8230374869999997, 2024-09-21T15:00:11.227 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.047271776000000001, 2024-09-21T15:00:11.227 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:11.227 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:11.227 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:11.227 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:11.227 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:11.227 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 97678 2024-09-21T15:00:11.227 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.227 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:11.228 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.228 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:11.228 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.308471+0000", 2024-09-21T15:00:11.228 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:11.228 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.228 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.228 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:11.228 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.308471+0000", 2024-09-21T15:00:11.228 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:11.228 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.228 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.228 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:11.229 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.308469+0000", 2024-09-21T15:00:11.229 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-21T15:00:11.229 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.229 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.229 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:11.229 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.317731+0000", 2024-09-21T15:00:11.229 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0092614949999999998 2024-09-21T15:00:11.229 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.229 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.229 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:11.229 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.317778+0000", 2024-09-21T15:00:11.229 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7574000000000002e-05 2024-09-21T15:00:11.230 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.230 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.230 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:11.230 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.317794+0000", 2024-09-21T15:00:11.230 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5537e-05 2024-09-21T15:00:11.230 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.230 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.230 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:11.230 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.317871+0000", 2024-09-21T15:00:11.230 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7163999999999995e-05 2024-09-21T15:00:11.230 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.230 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.231 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:11.231 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.318166+0000", 2024-09-21T15:00:11.231 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00029492400000000001 2024-09-21T15:00:11.231 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.231 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.231 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 1", 2024-09-21T15:00:11.231 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.320083+0000", 2024-09-21T15:00:11.231 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001916933 2024-09-21T15:00:11.231 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.231 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.231 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:11.231 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.354762+0000", 2024-09-21T15:00:11.231 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034679481999999998 2024-09-21T15:00:11.232 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.232 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.232 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:11.232 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.355494+0000", 2024-09-21T15:00:11.232 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00073167399999999997 2024-09-21T15:00:11.232 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.232 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.232 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:11.232 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.355683+0000", 2024-09-21T15:00:11.232 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00018951699999999999 2024-09-21T15:00:11.232 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.232 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.232 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:11.233 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.355743+0000", 2024-09-21T15:00:11.233 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9493000000000001e-05 2024-09-21T15:00:11.233 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:11.233 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:11.233 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:11.233 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.233 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.233 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:98562 3.9 3:911e52bc:::rbd_data.1136bfc79a70.000000000000000a:head [write 0~4194304 [fadvise_sequential+fadvise_nocache],write 4718592~114688 [fadvise_sequential+fadvise_nocache],write 4898816~147456 [fadvise_sequential+fadvise_nocache],write 5111808~229376 [fadvise_sequential+fadvise_nocache],write 5406720~147456 [fadvise_sequential+fadvise_nocache],write 5619712~229376 [fadvise_sequential+fadvise_nocache],write 5914624~147456 [fadvise_sequential+fadvise_nocache],write 6127616~147456 [fadvise_sequential+fadvise_nocache],write 6291456~65536 [fadvise_sequential+fadvise_nocache],write 6422528~147456 [fadvise_sequential+fadvise_nocache],write 6635520~147456 [fadvise_sequential+fadvise_nocache],write 6815744~49152 [fadvise_sequential+fadvise_nocache],write 6930432~147456 [fadvise_sequential+fadvise_nocache],write 7143424~147456 [fadvise_sequential+fadvise_nocache],write 7340032~32768 [fadvise_sequential+fadvise_nocache],write 7438336~147456 [fadvise_sequential+fadvise_nocache],write 7651328~147456 [fadvise_sequential+fadvise_nocache],write 7864320~16384 [fadvise_sequential+fadvise_nocache],write 7979008~114688 [fadvise_sequential+fadvise_nocache],write 8192000~114688 [fadvise_sequential+fadvise_nocache],write 8519680~81920 [fadvise_sequential+fadvise_nocache],write 8732672~38263 [fadvise_sequential+fadvise_nocache],write 8781824~32768 [fadvise_sequential+fadvise_nocache],write 9076736~32768 [fadvise_sequential+fadvise_nocache],write 9289728~32768 [fadvise_sequential+fadvise_nocache],write 9437184~81920 [fadvise_sequential+fadvise_nocache],write 9535488~196608 [fadvise_sequential+fadvise_nocache],write 9748480~196608 [fadvise_sequential+fadvise_nocache],write 9961472~65536 [fadvise_sequential+fadvise_nocache],write 10043392~196608 [fadvise_sequential+fadvise_nocache],write 10256384~196608 [fadvise_sequential+fadvise_nocache],write 10469376~65536 [fadvise_sequential+fadvise_nocache],write 10551296~196608 [fadvise_sequential+fadvise_nocache],write 10764288~196608 [fadvise_sequential+fadvise_nocache],write 10977280~65536 [fadvise_sequential+fadvise_nocache],write 11075584~98304 [fadvise_sequential+fadvise_nocache],write 11190272~65536 [fadvise_sequential+fadvise_nocache],write 11288576~98304 [fadvise_sequential+fadvise_nocache],write 11403264~65536 [fadvise_sequential+fadvise_nocache],write 11501568~49152 [fadvise_sequential+fadvise_nocache],write 11583488~98304 [fadvise_sequential+fadvise_nocache],write 11698176~65536 [fadvise_sequential+fadvise_nocache],write 11796480~98304 [fadvise_sequential+fadvise_nocache],write 11911168~65536 [fadvise_sequential+fadvise_nocache],write 12009472~49152 [fadvise_sequential+fadvise_nocache],write 12091392~98304 [fadvise_sequential+fadvise_nocache],write 12304384~98304 [fadvise_sequential+fadvise_nocache],write 12517376~65536 [fadvise_sequential+fadvise_nocache],write 12599296~98304 [fadvise_sequential+fadvise_nocache],write 12812288~98304 [fadvise_sequential+fadvise_nocache],write 13025280~180224 [fadvise_sequential+fadvise_nocache],write 13320192~98304 [fadvise_sequential+fadvise_nocache],write 13533184~180224 [fadvise_sequential+fadvise_nocache],write 13828096~98304 [fadvise_sequential+fadvise_nocache],write 14041088~98304 [fadvise_sequential+fadvise_nocache],write 14729216~81920 [fadvise_sequential+fadvise_nocache],write 14942208~81920 [fadvise_sequential+fadvise_nocache],write 15155200~49152 [fadvise_sequential+fadvise_nocache],write 15237120~81920 [fadvise_sequential+fadvise_nocache],write 15450112~81920 [fadvise_sequential+fadvise_nocache],write 15663104~65536 [fadvise_sequential+fadvise_nocache],write 15745024~81920 [fadvise_sequential+fadvise_nocache],write 15958016~81920 [fadvise_sequential+fadvise_nocache],write 16171008~163840 [fadvise_sequential+fadvise_nocache],write 16465920~81920 [fadvise_sequential+fadvise_nocache],write 16678912~81920 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:11.233 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.422553+0000", 2024-09-21T15:00:11.233 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 5.7089557810000002, 2024-09-21T15:00:11.233 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042104691, 2024-09-21T15:00:11.233 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:11.234 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:11.234 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:11.234 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:11.234 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:11.234 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 98562 2024-09-21T15:00:11.234 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.234 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:11.234 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.234 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:11.234 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.422553+0000", 2024-09-21T15:00:11.234 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:11.234 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.234 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.235 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:11.235 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.422553+0000", 2024-09-21T15:00:11.235 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:11.235 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.235 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.235 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:11.235 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.422551+0000", 2024-09-21T15:00:11.235 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-21T15:00:11.235 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.235 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.235 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:11.235 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.429465+0000", 2024-09-21T15:00:11.235 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0069139550000000003 2024-09-21T15:00:11.236 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.236 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.236 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:11.236 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.429847+0000", 2024-09-21T15:00:11.236 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00038256 2024-09-21T15:00:11.236 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.236 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.236 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:11.236 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.429866+0000", 2024-09-21T15:00:11.236 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8051e-05 2024-09-21T15:00:11.236 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.236 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.236 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:11.237 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.429946+0000", 2024-09-21T15:00:11.237 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0761999999999997e-05 2024-09-21T15:00:11.237 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.237 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.237 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:11.237 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.430257+0000", 2024-09-21T15:00:11.237 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00031120200000000001 2024-09-21T15:00:11.237 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.237 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.237 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 4", 2024-09-21T15:00:11.237 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.432045+0000", 2024-09-21T15:00:11.237 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0017872879999999999 2024-09-21T15:00:11.237 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.237 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.238 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:11.238 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.451614+0000", 2024-09-21T15:00:11.238 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019569521999999999 2024-09-21T15:00:11.238 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.238 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.238 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:11.238 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.464472+0000", 2024-09-21T15:00:11.238 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012857435 2024-09-21T15:00:11.238 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.238 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.238 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:11.238 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.464613+0000", 2024-09-21T15:00:11.238 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014159700000000001 2024-09-21T15:00:11.239 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:11.239 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:11.239 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:11.239 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.464658+0000", 2024-09-21T15:00:11.239 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4268999999999999e-05 2024-09-21T15:00:11.239 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:11.239 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:11.239 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:11.239 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:11.239 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:11.239 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:11.239 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 5, 6, 7, 4] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T15:00:11.240 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T15:00:11.240 INFO:tasks.thrashosds.thrasher:inject_pause on osd.3 2024-09-21T15:00:11.240 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-21T15:00:11.240 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-21T15:00:11.240 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok config set filestore_inject_stall 3 2024-09-21T15:00:11.241 INFO:teuthology.orchestra.run.smithi032.stdout:3223 write 0xe4659c1 thru 0xe46f888 (0x9ec8 bytes) 2024-09-21T15:00:11.242 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T15:00:11.243 INFO:teuthology.orchestra.run.smithi032.stdout:3224 trunc from 0xe46f889 to 0xba1d492 2024-09-21T15:00:11.243 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:11.243 INFO:teuthology.orchestra.run.smithi032.stdout:3230 writesame 0xad0c27c thru 0xad0dfe1 data_size 0x2(0x1d66 bytes) 2024-09-21T15:00:11.243 INFO:teuthology.orchestra.run.smithi032.stdout:3231 trunc from 0xba1d492 to 0xb2d703b 2024-09-21T15:00:11.244 INFO:teuthology.orchestra.run.smithi032.stderr:noscrub is unset 2024-09-21T15:00:11.245 INFO:teuthology.orchestra.run.smithi032.stdout:3233 punch from 0x399b12f to 0x399cd00, (0x1bd1 bytes) 2024-09-21T15:00:11.245 INFO:teuthology.orchestra.run.smithi032.stdout:3235 trunc from 0xb2d703b to 0x8b82405 2024-09-21T15:00:11.245 INFO:teuthology.orchestra.run.smithi032.stdout:3236 punch from 0x59dd4b9 to 0x59e39a3, (0x64ea bytes) 2024-09-21T15:00:11.245 INFO:teuthology.orchestra.run.smithi032.stdout:3237 read 0x7f2885a thru 0x7f2a8ca (0x2071 bytes) 2024-09-21T15:00:11.245 INFO:teuthology.orchestra.run.smithi032.stdout:3238 writesame 0xb06c930 thru 0xb074c4a data_size 0x1(0x831b bytes) 2024-09-21T15:00:11.245 INFO:teuthology.orchestra.run.smithi032.stdout:3240 write 0x7b8ca9f thru 0x7b907a8 (0x3d0a bytes) 2024-09-21T15:00:11.245 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:00:11.234+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:00:11.249 INFO:teuthology.orchestra.run.smithi032.stdout:3241 trunc from 0xb074c4b to 0xb9b7e58 2024-09-21T15:00:11.262 INFO:teuthology.orchestra.run.smithi032.stdout:3242 read 0x394a3f thru 0x3a31b7 (0xe779 bytes) 2024-09-21T15:00:11.283 INFO:teuthology.orchestra.run.smithi032.stdout:3243 trunc from 0xb9b7e58 to 0x252797f 2024-09-21T15:00:11.310 INFO:teuthology.orchestra.run.smithi032.stdout:3244 punch from 0x2356fff to 0x2364652, (0xd653 bytes) 2024-09-21T15:00:11.336 INFO:teuthology.orchestra.run.smithi032.stdout:3245 write 0xc017a6d thru 0xc01bdac (0x4340 bytes) 2024-09-21T15:00:11.342 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:00:11.340+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:00:11.343 INFO:teuthology.orchestra.run.smithi032.stdout:3248 write 0x82b3d96 thru 0x82c0c7a (0xcee5 bytes) 2024-09-21T15:00:11.352 INFO:teuthology.orchestra.run.smithi032.stdout:3249 read 0x7475639 thru 0x7481dcf (0xc797 bytes) 2024-09-21T15:00:11.352 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:11.352 INFO:teuthology.orchestra.run.smithi032.stdout:3252 punch from 0xa633b0e to 0xa6389af, (0x4ea1 bytes) 2024-09-21T15:00:11.352 INFO:teuthology.orchestra.run.smithi032.stdout:3254 punch from 0x8e498d1 to 0x8e554bb, (0xbbea bytes) 2024-09-21T15:00:11.353 INFO:teuthology.orchestra.run.smithi032.stdout:3255 punch from 0xad10f91 to 0xad1c9d1, (0xba40 bytes) 2024-09-21T15:00:11.353 INFO:teuthology.orchestra.run.smithi032.stdout:3256 read 0x8412a04 thru 0x8414b40 (0x213d bytes) 2024-09-21T15:00:11.353 INFO:teuthology.orchestra.run.smithi032.stdout:3257 punch from 0x512cf17 to 0x512fbc7, (0x2cb0 bytes) 2024-09-21T15:00:11.353 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:11.353 INFO:teuthology.orchestra.run.smithi032.stdout:3260 trunc from 0xc01bdad to 0xce75ca9 2024-09-21T15:00:11.363 INFO:teuthology.orchestra.run.smithi032.stdout:3262 read 0x4fdc698 thru 0x4fdd510 (0xe79 bytes) 2024-09-21T15:00:11.365 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:11.365 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "" 2024-09-21T15:00:11.366 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:11.366 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:11.367 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:00:11.367 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T15:00:11.367 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:11.367 INFO:teuthology.orchestra.run.smithi032.stdout:3264 trunc from 0xce75ca9 to 0xe2a1dde 2024-09-21T15:00:11.377 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T15:00:11.389 INFO:teuthology.orchestra.run.smithi032.stdout:3265 trunc from 0xe2a1dde to 0x846d193 2024-09-21T15:00:11.411 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:11.411 INFO:teuthology.orchestra.run.smithi032.stdout:3268 punch from 0x769d12 to 0x771de7, (0x80d5 bytes) 2024-09-21T15:00:11.413 INFO:teuthology.orchestra.run.smithi032.stdout:3269 trunc from 0x846d193 to 0x75d5252 2024-09-21T15:00:11.428 INFO:teuthology.orchestra.run.smithi032.stdout:3272 trunc from 0x75d5252 to 0x811e7b8 2024-09-21T15:00:11.440 INFO:teuthology.orchestra.run.smithi032.stdout:3273 write 0x57464bd thru 0x574b780 (0x52c4 bytes) 2024-09-21T15:00:11.443 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:00:11.438+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:00:11.447 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:11.447 INFO:teuthology.orchestra.run.smithi032.stdout:3275 punch from 0x4ba7b78 to 0x4baa684, (0x2b0c bytes) 2024-09-21T15:00:11.447 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:11.468 INFO:teuthology.orchestra.run.smithi032.stdout:3277 write 0xbd04d70 thru 0xbd05a87 (0xd18 bytes) 2024-09-21T15:00:11.474 INFO:teuthology.orchestra.run.smithi032.stdout:3278 write 0x80cfc8a thru 0x80d7b32 (0x7ea9 bytes) 2024-09-21T15:00:11.481 INFO:teuthology.orchestra.run.smithi032.stdout:3281 write 0xa2e3bc0 thru 0xa2e46a2 (0xae3 bytes) 2024-09-21T15:00:11.486 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:11.486 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:00:11.486 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T15:00:11.486 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T15:00:11.486 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:11.496 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T15:00:11.500 INFO:teuthology.orchestra.run.smithi032.stdout:3283 write 0xc58c843 thru 0xc58ee04 (0x25c2 bytes) 2024-09-21T15:00:11.507 INFO:teuthology.orchestra.run.smithi032.stdout:3284 read 0x25eb4a6 thru 0x25eec0b (0x3766 bytes) 2024-09-21T15:00:11.508 INFO:teuthology.orchestra.run.smithi032.stdout:3285 write 0xad5b967 thru 0xad6479c (0x8e36 bytes) 2024-09-21T15:00:11.515 INFO:teuthology.orchestra.run.smithi032.stdout:3286 write 0x3ae6fad thru 0x3aee659 (0x76ad bytes) 2024-09-21T15:00:11.530 INFO:teuthology.orchestra.run.smithi032.stdout:3290 trunc from 0xc58ee05 to 0x82c8c28 2024-09-21T15:00:11.544 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:00:11.540+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:00:11.547 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T15:00:11.555 INFO:teuthology.orchestra.run.smithi032.stdout:3291 read 0x1b48c1e thru 0x1b498e8 (0xccb bytes) 2024-09-21T15:00:11.570 INFO:teuthology.orchestra.run.smithi032.stdout:3293 write 0x9aa6e7b thru 0x9aa8319 (0x149f bytes) 2024-09-21T15:00:11.595 INFO:teuthology.orchestra.run.smithi032.stdout:3294 trunc from 0x9aa831a to 0x73938e 2024-09-21T15:00:11.605 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:11.605 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T15:00:11.605 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T15:00:11.605 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T15:00:11.605 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.605 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:1452 3.19 3:9d40fe72:::rbd_data.1136515ecb6b.0000000000000013:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T15:00:11.605 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:30.292755+0000", 2024-09-21T15:00:11.606 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 101.30673081499999, 2024-09-21T15:00:11.606 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043852191999999998, 2024-09-21T15:00:11.606 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:11.606 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:11.606 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:11.606 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:11.606 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:11.606 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1452 2024-09-21T15:00:11.606 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.606 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:11.606 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.606 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:11.606 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292755+0000", 2024-09-21T15:00:11.606 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:11.606 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.607 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.607 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:11.607 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292755+0000", 2024-09-21T15:00:11.607 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:11.607 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.607 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.607 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:11.607 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292754+0000", 2024-09-21T15:00:11.607 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:00:11.607 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.607 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.607 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:11.607 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292760+0000", 2024-09-21T15:00:11.607 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5029999999999997e-06 2024-09-21T15:00:11.607 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.607 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.608 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:11.608 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292771+0000", 2024-09-21T15:00:11.608 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1551e-05 2024-09-21T15:00:11.608 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.608 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.608 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:11.608 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292779+0000", 2024-09-21T15:00:11.608 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1559999999999998e-06 2024-09-21T15:00:11.608 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.608 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.608 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:11.608 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292896+0000", 2024-09-21T15:00:11.608 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000117304 2024-09-21T15:00:11.608 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.608 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.608 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:11.609 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.293124+0000", 2024-09-21T15:00:11.609 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00022819999999999999 2024-09-21T15:00:11.609 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.609 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.609 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 1", 2024-09-21T15:00:11.609 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.299967+0000", 2024-09-21T15:00:11.609 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0068427009999999996 2024-09-21T15:00:11.609 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.609 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.609 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:11.609 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336463+0000", 2024-09-21T15:00:11.609 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.036495842000000001 2024-09-21T15:00:11.609 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.609 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.609 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:11.609 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336480+0000", 2024-09-21T15:00:11.610 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7419999999999999e-05 2024-09-21T15:00:11.610 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.610 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.610 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:11.610 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336550+0000", 2024-09-21T15:00:11.610 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9951999999999997e-05 2024-09-21T15:00:11.610 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.610 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.610 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:11.610 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336607+0000", 2024-09-21T15:00:11.610 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7423000000000002e-05 2024-09-21T15:00:11.610 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:11.610 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:11.610 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:11.610 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.611 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.611 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:1567 3.8 3:139d3d55:::rbd_data.1136515ecb6b.0000000000000000:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T15:00:11.611 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:30.681087+0000", 2024-09-21T15:00:11.611 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 100.9183986, 2024-09-21T15:00:11.611 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.039009224000000002, 2024-09-21T15:00:11.611 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:11.611 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:11.611 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:11.611 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:11.611 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:11.611 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1567 2024-09-21T15:00:11.611 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.611 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:11.611 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.611 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:11.611 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681087+0000", 2024-09-21T15:00:11.612 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:11.612 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.612 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.612 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:11.612 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681087+0000", 2024-09-21T15:00:11.612 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:11.612 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.612 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.612 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:11.612 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681086+0000", 2024-09-21T15:00:11.612 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:00:11.612 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.612 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.612 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:11.612 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681097+0000", 2024-09-21T15:00:11.613 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.134e-05 2024-09-21T15:00:11.613 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.613 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.613 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:11.613 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681122+0000", 2024-09-21T15:00:11.613 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5313e-05 2024-09-21T15:00:11.613 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.613 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.613 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:11.613 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681138+0000", 2024-09-21T15:00:11.613 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5916000000000001e-05 2024-09-21T15:00:11.613 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.613 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.613 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:11.613 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681270+0000", 2024-09-21T15:00:11.614 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000131474 2024-09-21T15:00:11.614 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.614 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.614 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:11.614 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681416+0000", 2024-09-21T15:00:11.614 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014637 2024-09-21T15:00:11.614 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.614 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.614 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 7", 2024-09-21T15:00:11.614 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.693163+0000", 2024-09-21T15:00:11.614 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011747241 2024-09-21T15:00:11.614 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.614 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.614 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:11.614 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.719750+0000", 2024-09-21T15:00:11.615 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.026586852000000001 2024-09-21T15:00:11.615 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.615 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.615 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:11.615 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.719859+0000", 2024-09-21T15:00:11.615 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010925999999999999 2024-09-21T15:00:11.615 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.615 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.615 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:11.615 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.719951+0000", 2024-09-21T15:00:11.615 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1294999999999999e-05 2024-09-21T15:00:11.615 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.615 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.615 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:11.615 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.720097+0000", 2024-09-21T15:00:11.615 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014583 2024-09-21T15:00:11.616 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:11.616 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:11.616 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:11.616 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.616 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.616 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9381 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 303104~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:11.616 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158774+0000", 2024-09-21T15:00:11.616 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 92.440711746000005, 2024-09-21T15:00:11.616 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015309095999999999, 2024-09-21T15:00:11.616 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:11.616 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:11.616 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:11.616 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:11.616 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:11.617 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9381 2024-09-21T15:00:11.617 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.617 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:11.617 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.617 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:11.617 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158774+0000", 2024-09-21T15:00:11.617 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:11.617 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.617 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.617 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:11.617 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158774+0000", 2024-09-21T15:00:11.617 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:11.617 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.617 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.617 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:11.617 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158773+0000", 2024-09-21T15:00:11.618 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:00:11.618 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.618 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.618 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:11.618 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158779+0000", 2024-09-21T15:00:11.618 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6999999999999996e-06 2024-09-21T15:00:11.618 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.618 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.618 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:11.618 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158790+0000", 2024-09-21T15:00:11.618 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0594000000000001e-05 2024-09-21T15:00:11.618 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.618 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.618 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:11.618 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158798+0000", 2024-09-21T15:00:11.619 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5860000000000005e-06 2024-09-21T15:00:11.619 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.619 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.619 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:11.619 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173829+0000", 2024-09-21T15:00:11.619 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015030349 2024-09-21T15:00:11.619 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.619 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.619 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:11.619 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173918+0000", 2024-09-21T15:00:11.619 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9375000000000004e-05 2024-09-21T15:00:11.619 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.619 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.619 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:11.619 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.174083+0000", 2024-09-21T15:00:11.619 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016536200000000001 2024-09-21T15:00:11.620 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:11.620 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:11.620 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:11.620 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.620 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.620 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9382 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 442368~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:11.620 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158805+0000", 2024-09-21T15:00:11.620 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 92.440680841000002, 2024-09-21T15:00:11.620 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015592623, 2024-09-21T15:00:11.620 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:11.620 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:11.620 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:11.620 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:11.620 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:11.620 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9382 2024-09-21T15:00:11.620 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.621 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:11.621 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.621 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:11.621 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158805+0000", 2024-09-21T15:00:11.621 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:11.621 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.621 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.621 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:11.621 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158805+0000", 2024-09-21T15:00:11.621 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:11.621 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.621 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.621 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:11.621 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158805+0000", 2024-09-21T15:00:11.621 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:11.621 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.622 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.622 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:11.622 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158808+0000", 2024-09-21T15:00:11.622 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9179999999999996e-06 2024-09-21T15:00:11.622 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.622 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.622 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:11.622 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158817+0000", 2024-09-21T15:00:11.622 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9299999999999992e-06 2024-09-21T15:00:11.622 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.622 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.622 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:11.622 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158825+0000", 2024-09-21T15:00:11.622 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6739999999999997e-06 2024-09-21T15:00:11.622 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.623 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.623 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:11.623 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.174122+0000", 2024-09-21T15:00:11.623 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.01529694 2024-09-21T15:00:11.623 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.623 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.623 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:11.623 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.174185+0000", 2024-09-21T15:00:11.623 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3015000000000006e-05 2024-09-21T15:00:11.623 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.623 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.623 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:11.623 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.174398+0000", 2024-09-21T15:00:11.623 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021274799999999999 2024-09-21T15:00:11.623 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:11.623 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:11.624 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:11.624 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.624 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.624 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22702 3.19 3:98de45b8:::rbd_data.11366acc894.0000000000000000:19 [read 835584~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T15:00:11.624 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.140232+0000", 2024-09-21T15:00:11.624 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 81.459253558, 2024-09-21T15:00:11.624 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015338742000000001, 2024-09-21T15:00:11.624 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:11.624 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:11.624 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:11.624 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:11.624 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:11.624 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22702 2024-09-21T15:00:11.624 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.624 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:11.624 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.625 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:11.625 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140232+0000", 2024-09-21T15:00:11.625 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:11.625 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.625 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.625 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:11.625 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140232+0000", 2024-09-21T15:00:11.625 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:11.625 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.625 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.625 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:11.625 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140232+0000", 2024-09-21T15:00:11.625 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:11.625 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.625 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.625 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:11.626 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140234+0000", 2024-09-21T15:00:11.626 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.402e-06 2024-09-21T15:00:11.626 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.626 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.626 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:11.626 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140241+0000", 2024-09-21T15:00:11.626 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.054e-06 2024-09-21T15:00:11.626 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.626 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.626 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:11.626 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140246+0000", 2024-09-21T15:00:11.626 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0050000000000004e-06 2024-09-21T15:00:11.626 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.626 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.626 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:11.626 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.154315+0000", 2024-09-21T15:00:11.627 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014069440000000001 2024-09-21T15:00:11.627 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.627 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.627 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:11.627 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.154383+0000", 2024-09-21T15:00:11.627 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8151000000000005e-05 2024-09-21T15:00:11.627 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.627 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.627 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:11.627 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155571+0000", 2024-09-21T15:00:11.627 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001188029 2024-09-21T15:00:11.627 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:11.627 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:11.627 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:11.627 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.627 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.628 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22704 3.19 3:98de45b8:::rbd_data.11366acc894.0000000000000000:19 [read 966656~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T15:00:11.628 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.140266+0000", 2024-09-21T15:00:11.628 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 81.459220083000005, 2024-09-21T15:00:11.628 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015223443999999999, 2024-09-21T15:00:11.628 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:11.628 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:11.628 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:11.628 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:11.628 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:11.628 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22704 2024-09-21T15:00:11.628 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.628 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:11.628 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.628 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:11.628 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140266+0000", 2024-09-21T15:00:11.628 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:11.629 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.629 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.629 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:11.629 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140266+0000", 2024-09-21T15:00:11.629 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:11.629 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.629 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.629 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:11.629 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140266+0000", 2024-09-21T15:00:11.629 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:11.629 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.629 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.629 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:11.629 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140268+0000", 2024-09-21T15:00:11.629 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.475e-06 2024-09-21T15:00:11.629 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.630 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.630 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:11.630 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140274+0000", 2024-09-21T15:00:11.630 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.319e-06 2024-09-21T15:00:11.630 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.630 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.630 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:11.630 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140279+0000", 2024-09-21T15:00:11.630 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.882e-06 2024-09-21T15:00:11.630 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.630 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.630 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:11.630 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155264+0000", 2024-09-21T15:00:11.630 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014984655 2024-09-21T15:00:11.631 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.631 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.631 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:11.631 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155338+0000", 2024-09-21T15:00:11.631 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4091999999999995e-05 2024-09-21T15:00:11.631 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.631 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.631 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:11.631 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155489+0000", 2024-09-21T15:00:11.631 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000151349 2024-09-21T15:00:11.631 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:11.631 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:11.631 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:11.631 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.631 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.631 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22705 3.19 3:98de45b8:::rbd_data.11366acc894.0000000000000000:19 [read 1032192~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T15:00:11.631 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.140284+0000", 2024-09-21T15:00:11.632 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 81.459202226000002, 2024-09-21T15:00:11.632 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015467896, 2024-09-21T15:00:11.632 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:11.632 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:11.632 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:11.632 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:11.632 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:11.632 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22705 2024-09-21T15:00:11.632 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.632 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:11.632 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.632 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:11.632 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140284+0000", 2024-09-21T15:00:11.632 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:11.632 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.633 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.633 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:11.633 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140284+0000", 2024-09-21T15:00:11.633 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:11.633 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.633 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.633 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:11.633 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140283+0000", 2024-09-21T15:00:11.633 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:11.633 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.633 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.633 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:11.633 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140286+0000", 2024-09-21T15:00:11.633 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7130000000000001e-06 2024-09-21T15:00:11.633 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.634 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.634 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:11.634 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140292+0000", 2024-09-21T15:00:11.634 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3430000000000001e-06 2024-09-21T15:00:11.634 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.634 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.634 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:11.634 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140298+0000", 2024-09-21T15:00:11.634 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9730000000000002e-06 2024-09-21T15:00:11.634 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.634 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.634 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:11.634 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155510+0000", 2024-09-21T15:00:11.634 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015211738000000001 2024-09-21T15:00:11.634 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.635 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.635 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:11.635 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155598+0000", 2024-09-21T15:00:11.635 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7822e-05 2024-09-21T15:00:11.635 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.635 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.635 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:11.635 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155752+0000", 2024-09-21T15:00:11.635 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0001537 2024-09-21T15:00:11.635 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:11.635 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:11.635 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:11.635 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.635 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.635 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:37892 3.1b 3:d963c97f:::rbd_data.11365d19aa81.00000000000000be:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-21T15:00:11.635 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:04.029946+0000", 2024-09-21T15:00:11.635 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 67.569540458999995, 2024-09-21T15:00:11.636 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.017686021999999999, 2024-09-21T15:00:11.636 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:11.636 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:11.636 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:11.636 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:11.636 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:11.636 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 37892 2024-09-21T15:00:11.636 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.636 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:11.636 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.636 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:11.636 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.029946+0000", 2024-09-21T15:00:11.636 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:11.636 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.636 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.636 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:11.637 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.029946+0000", 2024-09-21T15:00:11.637 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:11.637 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.637 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.637 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:11.637 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.029944+0000", 2024-09-21T15:00:11.637 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:00:11.637 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.637 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.637 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:11.637 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.029953+0000", 2024-09-21T15:00:11.637 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7589999999999992e-06 2024-09-21T15:00:11.637 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.637 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.637 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:11.638 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.029971+0000", 2024-09-21T15:00:11.638 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7795000000000002e-05 2024-09-21T15:00:11.638 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.638 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.638 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:11.638 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.029984+0000", 2024-09-21T15:00:11.638 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.27e-05 2024-09-21T15:00:11.638 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.638 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.638 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:11.638 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.030072+0000", 2024-09-21T15:00:11.638 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8082000000000001e-05 2024-09-21T15:00:11.638 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.638 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.638 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:11.639 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.030276+0000", 2024-09-21T15:00:11.639 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00020375699999999999 2024-09-21T15:00:11.639 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.639 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.639 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-21T15:00:11.639 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.031634+0000", 2024-09-21T15:00:11.639 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0013586049999999999 2024-09-21T15:00:11.639 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.639 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.639 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:11.639 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.037554+0000", 2024-09-21T15:00:11.639 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0059197449999999997 2024-09-21T15:00:11.639 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.639 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.639 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:11.639 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.047490+0000", 2024-09-21T15:00:11.639 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0099361680000000004 2024-09-21T15:00:11.640 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.640 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.640 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:11.640 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.047569+0000", 2024-09-21T15:00:11.640 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8298000000000001e-05 2024-09-21T15:00:11.640 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.640 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.640 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:11.640 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.047632+0000", 2024-09-21T15:00:11.640 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3006e-05 2024-09-21T15:00:11.640 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:11.640 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:11.640 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:11.640 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.640 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.640 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:63820 3.19 3.626bc079 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-21T15:00:11.641 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:27.241293+0000", 2024-09-21T15:00:11.641 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 44.358193366000002, 2024-09-21T15:00:11.641 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.98723808199999996, 2024-09-21T15:00:11.641 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:11.641 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:11.641 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:11.641 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:11.641 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:11.641 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 63820 2024-09-21T15:00:11.641 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.641 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:11.641 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.641 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:11.641 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241293+0000", 2024-09-21T15:00:11.641 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:11.641 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.641 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.642 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:11.642 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241293+0000", 2024-09-21T15:00:11.642 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:11.642 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.642 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.642 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:11.642 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241291+0000", 2024-09-21T15:00:11.642 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:00:11.642 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.642 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.642 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:11.642 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241299+0000", 2024-09-21T15:00:11.642 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1529999999999997e-06 2024-09-21T15:00:11.642 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.642 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.643 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:11.643 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241317+0000", 2024-09-21T15:00:11.643 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7764000000000001e-05 2024-09-21T15:00:11.643 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.643 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.643 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:11.643 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241328+0000", 2024-09-21T15:00:11.643 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1245e-05 2024-09-21T15:00:11.643 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.643 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.643 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:11.643 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241407+0000", 2024-09-21T15:00:11.643 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8762000000000002e-05 2024-09-21T15:00:11.643 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.643 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.643 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:00:11.644 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241418+0000", 2024-09-21T15:00:11.644 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1695000000000001e-05 2024-09-21T15:00:11.644 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.644 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.644 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:11.644 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:28.228510+0000", 2024-09-21T15:00:11.644 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.98709166100000001 2024-09-21T15:00:11.644 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.644 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.644 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:11.644 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:28.228531+0000", 2024-09-21T15:00:11.644 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0696e-05 2024-09-21T15:00:11.644 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:11.644 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:11.644 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:11.645 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.645 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.645 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:73845 3.6 3:601bd01e:::rbd_data.1136f4939602.0000000000000015:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-21T15:00:11.645 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:42.393435+0000", 2024-09-21T15:00:11.645 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 29.206051254999998, 2024-09-21T15:00:11.645 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.030713345, 2024-09-21T15:00:11.645 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:11.645 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:11.645 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:11.645 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:11.645 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:11.645 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 73845 2024-09-21T15:00:11.645 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.645 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:11.645 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.645 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:11.646 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.393435+0000", 2024-09-21T15:00:11.646 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:11.646 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.646 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.646 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:11.646 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.393435+0000", 2024-09-21T15:00:11.646 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:11.646 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.646 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.646 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:11.646 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.393433+0000", 2024-09-21T15:00:11.646 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:00:11.646 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.646 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.647 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:11.647 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.393446+0000", 2024-09-21T15:00:11.647 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3013000000000001e-05 2024-09-21T15:00:11.647 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.647 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.647 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:11.647 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.393475+0000", 2024-09-21T15:00:11.647 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9615999999999999e-05 2024-09-21T15:00:11.647 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.647 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.647 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:11.647 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.393489+0000", 2024-09-21T15:00:11.647 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4103e-05 2024-09-21T15:00:11.648 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.648 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.648 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:11.648 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.393565+0000", 2024-09-21T15:00:11.648 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5195000000000001e-05 2024-09-21T15:00:11.648 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.648 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.648 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:11.648 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.393734+0000", 2024-09-21T15:00:11.648 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016909999999999999 2024-09-21T15:00:11.648 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.648 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.648 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 7", 2024-09-21T15:00:11.648 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.400243+0000", 2024-09-21T15:00:11.648 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0065093629999999998 2024-09-21T15:00:11.648 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.649 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.649 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:11.649 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.423721+0000", 2024-09-21T15:00:11.649 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.023477984 2024-09-21T15:00:11.649 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.649 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.649 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:11.649 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.423852+0000", 2024-09-21T15:00:11.649 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000131369 2024-09-21T15:00:11.649 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.649 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.649 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:11.649 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.424011+0000", 2024-09-21T15:00:11.649 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000158231 2024-09-21T15:00:11.649 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.650 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.650 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:11.650 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.424148+0000", 2024-09-21T15:00:11.650 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013732699999999999 2024-09-21T15:00:11.650 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:11.650 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:11.650 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:11.650 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.650 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.650 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:85605 3.7 3:e5bd5c60:::rbd_object_map.113637a2ddc2.0000000000000044:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115)", 2024-09-21T15:00:11.650 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:56.162490+0000", 2024-09-21T15:00:11.650 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 15.436995466999999, 2024-09-21T15:00:11.650 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015121122000000001, 2024-09-21T15:00:11.650 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:11.651 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:11.651 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:11.651 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:11.651 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:11.651 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 85605 2024-09-21T15:00:11.651 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.651 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:11.651 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.651 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:11.651 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.162490+0000", 2024-09-21T15:00:11.651 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:11.651 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.651 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.651 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:11.651 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.162490+0000", 2024-09-21T15:00:11.652 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:11.652 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.652 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.652 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:11.652 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.162490+0000", 2024-09-21T15:00:11.652 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:11.652 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.652 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.652 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:11.652 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.162494+0000", 2024-09-21T15:00:11.652 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.5020000000000001e-06 2024-09-21T15:00:11.652 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.652 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.652 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:11.652 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.162498+0000", 2024-09-21T15:00:11.653 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6920000000000001e-06 2024-09-21T15:00:11.653 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.653 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.653 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:11.653 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.162503+0000", 2024-09-21T15:00:11.653 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.617e-06 2024-09-21T15:00:11.653 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.653 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.653 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:11.653 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.174549+0000", 2024-09-21T15:00:11.653 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012046238000000001 2024-09-21T15:00:11.653 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.653 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.653 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:11.653 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.174640+0000", 2024-09-21T15:00:11.654 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0810999999999995e-05 2024-09-21T15:00:11.654 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.654 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.654 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T15:00:11.654 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.175156+0000", 2024-09-21T15:00:11.654 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00051634700000000001 2024-09-21T15:00:11.654 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.654 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.654 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:11.654 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.177347+0000", 2024-09-21T15:00:11.654 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0021907319999999999 2024-09-21T15:00:11.654 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.654 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.654 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:11.655 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.177407+0000", 2024-09-21T15:00:11.655 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9657999999999997e-05 2024-09-21T15:00:11.655 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.655 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.655 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:11.655 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.177498+0000", 2024-09-21T15:00:11.655 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1225999999999997e-05 2024-09-21T15:00:11.655 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.655 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.655 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:11.655 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.177612+0000", 2024-09-21T15:00:11.655 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011362999999999999 2024-09-21T15:00:11.655 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:11.655 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:11.655 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:11.656 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.656 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.656 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:93347 3.9 e119/90 3:92bf9a56:::rbd_object_map.1136b9c49b1c:head v 119'420, mlcod=119'415)", 2024-09-21T15:00:11.656 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:02.217952+0000", 2024-09-21T15:00:11.656 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.3815339049999995, 2024-09-21T15:00:11.656 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.016462039000000001, 2024-09-21T15:00:11.656 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:11.656 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:11.656 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:11.656 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.656 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:11.656 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:02.217952+0000", 2024-09-21T15:00:11.656 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:11.656 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.656 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.656 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:11.656 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:02.217952+0000", 2024-09-21T15:00:11.657 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:11.657 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.657 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.657 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:11.657 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:02.217951+0000", 2024-09-21T15:00:11.657 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:00:11.657 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.657 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.657 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:11.657 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:02.217963+0000", 2024-09-21T15:00:11.657 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2197e-05 2024-09-21T15:00:11.657 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.657 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.657 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:11.657 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:02.217997+0000", 2024-09-21T15:00:11.658 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4468999999999998e-05 2024-09-21T15:00:11.658 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.658 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.658 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:11.658 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:02.218012+0000", 2024-09-21T15:00:11.658 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.452e-05 2024-09-21T15:00:11.658 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.658 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.658 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:11.658 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:02.218095+0000", 2024-09-21T15:00:11.658 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2817000000000003e-05 2024-09-21T15:00:11.658 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.658 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.658 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:11.658 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:02.218126+0000", 2024-09-21T15:00:11.659 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0814e-05 2024-09-21T15:00:11.659 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.659 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.659 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:11.659 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:02.234371+0000", 2024-09-21T15:00:11.659 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.016245384000000002 2024-09-21T15:00:11.659 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.659 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.659 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:11.659 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:02.234414+0000", 2024-09-21T15:00:11.659 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3170999999999997e-05 2024-09-21T15:00:11.659 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:11.659 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:11.659 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:11.660 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.660 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.660 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:93349 3.19 3:9a74bbc0:::rbd_data.1136b9c49b1c.000000000000001c:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-21T15:00:11.660 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:02.218917+0000", 2024-09-21T15:00:11.660 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.3805684930000002, 2024-09-21T15:00:11.660 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015624626000000001, 2024-09-21T15:00:11.660 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:11.660 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:11.660 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:11.660 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:11.660 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:11.660 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 93349 2024-09-21T15:00:11.660 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.660 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:11.660 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.660 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:11.661 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:02.218917+0000", 2024-09-21T15:00:11.661 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:11.661 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.661 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.661 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:11.661 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:02.218917+0000", 2024-09-21T15:00:11.661 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:11.661 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.661 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.661 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:11.661 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:02.218916+0000", 2024-09-21T15:00:11.661 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:00:11.661 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.661 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.661 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:11.661 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:02.218931+0000", 2024-09-21T15:00:11.661 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4712000000000001e-05 2024-09-21T15:00:11.662 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.662 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.662 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:11.662 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:02.218949+0000", 2024-09-21T15:00:11.662 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8094999999999999e-05 2024-09-21T15:00:11.662 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.662 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.662 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:11.662 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:02.218958+0000", 2024-09-21T15:00:11.662 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3670000000000003e-06 2024-09-21T15:00:11.662 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.662 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.662 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:11.662 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:02.219029+0000", 2024-09-21T15:00:11.662 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0859999999999996e-05 2024-09-21T15:00:11.663 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.663 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.663 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:11.663 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:02.219159+0000", 2024-09-21T15:00:11.663 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000130211 2024-09-21T15:00:11.663 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.663 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.663 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 1", 2024-09-21T15:00:11.663 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:02.222970+0000", 2024-09-21T15:00:11.663 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0038108970000000002 2024-09-21T15:00:11.663 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.663 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.663 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:11.663 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:02.234423+0000", 2024-09-21T15:00:11.663 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011452413 2024-09-21T15:00:11.663 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.664 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.664 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:11.664 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:02.234445+0000", 2024-09-21T15:00:11.664 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2367000000000001e-05 2024-09-21T15:00:11.664 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.664 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.664 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:11.664 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:02.234499+0000", 2024-09-21T15:00:11.664 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3634999999999997e-05 2024-09-21T15:00:11.664 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.664 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.664 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:11.664 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:02.234542+0000", 2024-09-21T15:00:11.664 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3356999999999998e-05 2024-09-21T15:00:11.664 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:11.665 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:11.665 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:11.665 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.665 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.665 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:97619 3.1 3:8730dd9c:::rbd_data.1136bfc79a70.0000000000000000:head [write 0~4718592 [fadvise_sequential+fadvise_nocache],write 5242880~16384 [fadvise_sequential+fadvise_nocache],write 5324800~147456 [fadvise_sequential+fadvise_nocache],write 5537792~147456 [fadvise_sequential+fadvise_nocache],write 5750784~16384 [fadvise_sequential+fadvise_nocache],write 5832704~147456 [fadvise_sequential+fadvise_nocache],write 6045696~147456 [fadvise_sequential+fadvise_nocache],write 6258688~32768 [fadvise_sequential+fadvise_nocache],write 6340608~147456 [fadvise_sequential+fadvise_nocache],write 6553600~147456 [fadvise_sequential+fadvise_nocache],write 6766592~49152 [fadvise_sequential+fadvise_nocache],write 6848512~147456 [fadvise_sequential+fadvise_nocache],write 7061504~147456 [fadvise_sequential+fadvise_nocache],write 7274496~65536 [fadvise_sequential+fadvise_nocache],write 7356416~147456 [fadvise_sequential+fadvise_nocache],write 7569408~147456 [fadvise_sequential+fadvise_nocache],write 7782400~229376 [fadvise_sequential+fadvise_nocache],write 8077312~147456 [fadvise_sequential+fadvise_nocache],write 8290304~98304 [fadvise_sequential+fadvise_nocache],write 8404992~114688 [fadvise_sequential+fadvise_nocache],write 8617984~114688 [fadvise_sequential+fadvise_nocache],write 8830976~81920 [fadvise_sequential+fadvise_nocache],write 8994816~32768 [fadvise_sequential+fadvise_nocache],write 9207808~32768 [fadvise_sequential+fadvise_nocache],write 9420800~16384 [fadvise_sequential+fadvise_nocache],write 9502720~32768 [fadvise_sequential+fadvise_nocache],write 9715712~32768 [fadvise_sequential+fadvise_nocache],write 9928704~229376 [fadvise_sequential+fadvise_nocache],write 10174464~196608 [fadvise_sequential+fadvise_nocache],write 10387456~278528 [fadvise_sequential+fadvise_nocache],write 10682368~196608 [fadvise_sequential+fadvise_nocache],write 10895360~278528 [fadvise_sequential+fadvise_nocache],write 11190272~196608 [fadvise_sequential+fadvise_nocache],write 11403264~196608 [fadvise_sequential+fadvise_nocache],write 11616256~65536 [fadvise_sequential+fadvise_nocache],write 11714560~98304 [fadvise_sequential+fadvise_nocache],write 11829248~65536 [fadvise_sequential+fadvise_nocache],write 11927552~98304 [fadvise_sequential+fadvise_nocache],write 12042240~65536 [fadvise_sequential+fadvise_nocache],write 12124160~65536 [fadvise_sequential+fadvise_nocache],write 12222464~98304 [fadvise_sequential+fadvise_nocache],write 12337152~65536 [fadvise_sequential+fadvise_nocache],write 12435456~98304 [fadvise_sequential+fadvise_nocache],write 12550144~65536 [fadvise_sequential+fadvise_nocache],write 12730368~98304 [fadvise_sequential+fadvise_nocache],write 12943360~98304 [fadvise_sequential+fadvise_nocache],write 13107200~16384 [fadvise_sequential+fadvise_nocache],write 13238272~98304 [fadvise_sequential+fadvise_nocache],write 13451264~98304 [fadvise_sequential+fadvise_nocache],write 13746176~98304 [fadvise_sequential+fadvise_nocache],write 13959168~98304 [fadvise_sequential+fadvise_nocache],write 14254080~64264 [fadvise_sequential+fadvise_nocache],write 14319616~19759 [fadvise_sequential+fadvise_nocache],write 15204352~32768 [fadvise_sequential+fadvise_nocache],write 15368192~81920 [fadvise_sequential+fadvise_nocache],write 15581184~81920 [fadvise_sequential+fadvise_nocache],write 15728640~16384 [fadvise_sequential+fadvise_nocache],write 15876096~81920 [fadvise_sequential+fadvise_nocache],write 16089088~81920 [fadvise_sequential+fadvise_nocache],write 16384000~81920 [fadvise_sequential+fadvise_nocache],write 16596992~81920 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:11.665 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.255412+0000", 2024-09-21T15:00:11.665 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 6.3440737089999999, 2024-09-21T15:00:11.665 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.058037909999999998, 2024-09-21T15:00:11.665 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:11.665 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:11.665 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:11.665 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:11.665 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:11.665 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97619 2024-09-21T15:00:11.665 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.666 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:11.666 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.666 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:11.666 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.255412+0000", 2024-09-21T15:00:11.666 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:11.666 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.666 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.666 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:11.666 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.255412+0000", 2024-09-21T15:00:11.666 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:11.666 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.666 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.666 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:11.666 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.255410+0000", 2024-09-21T15:00:11.666 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-21T15:00:11.666 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.667 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.667 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:11.667 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.288087+0000", 2024-09-21T15:00:11.667 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.032677337000000001 2024-09-21T15:00:11.667 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.667 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.667 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:11.667 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.288114+0000", 2024-09-21T15:00:11.667 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6905000000000001e-05 2024-09-21T15:00:11.667 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.667 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.667 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:11.667 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.288128+0000", 2024-09-21T15:00:11.667 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3560999999999999e-05 2024-09-21T15:00:11.667 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.667 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.668 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:11.668 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.288193+0000", 2024-09-21T15:00:11.668 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5747000000000006e-05 2024-09-21T15:00:11.668 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.668 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.668 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:11.668 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.289299+0000", 2024-09-21T15:00:11.668 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0011056390000000001 2024-09-21T15:00:11.668 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.668 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.668 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T15:00:11.668 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.290870+0000", 2024-09-21T15:00:11.668 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001570718 2024-09-21T15:00:11.668 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.668 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.668 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:11.669 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.312525+0000", 2024-09-21T15:00:11.669 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.021655427000000001 2024-09-21T15:00:11.669 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.669 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.669 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:11.669 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.313312+0000", 2024-09-21T15:00:11.669 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00078665500000000004 2024-09-21T15:00:11.669 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.669 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.669 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:11.669 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.313413+0000", 2024-09-21T15:00:11.669 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010131700000000001 2024-09-21T15:00:11.669 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.669 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.669 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:11.669 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.313450+0000", 2024-09-21T15:00:11.670 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7032000000000001e-05 2024-09-21T15:00:11.670 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:11.670 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:11.670 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:11.670 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.670 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.670 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:97734 3.19 3:9fcf754f:::rbd_data.1136bfc79a70.0000000000000009:head [write 0~4194304 [fadvise_sequential+fadvise_nocache],write 4784128~147456 [fadvise_sequential+fadvise_nocache],write 4997120~147456 [fadvise_sequential+fadvise_nocache],write 5210112~32768 [fadvise_sequential+fadvise_nocache],write 5292032~147456 [fadvise_sequential+fadvise_nocache],write 5505024~147456 [fadvise_sequential+fadvise_nocache],write 5718016~49152 [fadvise_sequential+fadvise_nocache],write 5799936~147456 [fadvise_sequential+fadvise_nocache],write 6012928~147456 [fadvise_sequential+fadvise_nocache],write 6225920~65536 [fadvise_sequential+fadvise_nocache],write 6307840~147456 [fadvise_sequential+fadvise_nocache],write 6520832~147456 [fadvise_sequential+fadvise_nocache],write 6733824~229376 [fadvise_sequential+fadvise_nocache],write 7028736~147456 [fadvise_sequential+fadvise_nocache],write 7241728~229376 [fadvise_sequential+fadvise_nocache],write 7536640~147456 [fadvise_sequential+fadvise_nocache],write 7749632~229376 [fadvise_sequential+fadvise_nocache],write 8077312~114688 [fadvise_sequential+fadvise_nocache],write 8290304~98304 [fadvise_sequential+fadvise_nocache],write 8404992~81920 [fadvise_sequential+fadvise_nocache],write 8617984~81920 [fadvise_sequential+fadvise_nocache],write 8830976~81920 [fadvise_sequential+fadvise_nocache],write 8962048~32768 [fadvise_sequential+fadvise_nocache],write 9175040~32768 [fadvise_sequential+fadvise_nocache],write 9388032~32768 [fadvise_sequential+fadvise_nocache],write 9469952~32768 [fadvise_sequential+fadvise_nocache],write 9682944~20092 [fadvise_sequential+fadvise_nocache],write 9961472~163840 [fadvise_sequential+fadvise_nocache],write 10141696~196608 [fadvise_sequential+fadvise_nocache],write 10354688~278528 [fadvise_sequential+fadvise_nocache],write 10649600~196608 [fadvise_sequential+fadvise_nocache],write 10862592~196608 [fadvise_sequential+fadvise_nocache],write 11075584~65536 [fadvise_sequential+fadvise_nocache],write 11173888~98304 [fadvise_sequential+fadvise_nocache],write 11288576~65536 [fadvise_sequential+fadvise_nocache],write 11386880~98304 [fadvise_sequential+fadvise_nocache],write 11501568~65536 [fadvise_sequential+fadvise_nocache],write 11583488~65536 [fadvise_sequential+fadvise_nocache],write 11681792~98304 [fadvise_sequential+fadvise_nocache],write 11796480~65536 [fadvise_sequential+fadvise_nocache],write 11894784~98304 [fadvise_sequential+fadvise_nocache],write 12009472~65536 [fadvise_sequential+fadvise_nocache],write 12189696~98304 [fadvise_sequential+fadvise_nocache],write 12402688~98304 [fadvise_sequential+fadvise_nocache],write 12697600~98304 [fadvise_sequential+fadvise_nocache],write 12910592~98304 [fadvise_sequential+fadvise_nocache],write 13205504~98304 [fadvise_sequential+fadvise_nocache],write 13418496~98304 [fadvise_sequential+fadvise_nocache],write 13713408~98304 [fadvise_sequential+fadvise_nocache],write 13926400~98304 [fadvise_sequential+fadvise_nocache],write 14139392~16384 [fadvise_sequential+fadvise_nocache],write 14680064~16384 [fadvise_sequential+fadvise_nocache],write 14827520~81920 [fadvise_sequential+fadvise_nocache],write 15040512~81920 [fadvise_sequential+fadvise_nocache],write 15335424~81920 [fadvise_sequential+fadvise_nocache],write 15548416~81920 [fadvise_sequential+fadvise_nocache],write 15843328~81920 [fadvise_sequential+fadvise_nocache],write 16056320~81920 [fadvise_sequential+fadvise_nocache],write 16351232~81920 [fadvise_sequential+fadvise_nocache],write 16564224~81920 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:11.670 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.354331+0000", 2024-09-21T15:00:11.670 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 6.2451553190000002, 2024-09-21T15:00:11.670 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.086284870999999999, 2024-09-21T15:00:11.670 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:11.670 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:11.670 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:11.670 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:11.671 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:11.671 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97734 2024-09-21T15:00:11.671 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.671 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:11.671 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.671 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:11.671 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.354331+0000", 2024-09-21T15:00:11.671 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:11.671 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.671 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.671 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:11.671 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.354331+0000", 2024-09-21T15:00:11.671 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:11.671 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.671 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.671 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:11.672 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.354329+0000", 2024-09-21T15:00:11.672 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:00:11.672 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.672 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.672 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:11.672 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.370234+0000", 2024-09-21T15:00:11.672 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015905348999999999 2024-09-21T15:00:11.672 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.672 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.672 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:11.672 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.370286+0000", 2024-09-21T15:00:11.672 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1656999999999997e-05 2024-09-21T15:00:11.672 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.672 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.672 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:11.673 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.370315+0000", 2024-09-21T15:00:11.673 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9204e-05 2024-09-21T15:00:11.673 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.673 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.673 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:11.673 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.370395+0000", 2024-09-21T15:00:11.673 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0395999999999994e-05 2024-09-21T15:00:11.673 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.673 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.673 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:11.673 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.370604+0000", 2024-09-21T15:00:11.673 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000208684 2024-09-21T15:00:11.673 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.673 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.673 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 1", 2024-09-21T15:00:11.673 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.371701+0000", 2024-09-21T15:00:11.674 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0010973140000000001 2024-09-21T15:00:11.674 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.674 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.674 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:11.674 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.405485+0000", 2024-09-21T15:00:11.674 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.033783615000000003 2024-09-21T15:00:11.674 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.674 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.674 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:11.674 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.440211+0000", 2024-09-21T15:00:11.674 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034725757000000003 2024-09-21T15:00:11.674 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.674 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.674 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:11.674 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.440497+0000", 2024-09-21T15:00:11.675 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00028673099999999998 2024-09-21T15:00:11.675 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.675 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.675 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:11.675 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.440616+0000", 2024-09-21T15:00:11.675 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000118256 2024-09-21T15:00:11.675 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:11.675 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:11.675 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:11.675 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.675 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.675 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:98201 3.19 3:9efc5cfe:::rbd_data.1136b9c49b1c.0000000000000021:49 [read 3555328~32768 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:11.675 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.390232+0000", 2024-09-21T15:00:11.675 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 6.2092538859999999, 2024-09-21T15:00:11.675 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015244064999999999, 2024-09-21T15:00:11.675 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:11.676 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:11.676 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:11.676 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:11.676 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:11.676 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 98201 2024-09-21T15:00:11.676 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.676 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:11.676 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.676 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:11.676 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.390232+0000", 2024-09-21T15:00:11.676 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:11.676 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.676 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.676 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:11.676 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.390232+0000", 2024-09-21T15:00:11.676 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:11.677 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.677 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.677 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:11.677 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.390232+0000", 2024-09-21T15:00:11.677 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:11.677 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.677 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.677 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:11.677 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.390234+0000", 2024-09-21T15:00:11.677 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3159999999999999e-06 2024-09-21T15:00:11.677 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.677 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.677 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:11.677 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.390240+0000", 2024-09-21T15:00:11.677 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7189999999999998e-06 2024-09-21T15:00:11.677 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.678 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.678 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:11.678 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.390244+0000", 2024-09-21T15:00:11.678 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5530000000000003e-06 2024-09-21T15:00:11.678 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.678 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.678 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:11.678 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.405303+0000", 2024-09-21T15:00:11.678 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015058861999999999 2024-09-21T15:00:11.678 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.678 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.678 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:11.678 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.405381+0000", 2024-09-21T15:00:11.678 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7880999999999995e-05 2024-09-21T15:00:11.678 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.678 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.679 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:11.679 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.405476+0000", 2024-09-21T15:00:11.679 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5014999999999997e-05 2024-09-21T15:00:11.679 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:11.679 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:11.679 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:11.679 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.679 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.679 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:97734 3.19 3:9fcf754f:::rbd_data.1136bfc79a70.0000000000000009:head [write 0~4194304 [fadvise_sequential+fadvise_nocache],write 4784128~147456 [fadvise_sequential+fadvise_nocache],write 4997120~147456 [fadvise_sequential+fadvise_nocache],write 5210112~32768 [fadvise_sequential+fadvise_nocache],write 5292032~147456 [fadvise_sequential+fadvise_nocache],write 5505024~147456 [fadvise_sequential+fadvise_nocache],write 5718016~49152 [fadvise_sequential+fadvise_nocache],write 5799936~147456 [fadvise_sequential+fadvise_nocache],write 6012928~147456 [fadvise_sequential+fadvise_nocache],write 6225920~65536 [fadvise_sequential+fadvise_nocache],write 6307840~147456 [fadvise_sequential+fadvise_nocache],write 6520832~147456 [fadvise_sequential+fadvise_nocache],write 6733824~229376 [fadvise_sequential+fadvise_nocache],write 7028736~147456 [fadvise_sequential+fadvise_nocache],write 7241728~229376 [fadvise_sequential+fadvise_nocache],write 7536640~147456 [fadvise_sequential+fadvise_nocache],write 7749632~229376 [fadvise_sequential+fadvise_nocache],write 8077312~114688 [fadvise_sequential+fadvise_nocache],write 8290304~98304 [fadvise_sequential+fadvise_nocache],write 8404992~81920 [fadvise_sequential+fadvise_nocache],write 8617984~81920 [fadvise_sequential+fadvise_nocache],write 8830976~81920 [fadvise_sequential+fadvise_nocache],write 8962048~32768 [fadvise_sequential+fadvise_nocache],write 9175040~32768 [fadvise_sequential+fadvise_nocache],write 9388032~32768 [fadvise_sequential+fadvise_nocache],write 9469952~32768 [fadvise_sequential+fadvise_nocache],write 9682944~20092 [fadvise_sequential+fadvise_nocache],write 9961472~163840 [fadvise_sequential+fadvise_nocache],write 10141696~196608 [fadvise_sequential+fadvise_nocache],write 10354688~278528 [fadvise_sequential+fadvise_nocache],write 10649600~196608 [fadvise_sequential+fadvise_nocache],write 10862592~196608 [fadvise_sequential+fadvise_nocache],write 11075584~65536 [fadvise_sequential+fadvise_nocache],write 11173888~98304 [fadvise_sequential+fadvise_nocache],write 11288576~65536 [fadvise_sequential+fadvise_nocache],write 11386880~98304 [fadvise_sequential+fadvise_nocache],write 11501568~65536 [fadvise_sequential+fadvise_nocache],write 11583488~65536 [fadvise_sequential+fadvise_nocache],write 11681792~98304 [fadvise_sequential+fadvise_nocache],write 11796480~65536 [fadvise_sequential+fadvise_nocache],write 11894784~98304 [fadvise_sequential+fadvise_nocache],write 12009472~65536 [fadvise_sequential+fadvise_nocache],write 12189696~98304 [fadvise_sequential+fadvise_nocache],write 12402688~98304 [fadvise_sequential+fadvise_nocache],write 12697600~98304 [fadvise_sequential+fadvise_nocache],write 12910592~98304 [fadvise_sequential+fadvise_nocache],write 13205504~98304 [fadvise_sequential+fadvise_nocache],write 13418496~98304 [fadvise_sequential+fadvise_nocache],write 13713408~98304 [fadvise_sequential+fadvise_nocache],write 13926400~98304 [fadvise_sequential+fadvise_nocache],write 14139392~16384 [fadvise_sequential+fadvise_nocache],write 14680064~16384 [fadvise_sequential+fadvise_nocache],write 14827520~81920 [fadvise_sequential+fadvise_nocache],write 15040512~81920 [fadvise_sequential+fadvise_nocache],write 15335424~81920 [fadvise_sequential+fadvise_nocache],write 15548416~81920 [fadvise_sequential+fadvise_nocache],write 15843328~81920 [fadvise_sequential+fadvise_nocache],write 16056320~81920 [fadvise_sequential+fadvise_nocache],write 16351232~81920 [fadvise_sequential+fadvise_nocache],write 16564224~81920 [fadvise_sequential+fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:11.679 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.412887+0000", 2024-09-21T15:00:11.679 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 6.1865985950000004, 2024-09-21T15:00:11.679 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.027651709999999999, 2024-09-21T15:00:11.679 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:11.679 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "waiting for ondisk", 2024-09-21T15:00:11.679 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:11.680 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:11.680 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:11.680 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97734 2024-09-21T15:00:11.680 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.680 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:11.680 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.680 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:11.680 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.412887+0000", 2024-09-21T15:00:11.680 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:11.680 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.680 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.680 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:11.680 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.412887+0000", 2024-09-21T15:00:11.680 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:11.680 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.680 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.681 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:11.681 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.412886+0000", 2024-09-21T15:00:11.681 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:00:11.681 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.681 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.681 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:11.681 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.428395+0000", 2024-09-21T15:00:11.681 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015509448 2024-09-21T15:00:11.681 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.681 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.681 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:11.681 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.428417+0000", 2024-09-21T15:00:11.681 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1263999999999998e-05 2024-09-21T15:00:11.681 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.681 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.681 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:11.682 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.428425+0000", 2024-09-21T15:00:11.682 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8589999999999994e-06 2024-09-21T15:00:11.682 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.682 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.682 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:11.682 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.428481+0000", 2024-09-21T15:00:11.682 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5087000000000003e-05 2024-09-21T15:00:11.682 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.682 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.682 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for ondisk", 2024-09-21T15:00:11.682 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.428628+0000", 2024-09-21T15:00:11.682 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014714599999999999 2024-09-21T15:00:11.682 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.682 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.682 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:11.683 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.440539+0000", 2024-09-21T15:00:11.683 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011911424 2024-09-21T15:00:11.683 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:11.683 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:11.683 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:11.683 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.683 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.683 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:98562 3.9 e122/90 3:911e52bc:::rbd_data.1136bfc79a70.000000000000000a:head v 122'620, mlcod=121'618)", 2024-09-21T15:00:11.683 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.430966+0000", 2024-09-21T15:00:11.683 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 6.168520129, 2024-09-21T15:00:11.683 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.031245016, 2024-09-21T15:00:11.683 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:11.683 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:11.683 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:11.683 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.683 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:11.684 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.430966+0000", 2024-09-21T15:00:11.684 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:11.684 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.684 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.684 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:11.684 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.430966+0000", 2024-09-21T15:00:11.684 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:11.684 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.684 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.684 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:11.684 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.430965+0000", 2024-09-21T15:00:11.684 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:00:11.684 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.684 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.685 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:11.685 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.447521+0000", 2024-09-21T15:00:11.685 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.016556006000000002 2024-09-21T15:00:11.685 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.685 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.685 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:11.685 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.447543+0000", 2024-09-21T15:00:11.685 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2121e-05 2024-09-21T15:00:11.685 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.685 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.685 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:11.685 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.447551+0000", 2024-09-21T15:00:11.685 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6310000000000003e-06 2024-09-21T15:00:11.685 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.685 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.686 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:11.686 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.447632+0000", 2024-09-21T15:00:11.686 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1388000000000001e-05 2024-09-21T15:00:11.686 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.686 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.686 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:11.686 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.447662+0000", 2024-09-21T15:00:11.686 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0111999999999999e-05 2024-09-21T15:00:11.686 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.686 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.686 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:11.686 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.462185+0000", 2024-09-21T15:00:11.686 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014522863 2024-09-21T15:00:11.686 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.687 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.687 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:11.687 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.462211+0000", 2024-09-21T15:00:11.687 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5945e-05 2024-09-21T15:00:11.687 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:11.687 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:11.687 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:11.687 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.687 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.687 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:100151 3.8 3:17ab0d23:::rbd_data.1136ab4d6ff1.000000000000000a:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-21T15:00:11.687 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:10.762267+0000", 2024-09-21T15:00:11.687 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.83721862499999999, 2024-09-21T15:00:11.687 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.029791319, 2024-09-21T15:00:11.687 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:11.687 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:11.688 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:11.688 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:11.688 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:11.688 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 100151 2024-09-21T15:00:11.688 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.688 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:11.688 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.688 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:11.688 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:10.762267+0000", 2024-09-21T15:00:11.688 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:11.688 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.688 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.688 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:11.688 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:10.762267+0000", 2024-09-21T15:00:11.689 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:11.689 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.689 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.689 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:11.689 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:10.762264+0000", 2024-09-21T15:00:11.689 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999967 2024-09-21T15:00:11.689 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.689 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.689 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:11.689 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:10.762283+0000", 2024-09-21T15:00:11.689 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9072999999999998e-05 2024-09-21T15:00:11.689 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.689 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.689 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:11.690 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:10.762325+0000", 2024-09-21T15:00:11.690 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2222000000000003e-05 2024-09-21T15:00:11.690 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.690 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.690 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:11.690 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:10.762348+0000", 2024-09-21T15:00:11.690 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2857000000000002e-05 2024-09-21T15:00:11.690 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.690 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.690 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:11.690 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:10.762490+0000", 2024-09-21T15:00:11.690 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000141667 2024-09-21T15:00:11.690 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.691 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.691 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:11.691 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:10.762751+0000", 2024-09-21T15:00:11.691 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00026079 2024-09-21T15:00:11.691 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.691 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.691 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 7", 2024-09-21T15:00:11.691 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:10.772217+0000", 2024-09-21T15:00:11.691 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0094660960000000002 2024-09-21T15:00:11.691 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.691 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.691 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:11.691 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:10.791315+0000", 2024-09-21T15:00:11.691 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.019098674999999999 2024-09-21T15:00:11.691 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.692 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.692 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:11.692 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:10.791938+0000", 2024-09-21T15:00:11.692 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00062279200000000005 2024-09-21T15:00:11.692 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.692 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.692 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:11.692 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:10.792005+0000", 2024-09-21T15:00:11.692 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6884000000000005e-05 2024-09-21T15:00:11.692 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.692 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.692 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:11.692 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:10.792059+0000", 2024-09-21T15:00:11.692 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3497e-05 2024-09-21T15:00:11.692 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:11.693 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:11.693 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:11.693 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.693 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.693 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:100152 3.7 3:e02630f6:::rbd_data.1136ab4d6ff1.000000000000000b:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-21T15:00:11.693 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:10.763146+0000", 2024-09-21T15:00:11.693 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.83633950199999996, 2024-09-21T15:00:11.693 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.029305525999999998, 2024-09-21T15:00:11.693 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:11.693 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:11.693 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:11.693 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:11.693 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:11.693 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 100152 2024-09-21T15:00:11.693 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.694 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:11.694 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.694 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:11.694 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:10.763146+0000", 2024-09-21T15:00:11.694 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:11.694 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.694 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.694 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:11.694 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:10.763146+0000", 2024-09-21T15:00:11.694 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:11.694 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.694 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.694 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:11.694 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:10.763139+0000", 2024-09-21T15:00:11.695 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999924 2024-09-21T15:00:11.695 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.695 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.695 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:11.695 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:10.763158+0000", 2024-09-21T15:00:11.695 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9040000000000001e-05 2024-09-21T15:00:11.695 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.695 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.695 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:11.695 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:10.763185+0000", 2024-09-21T15:00:11.695 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6988999999999999e-05 2024-09-21T15:00:11.695 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.695 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.695 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:11.696 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:10.763198+0000", 2024-09-21T15:00:11.696 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.27e-05 2024-09-21T15:00:11.696 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.696 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.696 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:11.696 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:10.763303+0000", 2024-09-21T15:00:11.696 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000105451 2024-09-21T15:00:11.696 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.696 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.696 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:11.696 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:10.763557+0000", 2024-09-21T15:00:11.696 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00025405799999999999 2024-09-21T15:00:11.696 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.696 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.696 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T15:00:11.697 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:10.772841+0000", 2024-09-21T15:00:11.697 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0092834259999999991 2024-09-21T15:00:11.697 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.697 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.697 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:11.697 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:10.785078+0000", 2024-09-21T15:00:11.697 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012237572 2024-09-21T15:00:11.697 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.697 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.697 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:11.697 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:10.792383+0000", 2024-09-21T15:00:11.697 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0073048729999999999 2024-09-21T15:00:11.697 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.697 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.697 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:11.698 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:10.792423+0000", 2024-09-21T15:00:11.698 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0462e-05 2024-09-21T15:00:11.698 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:11.698 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:11.698 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:11.698 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:10.792452+0000", 2024-09-21T15:00:11.698 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8581999999999998e-05 2024-09-21T15:00:11.698 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:11.698 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:11.698 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:11.698 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:11.698 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:11.698 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:11.700 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T15:00:11.700 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:00:11.694+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:00:11.701 INFO:teuthology.orchestra.run.smithi032.stdout:3296 punch from 0x9d504 to 0xa3f76, (0x6a72 bytes) 2024-09-21T15:00:11.701 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:11.701 INFO:teuthology.orchestra.run.smithi032.stdout:3303 trunc from 0x73938e to 0x24ce78c 2024-09-21T15:00:11.701 INFO:teuthology.orchestra.run.smithi032.stdout:3304 read 0x1938f12 thru 0x1944669 (0xb758 bytes) 2024-09-21T15:00:11.701 INFO:teuthology.orchestra.run.smithi032.stdout:3305 write 0x150535 thru 0x1532b0 (0x2d7c bytes) 2024-09-21T15:00:11.701 INFO:teuthology.orchestra.run.smithi032.stdout:3307 write 0x3c59ead thru 0x3c65f55 (0xc0a9 bytes) 2024-09-21T15:00:11.701 INFO:teuthology.orchestra.run.smithi032.stdout:3308 read 0xeb5f08 thru 0xec121c (0xb315 bytes) 2024-09-21T15:00:11.701 INFO:teuthology.orchestra.run.smithi032.stdout:3310 punch from 0xcbd563 to 0xcc3abb, (0x6558 bytes) 2024-09-21T15:00:11.701 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:11.702 INFO:teuthology.orchestra.run.smithi032.stdout:3313 write 0xa4aa3a0 thru 0xa4ac4d4 (0x2135 bytes) 2024-09-21T15:00:11.702 INFO:teuthology.orchestra.run.smithi032.stdout:3315 trunc from 0xa4ac4d5 to 0x55f460f 2024-09-21T15:00:11.703 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T15:00:11.708 INFO:teuthology.orchestra.run.smithi032.stdout:3316 read 0x42bb3f6 thru 0x42ca288 (0xee93 bytes) 2024-09-21T15:00:11.710 INFO:teuthology.orchestra.run.smithi032.stdout:3319 trunc from 0x55f460f to 0x5e080b4 2024-09-21T15:00:11.729 INFO:teuthology.orchestra.run.smithi032.stdout:3320 trunc from 0x5e080b4 to 0x1237fda 2024-09-21T15:00:11.732 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T15:00:11.757 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T15:00:11.764 INFO:teuthology.orchestra.run.smithi032.stdout:3322 write 0x6486b1a thru 0x648b8b9 (0x4da0 bytes) 2024-09-21T15:00:11.770 INFO:teuthology.orchestra.run.smithi032.stdout:3323 write 0x2c9a92 thru 0x2d7cd0 (0xe23f bytes) 2024-09-21T15:00:11.774 INFO:teuthology.orchestra.run.smithi032.stdout:3324 write 0x1a5b5bb thru 0x1a60527 (0x4f6d bytes) 2024-09-21T15:00:11.776 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T15:00:11.777 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:11.777 INFO:teuthology.orchestra.run.smithi032.stdout:3326 read 0x24fd731 thru 0x250c0ff (0xe9cf bytes) 2024-09-21T15:00:11.796 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-21T15:00:11.797 INFO:teuthology.orchestra.run.smithi032.stdout:3327 writesame 0xb75a3ed thru 0xb75b058 data_size 0x1e(0xc6c bytes) 2024-09-21T15:00:11.797 INFO:teuthology.orchestra.run.smithi032.stdout:3328 writesame 0x315239e thru 0x31523f7 data_size 0x6(0x5a bytes) 2024-09-21T15:00:11.800 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:00:11.800+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:00:11.801 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:11.804 INFO:teuthology.orchestra.run.smithi032.stdout:3330 trunc from 0xb75b059 to 0xc9d7783 2024-09-21T15:00:11.824 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:11.824 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:00:11.824 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T15:00:11.824 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:11.834 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T15:00:11.837 INFO:teuthology.orchestra.run.smithi032.stdout:3331 trunc from 0xc9d7783 to 0xe7a32b 2024-09-21T15:00:11.877 INFO:teuthology.orchestra.run.smithi032.stdout:3332 writesame 0x4ce5eaf thru 0x4ce8d99 data_size 0x1(0x2eeb bytes) 2024-09-21T15:00:11.879 INFO:teuthology.orchestra.run.smithi032.stdout:3333 read 0x21adfb3 thru 0x21b2bec (0x4c3a bytes) 2024-09-21T15:00:11.880 INFO:teuthology.orchestra.run.smithi032.stdout:3334 read 0x12accd2 thru 0x12b311d (0x644c bytes) 2024-09-21T15:00:11.880 INFO:teuthology.orchestra.run.smithi032.stdout:3335 read 0x2b4f653 thru 0x2b581b7 (0x8b65 bytes) 2024-09-21T15:00:11.881 INFO:teuthology.orchestra.run.smithi032.stdout:3336 read 0x374db8e thru 0x3758850 (0xacc3 bytes) 2024-09-21T15:00:11.902 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:00:11.898+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:00:11.911 INFO:teuthology.orchestra.run.smithi032.stdout:3339 write 0xd8cd788 thru 0xd8cdc4e (0x4c7 bytes) 2024-09-21T15:00:11.918 INFO:teuthology.orchestra.run.smithi032.stdout:3340 read 0x7cc669f thru 0x7cd0bec (0xa54e bytes) 2024-09-21T15:00:11.918 INFO:teuthology.orchestra.run.smithi032.stdout:3342 write 0x42e1b8d thru 0x42f1989 (0xfdfd bytes) 2024-09-21T15:00:11.937 INFO:teuthology.orchestra.run.smithi032.stdout:3343 writesame 0xe000b41 thru 0xe00f4cf data_size 0x1(0xe98f bytes) 2024-09-21T15:00:11.937 INFO:teuthology.orchestra.run.smithi032.stdout:3344 writesame 0x32e2b61 thru 0x32e499d data_size 0x1(0x1e3d bytes) 2024-09-21T15:00:11.949 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:11.949 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:00:11.950 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T15:00:11.950 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T15:00:11.950 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:11.956 INFO:teuthology.orchestra.run.smithi032.stdout:3345 trunc from 0xe00f4d0 to 0x9e2bd08 2024-09-21T15:00:11.959 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T15:00:11.978 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:11.984 INFO:teuthology.orchestra.run.smithi032.stdout:3349 trunc from 0x9e2bd08 to 0x79c9c64 2024-09-21T15:00:11.998 INFO:teuthology.orchestra.run.smithi032.stdout:3351 punch from 0x5ffc84d to 0x60032cc, (0x6a7f bytes) 2024-09-21T15:00:11.998 INFO:teuthology.orchestra.run.smithi032.stdout:3352 write 0x25ddcbf thru 0x25e1158 (0x349a bytes) 2024-09-21T15:00:12.002 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:00:11.998+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:00:12.003 INFO:teuthology.orchestra.run.smithi032.stdout:3353 read 0x73ecb9e thru 0x73fc79f (0xfc02 bytes) 2024-09-21T15:00:12.003 INFO:teuthology.orchestra.run.smithi032.stdout:3354 punch from 0x5c59372 to 0x5c683cb, (0xf059 bytes) 2024-09-21T15:00:12.019 INFO:teuthology.orchestra.run.smithi032.stdout:3357 writesame 0x924fcd8 thru 0x9251fd5 data_size 0x3(0x22fe bytes) 2024-09-21T15:00:12.031 INFO:teuthology.orchestra.run.smithi032.stdout:3359 write 0xa719b21 thru 0xa71fec8 (0x63a8 bytes) 2024-09-21T15:00:12.031 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T15:00:12.040 INFO:teuthology.orchestra.run.smithi032.stdout:3360 read 0xc8c6d5 thru 0xc98e93 (0xc7bf bytes) 2024-09-21T15:00:12.040 INFO:teuthology.orchestra.run.smithi032.stdout:3361 write 0x4178294 thru 0x417b8e6 (0x3653 bytes) 2024-09-21T15:00:12.044 INFO:teuthology.orchestra.run.smithi032.stdout:3362 read 0x9080abb thru 0x908fb7a (0xf0c0 bytes) 2024-09-21T15:00:12.044 INFO:teuthology.orchestra.run.smithi032.stdout:3363 read 0x6924aef thru 0x692a03a (0x554c bytes) 2024-09-21T15:00:12.052 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T15:00:12.056 INFO:teuthology.orchestra.run.smithi032.stdout:3364 trunc from 0xa71fec9 to 0x4cbceff 2024-09-21T15:00:12.068 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:12.068 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:12.077 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T15:00:12.079 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T15:00:12.079 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:00:12.079 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:00:12.080 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:00:12.080 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:00:12.080 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:00:12.080 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:00:12.081 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:00:12.081 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:00:12.082 INFO:teuthology.orchestra.run.smithi032.stdout:3365 read 0x1668356 thru 0x16749d8 (0xc683 bytes) 2024-09-21T15:00:12.082 INFO:teuthology.orchestra.run.smithi032.stdout:3368 read 0x47fe441 thru 0x4802641 (0x4201 bytes) 2024-09-21T15:00:12.082 INFO:teuthology.orchestra.run.smithi032.stdout:3370 punch from 0x2fcffc7 to 0x2fde5d9, (0xe612 bytes) 2024-09-21T15:00:12.083 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:12.083 INFO:teuthology.orchestra.run.smithi032.stdout:3372 write 0x1d99d83 thru 0x1da722c (0xd4aa bytes) 2024-09-21T15:00:12.094 INFO:teuthology.orchestra.run.smithi032.stdout:3374 trunc from 0x4cbceff to 0x9e67a14 2024-09-21T15:00:12.103 INFO:teuthology.orchestra.run.smithi032.stdout:3375 write 0x6d5160f thru 0x6d5ec16 (0xd608 bytes) 2024-09-21T15:00:12.110 INFO:teuthology.orchestra.run.smithi032.stdout:3378 read 0x9e4b959 thru 0x9e5a5cd (0xec75 bytes) 2024-09-21T15:00:12.126 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:00:12.122+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:00:12.127 INFO:teuthology.orchestra.run.smithi032.stdout:3380 write 0xd64494b thru 0xd650fa9 (0xc65f bytes) 2024-09-21T15:00:12.136 INFO:teuthology.orchestra.run.smithi032.stdout:3381 writesame 0x5be345e thru 0x5beb3bd data_size 0x10(0x7f60 bytes) 2024-09-21T15:00:12.141 INFO:teuthology.orchestra.run.smithi032.stdout:3383 read 0x1b86e5d thru 0x1b8da4f (0x6bf3 bytes) 2024-09-21T15:00:12.142 INFO:teuthology.orchestra.run.smithi032.stdout:3385 write 0xb5e8c42 thru 0xb5eaf8e (0x234d bytes) 2024-09-21T15:00:12.145 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:12.145 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:12.145 INFO:teuthology.orchestra.run.smithi032.stdout:3389 writesame 0x3f226bf thru 0x3f253ee data_size 0x18(0x2d30 bytes) 2024-09-21T15:00:12.149 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:12.150 INFO:teuthology.orchestra.run.smithi032.stdout:3391 trunc from 0xd650faa to 0xdd7bb2e 2024-09-21T15:00:12.162 INFO:teuthology.orchestra.run.smithi032.stdout:3392 writesame 0xb65bb72 thru 0xb6642f2 data_size 0x1f(0x8781 bytes) 2024-09-21T15:00:12.170 INFO:teuthology.orchestra.run.smithi032.stdout:3394 punch from 0x2214eba to 0x221687b, (0x19c1 bytes) 2024-09-21T15:00:12.170 INFO:teuthology.orchestra.run.smithi032.stdout:3395 read 0x30bc11d thru 0x30c6352 (0xa236 bytes) 2024-09-21T15:00:12.171 INFO:teuthology.orchestra.run.smithi032.stdout:3396 read 0x5528dfe thru 0x552cc5e (0x3e61 bytes) 2024-09-21T15:00:12.172 INFO:teuthology.orchestra.run.smithi032.stdout:3397 write 0x8f0f010 thru 0x8f0ff53 (0xf44 bytes) 2024-09-21T15:00:12.185 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:12.185 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:12.190 INFO:teuthology.orchestra.run.smithi032.stdout:3398 trunc from 0xdd7bb2e to 0x82760d4 2024-09-21T15:00:12.195 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T15:00:12.203 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:00:12.200+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:00:12.215 INFO:teuthology.orchestra.run.smithi032.stdout:3399 punch from 0x4ac5b8e to 0x4acc1de, (0x6650 bytes) 2024-09-21T15:00:12.215 INFO:teuthology.orchestra.run.smithi032.stdout:3400 punch from 0x7a05d31 to 0x7a11846, (0xbb15 bytes) 2024-09-21T15:00:12.215 INFO:teuthology.orchestra.run.smithi032.stdout:3402 punch from 0x3d6245a to 0x3d69d3b, (0x78e1 bytes) 2024-09-21T15:00:12.215 INFO:teuthology.orchestra.run.smithi032.stdout:3404 read 0xd7c8ce thru 0xd8527a (0x89ad bytes) 2024-09-21T15:00:12.215 INFO:teuthology.orchestra.run.smithi032.stdout:3405 write 0xd21ee5 thru 0xd2f586 (0xd6a2 bytes) 2024-09-21T15:00:12.243 INFO:teuthology.orchestra.run.smithi032.stdout:3406 writesame 0xdd053d5 thru 0xdd12030 data_size 0x16(0xcc5c bytes) 2024-09-21T15:00:12.243 INFO:teuthology.orchestra.run.smithi032.stdout:3407 writesame 0x91254 thru 0x9826e data_size 0x1(0x701b bytes) 2024-09-21T15:00:12.264 INFO:teuthology.orchestra.run.smithi032.stdout:3408 write 0x6622d32 thru 0x662d424 (0xa6f3 bytes) 2024-09-21T15:00:12.271 INFO:teuthology.orchestra.run.smithi032.stdout:3412 writesame 0x10a1525 thru 0x10a2ada data_size 0x2(0x15b6 bytes) 2024-09-21T15:00:12.275 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:12.275 INFO:teuthology.orchestra.run.smithi032.stdout:3418 punch from 0xc0a4c36 to 0xc0b1856, (0xcc20 bytes) 2024-09-21T15:00:12.275 INFO:teuthology.orchestra.run.smithi032.stdout:3419 write 0x34b49f3 thru 0x34b85f6 (0x3c04 bytes) 2024-09-21T15:00:12.277 INFO:teuthology.orchestra.run.smithi032.stdout:3420 write 0x1627963 thru 0x16291ef (0x188d bytes) 2024-09-21T15:00:12.280 INFO:teuthology.orchestra.run.smithi032.stdout:3421 writesame 0xb20fe8b thru 0xb21784e data_size 0x2(0x79c4 bytes) 2024-09-21T15:00:12.294 INFO:teuthology.orchestra.run.smithi032.stdout:3422 read 0xd2bf5c5 thru 0xd2cf0d0 (0xfb0c bytes) 2024-09-21T15:00:12.295 INFO:teuthology.orchestra.run.smithi032.stdout:3423 writesame 0x6ea4825 thru 0x6eaa10b data_size 0x1(0x58e7 bytes) 2024-09-21T15:00:12.301 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:12.302 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:12.305 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:00:12.304+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:00:12.310 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:12.311 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T15:00:12.324 INFO:teuthology.orchestra.run.smithi032.stdout:3425 trunc from 0xdd12031 to 0x54b1d13 2024-09-21T15:00:12.363 INFO:teuthology.orchestra.run.smithi032.stdout:3426 trunc from 0x54b1d13 to 0xbfee2c6 2024-09-21T15:00:12.390 INFO:teuthology.orchestra.run.smithi032.stdout:3428 write 0xc9fe278 thru 0xca09d14 (0xba9d bytes) 2024-09-21T15:00:12.400 INFO:teuthology.orchestra.run.smithi032.stdout:3429 read 0x9063015 thru 0x9067c12 (0x4bfe bytes) 2024-09-21T15:00:12.401 INFO:teuthology.orchestra.run.smithi032.stdout:3430 write 0x52884fc thru 0x528efda (0x6adf bytes) 2024-09-21T15:00:12.406 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:00:12.398+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:00:12.409 INFO:teuthology.orchestra.run.smithi032.stdout:3431 read 0x20b137f thru 0x20be5b7 (0xd239 bytes) 2024-09-21T15:00:12.410 INFO:teuthology.orchestra.run.smithi032.stdout:3432 read 0x8cc92b4 thru 0x8cc98a3 (0x5f0 bytes) 2024-09-21T15:00:12.410 INFO:teuthology.orchestra.run.smithi032.stdout:3433 write 0x727c517 thru 0x727f886 (0x3370 bytes) 2024-09-21T15:00:12.414 INFO:teuthology.orchestra.run.smithi032.stdout:3434 write 0xb09622a thru 0xb09c0be (0x5e95 bytes) 2024-09-21T15:00:12.419 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:12.419 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:12.421 INFO:teuthology.orchestra.run.smithi032.stdout:3437 writesame 0x774c4c5 thru 0x7751efa data_size 0x9(0x5a36 bytes) 2024-09-21T15:00:12.426 INFO:teuthology.orchestra.run.smithi032.stdout:3438 read 0x5224459 thru 0x522a1c5 (0x5d6d bytes) 2024-09-21T15:00:12.427 INFO:teuthology.orchestra.run.smithi032.stdout:3440 writesame 0x35277b9 thru 0x3534390 data_size 0x2c(0xcbd8 bytes) 2024-09-21T15:00:12.429 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T15:00:12.440 INFO:teuthology.orchestra.run.smithi032.stdout:3442 trunc from 0xca09d15 to 0x9adc5bf 2024-09-21T15:00:12.465 INFO:teuthology.orchestra.run.smithi032.stdout:3443 writesame 0x81208ef thru 0x8130660 data_size 0x1(0xfd72 bytes) 2024-09-21T15:00:12.507 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:00:12.502+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:00:12.517 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:12.517 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:12.517 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:12.525 INFO:teuthology.orchestra.run.smithi032.stdout:3450 trunc from 0x9adc5bf to 0xccf9fbe 2024-09-21T15:00:12.537 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:12.537 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:12.537 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:12.537 INFO:teuthology.orchestra.run.smithi032.stdout:3453 write 0x950e110 thru 0x951d6cd (0xf5be bytes) 2024-09-21T15:00:12.540 INFO:teuthology.orchestra.run.smithi032.stdout:3454 writesame 0x6c79bac thru 0x6c80409 data_size 0x49(0x685e bytes) 2024-09-21T15:00:12.548 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T15:00:12.548 INFO:teuthology.orchestra.run.smithi032.stdout:3455 read 0xf32034 thru 0xf3e864 (0xc831 bytes) 2024-09-21T15:00:12.549 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:12.549 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:12.550 INFO:teuthology.orchestra.run.smithi032.stdout:3458 writesame 0x7dc6ecb thru 0x7dd1343 data_size 0x23(0xa479 bytes) 2024-09-21T15:00:12.553 INFO:teuthology.orchestra.run.smithi032.stdout:3459 write 0x8afc2f5 thru 0x8b0222c (0x5f38 bytes) 2024-09-21T15:00:12.560 INFO:teuthology.orchestra.run.smithi032.stdout:3460 write 0x92aed89 thru 0x92be5e3 (0xf85b bytes) 2024-09-21T15:00:12.567 INFO:teuthology.orchestra.run.smithi032.stdout:3461 trunc from 0xccf9fbe to 0xb18c535 2024-09-21T15:00:12.579 INFO:teuthology.orchestra.run.smithi032.stdout:3463 clone 26 order 21 su 32768 sc 10 2024-09-21T15:00:12.607 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:00:12.602+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:00:12.650 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:12.650 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:12.660 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T15:00:12.707 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:00:12.704+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:00:12.766 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:12.766 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:12.775 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T15:00:12.808 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:00:12.802+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:00:12.895 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:12.896 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:12.906 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T15:00:12.909 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:00:12.908+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:00:13.010 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:00:13.006+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:00:13.010 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:13.010 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:13.020 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T15:00:13.111 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:00:13.108+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:00:13.130 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:13.130 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:13.140 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T15:00:13.211 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:00:13.206+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:00:13.245 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T15:00:13.294 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:13.294 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:13.304 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T15:00:13.312 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:00:13.306+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:00:13.405 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:13.406 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:13.413 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:00:13.412+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:00:13.416 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T15:00:13.514 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:00:13.510+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:00:13.520 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:13.520 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:13.530 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T15:00:13.615 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:00:13.610+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:00:13.636 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:13.636 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:13.646 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T15:00:13.716 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:00:13.710+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:00:13.759 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:13.759 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:13.769 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T15:00:13.817 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:00:13.810+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:00:13.874 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:13.874 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:13.884 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T15:00:13.929 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:00:13.928+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:00:13.994 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:13.994 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:14.004 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T15:00:14.019 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:00:14.016+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:00:14.091 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T15:00:14.120 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:00:14.118+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:00:14.124 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:14.124 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:14.138 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T15:00:14.220 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:00:14.214+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:00:14.222 INFO:teuthology.orchestra.run.smithi032.stderr:nodeep-scrub is unset 2024-09-21T15:00:14.264 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T15:00:14.317 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T15:00:14.325 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:00:14.320+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:00:14.355 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T15:00:14.359 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:14.359 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:14.379 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T15:00:14.390 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T15:00:14.423 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-21T15:00:14.426 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:00:14.422+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:00:14.457 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T15:00:14.491 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T15:00:14.519 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:14.520 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T15:00:14.520 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.520 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:104908 3.8 3:17ab0d23:::rbd_data.1136ab4d6ff1.000000000000000a:4f [read 3375104~32768 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e130)", 2024-09-21T15:00:14.520 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:14.514936+0000", 2024-09-21T15:00:14.520 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.003946293, 2024-09-21T15:00:14.520 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0039781809999999999, 2024-09-21T15:00:14.520 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:14.520 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:14.520 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:14.520 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:14.520 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:14.520 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 104908 2024-09-21T15:00:14.521 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.521 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:14.521 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.521 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:14.521 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:14.514936+0000", 2024-09-21T15:00:14.521 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:14.521 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.521 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.521 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:14.521 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:14.514936+0000", 2024-09-21T15:00:14.521 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:14.521 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.521 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.521 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:14.522 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:14.514935+0000", 2024-09-21T15:00:14.522 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:14.522 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.522 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.522 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:14.522 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:14.514943+0000", 2024-09-21T15:00:14.522 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2230000000000001e-06 2024-09-21T15:00:14.522 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.522 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.522 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:14.522 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:14.514958+0000", 2024-09-21T15:00:14.522 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4800000000000001e-05 2024-09-21T15:00:14.522 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.522 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.522 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:14.523 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:14.514971+0000", 2024-09-21T15:00:14.523 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3128e-05 2024-09-21T15:00:14.523 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.523 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.523 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:14.523 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:14.518727+0000", 2024-09-21T15:00:14.523 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0037561629999999999 2024-09-21T15:00:14.523 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.523 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.523 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:14.523 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:14.518851+0000", 2024-09-21T15:00:14.523 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012409300000000001 2024-09-21T15:00:14.523 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:14.523 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:14.523 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:14.524 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.524 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.524 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:104909 3.8 3.c4b0d5e8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e130)", 2024-09-21T15:00:14.524 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:14.514977+0000", 2024-09-21T15:00:14.524 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0039049979999999998, 2024-09-21T15:00:14.524 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0040086269999999999, 2024-09-21T15:00:14.524 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:14.524 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-21T15:00:14.524 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:14.524 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:14.524 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:14.524 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 104909 2024-09-21T15:00:14.524 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.524 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:14.525 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.525 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:14.525 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:14.514977+0000", 2024-09-21T15:00:14.525 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:14.525 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.525 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.525 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:14.525 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:14.514977+0000", 2024-09-21T15:00:14.525 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:14.525 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.525 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.525 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:14.525 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:14.514977+0000", 2024-09-21T15:00:14.525 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:14.525 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.526 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.526 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:14.526 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:14.514980+0000", 2024-09-21T15:00:14.526 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3720000000000001e-06 2024-09-21T15:00:14.526 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.526 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.526 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:14.526 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:14.514995+0000", 2024-09-21T15:00:14.526 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4863e-05 2024-09-21T15:00:14.526 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.526 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.526 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:14.526 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:14.515001+0000", 2024-09-21T15:00:14.526 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7989999999999999e-06 2024-09-21T15:00:14.527 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:14.527 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:14.527 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:14.527 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.527 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.527 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:104910 3.8 3.c4b0d5e8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e130)", 2024-09-21T15:00:14.527 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:14.515006+0000", 2024-09-21T15:00:14.527 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0038758640000000001, 2024-09-21T15:00:14.527 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0040339959999999998, 2024-09-21T15:00:14.527 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:14.527 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-21T15:00:14.527 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:14.527 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:14.527 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:14.527 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 104910 2024-09-21T15:00:14.528 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.528 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:14.528 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.528 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:14.528 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:14.515006+0000", 2024-09-21T15:00:14.528 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:14.528 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.528 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.528 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:14.528 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:14.515006+0000", 2024-09-21T15:00:14.528 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:14.528 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.528 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.528 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:14.528 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:14.515006+0000", 2024-09-21T15:00:14.529 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:14.529 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.529 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.529 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:14.529 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:14.515009+0000", 2024-09-21T15:00:14.529 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3280000000000002e-06 2024-09-21T15:00:14.529 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.529 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.529 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:14.529 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:14.515019+0000", 2024-09-21T15:00:14.529 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6390000000000004e-06 2024-09-21T15:00:14.529 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.529 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.530 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:14.530 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:14.515024+0000", 2024-09-21T15:00:14.530 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5489999999999997e-06 2024-09-21T15:00:14.530 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:14.530 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:14.530 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:14.530 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.530 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.530 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:104911 3.8 3.c4b0d5e8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e130)", 2024-09-21T15:00:14.530 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:14.515029+0000", 2024-09-21T15:00:14.530 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0038530029999999998, 2024-09-21T15:00:14.530 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0040633290000000001, 2024-09-21T15:00:14.530 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:14.530 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "queued for pg", 2024-09-21T15:00:14.531 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:14.531 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:14.531 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:14.531 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 104911 2024-09-21T15:00:14.531 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.531 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:14.531 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.531 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:14.531 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:14.515029+0000", 2024-09-21T15:00:14.531 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:14.531 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.531 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.531 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:14.531 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:14.515029+0000", 2024-09-21T15:00:14.531 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:14.532 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.532 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.532 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:14.532 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:14.515029+0000", 2024-09-21T15:00:14.532 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:14.532 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.532 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.532 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:14.532 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:14.515032+0000", 2024-09-21T15:00:14.532 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0920000000000002e-06 2024-09-21T15:00:14.532 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.532 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.532 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:14.532 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:14.515047+0000", 2024-09-21T15:00:14.532 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.49e-05 2024-09-21T15:00:14.533 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.533 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.533 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:14.533 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:14.515053+0000", 2024-09-21T15:00:14.533 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1589999999999998e-06 2024-09-21T15:00:14.533 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:14.533 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:14.533 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:14.533 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:14.533 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-21T15:00:14.533 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 4 2024-09-21T15:00:14.533 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:14.534 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T15:00:14.534 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:00:14.534 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:00:14.535 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:00:14.535 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:00:14.535 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:00:14.535 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:00:14.535 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:00:14.535 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:00:14.535 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T15:00:14.537 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:00:14.532+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:00:14.635 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:00:14.630+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:00:14.650 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:14.650 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:00:14.650 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T15:00:14.650 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T15:00:14.650 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:14.660 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T15:00:14.736 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:00:14.732+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:00:14.818 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:14.818 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T15:00:14.818 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T15:00:14.818 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T15:00:14.818 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.818 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:1452 3.19 3:9d40fe72:::rbd_data.1136515ecb6b.0000000000000013:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T15:00:14.818 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:30.292755+0000", 2024-09-21T15:00:14.818 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 104.52093185, 2024-09-21T15:00:14.819 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043852191999999998, 2024-09-21T15:00:14.819 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:14.819 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:14.819 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:14.819 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:14.819 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:14.819 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1452 2024-09-21T15:00:14.819 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.819 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:14.819 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.819 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:14.819 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292755+0000", 2024-09-21T15:00:14.819 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:14.819 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.820 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.820 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:14.820 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292755+0000", 2024-09-21T15:00:14.820 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:14.820 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.820 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.820 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:14.820 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292754+0000", 2024-09-21T15:00:14.820 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:00:14.820 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.820 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.820 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:14.820 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292760+0000", 2024-09-21T15:00:14.820 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5029999999999997e-06 2024-09-21T15:00:14.821 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.821 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.821 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:14.821 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292771+0000", 2024-09-21T15:00:14.821 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1551e-05 2024-09-21T15:00:14.821 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.821 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.821 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:14.821 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292779+0000", 2024-09-21T15:00:14.821 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1559999999999998e-06 2024-09-21T15:00:14.821 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.821 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.821 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:14.821 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292896+0000", 2024-09-21T15:00:14.822 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000117304 2024-09-21T15:00:14.822 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.822 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.822 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:14.822 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.293124+0000", 2024-09-21T15:00:14.822 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00022819999999999999 2024-09-21T15:00:14.822 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.822 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.822 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 1", 2024-09-21T15:00:14.822 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.299967+0000", 2024-09-21T15:00:14.822 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0068427009999999996 2024-09-21T15:00:14.822 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.822 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.823 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:14.823 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336463+0000", 2024-09-21T15:00:14.823 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.036495842000000001 2024-09-21T15:00:14.823 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.823 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.823 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:14.823 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336480+0000", 2024-09-21T15:00:14.823 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7419999999999999e-05 2024-09-21T15:00:14.823 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.823 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.823 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:14.823 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336550+0000", 2024-09-21T15:00:14.823 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9951999999999997e-05 2024-09-21T15:00:14.823 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.823 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.824 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:14.824 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336607+0000", 2024-09-21T15:00:14.824 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7423000000000002e-05 2024-09-21T15:00:14.824 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:14.824 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:14.824 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:14.824 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.824 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.824 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:1567 3.8 3:139d3d55:::rbd_data.1136515ecb6b.0000000000000000:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T15:00:14.824 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:30.681087+0000", 2024-09-21T15:00:14.824 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 104.13259963500001, 2024-09-21T15:00:14.824 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.039009224000000002, 2024-09-21T15:00:14.824 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:14.824 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:14.825 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:14.825 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:14.825 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:14.825 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1567 2024-09-21T15:00:14.825 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.825 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:14.825 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.825 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:14.825 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681087+0000", 2024-09-21T15:00:14.825 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:14.825 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.825 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.825 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:14.825 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681087+0000", 2024-09-21T15:00:14.826 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:14.826 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.826 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.826 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:14.826 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681086+0000", 2024-09-21T15:00:14.826 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:00:14.826 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.826 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.826 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:14.826 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681097+0000", 2024-09-21T15:00:14.826 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.134e-05 2024-09-21T15:00:14.826 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.826 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.826 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:14.826 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681122+0000", 2024-09-21T15:00:14.827 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5313e-05 2024-09-21T15:00:14.827 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.827 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.827 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:14.827 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681138+0000", 2024-09-21T15:00:14.827 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5916000000000001e-05 2024-09-21T15:00:14.827 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.827 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.827 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:14.827 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681270+0000", 2024-09-21T15:00:14.827 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000131474 2024-09-21T15:00:14.827 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.827 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.827 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:14.827 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681416+0000", 2024-09-21T15:00:14.828 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014637 2024-09-21T15:00:14.828 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.828 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.828 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 7", 2024-09-21T15:00:14.828 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.693163+0000", 2024-09-21T15:00:14.828 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011747241 2024-09-21T15:00:14.828 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.828 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.828 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:14.828 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.719750+0000", 2024-09-21T15:00:14.828 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.026586852000000001 2024-09-21T15:00:14.828 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.828 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.828 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:14.828 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.719859+0000", 2024-09-21T15:00:14.828 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010925999999999999 2024-09-21T15:00:14.829 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.829 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.829 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:14.829 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.719951+0000", 2024-09-21T15:00:14.829 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1294999999999999e-05 2024-09-21T15:00:14.829 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.829 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.829 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:14.829 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.720097+0000", 2024-09-21T15:00:14.829 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014583 2024-09-21T15:00:14.829 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:14.829 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:14.829 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:14.829 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.829 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.830 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9381 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 303104~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:14.830 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158774+0000", 2024-09-21T15:00:14.830 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 95.654912780999993, 2024-09-21T15:00:14.830 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015309095999999999, 2024-09-21T15:00:14.830 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:14.830 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:14.830 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:14.830 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:14.830 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:14.830 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9381 2024-09-21T15:00:14.830 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.830 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:14.830 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.831 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:14.831 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158774+0000", 2024-09-21T15:00:14.831 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:14.831 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.831 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.831 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:14.831 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158774+0000", 2024-09-21T15:00:14.831 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:14.831 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.831 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.831 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:14.831 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158773+0000", 2024-09-21T15:00:14.831 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:00:14.832 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.832 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.832 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:14.832 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158779+0000", 2024-09-21T15:00:14.832 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6999999999999996e-06 2024-09-21T15:00:14.832 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.832 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.832 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:14.832 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158790+0000", 2024-09-21T15:00:14.832 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0594000000000001e-05 2024-09-21T15:00:14.832 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.832 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.832 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:14.832 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158798+0000", 2024-09-21T15:00:14.833 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5860000000000005e-06 2024-09-21T15:00:14.833 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.833 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.833 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:14.833 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173829+0000", 2024-09-21T15:00:14.833 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015030349 2024-09-21T15:00:14.833 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.833 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.833 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:14.833 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.173918+0000", 2024-09-21T15:00:14.833 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9375000000000004e-05 2024-09-21T15:00:14.833 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.833 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.833 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:14.833 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.174083+0000", 2024-09-21T15:00:14.834 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016536200000000001 2024-09-21T15:00:14.834 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:14.834 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:14.834 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:14.834 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.834 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.834 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:9382 3.2 3:45e0811e:::rbd_data.11363a21d2be.0000000000000002:10 [read 442368~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:14.834 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.158805+0000", 2024-09-21T15:00:14.834 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 95.654881876000005, 2024-09-21T15:00:14.834 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015592623, 2024-09-21T15:00:14.834 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:14.834 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:14.834 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:14.834 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:14.835 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:14.835 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 9382 2024-09-21T15:00:14.835 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.835 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:14.835 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.835 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:14.835 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158805+0000", 2024-09-21T15:00:14.835 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:14.835 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.835 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.835 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:14.835 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158805+0000", 2024-09-21T15:00:14.835 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:14.835 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.836 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.836 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:14.836 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158805+0000", 2024-09-21T15:00:14.836 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:14.836 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.836 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.836 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:14.836 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158808+0000", 2024-09-21T15:00:14.836 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9179999999999996e-06 2024-09-21T15:00:14.836 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.836 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.836 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:14.836 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158817+0000", 2024-09-21T15:00:14.836 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9299999999999992e-06 2024-09-21T15:00:14.836 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.836 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.837 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:14.837 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.158825+0000", 2024-09-21T15:00:14.837 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6739999999999997e-06 2024-09-21T15:00:14.837 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.837 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.837 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:14.837 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.174122+0000", 2024-09-21T15:00:14.837 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.01529694 2024-09-21T15:00:14.837 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.837 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.837 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:14.837 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.174185+0000", 2024-09-21T15:00:14.837 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3015000000000006e-05 2024-09-21T15:00:14.837 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.837 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.838 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:14.838 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.174398+0000", 2024-09-21T15:00:14.838 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021274799999999999 2024-09-21T15:00:14.838 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:14.838 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:14.838 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:14.838 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.838 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.838 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22702 3.19 3:98de45b8:::rbd_data.11366acc894.0000000000000000:19 [read 835584~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T15:00:14.838 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.140232+0000", 2024-09-21T15:00:14.838 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 84.673454593000002, 2024-09-21T15:00:14.838 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015338742000000001, 2024-09-21T15:00:14.838 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:14.838 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:14.839 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:14.839 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:14.839 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:14.839 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22702 2024-09-21T15:00:14.839 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.839 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:14.839 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.839 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:14.839 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140232+0000", 2024-09-21T15:00:14.839 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:14.839 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.839 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.839 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:14.839 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140232+0000", 2024-09-21T15:00:14.839 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:14.840 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.840 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.840 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:14.840 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140232+0000", 2024-09-21T15:00:14.840 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:14.840 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.840 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.840 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:14.840 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140234+0000", 2024-09-21T15:00:14.840 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.402e-06 2024-09-21T15:00:14.840 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.840 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.840 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:14.840 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140241+0000", 2024-09-21T15:00:14.840 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.054e-06 2024-09-21T15:00:14.841 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.841 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.841 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:14.841 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140246+0000", 2024-09-21T15:00:14.841 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0050000000000004e-06 2024-09-21T15:00:14.841 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.841 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.841 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:14.841 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.154315+0000", 2024-09-21T15:00:14.841 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014069440000000001 2024-09-21T15:00:14.841 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.841 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.841 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:14.842 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.154383+0000", 2024-09-21T15:00:14.842 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8151000000000005e-05 2024-09-21T15:00:14.842 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.842 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.842 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:14.842 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155571+0000", 2024-09-21T15:00:14.842 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001188029 2024-09-21T15:00:14.842 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:14.842 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:14.842 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:14.842 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.842 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.842 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22704 3.19 3:98de45b8:::rbd_data.11366acc894.0000000000000000:19 [read 966656~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T15:00:14.842 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.140266+0000", 2024-09-21T15:00:14.843 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 84.673421117999993, 2024-09-21T15:00:14.843 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015223443999999999, 2024-09-21T15:00:14.843 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:14.843 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:14.843 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:14.843 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:14.843 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:14.843 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22704 2024-09-21T15:00:14.843 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.843 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:14.843 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.843 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:14.843 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140266+0000", 2024-09-21T15:00:14.843 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:14.844 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.844 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.844 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:14.844 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140266+0000", 2024-09-21T15:00:14.844 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:14.844 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.844 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.844 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:14.844 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140266+0000", 2024-09-21T15:00:14.844 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:14.844 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.844 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.844 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:14.844 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140268+0000", 2024-09-21T15:00:14.845 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.475e-06 2024-09-21T15:00:14.845 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.845 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.845 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:14.845 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140274+0000", 2024-09-21T15:00:14.845 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.319e-06 2024-09-21T15:00:14.845 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.845 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.845 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:14.845 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140279+0000", 2024-09-21T15:00:14.845 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.882e-06 2024-09-21T15:00:14.845 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.845 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.845 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:14.846 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155264+0000", 2024-09-21T15:00:14.846 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014984655 2024-09-21T15:00:14.846 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.846 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.846 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:14.846 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155338+0000", 2024-09-21T15:00:14.846 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4091999999999995e-05 2024-09-21T15:00:14.846 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.846 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.846 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:14.846 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155489+0000", 2024-09-21T15:00:14.846 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000151349 2024-09-21T15:00:14.846 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:14.846 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:14.847 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:14.847 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.847 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.847 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:22705 3.19 3:98de45b8:::rbd_data.11366acc894.0000000000000000:19 [read 1032192~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T15:00:14.847 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:50.140284+0000", 2024-09-21T15:00:14.847 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 84.673403261000004, 2024-09-21T15:00:14.847 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015467896, 2024-09-21T15:00:14.847 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:14.847 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:14.847 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:14.847 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:14.847 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:14.847 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 22705 2024-09-21T15:00:14.847 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.848 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:14.848 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.848 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:14.848 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140284+0000", 2024-09-21T15:00:14.848 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:14.848 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.848 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.848 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:14.848 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140284+0000", 2024-09-21T15:00:14.848 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:14.848 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.848 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.848 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:14.848 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140283+0000", 2024-09-21T15:00:14.849 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:14.849 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.849 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.849 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:14.849 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140286+0000", 2024-09-21T15:00:14.849 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7130000000000001e-06 2024-09-21T15:00:14.849 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.849 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.849 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:14.849 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140292+0000", 2024-09-21T15:00:14.849 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3430000000000001e-06 2024-09-21T15:00:14.849 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.849 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.849 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:14.849 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.140298+0000", 2024-09-21T15:00:14.850 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9730000000000002e-06 2024-09-21T15:00:14.850 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.850 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.850 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:14.850 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155510+0000", 2024-09-21T15:00:14.850 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015211738000000001 2024-09-21T15:00:14.850 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.850 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.850 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:14.850 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155598+0000", 2024-09-21T15:00:14.850 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7822e-05 2024-09-21T15:00:14.850 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.850 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.850 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:14.851 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:50.155752+0000", 2024-09-21T15:00:14.851 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0001537 2024-09-21T15:00:14.851 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:14.851 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:14.851 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:14.851 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.851 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.851 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:37892 3.1b 3:d963c97f:::rbd_data.11365d19aa81.00000000000000be:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-21T15:00:14.851 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:04.029946+0000", 2024-09-21T15:00:14.851 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 70.783741493999997, 2024-09-21T15:00:14.851 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.017686021999999999, 2024-09-21T15:00:14.851 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:14.851 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:14.851 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:14.852 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:14.852 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:14.852 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 37892 2024-09-21T15:00:14.852 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.852 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:14.852 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.852 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:14.852 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.029946+0000", 2024-09-21T15:00:14.852 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:14.852 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.852 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.852 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:14.852 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.029946+0000", 2024-09-21T15:00:14.852 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:14.853 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.853 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.853 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:14.853 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.029944+0000", 2024-09-21T15:00:14.853 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:00:14.853 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.853 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.853 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:14.853 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.029953+0000", 2024-09-21T15:00:14.853 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7589999999999992e-06 2024-09-21T15:00:14.853 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.853 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.853 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:14.853 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.029971+0000", 2024-09-21T15:00:14.853 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7795000000000002e-05 2024-09-21T15:00:14.854 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.854 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.854 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:14.854 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.029984+0000", 2024-09-21T15:00:14.854 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.27e-05 2024-09-21T15:00:14.854 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.854 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.854 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:14.854 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.030072+0000", 2024-09-21T15:00:14.854 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8082000000000001e-05 2024-09-21T15:00:14.854 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.854 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.854 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:14.854 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.030276+0000", 2024-09-21T15:00:14.854 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00020375699999999999 2024-09-21T15:00:14.855 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.855 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.855 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-21T15:00:14.855 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.031634+0000", 2024-09-21T15:00:14.855 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0013586049999999999 2024-09-21T15:00:14.855 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.855 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.855 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:14.855 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.037554+0000", 2024-09-21T15:00:14.855 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0059197449999999997 2024-09-21T15:00:14.855 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.855 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.855 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:14.855 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.047490+0000", 2024-09-21T15:00:14.855 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0099361680000000004 2024-09-21T15:00:14.856 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.856 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.856 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:14.856 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.047569+0000", 2024-09-21T15:00:14.856 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8298000000000001e-05 2024-09-21T15:00:14.856 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.856 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.856 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:14.856 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.047632+0000", 2024-09-21T15:00:14.856 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3006e-05 2024-09-21T15:00:14.856 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:14.856 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:14.856 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:14.856 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.856 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.857 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:63820 3.19 3.626bc079 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-21T15:00:14.857 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:27.241293+0000", 2024-09-21T15:00:14.857 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 47.572394400999997, 2024-09-21T15:00:14.857 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.98723808199999996, 2024-09-21T15:00:14.857 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:14.857 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:14.857 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:14.857 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:14.857 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:14.857 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 63820 2024-09-21T15:00:14.857 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.857 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:14.857 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.857 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:14.857 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241293+0000", 2024-09-21T15:00:14.858 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:14.858 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.858 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.858 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:14.858 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241293+0000", 2024-09-21T15:00:14.858 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:14.858 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.858 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.858 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:14.858 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241291+0000", 2024-09-21T15:00:14.858 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:00:14.858 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.858 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.858 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:14.859 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241299+0000", 2024-09-21T15:00:14.859 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1529999999999997e-06 2024-09-21T15:00:14.859 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.859 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.859 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:14.859 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241317+0000", 2024-09-21T15:00:14.859 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7764000000000001e-05 2024-09-21T15:00:14.859 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.859 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.859 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:14.859 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241328+0000", 2024-09-21T15:00:14.859 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1245e-05 2024-09-21T15:00:14.859 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.859 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.860 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:14.860 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241407+0000", 2024-09-21T15:00:14.860 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8762000000000002e-05 2024-09-21T15:00:14.860 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.860 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.860 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:00:14.860 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241418+0000", 2024-09-21T15:00:14.860 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1695000000000001e-05 2024-09-21T15:00:14.860 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.860 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.860 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:14.860 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:28.228510+0000", 2024-09-21T15:00:14.860 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.98709166100000001 2024-09-21T15:00:14.860 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.860 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.861 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:14.861 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:28.228531+0000", 2024-09-21T15:00:14.861 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0696e-05 2024-09-21T15:00:14.861 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:14.861 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:14.861 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:14.861 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.861 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.861 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:73845 3.6 3:601bd01e:::rbd_data.1136f4939602.0000000000000015:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-21T15:00:14.861 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:42.393435+0000", 2024-09-21T15:00:14.861 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 32.420252290000001, 2024-09-21T15:00:14.861 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.030713345, 2024-09-21T15:00:14.861 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:14.861 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:14.861 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:14.862 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:14.862 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:14.862 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 73845 2024-09-21T15:00:14.862 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.862 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:14.862 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.862 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:14.862 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.393435+0000", 2024-09-21T15:00:14.862 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:14.862 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.862 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.862 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:14.862 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.393435+0000", 2024-09-21T15:00:14.862 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:14.863 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.863 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.863 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:14.863 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.393433+0000", 2024-09-21T15:00:14.863 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:00:14.863 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.863 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.863 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:14.863 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.393446+0000", 2024-09-21T15:00:14.863 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3013000000000001e-05 2024-09-21T15:00:14.863 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.863 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.863 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:14.863 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.393475+0000", 2024-09-21T15:00:14.864 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9615999999999999e-05 2024-09-21T15:00:14.864 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.864 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.864 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:14.864 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.393489+0000", 2024-09-21T15:00:14.864 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4103e-05 2024-09-21T15:00:14.864 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.864 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.864 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:14.864 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.393565+0000", 2024-09-21T15:00:14.864 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5195000000000001e-05 2024-09-21T15:00:14.864 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.864 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.864 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:14.865 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.393734+0000", 2024-09-21T15:00:14.865 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016909999999999999 2024-09-21T15:00:14.865 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.865 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.865 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 7", 2024-09-21T15:00:14.865 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.400243+0000", 2024-09-21T15:00:14.865 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0065093629999999998 2024-09-21T15:00:14.865 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.865 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.865 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:14.865 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.423721+0000", 2024-09-21T15:00:14.865 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.023477984 2024-09-21T15:00:14.865 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.865 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.866 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:14.866 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.423852+0000", 2024-09-21T15:00:14.866 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000131369 2024-09-21T15:00:14.866 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.866 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.866 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:14.866 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.424011+0000", 2024-09-21T15:00:14.866 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000158231 2024-09-21T15:00:14.866 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.866 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.866 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:14.866 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.424148+0000", 2024-09-21T15:00:14.866 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013732699999999999 2024-09-21T15:00:14.866 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:14.867 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:14.867 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:14.867 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.867 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.867 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:85605 3.7 3:e5bd5c60:::rbd_object_map.113637a2ddc2.0000000000000044:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115)", 2024-09-21T15:00:14.867 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:56.162490+0000", 2024-09-21T15:00:14.867 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 18.651196502000001, 2024-09-21T15:00:14.867 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015121122000000001, 2024-09-21T15:00:14.867 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:14.867 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:14.867 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:14.867 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:14.867 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:14.867 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 85605 2024-09-21T15:00:14.868 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.868 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:14.868 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.868 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:14.868 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.162490+0000", 2024-09-21T15:00:14.868 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:14.868 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.868 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.868 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:14.868 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.162490+0000", 2024-09-21T15:00:14.868 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:14.868 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.868 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.868 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:14.868 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.162490+0000", 2024-09-21T15:00:14.869 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:14.869 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.869 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.869 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:14.869 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.162494+0000", 2024-09-21T15:00:14.869 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.5020000000000001e-06 2024-09-21T15:00:14.869 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.869 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.869 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:14.869 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.162498+0000", 2024-09-21T15:00:14.869 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6920000000000001e-06 2024-09-21T15:00:14.869 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.869 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.869 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:14.869 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.162503+0000", 2024-09-21T15:00:14.870 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.617e-06 2024-09-21T15:00:14.870 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.870 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.870 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:14.870 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.174549+0000", 2024-09-21T15:00:14.870 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012046238000000001 2024-09-21T15:00:14.870 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.870 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.870 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:14.870 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.174640+0000", 2024-09-21T15:00:14.870 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0810999999999995e-05 2024-09-21T15:00:14.870 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.870 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.870 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T15:00:14.871 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.175156+0000", 2024-09-21T15:00:14.871 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00051634700000000001 2024-09-21T15:00:14.871 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.871 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.871 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:14.871 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.177347+0000", 2024-09-21T15:00:14.871 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0021907319999999999 2024-09-21T15:00:14.871 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.871 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.871 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:14.871 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.177407+0000", 2024-09-21T15:00:14.871 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.9657999999999997e-05 2024-09-21T15:00:14.871 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.872 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.872 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:14.872 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.177498+0000", 2024-09-21T15:00:14.872 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1225999999999997e-05 2024-09-21T15:00:14.872 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.872 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.872 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:14.872 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:56.177612+0000", 2024-09-21T15:00:14.872 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011362999999999999 2024-09-21T15:00:14.872 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:14.872 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:14.872 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:14.872 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.872 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.872 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:93347 3.9 e119/90 3:92bf9a56:::rbd_object_map.1136b9c49b1c:head v 119'420, mlcod=119'415)", 2024-09-21T15:00:14.873 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:02.217952+0000", 2024-09-21T15:00:14.873 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 12.59573494, 2024-09-21T15:00:14.873 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.016462039000000001, 2024-09-21T15:00:14.873 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:14.873 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:14.873 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:14.873 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.873 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:14.873 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:02.217952+0000", 2024-09-21T15:00:14.873 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:14.873 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.873 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.873 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:14.873 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:02.217952+0000", 2024-09-21T15:00:14.873 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:14.874 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.874 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.874 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:14.874 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:02.217951+0000", 2024-09-21T15:00:14.874 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:00:14.874 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.874 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.874 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:14.874 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:02.217963+0000", 2024-09-21T15:00:14.874 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2197e-05 2024-09-21T15:00:14.874 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.874 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.874 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:14.874 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:02.217997+0000", 2024-09-21T15:00:14.874 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4468999999999998e-05 2024-09-21T15:00:14.875 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.875 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.875 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:14.875 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:02.218012+0000", 2024-09-21T15:00:14.875 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.452e-05 2024-09-21T15:00:14.875 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.875 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.875 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:14.875 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:02.218095+0000", 2024-09-21T15:00:14.875 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2817000000000003e-05 2024-09-21T15:00:14.875 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.875 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.875 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:14.875 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:02.218126+0000", 2024-09-21T15:00:14.876 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0814e-05 2024-09-21T15:00:14.876 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.876 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.876 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:14.876 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:02.234371+0000", 2024-09-21T15:00:14.876 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.016245384000000002 2024-09-21T15:00:14.876 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.876 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.876 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:14.876 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:02.234414+0000", 2024-09-21T15:00:14.876 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3170999999999997e-05 2024-09-21T15:00:14.876 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:14.876 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:14.876 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:14.877 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.877 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.877 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:93349 3.19 3:9a74bbc0:::rbd_data.1136b9c49b1c.000000000000001c:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-21T15:00:14.877 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:02.218917+0000", 2024-09-21T15:00:14.877 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 12.594769528, 2024-09-21T15:00:14.877 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015624626000000001, 2024-09-21T15:00:14.877 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:14.877 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:14.877 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:14.877 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:14.877 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:14.877 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 93349 2024-09-21T15:00:14.877 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.877 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:14.878 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.878 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:14.878 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:02.218917+0000", 2024-09-21T15:00:14.878 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:14.878 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.878 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.878 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:14.878 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:02.218917+0000", 2024-09-21T15:00:14.878 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:14.878 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.878 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.878 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:14.878 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:02.218916+0000", 2024-09-21T15:00:14.878 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:00:14.879 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.879 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.879 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:14.879 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:02.218931+0000", 2024-09-21T15:00:14.879 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4712000000000001e-05 2024-09-21T15:00:14.879 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.879 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.879 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:14.879 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:02.218949+0000", 2024-09-21T15:00:14.879 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8094999999999999e-05 2024-09-21T15:00:14.879 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.879 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.879 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:14.879 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:02.218958+0000", 2024-09-21T15:00:14.879 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3670000000000003e-06 2024-09-21T15:00:14.880 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.880 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.880 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:14.880 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:02.219029+0000", 2024-09-21T15:00:14.880 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0859999999999996e-05 2024-09-21T15:00:14.880 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.880 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.880 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:14.880 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:02.219159+0000", 2024-09-21T15:00:14.880 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000130211 2024-09-21T15:00:14.880 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.880 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.880 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 1", 2024-09-21T15:00:14.881 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:02.222970+0000", 2024-09-21T15:00:14.881 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0038108970000000002 2024-09-21T15:00:14.881 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.881 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.881 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:14.881 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:02.234423+0000", 2024-09-21T15:00:14.881 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011452413 2024-09-21T15:00:14.881 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.881 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.881 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:14.881 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:02.234445+0000", 2024-09-21T15:00:14.881 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2367000000000001e-05 2024-09-21T15:00:14.881 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.881 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.882 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:14.882 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:02.234499+0000", 2024-09-21T15:00:14.882 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3634999999999997e-05 2024-09-21T15:00:14.882 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.882 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.882 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:14.882 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:02.234542+0000", 2024-09-21T15:00:14.882 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3356999999999998e-05 2024-09-21T15:00:14.882 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:14.882 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:14.882 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:14.882 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.882 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.883 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:97619 3.1 3:8730dd9c:::rbd_data.1136bfc79a70.0000000000000000:head [write 0~4718592 [fadvise_sequential+fadvise_nocache],write 5242880~16384 [fadvise_sequential+fadvise_nocache],write 5324800~147456 [fadvise_sequential+fadvise_nocache],write 5537792~147456 [fadvise_sequential+fadvise_nocache],write 5750784~16384 [fadvise_sequential+fadvise_nocache],write 5832704~147456 [fadvise_sequential+fadvise_nocache],write 6045696~147456 [fadvise_sequential+fadvise_nocache],write 6258688~32768 [fadvise_sequential+fadvise_nocache],write 6340608~147456 [fadvise_sequential+fadvise_nocache],write 6553600~147456 [fadvise_sequential+fadvise_nocache],write 6766592~49152 [fadvise_sequential+fadvise_nocache],write 6848512~147456 [fadvise_sequential+fadvise_nocache],write 7061504~147456 [fadvise_sequential+fadvise_nocache],write 7274496~65536 [fadvise_sequential+fadvise_nocache],write 7356416~147456 [fadvise_sequential+fadvise_nocache],write 7569408~147456 [fadvise_sequential+fadvise_nocache],write 7782400~229376 [fadvise_sequential+fadvise_nocache],write 8077312~147456 [fadvise_sequential+fadvise_nocache],write 8290304~98304 [fadvise_sequential+fadvise_nocache],write 8404992~114688 [fadvise_sequential+fadvise_nocache],write 8617984~114688 [fadvise_sequential+fadvise_nocache],write 8830976~81920 [fadvise_sequential+fadvise_nocache],write 8994816~32768 [fadvise_sequential+fadvise_nocache],write 9207808~32768 [fadvise_sequential+fadvise_nocache],write 9420800~16384 [fadvise_sequential+fadvise_nocache],write 9502720~32768 [fadvise_sequential+fadvise_nocache],write 9715712~32768 [fadvise_sequential+fadvise_nocache],write 9928704~229376 [fadvise_sequential+fadvise_nocache],write 10174464~196608 [fadvise_sequential+fadvise_nocache],write 10387456~278528 [fadvise_sequential+fadvise_nocache],write 10682368~196608 [fadvise_sequential+fadvise_nocache],write 10895360~278528 [fadvise_sequential+fadvise_nocache],write 11190272~196608 [fadvise_sequential+fadvise_nocache],write 11403264~196608 [fadvise_sequential+fadvise_nocache],write 11616256~65536 [fadvise_sequential+fadvise_nocache],write 11714560~98304 [fadvise_sequential+fadvise_nocache],write 11829248~65536 [fadvise_sequential+fadvise_nocache],write 11927552~98304 [fadvise_sequential+fadvise_nocache],write 12042240~65536 [fadvise_sequential+fadvise_nocache],write 12124160~65536 [fadvise_sequential+fadvise_nocache],write 12222464~98304 [fadvise_sequential+fadvise_nocache],write 12337152~65536 [fadvise_sequential+fadvise_nocache],write 12435456~98304 [fadvise_sequential+fadvise_nocache],write 12550144~65536 [fadvise_sequential+fadvise_nocache],write 12730368~98304 [fadvise_sequential+fadvise_nocache],write 12943360~98304 [fadvise_sequential+fadvise_nocache],write 13107200~16384 [fadvise_sequential+fadvise_nocache],write 13238272~98304 [fadvise_sequential+fadvise_nocache],write 13451264~98304 [fadvise_sequential+fadvise_nocache],write 13746176~98304 [fadvise_sequential+fadvise_nocache],write 13959168~98304 [fadvise_sequential+fadvise_nocache],write 14254080~64264 [fadvise_sequential+fadvise_nocache],write 14319616~19759 [fadvise_sequential+fadvise_nocache],write 15204352~32768 [fadvise_sequential+fadvise_nocache],write 15368192~81920 [fadvise_sequential+fadvise_nocache],write 15581184~81920 [fadvise_sequential+fadvise_nocache],write 15728640~16384 [fadvise_sequential+fadvise_nocache],write 15876096~81920 [fadvise_sequential+fadvise_nocache],write 16089088~81920 [fadvise_sequential+fadvise_nocache],write 16384000~81920 [fadvise_sequential+fadvise_nocache],write 16596992~81920 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:14.883 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.255412+0000", 2024-09-21T15:00:14.883 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.5582747440000002, 2024-09-21T15:00:14.883 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.058037909999999998, 2024-09-21T15:00:14.883 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:14.883 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:14.883 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:14.883 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:14.883 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:14.883 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97619 2024-09-21T15:00:14.883 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.883 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:14.883 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.883 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:14.884 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.255412+0000", 2024-09-21T15:00:14.884 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:14.884 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.884 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.884 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:14.884 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.255412+0000", 2024-09-21T15:00:14.884 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:14.884 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.884 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.884 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:14.884 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.255410+0000", 2024-09-21T15:00:14.884 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-21T15:00:14.884 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.884 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.885 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:14.885 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.288087+0000", 2024-09-21T15:00:14.885 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.032677337000000001 2024-09-21T15:00:14.885 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.885 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.885 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:14.885 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.288114+0000", 2024-09-21T15:00:14.885 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6905000000000001e-05 2024-09-21T15:00:14.885 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.885 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.885 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:14.885 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.288128+0000", 2024-09-21T15:00:14.885 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3560999999999999e-05 2024-09-21T15:00:14.885 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.886 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.886 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:14.886 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.288193+0000", 2024-09-21T15:00:14.886 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5747000000000006e-05 2024-09-21T15:00:14.886 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.886 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.886 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:14.886 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.289299+0000", 2024-09-21T15:00:14.886 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0011056390000000001 2024-09-21T15:00:14.886 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.886 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.886 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T15:00:14.886 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.290870+0000", 2024-09-21T15:00:14.886 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001570718 2024-09-21T15:00:14.887 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.887 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.887 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:14.887 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.312525+0000", 2024-09-21T15:00:14.887 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.021655427000000001 2024-09-21T15:00:14.887 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.887 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.887 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:14.887 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.313312+0000", 2024-09-21T15:00:14.887 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00078665500000000004 2024-09-21T15:00:14.887 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.887 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.887 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:14.887 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.313413+0000", 2024-09-21T15:00:14.888 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010131700000000001 2024-09-21T15:00:14.888 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.888 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.888 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:14.888 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.313450+0000", 2024-09-21T15:00:14.888 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7032000000000001e-05 2024-09-21T15:00:14.888 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:14.888 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:14.888 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:14.888 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.888 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.888 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:97734 3.19 3:9fcf754f:::rbd_data.1136bfc79a70.0000000000000009:head [write 0~4194304 [fadvise_sequential+fadvise_nocache],write 4784128~147456 [fadvise_sequential+fadvise_nocache],write 4997120~147456 [fadvise_sequential+fadvise_nocache],write 5210112~32768 [fadvise_sequential+fadvise_nocache],write 5292032~147456 [fadvise_sequential+fadvise_nocache],write 5505024~147456 [fadvise_sequential+fadvise_nocache],write 5718016~49152 [fadvise_sequential+fadvise_nocache],write 5799936~147456 [fadvise_sequential+fadvise_nocache],write 6012928~147456 [fadvise_sequential+fadvise_nocache],write 6225920~65536 [fadvise_sequential+fadvise_nocache],write 6307840~147456 [fadvise_sequential+fadvise_nocache],write 6520832~147456 [fadvise_sequential+fadvise_nocache],write 6733824~229376 [fadvise_sequential+fadvise_nocache],write 7028736~147456 [fadvise_sequential+fadvise_nocache],write 7241728~229376 [fadvise_sequential+fadvise_nocache],write 7536640~147456 [fadvise_sequential+fadvise_nocache],write 7749632~229376 [fadvise_sequential+fadvise_nocache],write 8077312~114688 [fadvise_sequential+fadvise_nocache],write 8290304~98304 [fadvise_sequential+fadvise_nocache],write 8404992~81920 [fadvise_sequential+fadvise_nocache],write 8617984~81920 [fadvise_sequential+fadvise_nocache],write 8830976~81920 [fadvise_sequential+fadvise_nocache],write 8962048~32768 [fadvise_sequential+fadvise_nocache],write 9175040~32768 [fadvise_sequential+fadvise_nocache],write 9388032~32768 [fadvise_sequential+fadvise_nocache],write 9469952~32768 [fadvise_sequential+fadvise_nocache],write 9682944~20092 [fadvise_sequential+fadvise_nocache],write 9961472~163840 [fadvise_sequential+fadvise_nocache],write 10141696~196608 [fadvise_sequential+fadvise_nocache],write 10354688~278528 [fadvise_sequential+fadvise_nocache],write 10649600~196608 [fadvise_sequential+fadvise_nocache],write 10862592~196608 [fadvise_sequential+fadvise_nocache],write 11075584~65536 [fadvise_sequential+fadvise_nocache],write 11173888~98304 [fadvise_sequential+fadvise_nocache],write 11288576~65536 [fadvise_sequential+fadvise_nocache],write 11386880~98304 [fadvise_sequential+fadvise_nocache],write 11501568~65536 [fadvise_sequential+fadvise_nocache],write 11583488~65536 [fadvise_sequential+fadvise_nocache],write 11681792~98304 [fadvise_sequential+fadvise_nocache],write 11796480~65536 [fadvise_sequential+fadvise_nocache],write 11894784~98304 [fadvise_sequential+fadvise_nocache],write 12009472~65536 [fadvise_sequential+fadvise_nocache],write 12189696~98304 [fadvise_sequential+fadvise_nocache],write 12402688~98304 [fadvise_sequential+fadvise_nocache],write 12697600~98304 [fadvise_sequential+fadvise_nocache],write 12910592~98304 [fadvise_sequential+fadvise_nocache],write 13205504~98304 [fadvise_sequential+fadvise_nocache],write 13418496~98304 [fadvise_sequential+fadvise_nocache],write 13713408~98304 [fadvise_sequential+fadvise_nocache],write 13926400~98304 [fadvise_sequential+fadvise_nocache],write 14139392~16384 [fadvise_sequential+fadvise_nocache],write 14680064~16384 [fadvise_sequential+fadvise_nocache],write 14827520~81920 [fadvise_sequential+fadvise_nocache],write 15040512~81920 [fadvise_sequential+fadvise_nocache],write 15335424~81920 [fadvise_sequential+fadvise_nocache],write 15548416~81920 [fadvise_sequential+fadvise_nocache],write 15843328~81920 [fadvise_sequential+fadvise_nocache],write 16056320~81920 [fadvise_sequential+fadvise_nocache],write 16351232~81920 [fadvise_sequential+fadvise_nocache],write 16564224~81920 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:14.888 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.354331+0000", 2024-09-21T15:00:14.888 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.4593563540000005, 2024-09-21T15:00:14.889 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.086284870999999999, 2024-09-21T15:00:14.889 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:14.889 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:14.889 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:14.889 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:14.889 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:14.889 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97734 2024-09-21T15:00:14.889 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.889 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:14.889 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.889 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:14.889 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.354331+0000", 2024-09-21T15:00:14.889 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:14.889 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.890 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.890 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:14.890 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.354331+0000", 2024-09-21T15:00:14.890 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:14.890 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.890 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.890 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:14.890 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.354329+0000", 2024-09-21T15:00:14.890 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:00:14.890 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.890 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.890 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:14.890 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.370234+0000", 2024-09-21T15:00:14.890 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015905348999999999 2024-09-21T15:00:14.890 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.891 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.891 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:14.891 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.370286+0000", 2024-09-21T15:00:14.891 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1656999999999997e-05 2024-09-21T15:00:14.891 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.891 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.891 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:14.891 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.370315+0000", 2024-09-21T15:00:14.891 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9204e-05 2024-09-21T15:00:14.891 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.891 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.891 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:14.891 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.370395+0000", 2024-09-21T15:00:14.891 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0395999999999994e-05 2024-09-21T15:00:14.892 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.892 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.892 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:14.892 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.370604+0000", 2024-09-21T15:00:14.892 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000208684 2024-09-21T15:00:14.892 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.892 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.892 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 1", 2024-09-21T15:00:14.892 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.371701+0000", 2024-09-21T15:00:14.892 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0010973140000000001 2024-09-21T15:00:14.892 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.892 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.892 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:14.892 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.405485+0000", 2024-09-21T15:00:14.892 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.033783615000000003 2024-09-21T15:00:14.893 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.893 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.893 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:14.893 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.440211+0000", 2024-09-21T15:00:14.893 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034725757000000003 2024-09-21T15:00:14.893 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.893 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.893 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:14.893 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.440497+0000", 2024-09-21T15:00:14.893 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00028673099999999998 2024-09-21T15:00:14.893 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.893 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.893 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:14.893 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.440616+0000", 2024-09-21T15:00:14.893 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000118256 2024-09-21T15:00:14.894 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:14.894 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:14.894 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:14.894 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.894 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.894 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:98201 3.19 3:9efc5cfe:::rbd_data.1136b9c49b1c.0000000000000021:49 [read 3555328~32768 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:14.894 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.390232+0000", 2024-09-21T15:00:14.894 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.4234549209999994, 2024-09-21T15:00:14.894 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015244064999999999, 2024-09-21T15:00:14.894 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:14.894 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:14.894 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:14.894 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:14.894 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:14.895 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 98201 2024-09-21T15:00:14.895 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.895 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:14.895 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.895 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:14.895 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.390232+0000", 2024-09-21T15:00:14.895 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:14.895 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.895 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.895 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:14.895 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.390232+0000", 2024-09-21T15:00:14.895 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:14.895 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.895 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.896 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:14.896 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.390232+0000", 2024-09-21T15:00:14.896 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:14.896 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.896 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.896 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:14.896 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.390234+0000", 2024-09-21T15:00:14.896 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3159999999999999e-06 2024-09-21T15:00:14.896 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.896 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.896 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:14.896 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.390240+0000", 2024-09-21T15:00:14.896 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7189999999999998e-06 2024-09-21T15:00:14.896 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.896 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.897 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:14.897 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.390244+0000", 2024-09-21T15:00:14.897 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5530000000000003e-06 2024-09-21T15:00:14.897 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.897 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.897 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:14.897 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.405303+0000", 2024-09-21T15:00:14.897 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015058861999999999 2024-09-21T15:00:14.897 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.897 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.897 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:14.897 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.405381+0000", 2024-09-21T15:00:14.897 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7880999999999995e-05 2024-09-21T15:00:14.897 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.897 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.897 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:14.898 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.405476+0000", 2024-09-21T15:00:14.898 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5014999999999997e-05 2024-09-21T15:00:14.898 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:14.898 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:14.898 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:14.898 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.898 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.898 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:97734 3.19 3:9fcf754f:::rbd_data.1136bfc79a70.0000000000000009:head [write 0~4194304 [fadvise_sequential+fadvise_nocache],write 4784128~147456 [fadvise_sequential+fadvise_nocache],write 4997120~147456 [fadvise_sequential+fadvise_nocache],write 5210112~32768 [fadvise_sequential+fadvise_nocache],write 5292032~147456 [fadvise_sequential+fadvise_nocache],write 5505024~147456 [fadvise_sequential+fadvise_nocache],write 5718016~49152 [fadvise_sequential+fadvise_nocache],write 5799936~147456 [fadvise_sequential+fadvise_nocache],write 6012928~147456 [fadvise_sequential+fadvise_nocache],write 6225920~65536 [fadvise_sequential+fadvise_nocache],write 6307840~147456 [fadvise_sequential+fadvise_nocache],write 6520832~147456 [fadvise_sequential+fadvise_nocache],write 6733824~229376 [fadvise_sequential+fadvise_nocache],write 7028736~147456 [fadvise_sequential+fadvise_nocache],write 7241728~229376 [fadvise_sequential+fadvise_nocache],write 7536640~147456 [fadvise_sequential+fadvise_nocache],write 7749632~229376 [fadvise_sequential+fadvise_nocache],write 8077312~114688 [fadvise_sequential+fadvise_nocache],write 8290304~98304 [fadvise_sequential+fadvise_nocache],write 8404992~81920 [fadvise_sequential+fadvise_nocache],write 8617984~81920 [fadvise_sequential+fadvise_nocache],write 8830976~81920 [fadvise_sequential+fadvise_nocache],write 8962048~32768 [fadvise_sequential+fadvise_nocache],write 9175040~32768 [fadvise_sequential+fadvise_nocache],write 9388032~32768 [fadvise_sequential+fadvise_nocache],write 9469952~32768 [fadvise_sequential+fadvise_nocache],write 9682944~20092 [fadvise_sequential+fadvise_nocache],write 9961472~163840 [fadvise_sequential+fadvise_nocache],write 10141696~196608 [fadvise_sequential+fadvise_nocache],write 10354688~278528 [fadvise_sequential+fadvise_nocache],write 10649600~196608 [fadvise_sequential+fadvise_nocache],write 10862592~196608 [fadvise_sequential+fadvise_nocache],write 11075584~65536 [fadvise_sequential+fadvise_nocache],write 11173888~98304 [fadvise_sequential+fadvise_nocache],write 11288576~65536 [fadvise_sequential+fadvise_nocache],write 11386880~98304 [fadvise_sequential+fadvise_nocache],write 11501568~65536 [fadvise_sequential+fadvise_nocache],write 11583488~65536 [fadvise_sequential+fadvise_nocache],write 11681792~98304 [fadvise_sequential+fadvise_nocache],write 11796480~65536 [fadvise_sequential+fadvise_nocache],write 11894784~98304 [fadvise_sequential+fadvise_nocache],write 12009472~65536 [fadvise_sequential+fadvise_nocache],write 12189696~98304 [fadvise_sequential+fadvise_nocache],write 12402688~98304 [fadvise_sequential+fadvise_nocache],write 12697600~98304 [fadvise_sequential+fadvise_nocache],write 12910592~98304 [fadvise_sequential+fadvise_nocache],write 13205504~98304 [fadvise_sequential+fadvise_nocache],write 13418496~98304 [fadvise_sequential+fadvise_nocache],write 13713408~98304 [fadvise_sequential+fadvise_nocache],write 13926400~98304 [fadvise_sequential+fadvise_nocache],write 14139392~16384 [fadvise_sequential+fadvise_nocache],write 14680064~16384 [fadvise_sequential+fadvise_nocache],write 14827520~81920 [fadvise_sequential+fadvise_nocache],write 15040512~81920 [fadvise_sequential+fadvise_nocache],write 15335424~81920 [fadvise_sequential+fadvise_nocache],write 15548416~81920 [fadvise_sequential+fadvise_nocache],write 15843328~81920 [fadvise_sequential+fadvise_nocache],write 16056320~81920 [fadvise_sequential+fadvise_nocache],write 16351232~81920 [fadvise_sequential+fadvise_nocache],write 16564224~81920 [fadvise_sequential+fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:14.898 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.412887+0000", 2024-09-21T15:00:14.898 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.4007996299999999, 2024-09-21T15:00:14.898 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.027651709999999999, 2024-09-21T15:00:14.898 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:14.898 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "waiting for ondisk", 2024-09-21T15:00:14.898 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:14.902 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:14.902 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:14.902 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97734 2024-09-21T15:00:14.902 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.902 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:14.902 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.902 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:14.902 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.412887+0000", 2024-09-21T15:00:14.902 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:14.902 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.902 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.902 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:14.902 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.412887+0000", 2024-09-21T15:00:14.903 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:14.903 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.903 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.903 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:14.903 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.412886+0000", 2024-09-21T15:00:14.903 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:00:14.903 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.903 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.903 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:14.903 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.428395+0000", 2024-09-21T15:00:14.903 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015509448 2024-09-21T15:00:14.903 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.903 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.904 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:14.904 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.428417+0000", 2024-09-21T15:00:14.904 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1263999999999998e-05 2024-09-21T15:00:14.904 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.904 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.904 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:14.904 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.428425+0000", 2024-09-21T15:00:14.904 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8589999999999994e-06 2024-09-21T15:00:14.904 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.904 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.904 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:14.904 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.428481+0000", 2024-09-21T15:00:14.904 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5087000000000003e-05 2024-09-21T15:00:14.904 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.905 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.905 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for ondisk", 2024-09-21T15:00:14.905 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.428628+0000", 2024-09-21T15:00:14.905 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014714599999999999 2024-09-21T15:00:14.905 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.905 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.905 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:14.905 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.440539+0000", 2024-09-21T15:00:14.905 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011911424 2024-09-21T15:00:14.905 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:14.905 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:14.905 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:14.905 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.905 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.906 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:98562 3.9 e122/90 3:911e52bc:::rbd_data.1136bfc79a70.000000000000000a:head v 122'620, mlcod=121'618)", 2024-09-21T15:00:14.906 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.430966+0000", 2024-09-21T15:00:14.906 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.3827211639999994, 2024-09-21T15:00:14.906 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.031245016, 2024-09-21T15:00:14.906 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:14.906 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:14.906 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:14.906 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.906 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:14.906 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.430966+0000", 2024-09-21T15:00:14.906 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:14.906 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.906 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.906 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:14.906 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.430966+0000", 2024-09-21T15:00:14.907 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:14.907 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.907 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.907 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:14.907 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.430965+0000", 2024-09-21T15:00:14.907 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:00:14.907 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.907 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.907 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:14.907 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.447521+0000", 2024-09-21T15:00:14.907 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.016556006000000002 2024-09-21T15:00:14.907 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.907 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.908 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:14.908 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.447543+0000", 2024-09-21T15:00:14.908 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2121e-05 2024-09-21T15:00:14.908 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.908 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.908 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:14.908 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.447551+0000", 2024-09-21T15:00:14.908 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6310000000000003e-06 2024-09-21T15:00:14.908 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.908 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.908 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:14.908 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.447632+0000", 2024-09-21T15:00:14.908 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1388000000000001e-05 2024-09-21T15:00:14.908 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.909 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.909 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:14.909 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.447662+0000", 2024-09-21T15:00:14.909 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0111999999999999e-05 2024-09-21T15:00:14.909 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.909 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.909 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:14.909 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.462185+0000", 2024-09-21T15:00:14.909 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014522863 2024-09-21T15:00:14.909 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.909 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.909 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:14.909 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.462211+0000", 2024-09-21T15:00:14.909 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5945e-05 2024-09-21T15:00:14.910 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:14.910 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:14.910 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:14.910 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.910 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.910 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:100151 3.8 3:17ab0d23:::rbd_data.1136ab4d6ff1.000000000000000a:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-21T15:00:14.910 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:10.762267+0000", 2024-09-21T15:00:14.910 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.0514196599999996, 2024-09-21T15:00:14.910 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.029791319, 2024-09-21T15:00:14.910 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:14.910 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:14.910 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:14.910 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:14.910 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:14.910 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 100151 2024-09-21T15:00:14.911 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.911 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:14.911 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.911 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:14.911 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:10.762267+0000", 2024-09-21T15:00:14.911 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:14.911 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.911 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.911 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:14.911 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:10.762267+0000", 2024-09-21T15:00:14.911 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:14.911 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.911 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.911 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:14.911 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:10.762264+0000", 2024-09-21T15:00:14.912 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999967 2024-09-21T15:00:14.912 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.912 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.912 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:14.912 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:10.762283+0000", 2024-09-21T15:00:14.912 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9072999999999998e-05 2024-09-21T15:00:14.912 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.912 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.912 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:14.912 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:10.762325+0000", 2024-09-21T15:00:14.912 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2222000000000003e-05 2024-09-21T15:00:14.912 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.912 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.912 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:14.913 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:10.762348+0000", 2024-09-21T15:00:14.913 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2857000000000002e-05 2024-09-21T15:00:14.913 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.913 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.913 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:14.913 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:10.762490+0000", 2024-09-21T15:00:14.913 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000141667 2024-09-21T15:00:14.913 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.913 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.913 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:14.913 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:10.762751+0000", 2024-09-21T15:00:14.913 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00026079 2024-09-21T15:00:14.913 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.913 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.914 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 7", 2024-09-21T15:00:14.914 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:10.772217+0000", 2024-09-21T15:00:14.914 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0094660960000000002 2024-09-21T15:00:14.914 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.914 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.914 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:14.914 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:10.791315+0000", 2024-09-21T15:00:14.914 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.019098674999999999 2024-09-21T15:00:14.914 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.914 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.914 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:14.914 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:10.791938+0000", 2024-09-21T15:00:14.914 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00062279200000000005 2024-09-21T15:00:14.914 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.915 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.915 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:14.915 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:10.792005+0000", 2024-09-21T15:00:14.915 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6884000000000005e-05 2024-09-21T15:00:14.915 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.915 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.915 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:14.915 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:10.792059+0000", 2024-09-21T15:00:14.915 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3497e-05 2024-09-21T15:00:14.915 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:14.915 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:14.915 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:14.915 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.915 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.915 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:100152 3.7 3:e02630f6:::rbd_data.1136ab4d6ff1.000000000000000b:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-21T15:00:14.916 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:10.763146+0000", 2024-09-21T15:00:14.916 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.0505405369999998, 2024-09-21T15:00:14.916 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.029305525999999998, 2024-09-21T15:00:14.916 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:14.916 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:14.916 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:14.916 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:14.916 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:14.916 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 100152 2024-09-21T15:00:14.916 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.916 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:14.916 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.916 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:14.916 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:10.763146+0000", 2024-09-21T15:00:14.916 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:14.917 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.917 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.917 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:14.917 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:10.763146+0000", 2024-09-21T15:00:14.917 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:14.917 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.917 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.917 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:14.917 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:10.763139+0000", 2024-09-21T15:00:14.917 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999924 2024-09-21T15:00:14.917 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.917 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.917 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:14.917 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:10.763158+0000", 2024-09-21T15:00:14.917 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9040000000000001e-05 2024-09-21T15:00:14.918 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.918 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.918 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:14.918 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:10.763185+0000", 2024-09-21T15:00:14.918 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6988999999999999e-05 2024-09-21T15:00:14.918 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.918 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.918 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:14.918 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:10.763198+0000", 2024-09-21T15:00:14.918 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.27e-05 2024-09-21T15:00:14.918 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.918 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.918 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:14.918 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:10.763303+0000", 2024-09-21T15:00:14.919 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000105451 2024-09-21T15:00:14.919 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.919 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.919 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:14.919 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:10.763557+0000", 2024-09-21T15:00:14.919 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00025405799999999999 2024-09-21T15:00:14.919 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.919 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.919 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T15:00:14.919 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:10.772841+0000", 2024-09-21T15:00:14.919 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0092834259999999991 2024-09-21T15:00:14.919 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.919 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.919 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:14.920 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:10.785078+0000", 2024-09-21T15:00:14.920 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012237572 2024-09-21T15:00:14.920 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.920 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.920 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:14.920 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:10.792383+0000", 2024-09-21T15:00:14.920 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0073048729999999999 2024-09-21T15:00:14.920 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.920 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.920 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:14.920 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:10.792423+0000", 2024-09-21T15:00:14.920 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0462e-05 2024-09-21T15:00:14.920 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:14.920 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:14.920 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:14.921 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:10.792452+0000", 2024-09-21T15:00:14.921 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8581999999999998e-05 2024-09-21T15:00:14.921 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:14.921 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:14.921 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:14.921 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:14.921 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:14.921 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:14.922 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T15:00:14.923 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:00:14.920+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:00:15.023 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:00:15.018+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:00:15.034 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:15.035 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:00:15.035 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T15:00:15.035 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:15.044 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T15:00:15.125 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:00:15.124+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:00:15.197 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:15.197 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:00:15.197 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T15:00:15.197 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T15:00:15.197 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:15.207 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T15:00:15.225 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:00:15.224+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:00:15.302 INFO:teuthology.orchestra.run.smithi032.stdout:checking clone #24, image image_client.0-clone24 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent25 2024-09-21T15:00:15.312 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:15.312 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T15:00:15.312 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T15:00:15.312 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T15:00:15.312 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.313 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45839 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:15.313 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159313+0000", 2024-09-21T15:00:15.313 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 65.151262617, 2024-09-21T15:00:15.313 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068238699, 2024-09-21T15:00:15.313 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:15.313 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:15.313 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:15.313 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:15.313 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:15.313 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45839 2024-09-21T15:00:15.313 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.313 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:15.313 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.313 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:15.314 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159313+0000", 2024-09-21T15:00:15.314 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:15.314 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.314 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.314 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:15.314 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159313+0000", 2024-09-21T15:00:15.314 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:15.314 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.314 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.314 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:15.314 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159313+0000", 2024-09-21T15:00:15.314 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:15.314 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.315 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.315 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:15.315 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159316+0000", 2024-09-21T15:00:15.315 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.033e-06 2024-09-21T15:00:15.315 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.315 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.315 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:15.315 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159322+0000", 2024-09-21T15:00:15.315 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1820000000000003e-06 2024-09-21T15:00:15.315 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.315 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.315 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:15.315 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159327+0000", 2024-09-21T15:00:15.315 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.062e-06 2024-09-21T15:00:15.316 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.316 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.316 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:15.316 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227544+0000", 2024-09-21T15:00:15.316 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068217142999999994 2024-09-21T15:00:15.316 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.316 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.316 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:15.316 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227552+0000", 2024-09-21T15:00:15.316 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6260000000000003e-06 2024-09-21T15:00:15.316 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:15.316 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:15.316 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:15.316 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.316 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.317 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45840 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:15.317 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159332+0000", 2024-09-21T15:00:15.317 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 65.151244320000004, 2024-09-21T15:00:15.317 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068262046000000007, 2024-09-21T15:00:15.317 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:15.317 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:15.317 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:15.317 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:15.317 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:15.317 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45840 2024-09-21T15:00:15.317 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.317 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:15.317 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.317 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:15.317 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159332+0000", 2024-09-21T15:00:15.318 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:15.318 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.318 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.318 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:15.318 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159332+0000", 2024-09-21T15:00:15.318 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:15.318 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.318 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.318 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:15.318 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159331+0000", 2024-09-21T15:00:15.318 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:15.318 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.318 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.318 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:15.318 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159351+0000", 2024-09-21T15:00:15.319 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0064999999999998e-05 2024-09-21T15:00:15.319 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.319 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.319 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:15.319 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159367+0000", 2024-09-21T15:00:15.319 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5308e-05 2024-09-21T15:00:15.319 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.319 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.319 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:15.319 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159373+0000", 2024-09-21T15:00:15.319 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1840000000000004e-06 2024-09-21T15:00:15.319 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.319 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.319 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:15.319 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227584+0000", 2024-09-21T15:00:15.320 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068211401000000005 2024-09-21T15:00:15.320 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.320 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.320 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:15.320 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227594+0000", 2024-09-21T15:00:15.320 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4120000000000007e-06 2024-09-21T15:00:15.320 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:15.320 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:15.320 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:15.320 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.320 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.320 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45841 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:15.320 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159379+0000", 2024-09-21T15:00:15.320 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 65.151197163999996, 2024-09-21T15:00:15.320 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068249039999999997, 2024-09-21T15:00:15.321 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:15.321 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:15.321 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:15.321 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:15.321 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:15.321 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45841 2024-09-21T15:00:15.321 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.321 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:15.321 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.321 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:15.321 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159379+0000", 2024-09-21T15:00:15.321 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:15.321 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.321 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.322 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:15.322 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159379+0000", 2024-09-21T15:00:15.322 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:15.322 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.322 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.322 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:15.322 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159378+0000", 2024-09-21T15:00:15.322 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:15.322 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.322 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.322 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:15.322 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159382+0000", 2024-09-21T15:00:15.322 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.641e-06 2024-09-21T15:00:15.322 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.322 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.323 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:15.323 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159421+0000", 2024-09-21T15:00:15.323 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9439e-05 2024-09-21T15:00:15.323 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.323 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.323 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:15.323 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159430+0000", 2024-09-21T15:00:15.323 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8429999999999999e-06 2024-09-21T15:00:15.323 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.323 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.323 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:15.323 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227617+0000", 2024-09-21T15:00:15.323 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068187084999999995 2024-09-21T15:00:15.323 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.324 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.324 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:15.324 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227628+0000", 2024-09-21T15:00:15.324 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0507e-05 2024-09-21T15:00:15.324 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:15.324 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:15.324 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:15.324 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.324 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.324 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45844 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:15.324 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159499+0000", 2024-09-21T15:00:15.324 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 65.151076778000004, 2024-09-21T15:00:15.324 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068255415, 2024-09-21T15:00:15.324 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:15.324 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:15.325 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:15.325 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:15.325 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:15.325 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45844 2024-09-21T15:00:15.325 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.325 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:15.325 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.325 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:15.325 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159499+0000", 2024-09-21T15:00:15.325 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:15.325 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.325 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.325 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:15.325 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159499+0000", 2024-09-21T15:00:15.325 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:15.325 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.326 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.326 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:15.326 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159499+0000", 2024-09-21T15:00:15.326 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:15.326 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.326 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.326 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:15.326 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159503+0000", 2024-09-21T15:00:15.326 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0999999999999997e-06 2024-09-21T15:00:15.326 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.326 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.326 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:15.326 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159512+0000", 2024-09-21T15:00:15.326 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5850000000000006e-06 2024-09-21T15:00:15.327 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.327 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.327 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:15.327 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159518+0000", 2024-09-21T15:00:15.327 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0909999999999998e-06 2024-09-21T15:00:15.327 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.327 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.327 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:15.327 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227743+0000", 2024-09-21T15:00:15.327 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068224536000000002 2024-09-21T15:00:15.327 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.327 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.327 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:15.328 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227755+0000", 2024-09-21T15:00:15.328 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1659e-05 2024-09-21T15:00:15.328 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:15.328 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:15.328 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:15.328 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.328 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.328 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45845 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:15.328 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159524+0000", 2024-09-21T15:00:15.328 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 65.151052418000006, 2024-09-21T15:00:15.328 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068267194000000003, 2024-09-21T15:00:15.328 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:15.328 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:15.328 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:15.328 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:15.329 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:15.329 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45845 2024-09-21T15:00:15.329 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.329 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:15.329 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.329 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:15.329 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159524+0000", 2024-09-21T15:00:15.329 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:15.329 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.329 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.329 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:15.329 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159524+0000", 2024-09-21T15:00:15.329 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:15.329 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.329 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.330 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:15.330 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159523+0000", 2024-09-21T15:00:15.330 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:15.330 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.330 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.330 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:15.330 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159526+0000", 2024-09-21T15:00:15.330 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1980000000000001e-06 2024-09-21T15:00:15.330 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.330 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.330 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:15.330 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159533+0000", 2024-09-21T15:00:15.330 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.348e-06 2024-09-21T15:00:15.330 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.330 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.331 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:15.331 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159546+0000", 2024-09-21T15:00:15.331 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3464e-05 2024-09-21T15:00:15.331 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.331 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.331 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:15.331 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227783+0000", 2024-09-21T15:00:15.331 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068236926000000003 2024-09-21T15:00:15.331 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.331 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.331 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:15.331 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227791+0000", 2024-09-21T15:00:15.331 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6669999999999996e-06 2024-09-21T15:00:15.331 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:15.332 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:15.332 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:15.332 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.332 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.332 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45846 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:15.332 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159584+0000", 2024-09-21T15:00:15.332 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 65.150991775999998, 2024-09-21T15:00:15.332 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068254767999999993, 2024-09-21T15:00:15.332 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:15.332 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:15.332 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:15.332 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:15.332 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:15.332 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45846 2024-09-21T15:00:15.332 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.333 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:15.333 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.333 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:15.333 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159584+0000", 2024-09-21T15:00:15.333 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:15.333 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.333 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.333 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:15.333 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159584+0000", 2024-09-21T15:00:15.333 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:15.333 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.333 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.333 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:15.333 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159583+0000", 2024-09-21T15:00:15.333 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:15.334 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.334 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.334 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:15.334 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159588+0000", 2024-09-21T15:00:15.334 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1330000000000001e-06 2024-09-21T15:00:15.334 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.334 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.334 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:15.334 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159596+0000", 2024-09-21T15:00:15.334 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5469999999999997e-06 2024-09-21T15:00:15.334 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.334 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.334 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:15.334 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159609+0000", 2024-09-21T15:00:15.335 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2785999999999999e-05 2024-09-21T15:00:15.335 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.335 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.335 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:15.335 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227830+0000", 2024-09-21T15:00:15.335 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068220879999999998 2024-09-21T15:00:15.335 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.335 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.335 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:15.335 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227839+0000", 2024-09-21T15:00:15.335 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1239999999999994e-06 2024-09-21T15:00:15.335 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:15.335 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:15.335 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:15.335 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.336 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.336 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45847 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:15.336 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159614+0000", 2024-09-21T15:00:15.336 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 65.150961699999996, 2024-09-21T15:00:15.336 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068260639999999997, 2024-09-21T15:00:15.336 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:15.336 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:15.336 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:15.336 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:15.336 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:15.336 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45847 2024-09-21T15:00:15.336 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.336 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:15.336 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.336 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:15.336 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159614+0000", 2024-09-21T15:00:15.337 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:15.337 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.337 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.337 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:15.337 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159614+0000", 2024-09-21T15:00:15.337 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:15.337 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.337 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.337 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:15.337 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159614+0000", 2024-09-21T15:00:15.337 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:15.337 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.337 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.337 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:15.337 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159617+0000", 2024-09-21T15:00:15.338 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2909999999999999e-06 2024-09-21T15:00:15.338 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.338 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.338 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:15.338 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159623+0000", 2024-09-21T15:00:15.338 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.511e-06 2024-09-21T15:00:15.338 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.338 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.338 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:15.338 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159631+0000", 2024-09-21T15:00:15.338 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.571e-06 2024-09-21T15:00:15.338 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.338 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.338 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:15.339 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227866+0000", 2024-09-21T15:00:15.339 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068235136000000002 2024-09-21T15:00:15.339 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.339 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.339 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:15.339 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227875+0000", 2024-09-21T15:00:15.339 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7199999999999995e-06 2024-09-21T15:00:15.339 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:15.339 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:15.339 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:15.339 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.339 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.339 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45848 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:15.339 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159636+0000", 2024-09-21T15:00:15.339 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 65.150939919999999, 2024-09-21T15:00:15.340 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068267283999999998, 2024-09-21T15:00:15.340 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:15.340 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:15.340 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:15.340 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:15.340 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:15.340 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45848 2024-09-21T15:00:15.340 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.340 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:15.340 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.340 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:15.340 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159636+0000", 2024-09-21T15:00:15.340 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:15.340 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.341 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.341 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:15.341 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159636+0000", 2024-09-21T15:00:15.341 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:15.341 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.341 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.341 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:15.341 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159636+0000", 2024-09-21T15:00:15.341 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:15.341 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.341 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.341 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:15.341 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159638+0000", 2024-09-21T15:00:15.341 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6560000000000001e-06 2024-09-21T15:00:15.342 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.342 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.342 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:15.342 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159644+0000", 2024-09-21T15:00:15.342 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0839999999999997e-06 2024-09-21T15:00:15.342 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.342 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.342 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:15.342 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159649+0000", 2024-09-21T15:00:15.342 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7500000000000003e-06 2024-09-21T15:00:15.342 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.342 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.342 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:15.342 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227892+0000", 2024-09-21T15:00:15.343 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068243315999999998 2024-09-21T15:00:15.343 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.343 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.343 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:15.343 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227903+0000", 2024-09-21T15:00:15.343 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0835999999999999e-05 2024-09-21T15:00:15.343 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:15.343 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:15.343 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:15.343 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.343 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.343 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45849 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:15.343 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159667+0000", 2024-09-21T15:00:15.343 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 65.150909106, 2024-09-21T15:00:15.344 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068291051000000005, 2024-09-21T15:00:15.344 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:15.344 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:15.344 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:15.344 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:15.344 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:15.344 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45849 2024-09-21T15:00:15.344 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.344 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:15.344 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.344 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:15.344 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159667+0000", 2024-09-21T15:00:15.344 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:15.344 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.344 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.345 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:15.345 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159667+0000", 2024-09-21T15:00:15.345 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:15.345 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.345 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.345 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:15.345 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159666+0000", 2024-09-21T15:00:15.345 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:15.345 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.345 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.345 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:15.345 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159670+0000", 2024-09-21T15:00:15.345 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.427e-06 2024-09-21T15:00:15.345 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.345 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.346 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:15.346 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159678+0000", 2024-09-21T15:00:15.346 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9869999999999999e-06 2024-09-21T15:00:15.346 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.346 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.346 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:15.346 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159688+0000", 2024-09-21T15:00:15.346 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0631999999999999e-05 2024-09-21T15:00:15.346 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.346 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.346 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:15.346 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227950+0000", 2024-09-21T15:00:15.346 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068261798999999998 2024-09-21T15:00:15.346 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.347 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.347 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:15.347 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227958+0000", 2024-09-21T15:00:15.347 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7510000000000008e-06 2024-09-21T15:00:15.347 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:15.347 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:15.347 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:15.347 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.347 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.347 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45850 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:15.347 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159695+0000", 2024-09-21T15:00:15.347 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 65.150881394999999, 2024-09-21T15:00:15.347 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068302999000000003, 2024-09-21T15:00:15.347 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:15.347 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:15.348 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:15.348 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:15.348 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:15.348 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45850 2024-09-21T15:00:15.348 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.348 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:15.348 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.348 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:15.348 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159695+0000", 2024-09-21T15:00:15.348 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:15.348 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.348 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.348 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:15.348 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159695+0000", 2024-09-21T15:00:15.349 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:15.349 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.349 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.349 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:15.349 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159694+0000", 2024-09-21T15:00:15.349 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:15.349 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.349 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.349 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:15.349 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159697+0000", 2024-09-21T15:00:15.349 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.258e-06 2024-09-21T15:00:15.349 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.349 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.349 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:15.350 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159704+0000", 2024-09-21T15:00:15.350 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7660000000000001e-06 2024-09-21T15:00:15.350 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.350 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.350 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:15.350 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159713+0000", 2024-09-21T15:00:15.350 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1779999999999992e-06 2024-09-21T15:00:15.350 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.350 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.350 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:15.350 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227989+0000", 2024-09-21T15:00:15.350 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068276015999999995 2024-09-21T15:00:15.350 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.351 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.351 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:15.351 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227998+0000", 2024-09-21T15:00:15.351 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2050000000000002e-06 2024-09-21T15:00:15.351 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:15.351 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:15.351 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:15.351 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.351 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.351 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45851 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:15.351 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159719+0000", 2024-09-21T15:00:15.351 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 65.150857262000002, 2024-09-21T15:00:15.351 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068320327, 2024-09-21T15:00:15.352 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:15.352 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:15.352 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:15.352 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:15.352 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:15.352 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45851 2024-09-21T15:00:15.352 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.352 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:15.352 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.352 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:15.352 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159719+0000", 2024-09-21T15:00:15.352 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:15.352 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.352 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.353 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:15.353 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159719+0000", 2024-09-21T15:00:15.353 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:15.353 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.353 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.353 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:15.353 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159718+0000", 2024-09-21T15:00:15.353 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:15.353 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.353 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.353 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:15.353 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159721+0000", 2024-09-21T15:00:15.353 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2169999999999999e-06 2024-09-21T15:00:15.353 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.353 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.354 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:15.354 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159730+0000", 2024-09-21T15:00:15.354 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5720000000000003e-06 2024-09-21T15:00:15.354 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.354 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.354 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:15.354 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159735+0000", 2024-09-21T15:00:15.354 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1100000000000002e-06 2024-09-21T15:00:15.354 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.354 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.354 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:15.354 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228029+0000", 2024-09-21T15:00:15.354 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068293624999999997 2024-09-21T15:00:15.354 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.355 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.355 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:15.355 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228039+0000", 2024-09-21T15:00:15.355 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0254e-05 2024-09-21T15:00:15.355 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:15.355 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:15.355 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:15.355 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.355 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.355 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45852 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:15.355 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159740+0000", 2024-09-21T15:00:15.355 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 65.150836093999999, 2024-09-21T15:00:15.355 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068340260999999999, 2024-09-21T15:00:15.355 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:15.355 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:15.356 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:15.356 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:15.356 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:15.356 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45852 2024-09-21T15:00:15.356 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.356 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:15.356 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.356 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:15.356 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159740+0000", 2024-09-21T15:00:15.356 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:15.356 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.356 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.356 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:15.356 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159740+0000", 2024-09-21T15:00:15.356 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:15.357 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.357 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.357 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:15.357 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159739+0000", 2024-09-21T15:00:15.357 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:15.357 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.357 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.357 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:15.357 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159757+0000", 2024-09-21T15:00:15.357 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7314999999999999e-05 2024-09-21T15:00:15.357 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.357 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.357 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:15.357 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159769+0000", 2024-09-21T15:00:15.358 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2376e-05 2024-09-21T15:00:15.358 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.358 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.358 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:15.358 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159775+0000", 2024-09-21T15:00:15.358 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2179999999999996e-06 2024-09-21T15:00:15.358 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.358 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.358 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:15.358 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228071+0000", 2024-09-21T15:00:15.358 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068295344999999993 2024-09-21T15:00:15.358 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.358 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.358 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:15.358 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228080+0000", 2024-09-21T15:00:15.359 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3959999999999995e-06 2024-09-21T15:00:15.359 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:15.359 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:15.359 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:15.359 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.359 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.359 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45853 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:15.359 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159781+0000", 2024-09-21T15:00:15.359 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 65.150794829999995, 2024-09-21T15:00:15.359 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068337093000000002, 2024-09-21T15:00:15.359 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:15.359 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:15.359 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:15.360 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:15.360 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:15.360 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45853 2024-09-21T15:00:15.360 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.360 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:15.360 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.360 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:15.360 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159781+0000", 2024-09-21T15:00:15.360 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:15.360 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.360 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.360 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:15.360 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159781+0000", 2024-09-21T15:00:15.360 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:15.360 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.361 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.361 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:15.361 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159781+0000", 2024-09-21T15:00:15.361 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:15.361 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.361 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.361 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:15.361 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159784+0000", 2024-09-21T15:00:15.361 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4759999999999998e-06 2024-09-21T15:00:15.361 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.361 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.361 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:15.361 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159792+0000", 2024-09-21T15:00:15.361 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0569999999999993e-06 2024-09-21T15:00:15.361 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.362 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.362 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:15.362 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159797+0000", 2024-09-21T15:00:15.362 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9640000000000002e-06 2024-09-21T15:00:15.362 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.362 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.362 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:15.362 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228111+0000", 2024-09-21T15:00:15.362 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068313737999999999 2024-09-21T15:00:15.362 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.362 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.362 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:15.362 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228118+0000", 2024-09-21T15:00:15.362 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.43e-06 2024-09-21T15:00:15.363 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:15.363 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:15.363 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:15.363 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.363 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.363 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45854 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:15.363 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159802+0000", 2024-09-21T15:00:15.363 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 65.150774179999999, 2024-09-21T15:00:15.363 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068358938999999994, 2024-09-21T15:00:15.363 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:15.363 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:15.363 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:15.363 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:15.363 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:15.363 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45854 2024-09-21T15:00:15.363 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.364 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:15.364 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.364 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:15.364 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159802+0000", 2024-09-21T15:00:15.364 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:15.364 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.364 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.364 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:15.364 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159802+0000", 2024-09-21T15:00:15.364 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:15.364 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.364 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.364 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:15.364 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159801+0000", 2024-09-21T15:00:15.365 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:15.365 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.365 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.365 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:15.365 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159804+0000", 2024-09-21T15:00:15.365 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0970000000000002e-06 2024-09-21T15:00:15.365 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.365 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.365 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:15.365 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159823+0000", 2024-09-21T15:00:15.365 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8895000000000001e-05 2024-09-21T15:00:15.365 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.365 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.365 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:15.366 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159837+0000", 2024-09-21T15:00:15.366 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3247999999999999e-05 2024-09-21T15:00:15.366 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.366 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.366 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:15.366 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228151+0000", 2024-09-21T15:00:15.366 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068314642999999994 2024-09-21T15:00:15.366 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.366 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.366 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:15.366 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228161+0000", 2024-09-21T15:00:15.366 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4250000000000004e-06 2024-09-21T15:00:15.366 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:15.366 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:15.366 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:15.366 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.367 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.367 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45983 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:15.367 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159848+0000", 2024-09-21T15:00:15.367 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 65.150727920999998, 2024-09-21T15:00:15.367 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068333801, 2024-09-21T15:00:15.367 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:15.367 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:15.367 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:15.367 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:15.367 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:15.367 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45983 2024-09-21T15:00:15.367 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.367 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:15.367 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.368 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:15.368 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159848+0000", 2024-09-21T15:00:15.368 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:15.368 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.368 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.368 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:15.368 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159848+0000", 2024-09-21T15:00:15.368 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:15.368 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.368 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.368 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:15.368 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159847+0000", 2024-09-21T15:00:15.368 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:15.368 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.368 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.369 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:15.369 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159856+0000", 2024-09-21T15:00:15.369 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4349999999999997e-06 2024-09-21T15:00:15.369 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.369 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.369 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:15.369 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159866+0000", 2024-09-21T15:00:15.369 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0076e-05 2024-09-21T15:00:15.369 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.369 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.369 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:15.369 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159872+0000", 2024-09-21T15:00:15.369 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8309999999999997e-06 2024-09-21T15:00:15.369 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.369 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.369 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:15.370 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228175+0000", 2024-09-21T15:00:15.370 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068302680000000005 2024-09-21T15:00:15.370 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.370 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.370 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:15.370 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228182+0000", 2024-09-21T15:00:15.370 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3150000000000003e-06 2024-09-21T15:00:15.370 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:15.370 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:15.370 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:15.370 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.370 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.370 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45984 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:15.370 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159877+0000", 2024-09-21T15:00:15.370 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 65.150698500999994, 2024-09-21T15:00:15.371 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068328448, 2024-09-21T15:00:15.371 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:15.371 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:15.371 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:15.371 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:15.371 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:15.371 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45984 2024-09-21T15:00:15.371 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.371 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:15.371 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.371 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:15.371 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159877+0000", 2024-09-21T15:00:15.371 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:15.371 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.371 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.372 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:15.372 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159877+0000", 2024-09-21T15:00:15.372 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:15.372 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.372 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.372 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:15.372 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159877+0000", 2024-09-21T15:00:15.372 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:15.372 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.372 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.372 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:15.372 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159880+0000", 2024-09-21T15:00:15.372 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7610000000000001e-06 2024-09-21T15:00:15.372 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.372 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.373 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:15.373 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159888+0000", 2024-09-21T15:00:15.373 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0330000000000003e-06 2024-09-21T15:00:15.373 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.373 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.373 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:15.373 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159893+0000", 2024-09-21T15:00:15.373 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3550000000000004e-06 2024-09-21T15:00:15.373 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.373 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.373 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:15.373 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228199+0000", 2024-09-21T15:00:15.373 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068306438999999997 2024-09-21T15:00:15.373 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.373 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.374 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:15.374 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228206+0000", 2024-09-21T15:00:15.374 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5599999999999999e-06 2024-09-21T15:00:15.374 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:15.374 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:15.374 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:15.374 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.374 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.374 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45985 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:15.374 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159897+0000", 2024-09-21T15:00:15.374 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 65.150678678000006, 2024-09-21T15:00:15.374 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068345664, 2024-09-21T15:00:15.374 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:15.374 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:15.375 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:15.375 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:15.375 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:15.375 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45985 2024-09-21T15:00:15.375 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.375 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:15.375 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.375 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:15.375 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159897+0000", 2024-09-21T15:00:15.375 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:15.375 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.375 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.375 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:15.376 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159897+0000", 2024-09-21T15:00:15.376 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:15.376 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.376 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.376 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:15.376 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159897+0000", 2024-09-21T15:00:15.376 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:15.376 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.376 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.376 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:15.376 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159900+0000", 2024-09-21T15:00:15.376 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.954e-06 2024-09-21T15:00:15.376 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.376 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.377 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:15.377 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159913+0000", 2024-09-21T15:00:15.377 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3106e-05 2024-09-21T15:00:15.377 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.377 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.377 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:15.377 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159918+0000", 2024-09-21T15:00:15.377 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5199999999999997e-06 2024-09-21T15:00:15.377 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.377 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.377 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:15.377 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228236+0000", 2024-09-21T15:00:15.377 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068317040999999995 2024-09-21T15:00:15.377 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.377 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.377 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:15.378 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228243+0000", 2024-09-21T15:00:15.378 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3880000000000002e-06 2024-09-21T15:00:15.378 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:15.378 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:15.378 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:15.378 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.378 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.378 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45986 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:15.378 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159924+0000", 2024-09-21T15:00:15.378 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 65.150652382999994, 2024-09-21T15:00:15.378 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068364362999999997, 2024-09-21T15:00:15.378 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:15.378 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:15.378 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:15.379 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:15.379 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:15.379 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45986 2024-09-21T15:00:15.379 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.379 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:15.379 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.379 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:15.379 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159924+0000", 2024-09-21T15:00:15.379 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:15.379 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.379 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.379 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:15.379 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159924+0000", 2024-09-21T15:00:15.379 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:15.379 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.379 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.380 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:15.380 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159923+0000", 2024-09-21T15:00:15.380 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:15.380 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.380 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.380 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:15.380 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159926+0000", 2024-09-21T15:00:15.380 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0510000000000001e-06 2024-09-21T15:00:15.380 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.380 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.380 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:15.380 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159932+0000", 2024-09-21T15:00:15.380 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4160000000000001e-06 2024-09-21T15:00:15.380 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.380 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.380 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:15.381 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159938+0000", 2024-09-21T15:00:15.381 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.383e-06 2024-09-21T15:00:15.381 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.381 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.381 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:15.381 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228275+0000", 2024-09-21T15:00:15.381 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068337007000000005 2024-09-21T15:00:15.381 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.381 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.381 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:15.381 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228288+0000", 2024-09-21T15:00:15.381 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3052e-05 2024-09-21T15:00:15.381 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:15.381 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:15.382 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:15.382 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.382 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.382 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45987 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:15.382 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159944+0000", 2024-09-21T15:00:15.382 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 65.150632158999997, 2024-09-21T15:00:15.382 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068387232000000006, 2024-09-21T15:00:15.382 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:15.382 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:15.382 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:15.382 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:15.382 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:15.382 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45987 2024-09-21T15:00:15.382 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.382 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:15.382 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.383 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:15.383 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159944+0000", 2024-09-21T15:00:15.383 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:15.383 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.383 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.383 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:15.383 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159944+0000", 2024-09-21T15:00:15.383 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:15.383 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.383 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.383 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:15.383 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159943+0000", 2024-09-21T15:00:15.383 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:15.383 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.384 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.384 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:15.384 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159946+0000", 2024-09-21T15:00:15.384 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6309999999999999e-06 2024-09-21T15:00:15.384 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.384 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.384 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:15.384 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159952+0000", 2024-09-21T15:00:15.384 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6540000000000004e-06 2024-09-21T15:00:15.384 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.384 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.384 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:15.384 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159957+0000", 2024-09-21T15:00:15.384 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9280000000000001e-06 2024-09-21T15:00:15.385 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.385 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.385 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:15.385 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228322+0000", 2024-09-21T15:00:15.385 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068365067000000002 2024-09-21T15:00:15.385 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.385 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.385 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:15.385 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228331+0000", 2024-09-21T15:00:15.385 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3270000000000007e-06 2024-09-21T15:00:15.385 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:15.385 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:15.385 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:15.385 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.385 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.386 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:97672 3.4 3:26046d29:::rbd_data.1136bfc79a70.0000000000000001:head [write 0~4718592 [fadvise_sequential+fadvise_nocache],write 5242880~131072 [fadvise_sequential+fadvise_nocache],write 5439488~147456 [fadvise_sequential+fadvise_nocache],write 5652480~229376 [fadvise_sequential+fadvise_nocache],write 5947392~147456 [fadvise_sequential+fadvise_nocache],write 6160384~229376 [fadvise_sequential+fadvise_nocache],write 6455296~147456 [fadvise_sequential+fadvise_nocache],write 6668288~229376 [fadvise_sequential+fadvise_nocache],write 6963200~147456 [fadvise_sequential+fadvise_nocache],write 7176192~147456 [fadvise_sequential+fadvise_nocache],write 7340032~65536 [fadvise_sequential+fadvise_nocache],write 7471104~147456 [fadvise_sequential+fadvise_nocache],write 7684096~147456 [fadvise_sequential+fadvise_nocache],write 7864320~49152 [fadvise_sequential+fadvise_nocache],write 7979008~147456 [fadvise_sequential+fadvise_nocache],write 8192000~147456 [fadvise_sequential+fadvise_nocache],write 8388608~32768 [fadvise_sequential+fadvise_nocache],write 8519680~114688 [fadvise_sequential+fadvise_nocache],write 8732672~114688 [fadvise_sequential+fadvise_nocache],write 8912896~16384 [fadvise_sequential+fadvise_nocache],write 9109504~32768 [fadvise_sequential+fadvise_nocache],write 9322496~32768 [fadvise_sequential+fadvise_nocache],write 9617408~32768 [fadvise_sequential+fadvise_nocache],write 9830400~32768 [fadvise_sequential+fadvise_nocache],write 9961472~98304 [fadvise_sequential+fadvise_nocache],write 10076160~196608 [fadvise_sequential+fadvise_nocache],write 10289152~278528 [fadvise_sequential+fadvise_nocache],write 10584064~196608 [fadvise_sequential+fadvise_nocache],write 10797056~196608 [fadvise_sequential+fadvise_nocache],write 11010048~65536 [fadvise_sequential+fadvise_nocache],write 11091968~196608 [fadvise_sequential+fadvise_nocache],write 11304960~196608 [fadvise_sequential+fadvise_nocache],write 11517952~65536 [fadvise_sequential+fadvise_nocache],write 11616256~98304 [fadvise_sequential+fadvise_nocache],write 11730944~65536 [fadvise_sequential+fadvise_nocache],write 11829248~98304 [fadvise_sequential+fadvise_nocache],write 11943936~65536 [fadvise_sequential+fadvise_nocache],write 12042240~49152 [fadvise_sequential+fadvise_nocache],write 12124160~98304 [fadvise_sequential+fadvise_nocache],write 12238848~65536 [fadvise_sequential+fadvise_nocache],write 12337152~98304 [fadvise_sequential+fadvise_nocache],write 12451840~65536 [fadvise_sequential+fadvise_nocache],write 12550144~32768 [fadvise_sequential+fadvise_nocache],write 12632064~98304 [fadvise_sequential+fadvise_nocache],write 12845056~98304 [fadvise_sequential+fadvise_nocache],write 13058048~49152 [fadvise_sequential+fadvise_nocache],write 13139968~98304 [fadvise_sequential+fadvise_nocache],write 13352960~98304 [fadvise_sequential+fadvise_nocache],write 13565952~65536 [fadvise_sequential+fadvise_nocache],write 13647872~98304 [fadvise_sequential+fadvise_nocache],write 13860864~98304 [fadvise_sequential+fadvise_nocache],write 14073856~81920 [fadvise_sequential+fadvise_nocache],write 15269888~81920 [fadvise_sequential+fadvise_nocache],write 15482880~81920 [fadvise_sequential+fadvise_nocache],write 15695872~32768 [fadvise_sequential+fadvise_nocache],write 15777792~81920 [fadvise_sequential+fadvise_nocache],write 15990784~81920 [fadvise_sequential+fadvise_nocache],write 16203776~49152 [fadvise_sequential+fadvise_nocache],write 16285696~81920 [fadvise_sequential+fadvise_nocache],write 16498688~81920 [fadvise_sequential+fadvise_nocache],write 16711680~65536 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:15.386 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.298669+0000", 2024-09-21T15:00:15.386 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 10.011907093, 2024-09-21T15:00:15.386 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.10282358900000001, 2024-09-21T15:00:15.386 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:15.386 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:15.386 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:15.386 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:15.386 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:15.386 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97672 2024-09-21T15:00:15.386 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.386 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:15.386 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.386 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:15.387 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.298669+0000", 2024-09-21T15:00:15.387 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:15.387 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.387 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.387 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:15.387 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.298669+0000", 2024-09-21T15:00:15.387 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:15.387 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.387 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.387 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:15.387 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.298667+0000", 2024-09-21T15:00:15.387 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:00:15.387 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.387 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.387 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:15.387 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.327458+0000", 2024-09-21T15:00:15.388 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.028790754000000002 2024-09-21T15:00:15.388 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.388 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.388 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:15.388 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.327492+0000", 2024-09-21T15:00:15.388 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3806999999999997e-05 2024-09-21T15:00:15.388 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.388 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.388 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:15.388 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.327507+0000", 2024-09-21T15:00:15.388 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5112e-05 2024-09-21T15:00:15.388 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.388 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.388 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:15.389 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.327597+0000", 2024-09-21T15:00:15.389 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9938999999999994e-05 2024-09-21T15:00:15.389 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.389 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.389 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:15.389 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.327927+0000", 2024-09-21T15:00:15.389 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000329766 2024-09-21T15:00:15.389 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.389 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.389 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T15:00:15.389 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.330735+0000", 2024-09-21T15:00:15.389 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0028086869999999998 2024-09-21T15:00:15.389 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.389 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.390 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:15.390 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.378299+0000", 2024-09-21T15:00:15.390 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.047563495999999997 2024-09-21T15:00:15.390 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.390 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.390 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:15.390 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.401248+0000", 2024-09-21T15:00:15.390 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.022949311 2024-09-21T15:00:15.390 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.390 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.390 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:15.390 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.401446+0000", 2024-09-21T15:00:15.390 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019831199999999999 2024-09-21T15:00:15.390 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.390 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.391 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:15.391 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.401492+0000", 2024-09-21T15:00:15.391 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6109000000000002e-05 2024-09-21T15:00:15.391 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:15.391 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:15.391 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:15.391 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:15.391 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:15.391 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:15.392 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T15:00:15.393 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:00:15.386+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:00:15.493 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:00:15.492+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:00:15.506 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:15.506 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:00:15.506 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T15:00:15.506 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:15.516 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T15:00:15.593 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:00:15.592+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:00:15.623 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:15.623 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:00:15.624 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T15:00:15.624 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T15:00:15.624 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:15.633 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T15:00:15.694 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:00:15.690+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:00:15.730 INFO:teuthology.orchestra.run.smithi032.stdout:3465 write 0x833f055 thru 0x8346ee8 (0x7e94 bytes) 2024-09-21T15:00:15.739 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:15.739 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T15:00:15.739 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T15:00:15.739 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T15:00:15.739 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.739 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:10651 3.12 3:4e9d23af:::rbd_data.1136f5ed5dd6.0000000000000020:head [write 0~81920 [fadvise_sequential+fadvise_nocache],write 90112~8192 [fadvise_sequential+fadvise_nocache],write 122880~122880 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:15.739 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.292476+0000", 2024-09-21T15:00:15.740 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 96.443468922999998, 2024-09-21T15:00:15.740 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012765669, 2024-09-21T15:00:15.740 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:15.740 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:15.740 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:15.740 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:15.740 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:15.740 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 10651 2024-09-21T15:00:15.740 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.740 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:15.740 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.740 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:15.740 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292476+0000", 2024-09-21T15:00:15.740 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:15.741 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.741 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.741 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:15.741 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292476+0000", 2024-09-21T15:00:15.741 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:15.741 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.741 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.741 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:15.741 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292475+0000", 2024-09-21T15:00:15.741 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:00:15.741 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.741 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.741 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:15.741 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292696+0000", 2024-09-21T15:00:15.742 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000220201 2024-09-21T15:00:15.742 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.742 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.742 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:15.742 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292711+0000", 2024-09-21T15:00:15.742 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5803000000000001e-05 2024-09-21T15:00:15.742 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.742 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.742 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:15.742 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292721+0000", 2024-09-21T15:00:15.742 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6160000000000008e-06 2024-09-21T15:00:15.742 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.742 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.742 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:15.742 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292757+0000", 2024-09-21T15:00:15.743 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6211999999999998e-05 2024-09-21T15:00:15.743 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.743 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.743 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:15.743 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292878+0000", 2024-09-21T15:00:15.743 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000120306 2024-09-21T15:00:15.743 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.743 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.743 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 3", 2024-09-21T15:00:15.743 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.293019+0000", 2024-09-21T15:00:15.743 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014186400000000001 2024-09-21T15:00:15.743 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.743 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.743 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:15.743 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.294898+0000", 2024-09-21T15:00:15.744 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001878434 2024-09-21T15:00:15.744 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.744 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.744 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:15.744 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.305162+0000", 2024-09-21T15:00:15.744 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.01026373 2024-09-21T15:00:15.744 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.744 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.744 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:15.744 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.305209+0000", 2024-09-21T15:00:15.744 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7615999999999999e-05 2024-09-21T15:00:15.744 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.744 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.744 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:15.744 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.305242+0000", 2024-09-21T15:00:15.745 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2979999999999999e-05 2024-09-21T15:00:15.745 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:15.745 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:15.745 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:15.745 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.745 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.745 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12193 3.4 3:25437d51:::rbd_data.11363a21d2be.0000000000000025:10 [read 425984~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:15.745 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.618755+0000", 2024-09-21T15:00:15.745 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 96.117190640000004, 2024-09-21T15:00:15.745 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086461379999999994, 2024-09-21T15:00:15.745 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:15.745 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:15.745 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:15.746 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:15.746 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:15.746 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12193 2024-09-21T15:00:15.746 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.746 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:15.746 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.746 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:15.746 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618755+0000", 2024-09-21T15:00:15.746 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:15.746 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.746 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.746 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:15.746 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618755+0000", 2024-09-21T15:00:15.746 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:15.747 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.747 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.747 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:15.747 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618754+0000", 2024-09-21T15:00:15.747 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:15.747 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.747 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.747 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:15.747 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618759+0000", 2024-09-21T15:00:15.747 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4020000000000004e-06 2024-09-21T15:00:15.747 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.747 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.747 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:15.747 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618767+0000", 2024-09-21T15:00:15.747 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7490000000000004e-06 2024-09-21T15:00:15.748 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.748 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.748 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:15.748 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.618773+0000", 2024-09-21T15:00:15.748 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0939999999999996e-06 2024-09-21T15:00:15.748 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.748 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.748 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:15.748 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.627115+0000", 2024-09-21T15:00:15.748 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0083416840000000003 2024-09-21T15:00:15.748 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.748 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.748 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:15.748 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.627214+0000", 2024-09-21T15:00:15.749 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9042000000000005e-05 2024-09-21T15:00:15.749 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.749 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.749 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:15.749 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.627401+0000", 2024-09-21T15:00:15.749 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018677200000000001 2024-09-21T15:00:15.749 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:15.749 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:15.749 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:15.749 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.749 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.749 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12214 3.4 3:21b43b45:::rbd_data.11363a21d2be.000000000000002b:10 [read 376832~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:15.749 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.619969+0000", 2024-09-21T15:00:15.749 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 96.115976372999995, 2024-09-21T15:00:15.749 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0090292089999999998, 2024-09-21T15:00:15.749 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:15.750 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:15.750 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:15.750 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:15.750 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:15.750 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12214 2024-09-21T15:00:15.750 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.750 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:15.750 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.750 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:15.750 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619969+0000", 2024-09-21T15:00:15.750 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:15.750 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.750 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.750 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:15.751 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619969+0000", 2024-09-21T15:00:15.751 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:15.751 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.751 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.751 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:15.751 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619969+0000", 2024-09-21T15:00:15.751 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:15.751 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.751 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.751 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:15.751 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619971+0000", 2024-09-21T15:00:15.751 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1830000000000001e-06 2024-09-21T15:00:15.751 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.752 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.752 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:15.752 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619976+0000", 2024-09-21T15:00:15.752 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7129999999999996e-06 2024-09-21T15:00:15.752 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.752 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.752 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:15.752 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619981+0000", 2024-09-21T15:00:15.752 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2460000000000003e-06 2024-09-21T15:00:15.752 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.752 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.752 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:15.752 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628602+0000", 2024-09-21T15:00:15.753 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086209519999999994 2024-09-21T15:00:15.753 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.753 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.753 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:15.753 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628734+0000", 2024-09-21T15:00:15.753 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000132242 2024-09-21T15:00:15.753 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.753 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.753 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:15.753 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628998+0000", 2024-09-21T15:00:15.753 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00026419799999999999 2024-09-21T15:00:15.753 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:15.753 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:15.754 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:15.754 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.754 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.754 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12215 3.4 3:21b43b45:::rbd_data.11363a21d2be.000000000000002b:10 [read 409600~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:15.754 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.619985+0000", 2024-09-21T15:00:15.754 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 96.115960306999995, 2024-09-21T15:00:15.754 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0095457319999999995, 2024-09-21T15:00:15.754 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:15.754 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:15.754 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:15.754 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:15.754 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:15.754 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12215 2024-09-21T15:00:15.755 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.755 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:15.755 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.755 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:15.755 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619985+0000", 2024-09-21T15:00:15.755 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:15.755 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.755 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.755 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:15.755 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619985+0000", 2024-09-21T15:00:15.755 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:15.755 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.755 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.755 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:15.755 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619985+0000", 2024-09-21T15:00:15.756 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:15.756 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.756 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.756 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:15.756 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619987+0000", 2024-09-21T15:00:15.756 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6129999999999998e-06 2024-09-21T15:00:15.756 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.756 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.756 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:15.756 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619993+0000", 2024-09-21T15:00:15.756 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0820000000000004e-06 2024-09-21T15:00:15.756 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.756 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.756 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:15.757 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619998+0000", 2024-09-21T15:00:15.757 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7759999999999997e-06 2024-09-21T15:00:15.757 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.757 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.757 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:15.757 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629022+0000", 2024-09-21T15:00:15.757 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0090237489999999993 2024-09-21T15:00:15.757 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.757 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.757 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:15.757 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629098+0000", 2024-09-21T15:00:15.757 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5735999999999995e-05 2024-09-21T15:00:15.757 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.757 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.757 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:15.758 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629531+0000", 2024-09-21T15:00:15.758 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00043310100000000003 2024-09-21T15:00:15.758 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:15.758 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:15.758 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:15.758 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.758 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.758 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12216 3.4 3:21b43b45:::rbd_data.11363a21d2be.000000000000002b:10 [read 516096~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:15.758 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.620003+0000", 2024-09-21T15:00:15.758 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 96.115942310999998, 2024-09-21T15:00:15.758 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0098850169999999994, 2024-09-21T15:00:15.758 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:15.758 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:15.758 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:15.759 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:15.759 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:15.759 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12216 2024-09-21T15:00:15.759 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.759 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:15.759 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.759 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:15.759 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620003+0000", 2024-09-21T15:00:15.759 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:15.759 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.759 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.759 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:15.759 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620003+0000", 2024-09-21T15:00:15.759 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:15.759 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.760 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.760 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:15.760 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620003+0000", 2024-09-21T15:00:15.760 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:15.760 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.760 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.760 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:15.760 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620006+0000", 2024-09-21T15:00:15.760 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4350000000000001e-06 2024-09-21T15:00:15.760 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.760 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.760 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:15.760 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620013+0000", 2024-09-21T15:00:15.760 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4219999999999997e-06 2024-09-21T15:00:15.760 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.761 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.761 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:15.761 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620017+0000", 2024-09-21T15:00:15.761 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.865e-06 2024-09-21T15:00:15.761 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.761 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.761 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:15.761 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629581+0000", 2024-09-21T15:00:15.761 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0095631989999999997 2024-09-21T15:00:15.761 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.761 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.761 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:15.761 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629691+0000", 2024-09-21T15:00:15.761 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000110586 2024-09-21T15:00:15.761 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.762 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.762 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:15.762 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629888+0000", 2024-09-21T15:00:15.762 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019691199999999999 2024-09-21T15:00:15.762 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:15.762 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:15.762 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:15.762 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.762 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.762 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12222 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 49152~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:15.762 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.620027+0000", 2024-09-21T15:00:15.762 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 96.115917964999994, 2024-09-21T15:00:15.762 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010228131999999999, 2024-09-21T15:00:15.762 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:15.763 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:15.763 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:15.763 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:15.763 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:15.763 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12222 2024-09-21T15:00:15.763 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.763 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:15.763 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.763 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:15.763 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620027+0000", 2024-09-21T15:00:15.763 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:15.763 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.763 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.763 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:15.763 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620027+0000", 2024-09-21T15:00:15.764 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:15.764 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.764 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.764 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:15.764 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620027+0000", 2024-09-21T15:00:15.764 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:15.764 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.764 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.764 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:15.764 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620030+0000", 2024-09-21T15:00:15.764 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0400000000000001e-06 2024-09-21T15:00:15.764 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.764 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.764 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:15.765 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620037+0000", 2024-09-21T15:00:15.765 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8220000000000001e-06 2024-09-21T15:00:15.765 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.765 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.765 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:15.765 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620042+0000", 2024-09-21T15:00:15.765 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.843e-06 2024-09-21T15:00:15.765 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.765 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.765 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:15.765 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629922+0000", 2024-09-21T15:00:15.765 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0098805100000000003 2024-09-21T15:00:15.765 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.765 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.766 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:15.766 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.630020+0000", 2024-09-21T15:00:15.766 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7596999999999994e-05 2024-09-21T15:00:15.766 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.766 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.766 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:15.766 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.630256+0000", 2024-09-21T15:00:15.766 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00023573299999999999 2024-09-21T15:00:15.766 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:15.766 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:15.766 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:15.766 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.766 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.766 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12260 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 57344~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:15.767 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.624587+0000", 2024-09-21T15:00:15.767 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 96.111358109999998, 2024-09-21T15:00:15.767 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0088282069999999994, 2024-09-21T15:00:15.767 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:15.767 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:15.767 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:15.767 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:15.767 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:15.767 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12260 2024-09-21T15:00:15.767 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.767 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:15.767 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.767 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:15.767 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624587+0000", 2024-09-21T15:00:15.768 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:15.768 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.768 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.768 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:15.768 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624587+0000", 2024-09-21T15:00:15.768 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:15.768 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.768 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.768 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:15.768 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624587+0000", 2024-09-21T15:00:15.768 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-21T15:00:15.768 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.768 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.769 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:15.769 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624589+0000", 2024-09-21T15:00:15.769 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4339999999999999e-06 2024-09-21T15:00:15.769 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.769 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.769 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:15.769 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624593+0000", 2024-09-21T15:00:15.769 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5859999999999998e-06 2024-09-21T15:00:15.769 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.769 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.769 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:15.769 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624596+0000", 2024-09-21T15:00:15.769 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5830000000000001e-06 2024-09-21T15:00:15.769 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.769 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.770 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:15.770 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.633132+0000", 2024-09-21T15:00:15.770 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.008536666 2024-09-21T15:00:15.770 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.770 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.770 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:15.770 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.633216+0000", 2024-09-21T15:00:15.770 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3387999999999995e-05 2024-09-21T15:00:15.770 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.770 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.770 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:15.770 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.633415+0000", 2024-09-21T15:00:15.770 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019976799999999999 2024-09-21T15:00:15.770 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:15.770 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:15.771 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:15.771 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.771 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.771 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12517 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 278528~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:15.771 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645214+0000", 2024-09-21T15:00:15.771 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 96.090731460000001, 2024-09-21T15:00:15.771 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086832899999999998, 2024-09-21T15:00:15.771 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:15.771 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:15.771 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:15.771 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:15.771 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:15.771 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12517 2024-09-21T15:00:15.771 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.772 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:15.772 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.772 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:15.772 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645214+0000", 2024-09-21T15:00:15.772 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:15.772 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.772 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.772 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:15.772 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645214+0000", 2024-09-21T15:00:15.772 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:15.772 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.772 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.772 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:15.772 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645214+0000", 2024-09-21T15:00:15.773 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:15.773 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.773 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.773 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:15.773 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645216+0000", 2024-09-21T15:00:15.773 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5579999999999999e-06 2024-09-21T15:00:15.773 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.773 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.773 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:15.773 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645224+0000", 2024-09-21T15:00:15.773 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2439999999999993e-06 2024-09-21T15:00:15.773 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.773 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.773 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:15.773 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645231+0000", 2024-09-21T15:00:15.774 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1859999999999997e-06 2024-09-21T15:00:15.774 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.774 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.774 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:15.774 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.653651+0000", 2024-09-21T15:00:15.774 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0084204220000000003 2024-09-21T15:00:15.774 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.774 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.774 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:15.774 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.653726+0000", 2024-09-21T15:00:15.774 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5356000000000002e-05 2024-09-21T15:00:15.774 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.774 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.774 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:15.775 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.653897+0000", 2024-09-21T15:00:15.775 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000170848 2024-09-21T15:00:15.775 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:15.775 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:15.775 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:15.775 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.775 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.775 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12518 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 385024~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:15.775 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645235+0000", 2024-09-21T15:00:15.775 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 96.090710233999999, 2024-09-21T15:00:15.775 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0089841020000000008, 2024-09-21T15:00:15.775 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:15.775 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:15.775 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:15.776 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:15.776 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:15.776 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12518 2024-09-21T15:00:15.776 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.776 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:15.776 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.776 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:15.776 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645235+0000", 2024-09-21T15:00:15.776 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:15.776 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.776 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.776 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:15.776 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645235+0000", 2024-09-21T15:00:15.776 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:15.777 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.777 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.777 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:15.777 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645235+0000", 2024-09-21T15:00:15.777 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:15.777 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.777 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.777 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:15.777 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645238+0000", 2024-09-21T15:00:15.777 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.869e-06 2024-09-21T15:00:15.777 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.777 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.777 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:15.778 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645244+0000", 2024-09-21T15:00:15.778 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8930000000000002e-06 2024-09-21T15:00:15.778 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.778 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.778 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:15.778 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645249+0000", 2024-09-21T15:00:15.778 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2859999999999999e-06 2024-09-21T15:00:15.778 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.778 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.778 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:15.778 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.653930+0000", 2024-09-21T15:00:15.778 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086815320000000005 2024-09-21T15:00:15.778 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.778 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.779 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:15.779 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654028+0000", 2024-09-21T15:00:15.779 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7578000000000007e-05 2024-09-21T15:00:15.779 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.779 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.779 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:15.779 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654219+0000", 2024-09-21T15:00:15.779 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019132099999999999 2024-09-21T15:00:15.779 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:15.779 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:15.779 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:15.779 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.779 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.779 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12538 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 147456~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:15.780 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645253+0000", 2024-09-21T15:00:15.780 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 96.090692610000005, 2024-09-21T15:00:15.780 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0092095360000000008, 2024-09-21T15:00:15.780 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:15.780 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:15.780 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:15.780 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:15.780 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:15.780 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12538 2024-09-21T15:00:15.780 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.780 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:15.780 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.780 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:15.781 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645253+0000", 2024-09-21T15:00:15.781 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:15.781 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.781 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.781 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:15.781 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645253+0000", 2024-09-21T15:00:15.781 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:15.781 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.781 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.781 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:15.781 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645252+0000", 2024-09-21T15:00:15.781 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:15.781 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.781 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.782 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:15.782 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645255+0000", 2024-09-21T15:00:15.782 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2290000000000002e-06 2024-09-21T15:00:15.782 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.782 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.782 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:15.782 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645267+0000", 2024-09-21T15:00:15.782 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2631e-05 2024-09-21T15:00:15.782 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.782 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.782 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:15.782 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645278+0000", 2024-09-21T15:00:15.782 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0847e-05 2024-09-21T15:00:15.783 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.783 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.783 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:15.783 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654251+0000", 2024-09-21T15:00:15.783 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0089731340000000007 2024-09-21T15:00:15.783 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.783 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.783 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:15.783 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654296+0000", 2024-09-21T15:00:15.783 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4353e-05 2024-09-21T15:00:15.783 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.783 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.783 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:15.783 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654462+0000", 2024-09-21T15:00:15.784 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016668499999999999 2024-09-21T15:00:15.784 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:15.784 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:15.784 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:15.784 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.784 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.784 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12539 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 253952~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:15.784 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645284+0000", 2024-09-21T15:00:15.784 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 96.090661389000005, 2024-09-21T15:00:15.784 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0094524029999999998, 2024-09-21T15:00:15.784 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:15.784 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:15.784 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:15.784 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:15.785 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:15.785 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12539 2024-09-21T15:00:15.785 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.785 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:15.785 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.785 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:15.785 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645284+0000", 2024-09-21T15:00:15.785 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:15.785 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.785 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.785 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:15.785 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645284+0000", 2024-09-21T15:00:15.785 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:15.786 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.786 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.786 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:15.786 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645284+0000", 2024-09-21T15:00:15.786 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:15.786 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.786 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.786 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:15.786 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645286+0000", 2024-09-21T15:00:15.786 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7159999999999999e-06 2024-09-21T15:00:15.786 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.786 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.786 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:15.786 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645292+0000", 2024-09-21T15:00:15.786 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7400000000000001e-06 2024-09-21T15:00:15.787 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.787 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.787 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:15.787 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645297+0000", 2024-09-21T15:00:15.787 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9019999999999998e-06 2024-09-21T15:00:15.787 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.787 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.787 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:15.787 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654490+0000", 2024-09-21T15:00:15.787 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0091932360000000005 2024-09-21T15:00:15.787 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.787 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.787 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:15.787 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654565+0000", 2024-09-21T15:00:15.787 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4657e-05 2024-09-21T15:00:15.788 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.788 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.788 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:15.788 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654736+0000", 2024-09-21T15:00:15.788 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017156199999999999 2024-09-21T15:00:15.788 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:15.788 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:15.788 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:15.788 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.788 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.788 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12540 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 286720~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:15.788 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645302+0000", 2024-09-21T15:00:15.788 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 96.090643119999996, 2024-09-21T15:00:15.788 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0096603090000000006, 2024-09-21T15:00:15.788 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:15.789 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:15.789 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:15.789 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:15.789 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:15.789 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12540 2024-09-21T15:00:15.789 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.789 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:15.789 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.789 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:15.789 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645302+0000", 2024-09-21T15:00:15.789 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:15.789 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.789 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.789 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:15.790 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645302+0000", 2024-09-21T15:00:15.790 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:15.790 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.790 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.790 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:15.790 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645302+0000", 2024-09-21T15:00:15.790 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:15.790 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.790 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.790 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:15.790 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645305+0000", 2024-09-21T15:00:15.790 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7329999999999999e-06 2024-09-21T15:00:15.790 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.791 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.791 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:15.791 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645316+0000", 2024-09-21T15:00:15.791 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1832e-05 2024-09-21T15:00:15.791 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.791 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.791 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:15.791 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645322+0000", 2024-09-21T15:00:15.791 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.169e-06 2024-09-21T15:00:15.791 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.791 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.791 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:15.791 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654778+0000", 2024-09-21T15:00:15.791 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0094559090000000002 2024-09-21T15:00:15.792 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.792 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.792 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:15.792 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654828+0000", 2024-09-21T15:00:15.792 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0461999999999999e-05 2024-09-21T15:00:15.792 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.792 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.792 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:15.792 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654963+0000", 2024-09-21T15:00:15.792 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000134583 2024-09-21T15:00:15.792 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:15.792 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:15.792 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:15.792 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.793 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.793 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12561 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:15.793 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645742+0000", 2024-09-21T15:00:15.793 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 96.090203853999995, 2024-09-21T15:00:15.793 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0093583810000000007, 2024-09-21T15:00:15.793 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:15.793 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:15.793 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:15.793 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:15.793 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:15.793 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12561 2024-09-21T15:00:15.793 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.793 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:15.793 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.794 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:15.794 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645742+0000", 2024-09-21T15:00:15.794 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:15.794 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.794 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.794 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:15.794 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645742+0000", 2024-09-21T15:00:15.794 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:15.794 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.794 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.794 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:15.794 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645741+0000", 2024-09-21T15:00:15.794 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:00:15.795 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.795 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.795 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:15.795 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645746+0000", 2024-09-21T15:00:15.795 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8379999999999998e-06 2024-09-21T15:00:15.795 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.795 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.795 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:15.795 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645757+0000", 2024-09-21T15:00:15.795 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0705e-05 2024-09-21T15:00:15.795 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.795 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.795 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:15.795 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645765+0000", 2024-09-21T15:00:15.796 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6729999999999992e-06 2024-09-21T15:00:15.796 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.796 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.796 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:15.796 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654993+0000", 2024-09-21T15:00:15.796 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0092287480000000002 2024-09-21T15:00:15.796 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.796 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.796 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:15.796 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.655050+0000", 2024-09-21T15:00:15.796 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6394999999999998e-05 2024-09-21T15:00:15.796 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.796 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.796 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:15.797 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.655100+0000", 2024-09-21T15:00:15.797 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0049000000000001e-05 2024-09-21T15:00:15.797 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:15.797 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:15.797 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:15.797 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.797 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.797 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:97619 3.1 e122/93 3:8730dd9c:::rbd_data.1136bfc79a70.0000000000000000:head v 122'167, mlcod=120'165)", 2024-09-21T15:00:15.797 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.292926+0000", 2024-09-21T15:00:15.797 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 10.443019786000001, 2024-09-21T15:00:15.797 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.020236437999999999, 2024-09-21T15:00:15.797 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:15.797 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:15.798 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:15.798 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.798 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:15.798 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.292926+0000", 2024-09-21T15:00:15.798 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:15.798 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.798 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.798 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:15.798 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.292926+0000", 2024-09-21T15:00:15.798 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:15.798 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.798 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.798 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:15.798 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.292925+0000", 2024-09-21T15:00:15.798 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:15.799 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.799 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.799 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:15.799 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.300394+0000", 2024-09-21T15:00:15.799 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0074683839999999998 2024-09-21T15:00:15.799 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.799 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.799 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:15.799 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.300420+0000", 2024-09-21T15:00:15.799 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6380000000000002e-05 2024-09-21T15:00:15.799 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.799 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.799 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:15.799 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.300432+0000", 2024-09-21T15:00:15.799 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1987999999999999e-05 2024-09-21T15:00:15.800 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.800 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.800 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:15.800 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.300533+0000", 2024-09-21T15:00:15.800 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000100883 2024-09-21T15:00:15.800 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.800 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.800 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:15.800 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.300562+0000", 2024-09-21T15:00:15.800 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9048999999999999e-05 2024-09-21T15:00:15.800 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.800 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.800 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:15.800 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.313121+0000", 2024-09-21T15:00:15.800 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012559038999999999 2024-09-21T15:00:15.800 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.801 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.801 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:15.801 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.313162+0000", 2024-09-21T15:00:15.801 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0998999999999998e-05 2024-09-21T15:00:15.801 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:15.801 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:15.801 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:15.801 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.801 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.801 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:97677 3.15 e122/24 3:a8e72133:::rbd_data.1136bfc79a70.0000000000000005:head v 122'567, mlcod=120'565)", 2024-09-21T15:00:15.801 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.320753+0000", 2024-09-21T15:00:15.801 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 10.415192198, 2024-09-21T15:00:15.801 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070639375000000004, 2024-09-21T15:00:15.801 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:15.802 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:15.802 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:15.802 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.802 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:15.802 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.320753+0000", 2024-09-21T15:00:15.802 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:15.802 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.802 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.802 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:15.802 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.320753+0000", 2024-09-21T15:00:15.802 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:15.802 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.802 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.802 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:15.802 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.320753+0000", 2024-09-21T15:00:15.803 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:15.803 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.803 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.803 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:15.803 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.375452+0000", 2024-09-21T15:00:15.803 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054698945999999998 2024-09-21T15:00:15.803 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.803 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.803 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:15.803 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.375476+0000", 2024-09-21T15:00:15.803 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4380999999999999e-05 2024-09-21T15:00:15.803 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.803 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.803 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:15.803 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.375485+0000", 2024-09-21T15:00:15.804 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0049999999999993e-06 2024-09-21T15:00:15.804 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.804 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.804 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:15.804 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.375552+0000", 2024-09-21T15:00:15.804 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6550999999999994e-05 2024-09-21T15:00:15.804 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.804 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.804 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:15.804 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.375578+0000", 2024-09-21T15:00:15.804 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6196000000000001e-05 2024-09-21T15:00:15.804 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.804 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.804 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:15.805 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.391319+0000", 2024-09-21T15:00:15.805 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015741271000000001 2024-09-21T15:00:15.805 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.805 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.805 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:15.805 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.391393+0000", 2024-09-21T15:00:15.805 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3287000000000005e-05 2024-09-21T15:00:15.805 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:15.805 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:15.805 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:15.805 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.805 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.805 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:97672 3.4 e122/51 3:26046d29:::rbd_data.1136bfc79a70.0000000000000001:head v 122'256, mlcod=120'254)", 2024-09-21T15:00:15.806 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.331696+0000", 2024-09-21T15:00:15.806 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 10.404249034999999, 2024-09-21T15:00:15.806 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041222507999999998, 2024-09-21T15:00:15.806 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:15.806 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:15.806 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:15.806 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.806 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:15.806 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.331696+0000", 2024-09-21T15:00:15.806 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:15.806 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.806 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.806 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:15.806 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.331696+0000", 2024-09-21T15:00:15.806 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:15.807 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.807 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.807 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:15.807 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.331695+0000", 2024-09-21T15:00:15.807 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:00:15.807 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.807 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.807 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:15.807 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.340175+0000", 2024-09-21T15:00:15.807 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0084799520000000007 2024-09-21T15:00:15.807 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.807 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.807 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:15.807 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.340208+0000", 2024-09-21T15:00:15.807 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2997999999999998e-05 2024-09-21T15:00:15.808 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.808 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.808 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:15.808 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.340223+0000", 2024-09-21T15:00:15.808 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5330999999999999e-05 2024-09-21T15:00:15.808 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.808 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.808 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:15.808 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.340305+0000", 2024-09-21T15:00:15.808 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2095999999999995e-05 2024-09-21T15:00:15.808 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.808 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.808 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:15.808 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.340337+0000", 2024-09-21T15:00:15.808 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2097000000000001e-05 2024-09-21T15:00:15.809 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.809 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.809 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:15.809 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.372863+0000", 2024-09-21T15:00:15.809 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.032525923999999998 2024-09-21T15:00:15.809 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.809 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.809 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:15.809 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.372919+0000", 2024-09-21T15:00:15.809 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5729000000000001e-05 2024-09-21T15:00:15.809 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:15.809 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:15.809 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:15.809 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.809 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.810 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:97675 3.b e122/24 3:d321acb8:::rbd_data.1136bfc79a70.0000000000000003:head v 122'323, mlcod=120'321)", 2024-09-21T15:00:15.810 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.341399+0000", 2024-09-21T15:00:15.810 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 10.394545963000001, 2024-09-21T15:00:15.810 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.027674555999999999, 2024-09-21T15:00:15.810 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:15.810 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:15.810 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:15.810 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.810 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:15.810 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.341399+0000", 2024-09-21T15:00:15.810 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:15.810 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.810 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.810 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:15.810 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.341399+0000", 2024-09-21T15:00:15.811 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:15.811 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.811 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.811 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:15.811 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.341398+0000", 2024-09-21T15:00:15.811 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:00:15.811 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.811 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.811 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:15.811 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.350707+0000", 2024-09-21T15:00:15.811 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0093094920000000008 2024-09-21T15:00:15.811 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.811 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.811 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:15.812 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.350730+0000", 2024-09-21T15:00:15.812 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3127999999999999e-05 2024-09-21T15:00:15.812 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.812 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.812 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:15.812 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.350741+0000", 2024-09-21T15:00:15.812 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0447e-05 2024-09-21T15:00:15.812 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.812 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.812 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:15.812 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.351206+0000", 2024-09-21T15:00:15.812 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000465439 2024-09-21T15:00:15.812 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.812 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.813 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:15.813 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.351245+0000", 2024-09-21T15:00:15.813 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8634999999999999e-05 2024-09-21T15:00:15.813 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.813 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.813 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:15.813 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.368992+0000", 2024-09-21T15:00:15.813 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.017747161000000001 2024-09-21T15:00:15.813 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.813 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.813 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:15.813 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.369074+0000", 2024-09-21T15:00:15.813 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1893999999999999e-05 2024-09-21T15:00:15.813 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:15.813 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:15.813 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:15.814 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.814 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.814 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:97733 3.4 e122/51 3:23762450:::rbd_data.1136bfc79a70.0000000000000008:head v 122'257, mlcod=120'254)", 2024-09-21T15:00:15.814 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.380585+0000", 2024-09-21T15:00:15.814 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 10.355359924, 2024-09-21T15:00:15.814 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.023704392000000001, 2024-09-21T15:00:15.814 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:15.814 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:15.814 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:15.814 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.814 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:15.814 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.380585+0000", 2024-09-21T15:00:15.814 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:15.814 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.814 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.815 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:15.815 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.380585+0000", 2024-09-21T15:00:15.815 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:15.815 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.815 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.815 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:15.815 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.380583+0000", 2024-09-21T15:00:15.815 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:00:15.815 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.815 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.815 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:15.815 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.386771+0000", 2024-09-21T15:00:15.815 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0061877160000000002 2024-09-21T15:00:15.815 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.816 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.816 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:15.816 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.386787+0000", 2024-09-21T15:00:15.816 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5596000000000001e-05 2024-09-21T15:00:15.816 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.816 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.816 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:15.816 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.386797+0000", 2024-09-21T15:00:15.816 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0316000000000001e-05 2024-09-21T15:00:15.816 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.816 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.816 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:15.816 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.386889+0000", 2024-09-21T15:00:15.816 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2335000000000003e-05 2024-09-21T15:00:15.817 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.817 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.817 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:15.817 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.386914+0000", 2024-09-21T15:00:15.817 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.499e-05 2024-09-21T15:00:15.817 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.817 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.817 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:15.817 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.404248+0000", 2024-09-21T15:00:15.817 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.017333237000000001 2024-09-21T15:00:15.817 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.817 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.817 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:15.817 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.404290+0000", 2024-09-21T15:00:15.817 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2327000000000003e-05 2024-09-21T15:00:15.818 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:15.818 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:15.818 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:15.818 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.818 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.818 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:104955 3.1 3:85de63d1:::rbd_data.1136d175325b.0000000000000033:head [write 0~32768 [fadvise_sequential+fadvise_nocache],write 65536~229376 [fadvise_sequential+fadvise_nocache],write 327680~229376 [fadvise_sequential+fadvise_nocache],write 589824~229376 [fadvise_sequential+fadvise_nocache],write 851968~229376 [fadvise_sequential+fadvise_nocache],write 1114112~229376 [fadvise_sequential+fadvise_nocache],write 1376256~229376 [fadvise_sequential+fadvise_nocache],write 1638400~229376 [fadvise_sequential+fadvise_nocache],write 1900544~196608 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-21T15:00:15.818 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:14.515375+0000", 2024-09-21T15:00:15.818 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.2205708689999999, 2024-09-21T15:00:15.818 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0095863219999999996, 2024-09-21T15:00:15.818 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:15.818 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:15.818 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:15.818 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:15.818 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:15.819 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 104955 2024-09-21T15:00:15.819 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.819 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:15.819 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.819 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:15.819 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:14.515375+0000", 2024-09-21T15:00:15.819 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:15.819 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.819 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.819 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:15.819 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:14.515375+0000", 2024-09-21T15:00:15.819 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:15.819 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.819 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.820 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:15.820 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:14.515374+0000", 2024-09-21T15:00:15.820 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:15.820 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.820 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.820 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:15.820 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:14.517138+0000", 2024-09-21T15:00:15.820 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0017643100000000001 2024-09-21T15:00:15.820 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.820 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.820 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:15.820 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:14.517151+0000", 2024-09-21T15:00:15.820 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3329999999999999e-05 2024-09-21T15:00:15.820 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.820 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.820 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:15.821 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:14.517160+0000", 2024-09-21T15:00:15.821 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6610000000000006e-06 2024-09-21T15:00:15.821 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.821 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.821 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:15.821 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:14.518305+0000", 2024-09-21T15:00:15.821 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001145189 2024-09-21T15:00:15.821 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.821 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.821 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:15.821 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:14.518433+0000", 2024-09-21T15:00:15.821 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000128045 2024-09-21T15:00:15.821 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.822 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.822 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-21T15:00:15.822 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:14.518627+0000", 2024-09-21T15:00:15.822 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019324300000000001 2024-09-21T15:00:15.822 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.822 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.822 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:15.822 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:14.522215+0000", 2024-09-21T15:00:15.822 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0035883740000000001 2024-09-21T15:00:15.822 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.822 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.822 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:15.822 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:14.524843+0000", 2024-09-21T15:00:15.822 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0026280359999999998 2024-09-21T15:00:15.823 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.823 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.823 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:15.823 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:14.524917+0000", 2024-09-21T15:00:15.823 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3924000000000006e-05 2024-09-21T15:00:15.823 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.823 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.823 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:15.823 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:14.524961+0000", 2024-09-21T15:00:15.823 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3906000000000003e-05 2024-09-21T15:00:15.823 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:15.823 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:15.823 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:15.824 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.824 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.824 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:105964 3.1 3:8730dd9c:::rbd_data.1136bfc79a70.0000000000000000:head [sparse-read 0~3145728] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-21T15:00:15.824 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:15.409532+0000", 2024-09-21T15:00:15.824 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.32641299400000001, 2024-09-21T15:00:15.824 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010457086000000001, 2024-09-21T15:00:15.824 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:15.824 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:15.824 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:15.824 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:15.824 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:15.824 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 105964 2024-09-21T15:00:15.824 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.824 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:15.825 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.825 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:15.825 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:15.409532+0000", 2024-09-21T15:00:15.825 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:15.825 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.825 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.825 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:15.825 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:15.409532+0000", 2024-09-21T15:00:15.825 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:15.825 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.825 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.825 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:15.825 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:15.409529+0000", 2024-09-21T15:00:15.825 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999971 2024-09-21T15:00:15.825 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.826 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.826 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:15.826 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:15.409545+0000", 2024-09-21T15:00:15.826 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5968999999999999e-05 2024-09-21T15:00:15.826 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.826 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.826 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:15.826 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:15.409583+0000", 2024-09-21T15:00:15.826 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7490000000000002e-05 2024-09-21T15:00:15.826 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.826 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.826 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:15.826 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:15.409600+0000", 2024-09-21T15:00:15.826 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7472000000000001e-05 2024-09-21T15:00:15.826 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.827 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.827 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:15.827 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:15.409757+0000", 2024-09-21T15:00:15.827 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015679100000000001 2024-09-21T15:00:15.827 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.827 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.827 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:15.827 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:15.409983+0000", 2024-09-21T15:00:15.827 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00022590500000000001 2024-09-21T15:00:15.827 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:15.827 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:15.827 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:15.827 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:15.419989+0000", 2024-09-21T15:00:15.827 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010006456 2024-09-21T15:00:15.827 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:15.828 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:15.828 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:15.828 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:15.828 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:15.828 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:15.830 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T15:00:15.830 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:00:15.822+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:00:15.831 INFO:teuthology.orchestra.run.smithi032.stdout:3466 writesame 0xe560910 thru 0xe564371 data_size 0x35(0x3a62 bytes) 2024-09-21T15:00:15.831 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:15.831 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:15.831 INFO:teuthology.orchestra.run.smithi032.stdout:3470 write 0xd966864 thru 0xd96bc9f (0x543c bytes) 2024-09-21T15:00:15.831 INFO:teuthology.orchestra.run.smithi032.stdout:3471 punch from 0x2e6c09e to 0x2e76c3b, (0xab9d bytes) 2024-09-21T15:00:15.831 INFO:teuthology.orchestra.run.smithi032.stdout:3473 punch from 0x2bd8c6 to 0x2ca8b6, (0xcff0 bytes) 2024-09-21T15:00:15.831 INFO:teuthology.orchestra.run.smithi032.stdout:3474 write 0x4aca41 thru 0x4b892a (0xbeea bytes) 2024-09-21T15:00:15.832 INFO:teuthology.orchestra.run.smithi032.stdout:3477 read 0x7fad1bd thru 0x7fbbf8f (0xedd3 bytes) 2024-09-21T15:00:15.832 INFO:teuthology.orchestra.run.smithi032.stdout:3479 read 0xd2a8476 thru 0xd2ae788 (0x6313 bytes) 2024-09-21T15:00:15.832 INFO:teuthology.orchestra.run.smithi032.stdout:3480 read 0x1177139 thru 0x1184151 (0xd019 bytes) 2024-09-21T15:00:15.832 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:15.832 INFO:teuthology.orchestra.run.smithi032.stdout:3487 read 0x8b1d85b thru 0x8b1e1e3 (0x989 bytes) 2024-09-21T15:00:15.832 INFO:teuthology.orchestra.run.smithi032.stdout:3488 read 0x5982774 thru 0x5985f0c (0x3799 bytes) 2024-09-21T15:00:15.832 INFO:teuthology.orchestra.run.smithi032.stdout:3489 trunc from 0xe564372 to 0x8b0921f 2024-09-21T15:00:15.832 INFO:teuthology.orchestra.run.smithi032.stdout:3490 read 0x2491ff7 thru 0x24930b4 (0x10be bytes) 2024-09-21T15:00:15.832 INFO:teuthology.orchestra.run.smithi032.stdout:3491 punch from 0x1a022aa to 0x1a0c297, (0x9fed bytes) 2024-09-21T15:00:15.832 INFO:teuthology.orchestra.run.smithi032.stdout:3492 read 0x8311bcd thru 0x831f469 (0xd89d bytes) 2024-09-21T15:00:15.832 INFO:teuthology.orchestra.run.smithi032.stdout:3493 read 0x4a27746 thru 0x4a35964 (0xe21f bytes) 2024-09-21T15:00:15.832 INFO:teuthology.orchestra.run.smithi032.stdout:3494 write 0x2357ced thru 0x2365c42 (0xdf56 bytes) 2024-09-21T15:00:15.832 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:15.832 INFO:teuthology.orchestra.run.smithi032.stdout:3496 write 0x7d75d5d thru 0x7d765ac (0x850 bytes) 2024-09-21T15:00:15.848 INFO:teuthology.orchestra.run.smithi032.stdout:3497 writesame 0xee0eda1 thru 0xee12473 data_size 0x7(0x36d3 bytes) 2024-09-21T15:00:15.848 INFO:teuthology.orchestra.run.smithi032.stdout:3499 read 0x3e3f91d thru 0x3e49741 (0x9e25 bytes) 2024-09-21T15:00:15.850 INFO:teuthology.orchestra.run.smithi032.stdout:3500 writesame 0xd1eed9b thru 0xd1f0ca0 data_size 0x16(0x1f06 bytes) 2024-09-21T15:00:15.861 INFO:teuthology.orchestra.run.smithi032.stdout:3501 punch from 0x3a2c453 to 0x3a2f0ad, (0x2c5a bytes) 2024-09-21T15:00:15.890 INFO:teuthology.orchestra.run.smithi032.stdout:3503 trunc from 0xee12474 to 0x125724e 2024-09-21T15:00:15.930 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:00:15.928+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:00:15.931 INFO:teuthology.orchestra.run.smithi032.stdout:3504 write 0x107495f thru 0x108314a (0xe7ec bytes) 2024-09-21T15:00:15.934 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:15.934 INFO:teuthology.orchestra.run.smithi032.stdout:3506 read 0x9e05f5 thru 0x9edd82 (0xd78e bytes) 2024-09-21T15:00:15.934 INFO:teuthology.orchestra.run.smithi032.stdout:3507 read 0x10eb66b thru 0x10f85ff (0xcf95 bytes) 2024-09-21T15:00:15.959 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:15.959 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:00:15.959 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T15:00:15.959 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:15.967 INFO:teuthology.orchestra.run.smithi032.stdout:3508 writesame 0xdc131e5 thru 0xdc1f4eb data_size 0x1(0xc307 bytes) 2024-09-21T15:00:15.967 INFO:teuthology.orchestra.run.smithi032.stdout:3512 punch from 0x17372d5 to 0x173e2c1, (0x6fec bytes) 2024-09-21T15:00:15.968 INFO:teuthology.orchestra.run.smithi032.stdout:3514 trunc from 0xdc1f4ec to 0xd4d82b5 2024-09-21T15:00:15.973 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T15:00:15.979 INFO:teuthology.orchestra.run.smithi032.stdout:3516 write 0x907e49e thru 0x9082aa3 (0x4606 bytes) 2024-09-21T15:00:15.987 INFO:teuthology.orchestra.run.smithi032.stdout:3519 punch from 0xa8fae7c to 0xa8fef96, (0x411a bytes) 2024-09-21T15:00:15.988 INFO:teuthology.orchestra.run.smithi032.stdout:3520 writesame 0xc94bbbd thru 0xc956f99 data_size 0x1(0xb3dd bytes) 2024-09-21T15:00:16.014 INFO:teuthology.orchestra.run.smithi032.stdout:3522 read 0x1514ea thru 0x160af5 (0xf60c bytes) 2024-09-21T15:00:16.015 INFO:teuthology.orchestra.run.smithi032.stdout:3523 punch from 0x70237ae to 0x70303cb, (0xcc1d bytes) 2024-09-21T15:00:16.016 INFO:teuthology.orchestra.run.smithi032.stdout:3524 write 0xca7a44f thru 0xca833a7 (0x8f59 bytes) 2024-09-21T15:00:16.024 INFO:teuthology.orchestra.run.smithi032.stdout:3526 read 0x2442fcb thru 0x244ef68 (0xbf9e bytes) 2024-09-21T15:00:16.024 INFO:teuthology.orchestra.run.smithi032.stdout:3528 punch from 0x5ecf90b to 0x5ed00a0, (0x795 bytes) 2024-09-21T15:00:16.024 INFO:teuthology.orchestra.run.smithi032.stdout:3529 write 0x22c0076 thru 0x22c7747 (0x76d2 bytes) 2024-09-21T15:00:16.031 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:00:16.028+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:00:16.031 INFO:teuthology.orchestra.run.smithi032.stdout:3530 writesame 0x5e976e9 thru 0x5e9d05f data_size 0x1(0x5977 bytes) 2024-09-21T15:00:16.052 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:16.052 INFO:teuthology.orchestra.run.smithi032.stdout:3533 write 0xb748db2 thru 0xb74f4e2 (0x6731 bytes) 2024-09-21T15:00:16.077 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:16.077 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:00:16.077 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T15:00:16.078 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T15:00:16.078 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:16.084 INFO:teuthology.orchestra.run.smithi032.stdout:3535 trunc from 0xd4d82b5 to 0x176a352 2024-09-21T15:00:16.087 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T15:00:16.109 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:16.109 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:16.127 INFO:teuthology.orchestra.run.smithi032.stdout:3538 write 0x406df43 thru 0x40719fd (0x3abb bytes) 2024-09-21T15:00:16.132 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:00:16.126+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:00:16.134 INFO:teuthology.orchestra.run.smithi032.stdout:3540 writesame 0x24d6bfb thru 0x24db23c data_size 0x17(0x4642 bytes) 2024-09-21T15:00:16.159 INFO:teuthology.orchestra.run.smithi032.stdout:3541 write 0x592afc5 thru 0x5935cc6 (0xad02 bytes) 2024-09-21T15:00:16.183 INFO:teuthology.orchestra.run.smithi032.stdout:3542 trunc from 0x5935cc7 to 0xc9c2aee 2024-09-21T15:00:16.195 INFO:teuthology.orchestra.run.smithi032.stdout:3544 writesame 0x1542170 thru 0x154ce59 data_size 0x1(0xacea bytes) 2024-09-21T15:00:16.197 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:16.197 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T15:00:16.197 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T15:00:16.197 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T15:00:16.197 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.197 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31476 3.c 3:35718b6b:::rbd_data.11367721c19.0000000000000014:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T15:00:16.197 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.505975+0000", 2024-09-21T15:00:16.197 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 77.687078786000001, 2024-09-21T15:00:16.197 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99515995899999998, 2024-09-21T15:00:16.197 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:16.197 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:16.197 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:16.198 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:16.198 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:16.198 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31476 2024-09-21T15:00:16.198 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.198 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:16.198 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.198 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:16.198 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.505975+0000", 2024-09-21T15:00:16.198 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:16.198 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.198 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.198 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:16.198 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.505975+0000", 2024-09-21T15:00:16.199 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:16.199 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.199 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.199 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:16.199 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.505971+0000", 2024-09-21T15:00:16.199 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999967 2024-09-21T15:00:16.199 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.199 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.199 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:16.199 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.505989+0000", 2024-09-21T15:00:16.199 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7985000000000001e-05 2024-09-21T15:00:16.199 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.199 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.199 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:16.200 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506050+0000", 2024-09-21T15:00:16.200 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0659000000000003e-05 2024-09-21T15:00:16.200 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.200 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.200 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:16.200 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506074+0000", 2024-09-21T15:00:16.200 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4156e-05 2024-09-21T15:00:16.200 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.200 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.200 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:16.200 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506225+0000", 2024-09-21T15:00:16.200 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015097 2024-09-21T15:00:16.200 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.200 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.201 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:00:16.201 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506256+0000", 2024-09-21T15:00:16.201 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0505999999999999e-05 2024-09-21T15:00:16.201 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.201 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.201 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:16.201 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.497974+0000", 2024-09-21T15:00:16.201 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99171825700000005 2024-09-21T15:00:16.201 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.201 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.201 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for missing object", 2024-09-21T15:00:16.201 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.498449+0000", 2024-09-21T15:00:16.201 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00047506399999999998 2024-09-21T15:00:16.201 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.202 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.202 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:16.202 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.501000+0000", 2024-09-21T15:00:16.202 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0025506919999999998 2024-09-21T15:00:16.202 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.202 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.202 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:16.202 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.501135+0000", 2024-09-21T15:00:16.202 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013524000000000001 2024-09-21T15:00:16.202 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:16.202 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:16.202 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:16.202 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.203 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.203 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31480 3.c 3:36ccd808:::rbd_data.11367721c19.0000000000000018:head [sparse-read 0~14821] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T15:00:16.203 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.506116+0000", 2024-09-21T15:00:16.203 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 77.686937443000005, 2024-09-21T15:00:16.203 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99316033599999998, 2024-09-21T15:00:16.203 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:16.203 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:16.203 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:16.203 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:16.203 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:16.203 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31480 2024-09-21T15:00:16.203 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.203 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:16.203 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.204 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:16.204 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506116+0000", 2024-09-21T15:00:16.204 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:16.204 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.204 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.204 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:16.204 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506116+0000", 2024-09-21T15:00:16.204 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:16.204 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.204 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.204 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:16.204 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506115+0000", 2024-09-21T15:00:16.204 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:00:16.205 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.205 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.205 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:16.205 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506124+0000", 2024-09-21T15:00:16.205 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0659999999999993e-06 2024-09-21T15:00:16.205 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.205 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.205 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:16.205 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506145+0000", 2024-09-21T15:00:16.205 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1447e-05 2024-09-21T15:00:16.205 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.205 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.205 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:16.205 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506161+0000", 2024-09-21T15:00:16.206 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5209e-05 2024-09-21T15:00:16.206 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.206 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.206 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:16.206 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506337+0000", 2024-09-21T15:00:16.206 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017642999999999999 2024-09-21T15:00:16.206 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.206 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.206 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:00:16.206 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506375+0000", 2024-09-21T15:00:16.206 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7536000000000001e-05 2024-09-21T15:00:16.206 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.206 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.206 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:16.207 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.498815+0000", 2024-09-21T15:00:16.207 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99244067999999996 2024-09-21T15:00:16.207 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.207 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.207 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:16.207 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.498950+0000", 2024-09-21T15:00:16.207 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013436700000000001 2024-09-21T15:00:16.207 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.207 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.207 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:16.207 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499277+0000", 2024-09-21T15:00:16.207 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00032696800000000002 2024-09-21T15:00:16.207 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:16.207 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:16.208 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:16.208 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.208 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.208 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31513 3.c 3:36de8523:::rbd_data.11367721c19.0000000000000039:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T15:00:16.208 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.509071+0000", 2024-09-21T15:00:16.208 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 77.683983243, 2024-09-21T15:00:16.208 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99038225099999999, 2024-09-21T15:00:16.208 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:16.208 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:16.208 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:16.208 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:16.208 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:16.208 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31513 2024-09-21T15:00:16.208 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.208 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:16.209 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.209 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:16.209 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509071+0000", 2024-09-21T15:00:16.209 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:16.209 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.209 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.209 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:16.209 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509071+0000", 2024-09-21T15:00:16.209 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:16.209 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.209 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.209 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:16.209 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509068+0000", 2024-09-21T15:00:16.209 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-21T15:00:16.209 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.209 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.210 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:16.210 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509082+0000", 2024-09-21T15:00:16.210 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3545e-05 2024-09-21T15:00:16.210 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.210 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.210 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:16.210 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509116+0000", 2024-09-21T15:00:16.210 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3942000000000001e-05 2024-09-21T15:00:16.210 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.210 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.210 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:16.210 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509134+0000", 2024-09-21T15:00:16.210 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8839999999999999e-05 2024-09-21T15:00:16.210 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.211 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.211 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:16.211 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509318+0000", 2024-09-21T15:00:16.211 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018326799999999999 2024-09-21T15:00:16.211 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.211 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.211 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:00:16.211 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509346+0000", 2024-09-21T15:00:16.211 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8019000000000001e-05 2024-09-21T15:00:16.211 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.211 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.211 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:16.211 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499330+0000", 2024-09-21T15:00:16.211 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.98998399299999995 2024-09-21T15:00:16.211 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.212 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.212 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:16.212 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499453+0000", 2024-09-21T15:00:16.212 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000123115 2024-09-21T15:00:16.212 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:16.212 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:16.212 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:16.212 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.212 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.212 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31515 3.c 3:35b36ddd:::rbd_data.11367721c19.000000000000003b:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T15:00:16.212 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.509177+0000", 2024-09-21T15:00:16.212 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 77.683876816999998, 2024-09-21T15:00:16.212 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99044870600000001, 2024-09-21T15:00:16.212 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:16.212 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:16.213 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:16.213 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:16.213 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:16.213 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31515 2024-09-21T15:00:16.213 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.213 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:16.213 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.213 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:16.213 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509177+0000", 2024-09-21T15:00:16.213 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:16.213 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.213 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.213 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:16.213 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509177+0000", 2024-09-21T15:00:16.214 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:16.214 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.214 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.214 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:16.214 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509176+0000", 2024-09-21T15:00:16.214 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:00:16.214 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.214 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.214 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:16.214 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509183+0000", 2024-09-21T15:00:16.214 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6720000000000004e-06 2024-09-21T15:00:16.214 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.214 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.214 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:16.215 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509269+0000", 2024-09-21T15:00:16.215 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5881e-05 2024-09-21T15:00:16.215 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.215 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.215 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:16.215 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509303+0000", 2024-09-21T15:00:16.215 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3367000000000001e-05 2024-09-21T15:00:16.215 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.215 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.215 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:16.215 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509425+0000", 2024-09-21T15:00:16.215 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000122199 2024-09-21T15:00:16.215 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.215 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.216 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:00:16.216 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509448+0000", 2024-09-21T15:00:16.216 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2688999999999999e-05 2024-09-21T15:00:16.216 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.216 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.216 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:16.216 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499501+0000", 2024-09-21T15:00:16.216 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99005306999999998 2024-09-21T15:00:16.216 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.216 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.216 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:16.216 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499626+0000", 2024-09-21T15:00:16.216 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012501400000000001 2024-09-21T15:00:16.216 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:16.216 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:16.216 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:16.217 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.217 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.217 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31531 3.c 3:314f0287:::rbd_data.11367721c19.000000000000004b:head [sparse-read 0~7106] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T15:00:16.217 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.509329+0000", 2024-09-21T15:00:16.217 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 77.683724796999996, 2024-09-21T15:00:16.217 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99103085300000004, 2024-09-21T15:00:16.217 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:16.217 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:16.217 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:16.217 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:16.217 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:16.217 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31531 2024-09-21T15:00:16.217 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.217 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:16.218 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.218 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:16.218 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509329+0000", 2024-09-21T15:00:16.218 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:16.218 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.218 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.218 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:16.218 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509329+0000", 2024-09-21T15:00:16.218 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:16.218 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.218 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.218 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:16.218 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509328+0000", 2024-09-21T15:00:16.218 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:00:16.218 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.219 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.219 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:16.219 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509336+0000", 2024-09-21T15:00:16.219 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3170000000000002e-06 2024-09-21T15:00:16.219 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.219 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.219 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:16.219 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509353+0000", 2024-09-21T15:00:16.219 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7214000000000002e-05 2024-09-21T15:00:16.219 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.219 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.219 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:16.219 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509366+0000", 2024-09-21T15:00:16.220 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.26e-05 2024-09-21T15:00:16.220 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.220 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.220 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:16.220 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509523+0000", 2024-09-21T15:00:16.220 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015708000000000001 2024-09-21T15:00:16.220 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.220 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.220 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:00:16.220 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509544+0000", 2024-09-21T15:00:16.220 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0783000000000001e-05 2024-09-21T15:00:16.220 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.220 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.220 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:16.221 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499727+0000", 2024-09-21T15:00:16.221 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99018345200000002 2024-09-21T15:00:16.221 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.221 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.221 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:16.221 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499949+0000", 2024-09-21T15:00:16.221 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000221358 2024-09-21T15:00:16.221 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.221 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.221 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:16.221 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.500360+0000", 2024-09-21T15:00:16.221 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00041130700000000002 2024-09-21T15:00:16.221 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:16.221 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:16.221 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:16.222 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.222 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.222 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:61580 3.b 3:d3369e4f:::rbd_data.1136862f872f.0000000000000010:head [write 0~557056 [fadvise_sequential+fadvise_nocache],zero 557056~16384,write 573440~49152 [fadvise_sequential+fadvise_nocache],zero 622592~16384,write 638976~49152 [fadvise_sequential+fadvise_nocache],zero 688128~16384,write 704512~49152 [fadvise_sequential+fadvise_nocache],zero 753664~16384,write 770048~49152 [fadvise_sequential+fadvise_nocache],zero 819200~16384,write 835584~49152 [fadvise_sequential+fadvise_nocache],zero 884736~16384,write 901120~49152 [fadvise_sequential+fadvise_nocache],zero 950272~16384,write 966656~49152 [fadvise_sequential+fadvise_nocache],zero 1015808~16384,write 1032192~16384 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-21T15:00:16.222 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:19.231716+0000", 2024-09-21T15:00:16.222 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 56.961337635, 2024-09-21T15:00:16.222 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0096097690000000006, 2024-09-21T15:00:16.222 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:16.222 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:16.222 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:16.222 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:16.222 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:16.222 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 61580 2024-09-21T15:00:16.222 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.223 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:16.223 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.223 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:16.223 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.231716+0000", 2024-09-21T15:00:16.223 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:16.223 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.223 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.223 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:16.223 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.231716+0000", 2024-09-21T15:00:16.223 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:16.223 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.223 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.223 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:16.223 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.231715+0000", 2024-09-21T15:00:16.224 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:00:16.224 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.224 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.224 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:16.224 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.235290+0000", 2024-09-21T15:00:16.224 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0035750399999999998 2024-09-21T15:00:16.224 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.224 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.224 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:16.224 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.235310+0000", 2024-09-21T15:00:16.224 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0120999999999999e-05 2024-09-21T15:00:16.224 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.224 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.224 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:16.225 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.235319+0000", 2024-09-21T15:00:16.225 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2420000000000006e-06 2024-09-21T15:00:16.225 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.225 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.225 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:16.225 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.235416+0000", 2024-09-21T15:00:16.225 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7204e-05 2024-09-21T15:00:16.225 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.225 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.225 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:16.225 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.235775+0000", 2024-09-21T15:00:16.225 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000358883 2024-09-21T15:00:16.225 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.225 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.226 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T15:00:16.226 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.236560+0000", 2024-09-21T15:00:16.226 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000784463 2024-09-21T15:00:16.226 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.226 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.226 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:16.226 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.241119+0000", 2024-09-21T15:00:16.226 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0045590589999999999 2024-09-21T15:00:16.226 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.226 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.226 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:16.226 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.241210+0000", 2024-09-21T15:00:16.226 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1617000000000001e-05 2024-09-21T15:00:16.226 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.227 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.227 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:16.227 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.241285+0000", 2024-09-21T15:00:16.227 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4470999999999999e-05 2024-09-21T15:00:16.227 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.227 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.227 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:16.227 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.241326+0000", 2024-09-21T15:00:16.227 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0967000000000003e-05 2024-09-21T15:00:16.227 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:16.227 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:16.227 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:16.227 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.228 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.228 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:73699 3.b 3:d293b54a:::rbd_data.1136f4939602.000000000000000a:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-21T15:00:16.228 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:41.971359+0000", 2024-09-21T15:00:16.228 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 34.221695079, 2024-09-21T15:00:16.228 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0082840980000000002, 2024-09-21T15:00:16.228 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:16.228 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:16.228 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:16.228 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:16.228 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:16.228 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 73699 2024-09-21T15:00:16.228 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.228 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:16.228 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.229 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:16.229 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.971359+0000", 2024-09-21T15:00:16.229 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:16.229 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.229 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.229 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:16.229 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.971359+0000", 2024-09-21T15:00:16.229 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:16.229 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.229 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.229 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:16.229 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.971356+0000", 2024-09-21T15:00:16.229 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-21T15:00:16.229 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.229 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.230 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:16.230 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.971370+0000", 2024-09-21T15:00:16.230 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3349999999999999e-05 2024-09-21T15:00:16.230 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.230 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.230 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:16.230 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.971393+0000", 2024-09-21T15:00:16.230 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3799999999999999e-05 2024-09-21T15:00:16.230 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.230 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.230 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:16.230 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.971405+0000", 2024-09-21T15:00:16.230 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1569000000000001e-05 2024-09-21T15:00:16.230 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.230 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.231 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:16.231 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.971510+0000", 2024-09-21T15:00:16.231 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010456899999999999 2024-09-21T15:00:16.231 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.231 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.231 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:16.231 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.971752+0000", 2024-09-21T15:00:16.231 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00024250300000000001 2024-09-21T15:00:16.231 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.231 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.231 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T15:00:16.231 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.973012+0000", 2024-09-21T15:00:16.231 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0012595429999999999 2024-09-21T15:00:16.231 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.231 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.232 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:16.232 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.979357+0000", 2024-09-21T15:00:16.232 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0063454469999999997 2024-09-21T15:00:16.232 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.232 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.232 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:16.232 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.979467+0000", 2024-09-21T15:00:16.232 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000109623 2024-09-21T15:00:16.232 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.232 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.232 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:16.232 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.979585+0000", 2024-09-21T15:00:16.232 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000118044 2024-09-21T15:00:16.232 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.232 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.232 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:16.233 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.979643+0000", 2024-09-21T15:00:16.233 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8069999999999998e-05 2024-09-21T15:00:16.233 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:16.233 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:16.233 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:16.233 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.233 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.233 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:82545 3.b 3:d4eb1ee5:::rbd_data.11368296f791.0000000000000000:head [write 0~671744 [fadvise_sequential+fadvise_nocache],write 704512~65536 [fadvise_sequential+fadvise_nocache],write 786432~49152 [fadvise_sequential+fadvise_nocache],write 868352~49152 [fadvise_sequential+fadvise_nocache],write 933888~65536 [fadvise_sequential+fadvise_nocache],write 1032192~32768 [fadvise_sequential+fadvise_nocache],write 1097728~65536 [fadvise_sequential+fadvise_nocache],write 1179648~49152 [fadvise_sequential+fadvise_nocache],write 1261568~49152 [fadvise_sequential+fadvise_nocache],write 1327104~65536 [fadvise_sequential+fadvise_nocache],write 1425408~32768 [fadvise_sequential+fadvise_nocache],write 1490944~65536 [fadvise_sequential+fadvise_nocache],write 1572864~8192 [fadvise_sequential+fadvise_nocache],write 1630208~49152 [fadvise_sequential+fadvise_nocache],write 1703936~40960 [fadvise_sequential+fadvise_nocache],write 1794048~40960 [fadvise_sequential+fadvise_nocache],write 1859584~49152 [fadvise_sequential+fadvise_nocache],write 1957888~8192 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:00:16.233 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:47.416089+0000", 2024-09-21T15:00:16.233 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 28.776965071999999, 2024-09-21T15:00:16.233 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011896455, 2024-09-21T15:00:16.233 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:16.233 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:16.233 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:16.234 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:16.234 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:16.234 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82545 2024-09-21T15:00:16.234 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.234 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:16.234 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.234 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:16.234 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.416089+0000", 2024-09-21T15:00:16.234 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:16.234 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.234 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.234 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:16.234 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.416089+0000", 2024-09-21T15:00:16.234 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:16.235 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.235 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.235 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:16.235 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.416087+0000", 2024-09-21T15:00:16.235 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:00:16.235 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.235 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.235 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:16.235 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.419643+0000", 2024-09-21T15:00:16.235 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0035555550000000002 2024-09-21T15:00:16.235 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.235 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.235 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:16.235 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.419663+0000", 2024-09-21T15:00:16.236 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0202000000000001e-05 2024-09-21T15:00:16.236 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.236 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.236 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:16.236 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.419676+0000", 2024-09-21T15:00:16.236 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3207000000000001e-05 2024-09-21T15:00:16.236 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.236 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.236 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:16.236 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.419759+0000", 2024-09-21T15:00:16.236 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3521000000000002e-05 2024-09-21T15:00:16.236 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.236 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.236 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op must wait for map", 2024-09-21T15:00:16.236 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.419768+0000", 2024-09-21T15:00:16.237 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4479999999999994e-06 2024-09-21T15:00:16.237 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.237 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.237 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:16.237 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.422171+0000", 2024-09-21T15:00:16.237 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0024035469999999998 2024-09-21T15:00:16.237 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.237 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.237 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:16.237 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.422350+0000", 2024-09-21T15:00:16.237 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000179001 2024-09-21T15:00:16.237 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.237 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.238 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T15:00:16.238 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.422710+0000", 2024-09-21T15:00:16.238 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000359878 2024-09-21T15:00:16.238 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.238 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.238 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:16.238 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.426676+0000", 2024-09-21T15:00:16.238 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0039655790000000003 2024-09-21T15:00:16.238 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.238 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.238 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:16.238 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.427884+0000", 2024-09-21T15:00:16.238 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001208207 2024-09-21T15:00:16.239 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.239 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.239 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:16.239 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.427953+0000", 2024-09-21T15:00:16.239 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8533000000000003e-05 2024-09-21T15:00:16.239 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.239 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.239 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:16.239 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.427985+0000", 2024-09-21T15:00:16.239 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2481999999999998e-05 2024-09-21T15:00:16.239 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:16.239 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:16.239 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:16.239 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.239 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.240 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:83735 3.b 3:d3640016:::rbd_data.11368296f791.0000000000000006:40 [sparse-read 393216~131072 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-21T15:00:16.240 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:51.156767+0000", 2024-09-21T15:00:16.240 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 25.036286448999999, 2024-09-21T15:00:16.240 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0082826410000000003, 2024-09-21T15:00:16.240 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:16.240 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:16.240 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:16.240 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:16.240 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:16.240 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 83735 2024-09-21T15:00:16.240 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.240 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:16.240 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.240 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:16.241 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156767+0000", 2024-09-21T15:00:16.241 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:16.241 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.241 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.241 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:16.241 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156767+0000", 2024-09-21T15:00:16.241 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:16.241 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.241 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.241 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:16.241 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156767+0000", 2024-09-21T15:00:16.241 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:16.241 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.241 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.241 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:16.241 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156771+0000", 2024-09-21T15:00:16.242 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0940000000000001e-06 2024-09-21T15:00:16.242 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.242 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.242 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:16.242 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156781+0000", 2024-09-21T15:00:16.242 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0267e-05 2024-09-21T15:00:16.242 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.242 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.242 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:16.242 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156789+0000", 2024-09-21T15:00:16.242 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7659999999999998e-06 2024-09-21T15:00:16.242 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.242 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.242 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:16.242 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.164742+0000", 2024-09-21T15:00:16.243 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0079534519999999997 2024-09-21T15:00:16.243 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.243 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.243 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:16.243 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.164820+0000", 2024-09-21T15:00:16.243 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7927000000000001e-05 2024-09-21T15:00:16.243 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.243 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.243 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:16.243 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165050+0000", 2024-09-21T15:00:16.243 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000229731 2024-09-21T15:00:16.243 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:16.243 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:16.243 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:16.244 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.244 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.244 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:83736 3.b 3:d3640016:::rbd_data.11368296f791.0000000000000006:40 [sparse-read 1835008~0 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-21T15:00:16.244 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:51.156796+0000", 2024-09-21T15:00:16.244 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 25.036257446, 2024-09-21T15:00:16.244 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0085139110000000007, 2024-09-21T15:00:16.244 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:16.244 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:16.244 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:16.244 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:16.244 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:16.244 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 83736 2024-09-21T15:00:16.244 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.244 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:16.244 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.245 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:16.245 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156796+0000", 2024-09-21T15:00:16.245 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:16.245 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.245 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.245 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:16.245 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156796+0000", 2024-09-21T15:00:16.245 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:16.245 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.245 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.245 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:16.245 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156796+0000", 2024-09-21T15:00:16.245 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:16.245 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.245 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.246 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:16.246 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156800+0000", 2024-09-21T15:00:16.246 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.34e-06 2024-09-21T15:00:16.246 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.246 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.246 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:16.246 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156813+0000", 2024-09-21T15:00:16.246 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2532e-05 2024-09-21T15:00:16.246 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.246 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.246 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:16.246 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156821+0000", 2024-09-21T15:00:16.246 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2069999999999995e-06 2024-09-21T15:00:16.246 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.247 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.247 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:16.247 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165100+0000", 2024-09-21T15:00:16.247 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0082788730000000008 2024-09-21T15:00:16.247 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.247 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.247 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:16.247 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165192+0000", 2024-09-21T15:00:16.247 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2095999999999994e-05 2024-09-21T15:00:16.247 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.247 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.247 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:16.247 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165310+0000", 2024-09-21T15:00:16.247 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000118461 2024-09-21T15:00:16.247 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:16.248 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:16.248 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:16.248 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.248 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.248 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T15:00:16.249 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:16.249 INFO:teuthology.orchestra.run.smithi032.stdout:3546 write 0xe7ad084 thru 0xe7bbd0f (0xec8c bytes) 2024-09-21T15:00:16.250 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:00:16.248+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:00:16.250 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:83739 3.b 3:d4eb1ee5:::rbd_data.11368296f791.0000000000000000:40 [sparse-read 524288~131072 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-21T15:00:16.250 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:51.156827+0000", 2024-09-21T15:00:16.251 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 25.036226363000001, 2024-09-21T15:00:16.251 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0087319900000000002, 2024-09-21T15:00:16.251 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:16.251 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:16.251 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:16.251 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:16.251 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:16.251 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 83739 2024-09-21T15:00:16.251 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.251 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:16.251 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.251 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:16.251 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156827+0000", 2024-09-21T15:00:16.252 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:16.252 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.252 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.252 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:16.252 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156827+0000", 2024-09-21T15:00:16.252 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:16.252 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.252 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.252 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:16.252 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156827+0000", 2024-09-21T15:00:16.252 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:16.252 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.252 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.253 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:16.253 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156831+0000", 2024-09-21T15:00:16.253 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9940000000000002e-06 2024-09-21T15:00:16.253 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.253 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.253 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:16.253 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156841+0000", 2024-09-21T15:00:16.253 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6760000000000002e-06 2024-09-21T15:00:16.253 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.253 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.253 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:16.253 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156853+0000", 2024-09-21T15:00:16.253 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1972e-05 2024-09-21T15:00:16.253 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.254 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.254 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:16.254 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165334+0000", 2024-09-21T15:00:16.254 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0084816939999999997 2024-09-21T15:00:16.254 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.254 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.254 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:16.254 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165375+0000", 2024-09-21T15:00:16.254 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0911000000000002e-05 2024-09-21T15:00:16.254 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.254 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.254 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:16.254 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165559+0000", 2024-09-21T15:00:16.254 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018429300000000001 2024-09-21T15:00:16.254 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:16.255 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:16.255 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:16.255 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.255 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.255 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:83740 3.b 3:d4eb1ee5:::rbd_data.11368296f791.0000000000000000:40 [sparse-read 1966080~0 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-21T15:00:16.255 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:51.156859+0000", 2024-09-21T15:00:16.255 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 25.036194373000001, 2024-09-21T15:00:16.255 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0088540089999999995, 2024-09-21T15:00:16.255 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:16.255 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:16.255 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:16.255 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:16.255 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:16.255 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 83740 2024-09-21T15:00:16.256 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.256 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:16.256 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.256 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:16.256 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156859+0000", 2024-09-21T15:00:16.256 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:16.256 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.256 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.256 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:16.256 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156859+0000", 2024-09-21T15:00:16.256 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:16.256 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.256 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.256 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:16.256 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156859+0000", 2024-09-21T15:00:16.257 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:16.257 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.257 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.257 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:16.257 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156863+0000", 2024-09-21T15:00:16.257 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.844e-06 2024-09-21T15:00:16.257 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.257 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.257 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:16.257 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156873+0000", 2024-09-21T15:00:16.257 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0069e-05 2024-09-21T15:00:16.257 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.257 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.257 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:16.257 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156880+0000", 2024-09-21T15:00:16.257 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4240000000000004e-06 2024-09-21T15:00:16.258 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.258 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.258 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:16.258 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165590+0000", 2024-09-21T15:00:16.258 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0087098100000000001 2024-09-21T15:00:16.258 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.258 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.258 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:16.258 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165656+0000", 2024-09-21T15:00:16.258 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5541000000000001e-05 2024-09-21T15:00:16.258 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.258 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.258 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:16.258 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165713+0000", 2024-09-21T15:00:16.259 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7865000000000002e-05 2024-09-21T15:00:16.259 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:16.259 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:16.259 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:16.259 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.259 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.259 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:83751 3.b 3:d3640016:::rbd_data.11368296f791.0000000000000006:40 [sparse-read 1179648~0 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-21T15:00:16.259 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:51.156888+0000", 2024-09-21T15:00:16.259 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 25.036166112, 2024-09-21T15:00:16.259 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0090039539999999998, 2024-09-21T15:00:16.259 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:16.259 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:16.259 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:16.259 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:16.260 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:16.260 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 83751 2024-09-21T15:00:16.260 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.260 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:16.260 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.260 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:16.260 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156888+0000", 2024-09-21T15:00:16.260 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:16.260 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.260 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.260 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:16.260 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156888+0000", 2024-09-21T15:00:16.260 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:16.260 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.260 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.261 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:16.261 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156887+0000", 2024-09-21T15:00:16.261 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:16.261 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.261 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.261 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:16.261 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156891+0000", 2024-09-21T15:00:16.261 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.518e-06 2024-09-21T15:00:16.261 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.261 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.261 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:16.261 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156900+0000", 2024-09-21T15:00:16.261 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1759999999999999e-06 2024-09-21T15:00:16.261 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.261 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.261 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:16.262 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156908+0000", 2024-09-21T15:00:16.262 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8439999999999998e-06 2024-09-21T15:00:16.262 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.262 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.262 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:16.262 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165738+0000", 2024-09-21T15:00:16.262 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0088305689999999999 2024-09-21T15:00:16.262 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.262 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.262 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:16.262 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165804+0000", 2024-09-21T15:00:16.262 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5535000000000002e-05 2024-09-21T15:00:16.262 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.262 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.262 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:16.263 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165892+0000", 2024-09-21T15:00:16.263 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7860000000000002e-05 2024-09-21T15:00:16.263 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:16.263 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:16.263 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:16.263 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.263 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.263 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:83752 3.b 3:d3640016:::rbd_data.11368296f791.0000000000000006:40 [sparse-read 2621440~0 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-21T15:00:16.263 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:51.156915+0000", 2024-09-21T15:00:16.263 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 25.036138565000002, 2024-09-21T15:00:16.263 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0090768180000000004, 2024-09-21T15:00:16.263 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:16.263 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:16.263 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:16.263 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:16.264 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:16.264 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 83752 2024-09-21T15:00:16.264 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.264 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:16.264 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.264 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:16.264 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156915+0000", 2024-09-21T15:00:16.264 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:16.264 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.264 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.264 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:16.264 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156915+0000", 2024-09-21T15:00:16.264 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:16.264 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.264 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.265 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:16.265 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156915+0000", 2024-09-21T15:00:16.265 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:16.265 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.265 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.265 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:16.265 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156919+0000", 2024-09-21T15:00:16.265 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2679999999999996e-06 2024-09-21T15:00:16.265 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.265 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.265 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:16.265 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156929+0000", 2024-09-21T15:00:16.265 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.031e-05 2024-09-21T15:00:16.265 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.266 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.266 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:16.266 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156937+0000", 2024-09-21T15:00:16.266 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8820000000000001e-06 2024-09-21T15:00:16.266 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.266 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.266 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:16.266 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165909+0000", 2024-09-21T15:00:16.266 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0089722989999999996 2024-09-21T15:00:16.266 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.266 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.266 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:16.266 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165948+0000", 2024-09-21T15:00:16.266 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8114999999999997e-05 2024-09-21T15:00:16.267 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.267 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.267 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:16.267 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165992+0000", 2024-09-21T15:00:16.267 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4503999999999999e-05 2024-09-21T15:00:16.267 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:16.267 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:16.267 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:16.267 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.267 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.267 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:83755 3.b 3:d4eb1ee5:::rbd_data.11368296f791.0000000000000000:40 [sparse-read 1310720~0 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-21T15:00:16.267 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:51.156944+0000", 2024-09-21T15:00:16.267 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 25.036109491000001, 2024-09-21T15:00:16.267 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.009159146, 2024-09-21T15:00:16.268 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:16.268 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:16.268 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:16.268 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:16.268 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:16.268 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 83755 2024-09-21T15:00:16.268 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.268 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:16.268 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.268 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:16.268 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156944+0000", 2024-09-21T15:00:16.268 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:16.268 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.268 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.268 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:16.269 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156944+0000", 2024-09-21T15:00:16.269 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:16.269 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.269 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.269 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:16.269 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156944+0000", 2024-09-21T15:00:16.269 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:16.269 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.269 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.269 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:16.269 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156948+0000", 2024-09-21T15:00:16.269 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5279999999999997e-06 2024-09-21T15:00:16.269 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.269 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.269 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:16.270 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156959+0000", 2024-09-21T15:00:16.270 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0215e-05 2024-09-21T15:00:16.270 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.270 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.270 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:16.270 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156967+0000", 2024-09-21T15:00:16.270 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6119999999999999e-06 2024-09-21T15:00:16.270 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.270 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.270 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:16.270 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.166007+0000", 2024-09-21T15:00:16.270 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0090395260000000009 2024-09-21T15:00:16.271 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.271 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.271 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:16.271 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.166053+0000", 2024-09-21T15:00:16.271 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6281e-05 2024-09-21T15:00:16.271 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.271 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.271 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:16.271 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.166103+0000", 2024-09-21T15:00:16.271 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0510000000000003e-05 2024-09-21T15:00:16.271 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:16.271 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:16.271 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:16.271 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.271 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.272 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:83763 3.b 3:d3640016:::rbd_data.11368296f791.0000000000000006:40 [sparse-read 524288~81920 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-21T15:00:16.272 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:51.157975+0000", 2024-09-21T15:00:16.272 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 25.035078382999998, 2024-09-21T15:00:16.272 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.008327348, 2024-09-21T15:00:16.272 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:16.272 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:16.272 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:16.272 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:16.272 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:16.272 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 83763 2024-09-21T15:00:16.272 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.272 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:16.272 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.272 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:16.272 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.157975+0000", 2024-09-21T15:00:16.272 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:16.273 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.273 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.273 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:16.273 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.157975+0000", 2024-09-21T15:00:16.273 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:16.273 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.273 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.273 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:16.273 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.157970+0000", 2024-09-21T15:00:16.273 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999943 2024-09-21T15:00:16.273 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.273 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.273 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:16.273 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.157985+0000", 2024-09-21T15:00:16.273 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4807000000000001e-05 2024-09-21T15:00:16.274 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.274 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.274 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:16.274 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.158012+0000", 2024-09-21T15:00:16.274 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7600999999999999e-05 2024-09-21T15:00:16.274 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.274 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.274 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:16.274 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.158019+0000", 2024-09-21T15:00:16.274 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9319999999999999e-06 2024-09-21T15:00:16.274 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.274 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.274 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:16.275 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.166118+0000", 2024-09-21T15:00:16.275 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0080992489999999993 2024-09-21T15:00:16.275 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.275 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.275 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:16.275 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.166156+0000", 2024-09-21T15:00:16.275 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7639999999999999e-05 2024-09-21T15:00:16.275 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.275 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.275 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:16.275 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.166303+0000", 2024-09-21T15:00:16.275 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000146799 2024-09-21T15:00:16.275 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:16.275 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:16.276 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:16.276 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.276 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.276 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:83764 3.b 3:d3640016:::rbd_data.11368296f791.0000000000000006:40 [sparse-read 1966080~0 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-21T15:00:16.276 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:51.158026+0000", 2024-09-21T15:00:16.276 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 25.03502795, 2024-09-21T15:00:16.276 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0083785839999999997, 2024-09-21T15:00:16.276 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:16.276 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:16.276 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:16.276 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:16.276 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:16.276 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 83764 2024-09-21T15:00:16.276 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.276 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:16.276 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.277 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:16.277 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.158026+0000", 2024-09-21T15:00:16.277 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:16.277 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.277 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.277 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:16.277 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.158026+0000", 2024-09-21T15:00:16.277 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:16.277 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.277 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.277 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:16.277 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.158025+0000", 2024-09-21T15:00:16.277 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:16.277 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.277 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.278 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:16.278 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.158053+0000", 2024-09-21T15:00:16.278 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7594e-05 2024-09-21T15:00:16.278 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.278 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.278 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:16.278 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.158061+0000", 2024-09-21T15:00:16.278 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2069999999999995e-06 2024-09-21T15:00:16.278 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.278 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.278 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:16.278 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.158067+0000", 2024-09-21T15:00:16.278 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3799999999999999e-06 2024-09-21T15:00:16.279 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.279 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.279 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:16.279 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.166321+0000", 2024-09-21T15:00:16.279 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0082539870000000008 2024-09-21T15:00:16.279 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.279 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.279 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:16.279 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.166360+0000", 2024-09-21T15:00:16.279 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8597999999999999e-05 2024-09-21T15:00:16.279 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.279 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.279 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:16.279 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.166404+0000", 2024-09-21T15:00:16.279 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4378000000000001e-05 2024-09-21T15:00:16.280 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:16.280 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:16.280 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:16.280 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.280 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.280 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:97675 3.b 3:d321acb8:::rbd_data.1136bfc79a70.0000000000000003:head [write 0~4718592 [fadvise_sequential+fadvise_nocache],write 5242880~147456 [fadvise_sequential+fadvise_nocache],write 5455872~147456 [fadvise_sequential+fadvise_nocache],write 5668864~229376 [fadvise_sequential+fadvise_nocache],write 5963776~147456 [fadvise_sequential+fadvise_nocache],write 6176768~229376 [fadvise_sequential+fadvise_nocache],write 6471680~147456 [fadvise_sequential+fadvise_nocache],write 6684672~229376 [fadvise_sequential+fadvise_nocache],write 6979584~147456 [fadvise_sequential+fadvise_nocache],write 7192576~229376 [fadvise_sequential+fadvise_nocache],write 7487488~147456 [fadvise_sequential+fadvise_nocache],write 7700480~147456 [fadvise_sequential+fadvise_nocache],write 7864320~65536 [fadvise_sequential+fadvise_nocache],write 7995392~147456 [fadvise_sequential+fadvise_nocache],write 8208384~147456 [fadvise_sequential+fadvise_nocache],write 8388608~49152 [fadvise_sequential+fadvise_nocache],write 8536064~114688 [fadvise_sequential+fadvise_nocache],write 8749056~114688 [fadvise_sequential+fadvise_nocache],write 8912896~32768 [fadvise_sequential+fadvise_nocache],write 9125888~32768 [fadvise_sequential+fadvise_nocache],write 9338880~32768 [fadvise_sequential+fadvise_nocache],write 9437184~16384 [fadvise_sequential+fadvise_nocache],write 9633792~32768 [fadvise_sequential+fadvise_nocache],write 9846784~32768 [fadvise_sequential+fadvise_nocache],write 9961472~114688 [fadvise_sequential+fadvise_nocache],write 10092544~196608 [fadvise_sequential+fadvise_nocache],write 10305536~278528 [fadvise_sequential+fadvise_nocache],write 10600448~196608 [fadvise_sequential+fadvise_nocache],write 10813440~278528 [fadvise_sequential+fadvise_nocache],write 11108352~196608 [fadvise_sequential+fadvise_nocache],write 11321344~196608 [fadvise_sequential+fadvise_nocache],write 11534336~65536 [fadvise_sequential+fadvise_nocache],write 11632640~98304 [fadvise_sequential+fadvise_nocache],write 11747328~65536 [fadvise_sequential+fadvise_nocache],write 11845632~98304 [fadvise_sequential+fadvise_nocache],write 11960320~65536 [fadvise_sequential+fadvise_nocache],write 12058624~49152 [fadvise_sequential+fadvise_nocache],write 12140544~98304 [fadvise_sequential+fadvise_nocache],write 12255232~65536 [fadvise_sequential+fadvise_nocache],write 12353536~98304 [fadvise_sequential+fadvise_nocache],write 12468224~65536 [fadvise_sequential+fadvise_nocache],write 12566528~16384 [fadvise_sequential+fadvise_nocache],write 12648448~98304 [fadvise_sequential+fadvise_nocache],write 12861440~98304 [fadvise_sequential+fadvise_nocache],write 13074432~32768 [fadvise_sequential+fadvise_nocache],write 13156352~98304 [fadvise_sequential+fadvise_nocache],write 13369344~98304 [fadvise_sequential+fadvise_nocache],write 13582336~49152 [fadvise_sequential+fadvise_nocache],write 13664256~98304 [fadvise_sequential+fadvise_nocache],write 13877248~98304 [fadvise_sequential+fadvise_nocache],write 14090240~65536 [fadvise_sequential+fadvise_nocache],write 15286272~81920 [fadvise_sequential+fadvise_nocache],write 15499264~81920 [fadvise_sequential+fadvise_nocache],write 15712256~16384 [fadvise_sequential+fadvise_nocache],write 15794176~81920 [fadvise_sequential+fadvise_nocache],write 16007168~81920 [fadvise_sequential+fadvise_nocache],write 16220160~32768 [fadvise_sequential+fadvise_nocache],write 16302080~81920 [fadvise_sequential+fadvise_nocache],write 16515072~81920 [fadvise_sequential+fadvise_nocache],write 16728064~49152 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:16.280 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.302412+0000", 2024-09-21T15:00:16.280 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 10.890642283, 2024-09-21T15:00:16.280 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.067380532000000007, 2024-09-21T15:00:16.280 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:16.280 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:16.280 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:16.280 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:16.280 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:16.280 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97675 2024-09-21T15:00:16.280 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.281 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:16.281 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.281 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:16.281 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.302412+0000", 2024-09-21T15:00:16.281 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:16.281 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.281 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.281 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:16.281 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.302412+0000", 2024-09-21T15:00:16.281 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:16.281 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.281 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.281 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:16.281 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.302409+0000", 2024-09-21T15:00:16.281 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-21T15:00:16.281 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.282 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.282 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:16.282 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328550+0000", 2024-09-21T15:00:16.282 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.026140674999999999 2024-09-21T15:00:16.282 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.282 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.282 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:16.282 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328584+0000", 2024-09-21T15:00:16.282 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4165999999999998e-05 2024-09-21T15:00:16.282 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.282 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.282 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:16.282 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328599+0000", 2024-09-21T15:00:16.282 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5505000000000001e-05 2024-09-21T15:00:16.283 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.283 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.283 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:16.283 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328704+0000", 2024-09-21T15:00:16.283 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010487199999999999 2024-09-21T15:00:16.283 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.283 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.283 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:16.283 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.329082+0000", 2024-09-21T15:00:16.283 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00037804899999999998 2024-09-21T15:00:16.283 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.283 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.283 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T15:00:16.283 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.331158+0000", 2024-09-21T15:00:16.284 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0020755449999999998 2024-09-21T15:00:16.284 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.284 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.284 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:16.284 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.360553+0000", 2024-09-21T15:00:16.284 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.029395553000000001 2024-09-21T15:00:16.284 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.284 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.284 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:16.284 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.369496+0000", 2024-09-21T15:00:16.284 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0089428619999999993 2024-09-21T15:00:16.284 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.284 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.284 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:16.284 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.369686+0000", 2024-09-21T15:00:16.285 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019000399999999999 2024-09-21T15:00:16.285 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.285 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.285 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:16.285 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.369792+0000", 2024-09-21T15:00:16.285 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000105775 2024-09-21T15:00:16.285 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:16.285 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:16.285 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:16.285 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.285 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.285 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:97693 3.b 3:d7d7b39b:::rbd_data.1136b9c49b1c.000000000000000c:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:16.285 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.328643+0000", 2024-09-21T15:00:16.285 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 10.864410497, 2024-09-21T15:00:16.285 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.031062136000000001, 2024-09-21T15:00:16.286 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:16.286 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:16.286 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:16.286 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:16.286 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:16.286 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97693 2024-09-21T15:00:16.286 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.286 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:16.286 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.286 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:16.286 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328643+0000", 2024-09-21T15:00:16.286 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:16.286 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.287 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.287 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:16.287 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328643+0000", 2024-09-21T15:00:16.287 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:16.287 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.287 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.287 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:16.287 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328642+0000", 2024-09-21T15:00:16.287 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:00:16.287 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.287 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.287 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:16.287 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328650+0000", 2024-09-21T15:00:16.287 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7770000000000008e-06 2024-09-21T15:00:16.287 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.288 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.288 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:16.288 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328662+0000", 2024-09-21T15:00:16.288 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2026e-05 2024-09-21T15:00:16.288 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.288 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.288 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:16.288 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328671+0000", 2024-09-21T15:00:16.288 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.585e-06 2024-09-21T15:00:16.288 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.288 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.288 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:16.288 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.359498+0000", 2024-09-21T15:00:16.288 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.030827001 2024-09-21T15:00:16.289 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.289 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.289 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:16.289 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.359603+0000", 2024-09-21T15:00:16.289 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010512400000000001 2024-09-21T15:00:16.289 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.289 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.289 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:16.289 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.359705+0000", 2024-09-21T15:00:16.289 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000102327 2024-09-21T15:00:16.289 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:16.289 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:16.289 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:16.289 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.289 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.290 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:97724 3.b 3:d4bf6bcf:::rbd_data.1136b9c49b1c.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:16.290 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.328704+0000", 2024-09-21T15:00:16.290 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 10.864350225000001, 2024-09-21T15:00:16.290 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.031157957, 2024-09-21T15:00:16.290 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:16.290 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:16.290 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:16.290 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:16.290 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:16.290 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97724 2024-09-21T15:00:16.290 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.290 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:16.290 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.290 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:16.290 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328704+0000", 2024-09-21T15:00:16.291 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:16.291 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.291 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.291 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:16.291 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328704+0000", 2024-09-21T15:00:16.291 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:16.291 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.291 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.291 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:16.291 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328703+0000", 2024-09-21T15:00:16.291 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:00:16.291 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.291 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.291 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:16.291 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328708+0000", 2024-09-21T15:00:16.292 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0050000000000004e-06 2024-09-21T15:00:16.292 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.292 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.292 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:16.292 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328719+0000", 2024-09-21T15:00:16.292 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1052e-05 2024-09-21T15:00:16.292 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.292 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.292 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:16.292 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328727+0000", 2024-09-21T15:00:16.292 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4519999999999997e-06 2024-09-21T15:00:16.292 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.292 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.292 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:16.292 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.359737+0000", 2024-09-21T15:00:16.292 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.031009341999999999 2024-09-21T15:00:16.293 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.293 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.293 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:16.293 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.359806+0000", 2024-09-21T15:00:16.293 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9750000000000001e-05 2024-09-21T15:00:16.293 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:16.293 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:16.293 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:16.293 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.359862+0000", 2024-09-21T15:00:16.293 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5084999999999999e-05 2024-09-21T15:00:16.293 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:16.293 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:16.293 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:16.293 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:16.293 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:16.293 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:16.294 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T15:00:16.295 INFO:teuthology.orchestra.run.smithi032.stdout:3547 read 0x21cca5 thru 0x22590a (0x8c66 bytes) 2024-09-21T15:00:16.295 INFO:teuthology.orchestra.run.smithi032.stdout:3552 trunc from 0xe7bbd10 to 0x8237b21 2024-09-21T15:00:16.295 INFO:teuthology.orchestra.run.smithi032.stdout:3554 punch from 0x6953be3 to 0x695f7fe, (0xbc1b bytes) 2024-09-21T15:00:16.295 INFO:teuthology.orchestra.run.smithi032.stdout:3555 write 0x73ea935 thru 0x73f8998 (0xe064 bytes) 2024-09-21T15:00:16.311 INFO:teuthology.orchestra.run.smithi032.stdout:3556 trunc from 0x8237b21 to 0x2455bf7 2024-09-21T15:00:16.348 INFO:teuthology.orchestra.run.smithi032.stdout:3557 write 0x4f715e1 thru 0x4f737e2 (0x2202 bytes) 2024-09-21T15:00:16.349 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:00:16.348+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:00:16.353 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:16.353 INFO:teuthology.orchestra.run.smithi032.stdout:3561 trunc from 0x4f737e3 to 0x53b6ac8 2024-09-21T15:00:16.365 INFO:teuthology.orchestra.run.smithi032.stdout:3562 trunc from 0x53b6ac8 to 0x37fde1e 2024-09-21T15:00:16.377 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 5, 6, 7, 4] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T15:00:16.377 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T15:00:16.377 INFO:tasks.thrashosds.thrasher:inject_pause on osd.4 2024-09-21T15:00:16.377 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-21T15:00:16.378 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-21T15:00:16.378 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok config set filestore_inject_stall 3 2024-09-21T15:00:16.396 INFO:teuthology.orchestra.run.smithi032.stdout:3563 writesame 0x3c97908 thru 0x3c9fa02 data_size 0x7(0x80fb bytes) 2024-09-21T15:00:16.396 INFO:teuthology.orchestra.run.smithi032.stdout:3564 write 0x3f0a1c thru 0x3f2c45 (0x222a bytes) 2024-09-21T15:00:16.416 INFO:teuthology.orchestra.run.smithi032.stdout:3566 trunc from 0x3c9fa03 to 0xd5d4ce0 2024-09-21T15:00:16.425 INFO:teuthology.orchestra.run.smithi032.stdout:3568 read 0x6ed4675 thru 0x6ed84f3 (0x3e7f bytes) 2024-09-21T15:00:16.426 INFO:teuthology.orchestra.run.smithi032.stdout:3569 write 0x75cd40c thru 0x75db32a (0xdf1f bytes) 2024-09-21T15:00:16.430 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:16.430 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:00:16.431 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T15:00:16.431 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:16.439 INFO:teuthology.orchestra.run.smithi032.stdout:3570 write 0x5484d53 thru 0x5492616 (0xd8c4 bytes) 2024-09-21T15:00:16.442 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T15:00:16.449 INFO:teuthology.orchestra.run.smithi032.stdout:3571 write 0x1d3e024 thru 0x1d480fd (0xa0da bytes) 2024-09-21T15:00:16.450 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:00:16.448+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:00:16.459 INFO:teuthology.orchestra.run.smithi032.stdout:3572 write 0xc1787cf thru 0xc17a260 (0x1a92 bytes) 2024-09-21T15:00:16.465 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:16.465 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:16.465 INFO:teuthology.orchestra.run.smithi032.stdout:3575 write 0xac3766 thru 0xac62ee (0x2b89 bytes) 2024-09-21T15:00:16.473 INFO:teuthology.orchestra.run.smithi032.stdout:3576 trunc from 0xd5d4ce0 to 0xa198d99 2024-09-21T15:00:16.492 INFO:teuthology.orchestra.run.smithi032.stdout:3577 read 0x4962765 thru 0x496afa2 (0x883e bytes) 2024-09-21T15:00:16.493 INFO:teuthology.orchestra.run.smithi032.stdout:3578 punch from 0x252238 to 0x26106d, (0xee35 bytes) 2024-09-21T15:00:16.499 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:16.499 INFO:teuthology.orchestra.run.smithi046.stdout: "success": "" 2024-09-21T15:00:16.499 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:16.508 INFO:teuthology.orchestra.run.smithi032.stdout:3579 writesame 0xc76d2ee thru 0xc779910 data_size 0x1(0xc623 bytes) 2024-09-21T15:00:16.508 INFO:teuthology.orchestra.run.smithi032.stdout:3580 punch from 0x9766bee to 0x97721ac, (0xb5be bytes) 2024-09-21T15:00:16.508 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:16.508 INFO:teuthology.orchestra.run.smithi032.stdout:3583 read 0x4a81d7d thru 0x4a887d9 (0x6a5d bytes) 2024-09-21T15:00:16.508 INFO:teuthology.orchestra.run.smithi032.stdout:3584 read 0x7505e97 thru 0x750ffa1 (0xa10b bytes) 2024-09-21T15:00:16.508 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:16.508 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:16.508 INFO:teuthology.orchestra.run.smithi032.stdout:3588 read 0x8cb71fa thru 0x8cc3b1f (0xc926 bytes) 2024-09-21T15:00:16.509 INFO:teuthology.orchestra.run.smithi032.stdout:3590 write 0xb43d243 thru 0xb44400a (0x6dc8 bytes) 2024-09-21T15:00:16.529 INFO:teuthology.orchestra.run.smithi032.stdout:3592 writesame 0xce7c075 thru 0xce880b0 data_size 0x12(0xc03c bytes) 2024-09-21T15:00:16.529 INFO:teuthology.orchestra.run.smithi032.stdout:3593 write 0x5cf2aca thru 0x5d029a1 (0xfed8 bytes) 2024-09-21T15:00:16.544 INFO:teuthology.orchestra.run.smithi032.stdout:3594 trunc from 0xce880b1 to 0x5e847eb 2024-09-21T15:00:16.550 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:00:16.548+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:00:16.552 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T15:00:16.563 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:16.563 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:00:16.563 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T15:00:16.563 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T15:00:16.563 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:16.569 INFO:teuthology.orchestra.run.smithi032.stdout:3595 trunc from 0x5e847eb to 0x3a766a7 2024-09-21T15:00:16.576 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T15:00:16.587 INFO:teuthology.orchestra.run.smithi032.stdout:3596 read 0x22261b5 thru 0x222d34a (0x7196 bytes) 2024-09-21T15:00:16.589 INFO:teuthology.orchestra.run.smithi032.stdout:3597 punch from 0xf1c2cf to 0xf1dbee, (0x191f bytes) 2024-09-21T15:00:16.589 INFO:teuthology.orchestra.run.smithi032.stdout:3598 read 0x3fe862 thru 0x40da01 (0xf1a0 bytes) 2024-09-21T15:00:16.590 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:16.590 INFO:teuthology.orchestra.run.smithi032.stdout:3601 read 0x3764bb0 thru 0x376bbfb (0x704c bytes) 2024-09-21T15:00:16.604 INFO:teuthology.orchestra.run.smithi032.stdout:3603 write 0x628ecb8 thru 0x6297d57 (0x90a0 bytes) 2024-09-21T15:00:16.613 INFO:teuthology.orchestra.run.smithi032.stdout:3604 read 0x4b0744 thru 0x4b272e (0x1feb bytes) 2024-09-21T15:00:16.614 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:16.614 INFO:teuthology.orchestra.run.smithi032.stdout:3606 read 0x5cf1324 thru 0x5cf7ec4 (0x6ba1 bytes) 2024-09-21T15:00:16.614 INFO:teuthology.orchestra.run.smithi032.stdout:3607 read 0x385a894 thru 0x3862f4d (0x86ba bytes) 2024-09-21T15:00:16.627 INFO:teuthology.orchestra.run.smithi032.stdout:3608 writesame 0x7d754ae thru 0x7d7cd7f data_size 0xf(0x78d2 bytes) 2024-09-21T15:00:16.627 INFO:teuthology.orchestra.run.smithi032.stdout:3611 write 0x7946d5d thru 0x7955352 (0xe5f6 bytes) 2024-09-21T15:00:16.641 INFO:teuthology.orchestra.run.smithi032.stdout:3612 write 0x2545d56 thru 0x2554f91 (0xf23c bytes) 2024-09-21T15:00:16.649 INFO:teuthology.orchestra.run.smithi032.stdout:3613 read 0x1d1a7dc thru 0x1d2a5f5 (0xfe1a bytes) 2024-09-21T15:00:16.650 INFO:teuthology.orchestra.run.smithi032.stdout:3614 punch from 0x5b15ccb to 0x5b1bc2c, (0x5f61 bytes) 2024-09-21T15:00:16.650 INFO:teuthology.orchestra.run.smithi032.stdout:3615 punch from 0x6e4818c to 0x6e55c3c, (0xdab0 bytes) 2024-09-21T15:00:16.650 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:00:16.648+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:00:16.667 INFO:teuthology.orchestra.run.smithi032.stdout:3616 write 0xd8305ef thru 0xd836460 (0x5e72 bytes) 2024-09-21T15:00:16.674 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:16.688 INFO:teuthology.orchestra.run.smithi032.stdout:3619 trunc from 0xd836461 to 0x47f4870 2024-09-21T15:00:16.704 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T15:00:16.713 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:16.713 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:16.726 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T15:00:16.727 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T15:00:16.732 INFO:teuthology.orchestra.run.smithi032.stdout:3621 write 0xddd4888 thru 0xdde0f04 (0xc67d bytes) 2024-09-21T15:00:16.744 INFO:teuthology.orchestra.run.smithi032.stdout:3622 writesame 0x3de6d7c thru 0x3ded310 data_size 0x7(0x6595 bytes) 2024-09-21T15:00:16.751 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:00:16.748+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:00:16.751 INFO:teuthology.orchestra.run.smithi032.stdout:3623 read 0x1622a19 thru 0x162ea84 (0xc06c bytes) 2024-09-21T15:00:16.752 INFO:teuthology.orchestra.run.smithi032.stdout:3624 writesame 0x7bcdc1c thru 0x7bda204 data_size 0x1(0xc5e9 bytes) 2024-09-21T15:00:16.753 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T15:00:16.770 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T15:00:16.797 INFO:teuthology.orchestra.run.smithi032.stdout:3626 trunc from 0xdde0f05 to 0xac454e5 2024-09-21T15:00:16.798 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-21T15:00:16.820 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T15:00:16.828 INFO:teuthology.orchestra.run.smithi032.stdout:3627 trunc from 0xac454e5 to 0x35da80e 2024-09-21T15:00:16.829 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:16.829 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:16.839 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T15:00:16.849 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T15:00:16.849 INFO:teuthology.orchestra.run.smithi032.stdout:3629 read 0x2bcaf4d thru 0x2bd325d (0x8311 bytes) 2024-09-21T15:00:16.852 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:00:16.846+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:00:16.862 INFO:teuthology.orchestra.run.smithi032.stdout:3630 write 0x654c9d9 thru 0x654df0b (0x1533 bytes) 2024-09-21T15:00:16.868 INFO:teuthology.orchestra.run.smithi032.stdout:3631 read 0x60b0622 thru 0x60bf027 (0xea06 bytes) 2024-09-21T15:00:16.868 INFO:teuthology.orchestra.run.smithi032.stdout:3633 read 0xbb99e6 thru 0xbc3637 (0x9c52 bytes) 2024-09-21T15:00:16.869 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:16.869 INFO:teuthology.orchestra.run.smithi032.stdout:3636 writesame 0x3fdacb0 thru 0x3fe3ea4 data_size 0x35(0x91f5 bytes) 2024-09-21T15:00:16.878 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T15:00:16.879 INFO:teuthology.orchestra.run.smithi032.stdout:3637 read 0x4e7648c thru 0x4e7846a (0x1fdf bytes) 2024-09-21T15:00:16.880 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:00:16.880 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:00:16.880 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:00:16.880 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:00:16.880 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:00:16.880 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:00:16.880 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:00:16.881 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:00:16.900 INFO:teuthology.orchestra.run.smithi032.stdout:3638 writesame 0xccca85d thru 0xccccb90 data_size 0x6(0x2334 bytes) 2024-09-21T15:00:16.908 INFO:teuthology.orchestra.run.smithi032.stdout:3639 trunc from 0xccccb91 to 0x7925330 2024-09-21T15:00:16.920 INFO:teuthology.orchestra.run.smithi032.stdout:3640 write 0x63879e6 thru 0x6393fee (0xc609 bytes) 2024-09-21T15:00:16.949 INFO:teuthology.orchestra.run.smithi032.stdout:3641 writesame 0xb4914ed thru 0xb49413c data_size 0x8(0x2c50 bytes) 2024-09-21T15:00:16.952 INFO:teuthology.orchestra.run.smithi032.stdout:3642 trunc from 0xb49413d to 0x9c54bc4 2024-09-21T15:00:16.952 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:00:16.952+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:00:16.958 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:16.958 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:16.963 INFO:teuthology.orchestra.run.smithi032.stdout:3643 write 0x6053f4f thru 0x605fc51 (0xbd03 bytes) 2024-09-21T15:00:16.969 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T15:00:16.986 INFO:teuthology.orchestra.run.smithi032.stdout:3644 write 0xad52f4e thru 0xad61a0f (0xeac2 bytes) 2024-09-21T15:00:17.000 INFO:teuthology.orchestra.run.smithi032.stdout:3647 writesame 0x8f42d7 thru 0x8fa281 data_size 0x1(0x5fab bytes) 2024-09-21T15:00:17.031 INFO:teuthology.orchestra.run.smithi032.stdout:3649 trunc from 0xad61a10 to 0x193de87 2024-09-21T15:00:17.054 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:00:17.052+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:00:17.059 INFO:teuthology.orchestra.run.smithi032.stdout:3650 trunc from 0x193de87 to 0x436b10a 2024-09-21T15:00:17.083 INFO:teuthology.orchestra.run.smithi032.stdout:3653 write 0x716d901 thru 0x71779c4 (0xa0c4 bytes) 2024-09-21T15:00:17.092 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:17.092 INFO:teuthology.orchestra.run.smithi032.stdout:3656 punch from 0x3f7ba70 to 0x3f85650, (0x9be0 bytes) 2024-09-21T15:00:17.104 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:17.104 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:17.115 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T15:00:17.115 INFO:teuthology.orchestra.run.smithi032.stdout:3658 writesame 0xc850be4 thru 0xc8598f5 data_size 0x27(0x8d12 bytes) 2024-09-21T15:00:17.116 INFO:teuthology.orchestra.run.smithi032.stdout:3659 trunc from 0xc8598f6 to 0xbc8af5a 2024-09-21T15:00:17.128 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:17.128 INFO:teuthology.orchestra.run.smithi032.stdout:3661 read 0x317bd19 thru 0x318107b (0x5363 bytes) 2024-09-21T15:00:17.128 INFO:teuthology.orchestra.run.smithi032.stdout:3663 write 0x938169a thru 0x938b267 (0x9bce bytes) 2024-09-21T15:00:17.138 INFO:teuthology.orchestra.run.smithi032.stdout:3664 read 0x8acdb4f thru 0x8ad5863 (0x7d15 bytes) 2024-09-21T15:00:17.151 INFO:teuthology.orchestra.run.smithi032.stdout:3666 write 0xc8e06c3 thru 0xc8e2ccf (0x260d bytes) 2024-09-21T15:00:17.155 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:00:17.150+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:00:17.158 INFO:teuthology.orchestra.run.smithi032.stdout:3667 read 0x99ea3e3 thru 0x99f0570 (0x618e bytes) 2024-09-21T15:00:17.158 INFO:teuthology.orchestra.run.smithi032.stdout:3668 writesame 0x9eaaafc thru 0x9eb607b data_size 0x2c(0xb580 bytes) 2024-09-21T15:00:17.168 INFO:teuthology.orchestra.run.smithi032.stdout:3669 read 0x57b197c thru 0x57b4367 (0x29ec bytes) 2024-09-21T15:00:17.168 INFO:teuthology.orchestra.run.smithi032.stdout:3670 write 0x428abef thru 0x428e391 (0x37a3 bytes) 2024-09-21T15:00:17.175 INFO:teuthology.orchestra.run.smithi032.stdout:3671 read 0xbfda26 thru 0xc03267 (0x5842 bytes) 2024-09-21T15:00:17.177 INFO:teuthology.orchestra.run.smithi032.stdout:3672 read 0x811eb24 thru 0x8127e6c (0x9349 bytes) 2024-09-21T15:00:17.177 INFO:teuthology.orchestra.run.smithi032.stdout:3674 read 0xbeba47b thru 0xbeba4c0 (0x46 bytes) 2024-09-21T15:00:17.177 INFO:teuthology.orchestra.run.smithi032.stdout:3675 punch from 0x5242395 to 0x5248074, (0x5cdf bytes) 2024-09-21T15:00:17.177 INFO:teuthology.orchestra.run.smithi032.stdout:3676 punch from 0xb25b82 to 0xb2e0df, (0x855d bytes) 2024-09-21T15:00:17.177 INFO:teuthology.orchestra.run.smithi032.stdout:3678 writesame 0x7711e2a thru 0x7716ef3 data_size 0x1b(0x50ca bytes) 2024-09-21T15:00:17.195 INFO:teuthology.orchestra.run.smithi032.stdout:3679 writesame 0xe576b83 thru 0xe57a5d9 data_size 0x1d(0x3a57 bytes) 2024-09-21T15:00:17.195 INFO:teuthology.orchestra.run.smithi032.stdout:3680 read 0x2bafb2e thru 0x2bbe82a (0xecfd bytes) 2024-09-21T15:00:17.200 INFO:teuthology.orchestra.run.smithi032.stdout:3681 trunc from 0xe57a5da to 0xb668c86 2024-09-21T15:00:17.217 INFO:teuthology.orchestra.run.smithi032.stderr:noscrub is set 2024-09-21T15:00:17.226 INFO:teuthology.orchestra.run.smithi032.stdout:3683 read 0x8c1fa62 thru 0x8c2d02d (0xd5cc bytes) 2024-09-21T15:00:17.231 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:17.231 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:17.241 INFO:teuthology.orchestra.run.smithi032.stdout:3684 trunc from 0xb668c86 to 0x276b82e 2024-09-21T15:00:17.251 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T15:00:17.293 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:00:17.292+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:00:17.297 INFO:teuthology.orchestra.run.smithi032.stdout:3686 write 0xbfa9392 thru 0xbfb64a0 (0xd10f bytes) 2024-09-21T15:00:17.307 INFO:teuthology.orchestra.run.smithi032.stdout:3687 read 0xa750587 thru 0xa755110 (0x4b8a bytes) 2024-09-21T15:00:17.307 INFO:teuthology.orchestra.run.smithi032.stdout:3688 punch from 0xee1c52 to 0xeebcb9, (0xa067 bytes) 2024-09-21T15:00:17.307 INFO:teuthology.orchestra.run.smithi032.stdout:3689 write 0x382091 thru 0x38253f (0x4af bytes) 2024-09-21T15:00:17.331 INFO:teuthology.orchestra.run.smithi032.stdout:3690 trunc from 0xbfb64a1 to 0x1840cd2 2024-09-21T15:00:17.356 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:00:17.352+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:00:17.357 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:17.377 INFO:teuthology.orchestra.run.smithi032.stdout:3692 writesame 0x707fed7 thru 0x708cd1d data_size 0x1(0xce47 bytes) 2024-09-21T15:00:17.377 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:17.377 INFO:teuthology.orchestra.run.smithi032.stdout:3694 read 0x3ca93b thru 0x3d1d48 (0x740e bytes) 2024-09-21T15:00:17.378 INFO:teuthology.orchestra.run.smithi032.stdout:3695 read 0x45ec4f2 thru 0x45f6484 (0x9f93 bytes) 2024-09-21T15:00:17.378 INFO:teuthology.orchestra.run.smithi032.stdout:3697 write 0x69f9ec5 thru 0x6a06807 (0xc943 bytes) 2024-09-21T15:00:17.387 INFO:teuthology.orchestra.run.smithi032.stdout:3698 read 0xe879d2 thru 0xe9340c (0xba3b bytes) 2024-09-21T15:00:17.401 INFO:teuthology.orchestra.run.smithi032.stdout:3699 trunc from 0x708cd1e to 0xac0fd 2024-09-21T15:00:17.413 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:17.413 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:17.425 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T15:00:17.429 INFO:teuthology.orchestra.run.smithi032.stdout:3700 read 0x952bb thru 0xa41d0 (0xef16 bytes) 2024-09-21T15:00:17.457 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:00:17.450+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:00:17.457 INFO:teuthology.orchestra.run.smithi032.stdout:3701 write 0xc369fcf thru 0xc376834 (0xc866 bytes) 2024-09-21T15:00:17.478 INFO:teuthology.orchestra.run.smithi032.stdout:3702 writesame 0xce6db45 thru 0xce7db1d data_size 0x1(0xffd9 bytes) 2024-09-21T15:00:17.478 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:17.478 INFO:teuthology.orchestra.run.smithi032.stdout:3704 write 0x7df37e4 thru 0x7e00135 (0xc952 bytes) 2024-09-21T15:00:17.491 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:17.493 INFO:teuthology.orchestra.run.smithi032.stdout:3707 trunc from 0xce7db1e to 0xbfcc16c 2024-09-21T15:00:17.508 INFO:teuthology.orchestra.run.smithi032.stdout:3708 read 0x3662449 thru 0x3666a0f (0x45c7 bytes) 2024-09-21T15:00:17.508 INFO:teuthology.orchestra.run.smithi032.stdout:3709 read 0xa3a7139 thru 0xa3b0a9a (0x9962 bytes) 2024-09-21T15:00:17.508 INFO:teuthology.orchestra.run.smithi032.stdout:3710 writesame 0x6a8b044 thru 0x6a95d5c data_size 0x1(0xad19 bytes) 2024-09-21T15:00:17.534 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:17.535 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:17.537 INFO:teuthology.orchestra.run.smithi032.stdout:3711 write 0x6d29e3a thru 0x6d356bb (0xb882 bytes) 2024-09-21T15:00:17.545 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T15:00:17.547 INFO:teuthology.orchestra.run.smithi032.stdout:3712 write 0x47be29b thru 0x47cb1f5 (0xcf5b bytes) 2024-09-21T15:00:17.555 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:17.555 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:17.556 INFO:teuthology.orchestra.run.smithi032.stdout:3716 punch from 0x67587b6 to 0x675f06e, (0x68b8 bytes) 2024-09-21T15:00:17.556 INFO:teuthology.orchestra.run.smithi032.stdout:3717 read 0x6ff9d87 thru 0x6ffceb1 (0x312b bytes) 2024-09-21T15:00:17.556 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:17.558 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:00:17.556+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:00:17.563 INFO:teuthology.orchestra.run.smithi032.stdout:3719 trunc from 0xbfcc16c to 0x72c6d26 2024-09-21T15:00:17.593 INFO:teuthology.orchestra.run.smithi032.stdout:3721 trunc from 0x72c6d26 to 0xc45ff77 2024-09-21T15:00:17.604 INFO:teuthology.orchestra.run.smithi032.stdout:3722 read 0x435b266 thru 0x436a4f6 (0xf291 bytes) 2024-09-21T15:00:17.605 INFO:teuthology.orchestra.run.smithi032.stdout:3723 punch from 0x690b10e to 0x6912266, (0x7158 bytes) 2024-09-21T15:00:17.619 INFO:teuthology.orchestra.run.smithi032.stdout:3725 trunc from 0xc45ff77 to 0x3ec9ec2 2024-09-21T15:00:17.654 INFO:teuthology.orchestra.run.smithi032.stdout:3726 write 0x4446271 thru 0x444b9f3 (0x5783 bytes) 2024-09-21T15:00:17.656 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:17.656 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:17.659 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:00:17.656+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:00:17.667 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T15:00:17.677 INFO:teuthology.orchestra.run.smithi032.stdout:3727 writesame 0x62e16ec thru 0x62e9f14 data_size 0x9(0x8829 bytes) 2024-09-21T15:00:17.677 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:17.677 INFO:teuthology.orchestra.run.smithi032.stdout:3729 write 0x257572d thru 0x2579ea7 (0x477b bytes) 2024-09-21T15:00:17.687 INFO:teuthology.orchestra.run.smithi032.stdout:3730 write 0x5405cd1 thru 0x540df94 (0x82c4 bytes) 2024-09-21T15:00:17.708 INFO:teuthology.orchestra.run.smithi032.stdout:3731 write 0x71e09c5 thru 0x71e9210 (0x884c bytes) 2024-09-21T15:00:17.717 INFO:teuthology.orchestra.run.smithi032.stdout:3732 write 0x4bebe72 thru 0x4bf6142 (0xa2d1 bytes) 2024-09-21T15:00:17.744 INFO:teuthology.orchestra.run.smithi032.stdout:3733 write 0xa6cd178 thru 0xa6d8236 (0xb0bf bytes) 2024-09-21T15:00:17.757 INFO:teuthology.orchestra.run.smithi032.stdout:3734 read 0x8c09a1f thru 0x8c14048 (0xa62a bytes) 2024-09-21T15:00:17.757 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:17.760 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:00:17.756+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:00:17.772 INFO:teuthology.orchestra.run.smithi032.stdout:3738 write 0xc6a0c29 thru 0xc6a9f71 (0x9349 bytes) 2024-09-21T15:00:17.780 INFO:teuthology.orchestra.run.smithi032.stdout:3739 write 0xc3ac393 thru 0xc3ae439 (0x20a7 bytes) 2024-09-21T15:00:17.786 INFO:teuthology.orchestra.run.smithi032.stdout:3740 read 0x822814f thru 0x822fe51 (0x7d03 bytes) 2024-09-21T15:00:17.786 INFO:teuthology.orchestra.run.smithi032.stdout:3741 write 0x11574f0 thru 0x11595d6 (0x20e7 bytes) 2024-09-21T15:00:17.789 INFO:teuthology.orchestra.run.smithi032.stdout:3742 read 0x9c52237 thru 0x9c60de0 (0xebaa bytes) 2024-09-21T15:00:17.807 INFO:teuthology.orchestra.run.smithi032.stdout:3743 trunc from 0xc6a9f72 to 0x17c5cef 2024-09-21T15:00:17.809 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:17.809 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:17.821 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T15:00:17.845 INFO:teuthology.orchestra.run.smithi032.stdout:3744 writesame 0x3c39fc3 thru 0x3c49a8b data_size 0x1f(0xfac9 bytes) 2024-09-21T15:00:17.867 INFO:teuthology.orchestra.run.smithi032.stdout:3747 writesame 0xb4b0681 thru 0xb4b336f data_size 0x1(0x2cef bytes) 2024-09-21T15:00:17.867 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:17.867 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:17.868 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:00:17.864+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:00:17.877 INFO:teuthology.orchestra.run.smithi032.stdout:3752 trunc from 0xb4b3370 to 0x5c2d890 2024-09-21T15:00:17.888 INFO:teuthology.orchestra.run.smithi032.stdout:3755 write 0x4117445 thru 0x411ebbf (0x777b bytes) 2024-09-21T15:00:17.897 INFO:teuthology.orchestra.run.smithi032.stdout:3756 clone 27 order 18 su 131072 sc 15 2024-09-21T15:00:17.957 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:17.957 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:17.961 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:00:17.960+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:00:17.967 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T15:00:18.063 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:00:18.064+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:00:18.085 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:18.086 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:18.096 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T15:00:18.164 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:00:18.158+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:00:18.199 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:18.200 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:18.210 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T15:00:18.264 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:00:18.264+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:00:18.313 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:18.313 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:18.323 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T15:00:18.365 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:00:18.358+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:00:18.423 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:18.424 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:18.433 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T15:00:18.466 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:00:18.464+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:00:18.567 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:00:18.562+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:00:18.585 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:18.585 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:18.595 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T15:00:18.668 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:00:18.664+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:00:18.706 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:18.706 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:18.716 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T15:00:18.768 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:00:18.768+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:00:18.821 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:18.821 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:18.831 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T15:00:18.869 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:00:18.868+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:00:18.895 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T15:00:18.941 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:18.941 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:18.950 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T15:00:18.970 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:00:18.968+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:00:19.048 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T15:00:19.062 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:19.062 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:19.070 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:00:19.068+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:00:19.072 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T15:00:19.072 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T15:00:19.094 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T15:00:19.120 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T15:00:19.146 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-21T15:00:19.169 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T15:00:19.171 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:00:19.166+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:00:19.179 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:19.179 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:19.189 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T15:00:19.193 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T15:00:19.217 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T15:00:19.220 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:00:19.220 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:00:19.220 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:00:19.220 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:00:19.220 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:00:19.220 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:00:19.220 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:00:19.230 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T15:00:19.271 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:00:19.266+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:00:19.366 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:19.367 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:00:19.367 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T15:00:19.367 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T15:00:19.367 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:19.372 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:00:19.368+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:00:19.376 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T15:00:19.473 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:00:19.466+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:00:19.484 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:19.484 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T15:00:19.484 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T15:00:19.484 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T15:00:19.484 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.484 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31476 3.c 3:35718b6b:::rbd_data.11367721c19.0000000000000014:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T15:00:19.484 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.505975+0000", 2024-09-21T15:00:19.484 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 80.974292836999993, 2024-09-21T15:00:19.484 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99515995899999998, 2024-09-21T15:00:19.485 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:19.485 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:19.485 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:19.485 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:19.485 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:19.485 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31476 2024-09-21T15:00:19.485 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.485 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:19.485 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.485 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:19.485 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.505975+0000", 2024-09-21T15:00:19.485 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:19.485 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.485 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.486 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:19.486 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.505975+0000", 2024-09-21T15:00:19.486 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:19.486 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.486 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.486 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:19.486 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.505971+0000", 2024-09-21T15:00:19.486 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999967 2024-09-21T15:00:19.486 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.486 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.486 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:19.486 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.505989+0000", 2024-09-21T15:00:19.486 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7985000000000001e-05 2024-09-21T15:00:19.486 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.486 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.487 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:19.487 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506050+0000", 2024-09-21T15:00:19.487 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0659000000000003e-05 2024-09-21T15:00:19.487 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.487 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.487 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:19.487 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506074+0000", 2024-09-21T15:00:19.487 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4156e-05 2024-09-21T15:00:19.487 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.487 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.487 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:19.487 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506225+0000", 2024-09-21T15:00:19.487 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015097 2024-09-21T15:00:19.487 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.488 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.488 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:00:19.488 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506256+0000", 2024-09-21T15:00:19.488 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0505999999999999e-05 2024-09-21T15:00:19.488 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.488 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.488 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:19.488 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.497974+0000", 2024-09-21T15:00:19.488 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99171825700000005 2024-09-21T15:00:19.488 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.488 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.488 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for missing object", 2024-09-21T15:00:19.488 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.498449+0000", 2024-09-21T15:00:19.488 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00047506399999999998 2024-09-21T15:00:19.489 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.489 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.489 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:19.489 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.501000+0000", 2024-09-21T15:00:19.489 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0025506919999999998 2024-09-21T15:00:19.489 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.489 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.489 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:19.489 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.501135+0000", 2024-09-21T15:00:19.489 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013524000000000001 2024-09-21T15:00:19.489 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:19.489 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:19.489 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:19.489 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.489 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.490 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31480 3.c 3:36ccd808:::rbd_data.11367721c19.0000000000000018:head [sparse-read 0~14821] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T15:00:19.490 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.506116+0000", 2024-09-21T15:00:19.490 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 80.974151493999997, 2024-09-21T15:00:19.490 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99316033599999998, 2024-09-21T15:00:19.490 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:19.490 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:19.490 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:19.490 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:19.490 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:19.490 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31480 2024-09-21T15:00:19.490 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.490 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:19.490 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.490 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:19.491 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506116+0000", 2024-09-21T15:00:19.491 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:19.491 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.491 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.491 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:19.491 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506116+0000", 2024-09-21T15:00:19.491 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:19.491 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.491 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.491 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:19.491 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506115+0000", 2024-09-21T15:00:19.491 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:00:19.491 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.491 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.492 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:19.492 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506124+0000", 2024-09-21T15:00:19.492 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0659999999999993e-06 2024-09-21T15:00:19.492 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.492 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.492 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:19.492 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506145+0000", 2024-09-21T15:00:19.492 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1447e-05 2024-09-21T15:00:19.492 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.492 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.492 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:19.492 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506161+0000", 2024-09-21T15:00:19.492 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5209e-05 2024-09-21T15:00:19.492 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.492 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.493 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:19.493 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506337+0000", 2024-09-21T15:00:19.493 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017642999999999999 2024-09-21T15:00:19.493 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.493 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.493 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:00:19.493 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506375+0000", 2024-09-21T15:00:19.493 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7536000000000001e-05 2024-09-21T15:00:19.493 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.493 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.493 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:19.493 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.498815+0000", 2024-09-21T15:00:19.493 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99244067999999996 2024-09-21T15:00:19.493 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.493 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.494 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:19.494 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.498950+0000", 2024-09-21T15:00:19.494 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013436700000000001 2024-09-21T15:00:19.494 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.494 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.494 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:19.494 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499277+0000", 2024-09-21T15:00:19.494 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00032696800000000002 2024-09-21T15:00:19.494 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:19.494 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:19.494 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:19.494 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.494 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.494 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31513 3.c 3:36de8523:::rbd_data.11367721c19.0000000000000039:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T15:00:19.495 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.509071+0000", 2024-09-21T15:00:19.495 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 80.971197294000007, 2024-09-21T15:00:19.495 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99038225099999999, 2024-09-21T15:00:19.495 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:19.495 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:19.495 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:19.495 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:19.495 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:19.495 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31513 2024-09-21T15:00:19.495 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.495 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:19.495 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.495 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:19.495 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509071+0000", 2024-09-21T15:00:19.496 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:19.496 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.496 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.496 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:19.496 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509071+0000", 2024-09-21T15:00:19.496 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:19.496 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.496 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.496 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:19.496 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509068+0000", 2024-09-21T15:00:19.496 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-21T15:00:19.496 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.496 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.497 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:19.497 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509082+0000", 2024-09-21T15:00:19.497 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3545e-05 2024-09-21T15:00:19.497 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.497 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.497 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:19.497 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509116+0000", 2024-09-21T15:00:19.497 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3942000000000001e-05 2024-09-21T15:00:19.497 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.497 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.497 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:19.497 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509134+0000", 2024-09-21T15:00:19.497 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8839999999999999e-05 2024-09-21T15:00:19.497 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.498 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.498 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:19.498 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509318+0000", 2024-09-21T15:00:19.498 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018326799999999999 2024-09-21T15:00:19.498 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.498 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.498 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:00:19.498 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509346+0000", 2024-09-21T15:00:19.498 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8019000000000001e-05 2024-09-21T15:00:19.498 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.498 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.498 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:19.498 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499330+0000", 2024-09-21T15:00:19.499 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.98998399299999995 2024-09-21T15:00:19.499 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.499 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.499 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:19.499 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499453+0000", 2024-09-21T15:00:19.499 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000123115 2024-09-21T15:00:19.499 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:19.499 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:19.499 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:19.499 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.499 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.499 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31515 3.c 3:35b36ddd:::rbd_data.11367721c19.000000000000003b:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T15:00:19.499 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.509177+0000", 2024-09-21T15:00:19.499 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 80.971090868000005, 2024-09-21T15:00:19.499 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99044870600000001, 2024-09-21T15:00:19.500 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:19.500 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:19.500 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:19.500 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:19.500 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:19.500 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31515 2024-09-21T15:00:19.500 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.500 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:19.500 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.500 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:19.500 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509177+0000", 2024-09-21T15:00:19.500 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:19.500 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.500 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.501 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:19.501 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509177+0000", 2024-09-21T15:00:19.501 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:19.501 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.501 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.501 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:19.501 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509176+0000", 2024-09-21T15:00:19.501 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:00:19.501 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.501 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.501 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:19.501 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509183+0000", 2024-09-21T15:00:19.501 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6720000000000004e-06 2024-09-21T15:00:19.501 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.502 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.502 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:19.502 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509269+0000", 2024-09-21T15:00:19.502 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5881e-05 2024-09-21T15:00:19.502 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.502 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.502 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:19.502 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509303+0000", 2024-09-21T15:00:19.502 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3367000000000001e-05 2024-09-21T15:00:19.502 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.502 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.502 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:19.502 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509425+0000", 2024-09-21T15:00:19.503 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000122199 2024-09-21T15:00:19.503 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.503 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.503 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:00:19.503 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509448+0000", 2024-09-21T15:00:19.503 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2688999999999999e-05 2024-09-21T15:00:19.503 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.503 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.503 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:19.503 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499501+0000", 2024-09-21T15:00:19.503 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99005306999999998 2024-09-21T15:00:19.503 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.503 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.504 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:19.504 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499626+0000", 2024-09-21T15:00:19.504 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012501400000000001 2024-09-21T15:00:19.504 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:19.504 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:19.504 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:19.504 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.504 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.504 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31531 3.c 3:314f0287:::rbd_data.11367721c19.000000000000004b:head [sparse-read 0~7106] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T15:00:19.504 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.509329+0000", 2024-09-21T15:00:19.504 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 80.970938848000003, 2024-09-21T15:00:19.504 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99103085300000004, 2024-09-21T15:00:19.504 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:19.504 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:19.505 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:19.505 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:19.505 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:19.505 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31531 2024-09-21T15:00:19.505 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.505 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:19.505 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.505 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:19.505 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509329+0000", 2024-09-21T15:00:19.505 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:19.505 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.505 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.505 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:19.506 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509329+0000", 2024-09-21T15:00:19.506 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:19.506 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.506 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.506 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:19.506 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509328+0000", 2024-09-21T15:00:19.506 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:00:19.506 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.506 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.506 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:19.506 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509336+0000", 2024-09-21T15:00:19.506 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3170000000000002e-06 2024-09-21T15:00:19.506 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.507 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.507 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:19.507 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509353+0000", 2024-09-21T15:00:19.507 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7214000000000002e-05 2024-09-21T15:00:19.507 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.507 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.507 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:19.507 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509366+0000", 2024-09-21T15:00:19.507 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.26e-05 2024-09-21T15:00:19.507 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.507 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.507 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:19.507 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509523+0000", 2024-09-21T15:00:19.507 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015708000000000001 2024-09-21T15:00:19.507 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.508 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.508 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:00:19.508 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509544+0000", 2024-09-21T15:00:19.508 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0783000000000001e-05 2024-09-21T15:00:19.508 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.508 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.508 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:19.508 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499727+0000", 2024-09-21T15:00:19.508 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99018345200000002 2024-09-21T15:00:19.508 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.508 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.508 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:19.508 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499949+0000", 2024-09-21T15:00:19.508 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000221358 2024-09-21T15:00:19.509 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.509 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.509 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:19.509 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.500360+0000", 2024-09-21T15:00:19.509 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00041130700000000002 2024-09-21T15:00:19.509 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:19.509 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:19.509 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:19.509 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.509 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.509 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:61580 3.b 3:d3369e4f:::rbd_data.1136862f872f.0000000000000010:head [write 0~557056 [fadvise_sequential+fadvise_nocache],zero 557056~16384,write 573440~49152 [fadvise_sequential+fadvise_nocache],zero 622592~16384,write 638976~49152 [fadvise_sequential+fadvise_nocache],zero 688128~16384,write 704512~49152 [fadvise_sequential+fadvise_nocache],zero 753664~16384,write 770048~49152 [fadvise_sequential+fadvise_nocache],zero 819200~16384,write 835584~49152 [fadvise_sequential+fadvise_nocache],zero 884736~16384,write 901120~49152 [fadvise_sequential+fadvise_nocache],zero 950272~16384,write 966656~49152 [fadvise_sequential+fadvise_nocache],zero 1015808~16384,write 1032192~16384 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-21T15:00:19.509 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:19.231716+0000", 2024-09-21T15:00:19.509 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 60.248551685999999, 2024-09-21T15:00:19.509 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0096097690000000006, 2024-09-21T15:00:19.510 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:19.510 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:19.510 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:19.510 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:19.510 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:19.510 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 61580 2024-09-21T15:00:19.510 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.510 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:19.510 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.510 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:19.510 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.231716+0000", 2024-09-21T15:00:19.510 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:19.510 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.510 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.511 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:19.511 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.231716+0000", 2024-09-21T15:00:19.511 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:19.511 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.511 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.511 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:19.511 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.231715+0000", 2024-09-21T15:00:19.511 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:00:19.511 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.511 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.511 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:19.511 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.235290+0000", 2024-09-21T15:00:19.511 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0035750399999999998 2024-09-21T15:00:19.511 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.512 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.512 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:19.512 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.235310+0000", 2024-09-21T15:00:19.512 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0120999999999999e-05 2024-09-21T15:00:19.512 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.512 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.512 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:19.512 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.235319+0000", 2024-09-21T15:00:19.512 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2420000000000006e-06 2024-09-21T15:00:19.512 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.512 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.512 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:19.512 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.235416+0000", 2024-09-21T15:00:19.512 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7204e-05 2024-09-21T15:00:19.512 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.513 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.513 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:19.513 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.235775+0000", 2024-09-21T15:00:19.513 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000358883 2024-09-21T15:00:19.513 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.513 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.513 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T15:00:19.513 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.236560+0000", 2024-09-21T15:00:19.513 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000784463 2024-09-21T15:00:19.513 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.513 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.513 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:19.513 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.241119+0000", 2024-09-21T15:00:19.513 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0045590589999999999 2024-09-21T15:00:19.514 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.514 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.514 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:19.514 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.241210+0000", 2024-09-21T15:00:19.514 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1617000000000001e-05 2024-09-21T15:00:19.514 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.514 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.514 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:19.514 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.241285+0000", 2024-09-21T15:00:19.514 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4470999999999999e-05 2024-09-21T15:00:19.514 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.514 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.514 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:19.514 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.241326+0000", 2024-09-21T15:00:19.515 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0967000000000003e-05 2024-09-21T15:00:19.515 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:19.515 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:19.515 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:19.515 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.515 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.515 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:73699 3.b 3:d293b54a:::rbd_data.1136f4939602.000000000000000a:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-21T15:00:19.515 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:41.971359+0000", 2024-09-21T15:00:19.515 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 37.508909129999999, 2024-09-21T15:00:19.515 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0082840980000000002, 2024-09-21T15:00:19.515 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:19.515 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:19.515 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:19.515 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:19.515 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:19.516 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 73699 2024-09-21T15:00:19.516 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.516 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:19.516 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.516 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:19.516 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.971359+0000", 2024-09-21T15:00:19.516 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:19.516 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.516 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.516 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:19.516 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.971359+0000", 2024-09-21T15:00:19.516 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:19.516 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.516 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.516 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:19.516 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.971356+0000", 2024-09-21T15:00:19.517 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-21T15:00:19.517 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.517 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.517 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:19.517 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.971370+0000", 2024-09-21T15:00:19.517 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3349999999999999e-05 2024-09-21T15:00:19.517 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.517 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.517 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:19.517 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.971393+0000", 2024-09-21T15:00:19.517 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3799999999999999e-05 2024-09-21T15:00:19.517 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.517 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.517 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:19.517 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.971405+0000", 2024-09-21T15:00:19.517 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1569000000000001e-05 2024-09-21T15:00:19.518 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.518 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.518 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:19.518 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.971510+0000", 2024-09-21T15:00:19.518 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010456899999999999 2024-09-21T15:00:19.518 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.518 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.518 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:19.518 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.971752+0000", 2024-09-21T15:00:19.518 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00024250300000000001 2024-09-21T15:00:19.518 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.518 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.518 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T15:00:19.518 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.973012+0000", 2024-09-21T15:00:19.518 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0012595429999999999 2024-09-21T15:00:19.519 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.519 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.519 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:19.519 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.979357+0000", 2024-09-21T15:00:19.519 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0063454469999999997 2024-09-21T15:00:19.519 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.519 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.519 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:19.519 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.979467+0000", 2024-09-21T15:00:19.519 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000109623 2024-09-21T15:00:19.519 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.519 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.519 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:19.519 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.979585+0000", 2024-09-21T15:00:19.519 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000118044 2024-09-21T15:00:19.520 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.520 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.520 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:19.520 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.979643+0000", 2024-09-21T15:00:19.520 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8069999999999998e-05 2024-09-21T15:00:19.520 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:19.520 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:19.520 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:19.520 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.520 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.520 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:82545 3.b 3:d4eb1ee5:::rbd_data.11368296f791.0000000000000000:head [write 0~671744 [fadvise_sequential+fadvise_nocache],write 704512~65536 [fadvise_sequential+fadvise_nocache],write 786432~49152 [fadvise_sequential+fadvise_nocache],write 868352~49152 [fadvise_sequential+fadvise_nocache],write 933888~65536 [fadvise_sequential+fadvise_nocache],write 1032192~32768 [fadvise_sequential+fadvise_nocache],write 1097728~65536 [fadvise_sequential+fadvise_nocache],write 1179648~49152 [fadvise_sequential+fadvise_nocache],write 1261568~49152 [fadvise_sequential+fadvise_nocache],write 1327104~65536 [fadvise_sequential+fadvise_nocache],write 1425408~32768 [fadvise_sequential+fadvise_nocache],write 1490944~65536 [fadvise_sequential+fadvise_nocache],write 1572864~8192 [fadvise_sequential+fadvise_nocache],write 1630208~49152 [fadvise_sequential+fadvise_nocache],write 1703936~40960 [fadvise_sequential+fadvise_nocache],write 1794048~40960 [fadvise_sequential+fadvise_nocache],write 1859584~49152 [fadvise_sequential+fadvise_nocache],write 1957888~8192 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:00:19.520 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:47.416089+0000", 2024-09-21T15:00:19.520 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 32.064179123000002, 2024-09-21T15:00:19.520 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011896455, 2024-09-21T15:00:19.520 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:19.521 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:19.521 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:19.521 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:19.521 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:19.521 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82545 2024-09-21T15:00:19.521 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.521 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:19.521 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.521 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:19.521 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.416089+0000", 2024-09-21T15:00:19.521 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:19.521 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.521 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.521 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:19.521 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.416089+0000", 2024-09-21T15:00:19.522 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:19.522 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.522 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.522 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:19.522 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.416087+0000", 2024-09-21T15:00:19.522 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:00:19.522 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.522 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.522 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:19.522 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.419643+0000", 2024-09-21T15:00:19.522 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0035555550000000002 2024-09-21T15:00:19.522 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.522 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.522 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:19.523 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.419663+0000", 2024-09-21T15:00:19.523 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0202000000000001e-05 2024-09-21T15:00:19.523 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.523 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.523 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:19.523 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.419676+0000", 2024-09-21T15:00:19.523 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3207000000000001e-05 2024-09-21T15:00:19.523 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.523 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.523 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:19.523 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.419759+0000", 2024-09-21T15:00:19.523 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3521000000000002e-05 2024-09-21T15:00:19.523 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.523 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.524 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op must wait for map", 2024-09-21T15:00:19.524 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.419768+0000", 2024-09-21T15:00:19.524 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4479999999999994e-06 2024-09-21T15:00:19.524 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.524 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.524 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:19.524 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.422171+0000", 2024-09-21T15:00:19.524 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0024035469999999998 2024-09-21T15:00:19.524 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.524 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.524 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:19.524 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.422350+0000", 2024-09-21T15:00:19.524 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000179001 2024-09-21T15:00:19.524 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.524 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.525 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T15:00:19.525 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.422710+0000", 2024-09-21T15:00:19.525 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000359878 2024-09-21T15:00:19.525 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.525 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.525 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:19.525 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.426676+0000", 2024-09-21T15:00:19.525 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0039655790000000003 2024-09-21T15:00:19.525 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.525 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.525 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:19.525 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.427884+0000", 2024-09-21T15:00:19.525 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001208207 2024-09-21T15:00:19.525 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.525 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.525 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:19.526 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.427953+0000", 2024-09-21T15:00:19.526 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8533000000000003e-05 2024-09-21T15:00:19.526 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.526 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.526 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:19.526 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.427985+0000", 2024-09-21T15:00:19.526 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2481999999999998e-05 2024-09-21T15:00:19.526 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:19.526 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:19.526 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:19.526 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.526 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.526 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:83735 3.b 3:d3640016:::rbd_data.11368296f791.0000000000000006:40 [sparse-read 393216~131072 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-21T15:00:19.526 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:51.156767+0000", 2024-09-21T15:00:19.527 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 28.323500500000002, 2024-09-21T15:00:19.527 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0082826410000000003, 2024-09-21T15:00:19.527 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:19.527 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:19.527 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:19.527 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:19.527 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:19.527 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 83735 2024-09-21T15:00:19.527 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.527 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:19.527 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.527 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:19.527 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156767+0000", 2024-09-21T15:00:19.527 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:19.527 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.527 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.528 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:19.528 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156767+0000", 2024-09-21T15:00:19.528 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:19.528 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.528 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.528 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:19.528 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156767+0000", 2024-09-21T15:00:19.528 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:19.528 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.528 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.528 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:19.528 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156771+0000", 2024-09-21T15:00:19.528 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0940000000000001e-06 2024-09-21T15:00:19.528 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.528 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.529 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:19.529 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156781+0000", 2024-09-21T15:00:19.529 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0267e-05 2024-09-21T15:00:19.529 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.529 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.529 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:19.529 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156789+0000", 2024-09-21T15:00:19.529 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7659999999999998e-06 2024-09-21T15:00:19.529 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.529 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.529 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:19.529 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.164742+0000", 2024-09-21T15:00:19.529 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0079534519999999997 2024-09-21T15:00:19.529 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.529 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.529 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:19.530 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.164820+0000", 2024-09-21T15:00:19.530 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7927000000000001e-05 2024-09-21T15:00:19.530 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.530 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.530 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:19.530 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165050+0000", 2024-09-21T15:00:19.530 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000229731 2024-09-21T15:00:19.530 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:19.530 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:19.530 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:19.530 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.530 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.530 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:83736 3.b 3:d3640016:::rbd_data.11368296f791.0000000000000006:40 [sparse-read 1835008~0 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-21T15:00:19.530 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:51.156796+0000", 2024-09-21T15:00:19.531 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 28.323471497, 2024-09-21T15:00:19.531 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0085139110000000007, 2024-09-21T15:00:19.531 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:19.531 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:19.531 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:19.531 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:19.531 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:19.531 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 83736 2024-09-21T15:00:19.531 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.531 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:19.531 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.531 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:19.531 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156796+0000", 2024-09-21T15:00:19.531 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:19.531 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.532 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.532 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:19.532 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156796+0000", 2024-09-21T15:00:19.532 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:19.532 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.532 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.532 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:19.532 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156796+0000", 2024-09-21T15:00:19.532 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:19.532 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.532 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.532 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:19.532 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156800+0000", 2024-09-21T15:00:19.532 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.34e-06 2024-09-21T15:00:19.532 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.532 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.533 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:19.533 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156813+0000", 2024-09-21T15:00:19.533 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2532e-05 2024-09-21T15:00:19.533 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.533 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.533 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:19.533 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156821+0000", 2024-09-21T15:00:19.533 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2069999999999995e-06 2024-09-21T15:00:19.533 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.533 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.533 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:19.533 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165100+0000", 2024-09-21T15:00:19.533 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0082788730000000008 2024-09-21T15:00:19.533 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.533 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.534 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:19.534 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165192+0000", 2024-09-21T15:00:19.534 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2095999999999994e-05 2024-09-21T15:00:19.534 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.534 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.534 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:19.534 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165310+0000", 2024-09-21T15:00:19.534 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000118461 2024-09-21T15:00:19.534 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:19.534 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:19.534 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:19.534 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.534 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.535 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:83739 3.b 3:d4eb1ee5:::rbd_data.11368296f791.0000000000000000:40 [sparse-read 524288~131072 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-21T15:00:19.535 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:51.156827+0000", 2024-09-21T15:00:19.535 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 28.323440414, 2024-09-21T15:00:19.535 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0087319900000000002, 2024-09-21T15:00:19.535 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:19.535 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:19.535 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:19.535 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:19.535 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:19.535 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 83739 2024-09-21T15:00:19.535 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.535 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:19.535 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.535 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:19.535 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156827+0000", 2024-09-21T15:00:19.536 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:19.536 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.536 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.536 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:19.536 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156827+0000", 2024-09-21T15:00:19.536 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:19.536 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.536 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.536 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:19.536 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156827+0000", 2024-09-21T15:00:19.536 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:19.536 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.536 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.536 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:19.536 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156831+0000", 2024-09-21T15:00:19.537 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9940000000000002e-06 2024-09-21T15:00:19.537 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.537 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.537 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:19.537 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156841+0000", 2024-09-21T15:00:19.537 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6760000000000002e-06 2024-09-21T15:00:19.537 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.537 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.537 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:19.537 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156853+0000", 2024-09-21T15:00:19.537 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1972e-05 2024-09-21T15:00:19.537 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.537 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.537 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:19.538 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165334+0000", 2024-09-21T15:00:19.538 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0084816939999999997 2024-09-21T15:00:19.538 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.538 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.538 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:19.538 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165375+0000", 2024-09-21T15:00:19.538 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0911000000000002e-05 2024-09-21T15:00:19.538 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.538 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.538 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:19.538 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165559+0000", 2024-09-21T15:00:19.538 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018429300000000001 2024-09-21T15:00:19.538 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:19.538 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:19.539 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:19.539 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.539 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.539 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:83740 3.b 3:d4eb1ee5:::rbd_data.11368296f791.0000000000000000:40 [sparse-read 1966080~0 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-21T15:00:19.539 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:51.156859+0000", 2024-09-21T15:00:19.539 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 28.323408424, 2024-09-21T15:00:19.539 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0088540089999999995, 2024-09-21T15:00:19.539 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:19.539 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:19.539 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:19.539 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:19.539 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:19.539 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 83740 2024-09-21T15:00:19.540 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.540 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:19.540 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.540 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:19.540 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156859+0000", 2024-09-21T15:00:19.540 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:19.540 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.540 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.540 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:19.540 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156859+0000", 2024-09-21T15:00:19.540 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:19.540 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.540 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.540 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:19.541 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156859+0000", 2024-09-21T15:00:19.541 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:19.541 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.541 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.541 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:19.541 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156863+0000", 2024-09-21T15:00:19.541 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.844e-06 2024-09-21T15:00:19.541 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.541 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.541 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:19.541 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156873+0000", 2024-09-21T15:00:19.541 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0069e-05 2024-09-21T15:00:19.541 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.541 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.541 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:19.542 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156880+0000", 2024-09-21T15:00:19.542 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4240000000000004e-06 2024-09-21T15:00:19.542 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.542 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.542 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:19.542 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165590+0000", 2024-09-21T15:00:19.542 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0087098100000000001 2024-09-21T15:00:19.542 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.542 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.542 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:19.542 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165656+0000", 2024-09-21T15:00:19.542 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5541000000000001e-05 2024-09-21T15:00:19.542 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.542 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.542 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:19.542 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165713+0000", 2024-09-21T15:00:19.543 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7865000000000002e-05 2024-09-21T15:00:19.543 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:19.543 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:19.543 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:19.543 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.543 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.543 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:83751 3.b 3:d3640016:::rbd_data.11368296f791.0000000000000006:40 [sparse-read 1179648~0 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-21T15:00:19.543 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:51.156888+0000", 2024-09-21T15:00:19.543 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 28.323380162999999, 2024-09-21T15:00:19.543 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0090039539999999998, 2024-09-21T15:00:19.543 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:19.543 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:19.543 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:19.543 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:19.544 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:19.544 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 83751 2024-09-21T15:00:19.544 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.544 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:19.544 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.544 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:19.544 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156888+0000", 2024-09-21T15:00:19.544 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:19.544 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.544 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.544 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:19.544 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156888+0000", 2024-09-21T15:00:19.544 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:19.544 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.544 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.545 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:19.545 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156887+0000", 2024-09-21T15:00:19.545 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:19.545 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.545 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.545 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:19.545 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156891+0000", 2024-09-21T15:00:19.545 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.518e-06 2024-09-21T15:00:19.545 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.545 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.545 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:19.545 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156900+0000", 2024-09-21T15:00:19.545 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1759999999999999e-06 2024-09-21T15:00:19.545 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.546 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.546 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:19.546 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156908+0000", 2024-09-21T15:00:19.546 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8439999999999998e-06 2024-09-21T15:00:19.546 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.546 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.546 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:19.546 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165738+0000", 2024-09-21T15:00:19.546 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0088305689999999999 2024-09-21T15:00:19.546 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.546 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.546 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:19.546 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165804+0000", 2024-09-21T15:00:19.546 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5535000000000002e-05 2024-09-21T15:00:19.547 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.547 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.547 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:19.547 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165892+0000", 2024-09-21T15:00:19.547 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7860000000000002e-05 2024-09-21T15:00:19.547 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:19.547 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:19.547 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:19.547 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.547 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.547 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:83752 3.b 3:d3640016:::rbd_data.11368296f791.0000000000000006:40 [sparse-read 2621440~0 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-21T15:00:19.547 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:51.156915+0000", 2024-09-21T15:00:19.547 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 28.323352616000001, 2024-09-21T15:00:19.547 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0090768180000000004, 2024-09-21T15:00:19.548 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:19.548 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:19.548 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:19.548 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:19.548 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:19.548 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 83752 2024-09-21T15:00:19.548 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.548 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:19.548 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.548 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:19.548 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156915+0000", 2024-09-21T15:00:19.548 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:19.548 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.548 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.549 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:19.549 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156915+0000", 2024-09-21T15:00:19.549 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:19.549 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.549 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.549 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:19.549 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156915+0000", 2024-09-21T15:00:19.549 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:19.549 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.549 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.549 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:19.549 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156919+0000", 2024-09-21T15:00:19.549 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2679999999999996e-06 2024-09-21T15:00:19.549 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.549 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.550 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:19.550 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156929+0000", 2024-09-21T15:00:19.550 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.031e-05 2024-09-21T15:00:19.550 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.550 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.550 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:19.550 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156937+0000", 2024-09-21T15:00:19.550 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8820000000000001e-06 2024-09-21T15:00:19.550 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.550 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.550 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:19.550 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165909+0000", 2024-09-21T15:00:19.550 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0089722989999999996 2024-09-21T15:00:19.550 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.551 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.551 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:19.551 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165948+0000", 2024-09-21T15:00:19.551 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8114999999999997e-05 2024-09-21T15:00:19.551 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.551 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.551 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:19.551 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165992+0000", 2024-09-21T15:00:19.551 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4503999999999999e-05 2024-09-21T15:00:19.551 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:19.551 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:19.551 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:19.551 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.551 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.551 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:83755 3.b 3:d4eb1ee5:::rbd_data.11368296f791.0000000000000000:40 [sparse-read 1310720~0 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-21T15:00:19.552 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:51.156944+0000", 2024-09-21T15:00:19.552 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 28.323323542000001, 2024-09-21T15:00:19.552 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.009159146, 2024-09-21T15:00:19.552 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:19.552 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:19.552 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:19.552 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:19.552 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:19.552 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 83755 2024-09-21T15:00:19.552 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.552 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:19.552 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.552 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:19.552 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156944+0000", 2024-09-21T15:00:19.552 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:19.553 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.553 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.553 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:19.553 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156944+0000", 2024-09-21T15:00:19.553 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:19.553 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.553 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.553 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:19.553 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156944+0000", 2024-09-21T15:00:19.553 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:19.553 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.553 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.553 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:19.553 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156948+0000", 2024-09-21T15:00:19.554 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5279999999999997e-06 2024-09-21T15:00:19.554 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.554 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.554 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:19.554 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156959+0000", 2024-09-21T15:00:19.554 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0215e-05 2024-09-21T15:00:19.554 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.554 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.554 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:19.554 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156967+0000", 2024-09-21T15:00:19.554 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6119999999999999e-06 2024-09-21T15:00:19.554 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.554 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.554 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:19.555 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.166007+0000", 2024-09-21T15:00:19.555 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0090395260000000009 2024-09-21T15:00:19.555 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.555 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.555 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:19.555 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.166053+0000", 2024-09-21T15:00:19.555 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6281e-05 2024-09-21T15:00:19.555 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.555 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.555 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:19.555 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.166103+0000", 2024-09-21T15:00:19.555 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0510000000000003e-05 2024-09-21T15:00:19.555 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:19.555 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:19.555 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:19.556 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.556 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.556 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:83763 3.b 3:d3640016:::rbd_data.11368296f791.0000000000000006:40 [sparse-read 524288~81920 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-21T15:00:19.556 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:51.157975+0000", 2024-09-21T15:00:19.556 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 28.322292434000001, 2024-09-21T15:00:19.556 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.008327348, 2024-09-21T15:00:19.556 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:19.556 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:19.556 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:19.556 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:19.556 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:19.556 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 83763 2024-09-21T15:00:19.556 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.556 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:19.556 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.556 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:19.557 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.157975+0000", 2024-09-21T15:00:19.557 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:19.557 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.557 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.557 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:19.557 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.157975+0000", 2024-09-21T15:00:19.557 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:19.557 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.557 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.557 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:19.557 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.157970+0000", 2024-09-21T15:00:19.557 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999943 2024-09-21T15:00:19.557 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.557 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.557 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:19.558 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.157985+0000", 2024-09-21T15:00:19.558 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4807000000000001e-05 2024-09-21T15:00:19.558 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.558 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.558 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:19.558 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.158012+0000", 2024-09-21T15:00:19.558 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7600999999999999e-05 2024-09-21T15:00:19.558 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.558 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.558 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:19.558 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.158019+0000", 2024-09-21T15:00:19.558 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9319999999999999e-06 2024-09-21T15:00:19.558 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.558 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.558 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:19.559 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.166118+0000", 2024-09-21T15:00:19.559 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0080992489999999993 2024-09-21T15:00:19.559 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.559 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.559 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:19.559 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.166156+0000", 2024-09-21T15:00:19.559 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7639999999999999e-05 2024-09-21T15:00:19.559 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.559 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.559 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:19.559 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.166303+0000", 2024-09-21T15:00:19.559 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000146799 2024-09-21T15:00:19.559 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:19.559 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:19.559 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:19.560 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.560 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.560 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:83764 3.b 3:d3640016:::rbd_data.11368296f791.0000000000000006:40 [sparse-read 1966080~0 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-21T15:00:19.560 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:51.158026+0000", 2024-09-21T15:00:19.560 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 28.322242000999999, 2024-09-21T15:00:19.560 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0083785839999999997, 2024-09-21T15:00:19.560 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:19.560 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:19.560 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:19.560 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:19.560 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:19.560 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 83764 2024-09-21T15:00:19.560 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.560 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:19.561 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.561 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:19.561 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.158026+0000", 2024-09-21T15:00:19.561 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:19.561 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.561 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.561 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:19.561 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.158026+0000", 2024-09-21T15:00:19.561 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:19.561 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.561 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.561 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:19.561 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.158025+0000", 2024-09-21T15:00:19.561 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:19.561 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.562 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.562 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:19.562 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.158053+0000", 2024-09-21T15:00:19.562 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7594e-05 2024-09-21T15:00:19.562 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.562 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.562 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:19.562 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.158061+0000", 2024-09-21T15:00:19.562 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2069999999999995e-06 2024-09-21T15:00:19.562 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.562 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.562 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:19.562 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.158067+0000", 2024-09-21T15:00:19.562 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3799999999999999e-06 2024-09-21T15:00:19.562 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.563 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.563 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:19.563 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.166321+0000", 2024-09-21T15:00:19.563 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0082539870000000008 2024-09-21T15:00:19.563 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.563 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.563 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:19.563 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.166360+0000", 2024-09-21T15:00:19.563 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8597999999999999e-05 2024-09-21T15:00:19.563 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.563 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.563 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:19.563 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.166404+0000", 2024-09-21T15:00:19.563 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4378000000000001e-05 2024-09-21T15:00:19.564 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:19.564 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:19.564 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:19.564 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.564 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.564 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:97675 3.b 3:d321acb8:::rbd_data.1136bfc79a70.0000000000000003:head [write 0~4718592 [fadvise_sequential+fadvise_nocache],write 5242880~147456 [fadvise_sequential+fadvise_nocache],write 5455872~147456 [fadvise_sequential+fadvise_nocache],write 5668864~229376 [fadvise_sequential+fadvise_nocache],write 5963776~147456 [fadvise_sequential+fadvise_nocache],write 6176768~229376 [fadvise_sequential+fadvise_nocache],write 6471680~147456 [fadvise_sequential+fadvise_nocache],write 6684672~229376 [fadvise_sequential+fadvise_nocache],write 6979584~147456 [fadvise_sequential+fadvise_nocache],write 7192576~229376 [fadvise_sequential+fadvise_nocache],write 7487488~147456 [fadvise_sequential+fadvise_nocache],write 7700480~147456 [fadvise_sequential+fadvise_nocache],write 7864320~65536 [fadvise_sequential+fadvise_nocache],write 7995392~147456 [fadvise_sequential+fadvise_nocache],write 8208384~147456 [fadvise_sequential+fadvise_nocache],write 8388608~49152 [fadvise_sequential+fadvise_nocache],write 8536064~114688 [fadvise_sequential+fadvise_nocache],write 8749056~114688 [fadvise_sequential+fadvise_nocache],write 8912896~32768 [fadvise_sequential+fadvise_nocache],write 9125888~32768 [fadvise_sequential+fadvise_nocache],write 9338880~32768 [fadvise_sequential+fadvise_nocache],write 9437184~16384 [fadvise_sequential+fadvise_nocache],write 9633792~32768 [fadvise_sequential+fadvise_nocache],write 9846784~32768 [fadvise_sequential+fadvise_nocache],write 9961472~114688 [fadvise_sequential+fadvise_nocache],write 10092544~196608 [fadvise_sequential+fadvise_nocache],write 10305536~278528 [fadvise_sequential+fadvise_nocache],write 10600448~196608 [fadvise_sequential+fadvise_nocache],write 10813440~278528 [fadvise_sequential+fadvise_nocache],write 11108352~196608 [fadvise_sequential+fadvise_nocache],write 11321344~196608 [fadvise_sequential+fadvise_nocache],write 11534336~65536 [fadvise_sequential+fadvise_nocache],write 11632640~98304 [fadvise_sequential+fadvise_nocache],write 11747328~65536 [fadvise_sequential+fadvise_nocache],write 11845632~98304 [fadvise_sequential+fadvise_nocache],write 11960320~65536 [fadvise_sequential+fadvise_nocache],write 12058624~49152 [fadvise_sequential+fadvise_nocache],write 12140544~98304 [fadvise_sequential+fadvise_nocache],write 12255232~65536 [fadvise_sequential+fadvise_nocache],write 12353536~98304 [fadvise_sequential+fadvise_nocache],write 12468224~65536 [fadvise_sequential+fadvise_nocache],write 12566528~16384 [fadvise_sequential+fadvise_nocache],write 12648448~98304 [fadvise_sequential+fadvise_nocache],write 12861440~98304 [fadvise_sequential+fadvise_nocache],write 13074432~32768 [fadvise_sequential+fadvise_nocache],write 13156352~98304 [fadvise_sequential+fadvise_nocache],write 13369344~98304 [fadvise_sequential+fadvise_nocache],write 13582336~49152 [fadvise_sequential+fadvise_nocache],write 13664256~98304 [fadvise_sequential+fadvise_nocache],write 13877248~98304 [fadvise_sequential+fadvise_nocache],write 14090240~65536 [fadvise_sequential+fadvise_nocache],write 15286272~81920 [fadvise_sequential+fadvise_nocache],write 15499264~81920 [fadvise_sequential+fadvise_nocache],write 15712256~16384 [fadvise_sequential+fadvise_nocache],write 15794176~81920 [fadvise_sequential+fadvise_nocache],write 16007168~81920 [fadvise_sequential+fadvise_nocache],write 16220160~32768 [fadvise_sequential+fadvise_nocache],write 16302080~81920 [fadvise_sequential+fadvise_nocache],write 16515072~81920 [fadvise_sequential+fadvise_nocache],write 16728064~49152 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:19.564 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.302412+0000", 2024-09-21T15:00:19.564 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.177856333999999, 2024-09-21T15:00:19.564 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.067380532000000007, 2024-09-21T15:00:19.564 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:19.564 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:19.564 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:19.564 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:19.564 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:19.565 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97675 2024-09-21T15:00:19.565 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.565 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:19.565 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.565 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:19.565 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.302412+0000", 2024-09-21T15:00:19.565 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:19.565 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.565 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.565 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:19.565 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.302412+0000", 2024-09-21T15:00:19.565 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:19.565 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.565 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.565 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:19.566 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.302409+0000", 2024-09-21T15:00:19.566 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-21T15:00:19.566 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.566 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.566 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:19.566 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328550+0000", 2024-09-21T15:00:19.566 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.026140674999999999 2024-09-21T15:00:19.566 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.566 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.566 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:19.566 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328584+0000", 2024-09-21T15:00:19.566 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4165999999999998e-05 2024-09-21T15:00:19.566 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.566 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.567 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:19.567 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328599+0000", 2024-09-21T15:00:19.567 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5505000000000001e-05 2024-09-21T15:00:19.567 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.567 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.567 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:19.567 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328704+0000", 2024-09-21T15:00:19.567 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010487199999999999 2024-09-21T15:00:19.567 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.567 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.567 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:19.567 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.329082+0000", 2024-09-21T15:00:19.567 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00037804899999999998 2024-09-21T15:00:19.567 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.567 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.568 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T15:00:19.568 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.331158+0000", 2024-09-21T15:00:19.568 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0020755449999999998 2024-09-21T15:00:19.568 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.568 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.568 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:19.568 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.360553+0000", 2024-09-21T15:00:19.568 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.029395553000000001 2024-09-21T15:00:19.568 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.568 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.568 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:19.568 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.369496+0000", 2024-09-21T15:00:19.568 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0089428619999999993 2024-09-21T15:00:19.568 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.569 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.569 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:19.569 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.369686+0000", 2024-09-21T15:00:19.569 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019000399999999999 2024-09-21T15:00:19.569 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.569 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.569 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:19.569 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.369792+0000", 2024-09-21T15:00:19.569 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000105775 2024-09-21T15:00:19.569 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:19.569 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:19.569 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:19.569 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.569 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.569 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:97693 3.b 3:d7d7b39b:::rbd_data.1136b9c49b1c.000000000000000c:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:19.570 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.328643+0000", 2024-09-21T15:00:19.570 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.151624547999999, 2024-09-21T15:00:19.570 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.031062136000000001, 2024-09-21T15:00:19.570 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:19.570 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:19.570 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:19.570 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:19.570 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:19.570 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97693 2024-09-21T15:00:19.570 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.570 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:19.570 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.570 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:19.570 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328643+0000", 2024-09-21T15:00:19.571 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:19.571 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.571 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.571 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:19.571 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328643+0000", 2024-09-21T15:00:19.571 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:19.571 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.571 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.571 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:19.571 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328642+0000", 2024-09-21T15:00:19.571 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:00:19.571 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.571 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.572 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:19.572 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328650+0000", 2024-09-21T15:00:19.572 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7770000000000008e-06 2024-09-21T15:00:19.572 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.572 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.572 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:19.572 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328662+0000", 2024-09-21T15:00:19.572 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2026e-05 2024-09-21T15:00:19.572 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.572 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.572 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:19.572 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328671+0000", 2024-09-21T15:00:19.572 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.585e-06 2024-09-21T15:00:19.572 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.572 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.572 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:19.573 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.359498+0000", 2024-09-21T15:00:19.573 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.030827001 2024-09-21T15:00:19.573 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.573 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.573 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:19.573 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.359603+0000", 2024-09-21T15:00:19.573 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010512400000000001 2024-09-21T15:00:19.573 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.573 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.573 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:19.573 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.359705+0000", 2024-09-21T15:00:19.573 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000102327 2024-09-21T15:00:19.573 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:19.573 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:19.573 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:19.574 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.574 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.574 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:97724 3.b 3:d4bf6bcf:::rbd_data.1136b9c49b1c.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:19.574 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.328704+0000", 2024-09-21T15:00:19.574 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.151564276, 2024-09-21T15:00:19.574 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.031157957, 2024-09-21T15:00:19.574 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:19.574 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:19.574 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:19.574 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:19.574 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:19.574 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97724 2024-09-21T15:00:19.574 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.574 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:19.575 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.575 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:19.575 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328704+0000", 2024-09-21T15:00:19.575 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:19.575 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.575 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.575 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:19.575 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328704+0000", 2024-09-21T15:00:19.575 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:19.575 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.575 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.575 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:19.575 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328703+0000", 2024-09-21T15:00:19.575 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:00:19.575 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.576 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.576 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:19.576 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328708+0000", 2024-09-21T15:00:19.576 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0050000000000004e-06 2024-09-21T15:00:19.576 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.576 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.576 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:19.576 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328719+0000", 2024-09-21T15:00:19.576 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1052e-05 2024-09-21T15:00:19.576 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.576 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.576 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:19.576 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328727+0000", 2024-09-21T15:00:19.576 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4519999999999997e-06 2024-09-21T15:00:19.576 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.576 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.577 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:19.577 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.359737+0000", 2024-09-21T15:00:19.577 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.031009341999999999 2024-09-21T15:00:19.577 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.577 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.577 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:19.577 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.359806+0000", 2024-09-21T15:00:19.577 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9750000000000001e-05 2024-09-21T15:00:19.577 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:19.577 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:19.577 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:19.577 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.359862+0000", 2024-09-21T15:00:19.577 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5084999999999999e-05 2024-09-21T15:00:19.577 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:19.577 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:19.577 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:19.578 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:19.578 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:19.578 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:19.579 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T15:00:19.579 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:00:19.576+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:00:19.680 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:00:19.678+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:00:19.689 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:19.689 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:00:19.689 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T15:00:19.689 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:19.699 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T15:00:19.781 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:00:19.778+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:00:19.809 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:19.809 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:00:19.809 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T15:00:19.809 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T15:00:19.809 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:19.818 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T15:00:19.882 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:00:19.880+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:00:19.925 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:19.925 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T15:00:19.925 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T15:00:19.925 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T15:00:19.925 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.926 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:1678 3.c 3:377cc7e1:::rbd_data.1136515ecb6b.000000000000000f:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T15:00:19.926 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:30.988085+0000", 2024-09-21T15:00:19.926 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 108.93538098499999, 2024-09-21T15:00:19.926 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.032974802999999997, 2024-09-21T15:00:19.926 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:19.926 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:19.926 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:19.926 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:19.926 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:19.926 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1678 2024-09-21T15:00:19.926 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.926 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:19.926 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.927 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:19.927 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988085+0000", 2024-09-21T15:00:19.927 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:19.927 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.927 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.927 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:19.927 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988085+0000", 2024-09-21T15:00:19.927 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:19.927 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.927 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.927 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:19.927 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988083+0000", 2024-09-21T15:00:19.927 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-21T15:00:19.927 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.928 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.928 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:19.928 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988093+0000", 2024-09-21T15:00:19.928 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8800000000000003e-06 2024-09-21T15:00:19.928 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.928 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.928 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:19.928 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988113+0000", 2024-09-21T15:00:19.928 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0217999999999999e-05 2024-09-21T15:00:19.928 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.928 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.928 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:19.928 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988123+0000", 2024-09-21T15:00:19.928 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5859999999999994e-06 2024-09-21T15:00:19.929 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.929 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.929 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:19.929 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988203+0000", 2024-09-21T15:00:19.929 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0834000000000006e-05 2024-09-21T15:00:19.929 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.929 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.929 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:19.929 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.988376+0000", 2024-09-21T15:00:19.929 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017301299999999999 2024-09-21T15:00:19.929 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.929 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.929 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 7", 2024-09-21T15:00:19.929 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:30.996338+0000", 2024-09-21T15:00:19.930 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0079618520000000002 2024-09-21T15:00:19.930 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.930 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.930 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:19.930 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.020881+0000", 2024-09-21T15:00:19.930 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024542731000000002 2024-09-21T15:00:19.930 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.930 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.930 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:19.930 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.020904+0000", 2024-09-21T15:00:19.930 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2861e-05 2024-09-21T15:00:19.930 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.930 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.930 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:19.931 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.020987+0000", 2024-09-21T15:00:19.931 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2806000000000006e-05 2024-09-21T15:00:19.931 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.931 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.931 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:19.931 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:31.021060+0000", 2024-09-21T15:00:19.931 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2871000000000001e-05 2024-09-21T15:00:19.931 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:19.931 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:19.931 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:19.931 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.931 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.931 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9403 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:19.931 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.157949+0000", 2024-09-21T15:00:19.932 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 100.765516338, 2024-09-21T15:00:19.932 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023540373999999999, 2024-09-21T15:00:19.932 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:19.932 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:19.932 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:19.932 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:19.932 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:19.932 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9403 2024-09-21T15:00:19.932 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.932 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:19.932 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.932 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:19.932 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157949+0000", 2024-09-21T15:00:19.932 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:19.932 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.933 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.933 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:19.933 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157949+0000", 2024-09-21T15:00:19.933 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:19.933 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.933 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.933 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:19.933 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157948+0000", 2024-09-21T15:00:19.933 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T15:00:19.933 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.933 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.933 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:19.933 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157956+0000", 2024-09-21T15:00:19.933 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7999999999999999e-06 2024-09-21T15:00:19.934 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.934 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.934 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:19.934 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157969+0000", 2024-09-21T15:00:19.934 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2792000000000001e-05 2024-09-21T15:00:19.934 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.934 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.934 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:19.934 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.157977+0000", 2024-09-21T15:00:19.934 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.704e-06 2024-09-21T15:00:19.934 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.934 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.934 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:19.935 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177246+0000", 2024-09-21T15:00:19.935 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019268264 2024-09-21T15:00:19.935 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.935 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.935 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:19.935 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177315+0000", 2024-09-21T15:00:19.935 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9480999999999994e-05 2024-09-21T15:00:19.935 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.935 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.935 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T15:00:19.935 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177615+0000", 2024-09-21T15:00:19.935 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00029929499999999999 2024-09-21T15:00:19.935 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.935 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.935 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:19.935 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181333+0000", 2024-09-21T15:00:19.936 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0037183149999999998 2024-09-21T15:00:19.936 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.936 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.936 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:19.936 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181374+0000", 2024-09-21T15:00:19.936 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.0723999999999998e-05 2024-09-21T15:00:19.936 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.936 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.936 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:19.936 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181434+0000", 2024-09-21T15:00:19.936 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0024e-05 2024-09-21T15:00:19.936 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.936 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.936 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:19.937 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.181490+0000", 2024-09-21T15:00:19.937 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6245000000000001e-05 2024-09-21T15:00:19.937 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:19.937 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:19.937 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:19.937 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.937 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.937 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9410 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:19.937 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.174103+0000", 2024-09-21T15:00:19.937 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 100.74936258, 2024-09-21T15:00:19.937 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025493703, 2024-09-21T15:00:19.937 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:19.937 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:19.937 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:19.938 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:19.938 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:19.938 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9410 2024-09-21T15:00:19.938 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.938 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:19.938 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.938 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:19.938 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.174103+0000", 2024-09-21T15:00:19.938 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:19.938 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.938 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.938 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:19.938 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.174103+0000", 2024-09-21T15:00:19.938 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:19.938 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.939 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.939 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:19.939 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.174102+0000", 2024-09-21T15:00:19.939 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:00:19.939 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.939 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.939 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:19.939 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.174108+0000", 2024-09-21T15:00:19.939 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1149999999999999e-06 2024-09-21T15:00:19.939 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.939 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.939 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:19.939 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.174117+0000", 2024-09-21T15:00:19.939 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4870000000000002e-06 2024-09-21T15:00:19.939 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.940 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.940 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:19.940 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.174122+0000", 2024-09-21T15:00:19.940 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9790000000000001e-06 2024-09-21T15:00:19.940 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.940 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.940 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:19.940 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.187830+0000", 2024-09-21T15:00:19.940 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013708048 2024-09-21T15:00:19.940 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.940 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.940 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:19.940 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.187913+0000", 2024-09-21T15:00:19.940 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2763999999999996e-05 2024-09-21T15:00:19.940 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.941 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.941 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T15:00:19.941 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.188230+0000", 2024-09-21T15:00:19.941 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00031745100000000001 2024-09-21T15:00:19.941 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.941 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.941 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:19.941 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.192708+0000", 2024-09-21T15:00:19.941 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0044783890000000002 2024-09-21T15:00:19.941 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.941 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.941 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:19.941 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199487+0000", 2024-09-21T15:00:19.941 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0067782149999999998 2024-09-21T15:00:19.941 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.942 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.942 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:19.942 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199560+0000", 2024-09-21T15:00:19.942 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3752999999999997e-05 2024-09-21T15:00:19.942 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.942 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.942 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:19.942 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199597+0000", 2024-09-21T15:00:19.942 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6526999999999998e-05 2024-09-21T15:00:19.942 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:19.942 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:19.942 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:19.942 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.942 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.943 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9411 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:19.943 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.175943+0000", 2024-09-21T15:00:19.943 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 100.747522561, 2024-09-21T15:00:19.943 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023773181000000001, 2024-09-21T15:00:19.943 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:19.943 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:19.943 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:19.943 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:19.943 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:19.943 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9411 2024-09-21T15:00:19.943 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.943 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:19.943 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.944 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:19.944 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175943+0000", 2024-09-21T15:00:19.944 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:19.944 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.944 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.944 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:19.944 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175943+0000", 2024-09-21T15:00:19.944 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:19.944 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.944 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.944 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:19.944 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175942+0000", 2024-09-21T15:00:19.944 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:00:19.944 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.945 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.945 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:19.945 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175948+0000", 2024-09-21T15:00:19.945 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9689999999999999e-06 2024-09-21T15:00:19.945 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.945 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.945 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:19.945 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175960+0000", 2024-09-21T15:00:19.945 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1745e-05 2024-09-21T15:00:19.945 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.945 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.945 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:19.945 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.175965+0000", 2024-09-21T15:00:19.945 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.203e-06 2024-09-21T15:00:19.945 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.945 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.946 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:19.946 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.189900+0000", 2024-09-21T15:00:19.946 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013934940999999999 2024-09-21T15:00:19.946 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.946 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.946 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:19.946 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.189988+0000", 2024-09-21T15:00:19.946 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8419999999999997e-05 2024-09-21T15:00:19.946 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.946 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.946 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T15:00:19.946 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.190321+0000", 2024-09-21T15:00:19.946 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00033231800000000001 2024-09-21T15:00:19.947 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.947 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.947 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:19.947 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.192746+0000", 2024-09-21T15:00:19.947 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0024254400000000001 2024-09-21T15:00:19.947 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.947 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.947 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:19.947 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199605+0000", 2024-09-21T15:00:19.947 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0068582809999999999 2024-09-21T15:00:19.947 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.947 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.947 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:19.947 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199680+0000", 2024-09-21T15:00:19.948 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5630000000000006e-05 2024-09-21T15:00:19.948 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.948 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.948 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:19.948 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199716+0000", 2024-09-21T15:00:19.948 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6248999999999998e-05 2024-09-21T15:00:19.948 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:19.948 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:19.948 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:19.948 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.948 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.948 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:9412 3.3 3:c63840eb:::rbd_object_map.1136f5ed5dd6.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:19.948 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:39.177402+0000", 2024-09-21T15:00:19.948 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 100.746063464, 2024-09-21T15:00:19.948 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022445590000000001, 2024-09-21T15:00:19.949 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:19.949 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:19.949 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:19.949 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:19.949 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:19.949 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 9412 2024-09-21T15:00:19.949 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.949 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:19.949 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.949 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:19.949 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177402+0000", 2024-09-21T15:00:19.949 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:19.949 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.949 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.950 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:19.950 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177402+0000", 2024-09-21T15:00:19.950 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:19.950 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.950 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.950 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:19.950 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177401+0000", 2024-09-21T15:00:19.950 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:00:19.950 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.950 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.950 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:19.950 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177409+0000", 2024-09-21T15:00:19.950 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4710000000000001e-06 2024-09-21T15:00:19.950 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.951 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.951 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:19.951 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177419+0000", 2024-09-21T15:00:19.951 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0351e-05 2024-09-21T15:00:19.951 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.951 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.951 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:19.951 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.177424+0000", 2024-09-21T15:00:19.951 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1919999999999996e-06 2024-09-21T15:00:19.951 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.951 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.951 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:19.951 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.191343+0000", 2024-09-21T15:00:19.951 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.01391878 2024-09-21T15:00:19.951 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.952 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.952 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:19.952 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.191446+0000", 2024-09-21T15:00:19.952 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000102453 2024-09-21T15:00:19.952 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.952 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.952 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T15:00:19.952 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.192200+0000", 2024-09-21T15:00:19.952 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00075476899999999997 2024-09-21T15:00:19.952 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.952 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.952 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:19.952 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.193866+0000", 2024-09-21T15:00:19.952 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0016657519999999999 2024-09-21T15:00:19.953 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.953 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.953 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:19.953 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199724+0000", 2024-09-21T15:00:19.953 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0058579529999999999 2024-09-21T15:00:19.953 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.953 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.953 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:19.953 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199803+0000", 2024-09-21T15:00:19.953 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8662e-05 2024-09-21T15:00:19.953 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.953 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.953 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:19.954 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:39.199848+0000", 2024-09-21T15:00:19.954 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5216000000000002e-05 2024-09-21T15:00:19.954 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:19.954 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:19.954 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:19.954 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.954 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.954 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:96341 3.15 3:aaa1da52:::rbd_data.1136b9c49b1c.0000000000000003:49 [read 212992~32768 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:19.954 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.200375+0000", 2024-09-21T15:00:19.954 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.723090607, 2024-09-21T15:00:19.954 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026255225, 2024-09-21T15:00:19.954 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:19.954 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:19.954 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:19.955 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:19.955 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:19.955 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 96341 2024-09-21T15:00:19.955 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.955 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:19.955 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.955 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:19.955 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200375+0000", 2024-09-21T15:00:19.955 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:19.955 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.955 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.955 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:19.955 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200375+0000", 2024-09-21T15:00:19.955 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:19.955 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.956 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.956 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:19.956 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200374+0000", 2024-09-21T15:00:19.956 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T15:00:19.956 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.956 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.956 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:19.956 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200381+0000", 2024-09-21T15:00:19.956 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6950000000000003e-06 2024-09-21T15:00:19.956 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.956 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.956 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:19.956 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200390+0000", 2024-09-21T15:00:19.956 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2750000000000001e-06 2024-09-21T15:00:19.957 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.957 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.957 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:19.957 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200397+0000", 2024-09-21T15:00:19.957 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.1389999999999998e-06 2024-09-21T15:00:19.957 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.957 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.957 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:19.957 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.216693+0000", 2024-09-21T15:00:19.957 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016296076 2024-09-21T15:00:19.957 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.957 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.957 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:19.957 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.216784+0000", 2024-09-21T15:00:19.958 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0484999999999998e-05 2024-09-21T15:00:19.958 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.958 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.958 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:19.958 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.226630+0000", 2024-09-21T15:00:19.958 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0098468730000000008 2024-09-21T15:00:19.958 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:19.958 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:19.958 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:19.958 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.958 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.958 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:96478 3.15 3:aee50b17:::rbd_data.1136b9c49b1c.0000000000000015:49 [read 1327104~49152 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:19.958 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.200926+0000", 2024-09-21T15:00:19.958 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.722539347, 2024-09-21T15:00:19.958 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021532795, 2024-09-21T15:00:19.959 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:19.959 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:19.959 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:19.959 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:19.959 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:19.959 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 96478 2024-09-21T15:00:19.959 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.959 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:19.959 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.959 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:19.959 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200926+0000", 2024-09-21T15:00:19.959 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:19.959 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.959 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.960 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:19.960 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200926+0000", 2024-09-21T15:00:19.960 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:19.960 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.960 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.960 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:19.960 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200926+0000", 2024-09-21T15:00:19.960 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:19.960 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.960 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.960 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:19.960 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200929+0000", 2024-09-21T15:00:19.961 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4449999999999999e-06 2024-09-21T15:00:19.961 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.961 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.961 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:19.961 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200934+0000", 2024-09-21T15:00:19.961 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2249999999999999e-06 2024-09-21T15:00:19.961 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.961 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.961 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:19.961 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200938+0000", 2024-09-21T15:00:19.961 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6739999999999996e-06 2024-09-21T15:00:19.961 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.961 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.961 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:19.962 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.222046+0000", 2024-09-21T15:00:19.962 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021107225 2024-09-21T15:00:19.962 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.962 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.962 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:19.962 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.222143+0000", 2024-09-21T15:00:19.962 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.7740999999999998e-05 2024-09-21T15:00:19.962 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.962 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.962 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:19.962 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.222459+0000", 2024-09-21T15:00:19.962 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00031578500000000001 2024-09-21T15:00:19.962 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:19.962 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:19.962 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:19.963 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.963 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.963 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:96479 3.15 3:aee50b17:::rbd_data.1136b9c49b1c.0000000000000015:49 [read 1769472~49152 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:19.963 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.200942+0000", 2024-09-21T15:00:19.963 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.722523313, 2024-09-21T15:00:19.963 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021994197, 2024-09-21T15:00:19.963 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:19.963 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:19.963 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:19.963 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:19.963 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:19.963 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 96479 2024-09-21T15:00:19.963 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.963 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:19.963 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.964 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:19.964 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200942+0000", 2024-09-21T15:00:19.964 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:19.964 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.964 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.964 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:19.964 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200942+0000", 2024-09-21T15:00:19.964 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:19.964 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.964 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.964 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:19.964 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200942+0000", 2024-09-21T15:00:19.964 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:19.964 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.965 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.965 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:19.965 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200945+0000", 2024-09-21T15:00:19.965 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3989999999999998e-06 2024-09-21T15:00:19.965 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.965 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.965 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:19.965 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200950+0000", 2024-09-21T15:00:19.965 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.631e-06 2024-09-21T15:00:19.965 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.965 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.965 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:19.965 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200955+0000", 2024-09-21T15:00:19.966 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6319999999999999e-06 2024-09-21T15:00:19.966 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.966 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.966 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:19.966 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.222484+0000", 2024-09-21T15:00:19.966 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021528988999999998 2024-09-21T15:00:19.966 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.966 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.966 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:19.966 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.222558+0000", 2024-09-21T15:00:19.966 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4239000000000005e-05 2024-09-21T15:00:19.966 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.966 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.966 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:19.967 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.222937+0000", 2024-09-21T15:00:19.967 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00037861999999999998 2024-09-21T15:00:19.967 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:19.967 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:19.967 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:19.967 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.967 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.967 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:96480 3.15 3:aee50b17:::rbd_data.1136b9c49b1c.0000000000000015:49 [read 2211840~49152 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:19.967 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.200959+0000", 2024-09-21T15:00:19.967 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.722507021, 2024-09-21T15:00:19.967 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022433293, 2024-09-21T15:00:19.967 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:19.967 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:19.967 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:19.968 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:19.968 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:19.968 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 96480 2024-09-21T15:00:19.968 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.968 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:19.968 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.968 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:19.968 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200959+0000", 2024-09-21T15:00:19.968 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:19.968 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.968 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.968 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:19.968 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200959+0000", 2024-09-21T15:00:19.968 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:19.969 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.969 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.969 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:19.969 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200958+0000", 2024-09-21T15:00:19.969 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:19.969 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.969 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.969 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:19.969 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200961+0000", 2024-09-21T15:00:19.969 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3549999999999999e-06 2024-09-21T15:00:19.969 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.969 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.969 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:19.969 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200966+0000", 2024-09-21T15:00:19.970 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5029999999999997e-06 2024-09-21T15:00:19.970 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.970 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.970 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:19.970 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200971+0000", 2024-09-21T15:00:19.970 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.561e-06 2024-09-21T15:00:19.970 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.970 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.970 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:19.970 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.222963+0000", 2024-09-21T15:00:19.970 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021992088 2024-09-21T15:00:19.970 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.970 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.970 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:19.971 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.223045+0000", 2024-09-21T15:00:19.971 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2464000000000002e-05 2024-09-21T15:00:19.971 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.971 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.971 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:19.971 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.223392+0000", 2024-09-21T15:00:19.971 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00034665299999999999 2024-09-21T15:00:19.971 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:19.971 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:19.971 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:19.971 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.971 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.971 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:96481 3.15 3:aee50b17:::rbd_data.1136b9c49b1c.0000000000000015:49 [read 2654208~49152 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:19.971 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.200975+0000", 2024-09-21T15:00:19.972 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.722490985, 2024-09-21T15:00:19.972 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022639701000000002, 2024-09-21T15:00:19.972 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:19.972 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:19.972 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:19.972 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:19.972 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:19.972 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 96481 2024-09-21T15:00:19.972 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.972 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:19.972 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.972 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:19.972 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200975+0000", 2024-09-21T15:00:19.972 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:19.972 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.973 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.973 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:19.973 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200975+0000", 2024-09-21T15:00:19.973 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:19.973 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.973 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.973 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:19.973 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200974+0000", 2024-09-21T15:00:19.973 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:19.973 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.973 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.973 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:19.973 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200977+0000", 2024-09-21T15:00:19.973 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.136e-06 2024-09-21T15:00:19.974 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.974 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.974 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:19.974 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200982+0000", 2024-09-21T15:00:19.974 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4129999999999997e-06 2024-09-21T15:00:19.974 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.974 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.974 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:19.974 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200987+0000", 2024-09-21T15:00:19.974 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5199999999999999e-06 2024-09-21T15:00:19.974 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.974 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.974 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:19.974 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.223420+0000", 2024-09-21T15:00:19.975 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022433611999999999 2024-09-21T15:00:19.975 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.975 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.975 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:19.975 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.223513+0000", 2024-09-21T15:00:19.975 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2477000000000002e-05 2024-09-21T15:00:19.975 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.975 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.975 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:19.975 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.223615+0000", 2024-09-21T15:00:19.975 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010186199999999999 2024-09-21T15:00:19.975 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:19.975 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:19.976 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:19.976 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.976 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.976 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:96482 3.15 3:aee50b17:::rbd_data.1136b9c49b1c.0000000000000015:49 [read 3096576~49152 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:19.976 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.200991+0000", 2024-09-21T15:00:19.976 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.722475105000001, 2024-09-21T15:00:19.976 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023020315, 2024-09-21T15:00:19.976 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:19.976 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:19.976 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:19.976 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:19.976 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:19.976 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 96482 2024-09-21T15:00:19.976 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.976 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:19.976 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.977 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:19.977 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200991+0000", 2024-09-21T15:00:19.977 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:19.977 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.977 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.977 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:19.977 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200991+0000", 2024-09-21T15:00:19.977 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:19.977 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.977 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.977 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:19.977 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200990+0000", 2024-09-21T15:00:19.977 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:19.977 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.977 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.978 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:19.978 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200993+0000", 2024-09-21T15:00:19.978 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4119999999999999e-06 2024-09-21T15:00:19.978 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.978 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.978 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:19.978 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.200998+0000", 2024-09-21T15:00:19.978 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4639999999999997e-06 2024-09-21T15:00:19.978 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.978 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.978 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:19.978 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.201003+0000", 2024-09-21T15:00:19.978 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8520000000000003e-06 2024-09-21T15:00:19.978 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.979 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.979 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:19.979 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.223660+0000", 2024-09-21T15:00:19.979 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022657295000000001 2024-09-21T15:00:19.979 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.979 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.979 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:19.979 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.223834+0000", 2024-09-21T15:00:19.979 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017377700000000001 2024-09-21T15:00:19.979 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.979 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.979 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:19.979 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.224011+0000", 2024-09-21T15:00:19.979 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017705999999999999 2024-09-21T15:00:19.979 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:19.980 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:19.980 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:19.980 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.980 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.980 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:97677 3.15 3:a8e72133:::rbd_data.1136bfc79a70.0000000000000005:head [write 0~4440027 [fadvise_sequential+fadvise_nocache],write 4751360~147456 [fadvise_sequential+fadvise_nocache],write 4964352~147456 [fadvise_sequential+fadvise_nocache],write 5177344~65536 [fadvise_sequential+fadvise_nocache],write 5259264~147456 [fadvise_sequential+fadvise_nocache],write 5472256~147456 [fadvise_sequential+fadvise_nocache],write 5685248~229376 [fadvise_sequential+fadvise_nocache],write 5980160~147456 [fadvise_sequential+fadvise_nocache],write 6193152~229376 [fadvise_sequential+fadvise_nocache],write 6488064~147456 [fadvise_sequential+fadvise_nocache],write 6701056~229376 [fadvise_sequential+fadvise_nocache],write 6995968~147456 [fadvise_sequential+fadvise_nocache],write 7208960~229376 [fadvise_sequential+fadvise_nocache],write 7503872~147456 [fadvise_sequential+fadvise_nocache],write 7716864~229376 [fadvise_sequential+fadvise_nocache],write 8044544~114688 [fadvise_sequential+fadvise_nocache],write 8257536~114688 [fadvise_sequential+fadvise_nocache],write 8388608~65536 [fadvise_sequential+fadvise_nocache],write 8552448~114688 [fadvise_sequential+fadvise_nocache],write 8765440~114688 [fadvise_sequential+fadvise_nocache],write 8929280~32768 [fadvise_sequential+fadvise_nocache],write 9142272~32768 [fadvise_sequential+fadvise_nocache],write 9355264~32768 [fadvise_sequential+fadvise_nocache],write 9437184~32768 [fadvise_sequential+fadvise_nocache],write 9650176~32768 [fadvise_sequential+fadvise_nocache],write 9863168~32768 [fadvise_sequential+fadvise_nocache],write 9961472~131072 [fadvise_sequential+fadvise_nocache],write 10108928~196608 [fadvise_sequential+fadvise_nocache],write 10321920~278528 [fadvise_sequential+fadvise_nocache],write 10616832~196608 [fadvise_sequential+fadvise_nocache],write 10829824~278528 [fadvise_sequential+fadvise_nocache],write 11124736~196608 [fadvise_sequential+fadvise_nocache],write 11337728~196608 [fadvise_sequential+fadvise_nocache],write 11550720~65536 [fadvise_sequential+fadvise_nocache],write 11649024~98304 [fadvise_sequential+fadvise_nocache],write 11763712~65536 [fadvise_sequential+fadvise_nocache],write 11862016~98304 [fadvise_sequential+fadvise_nocache],write 11976704~65536 [fadvise_sequential+fadvise_nocache],write 12058624~65536 [fadvise_sequential+fadvise_nocache],write 12156928~98304 [fadvise_sequential+fadvise_nocache],write 12271616~65536 [fadvise_sequential+fadvise_nocache],write 12369920~98304 [fadvise_sequential+fadvise_nocache],write 12484608~65536 [fadvise_sequential+fadvise_nocache],write 12664832~98304 [fadvise_sequential+fadvise_nocache],write 12877824~98304 [fadvise_sequential+fadvise_nocache],write 13090816~16384 [fadvise_sequential+fadvise_nocache],write 13172736~98304 [fadvise_sequential+fadvise_nocache],write 13385728~98304 [fadvise_sequential+fadvise_nocache],write 13598720~32768 [fadvise_sequential+fadvise_nocache],write 13680640~98304 [fadvise_sequential+fadvise_nocache],write 13893632~98304 [fadvise_sequential+fadvise_nocache],write 14106624~49152 [fadvise_sequential+fadvise_nocache],write 14794752~81920 [fadvise_sequential+fadvise_nocache],write 15007744~81920 [fadvise_sequential+fadvise_nocache],write 15302656~81920 [fadvise_sequential+fadvise_nocache],write 15515648~81920 [fadvise_sequential+fadvise_nocache],write 15810560~81920 [fadvise_sequential+fadvise_nocache],write 16023552~81920 [fadvise_sequential+fadvise_nocache],write 16236544~16384 [fadvise_sequential+fadvise_nocache],write 16318464~81920 [fadvise_sequential+fadvise_nocache],write 16531456~81920 [fadvise_sequential+fadvise_nocache],write 16744448~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:19.980 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.302946+0000", 2024-09-21T15:00:19.980 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.620519655000001, 2024-09-21T15:00:19.980 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.091231204999999996, 2024-09-21T15:00:19.980 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:19.980 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:19.980 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:19.980 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:19.980 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:19.980 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 97677 2024-09-21T15:00:19.981 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.981 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:19.981 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.981 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:19.981 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.302946+0000", 2024-09-21T15:00:19.981 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:19.981 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.981 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.981 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:19.981 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.302946+0000", 2024-09-21T15:00:19.981 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:19.981 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.981 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.981 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:19.981 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.302942+0000", 2024-09-21T15:00:19.982 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999962 2024-09-21T15:00:19.982 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.982 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.982 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:19.982 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311253+0000", 2024-09-21T15:00:19.982 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0083105360000000003 2024-09-21T15:00:19.982 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.982 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.982 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:19.982 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311283+0000", 2024-09-21T15:00:19.982 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0065000000000001e-05 2024-09-21T15:00:19.982 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.982 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.982 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:19.982 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311296+0000", 2024-09-21T15:00:19.983 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3206e-05 2024-09-21T15:00:19.983 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.983 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.983 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:19.983 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311404+0000", 2024-09-21T15:00:19.983 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000107514 2024-09-21T15:00:19.983 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.983 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.983 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:19.983 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311795+0000", 2024-09-21T15:00:19.983 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000390913 2024-09-21T15:00:19.983 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.983 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.983 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 6", 2024-09-21T15:00:19.984 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.314686+0000", 2024-09-21T15:00:19.984 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0028915659999999999 2024-09-21T15:00:19.984 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.984 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.984 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:19.984 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.368178+0000", 2024-09-21T15:00:19.984 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.053491706 2024-09-21T15:00:19.984 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.984 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.984 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:19.984 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.393925+0000", 2024-09-21T15:00:19.984 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025747289 2024-09-21T15:00:19.984 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.984 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.984 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:19.985 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.394127+0000", 2024-09-21T15:00:19.985 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000201444 2024-09-21T15:00:19.985 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.985 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.985 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:19.985 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.394177+0000", 2024-09-21T15:00:19.985 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0689000000000002e-05 2024-09-21T15:00:19.985 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:19.985 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:19.985 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:19.985 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.985 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.985 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:97679 3.f 3:f2a362b4:::rbd_data.1136bfc79a70.0000000000000007:head [write 0~4194304 [fadvise_sequential+fadvise_nocache],write 4767744~147456 [fadvise_sequential+fadvise_nocache],write 4980736~147456 [fadvise_sequential+fadvise_nocache],write 5193728~49152 [fadvise_sequential+fadvise_nocache],write 5275648~147456 [fadvise_sequential+fadvise_nocache],write 5488640~147456 [fadvise_sequential+fadvise_nocache],write 5701632~65536 [fadvise_sequential+fadvise_nocache],write 5783552~147456 [fadvise_sequential+fadvise_nocache],write 5996544~147456 [fadvise_sequential+fadvise_nocache],write 6209536~229376 [fadvise_sequential+fadvise_nocache],write 6504448~147456 [fadvise_sequential+fadvise_nocache],write 6717440~229376 [fadvise_sequential+fadvise_nocache],write 7012352~147456 [fadvise_sequential+fadvise_nocache],write 7225344~229376 [fadvise_sequential+fadvise_nocache],write 7520256~147456 [fadvise_sequential+fadvise_nocache],write 7733248~229376 [fadvise_sequential+fadvise_nocache],write 8060928~114688 [fadvise_sequential+fadvise_nocache],write 8273920~196608 [fadvise_sequential+fadvise_nocache],write 8568832~114688 [fadvise_sequential+fadvise_nocache],write 8781824~114688 [fadvise_sequential+fadvise_nocache],write 8945664~32768 [fadvise_sequential+fadvise_nocache],write 9158656~32768 [fadvise_sequential+fadvise_nocache],write 9371648~32768 [fadvise_sequential+fadvise_nocache],write 9453568~32768 [fadvise_sequential+fadvise_nocache],write 9666560~32768 [fadvise_sequential+fadvise_nocache],write 9879552~32768 [fadvise_sequential+fadvise_nocache],write 9961472~147456 [fadvise_sequential+fadvise_nocache],write 10125312~196608 [fadvise_sequential+fadvise_nocache],write 10338304~278528 [fadvise_sequential+fadvise_nocache],write 10633216~196608 [fadvise_sequential+fadvise_nocache],write 10846208~278528 [fadvise_sequential+fadvise_nocache],write 11157504~180224 [fadvise_sequential+fadvise_nocache],write 11370496~98304 [fadvise_sequential+fadvise_nocache],write 11485184~65536 [fadvise_sequential+fadvise_nocache],write 11567104~65536 [fadvise_sequential+fadvise_nocache],write 11665408~98304 [fadvise_sequential+fadvise_nocache],write 11780096~65536 [fadvise_sequential+fadvise_nocache],write 11878400~98304 [fadvise_sequential+fadvise_nocache],write 11993088~65536 [fadvise_sequential+fadvise_nocache],write 12075008~61231 [fadvise_sequential+fadvise_nocache],write 12173312~98304 [fadvise_sequential+fadvise_nocache],write 12386304~98304 [fadvise_sequential+fadvise_nocache],write 12681216~98304 [fadvise_sequential+fadvise_nocache],write 12894208~98304 [fadvise_sequential+fadvise_nocache],write 13189120~98304 [fadvise_sequential+fadvise_nocache],write 13402112~98304 [fadvise_sequential+fadvise_nocache],write 13615104~16384 [fadvise_sequential+fadvise_nocache],write 13697024~98304 [fadvise_sequential+fadvise_nocache],write 13910016~98304 [fadvise_sequential+fadvise_nocache],write 14123008~32768 [fadvise_sequential+fadvise_nocache],write 14811136~81920 [fadvise_sequential+fadvise_nocache],write 15024128~81920 [fadvise_sequential+fadvise_nocache],write 15319040~81920 [fadvise_sequential+fadvise_nocache],write 15532032~81920 [fadvise_sequential+fadvise_nocache],write 15826944~81920 [fadvise_sequential+fadvise_nocache],write 16039936~81920 [fadvise_sequential+fadvise_nocache],write 16334848~81920 [fadvise_sequential+fadvise_nocache],write 16547840~81920 [fadvise_sequential+fadvise_nocache],write 16760832~16384 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:19.985 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.311349+0000", 2024-09-21T15:00:19.986 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.612117258, 2024-09-21T15:00:19.986 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.048644222000000001, 2024-09-21T15:00:19.986 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:19.986 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:19.986 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:19.986 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:19.986 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:19.986 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 97679 2024-09-21T15:00:19.986 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.986 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:19.986 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.986 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:19.986 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311349+0000", 2024-09-21T15:00:19.987 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:19.987 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.987 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.987 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:19.987 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311349+0000", 2024-09-21T15:00:19.987 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:19.987 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.987 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.987 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:19.987 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311347+0000", 2024-09-21T15:00:19.987 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-21T15:00:19.987 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.987 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.987 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:19.987 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.322631+0000", 2024-09-21T15:00:19.987 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.011284307 2024-09-21T15:00:19.988 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.988 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.988 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:19.988 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.322676+0000", 2024-09-21T15:00:19.988 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5120000000000002e-05 2024-09-21T15:00:19.988 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.988 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.988 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:19.988 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.322694+0000", 2024-09-21T15:00:19.988 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8150000000000001e-05 2024-09-21T15:00:19.988 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.988 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.988 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:19.988 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.322781+0000", 2024-09-21T15:00:19.988 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.687e-05 2024-09-21T15:00:19.989 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.989 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.989 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:19.989 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.323108+0000", 2024-09-21T15:00:19.989 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000327363 2024-09-21T15:00:19.989 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.989 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.989 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 1", 2024-09-21T15:00:19.989 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.325174+0000", 2024-09-21T15:00:19.989 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0020656960000000001 2024-09-21T15:00:19.989 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.989 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.989 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:19.989 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.359764+0000", 2024-09-21T15:00:19.989 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034589987000000003 2024-09-21T15:00:19.989 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.990 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.990 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:19.990 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.359774+0000", 2024-09-21T15:00:19.990 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8730000000000002e-06 2024-09-21T15:00:19.990 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.990 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.990 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:19.990 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.359948+0000", 2024-09-21T15:00:19.990 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000174088 2024-09-21T15:00:19.990 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.990 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.990 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:19.990 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.359993+0000", 2024-09-21T15:00:19.990 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4706000000000002e-05 2024-09-21T15:00:19.991 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:19.991 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:19.991 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:19.991 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.991 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.991 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:97673 3.1d e122/24 3:bd91946e:::rbd_data.1136bfc79a70.0000000000000002:head v 122'162, mlcod=120'160)", 2024-09-21T15:00:19.991 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.313730+0000", 2024-09-21T15:00:19.991 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.609735329999999, 2024-09-21T15:00:19.991 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055373462999999998, 2024-09-21T15:00:19.991 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:19.991 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:19.991 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:19.991 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.991 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:19.991 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.313730+0000", 2024-09-21T15:00:19.992 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:19.992 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.992 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.992 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:19.992 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.313730+0000", 2024-09-21T15:00:19.992 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:19.992 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.992 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.992 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:19.992 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.313730+0000", 2024-09-21T15:00:19.992 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:19.992 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.992 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.992 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:19.992 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.336113+0000", 2024-09-21T15:00:19.993 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022382962999999999 2024-09-21T15:00:19.993 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.993 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.993 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:19.993 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.336140+0000", 2024-09-21T15:00:19.993 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7248999999999999e-05 2024-09-21T15:00:19.993 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.993 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.993 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:19.993 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.336151+0000", 2024-09-21T15:00:19.993 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1015e-05 2024-09-21T15:00:19.993 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.993 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.993 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:19.994 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.355428+0000", 2024-09-21T15:00:19.994 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019276379999999999 2024-09-21T15:00:19.994 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.994 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.994 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:19.994 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.355484+0000", 2024-09-21T15:00:19.994 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6820999999999997e-05 2024-09-21T15:00:19.994 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.994 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.994 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:19.994 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.369040+0000", 2024-09-21T15:00:19.994 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013555466 2024-09-21T15:00:19.994 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.995 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.995 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:19.995 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.369104+0000", 2024-09-21T15:00:19.995 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3986999999999996e-05 2024-09-21T15:00:19.995 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:19.995 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:19.995 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:19.995 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.995 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.995 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:97684 3.15 3:aaa1da52:::rbd_data.1136b9c49b1c.0000000000000003:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:19.995 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.331547+0000", 2024-09-21T15:00:19.995 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.591919224, 2024-09-21T15:00:19.995 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024267581, 2024-09-21T15:00:19.995 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:19.996 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:19.996 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:19.996 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:19.996 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:19.996 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 97684 2024-09-21T15:00:19.996 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.996 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:19.996 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.996 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:19.996 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.331547+0000", 2024-09-21T15:00:19.996 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:19.996 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.996 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.996 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:19.996 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.331547+0000", 2024-09-21T15:00:19.997 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:19.997 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.997 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.997 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:19.997 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.331544+0000", 2024-09-21T15:00:19.997 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T15:00:19.997 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.997 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.997 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:19.997 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.331553+0000", 2024-09-21T15:00:19.997 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8650000000000007e-06 2024-09-21T15:00:19.997 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.997 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.997 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:19.998 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.331571+0000", 2024-09-21T15:00:19.998 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8031000000000001e-05 2024-09-21T15:00:19.998 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.998 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.998 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:19.998 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.331582+0000", 2024-09-21T15:00:19.998 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0811e-05 2024-09-21T15:00:19.998 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.998 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.998 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:19.998 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.355617+0000", 2024-09-21T15:00:19.998 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024035117000000002 2024-09-21T15:00:19.998 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.998 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.999 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:19.999 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.355712+0000", 2024-09-21T15:00:19.999 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5297000000000006e-05 2024-09-21T15:00:19.999 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.999 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.999 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:19.999 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.355814+0000", 2024-09-21T15:00:19.999 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000101668 2024-09-21T15:00:19.999 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:19.999 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:19.999 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:19.999 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:19.999 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:19.999 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:97688 3.f 3:f6b1418f:::rbd_data.1136b9c49b1c.0000000000000007:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:19.999 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.331623+0000", 2024-09-21T15:00:20.000 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.591843223, 2024-09-21T15:00:20.000 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026357163999999999, 2024-09-21T15:00:20.000 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:20.000 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:20.000 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:20.000 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:20.000 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:20.000 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 97688 2024-09-21T15:00:20.000 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.000 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:20.000 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.000 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:20.000 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.331623+0000", 2024-09-21T15:00:20.001 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.001 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.001 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.001 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:20.001 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.331623+0000", 2024-09-21T15:00:20.001 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.001 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.001 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.001 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:20.001 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.331622+0000", 2024-09-21T15:00:20.001 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:00:20.001 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.001 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.001 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:20.001 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.331627+0000", 2024-09-21T15:00:20.002 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3199999999999999e-06 2024-09-21T15:00:20.002 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.002 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.002 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:20.002 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.331636+0000", 2024-09-21T15:00:20.002 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2070000000000001e-06 2024-09-21T15:00:20.002 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.002 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.002 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:20.002 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.331642+0000", 2024-09-21T15:00:20.002 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2539999999999998e-06 2024-09-21T15:00:20.002 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.002 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.002 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:20.002 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.357765+0000", 2024-09-21T15:00:20.003 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.026122777999999999 2024-09-21T15:00:20.003 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.003 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.003 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:20.003 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.357872+0000", 2024-09-21T15:00:20.003 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010690400000000001 2024-09-21T15:00:20.003 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.003 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.003 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:20.003 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.357980+0000", 2024-09-21T15:00:20.003 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010765200000000001 2024-09-21T15:00:20.003 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.003 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:20.004 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.004 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.004 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.004 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:97701 3.f 3:f043bda3:::rbd_data.1136b9c49b1c.0000000000000014:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:20.004 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.331656+0000", 2024-09-21T15:00:20.004 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.591809767000001, 2024-09-21T15:00:20.004 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.027952106000000001, 2024-09-21T15:00:20.004 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:20.004 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:20.004 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:20.004 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:20.004 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:20.004 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 97701 2024-09-21T15:00:20.004 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.005 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:20.005 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.005 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:20.005 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.331656+0000", 2024-09-21T15:00:20.005 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.005 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.005 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.005 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:20.005 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.331656+0000", 2024-09-21T15:00:20.005 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.005 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.005 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.005 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:20.005 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.331656+0000", 2024-09-21T15:00:20.005 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:20.006 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.006 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.006 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:20.006 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.331659+0000", 2024-09-21T15:00:20.006 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4640000000000002e-06 2024-09-21T15:00:20.006 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.006 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.006 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:20.006 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.333832+0000", 2024-09-21T15:00:20.006 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0021731820000000001 2024-09-21T15:00:20.006 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.006 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.006 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:20.006 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.333848+0000", 2024-09-21T15:00:20.007 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6107e-05 2024-09-21T15:00:20.007 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.007 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.007 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:20.007 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.358014+0000", 2024-09-21T15:00:20.007 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024165498000000001 2024-09-21T15:00:20.007 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.007 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.007 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:20.007 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.359472+0000", 2024-09-21T15:00:20.007 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001457999 2024-09-21T15:00:20.007 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.007 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.007 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:20.008 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.359608+0000", 2024-09-21T15:00:20.008 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013634899999999999 2024-09-21T15:00:20.008 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.008 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:20.008 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.008 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.008 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.008 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:97702 3.15 3:aee50b17:::rbd_data.1136b9c49b1c.0000000000000015:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:20.008 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.333871+0000", 2024-09-21T15:00:20.008 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.589595211000001, 2024-09-21T15:00:20.008 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022283784000000001, 2024-09-21T15:00:20.008 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:20.008 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:20.008 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:20.008 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:20.009 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:20.009 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 97702 2024-09-21T15:00:20.009 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.009 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:20.009 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.009 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:20.009 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.333871+0000", 2024-09-21T15:00:20.009 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.009 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.009 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.009 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:20.009 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.333871+0000", 2024-09-21T15:00:20.009 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.009 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.009 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.010 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:20.010 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.333868+0000", 2024-09-21T15:00:20.010 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T15:00:20.010 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.010 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.010 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:20.010 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.333881+0000", 2024-09-21T15:00:20.010 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2896e-05 2024-09-21T15:00:20.010 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.010 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.010 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:20.010 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.333890+0000", 2024-09-21T15:00:20.010 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.7900000000000005e-06 2024-09-21T15:00:20.010 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.010 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.011 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:20.011 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.333896+0000", 2024-09-21T15:00:20.011 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7060000000000001e-06 2024-09-21T15:00:20.011 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.011 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.011 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:20.011 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.355845+0000", 2024-09-21T15:00:20.011 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021948834 2024-09-21T15:00:20.011 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.011 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.011 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:20.011 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.355997+0000", 2024-09-21T15:00:20.011 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015249999999999999 2024-09-21T15:00:20.012 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.012 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.012 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:20.012 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.356154+0000", 2024-09-21T15:00:20.012 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000157267 2024-09-21T15:00:20.012 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.012 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:20.012 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.012 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.012 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.012 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:97712 3.f 3:f31de823:::rbd_data.1136b9c49b1c.000000000000001f:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:20.012 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.333902+0000", 2024-09-21T15:00:20.012 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.589563675000001, 2024-09-21T15:00:20.012 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025825147999999999, 2024-09-21T15:00:20.013 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:20.013 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:20.013 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:20.013 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:20.013 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:20.013 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 97712 2024-09-21T15:00:20.013 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.013 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:20.013 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.013 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:20.013 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.333902+0000", 2024-09-21T15:00:20.013 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.013 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.013 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.014 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:20.014 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.333902+0000", 2024-09-21T15:00:20.014 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.014 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.014 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.014 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:20.014 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.333902+0000", 2024-09-21T15:00:20.014 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:20.014 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.014 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.014 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:20.014 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.333905+0000", 2024-09-21T15:00:20.014 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6449999999999998e-06 2024-09-21T15:00:20.014 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.015 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.015 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:20.015 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.333911+0000", 2024-09-21T15:00:20.015 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9399999999999999e-06 2024-09-21T15:00:20.015 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.015 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.015 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:20.015 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.333916+0000", 2024-09-21T15:00:20.015 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8130000000000003e-06 2024-09-21T15:00:20.015 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.015 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.015 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:20.015 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.359634+0000", 2024-09-21T15:00:20.015 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025718039000000002 2024-09-21T15:00:20.016 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.016 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.016 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:20.016 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.359727+0000", 2024-09-21T15:00:20.016 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.3068999999999999e-05 2024-09-21T15:00:20.016 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.016 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:20.016 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.016 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.016 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.016 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:97718 3.15 3:afbff86b:::rbd_data.1136b9c49b1c.0000000000000025:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:20.016 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.333921+0000", 2024-09-21T15:00:20.016 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.58954469, 2024-09-21T15:00:20.016 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022577750000000001, 2024-09-21T15:00:20.017 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:20.017 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:20.017 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:20.017 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:20.017 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:20.017 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 97718 2024-09-21T15:00:20.017 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.017 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:20.017 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.017 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:20.017 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.333921+0000", 2024-09-21T15:00:20.017 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.017 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.017 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.018 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:20.018 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.333921+0000", 2024-09-21T15:00:20.018 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.018 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.018 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.018 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:20.018 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.333921+0000", 2024-09-21T15:00:20.018 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:20.018 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.018 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.018 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:20.018 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.333924+0000", 2024-09-21T15:00:20.018 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9110000000000002e-06 2024-09-21T15:00:20.019 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.019 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.019 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:20.019 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.333929+0000", 2024-09-21T15:00:20.019 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5659999999999997e-06 2024-09-21T15:00:20.019 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.019 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.019 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:20.019 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.333935+0000", 2024-09-21T15:00:20.019 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9020000000000004e-06 2024-09-21T15:00:20.019 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.019 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.019 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:20.020 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.356229+0000", 2024-09-21T15:00:20.020 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022294389000000001 2024-09-21T15:00:20.020 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.020 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.020 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:20.020 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.356406+0000", 2024-09-21T15:00:20.020 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017647800000000001 2024-09-21T15:00:20.020 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.020 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.020 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:20.020 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.356499+0000", 2024-09-21T15:00:20.020 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2928000000000002e-05 2024-09-21T15:00:20.020 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.020 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:20.021 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.021 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.021 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:20.021 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:20.022 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T15:00:20.023 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:00:20.018+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:00:20.123 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:00:20.118+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:00:20.142 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:20.143 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:00:20.143 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T15:00:20.143 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:20.150 INFO:teuthology.orchestra.run.smithi032.stderr:nodeep-scrub is set 2024-09-21T15:00:20.157 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T15:00:20.224 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:00:20.218+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:00:20.246 INFO:teuthology.orchestra.run.smithi032.stdout:checking clone #25, image image_client.0-clone25 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent26 2024-09-21T15:00:20.266 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:20.266 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:00:20.266 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T15:00:20.266 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T15:00:20.266 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:20.277 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T15:00:20.323 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:00:20.320+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:00:20.394 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:20.394 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T15:00:20.394 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T15:00:20.394 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T15:00:20.394 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.394 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T15:00:20.394 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.205754+0000", 2024-09-21T15:00:20.394 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 137.185751111, 2024-09-21T15:00:20.394 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.41117094199999998, 2024-09-21T15:00:20.394 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:20.395 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:20.395 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:20.395 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T15:00:20.395 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T15:00:20.395 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1 2024-09-21T15:00:20.395 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.395 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:20.395 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.395 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:20.395 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205754+0000", 2024-09-21T15:00:20.395 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.395 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.395 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.396 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:20.396 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205754+0000", 2024-09-21T15:00:20.396 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.396 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.396 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.396 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:20.396 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205752+0000", 2024-09-21T15:00:20.396 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T15:00:20.396 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.396 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.396 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:20.396 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205765+0000", 2024-09-21T15:00:20.396 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3230999999999999e-05 2024-09-21T15:00:20.396 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.397 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.397 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:20.397 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205782+0000", 2024-09-21T15:00:20.397 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6634e-05 2024-09-21T15:00:20.397 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.397 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.397 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:20.397 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205793+0000", 2024-09-21T15:00:20.397 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1297e-05 2024-09-21T15:00:20.397 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.397 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.397 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:20.397 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205867+0000", 2024-09-21T15:00:20.397 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4269000000000003e-05 2024-09-21T15:00:20.398 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.398 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.398 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:00:20.398 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205877+0000", 2024-09-21T15:00:20.398 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2690000000000005e-06 2024-09-21T15:00:20.398 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.398 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.398 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:20.398 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.613700+0000", 2024-09-21T15:00:20.398 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.407823455 2024-09-21T15:00:20.398 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.398 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.398 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:20.398 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.614203+0000", 2024-09-21T15:00:20.399 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00050311500000000005 2024-09-21T15:00:20.399 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.399 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.399 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T15:00:20.399 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.614750+0000", 2024-09-21T15:00:20.399 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00054738699999999998 2024-09-21T15:00:20.399 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.399 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.399 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:20.399 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616367+0000", 2024-09-21T15:00:20.399 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001616634 2024-09-21T15:00:20.399 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.399 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.399 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:20.400 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616574+0000", 2024-09-21T15:00:20.400 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00020668499999999999 2024-09-21T15:00:20.400 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.400 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.400 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:20.400 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616802+0000", 2024-09-21T15:00:20.400 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00022817900000000001 2024-09-21T15:00:20.400 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.400 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.400 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:20.400 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616925+0000", 2024-09-21T15:00:20.400 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012309699999999999 2024-09-21T15:00:20.400 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.400 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:20.401 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.401 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.401 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.401 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T15:00:20.401 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:04.493967+0000", 2024-09-21T15:00:20.401 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 135.89753872399999, 2024-09-21T15:00:20.401 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.015281075999999999, 2024-09-21T15:00:20.401 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:20.401 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:20.401 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:20.401 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T15:00:20.401 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T15:00:20.401 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 141 2024-09-21T15:00:20.401 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.401 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:20.401 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.402 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:20.402 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493967+0000", 2024-09-21T15:00:20.402 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.402 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.402 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.402 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:20.402 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493967+0000", 2024-09-21T15:00:20.402 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.402 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.403 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.403 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:20.403 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493965+0000", 2024-09-21T15:00:20.403 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T15:00:20.403 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.403 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.403 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:20.403 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493987+0000", 2024-09-21T15:00:20.403 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1662999999999999e-05 2024-09-21T15:00:20.403 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.403 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.403 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:20.403 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.494004+0000", 2024-09-21T15:00:20.403 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7311000000000001e-05 2024-09-21T15:00:20.404 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.404 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.404 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:20.404 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.494017+0000", 2024-09-21T15:00:20.404 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3202e-05 2024-09-21T15:00:20.404 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.404 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.404 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:20.404 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507678+0000", 2024-09-21T15:00:20.404 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013660432 2024-09-21T15:00:20.404 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.404 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.404 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:20.404 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507726+0000", 2024-09-21T15:00:20.404 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8154e-05 2024-09-21T15:00:20.405 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.405 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.405 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T15:00:20.405 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507797+0000", 2024-09-21T15:00:20.405 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.1110000000000002e-05 2024-09-21T15:00:20.405 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.405 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.405 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:20.405 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508710+0000", 2024-09-21T15:00:20.405 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00091339299999999997 2024-09-21T15:00:20.405 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.405 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.405 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:20.405 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.509108+0000", 2024-09-21T15:00:20.406 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00039718300000000001 2024-09-21T15:00:20.406 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.406 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.406 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:20.406 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.509186+0000", 2024-09-21T15:00:20.406 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8592999999999998e-05 2024-09-21T15:00:20.406 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.406 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.406 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:20.406 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.509248+0000", 2024-09-21T15:00:20.406 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1450000000000003e-05 2024-09-21T15:00:20.406 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.406 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:20.406 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.407 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.407 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.407 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6639 3.1a 3:592d80ed:::rbd_data.11361bed1f78.0000000000000005:d [read 0~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T15:00:20.407 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131650+0000", 2024-09-21T15:00:20.407 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 105.259855563, 2024-09-21T15:00:20.407 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016002832000000002, 2024-09-21T15:00:20.407 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:20.407 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:20.407 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:20.407 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:20.407 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:20.407 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6639 2024-09-21T15:00:20.407 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.407 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:20.407 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.408 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:20.408 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131650+0000", 2024-09-21T15:00:20.408 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.408 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.408 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.408 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:20.408 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131650+0000", 2024-09-21T15:00:20.408 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.408 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.408 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.408 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:20.408 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131649+0000", 2024-09-21T15:00:20.408 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:20.408 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.408 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.409 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:20.409 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131651+0000", 2024-09-21T15:00:20.409 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9199999999999998e-06 2024-09-21T15:00:20.409 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.409 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.409 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:20.409 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131656+0000", 2024-09-21T15:00:20.409 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9899999999999997e-06 2024-09-21T15:00:20.409 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.409 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.409 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:20.409 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131665+0000", 2024-09-21T15:00:20.409 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0999999999999993e-06 2024-09-21T15:00:20.409 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.409 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.410 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:20.410 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146309+0000", 2024-09-21T15:00:20.410 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014643893999999999 2024-09-21T15:00:20.410 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.410 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.410 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:20.410 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146392+0000", 2024-09-21T15:00:20.410 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2531e-05 2024-09-21T15:00:20.410 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.410 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.410 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:20.410 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147653+0000", 2024-09-21T15:00:20.410 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0012606710000000001 2024-09-21T15:00:20.410 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.411 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:20.411 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.411 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.411 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.411 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6640 3.1a 3:592d80ed:::rbd_data.11361bed1f78.0000000000000005:d [read 65536~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T15:00:20.411 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131670+0000", 2024-09-21T15:00:20.411 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 105.259835264, 2024-09-21T15:00:20.411 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016188859, 2024-09-21T15:00:20.411 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:20.411 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:20.411 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:20.411 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:20.411 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:20.411 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6640 2024-09-21T15:00:20.411 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.412 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:20.412 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.412 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:20.412 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131670+0000", 2024-09-21T15:00:20.412 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.412 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.412 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.412 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:20.412 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131670+0000", 2024-09-21T15:00:20.412 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.412 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.412 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.412 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:20.412 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131670+0000", 2024-09-21T15:00:20.413 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:00:20.413 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.413 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.413 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:20.413 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131672+0000", 2024-09-21T15:00:20.413 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.818e-06 2024-09-21T15:00:20.413 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.413 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.413 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:20.413 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131675+0000", 2024-09-21T15:00:20.413 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5930000000000002e-06 2024-09-21T15:00:20.413 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.413 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.413 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:20.413 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131678+0000", 2024-09-21T15:00:20.413 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6790000000000001e-06 2024-09-21T15:00:20.414 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.414 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.414 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:20.414 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147681+0000", 2024-09-21T15:00:20.414 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016002755 2024-09-21T15:00:20.414 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.414 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.414 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:20.414 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147762+0000", 2024-09-21T15:00:20.414 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1052999999999998e-05 2024-09-21T15:00:20.414 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.414 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.414 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:20.415 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147859+0000", 2024-09-21T15:00:20.415 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.7199000000000002e-05 2024-09-21T15:00:20.415 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.415 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:20.415 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.415 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.415 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.415 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6641 3.1a 3:592d80ed:::rbd_data.11361bed1f78.0000000000000005:d [read 147456~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T15:00:20.415 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131680+0000", 2024-09-21T15:00:20.415 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 105.259824976, 2024-09-21T15:00:20.415 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016431296000000001, 2024-09-21T15:00:20.415 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:20.415 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:20.415 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:20.415 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:20.416 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:20.416 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6641 2024-09-21T15:00:20.416 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.416 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:20.416 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.416 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:20.416 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131680+0000", 2024-09-21T15:00:20.416 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.416 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.416 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.416 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:20.416 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131680+0000", 2024-09-21T15:00:20.416 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.416 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.416 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.416 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:20.417 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131680+0000", 2024-09-21T15:00:20.417 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:00:20.417 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.417 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.417 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:20.417 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131681+0000", 2024-09-21T15:00:20.417 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2610000000000001e-06 2024-09-21T15:00:20.417 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.417 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.417 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:20.417 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131685+0000", 2024-09-21T15:00:20.417 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.8299999999999998e-06 2024-09-21T15:00:20.417 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.417 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.418 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:20.418 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131688+0000", 2024-09-21T15:00:20.418 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6189999999999998e-06 2024-09-21T15:00:20.418 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.418 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.418 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:20.418 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147884+0000", 2024-09-21T15:00:20.418 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016196465 2024-09-21T15:00:20.418 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.418 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.418 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:20.418 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147949+0000", 2024-09-21T15:00:20.418 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4867000000000001e-05 2024-09-21T15:00:20.418 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.418 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.419 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:20.419 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.148112+0000", 2024-09-21T15:00:20.419 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00016247700000000001 2024-09-21T15:00:20.419 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.419 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:20.419 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.419 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.419 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.419 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6642 3.1a 3:592d80ed:::rbd_data.11361bed1f78.0000000000000005:d [read 212992~32768 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T15:00:20.419 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131690+0000", 2024-09-21T15:00:20.419 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 105.259815098, 2024-09-21T15:00:20.419 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016751335999999999, 2024-09-21T15:00:20.419 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:20.419 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:20.419 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:20.420 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:20.420 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:20.420 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6642 2024-09-21T15:00:20.420 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.420 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:20.420 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.420 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:20.420 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131690+0000", 2024-09-21T15:00:20.420 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.420 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.420 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.420 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:20.420 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131690+0000", 2024-09-21T15:00:20.421 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.421 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.421 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.421 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:20.421 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131690+0000", 2024-09-21T15:00:20.421 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:00:20.421 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.421 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.421 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:20.421 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131691+0000", 2024-09-21T15:00:20.421 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3120000000000001e-06 2024-09-21T15:00:20.421 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.421 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.422 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:20.422 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131694+0000", 2024-09-21T15:00:20.422 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0070000000000002e-06 2024-09-21T15:00:20.422 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.422 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.422 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:20.422 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131700+0000", 2024-09-21T15:00:20.422 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0800000000000002e-06 2024-09-21T15:00:20.422 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.422 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.422 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:20.422 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.148135+0000", 2024-09-21T15:00:20.422 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016434250000000001 2024-09-21T15:00:20.422 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.423 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.423 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:20.423 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.148204+0000", 2024-09-21T15:00:20.423 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9466000000000002e-05 2024-09-21T15:00:20.423 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.423 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.423 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:20.423 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.148441+0000", 2024-09-21T15:00:20.423 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00023743700000000001 2024-09-21T15:00:20.423 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.423 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:20.423 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.423 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.423 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.423 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:61214 3.7 3:e28aec10:::rbd_header.1136862f872f:head [watch reconnect cookie 139864671932080 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-21T15:00:20.424 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:18.147172+0000", 2024-09-21T15:00:20.424 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 62.244333468000001, 2024-09-21T15:00:20.424 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.019264956, 2024-09-21T15:00:20.424 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:20.424 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:20.424 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:20.424 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:20.424 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:20.424 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 61214 2024-09-21T15:00:20.424 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.424 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:20.424 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.424 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:20.424 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147172+0000", 2024-09-21T15:00:20.424 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.424 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.425 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.425 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:20.425 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147172+0000", 2024-09-21T15:00:20.425 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.425 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.425 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.425 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:20.425 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147168+0000", 2024-09-21T15:00:20.425 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999962 2024-09-21T15:00:20.425 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.425 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.425 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:20.425 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147188+0000", 2024-09-21T15:00:20.425 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9717000000000001e-05 2024-09-21T15:00:20.425 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.426 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.426 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:20.426 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147239+0000", 2024-09-21T15:00:20.426 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1493999999999999e-05 2024-09-21T15:00:20.426 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.426 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.426 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:20.426 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147265+0000", 2024-09-21T15:00:20.426 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5746999999999999e-05 2024-09-21T15:00:20.426 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.426 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.426 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:20.426 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147372+0000", 2024-09-21T15:00:20.426 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010681200000000001 2024-09-21T15:00:20.427 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.427 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.427 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op must wait for map", 2024-09-21T15:00:20.427 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147380+0000", 2024-09-21T15:00:20.427 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7919999999999993e-06 2024-09-21T15:00:20.427 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.427 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.427 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:20.427 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.154025+0000", 2024-09-21T15:00:20.427 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0066454909999999999 2024-09-21T15:00:20.427 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.427 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.427 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:00:20.427 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.154041+0000", 2024-09-21T15:00:20.427 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5894000000000001e-05 2024-09-21T15:00:20.428 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.428 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.428 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:20.428 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.164345+0000", 2024-09-21T15:00:20.428 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0103040459999999 2024-09-21T15:00:20.428 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.428 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.428 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:20.428 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.164590+0000", 2024-09-21T15:00:20.428 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00024492699999999998 2024-09-21T15:00:20.428 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.428 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.428 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:20.428 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.166437+0000", 2024-09-21T15:00:20.429 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0018466330000000001 2024-09-21T15:00:20.429 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.429 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:20.429 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.429 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.429 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.429 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:61215 3.7 3:e28aec10:::rbd_header.1136862f872f:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-21T15:00:20.429 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:18.152008+0000", 2024-09-21T15:00:20.429 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 62.239497016000001, 2024-09-21T15:00:20.429 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.015644958, 2024-09-21T15:00:20.429 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:20.429 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:20.429 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:20.429 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:20.430 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:20.430 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 61215 2024-09-21T15:00:20.430 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.430 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:20.430 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.430 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:20.430 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152008+0000", 2024-09-21T15:00:20.430 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.430 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.430 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.430 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:20.430 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152008+0000", 2024-09-21T15:00:20.430 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.430 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.431 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.431 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:20.431 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152005+0000", 2024-09-21T15:00:20.431 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-21T15:00:20.431 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.431 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.431 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:20.431 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152025+0000", 2024-09-21T15:00:20.431 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9306000000000001e-05 2024-09-21T15:00:20.431 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.431 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.431 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:20.431 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152054+0000", 2024-09-21T15:00:20.431 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9189000000000001e-05 2024-09-21T15:00:20.431 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.432 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.432 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:20.432 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152078+0000", 2024-09-21T15:00:20.432 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4317000000000001e-05 2024-09-21T15:00:20.432 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.432 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.432 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:20.432 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152243+0000", 2024-09-21T15:00:20.432 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000164802 2024-09-21T15:00:20.432 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.432 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.432 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting_for_map not empty", 2024-09-21T15:00:20.432 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152251+0000", 2024-09-21T15:00:20.432 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0600000000000008e-06 2024-09-21T15:00:20.433 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.433 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.433 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:20.433 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.154741+0000", 2024-09-21T15:00:20.433 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0024900069999999998 2024-09-21T15:00:20.433 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.433 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.433 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:00:20.433 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.154753+0000", 2024-09-21T15:00:20.433 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1588e-05 2024-09-21T15:00:20.433 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.433 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.433 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:20.433 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.165380+0000", 2024-09-21T15:00:20.433 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0106272249999999 2024-09-21T15:00:20.434 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.434 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.434 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:20.434 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.165517+0000", 2024-09-21T15:00:20.434 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013737999999999999 2024-09-21T15:00:20.434 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.434 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.434 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 6", 2024-09-21T15:00:20.434 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.165938+0000", 2024-09-21T15:00:20.434 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00042082399999999998 2024-09-21T15:00:20.434 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.434 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.434 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:20.434 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.166795+0000", 2024-09-21T15:00:20.435 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00085674899999999997 2024-09-21T15:00:20.435 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.435 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.435 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:20.435 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.167598+0000", 2024-09-21T15:00:20.435 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00080335599999999997 2024-09-21T15:00:20.435 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.435 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.435 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:20.435 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.167632+0000", 2024-09-21T15:00:20.435 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4122000000000003e-05 2024-09-21T15:00:20.435 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.435 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.435 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:20.436 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.167653+0000", 2024-09-21T15:00:20.436 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0862000000000001e-05 2024-09-21T15:00:20.436 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.436 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:20.436 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.436 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.436 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.436 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:73698 3.10 3:0a4d95f6:::rbd_data.1136f4939602.000000000000000b:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-21T15:00:20.436 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:41.969870+0000", 2024-09-21T15:00:20.436 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 38.421634877999999, 2024-09-21T15:00:20.436 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.017502461, 2024-09-21T15:00:20.436 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:20.436 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:20.436 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:20.436 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:20.437 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:20.437 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 73698 2024-09-21T15:00:20.437 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.437 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:20.437 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.437 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:20.437 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.969870+0000", 2024-09-21T15:00:20.437 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.437 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.437 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.437 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:20.437 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.969870+0000", 2024-09-21T15:00:20.437 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.438 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.438 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.438 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:20.438 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.969868+0000", 2024-09-21T15:00:20.438 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T15:00:20.438 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.438 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.438 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:20.438 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.969885+0000", 2024-09-21T15:00:20.438 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7416999999999999e-05 2024-09-21T15:00:20.438 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.438 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.438 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:20.438 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.969918+0000", 2024-09-21T15:00:20.438 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3133000000000003e-05 2024-09-21T15:00:20.439 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.439 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.439 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:20.439 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.969934+0000", 2024-09-21T15:00:20.439 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6027000000000001e-05 2024-09-21T15:00:20.439 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.439 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.439 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:20.439 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.970021+0000", 2024-09-21T15:00:20.439 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6907e-05 2024-09-21T15:00:20.439 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.439 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.439 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:20.439 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.970148+0000", 2024-09-21T15:00:20.440 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012701499999999999 2024-09-21T15:00:20.440 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.440 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.440 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T15:00:20.440 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.972971+0000", 2024-09-21T15:00:20.440 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.002822873 2024-09-21T15:00:20.440 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.440 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.440 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:20.440 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.987124+0000", 2024-09-21T15:00:20.440 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0141529 2024-09-21T15:00:20.440 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.440 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.440 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:20.440 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.987146+0000", 2024-09-21T15:00:20.441 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1783000000000001e-05 2024-09-21T15:00:20.441 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.441 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.441 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:20.441 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.987258+0000", 2024-09-21T15:00:20.441 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000112555 2024-09-21T15:00:20.441 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.441 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.441 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:20.441 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.987373+0000", 2024-09-21T15:00:20.441 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000114434 2024-09-21T15:00:20.441 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.441 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:20.441 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.441 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.442 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.442 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:97638 3.1a 3:5e573525:::rbd_data.1136b9c49b1c.0000000000000026:49 [read 1277952~49152 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:20.442 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.276202+0000", 2024-09-21T15:00:20.442 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 15.115302773, 2024-09-21T15:00:20.442 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.015360357999999999, 2024-09-21T15:00:20.442 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:20.442 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:20.442 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:20.442 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:20.442 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:20.442 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 97638 2024-09-21T15:00:20.442 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.442 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:20.442 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.442 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:20.443 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276202+0000", 2024-09-21T15:00:20.443 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.443 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.443 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.443 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:20.443 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276202+0000", 2024-09-21T15:00:20.443 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.443 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.443 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.443 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:20.443 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276202+0000", 2024-09-21T15:00:20.443 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:20.443 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.443 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.443 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:20.444 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276204+0000", 2024-09-21T15:00:20.444 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5359999999999999e-06 2024-09-21T15:00:20.444 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.444 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.444 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:20.444 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276210+0000", 2024-09-21T15:00:20.444 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3040000000000004e-06 2024-09-21T15:00:20.444 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.444 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.444 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:20.444 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276214+0000", 2024-09-21T15:00:20.444 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6779999999999999e-06 2024-09-21T15:00:20.444 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.444 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.444 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:20.445 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.290563+0000", 2024-09-21T15:00:20.445 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014348346 2024-09-21T15:00:20.445 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.445 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.445 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:20.445 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.290623+0000", 2024-09-21T15:00:20.445 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0001999999999999e-05 2024-09-21T15:00:20.445 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.445 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.445 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:20.445 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.291563+0000", 2024-09-21T15:00:20.445 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00094003100000000003 2024-09-21T15:00:20.445 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.445 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:20.446 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.446 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.446 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.446 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:97678 3.d e122/24 3:b2f34c6d:::rbd_data.1136bfc79a70.0000000000000006:head v 122'234, mlcod=120'232)", 2024-09-21T15:00:20.446 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.321655+0000", 2024-09-21T15:00:20.446 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 15.069850099, 2024-09-21T15:00:20.446 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024509388999999999, 2024-09-21T15:00:20.446 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:20.446 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:20.446 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:20.446 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.446 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:20.446 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.321655+0000", 2024-09-21T15:00:20.447 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.447 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.447 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.447 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:20.447 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.321655+0000", 2024-09-21T15:00:20.447 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.447 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.447 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.447 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:20.447 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.321655+0000", 2024-09-21T15:00:20.447 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:20.447 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.447 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.447 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:20.448 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.329453+0000", 2024-09-21T15:00:20.448 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0077980560000000003 2024-09-21T15:00:20.448 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.448 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.448 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:20.448 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.329482+0000", 2024-09-21T15:00:20.448 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9153999999999999e-05 2024-09-21T15:00:20.448 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.448 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.448 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:20.448 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.329493+0000", 2024-09-21T15:00:20.448 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1103e-05 2024-09-21T15:00:20.448 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.448 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.449 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:20.449 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.329555+0000", 2024-09-21T15:00:20.449 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2345e-05 2024-09-21T15:00:20.449 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.449 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.449 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:20.449 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.329589+0000", 2024-09-21T15:00:20.449 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3908999999999997e-05 2024-09-21T15:00:20.449 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.449 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.449 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:20.449 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.346083+0000", 2024-09-21T15:00:20.449 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016493383 2024-09-21T15:00:20.449 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.450 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.450 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:20.450 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.346165+0000", 2024-09-21T15:00:20.450 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1796000000000001e-05 2024-09-21T15:00:20.450 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.450 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:20.450 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.450 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.450 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.450 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:97679 3.f e122/24 3:f2a362b4:::rbd_data.1136bfc79a70.0000000000000007:head v 122'331, mlcod=120'329)", 2024-09-21T15:00:20.450 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.337145+0000", 2024-09-21T15:00:20.451 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 15.054360089999999, 2024-09-21T15:00:20.451 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021912411, 2024-09-21T15:00:20.451 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:20.451 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:20.451 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:20.451 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.451 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:20.451 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.337145+0000", 2024-09-21T15:00:20.451 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.451 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.451 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.451 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:20.451 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.337145+0000", 2024-09-21T15:00:20.451 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.452 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.452 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.452 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:20.452 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.337145+0000", 2024-09-21T15:00:20.452 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:20.452 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.452 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.452 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:20.452 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.343899+0000", 2024-09-21T15:00:20.452 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0067541420000000003 2024-09-21T15:00:20.452 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.452 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.452 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:20.452 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.343921+0000", 2024-09-21T15:00:20.452 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2255e-05 2024-09-21T15:00:20.453 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.453 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.453 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:20.453 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.343933+0000", 2024-09-21T15:00:20.453 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1528e-05 2024-09-21T15:00:20.453 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.453 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.453 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:20.453 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.344001+0000", 2024-09-21T15:00:20.453 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8269999999999995e-05 2024-09-21T15:00:20.453 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.453 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.453 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:20.453 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.344029+0000", 2024-09-21T15:00:20.454 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8254000000000001e-05 2024-09-21T15:00:20.454 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.454 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.454 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:20.454 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.358901+0000", 2024-09-21T15:00:20.454 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0148719 2024-09-21T15:00:20.454 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.454 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.454 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:20.454 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.359058+0000", 2024-09-21T15:00:20.454 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000156446 2024-09-21T15:00:20.454 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.454 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:20.454 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.454 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.455 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.455 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:97676 3.1f e122/24 3:ffaa836a:::rbd_data.1136bfc79a70.0000000000000004:head v 122'312, mlcod=120'310)", 2024-09-21T15:00:20.455 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.348683+0000", 2024-09-21T15:00:20.455 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 15.042821925, 2024-09-21T15:00:20.455 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.033768802000000001, 2024-09-21T15:00:20.455 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:20.455 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:20.455 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:20.455 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.455 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:20.455 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.348683+0000", 2024-09-21T15:00:20.455 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.455 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.455 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.455 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:20.456 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.348683+0000", 2024-09-21T15:00:20.456 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.456 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.456 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.456 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:20.456 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.348683+0000", 2024-09-21T15:00:20.456 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:00:20.456 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.456 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.456 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:20.456 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.366937+0000", 2024-09-21T15:00:20.456 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018254415 2024-09-21T15:00:20.456 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.456 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.456 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:20.457 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.366959+0000", 2024-09-21T15:00:20.457 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1634e-05 2024-09-21T15:00:20.457 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.457 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.457 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:20.457 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.366971+0000", 2024-09-21T15:00:20.457 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2305e-05 2024-09-21T15:00:20.457 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.457 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.457 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:20.457 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.367034+0000", 2024-09-21T15:00:20.457 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3211000000000002e-05 2024-09-21T15:00:20.457 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.457 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.457 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:20.457 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.367056+0000", 2024-09-21T15:00:20.458 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2098000000000001e-05 2024-09-21T15:00:20.458 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.458 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.458 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:20.458 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.382400+0000", 2024-09-21T15:00:20.458 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.015343867000000001 2024-09-21T15:00:20.458 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.458 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.458 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:20.458 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.382452+0000", 2024-09-21T15:00:20.458 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1992e-05 2024-09-21T15:00:20.458 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.458 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:20.458 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.459 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.459 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.459 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:97734 3.19 e122/94 3:9fcf754f:::rbd_data.1136bfc79a70.0000000000000009:head v 122'314, mlcod=120'312)", 2024-09-21T15:00:20.459 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.378547+0000", 2024-09-21T15:00:20.459 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 15.012957947, 2024-09-21T15:00:20.459 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.063380675999999997, 2024-09-21T15:00:20.459 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:20.459 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:20.459 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:20.459 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.459 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:20.459 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.378547+0000", 2024-09-21T15:00:20.459 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.459 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.460 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.460 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:20.460 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.378547+0000", 2024-09-21T15:00:20.460 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.460 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.460 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.460 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:20.460 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.378547+0000", 2024-09-21T15:00:20.460 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:00:20.460 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.460 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.460 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:20.460 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.425118+0000", 2024-09-21T15:00:20.460 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046571166999999997 2024-09-21T15:00:20.461 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.461 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.461 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:20.461 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.425150+0000", 2024-09-21T15:00:20.461 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2150999999999998e-05 2024-09-21T15:00:20.461 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.461 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.461 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:20.461 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.425164+0000", 2024-09-21T15:00:20.461 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3302000000000001e-05 2024-09-21T15:00:20.461 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.461 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.461 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:20.461 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.425231+0000", 2024-09-21T15:00:20.462 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7106000000000004e-05 2024-09-21T15:00:20.462 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.462 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.462 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:20.462 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.425259+0000", 2024-09-21T15:00:20.462 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8087000000000001e-05 2024-09-21T15:00:20.462 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.462 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.462 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:20.462 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.441890+0000", 2024-09-21T15:00:20.462 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016631328000000001 2024-09-21T15:00:20.462 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.462 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.463 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:20.463 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.441928+0000", 2024-09-21T15:00:20.463 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7701999999999999e-05 2024-09-21T15:00:20.463 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.463 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:20.463 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.463 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.463 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.463 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:100187 3.10 3:08941a97:::rbd_data.1136ab4d6ff1.000000000000000e:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-21T15:00:20.463 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:10.869845+0000", 2024-09-21T15:00:20.463 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 9.5216600509999996, 2024-09-21T15:00:20.463 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029345237, 2024-09-21T15:00:20.463 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:20.463 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:20.464 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:20.464 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:20.464 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:20.464 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 100187 2024-09-21T15:00:20.464 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.464 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:20.464 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.464 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:20.464 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869845+0000", 2024-09-21T15:00:20.464 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.464 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.464 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.464 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:20.464 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869845+0000", 2024-09-21T15:00:20.465 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.465 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.465 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.465 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:20.465 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869844+0000", 2024-09-21T15:00:20.465 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T15:00:20.465 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.465 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.465 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:20.465 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869853+0000", 2024-09-21T15:00:20.465 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5719999999999992e-06 2024-09-21T15:00:20.465 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.465 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.465 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:20.466 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869874+0000", 2024-09-21T15:00:20.466 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0483e-05 2024-09-21T15:00:20.466 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.466 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.466 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:20.466 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869883+0000", 2024-09-21T15:00:20.466 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6080000000000002e-06 2024-09-21T15:00:20.466 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.466 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.466 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:20.466 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869991+0000", 2024-09-21T15:00:20.466 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000108013 2024-09-21T15:00:20.466 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.466 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.467 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:20.467 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.870258+0000", 2024-09-21T15:00:20.467 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00026629099999999999 2024-09-21T15:00:20.467 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.467 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.467 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T15:00:20.467 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.878656+0000", 2024-09-21T15:00:20.467 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0083989000000000008 2024-09-21T15:00:20.467 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.467 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.467 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:20.467 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.899081+0000", 2024-09-21T15:00:20.467 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.020424437 2024-09-21T15:00:20.467 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.467 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.468 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:20.468 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.899125+0000", 2024-09-21T15:00:20.468 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4440999999999997e-05 2024-09-21T15:00:20.468 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.468 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.468 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:20.468 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.899160+0000", 2024-09-21T15:00:20.468 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4518999999999999e-05 2024-09-21T15:00:20.468 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.468 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.468 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:20.468 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.899190+0000", 2024-09-21T15:00:20.468 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0574999999999998e-05 2024-09-21T15:00:20.468 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.469 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:20.469 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.469 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.469 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.469 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:100263 3.10 3:0ece142e:::rbd_data.1136ab4d6ff1.0000000000000008:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-21T15:00:20.469 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:11.052257+0000", 2024-09-21T15:00:20.469 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 9.3392479930000007, 2024-09-21T15:00:20.469 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042678025000000001, 2024-09-21T15:00:20.469 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:20.469 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:20.469 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:20.469 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:20.469 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:20.469 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 100263 2024-09-21T15:00:20.469 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.469 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:20.470 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.470 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:20.470 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052257+0000", 2024-09-21T15:00:20.470 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.470 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.470 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.470 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:20.470 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052257+0000", 2024-09-21T15:00:20.470 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.470 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.470 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.470 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:20.470 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052256+0000", 2024-09-21T15:00:20.470 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:00:20.471 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.471 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.471 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:20.471 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052263+0000", 2024-09-21T15:00:20.471 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4799999999999998e-06 2024-09-21T15:00:20.471 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.471 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.471 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:20.471 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052275+0000", 2024-09-21T15:00:20.471 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2386e-05 2024-09-21T15:00:20.471 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.471 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.471 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:20.472 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052281+0000", 2024-09-21T15:00:20.472 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6699999999999999e-06 2024-09-21T15:00:20.472 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.472 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.472 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:20.472 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052350+0000", 2024-09-21T15:00:20.472 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9099999999999999e-05 2024-09-21T15:00:20.472 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.472 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.472 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:20.472 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052456+0000", 2024-09-21T15:00:20.472 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000106669 2024-09-21T15:00:20.472 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.472 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.473 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T15:00:20.473 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.059694+0000", 2024-09-21T15:00:20.473 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.007237566 2024-09-21T15:00:20.473 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.473 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.473 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:20.473 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.094721+0000", 2024-09-21T15:00:20.473 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035026977000000001 2024-09-21T15:00:20.473 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.473 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.473 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:20.473 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.094746+0000", 2024-09-21T15:00:20.473 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5329000000000001e-05 2024-09-21T15:00:20.473 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.473 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.474 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:20.474 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.094850+0000", 2024-09-21T15:00:20.474 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000103452 2024-09-21T15:00:20.474 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.474 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.474 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:20.474 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.094935+0000", 2024-09-21T15:00:20.474 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5550000000000006e-05 2024-09-21T15:00:20.474 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.474 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:20.474 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.474 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.474 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.474 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:102266 3.1f e130/24 3:fec72c47:::rbd_data.1136d175325b.000000000000000c:head v 130'326, mlcod=128'324)", 2024-09-21T15:00:20.475 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:14.319401+0000", 2024-09-21T15:00:20.475 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 6.0721040530000003, 2024-09-21T15:00:20.475 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021068982999999999, 2024-09-21T15:00:20.475 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:20.475 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:20.475 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:20.475 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.475 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:20.475 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.319401+0000", 2024-09-21T15:00:20.475 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.475 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.475 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.475 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:20.475 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.319401+0000", 2024-09-21T15:00:20.476 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.476 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.476 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.476 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:20.476 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.319400+0000", 2024-09-21T15:00:20.476 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:00:20.476 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.476 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.476 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:20.476 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.337267+0000", 2024-09-21T15:00:20.476 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.017867141 2024-09-21T15:00:20.476 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.476 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.476 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:20.476 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.337278+0000", 2024-09-21T15:00:20.477 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0224000000000001e-05 2024-09-21T15:00:20.477 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.477 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.477 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:20.477 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.337285+0000", 2024-09-21T15:00:20.477 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.25e-06 2024-09-21T15:00:20.477 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.477 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.477 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:20.477 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.337330+0000", 2024-09-21T15:00:20.477 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5210000000000003e-05 2024-09-21T15:00:20.477 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.477 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.477 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:20.478 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.337350+0000", 2024-09-21T15:00:20.478 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9556e-05 2024-09-21T15:00:20.478 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.478 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.478 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:20.478 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.340418+0000", 2024-09-21T15:00:20.478 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.003068552 2024-09-21T15:00:20.478 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.478 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.478 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:20.478 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.340470+0000", 2024-09-21T15:00:20.478 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1897000000000001e-05 2024-09-21T15:00:20.478 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.478 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:20.478 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.479 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.479 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.479 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:102887 3.1c 3:3b11ae99:::rbd_object_map.1136d175325b.0000000000000050:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-21T15:00:20.479 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:14.368644+0000", 2024-09-21T15:00:20.479 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 6.0228610170000003, 2024-09-21T15:00:20.479 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.017021290000000001, 2024-09-21T15:00:20.479 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:20.479 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:20.479 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:20.479 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:20.479 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:20.479 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 102887 2024-09-21T15:00:20.479 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.479 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:20.479 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.480 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:20.480 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.368644+0000", 2024-09-21T15:00:20.480 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.480 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.480 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.480 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:20.480 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.368644+0000", 2024-09-21T15:00:20.480 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.480 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.480 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.480 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:20.480 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.368642+0000", 2024-09-21T15:00:20.480 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T15:00:20.480 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.480 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.480 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:20.481 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.368655+0000", 2024-09-21T15:00:20.481 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2808e-05 2024-09-21T15:00:20.481 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.481 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.481 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:20.481 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.368677+0000", 2024-09-21T15:00:20.481 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2753999999999999e-05 2024-09-21T15:00:20.481 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.481 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.481 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:20.481 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.368688+0000", 2024-09-21T15:00:20.481 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1228e-05 2024-09-21T15:00:20.481 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.481 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.482 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:20.482 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.368743+0000", 2024-09-21T15:00:20.482 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4608999999999998e-05 2024-09-21T15:00:20.482 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.482 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.482 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:20.482 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.368868+0000", 2024-09-21T15:00:20.482 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012473000000000001 2024-09-21T15:00:20.482 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.482 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.482 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 0", 2024-09-21T15:00:20.482 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.369385+0000", 2024-09-21T15:00:20.482 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00051684400000000003 2024-09-21T15:00:20.482 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.482 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.483 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:20.483 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.370755+0000", 2024-09-21T15:00:20.483 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0013701609999999999 2024-09-21T15:00:20.483 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.483 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.483 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:20.483 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.385531+0000", 2024-09-21T15:00:20.483 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014776648 2024-09-21T15:00:20.483 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.483 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.483 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:20.483 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.385575+0000", 2024-09-21T15:00:20.483 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3726000000000001e-05 2024-09-21T15:00:20.483 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.483 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.483 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:20.484 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.385666+0000", 2024-09-21T15:00:20.484 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0304999999999997e-05 2024-09-21T15:00:20.484 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.484 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:20.484 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.484 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.484 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.484 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:103706 3.1c 3:3b11ae99:::rbd_object_map.1136d175325b.0000000000000050:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-21T15:00:20.484 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:14.426370+0000", 2024-09-21T15:00:20.484 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 5.9651356800000004, 2024-09-21T15:00:20.484 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.015478897, 2024-09-21T15:00:20.484 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:20.484 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:20.484 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:20.484 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:20.485 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:20.485 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 103706 2024-09-21T15:00:20.485 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.485 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:20.485 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.485 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:20.485 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.426370+0000", 2024-09-21T15:00:20.485 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.485 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.485 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.485 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:20.485 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.426370+0000", 2024-09-21T15:00:20.485 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.485 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.485 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.485 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:20.486 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.426369+0000", 2024-09-21T15:00:20.486 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:00:20.486 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.486 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.486 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:20.486 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.426376+0000", 2024-09-21T15:00:20.486 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0999999999999998e-06 2024-09-21T15:00:20.486 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.486 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.486 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:20.486 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.426442+0000", 2024-09-21T15:00:20.486 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6445999999999994e-05 2024-09-21T15:00:20.486 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.486 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.487 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:20.487 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.426451+0000", 2024-09-21T15:00:20.487 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4940000000000003e-06 2024-09-21T15:00:20.487 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.487 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.487 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:20.487 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.426493+0000", 2024-09-21T15:00:20.487 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2094000000000001e-05 2024-09-21T15:00:20.487 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.487 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.487 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:20.487 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.426560+0000", 2024-09-21T15:00:20.487 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6774999999999997e-05 2024-09-21T15:00:20.487 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.487 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.488 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 0", 2024-09-21T15:00:20.488 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.426737+0000", 2024-09-21T15:00:20.488 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017769899999999999 2024-09-21T15:00:20.488 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.488 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.488 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:20.488 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.428561+0000", 2024-09-21T15:00:20.488 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001823917 2024-09-21T15:00:20.488 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.488 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.488 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:20.488 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.441762+0000", 2024-09-21T15:00:20.488 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013200462 2024-09-21T15:00:20.489 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.489 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.489 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:20.489 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.441811+0000", 2024-09-21T15:00:20.489 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8912000000000002e-05 2024-09-21T15:00:20.489 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.489 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.489 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:20.489 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.441848+0000", 2024-09-21T15:00:20.489 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7935000000000002e-05 2024-09-21T15:00:20.489 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.489 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:20.489 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.489 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.489 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.490 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:105083 3.1a 3:5d63845a:::rbd_data.1136d175325b.0000000000000037:head [write 0~32768 [fadvise_sequential+fadvise_nocache],write 65536~163840 [fadvise_sequential+fadvise_nocache],write 262144~32768 [fadvise_sequential+fadvise_nocache],write 327680~163840 [fadvise_sequential+fadvise_nocache],write 524288~32768 [fadvise_sequential+fadvise_nocache],write 589824~163840 [fadvise_sequential+fadvise_nocache],write 786432~32768 [fadvise_sequential+fadvise_nocache],write 851968~32768 [fadvise_sequential+fadvise_nocache],write 917504~98304 [fadvise_sequential+fadvise_nocache],write 1048576~32768 [fadvise_sequential+fadvise_nocache],write 1114112~32768 [fadvise_sequential+fadvise_nocache],write 1179648~98304 [fadvise_sequential+fadvise_nocache],write 1310720~32768 [fadvise_sequential+fadvise_nocache],write 1376256~32768 [fadvise_sequential+fadvise_nocache],write 1441792~98304 [fadvise_sequential+fadvise_nocache],write 1572864~32768 [fadvise_sequential+fadvise_nocache],write 1638400~32768 [fadvise_sequential+fadvise_nocache],write 1703936~98304 [fadvise_sequential+fadvise_nocache],write 1835008~32768 [fadvise_sequential+fadvise_nocache],write 1900544~32768 [fadvise_sequential+fadvise_nocache],write 1966080~98304 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-21T15:00:20.490 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:14.534201+0000", 2024-09-21T15:00:20.490 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 5.8573043389999997, 2024-09-21T15:00:20.490 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016474194000000001, 2024-09-21T15:00:20.490 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:20.490 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:20.490 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:20.490 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:20.490 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:20.490 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 105083 2024-09-21T15:00:20.490 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.490 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:20.490 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.490 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:20.490 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.534201+0000", 2024-09-21T15:00:20.491 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.491 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.491 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.491 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:20.491 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.534201+0000", 2024-09-21T15:00:20.491 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.491 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.491 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.491 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:20.491 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.534201+0000", 2024-09-21T15:00:20.491 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:20.491 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.491 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.491 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:20.491 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.534853+0000", 2024-09-21T15:00:20.492 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00065227099999999995 2024-09-21T15:00:20.492 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.492 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.492 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:20.492 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.534866+0000", 2024-09-21T15:00:20.492 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3553000000000001e-05 2024-09-21T15:00:20.492 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.492 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.492 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:20.492 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.534877+0000", 2024-09-21T15:00:20.492 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0186e-05 2024-09-21T15:00:20.492 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.492 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.492 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:20.492 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.534947+0000", 2024-09-21T15:00:20.492 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0417e-05 2024-09-21T15:00:20.493 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.493 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.493 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:20.493 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.535138+0000", 2024-09-21T15:00:20.493 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00019122200000000001 2024-09-21T15:00:20.493 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.493 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.493 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T15:00:20.493 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.535666+0000", 2024-09-21T15:00:20.493 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00052817499999999998 2024-09-21T15:00:20.493 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.493 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.493 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:20.493 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.543252+0000", 2024-09-21T15:00:20.493 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0075857889999999999 2024-09-21T15:00:20.494 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.494 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.494 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:20.494 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.550552+0000", 2024-09-21T15:00:20.494 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0072995869999999997 2024-09-21T15:00:20.494 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.494 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.494 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:20.494 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.550637+0000", 2024-09-21T15:00:20.494 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5750999999999999e-05 2024-09-21T15:00:20.494 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.494 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.494 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:20.495 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.550675+0000", 2024-09-21T15:00:20.495 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7602999999999999e-05 2024-09-21T15:00:20.495 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.495 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:20.495 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.495 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.495 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:20.495 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:20.496 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T15:00:20.497 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:00:20.492+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:00:20.545 INFO:teuthology.orchestra.run.smithi032.stdout:3757 punch from 0x17ee4da to 0x17f894c, (0xa472 bytes) 2024-09-21T15:00:20.560 INFO:teuthology.orchestra.run.smithi032.stdout:3758 trunc from 0x5c2d890 to 0x9273a0d 2024-09-21T15:00:20.572 INFO:teuthology.orchestra.run.smithi032.stdout:3759 trunc from 0x9273a0d to 0x999934e 2024-09-21T15:00:20.585 INFO:teuthology.orchestra.run.smithi032.stdout:3760 read 0x3569434 thru 0x356cd44 (0x3911 bytes) 2024-09-21T15:00:20.585 INFO:teuthology.orchestra.run.smithi032.stdout:3762 punch from 0x826c18f to 0x826e884, (0x26f5 bytes) 2024-09-21T15:00:20.585 INFO:teuthology.orchestra.run.smithi032.stdout:3763 read 0x9588500 thru 0x9590058 (0x7b59 bytes) 2024-09-21T15:00:20.585 INFO:teuthology.orchestra.run.smithi032.stdout:3764 read 0x8b32169 thru 0x8b40c7b (0xeb13 bytes) 2024-09-21T15:00:20.585 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:20.585 INFO:teuthology.orchestra.run.smithi032.stdout:3766 writesame 0x4f8b291 thru 0x4f8f402 data_size 0x1(0x4172 bytes) 2024-09-21T15:00:20.597 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:00:20.596+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:00:20.607 INFO:teuthology.orchestra.run.smithi032.stdout:3767 read 0x2f63596 thru 0x2f6e4fa (0xaf65 bytes) 2024-09-21T15:00:20.611 INFO:teuthology.orchestra.run.smithi032.stdout:3769 trunc from 0x999934e to 0x78e9cf7 2024-09-21T15:00:20.624 INFO:teuthology.orchestra.run.smithi032.stdout:3771 trunc from 0x78e9cf7 to 0x735fbb1 2024-09-21T15:00:20.635 INFO:teuthology.orchestra.run.smithi032.stdout:3772 punch from 0x5dfaa6 to 0x5e3f8a, (0x44e4 bytes) 2024-09-21T15:00:20.635 INFO:teuthology.orchestra.run.smithi032.stdout:3775 punch from 0x39169a1 to 0x391c15b, (0x57ba bytes) 2024-09-21T15:00:20.650 INFO:teuthology.orchestra.run.smithi032.stdout:3777 trunc from 0x735fbb1 to 0x1cac19 2024-09-21T15:00:20.653 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:20.653 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:00:20.653 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T15:00:20.653 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:20.667 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T15:00:20.694 INFO:teuthology.orchestra.run.smithi032.stdout:3781 trunc from 0x1cac19 to 0x72c05cd 2024-09-21T15:00:20.699 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:00:20.694+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:00:20.702 INFO:teuthology.orchestra.run.smithi032.stdout:3782 read 0x40a6760 thru 0x40b4382 (0xdc23 bytes) 2024-09-21T15:00:20.718 INFO:teuthology.orchestra.run.smithi032.stdout:3783 write 0xbeffa6e thru 0xbf052c0 (0x5853 bytes) 2024-09-21T15:00:20.728 INFO:teuthology.orchestra.run.smithi032.stdout:3784 punch from 0x8691db8 to 0x869372b, (0x1973 bytes) 2024-09-21T15:00:20.730 INFO:teuthology.orchestra.run.smithi032.stdout:3786 trunc from 0xbf052c1 to 0xa94666d 2024-09-21T15:00:20.772 INFO:teuthology.orchestra.run.smithi032.stdout:3787 write 0xd4de01f thru 0xd4defad (0xf8f bytes) 2024-09-21T15:00:20.774 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:20.774 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:00:20.774 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T15:00:20.774 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T15:00:20.774 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:20.779 INFO:teuthology.orchestra.run.smithi032.stdout:3788 write 0x61a1e96 thru 0x61aa3b6 (0x8521 bytes) 2024-09-21T15:00:20.785 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T15:00:20.788 INFO:teuthology.orchestra.run.smithi032.stdout:3789 write 0x5cd1468 thru 0x5cdb86d (0xa406 bytes) 2024-09-21T15:00:20.800 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:00:20.794+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:00:20.810 INFO:teuthology.orchestra.run.smithi032.stdout:3790 write 0xe5b494d thru 0xe5b623e (0x18f2 bytes) 2024-09-21T15:00:20.824 INFO:teuthology.orchestra.run.smithi032.stdout:3791 trunc from 0xe5b623f to 0xaf775ea 2024-09-21T15:00:20.848 INFO:teuthology.orchestra.run.smithi032.stdout:3792 write 0x24a430 thru 0x250503 (0x60d4 bytes) 2024-09-21T15:00:20.851 INFO:teuthology.orchestra.run.smithi032.stdout:3793 read 0x3b4ffb9 thru 0x3b55dbd (0x5e05 bytes) 2024-09-21T15:00:20.852 INFO:teuthology.orchestra.run.smithi032.stdout:3794 trunc from 0xaf775ea to 0xa6ae6a4 2024-09-21T15:00:20.864 INFO:teuthology.orchestra.run.smithi032.stdout:3796 read 0x8748621 thru 0x87514c9 (0x8ea9 bytes) 2024-09-21T15:00:20.864 INFO:teuthology.orchestra.run.smithi032.stdout:3797 punch from 0x3f54517 to 0x3f56da3, (0x288c bytes) 2024-09-21T15:00:20.872 INFO:teuthology.orchestra.run.smithi032.stdout:3799 trunc from 0xa6ae6a4 to 0x5d92368 2024-09-21T15:00:20.894 INFO:teuthology.orchestra.run.smithi032.stdout:3800 writesame 0x53d42a7 thru 0x53dc6b4 data_size 0x1(0x840e bytes) 2024-09-21T15:00:20.901 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:00:20.894+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:00:20.904 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:20.904 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T15:00:20.904 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T15:00:20.904 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T15:00:20.904 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.904 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:15130 3.2 3:47f1bbdb:::rbd_data.11361bed1f78.0000000000000067:head [sparse-read 0~147456] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-21T15:00:20.904 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:40.288284+0000", 2024-09-21T15:00:20.905 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 100.615164609, 2024-09-21T15:00:20.905 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1142935039999999, 2024-09-21T15:00:20.905 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:20.905 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:20.905 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:20.905 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:20.905 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:20.905 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 15130 2024-09-21T15:00:20.905 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.905 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:20.905 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.905 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:20.905 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288284+0000", 2024-09-21T15:00:20.905 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.905 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.906 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.906 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:20.906 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288284+0000", 2024-09-21T15:00:20.906 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.906 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.906 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.906 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:20.906 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288284+0000", 2024-09-21T15:00:20.906 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:20.906 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.906 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.906 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:20.906 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288288+0000", 2024-09-21T15:00:20.906 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5829999999999998e-06 2024-09-21T15:00:20.907 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.907 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.907 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:20.907 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288297+0000", 2024-09-21T15:00:20.907 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0669999999999998e-06 2024-09-21T15:00:20.907 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.907 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.907 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:20.907 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288305+0000", 2024-09-21T15:00:20.907 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8860000000000004e-06 2024-09-21T15:00:20.907 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.907 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.907 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:20.907 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288417+0000", 2024-09-21T15:00:20.908 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000112952 2024-09-21T15:00:20.908 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.908 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.908 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:00:20.908 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288428+0000", 2024-09-21T15:00:20.908 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0631e-05 2024-09-21T15:00:20.908 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.908 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.908 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:20.908 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.401876+0000", 2024-09-21T15:00:20.908 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1134479610000001 2024-09-21T15:00:20.908 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.908 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.908 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:20.908 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.402076+0000", 2024-09-21T15:00:20.908 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00019944200000000001 2024-09-21T15:00:20.909 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.909 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.909 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:20.909 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.402578+0000", 2024-09-21T15:00:20.909 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00050242899999999998 2024-09-21T15:00:20.909 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.909 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:20.909 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.909 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.909 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.909 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45708 3.2 3:415db91a:::rbd_data.1136ecbe5879.0000000000000000:28 [read 140288~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:20.909 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141801+0000", 2024-09-21T15:00:20.909 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 70.761648414999996, 2024-09-21T15:00:20.909 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034274200999999997, 2024-09-21T15:00:20.910 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:20.910 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:20.910 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:20.910 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:20.910 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:20.910 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45708 2024-09-21T15:00:20.910 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.910 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:20.910 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.910 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:20.910 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141801+0000", 2024-09-21T15:00:20.910 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.910 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.910 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.910 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:20.911 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141801+0000", 2024-09-21T15:00:20.911 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.911 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.911 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.911 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:20.911 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141800+0000", 2024-09-21T15:00:20.911 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:20.911 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.911 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.911 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:20.911 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141803+0000", 2024-09-21T15:00:20.911 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4210000000000002e-06 2024-09-21T15:00:20.911 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.911 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.911 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:20.912 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141812+0000", 2024-09-21T15:00:20.912 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.9800000000000004e-06 2024-09-21T15:00:20.912 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.912 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.912 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:20.912 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141816+0000", 2024-09-21T15:00:20.912 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4769999999999997e-06 2024-09-21T15:00:20.912 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.912 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.912 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:20.912 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.175865+0000", 2024-09-21T15:00:20.912 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034048875999999999 2024-09-21T15:00:20.912 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.912 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.912 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:20.912 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.175943+0000", 2024-09-21T15:00:20.913 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7464000000000002e-05 2024-09-21T15:00:20.913 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.913 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.913 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:20.913 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176075+0000", 2024-09-21T15:00:20.913 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013230800000000001 2024-09-21T15:00:20.913 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.913 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:20.913 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.913 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.913 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.913 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45709 3.2 3:415db91a:::rbd_data.1136ecbe5879.0000000000000000:28 [read 143360~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:20.913 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141820+0000", 2024-09-21T15:00:20.913 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 70.761628568000006, 2024-09-21T15:00:20.913 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034537536000000001, 2024-09-21T15:00:20.913 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:20.914 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:20.914 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:20.914 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:20.914 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:20.914 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45709 2024-09-21T15:00:20.914 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.914 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:20.914 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.914 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:20.914 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141820+0000", 2024-09-21T15:00:20.914 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.914 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.914 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.914 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:20.915 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141820+0000", 2024-09-21T15:00:20.915 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.915 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.915 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.915 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:20.915 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141820+0000", 2024-09-21T15:00:20.915 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:20.915 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.915 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.915 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:20.915 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141823+0000", 2024-09-21T15:00:20.915 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5110000000000002e-06 2024-09-21T15:00:20.915 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.915 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.915 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:20.916 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141829+0000", 2024-09-21T15:00:20.916 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0850000000000002e-06 2024-09-21T15:00:20.916 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.916 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.916 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:20.916 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141833+0000", 2024-09-21T15:00:20.916 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7620000000000003e-06 2024-09-21T15:00:20.916 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.916 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.916 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:20.916 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176103+0000", 2024-09-21T15:00:20.916 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034269933000000002 2024-09-21T15:00:20.916 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.916 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.916 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:20.916 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176161+0000", 2024-09-21T15:00:20.917 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7658999999999998e-05 2024-09-21T15:00:20.917 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.917 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.917 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:20.917 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176358+0000", 2024-09-21T15:00:20.917 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000196937 2024-09-21T15:00:20.917 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.917 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:20.917 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.917 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.917 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.917 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45710 3.2 3:415db91a:::rbd_data.1136ecbe5879.0000000000000000:28 [read 146432~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:20.917 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141838+0000", 2024-09-21T15:00:20.917 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 70.761611404999996, 2024-09-21T15:00:20.917 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034953674999999997, 2024-09-21T15:00:20.918 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:20.918 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:20.918 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:20.918 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:20.918 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:20.918 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45710 2024-09-21T15:00:20.918 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.918 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:20.918 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.918 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:20.918 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141838+0000", 2024-09-21T15:00:20.918 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.918 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.918 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.918 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:20.919 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141838+0000", 2024-09-21T15:00:20.919 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.919 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.919 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.919 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:20.919 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141837+0000", 2024-09-21T15:00:20.919 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:20.919 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.919 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.919 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:20.919 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141843+0000", 2024-09-21T15:00:20.919 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1619999999999996e-06 2024-09-21T15:00:20.919 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.919 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.919 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:20.920 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141850+0000", 2024-09-21T15:00:20.920 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4799999999999998e-06 2024-09-21T15:00:20.920 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.920 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.920 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:20.920 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141853+0000", 2024-09-21T15:00:20.920 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3050000000000001e-06 2024-09-21T15:00:20.920 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.920 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.920 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:20.920 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176490+0000", 2024-09-21T15:00:20.920 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034636413999999997 2024-09-21T15:00:20.920 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.920 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.920 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:20.920 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176606+0000", 2024-09-21T15:00:20.921 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000115841 2024-09-21T15:00:20.921 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.921 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.921 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:20.921 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176791+0000", 2024-09-21T15:00:20.921 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00018581500000000001 2024-09-21T15:00:20.921 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.921 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:20.921 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.921 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.921 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.921 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45711 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:20.921 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141856+0000", 2024-09-21T15:00:20.921 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 70.761593067999996, 2024-09-21T15:00:20.922 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034986308000000001, 2024-09-21T15:00:20.922 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:20.922 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:20.922 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:20.922 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:20.922 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:20.922 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45711 2024-09-21T15:00:20.922 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.922 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:20.922 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.922 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:20.922 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141856+0000", 2024-09-21T15:00:20.922 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.922 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.922 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.923 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:20.923 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141856+0000", 2024-09-21T15:00:20.923 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.923 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.923 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.923 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:20.923 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141856+0000", 2024-09-21T15:00:20.923 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:20.923 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.923 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.923 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:20.923 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141862+0000", 2024-09-21T15:00:20.923 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.567e-06 2024-09-21T15:00:20.923 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.923 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.924 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:20.924 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141868+0000", 2024-09-21T15:00:20.924 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9569999999999999e-06 2024-09-21T15:00:20.924 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.924 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.924 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:20.924 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141871+0000", 2024-09-21T15:00:20.924 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.146e-06 2024-09-21T15:00:20.924 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.924 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.924 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:20.924 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176831+0000", 2024-09-21T15:00:20.924 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034959935999999997 2024-09-21T15:00:20.924 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.924 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.924 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:20.925 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176842+0000", 2024-09-21T15:00:20.925 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1008e-05 2024-09-21T15:00:20.925 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.925 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:20.925 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.925 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.925 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.925 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45712 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:20.925 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141874+0000", 2024-09-21T15:00:20.925 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 70.761575172999997, 2024-09-21T15:00:20.925 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034993511999999997, 2024-09-21T15:00:20.925 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:20.925 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:20.925 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:20.925 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:20.926 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:20.926 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45712 2024-09-21T15:00:20.926 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.926 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:20.926 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.926 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:20.926 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141874+0000", 2024-09-21T15:00:20.926 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.926 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.926 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.926 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:20.926 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141874+0000", 2024-09-21T15:00:20.926 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.926 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.926 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.926 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:20.927 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141874+0000", 2024-09-21T15:00:20.927 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:20.927 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.927 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.927 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:20.927 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141875+0000", 2024-09-21T15:00:20.927 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5430000000000001e-06 2024-09-21T15:00:20.927 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.927 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.927 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:20.927 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141878+0000", 2024-09-21T15:00:20.927 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1420000000000002e-06 2024-09-21T15:00:20.927 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.927 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.927 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:20.927 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141881+0000", 2024-09-21T15:00:20.928 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.807e-06 2024-09-21T15:00:20.928 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.928 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.928 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:20.928 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176860+0000", 2024-09-21T15:00:20.928 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034979243 2024-09-21T15:00:20.928 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.928 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.928 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:20.928 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176867+0000", 2024-09-21T15:00:20.928 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0169999999999999e-06 2024-09-21T15:00:20.928 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.928 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:20.928 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.929 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.929 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.929 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45713 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:20.929 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141883+0000", 2024-09-21T15:00:20.929 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 70.761565598000004, 2024-09-21T15:00:20.929 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035010516999999998, 2024-09-21T15:00:20.929 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:20.929 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:20.929 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:20.929 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:20.929 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:20.929 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45713 2024-09-21T15:00:20.929 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.929 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:20.929 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.929 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:20.930 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141883+0000", 2024-09-21T15:00:20.930 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.930 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.930 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.930 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:20.930 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141883+0000", 2024-09-21T15:00:20.930 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.930 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.930 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.930 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:20.930 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141883+0000", 2024-09-21T15:00:20.930 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:20.930 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.930 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.930 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:20.931 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141885+0000", 2024-09-21T15:00:20.931 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4649999999999999e-06 2024-09-21T15:00:20.931 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.931 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.931 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:20.931 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141888+0000", 2024-09-21T15:00:20.931 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3890000000000001e-06 2024-09-21T15:00:20.931 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.931 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.931 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:20.931 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141891+0000", 2024-09-21T15:00:20.931 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9239999999999999e-06 2024-09-21T15:00:20.931 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.932 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.932 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:20.932 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176887+0000", 2024-09-21T15:00:20.932 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034996370999999998 2024-09-21T15:00:20.932 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.932 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.932 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:20.932 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176894+0000", 2024-09-21T15:00:20.932 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6069999999999996e-06 2024-09-21T15:00:20.932 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.932 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:20.932 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.932 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.932 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.933 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45714 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:20.933 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141893+0000", 2024-09-21T15:00:20.933 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 70.761555682999997, 2024-09-21T15:00:20.933 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035048215000000001, 2024-09-21T15:00:20.933 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:20.933 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:20.933 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:20.933 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:20.933 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:20.933 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45714 2024-09-21T15:00:20.933 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.933 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:20.933 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.933 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:20.933 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141893+0000", 2024-09-21T15:00:20.933 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.934 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.934 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.934 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:20.934 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141893+0000", 2024-09-21T15:00:20.934 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.934 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.934 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.934 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:20.934 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141893+0000", 2024-09-21T15:00:20.934 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:20.934 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.934 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.934 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:20.934 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141895+0000", 2024-09-21T15:00:20.934 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.703e-06 2024-09-21T15:00:20.935 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.935 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.935 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:20.935 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141900+0000", 2024-09-21T15:00:20.935 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.242e-06 2024-09-21T15:00:20.935 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.935 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.935 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:20.935 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141903+0000", 2024-09-21T15:00:20.935 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.117e-06 2024-09-21T15:00:20.935 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.935 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.935 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:20.935 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176930+0000", 2024-09-21T15:00:20.935 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035026750000000002 2024-09-21T15:00:20.936 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.936 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.936 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:20.936 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176942+0000", 2024-09-21T15:00:20.936 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1668000000000001e-05 2024-09-21T15:00:20.936 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.936 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:20.936 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.936 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.936 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.936 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45715 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:20.936 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141906+0000", 2024-09-21T15:00:20.936 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 70.761543524999993, 2024-09-21T15:00:20.936 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035066057999999997, 2024-09-21T15:00:20.937 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:20.937 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:20.937 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:20.937 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:20.937 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:20.937 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45715 2024-09-21T15:00:20.937 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.937 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:20.937 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.937 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:20.937 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141906+0000", 2024-09-21T15:00:20.937 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.937 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.937 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.937 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:20.937 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141906+0000", 2024-09-21T15:00:20.938 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.938 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.938 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.938 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:20.938 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141905+0000", 2024-09-21T15:00:20.938 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:20.938 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.938 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.938 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:20.938 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141907+0000", 2024-09-21T15:00:20.938 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5400000000000001e-06 2024-09-21T15:00:20.938 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.938 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.938 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:20.938 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141912+0000", 2024-09-21T15:00:20.939 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.502e-06 2024-09-21T15:00:20.939 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.939 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.939 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:20.939 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141915+0000", 2024-09-21T15:00:20.939 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0170000000000001e-06 2024-09-21T15:00:20.939 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.939 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.939 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:20.939 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176964+0000", 2024-09-21T15:00:20.939 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035048587999999999 2024-09-21T15:00:20.939 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.939 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.939 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:20.940 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176972+0000", 2024-09-21T15:00:20.940 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6580000000000002e-06 2024-09-21T15:00:20.940 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.940 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:20.940 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.940 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.940 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.940 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45716 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:20.940 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141918+0000", 2024-09-21T15:00:20.940 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 70.761531348999995, 2024-09-21T15:00:20.940 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035086223, 2024-09-21T15:00:20.940 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:20.940 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:20.940 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:20.940 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:20.941 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:20.941 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45716 2024-09-21T15:00:20.941 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.941 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:20.941 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.941 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:20.941 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141918+0000", 2024-09-21T15:00:20.941 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.941 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.941 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.941 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:20.941 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141918+0000", 2024-09-21T15:00:20.941 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.941 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.942 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.942 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:20.942 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141917+0000", 2024-09-21T15:00:20.942 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:20.942 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.942 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.942 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:20.942 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141919+0000", 2024-09-21T15:00:20.942 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5680000000000001e-06 2024-09-21T15:00:20.942 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.942 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.942 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:20.942 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141922+0000", 2024-09-21T15:00:20.942 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2789999999999999e-06 2024-09-21T15:00:20.942 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.942 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.943 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:20.943 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141925+0000", 2024-09-21T15:00:20.943 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9469999999999999e-06 2024-09-21T15:00:20.943 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.943 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.943 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:20.943 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176997+0000", 2024-09-21T15:00:20.943 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035071366999999999 2024-09-21T15:00:20.943 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.943 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.943 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:20.943 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177004+0000", 2024-09-21T15:00:20.943 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3309999999999998e-06 2024-09-21T15:00:20.943 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.944 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:20.944 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.944 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.944 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.944 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45717 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:20.944 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141928+0000", 2024-09-21T15:00:20.944 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 70.761521403000003, 2024-09-21T15:00:20.944 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035102573999999998, 2024-09-21T15:00:20.944 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:20.944 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:20.944 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:20.944 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:20.944 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:20.944 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45717 2024-09-21T15:00:20.944 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.945 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:20.945 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.945 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:20.945 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141928+0000", 2024-09-21T15:00:20.945 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.945 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.945 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.945 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:20.945 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141928+0000", 2024-09-21T15:00:20.945 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.945 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.945 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.945 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:20.945 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141927+0000", 2024-09-21T15:00:20.945 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:00:20.946 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.946 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.946 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:20.946 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141929+0000", 2024-09-21T15:00:20.946 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.533e-06 2024-09-21T15:00:20.946 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.946 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.946 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:20.946 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141934+0000", 2024-09-21T15:00:20.946 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1660000000000002e-06 2024-09-21T15:00:20.946 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.946 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.946 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:20.946 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141937+0000", 2024-09-21T15:00:20.947 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0309999999999999e-06 2024-09-21T15:00:20.947 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.947 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.947 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:20.947 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177023+0000", 2024-09-21T15:00:20.947 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035085595999999997 2024-09-21T15:00:20.947 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.947 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.947 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:20.947 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177030+0000", 2024-09-21T15:00:20.947 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.486e-06 2024-09-21T15:00:20.947 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.947 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:20.947 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.947 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.947 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.948 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45718 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:20.948 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141940+0000", 2024-09-21T15:00:20.948 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 70.761509544999996, 2024-09-21T15:00:20.948 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035124956999999998, 2024-09-21T15:00:20.948 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:20.948 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:20.948 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:20.948 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:20.948 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:20.948 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45718 2024-09-21T15:00:20.948 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.948 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:20.948 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.948 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:20.948 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141940+0000", 2024-09-21T15:00:20.949 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.949 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.949 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.949 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:20.949 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141940+0000", 2024-09-21T15:00:20.949 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.949 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.949 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.949 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:20.949 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141939+0000", 2024-09-21T15:00:20.949 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:20.949 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.949 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.949 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:20.950 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141941+0000", 2024-09-21T15:00:20.950 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.502e-06 2024-09-21T15:00:20.950 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.950 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.950 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:20.950 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141946+0000", 2024-09-21T15:00:20.950 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3449999999999996e-06 2024-09-21T15:00:20.950 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.950 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.950 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:20.950 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141949+0000", 2024-09-21T15:00:20.950 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9749999999999999e-06 2024-09-21T15:00:20.950 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.950 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.950 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:20.951 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177057+0000", 2024-09-21T15:00:20.951 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035108385999999998 2024-09-21T15:00:20.951 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.951 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.951 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:20.951 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177064+0000", 2024-09-21T15:00:20.951 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0099999999999998e-06 2024-09-21T15:00:20.951 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.951 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:20.951 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.951 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.951 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.951 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45719 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:20.952 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141952+0000", 2024-09-21T15:00:20.952 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 70.761497272, 2024-09-21T15:00:20.952 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035139108000000002, 2024-09-21T15:00:20.952 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:20.952 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:20.952 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:20.952 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:20.952 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:20.952 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45719 2024-09-21T15:00:20.952 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.952 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:20.952 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.952 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:20.952 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141952+0000", 2024-09-21T15:00:20.952 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.953 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.953 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.953 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:20.953 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141952+0000", 2024-09-21T15:00:20.953 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.953 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.953 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.953 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:20.953 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141952+0000", 2024-09-21T15:00:20.953 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:20.953 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.953 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.953 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:20.953 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141953+0000", 2024-09-21T15:00:20.953 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.486e-06 2024-09-21T15:00:20.953 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.954 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.954 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:20.954 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141959+0000", 2024-09-21T15:00:20.954 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4940000000000003e-06 2024-09-21T15:00:20.954 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.954 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.954 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:20.954 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141963+0000", 2024-09-21T15:00:20.954 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3540000000000002e-06 2024-09-21T15:00:20.954 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.954 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.954 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:20.954 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177084+0000", 2024-09-21T15:00:20.954 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035120983000000001 2024-09-21T15:00:20.955 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.955 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.955 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:20.955 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177091+0000", 2024-09-21T15:00:20.955 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0350000000000004e-06 2024-09-21T15:00:20.955 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.955 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:20.955 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.955 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.955 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.955 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45720 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:20.955 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141965+0000", 2024-09-21T15:00:20.955 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 70.761483804999997, 2024-09-21T15:00:20.955 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035155268000000003, 2024-09-21T15:00:20.955 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:20.956 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:20.956 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:20.956 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:20.956 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:20.956 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45720 2024-09-21T15:00:20.956 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.956 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:20.956 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.956 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:20.956 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141965+0000", 2024-09-21T15:00:20.956 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.956 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.956 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.956 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:20.956 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141965+0000", 2024-09-21T15:00:20.957 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.957 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.957 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.957 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:20.957 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141965+0000", 2024-09-21T15:00:20.957 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:20.957 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.957 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.957 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:20.957 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141966+0000", 2024-09-21T15:00:20.957 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.46e-06 2024-09-21T15:00:20.957 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.957 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.957 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:20.957 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141972+0000", 2024-09-21T15:00:20.957 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4210000000000003e-06 2024-09-21T15:00:20.958 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.958 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.958 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:20.958 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141975+0000", 2024-09-21T15:00:20.958 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.948e-06 2024-09-21T15:00:20.958 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.958 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.958 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:20.958 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177113+0000", 2024-09-21T15:00:20.958 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035137855000000003 2024-09-21T15:00:20.958 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.958 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.958 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:20.958 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177121+0000", 2024-09-21T15:00:20.959 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8269999999999998e-06 2024-09-21T15:00:20.959 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.959 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:20.959 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.959 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.959 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.959 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45721 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:20.959 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141977+0000", 2024-09-21T15:00:20.959 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 70.761471782000001, 2024-09-21T15:00:20.959 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035169984000000001, 2024-09-21T15:00:20.959 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:20.959 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:20.959 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:20.959 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:20.959 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:20.960 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45721 2024-09-21T15:00:20.960 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.960 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:20.960 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.960 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:20.960 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141977+0000", 2024-09-21T15:00:20.960 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.960 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.960 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.960 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:20.960 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141977+0000", 2024-09-21T15:00:20.960 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.960 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.960 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.960 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:20.961 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141977+0000", 2024-09-21T15:00:20.961 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:20.961 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.961 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.961 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:20.961 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141979+0000", 2024-09-21T15:00:20.961 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.511e-06 2024-09-21T15:00:20.961 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.961 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.961 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:20.961 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141982+0000", 2024-09-21T15:00:20.961 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4180000000000001e-06 2024-09-21T15:00:20.961 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.961 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.961 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:20.961 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141985+0000", 2024-09-21T15:00:20.962 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9450000000000002e-06 2024-09-21T15:00:20.962 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.962 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.962 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:20.962 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177139+0000", 2024-09-21T15:00:20.962 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035154409999999997 2024-09-21T15:00:20.962 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.962 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.962 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:20.962 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177147+0000", 2024-09-21T15:00:20.962 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9570000000000002e-06 2024-09-21T15:00:20.962 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.962 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:20.962 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.963 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.963 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.963 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45722 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:20.963 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141987+0000", 2024-09-21T15:00:20.963 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 70.761461811000004, 2024-09-21T15:00:20.963 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035188492000000002, 2024-09-21T15:00:20.963 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:20.963 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:20.963 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:20.963 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:20.963 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:20.963 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45722 2024-09-21T15:00:20.964 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.964 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:20.964 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.964 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:20.964 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141987+0000", 2024-09-21T15:00:20.964 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.964 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.964 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.964 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:20.964 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141987+0000", 2024-09-21T15:00:20.964 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.964 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.964 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.964 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:20.964 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141987+0000", 2024-09-21T15:00:20.965 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:20.965 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.965 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.965 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:20.965 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141988+0000", 2024-09-21T15:00:20.965 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.463e-06 2024-09-21T15:00:20.965 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.965 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.965 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:20.965 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141994+0000", 2024-09-21T15:00:20.965 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.361e-06 2024-09-21T15:00:20.965 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.965 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.965 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:20.965 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141997+0000", 2024-09-21T15:00:20.966 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0359999999999998e-06 2024-09-21T15:00:20.966 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.966 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.966 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:20.966 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177169+0000", 2024-09-21T15:00:20.966 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035171651999999998 2024-09-21T15:00:20.966 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.966 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.966 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:20.966 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177176+0000", 2024-09-21T15:00:20.966 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2189999999999998e-06 2024-09-21T15:00:20.966 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.966 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:20.966 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.967 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.967 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.967 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45723 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:20.967 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141999+0000", 2024-09-21T15:00:20.967 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 70.761449643999995, 2024-09-21T15:00:20.967 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035197701999999997, 2024-09-21T15:00:20.967 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:20.967 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:20.967 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:20.967 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:20.967 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:20.967 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45723 2024-09-21T15:00:20.967 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.967 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:20.967 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.967 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:20.968 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141999+0000", 2024-09-21T15:00:20.968 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.968 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.968 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.968 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:20.968 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141999+0000", 2024-09-21T15:00:20.968 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.968 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.968 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.968 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:20.968 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141999+0000", 2024-09-21T15:00:20.968 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:20.968 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.968 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.968 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:20.969 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142003+0000", 2024-09-21T15:00:20.969 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5049999999999999e-06 2024-09-21T15:00:20.969 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.969 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.969 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:20.969 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142008+0000", 2024-09-21T15:00:20.969 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6579999999999999e-06 2024-09-21T15:00:20.969 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.969 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.969 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:20.969 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142011+0000", 2024-09-21T15:00:20.969 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0529999999999998e-06 2024-09-21T15:00:20.969 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.969 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.970 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:20.970 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177194+0000", 2024-09-21T15:00:20.970 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035182221999999999 2024-09-21T15:00:20.970 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.970 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.970 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:20.970 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177197+0000", 2024-09-21T15:00:20.970 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5030000000000002e-06 2024-09-21T15:00:20.970 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.970 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:20.970 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.970 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.970 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.970 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45724 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:20.970 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.142015+0000", 2024-09-21T15:00:20.971 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 70.761433676999999, 2024-09-21T15:00:20.971 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035195345000000003, 2024-09-21T15:00:20.971 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:20.971 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:20.971 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:20.971 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:20.971 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:20.971 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45724 2024-09-21T15:00:20.971 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.971 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:20.971 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.971 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:20.971 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142015+0000", 2024-09-21T15:00:20.971 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.971 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.972 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.972 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:20.972 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142015+0000", 2024-09-21T15:00:20.972 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.972 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.972 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.972 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:20.972 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142015+0000", 2024-09-21T15:00:20.972 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:20.972 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.972 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.972 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:20.972 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142017+0000", 2024-09-21T15:00:20.972 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5540000000000001e-06 2024-09-21T15:00:20.972 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.973 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.973 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:20.973 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142022+0000", 2024-09-21T15:00:20.973 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4229999999999996e-06 2024-09-21T15:00:20.973 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.973 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.973 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:20.973 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142025+0000", 2024-09-21T15:00:20.973 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.056e-06 2024-09-21T15:00:20.973 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.973 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.973 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:20.973 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177207+0000", 2024-09-21T15:00:20.973 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035182089 2024-09-21T15:00:20.973 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.974 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.974 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:20.974 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177211+0000", 2024-09-21T15:00:20.974 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4649999999999999e-06 2024-09-21T15:00:20.974 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.974 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:20.974 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.974 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.974 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.974 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45725 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:20.974 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.142028+0000", 2024-09-21T15:00:20.974 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 70.761421397999996, 2024-09-21T15:00:20.974 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035195731000000001, 2024-09-21T15:00:20.974 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:20.974 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:20.974 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:20.975 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:20.975 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:20.975 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45725 2024-09-21T15:00:20.975 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.975 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:20.975 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.975 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:20.975 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142028+0000", 2024-09-21T15:00:20.975 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.975 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.975 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.975 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:20.975 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142028+0000", 2024-09-21T15:00:20.975 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.976 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.976 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.976 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:20.976 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142027+0000", 2024-09-21T15:00:20.976 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:20.976 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.976 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.976 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:20.976 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142029+0000", 2024-09-21T15:00:20.976 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5269999999999999e-06 2024-09-21T15:00:20.976 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.976 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.976 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:20.976 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142032+0000", 2024-09-21T15:00:20.976 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3249999999999999e-06 2024-09-21T15:00:20.976 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.977 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.977 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:20.977 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142035+0000", 2024-09-21T15:00:20.977 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.897e-06 2024-09-21T15:00:20.977 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.977 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.977 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:20.977 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177220+0000", 2024-09-21T15:00:20.977 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035184871 2024-09-21T15:00:20.977 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.977 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.977 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:20.977 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177223+0000", 2024-09-21T15:00:20.977 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3680000000000002e-06 2024-09-21T15:00:20.977 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.978 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:20.978 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.978 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.978 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.978 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45726 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:20.978 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.142037+0000", 2024-09-21T15:00:20.978 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 70.761411683000006, 2024-09-21T15:00:20.978 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035199273000000003, 2024-09-21T15:00:20.978 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:20.978 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:20.978 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:20.978 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:20.978 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:20.978 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45726 2024-09-21T15:00:20.979 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.979 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:20.979 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.979 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:20.979 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142037+0000", 2024-09-21T15:00:20.979 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.979 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.979 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.979 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:20.979 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142037+0000", 2024-09-21T15:00:20.979 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:20.979 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.979 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.979 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:20.979 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142037+0000", 2024-09-21T15:00:20.980 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:20.980 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.980 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.980 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:20.980 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142038+0000", 2024-09-21T15:00:20.980 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.344e-06 2024-09-21T15:00:20.980 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.980 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.980 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:20.980 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142044+0000", 2024-09-21T15:00:20.980 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1930000000000001e-06 2024-09-21T15:00:20.980 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.980 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.980 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:20.980 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142047+0000", 2024-09-21T15:00:20.980 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0280000000000001e-06 2024-09-21T15:00:20.981 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.981 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.981 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:20.981 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177233+0000", 2024-09-21T15:00:20.981 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035186394000000003 2024-09-21T15:00:20.981 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:20.981 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:20.981 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:20.981 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177237+0000", 2024-09-21T15:00:20.981 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5530000000000001e-06 2024-09-21T15:00:20.981 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.981 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:20.981 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.981 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:20.981 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:20.981 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:20.983 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T15:00:20.983 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:20.983 INFO:teuthology.orchestra.run.smithi032.stdout:3803 trunc from 0x5d92368 to 0xb347af8 2024-09-21T15:00:20.984 INFO:teuthology.orchestra.run.smithi032.stdout:3804 read 0xc00f70 thru 0xc10139 (0xf1ca bytes) 2024-09-21T15:00:20.984 INFO:teuthology.orchestra.run.smithi032.stdout:3806 read 0x3f48e10 thru 0x3f5656f (0xd760 bytes) 2024-09-21T15:00:20.984 INFO:teuthology.orchestra.run.smithi032.stdout:3807 writesame 0x15f84b4 thru 0x1605040 data_size 0x1(0xcb8d bytes) 2024-09-21T15:00:20.984 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:20.984 INFO:teuthology.orchestra.run.smithi032.stdout:3809 trunc from 0xb347af8 to 0xb4ae922 2024-09-21T15:00:20.997 INFO:teuthology.orchestra.run.smithi032.stdout:3811 punch from 0x623e8ea to 0x62473ce, (0x8ae4 bytes) 2024-09-21T15:00:20.997 INFO:teuthology.orchestra.run.smithi032.stdout:3812 punch from 0x868fc51 to 0x8692a83, (0x2e32 bytes) 2024-09-21T15:00:20.997 INFO:teuthology.orchestra.run.smithi032.stdout:3813 read 0x9bbaf8a thru 0x9bc23d8 (0x744f bytes) 2024-09-21T15:00:21.003 INFO:teuthology.orchestra.run.smithi032.stdout:3814 trunc from 0xb4ae922 to 0x7c12072 2024-09-21T15:00:21.004 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:00:21.000+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:00:21.013 INFO:teuthology.orchestra.run.smithi032.stdout:3816 punch from 0x6d8b47d to 0x6d9629e, (0xae21 bytes) 2024-09-21T15:00:21.013 INFO:teuthology.orchestra.run.smithi032.stdout:3817 writesame 0x2e16f7 thru 0x2e31e8 data_size 0x1(0x1af2 bytes) 2024-09-21T15:00:21.022 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:21.022 INFO:teuthology.orchestra.run.smithi032.stdout:3820 read 0x36f10ab thru 0x36f161b (0x571 bytes) 2024-09-21T15:00:21.022 INFO:teuthology.orchestra.run.smithi032.stdout:3822 write 0x5797698 thru 0x57a6580 (0xeee9 bytes) 2024-09-21T15:00:21.031 INFO:teuthology.orchestra.run.smithi032.stdout:3824 punch from 0x5202d32 to 0x520b3fc, (0x86ca bytes) 2024-09-21T15:00:21.031 INFO:teuthology.orchestra.run.smithi032.stdout:3825 write 0x248a24c thru 0x248e004 (0x3db9 bytes) 2024-09-21T15:00:21.059 INFO:teuthology.orchestra.run.smithi032.stdout:3826 write 0xd58687c thru 0xd58db0b (0x7290 bytes) 2024-09-21T15:00:21.067 INFO:teuthology.orchestra.run.smithi032.stdout:3828 write 0x2e1e90c thru 0x2e1f0c2 (0x7b7 bytes) 2024-09-21T15:00:21.073 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:21.074 INFO:teuthology.orchestra.run.smithi032.stdout:3830 punch from 0xd1d00b3 to 0xd1d73a6, (0x72f3 bytes) 2024-09-21T15:00:21.074 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:21.074 INFO:teuthology.orchestra.run.smithi032.stdout:3832 read 0xbcd2667 thru 0xbcdfe5d (0xd7f7 bytes) 2024-09-21T15:00:21.074 INFO:teuthology.orchestra.run.smithi032.stdout:3833 writesame 0x466b97c thru 0x467ac2f data_size 0x31(0xf2b4 bytes) 2024-09-21T15:00:21.081 INFO:teuthology.orchestra.run.smithi032.stdout:3834 punch from 0xb2b94cd to 0xb2c0f03, (0x7a36 bytes) 2024-09-21T15:00:21.081 INFO:teuthology.orchestra.run.smithi032.stdout:3835 read 0x81e9fa0 thru 0x81f673f (0xc7a0 bytes) 2024-09-21T15:00:21.081 INFO:teuthology.orchestra.run.smithi032.stdout:3837 writesame 0xb782f51 thru 0xb7874d3 data_size 0x1(0x4583 bytes) 2024-09-21T15:00:21.103 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:00:21.098+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:00:21.107 INFO:teuthology.orchestra.run.smithi032.stdout:3838 trunc from 0xd58db0c to 0xa9fb83b 2024-09-21T15:00:21.109 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:21.110 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:00:21.110 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T15:00:21.110 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:21.120 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T15:00:21.129 INFO:teuthology.orchestra.run.smithi032.stdout:3839 read 0x3dc6ee9 thru 0x3dccf86 (0x609e bytes) 2024-09-21T15:00:21.129 INFO:teuthology.orchestra.run.smithi032.stdout:3840 write 0x822ba98 thru 0x822f6d3 (0x3c3c bytes) 2024-09-21T15:00:21.136 INFO:teuthology.orchestra.run.smithi032.stdout:3841 punch from 0x228073e to 0x2288f20, (0x87e2 bytes) 2024-09-21T15:00:21.136 INFO:teuthology.orchestra.run.smithi032.stdout:3842 writesame 0xa981c7a thru 0xa9838b1 data_size 0x2b(0x1c38 bytes) 2024-09-21T15:00:21.143 INFO:teuthology.orchestra.run.smithi032.stdout:3843 writesame 0x6d8e1ba thru 0x6d935b7 data_size 0xd(0x53fe bytes) 2024-09-21T15:00:21.151 INFO:teuthology.orchestra.run.smithi032.stdout:3845 writesame 0xa02c13b thru 0xa033e56 data_size 0x44(0x7d1c bytes) 2024-09-21T15:00:21.159 INFO:teuthology.orchestra.run.smithi032.stdout:3849 writesame 0xcde5ff thru 0xce0326 data_size 0xc(0x1d28 bytes) 2024-09-21T15:00:21.168 INFO:teuthology.orchestra.run.smithi032.stdout:3850 write 0x285e60d thru 0x2869237 (0xac2b bytes) 2024-09-21T15:00:21.177 INFO:teuthology.orchestra.run.smithi032.stdout:3851 read 0x1b8be39 thru 0x1b8c3ae (0x576 bytes) 2024-09-21T15:00:21.187 INFO:teuthology.orchestra.run.smithi032.stdout:3852 trunc from 0xa9fb83b to 0xe7a38e9 2024-09-21T15:00:21.197 INFO:teuthology.orchestra.run.smithi032.stdout:3853 writesame 0x298a7f3 thru 0x2996b1e data_size 0x2(0xc32c bytes) 2024-09-21T15:00:21.203 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:00:21.200+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:00:21.222 INFO:teuthology.orchestra.run.smithi032.stdout:3856 write 0x2701613 thru 0x270eb21 (0xd50f bytes) 2024-09-21T15:00:21.229 INFO:teuthology.orchestra.run.smithi032.stdout:3857 read 0xe082fdd thru 0xe08856e (0x5592 bytes) 2024-09-21T15:00:21.230 INFO:teuthology.orchestra.run.smithi032.stdout:3858 read 0x597904f thru 0x5986008 (0xcfba bytes) 2024-09-21T15:00:21.231 INFO:teuthology.orchestra.run.smithi032.stdout:3859 clone 28 order 24 su 2097152 sc 4 2024-09-21T15:00:21.234 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T15:00:21.250 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:21.250 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:00:21.250 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T15:00:21.250 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T15:00:21.250 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:21.260 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T15:00:21.305 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:00:21.304+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:00:21.369 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T15:00:21.401 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T15:00:21.405 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:00:21.404+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:00:21.407 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T15:00:21.417 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:21.417 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T15:00:21.417 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T15:00:21.417 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T15:00:21.417 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.418 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4281.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-21T15:00:21.418 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:09.324416+0000", 2024-09-21T15:00:21.418 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 132.09025883000001, 2024-09-21T15:00:21.418 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.117837947, 2024-09-21T15:00:21.418 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:21.418 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:21.418 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:21.418 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4281", 2024-09-21T15:00:21.418 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/1518520303", 2024-09-21T15:00:21.418 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1 2024-09-21T15:00:21.418 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.418 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:21.418 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.418 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:21.419 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324416+0000", 2024-09-21T15:00:21.419 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:21.419 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.419 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.419 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:21.419 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324416+0000", 2024-09-21T15:00:21.419 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:21.419 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.419 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.419 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:21.419 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324414+0000", 2024-09-21T15:00:21.419 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T15:00:21.419 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.419 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.420 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:21.420 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324429+0000", 2024-09-21T15:00:21.420 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5637e-05 2024-09-21T15:00:21.420 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.420 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.420 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:21.420 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324448+0000", 2024-09-21T15:00:21.420 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9042000000000002e-05 2024-09-21T15:00:21.420 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.420 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.420 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:21.420 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324465+0000", 2024-09-21T15:00:21.420 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6767999999999999e-05 2024-09-21T15:00:21.420 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.420 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.421 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:21.421 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324571+0000", 2024-09-21T15:00:21.421 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000105714 2024-09-21T15:00:21.421 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.421 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.421 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op must wait for map", 2024-09-21T15:00:21.421 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324587+0000", 2024-09-21T15:00:21.421 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5753999999999999e-05 2024-09-21T15:00:21.421 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.421 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.421 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:21.421 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.325864+0000", 2024-09-21T15:00:21.421 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0012773350000000001 2024-09-21T15:00:21.421 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.422 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.422 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:00:21.422 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.325878+0000", 2024-09-21T15:00:21.422 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4498000000000001e-05 2024-09-21T15:00:21.422 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.422 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.422 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:21.422 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.439461+0000", 2024-09-21T15:00:21.422 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.11358212500000001 2024-09-21T15:00:21.422 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.422 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.422 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:21.422 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.439848+0000", 2024-09-21T15:00:21.423 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00038747800000000002 2024-09-21T15:00:21.423 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.423 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.423 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T15:00:21.423 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.440175+0000", 2024-09-21T15:00:21.423 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00032721900000000001 2024-09-21T15:00:21.423 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.423 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.423 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:21.423 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.441770+0000", 2024-09-21T15:00:21.423 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0015949409999999999 2024-09-21T15:00:21.423 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.423 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.423 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:21.424 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.442086+0000", 2024-09-21T15:00:21.424 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00031586299999999999 2024-09-21T15:00:21.424 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.424 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.424 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:21.424 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.442195+0000", 2024-09-21T15:00:21.424 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000108791 2024-09-21T15:00:21.424 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.424 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.424 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:21.424 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.442254+0000", 2024-09-21T15:00:21.424 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9038e-05 2024-09-21T15:00:21.424 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:21.424 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:21.425 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:21.425 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.425 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.425 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4281.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-21T15:00:21.425 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:09.447828+0000", 2024-09-21T15:00:21.425 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 131.96684682899999, 2024-09-21T15:00:21.425 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.32904393700000001, 2024-09-21T15:00:21.425 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:21.425 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:21.425 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:21.425 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4281", 2024-09-21T15:00:21.425 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/1518520303", 2024-09-21T15:00:21.425 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 2 2024-09-21T15:00:21.425 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.425 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:21.426 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.426 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:21.426 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447828+0000", 2024-09-21T15:00:21.426 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:21.426 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.426 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.426 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:21.426 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447828+0000", 2024-09-21T15:00:21.426 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:21.426 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.426 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.426 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:21.426 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447825+0000", 2024-09-21T15:00:21.427 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T15:00:21.427 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.427 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.427 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:21.427 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447841+0000", 2024-09-21T15:00:21.427 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5359999999999999e-05 2024-09-21T15:00:21.427 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.427 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.427 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:21.427 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447875+0000", 2024-09-21T15:00:21.427 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4388999999999999e-05 2024-09-21T15:00:21.427 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.427 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.427 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:21.427 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447903+0000", 2024-09-21T15:00:21.428 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7373e-05 2024-09-21T15:00:21.428 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.428 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.428 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:21.428 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447996+0000", 2024-09-21T15:00:21.428 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2954999999999993e-05 2024-09-21T15:00:21.428 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.428 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.428 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:00:21.428 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.448011+0000", 2024-09-21T15:00:21.428 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5038999999999999e-05 2024-09-21T15:00:21.428 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.428 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.428 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:21.428 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.776501+0000", 2024-09-21T15:00:21.429 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.32849012500000002 2024-09-21T15:00:21.429 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.429 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.429 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:21.429 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.776872+0000", 2024-09-21T15:00:21.429 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00037119600000000002 2024-09-21T15:00:21.429 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:21.429 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:21.429 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:21.429 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.429 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.429 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:752 3.14 3:28fc02e5:::rbd_data.113693316f36.0000000000000010:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-21T15:00:21.429 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:26.625579+0000", 2024-09-21T15:00:21.429 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 114.78909597400001, 2024-09-21T15:00:21.429 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042965426000000001, 2024-09-21T15:00:21.430 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:21.430 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:21.430 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:21.430 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:21.430 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:21.430 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 752 2024-09-21T15:00:21.430 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.430 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:21.430 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.430 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:21.430 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625579+0000", 2024-09-21T15:00:21.430 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:21.430 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.430 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.431 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:21.431 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625579+0000", 2024-09-21T15:00:21.431 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:21.431 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.431 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.431 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:21.431 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625577+0000", 2024-09-21T15:00:21.431 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T15:00:21.431 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.431 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.431 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:21.431 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625588+0000", 2024-09-21T15:00:21.431 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1555e-05 2024-09-21T15:00:21.432 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.432 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.432 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:21.432 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625610+0000", 2024-09-21T15:00:21.432 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2107999999999999e-05 2024-09-21T15:00:21.432 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.432 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.432 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:21.432 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625621+0000", 2024-09-21T15:00:21.432 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0762e-05 2024-09-21T15:00:21.432 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.432 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.432 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:21.432 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625692+0000", 2024-09-21T15:00:21.433 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.1203000000000003e-05 2024-09-21T15:00:21.433 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.433 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.433 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:21.433 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625872+0000", 2024-09-21T15:00:21.433 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017962800000000001 2024-09-21T15:00:21.433 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.433 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.433 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T15:00:21.433 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.633188+0000", 2024-09-21T15:00:21.433 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.007316014 2024-09-21T15:00:21.433 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.433 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.433 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:21.434 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.668357+0000", 2024-09-21T15:00:21.434 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035168933999999999 2024-09-21T15:00:21.434 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.434 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.434 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:21.434 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.668418+0000", 2024-09-21T15:00:21.434 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1282e-05 2024-09-21T15:00:21.434 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.434 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.434 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:21.434 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.668486+0000", 2024-09-21T15:00:21.434 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8468999999999998e-05 2024-09-21T15:00:21.434 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.435 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.435 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:21.435 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.668544+0000", 2024-09-21T15:00:21.435 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7785000000000003e-05 2024-09-21T15:00:21.435 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:21.435 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:21.435 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:21.435 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.435 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.435 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22741 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 245760~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T15:00:21.435 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141107+0000", 2024-09-21T15:00:21.435 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 91.273567614000001, 2024-09-21T15:00:21.436 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036407729, 2024-09-21T15:00:21.436 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:21.436 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:21.436 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:21.436 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:21.436 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:21.436 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22741 2024-09-21T15:00:21.436 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.436 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:21.436 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.436 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:21.436 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141107+0000", 2024-09-21T15:00:21.436 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:21.436 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.436 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.436 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:21.437 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141107+0000", 2024-09-21T15:00:21.437 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:21.437 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.437 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.437 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:21.437 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141107+0000", 2024-09-21T15:00:21.437 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:21.437 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.437 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.437 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:21.437 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141109+0000", 2024-09-21T15:00:21.437 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6529999999999998e-06 2024-09-21T15:00:21.437 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.437 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.438 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:21.438 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141115+0000", 2024-09-21T15:00:21.438 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9649999999999996e-06 2024-09-21T15:00:21.438 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.438 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.438 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:21.438 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141121+0000", 2024-09-21T15:00:21.438 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1209999999999998e-06 2024-09-21T15:00:21.438 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.438 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.438 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:21.438 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177242+0000", 2024-09-21T15:00:21.438 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036121706000000003 2024-09-21T15:00:21.438 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.439 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.439 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:21.439 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177301+0000", 2024-09-21T15:00:21.439 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9008000000000002e-05 2024-09-21T15:00:21.439 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.439 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.439 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:21.439 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177515+0000", 2024-09-21T15:00:21.439 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00021362 2024-09-21T15:00:21.439 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:21.439 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:21.439 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:21.439 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.439 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.440 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22742 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 311296~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T15:00:21.440 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141126+0000", 2024-09-21T15:00:21.440 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 91.273549259999996, 2024-09-21T15:00:21.440 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036653537, 2024-09-21T15:00:21.440 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:21.440 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:21.440 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:21.440 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:21.440 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:21.440 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22742 2024-09-21T15:00:21.440 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.440 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:21.440 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.440 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:21.440 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141126+0000", 2024-09-21T15:00:21.440 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:21.441 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.441 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.441 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:21.441 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141126+0000", 2024-09-21T15:00:21.441 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:21.441 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.441 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.441 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:21.441 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141125+0000", 2024-09-21T15:00:21.441 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:21.441 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.441 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.441 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:21.442 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141128+0000", 2024-09-21T15:00:21.442 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.886e-06 2024-09-21T15:00:21.442 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.442 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.442 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:21.442 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141137+0000", 2024-09-21T15:00:21.442 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4399999999999994e-06 2024-09-21T15:00:21.442 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.442 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.442 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:21.442 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141143+0000", 2024-09-21T15:00:21.442 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1410000000000004e-06 2024-09-21T15:00:21.442 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.443 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.443 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:21.443 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177543+0000", 2024-09-21T15:00:21.443 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036400544999999999 2024-09-21T15:00:21.443 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.443 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.443 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:21.443 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177605+0000", 2024-09-21T15:00:21.443 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2206999999999996e-05 2024-09-21T15:00:21.443 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.443 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.443 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:21.443 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177779+0000", 2024-09-21T15:00:21.443 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017368099999999999 2024-09-21T15:00:21.443 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:21.444 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:21.444 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:21.444 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.444 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.444 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22743 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 376832~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T15:00:21.444 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141147+0000", 2024-09-21T15:00:21.444 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 91.273527775999995, 2024-09-21T15:00:21.444 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036915158000000003, 2024-09-21T15:00:21.444 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:21.444 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:21.444 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:21.444 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:21.444 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:21.444 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22743 2024-09-21T15:00:21.445 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.445 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:21.445 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.445 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:21.445 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141147+0000", 2024-09-21T15:00:21.445 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:21.445 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.445 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.445 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:21.445 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141147+0000", 2024-09-21T15:00:21.445 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:21.445 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.445 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.445 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:21.446 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141147+0000", 2024-09-21T15:00:21.446 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:21.446 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.446 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.446 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:21.446 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141149+0000", 2024-09-21T15:00:21.446 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.79e-06 2024-09-21T15:00:21.446 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.446 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.446 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:21.446 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141155+0000", 2024-09-21T15:00:21.446 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6699999999999999e-06 2024-09-21T15:00:21.446 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.446 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.447 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:21.447 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141160+0000", 2024-09-21T15:00:21.447 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5759999999999999e-06 2024-09-21T15:00:21.447 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.447 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.447 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:21.447 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177817+0000", 2024-09-21T15:00:21.447 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036657404999999997 2024-09-21T15:00:21.447 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.447 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.447 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:21.447 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177892+0000", 2024-09-21T15:00:21.447 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4602000000000002e-05 2024-09-21T15:00:21.447 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.448 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.448 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:21.448 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178062+0000", 2024-09-21T15:00:21.448 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000170464 2024-09-21T15:00:21.448 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:21.448 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:21.448 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:21.448 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.448 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.448 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22744 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 442368~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T15:00:21.448 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141164+0000", 2024-09-21T15:00:21.448 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 91.273510642999995, 2024-09-21T15:00:21.448 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037120806999999999, 2024-09-21T15:00:21.448 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:21.449 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:21.449 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:21.449 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:21.449 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:21.449 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22744 2024-09-21T15:00:21.449 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.449 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:21.449 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.449 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:21.449 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141164+0000", 2024-09-21T15:00:21.449 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:21.449 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.449 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.449 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:21.449 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141164+0000", 2024-09-21T15:00:21.450 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:21.450 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.450 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.450 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:21.450 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141164+0000", 2024-09-21T15:00:21.450 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:21.450 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.450 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.450 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:21.450 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141167+0000", 2024-09-21T15:00:21.450 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7159999999999999e-06 2024-09-21T15:00:21.450 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.451 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.451 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:21.451 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141173+0000", 2024-09-21T15:00:21.451 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.071e-06 2024-09-21T15:00:21.451 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.451 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.451 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:21.451 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141177+0000", 2024-09-21T15:00:21.451 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3610000000000003e-06 2024-09-21T15:00:21.451 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.451 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.451 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:21.451 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178083+0000", 2024-09-21T15:00:21.452 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036906405000000003 2024-09-21T15:00:21.452 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.452 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.452 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:21.452 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178143+0000", 2024-09-21T15:00:21.452 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0056999999999998e-05 2024-09-21T15:00:21.452 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.452 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.452 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:21.452 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178285+0000", 2024-09-21T15:00:21.452 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000141555 2024-09-21T15:00:21.452 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:21.452 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:21.452 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:21.452 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.453 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.453 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22745 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 507904~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T15:00:21.453 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141182+0000", 2024-09-21T15:00:21.453 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 91.273492322999999, 2024-09-21T15:00:21.453 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037332331000000003, 2024-09-21T15:00:21.453 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:21.453 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:21.453 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:21.453 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:21.453 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:21.453 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22745 2024-09-21T15:00:21.453 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.453 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:21.453 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.453 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:21.454 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141182+0000", 2024-09-21T15:00:21.454 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:21.454 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.454 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.454 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:21.454 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141182+0000", 2024-09-21T15:00:21.454 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:21.454 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.454 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.454 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:21.454 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141182+0000", 2024-09-21T15:00:21.454 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:21.454 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.454 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.455 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:21.455 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141185+0000", 2024-09-21T15:00:21.455 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.464e-06 2024-09-21T15:00:21.455 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.455 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.455 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:21.455 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141200+0000", 2024-09-21T15:00:21.455 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5886999999999999e-05 2024-09-21T15:00:21.455 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.455 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.455 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:21.455 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141571+0000", 2024-09-21T15:00:21.455 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000370909 2024-09-21T15:00:21.455 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.456 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.456 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:21.456 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178318+0000", 2024-09-21T15:00:21.456 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036746244999999997 2024-09-21T15:00:21.456 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.456 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.456 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:21.456 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178367+0000", 2024-09-21T15:00:21.456 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9317000000000002e-05 2024-09-21T15:00:21.456 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.456 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.456 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:21.456 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178515+0000", 2024-09-21T15:00:21.456 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014784600000000001 2024-09-21T15:00:21.456 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:21.457 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:21.457 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:21.457 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.457 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.457 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22746 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 573440~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T15:00:21.457 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141599+0000", 2024-09-21T15:00:21.457 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 91.273075691000003, 2024-09-21T15:00:21.457 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037402249999999998, 2024-09-21T15:00:21.457 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:21.457 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:21.457 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:21.457 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:21.457 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:21.457 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22746 2024-09-21T15:00:21.458 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.458 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:21.458 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.458 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:21.458 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141599+0000", 2024-09-21T15:00:21.458 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:21.458 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.458 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.458 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:21.458 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141599+0000", 2024-09-21T15:00:21.458 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:21.458 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.458 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.458 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:21.459 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141598+0000", 2024-09-21T15:00:21.459 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:00:21.459 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.459 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.459 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:21.459 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141637+0000", 2024-09-21T15:00:21.459 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.9119999999999998e-05 2024-09-21T15:00:21.459 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.459 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.459 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:21.459 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141647+0000", 2024-09-21T15:00:21.459 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5570000000000001e-06 2024-09-21T15:00:21.459 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.460 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.460 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:21.460 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141654+0000", 2024-09-21T15:00:21.460 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2049999999999996e-06 2024-09-21T15:00:21.460 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.460 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.460 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:21.460 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178553+0000", 2024-09-21T15:00:21.460 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036898528 2024-09-21T15:00:21.460 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.460 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.460 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:21.460 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178625+0000", 2024-09-21T15:00:21.460 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2669999999999994e-05 2024-09-21T15:00:21.460 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.461 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.461 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:21.461 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179001+0000", 2024-09-21T15:00:21.461 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000375977 2024-09-21T15:00:21.461 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:21.461 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:21.461 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:21.461 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.461 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.461 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22747 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 638976~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T15:00:21.461 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141660+0000", 2024-09-21T15:00:21.461 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 91.273014548999996, 2024-09-21T15:00:21.461 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037617352999999999, 2024-09-21T15:00:21.462 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:21.462 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:21.462 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:21.462 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:21.462 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:21.462 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22747 2024-09-21T15:00:21.462 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.462 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:21.462 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.462 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:21.462 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141660+0000", 2024-09-21T15:00:21.462 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:21.462 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.463 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.463 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:21.463 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141660+0000", 2024-09-21T15:00:21.463 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:21.463 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.463 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.463 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:21.463 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141660+0000", 2024-09-21T15:00:21.463 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:21.463 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.463 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.463 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:21.463 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141663+0000", 2024-09-21T15:00:21.463 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.569e-06 2024-09-21T15:00:21.464 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.464 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.464 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:21.464 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141670+0000", 2024-09-21T15:00:21.464 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5719999999999999e-06 2024-09-21T15:00:21.464 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.464 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.464 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:21.464 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141675+0000", 2024-09-21T15:00:21.464 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2889999999999997e-06 2024-09-21T15:00:21.464 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.464 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.464 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:21.464 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179028+0000", 2024-09-21T15:00:21.465 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037353381999999997 2024-09-21T15:00:21.465 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.465 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.465 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:21.465 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179099+0000", 2024-09-21T15:00:21.465 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0259000000000006e-05 2024-09-21T15:00:21.465 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.465 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.465 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:21.465 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179278+0000", 2024-09-21T15:00:21.465 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000178908 2024-09-21T15:00:21.465 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:21.465 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:21.465 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:21.466 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.466 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.466 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22748 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 704512~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T15:00:21.466 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141680+0000", 2024-09-21T15:00:21.466 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 91.272994910999998, 2024-09-21T15:00:21.466 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037806166000000002, 2024-09-21T15:00:21.466 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:21.466 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:21.466 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:21.466 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:21.466 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:21.466 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22748 2024-09-21T15:00:21.466 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.466 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:21.467 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.467 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:21.467 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141680+0000", 2024-09-21T15:00:21.467 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:21.467 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.467 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.467 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:21.467 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141680+0000", 2024-09-21T15:00:21.467 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:21.467 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.467 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.467 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:21.467 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141679+0000", 2024-09-21T15:00:21.467 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:21.468 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.468 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.468 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:21.468 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141683+0000", 2024-09-21T15:00:21.468 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3230000000000002e-06 2024-09-21T15:00:21.468 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.468 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.468 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:21.468 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141690+0000", 2024-09-21T15:00:21.468 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8680000000000001e-06 2024-09-21T15:00:21.468 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.468 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.468 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:21.468 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141694+0000", 2024-09-21T15:00:21.468 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6770000000000003e-06 2024-09-21T15:00:21.469 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.469 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.469 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:21.469 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179300+0000", 2024-09-21T15:00:21.469 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037605567999999999 2024-09-21T15:00:21.469 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.469 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.469 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:21.469 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179355+0000", 2024-09-21T15:00:21.469 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4771000000000001e-05 2024-09-21T15:00:21.469 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.469 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.469 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:21.469 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179486+0000", 2024-09-21T15:00:21.470 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013141699999999999 2024-09-21T15:00:21.470 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:21.470 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:21.470 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:21.470 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.470 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.470 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22749 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 770048~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T15:00:21.470 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141698+0000", 2024-09-21T15:00:21.470 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 91.272976538999998, 2024-09-21T15:00:21.470 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037967813000000003, 2024-09-21T15:00:21.470 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:21.470 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:21.470 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:21.470 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:21.470 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:21.471 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22749 2024-09-21T15:00:21.471 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.471 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:21.471 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.471 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:21.471 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141698+0000", 2024-09-21T15:00:21.471 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:21.471 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.471 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.471 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:21.471 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141698+0000", 2024-09-21T15:00:21.471 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:21.471 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.471 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.472 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:21.472 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141698+0000", 2024-09-21T15:00:21.472 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:21.472 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.472 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.472 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:21.472 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141701+0000", 2024-09-21T15:00:21.472 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8399999999999999e-06 2024-09-21T15:00:21.472 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.472 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.472 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:21.472 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141706+0000", 2024-09-21T15:00:21.472 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4979999999999997e-06 2024-09-21T15:00:21.472 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.472 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.472 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:21.473 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141711+0000", 2024-09-21T15:00:21.473 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3850000000000004e-06 2024-09-21T15:00:21.473 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.473 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.473 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:21.473 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179504+0000", 2024-09-21T15:00:21.473 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037793514 2024-09-21T15:00:21.473 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.473 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.473 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:21.473 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179546+0000", 2024-09-21T15:00:21.473 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1467999999999997e-05 2024-09-21T15:00:21.473 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.473 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.474 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:21.474 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179666+0000", 2024-09-21T15:00:21.474 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000120499 2024-09-21T15:00:21.474 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:21.474 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:21.474 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:21.474 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.474 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.474 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22750 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 835584~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T15:00:21.474 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141715+0000", 2024-09-21T15:00:21.474 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 91.272960237999996, 2024-09-21T15:00:21.474 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038173179000000002, 2024-09-21T15:00:21.474 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:21.475 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:21.475 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:21.475 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:21.475 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:21.475 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22750 2024-09-21T15:00:21.475 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.475 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:21.475 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.475 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:21.475 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141715+0000", 2024-09-21T15:00:21.475 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:21.475 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.476 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.476 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:21.476 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141715+0000", 2024-09-21T15:00:21.476 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:21.476 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.476 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.476 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:21.476 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141714+0000", 2024-09-21T15:00:21.476 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:21.476 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.476 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.476 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:21.476 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141716+0000", 2024-09-21T15:00:21.476 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1730000000000002e-06 2024-09-21T15:00:21.477 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.477 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.477 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:21.477 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141722+0000", 2024-09-21T15:00:21.477 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.417e-06 2024-09-21T15:00:21.477 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.477 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.477 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:21.477 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141726+0000", 2024-09-21T15:00:21.477 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4830000000000001e-06 2024-09-21T15:00:21.477 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.477 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.477 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:21.477 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179683+0000", 2024-09-21T15:00:21.477 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037956977000000003 2024-09-21T15:00:21.478 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.478 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.478 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:21.478 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179751+0000", 2024-09-21T15:00:21.478 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7762000000000006e-05 2024-09-21T15:00:21.478 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.478 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.478 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:21.478 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179888+0000", 2024-09-21T15:00:21.478 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000136635 2024-09-21T15:00:21.478 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:21.478 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:21.478 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:21.479 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.479 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.479 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22751 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 901120~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T15:00:21.479 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141730+0000", 2024-09-21T15:00:21.479 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 91.272944439, 2024-09-21T15:00:21.479 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038357011000000003, 2024-09-21T15:00:21.479 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:21.479 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:21.479 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:21.479 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:21.479 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:21.479 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22751 2024-09-21T15:00:21.479 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.479 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:21.480 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.480 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:21.480 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141730+0000", 2024-09-21T15:00:21.480 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:21.480 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.480 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.480 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:21.480 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141730+0000", 2024-09-21T15:00:21.480 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:21.480 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.480 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.480 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:21.480 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141730+0000", 2024-09-21T15:00:21.480 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:21.480 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.480 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.481 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:21.481 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141737+0000", 2024-09-21T15:00:21.481 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.64e-06 2024-09-21T15:00:21.481 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.481 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.481 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:21.481 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141743+0000", 2024-09-21T15:00:21.481 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8130000000000001e-06 2024-09-21T15:00:21.481 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.481 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.481 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:21.481 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141747+0000", 2024-09-21T15:00:21.481 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4920000000000004e-06 2024-09-21T15:00:21.481 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.481 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.482 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:21.482 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179918+0000", 2024-09-21T15:00:21.482 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038170822 2024-09-21T15:00:21.482 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.482 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.482 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:21.482 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179960+0000", 2024-09-21T15:00:21.482 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1999000000000002e-05 2024-09-21T15:00:21.482 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.482 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.482 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:21.482 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180087+0000", 2024-09-21T15:00:21.482 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012752799999999999 2024-09-21T15:00:21.482 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:21.482 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:21.483 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:21.483 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.483 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.483 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22752 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 966656~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T15:00:21.483 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141751+0000", 2024-09-21T15:00:21.483 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 91.272924118999995, 2024-09-21T15:00:21.483 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038553084000000001, 2024-09-21T15:00:21.483 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:21.483 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:21.483 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:21.483 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:21.483 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:21.483 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22752 2024-09-21T15:00:21.483 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.484 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:21.484 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.484 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:21.484 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141751+0000", 2024-09-21T15:00:21.484 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:21.484 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.484 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.484 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:21.484 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141751+0000", 2024-09-21T15:00:21.484 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:21.484 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.484 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.484 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:21.484 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141750+0000", 2024-09-21T15:00:21.484 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:21.485 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.485 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.485 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:21.485 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141753+0000", 2024-09-21T15:00:21.485 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5900000000000002e-06 2024-09-21T15:00:21.485 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.485 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.485 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:21.485 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141759+0000", 2024-09-21T15:00:21.485 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.947e-06 2024-09-21T15:00:21.485 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.485 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.485 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:21.485 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141764+0000", 2024-09-21T15:00:21.485 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8319999999999996e-06 2024-09-21T15:00:21.485 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.486 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.486 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:21.486 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180108+0000", 2024-09-21T15:00:21.486 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038344095000000002 2024-09-21T15:00:21.486 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.486 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.486 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:21.486 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180168+0000", 2024-09-21T15:00:21.486 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0047000000000003e-05 2024-09-21T15:00:21.486 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.486 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.486 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:21.486 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180304+0000", 2024-09-21T15:00:21.486 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013592500000000001 2024-09-21T15:00:21.486 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:21.487 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:21.487 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:21.487 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.487 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.487 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22753 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 1032192~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T15:00:21.487 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141768+0000", 2024-09-21T15:00:21.487 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 91.272907244999999, 2024-09-21T15:00:21.487 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038818810000000002, 2024-09-21T15:00:21.487 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:21.487 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:21.487 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:21.487 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:21.487 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:21.487 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22753 2024-09-21T15:00:21.487 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.488 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:21.488 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.488 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:21.488 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141768+0000", 2024-09-21T15:00:21.488 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:21.488 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.488 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.488 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:21.488 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141768+0000", 2024-09-21T15:00:21.488 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:21.488 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.488 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.488 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:21.488 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141767+0000", 2024-09-21T15:00:21.488 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:21.488 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.489 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.489 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:21.489 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141770+0000", 2024-09-21T15:00:21.489 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3329999999999999e-06 2024-09-21T15:00:21.489 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.489 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.489 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:21.489 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141775+0000", 2024-09-21T15:00:21.489 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2669999999999997e-06 2024-09-21T15:00:21.489 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.489 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.489 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:21.489 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141779+0000", 2024-09-21T15:00:21.489 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2259999999999999e-06 2024-09-21T15:00:21.489 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.489 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.490 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:21.490 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180325+0000", 2024-09-21T15:00:21.490 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038546428000000001 2024-09-21T15:00:21.490 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.490 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.490 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:21.490 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180401+0000", 2024-09-21T15:00:21.490 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5437000000000003e-05 2024-09-21T15:00:21.490 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.490 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.490 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:21.490 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180586+0000", 2024-09-21T15:00:21.490 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000185429 2024-09-21T15:00:21.490 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:21.490 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:21.491 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:21.491 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.491 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.491 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:73793 3.11 3:8c8a112d:::rbd_data.1136f4939602.0000000000000003:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-21T15:00:21.491 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:42.225633+0000", 2024-09-21T15:00:21.491 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 39.189041330999999, 2024-09-21T15:00:21.491 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036337102000000003, 2024-09-21T15:00:21.491 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:21.491 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:21.491 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:21.491 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:21.491 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:21.491 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 73793 2024-09-21T15:00:21.491 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.491 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:21.492 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.492 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:21.492 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.225633+0000", 2024-09-21T15:00:21.492 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:21.492 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.492 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.492 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:21.492 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.225633+0000", 2024-09-21T15:00:21.492 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:21.492 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.492 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.492 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:21.492 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.225630+0000", 2024-09-21T15:00:21.492 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999967 2024-09-21T15:00:21.492 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.493 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.493 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:21.493 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.225649+0000", 2024-09-21T15:00:21.493 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9377000000000001e-05 2024-09-21T15:00:21.493 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.493 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.493 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:21.493 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.225689+0000", 2024-09-21T15:00:21.493 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.9879999999999998e-05 2024-09-21T15:00:21.493 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.493 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.493 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:21.493 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.225706+0000", 2024-09-21T15:00:21.493 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6537000000000001e-05 2024-09-21T15:00:21.493 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.494 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.494 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:21.494 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.225772+0000", 2024-09-21T15:00:21.494 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5894000000000003e-05 2024-09-21T15:00:21.494 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.494 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.494 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:21.494 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.225846+0000", 2024-09-21T15:00:21.494 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4165000000000006e-05 2024-09-21T15:00:21.494 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.494 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.494 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 1", 2024-09-21T15:00:21.494 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.231400+0000", 2024-09-21T15:00:21.495 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0055538269999999999 2024-09-21T15:00:21.495 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.495 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.495 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:21.495 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.261709+0000", 2024-09-21T15:00:21.495 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.030309750999999999 2024-09-21T15:00:21.495 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.495 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.495 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:21.495 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.261738+0000", 2024-09-21T15:00:21.495 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8569999999999999e-05 2024-09-21T15:00:21.495 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.495 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.495 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:21.495 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.261839+0000", 2024-09-21T15:00:21.496 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010118 2024-09-21T15:00:21.496 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.496 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.496 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:21.496 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.261971+0000", 2024-09-21T15:00:21.496 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000131414 2024-09-21T15:00:21.496 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:21.496 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:21.496 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:21.496 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.496 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.496 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:97673 3.1d 3:bd91946e:::rbd_data.1136bfc79a70.0000000000000002:head [write 0~4718592 [fadvise_sequential+fadvise_nocache],write 5242880~32768 [fadvise_sequential+fadvise_nocache],write 5341184~147456 [fadvise_sequential+fadvise_nocache],write 5554176~147456 [fadvise_sequential+fadvise_nocache],write 5767168~16384 [fadvise_sequential+fadvise_nocache],write 5849088~147456 [fadvise_sequential+fadvise_nocache],write 6062080~147456 [fadvise_sequential+fadvise_nocache],write 6275072~16384 [fadvise_sequential+fadvise_nocache],write 6356992~147456 [fadvise_sequential+fadvise_nocache],write 6569984~147456 [fadvise_sequential+fadvise_nocache],write 6782976~32768 [fadvise_sequential+fadvise_nocache],write 6864896~147456 [fadvise_sequential+fadvise_nocache],write 7077888~147456 [fadvise_sequential+fadvise_nocache],write 7290880~49152 [fadvise_sequential+fadvise_nocache],write 7372800~147456 [fadvise_sequential+fadvise_nocache],write 7585792~147456 [fadvise_sequential+fadvise_nocache],write 7798784~65536 [fadvise_sequential+fadvise_nocache],write 7880704~147456 [fadvise_sequential+fadvise_nocache],write 8093696~147456 [fadvise_sequential+fadvise_nocache],write 8306688~81920 [fadvise_sequential+fadvise_nocache],write 8421376~114688 [fadvise_sequential+fadvise_nocache],write 8634368~114688 [fadvise_sequential+fadvise_nocache],write 8847360~65536 [fadvise_sequential+fadvise_nocache],write 9011200~32768 [fadvise_sequential+fadvise_nocache],write 9224192~32768 [fadvise_sequential+fadvise_nocache],write 9519104~32768 [fadvise_sequential+fadvise_nocache],write 9732096~32768 [fadvise_sequential+fadvise_nocache],write 9945088~16384 [fadvise_sequential+fadvise_nocache],write 9977856~196608 [fadvise_sequential+fadvise_nocache],write 10190848~196608 [fadvise_sequential+fadvise_nocache],write 10403840~278528 [fadvise_sequential+fadvise_nocache],write 10698752~196608 [fadvise_sequential+fadvise_nocache],write 10911744~278528 [fadvise_sequential+fadvise_nocache],write 11206656~196608 [fadvise_sequential+fadvise_nocache],write 11419648~196608 [fadvise_sequential+fadvise_nocache],write 11632640~65536 [fadvise_sequential+fadvise_nocache],write 11730944~98304 [fadvise_sequential+fadvise_nocache],write 11845632~65536 [fadvise_sequential+fadvise_nocache],write 11943936~98304 [fadvise_sequential+fadvise_nocache],write 12058624~65536 [fadvise_sequential+fadvise_nocache],write 12140544~65536 [fadvise_sequential+fadvise_nocache],write 12238848~98304 [fadvise_sequential+fadvise_nocache],write 12353536~65536 [fadvise_sequential+fadvise_nocache],write 12451840~98304 [fadvise_sequential+fadvise_nocache],write 12566528~65536 [fadvise_sequential+fadvise_nocache],write 12746752~98304 [fadvise_sequential+fadvise_nocache],write 12959744~98304 [fadvise_sequential+fadvise_nocache],write 13107200~32768 [fadvise_sequential+fadvise_nocache],write 13254656~98304 [fadvise_sequential+fadvise_nocache],write 13467648~98304 [fadvise_sequential+fadvise_nocache],write 13631488~16384 [fadvise_sequential+fadvise_nocache],write 13762560~98304 [fadvise_sequential+fadvise_nocache],write 13975552~98304 [fadvise_sequential+fadvise_nocache],write 15204352~49152 [fadvise_sequential+fadvise_nocache],write 15384576~81920 [fadvise_sequential+fadvise_nocache],write 15597568~81920 [fadvise_sequential+fadvise_nocache],write 15728640~32768 [fadvise_sequential+fadvise_nocache],write 15892480~81920 [fadvise_sequential+fadvise_nocache],write 16105472~81920 [fadvise_sequential+fadvise_nocache],write 16252928~16384 [fadvise_sequential+fadvise_nocache],write 16400384~81920 [fadvise_sequential+fadvise_nocache],write 16613376~81920 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:21.496 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.300719+0000", 2024-09-21T15:00:21.496 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 16.113956056999999, 2024-09-21T15:00:21.496 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.068947114000000004, 2024-09-21T15:00:21.497 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:21.497 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:21.497 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:21.497 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:21.497 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:21.497 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 97673 2024-09-21T15:00:21.497 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.497 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:21.497 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.497 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:21.497 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.300719+0000", 2024-09-21T15:00:21.497 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:21.497 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.497 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.497 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:21.497 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.300719+0000", 2024-09-21T15:00:21.498 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:21.498 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.498 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.498 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:21.498 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.300716+0000", 2024-09-21T15:00:21.498 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-21T15:00:21.498 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.498 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.498 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:21.498 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.308380+0000", 2024-09-21T15:00:21.498 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0076638519999999996 2024-09-21T15:00:21.498 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.498 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.498 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:21.499 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.308418+0000", 2024-09-21T15:00:21.499 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.8380000000000002e-05 2024-09-21T15:00:21.499 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.499 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.499 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:21.499 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.308431+0000", 2024-09-21T15:00:21.499 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3213999999999999e-05 2024-09-21T15:00:21.499 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.499 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.499 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:21.499 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.308562+0000", 2024-09-21T15:00:21.499 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013015999999999999 2024-09-21T15:00:21.499 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.499 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.499 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:21.500 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.309173+0000", 2024-09-21T15:00:21.500 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00061185500000000002 2024-09-21T15:00:21.500 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.500 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.500 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 0", 2024-09-21T15:00:21.500 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311612+0000", 2024-09-21T15:00:21.500 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0024383949999999999 2024-09-21T15:00:21.500 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.500 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.500 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:21.500 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.355482+0000", 2024-09-21T15:00:21.500 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.043869940000000003 2024-09-21T15:00:21.500 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.500 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.500 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:21.501 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.369306+0000", 2024-09-21T15:00:21.501 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013824279 2024-09-21T15:00:21.501 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.501 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.501 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:21.501 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.369575+0000", 2024-09-21T15:00:21.501 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000269404 2024-09-21T15:00:21.501 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.501 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.501 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:21.501 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.369666+0000", 2024-09-21T15:00:21.501 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0426000000000005e-05 2024-09-21T15:00:21.501 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:21.501 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:21.501 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:21.501 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.502 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.502 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:97678 3.d 3:b2f34c6d:::rbd_data.1136bfc79a70.0000000000000006:head [write 0~4194304 [fadvise_sequential+fadvise_nocache],write 4718592~81920 [fadvise_sequential+fadvise_nocache],write 4866048~147456 [fadvise_sequential+fadvise_nocache],write 5079040~147456 [fadvise_sequential+fadvise_nocache],write 5242880~65536 [fadvise_sequential+fadvise_nocache],write 5373952~147456 [fadvise_sequential+fadvise_nocache],write 5586944~147456 [fadvise_sequential+fadvise_nocache],write 5767168~49152 [fadvise_sequential+fadvise_nocache],write 5881856~147456 [fadvise_sequential+fadvise_nocache],write 6094848~147456 [fadvise_sequential+fadvise_nocache],write 6291456~32768 [fadvise_sequential+fadvise_nocache],write 6389760~147456 [fadvise_sequential+fadvise_nocache],write 6602752~147456 [fadvise_sequential+fadvise_nocache],write 6815744~16384 [fadvise_sequential+fadvise_nocache],write 6897664~147456 [fadvise_sequential+fadvise_nocache],write 7110656~147456 [fadvise_sequential+fadvise_nocache],write 7323648~16384 [fadvise_sequential+fadvise_nocache],write 7405568~147456 [fadvise_sequential+fadvise_nocache],write 7618560~147456 [fadvise_sequential+fadvise_nocache],write 7831552~32768 [fadvise_sequential+fadvise_nocache],write 7946240~114688 [fadvise_sequential+fadvise_nocache],write 8159232~114688 [fadvise_sequential+fadvise_nocache],write 8372224~16384 [fadvise_sequential+fadvise_nocache],write 8454144~114688 [fadvise_sequential+fadvise_nocache],write 8667136~114688 [fadvise_sequential+fadvise_nocache],write 8880128~32768 [fadvise_sequential+fadvise_nocache],write 9043968~32768 [fadvise_sequential+fadvise_nocache],write 9256960~32768 [fadvise_sequential+fadvise_nocache],write 9551872~32768 [fadvise_sequential+fadvise_nocache],write 9764864~32768 [fadvise_sequential+fadvise_nocache],write 9961472~32768 [fadvise_sequential+fadvise_nocache],write 10010624~196608 [fadvise_sequential+fadvise_nocache],write 10223616~196608 [fadvise_sequential+fadvise_nocache],write 10436608~65536 [fadvise_sequential+fadvise_nocache],write 10518528~196608 [fadvise_sequential+fadvise_nocache],write 10731520~196608 [fadvise_sequential+fadvise_nocache],write 10944512~65536 [fadvise_sequential+fadvise_nocache],write 11026432~196608 [fadvise_sequential+fadvise_nocache],write 11239424~196608 [fadvise_sequential+fadvise_nocache],write 11452416~81920 [fadvise_sequential+fadvise_nocache],write 11550720~98304 [fadvise_sequential+fadvise_nocache],write 11665408~65536 [fadvise_sequential+fadvise_nocache],write 11763712~98304 [fadvise_sequential+fadvise_nocache],write 11878400~65536 [fadvise_sequential+fadvise_nocache],write 11976704~180224 [fadvise_sequential+fadvise_nocache],write 12173312~65536 [fadvise_sequential+fadvise_nocache],write 12271616~98304 [fadvise_sequential+fadvise_nocache],write 12386304~65536 [fadvise_sequential+fadvise_nocache],write 12484608~180224 [fadvise_sequential+fadvise_nocache],write 12779520~98304 [fadvise_sequential+fadvise_nocache],write 12992512~98304 [fadvise_sequential+fadvise_nocache],write 13107200~65536 [fadvise_sequential+fadvise_nocache],write 13287424~98304 [fadvise_sequential+fadvise_nocache],write 13500416~98304 [fadvise_sequential+fadvise_nocache],write 13631488~49152 [fadvise_sequential+fadvise_nocache],write 13795328~98304 [fadvise_sequential+fadvise_nocache],write 14008320~98304 [fadvise_sequential+fadvise_nocache],write 14696448~81920 [fadvise_sequential+fadvise_nocache],write 14909440~81920 [fadvise_sequential+fadvise_nocache],write 15122432~163840 [fadvise_sequential+fadvise_nocache],write 15417344~81920 [fadvise_sequential+fadvise_nocache],write 15630336~81920 [fadvise_sequential+fadvise_nocache],write 15728640~65536 [fadvise_sequential+fadvise_nocache],write 15925248~81920 [fadvise_sequential+fadvise_nocache],write 16138240~81920 [fadvise_sequential+fadvise_nocache],write 16252928~49152 [fadvise_sequential+fadvise_nocache],write 16433152~81920 [fadvise_sequential+fadvise_nocache],write 16646144~81920 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:21.502 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.308471+0000", 2024-09-21T15:00:21.502 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 16.106203606000001, 2024-09-21T15:00:21.502 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.047271776000000001, 2024-09-21T15:00:21.502 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:21.502 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:21.502 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:21.502 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:21.502 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:21.502 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 97678 2024-09-21T15:00:21.502 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.502 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:21.502 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.502 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:21.503 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.308471+0000", 2024-09-21T15:00:21.503 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:21.503 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.503 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.503 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:21.503 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.308471+0000", 2024-09-21T15:00:21.503 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:21.503 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.503 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.503 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:21.503 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.308469+0000", 2024-09-21T15:00:21.503 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-21T15:00:21.503 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.503 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.503 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:21.504 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.317731+0000", 2024-09-21T15:00:21.504 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0092614949999999998 2024-09-21T15:00:21.504 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.504 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.504 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:21.504 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.317778+0000", 2024-09-21T15:00:21.504 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7574000000000002e-05 2024-09-21T15:00:21.504 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.504 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.504 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:21.504 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.317794+0000", 2024-09-21T15:00:21.504 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5537e-05 2024-09-21T15:00:21.504 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.504 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.504 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:21.505 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.317871+0000", 2024-09-21T15:00:21.505 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7163999999999995e-05 2024-09-21T15:00:21.505 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.505 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.505 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:21.505 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.318166+0000", 2024-09-21T15:00:21.505 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00029492400000000001 2024-09-21T15:00:21.505 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.505 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.505 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 1", 2024-09-21T15:00:21.505 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.320083+0000", 2024-09-21T15:00:21.505 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001916933 2024-09-21T15:00:21.505 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.505 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.505 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:21.505 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.354762+0000", 2024-09-21T15:00:21.506 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034679481999999998 2024-09-21T15:00:21.506 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.506 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.506 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:21.506 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.355494+0000", 2024-09-21T15:00:21.506 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00073167399999999997 2024-09-21T15:00:21.506 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.506 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.506 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:21.506 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.355683+0000", 2024-09-21T15:00:21.506 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00018951699999999999 2024-09-21T15:00:21.506 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.506 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.506 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:21.506 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.355743+0000", 2024-09-21T15:00:21.507 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9493000000000001e-05 2024-09-21T15:00:21.507 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:21.507 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:21.507 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:21.507 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.507 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.507 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:98562 3.9 3:911e52bc:::rbd_data.1136bfc79a70.000000000000000a:head [write 0~4194304 [fadvise_sequential+fadvise_nocache],write 4718592~114688 [fadvise_sequential+fadvise_nocache],write 4898816~147456 [fadvise_sequential+fadvise_nocache],write 5111808~229376 [fadvise_sequential+fadvise_nocache],write 5406720~147456 [fadvise_sequential+fadvise_nocache],write 5619712~229376 [fadvise_sequential+fadvise_nocache],write 5914624~147456 [fadvise_sequential+fadvise_nocache],write 6127616~147456 [fadvise_sequential+fadvise_nocache],write 6291456~65536 [fadvise_sequential+fadvise_nocache],write 6422528~147456 [fadvise_sequential+fadvise_nocache],write 6635520~147456 [fadvise_sequential+fadvise_nocache],write 6815744~49152 [fadvise_sequential+fadvise_nocache],write 6930432~147456 [fadvise_sequential+fadvise_nocache],write 7143424~147456 [fadvise_sequential+fadvise_nocache],write 7340032~32768 [fadvise_sequential+fadvise_nocache],write 7438336~147456 [fadvise_sequential+fadvise_nocache],write 7651328~147456 [fadvise_sequential+fadvise_nocache],write 7864320~16384 [fadvise_sequential+fadvise_nocache],write 7979008~114688 [fadvise_sequential+fadvise_nocache],write 8192000~114688 [fadvise_sequential+fadvise_nocache],write 8519680~81920 [fadvise_sequential+fadvise_nocache],write 8732672~38263 [fadvise_sequential+fadvise_nocache],write 8781824~32768 [fadvise_sequential+fadvise_nocache],write 9076736~32768 [fadvise_sequential+fadvise_nocache],write 9289728~32768 [fadvise_sequential+fadvise_nocache],write 9437184~81920 [fadvise_sequential+fadvise_nocache],write 9535488~196608 [fadvise_sequential+fadvise_nocache],write 9748480~196608 [fadvise_sequential+fadvise_nocache],write 9961472~65536 [fadvise_sequential+fadvise_nocache],write 10043392~196608 [fadvise_sequential+fadvise_nocache],write 10256384~196608 [fadvise_sequential+fadvise_nocache],write 10469376~65536 [fadvise_sequential+fadvise_nocache],write 10551296~196608 [fadvise_sequential+fadvise_nocache],write 10764288~196608 [fadvise_sequential+fadvise_nocache],write 10977280~65536 [fadvise_sequential+fadvise_nocache],write 11075584~98304 [fadvise_sequential+fadvise_nocache],write 11190272~65536 [fadvise_sequential+fadvise_nocache],write 11288576~98304 [fadvise_sequential+fadvise_nocache],write 11403264~65536 [fadvise_sequential+fadvise_nocache],write 11501568~49152 [fadvise_sequential+fadvise_nocache],write 11583488~98304 [fadvise_sequential+fadvise_nocache],write 11698176~65536 [fadvise_sequential+fadvise_nocache],write 11796480~98304 [fadvise_sequential+fadvise_nocache],write 11911168~65536 [fadvise_sequential+fadvise_nocache],write 12009472~49152 [fadvise_sequential+fadvise_nocache],write 12091392~98304 [fadvise_sequential+fadvise_nocache],write 12304384~98304 [fadvise_sequential+fadvise_nocache],write 12517376~65536 [fadvise_sequential+fadvise_nocache],write 12599296~98304 [fadvise_sequential+fadvise_nocache],write 12812288~98304 [fadvise_sequential+fadvise_nocache],write 13025280~180224 [fadvise_sequential+fadvise_nocache],write 13320192~98304 [fadvise_sequential+fadvise_nocache],write 13533184~180224 [fadvise_sequential+fadvise_nocache],write 13828096~98304 [fadvise_sequential+fadvise_nocache],write 14041088~98304 [fadvise_sequential+fadvise_nocache],write 14729216~81920 [fadvise_sequential+fadvise_nocache],write 14942208~81920 [fadvise_sequential+fadvise_nocache],write 15155200~49152 [fadvise_sequential+fadvise_nocache],write 15237120~81920 [fadvise_sequential+fadvise_nocache],write 15450112~81920 [fadvise_sequential+fadvise_nocache],write 15663104~65536 [fadvise_sequential+fadvise_nocache],write 15745024~81920 [fadvise_sequential+fadvise_nocache],write 15958016~81920 [fadvise_sequential+fadvise_nocache],write 16171008~163840 [fadvise_sequential+fadvise_nocache],write 16465920~81920 [fadvise_sequential+fadvise_nocache],write 16678912~81920 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:21.507 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.422553+0000", 2024-09-21T15:00:21.507 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 15.992121900000001, 2024-09-21T15:00:21.507 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042104691, 2024-09-21T15:00:21.507 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:21.507 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:21.507 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:21.507 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:21.507 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:21.508 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 98562 2024-09-21T15:00:21.508 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.508 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:21.508 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.508 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:21.508 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.422553+0000", 2024-09-21T15:00:21.508 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:21.508 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.508 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.508 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:21.508 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.422553+0000", 2024-09-21T15:00:21.508 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:21.508 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.508 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.508 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:21.508 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.422551+0000", 2024-09-21T15:00:21.509 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-21T15:00:21.509 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.509 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.509 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:21.509 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.429465+0000", 2024-09-21T15:00:21.509 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0069139550000000003 2024-09-21T15:00:21.509 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.509 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.509 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:21.509 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.429847+0000", 2024-09-21T15:00:21.509 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00038256 2024-09-21T15:00:21.509 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.509 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.509 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:21.509 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.429866+0000", 2024-09-21T15:00:21.509 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8051e-05 2024-09-21T15:00:21.510 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.510 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.510 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:21.510 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.429946+0000", 2024-09-21T15:00:21.510 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0761999999999997e-05 2024-09-21T15:00:21.510 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.510 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.510 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:21.510 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.430257+0000", 2024-09-21T15:00:21.510 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00031120200000000001 2024-09-21T15:00:21.510 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.510 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.510 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 4", 2024-09-21T15:00:21.510 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.432045+0000", 2024-09-21T15:00:21.510 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0017872879999999999 2024-09-21T15:00:21.511 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.511 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.511 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:21.511 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.451614+0000", 2024-09-21T15:00:21.511 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019569521999999999 2024-09-21T15:00:21.511 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.511 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.511 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:21.511 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.464472+0000", 2024-09-21T15:00:21.511 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012857435 2024-09-21T15:00:21.511 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.511 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.511 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:21.511 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.464613+0000", 2024-09-21T15:00:21.511 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014159700000000001 2024-09-21T15:00:21.511 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.512 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.512 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:21.512 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.464658+0000", 2024-09-21T15:00:21.512 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4268999999999999e-05 2024-09-21T15:00:21.512 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:21.512 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:21.512 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:21.512 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:21.512 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:21.512 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:21.512 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 5, 6, 7, 4] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T15:00:21.512 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T15:00:21.513 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-21T15:00:21.513 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-21T15:00:21.514 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:00:21.506+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:00:21.514 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T15:00:21.515 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T15:00:21.515 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-21T15:00:21.515 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T15:00:21.515 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T15:00:21.524 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T15:00:21.528 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:00:21.528 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:00:21.528 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:00:21.528 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:00:21.528 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:00:21.529 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:00:21.529 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:00:21.529 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:00:21.614 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:00:21.610+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:00:21.633 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:21.634 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:21.644 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T15:00:21.715 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:00:21.710+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:00:21.748 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:21.748 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:21.757 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T15:00:21.816 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:00:21.812+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:00:21.827 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-21T15:00:21.828 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:21.828 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 135, 2024-09-21T15:00:21.828 INFO:teuthology.orchestra.run.smithi032.stdout: "fsid": "caf51e55-9234-4a26-bcd2-d60811962560", 2024-09-21T15:00:21.828 INFO:teuthology.orchestra.run.smithi032.stdout: "created": "2024-09-21T14:57:50.338822+0000", 2024-09-21T15:00:21.828 INFO:teuthology.orchestra.run.smithi032.stdout: "modified": "2024-09-21T15:00:20.135430+0000", 2024-09-21T15:00:21.828 INFO:teuthology.orchestra.run.smithi032.stdout: "last_up_change": "2024-09-21T14:58:01.177141+0000", 2024-09-21T15:00:21.828 INFO:teuthology.orchestra.run.smithi032.stdout: "last_in_change": "2024-09-21T14:59:27.207073+0000", 2024-09-21T15:00:21.828 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-21T15:00:21.828 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_num": 5806080, 2024-09-21T15:00:21.828 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_set": [ 2024-09-21T15:00:21.828 INFO:teuthology.orchestra.run.smithi032.stdout: "nodeep-scrub", 2024-09-21T15:00:21.828 INFO:teuthology.orchestra.run.smithi032.stdout: "noscrub", 2024-09-21T15:00:21.828 INFO:teuthology.orchestra.run.smithi032.stdout: "pglog_hardlimit", 2024-09-21T15:00:21.828 INFO:teuthology.orchestra.run.smithi032.stdout: "purged_snapdirs", 2024-09-21T15:00:21.828 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_deletes", 2024-09-21T15:00:21.829 INFO:teuthology.orchestra.run.smithi032.stdout: "sortbitwise" 2024-09-21T15:00:21.829 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-21T15:00:21.829 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_version": 5, 2024-09-21T15:00:21.829 INFO:teuthology.orchestra.run.smithi032.stdout: "full_ratio": 0.94999998807907104, 2024-09-21T15:00:21.829 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-21T15:00:21.829 INFO:teuthology.orchestra.run.smithi032.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-21T15:00:21.829 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_snapshot": "", 2024-09-21T15:00:21.829 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_max": 3, 2024-09-21T15:00:21.829 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd": 8, 2024-09-21T15:00:21.829 INFO:teuthology.orchestra.run.smithi032.stdout: "require_min_compat_client": "luminous", 2024-09-21T15:00:21.829 INFO:teuthology.orchestra.run.smithi032.stdout: "min_compat_client": "luminous", 2024-09-21T15:00:21.829 INFO:teuthology.orchestra.run.smithi032.stdout: "require_osd_release": "reef", 2024-09-21T15:00:21.829 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_crimson": false, 2024-09-21T15:00:21.829 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2024-09-21T15:00:21.830 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.830 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 1, 2024-09-21T15:00:21.830 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": ".mgr", 2024-09-21T15:00:21.830 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-09-21T14:58:01.842810+0000", 2024-09-21T15:00:21.830 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-09-21T15:00:21.830 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2024-09-21T15:00:21.830 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-09-21T15:00:21.830 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-09-21T15:00:21.830 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-09-21T15:00:21.830 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-09-21T15:00:21.830 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-09-21T15:00:21.830 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-09-21T15:00:21.830 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T15:00:21.830 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T15:00:21.831 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-09-21T15:00:21.831 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-09-21T15:00:21.831 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 1, 2024-09-21T15:00:21.831 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 1, 2024-09-21T15:00:21.831 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 1, 2024-09-21T15:00:21.831 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 1, 2024-09-21T15:00:21.831 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 1, 2024-09-21T15:00:21.831 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-09-21T15:00:21.831 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-09-21T15:00:21.831 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-09-21T15:00:21.831 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-09-21T15:00:21.831 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-09-21T15:00:21.831 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-09-21T15:00:21.831 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-09-21T15:00:21.831 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.832 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "18", 2024-09-21T15:00:21.832 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-09-21T15:00:21.832 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T15:00:21.832 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T15:00:21.832 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-09-21T15:00:21.832 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-09-21T15:00:21.832 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-09-21T15:00:21.832 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-09-21T15:00:21.832 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-09-21T15:00:21.832 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-09-21T15:00:21.832 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-09-21T15:00:21.832 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-09-21T15:00:21.832 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-09-21T15:00:21.832 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-09-21T15:00:21.832 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-09-21T15:00:21.832 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-09-21T15:00:21.833 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-09-21T15:00:21.833 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-09-21T15:00:21.833 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-09-21T15:00:21.833 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T15:00:21.833 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T15:00:21.833 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T15:00:21.833 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-09-21T15:00:21.833 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-09-21T15:00:21.833 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-09-21T15:00:21.833 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-09-21T15:00:21.833 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-09-21T15:00:21.833 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.833 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-09-21T15:00:21.833 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-09-21T15:00:21.833 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-09-21T15:00:21.834 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-09-21T15:00:21.834 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-09-21T15:00:21.834 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T15:00:21.834 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-09-21T15:00:21.834 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-09-21T15:00:21.834 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-09-21T15:00:21.834 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-09-21T15:00:21.834 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-09-21T15:00:21.834 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-09-21T15:00:21.834 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_max": 32, 2024-09-21T15:00:21.834 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 1 2024-09-21T15:00:21.834 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.834 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-09-21T15:00:21.835 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": {} 2024-09-21T15:00:21.835 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.835 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-09-21T15:00:21.835 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 8, 2024-09-21T15:00:21.835 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 8, 2024-09-21T15:00:21.835 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 0.25, 2024-09-21T15:00:21.835 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 2, 2024-09-21T15:00:21.835 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 2, 2024-09-21T15:00:21.835 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 0.97936630249023438, 2024-09-21T15:00:21.835 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 0.98000001907348633, 2024-09-21T15:00:21.835 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 0.98000001907348633 2024-09-21T15:00:21.835 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:21.835 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.835 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.835 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 2, 2024-09-21T15:00:21.836 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "rbd", 2024-09-21T15:00:21.836 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-09-21T14:58:08.284925+0000", 2024-09-21T15:00:21.836 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 8193, 2024-09-21T15:00:21.836 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T15:00:21.836 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-09-21T15:00:21.836 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-09-21T15:00:21.836 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-09-21T15:00:21.836 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-09-21T15:00:21.836 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-09-21T15:00:21.836 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-09-21T15:00:21.836 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T15:00:21.836 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T15:00:21.836 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-09-21T15:00:21.836 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-09-21T15:00:21.836 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2024-09-21T15:00:21.836 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2024-09-21T15:00:21.837 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2024-09-21T15:00:21.837 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2024-09-21T15:00:21.837 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2024-09-21T15:00:21.837 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-09-21T15:00:21.837 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-09-21T15:00:21.837 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-09-21T15:00:21.837 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-09-21T15:00:21.837 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-09-21T15:00:21.837 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-09-21T15:00:21.837 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-09-21T15:00:21.837 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.837 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "22", 2024-09-21T15:00:21.837 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-09-21T15:00:21.837 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T15:00:21.838 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T15:00:21.838 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-09-21T15:00:21.838 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-09-21T15:00:21.838 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 3, 2024-09-21T15:00:21.838 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 22, 2024-09-21T15:00:21.838 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-09-21T15:00:21.838 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-09-21T15:00:21.838 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-09-21T15:00:21.838 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-09-21T15:00:21.838 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-09-21T15:00:21.838 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-09-21T15:00:21.838 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-09-21T15:00:21.838 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-09-21T15:00:21.838 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-09-21T15:00:21.839 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-09-21T15:00:21.839 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-09-21T15:00:21.839 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T15:00:21.839 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T15:00:21.839 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T15:00:21.839 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-09-21T15:00:21.839 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-09-21T15:00:21.839 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-09-21T15:00:21.839 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-09-21T15:00:21.839 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-09-21T15:00:21.839 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.839 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-09-21T15:00:21.839 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-09-21T15:00:21.839 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-09-21T15:00:21.840 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-09-21T15:00:21.840 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-09-21T15:00:21.840 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T15:00:21.840 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-09-21T15:00:21.840 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-09-21T15:00:21.840 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-09-21T15:00:21.840 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-09-21T15:00:21.840 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-09-21T15:00:21.840 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2024-09-21T15:00:21.840 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-09-21T15:00:21.840 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd": {} 2024-09-21T15:00:21.840 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.840 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-09-21T15:00:21.841 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 4.9800000190734863, 2024-09-21T15:00:21.841 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 4.9800000190734863, 2024-09-21T15:00:21.841 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 0.87999999523162842, 2024-09-21T15:00:21.841 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 4.380000114440918, 2024-09-21T15:00:21.841 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 4.380000114440918, 2024-09-21T15:00:21.841 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 0.97936630249023438, 2024-09-21T15:00:21.841 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 0.98000001907348633, 2024-09-21T15:00:21.841 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 0.98000001907348633 2024-09-21T15:00:21.841 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:21.841 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.841 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.841 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 3, 2024-09-21T15:00:21.841 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "pool_client.0", 2024-09-21T15:00:21.841 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-09-21T14:58:17.510709+0000", 2024-09-21T15:00:21.841 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 8193, 2024-09-21T15:00:21.841 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T15:00:21.842 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-09-21T15:00:21.842 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-09-21T15:00:21.842 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-09-21T15:00:21.842 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-09-21T15:00:21.842 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-09-21T15:00:21.842 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-09-21T15:00:21.842 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T15:00:21.842 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T15:00:21.842 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-09-21T15:00:21.842 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-09-21T15:00:21.842 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 32, 2024-09-21T15:00:21.842 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 32, 2024-09-21T15:00:21.842 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 32, 2024-09-21T15:00:21.843 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 32, 2024-09-21T15:00:21.843 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 32, 2024-09-21T15:00:21.843 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-09-21T15:00:21.843 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-09-21T15:00:21.843 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-09-21T15:00:21.843 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-09-21T15:00:21.843 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-09-21T15:00:21.843 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-09-21T15:00:21.843 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-09-21T15:00:21.843 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.844 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "135", 2024-09-21T15:00:21.844 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-09-21T15:00:21.844 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T15:00:21.844 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T15:00:21.844 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-09-21T15:00:21.844 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-09-21T15:00:21.844 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 84, 2024-09-21T15:00:21.844 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 135, 2024-09-21T15:00:21.844 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-09-21T15:00:21.844 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-09-21T15:00:21.844 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-09-21T15:00:21.844 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-09-21T15:00:21.845 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-09-21T15:00:21.845 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-09-21T15:00:21.845 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-09-21T15:00:21.845 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-09-21T15:00:21.845 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-09-21T15:00:21.845 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-09-21T15:00:21.845 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-09-21T15:00:21.845 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T15:00:21.845 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T15:00:21.845 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T15:00:21.845 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-09-21T15:00:21.845 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-09-21T15:00:21.845 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-09-21T15:00:21.845 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-09-21T15:00:21.846 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-09-21T15:00:21.846 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.846 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-09-21T15:00:21.846 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-09-21T15:00:21.846 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-09-21T15:00:21.846 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-09-21T15:00:21.846 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-09-21T15:00:21.846 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T15:00:21.846 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-09-21T15:00:21.846 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-09-21T15:00:21.846 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-09-21T15:00:21.846 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-09-21T15:00:21.846 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-09-21T15:00:21.846 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2024-09-21T15:00:21.847 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-09-21T15:00:21.847 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd": {} 2024-09-21T15:00:21.847 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.847 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-09-21T15:00:21.847 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 2, 2024-09-21T15:00:21.847 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 2, 2024-09-21T15:00:21.847 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 1, 2024-09-21T15:00:21.847 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 2, 2024-09-21T15:00:21.847 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 2, 2024-09-21T15:00:21.847 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 0.97936630249023438, 2024-09-21T15:00:21.847 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 0.98000001907348633, 2024-09-21T15:00:21.847 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 0.98000001907348633 2024-09-21T15:00:21.847 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:21.847 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:21.847 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-21T15:00:21.847 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2024-09-21T15:00:21.848 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.848 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-09-21T15:00:21.848 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "a4a73316-3a94-42e9-9a51-deb46a66bb06", 2024-09-21T15:00:21.848 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-21T15:00:21.848 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-21T15:00:21.848 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-21T15:00:21.848 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-21T15:00:21.848 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-21T15:00:21.848 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-21T15:00:21.848 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2024-09-21T15:00:21.848 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 93, 2024-09-21T15:00:21.848 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-21T15:00:21.848 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-21T15:00:21.848 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-21T15:00:21.848 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:21.849 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.849 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:21.849 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6808", 2024-09-21T15:00:21.849 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4016672243 2024-09-21T15:00:21.849 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.849 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.849 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:21.849 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6809", 2024-09-21T15:00:21.849 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4016672243 2024-09-21T15:00:21.849 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:21.849 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:21.849 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.849 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-21T15:00:21.849 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:21.849 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.850 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:21.850 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6810", 2024-09-21T15:00:21.850 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4016672243 2024-09-21T15:00:21.850 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.850 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.850 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:21.850 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6811", 2024-09-21T15:00:21.850 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4016672243 2024-09-21T15:00:21.850 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:21.850 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:21.850 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.850 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-21T15:00:21.850 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:21.850 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.850 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:21.851 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6814", 2024-09-21T15:00:21.851 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4016672243 2024-09-21T15:00:21.851 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.851 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.851 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:21.851 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6815", 2024-09-21T15:00:21.851 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4016672243 2024-09-21T15:00:21.851 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:21.851 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:21.851 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.851 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-21T15:00:21.851 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:21.851 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.851 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:21.851 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6812", 2024-09-21T15:00:21.852 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4016672243 2024-09-21T15:00:21.852 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.852 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.852 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:21.852 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6813", 2024-09-21T15:00:21.852 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4016672243 2024-09-21T15:00:21.852 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:21.852 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:21.852 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.852 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6809/4016672243", 2024-09-21T15:00:21.852 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6811/4016672243", 2024-09-21T15:00:21.852 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6815/4016672243", 2024-09-21T15:00:21.852 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6813/4016672243", 2024-09-21T15:00:21.852 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-21T15:00:21.852 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-21T15:00:21.853 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-21T15:00:21.853 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:21.853 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.853 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.853 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-09-21T15:00:21.853 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "006757a0-94d1-4a9b-aef8-73634cc5828e", 2024-09-21T15:00:21.853 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-21T15:00:21.853 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-21T15:00:21.853 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-21T15:00:21.853 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-21T15:00:21.853 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-21T15:00:21.853 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-21T15:00:21.853 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2024-09-21T15:00:21.853 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 93, 2024-09-21T15:00:21.853 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-21T15:00:21.854 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-21T15:00:21.854 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-21T15:00:21.854 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:21.854 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.854 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:21.854 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6824", 2024-09-21T15:00:21.854 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 641090577 2024-09-21T15:00:21.854 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.854 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.854 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:21.854 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6825", 2024-09-21T15:00:21.854 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 641090577 2024-09-21T15:00:21.854 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:21.854 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:21.855 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.855 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-21T15:00:21.855 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:21.855 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.855 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:21.855 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6826", 2024-09-21T15:00:21.855 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 641090577 2024-09-21T15:00:21.855 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.855 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.855 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:21.855 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6827", 2024-09-21T15:00:21.855 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 641090577 2024-09-21T15:00:21.855 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:21.855 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:21.856 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.856 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-21T15:00:21.856 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:21.856 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.856 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:21.856 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6830", 2024-09-21T15:00:21.856 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 641090577 2024-09-21T15:00:21.856 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.856 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.856 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:21.856 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6831", 2024-09-21T15:00:21.856 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 641090577 2024-09-21T15:00:21.856 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:21.856 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:21.856 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.857 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-21T15:00:21.857 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:21.857 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.857 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:21.857 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6828", 2024-09-21T15:00:21.857 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 641090577 2024-09-21T15:00:21.857 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.857 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.857 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:21.857 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6829", 2024-09-21T15:00:21.857 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 641090577 2024-09-21T15:00:21.857 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:21.857 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:21.857 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.858 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6825/641090577", 2024-09-21T15:00:21.858 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6827/641090577", 2024-09-21T15:00:21.858 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6831/641090577", 2024-09-21T15:00:21.858 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6829/641090577", 2024-09-21T15:00:21.858 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-21T15:00:21.858 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-21T15:00:21.858 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-21T15:00:21.858 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:21.858 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.858 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.858 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-09-21T15:00:21.858 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "0b499a23-820e-4e6c-a558-adbf88b7014f", 2024-09-21T15:00:21.858 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-21T15:00:21.859 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-21T15:00:21.859 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 0.3514556884765625, 2024-09-21T15:00:21.859 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-21T15:00:21.859 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-21T15:00:21.859 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-21T15:00:21.859 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2024-09-21T15:00:21.859 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 51, 2024-09-21T15:00:21.859 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-21T15:00:21.859 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-21T15:00:21.859 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-21T15:00:21.859 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:21.859 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.859 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:21.859 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6800", 2024-09-21T15:00:21.860 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2609154414 2024-09-21T15:00:21.860 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.860 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.860 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:21.860 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6801", 2024-09-21T15:00:21.860 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2609154414 2024-09-21T15:00:21.860 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:21.860 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:21.860 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.860 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-21T15:00:21.860 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:21.860 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.860 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:21.861 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6802", 2024-09-21T15:00:21.861 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2609154414 2024-09-21T15:00:21.861 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.861 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.861 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:21.861 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6803", 2024-09-21T15:00:21.861 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2609154414 2024-09-21T15:00:21.861 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:21.861 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:21.861 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.861 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-21T15:00:21.861 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:21.861 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.862 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:21.862 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6806", 2024-09-21T15:00:21.862 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2609154414 2024-09-21T15:00:21.862 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.862 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.862 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:21.862 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6807", 2024-09-21T15:00:21.862 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2609154414 2024-09-21T15:00:21.862 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:21.862 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:21.862 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.862 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-21T15:00:21.862 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:21.863 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.863 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:21.863 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6804", 2024-09-21T15:00:21.863 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2609154414 2024-09-21T15:00:21.863 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.863 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.863 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:21.863 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6805", 2024-09-21T15:00:21.863 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2609154414 2024-09-21T15:00:21.863 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:21.863 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:21.863 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.863 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6801/2609154414", 2024-09-21T15:00:21.864 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6803/2609154414", 2024-09-21T15:00:21.864 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6807/2609154414", 2024-09-21T15:00:21.864 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6805/2609154414", 2024-09-21T15:00:21.864 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-21T15:00:21.864 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-21T15:00:21.864 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-21T15:00:21.864 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:21.864 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.864 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.864 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-09-21T15:00:21.864 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "5a57b3dd-4a07-45b7-b159-3cac4c7eb262", 2024-09-21T15:00:21.864 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-21T15:00:21.864 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-21T15:00:21.865 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-21T15:00:21.865 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-21T15:00:21.865 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-21T15:00:21.865 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-21T15:00:21.865 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2024-09-21T15:00:21.865 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 94, 2024-09-21T15:00:21.865 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-21T15:00:21.865 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-21T15:00:21.865 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-21T15:00:21.865 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:21.865 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.865 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:21.865 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6816", 2024-09-21T15:00:21.866 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1462563376 2024-09-21T15:00:21.866 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.866 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.866 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:21.866 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6817", 2024-09-21T15:00:21.866 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1462563376 2024-09-21T15:00:21.866 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:21.866 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:21.866 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.866 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-21T15:00:21.866 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:21.866 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.866 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:21.867 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6818", 2024-09-21T15:00:21.867 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1462563376 2024-09-21T15:00:21.867 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.867 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.867 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:21.867 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6819", 2024-09-21T15:00:21.867 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1462563376 2024-09-21T15:00:21.867 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:21.867 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:21.867 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.867 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-21T15:00:21.867 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:21.867 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.867 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:21.867 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6822", 2024-09-21T15:00:21.868 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1462563376 2024-09-21T15:00:21.868 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.868 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.868 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:21.868 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6823", 2024-09-21T15:00:21.868 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1462563376 2024-09-21T15:00:21.868 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:21.868 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:21.868 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.868 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-21T15:00:21.868 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:21.868 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.868 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:21.868 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6820", 2024-09-21T15:00:21.868 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1462563376 2024-09-21T15:00:21.869 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.869 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.869 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:21.869 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6821", 2024-09-21T15:00:21.869 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1462563376 2024-09-21T15:00:21.869 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:21.869 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:21.869 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.869 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6817/1462563376", 2024-09-21T15:00:21.869 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6819/1462563376", 2024-09-21T15:00:21.869 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6823/1462563376", 2024-09-21T15:00:21.869 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6821/1462563376", 2024-09-21T15:00:21.869 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-21T15:00:21.869 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-21T15:00:21.869 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-21T15:00:21.870 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:21.870 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.870 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.870 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-09-21T15:00:21.870 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "39a00807-7517-474e-8842-623dd7093cc5", 2024-09-21T15:00:21.870 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-21T15:00:21.870 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-21T15:00:21.870 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-21T15:00:21.870 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-21T15:00:21.870 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-21T15:00:21.870 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-21T15:00:21.870 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2024-09-21T15:00:21.870 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 94, 2024-09-21T15:00:21.870 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-21T15:00:21.871 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-21T15:00:21.871 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-21T15:00:21.871 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:21.871 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.871 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:21.871 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6801", 2024-09-21T15:00:21.871 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 144412042 2024-09-21T15:00:21.871 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.871 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.871 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:21.871 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6803", 2024-09-21T15:00:21.871 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 144412042 2024-09-21T15:00:21.871 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:21.871 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:21.872 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.872 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-21T15:00:21.872 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:21.872 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.872 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:21.872 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6805", 2024-09-21T15:00:21.872 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 144412042 2024-09-21T15:00:21.872 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.872 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.872 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:21.872 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6807", 2024-09-21T15:00:21.872 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 144412042 2024-09-21T15:00:21.872 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:21.872 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:21.872 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.872 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-21T15:00:21.873 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:21.873 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.873 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:21.873 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6812", 2024-09-21T15:00:21.873 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 144412042 2024-09-21T15:00:21.873 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.873 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.873 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:21.873 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6814", 2024-09-21T15:00:21.873 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 144412042 2024-09-21T15:00:21.873 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:21.873 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:21.873 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.873 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-21T15:00:21.873 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:21.874 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.874 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:21.874 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6809", 2024-09-21T15:00:21.874 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 144412042 2024-09-21T15:00:21.874 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.874 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.874 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:21.874 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6811", 2024-09-21T15:00:21.874 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 144412042 2024-09-21T15:00:21.874 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:21.874 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:21.874 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.874 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.46:6803/144412042", 2024-09-21T15:00:21.874 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.46:6807/144412042", 2024-09-21T15:00:21.875 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.46:6814/144412042", 2024-09-21T15:00:21.875 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.46:6811/144412042", 2024-09-21T15:00:21.875 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-21T15:00:21.875 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-21T15:00:21.875 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-21T15:00:21.875 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:21.875 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.875 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.875 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-09-21T15:00:21.875 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "edf9f045-ddeb-4504-a0fc-1a0e4e853550", 2024-09-21T15:00:21.875 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-21T15:00:21.875 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-21T15:00:21.875 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-21T15:00:21.875 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-21T15:00:21.876 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-21T15:00:21.876 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-21T15:00:21.876 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2024-09-21T15:00:21.876 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 75, 2024-09-21T15:00:21.876 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-21T15:00:21.876 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-21T15:00:21.876 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-21T15:00:21.876 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:21.876 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.876 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:21.876 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6800", 2024-09-21T15:00:21.876 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 849101442 2024-09-21T15:00:21.876 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.876 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.876 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:21.876 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6802", 2024-09-21T15:00:21.877 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 849101442 2024-09-21T15:00:21.877 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:21.877 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:21.877 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.877 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-21T15:00:21.877 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:21.877 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.877 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:21.877 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6804", 2024-09-21T15:00:21.877 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 849101442 2024-09-21T15:00:21.877 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.877 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.877 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:21.877 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6806", 2024-09-21T15:00:21.877 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 849101442 2024-09-21T15:00:21.878 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:21.878 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:21.878 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.878 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-21T15:00:21.878 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:21.878 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.878 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:21.878 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6813", 2024-09-21T15:00:21.878 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 849101442 2024-09-21T15:00:21.878 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.878 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.878 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:21.878 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6815", 2024-09-21T15:00:21.878 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 849101442 2024-09-21T15:00:21.878 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:21.879 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:21.879 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.879 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-21T15:00:21.879 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:21.879 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.879 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:21.879 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6808", 2024-09-21T15:00:21.879 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 849101442 2024-09-21T15:00:21.879 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.879 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.879 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:21.879 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6810", 2024-09-21T15:00:21.879 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 849101442 2024-09-21T15:00:21.879 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:21.880 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:21.880 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.880 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.46:6802/849101442", 2024-09-21T15:00:21.880 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.46:6806/849101442", 2024-09-21T15:00:21.880 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.46:6815/849101442", 2024-09-21T15:00:21.880 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.46:6810/849101442", 2024-09-21T15:00:21.880 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-21T15:00:21.880 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-21T15:00:21.880 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-21T15:00:21.880 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:21.880 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.880 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.880 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-09-21T15:00:21.880 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "c820ea6b-88fe-4421-83ec-17c466691fea", 2024-09-21T15:00:21.880 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-21T15:00:21.881 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-21T15:00:21.881 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-21T15:00:21.881 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 0.834930419921875, 2024-09-21T15:00:21.881 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-21T15:00:21.881 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-21T15:00:21.881 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2024-09-21T15:00:21.881 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 123, 2024-09-21T15:00:21.881 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-21T15:00:21.881 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-21T15:00:21.881 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-21T15:00:21.881 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:21.881 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.881 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:21.881 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6817", 2024-09-21T15:00:21.882 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 847189172 2024-09-21T15:00:21.882 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.882 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.882 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:21.882 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6818", 2024-09-21T15:00:21.882 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 847189172 2024-09-21T15:00:21.882 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:21.882 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:21.882 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.882 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-21T15:00:21.882 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:21.882 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.882 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:21.882 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6820", 2024-09-21T15:00:21.882 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 847189172 2024-09-21T15:00:21.883 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.883 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.883 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:21.883 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6822", 2024-09-21T15:00:21.883 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 847189172 2024-09-21T15:00:21.883 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:21.883 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:21.883 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.883 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-21T15:00:21.883 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:21.883 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.883 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:21.883 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6827", 2024-09-21T15:00:21.883 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 847189172 2024-09-21T15:00:21.884 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.884 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.884 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:21.884 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6828", 2024-09-21T15:00:21.884 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 847189172 2024-09-21T15:00:21.884 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:21.884 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:21.884 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.884 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-21T15:00:21.884 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:21.884 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.884 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:21.884 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6823", 2024-09-21T15:00:21.884 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 847189172 2024-09-21T15:00:21.884 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.885 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.885 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:21.885 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6825", 2024-09-21T15:00:21.885 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 847189172 2024-09-21T15:00:21.885 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:21.885 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:21.885 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.885 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.46:6818/847189172", 2024-09-21T15:00:21.885 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.46:6822/847189172", 2024-09-21T15:00:21.885 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.46:6828/847189172", 2024-09-21T15:00:21.885 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.46:6825/847189172", 2024-09-21T15:00:21.885 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-21T15:00:21.885 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-21T15:00:21.885 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-21T15:00:21.885 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:21.886 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.886 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.886 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-09-21T15:00:21.886 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "3a2f82a4-28a0-4fcb-9d6b-2285fa4d07ac", 2024-09-21T15:00:21.886 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-21T15:00:21.886 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-21T15:00:21.886 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 0.95001220703125, 2024-09-21T15:00:21.886 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-21T15:00:21.886 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-21T15:00:21.886 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-21T15:00:21.886 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 15, 2024-09-21T15:00:21.886 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 81, 2024-09-21T15:00:21.886 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-21T15:00:21.886 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-21T15:00:21.887 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-21T15:00:21.887 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:21.887 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.887 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:21.887 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6816", 2024-09-21T15:00:21.887 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3894963553 2024-09-21T15:00:21.887 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.887 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.887 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:21.887 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6819", 2024-09-21T15:00:21.887 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3894963553 2024-09-21T15:00:21.887 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:21.887 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:21.887 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.887 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-21T15:00:21.888 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:21.888 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.888 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:21.888 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6821", 2024-09-21T15:00:21.888 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3894963553 2024-09-21T15:00:21.888 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.888 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.888 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:21.888 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6824", 2024-09-21T15:00:21.888 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3894963553 2024-09-21T15:00:21.888 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:21.888 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:21.888 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.888 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-21T15:00:21.888 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:21.888 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.889 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:21.889 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6830", 2024-09-21T15:00:21.889 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3894963553 2024-09-21T15:00:21.889 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.889 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.889 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:21.889 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6831", 2024-09-21T15:00:21.889 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3894963553 2024-09-21T15:00:21.889 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:21.889 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:21.889 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.889 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-21T15:00:21.889 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:21.889 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.889 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:21.890 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6826", 2024-09-21T15:00:21.890 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3894963553 2024-09-21T15:00:21.890 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.890 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.890 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:21.890 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6829", 2024-09-21T15:00:21.890 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3894963553 2024-09-21T15:00:21.890 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:21.890 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:21.890 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.890 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.46:6819/3894963553", 2024-09-21T15:00:21.890 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.46:6824/3894963553", 2024-09-21T15:00:21.890 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.46:6831/3894963553", 2024-09-21T15:00:21.890 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.46:6829/3894963553", 2024-09-21T15:00:21.891 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-21T15:00:21.891 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-21T15:00:21.891 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-21T15:00:21.891 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:21.891 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:21.891 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-21T15:00:21.891 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_xinfo": [ 2024-09-21T15:00:21.891 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.891 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-09-21T15:00:21.891 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-21T15:00:21.891 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-21T15:00:21.891 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-21T15:00:21.891 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-21T15:00:21.891 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-21T15:00:21.892 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-21T14:57:57.458317+0000", 2024-09-21T15:00:21.892 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-21T15:00:21.892 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.892 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.892 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-09-21T15:00:21.892 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-21T15:00:21.892 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-21T15:00:21.892 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-21T15:00:21.892 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-21T15:00:21.892 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-21T15:00:21.892 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-21T14:57:57.907624+0000", 2024-09-21T15:00:21.892 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-21T15:00:21.892 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.892 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.893 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-09-21T15:00:21.893 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-21T15:00:21.893 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-21T15:00:21.893 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-21T15:00:21.893 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-21T15:00:21.893 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-21T15:00:21.893 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-21T14:57:57.278817+0000", 2024-09-21T15:00:21.893 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-21T15:00:21.893 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.893 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.893 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-09-21T15:00:21.893 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-21T15:00:21.893 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-21T15:00:21.893 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-21T15:00:21.894 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-21T15:00:21.894 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-21T15:00:21.894 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-21T14:57:57.500449+0000", 2024-09-21T15:00:21.894 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-21T15:00:21.894 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.894 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.894 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-09-21T15:00:21.894 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-21T15:00:21.894 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-21T15:00:21.894 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-21T15:00:21.894 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-21T15:00:21.894 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-21T15:00:21.894 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-21T14:57:57.504325+0000", 2024-09-21T15:00:21.895 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-21T15:00:21.895 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.895 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.895 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-09-21T15:00:21.895 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-21T15:00:21.895 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-21T15:00:21.895 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-21T15:00:21.895 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-21T15:00:21.895 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-21T15:00:21.895 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-21T14:57:57.777525+0000", 2024-09-21T15:00:21.895 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-21T15:00:21.895 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.895 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.895 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-09-21T15:00:21.895 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-21T15:00:21.896 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-21T15:00:21.896 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-21T15:00:21.896 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-21T15:00:21.896 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-21T15:00:21.896 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-21T14:57:57.698961+0000", 2024-09-21T15:00:21.896 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-21T15:00:21.896 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.896 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.896 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-09-21T15:00:21.896 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-21T15:00:21.896 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-21T15:00:21.896 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-21T15:00:21.896 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-21T15:00:21.897 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-21T15:00:21.897 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-21T14:57:57.932625+0000", 2024-09-21T15:00:21.897 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-21T15:00:21.897 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:21.897 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-21T15:00:21.897 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap": [ 2024-09-21T15:00:21.897 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.897 INFO:teuthology.orchestra.run.smithi032.stdout: "pgid": "3.2", 2024-09-21T15:00:21.897 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2024-09-21T15:00:21.897 INFO:teuthology.orchestra.run.smithi032.stdout: 2, 2024-09-21T15:00:21.897 INFO:teuthology.orchestra.run.smithi032.stdout: 3 2024-09-21T15:00:21.897 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:21.897 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:21.898 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-21T15:00:21.898 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_items": [], 2024-09-21T15:00:21.898 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_primaries": [], 2024-09-21T15:00:21.898 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_temp": [], 2024-09-21T15:00:21.898 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_temp": [], 2024-09-21T15:00:21.898 INFO:teuthology.orchestra.run.smithi032.stdout: "blocklist": {}, 2024-09-21T15:00:21.898 INFO:teuthology.orchestra.run.smithi032.stdout: "range_blocklist": {}, 2024-09-21T15:00:21.898 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profiles": { 2024-09-21T15:00:21.898 INFO:teuthology.orchestra.run.smithi032.stdout: "default": { 2024-09-21T15:00:21.898 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2024-09-21T15:00:21.898 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2024-09-21T15:00:21.898 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "1", 2024-09-21T15:00:21.898 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2024-09-21T15:00:21.899 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van" 2024-09-21T15:00:21.899 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:21.899 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:21.899 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps_queue": [ 2024-09-21T15:00:21.899 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.899 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 3, 2024-09-21T15:00:21.899 INFO:teuthology.orchestra.run.smithi032.stdout: "snaps": [ 2024-09-21T15:00:21.899 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.899 INFO:teuthology.orchestra.run.smithi032.stdout: "begin": 83, 2024-09-21T15:00:21.899 INFO:teuthology.orchestra.run.smithi032.stdout: "length": 1 2024-09-21T15:00:21.899 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:21.899 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:21.899 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:21.899 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-21T15:00:21.900 INFO:teuthology.orchestra.run.smithi032.stdout: "new_removed_snaps": [ 2024-09-21T15:00:21.900 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.900 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 3, 2024-09-21T15:00:21.900 INFO:teuthology.orchestra.run.smithi032.stdout: "snaps": [ 2024-09-21T15:00:21.900 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.900 INFO:teuthology.orchestra.run.smithi032.stdout: "begin": 83, 2024-09-21T15:00:21.900 INFO:teuthology.orchestra.run.smithi032.stdout: "length": 1 2024-09-21T15:00:21.900 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:21.900 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:21.900 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:21.900 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-21T15:00:21.900 INFO:teuthology.orchestra.run.smithi032.stdout: "new_purged_snaps": [ 2024-09-21T15:00:21.900 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.900 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 3, 2024-09-21T15:00:21.901 INFO:teuthology.orchestra.run.smithi032.stdout: "snaps": [ 2024-09-21T15:00:21.901 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:21.901 INFO:teuthology.orchestra.run.smithi032.stdout: "begin": 80, 2024-09-21T15:00:21.901 INFO:teuthology.orchestra.run.smithi032.stdout: "length": 1 2024-09-21T15:00:21.901 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:21.901 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:21.901 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:21.901 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-21T15:00:21.901 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_node_flags": {}, 2024-09-21T15:00:21.901 INFO:teuthology.orchestra.run.smithi032.stdout: "device_class_flags": {}, 2024-09-21T15:00:21.901 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode": { 2024-09-21T15:00:21.901 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_enabled": false, 2024-09-21T15:00:21.902 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_bucket_count": 0, 2024-09-21T15:00:21.902 INFO:teuthology.orchestra.run.smithi032.stdout: "degraded_stretch_mode": 0, 2024-09-21T15:00:21.902 INFO:teuthology.orchestra.run.smithi032.stdout: "recovering_stretch_mode": 0, 2024-09-21T15:00:21.902 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_bucket": 0 2024-09-21T15:00:21.902 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:21.902 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:21.903 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:21.903 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:21.904 INFO:tasks.thrashosds.thrasher:j is {'epoch': 135, 'fsid': 'caf51e55-9234-4a26-bcd2-d60811962560', 'created': '2024-09-21T14:57:50.338822+0000', 'modified': '2024-09-21T15:00:20.135430+0000', 'last_up_change': '2024-09-21T14:58:01.177141+0000', 'last_in_change': '2024-09-21T14:59:27.207073+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T14:58:01.842810+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.9793663024902344, 'average_primary_affinity': 0.9800000190734863, 'average_primary_affinity_weighted': 0.9800000190734863}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-21T14:58:08.284925+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 22, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 0.9793663024902344, 'average_primary_affinity': 0.9800000190734863, 'average_primary_affinity_weighted': 0.9800000190734863}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-21T14:58:17.510709+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '135', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 84, 'snap_epoch': 135, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.9793663024902344, 'average_primary_affinity': 0.9800000190734863, 'average_primary_affinity_weighted': 0.9800000190734863}}], 'osds': [{'osd': 0, 'uuid': 'a4a73316-3a94-42e9-9a51-deb46a66bb06', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 93, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6808', 'nonce': 4016672243}, {'type': 'v1', 'addr': '172.21.15.32:6809', 'nonce': 4016672243}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6810', 'nonce': 4016672243}, {'type': 'v1', 'addr': '172.21.15.32:6811', 'nonce': 4016672243}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6814', 'nonce': 4016672243}, {'type': 'v1', 'addr': '172.21.15.32:6815', 'nonce': 4016672243}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6812', 'nonce': 4016672243}, {'type': 'v1', 'addr': '172.21.15.32:6813', 'nonce': 4016672243}]}, 'public_addr': '172.21.15.32:6809/4016672243', 'cluster_addr': '172.21.15.32:6811/4016672243', 'heartbeat_back_addr': '172.21.15.32:6815/4016672243', 'heartbeat_front_addr': '172.21.15.32:6813/4016672243', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '006757a0-94d1-4a9b-aef8-73634cc5828e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 93, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6824', 'nonce': 641090577}, {'type': 'v1', 'addr': '172.21.15.32:6825', 'nonce': 641090577}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6826', 'nonce': 641090577}, {'type': 'v1', 'addr': '172.21.15.32:6827', 'nonce': 641090577}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6830', 'nonce': 641090577}, {'type': 'v1', 'addr': '172.21.15.32:6831', 'nonce': 641090577}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6828', 'nonce': 641090577}, {'type': 'v1', 'addr': '172.21.15.32:6829', 'nonce': 641090577}]}, 'public_addr': '172.21.15.32:6825/641090577', 'cluster_addr': '172.21.15.32:6827/641090577', 'heartbeat_back_addr': '172.21.15.32:6831/641090577', 'heartbeat_front_addr': '172.21.15.32:6829/641090577', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '0b499a23-820e-4e6c-a558-adbf88b7014f', 'up': 1, 'in': 1, 'weight': 0.3514556884765625, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 51, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6800', 'nonce': 2609154414}, {'type': 'v1', 'addr': '172.21.15.32:6801', 'nonce': 2609154414}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6802', 'nonce': 2609154414}, {'type': 'v1', 'addr': '172.21.15.32:6803', 'nonce': 2609154414}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6806', 'nonce': 2609154414}, {'type': 'v1', 'addr': '172.21.15.32:6807', 'nonce': 2609154414}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6804', 'nonce': 2609154414}, {'type': 'v1', 'addr': '172.21.15.32:6805', 'nonce': 2609154414}]}, 'public_addr': '172.21.15.32:6801/2609154414', 'cluster_addr': '172.21.15.32:6803/2609154414', 'heartbeat_back_addr': '172.21.15.32:6807/2609154414', 'heartbeat_front_addr': '172.21.15.32:6805/2609154414', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '5a57b3dd-4a07-45b7-b159-3cac4c7eb262', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 94, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6816', 'nonce': 1462563376}, {'type': 'v1', 'addr': '172.21.15.32:6817', 'nonce': 1462563376}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6818', 'nonce': 1462563376}, {'type': 'v1', 'addr': '172.21.15.32:6819', 'nonce': 1462563376}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6822', 'nonce': 1462563376}, {'type': 'v1', 'addr': '172.21.15.32:6823', 'nonce': 1462563376}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6820', 'nonce': 1462563376}, {'type': 'v1', 'addr': '172.21.15.32:6821', 'nonce': 1462563376}]}, 'public_addr': '172.21.15.32:6817/1462563376', 'cluster_addr': '172.21.15.32:6819/1462563376', 'heartbeat_back_addr': '172.21.15.32:6823/1462563376', 'heartbeat_front_addr': '172.21.15.32:6821/1462563376', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '39a00807-7517-474e-8842-623dd7093cc5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 94, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6801', 'nonce': 144412042}, {'type': 'v1', 'addr': '172.21.15.46:6803', 'nonce': 144412042}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6805', 'nonce': 144412042}, {'type': 'v1', 'addr': '172.21.15.46:6807', 'nonce': 144412042}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6812', 'nonce': 144412042}, {'type': 'v1', 'addr': '172.21.15.46:6814', 'nonce': 144412042}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6809', 'nonce': 144412042}, {'type': 'v1', 'addr': '172.21.15.46:6811', 'nonce': 144412042}]}, 'public_addr': '172.21.15.46:6803/144412042', 'cluster_addr': '172.21.15.46:6807/144412042', 'heartbeat_back_addr': '172.21.15.46:6814/144412042', 'heartbeat_front_addr': '172.21.15.46:6811/144412042', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'edf9f045-ddeb-4504-a0fc-1a0e4e853550', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 75, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6800', 'nonce': 849101442}, {'type': 'v1', 'addr': '172.21.15.46:6802', 'nonce': 849101442}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6804', 'nonce': 849101442}, {'type': 'v1', 'addr': '172.21.15.46:6806', 'nonce': 849101442}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6813', 'nonce': 849101442}, {'type': 'v1', 'addr': '172.21.15.46:6815', 'nonce': 849101442}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6808', 'nonce': 849101442}, {'type': 'v1', 'addr': '172.21.15.46:6810', 'nonce': 849101442}]}, 'public_addr': '172.21.15.46:6802/849101442', 'cluster_addr': '172.21.15.46:6806/849101442', 'heartbeat_back_addr': '172.21.15.46:6815/849101442', 'heartbeat_front_addr': '172.21.15.46:6810/849101442', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'c820ea6b-88fe-4421-83ec-17c466691fea', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.834930419921875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 123, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6817', 'nonce': 847189172}, {'type': 'v1', 'addr': '172.21.15.46:6818', 'nonce': 847189172}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6820', 'nonce': 847189172}, {'type': 'v1', 'addr': '172.21.15.46:6822', 'nonce': 847189172}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6827', 'nonce': 847189172}, {'type': 'v1', 'addr': '172.21.15.46:6828', 'nonce': 847189172}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6823', 'nonce': 847189172}, {'type': 'v1', 'addr': '172.21.15.46:6825', 'nonce': 847189172}]}, 'public_addr': '172.21.15.46:6818/847189172', 'cluster_addr': '172.21.15.46:6822/847189172', 'heartbeat_back_addr': '172.21.15.46:6828/847189172', 'heartbeat_front_addr': '172.21.15.46:6825/847189172', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '3a2f82a4-28a0-4fcb-9d6b-2285fa4d07ac', 'up': 1, 'in': 1, 'weight': 0.95001220703125, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 81, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6816', 'nonce': 3894963553}, {'type': 'v1', 'addr': '172.21.15.46:6819', 'nonce': 3894963553}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6821', 'nonce': 3894963553}, {'type': 'v1', 'addr': '172.21.15.46:6824', 'nonce': 3894963553}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6830', 'nonce': 3894963553}, {'type': 'v1', 'addr': '172.21.15.46:6831', 'nonce': 3894963553}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6826', 'nonce': 3894963553}, {'type': 'v1', 'addr': '172.21.15.46:6829', 'nonce': 3894963553}]}, 'public_addr': '172.21.15.46:6819/3894963553', 'cluster_addr': '172.21.15.46:6824/3894963553', 'heartbeat_back_addr': '172.21.15.46:6831/3894963553', 'heartbeat_front_addr': '172.21.15.46:6829/3894963553', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:57:57.458317+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:57:57.907624+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:57:57.278817+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:57:57.500449+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:57:57.504325+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:57:57.777525+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:57:57.698961+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:57:57.932625+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.2', 'osds': [2, 3]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 83, 'length': 1}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 83, 'length': 1}]}], 'new_purged_snaps': [{'pool': 3, 'snaps': [{'begin': 80, 'length': 1}]}], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-21T15:00:21.904 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T15:00:21.905 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T15:00:21.917 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:00:21.916+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:00:22.016 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:22.017 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:22.017 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:00:22.010+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:00:22.027 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T15:00:22.118 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:00:22.116+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:00:22.136 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:22.136 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:22.146 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T15:00:22.163 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T15:00:22.216 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-21T15:00:22.218 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-09-21T15:00:22.219 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:00:22.214+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:00:22.232 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":199,"stamp":"2024-09-21T15:00:21.811519+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":779182769,"num_objects":1026,"num_object_clones":0,"num_object_copies":2052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1026,"num_whiteouts":0,"num_read":109717,"num_read_kb":934360,"num_write":19484,"num_write_kb":701082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1206,"num_bytes_recovered":80408212,"num_keys_recovered":119,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12821,"ondisk_log_size":12821,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":379376,"kb_used_data":64020,"kb_used_omap":12,"kb_used_meta":315251,"kb_avail":754595344,"statfs":{"total":773094113280,"available":772705632256,"internally_reserved":0,"allocated":65556480,"data_stored":485094524,"data_compressed":10330938,"data_compressed_allocated":42442752,"data_compressed_original":473170405,"omap_allocated":12744,"internal_metadata":322817592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-130215104,"num_objects":-40,"num_object_clones":0,"num_object_copies":-80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-40,"num_whiteouts":0,"num_read":3804,"num_read_kb":17411,"num_write":1095,"num_write_kb":8504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.601301"},"pg_stats":[{"pgid":"3.1f","version":"135'352","reported_seq":3067,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-21T15:00:20.664548+0000","last_change":"2024-09-21T15:00:20.144173+0000","last_active":"2024-09-21T15:00:20.664548+0000","last_peered":"2024-09-21T15:00:20.664548+0000","last_clean":"2024-09-21T15:00:20.664548+0000","last_became_active":"2024-09-21T14:58:19.410055+0000","last_became_peered":"2024-09-21T14:58:19.410055+0000","last_unstale":"2024-09-21T15:00:20.664548+0000","last_undegraded":"2024-09-21T15:00:20.664548+0000","last_fullsized":"2024-09-21T15:00:20.664548+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":352,"log_dups_size":0,"ondisk_log_size":352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:49:13.944219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001324857,"stat_sum":{"num_bytes":31188599,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":2995,"num_read_kb":40496,"num_write":575,"num_write_kb":31882,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"53","length":"1"}]},{"pgid":"3.1e","version":"135'252","reported_seq":2182,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-21T15:00:20.665686+0000","last_change":"2024-09-21T15:00:20.143921+0000","last_active":"2024-09-21T15:00:20.665686+0000","last_peered":"2024-09-21T15:00:20.665686+0000","last_clean":"2024-09-21T15:00:20.665686+0000","last_became_active":"2024-09-21T14:59:13.130091+0000","last_became_peered":"2024-09-21T14:59:13.130091+0000","last_unstale":"2024-09-21T15:00:20.665686+0000","last_undegraded":"2024-09-21T15:00:20.665686+0000","last_fullsized":"2024-09-21T15:00:20.665686+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":252,"log_dups_size":0,"ondisk_log_size":252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:46:00.199136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001780083,"stat_sum":{"num_bytes":17859466,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":2107,"num_read_kb":19682,"num_write":342,"num_write_kb":13032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":3844512,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"53","length":"1"}]},{"pgid":"3.1d","version":"135'170","reported_seq":3079,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-21T15:00:20.150013+0000","last_change":"2024-09-21T15:00:20.150013+0000","last_active":"2024-09-21T15:00:20.150013+0000","last_peered":"2024-09-21T15:00:20.150013+0000","last_clean":"2024-09-21T15:00:20.150013+0000","last_became_active":"2024-09-21T14:58:19.411387+0000","last_became_peered":"2024-09-21T14:58:19.411387+0000","last_unstale":"2024-09-21T15:00:20.150013+0000","last_undegraded":"2024-09-21T15:00:20.150013+0000","last_fullsized":"2024-09-21T15:00:20.150013+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:37:57.597069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017624380000000001,"stat_sum":{"num_bytes":13092021,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":2660,"num_read_kb":19916,"num_write":283,"num_write_kb":19563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"53","length":"1"}]},{"pgid":"3.1c","version":"135'355","reported_seq":2449,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-21T15:00:20.665935+0000","last_change":"2024-09-21T15:00:20.153736+0000","last_active":"2024-09-21T15:00:20.665935+0000","last_peered":"2024-09-21T15:00:20.665935+0000","last_clean":"2024-09-21T15:00:20.665935+0000","last_became_active":"2024-09-21T14:58:19.411306+0000","last_became_peered":"2024-09-21T14:58:19.411306+0000","last_unstale":"2024-09-21T15:00:20.665935+0000","last_undegraded":"2024-09-21T15:00:20.665935+0000","last_fullsized":"2024-09-21T15:00:20.665935+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":355,"log_dups_size":0,"ondisk_log_size":355,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:20:27.079795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041969000000000002,"stat_sum":{"num_bytes":12624495,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":2530,"num_read_kb":15096,"num_write":523,"num_write_kb":18327,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"53","length":"1"}]},{"pgid":"3.1b","version":"135'339","reported_seq":2045,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-21T15:00:20.144009+0000","last_change":"2024-09-21T15:00:20.144009+0000","last_active":"2024-09-21T15:00:20.144009+0000","last_peered":"2024-09-21T15:00:20.144009+0000","last_clean":"2024-09-21T15:00:20.144009+0000","last_became_active":"2024-09-21T14:59:31.131051+0000","last_became_peered":"2024-09-21T14:59:31.131051+0000","last_unstale":"2024-09-21T15:00:20.144009+0000","last_undegraded":"2024-09-21T15:00:20.144009+0000","last_fullsized":"2024-09-21T15:00:20.144009+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":339,"log_dups_size":0,"ondisk_log_size":339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:21:37.320774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013927379999999999,"stat_sum":{"num_bytes":17687348,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":2029,"num_read_kb":23602,"num_write":423,"num_write_kb":13661,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":117,"num_bytes_recovered":7757116,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"53","length":"1"}]},{"pgid":"3.1a","version":"135'279","reported_seq":2653,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-21T15:00:20.864822+0000","last_change":"2024-09-21T15:00:20.153770+0000","last_active":"2024-09-21T15:00:20.864822+0000","last_peered":"2024-09-21T15:00:20.864822+0000","last_clean":"2024-09-21T15:00:20.864822+0000","last_became_active":"2024-09-21T14:58:19.409415+0000","last_became_peered":"2024-09-21T14:58:19.409415+0000","last_unstale":"2024-09-21T15:00:20.864822+0000","last_undegraded":"2024-09-21T15:00:20.864822+0000","last_fullsized":"2024-09-21T15:00:20.864822+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":279,"log_dups_size":0,"ondisk_log_size":279,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:28:46.336301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046801999999999998,"stat_sum":{"num_bytes":18340703,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":2587,"num_read_kb":23326,"num_write":386,"num_write_kb":17018,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"53","length":"1"}]},{"pgid":"3.19","version":"135'338","reported_seq":3125,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-21T15:00:20.143295+0000","last_change":"2024-09-21T15:00:20.143295+0000","last_active":"2024-09-21T15:00:20.143295+0000","last_peered":"2024-09-21T15:00:20.143295+0000","last_clean":"2024-09-21T15:00:20.143295+0000","last_became_active":"2024-09-21T14:59:32.130624+0000","last_became_peered":"2024-09-21T14:59:32.130624+0000","last_unstale":"2024-09-21T15:00:20.143295+0000","last_undegraded":"2024-09-21T15:00:20.143295+0000","last_fullsized":"2024-09-21T15:00:20.143295+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":338,"log_dups_size":0,"ondisk_log_size":338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:37:53.312733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013741269999999999,"stat_sum":{"num_bytes":34362905,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":3401,"num_read_kb":45868,"num_write":494,"num_write_kb":21957,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":128,"num_bytes_recovered":11896992,"num_keys_recovered":44,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"53","length":"1"}]},{"pgid":"3.18","version":"132'471","reported_seq":5782,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-21T15:00:20.143521+0000","last_change":"2024-09-21T15:00:20.143521+0000","last_active":"2024-09-21T15:00:20.143521+0000","last_peered":"2024-09-21T15:00:20.143521+0000","last_clean":"2024-09-21T15:00:20.143521+0000","last_became_active":"2024-09-21T14:58:19.409371+0000","last_became_peered":"2024-09-21T14:58:19.409371+0000","last_unstale":"2024-09-21T15:00:20.143521+0000","last_undegraded":"2024-09-21T15:00:20.143521+0000","last_fullsized":"2024-09-21T15:00:20.143521+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":471,"log_dups_size":0,"ondisk_log_size":471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:35:27.143445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017989530000000001,"stat_sum":{"num_bytes":20804570,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":6532,"num_read_kb":28786,"num_write":696,"num_write_kb":24123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"53","length":"1"}]},{"pgid":"3.17","version":"135'199","reported_seq":3098,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-21T15:00:20.326372+0000","last_change":"2024-09-21T15:00:20.144509+0000","last_active":"2024-09-21T15:00:20.326372+0000","last_peered":"2024-09-21T15:00:20.326372+0000","last_clean":"2024-09-21T15:00:20.326372+0000","last_became_active":"2024-09-21T14:58:19.410306+0000","last_became_peered":"2024-09-21T14:58:19.410306+0000","last_unstale":"2024-09-21T15:00:20.326372+0000","last_undegraded":"2024-09-21T15:00:20.326372+0000","last_fullsized":"2024-09-21T15:00:20.326372+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:33:50.089165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001222175,"stat_sum":{"num_bytes":21853084,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":2759,"num_read_kb":24304,"num_write":350,"num_write_kb":24363,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"53","length":"1"}]},{"pgid":"3.16","version":"135'582","reported_seq":3689,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-21T15:00:20.667930+0000","last_change":"2024-09-21T15:00:20.151671+0000","last_active":"2024-09-21T15:00:20.667930+0000","last_peered":"2024-09-21T15:00:20.667930+0000","last_clean":"2024-09-21T15:00:20.667930+0000","last_became_active":"2024-09-21T14:59:32.132898+0000","last_became_peered":"2024-09-21T14:59:32.132898+0000","last_unstale":"2024-09-21T15:00:20.667930+0000","last_undegraded":"2024-09-21T15:00:20.667930+0000","last_fullsized":"2024-09-21T15:00:20.667930+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":582,"log_dups_size":0,"ondisk_log_size":582,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:09:47.566958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028082749999999998,"stat_sum":{"num_bytes":26032116,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":4932,"num_read_kb":33087,"num_write":858,"num_write_kb":19766,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":153,"num_bytes_recovered":9621881,"num_keys_recovered":33,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"53","length":"1"}]},{"pgid":"3.15","version":"135'639","reported_seq":2975,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-21T15:00:20.324001+0000","last_change":"2024-09-21T15:00:20.238573+0000","last_active":"2024-09-21T15:00:20.324001+0000","last_peered":"2024-09-21T15:00:20.324001+0000","last_clean":"2024-09-21T15:00:20.324001+0000","last_became_active":"2024-09-21T14:58:19.410687+0000","last_became_peered":"2024-09-21T14:58:19.410687+0000","last_unstale":"2024-09-21T15:00:20.324001+0000","last_undegraded":"2024-09-21T15:00:20.324001+0000","last_fullsized":"2024-09-21T15:00:20.324001+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":639,"log_dups_size":0,"ondisk_log_size":639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:21:42.226982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044268999999999998,"stat_sum":{"num_bytes":33007291,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":4152,"num_read_kb":43861,"num_write":1091,"num_write_kb":29882,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"53","length":"1"}]},{"pgid":"3.14","version":"135'314","reported_seq":2083,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-21T15:00:20.728772+0000","last_change":"2024-09-21T15:00:20.150086+0000","last_active":"2024-09-21T15:00:20.728772+0000","last_peered":"2024-09-21T15:00:20.728772+0000","last_clean":"2024-09-21T15:00:20.728772+0000","last_became_active":"2024-09-21T14:58:19.407310+0000","last_became_peered":"2024-09-21T14:58:19.407310+0000","last_unstale":"2024-09-21T15:00:20.728772+0000","last_undegraded":"2024-09-21T15:00:20.728772+0000","last_fullsized":"2024-09-21T15:00:20.728772+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":314,"log_dups_size":0,"ondisk_log_size":314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:47:46.079115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00171222,"stat_sum":{"num_bytes":27524176,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":1971,"num_read_kb":29582,"num_write":401,"num_write_kb":17801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"53","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":129,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-21T15:00:20.148843+0000","last_change":"2024-09-21T14:58:11.348177+0000","last_active":"2024-09-21T15:00:20.148843+0000","last_peered":"2024-09-21T15:00:20.148843+0000","last_clean":"2024-09-21T15:00:20.148843+0000","last_became_active":"2024-09-21T14:58:09.328417+0000","last_became_peered":"2024-09-21T14:58:09.328417+0000","last_unstale":"2024-09-21T15:00:20.148843+0000","last_undegraded":"2024-09-21T15:00:20.148843+0000","last_fullsized":"2024-09-21T15:00:20.148843+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:52:22.671185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018917490000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"135'306","reported_seq":3185,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-21T15:00:20.675490+0000","last_change":"2024-09-21T15:00:20.144330+0000","last_active":"2024-09-21T15:00:20.675490+0000","last_peered":"2024-09-21T15:00:20.675490+0000","last_clean":"2024-09-21T15:00:20.675490+0000","last_became_active":"2024-09-21T14:59:32.131232+0000","last_became_peered":"2024-09-21T14:59:32.131232+0000","last_unstale":"2024-09-21T15:00:20.675490+0000","last_undegraded":"2024-09-21T15:00:20.675490+0000","last_fullsized":"2024-09-21T15:00:20.675490+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":306,"log_dups_size":0,"ondisk_log_size":306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:31:04.421782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001702568,"stat_sum":{"num_bytes":34132145,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":0,"num_read":3116,"num_read_kb":33379,"num_write":408,"num_write_kb":19974,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":3585713,"num_keys_recovered":9,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"53","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":54,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T15:00:19.248950+0000","last_change":"2024-09-21T14:59:28.228883+0000","last_active":"2024-09-21T15:00:19.248950+0000","last_peered":"2024-09-21T15:00:19.248950+0000","last_clean":"2024-09-21T15:00:19.248950+0000","last_became_active":"2024-09-21T14:59:28.228464+0000","last_became_peered":"2024-09-21T14:59:28.228464+0000","last_unstale":"2024-09-21T15:00:19.248950+0000","last_undegraded":"2024-09-21T15:00:19.248950+0000","last_fullsized":"2024-09-21T15:00:19.248950+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:00:26.310864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"135'688","reported_seq":2930,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-21T15:00:20.841901+0000","last_change":"2024-09-21T15:00:20.144434+0000","last_active":"2024-09-21T15:00:20.841901+0000","last_peered":"2024-09-21T15:00:20.841901+0000","last_clean":"2024-09-21T15:00:20.841901+0000","last_became_active":"2024-09-21T14:59:31.132581+0000","last_became_peered":"2024-09-21T14:59:31.132581+0000","last_unstale":"2024-09-21T15:00:20.841901+0000","last_undegraded":"2024-09-21T15:00:20.841901+0000","last_fullsized":"2024-09-21T15:00:20.841901+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":688,"log_dups_size":0,"ondisk_log_size":688,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:36:45.467515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086195100000000001,"stat_sum":{"num_bytes":25811478,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":4445,"num_read_kb":28167,"num_write":1012,"num_write_kb":21270,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":119,"num_bytes_recovered":6871924,"num_keys_recovered":11,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"53","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":145,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T15:00:19.234690+0000","last_change":"2024-09-21T14:59:28.231502+0000","last_active":"2024-09-21T15:00:19.234690+0000","last_peered":"2024-09-21T15:00:19.234690+0000","last_clean":"2024-09-21T15:00:19.234690+0000","last_became_active":"2024-09-21T14:59:28.231043+0000","last_became_peered":"2024-09-21T14:59:28.231043+0000","last_unstale":"2024-09-21T15:00:19.234690+0000","last_undegraded":"2024-09-21T15:00:19.234690+0000","last_fullsized":"2024-09-21T15:00:19.234690+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:09:46.773690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019338770000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"135'278","reported_seq":4143,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-21T15:00:20.155491+0000","last_change":"2024-09-21T15:00:20.155491+0000","last_active":"2024-09-21T15:00:20.155491+0000","last_peered":"2024-09-21T15:00:20.155491+0000","last_clean":"2024-09-21T15:00:20.155491+0000","last_became_active":"2024-09-21T15:00:07.159035+0000","last_became_peered":"2024-09-21T15:00:07.159035+0000","last_unstale":"2024-09-21T15:00:20.155491+0000","last_undegraded":"2024-09-21T15:00:20.155491+0000","last_fullsized":"2024-09-21T15:00:20.155491+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":278,"log_dups_size":0,"ondisk_log_size":278,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:09:15.716689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055134100000000001,"stat_sum":{"num_bytes":27038881,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":4119,"num_read_kb":36099,"num_write":522,"num_write_kb":34399,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"53","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":130,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T15:00:19.227964+0000","last_change":"2024-09-21T14:58:11.406176+0000","last_active":"2024-09-21T15:00:19.227964+0000","last_peered":"2024-09-21T15:00:19.227964+0000","last_clean":"2024-09-21T15:00:19.227964+0000","last_became_active":"2024-09-21T14:58:09.777465+0000","last_became_peered":"2024-09-21T14:58:09.777465+0000","last_unstale":"2024-09-21T15:00:19.227964+0000","last_undegraded":"2024-09-21T15:00:19.227964+0000","last_fullsized":"2024-09-21T15:00:19.227964+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:55:04.667816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095704000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"133'597","reported_seq":2219,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-21T15:00:20.219755+0000","last_change":"2024-09-21T15:00:20.144167+0000","last_active":"2024-09-21T15:00:20.219755+0000","last_peered":"2024-09-21T15:00:20.219755+0000","last_clean":"2024-09-21T15:00:20.219755+0000","last_became_active":"2024-09-21T14:58:19.408971+0000","last_became_peered":"2024-09-21T14:58:19.408971+0000","last_unstale":"2024-09-21T15:00:20.219755+0000","last_undegraded":"2024-09-21T15:00:20.219755+0000","last_fullsized":"2024-09-21T15:00:20.219755+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":597,"log_dups_size":0,"ondisk_log_size":597,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:29:52.642121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044485599999999998,"stat_sum":{"num_bytes":12010717,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":3390,"num_read_kb":15517,"num_write":993,"num_write_kb":14111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"53","length":"1"}]},{"pgid":"3.3","version":"135'383","reported_seq":1956,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-21T15:00:20.239500+0000","last_change":"2024-09-21T15:00:20.239500+0000","last_active":"2024-09-21T15:00:20.239500+0000","last_peered":"2024-09-21T15:00:20.239500+0000","last_clean":"2024-09-21T15:00:20.239500+0000","last_became_active":"2024-09-21T14:58:19.417453+0000","last_became_peered":"2024-09-21T14:58:19.417453+0000","last_unstale":"2024-09-21T15:00:20.239500+0000","last_undegraded":"2024-09-21T15:00:20.239500+0000","last_fullsized":"2024-09-21T15:00:20.239500+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:19:27.087272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037973999999999999,"stat_sum":{"num_bytes":13295449,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":2303,"num_read_kb":19181,"num_write":706,"num_write_kb":15409,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"53","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":131,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T15:00:19.235158+0000","last_change":"2024-09-21T14:58:11.347504+0000","last_active":"2024-09-21T15:00:19.235158+0000","last_peered":"2024-09-21T15:00:19.235158+0000","last_clean":"2024-09-21T15:00:19.235158+0000","last_became_active":"2024-09-21T14:58:09.775981+0000","last_became_peered":"2024-09-21T14:58:09.775981+0000","last_unstale":"2024-09-21T15:00:19.235158+0000","last_undegraded":"2024-09-21T15:00:19.235158+0000","last_fullsized":"2024-09-21T15:00:19.235158+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:47:16.286652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012568880000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"135'398","reported_seq":2123,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-21T15:00:20.796247+0000","last_change":"2024-09-21T15:00:20.143227+0000","last_active":"2024-09-21T15:00:20.796247+0000","last_peered":"2024-09-21T15:00:20.796247+0000","last_clean":"2024-09-21T15:00:20.796247+0000","last_became_active":"2024-09-21T14:58:19.405811+0000","last_became_peered":"2024-09-21T14:58:19.405811+0000","last_unstale":"2024-09-21T15:00:20.796247+0000","last_undegraded":"2024-09-21T15:00:20.796247+0000","last_fullsized":"2024-09-21T15:00:20.796247+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":398,"log_dups_size":0,"ondisk_log_size":398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:36:25.551607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001803624,"stat_sum":{"num_bytes":17157745,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":2624,"num_read_kb":21829,"num_write":535,"num_write_kb":22769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"53","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":126,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-21T15:00:15.209155+0000","last_change":"2024-09-21T14:58:11.342210+0000","last_active":"2024-09-21T15:00:15.209155+0000","last_peered":"2024-09-21T15:00:15.209155+0000","last_clean":"2024-09-21T15:00:15.209155+0000","last_became_active":"2024-09-21T14:58:09.330624+0000","last_became_peered":"2024-09-21T14:58:09.330624+0000","last_unstale":"2024-09-21T15:00:15.209155+0000","last_undegraded":"2024-09-21T15:00:15.209155+0000","last_fullsized":"2024-09-21T15:00:15.209155+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:29:43.229538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001051151,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"135'219","reported_seq":987,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-21T15:00:20.668749+0000","last_change":"2024-09-21T15:00:20.155829+0000","last_active":"2024-09-21T15:00:20.668749+0000","last_peered":"2024-09-21T15:00:20.668749+0000","last_clean":"2024-09-21T15:00:20.668749+0000","last_became_active":"2024-09-21T15:00:07.158207+0000","last_became_peered":"2024-09-21T15:00:07.158207+0000","last_unstale":"2024-09-21T15:00:20.668749+0000","last_undegraded":"2024-09-21T15:00:20.668749+0000","last_fullsized":"2024-09-21T15:00:20.668749+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":219,"log_dups_size":0,"ondisk_log_size":219,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:36:40.720592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061053099999999999,"stat_sum":{"num_bytes":29226045,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":1713,"num_read_kb":30167,"num_write":329,"num_write_kb":28224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":150,"num_bytes_recovered":8136552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"53","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":129,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T15:00:19.234045+0000","last_change":"2024-09-21T14:58:11.346766+0000","last_active":"2024-09-21T15:00:19.234045+0000","last_peered":"2024-09-21T15:00:19.234045+0000","last_clean":"2024-09-21T15:00:19.234045+0000","last_became_active":"2024-09-21T14:58:09.329748+0000","last_became_peered":"2024-09-21T14:58:09.329748+0000","last_unstale":"2024-09-21T15:00:19.234045+0000","last_undegraded":"2024-09-21T15:00:19.234045+0000","last_fullsized":"2024-09-21T15:00:19.234045+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:31:20.267675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052698000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"135'348","reported_seq":5189,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-21T15:00:20.665208+0000","last_change":"2024-09-21T15:00:20.144095+0000","last_active":"2024-09-21T15:00:20.665208+0000","last_peered":"2024-09-21T15:00:20.665208+0000","last_clean":"2024-09-21T15:00:20.665208+0000","last_became_active":"2024-09-21T14:58:41.401003+0000","last_became_peered":"2024-09-21T14:58:41.401003+0000","last_unstale":"2024-09-21T15:00:20.665208+0000","last_undegraded":"2024-09-21T15:00:20.665208+0000","last_fullsized":"2024-09-21T15:00:20.665208+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:05:37.644835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043765600000000003,"stat_sum":{"num_bytes":40989545,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":5925,"num_read_kb":40952,"num_write":393,"num_write_kb":16150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1403348,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"53","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":130,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T15:00:19.234478+0000","last_change":"2024-09-21T14:58:11.347700+0000","last_active":"2024-09-21T15:00:19.234478+0000","last_peered":"2024-09-21T15:00:19.234478+0000","last_clean":"2024-09-21T15:00:19.234478+0000","last_became_active":"2024-09-21T14:58:09.438842+0000","last_became_peered":"2024-09-21T14:58:09.438842+0000","last_unstale":"2024-09-21T15:00:19.234478+0000","last_undegraded":"2024-09-21T15:00:19.234478+0000","last_fullsized":"2024-09-21T15:00:19.234478+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:41:34.626249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001426194,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":278,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T15:00:19.238199+0000","last_change":"2024-09-21T14:58:03.616055+0000","last_active":"2024-09-21T15:00:19.238199+0000","last_peered":"2024-09-21T15:00:19.238199+0000","last_clean":"2024-09-21T15:00:19.238199+0000","last_became_active":"2024-09-21T14:58:03.613214+0000","last_became_peered":"2024-09-21T14:58:03.613214+0000","last_unstale":"2024-09-21T15:00:19.238199+0000","last_undegraded":"2024-09-21T15:00:19.238199+0000","last_fullsized":"2024-09-21T15:00:19.238199+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:02.180162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:02.180162+0000","last_clean_scrub_stamp":"2024-09-21T14:58:02.180162+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:29:05.860361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"135'525","reported_seq":3469,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-21T15:00:20.664803+0000","last_change":"2024-09-21T15:00:20.143225+0000","last_active":"2024-09-21T15:00:20.664803+0000","last_peered":"2024-09-21T15:00:20.664803+0000","last_clean":"2024-09-21T15:00:20.664803+0000","last_became_active":"2024-09-21T14:59:31.131460+0000","last_became_peered":"2024-09-21T14:59:31.131460+0000","last_unstale":"2024-09-21T15:00:20.664803+0000","last_undegraded":"2024-09-21T15:00:20.664803+0000","last_fullsized":"2024-09-21T15:00:20.664803+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":525,"log_dups_size":0,"ondisk_log_size":525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:01:38.353476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001651203,"stat_sum":{"num_bytes":31944180,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":3938,"num_read_kb":38113,"num_write":864,"num_write_kb":25315,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":119,"num_bytes_recovered":6097618,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"53","length":"1"}]},{"pgid":"3.9","version":"135'645","reported_seq":2193,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-21T15:00:20.329373+0000","last_change":"2024-09-21T15:00:20.149221+0000","last_active":"2024-09-21T15:00:20.329373+0000","last_peered":"2024-09-21T15:00:20.329373+0000","last_clean":"2024-09-21T15:00:20.329373+0000","last_became_active":"2024-09-21T14:59:28.233956+0000","last_became_peered":"2024-09-21T14:59:28.233956+0000","last_unstale":"2024-09-21T15:00:20.329373+0000","last_undegraded":"2024-09-21T15:00:20.329373+0000","last_fullsized":"2024-09-21T15:00:20.329373+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":645,"log_dups_size":0,"ondisk_log_size":645,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:29:09.682792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012266880000000001,"stat_sum":{"num_bytes":18330993,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":3432,"num_read_kb":24339,"num_write":1089,"num_write_kb":25134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":91,"num_bytes_recovered":6925016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"53","length":"1"}]},{"pgid":"3.a","version":"135'618","reported_seq":3291,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-21T15:00:20.676831+0000","last_change":"2024-09-21T15:00:20.151422+0000","last_active":"2024-09-21T15:00:20.676831+0000","last_peered":"2024-09-21T15:00:20.676831+0000","last_clean":"2024-09-21T15:00:20.676831+0000","last_became_active":"2024-09-21T14:58:19.407847+0000","last_became_peered":"2024-09-21T14:58:19.407847+0000","last_unstale":"2024-09-21T15:00:20.676831+0000","last_undegraded":"2024-09-21T15:00:20.676831+0000","last_fullsized":"2024-09-21T15:00:20.676831+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":618,"log_dups_size":0,"ondisk_log_size":618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:45:44.979236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029249699999999998,"stat_sum":{"num_bytes":18797925,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":0,"num_read":4962,"num_read_kb":26178,"num_write":869,"num_write_kb":28512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"53","length":"1"}]},{"pgid":"3.b","version":"131'344","reported_seq":2374,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-21T15:00:16.272646+0000","last_change":"2024-09-21T15:00:15.209964+0000","last_active":"2024-09-21T15:00:16.272646+0000","last_peered":"2024-09-21T15:00:16.272646+0000","last_clean":"2024-09-21T15:00:16.272646+0000","last_became_active":"2024-09-21T14:58:19.410679+0000","last_became_peered":"2024-09-21T14:58:19.410679+0000","last_unstale":"2024-09-21T15:00:16.272646+0000","last_undegraded":"2024-09-21T15:00:16.272646+0000","last_fullsized":"2024-09-21T15:00:16.272646+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":344,"log_dups_size":0,"ondisk_log_size":344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:07:40.758932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053074000000000003,"stat_sum":{"num_bytes":27790327,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":2481,"num_read_kb":32558,"num_write":561,"num_write_kb":34094,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"50","length":"1"}]},{"pgid":"3.c","version":"135'336","reported_seq":3688,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-21T15:00:20.667722+0000","last_change":"2024-09-21T15:00:20.239086+0000","last_active":"2024-09-21T15:00:20.667722+0000","last_peered":"2024-09-21T15:00:20.667722+0000","last_clean":"2024-09-21T15:00:20.667722+0000","last_became_active":"2024-09-21T14:59:25.130057+0000","last_became_peered":"2024-09-21T14:59:25.130057+0000","last_unstale":"2024-09-21T15:00:20.667722+0000","last_undegraded":"2024-09-21T15:00:20.667722+0000","last_fullsized":"2024-09-21T15:00:20.667722+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":336,"log_dups_size":0,"ondisk_log_size":336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:53:33.752550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000824236,"stat_sum":{"num_bytes":15284342,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":3870,"num_read_kb":20937,"num_write":488,"num_write_kb":16588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":9059600,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"53","length":"1"}]},{"pgid":"3.d","version":"135'301","reported_seq":2988,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-21T15:00:20.864219+0000","last_change":"2024-09-21T15:00:20.151374+0000","last_active":"2024-09-21T15:00:20.864219+0000","last_peered":"2024-09-21T15:00:20.864219+0000","last_clean":"2024-09-21T15:00:20.864219+0000","last_became_active":"2024-09-21T14:58:19.407884+0000","last_became_peered":"2024-09-21T14:58:19.407884+0000","last_unstale":"2024-09-21T15:00:20.864219+0000","last_undegraded":"2024-09-21T15:00:20.864219+0000","last_fullsized":"2024-09-21T15:00:20.864219+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":301,"log_dups_size":0,"ondisk_log_size":301,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:28:57.111615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028543610000000001,"stat_sum":{"num_bytes":34397991,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":2818,"num_read_kb":39957,"num_write":523,"num_write_kb":28441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"53","length":"1"}]},{"pgid":"3.e","version":"135'712","reported_seq":2662,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-21T15:00:20.239280+0000","last_change":"2024-09-21T15:00:20.239006+0000","last_active":"2024-09-21T15:00:20.239280+0000","last_peered":"2024-09-21T15:00:20.239280+0000","last_clean":"2024-09-21T15:00:20.239280+0000","last_became_active":"2024-09-21T14:59:13.131825+0000","last_became_peered":"2024-09-21T14:59:13.131825+0000","last_unstale":"2024-09-21T15:00:20.239280+0000","last_undegraded":"2024-09-21T15:00:20.239280+0000","last_fullsized":"2024-09-21T15:00:20.239280+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":712,"log_dups_size":0,"ondisk_log_size":712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:20:52.475349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087421999999999999,"stat_sum":{"num_bytes":10512224,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":4065,"num_read_kb":14538,"num_write":1187,"num_write_kb":11347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":2113894,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"53","length":"1"}]},{"pgid":"3.f","version":"135'355","reported_seq":3209,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-21T15:00:20.238568+0000","last_change":"2024-09-21T15:00:20.238568+0000","last_active":"2024-09-21T15:00:20.238568+0000","last_peered":"2024-09-21T15:00:20.238568+0000","last_clean":"2024-09-21T15:00:20.238568+0000","last_became_active":"2024-09-21T14:58:19.410406+0000","last_became_peered":"2024-09-21T14:58:19.410406+0000","last_unstale":"2024-09-21T15:00:20.238568+0000","last_undegraded":"2024-09-21T15:00:20.238568+0000","last_fullsized":"2024-09-21T15:00:20.238568+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":355,"log_dups_size":0,"ondisk_log_size":355,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:59:16.026939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050337799999999996,"stat_sum":{"num_bytes":29745096,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":3265,"num_read_kb":38571,"num_write":576,"num_write_kb":31284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"53","length":"1"}]},{"pgid":"3.10","version":"135'346","reported_seq":3500,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-21T15:00:20.677928+0000","last_change":"2024-09-21T15:00:20.153410+0000","last_active":"2024-09-21T15:00:20.677928+0000","last_peered":"2024-09-21T15:00:20.677928+0000","last_clean":"2024-09-21T15:00:20.677928+0000","last_became_active":"2024-09-21T14:58:19.409935+0000","last_became_peered":"2024-09-21T14:58:19.409935+0000","last_unstale":"2024-09-21T15:00:20.677928+0000","last_undegraded":"2024-09-21T15:00:20.677928+0000","last_fullsized":"2024-09-21T15:00:20.677928+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":346,"log_dups_size":0,"ondisk_log_size":346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:10:36.252629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057687199999999998,"stat_sum":{"num_bytes":44778579,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":3291,"num_read_kb":51552,"num_write":540,"num_write_kb":26164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"53","length":"1"}]},{"pgid":"3.11","version":"135'313","reported_seq":3331,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-21T15:00:20.247585+0000","last_change":"2024-09-21T15:00:20.148414+0000","last_active":"2024-09-21T15:00:20.247585+0000","last_peered":"2024-09-21T15:00:20.247585+0000","last_clean":"2024-09-21T15:00:20.247585+0000","last_became_active":"2024-09-21T14:59:10.128001+0000","last_became_peered":"2024-09-21T14:59:10.128001+0000","last_unstale":"2024-09-21T15:00:20.247585+0000","last_undegraded":"2024-09-21T15:00:20.247585+0000","last_fullsized":"2024-09-21T15:00:20.247585+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":313,"log_dups_size":0,"ondisk_log_size":313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:44:22.770961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071792199999999996,"stat_sum":{"num_bytes":15225199,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":3749,"num_read_kb":18874,"num_write":354,"num_write_kb":15462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":3094046,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"53","length":"1"}]},{"pgid":"3.12","version":"135'235","reported_seq":3756,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-21T15:00:20.669856+0000","last_change":"2024-09-21T15:00:20.155597+0000","last_active":"2024-09-21T15:00:20.669856+0000","last_peered":"2024-09-21T15:00:20.669856+0000","last_clean":"2024-09-21T15:00:20.669856+0000","last_became_active":"2024-09-21T15:00:07.155144+0000","last_became_peered":"2024-09-21T15:00:07.155144+0000","last_unstale":"2024-09-21T15:00:20.669856+0000","last_undegraded":"2024-09-21T15:00:20.669856+0000","last_fullsized":"2024-09-21T15:00:20.669856+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":235,"log_dups_size":0,"ondisk_log_size":235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:58:00.133639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061691899999999995,"stat_sum":{"num_bytes":19524684,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":3742,"num_read_kb":14195,"num_write":327,"num_write_kb":11013,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"53","length":"1"}]},{"pgid":"3.13","version":"135'464","reported_seq":2991,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-21T15:00:20.670554+0000","last_change":"2024-09-21T15:00:20.149024+0000","last_active":"2024-09-21T15:00:20.670554+0000","last_peered":"2024-09-21T15:00:20.670554+0000","last_clean":"2024-09-21T15:00:20.670554+0000","last_became_active":"2024-09-21T14:58:19.412030+0000","last_became_peered":"2024-09-21T14:58:19.412030+0000","last_unstale":"2024-09-21T15:00:20.670554+0000","last_undegraded":"2024-09-21T15:00:20.670554+0000","last_fullsized":"2024-09-21T15:00:20.670554+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":464,"log_dups_size":0,"ondisk_log_size":464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:44:29.282391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085379100000000001,"stat_sum":{"num_bytes":38222063,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":3698,"num_read_kb":41551,"num_write":625,"num_write_kb":21629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"53","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":778592382,"num_objects":1022,"num_object_clones":0,"num_object_copies":2044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1022,"num_whiteouts":0,"num_read":109601,"num_read_kb":934260,"num_write":19323,"num_write_kb":698664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1206,"num_bytes_recovered":80408212,"num_keys_recovered":119,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":52150272,"data_stored":469652168,"data_compressed":10342071,"data_compressed_allocated":41889792,"data_compressed_original":463897061,"omap_allocated":0,"internal_metadata":0},"log_size":12701,"ondisk_log_size":12701,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":10404,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509471,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49292,"kb_used_data":5892,"kb_used_omap":1,"kb_used_meta":43390,"kb_avail":94322548,"statfs":{"total":96636764160,"available":96586289152,"internally_reserved":0,"allocated":6033408,"data_stored":42186015,"data_compressed":876885,"data_compressed_allocated":3637248,"data_compressed_original":40988923,"omap_allocated":1589,"internal_metadata":44431819},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:00:03 2024","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.84499999999999997,"15min":0.83899999999999997},"min":{"1min":0.504,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.95099999999999996,"15min":0.95799999999999996},"min":{"1min":0.51400000000000001,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.4970000000000001,"5min":1.917,"15min":1.917},"last":1.0469999999999999}]},{"osd":1,"last update":"Sat Sep 21 15:00:03 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.78500000000000003,"15min":0.76900000000000002},"min":{"1min":0.626,"5min":0.312,"15min":0.312},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.879,"5min":0.83499999999999996,"15min":0.82699999999999996},"min":{"1min":0.53600000000000003,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.224,"5min":2.2149999999999999,"15min":2.2149999999999999},"last":1.0269999999999999}]},{"osd":2,"last update":"Sat Sep 21 15:00:03 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.81699999999999995,"15min":0.81100000000000005},"min":{"1min":0.495,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.1859999999999999,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.95099999999999996,"15min":0.94799999999999995},"min":{"1min":0.72199999999999998,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.3300000000000001,"5min":1.466,"15min":1.466},"last":0.58799999999999997}]},{"osd":3,"last update":"Sat Sep 21 15:00:03 2024","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.86199999999999999,"15min":0.85399999999999998},"min":{"1min":0.67200000000000004,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.3120000000000001,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":0.70399999999999996},{"interface":"front","average":{"1min":1.0429999999999999,"5min":0.97999999999999998,"15min":0.96999999999999997},"min":{"1min":0.68500000000000005,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.244,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.85599999999999998}]},{"osd":4,"last update":"Sat Sep 21 15:00:03 2024","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.93700000000000006,"15min":0.93799999999999994},"min":{"1min":0.64200000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.4510000000000001,"5min":2.7130000000000001,"15min":2.7130000000000001},"last":0.63700000000000001},{"interface":"front","average":{"1min":1.0369999999999999,"5min":0.94099999999999995,"15min":0.92500000000000004},"min":{"1min":0.59999999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.6499999999999999,"5min":2.125,"15min":2.125},"last":1.089}]},{"osd":5,"last update":"Sat Sep 21 15:00:03 2024","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.0409999999999999,"15min":1.0269999999999999},"min":{"1min":0.71099999999999997,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.4770000000000001,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.0640000000000001,"15min":1.0580000000000001},"min":{"1min":0.70799999999999996,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.8169999999999999,"5min":3.1880000000000002,"15min":3.1880000000000002},"last":1.1279999999999999}]},{"osd":6,"last update":"Sat Sep 21 15:00:03 2024","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.163,"15min":1.159},"min":{"1min":0.71299999999999997,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.6579999999999999,"5min":3.2730000000000001,"15min":3.2730000000000001},"last":1.5329999999999999},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.0469999999999999,"15min":1.0249999999999999},"min":{"1min":0.79100000000000004,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.6180000000000001,"5min":2.2879999999999998,"15min":2.2879999999999998},"last":0.94699999999999995}]}]},{"osd":6,"up_from":14,"seq":60129542175,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57900,"kb_used_data":8736,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94313940,"statfs":{"total":96636764160,"available":96577474560,"internally_reserved":0,"allocated":8945664,"data_stored":67851354,"data_compressed":1514116,"data_compressed_allocated":5976064,"data_compressed_original":66310895,"omap_allocated":1593,"internal_metadata":50330055},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:00:06 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.94399999999999995,"15min":0.95699999999999996},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.244,"5min":1.593,"15min":1.593},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.93600000000000005,"15min":0.95099999999999996},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.3360000000000001,"5min":1.587,"15min":1.587},"last":1.028}]},{"osd":1,"last update":"Sat Sep 21 15:00:06 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.90400000000000003,"15min":0.91400000000000003},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.1120000000000001,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.95699999999999996,"15min":0.97999999999999998},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.1819999999999999,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.51300000000000001}]},{"osd":2,"last update":"Sat Sep 21 15:00:06 2024","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.98499999999999999,"15min":0.99099999999999999},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.3560000000000001,"5min":1.758,"15min":1.758},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.89900000000000002,"15min":0.90900000000000003},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.3069999999999999,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.97299999999999998}]},{"osd":3,"last update":"Sat Sep 21 15:00:06 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.94499999999999995,"15min":0.95799999999999996},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.169,"5min":1.835,"15min":1.835},"last":0.66800000000000004},{"interface":"front","average":{"1min":1.012,"5min":1.1180000000000001,"15min":1.1359999999999999},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.4430000000000001,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":1.1659999999999999}]},{"osd":4,"last update":"Sat Sep 21 15:00:06 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.97699999999999998,"15min":1.0049999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.1379999999999999,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.93999999999999995,"5min":1.02,"15min":1.0329999999999999},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.411,"5min":1.897,"15min":1.897},"last":0.76200000000000001}]},{"osd":5,"last update":"Sat Sep 21 15:00:06 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":1.0860000000000001,"15min":1.1240000000000001},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.2589999999999999,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.79300000000000004},{"interface":"front","average":{"1min":1.01,"5min":1.0800000000000001,"15min":1.0920000000000001},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.3779999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":1.1100000000000001}]},{"osd":7,"last update":"Sat Sep 21 15:00:06 2024","interfaces":[{"interface":"back","average":{"1min":1.123,"5min":1.2350000000000001,"15min":1.2529999999999999},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.4530000000000001,"5min":1.8660000000000001,"15min":1.8660000000000001},"last":1.234},{"interface":"front","average":{"1min":1.044,"5min":1.26,"15min":1.296},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.385,"5min":2.004,"15min":2.004},"last":1.141}]}]},{"osd":5,"up_from":14,"seq":60129542176,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40140,"kb_used_data":9028,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94331700,"statfs":{"total":96636764160,"available":96595660800,"internally_reserved":0,"allocated":9244672,"data_stored":69889055,"data_compressed":1333558,"data_compressed_allocated":6025216,"data_compressed_original":68216804,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:00:01 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.96799999999999997,"15min":1.0109999999999999},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.98999999999999999,"5min":4.4800000000000004,"15min":4.4800000000000004},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.80800000000000005,"15min":0.82099999999999995},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.0329999999999999,"5min":1.181,"15min":1.181},"last":0.36699999999999999}]},{"osd":1,"last update":"Sat Sep 21 15:00:01 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.91000000000000003,"15min":0.92800000000000005},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.165,"5min":1.75,"15min":1.75},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.91900000000000004,"15min":0.93500000000000005},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.216,"5min":2.2400000000000002,"15min":2.2400000000000002},"last":0.88600000000000001}]},{"osd":2,"last update":"Sat Sep 21 15:00:01 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.95599999999999996,"15min":0.98799999999999999},"min":{"1min":0.54100000000000004,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.016,"5min":3.419,"15min":3.419},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.88500000000000001,"15min":0.90100000000000002},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.075,"5min":1.423,"15min":1.423},"last":0.53400000000000003}]},{"osd":3,"last update":"Sat Sep 21 15:00:01 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.80100000000000005,"15min":0.80700000000000005},"min":{"1min":0.54800000000000004,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.016,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.84899999999999998},{"interface":"front","average":{"1min":1.0069999999999999,"5min":1.081,"15min":1.093},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":2.6379999999999999,"5min":2.6379999999999999,"15min":2.6379999999999999},"last":0.59899999999999998}]},{"osd":4,"last update":"Sat Sep 21 15:00:01 2024","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":1.0129999999999999,"15min":1.024},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":2.5470000000000002,"5min":2.5470000000000002,"15min":2.5470000000000002},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.93999999999999995,"5min":1.075,"15min":1.097},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":2.5099999999999998,"5min":2.5099999999999998,"15min":2.5099999999999998},"last":0.41199999999999998}]},{"osd":6,"last update":"Sat Sep 21 15:00:01 2024","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.95699999999999996,"15min":0.97299999999999998},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.1539999999999999,"5min":1.244,"15min":1.244},"last":0.93100000000000005},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.325,"15min":1.379},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":2.6789999999999998,"5min":4.2469999999999999,"15min":4.2469999999999999},"last":0.73299999999999998}]},{"osd":7,"last update":"Sat Sep 21 15:00:05 2024","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.3200000000000001,"15min":1.363},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":2.7189999999999999,"5min":3.3010000000000002,"15min":3.3010000000000002},"last":0.76700000000000002},{"interface":"front","average":{"1min":1.0740000000000001,"5min":1.1339999999999999,"15min":1.1439999999999999},"min":{"1min":0.64200000000000002,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":2.5880000000000001,"5min":2.5880000000000001,"15min":2.5880000000000001},"last":0.90500000000000003}]}]},{"osd":4,"up_from":14,"seq":60129542175,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38524,"kb_used_data":7408,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94333316,"statfs":{"total":96636764160,"available":96597315584,"internally_reserved":0,"allocated":7585792,"data_stored":58697723,"data_compressed":1274475,"data_compressed_allocated":5050368,"data_compressed_original":57403287,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:00:04 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.76900000000000002,"15min":0.77900000000000003},"min":{"1min":0.36399999999999999,"5min":0.311,"15min":0.311},"max":{"1min":0.96799999999999997,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.92100000000000004,"15min":0.93799999999999994},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.486,"5min":1.486,"15min":1.486},"last":0.72999999999999998}]},{"osd":1,"last update":"Sat Sep 21 15:00:04 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.89000000000000001,"15min":0.91600000000000004},"min":{"1min":0.48099999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.92300000000000004,"5min":1.347,"15min":1.347},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.83899999999999997,"15min":0.84499999999999997},"min":{"1min":0.39600000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.0720000000000001,"5min":1.153,"15min":1.153},"last":0.61399999999999999}]},{"osd":2,"last update":"Sat Sep 21 15:00:04 2024","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.875,"15min":0.877},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.268,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":0.625},{"interface":"front","average":{"1min":0.91600000000000004,"5min":1.0129999999999999,"15min":1.0289999999999999},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.258,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.68100000000000005}]},{"osd":3,"last update":"Sat Sep 21 15:00:04 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.89300000000000002,"15min":0.89000000000000001},"min":{"1min":0.68100000000000005,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.407,"5min":1.538,"15min":1.538},"last":0.76800000000000002},{"interface":"front","average":{"1min":1.095,"5min":1.004,"15min":0.98899999999999999},"min":{"1min":0.63100000000000001,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.365,"5min":1.365,"15min":1.365},"last":0.56599999999999995}]},{"osd":5,"last update":"Sat Sep 21 15:00:04 2024","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.97899999999999998,"15min":0.98499999999999999},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.2210000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.872,"5min":0.95199999999999996,"15min":0.96499999999999997},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.1619999999999999,"5min":1.337,"15min":1.337},"last":0.67000000000000004}]},{"osd":6,"last update":"Sat Sep 21 15:00:04 2024","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":1.0009999999999999,"15min":1.0129999999999999},"min":{"1min":0.51400000000000001,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.2210000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":1.077,"5min":1.091,"15min":1.093},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.5449999999999999,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":0.45500000000000002}]},{"osd":7,"last update":"Sat Sep 21 15:00:04 2024","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":1.022,"15min":1.034},"min":{"1min":0.46000000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.3859999999999999,"5min":1.7010000000000001,"15min":1.7010000000000001},"last":0.81799999999999995},{"interface":"front","average":{"1min":1.202,"5min":1.214,"15min":1.216},"min":{"1min":0.78800000000000003,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.4590000000000001,"5min":1.53,"15min":1.53},"last":0.872}]}]},{"osd":3,"up_from":14,"seq":60129542176,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63688,"kb_used_data":10428,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94308152,"statfs":{"total":96636764160,"available":96571547648,"internally_reserved":0,"allocated":10678272,"data_stored":81038770,"data_compressed":1705530,"data_compressed_allocated":7118848,"data_compressed_original":79144774,"omap_allocated":1592,"internal_metadata":54524360},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:00:05 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.378,"5min":0.315,"15min":0.315},"max":{"1min":1.0109999999999999,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.71199999999999997,"15min":0.71999999999999997},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.89200000000000002,"5min":1.919,"15min":1.919},"last":0.57099999999999995}]},{"osd":1,"last update":"Sat Sep 21 15:00:05 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.67500000000000004,"15min":0.67600000000000005},"min":{"1min":0.33400000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":1.1240000000000001,"5min":1.135,"15min":1.135},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.80700000000000005,"15min":0.81699999999999995},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.1599999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.77400000000000002}]},{"osd":2,"last update":"Sat Sep 21 15:00:05 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.72799999999999998,"15min":0.72399999999999998},"min":{"1min":0.40200000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.83399999999999996,"15min":0.83499999999999996},"min":{"1min":0.47499999999999998,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.1759999999999999,"5min":1.254,"15min":1.254},"last":0.48999999999999999}]},{"osd":4,"last update":"Sat Sep 21 15:00:05 2024","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.95699999999999996,"15min":0.96299999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.2709999999999999,"5min":1.7609999999999999,"15min":1.7609999999999999},"last":1.1759999999999999},{"interface":"front","average":{"1min":0.91100000000000003,"5min":1.0189999999999999,"15min":1.0369999999999999},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.25,"5min":2.028,"15min":2.028},"last":1.0549999999999999}]},{"osd":5,"last update":"Sat Sep 21 15:00:05 2024","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.052,"15min":1.0569999999999999},"min":{"1min":0.77700000000000002,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.246,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.3440000000000001},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.0649999999999999,"15min":1.0660000000000001},"min":{"1min":0.70799999999999996,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.3600000000000001,"5min":1.431,"15min":1.431},"last":1.212}]},{"osd":6,"last update":"Sat Sep 21 15:00:05 2024","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.0860000000000001,"15min":1.0980000000000001},"min":{"1min":0.58599999999999997,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.3879999999999999,"5min":2.1219999999999999,"15min":2.1219999999999999},"last":1.0469999999999999},{"interface":"front","average":{"1min":1.046,"5min":1.097,"15min":1.105},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.7370000000000001,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":1.0920000000000001}]},{"osd":7,"last update":"Sat Sep 21 15:00:05 2024","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.1279999999999999,"15min":1.139},"min":{"1min":0.61799999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.4870000000000001,"5min":1.706,"15min":1.706},"last":0.92200000000000004},{"interface":"front","average":{"1min":1.093,"5min":1.173,"15min":1.1859999999999999},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.516,"5min":2.3380000000000001,"15min":2.3380000000000001},"last":1.28}]}]},{"osd":2,"up_from":14,"seq":60129542174,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31436,"kb_used_data":4412,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94340404,"statfs":{"total":96636764160,"available":96604573696,"internally_reserved":0,"allocated":4517888,"data_stored":21591538,"data_compressed":757641,"data_compressed_allocated":2220032,"data_compressed_original":20439711,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:00:00 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.62,"15min":0.62},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.97999999999999998,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.66100000000000003,"15min":0.65900000000000003},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":1.0429999999999999,"5min":1.244,"15min":1.244},"last":0.81100000000000005}]},{"osd":1,"last update":"Sat Sep 21 15:00:00 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.69699999999999995,"15min":0.68799999999999994},"min":{"1min":0.32400000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":2.6520000000000001,"5min":2.6520000000000001,"15min":2.6520000000000001},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.73999999999999999,"15min":0.76000000000000001},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.89300000000000002,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.85799999999999998}]},{"osd":3,"last update":"Sat Sep 21 15:00:00 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.72599999999999998,"15min":0.72999999999999998},"min":{"1min":0.38900000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.113,"5min":2.1259999999999999,"15min":2.1259999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":1.1359999999999999,"5min":0.88400000000000001,"15min":0.84199999999999997},"min":{"1min":0.52600000000000002,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":4.1440000000000001,"5min":4.1440000000000001,"15min":4.1440000000000001},"last":0.77600000000000002}]},{"osd":4,"last update":"Sat Sep 21 15:00:00 2024","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.96199999999999997,"15min":0.96599999999999997},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":2.899,"5min":2.899,"15min":2.899},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.95199999999999996,"15min":0.95999999999999996},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.1679999999999999,"5min":1.675,"15min":1.675},"last":1.256}]},{"osd":5,"last update":"Sat Sep 21 15:00:00 2024","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":1.0049999999999999,"15min":1.0069999999999999},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":2.8450000000000002,"5min":2.8450000000000002,"15min":2.8450000000000002},"last":0.89100000000000001},{"interface":"front","average":{"1min":1.002,"5min":1.0009999999999999,"15min":1.0009999999999999},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.502,"5min":1.546,"15min":1.546},"last":1.0389999999999999}]},{"osd":6,"last update":"Sat Sep 21 15:00:00 2024","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.97899999999999998,"15min":0.98399999999999999},"min":{"1min":0.40100000000000002,"5min":0.378,"15min":0.378},"max":{"1min":1.153,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":1.3220000000000001},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.05,"15min":1.05},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.399,"5min":1.446,"15min":1.446},"last":0.97999999999999998}]},{"osd":7,"last update":"Sat Sep 21 15:00:07 2024","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":1.0029999999999999,"15min":0.995},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":1.0249999999999999},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.097,"15min":1.091},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.5489999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":1.0660000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542175,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40644,"kb_used_data":9528,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94331196,"statfs":{"total":96636764160,"available":96595144704,"internally_reserved":0,"allocated":9756672,"data_stored":78469563,"data_compressed":1429607,"data_compressed_allocated":6619136,"data_compressed_original":76838360,"omap_allocated":1595,"internal_metadata":31848901},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:00:05 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.60299999999999998,"15min":0.61099999999999999},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.83599999999999997,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.60099999999999998,"15min":0.60899999999999999},"min":{"1min":0.252,"5min":0.216,"15min":0.216},"max":{"1min":0.90800000000000003,"5min":1.107,"15min":1.107},"last":0.55100000000000005}]},{"osd":2,"last update":"Sat Sep 21 15:00:05 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.68999999999999995,"15min":0.70499999999999996},"min":{"1min":0.313,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":1.399,"5min":2.5529999999999999,"15min":2.5529999999999999},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69599999999999995,"15min":0.69699999999999995},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.95499999999999996,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.47599999999999998}]},{"osd":3,"last update":"Sat Sep 21 15:00:05 2024","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.84399999999999997,"15min":0.84799999999999998},"min":{"1min":0.311,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":3.1469999999999998,"5min":3.1469999999999998,"15min":3.1469999999999998},"last":0.874},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.88700000000000001,"15min":0.89300000000000002},"min":{"1min":0.47199999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":3.2290000000000001,"5min":3.2320000000000002,"15min":3.2320000000000002},"last":0.64000000000000001}]},{"osd":4,"last update":"Sat Sep 21 15:00:05 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.89400000000000002,"15min":0.91600000000000004},"min":{"1min":0.38600000000000001,"5min":0.309,"15min":0.309},"max":{"1min":1.1950000000000001,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":1.3320000000000001},{"interface":"front","average":{"1min":1.0409999999999999,"5min":0.93999999999999995,"15min":0.92300000000000004},"min":{"1min":0.54200000000000004,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":4.9409999999999998,"5min":4.9409999999999998,"15min":4.9409999999999998},"last":1.024}]},{"osd":5,"last update":"Sat Sep 21 15:00:05 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.90200000000000002,"15min":0.90100000000000002},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":1.282},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.95899999999999996,"15min":0.96099999999999997},"min":{"1min":0.629,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.72699999999999998}]},{"osd":6,"last update":"Sat Sep 21 15:00:05 2024","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":1.026,"15min":1.054},"min":{"1min":0.42199999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.3959999999999999,"5min":2.3519999999999999,"15min":2.3519999999999999},"last":1.2190000000000001},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.94499999999999995,"15min":0.94699999999999995},"min":{"1min":0.58799999999999997,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.494,"5min":1.494,"15min":1.494},"last":1.085}]},{"osd":7,"last update":"Sat Sep 21 15:00:05 2024","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":1.0429999999999999,"15min":1.0629999999999999},"min":{"1min":0.48999999999999999,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.4590000000000001,"5min":2.2360000000000002,"15min":2.2360000000000002},"last":1.403},{"interface":"front","average":{"1min":0.96399999999999997,"5min":1.028,"15min":1.0389999999999999},"min":{"1min":0.66500000000000004,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.5369999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":1.1200000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542174,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57752,"kb_used_data":8588,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94314088,"statfs":{"total":96636764160,"available":96577626112,"internally_reserved":0,"allocated":8794112,"data_stored":65370506,"data_compressed":1439126,"data_compressed_allocated":5795840,"data_compressed_original":63827651,"omap_allocated":1593,"internal_metadata":50330055},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 21 15:00:04 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.69699999999999995,"15min":0.72499999999999998},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.97599999999999998,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.62,"15min":0.60699999999999998},"min":{"1min":0.42399999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.74199999999999999}]},{"osd":2,"last update":"Sat Sep 21 15:00:04 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.61399999999999999,"15min":0.60599999999999998},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.82999999999999996,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.74199999999999999,"15min":0.76400000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.94699999999999995,"5min":1.204,"15min":1.204},"last":0.60899999999999999}]},{"osd":3,"last update":"Sat Sep 21 15:00:04 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.747,"15min":0.748},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":1.0189999999999999},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.79500000000000004,"15min":0.81799999999999995},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.95699999999999996,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.52900000000000003}]},{"osd":4,"last update":"Sat Sep 21 15:00:04 2024","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":1.036,"15min":1.0549999999999999},"min":{"1min":0.56100000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.7,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.99299999999999999,"5min":1.054,"15min":1.0640000000000001},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.373,"5min":1.665,"15min":1.665},"last":0.94699999999999995}]},{"osd":5,"last update":"Sat Sep 21 15:00:04 2024","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":1.012,"15min":1.028},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.204,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":1.069},{"interface":"front","average":{"1min":0.875,"5min":1.05,"15min":1.079},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.319,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":1.115}]},{"osd":6,"last update":"Sat Sep 21 15:00:04 2024","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":1.0980000000000001,"15min":1.1259999999999999},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.621,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.99399999999999999,"5min":1.0549999999999999,"15min":1.0649999999999999},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.321,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.98499999999999999}]},{"osd":7,"last update":"Sat Sep 21 15:00:04 2024","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":1.052,"15min":1.0640000000000001},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.2769999999999999,"5min":1.323,"15min":1.323},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.94699999999999995,"5min":1.1120000000000001,"15min":1.1399999999999999},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.6719999999999999,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":0.92100000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5202,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5202,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":7102464,"data_stored":63161454,"data_compressed":1438334,"data_compressed_allocated":5705728,"data_compressed_original":62385859,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8007680,"data_stored":75301715,"data_compressed":1436211,"data_compressed_allocated":6483968,"data_compressed_original":74450392,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2859008,"data_stored":20255319,"data_compressed":752968,"data_compressed_allocated":2179072,"data_compressed_original":19878559,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8937472,"data_stored":78421784,"data_compressed":1695768,"data_compressed_allocated":6995968,"data_compressed_original":77301574,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":5869568,"data_stored":56618690,"data_compressed":1256572,"data_compressed_allocated":4952064,"data_compressed_original":56100759,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":7606272,"data_stored":68567105,"data_compressed":1351355,"data_compressed_allocated":6012928,"data_compressed_original":67680228,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":7290880,"data_stored":65873047,"data_compressed":1533890,"data_compressed_allocated":5918720,"data_compressed_original":65098479,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4476928,"data_stored":41453054,"data_compressed":876973,"data_compressed_allocated":3641344,"data_compressed_original":41001211,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T15:00:22.235 INFO:tasks.thrashosds.thrasher:Setting 3.a to [1, 0] 2024-09-21T15:00:22.235 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.a', '1', '0'] 2024-09-21T15:00:22.235 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.a 1 0 2024-09-21T15:00:22.258 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:22.258 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:22.269 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T15:00:22.320 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:00:22.314+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:00:22.375 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:22.375 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:22.386 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T15:00:22.427 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:00:22.422+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:00:22.522 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:00:22.514+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:00:22.537 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:22.537 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:22.546 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T15:00:22.622 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:00:22.620+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:00:22.657 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:22.657 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:22.666 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T15:00:22.723 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:00:22.718+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:00:22.771 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:22.771 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:22.781 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T15:00:22.824 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:00:22.818+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:00:22.924 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:00:22.918+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:00:22.932 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:22.932 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:22.942 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T15:00:23.025 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:00:23.024+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:00:23.048 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:23.048 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:23.058 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T15:00:23.126 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:00:23.118+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:00:23.176 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:23.176 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:23.186 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T15:00:23.227 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:00:23.222+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:00:23.260 INFO:teuthology.orchestra.run.smithi032.stderr:set 3.a pg_upmap mapping to [1,0] 2024-09-21T15:00:23.264 INFO:teuthology.orchestra.run.smithi032.stderr:noscrub is unset 2024-09-21T15:00:23.311 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:23.312 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:23.322 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T15:00:23.328 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:00:23.322+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:00:23.418 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:23.418 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:23.428 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T15:00:23.429 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:00:23.428+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:00:23.530 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:00:23.522+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:00:23.533 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:23.533 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:23.542 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T15:00:23.543 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T15:00:23.631 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:00:23.630+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:00:23.648 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:23.648 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:23.659 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T15:00:23.678 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T15:00:23.704 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T15:00:23.723 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T15:00:23.732 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:00:23.728+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:00:23.744 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T15:00:23.768 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:23.768 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T15:00:23.768 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T15:00:23.768 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T15:00:23.768 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.768 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T15:00:23.768 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.205754+0000", 2024-09-21T15:00:23.768 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 140.56188303499999, 2024-09-21T15:00:23.768 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.41117094199999998, 2024-09-21T15:00:23.768 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:23.769 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:23.769 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:23.769 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T15:00:23.769 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T15:00:23.769 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1 2024-09-21T15:00:23.769 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.769 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:23.769 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.769 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:23.769 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205754+0000", 2024-09-21T15:00:23.769 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:23.769 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.769 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.770 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:23.770 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205754+0000", 2024-09-21T15:00:23.770 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:23.770 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.770 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.770 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:23.770 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205752+0000", 2024-09-21T15:00:23.770 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T15:00:23.770 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.770 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.770 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:23.770 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205765+0000", 2024-09-21T15:00:23.770 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3230999999999999e-05 2024-09-21T15:00:23.770 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.771 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.771 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:23.771 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205782+0000", 2024-09-21T15:00:23.771 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6634e-05 2024-09-21T15:00:23.771 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.771 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.771 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:23.771 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205793+0000", 2024-09-21T15:00:23.771 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1297e-05 2024-09-21T15:00:23.771 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.771 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.771 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:23.771 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205867+0000", 2024-09-21T15:00:23.772 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4269000000000003e-05 2024-09-21T15:00:23.772 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.772 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.772 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:00:23.772 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205877+0000", 2024-09-21T15:00:23.772 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2690000000000005e-06 2024-09-21T15:00:23.772 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.772 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.772 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:23.772 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.613700+0000", 2024-09-21T15:00:23.772 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.407823455 2024-09-21T15:00:23.772 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.772 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.772 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:23.772 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.614203+0000", 2024-09-21T15:00:23.773 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00050311500000000005 2024-09-21T15:00:23.773 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.773 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.773 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T15:00:23.773 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.614750+0000", 2024-09-21T15:00:23.773 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00054738699999999998 2024-09-21T15:00:23.773 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.773 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.773 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:23.773 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616367+0000", 2024-09-21T15:00:23.773 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001616634 2024-09-21T15:00:23.773 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.774 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.774 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:23.774 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616574+0000", 2024-09-21T15:00:23.774 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00020668499999999999 2024-09-21T15:00:23.774 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.774 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.774 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:23.774 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616802+0000", 2024-09-21T15:00:23.774 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00022817900000000001 2024-09-21T15:00:23.774 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.774 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.774 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:23.774 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616925+0000", 2024-09-21T15:00:23.775 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012309699999999999 2024-09-21T15:00:23.775 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:23.775 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:23.775 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:23.775 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.775 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.775 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T15:00:23.775 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:04.493967+0000", 2024-09-21T15:00:23.775 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 139.27367064800001, 2024-09-21T15:00:23.775 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.015281075999999999, 2024-09-21T15:00:23.775 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:23.775 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:23.775 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:23.776 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T15:00:23.776 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T15:00:23.776 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 141 2024-09-21T15:00:23.776 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.776 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:23.776 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.776 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:23.776 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493967+0000", 2024-09-21T15:00:23.776 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:23.776 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.776 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.776 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:23.776 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493967+0000", 2024-09-21T15:00:23.776 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:23.777 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.777 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.777 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:23.777 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493965+0000", 2024-09-21T15:00:23.777 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T15:00:23.777 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.777 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.777 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:23.777 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.493987+0000", 2024-09-21T15:00:23.777 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1662999999999999e-05 2024-09-21T15:00:23.777 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.777 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.777 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:23.778 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.494004+0000", 2024-09-21T15:00:23.778 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7311000000000001e-05 2024-09-21T15:00:23.778 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.778 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.778 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:23.778 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.494017+0000", 2024-09-21T15:00:23.778 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3202e-05 2024-09-21T15:00:23.778 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.778 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.778 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:23.778 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507678+0000", 2024-09-21T15:00:23.778 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013660432 2024-09-21T15:00:23.778 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.779 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.779 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:23.779 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507726+0000", 2024-09-21T15:00:23.779 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8154e-05 2024-09-21T15:00:23.779 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.779 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.779 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T15:00:23.779 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.507797+0000", 2024-09-21T15:00:23.779 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.1110000000000002e-05 2024-09-21T15:00:23.779 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.779 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.779 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:23.779 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.508710+0000", 2024-09-21T15:00:23.779 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00091339299999999997 2024-09-21T15:00:23.780 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.780 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.780 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:23.780 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.509108+0000", 2024-09-21T15:00:23.780 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00039718300000000001 2024-09-21T15:00:23.780 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.780 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.780 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:23.780 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.509186+0000", 2024-09-21T15:00:23.780 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8592999999999998e-05 2024-09-21T15:00:23.780 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.780 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.780 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:23.780 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:04.509248+0000", 2024-09-21T15:00:23.781 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1450000000000003e-05 2024-09-21T15:00:23.781 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:23.781 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:23.781 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:23.781 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.781 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.781 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6639 3.1a 3:592d80ed:::rbd_data.11361bed1f78.0000000000000005:d [read 0~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T15:00:23.781 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131650+0000", 2024-09-21T15:00:23.781 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 108.63598748699999, 2024-09-21T15:00:23.781 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016002832000000002, 2024-09-21T15:00:23.781 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:23.781 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:23.781 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:23.781 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:23.781 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:23.782 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6639 2024-09-21T15:00:23.782 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.782 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:23.782 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.782 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:23.782 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131650+0000", 2024-09-21T15:00:23.782 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:23.782 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.782 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.782 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:23.782 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131650+0000", 2024-09-21T15:00:23.782 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:23.782 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.782 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.783 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:23.783 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131649+0000", 2024-09-21T15:00:23.783 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:23.783 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.783 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.783 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:23.783 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131651+0000", 2024-09-21T15:00:23.783 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9199999999999998e-06 2024-09-21T15:00:23.783 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.783 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.783 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:23.783 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131656+0000", 2024-09-21T15:00:23.783 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9899999999999997e-06 2024-09-21T15:00:23.784 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.784 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.784 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:23.784 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131665+0000", 2024-09-21T15:00:23.784 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0999999999999993e-06 2024-09-21T15:00:23.784 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.784 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.784 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:23.784 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146309+0000", 2024-09-21T15:00:23.784 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014643893999999999 2024-09-21T15:00:23.784 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.784 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.784 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:23.784 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146392+0000", 2024-09-21T15:00:23.784 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2531e-05 2024-09-21T15:00:23.785 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.785 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.785 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:23.785 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147653+0000", 2024-09-21T15:00:23.785 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0012606710000000001 2024-09-21T15:00:23.785 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:23.785 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:23.785 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:23.785 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.785 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.785 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6640 3.1a 3:592d80ed:::rbd_data.11361bed1f78.0000000000000005:d [read 65536~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T15:00:23.785 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131670+0000", 2024-09-21T15:00:23.785 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 108.635967188, 2024-09-21T15:00:23.785 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016188859, 2024-09-21T15:00:23.785 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:23.785 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:23.786 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:23.786 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:23.786 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:23.786 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6640 2024-09-21T15:00:23.786 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.786 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:23.786 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.786 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:23.786 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131670+0000", 2024-09-21T15:00:23.786 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:23.786 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.786 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.786 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:23.786 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131670+0000", 2024-09-21T15:00:23.787 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:23.787 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.787 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.787 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:23.787 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131670+0000", 2024-09-21T15:00:23.787 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:00:23.787 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.787 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.787 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:23.787 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131672+0000", 2024-09-21T15:00:23.787 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.818e-06 2024-09-21T15:00:23.787 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.787 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.787 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:23.787 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131675+0000", 2024-09-21T15:00:23.787 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5930000000000002e-06 2024-09-21T15:00:23.788 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.788 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.788 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:23.788 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131678+0000", 2024-09-21T15:00:23.788 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6790000000000001e-06 2024-09-21T15:00:23.788 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.788 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.788 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:23.788 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147681+0000", 2024-09-21T15:00:23.788 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016002755 2024-09-21T15:00:23.788 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.788 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.788 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:23.788 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147762+0000", 2024-09-21T15:00:23.789 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1052999999999998e-05 2024-09-21T15:00:23.789 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.789 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.789 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:23.789 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147859+0000", 2024-09-21T15:00:23.789 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.7199000000000002e-05 2024-09-21T15:00:23.789 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:23.789 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:23.789 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:23.789 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.789 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.789 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6641 3.1a 3:592d80ed:::rbd_data.11361bed1f78.0000000000000005:d [read 147456~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T15:00:23.789 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131680+0000", 2024-09-21T15:00:23.789 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 108.6359569, 2024-09-21T15:00:23.789 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016431296000000001, 2024-09-21T15:00:23.789 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:23.790 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:23.790 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:23.790 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:23.790 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:23.790 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6641 2024-09-21T15:00:23.790 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.790 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:23.790 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.790 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:23.790 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131680+0000", 2024-09-21T15:00:23.790 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:23.790 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.790 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.790 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:23.791 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131680+0000", 2024-09-21T15:00:23.791 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:23.791 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.791 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.791 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:23.791 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131680+0000", 2024-09-21T15:00:23.791 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:00:23.791 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.791 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.791 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:23.791 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131681+0000", 2024-09-21T15:00:23.791 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2610000000000001e-06 2024-09-21T15:00:23.791 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.791 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.791 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:23.792 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131685+0000", 2024-09-21T15:00:23.792 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.8299999999999998e-06 2024-09-21T15:00:23.792 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.792 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.792 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:23.792 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131688+0000", 2024-09-21T15:00:23.792 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6189999999999998e-06 2024-09-21T15:00:23.792 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.792 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.792 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:23.792 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147884+0000", 2024-09-21T15:00:23.792 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016196465 2024-09-21T15:00:23.792 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.792 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.793 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:23.793 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147949+0000", 2024-09-21T15:00:23.793 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4867000000000001e-05 2024-09-21T15:00:23.793 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.793 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.793 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:23.793 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.148112+0000", 2024-09-21T15:00:23.793 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00016247700000000001 2024-09-21T15:00:23.793 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:23.793 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:23.793 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:23.793 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.793 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.793 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6642 3.1a 3:592d80ed:::rbd_data.11361bed1f78.0000000000000005:d [read 212992~32768 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T15:00:23.793 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131690+0000", 2024-09-21T15:00:23.794 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 108.635947022, 2024-09-21T15:00:23.794 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016751335999999999, 2024-09-21T15:00:23.794 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:23.794 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:23.794 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:23.794 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:23.794 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:23.794 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6642 2024-09-21T15:00:23.794 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.794 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:23.794 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.794 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:23.794 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131690+0000", 2024-09-21T15:00:23.794 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:23.795 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.795 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.795 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:23.795 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131690+0000", 2024-09-21T15:00:23.795 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:23.795 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.795 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.795 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:23.795 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131690+0000", 2024-09-21T15:00:23.795 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:00:23.795 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.795 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.795 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:23.796 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131691+0000", 2024-09-21T15:00:23.796 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3120000000000001e-06 2024-09-21T15:00:23.796 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.796 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.796 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:23.796 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131694+0000", 2024-09-21T15:00:23.796 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0070000000000002e-06 2024-09-21T15:00:23.796 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.796 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.796 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:23.796 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131700+0000", 2024-09-21T15:00:23.796 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0800000000000002e-06 2024-09-21T15:00:23.796 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.796 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.796 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:23.797 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.148135+0000", 2024-09-21T15:00:23.797 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016434250000000001 2024-09-21T15:00:23.797 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.797 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.797 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:23.797 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.148204+0000", 2024-09-21T15:00:23.797 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9466000000000002e-05 2024-09-21T15:00:23.797 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.797 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.797 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:23.797 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.148441+0000", 2024-09-21T15:00:23.797 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00023743700000000001 2024-09-21T15:00:23.797 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:23.798 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:23.798 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:23.798 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.798 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.798 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:61214 3.7 3:e28aec10:::rbd_header.1136862f872f:head [watch reconnect cookie 139864671932080 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-21T15:00:23.798 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:18.147172+0000", 2024-09-21T15:00:23.798 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 65.620465392, 2024-09-21T15:00:23.798 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.019264956, 2024-09-21T15:00:23.798 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:23.798 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:23.798 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:23.798 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:23.798 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:23.799 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 61214 2024-09-21T15:00:23.799 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.799 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:23.799 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.799 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:23.799 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147172+0000", 2024-09-21T15:00:23.799 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:23.799 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.799 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.799 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:23.799 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147172+0000", 2024-09-21T15:00:23.799 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:23.799 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.800 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.800 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:23.800 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147168+0000", 2024-09-21T15:00:23.800 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999962 2024-09-21T15:00:23.800 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.800 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.800 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:23.800 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147188+0000", 2024-09-21T15:00:23.800 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9717000000000001e-05 2024-09-21T15:00:23.800 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.800 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.800 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:23.800 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147239+0000", 2024-09-21T15:00:23.800 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1493999999999999e-05 2024-09-21T15:00:23.800 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.800 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.801 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:23.801 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147265+0000", 2024-09-21T15:00:23.801 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5746999999999999e-05 2024-09-21T15:00:23.801 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.801 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.801 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:23.801 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147372+0000", 2024-09-21T15:00:23.801 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010681200000000001 2024-09-21T15:00:23.801 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.801 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.801 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op must wait for map", 2024-09-21T15:00:23.801 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147380+0000", 2024-09-21T15:00:23.801 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7919999999999993e-06 2024-09-21T15:00:23.801 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.802 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.802 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:23.802 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.154025+0000", 2024-09-21T15:00:23.802 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0066454909999999999 2024-09-21T15:00:23.802 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.802 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.802 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:00:23.802 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.154041+0000", 2024-09-21T15:00:23.802 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5894000000000001e-05 2024-09-21T15:00:23.802 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.802 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.802 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:23.802 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.164345+0000", 2024-09-21T15:00:23.802 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0103040459999999 2024-09-21T15:00:23.802 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.802 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.803 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:23.803 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.164590+0000", 2024-09-21T15:00:23.803 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00024492699999999998 2024-09-21T15:00:23.803 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.803 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.803 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:23.803 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.166437+0000", 2024-09-21T15:00:23.803 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0018466330000000001 2024-09-21T15:00:23.803 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:23.803 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:23.803 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:23.803 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.803 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.803 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:61215 3.7 3:e28aec10:::rbd_header.1136862f872f:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-21T15:00:23.803 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:18.152008+0000", 2024-09-21T15:00:23.804 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 65.615628939999993, 2024-09-21T15:00:23.804 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.015644958, 2024-09-21T15:00:23.804 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:23.804 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:23.804 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:23.804 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:23.804 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:23.804 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 61215 2024-09-21T15:00:23.804 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.804 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:23.804 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.804 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:23.804 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152008+0000", 2024-09-21T15:00:23.804 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:23.804 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.804 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.804 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:23.805 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152008+0000", 2024-09-21T15:00:23.805 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:23.805 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.805 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.805 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:23.805 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152005+0000", 2024-09-21T15:00:23.805 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-21T15:00:23.805 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.805 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.805 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:23.805 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152025+0000", 2024-09-21T15:00:23.805 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9306000000000001e-05 2024-09-21T15:00:23.805 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.805 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.805 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:23.806 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152054+0000", 2024-09-21T15:00:23.806 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9189000000000001e-05 2024-09-21T15:00:23.806 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.806 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.806 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:23.806 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152078+0000", 2024-09-21T15:00:23.806 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4317000000000001e-05 2024-09-21T15:00:23.806 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.806 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.806 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:23.806 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152243+0000", 2024-09-21T15:00:23.806 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000164802 2024-09-21T15:00:23.806 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.806 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.806 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting_for_map not empty", 2024-09-21T15:00:23.806 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152251+0000", 2024-09-21T15:00:23.807 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0600000000000008e-06 2024-09-21T15:00:23.807 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.807 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.807 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:23.807 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.154741+0000", 2024-09-21T15:00:23.807 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0024900069999999998 2024-09-21T15:00:23.807 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.807 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.807 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:00:23.807 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.154753+0000", 2024-09-21T15:00:23.807 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1588e-05 2024-09-21T15:00:23.807 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.807 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.807 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:23.807 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.165380+0000", 2024-09-21T15:00:23.807 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0106272249999999 2024-09-21T15:00:23.808 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.808 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.808 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:23.808 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.165517+0000", 2024-09-21T15:00:23.808 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013737999999999999 2024-09-21T15:00:23.808 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.808 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.808 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 6", 2024-09-21T15:00:23.808 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.165938+0000", 2024-09-21T15:00:23.808 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00042082399999999998 2024-09-21T15:00:23.808 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.808 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.808 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:23.808 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.166795+0000", 2024-09-21T15:00:23.808 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00085674899999999997 2024-09-21T15:00:23.809 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.809 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.809 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:23.809 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.167598+0000", 2024-09-21T15:00:23.809 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00080335599999999997 2024-09-21T15:00:23.809 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.809 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.809 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:23.809 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.167632+0000", 2024-09-21T15:00:23.809 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4122000000000003e-05 2024-09-21T15:00:23.809 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.809 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.809 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:23.809 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.167653+0000", 2024-09-21T15:00:23.809 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0862000000000001e-05 2024-09-21T15:00:23.810 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:23.810 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:23.810 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:23.810 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.810 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.810 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:73698 3.10 3:0a4d95f6:::rbd_data.1136f4939602.000000000000000b:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-21T15:00:23.810 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:41.969870+0000", 2024-09-21T15:00:23.810 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 41.797766801999998, 2024-09-21T15:00:23.810 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.017502461, 2024-09-21T15:00:23.810 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:23.810 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:23.810 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:23.810 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:23.810 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:23.810 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 73698 2024-09-21T15:00:23.810 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.811 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:23.811 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.811 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:23.811 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.969870+0000", 2024-09-21T15:00:23.811 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:23.811 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.811 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.811 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:23.811 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.969870+0000", 2024-09-21T15:00:23.811 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:23.811 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.811 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.811 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:23.811 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.969868+0000", 2024-09-21T15:00:23.811 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T15:00:23.811 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.812 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.812 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:23.812 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.969885+0000", 2024-09-21T15:00:23.812 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7416999999999999e-05 2024-09-21T15:00:23.812 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.812 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.812 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:23.812 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.969918+0000", 2024-09-21T15:00:23.812 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3133000000000003e-05 2024-09-21T15:00:23.812 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.812 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.812 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:23.812 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.969934+0000", 2024-09-21T15:00:23.812 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6027000000000001e-05 2024-09-21T15:00:23.812 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.813 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.813 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:23.813 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.970021+0000", 2024-09-21T15:00:23.813 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6907e-05 2024-09-21T15:00:23.813 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.813 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.813 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:23.813 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.970148+0000", 2024-09-21T15:00:23.813 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012701499999999999 2024-09-21T15:00:23.813 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.813 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.813 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T15:00:23.813 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.972971+0000", 2024-09-21T15:00:23.813 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.002822873 2024-09-21T15:00:23.813 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.813 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.814 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:23.814 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.987124+0000", 2024-09-21T15:00:23.814 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0141529 2024-09-21T15:00:23.814 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.814 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.814 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:23.814 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.987146+0000", 2024-09-21T15:00:23.814 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1783000000000001e-05 2024-09-21T15:00:23.814 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.814 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.814 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:23.814 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.987258+0000", 2024-09-21T15:00:23.814 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000112555 2024-09-21T15:00:23.814 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.814 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.814 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:23.815 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.987373+0000", 2024-09-21T15:00:23.815 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000114434 2024-09-21T15:00:23.815 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:23.815 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:23.815 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:23.815 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.815 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.815 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:97638 3.1a 3:5e573525:::rbd_data.1136b9c49b1c.0000000000000026:49 [read 1277952~49152 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:23.815 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.276202+0000", 2024-09-21T15:00:23.815 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.491434696999999, 2024-09-21T15:00:23.815 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.015360357999999999, 2024-09-21T15:00:23.815 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:23.815 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:23.815 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:23.815 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:23.816 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:23.816 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 97638 2024-09-21T15:00:23.816 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.816 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:23.816 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.816 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:23.816 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276202+0000", 2024-09-21T15:00:23.816 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:23.816 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.816 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.816 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:23.816 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276202+0000", 2024-09-21T15:00:23.816 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:23.816 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.816 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.817 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:23.817 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276202+0000", 2024-09-21T15:00:23.817 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:23.817 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.817 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.817 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:23.817 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276204+0000", 2024-09-21T15:00:23.817 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5359999999999999e-06 2024-09-21T15:00:23.817 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.817 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.817 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:23.817 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276210+0000", 2024-09-21T15:00:23.817 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3040000000000004e-06 2024-09-21T15:00:23.817 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.818 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.818 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:23.818 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276214+0000", 2024-09-21T15:00:23.818 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6779999999999999e-06 2024-09-21T15:00:23.818 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.818 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.818 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:23.818 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.290563+0000", 2024-09-21T15:00:23.818 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014348346 2024-09-21T15:00:23.818 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.818 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.818 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:23.818 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.290623+0000", 2024-09-21T15:00:23.818 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0001999999999999e-05 2024-09-21T15:00:23.819 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.819 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.819 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:23.819 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.291563+0000", 2024-09-21T15:00:23.819 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00094003100000000003 2024-09-21T15:00:23.819 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:23.819 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:23.819 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:23.819 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.819 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.819 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:97678 3.d e122/24 3:b2f34c6d:::rbd_data.1136bfc79a70.0000000000000006:head v 122'234, mlcod=120'232)", 2024-09-21T15:00:23.819 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.321655+0000", 2024-09-21T15:00:23.819 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.445982022999999, 2024-09-21T15:00:23.819 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024509388999999999, 2024-09-21T15:00:23.820 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:23.820 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:23.820 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:23.820 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.820 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:23.820 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.321655+0000", 2024-09-21T15:00:23.820 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:23.820 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.820 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.820 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:23.820 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.321655+0000", 2024-09-21T15:00:23.820 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:23.820 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.820 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.820 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:23.821 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.321655+0000", 2024-09-21T15:00:23.821 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:23.821 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.821 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.821 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:23.821 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.329453+0000", 2024-09-21T15:00:23.821 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0077980560000000003 2024-09-21T15:00:23.821 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.821 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.821 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:23.821 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.329482+0000", 2024-09-21T15:00:23.821 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9153999999999999e-05 2024-09-21T15:00:23.821 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.821 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.822 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:23.822 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.329493+0000", 2024-09-21T15:00:23.822 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1103e-05 2024-09-21T15:00:23.822 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.822 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.822 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:23.822 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.329555+0000", 2024-09-21T15:00:23.822 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2345e-05 2024-09-21T15:00:23.822 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.822 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.822 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:23.822 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.329589+0000", 2024-09-21T15:00:23.822 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3908999999999997e-05 2024-09-21T15:00:23.822 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.822 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.822 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:23.823 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.346083+0000", 2024-09-21T15:00:23.823 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016493383 2024-09-21T15:00:23.823 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.823 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.823 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:23.823 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.346165+0000", 2024-09-21T15:00:23.823 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1796000000000001e-05 2024-09-21T15:00:23.823 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:23.823 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:23.823 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:23.823 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.823 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.823 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:97679 3.f e122/24 3:f2a362b4:::rbd_data.1136bfc79a70.0000000000000007:head v 122'331, mlcod=120'329)", 2024-09-21T15:00:23.823 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.337145+0000", 2024-09-21T15:00:23.823 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.430492013999999, 2024-09-21T15:00:23.824 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021912411, 2024-09-21T15:00:23.824 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:23.824 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:23.824 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:23.824 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.824 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:23.824 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.337145+0000", 2024-09-21T15:00:23.824 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:23.824 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.824 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.824 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:23.824 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.337145+0000", 2024-09-21T15:00:23.824 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:23.824 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.825 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.825 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:23.825 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.337145+0000", 2024-09-21T15:00:23.825 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:23.825 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.825 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.825 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:23.825 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.343899+0000", 2024-09-21T15:00:23.825 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0067541420000000003 2024-09-21T15:00:23.825 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.825 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.825 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:23.825 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.343921+0000", 2024-09-21T15:00:23.825 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2255e-05 2024-09-21T15:00:23.825 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.825 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.826 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:23.826 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.343933+0000", 2024-09-21T15:00:23.826 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1528e-05 2024-09-21T15:00:23.826 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.826 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.826 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:23.826 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.344001+0000", 2024-09-21T15:00:23.826 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8269999999999995e-05 2024-09-21T15:00:23.826 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.826 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.826 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:23.826 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.344029+0000", 2024-09-21T15:00:23.826 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8254000000000001e-05 2024-09-21T15:00:23.826 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.826 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.827 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:23.827 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.358901+0000", 2024-09-21T15:00:23.827 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0148719 2024-09-21T15:00:23.827 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.827 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.827 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:23.827 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.359058+0000", 2024-09-21T15:00:23.827 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000156446 2024-09-21T15:00:23.827 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:23.827 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:23.827 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:23.827 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.827 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.827 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:97676 3.1f e122/24 3:ffaa836a:::rbd_data.1136bfc79a70.0000000000000004:head v 122'312, mlcod=120'310)", 2024-09-21T15:00:23.827 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.348683+0000", 2024-09-21T15:00:23.828 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.418953849000001, 2024-09-21T15:00:23.828 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.033768802000000001, 2024-09-21T15:00:23.828 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:23.828 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:23.828 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:23.828 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.828 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:23.828 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.348683+0000", 2024-09-21T15:00:23.828 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:23.828 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.828 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.828 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:23.828 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.348683+0000", 2024-09-21T15:00:23.828 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:23.828 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.828 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.829 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:23.829 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.348683+0000", 2024-09-21T15:00:23.829 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:00:23.829 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.829 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.829 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:23.829 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.366937+0000", 2024-09-21T15:00:23.829 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018254415 2024-09-21T15:00:23.829 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.829 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.829 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:23.829 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.366959+0000", 2024-09-21T15:00:23.829 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1634e-05 2024-09-21T15:00:23.829 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.829 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.830 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:23.830 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.366971+0000", 2024-09-21T15:00:23.830 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2305e-05 2024-09-21T15:00:23.830 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.830 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.830 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:23.830 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.367034+0000", 2024-09-21T15:00:23.830 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3211000000000002e-05 2024-09-21T15:00:23.830 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.830 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.830 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:23.830 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.367056+0000", 2024-09-21T15:00:23.830 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2098000000000001e-05 2024-09-21T15:00:23.830 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.830 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.830 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:23.831 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.382400+0000", 2024-09-21T15:00:23.831 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.015343867000000001 2024-09-21T15:00:23.831 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.831 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.831 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:23.831 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.382452+0000", 2024-09-21T15:00:23.831 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1992e-05 2024-09-21T15:00:23.831 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:23.831 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:23.831 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:23.831 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.831 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.831 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:97734 3.19 e122/94 3:9fcf754f:::rbd_data.1136bfc79a70.0000000000000009:head v 122'314, mlcod=120'312)", 2024-09-21T15:00:23.831 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.378547+0000", 2024-09-21T15:00:23.831 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.389089870999999, 2024-09-21T15:00:23.831 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.063380675999999997, 2024-09-21T15:00:23.832 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:23.832 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:23.832 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:23.832 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.832 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:23.832 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.378547+0000", 2024-09-21T15:00:23.832 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:23.832 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.832 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.832 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:23.832 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.378547+0000", 2024-09-21T15:00:23.832 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:23.832 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.832 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.832 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:23.832 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.378547+0000", 2024-09-21T15:00:23.833 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:00:23.833 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.833 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.833 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:23.833 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.425118+0000", 2024-09-21T15:00:23.833 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046571166999999997 2024-09-21T15:00:23.833 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.833 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.833 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:23.833 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.425150+0000", 2024-09-21T15:00:23.833 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2150999999999998e-05 2024-09-21T15:00:23.833 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.833 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.833 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:23.833 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.425164+0000", 2024-09-21T15:00:23.833 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3302000000000001e-05 2024-09-21T15:00:23.834 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.834 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.834 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:23.834 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.425231+0000", 2024-09-21T15:00:23.834 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7106000000000004e-05 2024-09-21T15:00:23.834 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.834 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.834 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:23.834 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.425259+0000", 2024-09-21T15:00:23.834 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8087000000000001e-05 2024-09-21T15:00:23.834 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.834 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.834 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:23.834 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.441890+0000", 2024-09-21T15:00:23.835 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016631328000000001 2024-09-21T15:00:23.835 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.835 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.835 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:23.835 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.441928+0000", 2024-09-21T15:00:23.835 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7701999999999999e-05 2024-09-21T15:00:23.835 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:23.835 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:23.835 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:23.835 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.835 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.835 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:100187 3.10 3:08941a97:::rbd_data.1136ab4d6ff1.000000000000000e:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-21T15:00:23.835 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:10.869845+0000", 2024-09-21T15:00:23.835 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 12.897791975000001, 2024-09-21T15:00:23.835 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029345237, 2024-09-21T15:00:23.835 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:23.836 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:23.836 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:23.836 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:23.836 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:23.836 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 100187 2024-09-21T15:00:23.836 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.836 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:23.836 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.836 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:23.836 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869845+0000", 2024-09-21T15:00:23.836 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:23.836 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.836 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.836 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:23.836 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869845+0000", 2024-09-21T15:00:23.836 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:23.836 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.837 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.837 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:23.837 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869844+0000", 2024-09-21T15:00:23.837 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T15:00:23.837 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.837 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.837 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:23.837 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869853+0000", 2024-09-21T15:00:23.837 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5719999999999992e-06 2024-09-21T15:00:23.837 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.837 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.837 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:23.837 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869874+0000", 2024-09-21T15:00:23.837 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0483e-05 2024-09-21T15:00:23.837 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.837 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.838 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:23.838 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869883+0000", 2024-09-21T15:00:23.838 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6080000000000002e-06 2024-09-21T15:00:23.838 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.838 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.838 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:23.838 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869991+0000", 2024-09-21T15:00:23.838 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000108013 2024-09-21T15:00:23.838 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.838 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.838 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:23.838 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.870258+0000", 2024-09-21T15:00:23.838 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00026629099999999999 2024-09-21T15:00:23.838 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.839 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.839 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T15:00:23.839 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.878656+0000", 2024-09-21T15:00:23.839 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0083989000000000008 2024-09-21T15:00:23.839 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.839 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.839 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:23.839 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.899081+0000", 2024-09-21T15:00:23.839 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.020424437 2024-09-21T15:00:23.839 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.839 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.839 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:23.839 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.899125+0000", 2024-09-21T15:00:23.839 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4440999999999997e-05 2024-09-21T15:00:23.839 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.840 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.840 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:23.840 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.899160+0000", 2024-09-21T15:00:23.840 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4518999999999999e-05 2024-09-21T15:00:23.840 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.840 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.840 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:23.840 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.899190+0000", 2024-09-21T15:00:23.840 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0574999999999998e-05 2024-09-21T15:00:23.840 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:23.840 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:23.840 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:23.840 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.840 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.840 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:100263 3.10 3:0ece142e:::rbd_data.1136ab4d6ff1.0000000000000008:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-21T15:00:23.840 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:11.052257+0000", 2024-09-21T15:00:23.841 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 12.715379917, 2024-09-21T15:00:23.841 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042678025000000001, 2024-09-21T15:00:23.841 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:23.841 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:23.841 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:23.841 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:23.841 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:23.841 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 100263 2024-09-21T15:00:23.841 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.841 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:23.841 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.841 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:23.841 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052257+0000", 2024-09-21T15:00:23.841 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:23.841 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.841 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.841 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:23.842 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052257+0000", 2024-09-21T15:00:23.842 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:23.842 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.842 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.842 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:23.842 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052256+0000", 2024-09-21T15:00:23.842 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:00:23.842 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.842 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.842 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:23.842 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052263+0000", 2024-09-21T15:00:23.842 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4799999999999998e-06 2024-09-21T15:00:23.842 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.842 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.842 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:23.842 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052275+0000", 2024-09-21T15:00:23.843 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2386e-05 2024-09-21T15:00:23.843 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.843 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.843 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:23.843 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052281+0000", 2024-09-21T15:00:23.843 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6699999999999999e-06 2024-09-21T15:00:23.843 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.843 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.843 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:23.843 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052350+0000", 2024-09-21T15:00:23.843 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9099999999999999e-05 2024-09-21T15:00:23.843 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.843 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.843 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:23.843 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052456+0000", 2024-09-21T15:00:23.843 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000106669 2024-09-21T15:00:23.844 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.844 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.844 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T15:00:23.844 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.059694+0000", 2024-09-21T15:00:23.844 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.007237566 2024-09-21T15:00:23.844 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.844 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.844 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:23.844 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.094721+0000", 2024-09-21T15:00:23.844 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035026977000000001 2024-09-21T15:00:23.844 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.844 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.844 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:23.844 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.094746+0000", 2024-09-21T15:00:23.844 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5329000000000001e-05 2024-09-21T15:00:23.845 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.845 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.845 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:23.845 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.094850+0000", 2024-09-21T15:00:23.845 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000103452 2024-09-21T15:00:23.845 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.845 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.845 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:23.845 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.094935+0000", 2024-09-21T15:00:23.845 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5550000000000006e-05 2024-09-21T15:00:23.845 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:23.845 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:23.845 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:23.845 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.845 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.845 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:102266 3.1f e130/24 3:fec72c47:::rbd_data.1136d175325b.000000000000000c:head v 130'326, mlcod=128'324)", 2024-09-21T15:00:23.846 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:14.319401+0000", 2024-09-21T15:00:23.846 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 9.4482359769999995, 2024-09-21T15:00:23.846 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021068982999999999, 2024-09-21T15:00:23.846 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:23.846 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:23.846 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:23.846 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.846 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:23.846 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.319401+0000", 2024-09-21T15:00:23.846 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:23.846 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.846 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.846 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:23.846 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.319401+0000", 2024-09-21T15:00:23.846 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:23.846 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.847 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.847 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:23.847 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.319400+0000", 2024-09-21T15:00:23.847 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:00:23.847 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.847 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.847 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:23.847 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.337267+0000", 2024-09-21T15:00:23.847 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.017867141 2024-09-21T15:00:23.847 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.847 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.847 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:23.847 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.337278+0000", 2024-09-21T15:00:23.847 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0224000000000001e-05 2024-09-21T15:00:23.848 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.848 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.848 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:23.848 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.337285+0000", 2024-09-21T15:00:23.848 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.25e-06 2024-09-21T15:00:23.848 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.848 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.848 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:23.848 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.337330+0000", 2024-09-21T15:00:23.848 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5210000000000003e-05 2024-09-21T15:00:23.848 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.848 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.848 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:23.848 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.337350+0000", 2024-09-21T15:00:23.848 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9556e-05 2024-09-21T15:00:23.848 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.848 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.849 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:23.849 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.340418+0000", 2024-09-21T15:00:23.849 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.003068552 2024-09-21T15:00:23.849 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.849 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.849 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:23.849 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.340470+0000", 2024-09-21T15:00:23.849 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1897000000000001e-05 2024-09-21T15:00:23.849 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:23.849 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:23.849 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:23.849 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.849 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.849 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:102887 3.1c 3:3b11ae99:::rbd_object_map.1136d175325b.0000000000000050:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-21T15:00:23.849 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:14.368644+0000", 2024-09-21T15:00:23.850 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 9.3989929409999995, 2024-09-21T15:00:23.850 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.017021290000000001, 2024-09-21T15:00:23.850 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:23.850 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:23.850 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:23.850 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:23.850 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:23.850 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 102887 2024-09-21T15:00:23.850 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.850 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:23.850 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.850 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:23.850 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.368644+0000", 2024-09-21T15:00:23.850 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:23.850 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.851 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.851 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:23.851 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.368644+0000", 2024-09-21T15:00:23.851 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:23.851 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.851 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.851 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:23.851 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.368642+0000", 2024-09-21T15:00:23.851 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T15:00:23.851 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.851 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.851 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:23.851 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.368655+0000", 2024-09-21T15:00:23.851 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2808e-05 2024-09-21T15:00:23.851 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.852 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.852 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:23.852 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.368677+0000", 2024-09-21T15:00:23.852 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2753999999999999e-05 2024-09-21T15:00:23.852 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.852 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.852 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:23.852 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.368688+0000", 2024-09-21T15:00:23.852 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1228e-05 2024-09-21T15:00:23.852 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.852 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.852 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:23.852 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.368743+0000", 2024-09-21T15:00:23.852 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4608999999999998e-05 2024-09-21T15:00:23.852 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.853 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.853 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:23.853 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.368868+0000", 2024-09-21T15:00:23.853 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012473000000000001 2024-09-21T15:00:23.853 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.853 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.853 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 0", 2024-09-21T15:00:23.853 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.369385+0000", 2024-09-21T15:00:23.853 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00051684400000000003 2024-09-21T15:00:23.853 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.853 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.853 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:23.853 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.370755+0000", 2024-09-21T15:00:23.853 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0013701609999999999 2024-09-21T15:00:23.853 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.854 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.854 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:23.854 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.385531+0000", 2024-09-21T15:00:23.854 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014776648 2024-09-21T15:00:23.854 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.854 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.854 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:23.854 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.385575+0000", 2024-09-21T15:00:23.854 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3726000000000001e-05 2024-09-21T15:00:23.854 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.854 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.854 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:23.854 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.385666+0000", 2024-09-21T15:00:23.854 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0304999999999997e-05 2024-09-21T15:00:23.855 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:23.855 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:23.855 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:23.855 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.855 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.855 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:103706 3.1c 3:3b11ae99:::rbd_object_map.1136d175325b.0000000000000050:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-21T15:00:23.855 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:14.426370+0000", 2024-09-21T15:00:23.855 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 9.3412676040000004, 2024-09-21T15:00:23.855 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.015478897, 2024-09-21T15:00:23.855 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:23.855 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:23.855 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:23.855 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:23.855 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:23.855 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 103706 2024-09-21T15:00:23.856 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.856 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:23.856 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.856 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:23.856 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.426370+0000", 2024-09-21T15:00:23.856 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:23.856 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.856 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.856 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:23.856 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.426370+0000", 2024-09-21T15:00:23.856 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:23.856 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.856 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.856 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:23.856 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.426369+0000", 2024-09-21T15:00:23.856 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:00:23.857 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.857 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.857 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:23.857 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.426376+0000", 2024-09-21T15:00:23.857 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0999999999999998e-06 2024-09-21T15:00:23.857 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.857 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.857 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:23.857 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.426442+0000", 2024-09-21T15:00:23.857 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6445999999999994e-05 2024-09-21T15:00:23.857 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.857 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.857 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:23.857 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.426451+0000", 2024-09-21T15:00:23.857 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4940000000000003e-06 2024-09-21T15:00:23.857 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.857 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.858 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:23.858 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.426493+0000", 2024-09-21T15:00:23.858 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2094000000000001e-05 2024-09-21T15:00:23.858 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.858 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.858 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:23.858 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.426560+0000", 2024-09-21T15:00:23.858 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6774999999999997e-05 2024-09-21T15:00:23.858 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.858 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.858 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 0", 2024-09-21T15:00:23.858 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.426737+0000", 2024-09-21T15:00:23.858 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017769899999999999 2024-09-21T15:00:23.858 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.859 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.859 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:23.859 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.428561+0000", 2024-09-21T15:00:23.859 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001823917 2024-09-21T15:00:23.859 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.859 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.859 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:23.859 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.441762+0000", 2024-09-21T15:00:23.859 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013200462 2024-09-21T15:00:23.859 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.859 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.859 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:23.859 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.441811+0000", 2024-09-21T15:00:23.859 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8912000000000002e-05 2024-09-21T15:00:23.860 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.860 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.860 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:23.860 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.441848+0000", 2024-09-21T15:00:23.860 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7935000000000002e-05 2024-09-21T15:00:23.860 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:23.860 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:23.860 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:23.860 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.860 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.860 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:105083 3.1a 3:5d63845a:::rbd_data.1136d175325b.0000000000000037:head [write 0~32768 [fadvise_sequential+fadvise_nocache],write 65536~163840 [fadvise_sequential+fadvise_nocache],write 262144~32768 [fadvise_sequential+fadvise_nocache],write 327680~163840 [fadvise_sequential+fadvise_nocache],write 524288~32768 [fadvise_sequential+fadvise_nocache],write 589824~163840 [fadvise_sequential+fadvise_nocache],write 786432~32768 [fadvise_sequential+fadvise_nocache],write 851968~32768 [fadvise_sequential+fadvise_nocache],write 917504~98304 [fadvise_sequential+fadvise_nocache],write 1048576~32768 [fadvise_sequential+fadvise_nocache],write 1114112~32768 [fadvise_sequential+fadvise_nocache],write 1179648~98304 [fadvise_sequential+fadvise_nocache],write 1310720~32768 [fadvise_sequential+fadvise_nocache],write 1376256~32768 [fadvise_sequential+fadvise_nocache],write 1441792~98304 [fadvise_sequential+fadvise_nocache],write 1572864~32768 [fadvise_sequential+fadvise_nocache],write 1638400~32768 [fadvise_sequential+fadvise_nocache],write 1703936~98304 [fadvise_sequential+fadvise_nocache],write 1835008~32768 [fadvise_sequential+fadvise_nocache],write 1900544~32768 [fadvise_sequential+fadvise_nocache],write 1966080~98304 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-21T15:00:23.860 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:14.534201+0000", 2024-09-21T15:00:23.860 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 9.2334362629999998, 2024-09-21T15:00:23.860 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016474194000000001, 2024-09-21T15:00:23.860 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:23.861 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:23.861 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:23.861 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:23.861 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:23.861 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 105083 2024-09-21T15:00:23.861 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.861 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:23.861 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.861 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:23.861 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.534201+0000", 2024-09-21T15:00:23.861 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:23.861 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.861 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.861 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:23.861 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.534201+0000", 2024-09-21T15:00:23.861 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:23.862 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.862 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.862 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:23.862 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.534201+0000", 2024-09-21T15:00:23.862 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:23.862 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.862 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.862 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:23.862 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.534853+0000", 2024-09-21T15:00:23.862 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00065227099999999995 2024-09-21T15:00:23.862 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.862 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.862 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:23.863 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.534866+0000", 2024-09-21T15:00:23.863 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3553000000000001e-05 2024-09-21T15:00:23.863 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.863 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.863 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:23.863 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.534877+0000", 2024-09-21T15:00:23.863 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0186e-05 2024-09-21T15:00:23.863 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.863 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.863 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:23.863 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.534947+0000", 2024-09-21T15:00:23.863 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0417e-05 2024-09-21T15:00:23.863 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.864 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.864 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:23.864 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.535138+0000", 2024-09-21T15:00:23.864 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00019122200000000001 2024-09-21T15:00:23.864 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.864 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.864 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T15:00:23.864 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.535666+0000", 2024-09-21T15:00:23.864 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00052817499999999998 2024-09-21T15:00:23.864 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.864 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.864 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:23.864 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.543252+0000", 2024-09-21T15:00:23.864 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0075857889999999999 2024-09-21T15:00:23.865 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.865 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.865 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:23.865 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.550552+0000", 2024-09-21T15:00:23.865 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0072995869999999997 2024-09-21T15:00:23.865 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.865 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.865 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:23.865 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.550637+0000", 2024-09-21T15:00:23.865 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5750999999999999e-05 2024-09-21T15:00:23.865 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:23.865 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:23.865 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:23.865 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.550675+0000", 2024-09-21T15:00:23.865 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7602999999999999e-05 2024-09-21T15:00:23.865 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:23.865 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:23.866 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:23.866 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:23.866 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:23.866 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:23.867 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T15:00:23.867 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:00:23.864+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:00:23.967 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:00:23.962+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:00:23.988 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-21T15:00:24.016 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T15:00:24.034 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:24.034 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:00:24.034 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T15:00:24.034 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:24.039 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T15:00:24.044 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T15:00:24.068 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:00:24.064+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:00:24.077 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T15:00:24.079 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:00:24.079 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:00:24.079 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:00:24.079 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:00:24.080 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:00:24.080 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:00:24.080 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:00:24.080 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:00:24.152 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:24.152 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:00:24.152 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T15:00:24.152 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T15:00:24.152 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:24.162 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T15:00:24.170 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:00:24.162+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:00:24.270 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:00:24.266+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:00:24.273 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:24.273 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T15:00:24.273 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T15:00:24.273 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T15:00:24.273 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.273 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:15130 3.2 3:47f1bbdb:::rbd_data.11361bed1f78.0000000000000067:head [sparse-read 0~147456] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-21T15:00:24.273 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:40.288284+0000", 2024-09-21T15:00:24.273 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 103.98454818899999, 2024-09-21T15:00:24.274 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1142935039999999, 2024-09-21T15:00:24.274 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:24.274 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:24.274 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:24.274 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:24.274 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:24.274 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 15130 2024-09-21T15:00:24.274 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.274 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:24.274 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.274 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:24.274 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288284+0000", 2024-09-21T15:00:24.274 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:24.274 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.275 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.275 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:24.275 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288284+0000", 2024-09-21T15:00:24.275 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:24.275 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.275 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.275 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:24.275 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288284+0000", 2024-09-21T15:00:24.275 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:24.275 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.275 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.275 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:24.275 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288288+0000", 2024-09-21T15:00:24.275 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5829999999999998e-06 2024-09-21T15:00:24.276 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.276 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.276 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:24.276 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288297+0000", 2024-09-21T15:00:24.276 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0669999999999998e-06 2024-09-21T15:00:24.276 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.276 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.276 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:24.276 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288305+0000", 2024-09-21T15:00:24.276 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8860000000000004e-06 2024-09-21T15:00:24.276 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.276 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.276 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:24.276 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288417+0000", 2024-09-21T15:00:24.276 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000112952 2024-09-21T15:00:24.276 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.277 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.277 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:00:24.277 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288428+0000", 2024-09-21T15:00:24.277 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0631e-05 2024-09-21T15:00:24.277 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.277 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.277 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:24.277 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.401876+0000", 2024-09-21T15:00:24.277 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1134479610000001 2024-09-21T15:00:24.277 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.277 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.277 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:24.277 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.402076+0000", 2024-09-21T15:00:24.277 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00019944200000000001 2024-09-21T15:00:24.277 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.277 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.278 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:24.278 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.402578+0000", 2024-09-21T15:00:24.278 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00050242899999999998 2024-09-21T15:00:24.278 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:24.278 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:24.278 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:24.278 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.278 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.278 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45708 3.2 3:415db91a:::rbd_data.1136ecbe5879.0000000000000000:28 [read 140288~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:24.278 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141801+0000", 2024-09-21T15:00:24.278 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 74.131031995000001, 2024-09-21T15:00:24.278 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034274200999999997, 2024-09-21T15:00:24.278 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:24.278 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:24.278 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:24.279 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:24.279 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:24.279 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45708 2024-09-21T15:00:24.279 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.279 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:24.279 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.279 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:24.279 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141801+0000", 2024-09-21T15:00:24.279 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:24.279 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.279 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.279 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:24.279 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141801+0000", 2024-09-21T15:00:24.279 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:24.279 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.280 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.280 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:24.280 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141800+0000", 2024-09-21T15:00:24.280 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:24.280 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.280 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.280 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:24.280 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141803+0000", 2024-09-21T15:00:24.280 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4210000000000002e-06 2024-09-21T15:00:24.280 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.280 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.280 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:24.280 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141812+0000", 2024-09-21T15:00:24.280 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.9800000000000004e-06 2024-09-21T15:00:24.280 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.280 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.281 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:24.281 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141816+0000", 2024-09-21T15:00:24.281 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4769999999999997e-06 2024-09-21T15:00:24.281 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.281 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.281 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:24.281 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.175865+0000", 2024-09-21T15:00:24.281 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034048875999999999 2024-09-21T15:00:24.281 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.281 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.281 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:24.281 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.175943+0000", 2024-09-21T15:00:24.281 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7464000000000002e-05 2024-09-21T15:00:24.281 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.282 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.282 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:24.282 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176075+0000", 2024-09-21T15:00:24.282 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013230800000000001 2024-09-21T15:00:24.282 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:24.282 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:24.282 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:24.282 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.282 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.282 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45709 3.2 3:415db91a:::rbd_data.1136ecbe5879.0000000000000000:28 [read 143360~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:24.282 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141820+0000", 2024-09-21T15:00:24.282 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 74.131012147999996, 2024-09-21T15:00:24.282 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034537536000000001, 2024-09-21T15:00:24.282 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:24.282 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:24.283 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:24.283 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:24.283 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:24.283 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45709 2024-09-21T15:00:24.283 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.283 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:24.283 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.283 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:24.283 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141820+0000", 2024-09-21T15:00:24.283 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:24.283 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.283 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.283 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:24.283 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141820+0000", 2024-09-21T15:00:24.284 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:24.284 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.284 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.284 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:24.284 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141820+0000", 2024-09-21T15:00:24.284 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:24.284 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.284 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.284 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:24.284 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141823+0000", 2024-09-21T15:00:24.284 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5110000000000002e-06 2024-09-21T15:00:24.284 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.285 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.285 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:24.285 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141829+0000", 2024-09-21T15:00:24.285 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0850000000000002e-06 2024-09-21T15:00:24.285 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.285 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.285 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:24.285 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141833+0000", 2024-09-21T15:00:24.285 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7620000000000003e-06 2024-09-21T15:00:24.285 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.285 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.285 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:24.285 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176103+0000", 2024-09-21T15:00:24.285 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034269933000000002 2024-09-21T15:00:24.285 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.285 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.286 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:24.286 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176161+0000", 2024-09-21T15:00:24.286 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7658999999999998e-05 2024-09-21T15:00:24.286 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.286 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.286 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:24.286 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176358+0000", 2024-09-21T15:00:24.286 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000196937 2024-09-21T15:00:24.286 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:24.286 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:24.286 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:24.286 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.286 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.286 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45710 3.2 3:415db91a:::rbd_data.1136ecbe5879.0000000000000000:28 [read 146432~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:24.287 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141838+0000", 2024-09-21T15:00:24.287 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 74.130994985000001, 2024-09-21T15:00:24.287 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034953674999999997, 2024-09-21T15:00:24.287 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:24.287 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:24.287 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:24.287 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:24.287 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:24.287 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45710 2024-09-21T15:00:24.287 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.287 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:24.287 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.287 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:24.287 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141838+0000", 2024-09-21T15:00:24.287 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:24.288 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.288 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.288 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:24.288 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141838+0000", 2024-09-21T15:00:24.288 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:24.288 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.288 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.288 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:24.288 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141837+0000", 2024-09-21T15:00:24.288 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:24.288 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.288 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.288 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:24.288 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141843+0000", 2024-09-21T15:00:24.288 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1619999999999996e-06 2024-09-21T15:00:24.288 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.289 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.289 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:24.289 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141850+0000", 2024-09-21T15:00:24.289 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4799999999999998e-06 2024-09-21T15:00:24.289 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.289 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.289 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:24.289 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141853+0000", 2024-09-21T15:00:24.289 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3050000000000001e-06 2024-09-21T15:00:24.289 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.289 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.289 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:24.289 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176490+0000", 2024-09-21T15:00:24.289 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034636413999999997 2024-09-21T15:00:24.289 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.289 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.290 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:24.290 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176606+0000", 2024-09-21T15:00:24.290 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000115841 2024-09-21T15:00:24.290 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.290 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.290 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:24.290 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176791+0000", 2024-09-21T15:00:24.290 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00018581500000000001 2024-09-21T15:00:24.290 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:24.290 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:24.290 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:24.290 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.290 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.290 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45711 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:24.290 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141856+0000", 2024-09-21T15:00:24.291 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 74.130976648000001, 2024-09-21T15:00:24.291 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034986308000000001, 2024-09-21T15:00:24.291 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:24.291 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:24.291 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:24.291 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:24.291 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:24.291 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45711 2024-09-21T15:00:24.291 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.291 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:24.291 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.291 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:24.291 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141856+0000", 2024-09-21T15:00:24.291 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:24.291 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.291 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.292 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:24.292 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141856+0000", 2024-09-21T15:00:24.292 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:24.292 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.292 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.292 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:24.292 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141856+0000", 2024-09-21T15:00:24.292 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:24.292 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.292 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.292 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:24.292 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141862+0000", 2024-09-21T15:00:24.292 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.567e-06 2024-09-21T15:00:24.292 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.292 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.292 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:24.293 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141868+0000", 2024-09-21T15:00:24.293 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9569999999999999e-06 2024-09-21T15:00:24.293 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.293 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.293 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:24.293 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141871+0000", 2024-09-21T15:00:24.293 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.146e-06 2024-09-21T15:00:24.293 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.293 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.293 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:24.293 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176831+0000", 2024-09-21T15:00:24.293 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034959935999999997 2024-09-21T15:00:24.293 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.293 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.294 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:24.294 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176842+0000", 2024-09-21T15:00:24.294 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1008e-05 2024-09-21T15:00:24.294 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:24.294 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:24.294 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:24.294 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.294 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.294 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45712 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:24.294 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141874+0000", 2024-09-21T15:00:24.294 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 74.130958753000002, 2024-09-21T15:00:24.294 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034993511999999997, 2024-09-21T15:00:24.294 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:24.294 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:24.294 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:24.295 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:24.295 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:24.295 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45712 2024-09-21T15:00:24.295 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.295 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:24.295 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.295 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:24.295 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141874+0000", 2024-09-21T15:00:24.295 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:24.295 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.295 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.295 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:24.295 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141874+0000", 2024-09-21T15:00:24.295 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:24.295 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.296 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.296 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:24.296 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141874+0000", 2024-09-21T15:00:24.296 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:24.296 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.296 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.296 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:24.296 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141875+0000", 2024-09-21T15:00:24.296 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5430000000000001e-06 2024-09-21T15:00:24.296 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.296 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.296 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:24.296 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141878+0000", 2024-09-21T15:00:24.296 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1420000000000002e-06 2024-09-21T15:00:24.296 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.296 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.297 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:24.297 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141881+0000", 2024-09-21T15:00:24.297 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.807e-06 2024-09-21T15:00:24.297 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.297 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.297 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:24.297 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176860+0000", 2024-09-21T15:00:24.297 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034979243 2024-09-21T15:00:24.297 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.297 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.297 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:24.297 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176867+0000", 2024-09-21T15:00:24.297 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0169999999999999e-06 2024-09-21T15:00:24.297 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:24.297 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:24.297 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:24.298 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.298 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.298 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45713 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:24.298 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141883+0000", 2024-09-21T15:00:24.298 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 74.130949177999995, 2024-09-21T15:00:24.298 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035010516999999998, 2024-09-21T15:00:24.298 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:24.298 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:24.298 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:24.298 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:24.298 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:24.298 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45713 2024-09-21T15:00:24.298 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.298 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:24.298 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.299 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:24.299 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141883+0000", 2024-09-21T15:00:24.299 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:24.299 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.299 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.299 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:24.299 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141883+0000", 2024-09-21T15:00:24.299 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:24.299 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.299 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.299 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:24.299 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141883+0000", 2024-09-21T15:00:24.299 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:24.299 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.299 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.300 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:24.300 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141885+0000", 2024-09-21T15:00:24.300 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4649999999999999e-06 2024-09-21T15:00:24.300 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.300 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.300 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:24.300 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141888+0000", 2024-09-21T15:00:24.300 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3890000000000001e-06 2024-09-21T15:00:24.300 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.300 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.300 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:24.300 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141891+0000", 2024-09-21T15:00:24.300 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9239999999999999e-06 2024-09-21T15:00:24.300 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.300 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.300 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:24.300 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176887+0000", 2024-09-21T15:00:24.301 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034996370999999998 2024-09-21T15:00:24.301 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.301 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.301 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:24.301 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176894+0000", 2024-09-21T15:00:24.301 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6069999999999996e-06 2024-09-21T15:00:24.301 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:24.301 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:24.301 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:24.301 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.301 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.301 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45714 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:24.301 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141893+0000", 2024-09-21T15:00:24.301 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 74.130939263000002, 2024-09-21T15:00:24.301 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035048215000000001, 2024-09-21T15:00:24.301 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:24.302 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:24.302 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:24.302 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:24.302 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:24.302 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45714 2024-09-21T15:00:24.302 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.302 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:24.302 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.302 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:24.302 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141893+0000", 2024-09-21T15:00:24.302 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:24.302 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.302 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.302 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:24.302 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141893+0000", 2024-09-21T15:00:24.302 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:24.303 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.303 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.303 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:24.303 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141893+0000", 2024-09-21T15:00:24.303 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:24.303 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.303 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.303 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:24.303 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141895+0000", 2024-09-21T15:00:24.303 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.703e-06 2024-09-21T15:00:24.303 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.303 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.303 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:24.303 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141900+0000", 2024-09-21T15:00:24.303 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.242e-06 2024-09-21T15:00:24.303 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.304 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.304 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:24.304 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141903+0000", 2024-09-21T15:00:24.304 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.117e-06 2024-09-21T15:00:24.304 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.304 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.304 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:24.304 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176930+0000", 2024-09-21T15:00:24.304 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035026750000000002 2024-09-21T15:00:24.304 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.304 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.304 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:24.304 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176942+0000", 2024-09-21T15:00:24.305 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1668000000000001e-05 2024-09-21T15:00:24.305 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:24.305 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:24.305 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:24.305 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.305 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.305 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45715 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:24.305 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141906+0000", 2024-09-21T15:00:24.305 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 74.130927104999998, 2024-09-21T15:00:24.305 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035066057999999997, 2024-09-21T15:00:24.305 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:24.305 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:24.305 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:24.306 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:24.306 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:24.306 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45715 2024-09-21T15:00:24.306 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.306 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:24.306 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.306 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:24.306 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141906+0000", 2024-09-21T15:00:24.306 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:24.306 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.306 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.306 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:24.306 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141906+0000", 2024-09-21T15:00:24.306 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:24.306 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.307 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.307 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:24.307 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141905+0000", 2024-09-21T15:00:24.307 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:24.307 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.307 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.307 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:24.307 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141907+0000", 2024-09-21T15:00:24.307 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5400000000000001e-06 2024-09-21T15:00:24.307 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.307 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.307 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:24.307 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141912+0000", 2024-09-21T15:00:24.307 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.502e-06 2024-09-21T15:00:24.307 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.308 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.308 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:24.308 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141915+0000", 2024-09-21T15:00:24.308 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0170000000000001e-06 2024-09-21T15:00:24.308 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.308 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.308 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:24.308 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176964+0000", 2024-09-21T15:00:24.308 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035048587999999999 2024-09-21T15:00:24.308 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.308 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.308 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:24.308 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176972+0000", 2024-09-21T15:00:24.308 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6580000000000002e-06 2024-09-21T15:00:24.308 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:24.309 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:24.309 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:24.309 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.309 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.309 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45716 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:24.309 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141918+0000", 2024-09-21T15:00:24.309 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 74.130914928999999, 2024-09-21T15:00:24.309 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035086223, 2024-09-21T15:00:24.309 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:24.309 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:24.309 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:24.309 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:24.309 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:24.309 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45716 2024-09-21T15:00:24.310 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.310 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:24.310 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.310 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:24.310 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141918+0000", 2024-09-21T15:00:24.310 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:24.310 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.310 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.310 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:24.310 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141918+0000", 2024-09-21T15:00:24.310 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:24.310 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.310 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.310 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:24.311 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141917+0000", 2024-09-21T15:00:24.311 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:24.311 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.311 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.311 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:24.311 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141919+0000", 2024-09-21T15:00:24.311 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5680000000000001e-06 2024-09-21T15:00:24.311 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.311 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.311 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:24.311 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141922+0000", 2024-09-21T15:00:24.311 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2789999999999999e-06 2024-09-21T15:00:24.311 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.311 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.312 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:24.312 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141925+0000", 2024-09-21T15:00:24.312 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9469999999999999e-06 2024-09-21T15:00:24.312 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.312 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.312 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:24.312 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176997+0000", 2024-09-21T15:00:24.312 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035071366999999999 2024-09-21T15:00:24.312 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.312 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.312 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:24.312 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177004+0000", 2024-09-21T15:00:24.312 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3309999999999998e-06 2024-09-21T15:00:24.312 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:24.312 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:24.312 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:24.313 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.313 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.313 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45717 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:24.313 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141928+0000", 2024-09-21T15:00:24.313 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 74.130904982999994, 2024-09-21T15:00:24.313 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035102573999999998, 2024-09-21T15:00:24.313 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:24.313 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:24.313 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:24.313 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:24.313 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:24.313 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45717 2024-09-21T15:00:24.313 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.313 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:24.314 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.314 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:24.314 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141928+0000", 2024-09-21T15:00:24.314 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:24.314 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.314 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.314 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:24.314 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141928+0000", 2024-09-21T15:00:24.314 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:24.314 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.314 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.314 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:24.314 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141927+0000", 2024-09-21T15:00:24.315 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:00:24.315 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.315 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.315 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:24.315 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141929+0000", 2024-09-21T15:00:24.315 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.533e-06 2024-09-21T15:00:24.315 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.315 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.315 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:24.315 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141934+0000", 2024-09-21T15:00:24.315 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1660000000000002e-06 2024-09-21T15:00:24.315 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.315 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.315 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:24.315 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141937+0000", 2024-09-21T15:00:24.316 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0309999999999999e-06 2024-09-21T15:00:24.316 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.316 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.316 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:24.316 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177023+0000", 2024-09-21T15:00:24.316 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035085595999999997 2024-09-21T15:00:24.316 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.316 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.316 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:24.316 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177030+0000", 2024-09-21T15:00:24.316 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.486e-06 2024-09-21T15:00:24.316 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:24.316 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:24.316 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:24.317 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.317 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.317 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45718 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:24.317 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141940+0000", 2024-09-21T15:00:24.317 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 74.130893125, 2024-09-21T15:00:24.317 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035124956999999998, 2024-09-21T15:00:24.317 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:24.317 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:24.317 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:24.317 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:24.317 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:24.317 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45718 2024-09-21T15:00:24.317 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.318 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:24.318 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.318 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:24.318 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141940+0000", 2024-09-21T15:00:24.318 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:24.318 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.318 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.318 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:24.318 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141940+0000", 2024-09-21T15:00:24.318 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:24.318 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.318 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.318 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:24.318 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141939+0000", 2024-09-21T15:00:24.318 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:24.319 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.319 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.319 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:24.319 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141941+0000", 2024-09-21T15:00:24.319 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.502e-06 2024-09-21T15:00:24.319 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.319 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.319 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:24.319 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141946+0000", 2024-09-21T15:00:24.319 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3449999999999996e-06 2024-09-21T15:00:24.319 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.319 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.319 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:24.319 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141949+0000", 2024-09-21T15:00:24.319 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9749999999999999e-06 2024-09-21T15:00:24.319 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.320 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.320 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:24.320 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177057+0000", 2024-09-21T15:00:24.320 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035108385999999998 2024-09-21T15:00:24.320 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.320 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.320 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:24.320 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177064+0000", 2024-09-21T15:00:24.320 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0099999999999998e-06 2024-09-21T15:00:24.320 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:24.320 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:24.320 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:24.320 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.320 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.320 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45719 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:24.321 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141952+0000", 2024-09-21T15:00:24.321 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 74.130880852000004, 2024-09-21T15:00:24.321 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035139108000000002, 2024-09-21T15:00:24.321 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:24.321 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:24.321 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:24.321 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:24.321 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:24.321 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45719 2024-09-21T15:00:24.321 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.321 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:24.321 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.321 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:24.321 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141952+0000", 2024-09-21T15:00:24.321 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:24.321 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.321 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.322 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:24.322 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141952+0000", 2024-09-21T15:00:24.322 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:24.322 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.322 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.322 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:24.322 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141952+0000", 2024-09-21T15:00:24.322 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:24.322 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.322 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.322 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:24.322 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141953+0000", 2024-09-21T15:00:24.322 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.486e-06 2024-09-21T15:00:24.322 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.323 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.323 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:24.323 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141959+0000", 2024-09-21T15:00:24.323 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4940000000000003e-06 2024-09-21T15:00:24.323 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.323 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.323 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:24.323 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141963+0000", 2024-09-21T15:00:24.323 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3540000000000002e-06 2024-09-21T15:00:24.323 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.323 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.323 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:24.323 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177084+0000", 2024-09-21T15:00:24.323 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035120983000000001 2024-09-21T15:00:24.324 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.324 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.324 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:24.324 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177091+0000", 2024-09-21T15:00:24.324 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0350000000000004e-06 2024-09-21T15:00:24.324 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:24.324 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:24.324 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:24.324 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.324 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.324 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45720 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:24.324 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141965+0000", 2024-09-21T15:00:24.324 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 74.130867385000002, 2024-09-21T15:00:24.324 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035155268000000003, 2024-09-21T15:00:24.324 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:24.325 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:24.325 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:24.325 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:24.325 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:24.325 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45720 2024-09-21T15:00:24.325 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.325 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:24.325 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.325 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:24.325 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141965+0000", 2024-09-21T15:00:24.325 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:24.325 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.325 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.325 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:24.326 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141965+0000", 2024-09-21T15:00:24.326 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:24.326 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.326 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.326 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:24.326 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141965+0000", 2024-09-21T15:00:24.326 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:24.326 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.326 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.326 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:24.326 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141966+0000", 2024-09-21T15:00:24.326 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.46e-06 2024-09-21T15:00:24.326 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.326 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.326 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:24.327 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141972+0000", 2024-09-21T15:00:24.327 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4210000000000003e-06 2024-09-21T15:00:24.327 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.327 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.327 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:24.327 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141975+0000", 2024-09-21T15:00:24.327 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.948e-06 2024-09-21T15:00:24.327 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.327 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.327 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:24.327 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177113+0000", 2024-09-21T15:00:24.327 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035137855000000003 2024-09-21T15:00:24.327 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.327 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.328 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:24.328 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177121+0000", 2024-09-21T15:00:24.328 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8269999999999998e-06 2024-09-21T15:00:24.328 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:24.328 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:24.328 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:24.328 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.328 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.328 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45721 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:24.328 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141977+0000", 2024-09-21T15:00:24.328 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 74.130855362000005, 2024-09-21T15:00:24.328 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035169984000000001, 2024-09-21T15:00:24.328 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:24.328 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:24.328 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:24.328 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:24.328 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:24.329 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45721 2024-09-21T15:00:24.329 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.329 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:24.329 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.329 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:24.329 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141977+0000", 2024-09-21T15:00:24.329 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:24.329 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.329 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.329 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:24.329 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141977+0000", 2024-09-21T15:00:24.329 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:24.329 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.329 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.329 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:24.329 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141977+0000", 2024-09-21T15:00:24.330 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:24.330 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.330 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.330 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:24.330 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141979+0000", 2024-09-21T15:00:24.330 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.511e-06 2024-09-21T15:00:24.330 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.330 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.330 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:24.330 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141982+0000", 2024-09-21T15:00:24.330 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4180000000000001e-06 2024-09-21T15:00:24.330 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.330 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.330 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:24.330 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141985+0000", 2024-09-21T15:00:24.331 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9450000000000002e-06 2024-09-21T15:00:24.331 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.331 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.331 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:24.331 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177139+0000", 2024-09-21T15:00:24.331 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035154409999999997 2024-09-21T15:00:24.331 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.331 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.331 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:24.331 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177147+0000", 2024-09-21T15:00:24.331 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9570000000000002e-06 2024-09-21T15:00:24.331 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:24.331 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:24.331 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:24.331 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.331 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.332 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45722 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:24.332 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141987+0000", 2024-09-21T15:00:24.332 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 74.130845390999994, 2024-09-21T15:00:24.332 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035188492000000002, 2024-09-21T15:00:24.332 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:24.332 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:24.332 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:24.332 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:24.332 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:24.332 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45722 2024-09-21T15:00:24.332 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.332 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:24.332 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.332 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:24.332 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141987+0000", 2024-09-21T15:00:24.333 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:24.333 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.333 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.333 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:24.333 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141987+0000", 2024-09-21T15:00:24.333 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:24.333 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.333 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.333 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:24.333 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141987+0000", 2024-09-21T15:00:24.333 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:24.333 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.333 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.333 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:24.334 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141988+0000", 2024-09-21T15:00:24.334 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.463e-06 2024-09-21T15:00:24.334 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.334 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.334 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:24.334 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141994+0000", 2024-09-21T15:00:24.334 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.361e-06 2024-09-21T15:00:24.334 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.334 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.334 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:24.334 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141997+0000", 2024-09-21T15:00:24.334 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0359999999999998e-06 2024-09-21T15:00:24.335 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.335 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.335 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:24.335 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177169+0000", 2024-09-21T15:00:24.335 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035171651999999998 2024-09-21T15:00:24.335 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.335 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.335 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:24.335 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177176+0000", 2024-09-21T15:00:24.335 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2189999999999998e-06 2024-09-21T15:00:24.335 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:24.335 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:24.335 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:24.335 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.335 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.335 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45723 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:24.336 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141999+0000", 2024-09-21T15:00:24.336 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 74.130833224, 2024-09-21T15:00:24.336 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035197701999999997, 2024-09-21T15:00:24.336 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:24.336 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:24.336 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:24.336 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:24.336 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:24.336 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45723 2024-09-21T15:00:24.336 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.336 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:24.336 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.336 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:24.336 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141999+0000", 2024-09-21T15:00:24.336 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:24.336 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.336 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.337 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:24.337 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141999+0000", 2024-09-21T15:00:24.337 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:24.337 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.337 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.337 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:24.337 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141999+0000", 2024-09-21T15:00:24.337 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:24.337 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.337 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.337 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:24.337 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142003+0000", 2024-09-21T15:00:24.337 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5049999999999999e-06 2024-09-21T15:00:24.337 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.337 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.338 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:24.338 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142008+0000", 2024-09-21T15:00:24.338 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6579999999999999e-06 2024-09-21T15:00:24.338 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.338 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.338 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:24.338 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142011+0000", 2024-09-21T15:00:24.338 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0529999999999998e-06 2024-09-21T15:00:24.338 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.338 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.338 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:24.338 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177194+0000", 2024-09-21T15:00:24.338 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035182221999999999 2024-09-21T15:00:24.338 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.338 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.339 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:24.339 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177197+0000", 2024-09-21T15:00:24.339 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5030000000000002e-06 2024-09-21T15:00:24.339 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:24.341 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:24.341 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:24.341 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.341 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.342 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45724 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:24.342 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.142015+0000", 2024-09-21T15:00:24.342 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 74.130817257000004, 2024-09-21T15:00:24.342 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035195345000000003, 2024-09-21T15:00:24.342 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:24.342 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:24.342 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:24.342 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:24.342 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:24.342 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45724 2024-09-21T15:00:24.342 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.342 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:24.342 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.342 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:24.343 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142015+0000", 2024-09-21T15:00:24.343 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:24.343 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.343 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.343 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:24.343 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142015+0000", 2024-09-21T15:00:24.343 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:24.343 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.343 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.343 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:24.343 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142015+0000", 2024-09-21T15:00:24.343 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:24.343 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.343 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.343 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:24.344 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142017+0000", 2024-09-21T15:00:24.344 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5540000000000001e-06 2024-09-21T15:00:24.344 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.344 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.344 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:24.344 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142022+0000", 2024-09-21T15:00:24.344 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4229999999999996e-06 2024-09-21T15:00:24.344 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.344 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.344 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:24.344 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142025+0000", 2024-09-21T15:00:24.344 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.056e-06 2024-09-21T15:00:24.344 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.344 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.344 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:24.345 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177207+0000", 2024-09-21T15:00:24.345 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035182089 2024-09-21T15:00:24.345 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.345 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.345 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:24.345 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177211+0000", 2024-09-21T15:00:24.345 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4649999999999999e-06 2024-09-21T15:00:24.345 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:24.345 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:24.345 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:24.345 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.345 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.345 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45725 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:24.345 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.142028+0000", 2024-09-21T15:00:24.345 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 74.130804978, 2024-09-21T15:00:24.346 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035195731000000001, 2024-09-21T15:00:24.346 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:24.346 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:24.346 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:24.346 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:24.346 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:24.346 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45725 2024-09-21T15:00:24.346 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.346 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:24.346 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.346 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:24.346 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142028+0000", 2024-09-21T15:00:24.346 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:24.347 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.347 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.347 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:24.347 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142028+0000", 2024-09-21T15:00:24.347 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:24.347 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.347 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.347 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:24.347 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142027+0000", 2024-09-21T15:00:24.347 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:24.347 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.347 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.347 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:24.347 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142029+0000", 2024-09-21T15:00:24.348 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5269999999999999e-06 2024-09-21T15:00:24.348 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.348 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.348 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:24.348 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142032+0000", 2024-09-21T15:00:24.348 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3249999999999999e-06 2024-09-21T15:00:24.348 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.348 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.348 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:24.348 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142035+0000", 2024-09-21T15:00:24.348 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.897e-06 2024-09-21T15:00:24.348 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.348 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.348 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:24.349 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177220+0000", 2024-09-21T15:00:24.349 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035184871 2024-09-21T15:00:24.349 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.349 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.349 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:24.349 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177223+0000", 2024-09-21T15:00:24.349 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3680000000000002e-06 2024-09-21T15:00:24.349 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:24.349 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:24.349 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:24.349 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.349 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.349 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45726 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:24.350 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.142037+0000", 2024-09-21T15:00:24.350 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 74.130795262999996, 2024-09-21T15:00:24.350 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035199273000000003, 2024-09-21T15:00:24.350 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:24.350 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:24.350 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:24.350 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:24.350 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:24.350 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45726 2024-09-21T15:00:24.350 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.350 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:24.351 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.351 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:24.351 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142037+0000", 2024-09-21T15:00:24.351 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:24.351 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.351 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.351 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:24.351 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142037+0000", 2024-09-21T15:00:24.351 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:24.351 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.351 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.351 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:24.352 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142037+0000", 2024-09-21T15:00:24.352 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:24.352 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.352 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.352 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:24.352 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142038+0000", 2024-09-21T15:00:24.352 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.344e-06 2024-09-21T15:00:24.352 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.352 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.352 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:24.352 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142044+0000", 2024-09-21T15:00:24.353 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1930000000000001e-06 2024-09-21T15:00:24.353 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.353 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.353 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:24.353 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142047+0000", 2024-09-21T15:00:24.353 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0280000000000001e-06 2024-09-21T15:00:24.353 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.353 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.353 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:24.353 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177233+0000", 2024-09-21T15:00:24.353 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035186394000000003 2024-09-21T15:00:24.353 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.353 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.354 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:24.354 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177237+0000", 2024-09-21T15:00:24.354 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5530000000000001e-06 2024-09-21T15:00:24.354 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:24.354 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:24.354 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:24.354 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:24.354 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:24.354 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:24.355 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T15:00:24.371 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:00:24.366+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:00:24.468 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:24.468 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:00:24.468 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T15:00:24.468 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:24.472 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:00:24.468+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:00:24.478 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T15:00:24.573 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:00:24.570+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:00:24.630 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:24.631 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:00:24.631 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T15:00:24.631 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T15:00:24.631 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:24.640 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T15:00:24.673 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:00:24.672+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:00:24.750 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:24.750 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T15:00:24.750 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T15:00:24.750 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T15:00:24.750 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.750 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4281.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-21T15:00:24.750 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:09.324416+0000", 2024-09-21T15:00:24.750 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 135.424645269, 2024-09-21T15:00:24.750 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.117837947, 2024-09-21T15:00:24.750 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:24.751 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:24.751 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:24.751 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4281", 2024-09-21T15:00:24.751 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/1518520303", 2024-09-21T15:00:24.751 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1 2024-09-21T15:00:24.751 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.751 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:24.751 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.751 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:24.751 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324416+0000", 2024-09-21T15:00:24.751 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:24.751 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.751 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.751 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:24.751 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324416+0000", 2024-09-21T15:00:24.752 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:24.752 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.752 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.752 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:24.752 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324414+0000", 2024-09-21T15:00:24.752 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T15:00:24.752 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.752 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.752 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:24.752 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324429+0000", 2024-09-21T15:00:24.752 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5637e-05 2024-09-21T15:00:24.752 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.752 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.752 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:24.752 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324448+0000", 2024-09-21T15:00:24.753 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9042000000000002e-05 2024-09-21T15:00:24.753 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.753 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.753 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:24.753 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324465+0000", 2024-09-21T15:00:24.753 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6767999999999999e-05 2024-09-21T15:00:24.753 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.753 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.753 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:24.753 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324571+0000", 2024-09-21T15:00:24.753 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000105714 2024-09-21T15:00:24.753 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.753 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.753 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op must wait for map", 2024-09-21T15:00:24.753 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324587+0000", 2024-09-21T15:00:24.754 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5753999999999999e-05 2024-09-21T15:00:24.754 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.754 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.754 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:24.754 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.325864+0000", 2024-09-21T15:00:24.754 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0012773350000000001 2024-09-21T15:00:24.754 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.754 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.754 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:00:24.754 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.325878+0000", 2024-09-21T15:00:24.754 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4498000000000001e-05 2024-09-21T15:00:24.754 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.754 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.754 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:24.755 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.439461+0000", 2024-09-21T15:00:24.755 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.11358212500000001 2024-09-21T15:00:24.755 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.755 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.755 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:24.755 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.439848+0000", 2024-09-21T15:00:24.755 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00038747800000000002 2024-09-21T15:00:24.755 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.755 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.755 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T15:00:24.755 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.440175+0000", 2024-09-21T15:00:24.755 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00032721900000000001 2024-09-21T15:00:24.755 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.755 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.755 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:24.755 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.441770+0000", 2024-09-21T15:00:24.756 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0015949409999999999 2024-09-21T15:00:24.756 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.756 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.756 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:24.756 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.442086+0000", 2024-09-21T15:00:24.756 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00031586299999999999 2024-09-21T15:00:24.756 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.756 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.756 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:24.756 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.442195+0000", 2024-09-21T15:00:24.756 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000108791 2024-09-21T15:00:24.756 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.756 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.756 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:24.757 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.442254+0000", 2024-09-21T15:00:24.757 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9038e-05 2024-09-21T15:00:24.757 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:24.757 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:24.757 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:24.757 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.757 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.757 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4281.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-21T15:00:24.757 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:09.447828+0000", 2024-09-21T15:00:24.757 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 135.301233268, 2024-09-21T15:00:24.757 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.32904393700000001, 2024-09-21T15:00:24.757 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:24.757 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:24.757 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:24.757 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4281", 2024-09-21T15:00:24.758 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/1518520303", 2024-09-21T15:00:24.758 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 2 2024-09-21T15:00:24.758 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.758 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:24.758 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.758 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:24.758 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447828+0000", 2024-09-21T15:00:24.758 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:24.758 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.758 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.758 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:24.758 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447828+0000", 2024-09-21T15:00:24.758 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:24.758 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.758 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.759 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:24.759 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447825+0000", 2024-09-21T15:00:24.759 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T15:00:24.759 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.759 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.759 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:24.759 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447841+0000", 2024-09-21T15:00:24.759 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5359999999999999e-05 2024-09-21T15:00:24.759 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.759 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.759 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:24.759 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447875+0000", 2024-09-21T15:00:24.759 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4388999999999999e-05 2024-09-21T15:00:24.759 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.759 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.760 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:24.760 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447903+0000", 2024-09-21T15:00:24.760 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7373e-05 2024-09-21T15:00:24.760 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.760 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.760 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:24.760 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447996+0000", 2024-09-21T15:00:24.760 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2954999999999993e-05 2024-09-21T15:00:24.760 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.760 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.760 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:00:24.760 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.448011+0000", 2024-09-21T15:00:24.760 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5038999999999999e-05 2024-09-21T15:00:24.760 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.760 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.760 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:24.761 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.776501+0000", 2024-09-21T15:00:24.761 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.32849012500000002 2024-09-21T15:00:24.761 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.761 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.761 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:24.761 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.776872+0000", 2024-09-21T15:00:24.761 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00037119600000000002 2024-09-21T15:00:24.761 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:24.761 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:24.761 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:24.761 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.761 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.761 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:752 3.14 3:28fc02e5:::rbd_data.113693316f36.0000000000000010:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-21T15:00:24.761 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:26.625579+0000", 2024-09-21T15:00:24.761 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 118.123482413, 2024-09-21T15:00:24.762 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042965426000000001, 2024-09-21T15:00:24.762 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:24.762 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:24.762 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:24.762 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:24.762 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:24.762 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 752 2024-09-21T15:00:24.762 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.762 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:24.762 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.762 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:24.762 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625579+0000", 2024-09-21T15:00:24.762 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:24.762 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.762 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.763 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:24.763 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625579+0000", 2024-09-21T15:00:24.763 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:24.763 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.763 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.763 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:24.763 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625577+0000", 2024-09-21T15:00:24.763 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T15:00:24.763 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.763 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.763 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:24.763 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625588+0000", 2024-09-21T15:00:24.763 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1555e-05 2024-09-21T15:00:24.763 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.764 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.764 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:24.764 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625610+0000", 2024-09-21T15:00:24.764 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2107999999999999e-05 2024-09-21T15:00:24.764 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.764 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.764 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:24.764 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625621+0000", 2024-09-21T15:00:24.764 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0762e-05 2024-09-21T15:00:24.764 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.764 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.764 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:24.764 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625692+0000", 2024-09-21T15:00:24.764 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.1203000000000003e-05 2024-09-21T15:00:24.765 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.765 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.765 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:24.765 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.625872+0000", 2024-09-21T15:00:24.765 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017962800000000001 2024-09-21T15:00:24.765 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.765 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.765 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T15:00:24.765 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.633188+0000", 2024-09-21T15:00:24.765 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.007316014 2024-09-21T15:00:24.765 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.765 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.765 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:24.765 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.668357+0000", 2024-09-21T15:00:24.765 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035168933999999999 2024-09-21T15:00:24.765 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.766 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.766 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:24.766 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.668418+0000", 2024-09-21T15:00:24.766 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1282e-05 2024-09-21T15:00:24.766 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.766 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.766 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:24.766 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.668486+0000", 2024-09-21T15:00:24.766 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8468999999999998e-05 2024-09-21T15:00:24.766 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.766 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.766 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:24.766 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:26.668544+0000", 2024-09-21T15:00:24.766 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7785000000000003e-05 2024-09-21T15:00:24.766 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:24.766 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:24.767 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:24.767 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.767 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.767 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22741 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 245760~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T15:00:24.767 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141107+0000", 2024-09-21T15:00:24.767 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 94.607954053, 2024-09-21T15:00:24.767 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036407729, 2024-09-21T15:00:24.767 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:24.767 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:24.767 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:24.767 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:24.767 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:24.767 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22741 2024-09-21T15:00:24.767 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.767 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:24.768 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.768 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:24.768 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141107+0000", 2024-09-21T15:00:24.768 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:24.768 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.768 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.768 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:24.768 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141107+0000", 2024-09-21T15:00:24.768 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:24.768 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.768 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.768 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:24.768 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141107+0000", 2024-09-21T15:00:24.768 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:24.769 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.769 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.769 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:24.769 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141109+0000", 2024-09-21T15:00:24.769 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6529999999999998e-06 2024-09-21T15:00:24.769 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.769 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.769 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:24.769 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141115+0000", 2024-09-21T15:00:24.769 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9649999999999996e-06 2024-09-21T15:00:24.769 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.769 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.769 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:24.769 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141121+0000", 2024-09-21T15:00:24.769 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1209999999999998e-06 2024-09-21T15:00:24.770 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.770 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.770 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:24.770 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177242+0000", 2024-09-21T15:00:24.770 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036121706000000003 2024-09-21T15:00:24.770 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.770 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.770 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:24.770 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177301+0000", 2024-09-21T15:00:24.770 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9008000000000002e-05 2024-09-21T15:00:24.770 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.770 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.770 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:24.770 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177515+0000", 2024-09-21T15:00:24.770 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00021362 2024-09-21T15:00:24.771 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:24.771 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:24.771 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:24.771 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.771 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.771 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22742 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 311296~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T15:00:24.771 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141126+0000", 2024-09-21T15:00:24.771 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 94.607935698999995, 2024-09-21T15:00:24.771 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036653537, 2024-09-21T15:00:24.771 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:24.771 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:24.771 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:24.771 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:24.771 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:24.771 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22742 2024-09-21T15:00:24.772 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.772 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:24.772 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.772 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:24.772 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141126+0000", 2024-09-21T15:00:24.772 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:24.772 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.772 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.772 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:24.772 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141126+0000", 2024-09-21T15:00:24.772 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:24.772 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.772 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.772 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:24.772 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141125+0000", 2024-09-21T15:00:24.772 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:24.773 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.773 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.773 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:24.773 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141128+0000", 2024-09-21T15:00:24.773 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.886e-06 2024-09-21T15:00:24.773 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.773 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.773 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:24.773 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141137+0000", 2024-09-21T15:00:24.773 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4399999999999994e-06 2024-09-21T15:00:24.773 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.773 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.773 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:24.773 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141143+0000", 2024-09-21T15:00:24.773 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1410000000000004e-06 2024-09-21T15:00:24.773 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.774 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.774 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:24.774 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177543+0000", 2024-09-21T15:00:24.774 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036400544999999999 2024-09-21T15:00:24.774 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.774 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.774 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:24.774 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177605+0000", 2024-09-21T15:00:24.774 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2206999999999996e-05 2024-09-21T15:00:24.774 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.774 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.774 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:24.774 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177779+0000", 2024-09-21T15:00:24.774 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017368099999999999 2024-09-21T15:00:24.775 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:24.775 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:24.775 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:24.775 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.775 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.775 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22743 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 376832~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T15:00:24.775 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141147+0000", 2024-09-21T15:00:24.775 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 94.607914214999994, 2024-09-21T15:00:24.775 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036915158000000003, 2024-09-21T15:00:24.775 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:24.775 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:24.775 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:24.775 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:24.775 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:24.775 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22743 2024-09-21T15:00:24.775 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.776 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:24.776 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.776 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:24.776 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141147+0000", 2024-09-21T15:00:24.776 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:24.776 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.776 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.776 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:24.776 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141147+0000", 2024-09-21T15:00:24.776 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:24.776 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.776 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.776 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:24.776 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141147+0000", 2024-09-21T15:00:24.776 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:24.777 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.777 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.777 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:24.777 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141149+0000", 2024-09-21T15:00:24.777 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.79e-06 2024-09-21T15:00:24.777 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.777 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.777 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:24.777 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141155+0000", 2024-09-21T15:00:24.777 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6699999999999999e-06 2024-09-21T15:00:24.777 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.777 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.777 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:24.777 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141160+0000", 2024-09-21T15:00:24.777 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5759999999999999e-06 2024-09-21T15:00:24.778 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.778 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.778 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:24.778 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177817+0000", 2024-09-21T15:00:24.778 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036657404999999997 2024-09-21T15:00:24.778 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.778 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.778 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:24.778 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.177892+0000", 2024-09-21T15:00:24.778 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4602000000000002e-05 2024-09-21T15:00:24.778 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.778 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.778 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:24.778 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178062+0000", 2024-09-21T15:00:24.778 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000170464 2024-09-21T15:00:24.778 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:24.779 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:24.779 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:24.779 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.779 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.779 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22744 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 442368~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T15:00:24.779 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141164+0000", 2024-09-21T15:00:24.779 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 94.607897081999994, 2024-09-21T15:00:24.779 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037120806999999999, 2024-09-21T15:00:24.779 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:24.779 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:24.779 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:24.779 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:24.779 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:24.779 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22744 2024-09-21T15:00:24.779 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.779 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:24.780 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.780 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:24.780 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141164+0000", 2024-09-21T15:00:24.780 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:24.780 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.780 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.780 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:24.780 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141164+0000", 2024-09-21T15:00:24.780 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:24.780 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.780 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.780 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:24.780 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141164+0000", 2024-09-21T15:00:24.780 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:24.780 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.780 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.781 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:24.781 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141167+0000", 2024-09-21T15:00:24.781 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7159999999999999e-06 2024-09-21T15:00:24.781 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.781 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.781 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:24.781 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141173+0000", 2024-09-21T15:00:24.781 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.071e-06 2024-09-21T15:00:24.781 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.781 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.781 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:24.781 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141177+0000", 2024-09-21T15:00:24.781 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3610000000000003e-06 2024-09-21T15:00:24.781 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.781 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.781 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:24.781 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178083+0000", 2024-09-21T15:00:24.782 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036906405000000003 2024-09-21T15:00:24.782 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.782 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.782 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:24.782 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178143+0000", 2024-09-21T15:00:24.782 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0056999999999998e-05 2024-09-21T15:00:24.782 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.782 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.782 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:24.782 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178285+0000", 2024-09-21T15:00:24.782 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000141555 2024-09-21T15:00:24.782 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:24.782 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:24.783 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:24.783 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.783 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.783 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22745 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 507904~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T15:00:24.783 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141182+0000", 2024-09-21T15:00:24.783 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 94.607878761999999, 2024-09-21T15:00:24.783 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037332331000000003, 2024-09-21T15:00:24.783 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:24.783 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:24.783 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:24.783 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:24.783 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:24.783 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22745 2024-09-21T15:00:24.783 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.783 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:24.784 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.784 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:24.784 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141182+0000", 2024-09-21T15:00:24.784 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:24.784 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.784 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.784 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:24.784 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141182+0000", 2024-09-21T15:00:24.784 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:24.784 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.784 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.784 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:24.784 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141182+0000", 2024-09-21T15:00:24.784 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:24.785 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.785 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.785 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:24.785 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141185+0000", 2024-09-21T15:00:24.785 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.464e-06 2024-09-21T15:00:24.785 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.785 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.785 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:24.785 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141200+0000", 2024-09-21T15:00:24.785 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5886999999999999e-05 2024-09-21T15:00:24.785 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.785 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.786 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:24.786 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141571+0000", 2024-09-21T15:00:24.786 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000370909 2024-09-21T15:00:24.786 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.786 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.786 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:24.786 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178318+0000", 2024-09-21T15:00:24.786 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036746244999999997 2024-09-21T15:00:24.786 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.786 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.786 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:24.786 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178367+0000", 2024-09-21T15:00:24.786 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9317000000000002e-05 2024-09-21T15:00:24.786 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.787 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.787 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:24.787 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178515+0000", 2024-09-21T15:00:24.787 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014784600000000001 2024-09-21T15:00:24.787 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:24.787 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:24.787 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:24.787 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.787 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.787 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22746 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 573440~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T15:00:24.787 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141599+0000", 2024-09-21T15:00:24.787 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 94.607462130000002, 2024-09-21T15:00:24.787 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037402249999999998, 2024-09-21T15:00:24.787 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:24.787 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:24.788 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:24.788 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:24.788 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:24.788 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22746 2024-09-21T15:00:24.788 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.788 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:24.788 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.788 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:24.788 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141599+0000", 2024-09-21T15:00:24.788 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:24.788 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.788 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.788 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:24.788 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141599+0000", 2024-09-21T15:00:24.788 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:24.789 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.789 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.789 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:24.789 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141598+0000", 2024-09-21T15:00:24.789 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:00:24.789 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.789 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.789 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:24.789 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141637+0000", 2024-09-21T15:00:24.789 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.9119999999999998e-05 2024-09-21T15:00:24.789 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.789 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.789 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:24.789 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141647+0000", 2024-09-21T15:00:24.789 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5570000000000001e-06 2024-09-21T15:00:24.790 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.790 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.790 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:24.790 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141654+0000", 2024-09-21T15:00:24.790 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2049999999999996e-06 2024-09-21T15:00:24.790 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.790 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.790 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:24.790 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178553+0000", 2024-09-21T15:00:24.790 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036898528 2024-09-21T15:00:24.790 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.790 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.790 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:24.790 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.178625+0000", 2024-09-21T15:00:24.791 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2669999999999994e-05 2024-09-21T15:00:24.791 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.791 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.791 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:24.791 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179001+0000", 2024-09-21T15:00:24.791 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000375977 2024-09-21T15:00:24.791 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:24.791 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:24.791 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:24.791 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.791 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.791 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22747 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 638976~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T15:00:24.791 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141660+0000", 2024-09-21T15:00:24.791 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 94.607400987999995, 2024-09-21T15:00:24.791 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037617352999999999, 2024-09-21T15:00:24.792 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:24.792 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:24.792 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:24.792 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:24.792 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:24.792 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22747 2024-09-21T15:00:24.792 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.792 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:24.792 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.792 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:24.792 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141660+0000", 2024-09-21T15:00:24.792 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:24.792 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.792 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.792 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:24.792 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141660+0000", 2024-09-21T15:00:24.793 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:24.793 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.793 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.793 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:24.793 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141660+0000", 2024-09-21T15:00:24.793 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:24.793 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.793 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.793 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:24.793 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141663+0000", 2024-09-21T15:00:24.793 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.569e-06 2024-09-21T15:00:24.793 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.793 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.793 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:24.793 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141670+0000", 2024-09-21T15:00:24.793 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5719999999999999e-06 2024-09-21T15:00:24.794 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.794 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.794 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:24.794 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141675+0000", 2024-09-21T15:00:24.794 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2889999999999997e-06 2024-09-21T15:00:24.794 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.794 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.794 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:24.794 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179028+0000", 2024-09-21T15:00:24.794 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037353381999999997 2024-09-21T15:00:24.794 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.794 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.795 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:24.795 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179099+0000", 2024-09-21T15:00:24.795 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0259000000000006e-05 2024-09-21T15:00:24.795 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.795 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.795 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:24.795 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179278+0000", 2024-09-21T15:00:24.795 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000178908 2024-09-21T15:00:24.795 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:24.795 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:24.795 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:24.795 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.795 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.795 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22748 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 704512~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T15:00:24.795 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141680+0000", 2024-09-21T15:00:24.795 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 94.607381349999997, 2024-09-21T15:00:24.796 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037806166000000002, 2024-09-21T15:00:24.796 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:24.796 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:24.796 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:24.796 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:24.796 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:24.796 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22748 2024-09-21T15:00:24.796 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.796 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:24.796 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.796 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:24.796 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141680+0000", 2024-09-21T15:00:24.796 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:24.796 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.796 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.796 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:24.797 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141680+0000", 2024-09-21T15:00:24.797 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:24.797 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.797 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.797 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:24.797 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141679+0000", 2024-09-21T15:00:24.797 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:24.797 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.797 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.797 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:24.797 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141683+0000", 2024-09-21T15:00:24.797 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3230000000000002e-06 2024-09-21T15:00:24.797 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.797 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.797 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:24.798 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141690+0000", 2024-09-21T15:00:24.798 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8680000000000001e-06 2024-09-21T15:00:24.798 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.798 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.798 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:24.798 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141694+0000", 2024-09-21T15:00:24.798 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6770000000000003e-06 2024-09-21T15:00:24.798 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.798 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.798 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:24.798 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179300+0000", 2024-09-21T15:00:24.798 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037605567999999999 2024-09-21T15:00:24.798 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.798 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.798 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:24.799 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179355+0000", 2024-09-21T15:00:24.799 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4771000000000001e-05 2024-09-21T15:00:24.799 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.799 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.799 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:24.799 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179486+0000", 2024-09-21T15:00:24.799 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013141699999999999 2024-09-21T15:00:24.799 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:24.799 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:24.799 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:24.799 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.799 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.799 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22749 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 770048~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T15:00:24.799 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141698+0000", 2024-09-21T15:00:24.799 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 94.607362977999998, 2024-09-21T15:00:24.799 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037967813000000003, 2024-09-21T15:00:24.800 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:24.800 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:24.800 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:24.800 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:24.800 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:24.800 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22749 2024-09-21T15:00:24.800 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.800 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:24.800 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.800 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:24.800 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141698+0000", 2024-09-21T15:00:24.800 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:24.800 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.800 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.800 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:24.800 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141698+0000", 2024-09-21T15:00:24.801 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:24.801 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.801 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.801 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:24.801 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141698+0000", 2024-09-21T15:00:24.801 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:24.801 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.801 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.801 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:24.801 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141701+0000", 2024-09-21T15:00:24.801 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8399999999999999e-06 2024-09-21T15:00:24.801 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.801 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.801 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:24.801 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141706+0000", 2024-09-21T15:00:24.801 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4979999999999997e-06 2024-09-21T15:00:24.802 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.802 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.802 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:24.802 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141711+0000", 2024-09-21T15:00:24.802 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3850000000000004e-06 2024-09-21T15:00:24.802 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.802 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.802 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:24.802 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179504+0000", 2024-09-21T15:00:24.802 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037793514 2024-09-21T15:00:24.802 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.802 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.802 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:24.803 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179546+0000", 2024-09-21T15:00:24.803 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1467999999999997e-05 2024-09-21T15:00:24.803 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.803 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.803 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:24.803 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179666+0000", 2024-09-21T15:00:24.803 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000120499 2024-09-21T15:00:24.803 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:24.803 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:24.803 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:24.803 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.803 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.803 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22750 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 835584~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T15:00:24.803 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141715+0000", 2024-09-21T15:00:24.803 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 94.607346676999995, 2024-09-21T15:00:24.804 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038173179000000002, 2024-09-21T15:00:24.804 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:24.804 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:24.804 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:24.804 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:24.804 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:24.804 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22750 2024-09-21T15:00:24.804 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.804 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:24.804 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.804 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:24.804 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141715+0000", 2024-09-21T15:00:24.804 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:24.804 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.804 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.805 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:24.805 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141715+0000", 2024-09-21T15:00:24.805 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:24.805 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.805 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.805 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:24.805 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141714+0000", 2024-09-21T15:00:24.805 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:24.805 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.805 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.805 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:24.805 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141716+0000", 2024-09-21T15:00:24.805 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1730000000000002e-06 2024-09-21T15:00:24.805 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.805 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.805 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:24.806 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141722+0000", 2024-09-21T15:00:24.806 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.417e-06 2024-09-21T15:00:24.806 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.806 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.806 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:24.806 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141726+0000", 2024-09-21T15:00:24.806 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4830000000000001e-06 2024-09-21T15:00:24.806 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.806 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.806 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:24.806 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179683+0000", 2024-09-21T15:00:24.806 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.037956977000000003 2024-09-21T15:00:24.806 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.806 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.806 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:24.807 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179751+0000", 2024-09-21T15:00:24.807 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7762000000000006e-05 2024-09-21T15:00:24.807 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.807 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.807 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:24.807 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179888+0000", 2024-09-21T15:00:24.807 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000136635 2024-09-21T15:00:24.807 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:24.807 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:24.807 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:24.807 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.807 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.807 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22751 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 901120~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T15:00:24.807 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141730+0000", 2024-09-21T15:00:24.807 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 94.607330877999999, 2024-09-21T15:00:24.808 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038357011000000003, 2024-09-21T15:00:24.808 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:24.808 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:24.808 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:24.808 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:24.808 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:24.808 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22751 2024-09-21T15:00:24.808 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.808 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:24.808 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.808 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:24.808 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141730+0000", 2024-09-21T15:00:24.808 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:24.808 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.808 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.808 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:24.808 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141730+0000", 2024-09-21T15:00:24.809 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:24.809 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.809 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.809 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:24.809 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141730+0000", 2024-09-21T15:00:24.809 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:24.809 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.809 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.809 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:24.809 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141737+0000", 2024-09-21T15:00:24.809 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.64e-06 2024-09-21T15:00:24.809 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.809 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.809 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:24.809 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141743+0000", 2024-09-21T15:00:24.809 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8130000000000001e-06 2024-09-21T15:00:24.810 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.810 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.810 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:24.810 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141747+0000", 2024-09-21T15:00:24.810 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4920000000000004e-06 2024-09-21T15:00:24.810 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.810 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.810 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:24.810 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179918+0000", 2024-09-21T15:00:24.810 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038170822 2024-09-21T15:00:24.810 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.810 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.810 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:24.810 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.179960+0000", 2024-09-21T15:00:24.810 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1999000000000002e-05 2024-09-21T15:00:24.811 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.811 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.811 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:24.811 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180087+0000", 2024-09-21T15:00:24.811 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012752799999999999 2024-09-21T15:00:24.811 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:24.811 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:24.811 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:24.811 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.811 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.811 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22752 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 966656~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T15:00:24.811 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141751+0000", 2024-09-21T15:00:24.811 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 94.607310557999995, 2024-09-21T15:00:24.811 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038553084000000001, 2024-09-21T15:00:24.811 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:24.811 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:24.812 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:24.812 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:24.812 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:24.812 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22752 2024-09-21T15:00:24.812 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.812 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:24.812 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.812 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:24.812 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141751+0000", 2024-09-21T15:00:24.812 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:24.812 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.812 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.812 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:24.812 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141751+0000", 2024-09-21T15:00:24.812 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:24.812 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.813 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.813 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:24.813 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141750+0000", 2024-09-21T15:00:24.813 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:24.813 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.813 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.813 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:24.813 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141753+0000", 2024-09-21T15:00:24.813 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5900000000000002e-06 2024-09-21T15:00:24.813 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.813 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.813 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:24.813 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141759+0000", 2024-09-21T15:00:24.813 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.947e-06 2024-09-21T15:00:24.813 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.813 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.813 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:24.814 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141764+0000", 2024-09-21T15:00:24.814 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8319999999999996e-06 2024-09-21T15:00:24.814 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.814 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.814 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:24.814 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180108+0000", 2024-09-21T15:00:24.814 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038344095000000002 2024-09-21T15:00:24.814 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.814 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.814 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:24.814 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180168+0000", 2024-09-21T15:00:24.814 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0047000000000003e-05 2024-09-21T15:00:24.814 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.814 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.815 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:24.815 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180304+0000", 2024-09-21T15:00:24.815 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013592500000000001 2024-09-21T15:00:24.815 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:24.815 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:24.815 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:24.815 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.815 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.815 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:22753 3.a 3:536dec60:::rbd_data.11366acc894.0000000000000003:19 [read 1032192~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-21T15:00:24.815 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:50.141768+0000", 2024-09-21T15:00:24.815 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 94.607293683999998, 2024-09-21T15:00:24.815 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038818810000000002, 2024-09-21T15:00:24.815 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:24.815 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:24.815 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:24.815 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:24.816 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:24.816 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 22753 2024-09-21T15:00:24.816 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.816 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:24.816 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.816 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:24.816 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141768+0000", 2024-09-21T15:00:24.816 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:24.816 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.816 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.816 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:24.816 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141768+0000", 2024-09-21T15:00:24.816 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:24.816 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.816 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.816 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:24.817 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141767+0000", 2024-09-21T15:00:24.817 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:24.817 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.817 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.817 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:24.817 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141770+0000", 2024-09-21T15:00:24.817 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3329999999999999e-06 2024-09-21T15:00:24.817 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.817 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.817 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:24.817 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141775+0000", 2024-09-21T15:00:24.817 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2669999999999997e-06 2024-09-21T15:00:24.817 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.817 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.817 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:24.817 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.141779+0000", 2024-09-21T15:00:24.818 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2259999999999999e-06 2024-09-21T15:00:24.818 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.818 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.818 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:24.818 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180325+0000", 2024-09-21T15:00:24.818 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.038546428000000001 2024-09-21T15:00:24.818 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.818 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.818 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:24.818 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180401+0000", 2024-09-21T15:00:24.818 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5437000000000003e-05 2024-09-21T15:00:24.818 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.818 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.818 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:24.818 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:50.180586+0000", 2024-09-21T15:00:24.819 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000185429 2024-09-21T15:00:24.819 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:24.819 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:24.819 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:24.819 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.819 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.819 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:73793 3.11 3:8c8a112d:::rbd_data.1136f4939602.0000000000000003:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-21T15:00:24.819 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:42.225633+0000", 2024-09-21T15:00:24.819 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 42.523427769999998, 2024-09-21T15:00:24.819 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.036337102000000003, 2024-09-21T15:00:24.819 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:24.819 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:24.819 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:24.819 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:24.819 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:24.820 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 73793 2024-09-21T15:00:24.820 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.820 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:24.820 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.820 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:24.820 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.225633+0000", 2024-09-21T15:00:24.820 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:24.820 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.820 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.820 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:24.820 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.225633+0000", 2024-09-21T15:00:24.820 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:24.820 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.820 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.820 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:24.821 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.225630+0000", 2024-09-21T15:00:24.821 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999967 2024-09-21T15:00:24.821 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.821 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.821 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:24.821 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.225649+0000", 2024-09-21T15:00:24.821 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9377000000000001e-05 2024-09-21T15:00:24.821 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.821 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.821 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:24.821 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.225689+0000", 2024-09-21T15:00:24.821 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.9879999999999998e-05 2024-09-21T15:00:24.821 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.821 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.821 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:24.821 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.225706+0000", 2024-09-21T15:00:24.822 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6537000000000001e-05 2024-09-21T15:00:24.822 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.822 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.822 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:24.822 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.225772+0000", 2024-09-21T15:00:24.822 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5894000000000003e-05 2024-09-21T15:00:24.822 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.822 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.822 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:24.822 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.225846+0000", 2024-09-21T15:00:24.822 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4165000000000006e-05 2024-09-21T15:00:24.822 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.822 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.822 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 1", 2024-09-21T15:00:24.822 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.231400+0000", 2024-09-21T15:00:24.822 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0055538269999999999 2024-09-21T15:00:24.823 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.823 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.823 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:24.823 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.261709+0000", 2024-09-21T15:00:24.823 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.030309750999999999 2024-09-21T15:00:24.823 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.823 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.823 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:24.823 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.261738+0000", 2024-09-21T15:00:24.823 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8569999999999999e-05 2024-09-21T15:00:24.823 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.823 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.823 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:24.823 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.261839+0000", 2024-09-21T15:00:24.823 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010118 2024-09-21T15:00:24.823 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.824 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.824 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:24.824 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:42.261971+0000", 2024-09-21T15:00:24.824 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000131414 2024-09-21T15:00:24.824 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:24.824 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:24.824 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:24.824 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.824 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.824 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:97673 3.1d 3:bd91946e:::rbd_data.1136bfc79a70.0000000000000002:head [write 0~4718592 [fadvise_sequential+fadvise_nocache],write 5242880~32768 [fadvise_sequential+fadvise_nocache],write 5341184~147456 [fadvise_sequential+fadvise_nocache],write 5554176~147456 [fadvise_sequential+fadvise_nocache],write 5767168~16384 [fadvise_sequential+fadvise_nocache],write 5849088~147456 [fadvise_sequential+fadvise_nocache],write 6062080~147456 [fadvise_sequential+fadvise_nocache],write 6275072~16384 [fadvise_sequential+fadvise_nocache],write 6356992~147456 [fadvise_sequential+fadvise_nocache],write 6569984~147456 [fadvise_sequential+fadvise_nocache],write 6782976~32768 [fadvise_sequential+fadvise_nocache],write 6864896~147456 [fadvise_sequential+fadvise_nocache],write 7077888~147456 [fadvise_sequential+fadvise_nocache],write 7290880~49152 [fadvise_sequential+fadvise_nocache],write 7372800~147456 [fadvise_sequential+fadvise_nocache],write 7585792~147456 [fadvise_sequential+fadvise_nocache],write 7798784~65536 [fadvise_sequential+fadvise_nocache],write 7880704~147456 [fadvise_sequential+fadvise_nocache],write 8093696~147456 [fadvise_sequential+fadvise_nocache],write 8306688~81920 [fadvise_sequential+fadvise_nocache],write 8421376~114688 [fadvise_sequential+fadvise_nocache],write 8634368~114688 [fadvise_sequential+fadvise_nocache],write 8847360~65536 [fadvise_sequential+fadvise_nocache],write 9011200~32768 [fadvise_sequential+fadvise_nocache],write 9224192~32768 [fadvise_sequential+fadvise_nocache],write 9519104~32768 [fadvise_sequential+fadvise_nocache],write 9732096~32768 [fadvise_sequential+fadvise_nocache],write 9945088~16384 [fadvise_sequential+fadvise_nocache],write 9977856~196608 [fadvise_sequential+fadvise_nocache],write 10190848~196608 [fadvise_sequential+fadvise_nocache],write 10403840~278528 [fadvise_sequential+fadvise_nocache],write 10698752~196608 [fadvise_sequential+fadvise_nocache],write 10911744~278528 [fadvise_sequential+fadvise_nocache],write 11206656~196608 [fadvise_sequential+fadvise_nocache],write 11419648~196608 [fadvise_sequential+fadvise_nocache],write 11632640~65536 [fadvise_sequential+fadvise_nocache],write 11730944~98304 [fadvise_sequential+fadvise_nocache],write 11845632~65536 [fadvise_sequential+fadvise_nocache],write 11943936~98304 [fadvise_sequential+fadvise_nocache],write 12058624~65536 [fadvise_sequential+fadvise_nocache],write 12140544~65536 [fadvise_sequential+fadvise_nocache],write 12238848~98304 [fadvise_sequential+fadvise_nocache],write 12353536~65536 [fadvise_sequential+fadvise_nocache],write 12451840~98304 [fadvise_sequential+fadvise_nocache],write 12566528~65536 [fadvise_sequential+fadvise_nocache],write 12746752~98304 [fadvise_sequential+fadvise_nocache],write 12959744~98304 [fadvise_sequential+fadvise_nocache],write 13107200~32768 [fadvise_sequential+fadvise_nocache],write 13254656~98304 [fadvise_sequential+fadvise_nocache],write 13467648~98304 [fadvise_sequential+fadvise_nocache],write 13631488~16384 [fadvise_sequential+fadvise_nocache],write 13762560~98304 [fadvise_sequential+fadvise_nocache],write 13975552~98304 [fadvise_sequential+fadvise_nocache],write 15204352~49152 [fadvise_sequential+fadvise_nocache],write 15384576~81920 [fadvise_sequential+fadvise_nocache],write 15597568~81920 [fadvise_sequential+fadvise_nocache],write 15728640~32768 [fadvise_sequential+fadvise_nocache],write 15892480~81920 [fadvise_sequential+fadvise_nocache],write 16105472~81920 [fadvise_sequential+fadvise_nocache],write 16252928~16384 [fadvise_sequential+fadvise_nocache],write 16400384~81920 [fadvise_sequential+fadvise_nocache],write 16613376~81920 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:24.824 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.300719+0000", 2024-09-21T15:00:24.824 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 19.448342495999999, 2024-09-21T15:00:24.824 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.068947114000000004, 2024-09-21T15:00:24.825 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:24.825 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:24.825 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:24.825 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:24.825 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:24.825 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 97673 2024-09-21T15:00:24.825 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.825 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:24.825 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.825 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:24.825 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.300719+0000", 2024-09-21T15:00:24.825 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:24.825 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.825 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.826 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:24.826 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.300719+0000", 2024-09-21T15:00:24.826 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:24.826 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.826 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.826 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:24.826 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.300716+0000", 2024-09-21T15:00:24.826 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-21T15:00:24.826 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.826 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.826 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:24.826 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.308380+0000", 2024-09-21T15:00:24.826 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0076638519999999996 2024-09-21T15:00:24.826 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.826 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.827 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:24.827 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.308418+0000", 2024-09-21T15:00:24.827 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.8380000000000002e-05 2024-09-21T15:00:24.827 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.827 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.827 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:24.827 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.308431+0000", 2024-09-21T15:00:24.827 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3213999999999999e-05 2024-09-21T15:00:24.827 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.827 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.827 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:24.827 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.308562+0000", 2024-09-21T15:00:24.827 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013015999999999999 2024-09-21T15:00:24.827 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.827 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.828 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:24.828 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.309173+0000", 2024-09-21T15:00:24.828 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00061185500000000002 2024-09-21T15:00:24.828 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.828 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.828 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 0", 2024-09-21T15:00:24.828 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311612+0000", 2024-09-21T15:00:24.828 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0024383949999999999 2024-09-21T15:00:24.828 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.828 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.828 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:24.828 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.355482+0000", 2024-09-21T15:00:24.828 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.043869940000000003 2024-09-21T15:00:24.828 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.829 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.829 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:24.829 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.369306+0000", 2024-09-21T15:00:24.829 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013824279 2024-09-21T15:00:24.829 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.829 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.829 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:24.829 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.369575+0000", 2024-09-21T15:00:24.829 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000269404 2024-09-21T15:00:24.829 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.829 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.829 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:24.829 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.369666+0000", 2024-09-21T15:00:24.829 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0426000000000005e-05 2024-09-21T15:00:24.829 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:24.829 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:24.830 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:24.830 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.830 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.830 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:97678 3.d 3:b2f34c6d:::rbd_data.1136bfc79a70.0000000000000006:head [write 0~4194304 [fadvise_sequential+fadvise_nocache],write 4718592~81920 [fadvise_sequential+fadvise_nocache],write 4866048~147456 [fadvise_sequential+fadvise_nocache],write 5079040~147456 [fadvise_sequential+fadvise_nocache],write 5242880~65536 [fadvise_sequential+fadvise_nocache],write 5373952~147456 [fadvise_sequential+fadvise_nocache],write 5586944~147456 [fadvise_sequential+fadvise_nocache],write 5767168~49152 [fadvise_sequential+fadvise_nocache],write 5881856~147456 [fadvise_sequential+fadvise_nocache],write 6094848~147456 [fadvise_sequential+fadvise_nocache],write 6291456~32768 [fadvise_sequential+fadvise_nocache],write 6389760~147456 [fadvise_sequential+fadvise_nocache],write 6602752~147456 [fadvise_sequential+fadvise_nocache],write 6815744~16384 [fadvise_sequential+fadvise_nocache],write 6897664~147456 [fadvise_sequential+fadvise_nocache],write 7110656~147456 [fadvise_sequential+fadvise_nocache],write 7323648~16384 [fadvise_sequential+fadvise_nocache],write 7405568~147456 [fadvise_sequential+fadvise_nocache],write 7618560~147456 [fadvise_sequential+fadvise_nocache],write 7831552~32768 [fadvise_sequential+fadvise_nocache],write 7946240~114688 [fadvise_sequential+fadvise_nocache],write 8159232~114688 [fadvise_sequential+fadvise_nocache],write 8372224~16384 [fadvise_sequential+fadvise_nocache],write 8454144~114688 [fadvise_sequential+fadvise_nocache],write 8667136~114688 [fadvise_sequential+fadvise_nocache],write 8880128~32768 [fadvise_sequential+fadvise_nocache],write 9043968~32768 [fadvise_sequential+fadvise_nocache],write 9256960~32768 [fadvise_sequential+fadvise_nocache],write 9551872~32768 [fadvise_sequential+fadvise_nocache],write 9764864~32768 [fadvise_sequential+fadvise_nocache],write 9961472~32768 [fadvise_sequential+fadvise_nocache],write 10010624~196608 [fadvise_sequential+fadvise_nocache],write 10223616~196608 [fadvise_sequential+fadvise_nocache],write 10436608~65536 [fadvise_sequential+fadvise_nocache],write 10518528~196608 [fadvise_sequential+fadvise_nocache],write 10731520~196608 [fadvise_sequential+fadvise_nocache],write 10944512~65536 [fadvise_sequential+fadvise_nocache],write 11026432~196608 [fadvise_sequential+fadvise_nocache],write 11239424~196608 [fadvise_sequential+fadvise_nocache],write 11452416~81920 [fadvise_sequential+fadvise_nocache],write 11550720~98304 [fadvise_sequential+fadvise_nocache],write 11665408~65536 [fadvise_sequential+fadvise_nocache],write 11763712~98304 [fadvise_sequential+fadvise_nocache],write 11878400~65536 [fadvise_sequential+fadvise_nocache],write 11976704~180224 [fadvise_sequential+fadvise_nocache],write 12173312~65536 [fadvise_sequential+fadvise_nocache],write 12271616~98304 [fadvise_sequential+fadvise_nocache],write 12386304~65536 [fadvise_sequential+fadvise_nocache],write 12484608~180224 [fadvise_sequential+fadvise_nocache],write 12779520~98304 [fadvise_sequential+fadvise_nocache],write 12992512~98304 [fadvise_sequential+fadvise_nocache],write 13107200~65536 [fadvise_sequential+fadvise_nocache],write 13287424~98304 [fadvise_sequential+fadvise_nocache],write 13500416~98304 [fadvise_sequential+fadvise_nocache],write 13631488~49152 [fadvise_sequential+fadvise_nocache],write 13795328~98304 [fadvise_sequential+fadvise_nocache],write 14008320~98304 [fadvise_sequential+fadvise_nocache],write 14696448~81920 [fadvise_sequential+fadvise_nocache],write 14909440~81920 [fadvise_sequential+fadvise_nocache],write 15122432~163840 [fadvise_sequential+fadvise_nocache],write 15417344~81920 [fadvise_sequential+fadvise_nocache],write 15630336~81920 [fadvise_sequential+fadvise_nocache],write 15728640~65536 [fadvise_sequential+fadvise_nocache],write 15925248~81920 [fadvise_sequential+fadvise_nocache],write 16138240~81920 [fadvise_sequential+fadvise_nocache],write 16252928~49152 [fadvise_sequential+fadvise_nocache],write 16433152~81920 [fadvise_sequential+fadvise_nocache],write 16646144~81920 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:24.830 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.308471+0000", 2024-09-21T15:00:24.830 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 19.440590045, 2024-09-21T15:00:24.830 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.047271776000000001, 2024-09-21T15:00:24.830 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:24.830 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:24.830 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:24.830 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:24.830 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:24.830 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 97678 2024-09-21T15:00:24.830 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.831 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:24.831 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.831 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:24.831 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.308471+0000", 2024-09-21T15:00:24.831 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:24.831 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.831 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.831 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:24.831 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.308471+0000", 2024-09-21T15:00:24.831 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:24.831 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.831 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.831 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:24.831 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.308469+0000", 2024-09-21T15:00:24.831 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-21T15:00:24.832 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.832 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.832 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:24.832 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.317731+0000", 2024-09-21T15:00:24.832 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0092614949999999998 2024-09-21T15:00:24.832 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.832 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.832 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:24.832 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.317778+0000", 2024-09-21T15:00:24.832 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7574000000000002e-05 2024-09-21T15:00:24.832 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.832 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.832 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:24.832 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.317794+0000", 2024-09-21T15:00:24.832 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5537e-05 2024-09-21T15:00:24.832 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.833 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.833 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:24.833 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.317871+0000", 2024-09-21T15:00:24.833 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7163999999999995e-05 2024-09-21T15:00:24.833 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.833 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.833 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:24.833 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.318166+0000", 2024-09-21T15:00:24.833 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00029492400000000001 2024-09-21T15:00:24.833 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.833 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.833 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 1", 2024-09-21T15:00:24.833 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.320083+0000", 2024-09-21T15:00:24.833 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001916933 2024-09-21T15:00:24.834 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.834 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.834 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:24.834 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.354762+0000", 2024-09-21T15:00:24.834 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034679481999999998 2024-09-21T15:00:24.834 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.834 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.834 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:24.834 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.355494+0000", 2024-09-21T15:00:24.834 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00073167399999999997 2024-09-21T15:00:24.834 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.834 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.834 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:24.835 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.355683+0000", 2024-09-21T15:00:24.835 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00018951699999999999 2024-09-21T15:00:24.835 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.835 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.835 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:24.835 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.355743+0000", 2024-09-21T15:00:24.835 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9493000000000001e-05 2024-09-21T15:00:24.835 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:24.835 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:24.835 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:24.835 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.835 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.835 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:98562 3.9 3:911e52bc:::rbd_data.1136bfc79a70.000000000000000a:head [write 0~4194304 [fadvise_sequential+fadvise_nocache],write 4718592~114688 [fadvise_sequential+fadvise_nocache],write 4898816~147456 [fadvise_sequential+fadvise_nocache],write 5111808~229376 [fadvise_sequential+fadvise_nocache],write 5406720~147456 [fadvise_sequential+fadvise_nocache],write 5619712~229376 [fadvise_sequential+fadvise_nocache],write 5914624~147456 [fadvise_sequential+fadvise_nocache],write 6127616~147456 [fadvise_sequential+fadvise_nocache],write 6291456~65536 [fadvise_sequential+fadvise_nocache],write 6422528~147456 [fadvise_sequential+fadvise_nocache],write 6635520~147456 [fadvise_sequential+fadvise_nocache],write 6815744~49152 [fadvise_sequential+fadvise_nocache],write 6930432~147456 [fadvise_sequential+fadvise_nocache],write 7143424~147456 [fadvise_sequential+fadvise_nocache],write 7340032~32768 [fadvise_sequential+fadvise_nocache],write 7438336~147456 [fadvise_sequential+fadvise_nocache],write 7651328~147456 [fadvise_sequential+fadvise_nocache],write 7864320~16384 [fadvise_sequential+fadvise_nocache],write 7979008~114688 [fadvise_sequential+fadvise_nocache],write 8192000~114688 [fadvise_sequential+fadvise_nocache],write 8519680~81920 [fadvise_sequential+fadvise_nocache],write 8732672~38263 [fadvise_sequential+fadvise_nocache],write 8781824~32768 [fadvise_sequential+fadvise_nocache],write 9076736~32768 [fadvise_sequential+fadvise_nocache],write 9289728~32768 [fadvise_sequential+fadvise_nocache],write 9437184~81920 [fadvise_sequential+fadvise_nocache],write 9535488~196608 [fadvise_sequential+fadvise_nocache],write 9748480~196608 [fadvise_sequential+fadvise_nocache],write 9961472~65536 [fadvise_sequential+fadvise_nocache],write 10043392~196608 [fadvise_sequential+fadvise_nocache],write 10256384~196608 [fadvise_sequential+fadvise_nocache],write 10469376~65536 [fadvise_sequential+fadvise_nocache],write 10551296~196608 [fadvise_sequential+fadvise_nocache],write 10764288~196608 [fadvise_sequential+fadvise_nocache],write 10977280~65536 [fadvise_sequential+fadvise_nocache],write 11075584~98304 [fadvise_sequential+fadvise_nocache],write 11190272~65536 [fadvise_sequential+fadvise_nocache],write 11288576~98304 [fadvise_sequential+fadvise_nocache],write 11403264~65536 [fadvise_sequential+fadvise_nocache],write 11501568~49152 [fadvise_sequential+fadvise_nocache],write 11583488~98304 [fadvise_sequential+fadvise_nocache],write 11698176~65536 [fadvise_sequential+fadvise_nocache],write 11796480~98304 [fadvise_sequential+fadvise_nocache],write 11911168~65536 [fadvise_sequential+fadvise_nocache],write 12009472~49152 [fadvise_sequential+fadvise_nocache],write 12091392~98304 [fadvise_sequential+fadvise_nocache],write 12304384~98304 [fadvise_sequential+fadvise_nocache],write 12517376~65536 [fadvise_sequential+fadvise_nocache],write 12599296~98304 [fadvise_sequential+fadvise_nocache],write 12812288~98304 [fadvise_sequential+fadvise_nocache],write 13025280~180224 [fadvise_sequential+fadvise_nocache],write 13320192~98304 [fadvise_sequential+fadvise_nocache],write 13533184~180224 [fadvise_sequential+fadvise_nocache],write 13828096~98304 [fadvise_sequential+fadvise_nocache],write 14041088~98304 [fadvise_sequential+fadvise_nocache],write 14729216~81920 [fadvise_sequential+fadvise_nocache],write 14942208~81920 [fadvise_sequential+fadvise_nocache],write 15155200~49152 [fadvise_sequential+fadvise_nocache],write 15237120~81920 [fadvise_sequential+fadvise_nocache],write 15450112~81920 [fadvise_sequential+fadvise_nocache],write 15663104~65536 [fadvise_sequential+fadvise_nocache],write 15745024~81920 [fadvise_sequential+fadvise_nocache],write 15958016~81920 [fadvise_sequential+fadvise_nocache],write 16171008~163840 [fadvise_sequential+fadvise_nocache],write 16465920~81920 [fadvise_sequential+fadvise_nocache],write 16678912~81920 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:24.835 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.422553+0000", 2024-09-21T15:00:24.835 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 19.326508339, 2024-09-21T15:00:24.835 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042104691, 2024-09-21T15:00:24.836 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:24.836 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:24.836 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:24.836 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:24.836 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:24.836 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 98562 2024-09-21T15:00:24.836 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.836 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:24.836 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.836 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:24.836 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.422553+0000", 2024-09-21T15:00:24.836 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:24.836 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.836 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.836 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:24.837 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.422553+0000", 2024-09-21T15:00:24.837 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:24.837 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.837 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.837 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:24.837 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.422551+0000", 2024-09-21T15:00:24.837 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-21T15:00:24.837 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.837 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.837 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:24.837 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.429465+0000", 2024-09-21T15:00:24.837 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0069139550000000003 2024-09-21T15:00:24.837 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.837 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.837 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:24.837 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.429847+0000", 2024-09-21T15:00:24.838 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00038256 2024-09-21T15:00:24.838 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.838 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.838 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:24.838 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.429866+0000", 2024-09-21T15:00:24.838 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8051e-05 2024-09-21T15:00:24.838 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.838 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.838 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:24.838 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.429946+0000", 2024-09-21T15:00:24.838 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0761999999999997e-05 2024-09-21T15:00:24.838 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.838 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.838 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:24.839 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.430257+0000", 2024-09-21T15:00:24.839 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00031120200000000001 2024-09-21T15:00:24.839 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.839 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.839 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 4", 2024-09-21T15:00:24.839 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.432045+0000", 2024-09-21T15:00:24.839 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0017872879999999999 2024-09-21T15:00:24.839 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.839 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.839 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:24.839 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.451614+0000", 2024-09-21T15:00:24.839 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019569521999999999 2024-09-21T15:00:24.839 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.839 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.839 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:24.840 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.464472+0000", 2024-09-21T15:00:24.840 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.012857435 2024-09-21T15:00:24.840 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.840 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.840 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:24.840 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.464613+0000", 2024-09-21T15:00:24.840 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014159700000000001 2024-09-21T15:00:24.840 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:24.840 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:24.840 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:24.840 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.464658+0000", 2024-09-21T15:00:24.840 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4268999999999999e-05 2024-09-21T15:00:24.840 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:24.840 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:24.840 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:24.841 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:24.841 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:24.841 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:24.842 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T15:00:24.842 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:00:24.840+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:00:24.943 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:00:24.938+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:00:24.961 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:24.961 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:00:24.962 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T15:00:24.962 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:24.972 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T15:00:25.043 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:00:25.038+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:00:25.076 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:25.077 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:00:25.077 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T15:00:25.077 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T15:00:25.077 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:25.086 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T15:00:25.145 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:00:25.144+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:00:25.203 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:25.203 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T15:00:25.203 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T15:00:25.203 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T15:00:25.203 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.203 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:1452 3.19 3:9d40fe72:::rbd_data.1136515ecb6b.0000000000000013:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T15:00:25.203 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:30.292755+0000", 2024-09-21T15:00:25.203 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 114.906923515, 2024-09-21T15:00:25.203 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.043852191999999998, 2024-09-21T15:00:25.203 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:25.203 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:25.204 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:25.204 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:25.204 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:25.204 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1452 2024-09-21T15:00:25.204 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.204 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:25.204 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.204 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:25.204 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292755+0000", 2024-09-21T15:00:25.204 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:25.204 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.204 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.204 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:25.204 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292755+0000", 2024-09-21T15:00:25.205 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:25.205 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.205 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.205 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:25.205 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292754+0000", 2024-09-21T15:00:25.205 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:00:25.205 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.205 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.205 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:25.205 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292760+0000", 2024-09-21T15:00:25.205 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5029999999999997e-06 2024-09-21T15:00:25.205 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.205 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.205 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:25.205 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292771+0000", 2024-09-21T15:00:25.206 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1551e-05 2024-09-21T15:00:25.206 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.206 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.206 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:25.206 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292779+0000", 2024-09-21T15:00:25.206 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1559999999999998e-06 2024-09-21T15:00:25.206 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.206 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.206 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:25.206 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.292896+0000", 2024-09-21T15:00:25.206 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000117304 2024-09-21T15:00:25.206 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.206 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.206 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:25.207 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.293124+0000", 2024-09-21T15:00:25.207 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00022819999999999999 2024-09-21T15:00:25.207 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.207 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.207 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 1", 2024-09-21T15:00:25.207 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.299967+0000", 2024-09-21T15:00:25.207 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0068427009999999996 2024-09-21T15:00:25.207 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.207 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.207 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:25.207 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336463+0000", 2024-09-21T15:00:25.207 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.036495842000000001 2024-09-21T15:00:25.207 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.208 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.208 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:25.208 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336480+0000", 2024-09-21T15:00:25.208 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7419999999999999e-05 2024-09-21T15:00:25.208 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.208 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.208 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:25.208 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336550+0000", 2024-09-21T15:00:25.208 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9951999999999997e-05 2024-09-21T15:00:25.208 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.208 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.208 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:25.208 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.336607+0000", 2024-09-21T15:00:25.208 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7423000000000002e-05 2024-09-21T15:00:25.208 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:25.208 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:25.209 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:25.209 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.209 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.209 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:1567 3.8 3:139d3d55:::rbd_data.1136515ecb6b.0000000000000000:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T15:00:25.209 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:30.681087+0000", 2024-09-21T15:00:25.209 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 114.5185913, 2024-09-21T15:00:25.209 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.039009224000000002, 2024-09-21T15:00:25.209 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:25.209 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:25.209 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:25.209 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:25.209 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:25.209 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 1567 2024-09-21T15:00:25.209 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.209 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:25.210 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.210 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:25.210 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681087+0000", 2024-09-21T15:00:25.210 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:25.210 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.210 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.210 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:25.210 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681087+0000", 2024-09-21T15:00:25.210 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:25.210 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.210 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.210 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:25.210 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681086+0000", 2024-09-21T15:00:25.210 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:00:25.210 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.211 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.211 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:25.211 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681097+0000", 2024-09-21T15:00:25.211 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.134e-05 2024-09-21T15:00:25.211 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.211 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.211 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:25.211 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681122+0000", 2024-09-21T15:00:25.211 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5313e-05 2024-09-21T15:00:25.211 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.211 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.211 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:25.211 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681138+0000", 2024-09-21T15:00:25.211 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5916000000000001e-05 2024-09-21T15:00:25.211 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.211 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.212 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:25.212 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681270+0000", 2024-09-21T15:00:25.212 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000131474 2024-09-21T15:00:25.212 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.212 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.212 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:25.212 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.681416+0000", 2024-09-21T15:00:25.212 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014637 2024-09-21T15:00:25.212 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.212 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.212 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 7", 2024-09-21T15:00:25.212 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.693163+0000", 2024-09-21T15:00:25.212 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011747241 2024-09-21T15:00:25.212 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.213 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.213 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:25.213 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.719750+0000", 2024-09-21T15:00:25.213 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.026586852000000001 2024-09-21T15:00:25.213 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.213 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.213 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:25.213 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.719859+0000", 2024-09-21T15:00:25.213 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010925999999999999 2024-09-21T15:00:25.213 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.213 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.213 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:25.213 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.719951+0000", 2024-09-21T15:00:25.213 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1294999999999999e-05 2024-09-21T15:00:25.213 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.214 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.214 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:25.214 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:30.720097+0000", 2024-09-21T15:00:25.214 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014583 2024-09-21T15:00:25.214 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:25.214 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:25.214 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:25.214 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.214 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.214 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:37892 3.1b 3:d963c97f:::rbd_data.11365d19aa81.00000000000000be:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-21T15:00:25.214 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:04.029946+0000", 2024-09-21T15:00:25.214 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 81.169733159000003, 2024-09-21T15:00:25.214 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.017686021999999999, 2024-09-21T15:00:25.214 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:25.214 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:25.215 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:25.215 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:25.215 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:25.215 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 37892 2024-09-21T15:00:25.215 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.215 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:25.215 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.215 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:25.215 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.029946+0000", 2024-09-21T15:00:25.215 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:25.215 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.215 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.215 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:25.215 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.029946+0000", 2024-09-21T15:00:25.215 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:25.216 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.216 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.216 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:25.216 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.029944+0000", 2024-09-21T15:00:25.216 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:00:25.216 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.216 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.216 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:25.216 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.029953+0000", 2024-09-21T15:00:25.216 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7589999999999992e-06 2024-09-21T15:00:25.216 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.216 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.216 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:25.216 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.029971+0000", 2024-09-21T15:00:25.216 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7795000000000002e-05 2024-09-21T15:00:25.216 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.217 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.217 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:25.217 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.029984+0000", 2024-09-21T15:00:25.217 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.27e-05 2024-09-21T15:00:25.217 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.217 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.217 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:25.217 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.030072+0000", 2024-09-21T15:00:25.217 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8082000000000001e-05 2024-09-21T15:00:25.217 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.217 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.217 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:25.217 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.030276+0000", 2024-09-21T15:00:25.218 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00020375699999999999 2024-09-21T15:00:25.218 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.218 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.218 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-21T15:00:25.218 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.031634+0000", 2024-09-21T15:00:25.218 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0013586049999999999 2024-09-21T15:00:25.218 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.218 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.218 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:25.218 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.037554+0000", 2024-09-21T15:00:25.218 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0059197449999999997 2024-09-21T15:00:25.218 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.218 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.218 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:25.219 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.047490+0000", 2024-09-21T15:00:25.219 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0099361680000000004 2024-09-21T15:00:25.219 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.219 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.219 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:25.219 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.047569+0000", 2024-09-21T15:00:25.219 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8298000000000001e-05 2024-09-21T15:00:25.219 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.219 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.219 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:25.219 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:04.047632+0000", 2024-09-21T15:00:25.219 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3006e-05 2024-09-21T15:00:25.219 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:25.219 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:25.220 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:25.220 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.220 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.220 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:63820 3.19 3.626bc079 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-21T15:00:25.220 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:27.241293+0000", 2024-09-21T15:00:25.220 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 57.958386066000003, 2024-09-21T15:00:25.220 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.98723808199999996, 2024-09-21T15:00:25.220 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:25.220 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:25.220 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:25.220 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:25.220 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:25.220 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 63820 2024-09-21T15:00:25.220 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.221 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:25.221 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.221 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:25.221 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241293+0000", 2024-09-21T15:00:25.221 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:25.221 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.221 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.221 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:25.221 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241293+0000", 2024-09-21T15:00:25.221 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:25.221 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.221 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.221 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:25.221 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241291+0000", 2024-09-21T15:00:25.221 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:00:25.222 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.222 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.222 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:25.222 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241299+0000", 2024-09-21T15:00:25.222 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1529999999999997e-06 2024-09-21T15:00:25.222 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.222 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.222 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:25.222 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241317+0000", 2024-09-21T15:00:25.222 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7764000000000001e-05 2024-09-21T15:00:25.222 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.222 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.222 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:25.222 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241328+0000", 2024-09-21T15:00:25.222 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1245e-05 2024-09-21T15:00:25.223 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.223 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.223 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:25.223 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241407+0000", 2024-09-21T15:00:25.223 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8762000000000002e-05 2024-09-21T15:00:25.223 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.223 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.223 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:00:25.223 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241418+0000", 2024-09-21T15:00:25.223 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1695000000000001e-05 2024-09-21T15:00:25.223 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.223 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.223 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:25.223 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:28.228510+0000", 2024-09-21T15:00:25.224 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.98709166100000001 2024-09-21T15:00:25.224 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.224 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.224 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:25.224 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:28.228531+0000", 2024-09-21T15:00:25.224 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0696e-05 2024-09-21T15:00:25.224 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:25.224 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:25.224 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:25.224 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.224 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.224 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:73845 3.6 3:601bd01e:::rbd_data.1136f4939602.0000000000000015:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-21T15:00:25.224 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:42.393435+0000", 2024-09-21T15:00:25.224 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.806243954999999, 2024-09-21T15:00:25.225 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.030713345, 2024-09-21T15:00:25.225 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:25.225 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:25.225 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:25.225 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:25.225 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:25.225 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 73845 2024-09-21T15:00:25.225 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.225 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:25.225 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.225 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:25.225 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.393435+0000", 2024-09-21T15:00:25.225 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:25.225 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.225 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.226 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:25.226 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.393435+0000", 2024-09-21T15:00:25.226 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:25.226 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.226 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.226 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:25.226 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.393433+0000", 2024-09-21T15:00:25.226 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:00:25.226 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.226 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.226 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:25.226 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.393446+0000", 2024-09-21T15:00:25.226 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3013000000000001e-05 2024-09-21T15:00:25.226 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.227 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.227 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:25.227 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.393475+0000", 2024-09-21T15:00:25.227 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9615999999999999e-05 2024-09-21T15:00:25.227 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.227 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.227 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:25.227 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.393489+0000", 2024-09-21T15:00:25.227 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4103e-05 2024-09-21T15:00:25.227 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.227 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.227 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:25.227 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.393565+0000", 2024-09-21T15:00:25.227 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5195000000000001e-05 2024-09-21T15:00:25.227 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.228 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.228 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:25.228 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.393734+0000", 2024-09-21T15:00:25.228 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016909999999999999 2024-09-21T15:00:25.228 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.228 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.228 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 7", 2024-09-21T15:00:25.228 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.400243+0000", 2024-09-21T15:00:25.228 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0065093629999999998 2024-09-21T15:00:25.228 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.228 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.228 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:25.228 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.423721+0000", 2024-09-21T15:00:25.228 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.023477984 2024-09-21T15:00:25.229 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.229 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.229 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:25.229 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.423852+0000", 2024-09-21T15:00:25.229 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000131369 2024-09-21T15:00:25.229 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.229 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.229 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:25.229 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.424011+0000", 2024-09-21T15:00:25.229 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000158231 2024-09-21T15:00:25.229 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.229 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.229 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:25.229 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:42.424148+0000", 2024-09-21T15:00:25.229 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013732699999999999 2024-09-21T15:00:25.230 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:25.230 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:25.230 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:25.230 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.230 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.230 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:93347 3.9 e119/90 3:92bf9a56:::rbd_object_map.1136b9c49b1c:head v 119'420, mlcod=119'415)", 2024-09-21T15:00:25.230 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:02.217952+0000", 2024-09-21T15:00:25.230 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.981726604999999, 2024-09-21T15:00:25.230 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.016462039000000001, 2024-09-21T15:00:25.230 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:25.230 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:25.230 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:25.230 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.230 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:25.230 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:02.217952+0000", 2024-09-21T15:00:25.231 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:25.231 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.231 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.231 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:25.231 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:02.217952+0000", 2024-09-21T15:00:25.231 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:25.231 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.231 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.231 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:25.231 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:02.217951+0000", 2024-09-21T15:00:25.231 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:00:25.231 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.231 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.231 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:25.231 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:02.217963+0000", 2024-09-21T15:00:25.232 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2197e-05 2024-09-21T15:00:25.232 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.232 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.232 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:25.232 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:02.217997+0000", 2024-09-21T15:00:25.232 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4468999999999998e-05 2024-09-21T15:00:25.232 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.232 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.232 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:25.232 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:02.218012+0000", 2024-09-21T15:00:25.232 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.452e-05 2024-09-21T15:00:25.232 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.232 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.232 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:25.233 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:02.218095+0000", 2024-09-21T15:00:25.233 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2817000000000003e-05 2024-09-21T15:00:25.233 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.233 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.233 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:25.233 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:02.218126+0000", 2024-09-21T15:00:25.233 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0814e-05 2024-09-21T15:00:25.233 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.233 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.233 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:25.233 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:02.234371+0000", 2024-09-21T15:00:25.233 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.016245384000000002 2024-09-21T15:00:25.233 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.233 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.233 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:25.234 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:02.234414+0000", 2024-09-21T15:00:25.234 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3170999999999997e-05 2024-09-21T15:00:25.234 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:25.234 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:25.234 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:25.234 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.234 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.234 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:93349 3.19 3:9a74bbc0:::rbd_data.1136b9c49b1c.000000000000001c:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-21T15:00:25.234 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:02.218917+0000", 2024-09-21T15:00:25.234 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 22.980761192999999, 2024-09-21T15:00:25.234 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015624626000000001, 2024-09-21T15:00:25.234 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:25.234 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:25.234 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:25.235 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:25.235 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:25.235 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 93349 2024-09-21T15:00:25.235 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.235 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:25.235 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.235 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:25.235 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:02.218917+0000", 2024-09-21T15:00:25.235 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:25.235 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.235 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.235 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:25.235 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:02.218917+0000", 2024-09-21T15:00:25.235 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:25.235 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.236 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.236 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:25.236 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:02.218916+0000", 2024-09-21T15:00:25.236 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:00:25.236 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.236 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.236 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:25.236 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:02.218931+0000", 2024-09-21T15:00:25.236 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4712000000000001e-05 2024-09-21T15:00:25.236 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.236 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.236 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:25.236 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:02.218949+0000", 2024-09-21T15:00:25.236 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8094999999999999e-05 2024-09-21T15:00:25.236 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.237 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.237 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:25.237 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:02.218958+0000", 2024-09-21T15:00:25.237 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3670000000000003e-06 2024-09-21T15:00:25.237 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.237 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.237 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:25.237 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:02.219029+0000", 2024-09-21T15:00:25.237 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0859999999999996e-05 2024-09-21T15:00:25.237 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.237 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.237 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:25.237 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:02.219159+0000", 2024-09-21T15:00:25.237 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000130211 2024-09-21T15:00:25.238 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.238 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.238 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 1", 2024-09-21T15:00:25.238 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:02.222970+0000", 2024-09-21T15:00:25.238 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0038108970000000002 2024-09-21T15:00:25.238 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.238 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.238 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:25.238 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:02.234423+0000", 2024-09-21T15:00:25.238 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011452413 2024-09-21T15:00:25.238 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.238 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.238 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:25.238 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:02.234445+0000", 2024-09-21T15:00:25.238 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2367000000000001e-05 2024-09-21T15:00:25.239 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.239 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.239 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:25.239 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:02.234499+0000", 2024-09-21T15:00:25.239 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3634999999999997e-05 2024-09-21T15:00:25.239 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.239 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.239 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:25.239 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:02.234542+0000", 2024-09-21T15:00:25.239 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3356999999999998e-05 2024-09-21T15:00:25.239 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:25.239 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:25.239 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:25.239 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.239 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.240 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:97619 3.1 3:8730dd9c:::rbd_data.1136bfc79a70.0000000000000000:head [write 0~4718592 [fadvise_sequential+fadvise_nocache],write 5242880~16384 [fadvise_sequential+fadvise_nocache],write 5324800~147456 [fadvise_sequential+fadvise_nocache],write 5537792~147456 [fadvise_sequential+fadvise_nocache],write 5750784~16384 [fadvise_sequential+fadvise_nocache],write 5832704~147456 [fadvise_sequential+fadvise_nocache],write 6045696~147456 [fadvise_sequential+fadvise_nocache],write 6258688~32768 [fadvise_sequential+fadvise_nocache],write 6340608~147456 [fadvise_sequential+fadvise_nocache],write 6553600~147456 [fadvise_sequential+fadvise_nocache],write 6766592~49152 [fadvise_sequential+fadvise_nocache],write 6848512~147456 [fadvise_sequential+fadvise_nocache],write 7061504~147456 [fadvise_sequential+fadvise_nocache],write 7274496~65536 [fadvise_sequential+fadvise_nocache],write 7356416~147456 [fadvise_sequential+fadvise_nocache],write 7569408~147456 [fadvise_sequential+fadvise_nocache],write 7782400~229376 [fadvise_sequential+fadvise_nocache],write 8077312~147456 [fadvise_sequential+fadvise_nocache],write 8290304~98304 [fadvise_sequential+fadvise_nocache],write 8404992~114688 [fadvise_sequential+fadvise_nocache],write 8617984~114688 [fadvise_sequential+fadvise_nocache],write 8830976~81920 [fadvise_sequential+fadvise_nocache],write 8994816~32768 [fadvise_sequential+fadvise_nocache],write 9207808~32768 [fadvise_sequential+fadvise_nocache],write 9420800~16384 [fadvise_sequential+fadvise_nocache],write 9502720~32768 [fadvise_sequential+fadvise_nocache],write 9715712~32768 [fadvise_sequential+fadvise_nocache],write 9928704~229376 [fadvise_sequential+fadvise_nocache],write 10174464~196608 [fadvise_sequential+fadvise_nocache],write 10387456~278528 [fadvise_sequential+fadvise_nocache],write 10682368~196608 [fadvise_sequential+fadvise_nocache],write 10895360~278528 [fadvise_sequential+fadvise_nocache],write 11190272~196608 [fadvise_sequential+fadvise_nocache],write 11403264~196608 [fadvise_sequential+fadvise_nocache],write 11616256~65536 [fadvise_sequential+fadvise_nocache],write 11714560~98304 [fadvise_sequential+fadvise_nocache],write 11829248~65536 [fadvise_sequential+fadvise_nocache],write 11927552~98304 [fadvise_sequential+fadvise_nocache],write 12042240~65536 [fadvise_sequential+fadvise_nocache],write 12124160~65536 [fadvise_sequential+fadvise_nocache],write 12222464~98304 [fadvise_sequential+fadvise_nocache],write 12337152~65536 [fadvise_sequential+fadvise_nocache],write 12435456~98304 [fadvise_sequential+fadvise_nocache],write 12550144~65536 [fadvise_sequential+fadvise_nocache],write 12730368~98304 [fadvise_sequential+fadvise_nocache],write 12943360~98304 [fadvise_sequential+fadvise_nocache],write 13107200~16384 [fadvise_sequential+fadvise_nocache],write 13238272~98304 [fadvise_sequential+fadvise_nocache],write 13451264~98304 [fadvise_sequential+fadvise_nocache],write 13746176~98304 [fadvise_sequential+fadvise_nocache],write 13959168~98304 [fadvise_sequential+fadvise_nocache],write 14254080~64264 [fadvise_sequential+fadvise_nocache],write 14319616~19759 [fadvise_sequential+fadvise_nocache],write 15204352~32768 [fadvise_sequential+fadvise_nocache],write 15368192~81920 [fadvise_sequential+fadvise_nocache],write 15581184~81920 [fadvise_sequential+fadvise_nocache],write 15728640~16384 [fadvise_sequential+fadvise_nocache],write 15876096~81920 [fadvise_sequential+fadvise_nocache],write 16089088~81920 [fadvise_sequential+fadvise_nocache],write 16384000~81920 [fadvise_sequential+fadvise_nocache],write 16596992~81920 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:25.240 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.255412+0000", 2024-09-21T15:00:25.240 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 19.944266409000001, 2024-09-21T15:00:25.240 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.058037909999999998, 2024-09-21T15:00:25.240 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:25.240 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:25.240 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:25.240 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:25.240 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:25.240 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97619 2024-09-21T15:00:25.240 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.240 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:25.240 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.240 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:25.240 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.255412+0000", 2024-09-21T15:00:25.241 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:25.241 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.241 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.241 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:25.241 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.255412+0000", 2024-09-21T15:00:25.241 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:25.241 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.241 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.241 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:25.241 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.255410+0000", 2024-09-21T15:00:25.241 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-21T15:00:25.241 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.241 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.241 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:25.241 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.288087+0000", 2024-09-21T15:00:25.242 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.032677337000000001 2024-09-21T15:00:25.242 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.242 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.242 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:25.242 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.288114+0000", 2024-09-21T15:00:25.242 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6905000000000001e-05 2024-09-21T15:00:25.242 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.242 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.242 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:25.242 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.288128+0000", 2024-09-21T15:00:25.242 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3560999999999999e-05 2024-09-21T15:00:25.242 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.242 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.242 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:25.243 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.288193+0000", 2024-09-21T15:00:25.243 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5747000000000006e-05 2024-09-21T15:00:25.243 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.243 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.243 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:25.243 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.289299+0000", 2024-09-21T15:00:25.243 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0011056390000000001 2024-09-21T15:00:25.243 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.243 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.243 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T15:00:25.243 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.290870+0000", 2024-09-21T15:00:25.243 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001570718 2024-09-21T15:00:25.243 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.243 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.243 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:25.244 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.312525+0000", 2024-09-21T15:00:25.244 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.021655427000000001 2024-09-21T15:00:25.244 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.244 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.244 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:25.244 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.313312+0000", 2024-09-21T15:00:25.244 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00078665500000000004 2024-09-21T15:00:25.244 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.244 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.244 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:25.244 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.313413+0000", 2024-09-21T15:00:25.244 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010131700000000001 2024-09-21T15:00:25.244 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.244 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.244 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:25.245 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.313450+0000", 2024-09-21T15:00:25.245 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7032000000000001e-05 2024-09-21T15:00:25.245 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:25.245 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:25.245 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:25.245 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.245 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.245 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:97734 3.19 3:9fcf754f:::rbd_data.1136bfc79a70.0000000000000009:head [write 0~4194304 [fadvise_sequential+fadvise_nocache],write 4784128~147456 [fadvise_sequential+fadvise_nocache],write 4997120~147456 [fadvise_sequential+fadvise_nocache],write 5210112~32768 [fadvise_sequential+fadvise_nocache],write 5292032~147456 [fadvise_sequential+fadvise_nocache],write 5505024~147456 [fadvise_sequential+fadvise_nocache],write 5718016~49152 [fadvise_sequential+fadvise_nocache],write 5799936~147456 [fadvise_sequential+fadvise_nocache],write 6012928~147456 [fadvise_sequential+fadvise_nocache],write 6225920~65536 [fadvise_sequential+fadvise_nocache],write 6307840~147456 [fadvise_sequential+fadvise_nocache],write 6520832~147456 [fadvise_sequential+fadvise_nocache],write 6733824~229376 [fadvise_sequential+fadvise_nocache],write 7028736~147456 [fadvise_sequential+fadvise_nocache],write 7241728~229376 [fadvise_sequential+fadvise_nocache],write 7536640~147456 [fadvise_sequential+fadvise_nocache],write 7749632~229376 [fadvise_sequential+fadvise_nocache],write 8077312~114688 [fadvise_sequential+fadvise_nocache],write 8290304~98304 [fadvise_sequential+fadvise_nocache],write 8404992~81920 [fadvise_sequential+fadvise_nocache],write 8617984~81920 [fadvise_sequential+fadvise_nocache],write 8830976~81920 [fadvise_sequential+fadvise_nocache],write 8962048~32768 [fadvise_sequential+fadvise_nocache],write 9175040~32768 [fadvise_sequential+fadvise_nocache],write 9388032~32768 [fadvise_sequential+fadvise_nocache],write 9469952~32768 [fadvise_sequential+fadvise_nocache],write 9682944~20092 [fadvise_sequential+fadvise_nocache],write 9961472~163840 [fadvise_sequential+fadvise_nocache],write 10141696~196608 [fadvise_sequential+fadvise_nocache],write 10354688~278528 [fadvise_sequential+fadvise_nocache],write 10649600~196608 [fadvise_sequential+fadvise_nocache],write 10862592~196608 [fadvise_sequential+fadvise_nocache],write 11075584~65536 [fadvise_sequential+fadvise_nocache],write 11173888~98304 [fadvise_sequential+fadvise_nocache],write 11288576~65536 [fadvise_sequential+fadvise_nocache],write 11386880~98304 [fadvise_sequential+fadvise_nocache],write 11501568~65536 [fadvise_sequential+fadvise_nocache],write 11583488~65536 [fadvise_sequential+fadvise_nocache],write 11681792~98304 [fadvise_sequential+fadvise_nocache],write 11796480~65536 [fadvise_sequential+fadvise_nocache],write 11894784~98304 [fadvise_sequential+fadvise_nocache],write 12009472~65536 [fadvise_sequential+fadvise_nocache],write 12189696~98304 [fadvise_sequential+fadvise_nocache],write 12402688~98304 [fadvise_sequential+fadvise_nocache],write 12697600~98304 [fadvise_sequential+fadvise_nocache],write 12910592~98304 [fadvise_sequential+fadvise_nocache],write 13205504~98304 [fadvise_sequential+fadvise_nocache],write 13418496~98304 [fadvise_sequential+fadvise_nocache],write 13713408~98304 [fadvise_sequential+fadvise_nocache],write 13926400~98304 [fadvise_sequential+fadvise_nocache],write 14139392~16384 [fadvise_sequential+fadvise_nocache],write 14680064~16384 [fadvise_sequential+fadvise_nocache],write 14827520~81920 [fadvise_sequential+fadvise_nocache],write 15040512~81920 [fadvise_sequential+fadvise_nocache],write 15335424~81920 [fadvise_sequential+fadvise_nocache],write 15548416~81920 [fadvise_sequential+fadvise_nocache],write 15843328~81920 [fadvise_sequential+fadvise_nocache],write 16056320~81920 [fadvise_sequential+fadvise_nocache],write 16351232~81920 [fadvise_sequential+fadvise_nocache],write 16564224~81920 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:25.245 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.354331+0000", 2024-09-21T15:00:25.245 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 19.845348018999999, 2024-09-21T15:00:25.245 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.086284870999999999, 2024-09-21T15:00:25.245 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:25.245 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:25.245 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:25.245 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:25.246 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:25.246 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97734 2024-09-21T15:00:25.246 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.246 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:25.246 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.246 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:25.246 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.354331+0000", 2024-09-21T15:00:25.246 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:25.246 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.246 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.246 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:25.246 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.354331+0000", 2024-09-21T15:00:25.246 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:25.246 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.246 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.246 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:25.247 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.354329+0000", 2024-09-21T15:00:25.247 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:00:25.247 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.247 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.247 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:25.247 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.370234+0000", 2024-09-21T15:00:25.247 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015905348999999999 2024-09-21T15:00:25.247 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.247 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.247 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:25.247 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.370286+0000", 2024-09-21T15:00:25.247 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1656999999999997e-05 2024-09-21T15:00:25.247 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.247 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.247 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:25.248 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.370315+0000", 2024-09-21T15:00:25.248 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9204e-05 2024-09-21T15:00:25.248 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.248 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.248 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:25.248 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.370395+0000", 2024-09-21T15:00:25.248 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0395999999999994e-05 2024-09-21T15:00:25.248 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.248 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.248 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:25.248 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.370604+0000", 2024-09-21T15:00:25.248 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000208684 2024-09-21T15:00:25.248 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.248 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.249 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 1", 2024-09-21T15:00:25.249 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.371701+0000", 2024-09-21T15:00:25.249 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0010973140000000001 2024-09-21T15:00:25.249 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.249 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.249 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:25.249 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.405485+0000", 2024-09-21T15:00:25.249 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.033783615000000003 2024-09-21T15:00:25.249 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.249 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.249 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:25.249 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.440211+0000", 2024-09-21T15:00:25.249 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034725757000000003 2024-09-21T15:00:25.249 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.249 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.249 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:25.250 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.440497+0000", 2024-09-21T15:00:25.250 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00028673099999999998 2024-09-21T15:00:25.250 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.250 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.250 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:25.250 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.440616+0000", 2024-09-21T15:00:25.250 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000118256 2024-09-21T15:00:25.250 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:25.250 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:25.250 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:25.250 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.250 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.250 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:97734 3.19 3:9fcf754f:::rbd_data.1136bfc79a70.0000000000000009:head [write 0~4194304 [fadvise_sequential+fadvise_nocache],write 4784128~147456 [fadvise_sequential+fadvise_nocache],write 4997120~147456 [fadvise_sequential+fadvise_nocache],write 5210112~32768 [fadvise_sequential+fadvise_nocache],write 5292032~147456 [fadvise_sequential+fadvise_nocache],write 5505024~147456 [fadvise_sequential+fadvise_nocache],write 5718016~49152 [fadvise_sequential+fadvise_nocache],write 5799936~147456 [fadvise_sequential+fadvise_nocache],write 6012928~147456 [fadvise_sequential+fadvise_nocache],write 6225920~65536 [fadvise_sequential+fadvise_nocache],write 6307840~147456 [fadvise_sequential+fadvise_nocache],write 6520832~147456 [fadvise_sequential+fadvise_nocache],write 6733824~229376 [fadvise_sequential+fadvise_nocache],write 7028736~147456 [fadvise_sequential+fadvise_nocache],write 7241728~229376 [fadvise_sequential+fadvise_nocache],write 7536640~147456 [fadvise_sequential+fadvise_nocache],write 7749632~229376 [fadvise_sequential+fadvise_nocache],write 8077312~114688 [fadvise_sequential+fadvise_nocache],write 8290304~98304 [fadvise_sequential+fadvise_nocache],write 8404992~81920 [fadvise_sequential+fadvise_nocache],write 8617984~81920 [fadvise_sequential+fadvise_nocache],write 8830976~81920 [fadvise_sequential+fadvise_nocache],write 8962048~32768 [fadvise_sequential+fadvise_nocache],write 9175040~32768 [fadvise_sequential+fadvise_nocache],write 9388032~32768 [fadvise_sequential+fadvise_nocache],write 9469952~32768 [fadvise_sequential+fadvise_nocache],write 9682944~20092 [fadvise_sequential+fadvise_nocache],write 9961472~163840 [fadvise_sequential+fadvise_nocache],write 10141696~196608 [fadvise_sequential+fadvise_nocache],write 10354688~278528 [fadvise_sequential+fadvise_nocache],write 10649600~196608 [fadvise_sequential+fadvise_nocache],write 10862592~196608 [fadvise_sequential+fadvise_nocache],write 11075584~65536 [fadvise_sequential+fadvise_nocache],write 11173888~98304 [fadvise_sequential+fadvise_nocache],write 11288576~65536 [fadvise_sequential+fadvise_nocache],write 11386880~98304 [fadvise_sequential+fadvise_nocache],write 11501568~65536 [fadvise_sequential+fadvise_nocache],write 11583488~65536 [fadvise_sequential+fadvise_nocache],write 11681792~98304 [fadvise_sequential+fadvise_nocache],write 11796480~65536 [fadvise_sequential+fadvise_nocache],write 11894784~98304 [fadvise_sequential+fadvise_nocache],write 12009472~65536 [fadvise_sequential+fadvise_nocache],write 12189696~98304 [fadvise_sequential+fadvise_nocache],write 12402688~98304 [fadvise_sequential+fadvise_nocache],write 12697600~98304 [fadvise_sequential+fadvise_nocache],write 12910592~98304 [fadvise_sequential+fadvise_nocache],write 13205504~98304 [fadvise_sequential+fadvise_nocache],write 13418496~98304 [fadvise_sequential+fadvise_nocache],write 13713408~98304 [fadvise_sequential+fadvise_nocache],write 13926400~98304 [fadvise_sequential+fadvise_nocache],write 14139392~16384 [fadvise_sequential+fadvise_nocache],write 14680064~16384 [fadvise_sequential+fadvise_nocache],write 14827520~81920 [fadvise_sequential+fadvise_nocache],write 15040512~81920 [fadvise_sequential+fadvise_nocache],write 15335424~81920 [fadvise_sequential+fadvise_nocache],write 15548416~81920 [fadvise_sequential+fadvise_nocache],write 15843328~81920 [fadvise_sequential+fadvise_nocache],write 16056320~81920 [fadvise_sequential+fadvise_nocache],write 16351232~81920 [fadvise_sequential+fadvise_nocache],write 16564224~81920 [fadvise_sequential+fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:25.250 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.412887+0000", 2024-09-21T15:00:25.250 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 19.786791295, 2024-09-21T15:00:25.251 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.027651709999999999, 2024-09-21T15:00:25.251 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:25.251 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "waiting for ondisk", 2024-09-21T15:00:25.251 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:25.251 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:25.251 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:25.251 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97734 2024-09-21T15:00:25.251 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.251 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:25.251 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.251 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:25.251 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.412887+0000", 2024-09-21T15:00:25.251 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:25.251 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.251 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.252 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:25.252 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.412887+0000", 2024-09-21T15:00:25.252 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:25.252 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.252 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.252 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:25.252 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.412886+0000", 2024-09-21T15:00:25.252 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:00:25.252 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.252 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.252 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:25.252 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.428395+0000", 2024-09-21T15:00:25.252 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015509448 2024-09-21T15:00:25.252 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.252 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.252 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:25.252 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.428417+0000", 2024-09-21T15:00:25.253 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1263999999999998e-05 2024-09-21T15:00:25.253 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.253 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.253 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:25.253 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.428425+0000", 2024-09-21T15:00:25.253 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8589999999999994e-06 2024-09-21T15:00:25.253 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.253 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.253 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:25.253 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.428481+0000", 2024-09-21T15:00:25.253 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5087000000000003e-05 2024-09-21T15:00:25.253 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.253 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.253 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for ondisk", 2024-09-21T15:00:25.253 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.428628+0000", 2024-09-21T15:00:25.254 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014714599999999999 2024-09-21T15:00:25.254 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.254 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.254 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:25.254 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.440539+0000", 2024-09-21T15:00:25.254 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011911424 2024-09-21T15:00:25.254 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:25.254 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:25.254 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:25.254 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.254 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.254 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:98562 3.9 e122/90 3:911e52bc:::rbd_data.1136bfc79a70.000000000000000a:head v 122'620, mlcod=121'618)", 2024-09-21T15:00:25.254 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.430966+0000", 2024-09-21T15:00:25.255 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 19.768712829000002, 2024-09-21T15:00:25.255 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.031245016, 2024-09-21T15:00:25.255 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:25.255 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:25.255 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:25.255 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.255 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:25.255 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.430966+0000", 2024-09-21T15:00:25.255 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:25.255 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.255 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.255 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:25.255 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.430966+0000", 2024-09-21T15:00:25.255 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:25.255 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.256 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.256 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:25.256 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.430965+0000", 2024-09-21T15:00:25.256 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:00:25.256 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.256 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.256 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:25.256 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.447521+0000", 2024-09-21T15:00:25.256 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.016556006000000002 2024-09-21T15:00:25.256 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.256 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.256 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:25.256 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.447543+0000", 2024-09-21T15:00:25.256 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2121e-05 2024-09-21T15:00:25.256 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.257 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.257 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:25.257 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.447551+0000", 2024-09-21T15:00:25.257 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6310000000000003e-06 2024-09-21T15:00:25.257 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.257 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.257 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:25.257 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.447632+0000", 2024-09-21T15:00:25.257 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1388000000000001e-05 2024-09-21T15:00:25.257 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.257 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.257 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:25.257 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.447662+0000", 2024-09-21T15:00:25.257 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0111999999999999e-05 2024-09-21T15:00:25.257 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.258 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.258 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:25.258 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.462185+0000", 2024-09-21T15:00:25.258 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014522863 2024-09-21T15:00:25.258 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.258 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.258 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:25.258 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.462211+0000", 2024-09-21T15:00:25.258 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5945e-05 2024-09-21T15:00:25.258 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:25.258 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:25.258 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:25.258 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.258 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.259 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:100151 3.8 3:17ab0d23:::rbd_data.1136ab4d6ff1.000000000000000a:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-21T15:00:25.259 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:10.762267+0000", 2024-09-21T15:00:25.259 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.437411324999999, 2024-09-21T15:00:25.259 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.029791319, 2024-09-21T15:00:25.259 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:25.259 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:25.259 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:25.259 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:25.259 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:25.259 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 100151 2024-09-21T15:00:25.259 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.259 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:25.259 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.259 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:25.259 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:10.762267+0000", 2024-09-21T15:00:25.260 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:25.260 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.260 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.260 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:25.260 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:10.762267+0000", 2024-09-21T15:00:25.260 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:25.260 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.260 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.260 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:25.260 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:10.762264+0000", 2024-09-21T15:00:25.260 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999967 2024-09-21T15:00:25.260 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.260 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.260 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:25.260 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:10.762283+0000", 2024-09-21T15:00:25.261 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9072999999999998e-05 2024-09-21T15:00:25.261 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.261 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.261 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:25.261 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:10.762325+0000", 2024-09-21T15:00:25.261 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2222000000000003e-05 2024-09-21T15:00:25.261 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.261 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.261 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:25.261 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:10.762348+0000", 2024-09-21T15:00:25.261 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2857000000000002e-05 2024-09-21T15:00:25.261 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.261 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.261 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:25.262 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:10.762490+0000", 2024-09-21T15:00:25.262 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000141667 2024-09-21T15:00:25.262 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.262 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.262 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:25.262 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:10.762751+0000", 2024-09-21T15:00:25.262 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00026079 2024-09-21T15:00:25.262 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.262 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.262 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 7", 2024-09-21T15:00:25.262 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:10.772217+0000", 2024-09-21T15:00:25.262 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0094660960000000002 2024-09-21T15:00:25.262 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.262 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.263 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:25.263 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:10.791315+0000", 2024-09-21T15:00:25.263 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.019098674999999999 2024-09-21T15:00:25.263 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.263 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.263 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:25.263 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:10.791938+0000", 2024-09-21T15:00:25.263 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00062279200000000005 2024-09-21T15:00:25.263 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.263 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.263 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:25.263 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:10.792005+0000", 2024-09-21T15:00:25.263 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6884000000000005e-05 2024-09-21T15:00:25.263 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.263 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.264 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:25.264 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:10.792059+0000", 2024-09-21T15:00:25.264 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3497e-05 2024-09-21T15:00:25.264 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:25.264 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:25.264 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:25.264 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.264 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.264 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:100152 3.7 3:e02630f6:::rbd_data.1136ab4d6ff1.000000000000000b:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-21T15:00:25.264 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:10.763146+0000", 2024-09-21T15:00:25.264 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.436532202, 2024-09-21T15:00:25.264 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.029305525999999998, 2024-09-21T15:00:25.264 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:25.264 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:25.265 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:25.265 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:25.265 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:25.265 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 100152 2024-09-21T15:00:25.265 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.265 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:25.265 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.265 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:25.265 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:10.763146+0000", 2024-09-21T15:00:25.265 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:25.265 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.265 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.265 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:25.265 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:10.763146+0000", 2024-09-21T15:00:25.265 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:25.265 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.266 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.266 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:25.266 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:10.763139+0000", 2024-09-21T15:00:25.266 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999924 2024-09-21T15:00:25.266 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.266 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.266 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:25.266 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:10.763158+0000", 2024-09-21T15:00:25.266 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9040000000000001e-05 2024-09-21T15:00:25.266 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.266 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.266 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:25.266 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:10.763185+0000", 2024-09-21T15:00:25.266 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6988999999999999e-05 2024-09-21T15:00:25.267 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.267 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.267 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:25.267 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:10.763198+0000", 2024-09-21T15:00:25.267 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.27e-05 2024-09-21T15:00:25.267 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.267 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.267 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:25.267 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:10.763303+0000", 2024-09-21T15:00:25.267 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000105451 2024-09-21T15:00:25.267 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.267 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.267 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:25.267 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:10.763557+0000", 2024-09-21T15:00:25.267 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00025405799999999999 2024-09-21T15:00:25.268 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.268 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.268 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T15:00:25.268 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:10.772841+0000", 2024-09-21T15:00:25.268 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0092834259999999991 2024-09-21T15:00:25.268 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.268 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.268 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:25.268 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:10.785078+0000", 2024-09-21T15:00:25.268 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012237572 2024-09-21T15:00:25.268 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.268 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.268 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:25.268 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:10.792383+0000", 2024-09-21T15:00:25.268 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0073048729999999999 2024-09-21T15:00:25.268 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.269 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.269 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:25.269 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:10.792423+0000", 2024-09-21T15:00:25.269 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0462e-05 2024-09-21T15:00:25.269 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.269 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.269 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:25.269 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:10.792452+0000", 2024-09-21T15:00:25.269 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8581999999999998e-05 2024-09-21T15:00:25.269 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:25.269 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:25.269 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:25.269 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.269 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.269 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:106195 3.1b 3:dacdb776:::rbd_data.1136d175325b.0000000000000066:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131)", 2024-09-21T15:00:25.270 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:15.993024+0000", 2024-09-21T15:00:25.270 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.2066544750000006, 2024-09-21T15:00:25.270 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.021198675, 2024-09-21T15:00:25.270 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:25.270 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:25.270 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:25.270 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:25.270 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:25.270 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 106195 2024-09-21T15:00:25.270 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.270 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:25.270 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.270 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:25.270 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:15.993024+0000", 2024-09-21T15:00:25.270 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:25.270 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.270 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.271 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:25.271 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:15.993024+0000", 2024-09-21T15:00:25.271 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:25.271 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.271 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.271 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:25.271 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:15.993023+0000", 2024-09-21T15:00:25.271 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:00:25.271 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.271 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.271 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:25.271 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:15.993031+0000", 2024-09-21T15:00:25.271 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.525e-06 2024-09-21T15:00:25.271 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.272 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.272 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:25.272 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:15.993047+0000", 2024-09-21T15:00:25.272 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6120000000000002e-05 2024-09-21T15:00:25.272 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.272 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.272 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:25.272 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:15.993054+0000", 2024-09-21T15:00:25.272 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6580000000000002e-06 2024-09-21T15:00:25.272 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.272 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.272 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:25.272 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:15.993127+0000", 2024-09-21T15:00:25.272 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2193000000000005e-05 2024-09-21T15:00:25.273 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.273 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.273 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:25.273 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:15.993272+0000", 2024-09-21T15:00:25.273 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014509500000000001 2024-09-21T15:00:25.273 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.273 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.273 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-21T15:00:25.273 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:15.997513+0000", 2024-09-21T15:00:25.273 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0042411810000000001 2024-09-21T15:00:25.273 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.273 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.273 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:25.274 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:16.014141+0000", 2024-09-21T15:00:25.274 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.016628428000000001 2024-09-21T15:00:25.274 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.274 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.274 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:25.274 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:16.014153+0000", 2024-09-21T15:00:25.274 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1585e-05 2024-09-21T15:00:25.274 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.274 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.274 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:25.274 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:16.014190+0000", 2024-09-21T15:00:25.274 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7583000000000003e-05 2024-09-21T15:00:25.274 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.275 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.275 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:25.275 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:16.014223+0000", 2024-09-21T15:00:25.275 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2465000000000002e-05 2024-09-21T15:00:25.275 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:25.275 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:25.275 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:25.275 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.275 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.275 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:106208 3.8 3:14331bdb:::rbd_data.1136d175325b.000000000000002f:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131)", 2024-09-21T15:00:25.275 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:16.034991+0000", 2024-09-21T15:00:25.275 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.1646874779999994, 2024-09-21T15:00:25.275 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.017265796, 2024-09-21T15:00:25.275 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:25.275 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:25.275 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:25.276 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:25.276 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:25.276 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 106208 2024-09-21T15:00:25.276 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.276 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:25.276 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.276 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:25.276 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:16.034991+0000", 2024-09-21T15:00:25.276 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:25.276 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.276 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.276 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:25.276 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:16.034991+0000", 2024-09-21T15:00:25.276 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:25.277 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.277 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.277 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:25.277 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:16.034990+0000", 2024-09-21T15:00:25.277 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:00:25.277 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.277 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.277 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:25.277 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:16.034998+0000", 2024-09-21T15:00:25.277 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5660000000000007e-06 2024-09-21T15:00:25.277 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.277 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.277 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:25.277 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:16.035017+0000", 2024-09-21T15:00:25.277 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8746e-05 2024-09-21T15:00:25.278 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.278 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.278 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:25.278 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:16.035026+0000", 2024-09-21T15:00:25.278 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4050000000000006e-06 2024-09-21T15:00:25.278 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.278 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.278 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:25.278 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:16.035117+0000", 2024-09-21T15:00:25.278 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0940999999999995e-05 2024-09-21T15:00:25.278 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.278 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.278 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:25.278 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:16.035274+0000", 2024-09-21T15:00:25.279 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000156298 2024-09-21T15:00:25.279 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.279 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.279 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 7", 2024-09-21T15:00:25.279 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:16.038518+0000", 2024-09-21T15:00:25.279 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00324442 2024-09-21T15:00:25.279 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.279 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.279 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:25.279 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:16.052111+0000", 2024-09-21T15:00:25.279 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.013592435999999999 2024-09-21T15:00:25.279 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.279 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.279 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:25.280 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:16.052128+0000", 2024-09-21T15:00:25.280 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7649e-05 2024-09-21T15:00:25.280 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.280 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.280 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:25.280 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:16.052202+0000", 2024-09-21T15:00:25.280 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3490000000000003e-05 2024-09-21T15:00:25.280 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.280 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.280 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:25.280 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:16.052257+0000", 2024-09-21T15:00:25.280 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5331000000000003e-05 2024-09-21T15:00:25.280 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:25.280 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:25.280 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:25.281 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.281 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.281 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:106571 3.7 3:e79b3980:::rbd_object_map.1136d175325b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131)", 2024-09-21T15:00:25.281 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:16.753145+0000", 2024-09-21T15:00:25.281 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 8.4465339539999995, 2024-09-21T15:00:25.281 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.031867624999999997, 2024-09-21T15:00:25.281 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:25.281 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:25.281 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:25.281 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:25.281 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:25.281 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 106571 2024-09-21T15:00:25.281 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.281 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:25.281 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.282 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:25.282 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:16.753145+0000", 2024-09-21T15:00:25.282 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:25.282 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.282 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.282 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:25.282 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:16.753145+0000", 2024-09-21T15:00:25.282 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:25.282 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.282 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.282 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:25.282 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:16.753144+0000", 2024-09-21T15:00:25.282 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:25.282 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.283 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.283 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:25.283 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:16.753148+0000", 2024-09-21T15:00:25.283 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4240000000000002e-06 2024-09-21T15:00:25.283 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.283 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.283 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:25.283 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:16.753163+0000", 2024-09-21T15:00:25.283 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5404e-05 2024-09-21T15:00:25.283 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.283 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.283 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:25.283 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:16.753168+0000", 2024-09-21T15:00:25.283 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7199999999999997e-06 2024-09-21T15:00:25.283 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.284 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.284 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:25.284 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:16.755580+0000", 2024-09-21T15:00:25.284 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0024117969999999998 2024-09-21T15:00:25.284 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.284 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.284 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:25.284 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:16.755649+0000", 2024-09-21T15:00:25.284 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8806000000000005e-05 2024-09-21T15:00:25.284 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.284 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.284 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T15:00:25.284 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:16.755931+0000", 2024-09-21T15:00:25.284 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00028237599999999998 2024-09-21T15:00:25.284 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.285 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.285 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:25.285 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:16.757409+0000", 2024-09-21T15:00:25.285 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0014781250000000001 2024-09-21T15:00:25.285 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.285 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.285 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:25.285 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:16.784837+0000", 2024-09-21T15:00:25.285 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.027427710000000001 2024-09-21T15:00:25.285 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.285 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.285 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:25.285 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:16.784911+0000", 2024-09-21T15:00:25.285 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4319000000000005e-05 2024-09-21T15:00:25.286 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.286 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.286 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:25.286 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:16.785012+0000", 2024-09-21T15:00:25.286 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010128600000000001 2024-09-21T15:00:25.286 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:25.286 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:25.286 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:25.286 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.286 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.286 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:106573 3.7 3:e1064da6:::rbd_data.1136d175325b.000000000000003e:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131)", 2024-09-21T15:00:25.286 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:16.757491+0000", 2024-09-21T15:00:25.286 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 8.4421881560000003, 2024-09-21T15:00:25.287 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.027733134, 2024-09-21T15:00:25.287 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:25.287 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:25.287 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:25.287 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:25.287 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:25.287 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 106573 2024-09-21T15:00:25.287 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.287 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:25.287 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.287 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:25.287 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:16.757491+0000", 2024-09-21T15:00:25.287 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:25.287 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.287 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.288 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:25.288 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:16.757491+0000", 2024-09-21T15:00:25.288 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:25.288 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.288 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.288 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:25.288 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:16.757490+0000", 2024-09-21T15:00:25.288 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:00:25.288 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.288 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.288 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:25.288 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:16.757496+0000", 2024-09-21T15:00:25.288 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8980000000000001e-06 2024-09-21T15:00:25.288 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.288 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.289 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:25.289 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:16.757504+0000", 2024-09-21T15:00:25.289 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5760000000000006e-06 2024-09-21T15:00:25.289 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.289 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.289 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:25.289 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:16.757510+0000", 2024-09-21T15:00:25.289 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4149999999999998e-06 2024-09-21T15:00:25.289 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.289 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.289 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:25.289 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:16.757599+0000", 2024-09-21T15:00:25.289 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8943999999999994e-05 2024-09-21T15:00:25.289 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.289 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.290 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:25.290 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:16.757726+0000", 2024-09-21T15:00:25.290 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000127427 2024-09-21T15:00:25.290 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.290 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.290 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T15:00:25.290 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:16.766790+0000", 2024-09-21T15:00:25.290 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0090637719999999995 2024-09-21T15:00:25.290 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.290 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.290 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:25.290 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:16.785066+0000", 2024-09-21T15:00:25.290 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.018276288000000002 2024-09-21T15:00:25.290 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.291 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.291 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:25.291 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:16.785081+0000", 2024-09-21T15:00:25.291 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5311999999999998e-05 2024-09-21T15:00:25.291 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.291 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.291 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:25.291 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:16.785138+0000", 2024-09-21T15:00:25.291 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6416000000000003e-05 2024-09-21T15:00:25.291 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.291 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.291 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:25.291 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:16.785224+0000", 2024-09-21T15:00:25.291 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5892999999999999e-05 2024-09-21T15:00:25.291 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:25.291 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:25.292 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:25.292 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.292 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.292 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop_reply(client.4406.0:106571 3.7 e131/93 ondisk, result = 0)", 2024-09-21T15:00:25.292 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:16.757972+0000", 2024-09-21T15:00:25.292 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 8.4417063189999997, 2024-09-21T15:00:25.292 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.027061808999999999, 2024-09-21T15:00:25.292 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:25.292 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:25.292 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:25.292 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.292 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:25.292 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:16.757972+0000", 2024-09-21T15:00:25.292 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:25.292 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.292 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.293 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:25.293 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:16.757972+0000", 2024-09-21T15:00:25.293 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:25.293 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.293 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.293 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:25.293 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:16.757971+0000", 2024-09-21T15:00:25.293 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:00:25.293 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.293 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.293 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:25.293 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:16.757980+0000", 2024-09-21T15:00:25.293 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.653e-06 2024-09-21T15:00:25.293 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.294 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.294 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:25.294 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:16.758003+0000", 2024-09-21T15:00:25.294 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2753000000000001e-05 2024-09-21T15:00:25.294 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.294 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.294 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:25.294 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:16.758013+0000", 2024-09-21T15:00:25.294 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.057e-05 2024-09-21T15:00:25.294 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.294 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.294 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:25.294 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:16.784816+0000", 2024-09-21T15:00:25.294 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.026802544000000001 2024-09-21T15:00:25.295 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.295 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.295 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:25.295 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:16.784828+0000", 2024-09-21T15:00:25.295 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2544999999999999e-05 2024-09-21T15:00:25.295 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.295 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.295 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:25.295 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:16.785034+0000", 2024-09-21T15:00:25.295 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00020573799999999999 2024-09-21T15:00:25.295 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:25.295 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:25.295 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:25.295 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.295 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.296 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:108547 3.7 3:e5a15d47:::rbd_data.1136d80ba95e.0000000000000056:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-21T15:00:25.296 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:20.958495+0000", 2024-09-21T15:00:25.296 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.2411839330000003, 2024-09-21T15:00:25.296 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.020027828000000001, 2024-09-21T15:00:25.296 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:25.296 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:25.296 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:25.296 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:25.296 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:25.296 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 108547 2024-09-21T15:00:25.296 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.296 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:25.296 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.296 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:25.296 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:20.958495+0000", 2024-09-21T15:00:25.297 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:25.297 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.297 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.297 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:25.297 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:20.958495+0000", 2024-09-21T15:00:25.297 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:25.297 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.297 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.297 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:25.297 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:20.958493+0000", 2024-09-21T15:00:25.297 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:00:25.297 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.297 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.297 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:25.297 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:20.958506+0000", 2024-09-21T15:00:25.297 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.307e-05 2024-09-21T15:00:25.298 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.298 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.298 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:25.298 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:20.958535+0000", 2024-09-21T15:00:25.298 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9488999999999998e-05 2024-09-21T15:00:25.298 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.298 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.298 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:25.298 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:20.958550+0000", 2024-09-21T15:00:25.298 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4569999999999999e-05 2024-09-21T15:00:25.298 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.298 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.298 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:25.298 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:20.958642+0000", 2024-09-21T15:00:25.298 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2610999999999998e-05 2024-09-21T15:00:25.299 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.299 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.299 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:25.299 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:20.958879+0000", 2024-09-21T15:00:25.299 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00023676200000000001 2024-09-21T15:00:25.299 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.299 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.299 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T15:00:25.299 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:20.963620+0000", 2024-09-21T15:00:25.299 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0047406499999999999 2024-09-21T15:00:25.299 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.299 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.299 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:25.299 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:20.978354+0000", 2024-09-21T15:00:25.299 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.014734206 2024-09-21T15:00:25.300 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.300 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.300 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:25.300 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:20.978375+0000", 2024-09-21T15:00:25.300 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.073e-05 2024-09-21T15:00:25.300 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.300 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.300 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:25.300 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:20.978460+0000", 2024-09-21T15:00:25.300 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5634e-05 2024-09-21T15:00:25.300 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.300 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.300 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:25.300 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:20.978523+0000", 2024-09-21T15:00:25.300 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2106000000000005e-05 2024-09-21T15:00:25.300 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:25.300 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:25.301 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:25.301 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.301 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.301 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:108591 3.1b 3:dc1bc5e9:::rbd_data.1136d80ba95e.00000000000002dd:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-21T15:00:25.301 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:21.085069+0000", 2024-09-21T15:00:25.301 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.1146094880000001, 2024-09-21T15:00:25.301 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015806928000000001, 2024-09-21T15:00:25.301 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:25.301 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:25.301 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:25.301 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:25.301 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:25.301 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 108591 2024-09-21T15:00:25.301 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.301 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:25.301 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.302 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:25.302 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:21.085069+0000", 2024-09-21T15:00:25.302 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:25.302 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.302 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.302 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:25.302 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:21.085069+0000", 2024-09-21T15:00:25.302 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:25.302 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.302 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.302 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:25.302 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:21.085067+0000", 2024-09-21T15:00:25.302 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:00:25.302 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.302 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.303 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:25.303 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:21.085079+0000", 2024-09-21T15:00:25.303 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1773999999999999e-05 2024-09-21T15:00:25.303 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.303 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.303 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:25.303 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:21.085106+0000", 2024-09-21T15:00:25.303 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6494000000000001e-05 2024-09-21T15:00:25.303 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.303 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.303 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:25.303 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:21.085119+0000", 2024-09-21T15:00:25.303 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3443e-05 2024-09-21T15:00:25.303 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.303 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.304 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:25.304 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:21.085203+0000", 2024-09-21T15:00:25.304 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4402999999999998e-05 2024-09-21T15:00:25.304 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.304 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.304 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:25.304 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:21.085387+0000", 2024-09-21T15:00:25.304 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018311299999999999 2024-09-21T15:00:25.304 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.304 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.304 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 0", 2024-09-21T15:00:25.304 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:21.089060+0000", 2024-09-21T15:00:25.304 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0036736049999999999 2024-09-21T15:00:25.304 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.304 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.305 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:25.305 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:21.100787+0000", 2024-09-21T15:00:25.305 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011726591 2024-09-21T15:00:25.305 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.305 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.305 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:25.305 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:21.100803+0000", 2024-09-21T15:00:25.305 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6307000000000002e-05 2024-09-21T15:00:25.305 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.305 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.305 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:25.305 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:21.100836+0000", 2024-09-21T15:00:25.305 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2982999999999999e-05 2024-09-21T15:00:25.305 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.305 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.305 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:25.306 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:21.100876+0000", 2024-09-21T15:00:25.306 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0040999999999998e-05 2024-09-21T15:00:25.306 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:25.306 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:25.306 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:25.306 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:25.306 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:25.306 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:25.306 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T15:00:25.308 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T15:00:25.308 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:00:25.302+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:00:25.408 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:00:25.404+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:00:25.426 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:25.427 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:00:25.427 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T15:00:25.427 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:25.437 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T15:00:25.509 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:00:25.502+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:00:25.541 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:25.541 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:00:25.541 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T15:00:25.541 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T15:00:25.541 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:25.551 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T15:00:25.610 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:00:25.606+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:00:25.662 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:25.662 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T15:00:25.662 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T15:00:25.662 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T15:00:25.662 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.662 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45839 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:25.662 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159313+0000", 2024-09-21T15:00:25.663 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 75.49932441, 2024-09-21T15:00:25.663 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068238699, 2024-09-21T15:00:25.663 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:25.663 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:25.663 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:25.663 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:25.663 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:25.663 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45839 2024-09-21T15:00:25.664 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.664 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:25.664 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.664 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:25.664 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159313+0000", 2024-09-21T15:00:25.664 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:25.664 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.665 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.665 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:25.665 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159313+0000", 2024-09-21T15:00:25.665 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:25.665 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.665 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.665 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:25.665 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159313+0000", 2024-09-21T15:00:25.665 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:25.665 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.665 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.665 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:25.665 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159316+0000", 2024-09-21T15:00:25.666 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.033e-06 2024-09-21T15:00:25.666 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.666 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.666 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:25.666 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159322+0000", 2024-09-21T15:00:25.666 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1820000000000003e-06 2024-09-21T15:00:25.666 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.666 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.666 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:25.666 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159327+0000", 2024-09-21T15:00:25.666 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.062e-06 2024-09-21T15:00:25.666 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.666 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.666 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:25.667 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227544+0000", 2024-09-21T15:00:25.667 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068217142999999994 2024-09-21T15:00:25.667 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.667 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.667 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:25.667 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227552+0000", 2024-09-21T15:00:25.667 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6260000000000003e-06 2024-09-21T15:00:25.667 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:25.667 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:25.667 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:25.667 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.667 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.667 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45840 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:25.667 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159332+0000", 2024-09-21T15:00:25.668 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 75.499306113000003, 2024-09-21T15:00:25.668 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068262046000000007, 2024-09-21T15:00:25.668 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:25.668 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:25.668 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:25.668 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:25.668 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:25.668 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45840 2024-09-21T15:00:25.668 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.668 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:25.668 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.668 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:25.668 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159332+0000", 2024-09-21T15:00:25.668 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:25.668 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.668 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.669 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:25.669 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159332+0000", 2024-09-21T15:00:25.669 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:25.669 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.669 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.669 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:25.669 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159331+0000", 2024-09-21T15:00:25.669 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:25.669 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.669 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.669 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:25.669 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159351+0000", 2024-09-21T15:00:25.669 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0064999999999998e-05 2024-09-21T15:00:25.670 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.670 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.670 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:25.670 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159367+0000", 2024-09-21T15:00:25.670 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5308e-05 2024-09-21T15:00:25.670 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.670 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.670 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:25.670 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159373+0000", 2024-09-21T15:00:25.670 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1840000000000004e-06 2024-09-21T15:00:25.670 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.670 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.670 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:25.670 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227584+0000", 2024-09-21T15:00:25.671 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068211401000000005 2024-09-21T15:00:25.671 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.671 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.671 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:25.671 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227594+0000", 2024-09-21T15:00:25.671 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4120000000000007e-06 2024-09-21T15:00:25.671 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:25.671 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:25.671 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:25.671 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.671 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.671 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45841 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:25.671 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159379+0000", 2024-09-21T15:00:25.671 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 75.499258956999995, 2024-09-21T15:00:25.671 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068249039999999997, 2024-09-21T15:00:25.672 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:25.672 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:25.672 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:25.672 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:25.672 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:25.672 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45841 2024-09-21T15:00:25.672 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.672 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:25.672 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.672 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:25.672 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159379+0000", 2024-09-21T15:00:25.672 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:25.672 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.673 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.673 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:25.673 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159379+0000", 2024-09-21T15:00:25.673 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:25.673 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.673 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.673 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:25.673 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159378+0000", 2024-09-21T15:00:25.673 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:25.673 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.673 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.673 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:25.673 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159382+0000", 2024-09-21T15:00:25.673 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.641e-06 2024-09-21T15:00:25.674 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.674 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.674 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:25.674 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159421+0000", 2024-09-21T15:00:25.674 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9439e-05 2024-09-21T15:00:25.674 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.674 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.674 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:25.674 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159430+0000", 2024-09-21T15:00:25.674 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8429999999999999e-06 2024-09-21T15:00:25.674 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.675 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.675 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:25.675 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227617+0000", 2024-09-21T15:00:25.675 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068187084999999995 2024-09-21T15:00:25.675 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.675 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.675 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:25.675 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227628+0000", 2024-09-21T15:00:25.675 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0507e-05 2024-09-21T15:00:25.675 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:25.675 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:25.675 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:25.675 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.675 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.675 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45844 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:25.676 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159499+0000", 2024-09-21T15:00:25.676 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 75.499138571000003, 2024-09-21T15:00:25.676 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068255415, 2024-09-21T15:00:25.676 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:25.676 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:25.676 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:25.676 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:25.676 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:25.676 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45844 2024-09-21T15:00:25.676 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.676 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:25.676 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.676 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:25.676 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159499+0000", 2024-09-21T15:00:25.677 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:25.677 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.677 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.677 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:25.677 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159499+0000", 2024-09-21T15:00:25.677 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:25.677 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.677 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.677 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:25.677 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159499+0000", 2024-09-21T15:00:25.677 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:25.677 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.677 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.677 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:25.678 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159503+0000", 2024-09-21T15:00:25.678 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0999999999999997e-06 2024-09-21T15:00:25.678 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.678 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.678 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:25.678 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159512+0000", 2024-09-21T15:00:25.678 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5850000000000006e-06 2024-09-21T15:00:25.678 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.678 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.678 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:25.678 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159518+0000", 2024-09-21T15:00:25.678 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0909999999999998e-06 2024-09-21T15:00:25.678 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.679 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.679 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:25.679 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227743+0000", 2024-09-21T15:00:25.679 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068224536000000002 2024-09-21T15:00:25.679 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.679 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.679 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:25.679 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227755+0000", 2024-09-21T15:00:25.679 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1659e-05 2024-09-21T15:00:25.679 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:25.679 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:25.679 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:25.679 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.679 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.680 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45845 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:25.680 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159524+0000", 2024-09-21T15:00:25.680 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 75.499114211000006, 2024-09-21T15:00:25.680 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068267194000000003, 2024-09-21T15:00:25.680 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:25.680 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:25.680 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:25.680 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:25.680 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:25.680 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45845 2024-09-21T15:00:25.680 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.680 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:25.680 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.680 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:25.680 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159524+0000", 2024-09-21T15:00:25.680 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:25.681 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.681 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.681 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:25.681 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159524+0000", 2024-09-21T15:00:25.681 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:25.681 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.681 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.681 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:25.681 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159523+0000", 2024-09-21T15:00:25.681 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:25.681 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.681 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.681 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:25.681 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159526+0000", 2024-09-21T15:00:25.682 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1980000000000001e-06 2024-09-21T15:00:25.682 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.682 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.682 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:25.682 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159533+0000", 2024-09-21T15:00:25.682 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.348e-06 2024-09-21T15:00:25.682 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.682 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.682 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:25.682 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159546+0000", 2024-09-21T15:00:25.682 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3464e-05 2024-09-21T15:00:25.682 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.682 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.682 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:25.683 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227783+0000", 2024-09-21T15:00:25.683 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068236926000000003 2024-09-21T15:00:25.683 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.683 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.683 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:25.683 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227791+0000", 2024-09-21T15:00:25.683 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6669999999999996e-06 2024-09-21T15:00:25.683 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:25.683 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:25.683 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:25.683 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.683 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.683 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45846 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:25.683 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159584+0000", 2024-09-21T15:00:25.683 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 75.499053568999997, 2024-09-21T15:00:25.684 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068254767999999993, 2024-09-21T15:00:25.684 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:25.684 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:25.684 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:25.684 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:25.684 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:25.684 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45846 2024-09-21T15:00:25.684 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.684 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:25.684 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.684 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:25.684 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159584+0000", 2024-09-21T15:00:25.684 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:25.684 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.684 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.685 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:25.685 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159584+0000", 2024-09-21T15:00:25.685 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:25.685 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.685 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.685 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:25.685 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159583+0000", 2024-09-21T15:00:25.685 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:25.685 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.685 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.685 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:25.685 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159588+0000", 2024-09-21T15:00:25.685 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1330000000000001e-06 2024-09-21T15:00:25.685 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.685 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.685 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:25.686 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159596+0000", 2024-09-21T15:00:25.686 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5469999999999997e-06 2024-09-21T15:00:25.686 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.686 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.686 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:25.686 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159609+0000", 2024-09-21T15:00:25.686 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2785999999999999e-05 2024-09-21T15:00:25.686 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.686 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.686 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:25.686 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227830+0000", 2024-09-21T15:00:25.686 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068220879999999998 2024-09-21T15:00:25.686 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.687 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.687 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:25.687 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227839+0000", 2024-09-21T15:00:25.687 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1239999999999994e-06 2024-09-21T15:00:25.687 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:25.687 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:25.687 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:25.687 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.687 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.687 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45847 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:25.687 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159614+0000", 2024-09-21T15:00:25.687 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 75.499023492999996, 2024-09-21T15:00:25.687 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068260639999999997, 2024-09-21T15:00:25.687 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:25.688 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:25.688 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:25.688 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:25.688 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:25.688 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45847 2024-09-21T15:00:25.688 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.688 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:25.688 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.688 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:25.688 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159614+0000", 2024-09-21T15:00:25.688 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:25.688 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.688 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.689 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:25.689 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159614+0000", 2024-09-21T15:00:25.689 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:25.689 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.689 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.689 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:25.689 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159614+0000", 2024-09-21T15:00:25.689 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:25.689 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.689 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.689 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:25.689 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159617+0000", 2024-09-21T15:00:25.689 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2909999999999999e-06 2024-09-21T15:00:25.689 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.689 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.690 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:25.690 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159623+0000", 2024-09-21T15:00:25.690 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.511e-06 2024-09-21T15:00:25.690 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.690 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.690 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:25.690 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159631+0000", 2024-09-21T15:00:25.690 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.571e-06 2024-09-21T15:00:25.690 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.690 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.690 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:25.690 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227866+0000", 2024-09-21T15:00:25.690 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068235136000000002 2024-09-21T15:00:25.690 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.691 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.691 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:25.691 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227875+0000", 2024-09-21T15:00:25.691 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7199999999999995e-06 2024-09-21T15:00:25.691 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:25.691 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:25.691 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:25.691 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.691 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.691 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45848 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:25.691 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159636+0000", 2024-09-21T15:00:25.691 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 75.499001712999998, 2024-09-21T15:00:25.691 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068267283999999998, 2024-09-21T15:00:25.691 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:25.692 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:25.692 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:25.692 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:25.692 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:25.692 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45848 2024-09-21T15:00:25.692 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.692 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:25.692 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.692 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:25.692 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159636+0000", 2024-09-21T15:00:25.692 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:25.692 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.692 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.692 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:25.693 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159636+0000", 2024-09-21T15:00:25.693 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:25.693 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.693 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.693 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:25.693 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159636+0000", 2024-09-21T15:00:25.693 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:25.693 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.693 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.693 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:25.693 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159638+0000", 2024-09-21T15:00:25.693 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6560000000000001e-06 2024-09-21T15:00:25.693 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.693 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.693 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:25.694 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159644+0000", 2024-09-21T15:00:25.694 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0839999999999997e-06 2024-09-21T15:00:25.694 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.694 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.694 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:25.694 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159649+0000", 2024-09-21T15:00:25.694 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7500000000000003e-06 2024-09-21T15:00:25.694 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.694 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.694 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:25.694 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227892+0000", 2024-09-21T15:00:25.694 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068243315999999998 2024-09-21T15:00:25.694 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.695 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.695 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:25.695 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227903+0000", 2024-09-21T15:00:25.695 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0835999999999999e-05 2024-09-21T15:00:25.695 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:25.695 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:25.695 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:25.695 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.695 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.695 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45849 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:25.695 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159667+0000", 2024-09-21T15:00:25.695 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 75.498970899, 2024-09-21T15:00:25.695 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068291051000000005, 2024-09-21T15:00:25.695 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:25.696 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:25.696 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:25.696 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:25.696 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:25.696 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45849 2024-09-21T15:00:25.696 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.696 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:25.696 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.696 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:25.696 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159667+0000", 2024-09-21T15:00:25.696 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:25.696 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.696 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.696 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:25.696 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159667+0000", 2024-09-21T15:00:25.697 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:25.697 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.697 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.697 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:25.697 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159666+0000", 2024-09-21T15:00:25.697 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:25.697 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.697 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.697 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:25.697 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159670+0000", 2024-09-21T15:00:25.697 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.427e-06 2024-09-21T15:00:25.697 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.697 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.697 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:25.698 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159678+0000", 2024-09-21T15:00:25.698 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9869999999999999e-06 2024-09-21T15:00:25.698 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.698 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.698 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:25.698 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159688+0000", 2024-09-21T15:00:25.698 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0631999999999999e-05 2024-09-21T15:00:25.698 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.698 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.698 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:25.698 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227950+0000", 2024-09-21T15:00:25.698 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068261798999999998 2024-09-21T15:00:25.698 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.698 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.699 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:25.699 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227958+0000", 2024-09-21T15:00:25.699 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7510000000000008e-06 2024-09-21T15:00:25.699 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:25.699 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:25.699 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:25.699 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.699 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.699 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45850 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:25.699 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159695+0000", 2024-09-21T15:00:25.699 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 75.498943187999998, 2024-09-21T15:00:25.699 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068302999000000003, 2024-09-21T15:00:25.699 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:25.699 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:25.699 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:25.700 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:25.700 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:25.700 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45850 2024-09-21T15:00:25.700 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.700 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:25.700 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.700 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:25.700 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159695+0000", 2024-09-21T15:00:25.700 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:25.700 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.700 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.700 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:25.700 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159695+0000", 2024-09-21T15:00:25.700 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:25.701 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.701 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.701 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:25.701 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159694+0000", 2024-09-21T15:00:25.701 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:25.701 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.701 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.701 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:25.701 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159697+0000", 2024-09-21T15:00:25.701 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.258e-06 2024-09-21T15:00:25.701 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.701 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.701 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:25.701 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159704+0000", 2024-09-21T15:00:25.701 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7660000000000001e-06 2024-09-21T15:00:25.701 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.702 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.702 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:25.702 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159713+0000", 2024-09-21T15:00:25.702 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1779999999999992e-06 2024-09-21T15:00:25.702 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.702 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.702 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:25.702 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227989+0000", 2024-09-21T15:00:25.702 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068276015999999995 2024-09-21T15:00:25.702 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.702 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.702 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:25.702 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227998+0000", 2024-09-21T15:00:25.702 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2050000000000002e-06 2024-09-21T15:00:25.702 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:25.703 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:25.703 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:25.703 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.703 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.703 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45851 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:25.703 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159719+0000", 2024-09-21T15:00:25.703 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 75.498919055000002, 2024-09-21T15:00:25.703 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068320327, 2024-09-21T15:00:25.703 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:25.703 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:25.703 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:25.703 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:25.703 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:25.703 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45851 2024-09-21T15:00:25.704 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.704 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:25.704 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.704 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:25.704 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159719+0000", 2024-09-21T15:00:25.704 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:25.704 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.704 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.704 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:25.704 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159719+0000", 2024-09-21T15:00:25.704 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:25.704 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.704 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.704 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:25.705 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159718+0000", 2024-09-21T15:00:25.705 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:25.705 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.705 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.705 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:25.705 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159721+0000", 2024-09-21T15:00:25.705 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2169999999999999e-06 2024-09-21T15:00:25.705 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.705 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.705 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:25.705 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159730+0000", 2024-09-21T15:00:25.705 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5720000000000003e-06 2024-09-21T15:00:25.705 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.705 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.706 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:25.706 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159735+0000", 2024-09-21T15:00:25.706 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1100000000000002e-06 2024-09-21T15:00:25.706 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.706 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.706 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:25.706 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228029+0000", 2024-09-21T15:00:25.706 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068293624999999997 2024-09-21T15:00:25.706 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.706 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.706 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:25.706 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228039+0000", 2024-09-21T15:00:25.706 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0254e-05 2024-09-21T15:00:25.706 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:25.707 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:25.707 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:25.707 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.707 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.707 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45852 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:25.707 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159740+0000", 2024-09-21T15:00:25.707 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 75.498897886999998, 2024-09-21T15:00:25.707 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068340260999999999, 2024-09-21T15:00:25.707 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:25.707 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:25.707 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:25.707 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:25.707 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:25.707 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45852 2024-09-21T15:00:25.707 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.708 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:25.708 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.708 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:25.708 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159740+0000", 2024-09-21T15:00:25.708 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:25.708 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.708 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.708 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:25.708 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159740+0000", 2024-09-21T15:00:25.708 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:25.708 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.708 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.708 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:25.708 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159739+0000", 2024-09-21T15:00:25.708 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:25.709 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.709 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.709 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:25.709 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159757+0000", 2024-09-21T15:00:25.709 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7314999999999999e-05 2024-09-21T15:00:25.709 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.709 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.709 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:25.709 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159769+0000", 2024-09-21T15:00:25.709 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2376e-05 2024-09-21T15:00:25.709 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.709 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.709 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:25.709 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159775+0000", 2024-09-21T15:00:25.709 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2179999999999996e-06 2024-09-21T15:00:25.710 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.710 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.710 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:25.710 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228071+0000", 2024-09-21T15:00:25.710 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068295344999999993 2024-09-21T15:00:25.710 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.710 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.710 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:25.710 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228080+0000", 2024-09-21T15:00:25.710 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3959999999999995e-06 2024-09-21T15:00:25.710 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:25.710 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:25.710 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:25.710 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.710 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.710 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45853 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:25.711 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159781+0000", 2024-09-21T15:00:25.711 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 75.498856622999995, 2024-09-21T15:00:25.711 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068337093000000002, 2024-09-21T15:00:25.711 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:25.711 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:25.711 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:25.711 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:25.711 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:25.711 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45853 2024-09-21T15:00:25.711 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.711 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:25.711 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.711 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:25.711 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159781+0000", 2024-09-21T15:00:25.712 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:25.712 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.712 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.712 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:25.712 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159781+0000", 2024-09-21T15:00:25.712 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:25.712 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.712 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.712 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:25.712 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159781+0000", 2024-09-21T15:00:25.712 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:25.712 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.712 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.712 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:25.712 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159784+0000", 2024-09-21T15:00:25.713 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4759999999999998e-06 2024-09-21T15:00:25.713 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.713 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.713 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:25.713 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159792+0000", 2024-09-21T15:00:25.713 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0569999999999993e-06 2024-09-21T15:00:25.713 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.713 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.713 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:25.713 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159797+0000", 2024-09-21T15:00:25.713 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9640000000000002e-06 2024-09-21T15:00:25.713 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.713 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.713 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:25.713 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228111+0000", 2024-09-21T15:00:25.713 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068313737999999999 2024-09-21T15:00:25.714 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.714 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.714 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:25.714 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228118+0000", 2024-09-21T15:00:25.714 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.43e-06 2024-09-21T15:00:25.714 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:25.714 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:25.714 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:25.714 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.714 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.714 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45854 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:25.714 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159802+0000", 2024-09-21T15:00:25.714 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 75.498835972999998, 2024-09-21T15:00:25.714 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068358938999999994, 2024-09-21T15:00:25.715 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:25.715 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:25.715 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:25.715 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:25.715 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:25.715 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45854 2024-09-21T15:00:25.715 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.715 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:25.715 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.715 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:25.715 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159802+0000", 2024-09-21T15:00:25.715 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:25.715 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.715 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.715 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:25.715 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159802+0000", 2024-09-21T15:00:25.716 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:25.716 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.716 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.716 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:25.716 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159801+0000", 2024-09-21T15:00:25.716 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:25.716 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.716 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.716 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:25.716 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159804+0000", 2024-09-21T15:00:25.716 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0970000000000002e-06 2024-09-21T15:00:25.716 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.716 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.716 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:25.716 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159823+0000", 2024-09-21T15:00:25.716 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8895000000000001e-05 2024-09-21T15:00:25.717 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.717 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.717 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:25.717 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159837+0000", 2024-09-21T15:00:25.717 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3247999999999999e-05 2024-09-21T15:00:25.717 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.717 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.717 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:25.717 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228151+0000", 2024-09-21T15:00:25.717 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068314642999999994 2024-09-21T15:00:25.717 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.717 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.717 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:25.717 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228161+0000", 2024-09-21T15:00:25.717 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4250000000000004e-06 2024-09-21T15:00:25.718 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:25.718 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:25.718 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:25.718 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.718 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.718 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45983 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:25.718 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159848+0000", 2024-09-21T15:00:25.718 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 75.498789713999997, 2024-09-21T15:00:25.718 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068333801, 2024-09-21T15:00:25.718 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:25.718 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:25.718 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:25.718 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:25.719 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:25.719 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45983 2024-09-21T15:00:25.719 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.719 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:25.719 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.719 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:25.719 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159848+0000", 2024-09-21T15:00:25.719 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:25.719 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.719 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.719 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:25.719 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159848+0000", 2024-09-21T15:00:25.719 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:25.719 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.719 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.719 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:25.720 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159847+0000", 2024-09-21T15:00:25.720 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:25.720 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.720 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.720 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:25.720 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159856+0000", 2024-09-21T15:00:25.720 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4349999999999997e-06 2024-09-21T15:00:25.720 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.720 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.720 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:25.720 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159866+0000", 2024-09-21T15:00:25.720 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0076e-05 2024-09-21T15:00:25.720 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.720 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.720 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:25.721 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159872+0000", 2024-09-21T15:00:25.721 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8309999999999997e-06 2024-09-21T15:00:25.721 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.721 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.721 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:25.721 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228175+0000", 2024-09-21T15:00:25.721 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068302680000000005 2024-09-21T15:00:25.721 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.721 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.721 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:25.721 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228182+0000", 2024-09-21T15:00:25.721 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3150000000000003e-06 2024-09-21T15:00:25.721 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:25.721 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:25.721 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:25.722 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.722 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.722 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45984 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:25.722 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159877+0000", 2024-09-21T15:00:25.722 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 75.498760293999993, 2024-09-21T15:00:25.722 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068328448, 2024-09-21T15:00:25.722 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:25.722 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:25.722 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:25.722 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:25.722 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:25.722 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45984 2024-09-21T15:00:25.722 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.723 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:25.723 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.723 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:25.723 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159877+0000", 2024-09-21T15:00:25.723 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:25.723 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.723 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.723 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:25.723 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159877+0000", 2024-09-21T15:00:25.723 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:25.723 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.723 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.723 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:25.723 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159877+0000", 2024-09-21T15:00:25.724 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:25.724 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.724 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.724 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:25.724 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159880+0000", 2024-09-21T15:00:25.724 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7610000000000001e-06 2024-09-21T15:00:25.724 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.724 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.724 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:25.724 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159888+0000", 2024-09-21T15:00:25.724 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0330000000000003e-06 2024-09-21T15:00:25.724 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.724 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.724 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:25.725 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159893+0000", 2024-09-21T15:00:25.725 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3550000000000004e-06 2024-09-21T15:00:25.725 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.725 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.725 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:25.725 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228199+0000", 2024-09-21T15:00:25.725 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068306438999999997 2024-09-21T15:00:25.725 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.725 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.725 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:25.725 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228206+0000", 2024-09-21T15:00:25.725 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5599999999999999e-06 2024-09-21T15:00:25.725 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:25.725 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:25.725 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:25.725 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.726 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.726 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45985 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:25.726 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159897+0000", 2024-09-21T15:00:25.726 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 75.498740471000005, 2024-09-21T15:00:25.726 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068345664, 2024-09-21T15:00:25.726 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:25.726 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:25.726 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:25.726 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:25.726 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:25.726 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45985 2024-09-21T15:00:25.726 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.726 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:25.726 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.726 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:25.727 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159897+0000", 2024-09-21T15:00:25.727 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:25.727 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.727 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.727 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:25.727 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159897+0000", 2024-09-21T15:00:25.727 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:25.727 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.727 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.727 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:25.727 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159897+0000", 2024-09-21T15:00:25.727 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:25.727 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.727 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.728 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:25.728 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159900+0000", 2024-09-21T15:00:25.728 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.954e-06 2024-09-21T15:00:25.728 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.728 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.728 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:25.728 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159913+0000", 2024-09-21T15:00:25.728 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3106e-05 2024-09-21T15:00:25.728 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.728 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.728 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:25.728 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159918+0000", 2024-09-21T15:00:25.728 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5199999999999997e-06 2024-09-21T15:00:25.728 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.728 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.729 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:25.729 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228236+0000", 2024-09-21T15:00:25.729 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068317040999999995 2024-09-21T15:00:25.729 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.729 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.729 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:25.729 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228243+0000", 2024-09-21T15:00:25.729 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3880000000000002e-06 2024-09-21T15:00:25.729 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:25.729 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:25.729 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:25.730 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.730 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.730 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45986 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:25.730 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159924+0000", 2024-09-21T15:00:25.730 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 75.498714175999993, 2024-09-21T15:00:25.730 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068364362999999997, 2024-09-21T15:00:25.730 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:25.730 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:25.730 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:25.730 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:25.730 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:25.730 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45986 2024-09-21T15:00:25.730 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.730 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:25.731 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.731 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:25.731 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159924+0000", 2024-09-21T15:00:25.731 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:25.731 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.731 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.731 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:25.731 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159924+0000", 2024-09-21T15:00:25.731 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:25.731 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.731 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.731 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:25.731 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159923+0000", 2024-09-21T15:00:25.731 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:25.732 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.732 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.732 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:25.732 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159926+0000", 2024-09-21T15:00:25.732 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0510000000000001e-06 2024-09-21T15:00:25.732 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.732 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.732 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:25.732 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159932+0000", 2024-09-21T15:00:25.732 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4160000000000001e-06 2024-09-21T15:00:25.732 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.732 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.732 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:25.732 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159938+0000", 2024-09-21T15:00:25.733 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.383e-06 2024-09-21T15:00:25.733 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.733 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.733 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:25.733 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228275+0000", 2024-09-21T15:00:25.733 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068337007000000005 2024-09-21T15:00:25.733 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.733 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.733 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:25.733 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228288+0000", 2024-09-21T15:00:25.733 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3052e-05 2024-09-21T15:00:25.733 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:25.733 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:25.733 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:25.734 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.734 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.734 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45987 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:25.734 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159944+0000", 2024-09-21T15:00:25.734 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 75.498693951999996, 2024-09-21T15:00:25.734 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068387232000000006, 2024-09-21T15:00:25.734 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:25.734 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:25.734 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:25.734 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:25.734 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:25.734 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45987 2024-09-21T15:00:25.734 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.735 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:25.735 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.735 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:25.735 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159944+0000", 2024-09-21T15:00:25.735 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:25.735 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.735 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.735 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:25.735 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159944+0000", 2024-09-21T15:00:25.735 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:25.735 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.735 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.735 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:25.735 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159943+0000", 2024-09-21T15:00:25.736 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:25.736 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.736 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.736 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:25.736 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159946+0000", 2024-09-21T15:00:25.736 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6309999999999999e-06 2024-09-21T15:00:25.736 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.736 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.736 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:25.736 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159952+0000", 2024-09-21T15:00:25.736 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6540000000000004e-06 2024-09-21T15:00:25.736 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.736 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.737 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:25.737 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159957+0000", 2024-09-21T15:00:25.737 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9280000000000001e-06 2024-09-21T15:00:25.737 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.737 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.737 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:25.737 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228322+0000", 2024-09-21T15:00:25.737 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068365067000000002 2024-09-21T15:00:25.737 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.737 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.737 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:25.737 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228331+0000", 2024-09-21T15:00:25.737 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3270000000000007e-06 2024-09-21T15:00:25.737 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:25.738 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:25.738 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:25.738 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.738 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.738 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:97672 3.4 3:26046d29:::rbd_data.1136bfc79a70.0000000000000001:head [write 0~4718592 [fadvise_sequential+fadvise_nocache],write 5242880~131072 [fadvise_sequential+fadvise_nocache],write 5439488~147456 [fadvise_sequential+fadvise_nocache],write 5652480~229376 [fadvise_sequential+fadvise_nocache],write 5947392~147456 [fadvise_sequential+fadvise_nocache],write 6160384~229376 [fadvise_sequential+fadvise_nocache],write 6455296~147456 [fadvise_sequential+fadvise_nocache],write 6668288~229376 [fadvise_sequential+fadvise_nocache],write 6963200~147456 [fadvise_sequential+fadvise_nocache],write 7176192~147456 [fadvise_sequential+fadvise_nocache],write 7340032~65536 [fadvise_sequential+fadvise_nocache],write 7471104~147456 [fadvise_sequential+fadvise_nocache],write 7684096~147456 [fadvise_sequential+fadvise_nocache],write 7864320~49152 [fadvise_sequential+fadvise_nocache],write 7979008~147456 [fadvise_sequential+fadvise_nocache],write 8192000~147456 [fadvise_sequential+fadvise_nocache],write 8388608~32768 [fadvise_sequential+fadvise_nocache],write 8519680~114688 [fadvise_sequential+fadvise_nocache],write 8732672~114688 [fadvise_sequential+fadvise_nocache],write 8912896~16384 [fadvise_sequential+fadvise_nocache],write 9109504~32768 [fadvise_sequential+fadvise_nocache],write 9322496~32768 [fadvise_sequential+fadvise_nocache],write 9617408~32768 [fadvise_sequential+fadvise_nocache],write 9830400~32768 [fadvise_sequential+fadvise_nocache],write 9961472~98304 [fadvise_sequential+fadvise_nocache],write 10076160~196608 [fadvise_sequential+fadvise_nocache],write 10289152~278528 [fadvise_sequential+fadvise_nocache],write 10584064~196608 [fadvise_sequential+fadvise_nocache],write 10797056~196608 [fadvise_sequential+fadvise_nocache],write 11010048~65536 [fadvise_sequential+fadvise_nocache],write 11091968~196608 [fadvise_sequential+fadvise_nocache],write 11304960~196608 [fadvise_sequential+fadvise_nocache],write 11517952~65536 [fadvise_sequential+fadvise_nocache],write 11616256~98304 [fadvise_sequential+fadvise_nocache],write 11730944~65536 [fadvise_sequential+fadvise_nocache],write 11829248~98304 [fadvise_sequential+fadvise_nocache],write 11943936~65536 [fadvise_sequential+fadvise_nocache],write 12042240~49152 [fadvise_sequential+fadvise_nocache],write 12124160~98304 [fadvise_sequential+fadvise_nocache],write 12238848~65536 [fadvise_sequential+fadvise_nocache],write 12337152~98304 [fadvise_sequential+fadvise_nocache],write 12451840~65536 [fadvise_sequential+fadvise_nocache],write 12550144~32768 [fadvise_sequential+fadvise_nocache],write 12632064~98304 [fadvise_sequential+fadvise_nocache],write 12845056~98304 [fadvise_sequential+fadvise_nocache],write 13058048~49152 [fadvise_sequential+fadvise_nocache],write 13139968~98304 [fadvise_sequential+fadvise_nocache],write 13352960~98304 [fadvise_sequential+fadvise_nocache],write 13565952~65536 [fadvise_sequential+fadvise_nocache],write 13647872~98304 [fadvise_sequential+fadvise_nocache],write 13860864~98304 [fadvise_sequential+fadvise_nocache],write 14073856~81920 [fadvise_sequential+fadvise_nocache],write 15269888~81920 [fadvise_sequential+fadvise_nocache],write 15482880~81920 [fadvise_sequential+fadvise_nocache],write 15695872~32768 [fadvise_sequential+fadvise_nocache],write 15777792~81920 [fadvise_sequential+fadvise_nocache],write 15990784~81920 [fadvise_sequential+fadvise_nocache],write 16203776~49152 [fadvise_sequential+fadvise_nocache],write 16285696~81920 [fadvise_sequential+fadvise_nocache],write 16498688~81920 [fadvise_sequential+fadvise_nocache],write 16711680~65536 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:25.738 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.298669+0000", 2024-09-21T15:00:25.738 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 20.359968886000001, 2024-09-21T15:00:25.738 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.10282358900000001, 2024-09-21T15:00:25.738 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:25.738 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:25.738 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:25.738 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:25.738 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:25.738 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97672 2024-09-21T15:00:25.739 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.739 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:25.739 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.739 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:25.739 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.298669+0000", 2024-09-21T15:00:25.739 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:25.739 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.739 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.739 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:25.739 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.298669+0000", 2024-09-21T15:00:25.739 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:25.739 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.740 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.740 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:25.740 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.298667+0000", 2024-09-21T15:00:25.740 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:00:25.740 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.740 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.740 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:25.740 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.327458+0000", 2024-09-21T15:00:25.740 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.028790754000000002 2024-09-21T15:00:25.740 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.740 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.740 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:25.740 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.327492+0000", 2024-09-21T15:00:25.740 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3806999999999997e-05 2024-09-21T15:00:25.741 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.741 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.741 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:25.741 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.327507+0000", 2024-09-21T15:00:25.741 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5112e-05 2024-09-21T15:00:25.741 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.741 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.741 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:25.741 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.327597+0000", 2024-09-21T15:00:25.741 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9938999999999994e-05 2024-09-21T15:00:25.741 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.741 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.741 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:25.741 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.327927+0000", 2024-09-21T15:00:25.742 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000329766 2024-09-21T15:00:25.742 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.742 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.742 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T15:00:25.742 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.330735+0000", 2024-09-21T15:00:25.742 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0028086869999999998 2024-09-21T15:00:25.742 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.742 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.742 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:25.742 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.378299+0000", 2024-09-21T15:00:25.742 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.047563495999999997 2024-09-21T15:00:25.742 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.742 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.742 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:25.742 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.401248+0000", 2024-09-21T15:00:25.743 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.022949311 2024-09-21T15:00:25.743 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.743 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.743 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:25.743 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.401446+0000", 2024-09-21T15:00:25.743 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019831199999999999 2024-09-21T15:00:25.743 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:25.743 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:25.743 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:25.743 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.401492+0000", 2024-09-21T15:00:25.743 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6109000000000002e-05 2024-09-21T15:00:25.743 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:25.743 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:25.743 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:25.744 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:25.744 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:25.744 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:25.745 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T15:00:25.746 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:00:25.738+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:00:25.846 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:00:25.844+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:00:25.901 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:25.901 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:00:25.901 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T15:00:25.901 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:25.912 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T15:00:25.947 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:00:25.944+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:00:26.024 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:26.024 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:00:26.024 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T15:00:26.024 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T15:00:26.024 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:26.034 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T15:00:26.048 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:00:26.042+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:00:26.095 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T15:00:26.141 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:26.141 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T15:00:26.141 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T15:00:26.142 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T15:00:26.142 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.142 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:10651 3.12 3:4e9d23af:::rbd_data.1136f5ed5dd6.0000000000000020:head [write 0~81920 [fadvise_sequential+fadvise_nocache],write 90112~8192 [fadvise_sequential+fadvise_nocache],write 122880~122880 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:26.142 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.292476+0000", 2024-09-21T15:00:26.142 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 106.84506107, 2024-09-21T15:00:26.142 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012765669, 2024-09-21T15:00:26.142 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:26.142 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:26.142 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:26.142 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:26.142 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:26.142 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 10651 2024-09-21T15:00:26.143 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.143 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:26.143 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.143 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:26.143 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292476+0000", 2024-09-21T15:00:26.143 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:26.143 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.143 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.143 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:26.143 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292476+0000", 2024-09-21T15:00:26.143 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:26.143 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.143 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.143 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:26.143 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292475+0000", 2024-09-21T15:00:26.144 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:00:26.144 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.144 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.144 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:26.144 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292696+0000", 2024-09-21T15:00:26.144 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000220201 2024-09-21T15:00:26.144 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.144 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.144 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:26.144 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292711+0000", 2024-09-21T15:00:26.144 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5803000000000001e-05 2024-09-21T15:00:26.144 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.144 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.144 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:26.144 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292721+0000", 2024-09-21T15:00:26.145 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6160000000000008e-06 2024-09-21T15:00:26.145 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.145 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.145 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:26.145 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292757+0000", 2024-09-21T15:00:26.145 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6211999999999998e-05 2024-09-21T15:00:26.145 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.145 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.145 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:26.145 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292878+0000", 2024-09-21T15:00:26.145 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000120306 2024-09-21T15:00:26.145 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.145 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.145 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 3", 2024-09-21T15:00:26.145 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.293019+0000", 2024-09-21T15:00:26.145 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014186400000000001 2024-09-21T15:00:26.146 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.146 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.146 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:26.146 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.294898+0000", 2024-09-21T15:00:26.146 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001878434 2024-09-21T15:00:26.146 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.146 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.146 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:26.146 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.305162+0000", 2024-09-21T15:00:26.146 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.01026373 2024-09-21T15:00:26.146 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.146 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.146 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:26.146 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.305209+0000", 2024-09-21T15:00:26.147 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7615999999999999e-05 2024-09-21T15:00:26.147 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.147 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.147 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:26.147 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.305242+0000", 2024-09-21T15:00:26.147 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2979999999999999e-05 2024-09-21T15:00:26.147 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:26.147 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:26.147 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:26.147 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.147 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.147 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12214 3.4 3:21b43b45:::rbd_data.11363a21d2be.000000000000002b:10 [read 376832~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:26.147 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.619969+0000", 2024-09-21T15:00:26.147 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 106.51756852, 2024-09-21T15:00:26.148 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0090292089999999998, 2024-09-21T15:00:26.148 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:26.148 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:26.148 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:26.148 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:26.148 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:26.148 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12214 2024-09-21T15:00:26.148 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.148 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:26.148 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.148 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:26.148 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619969+0000", 2024-09-21T15:00:26.148 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:26.148 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.148 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.149 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:26.149 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619969+0000", 2024-09-21T15:00:26.149 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:26.149 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.149 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.149 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:26.149 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619969+0000", 2024-09-21T15:00:26.149 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:26.149 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.149 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.149 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:26.149 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619971+0000", 2024-09-21T15:00:26.149 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1830000000000001e-06 2024-09-21T15:00:26.149 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.150 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.150 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:26.150 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619976+0000", 2024-09-21T15:00:26.150 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7129999999999996e-06 2024-09-21T15:00:26.150 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.150 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.150 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:26.150 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619981+0000", 2024-09-21T15:00:26.150 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2460000000000003e-06 2024-09-21T15:00:26.150 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.150 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.150 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:26.150 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628602+0000", 2024-09-21T15:00:26.150 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086209519999999994 2024-09-21T15:00:26.151 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.151 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.151 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:26.151 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628734+0000", 2024-09-21T15:00:26.151 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000132242 2024-09-21T15:00:26.151 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.151 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.151 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:26.151 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628998+0000", 2024-09-21T15:00:26.151 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00026419799999999999 2024-09-21T15:00:26.151 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:26.151 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:26.151 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:26.151 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.151 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.151 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12215 3.4 3:21b43b45:::rbd_data.11363a21d2be.000000000000002b:10 [read 409600~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:26.152 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.619985+0000", 2024-09-21T15:00:26.152 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 106.517552454, 2024-09-21T15:00:26.152 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0095457319999999995, 2024-09-21T15:00:26.152 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:26.152 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:26.152 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:26.152 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:26.152 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:26.152 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12215 2024-09-21T15:00:26.152 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.152 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:26.152 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.152 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:26.152 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619985+0000", 2024-09-21T15:00:26.152 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:26.153 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.153 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.153 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:26.153 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619985+0000", 2024-09-21T15:00:26.153 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:26.153 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.153 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.153 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:26.153 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619985+0000", 2024-09-21T15:00:26.153 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:26.153 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.153 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.153 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:26.153 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619987+0000", 2024-09-21T15:00:26.154 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6129999999999998e-06 2024-09-21T15:00:26.154 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.154 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.154 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:26.154 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619993+0000", 2024-09-21T15:00:26.154 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0820000000000004e-06 2024-09-21T15:00:26.154 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.154 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.154 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:26.154 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619998+0000", 2024-09-21T15:00:26.154 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7759999999999997e-06 2024-09-21T15:00:26.154 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.154 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.155 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:26.155 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629022+0000", 2024-09-21T15:00:26.155 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0090237489999999993 2024-09-21T15:00:26.155 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.155 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.155 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:26.155 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629098+0000", 2024-09-21T15:00:26.155 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5735999999999995e-05 2024-09-21T15:00:26.155 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.155 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.155 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:26.155 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629531+0000", 2024-09-21T15:00:26.155 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00043310100000000003 2024-09-21T15:00:26.155 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:26.156 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:26.156 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:26.156 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.156 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.156 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12216 3.4 3:21b43b45:::rbd_data.11363a21d2be.000000000000002b:10 [read 516096~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:26.156 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.620003+0000", 2024-09-21T15:00:26.156 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 106.517534458, 2024-09-21T15:00:26.156 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0098850169999999994, 2024-09-21T15:00:26.156 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:26.156 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:26.156 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:26.156 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:26.156 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:26.156 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12216 2024-09-21T15:00:26.156 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.157 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:26.157 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.157 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:26.157 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620003+0000", 2024-09-21T15:00:26.157 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:26.157 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.157 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.157 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:26.157 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620003+0000", 2024-09-21T15:00:26.157 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:26.157 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.157 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.157 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:26.157 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620003+0000", 2024-09-21T15:00:26.157 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:26.157 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.158 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.158 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:26.158 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620006+0000", 2024-09-21T15:00:26.158 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4350000000000001e-06 2024-09-21T15:00:26.158 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.158 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.158 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:26.158 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620013+0000", 2024-09-21T15:00:26.158 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4219999999999997e-06 2024-09-21T15:00:26.158 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.158 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.158 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:26.158 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620017+0000", 2024-09-21T15:00:26.158 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.865e-06 2024-09-21T15:00:26.159 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.159 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.159 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:26.159 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629581+0000", 2024-09-21T15:00:26.159 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0095631989999999997 2024-09-21T15:00:26.159 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.159 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.159 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:26.159 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629691+0000", 2024-09-21T15:00:26.159 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000110586 2024-09-21T15:00:26.159 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.159 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.159 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:26.159 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629888+0000", 2024-09-21T15:00:26.160 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019691199999999999 2024-09-21T15:00:26.160 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:26.160 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:26.160 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:26.160 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.160 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.160 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12222 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 49152~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:26.160 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.620027+0000", 2024-09-21T15:00:26.160 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 106.517510112, 2024-09-21T15:00:26.160 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010228131999999999, 2024-09-21T15:00:26.160 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:26.160 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:26.160 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:26.160 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:26.160 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:26.160 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12222 2024-09-21T15:00:26.161 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.161 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:26.161 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.161 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:26.161 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620027+0000", 2024-09-21T15:00:26.161 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:26.161 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.161 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.161 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:26.161 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620027+0000", 2024-09-21T15:00:26.161 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:26.161 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.161 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.161 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:26.161 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620027+0000", 2024-09-21T15:00:26.161 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:26.162 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.162 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.162 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:26.162 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620030+0000", 2024-09-21T15:00:26.162 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0400000000000001e-06 2024-09-21T15:00:26.162 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.162 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.162 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:26.162 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620037+0000", 2024-09-21T15:00:26.162 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8220000000000001e-06 2024-09-21T15:00:26.162 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.162 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.162 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:26.162 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620042+0000", 2024-09-21T15:00:26.162 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.843e-06 2024-09-21T15:00:26.162 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.163 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.163 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:26.163 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629922+0000", 2024-09-21T15:00:26.163 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0098805100000000003 2024-09-21T15:00:26.163 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.163 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.163 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:26.163 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.630020+0000", 2024-09-21T15:00:26.163 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7596999999999994e-05 2024-09-21T15:00:26.163 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.163 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.163 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:26.163 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.630256+0000", 2024-09-21T15:00:26.163 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00023573299999999999 2024-09-21T15:00:26.163 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:26.164 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:26.164 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:26.164 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.164 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.164 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12260 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 57344~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:26.164 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.624587+0000", 2024-09-21T15:00:26.164 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 106.512950257, 2024-09-21T15:00:26.164 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0088282069999999994, 2024-09-21T15:00:26.164 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:26.164 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:26.164 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:26.164 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:26.164 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:26.165 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12260 2024-09-21T15:00:26.165 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.165 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:26.165 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.165 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:26.165 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624587+0000", 2024-09-21T15:00:26.165 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:26.165 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.165 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.165 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:26.165 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624587+0000", 2024-09-21T15:00:26.165 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:26.165 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.165 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.165 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:26.166 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624587+0000", 2024-09-21T15:00:26.166 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-21T15:00:26.166 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.166 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.166 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:26.166 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624589+0000", 2024-09-21T15:00:26.166 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4339999999999999e-06 2024-09-21T15:00:26.166 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.166 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.166 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:26.166 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624593+0000", 2024-09-21T15:00:26.166 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5859999999999998e-06 2024-09-21T15:00:26.166 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.166 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.167 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:26.167 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624596+0000", 2024-09-21T15:00:26.167 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5830000000000001e-06 2024-09-21T15:00:26.167 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.167 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.167 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:26.167 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.633132+0000", 2024-09-21T15:00:26.167 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.008536666 2024-09-21T15:00:26.167 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.167 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.167 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:26.167 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.633216+0000", 2024-09-21T15:00:26.167 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3387999999999995e-05 2024-09-21T15:00:26.167 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.167 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.168 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:26.168 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.633415+0000", 2024-09-21T15:00:26.168 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019976799999999999 2024-09-21T15:00:26.168 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:26.168 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:26.168 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:26.168 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.168 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.168 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12517 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 278528~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:26.168 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645214+0000", 2024-09-21T15:00:26.168 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 106.492323607, 2024-09-21T15:00:26.168 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086832899999999998, 2024-09-21T15:00:26.168 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:26.168 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:26.168 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:26.168 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:26.169 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:26.169 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12517 2024-09-21T15:00:26.169 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.169 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:26.169 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.169 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:26.169 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645214+0000", 2024-09-21T15:00:26.169 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:26.169 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.169 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.169 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:26.169 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645214+0000", 2024-09-21T15:00:26.169 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:26.169 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.169 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.169 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:26.170 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645214+0000", 2024-09-21T15:00:26.170 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:26.170 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.170 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.170 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:26.170 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645216+0000", 2024-09-21T15:00:26.170 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5579999999999999e-06 2024-09-21T15:00:26.170 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.170 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.170 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:26.170 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645224+0000", 2024-09-21T15:00:26.170 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2439999999999993e-06 2024-09-21T15:00:26.170 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.171 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.171 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:26.171 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645231+0000", 2024-09-21T15:00:26.171 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1859999999999997e-06 2024-09-21T15:00:26.171 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.171 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.171 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:26.171 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.653651+0000", 2024-09-21T15:00:26.171 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0084204220000000003 2024-09-21T15:00:26.171 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.171 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.171 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:26.171 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.653726+0000", 2024-09-21T15:00:26.171 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5356000000000002e-05 2024-09-21T15:00:26.171 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.171 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.172 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:26.172 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.653897+0000", 2024-09-21T15:00:26.172 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000170848 2024-09-21T15:00:26.172 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:26.172 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:26.172 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:26.172 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.172 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.172 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12518 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 385024~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:26.172 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645235+0000", 2024-09-21T15:00:26.172 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 106.492302381, 2024-09-21T15:00:26.172 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0089841020000000008, 2024-09-21T15:00:26.172 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:26.172 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:26.172 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:26.172 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:26.173 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:26.173 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12518 2024-09-21T15:00:26.173 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.173 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:26.173 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.173 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:26.173 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645235+0000", 2024-09-21T15:00:26.173 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:26.173 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.173 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.173 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:26.173 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645235+0000", 2024-09-21T15:00:26.173 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:26.173 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.173 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.174 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:26.174 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645235+0000", 2024-09-21T15:00:26.174 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:26.174 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.174 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.174 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:26.174 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645238+0000", 2024-09-21T15:00:26.174 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.869e-06 2024-09-21T15:00:26.174 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.174 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.174 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:26.174 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645244+0000", 2024-09-21T15:00:26.174 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8930000000000002e-06 2024-09-21T15:00:26.174 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.175 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.175 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:26.175 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645249+0000", 2024-09-21T15:00:26.175 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2859999999999999e-06 2024-09-21T15:00:26.175 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.175 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.175 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:26.175 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.653930+0000", 2024-09-21T15:00:26.175 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086815320000000005 2024-09-21T15:00:26.175 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.175 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.175 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:26.175 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654028+0000", 2024-09-21T15:00:26.175 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7578000000000007e-05 2024-09-21T15:00:26.175 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.176 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.176 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:26.176 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654219+0000", 2024-09-21T15:00:26.176 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019132099999999999 2024-09-21T15:00:26.176 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:26.176 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:26.176 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:26.176 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.176 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.176 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12538 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 147456~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:26.176 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645253+0000", 2024-09-21T15:00:26.176 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 106.49228475699999, 2024-09-21T15:00:26.176 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0092095360000000008, 2024-09-21T15:00:26.177 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:26.177 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:26.177 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:26.177 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:26.177 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:26.177 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12538 2024-09-21T15:00:26.177 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.177 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:26.177 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.177 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:26.177 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645253+0000", 2024-09-21T15:00:26.177 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:26.177 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.177 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.178 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:26.178 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645253+0000", 2024-09-21T15:00:26.178 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:26.178 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.178 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.178 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:26.178 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645252+0000", 2024-09-21T15:00:26.178 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:26.178 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.178 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.178 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:26.178 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645255+0000", 2024-09-21T15:00:26.179 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2290000000000002e-06 2024-09-21T15:00:26.179 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.179 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.179 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:26.179 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645267+0000", 2024-09-21T15:00:26.179 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2631e-05 2024-09-21T15:00:26.179 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.179 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.179 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:26.179 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645278+0000", 2024-09-21T15:00:26.179 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0847e-05 2024-09-21T15:00:26.179 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.179 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.179 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:26.179 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654251+0000", 2024-09-21T15:00:26.180 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0089731340000000007 2024-09-21T15:00:26.180 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.180 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.180 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:26.180 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654296+0000", 2024-09-21T15:00:26.180 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4353e-05 2024-09-21T15:00:26.180 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.180 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.180 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:26.180 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654462+0000", 2024-09-21T15:00:26.180 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016668499999999999 2024-09-21T15:00:26.180 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:26.180 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:26.180 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:26.181 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.181 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.181 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12539 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 253952~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:26.181 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645284+0000", 2024-09-21T15:00:26.181 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 106.49225353600001, 2024-09-21T15:00:26.181 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0094524029999999998, 2024-09-21T15:00:26.181 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:26.181 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:26.181 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:26.181 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:26.181 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:26.181 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12539 2024-09-21T15:00:26.181 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.181 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:26.181 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.181 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:26.182 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645284+0000", 2024-09-21T15:00:26.182 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:26.182 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.182 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.182 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:26.182 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645284+0000", 2024-09-21T15:00:26.182 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:26.182 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.182 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.182 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:26.182 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645284+0000", 2024-09-21T15:00:26.182 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:26.182 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.182 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.183 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:26.183 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645286+0000", 2024-09-21T15:00:26.183 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7159999999999999e-06 2024-09-21T15:00:26.183 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.183 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.183 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:26.183 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645292+0000", 2024-09-21T15:00:26.183 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7400000000000001e-06 2024-09-21T15:00:26.183 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.183 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.183 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:26.183 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645297+0000", 2024-09-21T15:00:26.183 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9019999999999998e-06 2024-09-21T15:00:26.183 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.184 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.184 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:26.184 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654490+0000", 2024-09-21T15:00:26.184 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0091932360000000005 2024-09-21T15:00:26.184 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.184 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.184 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:26.184 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654565+0000", 2024-09-21T15:00:26.184 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4657e-05 2024-09-21T15:00:26.184 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.184 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.184 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:26.184 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654736+0000", 2024-09-21T15:00:26.184 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017156199999999999 2024-09-21T15:00:26.184 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:26.184 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:26.185 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:26.185 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.185 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.185 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12540 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 286720~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:26.185 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645302+0000", 2024-09-21T15:00:26.185 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 106.492235267, 2024-09-21T15:00:26.185 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0096603090000000006, 2024-09-21T15:00:26.185 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:26.185 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:26.185 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:26.185 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:26.185 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:26.185 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12540 2024-09-21T15:00:26.185 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.185 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:26.186 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.186 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:26.186 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645302+0000", 2024-09-21T15:00:26.186 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:26.186 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.186 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.186 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:26.186 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645302+0000", 2024-09-21T15:00:26.186 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:26.186 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.186 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.186 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:26.186 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645302+0000", 2024-09-21T15:00:26.186 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:26.186 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.187 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.187 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:26.187 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645305+0000", 2024-09-21T15:00:26.187 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7329999999999999e-06 2024-09-21T15:00:26.187 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.187 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.187 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:26.187 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645316+0000", 2024-09-21T15:00:26.187 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1832e-05 2024-09-21T15:00:26.187 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.187 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.187 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:26.187 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645322+0000", 2024-09-21T15:00:26.187 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.169e-06 2024-09-21T15:00:26.188 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.188 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.188 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:26.188 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654778+0000", 2024-09-21T15:00:26.188 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0094559090000000002 2024-09-21T15:00:26.188 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.188 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.188 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:26.188 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654828+0000", 2024-09-21T15:00:26.188 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0461999999999999e-05 2024-09-21T15:00:26.188 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.188 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.188 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:26.189 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654963+0000", 2024-09-21T15:00:26.189 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000134583 2024-09-21T15:00:26.189 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:26.189 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:26.189 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:26.189 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.189 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.189 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12561 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:26.189 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645742+0000", 2024-09-21T15:00:26.189 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 106.491796001, 2024-09-21T15:00:26.189 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0093583810000000007, 2024-09-21T15:00:26.189 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:26.189 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:26.190 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:26.190 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:26.190 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:26.190 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12561 2024-09-21T15:00:26.190 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.190 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:26.190 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.190 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:26.190 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645742+0000", 2024-09-21T15:00:26.190 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:26.190 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.190 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.190 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:26.190 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645742+0000", 2024-09-21T15:00:26.191 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:26.191 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.191 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.191 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:26.191 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645741+0000", 2024-09-21T15:00:26.191 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:00:26.191 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.191 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.191 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:26.191 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645746+0000", 2024-09-21T15:00:26.191 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8379999999999998e-06 2024-09-21T15:00:26.191 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.191 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.192 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:26.192 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645757+0000", 2024-09-21T15:00:26.192 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0705e-05 2024-09-21T15:00:26.192 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.192 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.192 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:26.192 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645765+0000", 2024-09-21T15:00:26.192 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6729999999999992e-06 2024-09-21T15:00:26.192 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.192 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.192 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:26.192 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654993+0000", 2024-09-21T15:00:26.192 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0092287480000000002 2024-09-21T15:00:26.192 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.192 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.193 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:26.193 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.655050+0000", 2024-09-21T15:00:26.193 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6394999999999998e-05 2024-09-21T15:00:26.193 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.193 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.193 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:26.193 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.655100+0000", 2024-09-21T15:00:26.193 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0049000000000001e-05 2024-09-21T15:00:26.193 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:26.193 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:26.193 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:26.193 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.193 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.193 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:97619 3.1 e122/93 3:8730dd9c:::rbd_data.1136bfc79a70.0000000000000000:head v 122'167, mlcod=120'165)", 2024-09-21T15:00:26.194 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.292926+0000", 2024-09-21T15:00:26.194 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 20.844611932999999, 2024-09-21T15:00:26.194 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.020236437999999999, 2024-09-21T15:00:26.194 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:26.194 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:26.194 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:26.194 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.194 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:26.194 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.292926+0000", 2024-09-21T15:00:26.194 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:26.194 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.194 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.194 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:26.194 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.292926+0000", 2024-09-21T15:00:26.195 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:26.195 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.195 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.195 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:26.195 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.292925+0000", 2024-09-21T15:00:26.195 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:26.195 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.195 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.195 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:26.195 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.300394+0000", 2024-09-21T15:00:26.195 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0074683839999999998 2024-09-21T15:00:26.195 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.195 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.196 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:26.196 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.300420+0000", 2024-09-21T15:00:26.196 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6380000000000002e-05 2024-09-21T15:00:26.196 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.196 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.196 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:26.196 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.300432+0000", 2024-09-21T15:00:26.196 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1987999999999999e-05 2024-09-21T15:00:26.196 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.196 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.196 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:26.196 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.300533+0000", 2024-09-21T15:00:26.196 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000100883 2024-09-21T15:00:26.196 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.197 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.197 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:26.197 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.300562+0000", 2024-09-21T15:00:26.197 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9048999999999999e-05 2024-09-21T15:00:26.197 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.197 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.197 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:26.197 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.313121+0000", 2024-09-21T15:00:26.197 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012559038999999999 2024-09-21T15:00:26.197 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.197 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.197 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:26.197 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.313162+0000", 2024-09-21T15:00:26.197 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0998999999999998e-05 2024-09-21T15:00:26.198 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:26.198 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:26.198 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:26.198 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.198 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.198 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:97677 3.15 e122/24 3:a8e72133:::rbd_data.1136bfc79a70.0000000000000005:head v 122'567, mlcod=120'565)", 2024-09-21T15:00:26.198 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.320753+0000", 2024-09-21T15:00:26.198 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 20.816784344999999, 2024-09-21T15:00:26.198 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070639375000000004, 2024-09-21T15:00:26.198 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:26.198 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:26.198 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:26.198 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.198 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:26.199 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.320753+0000", 2024-09-21T15:00:26.199 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:26.199 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.199 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.199 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:26.199 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.320753+0000", 2024-09-21T15:00:26.199 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:26.199 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.199 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.199 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:26.199 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.320753+0000", 2024-09-21T15:00:26.199 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:26.199 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.200 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.200 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:26.200 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.375452+0000", 2024-09-21T15:00:26.200 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054698945999999998 2024-09-21T15:00:26.200 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.200 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.200 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:26.200 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.375476+0000", 2024-09-21T15:00:26.200 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4380999999999999e-05 2024-09-21T15:00:26.200 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.200 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.200 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:26.200 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.375485+0000", 2024-09-21T15:00:26.200 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0049999999999993e-06 2024-09-21T15:00:26.200 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.201 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.201 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:26.201 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.375552+0000", 2024-09-21T15:00:26.201 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6550999999999994e-05 2024-09-21T15:00:26.201 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.201 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.201 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:26.201 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.375578+0000", 2024-09-21T15:00:26.201 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6196000000000001e-05 2024-09-21T15:00:26.201 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.201 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.201 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:26.201 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.391319+0000", 2024-09-21T15:00:26.201 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015741271000000001 2024-09-21T15:00:26.202 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.202 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.202 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:26.202 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.391393+0000", 2024-09-21T15:00:26.202 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3287000000000005e-05 2024-09-21T15:00:26.202 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:26.202 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:26.202 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:26.202 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.202 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.202 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:97672 3.4 e122/51 3:26046d29:::rbd_data.1136bfc79a70.0000000000000001:head v 122'256, mlcod=120'254)", 2024-09-21T15:00:26.202 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.331696+0000", 2024-09-21T15:00:26.202 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 20.805841182000002, 2024-09-21T15:00:26.202 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041222507999999998, 2024-09-21T15:00:26.203 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:26.203 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:26.203 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:26.203 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.203 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:26.203 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.331696+0000", 2024-09-21T15:00:26.203 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:26.203 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.203 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.203 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:26.203 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.331696+0000", 2024-09-21T15:00:26.203 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:26.203 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.203 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.204 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:26.204 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.331695+0000", 2024-09-21T15:00:26.204 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:00:26.204 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.204 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.204 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:26.204 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.340175+0000", 2024-09-21T15:00:26.204 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0084799520000000007 2024-09-21T15:00:26.204 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.204 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.204 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:26.204 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.340208+0000", 2024-09-21T15:00:26.204 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2997999999999998e-05 2024-09-21T15:00:26.204 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.204 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.205 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:26.205 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.340223+0000", 2024-09-21T15:00:26.205 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5330999999999999e-05 2024-09-21T15:00:26.205 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.205 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.205 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:26.205 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.340305+0000", 2024-09-21T15:00:26.205 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2095999999999995e-05 2024-09-21T15:00:26.205 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.205 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.205 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:26.205 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.340337+0000", 2024-09-21T15:00:26.205 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2097000000000001e-05 2024-09-21T15:00:26.205 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.205 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.206 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:26.206 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.372863+0000", 2024-09-21T15:00:26.206 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.032525923999999998 2024-09-21T15:00:26.206 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.206 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.206 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:26.206 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.372919+0000", 2024-09-21T15:00:26.206 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5729000000000001e-05 2024-09-21T15:00:26.206 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:26.206 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:26.206 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:26.206 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.206 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.206 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:97675 3.b e122/24 3:d321acb8:::rbd_data.1136bfc79a70.0000000000000003:head v 122'323, mlcod=120'321)", 2024-09-21T15:00:26.207 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.341399+0000", 2024-09-21T15:00:26.207 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 20.796138110000001, 2024-09-21T15:00:26.207 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.027674555999999999, 2024-09-21T15:00:26.207 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:26.207 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:26.207 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:26.207 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.207 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:26.207 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.341399+0000", 2024-09-21T15:00:26.207 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:26.207 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.207 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.207 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:26.207 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.341399+0000", 2024-09-21T15:00:26.208 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:26.208 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.208 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.208 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:26.208 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.341398+0000", 2024-09-21T15:00:26.208 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:00:26.208 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.208 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.208 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:26.208 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.350707+0000", 2024-09-21T15:00:26.208 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0093094920000000008 2024-09-21T15:00:26.208 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.208 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.208 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:26.208 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.350730+0000", 2024-09-21T15:00:26.209 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3127999999999999e-05 2024-09-21T15:00:26.209 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.209 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.209 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:26.209 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.350741+0000", 2024-09-21T15:00:26.209 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0447e-05 2024-09-21T15:00:26.209 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.209 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.209 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:26.209 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.351206+0000", 2024-09-21T15:00:26.209 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000465439 2024-09-21T15:00:26.209 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.209 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.209 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:26.209 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.351245+0000", 2024-09-21T15:00:26.210 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8634999999999999e-05 2024-09-21T15:00:26.210 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.210 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.210 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:26.210 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.368992+0000", 2024-09-21T15:00:26.210 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.017747161000000001 2024-09-21T15:00:26.210 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.210 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.210 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:26.210 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.369074+0000", 2024-09-21T15:00:26.210 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1893999999999999e-05 2024-09-21T15:00:26.210 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:26.210 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:26.210 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:26.211 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.211 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.211 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:97733 3.4 e122/51 3:23762450:::rbd_data.1136bfc79a70.0000000000000008:head v 122'257, mlcod=120'254)", 2024-09-21T15:00:26.211 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.380585+0000", 2024-09-21T15:00:26.211 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 20.756952071000001, 2024-09-21T15:00:26.211 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.023704392000000001, 2024-09-21T15:00:26.211 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:26.211 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:26.211 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:26.211 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.211 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:26.211 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.380585+0000", 2024-09-21T15:00:26.211 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:26.211 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.211 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.211 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:26.212 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.380585+0000", 2024-09-21T15:00:26.212 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:26.212 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.212 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.212 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:26.212 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.380583+0000", 2024-09-21T15:00:26.212 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:00:26.212 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.212 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.212 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:26.212 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.386771+0000", 2024-09-21T15:00:26.212 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0061877160000000002 2024-09-21T15:00:26.212 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.213 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.213 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:26.213 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.386787+0000", 2024-09-21T15:00:26.213 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5596000000000001e-05 2024-09-21T15:00:26.213 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.213 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.213 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:26.213 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.386797+0000", 2024-09-21T15:00:26.213 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0316000000000001e-05 2024-09-21T15:00:26.213 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.213 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.213 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:26.213 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.386889+0000", 2024-09-21T15:00:26.213 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2335000000000003e-05 2024-09-21T15:00:26.213 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.214 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.214 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:26.214 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.386914+0000", 2024-09-21T15:00:26.214 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.499e-05 2024-09-21T15:00:26.214 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.214 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.214 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:26.214 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.404248+0000", 2024-09-21T15:00:26.214 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.017333237000000001 2024-09-21T15:00:26.214 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.214 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.214 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:26.214 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.404290+0000", 2024-09-21T15:00:26.214 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2327000000000003e-05 2024-09-21T15:00:26.215 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:26.215 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:26.215 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:26.215 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.215 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.215 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:104955 3.1 3:85de63d1:::rbd_data.1136d175325b.0000000000000033:head [write 0~32768 [fadvise_sequential+fadvise_nocache],write 65536~229376 [fadvise_sequential+fadvise_nocache],write 327680~229376 [fadvise_sequential+fadvise_nocache],write 589824~229376 [fadvise_sequential+fadvise_nocache],write 851968~229376 [fadvise_sequential+fadvise_nocache],write 1114112~229376 [fadvise_sequential+fadvise_nocache],write 1376256~229376 [fadvise_sequential+fadvise_nocache],write 1638400~229376 [fadvise_sequential+fadvise_nocache],write 1900544~196608 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-21T15:00:26.215 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:14.515375+0000", 2024-09-21T15:00:26.215 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 11.622163016, 2024-09-21T15:00:26.215 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0095863219999999996, 2024-09-21T15:00:26.215 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:26.215 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:26.215 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:26.215 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:26.215 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:26.215 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 104955 2024-09-21T15:00:26.216 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.216 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:26.216 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.216 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:26.216 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:14.515375+0000", 2024-09-21T15:00:26.216 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:26.216 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.216 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.216 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:26.216 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:14.515375+0000", 2024-09-21T15:00:26.216 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:26.216 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.216 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.216 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:26.216 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:14.515374+0000", 2024-09-21T15:00:26.216 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:26.217 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.217 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.217 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:26.217 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:14.517138+0000", 2024-09-21T15:00:26.217 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0017643100000000001 2024-09-21T15:00:26.217 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.217 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.217 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:26.217 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:14.517151+0000", 2024-09-21T15:00:26.217 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3329999999999999e-05 2024-09-21T15:00:26.217 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.217 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.217 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:26.217 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:14.517160+0000", 2024-09-21T15:00:26.217 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6610000000000006e-06 2024-09-21T15:00:26.217 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.217 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.218 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:26.218 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:14.518305+0000", 2024-09-21T15:00:26.218 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001145189 2024-09-21T15:00:26.218 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.218 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.218 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:26.218 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:14.518433+0000", 2024-09-21T15:00:26.218 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000128045 2024-09-21T15:00:26.218 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.218 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.218 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-21T15:00:26.218 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:14.518627+0000", 2024-09-21T15:00:26.218 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019324300000000001 2024-09-21T15:00:26.218 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.218 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.219 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:26.219 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:14.522215+0000", 2024-09-21T15:00:26.219 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0035883740000000001 2024-09-21T15:00:26.219 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.219 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.219 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:26.219 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:14.524843+0000", 2024-09-21T15:00:26.219 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0026280359999999998 2024-09-21T15:00:26.219 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.219 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.219 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:26.219 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:14.524917+0000", 2024-09-21T15:00:26.219 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3924000000000006e-05 2024-09-21T15:00:26.219 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.219 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.220 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:26.220 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:14.524961+0000", 2024-09-21T15:00:26.220 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3906000000000003e-05 2024-09-21T15:00:26.220 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:26.220 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:26.220 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:26.220 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.220 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.220 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:105964 3.1 3:8730dd9c:::rbd_data.1136bfc79a70.0000000000000000:head [sparse-read 0~3145728] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-21T15:00:26.220 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:15.409532+0000", 2024-09-21T15:00:26.220 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 10.728005141000001, 2024-09-21T15:00:26.220 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010457086000000001, 2024-09-21T15:00:26.220 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:26.220 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:26.220 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:26.220 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:26.221 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:26.221 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 105964 2024-09-21T15:00:26.221 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.221 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:26.221 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.221 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:26.221 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:15.409532+0000", 2024-09-21T15:00:26.221 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:26.221 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.221 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.221 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:26.221 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:15.409532+0000", 2024-09-21T15:00:26.221 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:26.221 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.221 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.221 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:26.222 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:15.409529+0000", 2024-09-21T15:00:26.222 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999971 2024-09-21T15:00:26.222 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.222 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.222 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:26.222 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:15.409545+0000", 2024-09-21T15:00:26.222 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5968999999999999e-05 2024-09-21T15:00:26.222 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.222 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.222 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:26.222 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:15.409583+0000", 2024-09-21T15:00:26.222 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7490000000000002e-05 2024-09-21T15:00:26.222 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.222 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.222 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:26.223 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:15.409600+0000", 2024-09-21T15:00:26.223 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7472000000000001e-05 2024-09-21T15:00:26.223 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.223 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.223 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:26.223 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:15.409757+0000", 2024-09-21T15:00:26.223 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015679100000000001 2024-09-21T15:00:26.223 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.223 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.223 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:26.223 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:15.409983+0000", 2024-09-21T15:00:26.223 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00022590500000000001 2024-09-21T15:00:26.223 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.223 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.223 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:26.223 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:15.419989+0000", 2024-09-21T15:00:26.224 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010006456 2024-09-21T15:00:26.224 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:26.224 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:26.224 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:26.224 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.224 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.224 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:108350 3.12 3:4e90d7f9:::rbd_data.1136d80ba95e.0000000000000141:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-21T15:00:26.224 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:20.588954+0000", 2024-09-21T15:00:26.224 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.5485838510000001, 2024-09-21T15:00:26.224 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.017428454999999999, 2024-09-21T15:00:26.224 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:26.224 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:26.224 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:26.224 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:26.225 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:26.225 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 108350 2024-09-21T15:00:26.225 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.225 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:26.225 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.225 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:26.225 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:20.588954+0000", 2024-09-21T15:00:26.225 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:26.225 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.225 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.225 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:26.225 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:20.588954+0000", 2024-09-21T15:00:26.225 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:26.225 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.225 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.226 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:26.226 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:20.588951+0000", 2024-09-21T15:00:26.226 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-21T15:00:26.226 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.226 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.226 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:26.226 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:20.588967+0000", 2024-09-21T15:00:26.226 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5923999999999999e-05 2024-09-21T15:00:26.226 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.226 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.226 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:26.226 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:20.589000+0000", 2024-09-21T15:00:26.226 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2614999999999999e-05 2024-09-21T15:00:26.226 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.227 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.227 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:26.227 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:20.589014+0000", 2024-09-21T15:00:26.227 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4627000000000001e-05 2024-09-21T15:00:26.227 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.227 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.227 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:26.227 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:20.589148+0000", 2024-09-21T15:00:26.227 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000133934 2024-09-21T15:00:26.227 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.227 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.227 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:26.227 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:20.589548+0000", 2024-09-21T15:00:26.227 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00040010700000000002 2024-09-21T15:00:26.227 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.227 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.228 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 3", 2024-09-21T15:00:26.228 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:20.593837+0000", 2024-09-21T15:00:26.228 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0042885279999999998 2024-09-21T15:00:26.228 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.228 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.228 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:26.228 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:20.606168+0000", 2024-09-21T15:00:26.228 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012331482 2024-09-21T15:00:26.228 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.228 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.228 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:26.228 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:20.606198+0000", 2024-09-21T15:00:26.228 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9833e-05 2024-09-21T15:00:26.228 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.228 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.229 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:26.229 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:20.606299+0000", 2024-09-21T15:00:26.229 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000100388 2024-09-21T15:00:26.229 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:26.229 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:26.229 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:26.229 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:20.606382+0000", 2024-09-21T15:00:26.229 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3500999999999999e-05 2024-09-21T15:00:26.229 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:26.229 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:26.229 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:26.229 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:26.229 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:26.229 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:26.231 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T15:00:26.231 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:00:26.226+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:00:26.250 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T15:00:26.277 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T15:00:26.282 INFO:teuthology.orchestra.run.smithi032.stderr:nodeep-scrub is unset 2024-09-21T15:00:26.301 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T15:00:26.319 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T15:00:26.331 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:00:26.328+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:00:26.339 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-21T15:00:26.363 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T15:00:26.381 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:26.381 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:00:26.382 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T15:00:26.382 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:26.387 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T15:00:26.391 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T15:00:26.412 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T15:00:26.415 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:00:26.416 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:00:26.416 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:00:26.416 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:00:26.416 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:00:26.416 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:00:26.416 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:00:26.416 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:00:26.432 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:00:26.428+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:00:26.499 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:26.499 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:26.509 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T15:00:26.532 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:00:26.526+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:00:26.612 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:26.612 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:26.623 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T15:00:26.666 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:00:26.662+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:00:26.734 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:00:26.732+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:00:26.741 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:26.741 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:26.752 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T15:00:26.835 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:00:26.832+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:00:26.912 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:26.913 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:26.923 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T15:00:26.935 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:00:26.930+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:00:27.027 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:27.027 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:27.036 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:00:27.030+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:00:27.037 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T15:00:27.137 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:00:27.136+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:00:27.140 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:27.141 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:27.150 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T15:00:27.238 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:00:27.236+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:00:27.289 INFO:teuthology.orchestra.run.smithi032.stdout:checking clone #26, image image_client.0-clone26 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent27 2024-09-21T15:00:27.310 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:27.310 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:27.320 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T15:00:27.365 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:00:27.364+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:00:27.428 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:27.428 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:27.439 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T15:00:27.439 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:00:27.436+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:00:27.471 INFO:teuthology.orchestra.run.smithi032.stdout:3860 write 0x20e1399 thru 0x20e51cd (0x3e35 bytes) 2024-09-21T15:00:27.485 INFO:teuthology.orchestra.run.smithi032.stdout:3862 read 0x5b7e7de thru 0x5b89469 (0xac8c bytes) 2024-09-21T15:00:27.486 INFO:teuthology.orchestra.run.smithi032.stdout:3863 write 0xade341d thru 0xadeb7cf (0x83b3 bytes) 2024-09-21T15:00:27.498 INFO:teuthology.orchestra.run.smithi032.stdout:3865 trunc from 0xe7a38e9 to 0xbaf4e21 2024-09-21T15:00:27.518 INFO:teuthology.orchestra.run.smithi032.stdout:3866 punch from 0x479f761 to 0x47a1f87, (0x2826 bytes) 2024-09-21T15:00:27.519 INFO:teuthology.orchestra.run.smithi032.stdout:3867 write 0x71a9f66 thru 0x71b09cb (0x6a66 bytes) 2024-09-21T15:00:27.527 INFO:teuthology.orchestra.run.smithi032.stdout:3868 read 0x47562f8 thru 0x475c33a (0x6043 bytes) 2024-09-21T15:00:27.528 INFO:teuthology.orchestra.run.smithi032.stdout:3871 read 0x8b5368f thru 0x8b5b26d (0x7bdf bytes) 2024-09-21T15:00:27.534 INFO:teuthology.orchestra.run.smithi032.stdout:3872 trunc from 0xbaf4e21 to 0x8dab42f 2024-09-21T15:00:27.539 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:00:27.534+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:00:27.550 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:27.550 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:27.550 INFO:teuthology.orchestra.run.smithi032.stdout:3878 writesame 0x3c02ce thru 0x3c7bcf data_size 0x9(0x7902 bytes) 2024-09-21T15:00:27.552 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:27.552 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:27.557 INFO:teuthology.orchestra.run.smithi032.stdout:3879 write 0x86f306e thru 0x86ffea0 (0xce33 bytes) 2024-09-21T15:00:27.561 INFO:teuthology.orchestra.run.smithi032.stdout:3880 punch from 0x5b43cf8 to 0x5b4596a, (0x1c72 bytes) 2024-09-21T15:00:27.561 INFO:teuthology.orchestra.run.smithi032.stdout:3882 read 0x342072c thru 0x3422639 (0x1f0e bytes) 2024-09-21T15:00:27.562 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T15:00:27.563 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:27.565 INFO:teuthology.orchestra.run.smithi032.stdout:3884 trunc from 0x8dab42f to 0xab9a6be 2024-09-21T15:00:27.578 INFO:teuthology.orchestra.run.smithi032.stdout:3887 write 0x6fdf1cc thru 0x6fea802 (0xb637 bytes) 2024-09-21T15:00:27.590 INFO:teuthology.orchestra.run.smithi032.stdout:3888 trunc from 0xab9a6be to 0x67c741b 2024-09-21T15:00:27.615 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:27.615 INFO:teuthology.orchestra.run.smithi032.stdout:3890 punch from 0x189e7e8 to 0x18a7d9a, (0x95b2 bytes) 2024-09-21T15:00:27.615 INFO:teuthology.orchestra.run.smithi032.stdout:3891 punch from 0x1ee4eb6 to 0x1ef2eaf, (0xdff9 bytes) 2024-09-21T15:00:27.615 INFO:teuthology.orchestra.run.smithi032.stdout:3892 read 0xf53369 thru 0xf60026 (0xccbe bytes) 2024-09-21T15:00:27.632 INFO:teuthology.orchestra.run.smithi032.stdout:3893 write 0x8ba28e3 thru 0x8ba2f35 (0x653 bytes) 2024-09-21T15:00:27.638 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:27.638 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:27.638 INFO:teuthology.orchestra.run.smithi032.stdout:3897 punch from 0x5002f61 to 0x500d8a1, (0xa940 bytes) 2024-09-21T15:00:27.638 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:27.638 INFO:teuthology.orchestra.run.smithi032.stdout:3899 read 0x34ee737 thru 0x34f1101 (0x29cb bytes) 2024-09-21T15:00:27.639 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:27.639 INFO:teuthology.orchestra.run.smithi032.stdout:3904 punch from 0x4fd1598 to 0x4fd79e1, (0x6449 bytes) 2024-09-21T15:00:27.639 INFO:teuthology.orchestra.run.smithi032.stdout:3905 clone 29 order 21 su 8192 sc 11 2024-09-21T15:00:27.640 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:00:27.634+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:00:27.677 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:27.677 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:27.687 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T15:00:27.741 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:00:27.740+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:00:27.793 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:27.793 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:27.803 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T15:00:27.845 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:00:27.844+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:00:27.942 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:00:27.940+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:00:27.955 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:27.956 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:27.966 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T15:00:28.043 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:00:28.040+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:00:28.075 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:28.075 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:28.085 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T15:00:28.145 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:00:28.138+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:00:28.242 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:28.243 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:28.245 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:00:28.244+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:00:28.252 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T15:00:28.273 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 5, 6, 7, 4] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T15:00:28.273 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T15:00:28.273 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-21T15:00:28.273 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-21T15:00:28.295 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T15:00:28.346 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:00:28.342+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:00:28.376 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:28.376 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:28.386 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T15:00:28.430 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T15:00:28.447 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:00:28.444+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:00:28.493 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:28.493 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:28.503 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T15:00:28.547 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:00:28.544+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:00:28.574 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T15:00:28.607 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T15:00:28.610 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-21T15:00:28.610 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:28.610 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 142, 2024-09-21T15:00:28.610 INFO:teuthology.orchestra.run.smithi032.stdout: "fsid": "caf51e55-9234-4a26-bcd2-d60811962560", 2024-09-21T15:00:28.610 INFO:teuthology.orchestra.run.smithi032.stdout: "created": "2024-09-21T14:57:50.338822+0000", 2024-09-21T15:00:28.610 INFO:teuthology.orchestra.run.smithi032.stdout: "modified": "2024-09-21T15:00:28.279685+0000", 2024-09-21T15:00:28.610 INFO:teuthology.orchestra.run.smithi032.stdout: "last_up_change": "2024-09-21T14:58:01.177141+0000", 2024-09-21T15:00:28.610 INFO:teuthology.orchestra.run.smithi032.stdout: "last_in_change": "2024-09-21T14:59:27.207073+0000", 2024-09-21T15:00:28.611 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-21T15:00:28.611 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_num": 5799936, 2024-09-21T15:00:28.611 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_set": [ 2024-09-21T15:00:28.611 INFO:teuthology.orchestra.run.smithi032.stdout: "pglog_hardlimit", 2024-09-21T15:00:28.611 INFO:teuthology.orchestra.run.smithi032.stdout: "purged_snapdirs", 2024-09-21T15:00:28.611 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_deletes", 2024-09-21T15:00:28.611 INFO:teuthology.orchestra.run.smithi032.stdout: "sortbitwise" 2024-09-21T15:00:28.611 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-21T15:00:28.611 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_version": 5, 2024-09-21T15:00:28.611 INFO:teuthology.orchestra.run.smithi032.stdout: "full_ratio": 0.94999998807907104, 2024-09-21T15:00:28.611 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-21T15:00:28.611 INFO:teuthology.orchestra.run.smithi032.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-21T15:00:28.611 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_snapshot": "", 2024-09-21T15:00:28.611 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_max": 3, 2024-09-21T15:00:28.611 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd": 8, 2024-09-21T15:00:28.612 INFO:teuthology.orchestra.run.smithi032.stdout: "require_min_compat_client": "luminous", 2024-09-21T15:00:28.612 INFO:teuthology.orchestra.run.smithi032.stdout: "min_compat_client": "luminous", 2024-09-21T15:00:28.612 INFO:teuthology.orchestra.run.smithi032.stdout: "require_osd_release": "reef", 2024-09-21T15:00:28.612 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_crimson": false, 2024-09-21T15:00:28.612 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2024-09-21T15:00:28.612 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:28.612 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 1, 2024-09-21T15:00:28.612 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": ".mgr", 2024-09-21T15:00:28.612 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-09-21T14:58:01.842810+0000", 2024-09-21T15:00:28.612 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-09-21T15:00:28.612 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2024-09-21T15:00:28.612 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-09-21T15:00:28.612 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-09-21T15:00:28.612 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-09-21T15:00:28.612 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-09-21T15:00:28.613 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-09-21T15:00:28.613 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-09-21T15:00:28.613 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T15:00:28.613 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T15:00:28.613 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-09-21T15:00:28.613 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-09-21T15:00:28.613 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 1, 2024-09-21T15:00:28.613 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 1, 2024-09-21T15:00:28.613 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 1, 2024-09-21T15:00:28.613 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 1, 2024-09-21T15:00:28.613 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 1, 2024-09-21T15:00:28.613 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-09-21T15:00:28.613 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-09-21T15:00:28.613 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-09-21T15:00:28.613 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-09-21T15:00:28.614 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-09-21T15:00:28.614 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-09-21T15:00:28.614 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-09-21T15:00:28.614 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.614 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "18", 2024-09-21T15:00:28.614 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-09-21T15:00:28.614 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T15:00:28.614 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T15:00:28.614 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-09-21T15:00:28.614 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-09-21T15:00:28.614 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-09-21T15:00:28.614 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-09-21T15:00:28.614 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-09-21T15:00:28.615 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-09-21T15:00:28.615 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-09-21T15:00:28.615 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-09-21T15:00:28.615 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-09-21T15:00:28.615 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-09-21T15:00:28.615 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-09-21T15:00:28.615 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-09-21T15:00:28.615 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-09-21T15:00:28.615 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-09-21T15:00:28.615 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-09-21T15:00:28.615 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T15:00:28.615 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T15:00:28.615 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T15:00:28.615 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-09-21T15:00:28.616 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-09-21T15:00:28.616 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-09-21T15:00:28.616 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-09-21T15:00:28.616 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-09-21T15:00:28.616 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.616 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-09-21T15:00:28.616 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-09-21T15:00:28.616 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-09-21T15:00:28.616 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-09-21T15:00:28.616 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-09-21T15:00:28.616 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T15:00:28.616 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-09-21T15:00:28.616 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-09-21T15:00:28.617 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-09-21T15:00:28.617 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-09-21T15:00:28.617 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-09-21T15:00:28.617 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-09-21T15:00:28.617 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_max": 32, 2024-09-21T15:00:28.617 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 1 2024-09-21T15:00:28.617 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.617 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-09-21T15:00:28.617 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": {} 2024-09-21T15:00:28.617 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.617 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-09-21T15:00:28.617 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 8, 2024-09-21T15:00:28.617 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 8, 2024-09-21T15:00:28.617 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 0.25, 2024-09-21T15:00:28.617 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 2, 2024-09-21T15:00:28.617 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 2, 2024-09-21T15:00:28.618 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 0.97936630249023438, 2024-09-21T15:00:28.618 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 0.98000001907348633, 2024-09-21T15:00:28.618 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 0.98000001907348633 2024-09-21T15:00:28.618 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:28.618 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.618 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:28.618 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 2, 2024-09-21T15:00:28.618 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "rbd", 2024-09-21T15:00:28.618 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-09-21T14:58:08.284925+0000", 2024-09-21T15:00:28.618 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 8193, 2024-09-21T15:00:28.618 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T15:00:28.618 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-09-21T15:00:28.618 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-09-21T15:00:28.619 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-09-21T15:00:28.619 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-09-21T15:00:28.619 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-09-21T15:00:28.619 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-09-21T15:00:28.619 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T15:00:28.619 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T15:00:28.619 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-09-21T15:00:28.619 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-09-21T15:00:28.619 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2024-09-21T15:00:28.619 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2024-09-21T15:00:28.619 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2024-09-21T15:00:28.619 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2024-09-21T15:00:28.619 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2024-09-21T15:00:28.619 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-09-21T15:00:28.619 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-09-21T15:00:28.620 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-09-21T15:00:28.620 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-09-21T15:00:28.620 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-09-21T15:00:28.620 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-09-21T15:00:28.620 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-09-21T15:00:28.620 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.620 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "22", 2024-09-21T15:00:28.620 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-09-21T15:00:28.620 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T15:00:28.620 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T15:00:28.620 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-09-21T15:00:28.620 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-09-21T15:00:28.620 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 3, 2024-09-21T15:00:28.620 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 22, 2024-09-21T15:00:28.621 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-09-21T15:00:28.621 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-09-21T15:00:28.621 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-09-21T15:00:28.621 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-09-21T15:00:28.621 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-09-21T15:00:28.621 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-09-21T15:00:28.621 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-09-21T15:00:28.621 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-09-21T15:00:28.621 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-09-21T15:00:28.621 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-09-21T15:00:28.621 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-09-21T15:00:28.621 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T15:00:28.621 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T15:00:28.621 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T15:00:28.621 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-09-21T15:00:28.621 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-09-21T15:00:28.622 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-09-21T15:00:28.622 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-09-21T15:00:28.622 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-09-21T15:00:28.622 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.622 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-09-21T15:00:28.622 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-09-21T15:00:28.622 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-09-21T15:00:28.622 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-09-21T15:00:28.622 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-09-21T15:00:28.622 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T15:00:28.622 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-09-21T15:00:28.622 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-09-21T15:00:28.622 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-09-21T15:00:28.622 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-09-21T15:00:28.622 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-09-21T15:00:28.623 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2024-09-21T15:00:28.623 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-09-21T15:00:28.623 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd": {} 2024-09-21T15:00:28.623 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.623 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-09-21T15:00:28.623 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 4.9800000190734863, 2024-09-21T15:00:28.623 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 4.9800000190734863, 2024-09-21T15:00:28.623 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 0.87999999523162842, 2024-09-21T15:00:28.623 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 4.380000114440918, 2024-09-21T15:00:28.623 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 4.380000114440918, 2024-09-21T15:00:28.623 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 0.97936630249023438, 2024-09-21T15:00:28.623 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 0.98000001907348633, 2024-09-21T15:00:28.623 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 0.98000001907348633 2024-09-21T15:00:28.623 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:28.623 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.623 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:28.624 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 3, 2024-09-21T15:00:28.624 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "pool_client.0", 2024-09-21T15:00:28.624 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-09-21T14:58:17.510709+0000", 2024-09-21T15:00:28.624 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 8193, 2024-09-21T15:00:28.624 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T15:00:28.624 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-09-21T15:00:28.624 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-09-21T15:00:28.624 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-09-21T15:00:28.624 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-09-21T15:00:28.624 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-09-21T15:00:28.624 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-09-21T15:00:28.624 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T15:00:28.624 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T15:00:28.624 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-09-21T15:00:28.624 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-09-21T15:00:28.624 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 32, 2024-09-21T15:00:28.625 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 32, 2024-09-21T15:00:28.625 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 32, 2024-09-21T15:00:28.625 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 32, 2024-09-21T15:00:28.625 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 32, 2024-09-21T15:00:28.625 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-09-21T15:00:28.625 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-09-21T15:00:28.625 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-09-21T15:00:28.625 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-09-21T15:00:28.625 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-09-21T15:00:28.625 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-09-21T15:00:28.625 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-09-21T15:00:28.625 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.625 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "142", 2024-09-21T15:00:28.625 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-09-21T15:00:28.625 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T15:00:28.625 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T15:00:28.626 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-09-21T15:00:28.626 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-09-21T15:00:28.626 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 88, 2024-09-21T15:00:28.626 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 142, 2024-09-21T15:00:28.626 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-09-21T15:00:28.626 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-09-21T15:00:28.626 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-09-21T15:00:28.626 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-09-21T15:00:28.626 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-09-21T15:00:28.626 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-09-21T15:00:28.626 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-09-21T15:00:28.626 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-09-21T15:00:28.626 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-09-21T15:00:28.626 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-09-21T15:00:28.626 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-09-21T15:00:28.626 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T15:00:28.627 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T15:00:28.627 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T15:00:28.627 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-09-21T15:00:28.627 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-09-21T15:00:28.627 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-09-21T15:00:28.627 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-09-21T15:00:28.627 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-09-21T15:00:28.627 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.627 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-09-21T15:00:28.627 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-09-21T15:00:28.627 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-09-21T15:00:28.627 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-09-21T15:00:28.627 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-09-21T15:00:28.627 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T15:00:28.627 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-09-21T15:00:28.628 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-09-21T15:00:28.628 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-09-21T15:00:28.628 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-09-21T15:00:28.628 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-09-21T15:00:28.628 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2024-09-21T15:00:28.628 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-09-21T15:00:28.628 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd": {} 2024-09-21T15:00:28.628 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.628 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-09-21T15:00:28.628 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 1.75, 2024-09-21T15:00:28.628 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 1.75, 2024-09-21T15:00:28.628 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 1, 2024-09-21T15:00:28.628 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 1.75, 2024-09-21T15:00:28.628 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 1.75, 2024-09-21T15:00:28.628 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 0.97936630249023438, 2024-09-21T15:00:28.628 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 0.98000001907348633, 2024-09-21T15:00:28.628 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 0.98000001907348633 2024-09-21T15:00:28.629 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:28.629 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:28.629 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-21T15:00:28.629 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2024-09-21T15:00:28.629 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:28.629 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-09-21T15:00:28.629 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "a4a73316-3a94-42e9-9a51-deb46a66bb06", 2024-09-21T15:00:28.629 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-21T15:00:28.629 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-21T15:00:28.629 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-21T15:00:28.629 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-21T15:00:28.629 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-21T15:00:28.629 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-21T15:00:28.629 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2024-09-21T15:00:28.629 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 93, 2024-09-21T15:00:28.629 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-21T15:00:28.630 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-21T15:00:28.630 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-21T15:00:28.630 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:28.630 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:28.630 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:28.630 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6808", 2024-09-21T15:00:28.630 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4016672243 2024-09-21T15:00:28.630 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.630 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:28.630 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:28.630 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6809", 2024-09-21T15:00:28.630 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4016672243 2024-09-21T15:00:28.630 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:28.631 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:28.631 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.631 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-21T15:00:28.631 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:28.631 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:28.631 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:28.631 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6810", 2024-09-21T15:00:28.631 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4016672243 2024-09-21T15:00:28.631 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.631 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:28.631 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:28.631 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6811", 2024-09-21T15:00:28.631 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4016672243 2024-09-21T15:00:28.631 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:28.631 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:28.631 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.632 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-21T15:00:28.632 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:28.632 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:28.632 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:28.632 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6814", 2024-09-21T15:00:28.632 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4016672243 2024-09-21T15:00:28.632 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.632 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:28.632 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:28.632 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6815", 2024-09-21T15:00:28.632 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4016672243 2024-09-21T15:00:28.632 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:28.632 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:28.632 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.632 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-21T15:00:28.632 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:28.633 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:28.633 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:28.633 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6812", 2024-09-21T15:00:28.633 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4016672243 2024-09-21T15:00:28.633 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.633 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:28.633 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:28.633 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6813", 2024-09-21T15:00:28.633 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4016672243 2024-09-21T15:00:28.633 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:28.633 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:28.633 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.633 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6809/4016672243", 2024-09-21T15:00:28.633 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6811/4016672243", 2024-09-21T15:00:28.633 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6815/4016672243", 2024-09-21T15:00:28.633 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6813/4016672243", 2024-09-21T15:00:28.633 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-21T15:00:28.634 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-21T15:00:28.634 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-21T15:00:28.634 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:28.634 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.634 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:28.634 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-09-21T15:00:28.634 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "006757a0-94d1-4a9b-aef8-73634cc5828e", 2024-09-21T15:00:28.634 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-21T15:00:28.634 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-21T15:00:28.634 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-21T15:00:28.634 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-21T15:00:28.634 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-21T15:00:28.634 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-21T15:00:28.634 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2024-09-21T15:00:28.635 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 140, 2024-09-21T15:00:28.635 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-21T15:00:28.635 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-21T15:00:28.635 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-21T15:00:28.635 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:28.635 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:28.635 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:28.635 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6824", 2024-09-21T15:00:28.635 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 641090577 2024-09-21T15:00:28.635 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.635 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:28.635 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:28.635 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6825", 2024-09-21T15:00:28.635 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 641090577 2024-09-21T15:00:28.635 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:28.635 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:28.636 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.636 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-21T15:00:28.636 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:28.636 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:28.636 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:28.636 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6826", 2024-09-21T15:00:28.636 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 641090577 2024-09-21T15:00:28.636 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.636 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:28.636 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:28.636 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6827", 2024-09-21T15:00:28.636 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 641090577 2024-09-21T15:00:28.636 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:28.636 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:28.636 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.636 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-21T15:00:28.636 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:28.637 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:28.637 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:28.637 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6830", 2024-09-21T15:00:28.637 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 641090577 2024-09-21T15:00:28.637 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.637 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:28.637 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:28.637 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6831", 2024-09-21T15:00:28.637 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 641090577 2024-09-21T15:00:28.637 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:28.637 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:28.637 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.637 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-21T15:00:28.637 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:28.637 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:28.637 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:28.638 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6828", 2024-09-21T15:00:28.638 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 641090577 2024-09-21T15:00:28.638 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.638 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:28.638 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:28.638 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6829", 2024-09-21T15:00:28.638 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 641090577 2024-09-21T15:00:28.638 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:28.638 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:28.638 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.638 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6825/641090577", 2024-09-21T15:00:28.638 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6827/641090577", 2024-09-21T15:00:28.638 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6831/641090577", 2024-09-21T15:00:28.638 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6829/641090577", 2024-09-21T15:00:28.638 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-21T15:00:28.639 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-21T15:00:28.639 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-21T15:00:28.639 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:28.639 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.639 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:28.639 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-09-21T15:00:28.639 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "0b499a23-820e-4e6c-a558-adbf88b7014f", 2024-09-21T15:00:28.639 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-21T15:00:28.639 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-21T15:00:28.639 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 0.3514556884765625, 2024-09-21T15:00:28.639 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-21T15:00:28.639 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-21T15:00:28.639 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-21T15:00:28.639 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2024-09-21T15:00:28.639 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 51, 2024-09-21T15:00:28.639 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-21T15:00:28.640 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-21T15:00:28.640 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-21T15:00:28.640 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:28.640 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:28.640 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:28.640 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6800", 2024-09-21T15:00:28.640 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2609154414 2024-09-21T15:00:28.640 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.640 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:28.640 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:28.640 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6801", 2024-09-21T15:00:28.640 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2609154414 2024-09-21T15:00:28.640 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:28.640 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:28.640 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.640 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-21T15:00:28.640 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:28.641 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:28.641 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:28.641 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6802", 2024-09-21T15:00:28.641 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2609154414 2024-09-21T15:00:28.641 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.641 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:28.641 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:28.641 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6803", 2024-09-21T15:00:28.641 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2609154414 2024-09-21T15:00:28.641 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:28.641 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:28.641 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.641 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-21T15:00:28.641 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:28.641 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:28.642 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:28.642 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6806", 2024-09-21T15:00:28.642 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2609154414 2024-09-21T15:00:28.642 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.642 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:28.642 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:28.642 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6807", 2024-09-21T15:00:28.642 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2609154414 2024-09-21T15:00:28.642 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:28.642 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:28.642 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.642 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-21T15:00:28.642 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:28.642 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:28.642 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:28.642 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6804", 2024-09-21T15:00:28.643 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2609154414 2024-09-21T15:00:28.643 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.643 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:28.643 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:28.643 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6805", 2024-09-21T15:00:28.643 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2609154414 2024-09-21T15:00:28.643 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:28.643 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:28.643 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.643 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6801/2609154414", 2024-09-21T15:00:28.643 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6803/2609154414", 2024-09-21T15:00:28.643 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6807/2609154414", 2024-09-21T15:00:28.643 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6805/2609154414", 2024-09-21T15:00:28.643 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-21T15:00:28.643 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-21T15:00:28.644 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-21T15:00:28.644 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:28.644 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.644 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:28.644 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-09-21T15:00:28.644 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "5a57b3dd-4a07-45b7-b159-3cac4c7eb262", 2024-09-21T15:00:28.644 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-21T15:00:28.644 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-21T15:00:28.644 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-21T15:00:28.644 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-21T15:00:28.644 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-21T15:00:28.644 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-21T15:00:28.644 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2024-09-21T15:00:28.644 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 94, 2024-09-21T15:00:28.645 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-21T15:00:28.645 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-21T15:00:28.645 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-21T15:00:28.645 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:28.645 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:28.645 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:28.645 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6816", 2024-09-21T15:00:28.645 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1462563376 2024-09-21T15:00:28.645 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.645 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:28.645 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:28.645 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6817", 2024-09-21T15:00:28.645 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1462563376 2024-09-21T15:00:28.645 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:28.645 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:28.645 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.646 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-21T15:00:28.646 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:28.646 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:28.646 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:28.646 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6818", 2024-09-21T15:00:28.646 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1462563376 2024-09-21T15:00:28.646 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.646 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:28.646 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:28.646 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6819", 2024-09-21T15:00:28.646 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1462563376 2024-09-21T15:00:28.646 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:28.646 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:28.646 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.646 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-21T15:00:28.646 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:28.647 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:28.647 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:28.647 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6822", 2024-09-21T15:00:28.647 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1462563376 2024-09-21T15:00:28.647 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.647 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:28.647 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:28.647 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6823", 2024-09-21T15:00:28.647 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1462563376 2024-09-21T15:00:28.647 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:28.647 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:28.647 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.647 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-21T15:00:28.647 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:28.647 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:28.648 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:28.648 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6820", 2024-09-21T15:00:28.648 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1462563376 2024-09-21T15:00:28.648 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.648 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:28.648 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:28.648 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6821", 2024-09-21T15:00:28.648 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1462563376 2024-09-21T15:00:28.648 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:28.648 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:28.648 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.648 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6817/1462563376", 2024-09-21T15:00:28.648 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6819/1462563376", 2024-09-21T15:00:28.648 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6823/1462563376", 2024-09-21T15:00:28.648 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6821/1462563376", 2024-09-21T15:00:28.649 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-21T15:00:28.649 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-21T15:00:28.649 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-21T15:00:28.649 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:28.649 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.649 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:28.649 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-09-21T15:00:28.649 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "39a00807-7517-474e-8842-623dd7093cc5", 2024-09-21T15:00:28.649 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-21T15:00:28.649 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-21T15:00:28.649 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-21T15:00:28.649 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-21T15:00:28.649 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-21T15:00:28.649 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-21T15:00:28.649 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2024-09-21T15:00:28.649 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 94, 2024-09-21T15:00:28.650 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-21T15:00:28.650 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-21T15:00:28.650 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-21T15:00:28.650 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:28.650 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:28.650 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:28.650 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6801", 2024-09-21T15:00:28.650 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 144412042 2024-09-21T15:00:28.650 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.650 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:28.650 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:28.650 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6803", 2024-09-21T15:00:28.650 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 144412042 2024-09-21T15:00:28.650 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:28.650 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:28.650 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.651 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-21T15:00:28.651 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:28.651 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:28.651 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:28.651 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6805", 2024-09-21T15:00:28.651 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 144412042 2024-09-21T15:00:28.651 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.651 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:28.651 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:28.651 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6807", 2024-09-21T15:00:28.651 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 144412042 2024-09-21T15:00:28.651 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:28.651 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:28.651 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.651 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-21T15:00:28.651 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:28.651 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:28.652 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:28.652 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6812", 2024-09-21T15:00:28.652 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 144412042 2024-09-21T15:00:28.652 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.652 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:28.652 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:28.652 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6814", 2024-09-21T15:00:28.652 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 144412042 2024-09-21T15:00:28.652 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:28.652 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:28.652 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.652 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-21T15:00:28.652 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:28.652 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:28.652 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:28.652 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6809", 2024-09-21T15:00:28.653 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 144412042 2024-09-21T15:00:28.653 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.653 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:28.653 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:28.653 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6811", 2024-09-21T15:00:28.653 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 144412042 2024-09-21T15:00:28.653 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:28.653 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:28.653 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.653 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.46:6803/144412042", 2024-09-21T15:00:28.653 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.46:6807/144412042", 2024-09-21T15:00:28.653 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.46:6814/144412042", 2024-09-21T15:00:28.653 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.46:6811/144412042", 2024-09-21T15:00:28.653 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-21T15:00:28.653 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-21T15:00:28.654 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-21T15:00:28.654 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:28.654 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.654 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:28.654 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-09-21T15:00:28.654 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "edf9f045-ddeb-4504-a0fc-1a0e4e853550", 2024-09-21T15:00:28.654 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-21T15:00:28.654 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-21T15:00:28.654 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-21T15:00:28.654 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-21T15:00:28.654 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-21T15:00:28.654 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-21T15:00:28.654 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2024-09-21T15:00:28.654 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 75, 2024-09-21T15:00:28.655 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-21T15:00:28.655 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-21T15:00:28.655 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-21T15:00:28.655 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:28.655 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:28.655 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:28.655 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6800", 2024-09-21T15:00:28.655 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 849101442 2024-09-21T15:00:28.655 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.655 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:28.655 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:28.655 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6802", 2024-09-21T15:00:28.655 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 849101442 2024-09-21T15:00:28.655 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:28.655 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:28.655 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.655 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-21T15:00:28.656 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:28.656 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:28.656 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:28.656 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6804", 2024-09-21T15:00:28.656 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 849101442 2024-09-21T15:00:28.656 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.656 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:28.656 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:28.656 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6806", 2024-09-21T15:00:28.656 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 849101442 2024-09-21T15:00:28.656 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:28.656 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:28.656 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.656 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-21T15:00:28.656 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:28.656 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:28.657 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:28.657 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6813", 2024-09-21T15:00:28.657 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 849101442 2024-09-21T15:00:28.657 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.657 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:28.657 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:28.657 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6815", 2024-09-21T15:00:28.657 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 849101442 2024-09-21T15:00:28.657 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:28.657 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:28.657 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.657 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-21T15:00:28.657 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:28.657 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:28.657 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:28.657 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6808", 2024-09-21T15:00:28.658 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 849101442 2024-09-21T15:00:28.658 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.658 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:28.658 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:28.658 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6810", 2024-09-21T15:00:28.658 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 849101442 2024-09-21T15:00:28.658 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:28.658 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:28.658 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.658 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.46:6802/849101442", 2024-09-21T15:00:28.658 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.46:6806/849101442", 2024-09-21T15:00:28.658 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.46:6815/849101442", 2024-09-21T15:00:28.658 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.46:6810/849101442", 2024-09-21T15:00:28.658 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-21T15:00:28.658 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-21T15:00:28.658 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-21T15:00:28.659 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:28.659 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.659 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:28.659 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-09-21T15:00:28.659 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "c820ea6b-88fe-4421-83ec-17c466691fea", 2024-09-21T15:00:28.659 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-21T15:00:28.659 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-21T15:00:28.659 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-21T15:00:28.659 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 0.834930419921875, 2024-09-21T15:00:28.659 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-21T15:00:28.659 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-21T15:00:28.659 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2024-09-21T15:00:28.659 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 123, 2024-09-21T15:00:28.659 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-21T15:00:28.659 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-21T15:00:28.659 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-21T15:00:28.660 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:28.660 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:28.660 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:28.660 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6817", 2024-09-21T15:00:28.660 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 847189172 2024-09-21T15:00:28.660 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.660 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:28.660 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:28.660 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6818", 2024-09-21T15:00:28.660 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 847189172 2024-09-21T15:00:28.660 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:28.660 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:28.660 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.660 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-21T15:00:28.661 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:28.661 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:28.661 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:28.661 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6820", 2024-09-21T15:00:28.661 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 847189172 2024-09-21T15:00:28.661 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.661 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:28.661 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:28.661 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6822", 2024-09-21T15:00:28.661 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 847189172 2024-09-21T15:00:28.661 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:28.661 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:28.661 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.662 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-21T15:00:28.662 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:28.662 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:28.662 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:28.662 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6827", 2024-09-21T15:00:28.662 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 847189172 2024-09-21T15:00:28.662 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.662 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:28.662 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:28.662 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6828", 2024-09-21T15:00:28.662 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 847189172 2024-09-21T15:00:28.662 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:28.662 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:28.662 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.662 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-21T15:00:28.663 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:28.663 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:28.663 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:28.663 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6823", 2024-09-21T15:00:28.663 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 847189172 2024-09-21T15:00:28.663 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.663 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:28.663 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:28.663 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6825", 2024-09-21T15:00:28.663 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 847189172 2024-09-21T15:00:28.663 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:28.663 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:28.663 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.663 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.46:6818/847189172", 2024-09-21T15:00:28.663 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.46:6822/847189172", 2024-09-21T15:00:28.664 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.46:6828/847189172", 2024-09-21T15:00:28.664 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.46:6825/847189172", 2024-09-21T15:00:28.664 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-21T15:00:28.664 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-21T15:00:28.664 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-21T15:00:28.664 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:28.664 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.664 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:28.664 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-09-21T15:00:28.664 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "3a2f82a4-28a0-4fcb-9d6b-2285fa4d07ac", 2024-09-21T15:00:28.664 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-21T15:00:28.664 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-21T15:00:28.664 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 0.95001220703125, 2024-09-21T15:00:28.665 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-21T15:00:28.665 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-21T15:00:28.665 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-21T15:00:28.665 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 15, 2024-09-21T15:00:28.665 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 81, 2024-09-21T15:00:28.665 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-21T15:00:28.665 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-21T15:00:28.665 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-21T15:00:28.665 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:28.665 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:28.665 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:28.665 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6816", 2024-09-21T15:00:28.665 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3894963553 2024-09-21T15:00:28.665 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.665 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:28.666 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:28.666 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6819", 2024-09-21T15:00:28.666 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3894963553 2024-09-21T15:00:28.666 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:28.666 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:28.666 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.666 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-21T15:00:28.666 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:28.666 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:28.666 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:28.666 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6821", 2024-09-21T15:00:28.666 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3894963553 2024-09-21T15:00:28.666 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.666 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:28.667 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:28.667 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6824", 2024-09-21T15:00:28.667 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3894963553 2024-09-21T15:00:28.667 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:28.667 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:28.667 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.667 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-21T15:00:28.667 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:28.667 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:28.667 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:28.667 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6830", 2024-09-21T15:00:28.667 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3894963553 2024-09-21T15:00:28.667 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.667 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:28.667 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:28.668 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6831", 2024-09-21T15:00:28.668 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3894963553 2024-09-21T15:00:28.668 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:28.668 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:28.668 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.668 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-21T15:00:28.668 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:28.668 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:28.668 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:28.668 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6826", 2024-09-21T15:00:28.668 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3894963553 2024-09-21T15:00:28.668 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.668 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:28.668 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:28.668 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6829", 2024-09-21T15:00:28.669 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3894963553 2024-09-21T15:00:28.669 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:28.669 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:28.669 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.669 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.46:6819/3894963553", 2024-09-21T15:00:28.669 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.46:6824/3894963553", 2024-09-21T15:00:28.669 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.46:6831/3894963553", 2024-09-21T15:00:28.669 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.46:6829/3894963553", 2024-09-21T15:00:28.669 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-21T15:00:28.669 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-21T15:00:28.669 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-21T15:00:28.669 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:28.669 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:28.669 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-21T15:00:28.669 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_xinfo": [ 2024-09-21T15:00:28.669 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:28.670 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-09-21T15:00:28.670 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-21T15:00:28.670 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-21T15:00:28.670 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-21T15:00:28.670 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-21T15:00:28.670 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-21T15:00:28.670 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-21T14:57:57.458317+0000", 2024-09-21T15:00:28.670 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-21T15:00:28.670 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.670 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:28.670 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-09-21T15:00:28.670 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-21T15:00:28.670 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-21T15:00:28.670 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-21T15:00:28.671 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-21T15:00:28.671 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-21T15:00:28.671 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-21T14:57:57.907624+0000", 2024-09-21T15:00:28.671 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-21T15:00:28.671 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.671 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:28.671 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-09-21T15:00:28.671 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-21T15:00:28.671 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-21T15:00:28.671 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-21T15:00:28.671 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-21T15:00:28.671 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-21T15:00:28.671 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-21T14:57:57.278817+0000", 2024-09-21T15:00:28.672 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-21T15:00:28.672 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.672 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:28.672 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-09-21T15:00:28.672 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-21T15:00:28.672 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-21T15:00:28.672 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-21T15:00:28.672 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-21T15:00:28.672 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-21T15:00:28.672 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-21T14:57:57.500449+0000", 2024-09-21T15:00:28.672 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-21T15:00:28.672 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.672 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:28.673 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-09-21T15:00:28.673 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-21T15:00:28.673 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-21T15:00:28.673 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-21T15:00:28.673 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-21T15:00:28.673 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-21T15:00:28.673 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-21T14:57:57.504325+0000", 2024-09-21T15:00:28.673 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-21T15:00:28.673 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.673 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:28.673 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-09-21T15:00:28.673 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-21T15:00:28.673 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-21T15:00:28.673 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-21T15:00:28.674 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-21T15:00:28.674 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-21T15:00:28.674 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-21T14:57:57.777525+0000", 2024-09-21T15:00:28.674 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-21T15:00:28.674 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.674 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:28.674 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-09-21T15:00:28.674 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-21T15:00:28.674 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-21T15:00:28.674 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-21T15:00:28.674 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-21T15:00:28.674 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-21T15:00:28.674 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-21T14:57:57.698961+0000", 2024-09-21T15:00:28.674 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-21T15:00:28.675 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.675 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:28.675 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-09-21T15:00:28.675 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-21T15:00:28.675 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-21T15:00:28.675 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-21T15:00:28.675 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-21T15:00:28.675 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-21T15:00:28.675 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-21T14:57:57.932625+0000", 2024-09-21T15:00:28.675 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-21T15:00:28.675 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:28.675 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-21T15:00:28.675 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap": [ 2024-09-21T15:00:28.675 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:28.675 INFO:teuthology.orchestra.run.smithi032.stdout: "pgid": "3.2", 2024-09-21T15:00:28.675 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2024-09-21T15:00:28.676 INFO:teuthology.orchestra.run.smithi032.stdout: 2, 2024-09-21T15:00:28.676 INFO:teuthology.orchestra.run.smithi032.stdout: 3 2024-09-21T15:00:28.676 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:28.676 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.676 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:28.676 INFO:teuthology.orchestra.run.smithi032.stdout: "pgid": "3.a", 2024-09-21T15:00:28.676 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2024-09-21T15:00:28.676 INFO:teuthology.orchestra.run.smithi032.stdout: 1, 2024-09-21T15:00:28.676 INFO:teuthology.orchestra.run.smithi032.stdout: 0 2024-09-21T15:00:28.676 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:28.676 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:28.676 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-21T15:00:28.676 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_items": [], 2024-09-21T15:00:28.676 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_primaries": [], 2024-09-21T15:00:28.676 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_temp": [], 2024-09-21T15:00:28.677 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_temp": [], 2024-09-21T15:00:28.677 INFO:teuthology.orchestra.run.smithi032.stdout: "blocklist": {}, 2024-09-21T15:00:28.677 INFO:teuthology.orchestra.run.smithi032.stdout: "range_blocklist": {}, 2024-09-21T15:00:28.677 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profiles": { 2024-09-21T15:00:28.677 INFO:teuthology.orchestra.run.smithi032.stdout: "default": { 2024-09-21T15:00:28.677 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2024-09-21T15:00:28.677 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2024-09-21T15:00:28.677 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "1", 2024-09-21T15:00:28.677 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2024-09-21T15:00:28.677 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van" 2024-09-21T15:00:28.677 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:28.677 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:28.677 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps_queue": [ 2024-09-21T15:00:28.677 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:28.677 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 3, 2024-09-21T15:00:28.677 INFO:teuthology.orchestra.run.smithi032.stdout: "snaps": [ 2024-09-21T15:00:28.678 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:28.678 INFO:teuthology.orchestra.run.smithi032.stdout: "begin": 86, 2024-09-21T15:00:28.678 INFO:teuthology.orchestra.run.smithi032.stdout: "length": 1 2024-09-21T15:00:28.678 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:28.678 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:28.678 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:28.678 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-21T15:00:28.678 INFO:teuthology.orchestra.run.smithi032.stdout: "new_removed_snaps": [], 2024-09-21T15:00:28.678 INFO:teuthology.orchestra.run.smithi032.stdout: "new_purged_snaps": [], 2024-09-21T15:00:28.678 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_node_flags": {}, 2024-09-21T15:00:28.678 INFO:teuthology.orchestra.run.smithi032.stdout: "device_class_flags": {}, 2024-09-21T15:00:28.678 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode": { 2024-09-21T15:00:28.678 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_enabled": false, 2024-09-21T15:00:28.678 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_bucket_count": 0, 2024-09-21T15:00:28.678 INFO:teuthology.orchestra.run.smithi032.stdout: "degraded_stretch_mode": 0, 2024-09-21T15:00:28.679 INFO:teuthology.orchestra.run.smithi032.stdout: "recovering_stretch_mode": 0, 2024-09-21T15:00:28.679 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_bucket": 0 2024-09-21T15:00:28.679 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:28.679 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:28.680 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:28.680 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:28.681 INFO:tasks.thrashosds.thrasher:j is {'epoch': 142, 'fsid': 'caf51e55-9234-4a26-bcd2-d60811962560', 'created': '2024-09-21T14:57:50.338822+0000', 'modified': '2024-09-21T15:00:28.279685+0000', 'last_up_change': '2024-09-21T14:58:01.177141+0000', 'last_in_change': '2024-09-21T14:59:27.207073+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T14:58:01.842810+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.9793663024902344, 'average_primary_affinity': 0.9800000190734863, 'average_primary_affinity_weighted': 0.9800000190734863}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-21T14:58:08.284925+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 22, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 0.9793663024902344, 'average_primary_affinity': 0.9800000190734863, 'average_primary_affinity_weighted': 0.9800000190734863}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-21T14:58:17.510709+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '142', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 88, 'snap_epoch': 142, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.9793663024902344, 'average_primary_affinity': 0.9800000190734863, 'average_primary_affinity_weighted': 0.9800000190734863}}], 'osds': [{'osd': 0, 'uuid': 'a4a73316-3a94-42e9-9a51-deb46a66bb06', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 93, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6808', 'nonce': 4016672243}, {'type': 'v1', 'addr': '172.21.15.32:6809', 'nonce': 4016672243}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6810', 'nonce': 4016672243}, {'type': 'v1', 'addr': '172.21.15.32:6811', 'nonce': 4016672243}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6814', 'nonce': 4016672243}, {'type': 'v1', 'addr': '172.21.15.32:6815', 'nonce': 4016672243}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6812', 'nonce': 4016672243}, {'type': 'v1', 'addr': '172.21.15.32:6813', 'nonce': 4016672243}]}, 'public_addr': '172.21.15.32:6809/4016672243', 'cluster_addr': '172.21.15.32:6811/4016672243', 'heartbeat_back_addr': '172.21.15.32:6815/4016672243', 'heartbeat_front_addr': '172.21.15.32:6813/4016672243', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '006757a0-94d1-4a9b-aef8-73634cc5828e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 140, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6824', 'nonce': 641090577}, {'type': 'v1', 'addr': '172.21.15.32:6825', 'nonce': 641090577}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6826', 'nonce': 641090577}, {'type': 'v1', 'addr': '172.21.15.32:6827', 'nonce': 641090577}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6830', 'nonce': 641090577}, {'type': 'v1', 'addr': '172.21.15.32:6831', 'nonce': 641090577}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6828', 'nonce': 641090577}, {'type': 'v1', 'addr': '172.21.15.32:6829', 'nonce': 641090577}]}, 'public_addr': '172.21.15.32:6825/641090577', 'cluster_addr': '172.21.15.32:6827/641090577', 'heartbeat_back_addr': '172.21.15.32:6831/641090577', 'heartbeat_front_addr': '172.21.15.32:6829/641090577', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '0b499a23-820e-4e6c-a558-adbf88b7014f', 'up': 1, 'in': 1, 'weight': 0.3514556884765625, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 51, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6800', 'nonce': 2609154414}, {'type': 'v1', 'addr': '172.21.15.32:6801', 'nonce': 2609154414}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6802', 'nonce': 2609154414}, {'type': 'v1', 'addr': '172.21.15.32:6803', 'nonce': 2609154414}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6806', 'nonce': 2609154414}, {'type': 'v1', 'addr': '172.21.15.32:6807', 'nonce': 2609154414}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6804', 'nonce': 2609154414}, {'type': 'v1', 'addr': '172.21.15.32:6805', 'nonce': 2609154414}]}, 'public_addr': '172.21.15.32:6801/2609154414', 'cluster_addr': '172.21.15.32:6803/2609154414', 'heartbeat_back_addr': '172.21.15.32:6807/2609154414', 'heartbeat_front_addr': '172.21.15.32:6805/2609154414', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '5a57b3dd-4a07-45b7-b159-3cac4c7eb262', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 94, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6816', 'nonce': 1462563376}, {'type': 'v1', 'addr': '172.21.15.32:6817', 'nonce': 1462563376}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6818', 'nonce': 1462563376}, {'type': 'v1', 'addr': '172.21.15.32:6819', 'nonce': 1462563376}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6822', 'nonce': 1462563376}, {'type': 'v1', 'addr': '172.21.15.32:6823', 'nonce': 1462563376}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6820', 'nonce': 1462563376}, {'type': 'v1', 'addr': '172.21.15.32:6821', 'nonce': 1462563376}]}, 'public_addr': '172.21.15.32:6817/1462563376', 'cluster_addr': '172.21.15.32:6819/1462563376', 'heartbeat_back_addr': '172.21.15.32:6823/1462563376', 'heartbeat_front_addr': '172.21.15.32:6821/1462563376', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '39a00807-7517-474e-8842-623dd7093cc5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 94, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6801', 'nonce': 144412042}, {'type': 'v1', 'addr': '172.21.15.46:6803', 'nonce': 144412042}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6805', 'nonce': 144412042}, {'type': 'v1', 'addr': '172.21.15.46:6807', 'nonce': 144412042}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6812', 'nonce': 144412042}, {'type': 'v1', 'addr': '172.21.15.46:6814', 'nonce': 144412042}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6809', 'nonce': 144412042}, {'type': 'v1', 'addr': '172.21.15.46:6811', 'nonce': 144412042}]}, 'public_addr': '172.21.15.46:6803/144412042', 'cluster_addr': '172.21.15.46:6807/144412042', 'heartbeat_back_addr': '172.21.15.46:6814/144412042', 'heartbeat_front_addr': '172.21.15.46:6811/144412042', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'edf9f045-ddeb-4504-a0fc-1a0e4e853550', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 75, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6800', 'nonce': 849101442}, {'type': 'v1', 'addr': '172.21.15.46:6802', 'nonce': 849101442}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6804', 'nonce': 849101442}, {'type': 'v1', 'addr': '172.21.15.46:6806', 'nonce': 849101442}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6813', 'nonce': 849101442}, {'type': 'v1', 'addr': '172.21.15.46:6815', 'nonce': 849101442}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6808', 'nonce': 849101442}, {'type': 'v1', 'addr': '172.21.15.46:6810', 'nonce': 849101442}]}, 'public_addr': '172.21.15.46:6802/849101442', 'cluster_addr': '172.21.15.46:6806/849101442', 'heartbeat_back_addr': '172.21.15.46:6815/849101442', 'heartbeat_front_addr': '172.21.15.46:6810/849101442', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'c820ea6b-88fe-4421-83ec-17c466691fea', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.834930419921875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 123, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6817', 'nonce': 847189172}, {'type': 'v1', 'addr': '172.21.15.46:6818', 'nonce': 847189172}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6820', 'nonce': 847189172}, {'type': 'v1', 'addr': '172.21.15.46:6822', 'nonce': 847189172}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6827', 'nonce': 847189172}, {'type': 'v1', 'addr': '172.21.15.46:6828', 'nonce': 847189172}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6823', 'nonce': 847189172}, {'type': 'v1', 'addr': '172.21.15.46:6825', 'nonce': 847189172}]}, 'public_addr': '172.21.15.46:6818/847189172', 'cluster_addr': '172.21.15.46:6822/847189172', 'heartbeat_back_addr': '172.21.15.46:6828/847189172', 'heartbeat_front_addr': '172.21.15.46:6825/847189172', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '3a2f82a4-28a0-4fcb-9d6b-2285fa4d07ac', 'up': 1, 'in': 1, 'weight': 0.95001220703125, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 81, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6816', 'nonce': 3894963553}, {'type': 'v1', 'addr': '172.21.15.46:6819', 'nonce': 3894963553}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6821', 'nonce': 3894963553}, {'type': 'v1', 'addr': '172.21.15.46:6824', 'nonce': 3894963553}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6830', 'nonce': 3894963553}, {'type': 'v1', 'addr': '172.21.15.46:6831', 'nonce': 3894963553}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6826', 'nonce': 3894963553}, {'type': 'v1', 'addr': '172.21.15.46:6829', 'nonce': 3894963553}]}, 'public_addr': '172.21.15.46:6819/3894963553', 'cluster_addr': '172.21.15.46:6824/3894963553', 'heartbeat_back_addr': '172.21.15.46:6831/3894963553', 'heartbeat_front_addr': '172.21.15.46:6829/3894963553', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:57:57.458317+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:57:57.907624+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:57:57.278817+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:57:57.500449+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:57:57.504325+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:57:57.777525+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:57:57.698961+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:57:57.932625+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.2', 'osds': [2, 3]}, {'pgid': '3.a', 'osds': [1, 0]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 86, 'length': 1}]}], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-21T15:00:28.681 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T15:00:28.682 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T15:00:28.682 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T15:00:28.682 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T15:00:28.682 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:00:28.675+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:00:28.686 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-21T15:00:28.708 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T15:00:28.730 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T15:00:28.753 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T15:00:28.755 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:00:28.755 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:00:28.755 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:00:28.755 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:00:28.755 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:00:28.756 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:00:28.756 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:00:28.756 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:00:28.797 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:28.797 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:00:28.797 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T15:00:28.797 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:28.807 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T15:00:28.817 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:00:28.815+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:00:28.881 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:00:28.879+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:00:28.911 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:28.911 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:00:28.911 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T15:00:28.911 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T15:00:28.911 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:28.921 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T15:00:28.982 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:00:28.978+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:00:28.996 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-21T15:00:29.001 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-09-21T15:00:29.014 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":209,"stamp":"2024-09-21T15:00:28.288073+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":849812995,"num_objects":1056,"num_object_clones":0,"num_object_copies":2112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1056,"num_whiteouts":0,"num_read":110375,"num_read_kb":936549,"num_write":19657,"num_write_kb":703528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1306,"num_bytes_recovered":89636335,"num_keys_recovered":152,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12947,"ondisk_log_size":12947,"up":82,"acting":81,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":380096,"kb_used_data":64696,"kb_used_omap":12,"kb_used_meta":315251,"kb_avail":754594624,"statfs":{"total":773094113280,"available":772704894976,"internally_reserved":0,"allocated":66248704,"data_stored":477329338,"data_compressed":10644357,"data_compressed_allocated":42278912,"data_compressed_original":464985101,"omap_allocated":12744,"internal_metadata":322817592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":71613266,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":658,"num_read_kb":2189,"num_write":158,"num_write_kb":2196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":9228123,"num_keys_recovered":33,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.029496"},"pg_stats":[{"pgid":"3.1f","version":"139'367","reported_seq":3134,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-21T15:00:25.259711+0000","last_change":"2024-09-21T15:00:20.144173+0000","last_active":"2024-09-21T15:00:25.259711+0000","last_peered":"2024-09-21T15:00:25.259711+0000","last_clean":"2024-09-21T15:00:25.259711+0000","last_became_active":"2024-09-21T14:58:19.410055+0000","last_became_peered":"2024-09-21T14:58:19.410055+0000","last_unstale":"2024-09-21T15:00:25.259711+0000","last_undegraded":"2024-09-21T15:00:25.259711+0000","last_fullsized":"2024-09-21T15:00:25.259711+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":367,"log_dups_size":0,"ondisk_log_size":367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:49:13.944219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001324857,"stat_sum":{"num_bytes":31189406,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":3115,"num_read_kb":40656,"num_write":590,"num_write_kb":31894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"135'253","reported_seq":2192,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-21T15:00:25.144657+0000","last_change":"2024-09-21T15:00:20.143921+0000","last_active":"2024-09-21T15:00:25.144657+0000","last_peered":"2024-09-21T15:00:25.144657+0000","last_clean":"2024-09-21T15:00:25.144657+0000","last_became_active":"2024-09-21T14:59:13.130091+0000","last_became_peered":"2024-09-21T14:59:13.130091+0000","last_unstale":"2024-09-21T15:00:25.144657+0000","last_undegraded":"2024-09-21T15:00:25.144657+0000","last_fullsized":"2024-09-21T15:00:25.144657+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":253,"log_dups_size":0,"ondisk_log_size":253,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:46:00.199136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001780083,"stat_sum":{"num_bytes":17990538,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":2111,"num_read_kb":19717,"num_write":343,"num_write_kb":13067,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":3844512,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"135'172","reported_seq":3092,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-21T15:00:25.177536+0000","last_change":"2024-09-21T15:00:20.150013+0000","last_active":"2024-09-21T15:00:25.177536+0000","last_peered":"2024-09-21T15:00:25.177536+0000","last_clean":"2024-09-21T15:00:25.177536+0000","last_became_active":"2024-09-21T14:58:19.411387+0000","last_became_peered":"2024-09-21T14:58:19.411387+0000","last_unstale":"2024-09-21T15:00:25.177536+0000","last_undegraded":"2024-09-21T15:00:25.177536+0000","last_fullsized":"2024-09-21T15:00:25.177536+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":172,"log_dups_size":0,"ondisk_log_size":172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:37:57.597069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017624380000000001,"stat_sum":{"num_bytes":13166729,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":2666,"num_read_kb":19990,"num_write":285,"num_write_kb":19625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"136'356","reported_seq":2455,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-21T15:00:25.148922+0000","last_change":"2024-09-21T15:00:20.153736+0000","last_active":"2024-09-21T15:00:25.148922+0000","last_peered":"2024-09-21T15:00:25.148922+0000","last_clean":"2024-09-21T15:00:25.148922+0000","last_became_active":"2024-09-21T14:58:19.411306+0000","last_became_peered":"2024-09-21T14:58:19.411306+0000","last_unstale":"2024-09-21T15:00:25.148922+0000","last_undegraded":"2024-09-21T15:00:25.148922+0000","last_fullsized":"2024-09-21T15:00:25.148922+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":356,"log_dups_size":0,"ondisk_log_size":356,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:20:27.079795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041969000000000002,"stat_sum":{"num_bytes":12624495,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":2532,"num_read_kb":15098,"num_write":524,"num_write_kb":18328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"135'342","reported_seq":2057,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-21T15:00:25.170219+0000","last_change":"2024-09-21T15:00:20.144009+0000","last_active":"2024-09-21T15:00:25.170219+0000","last_peered":"2024-09-21T15:00:25.170219+0000","last_clean":"2024-09-21T15:00:25.170219+0000","last_became_active":"2024-09-21T14:59:31.131051+0000","last_became_peered":"2024-09-21T14:59:31.131051+0000","last_unstale":"2024-09-21T15:00:25.170219+0000","last_undegraded":"2024-09-21T15:00:25.170219+0000","last_fullsized":"2024-09-21T15:00:25.170219+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:21:37.320774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013927379999999999,"stat_sum":{"num_bytes":17949492,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":2033,"num_read_kb":23609,"num_write":426,"num_write_kb":13686,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":117,"num_bytes_recovered":7757116,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"135'279","reported_seq":2671,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-21T15:00:25.235143+0000","last_change":"2024-09-21T15:00:20.153770+0000","last_active":"2024-09-21T15:00:25.235143+0000","last_peered":"2024-09-21T15:00:25.235143+0000","last_clean":"2024-09-21T15:00:25.235143+0000","last_became_active":"2024-09-21T14:58:19.409415+0000","last_became_peered":"2024-09-21T14:58:19.409415+0000","last_unstale":"2024-09-21T15:00:25.235143+0000","last_undegraded":"2024-09-21T15:00:25.235143+0000","last_fullsized":"2024-09-21T15:00:25.235143+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":279,"log_dups_size":0,"ondisk_log_size":279,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:28:46.336301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046801999999999998,"stat_sum":{"num_bytes":18340703,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":2600,"num_read_kb":23339,"num_write":386,"num_write_kb":17018,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"135'340","reported_seq":3139,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-21T15:00:25.175501+0000","last_change":"2024-09-21T15:00:20.143295+0000","last_active":"2024-09-21T15:00:25.175501+0000","last_peered":"2024-09-21T15:00:25.175501+0000","last_clean":"2024-09-21T15:00:25.175501+0000","last_became_active":"2024-09-21T14:59:32.130624+0000","last_became_peered":"2024-09-21T14:59:32.130624+0000","last_unstale":"2024-09-21T15:00:25.175501+0000","last_undegraded":"2024-09-21T15:00:25.175501+0000","last_fullsized":"2024-09-21T15:00:25.175501+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":340,"log_dups_size":0,"ondisk_log_size":340,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:37:53.312733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013741269999999999,"stat_sum":{"num_bytes":34682398,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":3408,"num_read_kb":45932,"num_write":496,"num_write_kb":21980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":128,"num_bytes_recovered":11896992,"num_keys_recovered":44,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"132'471","reported_seq":5787,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-21T15:00:25.143779+0000","last_change":"2024-09-21T15:00:20.143521+0000","last_active":"2024-09-21T15:00:25.143779+0000","last_peered":"2024-09-21T15:00:25.143779+0000","last_clean":"2024-09-21T15:00:25.143779+0000","last_became_active":"2024-09-21T14:58:19.409371+0000","last_became_peered":"2024-09-21T14:58:19.409371+0000","last_unstale":"2024-09-21T15:00:25.143779+0000","last_undegraded":"2024-09-21T15:00:25.143779+0000","last_fullsized":"2024-09-21T15:00:25.143779+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":471,"log_dups_size":0,"ondisk_log_size":471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:35:27.143445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017989530000000001,"stat_sum":{"num_bytes":20804570,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":6532,"num_read_kb":28786,"num_write":696,"num_write_kb":24123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"135'199","reported_seq":3103,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-21T15:00:25.144448+0000","last_change":"2024-09-21T15:00:20.144509+0000","last_active":"2024-09-21T15:00:25.144448+0000","last_peered":"2024-09-21T15:00:25.144448+0000","last_clean":"2024-09-21T15:00:25.144448+0000","last_became_active":"2024-09-21T14:58:19.410306+0000","last_became_peered":"2024-09-21T14:58:19.410306+0000","last_unstale":"2024-09-21T15:00:25.144448+0000","last_undegraded":"2024-09-21T15:00:25.144448+0000","last_fullsized":"2024-09-21T15:00:25.144448+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:33:50.089165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001222175,"stat_sum":{"num_bytes":21853084,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":2759,"num_read_kb":24304,"num_write":350,"num_write_kb":24363,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"139'584","reported_seq":3700,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-21T15:00:25.188268+0000","last_change":"2024-09-21T15:00:20.151671+0000","last_active":"2024-09-21T15:00:25.188268+0000","last_peered":"2024-09-21T15:00:25.188268+0000","last_clean":"2024-09-21T15:00:25.188268+0000","last_became_active":"2024-09-21T14:59:32.132898+0000","last_became_peered":"2024-09-21T14:59:32.132898+0000","last_unstale":"2024-09-21T15:00:25.188268+0000","last_undegraded":"2024-09-21T15:00:25.188268+0000","last_fullsized":"2024-09-21T15:00:25.188268+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":584,"log_dups_size":0,"ondisk_log_size":584,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:09:47.566958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028082749999999998,"stat_sum":{"num_bytes":26467064,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":4936,"num_read_kb":33148,"num_write":860,"num_write_kb":19889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":153,"num_bytes_recovered":9621881,"num_keys_recovered":33,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"139'653","reported_seq":3000,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-21T15:00:25.262533+0000","last_change":"2024-09-21T15:00:20.238573+0000","last_active":"2024-09-21T15:00:25.262533+0000","last_peered":"2024-09-21T15:00:25.262533+0000","last_clean":"2024-09-21T15:00:25.262533+0000","last_became_active":"2024-09-21T14:58:19.410687+0000","last_became_peered":"2024-09-21T14:58:19.410687+0000","last_unstale":"2024-09-21T15:00:25.262533+0000","last_undegraded":"2024-09-21T15:00:25.262533+0000","last_fullsized":"2024-09-21T15:00:25.262533+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":653,"log_dups_size":0,"ondisk_log_size":653,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:21:42.226982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044268999999999998,"stat_sum":{"num_bytes":33396606,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":4210,"num_read_kb":43936,"num_write":1115,"num_write_kb":29936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"135'314","reported_seq":2088,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-21T15:00:25.154180+0000","last_change":"2024-09-21T15:00:20.150086+0000","last_active":"2024-09-21T15:00:25.154180+0000","last_peered":"2024-09-21T15:00:25.154180+0000","last_clean":"2024-09-21T15:00:25.154180+0000","last_became_active":"2024-09-21T14:58:19.407310+0000","last_became_peered":"2024-09-21T14:58:19.407310+0000","last_unstale":"2024-09-21T15:00:25.154180+0000","last_undegraded":"2024-09-21T15:00:25.154180+0000","last_fullsized":"2024-09-21T15:00:25.154180+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":314,"log_dups_size":0,"ondisk_log_size":314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:47:46.079115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00171222,"stat_sum":{"num_bytes":27524176,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":1971,"num_read_kb":29582,"num_write":401,"num_write_kb":17801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":133,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-21T15:00:24.266951+0000","last_change":"2024-09-21T14:58:11.348177+0000","last_active":"2024-09-21T15:00:24.266951+0000","last_peered":"2024-09-21T15:00:24.266951+0000","last_clean":"2024-09-21T15:00:24.266951+0000","last_became_active":"2024-09-21T14:58:09.328417+0000","last_became_peered":"2024-09-21T14:58:09.328417+0000","last_unstale":"2024-09-21T15:00:24.266951+0000","last_undegraded":"2024-09-21T15:00:24.266951+0000","last_fullsized":"2024-09-21T15:00:24.266951+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:52:22.671185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018917490000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"139'307","reported_seq":3196,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-21T15:00:25.171127+0000","last_change":"2024-09-21T15:00:20.144330+0000","last_active":"2024-09-21T15:00:25.171127+0000","last_peered":"2024-09-21T15:00:25.171127+0000","last_clean":"2024-09-21T15:00:25.171127+0000","last_became_active":"2024-09-21T14:59:32.131232+0000","last_became_peered":"2024-09-21T14:59:32.131232+0000","last_unstale":"2024-09-21T15:00:25.171127+0000","last_undegraded":"2024-09-21T15:00:25.171127+0000","last_fullsized":"2024-09-21T15:00:25.171127+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":307,"log_dups_size":0,"ondisk_log_size":307,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:31:04.421782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001702568,"stat_sum":{"num_bytes":46283728,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":3121,"num_read_kb":33635,"num_write":415,"num_write_kb":21012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":3585713,"num_keys_recovered":9,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":58,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-21T15:00:24.260929+0000","last_change":"2024-09-21T14:59:28.228883+0000","last_active":"2024-09-21T15:00:24.260929+0000","last_peered":"2024-09-21T15:00:24.260929+0000","last_clean":"2024-09-21T15:00:24.260929+0000","last_became_active":"2024-09-21T14:59:28.228464+0000","last_became_peered":"2024-09-21T14:59:28.228464+0000","last_unstale":"2024-09-21T15:00:24.260929+0000","last_undegraded":"2024-09-21T15:00:24.260929+0000","last_fullsized":"2024-09-21T15:00:24.260929+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:00:26.310864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"136'724","reported_seq":2982,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-21T15:00:25.186468+0000","last_change":"2024-09-21T15:00:20.144434+0000","last_active":"2024-09-21T15:00:25.186468+0000","last_peered":"2024-09-21T15:00:25.186468+0000","last_clean":"2024-09-21T15:00:25.186468+0000","last_became_active":"2024-09-21T14:59:31.132581+0000","last_became_peered":"2024-09-21T14:59:31.132581+0000","last_unstale":"2024-09-21T15:00:25.186468+0000","last_undegraded":"2024-09-21T15:00:25.186468+0000","last_fullsized":"2024-09-21T15:00:25.186468+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":724,"log_dups_size":0,"ondisk_log_size":724,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:36:45.467515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086195100000000001,"stat_sum":{"num_bytes":26157686,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":4627,"num_read_kb":28390,"num_write":1072,"num_write_kb":21364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":119,"num_bytes_recovered":6871924,"num_keys_recovered":11,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":149,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-21T15:00:24.266609+0000","last_change":"2024-09-21T14:59:28.231502+0000","last_active":"2024-09-21T15:00:24.266609+0000","last_peered":"2024-09-21T15:00:24.266609+0000","last_clean":"2024-09-21T15:00:24.266609+0000","last_became_active":"2024-09-21T14:59:28.231043+0000","last_became_peered":"2024-09-21T14:59:28.231043+0000","last_unstale":"2024-09-21T15:00:24.266609+0000","last_undegraded":"2024-09-21T15:00:24.266609+0000","last_fullsized":"2024-09-21T15:00:24.266609+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:09:46.773690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019338770000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"139'286","reported_seq":4160,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-21T15:00:25.263854+0000","last_change":"2024-09-21T15:00:20.155491+0000","last_active":"2024-09-21T15:00:25.263854+0000","last_peered":"2024-09-21T15:00:25.263854+0000","last_clean":"2024-09-21T15:00:25.263854+0000","last_became_active":"2024-09-21T15:00:07.159035+0000","last_became_peered":"2024-09-21T15:00:07.159035+0000","last_unstale":"2024-09-21T15:00:25.263854+0000","last_undegraded":"2024-09-21T15:00:25.263854+0000","last_fullsized":"2024-09-21T15:00:25.263854+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":286,"log_dups_size":0,"ondisk_log_size":286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:09:15.716689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055134100000000001,"stat_sum":{"num_bytes":32963144,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":4133,"num_read_kb":36107,"num_write":529,"num_write_kb":34522,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":134,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-21T15:00:24.260353+0000","last_change":"2024-09-21T14:58:11.406176+0000","last_active":"2024-09-21T15:00:24.260353+0000","last_peered":"2024-09-21T15:00:24.260353+0000","last_clean":"2024-09-21T15:00:24.260353+0000","last_became_active":"2024-09-21T14:58:09.777465+0000","last_became_peered":"2024-09-21T14:58:09.777465+0000","last_unstale":"2024-09-21T15:00:24.260353+0000","last_undegraded":"2024-09-21T15:00:24.260353+0000","last_fullsized":"2024-09-21T15:00:24.260353+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:55:04.667816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095704000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"133'597","reported_seq":2225,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-21T15:00:25.184044+0000","last_change":"2024-09-21T15:00:20.144167+0000","last_active":"2024-09-21T15:00:25.184044+0000","last_peered":"2024-09-21T15:00:25.184044+0000","last_clean":"2024-09-21T15:00:25.184044+0000","last_became_active":"2024-09-21T14:58:19.408971+0000","last_became_peered":"2024-09-21T14:58:19.408971+0000","last_unstale":"2024-09-21T15:00:25.184044+0000","last_undegraded":"2024-09-21T15:00:25.184044+0000","last_fullsized":"2024-09-21T15:00:25.184044+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":597,"log_dups_size":0,"ondisk_log_size":597,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:29:52.642121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044485599999999998,"stat_sum":{"num_bytes":12010717,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":3390,"num_read_kb":15517,"num_write":993,"num_write_kb":14111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"135'383","reported_seq":1962,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-21T15:00:25.145487+0000","last_change":"2024-09-21T15:00:20.239500+0000","last_active":"2024-09-21T15:00:25.145487+0000","last_peered":"2024-09-21T15:00:25.145487+0000","last_clean":"2024-09-21T15:00:25.145487+0000","last_became_active":"2024-09-21T14:58:19.417453+0000","last_became_peered":"2024-09-21T14:58:19.417453+0000","last_unstale":"2024-09-21T15:00:25.145487+0000","last_undegraded":"2024-09-21T15:00:25.145487+0000","last_fullsized":"2024-09-21T15:00:25.145487+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:19:27.087272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037973999999999999,"stat_sum":{"num_bytes":13295449,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":2303,"num_read_kb":19181,"num_write":706,"num_write_kb":15409,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":135,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-21T15:00:24.266710+0000","last_change":"2024-09-21T14:58:11.347504+0000","last_active":"2024-09-21T15:00:24.266710+0000","last_peered":"2024-09-21T15:00:24.266710+0000","last_clean":"2024-09-21T15:00:24.266710+0000","last_became_active":"2024-09-21T14:58:09.775981+0000","last_became_peered":"2024-09-21T14:58:09.775981+0000","last_unstale":"2024-09-21T15:00:24.266710+0000","last_undegraded":"2024-09-21T15:00:24.266710+0000","last_fullsized":"2024-09-21T15:00:24.266710+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:47:16.286652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012568880000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"139'402","reported_seq":2146,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-21T15:00:25.196861+0000","last_change":"2024-09-21T15:00:20.143227+0000","last_active":"2024-09-21T15:00:25.196861+0000","last_peered":"2024-09-21T15:00:25.196861+0000","last_clean":"2024-09-21T15:00:25.196861+0000","last_became_active":"2024-09-21T14:58:19.405811+0000","last_became_peered":"2024-09-21T14:58:19.405811+0000","last_unstale":"2024-09-21T15:00:25.196861+0000","last_undegraded":"2024-09-21T15:00:25.196861+0000","last_fullsized":"2024-09-21T15:00:25.196861+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":402,"log_dups_size":0,"ondisk_log_size":402,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:36:25.551607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001803624,"stat_sum":{"num_bytes":18300593,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":2638,"num_read_kb":22159,"num_write":540,"num_write_kb":22868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":135,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-21T15:00:25.181577+0000","last_change":"2024-09-21T14:58:11.342210+0000","last_active":"2024-09-21T15:00:25.181577+0000","last_peered":"2024-09-21T15:00:25.181577+0000","last_clean":"2024-09-21T15:00:25.181577+0000","last_became_active":"2024-09-21T14:58:09.330624+0000","last_became_peered":"2024-09-21T14:58:09.330624+0000","last_unstale":"2024-09-21T15:00:25.181577+0000","last_undegraded":"2024-09-21T15:00:25.181577+0000","last_fullsized":"2024-09-21T15:00:25.181577+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:29:43.229538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001051151,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"135'220","reported_seq":1000,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-21T15:00:25.175108+0000","last_change":"2024-09-21T15:00:20.155829+0000","last_active":"2024-09-21T15:00:25.175108+0000","last_peered":"2024-09-21T15:00:25.175108+0000","last_clean":"2024-09-21T15:00:25.175108+0000","last_became_active":"2024-09-21T15:00:07.158207+0000","last_became_peered":"2024-09-21T15:00:07.158207+0000","last_unstale":"2024-09-21T15:00:25.175108+0000","last_undegraded":"2024-09-21T15:00:25.175108+0000","last_fullsized":"2024-09-21T15:00:25.175108+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":220,"log_dups_size":0,"ondisk_log_size":220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:36:40.720592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061053099999999999,"stat_sum":{"num_bytes":29238822,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":1719,"num_read_kb":30244,"num_write":330,"num_write_kb":28231,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":150,"num_bytes_recovered":8136552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":133,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-21T15:00:24.266051+0000","last_change":"2024-09-21T14:58:11.346766+0000","last_active":"2024-09-21T15:00:24.266051+0000","last_peered":"2024-09-21T15:00:24.266051+0000","last_clean":"2024-09-21T15:00:24.266051+0000","last_became_active":"2024-09-21T14:58:09.329748+0000","last_became_peered":"2024-09-21T14:58:09.329748+0000","last_unstale":"2024-09-21T15:00:24.266051+0000","last_undegraded":"2024-09-21T15:00:24.266051+0000","last_fullsized":"2024-09-21T15:00:24.266051+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:31:20.267675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052698000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"135'348","reported_seq":5194,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-21T15:00:25.184377+0000","last_change":"2024-09-21T15:00:20.144095+0000","last_active":"2024-09-21T15:00:25.184377+0000","last_peered":"2024-09-21T15:00:25.184377+0000","last_clean":"2024-09-21T15:00:25.184377+0000","last_became_active":"2024-09-21T14:58:41.401003+0000","last_became_peered":"2024-09-21T14:58:41.401003+0000","last_unstale":"2024-09-21T15:00:25.184377+0000","last_undegraded":"2024-09-21T15:00:25.184377+0000","last_fullsized":"2024-09-21T15:00:25.184377+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:05:37.644835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043765600000000003,"stat_sum":{"num_bytes":40989545,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":5925,"num_read_kb":40952,"num_write":393,"num_write_kb":16150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1403348,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":134,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-21T15:00:24.266603+0000","last_change":"2024-09-21T14:58:11.347700+0000","last_active":"2024-09-21T15:00:24.266603+0000","last_peered":"2024-09-21T15:00:24.266603+0000","last_clean":"2024-09-21T15:00:24.266603+0000","last_became_active":"2024-09-21T14:58:09.438842+0000","last_became_peered":"2024-09-21T14:58:09.438842+0000","last_unstale":"2024-09-21T15:00:24.266603+0000","last_undegraded":"2024-09-21T15:00:24.266603+0000","last_fullsized":"2024-09-21T15:00:24.266603+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:41:34.626249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001426194,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":282,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-21T15:00:24.262359+0000","last_change":"2024-09-21T14:58:03.616055+0000","last_active":"2024-09-21T15:00:24.262359+0000","last_peered":"2024-09-21T15:00:24.262359+0000","last_clean":"2024-09-21T15:00:24.262359+0000","last_became_active":"2024-09-21T14:58:03.613214+0000","last_became_peered":"2024-09-21T14:58:03.613214+0000","last_unstale":"2024-09-21T15:00:24.262359+0000","last_undegraded":"2024-09-21T15:00:24.262359+0000","last_fullsized":"2024-09-21T15:00:24.262359+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:02.180162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:02.180162+0000","last_clean_scrub_stamp":"2024-09-21T14:58:02.180162+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:29:05.860361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"136'526","reported_seq":3479,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-21T15:00:25.227211+0000","last_change":"2024-09-21T15:00:20.143225+0000","last_active":"2024-09-21T15:00:25.227211+0000","last_peered":"2024-09-21T15:00:25.227211+0000","last_clean":"2024-09-21T15:00:25.227211+0000","last_became_active":"2024-09-21T14:59:31.131460+0000","last_became_peered":"2024-09-21T14:59:31.131460+0000","last_unstale":"2024-09-21T15:00:25.227211+0000","last_undegraded":"2024-09-21T15:00:25.227211+0000","last_fullsized":"2024-09-21T15:00:25.227211+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":526,"log_dups_size":0,"ondisk_log_size":526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:01:38.353476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001651203,"stat_sum":{"num_bytes":31944196,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":3946,"num_read_kb":38115,"num_write":865,"num_write_kb":25316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":119,"num_bytes_recovered":6097618,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"135'646","reported_seq":2203,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-21T15:00:25.154373+0000","last_change":"2024-09-21T15:00:20.149221+0000","last_active":"2024-09-21T15:00:25.154373+0000","last_peered":"2024-09-21T15:00:25.154373+0000","last_clean":"2024-09-21T15:00:25.154373+0000","last_became_active":"2024-09-21T14:59:28.233956+0000","last_became_peered":"2024-09-21T14:59:28.233956+0000","last_unstale":"2024-09-21T15:00:25.154373+0000","last_undegraded":"2024-09-21T15:00:25.154373+0000","last_fullsized":"2024-09-21T15:00:25.154373+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":646,"log_dups_size":0,"ondisk_log_size":646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:29:09.682792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012266880000000001,"stat_sum":{"num_bytes":18356978,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":3436,"num_read_kb":24365,"num_write":1090,"num_write_kb":25160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":91,"num_bytes_recovered":6925016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"135'619","reported_seq":3511,"reported_epoch":139,"state":"active+undersized+remapped","last_fresh":"2024-09-21T15:00:25.469822+0000","last_change":"2024-09-21T15:00:25.469822+0000","last_active":"2024-09-21T15:00:25.469822+0000","last_peered":"2024-09-21T15:00:25.469822+0000","last_clean":"2024-09-21T15:00:20.853145+0000","last_became_active":"2024-09-21T15:00:25.167619+0000","last_became_peered":"2024-09-21T15:00:25.167619+0000","last_unstale":"2024-09-21T15:00:25.469822+0000","last_undegraded":"2024-09-21T15:00:25.469822+0000","last_fullsized":"2024-09-21T15:00:25.148805+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":619,"log_dups_size":0,"ondisk_log_size":619,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:02:44.956221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029249699999999998,"stat_sum":{"num_bytes":18890884,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":4970,"num_read_kb":26508,"num_write":870,"num_write_kb":28561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":9228123,"num_keys_recovered":33,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1],"avail_no_missing":["1","0","3"],"object_location_counts":[{"shards":"1","objects":40}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"135'358","reported_seq":2407,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-21T15:00:26.323770+0000","last_change":"2024-09-21T15:00:26.324219+0000","last_active":"2024-09-21T15:00:26.323770+0000","last_peered":"2024-09-21T15:00:26.323770+0000","last_clean":"2024-09-21T15:00:26.323770+0000","last_became_active":"2024-09-21T14:58:19.410679+0000","last_became_peered":"2024-09-21T14:58:19.410679+0000","last_unstale":"2024-09-21T15:00:26.323770+0000","last_undegraded":"2024-09-21T15:00:26.323770+0000","last_fullsized":"2024-09-21T15:00:26.323770+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":358,"log_dups_size":0,"ondisk_log_size":358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:07:40.758932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068117099999999995,"stat_sum":{"num_bytes":26807287,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":2481,"num_read_kb":32558,"num_write":576,"num_write_kb":34344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"56","length":"1"}]},{"pgid":"3.c","version":"135'336","reported_seq":3696,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-21T15:00:25.145483+0000","last_change":"2024-09-21T15:00:20.239086+0000","last_active":"2024-09-21T15:00:25.145483+0000","last_peered":"2024-09-21T15:00:25.145483+0000","last_clean":"2024-09-21T15:00:25.145483+0000","last_became_active":"2024-09-21T14:59:25.130057+0000","last_became_peered":"2024-09-21T14:59:25.130057+0000","last_unstale":"2024-09-21T15:00:25.145483+0000","last_undegraded":"2024-09-21T15:00:25.145483+0000","last_fullsized":"2024-09-21T15:00:25.145483+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":336,"log_dups_size":0,"ondisk_log_size":336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:53:33.752550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000824236,"stat_sum":{"num_bytes":15284342,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":3873,"num_read_kb":20986,"num_write":488,"num_write_kb":16588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":9059600,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"139'313","reported_seq":3069,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-21T15:00:25.193822+0000","last_change":"2024-09-21T15:00:20.151374+0000","last_active":"2024-09-21T15:00:25.193822+0000","last_peered":"2024-09-21T15:00:25.193822+0000","last_clean":"2024-09-21T15:00:25.193822+0000","last_became_active":"2024-09-21T14:58:19.407884+0000","last_became_peered":"2024-09-21T14:58:19.407884+0000","last_unstale":"2024-09-21T15:00:25.193822+0000","last_undegraded":"2024-09-21T15:00:25.193822+0000","last_fullsized":"2024-09-21T15:00:25.193822+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":313,"log_dups_size":0,"ondisk_log_size":313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:28:57.111615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028543610000000001,"stat_sum":{"num_bytes":40543708,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":2997,"num_read_kb":40113,"num_write":535,"num_write_kb":28485,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"135'712","reported_seq":2667,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-21T15:00:25.145769+0000","last_change":"2024-09-21T15:00:20.239006+0000","last_active":"2024-09-21T15:00:25.145769+0000","last_peered":"2024-09-21T15:00:25.145769+0000","last_clean":"2024-09-21T15:00:25.145769+0000","last_became_active":"2024-09-21T14:59:13.131825+0000","last_became_peered":"2024-09-21T14:59:13.131825+0000","last_unstale":"2024-09-21T15:00:25.145769+0000","last_undegraded":"2024-09-21T15:00:25.145769+0000","last_fullsized":"2024-09-21T15:00:25.145769+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":712,"log_dups_size":0,"ondisk_log_size":712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:20:52.475349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087421999999999999,"stat_sum":{"num_bytes":10512224,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":4065,"num_read_kb":14538,"num_write":1187,"num_write_kb":11347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":2113894,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"139'358","reported_seq":3224,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-21T15:00:25.184580+0000","last_change":"2024-09-21T15:00:20.238568+0000","last_active":"2024-09-21T15:00:25.184580+0000","last_peered":"2024-09-21T15:00:25.184580+0000","last_clean":"2024-09-21T15:00:25.184580+0000","last_became_active":"2024-09-21T14:58:19.410406+0000","last_became_peered":"2024-09-21T14:58:19.410406+0000","last_unstale":"2024-09-21T15:00:25.184580+0000","last_undegraded":"2024-09-21T15:00:25.184580+0000","last_fullsized":"2024-09-21T15:00:25.184580+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":358,"log_dups_size":0,"ondisk_log_size":358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:59:16.026939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050337799999999996,"stat_sum":{"num_bytes":40642936,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":3272,"num_read_kb":38642,"num_write":581,"num_write_kb":31437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"135'347","reported_seq":3516,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-21T15:00:25.187225+0000","last_change":"2024-09-21T15:00:20.153410+0000","last_active":"2024-09-21T15:00:25.187225+0000","last_peered":"2024-09-21T15:00:25.187225+0000","last_clean":"2024-09-21T15:00:25.187225+0000","last_became_active":"2024-09-21T14:58:19.409935+0000","last_became_peered":"2024-09-21T14:58:19.409935+0000","last_unstale":"2024-09-21T15:00:25.187225+0000","last_undegraded":"2024-09-21T15:00:25.187225+0000","last_fullsized":"2024-09-21T15:00:25.187225+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":347,"log_dups_size":0,"ondisk_log_size":347,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:10:36.252629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057687199999999998,"stat_sum":{"num_bytes":44857867,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":3301,"num_read_kb":51722,"num_write":541,"num_write_kb":26185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"135'313","reported_seq":3336,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-21T15:00:25.153791+0000","last_change":"2024-09-21T15:00:20.148414+0000","last_active":"2024-09-21T15:00:25.153791+0000","last_peered":"2024-09-21T15:00:25.153791+0000","last_clean":"2024-09-21T15:00:25.153791+0000","last_became_active":"2024-09-21T14:59:10.128001+0000","last_became_peered":"2024-09-21T14:59:10.128001+0000","last_unstale":"2024-09-21T15:00:25.153791+0000","last_undegraded":"2024-09-21T15:00:25.153791+0000","last_fullsized":"2024-09-21T15:00:25.153791+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":313,"log_dups_size":0,"ondisk_log_size":313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:44:22.770961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071792199999999996,"stat_sum":{"num_bytes":15225199,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":3749,"num_read_kb":18874,"num_write":354,"num_write_kb":15462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":3094046,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"139'236","reported_seq":3762,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-21T15:00:25.173227+0000","last_change":"2024-09-21T15:00:20.155597+0000","last_active":"2024-09-21T15:00:25.173227+0000","last_peered":"2024-09-21T15:00:25.173227+0000","last_clean":"2024-09-21T15:00:25.173227+0000","last_became_active":"2024-09-21T15:00:07.155144+0000","last_became_peered":"2024-09-21T15:00:07.155144+0000","last_unstale":"2024-09-21T15:00:25.173227+0000","last_undegraded":"2024-09-21T15:00:25.173227+0000","last_fullsized":"2024-09-21T15:00:25.173227+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":236,"log_dups_size":0,"ondisk_log_size":236,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:58:00.133639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061691899999999995,"stat_sum":{"num_bytes":31662596,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":3742,"num_read_kb":14195,"num_write":329,"num_write_kb":11076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"139'466","reported_seq":2998,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-21T15:00:25.187815+0000","last_change":"2024-09-21T15:00:20.149024+0000","last_active":"2024-09-21T15:00:25.187815+0000","last_peered":"2024-09-21T15:00:25.187815+0000","last_clean":"2024-09-21T15:00:25.187815+0000","last_became_active":"2024-09-21T14:58:19.412030+0000","last_became_peered":"2024-09-21T14:58:19.412030+0000","last_unstale":"2024-09-21T15:00:25.187815+0000","last_undegraded":"2024-09-21T15:00:25.187815+0000","last_fullsized":"2024-09-21T15:00:25.187815+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":466,"log_dups_size":0,"ondisk_log_size":466,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:44:29.282391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085379100000000001,"stat_sum":{"num_bytes":59265446,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":3698,"num_read_kb":41551,"num_write":630,"num_write_kb":21772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":849222608,"num_objects":1052,"num_object_clones":0,"num_object_copies":2104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1052,"num_whiteouts":0,"num_read":110259,"num_read_kb":936449,"num_write":19496,"num_write_kb":701110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1306,"num_bytes_recovered":89636335,"num_keys_recovered":152,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":54116352,"data_stored":479842997,"data_compressed":10939599,"data_compressed_allocated":43180032,"data_compressed_original":473742349,"omap_allocated":0,"internal_metadata":0},"log_size":12827,"ondisk_log_size":12827,"up":64,"acting":63,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":10404,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509473,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49364,"kb_used_data":5952,"kb_used_omap":1,"kb_used_meta":43390,"kb_avail":94322476,"statfs":{"total":96636764160,"available":96586215424,"internally_reserved":0,"allocated":6094848,"data_stored":40235819,"data_compressed":891256,"data_compressed_allocated":3534848,"data_compressed_original":38959907,"omap_allocated":1589,"internal_metadata":44431819},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:00:03 2024","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.84499999999999997,"15min":0.83899999999999997},"min":{"1min":0.504,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.95099999999999996,"15min":0.95799999999999996},"min":{"1min":0.51400000000000001,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.4970000000000001,"5min":1.917,"15min":1.917},"last":0.66200000000000003}]},{"osd":1,"last update":"Sat Sep 21 15:00:03 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.78500000000000003,"15min":0.76900000000000002},"min":{"1min":0.626,"5min":0.312,"15min":0.312},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":1.0049999999999999},{"interface":"front","average":{"1min":0.879,"5min":0.83499999999999996,"15min":0.82699999999999996},"min":{"1min":0.53600000000000003,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.224,"5min":2.2149999999999999,"15min":2.2149999999999999},"last":1.208}]},{"osd":2,"last update":"Sat Sep 21 15:00:03 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.81699999999999995,"15min":0.81100000000000005},"min":{"1min":0.495,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.1859999999999999,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":1.169},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.95099999999999996,"15min":0.94799999999999995},"min":{"1min":0.72199999999999998,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.3300000000000001,"5min":1.466,"15min":1.466},"last":0.92700000000000005}]},{"osd":3,"last update":"Sat Sep 21 15:00:03 2024","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.86199999999999999,"15min":0.85399999999999998},"min":{"1min":0.67200000000000004,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.3120000000000001,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":1.127},{"interface":"front","average":{"1min":1.0429999999999999,"5min":0.97999999999999998,"15min":0.96999999999999997},"min":{"1min":0.68500000000000005,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.244,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.98599999999999999}]},{"osd":4,"last update":"Sat Sep 21 15:00:03 2024","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.93700000000000006,"15min":0.93799999999999994},"min":{"1min":0.64200000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.4510000000000001,"5min":2.7130000000000001,"15min":2.7130000000000001},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.0369999999999999,"5min":0.94099999999999995,"15min":0.92500000000000004},"min":{"1min":0.59999999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.6499999999999999,"5min":2.125,"15min":2.125},"last":0.95299999999999996}]},{"osd":5,"last update":"Sat Sep 21 15:00:03 2024","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.0409999999999999,"15min":1.0269999999999999},"min":{"1min":0.71099999999999997,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.4770000000000001,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":0.83699999999999997},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.0640000000000001,"15min":1.0580000000000001},"min":{"1min":0.70799999999999996,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.8169999999999999,"5min":3.1880000000000002,"15min":3.1880000000000002},"last":0.86799999999999999}]},{"osd":6,"last update":"Sat Sep 21 15:00:03 2024","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.163,"15min":1.159},"min":{"1min":0.71299999999999997,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.6579999999999999,"5min":3.2730000000000001,"15min":3.2730000000000001},"last":1.042},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.0469999999999999,"15min":1.0249999999999999},"min":{"1min":0.79100000000000004,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.6180000000000001,"5min":2.2879999999999998,"15min":2.2879999999999998},"last":1.2470000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542176,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57928,"kb_used_data":8760,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94313912,"statfs":{"total":96636764160,"available":96577445888,"internally_reserved":0,"allocated":8970240,"data_stored":66701824,"data_compressed":1521243,"data_compressed_allocated":5914624,"data_compressed_original":65127151,"omap_allocated":1593,"internal_metadata":50330055},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:00:06 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.94399999999999995,"15min":0.95699999999999996},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.244,"5min":1.593,"15min":1.593},"last":1.177},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.93600000000000005,"15min":0.95099999999999996},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.3360000000000001,"5min":1.587,"15min":1.587},"last":0.94899999999999995}]},{"osd":1,"last update":"Sat Sep 21 15:00:06 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.90400000000000003,"15min":0.91400000000000003},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.1120000000000001,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":1.3100000000000001},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.95699999999999996,"15min":0.97999999999999998},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.1819999999999999,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.73599999999999999}]},{"osd":2,"last update":"Sat Sep 21 15:00:06 2024","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.98499999999999999,"15min":0.99099999999999999},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.3560000000000001,"5min":1.758,"15min":1.758},"last":1.389},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.89900000000000002,"15min":0.90900000000000003},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.3069999999999999,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":1.353}]},{"osd":3,"last update":"Sat Sep 21 15:00:06 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.94499999999999995,"15min":0.95799999999999996},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.169,"5min":1.835,"15min":1.835},"last":1.498},{"interface":"front","average":{"1min":1.012,"5min":1.1180000000000001,"15min":1.1359999999999999},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.4430000000000001,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":1.4670000000000001}]},{"osd":4,"last update":"Sat Sep 21 15:00:06 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.97699999999999998,"15min":1.0049999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.1379999999999999,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.93999999999999995,"5min":1.02,"15min":1.0329999999999999},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.411,"5min":1.897,"15min":1.897},"last":0.623}]},{"osd":5,"last update":"Sat Sep 21 15:00:06 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":1.0860000000000001,"15min":1.1240000000000001},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.2589999999999999,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":1.1040000000000001},{"interface":"front","average":{"1min":1.01,"5min":1.0800000000000001,"15min":1.0920000000000001},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.3779999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":1.609}]},{"osd":7,"last update":"Sat Sep 21 15:00:06 2024","interfaces":[{"interface":"back","average":{"1min":1.123,"5min":1.2350000000000001,"15min":1.2529999999999999},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.4530000000000001,"5min":1.8660000000000001,"15min":1.8660000000000001},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.044,"5min":1.26,"15min":1.296},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.385,"5min":2.004,"15min":2.004},"last":0.997}]}]},{"osd":5,"up_from":14,"seq":60129542177,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40376,"kb_used_data":9252,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94331464,"statfs":{"total":96636764160,"available":96595419136,"internally_reserved":0,"allocated":9474048,"data_stored":70205813,"data_compressed":1425848,"data_compressed_allocated":6139904,"data_compressed_original":68474852,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:00:01 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.96799999999999997,"15min":1.0109999999999999},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.98999999999999999,"5min":4.4800000000000004,"15min":4.4800000000000004},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.80800000000000005,"15min":0.82099999999999995},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.0329999999999999,"5min":1.181,"15min":1.181},"last":0.60499999999999998}]},{"osd":1,"last update":"Sat Sep 21 15:00:01 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.91000000000000003,"15min":0.92800000000000005},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.165,"5min":1.75,"15min":1.75},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.91900000000000004,"15min":0.93500000000000005},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.216,"5min":2.2400000000000002,"15min":2.2400000000000002},"last":0.85899999999999999}]},{"osd":2,"last update":"Sat Sep 21 15:00:01 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.95599999999999996,"15min":0.98799999999999999},"min":{"1min":0.54100000000000004,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.016,"5min":3.419,"15min":3.419},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.88500000000000001,"15min":0.90100000000000002},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.075,"5min":1.423,"15min":1.423},"last":0.73299999999999998}]},{"osd":3,"last update":"Sat Sep 21 15:00:01 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.80100000000000005,"15min":0.80700000000000005},"min":{"1min":0.54800000000000004,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.016,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":1.0069999999999999,"5min":1.081,"15min":1.093},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":2.6379999999999999,"5min":2.6379999999999999,"15min":2.6379999999999999},"last":0.92400000000000004}]},{"osd":4,"last update":"Sat Sep 21 15:00:01 2024","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":1.0129999999999999,"15min":1.024},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":2.5470000000000002,"5min":2.5470000000000002,"15min":2.5470000000000002},"last":1.042},{"interface":"front","average":{"1min":0.93999999999999995,"5min":1.075,"15min":1.097},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":2.5099999999999998,"5min":2.5099999999999998,"15min":2.5099999999999998},"last":0.82099999999999995}]},{"osd":6,"last update":"Sat Sep 21 15:00:01 2024","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.95699999999999996,"15min":0.97299999999999998},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.1539999999999999,"5min":1.244,"15min":1.244},"last":0.98899999999999999},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.325,"15min":1.379},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":2.6789999999999998,"5min":4.2469999999999999,"15min":4.2469999999999999},"last":1.0940000000000001}]},{"osd":7,"last update":"Sat Sep 21 15:00:05 2024","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.3200000000000001,"15min":1.363},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":2.7189999999999999,"5min":3.3010000000000002,"15min":3.3010000000000002},"last":1.1359999999999999},{"interface":"front","average":{"1min":1.0740000000000001,"5min":1.1339999999999999,"15min":1.1439999999999999},"min":{"1min":0.64200000000000002,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":2.5880000000000001,"5min":2.5880000000000001,"15min":2.5880000000000001},"last":1.0700000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542176,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38708,"kb_used_data":7592,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94333132,"statfs":{"total":96636764160,"available":96597127168,"internally_reserved":0,"allocated":7774208,"data_stored":58701185,"data_compressed":1326363,"data_compressed_allocated":5099520,"data_compressed_original":57341847,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:00:04 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.76900000000000002,"15min":0.77900000000000003},"min":{"1min":0.36399999999999999,"5min":0.311,"15min":0.311},"max":{"1min":0.96799999999999997,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.92100000000000004,"15min":0.93799999999999994},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.486,"5min":1.486,"15min":1.486},"last":13.324}]},{"osd":1,"last update":"Sat Sep 21 15:00:04 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.89000000000000001,"15min":0.91600000000000004},"min":{"1min":0.48099999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.92300000000000004,"5min":1.347,"15min":1.347},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.83899999999999997,"15min":0.84499999999999997},"min":{"1min":0.39600000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.0720000000000001,"5min":1.153,"15min":1.153},"last":0.77500000000000002}]},{"osd":2,"last update":"Sat Sep 21 15:00:04 2024","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.875,"15min":0.877},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.268,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":1.048},{"interface":"front","average":{"1min":0.91600000000000004,"5min":1.0129999999999999,"15min":1.0289999999999999},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.258,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.94799999999999995}]},{"osd":3,"last update":"Sat Sep 21 15:00:04 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.89300000000000002,"15min":0.89000000000000001},"min":{"1min":0.68100000000000005,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.407,"5min":1.538,"15min":1.538},"last":0.90800000000000003},{"interface":"front","average":{"1min":1.095,"5min":1.004,"15min":0.98899999999999999},"min":{"1min":0.63100000000000001,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.365,"5min":1.365,"15min":1.365},"last":0.82099999999999995}]},{"osd":5,"last update":"Sat Sep 21 15:00:04 2024","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.97899999999999998,"15min":0.98499999999999999},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.2210000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.872,"5min":0.95199999999999996,"15min":0.96499999999999997},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.1619999999999999,"5min":1.337,"15min":1.337},"last":1.024}]},{"osd":6,"last update":"Sat Sep 21 15:00:04 2024","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":1.0009999999999999,"15min":1.0129999999999999},"min":{"1min":0.51400000000000001,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.2210000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":1.095},{"interface":"front","average":{"1min":1.077,"5min":1.091,"15min":1.093},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.5449999999999999,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":1.004}]},{"osd":7,"last update":"Sat Sep 21 15:00:04 2024","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":1.022,"15min":1.034},"min":{"1min":0.46000000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.3859999999999999,"5min":1.7010000000000001,"15min":1.7010000000000001},"last":0.97599999999999998},{"interface":"front","average":{"1min":1.202,"5min":1.214,"15min":1.216},"min":{"1min":0.78800000000000003,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.4590000000000001,"5min":1.53,"15min":1.53},"last":1.1240000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542177,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63744,"kb_used_data":10476,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94308096,"statfs":{"total":96636764160,"available":96571490304,"internally_reserved":0,"allocated":10727424,"data_stored":79454758,"data_compressed":1762356,"data_compressed_allocated":7073792,"data_compressed_original":77510470,"omap_allocated":1592,"internal_metadata":54524360},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:00:05 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.378,"5min":0.315,"15min":0.315},"max":{"1min":1.0109999999999999,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.71199999999999997,"15min":0.71999999999999997},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.89200000000000002,"5min":1.919,"15min":1.919},"last":0.52500000000000002}]},{"osd":1,"last update":"Sat Sep 21 15:00:05 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.67500000000000004,"15min":0.67600000000000005},"min":{"1min":0.33400000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":1.1240000000000001,"5min":1.135,"15min":1.135},"last":0.438},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.80700000000000005,"15min":0.81699999999999995},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.1599999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.40400000000000003}]},{"osd":2,"last update":"Sat Sep 21 15:00:05 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.72799999999999998,"15min":0.72399999999999998},"min":{"1min":0.40200000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.83399999999999996,"15min":0.83499999999999996},"min":{"1min":0.47499999999999998,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.1759999999999999,"5min":1.254,"15min":1.254},"last":0.56499999999999995}]},{"osd":4,"last update":"Sat Sep 21 15:00:05 2024","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.95699999999999996,"15min":0.96299999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.2709999999999999,"5min":1.7609999999999999,"15min":1.7609999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.91100000000000003,"5min":1.0189999999999999,"15min":1.0369999999999999},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.25,"5min":2.028,"15min":2.028},"last":0.69299999999999995}]},{"osd":5,"last update":"Sat Sep 21 15:00:05 2024","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.052,"15min":1.0569999999999999},"min":{"1min":0.77700000000000002,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.246,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":0.84999999999999998},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.0649999999999999,"15min":1.0660000000000001},"min":{"1min":0.70799999999999996,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.3600000000000001,"5min":1.431,"15min":1.431},"last":0.754}]},{"osd":6,"last update":"Sat Sep 21 15:00:05 2024","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.0860000000000001,"15min":1.0980000000000001},"min":{"1min":0.58599999999999997,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.3879999999999999,"5min":2.1219999999999999,"15min":2.1219999999999999},"last":1.0469999999999999},{"interface":"front","average":{"1min":1.046,"5min":1.097,"15min":1.105},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.7370000000000001,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":1.0920000000000001}]},{"osd":7,"last update":"Sat Sep 21 15:00:05 2024","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.1279999999999999,"15min":1.139},"min":{"1min":0.61799999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.4870000000000001,"5min":1.706,"15min":1.706},"last":0.95099999999999996},{"interface":"front","average":{"1min":1.093,"5min":1.173,"15min":1.1859999999999999},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.516,"5min":2.3380000000000001,"15min":2.3380000000000001},"last":0.80700000000000005}]}]},{"osd":2,"up_from":14,"seq":60129542175,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31436,"kb_used_data":4416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94340404,"statfs":{"total":96636764160,"available":96604573696,"internally_reserved":0,"allocated":4521984,"data_stored":21053250,"data_compressed":752968,"data_compressed_allocated":2179072,"data_compressed_original":19878559,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:00:00 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.62,"15min":0.62},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.97999999999999998,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.66100000000000003,"15min":0.65900000000000003},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":1.0429999999999999,"5min":1.244,"15min":1.244},"last":0.76000000000000001}]},{"osd":1,"last update":"Sat Sep 21 15:00:00 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.69699999999999995,"15min":0.68799999999999994},"min":{"1min":0.32400000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":2.6520000000000001,"5min":2.6520000000000001,"15min":2.6520000000000001},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.622,"5min":0.73999999999999999,"15min":0.76000000000000001},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.89300000000000002,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.93400000000000005}]},{"osd":3,"last update":"Sat Sep 21 15:00:00 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.72599999999999998,"15min":0.72999999999999998},"min":{"1min":0.38900000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.113,"5min":2.1259999999999999,"15min":2.1259999999999999},"last":0.97399999999999998},{"interface":"front","average":{"1min":1.1359999999999999,"5min":0.88400000000000001,"15min":0.84199999999999997},"min":{"1min":0.52600000000000002,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":4.1440000000000001,"5min":4.1440000000000001,"15min":4.1440000000000001},"last":1.0089999999999999}]},{"osd":4,"last update":"Sat Sep 21 15:00:00 2024","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.96199999999999997,"15min":0.96599999999999997},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":2.899,"5min":2.899,"15min":2.899},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.95199999999999996,"15min":0.95999999999999996},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.1679999999999999,"5min":1.675,"15min":1.675},"last":0.84099999999999997}]},{"osd":5,"last update":"Sat Sep 21 15:00:00 2024","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":1.0049999999999999,"15min":1.0069999999999999},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":2.8450000000000002,"5min":2.8450000000000002,"15min":2.8450000000000002},"last":0.88200000000000001},{"interface":"front","average":{"1min":1.002,"5min":1.0009999999999999,"15min":1.0009999999999999},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.502,"5min":1.546,"15min":1.546},"last":1.129}]},{"osd":6,"last update":"Sat Sep 21 15:00:00 2024","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.97899999999999998,"15min":0.98399999999999999},"min":{"1min":0.40100000000000002,"5min":0.378,"15min":0.378},"max":{"1min":1.153,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":1.0529999999999999},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.05,"15min":1.05},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.399,"5min":1.446,"15min":1.446},"last":1.091}]},{"osd":7,"last update":"Sat Sep 21 15:00:07 2024","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":1.0029999999999999,"15min":0.995},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":1.1679999999999999},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.097,"15min":1.091},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.5489999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":1.218}]}]},{"osd":1,"up_from":14,"seq":60129542176,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40704,"kb_used_data":9580,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94331136,"statfs":{"total":96636764160,"available":96595083264,"internally_reserved":0,"allocated":9809920,"data_stored":76850031,"data_compressed":1472262,"data_compressed_allocated":6565888,"data_compressed_original":75163096,"omap_allocated":1595,"internal_metadata":31848901},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:00:05 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.60299999999999998,"15min":0.61099999999999999},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.83599999999999997,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.60099999999999998,"15min":0.60899999999999999},"min":{"1min":0.252,"5min":0.216,"15min":0.216},"max":{"1min":0.90800000000000003,"5min":1.107,"15min":1.107},"last":0.65900000000000003}]},{"osd":2,"last update":"Sat Sep 21 15:00:05 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.68999999999999995,"15min":0.70499999999999996},"min":{"1min":0.313,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":1.399,"5min":2.5529999999999999,"15min":2.5529999999999999},"last":1.3460000000000001},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69599999999999995,"15min":0.69699999999999995},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.95499999999999996,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.85899999999999999}]},{"osd":3,"last update":"Sat Sep 21 15:00:05 2024","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.84399999999999997,"15min":0.84799999999999998},"min":{"1min":0.311,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":3.1469999999999998,"5min":3.1469999999999998,"15min":3.1469999999999998},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.88700000000000001,"15min":0.89300000000000002},"min":{"1min":0.47199999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":3.2290000000000001,"5min":3.2320000000000002,"15min":3.2320000000000002},"last":0.72799999999999998}]},{"osd":4,"last update":"Sat Sep 21 15:00:05 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.89400000000000002,"15min":0.91600000000000004},"min":{"1min":0.38600000000000001,"5min":0.309,"15min":0.309},"max":{"1min":1.1950000000000001,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":0.93899999999999995},{"interface":"front","average":{"1min":1.0409999999999999,"5min":0.93999999999999995,"15min":0.92300000000000004},"min":{"1min":0.54200000000000004,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":4.9409999999999998,"5min":4.9409999999999998,"15min":4.9409999999999998},"last":1.3009999999999999}]},{"osd":5,"last update":"Sat Sep 21 15:00:05 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.90200000000000002,"15min":0.90100000000000002},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":1.169},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.95899999999999996,"15min":0.96099999999999997},"min":{"1min":0.629,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":1.0920000000000001}]},{"osd":6,"last update":"Sat Sep 21 15:00:05 2024","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":1.026,"15min":1.054},"min":{"1min":0.42199999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.3959999999999999,"5min":2.3519999999999999,"15min":2.3519999999999999},"last":1.123},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.94499999999999995,"15min":0.94699999999999995},"min":{"1min":0.58799999999999997,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.494,"5min":1.494,"15min":1.494},"last":1.0029999999999999}]},{"osd":7,"last update":"Sat Sep 21 15:00:05 2024","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":1.0429999999999999,"15min":1.0629999999999999},"min":{"1min":0.48999999999999999,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.4590000000000001,"5min":2.2360000000000002,"15min":2.2360000000000002},"last":1.2130000000000001},{"interface":"front","average":{"1min":0.96399999999999997,"5min":1.028,"15min":1.0389999999999999},"min":{"1min":0.66500000000000004,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.5369999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":1.2529999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542175,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57836,"kb_used_data":8668,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94314004,"statfs":{"total":96636764160,"available":96577540096,"internally_reserved":0,"allocated":8876032,"data_stored":64126658,"data_compressed":1492061,"data_compressed_allocated":5771264,"data_compressed_original":62529219,"omap_allocated":1593,"internal_metadata":50330055},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 21 15:00:04 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.69699999999999995,"15min":0.72499999999999998},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.97599999999999998,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":1.1299999999999999},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.62,"15min":0.60699999999999998},"min":{"1min":0.42399999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":1.252}]},{"osd":2,"last update":"Sat Sep 21 15:00:04 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.61399999999999999,"15min":0.60599999999999998},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.82999999999999996,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.74199999999999999,"15min":0.76400000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.94699999999999995,"5min":1.204,"15min":1.204},"last":0.60399999999999998}]},{"osd":3,"last update":"Sat Sep 21 15:00:04 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.747,"15min":0.748},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":1.2130000000000001},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.79500000000000004,"15min":0.81799999999999995},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.95699999999999996,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.77200000000000002}]},{"osd":4,"last update":"Sat Sep 21 15:00:04 2024","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":1.036,"15min":1.0549999999999999},"min":{"1min":0.56100000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.7,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.99299999999999999,"5min":1.054,"15min":1.0640000000000001},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.373,"5min":1.665,"15min":1.665},"last":0.67800000000000005}]},{"osd":5,"last update":"Sat Sep 21 15:00:04 2024","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":1.012,"15min":1.028},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.204,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":1.171},{"interface":"front","average":{"1min":0.875,"5min":1.05,"15min":1.079},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.319,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":0.91600000000000004}]},{"osd":6,"last update":"Sat Sep 21 15:00:04 2024","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":1.0980000000000001,"15min":1.1259999999999999},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.621,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.99399999999999999,"5min":1.0549999999999999,"15min":1.0649999999999999},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.321,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":1.0509999999999999}]},{"osd":7,"last update":"Sat Sep 21 15:00:04 2024","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":1.052,"15min":1.0640000000000001},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.2769999999999999,"5min":1.323,"15min":1.323},"last":1.0920000000000001},{"interface":"front","average":{"1min":0.94699999999999995,"5min":1.1120000000000001,"15min":1.1399999999999999},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.6719999999999999,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":0.95299999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5202,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5202,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8187904,"data_stored":72788397,"data_compressed":1664711,"data_compressed_allocated":6565888,"data_compressed_original":71892675,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8167424,"data_stored":75584204,"data_compressed":1499377,"data_compressed_allocated":6569984,"data_compressed_original":74696152,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2859008,"data_stored":20255319,"data_compressed":752968,"data_compressed_allocated":2179072,"data_compressed_original":19878559,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":9191424,"data_stored":78960213,"data_compressed":1804145,"data_compressed_allocated":7143424,"data_compressed_original":77776710,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":6111232,"data_stored":57903254,"data_compressed":1326363,"data_compressed_allocated":5099520,"data_compressed_original":57341847,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":7753728,"data_stored":68817514,"data_compressed":1420646,"data_compressed_allocated":6103040,"data_compressed_original":67901412,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":7426048,"data_stored":66102522,"data_compressed":1580133,"data_compressed_allocated":5984256,"data_compressed_original":65295087,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4419584,"data_stored":39431574,"data_compressed":891256,"data_compressed_allocated":3534848,"data_compressed_original":38959907,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T15:00:29.017 INFO:tasks.thrashosds.thrasher:Setting 3.4 to [4, 2] 2024-09-21T15:00:29.017 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.4', '4', '2'] 2024-09-21T15:00:29.017 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.4 4 2 2024-09-21T15:00:29.035 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:29.035 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T15:00:29.035 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T15:00:29.035 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T15:00:29.035 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.035 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45839 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:29.035 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159313+0000", 2024-09-21T15:00:29.035 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 78.872114968000005, 2024-09-21T15:00:29.035 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068238699, 2024-09-21T15:00:29.035 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:29.035 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:29.035 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:29.035 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:29.036 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:29.036 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45839 2024-09-21T15:00:29.036 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.036 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:29.036 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.036 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:29.036 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159313+0000", 2024-09-21T15:00:29.036 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:29.036 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.036 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.036 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:29.036 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159313+0000", 2024-09-21T15:00:29.036 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:29.036 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.036 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.037 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:29.037 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159313+0000", 2024-09-21T15:00:29.037 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:29.037 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.037 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.037 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:29.037 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159316+0000", 2024-09-21T15:00:29.037 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.033e-06 2024-09-21T15:00:29.037 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.037 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.037 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:29.037 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159322+0000", 2024-09-21T15:00:29.037 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1820000000000003e-06 2024-09-21T15:00:29.037 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.037 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.038 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:29.038 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159327+0000", 2024-09-21T15:00:29.038 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.062e-06 2024-09-21T15:00:29.038 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.038 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.038 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:29.038 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227544+0000", 2024-09-21T15:00:29.038 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068217142999999994 2024-09-21T15:00:29.038 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.038 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.038 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:29.038 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227552+0000", 2024-09-21T15:00:29.038 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6260000000000003e-06 2024-09-21T15:00:29.038 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:29.038 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:29.039 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:29.039 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.039 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.039 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45840 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:29.039 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159332+0000", 2024-09-21T15:00:29.039 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 78.872096670999994, 2024-09-21T15:00:29.039 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068262046000000007, 2024-09-21T15:00:29.039 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:29.039 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:29.039 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:29.039 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:29.039 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:29.039 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45840 2024-09-21T15:00:29.039 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.040 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:29.040 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.040 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:29.040 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159332+0000", 2024-09-21T15:00:29.040 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:29.040 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.040 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.040 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:29.040 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159332+0000", 2024-09-21T15:00:29.040 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:29.040 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.040 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.040 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:29.040 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159331+0000", 2024-09-21T15:00:29.040 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:29.040 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.041 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.041 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:29.041 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159351+0000", 2024-09-21T15:00:29.041 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0064999999999998e-05 2024-09-21T15:00:29.041 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.041 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.041 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:29.041 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159367+0000", 2024-09-21T15:00:29.041 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5308e-05 2024-09-21T15:00:29.041 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.041 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.041 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:29.041 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159373+0000", 2024-09-21T15:00:29.041 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1840000000000004e-06 2024-09-21T15:00:29.042 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.042 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.042 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:29.042 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227584+0000", 2024-09-21T15:00:29.042 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068211401000000005 2024-09-21T15:00:29.042 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.042 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.042 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:29.042 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227594+0000", 2024-09-21T15:00:29.042 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4120000000000007e-06 2024-09-21T15:00:29.042 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:29.042 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:29.042 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:29.043 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.043 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.043 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45841 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:29.043 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159379+0000", 2024-09-21T15:00:29.043 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 78.872049515, 2024-09-21T15:00:29.043 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068249039999999997, 2024-09-21T15:00:29.043 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:29.043 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:29.043 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:29.043 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:29.043 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:29.043 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45841 2024-09-21T15:00:29.043 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.043 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:29.044 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.044 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:29.044 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159379+0000", 2024-09-21T15:00:29.044 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:29.044 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.044 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.044 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:29.044 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159379+0000", 2024-09-21T15:00:29.044 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:29.044 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.044 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.044 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:29.044 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159378+0000", 2024-09-21T15:00:29.044 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:29.045 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.045 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.045 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:29.045 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159382+0000", 2024-09-21T15:00:29.045 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.641e-06 2024-09-21T15:00:29.045 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.045 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.045 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:29.045 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159421+0000", 2024-09-21T15:00:29.045 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9439e-05 2024-09-21T15:00:29.045 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.045 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.045 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:29.045 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159430+0000", 2024-09-21T15:00:29.046 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8429999999999999e-06 2024-09-21T15:00:29.046 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.046 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.046 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:29.046 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227617+0000", 2024-09-21T15:00:29.046 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068187084999999995 2024-09-21T15:00:29.046 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.046 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.046 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:29.046 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227628+0000", 2024-09-21T15:00:29.046 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0507e-05 2024-09-21T15:00:29.046 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:29.046 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:29.046 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:29.047 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.047 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.047 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45844 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:29.047 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159499+0000", 2024-09-21T15:00:29.047 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 78.871929128999994, 2024-09-21T15:00:29.047 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068255415, 2024-09-21T15:00:29.047 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:29.047 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:29.047 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:29.047 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:29.047 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:29.047 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45844 2024-09-21T15:00:29.047 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.047 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:29.048 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.048 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:29.048 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159499+0000", 2024-09-21T15:00:29.048 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:29.048 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.048 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.048 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:29.048 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159499+0000", 2024-09-21T15:00:29.048 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:29.048 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.048 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.048 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:29.048 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159499+0000", 2024-09-21T15:00:29.048 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:29.048 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.048 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.049 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:29.049 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159503+0000", 2024-09-21T15:00:29.049 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0999999999999997e-06 2024-09-21T15:00:29.049 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.049 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.049 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:29.049 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159512+0000", 2024-09-21T15:00:29.049 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5850000000000006e-06 2024-09-21T15:00:29.049 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.049 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.049 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:29.049 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159518+0000", 2024-09-21T15:00:29.049 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0909999999999998e-06 2024-09-21T15:00:29.049 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.049 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.050 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:29.050 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227743+0000", 2024-09-21T15:00:29.050 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068224536000000002 2024-09-21T15:00:29.050 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.050 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.050 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:29.050 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227755+0000", 2024-09-21T15:00:29.050 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1659e-05 2024-09-21T15:00:29.050 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:29.050 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:29.050 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:29.050 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.050 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.050 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45845 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:29.050 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159524+0000", 2024-09-21T15:00:29.051 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 78.871904768999997, 2024-09-21T15:00:29.051 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068267194000000003, 2024-09-21T15:00:29.051 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:29.051 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:29.051 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:29.051 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:29.051 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:29.051 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45845 2024-09-21T15:00:29.051 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.051 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:29.051 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.051 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:29.051 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159524+0000", 2024-09-21T15:00:29.051 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:29.051 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.051 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.052 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:29.052 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159524+0000", 2024-09-21T15:00:29.052 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:29.052 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.052 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.052 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:29.052 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159523+0000", 2024-09-21T15:00:29.052 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:29.052 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.052 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.052 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:29.052 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159526+0000", 2024-09-21T15:00:29.052 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1980000000000001e-06 2024-09-21T15:00:29.052 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.052 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.053 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:29.053 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159533+0000", 2024-09-21T15:00:29.053 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.348e-06 2024-09-21T15:00:29.053 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.053 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.053 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:29.053 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159546+0000", 2024-09-21T15:00:29.053 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3464e-05 2024-09-21T15:00:29.053 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.053 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.053 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:29.053 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227783+0000", 2024-09-21T15:00:29.053 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068236926000000003 2024-09-21T15:00:29.053 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.054 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.054 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:29.054 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227791+0000", 2024-09-21T15:00:29.054 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6669999999999996e-06 2024-09-21T15:00:29.054 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:29.054 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:29.054 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:29.054 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.054 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.054 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45846 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:29.054 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159584+0000", 2024-09-21T15:00:29.054 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 78.871844127000003, 2024-09-21T15:00:29.054 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068254767999999993, 2024-09-21T15:00:29.054 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:29.055 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:29.055 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:29.055 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:29.055 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:29.055 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45846 2024-09-21T15:00:29.055 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.055 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:29.055 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.055 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:29.055 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159584+0000", 2024-09-21T15:00:29.055 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:29.055 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.055 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.055 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:29.055 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159584+0000", 2024-09-21T15:00:29.055 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:29.056 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.056 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.056 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:29.056 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159583+0000", 2024-09-21T15:00:29.056 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:29.056 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.056 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.056 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:29.056 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159588+0000", 2024-09-21T15:00:29.056 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1330000000000001e-06 2024-09-21T15:00:29.056 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.056 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.056 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:29.056 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159596+0000", 2024-09-21T15:00:29.056 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5469999999999997e-06 2024-09-21T15:00:29.056 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.057 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.057 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:29.057 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159609+0000", 2024-09-21T15:00:29.057 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2785999999999999e-05 2024-09-21T15:00:29.057 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.057 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.057 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:29.057 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227830+0000", 2024-09-21T15:00:29.057 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068220879999999998 2024-09-21T15:00:29.057 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.057 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.057 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:29.057 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227839+0000", 2024-09-21T15:00:29.057 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1239999999999994e-06 2024-09-21T15:00:29.057 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:29.057 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:29.058 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:29.058 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.058 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.058 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45847 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:29.058 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159614+0000", 2024-09-21T15:00:29.058 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 78.871814051000001, 2024-09-21T15:00:29.058 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068260639999999997, 2024-09-21T15:00:29.058 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:29.058 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:29.058 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:29.058 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:29.058 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:29.058 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45847 2024-09-21T15:00:29.058 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.058 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:29.058 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.059 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:29.059 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159614+0000", 2024-09-21T15:00:29.059 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:29.059 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.059 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.059 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:29.059 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159614+0000", 2024-09-21T15:00:29.059 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:29.059 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.059 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.059 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:29.059 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159614+0000", 2024-09-21T15:00:29.059 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:29.059 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.059 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.059 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:29.059 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159617+0000", 2024-09-21T15:00:29.060 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2909999999999999e-06 2024-09-21T15:00:29.060 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.060 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.060 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:29.060 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159623+0000", 2024-09-21T15:00:29.060 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.511e-06 2024-09-21T15:00:29.060 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.060 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.060 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:29.060 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159631+0000", 2024-09-21T15:00:29.060 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.571e-06 2024-09-21T15:00:29.060 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.060 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.061 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:29.061 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227866+0000", 2024-09-21T15:00:29.061 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068235136000000002 2024-09-21T15:00:29.061 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.061 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.061 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:29.061 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227875+0000", 2024-09-21T15:00:29.061 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7199999999999995e-06 2024-09-21T15:00:29.061 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:29.061 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:29.061 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:29.061 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.061 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.061 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45848 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:29.061 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159636+0000", 2024-09-21T15:00:29.062 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 78.871792271000004, 2024-09-21T15:00:29.062 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068267283999999998, 2024-09-21T15:00:29.062 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:29.062 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:29.062 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:29.062 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:29.062 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:29.062 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45848 2024-09-21T15:00:29.062 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.062 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:29.062 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.062 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:29.062 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159636+0000", 2024-09-21T15:00:29.063 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:29.063 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.063 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.063 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:29.063 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159636+0000", 2024-09-21T15:00:29.063 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:29.063 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.063 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.063 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:29.063 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159636+0000", 2024-09-21T15:00:29.063 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:29.063 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.063 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.063 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:29.064 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159638+0000", 2024-09-21T15:00:29.064 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6560000000000001e-06 2024-09-21T15:00:29.064 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.064 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.064 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:29.064 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159644+0000", 2024-09-21T15:00:29.064 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0839999999999997e-06 2024-09-21T15:00:29.064 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.064 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.064 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:29.064 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159649+0000", 2024-09-21T15:00:29.064 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7500000000000003e-06 2024-09-21T15:00:29.065 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.065 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.065 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:29.065 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227892+0000", 2024-09-21T15:00:29.065 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068243315999999998 2024-09-21T15:00:29.065 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.065 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.065 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:29.065 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227903+0000", 2024-09-21T15:00:29.065 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0835999999999999e-05 2024-09-21T15:00:29.065 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:29.065 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:29.065 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:29.065 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.066 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.066 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45849 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:29.066 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159667+0000", 2024-09-21T15:00:29.066 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 78.871761457000005, 2024-09-21T15:00:29.066 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068291051000000005, 2024-09-21T15:00:29.066 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:29.066 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:29.066 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:29.066 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:29.066 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:29.066 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45849 2024-09-21T15:00:29.066 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.066 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:29.066 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.067 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:29.067 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159667+0000", 2024-09-21T15:00:29.067 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:29.067 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.067 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.067 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:29.067 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159667+0000", 2024-09-21T15:00:29.067 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:29.067 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.067 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.067 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:29.067 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159666+0000", 2024-09-21T15:00:29.067 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:29.067 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.067 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.068 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:29.068 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159670+0000", 2024-09-21T15:00:29.068 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.427e-06 2024-09-21T15:00:29.068 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.068 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.068 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:29.068 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159678+0000", 2024-09-21T15:00:29.068 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9869999999999999e-06 2024-09-21T15:00:29.068 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.068 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.068 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:29.068 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159688+0000", 2024-09-21T15:00:29.068 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0631999999999999e-05 2024-09-21T15:00:29.068 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.069 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.069 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:29.069 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227950+0000", 2024-09-21T15:00:29.069 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068261798999999998 2024-09-21T15:00:29.069 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.069 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.069 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:29.069 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227958+0000", 2024-09-21T15:00:29.069 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7510000000000008e-06 2024-09-21T15:00:29.069 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:29.069 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:29.069 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:29.069 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.069 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.070 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45850 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:29.070 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159695+0000", 2024-09-21T15:00:29.070 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 78.871733746000004, 2024-09-21T15:00:29.070 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068302999000000003, 2024-09-21T15:00:29.070 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:29.070 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:29.070 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:29.070 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:29.070 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:29.070 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45850 2024-09-21T15:00:29.070 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.070 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:29.070 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.071 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:29.071 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159695+0000", 2024-09-21T15:00:29.071 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:29.071 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.071 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.071 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:29.071 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159695+0000", 2024-09-21T15:00:29.071 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:29.071 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.071 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.071 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:29.071 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159694+0000", 2024-09-21T15:00:29.071 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:29.071 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.071 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.072 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:29.072 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159697+0000", 2024-09-21T15:00:29.072 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.258e-06 2024-09-21T15:00:29.072 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.072 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.072 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:29.072 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159704+0000", 2024-09-21T15:00:29.072 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7660000000000001e-06 2024-09-21T15:00:29.072 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.072 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.072 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:29.072 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159713+0000", 2024-09-21T15:00:29.072 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1779999999999992e-06 2024-09-21T15:00:29.073 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.073 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.073 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:29.073 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227989+0000", 2024-09-21T15:00:29.073 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068276015999999995 2024-09-21T15:00:29.073 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.073 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.073 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:29.073 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227998+0000", 2024-09-21T15:00:29.073 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2050000000000002e-06 2024-09-21T15:00:29.073 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:29.073 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:29.073 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:29.073 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.074 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.074 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45851 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:29.074 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159719+0000", 2024-09-21T15:00:29.074 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 78.871709612999993, 2024-09-21T15:00:29.074 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068320327, 2024-09-21T15:00:29.074 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:29.074 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:29.074 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:29.074 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:29.074 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:29.074 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45851 2024-09-21T15:00:29.074 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.074 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:29.074 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.075 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:29.075 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159719+0000", 2024-09-21T15:00:29.075 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:29.075 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.075 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.075 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:29.075 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159719+0000", 2024-09-21T15:00:29.075 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:29.075 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.075 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.075 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:29.075 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159718+0000", 2024-09-21T15:00:29.075 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:29.075 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.076 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.076 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:29.076 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159721+0000", 2024-09-21T15:00:29.076 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2169999999999999e-06 2024-09-21T15:00:29.076 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.076 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.076 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:29.076 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159730+0000", 2024-09-21T15:00:29.076 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5720000000000003e-06 2024-09-21T15:00:29.076 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.076 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.076 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:29.076 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159735+0000", 2024-09-21T15:00:29.076 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1100000000000002e-06 2024-09-21T15:00:29.077 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.077 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.077 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:29.077 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228029+0000", 2024-09-21T15:00:29.077 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068293624999999997 2024-09-21T15:00:29.077 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.077 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.077 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:29.077 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228039+0000", 2024-09-21T15:00:29.077 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0254e-05 2024-09-21T15:00:29.077 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:29.077 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:29.077 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:29.077 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.078 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.078 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45852 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:29.078 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159740+0000", 2024-09-21T15:00:29.078 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 78.871688445000004, 2024-09-21T15:00:29.078 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068340260999999999, 2024-09-21T15:00:29.078 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:29.078 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:29.078 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:29.078 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:29.078 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:29.078 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45852 2024-09-21T15:00:29.078 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.078 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:29.079 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.079 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:29.079 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159740+0000", 2024-09-21T15:00:29.079 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:29.079 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.079 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.079 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:29.079 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159740+0000", 2024-09-21T15:00:29.079 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:29.079 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.079 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.079 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:29.079 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159739+0000", 2024-09-21T15:00:29.079 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:29.079 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.080 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.080 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:29.080 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159757+0000", 2024-09-21T15:00:29.080 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7314999999999999e-05 2024-09-21T15:00:29.080 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.080 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.080 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:29.080 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159769+0000", 2024-09-21T15:00:29.080 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2376e-05 2024-09-21T15:00:29.080 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.080 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.080 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:29.080 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159775+0000", 2024-09-21T15:00:29.080 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2179999999999996e-06 2024-09-21T15:00:29.081 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.081 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.081 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:29.081 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228071+0000", 2024-09-21T15:00:29.081 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068295344999999993 2024-09-21T15:00:29.081 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.081 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.081 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:29.081 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228080+0000", 2024-09-21T15:00:29.081 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3959999999999995e-06 2024-09-21T15:00:29.081 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:29.081 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:29.081 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:29.081 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.081 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.082 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45853 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:29.082 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159781+0000", 2024-09-21T15:00:29.082 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 78.871647181, 2024-09-21T15:00:29.082 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068337093000000002, 2024-09-21T15:00:29.082 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:29.082 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:29.082 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:29.082 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:29.082 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:29.082 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45853 2024-09-21T15:00:29.082 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.082 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:29.082 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.082 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:29.082 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159781+0000", 2024-09-21T15:00:29.083 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:29.083 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.083 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.083 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:29.083 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159781+0000", 2024-09-21T15:00:29.083 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:29.083 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.083 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.083 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:29.083 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159781+0000", 2024-09-21T15:00:29.083 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:29.083 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.083 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.083 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:29.083 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159784+0000", 2024-09-21T15:00:29.084 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4759999999999998e-06 2024-09-21T15:00:29.084 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.084 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.084 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:29.084 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159792+0000", 2024-09-21T15:00:29.084 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0569999999999993e-06 2024-09-21T15:00:29.084 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.084 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.084 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:29.084 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159797+0000", 2024-09-21T15:00:29.084 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9640000000000002e-06 2024-09-21T15:00:29.084 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.084 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.084 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:29.084 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228111+0000", 2024-09-21T15:00:29.085 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068313737999999999 2024-09-21T15:00:29.085 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.085 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.085 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:29.085 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228118+0000", 2024-09-21T15:00:29.085 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.43e-06 2024-09-21T15:00:29.085 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:29.085 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:29.085 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:29.085 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.085 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.085 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45854 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:29.085 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159802+0000", 2024-09-21T15:00:29.085 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 78.871626531000004, 2024-09-21T15:00:29.085 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068358938999999994, 2024-09-21T15:00:29.085 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:29.086 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:29.086 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:29.086 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:29.086 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:29.086 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45854 2024-09-21T15:00:29.086 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.086 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:29.086 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.086 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:29.086 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159802+0000", 2024-09-21T15:00:29.086 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:29.086 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.086 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.086 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:29.086 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159802+0000", 2024-09-21T15:00:29.087 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:29.087 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.087 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.087 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:29.087 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159801+0000", 2024-09-21T15:00:29.087 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:29.087 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.087 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.087 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:29.087 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159804+0000", 2024-09-21T15:00:29.087 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0970000000000002e-06 2024-09-21T15:00:29.087 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.087 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.088 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:29.088 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159823+0000", 2024-09-21T15:00:29.088 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8895000000000001e-05 2024-09-21T15:00:29.088 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.088 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.088 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:29.088 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159837+0000", 2024-09-21T15:00:29.088 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3247999999999999e-05 2024-09-21T15:00:29.088 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.088 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.088 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:29.088 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228151+0000", 2024-09-21T15:00:29.088 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068314642999999994 2024-09-21T15:00:29.088 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.088 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.088 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:29.089 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228161+0000", 2024-09-21T15:00:29.089 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4250000000000004e-06 2024-09-21T15:00:29.089 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:29.089 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:29.089 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:29.089 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.089 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.089 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45983 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:29.089 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159848+0000", 2024-09-21T15:00:29.089 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 78.871580272000003, 2024-09-21T15:00:29.089 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068333801, 2024-09-21T15:00:29.089 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:29.089 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:29.089 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:29.089 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:29.090 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:29.090 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45983 2024-09-21T15:00:29.090 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.090 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:29.090 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.090 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:29.090 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159848+0000", 2024-09-21T15:00:29.090 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:29.090 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.090 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.090 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:29.090 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159848+0000", 2024-09-21T15:00:29.090 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:29.090 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.091 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.091 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:29.091 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159847+0000", 2024-09-21T15:00:29.091 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:29.091 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.091 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.091 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:29.091 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159856+0000", 2024-09-21T15:00:29.091 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4349999999999997e-06 2024-09-21T15:00:29.091 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.091 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.091 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:29.091 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159866+0000", 2024-09-21T15:00:29.091 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0076e-05 2024-09-21T15:00:29.091 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.091 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.092 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:29.092 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159872+0000", 2024-09-21T15:00:29.092 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8309999999999997e-06 2024-09-21T15:00:29.092 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.092 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.092 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:29.092 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228175+0000", 2024-09-21T15:00:29.092 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068302680000000005 2024-09-21T15:00:29.092 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.092 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.092 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:29.092 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228182+0000", 2024-09-21T15:00:29.092 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3150000000000003e-06 2024-09-21T15:00:29.092 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:29.092 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:29.092 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:29.093 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.093 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.093 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45984 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:29.093 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159877+0000", 2024-09-21T15:00:29.093 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 78.871550851999999, 2024-09-21T15:00:29.093 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068328448, 2024-09-21T15:00:29.093 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:29.093 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:29.093 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:29.093 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:29.093 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:29.093 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45984 2024-09-21T15:00:29.093 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.093 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:29.093 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.094 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:29.094 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159877+0000", 2024-09-21T15:00:29.094 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:29.094 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.094 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.094 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:29.094 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159877+0000", 2024-09-21T15:00:29.094 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:29.094 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.094 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.094 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:29.094 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159877+0000", 2024-09-21T15:00:29.094 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:29.094 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.095 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.095 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:29.095 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159880+0000", 2024-09-21T15:00:29.095 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7610000000000001e-06 2024-09-21T15:00:29.095 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.095 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.095 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:29.095 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159888+0000", 2024-09-21T15:00:29.095 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0330000000000003e-06 2024-09-21T15:00:29.095 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.095 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.095 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:29.095 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159893+0000", 2024-09-21T15:00:29.095 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3550000000000004e-06 2024-09-21T15:00:29.095 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.096 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.096 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:29.096 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228199+0000", 2024-09-21T15:00:29.096 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068306438999999997 2024-09-21T15:00:29.096 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.096 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.096 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:29.096 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228206+0000", 2024-09-21T15:00:29.096 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5599999999999999e-06 2024-09-21T15:00:29.096 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:29.096 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:29.096 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:29.096 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.096 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.096 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45985 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:29.096 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159897+0000", 2024-09-21T15:00:29.097 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 78.871531028999996, 2024-09-21T15:00:29.097 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068345664, 2024-09-21T15:00:29.097 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:29.097 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:29.097 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:29.097 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:29.097 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:29.097 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45985 2024-09-21T15:00:29.097 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.097 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:29.097 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.097 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:29.097 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159897+0000", 2024-09-21T15:00:29.098 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:29.098 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.098 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.098 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:29.098 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159897+0000", 2024-09-21T15:00:29.098 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:29.098 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.098 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.098 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:29.098 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159897+0000", 2024-09-21T15:00:29.098 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:29.098 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.098 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.098 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:29.098 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159900+0000", 2024-09-21T15:00:29.098 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.954e-06 2024-09-21T15:00:29.099 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.099 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.099 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:29.099 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159913+0000", 2024-09-21T15:00:29.099 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3106e-05 2024-09-21T15:00:29.099 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.099 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.099 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:29.099 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159918+0000", 2024-09-21T15:00:29.099 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5199999999999997e-06 2024-09-21T15:00:29.099 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.099 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.099 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:29.099 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228236+0000", 2024-09-21T15:00:29.099 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068317040999999995 2024-09-21T15:00:29.099 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.100 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.100 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:29.100 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228243+0000", 2024-09-21T15:00:29.100 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3880000000000002e-06 2024-09-21T15:00:29.100 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:29.100 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:29.100 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:29.100 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.100 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.100 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45986 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:29.100 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159924+0000", 2024-09-21T15:00:29.100 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 78.871504733999998, 2024-09-21T15:00:29.100 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068364362999999997, 2024-09-21T15:00:29.100 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:29.100 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:29.100 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:29.100 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:29.101 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:29.101 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45986 2024-09-21T15:00:29.101 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.101 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:29.101 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.101 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:29.101 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159924+0000", 2024-09-21T15:00:29.101 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:29.101 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.101 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.101 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:29.101 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159924+0000", 2024-09-21T15:00:29.101 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:29.101 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.101 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.101 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:29.102 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159923+0000", 2024-09-21T15:00:29.102 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:29.102 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.102 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.102 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:29.102 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159926+0000", 2024-09-21T15:00:29.102 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0510000000000001e-06 2024-09-21T15:00:29.102 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.102 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.102 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:29.102 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159932+0000", 2024-09-21T15:00:29.102 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4160000000000001e-06 2024-09-21T15:00:29.102 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.102 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.102 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:29.102 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159938+0000", 2024-09-21T15:00:29.103 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.383e-06 2024-09-21T15:00:29.103 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.103 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.103 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:29.103 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228275+0000", 2024-09-21T15:00:29.103 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068337007000000005 2024-09-21T15:00:29.103 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.103 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.103 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:29.103 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228288+0000", 2024-09-21T15:00:29.103 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3052e-05 2024-09-21T15:00:29.103 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:29.103 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:29.103 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:29.104 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.104 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.104 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45987 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:29.104 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159944+0000", 2024-09-21T15:00:29.104 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 78.871484510000002, 2024-09-21T15:00:29.104 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068387232000000006, 2024-09-21T15:00:29.104 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:29.104 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:29.104 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:29.104 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:29.104 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:29.104 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45987 2024-09-21T15:00:29.104 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.104 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:29.105 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.105 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:29.105 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159944+0000", 2024-09-21T15:00:29.105 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:29.105 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.105 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.105 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:29.105 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159944+0000", 2024-09-21T15:00:29.105 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:29.105 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.105 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.105 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:29.105 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159943+0000", 2024-09-21T15:00:29.105 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:29.105 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.105 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.106 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:29.106 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159946+0000", 2024-09-21T15:00:29.106 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6309999999999999e-06 2024-09-21T15:00:29.106 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.106 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.106 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:29.106 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159952+0000", 2024-09-21T15:00:29.106 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6540000000000004e-06 2024-09-21T15:00:29.106 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.106 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.106 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:29.106 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159957+0000", 2024-09-21T15:00:29.106 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9280000000000001e-06 2024-09-21T15:00:29.106 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.106 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.106 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:29.107 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228322+0000", 2024-09-21T15:00:29.107 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068365067000000002 2024-09-21T15:00:29.107 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.107 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.107 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:29.107 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228331+0000", 2024-09-21T15:00:29.107 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3270000000000007e-06 2024-09-21T15:00:29.107 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:29.107 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:29.107 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:29.107 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.107 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.107 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:97672 3.4 3:26046d29:::rbd_data.1136bfc79a70.0000000000000001:head [write 0~4718592 [fadvise_sequential+fadvise_nocache],write 5242880~131072 [fadvise_sequential+fadvise_nocache],write 5439488~147456 [fadvise_sequential+fadvise_nocache],write 5652480~229376 [fadvise_sequential+fadvise_nocache],write 5947392~147456 [fadvise_sequential+fadvise_nocache],write 6160384~229376 [fadvise_sequential+fadvise_nocache],write 6455296~147456 [fadvise_sequential+fadvise_nocache],write 6668288~229376 [fadvise_sequential+fadvise_nocache],write 6963200~147456 [fadvise_sequential+fadvise_nocache],write 7176192~147456 [fadvise_sequential+fadvise_nocache],write 7340032~65536 [fadvise_sequential+fadvise_nocache],write 7471104~147456 [fadvise_sequential+fadvise_nocache],write 7684096~147456 [fadvise_sequential+fadvise_nocache],write 7864320~49152 [fadvise_sequential+fadvise_nocache],write 7979008~147456 [fadvise_sequential+fadvise_nocache],write 8192000~147456 [fadvise_sequential+fadvise_nocache],write 8388608~32768 [fadvise_sequential+fadvise_nocache],write 8519680~114688 [fadvise_sequential+fadvise_nocache],write 8732672~114688 [fadvise_sequential+fadvise_nocache],write 8912896~16384 [fadvise_sequential+fadvise_nocache],write 9109504~32768 [fadvise_sequential+fadvise_nocache],write 9322496~32768 [fadvise_sequential+fadvise_nocache],write 9617408~32768 [fadvise_sequential+fadvise_nocache],write 9830400~32768 [fadvise_sequential+fadvise_nocache],write 9961472~98304 [fadvise_sequential+fadvise_nocache],write 10076160~196608 [fadvise_sequential+fadvise_nocache],write 10289152~278528 [fadvise_sequential+fadvise_nocache],write 10584064~196608 [fadvise_sequential+fadvise_nocache],write 10797056~196608 [fadvise_sequential+fadvise_nocache],write 11010048~65536 [fadvise_sequential+fadvise_nocache],write 11091968~196608 [fadvise_sequential+fadvise_nocache],write 11304960~196608 [fadvise_sequential+fadvise_nocache],write 11517952~65536 [fadvise_sequential+fadvise_nocache],write 11616256~98304 [fadvise_sequential+fadvise_nocache],write 11730944~65536 [fadvise_sequential+fadvise_nocache],write 11829248~98304 [fadvise_sequential+fadvise_nocache],write 11943936~65536 [fadvise_sequential+fadvise_nocache],write 12042240~49152 [fadvise_sequential+fadvise_nocache],write 12124160~98304 [fadvise_sequential+fadvise_nocache],write 12238848~65536 [fadvise_sequential+fadvise_nocache],write 12337152~98304 [fadvise_sequential+fadvise_nocache],write 12451840~65536 [fadvise_sequential+fadvise_nocache],write 12550144~32768 [fadvise_sequential+fadvise_nocache],write 12632064~98304 [fadvise_sequential+fadvise_nocache],write 12845056~98304 [fadvise_sequential+fadvise_nocache],write 13058048~49152 [fadvise_sequential+fadvise_nocache],write 13139968~98304 [fadvise_sequential+fadvise_nocache],write 13352960~98304 [fadvise_sequential+fadvise_nocache],write 13565952~65536 [fadvise_sequential+fadvise_nocache],write 13647872~98304 [fadvise_sequential+fadvise_nocache],write 13860864~98304 [fadvise_sequential+fadvise_nocache],write 14073856~81920 [fadvise_sequential+fadvise_nocache],write 15269888~81920 [fadvise_sequential+fadvise_nocache],write 15482880~81920 [fadvise_sequential+fadvise_nocache],write 15695872~32768 [fadvise_sequential+fadvise_nocache],write 15777792~81920 [fadvise_sequential+fadvise_nocache],write 15990784~81920 [fadvise_sequential+fadvise_nocache],write 16203776~49152 [fadvise_sequential+fadvise_nocache],write 16285696~81920 [fadvise_sequential+fadvise_nocache],write 16498688~81920 [fadvise_sequential+fadvise_nocache],write 16711680~65536 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:29.107 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.298669+0000", 2024-09-21T15:00:29.107 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.732759443999999, 2024-09-21T15:00:29.107 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.10282358900000001, 2024-09-21T15:00:29.108 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:29.108 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:29.108 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:29.108 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:29.108 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:29.108 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97672 2024-09-21T15:00:29.108 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.108 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:29.108 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.108 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:29.108 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.298669+0000", 2024-09-21T15:00:29.108 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:29.108 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.108 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.108 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:29.108 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.298669+0000", 2024-09-21T15:00:29.108 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:29.109 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.109 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.109 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:29.109 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.298667+0000", 2024-09-21T15:00:29.109 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:00:29.109 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.109 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.109 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:29.109 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.327458+0000", 2024-09-21T15:00:29.109 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.028790754000000002 2024-09-21T15:00:29.109 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.109 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.109 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:29.109 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.327492+0000", 2024-09-21T15:00:29.109 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3806999999999997e-05 2024-09-21T15:00:29.109 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.110 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.110 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:29.110 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.327507+0000", 2024-09-21T15:00:29.110 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5112e-05 2024-09-21T15:00:29.110 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.110 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.110 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:29.110 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.327597+0000", 2024-09-21T15:00:29.110 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9938999999999994e-05 2024-09-21T15:00:29.110 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.110 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.110 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:29.110 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.327927+0000", 2024-09-21T15:00:29.110 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000329766 2024-09-21T15:00:29.110 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.110 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.111 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T15:00:29.111 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.330735+0000", 2024-09-21T15:00:29.111 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0028086869999999998 2024-09-21T15:00:29.111 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.111 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.111 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:29.111 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.378299+0000", 2024-09-21T15:00:29.111 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.047563495999999997 2024-09-21T15:00:29.111 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.111 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.111 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:29.111 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.401248+0000", 2024-09-21T15:00:29.111 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.022949311 2024-09-21T15:00:29.111 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.111 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.111 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:29.112 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.401446+0000", 2024-09-21T15:00:29.112 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019831199999999999 2024-09-21T15:00:29.112 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.112 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.112 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:29.112 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.401492+0000", 2024-09-21T15:00:29.112 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6109000000000002e-05 2024-09-21T15:00:29.112 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:29.112 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:29.112 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:29.112 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:29.112 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:29.112 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:29.113 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T15:00:29.114 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:00:29.111+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:00:29.214 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:00:29.211+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:00:29.226 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:29.226 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:00:29.227 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T15:00:29.227 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:29.237 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T15:00:29.301 INFO:teuthology.orchestra.run.smithi032.stderr:noscrub is set 2024-09-21T15:00:29.315 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:00:29.311+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:00:29.361 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:29.361 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:00:29.361 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T15:00:29.361 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T15:00:29.361 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:29.374 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T15:00:29.416 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:00:29.411+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:00:29.485 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:29.485 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T15:00:29.485 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T15:00:29.485 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T15:00:29.485 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.485 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:10651 3.12 3:4e9d23af:::rbd_data.1136f5ed5dd6.0000000000000020:head [write 0~81920 [fadvise_sequential+fadvise_nocache],write 90112~8192 [fadvise_sequential+fadvise_nocache],write 122880~122880 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:29.485 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.292476+0000", 2024-09-21T15:00:29.485 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 110.18968423699999, 2024-09-21T15:00:29.485 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012765669, 2024-09-21T15:00:29.485 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:29.485 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:29.486 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:29.486 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:29.486 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:29.486 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 10651 2024-09-21T15:00:29.486 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.486 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:29.486 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.486 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:29.486 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292476+0000", 2024-09-21T15:00:29.486 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:29.486 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.486 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.486 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:29.486 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292476+0000", 2024-09-21T15:00:29.487 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:29.487 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.487 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.487 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:29.487 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292475+0000", 2024-09-21T15:00:29.487 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:00:29.487 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.487 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.487 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:29.487 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292696+0000", 2024-09-21T15:00:29.487 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000220201 2024-09-21T15:00:29.487 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.487 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.487 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:29.488 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292711+0000", 2024-09-21T15:00:29.488 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5803000000000001e-05 2024-09-21T15:00:29.488 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.488 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.488 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:29.488 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292721+0000", 2024-09-21T15:00:29.488 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6160000000000008e-06 2024-09-21T15:00:29.488 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.488 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.488 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:29.488 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292757+0000", 2024-09-21T15:00:29.488 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6211999999999998e-05 2024-09-21T15:00:29.488 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.488 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.489 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:29.489 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.292878+0000", 2024-09-21T15:00:29.489 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000120306 2024-09-21T15:00:29.489 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.489 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.489 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 3", 2024-09-21T15:00:29.489 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.293019+0000", 2024-09-21T15:00:29.489 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014186400000000001 2024-09-21T15:00:29.489 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.489 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.489 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:29.489 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.294898+0000", 2024-09-21T15:00:29.489 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001878434 2024-09-21T15:00:29.489 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.489 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.489 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:29.490 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.305162+0000", 2024-09-21T15:00:29.490 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.01026373 2024-09-21T15:00:29.490 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.490 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.490 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:29.490 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.305209+0000", 2024-09-21T15:00:29.490 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7615999999999999e-05 2024-09-21T15:00:29.490 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.490 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.490 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:29.490 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.305242+0000", 2024-09-21T15:00:29.490 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2979999999999999e-05 2024-09-21T15:00:29.491 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:29.491 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:29.491 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:29.491 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.491 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.491 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12214 3.4 3:21b43b45:::rbd_data.11363a21d2be.000000000000002b:10 [read 376832~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:29.491 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.619969+0000", 2024-09-21T15:00:29.491 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 109.86219168700001, 2024-09-21T15:00:29.491 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0090292089999999998, 2024-09-21T15:00:29.491 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:29.491 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:29.491 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:29.491 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:29.492 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:29.492 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12214 2024-09-21T15:00:29.492 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.492 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:29.492 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.492 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:29.492 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619969+0000", 2024-09-21T15:00:29.492 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:29.492 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.492 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.492 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:29.492 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619969+0000", 2024-09-21T15:00:29.492 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:29.492 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.492 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.493 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:29.493 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619969+0000", 2024-09-21T15:00:29.493 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:29.493 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.493 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.493 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:29.493 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619971+0000", 2024-09-21T15:00:29.493 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1830000000000001e-06 2024-09-21T15:00:29.493 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.493 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.493 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:29.493 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619976+0000", 2024-09-21T15:00:29.493 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7129999999999996e-06 2024-09-21T15:00:29.493 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.493 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.494 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:29.494 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619981+0000", 2024-09-21T15:00:29.494 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2460000000000003e-06 2024-09-21T15:00:29.494 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.494 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.494 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:29.494 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628602+0000", 2024-09-21T15:00:29.494 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086209519999999994 2024-09-21T15:00:29.494 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.494 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.494 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:29.494 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628734+0000", 2024-09-21T15:00:29.494 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000132242 2024-09-21T15:00:29.495 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.495 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.495 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:29.495 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.628998+0000", 2024-09-21T15:00:29.495 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00026419799999999999 2024-09-21T15:00:29.495 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:29.495 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:29.495 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:29.495 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.495 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.495 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12215 3.4 3:21b43b45:::rbd_data.11363a21d2be.000000000000002b:10 [read 409600~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:29.495 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.619985+0000", 2024-09-21T15:00:29.495 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 109.86217562100001, 2024-09-21T15:00:29.495 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0095457319999999995, 2024-09-21T15:00:29.496 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:29.496 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:29.496 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:29.496 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:29.496 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:29.496 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12215 2024-09-21T15:00:29.496 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.496 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:29.496 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.496 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:29.496 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619985+0000", 2024-09-21T15:00:29.496 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:29.496 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.496 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.497 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:29.497 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619985+0000", 2024-09-21T15:00:29.497 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:29.497 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.497 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.497 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:29.497 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619985+0000", 2024-09-21T15:00:29.497 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:29.497 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.497 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.497 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:29.497 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619987+0000", 2024-09-21T15:00:29.497 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6129999999999998e-06 2024-09-21T15:00:29.497 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.498 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.498 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:29.498 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619993+0000", 2024-09-21T15:00:29.498 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0820000000000004e-06 2024-09-21T15:00:29.498 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.498 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.498 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:29.498 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.619998+0000", 2024-09-21T15:00:29.498 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7759999999999997e-06 2024-09-21T15:00:29.498 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.498 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.498 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:29.498 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629022+0000", 2024-09-21T15:00:29.498 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0090237489999999993 2024-09-21T15:00:29.498 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.499 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.499 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:29.499 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629098+0000", 2024-09-21T15:00:29.499 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5735999999999995e-05 2024-09-21T15:00:29.499 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.499 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.499 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:29.499 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629531+0000", 2024-09-21T15:00:29.499 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00043310100000000003 2024-09-21T15:00:29.499 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:29.499 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:29.499 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:29.499 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.499 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.500 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12216 3.4 3:21b43b45:::rbd_data.11363a21d2be.000000000000002b:10 [read 516096~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:29.500 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.620003+0000", 2024-09-21T15:00:29.500 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 109.86215762499999, 2024-09-21T15:00:29.500 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0098850169999999994, 2024-09-21T15:00:29.500 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:29.500 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:29.500 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:29.500 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:29.500 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:29.500 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12216 2024-09-21T15:00:29.500 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.500 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:29.500 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.501 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:29.501 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620003+0000", 2024-09-21T15:00:29.501 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:29.501 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.501 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.501 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:29.501 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620003+0000", 2024-09-21T15:00:29.501 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:29.501 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.501 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.501 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:29.501 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620003+0000", 2024-09-21T15:00:29.501 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:29.501 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.502 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.502 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:29.502 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620006+0000", 2024-09-21T15:00:29.502 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4350000000000001e-06 2024-09-21T15:00:29.502 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.502 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.502 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:29.502 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620013+0000", 2024-09-21T15:00:29.502 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4219999999999997e-06 2024-09-21T15:00:29.502 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.502 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.502 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:29.503 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620017+0000", 2024-09-21T15:00:29.503 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.865e-06 2024-09-21T15:00:29.503 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.503 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.503 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:29.503 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629581+0000", 2024-09-21T15:00:29.503 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0095631989999999997 2024-09-21T15:00:29.503 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.503 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.503 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:29.503 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629691+0000", 2024-09-21T15:00:29.503 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000110586 2024-09-21T15:00:29.503 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.503 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.504 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:29.504 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629888+0000", 2024-09-21T15:00:29.504 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019691199999999999 2024-09-21T15:00:29.504 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:29.504 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:29.504 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:29.504 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.504 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.504 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12222 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 49152~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:29.504 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.620027+0000", 2024-09-21T15:00:29.504 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 109.86213327900001, 2024-09-21T15:00:29.504 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010228131999999999, 2024-09-21T15:00:29.504 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:29.504 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:29.504 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:29.505 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:29.505 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:29.505 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12222 2024-09-21T15:00:29.505 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.505 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:29.505 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.505 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:29.505 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620027+0000", 2024-09-21T15:00:29.505 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:29.505 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.505 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.505 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:29.505 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620027+0000", 2024-09-21T15:00:29.505 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:29.505 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.506 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.506 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:29.506 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620027+0000", 2024-09-21T15:00:29.506 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:29.506 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.506 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.506 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:29.506 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620030+0000", 2024-09-21T15:00:29.506 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0400000000000001e-06 2024-09-21T15:00:29.506 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.506 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.506 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:29.506 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620037+0000", 2024-09-21T15:00:29.506 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8220000000000001e-06 2024-09-21T15:00:29.507 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.507 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.507 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:29.507 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.620042+0000", 2024-09-21T15:00:29.507 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.843e-06 2024-09-21T15:00:29.507 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.507 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.507 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:29.507 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.629922+0000", 2024-09-21T15:00:29.507 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0098805100000000003 2024-09-21T15:00:29.507 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.507 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.507 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:29.507 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.630020+0000", 2024-09-21T15:00:29.508 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7596999999999994e-05 2024-09-21T15:00:29.508 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.508 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.508 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:29.508 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.630256+0000", 2024-09-21T15:00:29.508 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00023573299999999999 2024-09-21T15:00:29.508 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:29.508 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:29.508 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:29.508 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.508 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.508 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12260 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 57344~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:29.508 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.624587+0000", 2024-09-21T15:00:29.508 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 109.85757342399999, 2024-09-21T15:00:29.508 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0088282069999999994, 2024-09-21T15:00:29.509 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:29.509 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:29.509 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:29.509 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:29.509 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:29.509 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12260 2024-09-21T15:00:29.509 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.509 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:29.509 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.509 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:29.509 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624587+0000", 2024-09-21T15:00:29.509 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:29.509 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.509 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.509 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:29.509 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624587+0000", 2024-09-21T15:00:29.510 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:29.510 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.510 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.510 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:29.510 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624587+0000", 2024-09-21T15:00:29.510 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967296 2024-09-21T15:00:29.510 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.510 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.510 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:29.510 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624589+0000", 2024-09-21T15:00:29.510 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4339999999999999e-06 2024-09-21T15:00:29.510 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.510 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.510 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:29.510 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624593+0000", 2024-09-21T15:00:29.510 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5859999999999998e-06 2024-09-21T15:00:29.511 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.511 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.511 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:29.511 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.624596+0000", 2024-09-21T15:00:29.511 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5830000000000001e-06 2024-09-21T15:00:29.511 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.511 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.511 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:29.511 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.633132+0000", 2024-09-21T15:00:29.511 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.008536666 2024-09-21T15:00:29.511 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.511 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.511 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:29.512 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.633216+0000", 2024-09-21T15:00:29.512 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3387999999999995e-05 2024-09-21T15:00:29.512 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.512 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.512 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:29.512 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.633415+0000", 2024-09-21T15:00:29.512 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019976799999999999 2024-09-21T15:00:29.512 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:29.512 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:29.512 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:29.512 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.512 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.512 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12518 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 385024~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:29.512 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645235+0000", 2024-09-21T15:00:29.512 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 109.836925548, 2024-09-21T15:00:29.513 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0089841020000000008, 2024-09-21T15:00:29.513 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:29.513 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:29.513 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:29.513 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:29.513 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:29.513 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12518 2024-09-21T15:00:29.513 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.513 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:29.513 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.513 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:29.513 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645235+0000", 2024-09-21T15:00:29.513 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:29.513 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.514 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.514 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:29.514 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645235+0000", 2024-09-21T15:00:29.514 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:29.514 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.514 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.514 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:29.514 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645235+0000", 2024-09-21T15:00:29.514 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:29.514 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.514 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.514 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:29.514 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645238+0000", 2024-09-21T15:00:29.515 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.869e-06 2024-09-21T15:00:29.515 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.515 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.515 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:29.515 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645244+0000", 2024-09-21T15:00:29.515 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8930000000000002e-06 2024-09-21T15:00:29.515 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.515 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.515 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:29.515 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645249+0000", 2024-09-21T15:00:29.515 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2859999999999999e-06 2024-09-21T15:00:29.515 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.515 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.515 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:29.515 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.653930+0000", 2024-09-21T15:00:29.516 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0086815320000000005 2024-09-21T15:00:29.516 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.516 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.516 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:29.516 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654028+0000", 2024-09-21T15:00:29.516 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7578000000000007e-05 2024-09-21T15:00:29.516 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.516 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.516 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:29.516 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654219+0000", 2024-09-21T15:00:29.516 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019132099999999999 2024-09-21T15:00:29.516 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:29.516 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:29.516 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:29.516 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.517 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.517 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12538 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 147456~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:29.517 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645253+0000", 2024-09-21T15:00:29.517 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 109.836907924, 2024-09-21T15:00:29.517 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0092095360000000008, 2024-09-21T15:00:29.517 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:29.517 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:29.517 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:29.517 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:29.517 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:29.517 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12538 2024-09-21T15:00:29.517 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.517 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:29.518 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.518 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:29.518 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645253+0000", 2024-09-21T15:00:29.518 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:29.518 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.518 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.518 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:29.518 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645253+0000", 2024-09-21T15:00:29.518 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:29.518 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.518 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.518 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:29.518 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645252+0000", 2024-09-21T15:00:29.518 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:29.519 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.519 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.519 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:29.519 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645255+0000", 2024-09-21T15:00:29.519 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2290000000000002e-06 2024-09-21T15:00:29.519 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.519 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.519 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:29.519 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645267+0000", 2024-09-21T15:00:29.519 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2631e-05 2024-09-21T15:00:29.519 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.519 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.519 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:29.519 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645278+0000", 2024-09-21T15:00:29.519 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0847e-05 2024-09-21T15:00:29.519 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.520 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.520 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:29.520 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654251+0000", 2024-09-21T15:00:29.520 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0089731340000000007 2024-09-21T15:00:29.520 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.520 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.520 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:29.520 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654296+0000", 2024-09-21T15:00:29.520 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4353e-05 2024-09-21T15:00:29.520 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.520 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.520 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:29.520 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654462+0000", 2024-09-21T15:00:29.520 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016668499999999999 2024-09-21T15:00:29.520 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:29.521 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:29.521 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:29.521 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.521 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.521 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12539 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 253952~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:29.521 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645284+0000", 2024-09-21T15:00:29.521 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 109.836876703, 2024-09-21T15:00:29.521 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0094524029999999998, 2024-09-21T15:00:29.521 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:29.521 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:29.521 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:29.521 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:29.521 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:29.521 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12539 2024-09-21T15:00:29.522 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.522 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:29.522 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.522 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:29.522 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645284+0000", 2024-09-21T15:00:29.522 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:29.522 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.522 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.522 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:29.522 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645284+0000", 2024-09-21T15:00:29.522 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:29.522 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.522 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.522 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:29.523 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645284+0000", 2024-09-21T15:00:29.523 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:29.523 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.523 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.523 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:29.523 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645286+0000", 2024-09-21T15:00:29.523 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7159999999999999e-06 2024-09-21T15:00:29.523 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.523 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.523 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:29.523 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645292+0000", 2024-09-21T15:00:29.523 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7400000000000001e-06 2024-09-21T15:00:29.523 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.523 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.523 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:29.523 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645297+0000", 2024-09-21T15:00:29.524 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9019999999999998e-06 2024-09-21T15:00:29.524 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.524 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.524 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:29.524 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654490+0000", 2024-09-21T15:00:29.524 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0091932360000000005 2024-09-21T15:00:29.524 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.524 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.524 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:29.524 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654565+0000", 2024-09-21T15:00:29.524 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4657e-05 2024-09-21T15:00:29.524 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.524 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.524 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:29.525 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654736+0000", 2024-09-21T15:00:29.525 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017156199999999999 2024-09-21T15:00:29.525 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:29.525 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:29.525 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:29.525 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.525 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.525 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12540 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:10 [read 286720~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:29.525 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645302+0000", 2024-09-21T15:00:29.525 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 109.83685843400001, 2024-09-21T15:00:29.525 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0096603090000000006, 2024-09-21T15:00:29.525 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:29.525 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:29.525 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:29.525 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:29.526 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:29.526 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12540 2024-09-21T15:00:29.526 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.526 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:29.526 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.526 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:29.526 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645302+0000", 2024-09-21T15:00:29.526 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:29.526 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.526 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.526 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:29.526 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645302+0000", 2024-09-21T15:00:29.526 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:29.526 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.526 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.526 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:29.527 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645302+0000", 2024-09-21T15:00:29.527 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:29.527 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.527 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.527 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:29.527 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645305+0000", 2024-09-21T15:00:29.527 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7329999999999999e-06 2024-09-21T15:00:29.527 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.527 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.527 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:29.527 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645316+0000", 2024-09-21T15:00:29.527 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1832e-05 2024-09-21T15:00:29.527 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.527 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.527 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:29.528 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645322+0000", 2024-09-21T15:00:29.528 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.169e-06 2024-09-21T15:00:29.528 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.528 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.528 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:29.528 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654778+0000", 2024-09-21T15:00:29.528 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0094559090000000002 2024-09-21T15:00:29.528 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.528 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.528 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:29.528 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654828+0000", 2024-09-21T15:00:29.528 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0461999999999999e-05 2024-09-21T15:00:29.528 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.528 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.528 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:29.528 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654963+0000", 2024-09-21T15:00:29.529 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000134583 2024-09-21T15:00:29.529 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:29.529 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:29.529 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:29.529 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.529 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.529 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:12561 3.4 3:20456089:::rbd_data.11363a21d2be.000000000000002e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-21T15:00:29.529 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:39.645742+0000", 2024-09-21T15:00:29.529 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 109.83641916800001, 2024-09-21T15:00:29.529 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0093583810000000007, 2024-09-21T15:00:29.529 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:29.529 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:29.529 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:29.529 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:29.529 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:29.529 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 12561 2024-09-21T15:00:29.530 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.530 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:29.530 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.530 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:29.530 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645742+0000", 2024-09-21T15:00:29.530 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:29.530 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.530 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.530 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:29.530 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645742+0000", 2024-09-21T15:00:29.530 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:29.530 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.530 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.530 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:29.531 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645741+0000", 2024-09-21T15:00:29.531 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:00:29.531 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.531 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.531 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:29.531 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645746+0000", 2024-09-21T15:00:29.531 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8379999999999998e-06 2024-09-21T15:00:29.531 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.531 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.531 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:29.531 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645757+0000", 2024-09-21T15:00:29.531 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0705e-05 2024-09-21T15:00:29.531 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.531 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.531 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:29.532 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.645765+0000", 2024-09-21T15:00:29.532 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6729999999999992e-06 2024-09-21T15:00:29.532 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.532 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.532 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:29.532 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.654993+0000", 2024-09-21T15:00:29.532 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0092287480000000002 2024-09-21T15:00:29.532 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.532 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.532 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:29.532 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.655050+0000", 2024-09-21T15:00:29.532 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6394999999999998e-05 2024-09-21T15:00:29.532 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.532 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.533 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:29.533 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:39.655100+0000", 2024-09-21T15:00:29.533 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0049000000000001e-05 2024-09-21T15:00:29.533 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:29.533 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:29.533 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:29.533 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.533 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.533 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:97619 3.1 e122/93 3:8730dd9c:::rbd_data.1136bfc79a70.0000000000000000:head v 122'167, mlcod=120'165)", 2024-09-21T15:00:29.533 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.292926+0000", 2024-09-21T15:00:29.533 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 24.189235100000001, 2024-09-21T15:00:29.533 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.020236437999999999, 2024-09-21T15:00:29.533 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:29.533 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:29.533 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:29.534 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.534 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:29.534 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.292926+0000", 2024-09-21T15:00:29.534 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:29.534 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.534 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.534 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:29.534 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.292926+0000", 2024-09-21T15:00:29.534 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:29.534 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.534 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.534 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:29.534 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.292925+0000", 2024-09-21T15:00:29.534 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:29.535 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.535 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.535 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:29.535 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.300394+0000", 2024-09-21T15:00:29.535 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0074683839999999998 2024-09-21T15:00:29.535 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.535 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.535 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:29.535 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.300420+0000", 2024-09-21T15:00:29.535 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6380000000000002e-05 2024-09-21T15:00:29.535 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.535 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.535 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:29.535 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.300432+0000", 2024-09-21T15:00:29.535 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1987999999999999e-05 2024-09-21T15:00:29.536 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.536 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.536 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:29.536 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.300533+0000", 2024-09-21T15:00:29.536 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000100883 2024-09-21T15:00:29.536 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.536 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.536 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:29.536 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.300562+0000", 2024-09-21T15:00:29.536 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9048999999999999e-05 2024-09-21T15:00:29.536 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.536 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.536 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:29.536 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.313121+0000", 2024-09-21T15:00:29.536 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012559038999999999 2024-09-21T15:00:29.536 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.537 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.537 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:29.537 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.313162+0000", 2024-09-21T15:00:29.537 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0998999999999998e-05 2024-09-21T15:00:29.537 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:29.537 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:29.537 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:29.537 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.537 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.537 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:97677 3.15 e122/24 3:a8e72133:::rbd_data.1136bfc79a70.0000000000000005:head v 122'567, mlcod=120'565)", 2024-09-21T15:00:29.537 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.320753+0000", 2024-09-21T15:00:29.537 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 24.161407512, 2024-09-21T15:00:29.537 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070639375000000004, 2024-09-21T15:00:29.537 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:29.537 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:29.537 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:29.537 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.538 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:29.538 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.320753+0000", 2024-09-21T15:00:29.538 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:29.538 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.538 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.538 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:29.538 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.320753+0000", 2024-09-21T15:00:29.538 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:29.538 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.538 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.538 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:29.538 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.320753+0000", 2024-09-21T15:00:29.538 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:29.538 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.539 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.539 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:29.539 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.375452+0000", 2024-09-21T15:00:29.539 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054698945999999998 2024-09-21T15:00:29.539 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.539 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.539 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:29.539 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.375476+0000", 2024-09-21T15:00:29.539 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4380999999999999e-05 2024-09-21T15:00:29.539 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.539 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.539 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:29.539 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.375485+0000", 2024-09-21T15:00:29.539 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0049999999999993e-06 2024-09-21T15:00:29.540 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.540 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.540 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:29.540 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.375552+0000", 2024-09-21T15:00:29.540 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6550999999999994e-05 2024-09-21T15:00:29.540 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.540 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.540 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:29.540 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.375578+0000", 2024-09-21T15:00:29.540 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6196000000000001e-05 2024-09-21T15:00:29.540 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.540 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.540 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:29.540 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.391319+0000", 2024-09-21T15:00:29.540 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015741271000000001 2024-09-21T15:00:29.541 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.541 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.541 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:29.541 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.391393+0000", 2024-09-21T15:00:29.541 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3287000000000005e-05 2024-09-21T15:00:29.541 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:29.541 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:29.541 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:29.541 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.541 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.541 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:97672 3.4 e122/51 3:26046d29:::rbd_data.1136bfc79a70.0000000000000001:head v 122'256, mlcod=120'254)", 2024-09-21T15:00:29.541 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.331696+0000", 2024-09-21T15:00:29.541 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 24.150464349, 2024-09-21T15:00:29.541 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041222507999999998, 2024-09-21T15:00:29.541 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:29.542 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:29.542 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:29.542 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.542 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:29.542 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.331696+0000", 2024-09-21T15:00:29.542 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:29.542 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.542 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.542 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:29.542 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.331696+0000", 2024-09-21T15:00:29.542 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:29.542 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.542 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.542 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:29.542 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.331695+0000", 2024-09-21T15:00:29.543 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:00:29.543 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.543 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.543 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:29.543 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.340175+0000", 2024-09-21T15:00:29.543 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0084799520000000007 2024-09-21T15:00:29.543 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.543 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.543 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:29.543 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.340208+0000", 2024-09-21T15:00:29.543 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2997999999999998e-05 2024-09-21T15:00:29.543 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.543 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.543 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:29.543 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.340223+0000", 2024-09-21T15:00:29.544 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5330999999999999e-05 2024-09-21T15:00:29.544 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.544 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.544 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:29.544 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.340305+0000", 2024-09-21T15:00:29.544 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2095999999999995e-05 2024-09-21T15:00:29.544 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.544 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.544 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:29.544 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.340337+0000", 2024-09-21T15:00:29.544 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2097000000000001e-05 2024-09-21T15:00:29.544 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.544 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.544 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:29.545 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.372863+0000", 2024-09-21T15:00:29.545 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.032525923999999998 2024-09-21T15:00:29.545 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.545 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.545 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:29.545 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.372919+0000", 2024-09-21T15:00:29.545 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5729000000000001e-05 2024-09-21T15:00:29.545 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:29.545 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:29.545 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:29.545 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.545 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.545 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:97675 3.b e122/24 3:d321acb8:::rbd_data.1136bfc79a70.0000000000000003:head v 122'323, mlcod=120'321)", 2024-09-21T15:00:29.545 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.341399+0000", 2024-09-21T15:00:29.545 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 24.140761276999999, 2024-09-21T15:00:29.546 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.027674555999999999, 2024-09-21T15:00:29.546 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:29.546 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:29.546 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:29.546 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.546 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:29.546 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.341399+0000", 2024-09-21T15:00:29.546 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:29.546 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.546 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.546 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:29.546 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.341399+0000", 2024-09-21T15:00:29.546 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:29.546 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.547 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.547 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:29.547 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.341398+0000", 2024-09-21T15:00:29.547 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:00:29.547 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.547 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.547 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:29.547 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.350707+0000", 2024-09-21T15:00:29.547 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0093094920000000008 2024-09-21T15:00:29.547 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.547 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.547 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:29.547 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.350730+0000", 2024-09-21T15:00:29.548 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3127999999999999e-05 2024-09-21T15:00:29.548 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.548 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.548 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:29.548 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.350741+0000", 2024-09-21T15:00:29.548 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0447e-05 2024-09-21T15:00:29.548 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.548 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.548 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:29.548 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.351206+0000", 2024-09-21T15:00:29.548 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000465439 2024-09-21T15:00:29.548 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.548 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.548 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:29.549 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.351245+0000", 2024-09-21T15:00:29.549 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8634999999999999e-05 2024-09-21T15:00:29.549 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.549 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.549 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:29.549 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.368992+0000", 2024-09-21T15:00:29.549 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.017747161000000001 2024-09-21T15:00:29.549 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.549 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.549 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:29.549 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.369074+0000", 2024-09-21T15:00:29.549 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1893999999999999e-05 2024-09-21T15:00:29.549 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:29.549 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:29.550 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:29.550 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.550 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.550 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:97733 3.4 e122/51 3:23762450:::rbd_data.1136bfc79a70.0000000000000008:head v 122'257, mlcod=120'254)", 2024-09-21T15:00:29.550 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.380585+0000", 2024-09-21T15:00:29.550 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 24.101575237999999, 2024-09-21T15:00:29.550 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.023704392000000001, 2024-09-21T15:00:29.550 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:29.550 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:29.550 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:29.550 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.550 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:29.550 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.380585+0000", 2024-09-21T15:00:29.550 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:29.551 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.551 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.551 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:29.551 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.380585+0000", 2024-09-21T15:00:29.551 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:29.551 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.551 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.551 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:29.551 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.380583+0000", 2024-09-21T15:00:29.551 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:00:29.551 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.551 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.551 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:29.551 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.386771+0000", 2024-09-21T15:00:29.551 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0061877160000000002 2024-09-21T15:00:29.552 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.552 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.552 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:29.552 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.386787+0000", 2024-09-21T15:00:29.552 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5596000000000001e-05 2024-09-21T15:00:29.552 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.552 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.552 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:29.552 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.386797+0000", 2024-09-21T15:00:29.552 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0316000000000001e-05 2024-09-21T15:00:29.552 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.552 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.552 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:29.552 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.386889+0000", 2024-09-21T15:00:29.553 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2335000000000003e-05 2024-09-21T15:00:29.553 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.553 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.553 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:29.553 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.386914+0000", 2024-09-21T15:00:29.553 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.499e-05 2024-09-21T15:00:29.553 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.553 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.553 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:29.553 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.404248+0000", 2024-09-21T15:00:29.553 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.017333237000000001 2024-09-21T15:00:29.553 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.553 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.553 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:29.553 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.404290+0000", 2024-09-21T15:00:29.554 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2327000000000003e-05 2024-09-21T15:00:29.554 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:29.554 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:29.554 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:29.554 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.554 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.554 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:104955 3.1 3:85de63d1:::rbd_data.1136d175325b.0000000000000033:head [write 0~32768 [fadvise_sequential+fadvise_nocache],write 65536~229376 [fadvise_sequential+fadvise_nocache],write 327680~229376 [fadvise_sequential+fadvise_nocache],write 589824~229376 [fadvise_sequential+fadvise_nocache],write 851968~229376 [fadvise_sequential+fadvise_nocache],write 1114112~229376 [fadvise_sequential+fadvise_nocache],write 1376256~229376 [fadvise_sequential+fadvise_nocache],write 1638400~229376 [fadvise_sequential+fadvise_nocache],write 1900544~196608 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-21T15:00:29.554 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:14.515375+0000", 2024-09-21T15:00:29.554 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.966786183, 2024-09-21T15:00:29.554 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0095863219999999996, 2024-09-21T15:00:29.554 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:29.554 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:29.554 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:29.554 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:29.555 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:29.555 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 104955 2024-09-21T15:00:29.555 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.555 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:29.555 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.555 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:29.555 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:14.515375+0000", 2024-09-21T15:00:29.555 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:29.555 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.555 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.555 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:29.555 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:14.515375+0000", 2024-09-21T15:00:29.555 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:29.555 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.556 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.556 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:29.556 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:14.515374+0000", 2024-09-21T15:00:29.556 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:29.556 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.556 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.556 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:29.556 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:14.517138+0000", 2024-09-21T15:00:29.556 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0017643100000000001 2024-09-21T15:00:29.556 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.556 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.556 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:29.556 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:14.517151+0000", 2024-09-21T15:00:29.556 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3329999999999999e-05 2024-09-21T15:00:29.557 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.557 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.557 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:29.557 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:14.517160+0000", 2024-09-21T15:00:29.557 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6610000000000006e-06 2024-09-21T15:00:29.557 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.557 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.557 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:29.557 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:14.518305+0000", 2024-09-21T15:00:29.557 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001145189 2024-09-21T15:00:29.557 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.557 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.557 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:29.557 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:14.518433+0000", 2024-09-21T15:00:29.557 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000128045 2024-09-21T15:00:29.558 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.558 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.558 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-21T15:00:29.558 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:14.518627+0000", 2024-09-21T15:00:29.558 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019324300000000001 2024-09-21T15:00:29.558 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.558 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.558 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:29.558 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:14.522215+0000", 2024-09-21T15:00:29.558 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0035883740000000001 2024-09-21T15:00:29.558 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.558 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.558 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:29.559 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:14.524843+0000", 2024-09-21T15:00:29.559 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0026280359999999998 2024-09-21T15:00:29.559 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.559 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.559 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:29.559 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:14.524917+0000", 2024-09-21T15:00:29.559 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3924000000000006e-05 2024-09-21T15:00:29.559 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.559 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.559 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:29.559 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:14.524961+0000", 2024-09-21T15:00:29.559 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3906000000000003e-05 2024-09-21T15:00:29.559 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:29.559 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:29.559 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:29.560 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.560 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.560 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:105964 3.1 3:8730dd9c:::rbd_data.1136bfc79a70.0000000000000000:head [sparse-read 0~3145728] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-21T15:00:29.560 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:15.409532+0000", 2024-09-21T15:00:29.560 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 14.072628308000001, 2024-09-21T15:00:29.560 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010457086000000001, 2024-09-21T15:00:29.560 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:29.560 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:29.560 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:29.560 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:29.560 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:29.560 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 105964 2024-09-21T15:00:29.560 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.560 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:29.560 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.561 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:29.561 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:15.409532+0000", 2024-09-21T15:00:29.561 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:29.561 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.561 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.561 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:29.561 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:15.409532+0000", 2024-09-21T15:00:29.561 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:29.561 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.561 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.561 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:29.561 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:15.409529+0000", 2024-09-21T15:00:29.561 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999971 2024-09-21T15:00:29.561 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.562 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.562 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:29.562 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:15.409545+0000", 2024-09-21T15:00:29.562 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5968999999999999e-05 2024-09-21T15:00:29.562 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.562 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.562 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:29.562 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:15.409583+0000", 2024-09-21T15:00:29.562 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7490000000000002e-05 2024-09-21T15:00:29.562 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.562 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.562 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:29.562 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:15.409600+0000", 2024-09-21T15:00:29.562 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7472000000000001e-05 2024-09-21T15:00:29.562 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.563 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.563 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:29.563 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:15.409757+0000", 2024-09-21T15:00:29.563 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015679100000000001 2024-09-21T15:00:29.563 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.563 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.563 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:29.563 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:15.409983+0000", 2024-09-21T15:00:29.563 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00022590500000000001 2024-09-21T15:00:29.563 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.563 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.563 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:29.563 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:15.419989+0000", 2024-09-21T15:00:29.563 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010006456 2024-09-21T15:00:29.563 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:29.564 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:29.564 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:29.564 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.564 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.564 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:108350 3.12 3:4e90d7f9:::rbd_data.1136d80ba95e.0000000000000141:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-21T15:00:29.564 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:20.588954+0000", 2024-09-21T15:00:29.564 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 8.893207018, 2024-09-21T15:00:29.564 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.017428454999999999, 2024-09-21T15:00:29.564 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:29.564 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:29.564 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:29.564 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:29.564 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:29.564 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 108350 2024-09-21T15:00:29.565 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.565 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:29.565 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.565 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:29.565 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:20.588954+0000", 2024-09-21T15:00:29.565 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:29.565 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.565 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.565 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:29.565 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:20.588954+0000", 2024-09-21T15:00:29.565 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:29.565 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.565 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.565 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:29.565 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:20.588951+0000", 2024-09-21T15:00:29.565 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-21T15:00:29.566 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.566 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.566 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:29.566 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:20.588967+0000", 2024-09-21T15:00:29.566 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5923999999999999e-05 2024-09-21T15:00:29.566 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.566 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.566 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:29.566 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:20.589000+0000", 2024-09-21T15:00:29.566 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2614999999999999e-05 2024-09-21T15:00:29.566 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.566 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.566 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:29.567 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:20.589014+0000", 2024-09-21T15:00:29.567 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4627000000000001e-05 2024-09-21T15:00:29.567 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.567 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.567 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:29.567 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:20.589148+0000", 2024-09-21T15:00:29.567 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000133934 2024-09-21T15:00:29.567 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.567 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.567 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:29.567 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:20.589548+0000", 2024-09-21T15:00:29.567 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00040010700000000002 2024-09-21T15:00:29.567 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.567 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.567 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 3", 2024-09-21T15:00:29.568 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:20.593837+0000", 2024-09-21T15:00:29.568 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0042885279999999998 2024-09-21T15:00:29.568 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.568 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.568 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:29.568 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:20.606168+0000", 2024-09-21T15:00:29.568 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012331482 2024-09-21T15:00:29.568 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.568 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.568 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:29.568 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:20.606198+0000", 2024-09-21T15:00:29.568 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9833e-05 2024-09-21T15:00:29.568 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.568 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.568 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:29.568 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:20.606299+0000", 2024-09-21T15:00:29.569 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000100388 2024-09-21T15:00:29.569 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.569 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.569 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:29.569 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:20.606382+0000", 2024-09-21T15:00:29.569 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3500999999999999e-05 2024-09-21T15:00:29.569 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:29.569 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:29.569 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:29.569 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.569 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.569 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112924 3.1 3:83450046:::rbd_data.1136b442e94a.0000000000000000:head [write 0~360448 [fadvise_sequential+fadvise_nocache],write 385024~573440 [fadvise_sequential+fadvise_nocache],write 1146880~573440 [fadvise_sequential+fadvise_nocache],write 1908736~188416 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:29.569 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.373102+0000", 2024-09-21T15:00:29.569 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.10905836200000001, 2024-09-21T15:00:29.569 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010965425000000001, 2024-09-21T15:00:29.570 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:29.570 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:29.570 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:29.570 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:29.570 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:29.570 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112924 2024-09-21T15:00:29.570 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.570 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:29.570 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.570 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:29.570 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.373102+0000", 2024-09-21T15:00:29.570 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:29.570 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.570 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.570 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:29.571 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.373102+0000", 2024-09-21T15:00:29.571 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:29.571 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.571 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.571 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:29.571 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.373100+0000", 2024-09-21T15:00:29.571 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-21T15:00:29.571 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.571 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.571 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:29.571 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.375945+0000", 2024-09-21T15:00:29.571 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0028451129999999998 2024-09-21T15:00:29.571 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.572 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.572 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:29.572 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.375973+0000", 2024-09-21T15:00:29.572 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.779e-05 2024-09-21T15:00:29.572 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.572 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.572 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:29.572 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.375986+0000", 2024-09-21T15:00:29.572 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3665999999999999e-05 2024-09-21T15:00:29.572 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.572 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.572 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:29.572 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.376069+0000", 2024-09-21T15:00:29.572 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3076000000000002e-05 2024-09-21T15:00:29.572 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.572 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.573 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op must wait for map", 2024-09-21T15:00:29.573 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.376078+0000", 2024-09-21T15:00:29.573 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6920000000000008e-06 2024-09-21T15:00:29.573 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.573 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.573 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:29.573 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.378274+0000", 2024-09-21T15:00:29.573 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0021954930000000002 2024-09-21T15:00:29.573 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.573 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.573 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:29.573 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.378482+0000", 2024-09-21T15:00:29.573 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00020811000000000001 2024-09-21T15:00:29.573 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.573 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.573 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 4", 2024-09-21T15:00:29.574 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.378845+0000", 2024-09-21T15:00:29.574 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00036328 2024-09-21T15:00:29.574 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.574 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.574 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:29.574 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.383269+0000", 2024-09-21T15:00:29.574 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0044236079999999999 2024-09-21T15:00:29.574 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.574 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.574 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:29.574 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.383949+0000", 2024-09-21T15:00:29.574 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00068067300000000002 2024-09-21T15:00:29.574 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.574 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.574 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:29.575 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.384021+0000", 2024-09-21T15:00:29.575 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1673000000000003e-05 2024-09-21T15:00:29.575 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:29.575 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:29.575 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:29.575 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.384068+0000", 2024-09-21T15:00:29.575 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6805999999999999e-05 2024-09-21T15:00:29.575 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:29.575 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:29.575 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:29.575 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:29.575 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:29.575 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:29.576 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T15:00:29.577 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:00:29.571+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:00:29.677 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:00:29.675+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:00:29.704 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:29.704 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:00:29.704 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T15:00:29.704 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:29.718 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T15:00:29.778 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:00:29.774+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:00:29.860 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:29.860 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:00:29.860 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T15:00:29.860 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T15:00:29.860 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:29.875 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T15:00:29.879 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:00:29.874+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:00:29.980 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:00:29.975+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:00:30.032 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:30.032 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T15:00:30.033 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T15:00:30.033 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T15:00:30.033 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.033 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31476 3.c 3:35718b6b:::rbd_data.11367721c19.0000000000000014:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T15:00:30.033 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.505975+0000", 2024-09-21T15:00:30.033 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 91.523308193000005, 2024-09-21T15:00:30.033 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99515995899999998, 2024-09-21T15:00:30.033 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:30.033 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:30.033 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:30.033 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:30.033 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:30.034 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31476 2024-09-21T15:00:30.034 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.034 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:30.034 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.034 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:30.034 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.505975+0000", 2024-09-21T15:00:30.034 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:30.034 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.034 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.034 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:30.034 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.505975+0000", 2024-09-21T15:00:30.034 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:30.034 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.035 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.035 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:30.035 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.505971+0000", 2024-09-21T15:00:30.035 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999967 2024-09-21T15:00:30.035 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.035 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.035 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:30.035 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.505989+0000", 2024-09-21T15:00:30.035 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7985000000000001e-05 2024-09-21T15:00:30.035 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.035 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.036 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:30.036 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506050+0000", 2024-09-21T15:00:30.036 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0659000000000003e-05 2024-09-21T15:00:30.036 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.036 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.036 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:30.036 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506074+0000", 2024-09-21T15:00:30.036 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4156e-05 2024-09-21T15:00:30.036 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.036 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.036 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:30.036 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506225+0000", 2024-09-21T15:00:30.036 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015097 2024-09-21T15:00:30.037 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.037 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.037 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:00:30.037 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506256+0000", 2024-09-21T15:00:30.037 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0505999999999999e-05 2024-09-21T15:00:30.037 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.037 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.037 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:30.037 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.497974+0000", 2024-09-21T15:00:30.037 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99171825700000005 2024-09-21T15:00:30.037 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.037 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.037 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for missing object", 2024-09-21T15:00:30.037 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.498449+0000", 2024-09-21T15:00:30.038 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00047506399999999998 2024-09-21T15:00:30.038 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.038 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.038 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:30.038 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.501000+0000", 2024-09-21T15:00:30.038 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0025506919999999998 2024-09-21T15:00:30.038 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.038 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.038 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:30.038 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.501135+0000", 2024-09-21T15:00:30.038 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013524000000000001 2024-09-21T15:00:30.038 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:30.038 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:30.039 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:30.039 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.039 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.039 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31480 3.c 3:36ccd808:::rbd_data.11367721c19.0000000000000018:head [sparse-read 0~14821] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T15:00:30.039 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.506116+0000", 2024-09-21T15:00:30.039 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 91.523166849999996, 2024-09-21T15:00:30.039 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99316033599999998, 2024-09-21T15:00:30.039 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:30.039 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:30.039 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:30.039 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:30.039 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:30.039 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31480 2024-09-21T15:00:30.039 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.040 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:30.040 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.040 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:30.040 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506116+0000", 2024-09-21T15:00:30.040 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:30.040 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.040 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.040 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:30.040 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506116+0000", 2024-09-21T15:00:30.040 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:30.040 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.040 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.040 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:30.040 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506115+0000", 2024-09-21T15:00:30.041 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:00:30.041 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.041 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.041 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:30.041 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506124+0000", 2024-09-21T15:00:30.041 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0659999999999993e-06 2024-09-21T15:00:30.041 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.041 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.041 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:30.041 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506145+0000", 2024-09-21T15:00:30.041 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1447e-05 2024-09-21T15:00:30.041 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.041 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.041 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:30.041 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506161+0000", 2024-09-21T15:00:30.042 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5209e-05 2024-09-21T15:00:30.042 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.042 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.042 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:30.042 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506337+0000", 2024-09-21T15:00:30.042 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017642999999999999 2024-09-21T15:00:30.042 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.042 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.042 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:00:30.042 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506375+0000", 2024-09-21T15:00:30.042 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7536000000000001e-05 2024-09-21T15:00:30.042 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.042 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.043 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:30.043 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.498815+0000", 2024-09-21T15:00:30.043 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99244067999999996 2024-09-21T15:00:30.043 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.043 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.043 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:30.043 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.498950+0000", 2024-09-21T15:00:30.043 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013436700000000001 2024-09-21T15:00:30.043 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.043 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.043 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:30.043 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499277+0000", 2024-09-21T15:00:30.043 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00032696800000000002 2024-09-21T15:00:30.043 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:30.044 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:30.044 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:30.044 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.044 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.044 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31513 3.c 3:36de8523:::rbd_data.11367721c19.0000000000000039:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T15:00:30.044 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.509071+0000", 2024-09-21T15:00:30.044 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 91.520212650000005, 2024-09-21T15:00:30.044 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99038225099999999, 2024-09-21T15:00:30.044 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:30.044 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:30.044 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:30.044 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:30.044 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:30.045 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31513 2024-09-21T15:00:30.045 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.045 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:30.045 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.045 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:30.045 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509071+0000", 2024-09-21T15:00:30.045 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:30.045 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.045 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.045 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:30.045 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509071+0000", 2024-09-21T15:00:30.045 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:30.045 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.046 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.046 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:30.046 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509068+0000", 2024-09-21T15:00:30.046 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-21T15:00:30.046 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.046 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.046 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:30.046 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509082+0000", 2024-09-21T15:00:30.046 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3545e-05 2024-09-21T15:00:30.046 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.046 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.046 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:30.046 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509116+0000", 2024-09-21T15:00:30.046 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3942000000000001e-05 2024-09-21T15:00:30.047 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.047 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.047 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:30.047 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509134+0000", 2024-09-21T15:00:30.047 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8839999999999999e-05 2024-09-21T15:00:30.047 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.047 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.047 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:30.047 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509318+0000", 2024-09-21T15:00:30.047 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018326799999999999 2024-09-21T15:00:30.047 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.047 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.048 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:00:30.048 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509346+0000", 2024-09-21T15:00:30.048 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8019000000000001e-05 2024-09-21T15:00:30.048 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.048 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.048 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:30.048 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499330+0000", 2024-09-21T15:00:30.048 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.98998399299999995 2024-09-21T15:00:30.048 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.048 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.048 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:30.048 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499453+0000", 2024-09-21T15:00:30.048 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000123115 2024-09-21T15:00:30.048 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:30.049 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:30.049 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:30.049 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.049 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.049 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31515 3.c 3:35b36ddd:::rbd_data.11367721c19.000000000000003b:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T15:00:30.049 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.509177+0000", 2024-09-21T15:00:30.049 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 91.520106224000003, 2024-09-21T15:00:30.049 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99044870600000001, 2024-09-21T15:00:30.049 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:30.049 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:30.049 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:30.049 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:30.049 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:30.049 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31515 2024-09-21T15:00:30.049 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.050 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:30.050 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.050 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:30.050 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509177+0000", 2024-09-21T15:00:30.050 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:30.050 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.050 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.050 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:30.050 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509177+0000", 2024-09-21T15:00:30.050 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:30.050 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.050 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.050 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:30.051 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509176+0000", 2024-09-21T15:00:30.051 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:00:30.051 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.051 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.051 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:30.051 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509183+0000", 2024-09-21T15:00:30.051 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6720000000000004e-06 2024-09-21T15:00:30.051 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.051 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.051 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:30.051 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509269+0000", 2024-09-21T15:00:30.051 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5881e-05 2024-09-21T15:00:30.051 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.051 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.052 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:30.052 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509303+0000", 2024-09-21T15:00:30.052 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3367000000000001e-05 2024-09-21T15:00:30.052 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.052 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.052 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:30.052 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509425+0000", 2024-09-21T15:00:30.052 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000122199 2024-09-21T15:00:30.052 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.052 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.052 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:00:30.052 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509448+0000", 2024-09-21T15:00:30.052 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2688999999999999e-05 2024-09-21T15:00:30.052 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.053 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.053 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:30.053 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499501+0000", 2024-09-21T15:00:30.053 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99005306999999998 2024-09-21T15:00:30.053 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.053 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.053 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:30.053 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499626+0000", 2024-09-21T15:00:30.053 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012501400000000001 2024-09-21T15:00:30.053 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:30.053 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:30.053 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:30.053 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.054 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.054 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31531 3.c 3:314f0287:::rbd_data.11367721c19.000000000000004b:head [sparse-read 0~7106] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T15:00:30.054 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.509329+0000", 2024-09-21T15:00:30.054 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 91.519954204000001, 2024-09-21T15:00:30.054 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99103085300000004, 2024-09-21T15:00:30.054 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:30.054 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:30.054 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:30.054 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:30.054 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:30.054 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31531 2024-09-21T15:00:30.054 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.054 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:30.055 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.055 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:30.055 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509329+0000", 2024-09-21T15:00:30.055 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:30.055 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.055 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.055 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:30.055 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509329+0000", 2024-09-21T15:00:30.055 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:30.055 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.055 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.055 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:30.055 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509328+0000", 2024-09-21T15:00:30.055 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:00:30.056 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.056 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.056 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:30.056 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509336+0000", 2024-09-21T15:00:30.056 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3170000000000002e-06 2024-09-21T15:00:30.056 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.056 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.056 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:30.056 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509353+0000", 2024-09-21T15:00:30.056 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7214000000000002e-05 2024-09-21T15:00:30.056 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.056 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.056 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:30.056 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509366+0000", 2024-09-21T15:00:30.056 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.26e-05 2024-09-21T15:00:30.057 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.057 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.057 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:30.057 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509523+0000", 2024-09-21T15:00:30.057 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015708000000000001 2024-09-21T15:00:30.057 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.057 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.057 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:00:30.057 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509544+0000", 2024-09-21T15:00:30.057 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0783000000000001e-05 2024-09-21T15:00:30.057 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.057 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.057 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:30.057 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499727+0000", 2024-09-21T15:00:30.057 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99018345200000002 2024-09-21T15:00:30.058 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.058 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.058 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:30.058 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499949+0000", 2024-09-21T15:00:30.058 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000221358 2024-09-21T15:00:30.058 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.058 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.058 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:30.058 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.500360+0000", 2024-09-21T15:00:30.058 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00041130700000000002 2024-09-21T15:00:30.058 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:30.058 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:30.058 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:30.058 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.059 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.059 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:61580 3.b 3:d3369e4f:::rbd_data.1136862f872f.0000000000000010:head [write 0~557056 [fadvise_sequential+fadvise_nocache],zero 557056~16384,write 573440~49152 [fadvise_sequential+fadvise_nocache],zero 622592~16384,write 638976~49152 [fadvise_sequential+fadvise_nocache],zero 688128~16384,write 704512~49152 [fadvise_sequential+fadvise_nocache],zero 753664~16384,write 770048~49152 [fadvise_sequential+fadvise_nocache],zero 819200~16384,write 835584~49152 [fadvise_sequential+fadvise_nocache],zero 884736~16384,write 901120~49152 [fadvise_sequential+fadvise_nocache],zero 950272~16384,write 966656~49152 [fadvise_sequential+fadvise_nocache],zero 1015808~16384,write 1032192~16384 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-21T15:00:30.059 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:19.231716+0000", 2024-09-21T15:00:30.059 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 70.797567041999997, 2024-09-21T15:00:30.059 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0096097690000000006, 2024-09-21T15:00:30.059 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:30.059 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:30.059 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:30.059 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:30.059 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:30.059 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 61580 2024-09-21T15:00:30.059 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.059 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:30.059 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.059 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:30.060 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.231716+0000", 2024-09-21T15:00:30.060 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:30.060 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.060 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.060 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:30.060 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.231716+0000", 2024-09-21T15:00:30.060 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:30.060 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.060 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.060 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:30.060 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.231715+0000", 2024-09-21T15:00:30.060 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:00:30.060 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.060 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.061 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:30.061 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.235290+0000", 2024-09-21T15:00:30.061 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0035750399999999998 2024-09-21T15:00:30.061 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.061 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.061 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:30.061 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.235310+0000", 2024-09-21T15:00:30.061 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0120999999999999e-05 2024-09-21T15:00:30.061 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.061 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.061 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:30.061 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.235319+0000", 2024-09-21T15:00:30.061 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2420000000000006e-06 2024-09-21T15:00:30.061 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.061 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.062 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:30.062 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.235416+0000", 2024-09-21T15:00:30.062 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7204e-05 2024-09-21T15:00:30.062 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.062 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.062 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:30.062 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.235775+0000", 2024-09-21T15:00:30.062 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000358883 2024-09-21T15:00:30.062 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.062 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.062 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T15:00:30.063 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.236560+0000", 2024-09-21T15:00:30.063 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000784463 2024-09-21T15:00:30.063 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.063 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.063 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:30.063 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.241119+0000", 2024-09-21T15:00:30.063 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0045590589999999999 2024-09-21T15:00:30.063 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.063 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.063 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:30.063 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.241210+0000", 2024-09-21T15:00:30.063 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1617000000000001e-05 2024-09-21T15:00:30.063 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.063 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.064 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:30.064 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.241285+0000", 2024-09-21T15:00:30.064 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4470999999999999e-05 2024-09-21T15:00:30.064 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.064 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.064 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:30.064 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.241326+0000", 2024-09-21T15:00:30.064 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0967000000000003e-05 2024-09-21T15:00:30.064 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:30.064 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:30.064 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:30.064 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.064 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.064 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:73699 3.b 3:d293b54a:::rbd_data.1136f4939602.000000000000000a:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-21T15:00:30.065 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:41.971359+0000", 2024-09-21T15:00:30.065 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 48.057924485999997, 2024-09-21T15:00:30.065 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0082840980000000002, 2024-09-21T15:00:30.065 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:30.065 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:30.065 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:30.065 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:30.065 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:30.065 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 73699 2024-09-21T15:00:30.065 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.065 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:30.065 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.065 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:30.065 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.971359+0000", 2024-09-21T15:00:30.066 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:30.066 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.066 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.066 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:30.066 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.971359+0000", 2024-09-21T15:00:30.066 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:30.066 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.066 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.066 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:30.066 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.971356+0000", 2024-09-21T15:00:30.066 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-21T15:00:30.066 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.066 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.066 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:30.067 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.971370+0000", 2024-09-21T15:00:30.067 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3349999999999999e-05 2024-09-21T15:00:30.067 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.067 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.067 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:30.067 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.971393+0000", 2024-09-21T15:00:30.067 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3799999999999999e-05 2024-09-21T15:00:30.067 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.067 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.067 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:30.067 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.971405+0000", 2024-09-21T15:00:30.067 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1569000000000001e-05 2024-09-21T15:00:30.067 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.067 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.068 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:30.068 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.971510+0000", 2024-09-21T15:00:30.068 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010456899999999999 2024-09-21T15:00:30.068 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.068 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.068 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:30.068 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.971752+0000", 2024-09-21T15:00:30.068 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00024250300000000001 2024-09-21T15:00:30.068 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.068 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.068 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T15:00:30.068 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.973012+0000", 2024-09-21T15:00:30.068 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0012595429999999999 2024-09-21T15:00:30.068 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.069 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.069 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:30.069 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.979357+0000", 2024-09-21T15:00:30.069 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0063454469999999997 2024-09-21T15:00:30.069 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.069 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.069 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:30.069 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.979467+0000", 2024-09-21T15:00:30.069 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000109623 2024-09-21T15:00:30.069 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.069 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.069 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:30.069 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.979585+0000", 2024-09-21T15:00:30.070 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000118044 2024-09-21T15:00:30.070 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.070 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.070 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:30.070 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.979643+0000", 2024-09-21T15:00:30.070 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8069999999999998e-05 2024-09-21T15:00:30.070 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:30.070 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:30.070 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:30.070 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.070 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.070 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:82545 3.b 3:d4eb1ee5:::rbd_data.11368296f791.0000000000000000:head [write 0~671744 [fadvise_sequential+fadvise_nocache],write 704512~65536 [fadvise_sequential+fadvise_nocache],write 786432~49152 [fadvise_sequential+fadvise_nocache],write 868352~49152 [fadvise_sequential+fadvise_nocache],write 933888~65536 [fadvise_sequential+fadvise_nocache],write 1032192~32768 [fadvise_sequential+fadvise_nocache],write 1097728~65536 [fadvise_sequential+fadvise_nocache],write 1179648~49152 [fadvise_sequential+fadvise_nocache],write 1261568~49152 [fadvise_sequential+fadvise_nocache],write 1327104~65536 [fadvise_sequential+fadvise_nocache],write 1425408~32768 [fadvise_sequential+fadvise_nocache],write 1490944~65536 [fadvise_sequential+fadvise_nocache],write 1572864~8192 [fadvise_sequential+fadvise_nocache],write 1630208~49152 [fadvise_sequential+fadvise_nocache],write 1703936~40960 [fadvise_sequential+fadvise_nocache],write 1794048~40960 [fadvise_sequential+fadvise_nocache],write 1859584~49152 [fadvise_sequential+fadvise_nocache],write 1957888~8192 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:00:30.071 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:47.416089+0000", 2024-09-21T15:00:30.071 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.613194479000001, 2024-09-21T15:00:30.071 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011896455, 2024-09-21T15:00:30.071 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:30.071 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:30.071 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:30.071 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:30.071 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:30.071 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82545 2024-09-21T15:00:30.071 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.071 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:30.071 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.071 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:30.071 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.416089+0000", 2024-09-21T15:00:30.071 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:30.072 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.072 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.072 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:30.072 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.416089+0000", 2024-09-21T15:00:30.072 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:30.072 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.072 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.072 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:30.072 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.416087+0000", 2024-09-21T15:00:30.072 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:00:30.072 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.072 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.072 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:30.072 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.419643+0000", 2024-09-21T15:00:30.072 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0035555550000000002 2024-09-21T15:00:30.073 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.073 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.073 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:30.073 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.419663+0000", 2024-09-21T15:00:30.073 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0202000000000001e-05 2024-09-21T15:00:30.073 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.073 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.073 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:30.073 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.419676+0000", 2024-09-21T15:00:30.073 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3207000000000001e-05 2024-09-21T15:00:30.073 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.073 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.073 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:30.074 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.419759+0000", 2024-09-21T15:00:30.074 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3521000000000002e-05 2024-09-21T15:00:30.074 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.074 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.074 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op must wait for map", 2024-09-21T15:00:30.074 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.419768+0000", 2024-09-21T15:00:30.074 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4479999999999994e-06 2024-09-21T15:00:30.074 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.074 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.074 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:30.074 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.422171+0000", 2024-09-21T15:00:30.074 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0024035469999999998 2024-09-21T15:00:30.074 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.075 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.075 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:30.075 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.422350+0000", 2024-09-21T15:00:30.075 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000179001 2024-09-21T15:00:30.075 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.075 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.075 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T15:00:30.075 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.422710+0000", 2024-09-21T15:00:30.075 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000359878 2024-09-21T15:00:30.075 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.075 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.075 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:30.075 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.426676+0000", 2024-09-21T15:00:30.075 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0039655790000000003 2024-09-21T15:00:30.075 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.075 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.076 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:30.076 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.427884+0000", 2024-09-21T15:00:30.076 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001208207 2024-09-21T15:00:30.076 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.076 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.076 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:30.076 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.427953+0000", 2024-09-21T15:00:30.076 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8533000000000003e-05 2024-09-21T15:00:30.076 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.076 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.076 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:30.076 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.427985+0000", 2024-09-21T15:00:30.076 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2481999999999998e-05 2024-09-21T15:00:30.076 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:30.076 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:30.076 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:30.077 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.077 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.077 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:83735 3.b 3:d3640016:::rbd_data.11368296f791.0000000000000006:40 [sparse-read 393216~131072 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-21T15:00:30.077 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:51.156767+0000", 2024-09-21T15:00:30.077 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 38.872515856, 2024-09-21T15:00:30.077 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0082826410000000003, 2024-09-21T15:00:30.077 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:30.077 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:30.077 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:30.077 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:30.077 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:30.077 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 83735 2024-09-21T15:00:30.077 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.077 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:30.078 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.078 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:30.078 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156767+0000", 2024-09-21T15:00:30.078 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:30.078 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.078 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.078 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:30.078 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156767+0000", 2024-09-21T15:00:30.078 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:30.078 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.078 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.078 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:30.078 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156767+0000", 2024-09-21T15:00:30.078 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:30.079 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.079 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.079 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:30.079 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156771+0000", 2024-09-21T15:00:30.079 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0940000000000001e-06 2024-09-21T15:00:30.079 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.079 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.079 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:30.079 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156781+0000", 2024-09-21T15:00:30.079 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0267e-05 2024-09-21T15:00:30.079 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.079 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.079 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:30.079 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156789+0000", 2024-09-21T15:00:30.079 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7659999999999998e-06 2024-09-21T15:00:30.080 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.080 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.080 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:30.080 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.164742+0000", 2024-09-21T15:00:30.080 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0079534519999999997 2024-09-21T15:00:30.080 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.080 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.080 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:30.080 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.164820+0000", 2024-09-21T15:00:30.080 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7927000000000001e-05 2024-09-21T15:00:30.080 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.080 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.080 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:30.080 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165050+0000", 2024-09-21T15:00:30.080 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000229731 2024-09-21T15:00:30.080 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:30.081 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:30.081 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:30.081 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.081 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.081 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:83736 3.b 3:d3640016:::rbd_data.11368296f791.0000000000000006:40 [sparse-read 1835008~0 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-21T15:00:30.081 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:51.156796+0000", 2024-09-21T15:00:30.081 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 38.872486852999998, 2024-09-21T15:00:30.081 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0085139110000000007, 2024-09-21T15:00:30.081 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:30.081 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:30.081 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:30.081 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:30.081 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:30.081 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 83736 2024-09-21T15:00:30.082 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.082 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:30.082 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.082 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:30.082 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156796+0000", 2024-09-21T15:00:30.082 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:30.082 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.082 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.082 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:30.082 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156796+0000", 2024-09-21T15:00:30.082 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:30.082 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.082 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.082 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:30.083 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156796+0000", 2024-09-21T15:00:30.083 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:30.083 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.083 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.083 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:30.083 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156800+0000", 2024-09-21T15:00:30.083 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.34e-06 2024-09-21T15:00:30.083 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.083 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.083 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:30.083 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156813+0000", 2024-09-21T15:00:30.083 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2532e-05 2024-09-21T15:00:30.083 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.084 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.084 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:30.084 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156821+0000", 2024-09-21T15:00:30.084 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2069999999999995e-06 2024-09-21T15:00:30.084 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.084 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.084 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:30.084 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165100+0000", 2024-09-21T15:00:30.084 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0082788730000000008 2024-09-21T15:00:30.084 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.084 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.084 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:30.084 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165192+0000", 2024-09-21T15:00:30.085 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2095999999999994e-05 2024-09-21T15:00:30.085 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.085 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.085 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:30.085 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165310+0000", 2024-09-21T15:00:30.085 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000118461 2024-09-21T15:00:30.085 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:30.085 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:30.085 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:30.085 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.085 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.085 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:83739 3.b 3:d4eb1ee5:::rbd_data.11368296f791.0000000000000000:40 [sparse-read 524288~131072 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-21T15:00:30.085 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:51.156827+0000", 2024-09-21T15:00:30.086 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 38.872455770000002, 2024-09-21T15:00:30.086 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0087319900000000002, 2024-09-21T15:00:30.086 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:30.086 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:30.086 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:30.086 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:30.086 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:30.086 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 83739 2024-09-21T15:00:30.086 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.086 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:30.086 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.086 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:30.086 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156827+0000", 2024-09-21T15:00:30.086 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:30.087 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.087 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.087 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:30.087 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156827+0000", 2024-09-21T15:00:30.087 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:30.087 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.087 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.087 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:30.087 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156827+0000", 2024-09-21T15:00:30.087 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:30.087 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.087 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.088 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:30.088 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156831+0000", 2024-09-21T15:00:30.088 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9940000000000002e-06 2024-09-21T15:00:30.088 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.088 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.088 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:30.088 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156841+0000", 2024-09-21T15:00:30.088 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6760000000000002e-06 2024-09-21T15:00:30.088 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.088 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.088 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:30.088 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156853+0000", 2024-09-21T15:00:30.088 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1972e-05 2024-09-21T15:00:30.089 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.089 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.089 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:30.089 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165334+0000", 2024-09-21T15:00:30.089 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0084816939999999997 2024-09-21T15:00:30.089 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.089 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.089 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:30.089 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165375+0000", 2024-09-21T15:00:30.089 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0911000000000002e-05 2024-09-21T15:00:30.089 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.089 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.090 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:30.090 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165559+0000", 2024-09-21T15:00:30.090 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018429300000000001 2024-09-21T15:00:30.090 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:30.090 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:30.090 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:30.090 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.090 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.090 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:83740 3.b 3:d4eb1ee5:::rbd_data.11368296f791.0000000000000000:40 [sparse-read 1966080~0 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-21T15:00:30.090 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:51.156859+0000", 2024-09-21T15:00:30.090 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 38.872423779999998, 2024-09-21T15:00:30.090 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0088540089999999995, 2024-09-21T15:00:30.091 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:30.091 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:30.091 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:30.091 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:30.091 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:30.091 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 83740 2024-09-21T15:00:30.091 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.091 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:30.091 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.091 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:30.091 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156859+0000", 2024-09-21T15:00:30.091 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:30.091 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.092 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.092 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:30.092 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156859+0000", 2024-09-21T15:00:30.092 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:30.092 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.092 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.092 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:30.092 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156859+0000", 2024-09-21T15:00:30.092 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:30.092 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.092 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.092 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:30.092 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156863+0000", 2024-09-21T15:00:30.092 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.844e-06 2024-09-21T15:00:30.093 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.093 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.093 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:30.093 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156873+0000", 2024-09-21T15:00:30.093 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0069e-05 2024-09-21T15:00:30.093 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.093 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.093 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:30.093 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156880+0000", 2024-09-21T15:00:30.093 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4240000000000004e-06 2024-09-21T15:00:30.093 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.093 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.093 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:30.094 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165590+0000", 2024-09-21T15:00:30.094 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0087098100000000001 2024-09-21T15:00:30.094 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.094 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.094 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:30.094 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165656+0000", 2024-09-21T15:00:30.094 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5541000000000001e-05 2024-09-21T15:00:30.094 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.094 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.094 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:30.094 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165713+0000", 2024-09-21T15:00:30.094 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7865000000000002e-05 2024-09-21T15:00:30.095 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:30.095 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:30.095 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:30.095 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.095 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.095 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:83751 3.b 3:d3640016:::rbd_data.11368296f791.0000000000000006:40 [sparse-read 1179648~0 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-21T15:00:30.095 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:51.156888+0000", 2024-09-21T15:00:30.095 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 38.872395519000001, 2024-09-21T15:00:30.095 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0090039539999999998, 2024-09-21T15:00:30.095 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:30.095 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:30.095 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:30.095 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:30.095 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:30.095 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 83751 2024-09-21T15:00:30.096 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.096 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:30.096 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.096 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:30.096 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156888+0000", 2024-09-21T15:00:30.096 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:30.096 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.096 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.096 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:30.096 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156888+0000", 2024-09-21T15:00:30.096 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:30.096 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.096 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.096 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:30.097 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156887+0000", 2024-09-21T15:00:30.097 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:30.097 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.097 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.097 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:30.097 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156891+0000", 2024-09-21T15:00:30.097 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.518e-06 2024-09-21T15:00:30.097 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.097 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.097 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:30.097 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156900+0000", 2024-09-21T15:00:30.097 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1759999999999999e-06 2024-09-21T15:00:30.097 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.097 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.098 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:30.098 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156908+0000", 2024-09-21T15:00:30.098 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8439999999999998e-06 2024-09-21T15:00:30.098 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.098 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.098 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:30.098 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165738+0000", 2024-09-21T15:00:30.098 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0088305689999999999 2024-09-21T15:00:30.098 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.098 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.098 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:30.098 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165804+0000", 2024-09-21T15:00:30.098 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5535000000000002e-05 2024-09-21T15:00:30.098 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.099 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.099 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:30.099 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165892+0000", 2024-09-21T15:00:30.099 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7860000000000002e-05 2024-09-21T15:00:30.099 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:30.099 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:30.099 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:30.099 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.099 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.099 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:83752 3.b 3:d3640016:::rbd_data.11368296f791.0000000000000006:40 [sparse-read 2621440~0 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-21T15:00:30.099 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:51.156915+0000", 2024-09-21T15:00:30.099 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 38.872367971999999, 2024-09-21T15:00:30.099 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0090768180000000004, 2024-09-21T15:00:30.099 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:30.099 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:30.100 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:30.100 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:30.100 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:30.100 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 83752 2024-09-21T15:00:30.100 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.100 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:30.100 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.100 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:30.100 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156915+0000", 2024-09-21T15:00:30.100 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:30.100 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.100 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.100 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:30.100 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156915+0000", 2024-09-21T15:00:30.101 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:30.101 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.101 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.101 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:30.101 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156915+0000", 2024-09-21T15:00:30.101 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:30.101 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.101 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.101 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:30.101 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156919+0000", 2024-09-21T15:00:30.101 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2679999999999996e-06 2024-09-21T15:00:30.101 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.101 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.101 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:30.101 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156929+0000", 2024-09-21T15:00:30.102 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.031e-05 2024-09-21T15:00:30.102 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.102 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.102 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:30.102 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156937+0000", 2024-09-21T15:00:30.102 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8820000000000001e-06 2024-09-21T15:00:30.102 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.102 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.102 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:30.102 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165909+0000", 2024-09-21T15:00:30.102 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0089722989999999996 2024-09-21T15:00:30.102 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.102 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.102 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:30.102 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165948+0000", 2024-09-21T15:00:30.103 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8114999999999997e-05 2024-09-21T15:00:30.103 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.103 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.103 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:30.103 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165992+0000", 2024-09-21T15:00:30.103 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4503999999999999e-05 2024-09-21T15:00:30.103 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:30.103 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:30.103 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:30.103 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.103 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.103 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:83755 3.b 3:d4eb1ee5:::rbd_data.11368296f791.0000000000000000:40 [sparse-read 1310720~0 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-21T15:00:30.103 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:51.156944+0000", 2024-09-21T15:00:30.103 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 38.872338898000002, 2024-09-21T15:00:30.104 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.009159146, 2024-09-21T15:00:30.104 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:30.104 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:30.104 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:30.104 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:30.104 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:30.104 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 83755 2024-09-21T15:00:30.104 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.104 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:30.104 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.104 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:30.104 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156944+0000", 2024-09-21T15:00:30.104 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:30.104 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.104 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.105 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:30.105 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156944+0000", 2024-09-21T15:00:30.105 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:30.105 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.105 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.105 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:30.105 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156944+0000", 2024-09-21T15:00:30.105 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:30.105 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.105 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.105 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:30.105 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156948+0000", 2024-09-21T15:00:30.105 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5279999999999997e-06 2024-09-21T15:00:30.105 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.105 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.106 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:30.106 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156959+0000", 2024-09-21T15:00:30.106 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0215e-05 2024-09-21T15:00:30.106 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.106 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.106 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:30.106 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156967+0000", 2024-09-21T15:00:30.106 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6119999999999999e-06 2024-09-21T15:00:30.106 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.106 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.106 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:30.106 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.166007+0000", 2024-09-21T15:00:30.106 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0090395260000000009 2024-09-21T15:00:30.106 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.106 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.107 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:30.107 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.166053+0000", 2024-09-21T15:00:30.107 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6281e-05 2024-09-21T15:00:30.107 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.107 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.107 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:30.107 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.166103+0000", 2024-09-21T15:00:30.107 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0510000000000003e-05 2024-09-21T15:00:30.107 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:30.107 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:30.107 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:30.107 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.107 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.107 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:83763 3.b 3:d3640016:::rbd_data.11368296f791.0000000000000006:40 [sparse-read 524288~81920 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-21T15:00:30.107 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:51.157975+0000", 2024-09-21T15:00:30.108 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 38.871307790000003, 2024-09-21T15:00:30.108 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.008327348, 2024-09-21T15:00:30.108 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:30.108 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:30.108 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:30.108 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:30.108 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:30.108 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 83763 2024-09-21T15:00:30.108 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.108 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:30.108 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.108 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:30.108 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.157975+0000", 2024-09-21T15:00:30.109 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:30.109 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.109 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.109 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:30.109 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.157975+0000", 2024-09-21T15:00:30.109 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:30.109 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.109 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.109 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:30.109 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.157970+0000", 2024-09-21T15:00:30.109 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999943 2024-09-21T15:00:30.109 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.109 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.109 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:30.109 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.157985+0000", 2024-09-21T15:00:30.110 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4807000000000001e-05 2024-09-21T15:00:30.110 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.110 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.110 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:30.110 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.158012+0000", 2024-09-21T15:00:30.110 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7600999999999999e-05 2024-09-21T15:00:30.110 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.110 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.110 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:30.110 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.158019+0000", 2024-09-21T15:00:30.110 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9319999999999999e-06 2024-09-21T15:00:30.110 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.110 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.110 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:30.110 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.166118+0000", 2024-09-21T15:00:30.111 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0080992489999999993 2024-09-21T15:00:30.111 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.111 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.111 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:30.111 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.166156+0000", 2024-09-21T15:00:30.111 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7639999999999999e-05 2024-09-21T15:00:30.111 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.111 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.111 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:30.111 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.166303+0000", 2024-09-21T15:00:30.111 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000146799 2024-09-21T15:00:30.111 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:30.111 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:30.111 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:30.111 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.111 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.112 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:83764 3.b 3:d3640016:::rbd_data.11368296f791.0000000000000006:40 [sparse-read 1966080~0 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-21T15:00:30.112 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:51.158026+0000", 2024-09-21T15:00:30.112 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 38.871257356999998, 2024-09-21T15:00:30.112 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0083785839999999997, 2024-09-21T15:00:30.112 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:30.112 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:30.112 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:30.112 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:30.112 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:30.112 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 83764 2024-09-21T15:00:30.112 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.112 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:30.112 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.112 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:30.112 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.158026+0000", 2024-09-21T15:00:30.113 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:30.113 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.113 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.113 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:30.113 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.158026+0000", 2024-09-21T15:00:30.113 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:30.113 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.113 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.113 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:30.113 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.158025+0000", 2024-09-21T15:00:30.113 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:30.113 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.113 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.113 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:30.114 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.158053+0000", 2024-09-21T15:00:30.114 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7594e-05 2024-09-21T15:00:30.114 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.114 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.114 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:30.114 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.158061+0000", 2024-09-21T15:00:30.114 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2069999999999995e-06 2024-09-21T15:00:30.114 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.114 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.114 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:30.114 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.158067+0000", 2024-09-21T15:00:30.114 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3799999999999999e-06 2024-09-21T15:00:30.114 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.114 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.115 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:30.115 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.166321+0000", 2024-09-21T15:00:30.115 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0082539870000000008 2024-09-21T15:00:30.115 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.115 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.115 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:30.115 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.166360+0000", 2024-09-21T15:00:30.115 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8597999999999999e-05 2024-09-21T15:00:30.115 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.115 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.115 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:30.115 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.166404+0000", 2024-09-21T15:00:30.115 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4378000000000001e-05 2024-09-21T15:00:30.115 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:30.115 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:30.116 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:30.116 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.116 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.116 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:97675 3.b 3:d321acb8:::rbd_data.1136bfc79a70.0000000000000003:head [write 0~4718592 [fadvise_sequential+fadvise_nocache],write 5242880~147456 [fadvise_sequential+fadvise_nocache],write 5455872~147456 [fadvise_sequential+fadvise_nocache],write 5668864~229376 [fadvise_sequential+fadvise_nocache],write 5963776~147456 [fadvise_sequential+fadvise_nocache],write 6176768~229376 [fadvise_sequential+fadvise_nocache],write 6471680~147456 [fadvise_sequential+fadvise_nocache],write 6684672~229376 [fadvise_sequential+fadvise_nocache],write 6979584~147456 [fadvise_sequential+fadvise_nocache],write 7192576~229376 [fadvise_sequential+fadvise_nocache],write 7487488~147456 [fadvise_sequential+fadvise_nocache],write 7700480~147456 [fadvise_sequential+fadvise_nocache],write 7864320~65536 [fadvise_sequential+fadvise_nocache],write 7995392~147456 [fadvise_sequential+fadvise_nocache],write 8208384~147456 [fadvise_sequential+fadvise_nocache],write 8388608~49152 [fadvise_sequential+fadvise_nocache],write 8536064~114688 [fadvise_sequential+fadvise_nocache],write 8749056~114688 [fadvise_sequential+fadvise_nocache],write 8912896~32768 [fadvise_sequential+fadvise_nocache],write 9125888~32768 [fadvise_sequential+fadvise_nocache],write 9338880~32768 [fadvise_sequential+fadvise_nocache],write 9437184~16384 [fadvise_sequential+fadvise_nocache],write 9633792~32768 [fadvise_sequential+fadvise_nocache],write 9846784~32768 [fadvise_sequential+fadvise_nocache],write 9961472~114688 [fadvise_sequential+fadvise_nocache],write 10092544~196608 [fadvise_sequential+fadvise_nocache],write 10305536~278528 [fadvise_sequential+fadvise_nocache],write 10600448~196608 [fadvise_sequential+fadvise_nocache],write 10813440~278528 [fadvise_sequential+fadvise_nocache],write 11108352~196608 [fadvise_sequential+fadvise_nocache],write 11321344~196608 [fadvise_sequential+fadvise_nocache],write 11534336~65536 [fadvise_sequential+fadvise_nocache],write 11632640~98304 [fadvise_sequential+fadvise_nocache],write 11747328~65536 [fadvise_sequential+fadvise_nocache],write 11845632~98304 [fadvise_sequential+fadvise_nocache],write 11960320~65536 [fadvise_sequential+fadvise_nocache],write 12058624~49152 [fadvise_sequential+fadvise_nocache],write 12140544~98304 [fadvise_sequential+fadvise_nocache],write 12255232~65536 [fadvise_sequential+fadvise_nocache],write 12353536~98304 [fadvise_sequential+fadvise_nocache],write 12468224~65536 [fadvise_sequential+fadvise_nocache],write 12566528~16384 [fadvise_sequential+fadvise_nocache],write 12648448~98304 [fadvise_sequential+fadvise_nocache],write 12861440~98304 [fadvise_sequential+fadvise_nocache],write 13074432~32768 [fadvise_sequential+fadvise_nocache],write 13156352~98304 [fadvise_sequential+fadvise_nocache],write 13369344~98304 [fadvise_sequential+fadvise_nocache],write 13582336~49152 [fadvise_sequential+fadvise_nocache],write 13664256~98304 [fadvise_sequential+fadvise_nocache],write 13877248~98304 [fadvise_sequential+fadvise_nocache],write 14090240~65536 [fadvise_sequential+fadvise_nocache],write 15286272~81920 [fadvise_sequential+fadvise_nocache],write 15499264~81920 [fadvise_sequential+fadvise_nocache],write 15712256~16384 [fadvise_sequential+fadvise_nocache],write 15794176~81920 [fadvise_sequential+fadvise_nocache],write 16007168~81920 [fadvise_sequential+fadvise_nocache],write 16220160~32768 [fadvise_sequential+fadvise_nocache],write 16302080~81920 [fadvise_sequential+fadvise_nocache],write 16515072~81920 [fadvise_sequential+fadvise_nocache],write 16728064~49152 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:30.116 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.302412+0000", 2024-09-21T15:00:30.116 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 24.726871689999999, 2024-09-21T15:00:30.116 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.067380532000000007, 2024-09-21T15:00:30.116 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:30.116 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:30.116 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:30.116 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:30.116 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:30.116 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97675 2024-09-21T15:00:30.116 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.116 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:30.116 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.117 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:30.117 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.302412+0000", 2024-09-21T15:00:30.117 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:30.117 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.117 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.117 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:30.117 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.302412+0000", 2024-09-21T15:00:30.117 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:30.117 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.117 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.117 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:30.117 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.302409+0000", 2024-09-21T15:00:30.117 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-21T15:00:30.117 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.118 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.118 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:30.118 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328550+0000", 2024-09-21T15:00:30.118 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.026140674999999999 2024-09-21T15:00:30.118 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.118 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.118 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:30.118 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328584+0000", 2024-09-21T15:00:30.118 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4165999999999998e-05 2024-09-21T15:00:30.118 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.118 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.118 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:30.118 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328599+0000", 2024-09-21T15:00:30.118 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5505000000000001e-05 2024-09-21T15:00:30.119 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.119 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.119 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:30.119 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328704+0000", 2024-09-21T15:00:30.119 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010487199999999999 2024-09-21T15:00:30.119 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.119 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.119 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:30.119 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.329082+0000", 2024-09-21T15:00:30.119 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00037804899999999998 2024-09-21T15:00:30.119 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.119 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.119 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T15:00:30.119 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.331158+0000", 2024-09-21T15:00:30.120 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0020755449999999998 2024-09-21T15:00:30.120 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.120 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.120 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:30.120 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.360553+0000", 2024-09-21T15:00:30.120 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.029395553000000001 2024-09-21T15:00:30.120 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.120 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.120 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:30.120 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.369496+0000", 2024-09-21T15:00:30.120 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0089428619999999993 2024-09-21T15:00:30.120 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.120 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.120 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:30.120 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.369686+0000", 2024-09-21T15:00:30.120 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019000399999999999 2024-09-21T15:00:30.121 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.121 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.121 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:30.121 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.369792+0000", 2024-09-21T15:00:30.121 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000105775 2024-09-21T15:00:30.121 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:30.121 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:30.121 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:30.121 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.121 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.121 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:97693 3.b 3:d7d7b39b:::rbd_data.1136b9c49b1c.000000000000000c:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:30.121 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.328643+0000", 2024-09-21T15:00:30.121 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 24.700639903999999, 2024-09-21T15:00:30.121 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.031062136000000001, 2024-09-21T15:00:30.122 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:30.122 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:30.122 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:30.122 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:30.122 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:30.122 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97693 2024-09-21T15:00:30.122 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.122 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:30.122 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.122 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:30.122 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328643+0000", 2024-09-21T15:00:30.122 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:30.122 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.122 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.122 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:30.123 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328643+0000", 2024-09-21T15:00:30.123 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:30.123 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.123 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.123 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:30.123 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328642+0000", 2024-09-21T15:00:30.123 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:00:30.123 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.123 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.123 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:30.123 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328650+0000", 2024-09-21T15:00:30.123 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7770000000000008e-06 2024-09-21T15:00:30.123 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.123 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.124 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:30.124 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328662+0000", 2024-09-21T15:00:30.124 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2026e-05 2024-09-21T15:00:30.124 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.124 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.124 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:30.124 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328671+0000", 2024-09-21T15:00:30.124 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.585e-06 2024-09-21T15:00:30.124 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.124 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.124 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:30.124 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.359498+0000", 2024-09-21T15:00:30.124 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.030827001 2024-09-21T15:00:30.124 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.125 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.125 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:30.125 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.359603+0000", 2024-09-21T15:00:30.125 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010512400000000001 2024-09-21T15:00:30.125 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.125 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.125 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:30.125 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.359705+0000", 2024-09-21T15:00:30.125 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000102327 2024-09-21T15:00:30.125 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:30.125 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:30.125 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:30.125 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.125 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.126 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:97724 3.b 3:d4bf6bcf:::rbd_data.1136b9c49b1c.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:30.126 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.328704+0000", 2024-09-21T15:00:30.126 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 24.700579632, 2024-09-21T15:00:30.126 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.031157957, 2024-09-21T15:00:30.126 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:30.126 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:30.126 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:30.126 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:30.126 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:30.126 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97724 2024-09-21T15:00:30.126 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.126 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:30.126 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.126 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:30.127 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328704+0000", 2024-09-21T15:00:30.127 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:30.127 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.127 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.127 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:30.127 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328704+0000", 2024-09-21T15:00:30.127 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:30.127 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.127 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.127 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:30.127 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328703+0000", 2024-09-21T15:00:30.127 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:00:30.127 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.127 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.127 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:30.128 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328708+0000", 2024-09-21T15:00:30.128 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0050000000000004e-06 2024-09-21T15:00:30.128 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.128 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.128 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:30.128 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328719+0000", 2024-09-21T15:00:30.128 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1052e-05 2024-09-21T15:00:30.128 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.128 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.128 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:30.128 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328727+0000", 2024-09-21T15:00:30.128 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4519999999999997e-06 2024-09-21T15:00:30.128 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.128 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.128 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:30.128 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.359737+0000", 2024-09-21T15:00:30.129 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.031009341999999999 2024-09-21T15:00:30.129 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.129 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.129 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:30.129 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.359806+0000", 2024-09-21T15:00:30.129 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9750000000000001e-05 2024-09-21T15:00:30.129 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:30.129 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:30.129 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:30.129 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.359862+0000", 2024-09-21T15:00:30.129 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5084999999999999e-05 2024-09-21T15:00:30.129 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:30.129 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:30.129 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:30.129 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:30.129 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:30.130 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:30.131 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:00:30.126+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:00:30.131 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T15:00:30.232 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:00:30.227+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:00:30.238 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:30.238 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:00:30.238 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T15:00:30.238 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:30.248 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T15:00:30.318 INFO:teuthology.orchestra.run.smithi032.stderr:set 3.4 pg_upmap mapping to [4,2] 2024-09-21T15:00:30.332 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:00:30.326+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:00:30.356 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:30.356 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:00:30.356 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T15:00:30.356 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T15:00:30.356 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:30.367 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T15:00:30.433 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:00:30.426+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:00:30.434 INFO:teuthology.orchestra.run.smithi032.stdout:checking clone #27, image image_client.0-clone27 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent28 2024-09-21T15:00:30.474 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:30.475 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T15:00:30.475 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T15:00:30.475 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T15:00:30.475 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.475 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:97677 3.15 3:a8e72133:::rbd_data.1136bfc79a70.0000000000000005:head [write 0~4440027 [fadvise_sequential+fadvise_nocache],write 4751360~147456 [fadvise_sequential+fadvise_nocache],write 4964352~147456 [fadvise_sequential+fadvise_nocache],write 5177344~65536 [fadvise_sequential+fadvise_nocache],write 5259264~147456 [fadvise_sequential+fadvise_nocache],write 5472256~147456 [fadvise_sequential+fadvise_nocache],write 5685248~229376 [fadvise_sequential+fadvise_nocache],write 5980160~147456 [fadvise_sequential+fadvise_nocache],write 6193152~229376 [fadvise_sequential+fadvise_nocache],write 6488064~147456 [fadvise_sequential+fadvise_nocache],write 6701056~229376 [fadvise_sequential+fadvise_nocache],write 6995968~147456 [fadvise_sequential+fadvise_nocache],write 7208960~229376 [fadvise_sequential+fadvise_nocache],write 7503872~147456 [fadvise_sequential+fadvise_nocache],write 7716864~229376 [fadvise_sequential+fadvise_nocache],write 8044544~114688 [fadvise_sequential+fadvise_nocache],write 8257536~114688 [fadvise_sequential+fadvise_nocache],write 8388608~65536 [fadvise_sequential+fadvise_nocache],write 8552448~114688 [fadvise_sequential+fadvise_nocache],write 8765440~114688 [fadvise_sequential+fadvise_nocache],write 8929280~32768 [fadvise_sequential+fadvise_nocache],write 9142272~32768 [fadvise_sequential+fadvise_nocache],write 9355264~32768 [fadvise_sequential+fadvise_nocache],write 9437184~32768 [fadvise_sequential+fadvise_nocache],write 9650176~32768 [fadvise_sequential+fadvise_nocache],write 9863168~32768 [fadvise_sequential+fadvise_nocache],write 9961472~131072 [fadvise_sequential+fadvise_nocache],write 10108928~196608 [fadvise_sequential+fadvise_nocache],write 10321920~278528 [fadvise_sequential+fadvise_nocache],write 10616832~196608 [fadvise_sequential+fadvise_nocache],write 10829824~278528 [fadvise_sequential+fadvise_nocache],write 11124736~196608 [fadvise_sequential+fadvise_nocache],write 11337728~196608 [fadvise_sequential+fadvise_nocache],write 11550720~65536 [fadvise_sequential+fadvise_nocache],write 11649024~98304 [fadvise_sequential+fadvise_nocache],write 11763712~65536 [fadvise_sequential+fadvise_nocache],write 11862016~98304 [fadvise_sequential+fadvise_nocache],write 11976704~65536 [fadvise_sequential+fadvise_nocache],write 12058624~65536 [fadvise_sequential+fadvise_nocache],write 12156928~98304 [fadvise_sequential+fadvise_nocache],write 12271616~65536 [fadvise_sequential+fadvise_nocache],write 12369920~98304 [fadvise_sequential+fadvise_nocache],write 12484608~65536 [fadvise_sequential+fadvise_nocache],write 12664832~98304 [fadvise_sequential+fadvise_nocache],write 12877824~98304 [fadvise_sequential+fadvise_nocache],write 13090816~16384 [fadvise_sequential+fadvise_nocache],write 13172736~98304 [fadvise_sequential+fadvise_nocache],write 13385728~98304 [fadvise_sequential+fadvise_nocache],write 13598720~32768 [fadvise_sequential+fadvise_nocache],write 13680640~98304 [fadvise_sequential+fadvise_nocache],write 13893632~98304 [fadvise_sequential+fadvise_nocache],write 14106624~49152 [fadvise_sequential+fadvise_nocache],write 14794752~81920 [fadvise_sequential+fadvise_nocache],write 15007744~81920 [fadvise_sequential+fadvise_nocache],write 15302656~81920 [fadvise_sequential+fadvise_nocache],write 15515648~81920 [fadvise_sequential+fadvise_nocache],write 15810560~81920 [fadvise_sequential+fadvise_nocache],write 16023552~81920 [fadvise_sequential+fadvise_nocache],write 16236544~16384 [fadvise_sequential+fadvise_nocache],write 16318464~81920 [fadvise_sequential+fadvise_nocache],write 16531456~81920 [fadvise_sequential+fadvise_nocache],write 16744448~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:30.475 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.302946+0000", 2024-09-21T15:00:30.475 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 25.170412384999999, 2024-09-21T15:00:30.475 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.091231204999999996, 2024-09-21T15:00:30.475 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:30.475 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:30.475 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:30.475 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:30.475 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:30.476 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 97677 2024-09-21T15:00:30.476 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.476 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:30.476 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.476 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:30.476 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.302946+0000", 2024-09-21T15:00:30.476 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:30.476 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.476 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.476 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:30.476 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.302946+0000", 2024-09-21T15:00:30.476 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:30.476 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.476 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.476 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:30.477 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.302942+0000", 2024-09-21T15:00:30.477 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999962 2024-09-21T15:00:30.477 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.477 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.477 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:30.477 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311253+0000", 2024-09-21T15:00:30.477 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0083105360000000003 2024-09-21T15:00:30.477 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.477 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.477 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:30.477 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311283+0000", 2024-09-21T15:00:30.477 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0065000000000001e-05 2024-09-21T15:00:30.477 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.477 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.477 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:30.478 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311296+0000", 2024-09-21T15:00:30.478 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3206e-05 2024-09-21T15:00:30.478 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.478 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.478 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:30.478 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311404+0000", 2024-09-21T15:00:30.478 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000107514 2024-09-21T15:00:30.478 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.478 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.478 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:30.478 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311795+0000", 2024-09-21T15:00:30.478 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000390913 2024-09-21T15:00:30.478 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.478 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.478 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 6", 2024-09-21T15:00:30.479 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.314686+0000", 2024-09-21T15:00:30.479 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0028915659999999999 2024-09-21T15:00:30.479 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.479 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.479 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:30.479 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.368178+0000", 2024-09-21T15:00:30.479 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.053491706 2024-09-21T15:00:30.479 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.479 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.479 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:30.479 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.393925+0000", 2024-09-21T15:00:30.479 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025747289 2024-09-21T15:00:30.479 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.479 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.479 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:30.480 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.394127+0000", 2024-09-21T15:00:30.480 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000201444 2024-09-21T15:00:30.480 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.480 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.480 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:30.480 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.394177+0000", 2024-09-21T15:00:30.480 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0689000000000002e-05 2024-09-21T15:00:30.480 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:30.480 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:30.480 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:30.480 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.480 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.480 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:97673 3.1d e122/24 3:bd91946e:::rbd_data.1136bfc79a70.0000000000000002:head v 122'162, mlcod=120'160)", 2024-09-21T15:00:30.480 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.313730+0000", 2024-09-21T15:00:30.481 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 25.159628059999999, 2024-09-21T15:00:30.481 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055373462999999998, 2024-09-21T15:00:30.481 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:30.481 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:30.481 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:30.481 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.481 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:30.481 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.313730+0000", 2024-09-21T15:00:30.481 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:30.481 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.481 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.481 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:30.481 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.313730+0000", 2024-09-21T15:00:30.481 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:30.481 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.482 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.482 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:30.482 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.313730+0000", 2024-09-21T15:00:30.482 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:30.482 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.482 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.482 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:30.482 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.336113+0000", 2024-09-21T15:00:30.482 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022382962999999999 2024-09-21T15:00:30.482 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.482 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.482 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:30.482 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.336140+0000", 2024-09-21T15:00:30.483 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7248999999999999e-05 2024-09-21T15:00:30.483 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.483 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.483 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:30.483 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.336151+0000", 2024-09-21T15:00:30.483 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1015e-05 2024-09-21T15:00:30.483 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.483 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.483 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:30.483 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.355428+0000", 2024-09-21T15:00:30.483 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019276379999999999 2024-09-21T15:00:30.483 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.484 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.484 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:30.484 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.355484+0000", 2024-09-21T15:00:30.484 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6820999999999997e-05 2024-09-21T15:00:30.484 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.484 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.484 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:30.484 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.369040+0000", 2024-09-21T15:00:30.484 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013555466 2024-09-21T15:00:30.484 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.484 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.484 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:30.484 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.369104+0000", 2024-09-21T15:00:30.485 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3986999999999996e-05 2024-09-21T15:00:30.485 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:30.485 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:30.485 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:30.485 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.485 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.485 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:113889 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 5300224~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:30.485 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.501880+0000", 2024-09-21T15:00:30.485 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.97147814700000001, 2024-09-21T15:00:30.485 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052644204999999999, 2024-09-21T15:00:30.485 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:30.485 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:30.485 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:30.485 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:30.486 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:30.486 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 113889 2024-09-21T15:00:30.486 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.486 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:30.486 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.486 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:30.486 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501880+0000", 2024-09-21T15:00:30.486 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:30.486 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.486 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.486 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:30.486 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501880+0000", 2024-09-21T15:00:30.486 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:30.486 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.487 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.487 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:30.487 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501880+0000", 2024-09-21T15:00:30.487 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:00:30.487 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.487 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.487 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:30.487 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501882+0000", 2024-09-21T15:00:30.487 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.364e-06 2024-09-21T15:00:30.487 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.487 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.487 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:30.487 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501885+0000", 2024-09-21T15:00:30.488 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4199999999999999e-06 2024-09-21T15:00:30.488 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.488 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.488 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:30.488 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501888+0000", 2024-09-21T15:00:30.488 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8669999999999998e-06 2024-09-21T15:00:30.488 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.488 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.488 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:30.488 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.554369+0000", 2024-09-21T15:00:30.488 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052481456000000003 2024-09-21T15:00:30.488 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.488 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.489 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:30.489 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.554449+0000", 2024-09-21T15:00:30.489 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9412000000000005e-05 2024-09-21T15:00:30.489 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.489 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.489 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:30.489 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.554525+0000", 2024-09-21T15:00:30.489 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5908999999999994e-05 2024-09-21T15:00:30.489 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:30.489 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:30.489 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:30.489 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.489 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.489 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:113890 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 5390336~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:30.489 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.501890+0000", 2024-09-21T15:00:30.490 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.97146843199999999, 2024-09-21T15:00:30.490 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052785212999999997, 2024-09-21T15:00:30.490 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:30.490 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:30.490 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:30.490 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:30.490 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:30.490 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 113890 2024-09-21T15:00:30.490 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.490 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:30.490 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.490 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:30.490 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501890+0000", 2024-09-21T15:00:30.490 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:30.491 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.491 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.491 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:30.491 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501890+0000", 2024-09-21T15:00:30.491 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:30.491 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.491 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.491 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:30.491 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501890+0000", 2024-09-21T15:00:30.491 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:00:30.491 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.491 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.491 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:30.491 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501891+0000", 2024-09-21T15:00:30.491 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3599999999999999e-06 2024-09-21T15:00:30.492 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.492 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.492 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:30.492 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501894+0000", 2024-09-21T15:00:30.492 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.717e-06 2024-09-21T15:00:30.492 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.492 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.492 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:30.492 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501897+0000", 2024-09-21T15:00:30.492 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5790000000000002e-06 2024-09-21T15:00:30.492 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.492 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.492 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:30.492 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.554569+0000", 2024-09-21T15:00:30.493 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052672434999999997 2024-09-21T15:00:30.493 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.493 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.493 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:30.493 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.554623+0000", 2024-09-21T15:00:30.493 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4509999999999998e-05 2024-09-21T15:00:30.493 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.493 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.493 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:30.493 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.554675+0000", 2024-09-21T15:00:30.493 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1833999999999999e-05 2024-09-21T15:00:30.493 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:30.493 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:30.493 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:30.494 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.494 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.494 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:113891 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 5480448~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:30.494 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.501899+0000", 2024-09-21T15:00:30.494 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.97145994599999996, 2024-09-21T15:00:30.494 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052991637000000001, 2024-09-21T15:00:30.494 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:30.494 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:30.494 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:30.494 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:30.494 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:30.494 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 113891 2024-09-21T15:00:30.494 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.495 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:30.495 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.495 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:30.495 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501899+0000", 2024-09-21T15:00:30.495 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:30.495 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.495 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.495 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:30.495 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501899+0000", 2024-09-21T15:00:30.495 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:30.495 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.495 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.495 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:30.495 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501898+0000", 2024-09-21T15:00:30.496 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:00:30.496 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.496 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.496 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:30.496 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501900+0000", 2024-09-21T15:00:30.496 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.356e-06 2024-09-21T15:00:30.496 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.496 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.496 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:30.496 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501902+0000", 2024-09-21T15:00:30.496 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6620000000000001e-06 2024-09-21T15:00:30.496 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.497 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.497 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:30.497 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501905+0000", 2024-09-21T15:00:30.497 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.492e-06 2024-09-21T15:00:30.497 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.497 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.497 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:30.497 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.554707+0000", 2024-09-21T15:00:30.497 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052802618000000003 2024-09-21T15:00:30.497 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.497 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.497 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:30.497 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.554798+0000", 2024-09-21T15:00:30.498 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0976000000000004e-05 2024-09-21T15:00:30.498 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.498 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.498 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:30.498 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.554890+0000", 2024-09-21T15:00:30.498 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1755999999999994e-05 2024-09-21T15:00:30.498 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:30.498 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:30.498 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:30.498 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.498 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.498 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:113892 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 5570560~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:30.498 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.501907+0000", 2024-09-21T15:00:30.499 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.97145151399999996, 2024-09-21T15:00:30.499 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.053089242000000002, 2024-09-21T15:00:30.499 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:30.499 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:30.499 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:30.499 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:30.499 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:30.499 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 113892 2024-09-21T15:00:30.499 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.499 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:30.499 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.499 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:30.500 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501907+0000", 2024-09-21T15:00:30.500 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:30.500 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.500 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.500 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:30.500 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501907+0000", 2024-09-21T15:00:30.500 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:30.500 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.500 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.500 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:30.500 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501907+0000", 2024-09-21T15:00:30.500 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:00:30.500 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.500 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.500 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:30.501 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501908+0000", 2024-09-21T15:00:30.501 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2860000000000001e-06 2024-09-21T15:00:30.501 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.501 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.501 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:30.501 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501913+0000", 2024-09-21T15:00:30.501 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1320000000000002e-06 2024-09-21T15:00:30.501 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.501 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.501 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:30.501 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501916+0000", 2024-09-21T15:00:30.501 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5330000000000001e-06 2024-09-21T15:00:30.501 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.501 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.501 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:30.502 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.554904+0000", 2024-09-21T15:00:30.502 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052988631000000001 2024-09-21T15:00:30.502 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.502 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.502 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:30.502 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.554945+0000", 2024-09-21T15:00:30.502 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.0695999999999998e-05 2024-09-21T15:00:30.502 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.502 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.502 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:30.502 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.554996+0000", 2024-09-21T15:00:30.502 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1193999999999998e-05 2024-09-21T15:00:30.502 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:30.502 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:30.502 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:30.503 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.503 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.503 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:113893 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 5660672~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:30.503 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.501918+0000", 2024-09-21T15:00:30.503 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.97144053699999999, 2024-09-21T15:00:30.503 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.053245595, 2024-09-21T15:00:30.503 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:30.503 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:30.503 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:30.503 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:30.503 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:30.503 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 113893 2024-09-21T15:00:30.503 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.504 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:30.504 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.504 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:30.504 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501918+0000", 2024-09-21T15:00:30.504 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:30.504 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.504 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.504 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:30.504 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501918+0000", 2024-09-21T15:00:30.504 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:30.504 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.504 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.504 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:30.504 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501918+0000", 2024-09-21T15:00:30.504 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:30.505 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.505 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.505 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:30.505 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501920+0000", 2024-09-21T15:00:30.505 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9800000000000001e-06 2024-09-21T15:00:30.505 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.505 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.505 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:30.505 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501928+0000", 2024-09-21T15:00:30.505 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7999999999999999e-06 2024-09-21T15:00:30.505 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.505 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.505 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:30.505 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501930+0000", 2024-09-21T15:00:30.505 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.785e-06 2024-09-21T15:00:30.505 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.506 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.506 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:30.506 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555019+0000", 2024-09-21T15:00:30.506 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.053089154 2024-09-21T15:00:30.506 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.506 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.506 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:30.506 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555065+0000", 2024-09-21T15:00:30.506 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5485000000000002e-05 2024-09-21T15:00:30.506 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.506 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.506 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:30.506 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555164+0000", 2024-09-21T15:00:30.506 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8629999999999996e-05 2024-09-21T15:00:30.507 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:30.507 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:30.507 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:30.507 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.507 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.507 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:113894 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 5750784~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:30.507 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.502384+0000", 2024-09-21T15:00:30.507 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.97097460000000002, 2024-09-21T15:00:30.507 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052884821999999998, 2024-09-21T15:00:30.507 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:30.507 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:30.507 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:30.507 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:30.507 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:30.507 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 113894 2024-09-21T15:00:30.507 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.508 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:30.508 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.508 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:30.508 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502384+0000", 2024-09-21T15:00:30.508 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:30.508 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.508 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.508 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:30.508 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502384+0000", 2024-09-21T15:00:30.508 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:30.508 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.508 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.508 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:30.508 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501933+0000", 2024-09-21T15:00:30.508 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9995484 2024-09-21T15:00:30.509 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.509 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.509 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:30.509 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502390+0000", 2024-09-21T15:00:30.509 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00045714099999999999 2024-09-21T15:00:30.509 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.509 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.509 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:30.509 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502402+0000", 2024-09-21T15:00:30.509 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.236e-05 2024-09-21T15:00:30.509 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.509 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.509 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:30.510 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502411+0000", 2024-09-21T15:00:30.510 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8689999999999993e-06 2024-09-21T15:00:30.510 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.510 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.510 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:30.510 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555178+0000", 2024-09-21T15:00:30.510 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052767050000000003 2024-09-21T15:00:30.510 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.510 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.510 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:30.510 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555215+0000", 2024-09-21T15:00:30.510 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.667e-05 2024-09-21T15:00:30.510 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.511 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.511 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:30.511 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555269+0000", 2024-09-21T15:00:30.511 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4109e-05 2024-09-21T15:00:30.511 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:30.511 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:30.511 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:30.511 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.511 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.511 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:113895 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 5840896~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:30.511 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.502651+0000", 2024-09-21T15:00:30.511 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.97070798700000005, 2024-09-21T15:00:30.511 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052770692000000001, 2024-09-21T15:00:30.511 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:30.512 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:30.512 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:30.512 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:30.512 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:30.512 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 113895 2024-09-21T15:00:30.512 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.512 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:30.512 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.512 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:30.512 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502651+0000", 2024-09-21T15:00:30.512 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:30.512 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.512 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.512 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:30.512 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502651+0000", 2024-09-21T15:00:30.513 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:30.513 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.513 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.513 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:30.513 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502421+0000", 2024-09-21T15:00:30.513 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9997702 2024-09-21T15:00:30.513 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.513 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.513 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:30.513 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502658+0000", 2024-09-21T15:00:30.513 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000237264 2024-09-21T15:00:30.513 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.513 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.513 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:30.514 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502673+0000", 2024-09-21T15:00:30.514 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4474999999999999e-05 2024-09-21T15:00:30.514 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.514 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.514 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:30.514 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502681+0000", 2024-09-21T15:00:30.514 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6119999999999999e-06 2024-09-21T15:00:30.514 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.515 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.515 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:30.515 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555285+0000", 2024-09-21T15:00:30.515 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052603860000000002 2024-09-21T15:00:30.515 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.515 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.515 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:30.515 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555348+0000", 2024-09-21T15:00:30.515 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3411999999999995e-05 2024-09-21T15:00:30.515 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.515 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.516 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:30.516 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555421+0000", 2024-09-21T15:00:30.516 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2809e-05 2024-09-21T15:00:30.516 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:30.516 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:30.516 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:30.516 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.516 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.516 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:113896 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 5931008~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:30.516 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.502862+0000", 2024-09-21T15:00:30.516 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.97049643900000004, 2024-09-21T15:00:30.516 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052660208, 2024-09-21T15:00:30.516 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:30.517 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:30.517 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:30.517 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:30.517 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:30.517 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 113896 2024-09-21T15:00:30.517 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.517 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:30.517 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.517 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:30.517 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502862+0000", 2024-09-21T15:00:30.517 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:30.517 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.517 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.518 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:30.518 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502862+0000", 2024-09-21T15:00:30.518 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:30.518 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.518 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.518 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:30.518 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502689+0000", 2024-09-21T15:00:30.518 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9998264 2024-09-21T15:00:30.518 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.518 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.518 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:30.518 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502873+0000", 2024-09-21T15:00:30.518 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000184146 2024-09-21T15:00:30.518 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.518 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.519 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:30.519 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502884+0000", 2024-09-21T15:00:30.519 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1293e-05 2024-09-21T15:00:30.519 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.519 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.519 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:30.519 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502891+0000", 2024-09-21T15:00:30.519 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4259999999999997e-06 2024-09-21T15:00:30.519 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.519 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.519 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:30.519 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555435+0000", 2024-09-21T15:00:30.519 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052543053999999999 2024-09-21T15:00:30.519 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.519 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.520 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:30.520 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555471+0000", 2024-09-21T15:00:30.520 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6454e-05 2024-09-21T15:00:30.520 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.520 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.520 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:30.520 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555522+0000", 2024-09-21T15:00:30.520 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1314999999999999e-05 2024-09-21T15:00:30.520 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:30.520 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:30.520 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:30.520 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.520 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.520 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:113897 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 6021120~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:30.521 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.503004+0000", 2024-09-21T15:00:30.521 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.97035433199999999, 2024-09-21T15:00:30.521 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052669499000000002, 2024-09-21T15:00:30.521 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:30.521 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:30.521 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:30.521 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:30.521 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:30.521 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 113897 2024-09-21T15:00:30.521 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.521 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:30.521 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.521 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:30.521 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503004+0000", 2024-09-21T15:00:30.522 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:30.522 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.522 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.522 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:30.522 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503004+0000", 2024-09-21T15:00:30.522 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:30.522 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.522 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.522 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:30.522 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502898+0000", 2024-09-21T15:00:30.522 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9998937 2024-09-21T15:00:30.522 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.522 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.522 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:30.522 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503010+0000", 2024-09-21T15:00:30.523 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000112396 2024-09-21T15:00:30.523 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.523 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.523 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:30.523 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503023+0000", 2024-09-21T15:00:30.523 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2206e-05 2024-09-21T15:00:30.523 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.523 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.523 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:30.523 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503031+0000", 2024-09-21T15:00:30.523 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3780000000000001e-06 2024-09-21T15:00:30.523 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.523 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.523 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:30.523 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555549+0000", 2024-09-21T15:00:30.524 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052517909000000002 2024-09-21T15:00:30.524 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.524 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.524 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:30.524 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555594+0000", 2024-09-21T15:00:30.524 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5385e-05 2024-09-21T15:00:30.524 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.524 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.524 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:30.524 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555674+0000", 2024-09-21T15:00:30.524 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9447e-05 2024-09-21T15:00:30.524 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:30.524 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:30.524 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:30.525 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.525 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.525 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:113898 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 6111232~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:30.525 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.503039+0000", 2024-09-21T15:00:30.525 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.970319547, 2024-09-21T15:00:30.525 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052754290000000002, 2024-09-21T15:00:30.525 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:30.525 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:30.525 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:30.525 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:30.525 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:30.525 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 113898 2024-09-21T15:00:30.525 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.525 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:30.525 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.525 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:30.525 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503039+0000", 2024-09-21T15:00:30.526 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:30.526 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.526 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.526 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:30.526 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503039+0000", 2024-09-21T15:00:30.526 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:30.526 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.526 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.526 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:30.526 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503038+0000", 2024-09-21T15:00:30.526 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:00:30.526 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.526 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.526 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:30.526 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503042+0000", 2024-09-21T15:00:30.527 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.0640000000000004e-06 2024-09-21T15:00:30.527 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.527 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.527 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:30.527 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503049+0000", 2024-09-21T15:00:30.527 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5579999999999997e-06 2024-09-21T15:00:30.527 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.527 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.527 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:30.527 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503054+0000", 2024-09-21T15:00:30.527 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6550000000000001e-06 2024-09-21T15:00:30.527 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.527 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.527 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:30.527 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555693+0000", 2024-09-21T15:00:30.527 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052638216000000002 2024-09-21T15:00:30.528 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.528 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.528 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:30.528 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555736+0000", 2024-09-21T15:00:30.528 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3619999999999999e-05 2024-09-21T15:00:30.528 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.528 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.528 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:30.528 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555793+0000", 2024-09-21T15:00:30.528 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6956999999999997e-05 2024-09-21T15:00:30.528 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:30.528 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:30.528 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:30.528 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.528 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.528 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:113899 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 6201344~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:30.529 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.503255+0000", 2024-09-21T15:00:30.529 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.97010327399999996, 2024-09-21T15:00:30.529 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052648161999999998, 2024-09-21T15:00:30.529 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:30.529 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:30.529 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:30.529 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:30.529 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:30.529 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 113899 2024-09-21T15:00:30.529 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.529 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:30.529 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.529 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:30.529 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503255+0000", 2024-09-21T15:00:30.529 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:30.529 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.529 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.530 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:30.530 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503255+0000", 2024-09-21T15:00:30.530 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:30.530 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.530 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.530 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:30.530 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503058+0000", 2024-09-21T15:00:30.530 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9998021 2024-09-21T15:00:30.530 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.530 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.530 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:30.530 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503262+0000", 2024-09-21T15:00:30.530 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00020421299999999999 2024-09-21T15:00:30.530 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.530 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.531 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:30.531 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503274+0000", 2024-09-21T15:00:30.531 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2052000000000001e-05 2024-09-21T15:00:30.531 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.531 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.531 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:30.531 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503282+0000", 2024-09-21T15:00:30.531 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.977e-06 2024-09-21T15:00:30.531 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.531 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.531 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:30.531 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555808+0000", 2024-09-21T15:00:30.531 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052526150000000001 2024-09-21T15:00:30.531 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.531 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.531 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:30.532 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555851+0000", 2024-09-21T15:00:30.532 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2846999999999998e-05 2024-09-21T15:00:30.532 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.532 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.532 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:30.532 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555903+0000", 2024-09-21T15:00:30.532 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2607e-05 2024-09-21T15:00:30.532 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:30.532 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:30.532 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:30.532 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.532 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.532 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:113900 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 6307840~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:30.532 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.503445+0000", 2024-09-21T15:00:30.533 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.96991333899999999, 2024-09-21T15:00:30.533 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052589546000000001, 2024-09-21T15:00:30.533 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:30.533 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:30.533 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:30.533 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:30.533 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:30.533 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 113900 2024-09-21T15:00:30.533 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.533 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:30.533 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.533 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:30.533 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503445+0000", 2024-09-21T15:00:30.533 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:30.533 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.533 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.534 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:30.534 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503445+0000", 2024-09-21T15:00:30.534 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:30.534 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.534 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.534 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:30.534 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503290+0000", 2024-09-21T15:00:30.534 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9998446 2024-09-21T15:00:30.534 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.534 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.534 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:30.534 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503451+0000", 2024-09-21T15:00:30.534 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00016111000000000001 2024-09-21T15:00:30.534 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.535 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.535 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:30.535 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503461+0000", 2024-09-21T15:00:30.535 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8740000000000007e-06 2024-09-21T15:00:30.535 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.535 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.535 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:30.535 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503468+0000", 2024-09-21T15:00:30.535 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.187e-06 2024-09-21T15:00:30.535 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.535 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.535 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:30.535 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555930+0000", 2024-09-21T15:00:30.535 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052462404999999997 2024-09-21T15:00:30.535 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.535 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.536 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:30.536 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555971+0000", 2024-09-21T15:00:30.536 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.0419e-05 2024-09-21T15:00:30.536 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.536 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.536 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:30.536 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.556035+0000", 2024-09-21T15:00:30.536 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3948999999999994e-05 2024-09-21T15:00:30.536 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:30.536 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:30.536 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:30.536 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.536 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.536 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:114050 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 4767744~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:30.536 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.514988+0000", 2024-09-21T15:00:30.536 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.95837082799999995, 2024-09-21T15:00:30.537 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052614432000000003, 2024-09-21T15:00:30.537 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:30.537 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:30.537 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:30.537 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:30.537 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:30.537 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 114050 2024-09-21T15:00:30.537 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.537 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:30.537 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.537 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:30.537 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.514988+0000", 2024-09-21T15:00:30.537 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:30.537 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.537 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.537 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:30.538 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.514988+0000", 2024-09-21T15:00:30.538 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:30.538 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.538 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.538 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:30.538 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.514987+0000", 2024-09-21T15:00:30.538 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:30.538 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.538 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.538 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:30.538 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.514990+0000", 2024-09-21T15:00:30.538 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4739999999999999e-06 2024-09-21T15:00:30.539 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.539 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.539 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:30.539 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.514995+0000", 2024-09-21T15:00:30.539 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1939999999999997e-06 2024-09-21T15:00:30.539 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.539 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.539 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:30.539 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.514999+0000", 2024-09-21T15:00:30.539 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.245e-06 2024-09-21T15:00:30.539 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.539 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.539 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:30.539 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.567498+0000", 2024-09-21T15:00:30.539 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052498516000000002 2024-09-21T15:00:30.539 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.540 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.540 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:30.540 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.567541+0000", 2024-09-21T15:00:30.540 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3516000000000001e-05 2024-09-21T15:00:30.540 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.540 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.540 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:30.540 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.567602+0000", 2024-09-21T15:00:30.540 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0800000000000001e-05 2024-09-21T15:00:30.540 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:30.540 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:30.540 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:30.540 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.540 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.540 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:114051 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 4857856~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:30.540 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.515003+0000", 2024-09-21T15:00:30.541 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.95835511299999998, 2024-09-21T15:00:30.541 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052709245000000002, 2024-09-21T15:00:30.541 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:30.541 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:30.541 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:30.541 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:30.541 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:30.541 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 114051 2024-09-21T15:00:30.541 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.541 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:30.541 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.541 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:30.541 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515003+0000", 2024-09-21T15:00:30.541 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:30.541 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.542 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.542 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:30.542 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515003+0000", 2024-09-21T15:00:30.542 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:30.542 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.542 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.542 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:30.542 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515003+0000", 2024-09-21T15:00:30.542 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:30.542 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.542 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.542 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:30.542 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515005+0000", 2024-09-21T15:00:30.542 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3499999999999999e-06 2024-09-21T15:00:30.543 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.543 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.543 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:30.543 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515011+0000", 2024-09-21T15:00:30.543 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2850000000000002e-06 2024-09-21T15:00:30.543 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.543 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.543 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:30.543 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515017+0000", 2024-09-21T15:00:30.543 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0059999999999998e-06 2024-09-21T15:00:30.543 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.543 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.544 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:30.544 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.567618+0000", 2024-09-21T15:00:30.544 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052600806999999999 2024-09-21T15:00:30.544 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.544 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.544 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:30.544 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.567655+0000", 2024-09-21T15:00:30.544 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7866e-05 2024-09-21T15:00:30.544 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.544 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.544 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:30.544 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.567713+0000", 2024-09-21T15:00:30.544 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7281999999999998e-05 2024-09-21T15:00:30.544 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:30.545 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:30.545 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:30.545 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.545 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.545 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:114052 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 4947968~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:30.545 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.515046+0000", 2024-09-21T15:00:30.545 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.95831245399999998, 2024-09-21T15:00:30.545 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052795975000000002, 2024-09-21T15:00:30.545 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:30.545 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:30.545 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:30.545 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:30.545 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:30.545 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 114052 2024-09-21T15:00:30.545 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.545 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:30.546 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.546 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:30.546 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515046+0000", 2024-09-21T15:00:30.546 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:30.546 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.546 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.546 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:30.546 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515046+0000", 2024-09-21T15:00:30.546 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:30.546 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.546 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.546 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:30.546 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515046+0000", 2024-09-21T15:00:30.546 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:30.546 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.547 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.547 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:30.547 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515050+0000", 2024-09-21T15:00:30.547 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1359999999999999e-06 2024-09-21T15:00:30.547 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.547 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.547 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:30.547 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515060+0000", 2024-09-21T15:00:30.547 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0339e-05 2024-09-21T15:00:30.547 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.547 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.547 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:30.547 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515068+0000", 2024-09-21T15:00:30.547 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1149999999999994e-06 2024-09-21T15:00:30.547 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.547 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.547 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:30.548 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.567733+0000", 2024-09-21T15:00:30.548 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052664872000000001 2024-09-21T15:00:30.548 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.548 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.548 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:30.548 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.567778+0000", 2024-09-21T15:00:30.548 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4966999999999998e-05 2024-09-21T15:00:30.548 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.548 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.548 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:30.548 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.567842+0000", 2024-09-21T15:00:30.548 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4114000000000003e-05 2024-09-21T15:00:30.548 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:30.548 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:30.548 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:30.548 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.549 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.549 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:114053 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 5038080~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:30.549 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.515076+0000", 2024-09-21T15:00:30.549 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.95828259400000004, 2024-09-21T15:00:30.549 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052886112999999998, 2024-09-21T15:00:30.549 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:30.549 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:30.549 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:30.549 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:30.549 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:30.549 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 114053 2024-09-21T15:00:30.549 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.549 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:30.549 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.550 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:30.550 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515076+0000", 2024-09-21T15:00:30.550 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:30.550 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.550 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.550 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:30.550 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515076+0000", 2024-09-21T15:00:30.550 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:30.550 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.550 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.550 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:30.550 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515075+0000", 2024-09-21T15:00:30.550 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:30.550 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.550 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.551 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:30.551 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515083+0000", 2024-09-21T15:00:30.551 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0520000000000002e-06 2024-09-21T15:00:30.551 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.551 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.551 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:30.551 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515092+0000", 2024-09-21T15:00:30.551 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1440000000000003e-06 2024-09-21T15:00:30.551 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.551 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.551 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:30.551 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515098+0000", 2024-09-21T15:00:30.551 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1090000000000003e-06 2024-09-21T15:00:30.551 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.551 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.551 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:30.552 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.567871+0000", 2024-09-21T15:00:30.552 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052772966999999997 2024-09-21T15:00:30.552 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.552 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.552 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:30.552 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.567909+0000", 2024-09-21T15:00:30.552 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.8380000000000002e-05 2024-09-21T15:00:30.552 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.552 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.552 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:30.552 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.567962+0000", 2024-09-21T15:00:30.552 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3068000000000001e-05 2024-09-21T15:00:30.552 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:30.552 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:30.552 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:30.552 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.553 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.553 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:114055 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 5218304~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:30.553 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.515526+0000", 2024-09-21T15:00:30.553 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.95783215399999999, 2024-09-21T15:00:30.553 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052684278000000001, 2024-09-21T15:00:30.553 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:30.553 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:30.553 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:30.553 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:30.553 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:30.553 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 114055 2024-09-21T15:00:30.553 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.553 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:30.553 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.553 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:30.554 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515526+0000", 2024-09-21T15:00:30.554 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:30.554 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.554 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.554 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:30.554 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515526+0000", 2024-09-21T15:00:30.554 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:30.554 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.554 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.554 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:30.554 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515526+0000", 2024-09-21T15:00:30.554 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:30.554 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.554 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.555 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:30.555 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515529+0000", 2024-09-21T15:00:30.555 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.596e-06 2024-09-21T15:00:30.555 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.555 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.555 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:30.555 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515536+0000", 2024-09-21T15:00:30.555 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7399999999999998e-06 2024-09-21T15:00:30.555 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.555 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.555 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:30.555 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515542+0000", 2024-09-21T15:00:30.555 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5740000000000003e-06 2024-09-21T15:00:30.555 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.555 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.555 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:30.556 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.568116+0000", 2024-09-21T15:00:30.556 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052574152999999998 2024-09-21T15:00:30.556 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.556 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.556 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:30.556 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.568154+0000", 2024-09-21T15:00:30.556 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7926000000000002e-05 2024-09-21T15:00:30.556 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.556 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.556 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:30.556 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.568211+0000", 2024-09-21T15:00:30.556 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6858999999999999e-05 2024-09-21T15:00:30.556 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:30.556 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:30.556 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:30.556 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.557 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.557 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:114056 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 5308416~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:30.557 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.515547+0000", 2024-09-21T15:00:30.557 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.95781112899999998, 2024-09-21T15:00:30.557 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052792795000000003, 2024-09-21T15:00:30.557 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:30.557 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:30.557 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:30.557 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:30.557 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:30.557 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 114056 2024-09-21T15:00:30.557 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.557 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:30.557 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.557 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:30.558 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515547+0000", 2024-09-21T15:00:30.558 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:30.558 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.558 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.558 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:30.558 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515547+0000", 2024-09-21T15:00:30.558 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:30.558 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.558 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.558 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:30.558 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515547+0000", 2024-09-21T15:00:30.558 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:30.558 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.559 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.559 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:30.559 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515550+0000", 2024-09-21T15:00:30.559 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2770000000000001e-06 2024-09-21T15:00:30.559 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.559 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.559 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:30.559 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515554+0000", 2024-09-21T15:00:30.559 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.087e-06 2024-09-21T15:00:30.559 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.559 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.559 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:30.559 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515558+0000", 2024-09-21T15:00:30.560 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1709999999999998e-06 2024-09-21T15:00:30.560 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.560 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.560 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:30.560 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.568234+0000", 2024-09-21T15:00:30.560 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052676675999999999 2024-09-21T15:00:30.560 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.560 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.560 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:30.560 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.568277+0000", 2024-09-21T15:00:30.560 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2400000000000001e-05 2024-09-21T15:00:30.560 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.560 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.560 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:30.561 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.568340+0000", 2024-09-21T15:00:30.561 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3581e-05 2024-09-21T15:00:30.561 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:30.561 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:30.561 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:30.561 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:30.561 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:30.561 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:30.562 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T15:00:30.563 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:00:30.558+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:00:30.660 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:30.660 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:00:30.660 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T15:00:30.660 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:30.663 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:00:30.662+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:00:30.669 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T15:00:30.764 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:00:30.758+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:00:30.771 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T15:00:30.778 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:30.778 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:00:30.778 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T15:00:30.778 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T15:00:30.778 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:30.788 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T15:00:30.865 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:00:30.858+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:00:30.897 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:30.897 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T15:00:30.897 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T15:00:30.897 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T15:00:30.897 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.897 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T15:00:30.897 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.205754+0000", 2024-09-21T15:00:30.897 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 147.68957600499999, 2024-09-21T15:00:30.897 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.41117094199999998, 2024-09-21T15:00:30.898 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:30.898 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:30.898 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:30.898 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T15:00:30.898 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T15:00:30.898 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1 2024-09-21T15:00:30.898 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.898 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:30.898 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.898 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:30.898 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205754+0000", 2024-09-21T15:00:30.898 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:30.898 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.898 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.899 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:30.899 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205754+0000", 2024-09-21T15:00:30.899 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:30.899 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.899 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.899 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:30.899 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205752+0000", 2024-09-21T15:00:30.899 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T15:00:30.899 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.899 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.899 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:30.899 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205765+0000", 2024-09-21T15:00:30.899 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3230999999999999e-05 2024-09-21T15:00:30.900 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.900 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.900 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:30.900 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205782+0000", 2024-09-21T15:00:30.900 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6634e-05 2024-09-21T15:00:30.900 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.900 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.900 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:30.900 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205793+0000", 2024-09-21T15:00:30.900 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1297e-05 2024-09-21T15:00:30.900 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.900 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.900 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:30.900 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205867+0000", 2024-09-21T15:00:30.900 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4269000000000003e-05 2024-09-21T15:00:30.901 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.901 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.901 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:00:30.901 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205877+0000", 2024-09-21T15:00:30.901 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2690000000000005e-06 2024-09-21T15:00:30.901 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.901 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.901 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:30.901 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.613700+0000", 2024-09-21T15:00:30.901 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.407823455 2024-09-21T15:00:30.901 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.901 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.901 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:30.901 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.614203+0000", 2024-09-21T15:00:30.901 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00050311500000000005 2024-09-21T15:00:30.902 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.902 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.902 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T15:00:30.902 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.614750+0000", 2024-09-21T15:00:30.902 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00054738699999999998 2024-09-21T15:00:30.902 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.902 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.902 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:30.902 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616367+0000", 2024-09-21T15:00:30.902 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001616634 2024-09-21T15:00:30.902 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.902 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.902 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:30.902 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616574+0000", 2024-09-21T15:00:30.902 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00020668499999999999 2024-09-21T15:00:30.902 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.903 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.903 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:30.903 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616802+0000", 2024-09-21T15:00:30.903 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00022817900000000001 2024-09-21T15:00:30.903 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.903 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.903 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:30.903 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616925+0000", 2024-09-21T15:00:30.903 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012309699999999999 2024-09-21T15:00:30.903 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:30.903 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:30.903 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:30.903 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.903 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.903 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6639 3.1a 3:592d80ed:::rbd_data.11361bed1f78.0000000000000005:d [read 0~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T15:00:30.903 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131650+0000", 2024-09-21T15:00:30.904 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 115.76368045700001, 2024-09-21T15:00:30.904 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016002832000000002, 2024-09-21T15:00:30.904 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:30.904 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:30.904 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:30.904 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:30.904 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:30.904 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6639 2024-09-21T15:00:30.904 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.904 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:30.904 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.904 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:30.904 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131650+0000", 2024-09-21T15:00:30.904 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:30.904 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.905 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.905 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:30.905 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131650+0000", 2024-09-21T15:00:30.905 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:30.905 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.905 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.905 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:30.905 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131649+0000", 2024-09-21T15:00:30.905 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:30.905 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.905 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.905 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:30.905 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131651+0000", 2024-09-21T15:00:30.905 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9199999999999998e-06 2024-09-21T15:00:30.905 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.905 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.906 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:30.906 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131656+0000", 2024-09-21T15:00:30.906 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9899999999999997e-06 2024-09-21T15:00:30.906 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.906 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.906 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:30.906 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131665+0000", 2024-09-21T15:00:30.906 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0999999999999993e-06 2024-09-21T15:00:30.906 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.906 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.906 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:30.906 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146309+0000", 2024-09-21T15:00:30.906 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014643893999999999 2024-09-21T15:00:30.906 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.906 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.906 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:30.907 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146392+0000", 2024-09-21T15:00:30.907 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2531e-05 2024-09-21T15:00:30.907 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.907 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.907 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:30.907 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147653+0000", 2024-09-21T15:00:30.907 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0012606710000000001 2024-09-21T15:00:30.907 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:30.907 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:30.907 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:30.907 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.907 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.907 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6640 3.1a 3:592d80ed:::rbd_data.11361bed1f78.0000000000000005:d [read 65536~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T15:00:30.907 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131670+0000", 2024-09-21T15:00:30.907 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 115.76366015799999, 2024-09-21T15:00:30.908 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016188859, 2024-09-21T15:00:30.908 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:30.908 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:30.908 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:30.908 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:30.908 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:30.908 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6640 2024-09-21T15:00:30.908 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.908 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:30.908 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.908 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:30.908 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131670+0000", 2024-09-21T15:00:30.908 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:30.908 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.908 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.909 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:30.909 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131670+0000", 2024-09-21T15:00:30.909 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:30.909 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.909 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.909 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:30.909 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131670+0000", 2024-09-21T15:00:30.909 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:00:30.909 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.909 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.909 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:30.909 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131672+0000", 2024-09-21T15:00:30.909 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.818e-06 2024-09-21T15:00:30.909 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.909 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.909 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:30.910 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131675+0000", 2024-09-21T15:00:30.910 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5930000000000002e-06 2024-09-21T15:00:30.910 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.910 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.910 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:30.910 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131678+0000", 2024-09-21T15:00:30.910 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6790000000000001e-06 2024-09-21T15:00:30.910 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.910 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.910 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:30.910 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147681+0000", 2024-09-21T15:00:30.910 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016002755 2024-09-21T15:00:30.910 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.910 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.910 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:30.911 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147762+0000", 2024-09-21T15:00:30.911 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1052999999999998e-05 2024-09-21T15:00:30.911 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.911 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.911 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:30.911 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147859+0000", 2024-09-21T15:00:30.911 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.7199000000000002e-05 2024-09-21T15:00:30.911 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:30.911 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:30.911 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:30.911 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.911 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.911 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6641 3.1a 3:592d80ed:::rbd_data.11361bed1f78.0000000000000005:d [read 147456~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T15:00:30.911 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131680+0000", 2024-09-21T15:00:30.912 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 115.76364986999999, 2024-09-21T15:00:30.912 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016431296000000001, 2024-09-21T15:00:30.912 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:30.912 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:30.912 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:30.912 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:30.912 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:30.912 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6641 2024-09-21T15:00:30.912 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.912 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:30.912 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.912 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:30.912 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131680+0000", 2024-09-21T15:00:30.912 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:30.912 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.912 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.913 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:30.913 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131680+0000", 2024-09-21T15:00:30.913 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:30.913 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.913 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.913 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:30.913 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131680+0000", 2024-09-21T15:00:30.913 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:00:30.913 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.913 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.913 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:30.913 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131681+0000", 2024-09-21T15:00:30.913 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2610000000000001e-06 2024-09-21T15:00:30.913 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.913 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.914 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:30.914 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131685+0000", 2024-09-21T15:00:30.914 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.8299999999999998e-06 2024-09-21T15:00:30.914 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.914 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.914 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:30.914 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131688+0000", 2024-09-21T15:00:30.914 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6189999999999998e-06 2024-09-21T15:00:30.914 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.914 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.914 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:30.914 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147884+0000", 2024-09-21T15:00:30.914 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016196465 2024-09-21T15:00:30.914 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.914 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.915 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:30.915 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147949+0000", 2024-09-21T15:00:30.915 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4867000000000001e-05 2024-09-21T15:00:30.915 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.915 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.915 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:30.915 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.148112+0000", 2024-09-21T15:00:30.915 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00016247700000000001 2024-09-21T15:00:30.915 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:30.915 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:30.915 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:30.915 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.915 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.915 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6642 3.1a 3:592d80ed:::rbd_data.11361bed1f78.0000000000000005:d [read 212992~32768 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T15:00:30.915 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131690+0000", 2024-09-21T15:00:30.915 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 115.76363999199999, 2024-09-21T15:00:30.916 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016751335999999999, 2024-09-21T15:00:30.916 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:30.916 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:30.916 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:30.916 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:30.916 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:30.916 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6642 2024-09-21T15:00:30.916 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.916 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:30.916 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.916 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:30.916 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131690+0000", 2024-09-21T15:00:30.916 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:30.916 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.916 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.916 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:30.917 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131690+0000", 2024-09-21T15:00:30.917 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:30.917 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.917 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.917 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:30.917 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131690+0000", 2024-09-21T15:00:30.917 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:00:30.917 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.917 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.917 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:30.917 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131691+0000", 2024-09-21T15:00:30.917 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3120000000000001e-06 2024-09-21T15:00:30.917 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.917 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.917 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:30.917 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131694+0000", 2024-09-21T15:00:30.918 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0070000000000002e-06 2024-09-21T15:00:30.918 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.918 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.918 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:30.918 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131700+0000", 2024-09-21T15:00:30.918 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0800000000000002e-06 2024-09-21T15:00:30.918 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.918 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.918 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:30.918 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.148135+0000", 2024-09-21T15:00:30.918 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016434250000000001 2024-09-21T15:00:30.918 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.918 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.918 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:30.918 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.148204+0000", 2024-09-21T15:00:30.919 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9466000000000002e-05 2024-09-21T15:00:30.919 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.919 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.919 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:30.919 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.148441+0000", 2024-09-21T15:00:30.919 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00023743700000000001 2024-09-21T15:00:30.919 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:30.919 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:30.919 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:30.919 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.919 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.919 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:61214 3.7 3:e28aec10:::rbd_header.1136862f872f:head [watch reconnect cookie 139864671932080 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-21T15:00:30.919 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:18.147172+0000", 2024-09-21T15:00:30.919 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 72.748158361999998, 2024-09-21T15:00:30.919 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.019264956, 2024-09-21T15:00:30.919 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:30.920 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:30.920 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:30.920 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:30.920 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:30.920 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 61214 2024-09-21T15:00:30.920 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.920 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:30.920 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.920 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:30.920 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147172+0000", 2024-09-21T15:00:30.920 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:30.920 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.920 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.920 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:30.920 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147172+0000", 2024-09-21T15:00:30.920 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:30.920 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.921 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.921 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:30.921 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147168+0000", 2024-09-21T15:00:30.921 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999962 2024-09-21T15:00:30.921 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.921 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.921 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:30.921 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147188+0000", 2024-09-21T15:00:30.921 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9717000000000001e-05 2024-09-21T15:00:30.921 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.921 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.921 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:30.921 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147239+0000", 2024-09-21T15:00:30.921 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1493999999999999e-05 2024-09-21T15:00:30.921 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.922 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.922 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:30.922 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147265+0000", 2024-09-21T15:00:30.922 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5746999999999999e-05 2024-09-21T15:00:30.922 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.922 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.922 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:30.922 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147372+0000", 2024-09-21T15:00:30.922 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010681200000000001 2024-09-21T15:00:30.922 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.922 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.922 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op must wait for map", 2024-09-21T15:00:30.922 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147380+0000", 2024-09-21T15:00:30.922 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7919999999999993e-06 2024-09-21T15:00:30.922 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.923 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.923 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:30.923 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.154025+0000", 2024-09-21T15:00:30.923 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0066454909999999999 2024-09-21T15:00:30.923 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.923 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.923 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:00:30.923 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.154041+0000", 2024-09-21T15:00:30.923 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5894000000000001e-05 2024-09-21T15:00:30.923 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.923 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.923 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:30.923 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.164345+0000", 2024-09-21T15:00:30.923 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0103040459999999 2024-09-21T15:00:30.923 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.924 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.924 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:30.924 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.164590+0000", 2024-09-21T15:00:30.924 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00024492699999999998 2024-09-21T15:00:30.924 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.924 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.924 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:30.924 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.166437+0000", 2024-09-21T15:00:30.924 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0018466330000000001 2024-09-21T15:00:30.924 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:30.924 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:30.924 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:30.924 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.924 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.924 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:61215 3.7 3:e28aec10:::rbd_header.1136862f872f:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-21T15:00:30.925 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:18.152008+0000", 2024-09-21T15:00:30.925 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 72.743321910000006, 2024-09-21T15:00:30.925 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.015644958, 2024-09-21T15:00:30.925 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:30.925 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:30.925 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:30.925 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:30.925 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:30.925 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 61215 2024-09-21T15:00:30.925 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.925 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:30.925 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.925 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:30.925 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152008+0000", 2024-09-21T15:00:30.925 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:30.925 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.925 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.926 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:30.926 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152008+0000", 2024-09-21T15:00:30.926 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:30.926 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.926 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.926 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:30.926 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152005+0000", 2024-09-21T15:00:30.926 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-21T15:00:30.926 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.926 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.926 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:30.926 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152025+0000", 2024-09-21T15:00:30.926 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9306000000000001e-05 2024-09-21T15:00:30.926 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.926 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.927 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:30.927 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152054+0000", 2024-09-21T15:00:30.927 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9189000000000001e-05 2024-09-21T15:00:30.927 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.927 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.927 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:30.927 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152078+0000", 2024-09-21T15:00:30.927 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4317000000000001e-05 2024-09-21T15:00:30.927 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.927 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.927 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:30.927 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152243+0000", 2024-09-21T15:00:30.927 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000164802 2024-09-21T15:00:30.927 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.927 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.927 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting_for_map not empty", 2024-09-21T15:00:30.928 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152251+0000", 2024-09-21T15:00:30.928 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0600000000000008e-06 2024-09-21T15:00:30.928 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.928 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.928 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:30.928 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.154741+0000", 2024-09-21T15:00:30.928 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0024900069999999998 2024-09-21T15:00:30.928 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.928 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.928 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:00:30.928 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.154753+0000", 2024-09-21T15:00:30.928 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1588e-05 2024-09-21T15:00:30.928 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.928 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.929 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:30.929 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.165380+0000", 2024-09-21T15:00:30.929 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0106272249999999 2024-09-21T15:00:30.929 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.929 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.929 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:30.929 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.165517+0000", 2024-09-21T15:00:30.929 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013737999999999999 2024-09-21T15:00:30.929 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.929 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.929 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 6", 2024-09-21T15:00:30.929 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.165938+0000", 2024-09-21T15:00:30.929 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00042082399999999998 2024-09-21T15:00:30.929 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.929 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.929 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:30.930 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.166795+0000", 2024-09-21T15:00:30.930 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00085674899999999997 2024-09-21T15:00:30.930 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.930 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.930 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:30.930 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.167598+0000", 2024-09-21T15:00:30.930 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00080335599999999997 2024-09-21T15:00:30.930 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.930 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.930 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:30.930 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.167632+0000", 2024-09-21T15:00:30.930 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4122000000000003e-05 2024-09-21T15:00:30.930 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.930 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.930 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:30.930 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.167653+0000", 2024-09-21T15:00:30.931 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0862000000000001e-05 2024-09-21T15:00:30.931 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:30.931 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:30.931 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:30.931 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.931 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.931 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:73698 3.10 3:0a4d95f6:::rbd_data.1136f4939602.000000000000000b:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-21T15:00:30.931 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:41.969870+0000", 2024-09-21T15:00:30.931 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 48.925459771999996, 2024-09-21T15:00:30.931 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.017502461, 2024-09-21T15:00:30.931 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:30.931 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:30.931 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:30.931 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:30.931 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:30.931 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 73698 2024-09-21T15:00:30.932 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.932 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:30.932 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.932 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:30.932 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.969870+0000", 2024-09-21T15:00:30.932 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:30.932 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.932 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.932 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:30.932 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.969870+0000", 2024-09-21T15:00:30.932 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:30.932 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.932 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.932 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:30.932 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.969868+0000", 2024-09-21T15:00:30.933 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T15:00:30.933 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.933 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.933 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:30.933 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.969885+0000", 2024-09-21T15:00:30.933 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7416999999999999e-05 2024-09-21T15:00:30.933 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.933 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.933 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:30.933 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.969918+0000", 2024-09-21T15:00:30.933 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3133000000000003e-05 2024-09-21T15:00:30.933 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.933 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.933 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:30.933 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.969934+0000", 2024-09-21T15:00:30.933 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6027000000000001e-05 2024-09-21T15:00:30.934 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.934 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.934 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:30.934 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.970021+0000", 2024-09-21T15:00:30.934 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6907e-05 2024-09-21T15:00:30.934 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.934 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.934 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:30.934 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.970148+0000", 2024-09-21T15:00:30.934 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012701499999999999 2024-09-21T15:00:30.934 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.934 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.934 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T15:00:30.934 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.972971+0000", 2024-09-21T15:00:30.935 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.002822873 2024-09-21T15:00:30.935 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.935 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.935 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:30.935 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.987124+0000", 2024-09-21T15:00:30.935 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0141529 2024-09-21T15:00:30.935 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.935 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.935 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:30.935 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.987146+0000", 2024-09-21T15:00:30.935 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1783000000000001e-05 2024-09-21T15:00:30.935 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.935 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.935 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:30.935 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.987258+0000", 2024-09-21T15:00:30.935 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000112555 2024-09-21T15:00:30.936 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.936 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.936 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:30.936 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.987373+0000", 2024-09-21T15:00:30.936 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000114434 2024-09-21T15:00:30.936 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:30.936 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:30.936 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:30.936 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.936 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.936 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:97638 3.1a 3:5e573525:::rbd_data.1136b9c49b1c.0000000000000026:49 [read 1277952~49152 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:30.936 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.276202+0000", 2024-09-21T15:00:30.936 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 25.619127667000001, 2024-09-21T15:00:30.936 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.015360357999999999, 2024-09-21T15:00:30.936 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:30.937 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:30.937 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:30.937 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:30.937 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:30.937 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 97638 2024-09-21T15:00:30.937 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.937 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:30.937 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.937 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:30.937 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276202+0000", 2024-09-21T15:00:30.937 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:30.937 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.937 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.937 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:30.937 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276202+0000", 2024-09-21T15:00:30.937 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:30.937 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.938 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.938 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:30.938 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276202+0000", 2024-09-21T15:00:30.938 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:30.938 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.938 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.938 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:30.938 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276204+0000", 2024-09-21T15:00:30.938 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5359999999999999e-06 2024-09-21T15:00:30.938 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.938 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.938 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:30.938 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276210+0000", 2024-09-21T15:00:30.938 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3040000000000004e-06 2024-09-21T15:00:30.938 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.939 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.939 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:30.939 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276214+0000", 2024-09-21T15:00:30.939 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6779999999999999e-06 2024-09-21T15:00:30.939 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.939 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.939 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:30.939 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.290563+0000", 2024-09-21T15:00:30.939 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014348346 2024-09-21T15:00:30.939 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.939 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.939 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:30.939 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.290623+0000", 2024-09-21T15:00:30.939 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0001999999999999e-05 2024-09-21T15:00:30.939 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.939 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.940 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:30.940 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.291563+0000", 2024-09-21T15:00:30.940 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00094003100000000003 2024-09-21T15:00:30.940 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:30.940 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:30.940 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:30.940 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.940 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.940 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:97678 3.d e122/24 3:b2f34c6d:::rbd_data.1136bfc79a70.0000000000000006:head v 122'234, mlcod=120'232)", 2024-09-21T15:00:30.940 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.321655+0000", 2024-09-21T15:00:30.940 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 25.573674993000001, 2024-09-21T15:00:30.940 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024509388999999999, 2024-09-21T15:00:30.940 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:30.940 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:30.940 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:30.940 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.941 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:30.941 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.321655+0000", 2024-09-21T15:00:30.941 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:30.941 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.941 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.941 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:30.941 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.321655+0000", 2024-09-21T15:00:30.941 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:30.941 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.941 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.941 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:30.941 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.321655+0000", 2024-09-21T15:00:30.941 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:30.941 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.941 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.941 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:30.941 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.329453+0000", 2024-09-21T15:00:30.942 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0077980560000000003 2024-09-21T15:00:30.942 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.942 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.942 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:30.942 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.329482+0000", 2024-09-21T15:00:30.942 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9153999999999999e-05 2024-09-21T15:00:30.942 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.942 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.942 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:30.942 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.329493+0000", 2024-09-21T15:00:30.942 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1103e-05 2024-09-21T15:00:30.942 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.942 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.942 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:30.942 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.329555+0000", 2024-09-21T15:00:30.943 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2345e-05 2024-09-21T15:00:30.943 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.943 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.943 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:30.943 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.329589+0000", 2024-09-21T15:00:30.943 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3908999999999997e-05 2024-09-21T15:00:30.943 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.943 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.943 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:30.943 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.346083+0000", 2024-09-21T15:00:30.943 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016493383 2024-09-21T15:00:30.943 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.943 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.943 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:30.943 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.346165+0000", 2024-09-21T15:00:30.944 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1796000000000001e-05 2024-09-21T15:00:30.944 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:30.944 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:30.944 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:30.944 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.944 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.944 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:97679 3.f e122/24 3:f2a362b4:::rbd_data.1136bfc79a70.0000000000000007:head v 122'331, mlcod=120'329)", 2024-09-21T15:00:30.944 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.337145+0000", 2024-09-21T15:00:30.944 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 25.558184984, 2024-09-21T15:00:30.944 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021912411, 2024-09-21T15:00:30.944 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:30.944 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:30.944 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:30.944 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.945 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:30.945 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.337145+0000", 2024-09-21T15:00:30.945 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:30.945 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.945 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.945 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:30.945 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.337145+0000", 2024-09-21T15:00:30.945 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:30.945 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.945 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.945 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:30.945 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.337145+0000", 2024-09-21T15:00:30.945 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:30.945 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.945 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.945 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:30.946 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.343899+0000", 2024-09-21T15:00:30.946 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0067541420000000003 2024-09-21T15:00:30.946 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.946 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.946 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:30.946 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.343921+0000", 2024-09-21T15:00:30.946 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2255e-05 2024-09-21T15:00:30.946 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.946 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.946 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:30.946 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.343933+0000", 2024-09-21T15:00:30.946 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1528e-05 2024-09-21T15:00:30.946 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.946 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.946 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:30.946 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.344001+0000", 2024-09-21T15:00:30.947 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8269999999999995e-05 2024-09-21T15:00:30.947 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.947 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.947 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:30.947 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.344029+0000", 2024-09-21T15:00:30.947 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8254000000000001e-05 2024-09-21T15:00:30.947 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.947 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.947 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:30.947 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.358901+0000", 2024-09-21T15:00:30.947 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0148719 2024-09-21T15:00:30.947 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.947 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.947 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:30.947 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.359058+0000", 2024-09-21T15:00:30.947 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000156446 2024-09-21T15:00:30.948 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:30.948 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:30.948 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:30.948 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.948 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.948 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:97676 3.1f e122/24 3:ffaa836a:::rbd_data.1136bfc79a70.0000000000000004:head v 122'312, mlcod=120'310)", 2024-09-21T15:00:30.948 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.348683+0000", 2024-09-21T15:00:30.948 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 25.546646818999999, 2024-09-21T15:00:30.948 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.033768802000000001, 2024-09-21T15:00:30.948 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:30.948 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:30.948 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:30.948 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.948 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:30.948 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.348683+0000", 2024-09-21T15:00:30.948 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:30.949 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.949 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.949 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:30.949 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.348683+0000", 2024-09-21T15:00:30.949 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:30.949 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.949 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.949 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:30.949 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.348683+0000", 2024-09-21T15:00:30.949 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:00:30.949 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.949 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.949 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:30.949 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.366937+0000", 2024-09-21T15:00:30.949 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018254415 2024-09-21T15:00:30.950 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.950 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.950 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:30.950 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.366959+0000", 2024-09-21T15:00:30.950 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1634e-05 2024-09-21T15:00:30.950 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.950 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.950 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:30.950 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.366971+0000", 2024-09-21T15:00:30.950 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2305e-05 2024-09-21T15:00:30.950 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.950 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.950 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:30.950 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.367034+0000", 2024-09-21T15:00:30.950 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3211000000000002e-05 2024-09-21T15:00:30.951 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.951 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.951 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:30.951 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.367056+0000", 2024-09-21T15:00:30.951 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2098000000000001e-05 2024-09-21T15:00:30.951 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.951 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.951 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:30.951 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.382400+0000", 2024-09-21T15:00:30.951 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.015343867000000001 2024-09-21T15:00:30.951 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.951 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.951 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:30.951 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.382452+0000", 2024-09-21T15:00:30.951 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1992e-05 2024-09-21T15:00:30.952 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:30.952 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:30.952 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:30.952 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.952 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.952 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:97734 3.19 e122/94 3:9fcf754f:::rbd_data.1136bfc79a70.0000000000000009:head v 122'314, mlcod=120'312)", 2024-09-21T15:00:30.952 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.378547+0000", 2024-09-21T15:00:30.952 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 25.516782841000001, 2024-09-21T15:00:30.952 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.063380675999999997, 2024-09-21T15:00:30.952 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:30.952 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:30.952 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:30.952 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.952 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:30.952 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.378547+0000", 2024-09-21T15:00:30.953 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:30.953 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.953 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.953 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:30.953 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.378547+0000", 2024-09-21T15:00:30.953 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:30.953 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.953 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.953 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:30.953 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.378547+0000", 2024-09-21T15:00:30.953 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:00:30.953 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.953 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.953 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:30.953 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.425118+0000", 2024-09-21T15:00:30.953 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046571166999999997 2024-09-21T15:00:30.954 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.954 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.954 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:30.954 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.425150+0000", 2024-09-21T15:00:30.954 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2150999999999998e-05 2024-09-21T15:00:30.954 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.954 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.954 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:30.954 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.425164+0000", 2024-09-21T15:00:30.954 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3302000000000001e-05 2024-09-21T15:00:30.954 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.954 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.954 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:30.954 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.425231+0000", 2024-09-21T15:00:30.954 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7106000000000004e-05 2024-09-21T15:00:30.955 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.955 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.955 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:30.955 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.425259+0000", 2024-09-21T15:00:30.955 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8087000000000001e-05 2024-09-21T15:00:30.955 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.955 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.955 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:30.955 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.441890+0000", 2024-09-21T15:00:30.955 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016631328000000001 2024-09-21T15:00:30.955 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.955 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.955 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:30.955 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.441928+0000", 2024-09-21T15:00:30.956 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7701999999999999e-05 2024-09-21T15:00:30.956 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:30.956 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:30.956 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:30.956 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.956 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.956 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:100187 3.10 3:08941a97:::rbd_data.1136ab4d6ff1.000000000000000e:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-21T15:00:30.956 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:10.869845+0000", 2024-09-21T15:00:30.956 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 20.025484944999999, 2024-09-21T15:00:30.956 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029345237, 2024-09-21T15:00:30.956 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:30.956 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:30.956 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:30.956 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:30.956 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:30.956 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 100187 2024-09-21T15:00:30.957 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.957 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:30.957 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.957 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:30.957 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869845+0000", 2024-09-21T15:00:30.957 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:30.957 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.957 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.957 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:30.957 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869845+0000", 2024-09-21T15:00:30.957 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:30.957 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.957 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.957 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:30.957 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869844+0000", 2024-09-21T15:00:30.957 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T15:00:30.958 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.958 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.958 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:30.958 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869853+0000", 2024-09-21T15:00:30.958 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5719999999999992e-06 2024-09-21T15:00:30.958 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.958 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.958 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:30.958 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869874+0000", 2024-09-21T15:00:30.958 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0483e-05 2024-09-21T15:00:30.958 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.958 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.958 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:30.958 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869883+0000", 2024-09-21T15:00:30.959 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6080000000000002e-06 2024-09-21T15:00:30.959 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.959 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.959 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:30.959 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869991+0000", 2024-09-21T15:00:30.959 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000108013 2024-09-21T15:00:30.959 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.959 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.959 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:30.959 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.870258+0000", 2024-09-21T15:00:30.959 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00026629099999999999 2024-09-21T15:00:30.959 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.959 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.959 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T15:00:30.959 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.878656+0000", 2024-09-21T15:00:30.960 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0083989000000000008 2024-09-21T15:00:30.960 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.960 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.960 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:30.960 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.899081+0000", 2024-09-21T15:00:30.960 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.020424437 2024-09-21T15:00:30.960 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.960 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.960 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:30.960 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.899125+0000", 2024-09-21T15:00:30.960 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4440999999999997e-05 2024-09-21T15:00:30.960 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.960 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.960 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:30.960 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.899160+0000", 2024-09-21T15:00:30.960 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4518999999999999e-05 2024-09-21T15:00:30.961 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.961 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.961 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:30.961 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.899190+0000", 2024-09-21T15:00:30.961 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0574999999999998e-05 2024-09-21T15:00:30.961 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:30.961 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:30.961 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:30.961 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.961 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.961 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:100263 3.10 3:0ece142e:::rbd_data.1136ab4d6ff1.0000000000000008:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-21T15:00:30.961 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:11.052257+0000", 2024-09-21T15:00:30.961 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 19.843072887000002, 2024-09-21T15:00:30.961 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042678025000000001, 2024-09-21T15:00:30.961 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:30.961 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:30.962 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:30.962 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:30.962 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:30.962 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 100263 2024-09-21T15:00:30.962 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.962 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:30.962 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.962 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:30.962 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052257+0000", 2024-09-21T15:00:30.962 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:30.962 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.962 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.962 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:30.962 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052257+0000", 2024-09-21T15:00:30.963 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:30.963 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.963 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.963 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:30.963 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052256+0000", 2024-09-21T15:00:30.963 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:00:30.963 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.963 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.963 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:30.963 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052263+0000", 2024-09-21T15:00:30.963 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4799999999999998e-06 2024-09-21T15:00:30.963 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.963 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.963 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:30.964 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052275+0000", 2024-09-21T15:00:30.964 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2386e-05 2024-09-21T15:00:30.964 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.964 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.964 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:30.964 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052281+0000", 2024-09-21T15:00:30.964 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6699999999999999e-06 2024-09-21T15:00:30.964 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.964 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.964 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:30.964 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052350+0000", 2024-09-21T15:00:30.964 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9099999999999999e-05 2024-09-21T15:00:30.964 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.964 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.965 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:30.965 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052456+0000", 2024-09-21T15:00:30.965 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000106669 2024-09-21T15:00:30.965 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.965 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.965 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T15:00:30.965 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.059694+0000", 2024-09-21T15:00:30.965 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.007237566 2024-09-21T15:00:30.965 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.965 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.965 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:30.965 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.094721+0000", 2024-09-21T15:00:30.965 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035026977000000001 2024-09-21T15:00:30.965 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.966 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.966 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:30.966 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.094746+0000", 2024-09-21T15:00:30.966 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5329000000000001e-05 2024-09-21T15:00:30.966 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.966 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.966 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:30.966 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.094850+0000", 2024-09-21T15:00:30.966 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000103452 2024-09-21T15:00:30.966 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.966 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.966 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:30.966 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.094935+0000", 2024-09-21T15:00:30.966 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5550000000000006e-05 2024-09-21T15:00:30.966 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:30.966 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:30.967 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:30.967 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.967 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.967 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:102266 3.1f e130/24 3:fec72c47:::rbd_data.1136d175325b.000000000000000c:head v 130'326, mlcod=128'324)", 2024-09-21T15:00:30.967 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:14.319401+0000", 2024-09-21T15:00:30.967 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 16.575928947000001, 2024-09-21T15:00:30.967 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021068982999999999, 2024-09-21T15:00:30.967 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:30.967 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:30.967 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:30.967 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.967 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:30.967 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.319401+0000", 2024-09-21T15:00:30.967 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:30.967 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.967 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.968 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:30.968 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.319401+0000", 2024-09-21T15:00:30.968 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:30.968 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.968 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.968 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:30.968 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.319400+0000", 2024-09-21T15:00:30.968 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:00:30.968 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.968 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.968 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:30.968 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.337267+0000", 2024-09-21T15:00:30.968 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.017867141 2024-09-21T15:00:30.968 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.968 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.968 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:30.969 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.337278+0000", 2024-09-21T15:00:30.969 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0224000000000001e-05 2024-09-21T15:00:30.969 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.969 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.969 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:30.969 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.337285+0000", 2024-09-21T15:00:30.969 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.25e-06 2024-09-21T15:00:30.969 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.969 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.969 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:30.969 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.337330+0000", 2024-09-21T15:00:30.969 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5210000000000003e-05 2024-09-21T15:00:30.969 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.969 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.969 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:30.969 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.337350+0000", 2024-09-21T15:00:30.970 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9556e-05 2024-09-21T15:00:30.970 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.970 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.970 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:30.970 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.340418+0000", 2024-09-21T15:00:30.970 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.003068552 2024-09-21T15:00:30.970 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.970 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.970 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:30.970 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.340470+0000", 2024-09-21T15:00:30.970 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1897000000000001e-05 2024-09-21T15:00:30.970 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:30.970 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:30.970 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:30.970 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.970 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.971 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:102887 3.1c 3:3b11ae99:::rbd_object_map.1136d175325b.0000000000000050:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-21T15:00:30.971 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:14.368644+0000", 2024-09-21T15:00:30.971 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 16.526685911000001, 2024-09-21T15:00:30.971 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.017021290000000001, 2024-09-21T15:00:30.971 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:30.971 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:30.971 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:30.971 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:30.971 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:30.971 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 102887 2024-09-21T15:00:30.971 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.971 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:30.971 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.971 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:30.971 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.368644+0000", 2024-09-21T15:00:30.971 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:30.972 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.972 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.972 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:30.972 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.368644+0000", 2024-09-21T15:00:30.972 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:30.972 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.972 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.972 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:30.972 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.368642+0000", 2024-09-21T15:00:30.972 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T15:00:30.972 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.972 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.972 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:30.972 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.368655+0000", 2024-09-21T15:00:30.972 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2808e-05 2024-09-21T15:00:30.972 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.973 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.973 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:30.973 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.368677+0000", 2024-09-21T15:00:30.973 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2753999999999999e-05 2024-09-21T15:00:30.973 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.973 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.973 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:30.973 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.368688+0000", 2024-09-21T15:00:30.973 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1228e-05 2024-09-21T15:00:30.973 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.973 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.973 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:30.973 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.368743+0000", 2024-09-21T15:00:30.973 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4608999999999998e-05 2024-09-21T15:00:30.973 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.974 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.974 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:30.974 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.368868+0000", 2024-09-21T15:00:30.974 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012473000000000001 2024-09-21T15:00:30.974 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.974 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.974 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 0", 2024-09-21T15:00:30.974 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.369385+0000", 2024-09-21T15:00:30.974 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00051684400000000003 2024-09-21T15:00:30.974 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.974 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.974 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:30.974 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.370755+0000", 2024-09-21T15:00:30.974 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0013701609999999999 2024-09-21T15:00:30.974 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.974 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.975 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:30.975 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.385531+0000", 2024-09-21T15:00:30.975 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014776648 2024-09-21T15:00:30.975 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.975 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.975 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:30.975 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.385575+0000", 2024-09-21T15:00:30.975 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3726000000000001e-05 2024-09-21T15:00:30.975 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.975 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.975 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:30.975 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.385666+0000", 2024-09-21T15:00:30.975 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0304999999999997e-05 2024-09-21T15:00:30.975 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:30.975 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:30.975 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:30.976 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.976 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.976 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:103706 3.1c 3:3b11ae99:::rbd_object_map.1136d175325b.0000000000000050:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-21T15:00:30.976 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:14.426370+0000", 2024-09-21T15:00:30.976 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 16.468960574, 2024-09-21T15:00:30.976 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.015478897, 2024-09-21T15:00:30.976 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:30.976 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:30.976 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:30.976 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:30.976 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:30.976 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 103706 2024-09-21T15:00:30.976 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.976 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:30.976 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.976 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:30.977 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.426370+0000", 2024-09-21T15:00:30.977 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:30.977 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.977 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.977 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:30.977 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.426370+0000", 2024-09-21T15:00:30.977 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:30.977 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.977 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.977 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:30.977 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.426369+0000", 2024-09-21T15:00:30.977 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:00:30.977 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.977 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.977 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:30.977 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.426376+0000", 2024-09-21T15:00:30.977 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0999999999999998e-06 2024-09-21T15:00:30.978 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.978 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.978 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:30.978 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.426442+0000", 2024-09-21T15:00:30.978 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6445999999999994e-05 2024-09-21T15:00:30.978 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.978 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.978 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:30.978 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.426451+0000", 2024-09-21T15:00:30.978 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4940000000000003e-06 2024-09-21T15:00:30.978 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.978 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.978 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:30.978 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.426493+0000", 2024-09-21T15:00:30.978 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2094000000000001e-05 2024-09-21T15:00:30.979 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.979 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.979 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:30.979 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.426560+0000", 2024-09-21T15:00:30.979 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6774999999999997e-05 2024-09-21T15:00:30.979 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.979 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.979 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 0", 2024-09-21T15:00:30.979 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.426737+0000", 2024-09-21T15:00:30.979 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017769899999999999 2024-09-21T15:00:30.979 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.979 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.979 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:30.979 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.428561+0000", 2024-09-21T15:00:30.979 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001823917 2024-09-21T15:00:30.979 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.979 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.980 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:30.980 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.441762+0000", 2024-09-21T15:00:30.980 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013200462 2024-09-21T15:00:30.980 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.980 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.980 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:30.980 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.441811+0000", 2024-09-21T15:00:30.980 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8912000000000002e-05 2024-09-21T15:00:30.980 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.980 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.980 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:30.980 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.441848+0000", 2024-09-21T15:00:30.980 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7935000000000002e-05 2024-09-21T15:00:30.980 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:30.980 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:30.980 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:30.981 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.981 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.981 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:105083 3.1a 3:5d63845a:::rbd_data.1136d175325b.0000000000000037:head [write 0~32768 [fadvise_sequential+fadvise_nocache],write 65536~163840 [fadvise_sequential+fadvise_nocache],write 262144~32768 [fadvise_sequential+fadvise_nocache],write 327680~163840 [fadvise_sequential+fadvise_nocache],write 524288~32768 [fadvise_sequential+fadvise_nocache],write 589824~163840 [fadvise_sequential+fadvise_nocache],write 786432~32768 [fadvise_sequential+fadvise_nocache],write 851968~32768 [fadvise_sequential+fadvise_nocache],write 917504~98304 [fadvise_sequential+fadvise_nocache],write 1048576~32768 [fadvise_sequential+fadvise_nocache],write 1114112~32768 [fadvise_sequential+fadvise_nocache],write 1179648~98304 [fadvise_sequential+fadvise_nocache],write 1310720~32768 [fadvise_sequential+fadvise_nocache],write 1376256~32768 [fadvise_sequential+fadvise_nocache],write 1441792~98304 [fadvise_sequential+fadvise_nocache],write 1572864~32768 [fadvise_sequential+fadvise_nocache],write 1638400~32768 [fadvise_sequential+fadvise_nocache],write 1703936~98304 [fadvise_sequential+fadvise_nocache],write 1835008~32768 [fadvise_sequential+fadvise_nocache],write 1900544~32768 [fadvise_sequential+fadvise_nocache],write 1966080~98304 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-21T15:00:30.981 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:14.534201+0000", 2024-09-21T15:00:30.981 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 16.361129233, 2024-09-21T15:00:30.981 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016474194000000001, 2024-09-21T15:00:30.981 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:30.981 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:30.981 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:30.981 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:30.981 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:30.981 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 105083 2024-09-21T15:00:30.981 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.981 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:30.981 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.981 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:30.982 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.534201+0000", 2024-09-21T15:00:30.982 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:30.982 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.982 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.982 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:30.982 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.534201+0000", 2024-09-21T15:00:30.982 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:30.982 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.982 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.982 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:30.982 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.534201+0000", 2024-09-21T15:00:30.982 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:30.983 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.983 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.983 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:30.983 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.534853+0000", 2024-09-21T15:00:30.983 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00065227099999999995 2024-09-21T15:00:30.983 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.983 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.983 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:30.983 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.534866+0000", 2024-09-21T15:00:30.983 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3553000000000001e-05 2024-09-21T15:00:30.983 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.983 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.983 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:30.983 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.534877+0000", 2024-09-21T15:00:30.983 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0186e-05 2024-09-21T15:00:30.984 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.984 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.984 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:30.984 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.534947+0000", 2024-09-21T15:00:30.984 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0417e-05 2024-09-21T15:00:30.984 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.984 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.984 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:30.984 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.535138+0000", 2024-09-21T15:00:30.984 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00019122200000000001 2024-09-21T15:00:30.984 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.984 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.984 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T15:00:30.984 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.535666+0000", 2024-09-21T15:00:30.985 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00052817499999999998 2024-09-21T15:00:30.985 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.985 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.985 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:30.985 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.543252+0000", 2024-09-21T15:00:30.985 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0075857889999999999 2024-09-21T15:00:30.985 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.985 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.985 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:30.985 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.550552+0000", 2024-09-21T15:00:30.985 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0072995869999999997 2024-09-21T15:00:30.985 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.985 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.985 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:30.985 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.550637+0000", 2024-09-21T15:00:30.986 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5750999999999999e-05 2024-09-21T15:00:30.986 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.986 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.986 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:30.986 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.550675+0000", 2024-09-21T15:00:30.986 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7602999999999999e-05 2024-09-21T15:00:30.986 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:30.986 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:30.986 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:30.986 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.986 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.986 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:113281 3.10 3:0e1b39db:::rbd_data.1136b442e94a.0000000000000009:head [write 0~360448 [fadvise_sequential+fadvise_nocache],write 376832~573440 [fadvise_sequential+fadvise_nocache],write 1138688~573440 [fadvise_sequential+fadvise_nocache],write 1900544~196608 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:30.986 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.487425+0000", 2024-09-21T15:00:30.986 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.4079052910000001, 2024-09-21T15:00:30.986 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.015461361999999999, 2024-09-21T15:00:30.987 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:30.987 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:30.987 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:30.987 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:30.987 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:30.987 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 113281 2024-09-21T15:00:30.987 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.987 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:30.987 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.987 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:30.987 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.487425+0000", 2024-09-21T15:00:30.987 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:30.987 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.988 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.988 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:30.988 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.487425+0000", 2024-09-21T15:00:30.988 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:30.988 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.988 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.988 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:30.988 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.487423+0000", 2024-09-21T15:00:30.988 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-21T15:00:30.988 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.988 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.988 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:30.988 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.488151+0000", 2024-09-21T15:00:30.988 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00072770699999999996 2024-09-21T15:00:30.988 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.989 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.989 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:30.989 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.488175+0000", 2024-09-21T15:00:30.989 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3896e-05 2024-09-21T15:00:30.989 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.989 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.989 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:30.989 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.488184+0000", 2024-09-21T15:00:30.989 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9000000000000001e-06 2024-09-21T15:00:30.989 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.989 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.989 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:30.989 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.488918+0000", 2024-09-21T15:00:30.990 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00073320500000000001 2024-09-21T15:00:30.990 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.990 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.990 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:30.990 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.489175+0000", 2024-09-21T15:00:30.990 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00025761799999999999 2024-09-21T15:00:30.990 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.990 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.990 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T15:00:30.990 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.489479+0000", 2024-09-21T15:00:30.990 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00030419099999999999 2024-09-21T15:00:30.990 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.990 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.991 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:30.991 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.498472+0000", 2024-09-21T15:00:30.991 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0089922319999999993 2024-09-21T15:00:30.991 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.991 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.991 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:30.991 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502751+0000", 2024-09-21T15:00:30.991 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0042795309999999996 2024-09-21T15:00:30.991 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.991 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.991 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:30.992 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502843+0000", 2024-09-21T15:00:30.992 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1926e-05 2024-09-21T15:00:30.992 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:30.992 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:30.992 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:30.992 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502886+0000", 2024-09-21T15:00:30.992 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3087999999999998e-05 2024-09-21T15:00:30.992 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:30.992 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:30.992 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:30.992 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:30.992 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:30.992 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:30.994 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T15:00:30.994 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T15:00:30.994 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T15:00:30.995 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T15:00:30.995 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T15:00:30.995 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:00:30.991+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:00:31.017 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-21T15:00:31.042 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T15:00:31.073 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T15:00:31.094 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:00:31.090+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:00:31.102 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T15:00:31.104 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:00:31.105 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:00:31.105 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:00:31.105 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:00:31.105 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:00:31.105 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:00:31.105 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:00:31.105 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:00:31.106 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:31.106 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:31.116 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T15:00:31.195 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:00:31.191+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:00:31.270 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:31.270 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:31.280 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T15:00:31.296 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:00:31.294+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:00:31.313 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T15:00:31.397 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:00:31.390+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:00:31.421 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:31.421 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:31.432 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T15:00:31.498 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:00:31.494+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:00:31.536 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:31.536 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:31.546 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T15:00:31.599 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:00:31.595+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:00:31.654 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:31.654 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:31.664 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T15:00:31.709 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:00:31.707+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:00:31.767 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:31.767 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:31.777 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T15:00:31.822 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:00:31.818+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:00:31.895 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:31.895 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:31.902 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:00:31.899+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:00:31.905 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T15:00:32.003 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:00:31.999+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:00:32.009 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:32.009 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:32.019 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T15:00:32.103 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:00:32.103+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:00:32.128 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:32.128 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:32.138 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T15:00:32.204 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:00:32.199+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:00:32.241 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:32.242 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:32.251 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T15:00:32.304 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:00:32.303+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:00:32.333 INFO:teuthology.orchestra.run.smithi032.stderr:nodeep-scrub is set 2024-09-21T15:00:32.363 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:32.363 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:32.373 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T15:00:32.405 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:00:32.398+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:00:32.474 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:32.474 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:32.483 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T15:00:32.506 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:00:32.503+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:00:32.595 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:32.595 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:32.606 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T15:00:32.607 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:00:32.602+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:00:32.707 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:00:32.703+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:00:32.781 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:32.781 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:32.792 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T15:00:32.808 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:00:32.803+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:00:32.908 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:00:32.902+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:00:32.915 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:32.915 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:32.924 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T15:00:32.939 INFO:teuthology.orchestra.run.smithi032.stdout:3907 trunc from 0x8ba2f36 to 0x1778a53 2024-09-21T15:00:32.981 INFO:teuthology.orchestra.run.smithi032.stdout:3908 read 0x1011074 thru 0x1017f3e (0x6ecb bytes) 2024-09-21T15:00:32.982 INFO:teuthology.orchestra.run.smithi032.stdout:3909 read 0x13a0e6 thru 0x13f0bf (0x4fda bytes) 2024-09-21T15:00:33.003 INFO:teuthology.orchestra.run.smithi032.stdout:3910 write 0x881731d thru 0x881d2f7 (0x5fdb bytes) 2024-09-21T15:00:33.009 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:00:33.002+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:00:33.014 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:33.014 INFO:teuthology.orchestra.run.smithi032.stdout:3913 read 0xae97c5 thru 0xaea5fa (0xe36 bytes) 2024-09-21T15:00:33.015 INFO:teuthology.orchestra.run.smithi032.stdout:3916 read 0x733c25d thru 0x734ab3e (0xe8e2 bytes) 2024-09-21T15:00:33.015 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:33.026 INFO:teuthology.orchestra.run.smithi032.stdout:3918 trunc from 0x881d2f8 to 0xea052b6 2024-09-21T15:00:33.038 INFO:teuthology.orchestra.run.smithi032.stdout:3920 read 0xd62e433 thru 0xd6386ee (0xa2bc bytes) 2024-09-21T15:00:33.038 INFO:teuthology.orchestra.run.smithi032.stdout:3921 write 0xb0e07d6 thru 0xb0ecb11 (0xc33c bytes) 2024-09-21T15:00:33.038 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:33.039 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:33.049 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T15:00:33.054 INFO:teuthology.orchestra.run.smithi032.stdout:3922 write 0xa262a68 thru 0xa272808 (0xfda1 bytes) 2024-09-21T15:00:33.069 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:33.069 INFO:teuthology.orchestra.run.smithi032.stdout:3924 read 0x3b5bd5b thru 0x3b5e72e (0x29d4 bytes) 2024-09-21T15:00:33.069 INFO:teuthology.orchestra.run.smithi032.stdout:3925 read 0x600b29a thru 0x600f6cf (0x4436 bytes) 2024-09-21T15:00:33.069 INFO:teuthology.orchestra.run.smithi032.stdout:3929 punch from 0x3c63417 to 0x3c72f34, (0xfb1d bytes) 2024-09-21T15:00:33.079 INFO:teuthology.orchestra.run.smithi032.stdout:3930 trunc from 0xea052b6 to 0x91238af 2024-09-21T15:00:33.105 INFO:teuthology.orchestra.run.smithi032.stdout:3931 read 0x4c2a6a4 thru 0x4c2cd8e (0x26eb bytes) 2024-09-21T15:00:33.105 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:33.110 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:00:33.106+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:00:33.113 INFO:teuthology.orchestra.run.smithi032.stdout:3935 trunc from 0x91238af to 0x5249f85 2024-09-21T15:00:33.117 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T15:00:33.148 INFO:teuthology.orchestra.run.smithi032.stdout:3936 trunc from 0x5249f85 to 0xc355c9a 2024-09-21T15:00:33.159 INFO:teuthology.orchestra.run.smithi032.stdout:3938 writesame 0x5e6567d thru 0x5e66ee6 data_size 0x19(0x186a bytes) 2024-09-21T15:00:33.170 INFO:teuthology.orchestra.run.smithi032.stdout:3941 trunc from 0xc355c9a to 0xac7b659 2024-09-21T15:00:33.181 INFO:teuthology.orchestra.run.smithi032.stdout:3942 read 0xfb09dd thru 0xfbedd3 (0xe3f7 bytes) 2024-09-21T15:00:33.183 INFO:teuthology.orchestra.run.smithi032.stdout:3943 write 0x4e0c4cc thru 0x4e165f2 (0xa127 bytes) 2024-09-21T15:00:33.207 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:33.207 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:33.210 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:00:33.206+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:00:33.211 INFO:teuthology.orchestra.run.smithi032.stdout:3944 trunc from 0xac7b659 to 0x1b0f712 2024-09-21T15:00:33.218 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T15:00:33.234 INFO:teuthology.orchestra.run.smithi032.stdout:3945 read 0xca7f86 thru 0xcb7c75 (0xfcf0 bytes) 2024-09-21T15:00:33.235 INFO:teuthology.orchestra.run.smithi032.stdout:3946 writesame 0x1a16fa thru 0x1abab8 data_size 0x3(0xa3bf bytes) 2024-09-21T15:00:33.245 INFO:teuthology.orchestra.run.smithi032.stdout:3947 read 0x405e7d thru 0x40fc09 (0x9d8d bytes) 2024-09-21T15:00:33.250 INFO:teuthology.orchestra.run.smithi032.stdout:3948 trunc from 0x1b0f712 to 0x35d1e77 2024-09-21T15:00:33.267 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T15:00:33.272 INFO:teuthology.orchestra.run.smithi032.stdout:3949 write 0x61ef43c thru 0x61fc8ba (0xd47f bytes) 2024-09-21T15:00:33.287 INFO:teuthology.orchestra.run.smithi032.stdout:3950 read 0x169c9e2 thru 0x16ab92b (0xef4a bytes) 2024-09-21T15:00:33.290 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T15:00:33.301 INFO:teuthology.orchestra.run.smithi032.stdout:3953 write 0x9baab06 thru 0x9bb10dd (0x65d8 bytes) 2024-09-21T15:00:33.311 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T15:00:33.311 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:00:33.307+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:00:33.313 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:33.313 INFO:teuthology.orchestra.run.smithi032.stdout:3958 punch from 0x6cda2dd to 0x6cdea2d, (0x4750 bytes) 2024-09-21T15:00:33.313 INFO:teuthology.orchestra.run.smithi032.stdout:3959 read 0x3d4fc42 thru 0x3d5877e (0x8b3d bytes) 2024-09-21T15:00:33.313 INFO:teuthology.orchestra.run.smithi032.stdout:3960 punch from 0x6014bef to 0x60200c0, (0xb4d1 bytes) 2024-09-21T15:00:33.313 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:33.313 INFO:teuthology.orchestra.run.smithi032.stdout:3964 read 0x15fff57 thru 0x1608717 (0x87c1 bytes) 2024-09-21T15:00:33.314 INFO:teuthology.orchestra.run.smithi032.stdout:3965 write 0x9810f16 thru 0x9819fa2 (0x908d bytes) 2024-09-21T15:00:33.329 INFO:teuthology.orchestra.run.smithi032.stdout:3966 write 0x451864f thru 0x451c571 (0x3f23 bytes) 2024-09-21T15:00:33.336 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T15:00:33.357 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-21T15:00:33.376 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:33.376 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:33.379 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T15:00:33.385 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T15:00:33.401 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T15:00:33.413 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:00:33.406+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:00:33.425 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T15:00:33.427 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:00:33.427 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:00:33.427 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:00:33.427 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:00:33.427 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:00:33.427 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:00:33.427 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:00:33.427 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:00:33.496 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:33.496 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:00:33.497 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T15:00:33.497 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:33.506 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T15:00:33.514 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:00:33.506+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:00:33.615 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:00:33.611+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:00:33.616 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:33.616 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:00:33.616 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T15:00:33.616 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T15:00:33.617 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:33.626 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T15:00:33.716 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:00:33.714+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:00:33.734 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:33.734 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T15:00:33.734 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T15:00:33.734 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T15:00:33.734 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.734 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:97677 3.15 3:a8e72133:::rbd_data.1136bfc79a70.0000000000000005:head [write 0~4440027 [fadvise_sequential+fadvise_nocache],write 4751360~147456 [fadvise_sequential+fadvise_nocache],write 4964352~147456 [fadvise_sequential+fadvise_nocache],write 5177344~65536 [fadvise_sequential+fadvise_nocache],write 5259264~147456 [fadvise_sequential+fadvise_nocache],write 5472256~147456 [fadvise_sequential+fadvise_nocache],write 5685248~229376 [fadvise_sequential+fadvise_nocache],write 5980160~147456 [fadvise_sequential+fadvise_nocache],write 6193152~229376 [fadvise_sequential+fadvise_nocache],write 6488064~147456 [fadvise_sequential+fadvise_nocache],write 6701056~229376 [fadvise_sequential+fadvise_nocache],write 6995968~147456 [fadvise_sequential+fadvise_nocache],write 7208960~229376 [fadvise_sequential+fadvise_nocache],write 7503872~147456 [fadvise_sequential+fadvise_nocache],write 7716864~229376 [fadvise_sequential+fadvise_nocache],write 8044544~114688 [fadvise_sequential+fadvise_nocache],write 8257536~114688 [fadvise_sequential+fadvise_nocache],write 8388608~65536 [fadvise_sequential+fadvise_nocache],write 8552448~114688 [fadvise_sequential+fadvise_nocache],write 8765440~114688 [fadvise_sequential+fadvise_nocache],write 8929280~32768 [fadvise_sequential+fadvise_nocache],write 9142272~32768 [fadvise_sequential+fadvise_nocache],write 9355264~32768 [fadvise_sequential+fadvise_nocache],write 9437184~32768 [fadvise_sequential+fadvise_nocache],write 9650176~32768 [fadvise_sequential+fadvise_nocache],write 9863168~32768 [fadvise_sequential+fadvise_nocache],write 9961472~131072 [fadvise_sequential+fadvise_nocache],write 10108928~196608 [fadvise_sequential+fadvise_nocache],write 10321920~278528 [fadvise_sequential+fadvise_nocache],write 10616832~196608 [fadvise_sequential+fadvise_nocache],write 10829824~278528 [fadvise_sequential+fadvise_nocache],write 11124736~196608 [fadvise_sequential+fadvise_nocache],write 11337728~196608 [fadvise_sequential+fadvise_nocache],write 11550720~65536 [fadvise_sequential+fadvise_nocache],write 11649024~98304 [fadvise_sequential+fadvise_nocache],write 11763712~65536 [fadvise_sequential+fadvise_nocache],write 11862016~98304 [fadvise_sequential+fadvise_nocache],write 11976704~65536 [fadvise_sequential+fadvise_nocache],write 12058624~65536 [fadvise_sequential+fadvise_nocache],write 12156928~98304 [fadvise_sequential+fadvise_nocache],write 12271616~65536 [fadvise_sequential+fadvise_nocache],write 12369920~98304 [fadvise_sequential+fadvise_nocache],write 12484608~65536 [fadvise_sequential+fadvise_nocache],write 12664832~98304 [fadvise_sequential+fadvise_nocache],write 12877824~98304 [fadvise_sequential+fadvise_nocache],write 13090816~16384 [fadvise_sequential+fadvise_nocache],write 13172736~98304 [fadvise_sequential+fadvise_nocache],write 13385728~98304 [fadvise_sequential+fadvise_nocache],write 13598720~32768 [fadvise_sequential+fadvise_nocache],write 13680640~98304 [fadvise_sequential+fadvise_nocache],write 13893632~98304 [fadvise_sequential+fadvise_nocache],write 14106624~49152 [fadvise_sequential+fadvise_nocache],write 14794752~81920 [fadvise_sequential+fadvise_nocache],write 15007744~81920 [fadvise_sequential+fadvise_nocache],write 15302656~81920 [fadvise_sequential+fadvise_nocache],write 15515648~81920 [fadvise_sequential+fadvise_nocache],write 15810560~81920 [fadvise_sequential+fadvise_nocache],write 16023552~81920 [fadvise_sequential+fadvise_nocache],write 16236544~16384 [fadvise_sequential+fadvise_nocache],write 16318464~81920 [fadvise_sequential+fadvise_nocache],write 16531456~81920 [fadvise_sequential+fadvise_nocache],write 16744448~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:33.734 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.302946+0000", 2024-09-21T15:00:33.734 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 28.430411495000001, 2024-09-21T15:00:33.734 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.091231204999999996, 2024-09-21T15:00:33.734 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:33.734 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:33.734 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:33.735 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:33.735 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:33.735 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 97677 2024-09-21T15:00:33.735 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.735 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:33.735 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.735 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:33.735 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.302946+0000", 2024-09-21T15:00:33.735 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:33.735 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.735 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.735 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:33.735 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.302946+0000", 2024-09-21T15:00:33.735 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:33.735 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.736 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.736 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:33.736 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.302942+0000", 2024-09-21T15:00:33.736 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999962 2024-09-21T15:00:33.736 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.736 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.736 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:33.736 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311253+0000", 2024-09-21T15:00:33.736 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0083105360000000003 2024-09-21T15:00:33.736 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.736 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.736 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:33.736 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311283+0000", 2024-09-21T15:00:33.736 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0065000000000001e-05 2024-09-21T15:00:33.736 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.737 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.737 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:33.737 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311296+0000", 2024-09-21T15:00:33.737 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3206e-05 2024-09-21T15:00:33.737 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.737 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.737 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:33.737 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311404+0000", 2024-09-21T15:00:33.737 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000107514 2024-09-21T15:00:33.737 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.737 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.737 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:33.737 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311795+0000", 2024-09-21T15:00:33.737 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000390913 2024-09-21T15:00:33.737 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.738 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.738 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 6", 2024-09-21T15:00:33.738 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.314686+0000", 2024-09-21T15:00:33.738 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0028915659999999999 2024-09-21T15:00:33.738 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.738 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.738 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:33.738 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.368178+0000", 2024-09-21T15:00:33.738 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.053491706 2024-09-21T15:00:33.738 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.738 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.738 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:33.738 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.393925+0000", 2024-09-21T15:00:33.738 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025747289 2024-09-21T15:00:33.739 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.739 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.739 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:33.739 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.394127+0000", 2024-09-21T15:00:33.739 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000201444 2024-09-21T15:00:33.739 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.739 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.739 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:33.739 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.394177+0000", 2024-09-21T15:00:33.739 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0689000000000002e-05 2024-09-21T15:00:33.739 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:33.739 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:33.739 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:33.739 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.739 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.740 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:97673 3.1d e122/24 3:bd91946e:::rbd_data.1136bfc79a70.0000000000000002:head v 122'162, mlcod=120'160)", 2024-09-21T15:00:33.740 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.313730+0000", 2024-09-21T15:00:33.740 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 28.419627169999998, 2024-09-21T15:00:33.740 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055373462999999998, 2024-09-21T15:00:33.740 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:33.740 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:33.740 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:33.740 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.740 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:33.740 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.313730+0000", 2024-09-21T15:00:33.740 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:33.740 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.740 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.740 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:33.740 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.313730+0000", 2024-09-21T15:00:33.741 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:33.741 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.741 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.741 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:33.741 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.313730+0000", 2024-09-21T15:00:33.741 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:33.741 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.741 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.741 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:33.741 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.336113+0000", 2024-09-21T15:00:33.741 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022382962999999999 2024-09-21T15:00:33.741 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.741 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.741 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:33.742 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.336140+0000", 2024-09-21T15:00:33.742 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7248999999999999e-05 2024-09-21T15:00:33.742 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.742 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.742 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:33.742 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.336151+0000", 2024-09-21T15:00:33.742 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1015e-05 2024-09-21T15:00:33.742 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.742 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.742 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:33.742 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.355428+0000", 2024-09-21T15:00:33.742 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019276379999999999 2024-09-21T15:00:33.742 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.742 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.742 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:33.743 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.355484+0000", 2024-09-21T15:00:33.743 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6820999999999997e-05 2024-09-21T15:00:33.743 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.743 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.743 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:33.743 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.369040+0000", 2024-09-21T15:00:33.743 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013555466 2024-09-21T15:00:33.743 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.743 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.743 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:33.743 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.369104+0000", 2024-09-21T15:00:33.743 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3986999999999996e-05 2024-09-21T15:00:33.743 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:33.743 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:33.744 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:33.744 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.744 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.744 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:113889 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 5300224~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:33.744 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.501880+0000", 2024-09-21T15:00:33.744 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.2314772569999999, 2024-09-21T15:00:33.744 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052644204999999999, 2024-09-21T15:00:33.746 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:33.746 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:33.746 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:33.746 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:33.746 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:33.746 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 113889 2024-09-21T15:00:33.746 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.746 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:33.747 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.747 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:33.747 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501880+0000", 2024-09-21T15:00:33.747 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:33.747 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.747 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.747 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:33.747 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501880+0000", 2024-09-21T15:00:33.747 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:33.747 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.747 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.747 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:33.747 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501880+0000", 2024-09-21T15:00:33.747 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:00:33.747 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.748 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.748 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:33.748 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501882+0000", 2024-09-21T15:00:33.748 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.364e-06 2024-09-21T15:00:33.748 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.748 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.748 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:33.748 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501885+0000", 2024-09-21T15:00:33.748 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4199999999999999e-06 2024-09-21T15:00:33.748 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.748 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.748 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:33.748 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501888+0000", 2024-09-21T15:00:33.748 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8669999999999998e-06 2024-09-21T15:00:33.748 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.749 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.749 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:33.749 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.554369+0000", 2024-09-21T15:00:33.749 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052481456000000003 2024-09-21T15:00:33.749 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.749 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.749 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:33.749 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.554449+0000", 2024-09-21T15:00:33.749 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9412000000000005e-05 2024-09-21T15:00:33.749 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.749 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.749 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:33.749 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.554525+0000", 2024-09-21T15:00:33.749 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5908999999999994e-05 2024-09-21T15:00:33.749 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:33.750 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:33.750 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:33.750 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.750 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.750 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:113890 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 5390336~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:33.750 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.501890+0000", 2024-09-21T15:00:33.750 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.2314675419999999, 2024-09-21T15:00:33.750 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052785212999999997, 2024-09-21T15:00:33.750 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:33.750 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:33.750 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:33.750 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:33.750 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:33.750 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 113890 2024-09-21T15:00:33.750 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.750 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:33.751 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.751 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:33.751 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501890+0000", 2024-09-21T15:00:33.751 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:33.751 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.751 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.751 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:33.751 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501890+0000", 2024-09-21T15:00:33.751 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:33.751 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.751 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.751 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:33.751 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501890+0000", 2024-09-21T15:00:33.751 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:00:33.752 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.752 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.752 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:33.752 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501891+0000", 2024-09-21T15:00:33.752 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3599999999999999e-06 2024-09-21T15:00:33.752 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.752 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.752 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:33.752 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501894+0000", 2024-09-21T15:00:33.752 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.717e-06 2024-09-21T15:00:33.752 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.752 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.752 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:33.752 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501897+0000", 2024-09-21T15:00:33.752 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5790000000000002e-06 2024-09-21T15:00:33.753 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.753 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.753 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:33.753 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.554569+0000", 2024-09-21T15:00:33.753 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052672434999999997 2024-09-21T15:00:33.753 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.753 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.753 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:33.753 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.554623+0000", 2024-09-21T15:00:33.753 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4509999999999998e-05 2024-09-21T15:00:33.753 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.753 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.753 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:33.753 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.554675+0000", 2024-09-21T15:00:33.753 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1833999999999999e-05 2024-09-21T15:00:33.753 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:33.753 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:33.754 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:33.754 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.754 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.754 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:113891 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 5480448~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:33.754 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.501899+0000", 2024-09-21T15:00:33.754 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.2314590560000003, 2024-09-21T15:00:33.754 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052991637000000001, 2024-09-21T15:00:33.754 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:33.754 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:33.754 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:33.754 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:33.754 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:33.754 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 113891 2024-09-21T15:00:33.754 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.754 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:33.755 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.755 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:33.755 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501899+0000", 2024-09-21T15:00:33.755 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:33.755 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.755 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.755 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:33.755 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501899+0000", 2024-09-21T15:00:33.755 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:33.755 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.755 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.755 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:33.755 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501898+0000", 2024-09-21T15:00:33.755 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:00:33.755 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.756 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.756 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:33.756 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501900+0000", 2024-09-21T15:00:33.756 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.356e-06 2024-09-21T15:00:33.756 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.756 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.756 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:33.756 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501902+0000", 2024-09-21T15:00:33.756 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6620000000000001e-06 2024-09-21T15:00:33.756 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.756 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.756 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:33.756 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501905+0000", 2024-09-21T15:00:33.756 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.492e-06 2024-09-21T15:00:33.756 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.757 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.757 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:33.757 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.554707+0000", 2024-09-21T15:00:33.757 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052802618000000003 2024-09-21T15:00:33.757 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.757 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.757 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:33.757 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.554798+0000", 2024-09-21T15:00:33.757 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0976000000000004e-05 2024-09-21T15:00:33.757 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.757 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.757 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:33.757 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.554890+0000", 2024-09-21T15:00:33.757 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1755999999999994e-05 2024-09-21T15:00:33.757 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:33.758 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:33.758 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:33.758 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.758 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.758 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:113892 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 5570560~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:33.758 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.501907+0000", 2024-09-21T15:00:33.758 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.2314506239999998, 2024-09-21T15:00:33.758 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.053089242000000002, 2024-09-21T15:00:33.758 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:33.758 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:33.758 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:33.758 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:33.758 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:33.758 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 113892 2024-09-21T15:00:33.758 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.759 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:33.759 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.759 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:33.759 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501907+0000", 2024-09-21T15:00:33.759 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:33.759 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.759 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.759 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:33.759 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501907+0000", 2024-09-21T15:00:33.759 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:33.759 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.759 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.759 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:33.759 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501907+0000", 2024-09-21T15:00:33.759 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:00:33.760 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.760 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.760 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:33.760 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501908+0000", 2024-09-21T15:00:33.760 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2860000000000001e-06 2024-09-21T15:00:33.760 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.760 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.760 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:33.760 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501913+0000", 2024-09-21T15:00:33.760 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1320000000000002e-06 2024-09-21T15:00:33.760 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.760 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.760 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:33.760 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501916+0000", 2024-09-21T15:00:33.760 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5330000000000001e-06 2024-09-21T15:00:33.761 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.761 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.761 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:33.761 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.554904+0000", 2024-09-21T15:00:33.761 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052988631000000001 2024-09-21T15:00:33.761 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.761 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.761 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:33.761 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.554945+0000", 2024-09-21T15:00:33.761 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.0695999999999998e-05 2024-09-21T15:00:33.761 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.761 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.761 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:33.761 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.554996+0000", 2024-09-21T15:00:33.761 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1193999999999998e-05 2024-09-21T15:00:33.762 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:33.762 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:33.762 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:33.762 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.762 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.762 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:113893 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 5660672~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:33.762 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.501918+0000", 2024-09-21T15:00:33.762 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.2314396470000002, 2024-09-21T15:00:33.762 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.053245595, 2024-09-21T15:00:33.762 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:33.762 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:33.762 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:33.762 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:33.762 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:33.763 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 113893 2024-09-21T15:00:33.763 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.763 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:33.763 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.763 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:33.763 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501918+0000", 2024-09-21T15:00:33.763 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:33.763 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.763 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.763 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:33.763 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501918+0000", 2024-09-21T15:00:33.763 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:33.763 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.763 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.764 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:33.764 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501918+0000", 2024-09-21T15:00:33.764 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:33.764 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.764 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.764 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:33.764 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501920+0000", 2024-09-21T15:00:33.764 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9800000000000001e-06 2024-09-21T15:00:33.764 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.764 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.764 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:33.764 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501928+0000", 2024-09-21T15:00:33.764 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7999999999999999e-06 2024-09-21T15:00:33.764 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.764 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.765 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:33.765 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501930+0000", 2024-09-21T15:00:33.765 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.785e-06 2024-09-21T15:00:33.765 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.765 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.765 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:33.765 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555019+0000", 2024-09-21T15:00:33.765 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.053089154 2024-09-21T15:00:33.765 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.765 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.765 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:33.765 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555065+0000", 2024-09-21T15:00:33.765 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5485000000000002e-05 2024-09-21T15:00:33.765 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.765 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.765 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:33.766 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555164+0000", 2024-09-21T15:00:33.766 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8629999999999996e-05 2024-09-21T15:00:33.766 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:33.766 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:33.766 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:33.766 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.766 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.766 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:113894 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 5750784~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:33.766 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.502384+0000", 2024-09-21T15:00:33.766 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.2309737099999998, 2024-09-21T15:00:33.766 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052884821999999998, 2024-09-21T15:00:33.766 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:33.766 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:33.766 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:33.767 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:33.767 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:33.767 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 113894 2024-09-21T15:00:33.767 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.767 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:33.767 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.767 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:33.767 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502384+0000", 2024-09-21T15:00:33.767 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:33.767 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.767 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.767 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:33.767 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502384+0000", 2024-09-21T15:00:33.767 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:33.767 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.767 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.767 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:33.768 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501933+0000", 2024-09-21T15:00:33.768 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9995484 2024-09-21T15:00:33.768 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.768 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.768 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:33.768 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502390+0000", 2024-09-21T15:00:33.768 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00045714099999999999 2024-09-21T15:00:33.768 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.768 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.768 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:33.768 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502402+0000", 2024-09-21T15:00:33.768 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.236e-05 2024-09-21T15:00:33.768 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.768 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.768 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:33.769 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502411+0000", 2024-09-21T15:00:33.769 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8689999999999993e-06 2024-09-21T15:00:33.769 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.769 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.769 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:33.769 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555178+0000", 2024-09-21T15:00:33.769 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052767050000000003 2024-09-21T15:00:33.769 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.769 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.769 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:33.769 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555215+0000", 2024-09-21T15:00:33.769 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.667e-05 2024-09-21T15:00:33.769 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.769 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.769 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:33.770 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555269+0000", 2024-09-21T15:00:33.770 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4109e-05 2024-09-21T15:00:33.770 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:33.770 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:33.770 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:33.770 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.770 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.770 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:113895 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 5840896~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:33.770 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.502651+0000", 2024-09-21T15:00:33.770 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.2307070969999998, 2024-09-21T15:00:33.770 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052770692000000001, 2024-09-21T15:00:33.770 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:33.770 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:33.770 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:33.771 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:33.771 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:33.771 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 113895 2024-09-21T15:00:33.771 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.771 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:33.771 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.771 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:33.771 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502651+0000", 2024-09-21T15:00:33.771 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:33.771 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.771 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.771 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:33.771 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502651+0000", 2024-09-21T15:00:33.771 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:33.771 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.772 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.772 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:33.772 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502421+0000", 2024-09-21T15:00:33.772 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9997702 2024-09-21T15:00:33.772 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.772 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.772 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:33.772 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502658+0000", 2024-09-21T15:00:33.772 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000237264 2024-09-21T15:00:33.772 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.772 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.772 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:33.772 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502673+0000", 2024-09-21T15:00:33.772 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4474999999999999e-05 2024-09-21T15:00:33.772 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.772 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.773 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:33.773 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502681+0000", 2024-09-21T15:00:33.773 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6119999999999999e-06 2024-09-21T15:00:33.773 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.773 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.773 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:33.773 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555285+0000", 2024-09-21T15:00:33.773 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052603860000000002 2024-09-21T15:00:33.773 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.773 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.773 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:33.773 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555348+0000", 2024-09-21T15:00:33.773 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3411999999999995e-05 2024-09-21T15:00:33.773 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.774 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.774 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:33.774 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555421+0000", 2024-09-21T15:00:33.774 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2809e-05 2024-09-21T15:00:33.774 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:33.774 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:33.774 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:33.774 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.774 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.774 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:113896 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 5931008~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:33.774 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.502862+0000", 2024-09-21T15:00:33.774 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.2304955489999996, 2024-09-21T15:00:33.774 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052660208, 2024-09-21T15:00:33.774 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:33.775 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:33.775 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:33.775 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:33.775 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:33.775 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 113896 2024-09-21T15:00:33.775 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.775 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:33.775 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.775 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:33.775 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502862+0000", 2024-09-21T15:00:33.775 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:33.775 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.775 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.775 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:33.775 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502862+0000", 2024-09-21T15:00:33.776 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:33.776 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.776 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.776 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:33.776 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502689+0000", 2024-09-21T15:00:33.776 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9998264 2024-09-21T15:00:33.776 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.776 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.776 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:33.776 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502873+0000", 2024-09-21T15:00:33.776 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000184146 2024-09-21T15:00:33.776 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.776 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.776 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:33.776 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502884+0000", 2024-09-21T15:00:33.776 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1293e-05 2024-09-21T15:00:33.777 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.777 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.777 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:33.777 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502891+0000", 2024-09-21T15:00:33.777 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4259999999999997e-06 2024-09-21T15:00:33.777 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.777 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.777 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:33.777 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555435+0000", 2024-09-21T15:00:33.777 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052543053999999999 2024-09-21T15:00:33.777 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.777 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.777 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:33.777 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555471+0000", 2024-09-21T15:00:33.777 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6454e-05 2024-09-21T15:00:33.777 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.778 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.778 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:33.778 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555522+0000", 2024-09-21T15:00:33.778 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1314999999999999e-05 2024-09-21T15:00:33.778 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:33.778 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:33.778 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:33.778 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.778 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.778 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:113897 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 6021120~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:33.778 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.503004+0000", 2024-09-21T15:00:33.778 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.2303534420000002, 2024-09-21T15:00:33.778 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052669499000000002, 2024-09-21T15:00:33.778 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:33.778 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:33.779 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:33.779 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:33.779 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:33.779 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 113897 2024-09-21T15:00:33.779 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.779 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:33.779 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.779 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:33.779 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503004+0000", 2024-09-21T15:00:33.779 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:33.779 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.779 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.779 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:33.779 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503004+0000", 2024-09-21T15:00:33.779 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:33.779 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.780 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.780 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:33.780 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502898+0000", 2024-09-21T15:00:33.780 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9998937 2024-09-21T15:00:33.780 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.780 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.780 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:33.780 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503010+0000", 2024-09-21T15:00:33.780 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000112396 2024-09-21T15:00:33.780 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.780 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.780 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:33.780 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503023+0000", 2024-09-21T15:00:33.780 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2206e-05 2024-09-21T15:00:33.781 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.781 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.781 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:33.781 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503031+0000", 2024-09-21T15:00:33.781 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3780000000000001e-06 2024-09-21T15:00:33.781 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.781 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.781 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:33.781 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555549+0000", 2024-09-21T15:00:33.781 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052517909000000002 2024-09-21T15:00:33.781 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.781 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.781 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:33.781 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555594+0000", 2024-09-21T15:00:33.781 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5385e-05 2024-09-21T15:00:33.782 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.782 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.782 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:33.782 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555674+0000", 2024-09-21T15:00:33.782 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9447e-05 2024-09-21T15:00:33.782 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:33.782 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:33.782 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:33.782 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.782 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.782 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:113898 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 6111232~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:33.782 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.503039+0000", 2024-09-21T15:00:33.782 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.2303186569999998, 2024-09-21T15:00:33.783 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052754290000000002, 2024-09-21T15:00:33.783 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:33.783 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:33.783 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:33.783 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:33.783 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:33.783 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 113898 2024-09-21T15:00:33.783 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.783 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:33.783 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.783 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:33.783 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503039+0000", 2024-09-21T15:00:33.783 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:33.783 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.784 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.784 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:33.784 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503039+0000", 2024-09-21T15:00:33.784 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:33.784 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.784 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.784 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:33.784 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503038+0000", 2024-09-21T15:00:33.784 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:00:33.784 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.784 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.784 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:33.784 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503042+0000", 2024-09-21T15:00:33.784 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.0640000000000004e-06 2024-09-21T15:00:33.785 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.785 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.785 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:33.785 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503049+0000", 2024-09-21T15:00:33.785 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5579999999999997e-06 2024-09-21T15:00:33.785 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.785 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.785 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:33.785 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503054+0000", 2024-09-21T15:00:33.785 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6550000000000001e-06 2024-09-21T15:00:33.785 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.785 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.785 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:33.785 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555693+0000", 2024-09-21T15:00:33.785 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052638216000000002 2024-09-21T15:00:33.786 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.786 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.786 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:33.786 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555736+0000", 2024-09-21T15:00:33.786 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3619999999999999e-05 2024-09-21T15:00:33.786 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.786 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.786 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:33.786 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555793+0000", 2024-09-21T15:00:33.786 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6956999999999997e-05 2024-09-21T15:00:33.786 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:33.786 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:33.786 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:33.786 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.787 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.787 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:113899 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 6201344~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:33.787 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.503255+0000", 2024-09-21T15:00:33.787 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.2301023840000003, 2024-09-21T15:00:33.787 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052648161999999998, 2024-09-21T15:00:33.787 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:33.787 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:33.787 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:33.787 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:33.787 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:33.787 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 113899 2024-09-21T15:00:33.787 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.787 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:33.787 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.787 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:33.788 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503255+0000", 2024-09-21T15:00:33.788 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:33.788 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.788 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.788 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:33.788 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503255+0000", 2024-09-21T15:00:33.788 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:33.788 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.788 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.788 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:33.788 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503058+0000", 2024-09-21T15:00:33.788 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9998021 2024-09-21T15:00:33.788 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.788 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.789 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:33.789 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503262+0000", 2024-09-21T15:00:33.789 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00020421299999999999 2024-09-21T15:00:33.789 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.789 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.789 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:33.789 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503274+0000", 2024-09-21T15:00:33.789 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2052000000000001e-05 2024-09-21T15:00:33.789 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.789 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.789 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:33.789 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503282+0000", 2024-09-21T15:00:33.789 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.977e-06 2024-09-21T15:00:33.789 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.789 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.789 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:33.790 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555808+0000", 2024-09-21T15:00:33.790 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052526150000000001 2024-09-21T15:00:33.790 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.790 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.790 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:33.790 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555851+0000", 2024-09-21T15:00:33.790 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2846999999999998e-05 2024-09-21T15:00:33.790 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.790 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.790 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:33.790 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555903+0000", 2024-09-21T15:00:33.790 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2607e-05 2024-09-21T15:00:33.790 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:33.790 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:33.790 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:33.791 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.791 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.791 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:113900 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 6307840~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:33.791 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.503445+0000", 2024-09-21T15:00:33.791 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.2299124490000004, 2024-09-21T15:00:33.791 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052589546000000001, 2024-09-21T15:00:33.791 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:33.791 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:33.791 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:33.791 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:33.791 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:33.791 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 113900 2024-09-21T15:00:33.791 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.791 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:33.791 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.791 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:33.792 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503445+0000", 2024-09-21T15:00:33.792 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:33.792 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.792 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.792 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:33.792 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503445+0000", 2024-09-21T15:00:33.792 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:33.792 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.792 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.792 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:33.792 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503290+0000", 2024-09-21T15:00:33.792 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9998446 2024-09-21T15:00:33.792 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.792 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.792 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:33.793 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503451+0000", 2024-09-21T15:00:33.793 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00016111000000000001 2024-09-21T15:00:33.793 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.793 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.793 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:33.793 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503461+0000", 2024-09-21T15:00:33.793 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8740000000000007e-06 2024-09-21T15:00:33.793 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.793 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.793 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:33.793 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503468+0000", 2024-09-21T15:00:33.793 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.187e-06 2024-09-21T15:00:33.793 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.793 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.793 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:33.793 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555930+0000", 2024-09-21T15:00:33.793 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052462404999999997 2024-09-21T15:00:33.794 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.794 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.794 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:33.794 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555971+0000", 2024-09-21T15:00:33.794 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.0419e-05 2024-09-21T15:00:33.794 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.794 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.794 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:33.794 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.556035+0000", 2024-09-21T15:00:33.794 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3948999999999994e-05 2024-09-21T15:00:33.794 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:33.794 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:33.794 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:33.795 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.795 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.795 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:114050 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 4767744~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:33.795 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.514988+0000", 2024-09-21T15:00:33.795 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.2183699380000004, 2024-09-21T15:00:33.795 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052614432000000003, 2024-09-21T15:00:33.795 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:33.795 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:33.795 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:33.795 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:33.795 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:33.795 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 114050 2024-09-21T15:00:33.795 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.795 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:33.796 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.796 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:33.796 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.514988+0000", 2024-09-21T15:00:33.796 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:33.796 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.796 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.796 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:33.796 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.514988+0000", 2024-09-21T15:00:33.796 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:33.796 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.796 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.796 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:33.796 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.514987+0000", 2024-09-21T15:00:33.796 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:33.796 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.796 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.797 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:33.797 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.514990+0000", 2024-09-21T15:00:33.797 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4739999999999999e-06 2024-09-21T15:00:33.797 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.797 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.797 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:33.797 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.514995+0000", 2024-09-21T15:00:33.797 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1939999999999997e-06 2024-09-21T15:00:33.797 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.797 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.797 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:33.797 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.514999+0000", 2024-09-21T15:00:33.797 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.245e-06 2024-09-21T15:00:33.797 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.797 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.797 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:33.798 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.567498+0000", 2024-09-21T15:00:33.798 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052498516000000002 2024-09-21T15:00:33.798 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.798 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.798 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:33.798 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.567541+0000", 2024-09-21T15:00:33.798 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3516000000000001e-05 2024-09-21T15:00:33.798 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.798 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.798 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:33.798 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.567602+0000", 2024-09-21T15:00:33.798 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0800000000000001e-05 2024-09-21T15:00:33.798 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:33.798 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:33.798 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:33.799 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.799 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.799 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:114051 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 4857856~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:33.799 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.515003+0000", 2024-09-21T15:00:33.799 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.2183542230000004, 2024-09-21T15:00:33.799 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052709245000000002, 2024-09-21T15:00:33.799 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:33.799 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:33.799 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:33.799 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:33.799 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:33.799 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 114051 2024-09-21T15:00:33.799 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.799 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:33.800 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.800 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:33.800 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515003+0000", 2024-09-21T15:00:33.800 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:33.800 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.800 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.800 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:33.800 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515003+0000", 2024-09-21T15:00:33.800 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:33.800 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.800 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.800 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:33.800 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515003+0000", 2024-09-21T15:00:33.800 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:33.800 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.800 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.801 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:33.801 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515005+0000", 2024-09-21T15:00:33.801 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3499999999999999e-06 2024-09-21T15:00:33.801 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.801 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.801 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:33.801 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515011+0000", 2024-09-21T15:00:33.801 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2850000000000002e-06 2024-09-21T15:00:33.801 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.801 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.801 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:33.801 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515017+0000", 2024-09-21T15:00:33.801 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0059999999999998e-06 2024-09-21T15:00:33.801 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.801 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.801 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:33.802 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.567618+0000", 2024-09-21T15:00:33.802 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052600806999999999 2024-09-21T15:00:33.802 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.802 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.802 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:33.802 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.567655+0000", 2024-09-21T15:00:33.802 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7866e-05 2024-09-21T15:00:33.802 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.802 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.802 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:33.802 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.567713+0000", 2024-09-21T15:00:33.802 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7281999999999998e-05 2024-09-21T15:00:33.802 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:33.802 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:33.803 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:33.803 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.803 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.803 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:114052 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 4947968~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:33.803 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.515046+0000", 2024-09-21T15:00:33.803 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.2183115640000004, 2024-09-21T15:00:33.803 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052795975000000002, 2024-09-21T15:00:33.803 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:33.803 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:33.803 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:33.803 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:33.803 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:33.803 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 114052 2024-09-21T15:00:33.803 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.803 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:33.804 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.804 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:33.804 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515046+0000", 2024-09-21T15:00:33.804 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:33.804 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.804 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.804 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:33.804 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515046+0000", 2024-09-21T15:00:33.804 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:33.804 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.804 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.804 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:33.804 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515046+0000", 2024-09-21T15:00:33.804 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:33.804 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.805 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.805 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:33.805 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515050+0000", 2024-09-21T15:00:33.805 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1359999999999999e-06 2024-09-21T15:00:33.805 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.805 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.805 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:33.805 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515060+0000", 2024-09-21T15:00:33.805 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0339e-05 2024-09-21T15:00:33.805 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.805 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.805 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:33.805 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515068+0000", 2024-09-21T15:00:33.805 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1149999999999994e-06 2024-09-21T15:00:33.805 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.806 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.806 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:33.806 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.567733+0000", 2024-09-21T15:00:33.806 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052664872000000001 2024-09-21T15:00:33.806 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.806 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.806 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:33.806 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.567778+0000", 2024-09-21T15:00:33.806 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4966999999999998e-05 2024-09-21T15:00:33.806 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.806 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.806 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:33.806 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.567842+0000", 2024-09-21T15:00:33.806 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4114000000000003e-05 2024-09-21T15:00:33.806 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:33.807 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:33.807 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:33.807 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.807 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.807 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:114053 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 5038080~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:33.807 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.515076+0000", 2024-09-21T15:00:33.807 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.2182817039999998, 2024-09-21T15:00:33.807 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052886112999999998, 2024-09-21T15:00:33.807 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:33.807 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:33.807 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:33.807 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:33.807 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:33.807 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 114053 2024-09-21T15:00:33.808 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.808 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:33.808 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.808 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:33.808 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515076+0000", 2024-09-21T15:00:33.808 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:33.808 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.808 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.808 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:33.808 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515076+0000", 2024-09-21T15:00:33.808 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:33.808 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.808 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.808 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:33.808 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515075+0000", 2024-09-21T15:00:33.809 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:33.809 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.809 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.809 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:33.809 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515083+0000", 2024-09-21T15:00:33.809 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0520000000000002e-06 2024-09-21T15:00:33.809 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.809 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.809 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:33.809 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515092+0000", 2024-09-21T15:00:33.809 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1440000000000003e-06 2024-09-21T15:00:33.809 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.809 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.809 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:33.810 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515098+0000", 2024-09-21T15:00:33.810 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1090000000000003e-06 2024-09-21T15:00:33.810 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.810 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.810 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:33.810 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.567871+0000", 2024-09-21T15:00:33.810 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052772966999999997 2024-09-21T15:00:33.810 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.810 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.810 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:33.810 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.567909+0000", 2024-09-21T15:00:33.810 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.8380000000000002e-05 2024-09-21T15:00:33.810 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.811 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.811 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:33.811 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.567962+0000", 2024-09-21T15:00:33.811 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3068000000000001e-05 2024-09-21T15:00:33.811 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:33.811 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:33.811 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:33.811 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.811 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.811 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:114055 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 5218304~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:33.811 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.515526+0000", 2024-09-21T15:00:33.811 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.217831264, 2024-09-21T15:00:33.811 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052684278000000001, 2024-09-21T15:00:33.811 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:33.811 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:33.812 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:33.812 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:33.812 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:33.812 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 114055 2024-09-21T15:00:33.812 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.812 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:33.812 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.812 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:33.812 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515526+0000", 2024-09-21T15:00:33.812 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:33.812 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.812 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.812 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:33.812 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515526+0000", 2024-09-21T15:00:33.813 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:33.813 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.813 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.813 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:33.813 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515526+0000", 2024-09-21T15:00:33.813 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:33.813 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.813 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.813 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:33.813 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515529+0000", 2024-09-21T15:00:33.813 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.596e-06 2024-09-21T15:00:33.813 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.813 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.813 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:33.813 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515536+0000", 2024-09-21T15:00:33.814 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7399999999999998e-06 2024-09-21T15:00:33.814 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.814 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.814 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:33.814 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515542+0000", 2024-09-21T15:00:33.814 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5740000000000003e-06 2024-09-21T15:00:33.814 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.814 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.814 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:33.814 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.568116+0000", 2024-09-21T15:00:33.814 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052574152999999998 2024-09-21T15:00:33.814 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.814 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.815 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:33.815 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.568154+0000", 2024-09-21T15:00:33.815 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7926000000000002e-05 2024-09-21T15:00:33.815 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.815 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.815 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:33.815 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.568211+0000", 2024-09-21T15:00:33.815 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6858999999999999e-05 2024-09-21T15:00:33.815 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:33.815 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:33.815 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:33.815 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.815 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.815 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:114056 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 5308416~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:33.815 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.515547+0000", 2024-09-21T15:00:33.816 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.2178102390000003, 2024-09-21T15:00:33.816 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052792795000000003, 2024-09-21T15:00:33.816 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:33.816 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:33.816 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:33.816 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:33.816 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:33.816 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 114056 2024-09-21T15:00:33.816 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.816 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:33.816 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.816 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:33.816 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515547+0000", 2024-09-21T15:00:33.816 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:33.816 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.816 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.817 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:33.817 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515547+0000", 2024-09-21T15:00:33.817 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:33.817 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.817 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.817 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:33.817 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515547+0000", 2024-09-21T15:00:33.817 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:33.817 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.817 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.817 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:33.817 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515550+0000", 2024-09-21T15:00:33.817 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2770000000000001e-06 2024-09-21T15:00:33.817 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.817 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.817 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:33.818 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515554+0000", 2024-09-21T15:00:33.818 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.087e-06 2024-09-21T15:00:33.818 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.818 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.818 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:33.818 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515558+0000", 2024-09-21T15:00:33.818 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1709999999999998e-06 2024-09-21T15:00:33.818 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.818 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.818 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:33.818 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.568234+0000", 2024-09-21T15:00:33.818 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052676675999999999 2024-09-21T15:00:33.818 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.818 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.819 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:33.819 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.568277+0000", 2024-09-21T15:00:33.819 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2400000000000001e-05 2024-09-21T15:00:33.819 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:33.819 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:33.819 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:33.819 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.568340+0000", 2024-09-21T15:00:33.819 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3581e-05 2024-09-21T15:00:33.819 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:33.819 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:33.819 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:33.819 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:33.819 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:33.819 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:33.821 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T15:00:33.821 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:00:33.819+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:00:33.922 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:00:33.918+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:00:33.936 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:33.936 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:00:33.936 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T15:00:33.936 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:33.946 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T15:00:34.023 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:00:34.022+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:00:34.053 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:34.053 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:00:34.053 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T15:00:34.053 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T15:00:34.053 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:34.063 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T15:00:34.124 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:00:34.122+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:00:34.175 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:34.175 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T15:00:34.175 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T15:00:34.175 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T15:00:34.175 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.175 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T15:00:34.175 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.205754+0000", 2024-09-21T15:00:34.175 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 150.96710307999999, 2024-09-21T15:00:34.175 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.41117094199999998, 2024-09-21T15:00:34.176 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:34.176 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:34.176 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:34.176 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T15:00:34.176 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T15:00:34.176 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1 2024-09-21T15:00:34.176 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.176 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:34.176 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.176 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:34.176 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205754+0000", 2024-09-21T15:00:34.176 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:34.176 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.176 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.176 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:34.176 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205754+0000", 2024-09-21T15:00:34.177 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:34.177 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.177 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.177 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:34.177 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205752+0000", 2024-09-21T15:00:34.177 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T15:00:34.177 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.177 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.177 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:34.177 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205765+0000", 2024-09-21T15:00:34.177 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3230999999999999e-05 2024-09-21T15:00:34.177 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.177 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.177 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:34.177 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205782+0000", 2024-09-21T15:00:34.177 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6634e-05 2024-09-21T15:00:34.178 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.178 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.178 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:34.178 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205793+0000", 2024-09-21T15:00:34.178 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1297e-05 2024-09-21T15:00:34.178 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.178 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.178 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:34.178 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205867+0000", 2024-09-21T15:00:34.178 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4269000000000003e-05 2024-09-21T15:00:34.178 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.178 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.178 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:00:34.178 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205877+0000", 2024-09-21T15:00:34.178 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2690000000000005e-06 2024-09-21T15:00:34.179 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.179 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.179 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:34.179 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.613700+0000", 2024-09-21T15:00:34.179 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.407823455 2024-09-21T15:00:34.179 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.179 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.179 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:34.179 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.614203+0000", 2024-09-21T15:00:34.179 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00050311500000000005 2024-09-21T15:00:34.179 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.179 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.179 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T15:00:34.179 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.614750+0000", 2024-09-21T15:00:34.179 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00054738699999999998 2024-09-21T15:00:34.179 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.180 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.180 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:34.180 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616367+0000", 2024-09-21T15:00:34.180 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001616634 2024-09-21T15:00:34.180 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.180 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.180 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:34.180 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616574+0000", 2024-09-21T15:00:34.180 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00020668499999999999 2024-09-21T15:00:34.180 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.180 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.180 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:34.180 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616802+0000", 2024-09-21T15:00:34.180 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00022817900000000001 2024-09-21T15:00:34.181 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.181 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.181 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:34.181 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616925+0000", 2024-09-21T15:00:34.181 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012309699999999999 2024-09-21T15:00:34.181 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:34.181 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:34.181 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:34.181 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.181 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.181 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6639 3.1a 3:592d80ed:::rbd_data.11361bed1f78.0000000000000005:d [read 0~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T15:00:34.181 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131650+0000", 2024-09-21T15:00:34.181 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 119.041207532, 2024-09-21T15:00:34.181 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016002832000000002, 2024-09-21T15:00:34.181 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:34.181 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:34.182 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:34.182 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:34.182 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:34.182 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6639 2024-09-21T15:00:34.182 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.182 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:34.182 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.182 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:34.182 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131650+0000", 2024-09-21T15:00:34.182 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:34.182 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.182 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.182 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:34.182 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131650+0000", 2024-09-21T15:00:34.182 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:34.183 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.183 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.183 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:34.183 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131649+0000", 2024-09-21T15:00:34.183 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:34.183 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.183 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.183 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:34.183 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131651+0000", 2024-09-21T15:00:34.183 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9199999999999998e-06 2024-09-21T15:00:34.183 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.183 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.183 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:34.183 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131656+0000", 2024-09-21T15:00:34.184 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9899999999999997e-06 2024-09-21T15:00:34.184 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.184 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.184 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:34.184 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131665+0000", 2024-09-21T15:00:34.184 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0999999999999993e-06 2024-09-21T15:00:34.184 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.184 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.184 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:34.184 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146309+0000", 2024-09-21T15:00:34.184 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014643893999999999 2024-09-21T15:00:34.184 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.184 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.184 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:34.184 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146392+0000", 2024-09-21T15:00:34.185 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2531e-05 2024-09-21T15:00:34.185 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.185 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.185 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:34.185 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147653+0000", 2024-09-21T15:00:34.185 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0012606710000000001 2024-09-21T15:00:34.185 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:34.185 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:34.185 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:34.185 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.185 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.185 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6640 3.1a 3:592d80ed:::rbd_data.11361bed1f78.0000000000000005:d [read 65536~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T15:00:34.185 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131670+0000", 2024-09-21T15:00:34.185 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 119.041187233, 2024-09-21T15:00:34.186 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016188859, 2024-09-21T15:00:34.186 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:34.186 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:34.186 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:34.186 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:34.186 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:34.186 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6640 2024-09-21T15:00:34.186 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.186 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:34.186 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.186 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:34.186 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131670+0000", 2024-09-21T15:00:34.186 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:34.187 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.187 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.187 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:34.187 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131670+0000", 2024-09-21T15:00:34.187 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:34.187 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.187 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.187 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:34.187 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131670+0000", 2024-09-21T15:00:34.187 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:00:34.187 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.187 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.187 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:34.187 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131672+0000", 2024-09-21T15:00:34.187 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.818e-06 2024-09-21T15:00:34.188 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.188 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.188 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:34.188 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131675+0000", 2024-09-21T15:00:34.188 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5930000000000002e-06 2024-09-21T15:00:34.188 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.188 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.188 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:34.188 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131678+0000", 2024-09-21T15:00:34.188 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6790000000000001e-06 2024-09-21T15:00:34.188 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.188 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.188 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:34.188 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147681+0000", 2024-09-21T15:00:34.189 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016002755 2024-09-21T15:00:34.189 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.189 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.189 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:34.189 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147762+0000", 2024-09-21T15:00:34.189 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1052999999999998e-05 2024-09-21T15:00:34.189 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.189 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.189 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:34.189 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147859+0000", 2024-09-21T15:00:34.189 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.7199000000000002e-05 2024-09-21T15:00:34.189 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:34.189 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:34.189 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:34.189 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.189 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.190 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6641 3.1a 3:592d80ed:::rbd_data.11361bed1f78.0000000000000005:d [read 147456~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T15:00:34.190 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131680+0000", 2024-09-21T15:00:34.190 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 119.041176945, 2024-09-21T15:00:34.190 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016431296000000001, 2024-09-21T15:00:34.190 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:34.190 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:34.190 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:34.190 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:34.190 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:34.190 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6641 2024-09-21T15:00:34.190 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.190 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:34.190 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.191 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:34.191 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131680+0000", 2024-09-21T15:00:34.191 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:34.191 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.191 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.191 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:34.191 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131680+0000", 2024-09-21T15:00:34.191 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:34.191 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.191 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.191 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:34.191 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131680+0000", 2024-09-21T15:00:34.191 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:00:34.191 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.191 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.191 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:34.192 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131681+0000", 2024-09-21T15:00:34.192 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2610000000000001e-06 2024-09-21T15:00:34.192 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.192 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.192 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:34.192 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131685+0000", 2024-09-21T15:00:34.192 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.8299999999999998e-06 2024-09-21T15:00:34.192 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.192 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.192 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:34.192 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131688+0000", 2024-09-21T15:00:34.192 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6189999999999998e-06 2024-09-21T15:00:34.192 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.192 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.192 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:34.193 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147884+0000", 2024-09-21T15:00:34.193 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016196465 2024-09-21T15:00:34.193 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.193 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.193 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:34.193 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147949+0000", 2024-09-21T15:00:34.193 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4867000000000001e-05 2024-09-21T15:00:34.193 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.193 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.193 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:34.193 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.148112+0000", 2024-09-21T15:00:34.193 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00016247700000000001 2024-09-21T15:00:34.193 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:34.193 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:34.194 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:34.194 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.194 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.194 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6642 3.1a 3:592d80ed:::rbd_data.11361bed1f78.0000000000000005:d [read 212992~32768 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T15:00:34.194 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131690+0000", 2024-09-21T15:00:34.194 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 119.041167067, 2024-09-21T15:00:34.194 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016751335999999999, 2024-09-21T15:00:34.194 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:34.194 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:34.194 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:34.194 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:34.194 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:34.194 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6642 2024-09-21T15:00:34.194 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.194 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:34.195 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.195 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:34.195 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131690+0000", 2024-09-21T15:00:34.195 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:34.195 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.195 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.195 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:34.195 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131690+0000", 2024-09-21T15:00:34.195 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:34.195 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.195 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.195 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:34.195 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131690+0000", 2024-09-21T15:00:34.195 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:00:34.196 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.196 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.196 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:34.196 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131691+0000", 2024-09-21T15:00:34.196 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3120000000000001e-06 2024-09-21T15:00:34.196 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.196 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.196 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:34.196 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131694+0000", 2024-09-21T15:00:34.196 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0070000000000002e-06 2024-09-21T15:00:34.196 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.196 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.196 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:34.196 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131700+0000", 2024-09-21T15:00:34.197 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0800000000000002e-06 2024-09-21T15:00:34.197 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.197 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.197 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:34.197 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.148135+0000", 2024-09-21T15:00:34.197 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016434250000000001 2024-09-21T15:00:34.197 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.197 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.197 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:34.197 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.148204+0000", 2024-09-21T15:00:34.197 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9466000000000002e-05 2024-09-21T15:00:34.197 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.197 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.197 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:34.197 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.148441+0000", 2024-09-21T15:00:34.198 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00023743700000000001 2024-09-21T15:00:34.198 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:34.198 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:34.198 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:34.198 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.198 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.198 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:61214 3.7 3:e28aec10:::rbd_header.1136862f872f:head [watch reconnect cookie 139864671932080 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-21T15:00:34.198 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:18.147172+0000", 2024-09-21T15:00:34.198 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 76.025685437000007, 2024-09-21T15:00:34.198 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.019264956, 2024-09-21T15:00:34.198 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:34.198 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:34.198 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:34.198 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:34.198 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:34.199 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 61214 2024-09-21T15:00:34.199 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.199 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:34.199 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.199 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:34.199 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147172+0000", 2024-09-21T15:00:34.199 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:34.199 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.199 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.199 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:34.199 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147172+0000", 2024-09-21T15:00:34.199 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:34.199 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.199 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.199 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:34.200 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147168+0000", 2024-09-21T15:00:34.200 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999962 2024-09-21T15:00:34.200 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.200 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.200 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:34.200 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147188+0000", 2024-09-21T15:00:34.200 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9717000000000001e-05 2024-09-21T15:00:34.200 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.200 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.200 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:34.200 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147239+0000", 2024-09-21T15:00:34.200 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1493999999999999e-05 2024-09-21T15:00:34.200 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.200 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.200 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:34.201 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147265+0000", 2024-09-21T15:00:34.201 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5746999999999999e-05 2024-09-21T15:00:34.201 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.201 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.201 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:34.201 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147372+0000", 2024-09-21T15:00:34.201 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010681200000000001 2024-09-21T15:00:34.201 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.201 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.201 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op must wait for map", 2024-09-21T15:00:34.201 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147380+0000", 2024-09-21T15:00:34.201 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7919999999999993e-06 2024-09-21T15:00:34.201 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.201 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.201 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:34.202 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.154025+0000", 2024-09-21T15:00:34.202 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0066454909999999999 2024-09-21T15:00:34.202 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.202 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.202 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:00:34.202 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.154041+0000", 2024-09-21T15:00:34.202 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5894000000000001e-05 2024-09-21T15:00:34.202 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.202 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.202 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:34.202 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.164345+0000", 2024-09-21T15:00:34.202 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0103040459999999 2024-09-21T15:00:34.202 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.202 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.202 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:34.203 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.164590+0000", 2024-09-21T15:00:34.203 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00024492699999999998 2024-09-21T15:00:34.203 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.203 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.203 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:34.203 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.166437+0000", 2024-09-21T15:00:34.203 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0018466330000000001 2024-09-21T15:00:34.203 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:34.203 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:34.203 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:34.203 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.203 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.203 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:61215 3.7 3:e28aec10:::rbd_header.1136862f872f:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-21T15:00:34.203 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:18.152008+0000", 2024-09-21T15:00:34.204 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 76.020848985000001, 2024-09-21T15:00:34.204 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.015644958, 2024-09-21T15:00:34.204 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:34.204 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:34.204 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:34.204 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:34.204 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:34.204 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 61215 2024-09-21T15:00:34.204 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.204 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:34.204 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.204 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:34.204 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152008+0000", 2024-09-21T15:00:34.204 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:34.204 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.205 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.205 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:34.205 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152008+0000", 2024-09-21T15:00:34.205 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:34.205 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.205 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.205 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:34.205 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152005+0000", 2024-09-21T15:00:34.205 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-21T15:00:34.205 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.205 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.205 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:34.205 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152025+0000", 2024-09-21T15:00:34.205 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9306000000000001e-05 2024-09-21T15:00:34.206 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.206 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.206 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:34.206 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152054+0000", 2024-09-21T15:00:34.206 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9189000000000001e-05 2024-09-21T15:00:34.206 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.206 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.206 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:34.206 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152078+0000", 2024-09-21T15:00:34.206 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4317000000000001e-05 2024-09-21T15:00:34.206 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.206 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.206 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:34.206 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152243+0000", 2024-09-21T15:00:34.207 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000164802 2024-09-21T15:00:34.207 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.207 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.207 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting_for_map not empty", 2024-09-21T15:00:34.207 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152251+0000", 2024-09-21T15:00:34.207 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0600000000000008e-06 2024-09-21T15:00:34.207 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.207 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.207 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:34.207 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.154741+0000", 2024-09-21T15:00:34.207 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0024900069999999998 2024-09-21T15:00:34.207 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.207 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.207 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:00:34.208 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.154753+0000", 2024-09-21T15:00:34.208 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1588e-05 2024-09-21T15:00:34.208 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.208 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.208 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:34.208 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.165380+0000", 2024-09-21T15:00:34.208 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0106272249999999 2024-09-21T15:00:34.208 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.208 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.208 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:34.208 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.165517+0000", 2024-09-21T15:00:34.208 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013737999999999999 2024-09-21T15:00:34.208 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.208 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.209 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 6", 2024-09-21T15:00:34.209 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.165938+0000", 2024-09-21T15:00:34.209 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00042082399999999998 2024-09-21T15:00:34.209 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.209 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.209 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:34.209 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.166795+0000", 2024-09-21T15:00:34.209 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00085674899999999997 2024-09-21T15:00:34.209 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.209 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.209 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:34.209 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.167598+0000", 2024-09-21T15:00:34.209 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00080335599999999997 2024-09-21T15:00:34.209 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.209 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.209 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:34.210 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.167632+0000", 2024-09-21T15:00:34.210 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4122000000000003e-05 2024-09-21T15:00:34.210 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.210 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.210 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:34.210 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.167653+0000", 2024-09-21T15:00:34.210 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0862000000000001e-05 2024-09-21T15:00:34.210 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:34.210 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:34.210 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:34.210 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.210 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.210 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:73698 3.10 3:0a4d95f6:::rbd_data.1136f4939602.000000000000000b:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-21T15:00:34.210 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:41.969870+0000", 2024-09-21T15:00:34.210 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 52.202986846999998, 2024-09-21T15:00:34.210 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.017502461, 2024-09-21T15:00:34.211 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:34.211 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:34.211 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:34.211 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:34.211 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:34.211 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 73698 2024-09-21T15:00:34.211 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.211 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:34.211 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.211 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:34.211 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.969870+0000", 2024-09-21T15:00:34.211 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:34.211 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.211 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.211 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:34.211 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.969870+0000", 2024-09-21T15:00:34.212 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:34.212 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.212 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.212 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:34.212 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.969868+0000", 2024-09-21T15:00:34.212 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T15:00:34.212 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.212 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.212 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:34.212 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.969885+0000", 2024-09-21T15:00:34.212 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7416999999999999e-05 2024-09-21T15:00:34.212 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.212 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.212 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:34.212 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.969918+0000", 2024-09-21T15:00:34.213 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3133000000000003e-05 2024-09-21T15:00:34.213 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.213 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.213 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:34.213 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.969934+0000", 2024-09-21T15:00:34.213 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6027000000000001e-05 2024-09-21T15:00:34.213 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.213 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.213 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:34.213 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.970021+0000", 2024-09-21T15:00:34.213 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6907e-05 2024-09-21T15:00:34.213 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.213 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.213 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:34.213 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.970148+0000", 2024-09-21T15:00:34.214 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012701499999999999 2024-09-21T15:00:34.214 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.214 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.214 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T15:00:34.214 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.972971+0000", 2024-09-21T15:00:34.214 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.002822873 2024-09-21T15:00:34.214 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.214 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.214 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:34.214 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.987124+0000", 2024-09-21T15:00:34.214 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0141529 2024-09-21T15:00:34.214 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.214 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.214 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:34.214 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.987146+0000", 2024-09-21T15:00:34.215 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1783000000000001e-05 2024-09-21T15:00:34.215 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.215 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.215 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:34.215 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.987258+0000", 2024-09-21T15:00:34.215 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000112555 2024-09-21T15:00:34.215 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.215 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.215 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:34.215 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.987373+0000", 2024-09-21T15:00:34.215 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000114434 2024-09-21T15:00:34.215 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:34.215 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:34.215 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:34.215 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.215 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.216 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:97638 3.1a 3:5e573525:::rbd_data.1136b9c49b1c.0000000000000026:49 [read 1277952~49152 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:34.216 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.276202+0000", 2024-09-21T15:00:34.216 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 28.896654741999999, 2024-09-21T15:00:34.216 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.015360357999999999, 2024-09-21T15:00:34.216 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:34.216 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:34.216 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:34.216 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:34.216 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:34.216 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 97638 2024-09-21T15:00:34.216 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.216 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:34.216 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.216 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:34.217 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276202+0000", 2024-09-21T15:00:34.217 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:34.217 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.217 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.217 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:34.217 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276202+0000", 2024-09-21T15:00:34.217 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:34.217 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.217 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.217 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:34.217 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276202+0000", 2024-09-21T15:00:34.217 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:34.217 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.217 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.217 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:34.218 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276204+0000", 2024-09-21T15:00:34.218 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5359999999999999e-06 2024-09-21T15:00:34.218 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.218 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.218 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:34.218 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276210+0000", 2024-09-21T15:00:34.218 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3040000000000004e-06 2024-09-21T15:00:34.218 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.218 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.218 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:34.218 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276214+0000", 2024-09-21T15:00:34.218 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6779999999999999e-06 2024-09-21T15:00:34.218 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.218 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.218 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:34.219 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.290563+0000", 2024-09-21T15:00:34.219 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014348346 2024-09-21T15:00:34.219 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.219 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.219 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:34.219 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.290623+0000", 2024-09-21T15:00:34.219 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0001999999999999e-05 2024-09-21T15:00:34.219 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.219 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.219 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:34.219 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.291563+0000", 2024-09-21T15:00:34.219 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00094003100000000003 2024-09-21T15:00:34.219 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:34.219 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:34.219 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:34.219 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.220 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.220 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:97678 3.d e122/24 3:b2f34c6d:::rbd_data.1136bfc79a70.0000000000000006:head v 122'234, mlcod=120'232)", 2024-09-21T15:00:34.220 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.321655+0000", 2024-09-21T15:00:34.220 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 28.851202067999999, 2024-09-21T15:00:34.220 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024509388999999999, 2024-09-21T15:00:34.220 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:34.220 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:34.220 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:34.220 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.220 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:34.220 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.321655+0000", 2024-09-21T15:00:34.220 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:34.220 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.220 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.220 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:34.220 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.321655+0000", 2024-09-21T15:00:34.221 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:34.221 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.221 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.221 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:34.221 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.321655+0000", 2024-09-21T15:00:34.221 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:34.221 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.221 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.221 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:34.221 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.329453+0000", 2024-09-21T15:00:34.221 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0077980560000000003 2024-09-21T15:00:34.221 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.221 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.221 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:34.221 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.329482+0000", 2024-09-21T15:00:34.221 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9153999999999999e-05 2024-09-21T15:00:34.222 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.222 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.222 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:34.222 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.329493+0000", 2024-09-21T15:00:34.222 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1103e-05 2024-09-21T15:00:34.222 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.222 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.222 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:34.222 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.329555+0000", 2024-09-21T15:00:34.222 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2345e-05 2024-09-21T15:00:34.222 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.222 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.222 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:34.222 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.329589+0000", 2024-09-21T15:00:34.222 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3908999999999997e-05 2024-09-21T15:00:34.222 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.223 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.223 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:34.223 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.346083+0000", 2024-09-21T15:00:34.223 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016493383 2024-09-21T15:00:34.223 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.223 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.223 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:34.223 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.346165+0000", 2024-09-21T15:00:34.223 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1796000000000001e-05 2024-09-21T15:00:34.223 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:34.223 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:34.223 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:34.223 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.223 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.223 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:97679 3.f e122/24 3:f2a362b4:::rbd_data.1136bfc79a70.0000000000000007:head v 122'331, mlcod=120'329)", 2024-09-21T15:00:34.224 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.337145+0000", 2024-09-21T15:00:34.224 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 28.835712058999999, 2024-09-21T15:00:34.224 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021912411, 2024-09-21T15:00:34.224 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:34.224 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:34.224 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:34.224 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.224 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:34.224 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.337145+0000", 2024-09-21T15:00:34.224 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:34.224 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.224 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.224 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:34.224 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.337145+0000", 2024-09-21T15:00:34.224 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:34.224 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.225 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.225 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:34.225 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.337145+0000", 2024-09-21T15:00:34.225 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:34.225 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.225 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.225 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:34.225 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.343899+0000", 2024-09-21T15:00:34.225 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0067541420000000003 2024-09-21T15:00:34.225 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.225 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.225 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:34.225 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.343921+0000", 2024-09-21T15:00:34.225 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2255e-05 2024-09-21T15:00:34.225 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.225 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.226 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:34.226 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.343933+0000", 2024-09-21T15:00:34.226 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1528e-05 2024-09-21T15:00:34.226 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.226 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.226 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:34.226 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.344001+0000", 2024-09-21T15:00:34.226 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8269999999999995e-05 2024-09-21T15:00:34.226 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.226 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.226 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:34.226 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.344029+0000", 2024-09-21T15:00:34.226 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8254000000000001e-05 2024-09-21T15:00:34.226 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.226 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.227 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:34.227 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.358901+0000", 2024-09-21T15:00:34.227 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0148719 2024-09-21T15:00:34.227 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.227 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.227 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:34.227 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.359058+0000", 2024-09-21T15:00:34.227 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000156446 2024-09-21T15:00:34.227 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:34.227 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:34.227 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:34.227 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.227 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.227 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:97676 3.1f e122/24 3:ffaa836a:::rbd_data.1136bfc79a70.0000000000000004:head v 122'312, mlcod=120'310)", 2024-09-21T15:00:34.227 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.348683+0000", 2024-09-21T15:00:34.228 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 28.824173894000001, 2024-09-21T15:00:34.228 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.033768802000000001, 2024-09-21T15:00:34.228 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:34.228 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:34.228 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:34.228 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.228 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:34.228 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.348683+0000", 2024-09-21T15:00:34.228 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:34.228 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.228 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.228 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:34.228 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.348683+0000", 2024-09-21T15:00:34.228 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:34.228 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.228 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.229 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:34.229 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.348683+0000", 2024-09-21T15:00:34.229 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:00:34.229 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.229 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.229 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:34.229 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.366937+0000", 2024-09-21T15:00:34.229 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018254415 2024-09-21T15:00:34.229 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.229 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.229 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:34.229 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.366959+0000", 2024-09-21T15:00:34.229 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1634e-05 2024-09-21T15:00:34.229 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.229 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.230 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:34.230 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.366971+0000", 2024-09-21T15:00:34.230 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2305e-05 2024-09-21T15:00:34.230 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.230 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.230 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:34.230 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.367034+0000", 2024-09-21T15:00:34.230 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3211000000000002e-05 2024-09-21T15:00:34.230 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.230 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.230 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:34.230 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.367056+0000", 2024-09-21T15:00:34.230 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2098000000000001e-05 2024-09-21T15:00:34.230 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.230 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.231 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:34.231 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.382400+0000", 2024-09-21T15:00:34.231 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.015343867000000001 2024-09-21T15:00:34.231 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.231 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.231 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:34.231 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.382452+0000", 2024-09-21T15:00:34.231 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1992e-05 2024-09-21T15:00:34.231 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:34.231 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:34.231 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:34.231 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.231 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.231 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:97734 3.19 e122/94 3:9fcf754f:::rbd_data.1136bfc79a70.0000000000000009:head v 122'314, mlcod=120'312)", 2024-09-21T15:00:34.231 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.378547+0000", 2024-09-21T15:00:34.231 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 28.794309916, 2024-09-21T15:00:34.232 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.063380675999999997, 2024-09-21T15:00:34.232 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:34.232 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:34.232 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:34.232 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.232 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:34.232 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.378547+0000", 2024-09-21T15:00:34.232 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:34.232 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.232 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.232 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:34.232 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.378547+0000", 2024-09-21T15:00:34.232 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:34.232 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.232 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.232 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:34.233 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.378547+0000", 2024-09-21T15:00:34.233 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:00:34.233 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.233 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.233 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:34.233 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.425118+0000", 2024-09-21T15:00:34.233 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046571166999999997 2024-09-21T15:00:34.233 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.233 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.233 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:34.233 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.425150+0000", 2024-09-21T15:00:34.233 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2150999999999998e-05 2024-09-21T15:00:34.233 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.233 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.233 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:34.234 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.425164+0000", 2024-09-21T15:00:34.234 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3302000000000001e-05 2024-09-21T15:00:34.234 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.234 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.234 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:34.234 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.425231+0000", 2024-09-21T15:00:34.234 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7106000000000004e-05 2024-09-21T15:00:34.234 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.234 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.234 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:34.234 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.425259+0000", 2024-09-21T15:00:34.234 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8087000000000001e-05 2024-09-21T15:00:34.235 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.235 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.235 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:34.235 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.441890+0000", 2024-09-21T15:00:34.235 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016631328000000001 2024-09-21T15:00:34.235 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.235 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.235 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:34.235 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.441928+0000", 2024-09-21T15:00:34.235 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7701999999999999e-05 2024-09-21T15:00:34.235 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:34.235 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:34.235 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:34.235 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.235 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.236 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:100187 3.10 3:08941a97:::rbd_data.1136ab4d6ff1.000000000000000e:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-21T15:00:34.236 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:10.869845+0000", 2024-09-21T15:00:34.236 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 23.303012020000001, 2024-09-21T15:00:34.236 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029345237, 2024-09-21T15:00:34.236 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:34.236 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:34.236 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:34.236 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:34.236 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:34.236 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 100187 2024-09-21T15:00:34.236 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.236 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:34.236 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.236 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:34.236 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869845+0000", 2024-09-21T15:00:34.237 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:34.237 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.237 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.237 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:34.237 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869845+0000", 2024-09-21T15:00:34.237 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:34.237 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.237 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.237 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:34.237 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869844+0000", 2024-09-21T15:00:34.237 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T15:00:34.237 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.237 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.237 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:34.237 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869853+0000", 2024-09-21T15:00:34.238 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5719999999999992e-06 2024-09-21T15:00:34.238 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.238 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.238 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:34.238 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869874+0000", 2024-09-21T15:00:34.238 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0483e-05 2024-09-21T15:00:34.238 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.238 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.238 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:34.238 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869883+0000", 2024-09-21T15:00:34.238 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6080000000000002e-06 2024-09-21T15:00:34.238 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.238 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.238 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:34.239 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869991+0000", 2024-09-21T15:00:34.239 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000108013 2024-09-21T15:00:34.239 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.239 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.239 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:34.239 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.870258+0000", 2024-09-21T15:00:34.239 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00026629099999999999 2024-09-21T15:00:34.239 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.239 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.239 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T15:00:34.239 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.878656+0000", 2024-09-21T15:00:34.239 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0083989000000000008 2024-09-21T15:00:34.239 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.240 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.240 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:34.240 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.899081+0000", 2024-09-21T15:00:34.240 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.020424437 2024-09-21T15:00:34.240 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.240 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.240 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:34.240 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.899125+0000", 2024-09-21T15:00:34.240 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4440999999999997e-05 2024-09-21T15:00:34.240 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.240 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.240 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:34.240 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.899160+0000", 2024-09-21T15:00:34.240 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4518999999999999e-05 2024-09-21T15:00:34.240 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.241 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.241 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:34.241 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.899190+0000", 2024-09-21T15:00:34.241 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0574999999999998e-05 2024-09-21T15:00:34.241 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:34.241 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:34.241 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:34.241 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.241 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.241 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:100263 3.10 3:0ece142e:::rbd_data.1136ab4d6ff1.0000000000000008:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-21T15:00:34.241 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:11.052257+0000", 2024-09-21T15:00:34.241 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 23.120599962, 2024-09-21T15:00:34.241 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042678025000000001, 2024-09-21T15:00:34.241 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:34.241 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:34.242 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:34.242 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:34.242 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:34.242 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 100263 2024-09-21T15:00:34.242 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.242 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:34.242 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.242 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:34.242 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052257+0000", 2024-09-21T15:00:34.242 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:34.242 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.242 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.242 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:34.242 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052257+0000", 2024-09-21T15:00:34.242 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:34.242 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.243 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.243 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:34.243 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052256+0000", 2024-09-21T15:00:34.243 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:00:34.243 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.243 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.243 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:34.243 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052263+0000", 2024-09-21T15:00:34.243 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4799999999999998e-06 2024-09-21T15:00:34.243 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.243 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.243 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:34.243 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052275+0000", 2024-09-21T15:00:34.243 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2386e-05 2024-09-21T15:00:34.243 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.244 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.244 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:34.244 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052281+0000", 2024-09-21T15:00:34.244 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6699999999999999e-06 2024-09-21T15:00:34.244 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.244 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.244 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:34.244 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052350+0000", 2024-09-21T15:00:34.244 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9099999999999999e-05 2024-09-21T15:00:34.244 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.244 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.244 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:34.244 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052456+0000", 2024-09-21T15:00:34.244 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000106669 2024-09-21T15:00:34.244 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.245 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.245 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T15:00:34.245 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.059694+0000", 2024-09-21T15:00:34.245 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.007237566 2024-09-21T15:00:34.245 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.245 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.245 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:34.245 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.094721+0000", 2024-09-21T15:00:34.245 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035026977000000001 2024-09-21T15:00:34.245 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.245 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.245 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:34.245 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.094746+0000", 2024-09-21T15:00:34.245 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5329000000000001e-05 2024-09-21T15:00:34.245 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.245 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.246 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:34.246 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.094850+0000", 2024-09-21T15:00:34.246 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000103452 2024-09-21T15:00:34.246 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.246 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.246 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:34.246 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.094935+0000", 2024-09-21T15:00:34.246 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5550000000000006e-05 2024-09-21T15:00:34.246 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:34.246 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:34.246 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:34.246 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.246 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.246 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:102266 3.1f e130/24 3:fec72c47:::rbd_data.1136d175325b.000000000000000c:head v 130'326, mlcod=128'324)", 2024-09-21T15:00:34.246 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:14.319401+0000", 2024-09-21T15:00:34.246 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 19.853456022, 2024-09-21T15:00:34.247 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021068982999999999, 2024-09-21T15:00:34.247 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:34.247 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:34.247 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:34.247 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.247 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:34.247 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.319401+0000", 2024-09-21T15:00:34.247 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:34.247 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.247 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.247 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:34.247 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.319401+0000", 2024-09-21T15:00:34.247 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:34.247 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.247 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.247 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:34.248 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.319400+0000", 2024-09-21T15:00:34.248 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:00:34.248 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.248 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.248 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:34.248 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.337267+0000", 2024-09-21T15:00:34.248 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.017867141 2024-09-21T15:00:34.248 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.248 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.248 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:34.248 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.337278+0000", 2024-09-21T15:00:34.248 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0224000000000001e-05 2024-09-21T15:00:34.248 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.248 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.248 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:34.248 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.337285+0000", 2024-09-21T15:00:34.249 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.25e-06 2024-09-21T15:00:34.249 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.249 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.249 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:34.249 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.337330+0000", 2024-09-21T15:00:34.249 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5210000000000003e-05 2024-09-21T15:00:34.249 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.249 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.249 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:34.249 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.337350+0000", 2024-09-21T15:00:34.249 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9556e-05 2024-09-21T15:00:34.249 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.249 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.249 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:34.249 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.340418+0000", 2024-09-21T15:00:34.249 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.003068552 2024-09-21T15:00:34.249 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.250 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.250 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:34.250 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.340470+0000", 2024-09-21T15:00:34.250 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1897000000000001e-05 2024-09-21T15:00:34.250 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:34.250 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:34.250 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:34.250 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.250 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.250 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:102887 3.1c 3:3b11ae99:::rbd_object_map.1136d175325b.0000000000000050:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-21T15:00:34.250 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:14.368644+0000", 2024-09-21T15:00:34.250 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 19.804212986, 2024-09-21T15:00:34.250 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.017021290000000001, 2024-09-21T15:00:34.250 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:34.250 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:34.251 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:34.251 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:34.251 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:34.251 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 102887 2024-09-21T15:00:34.251 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.251 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:34.251 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.251 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:34.251 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.368644+0000", 2024-09-21T15:00:34.251 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:34.251 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.251 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.251 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:34.251 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.368644+0000", 2024-09-21T15:00:34.251 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:34.251 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.251 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.252 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:34.252 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.368642+0000", 2024-09-21T15:00:34.252 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T15:00:34.252 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.252 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.252 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:34.252 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.368655+0000", 2024-09-21T15:00:34.252 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2808e-05 2024-09-21T15:00:34.252 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.252 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.252 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:34.252 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.368677+0000", 2024-09-21T15:00:34.252 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2753999999999999e-05 2024-09-21T15:00:34.252 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.252 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.252 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:34.253 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.368688+0000", 2024-09-21T15:00:34.253 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1228e-05 2024-09-21T15:00:34.253 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.253 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.253 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:34.253 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.368743+0000", 2024-09-21T15:00:34.253 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4608999999999998e-05 2024-09-21T15:00:34.253 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.253 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.253 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:34.253 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.368868+0000", 2024-09-21T15:00:34.253 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012473000000000001 2024-09-21T15:00:34.253 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.253 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.253 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 0", 2024-09-21T15:00:34.253 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.369385+0000", 2024-09-21T15:00:34.254 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00051684400000000003 2024-09-21T15:00:34.254 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.254 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.254 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:34.254 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.370755+0000", 2024-09-21T15:00:34.254 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0013701609999999999 2024-09-21T15:00:34.254 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.254 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.254 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:34.254 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.385531+0000", 2024-09-21T15:00:34.254 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014776648 2024-09-21T15:00:34.254 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.254 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.254 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:34.255 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.385575+0000", 2024-09-21T15:00:34.255 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3726000000000001e-05 2024-09-21T15:00:34.255 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.255 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.255 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:34.255 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.385666+0000", 2024-09-21T15:00:34.255 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0304999999999997e-05 2024-09-21T15:00:34.255 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:34.255 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:34.255 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:34.255 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.255 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.255 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:103706 3.1c 3:3b11ae99:::rbd_object_map.1136d175325b.0000000000000050:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-21T15:00:34.255 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:14.426370+0000", 2024-09-21T15:00:34.255 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 19.746487648999999, 2024-09-21T15:00:34.256 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.015478897, 2024-09-21T15:00:34.256 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:34.256 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:34.256 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:34.256 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:34.256 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:34.256 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 103706 2024-09-21T15:00:34.256 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.256 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:34.256 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.256 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:34.256 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.426370+0000", 2024-09-21T15:00:34.256 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:34.256 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.256 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.256 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:34.256 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.426370+0000", 2024-09-21T15:00:34.257 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:34.257 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.257 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.257 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:34.257 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.426369+0000", 2024-09-21T15:00:34.257 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:00:34.257 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.257 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.257 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:34.257 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.426376+0000", 2024-09-21T15:00:34.257 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0999999999999998e-06 2024-09-21T15:00:34.257 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.257 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.257 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:34.257 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.426442+0000", 2024-09-21T15:00:34.257 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6445999999999994e-05 2024-09-21T15:00:34.258 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.258 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.258 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:34.258 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.426451+0000", 2024-09-21T15:00:34.258 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4940000000000003e-06 2024-09-21T15:00:34.258 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.258 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.258 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:34.258 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.426493+0000", 2024-09-21T15:00:34.258 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2094000000000001e-05 2024-09-21T15:00:34.258 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.258 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.258 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:34.258 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.426560+0000", 2024-09-21T15:00:34.258 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6774999999999997e-05 2024-09-21T15:00:34.258 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.259 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.259 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 0", 2024-09-21T15:00:34.259 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.426737+0000", 2024-09-21T15:00:34.259 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017769899999999999 2024-09-21T15:00:34.259 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.259 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.259 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:34.259 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.428561+0000", 2024-09-21T15:00:34.259 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001823917 2024-09-21T15:00:34.259 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.259 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.259 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:34.259 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.441762+0000", 2024-09-21T15:00:34.259 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013200462 2024-09-21T15:00:34.259 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.259 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.260 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:34.260 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.441811+0000", 2024-09-21T15:00:34.260 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8912000000000002e-05 2024-09-21T15:00:34.260 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.260 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.260 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:34.260 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.441848+0000", 2024-09-21T15:00:34.260 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7935000000000002e-05 2024-09-21T15:00:34.260 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:34.260 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:34.260 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:34.260 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.260 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.260 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:105083 3.1a 3:5d63845a:::rbd_data.1136d175325b.0000000000000037:head [write 0~32768 [fadvise_sequential+fadvise_nocache],write 65536~163840 [fadvise_sequential+fadvise_nocache],write 262144~32768 [fadvise_sequential+fadvise_nocache],write 327680~163840 [fadvise_sequential+fadvise_nocache],write 524288~32768 [fadvise_sequential+fadvise_nocache],write 589824~163840 [fadvise_sequential+fadvise_nocache],write 786432~32768 [fadvise_sequential+fadvise_nocache],write 851968~32768 [fadvise_sequential+fadvise_nocache],write 917504~98304 [fadvise_sequential+fadvise_nocache],write 1048576~32768 [fadvise_sequential+fadvise_nocache],write 1114112~32768 [fadvise_sequential+fadvise_nocache],write 1179648~98304 [fadvise_sequential+fadvise_nocache],write 1310720~32768 [fadvise_sequential+fadvise_nocache],write 1376256~32768 [fadvise_sequential+fadvise_nocache],write 1441792~98304 [fadvise_sequential+fadvise_nocache],write 1572864~32768 [fadvise_sequential+fadvise_nocache],write 1638400~32768 [fadvise_sequential+fadvise_nocache],write 1703936~98304 [fadvise_sequential+fadvise_nocache],write 1835008~32768 [fadvise_sequential+fadvise_nocache],write 1900544~32768 [fadvise_sequential+fadvise_nocache],write 1966080~98304 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-21T15:00:34.260 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:14.534201+0000", 2024-09-21T15:00:34.260 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 19.638656308000002, 2024-09-21T15:00:34.261 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016474194000000001, 2024-09-21T15:00:34.261 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:34.261 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:34.261 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:34.261 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:34.261 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:34.261 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 105083 2024-09-21T15:00:34.261 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.261 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:34.261 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.261 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:34.261 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.534201+0000", 2024-09-21T15:00:34.261 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:34.261 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.261 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.261 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:34.262 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.534201+0000", 2024-09-21T15:00:34.262 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:34.262 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.262 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.262 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:34.262 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.534201+0000", 2024-09-21T15:00:34.262 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:34.262 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.262 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.262 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:34.262 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.534853+0000", 2024-09-21T15:00:34.262 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00065227099999999995 2024-09-21T15:00:34.262 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.262 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.262 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:34.262 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.534866+0000", 2024-09-21T15:00:34.263 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3553000000000001e-05 2024-09-21T15:00:34.263 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.263 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.263 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:34.263 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.534877+0000", 2024-09-21T15:00:34.263 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0186e-05 2024-09-21T15:00:34.263 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.263 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.263 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:34.263 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.534947+0000", 2024-09-21T15:00:34.263 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0417e-05 2024-09-21T15:00:34.263 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.263 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.263 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:34.263 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.535138+0000", 2024-09-21T15:00:34.263 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00019122200000000001 2024-09-21T15:00:34.264 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.264 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.264 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T15:00:34.264 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.535666+0000", 2024-09-21T15:00:34.264 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00052817499999999998 2024-09-21T15:00:34.264 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.264 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.264 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:34.264 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.543252+0000", 2024-09-21T15:00:34.264 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0075857889999999999 2024-09-21T15:00:34.264 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.264 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.264 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:34.264 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.550552+0000", 2024-09-21T15:00:34.264 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0072995869999999997 2024-09-21T15:00:34.264 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.265 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.265 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:34.265 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.550637+0000", 2024-09-21T15:00:34.265 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5750999999999999e-05 2024-09-21T15:00:34.265 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.265 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.265 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:34.265 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.550675+0000", 2024-09-21T15:00:34.265 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7602999999999999e-05 2024-09-21T15:00:34.265 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:34.265 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:34.265 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:34.265 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.265 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.265 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:113281 3.10 3:0e1b39db:::rbd_data.1136b442e94a.0000000000000009:head [write 0~360448 [fadvise_sequential+fadvise_nocache],write 376832~573440 [fadvise_sequential+fadvise_nocache],write 1138688~573440 [fadvise_sequential+fadvise_nocache],write 1900544~196608 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:34.266 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.487425+0000", 2024-09-21T15:00:34.266 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.6854323659999997, 2024-09-21T15:00:34.266 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.015461361999999999, 2024-09-21T15:00:34.266 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:34.266 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:34.266 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:34.266 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:34.266 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:34.266 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 113281 2024-09-21T15:00:34.266 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.266 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:34.266 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.266 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:34.266 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.487425+0000", 2024-09-21T15:00:34.266 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:34.266 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.267 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.267 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:34.267 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.487425+0000", 2024-09-21T15:00:34.267 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:34.267 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.267 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.267 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:34.267 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.487423+0000", 2024-09-21T15:00:34.267 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-21T15:00:34.267 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.267 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.267 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:34.267 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.488151+0000", 2024-09-21T15:00:34.267 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00072770699999999996 2024-09-21T15:00:34.267 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.268 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.268 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:34.268 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.488175+0000", 2024-09-21T15:00:34.268 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3896e-05 2024-09-21T15:00:34.268 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.268 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.268 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:34.268 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.488184+0000", 2024-09-21T15:00:34.268 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9000000000000001e-06 2024-09-21T15:00:34.268 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.268 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.268 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:34.268 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.488918+0000", 2024-09-21T15:00:34.268 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00073320500000000001 2024-09-21T15:00:34.268 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.268 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.268 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:34.269 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.489175+0000", 2024-09-21T15:00:34.269 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00025761799999999999 2024-09-21T15:00:34.269 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.269 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.269 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T15:00:34.269 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.489479+0000", 2024-09-21T15:00:34.269 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00030419099999999999 2024-09-21T15:00:34.269 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.269 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.269 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:34.269 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.498472+0000", 2024-09-21T15:00:34.269 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0089922319999999993 2024-09-21T15:00:34.269 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.269 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.269 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:34.269 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502751+0000", 2024-09-21T15:00:34.270 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0042795309999999996 2024-09-21T15:00:34.270 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.270 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.270 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:34.270 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502843+0000", 2024-09-21T15:00:34.270 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1926e-05 2024-09-21T15:00:34.270 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.270 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.270 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:34.270 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502886+0000", 2024-09-21T15:00:34.270 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3087999999999998e-05 2024-09-21T15:00:34.270 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:34.270 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:34.270 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:34.271 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:34.271 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:34.271 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:34.272 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T15:00:34.273 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:00:34.270+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:00:34.349 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T15:00:34.351 INFO:teuthology.orchestra.run.smithi032.stdout:3968 writesame 0x735a8b8 thru 0x735e013 data_size 0x6(0x375c bytes) 2024-09-21T15:00:34.373 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:00:34.366+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:00:34.384 INFO:teuthology.orchestra.run.smithi032.stdout:3970 write 0xeb8c5ff thru 0xeb945eb (0x7fed bytes) 2024-09-21T15:00:34.399 INFO:teuthology.orchestra.run.smithi032.stdout:3972 punch from 0xd11faea to 0xd127843, (0x7d59 bytes) 2024-09-21T15:00:34.399 INFO:teuthology.orchestra.run.smithi032.stdout:3974 read 0x76916c3 thru 0x769ab30 (0x946e bytes) 2024-09-21T15:00:34.399 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:34.399 INFO:teuthology.orchestra.run.smithi032.stdout:3977 write 0x609d919 thru 0x60aacb7 (0xd39f bytes) 2024-09-21T15:00:34.409 INFO:teuthology.orchestra.run.smithi032.stdout:3978 trunc from 0xeb945ec to 0xe01a4c9 2024-09-21T15:00:34.415 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:34.415 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:00:34.415 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T15:00:34.415 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:34.419 INFO:teuthology.orchestra.run.smithi032.stdout:3980 writesame 0x53a2097 thru 0x53a6ebe data_size 0x52(0x4e28 bytes) 2024-09-21T15:00:34.426 INFO:teuthology.orchestra.run.smithi032.stdout:3981 writesame 0xca839cb thru 0xca8e63e data_size 0x4e(0xac74 bytes) 2024-09-21T15:00:34.429 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T15:00:34.442 INFO:teuthology.orchestra.run.smithi032.stdout:3982 writesame 0x27eab1d thru 0x27ebb35 data_size 0x1(0x1019 bytes) 2024-09-21T15:00:34.468 INFO:teuthology.orchestra.run.smithi032.stdout:3984 trunc from 0xe01a4c9 to 0xc38074 2024-09-21T15:00:34.474 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:00:34.470+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:00:34.519 INFO:teuthology.orchestra.run.smithi032.stdout:3985 trunc from 0xc38074 to 0x81d96c9 2024-09-21T15:00:34.529 INFO:teuthology.orchestra.run.smithi032.stdout:3986 punch from 0x17854fa to 0x178fc11, (0xa717 bytes) 2024-09-21T15:00:34.549 INFO:teuthology.orchestra.run.smithi032.stdout:3988 write 0xd28641c thru 0xd294414 (0xdff9 bytes) 2024-09-21T15:00:34.554 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:34.554 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:00:34.554 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T15:00:34.554 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T15:00:34.554 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:34.567 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T15:00:34.569 INFO:teuthology.orchestra.run.smithi032.stdout:3989 write 0xc914311 thru 0xc919e31 (0x5b21 bytes) 2024-09-21T15:00:34.574 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:00:34.570+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:00:34.581 INFO:teuthology.orchestra.run.smithi032.stdout:3990 write 0xe36116c thru 0xe368e46 (0x7cdb bytes) 2024-09-21T15:00:34.603 INFO:teuthology.orchestra.run.smithi032.stdout:3991 trunc from 0xe368e47 to 0x829b3c4 2024-09-21T15:00:34.623 INFO:teuthology.orchestra.run.smithi032.stdout:3993 read 0x1cb11b0 thru 0x1cbdd6b (0xcbbc bytes) 2024-09-21T15:00:34.623 INFO:teuthology.orchestra.run.smithi032.stdout:3994 read 0x340aa6 thru 0x34d71d (0xcc78 bytes) 2024-09-21T15:00:34.625 INFO:teuthology.orchestra.run.smithi032.stdout:3995 punch from 0x4340ca5 to 0x4341ccc, (0x1027 bytes) 2024-09-21T15:00:34.625 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:34.625 INFO:teuthology.orchestra.run.smithi032.stdout:3998 writesame 0x181904 thru 0x18a06c data_size 0x1(0x8769 bytes) 2024-09-21T15:00:34.639 INFO:teuthology.orchestra.run.smithi032.stdout:3999 writesame 0x7153289 thru 0x7160273 data_size 0x1f(0xcfeb bytes) 2024-09-21T15:00:34.655 INFO:teuthology.orchestra.run.smithi032.stdout:4000 write 0x7ef8836 thru 0x7f0721a (0xe9e5 bytes) 2024-09-21T15:00:34.675 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:00:34.670+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:00:34.680 INFO:teuthology.orchestra.run.smithi032.stdout:4001 writesame 0xe2e2d8c thru 0xe2f164e data_size 0x1(0xe8c3 bytes) 2024-09-21T15:00:34.681 INFO:teuthology.orchestra.run.smithi032.stdout:4002 read 0x21191f5 thru 0x211fbac (0x69b8 bytes) 2024-09-21T15:00:34.681 INFO:teuthology.orchestra.run.smithi032.stdout:4003 punch from 0xd527568 to 0xd528b66, (0x15fe bytes) 2024-09-21T15:00:34.681 INFO:teuthology.orchestra.run.smithi032.stdout:4004 writesame 0xd7f938a thru 0xd804932 data_size 0x47(0xb5a9 bytes) 2024-09-21T15:00:34.685 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:34.685 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T15:00:34.685 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T15:00:34.685 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T15:00:34.685 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.685 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:15130 3.2 3:47f1bbdb:::rbd_data.11361bed1f78.0000000000000067:head [sparse-read 0~147456] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-21T15:00:34.685 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:40.288284+0000", 2024-09-21T15:00:34.686 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 114.39693361499999, 2024-09-21T15:00:34.686 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1142935039999999, 2024-09-21T15:00:34.686 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:34.686 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:34.686 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:34.686 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:34.686 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:34.686 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 15130 2024-09-21T15:00:34.686 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.686 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:34.686 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.686 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:34.686 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288284+0000", 2024-09-21T15:00:34.686 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:34.687 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.687 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.687 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:34.687 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288284+0000", 2024-09-21T15:00:34.687 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:34.687 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.687 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.687 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:34.687 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288284+0000", 2024-09-21T15:00:34.687 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:34.687 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.687 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.687 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:34.687 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288288+0000", 2024-09-21T15:00:34.688 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5829999999999998e-06 2024-09-21T15:00:34.688 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.688 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.688 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:34.688 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288297+0000", 2024-09-21T15:00:34.688 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0669999999999998e-06 2024-09-21T15:00:34.688 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.688 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.688 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:34.688 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288305+0000", 2024-09-21T15:00:34.688 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8860000000000004e-06 2024-09-21T15:00:34.688 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.688 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.688 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:34.688 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288417+0000", 2024-09-21T15:00:34.689 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000112952 2024-09-21T15:00:34.689 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.689 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.689 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:00:34.689 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288428+0000", 2024-09-21T15:00:34.689 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0631e-05 2024-09-21T15:00:34.689 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.689 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.689 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:34.689 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.401876+0000", 2024-09-21T15:00:34.689 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1134479610000001 2024-09-21T15:00:34.689 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.689 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.689 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:34.689 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.402076+0000", 2024-09-21T15:00:34.690 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00019944200000000001 2024-09-21T15:00:34.690 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.690 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.690 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:34.690 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.402578+0000", 2024-09-21T15:00:34.690 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00050242899999999998 2024-09-21T15:00:34.690 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:34.690 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:34.690 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:34.690 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.690 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.690 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45708 3.2 3:415db91a:::rbd_data.1136ecbe5879.0000000000000000:28 [read 140288~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:34.690 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141801+0000", 2024-09-21T15:00:34.690 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 84.543417421000001, 2024-09-21T15:00:34.691 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034274200999999997, 2024-09-21T15:00:34.691 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:34.691 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:34.691 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:34.691 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:34.691 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:34.691 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45708 2024-09-21T15:00:34.691 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.691 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:34.691 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.691 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:34.691 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141801+0000", 2024-09-21T15:00:34.691 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:34.691 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.691 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.691 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:34.692 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141801+0000", 2024-09-21T15:00:34.692 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:34.692 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.692 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.692 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:34.692 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141800+0000", 2024-09-21T15:00:34.692 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:34.692 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.692 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.692 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:34.692 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141803+0000", 2024-09-21T15:00:34.692 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4210000000000002e-06 2024-09-21T15:00:34.692 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.692 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.693 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:34.693 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141812+0000", 2024-09-21T15:00:34.693 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.9800000000000004e-06 2024-09-21T15:00:34.693 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.693 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.693 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:34.693 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141816+0000", 2024-09-21T15:00:34.693 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4769999999999997e-06 2024-09-21T15:00:34.693 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.693 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.693 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:34.693 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.175865+0000", 2024-09-21T15:00:34.693 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034048875999999999 2024-09-21T15:00:34.693 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.693 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.694 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:34.694 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.175943+0000", 2024-09-21T15:00:34.694 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7464000000000002e-05 2024-09-21T15:00:34.694 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.694 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.694 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:34.694 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176075+0000", 2024-09-21T15:00:34.694 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013230800000000001 2024-09-21T15:00:34.694 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:34.694 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:34.694 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:34.694 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.694 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.694 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45709 3.2 3:415db91a:::rbd_data.1136ecbe5879.0000000000000000:28 [read 143360~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:34.694 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141820+0000", 2024-09-21T15:00:34.695 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 84.543397573999997, 2024-09-21T15:00:34.695 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034537536000000001, 2024-09-21T15:00:34.695 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:34.695 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:34.695 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:34.695 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:34.695 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:34.695 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45709 2024-09-21T15:00:34.695 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.695 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:34.695 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.695 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:34.695 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141820+0000", 2024-09-21T15:00:34.695 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:34.695 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.696 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.696 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:34.696 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141820+0000", 2024-09-21T15:00:34.696 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:34.696 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.696 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.696 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:34.696 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141820+0000", 2024-09-21T15:00:34.696 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:34.696 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.696 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.696 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:34.696 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141823+0000", 2024-09-21T15:00:34.696 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5110000000000002e-06 2024-09-21T15:00:34.696 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.697 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.697 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:34.697 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141829+0000", 2024-09-21T15:00:34.697 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0850000000000002e-06 2024-09-21T15:00:34.697 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.697 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.697 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:34.697 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141833+0000", 2024-09-21T15:00:34.697 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7620000000000003e-06 2024-09-21T15:00:34.697 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.697 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.697 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:34.697 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176103+0000", 2024-09-21T15:00:34.697 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034269933000000002 2024-09-21T15:00:34.697 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.698 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.698 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:34.698 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176161+0000", 2024-09-21T15:00:34.698 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7658999999999998e-05 2024-09-21T15:00:34.698 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.698 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.698 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:34.698 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176358+0000", 2024-09-21T15:00:34.698 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000196937 2024-09-21T15:00:34.698 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:34.698 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:34.698 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:34.698 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.699 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.699 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45710 3.2 3:415db91a:::rbd_data.1136ecbe5879.0000000000000000:28 [read 146432~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:34.699 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141838+0000", 2024-09-21T15:00:34.699 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 84.543380411000001, 2024-09-21T15:00:34.699 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034953674999999997, 2024-09-21T15:00:34.699 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:34.699 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:34.699 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:34.699 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:34.699 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:34.699 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45710 2024-09-21T15:00:34.699 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.699 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:34.699 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.699 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:34.699 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141838+0000", 2024-09-21T15:00:34.700 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:34.700 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.700 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.700 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:34.700 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141838+0000", 2024-09-21T15:00:34.700 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:34.700 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.700 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.700 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:34.700 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141837+0000", 2024-09-21T15:00:34.700 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:34.700 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.700 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.700 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:34.700 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141843+0000", 2024-09-21T15:00:34.701 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1619999999999996e-06 2024-09-21T15:00:34.701 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.701 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.701 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:34.701 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141850+0000", 2024-09-21T15:00:34.701 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4799999999999998e-06 2024-09-21T15:00:34.701 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.701 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.701 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:34.701 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141853+0000", 2024-09-21T15:00:34.701 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3050000000000001e-06 2024-09-21T15:00:34.701 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.701 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.701 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:34.701 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176490+0000", 2024-09-21T15:00:34.701 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034636413999999997 2024-09-21T15:00:34.702 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.702 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.702 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:34.702 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176606+0000", 2024-09-21T15:00:34.702 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000115841 2024-09-21T15:00:34.702 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.702 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.702 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:34.702 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176791+0000", 2024-09-21T15:00:34.702 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00018581500000000001 2024-09-21T15:00:34.702 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:34.702 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:34.702 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:34.702 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.702 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.703 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45711 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:34.703 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141856+0000", 2024-09-21T15:00:34.703 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 84.543362074000001, 2024-09-21T15:00:34.703 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034986308000000001, 2024-09-21T15:00:34.703 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:34.703 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:34.703 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:34.703 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:34.703 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:34.703 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45711 2024-09-21T15:00:34.703 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.703 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:34.703 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.703 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:34.703 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141856+0000", 2024-09-21T15:00:34.703 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:34.704 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.704 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.704 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:34.704 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141856+0000", 2024-09-21T15:00:34.704 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:34.704 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.704 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.704 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:34.704 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141856+0000", 2024-09-21T15:00:34.704 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:34.704 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.704 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.704 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:34.704 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141862+0000", 2024-09-21T15:00:34.704 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.567e-06 2024-09-21T15:00:34.705 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.705 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.705 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:34.705 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141868+0000", 2024-09-21T15:00:34.705 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9569999999999999e-06 2024-09-21T15:00:34.705 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.705 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.705 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:34.705 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141871+0000", 2024-09-21T15:00:34.705 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.146e-06 2024-09-21T15:00:34.705 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.705 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.705 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:34.705 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176831+0000", 2024-09-21T15:00:34.705 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034959935999999997 2024-09-21T15:00:34.706 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.706 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.706 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:34.706 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176842+0000", 2024-09-21T15:00:34.706 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1008e-05 2024-09-21T15:00:34.706 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:34.706 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:34.706 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:34.706 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.706 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.706 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45712 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:34.706 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141874+0000", 2024-09-21T15:00:34.706 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 84.543344179000002, 2024-09-21T15:00:34.706 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034993511999999997, 2024-09-21T15:00:34.707 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:34.707 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:34.707 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:34.707 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:34.707 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:34.707 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45712 2024-09-21T15:00:34.707 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.707 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:34.707 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.707 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:34.707 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141874+0000", 2024-09-21T15:00:34.707 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:34.707 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.707 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.708 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:34.708 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141874+0000", 2024-09-21T15:00:34.708 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:34.708 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.708 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.708 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:34.708 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141874+0000", 2024-09-21T15:00:34.708 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:34.708 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.708 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.708 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:34.708 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141875+0000", 2024-09-21T15:00:34.708 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5430000000000001e-06 2024-09-21T15:00:34.708 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.708 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.709 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:34.709 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141878+0000", 2024-09-21T15:00:34.709 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1420000000000002e-06 2024-09-21T15:00:34.709 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.709 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.709 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:34.709 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141881+0000", 2024-09-21T15:00:34.709 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.807e-06 2024-09-21T15:00:34.709 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.709 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.709 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:34.709 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176860+0000", 2024-09-21T15:00:34.709 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034979243 2024-09-21T15:00:34.709 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.709 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.709 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:34.710 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176867+0000", 2024-09-21T15:00:34.710 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0169999999999999e-06 2024-09-21T15:00:34.710 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:34.710 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:34.710 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:34.710 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.710 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.710 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45713 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:34.710 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141883+0000", 2024-09-21T15:00:34.710 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 84.543334603999995, 2024-09-21T15:00:34.710 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035010516999999998, 2024-09-21T15:00:34.710 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:34.710 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:34.710 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:34.711 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:34.711 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:34.711 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45713 2024-09-21T15:00:34.711 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.711 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:34.711 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.711 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:34.711 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141883+0000", 2024-09-21T15:00:34.711 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:34.711 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.711 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.711 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:34.711 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141883+0000", 2024-09-21T15:00:34.711 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:34.711 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.712 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.712 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:34.712 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141883+0000", 2024-09-21T15:00:34.712 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:34.712 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.712 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.712 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:34.712 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141885+0000", 2024-09-21T15:00:34.712 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4649999999999999e-06 2024-09-21T15:00:34.712 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.712 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.712 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:34.712 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141888+0000", 2024-09-21T15:00:34.712 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3890000000000001e-06 2024-09-21T15:00:34.712 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.713 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.713 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:34.713 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141891+0000", 2024-09-21T15:00:34.713 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9239999999999999e-06 2024-09-21T15:00:34.713 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.713 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.713 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:34.713 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176887+0000", 2024-09-21T15:00:34.713 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034996370999999998 2024-09-21T15:00:34.713 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.713 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.713 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:34.713 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176894+0000", 2024-09-21T15:00:34.713 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6069999999999996e-06 2024-09-21T15:00:34.713 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:34.713 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:34.714 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:34.714 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.714 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.714 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45714 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:34.714 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141893+0000", 2024-09-21T15:00:34.714 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 84.543324689000002, 2024-09-21T15:00:34.714 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035048215000000001, 2024-09-21T15:00:34.714 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:34.714 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:34.714 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:34.714 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:34.714 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:34.714 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45714 2024-09-21T15:00:34.714 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.715 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:34.715 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.715 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:34.715 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141893+0000", 2024-09-21T15:00:34.715 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:34.715 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.715 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.715 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:34.715 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141893+0000", 2024-09-21T15:00:34.715 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:34.715 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.715 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.715 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:34.715 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141893+0000", 2024-09-21T15:00:34.716 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:34.716 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.716 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.716 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:34.716 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141895+0000", 2024-09-21T15:00:34.716 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.703e-06 2024-09-21T15:00:34.716 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.716 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.716 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:34.716 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141900+0000", 2024-09-21T15:00:34.716 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.242e-06 2024-09-21T15:00:34.716 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.716 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.716 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:34.716 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141903+0000", 2024-09-21T15:00:34.717 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.117e-06 2024-09-21T15:00:34.717 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.717 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.717 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:34.717 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176930+0000", 2024-09-21T15:00:34.717 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035026750000000002 2024-09-21T15:00:34.717 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.717 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.717 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:34.717 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176942+0000", 2024-09-21T15:00:34.717 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1668000000000001e-05 2024-09-21T15:00:34.717 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:34.717 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:34.717 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:34.718 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.718 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.718 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45715 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:34.718 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141906+0000", 2024-09-21T15:00:34.718 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 84.543312530999998, 2024-09-21T15:00:34.718 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035066057999999997, 2024-09-21T15:00:34.718 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:34.718 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:34.718 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:34.718 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:34.718 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:34.718 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45715 2024-09-21T15:00:34.718 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.718 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:34.719 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.719 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:34.719 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141906+0000", 2024-09-21T15:00:34.719 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:34.719 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.719 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.719 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:34.719 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141906+0000", 2024-09-21T15:00:34.719 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:34.719 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.719 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.719 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:34.719 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141905+0000", 2024-09-21T15:00:34.719 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:34.720 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.720 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.720 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:34.720 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141907+0000", 2024-09-21T15:00:34.720 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5400000000000001e-06 2024-09-21T15:00:34.720 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.720 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.720 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:34.720 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141912+0000", 2024-09-21T15:00:34.720 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.502e-06 2024-09-21T15:00:34.720 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.720 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.720 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:34.720 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141915+0000", 2024-09-21T15:00:34.721 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0170000000000001e-06 2024-09-21T15:00:34.721 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.721 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.721 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:34.721 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176964+0000", 2024-09-21T15:00:34.721 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035048587999999999 2024-09-21T15:00:34.721 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.721 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.721 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:34.721 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176972+0000", 2024-09-21T15:00:34.721 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6580000000000002e-06 2024-09-21T15:00:34.721 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:34.721 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:34.721 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:34.721 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.722 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.722 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45716 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:34.722 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141918+0000", 2024-09-21T15:00:34.722 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 84.543300355, 2024-09-21T15:00:34.722 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035086223, 2024-09-21T15:00:34.722 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:34.722 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:34.722 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:34.722 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:34.722 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:34.722 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45716 2024-09-21T15:00:34.722 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.722 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:34.722 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.722 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:34.723 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141918+0000", 2024-09-21T15:00:34.723 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:34.723 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.723 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.723 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:34.723 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141918+0000", 2024-09-21T15:00:34.723 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:34.723 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.723 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.723 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:34.723 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141917+0000", 2024-09-21T15:00:34.723 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:34.723 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.724 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.724 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:34.724 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141919+0000", 2024-09-21T15:00:34.724 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5680000000000001e-06 2024-09-21T15:00:34.724 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.724 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.724 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:34.724 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141922+0000", 2024-09-21T15:00:34.724 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2789999999999999e-06 2024-09-21T15:00:34.724 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.724 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.724 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:34.724 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141925+0000", 2024-09-21T15:00:34.724 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9469999999999999e-06 2024-09-21T15:00:34.724 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.725 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.725 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:34.725 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176997+0000", 2024-09-21T15:00:34.725 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035071366999999999 2024-09-21T15:00:34.725 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.725 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.725 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:34.725 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177004+0000", 2024-09-21T15:00:34.725 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3309999999999998e-06 2024-09-21T15:00:34.725 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:34.725 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:34.725 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:34.725 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.725 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.725 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45717 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:34.726 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141928+0000", 2024-09-21T15:00:34.726 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 84.543290408999994, 2024-09-21T15:00:34.726 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035102573999999998, 2024-09-21T15:00:34.726 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:34.726 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:34.726 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:34.726 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:34.726 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:34.726 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45717 2024-09-21T15:00:34.726 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.726 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:34.726 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.726 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:34.726 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141928+0000", 2024-09-21T15:00:34.727 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:34.727 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.727 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.727 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:34.727 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141928+0000", 2024-09-21T15:00:34.727 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:34.727 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.727 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.727 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:34.727 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141927+0000", 2024-09-21T15:00:34.727 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:00:34.727 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.727 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.727 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:34.727 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141929+0000", 2024-09-21T15:00:34.728 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.533e-06 2024-09-21T15:00:34.728 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.728 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.728 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:34.728 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141934+0000", 2024-09-21T15:00:34.728 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1660000000000002e-06 2024-09-21T15:00:34.728 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.728 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.728 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:34.728 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141937+0000", 2024-09-21T15:00:34.728 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0309999999999999e-06 2024-09-21T15:00:34.728 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.728 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.728 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:34.729 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177023+0000", 2024-09-21T15:00:34.729 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035085595999999997 2024-09-21T15:00:34.729 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.729 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.729 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:34.729 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177030+0000", 2024-09-21T15:00:34.729 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.486e-06 2024-09-21T15:00:34.729 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:34.729 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:34.729 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:34.729 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.729 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.729 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45718 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:34.729 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141940+0000", 2024-09-21T15:00:34.730 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 84.543278551, 2024-09-21T15:00:34.730 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035124956999999998, 2024-09-21T15:00:34.730 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:34.730 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:34.730 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:34.730 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:34.730 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:34.730 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45718 2024-09-21T15:00:34.730 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.730 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:34.730 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.730 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:34.730 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141940+0000", 2024-09-21T15:00:34.730 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:34.731 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.731 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.731 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:34.731 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141940+0000", 2024-09-21T15:00:34.731 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:34.731 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.731 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.731 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:34.731 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141939+0000", 2024-09-21T15:00:34.731 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:34.731 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.731 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.731 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:34.731 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141941+0000", 2024-09-21T15:00:34.731 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.502e-06 2024-09-21T15:00:34.732 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.732 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.732 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:34.732 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141946+0000", 2024-09-21T15:00:34.732 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3449999999999996e-06 2024-09-21T15:00:34.732 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.732 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.732 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:34.732 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141949+0000", 2024-09-21T15:00:34.732 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9749999999999999e-06 2024-09-21T15:00:34.732 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.732 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.732 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:34.732 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177057+0000", 2024-09-21T15:00:34.732 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035108385999999998 2024-09-21T15:00:34.732 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.733 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.733 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:34.733 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177064+0000", 2024-09-21T15:00:34.733 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0099999999999998e-06 2024-09-21T15:00:34.733 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:34.733 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:34.733 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:34.733 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.733 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.733 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45719 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:34.733 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141952+0000", 2024-09-21T15:00:34.733 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 84.543266278000004, 2024-09-21T15:00:34.733 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035139108000000002, 2024-09-21T15:00:34.733 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:34.733 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:34.734 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:34.734 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:34.734 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:34.734 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45719 2024-09-21T15:00:34.734 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.734 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:34.734 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.734 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:34.734 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141952+0000", 2024-09-21T15:00:34.734 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:34.734 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.734 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.734 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:34.734 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141952+0000", 2024-09-21T15:00:34.734 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:34.735 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.735 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.735 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:34.735 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141952+0000", 2024-09-21T15:00:34.735 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:34.735 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.735 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.735 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:34.735 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141953+0000", 2024-09-21T15:00:34.735 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.486e-06 2024-09-21T15:00:34.735 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.735 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.735 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:34.735 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141959+0000", 2024-09-21T15:00:34.735 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4940000000000003e-06 2024-09-21T15:00:34.736 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.736 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.736 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:34.736 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141963+0000", 2024-09-21T15:00:34.736 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3540000000000002e-06 2024-09-21T15:00:34.736 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.736 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.736 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:34.736 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177084+0000", 2024-09-21T15:00:34.736 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035120983000000001 2024-09-21T15:00:34.736 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.736 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.736 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:34.736 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177091+0000", 2024-09-21T15:00:34.736 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0350000000000004e-06 2024-09-21T15:00:34.736 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:34.737 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:34.737 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:34.737 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.737 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.737 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45720 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:34.737 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141965+0000", 2024-09-21T15:00:34.737 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 84.543252811000002, 2024-09-21T15:00:34.737 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035155268000000003, 2024-09-21T15:00:34.737 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:34.737 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:34.737 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:34.737 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:34.737 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:34.737 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45720 2024-09-21T15:00:34.737 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.738 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:34.738 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.738 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:34.738 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141965+0000", 2024-09-21T15:00:34.738 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:34.738 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.738 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.738 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:34.738 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141965+0000", 2024-09-21T15:00:34.738 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:34.738 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.738 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.738 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:34.738 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141965+0000", 2024-09-21T15:00:34.738 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:34.739 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.739 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.739 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:34.739 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141966+0000", 2024-09-21T15:00:34.739 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.46e-06 2024-09-21T15:00:34.739 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.739 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.739 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:34.739 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141972+0000", 2024-09-21T15:00:34.739 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4210000000000003e-06 2024-09-21T15:00:34.739 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.739 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.739 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:34.739 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141975+0000", 2024-09-21T15:00:34.740 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.948e-06 2024-09-21T15:00:34.740 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.740 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.740 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:34.740 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177113+0000", 2024-09-21T15:00:34.740 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035137855000000003 2024-09-21T15:00:34.740 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.740 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.740 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:34.740 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177121+0000", 2024-09-21T15:00:34.740 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8269999999999998e-06 2024-09-21T15:00:34.740 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:34.740 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:34.740 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:34.740 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.740 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.741 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45721 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:34.741 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141977+0000", 2024-09-21T15:00:34.741 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 84.543240788000006, 2024-09-21T15:00:34.741 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035169984000000001, 2024-09-21T15:00:34.741 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:34.741 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:34.741 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:34.741 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:34.741 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:34.741 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45721 2024-09-21T15:00:34.741 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.741 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:34.741 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.741 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:34.742 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141977+0000", 2024-09-21T15:00:34.742 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:34.742 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.742 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.742 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:34.742 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141977+0000", 2024-09-21T15:00:34.742 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:34.742 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.742 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.742 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:34.742 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141977+0000", 2024-09-21T15:00:34.742 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:34.742 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.742 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.742 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:34.743 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141979+0000", 2024-09-21T15:00:34.743 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.511e-06 2024-09-21T15:00:34.743 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.743 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.743 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:34.743 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141982+0000", 2024-09-21T15:00:34.743 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4180000000000001e-06 2024-09-21T15:00:34.743 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.743 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.743 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:34.743 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141985+0000", 2024-09-21T15:00:34.743 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9450000000000002e-06 2024-09-21T15:00:34.743 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.743 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.743 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:34.743 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177139+0000", 2024-09-21T15:00:34.744 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035154409999999997 2024-09-21T15:00:34.744 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.744 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.744 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:34.744 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177147+0000", 2024-09-21T15:00:34.744 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9570000000000002e-06 2024-09-21T15:00:34.744 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:34.744 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:34.744 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:34.744 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.744 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.744 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45722 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:34.744 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141987+0000", 2024-09-21T15:00:34.744 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 84.543230816999994, 2024-09-21T15:00:34.745 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035188492000000002, 2024-09-21T15:00:34.745 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:34.745 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:34.745 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:34.745 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:34.745 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:34.745 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45722 2024-09-21T15:00:34.745 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.745 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:34.745 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.745 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:34.745 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141987+0000", 2024-09-21T15:00:34.745 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:34.745 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.745 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.746 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:34.746 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141987+0000", 2024-09-21T15:00:34.746 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:34.746 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.746 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.746 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:34.746 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141987+0000", 2024-09-21T15:00:34.746 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:34.746 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.746 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.746 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:34.746 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141988+0000", 2024-09-21T15:00:34.746 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.463e-06 2024-09-21T15:00:34.746 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.746 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.747 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:34.747 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141994+0000", 2024-09-21T15:00:34.747 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.361e-06 2024-09-21T15:00:34.747 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.747 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.747 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:34.747 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141997+0000", 2024-09-21T15:00:34.747 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0359999999999998e-06 2024-09-21T15:00:34.747 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.747 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.747 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:34.747 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177169+0000", 2024-09-21T15:00:34.747 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035171651999999998 2024-09-21T15:00:34.747 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.747 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.748 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:34.748 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177176+0000", 2024-09-21T15:00:34.748 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2189999999999998e-06 2024-09-21T15:00:34.748 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:34.748 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:34.748 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:34.748 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.748 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.748 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45723 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:34.748 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141999+0000", 2024-09-21T15:00:34.748 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 84.54321865, 2024-09-21T15:00:34.748 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035197701999999997, 2024-09-21T15:00:34.748 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:34.748 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:34.748 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:34.749 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:34.749 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:34.749 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45723 2024-09-21T15:00:34.749 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.749 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:34.749 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.749 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:34.749 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141999+0000", 2024-09-21T15:00:34.749 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:34.749 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.749 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.749 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:34.749 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141999+0000", 2024-09-21T15:00:34.749 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:34.750 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.750 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.750 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:34.750 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141999+0000", 2024-09-21T15:00:34.750 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:34.750 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.750 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.750 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:34.750 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142003+0000", 2024-09-21T15:00:34.750 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5049999999999999e-06 2024-09-21T15:00:34.750 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.750 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.750 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:34.750 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142008+0000", 2024-09-21T15:00:34.750 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6579999999999999e-06 2024-09-21T15:00:34.751 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.751 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.751 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:34.751 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142011+0000", 2024-09-21T15:00:34.751 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0529999999999998e-06 2024-09-21T15:00:34.751 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.751 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.751 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:34.751 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177194+0000", 2024-09-21T15:00:34.751 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035182221999999999 2024-09-21T15:00:34.751 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.751 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.751 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:34.751 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177197+0000", 2024-09-21T15:00:34.752 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5030000000000002e-06 2024-09-21T15:00:34.752 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:34.752 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:34.752 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:34.752 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.752 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.752 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45724 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:34.752 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.142015+0000", 2024-09-21T15:00:34.752 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 84.543202683000004, 2024-09-21T15:00:34.752 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035195345000000003, 2024-09-21T15:00:34.752 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:34.752 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:34.752 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:34.752 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:34.752 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:34.753 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45724 2024-09-21T15:00:34.753 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.753 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:34.753 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.753 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:34.753 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142015+0000", 2024-09-21T15:00:34.753 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:34.753 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.753 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.753 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:34.753 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142015+0000", 2024-09-21T15:00:34.753 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:34.753 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.753 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.753 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:34.754 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142015+0000", 2024-09-21T15:00:34.754 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:34.754 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.754 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.754 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:34.754 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142017+0000", 2024-09-21T15:00:34.754 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5540000000000001e-06 2024-09-21T15:00:34.754 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.754 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.754 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:34.754 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142022+0000", 2024-09-21T15:00:34.754 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4229999999999996e-06 2024-09-21T15:00:34.754 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.754 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.754 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:34.755 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142025+0000", 2024-09-21T15:00:34.755 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.056e-06 2024-09-21T15:00:34.755 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.755 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.755 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:34.755 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177207+0000", 2024-09-21T15:00:34.755 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035182089 2024-09-21T15:00:34.755 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.755 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.755 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:34.755 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177211+0000", 2024-09-21T15:00:34.755 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4649999999999999e-06 2024-09-21T15:00:34.755 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:34.755 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:34.755 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:34.756 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.756 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.756 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45725 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:34.756 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.142028+0000", 2024-09-21T15:00:34.756 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 84.543190404000001, 2024-09-21T15:00:34.756 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035195731000000001, 2024-09-21T15:00:34.756 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:34.756 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:34.756 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:34.756 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:34.756 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:34.756 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45725 2024-09-21T15:00:34.756 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.756 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:34.756 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.757 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:34.757 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142028+0000", 2024-09-21T15:00:34.757 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:34.757 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.757 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.757 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:34.757 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142028+0000", 2024-09-21T15:00:34.757 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:34.757 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.757 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.757 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:34.757 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142027+0000", 2024-09-21T15:00:34.757 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:34.757 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.757 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.757 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:34.758 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142029+0000", 2024-09-21T15:00:34.758 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5269999999999999e-06 2024-09-21T15:00:34.758 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.758 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.758 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:34.758 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142032+0000", 2024-09-21T15:00:34.758 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3249999999999999e-06 2024-09-21T15:00:34.758 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.758 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.758 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:34.758 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142035+0000", 2024-09-21T15:00:34.758 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.897e-06 2024-09-21T15:00:34.758 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.759 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.759 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:34.759 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177220+0000", 2024-09-21T15:00:34.759 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035184871 2024-09-21T15:00:34.759 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.759 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.759 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:34.759 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177223+0000", 2024-09-21T15:00:34.759 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3680000000000002e-06 2024-09-21T15:00:34.759 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:34.759 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:34.759 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:34.759 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.759 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.760 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45726 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:34.760 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.142037+0000", 2024-09-21T15:00:34.760 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 84.543180688999996, 2024-09-21T15:00:34.760 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035199273000000003, 2024-09-21T15:00:34.760 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:34.760 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:34.760 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:34.760 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:34.760 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:34.760 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45726 2024-09-21T15:00:34.760 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.760 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:34.760 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.760 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:34.760 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142037+0000", 2024-09-21T15:00:34.761 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:34.761 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.761 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.761 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:34.761 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142037+0000", 2024-09-21T15:00:34.761 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:34.761 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.761 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.761 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:34.761 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142037+0000", 2024-09-21T15:00:34.761 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:34.761 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.761 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.761 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:34.761 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142038+0000", 2024-09-21T15:00:34.761 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.344e-06 2024-09-21T15:00:34.762 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.762 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.762 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:34.762 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142044+0000", 2024-09-21T15:00:34.762 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1930000000000001e-06 2024-09-21T15:00:34.762 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.762 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.762 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:34.762 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142047+0000", 2024-09-21T15:00:34.762 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0280000000000001e-06 2024-09-21T15:00:34.762 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.762 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.762 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:34.762 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177233+0000", 2024-09-21T15:00:34.763 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035186394000000003 2024-09-21T15:00:34.763 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:34.763 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:34.763 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:34.763 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177237+0000", 2024-09-21T15:00:34.763 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5530000000000001e-06 2024-09-21T15:00:34.763 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:34.763 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:34.763 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:34.763 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:34.763 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:34.763 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:34.764 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T15:00:34.766 INFO:teuthology.orchestra.run.smithi032.stdout:4005 trunc from 0xe2f164f to 0x855a158 2024-09-21T15:00:34.766 INFO:teuthology.orchestra.run.smithi032.stdout:4010 punch from 0x75daf14 to 0x75e5cda, (0xadc6 bytes) 2024-09-21T15:00:34.766 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:34.766 INFO:teuthology.orchestra.run.smithi032.stdout:4013 write 0x8424762 thru 0x8432ec3 (0xe762 bytes) 2024-09-21T15:00:34.766 INFO:teuthology.orchestra.run.smithi032.stdout:4014 read 0x3db248a thru 0x3dbb1c0 (0x8d37 bytes) 2024-09-21T15:00:34.766 INFO:teuthology.orchestra.run.smithi032.stdout:4015 punch from 0x259a3a2 to 0x259a4db, (0x139 bytes) 2024-09-21T15:00:34.766 INFO:teuthology.orchestra.run.smithi032.stdout:4016 trunc from 0x855a158 to 0xd067474 2024-09-21T15:00:34.766 INFO:teuthology.orchestra.run.smithi032.stdout:4017 punch from 0xb968db7 to 0xb96a0d0, (0x1319 bytes) 2024-09-21T15:00:34.766 INFO:teuthology.orchestra.run.smithi032.stdout:4018 writesame 0x55c6c4f thru 0x55cab1f data_size 0x1(0x3ed1 bytes) 2024-09-21T15:00:34.776 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:00:34.771+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:00:34.777 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:34.778 INFO:teuthology.orchestra.run.smithi032.stdout:4021 punch from 0x89b8609 to 0x89c4c2a, (0xc621 bytes) 2024-09-21T15:00:34.796 INFO:teuthology.orchestra.run.smithi032.stdout:4022 trunc from 0xd067474 to 0x2e67df1 2024-09-21T15:00:34.846 INFO:teuthology.orchestra.run.smithi032.stdout:4024 writesame 0xeb7ddd4 thru 0xeb81bfe data_size 0x5(0x3e2b bytes) 2024-09-21T15:00:34.846 INFO:teuthology.orchestra.run.smithi032.stdout:4025 punch from 0x35562ab to 0x3564f04, (0xec59 bytes) 2024-09-21T15:00:34.846 INFO:teuthology.orchestra.run.smithi032.stdout:4026 punch from 0x512941f to 0x5131cbb, (0x889c bytes) 2024-09-21T15:00:34.846 INFO:teuthology.orchestra.run.smithi032.stdout:4028 writesame 0x1bf23e9 thru 0x1bf9d22 data_size 0x3b(0x793a bytes) 2024-09-21T15:00:34.855 INFO:teuthology.orchestra.run.smithi032.stdout:4029 read 0x7b6b84b thru 0x7b72874 (0x702a bytes) 2024-09-21T15:00:34.855 INFO:teuthology.orchestra.run.smithi032.stdout:4030 write 0x2cf6e0b thru 0x2cf8fe5 (0x21db bytes) 2024-09-21T15:00:34.863 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:34.863 INFO:teuthology.orchestra.run.smithi032.stdout:4032 read 0x56b82a9 thru 0x56c5870 (0xd5c8 bytes) 2024-09-21T15:00:34.863 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:34.863 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:34.863 INFO:teuthology.orchestra.run.smithi032.stdout:4035 read 0xd24d308 thru 0xd25bfa9 (0xeca2 bytes) 2024-09-21T15:00:34.863 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:34.863 INFO:teuthology.orchestra.run.smithi032.stdout:4038 write 0xbb51673 thru 0xbb5ba9f (0xa42d bytes) 2024-09-21T15:00:34.877 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:00:34.871+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:00:34.877 INFO:teuthology.orchestra.run.smithi032.stdout:4039 read 0x2a0bc0e thru 0x2a16565 (0xa958 bytes) 2024-09-21T15:00:34.878 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:34.879 INFO:teuthology.orchestra.run.smithi032.stdout:4041 read 0xb5e8d33 thru 0xb5f4cef (0xbfbd bytes) 2024-09-21T15:00:34.879 INFO:teuthology.orchestra.run.smithi032.stdout:4042 write 0xe4fd909 thru 0xe502370 (0x4a68 bytes) 2024-09-21T15:00:34.888 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:34.888 INFO:teuthology.orchestra.run.smithi032.stdout:4045 writesame 0xdd51df2 thru 0xdd585b5 data_size 0x3a(0x67c4 bytes) 2024-09-21T15:00:34.895 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:34.895 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:00:34.895 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T15:00:34.895 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:34.896 INFO:teuthology.orchestra.run.smithi032.stdout:4046 write 0xa1c8bb4 thru 0xa1d3da9 (0xb1f6 bytes) 2024-09-21T15:00:34.906 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T15:00:34.911 INFO:teuthology.orchestra.run.smithi032.stdout:4048 read 0xc2ad9a9 thru 0xc2bcf3a (0xf592 bytes) 2024-09-21T15:00:34.929 INFO:teuthology.orchestra.run.smithi032.stdout:4049 trunc from 0xeb81bff to 0x21ae2c0 2024-09-21T15:00:34.956 INFO:teuthology.orchestra.run.smithi032.stdout:4050 read 0x1d0db0f thru 0x1d18c63 (0xb155 bytes) 2024-09-21T15:00:34.977 INFO:teuthology.orchestra.run.smithi032.stdout:4051 write 0x8d1ac33 thru 0x8d2464f (0x9a1d bytes) 2024-09-21T15:00:34.977 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:00:34.970+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:00:34.990 INFO:teuthology.orchestra.run.smithi032.stdout:4052 read 0x2f5e3b0 thru 0x2f67c4a (0x989b bytes) 2024-09-21T15:00:35.008 INFO:teuthology.orchestra.run.smithi032.stdout:4053 write 0xe383b80 thru 0xe391eee (0xe36f bytes) 2024-09-21T15:00:35.031 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:35.031 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:00:35.031 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T15:00:35.031 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T15:00:35.031 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:35.038 INFO:teuthology.orchestra.run.smithi032.stdout:4054 trunc from 0xe391eef to 0x7c31c99 2024-09-21T15:00:35.044 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T15:00:35.061 INFO:teuthology.orchestra.run.smithi032.stdout:4055 trunc from 0x7c31c99 to 0x5a80b5c 2024-09-21T15:00:35.072 INFO:teuthology.orchestra.run.smithi032.stdout:4059 punch from 0x477c55a to 0x478a1c1, (0xdc67 bytes) 2024-09-21T15:00:35.072 INFO:teuthology.orchestra.run.smithi032.stdout:4060 read 0x4ce2375 thru 0x4cefcb3 (0xd93f bytes) 2024-09-21T15:00:35.072 INFO:teuthology.orchestra.run.smithi032.stdout:4061 write 0x5a23b34 thru 0x5a33790 (0xfc5d bytes) 2024-09-21T15:00:35.078 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:00:35.075+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:00:35.092 INFO:teuthology.orchestra.run.smithi032.stdout:4062 read 0x4041e81 thru 0x4041e90 (0x10 bytes) 2024-09-21T15:00:35.092 INFO:teuthology.orchestra.run.smithi032.stdout:4063 read 0x5441aaa thru 0x54477bb (0x5d12 bytes) 2024-09-21T15:00:35.093 INFO:teuthology.orchestra.run.smithi032.stdout:4064 trunc from 0x5a80b5c to 0x4ba8895 2024-09-21T15:00:35.112 INFO:teuthology.orchestra.run.smithi032.stdout:4065 punch from 0x1919741 to 0x191d238, (0x3af7 bytes) 2024-09-21T15:00:35.127 INFO:teuthology.orchestra.run.smithi032.stdout:4067 trunc from 0x4ba8895 to 0xcbac495 2024-09-21T15:00:35.137 INFO:teuthology.orchestra.run.smithi032.stdout:4068 read 0x1e23ffb thru 0x1e274b2 (0x34b8 bytes) 2024-09-21T15:00:35.137 INFO:teuthology.orchestra.run.smithi032.stdout:4069 read 0xb664a6a thru 0xb6701e1 (0xb778 bytes) 2024-09-21T15:00:35.148 INFO:teuthology.orchestra.run.smithi032.stdout:4071 trunc from 0xcbac495 to 0x50e0540 2024-09-21T15:00:35.161 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:35.161 INFO:teuthology.orchestra.run.smithi032.stdout:4074 read 0x1dd70a5 thru 0x1de3656 (0xc5b2 bytes) 2024-09-21T15:00:35.163 INFO:teuthology.orchestra.run.smithi032.stdout:4075 read 0x1022c6a thru 0x102d8a4 (0xac3b bytes) 2024-09-21T15:00:35.164 INFO:teuthology.orchestra.run.smithi032.stdout:4077 write 0x33a256e thru 0x33a757b (0x500e bytes) 2024-09-21T15:00:35.175 INFO:teuthology.orchestra.run.smithi032.stdout:4079 write 0x29025d2 thru 0x290a8c2 (0x82f1 bytes) 2024-09-21T15:00:35.179 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:00:35.174+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:00:35.182 INFO:teuthology.orchestra.run.smithi032.stdout:4080 clone 30 order 18 su 32768 sc 11 2024-09-21T15:00:35.191 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:35.191 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T15:00:35.191 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T15:00:35.191 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T15:00:35.192 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.192 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4281.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-21T15:00:35.192 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:09.324416+0000", 2024-09-21T15:00:35.192 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 145.865972215, 2024-09-21T15:00:35.192 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.117837947, 2024-09-21T15:00:35.192 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:35.192 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:35.192 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:35.192 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4281", 2024-09-21T15:00:35.192 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/1518520303", 2024-09-21T15:00:35.192 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1 2024-09-21T15:00:35.192 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.192 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:35.192 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.192 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:35.192 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324416+0000", 2024-09-21T15:00:35.193 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:35.193 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.193 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.193 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:35.193 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324416+0000", 2024-09-21T15:00:35.193 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:35.193 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.193 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.193 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:35.193 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324414+0000", 2024-09-21T15:00:35.193 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T15:00:35.193 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.193 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.193 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:35.194 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324429+0000", 2024-09-21T15:00:35.194 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5637e-05 2024-09-21T15:00:35.194 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.194 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.194 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:35.194 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324448+0000", 2024-09-21T15:00:35.194 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9042000000000002e-05 2024-09-21T15:00:35.194 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.194 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.194 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:35.194 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324465+0000", 2024-09-21T15:00:35.194 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6767999999999999e-05 2024-09-21T15:00:35.194 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.194 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.194 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:35.195 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324571+0000", 2024-09-21T15:00:35.195 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000105714 2024-09-21T15:00:35.195 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.195 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.195 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op must wait for map", 2024-09-21T15:00:35.195 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324587+0000", 2024-09-21T15:00:35.195 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5753999999999999e-05 2024-09-21T15:00:35.195 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.195 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.195 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:35.195 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.325864+0000", 2024-09-21T15:00:35.195 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0012773350000000001 2024-09-21T15:00:35.195 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.195 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.195 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:00:35.196 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.325878+0000", 2024-09-21T15:00:35.196 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4498000000000001e-05 2024-09-21T15:00:35.196 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.196 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.196 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:35.196 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.439461+0000", 2024-09-21T15:00:35.196 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.11358212500000001 2024-09-21T15:00:35.196 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.196 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.196 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:35.196 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.439848+0000", 2024-09-21T15:00:35.196 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00038747800000000002 2024-09-21T15:00:35.196 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.196 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.196 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T15:00:35.196 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.440175+0000", 2024-09-21T15:00:35.197 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00032721900000000001 2024-09-21T15:00:35.197 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.197 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.197 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:35.197 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.441770+0000", 2024-09-21T15:00:35.197 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0015949409999999999 2024-09-21T15:00:35.197 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.197 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.197 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:35.197 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.442086+0000", 2024-09-21T15:00:35.197 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00031586299999999999 2024-09-21T15:00:35.197 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.197 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.197 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:35.197 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.442195+0000", 2024-09-21T15:00:35.197 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000108791 2024-09-21T15:00:35.198 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.198 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.198 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:35.198 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.442254+0000", 2024-09-21T15:00:35.198 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9038e-05 2024-09-21T15:00:35.198 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:35.198 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:35.198 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:35.198 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.198 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.198 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4281.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-21T15:00:35.198 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:09.447828+0000", 2024-09-21T15:00:35.198 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 145.74256021400001, 2024-09-21T15:00:35.198 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.32904393700000001, 2024-09-21T15:00:35.198 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:35.199 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:35.199 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:35.199 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4281", 2024-09-21T15:00:35.199 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/1518520303", 2024-09-21T15:00:35.199 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 2 2024-09-21T15:00:35.199 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.199 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:35.199 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.199 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:35.199 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447828+0000", 2024-09-21T15:00:35.199 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:35.199 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.199 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.199 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:35.199 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447828+0000", 2024-09-21T15:00:35.200 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:35.200 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.200 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.200 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:35.200 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447825+0000", 2024-09-21T15:00:35.200 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T15:00:35.200 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.200 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.200 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:35.200 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447841+0000", 2024-09-21T15:00:35.200 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5359999999999999e-05 2024-09-21T15:00:35.200 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.200 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.200 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:35.201 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447875+0000", 2024-09-21T15:00:35.201 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4388999999999999e-05 2024-09-21T15:00:35.201 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.201 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.201 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:35.201 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447903+0000", 2024-09-21T15:00:35.201 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7373e-05 2024-09-21T15:00:35.201 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.201 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.201 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:35.201 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447996+0000", 2024-09-21T15:00:35.201 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2954999999999993e-05 2024-09-21T15:00:35.201 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.201 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.201 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:00:35.201 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.448011+0000", 2024-09-21T15:00:35.202 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5038999999999999e-05 2024-09-21T15:00:35.202 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.202 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.202 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:35.202 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.776501+0000", 2024-09-21T15:00:35.202 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.32849012500000002 2024-09-21T15:00:35.202 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.202 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.202 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:35.202 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.776872+0000", 2024-09-21T15:00:35.202 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00037119600000000002 2024-09-21T15:00:35.202 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:35.202 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:35.202 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:35.203 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.203 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.203 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:97673 3.1d 3:bd91946e:::rbd_data.1136bfc79a70.0000000000000002:head [write 0~4718592 [fadvise_sequential+fadvise_nocache],write 5242880~32768 [fadvise_sequential+fadvise_nocache],write 5341184~147456 [fadvise_sequential+fadvise_nocache],write 5554176~147456 [fadvise_sequential+fadvise_nocache],write 5767168~16384 [fadvise_sequential+fadvise_nocache],write 5849088~147456 [fadvise_sequential+fadvise_nocache],write 6062080~147456 [fadvise_sequential+fadvise_nocache],write 6275072~16384 [fadvise_sequential+fadvise_nocache],write 6356992~147456 [fadvise_sequential+fadvise_nocache],write 6569984~147456 [fadvise_sequential+fadvise_nocache],write 6782976~32768 [fadvise_sequential+fadvise_nocache],write 6864896~147456 [fadvise_sequential+fadvise_nocache],write 7077888~147456 [fadvise_sequential+fadvise_nocache],write 7290880~49152 [fadvise_sequential+fadvise_nocache],write 7372800~147456 [fadvise_sequential+fadvise_nocache],write 7585792~147456 [fadvise_sequential+fadvise_nocache],write 7798784~65536 [fadvise_sequential+fadvise_nocache],write 7880704~147456 [fadvise_sequential+fadvise_nocache],write 8093696~147456 [fadvise_sequential+fadvise_nocache],write 8306688~81920 [fadvise_sequential+fadvise_nocache],write 8421376~114688 [fadvise_sequential+fadvise_nocache],write 8634368~114688 [fadvise_sequential+fadvise_nocache],write 8847360~65536 [fadvise_sequential+fadvise_nocache],write 9011200~32768 [fadvise_sequential+fadvise_nocache],write 9224192~32768 [fadvise_sequential+fadvise_nocache],write 9519104~32768 [fadvise_sequential+fadvise_nocache],write 9732096~32768 [fadvise_sequential+fadvise_nocache],write 9945088~16384 [fadvise_sequential+fadvise_nocache],write 9977856~196608 [fadvise_sequential+fadvise_nocache],write 10190848~196608 [fadvise_sequential+fadvise_nocache],write 10403840~278528 [fadvise_sequential+fadvise_nocache],write 10698752~196608 [fadvise_sequential+fadvise_nocache],write 10911744~278528 [fadvise_sequential+fadvise_nocache],write 11206656~196608 [fadvise_sequential+fadvise_nocache],write 11419648~196608 [fadvise_sequential+fadvise_nocache],write 11632640~65536 [fadvise_sequential+fadvise_nocache],write 11730944~98304 [fadvise_sequential+fadvise_nocache],write 11845632~65536 [fadvise_sequential+fadvise_nocache],write 11943936~98304 [fadvise_sequential+fadvise_nocache],write 12058624~65536 [fadvise_sequential+fadvise_nocache],write 12140544~65536 [fadvise_sequential+fadvise_nocache],write 12238848~98304 [fadvise_sequential+fadvise_nocache],write 12353536~65536 [fadvise_sequential+fadvise_nocache],write 12451840~98304 [fadvise_sequential+fadvise_nocache],write 12566528~65536 [fadvise_sequential+fadvise_nocache],write 12746752~98304 [fadvise_sequential+fadvise_nocache],write 12959744~98304 [fadvise_sequential+fadvise_nocache],write 13107200~32768 [fadvise_sequential+fadvise_nocache],write 13254656~98304 [fadvise_sequential+fadvise_nocache],write 13467648~98304 [fadvise_sequential+fadvise_nocache],write 13631488~16384 [fadvise_sequential+fadvise_nocache],write 13762560~98304 [fadvise_sequential+fadvise_nocache],write 13975552~98304 [fadvise_sequential+fadvise_nocache],write 15204352~49152 [fadvise_sequential+fadvise_nocache],write 15384576~81920 [fadvise_sequential+fadvise_nocache],write 15597568~81920 [fadvise_sequential+fadvise_nocache],write 15728640~32768 [fadvise_sequential+fadvise_nocache],write 15892480~81920 [fadvise_sequential+fadvise_nocache],write 16105472~81920 [fadvise_sequential+fadvise_nocache],write 16252928~16384 [fadvise_sequential+fadvise_nocache],write 16400384~81920 [fadvise_sequential+fadvise_nocache],write 16613376~81920 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:35.203 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.300719+0000", 2024-09-21T15:00:35.203 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 29.889669441999999, 2024-09-21T15:00:35.203 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.068947114000000004, 2024-09-21T15:00:35.203 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:35.203 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:35.203 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:35.203 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:35.203 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:35.203 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 97673 2024-09-21T15:00:35.203 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.203 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:35.204 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.204 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:35.204 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.300719+0000", 2024-09-21T15:00:35.204 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:35.204 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.204 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.204 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:35.204 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.300719+0000", 2024-09-21T15:00:35.204 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:35.204 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.204 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.204 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:35.204 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.300716+0000", 2024-09-21T15:00:35.204 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-21T15:00:35.204 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.205 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.205 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:35.205 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.308380+0000", 2024-09-21T15:00:35.205 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0076638519999999996 2024-09-21T15:00:35.205 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.205 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.205 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:35.205 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.308418+0000", 2024-09-21T15:00:35.205 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.8380000000000002e-05 2024-09-21T15:00:35.205 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.205 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.205 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:35.205 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.308431+0000", 2024-09-21T15:00:35.205 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3213999999999999e-05 2024-09-21T15:00:35.205 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.206 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.206 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:35.206 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.308562+0000", 2024-09-21T15:00:35.206 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013015999999999999 2024-09-21T15:00:35.206 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.206 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.206 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:35.206 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.309173+0000", 2024-09-21T15:00:35.206 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00061185500000000002 2024-09-21T15:00:35.206 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.206 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.206 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 0", 2024-09-21T15:00:35.206 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311612+0000", 2024-09-21T15:00:35.206 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0024383949999999999 2024-09-21T15:00:35.206 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.207 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.207 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:35.207 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.355482+0000", 2024-09-21T15:00:35.207 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.043869940000000003 2024-09-21T15:00:35.207 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.207 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.207 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:35.207 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.369306+0000", 2024-09-21T15:00:35.207 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013824279 2024-09-21T15:00:35.207 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.207 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.207 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:35.207 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.369575+0000", 2024-09-21T15:00:35.207 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000269404 2024-09-21T15:00:35.207 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.207 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.208 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:35.208 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.369666+0000", 2024-09-21T15:00:35.208 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0426000000000005e-05 2024-09-21T15:00:35.208 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:35.208 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:35.208 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:35.208 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.208 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.208 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:112719 3.13 3:cd56ff5a:::rbd_data.1136b1194d20.0000000000000002:58 [read 1998848~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:35.208 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.387056+0000", 2024-09-21T15:00:35.208 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 5.8033320960000001, 2024-09-21T15:00:35.208 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055608623000000003, 2024-09-21T15:00:35.208 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:35.208 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:35.208 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:35.208 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:35.209 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:35.209 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 112719 2024-09-21T15:00:35.209 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.209 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:35.209 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.209 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:35.209 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.387056+0000", 2024-09-21T15:00:35.209 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:35.209 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.209 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.209 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:35.209 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.387056+0000", 2024-09-21T15:00:35.209 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:35.209 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.209 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.209 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:35.210 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.387056+0000", 2024-09-21T15:00:35.210 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:35.210 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.210 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.210 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:35.210 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.387058+0000", 2024-09-21T15:00:35.210 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.683e-06 2024-09-21T15:00:35.210 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.210 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.210 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:35.210 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.387064+0000", 2024-09-21T15:00:35.210 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6720000000000001e-06 2024-09-21T15:00:35.210 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.210 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.210 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:35.211 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.387069+0000", 2024-09-21T15:00:35.211 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8999999999999997e-06 2024-09-21T15:00:35.211 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.211 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.211 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:35.211 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.442514+0000", 2024-09-21T15:00:35.211 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055444881000000001 2024-09-21T15:00:35.211 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.211 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.211 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:35.211 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.442565+0000", 2024-09-21T15:00:35.211 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1010000000000001e-05 2024-09-21T15:00:35.211 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.211 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.212 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:35.212 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.442665+0000", 2024-09-21T15:00:35.212 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9840000000000006e-05 2024-09-21T15:00:35.212 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:35.212 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:35.212 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:35.212 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.212 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.212 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:112720 3.13 3:cd56ff5a:::rbd_data.1136b1194d20.0000000000000002:58 [read 2088960~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:35.212 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.387074+0000", 2024-09-21T15:00:35.212 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 5.8033145810000004, 2024-09-21T15:00:35.212 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055824591999999999, 2024-09-21T15:00:35.212 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:35.212 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:35.212 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:35.213 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:35.213 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:35.213 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 112720 2024-09-21T15:00:35.213 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.213 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:35.213 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.213 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:35.213 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.387074+0000", 2024-09-21T15:00:35.213 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:35.213 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.213 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.213 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:35.213 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.387074+0000", 2024-09-21T15:00:35.213 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:35.213 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.213 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.214 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:35.214 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.387073+0000", 2024-09-21T15:00:35.214 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:35.214 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.214 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.214 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:35.214 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.387076+0000", 2024-09-21T15:00:35.214 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4710000000000001e-06 2024-09-21T15:00:35.214 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.214 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.214 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:35.214 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.387085+0000", 2024-09-21T15:00:35.214 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2110000000000004e-06 2024-09-21T15:00:35.214 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.215 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.215 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:35.215 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.387090+0000", 2024-09-21T15:00:35.215 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.904e-06 2024-09-21T15:00:35.215 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.215 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.215 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:35.215 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.442702+0000", 2024-09-21T15:00:35.215 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055612251000000001 2024-09-21T15:00:35.215 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.215 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.215 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:35.215 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.442780+0000", 2024-09-21T15:00:35.215 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8232999999999994e-05 2024-09-21T15:00:35.215 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.215 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.216 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:35.216 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.442898+0000", 2024-09-21T15:00:35.216 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000117889 2024-09-21T15:00:35.216 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:35.216 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:35.216 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:35.216 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.216 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.216 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:112721 3.13 3:cd56ff5a:::rbd_data.1136b1194d20.0000000000000002:58 [read 2105344~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:35.216 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.387095+0000", 2024-09-21T15:00:35.216 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 5.8032934909999998, 2024-09-21T15:00:35.216 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.056000182000000003, 2024-09-21T15:00:35.216 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:35.217 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:35.217 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:35.217 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:35.217 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:35.217 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 112721 2024-09-21T15:00:35.217 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.217 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:35.217 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.217 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:35.217 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.387095+0000", 2024-09-21T15:00:35.217 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:35.217 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.217 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.217 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:35.217 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.387095+0000", 2024-09-21T15:00:35.217 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:35.217 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.218 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.218 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:35.218 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.387094+0000", 2024-09-21T15:00:35.218 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:35.218 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.218 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.218 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:35.218 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.387097+0000", 2024-09-21T15:00:35.218 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5940000000000001e-06 2024-09-21T15:00:35.218 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.218 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.218 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:35.218 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.387106+0000", 2024-09-21T15:00:35.218 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.9439999999999994e-06 2024-09-21T15:00:35.219 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.219 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.219 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:35.219 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.387111+0000", 2024-09-21T15:00:35.219 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3900000000000001e-06 2024-09-21T15:00:35.219 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.219 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.219 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:35.219 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.442934+0000", 2024-09-21T15:00:35.219 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055822996 2024-09-21T15:00:35.219 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.219 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.219 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:35.219 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.442998+0000", 2024-09-21T15:00:35.219 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3746999999999998e-05 2024-09-21T15:00:35.219 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.220 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.220 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:35.220 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.443095+0000", 2024-09-21T15:00:35.220 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6848999999999994e-05 2024-09-21T15:00:35.220 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:35.220 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:35.220 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:35.220 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.220 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.220 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:112733 3.13 3:cd56ff5a:::rbd_data.1136b1194d20.0000000000000002:58 [read 3186688~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:35.220 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.390744+0000", 2024-09-21T15:00:35.220 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 5.7996443129999999, 2024-09-21T15:00:35.220 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055498697999999999, 2024-09-21T15:00:35.220 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:35.220 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:35.221 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:35.221 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:35.221 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:35.221 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 112733 2024-09-21T15:00:35.221 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.221 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:35.221 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.221 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:35.221 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390744+0000", 2024-09-21T15:00:35.221 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:35.221 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.221 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.221 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:35.221 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390744+0000", 2024-09-21T15:00:35.221 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:35.221 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.222 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.222 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:35.222 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390743+0000", 2024-09-21T15:00:35.222 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:35.222 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.222 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.222 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:35.222 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390746+0000", 2024-09-21T15:00:35.222 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9510000000000002e-06 2024-09-21T15:00:35.222 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.222 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.222 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:35.223 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390751+0000", 2024-09-21T15:00:35.223 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.357e-06 2024-09-21T15:00:35.223 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.223 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.223 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:35.223 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390753+0000", 2024-09-21T15:00:35.223 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7310000000000002e-06 2024-09-21T15:00:35.223 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.223 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.223 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:35.223 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.445994+0000", 2024-09-21T15:00:35.223 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055240247999999999 2024-09-21T15:00:35.223 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.223 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.224 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:35.224 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.446111+0000", 2024-09-21T15:00:35.224 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00011699799999999999 2024-09-21T15:00:35.224 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.224 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.224 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:35.224 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.446243+0000", 2024-09-21T15:00:35.224 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013196099999999999 2024-09-21T15:00:35.224 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:35.224 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:35.224 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:35.224 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.224 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.224 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:112734 3.13 3:cd56ff5a:::rbd_data.1136b1194d20.0000000000000002:58 [read 3276800~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:35.224 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.390755+0000", 2024-09-21T15:00:35.225 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 5.7996326900000001, 2024-09-21T15:00:35.225 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.05569313, 2024-09-21T15:00:35.225 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:35.225 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:35.225 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:35.225 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:35.225 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:35.225 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 112734 2024-09-21T15:00:35.225 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.225 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:35.225 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.225 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:35.225 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390755+0000", 2024-09-21T15:00:35.225 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:35.226 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.226 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.226 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:35.226 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390755+0000", 2024-09-21T15:00:35.226 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:35.226 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.226 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.226 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:35.226 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390755+0000", 2024-09-21T15:00:35.226 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:00:35.226 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.226 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.226 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:35.226 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390757+0000", 2024-09-21T15:00:35.227 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5060000000000001e-06 2024-09-21T15:00:35.227 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.227 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.227 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:35.227 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390760+0000", 2024-09-21T15:00:35.227 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.79e-06 2024-09-21T15:00:35.227 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.227 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.227 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:35.227 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390762+0000", 2024-09-21T15:00:35.227 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.661e-06 2024-09-21T15:00:35.227 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.227 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.227 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:35.228 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.446280+0000", 2024-09-21T15:00:35.228 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055517761999999998 2024-09-21T15:00:35.228 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.228 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.228 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:35.228 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.446365+0000", 2024-09-21T15:00:35.228 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5208000000000001e-05 2024-09-21T15:00:35.228 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.228 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.228 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:35.228 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.446449+0000", 2024-09-21T15:00:35.228 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3417000000000004e-05 2024-09-21T15:00:35.228 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:35.228 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:35.228 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:35.229 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.229 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.229 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:112735 3.13 3:cd56ff5a:::rbd_data.1136b1194d20.0000000000000002:58 [read 3366912~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:35.229 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.390764+0000", 2024-09-21T15:00:35.229 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 5.7996238800000004, 2024-09-21T15:00:35.229 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055846502999999999, 2024-09-21T15:00:35.229 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:35.229 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:35.229 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:35.229 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:35.229 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:35.229 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 112735 2024-09-21T15:00:35.229 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.229 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:35.229 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.229 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:35.230 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390764+0000", 2024-09-21T15:00:35.230 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:35.230 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.230 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.230 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:35.230 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390764+0000", 2024-09-21T15:00:35.230 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:35.230 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.230 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.230 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:35.230 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390764+0000", 2024-09-21T15:00:35.230 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:00:35.230 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.230 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.230 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:35.231 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390771+0000", 2024-09-21T15:00:35.231 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.1380000000000001e-06 2024-09-21T15:00:35.231 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.231 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.231 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:35.231 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390774+0000", 2024-09-21T15:00:35.231 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1370000000000002e-06 2024-09-21T15:00:35.231 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.231 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.231 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:35.231 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390777+0000", 2024-09-21T15:00:35.231 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7470000000000001e-06 2024-09-21T15:00:35.231 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.231 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.231 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:35.231 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.446488+0000", 2024-09-21T15:00:35.232 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055711016000000002 2024-09-21T15:00:35.232 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.232 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.232 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:35.232 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.446543+0000", 2024-09-21T15:00:35.232 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5148999999999997e-05 2024-09-21T15:00:35.232 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.232 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.232 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:35.232 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.446611+0000", 2024-09-21T15:00:35.232 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7533999999999994e-05 2024-09-21T15:00:35.232 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:35.232 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:35.232 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:35.232 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.233 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.233 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:112736 3.13 3:cd56ff5a:::rbd_data.1136b1194d20.0000000000000002:58 [read 3457024~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:35.233 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.390779+0000", 2024-09-21T15:00:35.233 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 5.7996089360000003, 2024-09-21T15:00:35.233 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.056179989, 2024-09-21T15:00:35.233 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:35.233 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:35.233 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:35.233 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:35.233 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:35.233 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 112736 2024-09-21T15:00:35.233 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.233 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:35.233 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.233 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:35.233 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390779+0000", 2024-09-21T15:00:35.234 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:35.234 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.234 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.234 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:35.234 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390779+0000", 2024-09-21T15:00:35.234 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:35.234 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.234 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.234 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:35.234 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390779+0000", 2024-09-21T15:00:35.234 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:00:35.234 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.234 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.234 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:35.234 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390780+0000", 2024-09-21T15:00:35.234 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4360000000000001e-06 2024-09-21T15:00:35.235 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.235 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.235 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:35.235 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390783+0000", 2024-09-21T15:00:35.235 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.818e-06 2024-09-21T15:00:35.235 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.235 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.235 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:35.235 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390786+0000", 2024-09-21T15:00:35.235 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4439999999999998e-06 2024-09-21T15:00:35.235 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.235 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.235 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:35.235 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.446668+0000", 2024-09-21T15:00:35.235 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055881910999999999 2024-09-21T15:00:35.236 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.236 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.236 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:35.236 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.446797+0000", 2024-09-21T15:00:35.236 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012924599999999999 2024-09-21T15:00:35.236 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.236 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.236 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:35.236 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.446959+0000", 2024-09-21T15:00:35.236 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000162356 2024-09-21T15:00:35.236 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:35.236 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:35.236 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:35.236 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.236 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.236 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:118215 3.d 3:b5b5db2f:::rbd_data.1136b1194d20.0000000000000005:58 [read 5160960~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:35.236 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.833406+0000", 2024-09-21T15:00:35.237 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 5.3569826379999999, 2024-09-21T15:00:35.237 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055458362999999997, 2024-09-21T15:00:35.237 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:35.237 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:35.237 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:35.237 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:35.237 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:35.237 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 118215 2024-09-21T15:00:35.237 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.237 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:35.237 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.237 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:35.237 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833406+0000", 2024-09-21T15:00:35.237 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:35.237 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.238 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.238 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:35.238 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833406+0000", 2024-09-21T15:00:35.238 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:35.238 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.238 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.238 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:35.238 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833405+0000", 2024-09-21T15:00:35.238 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:35.238 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.238 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.238 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:35.238 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833409+0000", 2024-09-21T15:00:35.239 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.8099999999999999e-06 2024-09-21T15:00:35.239 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.239 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.239 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:35.239 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833416+0000", 2024-09-21T15:00:35.239 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3989999999999998e-06 2024-09-21T15:00:35.239 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.239 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.239 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:35.239 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833420+0000", 2024-09-21T15:00:35.239 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7189999999999999e-06 2024-09-21T15:00:35.239 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.239 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.239 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:35.239 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.888444+0000", 2024-09-21T15:00:35.239 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055024311999999999 2024-09-21T15:00:35.240 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.240 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.240 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:35.240 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.888630+0000", 2024-09-21T15:00:35.240 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00018579800000000001 2024-09-21T15:00:35.240 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.240 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.240 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:35.240 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.888864+0000", 2024-09-21T15:00:35.240 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000234009 2024-09-21T15:00:35.240 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:35.240 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:35.240 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:35.240 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.240 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.240 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:118216 3.d 3:b5b5db2f:::rbd_data.1136b1194d20.0000000000000005:58 [read 5251072~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:35.241 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.833425+0000", 2024-09-21T15:00:35.241 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 5.3569632059999996, 2024-09-21T15:00:35.241 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055891158000000003, 2024-09-21T15:00:35.241 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:35.241 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:35.241 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:35.241 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:35.241 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:35.241 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 118216 2024-09-21T15:00:35.241 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.241 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:35.241 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.241 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:35.241 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833425+0000", 2024-09-21T15:00:35.241 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:35.242 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.242 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.242 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:35.242 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833425+0000", 2024-09-21T15:00:35.242 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:35.242 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.242 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.242 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:35.242 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833425+0000", 2024-09-21T15:00:35.242 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:35.242 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.242 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.242 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:35.242 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833427+0000", 2024-09-21T15:00:35.243 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.137e-06 2024-09-21T15:00:35.243 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.243 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.243 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:35.243 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833430+0000", 2024-09-21T15:00:35.243 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5379999999999998e-06 2024-09-21T15:00:35.243 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.243 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.243 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:35.243 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833433+0000", 2024-09-21T15:00:35.243 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0599999999999999e-06 2024-09-21T15:00:35.243 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.243 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.243 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:35.243 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.888919+0000", 2024-09-21T15:00:35.244 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055485381 2024-09-21T15:00:35.244 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.244 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.244 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:35.244 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.889088+0000", 2024-09-21T15:00:35.244 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000169364 2024-09-21T15:00:35.244 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.244 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.244 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:35.244 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.889316+0000", 2024-09-21T15:00:35.244 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00022798600000000001 2024-09-21T15:00:35.244 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:35.244 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:35.244 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:35.244 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.245 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.245 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:118217 3.d 3:b5b5db2f:::rbd_data.1136b1194d20.0000000000000005:58 [read 5341184~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:35.245 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.833436+0000", 2024-09-21T15:00:35.245 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 5.3569517790000001, 2024-09-21T15:00:35.245 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.056045230000000001, 2024-09-21T15:00:35.245 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:35.245 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:35.245 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:35.245 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:35.245 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:35.245 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 118217 2024-09-21T15:00:35.245 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.245 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:35.245 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.245 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:35.245 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833436+0000", 2024-09-21T15:00:35.246 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:35.246 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.246 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.246 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:35.246 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833436+0000", 2024-09-21T15:00:35.246 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:35.246 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.246 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.246 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:35.246 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833436+0000", 2024-09-21T15:00:35.246 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:35.246 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.246 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.246 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:35.246 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833438+0000", 2024-09-21T15:00:35.247 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.826e-06 2024-09-21T15:00:35.247 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.247 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.247 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:35.247 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833441+0000", 2024-09-21T15:00:35.247 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3280000000000002e-06 2024-09-21T15:00:35.247 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.247 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.247 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:35.247 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833447+0000", 2024-09-21T15:00:35.247 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2660000000000001e-06 2024-09-21T15:00:35.247 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.247 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.247 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:35.247 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.889362+0000", 2024-09-21T15:00:35.248 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055915323000000003 2024-09-21T15:00:35.248 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.248 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.248 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:35.248 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.889414+0000", 2024-09-21T15:00:35.248 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1711e-05 2024-09-21T15:00:35.248 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.248 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.248 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:35.248 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.889482+0000", 2024-09-21T15:00:35.248 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8056e-05 2024-09-21T15:00:35.248 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:35.248 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:35.248 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:35.248 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.248 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.249 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:118218 3.d 3:b5b5db2f:::rbd_data.1136b1194d20.0000000000000005:58 [read 5431296~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:35.249 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.833451+0000", 2024-09-21T15:00:35.249 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 5.3569367440000004, 2024-09-21T15:00:35.249 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.056286765000000002, 2024-09-21T15:00:35.249 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:35.249 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:35.249 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:35.249 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:35.249 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:35.249 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 118218 2024-09-21T15:00:35.249 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.249 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:35.249 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.249 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:35.249 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833451+0000", 2024-09-21T15:00:35.250 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:35.250 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.250 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.250 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:35.250 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833451+0000", 2024-09-21T15:00:35.250 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:35.250 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.250 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.250 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:35.250 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833451+0000", 2024-09-21T15:00:35.250 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:35.250 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.250 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.250 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:35.251 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833454+0000", 2024-09-21T15:00:35.251 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9670000000000001e-06 2024-09-21T15:00:35.251 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.251 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.251 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:35.251 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833461+0000", 2024-09-21T15:00:35.251 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0489999999999998e-06 2024-09-21T15:00:35.251 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.251 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.251 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:35.251 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833468+0000", 2024-09-21T15:00:35.251 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0310000000000001e-06 2024-09-21T15:00:35.251 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.251 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.251 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:35.251 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.889539+0000", 2024-09-21T15:00:35.252 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.056071217 2024-09-21T15:00:35.252 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.252 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.252 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:35.252 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.889621+0000", 2024-09-21T15:00:35.252 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1528999999999998e-05 2024-09-21T15:00:35.252 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.252 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.252 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:35.252 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.889738+0000", 2024-09-21T15:00:35.252 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00011759799999999999 2024-09-21T15:00:35.252 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:35.252 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:35.252 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:35.252 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.252 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.253 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:118220 3.d 3:b5b5db2f:::rbd_data.1136b1194d20.0000000000000005:58 [read 5611520~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:35.253 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.834528+0000", 2024-09-21T15:00:35.253 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 5.355859755, 2024-09-21T15:00:35.253 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055660376999999997, 2024-09-21T15:00:35.253 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:35.253 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:35.253 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:35.253 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:35.253 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:35.253 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 118220 2024-09-21T15:00:35.253 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.253 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:35.253 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.253 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:35.253 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834528+0000", 2024-09-21T15:00:35.254 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:35.254 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.254 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.254 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:35.254 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834528+0000", 2024-09-21T15:00:35.254 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:35.254 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.254 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.254 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:35.254 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834528+0000", 2024-09-21T15:00:35.254 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:35.254 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.254 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.254 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:35.254 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834532+0000", 2024-09-21T15:00:35.255 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.9509999999999999e-06 2024-09-21T15:00:35.255 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.255 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.255 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:35.255 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834537+0000", 2024-09-21T15:00:35.255 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6899999999999997e-06 2024-09-21T15:00:35.255 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.255 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.255 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:35.255 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834541+0000", 2024-09-21T15:00:35.255 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.76e-06 2024-09-21T15:00:35.255 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.255 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.255 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:35.255 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.889962+0000", 2024-09-21T15:00:35.255 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055421249999999998 2024-09-21T15:00:35.256 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.256 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.256 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:35.256 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.890074+0000", 2024-09-21T15:00:35.256 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00011124300000000001 2024-09-21T15:00:35.256 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.256 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.256 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:35.256 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.890189+0000", 2024-09-21T15:00:35.256 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000115151 2024-09-21T15:00:35.256 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:35.256 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:35.256 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:35.256 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.256 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.256 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:118221 3.d 3:b5b5db2f:::rbd_data.1136b1194d20.0000000000000005:58 [read 5701632~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:35.257 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.834544+0000", 2024-09-21T15:00:35.257 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 5.3558438329999998, 2024-09-21T15:00:35.257 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.056071875, 2024-09-21T15:00:35.257 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:35.257 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:35.257 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:35.257 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:35.257 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:35.257 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 118221 2024-09-21T15:00:35.257 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.257 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:35.257 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.257 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:35.257 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834544+0000", 2024-09-21T15:00:35.257 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:35.257 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.258 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.258 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:35.258 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834544+0000", 2024-09-21T15:00:35.258 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:35.258 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.258 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.258 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:35.258 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834544+0000", 2024-09-21T15:00:35.258 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:35.258 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.258 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.258 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:35.258 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834546+0000", 2024-09-21T15:00:35.258 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.458e-06 2024-09-21T15:00:35.258 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.259 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.259 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:35.259 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834550+0000", 2024-09-21T15:00:35.259 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.563e-06 2024-09-21T15:00:35.259 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.259 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.259 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:35.259 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834553+0000", 2024-09-21T15:00:35.259 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.253e-06 2024-09-21T15:00:35.259 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.259 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.259 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:35.259 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.890246+0000", 2024-09-21T15:00:35.259 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055692561000000002 2024-09-21T15:00:35.259 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.259 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.259 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:35.260 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.890397+0000", 2024-09-21T15:00:35.260 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015129999999999999 2024-09-21T15:00:35.260 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.260 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.260 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:35.260 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.890616+0000", 2024-09-21T15:00:35.260 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00021912799999999999 2024-09-21T15:00:35.260 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:35.260 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:35.260 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:35.260 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.260 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.260 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:118222 3.d 3:b5b5db2f:::rbd_data.1136b1194d20.0000000000000005:58 [read 5791744~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:35.260 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.834557+0000", 2024-09-21T15:00:35.260 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 5.3558310220000003, 2024-09-21T15:00:35.260 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.056417907000000003, 2024-09-21T15:00:35.261 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:35.261 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:35.261 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:35.261 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:35.261 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:35.261 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 118222 2024-09-21T15:00:35.261 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.261 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:35.261 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.261 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:35.261 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834557+0000", 2024-09-21T15:00:35.261 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:35.261 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.261 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.261 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:35.261 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834557+0000", 2024-09-21T15:00:35.262 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:35.262 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.262 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.262 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:35.262 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834557+0000", 2024-09-21T15:00:35.262 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:35.262 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.262 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.262 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:35.262 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834559+0000", 2024-09-21T15:00:35.262 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.956e-06 2024-09-21T15:00:35.262 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.262 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.262 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:35.262 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834562+0000", 2024-09-21T15:00:35.263 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.518e-06 2024-09-21T15:00:35.263 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.263 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.263 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:35.263 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834566+0000", 2024-09-21T15:00:35.263 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.529e-06 2024-09-21T15:00:35.263 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.263 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.263 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:35.263 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.890671+0000", 2024-09-21T15:00:35.263 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.056105255 2024-09-21T15:00:35.263 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.263 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.263 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:35.263 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.890788+0000", 2024-09-21T15:00:35.264 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00011715899999999999 2024-09-21T15:00:35.264 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.264 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.264 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:35.264 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.890975+0000", 2024-09-21T15:00:35.264 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00018678100000000001 2024-09-21T15:00:35.264 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:35.264 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:35.264 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:35.264 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.264 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.264 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:118223 3.d 3:b5b5db2f:::rbd_data.1136b1194d20.0000000000000005:58 [read 5881856~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:35.264 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.834569+0000", 2024-09-21T15:00:35.264 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 5.3558194649999997, 2024-09-21T15:00:35.265 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.056618784999999998, 2024-09-21T15:00:35.265 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:35.265 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:35.265 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:35.265 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:35.265 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:35.265 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 118223 2024-09-21T15:00:35.265 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.265 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:35.265 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.265 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:35.265 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834569+0000", 2024-09-21T15:00:35.265 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:35.265 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.265 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.265 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:35.266 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834569+0000", 2024-09-21T15:00:35.266 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:35.266 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.266 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.266 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:35.266 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834568+0000", 2024-09-21T15:00:35.266 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:35.266 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.266 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.266 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:35.266 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834570+0000", 2024-09-21T15:00:35.266 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8309999999999999e-06 2024-09-21T15:00:35.266 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.266 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.266 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:35.267 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834574+0000", 2024-09-21T15:00:35.267 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3720000000000001e-06 2024-09-21T15:00:35.267 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.267 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.267 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:35.267 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834577+0000", 2024-09-21T15:00:35.267 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3019999999999999e-06 2024-09-21T15:00:35.267 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.267 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.267 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:35.267 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.891014+0000", 2024-09-21T15:00:35.267 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.056437322999999998 2024-09-21T15:00:35.267 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.267 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.267 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:35.267 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.891101+0000", 2024-09-21T15:00:35.268 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6341999999999994e-05 2024-09-21T15:00:35.268 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.268 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.268 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:35.268 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.891187+0000", 2024-09-21T15:00:35.268 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6889000000000001e-05 2024-09-21T15:00:35.268 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:35.268 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:35.268 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:35.268 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.268 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.268 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:118224 3.d 3:b5b5db2f:::rbd_data.1136b1194d20.0000000000000005:58 [read 5971968~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:35.268 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.834580+0000", 2024-09-21T15:00:35.268 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 5.355808498, 2024-09-21T15:00:35.268 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.056770289000000002, 2024-09-21T15:00:35.268 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:35.269 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:35.269 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:35.269 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:35.269 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:35.269 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 118224 2024-09-21T15:00:35.269 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.269 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:35.269 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.269 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:35.269 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834580+0000", 2024-09-21T15:00:35.269 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:35.269 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.269 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.269 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:35.269 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834580+0000", 2024-09-21T15:00:35.269 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:35.270 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.270 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.270 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:35.270 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834579+0000", 2024-09-21T15:00:35.270 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:35.270 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.270 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.270 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:35.270 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834581+0000", 2024-09-21T15:00:35.270 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8160000000000001e-06 2024-09-21T15:00:35.270 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.270 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.270 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:35.270 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834585+0000", 2024-09-21T15:00:35.271 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.568e-06 2024-09-21T15:00:35.271 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.271 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.271 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:35.271 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834588+0000", 2024-09-21T15:00:35.271 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1810000000000001e-06 2024-09-21T15:00:35.271 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.271 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.271 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:35.271 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.891216+0000", 2024-09-21T15:00:35.271 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.056628127 2024-09-21T15:00:35.271 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.271 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.271 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:35.271 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.891273+0000", 2024-09-21T15:00:35.272 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7364000000000002e-05 2024-09-21T15:00:35.272 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.272 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.272 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:35.272 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.891350+0000", 2024-09-21T15:00:35.272 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6509999999999998e-05 2024-09-21T15:00:35.272 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:35.272 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:35.272 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:35.272 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.272 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.272 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:118225 3.d 3:b5b5db2f:::rbd_data.1136b1194d20.0000000000000005:58 [read 6062080~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:35.272 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.834591+0000", 2024-09-21T15:00:35.272 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 5.3557974870000002, 2024-09-21T15:00:35.272 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.056883207999999998, 2024-09-21T15:00:35.272 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:35.273 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:35.273 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:35.273 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:35.273 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:35.273 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 118225 2024-09-21T15:00:35.273 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.273 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:35.273 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.273 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:35.273 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834591+0000", 2024-09-21T15:00:35.273 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:35.273 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.273 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.273 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:35.273 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834591+0000", 2024-09-21T15:00:35.274 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:35.274 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.274 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.274 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:35.274 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834590+0000", 2024-09-21T15:00:35.274 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:35.274 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.274 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.274 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:35.274 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834592+0000", 2024-09-21T15:00:35.274 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8810000000000001e-06 2024-09-21T15:00:35.274 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.274 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.274 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:35.274 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834596+0000", 2024-09-21T15:00:35.275 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.439e-06 2024-09-21T15:00:35.275 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.275 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.275 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:35.275 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834599+0000", 2024-09-21T15:00:35.275 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0740000000000001e-06 2024-09-21T15:00:35.275 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.275 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.275 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:35.275 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.891369+0000", 2024-09-21T15:00:35.275 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.056769872999999998 2024-09-21T15:00:35.275 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.275 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.275 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:35.275 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.891415+0000", 2024-09-21T15:00:35.276 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6257000000000001e-05 2024-09-21T15:00:35.276 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:35.276 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:35.276 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:35.276 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.891474+0000", 2024-09-21T15:00:35.276 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8964e-05 2024-09-21T15:00:35.276 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:35.276 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:35.276 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:35.276 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:35.276 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:35.276 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:35.277 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T15:00:35.321 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:00:35.319+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:00:35.333 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 5, 6, 7, 4] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T15:00:35.333 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T15:00:35.334 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-21T15:00:35.334 INFO:tasks.thrashosds.thrasher:Setting osd 2 primary_affinity to 1.000000 2024-09-21T15:00:35.334 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-09-21T15:00:35.352 INFO:teuthology.orchestra.run.smithi032.stderr:noscrub is unset 2024-09-21T15:00:35.381 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:00:35.379+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:00:35.401 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:35.401 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T15:00:35.401 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.401 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:123150 3.4 3:217cf3de:::rbd_object_map.113625862118:head [call rbd.object_map_resize in=29b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e149)", 2024-09-21T15:00:35.401 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:35.398384+0000", 2024-09-21T15:00:35.401 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.0017180590000000001, 2024-09-21T15:00:35.401 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0017955040000000001, 2024-09-21T15:00:35.401 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:35.401 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "waiting for sub ops", 2024-09-21T15:00:35.402 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:35.402 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:35.402 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:35.402 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 123150 2024-09-21T15:00:35.402 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.402 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:35.402 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.402 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:35.402 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:35.398384+0000", 2024-09-21T15:00:35.402 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:35.402 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.402 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.402 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:35.402 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:35.398384+0000", 2024-09-21T15:00:35.403 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:35.403 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.403 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.403 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:35.403 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:35.398382+0000", 2024-09-21T15:00:35.403 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:00:35.403 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.403 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.403 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:35.403 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:35.398391+0000", 2024-09-21T15:00:35.403 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4919999999999999e-06 2024-09-21T15:00:35.403 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.403 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.403 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:35.403 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:35.398411+0000", 2024-09-21T15:00:35.404 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9310999999999999e-05 2024-09-21T15:00:35.404 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.404 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.404 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:35.404 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:35.398420+0000", 2024-09-21T15:00:35.404 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3789999999999995e-06 2024-09-21T15:00:35.404 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.404 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.404 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:35.404 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:35.398550+0000", 2024-09-21T15:00:35.404 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000129677 2024-09-21T15:00:35.404 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.404 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.404 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:35.404 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:35.398824+0000", 2024-09-21T15:00:35.405 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00027411699999999999 2024-09-21T15:00:35.405 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.405 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.405 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 2", 2024-09-21T15:00:35.405 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:35.399154+0000", 2024-09-21T15:00:35.405 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00032969899999999998 2024-09-21T15:00:35.405 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:35.405 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:35.405 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:35.405 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:35.405 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-21T15:00:35.405 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 1 2024-09-21T15:00:35.405 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:35.412 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T15:00:35.443 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T15:00:35.493 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:00:35.491+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:00:35.517 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:35.517 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:00:35.517 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T15:00:35.517 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T15:00:35.517 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:35.527 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T15:00:35.582 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:00:35.582+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:00:35.604 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T15:00:35.625 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T15:00:35.642 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:35.642 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T15:00:35.642 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T15:00:35.642 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T15:00:35.642 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.642 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:63820 3.19 3.626bc079 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-21T15:00:35.642 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:27.241293+0000", 2024-09-21T15:00:35.642 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 68.396646302999997, 2024-09-21T15:00:35.642 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.98723808199999996, 2024-09-21T15:00:35.643 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:35.643 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:35.643 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:35.643 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:35.643 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:35.643 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 63820 2024-09-21T15:00:35.643 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.643 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:35.643 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.643 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:35.643 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241293+0000", 2024-09-21T15:00:35.643 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:35.643 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.643 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.644 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:35.644 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241293+0000", 2024-09-21T15:00:35.644 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:35.644 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.644 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.644 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:35.644 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241291+0000", 2024-09-21T15:00:35.644 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:00:35.644 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.644 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.644 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:35.644 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241299+0000", 2024-09-21T15:00:35.644 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1529999999999997e-06 2024-09-21T15:00:35.644 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.644 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.645 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:35.645 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241317+0000", 2024-09-21T15:00:35.645 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7764000000000001e-05 2024-09-21T15:00:35.645 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.645 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.645 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:35.645 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241328+0000", 2024-09-21T15:00:35.645 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1245e-05 2024-09-21T15:00:35.645 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.645 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.645 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:35.645 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241407+0000", 2024-09-21T15:00:35.645 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8762000000000002e-05 2024-09-21T15:00:35.645 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.645 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.645 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:00:35.646 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241418+0000", 2024-09-21T15:00:35.646 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1695000000000001e-05 2024-09-21T15:00:35.646 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.646 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.646 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:35.646 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:28.228510+0000", 2024-09-21T15:00:35.646 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.98709166100000001 2024-09-21T15:00:35.646 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.646 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.646 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:35.646 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:28.228531+0000", 2024-09-21T15:00:35.646 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0696e-05 2024-09-21T15:00:35.646 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:35.646 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:35.646 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:35.647 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.647 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.647 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:97734 3.19 3:9fcf754f:::rbd_data.1136bfc79a70.0000000000000009:head [write 0~4194304 [fadvise_sequential+fadvise_nocache],write 4784128~147456 [fadvise_sequential+fadvise_nocache],write 4997120~147456 [fadvise_sequential+fadvise_nocache],write 5210112~32768 [fadvise_sequential+fadvise_nocache],write 5292032~147456 [fadvise_sequential+fadvise_nocache],write 5505024~147456 [fadvise_sequential+fadvise_nocache],write 5718016~49152 [fadvise_sequential+fadvise_nocache],write 5799936~147456 [fadvise_sequential+fadvise_nocache],write 6012928~147456 [fadvise_sequential+fadvise_nocache],write 6225920~65536 [fadvise_sequential+fadvise_nocache],write 6307840~147456 [fadvise_sequential+fadvise_nocache],write 6520832~147456 [fadvise_sequential+fadvise_nocache],write 6733824~229376 [fadvise_sequential+fadvise_nocache],write 7028736~147456 [fadvise_sequential+fadvise_nocache],write 7241728~229376 [fadvise_sequential+fadvise_nocache],write 7536640~147456 [fadvise_sequential+fadvise_nocache],write 7749632~229376 [fadvise_sequential+fadvise_nocache],write 8077312~114688 [fadvise_sequential+fadvise_nocache],write 8290304~98304 [fadvise_sequential+fadvise_nocache],write 8404992~81920 [fadvise_sequential+fadvise_nocache],write 8617984~81920 [fadvise_sequential+fadvise_nocache],write 8830976~81920 [fadvise_sequential+fadvise_nocache],write 8962048~32768 [fadvise_sequential+fadvise_nocache],write 9175040~32768 [fadvise_sequential+fadvise_nocache],write 9388032~32768 [fadvise_sequential+fadvise_nocache],write 9469952~32768 [fadvise_sequential+fadvise_nocache],write 9682944~20092 [fadvise_sequential+fadvise_nocache],write 9961472~163840 [fadvise_sequential+fadvise_nocache],write 10141696~196608 [fadvise_sequential+fadvise_nocache],write 10354688~278528 [fadvise_sequential+fadvise_nocache],write 10649600~196608 [fadvise_sequential+fadvise_nocache],write 10862592~196608 [fadvise_sequential+fadvise_nocache],write 11075584~65536 [fadvise_sequential+fadvise_nocache],write 11173888~98304 [fadvise_sequential+fadvise_nocache],write 11288576~65536 [fadvise_sequential+fadvise_nocache],write 11386880~98304 [fadvise_sequential+fadvise_nocache],write 11501568~65536 [fadvise_sequential+fadvise_nocache],write 11583488~65536 [fadvise_sequential+fadvise_nocache],write 11681792~98304 [fadvise_sequential+fadvise_nocache],write 11796480~65536 [fadvise_sequential+fadvise_nocache],write 11894784~98304 [fadvise_sequential+fadvise_nocache],write 12009472~65536 [fadvise_sequential+fadvise_nocache],write 12189696~98304 [fadvise_sequential+fadvise_nocache],write 12402688~98304 [fadvise_sequential+fadvise_nocache],write 12697600~98304 [fadvise_sequential+fadvise_nocache],write 12910592~98304 [fadvise_sequential+fadvise_nocache],write 13205504~98304 [fadvise_sequential+fadvise_nocache],write 13418496~98304 [fadvise_sequential+fadvise_nocache],write 13713408~98304 [fadvise_sequential+fadvise_nocache],write 13926400~98304 [fadvise_sequential+fadvise_nocache],write 14139392~16384 [fadvise_sequential+fadvise_nocache],write 14680064~16384 [fadvise_sequential+fadvise_nocache],write 14827520~81920 [fadvise_sequential+fadvise_nocache],write 15040512~81920 [fadvise_sequential+fadvise_nocache],write 15335424~81920 [fadvise_sequential+fadvise_nocache],write 15548416~81920 [fadvise_sequential+fadvise_nocache],write 15843328~81920 [fadvise_sequential+fadvise_nocache],write 16056320~81920 [fadvise_sequential+fadvise_nocache],write 16351232~81920 [fadvise_sequential+fadvise_nocache],write 16564224~81920 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:35.647 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.354331+0000", 2024-09-21T15:00:35.647 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 30.283608256000001, 2024-09-21T15:00:35.647 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.086284870999999999, 2024-09-21T15:00:35.647 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:35.647 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:35.647 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:35.647 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:35.647 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:35.647 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97734 2024-09-21T15:00:35.647 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.647 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:35.648 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.648 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:35.648 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.354331+0000", 2024-09-21T15:00:35.648 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:35.648 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.648 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.648 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:35.648 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.354331+0000", 2024-09-21T15:00:35.648 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:35.648 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.648 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.648 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:35.648 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.354329+0000", 2024-09-21T15:00:35.648 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:00:35.648 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.648 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.649 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:35.649 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.370234+0000", 2024-09-21T15:00:35.649 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015905348999999999 2024-09-21T15:00:35.649 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.649 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.649 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:35.649 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.370286+0000", 2024-09-21T15:00:35.649 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1656999999999997e-05 2024-09-21T15:00:35.649 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.649 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.649 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:35.649 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.370315+0000", 2024-09-21T15:00:35.649 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9204e-05 2024-09-21T15:00:35.649 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.649 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.650 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:35.650 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.370395+0000", 2024-09-21T15:00:35.650 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0395999999999994e-05 2024-09-21T15:00:35.650 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.650 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.650 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:35.650 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.370604+0000", 2024-09-21T15:00:35.650 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000208684 2024-09-21T15:00:35.650 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.650 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.650 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 1", 2024-09-21T15:00:35.650 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.371701+0000", 2024-09-21T15:00:35.650 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0010973140000000001 2024-09-21T15:00:35.650 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.650 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.651 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:35.651 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.405485+0000", 2024-09-21T15:00:35.651 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.033783615000000003 2024-09-21T15:00:35.651 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.651 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.651 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:35.651 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.440211+0000", 2024-09-21T15:00:35.651 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034725757000000003 2024-09-21T15:00:35.651 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.651 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.651 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:35.651 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.440497+0000", 2024-09-21T15:00:35.651 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00028673099999999998 2024-09-21T15:00:35.651 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.651 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.652 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:35.652 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.440616+0000", 2024-09-21T15:00:35.652 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000118256 2024-09-21T15:00:35.652 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:35.652 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:35.652 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:35.652 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.652 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.652 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112831 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 1515520~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:35.652 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.406761+0000", 2024-09-21T15:00:35.652 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 6.2311775579999997, 2024-09-21T15:00:35.652 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060265666000000002, 2024-09-21T15:00:35.652 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:35.652 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:35.652 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:35.652 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:35.653 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:35.653 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112831 2024-09-21T15:00:35.653 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.653 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:35.653 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.653 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:35.653 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406761+0000", 2024-09-21T15:00:35.653 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:35.653 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.653 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.653 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:35.653 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406761+0000", 2024-09-21T15:00:35.653 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:35.653 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.653 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.653 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:35.654 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406761+0000", 2024-09-21T15:00:35.654 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:35.654 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.654 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.654 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:35.654 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406764+0000", 2024-09-21T15:00:35.654 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0319999999999999e-06 2024-09-21T15:00:35.654 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.654 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.654 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:35.654 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406776+0000", 2024-09-21T15:00:35.654 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2357999999999999e-05 2024-09-21T15:00:35.654 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.654 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.655 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:35.655 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406784+0000", 2024-09-21T15:00:35.655 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6059999999999997e-06 2024-09-21T15:00:35.655 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.655 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.655 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:35.655 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.466595+0000", 2024-09-21T15:00:35.655 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059811062999999998 2024-09-21T15:00:35.655 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.655 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.655 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:35.655 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.466782+0000", 2024-09-21T15:00:35.655 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000186622 2024-09-21T15:00:35.655 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.656 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.656 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:35.656 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.467027+0000", 2024-09-21T15:00:35.656 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000245455 2024-09-21T15:00:35.656 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:35.656 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:35.656 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:35.656 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.656 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.656 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112832 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 1605632~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:35.656 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.406788+0000", 2024-09-21T15:00:35.656 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 6.2311509679999997, 2024-09-21T15:00:35.656 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060452065999999999, 2024-09-21T15:00:35.656 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:35.656 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:35.656 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:35.657 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:35.657 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:35.657 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112832 2024-09-21T15:00:35.657 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.657 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:35.657 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.657 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:35.657 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406788+0000", 2024-09-21T15:00:35.657 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:35.657 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.657 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.657 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:35.657 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406788+0000", 2024-09-21T15:00:35.657 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:35.657 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.657 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.658 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:35.658 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406788+0000", 2024-09-21T15:00:35.658 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:35.658 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.658 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.658 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:35.658 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406798+0000", 2024-09-21T15:00:35.658 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0096e-05 2024-09-21T15:00:35.658 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.658 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.658 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:35.658 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406803+0000", 2024-09-21T15:00:35.658 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2020000000000003e-06 2024-09-21T15:00:35.658 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.658 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.659 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:35.659 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406808+0000", 2024-09-21T15:00:35.659 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8239999999999999e-06 2024-09-21T15:00:35.659 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.659 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.659 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:35.659 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.467049+0000", 2024-09-21T15:00:35.659 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060241754000000002 2024-09-21T15:00:35.659 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.659 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.659 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:35.659 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.467134+0000", 2024-09-21T15:00:35.659 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4216999999999997e-05 2024-09-21T15:00:35.659 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.659 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.659 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:35.660 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.467240+0000", 2024-09-21T15:00:35.660 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000106304 2024-09-21T15:00:35.660 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:35.660 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:35.660 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:35.660 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.660 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.660 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112843 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 2613248~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:35.660 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.410273+0000", 2024-09-21T15:00:35.660 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 6.227666374, 2024-09-21T15:00:35.660 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059854931, 2024-09-21T15:00:35.660 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:35.660 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:35.660 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:35.660 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:35.660 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:35.661 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112843 2024-09-21T15:00:35.661 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.661 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:35.661 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.661 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:35.661 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410273+0000", 2024-09-21T15:00:35.661 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:35.661 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.661 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.661 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:35.661 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410273+0000", 2024-09-21T15:00:35.661 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:35.661 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.661 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.661 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:35.661 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410272+0000", 2024-09-21T15:00:35.662 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:35.662 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.662 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.662 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:35.662 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410274+0000", 2024-09-21T15:00:35.662 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.905e-06 2024-09-21T15:00:35.662 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.662 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.662 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:35.662 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410283+0000", 2024-09-21T15:00:35.662 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3440000000000007e-06 2024-09-21T15:00:35.662 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.662 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.662 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:35.662 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410294+0000", 2024-09-21T15:00:35.663 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0271e-05 2024-09-21T15:00:35.663 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.663 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.663 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:35.663 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.469917+0000", 2024-09-21T15:00:35.663 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059623738000000003 2024-09-21T15:00:35.663 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.663 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.663 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:35.663 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470002+0000", 2024-09-21T15:00:35.663 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4270999999999993e-05 2024-09-21T15:00:35.663 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.663 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.663 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:35.663 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470127+0000", 2024-09-21T15:00:35.664 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000125711 2024-09-21T15:00:35.664 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:35.664 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:35.664 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:35.664 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.664 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.664 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112844 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 2703360~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:35.664 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.410326+0000", 2024-09-21T15:00:35.664 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 6.2276126539999996, 2024-09-21T15:00:35.664 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060086056999999998, 2024-09-21T15:00:35.664 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:35.664 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:35.664 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:35.664 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:35.664 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:35.664 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112844 2024-09-21T15:00:35.665 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.665 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:35.665 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.665 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:35.665 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410326+0000", 2024-09-21T15:00:35.665 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:35.665 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.665 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.665 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:35.665 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410326+0000", 2024-09-21T15:00:35.665 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:35.665 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.665 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.665 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:35.665 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410326+0000", 2024-09-21T15:00:35.666 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:00:35.666 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.666 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.666 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:35.666 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410352+0000", 2024-09-21T15:00:35.666 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6872999999999998e-05 2024-09-21T15:00:35.666 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.666 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.666 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:35.666 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410374+0000", 2024-09-21T15:00:35.666 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1141e-05 2024-09-21T15:00:35.666 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.666 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.666 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:35.667 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410385+0000", 2024-09-21T15:00:35.667 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1456e-05 2024-09-21T15:00:35.667 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.667 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.667 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:35.667 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470174+0000", 2024-09-21T15:00:35.667 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059788574999999997 2024-09-21T15:00:35.667 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.667 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.667 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:35.667 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470277+0000", 2024-09-21T15:00:35.667 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000103838 2024-09-21T15:00:35.667 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.667 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.667 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:35.668 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470412+0000", 2024-09-21T15:00:35.668 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013489700000000001 2024-09-21T15:00:35.668 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:35.668 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:35.668 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:35.668 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.668 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.668 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112845 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 2793472~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:35.668 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.410394+0000", 2024-09-21T15:00:35.668 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 6.2275451799999999, 2024-09-21T15:00:35.668 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060244173999999998, 2024-09-21T15:00:35.668 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:35.668 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:35.668 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:35.668 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:35.669 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:35.669 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112845 2024-09-21T15:00:35.669 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.669 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:35.669 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.669 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:35.669 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410394+0000", 2024-09-21T15:00:35.669 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:35.669 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.669 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.669 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:35.669 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410394+0000", 2024-09-21T15:00:35.669 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:35.669 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.669 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.670 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:35.670 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410393+0000", 2024-09-21T15:00:35.670 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:35.670 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.670 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.670 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:35.670 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410397+0000", 2024-09-21T15:00:35.670 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9430000000000002e-06 2024-09-21T15:00:35.670 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.670 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.670 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:35.670 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410406+0000", 2024-09-21T15:00:35.670 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.799e-06 2024-09-21T15:00:35.670 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.670 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.671 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:35.671 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410418+0000", 2024-09-21T15:00:35.671 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2225999999999999e-05 2024-09-21T15:00:35.671 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.671 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.671 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:35.671 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470444+0000", 2024-09-21T15:00:35.671 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060026311999999998 2024-09-21T15:00:35.671 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.671 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.671 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:35.671 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470535+0000", 2024-09-21T15:00:35.671 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0173999999999994e-05 2024-09-21T15:00:35.671 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.671 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.671 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:35.672 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470638+0000", 2024-09-21T15:00:35.672 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010333 2024-09-21T15:00:35.672 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:35.672 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:35.672 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:35.672 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.672 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.672 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112846 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 2883584~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:35.672 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.410425+0000", 2024-09-21T15:00:35.672 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 6.2275139670000002, 2024-09-21T15:00:35.672 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060459247000000001, 2024-09-21T15:00:35.672 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:35.672 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:35.672 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:35.673 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:35.673 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:35.673 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112846 2024-09-21T15:00:35.673 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.673 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:35.673 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.673 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:35.673 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410425+0000", 2024-09-21T15:00:35.673 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:35.673 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.673 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.673 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:35.673 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410425+0000", 2024-09-21T15:00:35.673 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:35.673 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.674 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.674 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:35.674 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410424+0000", 2024-09-21T15:00:35.674 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:35.674 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.674 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.674 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:35.674 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410428+0000", 2024-09-21T15:00:35.674 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4709999999999999e-06 2024-09-21T15:00:35.674 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.674 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.674 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:35.674 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410446+0000", 2024-09-21T15:00:35.674 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8354000000000001e-05 2024-09-21T15:00:35.674 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.675 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.675 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:35.675 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410456+0000", 2024-09-21T15:00:35.675 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6709999999999994e-06 2024-09-21T15:00:35.675 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.675 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.675 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:35.675 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470661+0000", 2024-09-21T15:00:35.675 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060205303000000002 2024-09-21T15:00:35.675 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.675 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.675 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:35.675 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470756+0000", 2024-09-21T15:00:35.675 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4401999999999995e-05 2024-09-21T15:00:35.675 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.676 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.676 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:35.676 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470884+0000", 2024-09-21T15:00:35.676 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012860500000000001 2024-09-21T15:00:35.676 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:35.676 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:35.676 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:35.676 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.676 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.676 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112847 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 2973696~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:35.676 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.410462+0000", 2024-09-21T15:00:35.676 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 6.2274769230000002, 2024-09-21T15:00:35.676 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060653836000000003, 2024-09-21T15:00:35.676 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:35.677 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:35.677 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:35.677 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:35.677 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:35.677 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112847 2024-09-21T15:00:35.677 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.677 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:35.677 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.677 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:35.677 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410462+0000", 2024-09-21T15:00:35.677 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:35.677 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.677 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.677 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:35.677 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410462+0000", 2024-09-21T15:00:35.677 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:35.678 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.678 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.678 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:35.678 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410461+0000", 2024-09-21T15:00:35.678 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:35.678 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.678 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.678 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:35.678 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410465+0000", 2024-09-21T15:00:35.678 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6899999999999998e-06 2024-09-21T15:00:35.678 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.678 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.678 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:35.678 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410474+0000", 2024-09-21T15:00:35.678 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4179999999999997e-06 2024-09-21T15:00:35.679 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.679 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.679 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:35.679 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410508+0000", 2024-09-21T15:00:35.679 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4282999999999997e-05 2024-09-21T15:00:35.679 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.679 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.679 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:35.679 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470922+0000", 2024-09-21T15:00:35.679 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060413820999999999 2024-09-21T15:00:35.679 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.679 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.679 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:35.679 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471008+0000", 2024-09-21T15:00:35.679 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6379999999999996e-05 2024-09-21T15:00:35.680 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.680 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.680 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:35.680 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471116+0000", 2024-09-21T15:00:35.680 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000107806 2024-09-21T15:00:35.680 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:35.680 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:35.680 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:35.680 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.680 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.680 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112848 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 3063808~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:35.680 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.410525+0000", 2024-09-21T15:00:35.680 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 6.22741364, 2024-09-21T15:00:35.680 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060880735999999998, 2024-09-21T15:00:35.680 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:35.681 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:35.681 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:35.681 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:35.681 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:35.681 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112848 2024-09-21T15:00:35.681 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.681 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:35.681 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.681 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:35.681 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410525+0000", 2024-09-21T15:00:35.681 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:35.681 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.681 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.681 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:35.681 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410525+0000", 2024-09-21T15:00:35.681 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:35.681 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.682 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.682 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:35.682 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410525+0000", 2024-09-21T15:00:35.682 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:35.682 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.682 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.682 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:35.682 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410529+0000", 2024-09-21T15:00:35.682 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1760000000000003e-06 2024-09-21T15:00:35.682 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.682 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.682 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:35.682 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410551+0000", 2024-09-21T15:00:35.682 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2099999999999998e-05 2024-09-21T15:00:35.682 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.683 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.683 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:35.683 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410571+0000", 2024-09-21T15:00:35.683 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0159000000000001e-05 2024-09-21T15:00:35.683 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.683 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.683 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:35.683 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471164+0000", 2024-09-21T15:00:35.683 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.06059287 2024-09-21T15:00:35.683 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.683 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.683 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:35.683 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471271+0000", 2024-09-21T15:00:35.683 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000106635 2024-09-21T15:00:35.683 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.683 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.684 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:35.684 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471406+0000", 2024-09-21T15:00:35.684 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000135399 2024-09-21T15:00:35.684 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:35.684 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:35.684 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:35.684 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.684 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.684 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112849 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 3153920~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:35.684 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.410579+0000", 2024-09-21T15:00:35.684 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 6.2273603450000001, 2024-09-21T15:00:35.684 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.061057304999999999, 2024-09-21T15:00:35.684 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:35.684 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:35.684 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:35.685 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:35.685 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:35.685 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112849 2024-09-21T15:00:35.685 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.685 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:35.685 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.685 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:35.685 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410579+0000", 2024-09-21T15:00:35.685 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:35.685 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.685 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.685 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:35.685 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410579+0000", 2024-09-21T15:00:35.685 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:35.685 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.685 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.686 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:35.686 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410578+0000", 2024-09-21T15:00:35.686 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:35.686 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.686 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.686 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:35.686 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410582+0000", 2024-09-21T15:00:35.686 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.036e-06 2024-09-21T15:00:35.686 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.686 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.686 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:35.686 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410592+0000", 2024-09-21T15:00:35.686 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9850000000000001e-06 2024-09-21T15:00:35.686 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.686 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.686 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:35.687 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410604+0000", 2024-09-21T15:00:35.687 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2051e-05 2024-09-21T15:00:35.687 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.687 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.687 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:35.687 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471444+0000", 2024-09-21T15:00:35.687 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060839722999999998 2024-09-21T15:00:35.687 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.687 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.687 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:35.687 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471530+0000", 2024-09-21T15:00:35.687 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6359000000000004e-05 2024-09-21T15:00:35.687 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.687 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.687 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:35.687 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471636+0000", 2024-09-21T15:00:35.688 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010576700000000001 2024-09-21T15:00:35.688 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:35.688 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:35.688 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:35.688 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.688 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.688 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112850 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 3244032~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:35.688 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.410698+0000", 2024-09-21T15:00:35.688 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 6.2272410029999996, 2024-09-21T15:00:35.688 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.061164118000000003, 2024-09-21T15:00:35.688 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:35.688 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:35.688 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:35.688 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:35.688 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:35.689 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112850 2024-09-21T15:00:35.689 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.689 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:35.689 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.689 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:35.689 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410698+0000", 2024-09-21T15:00:35.689 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:35.689 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.689 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.689 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:35.689 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410698+0000", 2024-09-21T15:00:35.689 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:35.689 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.689 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.689 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:35.689 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410610+0000", 2024-09-21T15:00:35.690 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999123 2024-09-21T15:00:35.690 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.690 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.690 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:35.690 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410707+0000", 2024-09-21T15:00:35.690 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7093e-05 2024-09-21T15:00:35.690 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.690 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.690 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:35.690 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410728+0000", 2024-09-21T15:00:35.690 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0275000000000002e-05 2024-09-21T15:00:35.690 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.690 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.690 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:35.690 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410740+0000", 2024-09-21T15:00:35.691 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2561999999999999e-05 2024-09-21T15:00:35.691 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.691 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.691 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:35.691 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471664+0000", 2024-09-21T15:00:35.691 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060923979000000003 2024-09-21T15:00:35.691 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.691 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.691 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:35.691 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471753+0000", 2024-09-21T15:00:35.691 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8265999999999998e-05 2024-09-21T15:00:35.691 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.692 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.692 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:35.692 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471862+0000", 2024-09-21T15:00:35.692 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000109474 2024-09-21T15:00:35.692 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:35.692 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:35.692 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:35.692 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.692 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.692 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112864 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 4521984~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:35.692 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.414789+0000", 2024-09-21T15:00:35.692 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 6.2231499780000004, 2024-09-21T15:00:35.692 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059845903999999998, 2024-09-21T15:00:35.693 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:35.693 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:35.693 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:35.693 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:35.693 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:35.693 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112864 2024-09-21T15:00:35.693 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.693 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:35.693 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.693 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:35.693 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414789+0000", 2024-09-21T15:00:35.693 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:35.693 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.693 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.694 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:35.694 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414789+0000", 2024-09-21T15:00:35.694 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:35.694 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.694 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.694 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:35.694 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414788+0000", 2024-09-21T15:00:35.694 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:35.694 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.694 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.694 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:35.694 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414790+0000", 2024-09-21T15:00:35.694 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.995e-06 2024-09-21T15:00:35.695 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.695 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.695 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:35.695 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414801+0000", 2024-09-21T15:00:35.695 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.01e-05 2024-09-21T15:00:35.695 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.695 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.695 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:35.695 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414804+0000", 2024-09-21T15:00:35.695 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.563e-06 2024-09-21T15:00:35.695 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.695 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.695 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:35.696 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.474499+0000", 2024-09-21T15:00:35.696 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059695208 2024-09-21T15:00:35.696 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.696 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.696 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:35.696 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.474561+0000", 2024-09-21T15:00:35.696 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1672000000000002e-05 2024-09-21T15:00:35.696 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.696 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.696 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:35.696 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.474635+0000", 2024-09-21T15:00:35.696 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3792000000000001e-05 2024-09-21T15:00:35.696 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:35.696 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:35.697 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:35.697 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.697 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.697 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112865 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 4612096~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:35.697 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.414807+0000", 2024-09-21T15:00:35.697 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 6.2231315819999997, 2024-09-21T15:00:35.697 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060003506999999998, 2024-09-21T15:00:35.697 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:35.697 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:35.697 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:35.697 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:35.697 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:35.697 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112865 2024-09-21T15:00:35.697 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.697 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:35.698 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.698 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:35.698 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414807+0000", 2024-09-21T15:00:35.698 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:35.698 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.698 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.698 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:35.698 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414807+0000", 2024-09-21T15:00:35.698 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:35.698 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.698 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.698 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:35.698 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414807+0000", 2024-09-21T15:00:35.698 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:35.698 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.699 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.699 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:35.699 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414809+0000", 2024-09-21T15:00:35.699 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0509999999999999e-06 2024-09-21T15:00:35.699 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.699 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.699 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:35.699 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414813+0000", 2024-09-21T15:00:35.699 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0960000000000003e-06 2024-09-21T15:00:35.699 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.699 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.699 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:35.699 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414817+0000", 2024-09-21T15:00:35.699 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.296e-06 2024-09-21T15:00:35.699 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.700 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.700 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:35.700 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.474671+0000", 2024-09-21T15:00:35.700 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059853663000000001 2024-09-21T15:00:35.700 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.700 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.700 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:35.700 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.474735+0000", 2024-09-21T15:00:35.700 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3653999999999997e-05 2024-09-21T15:00:35.700 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.700 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.700 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:35.700 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.474811+0000", 2024-09-21T15:00:35.700 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6074000000000004e-05 2024-09-21T15:00:35.700 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:35.701 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:35.701 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:35.701 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.701 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.701 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112866 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 4702208~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:35.701 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.414820+0000", 2024-09-21T15:00:35.701 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 6.2231186620000001, 2024-09-21T15:00:35.701 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060161992999999997, 2024-09-21T15:00:35.701 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:35.701 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:35.701 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:35.701 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:35.701 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:35.701 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112866 2024-09-21T15:00:35.701 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.702 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:35.702 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.702 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:35.702 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414820+0000", 2024-09-21T15:00:35.702 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:35.702 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.702 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.702 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:35.702 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414820+0000", 2024-09-21T15:00:35.702 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:35.702 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.702 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.702 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:35.702 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414820+0000", 2024-09-21T15:00:35.703 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:35.703 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.703 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.703 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:35.703 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414822+0000", 2024-09-21T15:00:35.703 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8509999999999999e-06 2024-09-21T15:00:35.703 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.703 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.703 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:35.703 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414826+0000", 2024-09-21T15:00:35.703 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8190000000000002e-06 2024-09-21T15:00:35.703 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.703 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.703 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:35.703 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414837+0000", 2024-09-21T15:00:35.704 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1232e-05 2024-09-21T15:00:35.704 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.704 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.704 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:35.704 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.474846+0000", 2024-09-21T15:00:35.704 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060009506999999997 2024-09-21T15:00:35.704 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.704 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.704 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:35.704 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.474910+0000", 2024-09-21T15:00:35.704 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3918999999999996e-05 2024-09-21T15:00:35.704 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.704 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.704 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:35.704 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.474982+0000", 2024-09-21T15:00:35.705 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1958999999999993e-05 2024-09-21T15:00:35.705 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:35.705 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:35.705 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:35.705 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.705 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.705 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112867 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 4792320~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:35.705 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.414840+0000", 2024-09-21T15:00:35.705 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 6.223099167, 2024-09-21T15:00:35.705 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060308542999999999, 2024-09-21T15:00:35.705 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:35.705 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:35.705 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:35.705 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:35.705 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:35.705 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112867 2024-09-21T15:00:35.706 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.706 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:35.706 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.706 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:35.706 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414840+0000", 2024-09-21T15:00:35.706 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:35.706 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.706 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.706 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:35.706 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414840+0000", 2024-09-21T15:00:35.706 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:35.706 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.706 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.706 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:35.707 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414839+0000", 2024-09-21T15:00:35.707 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:35.707 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.707 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.707 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:35.707 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414841+0000", 2024-09-21T15:00:35.707 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8500000000000001e-06 2024-09-21T15:00:35.707 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.707 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.707 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:35.707 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414846+0000", 2024-09-21T15:00:35.707 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7700000000000001e-06 2024-09-21T15:00:35.707 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.707 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.708 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:35.708 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414850+0000", 2024-09-21T15:00:35.708 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1389999999999997e-06 2024-09-21T15:00:35.708 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.708 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.708 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:35.708 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.475016+0000", 2024-09-21T15:00:35.708 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.06016614 2024-09-21T15:00:35.708 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.708 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.708 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:35.708 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.475073+0000", 2024-09-21T15:00:35.708 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6295000000000002e-05 2024-09-21T15:00:35.708 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.708 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.709 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:35.709 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.475148+0000", 2024-09-21T15:00:35.709 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5647000000000003e-05 2024-09-21T15:00:35.709 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:35.709 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:35.709 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:35.709 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.709 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.709 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:113030 3.6 3:60420fc2:::rbd_data.1136b442e94a.0000000000000001:head [write 0~360448 [fadvise_sequential+fadvise_nocache],write 385024~573440 [fadvise_sequential+fadvise_nocache],write 1146880~573440 [fadvise_sequential+fadvise_nocache],write 1908736~188416 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:35.709 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.422420+0000", 2024-09-21T15:00:35.709 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 6.215518522, 2024-09-21T15:00:35.709 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.063268959999999999, 2024-09-21T15:00:35.709 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:35.709 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:35.709 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:35.710 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:35.710 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:35.710 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 113030 2024-09-21T15:00:35.710 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.710 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:35.710 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.710 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:35.710 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.422420+0000", 2024-09-21T15:00:35.710 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:35.710 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.710 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.710 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:35.710 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.422420+0000", 2024-09-21T15:00:35.710 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:35.711 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.711 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.711 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:35.711 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.422420+0000", 2024-09-21T15:00:35.711 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:35.711 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.711 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.711 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:35.711 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.424657+0000", 2024-09-21T15:00:35.711 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0022369780000000001 2024-09-21T15:00:35.711 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.711 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.711 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:35.711 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.424673+0000", 2024-09-21T15:00:35.711 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6141e-05 2024-09-21T15:00:35.711 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.712 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.712 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:35.712 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.424689+0000", 2024-09-21T15:00:35.712 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5897999999999999e-05 2024-09-21T15:00:35.712 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.712 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.712 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:35.712 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.478005+0000", 2024-09-21T15:00:35.712 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.05331636 2024-09-21T15:00:35.712 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.712 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.712 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:35.712 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.478133+0000", 2024-09-21T15:00:35.712 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012736799999999999 2024-09-21T15:00:35.712 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.713 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.713 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 7", 2024-09-21T15:00:35.713 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.478307+0000", 2024-09-21T15:00:35.713 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017491499999999999 2024-09-21T15:00:35.713 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.713 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.713 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:35.713 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.483611+0000", 2024-09-21T15:00:35.713 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0053039389999999997 2024-09-21T15:00:35.713 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.713 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.713 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:35.713 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.485553+0000", 2024-09-21T15:00:35.713 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0019418759999999999 2024-09-21T15:00:35.713 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.713 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.714 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:35.714 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.485645+0000", 2024-09-21T15:00:35.714 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2138000000000005e-05 2024-09-21T15:00:35.714 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.714 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.714 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:35.714 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.485689+0000", 2024-09-21T15:00:35.714 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3922999999999999e-05 2024-09-21T15:00:35.714 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:35.714 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:35.714 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:35.714 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.714 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.715 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:114290 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 6946816~8192 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:35.715 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.558087+0000", 2024-09-21T15:00:35.715 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 6.0798517250000002, 2024-09-21T15:00:35.715 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059973091999999999, 2024-09-21T15:00:35.715 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:35.715 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:35.715 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:35.715 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:35.715 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:35.715 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 114290 2024-09-21T15:00:35.715 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.715 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:35.715 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.715 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:35.715 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.558087+0000", 2024-09-21T15:00:35.716 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:35.716 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.716 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.716 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:35.716 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.558087+0000", 2024-09-21T15:00:35.716 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:35.716 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.716 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.716 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:35.716 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.558087+0000", 2024-09-21T15:00:35.716 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:35.716 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.716 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.716 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:35.717 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.558092+0000", 2024-09-21T15:00:35.717 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2379999999999997e-06 2024-09-21T15:00:35.717 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.717 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.717 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:35.717 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.558095+0000", 2024-09-21T15:00:35.717 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.275e-06 2024-09-21T15:00:35.717 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.717 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.717 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:35.717 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.558099+0000", 2024-09-21T15:00:35.717 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6679999999999999e-06 2024-09-21T15:00:35.717 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.717 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.717 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:35.718 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.617757+0000", 2024-09-21T15:00:35.718 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059657531999999999 2024-09-21T15:00:35.718 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.718 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.718 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:35.718 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.617891+0000", 2024-09-21T15:00:35.718 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000134478 2024-09-21T15:00:35.718 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.718 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.718 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:35.718 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.618060+0000", 2024-09-21T15:00:35.718 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016925100000000001 2024-09-21T15:00:35.718 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:35.718 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:35.718 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:35.719 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.719 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.719 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:114312 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 8945664~8192 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:35.719 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.562396+0000", 2024-09-21T15:00:35.719 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 6.0755433769999998, 2024-09-21T15:00:35.719 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059832396000000003, 2024-09-21T15:00:35.719 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:35.719 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:35.719 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:35.719 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:35.719 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:35.719 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 114312 2024-09-21T15:00:35.719 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.719 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:35.719 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.720 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:35.720 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562396+0000", 2024-09-21T15:00:35.720 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:35.720 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.720 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.720 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:35.720 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562396+0000", 2024-09-21T15:00:35.720 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:35.720 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.720 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.720 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:35.720 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562395+0000", 2024-09-21T15:00:35.720 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:35.720 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.720 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.721 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:35.721 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562397+0000", 2024-09-21T15:00:35.721 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9e-06 2024-09-21T15:00:35.721 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.721 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.721 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:35.721 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562401+0000", 2024-09-21T15:00:35.721 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8099999999999999e-06 2024-09-21T15:00:35.721 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.721 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.721 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:35.721 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562405+0000", 2024-09-21T15:00:35.721 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.5710000000000002e-06 2024-09-21T15:00:35.721 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.721 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.722 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:35.722 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.621997+0000", 2024-09-21T15:00:35.722 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059592118999999999 2024-09-21T15:00:35.722 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.722 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.722 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:35.722 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.622121+0000", 2024-09-21T15:00:35.722 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012411599999999999 2024-09-21T15:00:35.722 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.722 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.722 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:35.722 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.622228+0000", 2024-09-21T15:00:35.722 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000107184 2024-09-21T15:00:35.722 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:35.722 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:35.723 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:35.723 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.723 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.723 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:114313 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 9035776~8192 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:35.723 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.562408+0000", 2024-09-21T15:00:35.723 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 6.0755313959999997, 2024-09-21T15:00:35.723 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059975560999999997, 2024-09-21T15:00:35.723 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:35.723 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:35.723 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:35.723 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:35.723 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:35.723 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 114313 2024-09-21T15:00:35.723 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.724 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:35.724 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.724 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:35.724 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562408+0000", 2024-09-21T15:00:35.724 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:35.724 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.724 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.724 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:35.724 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562408+0000", 2024-09-21T15:00:35.724 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:35.724 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.724 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.724 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:35.724 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562407+0000", 2024-09-21T15:00:35.724 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:35.725 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.725 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.725 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:35.725 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562409+0000", 2024-09-21T15:00:35.725 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9700000000000002e-06 2024-09-21T15:00:35.725 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.725 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.725 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:35.725 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562421+0000", 2024-09-21T15:00:35.725 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2094e-05 2024-09-21T15:00:35.725 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.725 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.725 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:35.725 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562427+0000", 2024-09-21T15:00:35.725 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7690000000000002e-06 2024-09-21T15:00:35.726 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.726 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.726 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:35.726 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.622264+0000", 2024-09-21T15:00:35.726 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059837488000000001 2024-09-21T15:00:35.726 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.726 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.726 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:35.726 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.622319+0000", 2024-09-21T15:00:35.726 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4064999999999998e-05 2024-09-21T15:00:35.726 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:35.726 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:35.727 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:35.727 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.622383+0000", 2024-09-21T15:00:35.727 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4506999999999998e-05 2024-09-21T15:00:35.727 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:35.727 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:35.727 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:35.727 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:35.727 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:35.727 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:35.728 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T15:00:35.728 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T15:00:35.729 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-21T15:00:35.729 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T15:00:35.729 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:00:35.723+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:00:35.729 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T15:00:35.738 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T15:00:35.762 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T15:00:35.765 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:00:35.765 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:00:35.765 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:00:35.766 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:00:35.766 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:00:35.766 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:00:35.766 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:00:35.766 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:00:35.830 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:00:35.822+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:00:35.847 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:35.848 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:35.859 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T15:00:35.931 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:00:35.927+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:00:35.969 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:35.969 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:35.979 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T15:00:36.032 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:00:36.031+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:00:36.087 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:36.087 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:36.098 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T15:00:36.133 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:00:36.127+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:00:36.207 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:36.207 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:36.217 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T15:00:36.266 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:00:36.266+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:00:36.329 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:36.330 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:36.334 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:00:36.331+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:00:36.339 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T15:00:36.369 INFO:teuthology.orchestra.run.smithi032.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-21T15:00:36.434 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:00:36.434+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:00:36.484 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:36.484 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:36.499 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T15:00:36.535 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:00:36.534+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:00:36.637 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:00:36.635+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:00:36.642 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:36.642 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:36.655 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T15:00:36.736 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:00:36.731+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:00:36.807 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:36.807 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:36.817 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T15:00:36.837 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:00:36.834+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:00:36.919 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:36.919 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:36.929 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T15:00:36.971 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:00:36.970+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:00:37.039 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:00:37.038+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:00:37.049 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:37.050 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:37.060 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T15:00:37.140 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:00:37.138+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:00:37.169 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:37.169 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:37.178 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T15:00:37.241 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:00:37.239+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:00:37.287 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:37.287 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:37.297 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T15:00:37.341 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:00:37.338+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:00:37.367 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T15:00:37.415 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:37.415 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:37.426 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T15:00:37.442 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:00:37.438+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:00:37.466 INFO:teuthology.orchestra.run.smithi032.stdout:checking clone #28, image image_client.0-clone28 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent29 2024-09-21T15:00:37.532 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:37.533 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:37.543 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:00:37.539+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:00:37.543 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T15:00:37.643 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:00:37.639+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:00:37.645 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:37.645 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:37.656 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T15:00:37.744 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:00:37.739+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:00:37.748 INFO:teuthology.orchestra.run.smithi032.stdout:4081 writesame 0xce0109c thru 0xce03bcd data_size 0x39(0x2b32 bytes) 2024-09-21T15:00:37.748 INFO:teuthology.orchestra.run.smithi032.stdout:4082 read 0x27b45cf thru 0x27c406f (0xfaa1 bytes) 2024-09-21T15:00:37.749 INFO:teuthology.orchestra.run.smithi032.stdout:4083 write 0xc41655 thru 0xc453b2 (0x3d5e bytes) 2024-09-21T15:00:37.753 INFO:teuthology.orchestra.run.smithi032.stdout:4084 trunc from 0xce03bce to 0xcb8bb44 2024-09-21T15:00:37.764 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:37.764 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:37.764 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:37.764 INFO:teuthology.orchestra.run.smithi032.stdout:4089 writesame 0xa655ee8 thru 0xa65ac47 data_size 0x10(0x4d60 bytes) 2024-09-21T15:00:37.773 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:37.783 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T15:00:37.791 INFO:teuthology.orchestra.run.smithi032.stdout:4092 trunc from 0xcb8bb44 to 0x2085ce7 2024-09-21T15:00:37.825 INFO:teuthology.orchestra.run.smithi032.stdout:4093 punch from 0xe6248b to 0xe627bd, (0x332 bytes) 2024-09-21T15:00:37.825 INFO:teuthology.orchestra.run.smithi032.stdout:4094 punch from 0xcd84e9 to 0xcdb996, (0x34ad bytes) 2024-09-21T15:00:37.825 INFO:teuthology.orchestra.run.smithi032.stdout:4095 write 0x103c917 thru 0x104c66c (0xfd56 bytes) 2024-09-21T15:00:37.834 INFO:teuthology.orchestra.run.smithi032.stdout:4096 punch from 0xee094d to 0xee642d, (0x5ae0 bytes) 2024-09-21T15:00:37.834 INFO:teuthology.orchestra.run.smithi032.stdout:4097 writesame 0x1ad0564 thru 0x1adb7b6 data_size 0x1(0xb253 bytes) 2024-09-21T15:00:37.835 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:37.835 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:37.845 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:00:37.841+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:00:37.847 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T15:00:37.863 INFO:teuthology.orchestra.run.smithi032.stdout:4098 read 0x1609912 thru 0x160a493 (0xb82 bytes) 2024-09-21T15:00:37.885 INFO:teuthology.orchestra.run.smithi032.stdout:4099 write 0xa3220ff thru 0xa3299f6 (0x78f8 bytes) 2024-09-21T15:00:37.894 INFO:teuthology.orchestra.run.smithi032.stdout:4100 punch from 0x96df60 to 0x96fbdf, (0x1c7f bytes) 2024-09-21T15:00:37.894 INFO:teuthology.orchestra.run.smithi032.stdout:4101 write 0x627a457 thru 0x6286b39 (0xc6e3 bytes) 2024-09-21T15:00:37.902 INFO:teuthology.orchestra.run.smithi032.stdout:4103 read 0x57dd272 thru 0x57df745 (0x24d4 bytes) 2024-09-21T15:00:37.902 INFO:teuthology.orchestra.run.smithi032.stdout:4104 writesame 0x52967f6 thru 0x52a092f data_size 0x9(0xa13a bytes) 2024-09-21T15:00:37.914 INFO:teuthology.orchestra.run.smithi032.stdout:4105 writesame 0x915b236 thru 0x916a588 data_size 0x1(0xf353 bytes) 2024-09-21T15:00:37.946 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:00:37.941+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:00:37.946 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T15:00:37.951 INFO:teuthology.orchestra.run.smithi032.stdout:4106 punch from 0x7d7e72b to 0x7d8bf84, (0xd859 bytes) 2024-09-21T15:00:37.951 INFO:teuthology.orchestra.run.smithi032.stdout:4107 write 0x422561e thru 0x4227801 (0x21e4 bytes) 2024-09-21T15:00:37.958 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:37.958 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:37.958 INFO:teuthology.orchestra.run.smithi032.stdout:4110 trunc from 0xa3299f7 to 0xa219e1f 2024-09-21T15:00:37.968 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T15:00:37.979 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:37.979 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:37.979 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:37.987 INFO:teuthology.orchestra.run.smithi032.stdout:4115 trunc from 0xa219e1f to 0x646ed3d 2024-09-21T15:00:37.990 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T15:00:37.992 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T15:00:38.008 INFO:teuthology.orchestra.run.smithi032.stdout:4117 read 0x478944c thru 0x4789a9a (0x64f bytes) 2024-09-21T15:00:38.008 INFO:teuthology.orchestra.run.smithi032.stdout:4118 write 0x4e8991e thru 0x4e8bf5d (0x2640 bytes) 2024-09-21T15:00:38.013 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T15:00:38.014 INFO:teuthology.orchestra.run.smithi032.stdout:4120 read 0x4bcca thru 0x5647a (0xa7b1 bytes) 2024-09-21T15:00:38.015 INFO:teuthology.orchestra.run.smithi032.stdout:4121 read 0xbfbfeb thru 0xc0a58e (0xe5a4 bytes) 2024-09-21T15:00:38.029 INFO:teuthology.orchestra.run.smithi032.stdout:4122 trunc from 0x646ed3d to 0xcd52a7d 2024-09-21T15:00:38.044 INFO:teuthology.orchestra.run.smithi032.stdout:4125 trunc from 0xcd52a7d to 0xa237b3a 2024-09-21T15:00:38.047 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:00:38.045+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:00:38.048 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-21T15:00:38.054 INFO:teuthology.orchestra.run.smithi032.stdout:4126 read 0x58769d0 thru 0x587af88 (0x45b9 bytes) 2024-09-21T15:00:38.058 INFO:teuthology.orchestra.run.smithi032.stdout:4127 trunc from 0xa237b3a to 0x7f8fed9 2024-09-21T15:00:38.069 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:38.083 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T15:00:38.088 INFO:teuthology.orchestra.run.smithi032.stdout:4129 write 0xc72e850 thru 0xc734bcd (0x637e bytes) 2024-09-21T15:00:38.096 INFO:teuthology.orchestra.run.smithi032.stdout:4131 write 0xb494f00 thru 0xb49b90e (0x6a0f bytes) 2024-09-21T15:00:38.104 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:38.104 INFO:teuthology.orchestra.run.smithi032.stdout:4133 write 0x825c102 thru 0x8262f38 (0x6e37 bytes) 2024-09-21T15:00:38.114 INFO:teuthology.orchestra.run.smithi032.stdout:4134 write 0xa4a20e7 thru 0xa4a5d44 (0x3c5e bytes) 2024-09-21T15:00:38.115 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T15:00:38.120 INFO:teuthology.orchestra.run.smithi032.stdout:4138 write 0xaf6067a thru 0xaf66bd9 (0x6560 bytes) 2024-09-21T15:00:38.126 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:38.126 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T15:00:38.126 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T15:00:38.126 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T15:00:38.126 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.126 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:15130 3.2 3:47f1bbdb:::rbd_data.11361bed1f78.0000000000000067:head [sparse-read 0~147456] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-21T15:00:38.126 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:40.288284+0000", 2024-09-21T15:00:38.127 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 117.836896448, 2024-09-21T15:00:38.127 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1142935039999999, 2024-09-21T15:00:38.127 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:38.127 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:38.127 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:38.127 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:38.127 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:38.127 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 15130 2024-09-21T15:00:38.127 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.127 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:38.127 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.127 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:38.127 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288284+0000", 2024-09-21T15:00:38.127 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:38.127 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.128 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.128 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:38.128 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288284+0000", 2024-09-21T15:00:38.128 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:38.128 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.128 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.128 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:38.128 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288284+0000", 2024-09-21T15:00:38.128 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:38.128 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.128 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.128 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:38.128 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288288+0000", 2024-09-21T15:00:38.128 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5829999999999998e-06 2024-09-21T15:00:38.128 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.128 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.129 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:38.129 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288297+0000", 2024-09-21T15:00:38.129 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0669999999999998e-06 2024-09-21T15:00:38.129 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.129 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.129 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:38.129 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288305+0000", 2024-09-21T15:00:38.129 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8860000000000004e-06 2024-09-21T15:00:38.129 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.129 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.129 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:38.129 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288417+0000", 2024-09-21T15:00:38.129 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000112952 2024-09-21T15:00:38.129 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.129 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.130 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:00:38.130 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288428+0000", 2024-09-21T15:00:38.130 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0631e-05 2024-09-21T15:00:38.130 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.130 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.130 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:38.130 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.401876+0000", 2024-09-21T15:00:38.130 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1134479610000001 2024-09-21T15:00:38.130 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.130 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.130 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:38.130 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.402076+0000", 2024-09-21T15:00:38.130 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00019944200000000001 2024-09-21T15:00:38.130 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.130 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.131 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:38.131 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.402578+0000", 2024-09-21T15:00:38.131 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00050242899999999998 2024-09-21T15:00:38.131 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:38.131 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:38.131 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:38.131 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.131 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.131 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45708 3.2 3:415db91a:::rbd_data.1136ecbe5879.0000000000000000:28 [read 140288~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:38.131 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141801+0000", 2024-09-21T15:00:38.131 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 87.983380253999997, 2024-09-21T15:00:38.131 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034274200999999997, 2024-09-21T15:00:38.131 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:38.131 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:38.131 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:38.131 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:38.132 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:38.132 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45708 2024-09-21T15:00:38.132 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.132 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:38.132 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.132 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:38.132 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141801+0000", 2024-09-21T15:00:38.132 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:38.132 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.132 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.132 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:38.132 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141801+0000", 2024-09-21T15:00:38.132 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:38.132 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.133 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.133 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:38.133 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141800+0000", 2024-09-21T15:00:38.133 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:38.133 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.133 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.133 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:38.133 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141803+0000", 2024-09-21T15:00:38.133 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4210000000000002e-06 2024-09-21T15:00:38.133 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.133 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.133 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:38.133 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141812+0000", 2024-09-21T15:00:38.133 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.9800000000000004e-06 2024-09-21T15:00:38.133 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.134 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.134 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:38.134 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141816+0000", 2024-09-21T15:00:38.134 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4769999999999997e-06 2024-09-21T15:00:38.134 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.134 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.134 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:38.134 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.175865+0000", 2024-09-21T15:00:38.134 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034048875999999999 2024-09-21T15:00:38.134 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.134 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.134 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:38.134 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.175943+0000", 2024-09-21T15:00:38.135 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7464000000000002e-05 2024-09-21T15:00:38.135 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.135 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.135 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:38.135 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176075+0000", 2024-09-21T15:00:38.135 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013230800000000001 2024-09-21T15:00:38.135 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:38.135 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:38.135 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:38.135 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.135 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.135 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45709 3.2 3:415db91a:::rbd_data.1136ecbe5879.0000000000000000:28 [read 143360~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:38.135 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141820+0000", 2024-09-21T15:00:38.135 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 87.983360407000006, 2024-09-21T15:00:38.135 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034537536000000001, 2024-09-21T15:00:38.135 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:38.136 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:38.136 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:38.136 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:38.136 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:38.136 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45709 2024-09-21T15:00:38.136 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.136 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:38.136 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.136 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:38.136 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141820+0000", 2024-09-21T15:00:38.136 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:38.136 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.136 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.136 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:38.136 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141820+0000", 2024-09-21T15:00:38.137 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:38.137 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.137 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.137 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:38.137 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141820+0000", 2024-09-21T15:00:38.137 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:38.137 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.137 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.137 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:38.137 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141823+0000", 2024-09-21T15:00:38.137 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5110000000000002e-06 2024-09-21T15:00:38.137 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.137 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.137 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:38.137 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141829+0000", 2024-09-21T15:00:38.138 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0850000000000002e-06 2024-09-21T15:00:38.138 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.138 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.138 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:38.138 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141833+0000", 2024-09-21T15:00:38.138 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7620000000000003e-06 2024-09-21T15:00:38.138 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.138 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.138 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:38.138 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176103+0000", 2024-09-21T15:00:38.138 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034269933000000002 2024-09-21T15:00:38.138 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.138 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.138 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:38.138 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176161+0000", 2024-09-21T15:00:38.139 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7658999999999998e-05 2024-09-21T15:00:38.139 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.139 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.139 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:38.139 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176358+0000", 2024-09-21T15:00:38.139 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000196937 2024-09-21T15:00:38.139 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:38.139 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:38.139 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:38.139 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.139 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.139 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45710 3.2 3:415db91a:::rbd_data.1136ecbe5879.0000000000000000:28 [read 146432~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:38.139 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141838+0000", 2024-09-21T15:00:38.139 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 87.983343243999997, 2024-09-21T15:00:38.140 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034953674999999997, 2024-09-21T15:00:38.140 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:38.140 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:38.140 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:38.140 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:38.140 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:38.140 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45710 2024-09-21T15:00:38.140 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.140 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:38.140 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.140 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:38.140 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141838+0000", 2024-09-21T15:00:38.140 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:38.140 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.140 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.140 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:38.141 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141838+0000", 2024-09-21T15:00:38.141 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:38.141 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.141 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.141 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:38.141 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141837+0000", 2024-09-21T15:00:38.141 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:38.141 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.141 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.141 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:38.141 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141843+0000", 2024-09-21T15:00:38.141 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1619999999999996e-06 2024-09-21T15:00:38.141 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.141 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.141 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:38.141 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141850+0000", 2024-09-21T15:00:38.142 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4799999999999998e-06 2024-09-21T15:00:38.142 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.142 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.142 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:38.142 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141853+0000", 2024-09-21T15:00:38.142 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3050000000000001e-06 2024-09-21T15:00:38.142 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.142 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.142 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:38.142 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176490+0000", 2024-09-21T15:00:38.142 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034636413999999997 2024-09-21T15:00:38.142 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.142 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.142 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:38.142 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176606+0000", 2024-09-21T15:00:38.143 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000115841 2024-09-21T15:00:38.143 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.143 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.143 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:38.143 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176791+0000", 2024-09-21T15:00:38.143 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00018581500000000001 2024-09-21T15:00:38.143 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:38.143 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:38.143 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:38.143 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.143 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.143 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45711 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:38.143 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141856+0000", 2024-09-21T15:00:38.143 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 87.983324906999997, 2024-09-21T15:00:38.143 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034986308000000001, 2024-09-21T15:00:38.143 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:38.144 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:38.144 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:38.144 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:38.144 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:38.144 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45711 2024-09-21T15:00:38.144 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.144 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:38.144 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.144 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:38.144 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141856+0000", 2024-09-21T15:00:38.144 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:38.144 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.144 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.144 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:38.144 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141856+0000", 2024-09-21T15:00:38.145 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:38.145 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.145 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.145 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:38.145 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141856+0000", 2024-09-21T15:00:38.145 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:38.145 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.145 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.145 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:38.145 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141862+0000", 2024-09-21T15:00:38.145 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.567e-06 2024-09-21T15:00:38.145 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.145 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.145 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:38.145 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141868+0000", 2024-09-21T15:00:38.145 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9569999999999999e-06 2024-09-21T15:00:38.146 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.146 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.146 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:38.146 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141871+0000", 2024-09-21T15:00:38.146 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.146e-06 2024-09-21T15:00:38.146 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.146 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.146 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:38.146 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176831+0000", 2024-09-21T15:00:38.146 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034959935999999997 2024-09-21T15:00:38.146 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.146 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.146 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:38.146 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176842+0000", 2024-09-21T15:00:38.146 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1008e-05 2024-09-21T15:00:38.146 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:38.147 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:38.147 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:38.147 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.147 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.147 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45712 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:38.147 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141874+0000", 2024-09-21T15:00:38.147 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 87.983307011999997, 2024-09-21T15:00:38.147 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034993511999999997, 2024-09-21T15:00:38.147 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:38.147 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:38.147 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:38.147 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:38.147 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:38.147 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45712 2024-09-21T15:00:38.147 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.148 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:38.148 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.148 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:38.148 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141874+0000", 2024-09-21T15:00:38.148 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:38.148 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.148 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.148 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:38.148 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141874+0000", 2024-09-21T15:00:38.148 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:38.148 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.148 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.148 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:38.148 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141874+0000", 2024-09-21T15:00:38.148 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:38.148 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.149 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.149 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:38.149 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141875+0000", 2024-09-21T15:00:38.149 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5430000000000001e-06 2024-09-21T15:00:38.149 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.149 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.149 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:38.149 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141878+0000", 2024-09-21T15:00:38.149 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1420000000000002e-06 2024-09-21T15:00:38.149 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.149 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.149 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:38.149 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141881+0000", 2024-09-21T15:00:38.149 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.807e-06 2024-09-21T15:00:38.149 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.150 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.150 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:38.150 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176860+0000", 2024-09-21T15:00:38.150 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034979243 2024-09-21T15:00:38.150 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.150 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.150 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:38.150 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176867+0000", 2024-09-21T15:00:38.150 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0169999999999999e-06 2024-09-21T15:00:38.150 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:38.150 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:38.150 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:38.150 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.150 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.151 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45713 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:38.151 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141883+0000", 2024-09-21T15:00:38.151 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 87.983297437000004, 2024-09-21T15:00:38.151 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035010516999999998, 2024-09-21T15:00:38.151 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:38.151 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:38.151 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:38.151 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:38.151 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:38.151 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45713 2024-09-21T15:00:38.151 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.151 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:38.151 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.151 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:38.151 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141883+0000", 2024-09-21T15:00:38.151 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:38.151 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.152 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.152 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:38.152 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141883+0000", 2024-09-21T15:00:38.152 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:38.152 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.152 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.152 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:38.152 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141883+0000", 2024-09-21T15:00:38.152 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:38.152 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.152 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.152 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:38.152 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141885+0000", 2024-09-21T15:00:38.152 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4649999999999999e-06 2024-09-21T15:00:38.152 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.152 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.153 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:38.153 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141888+0000", 2024-09-21T15:00:38.153 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3890000000000001e-06 2024-09-21T15:00:38.153 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.153 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.153 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:38.153 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141891+0000", 2024-09-21T15:00:38.153 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9239999999999999e-06 2024-09-21T15:00:38.153 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.153 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.153 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:38.153 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176887+0000", 2024-09-21T15:00:38.153 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034996370999999998 2024-09-21T15:00:38.153 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.153 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.154 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:38.154 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176894+0000", 2024-09-21T15:00:38.154 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6069999999999996e-06 2024-09-21T15:00:38.154 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:38.154 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:38.154 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:38.154 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.154 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.154 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45714 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:38.154 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141893+0000", 2024-09-21T15:00:38.154 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 87.983287521999998, 2024-09-21T15:00:38.154 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035048215000000001, 2024-09-21T15:00:38.154 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:38.154 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:38.154 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:38.155 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:38.155 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:38.155 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45714 2024-09-21T15:00:38.155 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.155 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:38.155 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.155 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:38.155 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141893+0000", 2024-09-21T15:00:38.155 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:38.155 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.155 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.155 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:38.155 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141893+0000", 2024-09-21T15:00:38.155 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:38.155 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.155 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.156 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:38.156 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141893+0000", 2024-09-21T15:00:38.156 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:38.156 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.156 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.156 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:38.156 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141895+0000", 2024-09-21T15:00:38.156 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.703e-06 2024-09-21T15:00:38.156 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.156 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.156 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:38.156 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141900+0000", 2024-09-21T15:00:38.156 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.242e-06 2024-09-21T15:00:38.156 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.156 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.156 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:38.157 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141903+0000", 2024-09-21T15:00:38.157 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.117e-06 2024-09-21T15:00:38.157 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.157 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.157 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:38.157 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176930+0000", 2024-09-21T15:00:38.157 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035026750000000002 2024-09-21T15:00:38.157 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.157 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.157 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:38.157 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176942+0000", 2024-09-21T15:00:38.157 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1668000000000001e-05 2024-09-21T15:00:38.157 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:38.157 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:38.157 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:38.157 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.158 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.158 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45715 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:38.158 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141906+0000", 2024-09-21T15:00:38.158 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 87.983275363999994, 2024-09-21T15:00:38.158 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035066057999999997, 2024-09-21T15:00:38.158 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:38.158 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:38.158 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:38.158 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:38.158 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:38.158 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45715 2024-09-21T15:00:38.158 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.158 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:38.158 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.158 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:38.158 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141906+0000", 2024-09-21T15:00:38.159 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:38.159 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.159 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.159 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:38.159 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141906+0000", 2024-09-21T15:00:38.159 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:38.159 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.159 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.159 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:38.159 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141905+0000", 2024-09-21T15:00:38.159 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:38.159 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.159 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.159 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:38.159 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141907+0000", 2024-09-21T15:00:38.159 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5400000000000001e-06 2024-09-21T15:00:38.160 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.160 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.160 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:38.160 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141912+0000", 2024-09-21T15:00:38.160 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.502e-06 2024-09-21T15:00:38.160 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.160 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.160 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:38.160 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141915+0000", 2024-09-21T15:00:38.160 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0170000000000001e-06 2024-09-21T15:00:38.160 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.160 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.160 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:38.160 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176964+0000", 2024-09-21T15:00:38.160 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035048587999999999 2024-09-21T15:00:38.160 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.160 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.161 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:38.161 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176972+0000", 2024-09-21T15:00:38.161 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6580000000000002e-06 2024-09-21T15:00:38.161 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:38.161 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:38.161 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:38.161 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.161 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.161 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45716 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:38.161 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141918+0000", 2024-09-21T15:00:38.161 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 87.983263187999995, 2024-09-21T15:00:38.161 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035086223, 2024-09-21T15:00:38.161 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:38.161 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:38.161 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:38.161 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:38.162 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:38.162 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45716 2024-09-21T15:00:38.162 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.162 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:38.162 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.162 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:38.162 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141918+0000", 2024-09-21T15:00:38.162 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:38.162 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.162 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.162 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:38.162 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141918+0000", 2024-09-21T15:00:38.162 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:38.162 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.162 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.162 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:38.163 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141917+0000", 2024-09-21T15:00:38.163 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:38.163 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.163 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.163 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:38.163 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141919+0000", 2024-09-21T15:00:38.163 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5680000000000001e-06 2024-09-21T15:00:38.163 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.163 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.163 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:38.163 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141922+0000", 2024-09-21T15:00:38.163 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2789999999999999e-06 2024-09-21T15:00:38.163 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.163 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.163 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:38.163 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141925+0000", 2024-09-21T15:00:38.163 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9469999999999999e-06 2024-09-21T15:00:38.164 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.164 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.164 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:38.164 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176997+0000", 2024-09-21T15:00:38.164 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035071366999999999 2024-09-21T15:00:38.164 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.164 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.164 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:38.164 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177004+0000", 2024-09-21T15:00:38.164 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3309999999999998e-06 2024-09-21T15:00:38.164 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:38.164 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:38.164 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:38.164 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.165 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.165 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45717 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:38.165 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141928+0000", 2024-09-21T15:00:38.165 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 87.983253242000004, 2024-09-21T15:00:38.165 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035102573999999998, 2024-09-21T15:00:38.165 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:38.165 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:38.165 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:38.165 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:38.165 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:38.165 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45717 2024-09-21T15:00:38.165 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.165 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:38.165 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.165 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:38.166 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141928+0000", 2024-09-21T15:00:38.166 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:38.166 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.166 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.166 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:38.166 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141928+0000", 2024-09-21T15:00:38.166 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:38.166 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.166 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.166 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:38.166 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141927+0000", 2024-09-21T15:00:38.166 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:00:38.166 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.166 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.166 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:38.167 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141929+0000", 2024-09-21T15:00:38.167 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.533e-06 2024-09-21T15:00:38.167 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.167 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.167 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:38.167 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141934+0000", 2024-09-21T15:00:38.167 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1660000000000002e-06 2024-09-21T15:00:38.167 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.167 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.167 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:38.167 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141937+0000", 2024-09-21T15:00:38.167 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0309999999999999e-06 2024-09-21T15:00:38.167 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.167 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.167 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:38.167 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177023+0000", 2024-09-21T15:00:38.168 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035085595999999997 2024-09-21T15:00:38.168 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.168 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.168 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:38.168 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177030+0000", 2024-09-21T15:00:38.168 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.486e-06 2024-09-21T15:00:38.168 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:38.168 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:38.168 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:38.168 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.168 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.168 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45718 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:38.168 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141940+0000", 2024-09-21T15:00:38.168 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 87.983241383999996, 2024-09-21T15:00:38.168 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035124956999999998, 2024-09-21T15:00:38.168 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:38.169 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:38.169 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:38.169 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:38.169 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:38.169 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45718 2024-09-21T15:00:38.169 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.169 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:38.169 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.169 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:38.169 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141940+0000", 2024-09-21T15:00:38.169 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:38.169 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.169 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.169 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:38.169 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141940+0000", 2024-09-21T15:00:38.169 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:38.170 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.170 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.170 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:38.170 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141939+0000", 2024-09-21T15:00:38.170 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:38.170 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.170 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.170 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:38.170 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141941+0000", 2024-09-21T15:00:38.170 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.502e-06 2024-09-21T15:00:38.170 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.170 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.170 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:38.170 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141946+0000", 2024-09-21T15:00:38.170 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3449999999999996e-06 2024-09-21T15:00:38.170 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.171 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.171 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:38.171 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141949+0000", 2024-09-21T15:00:38.171 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9749999999999999e-06 2024-09-21T15:00:38.171 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.171 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.171 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:38.171 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177057+0000", 2024-09-21T15:00:38.171 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035108385999999998 2024-09-21T15:00:38.171 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.171 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.171 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:38.171 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177064+0000", 2024-09-21T15:00:38.171 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0099999999999998e-06 2024-09-21T15:00:38.171 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:38.171 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:38.172 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:38.172 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.172 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.172 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45719 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:38.172 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141952+0000", 2024-09-21T15:00:38.172 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 87.983229111, 2024-09-21T15:00:38.172 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035139108000000002, 2024-09-21T15:00:38.172 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:38.172 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:38.172 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:38.172 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:38.172 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:38.172 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45719 2024-09-21T15:00:38.172 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.172 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:38.172 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.172 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:38.173 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141952+0000", 2024-09-21T15:00:38.173 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:38.173 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.173 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.173 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:38.173 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141952+0000", 2024-09-21T15:00:38.173 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:38.173 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.173 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.173 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:38.173 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141952+0000", 2024-09-21T15:00:38.173 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:38.173 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.173 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.173 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:38.174 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141953+0000", 2024-09-21T15:00:38.174 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.486e-06 2024-09-21T15:00:38.174 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.174 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.174 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:38.174 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141959+0000", 2024-09-21T15:00:38.174 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4940000000000003e-06 2024-09-21T15:00:38.174 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.174 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.174 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:38.174 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141963+0000", 2024-09-21T15:00:38.174 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3540000000000002e-06 2024-09-21T15:00:38.174 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.174 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.174 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:38.175 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177084+0000", 2024-09-21T15:00:38.175 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035120983000000001 2024-09-21T15:00:38.175 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.175 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.175 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:38.175 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177091+0000", 2024-09-21T15:00:38.175 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0350000000000004e-06 2024-09-21T15:00:38.175 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:38.175 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:38.175 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:38.175 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.175 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.175 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45720 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:38.175 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141965+0000", 2024-09-21T15:00:38.175 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 87.983215643999998, 2024-09-21T15:00:38.175 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035155268000000003, 2024-09-21T15:00:38.176 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:38.176 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:38.176 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:38.176 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:38.176 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:38.176 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45720 2024-09-21T15:00:38.176 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.176 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:38.176 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.176 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:38.176 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141965+0000", 2024-09-21T15:00:38.176 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:38.176 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.176 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.176 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:38.177 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141965+0000", 2024-09-21T15:00:38.177 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:38.177 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.177 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.177 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:38.177 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141965+0000", 2024-09-21T15:00:38.177 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:38.177 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.177 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.177 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:38.177 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141966+0000", 2024-09-21T15:00:38.177 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.46e-06 2024-09-21T15:00:38.177 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.177 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.177 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:38.177 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141972+0000", 2024-09-21T15:00:38.178 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4210000000000003e-06 2024-09-21T15:00:38.178 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.178 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.178 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:38.178 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141975+0000", 2024-09-21T15:00:38.178 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.948e-06 2024-09-21T15:00:38.178 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.178 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.178 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:38.178 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177113+0000", 2024-09-21T15:00:38.178 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035137855000000003 2024-09-21T15:00:38.178 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.178 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.178 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:38.178 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177121+0000", 2024-09-21T15:00:38.178 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8269999999999998e-06 2024-09-21T15:00:38.179 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:38.179 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:38.179 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:38.179 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.179 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.179 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45721 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:38.179 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141977+0000", 2024-09-21T15:00:38.179 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 87.983203621000001, 2024-09-21T15:00:38.179 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035169984000000001, 2024-09-21T15:00:38.179 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:38.179 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:38.179 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:38.179 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:38.179 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:38.179 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45721 2024-09-21T15:00:38.179 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.180 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:38.180 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.180 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:38.180 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141977+0000", 2024-09-21T15:00:38.180 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:38.180 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.180 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.180 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:38.180 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141977+0000", 2024-09-21T15:00:38.180 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:38.180 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.180 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.180 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:38.180 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141977+0000", 2024-09-21T15:00:38.180 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:38.180 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.180 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.181 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:38.181 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141979+0000", 2024-09-21T15:00:38.181 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.511e-06 2024-09-21T15:00:38.181 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.181 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.181 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:38.181 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141982+0000", 2024-09-21T15:00:38.181 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4180000000000001e-06 2024-09-21T15:00:38.181 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.181 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.181 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:38.181 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141985+0000", 2024-09-21T15:00:38.181 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9450000000000002e-06 2024-09-21T15:00:38.181 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.181 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.182 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:38.182 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177139+0000", 2024-09-21T15:00:38.182 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035154409999999997 2024-09-21T15:00:38.182 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.182 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.182 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:38.182 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177147+0000", 2024-09-21T15:00:38.182 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9570000000000002e-06 2024-09-21T15:00:38.182 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:38.182 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:38.182 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:38.182 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.182 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.182 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45722 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:38.182 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141987+0000", 2024-09-21T15:00:38.182 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 87.983193650000004, 2024-09-21T15:00:38.183 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035188492000000002, 2024-09-21T15:00:38.183 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:38.183 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:38.183 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:38.183 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:38.183 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:38.183 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45722 2024-09-21T15:00:38.183 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.183 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:38.183 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.183 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:38.183 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141987+0000", 2024-09-21T15:00:38.183 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:38.183 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.183 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.183 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:38.184 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141987+0000", 2024-09-21T15:00:38.184 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:38.184 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.184 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.184 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:38.184 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141987+0000", 2024-09-21T15:00:38.184 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:38.184 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.184 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.184 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:38.184 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141988+0000", 2024-09-21T15:00:38.184 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.463e-06 2024-09-21T15:00:38.184 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.184 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.184 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:38.185 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141994+0000", 2024-09-21T15:00:38.185 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.361e-06 2024-09-21T15:00:38.185 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.185 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.185 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:38.185 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141997+0000", 2024-09-21T15:00:38.185 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0359999999999998e-06 2024-09-21T15:00:38.185 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.185 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.185 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:38.185 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177169+0000", 2024-09-21T15:00:38.185 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035171651999999998 2024-09-21T15:00:38.185 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.185 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.186 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:38.186 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177176+0000", 2024-09-21T15:00:38.186 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2189999999999998e-06 2024-09-21T15:00:38.186 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:38.186 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:38.186 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:38.186 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.186 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.186 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45723 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:38.186 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141999+0000", 2024-09-21T15:00:38.186 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 87.983181482999996, 2024-09-21T15:00:38.186 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035197701999999997, 2024-09-21T15:00:38.186 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:38.186 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:38.186 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:38.186 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:38.187 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:38.187 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45723 2024-09-21T15:00:38.187 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.187 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:38.187 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.187 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:38.187 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141999+0000", 2024-09-21T15:00:38.187 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:38.187 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.187 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.187 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:38.187 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141999+0000", 2024-09-21T15:00:38.187 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:38.187 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.187 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.187 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:38.188 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141999+0000", 2024-09-21T15:00:38.188 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:38.188 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.188 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.188 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:38.188 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142003+0000", 2024-09-21T15:00:38.188 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5049999999999999e-06 2024-09-21T15:00:38.188 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.188 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.188 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:38.188 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142008+0000", 2024-09-21T15:00:38.188 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6579999999999999e-06 2024-09-21T15:00:38.188 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.188 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.188 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:38.188 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142011+0000", 2024-09-21T15:00:38.188 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0529999999999998e-06 2024-09-21T15:00:38.189 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.189 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.189 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:38.189 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177194+0000", 2024-09-21T15:00:38.189 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035182221999999999 2024-09-21T15:00:38.189 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.189 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.189 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:38.189 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177197+0000", 2024-09-21T15:00:38.189 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5030000000000002e-06 2024-09-21T15:00:38.189 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:38.189 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:38.189 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:38.189 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.189 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.189 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45724 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:38.190 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.142015+0000", 2024-09-21T15:00:38.190 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 87.983165516, 2024-09-21T15:00:38.190 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035195345000000003, 2024-09-21T15:00:38.190 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:38.190 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:38.190 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:38.190 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:38.190 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:38.190 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45724 2024-09-21T15:00:38.190 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.190 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:38.190 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.190 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:38.190 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142015+0000", 2024-09-21T15:00:38.190 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:38.190 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.190 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.191 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:38.191 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142015+0000", 2024-09-21T15:00:38.191 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:38.191 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.191 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.191 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:38.191 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142015+0000", 2024-09-21T15:00:38.191 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:38.191 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.191 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.191 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:38.191 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142017+0000", 2024-09-21T15:00:38.191 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5540000000000001e-06 2024-09-21T15:00:38.191 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.191 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.191 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:38.192 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142022+0000", 2024-09-21T15:00:38.192 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4229999999999996e-06 2024-09-21T15:00:38.192 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.192 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.192 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:38.192 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142025+0000", 2024-09-21T15:00:38.192 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.056e-06 2024-09-21T15:00:38.192 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.192 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.192 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:38.192 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177207+0000", 2024-09-21T15:00:38.192 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035182089 2024-09-21T15:00:38.192 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.192 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.192 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:38.192 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177211+0000", 2024-09-21T15:00:38.193 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4649999999999999e-06 2024-09-21T15:00:38.193 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:38.193 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:38.193 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:38.193 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.193 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.193 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45725 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:38.193 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.142028+0000", 2024-09-21T15:00:38.193 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 87.983153236999996, 2024-09-21T15:00:38.193 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035195731000000001, 2024-09-21T15:00:38.193 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:38.193 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:38.193 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:38.193 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:38.193 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:38.194 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45725 2024-09-21T15:00:38.194 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.194 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:38.194 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.194 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:38.194 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142028+0000", 2024-09-21T15:00:38.194 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:38.194 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.194 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.194 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:38.194 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142028+0000", 2024-09-21T15:00:38.194 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:38.194 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.194 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.194 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:38.195 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142027+0000", 2024-09-21T15:00:38.195 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:38.195 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.195 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.195 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:38.195 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142029+0000", 2024-09-21T15:00:38.195 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5269999999999999e-06 2024-09-21T15:00:38.195 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.195 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.195 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:38.195 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142032+0000", 2024-09-21T15:00:38.195 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3249999999999999e-06 2024-09-21T15:00:38.195 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.195 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.195 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:38.195 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142035+0000", 2024-09-21T15:00:38.196 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.897e-06 2024-09-21T15:00:38.196 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.196 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.196 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:38.196 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177220+0000", 2024-09-21T15:00:38.196 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035184871 2024-09-21T15:00:38.196 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.196 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.196 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:38.196 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177223+0000", 2024-09-21T15:00:38.196 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3680000000000002e-06 2024-09-21T15:00:38.196 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:38.196 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:38.196 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:38.196 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.196 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.197 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45726 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:38.197 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.142037+0000", 2024-09-21T15:00:38.197 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 87.983143522000006, 2024-09-21T15:00:38.197 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035199273000000003, 2024-09-21T15:00:38.197 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:38.197 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:38.197 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:38.197 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:38.197 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:38.197 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45726 2024-09-21T15:00:38.197 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.197 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:38.197 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.197 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:38.197 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142037+0000", 2024-09-21T15:00:38.197 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:38.197 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.198 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.198 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:38.198 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142037+0000", 2024-09-21T15:00:38.198 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:38.198 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.198 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.198 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:38.198 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142037+0000", 2024-09-21T15:00:38.198 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:38.198 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.198 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.198 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:38.198 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142038+0000", 2024-09-21T15:00:38.198 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.344e-06 2024-09-21T15:00:38.198 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.198 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.199 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:38.199 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142044+0000", 2024-09-21T15:00:38.199 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1930000000000001e-06 2024-09-21T15:00:38.199 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.199 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.199 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:38.199 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142047+0000", 2024-09-21T15:00:38.199 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0280000000000001e-06 2024-09-21T15:00:38.199 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.199 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.199 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:38.199 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177233+0000", 2024-09-21T15:00:38.199 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035186394000000003 2024-09-21T15:00:38.199 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.199 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.199 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:38.199 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177237+0000", 2024-09-21T15:00:38.200 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5530000000000001e-06 2024-09-21T15:00:38.200 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:38.200 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:38.200 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:38.200 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:38.200 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:38.200 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:38.201 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T15:00:38.201 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T15:00:38.202 INFO:teuthology.orchestra.run.smithi032.stdout:4139 write 0xd98729d thru 0xd9919c7 (0xa72b bytes) 2024-09-21T15:00:38.202 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:00:38.202 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:00:38.202 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:00:38.202 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:00:38.202 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:00:38.202 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:00:38.202 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:00:38.203 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:00:38.203 INFO:teuthology.orchestra.run.smithi032.stdout:4140 punch from 0xc059449 to 0xc05d423, (0x3fda bytes) 2024-09-21T15:00:38.203 INFO:teuthology.orchestra.run.smithi032.stdout:4141 write 0xbebef56 thru 0xbecb01c (0xc0c7 bytes) 2024-09-21T15:00:38.204 INFO:teuthology.orchestra.run.smithi032.stdout:4142 writesame 0x9021b26 thru 0x902b5ae data_size 0x1(0x9a89 bytes) 2024-09-21T15:00:38.204 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:38.204 INFO:teuthology.orchestra.run.smithi032.stdout:4144 write 0x377a704 thru 0x377c624 (0x1f21 bytes) 2024-09-21T15:00:38.204 INFO:teuthology.orchestra.run.smithi032.stdout:4146 trunc from 0xd9919c8 to 0xd61f2f6 2024-09-21T15:00:38.205 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:00:38.197+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:00:38.216 INFO:teuthology.orchestra.run.smithi032.stdout:4147 write 0x2a74934 thru 0x2a7db0a (0x91d7 bytes) 2024-09-21T15:00:38.226 INFO:teuthology.orchestra.run.smithi032.stdout:4148 write 0xcb2d2d thru 0xcb7e57 (0x512b bytes) 2024-09-21T15:00:38.230 INFO:teuthology.orchestra.run.smithi032.stdout:4149 write 0x9d43a92 thru 0x9d4b5fa (0x7b69 bytes) 2024-09-21T15:00:38.239 INFO:teuthology.orchestra.run.smithi032.stdout:4150 write 0x9bd4362 thru 0x9bdd003 (0x8ca2 bytes) 2024-09-21T15:00:38.247 INFO:teuthology.orchestra.run.smithi032.stdout:4151 write 0xb552231 thru 0xb556190 (0x3f60 bytes) 2024-09-21T15:00:38.255 INFO:teuthology.orchestra.run.smithi032.stdout:4152 writesame 0xa921883 thru 0xa930b8a data_size 0x1c(0xf308 bytes) 2024-09-21T15:00:38.265 INFO:teuthology.orchestra.run.smithi032.stdout:4153 write 0x6230c56 thru 0x623736a (0x6715 bytes) 2024-09-21T15:00:38.272 INFO:teuthology.orchestra.run.smithi032.stdout:4156 read 0xb7d5829 thru 0xb7e1d86 (0xc55e bytes) 2024-09-21T15:00:38.272 INFO:teuthology.orchestra.run.smithi032.stdout:4158 write 0xd1ce37e thru 0xd1d3281 (0x4f04 bytes) 2024-09-21T15:00:38.282 INFO:teuthology.orchestra.run.smithi032.stdout:4159 read 0x4dff2dd thru 0x4e06f7b (0x7c9f bytes) 2024-09-21T15:00:38.283 INFO:teuthology.orchestra.run.smithi032.stdout:4160 punch from 0x901ffd3 to 0x90256db, (0x5708 bytes) 2024-09-21T15:00:38.283 INFO:teuthology.orchestra.run.smithi032.stdout:4161 write 0x13d9289 thru 0x13dc249 (0x2fc1 bytes) 2024-09-21T15:00:38.289 INFO:teuthology.orchestra.run.smithi032.stdout:4162 write 0x61392ea thru 0x61413a3 (0x80ba bytes) 2024-09-21T15:00:38.297 INFO:teuthology.orchestra.run.smithi032.stdout:4165 write 0xa9b3f38 thru 0xa9b4ce1 (0xdaa bytes) 2024-09-21T15:00:38.301 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:00:38.299+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:00:38.304 INFO:teuthology.orchestra.run.smithi032.stdout:4166 writesame 0xa830857 thru 0xa8349b8 data_size 0x1(0x4162 bytes) 2024-09-21T15:00:38.341 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:38.341 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:00:38.341 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T15:00:38.341 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:38.349 INFO:teuthology.orchestra.run.smithi032.stdout:4170 trunc from 0xd61f2f6 to 0x1ab2d7d 2024-09-21T15:00:38.355 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T15:00:38.382 INFO:teuthology.orchestra.run.smithi032.stderr:nodeep-scrub is unset 2024-09-21T15:00:38.393 INFO:teuthology.orchestra.run.smithi032.stdout:4171 punch from 0x18b5f66 to 0x18be598, (0x8632 bytes) 2024-09-21T15:00:38.393 INFO:teuthology.orchestra.run.smithi032.stdout:4172 read 0x151d309 thru 0x1522ec9 (0x5bc1 bytes) 2024-09-21T15:00:38.395 INFO:teuthology.orchestra.run.smithi032.stdout:4174 trunc from 0x1ab2d7d to 0x5935aa 2024-09-21T15:00:38.437 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:00:38.435+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:00:38.458 INFO:teuthology.orchestra.run.smithi032.stdout:4176 write 0x41caebe thru 0x41d3a0d (0x8b50 bytes) 2024-09-21T15:00:38.477 INFO:teuthology.orchestra.run.smithi032.stdout:4177 write 0x4f9ee57 thru 0x4f9efa2 (0x14c bytes) 2024-09-21T15:00:38.484 INFO:teuthology.orchestra.run.smithi032.stdout:4178 writesame 0x412393b thru 0x412ec76 data_size 0x2(0xb33c bytes) 2024-09-21T15:00:38.497 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:38.497 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:00:38.497 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T15:00:38.497 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T15:00:38.497 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:38.503 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:00:38.499+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:00:38.505 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:38.505 INFO:teuthology.orchestra.run.smithi032.stdout:4180 write 0x1290b12 thru 0x129dfdb (0xd4ca bytes) 2024-09-21T15:00:38.507 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T15:00:38.525 INFO:teuthology.orchestra.run.smithi032.stdout:4181 write 0x5a25342 thru 0x5a303e7 (0xb0a6 bytes) 2024-09-21T15:00:38.538 INFO:teuthology.orchestra.run.smithi032.stdout:4182 read 0x11e8b32 thru 0x11f2cde (0xa1ad bytes) 2024-09-21T15:00:38.539 INFO:teuthology.orchestra.run.smithi032.stdout:4183 trunc from 0x5a303e8 to 0x6142c02 2024-09-21T15:00:38.550 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:38.550 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:38.550 INFO:teuthology.orchestra.run.smithi032.stdout:4187 write 0x419e90b thru 0x41a7556 (0x8c4c bytes) 2024-09-21T15:00:38.558 INFO:teuthology.orchestra.run.smithi032.stdout:4188 punch from 0x23ffee2 to 0x240f22f, (0xf34d bytes) 2024-09-21T15:00:38.558 INFO:teuthology.orchestra.run.smithi032.stdout:4189 punch from 0x1d49368 to 0x1d57e2e, (0xeac6 bytes) 2024-09-21T15:00:38.564 INFO:teuthology.orchestra.run.smithi032.stdout:4191 trunc from 0x6142c02 to 0x2c1e3ae 2024-09-21T15:00:38.585 INFO:teuthology.orchestra.run.smithi032.stdout:4192 write 0xf05d96 thru 0xf07ef9 (0x2164 bytes) 2024-09-21T15:00:38.591 INFO:teuthology.orchestra.run.smithi032.stdout:4193 write 0xfc45bb thru 0xfca307 (0x5d4d bytes) 2024-09-21T15:00:38.599 INFO:teuthology.orchestra.run.smithi032.stdout:4194 punch from 0x9373ef to 0x9468a6, (0xf4b7 bytes) 2024-09-21T15:00:38.599 INFO:teuthology.orchestra.run.smithi032.stdout:4195 read 0xdc4209 thru 0xdca1fd (0x5ff5 bytes) 2024-09-21T15:00:38.604 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:00:38.599+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:00:38.618 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:38.618 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T15:00:38.618 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T15:00:38.618 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T15:00:38.619 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.619 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4281.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-21T15:00:38.619 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:09.324416+0000", 2024-09-21T15:00:38.619 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 149.293465481, 2024-09-21T15:00:38.619 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.117837947, 2024-09-21T15:00:38.619 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:38.619 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:38.619 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:38.619 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4281", 2024-09-21T15:00:38.619 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/1518520303", 2024-09-21T15:00:38.619 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1 2024-09-21T15:00:38.619 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.619 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:38.619 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.619 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:38.620 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324416+0000", 2024-09-21T15:00:38.620 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:38.620 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.620 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.620 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:38.620 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324416+0000", 2024-09-21T15:00:38.620 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:38.620 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.620 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.620 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:38.620 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324414+0000", 2024-09-21T15:00:38.620 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T15:00:38.620 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.620 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.620 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:38.621 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324429+0000", 2024-09-21T15:00:38.621 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5637e-05 2024-09-21T15:00:38.621 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.621 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.621 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:38.621 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324448+0000", 2024-09-21T15:00:38.621 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9042000000000002e-05 2024-09-21T15:00:38.621 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.621 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.621 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:38.621 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324465+0000", 2024-09-21T15:00:38.621 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6767999999999999e-05 2024-09-21T15:00:38.621 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.621 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.621 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:38.622 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324571+0000", 2024-09-21T15:00:38.622 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000105714 2024-09-21T15:00:38.622 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.622 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.622 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op must wait for map", 2024-09-21T15:00:38.622 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324587+0000", 2024-09-21T15:00:38.622 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5753999999999999e-05 2024-09-21T15:00:38.622 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.622 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.622 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:38.622 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.325864+0000", 2024-09-21T15:00:38.622 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0012773350000000001 2024-09-21T15:00:38.622 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.622 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.622 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:00:38.623 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.325878+0000", 2024-09-21T15:00:38.623 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4498000000000001e-05 2024-09-21T15:00:38.623 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.623 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.623 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:38.623 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.439461+0000", 2024-09-21T15:00:38.623 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.11358212500000001 2024-09-21T15:00:38.623 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.623 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.623 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:38.623 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.439848+0000", 2024-09-21T15:00:38.623 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00038747800000000002 2024-09-21T15:00:38.623 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.623 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.623 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T15:00:38.624 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.440175+0000", 2024-09-21T15:00:38.624 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00032721900000000001 2024-09-21T15:00:38.624 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.624 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.624 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:38.624 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.441770+0000", 2024-09-21T15:00:38.624 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0015949409999999999 2024-09-21T15:00:38.624 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.624 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.624 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:38.624 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.442086+0000", 2024-09-21T15:00:38.624 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00031586299999999999 2024-09-21T15:00:38.624 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.624 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.624 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:38.625 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.442195+0000", 2024-09-21T15:00:38.625 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000108791 2024-09-21T15:00:38.625 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.625 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.625 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:38.625 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.442254+0000", 2024-09-21T15:00:38.625 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9038e-05 2024-09-21T15:00:38.625 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:38.625 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:38.625 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:38.625 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.625 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.625 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4281.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-21T15:00:38.625 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:09.447828+0000", 2024-09-21T15:00:38.625 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 149.17005348000001, 2024-09-21T15:00:38.626 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.32904393700000001, 2024-09-21T15:00:38.626 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:38.626 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:38.626 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:38.626 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4281", 2024-09-21T15:00:38.626 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/1518520303", 2024-09-21T15:00:38.626 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 2 2024-09-21T15:00:38.626 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.626 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:38.626 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.626 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:38.626 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447828+0000", 2024-09-21T15:00:38.626 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:38.626 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.627 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.627 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:38.627 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447828+0000", 2024-09-21T15:00:38.627 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:38.627 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.627 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.627 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:38.627 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447825+0000", 2024-09-21T15:00:38.627 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T15:00:38.627 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.627 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.627 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:38.627 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447841+0000", 2024-09-21T15:00:38.627 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5359999999999999e-05 2024-09-21T15:00:38.627 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.628 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.628 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:38.628 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447875+0000", 2024-09-21T15:00:38.628 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4388999999999999e-05 2024-09-21T15:00:38.628 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.628 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.628 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:38.628 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447903+0000", 2024-09-21T15:00:38.628 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7373e-05 2024-09-21T15:00:38.628 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.628 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.628 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:38.628 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447996+0000", 2024-09-21T15:00:38.628 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2954999999999993e-05 2024-09-21T15:00:38.629 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.629 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.629 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:00:38.629 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.448011+0000", 2024-09-21T15:00:38.629 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5038999999999999e-05 2024-09-21T15:00:38.629 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.629 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.629 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:38.629 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.776501+0000", 2024-09-21T15:00:38.629 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.32849012500000002 2024-09-21T15:00:38.629 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.629 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.629 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:38.629 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.776872+0000", 2024-09-21T15:00:38.629 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00037119600000000002 2024-09-21T15:00:38.630 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:38.630 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:38.630 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:38.630 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.630 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.630 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:97673 3.1d 3:bd91946e:::rbd_data.1136bfc79a70.0000000000000002:head [write 0~4718592 [fadvise_sequential+fadvise_nocache],write 5242880~32768 [fadvise_sequential+fadvise_nocache],write 5341184~147456 [fadvise_sequential+fadvise_nocache],write 5554176~147456 [fadvise_sequential+fadvise_nocache],write 5767168~16384 [fadvise_sequential+fadvise_nocache],write 5849088~147456 [fadvise_sequential+fadvise_nocache],write 6062080~147456 [fadvise_sequential+fadvise_nocache],write 6275072~16384 [fadvise_sequential+fadvise_nocache],write 6356992~147456 [fadvise_sequential+fadvise_nocache],write 6569984~147456 [fadvise_sequential+fadvise_nocache],write 6782976~32768 [fadvise_sequential+fadvise_nocache],write 6864896~147456 [fadvise_sequential+fadvise_nocache],write 7077888~147456 [fadvise_sequential+fadvise_nocache],write 7290880~49152 [fadvise_sequential+fadvise_nocache],write 7372800~147456 [fadvise_sequential+fadvise_nocache],write 7585792~147456 [fadvise_sequential+fadvise_nocache],write 7798784~65536 [fadvise_sequential+fadvise_nocache],write 7880704~147456 [fadvise_sequential+fadvise_nocache],write 8093696~147456 [fadvise_sequential+fadvise_nocache],write 8306688~81920 [fadvise_sequential+fadvise_nocache],write 8421376~114688 [fadvise_sequential+fadvise_nocache],write 8634368~114688 [fadvise_sequential+fadvise_nocache],write 8847360~65536 [fadvise_sequential+fadvise_nocache],write 9011200~32768 [fadvise_sequential+fadvise_nocache],write 9224192~32768 [fadvise_sequential+fadvise_nocache],write 9519104~32768 [fadvise_sequential+fadvise_nocache],write 9732096~32768 [fadvise_sequential+fadvise_nocache],write 9945088~16384 [fadvise_sequential+fadvise_nocache],write 9977856~196608 [fadvise_sequential+fadvise_nocache],write 10190848~196608 [fadvise_sequential+fadvise_nocache],write 10403840~278528 [fadvise_sequential+fadvise_nocache],write 10698752~196608 [fadvise_sequential+fadvise_nocache],write 10911744~278528 [fadvise_sequential+fadvise_nocache],write 11206656~196608 [fadvise_sequential+fadvise_nocache],write 11419648~196608 [fadvise_sequential+fadvise_nocache],write 11632640~65536 [fadvise_sequential+fadvise_nocache],write 11730944~98304 [fadvise_sequential+fadvise_nocache],write 11845632~65536 [fadvise_sequential+fadvise_nocache],write 11943936~98304 [fadvise_sequential+fadvise_nocache],write 12058624~65536 [fadvise_sequential+fadvise_nocache],write 12140544~65536 [fadvise_sequential+fadvise_nocache],write 12238848~98304 [fadvise_sequential+fadvise_nocache],write 12353536~65536 [fadvise_sequential+fadvise_nocache],write 12451840~98304 [fadvise_sequential+fadvise_nocache],write 12566528~65536 [fadvise_sequential+fadvise_nocache],write 12746752~98304 [fadvise_sequential+fadvise_nocache],write 12959744~98304 [fadvise_sequential+fadvise_nocache],write 13107200~32768 [fadvise_sequential+fadvise_nocache],write 13254656~98304 [fadvise_sequential+fadvise_nocache],write 13467648~98304 [fadvise_sequential+fadvise_nocache],write 13631488~16384 [fadvise_sequential+fadvise_nocache],write 13762560~98304 [fadvise_sequential+fadvise_nocache],write 13975552~98304 [fadvise_sequential+fadvise_nocache],write 15204352~49152 [fadvise_sequential+fadvise_nocache],write 15384576~81920 [fadvise_sequential+fadvise_nocache],write 15597568~81920 [fadvise_sequential+fadvise_nocache],write 15728640~32768 [fadvise_sequential+fadvise_nocache],write 15892480~81920 [fadvise_sequential+fadvise_nocache],write 16105472~81920 [fadvise_sequential+fadvise_nocache],write 16252928~16384 [fadvise_sequential+fadvise_nocache],write 16400384~81920 [fadvise_sequential+fadvise_nocache],write 16613376~81920 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:38.630 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.300719+0000", 2024-09-21T15:00:38.630 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 33.317162707999998, 2024-09-21T15:00:38.630 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.068947114000000004, 2024-09-21T15:00:38.630 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:38.630 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:38.630 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:38.630 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:38.630 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:38.630 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 97673 2024-09-21T15:00:38.631 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.631 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:38.631 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.631 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:38.631 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.300719+0000", 2024-09-21T15:00:38.631 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:38.631 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.631 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.631 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:38.631 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.300719+0000", 2024-09-21T15:00:38.631 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:38.631 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.631 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.631 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:38.631 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.300716+0000", 2024-09-21T15:00:38.631 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-21T15:00:38.632 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.632 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.632 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:38.632 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.308380+0000", 2024-09-21T15:00:38.632 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0076638519999999996 2024-09-21T15:00:38.632 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.632 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.632 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:38.632 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.308418+0000", 2024-09-21T15:00:38.632 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.8380000000000002e-05 2024-09-21T15:00:38.632 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.632 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.632 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:38.632 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.308431+0000", 2024-09-21T15:00:38.632 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3213999999999999e-05 2024-09-21T15:00:38.632 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.633 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.633 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:38.633 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.308562+0000", 2024-09-21T15:00:38.633 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013015999999999999 2024-09-21T15:00:38.633 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.633 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.633 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:38.633 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.309173+0000", 2024-09-21T15:00:38.633 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00061185500000000002 2024-09-21T15:00:38.633 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.633 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.633 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 0", 2024-09-21T15:00:38.633 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311612+0000", 2024-09-21T15:00:38.633 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0024383949999999999 2024-09-21T15:00:38.633 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.634 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.634 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:38.634 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.355482+0000", 2024-09-21T15:00:38.634 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.043869940000000003 2024-09-21T15:00:38.634 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.634 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.634 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:38.634 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.369306+0000", 2024-09-21T15:00:38.634 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013824279 2024-09-21T15:00:38.634 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.634 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.634 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:38.634 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.369575+0000", 2024-09-21T15:00:38.634 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000269404 2024-09-21T15:00:38.634 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.635 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.635 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:38.635 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.369666+0000", 2024-09-21T15:00:38.635 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0426000000000005e-05 2024-09-21T15:00:38.635 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:38.635 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:38.635 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:38.635 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.635 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.635 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:112719 3.13 3:cd56ff5a:::rbd_data.1136b1194d20.0000000000000002:58 [read 1998848~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:38.635 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.387056+0000", 2024-09-21T15:00:38.635 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 9.2308253619999991, 2024-09-21T15:00:38.635 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055608623000000003, 2024-09-21T15:00:38.635 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:38.635 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:38.635 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:38.636 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:38.636 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:38.636 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 112719 2024-09-21T15:00:38.636 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.636 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:38.636 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.636 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:38.636 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.387056+0000", 2024-09-21T15:00:38.636 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:38.636 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.636 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.636 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:38.636 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.387056+0000", 2024-09-21T15:00:38.636 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:38.636 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.637 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.637 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:38.637 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.387056+0000", 2024-09-21T15:00:38.637 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:38.637 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.637 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.637 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:38.637 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.387058+0000", 2024-09-21T15:00:38.637 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.683e-06 2024-09-21T15:00:38.637 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.637 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.637 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:38.637 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.387064+0000", 2024-09-21T15:00:38.637 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6720000000000001e-06 2024-09-21T15:00:38.638 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.638 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.638 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:38.638 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.387069+0000", 2024-09-21T15:00:38.638 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8999999999999997e-06 2024-09-21T15:00:38.638 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.638 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.638 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:38.638 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.442514+0000", 2024-09-21T15:00:38.638 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055444881000000001 2024-09-21T15:00:38.638 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.638 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.638 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:38.638 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.442565+0000", 2024-09-21T15:00:38.638 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1010000000000001e-05 2024-09-21T15:00:38.639 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.639 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.639 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:38.639 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.442665+0000", 2024-09-21T15:00:38.639 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9840000000000006e-05 2024-09-21T15:00:38.639 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:38.639 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:38.639 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:38.639 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.639 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.639 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:112720 3.13 3:cd56ff5a:::rbd_data.1136b1194d20.0000000000000002:58 [read 2088960~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:38.639 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.387074+0000", 2024-09-21T15:00:38.639 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 9.2308078469999995, 2024-09-21T15:00:38.640 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055824591999999999, 2024-09-21T15:00:38.640 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:38.640 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:38.640 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:38.640 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:38.640 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:38.640 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 112720 2024-09-21T15:00:38.640 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.640 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:38.640 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.640 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:38.640 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.387074+0000", 2024-09-21T15:00:38.640 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:38.640 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.641 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.641 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:38.641 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.387074+0000", 2024-09-21T15:00:38.641 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:38.641 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.641 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.641 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:38.641 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.387073+0000", 2024-09-21T15:00:38.641 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:38.641 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.641 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.641 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:38.641 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.387076+0000", 2024-09-21T15:00:38.641 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4710000000000001e-06 2024-09-21T15:00:38.642 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.642 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.642 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:38.642 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.387085+0000", 2024-09-21T15:00:38.642 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2110000000000004e-06 2024-09-21T15:00:38.642 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.642 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.642 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:38.642 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.387090+0000", 2024-09-21T15:00:38.642 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.904e-06 2024-09-21T15:00:38.642 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.642 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.642 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:38.642 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.442702+0000", 2024-09-21T15:00:38.642 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055612251000000001 2024-09-21T15:00:38.643 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.643 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.643 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:38.643 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.442780+0000", 2024-09-21T15:00:38.643 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8232999999999994e-05 2024-09-21T15:00:38.646 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.646 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.646 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:38.646 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.442898+0000", 2024-09-21T15:00:38.646 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000117889 2024-09-21T15:00:38.646 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:38.646 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:38.647 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:38.647 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.647 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.647 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:112721 3.13 3:cd56ff5a:::rbd_data.1136b1194d20.0000000000000002:58 [read 2105344~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:38.647 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.387095+0000", 2024-09-21T15:00:38.647 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 9.2307867570000006, 2024-09-21T15:00:38.647 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.056000182000000003, 2024-09-21T15:00:38.647 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:38.647 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:38.647 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:38.647 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:38.647 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:38.647 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 112721 2024-09-21T15:00:38.647 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.647 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:38.648 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.648 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:38.648 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.387095+0000", 2024-09-21T15:00:38.648 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:38.648 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.648 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.648 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:38.648 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.387095+0000", 2024-09-21T15:00:38.648 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:38.648 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.648 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.648 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:38.648 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.387094+0000", 2024-09-21T15:00:38.648 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:38.648 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.648 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.649 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:38.649 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.387097+0000", 2024-09-21T15:00:38.649 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5940000000000001e-06 2024-09-21T15:00:38.649 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.649 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.649 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:38.649 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.387106+0000", 2024-09-21T15:00:38.649 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.9439999999999994e-06 2024-09-21T15:00:38.649 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.649 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.649 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:38.649 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.387111+0000", 2024-09-21T15:00:38.649 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3900000000000001e-06 2024-09-21T15:00:38.649 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.650 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.650 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:38.650 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.442934+0000", 2024-09-21T15:00:38.650 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055822996 2024-09-21T15:00:38.650 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.650 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.650 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:38.650 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.442998+0000", 2024-09-21T15:00:38.650 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3746999999999998e-05 2024-09-21T15:00:38.650 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.650 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.650 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:38.650 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.443095+0000", 2024-09-21T15:00:38.651 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6848999999999994e-05 2024-09-21T15:00:38.651 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:38.651 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:38.651 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:38.651 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.651 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.651 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:112733 3.13 3:cd56ff5a:::rbd_data.1136b1194d20.0000000000000002:58 [read 3186688~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:38.651 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.390744+0000", 2024-09-21T15:00:38.651 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 9.2271375790000008, 2024-09-21T15:00:38.651 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055498697999999999, 2024-09-21T15:00:38.651 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:38.651 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:38.651 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:38.651 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:38.652 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:38.652 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 112733 2024-09-21T15:00:38.652 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.652 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:38.652 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.652 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:38.652 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390744+0000", 2024-09-21T15:00:38.652 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:38.652 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.652 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.652 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:38.652 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390744+0000", 2024-09-21T15:00:38.652 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:38.652 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.652 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.653 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:38.653 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390743+0000", 2024-09-21T15:00:38.653 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:38.653 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.653 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.653 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:38.653 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390746+0000", 2024-09-21T15:00:38.653 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9510000000000002e-06 2024-09-21T15:00:38.653 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.653 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.653 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:38.653 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390751+0000", 2024-09-21T15:00:38.653 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.357e-06 2024-09-21T15:00:38.653 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.653 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.654 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:38.654 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390753+0000", 2024-09-21T15:00:38.654 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7310000000000002e-06 2024-09-21T15:00:38.654 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.654 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.654 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:38.654 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.445994+0000", 2024-09-21T15:00:38.654 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055240247999999999 2024-09-21T15:00:38.654 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.654 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.654 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:38.654 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.446111+0000", 2024-09-21T15:00:38.654 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00011699799999999999 2024-09-21T15:00:38.654 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.654 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.655 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:38.655 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.446243+0000", 2024-09-21T15:00:38.655 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013196099999999999 2024-09-21T15:00:38.655 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:38.655 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:38.655 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:38.655 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.655 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.655 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:112734 3.13 3:cd56ff5a:::rbd_data.1136b1194d20.0000000000000002:58 [read 3276800~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:38.655 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.390755+0000", 2024-09-21T15:00:38.655 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 9.2271259560000001, 2024-09-21T15:00:38.655 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.05569313, 2024-09-21T15:00:38.655 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:38.656 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:38.656 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:38.656 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:38.656 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:38.656 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 112734 2024-09-21T15:00:38.656 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.656 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:38.656 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.656 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:38.656 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390755+0000", 2024-09-21T15:00:38.656 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:38.656 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.656 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.656 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:38.656 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390755+0000", 2024-09-21T15:00:38.657 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:38.657 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.657 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.657 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:38.657 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390755+0000", 2024-09-21T15:00:38.657 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:00:38.657 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.657 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.657 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:38.657 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390757+0000", 2024-09-21T15:00:38.657 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5060000000000001e-06 2024-09-21T15:00:38.657 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.657 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.658 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:38.658 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390760+0000", 2024-09-21T15:00:38.658 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.79e-06 2024-09-21T15:00:38.658 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.658 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.658 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:38.658 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390762+0000", 2024-09-21T15:00:38.658 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.661e-06 2024-09-21T15:00:38.658 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.658 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.658 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:38.658 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.446280+0000", 2024-09-21T15:00:38.658 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055517761999999998 2024-09-21T15:00:38.658 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.658 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.659 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:38.659 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.446365+0000", 2024-09-21T15:00:38.659 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5208000000000001e-05 2024-09-21T15:00:38.659 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.659 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.659 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:38.659 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.446449+0000", 2024-09-21T15:00:38.659 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3417000000000004e-05 2024-09-21T15:00:38.659 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:38.659 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:38.659 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:38.659 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.659 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.659 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:112735 3.13 3:cd56ff5a:::rbd_data.1136b1194d20.0000000000000002:58 [read 3366912~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:38.660 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.390764+0000", 2024-09-21T15:00:38.660 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 9.2271171459999994, 2024-09-21T15:00:38.660 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055846502999999999, 2024-09-21T15:00:38.660 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:38.660 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:38.660 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:38.660 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:38.660 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:38.660 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 112735 2024-09-21T15:00:38.660 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.660 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:38.660 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.660 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:38.660 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390764+0000", 2024-09-21T15:00:38.661 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:38.661 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.661 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.661 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:38.661 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390764+0000", 2024-09-21T15:00:38.661 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:38.661 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.661 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.661 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:38.661 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390764+0000", 2024-09-21T15:00:38.661 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:00:38.661 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.661 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.661 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:38.661 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390771+0000", 2024-09-21T15:00:38.662 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.1380000000000001e-06 2024-09-21T15:00:38.662 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.662 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.662 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:38.662 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390774+0000", 2024-09-21T15:00:38.662 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1370000000000002e-06 2024-09-21T15:00:38.662 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.662 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.662 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:38.662 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390777+0000", 2024-09-21T15:00:38.662 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7470000000000001e-06 2024-09-21T15:00:38.662 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.662 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.662 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:38.662 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.446488+0000", 2024-09-21T15:00:38.663 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055711016000000002 2024-09-21T15:00:38.663 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.663 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.663 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:38.663 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.446543+0000", 2024-09-21T15:00:38.663 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5148999999999997e-05 2024-09-21T15:00:38.663 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.663 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.663 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:38.663 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.446611+0000", 2024-09-21T15:00:38.663 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7533999999999994e-05 2024-09-21T15:00:38.663 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:38.663 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:38.663 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:38.663 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.663 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.664 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:112736 3.13 3:cd56ff5a:::rbd_data.1136b1194d20.0000000000000002:58 [read 3457024~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:38.664 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.390779+0000", 2024-09-21T15:00:38.664 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 9.2271022019999993, 2024-09-21T15:00:38.664 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.056179989, 2024-09-21T15:00:38.664 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:38.664 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:38.664 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:38.664 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:38.664 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:38.664 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 112736 2024-09-21T15:00:38.664 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.664 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:38.664 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.664 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:38.664 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390779+0000", 2024-09-21T15:00:38.665 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:38.665 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.665 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.665 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:38.665 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390779+0000", 2024-09-21T15:00:38.665 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:38.665 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.665 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.665 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:38.665 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390779+0000", 2024-09-21T15:00:38.665 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:00:38.665 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.665 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.665 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:38.665 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390780+0000", 2024-09-21T15:00:38.666 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4360000000000001e-06 2024-09-21T15:00:38.666 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.666 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.666 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:38.666 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390783+0000", 2024-09-21T15:00:38.666 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.818e-06 2024-09-21T15:00:38.666 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.666 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.666 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:38.666 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390786+0000", 2024-09-21T15:00:38.666 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4439999999999998e-06 2024-09-21T15:00:38.666 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.666 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.666 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:38.666 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.446668+0000", 2024-09-21T15:00:38.666 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055881910999999999 2024-09-21T15:00:38.667 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.667 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.667 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:38.667 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.446797+0000", 2024-09-21T15:00:38.667 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012924599999999999 2024-09-21T15:00:38.667 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.667 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.667 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:38.667 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.446959+0000", 2024-09-21T15:00:38.667 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000162356 2024-09-21T15:00:38.667 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:38.667 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:38.667 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:38.667 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.667 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.668 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:118215 3.d 3:b5b5db2f:::rbd_data.1136b1194d20.0000000000000005:58 [read 5160960~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:38.668 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.833406+0000", 2024-09-21T15:00:38.668 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 8.7844759040000007, 2024-09-21T15:00:38.668 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055458362999999997, 2024-09-21T15:00:38.668 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:38.668 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:38.668 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:38.668 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:38.668 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:38.668 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 118215 2024-09-21T15:00:38.668 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.668 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:38.668 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.668 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:38.668 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833406+0000", 2024-09-21T15:00:38.668 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:38.669 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.669 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.669 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:38.669 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833406+0000", 2024-09-21T15:00:38.669 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:38.669 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.669 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.669 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:38.669 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833405+0000", 2024-09-21T15:00:38.669 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:38.669 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.669 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.669 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:38.669 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833409+0000", 2024-09-21T15:00:38.669 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.8099999999999999e-06 2024-09-21T15:00:38.670 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.670 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.670 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:38.670 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833416+0000", 2024-09-21T15:00:38.670 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3989999999999998e-06 2024-09-21T15:00:38.670 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.670 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.670 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:38.670 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833420+0000", 2024-09-21T15:00:38.670 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7189999999999999e-06 2024-09-21T15:00:38.670 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.670 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.670 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:38.670 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.888444+0000", 2024-09-21T15:00:38.670 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055024311999999999 2024-09-21T15:00:38.671 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.671 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.671 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:38.671 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.888630+0000", 2024-09-21T15:00:38.671 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00018579800000000001 2024-09-21T15:00:38.671 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.671 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.671 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:38.671 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.888864+0000", 2024-09-21T15:00:38.671 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000234009 2024-09-21T15:00:38.671 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:38.671 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:38.671 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:38.671 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.672 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.672 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:118216 3.d 3:b5b5db2f:::rbd_data.1136b1194d20.0000000000000005:58 [read 5251072~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:38.672 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.833425+0000", 2024-09-21T15:00:38.672 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 8.7844564720000005, 2024-09-21T15:00:38.672 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055891158000000003, 2024-09-21T15:00:38.672 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:38.672 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:38.672 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:38.672 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:38.672 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:38.672 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 118216 2024-09-21T15:00:38.672 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.672 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:38.672 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.673 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:38.673 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833425+0000", 2024-09-21T15:00:38.673 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:38.673 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.673 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.673 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:38.673 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833425+0000", 2024-09-21T15:00:38.673 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:38.673 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.673 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.673 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:38.673 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833425+0000", 2024-09-21T15:00:38.673 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:38.673 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.673 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.674 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:38.674 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833427+0000", 2024-09-21T15:00:38.674 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.137e-06 2024-09-21T15:00:38.674 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.674 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.674 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:38.674 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833430+0000", 2024-09-21T15:00:38.674 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5379999999999998e-06 2024-09-21T15:00:38.674 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.674 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.674 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:38.674 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833433+0000", 2024-09-21T15:00:38.674 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0599999999999999e-06 2024-09-21T15:00:38.674 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.674 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.674 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:38.675 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.888919+0000", 2024-09-21T15:00:38.675 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055485381 2024-09-21T15:00:38.675 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.675 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.675 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:38.675 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.889088+0000", 2024-09-21T15:00:38.675 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000169364 2024-09-21T15:00:38.675 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.675 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.675 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:38.675 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.889316+0000", 2024-09-21T15:00:38.675 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00022798600000000001 2024-09-21T15:00:38.675 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:38.675 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:38.675 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:38.676 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.676 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.676 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:118217 3.d 3:b5b5db2f:::rbd_data.1136b1194d20.0000000000000005:58 [read 5341184~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:38.676 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.833436+0000", 2024-09-21T15:00:38.676 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 8.784445045, 2024-09-21T15:00:38.676 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.056045230000000001, 2024-09-21T15:00:38.676 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:38.676 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:38.676 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:38.676 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:38.676 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:38.676 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 118217 2024-09-21T15:00:38.676 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.676 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:38.676 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.676 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:38.677 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833436+0000", 2024-09-21T15:00:38.677 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:38.677 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.677 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.677 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:38.677 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833436+0000", 2024-09-21T15:00:38.677 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:38.677 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.677 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.677 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:38.677 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833436+0000", 2024-09-21T15:00:38.677 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:38.677 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.677 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.677 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:38.678 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833438+0000", 2024-09-21T15:00:38.678 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.826e-06 2024-09-21T15:00:38.678 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.678 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.678 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:38.678 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833441+0000", 2024-09-21T15:00:38.678 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3280000000000002e-06 2024-09-21T15:00:38.678 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.678 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.678 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:38.678 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833447+0000", 2024-09-21T15:00:38.678 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2660000000000001e-06 2024-09-21T15:00:38.678 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.678 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.679 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:38.679 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.889362+0000", 2024-09-21T15:00:38.679 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055915323000000003 2024-09-21T15:00:38.679 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.679 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.679 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:38.679 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.889414+0000", 2024-09-21T15:00:38.679 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1711e-05 2024-09-21T15:00:38.679 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.679 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.679 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:38.679 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.889482+0000", 2024-09-21T15:00:38.679 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8056e-05 2024-09-21T15:00:38.679 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:38.679 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:38.680 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:38.680 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.680 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.680 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:118218 3.d 3:b5b5db2f:::rbd_data.1136b1194d20.0000000000000005:58 [read 5431296~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:38.680 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.833451+0000", 2024-09-21T15:00:38.680 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 8.7844300099999995, 2024-09-21T15:00:38.680 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.056286765000000002, 2024-09-21T15:00:38.680 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:38.680 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:38.680 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:38.680 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:38.680 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:38.680 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 118218 2024-09-21T15:00:38.680 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.680 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:38.681 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.681 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:38.681 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833451+0000", 2024-09-21T15:00:38.681 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:38.681 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.681 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.681 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:38.681 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833451+0000", 2024-09-21T15:00:38.681 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:38.681 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.681 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.681 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:38.681 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833451+0000", 2024-09-21T15:00:38.681 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:38.681 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.681 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.682 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:38.682 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833454+0000", 2024-09-21T15:00:38.682 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9670000000000001e-06 2024-09-21T15:00:38.682 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.682 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.682 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:38.682 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833461+0000", 2024-09-21T15:00:38.682 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0489999999999998e-06 2024-09-21T15:00:38.682 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.682 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.682 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:38.682 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833468+0000", 2024-09-21T15:00:38.682 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0310000000000001e-06 2024-09-21T15:00:38.682 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.683 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.683 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:38.683 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.889539+0000", 2024-09-21T15:00:38.683 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.056071217 2024-09-21T15:00:38.683 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.683 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.683 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:38.683 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.889621+0000", 2024-09-21T15:00:38.683 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1528999999999998e-05 2024-09-21T15:00:38.683 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.683 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.683 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:38.683 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.889738+0000", 2024-09-21T15:00:38.683 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00011759799999999999 2024-09-21T15:00:38.683 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:38.684 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:38.684 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:38.684 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.684 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.684 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:118220 3.d 3:b5b5db2f:::rbd_data.1136b1194d20.0000000000000005:58 [read 5611520~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:38.684 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.834528+0000", 2024-09-21T15:00:38.684 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 8.7833530209999999, 2024-09-21T15:00:38.684 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055660376999999997, 2024-09-21T15:00:38.684 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:38.684 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:38.684 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:38.684 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:38.685 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:38.685 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 118220 2024-09-21T15:00:38.685 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.685 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:38.685 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.685 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:38.685 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834528+0000", 2024-09-21T15:00:38.685 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:38.685 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.685 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.685 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:38.685 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834528+0000", 2024-09-21T15:00:38.685 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:38.686 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.686 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.686 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:38.686 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834528+0000", 2024-09-21T15:00:38.686 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:38.686 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.686 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.686 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:38.686 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834532+0000", 2024-09-21T15:00:38.686 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.9509999999999999e-06 2024-09-21T15:00:38.686 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.686 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.686 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:38.686 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834537+0000", 2024-09-21T15:00:38.687 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6899999999999997e-06 2024-09-21T15:00:38.687 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.687 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.687 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:38.687 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834541+0000", 2024-09-21T15:00:38.687 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.76e-06 2024-09-21T15:00:38.687 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.687 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.687 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:38.687 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.889962+0000", 2024-09-21T15:00:38.687 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055421249999999998 2024-09-21T15:00:38.687 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.687 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.687 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:38.688 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.890074+0000", 2024-09-21T15:00:38.688 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00011124300000000001 2024-09-21T15:00:38.688 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.688 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.688 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:38.688 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.890189+0000", 2024-09-21T15:00:38.688 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000115151 2024-09-21T15:00:38.688 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:38.688 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:38.688 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:38.688 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.688 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.688 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:118221 3.d 3:b5b5db2f:::rbd_data.1136b1194d20.0000000000000005:58 [read 5701632~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:38.688 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.834544+0000", 2024-09-21T15:00:38.689 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 8.7833370990000006, 2024-09-21T15:00:38.689 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.056071875, 2024-09-21T15:00:38.689 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:38.689 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:38.689 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:38.689 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:38.689 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:38.689 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 118221 2024-09-21T15:00:38.689 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.689 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:38.689 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.689 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:38.689 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834544+0000", 2024-09-21T15:00:38.689 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:38.690 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.690 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.690 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:38.690 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834544+0000", 2024-09-21T15:00:38.690 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:38.690 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.690 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.690 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:38.690 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834544+0000", 2024-09-21T15:00:38.690 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:38.690 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.690 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.690 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:38.690 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834546+0000", 2024-09-21T15:00:38.691 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.458e-06 2024-09-21T15:00:38.691 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.691 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.691 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:38.691 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834550+0000", 2024-09-21T15:00:38.691 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.563e-06 2024-09-21T15:00:38.691 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.691 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.691 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:38.691 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834553+0000", 2024-09-21T15:00:38.691 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.253e-06 2024-09-21T15:00:38.691 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.691 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.691 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:38.692 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.890246+0000", 2024-09-21T15:00:38.692 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055692561000000002 2024-09-21T15:00:38.692 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.692 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.692 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:38.692 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.890397+0000", 2024-09-21T15:00:38.692 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015129999999999999 2024-09-21T15:00:38.692 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.692 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.692 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:38.692 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.890616+0000", 2024-09-21T15:00:38.692 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00021912799999999999 2024-09-21T15:00:38.692 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:38.692 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:38.692 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:38.692 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.693 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.693 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:118222 3.d 3:b5b5db2f:::rbd_data.1136b1194d20.0000000000000005:58 [read 5791744~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:38.693 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.834557+0000", 2024-09-21T15:00:38.693 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 8.7833242879999993, 2024-09-21T15:00:38.693 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.056417907000000003, 2024-09-21T15:00:38.693 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:38.693 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:38.693 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:38.693 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:38.693 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:38.693 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 118222 2024-09-21T15:00:38.693 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.693 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:38.693 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.694 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:38.694 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834557+0000", 2024-09-21T15:00:38.694 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:38.694 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.694 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.694 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:38.694 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834557+0000", 2024-09-21T15:00:38.694 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:38.694 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.694 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.694 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:38.694 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834557+0000", 2024-09-21T15:00:38.694 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:38.694 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.694 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.695 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:38.695 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834559+0000", 2024-09-21T15:00:38.695 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.956e-06 2024-09-21T15:00:38.695 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.695 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.695 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:38.695 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834562+0000", 2024-09-21T15:00:38.695 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.518e-06 2024-09-21T15:00:38.695 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.695 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.695 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:38.695 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834566+0000", 2024-09-21T15:00:38.695 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.529e-06 2024-09-21T15:00:38.695 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.695 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.696 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:38.696 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.890671+0000", 2024-09-21T15:00:38.696 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.056105255 2024-09-21T15:00:38.696 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.696 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.696 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:38.696 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.890788+0000", 2024-09-21T15:00:38.696 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00011715899999999999 2024-09-21T15:00:38.696 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.696 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.696 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:38.696 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.890975+0000", 2024-09-21T15:00:38.696 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00018678100000000001 2024-09-21T15:00:38.696 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:38.696 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:38.697 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:38.697 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.697 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.697 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:118223 3.d 3:b5b5db2f:::rbd_data.1136b1194d20.0000000000000005:58 [read 5881856~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:38.697 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.834569+0000", 2024-09-21T15:00:38.697 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 8.7833127310000005, 2024-09-21T15:00:38.697 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.056618784999999998, 2024-09-21T15:00:38.697 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:38.697 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:38.697 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:38.697 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:38.697 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:38.697 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 118223 2024-09-21T15:00:38.697 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.697 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:38.698 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.698 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:38.698 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834569+0000", 2024-09-21T15:00:38.698 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:38.698 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.698 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.698 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:38.698 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834569+0000", 2024-09-21T15:00:38.698 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:38.698 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.698 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.698 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:38.698 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834568+0000", 2024-09-21T15:00:38.699 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:38.699 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.699 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.699 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:38.699 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834570+0000", 2024-09-21T15:00:38.699 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8309999999999999e-06 2024-09-21T15:00:38.699 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.699 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.699 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:38.699 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834574+0000", 2024-09-21T15:00:38.699 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3720000000000001e-06 2024-09-21T15:00:38.699 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.699 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.699 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:38.700 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834577+0000", 2024-09-21T15:00:38.700 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3019999999999999e-06 2024-09-21T15:00:38.700 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.700 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.700 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:38.700 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.891014+0000", 2024-09-21T15:00:38.700 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.056437322999999998 2024-09-21T15:00:38.700 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.700 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.700 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:38.700 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.891101+0000", 2024-09-21T15:00:38.700 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6341999999999994e-05 2024-09-21T15:00:38.700 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.700 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.700 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:38.701 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.891187+0000", 2024-09-21T15:00:38.701 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6889000000000001e-05 2024-09-21T15:00:38.701 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:38.701 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:38.701 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:38.701 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.701 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.701 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:118224 3.d 3:b5b5db2f:::rbd_data.1136b1194d20.0000000000000005:58 [read 5971968~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:38.701 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.834580+0000", 2024-09-21T15:00:38.701 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 8.7833017640000008, 2024-09-21T15:00:38.701 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.056770289000000002, 2024-09-21T15:00:38.701 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:38.701 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:38.701 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:38.702 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:38.702 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:38.702 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 118224 2024-09-21T15:00:38.702 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.702 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:38.702 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.702 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:38.702 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834580+0000", 2024-09-21T15:00:38.702 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:38.702 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.702 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.702 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:38.702 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834580+0000", 2024-09-21T15:00:38.702 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:38.703 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.703 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.703 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:38.703 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834579+0000", 2024-09-21T15:00:38.703 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:38.703 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.703 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.703 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:38.703 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834581+0000", 2024-09-21T15:00:38.703 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8160000000000001e-06 2024-09-21T15:00:38.703 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.703 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.703 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:38.704 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834585+0000", 2024-09-21T15:00:38.704 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.568e-06 2024-09-21T15:00:38.704 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.704 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.704 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:38.704 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834588+0000", 2024-09-21T15:00:38.704 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1810000000000001e-06 2024-09-21T15:00:38.704 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.704 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.704 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:38.704 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.891216+0000", 2024-09-21T15:00:38.704 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.056628127 2024-09-21T15:00:38.704 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.704 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.705 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:38.705 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.891273+0000", 2024-09-21T15:00:38.705 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7364000000000002e-05 2024-09-21T15:00:38.705 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.705 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.705 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:38.705 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.891350+0000", 2024-09-21T15:00:38.705 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6509999999999998e-05 2024-09-21T15:00:38.705 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:38.705 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:38.705 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:38.705 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.705 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.705 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:118225 3.d 3:b5b5db2f:::rbd_data.1136b1194d20.0000000000000005:58 [read 6062080~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:38.705 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.834591+0000", 2024-09-21T15:00:38.706 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 8.7832907529999993, 2024-09-21T15:00:38.706 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.056883207999999998, 2024-09-21T15:00:38.706 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:38.706 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:38.706 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:38.706 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:38.706 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:38.706 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 118225 2024-09-21T15:00:38.706 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.706 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:38.706 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.706 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:38.706 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834591+0000", 2024-09-21T15:00:38.706 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:38.707 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.707 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.707 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:38.707 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834591+0000", 2024-09-21T15:00:38.707 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:38.707 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.707 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.707 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:38.707 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834590+0000", 2024-09-21T15:00:38.707 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:38.707 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.707 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.707 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:38.707 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834592+0000", 2024-09-21T15:00:38.707 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8810000000000001e-06 2024-09-21T15:00:38.708 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.708 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.708 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:38.708 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834596+0000", 2024-09-21T15:00:38.708 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.439e-06 2024-09-21T15:00:38.708 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.708 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.708 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:38.708 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834599+0000", 2024-09-21T15:00:38.708 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0740000000000001e-06 2024-09-21T15:00:38.708 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.708 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.708 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:38.708 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.891369+0000", 2024-09-21T15:00:38.709 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.056769872999999998 2024-09-21T15:00:38.709 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.709 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.709 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:38.709 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.891415+0000", 2024-09-21T15:00:38.709 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6257000000000001e-05 2024-09-21T15:00:38.709 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:38.709 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:38.709 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:38.709 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.891474+0000", 2024-09-21T15:00:38.709 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8964e-05 2024-09-21T15:00:38.709 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:38.709 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:38.709 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:38.710 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:38.710 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:38.710 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:38.710 INFO:teuthology.orchestra.run.smithi032.stdout:4196 write 0x94ba566 thru 0x94bc9ad (0x2448 bytes) 2024-09-21T15:00:38.711 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T15:00:38.712 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:38.712 INFO:teuthology.orchestra.run.smithi032.stdout:4198 trunc from 0x94bc9ae to 0x21edbb8 2024-09-21T15:00:38.712 INFO:teuthology.orchestra.run.smithi032.stdout:4199 trunc from 0x21edbb8 to 0x175045d 2024-09-21T15:00:38.712 INFO:teuthology.orchestra.run.smithi032.stdout:4200 read 0xebf24e thru 0xec2da3 (0x3b56 bytes) 2024-09-21T15:00:38.713 INFO:teuthology.orchestra.run.smithi032.stdout:4201 punch from 0x141085e to 0x141ea50, (0xe1f2 bytes) 2024-09-21T15:00:38.713 INFO:teuthology.orchestra.run.smithi032.stdout:4202 punch from 0x127eba1 to 0x1287cab, (0x910a bytes) 2024-09-21T15:00:38.713 INFO:teuthology.orchestra.run.smithi032.stdout:4203 write 0x976c498 thru 0x9779d76 (0xd8df bytes) 2024-09-21T15:00:38.713 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:00:38.707+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:00:38.713 INFO:teuthology.orchestra.run.smithi032.stdout:4205 write 0x70743ef thru 0x707b06b (0x6c7d bytes) 2024-09-21T15:00:38.745 INFO:teuthology.orchestra.run.smithi032.stdout:4206 writesame 0xd8902a0 thru 0xd898dd5 data_size 0x67(0x8b36 bytes) 2024-09-21T15:00:38.745 INFO:teuthology.orchestra.run.smithi032.stdout:4208 punch from 0xd1d182 to 0xd2a894, (0xd712 bytes) 2024-09-21T15:00:38.746 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:38.746 INFO:teuthology.orchestra.run.smithi032.stdout:4211 write 0x79416e6 thru 0x794446a (0x2d85 bytes) 2024-09-21T15:00:38.769 INFO:teuthology.orchestra.run.smithi032.stdout:4212 writesame 0xeb9846b thru 0xeb9dd9d data_size 0x1(0x5933 bytes) 2024-09-21T15:00:38.769 INFO:teuthology.orchestra.run.smithi032.stdout:4213 punch from 0xbc09dbe to 0xbc0f3be, (0x5600 bytes) 2024-09-21T15:00:38.782 INFO:teuthology.orchestra.run.smithi032.stdout:4216 write 0xec178e4 thru 0xec1da52 (0x616f bytes) 2024-09-21T15:00:38.791 INFO:teuthology.orchestra.run.smithi032.stdout:4217 read 0x9db4a5d thru 0x9dc17b1 (0xcd55 bytes) 2024-09-21T15:00:38.791 INFO:teuthology.orchestra.run.smithi032.stdout:4218 read 0x646e49e thru 0x64781ab (0x9d0e bytes) 2024-09-21T15:00:38.791 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:38.791 INFO:teuthology.orchestra.run.smithi032.stdout:4221 punch from 0xdb5af1a to 0xdb5f172, (0x4258 bytes) 2024-09-21T15:00:38.791 INFO:teuthology.orchestra.run.smithi032.stdout:4223 write 0x637516e thru 0x637884f (0x36e2 bytes) 2024-09-21T15:00:38.799 INFO:teuthology.orchestra.run.smithi032.stdout:4224 read 0x345827d thru 0x345d2de (0x5062 bytes) 2024-09-21T15:00:38.799 INFO:teuthology.orchestra.run.smithi032.stdout:4225 write 0x8944924 thru 0x894e79b (0x9e78 bytes) 2024-09-21T15:00:38.808 INFO:teuthology.orchestra.run.smithi032.stdout:4226 read 0xd43e9e3 thru 0xd440532 (0x1b50 bytes) 2024-09-21T15:00:38.808 INFO:teuthology.orchestra.run.smithi032.stdout:4228 read 0x71af85e thru 0x71b0c08 (0x13ab bytes) 2024-09-21T15:00:38.808 INFO:teuthology.orchestra.run.smithi032.stdout:4229 punch from 0x7a46908 to 0x7a556e9, (0xede1 bytes) 2024-09-21T15:00:38.808 INFO:teuthology.orchestra.run.smithi032.stdout:4230 clone 31 order 21 su 524288 sc 3 2024-09-21T15:00:38.811 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:00:38.807+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:00:38.825 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:38.825 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:00:38.826 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T15:00:38.826 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:38.835 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T15:00:38.912 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:00:38.907+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:00:38.942 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:38.942 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:00:38.942 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T15:00:38.942 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T15:00:38.942 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:38.951 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T15:00:39.013 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:00:39.011+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:00:39.059 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:39.059 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T15:00:39.059 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T15:00:39.060 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T15:00:39.060 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.060 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:63820 3.19 3.626bc079 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-21T15:00:39.060 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:27.241293+0000", 2024-09-21T15:00:39.060 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 71.814173619000002, 2024-09-21T15:00:39.060 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.98723808199999996, 2024-09-21T15:00:39.060 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:39.060 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:39.060 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:39.060 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:39.060 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:39.060 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 63820 2024-09-21T15:00:39.060 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.060 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:39.060 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.061 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:39.061 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241293+0000", 2024-09-21T15:00:39.061 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.061 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.061 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.061 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:39.061 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241293+0000", 2024-09-21T15:00:39.061 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.061 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.061 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.061 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:39.061 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241291+0000", 2024-09-21T15:00:39.061 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:00:39.061 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.061 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.061 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:39.062 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241299+0000", 2024-09-21T15:00:39.062 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1529999999999997e-06 2024-09-21T15:00:39.062 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.062 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.062 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:39.062 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241317+0000", 2024-09-21T15:00:39.062 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7764000000000001e-05 2024-09-21T15:00:39.062 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.062 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.062 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:39.062 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241328+0000", 2024-09-21T15:00:39.062 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1245e-05 2024-09-21T15:00:39.062 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.062 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.063 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:39.063 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241407+0000", 2024-09-21T15:00:39.063 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8762000000000002e-05 2024-09-21T15:00:39.063 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.063 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.063 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:00:39.063 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241418+0000", 2024-09-21T15:00:39.063 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1695000000000001e-05 2024-09-21T15:00:39.063 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.063 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.063 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:39.063 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:28.228510+0000", 2024-09-21T15:00:39.063 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.98709166100000001 2024-09-21T15:00:39.063 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.063 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.064 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:39.064 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:28.228531+0000", 2024-09-21T15:00:39.064 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0696e-05 2024-09-21T15:00:39.064 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.064 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:39.064 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.064 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.064 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.064 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:97734 3.19 3:9fcf754f:::rbd_data.1136bfc79a70.0000000000000009:head [write 0~4194304 [fadvise_sequential+fadvise_nocache],write 4784128~147456 [fadvise_sequential+fadvise_nocache],write 4997120~147456 [fadvise_sequential+fadvise_nocache],write 5210112~32768 [fadvise_sequential+fadvise_nocache],write 5292032~147456 [fadvise_sequential+fadvise_nocache],write 5505024~147456 [fadvise_sequential+fadvise_nocache],write 5718016~49152 [fadvise_sequential+fadvise_nocache],write 5799936~147456 [fadvise_sequential+fadvise_nocache],write 6012928~147456 [fadvise_sequential+fadvise_nocache],write 6225920~65536 [fadvise_sequential+fadvise_nocache],write 6307840~147456 [fadvise_sequential+fadvise_nocache],write 6520832~147456 [fadvise_sequential+fadvise_nocache],write 6733824~229376 [fadvise_sequential+fadvise_nocache],write 7028736~147456 [fadvise_sequential+fadvise_nocache],write 7241728~229376 [fadvise_sequential+fadvise_nocache],write 7536640~147456 [fadvise_sequential+fadvise_nocache],write 7749632~229376 [fadvise_sequential+fadvise_nocache],write 8077312~114688 [fadvise_sequential+fadvise_nocache],write 8290304~98304 [fadvise_sequential+fadvise_nocache],write 8404992~81920 [fadvise_sequential+fadvise_nocache],write 8617984~81920 [fadvise_sequential+fadvise_nocache],write 8830976~81920 [fadvise_sequential+fadvise_nocache],write 8962048~32768 [fadvise_sequential+fadvise_nocache],write 9175040~32768 [fadvise_sequential+fadvise_nocache],write 9388032~32768 [fadvise_sequential+fadvise_nocache],write 9469952~32768 [fadvise_sequential+fadvise_nocache],write 9682944~20092 [fadvise_sequential+fadvise_nocache],write 9961472~163840 [fadvise_sequential+fadvise_nocache],write 10141696~196608 [fadvise_sequential+fadvise_nocache],write 10354688~278528 [fadvise_sequential+fadvise_nocache],write 10649600~196608 [fadvise_sequential+fadvise_nocache],write 10862592~196608 [fadvise_sequential+fadvise_nocache],write 11075584~65536 [fadvise_sequential+fadvise_nocache],write 11173888~98304 [fadvise_sequential+fadvise_nocache],write 11288576~65536 [fadvise_sequential+fadvise_nocache],write 11386880~98304 [fadvise_sequential+fadvise_nocache],write 11501568~65536 [fadvise_sequential+fadvise_nocache],write 11583488~65536 [fadvise_sequential+fadvise_nocache],write 11681792~98304 [fadvise_sequential+fadvise_nocache],write 11796480~65536 [fadvise_sequential+fadvise_nocache],write 11894784~98304 [fadvise_sequential+fadvise_nocache],write 12009472~65536 [fadvise_sequential+fadvise_nocache],write 12189696~98304 [fadvise_sequential+fadvise_nocache],write 12402688~98304 [fadvise_sequential+fadvise_nocache],write 12697600~98304 [fadvise_sequential+fadvise_nocache],write 12910592~98304 [fadvise_sequential+fadvise_nocache],write 13205504~98304 [fadvise_sequential+fadvise_nocache],write 13418496~98304 [fadvise_sequential+fadvise_nocache],write 13713408~98304 [fadvise_sequential+fadvise_nocache],write 13926400~98304 [fadvise_sequential+fadvise_nocache],write 14139392~16384 [fadvise_sequential+fadvise_nocache],write 14680064~16384 [fadvise_sequential+fadvise_nocache],write 14827520~81920 [fadvise_sequential+fadvise_nocache],write 15040512~81920 [fadvise_sequential+fadvise_nocache],write 15335424~81920 [fadvise_sequential+fadvise_nocache],write 15548416~81920 [fadvise_sequential+fadvise_nocache],write 15843328~81920 [fadvise_sequential+fadvise_nocache],write 16056320~81920 [fadvise_sequential+fadvise_nocache],write 16351232~81920 [fadvise_sequential+fadvise_nocache],write 16564224~81920 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:39.064 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.354331+0000", 2024-09-21T15:00:39.064 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 33.701135571999998, 2024-09-21T15:00:39.064 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.086284870999999999, 2024-09-21T15:00:39.064 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:39.064 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:39.064 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:39.065 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:39.065 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:39.065 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97734 2024-09-21T15:00:39.065 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.065 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:39.065 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.065 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:39.065 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.354331+0000", 2024-09-21T15:00:39.065 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.065 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.065 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.065 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:39.065 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.354331+0000", 2024-09-21T15:00:39.065 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.065 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.065 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.066 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:39.066 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.354329+0000", 2024-09-21T15:00:39.066 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:00:39.066 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.066 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.066 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:39.066 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.370234+0000", 2024-09-21T15:00:39.066 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015905348999999999 2024-09-21T15:00:39.066 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.066 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.066 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:39.066 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.370286+0000", 2024-09-21T15:00:39.066 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1656999999999997e-05 2024-09-21T15:00:39.067 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.067 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.067 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:39.067 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.370315+0000", 2024-09-21T15:00:39.067 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9204e-05 2024-09-21T15:00:39.067 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.067 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.067 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:39.067 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.370395+0000", 2024-09-21T15:00:39.067 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0395999999999994e-05 2024-09-21T15:00:39.067 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.067 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.067 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:39.067 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.370604+0000", 2024-09-21T15:00:39.067 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000208684 2024-09-21T15:00:39.068 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.068 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.068 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 1", 2024-09-21T15:00:39.068 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.371701+0000", 2024-09-21T15:00:39.068 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0010973140000000001 2024-09-21T15:00:39.068 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.068 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.068 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:39.068 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.405485+0000", 2024-09-21T15:00:39.068 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.033783615000000003 2024-09-21T15:00:39.068 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.068 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.068 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:39.068 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.440211+0000", 2024-09-21T15:00:39.068 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034725757000000003 2024-09-21T15:00:39.068 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.069 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.069 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:39.069 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.440497+0000", 2024-09-21T15:00:39.069 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00028673099999999998 2024-09-21T15:00:39.069 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.069 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.069 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:39.069 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.440616+0000", 2024-09-21T15:00:39.069 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000118256 2024-09-21T15:00:39.069 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.069 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:39.069 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.069 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.069 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.069 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112831 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 1515520~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:39.069 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.406761+0000", 2024-09-21T15:00:39.070 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.6487048739999999, 2024-09-21T15:00:39.070 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060265666000000002, 2024-09-21T15:00:39.070 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:39.070 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:39.070 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:39.070 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:39.070 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:39.070 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112831 2024-09-21T15:00:39.070 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.070 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:39.070 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.070 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:39.070 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406761+0000", 2024-09-21T15:00:39.071 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.071 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.071 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.071 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:39.071 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406761+0000", 2024-09-21T15:00:39.071 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.071 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.071 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.071 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:39.071 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406761+0000", 2024-09-21T15:00:39.071 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:39.071 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.071 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.071 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:39.071 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406764+0000", 2024-09-21T15:00:39.072 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0319999999999999e-06 2024-09-21T15:00:39.072 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.072 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.072 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:39.072 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406776+0000", 2024-09-21T15:00:39.072 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2357999999999999e-05 2024-09-21T15:00:39.072 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.072 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.072 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:39.072 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406784+0000", 2024-09-21T15:00:39.072 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6059999999999997e-06 2024-09-21T15:00:39.072 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.072 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.072 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:39.073 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.466595+0000", 2024-09-21T15:00:39.073 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059811062999999998 2024-09-21T15:00:39.073 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.073 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.073 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:39.073 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.466782+0000", 2024-09-21T15:00:39.073 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000186622 2024-09-21T15:00:39.073 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.073 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.073 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:39.073 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.467027+0000", 2024-09-21T15:00:39.073 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000245455 2024-09-21T15:00:39.073 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.073 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:39.073 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.074 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.074 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.074 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112832 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 1605632~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:39.074 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.406788+0000", 2024-09-21T15:00:39.074 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.6486782840000007, 2024-09-21T15:00:39.074 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060452065999999999, 2024-09-21T15:00:39.074 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:39.074 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:39.074 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:39.074 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:39.074 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:39.074 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112832 2024-09-21T15:00:39.074 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.075 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:39.075 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.075 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:39.075 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406788+0000", 2024-09-21T15:00:39.075 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.075 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.075 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.075 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:39.075 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406788+0000", 2024-09-21T15:00:39.075 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.075 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.075 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.075 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:39.075 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406788+0000", 2024-09-21T15:00:39.075 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:39.076 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.076 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.076 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:39.076 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406798+0000", 2024-09-21T15:00:39.076 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0096e-05 2024-09-21T15:00:39.076 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.076 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.076 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:39.076 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406803+0000", 2024-09-21T15:00:39.076 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2020000000000003e-06 2024-09-21T15:00:39.076 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.076 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.076 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:39.076 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406808+0000", 2024-09-21T15:00:39.076 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8239999999999999e-06 2024-09-21T15:00:39.076 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.077 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.077 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:39.077 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.467049+0000", 2024-09-21T15:00:39.077 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060241754000000002 2024-09-21T15:00:39.077 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.077 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.077 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:39.077 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.467134+0000", 2024-09-21T15:00:39.077 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4216999999999997e-05 2024-09-21T15:00:39.077 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.077 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.077 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:39.077 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.467240+0000", 2024-09-21T15:00:39.077 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000106304 2024-09-21T15:00:39.077 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.077 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:39.078 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.078 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.078 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.078 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112843 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 2613248~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:39.078 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.410273+0000", 2024-09-21T15:00:39.078 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.6451936899999993, 2024-09-21T15:00:39.078 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059854931, 2024-09-21T15:00:39.078 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:39.078 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:39.078 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:39.078 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:39.078 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:39.078 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112843 2024-09-21T15:00:39.078 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.078 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:39.079 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.079 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:39.079 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410273+0000", 2024-09-21T15:00:39.079 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.079 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.079 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.079 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:39.079 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410273+0000", 2024-09-21T15:00:39.079 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.079 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.079 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.079 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:39.079 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410272+0000", 2024-09-21T15:00:39.079 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:39.079 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.079 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.080 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:39.080 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410274+0000", 2024-09-21T15:00:39.080 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.905e-06 2024-09-21T15:00:39.080 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.080 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.080 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:39.080 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410283+0000", 2024-09-21T15:00:39.080 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3440000000000007e-06 2024-09-21T15:00:39.080 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.080 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.080 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:39.080 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410294+0000", 2024-09-21T15:00:39.080 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0271e-05 2024-09-21T15:00:39.080 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.080 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.081 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:39.081 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.469917+0000", 2024-09-21T15:00:39.081 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059623738000000003 2024-09-21T15:00:39.081 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.081 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.081 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:39.081 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470002+0000", 2024-09-21T15:00:39.081 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4270999999999993e-05 2024-09-21T15:00:39.081 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.081 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.081 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:39.081 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470127+0000", 2024-09-21T15:00:39.081 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000125711 2024-09-21T15:00:39.081 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.081 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:39.081 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.082 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.082 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.082 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112844 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 2703360~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:39.082 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.410326+0000", 2024-09-21T15:00:39.082 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.6451399700000007, 2024-09-21T15:00:39.082 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060086056999999998, 2024-09-21T15:00:39.082 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:39.082 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:39.082 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:39.082 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:39.082 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:39.082 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112844 2024-09-21T15:00:39.082 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.082 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:39.082 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.083 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:39.083 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410326+0000", 2024-09-21T15:00:39.083 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.083 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.083 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.083 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:39.083 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410326+0000", 2024-09-21T15:00:39.083 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.083 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.083 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.083 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:39.083 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410326+0000", 2024-09-21T15:00:39.083 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:00:39.083 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.083 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.084 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:39.084 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410352+0000", 2024-09-21T15:00:39.084 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6872999999999998e-05 2024-09-21T15:00:39.084 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.084 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.084 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:39.084 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410374+0000", 2024-09-21T15:00:39.084 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1141e-05 2024-09-21T15:00:39.084 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.084 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.084 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:39.084 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410385+0000", 2024-09-21T15:00:39.084 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1456e-05 2024-09-21T15:00:39.084 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.084 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.085 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:39.085 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470174+0000", 2024-09-21T15:00:39.085 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059788574999999997 2024-09-21T15:00:39.085 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.085 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.085 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:39.085 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470277+0000", 2024-09-21T15:00:39.085 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000103838 2024-09-21T15:00:39.085 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.085 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.085 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:39.085 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470412+0000", 2024-09-21T15:00:39.085 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013489700000000001 2024-09-21T15:00:39.085 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.085 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:39.086 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.086 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.086 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.086 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112845 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 2793472~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:39.086 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.410394+0000", 2024-09-21T15:00:39.086 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.6450724960000009, 2024-09-21T15:00:39.086 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060244173999999998, 2024-09-21T15:00:39.086 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:39.086 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:39.086 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:39.086 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:39.086 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:39.086 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112845 2024-09-21T15:00:39.087 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.087 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:39.087 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.087 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:39.087 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410394+0000", 2024-09-21T15:00:39.087 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.087 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.087 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.087 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:39.087 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410394+0000", 2024-09-21T15:00:39.087 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.087 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.087 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.087 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:39.088 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410393+0000", 2024-09-21T15:00:39.088 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:39.088 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.088 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.088 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:39.088 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410397+0000", 2024-09-21T15:00:39.088 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9430000000000002e-06 2024-09-21T15:00:39.088 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.088 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.089 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:39.090 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410406+0000", 2024-09-21T15:00:39.090 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.799e-06 2024-09-21T15:00:39.090 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.090 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.090 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:39.090 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410418+0000", 2024-09-21T15:00:39.090 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2225999999999999e-05 2024-09-21T15:00:39.090 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.090 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.090 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:39.090 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470444+0000", 2024-09-21T15:00:39.090 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060026311999999998 2024-09-21T15:00:39.090 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.091 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.091 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:39.091 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470535+0000", 2024-09-21T15:00:39.091 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0173999999999994e-05 2024-09-21T15:00:39.091 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.091 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.091 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:39.091 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470638+0000", 2024-09-21T15:00:39.091 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010333 2024-09-21T15:00:39.091 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.091 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:39.091 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.091 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.091 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.091 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112846 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 2883584~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:39.092 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.410425+0000", 2024-09-21T15:00:39.092 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.6450412829999994, 2024-09-21T15:00:39.092 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060459247000000001, 2024-09-21T15:00:39.092 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:39.092 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:39.092 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:39.092 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:39.092 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:39.092 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112846 2024-09-21T15:00:39.092 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.092 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:39.092 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.092 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:39.092 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410425+0000", 2024-09-21T15:00:39.093 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.093 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.093 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.093 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:39.093 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410425+0000", 2024-09-21T15:00:39.093 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.093 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.093 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.093 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:39.093 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410424+0000", 2024-09-21T15:00:39.093 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:39.093 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.093 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.093 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:39.093 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410428+0000", 2024-09-21T15:00:39.094 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4709999999999999e-06 2024-09-21T15:00:39.094 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.094 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.094 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:39.094 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410446+0000", 2024-09-21T15:00:39.094 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8354000000000001e-05 2024-09-21T15:00:39.094 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.094 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.094 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:39.094 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410456+0000", 2024-09-21T15:00:39.094 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6709999999999994e-06 2024-09-21T15:00:39.094 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.094 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.094 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:39.095 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470661+0000", 2024-09-21T15:00:39.095 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060205303000000002 2024-09-21T15:00:39.095 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.095 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.095 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:39.095 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470756+0000", 2024-09-21T15:00:39.095 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4401999999999995e-05 2024-09-21T15:00:39.095 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.095 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.095 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:39.095 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470884+0000", 2024-09-21T15:00:39.095 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012860500000000001 2024-09-21T15:00:39.095 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.095 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:39.096 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.096 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.096 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.096 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112847 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 2973696~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:39.096 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.410462+0000", 2024-09-21T15:00:39.096 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.6450042390000004, 2024-09-21T15:00:39.096 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060653836000000003, 2024-09-21T15:00:39.096 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:39.096 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:39.096 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:39.096 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:39.096 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:39.096 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112847 2024-09-21T15:00:39.096 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.097 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:39.097 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.097 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:39.097 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410462+0000", 2024-09-21T15:00:39.097 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.097 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.097 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.097 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:39.097 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410462+0000", 2024-09-21T15:00:39.097 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.097 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.097 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.097 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:39.097 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410461+0000", 2024-09-21T15:00:39.097 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:39.098 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.098 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.098 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:39.098 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410465+0000", 2024-09-21T15:00:39.098 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6899999999999998e-06 2024-09-21T15:00:39.098 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.098 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.098 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:39.098 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410474+0000", 2024-09-21T15:00:39.098 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4179999999999997e-06 2024-09-21T15:00:39.098 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.098 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.098 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:39.098 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410508+0000", 2024-09-21T15:00:39.098 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4282999999999997e-05 2024-09-21T15:00:39.099 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.099 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.099 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:39.099 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470922+0000", 2024-09-21T15:00:39.099 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060413820999999999 2024-09-21T15:00:39.099 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.099 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.099 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:39.099 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471008+0000", 2024-09-21T15:00:39.099 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6379999999999996e-05 2024-09-21T15:00:39.099 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.099 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.099 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:39.099 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471116+0000", 2024-09-21T15:00:39.099 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000107806 2024-09-21T15:00:39.100 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.100 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:39.100 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.100 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.100 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.100 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112848 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 3063808~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:39.100 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.410525+0000", 2024-09-21T15:00:39.100 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.6449409559999992, 2024-09-21T15:00:39.100 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060880735999999998, 2024-09-21T15:00:39.100 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:39.100 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:39.100 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:39.100 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:39.100 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:39.100 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112848 2024-09-21T15:00:39.101 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.101 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:39.101 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.101 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:39.101 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410525+0000", 2024-09-21T15:00:39.101 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.101 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.101 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.101 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:39.101 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410525+0000", 2024-09-21T15:00:39.101 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.101 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.101 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.101 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:39.101 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410525+0000", 2024-09-21T15:00:39.102 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:39.102 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.102 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.102 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:39.102 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410529+0000", 2024-09-21T15:00:39.102 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1760000000000003e-06 2024-09-21T15:00:39.102 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.102 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.102 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:39.102 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410551+0000", 2024-09-21T15:00:39.102 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2099999999999998e-05 2024-09-21T15:00:39.102 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.102 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.102 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:39.102 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410571+0000", 2024-09-21T15:00:39.103 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0159000000000001e-05 2024-09-21T15:00:39.103 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.103 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.103 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:39.103 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471164+0000", 2024-09-21T15:00:39.103 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.06059287 2024-09-21T15:00:39.103 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.103 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.103 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:39.103 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471271+0000", 2024-09-21T15:00:39.103 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000106635 2024-09-21T15:00:39.103 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.103 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.103 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:39.104 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471406+0000", 2024-09-21T15:00:39.104 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000135399 2024-09-21T15:00:39.104 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.104 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:39.104 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.104 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.104 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.104 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112849 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 3153920~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:39.104 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.410579+0000", 2024-09-21T15:00:39.104 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.6448876610000003, 2024-09-21T15:00:39.104 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.061057304999999999, 2024-09-21T15:00:39.104 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:39.104 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:39.104 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:39.105 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:39.105 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:39.105 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112849 2024-09-21T15:00:39.105 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.105 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:39.105 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.105 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:39.105 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410579+0000", 2024-09-21T15:00:39.105 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.105 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.105 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.105 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:39.105 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410579+0000", 2024-09-21T15:00:39.105 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.106 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.106 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.106 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:39.106 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410578+0000", 2024-09-21T15:00:39.106 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:39.106 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.106 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.106 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:39.106 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410582+0000", 2024-09-21T15:00:39.106 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.036e-06 2024-09-21T15:00:39.106 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.106 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.106 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:39.106 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410592+0000", 2024-09-21T15:00:39.106 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9850000000000001e-06 2024-09-21T15:00:39.107 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.107 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.107 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:39.107 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410604+0000", 2024-09-21T15:00:39.107 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2051e-05 2024-09-21T15:00:39.107 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.107 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.107 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:39.107 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471444+0000", 2024-09-21T15:00:39.107 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060839722999999998 2024-09-21T15:00:39.107 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.107 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.107 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:39.107 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471530+0000", 2024-09-21T15:00:39.108 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6359000000000004e-05 2024-09-21T15:00:39.108 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.108 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.108 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:39.108 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471636+0000", 2024-09-21T15:00:39.108 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010576700000000001 2024-09-21T15:00:39.108 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.108 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:39.108 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.108 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.108 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.108 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112850 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 3244032~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:39.108 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.410698+0000", 2024-09-21T15:00:39.108 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.6447683190000006, 2024-09-21T15:00:39.109 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.061164118000000003, 2024-09-21T15:00:39.109 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:39.109 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:39.109 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:39.109 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:39.109 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:39.109 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112850 2024-09-21T15:00:39.109 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.109 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:39.109 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.109 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:39.109 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410698+0000", 2024-09-21T15:00:39.109 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.109 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.110 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.110 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:39.110 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410698+0000", 2024-09-21T15:00:39.110 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.110 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.110 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.110 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:39.110 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410610+0000", 2024-09-21T15:00:39.110 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999123 2024-09-21T15:00:39.110 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.110 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.110 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:39.110 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410707+0000", 2024-09-21T15:00:39.111 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7093e-05 2024-09-21T15:00:39.111 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.111 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.111 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:39.111 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410728+0000", 2024-09-21T15:00:39.111 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0275000000000002e-05 2024-09-21T15:00:39.111 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.111 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.111 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:39.111 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410740+0000", 2024-09-21T15:00:39.111 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2561999999999999e-05 2024-09-21T15:00:39.111 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.111 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.111 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:39.111 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471664+0000", 2024-09-21T15:00:39.112 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060923979000000003 2024-09-21T15:00:39.112 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.112 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.112 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:39.112 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471753+0000", 2024-09-21T15:00:39.112 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8265999999999998e-05 2024-09-21T15:00:39.112 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.112 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.112 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:39.112 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471862+0000", 2024-09-21T15:00:39.112 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000109474 2024-09-21T15:00:39.112 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.112 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:39.112 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.112 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.113 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.113 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112864 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 4521984~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:39.113 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.414789+0000", 2024-09-21T15:00:39.113 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.6406772939999996, 2024-09-21T15:00:39.113 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059845903999999998, 2024-09-21T15:00:39.113 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:39.113 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:39.113 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:39.113 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:39.113 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:39.113 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112864 2024-09-21T15:00:39.113 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.113 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:39.113 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.113 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:39.114 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414789+0000", 2024-09-21T15:00:39.114 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.114 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.114 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.114 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:39.114 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414789+0000", 2024-09-21T15:00:39.114 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.114 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.114 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.114 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:39.114 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414788+0000", 2024-09-21T15:00:39.114 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:39.114 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.114 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.114 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:39.115 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414790+0000", 2024-09-21T15:00:39.115 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.995e-06 2024-09-21T15:00:39.115 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.115 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.115 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:39.115 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414801+0000", 2024-09-21T15:00:39.115 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.01e-05 2024-09-21T15:00:39.115 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.115 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.115 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:39.115 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414804+0000", 2024-09-21T15:00:39.115 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.563e-06 2024-09-21T15:00:39.115 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.115 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.116 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:39.116 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.474499+0000", 2024-09-21T15:00:39.116 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059695208 2024-09-21T15:00:39.116 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.116 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.116 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:39.116 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.474561+0000", 2024-09-21T15:00:39.116 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1672000000000002e-05 2024-09-21T15:00:39.116 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.116 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.116 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:39.116 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.474635+0000", 2024-09-21T15:00:39.116 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3792000000000001e-05 2024-09-21T15:00:39.117 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.117 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:39.117 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.117 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.117 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.117 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112865 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 4612096~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:39.117 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.414807+0000", 2024-09-21T15:00:39.117 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.6406588979999999, 2024-09-21T15:00:39.117 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060003506999999998, 2024-09-21T15:00:39.117 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:39.117 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:39.117 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:39.117 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:39.117 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:39.118 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112865 2024-09-21T15:00:39.118 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.118 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:39.118 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.118 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:39.118 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414807+0000", 2024-09-21T15:00:39.118 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.118 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.118 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.118 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:39.118 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414807+0000", 2024-09-21T15:00:39.118 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.118 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.119 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.119 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:39.119 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414807+0000", 2024-09-21T15:00:39.119 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:39.119 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.119 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.119 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:39.119 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414809+0000", 2024-09-21T15:00:39.119 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0509999999999999e-06 2024-09-21T15:00:39.119 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.119 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.119 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:39.119 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414813+0000", 2024-09-21T15:00:39.119 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0960000000000003e-06 2024-09-21T15:00:39.120 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.120 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.120 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:39.120 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414817+0000", 2024-09-21T15:00:39.120 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.296e-06 2024-09-21T15:00:39.120 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.120 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.120 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:39.120 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.474671+0000", 2024-09-21T15:00:39.120 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059853663000000001 2024-09-21T15:00:39.120 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.120 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.120 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:39.121 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.474735+0000", 2024-09-21T15:00:39.121 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3653999999999997e-05 2024-09-21T15:00:39.121 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.121 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.121 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:39.121 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.474811+0000", 2024-09-21T15:00:39.121 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6074000000000004e-05 2024-09-21T15:00:39.121 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.121 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:39.121 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.121 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.121 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.121 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112866 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 4702208~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:39.121 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.414820+0000", 2024-09-21T15:00:39.121 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.6406459780000002, 2024-09-21T15:00:39.122 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060161992999999997, 2024-09-21T15:00:39.122 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:39.122 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:39.122 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:39.122 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:39.122 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:39.122 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112866 2024-09-21T15:00:39.122 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.122 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:39.122 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.122 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:39.122 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414820+0000", 2024-09-21T15:00:39.122 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.122 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.123 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.123 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:39.123 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414820+0000", 2024-09-21T15:00:39.123 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.123 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.123 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.123 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:39.123 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414820+0000", 2024-09-21T15:00:39.123 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:39.123 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.123 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.123 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:39.123 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414822+0000", 2024-09-21T15:00:39.123 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8509999999999999e-06 2024-09-21T15:00:39.124 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.124 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.124 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:39.124 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414826+0000", 2024-09-21T15:00:39.124 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8190000000000002e-06 2024-09-21T15:00:39.124 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.124 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.124 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:39.124 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414837+0000", 2024-09-21T15:00:39.124 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1232e-05 2024-09-21T15:00:39.124 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.124 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.124 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:39.124 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.474846+0000", 2024-09-21T15:00:39.124 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060009506999999997 2024-09-21T15:00:39.124 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.125 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.125 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:39.125 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.474910+0000", 2024-09-21T15:00:39.125 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3918999999999996e-05 2024-09-21T15:00:39.125 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.125 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.125 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:39.125 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.474982+0000", 2024-09-21T15:00:39.125 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1958999999999993e-05 2024-09-21T15:00:39.125 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.125 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:39.125 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.125 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.125 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.125 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112867 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 4792320~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:39.125 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.414840+0000", 2024-09-21T15:00:39.126 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.6406264830000001, 2024-09-21T15:00:39.126 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060308542999999999, 2024-09-21T15:00:39.126 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:39.126 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:39.126 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:39.126 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:39.126 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:39.126 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112867 2024-09-21T15:00:39.126 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.126 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:39.126 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.126 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:39.126 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414840+0000", 2024-09-21T15:00:39.126 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.127 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.127 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.127 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:39.127 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414840+0000", 2024-09-21T15:00:39.127 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.127 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.127 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.127 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:39.127 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414839+0000", 2024-09-21T15:00:39.127 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:39.127 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.127 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.127 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:39.127 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414841+0000", 2024-09-21T15:00:39.127 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8500000000000001e-06 2024-09-21T15:00:39.127 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.128 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.128 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:39.128 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414846+0000", 2024-09-21T15:00:39.128 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7700000000000001e-06 2024-09-21T15:00:39.128 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.128 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.128 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:39.128 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414850+0000", 2024-09-21T15:00:39.128 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1389999999999997e-06 2024-09-21T15:00:39.128 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.128 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.128 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:39.128 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.475016+0000", 2024-09-21T15:00:39.128 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.06016614 2024-09-21T15:00:39.128 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.129 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.129 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:39.129 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.475073+0000", 2024-09-21T15:00:39.129 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6295000000000002e-05 2024-09-21T15:00:39.129 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.129 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.129 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:39.129 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.475148+0000", 2024-09-21T15:00:39.129 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5647000000000003e-05 2024-09-21T15:00:39.129 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.129 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:39.129 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.129 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.129 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.129 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:113030 3.6 3:60420fc2:::rbd_data.1136b442e94a.0000000000000001:head [write 0~360448 [fadvise_sequential+fadvise_nocache],write 385024~573440 [fadvise_sequential+fadvise_nocache],write 1146880~573440 [fadvise_sequential+fadvise_nocache],write 1908736~188416 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:39.130 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.422420+0000", 2024-09-21T15:00:39.130 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.6330458379999993, 2024-09-21T15:00:39.130 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.063268959999999999, 2024-09-21T15:00:39.130 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:39.130 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:39.130 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:39.130 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:39.130 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:39.130 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 113030 2024-09-21T15:00:39.130 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.130 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:39.130 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.130 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:39.131 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.422420+0000", 2024-09-21T15:00:39.131 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.131 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.131 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.131 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:39.131 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.422420+0000", 2024-09-21T15:00:39.131 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.131 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.131 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.131 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:39.131 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.422420+0000", 2024-09-21T15:00:39.131 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:39.131 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.131 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.132 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:39.132 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.424657+0000", 2024-09-21T15:00:39.132 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0022369780000000001 2024-09-21T15:00:39.132 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.132 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.132 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:39.132 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.424673+0000", 2024-09-21T15:00:39.132 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6141e-05 2024-09-21T15:00:39.132 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.132 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.132 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:39.132 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.424689+0000", 2024-09-21T15:00:39.132 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5897999999999999e-05 2024-09-21T15:00:39.132 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.132 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.133 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:39.133 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.478005+0000", 2024-09-21T15:00:39.133 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.05331636 2024-09-21T15:00:39.133 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.133 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.133 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:39.133 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.478133+0000", 2024-09-21T15:00:39.133 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012736799999999999 2024-09-21T15:00:39.133 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.133 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.133 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 7", 2024-09-21T15:00:39.133 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.478307+0000", 2024-09-21T15:00:39.133 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017491499999999999 2024-09-21T15:00:39.133 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.134 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.134 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:39.134 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.483611+0000", 2024-09-21T15:00:39.134 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0053039389999999997 2024-09-21T15:00:39.134 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.134 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.134 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:39.134 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.485553+0000", 2024-09-21T15:00:39.134 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0019418759999999999 2024-09-21T15:00:39.134 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.134 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.134 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:39.134 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.485645+0000", 2024-09-21T15:00:39.134 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2138000000000005e-05 2024-09-21T15:00:39.135 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.135 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.135 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:39.135 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.485689+0000", 2024-09-21T15:00:39.135 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3922999999999999e-05 2024-09-21T15:00:39.135 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.135 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:39.135 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.135 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.135 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.135 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:114290 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 6946816~8192 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:39.135 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.558087+0000", 2024-09-21T15:00:39.135 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.4973790410000003, 2024-09-21T15:00:39.135 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059973091999999999, 2024-09-21T15:00:39.136 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:39.136 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:39.136 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:39.136 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:39.136 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:39.136 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 114290 2024-09-21T15:00:39.136 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.136 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:39.136 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.136 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:39.136 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.558087+0000", 2024-09-21T15:00:39.136 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.136 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.136 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.136 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:39.137 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.558087+0000", 2024-09-21T15:00:39.137 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.137 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.137 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.137 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:39.137 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.558087+0000", 2024-09-21T15:00:39.137 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:39.137 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.137 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.137 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:39.137 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.558092+0000", 2024-09-21T15:00:39.137 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2379999999999997e-06 2024-09-21T15:00:39.137 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.137 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.138 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:39.138 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.558095+0000", 2024-09-21T15:00:39.138 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.275e-06 2024-09-21T15:00:39.138 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.138 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.138 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:39.138 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.558099+0000", 2024-09-21T15:00:39.138 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6679999999999999e-06 2024-09-21T15:00:39.138 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.138 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.138 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:39.138 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.617757+0000", 2024-09-21T15:00:39.138 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059657531999999999 2024-09-21T15:00:39.139 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.139 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.139 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:39.139 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.617891+0000", 2024-09-21T15:00:39.139 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000134478 2024-09-21T15:00:39.139 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.139 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.139 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:39.139 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.618060+0000", 2024-09-21T15:00:39.139 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016925100000000001 2024-09-21T15:00:39.139 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.139 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:39.139 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.139 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.139 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.140 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:114312 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 8945664~8192 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:39.140 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.562396+0000", 2024-09-21T15:00:39.140 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.4930706929999999, 2024-09-21T15:00:39.140 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059832396000000003, 2024-09-21T15:00:39.140 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:39.140 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:39.140 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:39.140 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:39.140 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:39.140 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 114312 2024-09-21T15:00:39.140 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.140 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:39.140 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.140 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:39.141 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562396+0000", 2024-09-21T15:00:39.141 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.141 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.141 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.141 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:39.141 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562396+0000", 2024-09-21T15:00:39.141 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.141 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.141 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.141 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:39.141 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562395+0000", 2024-09-21T15:00:39.141 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:39.141 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.141 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.142 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:39.142 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562397+0000", 2024-09-21T15:00:39.142 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9e-06 2024-09-21T15:00:39.142 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.142 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.142 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:39.142 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562401+0000", 2024-09-21T15:00:39.142 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8099999999999999e-06 2024-09-21T15:00:39.142 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.142 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.142 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:39.142 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562405+0000", 2024-09-21T15:00:39.142 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.5710000000000002e-06 2024-09-21T15:00:39.142 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.143 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.143 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:39.143 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.621997+0000", 2024-09-21T15:00:39.143 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059592118999999999 2024-09-21T15:00:39.143 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.143 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.143 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:39.143 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.622121+0000", 2024-09-21T15:00:39.143 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012411599999999999 2024-09-21T15:00:39.143 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.143 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.143 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:39.143 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.622228+0000", 2024-09-21T15:00:39.143 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000107184 2024-09-21T15:00:39.144 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.144 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:39.144 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.144 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.144 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.144 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:114313 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 9035776~8192 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:39.144 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.562408+0000", 2024-09-21T15:00:39.144 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.4930587119999998, 2024-09-21T15:00:39.144 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059975560999999997, 2024-09-21T15:00:39.144 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:39.144 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:39.144 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:39.144 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:39.144 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:39.145 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 114313 2024-09-21T15:00:39.145 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.145 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:39.145 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.145 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:39.145 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562408+0000", 2024-09-21T15:00:39.145 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.145 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.145 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.145 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:39.145 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562408+0000", 2024-09-21T15:00:39.145 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.145 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.145 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.145 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:39.146 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562407+0000", 2024-09-21T15:00:39.146 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:39.146 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.146 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.146 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:39.146 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562409+0000", 2024-09-21T15:00:39.146 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9700000000000002e-06 2024-09-21T15:00:39.146 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.146 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.146 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:39.146 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562421+0000", 2024-09-21T15:00:39.146 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2094e-05 2024-09-21T15:00:39.146 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.147 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.147 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:39.147 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562427+0000", 2024-09-21T15:00:39.147 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7690000000000002e-06 2024-09-21T15:00:39.147 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.147 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.147 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:39.147 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.622264+0000", 2024-09-21T15:00:39.147 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059837488000000001 2024-09-21T15:00:39.147 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.147 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.147 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:39.147 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.622319+0000", 2024-09-21T15:00:39.147 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4064999999999998e-05 2024-09-21T15:00:39.147 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.148 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.148 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:39.148 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.622383+0000", 2024-09-21T15:00:39.148 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4506999999999998e-05 2024-09-21T15:00:39.148 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.148 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:39.148 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.148 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.148 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:39.148 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:39.150 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T15:00:39.150 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:00:39.145+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:00:39.250 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:00:39.249+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:00:39.265 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:39.265 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:00:39.265 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T15:00:39.265 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:39.275 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T15:00:39.351 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:00:39.349+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:00:39.431 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:39.431 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:00:39.431 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T15:00:39.431 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T15:00:39.431 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:39.441 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T15:00:39.452 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:00:39.447+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:00:39.553 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:00:39.549+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:00:39.554 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:39.555 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T15:00:39.555 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T15:00:39.555 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T15:00:39.555 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.555 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45839 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:39.555 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159313+0000", 2024-09-21T15:00:39.555 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 89.391726270000007, 2024-09-21T15:00:39.555 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068238699, 2024-09-21T15:00:39.555 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:39.555 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:39.555 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:39.555 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:39.555 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:39.555 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45839 2024-09-21T15:00:39.555 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.556 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:39.556 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.556 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:39.556 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159313+0000", 2024-09-21T15:00:39.556 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.556 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.556 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.556 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:39.556 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159313+0000", 2024-09-21T15:00:39.556 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.556 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.556 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.556 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:39.556 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159313+0000", 2024-09-21T15:00:39.556 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:39.557 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.557 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.557 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:39.557 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159316+0000", 2024-09-21T15:00:39.557 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.033e-06 2024-09-21T15:00:39.557 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.557 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.557 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:39.557 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159322+0000", 2024-09-21T15:00:39.557 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1820000000000003e-06 2024-09-21T15:00:39.557 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.557 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.557 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:39.557 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159327+0000", 2024-09-21T15:00:39.557 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.062e-06 2024-09-21T15:00:39.558 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.558 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.558 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:39.558 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227544+0000", 2024-09-21T15:00:39.558 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068217142999999994 2024-09-21T15:00:39.558 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.558 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.558 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:39.558 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227552+0000", 2024-09-21T15:00:39.558 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6260000000000003e-06 2024-09-21T15:00:39.558 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.558 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:39.558 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.559 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.559 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.559 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45840 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:39.559 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159332+0000", 2024-09-21T15:00:39.559 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 89.391707972999995, 2024-09-21T15:00:39.559 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068262046000000007, 2024-09-21T15:00:39.559 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:39.559 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:39.559 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:39.559 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:39.559 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:39.559 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45840 2024-09-21T15:00:39.559 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.559 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:39.559 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.560 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:39.560 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159332+0000", 2024-09-21T15:00:39.560 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.560 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.560 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.560 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:39.560 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159332+0000", 2024-09-21T15:00:39.560 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.560 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.560 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.560 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:39.560 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159331+0000", 2024-09-21T15:00:39.560 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:39.560 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.561 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.561 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:39.561 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159351+0000", 2024-09-21T15:00:39.561 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0064999999999998e-05 2024-09-21T15:00:39.561 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.561 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.561 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:39.561 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159367+0000", 2024-09-21T15:00:39.561 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5308e-05 2024-09-21T15:00:39.561 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.561 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.561 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:39.561 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159373+0000", 2024-09-21T15:00:39.561 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1840000000000004e-06 2024-09-21T15:00:39.561 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.562 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.562 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:39.562 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227584+0000", 2024-09-21T15:00:39.562 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068211401000000005 2024-09-21T15:00:39.562 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.562 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.562 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:39.562 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227594+0000", 2024-09-21T15:00:39.562 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4120000000000007e-06 2024-09-21T15:00:39.562 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.562 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:39.562 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.562 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.562 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.562 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45841 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:39.563 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159379+0000", 2024-09-21T15:00:39.563 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 89.391660817000002, 2024-09-21T15:00:39.563 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068249039999999997, 2024-09-21T15:00:39.563 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:39.563 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:39.563 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:39.563 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:39.563 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:39.563 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45841 2024-09-21T15:00:39.563 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.563 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:39.563 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.563 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:39.563 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159379+0000", 2024-09-21T15:00:39.563 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.564 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.564 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.564 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:39.564 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159379+0000", 2024-09-21T15:00:39.564 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.564 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.564 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.564 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:39.564 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159378+0000", 2024-09-21T15:00:39.564 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:39.564 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.564 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.564 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:39.565 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159382+0000", 2024-09-21T15:00:39.565 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.641e-06 2024-09-21T15:00:39.565 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.565 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.565 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:39.565 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159421+0000", 2024-09-21T15:00:39.565 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9439e-05 2024-09-21T15:00:39.565 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.565 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.565 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:39.565 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159430+0000", 2024-09-21T15:00:39.565 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8429999999999999e-06 2024-09-21T15:00:39.565 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.565 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.566 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:39.566 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227617+0000", 2024-09-21T15:00:39.566 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068187084999999995 2024-09-21T15:00:39.566 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.566 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.566 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:39.566 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227628+0000", 2024-09-21T15:00:39.566 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0507e-05 2024-09-21T15:00:39.566 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.566 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:39.566 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.566 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.566 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.566 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45844 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:39.566 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159499+0000", 2024-09-21T15:00:39.567 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 89.391540430999996, 2024-09-21T15:00:39.567 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068255415, 2024-09-21T15:00:39.567 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:39.567 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:39.567 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:39.567 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:39.567 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:39.567 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45844 2024-09-21T15:00:39.567 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.567 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:39.567 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.567 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:39.567 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159499+0000", 2024-09-21T15:00:39.567 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.567 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.568 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.568 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:39.568 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159499+0000", 2024-09-21T15:00:39.568 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.568 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.568 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.568 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:39.568 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159499+0000", 2024-09-21T15:00:39.568 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:39.568 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.568 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.568 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:39.568 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159503+0000", 2024-09-21T15:00:39.568 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0999999999999997e-06 2024-09-21T15:00:39.568 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.569 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.569 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:39.569 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159512+0000", 2024-09-21T15:00:39.569 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5850000000000006e-06 2024-09-21T15:00:39.569 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.569 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.569 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:39.569 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159518+0000", 2024-09-21T15:00:39.569 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0909999999999998e-06 2024-09-21T15:00:39.569 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.569 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.569 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:39.569 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227743+0000", 2024-09-21T15:00:39.569 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068224536000000002 2024-09-21T15:00:39.569 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.570 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.570 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:39.570 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227755+0000", 2024-09-21T15:00:39.570 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1659e-05 2024-09-21T15:00:39.570 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.570 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:39.570 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.570 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.570 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.570 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45845 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:39.570 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159524+0000", 2024-09-21T15:00:39.570 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 89.391516070999998, 2024-09-21T15:00:39.570 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068267194000000003, 2024-09-21T15:00:39.570 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:39.570 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:39.571 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:39.571 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:39.571 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:39.571 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45845 2024-09-21T15:00:39.571 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.571 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:39.571 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.571 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:39.571 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159524+0000", 2024-09-21T15:00:39.571 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.571 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.571 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.571 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:39.571 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159524+0000", 2024-09-21T15:00:39.571 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.572 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.572 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.572 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:39.572 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159523+0000", 2024-09-21T15:00:39.572 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:39.572 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.572 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.572 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:39.572 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159526+0000", 2024-09-21T15:00:39.572 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1980000000000001e-06 2024-09-21T15:00:39.572 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.572 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.572 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:39.572 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159533+0000", 2024-09-21T15:00:39.572 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.348e-06 2024-09-21T15:00:39.573 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.573 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.573 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:39.573 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159546+0000", 2024-09-21T15:00:39.573 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3464e-05 2024-09-21T15:00:39.573 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.573 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.573 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:39.573 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227783+0000", 2024-09-21T15:00:39.573 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068236926000000003 2024-09-21T15:00:39.573 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.573 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.573 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:39.573 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227791+0000", 2024-09-21T15:00:39.573 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6669999999999996e-06 2024-09-21T15:00:39.573 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.574 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:39.574 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.574 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.574 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.574 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45846 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:39.574 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159584+0000", 2024-09-21T15:00:39.574 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 89.391455429000004, 2024-09-21T15:00:39.574 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068254767999999993, 2024-09-21T15:00:39.574 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:39.574 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:39.574 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:39.574 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:39.574 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:39.574 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45846 2024-09-21T15:00:39.575 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.575 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:39.575 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.575 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:39.575 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159584+0000", 2024-09-21T15:00:39.575 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.575 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.575 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.575 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:39.575 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159584+0000", 2024-09-21T15:00:39.575 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.575 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.575 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.575 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:39.575 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159583+0000", 2024-09-21T15:00:39.576 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:39.576 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.576 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.576 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:39.576 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159588+0000", 2024-09-21T15:00:39.576 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1330000000000001e-06 2024-09-21T15:00:39.576 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.576 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.576 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:39.576 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159596+0000", 2024-09-21T15:00:39.576 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5469999999999997e-06 2024-09-21T15:00:39.576 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.576 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.576 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:39.576 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159609+0000", 2024-09-21T15:00:39.577 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2785999999999999e-05 2024-09-21T15:00:39.577 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.577 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.577 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:39.577 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227830+0000", 2024-09-21T15:00:39.577 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068220879999999998 2024-09-21T15:00:39.577 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.577 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.577 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:39.577 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227839+0000", 2024-09-21T15:00:39.577 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1239999999999994e-06 2024-09-21T15:00:39.577 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.577 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:39.577 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.577 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.577 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.578 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45847 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:39.578 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159614+0000", 2024-09-21T15:00:39.578 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 89.391425353000002, 2024-09-21T15:00:39.578 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068260639999999997, 2024-09-21T15:00:39.578 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:39.578 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:39.578 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:39.578 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:39.578 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:39.578 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45847 2024-09-21T15:00:39.578 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.578 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:39.578 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.578 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:39.579 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159614+0000", 2024-09-21T15:00:39.579 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.579 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.579 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.579 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:39.579 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159614+0000", 2024-09-21T15:00:39.579 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.579 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.579 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.579 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:39.579 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159614+0000", 2024-09-21T15:00:39.579 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:39.579 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.579 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.579 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:39.579 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159617+0000", 2024-09-21T15:00:39.580 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2909999999999999e-06 2024-09-21T15:00:39.580 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.580 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.580 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:39.580 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159623+0000", 2024-09-21T15:00:39.580 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.511e-06 2024-09-21T15:00:39.580 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.580 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.580 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:39.580 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159631+0000", 2024-09-21T15:00:39.580 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.571e-06 2024-09-21T15:00:39.580 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.580 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.580 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:39.580 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227866+0000", 2024-09-21T15:00:39.581 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068235136000000002 2024-09-21T15:00:39.581 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.581 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.581 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:39.581 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227875+0000", 2024-09-21T15:00:39.581 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7199999999999995e-06 2024-09-21T15:00:39.581 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.581 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:39.581 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.581 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.581 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.581 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45848 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:39.581 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159636+0000", 2024-09-21T15:00:39.581 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 89.391403573000005, 2024-09-21T15:00:39.581 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068267283999999998, 2024-09-21T15:00:39.582 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:39.582 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:39.582 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:39.582 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:39.582 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:39.582 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45848 2024-09-21T15:00:39.582 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.582 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:39.582 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.582 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:39.582 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159636+0000", 2024-09-21T15:00:39.582 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.582 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.582 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.583 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:39.583 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159636+0000", 2024-09-21T15:00:39.583 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.583 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.583 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.583 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:39.583 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159636+0000", 2024-09-21T15:00:39.583 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:39.583 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.583 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.583 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:39.583 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159638+0000", 2024-09-21T15:00:39.583 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6560000000000001e-06 2024-09-21T15:00:39.584 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.584 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.584 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:39.584 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159644+0000", 2024-09-21T15:00:39.584 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0839999999999997e-06 2024-09-21T15:00:39.584 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.584 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.584 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:39.584 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159649+0000", 2024-09-21T15:00:39.584 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7500000000000003e-06 2024-09-21T15:00:39.584 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.584 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.584 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:39.584 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227892+0000", 2024-09-21T15:00:39.584 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068243315999999998 2024-09-21T15:00:39.585 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.585 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.585 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:39.585 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227903+0000", 2024-09-21T15:00:39.585 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0835999999999999e-05 2024-09-21T15:00:39.585 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.585 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:39.585 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.585 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.585 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.585 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45849 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:39.585 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159667+0000", 2024-09-21T15:00:39.585 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 89.391372759000006, 2024-09-21T15:00:39.585 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068291051000000005, 2024-09-21T15:00:39.585 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:39.585 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:39.586 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:39.586 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:39.586 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:39.586 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45849 2024-09-21T15:00:39.586 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.586 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:39.586 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.586 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:39.586 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159667+0000", 2024-09-21T15:00:39.586 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.586 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.586 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.586 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:39.586 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159667+0000", 2024-09-21T15:00:39.587 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.587 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.587 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.587 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:39.587 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159666+0000", 2024-09-21T15:00:39.587 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:39.587 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.587 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.587 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:39.587 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159670+0000", 2024-09-21T15:00:39.587 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.427e-06 2024-09-21T15:00:39.587 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.587 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.587 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:39.587 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159678+0000", 2024-09-21T15:00:39.588 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9869999999999999e-06 2024-09-21T15:00:39.588 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.588 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.588 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:39.588 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159688+0000", 2024-09-21T15:00:39.588 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0631999999999999e-05 2024-09-21T15:00:39.588 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.588 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.588 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:39.588 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227950+0000", 2024-09-21T15:00:39.588 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068261798999999998 2024-09-21T15:00:39.588 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.588 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.588 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:39.589 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227958+0000", 2024-09-21T15:00:39.589 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7510000000000008e-06 2024-09-21T15:00:39.589 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.589 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:39.589 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.589 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.589 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.589 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45850 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:39.589 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159695+0000", 2024-09-21T15:00:39.589 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 89.391345048000005, 2024-09-21T15:00:39.589 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068302999000000003, 2024-09-21T15:00:39.589 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:39.589 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:39.589 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:39.589 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:39.590 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:39.590 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45850 2024-09-21T15:00:39.590 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.590 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:39.590 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.590 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:39.590 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159695+0000", 2024-09-21T15:00:39.590 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.590 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.590 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.590 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:39.590 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159695+0000", 2024-09-21T15:00:39.590 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.590 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.590 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.591 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:39.591 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159694+0000", 2024-09-21T15:00:39.591 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:39.591 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.591 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.591 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:39.591 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159697+0000", 2024-09-21T15:00:39.591 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.258e-06 2024-09-21T15:00:39.591 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.591 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.591 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:39.591 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159704+0000", 2024-09-21T15:00:39.591 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7660000000000001e-06 2024-09-21T15:00:39.591 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.591 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.592 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:39.592 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159713+0000", 2024-09-21T15:00:39.592 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1779999999999992e-06 2024-09-21T15:00:39.592 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.592 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.592 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:39.592 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227989+0000", 2024-09-21T15:00:39.592 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068276015999999995 2024-09-21T15:00:39.592 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.592 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.592 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:39.592 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227998+0000", 2024-09-21T15:00:39.592 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2050000000000002e-06 2024-09-21T15:00:39.592 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.593 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:39.593 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.593 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.593 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.593 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45851 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:39.593 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159719+0000", 2024-09-21T15:00:39.593 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 89.391320914999994, 2024-09-21T15:00:39.593 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068320327, 2024-09-21T15:00:39.593 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:39.593 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:39.593 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:39.593 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:39.593 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:39.593 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45851 2024-09-21T15:00:39.593 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.594 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:39.594 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.594 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:39.594 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159719+0000", 2024-09-21T15:00:39.594 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.594 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.594 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.594 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:39.594 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159719+0000", 2024-09-21T15:00:39.594 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.594 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.594 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.594 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:39.594 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159718+0000", 2024-09-21T15:00:39.594 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:39.595 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.595 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.595 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:39.595 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159721+0000", 2024-09-21T15:00:39.595 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2169999999999999e-06 2024-09-21T15:00:39.595 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.595 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.595 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:39.595 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159730+0000", 2024-09-21T15:00:39.595 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5720000000000003e-06 2024-09-21T15:00:39.595 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.595 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.595 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:39.595 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159735+0000", 2024-09-21T15:00:39.595 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1100000000000002e-06 2024-09-21T15:00:39.596 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.596 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.596 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:39.596 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228029+0000", 2024-09-21T15:00:39.596 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068293624999999997 2024-09-21T15:00:39.596 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.596 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.596 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:39.596 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228039+0000", 2024-09-21T15:00:39.596 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0254e-05 2024-09-21T15:00:39.596 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.596 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:39.596 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.596 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.596 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.597 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45852 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:39.597 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159740+0000", 2024-09-21T15:00:39.597 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 89.391299747000005, 2024-09-21T15:00:39.597 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068340260999999999, 2024-09-21T15:00:39.597 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:39.597 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:39.597 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:39.597 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:39.597 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:39.597 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45852 2024-09-21T15:00:39.597 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.597 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:39.597 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.597 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:39.597 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159740+0000", 2024-09-21T15:00:39.598 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.598 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.598 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.598 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:39.598 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159740+0000", 2024-09-21T15:00:39.598 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.598 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.598 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.598 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:39.598 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159739+0000", 2024-09-21T15:00:39.598 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:39.598 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.598 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.598 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:39.599 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159757+0000", 2024-09-21T15:00:39.599 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7314999999999999e-05 2024-09-21T15:00:39.599 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.599 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.599 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:39.599 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159769+0000", 2024-09-21T15:00:39.599 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2376e-05 2024-09-21T15:00:39.599 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.599 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.599 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:39.599 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159775+0000", 2024-09-21T15:00:39.599 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2179999999999996e-06 2024-09-21T15:00:39.599 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.599 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.600 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:39.600 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228071+0000", 2024-09-21T15:00:39.600 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068295344999999993 2024-09-21T15:00:39.600 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.600 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.600 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:39.600 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228080+0000", 2024-09-21T15:00:39.600 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3959999999999995e-06 2024-09-21T15:00:39.600 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.600 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:39.600 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.600 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.600 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.600 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45853 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:39.600 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159781+0000", 2024-09-21T15:00:39.600 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 89.391258483000001, 2024-09-21T15:00:39.601 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068337093000000002, 2024-09-21T15:00:39.601 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:39.601 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:39.601 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:39.601 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:39.601 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:39.601 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45853 2024-09-21T15:00:39.601 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.601 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:39.601 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.601 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:39.601 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159781+0000", 2024-09-21T15:00:39.601 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.601 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.601 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.602 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:39.602 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159781+0000", 2024-09-21T15:00:39.602 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.602 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.602 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.602 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:39.602 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159781+0000", 2024-09-21T15:00:39.602 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:39.602 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.602 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.602 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:39.602 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159784+0000", 2024-09-21T15:00:39.602 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4759999999999998e-06 2024-09-21T15:00:39.602 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.603 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.603 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:39.603 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159792+0000", 2024-09-21T15:00:39.603 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0569999999999993e-06 2024-09-21T15:00:39.603 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.603 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.603 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:39.603 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159797+0000", 2024-09-21T15:00:39.603 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9640000000000002e-06 2024-09-21T15:00:39.603 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.603 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.603 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:39.603 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228111+0000", 2024-09-21T15:00:39.603 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068313737999999999 2024-09-21T15:00:39.603 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.604 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.604 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:39.604 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228118+0000", 2024-09-21T15:00:39.604 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.43e-06 2024-09-21T15:00:39.604 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.604 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:39.604 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.604 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.604 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.604 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45854 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:39.604 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159802+0000", 2024-09-21T15:00:39.604 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 89.391237833000005, 2024-09-21T15:00:39.604 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068358938999999994, 2024-09-21T15:00:39.604 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:39.605 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:39.605 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:39.605 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:39.605 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:39.605 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45854 2024-09-21T15:00:39.605 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.605 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:39.605 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.605 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:39.605 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159802+0000", 2024-09-21T15:00:39.605 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.605 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.605 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.605 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:39.605 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159802+0000", 2024-09-21T15:00:39.606 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.606 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.606 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.606 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:39.606 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159801+0000", 2024-09-21T15:00:39.606 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:39.606 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.606 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.606 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:39.606 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159804+0000", 2024-09-21T15:00:39.606 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0970000000000002e-06 2024-09-21T15:00:39.606 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.606 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.606 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:39.607 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159823+0000", 2024-09-21T15:00:39.607 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8895000000000001e-05 2024-09-21T15:00:39.607 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.607 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.607 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:39.607 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159837+0000", 2024-09-21T15:00:39.607 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3247999999999999e-05 2024-09-21T15:00:39.607 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.607 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.607 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:39.607 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228151+0000", 2024-09-21T15:00:39.607 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068314642999999994 2024-09-21T15:00:39.607 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.607 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.608 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:39.608 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228161+0000", 2024-09-21T15:00:39.608 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4250000000000004e-06 2024-09-21T15:00:39.608 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.608 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:39.608 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.608 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.608 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.608 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45983 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:39.608 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159848+0000", 2024-09-21T15:00:39.608 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 89.391191574000004, 2024-09-21T15:00:39.608 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068333801, 2024-09-21T15:00:39.608 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:39.608 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:39.608 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:39.609 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:39.609 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:39.609 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45983 2024-09-21T15:00:39.609 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.609 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:39.609 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.609 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:39.609 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159848+0000", 2024-09-21T15:00:39.609 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.609 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.609 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.609 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:39.609 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159848+0000", 2024-09-21T15:00:39.609 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.609 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.610 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.610 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:39.610 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159847+0000", 2024-09-21T15:00:39.610 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:39.610 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.610 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.610 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:39.610 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159856+0000", 2024-09-21T15:00:39.610 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4349999999999997e-06 2024-09-21T15:00:39.610 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.610 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.610 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:39.610 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159866+0000", 2024-09-21T15:00:39.610 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0076e-05 2024-09-21T15:00:39.611 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.611 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.611 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:39.611 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159872+0000", 2024-09-21T15:00:39.611 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8309999999999997e-06 2024-09-21T15:00:39.611 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.611 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.611 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:39.611 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228175+0000", 2024-09-21T15:00:39.611 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068302680000000005 2024-09-21T15:00:39.611 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.611 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.611 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:39.611 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228182+0000", 2024-09-21T15:00:39.612 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3150000000000003e-06 2024-09-21T15:00:39.612 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.612 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:39.612 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.612 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.612 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.612 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45984 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:39.612 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159877+0000", 2024-09-21T15:00:39.612 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 89.391162154, 2024-09-21T15:00:39.612 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068328448, 2024-09-21T15:00:39.612 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:39.612 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:39.612 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:39.612 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:39.613 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:39.613 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45984 2024-09-21T15:00:39.613 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.613 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:39.613 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.613 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:39.613 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159877+0000", 2024-09-21T15:00:39.613 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.613 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.613 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.613 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:39.613 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159877+0000", 2024-09-21T15:00:39.613 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.614 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.614 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.614 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:39.614 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159877+0000", 2024-09-21T15:00:39.614 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:39.614 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.614 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.614 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:39.614 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159880+0000", 2024-09-21T15:00:39.614 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7610000000000001e-06 2024-09-21T15:00:39.614 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.614 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.614 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:39.614 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159888+0000", 2024-09-21T15:00:39.614 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0330000000000003e-06 2024-09-21T15:00:39.614 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.615 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.615 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:39.615 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159893+0000", 2024-09-21T15:00:39.615 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3550000000000004e-06 2024-09-21T15:00:39.615 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.615 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.615 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:39.615 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228199+0000", 2024-09-21T15:00:39.615 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068306438999999997 2024-09-21T15:00:39.615 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.615 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.615 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:39.615 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228206+0000", 2024-09-21T15:00:39.615 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5599999999999999e-06 2024-09-21T15:00:39.616 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.616 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:39.616 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.616 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.616 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.616 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45985 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:39.616 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159897+0000", 2024-09-21T15:00:39.616 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 89.391142330999998, 2024-09-21T15:00:39.616 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068345664, 2024-09-21T15:00:39.616 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:39.616 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:39.616 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:39.616 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:39.616 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:39.616 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45985 2024-09-21T15:00:39.617 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.617 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:39.617 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.617 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:39.617 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159897+0000", 2024-09-21T15:00:39.617 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.617 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.617 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.617 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:39.617 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159897+0000", 2024-09-21T15:00:39.617 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.617 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.617 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.617 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:39.617 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159897+0000", 2024-09-21T15:00:39.618 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:39.618 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.618 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.618 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:39.618 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159900+0000", 2024-09-21T15:00:39.618 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.954e-06 2024-09-21T15:00:39.618 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.618 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.618 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:39.618 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159913+0000", 2024-09-21T15:00:39.618 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3106e-05 2024-09-21T15:00:39.618 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.618 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.618 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:39.619 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159918+0000", 2024-09-21T15:00:39.619 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5199999999999997e-06 2024-09-21T15:00:39.619 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.619 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.619 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:39.619 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228236+0000", 2024-09-21T15:00:39.619 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068317040999999995 2024-09-21T15:00:39.619 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.619 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.619 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:39.619 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228243+0000", 2024-09-21T15:00:39.619 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3880000000000002e-06 2024-09-21T15:00:39.619 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.619 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:39.619 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.620 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.620 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.620 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45986 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:39.620 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159924+0000", 2024-09-21T15:00:39.620 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 89.391116036, 2024-09-21T15:00:39.620 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068364362999999997, 2024-09-21T15:00:39.620 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:39.620 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:39.620 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:39.620 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:39.620 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:39.620 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45986 2024-09-21T15:00:39.620 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.620 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:39.620 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.620 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:39.621 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159924+0000", 2024-09-21T15:00:39.621 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.621 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.621 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.621 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:39.621 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159924+0000", 2024-09-21T15:00:39.621 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.621 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.621 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.621 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:39.621 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159923+0000", 2024-09-21T15:00:39.621 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:39.621 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.621 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.621 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:39.621 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159926+0000", 2024-09-21T15:00:39.622 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0510000000000001e-06 2024-09-21T15:00:39.622 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.622 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.622 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:39.622 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159932+0000", 2024-09-21T15:00:39.622 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4160000000000001e-06 2024-09-21T15:00:39.622 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.622 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.622 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:39.622 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159938+0000", 2024-09-21T15:00:39.622 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.383e-06 2024-09-21T15:00:39.622 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.622 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.623 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:39.623 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228275+0000", 2024-09-21T15:00:39.623 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068337007000000005 2024-09-21T15:00:39.623 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.623 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.623 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:39.623 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228288+0000", 2024-09-21T15:00:39.623 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3052e-05 2024-09-21T15:00:39.623 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.623 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:39.623 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.623 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.623 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.623 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45987 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:39.623 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159944+0000", 2024-09-21T15:00:39.624 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 89.391095812000003, 2024-09-21T15:00:39.624 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068387232000000006, 2024-09-21T15:00:39.624 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:39.624 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:39.624 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:39.624 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:39.624 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:39.624 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45987 2024-09-21T15:00:39.624 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.624 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:39.624 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.624 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:39.624 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159944+0000", 2024-09-21T15:00:39.624 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.624 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.624 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.625 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:39.625 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159944+0000", 2024-09-21T15:00:39.625 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.625 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.625 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.625 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:39.625 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159943+0000", 2024-09-21T15:00:39.625 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:39.625 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.625 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.625 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:39.625 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159946+0000", 2024-09-21T15:00:39.625 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6309999999999999e-06 2024-09-21T15:00:39.625 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.625 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.626 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:39.626 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159952+0000", 2024-09-21T15:00:39.626 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6540000000000004e-06 2024-09-21T15:00:39.626 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.626 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.626 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:39.626 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159957+0000", 2024-09-21T15:00:39.626 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9280000000000001e-06 2024-09-21T15:00:39.626 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.626 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.626 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:39.626 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228322+0000", 2024-09-21T15:00:39.626 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068365067000000002 2024-09-21T15:00:39.626 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.626 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.626 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:39.627 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228331+0000", 2024-09-21T15:00:39.627 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3270000000000007e-06 2024-09-21T15:00:39.627 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.627 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:39.627 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.627 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.627 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.627 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:97672 3.4 3:26046d29:::rbd_data.1136bfc79a70.0000000000000001:head [write 0~4718592 [fadvise_sequential+fadvise_nocache],write 5242880~131072 [fadvise_sequential+fadvise_nocache],write 5439488~147456 [fadvise_sequential+fadvise_nocache],write 5652480~229376 [fadvise_sequential+fadvise_nocache],write 5947392~147456 [fadvise_sequential+fadvise_nocache],write 6160384~229376 [fadvise_sequential+fadvise_nocache],write 6455296~147456 [fadvise_sequential+fadvise_nocache],write 6668288~229376 [fadvise_sequential+fadvise_nocache],write 6963200~147456 [fadvise_sequential+fadvise_nocache],write 7176192~147456 [fadvise_sequential+fadvise_nocache],write 7340032~65536 [fadvise_sequential+fadvise_nocache],write 7471104~147456 [fadvise_sequential+fadvise_nocache],write 7684096~147456 [fadvise_sequential+fadvise_nocache],write 7864320~49152 [fadvise_sequential+fadvise_nocache],write 7979008~147456 [fadvise_sequential+fadvise_nocache],write 8192000~147456 [fadvise_sequential+fadvise_nocache],write 8388608~32768 [fadvise_sequential+fadvise_nocache],write 8519680~114688 [fadvise_sequential+fadvise_nocache],write 8732672~114688 [fadvise_sequential+fadvise_nocache],write 8912896~16384 [fadvise_sequential+fadvise_nocache],write 9109504~32768 [fadvise_sequential+fadvise_nocache],write 9322496~32768 [fadvise_sequential+fadvise_nocache],write 9617408~32768 [fadvise_sequential+fadvise_nocache],write 9830400~32768 [fadvise_sequential+fadvise_nocache],write 9961472~98304 [fadvise_sequential+fadvise_nocache],write 10076160~196608 [fadvise_sequential+fadvise_nocache],write 10289152~278528 [fadvise_sequential+fadvise_nocache],write 10584064~196608 [fadvise_sequential+fadvise_nocache],write 10797056~196608 [fadvise_sequential+fadvise_nocache],write 11010048~65536 [fadvise_sequential+fadvise_nocache],write 11091968~196608 [fadvise_sequential+fadvise_nocache],write 11304960~196608 [fadvise_sequential+fadvise_nocache],write 11517952~65536 [fadvise_sequential+fadvise_nocache],write 11616256~98304 [fadvise_sequential+fadvise_nocache],write 11730944~65536 [fadvise_sequential+fadvise_nocache],write 11829248~98304 [fadvise_sequential+fadvise_nocache],write 11943936~65536 [fadvise_sequential+fadvise_nocache],write 12042240~49152 [fadvise_sequential+fadvise_nocache],write 12124160~98304 [fadvise_sequential+fadvise_nocache],write 12238848~65536 [fadvise_sequential+fadvise_nocache],write 12337152~98304 [fadvise_sequential+fadvise_nocache],write 12451840~65536 [fadvise_sequential+fadvise_nocache],write 12550144~32768 [fadvise_sequential+fadvise_nocache],write 12632064~98304 [fadvise_sequential+fadvise_nocache],write 12845056~98304 [fadvise_sequential+fadvise_nocache],write 13058048~49152 [fadvise_sequential+fadvise_nocache],write 13139968~98304 [fadvise_sequential+fadvise_nocache],write 13352960~98304 [fadvise_sequential+fadvise_nocache],write 13565952~65536 [fadvise_sequential+fadvise_nocache],write 13647872~98304 [fadvise_sequential+fadvise_nocache],write 13860864~98304 [fadvise_sequential+fadvise_nocache],write 14073856~81920 [fadvise_sequential+fadvise_nocache],write 15269888~81920 [fadvise_sequential+fadvise_nocache],write 15482880~81920 [fadvise_sequential+fadvise_nocache],write 15695872~32768 [fadvise_sequential+fadvise_nocache],write 15777792~81920 [fadvise_sequential+fadvise_nocache],write 15990784~81920 [fadvise_sequential+fadvise_nocache],write 16203776~49152 [fadvise_sequential+fadvise_nocache],write 16285696~81920 [fadvise_sequential+fadvise_nocache],write 16498688~81920 [fadvise_sequential+fadvise_nocache],write 16711680~65536 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:39.627 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.298669+0000", 2024-09-21T15:00:39.627 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 34.252370745999997, 2024-09-21T15:00:39.627 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.10282358900000001, 2024-09-21T15:00:39.627 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:39.627 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:39.627 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:39.628 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:39.628 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:39.628 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97672 2024-09-21T15:00:39.628 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.628 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:39.628 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.628 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:39.628 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.298669+0000", 2024-09-21T15:00:39.628 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.628 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.628 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.628 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:39.628 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.298669+0000", 2024-09-21T15:00:39.628 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.628 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.629 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.629 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:39.629 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.298667+0000", 2024-09-21T15:00:39.629 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:00:39.629 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.629 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.629 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:39.629 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.327458+0000", 2024-09-21T15:00:39.629 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.028790754000000002 2024-09-21T15:00:39.629 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.629 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.629 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:39.629 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.327492+0000", 2024-09-21T15:00:39.629 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3806999999999997e-05 2024-09-21T15:00:39.629 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.629 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.630 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:39.630 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.327507+0000", 2024-09-21T15:00:39.630 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5112e-05 2024-09-21T15:00:39.630 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.630 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.630 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:39.630 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.327597+0000", 2024-09-21T15:00:39.630 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9938999999999994e-05 2024-09-21T15:00:39.630 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.630 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.630 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:39.630 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.327927+0000", 2024-09-21T15:00:39.630 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000329766 2024-09-21T15:00:39.630 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.630 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.631 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T15:00:39.631 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.330735+0000", 2024-09-21T15:00:39.631 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0028086869999999998 2024-09-21T15:00:39.631 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.631 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.631 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:39.631 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.378299+0000", 2024-09-21T15:00:39.631 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.047563495999999997 2024-09-21T15:00:39.631 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.631 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.631 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:39.631 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.401248+0000", 2024-09-21T15:00:39.631 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.022949311 2024-09-21T15:00:39.631 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.631 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.632 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:39.632 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.401446+0000", 2024-09-21T15:00:39.632 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019831199999999999 2024-09-21T15:00:39.632 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.632 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.632 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:39.632 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.401492+0000", 2024-09-21T15:00:39.632 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6109000000000002e-05 2024-09-21T15:00:39.632 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.632 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:39.632 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.632 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.632 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:39.632 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:39.633 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T15:00:39.653 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:00:39.651+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:00:39.746 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:39.746 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:00:39.746 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T15:00:39.746 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:39.755 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:00:39.755+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:00:39.756 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T15:00:39.855 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:00:39.851+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:00:39.859 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:39.859 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:00:39.859 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T15:00:39.859 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T15:00:39.859 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:39.869 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T15:00:39.956 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:00:39.953+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:00:39.979 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:39.980 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T15:00:39.980 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T15:00:39.980 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T15:00:39.980 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.980 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:97677 3.15 e122/24 3:a8e72133:::rbd_data.1136bfc79a70.0000000000000005:head v 122'567, mlcod=120'565)", 2024-09-21T15:00:39.980 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.320753+0000", 2024-09-21T15:00:39.980 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 34.655295662, 2024-09-21T15:00:39.980 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070639375000000004, 2024-09-21T15:00:39.980 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:39.980 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:39.980 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:39.980 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.980 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:39.980 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.320753+0000", 2024-09-21T15:00:39.980 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.981 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.981 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.981 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:39.981 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.320753+0000", 2024-09-21T15:00:39.981 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.981 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.981 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.981 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:39.981 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.320753+0000", 2024-09-21T15:00:39.981 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:39.981 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.981 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.981 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:39.982 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.375452+0000", 2024-09-21T15:00:39.982 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054698945999999998 2024-09-21T15:00:39.982 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.982 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.982 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:39.982 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.375476+0000", 2024-09-21T15:00:39.982 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4380999999999999e-05 2024-09-21T15:00:39.982 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.982 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.982 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:39.982 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.375485+0000", 2024-09-21T15:00:39.982 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0049999999999993e-06 2024-09-21T15:00:39.982 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.983 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.983 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:39.983 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.375552+0000", 2024-09-21T15:00:39.983 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6550999999999994e-05 2024-09-21T15:00:39.983 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.983 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.983 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:39.983 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.375578+0000", 2024-09-21T15:00:39.983 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6196000000000001e-05 2024-09-21T15:00:39.983 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.983 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.983 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:39.983 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.391319+0000", 2024-09-21T15:00:39.983 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015741271000000001 2024-09-21T15:00:39.984 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.984 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.984 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:39.984 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.391393+0000", 2024-09-21T15:00:39.984 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3287000000000005e-05 2024-09-21T15:00:39.984 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.984 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:39.984 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.984 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.984 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.984 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:97672 3.4 e122/51 3:26046d29:::rbd_data.1136bfc79a70.0000000000000001:head v 122'256, mlcod=120'254)", 2024-09-21T15:00:39.984 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.331696+0000", 2024-09-21T15:00:39.984 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 34.644352499, 2024-09-21T15:00:39.984 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041222507999999998, 2024-09-21T15:00:39.985 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:39.985 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:39.985 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:39.985 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.985 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:39.985 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.331696+0000", 2024-09-21T15:00:39.985 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.985 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.985 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.985 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:39.985 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.331696+0000", 2024-09-21T15:00:39.985 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.985 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.985 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.985 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:39.986 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.331695+0000", 2024-09-21T15:00:39.986 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:00:39.986 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.986 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.986 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:39.986 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.340175+0000", 2024-09-21T15:00:39.986 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0084799520000000007 2024-09-21T15:00:39.986 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.986 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.986 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:39.986 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.340208+0000", 2024-09-21T15:00:39.986 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2997999999999998e-05 2024-09-21T15:00:39.986 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.987 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.987 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:39.987 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.340223+0000", 2024-09-21T15:00:39.987 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5330999999999999e-05 2024-09-21T15:00:39.987 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.987 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.987 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:39.987 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.340305+0000", 2024-09-21T15:00:39.987 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2095999999999995e-05 2024-09-21T15:00:39.987 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.987 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.987 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:39.987 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.340337+0000", 2024-09-21T15:00:39.987 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2097000000000001e-05 2024-09-21T15:00:39.987 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.988 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.988 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:39.988 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.372863+0000", 2024-09-21T15:00:39.988 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.032525923999999998 2024-09-21T15:00:39.988 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.988 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.988 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:39.988 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.372919+0000", 2024-09-21T15:00:39.988 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5729000000000001e-05 2024-09-21T15:00:39.988 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.988 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:39.988 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.988 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.988 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.988 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:116177 3.12 3:4efbaa3f:::rbd_data.1136b1194d20.0000000000000006:58 [read 5791744~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:39.989 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.712449+0000", 2024-09-21T15:00:39.989 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 10.263599817999999, 2024-09-21T15:00:39.989 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.033824706000000003, 2024-09-21T15:00:39.989 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:39.989 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:39.989 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:39.989 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:39.989 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:39.989 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116177 2024-09-21T15:00:39.989 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.989 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:39.989 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.989 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:39.989 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712449+0000", 2024-09-21T15:00:39.990 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.990 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.990 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.990 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:39.990 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712449+0000", 2024-09-21T15:00:39.990 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.990 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.990 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.990 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:39.990 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712448+0000", 2024-09-21T15:00:39.990 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:39.990 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.990 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.990 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:39.990 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712451+0000", 2024-09-21T15:00:39.991 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5330000000000001e-06 2024-09-21T15:00:39.991 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.991 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.991 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:39.991 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712458+0000", 2024-09-21T15:00:39.991 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5239999999999997e-06 2024-09-21T15:00:39.991 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.991 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.991 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:39.991 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712467+0000", 2024-09-21T15:00:39.991 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7189999999999996e-06 2024-09-21T15:00:39.991 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.991 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.991 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:39.992 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.746189+0000", 2024-09-21T15:00:39.992 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.033722203999999999 2024-09-21T15:00:39.992 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.992 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.992 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:39.992 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.746228+0000", 2024-09-21T15:00:39.992 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8908000000000001e-05 2024-09-21T15:00:39.992 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.992 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.992 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:39.992 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.746274+0000", 2024-09-21T15:00:39.992 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5380000000000003e-05 2024-09-21T15:00:39.992 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.992 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:39.992 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.993 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.993 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.993 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:116178 3.12 3:4efbaa3f:::rbd_data.1136b1194d20.0000000000000006:58 [read 5881856~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:39.993 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.712472+0000", 2024-09-21T15:00:39.993 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 10.263577166999999, 2024-09-21T15:00:39.993 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.033958110999999999, 2024-09-21T15:00:39.993 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:39.993 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:39.993 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:39.993 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:39.993 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:39.993 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116178 2024-09-21T15:00:39.993 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.993 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:39.993 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.994 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:39.994 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712472+0000", 2024-09-21T15:00:39.994 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.994 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.994 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.994 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:39.994 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712472+0000", 2024-09-21T15:00:39.994 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.994 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.994 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.994 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:39.994 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712471+0000", 2024-09-21T15:00:39.994 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:39.994 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.995 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.995 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:39.995 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712474+0000", 2024-09-21T15:00:39.995 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4389999999999999e-06 2024-09-21T15:00:39.995 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.995 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.995 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:39.995 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712479+0000", 2024-09-21T15:00:39.995 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4269999999999999e-06 2024-09-21T15:00:39.995 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.995 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.995 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:39.995 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712483+0000", 2024-09-21T15:00:39.996 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8449999999999996e-06 2024-09-21T15:00:39.996 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.996 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.996 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:39.996 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.746287+0000", 2024-09-21T15:00:39.996 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.033804381000000001 2024-09-21T15:00:39.996 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.996 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.996 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:39.996 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.746337+0000", 2024-09-21T15:00:39.996 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9622e-05 2024-09-21T15:00:39.996 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.996 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.996 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:39.996 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.746430+0000", 2024-09-21T15:00:39.997 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2800000000000006e-05 2024-09-21T15:00:39.997 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.997 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:39.997 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:39.997 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.997 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.997 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:116179 3.12 3:4efbaa3f:::rbd_data.1136b1194d20.0000000000000006:58 [read 5971968~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:39.997 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.712487+0000", 2024-09-21T15:00:39.997 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 10.263562325000001, 2024-09-21T15:00:39.997 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034144503, 2024-09-21T15:00:39.997 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:39.997 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:39.997 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:39.997 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:39.998 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:39.998 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116179 2024-09-21T15:00:39.998 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.998 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:39.998 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.998 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:39.998 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712487+0000", 2024-09-21T15:00:39.998 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.998 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.998 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.998 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:39.998 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712487+0000", 2024-09-21T15:00:39.998 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:39.998 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.999 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.999 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:39.999 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712486+0000", 2024-09-21T15:00:39.999 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:39.999 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.999 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.999 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:39.999 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712488+0000", 2024-09-21T15:00:39.999 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2000000000000001e-06 2024-09-21T15:00:39.999 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:39.999 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:39.999 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:39.999 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712493+0000", 2024-09-21T15:00:39.999 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9429999999999999e-06 2024-09-21T15:00:39.999 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.000 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.000 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:40.000 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712497+0000", 2024-09-21T15:00:40.000 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9689999999999996e-06 2024-09-21T15:00:40.000 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.000 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.000 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:40.000 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.746459+0000", 2024-09-21T15:00:40.000 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.033961365 2024-09-21T15:00:40.000 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.000 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.000 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:40.000 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.746538+0000", 2024-09-21T15:00:40.000 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9073999999999995e-05 2024-09-21T15:00:40.000 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.000 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.001 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:40.001 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.746631+0000", 2024-09-21T15:00:40.001 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3375999999999995e-05 2024-09-21T15:00:40.001 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:40.001 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:40.001 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:40.001 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.001 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.001 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:116180 3.12 3:4efbaa3f:::rbd_data.1136b1194d20.0000000000000006:58 [read 6062080~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:40.001 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.712501+0000", 2024-09-21T15:00:40.001 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 10.263548127, 2024-09-21T15:00:40.001 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034331168000000002, 2024-09-21T15:00:40.001 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:40.001 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:40.001 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:40.001 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:40.002 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:40.002 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116180 2024-09-21T15:00:40.002 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.002 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:40.002 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.002 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:40.002 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712501+0000", 2024-09-21T15:00:40.002 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:40.002 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.002 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.002 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:40.002 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712501+0000", 2024-09-21T15:00:40.002 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:40.002 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.003 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.003 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:40.003 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712500+0000", 2024-09-21T15:00:40.003 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:40.003 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.003 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.003 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:40.003 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712502+0000", 2024-09-21T15:00:40.003 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.035e-06 2024-09-21T15:00:40.003 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.003 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.003 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:40.003 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712508+0000", 2024-09-21T15:00:40.003 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4720000000000003e-06 2024-09-21T15:00:40.003 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.004 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.004 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:40.004 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712512+0000", 2024-09-21T15:00:40.004 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7129999999999999e-06 2024-09-21T15:00:40.004 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.004 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.004 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:40.004 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.746662+0000", 2024-09-21T15:00:40.004 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034149837000000002 2024-09-21T15:00:40.004 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.004 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.004 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:40.004 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.746740+0000", 2024-09-21T15:00:40.004 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8350999999999996e-05 2024-09-21T15:00:40.004 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.005 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.005 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:40.005 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.746832+0000", 2024-09-21T15:00:40.005 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2046000000000006e-05 2024-09-21T15:00:40.005 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:40.005 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:40.005 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:40.005 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.005 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.005 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:116181 3.12 3:4efbaa3f:::rbd_data.1136b1194d20.0000000000000006:58 [read 6152192~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:40.005 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.712515+0000", 2024-09-21T15:00:40.005 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 10.263533781, 2024-09-21T15:00:40.005 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034525820999999998, 2024-09-21T15:00:40.005 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:40.005 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:40.006 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:40.006 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:40.006 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:40.006 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116181 2024-09-21T15:00:40.006 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.006 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:40.006 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.006 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:40.006 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712515+0000", 2024-09-21T15:00:40.006 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:40.006 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.006 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.006 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:40.007 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712515+0000", 2024-09-21T15:00:40.007 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:40.007 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.007 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.007 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:40.007 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712515+0000", 2024-09-21T15:00:40.007 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:40.007 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.007 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.007 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:40.007 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712522+0000", 2024-09-21T15:00:40.007 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8000000000000001e-06 2024-09-21T15:00:40.007 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.007 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.007 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:40.008 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712528+0000", 2024-09-21T15:00:40.008 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0650000000000004e-06 2024-09-21T15:00:40.008 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.008 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.008 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:40.008 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712533+0000", 2024-09-21T15:00:40.008 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.079e-06 2024-09-21T15:00:40.008 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.008 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.008 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:40.008 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.746864+0000", 2024-09-21T15:00:40.008 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034330830999999999 2024-09-21T15:00:40.008 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.008 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.008 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:40.009 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.746943+0000", 2024-09-21T15:00:40.009 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9221999999999995e-05 2024-09-21T15:00:40.009 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.009 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.009 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:40.009 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.747041+0000", 2024-09-21T15:00:40.009 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.8114000000000003e-05 2024-09-21T15:00:40.009 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:40.009 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:40.009 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:40.009 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.009 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.009 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:116182 3.12 3:4efbaa3f:::rbd_data.1136b1194d20.0000000000000006:58 [read 6242304~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:40.009 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.712538+0000", 2024-09-21T15:00:40.010 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 10.263510508, 2024-09-21T15:00:40.010 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034702127999999999, 2024-09-21T15:00:40.010 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:40.010 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:40.010 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:40.010 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:40.010 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:40.010 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116182 2024-09-21T15:00:40.010 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.010 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:40.010 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.010 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:40.010 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712538+0000", 2024-09-21T15:00:40.010 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:40.010 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.011 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.011 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:40.011 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712538+0000", 2024-09-21T15:00:40.011 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:40.011 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.011 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.011 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:40.011 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712538+0000", 2024-09-21T15:00:40.011 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:40.011 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.011 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.011 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:40.011 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712541+0000", 2024-09-21T15:00:40.011 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5330000000000001e-06 2024-09-21T15:00:40.011 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.012 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.012 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:40.012 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712547+0000", 2024-09-21T15:00:40.012 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1109999999999996e-06 2024-09-21T15:00:40.012 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.012 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.012 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:40.012 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712551+0000", 2024-09-21T15:00:40.012 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.685e-06 2024-09-21T15:00:40.012 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.012 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.012 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:40.012 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.747067+0000", 2024-09-21T15:00:40.012 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034516181 2024-09-21T15:00:40.013 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.013 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.013 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:40.013 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.747152+0000", 2024-09-21T15:00:40.013 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4075999999999999e-05 2024-09-21T15:00:40.013 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.013 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.013 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:40.013 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.747240+0000", 2024-09-21T15:00:40.013 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8924000000000004e-05 2024-09-21T15:00:40.013 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:40.013 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:40.013 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:40.013 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.014 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.014 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:116216 3.12 3:4efbaa3f:::rbd_data.1136b1194d20.0000000000000006:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:40.014 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.713202+0000", 2024-09-21T15:00:40.014 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 10.262847002999999, 2024-09-21T15:00:40.014 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034182227000000003, 2024-09-21T15:00:40.014 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:40.014 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:40.014 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:40.014 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:40.014 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:40.014 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116216 2024-09-21T15:00:40.014 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.014 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:40.014 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.015 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:40.015 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.713202+0000", 2024-09-21T15:00:40.015 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:40.015 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.015 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.015 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:40.015 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.713202+0000", 2024-09-21T15:00:40.015 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:40.015 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.015 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.015 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:40.015 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.713201+0000", 2024-09-21T15:00:40.015 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:40.015 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.016 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.016 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:40.016 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.713204+0000", 2024-09-21T15:00:40.016 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7410000000000001e-06 2024-09-21T15:00:40.016 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.016 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.016 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:40.016 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.713211+0000", 2024-09-21T15:00:40.016 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1110000000000002e-06 2024-09-21T15:00:40.016 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.016 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.016 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:40.016 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.713217+0000", 2024-09-21T15:00:40.016 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.446e-06 2024-09-21T15:00:40.017 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.017 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.017 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:40.017 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.747263+0000", 2024-09-21T15:00:40.017 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034046598999999997 2024-09-21T15:00:40.017 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.017 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.017 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:40.017 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.747321+0000", 2024-09-21T15:00:40.017 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7778999999999997e-05 2024-09-21T15:00:40.017 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.017 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.017 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:40.017 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.747384+0000", 2024-09-21T15:00:40.018 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3083000000000006e-05 2024-09-21T15:00:40.018 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:40.018 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:40.018 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:40.018 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.018 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.018 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:118512 3.12 3:4efbaa3f:::rbd_data.1136b1194d20.0000000000000006:58 [read 10043392~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:40.018 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.893240+0000", 2024-09-21T15:00:40.018 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 10.082808415000001, 2024-09-21T15:00:40.018 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.033811941999999998, 2024-09-21T15:00:40.018 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:40.018 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:40.018 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:40.019 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:40.019 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:40.019 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 118512 2024-09-21T15:00:40.019 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.019 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:40.019 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.019 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:40.019 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893240+0000", 2024-09-21T15:00:40.019 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:40.019 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.019 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.019 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:40.019 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893240+0000", 2024-09-21T15:00:40.019 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:40.020 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.020 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.020 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:40.020 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893240+0000", 2024-09-21T15:00:40.020 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:40.020 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.020 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.020 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:40.020 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893242+0000", 2024-09-21T15:00:40.020 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.717e-06 2024-09-21T15:00:40.020 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.020 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.020 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:40.020 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893245+0000", 2024-09-21T15:00:40.020 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3380000000000001e-06 2024-09-21T15:00:40.021 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.021 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.021 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:40.021 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893248+0000", 2024-09-21T15:00:40.021 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9670000000000001e-06 2024-09-21T15:00:40.021 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.021 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.021 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:40.021 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.926868+0000", 2024-09-21T15:00:40.021 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.033619938000000002 2024-09-21T15:00:40.021 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.021 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.021 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:40.021 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.926964+0000", 2024-09-21T15:00:40.022 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5624999999999993e-05 2024-09-21T15:00:40.022 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.022 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.022 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:40.022 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.927052+0000", 2024-09-21T15:00:40.022 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8632000000000001e-05 2024-09-21T15:00:40.022 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:40.022 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:40.022 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:40.022 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.022 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.022 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:118513 3.12 3:4efbaa3f:::rbd_data.1136b1194d20.0000000000000006:58 [read 10133504~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:40.022 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.893251+0000", 2024-09-21T15:00:40.023 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 10.082798169, 2024-09-21T15:00:40.023 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.033992324999999997, 2024-09-21T15:00:40.023 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:40.023 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:40.023 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:40.023 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:40.023 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:40.023 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 118513 2024-09-21T15:00:40.023 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.023 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:40.023 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.023 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:40.023 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893251+0000", 2024-09-21T15:00:40.023 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:40.024 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.024 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.024 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:40.024 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893251+0000", 2024-09-21T15:00:40.024 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:40.024 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.024 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.024 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:40.024 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893250+0000", 2024-09-21T15:00:40.024 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:40.024 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.024 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.024 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:40.025 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893252+0000", 2024-09-21T15:00:40.025 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.666e-06 2024-09-21T15:00:40.025 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.025 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.025 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:40.025 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893255+0000", 2024-09-21T15:00:40.025 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.287e-06 2024-09-21T15:00:40.025 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.025 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.025 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:40.025 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893258+0000", 2024-09-21T15:00:40.025 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.027e-06 2024-09-21T15:00:40.025 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.025 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.026 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:40.026 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.927083+0000", 2024-09-21T15:00:40.026 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.033824262000000001 2024-09-21T15:00:40.026 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.026 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.026 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:40.026 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.927154+0000", 2024-09-21T15:00:40.026 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1099000000000005e-05 2024-09-21T15:00:40.026 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.026 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.026 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:40.026 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.927243+0000", 2024-09-21T15:00:40.026 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9266999999999998e-05 2024-09-21T15:00:40.026 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:40.027 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:40.027 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:40.027 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.027 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.027 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:118514 3.12 3:4efbaa3f:::rbd_data.1136b1194d20.0000000000000006:58 [read 10223616~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:40.027 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.893261+0000", 2024-09-21T15:00:40.027 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 10.082787808999999, 2024-09-21T15:00:40.027 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034165500000000001, 2024-09-21T15:00:40.027 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:40.027 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:40.027 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:40.027 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:40.027 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:40.027 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 118514 2024-09-21T15:00:40.028 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.028 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:40.028 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.028 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:40.028 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893261+0000", 2024-09-21T15:00:40.028 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:40.028 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.028 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.028 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:40.028 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893261+0000", 2024-09-21T15:00:40.028 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:40.028 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.028 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.028 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:40.028 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893261+0000", 2024-09-21T15:00:40.029 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:40.029 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.029 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.029 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:40.029 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893262+0000", 2024-09-21T15:00:40.029 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.787e-06 2024-09-21T15:00:40.029 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.029 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.029 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:40.029 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893267+0000", 2024-09-21T15:00:40.029 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6399999999999996e-06 2024-09-21T15:00:40.029 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.029 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.029 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:40.030 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893270+0000", 2024-09-21T15:00:40.030 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9610000000000001e-06 2024-09-21T15:00:40.030 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.030 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.030 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:40.030 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.927271+0000", 2024-09-21T15:00:40.030 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034001253000000002 2024-09-21T15:00:40.030 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.030 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.030 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:40.030 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.927344+0000", 2024-09-21T15:00:40.030 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2962999999999999e-05 2024-09-21T15:00:40.030 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.031 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.031 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:40.031 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.927427+0000", 2024-09-21T15:00:40.031 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2292000000000004e-05 2024-09-21T15:00:40.031 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:40.031 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:40.031 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:40.031 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.031 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.031 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:118515 3.12 3:4efbaa3f:::rbd_data.1136b1194d20.0000000000000006:58 [read 10313728~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:40.031 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.893273+0000", 2024-09-21T15:00:40.031 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 10.082775434, 2024-09-21T15:00:40.031 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034333437000000001, 2024-09-21T15:00:40.031 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:40.032 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:40.032 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:40.032 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:40.032 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:40.032 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 118515 2024-09-21T15:00:40.032 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.032 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:40.032 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.032 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:40.032 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893273+0000", 2024-09-21T15:00:40.032 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:40.032 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.032 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.032 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:40.032 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893273+0000", 2024-09-21T15:00:40.033 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:40.033 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.033 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.033 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:40.033 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893273+0000", 2024-09-21T15:00:40.033 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:40.033 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.033 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.033 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:40.033 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893275+0000", 2024-09-21T15:00:40.033 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6789999999999999e-06 2024-09-21T15:00:40.033 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.033 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.033 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:40.033 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893278+0000", 2024-09-21T15:00:40.034 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2440000000000002e-06 2024-09-21T15:00:40.034 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.034 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.034 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:40.034 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893282+0000", 2024-09-21T15:00:40.034 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.89e-06 2024-09-21T15:00:40.034 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.034 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.034 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:40.034 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.927454+0000", 2024-09-21T15:00:40.034 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034172077000000002 2024-09-21T15:00:40.034 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.034 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.035 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:40.035 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.927525+0000", 2024-09-21T15:00:40.035 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0963999999999993e-05 2024-09-21T15:00:40.035 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.035 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.035 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:40.035 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.927607+0000", 2024-09-21T15:00:40.035 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1868999999999998e-05 2024-09-21T15:00:40.035 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:40.035 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:40.035 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:40.035 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.035 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.035 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:118516 3.12 3:4efbaa3f:::rbd_data.1136b1194d20.0000000000000006:58 [read 10403840~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:40.036 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.893284+0000", 2024-09-21T15:00:40.036 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 10.082764492999999, 2024-09-21T15:00:40.036 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034589559999999998, 2024-09-21T15:00:40.036 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:40.036 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:40.036 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:40.036 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:40.036 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:40.036 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 118516 2024-09-21T15:00:40.036 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.036 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:40.036 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.036 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:40.036 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893284+0000", 2024-09-21T15:00:40.036 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:40.037 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.037 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.037 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:40.037 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893284+0000", 2024-09-21T15:00:40.037 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:40.037 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.037 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.037 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:40.037 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893284+0000", 2024-09-21T15:00:40.037 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:40.037 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.037 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.037 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:40.037 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893291+0000", 2024-09-21T15:00:40.037 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6200000000000001e-06 2024-09-21T15:00:40.038 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.038 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.038 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:40.038 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893294+0000", 2024-09-21T15:00:40.038 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6729999999999998e-06 2024-09-21T15:00:40.038 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.038 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.038 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:40.038 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893297+0000", 2024-09-21T15:00:40.038 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0510000000000001e-06 2024-09-21T15:00:40.038 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.038 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.038 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:40.038 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.927694+0000", 2024-09-21T15:00:40.039 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034397018000000001 2024-09-21T15:00:40.039 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.039 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.039 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:40.039 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.927767+0000", 2024-09-21T15:00:40.039 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2379000000000006e-05 2024-09-21T15:00:40.039 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.039 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.039 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:40.039 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.927874+0000", 2024-09-21T15:00:40.039 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010709000000000001 2024-09-21T15:00:40.039 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:40.039 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:40.039 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:40.040 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.040 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.040 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:118696 3.12 3:4efbaa3f:::rbd_data.1136b1194d20.0000000000000006:58 [read 9961472~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:40.040 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.902583+0000", 2024-09-21T15:00:40.040 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 10.073465446, 2024-09-21T15:00:40.040 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.033756517999999999, 2024-09-21T15:00:40.040 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:40.040 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:40.040 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:40.040 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:40.040 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:40.040 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 118696 2024-09-21T15:00:40.040 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.040 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:40.041 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.041 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:40.041 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902583+0000", 2024-09-21T15:00:40.041 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:40.041 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.041 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.041 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:40.041 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902583+0000", 2024-09-21T15:00:40.041 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:40.041 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.041 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.041 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:40.041 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902583+0000", 2024-09-21T15:00:40.041 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:40.041 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.041 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.042 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:40.042 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902585+0000", 2024-09-21T15:00:40.042 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7060000000000001e-06 2024-09-21T15:00:40.042 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.042 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.042 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:40.042 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902588+0000", 2024-09-21T15:00:40.042 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.128e-06 2024-09-21T15:00:40.042 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.042 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.042 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:40.042 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902592+0000", 2024-09-21T15:00:40.042 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0990000000000001e-06 2024-09-21T15:00:40.042 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.043 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.043 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:40.043 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.936161+0000", 2024-09-21T15:00:40.043 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.033569103000000003 2024-09-21T15:00:40.043 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.043 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.043 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:40.043 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.936241+0000", 2024-09-21T15:00:40.043 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9437000000000005e-05 2024-09-21T15:00:40.043 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.043 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.043 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:40.043 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.936340+0000", 2024-09-21T15:00:40.043 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9302999999999994e-05 2024-09-21T15:00:40.043 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:40.044 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:40.044 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:40.044 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.044 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.044 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:118697 3.12 3:4efbaa3f:::rbd_data.1136b1194d20.0000000000000006:58 [read 10051584~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:40.044 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.902595+0000", 2024-09-21T15:00:40.044 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 10.07345351, 2024-09-21T15:00:40.044 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.033875518, 2024-09-21T15:00:40.044 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:40.044 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:40.044 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:40.044 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:40.044 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:40.044 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 118697 2024-09-21T15:00:40.044 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.044 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:40.045 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.045 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:40.045 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902595+0000", 2024-09-21T15:00:40.045 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:40.045 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.045 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.045 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:40.045 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902595+0000", 2024-09-21T15:00:40.045 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:40.045 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.045 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.045 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:40.045 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902595+0000", 2024-09-21T15:00:40.045 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:40.045 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.046 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.046 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:40.046 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902597+0000", 2024-09-21T15:00:40.046 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7889999999999999e-06 2024-09-21T15:00:40.046 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.046 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.046 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:40.046 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902601+0000", 2024-09-21T15:00:40.046 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.16e-06 2024-09-21T15:00:40.046 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.046 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.046 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:40.046 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902605+0000", 2024-09-21T15:00:40.046 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7560000000000001e-06 2024-09-21T15:00:40.046 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.047 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.047 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:40.047 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.936370+0000", 2024-09-21T15:00:40.047 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.033765114999999998 2024-09-21T15:00:40.047 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.047 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.047 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:40.047 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.936413+0000", 2024-09-21T15:00:40.047 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3399000000000002e-05 2024-09-21T15:00:40.047 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.047 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.047 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:40.048 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.936471+0000", 2024-09-21T15:00:40.048 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7726000000000003e-05 2024-09-21T15:00:40.048 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:40.048 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:40.048 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:40.048 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.048 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.048 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:118698 3.12 3:4efbaa3f:::rbd_data.1136b1194d20.0000000000000006:58 [read 10141696~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:40.048 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.902607+0000", 2024-09-21T15:00:40.048 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 10.073441916, 2024-09-21T15:00:40.048 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034062306000000001, 2024-09-21T15:00:40.048 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:40.048 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:40.048 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:40.049 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:40.049 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:40.049 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 118698 2024-09-21T15:00:40.049 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.049 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:40.049 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.049 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:40.049 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902607+0000", 2024-09-21T15:00:40.049 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:40.049 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.049 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.049 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:40.049 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902607+0000", 2024-09-21T15:00:40.049 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:40.049 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.050 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.050 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:40.050 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902607+0000", 2024-09-21T15:00:40.050 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:40.050 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.050 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.050 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:40.050 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902608+0000", 2024-09-21T15:00:40.050 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5910000000000001e-06 2024-09-21T15:00:40.050 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.050 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.050 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:40.050 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902611+0000", 2024-09-21T15:00:40.050 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.965e-06 2024-09-21T15:00:40.050 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.051 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.051 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:40.051 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902615+0000", 2024-09-21T15:00:40.051 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0049999999999998e-06 2024-09-21T15:00:40.051 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.051 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.051 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:40.051 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.936501+0000", 2024-09-21T15:00:40.051 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.033885884999999998 2024-09-21T15:00:40.051 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.051 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.051 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:40.051 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.936584+0000", 2024-09-21T15:00:40.051 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2650000000000003e-05 2024-09-21T15:00:40.052 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.052 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.052 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:40.052 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.936669+0000", 2024-09-21T15:00:40.052 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5461000000000001e-05 2024-09-21T15:00:40.052 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:40.052 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:40.052 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:40.052 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.052 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.052 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:118699 3.12 3:4efbaa3f:::rbd_data.1136b1194d20.0000000000000006:58 [read 10231808~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:40.052 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.902625+0000", 2024-09-21T15:00:40.052 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 10.073423784999999, 2024-09-21T15:00:40.052 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034245128999999999, 2024-09-21T15:00:40.052 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:40.052 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:40.053 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:40.053 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:40.053 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:40.053 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 118699 2024-09-21T15:00:40.053 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.053 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:40.053 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.053 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:40.053 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902625+0000", 2024-09-21T15:00:40.053 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:40.053 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.053 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.053 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:40.053 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902625+0000", 2024-09-21T15:00:40.053 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:40.054 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.054 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.054 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:40.054 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902625+0000", 2024-09-21T15:00:40.054 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:40.054 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.054 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.054 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:40.054 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902627+0000", 2024-09-21T15:00:40.054 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7269999999999999e-06 2024-09-21T15:00:40.054 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.054 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.054 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:40.054 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902630+0000", 2024-09-21T15:00:40.054 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.5200000000000002e-06 2024-09-21T15:00:40.055 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.055 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.055 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:40.055 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902633+0000", 2024-09-21T15:00:40.055 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8399999999999999e-06 2024-09-21T15:00:40.055 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.055 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.055 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:40.055 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.936700+0000", 2024-09-21T15:00:40.055 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034066723 2024-09-21T15:00:40.055 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.055 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.055 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:40.055 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.936778+0000", 2024-09-21T15:00:40.055 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8713000000000003e-05 2024-09-21T15:00:40.056 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.056 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.056 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:40.056 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.936870+0000", 2024-09-21T15:00:40.056 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1885000000000005e-05 2024-09-21T15:00:40.056 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:40.056 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:40.056 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:40.056 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.056 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.056 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:118700 3.12 3:4efbaa3f:::rbd_data.1136b1194d20.0000000000000006:58 [read 10321920~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:40.056 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.902635+0000", 2024-09-21T15:00:40.056 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 10.073413578, 2024-09-21T15:00:40.056 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034423665999999999, 2024-09-21T15:00:40.056 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:40.057 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:40.057 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:40.057 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:40.057 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:40.057 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 118700 2024-09-21T15:00:40.057 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.057 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:40.057 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.057 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:40.057 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902635+0000", 2024-09-21T15:00:40.057 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:40.057 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.057 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.057 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:40.058 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902635+0000", 2024-09-21T15:00:40.058 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:40.058 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.058 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.058 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:40.058 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902635+0000", 2024-09-21T15:00:40.058 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:40.058 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.058 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.058 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:40.058 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902637+0000", 2024-09-21T15:00:40.058 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.734e-06 2024-09-21T15:00:40.058 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.058 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.058 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:40.059 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902640+0000", 2024-09-21T15:00:40.059 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9720000000000001e-06 2024-09-21T15:00:40.059 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.059 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.059 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:40.059 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902642+0000", 2024-09-21T15:00:40.059 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.616e-06 2024-09-21T15:00:40.059 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.059 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.059 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:40.059 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.936904+0000", 2024-09-21T15:00:40.059 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034261982000000003 2024-09-21T15:00:40.059 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.059 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.060 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:40.060 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.936974+0000", 2024-09-21T15:00:40.060 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9406000000000006e-05 2024-09-21T15:00:40.060 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.060 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.060 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:40.060 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.937059+0000", 2024-09-21T15:00:40.060 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5315999999999994e-05 2024-09-21T15:00:40.060 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:40.060 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:40.060 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:40.060 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.060 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.060 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:118701 3.12 3:4efbaa3f:::rbd_data.1136b1194d20.0000000000000006:58 [read 10412032~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:40.060 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.902645+0000", 2024-09-21T15:00:40.061 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 10.073404134, 2024-09-21T15:00:40.061 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034600791999999998, 2024-09-21T15:00:40.061 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:40.061 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:40.061 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:40.061 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:40.061 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:40.061 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 118701 2024-09-21T15:00:40.061 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.061 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:40.061 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.061 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:40.061 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902645+0000", 2024-09-21T15:00:40.061 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:40.061 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.061 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.062 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:40.062 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902645+0000", 2024-09-21T15:00:40.062 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:40.062 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.062 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.062 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:40.062 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902644+0000", 2024-09-21T15:00:40.062 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:40.062 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.062 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.062 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:40.062 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902646+0000", 2024-09-21T15:00:40.062 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6169999999999999e-06 2024-09-21T15:00:40.062 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.062 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.063 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:40.063 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902649+0000", 2024-09-21T15:00:40.063 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9450000000000002e-06 2024-09-21T15:00:40.063 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.063 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.063 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:40.063 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902652+0000", 2024-09-21T15:00:40.063 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7889999999999999e-06 2024-09-21T15:00:40.063 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.063 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.063 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:40.063 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.937086+0000", 2024-09-21T15:00:40.063 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034433832999999997 2024-09-21T15:00:40.063 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.063 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.064 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:40.064 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.937152+0000", 2024-09-21T15:00:40.064 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5934999999999998e-05 2024-09-21T15:00:40.064 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.064 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.064 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:40.064 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.937246+0000", 2024-09-21T15:00:40.064 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3923000000000002e-05 2024-09-21T15:00:40.064 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:40.064 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:40.064 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:40.064 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:40.064 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:40.064 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:40.066 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T15:00:40.066 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:00:40.061+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:00:40.167 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:00:40.165+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:00:40.204 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T15:00:40.240 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:40.240 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T15:00:40.240 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.240 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:129061 3.8 e154/93 3:1461ec96:::rbd_data.11361de34f8.000000000000004d:head v 154'556, mlcod=152'554)", 2024-09-21T15:00:40.240 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:40.238882+0000", 2024-09-21T15:00:40.240 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.000407601, 2024-09-21T15:00:40.240 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00042537899999999998, 2024-09-21T15:00:40.240 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:40.240 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:40.240 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:40.240 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.240 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:40.241 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:40.238882+0000", 2024-09-21T15:00:40.241 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:40.241 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.241 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.241 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:40.241 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:40.238882+0000", 2024-09-21T15:00:40.241 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:40.241 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.241 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.241 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:40.241 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:40.238882+0000", 2024-09-21T15:00:40.241 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:40.241 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.241 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.241 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:40.242 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:40.238960+0000", 2024-09-21T15:00:40.242 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8891000000000001e-05 2024-09-21T15:00:40.242 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.242 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.242 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:40.242 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:40.238982+0000", 2024-09-21T15:00:40.242 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1569999999999998e-05 2024-09-21T15:00:40.242 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.242 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.242 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:40.242 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:40.238998+0000", 2024-09-21T15:00:40.242 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5826999999999999e-05 2024-09-21T15:00:40.242 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.242 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.242 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:40.242 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:40.239075+0000", 2024-09-21T15:00:40.243 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6904999999999996e-05 2024-09-21T15:00:40.243 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.243 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.243 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:40.243 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:40.239095+0000", 2024-09-21T15:00:40.243 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0347999999999999e-05 2024-09-21T15:00:40.243 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:40.243 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:40.243 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:40.243 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:40.243 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-21T15:00:40.243 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 1 2024-09-21T15:00:40.243 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:40.250 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T15:00:40.267 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:00:40.263+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:00:40.355 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T15:00:40.357 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:40.357 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:00:40.357 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T15:00:40.357 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T15:00:40.357 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:40.367 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T15:00:40.368 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:00:40.363+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:00:40.378 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T15:00:40.396 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T15:00:40.402 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T15:00:40.421 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T15:00:40.443 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-21T15:00:40.465 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T15:00:40.468 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:00:40.463+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:00:40.475 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:40.475 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T15:00:40.475 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T15:00:40.475 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T15:00:40.476 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.476 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31476 3.c 3:35718b6b:::rbd_data.11367721c19.0000000000000014:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T15:00:40.476 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.505975+0000", 2024-09-21T15:00:40.476 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 101.96558004000001, 2024-09-21T15:00:40.476 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99515995899999998, 2024-09-21T15:00:40.476 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:40.476 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:40.476 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:40.476 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:40.476 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:40.476 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31476 2024-09-21T15:00:40.476 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.476 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:40.476 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.476 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:40.477 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.505975+0000", 2024-09-21T15:00:40.477 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:40.477 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.477 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.477 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:40.477 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.505975+0000", 2024-09-21T15:00:40.477 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:40.477 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.477 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.477 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:40.477 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.505971+0000", 2024-09-21T15:00:40.477 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999967 2024-09-21T15:00:40.477 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.477 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.477 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:40.478 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.505989+0000", 2024-09-21T15:00:40.478 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7985000000000001e-05 2024-09-21T15:00:40.478 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.478 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.478 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:40.478 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506050+0000", 2024-09-21T15:00:40.478 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0659000000000003e-05 2024-09-21T15:00:40.478 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.478 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.478 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:40.478 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506074+0000", 2024-09-21T15:00:40.478 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4156e-05 2024-09-21T15:00:40.478 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.478 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.478 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:40.479 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506225+0000", 2024-09-21T15:00:40.479 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015097 2024-09-21T15:00:40.479 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.479 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.479 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:00:40.479 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506256+0000", 2024-09-21T15:00:40.479 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0505999999999999e-05 2024-09-21T15:00:40.479 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.479 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.479 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:40.479 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.497974+0000", 2024-09-21T15:00:40.479 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99171825700000005 2024-09-21T15:00:40.479 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.479 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.479 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for missing object", 2024-09-21T15:00:40.479 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.498449+0000", 2024-09-21T15:00:40.480 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00047506399999999998 2024-09-21T15:00:40.480 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.480 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.480 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:40.480 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.501000+0000", 2024-09-21T15:00:40.480 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0025506919999999998 2024-09-21T15:00:40.480 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.480 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.480 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:40.480 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.501135+0000", 2024-09-21T15:00:40.480 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013524000000000001 2024-09-21T15:00:40.480 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:40.480 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:40.480 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:40.480 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.481 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.481 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31480 3.c 3:36ccd808:::rbd_data.11367721c19.0000000000000018:head [sparse-read 0~14821] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T15:00:40.481 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.506116+0000", 2024-09-21T15:00:40.481 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 101.965438697, 2024-09-21T15:00:40.481 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99316033599999998, 2024-09-21T15:00:40.481 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:40.481 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:40.481 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:40.481 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:40.481 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:40.481 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31480 2024-09-21T15:00:40.481 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.481 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:40.481 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.481 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:40.482 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506116+0000", 2024-09-21T15:00:40.482 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:40.482 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.482 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.482 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:40.482 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506116+0000", 2024-09-21T15:00:40.482 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:40.482 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.482 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.482 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:40.482 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506115+0000", 2024-09-21T15:00:40.482 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:00:40.482 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.483 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.483 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:40.483 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506124+0000", 2024-09-21T15:00:40.483 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0659999999999993e-06 2024-09-21T15:00:40.483 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.483 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.483 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:40.483 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506145+0000", 2024-09-21T15:00:40.483 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1447e-05 2024-09-21T15:00:40.483 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.483 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.483 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:40.483 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506161+0000", 2024-09-21T15:00:40.484 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5209e-05 2024-09-21T15:00:40.484 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.484 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.484 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:40.484 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506337+0000", 2024-09-21T15:00:40.484 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017642999999999999 2024-09-21T15:00:40.484 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.484 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.484 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:00:40.484 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506375+0000", 2024-09-21T15:00:40.484 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7536000000000001e-05 2024-09-21T15:00:40.484 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.484 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.484 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:40.484 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.498815+0000", 2024-09-21T15:00:40.485 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99244067999999996 2024-09-21T15:00:40.485 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.485 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.485 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:40.485 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.498950+0000", 2024-09-21T15:00:40.485 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013436700000000001 2024-09-21T15:00:40.485 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.485 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.485 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:40.485 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499277+0000", 2024-09-21T15:00:40.485 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00032696800000000002 2024-09-21T15:00:40.485 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:40.485 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:40.485 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:40.485 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.485 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.486 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31513 3.c 3:36de8523:::rbd_data.11367721c19.0000000000000039:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T15:00:40.486 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.509071+0000", 2024-09-21T15:00:40.486 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 101.96248449700001, 2024-09-21T15:00:40.486 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99038225099999999, 2024-09-21T15:00:40.486 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:40.486 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:40.486 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:40.486 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:40.486 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:40.486 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31513 2024-09-21T15:00:40.486 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.486 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:40.486 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.487 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:40.487 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509071+0000", 2024-09-21T15:00:40.487 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:40.487 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.487 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.487 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:40.487 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509071+0000", 2024-09-21T15:00:40.487 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:40.487 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.487 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.487 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:40.487 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509068+0000", 2024-09-21T15:00:40.487 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-21T15:00:40.487 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.488 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.488 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:40.488 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509082+0000", 2024-09-21T15:00:40.488 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3545e-05 2024-09-21T15:00:40.488 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.488 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.488 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:40.488 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509116+0000", 2024-09-21T15:00:40.488 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3942000000000001e-05 2024-09-21T15:00:40.488 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.488 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.488 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:40.488 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509134+0000", 2024-09-21T15:00:40.488 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8839999999999999e-05 2024-09-21T15:00:40.488 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.489 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.489 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:40.489 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509318+0000", 2024-09-21T15:00:40.489 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018326799999999999 2024-09-21T15:00:40.489 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.489 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.489 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:00:40.489 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509346+0000", 2024-09-21T15:00:40.489 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8019000000000001e-05 2024-09-21T15:00:40.489 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.489 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.489 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:40.489 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499330+0000", 2024-09-21T15:00:40.489 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.98998399299999995 2024-09-21T15:00:40.490 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.490 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.490 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:40.490 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499453+0000", 2024-09-21T15:00:40.490 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000123115 2024-09-21T15:00:40.490 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:40.490 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:40.490 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:40.490 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.490 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.490 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31515 3.c 3:35b36ddd:::rbd_data.11367721c19.000000000000003b:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T15:00:40.490 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.509177+0000", 2024-09-21T15:00:40.490 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 101.962378071, 2024-09-21T15:00:40.491 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99044870600000001, 2024-09-21T15:00:40.491 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:40.491 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:40.491 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:40.491 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:40.491 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:40.491 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31515 2024-09-21T15:00:40.491 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.491 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:40.491 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.491 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:40.491 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509177+0000", 2024-09-21T15:00:40.491 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:40.491 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.491 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.492 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:40.492 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509177+0000", 2024-09-21T15:00:40.492 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:40.492 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.492 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.492 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:40.492 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509176+0000", 2024-09-21T15:00:40.492 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:00:40.492 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.492 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.492 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:40.492 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509183+0000", 2024-09-21T15:00:40.492 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6720000000000004e-06 2024-09-21T15:00:40.492 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.492 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.492 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:40.493 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509269+0000", 2024-09-21T15:00:40.493 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5881e-05 2024-09-21T15:00:40.493 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.493 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.493 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:40.493 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509303+0000", 2024-09-21T15:00:40.493 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3367000000000001e-05 2024-09-21T15:00:40.493 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.493 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.493 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:40.493 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509425+0000", 2024-09-21T15:00:40.493 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000122199 2024-09-21T15:00:40.493 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.493 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.494 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:00:40.494 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509448+0000", 2024-09-21T15:00:40.494 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2688999999999999e-05 2024-09-21T15:00:40.494 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.494 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.494 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:40.494 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499501+0000", 2024-09-21T15:00:40.494 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99005306999999998 2024-09-21T15:00:40.494 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.494 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.494 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:40.494 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499626+0000", 2024-09-21T15:00:40.494 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012501400000000001 2024-09-21T15:00:40.495 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:40.495 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:40.495 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:40.495 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.495 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.495 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31531 3.c 3:314f0287:::rbd_data.11367721c19.000000000000004b:head [sparse-read 0~7106] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T15:00:40.495 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.509329+0000", 2024-09-21T15:00:40.495 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 101.962226051, 2024-09-21T15:00:40.495 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99103085300000004, 2024-09-21T15:00:40.495 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:40.495 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:40.495 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:40.495 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:40.495 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:40.495 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31531 2024-09-21T15:00:40.496 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.496 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:40.496 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.496 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:40.496 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509329+0000", 2024-09-21T15:00:40.496 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:40.496 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.496 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.496 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:40.496 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509329+0000", 2024-09-21T15:00:40.496 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:40.496 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.496 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.496 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:40.496 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509328+0000", 2024-09-21T15:00:40.497 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:00:40.497 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.497 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.497 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:40.497 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509336+0000", 2024-09-21T15:00:40.497 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3170000000000002e-06 2024-09-21T15:00:40.497 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.497 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.497 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:40.497 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509353+0000", 2024-09-21T15:00:40.497 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7214000000000002e-05 2024-09-21T15:00:40.497 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.497 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.497 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:40.498 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509366+0000", 2024-09-21T15:00:40.498 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.26e-05 2024-09-21T15:00:40.498 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.498 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.498 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:40.498 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509523+0000", 2024-09-21T15:00:40.498 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015708000000000001 2024-09-21T15:00:40.498 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.498 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.498 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:00:40.498 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509544+0000", 2024-09-21T15:00:40.498 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0783000000000001e-05 2024-09-21T15:00:40.498 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.499 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.499 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:40.499 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499727+0000", 2024-09-21T15:00:40.499 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99018345200000002 2024-09-21T15:00:40.499 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.499 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.499 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:40.499 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499949+0000", 2024-09-21T15:00:40.499 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000221358 2024-09-21T15:00:40.499 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.499 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.499 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:40.499 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.500360+0000", 2024-09-21T15:00:40.499 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00041130700000000002 2024-09-21T15:00:40.499 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:40.500 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:40.500 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:40.500 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.500 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.500 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:61580 3.b 3:d3369e4f:::rbd_data.1136862f872f.0000000000000010:head [write 0~557056 [fadvise_sequential+fadvise_nocache],zero 557056~16384,write 573440~49152 [fadvise_sequential+fadvise_nocache],zero 622592~16384,write 638976~49152 [fadvise_sequential+fadvise_nocache],zero 688128~16384,write 704512~49152 [fadvise_sequential+fadvise_nocache],zero 753664~16384,write 770048~49152 [fadvise_sequential+fadvise_nocache],zero 819200~16384,write 835584~49152 [fadvise_sequential+fadvise_nocache],zero 884736~16384,write 901120~49152 [fadvise_sequential+fadvise_nocache],zero 950272~16384,write 966656~49152 [fadvise_sequential+fadvise_nocache],zero 1015808~16384,write 1032192~16384 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-21T15:00:40.500 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:19.231716+0000", 2024-09-21T15:00:40.500 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 81.239838888999998, 2024-09-21T15:00:40.500 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0096097690000000006, 2024-09-21T15:00:40.500 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:40.500 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:40.500 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:40.500 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:40.500 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:40.500 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 61580 2024-09-21T15:00:40.500 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.500 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:40.501 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.501 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:40.501 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.231716+0000", 2024-09-21T15:00:40.501 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:40.501 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.501 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.501 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:40.501 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.231716+0000", 2024-09-21T15:00:40.501 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:40.501 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.501 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.501 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:40.501 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.231715+0000", 2024-09-21T15:00:40.501 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:00:40.501 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.502 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.502 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:40.502 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.235290+0000", 2024-09-21T15:00:40.502 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0035750399999999998 2024-09-21T15:00:40.502 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.502 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.502 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:40.502 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.235310+0000", 2024-09-21T15:00:40.502 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0120999999999999e-05 2024-09-21T15:00:40.502 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.502 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.502 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:40.502 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.235319+0000", 2024-09-21T15:00:40.502 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2420000000000006e-06 2024-09-21T15:00:40.502 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.503 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.503 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:40.503 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.235416+0000", 2024-09-21T15:00:40.503 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7204e-05 2024-09-21T15:00:40.503 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.503 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.503 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:40.503 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.235775+0000", 2024-09-21T15:00:40.503 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000358883 2024-09-21T15:00:40.503 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.503 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.503 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T15:00:40.503 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.236560+0000", 2024-09-21T15:00:40.503 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000784463 2024-09-21T15:00:40.503 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.503 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.504 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:40.504 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.241119+0000", 2024-09-21T15:00:40.504 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0045590589999999999 2024-09-21T15:00:40.504 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.504 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.504 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:40.504 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.241210+0000", 2024-09-21T15:00:40.504 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1617000000000001e-05 2024-09-21T15:00:40.504 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.504 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.504 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:40.504 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.241285+0000", 2024-09-21T15:00:40.504 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4470999999999999e-05 2024-09-21T15:00:40.504 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.505 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.505 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:40.505 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.241326+0000", 2024-09-21T15:00:40.505 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0967000000000003e-05 2024-09-21T15:00:40.505 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:40.505 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:40.505 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:40.505 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.505 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.505 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:73699 3.b 3:d293b54a:::rbd_data.1136f4939602.000000000000000a:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-21T15:00:40.505 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:41.971359+0000", 2024-09-21T15:00:40.505 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 58.500196332999998, 2024-09-21T15:00:40.505 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0082840980000000002, 2024-09-21T15:00:40.506 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:40.506 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:40.506 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:40.506 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:40.506 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:40.506 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 73699 2024-09-21T15:00:40.506 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.506 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:40.506 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.506 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:40.506 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.971359+0000", 2024-09-21T15:00:40.506 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:40.506 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.506 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.506 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:40.507 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.971359+0000", 2024-09-21T15:00:40.507 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:40.507 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.507 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.507 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:40.507 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.971356+0000", 2024-09-21T15:00:40.507 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-21T15:00:40.507 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.507 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.507 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:40.507 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.971370+0000", 2024-09-21T15:00:40.507 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3349999999999999e-05 2024-09-21T15:00:40.507 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.507 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.507 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:40.508 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.971393+0000", 2024-09-21T15:00:40.508 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3799999999999999e-05 2024-09-21T15:00:40.508 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.508 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.508 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:40.508 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.971405+0000", 2024-09-21T15:00:40.508 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1569000000000001e-05 2024-09-21T15:00:40.508 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.508 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.508 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:40.508 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.971510+0000", 2024-09-21T15:00:40.508 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010456899999999999 2024-09-21T15:00:40.508 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.508 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.509 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:40.509 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.971752+0000", 2024-09-21T15:00:40.509 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00024250300000000001 2024-09-21T15:00:40.509 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.509 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.509 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T15:00:40.509 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.973012+0000", 2024-09-21T15:00:40.509 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0012595429999999999 2024-09-21T15:00:40.509 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.509 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.509 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:40.509 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.979357+0000", 2024-09-21T15:00:40.509 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0063454469999999997 2024-09-21T15:00:40.509 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.509 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.510 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:40.510 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.979467+0000", 2024-09-21T15:00:40.510 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000109623 2024-09-21T15:00:40.510 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.510 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.510 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:40.510 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.979585+0000", 2024-09-21T15:00:40.510 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000118044 2024-09-21T15:00:40.510 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.510 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.510 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:40.510 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.979643+0000", 2024-09-21T15:00:40.510 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8069999999999998e-05 2024-09-21T15:00:40.511 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:40.511 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:40.511 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:40.511 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.511 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.511 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:82545 3.b 3:d4eb1ee5:::rbd_data.11368296f791.0000000000000000:head [write 0~671744 [fadvise_sequential+fadvise_nocache],write 704512~65536 [fadvise_sequential+fadvise_nocache],write 786432~49152 [fadvise_sequential+fadvise_nocache],write 868352~49152 [fadvise_sequential+fadvise_nocache],write 933888~65536 [fadvise_sequential+fadvise_nocache],write 1032192~32768 [fadvise_sequential+fadvise_nocache],write 1097728~65536 [fadvise_sequential+fadvise_nocache],write 1179648~49152 [fadvise_sequential+fadvise_nocache],write 1261568~49152 [fadvise_sequential+fadvise_nocache],write 1327104~65536 [fadvise_sequential+fadvise_nocache],write 1425408~32768 [fadvise_sequential+fadvise_nocache],write 1490944~65536 [fadvise_sequential+fadvise_nocache],write 1572864~8192 [fadvise_sequential+fadvise_nocache],write 1630208~49152 [fadvise_sequential+fadvise_nocache],write 1703936~40960 [fadvise_sequential+fadvise_nocache],write 1794048~40960 [fadvise_sequential+fadvise_nocache],write 1859584~49152 [fadvise_sequential+fadvise_nocache],write 1957888~8192 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:00:40.511 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:47.416089+0000", 2024-09-21T15:00:40.511 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 53.055466326000001, 2024-09-21T15:00:40.511 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011896455, 2024-09-21T15:00:40.511 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:40.511 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:40.511 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:40.511 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:40.511 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:40.512 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82545 2024-09-21T15:00:40.512 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.512 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:40.512 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.512 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:40.512 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.416089+0000", 2024-09-21T15:00:40.512 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:40.512 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.512 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.512 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:40.512 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.416089+0000", 2024-09-21T15:00:40.512 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:40.512 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.512 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.513 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:40.513 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.416087+0000", 2024-09-21T15:00:40.513 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:00:40.513 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.513 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.513 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:40.513 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.419643+0000", 2024-09-21T15:00:40.513 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0035555550000000002 2024-09-21T15:00:40.513 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.513 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.513 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:40.513 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.419663+0000", 2024-09-21T15:00:40.513 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0202000000000001e-05 2024-09-21T15:00:40.513 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.513 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.514 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:40.514 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.419676+0000", 2024-09-21T15:00:40.514 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3207000000000001e-05 2024-09-21T15:00:40.514 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.514 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.514 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:40.514 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.419759+0000", 2024-09-21T15:00:40.514 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3521000000000002e-05 2024-09-21T15:00:40.514 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.514 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.514 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op must wait for map", 2024-09-21T15:00:40.514 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.419768+0000", 2024-09-21T15:00:40.514 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4479999999999994e-06 2024-09-21T15:00:40.514 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.515 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.515 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:40.515 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.422171+0000", 2024-09-21T15:00:40.515 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0024035469999999998 2024-09-21T15:00:40.515 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.515 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.515 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:40.515 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.422350+0000", 2024-09-21T15:00:40.515 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000179001 2024-09-21T15:00:40.515 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.515 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.515 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T15:00:40.515 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.422710+0000", 2024-09-21T15:00:40.515 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000359878 2024-09-21T15:00:40.515 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.516 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.516 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:40.516 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.426676+0000", 2024-09-21T15:00:40.516 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0039655790000000003 2024-09-21T15:00:40.516 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.516 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.516 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:40.516 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.427884+0000", 2024-09-21T15:00:40.516 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001208207 2024-09-21T15:00:40.516 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.516 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.516 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:40.516 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.427953+0000", 2024-09-21T15:00:40.516 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8533000000000003e-05 2024-09-21T15:00:40.516 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.517 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.517 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:40.517 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.427985+0000", 2024-09-21T15:00:40.517 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2481999999999998e-05 2024-09-21T15:00:40.517 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:40.517 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:40.517 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:40.517 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.517 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.517 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:83735 3.b 3:d3640016:::rbd_data.11368296f791.0000000000000006:40 [sparse-read 393216~131072 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-21T15:00:40.517 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:51.156767+0000", 2024-09-21T15:00:40.517 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 49.314787703, 2024-09-21T15:00:40.517 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0082826410000000003, 2024-09-21T15:00:40.517 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:40.517 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:40.518 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:40.518 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:40.518 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:40.518 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 83735 2024-09-21T15:00:40.518 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.518 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:40.518 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.518 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:40.518 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156767+0000", 2024-09-21T15:00:40.518 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:40.518 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.518 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.518 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:40.518 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156767+0000", 2024-09-21T15:00:40.519 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:40.519 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.519 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.519 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:40.519 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156767+0000", 2024-09-21T15:00:40.519 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:40.519 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.519 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.519 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:40.519 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156771+0000", 2024-09-21T15:00:40.519 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0940000000000001e-06 2024-09-21T15:00:40.519 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.519 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.519 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:40.519 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156781+0000", 2024-09-21T15:00:40.520 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0267e-05 2024-09-21T15:00:40.520 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.520 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.520 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:40.520 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156789+0000", 2024-09-21T15:00:40.520 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7659999999999998e-06 2024-09-21T15:00:40.520 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.520 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.520 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:40.520 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.164742+0000", 2024-09-21T15:00:40.520 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0079534519999999997 2024-09-21T15:00:40.520 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.520 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.521 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:40.521 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.164820+0000", 2024-09-21T15:00:40.521 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7927000000000001e-05 2024-09-21T15:00:40.521 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.521 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.521 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:40.521 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165050+0000", 2024-09-21T15:00:40.521 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000229731 2024-09-21T15:00:40.521 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:40.521 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:40.521 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:40.521 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.521 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.521 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:83736 3.b 3:d3640016:::rbd_data.11368296f791.0000000000000006:40 [sparse-read 1835008~0 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-21T15:00:40.521 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:51.156796+0000", 2024-09-21T15:00:40.522 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 49.314758699999999, 2024-09-21T15:00:40.522 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0085139110000000007, 2024-09-21T15:00:40.522 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:40.522 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:40.522 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:40.522 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:40.522 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:40.522 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 83736 2024-09-21T15:00:40.522 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.522 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:40.522 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.522 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:40.522 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156796+0000", 2024-09-21T15:00:40.522 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:40.522 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.523 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.523 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:40.523 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156796+0000", 2024-09-21T15:00:40.523 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:40.523 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.523 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.523 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:40.523 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156796+0000", 2024-09-21T15:00:40.523 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:40.523 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.523 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.523 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:40.523 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156800+0000", 2024-09-21T15:00:40.523 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.34e-06 2024-09-21T15:00:40.523 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.523 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.524 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:40.524 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156813+0000", 2024-09-21T15:00:40.524 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2532e-05 2024-09-21T15:00:40.524 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.524 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.524 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:40.524 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156821+0000", 2024-09-21T15:00:40.524 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2069999999999995e-06 2024-09-21T15:00:40.524 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.524 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.524 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:40.524 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165100+0000", 2024-09-21T15:00:40.524 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0082788730000000008 2024-09-21T15:00:40.524 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.524 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.524 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:40.525 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165192+0000", 2024-09-21T15:00:40.525 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2095999999999994e-05 2024-09-21T15:00:40.525 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.525 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.525 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:40.525 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165310+0000", 2024-09-21T15:00:40.525 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000118461 2024-09-21T15:00:40.525 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:40.525 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:40.525 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:40.525 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.525 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.525 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:83739 3.b 3:d4eb1ee5:::rbd_data.11368296f791.0000000000000000:40 [sparse-read 524288~131072 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-21T15:00:40.525 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:51.156827+0000", 2024-09-21T15:00:40.526 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 49.314727617000003, 2024-09-21T15:00:40.526 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0087319900000000002, 2024-09-21T15:00:40.526 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:40.526 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:40.526 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:40.526 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:40.526 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:40.526 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 83739 2024-09-21T15:00:40.526 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.526 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:40.526 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.526 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:40.526 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156827+0000", 2024-09-21T15:00:40.526 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:40.526 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.527 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.527 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:40.527 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156827+0000", 2024-09-21T15:00:40.527 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:40.527 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.527 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.527 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:40.527 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156827+0000", 2024-09-21T15:00:40.527 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:40.527 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.527 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.527 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:40.527 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156831+0000", 2024-09-21T15:00:40.527 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9940000000000002e-06 2024-09-21T15:00:40.527 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.528 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.528 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:40.528 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156841+0000", 2024-09-21T15:00:40.528 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6760000000000002e-06 2024-09-21T15:00:40.528 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.528 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.528 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:40.528 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156853+0000", 2024-09-21T15:00:40.528 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1972e-05 2024-09-21T15:00:40.528 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.528 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.528 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:40.528 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165334+0000", 2024-09-21T15:00:40.528 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0084816939999999997 2024-09-21T15:00:40.528 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.529 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.529 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:40.529 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165375+0000", 2024-09-21T15:00:40.529 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0911000000000002e-05 2024-09-21T15:00:40.529 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.529 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.529 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:40.529 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165559+0000", 2024-09-21T15:00:40.529 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018429300000000001 2024-09-21T15:00:40.529 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:40.529 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:40.529 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:40.529 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.529 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.530 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:83740 3.b 3:d4eb1ee5:::rbd_data.11368296f791.0000000000000000:40 [sparse-read 1966080~0 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-21T15:00:40.530 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:51.156859+0000", 2024-09-21T15:00:40.530 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 49.314695626999999, 2024-09-21T15:00:40.530 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0088540089999999995, 2024-09-21T15:00:40.530 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:40.530 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:40.530 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:40.530 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:40.530 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:40.530 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 83740 2024-09-21T15:00:40.530 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.530 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:40.530 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.530 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:40.530 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156859+0000", 2024-09-21T15:00:40.531 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:40.531 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.531 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.531 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:40.531 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156859+0000", 2024-09-21T15:00:40.531 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:40.531 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.531 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.531 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:40.531 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156859+0000", 2024-09-21T15:00:40.531 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:40.531 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.531 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.531 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:40.531 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156863+0000", 2024-09-21T15:00:40.531 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.844e-06 2024-09-21T15:00:40.532 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.532 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.532 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:40.532 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156873+0000", 2024-09-21T15:00:40.532 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0069e-05 2024-09-21T15:00:40.532 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.532 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.532 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:40.532 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156880+0000", 2024-09-21T15:00:40.532 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4240000000000004e-06 2024-09-21T15:00:40.532 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.532 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.532 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:40.532 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165590+0000", 2024-09-21T15:00:40.533 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0087098100000000001 2024-09-21T15:00:40.533 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.533 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.533 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:40.533 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165656+0000", 2024-09-21T15:00:40.533 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5541000000000001e-05 2024-09-21T15:00:40.533 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.533 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.533 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:40.533 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165713+0000", 2024-09-21T15:00:40.533 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7865000000000002e-05 2024-09-21T15:00:40.533 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:40.533 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:40.533 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:40.534 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.534 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.534 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:83751 3.b 3:d3640016:::rbd_data.11368296f791.0000000000000006:40 [sparse-read 1179648~0 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-21T15:00:40.534 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:51.156888+0000", 2024-09-21T15:00:40.534 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 49.314667366000002, 2024-09-21T15:00:40.534 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0090039539999999998, 2024-09-21T15:00:40.534 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:40.534 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:40.534 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:40.534 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:40.534 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:40.534 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 83751 2024-09-21T15:00:40.534 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.534 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:40.534 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.534 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:40.535 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156888+0000", 2024-09-21T15:00:40.535 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:40.535 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.535 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.535 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:40.535 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156888+0000", 2024-09-21T15:00:40.535 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:40.535 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.535 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.535 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:40.535 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156887+0000", 2024-09-21T15:00:40.535 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:40.535 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.535 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.536 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:40.536 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156891+0000", 2024-09-21T15:00:40.536 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.518e-06 2024-09-21T15:00:40.536 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.536 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.536 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:40.536 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156900+0000", 2024-09-21T15:00:40.536 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1759999999999999e-06 2024-09-21T15:00:40.536 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.536 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.536 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:40.536 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156908+0000", 2024-09-21T15:00:40.536 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8439999999999998e-06 2024-09-21T15:00:40.536 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.536 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.536 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:40.537 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165738+0000", 2024-09-21T15:00:40.537 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0088305689999999999 2024-09-21T15:00:40.537 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.537 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.537 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:40.537 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165804+0000", 2024-09-21T15:00:40.537 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5535000000000002e-05 2024-09-21T15:00:40.537 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.537 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.537 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:40.537 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165892+0000", 2024-09-21T15:00:40.537 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7860000000000002e-05 2024-09-21T15:00:40.537 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:40.537 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:40.537 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:40.538 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.538 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.538 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:83752 3.b 3:d3640016:::rbd_data.11368296f791.0000000000000006:40 [sparse-read 2621440~0 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-21T15:00:40.538 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:51.156915+0000", 2024-09-21T15:00:40.538 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 49.314639819, 2024-09-21T15:00:40.538 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0090768180000000004, 2024-09-21T15:00:40.538 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:40.538 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:40.538 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:40.538 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:40.538 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:40.538 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 83752 2024-09-21T15:00:40.538 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.538 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:40.538 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.539 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:40.539 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156915+0000", 2024-09-21T15:00:40.539 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:40.539 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.539 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.539 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:40.539 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156915+0000", 2024-09-21T15:00:40.539 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:40.539 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.539 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.539 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:40.539 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156915+0000", 2024-09-21T15:00:40.539 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:40.539 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.540 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.540 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:40.540 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156919+0000", 2024-09-21T15:00:40.540 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2679999999999996e-06 2024-09-21T15:00:40.540 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.540 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.540 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:40.540 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156929+0000", 2024-09-21T15:00:40.540 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.031e-05 2024-09-21T15:00:40.540 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.540 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.540 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:40.540 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156937+0000", 2024-09-21T15:00:40.540 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8820000000000001e-06 2024-09-21T15:00:40.540 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.541 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.541 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:40.541 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165909+0000", 2024-09-21T15:00:40.541 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0089722989999999996 2024-09-21T15:00:40.541 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.541 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.541 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:40.541 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165948+0000", 2024-09-21T15:00:40.541 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8114999999999997e-05 2024-09-21T15:00:40.541 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.541 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.541 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:40.541 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165992+0000", 2024-09-21T15:00:40.541 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4503999999999999e-05 2024-09-21T15:00:40.541 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:40.541 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:40.542 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:40.542 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.542 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.542 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:83755 3.b 3:d4eb1ee5:::rbd_data.11368296f791.0000000000000000:40 [sparse-read 1310720~0 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-21T15:00:40.542 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:51.156944+0000", 2024-09-21T15:00:40.542 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 49.314610745000003, 2024-09-21T15:00:40.542 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.009159146, 2024-09-21T15:00:40.542 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:40.542 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:40.542 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:40.542 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:40.542 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:40.542 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 83755 2024-09-21T15:00:40.542 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.542 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:40.542 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.543 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:40.543 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156944+0000", 2024-09-21T15:00:40.543 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:40.543 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.543 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.543 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:40.543 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156944+0000", 2024-09-21T15:00:40.543 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:40.543 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.543 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.543 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:40.543 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156944+0000", 2024-09-21T15:00:40.543 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:40.543 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.543 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.544 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:40.544 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156948+0000", 2024-09-21T15:00:40.544 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5279999999999997e-06 2024-09-21T15:00:40.544 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.544 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.544 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:40.544 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156959+0000", 2024-09-21T15:00:40.544 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0215e-05 2024-09-21T15:00:40.544 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.544 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.544 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:40.544 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156967+0000", 2024-09-21T15:00:40.544 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6119999999999999e-06 2024-09-21T15:00:40.544 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.544 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.544 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:40.545 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.166007+0000", 2024-09-21T15:00:40.545 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0090395260000000009 2024-09-21T15:00:40.545 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.545 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.545 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:40.545 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.166053+0000", 2024-09-21T15:00:40.545 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6281e-05 2024-09-21T15:00:40.545 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.545 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.545 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:40.545 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.166103+0000", 2024-09-21T15:00:40.545 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0510000000000003e-05 2024-09-21T15:00:40.545 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:40.545 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:40.545 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:40.546 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.546 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.546 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:83763 3.b 3:d3640016:::rbd_data.11368296f791.0000000000000006:40 [sparse-read 524288~81920 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-21T15:00:40.546 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:51.157975+0000", 2024-09-21T15:00:40.546 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 49.313579636999997, 2024-09-21T15:00:40.546 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.008327348, 2024-09-21T15:00:40.546 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:40.546 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:40.546 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:40.546 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:40.546 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:40.546 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 83763 2024-09-21T15:00:40.546 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.546 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:40.546 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.547 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:40.547 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.157975+0000", 2024-09-21T15:00:40.547 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:40.547 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.547 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.547 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:40.547 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.157975+0000", 2024-09-21T15:00:40.547 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:40.547 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.547 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.547 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:40.547 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.157970+0000", 2024-09-21T15:00:40.547 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999943 2024-09-21T15:00:40.547 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.547 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.548 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:40.548 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.157985+0000", 2024-09-21T15:00:40.548 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4807000000000001e-05 2024-09-21T15:00:40.548 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.548 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.548 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:40.548 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.158012+0000", 2024-09-21T15:00:40.548 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7600999999999999e-05 2024-09-21T15:00:40.548 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.548 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.548 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:40.548 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.158019+0000", 2024-09-21T15:00:40.548 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9319999999999999e-06 2024-09-21T15:00:40.548 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.548 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.548 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:40.549 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.166118+0000", 2024-09-21T15:00:40.549 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0080992489999999993 2024-09-21T15:00:40.549 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.549 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.549 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:40.549 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.166156+0000", 2024-09-21T15:00:40.549 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7639999999999999e-05 2024-09-21T15:00:40.549 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.549 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.549 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:40.549 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.166303+0000", 2024-09-21T15:00:40.549 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000146799 2024-09-21T15:00:40.549 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:40.549 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:40.550 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:40.550 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.550 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.550 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:83764 3.b 3:d3640016:::rbd_data.11368296f791.0000000000000006:40 [sparse-read 1966080~0 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-21T15:00:40.550 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:51.158026+0000", 2024-09-21T15:00:40.550 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 49.313529203999998, 2024-09-21T15:00:40.550 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0083785839999999997, 2024-09-21T15:00:40.550 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:40.550 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:40.550 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:40.550 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:40.550 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:40.550 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 83764 2024-09-21T15:00:40.550 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.550 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:40.551 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.551 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:40.551 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.158026+0000", 2024-09-21T15:00:40.551 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:40.551 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.551 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.551 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:40.551 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.158026+0000", 2024-09-21T15:00:40.551 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:40.551 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.551 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.551 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:40.551 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.158025+0000", 2024-09-21T15:00:40.551 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:40.551 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.552 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.552 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:40.552 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.158053+0000", 2024-09-21T15:00:40.552 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7594e-05 2024-09-21T15:00:40.552 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.552 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.552 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:40.552 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.158061+0000", 2024-09-21T15:00:40.552 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2069999999999995e-06 2024-09-21T15:00:40.552 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.552 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.552 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:40.552 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.158067+0000", 2024-09-21T15:00:40.552 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3799999999999999e-06 2024-09-21T15:00:40.552 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.552 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.553 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:40.553 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.166321+0000", 2024-09-21T15:00:40.553 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0082539870000000008 2024-09-21T15:00:40.553 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.553 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.553 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:40.553 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.166360+0000", 2024-09-21T15:00:40.553 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8597999999999999e-05 2024-09-21T15:00:40.553 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.553 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.553 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:40.553 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.166404+0000", 2024-09-21T15:00:40.553 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4378000000000001e-05 2024-09-21T15:00:40.553 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:40.553 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:40.554 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:40.554 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.554 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.554 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:97675 3.b 3:d321acb8:::rbd_data.1136bfc79a70.0000000000000003:head [write 0~4718592 [fadvise_sequential+fadvise_nocache],write 5242880~147456 [fadvise_sequential+fadvise_nocache],write 5455872~147456 [fadvise_sequential+fadvise_nocache],write 5668864~229376 [fadvise_sequential+fadvise_nocache],write 5963776~147456 [fadvise_sequential+fadvise_nocache],write 6176768~229376 [fadvise_sequential+fadvise_nocache],write 6471680~147456 [fadvise_sequential+fadvise_nocache],write 6684672~229376 [fadvise_sequential+fadvise_nocache],write 6979584~147456 [fadvise_sequential+fadvise_nocache],write 7192576~229376 [fadvise_sequential+fadvise_nocache],write 7487488~147456 [fadvise_sequential+fadvise_nocache],write 7700480~147456 [fadvise_sequential+fadvise_nocache],write 7864320~65536 [fadvise_sequential+fadvise_nocache],write 7995392~147456 [fadvise_sequential+fadvise_nocache],write 8208384~147456 [fadvise_sequential+fadvise_nocache],write 8388608~49152 [fadvise_sequential+fadvise_nocache],write 8536064~114688 [fadvise_sequential+fadvise_nocache],write 8749056~114688 [fadvise_sequential+fadvise_nocache],write 8912896~32768 [fadvise_sequential+fadvise_nocache],write 9125888~32768 [fadvise_sequential+fadvise_nocache],write 9338880~32768 [fadvise_sequential+fadvise_nocache],write 9437184~16384 [fadvise_sequential+fadvise_nocache],write 9633792~32768 [fadvise_sequential+fadvise_nocache],write 9846784~32768 [fadvise_sequential+fadvise_nocache],write 9961472~114688 [fadvise_sequential+fadvise_nocache],write 10092544~196608 [fadvise_sequential+fadvise_nocache],write 10305536~278528 [fadvise_sequential+fadvise_nocache],write 10600448~196608 [fadvise_sequential+fadvise_nocache],write 10813440~278528 [fadvise_sequential+fadvise_nocache],write 11108352~196608 [fadvise_sequential+fadvise_nocache],write 11321344~196608 [fadvise_sequential+fadvise_nocache],write 11534336~65536 [fadvise_sequential+fadvise_nocache],write 11632640~98304 [fadvise_sequential+fadvise_nocache],write 11747328~65536 [fadvise_sequential+fadvise_nocache],write 11845632~98304 [fadvise_sequential+fadvise_nocache],write 11960320~65536 [fadvise_sequential+fadvise_nocache],write 12058624~49152 [fadvise_sequential+fadvise_nocache],write 12140544~98304 [fadvise_sequential+fadvise_nocache],write 12255232~65536 [fadvise_sequential+fadvise_nocache],write 12353536~98304 [fadvise_sequential+fadvise_nocache],write 12468224~65536 [fadvise_sequential+fadvise_nocache],write 12566528~16384 [fadvise_sequential+fadvise_nocache],write 12648448~98304 [fadvise_sequential+fadvise_nocache],write 12861440~98304 [fadvise_sequential+fadvise_nocache],write 13074432~32768 [fadvise_sequential+fadvise_nocache],write 13156352~98304 [fadvise_sequential+fadvise_nocache],write 13369344~98304 [fadvise_sequential+fadvise_nocache],write 13582336~49152 [fadvise_sequential+fadvise_nocache],write 13664256~98304 [fadvise_sequential+fadvise_nocache],write 13877248~98304 [fadvise_sequential+fadvise_nocache],write 14090240~65536 [fadvise_sequential+fadvise_nocache],write 15286272~81920 [fadvise_sequential+fadvise_nocache],write 15499264~81920 [fadvise_sequential+fadvise_nocache],write 15712256~16384 [fadvise_sequential+fadvise_nocache],write 15794176~81920 [fadvise_sequential+fadvise_nocache],write 16007168~81920 [fadvise_sequential+fadvise_nocache],write 16220160~32768 [fadvise_sequential+fadvise_nocache],write 16302080~81920 [fadvise_sequential+fadvise_nocache],write 16515072~81920 [fadvise_sequential+fadvise_nocache],write 16728064~49152 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:40.554 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.302412+0000", 2024-09-21T15:00:40.554 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 35.169143536999997, 2024-09-21T15:00:40.554 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.067380532000000007, 2024-09-21T15:00:40.554 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:40.554 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:40.554 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:40.554 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:40.554 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:40.554 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97675 2024-09-21T15:00:40.554 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.554 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:40.555 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.555 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:40.555 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.302412+0000", 2024-09-21T15:00:40.555 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:40.555 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.555 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.555 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:40.555 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.302412+0000", 2024-09-21T15:00:40.555 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:40.555 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.555 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.555 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:40.555 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.302409+0000", 2024-09-21T15:00:40.555 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-21T15:00:40.556 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.556 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.556 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:40.556 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328550+0000", 2024-09-21T15:00:40.556 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.026140674999999999 2024-09-21T15:00:40.556 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.556 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.556 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:40.556 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328584+0000", 2024-09-21T15:00:40.556 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4165999999999998e-05 2024-09-21T15:00:40.556 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.556 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.556 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:40.556 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328599+0000", 2024-09-21T15:00:40.556 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5505000000000001e-05 2024-09-21T15:00:40.556 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.557 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.557 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:40.557 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328704+0000", 2024-09-21T15:00:40.557 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010487199999999999 2024-09-21T15:00:40.557 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.557 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.557 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:40.557 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.329082+0000", 2024-09-21T15:00:40.557 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00037804899999999998 2024-09-21T15:00:40.557 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.557 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.557 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T15:00:40.557 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.331158+0000", 2024-09-21T15:00:40.557 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0020755449999999998 2024-09-21T15:00:40.557 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.558 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.558 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:40.558 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.360553+0000", 2024-09-21T15:00:40.558 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.029395553000000001 2024-09-21T15:00:40.558 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.558 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.558 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:40.558 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.369496+0000", 2024-09-21T15:00:40.558 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0089428619999999993 2024-09-21T15:00:40.558 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.558 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.558 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:40.558 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.369686+0000", 2024-09-21T15:00:40.558 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019000399999999999 2024-09-21T15:00:40.558 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.559 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.559 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:40.559 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.369792+0000", 2024-09-21T15:00:40.559 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000105775 2024-09-21T15:00:40.559 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:40.559 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:40.559 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:40.559 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.559 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.559 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:97693 3.b 3:d7d7b39b:::rbd_data.1136b9c49b1c.000000000000000c:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:40.559 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.328643+0000", 2024-09-21T15:00:40.559 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 35.142911751, 2024-09-21T15:00:40.559 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.031062136000000001, 2024-09-21T15:00:40.559 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:40.559 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:40.560 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:40.560 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:40.560 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:40.560 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97693 2024-09-21T15:00:40.560 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.560 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:40.560 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.560 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:40.560 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328643+0000", 2024-09-21T15:00:40.560 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:40.560 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.560 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.560 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:40.560 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328643+0000", 2024-09-21T15:00:40.560 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:40.560 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.561 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.561 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:40.561 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328642+0000", 2024-09-21T15:00:40.561 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:00:40.561 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.561 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.561 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:40.561 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328650+0000", 2024-09-21T15:00:40.561 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7770000000000008e-06 2024-09-21T15:00:40.561 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.561 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.561 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:40.561 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328662+0000", 2024-09-21T15:00:40.561 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2026e-05 2024-09-21T15:00:40.561 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.562 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.562 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:40.562 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328671+0000", 2024-09-21T15:00:40.562 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.585e-06 2024-09-21T15:00:40.562 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.562 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.562 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:40.562 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.359498+0000", 2024-09-21T15:00:40.562 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.030827001 2024-09-21T15:00:40.562 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.562 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.562 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:40.562 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.359603+0000", 2024-09-21T15:00:40.562 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010512400000000001 2024-09-21T15:00:40.562 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.563 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.563 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:40.563 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.359705+0000", 2024-09-21T15:00:40.563 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000102327 2024-09-21T15:00:40.563 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:40.563 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:40.563 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:40.563 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.563 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.563 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:97724 3.b 3:d4bf6bcf:::rbd_data.1136b9c49b1c.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:40.563 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.328704+0000", 2024-09-21T15:00:40.563 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 35.142851479000001, 2024-09-21T15:00:40.563 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.031157957, 2024-09-21T15:00:40.563 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:40.563 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:40.564 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:40.564 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:40.564 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:40.564 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97724 2024-09-21T15:00:40.564 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.564 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:40.564 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.564 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:40.564 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328704+0000", 2024-09-21T15:00:40.564 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:40.564 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.564 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.564 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:40.564 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328704+0000", 2024-09-21T15:00:40.564 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:40.564 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.565 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.565 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:40.565 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328703+0000", 2024-09-21T15:00:40.565 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:00:40.565 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.565 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.565 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:40.565 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328708+0000", 2024-09-21T15:00:40.565 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0050000000000004e-06 2024-09-21T15:00:40.565 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.565 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.565 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:40.565 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328719+0000", 2024-09-21T15:00:40.565 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1052e-05 2024-09-21T15:00:40.565 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.566 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.566 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:40.566 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328727+0000", 2024-09-21T15:00:40.566 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4519999999999997e-06 2024-09-21T15:00:40.566 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.566 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.566 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:40.566 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.359737+0000", 2024-09-21T15:00:40.566 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.031009341999999999 2024-09-21T15:00:40.566 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.566 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.566 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:40.566 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.359806+0000", 2024-09-21T15:00:40.566 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9750000000000001e-05 2024-09-21T15:00:40.567 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:40.567 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:40.567 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:40.567 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.359862+0000", 2024-09-21T15:00:40.567 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5084999999999999e-05 2024-09-21T15:00:40.567 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:40.567 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:40.567 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:40.567 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:40.567 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:40.567 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:40.568 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T15:00:40.568 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T15:00:40.569 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T15:00:40.569 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:00:40.569 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:00:40.569 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:00:40.569 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:00:40.570 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:00:40.570 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:00:40.570 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:00:40.570 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:00:40.614 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:00:40.611+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:00:40.671 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:00:40.669+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:00:40.721 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:40.721 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:40.731 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T15:00:40.772 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:00:40.769+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:00:40.839 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:40.839 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:40.849 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T15:00:40.873 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:00:40.869+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:00:40.957 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:40.957 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:40.967 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T15:00:40.974 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:00:40.971+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:00:41.075 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:00:41.071+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:00:41.076 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:41.076 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:41.086 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T15:00:41.159 INFO:teuthology.orchestra.run.smithi032.stderr:noscrub is set 2024-09-21T15:00:41.177 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:00:41.173+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:00:41.246 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:41.246 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:41.257 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T15:00:41.277 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:00:41.273+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:00:41.342 INFO:teuthology.orchestra.run.smithi032.stdout:checking clone #29, image image_client.0-clone29 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent30 2024-09-21T15:00:41.367 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:41.367 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:41.379 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T15:00:41.379 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:00:41.377+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:00:41.385 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 5, 6, 7, 4] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T15:00:41.385 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T15:00:41.385 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-21T15:00:41.386 INFO:tasks.thrashosds.thrasher:Setting osd 3 primary_affinity to 0.833837 2024-09-21T15:00:41.386 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 0.8338365970058557 2024-09-21T15:00:41.480 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:00:41.475+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:00:41.486 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:41.486 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:41.496 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T15:00:41.516 INFO:teuthology.orchestra.run.smithi032.stdout:4231 write 0x95ca9f1 thru 0x95cc447 (0x1a57 bytes) 2024-09-21T15:00:41.529 INFO:teuthology.orchestra.run.smithi032.stdout:4232 punch from 0x88b2120 to 0x88b7a0d, (0x58ed bytes) 2024-09-21T15:00:41.529 INFO:teuthology.orchestra.run.smithi032.stdout:4233 write 0x52bb39 thru 0x52c70c (0xbd4 bytes) 2024-09-21T15:00:41.540 INFO:teuthology.orchestra.run.smithi032.stdout:4234 trunc from 0xec1da53 to 0x95665fe 2024-09-21T15:00:41.563 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:41.563 INFO:teuthology.orchestra.run.smithi032.stdout:4239 read 0x1e2a09c thru 0x1e35fa2 (0xbf07 bytes) 2024-09-21T15:00:41.563 INFO:teuthology.orchestra.run.smithi032.stdout:4240 read 0x1c63fa3 thru 0x1c6ee6a (0xaec8 bytes) 2024-09-21T15:00:41.563 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:41.563 INFO:teuthology.orchestra.run.smithi032.stdout:4242 punch from 0x2ce74e6 to 0x2cf426f, (0xcd89 bytes) 2024-09-21T15:00:41.563 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:41.564 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:41.564 INFO:teuthology.orchestra.run.smithi032.stdout:4245 write 0xecd0f0 thru 0xedbb9a (0xeaab bytes) 2024-09-21T15:00:41.567 INFO:teuthology.orchestra.run.smithi032.stdout:4247 punch from 0xa02396 to 0xa0e1f0, (0xbe5a bytes) 2024-09-21T15:00:41.567 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:41.567 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:41.577 INFO:teuthology.orchestra.run.smithi032.stdout:4251 trunc from 0x95665fe to 0xeaef555 2024-09-21T15:00:41.581 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:00:41.577+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:00:41.589 INFO:teuthology.orchestra.run.smithi032.stdout:4252 punch from 0x22d7b68 to 0x22e40bf, (0xc557 bytes) 2024-09-21T15:00:41.590 INFO:teuthology.orchestra.run.smithi032.stdout:4253 writesame 0x7d5df73 thru 0x7d6d328 data_size 0x22(0xf3b6 bytes) 2024-09-21T15:00:41.597 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:41.597 INFO:teuthology.orchestra.run.smithi032.stdout:4257 writesame 0xda08a33 thru 0xda0b78c data_size 0x36(0x2d5a bytes) 2024-09-21T15:00:41.604 INFO:teuthology.orchestra.run.smithi032.stdout:4259 punch from 0x167bb8e to 0x167c7c2, (0xc34 bytes) 2024-09-21T15:00:41.604 INFO:teuthology.orchestra.run.smithi032.stdout:4260 read 0x936e7f4 thru 0x9377e19 (0x9626 bytes) 2024-09-21T15:00:41.611 INFO:teuthology.orchestra.run.smithi032.stdout:4261 trunc from 0xeaef555 to 0xa3d8cc7 2024-09-21T15:00:41.630 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:41.630 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:41.631 INFO:teuthology.orchestra.run.smithi032.stdout:4263 writesame 0x2f8180f thru 0x2f87f82 data_size 0x6(0x6774 bytes) 2024-09-21T15:00:41.639 INFO:teuthology.orchestra.run.smithi032.stdout:4265 read 0x540f941 thru 0x541a8e7 (0xafa7 bytes) 2024-09-21T15:00:41.639 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:41.639 INFO:teuthology.orchestra.run.smithi032.stdout:4267 read 0x71c35c1 thru 0x71cc7bf (0x91ff bytes) 2024-09-21T15:00:41.640 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:41.640 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:41.640 INFO:teuthology.orchestra.run.smithi032.stdout:4270 write 0x4577d33 thru 0x458693b (0xec09 bytes) 2024-09-21T15:00:41.641 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T15:00:41.647 INFO:teuthology.orchestra.run.smithi032.stdout:4271 read 0x3ade798 thru 0x3ae109a (0x2903 bytes) 2024-09-21T15:00:41.647 INFO:teuthology.orchestra.run.smithi032.stdout:4272 write 0x236a4a7 thru 0x237923d (0xed97 bytes) 2024-09-21T15:00:41.654 INFO:teuthology.orchestra.run.smithi032.stdout:4274 read 0x3bd3219 thru 0x3bd7ecd (0x4cb5 bytes) 2024-09-21T15:00:41.667 INFO:teuthology.orchestra.run.smithi032.stdout:4275 write 0xb0aa445 thru 0xb0b4da8 (0xa964 bytes) 2024-09-21T15:00:41.672 INFO:teuthology.orchestra.run.smithi032.stdout:4276 writesame 0x5060262 thru 0x5060276 data_size 0x1(0x15 bytes) 2024-09-21T15:00:41.678 INFO:teuthology.orchestra.run.smithi032.stdout:4278 write 0x1f60513 thru 0x1f61ff9 (0x1ae7 bytes) 2024-09-21T15:00:41.681 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:00:41.677+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:00:41.699 INFO:teuthology.orchestra.run.smithi032.stdout:4280 writesame 0xbc4cbd0 thru 0xbc52d47 data_size 0x4(0x6178 bytes) 2024-09-21T15:00:41.699 INFO:teuthology.orchestra.run.smithi032.stdout:4281 writesame 0x155bb5c thru 0x1566515 data_size 0x6e(0xa9ba bytes) 2024-09-21T15:00:41.703 INFO:teuthology.orchestra.run.smithi032.stdout:4284 read 0xfd3aa5 thru 0xfd7744 (0x3ca0 bytes) 2024-09-21T15:00:41.703 INFO:teuthology.orchestra.run.smithi032.stdout:4285 punch from 0x5fff2e0 to 0x600d6ea, (0xe40a bytes) 2024-09-21T15:00:41.703 INFO:teuthology.orchestra.run.smithi032.stdout:4286 writesame 0x36c61d2 thru 0x36c7cc4 data_size 0x1(0x1af3 bytes) 2024-09-21T15:00:41.733 INFO:teuthology.orchestra.run.smithi032.stdout:4288 trunc from 0xbc52d48 to 0x1067698 2024-09-21T15:00:41.759 INFO:teuthology.orchestra.run.smithi032.stdout:4289 read 0x72b8b6 thru 0x735cc3 (0xa40e bytes) 2024-09-21T15:00:41.759 INFO:teuthology.orchestra.run.smithi032.stdout:4290 write 0x94e05e thru 0x95db29 (0xfacc bytes) 2024-09-21T15:00:41.761 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:41.761 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:41.764 INFO:teuthology.orchestra.run.smithi032.stdout:4291 read 0x35481b thru 0x35c21d (0x7a03 bytes) 2024-09-21T15:00:41.765 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:41.782 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:00:41.777+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:00:41.787 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T15:00:41.797 INFO:teuthology.orchestra.run.smithi032.stdout:4293 writesame 0xc786282 thru 0xc795a10 data_size 0xa9(0xf78f bytes) 2024-09-21T15:00:41.797 INFO:teuthology.orchestra.run.smithi032.stdout:4294 write 0x402c364 thru 0x4031378 (0x5015 bytes) 2024-09-21T15:00:41.802 INFO:teuthology.orchestra.run.smithi032.stdout:4296 write 0xc728429 thru 0xc72c000 (0x3bd8 bytes) 2024-09-21T15:00:41.809 INFO:teuthology.orchestra.run.smithi032.stdout:4297 trunc from 0xc795a11 to 0xca804bb 2024-09-21T15:00:41.821 INFO:teuthology.orchestra.run.smithi032.stdout:4298 punch from 0x5a60883 to 0x5a6de1a, (0xd597 bytes) 2024-09-21T15:00:41.821 INFO:teuthology.orchestra.run.smithi032.stdout:4300 read 0x50d3487 thru 0x50e1f0b (0xea85 bytes) 2024-09-21T15:00:41.821 INFO:teuthology.orchestra.run.smithi032.stdout:4301 write 0x89915d thru 0x8a22b0 (0x9154 bytes) 2024-09-21T15:00:41.829 INFO:teuthology.orchestra.run.smithi032.stdout:4303 punch from 0xc66edaf to 0xc66fbd2, (0xe23 bytes) 2024-09-21T15:00:41.829 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:41.829 INFO:teuthology.orchestra.run.smithi032.stdout:4306 read 0x9ab8754 thru 0x9ab92cf (0xb7c bytes) 2024-09-21T15:00:41.829 INFO:teuthology.orchestra.run.smithi032.stdout:4307 write 0x4916940 thru 0x491f7fa (0x8ebb bytes) 2024-09-21T15:00:41.834 INFO:teuthology.orchestra.run.smithi032.stdout:4308 read 0xb257665 thru 0xb25ef0b (0x78a7 bytes) 2024-09-21T15:00:41.852 INFO:teuthology.orchestra.run.smithi032.stdout:4310 write 0xe5fdd2b thru 0xe5ffa93 (0x1d69 bytes) 2024-09-21T15:00:41.860 INFO:teuthology.orchestra.run.smithi032.stdout:4311 read 0x4ca526c thru 0x4cb1b82 (0xc917 bytes) 2024-09-21T15:00:41.860 INFO:teuthology.orchestra.run.smithi032.stdout:4312 writesame 0x31dc1b7 thru 0x31ea09e data_size 0x1c(0xdee8 bytes) 2024-09-21T15:00:41.869 INFO:teuthology.orchestra.run.smithi032.stdout:4313 punch from 0x242946e to 0x24327ca, (0x935c bytes) 2024-09-21T15:00:41.869 INFO:teuthology.orchestra.run.smithi032.stdout:4315 read 0x68b101e thru 0x68b416e (0x3151 bytes) 2024-09-21T15:00:41.882 INFO:teuthology.orchestra.run.smithi032.stdout:4316 write 0xe628c91 thru 0xe63298e (0x9cfe bytes) 2024-09-21T15:00:41.882 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:00:41.881+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:00:41.889 INFO:teuthology.orchestra.run.smithi032.stdout:4317 read 0x7ebddbc thru 0x7ec2df3 (0x5038 bytes) 2024-09-21T15:00:41.889 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:41.889 INFO:teuthology.orchestra.run.smithi032.stdout:4320 read 0xc1cf1a1 thru 0xc1d01e5 (0x1045 bytes) 2024-09-21T15:00:41.889 INFO:teuthology.orchestra.run.smithi032.stdout:4321 write 0xa19d543 thru 0xa1a55e5 (0x80a3 bytes) 2024-09-21T15:00:41.895 INFO:teuthology.orchestra.run.smithi032.stdout:4322 read 0x6060230 thru 0x6068621 (0x83f2 bytes) 2024-09-21T15:00:41.895 INFO:teuthology.orchestra.run.smithi032.stdout:4324 write 0xb93d0d9 thru 0xb9436fa (0x6622 bytes) 2024-09-21T15:00:41.898 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:41.898 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:41.909 INFO:teuthology.orchestra.run.smithi032.stdout:4325 trunc from 0xe63298f to 0xa2434e8 2024-09-21T15:00:41.910 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T15:00:41.932 INFO:teuthology.orchestra.run.smithi032.stdout:4327 write 0x572c9e4 thru 0x572d203 (0x820 bytes) 2024-09-21T15:00:41.938 INFO:teuthology.orchestra.run.smithi032.stdout:4328 read 0x2146918 thru 0x214a5d5 (0x3cbe bytes) 2024-09-21T15:00:41.939 INFO:teuthology.orchestra.run.smithi032.stdout:truncating to largest ever: 0xeda8ff9 2024-09-21T15:00:41.947 INFO:teuthology.orchestra.run.smithi032.stdout:4329 trunc from 0xa2434e8 to 0xeda8ff9 2024-09-21T15:00:41.960 INFO:teuthology.orchestra.run.smithi032.stdout:4330 read 0x2cb8faf thru 0x2cc61ed (0xd23f bytes) 2024-09-21T15:00:41.961 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:41.961 INFO:teuthology.orchestra.run.smithi032.stdout:4333 punch from 0x8423d56 to 0x842f33b, (0xb5e5 bytes) 2024-09-21T15:00:41.983 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:00:41.979+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:00:41.983 INFO:teuthology.orchestra.run.smithi032.stdout:4334 trunc from 0xeda8ff9 to 0x2b9259c 2024-09-21T15:00:42.007 INFO:teuthology.orchestra.run.smithi032.stdout:4335 read 0x239cb75 thru 0x23a9532 (0xc9be bytes) 2024-09-21T15:00:42.007 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:42.039 INFO:teuthology.orchestra.run.smithi032.stdout:4337 write 0xd90fe62 thru 0xd91388f (0x3a2e bytes) 2024-09-21T15:00:42.046 INFO:teuthology.orchestra.run.smithi032.stdout:4338 write 0xb8cf1d2 thru 0xb8d4212 (0x5041 bytes) 2024-09-21T15:00:42.055 INFO:teuthology.orchestra.run.smithi032.stdout:4339 read 0x4604bb2 thru 0x460bd65 (0x71b4 bytes) 2024-09-21T15:00:42.055 INFO:teuthology.orchestra.run.smithi032.stdout:4340 punch from 0xd3bb2bd to 0xd3c8a58, (0xd79b bytes) 2024-09-21T15:00:42.056 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:42.056 INFO:teuthology.orchestra.run.smithi032.stdout:4345 writesame 0x62781e3 thru 0x6284dcb data_size 0x1(0xcbe9 bytes) 2024-09-21T15:00:42.068 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:42.068 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:42.079 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T15:00:42.081 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:42.081 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:42.081 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:42.081 INFO:teuthology.orchestra.run.smithi032.stdout:4350 writesame 0x1d3e566 thru 0x1d4b271 data_size 0x16(0xcd0c bytes) 2024-09-21T15:00:42.093 INFO:teuthology.orchestra.run.smithi032.stdout:4351 trunc from 0xd913890 to 0x9372df6 2024-09-21T15:00:42.094 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:00:42.091+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:00:42.113 INFO:teuthology.orchestra.run.smithi032.stdout:4353 trunc from 0x9372df6 to 0x8f51d9d 2024-09-21T15:00:42.125 INFO:teuthology.orchestra.run.smithi032.stdout:4354 trunc from 0x8f51d9d to 0x8a9d30e 2024-09-21T15:00:42.155 INFO:teuthology.orchestra.run.smithi032.stdout:4356 write 0xa6443ce thru 0xa64b88d (0x74c0 bytes) 2024-09-21T15:00:42.163 INFO:teuthology.orchestra.run.smithi032.stdout:4357 punch from 0x5eed5ca to 0x5ef8ee9, (0xb91f bytes) 2024-09-21T15:00:42.163 INFO:teuthology.orchestra.run.smithi032.stdout:4358 punch from 0x698a7fd to 0x698caa5, (0x22a8 bytes) 2024-09-21T15:00:42.163 INFO:teuthology.orchestra.run.smithi032.stdout:4360 read 0x94f39f4 thru 0x94f39fe (0xb bytes) 2024-09-21T15:00:42.163 INFO:teuthology.orchestra.run.smithi032.stdout:4362 punch from 0x6bec61e to 0x6bfc0e3, (0xfac5 bytes) 2024-09-21T15:00:42.164 INFO:teuthology.orchestra.run.smithi032.stdout:4363 read 0x8b7cdc1 thru 0x8b8cc50 (0xfe90 bytes) 2024-09-21T15:00:42.164 INFO:teuthology.orchestra.run.smithi032.stdout:4364 punch from 0x899db71 to 0x89a576f, (0x7bfe bytes) 2024-09-21T15:00:42.164 INFO:teuthology.orchestra.run.smithi032.stdout:4365 writesame 0x14d340d thru 0x14d6d56 data_size 0x1(0x394a bytes) 2024-09-21T15:00:42.179 INFO:teuthology.orchestra.run.smithi032.stdout:4366 read 0x4c435fe thru 0x4c535a8 (0xffab bytes) 2024-09-21T15:00:42.179 INFO:teuthology.orchestra.run.smithi032.stdout:4368 write 0x1a7384b thru 0x1a7a088 (0x683e bytes) 2024-09-21T15:00:42.183 INFO:teuthology.orchestra.run.smithi032.stdout:4369 read 0x2ce9bb7 thru 0x2cf2500 (0x894a bytes) 2024-09-21T15:00:42.183 INFO:teuthology.orchestra.run.smithi032.stdout:4371 read 0x4a1c4a0 thru 0x4a275cc (0xb12d bytes) 2024-09-21T15:00:42.183 INFO:teuthology.orchestra.run.smithi032.stdout:4372 punch from 0x13c485f to 0x13cb1b4, (0x6955 bytes) 2024-09-21T15:00:42.184 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:00:42.181+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:00:42.192 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:42.192 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:42.202 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T15:00:42.203 INFO:teuthology.orchestra.run.smithi032.stdout:4373 write 0xec0f70e thru 0xec1b7cf (0xc0c2 bytes) 2024-09-21T15:00:42.211 INFO:teuthology.orchestra.run.smithi032.stdout:4374 read 0x1d07959 thru 0x1d15f7f (0xe627 bytes) 2024-09-21T15:00:42.212 INFO:teuthology.orchestra.run.smithi032.stdout:4375 write 0xc7e54a2 thru 0xc7f2af6 (0xd655 bytes) 2024-09-21T15:00:42.220 INFO:teuthology.orchestra.run.smithi032.stdout:4376 writesame 0xdd168f thru 0xdded84 data_size 0x5(0xd6f6 bytes) 2024-09-21T15:00:42.232 INFO:teuthology.orchestra.run.smithi032.stdout:4377 read 0x390899e thru 0x391401e (0xb681 bytes) 2024-09-21T15:00:42.232 INFO:teuthology.orchestra.run.smithi032.stdout:4378 read 0x1fc01cb thru 0x1fcd0ca (0xcf00 bytes) 2024-09-21T15:00:42.232 INFO:teuthology.orchestra.run.smithi032.stdout:4379 writesame 0xade7027 thru 0xadf0a0d data_size 0x17(0x99e7 bytes) 2024-09-21T15:00:42.240 INFO:teuthology.orchestra.run.smithi032.stdout:4380 writesame 0x4182ae thru 0x41b5a6 data_size 0x1(0x32f9 bytes) 2024-09-21T15:00:42.255 INFO:teuthology.orchestra.run.smithi032.stdout:4383 punch from 0x4402637 to 0x44103af, (0xdd78 bytes) 2024-09-21T15:00:42.255 INFO:teuthology.orchestra.run.smithi032.stdout:4384 read 0xa5456cd thru 0xa546558 (0xe8c bytes) 2024-09-21T15:00:42.255 INFO:teuthology.orchestra.run.smithi032.stdout:4385 writesame 0xdb0cb4b thru 0xdb0f469 data_size 0x21(0x291f bytes) 2024-09-21T15:00:42.260 INFO:teuthology.orchestra.run.smithi032.stdout:4386 write 0xa4e3858 thru 0xa4ee4fc (0xaca5 bytes) 2024-09-21T15:00:42.265 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:42.265 INFO:teuthology.orchestra.run.smithi032.stdout:4389 write 0xa84af60 thru 0xa856d7b (0xbe1c bytes) 2024-09-21T15:00:42.272 INFO:teuthology.orchestra.run.smithi032.stdout:4390 writesame 0x93106d2 thru 0x9310c51 data_size 0x10(0x580 bytes) 2024-09-21T15:00:42.279 INFO:teuthology.orchestra.run.smithi032.stdout:4391 read 0xdae8ff0 thru 0xdaef19e (0x61af bytes) 2024-09-21T15:00:42.279 INFO:teuthology.orchestra.run.smithi032.stdout:4392 read 0x7159260 thru 0x7165986 (0xc727 bytes) 2024-09-21T15:00:42.279 INFO:teuthology.orchestra.run.smithi032.stdout:4394 punch from 0x2c79f69 to 0x2c852c7, (0xb35e bytes) 2024-09-21T15:00:42.279 INFO:teuthology.orchestra.run.smithi032.stdout:4396 punch from 0x1f84c60 to 0x1f94814, (0xfbb4 bytes) 2024-09-21T15:00:42.279 INFO:teuthology.orchestra.run.smithi032.stdout:4397 punch from 0x4e77db1 to 0x4e7a70a, (0x2959 bytes) 2024-09-21T15:00:42.279 INFO:teuthology.orchestra.run.smithi032.stdout:4398 writesame 0x1aa125e thru 0x1aa5c1f data_size 0x9(0x49c2 bytes) 2024-09-21T15:00:42.284 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:00:42.279+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:00:42.288 INFO:teuthology.orchestra.run.smithi032.stdout:4399 write 0x360ec4f thru 0x361e413 (0xf7c5 bytes) 2024-09-21T15:00:42.296 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:42.296 INFO:teuthology.orchestra.run.smithi032.stdout:4401 write 0xcfb73e6 thru 0xcfc42ff (0xcf1a bytes) 2024-09-21T15:00:42.304 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:42.304 INFO:teuthology.orchestra.run.smithi032.stdout:4404 punch from 0x64f9783 to 0x65091a5, (0xfa22 bytes) 2024-09-21T15:00:42.304 INFO:teuthology.orchestra.run.smithi032.stdout:4405 writesame 0x4be6eaa thru 0x4bea5c1 data_size 0x2b(0x3718 bytes) 2024-09-21T15:00:42.311 INFO:teuthology.orchestra.run.smithi032.stdout:4406 write 0xdca2195 thru 0xdca46ee (0x255a bytes) 2024-09-21T15:00:42.315 INFO:teuthology.orchestra.run.smithi032.stdout:4407 writesame 0x2829aff thru 0x2832bcd data_size 0x9(0x90cf bytes) 2024-09-21T15:00:42.325 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:42.325 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:42.326 INFO:teuthology.orchestra.run.smithi032.stdout:4408 punch from 0xadd4e49 to 0xaddca6c, (0x7c23 bytes) 2024-09-21T15:00:42.326 INFO:teuthology.orchestra.run.smithi032.stdout:4409 writesame 0x89e0bfd thru 0x89e0f0c data_size 0x7(0x310 bytes) 2024-09-21T15:00:42.333 INFO:teuthology.orchestra.run.smithi032.stdout:4410 write 0xccd8206 thru 0xcce47a3 (0xc59e bytes) 2024-09-21T15:00:42.336 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T15:00:42.337 INFO:teuthology.orchestra.run.smithi032.stdout:4414 write 0x4e869c6 thru 0x4e8f783 (0x8dbe bytes) 2024-09-21T15:00:42.344 INFO:teuthology.orchestra.run.smithi032.stdout:4416 read 0x9840bd8 thru 0x984154b (0x974 bytes) 2024-09-21T15:00:42.344 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:42.370 INFO:teuthology.orchestra.run.smithi032.stdout:4419 trunc from 0xec1b7d0 to 0x2a542f5 2024-09-21T15:00:42.385 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:00:42.383+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:00:42.393 INFO:teuthology.orchestra.run.smithi032.stdout:4421 read 0x78c729 thru 0x790340 (0x3c18 bytes) 2024-09-21T15:00:42.408 INFO:teuthology.orchestra.run.smithi032.stderr:set osd.3 primary-affinity to 0.833837 (8546462) 2024-09-21T15:00:42.412 INFO:teuthology.orchestra.run.smithi032.stdout:4422 writesame 0x6866f53 thru 0x6873abb data_size 0x2b(0xcb69 bytes) 2024-09-21T15:00:42.412 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:42.431 INFO:teuthology.orchestra.run.smithi032.stdout:4424 write 0xa95e325 thru 0xa96762e (0x930a bytes) 2024-09-21T15:00:42.436 INFO:teuthology.orchestra.run.smithi032.stdout:4425 writesame 0xcf5670 thru 0xcff657 data_size 0x88(0x9fe8 bytes) 2024-09-21T15:00:42.439 INFO:teuthology.orchestra.run.smithi032.stdout:4427 read 0xa5f58be thru 0xa5f60e6 (0x829 bytes) 2024-09-21T15:00:42.439 INFO:teuthology.orchestra.run.smithi032.stdout:4428 write 0x22815dd thru 0x228b60a (0xa02e bytes) 2024-09-21T15:00:42.445 INFO:teuthology.orchestra.run.smithi032.stdout:4429 read 0xcb10e5 thru 0xcb844f (0x736b bytes) 2024-09-21T15:00:42.446 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:42.446 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:42.456 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T15:00:42.466 INFO:teuthology.orchestra.run.smithi032.stdout:4430 write 0xe0cf433 thru 0xe0d7301 (0x7ecf bytes) 2024-09-21T15:00:42.474 INFO:teuthology.orchestra.run.smithi032.stdout:4431 write 0x20515cb thru 0x205c337 (0xad6d bytes) 2024-09-21T15:00:42.481 INFO:teuthology.orchestra.run.smithi032.stdout:4432 read 0x7f0693d thru 0x7f0a99b (0x405f bytes) 2024-09-21T15:00:42.482 INFO:teuthology.orchestra.run.smithi032.stdout:4433 read 0xb38be6f thru 0xb38e488 (0x261a bytes) 2024-09-21T15:00:42.482 INFO:teuthology.orchestra.run.smithi032.stdout:4434 write 0xa97e4e thru 0xa9b56d (0x3720 bytes) 2024-09-21T15:00:42.485 INFO:teuthology.orchestra.run.smithi032.stdout:4435 writesame 0x343a8b4 thru 0x343fa8f data_size 0x3e(0x51dc bytes) 2024-09-21T15:00:42.486 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:00:42.483+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:00:42.493 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:42.493 INFO:teuthology.orchestra.run.smithi032.stdout:4438 read 0x827b080 thru 0x8283dd8 (0x8d59 bytes) 2024-09-21T15:00:42.493 INFO:teuthology.orchestra.run.smithi032.stdout:4439 read 0x4798665 thru 0x47a39ec (0xb388 bytes) 2024-09-21T15:00:42.493 INFO:teuthology.orchestra.run.smithi032.stdout:4443 punch from 0x919a26 to 0x921226, (0x7800 bytes) 2024-09-21T15:00:42.494 INFO:teuthology.orchestra.run.smithi032.stdout:4444 write 0x5799223 thru 0x579aea3 (0x1c81 bytes) 2024-09-21T15:00:42.500 INFO:teuthology.orchestra.run.smithi032.stdout:4446 writesame 0x29a5378 thru 0x29ab9d7 data_size 0x1c(0x6660 bytes) 2024-09-21T15:00:42.504 INFO:teuthology.orchestra.run.smithi032.stdout:4447 writesame 0x2d65d2d thru 0x2d6656b data_size 0x1(0x83f bytes) 2024-09-21T15:00:42.512 INFO:teuthology.orchestra.run.smithi032.stdout:4448 write 0x8e9b8c2 thru 0x8ea9dfb (0xe53a bytes) 2024-09-21T15:00:42.544 INFO:teuthology.orchestra.run.smithi032.stdout:4449 trunc from 0xe0d7302 to 0x3577e1f 2024-09-21T15:00:42.565 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:42.566 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:42.571 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T15:00:42.576 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T15:00:42.582 INFO:teuthology.orchestra.run.smithi032.stdout:4450 write 0x3951868 thru 0x3954897 (0x3030 bytes) 2024-09-21T15:00:42.587 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:00:42.585+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:00:42.589 INFO:teuthology.orchestra.run.smithi032.stdout:4451 read 0x2097d23 thru 0x209fd4c (0x802a bytes) 2024-09-21T15:00:42.604 INFO:teuthology.orchestra.run.smithi032.stdout:4452 trunc from 0x3954898 to 0xac2b898 2024-09-21T15:00:42.615 INFO:teuthology.orchestra.run.smithi032.stdout:4453 read 0x9c25b4 thru 0x9cc2fc (0x9d49 bytes) 2024-09-21T15:00:42.615 INFO:teuthology.orchestra.run.smithi032.stdout:4455 read 0x57b50db thru 0x57bd25b (0x8181 bytes) 2024-09-21T15:00:42.629 INFO:teuthology.orchestra.run.smithi032.stdout:4458 trunc from 0xac2b898 to 0x33cd44e 2024-09-21T15:00:42.655 INFO:teuthology.orchestra.run.smithi032.stdout:4460 trunc from 0x33cd44e to 0x3ea0328 2024-09-21T15:00:42.666 INFO:teuthology.orchestra.run.smithi032.stdout:4462 write 0x5cf279 thru 0x5cfe6f (0xbf7 bytes) 2024-09-21T15:00:42.670 INFO:teuthology.orchestra.run.smithi032.stdout:4463 read 0x214b441 thru 0x215535f (0x9f1f bytes) 2024-09-21T15:00:42.682 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:42.683 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:42.688 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:00:42.685+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:00:42.690 INFO:teuthology.orchestra.run.smithi032.stdout:4464 trunc from 0x3ea0328 to 0xea70a2c 2024-09-21T15:00:42.693 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T15:00:42.702 INFO:teuthology.orchestra.run.smithi032.stdout:4465 punch from 0x2fdb2e8 to 0x2fdbc49, (0x961 bytes) 2024-09-21T15:00:42.702 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:42.702 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:42.703 INFO:teuthology.orchestra.run.smithi032.stdout:4469 read 0x2016886 thru 0x2019be0 (0x335b bytes) 2024-09-21T15:00:42.723 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T15:00:42.723 INFO:teuthology.orchestra.run.smithi032.stdout:4470 trunc from 0xea70a2c to 0x282b18e 2024-09-21T15:00:42.743 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T15:00:42.764 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T15:00:42.774 INFO:teuthology.orchestra.run.smithi032.stdout:4471 write 0xaf4182c thru 0xaf4da69 (0xc23e bytes) 2024-09-21T15:00:42.780 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:42.780 INFO:teuthology.orchestra.run.smithi032.stdout:4475 read 0x73446f5 thru 0x734f9c7 (0xb2d3 bytes) 2024-09-21T15:00:42.781 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:42.781 INFO:teuthology.orchestra.run.smithi032.stdout:4478 write 0xa8266f0 thru 0xa82fa51 (0x9362 bytes) 2024-09-21T15:00:42.786 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T15:00:42.788 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:42.788 INFO:teuthology.orchestra.run.smithi032.stdout:4482 punch from 0x240884e to 0x24148e2, (0xc094 bytes) 2024-09-21T15:00:42.788 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:00:42.785+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:00:42.800 INFO:teuthology.orchestra.run.smithi032.stdout:4483 write 0xaf6c092 thru 0xaf6e86f (0x27de bytes) 2024-09-21T15:00:42.801 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:42.801 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:42.803 INFO:teuthology.orchestra.run.smithi032.stdout:4484 write 0x80dbba4 thru 0x80df66b (0x3ac8 bytes) 2024-09-21T15:00:42.807 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-21T15:00:42.810 INFO:teuthology.orchestra.run.smithi032.stdout:4486 write 0xa0dbf9f thru 0xa0e1b13 (0x5b75 bytes) 2024-09-21T15:00:42.812 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T15:00:42.816 INFO:teuthology.orchestra.run.smithi032.stdout:4489 read 0x1b74526 thru 0x1b7522c (0xd07 bytes) 2024-09-21T15:00:42.816 INFO:teuthology.orchestra.run.smithi032.stdout:4490 read 0x12ffda8 thru 0x13064ae (0x6707 bytes) 2024-09-21T15:00:42.816 INFO:teuthology.orchestra.run.smithi032.stdout:4491 write 0x3d2e114 thru 0x3d35e82 (0x7d6f bytes) 2024-09-21T15:00:42.822 INFO:teuthology.orchestra.run.smithi032.stdout:4494 read 0x78aab3b thru 0x78b3910 (0x8dd6 bytes) 2024-09-21T15:00:42.836 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T15:00:42.837 INFO:teuthology.orchestra.run.smithi032.stdout:4496 write 0xbf0545d thru 0xbf08515 (0x30b9 bytes) 2024-09-21T15:00:42.843 INFO:teuthology.orchestra.run.smithi032.stdout:4497 writesame 0x424a472 thru 0x4254bcc data_size 0x1(0xa75b bytes) 2024-09-21T15:00:42.863 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T15:00:42.884 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T15:00:42.884 INFO:teuthology.orchestra.run.smithi032.stdout:4498 read 0x493ef78 thru 0x4945087 (0x6110 bytes) 2024-09-21T15:00:42.885 INFO:teuthology.orchestra.run.smithi032.stdout:4499 writesame 0xb3b1574 thru 0xb3bfa81 data_size 0x57(0xe50e bytes) 2024-09-21T15:00:42.887 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:00:42.887 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:00:42.887 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:00:42.887 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:00:42.888 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:00:42.888 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:00:42.888 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:00:42.888 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:00:42.889 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:42.889 INFO:teuthology.orchestra.run.smithi032.stdout:4501 read 0x68f678 thru 0x691c7a (0x2603 bytes) 2024-09-21T15:00:42.889 INFO:teuthology.orchestra.run.smithi032.stdout:4502 write 0xad7afec thru 0xad7cc6b (0x1c80 bytes) 2024-09-21T15:00:42.889 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:00:42.885+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:00:42.897 INFO:teuthology.orchestra.run.smithi032.stdout:4503 writesame 0x7f3537c thru 0x7f3ff90 data_size 0x1(0xac15 bytes) 2024-09-21T15:00:42.930 INFO:teuthology.orchestra.run.smithi032.stdout:4504 read 0x33b0c56 thru 0x33b5949 (0x4cf4 bytes) 2024-09-21T15:00:42.930 INFO:teuthology.orchestra.run.smithi032.stdout:4505 write 0x957ca08 thru 0x958955b (0xcb54 bytes) 2024-09-21T15:00:42.939 INFO:teuthology.orchestra.run.smithi032.stdout:4506 punch from 0x3355163 to 0x335576b, (0x608 bytes) 2024-09-21T15:00:42.939 INFO:teuthology.orchestra.run.smithi032.stdout:4508 write 0x384e6e6 thru 0x385dc07 (0xf522 bytes) 2024-09-21T15:00:42.944 INFO:teuthology.orchestra.run.smithi032.stdout:4510 writesame 0xbebf390 thru 0xbebfa9f data_size 0x8(0x710 bytes) 2024-09-21T15:00:42.950 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:42.950 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:42.950 INFO:teuthology.orchestra.run.smithi032.stdout:4513 writesame 0x163797d thru 0x163ecee data_size 0x7(0x7372 bytes) 2024-09-21T15:00:42.956 INFO:teuthology.orchestra.run.smithi032.stdout:4514 punch from 0x15b42f5 to 0x15bd91f, (0x962a bytes) 2024-09-21T15:00:42.957 INFO:teuthology.orchestra.run.smithi032.stdout:4515 punch from 0x72d1200 to 0x72df6b1, (0xe4b1 bytes) 2024-09-21T15:00:42.957 INFO:teuthology.orchestra.run.smithi032.stdout:4516 read 0x6cae270 thru 0x6cbc069 (0xddfa bytes) 2024-09-21T15:00:42.957 INFO:teuthology.orchestra.run.smithi032.stdout:4517 punch from 0x58e331b to 0x58e8b06, (0x57eb bytes) 2024-09-21T15:00:42.957 INFO:teuthology.orchestra.run.smithi032.stdout:4519 write 0x14aea41 thru 0x14b9bc1 (0xb181 bytes) 2024-09-21T15:00:42.960 INFO:teuthology.orchestra.run.smithi032.stdout:4520 clone 32 order 21 su 131072 sc 2 2024-09-21T15:00:42.967 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:42.967 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T15:00:42.967 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T15:00:42.967 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T15:00:42.967 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:42.967 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45839 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:42.967 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159313+0000", 2024-09-21T15:00:42.967 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 92.803990518000006, 2024-09-21T15:00:42.967 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068238699, 2024-09-21T15:00:42.967 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:42.968 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:42.968 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:42.968 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:42.968 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:42.968 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45839 2024-09-21T15:00:42.968 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:42.968 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:42.968 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:42.968 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:42.968 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159313+0000", 2024-09-21T15:00:42.968 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:42.968 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:42.968 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:42.968 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:42.968 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159313+0000", 2024-09-21T15:00:42.969 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:42.969 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:42.969 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:42.969 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:42.969 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159313+0000", 2024-09-21T15:00:42.969 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:42.969 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:42.969 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:42.969 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:42.969 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159316+0000", 2024-09-21T15:00:42.969 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.033e-06 2024-09-21T15:00:42.969 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:42.969 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:42.969 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:42.969 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159322+0000", 2024-09-21T15:00:42.969 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1820000000000003e-06 2024-09-21T15:00:42.970 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:42.970 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:42.970 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:42.970 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159327+0000", 2024-09-21T15:00:42.970 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.062e-06 2024-09-21T15:00:42.970 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:42.970 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:42.970 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:42.970 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227544+0000", 2024-09-21T15:00:42.970 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068217142999999994 2024-09-21T15:00:42.970 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:42.970 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:42.970 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:42.970 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227552+0000", 2024-09-21T15:00:42.971 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6260000000000003e-06 2024-09-21T15:00:42.971 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:42.971 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:42.971 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:42.971 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:42.971 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:42.971 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45840 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:42.971 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159332+0000", 2024-09-21T15:00:42.971 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 92.803972220999995, 2024-09-21T15:00:42.971 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068262046000000007, 2024-09-21T15:00:42.971 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:42.971 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:42.971 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:42.971 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:42.971 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:42.971 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45840 2024-09-21T15:00:42.972 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:42.972 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:42.972 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:42.972 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:42.972 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159332+0000", 2024-09-21T15:00:42.972 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:42.972 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:42.972 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:42.972 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:42.972 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159332+0000", 2024-09-21T15:00:42.972 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:42.972 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:42.972 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:42.972 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:42.972 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159331+0000", 2024-09-21T15:00:42.973 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:42.973 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:42.973 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:42.973 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:42.973 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159351+0000", 2024-09-21T15:00:42.973 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0064999999999998e-05 2024-09-21T15:00:42.973 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:42.973 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:42.973 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:42.973 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159367+0000", 2024-09-21T15:00:42.973 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5308e-05 2024-09-21T15:00:42.973 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:42.973 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:42.973 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:42.973 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159373+0000", 2024-09-21T15:00:42.974 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1840000000000004e-06 2024-09-21T15:00:42.974 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:42.974 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:42.974 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:42.974 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227584+0000", 2024-09-21T15:00:42.974 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068211401000000005 2024-09-21T15:00:42.974 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:42.974 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:42.974 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:42.974 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227594+0000", 2024-09-21T15:00:42.974 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4120000000000007e-06 2024-09-21T15:00:42.974 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:42.974 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:42.974 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:42.975 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:42.975 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:42.975 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45841 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:42.975 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159379+0000", 2024-09-21T15:00:42.975 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 92.803925065000001, 2024-09-21T15:00:42.975 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068249039999999997, 2024-09-21T15:00:42.975 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:42.975 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:42.975 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:42.975 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:42.975 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:42.975 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45841 2024-09-21T15:00:42.975 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:42.975 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:42.975 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:42.976 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:42.976 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159379+0000", 2024-09-21T15:00:42.976 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:42.976 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:42.976 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:42.976 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:42.976 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159379+0000", 2024-09-21T15:00:42.976 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:42.976 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:42.976 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:42.976 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:42.976 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159378+0000", 2024-09-21T15:00:42.976 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:42.976 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:42.977 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:42.977 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:42.977 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159382+0000", 2024-09-21T15:00:42.977 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.641e-06 2024-09-21T15:00:42.977 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:42.977 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:42.977 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:42.977 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159421+0000", 2024-09-21T15:00:42.977 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9439e-05 2024-09-21T15:00:42.977 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:42.977 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:42.977 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:42.977 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159430+0000", 2024-09-21T15:00:42.977 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8429999999999999e-06 2024-09-21T15:00:42.977 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:42.978 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:42.978 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:42.978 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227617+0000", 2024-09-21T15:00:42.978 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068187084999999995 2024-09-21T15:00:42.978 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:42.978 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:42.978 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:42.978 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227628+0000", 2024-09-21T15:00:42.978 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0507e-05 2024-09-21T15:00:42.978 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:42.978 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:42.978 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:42.978 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:42.978 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:42.978 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45844 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:42.978 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159499+0000", 2024-09-21T15:00:42.979 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 92.803804678999995, 2024-09-21T15:00:42.979 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068255415, 2024-09-21T15:00:42.979 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:42.979 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:42.979 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:42.979 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:42.979 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:42.979 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45844 2024-09-21T15:00:42.979 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:42.979 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:42.979 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:42.979 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:42.979 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159499+0000", 2024-09-21T15:00:42.979 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:42.980 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:42.980 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:42.980 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:42.980 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159499+0000", 2024-09-21T15:00:42.980 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:42.980 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:42.980 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:42.980 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:42.980 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159499+0000", 2024-09-21T15:00:42.980 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:42.980 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:42.980 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:42.980 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:42.980 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159503+0000", 2024-09-21T15:00:42.980 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0999999999999997e-06 2024-09-21T15:00:42.980 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:42.981 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:42.981 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:42.981 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159512+0000", 2024-09-21T15:00:42.981 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5850000000000006e-06 2024-09-21T15:00:42.981 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:42.981 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:42.981 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:42.981 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159518+0000", 2024-09-21T15:00:42.981 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0909999999999998e-06 2024-09-21T15:00:42.981 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:42.981 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:42.981 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:42.981 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227743+0000", 2024-09-21T15:00:42.981 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068224536000000002 2024-09-21T15:00:42.981 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:42.981 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:42.982 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:42.982 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227755+0000", 2024-09-21T15:00:42.982 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1659e-05 2024-09-21T15:00:42.982 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:42.982 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:42.982 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:42.982 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:42.982 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:42.982 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45845 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:42.982 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159524+0000", 2024-09-21T15:00:42.982 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 92.803780318999998, 2024-09-21T15:00:42.982 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068267194000000003, 2024-09-21T15:00:42.982 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:42.982 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:42.982 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:42.982 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:42.983 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:42.983 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45845 2024-09-21T15:00:42.983 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:42.983 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:42.983 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:42.983 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:42.983 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159524+0000", 2024-09-21T15:00:42.983 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:42.983 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:42.983 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:42.983 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:42.983 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159524+0000", 2024-09-21T15:00:42.983 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:42.983 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:42.983 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:42.983 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:42.984 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159523+0000", 2024-09-21T15:00:42.984 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:42.984 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:42.984 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:42.984 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:42.984 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159526+0000", 2024-09-21T15:00:42.984 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1980000000000001e-06 2024-09-21T15:00:42.984 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:42.984 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:42.984 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:42.984 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159533+0000", 2024-09-21T15:00:42.984 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.348e-06 2024-09-21T15:00:42.984 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:42.984 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:42.985 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:42.985 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159546+0000", 2024-09-21T15:00:42.985 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3464e-05 2024-09-21T15:00:42.985 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:42.985 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:42.985 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:42.985 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227783+0000", 2024-09-21T15:00:42.985 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068236926000000003 2024-09-21T15:00:42.985 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:42.985 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:42.985 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:42.985 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227791+0000", 2024-09-21T15:00:42.985 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6669999999999996e-06 2024-09-21T15:00:42.985 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:42.985 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:42.986 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:42.986 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:42.986 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:42.986 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45846 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:42.986 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159584+0000", 2024-09-21T15:00:42.986 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 92.803719677000004, 2024-09-21T15:00:42.986 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068254767999999993, 2024-09-21T15:00:42.986 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:42.986 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:42.986 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:42.986 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:42.986 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:42.986 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45846 2024-09-21T15:00:42.987 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:42.987 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:42.987 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:42.987 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:42.987 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159584+0000", 2024-09-21T15:00:42.987 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:42.987 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:42.987 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:42.987 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:42.987 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159584+0000", 2024-09-21T15:00:42.987 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:42.987 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:42.987 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:42.987 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:42.987 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159583+0000", 2024-09-21T15:00:42.988 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:42.988 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:42.988 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:42.988 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:42.988 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159588+0000", 2024-09-21T15:00:42.988 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1330000000000001e-06 2024-09-21T15:00:42.988 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:42.988 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:42.988 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:42.988 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159596+0000", 2024-09-21T15:00:42.988 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5469999999999997e-06 2024-09-21T15:00:42.988 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:42.988 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:42.988 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:42.988 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159609+0000", 2024-09-21T15:00:42.989 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2785999999999999e-05 2024-09-21T15:00:42.989 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:42.989 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:42.989 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:42.989 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227830+0000", 2024-09-21T15:00:42.989 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068220879999999998 2024-09-21T15:00:42.989 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:42.989 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:42.989 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:42.989 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227839+0000", 2024-09-21T15:00:42.989 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1239999999999994e-06 2024-09-21T15:00:42.989 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:42.989 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:42.990 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:42.990 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:42.990 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:42.990 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45847 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:42.990 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159614+0000", 2024-09-21T15:00:42.990 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 92.803689601000002, 2024-09-21T15:00:42.990 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068260639999999997, 2024-09-21T15:00:42.990 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:42.990 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:42.990 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:42.990 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:42.990 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:42.990 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45847 2024-09-21T15:00:42.990 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:42.990 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:42.991 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:42.991 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:42.991 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159614+0000", 2024-09-21T15:00:42.991 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:42.991 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:42.991 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:42.991 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:42.991 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159614+0000", 2024-09-21T15:00:42.991 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:42.991 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:42.991 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:42.991 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:42.991 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159614+0000", 2024-09-21T15:00:42.991 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:42.991 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:42.992 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:42.992 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:42.992 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159617+0000", 2024-09-21T15:00:42.992 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2909999999999999e-06 2024-09-21T15:00:42.992 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:42.992 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:42.992 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:42.992 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159623+0000", 2024-09-21T15:00:42.992 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.511e-06 2024-09-21T15:00:42.992 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:42.992 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:42.992 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:42.992 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159631+0000", 2024-09-21T15:00:42.992 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.571e-06 2024-09-21T15:00:42.992 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:42.992 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:42.993 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:42.993 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227866+0000", 2024-09-21T15:00:42.993 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068235136000000002 2024-09-21T15:00:42.993 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:42.993 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:42.993 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:42.993 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227875+0000", 2024-09-21T15:00:42.993 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7199999999999995e-06 2024-09-21T15:00:42.993 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:42.993 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:42.993 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:42.993 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:42.993 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:42.993 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45848 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:42.993 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159636+0000", 2024-09-21T15:00:42.994 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 92.803667821000005, 2024-09-21T15:00:42.994 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068267283999999998, 2024-09-21T15:00:42.994 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:42.994 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:42.994 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:42.994 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:42.994 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:42.994 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45848 2024-09-21T15:00:42.994 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:42.994 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:42.994 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:42.994 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:42.994 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159636+0000", 2024-09-21T15:00:42.995 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:42.995 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:42.995 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:42.995 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:42.995 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159636+0000", 2024-09-21T15:00:42.995 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:42.995 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:42.995 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:42.995 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:42.995 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159636+0000", 2024-09-21T15:00:42.995 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:42.995 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:42.995 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:42.995 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:42.995 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159638+0000", 2024-09-21T15:00:42.996 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6560000000000001e-06 2024-09-21T15:00:42.996 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:42.996 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:42.996 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:42.996 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159644+0000", 2024-09-21T15:00:42.996 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0839999999999997e-06 2024-09-21T15:00:42.996 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:42.996 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:42.996 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:42.996 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159649+0000", 2024-09-21T15:00:42.996 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7500000000000003e-06 2024-09-21T15:00:42.996 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:42.996 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:42.996 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:42.996 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227892+0000", 2024-09-21T15:00:42.996 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068243315999999998 2024-09-21T15:00:42.997 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:42.997 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:42.997 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:42.997 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227903+0000", 2024-09-21T15:00:42.997 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0835999999999999e-05 2024-09-21T15:00:42.997 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:42.997 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:42.997 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:42.997 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:42.997 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:42.997 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45849 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:42.997 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159667+0000", 2024-09-21T15:00:42.997 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 92.803637007000006, 2024-09-21T15:00:42.997 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068291051000000005, 2024-09-21T15:00:42.997 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:42.997 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:42.998 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:42.998 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:42.998 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:42.998 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45849 2024-09-21T15:00:42.998 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:42.998 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:42.998 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:42.998 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:42.998 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159667+0000", 2024-09-21T15:00:42.998 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:42.998 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:42.998 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:42.998 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:42.998 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159667+0000", 2024-09-21T15:00:42.998 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:42.998 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:42.999 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:42.999 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:42.999 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159666+0000", 2024-09-21T15:00:42.999 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:42.999 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:42.999 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:42.999 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:42.999 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159670+0000", 2024-09-21T15:00:42.999 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.427e-06 2024-09-21T15:00:42.999 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:42.999 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:42.999 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:42.999 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159678+0000", 2024-09-21T15:00:42.999 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9869999999999999e-06 2024-09-21T15:00:42.999 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.000 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.000 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:43.000 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159688+0000", 2024-09-21T15:00:43.000 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0631999999999999e-05 2024-09-21T15:00:43.000 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.000 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.000 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:43.000 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227950+0000", 2024-09-21T15:00:43.000 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068261798999999998 2024-09-21T15:00:43.000 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.000 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.000 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:43.000 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227958+0000", 2024-09-21T15:00:43.000 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7510000000000008e-06 2024-09-21T15:00:43.000 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.000 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:43.001 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.001 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.001 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.001 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45850 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:43.001 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159695+0000", 2024-09-21T15:00:43.001 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 92.803609296000005, 2024-09-21T15:00:43.001 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068302999000000003, 2024-09-21T15:00:43.001 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:43.001 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:43.001 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:43.001 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:43.001 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:43.001 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45850 2024-09-21T15:00:43.001 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.002 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:43.002 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.002 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:43.002 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159695+0000", 2024-09-21T15:00:43.002 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.002 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.002 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.002 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:43.002 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159695+0000", 2024-09-21T15:00:43.002 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.002 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.002 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.002 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:43.002 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159694+0000", 2024-09-21T15:00:43.002 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:43.002 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.003 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.003 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:43.003 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159697+0000", 2024-09-21T15:00:43.003 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.258e-06 2024-09-21T15:00:43.003 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.003 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.003 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:43.003 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159704+0000", 2024-09-21T15:00:43.003 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7660000000000001e-06 2024-09-21T15:00:43.003 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.003 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.003 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:43.003 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159713+0000", 2024-09-21T15:00:43.003 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1779999999999992e-06 2024-09-21T15:00:43.003 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.004 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.004 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:43.004 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227989+0000", 2024-09-21T15:00:43.004 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068276015999999995 2024-09-21T15:00:43.004 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.004 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.004 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:43.004 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227998+0000", 2024-09-21T15:00:43.004 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2050000000000002e-06 2024-09-21T15:00:43.004 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.004 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:43.004 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.004 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.004 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.004 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45851 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:43.005 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159719+0000", 2024-09-21T15:00:43.005 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 92.803585162999994, 2024-09-21T15:00:43.005 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068320327, 2024-09-21T15:00:43.005 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:43.005 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:43.005 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:43.005 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:43.005 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:43.005 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45851 2024-09-21T15:00:43.005 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.005 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:43.005 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.005 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:43.005 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159719+0000", 2024-09-21T15:00:43.005 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.005 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.006 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.006 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:43.006 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159719+0000", 2024-09-21T15:00:43.006 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.006 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.006 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.006 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:43.006 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159718+0000", 2024-09-21T15:00:43.006 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:43.006 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.006 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.006 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:43.006 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159721+0000", 2024-09-21T15:00:43.006 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2169999999999999e-06 2024-09-21T15:00:43.007 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.007 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.007 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:43.007 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159730+0000", 2024-09-21T15:00:43.007 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5720000000000003e-06 2024-09-21T15:00:43.007 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.007 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.007 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:43.007 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159735+0000", 2024-09-21T15:00:43.007 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1100000000000002e-06 2024-09-21T15:00:43.007 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.007 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.007 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:43.007 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228029+0000", 2024-09-21T15:00:43.008 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068293624999999997 2024-09-21T15:00:43.008 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.008 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.008 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:43.008 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228039+0000", 2024-09-21T15:00:43.008 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0254e-05 2024-09-21T15:00:43.008 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.008 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:43.008 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.008 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.008 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.008 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45852 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:43.008 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159740+0000", 2024-09-21T15:00:43.008 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 92.803563995000005, 2024-09-21T15:00:43.008 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068340260999999999, 2024-09-21T15:00:43.008 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:43.009 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:43.009 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:43.009 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:43.009 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:43.009 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45852 2024-09-21T15:00:43.009 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.009 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:43.009 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.009 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:43.009 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159740+0000", 2024-09-21T15:00:43.009 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.009 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.009 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.009 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:43.009 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159740+0000", 2024-09-21T15:00:43.009 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.010 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.010 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.010 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:43.010 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159739+0000", 2024-09-21T15:00:43.010 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:43.010 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.010 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.010 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:43.010 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159757+0000", 2024-09-21T15:00:43.010 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7314999999999999e-05 2024-09-21T15:00:43.010 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.010 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.010 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:43.010 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159769+0000", 2024-09-21T15:00:43.010 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2376e-05 2024-09-21T15:00:43.010 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.011 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.011 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:43.011 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159775+0000", 2024-09-21T15:00:43.011 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2179999999999996e-06 2024-09-21T15:00:43.011 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.011 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.011 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:43.011 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228071+0000", 2024-09-21T15:00:43.011 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068295344999999993 2024-09-21T15:00:43.011 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.011 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.011 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:43.011 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228080+0000", 2024-09-21T15:00:43.011 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3959999999999995e-06 2024-09-21T15:00:43.011 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.011 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:43.012 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.012 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.012 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.012 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45853 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:43.012 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159781+0000", 2024-09-21T15:00:43.012 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 92.803522731000001, 2024-09-21T15:00:43.012 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068337093000000002, 2024-09-21T15:00:43.012 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:43.012 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:43.012 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:43.012 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:43.012 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:43.012 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45853 2024-09-21T15:00:43.013 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.013 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:43.013 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.013 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:43.013 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159781+0000", 2024-09-21T15:00:43.013 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.013 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.013 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.013 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:43.013 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159781+0000", 2024-09-21T15:00:43.013 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.013 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.013 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.013 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:43.013 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159781+0000", 2024-09-21T15:00:43.013 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:43.014 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.014 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.014 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:43.014 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159784+0000", 2024-09-21T15:00:43.014 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4759999999999998e-06 2024-09-21T15:00:43.014 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.014 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.014 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:43.014 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159792+0000", 2024-09-21T15:00:43.014 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0569999999999993e-06 2024-09-21T15:00:43.014 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.014 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.014 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:43.014 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159797+0000", 2024-09-21T15:00:43.015 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9640000000000002e-06 2024-09-21T15:00:43.015 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.015 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.015 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:43.015 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228111+0000", 2024-09-21T15:00:43.015 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068313737999999999 2024-09-21T15:00:43.015 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.015 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.015 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:43.015 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228118+0000", 2024-09-21T15:00:43.015 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.43e-06 2024-09-21T15:00:43.015 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.015 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:43.015 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.015 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.015 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.016 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45854 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:43.016 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159802+0000", 2024-09-21T15:00:43.016 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 92.803502081000005, 2024-09-21T15:00:43.016 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068358938999999994, 2024-09-21T15:00:43.016 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:43.016 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:43.016 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:43.016 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:43.016 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:43.016 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45854 2024-09-21T15:00:43.016 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.016 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:43.016 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.016 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:43.016 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159802+0000", 2024-09-21T15:00:43.017 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.017 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.017 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.017 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:43.017 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159802+0000", 2024-09-21T15:00:43.017 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.017 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.017 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.017 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:43.017 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159801+0000", 2024-09-21T15:00:43.017 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:43.017 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.017 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.017 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:43.017 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159804+0000", 2024-09-21T15:00:43.017 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0970000000000002e-06 2024-09-21T15:00:43.018 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.018 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.018 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:43.018 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159823+0000", 2024-09-21T15:00:43.018 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8895000000000001e-05 2024-09-21T15:00:43.018 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.018 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.018 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:43.018 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159837+0000", 2024-09-21T15:00:43.018 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3247999999999999e-05 2024-09-21T15:00:43.018 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.018 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.018 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:43.018 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228151+0000", 2024-09-21T15:00:43.019 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068314642999999994 2024-09-21T15:00:43.019 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.019 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.019 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:43.019 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228161+0000", 2024-09-21T15:00:43.019 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4250000000000004e-06 2024-09-21T15:00:43.019 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.019 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:43.019 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.019 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.019 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.019 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45983 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:43.019 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159848+0000", 2024-09-21T15:00:43.019 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 92.803455822000004, 2024-09-21T15:00:43.019 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068333801, 2024-09-21T15:00:43.019 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:43.020 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:43.020 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:43.020 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:43.020 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:43.020 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45983 2024-09-21T15:00:43.020 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.020 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:43.020 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.020 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:43.020 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159848+0000", 2024-09-21T15:00:43.020 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.020 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.020 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.020 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:43.020 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159848+0000", 2024-09-21T15:00:43.021 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.021 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.021 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.021 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:43.021 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159847+0000", 2024-09-21T15:00:43.021 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:43.021 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.021 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.021 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:43.021 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159856+0000", 2024-09-21T15:00:43.021 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4349999999999997e-06 2024-09-21T15:00:43.021 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.021 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.021 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:43.021 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159866+0000", 2024-09-21T15:00:43.021 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0076e-05 2024-09-21T15:00:43.021 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.022 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.022 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:43.022 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159872+0000", 2024-09-21T15:00:43.022 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8309999999999997e-06 2024-09-21T15:00:43.022 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.022 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.022 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:43.022 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228175+0000", 2024-09-21T15:00:43.022 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068302680000000005 2024-09-21T15:00:43.022 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.022 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.022 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:43.022 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228182+0000", 2024-09-21T15:00:43.022 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3150000000000003e-06 2024-09-21T15:00:43.022 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.023 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:43.023 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.023 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.023 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.023 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45984 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:43.023 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159877+0000", 2024-09-21T15:00:43.023 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 92.803426401999999, 2024-09-21T15:00:43.023 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068328448, 2024-09-21T15:00:43.023 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:43.023 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:43.023 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:43.023 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:43.023 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:43.023 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45984 2024-09-21T15:00:43.023 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.023 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:43.024 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.024 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:43.024 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159877+0000", 2024-09-21T15:00:43.024 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.024 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.024 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.024 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:43.024 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159877+0000", 2024-09-21T15:00:43.024 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.024 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.024 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.024 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:43.024 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159877+0000", 2024-09-21T15:00:43.024 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:43.024 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.025 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.025 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:43.025 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159880+0000", 2024-09-21T15:00:43.025 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7610000000000001e-06 2024-09-21T15:00:43.025 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.025 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.025 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:43.025 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159888+0000", 2024-09-21T15:00:43.025 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0330000000000003e-06 2024-09-21T15:00:43.025 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.025 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.025 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:43.025 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159893+0000", 2024-09-21T15:00:43.025 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3550000000000004e-06 2024-09-21T15:00:43.025 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.025 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.026 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:43.026 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228199+0000", 2024-09-21T15:00:43.026 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068306438999999997 2024-09-21T15:00:43.026 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.026 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.026 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:43.026 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228206+0000", 2024-09-21T15:00:43.026 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5599999999999999e-06 2024-09-21T15:00:43.026 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.026 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:43.026 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.026 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.026 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.026 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45985 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:43.026 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159897+0000", 2024-09-21T15:00:43.026 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 92.803406578999997, 2024-09-21T15:00:43.027 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068345664, 2024-09-21T15:00:43.027 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:43.027 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:43.027 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:43.027 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:43.027 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:43.027 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45985 2024-09-21T15:00:43.027 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.027 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:43.027 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.027 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:43.027 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159897+0000", 2024-09-21T15:00:43.027 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.027 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.027 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.027 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:43.028 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159897+0000", 2024-09-21T15:00:43.028 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.028 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.028 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.028 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:43.028 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159897+0000", 2024-09-21T15:00:43.028 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:43.028 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.028 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.028 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:43.028 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159900+0000", 2024-09-21T15:00:43.028 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.954e-06 2024-09-21T15:00:43.028 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.028 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.028 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:43.028 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159913+0000", 2024-09-21T15:00:43.028 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3106e-05 2024-09-21T15:00:43.029 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.029 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.029 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:43.029 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159918+0000", 2024-09-21T15:00:43.029 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5199999999999997e-06 2024-09-21T15:00:43.029 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.029 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.029 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:43.029 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228236+0000", 2024-09-21T15:00:43.029 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068317040999999995 2024-09-21T15:00:43.029 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.029 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.029 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:43.029 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228243+0000", 2024-09-21T15:00:43.029 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3880000000000002e-06 2024-09-21T15:00:43.029 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.030 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:43.030 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.030 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.030 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.030 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45986 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:43.030 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159924+0000", 2024-09-21T15:00:43.030 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 92.803380283999999, 2024-09-21T15:00:43.030 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068364362999999997, 2024-09-21T15:00:43.030 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:43.030 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:43.030 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:43.030 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:43.030 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:43.030 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45986 2024-09-21T15:00:43.030 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.031 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:43.031 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.031 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:43.031 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159924+0000", 2024-09-21T15:00:43.031 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.031 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.031 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.031 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:43.031 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159924+0000", 2024-09-21T15:00:43.031 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.031 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.031 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.031 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:43.031 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159923+0000", 2024-09-21T15:00:43.031 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:43.031 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.031 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.032 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:43.032 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159926+0000", 2024-09-21T15:00:43.032 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0510000000000001e-06 2024-09-21T15:00:43.032 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.032 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.032 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:43.032 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159932+0000", 2024-09-21T15:00:43.032 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4160000000000001e-06 2024-09-21T15:00:43.032 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.032 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.032 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:43.032 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159938+0000", 2024-09-21T15:00:43.032 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.383e-06 2024-09-21T15:00:43.032 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.032 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.032 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:43.033 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228275+0000", 2024-09-21T15:00:43.033 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068337007000000005 2024-09-21T15:00:43.033 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.033 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.033 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:43.033 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228288+0000", 2024-09-21T15:00:43.033 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3052e-05 2024-09-21T15:00:43.033 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.033 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:43.033 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.033 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.033 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.033 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45987 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:43.033 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159944+0000", 2024-09-21T15:00:43.033 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 92.803360060000003, 2024-09-21T15:00:43.034 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068387232000000006, 2024-09-21T15:00:43.034 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:43.034 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:43.034 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:43.034 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:43.034 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:43.034 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45987 2024-09-21T15:00:43.034 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.034 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:43.034 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.034 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:43.034 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159944+0000", 2024-09-21T15:00:43.034 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.034 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.034 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.034 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:43.035 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159944+0000", 2024-09-21T15:00:43.035 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.035 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.035 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.035 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:43.035 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159943+0000", 2024-09-21T15:00:43.035 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:43.035 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.035 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.035 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:43.035 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159946+0000", 2024-09-21T15:00:43.035 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6309999999999999e-06 2024-09-21T15:00:43.035 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.035 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.035 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:43.035 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159952+0000", 2024-09-21T15:00:43.036 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6540000000000004e-06 2024-09-21T15:00:43.036 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.036 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.036 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:43.036 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159957+0000", 2024-09-21T15:00:43.036 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9280000000000001e-06 2024-09-21T15:00:43.036 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.036 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.036 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:43.036 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228322+0000", 2024-09-21T15:00:43.036 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068365067000000002 2024-09-21T15:00:43.036 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.036 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.036 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:43.036 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228331+0000", 2024-09-21T15:00:43.036 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3270000000000007e-06 2024-09-21T15:00:43.037 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.037 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:43.037 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.037 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.037 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.037 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:97672 3.4 3:26046d29:::rbd_data.1136bfc79a70.0000000000000001:head [write 0~4718592 [fadvise_sequential+fadvise_nocache],write 5242880~131072 [fadvise_sequential+fadvise_nocache],write 5439488~147456 [fadvise_sequential+fadvise_nocache],write 5652480~229376 [fadvise_sequential+fadvise_nocache],write 5947392~147456 [fadvise_sequential+fadvise_nocache],write 6160384~229376 [fadvise_sequential+fadvise_nocache],write 6455296~147456 [fadvise_sequential+fadvise_nocache],write 6668288~229376 [fadvise_sequential+fadvise_nocache],write 6963200~147456 [fadvise_sequential+fadvise_nocache],write 7176192~147456 [fadvise_sequential+fadvise_nocache],write 7340032~65536 [fadvise_sequential+fadvise_nocache],write 7471104~147456 [fadvise_sequential+fadvise_nocache],write 7684096~147456 [fadvise_sequential+fadvise_nocache],write 7864320~49152 [fadvise_sequential+fadvise_nocache],write 7979008~147456 [fadvise_sequential+fadvise_nocache],write 8192000~147456 [fadvise_sequential+fadvise_nocache],write 8388608~32768 [fadvise_sequential+fadvise_nocache],write 8519680~114688 [fadvise_sequential+fadvise_nocache],write 8732672~114688 [fadvise_sequential+fadvise_nocache],write 8912896~16384 [fadvise_sequential+fadvise_nocache],write 9109504~32768 [fadvise_sequential+fadvise_nocache],write 9322496~32768 [fadvise_sequential+fadvise_nocache],write 9617408~32768 [fadvise_sequential+fadvise_nocache],write 9830400~32768 [fadvise_sequential+fadvise_nocache],write 9961472~98304 [fadvise_sequential+fadvise_nocache],write 10076160~196608 [fadvise_sequential+fadvise_nocache],write 10289152~278528 [fadvise_sequential+fadvise_nocache],write 10584064~196608 [fadvise_sequential+fadvise_nocache],write 10797056~196608 [fadvise_sequential+fadvise_nocache],write 11010048~65536 [fadvise_sequential+fadvise_nocache],write 11091968~196608 [fadvise_sequential+fadvise_nocache],write 11304960~196608 [fadvise_sequential+fadvise_nocache],write 11517952~65536 [fadvise_sequential+fadvise_nocache],write 11616256~98304 [fadvise_sequential+fadvise_nocache],write 11730944~65536 [fadvise_sequential+fadvise_nocache],write 11829248~98304 [fadvise_sequential+fadvise_nocache],write 11943936~65536 [fadvise_sequential+fadvise_nocache],write 12042240~49152 [fadvise_sequential+fadvise_nocache],write 12124160~98304 [fadvise_sequential+fadvise_nocache],write 12238848~65536 [fadvise_sequential+fadvise_nocache],write 12337152~98304 [fadvise_sequential+fadvise_nocache],write 12451840~65536 [fadvise_sequential+fadvise_nocache],write 12550144~32768 [fadvise_sequential+fadvise_nocache],write 12632064~98304 [fadvise_sequential+fadvise_nocache],write 12845056~98304 [fadvise_sequential+fadvise_nocache],write 13058048~49152 [fadvise_sequential+fadvise_nocache],write 13139968~98304 [fadvise_sequential+fadvise_nocache],write 13352960~98304 [fadvise_sequential+fadvise_nocache],write 13565952~65536 [fadvise_sequential+fadvise_nocache],write 13647872~98304 [fadvise_sequential+fadvise_nocache],write 13860864~98304 [fadvise_sequential+fadvise_nocache],write 14073856~81920 [fadvise_sequential+fadvise_nocache],write 15269888~81920 [fadvise_sequential+fadvise_nocache],write 15482880~81920 [fadvise_sequential+fadvise_nocache],write 15695872~32768 [fadvise_sequential+fadvise_nocache],write 15777792~81920 [fadvise_sequential+fadvise_nocache],write 15990784~81920 [fadvise_sequential+fadvise_nocache],write 16203776~49152 [fadvise_sequential+fadvise_nocache],write 16285696~81920 [fadvise_sequential+fadvise_nocache],write 16498688~81920 [fadvise_sequential+fadvise_nocache],write 16711680~65536 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:43.037 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.298669+0000", 2024-09-21T15:00:43.037 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 37.664634993999996, 2024-09-21T15:00:43.037 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.10282358900000001, 2024-09-21T15:00:43.037 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:43.037 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:43.037 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:43.037 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:43.037 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:43.037 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97672 2024-09-21T15:00:43.037 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.038 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:43.038 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.038 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:43.038 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.298669+0000", 2024-09-21T15:00:43.038 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.038 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.038 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.038 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:43.038 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.298669+0000", 2024-09-21T15:00:43.038 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.038 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.038 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.038 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:43.038 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.298667+0000", 2024-09-21T15:00:43.038 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:00:43.038 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.039 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.039 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:43.039 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.327458+0000", 2024-09-21T15:00:43.039 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.028790754000000002 2024-09-21T15:00:43.039 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.039 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.039 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:43.039 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.327492+0000", 2024-09-21T15:00:43.039 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3806999999999997e-05 2024-09-21T15:00:43.039 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.039 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.039 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:43.039 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.327507+0000", 2024-09-21T15:00:43.039 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5112e-05 2024-09-21T15:00:43.039 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.040 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.040 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:43.040 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.327597+0000", 2024-09-21T15:00:43.040 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9938999999999994e-05 2024-09-21T15:00:43.040 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.040 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.040 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:43.040 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.327927+0000", 2024-09-21T15:00:43.040 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000329766 2024-09-21T15:00:43.040 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.040 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.040 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T15:00:43.040 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.330735+0000", 2024-09-21T15:00:43.040 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0028086869999999998 2024-09-21T15:00:43.040 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.040 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.041 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:43.041 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.378299+0000", 2024-09-21T15:00:43.041 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.047563495999999997 2024-09-21T15:00:43.041 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.041 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.041 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:43.041 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.401248+0000", 2024-09-21T15:00:43.041 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.022949311 2024-09-21T15:00:43.041 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.041 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.041 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:43.041 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.401446+0000", 2024-09-21T15:00:43.041 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019831199999999999 2024-09-21T15:00:43.041 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.041 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.041 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:43.042 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.401492+0000", 2024-09-21T15:00:43.042 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6109000000000002e-05 2024-09-21T15:00:43.042 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.042 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:43.042 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.042 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.042 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:43.042 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:43.043 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T15:00:43.043 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:00:43.039+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:00:43.144 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:00:43.141+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:00:43.154 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:43.154 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:00:43.154 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T15:00:43.155 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:43.164 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T15:00:43.172 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T15:00:43.244 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:00:43.239+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:00:43.268 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:43.268 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:00:43.268 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T15:00:43.268 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T15:00:43.268 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:43.278 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T15:00:43.345 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:00:43.343+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:00:43.391 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:43.391 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T15:00:43.391 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T15:00:43.391 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T15:00:43.391 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.391 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:97677 3.15 e122/24 3:a8e72133:::rbd_data.1136bfc79a70.0000000000000005:head v 122'567, mlcod=120'565)", 2024-09-21T15:00:43.392 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.320753+0000", 2024-09-21T15:00:43.392 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 38.066770167999998, 2024-09-21T15:00:43.392 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070639375000000004, 2024-09-21T15:00:43.392 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:43.392 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:43.392 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:43.392 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.392 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:43.392 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.320753+0000", 2024-09-21T15:00:43.392 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.392 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.392 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.392 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:43.393 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.320753+0000", 2024-09-21T15:00:43.393 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.393 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.393 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.393 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:43.393 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.320753+0000", 2024-09-21T15:00:43.393 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:43.393 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.393 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.393 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:43.393 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.375452+0000", 2024-09-21T15:00:43.393 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054698945999999998 2024-09-21T15:00:43.393 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.393 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.393 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:43.393 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.375476+0000", 2024-09-21T15:00:43.394 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4380999999999999e-05 2024-09-21T15:00:43.394 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.394 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.394 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:43.394 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.375485+0000", 2024-09-21T15:00:43.394 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0049999999999993e-06 2024-09-21T15:00:43.394 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.394 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.394 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:43.394 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.375552+0000", 2024-09-21T15:00:43.394 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6550999999999994e-05 2024-09-21T15:00:43.394 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.394 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.394 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:43.395 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.375578+0000", 2024-09-21T15:00:43.395 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6196000000000001e-05 2024-09-21T15:00:43.395 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.395 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.395 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:43.395 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.391319+0000", 2024-09-21T15:00:43.395 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015741271000000001 2024-09-21T15:00:43.395 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.395 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.395 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:43.395 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.391393+0000", 2024-09-21T15:00:43.395 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3287000000000005e-05 2024-09-21T15:00:43.395 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.396 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:43.396 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.396 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.396 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.396 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:97672 3.4 e122/51 3:26046d29:::rbd_data.1136bfc79a70.0000000000000001:head v 122'256, mlcod=120'254)", 2024-09-21T15:00:43.396 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.331696+0000", 2024-09-21T15:00:43.396 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 38.055827004999998, 2024-09-21T15:00:43.396 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041222507999999998, 2024-09-21T15:00:43.396 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:43.396 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:43.396 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:43.396 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.396 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:43.397 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.331696+0000", 2024-09-21T15:00:43.397 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.397 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.397 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.397 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:43.397 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.331696+0000", 2024-09-21T15:00:43.397 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.397 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.397 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.397 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:43.397 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.331695+0000", 2024-09-21T15:00:43.397 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:00:43.397 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.397 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.397 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:43.398 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.340175+0000", 2024-09-21T15:00:43.398 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0084799520000000007 2024-09-21T15:00:43.398 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.398 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.398 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:43.398 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.340208+0000", 2024-09-21T15:00:43.398 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2997999999999998e-05 2024-09-21T15:00:43.398 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.398 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.398 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:43.398 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.340223+0000", 2024-09-21T15:00:43.398 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5330999999999999e-05 2024-09-21T15:00:43.398 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.398 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.398 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:43.399 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.340305+0000", 2024-09-21T15:00:43.399 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2095999999999995e-05 2024-09-21T15:00:43.399 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.399 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.399 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:43.399 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.340337+0000", 2024-09-21T15:00:43.399 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2097000000000001e-05 2024-09-21T15:00:43.399 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.399 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.399 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:43.399 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.372863+0000", 2024-09-21T15:00:43.399 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.032525923999999998 2024-09-21T15:00:43.399 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.399 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.399 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:43.399 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.372919+0000", 2024-09-21T15:00:43.400 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5729000000000001e-05 2024-09-21T15:00:43.400 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.400 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:43.400 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.400 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.400 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.400 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:116177 3.12 3:4efbaa3f:::rbd_data.1136b1194d20.0000000000000006:58 [read 5791744~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:43.400 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.712449+0000", 2024-09-21T15:00:43.400 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 13.675074324000001, 2024-09-21T15:00:43.400 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.033824706000000003, 2024-09-21T15:00:43.400 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:43.400 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:43.400 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:43.400 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:43.400 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:43.400 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116177 2024-09-21T15:00:43.400 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.401 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:43.401 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.401 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:43.401 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712449+0000", 2024-09-21T15:00:43.401 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.401 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.401 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.401 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:43.401 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712449+0000", 2024-09-21T15:00:43.401 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.401 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.401 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.401 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:43.401 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712448+0000", 2024-09-21T15:00:43.401 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:43.401 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.401 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.402 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:43.402 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712451+0000", 2024-09-21T15:00:43.402 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5330000000000001e-06 2024-09-21T15:00:43.402 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.402 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.402 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:43.402 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712458+0000", 2024-09-21T15:00:43.402 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5239999999999997e-06 2024-09-21T15:00:43.402 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.402 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.402 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:43.402 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712467+0000", 2024-09-21T15:00:43.402 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7189999999999996e-06 2024-09-21T15:00:43.402 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.403 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.403 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:43.403 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.746189+0000", 2024-09-21T15:00:43.403 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.033722203999999999 2024-09-21T15:00:43.403 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.403 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.403 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:43.403 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.746228+0000", 2024-09-21T15:00:43.403 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8908000000000001e-05 2024-09-21T15:00:43.403 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.403 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.403 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:43.403 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.746274+0000", 2024-09-21T15:00:43.403 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5380000000000003e-05 2024-09-21T15:00:43.403 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.403 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:43.404 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.404 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.404 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.404 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:116178 3.12 3:4efbaa3f:::rbd_data.1136b1194d20.0000000000000006:58 [read 5881856~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:43.404 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.712472+0000", 2024-09-21T15:00:43.404 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 13.675051673, 2024-09-21T15:00:43.404 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.033958110999999999, 2024-09-21T15:00:43.404 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:43.404 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:43.404 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:43.404 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:43.404 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:43.404 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116178 2024-09-21T15:00:43.404 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.404 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:43.404 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.405 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:43.405 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712472+0000", 2024-09-21T15:00:43.405 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.405 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.405 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.405 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:43.405 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712472+0000", 2024-09-21T15:00:43.405 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.405 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.405 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.405 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:43.405 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712471+0000", 2024-09-21T15:00:43.405 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:43.405 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.405 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.406 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:43.406 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712474+0000", 2024-09-21T15:00:43.406 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4389999999999999e-06 2024-09-21T15:00:43.406 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.406 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.406 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:43.406 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712479+0000", 2024-09-21T15:00:43.406 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4269999999999999e-06 2024-09-21T15:00:43.406 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.406 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.406 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:43.406 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712483+0000", 2024-09-21T15:00:43.406 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8449999999999996e-06 2024-09-21T15:00:43.406 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.407 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.407 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:43.407 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.746287+0000", 2024-09-21T15:00:43.407 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.033804381000000001 2024-09-21T15:00:43.407 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.407 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.407 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:43.407 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.746337+0000", 2024-09-21T15:00:43.407 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9622e-05 2024-09-21T15:00:43.407 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.407 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.407 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:43.407 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.746430+0000", 2024-09-21T15:00:43.407 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2800000000000006e-05 2024-09-21T15:00:43.408 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.408 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:43.408 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.408 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.408 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.408 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:116179 3.12 3:4efbaa3f:::rbd_data.1136b1194d20.0000000000000006:58 [read 5971968~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:43.408 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.712487+0000", 2024-09-21T15:00:43.408 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 13.675036831, 2024-09-21T15:00:43.408 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034144503, 2024-09-21T15:00:43.408 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:43.408 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:43.408 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:43.408 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:43.408 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:43.408 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116179 2024-09-21T15:00:43.408 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.409 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:43.409 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.409 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:43.409 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712487+0000", 2024-09-21T15:00:43.409 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.409 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.409 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.409 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:43.409 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712487+0000", 2024-09-21T15:00:43.409 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.409 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.409 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.409 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:43.409 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712486+0000", 2024-09-21T15:00:43.409 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:43.409 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.409 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.410 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:43.410 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712488+0000", 2024-09-21T15:00:43.410 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2000000000000001e-06 2024-09-21T15:00:43.410 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.410 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.410 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:43.410 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712493+0000", 2024-09-21T15:00:43.410 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9429999999999999e-06 2024-09-21T15:00:43.410 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.410 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.410 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:43.410 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712497+0000", 2024-09-21T15:00:43.410 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9689999999999996e-06 2024-09-21T15:00:43.410 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.410 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.411 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:43.411 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.746459+0000", 2024-09-21T15:00:43.411 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.033961365 2024-09-21T15:00:43.411 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.411 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.411 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:43.411 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.746538+0000", 2024-09-21T15:00:43.411 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9073999999999995e-05 2024-09-21T15:00:43.411 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.411 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.411 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:43.411 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.746631+0000", 2024-09-21T15:00:43.411 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3375999999999995e-05 2024-09-21T15:00:43.411 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.411 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:43.411 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.411 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.412 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.412 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:116180 3.12 3:4efbaa3f:::rbd_data.1136b1194d20.0000000000000006:58 [read 6062080~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:43.412 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.712501+0000", 2024-09-21T15:00:43.412 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 13.675022632999999, 2024-09-21T15:00:43.412 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034331168000000002, 2024-09-21T15:00:43.412 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:43.412 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:43.412 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:43.412 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:43.412 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:43.412 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116180 2024-09-21T15:00:43.412 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.412 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:43.412 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.412 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:43.412 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712501+0000", 2024-09-21T15:00:43.413 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.413 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.413 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.413 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:43.413 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712501+0000", 2024-09-21T15:00:43.413 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.413 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.413 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.413 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:43.413 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712500+0000", 2024-09-21T15:00:43.413 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:43.413 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.413 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.413 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:43.413 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712502+0000", 2024-09-21T15:00:43.414 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.035e-06 2024-09-21T15:00:43.414 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.414 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.414 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:43.414 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712508+0000", 2024-09-21T15:00:43.414 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4720000000000003e-06 2024-09-21T15:00:43.414 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.414 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.414 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:43.414 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712512+0000", 2024-09-21T15:00:43.414 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7129999999999999e-06 2024-09-21T15:00:43.414 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.414 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.414 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:43.415 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.746662+0000", 2024-09-21T15:00:43.415 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034149837000000002 2024-09-21T15:00:43.415 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.415 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.415 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:43.415 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.746740+0000", 2024-09-21T15:00:43.415 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8350999999999996e-05 2024-09-21T15:00:43.415 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.415 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.415 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:43.415 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.746832+0000", 2024-09-21T15:00:43.415 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2046000000000006e-05 2024-09-21T15:00:43.415 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.415 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:43.415 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.415 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.416 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.416 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:116181 3.12 3:4efbaa3f:::rbd_data.1136b1194d20.0000000000000006:58 [read 6152192~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:43.416 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.712515+0000", 2024-09-21T15:00:43.416 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 13.675008287000001, 2024-09-21T15:00:43.416 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034525820999999998, 2024-09-21T15:00:43.416 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:43.416 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:43.416 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:43.416 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:43.416 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:43.416 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116181 2024-09-21T15:00:43.416 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.416 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:43.416 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.416 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:43.416 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712515+0000", 2024-09-21T15:00:43.417 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.417 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.417 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.417 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:43.417 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712515+0000", 2024-09-21T15:00:43.417 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.417 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.417 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.417 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:43.417 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712515+0000", 2024-09-21T15:00:43.417 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:43.417 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.417 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.417 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:43.417 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712522+0000", 2024-09-21T15:00:43.417 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8000000000000001e-06 2024-09-21T15:00:43.418 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.418 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.418 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:43.418 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712528+0000", 2024-09-21T15:00:43.418 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0650000000000004e-06 2024-09-21T15:00:43.418 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.418 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.418 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:43.418 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712533+0000", 2024-09-21T15:00:43.418 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.079e-06 2024-09-21T15:00:43.418 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.418 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.418 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:43.418 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.746864+0000", 2024-09-21T15:00:43.418 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034330830999999999 2024-09-21T15:00:43.419 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.419 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.419 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:43.419 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.746943+0000", 2024-09-21T15:00:43.419 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9221999999999995e-05 2024-09-21T15:00:43.419 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.419 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.419 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:43.419 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.747041+0000", 2024-09-21T15:00:43.419 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.8114000000000003e-05 2024-09-21T15:00:43.419 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.419 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:43.419 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.419 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.420 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.420 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:116182 3.12 3:4efbaa3f:::rbd_data.1136b1194d20.0000000000000006:58 [read 6242304~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:43.420 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.712538+0000", 2024-09-21T15:00:43.420 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 13.674985014000001, 2024-09-21T15:00:43.420 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034702127999999999, 2024-09-21T15:00:43.420 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:43.420 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:43.420 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:43.420 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:43.420 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:43.420 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116182 2024-09-21T15:00:43.420 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.420 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:43.420 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.420 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:43.420 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712538+0000", 2024-09-21T15:00:43.421 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.421 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.421 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.421 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:43.421 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712538+0000", 2024-09-21T15:00:43.421 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.421 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.421 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.421 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:43.421 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712538+0000", 2024-09-21T15:00:43.421 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:43.421 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.421 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.421 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:43.421 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712541+0000", 2024-09-21T15:00:43.421 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5330000000000001e-06 2024-09-21T15:00:43.421 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.422 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.422 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:43.422 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712547+0000", 2024-09-21T15:00:43.422 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1109999999999996e-06 2024-09-21T15:00:43.422 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.422 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.422 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:43.422 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712551+0000", 2024-09-21T15:00:43.422 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.685e-06 2024-09-21T15:00:43.422 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.422 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.422 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:43.422 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.747067+0000", 2024-09-21T15:00:43.422 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034516181 2024-09-21T15:00:43.423 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.423 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.423 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:43.423 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.747152+0000", 2024-09-21T15:00:43.423 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4075999999999999e-05 2024-09-21T15:00:43.423 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.423 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.423 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:43.423 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.747240+0000", 2024-09-21T15:00:43.423 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8924000000000004e-05 2024-09-21T15:00:43.423 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.423 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:43.423 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.423 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.423 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.423 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:116216 3.12 3:4efbaa3f:::rbd_data.1136b1194d20.0000000000000006:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:43.424 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.713202+0000", 2024-09-21T15:00:43.424 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 13.674321509, 2024-09-21T15:00:43.424 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034182227000000003, 2024-09-21T15:00:43.424 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:43.424 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:43.424 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:43.424 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:43.424 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:43.424 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116216 2024-09-21T15:00:43.424 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.424 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:43.424 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.424 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:43.424 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.713202+0000", 2024-09-21T15:00:43.424 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.424 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.425 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.425 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:43.425 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.713202+0000", 2024-09-21T15:00:43.425 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.425 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.425 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.425 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:43.425 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.713201+0000", 2024-09-21T15:00:43.425 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:43.425 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.425 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.425 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:43.425 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.713204+0000", 2024-09-21T15:00:43.425 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7410000000000001e-06 2024-09-21T15:00:43.425 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.425 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.426 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:43.426 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.713211+0000", 2024-09-21T15:00:43.426 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1110000000000002e-06 2024-09-21T15:00:43.426 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.426 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.426 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:43.426 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.713217+0000", 2024-09-21T15:00:43.426 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.446e-06 2024-09-21T15:00:43.426 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.426 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.426 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:43.426 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.747263+0000", 2024-09-21T15:00:43.426 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034046598999999997 2024-09-21T15:00:43.426 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.426 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.427 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:43.427 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.747321+0000", 2024-09-21T15:00:43.427 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7778999999999997e-05 2024-09-21T15:00:43.427 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.427 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.427 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:43.427 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.747384+0000", 2024-09-21T15:00:43.427 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3083000000000006e-05 2024-09-21T15:00:43.427 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.427 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:43.427 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.427 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.427 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.427 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:118512 3.12 3:4efbaa3f:::rbd_data.1136b1194d20.0000000000000006:58 [read 10043392~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:43.427 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.893240+0000", 2024-09-21T15:00:43.427 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 13.494282921, 2024-09-21T15:00:43.427 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.033811941999999998, 2024-09-21T15:00:43.428 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:43.428 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:43.428 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:43.428 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:43.428 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:43.428 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 118512 2024-09-21T15:00:43.428 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.428 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:43.428 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.428 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:43.428 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893240+0000", 2024-09-21T15:00:43.428 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.428 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.428 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.428 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:43.428 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893240+0000", 2024-09-21T15:00:43.428 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.429 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.429 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.429 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:43.429 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893240+0000", 2024-09-21T15:00:43.429 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:43.429 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.429 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.429 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:43.429 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893242+0000", 2024-09-21T15:00:43.429 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.717e-06 2024-09-21T15:00:43.429 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.429 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.429 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:43.429 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893245+0000", 2024-09-21T15:00:43.429 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3380000000000001e-06 2024-09-21T15:00:43.430 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.430 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.430 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:43.430 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893248+0000", 2024-09-21T15:00:43.430 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9670000000000001e-06 2024-09-21T15:00:43.430 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.430 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.430 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:43.430 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.926868+0000", 2024-09-21T15:00:43.430 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.033619938000000002 2024-09-21T15:00:43.430 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.430 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.430 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:43.430 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.926964+0000", 2024-09-21T15:00:43.430 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5624999999999993e-05 2024-09-21T15:00:43.430 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.431 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.431 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:43.431 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.927052+0000", 2024-09-21T15:00:43.431 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8632000000000001e-05 2024-09-21T15:00:43.431 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.431 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:43.431 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.431 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.431 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.431 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:118513 3.12 3:4efbaa3f:::rbd_data.1136b1194d20.0000000000000006:58 [read 10133504~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:43.431 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.893251+0000", 2024-09-21T15:00:43.431 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 13.494272674999999, 2024-09-21T15:00:43.431 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.033992324999999997, 2024-09-21T15:00:43.431 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:43.431 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:43.432 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:43.432 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:43.432 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:43.432 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 118513 2024-09-21T15:00:43.432 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.432 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:43.432 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.432 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:43.432 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893251+0000", 2024-09-21T15:00:43.432 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.432 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.432 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.432 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:43.432 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893251+0000", 2024-09-21T15:00:43.432 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.432 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.433 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.433 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:43.433 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893250+0000", 2024-09-21T15:00:43.433 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:43.433 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.433 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.433 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:43.433 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893252+0000", 2024-09-21T15:00:43.433 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.666e-06 2024-09-21T15:00:43.433 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.433 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.433 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:43.433 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893255+0000", 2024-09-21T15:00:43.433 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.287e-06 2024-09-21T15:00:43.433 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.433 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.434 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:43.434 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893258+0000", 2024-09-21T15:00:43.434 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.027e-06 2024-09-21T15:00:43.434 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.434 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.434 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:43.434 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.927083+0000", 2024-09-21T15:00:43.434 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.033824262000000001 2024-09-21T15:00:43.434 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.434 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.434 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:43.434 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.927154+0000", 2024-09-21T15:00:43.434 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1099000000000005e-05 2024-09-21T15:00:43.434 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.435 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.435 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:43.435 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.927243+0000", 2024-09-21T15:00:43.435 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9266999999999998e-05 2024-09-21T15:00:43.435 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.435 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:43.435 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.435 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.435 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.435 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:118514 3.12 3:4efbaa3f:::rbd_data.1136b1194d20.0000000000000006:58 [read 10223616~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:43.435 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.893261+0000", 2024-09-21T15:00:43.435 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 13.494262315, 2024-09-21T15:00:43.435 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034165500000000001, 2024-09-21T15:00:43.435 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:43.435 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:43.435 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:43.436 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:43.436 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:43.436 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 118514 2024-09-21T15:00:43.436 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.436 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:43.436 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.436 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:43.436 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893261+0000", 2024-09-21T15:00:43.436 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.436 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.436 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.436 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:43.436 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893261+0000", 2024-09-21T15:00:43.436 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.436 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.436 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.437 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:43.437 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893261+0000", 2024-09-21T15:00:43.437 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:43.437 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.437 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.437 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:43.437 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893262+0000", 2024-09-21T15:00:43.437 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.787e-06 2024-09-21T15:00:43.437 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.437 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.437 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:43.437 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893267+0000", 2024-09-21T15:00:43.437 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6399999999999996e-06 2024-09-21T15:00:43.437 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.437 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.437 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:43.438 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893270+0000", 2024-09-21T15:00:43.438 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9610000000000001e-06 2024-09-21T15:00:43.438 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.438 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.438 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:43.438 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.927271+0000", 2024-09-21T15:00:43.438 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034001253000000002 2024-09-21T15:00:43.438 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.438 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.438 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:43.438 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.927344+0000", 2024-09-21T15:00:43.438 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2962999999999999e-05 2024-09-21T15:00:43.438 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.438 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.438 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:43.438 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.927427+0000", 2024-09-21T15:00:43.439 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2292000000000004e-05 2024-09-21T15:00:43.439 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.439 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:43.439 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.439 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.439 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.439 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:118515 3.12 3:4efbaa3f:::rbd_data.1136b1194d20.0000000000000006:58 [read 10313728~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:43.439 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.893273+0000", 2024-09-21T15:00:43.439 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 13.49424994, 2024-09-21T15:00:43.439 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034333437000000001, 2024-09-21T15:00:43.439 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:43.439 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:43.439 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:43.439 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:43.439 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:43.439 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 118515 2024-09-21T15:00:43.440 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.440 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:43.440 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.440 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:43.440 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893273+0000", 2024-09-21T15:00:43.440 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.440 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.440 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.440 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:43.440 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893273+0000", 2024-09-21T15:00:43.440 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.440 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.440 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.440 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:43.440 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893273+0000", 2024-09-21T15:00:43.440 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:43.441 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.441 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.441 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:43.441 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893275+0000", 2024-09-21T15:00:43.441 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6789999999999999e-06 2024-09-21T15:00:43.441 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.441 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.441 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:43.441 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893278+0000", 2024-09-21T15:00:43.441 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2440000000000002e-06 2024-09-21T15:00:43.441 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.441 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.441 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:43.441 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893282+0000", 2024-09-21T15:00:43.441 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.89e-06 2024-09-21T15:00:43.441 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.442 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.442 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:43.442 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.927454+0000", 2024-09-21T15:00:43.442 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034172077000000002 2024-09-21T15:00:43.442 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.442 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.442 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:43.442 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.927525+0000", 2024-09-21T15:00:43.442 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0963999999999993e-05 2024-09-21T15:00:43.442 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.442 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.442 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:43.442 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.927607+0000", 2024-09-21T15:00:43.442 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1868999999999998e-05 2024-09-21T15:00:43.442 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.443 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:43.443 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.443 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.443 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.443 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:118516 3.12 3:4efbaa3f:::rbd_data.1136b1194d20.0000000000000006:58 [read 10403840~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:43.443 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.893284+0000", 2024-09-21T15:00:43.443 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 13.494238999, 2024-09-21T15:00:43.443 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034589559999999998, 2024-09-21T15:00:43.443 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:43.443 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:43.443 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:43.443 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:43.443 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:43.443 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 118516 2024-09-21T15:00:43.443 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.443 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:43.444 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.444 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:43.444 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893284+0000", 2024-09-21T15:00:43.444 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.444 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.444 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.444 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:43.444 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893284+0000", 2024-09-21T15:00:43.444 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.444 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.444 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.444 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:43.444 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893284+0000", 2024-09-21T15:00:43.444 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:43.444 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.445 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.445 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:43.445 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893291+0000", 2024-09-21T15:00:43.445 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6200000000000001e-06 2024-09-21T15:00:43.445 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.445 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.445 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:43.445 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893294+0000", 2024-09-21T15:00:43.445 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6729999999999998e-06 2024-09-21T15:00:43.445 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.445 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.445 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:43.445 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893297+0000", 2024-09-21T15:00:43.445 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0510000000000001e-06 2024-09-21T15:00:43.445 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.445 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.446 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:43.446 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.927694+0000", 2024-09-21T15:00:43.446 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034397018000000001 2024-09-21T15:00:43.446 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.446 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.446 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:43.446 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.927767+0000", 2024-09-21T15:00:43.446 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2379000000000006e-05 2024-09-21T15:00:43.446 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.446 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.446 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:43.446 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.927874+0000", 2024-09-21T15:00:43.446 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010709000000000001 2024-09-21T15:00:43.446 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.446 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:43.446 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.447 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.447 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.447 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:118696 3.12 3:4efbaa3f:::rbd_data.1136b1194d20.0000000000000006:58 [read 9961472~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:43.447 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.902583+0000", 2024-09-21T15:00:43.447 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 13.484939952, 2024-09-21T15:00:43.447 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.033756517999999999, 2024-09-21T15:00:43.447 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:43.447 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:43.447 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:43.447 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:43.447 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:43.447 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 118696 2024-09-21T15:00:43.447 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.447 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:43.448 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.448 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:43.448 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902583+0000", 2024-09-21T15:00:43.448 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.448 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.448 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.448 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:43.448 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902583+0000", 2024-09-21T15:00:43.448 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.448 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.448 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.448 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:43.448 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902583+0000", 2024-09-21T15:00:43.448 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:43.448 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.448 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.449 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:43.449 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902585+0000", 2024-09-21T15:00:43.449 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7060000000000001e-06 2024-09-21T15:00:43.449 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.449 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.449 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:43.449 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902588+0000", 2024-09-21T15:00:43.449 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.128e-06 2024-09-21T15:00:43.449 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.449 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.449 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:43.449 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902592+0000", 2024-09-21T15:00:43.449 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0990000000000001e-06 2024-09-21T15:00:43.449 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.449 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.449 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:43.450 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.936161+0000", 2024-09-21T15:00:43.450 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.033569103000000003 2024-09-21T15:00:43.450 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.450 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.450 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:43.450 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.936241+0000", 2024-09-21T15:00:43.450 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9437000000000005e-05 2024-09-21T15:00:43.450 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.450 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.450 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:43.450 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.936340+0000", 2024-09-21T15:00:43.450 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9302999999999994e-05 2024-09-21T15:00:43.450 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.450 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:43.450 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.451 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.451 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.451 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:118697 3.12 3:4efbaa3f:::rbd_data.1136b1194d20.0000000000000006:58 [read 10051584~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:43.451 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.902595+0000", 2024-09-21T15:00:43.451 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 13.484928016, 2024-09-21T15:00:43.451 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.033875518, 2024-09-21T15:00:43.451 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:43.451 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:43.451 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:43.451 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:43.451 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:43.451 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 118697 2024-09-21T15:00:43.451 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.451 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:43.451 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.451 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:43.452 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902595+0000", 2024-09-21T15:00:43.452 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.452 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.452 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.452 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:43.452 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902595+0000", 2024-09-21T15:00:43.452 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.452 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.452 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.452 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:43.452 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902595+0000", 2024-09-21T15:00:43.452 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:43.452 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.452 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.452 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:43.452 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902597+0000", 2024-09-21T15:00:43.452 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7889999999999999e-06 2024-09-21T15:00:43.453 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.453 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.453 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:43.453 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902601+0000", 2024-09-21T15:00:43.453 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.16e-06 2024-09-21T15:00:43.453 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.453 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.453 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:43.453 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902605+0000", 2024-09-21T15:00:43.453 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7560000000000001e-06 2024-09-21T15:00:43.453 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.453 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.453 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:43.453 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.936370+0000", 2024-09-21T15:00:43.453 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.033765114999999998 2024-09-21T15:00:43.454 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.454 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.454 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:43.454 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.936413+0000", 2024-09-21T15:00:43.454 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3399000000000002e-05 2024-09-21T15:00:43.454 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.454 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.454 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:43.454 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.936471+0000", 2024-09-21T15:00:43.454 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7726000000000003e-05 2024-09-21T15:00:43.454 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.454 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:43.454 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.454 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.454 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.455 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:118698 3.12 3:4efbaa3f:::rbd_data.1136b1194d20.0000000000000006:58 [read 10141696~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:43.455 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.902607+0000", 2024-09-21T15:00:43.455 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 13.484916422, 2024-09-21T15:00:43.455 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034062306000000001, 2024-09-21T15:00:43.455 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:43.455 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:43.455 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:43.455 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:43.455 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:43.455 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 118698 2024-09-21T15:00:43.455 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.455 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:43.455 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.455 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:43.456 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902607+0000", 2024-09-21T15:00:43.456 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.456 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.456 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.456 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:43.456 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902607+0000", 2024-09-21T15:00:43.456 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.456 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.456 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.456 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:43.456 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902607+0000", 2024-09-21T15:00:43.456 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:43.456 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.456 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.456 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:43.456 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902608+0000", 2024-09-21T15:00:43.457 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5910000000000001e-06 2024-09-21T15:00:43.457 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.457 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.457 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:43.457 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902611+0000", 2024-09-21T15:00:43.457 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.965e-06 2024-09-21T15:00:43.457 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.457 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.457 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:43.457 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902615+0000", 2024-09-21T15:00:43.457 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0049999999999998e-06 2024-09-21T15:00:43.457 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.457 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.457 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:43.457 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.936501+0000", 2024-09-21T15:00:43.457 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.033885884999999998 2024-09-21T15:00:43.458 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.458 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.458 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:43.458 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.936584+0000", 2024-09-21T15:00:43.458 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2650000000000003e-05 2024-09-21T15:00:43.458 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.458 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.458 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:43.458 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.936669+0000", 2024-09-21T15:00:43.458 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5461000000000001e-05 2024-09-21T15:00:43.458 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.458 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:43.458 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.458 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.458 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.458 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:118699 3.12 3:4efbaa3f:::rbd_data.1136b1194d20.0000000000000006:58 [read 10231808~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:43.459 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.902625+0000", 2024-09-21T15:00:43.459 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 13.484898291, 2024-09-21T15:00:43.459 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034245128999999999, 2024-09-21T15:00:43.459 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:43.459 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:43.459 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:43.459 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:43.459 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:43.459 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 118699 2024-09-21T15:00:43.459 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.459 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:43.459 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.459 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:43.459 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902625+0000", 2024-09-21T15:00:43.459 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.459 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.460 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.460 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:43.460 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902625+0000", 2024-09-21T15:00:43.460 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.460 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.460 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.460 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:43.460 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902625+0000", 2024-09-21T15:00:43.460 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:43.460 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.460 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.460 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:43.460 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902627+0000", 2024-09-21T15:00:43.460 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7269999999999999e-06 2024-09-21T15:00:43.460 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.460 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.460 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:43.461 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902630+0000", 2024-09-21T15:00:43.461 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.5200000000000002e-06 2024-09-21T15:00:43.461 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.461 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.461 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:43.461 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902633+0000", 2024-09-21T15:00:43.461 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8399999999999999e-06 2024-09-21T15:00:43.461 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.461 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.461 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:43.461 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.936700+0000", 2024-09-21T15:00:43.461 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034066723 2024-09-21T15:00:43.461 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.461 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.461 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:43.461 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.936778+0000", 2024-09-21T15:00:43.462 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8713000000000003e-05 2024-09-21T15:00:43.462 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.462 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.462 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:43.462 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.936870+0000", 2024-09-21T15:00:43.462 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1885000000000005e-05 2024-09-21T15:00:43.462 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.462 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:43.462 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.462 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.462 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.462 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:118700 3.12 3:4efbaa3f:::rbd_data.1136b1194d20.0000000000000006:58 [read 10321920~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:43.462 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.902635+0000", 2024-09-21T15:00:43.463 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 13.484888084, 2024-09-21T15:00:43.463 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034423665999999999, 2024-09-21T15:00:43.463 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:43.463 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:43.463 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:43.463 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:43.463 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:43.463 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 118700 2024-09-21T15:00:43.463 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.463 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:43.463 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.463 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:43.463 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902635+0000", 2024-09-21T15:00:43.463 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.463 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.463 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.464 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:43.464 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902635+0000", 2024-09-21T15:00:43.464 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.464 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.464 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.464 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:43.464 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902635+0000", 2024-09-21T15:00:43.464 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:43.464 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.464 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.464 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:43.464 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902637+0000", 2024-09-21T15:00:43.464 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.734e-06 2024-09-21T15:00:43.464 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.464 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.464 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:43.465 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902640+0000", 2024-09-21T15:00:43.465 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9720000000000001e-06 2024-09-21T15:00:43.465 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.465 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.465 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:43.465 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902642+0000", 2024-09-21T15:00:43.465 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.616e-06 2024-09-21T15:00:43.465 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.465 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.465 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:43.465 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.936904+0000", 2024-09-21T15:00:43.465 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034261982000000003 2024-09-21T15:00:43.465 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.465 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.465 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:43.465 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.936974+0000", 2024-09-21T15:00:43.466 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9406000000000006e-05 2024-09-21T15:00:43.466 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.466 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.466 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:43.466 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.937059+0000", 2024-09-21T15:00:43.466 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5315999999999994e-05 2024-09-21T15:00:43.466 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.466 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:43.466 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.466 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.466 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.466 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:118701 3.12 3:4efbaa3f:::rbd_data.1136b1194d20.0000000000000006:58 [read 10412032~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:43.466 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.902645+0000", 2024-09-21T15:00:43.466 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 13.48487864, 2024-09-21T15:00:43.466 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034600791999999998, 2024-09-21T15:00:43.466 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:43.467 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:43.467 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:43.467 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:43.467 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:43.467 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 118701 2024-09-21T15:00:43.467 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.467 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:43.467 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.467 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:43.467 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902645+0000", 2024-09-21T15:00:43.467 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.467 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.467 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.467 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:43.467 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902645+0000", 2024-09-21T15:00:43.467 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.468 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.468 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.468 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:43.468 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902644+0000", 2024-09-21T15:00:43.468 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:43.468 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.468 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.468 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:43.468 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902646+0000", 2024-09-21T15:00:43.468 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6169999999999999e-06 2024-09-21T15:00:43.468 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.468 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.468 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:43.468 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902649+0000", 2024-09-21T15:00:43.468 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9450000000000002e-06 2024-09-21T15:00:43.468 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.468 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.469 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:43.469 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902652+0000", 2024-09-21T15:00:43.469 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7889999999999999e-06 2024-09-21T15:00:43.469 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.469 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.469 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:43.469 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.937086+0000", 2024-09-21T15:00:43.469 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034433832999999997 2024-09-21T15:00:43.469 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.469 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.469 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:43.469 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.937152+0000", 2024-09-21T15:00:43.469 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5934999999999998e-05 2024-09-21T15:00:43.469 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.470 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.470 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:43.470 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.937246+0000", 2024-09-21T15:00:43.470 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3923000000000002e-05 2024-09-21T15:00:43.470 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.470 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:43.470 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.470 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.470 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:43.470 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:43.471 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T15:00:43.472 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:00:43.467+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:00:43.572 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:00:43.567+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:00:43.585 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:43.586 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:00:43.586 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T15:00:43.586 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:43.596 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T15:00:43.673 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:00:43.669+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:00:43.707 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:43.707 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:00:43.707 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T15:00:43.707 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T15:00:43.707 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:43.716 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T15:00:43.775 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:00:43.771+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:00:43.830 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:43.830 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T15:00:43.830 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T15:00:43.830 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T15:00:43.830 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.830 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31476 3.c 3:35718b6b:::rbd_data.11367721c19.0000000000000014:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T15:00:43.831 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.505975+0000", 2024-09-21T15:00:43.831 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 105.320539497, 2024-09-21T15:00:43.831 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99515995899999998, 2024-09-21T15:00:43.831 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:43.831 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:43.831 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:43.831 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:43.831 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:43.831 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31476 2024-09-21T15:00:43.831 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.831 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:43.831 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.831 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:43.831 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.505975+0000", 2024-09-21T15:00:43.831 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.832 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.832 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.832 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:43.832 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.505975+0000", 2024-09-21T15:00:43.832 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.832 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.832 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.832 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:43.832 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.505971+0000", 2024-09-21T15:00:43.832 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999967 2024-09-21T15:00:43.832 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.832 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.832 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:43.832 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.505989+0000", 2024-09-21T15:00:43.832 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7985000000000001e-05 2024-09-21T15:00:43.832 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.833 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.833 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:43.833 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506050+0000", 2024-09-21T15:00:43.833 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0659000000000003e-05 2024-09-21T15:00:43.833 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.833 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.833 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:43.833 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506074+0000", 2024-09-21T15:00:43.833 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4156e-05 2024-09-21T15:00:43.833 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.833 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.833 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:43.833 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506225+0000", 2024-09-21T15:00:43.833 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015097 2024-09-21T15:00:43.833 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.833 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.834 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:00:43.834 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506256+0000", 2024-09-21T15:00:43.834 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0505999999999999e-05 2024-09-21T15:00:43.834 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.834 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.834 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:43.834 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.497974+0000", 2024-09-21T15:00:43.834 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99171825700000005 2024-09-21T15:00:43.834 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.834 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.834 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for missing object", 2024-09-21T15:00:43.834 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.498449+0000", 2024-09-21T15:00:43.834 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00047506399999999998 2024-09-21T15:00:43.834 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.834 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.835 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:43.835 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.501000+0000", 2024-09-21T15:00:43.835 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0025506919999999998 2024-09-21T15:00:43.835 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.835 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.835 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:43.835 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.501135+0000", 2024-09-21T15:00:43.835 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013524000000000001 2024-09-21T15:00:43.835 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.835 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:43.835 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.835 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.835 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.835 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31480 3.c 3:36ccd808:::rbd_data.11367721c19.0000000000000018:head [sparse-read 0~14821] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T15:00:43.835 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.506116+0000", 2024-09-21T15:00:43.835 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 105.320398154, 2024-09-21T15:00:43.836 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99316033599999998, 2024-09-21T15:00:43.836 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:43.836 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:43.836 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:43.836 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:43.836 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:43.836 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31480 2024-09-21T15:00:43.836 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.836 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:43.836 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.836 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:43.836 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506116+0000", 2024-09-21T15:00:43.836 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.836 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.836 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.836 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:43.837 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506116+0000", 2024-09-21T15:00:43.837 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.837 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.837 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.837 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:43.837 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506115+0000", 2024-09-21T15:00:43.837 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:00:43.837 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.837 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.837 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:43.837 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506124+0000", 2024-09-21T15:00:43.837 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0659999999999993e-06 2024-09-21T15:00:43.837 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.837 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.837 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:43.837 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506145+0000", 2024-09-21T15:00:43.837 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1447e-05 2024-09-21T15:00:43.838 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.838 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.838 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:43.838 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506161+0000", 2024-09-21T15:00:43.838 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5209e-05 2024-09-21T15:00:43.838 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.838 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.838 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:43.838 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506337+0000", 2024-09-21T15:00:43.838 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017642999999999999 2024-09-21T15:00:43.838 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.838 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.838 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:00:43.838 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506375+0000", 2024-09-21T15:00:43.838 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7536000000000001e-05 2024-09-21T15:00:43.839 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.839 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.839 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:43.839 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.498815+0000", 2024-09-21T15:00:43.839 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99244067999999996 2024-09-21T15:00:43.839 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.839 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.839 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:43.839 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.498950+0000", 2024-09-21T15:00:43.839 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013436700000000001 2024-09-21T15:00:43.839 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.839 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.839 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:43.839 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499277+0000", 2024-09-21T15:00:43.840 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00032696800000000002 2024-09-21T15:00:43.840 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.840 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:43.840 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.840 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.840 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.840 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31513 3.c 3:36de8523:::rbd_data.11367721c19.0000000000000039:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T15:00:43.840 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.509071+0000", 2024-09-21T15:00:43.840 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 105.317443954, 2024-09-21T15:00:43.840 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99038225099999999, 2024-09-21T15:00:43.840 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:43.840 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:43.840 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:43.840 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:43.840 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:43.840 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31513 2024-09-21T15:00:43.841 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.841 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:43.841 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.841 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:43.841 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509071+0000", 2024-09-21T15:00:43.841 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.841 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.841 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.841 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:43.841 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509071+0000", 2024-09-21T15:00:43.841 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.841 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.841 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.841 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:43.841 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509068+0000", 2024-09-21T15:00:43.841 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-21T15:00:43.841 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.842 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.842 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:43.842 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509082+0000", 2024-09-21T15:00:43.842 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3545e-05 2024-09-21T15:00:43.842 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.842 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.842 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:43.842 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509116+0000", 2024-09-21T15:00:43.842 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3942000000000001e-05 2024-09-21T15:00:43.842 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.842 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.842 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:43.842 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509134+0000", 2024-09-21T15:00:43.842 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8839999999999999e-05 2024-09-21T15:00:43.843 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.843 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.843 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:43.843 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509318+0000", 2024-09-21T15:00:43.843 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018326799999999999 2024-09-21T15:00:43.843 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.843 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.843 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:00:43.843 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509346+0000", 2024-09-21T15:00:43.843 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8019000000000001e-05 2024-09-21T15:00:43.843 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.843 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.843 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:43.843 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499330+0000", 2024-09-21T15:00:43.843 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.98998399299999995 2024-09-21T15:00:43.844 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.844 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.844 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:43.844 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499453+0000", 2024-09-21T15:00:43.844 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000123115 2024-09-21T15:00:43.844 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.844 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:43.844 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.844 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.844 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.844 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31515 3.c 3:35b36ddd:::rbd_data.11367721c19.000000000000003b:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T15:00:43.844 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.509177+0000", 2024-09-21T15:00:43.844 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 105.317337528, 2024-09-21T15:00:43.844 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99044870600000001, 2024-09-21T15:00:43.844 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:43.845 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:43.845 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:43.845 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:43.845 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:43.845 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31515 2024-09-21T15:00:43.845 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.845 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:43.845 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.845 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:43.845 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509177+0000", 2024-09-21T15:00:43.845 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.845 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.845 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.845 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:43.845 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509177+0000", 2024-09-21T15:00:43.845 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.846 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.846 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.846 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:43.846 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509176+0000", 2024-09-21T15:00:43.846 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:00:43.846 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.846 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.846 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:43.846 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509183+0000", 2024-09-21T15:00:43.846 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6720000000000004e-06 2024-09-21T15:00:43.846 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.846 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.846 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:43.846 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509269+0000", 2024-09-21T15:00:43.846 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5881e-05 2024-09-21T15:00:43.847 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.847 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.847 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:43.847 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509303+0000", 2024-09-21T15:00:43.847 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3367000000000001e-05 2024-09-21T15:00:43.847 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.847 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.847 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:43.847 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509425+0000", 2024-09-21T15:00:43.847 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000122199 2024-09-21T15:00:43.847 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.847 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.847 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:00:43.847 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509448+0000", 2024-09-21T15:00:43.847 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2688999999999999e-05 2024-09-21T15:00:43.847 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.848 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.848 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:43.848 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499501+0000", 2024-09-21T15:00:43.848 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99005306999999998 2024-09-21T15:00:43.848 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.848 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.848 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:43.848 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499626+0000", 2024-09-21T15:00:43.848 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012501400000000001 2024-09-21T15:00:43.848 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.848 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:43.848 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.848 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.848 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.848 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31531 3.c 3:314f0287:::rbd_data.11367721c19.000000000000004b:head [sparse-read 0~7106] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T15:00:43.848 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.509329+0000", 2024-09-21T15:00:43.848 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 105.31718550799999, 2024-09-21T15:00:43.849 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99103085300000004, 2024-09-21T15:00:43.849 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:43.849 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:43.849 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:43.849 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:43.849 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:43.849 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31531 2024-09-21T15:00:43.849 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.849 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:43.849 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.849 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:43.849 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509329+0000", 2024-09-21T15:00:43.849 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.849 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.849 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.849 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:43.850 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509329+0000", 2024-09-21T15:00:43.850 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.850 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.850 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.850 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:43.850 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509328+0000", 2024-09-21T15:00:43.850 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:00:43.850 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.850 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.850 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:43.850 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509336+0000", 2024-09-21T15:00:43.850 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3170000000000002e-06 2024-09-21T15:00:43.850 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.850 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.850 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:43.850 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509353+0000", 2024-09-21T15:00:43.851 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7214000000000002e-05 2024-09-21T15:00:43.851 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.851 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.851 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:43.851 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509366+0000", 2024-09-21T15:00:43.851 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.26e-05 2024-09-21T15:00:43.851 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.851 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.851 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:43.851 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509523+0000", 2024-09-21T15:00:43.851 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015708000000000001 2024-09-21T15:00:43.851 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.851 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.851 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:00:43.851 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509544+0000", 2024-09-21T15:00:43.851 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0783000000000001e-05 2024-09-21T15:00:43.852 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.852 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.852 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:43.852 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499727+0000", 2024-09-21T15:00:43.852 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99018345200000002 2024-09-21T15:00:43.852 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.852 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.852 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:43.852 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499949+0000", 2024-09-21T15:00:43.852 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000221358 2024-09-21T15:00:43.852 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.852 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.852 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:43.852 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.500360+0000", 2024-09-21T15:00:43.852 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00041130700000000002 2024-09-21T15:00:43.852 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.852 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:43.853 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.853 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.853 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.853 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:61580 3.b 3:d3369e4f:::rbd_data.1136862f872f.0000000000000010:head [write 0~557056 [fadvise_sequential+fadvise_nocache],zero 557056~16384,write 573440~49152 [fadvise_sequential+fadvise_nocache],zero 622592~16384,write 638976~49152 [fadvise_sequential+fadvise_nocache],zero 688128~16384,write 704512~49152 [fadvise_sequential+fadvise_nocache],zero 753664~16384,write 770048~49152 [fadvise_sequential+fadvise_nocache],zero 819200~16384,write 835584~49152 [fadvise_sequential+fadvise_nocache],zero 884736~16384,write 901120~49152 [fadvise_sequential+fadvise_nocache],zero 950272~16384,write 966656~49152 [fadvise_sequential+fadvise_nocache],zero 1015808~16384,write 1032192~16384 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-21T15:00:43.853 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:19.231716+0000", 2024-09-21T15:00:43.853 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 84.594798346000005, 2024-09-21T15:00:43.853 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0096097690000000006, 2024-09-21T15:00:43.853 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:43.853 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:43.853 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:43.853 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:43.853 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:43.853 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 61580 2024-09-21T15:00:43.853 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.853 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:43.853 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.854 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:43.854 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.231716+0000", 2024-09-21T15:00:43.854 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.854 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.854 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.854 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:43.854 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.231716+0000", 2024-09-21T15:00:43.854 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.854 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.854 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.854 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:43.854 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.231715+0000", 2024-09-21T15:00:43.854 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:00:43.854 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.854 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.855 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:43.855 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.235290+0000", 2024-09-21T15:00:43.855 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0035750399999999998 2024-09-21T15:00:43.855 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.855 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.855 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:43.855 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.235310+0000", 2024-09-21T15:00:43.855 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0120999999999999e-05 2024-09-21T15:00:43.855 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.855 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.855 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:43.855 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.235319+0000", 2024-09-21T15:00:43.855 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2420000000000006e-06 2024-09-21T15:00:43.855 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.855 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.856 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:43.856 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.235416+0000", 2024-09-21T15:00:43.856 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7204e-05 2024-09-21T15:00:43.856 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.856 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.856 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:43.856 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.235775+0000", 2024-09-21T15:00:43.856 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000358883 2024-09-21T15:00:43.856 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.856 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.856 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T15:00:43.856 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.236560+0000", 2024-09-21T15:00:43.856 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000784463 2024-09-21T15:00:43.856 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.856 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.856 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:43.857 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.241119+0000", 2024-09-21T15:00:43.857 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0045590589999999999 2024-09-21T15:00:43.857 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.857 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.857 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:43.857 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.241210+0000", 2024-09-21T15:00:43.857 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1617000000000001e-05 2024-09-21T15:00:43.857 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.857 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.857 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:43.857 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.241285+0000", 2024-09-21T15:00:43.857 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4470999999999999e-05 2024-09-21T15:00:43.857 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.857 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.857 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:43.857 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:19.241326+0000", 2024-09-21T15:00:43.857 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0967000000000003e-05 2024-09-21T15:00:43.858 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.858 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:43.858 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.858 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.858 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.858 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:73699 3.b 3:d293b54a:::rbd_data.1136f4939602.000000000000000a:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-21T15:00:43.858 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:41.971359+0000", 2024-09-21T15:00:43.858 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 61.855155789999998, 2024-09-21T15:00:43.858 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0082840980000000002, 2024-09-21T15:00:43.858 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:43.858 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:43.858 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:43.858 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:43.858 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:43.858 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 73699 2024-09-21T15:00:43.859 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.859 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:43.859 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.859 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:43.859 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.971359+0000", 2024-09-21T15:00:43.859 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.859 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.859 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.859 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:43.859 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.971359+0000", 2024-09-21T15:00:43.859 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.859 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.859 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.859 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:43.859 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.971356+0000", 2024-09-21T15:00:43.859 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-21T15:00:43.860 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.860 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.860 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:43.860 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.971370+0000", 2024-09-21T15:00:43.860 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3349999999999999e-05 2024-09-21T15:00:43.860 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.860 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.860 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:43.860 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.971393+0000", 2024-09-21T15:00:43.860 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3799999999999999e-05 2024-09-21T15:00:43.860 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.860 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.860 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:43.860 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.971405+0000", 2024-09-21T15:00:43.860 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1569000000000001e-05 2024-09-21T15:00:43.860 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.861 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.861 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:43.861 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.971510+0000", 2024-09-21T15:00:43.861 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010456899999999999 2024-09-21T15:00:43.861 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.861 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.861 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:43.861 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.971752+0000", 2024-09-21T15:00:43.861 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00024250300000000001 2024-09-21T15:00:43.861 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.861 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.861 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T15:00:43.861 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.973012+0000", 2024-09-21T15:00:43.861 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0012595429999999999 2024-09-21T15:00:43.861 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.861 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.862 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:43.862 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.979357+0000", 2024-09-21T15:00:43.862 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0063454469999999997 2024-09-21T15:00:43.862 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.862 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.862 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:43.862 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.979467+0000", 2024-09-21T15:00:43.862 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000109623 2024-09-21T15:00:43.862 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.862 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.862 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:43.862 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.979585+0000", 2024-09-21T15:00:43.862 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000118044 2024-09-21T15:00:43.862 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.863 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.863 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:43.863 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:41.979643+0000", 2024-09-21T15:00:43.863 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8069999999999998e-05 2024-09-21T15:00:43.863 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.863 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:43.863 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.863 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.863 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.863 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:82545 3.b 3:d4eb1ee5:::rbd_data.11368296f791.0000000000000000:head [write 0~671744 [fadvise_sequential+fadvise_nocache],write 704512~65536 [fadvise_sequential+fadvise_nocache],write 786432~49152 [fadvise_sequential+fadvise_nocache],write 868352~49152 [fadvise_sequential+fadvise_nocache],write 933888~65536 [fadvise_sequential+fadvise_nocache],write 1032192~32768 [fadvise_sequential+fadvise_nocache],write 1097728~65536 [fadvise_sequential+fadvise_nocache],write 1179648~49152 [fadvise_sequential+fadvise_nocache],write 1261568~49152 [fadvise_sequential+fadvise_nocache],write 1327104~65536 [fadvise_sequential+fadvise_nocache],write 1425408~32768 [fadvise_sequential+fadvise_nocache],write 1490944~65536 [fadvise_sequential+fadvise_nocache],write 1572864~8192 [fadvise_sequential+fadvise_nocache],write 1630208~49152 [fadvise_sequential+fadvise_nocache],write 1703936~40960 [fadvise_sequential+fadvise_nocache],write 1794048~40960 [fadvise_sequential+fadvise_nocache],write 1859584~49152 [fadvise_sequential+fadvise_nocache],write 1957888~8192 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:00:43.863 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:47.416089+0000", 2024-09-21T15:00:43.863 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 56.410425783000001, 2024-09-21T15:00:43.863 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011896455, 2024-09-21T15:00:43.863 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:43.863 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:43.864 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:43.864 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:43.864 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:43.864 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82545 2024-09-21T15:00:43.864 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.864 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:43.864 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.864 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:43.864 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.416089+0000", 2024-09-21T15:00:43.864 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.864 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.864 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.864 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:43.864 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.416089+0000", 2024-09-21T15:00:43.864 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.865 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.865 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.865 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:43.865 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.416087+0000", 2024-09-21T15:00:43.865 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:00:43.865 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.865 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.865 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:43.865 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.419643+0000", 2024-09-21T15:00:43.865 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0035555550000000002 2024-09-21T15:00:43.865 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.865 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.865 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:43.865 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.419663+0000", 2024-09-21T15:00:43.865 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0202000000000001e-05 2024-09-21T15:00:43.866 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.866 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.866 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:43.866 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.419676+0000", 2024-09-21T15:00:43.866 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3207000000000001e-05 2024-09-21T15:00:43.866 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.866 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.866 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:43.866 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.419759+0000", 2024-09-21T15:00:43.866 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3521000000000002e-05 2024-09-21T15:00:43.866 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.866 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.866 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op must wait for map", 2024-09-21T15:00:43.867 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.419768+0000", 2024-09-21T15:00:43.867 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4479999999999994e-06 2024-09-21T15:00:43.867 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.867 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.867 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:43.867 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.422171+0000", 2024-09-21T15:00:43.867 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0024035469999999998 2024-09-21T15:00:43.867 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.867 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.867 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:43.867 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.422350+0000", 2024-09-21T15:00:43.867 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000179001 2024-09-21T15:00:43.867 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.867 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.867 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T15:00:43.868 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.422710+0000", 2024-09-21T15:00:43.868 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000359878 2024-09-21T15:00:43.868 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.868 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.868 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:43.868 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.426676+0000", 2024-09-21T15:00:43.868 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0039655790000000003 2024-09-21T15:00:43.868 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.868 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.868 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:43.868 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.427884+0000", 2024-09-21T15:00:43.868 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001208207 2024-09-21T15:00:43.868 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.868 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.868 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:43.868 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.427953+0000", 2024-09-21T15:00:43.869 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8533000000000003e-05 2024-09-21T15:00:43.869 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.869 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.869 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:43.869 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.427985+0000", 2024-09-21T15:00:43.869 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2481999999999998e-05 2024-09-21T15:00:43.869 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.869 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:43.869 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.869 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.869 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.869 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:83735 3.b 3:d3640016:::rbd_data.11368296f791.0000000000000006:40 [sparse-read 393216~131072 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-21T15:00:43.869 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:51.156767+0000", 2024-09-21T15:00:43.869 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 52.66974716, 2024-09-21T15:00:43.869 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0082826410000000003, 2024-09-21T15:00:43.870 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:43.870 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:43.870 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:43.870 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:43.870 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:43.870 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 83735 2024-09-21T15:00:43.870 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.870 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:43.870 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.870 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:43.870 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156767+0000", 2024-09-21T15:00:43.870 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.870 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.870 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.870 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:43.870 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156767+0000", 2024-09-21T15:00:43.871 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.871 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.871 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.871 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:43.871 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156767+0000", 2024-09-21T15:00:43.871 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:43.871 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.871 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.871 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:43.871 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156771+0000", 2024-09-21T15:00:43.871 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0940000000000001e-06 2024-09-21T15:00:43.871 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.871 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.871 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:43.871 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156781+0000", 2024-09-21T15:00:43.871 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0267e-05 2024-09-21T15:00:43.872 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.872 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.872 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:43.872 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156789+0000", 2024-09-21T15:00:43.872 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7659999999999998e-06 2024-09-21T15:00:43.872 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.872 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.872 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:43.872 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.164742+0000", 2024-09-21T15:00:43.872 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0079534519999999997 2024-09-21T15:00:43.872 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.872 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.872 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:43.872 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.164820+0000", 2024-09-21T15:00:43.872 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7927000000000001e-05 2024-09-21T15:00:43.872 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.873 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.873 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:43.873 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165050+0000", 2024-09-21T15:00:43.873 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000229731 2024-09-21T15:00:43.873 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.873 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:43.873 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.873 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.873 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.873 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:83736 3.b 3:d3640016:::rbd_data.11368296f791.0000000000000006:40 [sparse-read 1835008~0 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-21T15:00:43.873 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:51.156796+0000", 2024-09-21T15:00:43.873 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 52.669718156999998, 2024-09-21T15:00:43.873 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0085139110000000007, 2024-09-21T15:00:43.873 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:43.873 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:43.873 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:43.874 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:43.874 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:43.874 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 83736 2024-09-21T15:00:43.874 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.874 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:43.874 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.874 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:43.874 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156796+0000", 2024-09-21T15:00:43.874 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.874 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.874 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.874 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:43.874 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156796+0000", 2024-09-21T15:00:43.874 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.875 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.875 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.875 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:43.875 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156796+0000", 2024-09-21T15:00:43.875 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:43.875 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.875 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.875 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:43.875 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156800+0000", 2024-09-21T15:00:43.875 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.34e-06 2024-09-21T15:00:43.875 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.875 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.875 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:43.875 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156813+0000", 2024-09-21T15:00:43.876 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2532e-05 2024-09-21T15:00:43.876 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.876 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.876 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:43.876 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156821+0000", 2024-09-21T15:00:43.876 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2069999999999995e-06 2024-09-21T15:00:43.876 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.876 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.876 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:43.876 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165100+0000", 2024-09-21T15:00:43.876 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0082788730000000008 2024-09-21T15:00:43.876 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.876 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.876 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:43.876 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165192+0000", 2024-09-21T15:00:43.876 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2095999999999994e-05 2024-09-21T15:00:43.877 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.877 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.877 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:43.877 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165310+0000", 2024-09-21T15:00:43.877 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000118461 2024-09-21T15:00:43.877 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.877 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:43.877 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.877 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.877 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.877 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:83739 3.b 3:d4eb1ee5:::rbd_data.11368296f791.0000000000000000:40 [sparse-read 524288~131072 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-21T15:00:43.877 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:51.156827+0000", 2024-09-21T15:00:43.877 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 52.669687074000002, 2024-09-21T15:00:43.877 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0087319900000000002, 2024-09-21T15:00:43.878 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:43.878 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:43.878 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:43.878 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:43.878 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:43.878 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 83739 2024-09-21T15:00:43.878 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.878 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:43.878 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.878 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:43.878 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156827+0000", 2024-09-21T15:00:43.878 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.878 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.878 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.879 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:43.879 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156827+0000", 2024-09-21T15:00:43.879 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.879 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.879 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.879 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:43.879 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156827+0000", 2024-09-21T15:00:43.879 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:43.879 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.879 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.879 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:43.879 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156831+0000", 2024-09-21T15:00:43.879 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9940000000000002e-06 2024-09-21T15:00:43.879 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.879 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.879 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:43.880 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156841+0000", 2024-09-21T15:00:43.880 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6760000000000002e-06 2024-09-21T15:00:43.880 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.880 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.880 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:43.880 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156853+0000", 2024-09-21T15:00:43.880 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1972e-05 2024-09-21T15:00:43.880 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.880 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.880 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:43.880 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165334+0000", 2024-09-21T15:00:43.880 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0084816939999999997 2024-09-21T15:00:43.880 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.880 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.880 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:43.880 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165375+0000", 2024-09-21T15:00:43.881 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0911000000000002e-05 2024-09-21T15:00:43.881 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.881 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.881 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:43.881 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165559+0000", 2024-09-21T15:00:43.881 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018429300000000001 2024-09-21T15:00:43.881 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.881 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:43.881 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.881 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.881 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.881 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:83740 3.b 3:d4eb1ee5:::rbd_data.11368296f791.0000000000000000:40 [sparse-read 1966080~0 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-21T15:00:43.881 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:51.156859+0000", 2024-09-21T15:00:43.881 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 52.669655083999999, 2024-09-21T15:00:43.881 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0088540089999999995, 2024-09-21T15:00:43.882 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:43.882 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:43.882 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:43.882 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:43.882 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:43.882 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 83740 2024-09-21T15:00:43.882 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.882 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:43.882 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.882 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:43.882 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156859+0000", 2024-09-21T15:00:43.882 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.882 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.882 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.882 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:43.883 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156859+0000", 2024-09-21T15:00:43.883 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.883 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.883 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.883 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:43.883 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156859+0000", 2024-09-21T15:00:43.883 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:43.883 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.883 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.883 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:43.883 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156863+0000", 2024-09-21T15:00:43.883 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.844e-06 2024-09-21T15:00:43.883 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.883 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.883 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:43.883 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156873+0000", 2024-09-21T15:00:43.884 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0069e-05 2024-09-21T15:00:43.884 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.884 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.884 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:43.884 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156880+0000", 2024-09-21T15:00:43.884 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4240000000000004e-06 2024-09-21T15:00:43.884 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.884 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.884 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:43.884 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165590+0000", 2024-09-21T15:00:43.884 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0087098100000000001 2024-09-21T15:00:43.884 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.884 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.884 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:43.884 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165656+0000", 2024-09-21T15:00:43.885 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5541000000000001e-05 2024-09-21T15:00:43.885 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.885 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.885 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:43.885 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165713+0000", 2024-09-21T15:00:43.885 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7865000000000002e-05 2024-09-21T15:00:43.885 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.885 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:43.885 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.885 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.885 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.885 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:83751 3.b 3:d3640016:::rbd_data.11368296f791.0000000000000006:40 [sparse-read 1179648~0 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-21T15:00:43.885 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:51.156888+0000", 2024-09-21T15:00:43.885 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 52.669626823000002, 2024-09-21T15:00:43.886 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0090039539999999998, 2024-09-21T15:00:43.886 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:43.886 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:43.886 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:43.886 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:43.886 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:43.886 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 83751 2024-09-21T15:00:43.886 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.886 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:43.886 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.886 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:43.886 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156888+0000", 2024-09-21T15:00:43.886 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.886 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.887 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.887 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:43.887 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156888+0000", 2024-09-21T15:00:43.887 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.887 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.887 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.887 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:43.887 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156887+0000", 2024-09-21T15:00:43.887 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:43.887 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.887 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.887 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:43.887 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156891+0000", 2024-09-21T15:00:43.887 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.518e-06 2024-09-21T15:00:43.887 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.888 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.888 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:43.888 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156900+0000", 2024-09-21T15:00:43.888 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1759999999999999e-06 2024-09-21T15:00:43.888 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.888 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.888 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:43.888 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156908+0000", 2024-09-21T15:00:43.888 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8439999999999998e-06 2024-09-21T15:00:43.888 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.888 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.888 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:43.888 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165738+0000", 2024-09-21T15:00:43.888 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0088305689999999999 2024-09-21T15:00:43.888 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.888 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.889 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:43.889 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165804+0000", 2024-09-21T15:00:43.889 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5535000000000002e-05 2024-09-21T15:00:43.889 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.889 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.889 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:43.889 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165892+0000", 2024-09-21T15:00:43.889 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7860000000000002e-05 2024-09-21T15:00:43.889 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.889 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:43.889 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.889 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.889 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.889 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:83752 3.b 3:d3640016:::rbd_data.11368296f791.0000000000000006:40 [sparse-read 2621440~0 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-21T15:00:43.889 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:51.156915+0000", 2024-09-21T15:00:43.889 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 52.669599276, 2024-09-21T15:00:43.890 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0090768180000000004, 2024-09-21T15:00:43.890 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:43.890 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:43.890 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:43.890 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:43.890 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:43.890 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 83752 2024-09-21T15:00:43.890 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.890 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:43.890 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.890 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:43.890 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156915+0000", 2024-09-21T15:00:43.890 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.890 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.891 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.891 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:43.891 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156915+0000", 2024-09-21T15:00:43.891 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.891 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.891 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.891 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:43.891 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156915+0000", 2024-09-21T15:00:43.891 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:43.891 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.891 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.891 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:43.891 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156919+0000", 2024-09-21T15:00:43.891 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2679999999999996e-06 2024-09-21T15:00:43.891 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.891 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.892 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:43.892 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156929+0000", 2024-09-21T15:00:43.892 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.031e-05 2024-09-21T15:00:43.892 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.892 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.892 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:43.892 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156937+0000", 2024-09-21T15:00:43.892 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8820000000000001e-06 2024-09-21T15:00:43.892 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.892 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.892 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:43.892 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165909+0000", 2024-09-21T15:00:43.892 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0089722989999999996 2024-09-21T15:00:43.892 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.892 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.892 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:43.893 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165948+0000", 2024-09-21T15:00:43.893 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8114999999999997e-05 2024-09-21T15:00:43.893 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.893 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.893 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:43.893 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.165992+0000", 2024-09-21T15:00:43.893 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4503999999999999e-05 2024-09-21T15:00:43.893 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.893 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:43.893 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.893 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.893 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.893 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:83755 3.b 3:d4eb1ee5:::rbd_data.11368296f791.0000000000000000:40 [sparse-read 1310720~0 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-21T15:00:43.893 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:51.156944+0000", 2024-09-21T15:00:43.893 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 52.669570202000003, 2024-09-21T15:00:43.894 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.009159146, 2024-09-21T15:00:43.894 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:43.894 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:43.894 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:43.894 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:43.894 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:43.894 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 83755 2024-09-21T15:00:43.894 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.894 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:43.894 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.894 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:43.894 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156944+0000", 2024-09-21T15:00:43.894 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.894 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.895 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.895 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:43.895 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156944+0000", 2024-09-21T15:00:43.895 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.895 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.895 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.895 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:43.895 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156944+0000", 2024-09-21T15:00:43.895 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:43.895 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.895 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.895 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:43.895 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156948+0000", 2024-09-21T15:00:43.895 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5279999999999997e-06 2024-09-21T15:00:43.896 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.896 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.896 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:43.896 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156959+0000", 2024-09-21T15:00:43.896 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0215e-05 2024-09-21T15:00:43.896 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.896 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.896 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:43.896 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.156967+0000", 2024-09-21T15:00:43.896 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6119999999999999e-06 2024-09-21T15:00:43.896 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.896 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.896 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:43.896 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.166007+0000", 2024-09-21T15:00:43.896 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0090395260000000009 2024-09-21T15:00:43.897 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.897 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.897 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:43.897 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.166053+0000", 2024-09-21T15:00:43.897 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6281e-05 2024-09-21T15:00:43.897 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.897 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.897 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:43.897 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.166103+0000", 2024-09-21T15:00:43.897 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0510000000000003e-05 2024-09-21T15:00:43.897 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.897 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:43.897 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.897 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.897 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.897 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:83763 3.b 3:d3640016:::rbd_data.11368296f791.0000000000000006:40 [sparse-read 524288~81920 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-21T15:00:43.898 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:51.157975+0000", 2024-09-21T15:00:43.898 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 52.668539094000003, 2024-09-21T15:00:43.898 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.008327348, 2024-09-21T15:00:43.898 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:43.898 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:43.898 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:43.898 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:43.898 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:43.898 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 83763 2024-09-21T15:00:43.898 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.898 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:43.898 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.898 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:43.898 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.157975+0000", 2024-09-21T15:00:43.898 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.898 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.899 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.899 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:43.899 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.157975+0000", 2024-09-21T15:00:43.899 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.899 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.899 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.899 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:43.899 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.157970+0000", 2024-09-21T15:00:43.899 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999943 2024-09-21T15:00:43.899 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.899 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.899 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:43.899 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.157985+0000", 2024-09-21T15:00:43.899 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4807000000000001e-05 2024-09-21T15:00:43.899 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.899 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.900 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:43.900 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.158012+0000", 2024-09-21T15:00:43.900 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7600999999999999e-05 2024-09-21T15:00:43.900 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.900 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.900 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:43.900 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.158019+0000", 2024-09-21T15:00:43.900 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9319999999999999e-06 2024-09-21T15:00:43.900 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.900 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.900 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:43.900 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.166118+0000", 2024-09-21T15:00:43.900 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0080992489999999993 2024-09-21T15:00:43.900 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.900 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.900 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:43.901 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.166156+0000", 2024-09-21T15:00:43.901 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7639999999999999e-05 2024-09-21T15:00:43.901 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.901 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.901 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:43.901 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.166303+0000", 2024-09-21T15:00:43.901 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000146799 2024-09-21T15:00:43.901 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.901 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:43.901 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.901 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.901 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.901 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:83764 3.b 3:d3640016:::rbd_data.11368296f791.0000000000000006:40 [sparse-read 1966080~0 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-21T15:00:43.901 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:51.158026+0000", 2024-09-21T15:00:43.901 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 52.668488660999998, 2024-09-21T15:00:43.901 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0083785839999999997, 2024-09-21T15:00:43.901 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:43.902 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:43.902 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:43.902 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:43.902 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:43.902 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 83764 2024-09-21T15:00:43.902 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.902 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:43.902 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.902 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:43.902 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.158026+0000", 2024-09-21T15:00:43.902 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.902 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.902 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.902 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:43.902 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.158026+0000", 2024-09-21T15:00:43.903 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.903 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.903 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.903 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:43.903 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.158025+0000", 2024-09-21T15:00:43.903 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:43.903 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.903 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.903 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:43.903 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.158053+0000", 2024-09-21T15:00:43.903 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7594e-05 2024-09-21T15:00:43.903 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.903 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.903 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:43.903 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.158061+0000", 2024-09-21T15:00:43.903 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2069999999999995e-06 2024-09-21T15:00:43.904 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.904 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.904 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:43.904 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.158067+0000", 2024-09-21T15:00:43.904 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3799999999999999e-06 2024-09-21T15:00:43.904 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.904 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.904 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:43.904 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.166321+0000", 2024-09-21T15:00:43.904 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0082539870000000008 2024-09-21T15:00:43.904 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.904 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.904 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:43.904 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.166360+0000", 2024-09-21T15:00:43.905 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8597999999999999e-05 2024-09-21T15:00:43.905 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.905 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.905 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:43.905 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:51.166404+0000", 2024-09-21T15:00:43.905 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4378000000000001e-05 2024-09-21T15:00:43.905 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.905 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:43.905 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.905 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.905 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.905 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:97675 3.b 3:d321acb8:::rbd_data.1136bfc79a70.0000000000000003:head [write 0~4718592 [fadvise_sequential+fadvise_nocache],write 5242880~147456 [fadvise_sequential+fadvise_nocache],write 5455872~147456 [fadvise_sequential+fadvise_nocache],write 5668864~229376 [fadvise_sequential+fadvise_nocache],write 5963776~147456 [fadvise_sequential+fadvise_nocache],write 6176768~229376 [fadvise_sequential+fadvise_nocache],write 6471680~147456 [fadvise_sequential+fadvise_nocache],write 6684672~229376 [fadvise_sequential+fadvise_nocache],write 6979584~147456 [fadvise_sequential+fadvise_nocache],write 7192576~229376 [fadvise_sequential+fadvise_nocache],write 7487488~147456 [fadvise_sequential+fadvise_nocache],write 7700480~147456 [fadvise_sequential+fadvise_nocache],write 7864320~65536 [fadvise_sequential+fadvise_nocache],write 7995392~147456 [fadvise_sequential+fadvise_nocache],write 8208384~147456 [fadvise_sequential+fadvise_nocache],write 8388608~49152 [fadvise_sequential+fadvise_nocache],write 8536064~114688 [fadvise_sequential+fadvise_nocache],write 8749056~114688 [fadvise_sequential+fadvise_nocache],write 8912896~32768 [fadvise_sequential+fadvise_nocache],write 9125888~32768 [fadvise_sequential+fadvise_nocache],write 9338880~32768 [fadvise_sequential+fadvise_nocache],write 9437184~16384 [fadvise_sequential+fadvise_nocache],write 9633792~32768 [fadvise_sequential+fadvise_nocache],write 9846784~32768 [fadvise_sequential+fadvise_nocache],write 9961472~114688 [fadvise_sequential+fadvise_nocache],write 10092544~196608 [fadvise_sequential+fadvise_nocache],write 10305536~278528 [fadvise_sequential+fadvise_nocache],write 10600448~196608 [fadvise_sequential+fadvise_nocache],write 10813440~278528 [fadvise_sequential+fadvise_nocache],write 11108352~196608 [fadvise_sequential+fadvise_nocache],write 11321344~196608 [fadvise_sequential+fadvise_nocache],write 11534336~65536 [fadvise_sequential+fadvise_nocache],write 11632640~98304 [fadvise_sequential+fadvise_nocache],write 11747328~65536 [fadvise_sequential+fadvise_nocache],write 11845632~98304 [fadvise_sequential+fadvise_nocache],write 11960320~65536 [fadvise_sequential+fadvise_nocache],write 12058624~49152 [fadvise_sequential+fadvise_nocache],write 12140544~98304 [fadvise_sequential+fadvise_nocache],write 12255232~65536 [fadvise_sequential+fadvise_nocache],write 12353536~98304 [fadvise_sequential+fadvise_nocache],write 12468224~65536 [fadvise_sequential+fadvise_nocache],write 12566528~16384 [fadvise_sequential+fadvise_nocache],write 12648448~98304 [fadvise_sequential+fadvise_nocache],write 12861440~98304 [fadvise_sequential+fadvise_nocache],write 13074432~32768 [fadvise_sequential+fadvise_nocache],write 13156352~98304 [fadvise_sequential+fadvise_nocache],write 13369344~98304 [fadvise_sequential+fadvise_nocache],write 13582336~49152 [fadvise_sequential+fadvise_nocache],write 13664256~98304 [fadvise_sequential+fadvise_nocache],write 13877248~98304 [fadvise_sequential+fadvise_nocache],write 14090240~65536 [fadvise_sequential+fadvise_nocache],write 15286272~81920 [fadvise_sequential+fadvise_nocache],write 15499264~81920 [fadvise_sequential+fadvise_nocache],write 15712256~16384 [fadvise_sequential+fadvise_nocache],write 15794176~81920 [fadvise_sequential+fadvise_nocache],write 16007168~81920 [fadvise_sequential+fadvise_nocache],write 16220160~32768 [fadvise_sequential+fadvise_nocache],write 16302080~81920 [fadvise_sequential+fadvise_nocache],write 16515072~81920 [fadvise_sequential+fadvise_nocache],write 16728064~49152 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:43.905 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.302412+0000", 2024-09-21T15:00:43.905 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 38.524102994000003, 2024-09-21T15:00:43.905 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.067380532000000007, 2024-09-21T15:00:43.906 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:43.906 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:43.906 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:43.906 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:43.906 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:43.906 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97675 2024-09-21T15:00:43.906 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.906 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:43.906 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.906 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:43.906 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.302412+0000", 2024-09-21T15:00:43.906 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.906 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.906 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.906 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:43.906 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.302412+0000", 2024-09-21T15:00:43.907 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.907 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.907 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.907 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:43.907 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.302409+0000", 2024-09-21T15:00:43.907 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-21T15:00:43.907 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.907 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.907 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:43.907 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328550+0000", 2024-09-21T15:00:43.907 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.026140674999999999 2024-09-21T15:00:43.907 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.907 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.907 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:43.907 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328584+0000", 2024-09-21T15:00:43.907 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4165999999999998e-05 2024-09-21T15:00:43.908 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.908 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.908 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:43.908 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328599+0000", 2024-09-21T15:00:43.908 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5505000000000001e-05 2024-09-21T15:00:43.908 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.908 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.908 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:43.908 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328704+0000", 2024-09-21T15:00:43.908 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010487199999999999 2024-09-21T15:00:43.908 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.908 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.908 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:43.908 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.329082+0000", 2024-09-21T15:00:43.908 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00037804899999999998 2024-09-21T15:00:43.908 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.908 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.909 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T15:00:43.909 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.331158+0000", 2024-09-21T15:00:43.909 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0020755449999999998 2024-09-21T15:00:43.909 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.909 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.909 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:43.909 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.360553+0000", 2024-09-21T15:00:43.909 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.029395553000000001 2024-09-21T15:00:43.909 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.909 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.909 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:43.909 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.369496+0000", 2024-09-21T15:00:43.909 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0089428619999999993 2024-09-21T15:00:43.909 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.909 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.909 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:43.910 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.369686+0000", 2024-09-21T15:00:43.910 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019000399999999999 2024-09-21T15:00:43.910 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.910 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.910 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:43.910 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.369792+0000", 2024-09-21T15:00:43.910 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000105775 2024-09-21T15:00:43.910 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.910 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:43.910 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.910 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.910 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.910 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:97693 3.b 3:d7d7b39b:::rbd_data.1136b9c49b1c.000000000000000c:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:43.910 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.328643+0000", 2024-09-21T15:00:43.910 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 38.497871207999999, 2024-09-21T15:00:43.911 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.031062136000000001, 2024-09-21T15:00:43.911 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:43.911 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:43.911 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:43.911 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:43.911 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:43.911 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97693 2024-09-21T15:00:43.911 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.911 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:43.911 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.911 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:43.911 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328643+0000", 2024-09-21T15:00:43.911 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.911 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.911 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.911 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:43.912 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328643+0000", 2024-09-21T15:00:43.912 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.912 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.912 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.912 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:43.912 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328642+0000", 2024-09-21T15:00:43.912 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:00:43.912 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.912 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.912 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:43.912 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328650+0000", 2024-09-21T15:00:43.912 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7770000000000008e-06 2024-09-21T15:00:43.912 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.912 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.912 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:43.913 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328662+0000", 2024-09-21T15:00:43.913 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2026e-05 2024-09-21T15:00:43.913 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.913 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.913 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:43.913 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328671+0000", 2024-09-21T15:00:43.913 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.585e-06 2024-09-21T15:00:43.913 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.913 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.913 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:43.913 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.359498+0000", 2024-09-21T15:00:43.913 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.030827001 2024-09-21T15:00:43.913 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.913 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.914 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:43.914 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.359603+0000", 2024-09-21T15:00:43.914 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010512400000000001 2024-09-21T15:00:43.914 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.914 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.914 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:43.914 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.359705+0000", 2024-09-21T15:00:43.914 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000102327 2024-09-21T15:00:43.914 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.914 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:43.914 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.914 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.914 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.914 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:97724 3.b 3:d4bf6bcf:::rbd_data.1136b9c49b1c.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:43.914 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.328704+0000", 2024-09-21T15:00:43.914 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 38.497810936, 2024-09-21T15:00:43.915 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.031157957, 2024-09-21T15:00:43.915 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:43.915 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:43.915 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:43.915 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:43.915 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:43.915 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97724 2024-09-21T15:00:43.915 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.915 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:43.915 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.915 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:43.915 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328704+0000", 2024-09-21T15:00:43.915 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.915 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.915 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.916 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:43.916 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328704+0000", 2024-09-21T15:00:43.916 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:43.916 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.916 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.916 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:43.916 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328703+0000", 2024-09-21T15:00:43.916 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:00:43.916 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.916 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.916 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:43.916 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328708+0000", 2024-09-21T15:00:43.916 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0050000000000004e-06 2024-09-21T15:00:43.916 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.916 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.917 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:43.917 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328719+0000", 2024-09-21T15:00:43.917 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1052e-05 2024-09-21T15:00:43.917 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.917 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.917 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:43.917 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328727+0000", 2024-09-21T15:00:43.917 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4519999999999997e-06 2024-09-21T15:00:43.917 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.917 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.917 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:43.917 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.359737+0000", 2024-09-21T15:00:43.917 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.031009341999999999 2024-09-21T15:00:43.917 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.917 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.917 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:43.918 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.359806+0000", 2024-09-21T15:00:43.918 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9750000000000001e-05 2024-09-21T15:00:43.918 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:43.918 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:43.918 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:43.918 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.359862+0000", 2024-09-21T15:00:43.918 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5084999999999999e-05 2024-09-21T15:00:43.918 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.918 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:43.918 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.918 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:43.918 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:43.918 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:43.919 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T15:00:43.920 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:00:43.915+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:00:44.020 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:00:44.017+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:00:44.036 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:44.036 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:00:44.036 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T15:00:44.036 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:44.046 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T15:00:44.121 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:00:44.117+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:00:44.152 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:44.152 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:00:44.152 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T15:00:44.152 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T15:00:44.152 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:44.162 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T15:00:44.221 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:00:44.219+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:00:44.267 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:44.268 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T15:00:44.268 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T15:00:44.268 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T15:00:44.268 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.268 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:97677 3.15 3:a8e72133:::rbd_data.1136bfc79a70.0000000000000005:head [write 0~4440027 [fadvise_sequential+fadvise_nocache],write 4751360~147456 [fadvise_sequential+fadvise_nocache],write 4964352~147456 [fadvise_sequential+fadvise_nocache],write 5177344~65536 [fadvise_sequential+fadvise_nocache],write 5259264~147456 [fadvise_sequential+fadvise_nocache],write 5472256~147456 [fadvise_sequential+fadvise_nocache],write 5685248~229376 [fadvise_sequential+fadvise_nocache],write 5980160~147456 [fadvise_sequential+fadvise_nocache],write 6193152~229376 [fadvise_sequential+fadvise_nocache],write 6488064~147456 [fadvise_sequential+fadvise_nocache],write 6701056~229376 [fadvise_sequential+fadvise_nocache],write 6995968~147456 [fadvise_sequential+fadvise_nocache],write 7208960~229376 [fadvise_sequential+fadvise_nocache],write 7503872~147456 [fadvise_sequential+fadvise_nocache],write 7716864~229376 [fadvise_sequential+fadvise_nocache],write 8044544~114688 [fadvise_sequential+fadvise_nocache],write 8257536~114688 [fadvise_sequential+fadvise_nocache],write 8388608~65536 [fadvise_sequential+fadvise_nocache],write 8552448~114688 [fadvise_sequential+fadvise_nocache],write 8765440~114688 [fadvise_sequential+fadvise_nocache],write 8929280~32768 [fadvise_sequential+fadvise_nocache],write 9142272~32768 [fadvise_sequential+fadvise_nocache],write 9355264~32768 [fadvise_sequential+fadvise_nocache],write 9437184~32768 [fadvise_sequential+fadvise_nocache],write 9650176~32768 [fadvise_sequential+fadvise_nocache],write 9863168~32768 [fadvise_sequential+fadvise_nocache],write 9961472~131072 [fadvise_sequential+fadvise_nocache],write 10108928~196608 [fadvise_sequential+fadvise_nocache],write 10321920~278528 [fadvise_sequential+fadvise_nocache],write 10616832~196608 [fadvise_sequential+fadvise_nocache],write 10829824~278528 [fadvise_sequential+fadvise_nocache],write 11124736~196608 [fadvise_sequential+fadvise_nocache],write 11337728~196608 [fadvise_sequential+fadvise_nocache],write 11550720~65536 [fadvise_sequential+fadvise_nocache],write 11649024~98304 [fadvise_sequential+fadvise_nocache],write 11763712~65536 [fadvise_sequential+fadvise_nocache],write 11862016~98304 [fadvise_sequential+fadvise_nocache],write 11976704~65536 [fadvise_sequential+fadvise_nocache],write 12058624~65536 [fadvise_sequential+fadvise_nocache],write 12156928~98304 [fadvise_sequential+fadvise_nocache],write 12271616~65536 [fadvise_sequential+fadvise_nocache],write 12369920~98304 [fadvise_sequential+fadvise_nocache],write 12484608~65536 [fadvise_sequential+fadvise_nocache],write 12664832~98304 [fadvise_sequential+fadvise_nocache],write 12877824~98304 [fadvise_sequential+fadvise_nocache],write 13090816~16384 [fadvise_sequential+fadvise_nocache],write 13172736~98304 [fadvise_sequential+fadvise_nocache],write 13385728~98304 [fadvise_sequential+fadvise_nocache],write 13598720~32768 [fadvise_sequential+fadvise_nocache],write 13680640~98304 [fadvise_sequential+fadvise_nocache],write 13893632~98304 [fadvise_sequential+fadvise_nocache],write 14106624~49152 [fadvise_sequential+fadvise_nocache],write 14794752~81920 [fadvise_sequential+fadvise_nocache],write 15007744~81920 [fadvise_sequential+fadvise_nocache],write 15302656~81920 [fadvise_sequential+fadvise_nocache],write 15515648~81920 [fadvise_sequential+fadvise_nocache],write 15810560~81920 [fadvise_sequential+fadvise_nocache],write 16023552~81920 [fadvise_sequential+fadvise_nocache],write 16236544~16384 [fadvise_sequential+fadvise_nocache],write 16318464~81920 [fadvise_sequential+fadvise_nocache],write 16531456~81920 [fadvise_sequential+fadvise_nocache],write 16744448~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:44.268 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.302946+0000", 2024-09-21T15:00:44.268 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 38.964173557999999, 2024-09-21T15:00:44.268 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.091231204999999996, 2024-09-21T15:00:44.268 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:44.268 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:44.268 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:44.268 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:44.268 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:44.268 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 97677 2024-09-21T15:00:44.268 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.269 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:44.269 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.269 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:44.269 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.302946+0000", 2024-09-21T15:00:44.269 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:44.269 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.269 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.269 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:44.269 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.302946+0000", 2024-09-21T15:00:44.269 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:44.269 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.269 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.269 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:44.269 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.302942+0000", 2024-09-21T15:00:44.269 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999962 2024-09-21T15:00:44.269 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.270 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.270 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:44.270 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311253+0000", 2024-09-21T15:00:44.270 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0083105360000000003 2024-09-21T15:00:44.270 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.270 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.270 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:44.270 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311283+0000", 2024-09-21T15:00:44.270 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0065000000000001e-05 2024-09-21T15:00:44.270 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.270 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.270 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:44.270 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311296+0000", 2024-09-21T15:00:44.270 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3206e-05 2024-09-21T15:00:44.270 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.270 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.271 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:44.271 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311404+0000", 2024-09-21T15:00:44.271 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000107514 2024-09-21T15:00:44.271 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.271 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.271 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:44.271 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311795+0000", 2024-09-21T15:00:44.271 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000390913 2024-09-21T15:00:44.271 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.271 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.271 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 6", 2024-09-21T15:00:44.271 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.314686+0000", 2024-09-21T15:00:44.271 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0028915659999999999 2024-09-21T15:00:44.271 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.271 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.271 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:44.272 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.368178+0000", 2024-09-21T15:00:44.272 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.053491706 2024-09-21T15:00:44.272 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.272 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.272 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:44.272 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.393925+0000", 2024-09-21T15:00:44.272 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025747289 2024-09-21T15:00:44.272 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.272 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.272 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:44.272 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.394127+0000", 2024-09-21T15:00:44.272 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000201444 2024-09-21T15:00:44.272 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.272 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.272 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:44.272 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.394177+0000", 2024-09-21T15:00:44.273 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0689000000000002e-05 2024-09-21T15:00:44.273 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:44.273 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:44.273 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:44.273 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.273 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.273 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:97673 3.1d e122/24 3:bd91946e:::rbd_data.1136bfc79a70.0000000000000002:head v 122'162, mlcod=120'160)", 2024-09-21T15:00:44.273 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.313730+0000", 2024-09-21T15:00:44.273 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 38.953389233000003, 2024-09-21T15:00:44.273 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055373462999999998, 2024-09-21T15:00:44.273 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:44.273 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:44.273 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:44.273 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.273 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:44.273 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.313730+0000", 2024-09-21T15:00:44.273 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:44.274 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.274 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.274 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:44.274 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.313730+0000", 2024-09-21T15:00:44.274 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:44.274 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.274 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.274 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:44.274 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.313730+0000", 2024-09-21T15:00:44.274 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:44.274 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.274 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.274 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:44.274 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.336113+0000", 2024-09-21T15:00:44.275 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022382962999999999 2024-09-21T15:00:44.275 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.275 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.275 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:44.275 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.336140+0000", 2024-09-21T15:00:44.275 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7248999999999999e-05 2024-09-21T15:00:44.275 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.275 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.275 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:44.275 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.336151+0000", 2024-09-21T15:00:44.275 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1015e-05 2024-09-21T15:00:44.275 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.275 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.275 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:44.275 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.355428+0000", 2024-09-21T15:00:44.275 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019276379999999999 2024-09-21T15:00:44.276 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.276 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.276 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:44.276 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.355484+0000", 2024-09-21T15:00:44.276 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6820999999999997e-05 2024-09-21T15:00:44.276 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.276 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.276 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:44.276 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.369040+0000", 2024-09-21T15:00:44.276 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013555466 2024-09-21T15:00:44.276 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.276 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.276 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:44.276 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.369104+0000", 2024-09-21T15:00:44.276 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3986999999999996e-05 2024-09-21T15:00:44.276 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:44.277 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:44.277 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:44.277 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.277 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.277 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:113889 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 5300224~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:44.277 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.501880+0000", 2024-09-21T15:00:44.277 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.765239319999999, 2024-09-21T15:00:44.277 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052644204999999999, 2024-09-21T15:00:44.277 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:44.277 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:44.277 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:44.277 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:44.277 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:44.277 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 113889 2024-09-21T15:00:44.277 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.278 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:44.278 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.278 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:44.278 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501880+0000", 2024-09-21T15:00:44.278 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:44.278 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.278 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.278 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:44.278 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501880+0000", 2024-09-21T15:00:44.278 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:44.278 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.278 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.278 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:44.278 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501880+0000", 2024-09-21T15:00:44.279 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:00:44.279 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.279 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.279 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:44.279 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501882+0000", 2024-09-21T15:00:44.279 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.364e-06 2024-09-21T15:00:44.279 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.279 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.279 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:44.279 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501885+0000", 2024-09-21T15:00:44.279 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4199999999999999e-06 2024-09-21T15:00:44.279 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.279 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.279 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:44.279 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501888+0000", 2024-09-21T15:00:44.279 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8669999999999998e-06 2024-09-21T15:00:44.280 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.280 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.280 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:44.280 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.554369+0000", 2024-09-21T15:00:44.280 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052481456000000003 2024-09-21T15:00:44.280 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.280 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.280 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:44.280 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.554449+0000", 2024-09-21T15:00:44.280 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9412000000000005e-05 2024-09-21T15:00:44.280 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.280 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.280 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:44.280 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.554525+0000", 2024-09-21T15:00:44.280 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5908999999999994e-05 2024-09-21T15:00:44.280 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:44.281 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:44.281 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:44.281 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.281 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.281 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:113890 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 5390336~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:44.281 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.501890+0000", 2024-09-21T15:00:44.281 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.765229605, 2024-09-21T15:00:44.281 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052785212999999997, 2024-09-21T15:00:44.281 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:44.281 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:44.281 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:44.281 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:44.281 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:44.281 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 113890 2024-09-21T15:00:44.281 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.282 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:44.282 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.282 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:44.282 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501890+0000", 2024-09-21T15:00:44.282 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:44.282 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.282 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.282 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:44.282 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501890+0000", 2024-09-21T15:00:44.282 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:44.282 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.282 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.282 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:44.282 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501890+0000", 2024-09-21T15:00:44.282 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:00:44.283 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.283 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.283 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:44.283 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501891+0000", 2024-09-21T15:00:44.283 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3599999999999999e-06 2024-09-21T15:00:44.283 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.283 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.283 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:44.283 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501894+0000", 2024-09-21T15:00:44.283 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.717e-06 2024-09-21T15:00:44.283 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.283 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.283 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:44.283 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501897+0000", 2024-09-21T15:00:44.283 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5790000000000002e-06 2024-09-21T15:00:44.283 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.284 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.284 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:44.284 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.554569+0000", 2024-09-21T15:00:44.284 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052672434999999997 2024-09-21T15:00:44.284 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.284 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.284 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:44.284 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.554623+0000", 2024-09-21T15:00:44.284 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4509999999999998e-05 2024-09-21T15:00:44.284 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.284 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.284 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:44.284 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.554675+0000", 2024-09-21T15:00:44.284 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1833999999999999e-05 2024-09-21T15:00:44.284 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:44.285 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:44.285 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:44.285 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.285 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.285 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:113891 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 5480448~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:44.285 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.501899+0000", 2024-09-21T15:00:44.285 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.765221119, 2024-09-21T15:00:44.285 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052991637000000001, 2024-09-21T15:00:44.285 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:44.285 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:44.285 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:44.285 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:44.285 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:44.285 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 113891 2024-09-21T15:00:44.285 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.285 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:44.286 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.286 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:44.286 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501899+0000", 2024-09-21T15:00:44.286 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:44.286 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.286 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.286 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:44.286 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501899+0000", 2024-09-21T15:00:44.286 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:44.286 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.286 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.286 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:44.286 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501898+0000", 2024-09-21T15:00:44.286 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:00:44.286 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.287 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.287 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:44.287 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501900+0000", 2024-09-21T15:00:44.287 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.356e-06 2024-09-21T15:00:44.287 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.287 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.287 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:44.287 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501902+0000", 2024-09-21T15:00:44.287 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6620000000000001e-06 2024-09-21T15:00:44.287 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.287 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.287 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:44.287 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501905+0000", 2024-09-21T15:00:44.287 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.492e-06 2024-09-21T15:00:44.288 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.288 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.288 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:44.288 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.554707+0000", 2024-09-21T15:00:44.288 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052802618000000003 2024-09-21T15:00:44.288 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.288 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.288 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:44.288 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.554798+0000", 2024-09-21T15:00:44.288 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0976000000000004e-05 2024-09-21T15:00:44.288 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.288 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.288 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:44.288 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.554890+0000", 2024-09-21T15:00:44.288 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1755999999999994e-05 2024-09-21T15:00:44.289 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:44.289 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:44.289 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:44.289 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.289 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.289 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:113892 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 5570560~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:44.289 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.501907+0000", 2024-09-21T15:00:44.289 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.765212687, 2024-09-21T15:00:44.289 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.053089242000000002, 2024-09-21T15:00:44.289 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:44.289 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:44.289 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:44.289 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:44.289 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:44.289 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 113892 2024-09-21T15:00:44.290 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.290 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:44.290 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.290 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:44.290 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501907+0000", 2024-09-21T15:00:44.290 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:44.290 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.290 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.290 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:44.290 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501907+0000", 2024-09-21T15:00:44.290 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:44.290 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.290 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.290 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:44.291 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501907+0000", 2024-09-21T15:00:44.291 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:00:44.291 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.291 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.291 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:44.291 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501908+0000", 2024-09-21T15:00:44.291 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2860000000000001e-06 2024-09-21T15:00:44.291 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.291 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.291 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:44.291 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501913+0000", 2024-09-21T15:00:44.291 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1320000000000002e-06 2024-09-21T15:00:44.291 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.291 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.291 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:44.292 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501916+0000", 2024-09-21T15:00:44.292 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5330000000000001e-06 2024-09-21T15:00:44.292 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.292 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.292 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:44.292 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.554904+0000", 2024-09-21T15:00:44.292 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052988631000000001 2024-09-21T15:00:44.292 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.292 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.292 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:44.292 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.554945+0000", 2024-09-21T15:00:44.292 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.0695999999999998e-05 2024-09-21T15:00:44.292 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.292 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.292 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:44.293 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.554996+0000", 2024-09-21T15:00:44.293 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1193999999999998e-05 2024-09-21T15:00:44.293 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:44.293 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:44.293 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:44.293 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.293 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.293 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:113893 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 5660672~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:44.293 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.501918+0000", 2024-09-21T15:00:44.293 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.765201709999999, 2024-09-21T15:00:44.293 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.053245595, 2024-09-21T15:00:44.293 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:44.293 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:44.293 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:44.294 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:44.294 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:44.294 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 113893 2024-09-21T15:00:44.294 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.294 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:44.294 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.294 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:44.294 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501918+0000", 2024-09-21T15:00:44.294 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:44.294 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.294 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.294 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:44.294 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501918+0000", 2024-09-21T15:00:44.294 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:44.295 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.295 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.295 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:44.295 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501918+0000", 2024-09-21T15:00:44.295 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:44.295 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.295 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.295 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:44.295 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501920+0000", 2024-09-21T15:00:44.295 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9800000000000001e-06 2024-09-21T15:00:44.295 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.295 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.295 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:44.295 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501928+0000", 2024-09-21T15:00:44.295 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7999999999999999e-06 2024-09-21T15:00:44.295 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.296 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.296 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:44.296 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501930+0000", 2024-09-21T15:00:44.296 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.785e-06 2024-09-21T15:00:44.296 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.296 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.296 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:44.296 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555019+0000", 2024-09-21T15:00:44.296 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.053089154 2024-09-21T15:00:44.296 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.296 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.296 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:44.296 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555065+0000", 2024-09-21T15:00:44.296 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5485000000000002e-05 2024-09-21T15:00:44.297 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.297 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.297 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:44.297 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555164+0000", 2024-09-21T15:00:44.297 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8629999999999996e-05 2024-09-21T15:00:44.297 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:44.297 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:44.297 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:44.297 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.297 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.297 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:113894 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 5750784~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:44.297 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.502384+0000", 2024-09-21T15:00:44.297 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.764735773, 2024-09-21T15:00:44.297 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052884821999999998, 2024-09-21T15:00:44.297 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:44.298 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:44.298 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:44.298 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:44.298 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:44.298 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 113894 2024-09-21T15:00:44.298 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.298 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:44.298 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.298 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:44.298 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502384+0000", 2024-09-21T15:00:44.298 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:44.298 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.298 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.298 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:44.299 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502384+0000", 2024-09-21T15:00:44.299 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:44.299 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.299 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.299 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:44.299 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501933+0000", 2024-09-21T15:00:44.299 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9995484 2024-09-21T15:00:44.299 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.299 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.299 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:44.299 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502390+0000", 2024-09-21T15:00:44.299 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00045714099999999999 2024-09-21T15:00:44.299 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.299 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.299 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:44.299 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502402+0000", 2024-09-21T15:00:44.300 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.236e-05 2024-09-21T15:00:44.300 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.300 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.300 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:44.300 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502411+0000", 2024-09-21T15:00:44.300 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8689999999999993e-06 2024-09-21T15:00:44.300 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.300 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.300 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:44.300 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555178+0000", 2024-09-21T15:00:44.300 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052767050000000003 2024-09-21T15:00:44.300 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.300 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.300 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:44.300 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555215+0000", 2024-09-21T15:00:44.301 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.667e-05 2024-09-21T15:00:44.301 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.301 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.301 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:44.301 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555269+0000", 2024-09-21T15:00:44.301 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4109e-05 2024-09-21T15:00:44.301 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:44.301 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:44.301 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:44.301 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.301 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.301 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:113895 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 5840896~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:44.301 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.502651+0000", 2024-09-21T15:00:44.301 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.764469160000001, 2024-09-21T15:00:44.301 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052770692000000001, 2024-09-21T15:00:44.302 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:44.302 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:44.302 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:44.302 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:44.302 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:44.302 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 113895 2024-09-21T15:00:44.302 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.302 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:44.302 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.302 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:44.302 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502651+0000", 2024-09-21T15:00:44.302 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:44.302 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.302 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.303 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:44.303 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502651+0000", 2024-09-21T15:00:44.303 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:44.303 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.303 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.303 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:44.303 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502421+0000", 2024-09-21T15:00:44.303 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9997702 2024-09-21T15:00:44.303 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.303 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.303 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:44.303 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502658+0000", 2024-09-21T15:00:44.303 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000237264 2024-09-21T15:00:44.303 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.303 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.304 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:44.304 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502673+0000", 2024-09-21T15:00:44.304 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4474999999999999e-05 2024-09-21T15:00:44.304 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.304 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.304 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:44.304 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502681+0000", 2024-09-21T15:00:44.304 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6119999999999999e-06 2024-09-21T15:00:44.304 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.304 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.304 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:44.304 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555285+0000", 2024-09-21T15:00:44.304 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052603860000000002 2024-09-21T15:00:44.304 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.305 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.305 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:44.305 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555348+0000", 2024-09-21T15:00:44.305 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3411999999999995e-05 2024-09-21T15:00:44.305 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.305 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.305 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:44.305 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555421+0000", 2024-09-21T15:00:44.305 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2809e-05 2024-09-21T15:00:44.305 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:44.305 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:44.305 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:44.305 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.305 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.305 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:113896 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 5931008~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:44.306 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.502862+0000", 2024-09-21T15:00:44.306 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.764257612, 2024-09-21T15:00:44.306 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052660208, 2024-09-21T15:00:44.306 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:44.306 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:44.306 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:44.306 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:44.306 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:44.306 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 113896 2024-09-21T15:00:44.306 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.306 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:44.306 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.306 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:44.306 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502862+0000", 2024-09-21T15:00:44.307 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:44.307 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.307 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.307 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:44.307 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502862+0000", 2024-09-21T15:00:44.307 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:44.307 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.307 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.307 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:44.307 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502689+0000", 2024-09-21T15:00:44.307 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9998264 2024-09-21T15:00:44.307 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.307 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.307 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:44.307 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502873+0000", 2024-09-21T15:00:44.308 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000184146 2024-09-21T15:00:44.308 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.308 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.308 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:44.308 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502884+0000", 2024-09-21T15:00:44.308 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1293e-05 2024-09-21T15:00:44.308 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.308 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.308 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:44.308 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502891+0000", 2024-09-21T15:00:44.308 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4259999999999997e-06 2024-09-21T15:00:44.308 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.308 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.308 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:44.308 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555435+0000", 2024-09-21T15:00:44.309 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052543053999999999 2024-09-21T15:00:44.309 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.309 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.309 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:44.309 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555471+0000", 2024-09-21T15:00:44.309 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6454e-05 2024-09-21T15:00:44.309 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.309 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.309 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:44.309 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555522+0000", 2024-09-21T15:00:44.309 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1314999999999999e-05 2024-09-21T15:00:44.309 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:44.309 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:44.309 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:44.309 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.310 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.310 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:113897 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 6021120~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:44.310 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.503004+0000", 2024-09-21T15:00:44.310 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.764115504999999, 2024-09-21T15:00:44.310 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052669499000000002, 2024-09-21T15:00:44.310 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:44.310 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:44.310 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:44.310 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:44.310 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:44.310 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 113897 2024-09-21T15:00:44.310 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.310 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:44.310 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.310 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:44.311 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503004+0000", 2024-09-21T15:00:44.311 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:44.311 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.311 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.311 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:44.311 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503004+0000", 2024-09-21T15:00:44.311 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:44.311 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.311 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.311 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:44.311 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502898+0000", 2024-09-21T15:00:44.311 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9998937 2024-09-21T15:00:44.311 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.311 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.311 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:44.312 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503010+0000", 2024-09-21T15:00:44.312 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000112396 2024-09-21T15:00:44.312 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.312 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.312 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:44.312 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503023+0000", 2024-09-21T15:00:44.312 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2206e-05 2024-09-21T15:00:44.312 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.312 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.312 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:44.312 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503031+0000", 2024-09-21T15:00:44.312 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3780000000000001e-06 2024-09-21T15:00:44.312 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.312 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.312 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:44.313 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555549+0000", 2024-09-21T15:00:44.313 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052517909000000002 2024-09-21T15:00:44.313 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.313 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.313 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:44.313 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555594+0000", 2024-09-21T15:00:44.313 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5385e-05 2024-09-21T15:00:44.313 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.313 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.313 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:44.313 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555674+0000", 2024-09-21T15:00:44.313 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9447e-05 2024-09-21T15:00:44.313 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:44.313 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:44.313 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:44.314 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.314 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.314 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:113898 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 6111232~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:44.314 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.503039+0000", 2024-09-21T15:00:44.314 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.764080720000001, 2024-09-21T15:00:44.314 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052754290000000002, 2024-09-21T15:00:44.314 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:44.314 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:44.314 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:44.314 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:44.314 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:44.314 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 113898 2024-09-21T15:00:44.314 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.314 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:44.315 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.315 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:44.315 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503039+0000", 2024-09-21T15:00:44.315 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:44.315 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.315 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.315 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:44.315 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503039+0000", 2024-09-21T15:00:44.315 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:44.315 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.315 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.315 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:44.315 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503038+0000", 2024-09-21T15:00:44.315 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:00:44.315 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.316 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.316 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:44.316 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503042+0000", 2024-09-21T15:00:44.316 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.0640000000000004e-06 2024-09-21T15:00:44.316 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.316 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.316 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:44.316 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503049+0000", 2024-09-21T15:00:44.316 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5579999999999997e-06 2024-09-21T15:00:44.316 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.316 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.316 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:44.316 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503054+0000", 2024-09-21T15:00:44.316 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6550000000000001e-06 2024-09-21T15:00:44.316 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.317 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.317 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:44.317 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555693+0000", 2024-09-21T15:00:44.317 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052638216000000002 2024-09-21T15:00:44.317 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.317 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.317 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:44.317 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555736+0000", 2024-09-21T15:00:44.317 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3619999999999999e-05 2024-09-21T15:00:44.317 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.317 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.317 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:44.317 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555793+0000", 2024-09-21T15:00:44.317 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6956999999999997e-05 2024-09-21T15:00:44.317 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:44.318 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:44.318 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:44.318 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.318 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.318 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:113899 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 6201344~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:44.318 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.503255+0000", 2024-09-21T15:00:44.318 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.763864447, 2024-09-21T15:00:44.318 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052648161999999998, 2024-09-21T15:00:44.318 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:44.318 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:44.318 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:44.318 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:44.318 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:44.318 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 113899 2024-09-21T15:00:44.318 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.319 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:44.319 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.319 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:44.319 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503255+0000", 2024-09-21T15:00:44.319 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:44.319 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.319 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.319 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:44.319 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503255+0000", 2024-09-21T15:00:44.319 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:44.319 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.319 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.319 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:44.319 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503058+0000", 2024-09-21T15:00:44.319 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9998021 2024-09-21T15:00:44.320 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.320 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.320 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:44.320 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503262+0000", 2024-09-21T15:00:44.320 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00020421299999999999 2024-09-21T15:00:44.320 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.320 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.320 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:44.320 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503274+0000", 2024-09-21T15:00:44.320 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2052000000000001e-05 2024-09-21T15:00:44.320 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.320 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.320 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:44.320 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503282+0000", 2024-09-21T15:00:44.320 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.977e-06 2024-09-21T15:00:44.321 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.321 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.321 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:44.321 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555808+0000", 2024-09-21T15:00:44.321 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052526150000000001 2024-09-21T15:00:44.321 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.321 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.321 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:44.321 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555851+0000", 2024-09-21T15:00:44.321 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2846999999999998e-05 2024-09-21T15:00:44.321 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.321 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.321 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:44.321 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555903+0000", 2024-09-21T15:00:44.321 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2607e-05 2024-09-21T15:00:44.322 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:44.322 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:44.322 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:44.322 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.322 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.322 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:113900 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 6307840~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:44.322 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.503445+0000", 2024-09-21T15:00:44.322 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.763674512, 2024-09-21T15:00:44.322 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052589546000000001, 2024-09-21T15:00:44.322 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:44.322 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:44.322 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:44.322 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:44.322 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:44.322 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 113900 2024-09-21T15:00:44.323 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.323 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:44.323 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.323 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:44.323 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503445+0000", 2024-09-21T15:00:44.323 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:44.323 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.323 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.323 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:44.323 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503445+0000", 2024-09-21T15:00:44.323 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:44.323 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.323 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.323 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:44.323 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503290+0000", 2024-09-21T15:00:44.324 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9998446 2024-09-21T15:00:44.324 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.324 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.324 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:44.324 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503451+0000", 2024-09-21T15:00:44.324 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00016111000000000001 2024-09-21T15:00:44.324 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.324 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.324 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:44.324 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503461+0000", 2024-09-21T15:00:44.324 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8740000000000007e-06 2024-09-21T15:00:44.324 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.324 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.324 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:44.325 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503468+0000", 2024-09-21T15:00:44.325 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.187e-06 2024-09-21T15:00:44.325 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.325 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.325 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:44.325 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555930+0000", 2024-09-21T15:00:44.325 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052462404999999997 2024-09-21T15:00:44.325 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.325 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.325 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:44.325 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555971+0000", 2024-09-21T15:00:44.325 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.0419e-05 2024-09-21T15:00:44.325 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.325 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.325 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:44.325 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.556035+0000", 2024-09-21T15:00:44.326 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3948999999999994e-05 2024-09-21T15:00:44.326 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:44.326 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:44.326 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:44.326 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.326 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.326 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:114050 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 4767744~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:44.326 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.514988+0000", 2024-09-21T15:00:44.326 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.752132001, 2024-09-21T15:00:44.326 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052614432000000003, 2024-09-21T15:00:44.326 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:44.326 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:44.326 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:44.326 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:44.327 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:44.327 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 114050 2024-09-21T15:00:44.327 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.327 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:44.327 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.327 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:44.327 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.514988+0000", 2024-09-21T15:00:44.327 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:44.327 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.327 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.327 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:44.327 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.514988+0000", 2024-09-21T15:00:44.327 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:44.327 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.327 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.328 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:44.328 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.514987+0000", 2024-09-21T15:00:44.328 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:44.328 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.328 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.328 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:44.328 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.514990+0000", 2024-09-21T15:00:44.328 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4739999999999999e-06 2024-09-21T15:00:44.328 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.328 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.328 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:44.328 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.514995+0000", 2024-09-21T15:00:44.328 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1939999999999997e-06 2024-09-21T15:00:44.328 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.328 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.329 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:44.329 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.514999+0000", 2024-09-21T15:00:44.329 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.245e-06 2024-09-21T15:00:44.329 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.329 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.329 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:44.329 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.567498+0000", 2024-09-21T15:00:44.329 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052498516000000002 2024-09-21T15:00:44.329 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.329 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.329 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:44.329 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.567541+0000", 2024-09-21T15:00:44.329 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3516000000000001e-05 2024-09-21T15:00:44.329 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.329 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.330 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:44.330 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.567602+0000", 2024-09-21T15:00:44.330 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0800000000000001e-05 2024-09-21T15:00:44.330 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:44.330 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:44.330 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:44.330 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.330 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.330 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:114051 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 4857856~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:44.330 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.515003+0000", 2024-09-21T15:00:44.330 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.752116286, 2024-09-21T15:00:44.330 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052709245000000002, 2024-09-21T15:00:44.330 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:44.330 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:44.330 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:44.331 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:44.331 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:44.331 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 114051 2024-09-21T15:00:44.331 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.331 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:44.331 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.331 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:44.331 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515003+0000", 2024-09-21T15:00:44.331 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:44.331 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.331 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.331 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:44.331 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515003+0000", 2024-09-21T15:00:44.331 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:44.331 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.332 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.332 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:44.332 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515003+0000", 2024-09-21T15:00:44.332 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:44.332 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.332 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.332 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:44.332 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515005+0000", 2024-09-21T15:00:44.332 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3499999999999999e-06 2024-09-21T15:00:44.332 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.332 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.332 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:44.332 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515011+0000", 2024-09-21T15:00:44.332 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2850000000000002e-06 2024-09-21T15:00:44.333 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.333 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.333 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:44.333 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515017+0000", 2024-09-21T15:00:44.333 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0059999999999998e-06 2024-09-21T15:00:44.333 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.333 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.333 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:44.333 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.567618+0000", 2024-09-21T15:00:44.333 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052600806999999999 2024-09-21T15:00:44.333 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.333 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.333 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:44.333 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.567655+0000", 2024-09-21T15:00:44.334 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7866e-05 2024-09-21T15:00:44.334 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.334 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.334 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:44.334 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.567713+0000", 2024-09-21T15:00:44.334 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7281999999999998e-05 2024-09-21T15:00:44.334 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:44.334 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:44.334 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:44.334 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.334 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.334 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:114052 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 4947968~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:44.334 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.515046+0000", 2024-09-21T15:00:44.334 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.752073627, 2024-09-21T15:00:44.335 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052795975000000002, 2024-09-21T15:00:44.335 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:44.335 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:44.335 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:44.335 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:44.335 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:44.335 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 114052 2024-09-21T15:00:44.335 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.335 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:44.335 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.335 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:44.335 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515046+0000", 2024-09-21T15:00:44.335 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:44.335 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.335 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.335 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:44.336 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515046+0000", 2024-09-21T15:00:44.336 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:44.336 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.336 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.336 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:44.336 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515046+0000", 2024-09-21T15:00:44.336 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:44.336 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.336 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.336 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:44.336 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515050+0000", 2024-09-21T15:00:44.336 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1359999999999999e-06 2024-09-21T15:00:44.336 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.336 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.336 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:44.336 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515060+0000", 2024-09-21T15:00:44.336 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0339e-05 2024-09-21T15:00:44.337 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.337 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.337 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:44.337 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515068+0000", 2024-09-21T15:00:44.337 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1149999999999994e-06 2024-09-21T15:00:44.337 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.337 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.337 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:44.337 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.567733+0000", 2024-09-21T15:00:44.337 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052664872000000001 2024-09-21T15:00:44.337 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.337 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.337 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:44.337 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.567778+0000", 2024-09-21T15:00:44.337 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4966999999999998e-05 2024-09-21T15:00:44.337 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.338 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.338 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:44.338 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.567842+0000", 2024-09-21T15:00:44.338 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4114000000000003e-05 2024-09-21T15:00:44.338 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:44.338 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:44.338 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:44.338 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.338 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.338 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:114053 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 5038080~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:44.338 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.515076+0000", 2024-09-21T15:00:44.338 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.752043767, 2024-09-21T15:00:44.338 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052886112999999998, 2024-09-21T15:00:44.338 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:44.338 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:44.338 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:44.339 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:44.339 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:44.339 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 114053 2024-09-21T15:00:44.339 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.339 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:44.339 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.339 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:44.339 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515076+0000", 2024-09-21T15:00:44.339 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:44.339 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.339 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.339 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:44.339 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515076+0000", 2024-09-21T15:00:44.339 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:44.339 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.339 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.340 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:44.340 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515075+0000", 2024-09-21T15:00:44.340 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:44.340 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.340 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.340 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:44.340 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515083+0000", 2024-09-21T15:00:44.340 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0520000000000002e-06 2024-09-21T15:00:44.340 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.340 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.340 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:44.340 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515092+0000", 2024-09-21T15:00:44.340 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1440000000000003e-06 2024-09-21T15:00:44.340 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.340 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.340 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:44.341 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515098+0000", 2024-09-21T15:00:44.341 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1090000000000003e-06 2024-09-21T15:00:44.341 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.341 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.341 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:44.341 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.567871+0000", 2024-09-21T15:00:44.341 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052772966999999997 2024-09-21T15:00:44.341 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.341 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.341 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:44.341 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.567909+0000", 2024-09-21T15:00:44.341 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.8380000000000002e-05 2024-09-21T15:00:44.341 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.341 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.342 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:44.342 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.567962+0000", 2024-09-21T15:00:44.342 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3068000000000001e-05 2024-09-21T15:00:44.342 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:44.342 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:44.342 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:44.342 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.342 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.342 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:114055 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 5218304~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:44.342 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.515526+0000", 2024-09-21T15:00:44.342 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.751593327, 2024-09-21T15:00:44.342 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052684278000000001, 2024-09-21T15:00:44.342 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:44.342 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:44.342 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:44.343 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:44.343 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:44.343 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 114055 2024-09-21T15:00:44.343 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.343 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:44.343 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.343 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:44.343 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515526+0000", 2024-09-21T15:00:44.343 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:44.343 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.343 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.343 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:44.343 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515526+0000", 2024-09-21T15:00:44.343 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:44.343 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.344 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.344 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:44.344 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515526+0000", 2024-09-21T15:00:44.344 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:44.344 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.344 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.344 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:44.344 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515529+0000", 2024-09-21T15:00:44.344 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.596e-06 2024-09-21T15:00:44.344 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.344 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.344 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:44.344 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515536+0000", 2024-09-21T15:00:44.344 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7399999999999998e-06 2024-09-21T15:00:44.345 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.345 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.345 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:44.345 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515542+0000", 2024-09-21T15:00:44.345 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5740000000000003e-06 2024-09-21T15:00:44.345 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.345 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.345 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:44.345 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.568116+0000", 2024-09-21T15:00:44.345 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052574152999999998 2024-09-21T15:00:44.345 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.345 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.345 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:44.345 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.568154+0000", 2024-09-21T15:00:44.345 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7926000000000002e-05 2024-09-21T15:00:44.346 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.346 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.346 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:44.346 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.568211+0000", 2024-09-21T15:00:44.346 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6858999999999999e-05 2024-09-21T15:00:44.346 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:44.346 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:44.346 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:44.346 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.346 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.346 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:114056 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 5308416~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:44.346 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.515547+0000", 2024-09-21T15:00:44.346 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.751572302, 2024-09-21T15:00:44.347 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052792795000000003, 2024-09-21T15:00:44.347 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:44.347 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:44.347 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:44.347 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:44.347 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:44.347 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 114056 2024-09-21T15:00:44.347 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.347 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:44.347 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.347 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:44.347 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515547+0000", 2024-09-21T15:00:44.347 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:44.347 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.347 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.348 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:44.348 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515547+0000", 2024-09-21T15:00:44.348 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:44.348 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.348 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.348 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:44.348 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515547+0000", 2024-09-21T15:00:44.348 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:44.348 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.348 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.348 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:44.348 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515550+0000", 2024-09-21T15:00:44.348 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2770000000000001e-06 2024-09-21T15:00:44.348 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.349 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.349 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:44.349 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515554+0000", 2024-09-21T15:00:44.349 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.087e-06 2024-09-21T15:00:44.349 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.349 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.349 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:44.349 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515558+0000", 2024-09-21T15:00:44.349 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1709999999999998e-06 2024-09-21T15:00:44.349 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.349 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.349 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:44.349 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.568234+0000", 2024-09-21T15:00:44.349 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052676675999999999 2024-09-21T15:00:44.349 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.350 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.350 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:44.350 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.568277+0000", 2024-09-21T15:00:44.350 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2400000000000001e-05 2024-09-21T15:00:44.350 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.350 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.350 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:44.350 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.568340+0000", 2024-09-21T15:00:44.350 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3581e-05 2024-09-21T15:00:44.350 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:44.350 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:44.350 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:44.350 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:44.350 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:44.351 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:44.352 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T15:00:44.352 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:00:44.349+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:00:44.423 INFO:teuthology.orchestra.run.smithi032.stderr:nodeep-scrub is set 2024-09-21T15:00:44.453 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:00:44.451+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:00:44.492 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:44.492 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:00:44.492 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T15:00:44.492 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:44.506 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T15:00:44.553 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:00:44.549+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:00:44.609 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:44.610 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:00:44.610 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T15:00:44.610 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T15:00:44.610 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:44.620 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T15:00:44.654 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:00:44.651+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:00:44.729 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:44.729 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T15:00:44.729 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T15:00:44.729 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T15:00:44.729 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.729 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T15:00:44.729 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.205754+0000", 2024-09-21T15:00:44.729 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 161.52034251500001, 2024-09-21T15:00:44.729 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.41117094199999998, 2024-09-21T15:00:44.730 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:44.730 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:44.730 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:44.730 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T15:00:44.730 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T15:00:44.730 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1 2024-09-21T15:00:44.730 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.730 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:44.730 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.730 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:44.730 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205754+0000", 2024-09-21T15:00:44.730 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:44.730 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.731 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.731 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:44.731 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205754+0000", 2024-09-21T15:00:44.731 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:44.731 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.731 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.731 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:44.731 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205752+0000", 2024-09-21T15:00:44.731 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T15:00:44.731 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.731 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.731 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:44.731 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205765+0000", 2024-09-21T15:00:44.731 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3230999999999999e-05 2024-09-21T15:00:44.731 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.731 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.732 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:44.732 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205782+0000", 2024-09-21T15:00:44.732 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6634e-05 2024-09-21T15:00:44.732 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.732 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.732 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:44.732 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205793+0000", 2024-09-21T15:00:44.732 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1297e-05 2024-09-21T15:00:44.732 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.732 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.732 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:44.732 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205867+0000", 2024-09-21T15:00:44.732 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4269000000000003e-05 2024-09-21T15:00:44.732 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.732 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.733 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:00:44.733 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205877+0000", 2024-09-21T15:00:44.733 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2690000000000005e-06 2024-09-21T15:00:44.733 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.733 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.733 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:44.733 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.613700+0000", 2024-09-21T15:00:44.733 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.407823455 2024-09-21T15:00:44.733 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.733 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.733 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:44.733 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.614203+0000", 2024-09-21T15:00:44.733 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00050311500000000005 2024-09-21T15:00:44.733 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.734 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.734 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T15:00:44.734 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.614750+0000", 2024-09-21T15:00:44.734 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00054738699999999998 2024-09-21T15:00:44.734 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.734 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.734 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:44.734 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616367+0000", 2024-09-21T15:00:44.734 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001616634 2024-09-21T15:00:44.734 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.734 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.734 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:44.734 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616574+0000", 2024-09-21T15:00:44.735 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00020668499999999999 2024-09-21T15:00:44.735 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.735 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.735 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:44.735 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616802+0000", 2024-09-21T15:00:44.735 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00022817900000000001 2024-09-21T15:00:44.735 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.735 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.735 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:44.735 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616925+0000", 2024-09-21T15:00:44.735 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012309699999999999 2024-09-21T15:00:44.735 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:44.735 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:44.735 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:44.735 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.735 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.736 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6639 3.1a 3:592d80ed:::rbd_data.11361bed1f78.0000000000000005:d [read 0~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T15:00:44.736 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131650+0000", 2024-09-21T15:00:44.736 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 129.59444696700001, 2024-09-21T15:00:44.736 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016002832000000002, 2024-09-21T15:00:44.736 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:44.736 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:44.736 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:44.736 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:44.736 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:44.736 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6639 2024-09-21T15:00:44.736 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.736 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:44.736 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.736 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:44.736 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131650+0000", 2024-09-21T15:00:44.737 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:44.737 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.737 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.737 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:44.737 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131650+0000", 2024-09-21T15:00:44.737 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:44.737 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.737 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.737 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:44.737 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131649+0000", 2024-09-21T15:00:44.737 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:44.737 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.737 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.737 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:44.738 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131651+0000", 2024-09-21T15:00:44.738 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9199999999999998e-06 2024-09-21T15:00:44.738 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.738 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.738 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:44.738 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131656+0000", 2024-09-21T15:00:44.738 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9899999999999997e-06 2024-09-21T15:00:44.738 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.738 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.738 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:44.738 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131665+0000", 2024-09-21T15:00:44.738 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0999999999999993e-06 2024-09-21T15:00:44.738 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.738 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.738 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:44.739 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146309+0000", 2024-09-21T15:00:44.739 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014643893999999999 2024-09-21T15:00:44.739 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.739 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.739 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:44.739 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146392+0000", 2024-09-21T15:00:44.739 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2531e-05 2024-09-21T15:00:44.739 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.739 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.739 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:44.739 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147653+0000", 2024-09-21T15:00:44.739 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0012606710000000001 2024-09-21T15:00:44.739 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:44.739 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:44.739 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:44.739 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.740 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.740 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6640 3.1a 3:592d80ed:::rbd_data.11361bed1f78.0000000000000005:d [read 65536~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T15:00:44.740 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131670+0000", 2024-09-21T15:00:44.740 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 129.59442666800001, 2024-09-21T15:00:44.740 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016188859, 2024-09-21T15:00:44.740 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:44.740 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:44.740 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:44.740 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:44.740 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:44.740 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6640 2024-09-21T15:00:44.740 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.740 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:44.740 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.740 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:44.740 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131670+0000", 2024-09-21T15:00:44.741 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:44.741 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.741 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.741 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:44.741 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131670+0000", 2024-09-21T15:00:44.741 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:44.741 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.741 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.741 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:44.741 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131670+0000", 2024-09-21T15:00:44.741 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:00:44.741 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.741 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.741 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:44.741 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131672+0000", 2024-09-21T15:00:44.742 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.818e-06 2024-09-21T15:00:44.742 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.742 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.742 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:44.742 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131675+0000", 2024-09-21T15:00:44.742 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5930000000000002e-06 2024-09-21T15:00:44.742 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.742 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.742 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:44.742 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131678+0000", 2024-09-21T15:00:44.742 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6790000000000001e-06 2024-09-21T15:00:44.742 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.742 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.742 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:44.742 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147681+0000", 2024-09-21T15:00:44.742 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016002755 2024-09-21T15:00:44.743 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.743 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.743 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:44.743 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147762+0000", 2024-09-21T15:00:44.743 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1052999999999998e-05 2024-09-21T15:00:44.743 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.743 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.743 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:44.743 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147859+0000", 2024-09-21T15:00:44.743 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.7199000000000002e-05 2024-09-21T15:00:44.743 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:44.743 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:44.743 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:44.743 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.744 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.744 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6641 3.1a 3:592d80ed:::rbd_data.11361bed1f78.0000000000000005:d [read 147456~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T15:00:44.744 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131680+0000", 2024-09-21T15:00:44.744 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 129.59441638000001, 2024-09-21T15:00:44.744 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016431296000000001, 2024-09-21T15:00:44.744 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:44.744 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:44.744 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:44.744 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:44.744 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:44.744 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6641 2024-09-21T15:00:44.744 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.744 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:44.744 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.744 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:44.745 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131680+0000", 2024-09-21T15:00:44.745 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:44.745 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.745 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.745 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:44.745 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131680+0000", 2024-09-21T15:00:44.745 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:44.745 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.745 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.745 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:44.745 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131680+0000", 2024-09-21T15:00:44.745 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:00:44.745 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.745 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.745 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:44.745 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131681+0000", 2024-09-21T15:00:44.746 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2610000000000001e-06 2024-09-21T15:00:44.746 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.746 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.746 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:44.746 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131685+0000", 2024-09-21T15:00:44.746 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.8299999999999998e-06 2024-09-21T15:00:44.746 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.746 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.746 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:44.746 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131688+0000", 2024-09-21T15:00:44.746 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6189999999999998e-06 2024-09-21T15:00:44.746 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.746 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.746 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:44.746 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147884+0000", 2024-09-21T15:00:44.746 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016196465 2024-09-21T15:00:44.747 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.747 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.747 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:44.747 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147949+0000", 2024-09-21T15:00:44.747 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4867000000000001e-05 2024-09-21T15:00:44.747 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.747 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.747 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:44.747 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.148112+0000", 2024-09-21T15:00:44.747 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00016247700000000001 2024-09-21T15:00:44.747 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:44.747 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:44.747 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:44.747 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.747 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.747 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6642 3.1a 3:592d80ed:::rbd_data.11361bed1f78.0000000000000005:d [read 212992~32768 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T15:00:44.748 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131690+0000", 2024-09-21T15:00:44.748 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 129.594406502, 2024-09-21T15:00:44.748 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016751335999999999, 2024-09-21T15:00:44.748 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:44.748 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:44.748 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:44.748 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:44.748 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:44.748 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6642 2024-09-21T15:00:44.748 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.748 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:44.748 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.748 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:44.748 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131690+0000", 2024-09-21T15:00:44.748 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:44.748 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.749 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.749 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:44.749 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131690+0000", 2024-09-21T15:00:44.749 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:44.749 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.749 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.749 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:44.749 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131690+0000", 2024-09-21T15:00:44.749 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:00:44.749 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.749 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.749 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:44.749 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131691+0000", 2024-09-21T15:00:44.749 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3120000000000001e-06 2024-09-21T15:00:44.749 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.749 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.750 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:44.750 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131694+0000", 2024-09-21T15:00:44.750 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0070000000000002e-06 2024-09-21T15:00:44.750 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.750 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.750 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:44.750 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131700+0000", 2024-09-21T15:00:44.750 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0800000000000002e-06 2024-09-21T15:00:44.750 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.750 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.750 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:44.750 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.148135+0000", 2024-09-21T15:00:44.750 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016434250000000001 2024-09-21T15:00:44.750 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.750 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.751 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:44.751 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.148204+0000", 2024-09-21T15:00:44.751 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9466000000000002e-05 2024-09-21T15:00:44.751 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.751 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.751 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:44.751 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.148441+0000", 2024-09-21T15:00:44.751 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00023743700000000001 2024-09-21T15:00:44.751 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:44.751 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:44.751 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:44.751 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.751 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.751 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:61214 3.7 3:e28aec10:::rbd_header.1136862f872f:head [watch reconnect cookie 139864671932080 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-21T15:00:44.751 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:18.147172+0000", 2024-09-21T15:00:44.751 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 86.578924872000002, 2024-09-21T15:00:44.752 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.019264956, 2024-09-21T15:00:44.752 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:44.752 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:44.752 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:44.752 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:44.752 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:44.752 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 61214 2024-09-21T15:00:44.752 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.752 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:44.752 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.752 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:44.752 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147172+0000", 2024-09-21T15:00:44.752 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:44.752 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.752 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.752 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:44.753 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147172+0000", 2024-09-21T15:00:44.753 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:44.753 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.753 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.753 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:44.753 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147168+0000", 2024-09-21T15:00:44.753 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999962 2024-09-21T15:00:44.753 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.753 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.753 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:44.753 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147188+0000", 2024-09-21T15:00:44.753 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9717000000000001e-05 2024-09-21T15:00:44.753 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.753 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.753 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:44.753 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147239+0000", 2024-09-21T15:00:44.754 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1493999999999999e-05 2024-09-21T15:00:44.754 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.754 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.754 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:44.754 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147265+0000", 2024-09-21T15:00:44.754 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5746999999999999e-05 2024-09-21T15:00:44.754 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.754 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.754 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:44.754 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147372+0000", 2024-09-21T15:00:44.754 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010681200000000001 2024-09-21T15:00:44.754 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.754 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.754 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op must wait for map", 2024-09-21T15:00:44.754 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147380+0000", 2024-09-21T15:00:44.755 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7919999999999993e-06 2024-09-21T15:00:44.755 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.755 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.755 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:44.755 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.154025+0000", 2024-09-21T15:00:44.755 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0066454909999999999 2024-09-21T15:00:44.755 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.755 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.755 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:00:44.755 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.154041+0000", 2024-09-21T15:00:44.755 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5894000000000001e-05 2024-09-21T15:00:44.755 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.755 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.755 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:44.755 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.164345+0000", 2024-09-21T15:00:44.756 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0103040459999999 2024-09-21T15:00:44.756 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.756 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.756 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:44.756 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.164590+0000", 2024-09-21T15:00:44.756 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00024492699999999998 2024-09-21T15:00:44.756 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.756 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.756 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:44.756 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.166437+0000", 2024-09-21T15:00:44.756 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0018466330000000001 2024-09-21T15:00:44.756 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:44.756 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:44.756 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:44.756 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.756 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.756 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:61215 3.7 3:e28aec10:::rbd_header.1136862f872f:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-21T15:00:44.757 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:18.152008+0000", 2024-09-21T15:00:44.757 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 86.574088419999995, 2024-09-21T15:00:44.757 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.015644958, 2024-09-21T15:00:44.757 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:44.757 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:44.757 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:44.757 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:44.757 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:44.757 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 61215 2024-09-21T15:00:44.757 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.757 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:44.757 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.757 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:44.757 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152008+0000", 2024-09-21T15:00:44.758 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:44.758 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.758 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.758 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:44.758 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152008+0000", 2024-09-21T15:00:44.758 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:44.758 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.758 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.758 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:44.758 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152005+0000", 2024-09-21T15:00:44.758 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-21T15:00:44.758 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.758 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.758 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:44.758 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152025+0000", 2024-09-21T15:00:44.758 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9306000000000001e-05 2024-09-21T15:00:44.759 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.759 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.759 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:44.759 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152054+0000", 2024-09-21T15:00:44.759 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9189000000000001e-05 2024-09-21T15:00:44.759 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.759 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.759 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:44.759 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152078+0000", 2024-09-21T15:00:44.759 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4317000000000001e-05 2024-09-21T15:00:44.759 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.759 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.759 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:44.759 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152243+0000", 2024-09-21T15:00:44.759 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000164802 2024-09-21T15:00:44.759 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.760 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.760 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting_for_map not empty", 2024-09-21T15:00:44.760 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152251+0000", 2024-09-21T15:00:44.760 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0600000000000008e-06 2024-09-21T15:00:44.760 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.760 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.760 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:44.760 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.154741+0000", 2024-09-21T15:00:44.760 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0024900069999999998 2024-09-21T15:00:44.760 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.760 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.760 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:00:44.760 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.154753+0000", 2024-09-21T15:00:44.760 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1588e-05 2024-09-21T15:00:44.760 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.760 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.761 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:44.761 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.165380+0000", 2024-09-21T15:00:44.761 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0106272249999999 2024-09-21T15:00:44.761 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.761 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.761 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:44.761 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.165517+0000", 2024-09-21T15:00:44.761 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013737999999999999 2024-09-21T15:00:44.761 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.761 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.761 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 6", 2024-09-21T15:00:44.761 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.165938+0000", 2024-09-21T15:00:44.761 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00042082399999999998 2024-09-21T15:00:44.761 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.761 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.761 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:44.762 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.166795+0000", 2024-09-21T15:00:44.762 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00085674899999999997 2024-09-21T15:00:44.762 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.762 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.762 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:44.762 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.167598+0000", 2024-09-21T15:00:44.762 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00080335599999999997 2024-09-21T15:00:44.762 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.762 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.762 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:44.762 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.167632+0000", 2024-09-21T15:00:44.762 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4122000000000003e-05 2024-09-21T15:00:44.762 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.762 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.762 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:44.762 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.167653+0000", 2024-09-21T15:00:44.763 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0862000000000001e-05 2024-09-21T15:00:44.763 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:44.763 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:44.763 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:44.763 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.763 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.763 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:73698 3.10 3:0a4d95f6:::rbd_data.1136f4939602.000000000000000b:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-21T15:00:44.763 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:41.969870+0000", 2024-09-21T15:00:44.763 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 62.756226282, 2024-09-21T15:00:44.763 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.017502461, 2024-09-21T15:00:44.763 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:44.763 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:44.763 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:44.763 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:44.763 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:44.763 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 73698 2024-09-21T15:00:44.764 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.764 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:44.764 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.764 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:44.764 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.969870+0000", 2024-09-21T15:00:44.764 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:44.764 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.764 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.764 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:44.764 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.969870+0000", 2024-09-21T15:00:44.764 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:44.764 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.764 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.764 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:44.764 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.969868+0000", 2024-09-21T15:00:44.764 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T15:00:44.765 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.765 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.765 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:44.765 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.969885+0000", 2024-09-21T15:00:44.765 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7416999999999999e-05 2024-09-21T15:00:44.765 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.765 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.765 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:44.765 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.969918+0000", 2024-09-21T15:00:44.765 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3133000000000003e-05 2024-09-21T15:00:44.765 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.765 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.765 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:44.765 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.969934+0000", 2024-09-21T15:00:44.765 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6027000000000001e-05 2024-09-21T15:00:44.765 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.766 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.766 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:44.766 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.970021+0000", 2024-09-21T15:00:44.766 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6907e-05 2024-09-21T15:00:44.766 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.766 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.766 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:44.766 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.970148+0000", 2024-09-21T15:00:44.766 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012701499999999999 2024-09-21T15:00:44.766 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.766 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.766 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T15:00:44.766 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.972971+0000", 2024-09-21T15:00:44.766 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.002822873 2024-09-21T15:00:44.766 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.767 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.767 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:44.767 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.987124+0000", 2024-09-21T15:00:44.767 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0141529 2024-09-21T15:00:44.767 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.767 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.767 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:44.767 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.987146+0000", 2024-09-21T15:00:44.767 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1783000000000001e-05 2024-09-21T15:00:44.767 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.767 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.767 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:44.767 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.987258+0000", 2024-09-21T15:00:44.767 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000112555 2024-09-21T15:00:44.767 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.768 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.768 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:44.768 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.987373+0000", 2024-09-21T15:00:44.768 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000114434 2024-09-21T15:00:44.768 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:44.768 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:44.768 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:44.768 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.768 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.768 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:97638 3.1a 3:5e573525:::rbd_data.1136b9c49b1c.0000000000000026:49 [read 1277952~49152 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:44.768 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.276202+0000", 2024-09-21T15:00:44.768 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 39.449894176999997, 2024-09-21T15:00:44.768 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.015360357999999999, 2024-09-21T15:00:44.768 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:44.768 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:44.768 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:44.769 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:44.769 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:44.769 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 97638 2024-09-21T15:00:44.769 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.769 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:44.769 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.769 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:44.769 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276202+0000", 2024-09-21T15:00:44.769 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:44.769 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.769 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.769 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:44.769 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276202+0000", 2024-09-21T15:00:44.769 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:44.769 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.769 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.770 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:44.770 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276202+0000", 2024-09-21T15:00:44.770 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:44.770 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.770 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.770 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:44.770 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276204+0000", 2024-09-21T15:00:44.770 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5359999999999999e-06 2024-09-21T15:00:44.770 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.770 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.770 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:44.770 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276210+0000", 2024-09-21T15:00:44.770 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3040000000000004e-06 2024-09-21T15:00:44.770 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.770 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.770 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:44.771 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276214+0000", 2024-09-21T15:00:44.771 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6779999999999999e-06 2024-09-21T15:00:44.771 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.771 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.771 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:44.771 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.290563+0000", 2024-09-21T15:00:44.771 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014348346 2024-09-21T15:00:44.771 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.771 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.771 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:44.771 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.290623+0000", 2024-09-21T15:00:44.771 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0001999999999999e-05 2024-09-21T15:00:44.771 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.771 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.771 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:44.771 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.291563+0000", 2024-09-21T15:00:44.772 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00094003100000000003 2024-09-21T15:00:44.772 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:44.772 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:44.772 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:44.772 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.772 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.772 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:97678 3.d e122/24 3:b2f34c6d:::rbd_data.1136bfc79a70.0000000000000006:head v 122'234, mlcod=120'232)", 2024-09-21T15:00:44.772 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.321655+0000", 2024-09-21T15:00:44.772 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 39.404441503000001, 2024-09-21T15:00:44.772 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024509388999999999, 2024-09-21T15:00:44.772 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:44.772 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:44.772 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:44.772 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.772 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:44.772 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.321655+0000", 2024-09-21T15:00:44.772 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:44.773 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.773 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.773 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:44.773 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.321655+0000", 2024-09-21T15:00:44.773 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:44.773 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.773 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.773 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:44.773 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.321655+0000", 2024-09-21T15:00:44.773 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:44.773 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.773 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.773 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:44.773 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.329453+0000", 2024-09-21T15:00:44.773 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0077980560000000003 2024-09-21T15:00:44.774 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.774 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.774 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:44.774 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.329482+0000", 2024-09-21T15:00:44.774 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9153999999999999e-05 2024-09-21T15:00:44.774 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.774 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.774 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:44.774 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.329493+0000", 2024-09-21T15:00:44.774 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1103e-05 2024-09-21T15:00:44.774 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.774 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.774 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:44.774 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.329555+0000", 2024-09-21T15:00:44.774 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2345e-05 2024-09-21T15:00:44.775 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.775 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.775 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:44.775 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.329589+0000", 2024-09-21T15:00:44.775 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3908999999999997e-05 2024-09-21T15:00:44.775 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.775 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.775 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:44.775 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.346083+0000", 2024-09-21T15:00:44.775 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016493383 2024-09-21T15:00:44.775 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.775 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.775 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:44.775 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.346165+0000", 2024-09-21T15:00:44.775 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1796000000000001e-05 2024-09-21T15:00:44.775 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:44.776 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:44.776 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:44.776 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.776 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.776 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:97679 3.f e122/24 3:f2a362b4:::rbd_data.1136bfc79a70.0000000000000007:head v 122'331, mlcod=120'329)", 2024-09-21T15:00:44.776 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.337145+0000", 2024-09-21T15:00:44.776 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 39.388951493999997, 2024-09-21T15:00:44.776 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021912411, 2024-09-21T15:00:44.776 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:44.776 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:44.776 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:44.776 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.776 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:44.776 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.337145+0000", 2024-09-21T15:00:44.776 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:44.776 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.777 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.777 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:44.777 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.337145+0000", 2024-09-21T15:00:44.777 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:44.777 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.777 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.777 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:44.777 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.337145+0000", 2024-09-21T15:00:44.777 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:44.777 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.777 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.777 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:44.777 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.343899+0000", 2024-09-21T15:00:44.777 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0067541420000000003 2024-09-21T15:00:44.777 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.777 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.778 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:44.778 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.343921+0000", 2024-09-21T15:00:44.778 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2255e-05 2024-09-21T15:00:44.778 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.778 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.778 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:44.778 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.343933+0000", 2024-09-21T15:00:44.778 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1528e-05 2024-09-21T15:00:44.778 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.778 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.778 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:44.778 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.344001+0000", 2024-09-21T15:00:44.778 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8269999999999995e-05 2024-09-21T15:00:44.778 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.779 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.779 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:44.779 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.344029+0000", 2024-09-21T15:00:44.779 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8254000000000001e-05 2024-09-21T15:00:44.779 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.779 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.779 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:44.779 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.358901+0000", 2024-09-21T15:00:44.779 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0148719 2024-09-21T15:00:44.779 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.779 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.779 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:44.779 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.359058+0000", 2024-09-21T15:00:44.779 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000156446 2024-09-21T15:00:44.779 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:44.780 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:44.780 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:44.780 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.780 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.780 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:97676 3.1f e122/24 3:ffaa836a:::rbd_data.1136bfc79a70.0000000000000004:head v 122'312, mlcod=120'310)", 2024-09-21T15:00:44.780 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.348683+0000", 2024-09-21T15:00:44.780 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 39.377413328999999, 2024-09-21T15:00:44.780 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.033768802000000001, 2024-09-21T15:00:44.780 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:44.780 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:44.780 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:44.780 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.780 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:44.780 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.348683+0000", 2024-09-21T15:00:44.780 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:44.781 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.781 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.781 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:44.781 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.348683+0000", 2024-09-21T15:00:44.781 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:44.781 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.781 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.781 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:44.781 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.348683+0000", 2024-09-21T15:00:44.781 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:00:44.781 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.781 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.781 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:44.781 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.366937+0000", 2024-09-21T15:00:44.782 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018254415 2024-09-21T15:00:44.782 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.782 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.782 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:44.782 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.366959+0000", 2024-09-21T15:00:44.782 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1634e-05 2024-09-21T15:00:44.782 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.782 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.782 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:44.782 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.366971+0000", 2024-09-21T15:00:44.782 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2305e-05 2024-09-21T15:00:44.782 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.782 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.782 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:44.783 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.367034+0000", 2024-09-21T15:00:44.783 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3211000000000002e-05 2024-09-21T15:00:44.783 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.783 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.783 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:44.783 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.367056+0000", 2024-09-21T15:00:44.783 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2098000000000001e-05 2024-09-21T15:00:44.783 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.783 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.783 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:44.783 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.382400+0000", 2024-09-21T15:00:44.783 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.015343867000000001 2024-09-21T15:00:44.783 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.783 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.784 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:44.784 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.382452+0000", 2024-09-21T15:00:44.784 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1992e-05 2024-09-21T15:00:44.784 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:44.784 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:44.784 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:44.784 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.784 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.784 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:97734 3.19 e122/94 3:9fcf754f:::rbd_data.1136bfc79a70.0000000000000009:head v 122'314, mlcod=120'312)", 2024-09-21T15:00:44.784 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.378547+0000", 2024-09-21T15:00:44.784 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 39.347549350999998, 2024-09-21T15:00:44.784 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.063380675999999997, 2024-09-21T15:00:44.784 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:44.785 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:44.785 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:44.785 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.785 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:44.785 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.378547+0000", 2024-09-21T15:00:44.785 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:44.785 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.785 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.785 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:44.785 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.378547+0000", 2024-09-21T15:00:44.785 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:44.785 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.785 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.785 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:44.786 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.378547+0000", 2024-09-21T15:00:44.786 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:00:44.786 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.786 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.786 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:44.786 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.425118+0000", 2024-09-21T15:00:44.786 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046571166999999997 2024-09-21T15:00:44.786 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.786 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.786 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:44.786 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.425150+0000", 2024-09-21T15:00:44.786 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2150999999999998e-05 2024-09-21T15:00:44.786 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.786 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.787 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:44.787 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.425164+0000", 2024-09-21T15:00:44.787 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3302000000000001e-05 2024-09-21T15:00:44.787 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.787 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.787 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:44.787 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.425231+0000", 2024-09-21T15:00:44.787 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7106000000000004e-05 2024-09-21T15:00:44.787 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.787 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.787 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:44.787 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.425259+0000", 2024-09-21T15:00:44.787 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8087000000000001e-05 2024-09-21T15:00:44.787 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.788 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.788 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:44.788 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.441890+0000", 2024-09-21T15:00:44.788 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016631328000000001 2024-09-21T15:00:44.788 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.788 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.788 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:44.788 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.441928+0000", 2024-09-21T15:00:44.788 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7701999999999999e-05 2024-09-21T15:00:44.788 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:44.788 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:44.788 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:44.788 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.788 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.789 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:100187 3.10 3:08941a97:::rbd_data.1136ab4d6ff1.000000000000000e:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-21T15:00:44.789 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:10.869845+0000", 2024-09-21T15:00:44.789 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 33.856251454999999, 2024-09-21T15:00:44.789 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029345237, 2024-09-21T15:00:44.789 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:44.789 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:44.789 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:44.789 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:44.789 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:44.789 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 100187 2024-09-21T15:00:44.789 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.789 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:44.789 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.789 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:44.789 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869845+0000", 2024-09-21T15:00:44.790 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:44.790 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.790 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.790 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:44.790 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869845+0000", 2024-09-21T15:00:44.790 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:44.790 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.790 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.790 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:44.790 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869844+0000", 2024-09-21T15:00:44.790 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T15:00:44.790 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.790 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.790 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:44.791 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869853+0000", 2024-09-21T15:00:44.791 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5719999999999992e-06 2024-09-21T15:00:44.791 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.791 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.791 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:44.791 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869874+0000", 2024-09-21T15:00:44.791 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0483e-05 2024-09-21T15:00:44.791 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.791 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.791 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:44.791 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869883+0000", 2024-09-21T15:00:44.791 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6080000000000002e-06 2024-09-21T15:00:44.791 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.791 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.791 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:44.791 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869991+0000", 2024-09-21T15:00:44.792 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000108013 2024-09-21T15:00:44.792 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.792 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.792 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:44.792 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.870258+0000", 2024-09-21T15:00:44.792 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00026629099999999999 2024-09-21T15:00:44.792 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.792 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.792 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T15:00:44.792 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.878656+0000", 2024-09-21T15:00:44.792 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0083989000000000008 2024-09-21T15:00:44.792 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.792 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.792 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:44.793 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.899081+0000", 2024-09-21T15:00:44.793 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.020424437 2024-09-21T15:00:44.793 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.793 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.793 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:44.793 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.899125+0000", 2024-09-21T15:00:44.793 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4440999999999997e-05 2024-09-21T15:00:44.793 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.793 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.793 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:44.793 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.899160+0000", 2024-09-21T15:00:44.793 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4518999999999999e-05 2024-09-21T15:00:44.793 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.794 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.794 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:44.794 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.899190+0000", 2024-09-21T15:00:44.794 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0574999999999998e-05 2024-09-21T15:00:44.794 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:44.794 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:44.794 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:44.794 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.794 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.794 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:100263 3.10 3:0ece142e:::rbd_data.1136ab4d6ff1.0000000000000008:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-21T15:00:44.794 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:11.052257+0000", 2024-09-21T15:00:44.794 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 33.673839397000002, 2024-09-21T15:00:44.794 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042678025000000001, 2024-09-21T15:00:44.795 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:44.795 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:44.795 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:44.795 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:44.795 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:44.795 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 100263 2024-09-21T15:00:44.795 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.795 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:44.795 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.795 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:44.795 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052257+0000", 2024-09-21T15:00:44.795 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:44.795 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.796 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.796 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:44.796 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052257+0000", 2024-09-21T15:00:44.796 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:44.796 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.796 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.796 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:44.796 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052256+0000", 2024-09-21T15:00:44.796 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:00:44.796 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.796 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.796 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:44.796 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052263+0000", 2024-09-21T15:00:44.796 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4799999999999998e-06 2024-09-21T15:00:44.797 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.797 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.797 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:44.797 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052275+0000", 2024-09-21T15:00:44.797 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2386e-05 2024-09-21T15:00:44.797 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.797 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.797 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:44.797 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052281+0000", 2024-09-21T15:00:44.797 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6699999999999999e-06 2024-09-21T15:00:44.797 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.797 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.797 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:44.797 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052350+0000", 2024-09-21T15:00:44.797 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9099999999999999e-05 2024-09-21T15:00:44.797 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.798 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.798 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:44.798 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052456+0000", 2024-09-21T15:00:44.798 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000106669 2024-09-21T15:00:44.798 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.798 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.798 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T15:00:44.798 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.059694+0000", 2024-09-21T15:00:44.798 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.007237566 2024-09-21T15:00:44.798 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.798 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.798 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:44.798 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.094721+0000", 2024-09-21T15:00:44.799 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035026977000000001 2024-09-21T15:00:44.799 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.799 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.799 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:44.799 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.094746+0000", 2024-09-21T15:00:44.799 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5329000000000001e-05 2024-09-21T15:00:44.799 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.799 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.799 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:44.799 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.094850+0000", 2024-09-21T15:00:44.799 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000103452 2024-09-21T15:00:44.799 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.799 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.799 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:44.800 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.094935+0000", 2024-09-21T15:00:44.800 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5550000000000006e-05 2024-09-21T15:00:44.800 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:44.800 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:44.800 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:44.800 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.800 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.800 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:102266 3.1f e130/24 3:fec72c47:::rbd_data.1136d175325b.000000000000000c:head v 130'326, mlcod=128'324)", 2024-09-21T15:00:44.800 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:14.319401+0000", 2024-09-21T15:00:44.800 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 30.406695457000001, 2024-09-21T15:00:44.800 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021068982999999999, 2024-09-21T15:00:44.800 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:44.800 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:44.800 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:44.800 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.800 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:44.801 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.319401+0000", 2024-09-21T15:00:44.801 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:44.801 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.801 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.801 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:44.801 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.319401+0000", 2024-09-21T15:00:44.801 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:44.801 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.801 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.801 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:44.801 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.319400+0000", 2024-09-21T15:00:44.801 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:00:44.801 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.801 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.801 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:44.801 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.337267+0000", 2024-09-21T15:00:44.802 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.017867141 2024-09-21T15:00:44.802 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.802 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.802 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:44.802 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.337278+0000", 2024-09-21T15:00:44.802 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0224000000000001e-05 2024-09-21T15:00:44.802 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.802 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.802 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:44.802 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.337285+0000", 2024-09-21T15:00:44.802 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.25e-06 2024-09-21T15:00:44.802 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.802 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.802 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:44.803 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.337330+0000", 2024-09-21T15:00:44.803 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5210000000000003e-05 2024-09-21T15:00:44.803 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.803 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.803 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:44.803 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.337350+0000", 2024-09-21T15:00:44.803 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9556e-05 2024-09-21T15:00:44.803 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.803 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.803 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:44.803 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.340418+0000", 2024-09-21T15:00:44.803 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.003068552 2024-09-21T15:00:44.803 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.803 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.803 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:44.803 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.340470+0000", 2024-09-21T15:00:44.804 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1897000000000001e-05 2024-09-21T15:00:44.804 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:44.804 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:44.804 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:44.804 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.804 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.804 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:102887 3.1c 3:3b11ae99:::rbd_object_map.1136d175325b.0000000000000050:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-21T15:00:44.804 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:14.368644+0000", 2024-09-21T15:00:44.804 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 30.357452421000001, 2024-09-21T15:00:44.804 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.017021290000000001, 2024-09-21T15:00:44.804 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:44.804 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:44.804 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:44.804 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:44.804 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:44.805 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 102887 2024-09-21T15:00:44.805 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.805 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:44.805 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.805 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:44.805 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.368644+0000", 2024-09-21T15:00:44.805 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:44.805 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.805 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.805 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:44.805 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.368644+0000", 2024-09-21T15:00:44.805 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:44.805 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.805 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.805 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:44.805 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.368642+0000", 2024-09-21T15:00:44.806 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T15:00:44.806 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.806 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.806 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:44.806 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.368655+0000", 2024-09-21T15:00:44.806 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2808e-05 2024-09-21T15:00:44.806 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.806 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.806 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:44.806 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.368677+0000", 2024-09-21T15:00:44.806 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2753999999999999e-05 2024-09-21T15:00:44.806 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.806 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.806 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:44.807 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.368688+0000", 2024-09-21T15:00:44.807 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1228e-05 2024-09-21T15:00:44.807 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.807 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.807 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:44.807 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.368743+0000", 2024-09-21T15:00:44.807 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4608999999999998e-05 2024-09-21T15:00:44.807 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.807 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.807 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:44.807 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.368868+0000", 2024-09-21T15:00:44.807 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012473000000000001 2024-09-21T15:00:44.807 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.807 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.807 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 0", 2024-09-21T15:00:44.808 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.369385+0000", 2024-09-21T15:00:44.808 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00051684400000000003 2024-09-21T15:00:44.808 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.808 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.808 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:44.808 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.370755+0000", 2024-09-21T15:00:44.808 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0013701609999999999 2024-09-21T15:00:44.808 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.808 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.808 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:44.808 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.385531+0000", 2024-09-21T15:00:44.808 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014776648 2024-09-21T15:00:44.808 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.808 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.808 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:44.809 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.385575+0000", 2024-09-21T15:00:44.809 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3726000000000001e-05 2024-09-21T15:00:44.809 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.809 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.809 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:44.809 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.385666+0000", 2024-09-21T15:00:44.809 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0304999999999997e-05 2024-09-21T15:00:44.809 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:44.809 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:44.809 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:44.809 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.809 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.809 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:103706 3.1c 3:3b11ae99:::rbd_object_map.1136d175325b.0000000000000050:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-21T15:00:44.809 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:14.426370+0000", 2024-09-21T15:00:44.809 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 30.299727084000001, 2024-09-21T15:00:44.809 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.015478897, 2024-09-21T15:00:44.810 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:44.810 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:44.810 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:44.810 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:44.810 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:44.810 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 103706 2024-09-21T15:00:44.810 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.810 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:44.810 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.810 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:44.810 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.426370+0000", 2024-09-21T15:00:44.810 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:44.810 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.811 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.811 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:44.811 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.426370+0000", 2024-09-21T15:00:44.811 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:44.811 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.811 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.811 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:44.811 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.426369+0000", 2024-09-21T15:00:44.811 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:00:44.811 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.811 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.811 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:44.811 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.426376+0000", 2024-09-21T15:00:44.811 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0999999999999998e-06 2024-09-21T15:00:44.811 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.811 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.812 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:44.812 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.426442+0000", 2024-09-21T15:00:44.812 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6445999999999994e-05 2024-09-21T15:00:44.812 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.812 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.812 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:44.812 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.426451+0000", 2024-09-21T15:00:44.812 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4940000000000003e-06 2024-09-21T15:00:44.812 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.812 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.812 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:44.812 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.426493+0000", 2024-09-21T15:00:44.812 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2094000000000001e-05 2024-09-21T15:00:44.812 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.812 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.812 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:44.813 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.426560+0000", 2024-09-21T15:00:44.813 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6774999999999997e-05 2024-09-21T15:00:44.813 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.813 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.813 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 0", 2024-09-21T15:00:44.813 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.426737+0000", 2024-09-21T15:00:44.813 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017769899999999999 2024-09-21T15:00:44.813 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.813 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.813 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:44.813 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.428561+0000", 2024-09-21T15:00:44.813 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001823917 2024-09-21T15:00:44.813 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.813 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.813 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:44.814 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.441762+0000", 2024-09-21T15:00:44.814 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013200462 2024-09-21T15:00:44.814 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.814 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.814 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:44.814 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.441811+0000", 2024-09-21T15:00:44.814 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8912000000000002e-05 2024-09-21T15:00:44.814 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.814 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.814 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:44.814 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.441848+0000", 2024-09-21T15:00:44.814 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7935000000000002e-05 2024-09-21T15:00:44.814 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:44.815 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:44.815 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:44.815 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.815 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.815 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:105083 3.1a 3:5d63845a:::rbd_data.1136d175325b.0000000000000037:head [write 0~32768 [fadvise_sequential+fadvise_nocache],write 65536~163840 [fadvise_sequential+fadvise_nocache],write 262144~32768 [fadvise_sequential+fadvise_nocache],write 327680~163840 [fadvise_sequential+fadvise_nocache],write 524288~32768 [fadvise_sequential+fadvise_nocache],write 589824~163840 [fadvise_sequential+fadvise_nocache],write 786432~32768 [fadvise_sequential+fadvise_nocache],write 851968~32768 [fadvise_sequential+fadvise_nocache],write 917504~98304 [fadvise_sequential+fadvise_nocache],write 1048576~32768 [fadvise_sequential+fadvise_nocache],write 1114112~32768 [fadvise_sequential+fadvise_nocache],write 1179648~98304 [fadvise_sequential+fadvise_nocache],write 1310720~32768 [fadvise_sequential+fadvise_nocache],write 1376256~32768 [fadvise_sequential+fadvise_nocache],write 1441792~98304 [fadvise_sequential+fadvise_nocache],write 1572864~32768 [fadvise_sequential+fadvise_nocache],write 1638400~32768 [fadvise_sequential+fadvise_nocache],write 1703936~98304 [fadvise_sequential+fadvise_nocache],write 1835008~32768 [fadvise_sequential+fadvise_nocache],write 1900544~32768 [fadvise_sequential+fadvise_nocache],write 1966080~98304 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-21T15:00:44.815 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:14.534201+0000", 2024-09-21T15:00:44.815 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 30.191895743, 2024-09-21T15:00:44.815 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016474194000000001, 2024-09-21T15:00:44.815 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:44.815 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:44.815 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:44.815 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:44.815 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:44.816 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 105083 2024-09-21T15:00:44.816 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.816 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:44.816 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.816 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:44.816 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.534201+0000", 2024-09-21T15:00:44.816 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:44.816 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.816 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.816 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:44.816 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.534201+0000", 2024-09-21T15:00:44.816 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:44.816 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.816 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.817 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:44.817 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.534201+0000", 2024-09-21T15:00:44.817 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:44.817 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.817 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.817 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:44.817 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.534853+0000", 2024-09-21T15:00:44.817 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00065227099999999995 2024-09-21T15:00:44.817 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.817 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.817 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:44.817 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.534866+0000", 2024-09-21T15:00:44.817 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3553000000000001e-05 2024-09-21T15:00:44.818 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.818 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.818 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:44.818 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.534877+0000", 2024-09-21T15:00:44.818 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0186e-05 2024-09-21T15:00:44.818 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.818 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.818 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:44.818 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.534947+0000", 2024-09-21T15:00:44.818 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0417e-05 2024-09-21T15:00:44.818 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.818 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.818 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:44.818 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.535138+0000", 2024-09-21T15:00:44.819 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00019122200000000001 2024-09-21T15:00:44.819 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.819 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.819 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T15:00:44.819 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.535666+0000", 2024-09-21T15:00:44.819 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00052817499999999998 2024-09-21T15:00:44.819 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.819 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.819 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:44.819 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.543252+0000", 2024-09-21T15:00:44.819 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0075857889999999999 2024-09-21T15:00:44.819 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.819 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.819 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:44.820 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.550552+0000", 2024-09-21T15:00:44.820 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0072995869999999997 2024-09-21T15:00:44.820 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.820 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.820 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:44.820 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.550637+0000", 2024-09-21T15:00:44.820 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5750999999999999e-05 2024-09-21T15:00:44.820 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.820 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.820 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:44.820 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.550675+0000", 2024-09-21T15:00:44.820 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7602999999999999e-05 2024-09-21T15:00:44.820 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:44.820 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:44.820 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:44.821 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.821 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.821 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:113281 3.10 3:0e1b39db:::rbd_data.1136b442e94a.0000000000000009:head [write 0~360448 [fadvise_sequential+fadvise_nocache],write 376832~573440 [fadvise_sequential+fadvise_nocache],write 1138688~573440 [fadvise_sequential+fadvise_nocache],write 1900544~196608 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:44.821 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.487425+0000", 2024-09-21T15:00:44.821 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 15.238671801000001, 2024-09-21T15:00:44.821 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.015461361999999999, 2024-09-21T15:00:44.821 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:44.821 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:44.821 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:44.821 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:44.821 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:44.821 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 113281 2024-09-21T15:00:44.821 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.821 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:44.822 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.822 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:44.822 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.487425+0000", 2024-09-21T15:00:44.822 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:44.822 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.822 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.822 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:44.822 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.487425+0000", 2024-09-21T15:00:44.822 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:44.822 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.822 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.822 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:44.822 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.487423+0000", 2024-09-21T15:00:44.823 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-21T15:00:44.823 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.823 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.823 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:44.823 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.488151+0000", 2024-09-21T15:00:44.823 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00072770699999999996 2024-09-21T15:00:44.823 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.823 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.823 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:44.823 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.488175+0000", 2024-09-21T15:00:44.823 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3896e-05 2024-09-21T15:00:44.823 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.823 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.823 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:44.824 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.488184+0000", 2024-09-21T15:00:44.824 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9000000000000001e-06 2024-09-21T15:00:44.824 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.824 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.824 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:44.824 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.488918+0000", 2024-09-21T15:00:44.824 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00073320500000000001 2024-09-21T15:00:44.824 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.824 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.824 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:44.824 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.489175+0000", 2024-09-21T15:00:44.824 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00025761799999999999 2024-09-21T15:00:44.824 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.824 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.825 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T15:00:44.825 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.489479+0000", 2024-09-21T15:00:44.825 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00030419099999999999 2024-09-21T15:00:44.825 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.825 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.825 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:44.825 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.498472+0000", 2024-09-21T15:00:44.825 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0089922319999999993 2024-09-21T15:00:44.825 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.825 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.825 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:44.825 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502751+0000", 2024-09-21T15:00:44.825 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0042795309999999996 2024-09-21T15:00:44.825 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.825 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.825 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:44.826 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502843+0000", 2024-09-21T15:00:44.826 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1926e-05 2024-09-21T15:00:44.826 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:44.826 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:44.826 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:44.826 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502886+0000", 2024-09-21T15:00:44.826 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3087999999999998e-05 2024-09-21T15:00:44.826 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:44.826 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:44.826 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:44.826 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:44.826 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:44.826 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:44.828 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T15:00:44.828 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:00:44.823+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:00:44.900 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T15:00:44.928 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:00:44.923+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:00:44.985 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:44.985 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:00:44.985 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T15:00:44.985 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:44.995 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T15:00:45.029 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:00:45.025+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:00:45.045 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T15:00:45.068 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T15:00:45.089 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T15:00:45.111 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T15:00:45.129 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:00:45.125+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:00:45.133 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-21T15:00:45.155 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:45.155 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:45.163 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T15:00:45.167 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T15:00:45.185 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T15:00:45.208 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T15:00:45.210 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:00:45.210 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:00:45.210 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:00:45.210 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:00:45.210 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:00:45.210 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:00:45.211 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:00:45.211 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:00:45.230 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:00:45.227+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:00:45.283 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:45.283 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:45.293 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T15:00:45.316 INFO:teuthology.orchestra.run.smithi032.stdout:checking clone #30, image image_client.0-clone30 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent31 2024-09-21T15:00:45.331 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:00:45.327+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:00:45.406 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:45.406 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:45.418 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T15:00:45.461 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:00:45.459+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:00:45.532 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:00:45.529+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:00:45.572 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:45.573 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:45.583 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T15:00:45.634 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:00:45.629+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:00:45.690 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:45.691 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:45.701 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T15:00:45.747 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:00:45.745+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:00:45.763 INFO:teuthology.orchestra.run.smithi032.stdout:4521 read 0xbb6c18 thru 0xbc69c3 (0xfdac bytes) 2024-09-21T15:00:45.764 INFO:teuthology.orchestra.run.smithi032.stdout:4524 read 0x1ec18e6 thru 0x1ec222c (0x947 bytes) 2024-09-21T15:00:45.765 INFO:teuthology.orchestra.run.smithi032.stdout:4525 punch from 0x8fafe36 to 0x8fb3b66, (0x3d30 bytes) 2024-09-21T15:00:45.771 INFO:teuthology.orchestra.run.smithi032.stdout:4526 read 0xa38a3b3 thru 0xa39838b (0xdfd9 bytes) 2024-09-21T15:00:45.772 INFO:teuthology.orchestra.run.smithi032.stdout:4527 read 0x329f442 thru 0x32a3b05 (0x46c4 bytes) 2024-09-21T15:00:45.772 INFO:teuthology.orchestra.run.smithi032.stdout:4529 write 0x4411c45 thru 0x441af90 (0x934c bytes) 2024-09-21T15:00:45.780 INFO:teuthology.orchestra.run.smithi032.stdout:4530 punch from 0x7670a33 to 0x7680325, (0xf8f2 bytes) 2024-09-21T15:00:45.780 INFO:teuthology.orchestra.run.smithi032.stdout:4533 write 0xae619c3 thru 0xae71480 (0xfabe bytes) 2024-09-21T15:00:45.783 INFO:teuthology.orchestra.run.smithi032.stdout:4534 read 0x646107c thru 0x646fac2 (0xea47 bytes) 2024-09-21T15:00:45.783 INFO:teuthology.orchestra.run.smithi032.stdout:4535 write 0x4874390 thru 0x48749a8 (0x619 bytes) 2024-09-21T15:00:45.809 INFO:teuthology.orchestra.run.smithi032.stdout:4536 trunc from 0xbf08516 to 0x93a098 2024-09-21T15:00:45.823 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:45.823 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:45.835 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T15:00:45.835 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:00:45.831+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:00:45.838 INFO:teuthology.orchestra.run.smithi032.stdout:4537 read 0x27ba20 thru 0x287256 (0xb837 bytes) 2024-09-21T15:00:45.839 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:45.839 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:45.839 INFO:teuthology.orchestra.run.smithi032.stdout:4540 read 0x2562c3 thru 0x258bce (0x290c bytes) 2024-09-21T15:00:45.852 INFO:teuthology.orchestra.run.smithi032.stdout:4541 write 0x3bd0c97 thru 0x3bd51f2 (0x455c bytes) 2024-09-21T15:00:45.857 INFO:teuthology.orchestra.run.smithi032.stdout:4542 read 0x27d698 thru 0x28b3d7 (0xdd40 bytes) 2024-09-21T15:00:45.890 INFO:teuthology.orchestra.run.smithi032.stdout:4543 writesame 0xe269d1c thru 0xe276a04 data_size 0x1(0xcce9 bytes) 2024-09-21T15:00:45.890 INFO:teuthology.orchestra.run.smithi032.stdout:4545 read 0x856dec4 thru 0x856e3fc (0x539 bytes) 2024-09-21T15:00:45.891 INFO:teuthology.orchestra.run.smithi032.stdout:4546 writesame 0x2f6232b thru 0x2f6bb37 data_size 0x3(0x980d bytes) 2024-09-21T15:00:45.905 INFO:teuthology.orchestra.run.smithi032.stdout:4547 writesame 0x2cbf57e thru 0x2cca63d data_size 0x40(0xb0c0 bytes) 2024-09-21T15:00:45.912 INFO:teuthology.orchestra.run.smithi032.stdout:4548 punch from 0x79953d8 to 0x799f59a, (0xa1c2 bytes) 2024-09-21T15:00:45.912 INFO:teuthology.orchestra.run.smithi032.stdout:4549 write 0x662d3d3 thru 0x662e52b (0x1159 bytes) 2024-09-21T15:00:45.930 INFO:teuthology.orchestra.run.smithi032.stdout:4550 write 0xe88fcf1 thru 0xe892ce0 (0x2ff0 bytes) 2024-09-21T15:00:45.935 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:00:45.933+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:00:45.937 INFO:teuthology.orchestra.run.smithi032.stdout:4551 write 0x1dcb188 thru 0x1dce3ab (0x3224 bytes) 2024-09-21T15:00:45.940 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:45.940 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:45.944 INFO:teuthology.orchestra.run.smithi032.stdout:4552 read 0xd774dfd thru 0xd7797a0 (0x49a4 bytes) 2024-09-21T15:00:45.951 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T15:00:45.972 INFO:teuthology.orchestra.run.smithi032.stdout:4558 trunc from 0xe892ce1 to 0xcd15b3 2024-09-21T15:00:46.027 INFO:teuthology.orchestra.run.smithi032.stdout:4559 write 0xd67d877 thru 0xd6834b5 (0x5c3f bytes) 2024-09-21T15:00:46.036 INFO:teuthology.orchestra.run.smithi032.stdout:4560 write 0xabf7e1f thru 0xabfacda (0x2ebc bytes) 2024-09-21T15:00:46.036 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:00:46.033+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:00:46.042 INFO:teuthology.orchestra.run.smithi032.stdout:4561 punch from 0x936412b to 0x93710a9, (0xcf7e bytes) 2024-09-21T15:00:46.042 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:46.042 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:46.042 INFO:teuthology.orchestra.run.smithi032.stdout:4566 read 0xb3a4dd7 thru 0xb3a99c4 (0x4bee bytes) 2024-09-21T15:00:46.042 INFO:teuthology.orchestra.run.smithi032.stdout:4568 write 0xb60c3ea thru 0xb613f25 (0x7b3c bytes) 2024-09-21T15:00:46.048 INFO:teuthology.orchestra.run.smithi032.stdout:4569 read 0xb8366d6 thru 0xb838468 (0x1d93 bytes) 2024-09-21T15:00:46.048 INFO:teuthology.orchestra.run.smithi032.stdout:4571 write 0x6161fb6 thru 0x61644ff (0x254a bytes) 2024-09-21T15:00:46.055 INFO:teuthology.orchestra.run.smithi032.stdout:4572 read 0x650d768 thru 0x65144c4 (0x6d5d bytes) 2024-09-21T15:00:46.055 INFO:teuthology.orchestra.run.smithi032.stdout:4574 punch from 0x91abd0b to 0x91ba718, (0xea0d bytes) 2024-09-21T15:00:46.055 INFO:teuthology.orchestra.run.smithi032.stdout:4575 read 0x2e03d52 thru 0x2e09823 (0x5ad2 bytes) 2024-09-21T15:00:46.055 INFO:teuthology.orchestra.run.smithi032.stdout:4576 punch from 0x1906d47 to 0x190e91b, (0x7bd4 bytes) 2024-09-21T15:00:46.055 INFO:teuthology.orchestra.run.smithi032.stdout:4577 write 0x90dd006 thru 0x90e792e (0xa929 bytes) 2024-09-21T15:00:46.077 INFO:teuthology.orchestra.run.smithi032.stdout:4578 write 0xe3a671c thru 0xe3ad3bf (0x6ca4 bytes) 2024-09-21T15:00:46.085 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:46.085 INFO:teuthology.orchestra.run.smithi032.stdout:4582 writesame 0x79386c0 thru 0x794439c data_size 0x1(0xbcdd bytes) 2024-09-21T15:00:46.123 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:46.123 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:46.131 INFO:teuthology.orchestra.run.smithi032.stdout:4583 trunc from 0xe3ad3c0 to 0x8e394de 2024-09-21T15:00:46.133 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T15:00:46.137 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:00:46.133+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:00:46.154 INFO:teuthology.orchestra.run.smithi032.stdout:4584 trunc from 0x8e394de to 0x977e9a2 2024-09-21T15:00:46.164 INFO:teuthology.orchestra.run.smithi032.stdout:4585 punch from 0x72ee10a to 0x72f5466, (0x735c bytes) 2024-09-21T15:00:46.177 INFO:teuthology.orchestra.run.smithi032.stdout:4586 write 0xa2ed77a thru 0xa2f6839 (0x90c0 bytes) 2024-09-21T15:00:46.184 INFO:teuthology.orchestra.run.smithi032.stdout:4587 read 0x65a37a2 thru 0x65b28cd (0xf12c bytes) 2024-09-21T15:00:46.184 INFO:teuthology.orchestra.run.smithi032.stdout:4589 write 0x368c003 thru 0x368cad1 (0xacf bytes) 2024-09-21T15:00:46.191 INFO:teuthology.orchestra.run.smithi032.stdout:4590 punch from 0x28090e8 to 0x2815ae4, (0xc9fc bytes) 2024-09-21T15:00:46.191 INFO:teuthology.orchestra.run.smithi032.stdout:4591 read 0x72b83c4 thru 0x72ba702 (0x233f bytes) 2024-09-21T15:00:46.191 INFO:teuthology.orchestra.run.smithi032.stdout:4592 punch from 0x47094c5 to 0x471239e, (0x8ed9 bytes) 2024-09-21T15:00:46.191 INFO:teuthology.orchestra.run.smithi032.stdout:4594 write 0x44c18d7 thru 0x44cf2ba (0xd9e4 bytes) 2024-09-21T15:00:46.198 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:46.198 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:46.198 INFO:teuthology.orchestra.run.smithi032.stdout:4597 write 0x59f554f thru 0x5a03b87 (0xe639 bytes) 2024-09-21T15:00:46.209 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:46.209 INFO:teuthology.orchestra.run.smithi032.stdout:4599 writesame 0x1c86563 thru 0x1c91b86 data_size 0x2(0xb624 bytes) 2024-09-21T15:00:46.232 INFO:teuthology.orchestra.run.smithi032.stdout:4600 read 0x6315c64 thru 0x6318d9f (0x313c bytes) 2024-09-21T15:00:46.232 INFO:teuthology.orchestra.run.smithi032.stdout:4601 write 0x464afb0 thru 0x4650430 (0x5481 bytes) 2024-09-21T15:00:46.235 INFO:teuthology.orchestra.run.smithi032.stdout:4602 writesame 0x98d139f thru 0x98d2ef1 data_size 0x1(0x1b53 bytes) 2024-09-21T15:00:46.237 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:00:46.233+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:00:46.241 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:46.241 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:46.251 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T15:00:46.257 INFO:teuthology.orchestra.run.smithi032.stdout:4603 trunc from 0xa2f683a to 0x5e07438 2024-09-21T15:00:46.277 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:46.277 INFO:teuthology.orchestra.run.smithi032.stdout:4605 write 0x1f26005 thru 0x1f2ca7c (0x6a78 bytes) 2024-09-21T15:00:46.282 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:46.288 INFO:teuthology.orchestra.run.smithi032.stdout:4609 trunc from 0x5e07438 to 0x31fb3f3 2024-09-21T15:00:46.316 INFO:teuthology.orchestra.run.smithi032.stdout:4610 trunc from 0x31fb3f3 to 0x7dee635 2024-09-21T15:00:46.324 INFO:teuthology.orchestra.run.smithi032.stdout:4611 read 0x1c95c87 thru 0x1c9ba2f (0x5da9 bytes) 2024-09-21T15:00:46.326 INFO:teuthology.orchestra.run.smithi032.stdout:4612 punch from 0x3174926 to 0x317dae9, (0x91c3 bytes) 2024-09-21T15:00:46.326 INFO:teuthology.orchestra.run.smithi032.stdout:4614 read 0x5045b68 thru 0x50541f2 (0xe68b bytes) 2024-09-21T15:00:46.326 INFO:teuthology.orchestra.run.smithi032.stdout:4615 writesame 0x60467e6 thru 0x6056233 data_size 0x2e(0xfa4e bytes) 2024-09-21T15:00:46.333 INFO:teuthology.orchestra.run.smithi032.stdout:4616 write 0x6d4775 thru 0x6d5774 (0x1000 bytes) 2024-09-21T15:00:46.338 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:00:46.335+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:00:46.354 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:46.354 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:46.355 INFO:teuthology.orchestra.run.smithi032.stdout:4617 writesame 0xc7017b7 thru 0xc70466d data_size 0x1(0x2eb7 bytes) 2024-09-21T15:00:46.355 INFO:teuthology.orchestra.run.smithi032.stdout:4618 punch from 0x126a5f9 to 0x1279b1b, (0xf522 bytes) 2024-09-21T15:00:46.355 INFO:teuthology.orchestra.run.smithi032.stdout:4619 read 0x8a218dc thru 0x8a2ec42 (0xd367 bytes) 2024-09-21T15:00:46.355 INFO:teuthology.orchestra.run.smithi032.stdout:4621 write 0x543b319 thru 0x543ca10 (0x16f8 bytes) 2024-09-21T15:00:46.361 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:46.361 INFO:teuthology.orchestra.run.smithi032.stdout:4624 writesame 0x30517be thru 0x305de9a data_size 0x1(0xc6dd bytes) 2024-09-21T15:00:46.365 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T15:00:46.411 INFO:teuthology.orchestra.run.smithi032.stdout:4625 trunc from 0xc70466e to 0x8ab01ca 2024-09-21T15:00:46.425 INFO:teuthology.orchestra.run.smithi032.stdout:4627 read 0x85214dd thru 0x8531217 (0xfd3b bytes) 2024-09-21T15:00:46.426 INFO:teuthology.orchestra.run.smithi032.stdout:4629 writesame 0x599dd53 thru 0x59ac783 data_size 0x1(0xea31 bytes) 2024-09-21T15:00:46.432 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T15:00:46.461 INFO:teuthology.orchestra.run.smithi032.stdout:4632 write 0x638bb97 thru 0x63936b1 (0x7b1b bytes) 2024-09-21T15:00:46.461 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:00:46.459+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:00:46.464 INFO:teuthology.orchestra.run.smithi032.stdout:4633 writesame 0x3a213f0 thru 0x3a2e212 data_size 0x1(0xce23 bytes) 2024-09-21T15:00:46.468 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:46.468 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:46.478 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T15:00:46.499 INFO:teuthology.orchestra.run.smithi032.stdout:4634 read 0x1477a80 thru 0x148653d (0xeabe bytes) 2024-09-21T15:00:46.499 INFO:teuthology.orchestra.run.smithi032.stdout:4635 punch from 0x7a49af4 to 0x7a55f50, (0xc45c bytes) 2024-09-21T15:00:46.499 INFO:teuthology.orchestra.run.smithi032.stdout:4636 punch from 0x4452a2e to 0x445cd60, (0xa332 bytes) 2024-09-21T15:00:46.500 INFO:teuthology.orchestra.run.smithi032.stdout:4637 read 0x5e04219 thru 0x5e077f2 (0x35da bytes) 2024-09-21T15:00:46.500 INFO:teuthology.orchestra.run.smithi032.stdout:4638 write 0x220bff thru 0x2283dc (0x77de bytes) 2024-09-21T15:00:46.503 INFO:teuthology.orchestra.run.smithi032.stdout:4639 read 0x4487ab thru 0x44a34e (0x1ba4 bytes) 2024-09-21T15:00:46.521 INFO:teuthology.orchestra.run.smithi032.stdout:4640 write 0xde69e50 thru 0xde77c1a (0xddcb bytes) 2024-09-21T15:00:46.528 INFO:teuthology.orchestra.run.smithi032.stdout:4641 writesame 0x1ed3286 thru 0x1edd94f data_size 0x25(0xa6ca bytes) 2024-09-21T15:00:46.532 INFO:teuthology.orchestra.run.smithi032.stdout:4642 writesame 0xbfa4823 thru 0xbfb30fe data_size 0xe(0xe8dc bytes) 2024-09-21T15:00:46.538 INFO:teuthology.orchestra.run.smithi032.stdout:4643 write 0x5630e6e thru 0x56333c5 (0x2558 bytes) 2024-09-21T15:00:46.539 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:00:46.535+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:00:46.541 INFO:teuthology.orchestra.run.smithi032.stdout:4644 read 0x296ace6 thru 0x2970575 (0x5890 bytes) 2024-09-21T15:00:46.541 INFO:teuthology.orchestra.run.smithi032.stdout:4645 writesame 0xa3d575b thru 0xa3dafce data_size 0x11(0x5874 bytes) 2024-09-21T15:00:46.548 INFO:teuthology.orchestra.run.smithi032.stdout:4646 write 0x12fc5cc thru 0x13050b4 (0x8ae9 bytes) 2024-09-21T15:00:46.562 INFO:teuthology.orchestra.run.smithi032.stdout:4647 writesame 0xdfad588 thru 0xdfb6b41 data_size 0x5(0x95ba bytes) 2024-09-21T15:00:46.565 INFO:teuthology.orchestra.run.smithi032.stdout:4648 read 0x2070eb0 thru 0x20770d1 (0x6222 bytes) 2024-09-21T15:00:46.565 INFO:teuthology.orchestra.run.smithi032.stdout:4649 trunc from 0xdfb6b42 to 0xe025153 2024-09-21T15:00:46.572 INFO:teuthology.orchestra.run.smithi032.stdout:4650 write 0x27664e9 thru 0x276e7e0 (0x82f8 bytes) 2024-09-21T15:00:46.578 INFO:teuthology.orchestra.run.smithi032.stdout:4651 write 0x1f6ca3c thru 0x1f6fe6b (0x3430 bytes) 2024-09-21T15:00:46.583 INFO:teuthology.orchestra.run.smithi032.stdout:4652 trunc from 0xe025153 to 0xcf75740 2024-09-21T15:00:46.584 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:46.584 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:46.594 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T15:00:46.602 INFO:teuthology.orchestra.run.smithi032.stdout:4653 read 0xa4058c7 thru 0xa413acf (0xe209 bytes) 2024-09-21T15:00:46.602 INFO:teuthology.orchestra.run.smithi032.stdout:4655 read 0x19ee5be thru 0x19f2d4d (0x4790 bytes) 2024-09-21T15:00:46.603 INFO:teuthology.orchestra.run.smithi032.stdout:4656 punch from 0x2476f5 to 0x24b23f, (0x3b4a bytes) 2024-09-21T15:00:46.603 INFO:teuthology.orchestra.run.smithi032.stdout:4657 read 0x54da501 thru 0x54e8d4d (0xe84d bytes) 2024-09-21T15:00:46.603 INFO:teuthology.orchestra.run.smithi032.stdout:4658 write 0xc400a83 thru 0xc4075e0 (0x6b5e bytes) 2024-09-21T15:00:46.610 INFO:teuthology.orchestra.run.smithi032.stdout:4659 read 0xb90f5bf thru 0xb91e2d0 (0xed12 bytes) 2024-09-21T15:00:46.610 INFO:teuthology.orchestra.run.smithi032.stdout:4660 punch from 0x546da77 to 0x547607c, (0x8605 bytes) 2024-09-21T15:00:46.610 INFO:teuthology.orchestra.run.smithi032.stdout:4661 write 0x31acca3 thru 0x31b6a29 (0x9d87 bytes) 2024-09-21T15:00:46.616 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:46.616 INFO:teuthology.orchestra.run.smithi032.stdout:4663 punch from 0x815e13b to 0x816d0e6, (0xefab bytes) 2024-09-21T15:00:46.616 INFO:teuthology.orchestra.run.smithi032.stdout:4665 punch from 0xccfa8d5 to 0xccfecd4, (0x43ff bytes) 2024-09-21T15:00:46.616 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:46.616 INFO:teuthology.orchestra.run.smithi032.stdout:4668 write 0x51f127e thru 0x51fa080 (0x8e03 bytes) 2024-09-21T15:00:46.622 INFO:teuthology.orchestra.run.smithi032.stdout:4670 write 0x97005cc thru 0x97044c1 (0x3ef6 bytes) 2024-09-21T15:00:46.629 INFO:teuthology.orchestra.run.smithi032.stdout:4672 trunc from 0xcf75740 to 0xd597bc2 2024-09-21T15:00:46.640 INFO:teuthology.orchestra.run.smithi032.stdout:4673 write 0x7037079 thru 0x703b4a7 (0x442f bytes) 2024-09-21T15:00:46.640 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:00:46.635+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:00:46.646 INFO:teuthology.orchestra.run.smithi032.stdout:4674 write 0x3995b84 thru 0x39a2483 (0xc900 bytes) 2024-09-21T15:00:46.652 INFO:teuthology.orchestra.run.smithi032.stdout:4678 read 0xa3f4060 thru 0xa3f49a2 (0x943 bytes) 2024-09-21T15:00:46.652 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:46.652 INFO:teuthology.orchestra.run.smithi032.stdout:4680 writesame 0x498541f thru 0x498bf19 data_size 0x33(0x6afb bytes) 2024-09-21T15:00:46.659 INFO:teuthology.orchestra.run.smithi032.stdout:4682 writesame 0x3a96b48 thru 0x3aa67e8 data_size 0x1(0xfca1 bytes) 2024-09-21T15:00:46.692 INFO:teuthology.orchestra.run.smithi032.stdout:4683 read 0xc27062d thru 0xc278543 (0x7f17 bytes) 2024-09-21T15:00:46.692 INFO:teuthology.orchestra.run.smithi032.stdout:4684 writesame 0x3ea1825 thru 0x3ea9dd7 data_size 0x3(0x85b3 bytes) 2024-09-21T15:00:46.701 INFO:teuthology.orchestra.run.smithi032.stdout:4685 write 0x501be48 thru 0x50268fa (0xaab3 bytes) 2024-09-21T15:00:46.705 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:46.705 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:46.707 INFO:teuthology.orchestra.run.smithi032.stdout:4686 write 0xf19ac5 thru 0xf19b2a (0x66 bytes) 2024-09-21T15:00:46.715 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T15:00:46.719 INFO:teuthology.orchestra.run.smithi032.stdout:4687 trunc from 0xd597bc2 to 0x8708d99 2024-09-21T15:00:46.741 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:00:46.737+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:00:46.754 INFO:teuthology.orchestra.run.smithi032.stdout:4688 write 0xee3702d thru 0xee3c2e7 (0x52bb bytes) 2024-09-21T15:00:46.760 INFO:teuthology.orchestra.run.smithi032.stdout:4689 write 0xea25046 thru 0xea28158 (0x3113 bytes) 2024-09-21T15:00:46.767 INFO:teuthology.orchestra.run.smithi032.stdout:4690 read 0x69784a9 thru 0x6986965 (0xe4bd bytes) 2024-09-21T15:00:46.767 INFO:teuthology.orchestra.run.smithi032.stdout:4692 write 0xe4f5616 thru 0xe4f6529 (0xf14 bytes) 2024-09-21T15:00:46.773 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:46.773 INFO:teuthology.orchestra.run.smithi032.stdout:4694 write 0xb1b8216 thru 0xb1c5182 (0xcf6d bytes) 2024-09-21T15:00:46.780 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:46.780 INFO:teuthology.orchestra.run.smithi032.stdout:4696 writesame 0x88e9fb8 thru 0x88ea41c data_size 0x9(0x465 bytes) 2024-09-21T15:00:46.786 INFO:teuthology.orchestra.run.smithi032.stdout:4705 write 0x57012bb thru 0x5702649 (0x138f bytes) 2024-09-21T15:00:46.792 INFO:teuthology.orchestra.run.smithi032.stdout:4706 read 0x1d7571d thru 0x1d7ea3d (0x9321 bytes) 2024-09-21T15:00:46.793 INFO:teuthology.orchestra.run.smithi032.stdout:4707 read 0xebed516 thru 0xebef31b (0x1e06 bytes) 2024-09-21T15:00:46.796 INFO:teuthology.orchestra.run.smithi032.stdout:4708 trunc from 0xee3c2e8 to 0xdf9b456 2024-09-21T15:00:46.815 INFO:teuthology.orchestra.run.smithi032.stdout:4709 read 0xcfebfd3 thru 0xcff7f4d (0xbf7b bytes) 2024-09-21T15:00:46.815 INFO:teuthology.orchestra.run.smithi032.stdout:4710 writesame 0x8461d4b thru 0x8471140 data_size 0x15(0xf3f6 bytes) 2024-09-21T15:00:46.816 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:46.816 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:46.822 INFO:teuthology.orchestra.run.smithi032.stdout:4712 write 0xdf56219 thru 0xdf63d22 (0xdb0a bytes) 2024-09-21T15:00:46.826 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T15:00:46.831 INFO:teuthology.orchestra.run.smithi032.stdout:4713 write 0xc8d8361 thru 0xc8e64d1 (0xe171 bytes) 2024-09-21T15:00:46.842 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:00:46.837+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:00:46.849 INFO:teuthology.orchestra.run.smithi032.stdout:4717 trunc from 0xdf9b456 to 0x95d18c3 2024-09-21T15:00:46.871 INFO:teuthology.orchestra.run.smithi032.stdout:4719 trunc from 0x95d18c3 to 0x73981b2 2024-09-21T15:00:46.893 INFO:teuthology.orchestra.run.smithi032.stdout:4720 punch from 0x2f71855 to 0x2f80bd2, (0xf37d bytes) 2024-09-21T15:00:46.893 INFO:teuthology.orchestra.run.smithi032.stdout:4721 write 0x4e597d6 thru 0x4e617d6 (0x8001 bytes) 2024-09-21T15:00:46.901 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:46.901 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:46.901 INFO:teuthology.orchestra.run.smithi032.stdout:4725 punch from 0x72e35a0 to 0x72ec1de, (0x8c3e bytes) 2024-09-21T15:00:46.901 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:46.901 INFO:teuthology.orchestra.run.smithi032.stdout:4729 writesame 0x607c0d0 thru 0x6084f66 data_size 0x1(0x8e97 bytes) 2024-09-21T15:00:46.927 INFO:teuthology.orchestra.run.smithi032.stdout:4730 trunc from 0x73981b2 to 0xb8dd8c9 2024-09-21T15:00:46.932 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:46.932 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:46.935 INFO:teuthology.orchestra.run.smithi032.stdout:4731 read 0x71f543 thru 0x72171f (0x21dd bytes) 2024-09-21T15:00:46.937 INFO:teuthology.orchestra.run.smithi032.stdout:4732 write 0x32513d7 thru 0x325499e (0x35c8 bytes) 2024-09-21T15:00:46.940 INFO:teuthology.orchestra.run.smithi032.stdout:4734 write 0x4904fd5 thru 0x490e6a4 (0x96d0 bytes) 2024-09-21T15:00:46.942 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T15:00:46.944 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:46.949 INFO:teuthology.orchestra.run.smithi032.stdout:4736 trunc from 0xb8dd8c9 to 0xdfcb46f 2024-09-21T15:00:46.962 INFO:teuthology.orchestra.run.smithi032.stdout:4737 writesame 0x7635f40 thru 0x763e3ea data_size 0x1(0x84ab bytes) 2024-09-21T15:00:46.983 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:00:46.981+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:00:46.995 INFO:teuthology.orchestra.run.smithi032.stdout:4738 punch from 0x31e9c21 to 0x31f5fea, (0xc3c9 bytes) 2024-09-21T15:00:46.995 INFO:teuthology.orchestra.run.smithi032.stdout:4739 write 0x2125891 thru 0x212f2a4 (0x9a14 bytes) 2024-09-21T15:00:47.002 INFO:teuthology.orchestra.run.smithi032.stdout:4740 writesame 0x4e09a4d thru 0x4e0e37d data_size 0x1(0x4931 bytes) 2024-09-21T15:00:47.017 INFO:teuthology.orchestra.run.smithi032.stdout:4741 write 0xb4983ee thru 0xb4a2b8a (0xa79d bytes) 2024-09-21T15:00:47.024 INFO:teuthology.orchestra.run.smithi032.stdout:4743 write 0x38165ec thru 0x3825c38 (0xf64d bytes) 2024-09-21T15:00:47.037 INFO:teuthology.orchestra.run.smithi032.stdout:4744 trunc from 0xdfcb46f to 0x9b4edf4 2024-09-21T15:00:47.044 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:00:47.041+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:00:47.054 INFO:teuthology.orchestra.run.smithi032.stdout:4746 write 0x802a79d thru 0x8037e5d (0xd6c1 bytes) 2024-09-21T15:00:47.061 INFO:teuthology.orchestra.run.smithi032.stdout:4747 read 0x33097b3 thru 0x330af14 (0x1762 bytes) 2024-09-21T15:00:47.062 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:47.066 INFO:teuthology.orchestra.run.smithi032.stdout:4749 trunc from 0x9b4edf4 to 0xb6b4dcb 2024-09-21T15:00:47.096 INFO:teuthology.orchestra.run.smithi032.stdout:4750 trunc from 0xb6b4dcb to 0x32e8857 2024-09-21T15:00:47.096 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:47.096 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:47.108 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T15:00:47.139 INFO:teuthology.orchestra.run.smithi032.stdout:4751 writesame 0x8a4b136 thru 0x8a581f7 data_size 0x9(0xd0c2 bytes) 2024-09-21T15:00:47.139 INFO:teuthology.orchestra.run.smithi032.stdout:4752 write 0x6ec923c thru 0x6eca07f (0xe44 bytes) 2024-09-21T15:00:47.144 INFO:teuthology.orchestra.run.smithi032.stdout:4753 read 0x2fc928 thru 0x303069 (0x6742 bytes) 2024-09-21T15:00:47.145 INFO:teuthology.orchestra.run.smithi032.stdout:4754 write 0x72a31db thru 0x72b070f (0xd535 bytes) 2024-09-21T15:00:47.145 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:00:47.141+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:00:47.151 INFO:teuthology.orchestra.run.smithi032.stdout:4756 read 0x2398459 thru 0x239ca11 (0x45b9 bytes) 2024-09-21T15:00:47.151 INFO:teuthology.orchestra.run.smithi032.stdout:4757 read 0x51e3bf9 thru 0x51f0fdd (0xd3e5 bytes) 2024-09-21T15:00:47.151 INFO:teuthology.orchestra.run.smithi032.stdout:4758 write 0x5d1fad4 thru 0x5d2aa5a (0xaf87 bytes) 2024-09-21T15:00:47.158 INFO:teuthology.orchestra.run.smithi032.stdout:4760 write 0x6b0d58a thru 0x6b11d9e (0x4815 bytes) 2024-09-21T15:00:47.165 INFO:teuthology.orchestra.run.smithi032.stdout:4762 write 0x7e4e89b thru 0x7e557a6 (0x6f0c bytes) 2024-09-21T15:00:47.170 INFO:teuthology.orchestra.run.smithi032.stdout:4763 punch from 0x8302a77 to 0x831104e, (0xe5d7 bytes) 2024-09-21T15:00:47.193 INFO:teuthology.orchestra.run.smithi032.stdout:4764 write 0xe3383f6 thru 0xe3413ae (0x8fb9 bytes) 2024-09-21T15:00:47.201 INFO:teuthology.orchestra.run.smithi032.stdout:4768 punch from 0xc123ad4 to 0xc126755, (0x2c81 bytes) 2024-09-21T15:00:47.201 INFO:teuthology.orchestra.run.smithi032.stdout:4770 punch from 0xd4093ea to 0xd40de27, (0x4a3d bytes) 2024-09-21T15:00:47.201 INFO:teuthology.orchestra.run.smithi032.stdout:4773 write 0x7948dc6 thru 0x794b327 (0x2562 bytes) 2024-09-21T15:00:47.207 INFO:teuthology.orchestra.run.smithi032.stdout:4776 writesame 0xb6117ab thru 0xb611cc9 data_size 0x3(0x51f bytes) 2024-09-21T15:00:47.211 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:47.211 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:47.213 INFO:teuthology.orchestra.run.smithi032.stdout:4779 read 0x1363b02 thru 0x1368d42 (0x5241 bytes) 2024-09-21T15:00:47.213 INFO:teuthology.orchestra.run.smithi032.stdout:4780 writesame 0x17199f5 thru 0x1727002 data_size 0x3(0xd60e bytes) 2024-09-21T15:00:47.221 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T15:00:47.224 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T15:00:47.252 INFO:teuthology.orchestra.run.smithi032.stdout:4781 trunc from 0xe3413af to 0x3af8b04 2024-09-21T15:00:47.253 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:00:47.251+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:00:47.273 INFO:teuthology.orchestra.run.smithi032.stdout:4783 writesame 0x26a5dc9 thru 0x26ae87d data_size 0x1(0x8ab5 bytes) 2024-09-21T15:00:47.305 INFO:teuthology.orchestra.run.smithi032.stdout:4784 read 0x7a8593 thru 0x7ad565 (0x4fd3 bytes) 2024-09-21T15:00:47.306 INFO:teuthology.orchestra.run.smithi032.stdout:4787 read 0x17fa389 thru 0x17fba1c (0x1694 bytes) 2024-09-21T15:00:47.325 INFO:teuthology.orchestra.run.smithi032.stdout:4788 trunc from 0x3af8b04 to 0xdd5ca47 2024-09-21T15:00:47.334 INFO:teuthology.orchestra.run.smithi032.stdout:4789 writesame 0x453c6a6 thru 0x45412df data_size 0xb(0x4c3a bytes) 2024-09-21T15:00:47.343 INFO:teuthology.orchestra.run.smithi032.stdout:4790 read 0xc302bd4 thru 0xc3066d9 (0x3b06 bytes) 2024-09-21T15:00:47.343 INFO:teuthology.orchestra.run.smithi032.stdout:4791 punch from 0x419046c to 0x419c39b, (0xbf2f bytes) 2024-09-21T15:00:47.343 INFO:teuthology.orchestra.run.smithi032.stdout:4792 read 0x99922ac thru 0x9996ed0 (0x4c25 bytes) 2024-09-21T15:00:47.343 INFO:teuthology.orchestra.run.smithi032.stdout:4794 read 0x636f844 thru 0x637a320 (0xaadd bytes) 2024-09-21T15:00:47.346 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:00:47.341+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:00:47.352 INFO:teuthology.orchestra.run.smithi032.stdout:4795 trunc from 0xdd5ca47 to 0x831901f 2024-09-21T15:00:47.359 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:47.359 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:47.360 INFO:teuthology.orchestra.run.smithi032.stdout:4797 read 0x7f5f02e thru 0x7f6078b (0x175e bytes) 2024-09-21T15:00:47.361 INFO:teuthology.orchestra.run.smithi032.stdout:4798 read 0x4fbf6d2 thru 0x4fc83c9 (0x8cf8 bytes) 2024-09-21T15:00:47.361 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:47.361 INFO:teuthology.orchestra.run.smithi032.stdout:4800 punch from 0x7f4ef51 to 0x7f5e8cc, (0xf97b bytes) 2024-09-21T15:00:47.361 INFO:teuthology.orchestra.run.smithi032.stdout:4801 write 0x6704849 thru 0x6708d06 (0x44be bytes) 2024-09-21T15:00:47.371 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T15:00:47.372 INFO:teuthology.orchestra.run.smithi032.stdout:4802 trunc from 0x831901f to 0x567691e 2024-09-21T15:00:47.398 INFO:teuthology.orchestra.run.smithi032.stdout:4803 trunc from 0x567691e to 0xc7c7130 2024-09-21T15:00:47.398 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T15:00:47.407 INFO:teuthology.orchestra.run.smithi032.stdout:4804 read 0x15ead44 thru 0x15ef521 (0x47de bytes) 2024-09-21T15:00:47.408 INFO:teuthology.orchestra.run.smithi032.stdout:4805 read 0x28faecb thru 0x2904a02 (0x9b38 bytes) 2024-09-21T15:00:47.408 INFO:teuthology.orchestra.run.smithi032.stdout:4806 writesame 0x13daefa thru 0x13e9e77 data_size 0x5(0xef7e bytes) 2024-09-21T15:00:47.415 INFO:teuthology.orchestra.run.smithi032.stdout:4807 writesame 0x7cd5837 thru 0x7cd709b data_size 0x1(0x1865 bytes) 2024-09-21T15:00:47.423 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 5, 6, 7, 4] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T15:00:47.423 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T15:00:47.423 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-21T15:00:47.423 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-21T15:00:47.424 INFO:teuthology.orchestra.run.smithi032.stdout:4808 read 0xa206f08 thru 0xa20bc18 (0x4d11 bytes) 2024-09-21T15:00:47.425 INFO:teuthology.orchestra.run.smithi032.stdout:4809 write 0x2d2f297 thru 0x2d37655 (0x83bf bytes) 2024-09-21T15:00:47.428 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T15:00:47.432 INFO:teuthology.orchestra.run.smithi032.stdout:4810 write 0x7680a2a thru 0x768636e (0x5945 bytes) 2024-09-21T15:00:47.440 INFO:teuthology.orchestra.run.smithi032.stderr:noscrub is unset 2024-09-21T15:00:47.446 INFO:teuthology.orchestra.run.smithi032.stdout:4813 punch from 0x7b7abf0 to 0x7b818b5, (0x6cc5 bytes) 2024-09-21T15:00:47.446 INFO:teuthology.orchestra.run.smithi032.stdout:4814 punch from 0x19c9ff7 to 0x19ca752, (0x75b bytes) 2024-09-21T15:00:47.446 INFO:teuthology.orchestra.run.smithi032.stdout:4815 punch from 0x17a3fff to 0x17a772e, (0x372f bytes) 2024-09-21T15:00:47.446 INFO:teuthology.orchestra.run.smithi032.stdout:4816 punch from 0x99ab226 to 0x99b72f7, (0xc0d1 bytes) 2024-09-21T15:00:47.446 INFO:teuthology.orchestra.run.smithi032.stdout:4818 punch from 0xb86ee38 to 0xb8755b7, (0x677f bytes) 2024-09-21T15:00:47.446 INFO:teuthology.orchestra.run.smithi032.stdout:4819 punch from 0x12e5c45 to 0x12e85ef, (0x29aa bytes) 2024-09-21T15:00:47.446 INFO:teuthology.orchestra.run.smithi032.stdout:4820 read 0xb8d95c1 thru 0xb8e6d34 (0xd774 bytes) 2024-09-21T15:00:47.447 INFO:teuthology.orchestra.run.smithi032.stdout:4821 read 0x12e643 thru 0x136e6c (0x882a bytes) 2024-09-21T15:00:47.447 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:00:47.443+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:00:47.447 INFO:teuthology.orchestra.run.smithi032.stdout:4822 write 0x4657fca thru 0x4665832 (0xd869 bytes) 2024-09-21T15:00:47.450 INFO:teuthology.orchestra.run.smithi032.stdout:4823 write 0xc77eaf9 thru 0xc783f9c (0x54a4 bytes) 2024-09-21T15:00:47.453 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T15:00:47.461 INFO:teuthology.orchestra.run.smithi032.stdout:4825 punch from 0x8a534aa to 0x8a57a4d, (0x45a3 bytes) 2024-09-21T15:00:47.475 INFO:teuthology.orchestra.run.smithi032.stdout:4827 trunc from 0xc7c7130 to 0x4249229 2024-09-21T15:00:47.476 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T15:00:47.494 INFO:teuthology.orchestra.run.smithi032.stdout:4830 write 0x1f10eb6 thru 0x1f132c8 (0x2413 bytes) 2024-09-21T15:00:47.497 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:47.497 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:47.501 INFO:teuthology.orchestra.run.smithi032.stdout:4836 trunc from 0x4249229 to 0x1d40402 2024-09-21T15:00:47.507 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-21T15:00:47.531 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:47.531 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:00:47.531 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T15:00:47.531 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T15:00:47.531 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:47.533 INFO:teuthology.orchestra.run.smithi032.stdout:4838 trunc from 0x1d40402 to 0x7844da0 2024-09-21T15:00:47.535 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T15:00:47.541 INFO:teuthology.orchestra.run.smithi032.stdout:4839 write 0x45f46e3 thru 0x4603a53 (0xf371 bytes) 2024-09-21T15:00:47.544 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T15:00:47.547 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:00:47.545+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:00:47.551 INFO:teuthology.orchestra.run.smithi032.stdout:4841 write 0x17646af thru 0x1768d4d (0x469f bytes) 2024-09-21T15:00:47.554 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:47.567 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T15:00:47.568 INFO:teuthology.orchestra.run.smithi032.stdout:4847 writesame 0xaeb71b1 thru 0xaebb256 data_size 0x19(0x40a6 bytes) 2024-09-21T15:00:47.568 INFO:teuthology.orchestra.run.smithi032.stdout:4848 read 0x930154e thru 0x930e57f (0xd032 bytes) 2024-09-21T15:00:47.576 INFO:teuthology.orchestra.run.smithi032.stdout:4849 write 0xb2a4a19 thru 0xb2b1730 (0xcd18 bytes) 2024-09-21T15:00:47.582 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:47.594 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T15:00:47.597 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:00:47.597 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:00:47.597 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:00:47.597 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:00:47.597 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:00:47.597 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:00:47.597 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:00:47.597 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:00:47.597 INFO:teuthology.orchestra.run.smithi032.stdout:4851 trunc from 0xb2b1731 to 0x2226afa 2024-09-21T15:00:47.617 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:47.617 INFO:teuthology.orchestra.run.smithi032.stdout:4856 read 0x1617162 thru 0x1623986 (0xc825 bytes) 2024-09-21T15:00:47.618 INFO:teuthology.orchestra.run.smithi032.stdout:4857 punch from 0x78c7a7 to 0x793f33, (0x778c bytes) 2024-09-21T15:00:47.641 INFO:teuthology.orchestra.run.smithi032.stdout:4858 writesame 0x9b676b3 thru 0x9b7046d data_size 0x1(0x8dbb bytes) 2024-09-21T15:00:47.641 INFO:teuthology.orchestra.run.smithi032.stdout:4859 read 0x32c3a8c thru 0x32d0d26 (0xd29b bytes) 2024-09-21T15:00:47.648 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:00:47.643+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:00:47.659 INFO:teuthology.orchestra.run.smithi032.stdout:4861 write 0xe370531 thru 0xe37f7ae (0xf27e bytes) 2024-09-21T15:00:47.666 INFO:teuthology.orchestra.run.smithi032.stdout:4862 read 0x1a6d166 thru 0x1a73285 (0x6120 bytes) 2024-09-21T15:00:47.666 INFO:teuthology.orchestra.run.smithi032.stdout:4864 write 0x6a73cf6 thru 0x6a77cd6 (0x3fe1 bytes) 2024-09-21T15:00:47.671 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:47.673 INFO:teuthology.orchestra.run.smithi032.stdout:4867 trunc from 0xe37f7af to 0xd2ca730 2024-09-21T15:00:47.691 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:47.691 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T15:00:47.692 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T15:00:47.692 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T15:00:47.692 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.692 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:97677 3.15 3:a8e72133:::rbd_data.1136bfc79a70.0000000000000005:head [write 0~4440027 [fadvise_sequential+fadvise_nocache],write 4751360~147456 [fadvise_sequential+fadvise_nocache],write 4964352~147456 [fadvise_sequential+fadvise_nocache],write 5177344~65536 [fadvise_sequential+fadvise_nocache],write 5259264~147456 [fadvise_sequential+fadvise_nocache],write 5472256~147456 [fadvise_sequential+fadvise_nocache],write 5685248~229376 [fadvise_sequential+fadvise_nocache],write 5980160~147456 [fadvise_sequential+fadvise_nocache],write 6193152~229376 [fadvise_sequential+fadvise_nocache],write 6488064~147456 [fadvise_sequential+fadvise_nocache],write 6701056~229376 [fadvise_sequential+fadvise_nocache],write 6995968~147456 [fadvise_sequential+fadvise_nocache],write 7208960~229376 [fadvise_sequential+fadvise_nocache],write 7503872~147456 [fadvise_sequential+fadvise_nocache],write 7716864~229376 [fadvise_sequential+fadvise_nocache],write 8044544~114688 [fadvise_sequential+fadvise_nocache],write 8257536~114688 [fadvise_sequential+fadvise_nocache],write 8388608~65536 [fadvise_sequential+fadvise_nocache],write 8552448~114688 [fadvise_sequential+fadvise_nocache],write 8765440~114688 [fadvise_sequential+fadvise_nocache],write 8929280~32768 [fadvise_sequential+fadvise_nocache],write 9142272~32768 [fadvise_sequential+fadvise_nocache],write 9355264~32768 [fadvise_sequential+fadvise_nocache],write 9437184~32768 [fadvise_sequential+fadvise_nocache],write 9650176~32768 [fadvise_sequential+fadvise_nocache],write 9863168~32768 [fadvise_sequential+fadvise_nocache],write 9961472~131072 [fadvise_sequential+fadvise_nocache],write 10108928~196608 [fadvise_sequential+fadvise_nocache],write 10321920~278528 [fadvise_sequential+fadvise_nocache],write 10616832~196608 [fadvise_sequential+fadvise_nocache],write 10829824~278528 [fadvise_sequential+fadvise_nocache],write 11124736~196608 [fadvise_sequential+fadvise_nocache],write 11337728~196608 [fadvise_sequential+fadvise_nocache],write 11550720~65536 [fadvise_sequential+fadvise_nocache],write 11649024~98304 [fadvise_sequential+fadvise_nocache],write 11763712~65536 [fadvise_sequential+fadvise_nocache],write 11862016~98304 [fadvise_sequential+fadvise_nocache],write 11976704~65536 [fadvise_sequential+fadvise_nocache],write 12058624~65536 [fadvise_sequential+fadvise_nocache],write 12156928~98304 [fadvise_sequential+fadvise_nocache],write 12271616~65536 [fadvise_sequential+fadvise_nocache],write 12369920~98304 [fadvise_sequential+fadvise_nocache],write 12484608~65536 [fadvise_sequential+fadvise_nocache],write 12664832~98304 [fadvise_sequential+fadvise_nocache],write 12877824~98304 [fadvise_sequential+fadvise_nocache],write 13090816~16384 [fadvise_sequential+fadvise_nocache],write 13172736~98304 [fadvise_sequential+fadvise_nocache],write 13385728~98304 [fadvise_sequential+fadvise_nocache],write 13598720~32768 [fadvise_sequential+fadvise_nocache],write 13680640~98304 [fadvise_sequential+fadvise_nocache],write 13893632~98304 [fadvise_sequential+fadvise_nocache],write 14106624~49152 [fadvise_sequential+fadvise_nocache],write 14794752~81920 [fadvise_sequential+fadvise_nocache],write 15007744~81920 [fadvise_sequential+fadvise_nocache],write 15302656~81920 [fadvise_sequential+fadvise_nocache],write 15515648~81920 [fadvise_sequential+fadvise_nocache],write 15810560~81920 [fadvise_sequential+fadvise_nocache],write 16023552~81920 [fadvise_sequential+fadvise_nocache],write 16236544~16384 [fadvise_sequential+fadvise_nocache],write 16318464~81920 [fadvise_sequential+fadvise_nocache],write 16531456~81920 [fadvise_sequential+fadvise_nocache],write 16744448~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:47.692 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.302946+0000", 2024-09-21T15:00:47.692 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 42.386960367999997, 2024-09-21T15:00:47.692 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.091231204999999996, 2024-09-21T15:00:47.692 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:47.692 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:47.692 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:47.692 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:47.692 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:47.692 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 97677 2024-09-21T15:00:47.692 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.693 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:47.693 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.693 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:47.693 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.302946+0000", 2024-09-21T15:00:47.693 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:47.693 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.693 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.693 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:47.693 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.302946+0000", 2024-09-21T15:00:47.693 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:47.693 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.693 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.693 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:47.693 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.302942+0000", 2024-09-21T15:00:47.694 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999962 2024-09-21T15:00:47.694 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.694 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.694 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:47.694 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311253+0000", 2024-09-21T15:00:47.694 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0083105360000000003 2024-09-21T15:00:47.694 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.694 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.694 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:47.694 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311283+0000", 2024-09-21T15:00:47.694 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0065000000000001e-05 2024-09-21T15:00:47.694 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.694 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.695 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:47.695 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311296+0000", 2024-09-21T15:00:47.695 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3206e-05 2024-09-21T15:00:47.695 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.695 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.695 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:47.695 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311404+0000", 2024-09-21T15:00:47.695 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000107514 2024-09-21T15:00:47.695 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.695 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.695 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:47.695 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311795+0000", 2024-09-21T15:00:47.695 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000390913 2024-09-21T15:00:47.695 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.696 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.696 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 6", 2024-09-21T15:00:47.696 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.314686+0000", 2024-09-21T15:00:47.696 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0028915659999999999 2024-09-21T15:00:47.696 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.696 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.696 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:47.696 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.368178+0000", 2024-09-21T15:00:47.696 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.053491706 2024-09-21T15:00:47.696 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.696 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.696 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:47.696 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.393925+0000", 2024-09-21T15:00:47.696 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025747289 2024-09-21T15:00:47.696 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.697 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.697 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:47.697 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.394127+0000", 2024-09-21T15:00:47.697 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000201444 2024-09-21T15:00:47.697 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.697 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.697 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:47.697 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.394177+0000", 2024-09-21T15:00:47.697 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0689000000000002e-05 2024-09-21T15:00:47.697 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:47.697 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:47.697 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:47.697 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.698 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.698 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:97673 3.1d e122/24 3:bd91946e:::rbd_data.1136bfc79a70.0000000000000002:head v 122'162, mlcod=120'160)", 2024-09-21T15:00:47.698 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.313730+0000", 2024-09-21T15:00:47.698 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 42.376176043000001, 2024-09-21T15:00:47.698 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055373462999999998, 2024-09-21T15:00:47.698 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:47.698 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:47.698 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:47.698 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.698 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:47.698 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.313730+0000", 2024-09-21T15:00:47.698 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:47.698 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.698 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.699 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:47.699 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.313730+0000", 2024-09-21T15:00:47.699 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:47.699 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.699 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.699 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:47.699 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.313730+0000", 2024-09-21T15:00:47.699 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:47.699 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.699 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.699 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:47.699 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.336113+0000", 2024-09-21T15:00:47.699 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022382962999999999 2024-09-21T15:00:47.699 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.700 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.700 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:47.700 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.336140+0000", 2024-09-21T15:00:47.700 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7248999999999999e-05 2024-09-21T15:00:47.700 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.700 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.700 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:47.700 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.336151+0000", 2024-09-21T15:00:47.700 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1015e-05 2024-09-21T15:00:47.700 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.700 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.700 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:47.700 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.355428+0000", 2024-09-21T15:00:47.700 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019276379999999999 2024-09-21T15:00:47.700 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.700 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.701 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:47.701 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.355484+0000", 2024-09-21T15:00:47.701 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6820999999999997e-05 2024-09-21T15:00:47.701 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.701 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.701 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:47.701 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.369040+0000", 2024-09-21T15:00:47.701 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013555466 2024-09-21T15:00:47.701 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.701 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.701 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:47.701 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.369104+0000", 2024-09-21T15:00:47.701 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3986999999999996e-05 2024-09-21T15:00:47.701 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:47.701 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:47.701 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:47.702 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.702 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.702 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:113889 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 5300224~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:47.702 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.501880+0000", 2024-09-21T15:00:47.702 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.188026130000001, 2024-09-21T15:00:47.702 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052644204999999999, 2024-09-21T15:00:47.702 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:47.702 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:47.702 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:47.702 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:47.702 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:47.702 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 113889 2024-09-21T15:00:47.702 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.702 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:47.702 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.703 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:47.703 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501880+0000", 2024-09-21T15:00:47.703 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:47.703 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.703 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.703 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:47.703 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501880+0000", 2024-09-21T15:00:47.703 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:47.703 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.703 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.703 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:47.703 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501880+0000", 2024-09-21T15:00:47.703 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:00:47.703 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.704 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.704 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:47.704 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501882+0000", 2024-09-21T15:00:47.704 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.364e-06 2024-09-21T15:00:47.704 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.704 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.704 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:47.704 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501885+0000", 2024-09-21T15:00:47.704 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4199999999999999e-06 2024-09-21T15:00:47.704 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.704 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.704 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:47.704 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501888+0000", 2024-09-21T15:00:47.704 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8669999999999998e-06 2024-09-21T15:00:47.704 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.705 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.705 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:47.705 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.554369+0000", 2024-09-21T15:00:47.705 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052481456000000003 2024-09-21T15:00:47.705 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.705 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.705 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:47.705 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.554449+0000", 2024-09-21T15:00:47.705 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9412000000000005e-05 2024-09-21T15:00:47.705 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.705 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.705 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:47.705 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.554525+0000", 2024-09-21T15:00:47.705 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5908999999999994e-05 2024-09-21T15:00:47.705 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:47.705 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:47.706 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:47.706 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.706 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.706 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:113890 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 5390336~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:47.706 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.501890+0000", 2024-09-21T15:00:47.706 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.188016415, 2024-09-21T15:00:47.706 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052785212999999997, 2024-09-21T15:00:47.706 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:47.706 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:47.706 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:47.706 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:47.706 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:47.706 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 113890 2024-09-21T15:00:47.706 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.707 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:47.707 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.707 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:47.707 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501890+0000", 2024-09-21T15:00:47.707 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:47.707 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.707 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.707 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:47.707 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501890+0000", 2024-09-21T15:00:47.707 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:47.707 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.707 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.707 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:47.708 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501890+0000", 2024-09-21T15:00:47.708 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:00:47.708 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.708 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.708 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:47.708 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501891+0000", 2024-09-21T15:00:47.708 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3599999999999999e-06 2024-09-21T15:00:47.708 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.708 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.708 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:47.708 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501894+0000", 2024-09-21T15:00:47.708 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.717e-06 2024-09-21T15:00:47.708 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.708 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.709 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:47.709 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501897+0000", 2024-09-21T15:00:47.709 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5790000000000002e-06 2024-09-21T15:00:47.709 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.709 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.709 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:47.709 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.554569+0000", 2024-09-21T15:00:47.709 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052672434999999997 2024-09-21T15:00:47.709 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.709 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.709 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:47.709 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.554623+0000", 2024-09-21T15:00:47.709 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4509999999999998e-05 2024-09-21T15:00:47.709 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.710 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.710 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:47.710 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.554675+0000", 2024-09-21T15:00:47.710 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1833999999999999e-05 2024-09-21T15:00:47.710 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:47.710 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:47.710 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:47.710 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.710 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.710 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:113891 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 5480448~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:47.710 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.501899+0000", 2024-09-21T15:00:47.710 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.188007929000001, 2024-09-21T15:00:47.710 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052991637000000001, 2024-09-21T15:00:47.711 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:47.711 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:47.711 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:47.711 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:47.711 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:47.711 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 113891 2024-09-21T15:00:47.711 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.711 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:47.711 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.711 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:47.711 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501899+0000", 2024-09-21T15:00:47.711 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:47.711 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.711 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.711 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:47.712 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501899+0000", 2024-09-21T15:00:47.712 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:47.712 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.712 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.712 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:47.712 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501898+0000", 2024-09-21T15:00:47.712 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:00:47.712 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.712 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.712 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:47.712 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501900+0000", 2024-09-21T15:00:47.712 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.356e-06 2024-09-21T15:00:47.712 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.712 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.712 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:47.713 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501902+0000", 2024-09-21T15:00:47.713 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6620000000000001e-06 2024-09-21T15:00:47.713 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.713 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.713 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:47.713 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501905+0000", 2024-09-21T15:00:47.713 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.492e-06 2024-09-21T15:00:47.713 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.713 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.713 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:47.713 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.554707+0000", 2024-09-21T15:00:47.713 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052802618000000003 2024-09-21T15:00:47.713 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.713 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.714 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:47.714 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.554798+0000", 2024-09-21T15:00:47.714 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0976000000000004e-05 2024-09-21T15:00:47.714 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.714 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.714 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:47.714 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.554890+0000", 2024-09-21T15:00:47.714 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1755999999999994e-05 2024-09-21T15:00:47.714 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:47.714 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:47.714 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:47.714 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.714 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.715 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:113892 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 5570560~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:47.715 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.501907+0000", 2024-09-21T15:00:47.715 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.187999497, 2024-09-21T15:00:47.715 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.053089242000000002, 2024-09-21T15:00:47.715 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:47.715 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:47.715 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:47.715 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:47.715 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:47.715 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 113892 2024-09-21T15:00:47.715 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.715 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:47.715 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.715 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:47.716 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501907+0000", 2024-09-21T15:00:47.716 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:47.716 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.716 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.716 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:47.716 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501907+0000", 2024-09-21T15:00:47.716 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:47.716 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.716 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.716 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:47.716 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501907+0000", 2024-09-21T15:00:47.716 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:00:47.716 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.716 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.716 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:47.717 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501908+0000", 2024-09-21T15:00:47.717 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2860000000000001e-06 2024-09-21T15:00:47.717 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.717 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.717 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:47.717 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501913+0000", 2024-09-21T15:00:47.717 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1320000000000002e-06 2024-09-21T15:00:47.717 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.717 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.717 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:47.717 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501916+0000", 2024-09-21T15:00:47.717 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5330000000000001e-06 2024-09-21T15:00:47.717 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.717 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.718 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:47.718 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.554904+0000", 2024-09-21T15:00:47.718 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052988631000000001 2024-09-21T15:00:47.718 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.718 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.718 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:47.718 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.554945+0000", 2024-09-21T15:00:47.718 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.0695999999999998e-05 2024-09-21T15:00:47.718 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.718 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.718 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:47.718 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.554996+0000", 2024-09-21T15:00:47.718 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1193999999999998e-05 2024-09-21T15:00:47.718 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:47.719 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:47.719 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:47.719 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.719 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.719 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:113893 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 5660672~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:47.719 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.501918+0000", 2024-09-21T15:00:47.719 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.187988520000001, 2024-09-21T15:00:47.719 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.053245595, 2024-09-21T15:00:47.719 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:47.719 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:47.719 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:47.719 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:47.719 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:47.719 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 113893 2024-09-21T15:00:47.720 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.720 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:47.720 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.720 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:47.720 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501918+0000", 2024-09-21T15:00:47.720 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:47.720 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.720 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.720 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:47.720 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501918+0000", 2024-09-21T15:00:47.720 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:47.720 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.720 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.720 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:47.720 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501918+0000", 2024-09-21T15:00:47.721 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:47.721 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.721 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.721 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:47.721 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501920+0000", 2024-09-21T15:00:47.721 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9800000000000001e-06 2024-09-21T15:00:47.721 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.721 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.721 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:47.721 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501928+0000", 2024-09-21T15:00:47.721 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7999999999999999e-06 2024-09-21T15:00:47.721 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.721 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.722 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:47.722 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501930+0000", 2024-09-21T15:00:47.722 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.785e-06 2024-09-21T15:00:47.722 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.722 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.722 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:47.722 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555019+0000", 2024-09-21T15:00:47.722 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.053089154 2024-09-21T15:00:47.722 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.722 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.722 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:47.722 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555065+0000", 2024-09-21T15:00:47.722 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5485000000000002e-05 2024-09-21T15:00:47.722 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.723 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.723 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:47.723 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555164+0000", 2024-09-21T15:00:47.723 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8629999999999996e-05 2024-09-21T15:00:47.723 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:47.723 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:47.723 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:47.723 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.723 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.723 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:113894 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 5750784~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:47.723 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.502384+0000", 2024-09-21T15:00:47.723 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.187522583, 2024-09-21T15:00:47.723 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052884821999999998, 2024-09-21T15:00:47.723 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:47.724 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:47.724 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:47.724 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:47.724 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:47.724 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 113894 2024-09-21T15:00:47.724 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.724 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:47.724 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.724 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:47.724 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502384+0000", 2024-09-21T15:00:47.724 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:47.724 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.724 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.724 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:47.725 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502384+0000", 2024-09-21T15:00:47.725 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:47.725 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.725 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.725 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:47.725 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501933+0000", 2024-09-21T15:00:47.725 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9995484 2024-09-21T15:00:47.725 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.725 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.725 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:47.725 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502390+0000", 2024-09-21T15:00:47.725 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00045714099999999999 2024-09-21T15:00:47.725 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.725 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.725 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:47.726 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502402+0000", 2024-09-21T15:00:47.726 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.236e-05 2024-09-21T15:00:47.726 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.726 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.726 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:47.726 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502411+0000", 2024-09-21T15:00:47.726 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8689999999999993e-06 2024-09-21T15:00:47.726 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.726 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.726 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:47.726 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555178+0000", 2024-09-21T15:00:47.726 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052767050000000003 2024-09-21T15:00:47.726 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.727 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.727 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:47.727 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555215+0000", 2024-09-21T15:00:47.727 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.667e-05 2024-09-21T15:00:47.727 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.727 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.727 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:47.727 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555269+0000", 2024-09-21T15:00:47.727 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4109e-05 2024-09-21T15:00:47.727 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:47.727 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:47.727 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:47.727 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.727 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.727 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:113895 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 5840896~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:47.728 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.502651+0000", 2024-09-21T15:00:47.728 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.187255969999999, 2024-09-21T15:00:47.728 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052770692000000001, 2024-09-21T15:00:47.728 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:47.728 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:47.728 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:47.728 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:47.728 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:47.728 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 113895 2024-09-21T15:00:47.728 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.728 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:47.728 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.728 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:47.728 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502651+0000", 2024-09-21T15:00:47.729 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:47.729 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.729 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.729 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:47.729 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502651+0000", 2024-09-21T15:00:47.729 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:47.729 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.729 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.729 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:47.729 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502421+0000", 2024-09-21T15:00:47.729 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9997702 2024-09-21T15:00:47.729 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.729 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.729 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:47.729 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502658+0000", 2024-09-21T15:00:47.730 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000237264 2024-09-21T15:00:47.730 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.730 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.730 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:47.730 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502673+0000", 2024-09-21T15:00:47.730 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4474999999999999e-05 2024-09-21T15:00:47.730 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.730 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.730 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:47.730 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502681+0000", 2024-09-21T15:00:47.730 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6119999999999999e-06 2024-09-21T15:00:47.730 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.730 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.730 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:47.730 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555285+0000", 2024-09-21T15:00:47.731 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052603860000000002 2024-09-21T15:00:47.731 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.731 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.731 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:47.731 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555348+0000", 2024-09-21T15:00:47.731 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3411999999999995e-05 2024-09-21T15:00:47.731 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.731 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.731 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:47.731 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555421+0000", 2024-09-21T15:00:47.731 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2809e-05 2024-09-21T15:00:47.731 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:47.731 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:47.731 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:47.731 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.732 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.732 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:113896 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 5931008~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:47.732 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.502862+0000", 2024-09-21T15:00:47.732 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.187044422, 2024-09-21T15:00:47.732 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052660208, 2024-09-21T15:00:47.732 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:47.732 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:47.732 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:47.732 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:47.732 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:47.732 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 113896 2024-09-21T15:00:47.732 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.732 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:47.732 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.733 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:47.733 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502862+0000", 2024-09-21T15:00:47.733 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:47.733 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.733 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.733 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:47.733 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502862+0000", 2024-09-21T15:00:47.733 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:47.733 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.733 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.733 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:47.733 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502689+0000", 2024-09-21T15:00:47.733 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9998264 2024-09-21T15:00:47.733 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.733 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.733 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:47.734 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502873+0000", 2024-09-21T15:00:47.734 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000184146 2024-09-21T15:00:47.734 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.734 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.734 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:47.734 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502884+0000", 2024-09-21T15:00:47.734 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1293e-05 2024-09-21T15:00:47.734 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.734 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.734 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:47.734 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502891+0000", 2024-09-21T15:00:47.734 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4259999999999997e-06 2024-09-21T15:00:47.734 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.734 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.735 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:47.735 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555435+0000", 2024-09-21T15:00:47.735 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052543053999999999 2024-09-21T15:00:47.735 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.735 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.735 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:47.735 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555471+0000", 2024-09-21T15:00:47.735 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6454e-05 2024-09-21T15:00:47.735 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.735 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.735 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:47.735 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555522+0000", 2024-09-21T15:00:47.735 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1314999999999999e-05 2024-09-21T15:00:47.735 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:47.736 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:47.736 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:47.736 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.736 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.736 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:113897 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 6021120~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:47.736 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.503004+0000", 2024-09-21T15:00:47.736 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.186902315000001, 2024-09-21T15:00:47.736 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052669499000000002, 2024-09-21T15:00:47.736 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:47.736 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:47.736 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:47.736 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:47.736 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:47.736 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 113897 2024-09-21T15:00:47.736 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.737 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:47.737 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.737 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:47.737 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503004+0000", 2024-09-21T15:00:47.737 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:47.737 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.737 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.737 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:47.737 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503004+0000", 2024-09-21T15:00:47.737 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:47.737 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.737 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.737 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:47.737 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502898+0000", 2024-09-21T15:00:47.737 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9998937 2024-09-21T15:00:47.738 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.738 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.738 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:47.738 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503010+0000", 2024-09-21T15:00:47.738 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000112396 2024-09-21T15:00:47.738 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.738 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.738 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:47.738 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503023+0000", 2024-09-21T15:00:47.738 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2206e-05 2024-09-21T15:00:47.738 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.738 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.738 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:47.738 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503031+0000", 2024-09-21T15:00:47.738 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3780000000000001e-06 2024-09-21T15:00:47.739 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.739 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.739 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:47.739 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555549+0000", 2024-09-21T15:00:47.739 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052517909000000002 2024-09-21T15:00:47.739 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.739 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.739 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:47.739 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555594+0000", 2024-09-21T15:00:47.739 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5385e-05 2024-09-21T15:00:47.739 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.739 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.739 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:47.739 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555674+0000", 2024-09-21T15:00:47.740 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9447e-05 2024-09-21T15:00:47.740 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:47.740 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:47.740 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:47.740 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.740 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.740 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:113898 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 6111232~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:47.740 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.503039+0000", 2024-09-21T15:00:47.740 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.186867530000001, 2024-09-21T15:00:47.740 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052754290000000002, 2024-09-21T15:00:47.740 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:47.740 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:47.740 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:47.740 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:47.740 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:47.741 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 113898 2024-09-21T15:00:47.741 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.741 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:47.741 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.741 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:47.741 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503039+0000", 2024-09-21T15:00:47.741 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:47.741 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.741 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.741 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:47.741 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503039+0000", 2024-09-21T15:00:47.741 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:47.741 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.741 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.741 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:47.742 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503038+0000", 2024-09-21T15:00:47.742 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:00:47.742 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.742 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.742 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:47.742 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503042+0000", 2024-09-21T15:00:47.742 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.0640000000000004e-06 2024-09-21T15:00:47.742 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.742 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.742 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:47.742 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503049+0000", 2024-09-21T15:00:47.742 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5579999999999997e-06 2024-09-21T15:00:47.742 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.742 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.743 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:47.743 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503054+0000", 2024-09-21T15:00:47.743 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6550000000000001e-06 2024-09-21T15:00:47.743 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.743 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.743 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:47.743 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555693+0000", 2024-09-21T15:00:47.743 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052638216000000002 2024-09-21T15:00:47.743 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.743 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.743 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:47.743 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555736+0000", 2024-09-21T15:00:47.743 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3619999999999999e-05 2024-09-21T15:00:47.743 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.743 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.744 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:47.744 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555793+0000", 2024-09-21T15:00:47.744 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6956999999999997e-05 2024-09-21T15:00:47.744 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:47.744 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:47.744 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:47.744 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.744 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.744 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:113899 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 6201344~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:47.744 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.503255+0000", 2024-09-21T15:00:47.744 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.186651257000001, 2024-09-21T15:00:47.744 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052648161999999998, 2024-09-21T15:00:47.744 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:47.744 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:47.745 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:47.745 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:47.745 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:47.745 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 113899 2024-09-21T15:00:47.745 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.745 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:47.745 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.745 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:47.745 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503255+0000", 2024-09-21T15:00:47.745 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:47.745 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.745 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.745 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:47.745 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503255+0000", 2024-09-21T15:00:47.745 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:47.746 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.746 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.746 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:47.746 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503058+0000", 2024-09-21T15:00:47.746 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9998021 2024-09-21T15:00:47.746 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.746 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.746 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:47.746 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503262+0000", 2024-09-21T15:00:47.746 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00020421299999999999 2024-09-21T15:00:47.746 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.746 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.746 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:47.746 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503274+0000", 2024-09-21T15:00:47.747 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2052000000000001e-05 2024-09-21T15:00:47.747 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.747 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.747 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:47.747 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503282+0000", 2024-09-21T15:00:47.747 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.977e-06 2024-09-21T15:00:47.747 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.747 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.747 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:47.747 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555808+0000", 2024-09-21T15:00:47.747 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052526150000000001 2024-09-21T15:00:47.747 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.747 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.747 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:47.747 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555851+0000", 2024-09-21T15:00:47.748 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2846999999999998e-05 2024-09-21T15:00:47.748 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.748 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.748 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:47.748 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555903+0000", 2024-09-21T15:00:47.748 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2607e-05 2024-09-21T15:00:47.748 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:47.748 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:47.748 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:47.748 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.748 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.748 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:113900 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 6307840~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:47.748 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.503445+0000", 2024-09-21T15:00:47.748 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.186461322, 2024-09-21T15:00:47.748 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052589546000000001, 2024-09-21T15:00:47.749 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:47.749 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:47.749 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:47.749 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:47.749 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:47.749 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 113900 2024-09-21T15:00:47.749 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.749 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:47.749 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.749 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:47.749 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503445+0000", 2024-09-21T15:00:47.749 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:47.749 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.749 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.749 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:47.749 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503445+0000", 2024-09-21T15:00:47.750 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:47.750 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.750 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.750 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:47.750 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503290+0000", 2024-09-21T15:00:47.750 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9998446 2024-09-21T15:00:47.750 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.750 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.750 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:47.750 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503451+0000", 2024-09-21T15:00:47.750 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00016111000000000001 2024-09-21T15:00:47.750 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.750 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.750 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:47.750 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503461+0000", 2024-09-21T15:00:47.751 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8740000000000007e-06 2024-09-21T15:00:47.751 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.751 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.751 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:47.751 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503468+0000", 2024-09-21T15:00:47.751 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.187e-06 2024-09-21T15:00:47.751 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.751 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.751 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:47.751 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555930+0000", 2024-09-21T15:00:47.751 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052462404999999997 2024-09-21T15:00:47.751 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.751 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.751 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:47.751 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555971+0000", 2024-09-21T15:00:47.751 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.0419e-05 2024-09-21T15:00:47.752 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.752 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.752 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:47.752 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.556035+0000", 2024-09-21T15:00:47.752 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3948999999999994e-05 2024-09-21T15:00:47.752 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:47.752 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:47.752 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:47.752 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.752 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.752 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:114050 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 4767744~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:47.752 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.514988+0000", 2024-09-21T15:00:47.752 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.174918811000001, 2024-09-21T15:00:47.752 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052614432000000003, 2024-09-21T15:00:47.752 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:47.753 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:47.753 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:47.753 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:47.753 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:47.753 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 114050 2024-09-21T15:00:47.753 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.753 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:47.753 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.753 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:47.753 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.514988+0000", 2024-09-21T15:00:47.753 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:47.753 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.753 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.753 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:47.754 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.514988+0000", 2024-09-21T15:00:47.754 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:47.754 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.754 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.754 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:47.754 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.514987+0000", 2024-09-21T15:00:47.754 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:47.754 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.754 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.754 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:47.754 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.514990+0000", 2024-09-21T15:00:47.754 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4739999999999999e-06 2024-09-21T15:00:47.754 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.754 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.755 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:47.755 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.514995+0000", 2024-09-21T15:00:47.755 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1939999999999997e-06 2024-09-21T15:00:47.755 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.755 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.755 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:47.755 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.514999+0000", 2024-09-21T15:00:47.755 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.245e-06 2024-09-21T15:00:47.755 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.755 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.755 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:47.755 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.567498+0000", 2024-09-21T15:00:47.755 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052498516000000002 2024-09-21T15:00:47.755 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.755 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.756 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:47.756 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.567541+0000", 2024-09-21T15:00:47.756 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3516000000000001e-05 2024-09-21T15:00:47.756 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.756 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.756 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:47.756 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.567602+0000", 2024-09-21T15:00:47.756 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0800000000000001e-05 2024-09-21T15:00:47.756 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:47.756 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:47.756 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:47.756 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.756 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.756 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:114051 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 4857856~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:47.756 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.515003+0000", 2024-09-21T15:00:47.757 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.174903096000001, 2024-09-21T15:00:47.757 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052709245000000002, 2024-09-21T15:00:47.757 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:47.757 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:47.757 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:47.757 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:47.757 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:47.757 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 114051 2024-09-21T15:00:47.757 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.757 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:47.757 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.757 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:47.757 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515003+0000", 2024-09-21T15:00:47.757 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:47.757 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.757 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.758 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:47.758 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515003+0000", 2024-09-21T15:00:47.758 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:47.758 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.758 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.758 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:47.758 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515003+0000", 2024-09-21T15:00:47.758 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:47.758 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.758 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.758 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:47.758 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515005+0000", 2024-09-21T15:00:47.758 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3499999999999999e-06 2024-09-21T15:00:47.758 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.759 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.759 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:47.759 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515011+0000", 2024-09-21T15:00:47.759 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2850000000000002e-06 2024-09-21T15:00:47.759 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.759 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.759 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:47.759 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515017+0000", 2024-09-21T15:00:47.759 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0059999999999998e-06 2024-09-21T15:00:47.759 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.759 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.759 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:47.759 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.567618+0000", 2024-09-21T15:00:47.759 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052600806999999999 2024-09-21T15:00:47.759 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.760 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.760 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:47.760 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.567655+0000", 2024-09-21T15:00:47.760 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7866e-05 2024-09-21T15:00:47.760 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.760 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.760 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:47.760 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.567713+0000", 2024-09-21T15:00:47.760 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7281999999999998e-05 2024-09-21T15:00:47.760 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:47.760 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:47.760 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:47.760 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.760 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.760 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:114052 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 4947968~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:47.761 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.515046+0000", 2024-09-21T15:00:47.761 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.174860437, 2024-09-21T15:00:47.761 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052795975000000002, 2024-09-21T15:00:47.761 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:47.761 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:47.761 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:47.761 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:47.761 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:47.761 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 114052 2024-09-21T15:00:47.761 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.761 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:47.761 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.761 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:47.761 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515046+0000", 2024-09-21T15:00:47.761 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:47.761 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.762 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.762 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:47.762 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515046+0000", 2024-09-21T15:00:47.762 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:47.762 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.762 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.762 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:47.762 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515046+0000", 2024-09-21T15:00:47.762 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:47.762 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.762 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.762 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:47.762 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515050+0000", 2024-09-21T15:00:47.762 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1359999999999999e-06 2024-09-21T15:00:47.762 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.762 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.763 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:47.763 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515060+0000", 2024-09-21T15:00:47.763 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0339e-05 2024-09-21T15:00:47.763 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.763 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.763 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:47.763 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515068+0000", 2024-09-21T15:00:47.763 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1149999999999994e-06 2024-09-21T15:00:47.763 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.763 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.763 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:47.763 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.567733+0000", 2024-09-21T15:00:47.763 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052664872000000001 2024-09-21T15:00:47.763 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.763 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.764 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:47.764 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.567778+0000", 2024-09-21T15:00:47.764 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4966999999999998e-05 2024-09-21T15:00:47.764 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.764 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.764 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:47.764 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.567842+0000", 2024-09-21T15:00:47.764 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4114000000000003e-05 2024-09-21T15:00:47.764 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:47.764 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:47.764 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:47.764 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.764 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.764 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:114053 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 5038080~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:47.764 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.515076+0000", 2024-09-21T15:00:47.765 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.174830577000002, 2024-09-21T15:00:47.765 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052886112999999998, 2024-09-21T15:00:47.765 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:47.765 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:47.765 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:47.765 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:47.765 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:47.765 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 114053 2024-09-21T15:00:47.765 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.765 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:47.765 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.765 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:47.765 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515076+0000", 2024-09-21T15:00:47.765 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:47.766 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.766 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.766 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:47.766 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515076+0000", 2024-09-21T15:00:47.766 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:47.766 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.766 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.766 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:47.766 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515075+0000", 2024-09-21T15:00:47.766 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:47.766 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.766 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.766 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:47.766 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515083+0000", 2024-09-21T15:00:47.767 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0520000000000002e-06 2024-09-21T15:00:47.767 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.767 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.767 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:47.767 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515092+0000", 2024-09-21T15:00:47.767 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1440000000000003e-06 2024-09-21T15:00:47.767 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.767 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.767 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:47.767 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515098+0000", 2024-09-21T15:00:47.767 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1090000000000003e-06 2024-09-21T15:00:47.767 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.767 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.767 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:47.768 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.567871+0000", 2024-09-21T15:00:47.768 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052772966999999997 2024-09-21T15:00:47.768 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.768 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.768 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:47.768 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.567909+0000", 2024-09-21T15:00:47.768 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.8380000000000002e-05 2024-09-21T15:00:47.768 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.768 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.768 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:47.768 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.567962+0000", 2024-09-21T15:00:47.768 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3068000000000001e-05 2024-09-21T15:00:47.768 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:47.768 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:47.768 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:47.769 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.769 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.769 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:114055 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 5218304~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:47.769 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.515526+0000", 2024-09-21T15:00:47.769 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.174380137, 2024-09-21T15:00:47.769 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052684278000000001, 2024-09-21T15:00:47.769 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:47.769 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:47.769 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:47.769 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:47.769 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:47.769 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 114055 2024-09-21T15:00:47.769 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.769 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:47.769 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.770 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:47.770 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515526+0000", 2024-09-21T15:00:47.770 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:47.770 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.770 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.770 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:47.770 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515526+0000", 2024-09-21T15:00:47.770 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:47.770 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.770 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.770 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:47.770 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515526+0000", 2024-09-21T15:00:47.770 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:47.770 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.771 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.771 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:47.771 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515529+0000", 2024-09-21T15:00:47.771 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.596e-06 2024-09-21T15:00:47.771 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.771 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.771 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:47.771 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515536+0000", 2024-09-21T15:00:47.771 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7399999999999998e-06 2024-09-21T15:00:47.771 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.771 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.771 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:47.771 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515542+0000", 2024-09-21T15:00:47.771 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5740000000000003e-06 2024-09-21T15:00:47.772 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.772 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.772 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:47.772 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.568116+0000", 2024-09-21T15:00:47.772 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052574152999999998 2024-09-21T15:00:47.772 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.772 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.772 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:47.772 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.568154+0000", 2024-09-21T15:00:47.772 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7926000000000002e-05 2024-09-21T15:00:47.772 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.772 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.772 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:47.772 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.568211+0000", 2024-09-21T15:00:47.772 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6858999999999999e-05 2024-09-21T15:00:47.773 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:47.773 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:47.773 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:47.773 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.773 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.773 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:114056 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 5308416~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:47.773 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.515547+0000", 2024-09-21T15:00:47.773 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.174359112000001, 2024-09-21T15:00:47.773 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052792795000000003, 2024-09-21T15:00:47.773 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:47.773 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:47.773 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:47.773 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:47.773 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:47.773 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 114056 2024-09-21T15:00:47.774 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.774 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:47.774 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.774 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:47.774 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515547+0000", 2024-09-21T15:00:47.774 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:47.774 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.774 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.774 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:47.774 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515547+0000", 2024-09-21T15:00:47.774 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:47.774 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.774 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.774 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:47.775 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515547+0000", 2024-09-21T15:00:47.775 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:47.775 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.775 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.775 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:47.775 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515550+0000", 2024-09-21T15:00:47.775 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2770000000000001e-06 2024-09-21T15:00:47.775 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.775 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.775 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:47.775 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515554+0000", 2024-09-21T15:00:47.775 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.087e-06 2024-09-21T15:00:47.775 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.776 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.776 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:47.776 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515558+0000", 2024-09-21T15:00:47.776 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1709999999999998e-06 2024-09-21T15:00:47.776 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.776 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.776 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:47.776 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.568234+0000", 2024-09-21T15:00:47.776 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052676675999999999 2024-09-21T15:00:47.776 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.776 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.776 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:47.776 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.568277+0000", 2024-09-21T15:00:47.776 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2400000000000001e-05 2024-09-21T15:00:47.776 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.777 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.777 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:47.777 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.568340+0000", 2024-09-21T15:00:47.777 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3581e-05 2024-09-21T15:00:47.777 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:47.777 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:47.777 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:47.777 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:47.777 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:47.777 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:47.778 INFO:teuthology.orchestra.run.smithi032.stdout:4868 write 0xe9c2526 thru 0xe9c3e10 (0x18eb bytes) 2024-09-21T15:00:47.779 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T15:00:47.780 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:47.780 INFO:teuthology.orchestra.run.smithi032.stdout:4872 read 0x481f425 thru 0x4821b2d (0x2709 bytes) 2024-09-21T15:00:47.780 INFO:teuthology.orchestra.run.smithi032.stdout:4873 write 0xcd5698d thru 0xcd5d900 (0x6f74 bytes) 2024-09-21T15:00:47.780 INFO:teuthology.orchestra.run.smithi032.stdout:4874 read 0x1317a0c thru 0x13269a9 (0xef9e bytes) 2024-09-21T15:00:47.780 INFO:teuthology.orchestra.run.smithi032.stdout:4875 writesame 0xd76bb2f thru 0xd778700 data_size 0x7(0xcbd2 bytes) 2024-09-21T15:00:47.780 INFO:teuthology.orchestra.run.smithi032.stdout:4876 read 0x2623030 thru 0x262f44f (0xc420 bytes) 2024-09-21T15:00:47.780 INFO:teuthology.orchestra.run.smithi032.stdout:4877 write 0x97696e6 thru 0x9778a71 (0xf38c bytes) 2024-09-21T15:00:47.780 INFO:teuthology.orchestra.run.smithi032.stdout:4878 read 0x53b583e thru 0x53b88cf (0x3092 bytes) 2024-09-21T15:00:47.781 INFO:teuthology.orchestra.run.smithi032.stdout:4879 write 0x6e39121 thru 0x6e3ebb6 (0x5a96 bytes) 2024-09-21T15:00:47.781 INFO:teuthology.orchestra.run.smithi032.stdout:4880 writesame 0xec4c42 thru 0xecf828 data_size 0x1(0xabe7 bytes) 2024-09-21T15:00:47.782 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-21T15:00:47.782 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:47.782 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 160, 2024-09-21T15:00:47.782 INFO:teuthology.orchestra.run.smithi032.stdout: "fsid": "caf51e55-9234-4a26-bcd2-d60811962560", 2024-09-21T15:00:47.782 INFO:teuthology.orchestra.run.smithi032.stdout: "created": "2024-09-21T14:57:50.338822+0000", 2024-09-21T15:00:47.782 INFO:teuthology.orchestra.run.smithi032.stdout: "modified": "2024-09-21T15:00:47.422685+0000", 2024-09-21T15:00:47.782 INFO:teuthology.orchestra.run.smithi032.stdout: "last_up_change": "2024-09-21T14:58:01.177141+0000", 2024-09-21T15:00:47.782 INFO:teuthology.orchestra.run.smithi032.stdout: "last_in_change": "2024-09-21T14:59:27.207073+0000", 2024-09-21T15:00:47.782 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-21T15:00:47.782 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_num": 5804032, 2024-09-21T15:00:47.782 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_set": [ 2024-09-21T15:00:47.782 INFO:teuthology.orchestra.run.smithi032.stdout: "nodeep-scrub", 2024-09-21T15:00:47.782 INFO:teuthology.orchestra.run.smithi032.stdout: "pglog_hardlimit", 2024-09-21T15:00:47.783 INFO:teuthology.orchestra.run.smithi032.stdout: "purged_snapdirs", 2024-09-21T15:00:47.783 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_deletes", 2024-09-21T15:00:47.783 INFO:teuthology.orchestra.run.smithi032.stdout: "sortbitwise" 2024-09-21T15:00:47.783 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-21T15:00:47.783 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_version": 5, 2024-09-21T15:00:47.783 INFO:teuthology.orchestra.run.smithi032.stdout: "full_ratio": 0.94999998807907104, 2024-09-21T15:00:47.783 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-21T15:00:47.783 INFO:teuthology.orchestra.run.smithi032.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-21T15:00:47.783 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_snapshot": "", 2024-09-21T15:00:47.783 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_max": 3, 2024-09-21T15:00:47.783 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd": 8, 2024-09-21T15:00:47.783 INFO:teuthology.orchestra.run.smithi032.stdout: "require_min_compat_client": "luminous", 2024-09-21T15:00:47.783 INFO:teuthology.orchestra.run.smithi032.stdout: "min_compat_client": "luminous", 2024-09-21T15:00:47.783 INFO:teuthology.orchestra.run.smithi032.stdout: "require_osd_release": "reef", 2024-09-21T15:00:47.784 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_crimson": false, 2024-09-21T15:00:47.784 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2024-09-21T15:00:47.784 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.784 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 1, 2024-09-21T15:00:47.784 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": ".mgr", 2024-09-21T15:00:47.784 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-09-21T14:58:01.842810+0000", 2024-09-21T15:00:47.784 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-09-21T15:00:47.784 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2024-09-21T15:00:47.784 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-09-21T15:00:47.784 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-09-21T15:00:47.784 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-09-21T15:00:47.784 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-09-21T15:00:47.784 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-09-21T15:00:47.784 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-09-21T15:00:47.785 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T15:00:47.785 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T15:00:47.785 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-09-21T15:00:47.785 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-09-21T15:00:47.785 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 1, 2024-09-21T15:00:47.785 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 1, 2024-09-21T15:00:47.785 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 1, 2024-09-21T15:00:47.785 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 1, 2024-09-21T15:00:47.785 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 1, 2024-09-21T15:00:47.785 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-09-21T15:00:47.785 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-09-21T15:00:47.785 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-09-21T15:00:47.785 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-09-21T15:00:47.785 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-09-21T15:00:47.786 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-09-21T15:00:47.786 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-09-21T15:00:47.786 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.786 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "18", 2024-09-21T15:00:47.786 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-09-21T15:00:47.786 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T15:00:47.786 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T15:00:47.786 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-09-21T15:00:47.786 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-09-21T15:00:47.786 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-09-21T15:00:47.786 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-09-21T15:00:47.786 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-09-21T15:00:47.786 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-09-21T15:00:47.787 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-09-21T15:00:47.787 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-09-21T15:00:47.787 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-09-21T15:00:47.787 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-09-21T15:00:47.787 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-09-21T15:00:47.787 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-09-21T15:00:47.787 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-09-21T15:00:47.787 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-09-21T15:00:47.787 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-09-21T15:00:47.787 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T15:00:47.787 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T15:00:47.787 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T15:00:47.787 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-09-21T15:00:47.787 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-09-21T15:00:47.787 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-09-21T15:00:47.788 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-09-21T15:00:47.788 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-09-21T15:00:47.788 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.788 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-09-21T15:00:47.788 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-09-21T15:00:47.788 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-09-21T15:00:47.788 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-09-21T15:00:47.788 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-09-21T15:00:47.788 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T15:00:47.788 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-09-21T15:00:47.788 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-09-21T15:00:47.788 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-09-21T15:00:47.788 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-09-21T15:00:47.788 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-09-21T15:00:47.789 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-09-21T15:00:47.789 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_max": 32, 2024-09-21T15:00:47.789 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 1 2024-09-21T15:00:47.789 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.789 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-09-21T15:00:47.789 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": {} 2024-09-21T15:00:47.789 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.789 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-09-21T15:00:47.789 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 8, 2024-09-21T15:00:47.789 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 8, 2024-09-21T15:00:47.789 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 0.25, 2024-09-21T15:00:47.789 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 2, 2024-09-21T15:00:47.789 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 2, 2024-09-21T15:00:47.789 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 0.95859527587890625, 2024-09-21T15:00:47.790 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 0.95999997854232788, 2024-09-21T15:00:47.790 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 0.95999997854232788 2024-09-21T15:00:47.790 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:47.790 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.790 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.790 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 2, 2024-09-21T15:00:47.790 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "rbd", 2024-09-21T15:00:47.790 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-09-21T14:58:08.284925+0000", 2024-09-21T15:00:47.790 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 8193, 2024-09-21T15:00:47.790 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T15:00:47.790 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-09-21T15:00:47.790 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-09-21T15:00:47.790 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-09-21T15:00:47.790 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-09-21T15:00:47.791 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-09-21T15:00:47.791 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-09-21T15:00:47.791 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T15:00:47.791 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T15:00:47.791 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-09-21T15:00:47.791 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-09-21T15:00:47.791 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2024-09-21T15:00:47.791 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2024-09-21T15:00:47.791 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2024-09-21T15:00:47.791 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2024-09-21T15:00:47.791 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2024-09-21T15:00:47.791 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-09-21T15:00:47.791 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-09-21T15:00:47.791 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-09-21T15:00:47.792 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-09-21T15:00:47.792 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-09-21T15:00:47.792 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-09-21T15:00:47.792 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-09-21T15:00:47.792 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.792 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "22", 2024-09-21T15:00:47.792 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-09-21T15:00:47.792 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T15:00:47.792 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T15:00:47.792 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-09-21T15:00:47.792 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-09-21T15:00:47.792 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 3, 2024-09-21T15:00:47.792 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 22, 2024-09-21T15:00:47.792 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-09-21T15:00:47.793 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-09-21T15:00:47.793 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-09-21T15:00:47.793 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-09-21T15:00:47.793 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-09-21T15:00:47.793 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-09-21T15:00:47.793 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-09-21T15:00:47.793 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-09-21T15:00:47.793 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-09-21T15:00:47.793 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-09-21T15:00:47.793 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-09-21T15:00:47.793 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T15:00:47.793 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T15:00:47.793 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T15:00:47.793 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-09-21T15:00:47.794 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-09-21T15:00:47.794 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-09-21T15:00:47.794 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-09-21T15:00:47.794 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-09-21T15:00:47.794 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.794 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-09-21T15:00:47.794 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-09-21T15:00:47.794 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-09-21T15:00:47.794 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-09-21T15:00:47.794 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-09-21T15:00:47.794 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T15:00:47.794 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-09-21T15:00:47.794 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-09-21T15:00:47.794 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-09-21T15:00:47.795 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-09-21T15:00:47.795 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-09-21T15:00:47.795 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2024-09-21T15:00:47.795 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-09-21T15:00:47.795 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd": {} 2024-09-21T15:00:47.795 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.795 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-09-21T15:00:47.795 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 4.9800000190734863, 2024-09-21T15:00:47.795 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 4.9800000190734863, 2024-09-21T15:00:47.795 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 0.87999999523162842, 2024-09-21T15:00:47.795 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 4.380000114440918, 2024-09-21T15:00:47.795 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 4.380000114440918, 2024-09-21T15:00:47.795 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 0.95859527587890625, 2024-09-21T15:00:47.795 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 0.95999997854232788, 2024-09-21T15:00:47.796 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 0.95999997854232788 2024-09-21T15:00:47.796 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:47.796 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.796 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.796 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 3, 2024-09-21T15:00:47.796 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "pool_client.0", 2024-09-21T15:00:47.796 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-09-21T14:58:17.510709+0000", 2024-09-21T15:00:47.796 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 8193, 2024-09-21T15:00:47.796 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T15:00:47.796 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-09-21T15:00:47.796 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-09-21T15:00:47.796 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-09-21T15:00:47.796 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-09-21T15:00:47.796 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-09-21T15:00:47.796 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-09-21T15:00:47.796 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T15:00:47.797 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T15:00:47.797 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-09-21T15:00:47.797 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-09-21T15:00:47.797 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 32, 2024-09-21T15:00:47.797 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 32, 2024-09-21T15:00:47.797 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 32, 2024-09-21T15:00:47.797 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 32, 2024-09-21T15:00:47.797 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 32, 2024-09-21T15:00:47.797 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-09-21T15:00:47.797 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-09-21T15:00:47.797 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-09-21T15:00:47.797 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-09-21T15:00:47.797 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-09-21T15:00:47.797 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-09-21T15:00:47.797 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-09-21T15:00:47.797 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.798 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "159", 2024-09-21T15:00:47.798 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-09-21T15:00:47.798 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T15:00:47.798 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T15:00:47.798 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-09-21T15:00:47.798 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-09-21T15:00:47.798 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 99, 2024-09-21T15:00:47.798 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 159, 2024-09-21T15:00:47.798 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-09-21T15:00:47.798 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-09-21T15:00:47.798 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-09-21T15:00:47.798 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-09-21T15:00:47.798 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-09-21T15:00:47.798 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-09-21T15:00:47.798 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-09-21T15:00:47.799 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-09-21T15:00:47.799 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-09-21T15:00:47.799 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-09-21T15:00:47.799 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-09-21T15:00:47.799 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T15:00:47.799 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T15:00:47.799 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T15:00:47.799 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-09-21T15:00:47.799 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-09-21T15:00:47.799 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-09-21T15:00:47.799 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-09-21T15:00:47.799 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-09-21T15:00:47.799 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.799 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-09-21T15:00:47.800 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-09-21T15:00:47.800 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-09-21T15:00:47.800 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-09-21T15:00:47.800 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-09-21T15:00:47.800 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T15:00:47.800 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-09-21T15:00:47.800 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-09-21T15:00:47.800 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-09-21T15:00:47.800 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-09-21T15:00:47.800 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-09-21T15:00:47.800 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2024-09-21T15:00:47.800 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-09-21T15:00:47.800 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd": {} 2024-09-21T15:00:47.800 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.800 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-09-21T15:00:47.801 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 1.75, 2024-09-21T15:00:47.801 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 1.75, 2024-09-21T15:00:47.801 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 1, 2024-09-21T15:00:47.801 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 1.75, 2024-09-21T15:00:47.801 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 1.75, 2024-09-21T15:00:47.801 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 0.95859527587890625, 2024-09-21T15:00:47.801 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 0.95999997854232788, 2024-09-21T15:00:47.801 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 0.95999997854232788 2024-09-21T15:00:47.801 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:47.801 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:47.801 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-21T15:00:47.801 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2024-09-21T15:00:47.801 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.801 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-09-21T15:00:47.801 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "a4a73316-3a94-42e9-9a51-deb46a66bb06", 2024-09-21T15:00:47.802 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-21T15:00:47.802 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-21T15:00:47.802 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-21T15:00:47.802 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-21T15:00:47.802 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-21T15:00:47.802 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-21T15:00:47.802 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2024-09-21T15:00:47.802 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 93, 2024-09-21T15:00:47.802 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-21T15:00:47.802 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-21T15:00:47.802 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-21T15:00:47.802 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:47.802 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.802 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:47.802 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6808", 2024-09-21T15:00:47.802 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4016672243 2024-09-21T15:00:47.803 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.803 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.803 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:47.803 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6809", 2024-09-21T15:00:47.803 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4016672243 2024-09-21T15:00:47.803 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:47.803 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:47.803 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.803 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-21T15:00:47.803 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:47.803 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.803 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:47.803 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6810", 2024-09-21T15:00:47.803 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4016672243 2024-09-21T15:00:47.804 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.804 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.804 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:47.804 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6811", 2024-09-21T15:00:47.804 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4016672243 2024-09-21T15:00:47.804 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:47.804 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:47.804 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.804 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-21T15:00:47.804 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:47.804 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.804 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:47.804 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6814", 2024-09-21T15:00:47.804 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4016672243 2024-09-21T15:00:47.804 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.805 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.805 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:47.805 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6815", 2024-09-21T15:00:47.805 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4016672243 2024-09-21T15:00:47.805 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:47.805 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:47.805 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.805 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-21T15:00:47.805 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:47.805 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.805 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:47.805 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6812", 2024-09-21T15:00:47.805 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4016672243 2024-09-21T15:00:47.805 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.805 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.806 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:47.806 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6813", 2024-09-21T15:00:47.806 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4016672243 2024-09-21T15:00:47.806 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:47.806 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:47.806 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.806 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6809/4016672243", 2024-09-21T15:00:47.806 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6811/4016672243", 2024-09-21T15:00:47.806 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6815/4016672243", 2024-09-21T15:00:47.806 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6813/4016672243", 2024-09-21T15:00:47.806 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-21T15:00:47.806 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-21T15:00:47.806 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-21T15:00:47.806 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:47.807 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.807 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.807 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-09-21T15:00:47.807 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "006757a0-94d1-4a9b-aef8-73634cc5828e", 2024-09-21T15:00:47.807 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-21T15:00:47.807 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-21T15:00:47.807 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-21T15:00:47.807 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-21T15:00:47.807 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-21T15:00:47.807 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-21T15:00:47.807 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2024-09-21T15:00:47.807 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 140, 2024-09-21T15:00:47.807 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-21T15:00:47.807 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-21T15:00:47.807 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-21T15:00:47.807 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:47.808 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.808 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:47.808 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6824", 2024-09-21T15:00:47.808 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 641090577 2024-09-21T15:00:47.808 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.808 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.808 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:47.808 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6825", 2024-09-21T15:00:47.808 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 641090577 2024-09-21T15:00:47.808 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:47.808 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:47.808 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.808 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-21T15:00:47.808 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:47.808 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.809 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:47.809 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6826", 2024-09-21T15:00:47.809 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 641090577 2024-09-21T15:00:47.809 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.809 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.809 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:47.809 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6827", 2024-09-21T15:00:47.809 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 641090577 2024-09-21T15:00:47.809 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:47.809 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:47.809 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.809 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-21T15:00:47.809 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:47.809 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.809 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:47.810 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6830", 2024-09-21T15:00:47.810 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 641090577 2024-09-21T15:00:47.810 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.810 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.810 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:47.810 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6831", 2024-09-21T15:00:47.810 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 641090577 2024-09-21T15:00:47.810 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:47.810 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:47.810 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.810 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-21T15:00:47.810 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:47.810 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.810 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:47.810 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6828", 2024-09-21T15:00:47.811 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 641090577 2024-09-21T15:00:47.811 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.811 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.811 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:47.811 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6829", 2024-09-21T15:00:47.811 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 641090577 2024-09-21T15:00:47.811 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:47.811 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:47.811 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.811 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6825/641090577", 2024-09-21T15:00:47.811 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6827/641090577", 2024-09-21T15:00:47.811 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6831/641090577", 2024-09-21T15:00:47.811 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6829/641090577", 2024-09-21T15:00:47.811 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-21T15:00:47.812 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-21T15:00:47.812 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-21T15:00:47.812 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:47.812 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.812 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.812 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-09-21T15:00:47.812 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "0b499a23-820e-4e6c-a558-adbf88b7014f", 2024-09-21T15:00:47.812 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-21T15:00:47.812 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-21T15:00:47.812 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 0.3514556884765625, 2024-09-21T15:00:47.812 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-21T15:00:47.812 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-21T15:00:47.812 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-21T15:00:47.812 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2024-09-21T15:00:47.812 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 51, 2024-09-21T15:00:47.813 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-21T15:00:47.813 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-21T15:00:47.813 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-21T15:00:47.813 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:47.813 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.813 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:47.813 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6800", 2024-09-21T15:00:47.813 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2609154414 2024-09-21T15:00:47.813 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.813 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.813 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:47.813 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6801", 2024-09-21T15:00:47.813 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2609154414 2024-09-21T15:00:47.813 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:47.813 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:47.814 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.814 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-21T15:00:47.814 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:47.814 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.814 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:47.814 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6802", 2024-09-21T15:00:47.814 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2609154414 2024-09-21T15:00:47.814 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.814 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.814 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:47.814 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6803", 2024-09-21T15:00:47.814 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2609154414 2024-09-21T15:00:47.814 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:47.814 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:47.814 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.815 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-21T15:00:47.815 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:47.815 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.815 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:47.815 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6806", 2024-09-21T15:00:47.815 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2609154414 2024-09-21T15:00:47.815 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.815 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.815 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:47.815 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6807", 2024-09-21T15:00:47.815 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2609154414 2024-09-21T15:00:47.815 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:47.815 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:47.815 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.815 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-21T15:00:47.816 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:47.816 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.816 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:47.816 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6804", 2024-09-21T15:00:47.816 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2609154414 2024-09-21T15:00:47.816 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.816 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.816 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:47.816 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6805", 2024-09-21T15:00:47.816 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2609154414 2024-09-21T15:00:47.816 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:47.816 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:47.816 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.816 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6801/2609154414", 2024-09-21T15:00:47.816 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6803/2609154414", 2024-09-21T15:00:47.816 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6807/2609154414", 2024-09-21T15:00:47.817 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6805/2609154414", 2024-09-21T15:00:47.817 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-21T15:00:47.817 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-21T15:00:47.817 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-21T15:00:47.817 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:47.817 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.817 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.817 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-09-21T15:00:47.817 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "5a57b3dd-4a07-45b7-b159-3cac4c7eb262", 2024-09-21T15:00:47.817 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-21T15:00:47.817 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-21T15:00:47.817 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-21T15:00:47.817 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 0.833831787109375, 2024-09-21T15:00:47.817 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-21T15:00:47.818 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-21T15:00:47.818 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2024-09-21T15:00:47.818 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 94, 2024-09-21T15:00:47.818 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-21T15:00:47.818 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-21T15:00:47.818 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-21T15:00:47.818 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:47.818 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.818 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:47.818 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6816", 2024-09-21T15:00:47.818 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1462563376 2024-09-21T15:00:47.818 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.818 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.818 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:47.819 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6817", 2024-09-21T15:00:47.819 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1462563376 2024-09-21T15:00:47.819 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:47.819 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:47.819 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.819 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-21T15:00:47.819 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:47.819 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.819 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:47.819 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6818", 2024-09-21T15:00:47.819 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1462563376 2024-09-21T15:00:47.819 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.819 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.819 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:47.819 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6819", 2024-09-21T15:00:47.820 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1462563376 2024-09-21T15:00:47.820 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:47.820 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:47.820 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.820 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-21T15:00:47.820 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:47.820 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.820 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:47.820 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6822", 2024-09-21T15:00:47.820 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1462563376 2024-09-21T15:00:47.820 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.820 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.820 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:47.820 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6823", 2024-09-21T15:00:47.821 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1462563376 2024-09-21T15:00:47.821 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:47.821 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:47.821 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.821 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-21T15:00:47.821 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:47.821 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.821 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:47.821 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6820", 2024-09-21T15:00:47.821 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1462563376 2024-09-21T15:00:47.821 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.821 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.821 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:47.821 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6821", 2024-09-21T15:00:47.822 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1462563376 2024-09-21T15:00:47.822 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:47.822 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:47.822 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.822 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6817/1462563376", 2024-09-21T15:00:47.822 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6819/1462563376", 2024-09-21T15:00:47.822 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6823/1462563376", 2024-09-21T15:00:47.822 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6821/1462563376", 2024-09-21T15:00:47.822 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-21T15:00:47.822 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-21T15:00:47.822 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-21T15:00:47.822 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:47.822 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.823 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.823 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-09-21T15:00:47.823 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "39a00807-7517-474e-8842-623dd7093cc5", 2024-09-21T15:00:47.823 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-21T15:00:47.823 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-21T15:00:47.823 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-21T15:00:47.823 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-21T15:00:47.823 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-21T15:00:47.823 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-21T15:00:47.823 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2024-09-21T15:00:47.823 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 147, 2024-09-21T15:00:47.823 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-21T15:00:47.823 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-21T15:00:47.823 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-21T15:00:47.823 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:47.824 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.824 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:47.824 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6801", 2024-09-21T15:00:47.824 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 144412042 2024-09-21T15:00:47.824 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.824 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.824 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:47.824 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6803", 2024-09-21T15:00:47.824 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 144412042 2024-09-21T15:00:47.824 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:47.824 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:47.824 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.824 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-21T15:00:47.824 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:47.824 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.825 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:47.825 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6805", 2024-09-21T15:00:47.825 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 144412042 2024-09-21T15:00:47.825 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.825 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.825 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:47.825 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6807", 2024-09-21T15:00:47.825 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 144412042 2024-09-21T15:00:47.825 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:47.825 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:47.825 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.825 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-21T15:00:47.825 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:47.825 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.826 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:47.826 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6812", 2024-09-21T15:00:47.826 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 144412042 2024-09-21T15:00:47.826 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.826 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.826 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:47.826 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6814", 2024-09-21T15:00:47.826 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 144412042 2024-09-21T15:00:47.826 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:47.826 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:47.826 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.826 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-21T15:00:47.826 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:47.826 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.827 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:47.827 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6809", 2024-09-21T15:00:47.827 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 144412042 2024-09-21T15:00:47.827 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.827 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.827 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:47.827 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6811", 2024-09-21T15:00:47.827 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 144412042 2024-09-21T15:00:47.827 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:47.827 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:47.827 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.827 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.46:6803/144412042", 2024-09-21T15:00:47.827 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.46:6807/144412042", 2024-09-21T15:00:47.827 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.46:6814/144412042", 2024-09-21T15:00:47.827 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.46:6811/144412042", 2024-09-21T15:00:47.828 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-21T15:00:47.828 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-21T15:00:47.828 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-21T15:00:47.828 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:47.828 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.828 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.828 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-09-21T15:00:47.828 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "edf9f045-ddeb-4504-a0fc-1a0e4e853550", 2024-09-21T15:00:47.828 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-21T15:00:47.828 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-21T15:00:47.828 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-21T15:00:47.828 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-21T15:00:47.828 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-21T15:00:47.828 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-21T15:00:47.829 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2024-09-21T15:00:47.829 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 146, 2024-09-21T15:00:47.829 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-21T15:00:47.829 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-21T15:00:47.829 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-21T15:00:47.829 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:47.829 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.829 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:47.829 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6800", 2024-09-21T15:00:47.829 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 849101442 2024-09-21T15:00:47.829 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.829 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.829 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:47.829 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6802", 2024-09-21T15:00:47.829 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 849101442 2024-09-21T15:00:47.830 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:47.830 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:47.830 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.830 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-21T15:00:47.830 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:47.830 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.830 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:47.830 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6804", 2024-09-21T15:00:47.830 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 849101442 2024-09-21T15:00:47.830 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.830 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.830 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:47.830 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6806", 2024-09-21T15:00:47.830 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 849101442 2024-09-21T15:00:47.831 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:47.831 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:47.831 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.831 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-21T15:00:47.831 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:47.831 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.831 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:47.831 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6813", 2024-09-21T15:00:47.831 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 849101442 2024-09-21T15:00:47.831 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.831 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.831 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:47.831 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6815", 2024-09-21T15:00:47.831 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 849101442 2024-09-21T15:00:47.832 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:47.832 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:47.832 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.832 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-21T15:00:47.832 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:47.832 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.832 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:47.832 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6808", 2024-09-21T15:00:47.832 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 849101442 2024-09-21T15:00:47.832 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.832 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.832 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:47.832 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6810", 2024-09-21T15:00:47.833 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 849101442 2024-09-21T15:00:47.833 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:47.833 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:47.833 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.833 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.46:6802/849101442", 2024-09-21T15:00:47.833 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.46:6806/849101442", 2024-09-21T15:00:47.833 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.46:6815/849101442", 2024-09-21T15:00:47.833 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.46:6810/849101442", 2024-09-21T15:00:47.833 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-21T15:00:47.833 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-21T15:00:47.833 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-21T15:00:47.833 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:47.833 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.833 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.834 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-09-21T15:00:47.834 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "c820ea6b-88fe-4421-83ec-17c466691fea", 2024-09-21T15:00:47.834 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-21T15:00:47.834 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-21T15:00:47.834 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-21T15:00:47.834 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 0.834930419921875, 2024-09-21T15:00:47.834 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-21T15:00:47.834 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-21T15:00:47.834 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2024-09-21T15:00:47.834 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 123, 2024-09-21T15:00:47.834 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-21T15:00:47.834 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-21T15:00:47.834 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-21T15:00:47.834 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:47.835 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.835 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:47.835 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6817", 2024-09-21T15:00:47.835 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 847189172 2024-09-21T15:00:47.835 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.835 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.835 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:47.835 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6818", 2024-09-21T15:00:47.835 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 847189172 2024-09-21T15:00:47.835 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:47.835 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:47.835 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.835 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-21T15:00:47.835 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:47.835 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.836 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:47.836 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6820", 2024-09-21T15:00:47.836 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 847189172 2024-09-21T15:00:47.836 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.836 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.836 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:47.836 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6822", 2024-09-21T15:00:47.836 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 847189172 2024-09-21T15:00:47.836 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:47.836 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:47.836 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.836 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-21T15:00:47.836 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:47.836 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.836 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:47.837 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6827", 2024-09-21T15:00:47.837 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 847189172 2024-09-21T15:00:47.837 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.837 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.837 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:47.837 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6828", 2024-09-21T15:00:47.837 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 847189172 2024-09-21T15:00:47.837 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:47.837 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:47.837 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.837 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-21T15:00:47.837 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:47.837 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.837 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:47.838 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6823", 2024-09-21T15:00:47.838 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 847189172 2024-09-21T15:00:47.838 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.838 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.838 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:47.838 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6825", 2024-09-21T15:00:47.838 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 847189172 2024-09-21T15:00:47.838 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:47.838 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:47.838 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.838 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.46:6818/847189172", 2024-09-21T15:00:47.838 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.46:6822/847189172", 2024-09-21T15:00:47.838 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.46:6828/847189172", 2024-09-21T15:00:47.838 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.46:6825/847189172", 2024-09-21T15:00:47.839 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-21T15:00:47.839 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-21T15:00:47.839 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-21T15:00:47.839 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:47.839 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.839 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.839 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-09-21T15:00:47.839 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "3a2f82a4-28a0-4fcb-9d6b-2285fa4d07ac", 2024-09-21T15:00:47.839 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-21T15:00:47.839 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-21T15:00:47.839 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 0.95001220703125, 2024-09-21T15:00:47.839 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-21T15:00:47.839 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-21T15:00:47.839 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-21T15:00:47.839 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 15, 2024-09-21T15:00:47.840 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 81, 2024-09-21T15:00:47.840 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-21T15:00:47.840 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-21T15:00:47.840 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-21T15:00:47.840 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:47.840 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.840 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:47.840 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6816", 2024-09-21T15:00:47.840 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3894963553 2024-09-21T15:00:47.840 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.840 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.840 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:47.840 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6819", 2024-09-21T15:00:47.840 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3894963553 2024-09-21T15:00:47.841 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:47.841 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:47.841 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.841 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-21T15:00:47.841 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:47.841 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.841 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:47.841 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6821", 2024-09-21T15:00:47.841 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3894963553 2024-09-21T15:00:47.841 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.841 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.841 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:47.841 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6824", 2024-09-21T15:00:47.841 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3894963553 2024-09-21T15:00:47.842 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:47.842 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:47.842 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.842 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-21T15:00:47.842 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:47.842 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.842 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:47.842 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6830", 2024-09-21T15:00:47.842 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3894963553 2024-09-21T15:00:47.842 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.842 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.842 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:47.842 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6831", 2024-09-21T15:00:47.842 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3894963553 2024-09-21T15:00:47.843 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:47.843 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:47.843 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.843 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-21T15:00:47.843 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:00:47.843 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.843 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:00:47.843 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6826", 2024-09-21T15:00:47.843 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3894963553 2024-09-21T15:00:47.843 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.843 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.843 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:00:47.843 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6829", 2024-09-21T15:00:47.843 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3894963553 2024-09-21T15:00:47.844 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:47.844 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:47.844 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.844 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.46:6819/3894963553", 2024-09-21T15:00:47.844 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.46:6824/3894963553", 2024-09-21T15:00:47.844 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.46:6831/3894963553", 2024-09-21T15:00:47.844 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.46:6829/3894963553", 2024-09-21T15:00:47.844 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-21T15:00:47.844 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-21T15:00:47.844 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-21T15:00:47.844 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:47.844 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:47.844 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-21T15:00:47.844 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_xinfo": [ 2024-09-21T15:00:47.844 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.845 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-09-21T15:00:47.845 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-21T15:00:47.845 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-21T15:00:47.845 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-21T15:00:47.845 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-21T15:00:47.845 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-21T15:00:47.845 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-21T14:57:57.458317+0000", 2024-09-21T15:00:47.845 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-21T15:00:47.845 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.845 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.845 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-09-21T15:00:47.845 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-21T15:00:47.845 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-21T15:00:47.845 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-21T15:00:47.846 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-21T15:00:47.846 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-21T15:00:47.846 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-21T14:57:57.907624+0000", 2024-09-21T15:00:47.846 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-21T15:00:47.846 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.846 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.846 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-09-21T15:00:47.846 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-21T15:00:47.846 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-21T15:00:47.846 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-21T15:00:47.846 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-21T15:00:47.846 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-21T15:00:47.846 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-21T14:57:57.278817+0000", 2024-09-21T15:00:47.847 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-21T15:00:47.847 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.847 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.847 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-09-21T15:00:47.847 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-21T15:00:47.847 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-21T15:00:47.847 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-21T15:00:47.847 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-21T15:00:47.847 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-21T15:00:47.847 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-21T14:57:57.500449+0000", 2024-09-21T15:00:47.847 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-21T15:00:47.847 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.847 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.847 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-09-21T15:00:47.847 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-21T15:00:47.848 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-21T15:00:47.848 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-21T15:00:47.848 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-21T15:00:47.848 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-21T15:00:47.848 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-21T14:57:57.504325+0000", 2024-09-21T15:00:47.848 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-21T15:00:47.848 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.848 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.848 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-09-21T15:00:47.848 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-21T15:00:47.848 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-21T15:00:47.848 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-21T15:00:47.848 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-21T15:00:47.848 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-21T15:00:47.849 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-21T14:57:57.777525+0000", 2024-09-21T15:00:47.849 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-21T15:00:47.849 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.849 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.849 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-09-21T15:00:47.849 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-21T15:00:47.849 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-21T15:00:47.849 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-21T15:00:47.849 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-21T15:00:47.849 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-21T15:00:47.849 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-21T14:57:57.698961+0000", 2024-09-21T15:00:47.849 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-21T15:00:47.849 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.849 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.850 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-09-21T15:00:47.850 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-21T15:00:47.850 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-21T15:00:47.850 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-21T15:00:47.850 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-21T15:00:47.850 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-21T15:00:47.850 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-21T14:57:57.932625+0000", 2024-09-21T15:00:47.850 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-21T15:00:47.850 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:47.850 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-21T15:00:47.850 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap": [ 2024-09-21T15:00:47.850 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.850 INFO:teuthology.orchestra.run.smithi032.stdout: "pgid": "3.2", 2024-09-21T15:00:47.850 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2024-09-21T15:00:47.851 INFO:teuthology.orchestra.run.smithi032.stdout: 2, 2024-09-21T15:00:47.851 INFO:teuthology.orchestra.run.smithi032.stdout: 3 2024-09-21T15:00:47.851 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:47.851 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.851 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.851 INFO:teuthology.orchestra.run.smithi032.stdout: "pgid": "3.4", 2024-09-21T15:00:47.851 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2024-09-21T15:00:47.851 INFO:teuthology.orchestra.run.smithi032.stdout: 4, 2024-09-21T15:00:47.851 INFO:teuthology.orchestra.run.smithi032.stdout: 2 2024-09-21T15:00:47.851 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:47.851 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.851 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.851 INFO:teuthology.orchestra.run.smithi032.stdout: "pgid": "3.a", 2024-09-21T15:00:47.851 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2024-09-21T15:00:47.852 INFO:teuthology.orchestra.run.smithi032.stdout: 1, 2024-09-21T15:00:47.852 INFO:teuthology.orchestra.run.smithi032.stdout: 0 2024-09-21T15:00:47.852 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:47.852 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:47.852 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-21T15:00:47.852 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_items": [], 2024-09-21T15:00:47.852 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_primaries": [], 2024-09-21T15:00:47.852 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_temp": [], 2024-09-21T15:00:47.852 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_temp": [], 2024-09-21T15:00:47.852 INFO:teuthology.orchestra.run.smithi032.stdout: "blocklist": {}, 2024-09-21T15:00:47.852 INFO:teuthology.orchestra.run.smithi032.stdout: "range_blocklist": {}, 2024-09-21T15:00:47.852 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profiles": { 2024-09-21T15:00:47.852 INFO:teuthology.orchestra.run.smithi032.stdout: "default": { 2024-09-21T15:00:47.852 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2024-09-21T15:00:47.853 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2024-09-21T15:00:47.853 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "1", 2024-09-21T15:00:47.853 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2024-09-21T15:00:47.853 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van" 2024-09-21T15:00:47.853 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:47.853 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.853 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps_queue": [ 2024-09-21T15:00:47.853 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.853 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 3, 2024-09-21T15:00:47.853 INFO:teuthology.orchestra.run.smithi032.stdout: "snaps": [ 2024-09-21T15:00:47.853 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.853 INFO:teuthology.orchestra.run.smithi032.stdout: "begin": 95, 2024-09-21T15:00:47.853 INFO:teuthology.orchestra.run.smithi032.stdout: "length": 1 2024-09-21T15:00:47.853 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:47.853 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:47.854 INFO:teuthology.orchestra.run.smithi032.stdout: "begin": 98, 2024-09-21T15:00:47.854 INFO:teuthology.orchestra.run.smithi032.stdout: "length": 1 2024-09-21T15:00:47.854 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:47.854 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:47.854 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:47.854 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-21T15:00:47.854 INFO:teuthology.orchestra.run.smithi032.stdout: "new_removed_snaps": [], 2024-09-21T15:00:47.854 INFO:teuthology.orchestra.run.smithi032.stdout: "new_purged_snaps": [], 2024-09-21T15:00:47.854 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_node_flags": {}, 2024-09-21T15:00:47.854 INFO:teuthology.orchestra.run.smithi032.stdout: "device_class_flags": {}, 2024-09-21T15:00:47.854 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode": { 2024-09-21T15:00:47.854 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_enabled": false, 2024-09-21T15:00:47.854 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_bucket_count": 0, 2024-09-21T15:00:47.854 INFO:teuthology.orchestra.run.smithi032.stdout: "degraded_stretch_mode": 0, 2024-09-21T15:00:47.855 INFO:teuthology.orchestra.run.smithi032.stdout: "recovering_stretch_mode": 0, 2024-09-21T15:00:47.855 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_bucket": 0 2024-09-21T15:00:47.855 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:47.855 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:47.856 INFO:tasks.thrashosds.thrasher:j is {'epoch': 160, 'fsid': 'caf51e55-9234-4a26-bcd2-d60811962560', 'created': '2024-09-21T14:57:50.338822+0000', 'modified': '2024-09-21T15:00:47.422685+0000', 'last_up_change': '2024-09-21T14:58:01.177141+0000', 'last_in_change': '2024-09-21T14:59:27.207073+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T14:58:01.842810+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.9585952758789062, 'average_primary_affinity': 0.9599999785423279, 'average_primary_affinity_weighted': 0.9599999785423279}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-21T14:58:08.284925+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 22, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 0.9585952758789062, 'average_primary_affinity': 0.9599999785423279, 'average_primary_affinity_weighted': 0.9599999785423279}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-21T14:58:17.510709+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '159', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 99, 'snap_epoch': 159, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.9585952758789062, 'average_primary_affinity': 0.9599999785423279, 'average_primary_affinity_weighted': 0.9599999785423279}}], 'osds': [{'osd': 0, 'uuid': 'a4a73316-3a94-42e9-9a51-deb46a66bb06', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 93, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6808', 'nonce': 4016672243}, {'type': 'v1', 'addr': '172.21.15.32:6809', 'nonce': 4016672243}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6810', 'nonce': 4016672243}, {'type': 'v1', 'addr': '172.21.15.32:6811', 'nonce': 4016672243}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6814', 'nonce': 4016672243}, {'type': 'v1', 'addr': '172.21.15.32:6815', 'nonce': 4016672243}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6812', 'nonce': 4016672243}, {'type': 'v1', 'addr': '172.21.15.32:6813', 'nonce': 4016672243}]}, 'public_addr': '172.21.15.32:6809/4016672243', 'cluster_addr': '172.21.15.32:6811/4016672243', 'heartbeat_back_addr': '172.21.15.32:6815/4016672243', 'heartbeat_front_addr': '172.21.15.32:6813/4016672243', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '006757a0-94d1-4a9b-aef8-73634cc5828e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 140, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6824', 'nonce': 641090577}, {'type': 'v1', 'addr': '172.21.15.32:6825', 'nonce': 641090577}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6826', 'nonce': 641090577}, {'type': 'v1', 'addr': '172.21.15.32:6827', 'nonce': 641090577}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6830', 'nonce': 641090577}, {'type': 'v1', 'addr': '172.21.15.32:6831', 'nonce': 641090577}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6828', 'nonce': 641090577}, {'type': 'v1', 'addr': '172.21.15.32:6829', 'nonce': 641090577}]}, 'public_addr': '172.21.15.32:6825/641090577', 'cluster_addr': '172.21.15.32:6827/641090577', 'heartbeat_back_addr': '172.21.15.32:6831/641090577', 'heartbeat_front_addr': '172.21.15.32:6829/641090577', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '0b499a23-820e-4e6c-a558-adbf88b7014f', 'up': 1, 'in': 1, 'weight': 0.3514556884765625, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 51, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6800', 'nonce': 2609154414}, {'type': 'v1', 'addr': '172.21.15.32:6801', 'nonce': 2609154414}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6802', 'nonce': 2609154414}, {'type': 'v1', 'addr': '172.21.15.32:6803', 'nonce': 2609154414}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6806', 'nonce': 2609154414}, {'type': 'v1', 'addr': '172.21.15.32:6807', 'nonce': 2609154414}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6804', 'nonce': 2609154414}, {'type': 'v1', 'addr': '172.21.15.32:6805', 'nonce': 2609154414}]}, 'public_addr': '172.21.15.32:6801/2609154414', 'cluster_addr': '172.21.15.32:6803/2609154414', 'heartbeat_back_addr': '172.21.15.32:6807/2609154414', 'heartbeat_front_addr': '172.21.15.32:6805/2609154414', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '5a57b3dd-4a07-45b7-b159-3cac4c7eb262', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.833831787109375, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 94, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6816', 'nonce': 1462563376}, {'type': 'v1', 'addr': '172.21.15.32:6817', 'nonce': 1462563376}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6818', 'nonce': 1462563376}, {'type': 'v1', 'addr': '172.21.15.32:6819', 'nonce': 1462563376}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6822', 'nonce': 1462563376}, {'type': 'v1', 'addr': '172.21.15.32:6823', 'nonce': 1462563376}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6820', 'nonce': 1462563376}, {'type': 'v1', 'addr': '172.21.15.32:6821', 'nonce': 1462563376}]}, 'public_addr': '172.21.15.32:6817/1462563376', 'cluster_addr': '172.21.15.32:6819/1462563376', 'heartbeat_back_addr': '172.21.15.32:6823/1462563376', 'heartbeat_front_addr': '172.21.15.32:6821/1462563376', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '39a00807-7517-474e-8842-623dd7093cc5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 147, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6801', 'nonce': 144412042}, {'type': 'v1', 'addr': '172.21.15.46:6803', 'nonce': 144412042}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6805', 'nonce': 144412042}, {'type': 'v1', 'addr': '172.21.15.46:6807', 'nonce': 144412042}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6812', 'nonce': 144412042}, {'type': 'v1', 'addr': '172.21.15.46:6814', 'nonce': 144412042}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6809', 'nonce': 144412042}, {'type': 'v1', 'addr': '172.21.15.46:6811', 'nonce': 144412042}]}, 'public_addr': '172.21.15.46:6803/144412042', 'cluster_addr': '172.21.15.46:6807/144412042', 'heartbeat_back_addr': '172.21.15.46:6814/144412042', 'heartbeat_front_addr': '172.21.15.46:6811/144412042', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'edf9f045-ddeb-4504-a0fc-1a0e4e853550', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 146, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6800', 'nonce': 849101442}, {'type': 'v1', 'addr': '172.21.15.46:6802', 'nonce': 849101442}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6804', 'nonce': 849101442}, {'type': 'v1', 'addr': '172.21.15.46:6806', 'nonce': 849101442}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6813', 'nonce': 849101442}, {'type': 'v1', 'addr': '172.21.15.46:6815', 'nonce': 849101442}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6808', 'nonce': 849101442}, {'type': 'v1', 'addr': '172.21.15.46:6810', 'nonce': 849101442}]}, 'public_addr': '172.21.15.46:6802/849101442', 'cluster_addr': '172.21.15.46:6806/849101442', 'heartbeat_back_addr': '172.21.15.46:6815/849101442', 'heartbeat_front_addr': '172.21.15.46:6810/849101442', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'c820ea6b-88fe-4421-83ec-17c466691fea', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.834930419921875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 123, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6817', 'nonce': 847189172}, {'type': 'v1', 'addr': '172.21.15.46:6818', 'nonce': 847189172}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6820', 'nonce': 847189172}, {'type': 'v1', 'addr': '172.21.15.46:6822', 'nonce': 847189172}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6827', 'nonce': 847189172}, {'type': 'v1', 'addr': '172.21.15.46:6828', 'nonce': 847189172}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6823', 'nonce': 847189172}, {'type': 'v1', 'addr': '172.21.15.46:6825', 'nonce': 847189172}]}, 'public_addr': '172.21.15.46:6818/847189172', 'cluster_addr': '172.21.15.46:6822/847189172', 'heartbeat_back_addr': '172.21.15.46:6828/847189172', 'heartbeat_front_addr': '172.21.15.46:6825/847189172', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '3a2f82a4-28a0-4fcb-9d6b-2285fa4d07ac', 'up': 1, 'in': 1, 'weight': 0.95001220703125, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 81, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6816', 'nonce': 3894963553}, {'type': 'v1', 'addr': '172.21.15.46:6819', 'nonce': 3894963553}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6821', 'nonce': 3894963553}, {'type': 'v1', 'addr': '172.21.15.46:6824', 'nonce': 3894963553}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6830', 'nonce': 3894963553}, {'type': 'v1', 'addr': '172.21.15.46:6831', 'nonce': 3894963553}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6826', 'nonce': 3894963553}, {'type': 'v1', 'addr': '172.21.15.46:6829', 'nonce': 3894963553}]}, 'public_addr': '172.21.15.46:6819/3894963553', 'cluster_addr': '172.21.15.46:6824/3894963553', 'heartbeat_back_addr': '172.21.15.46:6831/3894963553', 'heartbeat_front_addr': '172.21.15.46:6829/3894963553', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:57:57.458317+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:57:57.907624+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:57:57.278817+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:57:57.500449+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:57:57.504325+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:57:57.777525+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:57:57.698961+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:57:57.932625+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.2', 'osds': [2, 3]}, {'pgid': '3.4', 'osds': [4, 2]}, {'pgid': '3.a', 'osds': [1, 0]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 95, 'length': 1}, {'begin': 98, 'length': 1}]}], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-21T15:00:47.856 INFO:tasks.thrashosds.thrasher:No pg_upmap entries; doing nothing 2024-09-21T15:00:47.856 INFO:teuthology.orchestra.run.smithi032.stdout:4881 trunc from 0xe9c3e11 to 0xd368644 2024-09-21T15:00:47.856 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:00:47.775+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:00:47.857 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:47.858 INFO:teuthology.orchestra.run.smithi032.stdout:4883 write 0xb300b5f thru 0xb308714 (0x7bb6 bytes) 2024-09-21T15:00:47.858 INFO:teuthology.orchestra.run.smithi032.stdout:4885 punch from 0x395ce5e to 0x3969080, (0xc222 bytes) 2024-09-21T15:00:47.858 INFO:teuthology.orchestra.run.smithi032.stdout:4887 write 0x7b5a847 thru 0x7b5fa28 (0x51e2 bytes) 2024-09-21T15:00:47.858 INFO:teuthology.orchestra.run.smithi032.stdout:4888 read 0x4bc3657 thru 0x4bc48c0 (0x126a bytes) 2024-09-21T15:00:47.858 INFO:teuthology.orchestra.run.smithi032.stdout:4889 trunc from 0xd368644 to 0xb9412eb 2024-09-21T15:00:47.858 INFO:teuthology.orchestra.run.smithi032.stdout:4890 write 0x331342 thru 0x338d06 (0x79c5 bytes) 2024-09-21T15:00:47.858 INFO:teuthology.orchestra.run.smithi032.stdout:4891 write 0xe9fb7f1 thru 0xea0ac29 (0xf439 bytes) 2024-09-21T15:00:47.865 INFO:teuthology.orchestra.run.smithi032.stdout:4893 writesame 0xc5834d3 thru 0xc583600 data_size 0x1(0x12e bytes) 2024-09-21T15:00:47.879 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:00:47.877+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:00:47.902 INFO:teuthology.orchestra.run.smithi032.stdout:4894 trunc from 0xea0ac2a to 0x971e34 2024-09-21T15:00:47.939 INFO:teuthology.orchestra.run.smithi032.stdout:4895 write 0x30d07b7 thru 0x30d1006 (0x850 bytes) 2024-09-21T15:00:47.958 INFO:teuthology.orchestra.run.smithi032.stdout:4896 trunc from 0x30d1007 to 0xb6d51ec 2024-09-21T15:00:47.967 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:47.970 INFO:teuthology.orchestra.run.smithi032.stdout:4898 trunc from 0xb6d51ec to 0xd06f33c 2024-09-21T15:00:47.978 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:47.978 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:00:47.978 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T15:00:47.978 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:47.979 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:00:47.975+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:00:47.981 INFO:teuthology.orchestra.run.smithi032.stdout:4899 read 0x124872e thru 0x125827f (0xfb52 bytes) 2024-09-21T15:00:47.981 INFO:teuthology.orchestra.run.smithi032.stdout:4901 read 0xacc9ed3 thru 0xaccc632 (0x2760 bytes) 2024-09-21T15:00:47.981 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:47.981 INFO:teuthology.orchestra.run.smithi032.stdout:4903 punch from 0x19a4be3 to 0x19b2e8a, (0xe2a7 bytes) 2024-09-21T15:00:47.981 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:47.981 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:47.981 INFO:teuthology.orchestra.run.smithi032.stdout:4906 punch from 0x68e6603 to 0x68efc73, (0x9670 bytes) 2024-09-21T15:00:47.981 INFO:teuthology.orchestra.run.smithi032.stdout:4909 write 0x2859da8 thru 0x285f1dc (0x5435 bytes) 2024-09-21T15:00:47.987 INFO:teuthology.orchestra.run.smithi032.stdout:4910 write 0xb9fbc75 thru 0xba021ad (0x6539 bytes) 2024-09-21T15:00:47.989 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T15:00:47.996 INFO:teuthology.orchestra.run.smithi032.stdout:4911 punch from 0x3ca7953 to 0x3cac2b2, (0x495f bytes) 2024-09-21T15:00:47.996 INFO:teuthology.orchestra.run.smithi032.stdout:4912 punch from 0x99f0e51 to 0x99f69c2, (0x5b71 bytes) 2024-09-21T15:00:47.996 INFO:teuthology.orchestra.run.smithi032.stdout:4913 punch from 0x952ec74 to 0x953c6ad, (0xda39 bytes) 2024-09-21T15:00:47.996 INFO:teuthology.orchestra.run.smithi032.stdout:4914 punch from 0xcafb4a3 to 0xcafcd0b, (0x1868 bytes) 2024-09-21T15:00:47.996 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:47.996 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:47.996 INFO:teuthology.orchestra.run.smithi032.stdout:4917 punch from 0x200e242 to 0x2015251, (0x700f bytes) 2024-09-21T15:00:47.996 INFO:teuthology.orchestra.run.smithi032.stdout:4918 write 0xa26e57d thru 0xa273a98 (0x551c bytes) 2024-09-21T15:00:48.000 INFO:teuthology.orchestra.run.smithi032.stdout:4920 read 0x9ee0dd5 thru 0x9eee186 (0xd3b2 bytes) 2024-09-21T15:00:48.016 INFO:teuthology.orchestra.run.smithi032.stdout:4923 trunc from 0xd06f33c to 0x3e77abf 2024-09-21T15:00:48.033 INFO:teuthology.orchestra.run.smithi032.stdout:4924 clone 33 order 19 su 32768 sc 10 2024-09-21T15:00:48.080 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:00:48.075+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:00:48.098 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:48.098 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:00:48.098 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T15:00:48.098 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T15:00:48.098 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:48.107 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T15:00:48.181 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:00:48.177+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:00:48.266 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:48.266 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T15:00:48.266 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T15:00:48.266 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T15:00:48.266 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.266 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T15:00:48.267 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.205754+0000", 2024-09-21T15:00:48.267 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 165.05792434099999, 2024-09-21T15:00:48.267 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.41117094199999998, 2024-09-21T15:00:48.267 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:48.267 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:48.267 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:48.267 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T15:00:48.267 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T15:00:48.267 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1 2024-09-21T15:00:48.267 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.267 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:48.267 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.267 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:48.267 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205754+0000", 2024-09-21T15:00:48.268 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:48.268 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.268 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.268 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:48.268 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205754+0000", 2024-09-21T15:00:48.268 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:48.268 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.268 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.268 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:48.268 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205752+0000", 2024-09-21T15:00:48.268 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T15:00:48.268 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.268 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.268 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:48.268 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205765+0000", 2024-09-21T15:00:48.269 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3230999999999999e-05 2024-09-21T15:00:48.269 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.269 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.269 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:48.269 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205782+0000", 2024-09-21T15:00:48.269 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6634e-05 2024-09-21T15:00:48.269 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.269 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.269 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:48.269 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205793+0000", 2024-09-21T15:00:48.269 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1297e-05 2024-09-21T15:00:48.269 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.269 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.269 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:48.269 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205867+0000", 2024-09-21T15:00:48.269 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4269000000000003e-05 2024-09-21T15:00:48.270 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.270 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.270 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:00:48.270 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205877+0000", 2024-09-21T15:00:48.270 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2690000000000005e-06 2024-09-21T15:00:48.270 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.270 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.270 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:48.270 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.613700+0000", 2024-09-21T15:00:48.270 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.407823455 2024-09-21T15:00:48.270 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.270 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.270 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:48.270 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.614203+0000", 2024-09-21T15:00:48.271 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00050311500000000005 2024-09-21T15:00:48.271 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.271 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.271 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T15:00:48.271 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.614750+0000", 2024-09-21T15:00:48.271 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00054738699999999998 2024-09-21T15:00:48.271 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.271 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.271 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:48.271 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616367+0000", 2024-09-21T15:00:48.271 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001616634 2024-09-21T15:00:48.271 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.271 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.271 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:48.272 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616574+0000", 2024-09-21T15:00:48.272 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00020668499999999999 2024-09-21T15:00:48.272 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.272 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.272 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:48.272 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616802+0000", 2024-09-21T15:00:48.272 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00022817900000000001 2024-09-21T15:00:48.272 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.272 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.272 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:48.272 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616925+0000", 2024-09-21T15:00:48.272 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012309699999999999 2024-09-21T15:00:48.272 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:48.272 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:48.273 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:48.273 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.273 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.273 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6639 3.1a 3:592d80ed:::rbd_data.11361bed1f78.0000000000000005:d [read 0~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T15:00:48.273 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131650+0000", 2024-09-21T15:00:48.273 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 133.13202879299999, 2024-09-21T15:00:48.273 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016002832000000002, 2024-09-21T15:00:48.273 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:48.273 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:48.273 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:48.273 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:48.273 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:48.273 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6639 2024-09-21T15:00:48.273 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.274 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:48.274 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.274 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:48.274 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131650+0000", 2024-09-21T15:00:48.274 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:48.274 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.274 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.274 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:48.274 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131650+0000", 2024-09-21T15:00:48.274 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:48.274 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.274 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.274 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:48.274 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131649+0000", 2024-09-21T15:00:48.275 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:48.275 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.275 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.275 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:48.275 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131651+0000", 2024-09-21T15:00:48.275 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9199999999999998e-06 2024-09-21T15:00:48.275 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.275 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.275 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:48.275 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131656+0000", 2024-09-21T15:00:48.275 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9899999999999997e-06 2024-09-21T15:00:48.275 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.275 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.275 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:48.276 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131665+0000", 2024-09-21T15:00:48.276 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0999999999999993e-06 2024-09-21T15:00:48.276 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.276 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.276 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:48.276 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146309+0000", 2024-09-21T15:00:48.276 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014643893999999999 2024-09-21T15:00:48.276 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.276 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.276 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:48.276 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.146392+0000", 2024-09-21T15:00:48.276 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2531e-05 2024-09-21T15:00:48.276 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.276 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.276 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:48.277 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147653+0000", 2024-09-21T15:00:48.277 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0012606710000000001 2024-09-21T15:00:48.277 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:48.277 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:48.277 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:48.277 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.277 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.277 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6640 3.1a 3:592d80ed:::rbd_data.11361bed1f78.0000000000000005:d [read 65536~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T15:00:48.277 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131670+0000", 2024-09-21T15:00:48.277 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 133.13200849399999, 2024-09-21T15:00:48.277 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016188859, 2024-09-21T15:00:48.277 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:48.277 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:48.277 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:48.277 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:48.277 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:48.278 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6640 2024-09-21T15:00:48.278 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.278 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:48.278 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.278 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:48.278 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131670+0000", 2024-09-21T15:00:48.278 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:48.278 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.278 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.278 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:48.278 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131670+0000", 2024-09-21T15:00:48.278 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:48.278 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.278 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.279 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:48.279 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131670+0000", 2024-09-21T15:00:48.279 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:00:48.279 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.279 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.279 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:48.279 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131672+0000", 2024-09-21T15:00:48.279 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.818e-06 2024-09-21T15:00:48.279 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.279 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.279 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:48.279 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131675+0000", 2024-09-21T15:00:48.279 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5930000000000002e-06 2024-09-21T15:00:48.279 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.279 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.279 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:48.280 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131678+0000", 2024-09-21T15:00:48.280 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6790000000000001e-06 2024-09-21T15:00:48.280 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.280 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.280 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:48.280 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147681+0000", 2024-09-21T15:00:48.280 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016002755 2024-09-21T15:00:48.280 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.280 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.280 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:48.280 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147762+0000", 2024-09-21T15:00:48.280 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1052999999999998e-05 2024-09-21T15:00:48.280 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.280 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.280 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:48.281 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147859+0000", 2024-09-21T15:00:48.281 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.7199000000000002e-05 2024-09-21T15:00:48.281 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:48.281 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:48.281 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:48.281 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.281 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.281 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6641 3.1a 3:592d80ed:::rbd_data.11361bed1f78.0000000000000005:d [read 147456~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T15:00:48.281 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131680+0000", 2024-09-21T15:00:48.281 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 133.13199820599999, 2024-09-21T15:00:48.281 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016431296000000001, 2024-09-21T15:00:48.281 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:48.281 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:48.281 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:48.281 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:48.281 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:48.282 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6641 2024-09-21T15:00:48.282 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.282 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:48.282 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.282 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:48.282 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131680+0000", 2024-09-21T15:00:48.282 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:48.282 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.282 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.282 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:48.282 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131680+0000", 2024-09-21T15:00:48.282 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:48.282 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.282 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.283 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:48.283 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131680+0000", 2024-09-21T15:00:48.283 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:00:48.283 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.283 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.283 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:48.283 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131681+0000", 2024-09-21T15:00:48.283 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2610000000000001e-06 2024-09-21T15:00:48.283 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.283 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.283 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:48.283 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131685+0000", 2024-09-21T15:00:48.283 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.8299999999999998e-06 2024-09-21T15:00:48.283 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.284 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.284 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:48.284 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131688+0000", 2024-09-21T15:00:48.284 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6189999999999998e-06 2024-09-21T15:00:48.284 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.284 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.284 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:48.284 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147884+0000", 2024-09-21T15:00:48.284 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016196465 2024-09-21T15:00:48.284 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.284 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.284 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:48.284 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.147949+0000", 2024-09-21T15:00:48.284 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4867000000000001e-05 2024-09-21T15:00:48.284 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.285 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.285 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:48.285 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.148112+0000", 2024-09-21T15:00:48.285 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00016247700000000001 2024-09-21T15:00:48.285 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:48.285 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:48.285 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:48.285 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.285 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.285 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:6642 3.1a 3:592d80ed:::rbd_data.11361bed1f78.0000000000000005:d [read 212992~32768 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-21T15:00:48.285 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:35.131690+0000", 2024-09-21T15:00:48.285 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 133.13198832800001, 2024-09-21T15:00:48.285 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016751335999999999, 2024-09-21T15:00:48.285 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:48.286 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:48.286 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:48.286 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:48.286 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:48.286 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 6642 2024-09-21T15:00:48.286 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.286 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:48.286 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.286 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:48.286 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131690+0000", 2024-09-21T15:00:48.286 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:48.286 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.286 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.286 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:48.286 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131690+0000", 2024-09-21T15:00:48.287 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:48.287 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.287 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.287 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:48.287 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131690+0000", 2024-09-21T15:00:48.287 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:00:48.287 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.287 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.287 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:48.287 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131691+0000", 2024-09-21T15:00:48.287 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3120000000000001e-06 2024-09-21T15:00:48.287 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.287 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.287 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:48.288 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131694+0000", 2024-09-21T15:00:48.288 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0070000000000002e-06 2024-09-21T15:00:48.288 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.288 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.288 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:48.288 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.131700+0000", 2024-09-21T15:00:48.288 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0800000000000002e-06 2024-09-21T15:00:48.288 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.288 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.288 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:48.288 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.148135+0000", 2024-09-21T15:00:48.288 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016434250000000001 2024-09-21T15:00:48.288 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.288 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.288 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:48.288 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.148204+0000", 2024-09-21T15:00:48.289 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9466000000000002e-05 2024-09-21T15:00:48.289 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.289 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.289 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:48.289 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:35.148441+0000", 2024-09-21T15:00:48.289 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00023743700000000001 2024-09-21T15:00:48.289 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:48.289 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:48.289 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:48.289 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.289 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.289 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:61214 3.7 3:e28aec10:::rbd_header.1136862f872f:head [watch reconnect cookie 139864671932080 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-21T15:00:48.289 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:18.147172+0000", 2024-09-21T15:00:48.289 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 90.116506697999995, 2024-09-21T15:00:48.289 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.019264956, 2024-09-21T15:00:48.290 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:48.290 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:48.290 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:48.290 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:48.290 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:48.290 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 61214 2024-09-21T15:00:48.290 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.290 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:48.290 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.290 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:48.290 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147172+0000", 2024-09-21T15:00:48.290 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:48.290 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.290 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.291 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:48.291 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147172+0000", 2024-09-21T15:00:48.291 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:48.291 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.291 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.291 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:48.291 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147168+0000", 2024-09-21T15:00:48.291 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999962 2024-09-21T15:00:48.291 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.291 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.291 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:48.291 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147188+0000", 2024-09-21T15:00:48.291 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9717000000000001e-05 2024-09-21T15:00:48.291 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.291 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.292 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:48.292 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147239+0000", 2024-09-21T15:00:48.292 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1493999999999999e-05 2024-09-21T15:00:48.292 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.292 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.292 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:48.292 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147265+0000", 2024-09-21T15:00:48.292 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5746999999999999e-05 2024-09-21T15:00:48.292 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.292 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.292 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:48.292 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147372+0000", 2024-09-21T15:00:48.292 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010681200000000001 2024-09-21T15:00:48.292 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.293 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.293 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op must wait for map", 2024-09-21T15:00:48.293 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147380+0000", 2024-09-21T15:00:48.293 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7919999999999993e-06 2024-09-21T15:00:48.293 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.293 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.293 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:48.293 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.154025+0000", 2024-09-21T15:00:48.293 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0066454909999999999 2024-09-21T15:00:48.293 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.293 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.293 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:00:48.293 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.154041+0000", 2024-09-21T15:00:48.293 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5894000000000001e-05 2024-09-21T15:00:48.293 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.294 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.294 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:48.294 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.164345+0000", 2024-09-21T15:00:48.294 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0103040459999999 2024-09-21T15:00:48.294 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.294 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.294 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:48.294 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.164590+0000", 2024-09-21T15:00:48.294 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00024492699999999998 2024-09-21T15:00:48.294 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.294 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.294 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:48.294 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.166437+0000", 2024-09-21T15:00:48.295 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0018466330000000001 2024-09-21T15:00:48.295 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:48.295 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:48.295 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:48.295 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.295 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.295 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:61215 3.7 3:e28aec10:::rbd_header.1136862f872f:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-21T15:00:48.295 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:18.152008+0000", 2024-09-21T15:00:48.295 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 90.111670246000003, 2024-09-21T15:00:48.295 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.015644958, 2024-09-21T15:00:48.295 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:48.295 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:48.295 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:48.295 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:48.295 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:48.296 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 61215 2024-09-21T15:00:48.296 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.296 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:48.296 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.296 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:48.296 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152008+0000", 2024-09-21T15:00:48.296 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:48.296 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.296 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.296 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:48.296 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152008+0000", 2024-09-21T15:00:48.296 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:48.296 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.296 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.296 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:48.296 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152005+0000", 2024-09-21T15:00:48.297 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-21T15:00:48.297 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.297 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.297 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:48.297 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152025+0000", 2024-09-21T15:00:48.297 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9306000000000001e-05 2024-09-21T15:00:48.297 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.297 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.297 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:48.297 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152054+0000", 2024-09-21T15:00:48.297 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9189000000000001e-05 2024-09-21T15:00:48.297 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.297 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.297 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:48.298 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152078+0000", 2024-09-21T15:00:48.298 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4317000000000001e-05 2024-09-21T15:00:48.298 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.298 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.298 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:48.298 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152243+0000", 2024-09-21T15:00:48.298 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000164802 2024-09-21T15:00:48.298 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.298 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.298 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting_for_map not empty", 2024-09-21T15:00:48.298 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152251+0000", 2024-09-21T15:00:48.298 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0600000000000008e-06 2024-09-21T15:00:48.298 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.298 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.299 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:48.299 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.154741+0000", 2024-09-21T15:00:48.299 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0024900069999999998 2024-09-21T15:00:48.299 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.299 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.299 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:00:48.299 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.154753+0000", 2024-09-21T15:00:48.299 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1588e-05 2024-09-21T15:00:48.299 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.299 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.299 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:48.299 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.165380+0000", 2024-09-21T15:00:48.299 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0106272249999999 2024-09-21T15:00:48.299 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.299 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.300 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:48.300 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.165517+0000", 2024-09-21T15:00:48.300 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013737999999999999 2024-09-21T15:00:48.300 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.300 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.300 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 6", 2024-09-21T15:00:48.300 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.165938+0000", 2024-09-21T15:00:48.300 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00042082399999999998 2024-09-21T15:00:48.300 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.300 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.300 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:48.300 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.166795+0000", 2024-09-21T15:00:48.300 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00085674899999999997 2024-09-21T15:00:48.300 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.300 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.300 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:48.300 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.167598+0000", 2024-09-21T15:00:48.301 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00080335599999999997 2024-09-21T15:00:48.301 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.301 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.301 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:48.301 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.167632+0000", 2024-09-21T15:00:48.301 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4122000000000003e-05 2024-09-21T15:00:48.301 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.301 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.301 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:48.301 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.167653+0000", 2024-09-21T15:00:48.301 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0862000000000001e-05 2024-09-21T15:00:48.301 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:48.301 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:48.301 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:48.301 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.301 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.302 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:73698 3.10 3:0a4d95f6:::rbd_data.1136f4939602.000000000000000b:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-21T15:00:48.302 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:41.969870+0000", 2024-09-21T15:00:48.302 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 66.293808107999993, 2024-09-21T15:00:48.302 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.017502461, 2024-09-21T15:00:48.302 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:48.302 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:48.302 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:48.302 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:48.302 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:48.302 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 73698 2024-09-21T15:00:48.302 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.302 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:48.302 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.302 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:48.303 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.969870+0000", 2024-09-21T15:00:48.303 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:48.303 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.303 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.303 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:48.303 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.969870+0000", 2024-09-21T15:00:48.303 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:48.303 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.303 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.303 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:48.303 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.969868+0000", 2024-09-21T15:00:48.303 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T15:00:48.303 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.303 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.303 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:48.304 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.969885+0000", 2024-09-21T15:00:48.304 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7416999999999999e-05 2024-09-21T15:00:48.304 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.304 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.304 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:48.304 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.969918+0000", 2024-09-21T15:00:48.304 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3133000000000003e-05 2024-09-21T15:00:48.304 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.304 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.304 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:48.304 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.969934+0000", 2024-09-21T15:00:48.304 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6027000000000001e-05 2024-09-21T15:00:48.304 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.304 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.304 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:48.305 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.970021+0000", 2024-09-21T15:00:48.305 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6907e-05 2024-09-21T15:00:48.305 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.305 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.305 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:48.305 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.970148+0000", 2024-09-21T15:00:48.305 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012701499999999999 2024-09-21T15:00:48.305 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.305 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.305 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T15:00:48.305 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.972971+0000", 2024-09-21T15:00:48.305 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.002822873 2024-09-21T15:00:48.305 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.305 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.305 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:48.306 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.987124+0000", 2024-09-21T15:00:48.306 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0141529 2024-09-21T15:00:48.306 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.306 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.306 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:48.306 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.987146+0000", 2024-09-21T15:00:48.306 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1783000000000001e-05 2024-09-21T15:00:48.306 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.306 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.306 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:48.306 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.987258+0000", 2024-09-21T15:00:48.306 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000112555 2024-09-21T15:00:48.306 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.306 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.306 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:48.307 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:41.987373+0000", 2024-09-21T15:00:48.307 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000114434 2024-09-21T15:00:48.307 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:48.307 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:48.307 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:48.307 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.307 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.307 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:97638 3.1a 3:5e573525:::rbd_data.1136b9c49b1c.0000000000000026:49 [read 1277952~49152 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:48.307 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.276202+0000", 2024-09-21T15:00:48.307 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 42.987476002999998, 2024-09-21T15:00:48.307 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.015360357999999999, 2024-09-21T15:00:48.307 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:48.307 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:48.307 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:48.307 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:48.308 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:48.308 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 97638 2024-09-21T15:00:48.308 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.308 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:48.308 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.308 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:48.308 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276202+0000", 2024-09-21T15:00:48.308 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:48.308 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.308 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.308 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:48.308 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276202+0000", 2024-09-21T15:00:48.308 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:48.308 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.309 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.309 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:48.309 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276202+0000", 2024-09-21T15:00:48.309 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:48.309 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.309 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.309 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:48.309 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276204+0000", 2024-09-21T15:00:48.309 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5359999999999999e-06 2024-09-21T15:00:48.309 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.309 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.309 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:48.309 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276210+0000", 2024-09-21T15:00:48.309 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3040000000000004e-06 2024-09-21T15:00:48.309 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.310 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.310 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:48.310 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.276214+0000", 2024-09-21T15:00:48.310 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6779999999999999e-06 2024-09-21T15:00:48.310 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.310 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.310 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:48.310 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.290563+0000", 2024-09-21T15:00:48.310 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014348346 2024-09-21T15:00:48.310 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.310 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.310 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:48.310 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.290623+0000", 2024-09-21T15:00:48.310 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0001999999999999e-05 2024-09-21T15:00:48.310 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.311 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.311 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:48.311 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.291563+0000", 2024-09-21T15:00:48.311 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00094003100000000003 2024-09-21T15:00:48.311 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:48.311 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:48.311 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:48.311 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.311 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.311 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:97678 3.d e122/24 3:b2f34c6d:::rbd_data.1136bfc79a70.0000000000000006:head v 122'234, mlcod=120'232)", 2024-09-21T15:00:48.311 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.321655+0000", 2024-09-21T15:00:48.311 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 42.942023329000001, 2024-09-21T15:00:48.311 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024509388999999999, 2024-09-21T15:00:48.311 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:48.311 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:48.312 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:48.312 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.312 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:48.312 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.321655+0000", 2024-09-21T15:00:48.312 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:48.312 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.312 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.312 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:48.312 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.321655+0000", 2024-09-21T15:00:48.312 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:48.312 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.312 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.312 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:48.312 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.321655+0000", 2024-09-21T15:00:48.312 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:48.313 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.313 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.313 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:48.313 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.329453+0000", 2024-09-21T15:00:48.313 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0077980560000000003 2024-09-21T15:00:48.313 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.313 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.313 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:48.313 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.329482+0000", 2024-09-21T15:00:48.313 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9153999999999999e-05 2024-09-21T15:00:48.313 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.313 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.313 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:48.313 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.329493+0000", 2024-09-21T15:00:48.313 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1103e-05 2024-09-21T15:00:48.314 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.314 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.314 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:48.314 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.329555+0000", 2024-09-21T15:00:48.314 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2345e-05 2024-09-21T15:00:48.314 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.314 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.314 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:48.314 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.329589+0000", 2024-09-21T15:00:48.314 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3908999999999997e-05 2024-09-21T15:00:48.314 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.314 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.314 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:48.314 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.346083+0000", 2024-09-21T15:00:48.315 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016493383 2024-09-21T15:00:48.315 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.315 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.315 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:48.315 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.346165+0000", 2024-09-21T15:00:48.315 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1796000000000001e-05 2024-09-21T15:00:48.315 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:48.315 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:48.315 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:48.315 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.315 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.315 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:97679 3.f e122/24 3:f2a362b4:::rbd_data.1136bfc79a70.0000000000000007:head v 122'331, mlcod=120'329)", 2024-09-21T15:00:48.315 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.337145+0000", 2024-09-21T15:00:48.315 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 42.926533319999997, 2024-09-21T15:00:48.315 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021912411, 2024-09-21T15:00:48.315 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:48.316 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:48.316 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:48.316 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.316 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:48.316 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.337145+0000", 2024-09-21T15:00:48.316 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:48.316 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.316 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.316 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:48.316 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.337145+0000", 2024-09-21T15:00:48.316 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:48.316 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.316 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.316 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:48.316 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.337145+0000", 2024-09-21T15:00:48.316 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:48.317 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.317 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.317 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:48.317 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.343899+0000", 2024-09-21T15:00:48.317 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0067541420000000003 2024-09-21T15:00:48.317 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.317 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.317 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:48.317 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.343921+0000", 2024-09-21T15:00:48.317 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2255e-05 2024-09-21T15:00:48.317 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.317 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.317 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:48.317 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.343933+0000", 2024-09-21T15:00:48.317 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1528e-05 2024-09-21T15:00:48.317 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.317 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.318 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:48.318 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.344001+0000", 2024-09-21T15:00:48.318 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8269999999999995e-05 2024-09-21T15:00:48.318 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.318 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.318 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:48.318 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.344029+0000", 2024-09-21T15:00:48.318 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8254000000000001e-05 2024-09-21T15:00:48.318 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.318 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.318 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:48.318 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.358901+0000", 2024-09-21T15:00:48.318 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0148719 2024-09-21T15:00:48.318 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.318 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.319 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:48.319 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.359058+0000", 2024-09-21T15:00:48.319 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000156446 2024-09-21T15:00:48.319 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:48.319 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:48.319 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:48.319 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.319 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.319 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:97676 3.1f e122/24 3:ffaa836a:::rbd_data.1136bfc79a70.0000000000000004:head v 122'312, mlcod=120'310)", 2024-09-21T15:00:48.319 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.348683+0000", 2024-09-21T15:00:48.319 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 42.914995155, 2024-09-21T15:00:48.319 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.033768802000000001, 2024-09-21T15:00:48.319 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:48.319 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:48.319 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:48.320 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.320 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:48.320 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.348683+0000", 2024-09-21T15:00:48.320 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:48.320 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.320 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.320 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:48.320 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.348683+0000", 2024-09-21T15:00:48.320 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:48.320 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.320 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.320 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:48.320 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.348683+0000", 2024-09-21T15:00:48.320 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:00:48.321 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.321 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.321 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:48.321 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.366937+0000", 2024-09-21T15:00:48.321 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018254415 2024-09-21T15:00:48.321 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.321 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.321 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:48.321 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.366959+0000", 2024-09-21T15:00:48.321 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1634e-05 2024-09-21T15:00:48.321 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.321 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.321 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:48.321 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.366971+0000", 2024-09-21T15:00:48.321 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2305e-05 2024-09-21T15:00:48.322 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.322 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.322 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:48.322 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.367034+0000", 2024-09-21T15:00:48.322 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3211000000000002e-05 2024-09-21T15:00:48.322 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.322 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.322 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:48.322 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.367056+0000", 2024-09-21T15:00:48.322 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2098000000000001e-05 2024-09-21T15:00:48.322 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.322 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.322 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:48.323 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.382400+0000", 2024-09-21T15:00:48.323 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.015343867000000001 2024-09-21T15:00:48.323 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.323 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.323 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:48.323 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.382452+0000", 2024-09-21T15:00:48.323 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1992e-05 2024-09-21T15:00:48.323 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:48.323 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:48.323 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:48.323 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.323 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.323 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:97734 3.19 e122/94 3:9fcf754f:::rbd_data.1136bfc79a70.0000000000000009:head v 122'314, mlcod=120'312)", 2024-09-21T15:00:48.324 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.378547+0000", 2024-09-21T15:00:48.324 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 42.885131176999998, 2024-09-21T15:00:48.324 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.063380675999999997, 2024-09-21T15:00:48.324 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:48.324 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:48.324 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:48.324 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.324 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:48.324 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.378547+0000", 2024-09-21T15:00:48.324 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:48.324 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.324 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.324 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:48.324 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.378547+0000", 2024-09-21T15:00:48.325 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:48.325 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.325 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.325 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:48.325 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.378547+0000", 2024-09-21T15:00:48.325 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:00:48.325 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.325 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.325 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:48.325 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.425118+0000", 2024-09-21T15:00:48.325 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046571166999999997 2024-09-21T15:00:48.325 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.325 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.325 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:48.326 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.425150+0000", 2024-09-21T15:00:48.326 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2150999999999998e-05 2024-09-21T15:00:48.326 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.326 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.326 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:48.326 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.425164+0000", 2024-09-21T15:00:48.326 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3302000000000001e-05 2024-09-21T15:00:48.326 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.326 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.326 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:48.326 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.425231+0000", 2024-09-21T15:00:48.326 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7106000000000004e-05 2024-09-21T15:00:48.326 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.327 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.327 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:48.327 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.425259+0000", 2024-09-21T15:00:48.327 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8087000000000001e-05 2024-09-21T15:00:48.327 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.327 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.327 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:48.327 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.441890+0000", 2024-09-21T15:00:48.327 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016631328000000001 2024-09-21T15:00:48.327 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.327 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.327 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:48.327 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.441928+0000", 2024-09-21T15:00:48.327 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7701999999999999e-05 2024-09-21T15:00:48.327 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:48.328 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:48.328 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:48.328 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.328 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.328 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:100187 3.10 3:08941a97:::rbd_data.1136ab4d6ff1.000000000000000e:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-21T15:00:48.328 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:10.869845+0000", 2024-09-21T15:00:48.328 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 37.393833280999999, 2024-09-21T15:00:48.328 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029345237, 2024-09-21T15:00:48.328 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:48.328 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:48.328 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:48.328 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:48.328 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:48.328 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 100187 2024-09-21T15:00:48.329 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.329 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:48.329 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.329 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:48.329 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869845+0000", 2024-09-21T15:00:48.329 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:48.329 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.329 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.329 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:48.329 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869845+0000", 2024-09-21T15:00:48.329 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:48.329 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.329 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.329 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:48.330 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869844+0000", 2024-09-21T15:00:48.330 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T15:00:48.330 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.330 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.330 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:48.330 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869853+0000", 2024-09-21T15:00:48.330 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5719999999999992e-06 2024-09-21T15:00:48.330 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.330 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.330 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:48.330 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869874+0000", 2024-09-21T15:00:48.330 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0483e-05 2024-09-21T15:00:48.330 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.330 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.331 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:48.331 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869883+0000", 2024-09-21T15:00:48.331 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6080000000000002e-06 2024-09-21T15:00:48.331 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.331 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.331 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:48.331 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869991+0000", 2024-09-21T15:00:48.331 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000108013 2024-09-21T15:00:48.331 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.331 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.331 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:48.331 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.870258+0000", 2024-09-21T15:00:48.331 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00026629099999999999 2024-09-21T15:00:48.331 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.331 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.332 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T15:00:48.332 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.878656+0000", 2024-09-21T15:00:48.332 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0083989000000000008 2024-09-21T15:00:48.332 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.332 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.332 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:48.332 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.899081+0000", 2024-09-21T15:00:48.332 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.020424437 2024-09-21T15:00:48.332 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.332 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.332 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:48.332 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.899125+0000", 2024-09-21T15:00:48.332 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4440999999999997e-05 2024-09-21T15:00:48.332 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.332 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.333 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:48.333 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.899160+0000", 2024-09-21T15:00:48.333 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4518999999999999e-05 2024-09-21T15:00:48.333 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.333 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.333 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:48.333 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.899190+0000", 2024-09-21T15:00:48.333 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0574999999999998e-05 2024-09-21T15:00:48.333 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:48.333 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:48.333 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:48.333 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.333 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.333 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:100263 3.10 3:0ece142e:::rbd_data.1136ab4d6ff1.0000000000000008:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-21T15:00:48.333 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:11.052257+0000", 2024-09-21T15:00:48.334 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 37.211421223000002, 2024-09-21T15:00:48.334 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042678025000000001, 2024-09-21T15:00:48.334 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:48.334 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:48.334 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:48.334 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:48.334 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:48.334 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 100263 2024-09-21T15:00:48.334 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.334 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:48.334 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.334 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:48.334 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052257+0000", 2024-09-21T15:00:48.334 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:48.335 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.335 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.335 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:48.335 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052257+0000", 2024-09-21T15:00:48.335 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:48.335 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.335 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.335 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:48.335 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052256+0000", 2024-09-21T15:00:48.335 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:00:48.335 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.335 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.335 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:48.335 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052263+0000", 2024-09-21T15:00:48.335 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4799999999999998e-06 2024-09-21T15:00:48.336 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.336 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.336 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:48.336 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052275+0000", 2024-09-21T15:00:48.336 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2386e-05 2024-09-21T15:00:48.336 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.336 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.336 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:48.336 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052281+0000", 2024-09-21T15:00:48.336 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6699999999999999e-06 2024-09-21T15:00:48.336 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.336 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.336 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:48.336 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052350+0000", 2024-09-21T15:00:48.336 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9099999999999999e-05 2024-09-21T15:00:48.336 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.337 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.337 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:48.337 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052456+0000", 2024-09-21T15:00:48.337 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000106669 2024-09-21T15:00:48.337 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.337 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.337 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T15:00:48.337 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.059694+0000", 2024-09-21T15:00:48.337 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.007237566 2024-09-21T15:00:48.337 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.337 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.337 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:48.337 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.094721+0000", 2024-09-21T15:00:48.337 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035026977000000001 2024-09-21T15:00:48.337 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.337 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.338 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:48.338 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.094746+0000", 2024-09-21T15:00:48.338 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5329000000000001e-05 2024-09-21T15:00:48.338 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.338 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.338 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:48.338 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.094850+0000", 2024-09-21T15:00:48.338 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000103452 2024-09-21T15:00:48.338 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.338 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.338 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:48.338 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.094935+0000", 2024-09-21T15:00:48.338 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5550000000000006e-05 2024-09-21T15:00:48.338 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:48.338 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:48.339 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:48.339 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.339 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.339 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:102266 3.1f e130/24 3:fec72c47:::rbd_data.1136d175325b.000000000000000c:head v 130'326, mlcod=128'324)", 2024-09-21T15:00:48.339 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:14.319401+0000", 2024-09-21T15:00:48.339 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 33.944277282999998, 2024-09-21T15:00:48.339 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021068982999999999, 2024-09-21T15:00:48.339 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:48.339 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:48.339 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:48.339 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.339 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:48.339 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.319401+0000", 2024-09-21T15:00:48.339 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:48.339 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.340 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.340 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:48.340 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.319401+0000", 2024-09-21T15:00:48.340 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:48.340 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.340 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.340 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:48.340 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.319400+0000", 2024-09-21T15:00:48.340 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:00:48.340 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.340 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.340 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:48.340 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.337267+0000", 2024-09-21T15:00:48.340 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.017867141 2024-09-21T15:00:48.340 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.340 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.340 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:48.341 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.337278+0000", 2024-09-21T15:00:48.341 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0224000000000001e-05 2024-09-21T15:00:48.341 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.341 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.341 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:48.341 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.337285+0000", 2024-09-21T15:00:48.341 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.25e-06 2024-09-21T15:00:48.341 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.341 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.341 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:48.341 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.337330+0000", 2024-09-21T15:00:48.341 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5210000000000003e-05 2024-09-21T15:00:48.341 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.341 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.341 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:48.342 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.337350+0000", 2024-09-21T15:00:48.342 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9556e-05 2024-09-21T15:00:48.342 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.342 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.342 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:48.342 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.340418+0000", 2024-09-21T15:00:48.342 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.003068552 2024-09-21T15:00:48.342 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.342 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.342 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:48.342 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.340470+0000", 2024-09-21T15:00:48.342 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1897000000000001e-05 2024-09-21T15:00:48.342 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:48.342 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:48.342 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:48.343 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.343 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.343 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:102887 3.1c 3:3b11ae99:::rbd_object_map.1136d175325b.0000000000000050:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-21T15:00:48.343 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:14.368644+0000", 2024-09-21T15:00:48.343 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 33.895034246999998, 2024-09-21T15:00:48.343 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.017021290000000001, 2024-09-21T15:00:48.343 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:48.343 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:48.343 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:48.343 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:48.343 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:48.343 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 102887 2024-09-21T15:00:48.343 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.343 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:48.344 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.344 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:48.344 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.368644+0000", 2024-09-21T15:00:48.344 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:48.344 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.344 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.344 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:48.344 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.368644+0000", 2024-09-21T15:00:48.344 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:48.344 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.344 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.344 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:48.344 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.368642+0000", 2024-09-21T15:00:48.344 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T15:00:48.345 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.345 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.345 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:48.345 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.368655+0000", 2024-09-21T15:00:48.345 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2808e-05 2024-09-21T15:00:48.345 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.345 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.345 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:48.345 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.368677+0000", 2024-09-21T15:00:48.345 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2753999999999999e-05 2024-09-21T15:00:48.345 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.345 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.345 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:48.345 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.368688+0000", 2024-09-21T15:00:48.346 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1228e-05 2024-09-21T15:00:48.346 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.346 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.346 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:48.346 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.368743+0000", 2024-09-21T15:00:48.346 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4608999999999998e-05 2024-09-21T15:00:48.346 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.346 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.346 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:48.346 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.368868+0000", 2024-09-21T15:00:48.346 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012473000000000001 2024-09-21T15:00:48.346 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.346 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.346 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 0", 2024-09-21T15:00:48.347 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.369385+0000", 2024-09-21T15:00:48.347 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00051684400000000003 2024-09-21T15:00:48.347 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.347 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.347 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:48.347 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.370755+0000", 2024-09-21T15:00:48.347 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0013701609999999999 2024-09-21T15:00:48.347 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.347 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.347 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:48.347 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.385531+0000", 2024-09-21T15:00:48.347 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.014776648 2024-09-21T15:00:48.347 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.347 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.348 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:48.348 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.385575+0000", 2024-09-21T15:00:48.348 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3726000000000001e-05 2024-09-21T15:00:48.348 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.348 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.348 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:48.348 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.385666+0000", 2024-09-21T15:00:48.348 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0304999999999997e-05 2024-09-21T15:00:48.348 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:48.348 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:48.348 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:48.348 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.348 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.348 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:103706 3.1c 3:3b11ae99:::rbd_object_map.1136d175325b.0000000000000050:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-21T15:00:48.348 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:14.426370+0000", 2024-09-21T15:00:48.349 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 33.837308909999997, 2024-09-21T15:00:48.349 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.015478897, 2024-09-21T15:00:48.349 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:48.349 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:48.349 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:48.349 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:48.349 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:48.349 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 103706 2024-09-21T15:00:48.349 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.349 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:48.349 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.349 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:48.349 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.426370+0000", 2024-09-21T15:00:48.349 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:48.350 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.350 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.350 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:48.350 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.426370+0000", 2024-09-21T15:00:48.350 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:48.350 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.350 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.350 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:48.350 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.426369+0000", 2024-09-21T15:00:48.350 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:00:48.350 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.350 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.350 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:48.350 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.426376+0000", 2024-09-21T15:00:48.351 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0999999999999998e-06 2024-09-21T15:00:48.351 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.351 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.351 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:48.351 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.426442+0000", 2024-09-21T15:00:48.351 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6445999999999994e-05 2024-09-21T15:00:48.351 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.351 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.351 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:48.351 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.426451+0000", 2024-09-21T15:00:48.351 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4940000000000003e-06 2024-09-21T15:00:48.351 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.351 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.351 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:48.352 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.426493+0000", 2024-09-21T15:00:48.352 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2094000000000001e-05 2024-09-21T15:00:48.352 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.352 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.352 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:48.352 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.426560+0000", 2024-09-21T15:00:48.352 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6774999999999997e-05 2024-09-21T15:00:48.352 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.352 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.352 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 0", 2024-09-21T15:00:48.352 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.426737+0000", 2024-09-21T15:00:48.352 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017769899999999999 2024-09-21T15:00:48.352 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.352 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.352 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:48.353 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.428561+0000", 2024-09-21T15:00:48.353 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001823917 2024-09-21T15:00:48.353 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.353 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.353 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:48.353 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.441762+0000", 2024-09-21T15:00:48.353 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013200462 2024-09-21T15:00:48.353 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.353 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.353 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:48.353 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.441811+0000", 2024-09-21T15:00:48.353 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8912000000000002e-05 2024-09-21T15:00:48.353 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.353 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.354 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:48.354 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.441848+0000", 2024-09-21T15:00:48.354 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7935000000000002e-05 2024-09-21T15:00:48.354 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:48.354 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:48.354 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:48.354 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.354 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.354 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:105083 3.1a 3:5d63845a:::rbd_data.1136d175325b.0000000000000037:head [write 0~32768 [fadvise_sequential+fadvise_nocache],write 65536~163840 [fadvise_sequential+fadvise_nocache],write 262144~32768 [fadvise_sequential+fadvise_nocache],write 327680~163840 [fadvise_sequential+fadvise_nocache],write 524288~32768 [fadvise_sequential+fadvise_nocache],write 589824~163840 [fadvise_sequential+fadvise_nocache],write 786432~32768 [fadvise_sequential+fadvise_nocache],write 851968~32768 [fadvise_sequential+fadvise_nocache],write 917504~98304 [fadvise_sequential+fadvise_nocache],write 1048576~32768 [fadvise_sequential+fadvise_nocache],write 1114112~32768 [fadvise_sequential+fadvise_nocache],write 1179648~98304 [fadvise_sequential+fadvise_nocache],write 1310720~32768 [fadvise_sequential+fadvise_nocache],write 1376256~32768 [fadvise_sequential+fadvise_nocache],write 1441792~98304 [fadvise_sequential+fadvise_nocache],write 1572864~32768 [fadvise_sequential+fadvise_nocache],write 1638400~32768 [fadvise_sequential+fadvise_nocache],write 1703936~98304 [fadvise_sequential+fadvise_nocache],write 1835008~32768 [fadvise_sequential+fadvise_nocache],write 1900544~32768 [fadvise_sequential+fadvise_nocache],write 1966080~98304 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-21T15:00:48.354 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:14.534201+0000", 2024-09-21T15:00:48.354 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 33.729477568999997, 2024-09-21T15:00:48.354 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016474194000000001, 2024-09-21T15:00:48.354 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:48.354 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:48.355 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:48.355 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:48.355 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:48.355 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 105083 2024-09-21T15:00:48.355 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.355 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:48.355 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.355 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:48.355 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.534201+0000", 2024-09-21T15:00:48.355 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:48.355 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.355 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.355 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:48.355 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.534201+0000", 2024-09-21T15:00:48.356 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:48.356 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.356 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.356 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:48.356 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.534201+0000", 2024-09-21T15:00:48.356 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:48.356 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.356 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.356 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:48.356 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.534853+0000", 2024-09-21T15:00:48.356 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00065227099999999995 2024-09-21T15:00:48.356 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.356 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.356 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:48.356 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.534866+0000", 2024-09-21T15:00:48.357 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3553000000000001e-05 2024-09-21T15:00:48.357 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.357 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.357 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:48.357 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.534877+0000", 2024-09-21T15:00:48.357 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0186e-05 2024-09-21T15:00:48.357 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.357 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.357 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:48.357 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.534947+0000", 2024-09-21T15:00:48.357 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0417e-05 2024-09-21T15:00:48.357 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.357 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.358 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:48.358 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.535138+0000", 2024-09-21T15:00:48.358 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00019122200000000001 2024-09-21T15:00:48.358 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.358 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.358 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T15:00:48.358 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.535666+0000", 2024-09-21T15:00:48.358 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00052817499999999998 2024-09-21T15:00:48.358 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.358 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.358 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:48.358 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.543252+0000", 2024-09-21T15:00:48.358 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0075857889999999999 2024-09-21T15:00:48.358 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.358 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.359 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:48.359 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.550552+0000", 2024-09-21T15:00:48.359 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0072995869999999997 2024-09-21T15:00:48.359 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.359 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.359 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:48.359 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.550637+0000", 2024-09-21T15:00:48.359 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5750999999999999e-05 2024-09-21T15:00:48.359 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.359 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.359 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:48.359 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:14.550675+0000", 2024-09-21T15:00:48.359 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7602999999999999e-05 2024-09-21T15:00:48.359 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:48.360 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:48.360 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:48.360 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.360 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.360 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:113281 3.10 3:0e1b39db:::rbd_data.1136b442e94a.0000000000000009:head [write 0~360448 [fadvise_sequential+fadvise_nocache],write 376832~573440 [fadvise_sequential+fadvise_nocache],write 1138688~573440 [fadvise_sequential+fadvise_nocache],write 1900544~196608 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:48.360 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.487425+0000", 2024-09-21T15:00:48.360 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.776253626999999, 2024-09-21T15:00:48.360 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.015461361999999999, 2024-09-21T15:00:48.360 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:48.360 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:48.360 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:48.360 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:48.360 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:48.360 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 113281 2024-09-21T15:00:48.360 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.361 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:48.361 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.361 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:48.361 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.487425+0000", 2024-09-21T15:00:48.361 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:48.361 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.361 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.361 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:48.361 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.487425+0000", 2024-09-21T15:00:48.361 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:48.361 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.361 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.361 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:48.361 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.487423+0000", 2024-09-21T15:00:48.361 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-21T15:00:48.361 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.362 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.362 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:48.362 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.488151+0000", 2024-09-21T15:00:48.362 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00072770699999999996 2024-09-21T15:00:48.362 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.362 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.362 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:48.362 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.488175+0000", 2024-09-21T15:00:48.362 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3896e-05 2024-09-21T15:00:48.362 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.362 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.362 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:48.362 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.488184+0000", 2024-09-21T15:00:48.362 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9000000000000001e-06 2024-09-21T15:00:48.362 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.363 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.363 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:48.363 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.488918+0000", 2024-09-21T15:00:48.363 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00073320500000000001 2024-09-21T15:00:48.363 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.363 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.363 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:48.363 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.489175+0000", 2024-09-21T15:00:48.363 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00025761799999999999 2024-09-21T15:00:48.363 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.363 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.363 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T15:00:48.363 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.489479+0000", 2024-09-21T15:00:48.363 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00030419099999999999 2024-09-21T15:00:48.363 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.364 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.364 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:48.364 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.498472+0000", 2024-09-21T15:00:48.364 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0089922319999999993 2024-09-21T15:00:48.364 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.364 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.364 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:48.364 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502751+0000", 2024-09-21T15:00:48.364 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0042795309999999996 2024-09-21T15:00:48.364 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.364 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.364 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:48.364 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502843+0000", 2024-09-21T15:00:48.364 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1926e-05 2024-09-21T15:00:48.364 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.365 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.365 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:48.365 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502886+0000", 2024-09-21T15:00:48.365 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3087999999999998e-05 2024-09-21T15:00:48.365 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:48.365 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:48.365 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:48.365 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:48.365 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:48.365 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:48.366 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T15:00:48.367 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:00:48.363+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:00:48.467 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:00:48.463+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:00:48.494 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:48.494 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:00:48.494 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T15:00:48.494 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:48.505 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T15:00:48.568 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:00:48.565+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:00:48.658 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:48.659 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:00:48.659 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T15:00:48.659 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T15:00:48.659 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:48.668 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T15:00:48.669 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:00:48.665+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:00:48.769 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:00:48.767+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:00:48.782 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:48.782 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T15:00:48.782 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T15:00:48.782 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T15:00:48.782 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.782 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:15130 3.2 3:47f1bbdb:::rbd_data.11361bed1f78.0000000000000067:head [sparse-read 0~147456] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-21T15:00:48.783 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:40.288284+0000", 2024-09-21T15:00:48.783 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 128.49175647800001, 2024-09-21T15:00:48.783 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1142935039999999, 2024-09-21T15:00:48.783 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:48.783 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:48.783 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:48.783 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:48.783 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:48.783 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 15130 2024-09-21T15:00:48.783 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.783 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:48.783 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.783 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:48.783 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288284+0000", 2024-09-21T15:00:48.783 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:48.784 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.784 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.784 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:48.784 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288284+0000", 2024-09-21T15:00:48.784 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:48.784 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.784 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.784 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:48.784 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288284+0000", 2024-09-21T15:00:48.784 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:48.784 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.784 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.784 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:48.785 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288288+0000", 2024-09-21T15:00:48.785 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5829999999999998e-06 2024-09-21T15:00:48.785 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.785 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.785 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:48.785 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288297+0000", 2024-09-21T15:00:48.785 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0669999999999998e-06 2024-09-21T15:00:48.785 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.785 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.785 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:48.785 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288305+0000", 2024-09-21T15:00:48.785 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8860000000000004e-06 2024-09-21T15:00:48.785 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.785 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.785 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:48.786 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288417+0000", 2024-09-21T15:00:48.786 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000112952 2024-09-21T15:00:48.786 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.786 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.786 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:00:48.786 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288428+0000", 2024-09-21T15:00:48.786 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0631e-05 2024-09-21T15:00:48.786 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.786 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.786 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:48.786 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.401876+0000", 2024-09-21T15:00:48.786 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1134479610000001 2024-09-21T15:00:48.786 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.786 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.787 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:48.787 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.402076+0000", 2024-09-21T15:00:48.787 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00019944200000000001 2024-09-21T15:00:48.787 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.787 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.787 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:48.787 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.402578+0000", 2024-09-21T15:00:48.787 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00050242899999999998 2024-09-21T15:00:48.787 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:48.787 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:48.787 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:48.787 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.787 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.787 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45708 3.2 3:415db91a:::rbd_data.1136ecbe5879.0000000000000000:28 [read 140288~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:48.787 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141801+0000", 2024-09-21T15:00:48.788 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 98.638240284000005, 2024-09-21T15:00:48.788 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034274200999999997, 2024-09-21T15:00:48.788 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:48.788 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:48.788 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:48.788 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:48.788 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:48.788 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45708 2024-09-21T15:00:48.788 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.788 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:48.788 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.788 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:48.788 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141801+0000", 2024-09-21T15:00:48.788 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:48.788 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.788 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.789 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:48.789 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141801+0000", 2024-09-21T15:00:48.789 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:48.789 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.789 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.789 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:48.789 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141800+0000", 2024-09-21T15:00:48.789 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:48.789 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.789 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.789 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:48.789 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141803+0000", 2024-09-21T15:00:48.789 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4210000000000002e-06 2024-09-21T15:00:48.789 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.789 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.789 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:48.790 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141812+0000", 2024-09-21T15:00:48.790 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.9800000000000004e-06 2024-09-21T15:00:48.790 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.790 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.790 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:48.790 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141816+0000", 2024-09-21T15:00:48.790 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4769999999999997e-06 2024-09-21T15:00:48.790 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.790 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.790 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:48.790 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.175865+0000", 2024-09-21T15:00:48.790 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034048875999999999 2024-09-21T15:00:48.790 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.791 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.791 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:48.791 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.175943+0000", 2024-09-21T15:00:48.791 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7464000000000002e-05 2024-09-21T15:00:48.791 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.791 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.791 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:48.791 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176075+0000", 2024-09-21T15:00:48.791 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013230800000000001 2024-09-21T15:00:48.791 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:48.791 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:48.791 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:48.791 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.791 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.791 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45709 3.2 3:415db91a:::rbd_data.1136ecbe5879.0000000000000000:28 [read 143360~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:48.791 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141820+0000", 2024-09-21T15:00:48.792 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 98.638220437000001, 2024-09-21T15:00:48.792 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034537536000000001, 2024-09-21T15:00:48.792 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:48.792 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:48.792 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:48.792 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:48.792 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:48.792 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45709 2024-09-21T15:00:48.792 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.792 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:48.792 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.792 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:48.792 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141820+0000", 2024-09-21T15:00:48.792 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:48.792 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.792 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.793 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:48.793 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141820+0000", 2024-09-21T15:00:48.793 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:48.793 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.793 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.793 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:48.793 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141820+0000", 2024-09-21T15:00:48.793 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:48.793 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.793 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.793 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:48.793 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141823+0000", 2024-09-21T15:00:48.793 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5110000000000002e-06 2024-09-21T15:00:48.793 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.793 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.794 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:48.794 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141829+0000", 2024-09-21T15:00:48.794 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0850000000000002e-06 2024-09-21T15:00:48.794 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.794 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.794 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:48.794 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141833+0000", 2024-09-21T15:00:48.794 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7620000000000003e-06 2024-09-21T15:00:48.794 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.794 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.794 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:48.794 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176103+0000", 2024-09-21T15:00:48.794 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034269933000000002 2024-09-21T15:00:48.795 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.795 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.795 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:48.795 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176161+0000", 2024-09-21T15:00:48.795 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7658999999999998e-05 2024-09-21T15:00:48.795 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.795 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.795 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:48.795 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176358+0000", 2024-09-21T15:00:48.795 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000196937 2024-09-21T15:00:48.795 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:48.795 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:48.795 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:48.795 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.796 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.796 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45710 3.2 3:415db91a:::rbd_data.1136ecbe5879.0000000000000000:28 [read 146432~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:48.796 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141838+0000", 2024-09-21T15:00:48.796 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 98.638203274000006, 2024-09-21T15:00:48.796 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034953674999999997, 2024-09-21T15:00:48.796 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:48.796 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:48.796 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:48.796 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:48.796 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:48.796 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45710 2024-09-21T15:00:48.796 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.796 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:48.796 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.796 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:48.796 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141838+0000", 2024-09-21T15:00:48.797 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:48.797 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.797 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.797 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:48.797 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141838+0000", 2024-09-21T15:00:48.797 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:48.797 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.797 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.797 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:48.797 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141837+0000", 2024-09-21T15:00:48.797 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:48.797 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.797 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.797 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:48.797 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141843+0000", 2024-09-21T15:00:48.797 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1619999999999996e-06 2024-09-21T15:00:48.798 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.798 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.798 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:48.798 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141850+0000", 2024-09-21T15:00:48.798 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4799999999999998e-06 2024-09-21T15:00:48.798 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.798 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.798 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:48.798 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141853+0000", 2024-09-21T15:00:48.798 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3050000000000001e-06 2024-09-21T15:00:48.798 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.798 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.798 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:48.798 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176490+0000", 2024-09-21T15:00:48.799 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034636413999999997 2024-09-21T15:00:48.799 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.799 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.799 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:48.799 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176606+0000", 2024-09-21T15:00:48.799 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000115841 2024-09-21T15:00:48.799 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.799 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.799 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:48.799 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176791+0000", 2024-09-21T15:00:48.799 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00018581500000000001 2024-09-21T15:00:48.799 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:48.799 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:48.799 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:48.799 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.800 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.800 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45711 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:48.800 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141856+0000", 2024-09-21T15:00:48.800 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 98.638184937000005, 2024-09-21T15:00:48.800 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034986308000000001, 2024-09-21T15:00:48.800 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:48.800 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:48.800 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:48.800 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:48.800 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:48.800 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45711 2024-09-21T15:00:48.800 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.800 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:48.800 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.800 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:48.800 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141856+0000", 2024-09-21T15:00:48.801 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:48.801 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.801 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.801 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:48.801 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141856+0000", 2024-09-21T15:00:48.801 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:48.801 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.801 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.801 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:48.801 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141856+0000", 2024-09-21T15:00:48.801 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:48.801 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.801 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.801 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:48.801 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141862+0000", 2024-09-21T15:00:48.801 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.567e-06 2024-09-21T15:00:48.802 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.802 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.802 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:48.802 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141868+0000", 2024-09-21T15:00:48.802 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9569999999999999e-06 2024-09-21T15:00:48.802 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.802 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.802 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:48.802 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141871+0000", 2024-09-21T15:00:48.802 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.146e-06 2024-09-21T15:00:48.802 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.802 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.802 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:48.802 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176831+0000", 2024-09-21T15:00:48.802 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034959935999999997 2024-09-21T15:00:48.803 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.803 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.803 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:48.803 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176842+0000", 2024-09-21T15:00:48.803 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1008e-05 2024-09-21T15:00:48.803 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:48.803 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:48.803 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:48.803 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.803 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.803 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45712 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:48.803 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141874+0000", 2024-09-21T15:00:48.803 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 98.638167042000006, 2024-09-21T15:00:48.803 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034993511999999997, 2024-09-21T15:00:48.804 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:48.804 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:48.804 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:48.804 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:48.804 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:48.804 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45712 2024-09-21T15:00:48.804 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.804 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:48.804 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.804 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:48.804 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141874+0000", 2024-09-21T15:00:48.804 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:48.804 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.805 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.805 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:48.805 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141874+0000", 2024-09-21T15:00:48.805 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:48.805 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.805 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.805 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:48.805 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141874+0000", 2024-09-21T15:00:48.805 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:48.805 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.805 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.805 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:48.805 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141875+0000", 2024-09-21T15:00:48.805 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5430000000000001e-06 2024-09-21T15:00:48.805 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.806 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.806 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:48.806 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141878+0000", 2024-09-21T15:00:48.806 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1420000000000002e-06 2024-09-21T15:00:48.806 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.806 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.806 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:48.806 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141881+0000", 2024-09-21T15:00:48.806 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.807e-06 2024-09-21T15:00:48.806 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.806 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.806 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:48.806 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176860+0000", 2024-09-21T15:00:48.806 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034979243 2024-09-21T15:00:48.807 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.807 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.807 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:48.807 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176867+0000", 2024-09-21T15:00:48.807 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0169999999999999e-06 2024-09-21T15:00:48.807 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:48.807 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:48.807 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:48.807 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.807 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.807 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45713 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:48.807 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141883+0000", 2024-09-21T15:00:48.807 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 98.638157466999999, 2024-09-21T15:00:48.807 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035010516999999998, 2024-09-21T15:00:48.807 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:48.808 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:48.808 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:48.808 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:48.808 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:48.808 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45713 2024-09-21T15:00:48.808 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.808 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:48.808 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.808 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:48.808 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141883+0000", 2024-09-21T15:00:48.808 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:48.808 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.808 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.808 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:48.808 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141883+0000", 2024-09-21T15:00:48.808 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:48.809 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.809 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.809 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:48.809 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141883+0000", 2024-09-21T15:00:48.809 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:48.809 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.809 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.809 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:48.809 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141885+0000", 2024-09-21T15:00:48.809 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4649999999999999e-06 2024-09-21T15:00:48.809 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.809 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.809 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:48.809 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141888+0000", 2024-09-21T15:00:48.809 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3890000000000001e-06 2024-09-21T15:00:48.809 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.810 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.810 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:48.810 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141891+0000", 2024-09-21T15:00:48.810 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9239999999999999e-06 2024-09-21T15:00:48.810 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.810 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.810 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:48.810 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176887+0000", 2024-09-21T15:00:48.810 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034996370999999998 2024-09-21T15:00:48.810 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.810 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.810 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:48.810 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176894+0000", 2024-09-21T15:00:48.811 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6069999999999996e-06 2024-09-21T15:00:48.811 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:48.811 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:48.811 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:48.811 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.811 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.811 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45714 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:48.811 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141893+0000", 2024-09-21T15:00:48.811 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 98.638147552000007, 2024-09-21T15:00:48.811 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035048215000000001, 2024-09-21T15:00:48.811 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:48.811 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:48.811 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:48.811 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:48.811 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:48.812 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45714 2024-09-21T15:00:48.812 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.812 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:48.812 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.812 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:48.812 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141893+0000", 2024-09-21T15:00:48.812 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:48.812 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.812 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.812 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:48.812 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141893+0000", 2024-09-21T15:00:48.812 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:48.812 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.812 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.812 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:48.812 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141893+0000", 2024-09-21T15:00:48.813 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:48.813 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.813 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.813 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:48.813 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141895+0000", 2024-09-21T15:00:48.813 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.703e-06 2024-09-21T15:00:48.813 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.813 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.813 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:48.813 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141900+0000", 2024-09-21T15:00:48.813 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.242e-06 2024-09-21T15:00:48.813 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.813 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.814 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:48.814 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141903+0000", 2024-09-21T15:00:48.814 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.117e-06 2024-09-21T15:00:48.814 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.814 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.814 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:48.814 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176930+0000", 2024-09-21T15:00:48.814 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035026750000000002 2024-09-21T15:00:48.814 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.814 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.814 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:48.814 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176942+0000", 2024-09-21T15:00:48.814 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1668000000000001e-05 2024-09-21T15:00:48.814 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:48.815 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:48.815 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:48.815 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.815 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.815 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45715 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:48.815 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141906+0000", 2024-09-21T15:00:48.815 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 98.638135394000003, 2024-09-21T15:00:48.815 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035066057999999997, 2024-09-21T15:00:48.815 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:48.815 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:48.815 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:48.815 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:48.815 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:48.815 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45715 2024-09-21T15:00:48.816 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.816 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:48.816 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.816 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:48.816 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141906+0000", 2024-09-21T15:00:48.816 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:48.816 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.816 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.816 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:48.816 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141906+0000", 2024-09-21T15:00:48.816 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:48.816 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.816 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.816 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:48.816 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141905+0000", 2024-09-21T15:00:48.816 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:48.817 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.817 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.817 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:48.817 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141907+0000", 2024-09-21T15:00:48.817 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5400000000000001e-06 2024-09-21T15:00:48.817 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.817 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.818 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:48.818 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141912+0000", 2024-09-21T15:00:48.818 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.502e-06 2024-09-21T15:00:48.818 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.818 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.818 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:48.818 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141915+0000", 2024-09-21T15:00:48.818 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0170000000000001e-06 2024-09-21T15:00:48.819 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.819 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.819 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:48.819 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176964+0000", 2024-09-21T15:00:48.819 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035048587999999999 2024-09-21T15:00:48.819 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.819 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.820 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:48.820 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176972+0000", 2024-09-21T15:00:48.820 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6580000000000002e-06 2024-09-21T15:00:48.820 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:48.820 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:48.820 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:48.820 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.821 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.821 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45716 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:48.821 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141918+0000", 2024-09-21T15:00:48.821 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 98.638123218000004, 2024-09-21T15:00:48.821 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035086223, 2024-09-21T15:00:48.821 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:48.822 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:48.822 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:48.822 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:48.822 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:48.822 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45716 2024-09-21T15:00:48.822 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.822 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:48.823 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.823 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:48.823 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141918+0000", 2024-09-21T15:00:48.823 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:48.823 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.823 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.824 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:48.824 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141918+0000", 2024-09-21T15:00:48.824 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:48.824 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.824 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.825 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:48.825 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141917+0000", 2024-09-21T15:00:48.825 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:48.825 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.825 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.825 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:48.826 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141919+0000", 2024-09-21T15:00:48.826 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5680000000000001e-06 2024-09-21T15:00:48.826 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.826 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.826 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:48.826 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141922+0000", 2024-09-21T15:00:48.827 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2789999999999999e-06 2024-09-21T15:00:48.827 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.827 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.827 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:48.827 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141925+0000", 2024-09-21T15:00:48.827 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9469999999999999e-06 2024-09-21T15:00:48.827 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.828 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.828 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:48.828 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176997+0000", 2024-09-21T15:00:48.828 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035071366999999999 2024-09-21T15:00:48.828 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.828 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.829 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:48.829 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177004+0000", 2024-09-21T15:00:48.829 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3309999999999998e-06 2024-09-21T15:00:48.829 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:48.829 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:48.829 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:48.829 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.830 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.830 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45717 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:48.830 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141928+0000", 2024-09-21T15:00:48.830 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 98.638113271999998, 2024-09-21T15:00:48.830 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035102573999999998, 2024-09-21T15:00:48.830 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:48.831 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:48.831 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:48.831 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:48.831 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:48.831 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45717 2024-09-21T15:00:48.831 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.832 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:48.832 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.832 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:48.832 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141928+0000", 2024-09-21T15:00:48.832 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:48.832 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.833 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.833 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:48.833 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141928+0000", 2024-09-21T15:00:48.833 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:48.833 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.833 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.833 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:48.834 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141927+0000", 2024-09-21T15:00:48.834 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:00:48.834 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.834 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.834 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:48.834 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141929+0000", 2024-09-21T15:00:48.835 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.533e-06 2024-09-21T15:00:48.835 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.835 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.835 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:48.835 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141934+0000", 2024-09-21T15:00:48.835 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1660000000000002e-06 2024-09-21T15:00:48.836 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.836 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.836 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:48.836 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141937+0000", 2024-09-21T15:00:48.836 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0309999999999999e-06 2024-09-21T15:00:48.836 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.836 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.837 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:48.837 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177023+0000", 2024-09-21T15:00:48.837 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035085595999999997 2024-09-21T15:00:48.837 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.837 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.837 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:48.837 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177030+0000", 2024-09-21T15:00:48.838 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.486e-06 2024-09-21T15:00:48.838 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:48.838 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:48.838 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:48.838 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.838 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.838 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45718 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:48.839 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141940+0000", 2024-09-21T15:00:48.839 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 98.638101414000005, 2024-09-21T15:00:48.839 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035124956999999998, 2024-09-21T15:00:48.839 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:48.839 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:48.839 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:48.839 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:48.839 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:48.839 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45718 2024-09-21T15:00:48.840 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.840 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:48.840 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.840 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:48.840 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141940+0000", 2024-09-21T15:00:48.840 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:48.840 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.840 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.840 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:48.841 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141940+0000", 2024-09-21T15:00:48.841 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:48.841 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.841 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.841 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:48.841 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141939+0000", 2024-09-21T15:00:48.841 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:48.841 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.841 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.842 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:48.842 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141941+0000", 2024-09-21T15:00:48.842 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.502e-06 2024-09-21T15:00:48.842 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.842 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.842 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:48.842 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141946+0000", 2024-09-21T15:00:48.842 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3449999999999996e-06 2024-09-21T15:00:48.842 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.843 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.843 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:48.843 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141949+0000", 2024-09-21T15:00:48.843 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9749999999999999e-06 2024-09-21T15:00:48.843 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.843 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.843 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:48.843 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177057+0000", 2024-09-21T15:00:48.843 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035108385999999998 2024-09-21T15:00:48.843 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.843 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.843 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:48.843 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177064+0000", 2024-09-21T15:00:48.844 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0099999999999998e-06 2024-09-21T15:00:48.844 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:48.844 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:48.844 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:48.844 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.844 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.844 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45719 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:48.844 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141952+0000", 2024-09-21T15:00:48.844 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 98.638089140999995, 2024-09-21T15:00:48.845 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035139108000000002, 2024-09-21T15:00:48.845 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:48.845 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:48.845 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:48.845 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:48.845 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:48.845 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45719 2024-09-21T15:00:48.845 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.845 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:48.845 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.846 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:48.846 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141952+0000", 2024-09-21T15:00:48.846 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:48.846 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.846 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.846 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:48.846 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141952+0000", 2024-09-21T15:00:48.846 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:48.846 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.846 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.846 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:48.846 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141952+0000", 2024-09-21T15:00:48.846 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:48.847 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.847 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.847 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:48.847 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141953+0000", 2024-09-21T15:00:48.847 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.486e-06 2024-09-21T15:00:48.847 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.847 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.847 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:48.847 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141959+0000", 2024-09-21T15:00:48.847 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4940000000000003e-06 2024-09-21T15:00:48.847 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.847 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.847 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:48.847 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141963+0000", 2024-09-21T15:00:48.848 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3540000000000002e-06 2024-09-21T15:00:48.848 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.848 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.848 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:48.848 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177084+0000", 2024-09-21T15:00:48.848 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035120983000000001 2024-09-21T15:00:48.848 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.848 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.848 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:48.848 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177091+0000", 2024-09-21T15:00:48.848 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0350000000000004e-06 2024-09-21T15:00:48.848 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:48.848 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:48.848 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:48.848 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.849 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.849 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45720 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:48.849 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141965+0000", 2024-09-21T15:00:48.849 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 98.638075674000007, 2024-09-21T15:00:48.849 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035155268000000003, 2024-09-21T15:00:48.849 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:48.849 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:48.849 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:48.849 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:48.849 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:48.849 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45720 2024-09-21T15:00:48.849 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.849 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:48.849 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.849 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:48.850 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141965+0000", 2024-09-21T15:00:48.850 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:48.850 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.850 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.850 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:48.850 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141965+0000", 2024-09-21T15:00:48.850 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:48.850 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.850 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.850 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:48.850 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141965+0000", 2024-09-21T15:00:48.850 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:48.850 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.850 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.851 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:48.851 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141966+0000", 2024-09-21T15:00:48.851 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.46e-06 2024-09-21T15:00:48.851 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.851 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.851 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:48.851 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141972+0000", 2024-09-21T15:00:48.851 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4210000000000003e-06 2024-09-21T15:00:48.851 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.851 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.851 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:48.851 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141975+0000", 2024-09-21T15:00:48.851 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.948e-06 2024-09-21T15:00:48.851 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.852 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.852 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:48.852 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177113+0000", 2024-09-21T15:00:48.852 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035137855000000003 2024-09-21T15:00:48.852 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.852 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.852 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:48.852 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177121+0000", 2024-09-21T15:00:48.852 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8269999999999998e-06 2024-09-21T15:00:48.852 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:48.852 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:48.852 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:48.852 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.852 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.852 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45721 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:48.853 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141977+0000", 2024-09-21T15:00:48.853 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 98.638063650999996, 2024-09-21T15:00:48.853 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035169984000000001, 2024-09-21T15:00:48.853 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:48.853 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:48.853 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:48.853 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:48.853 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:48.853 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45721 2024-09-21T15:00:48.853 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.853 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:48.853 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.853 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:48.853 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141977+0000", 2024-09-21T15:00:48.854 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:48.854 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.854 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.854 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:48.854 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141977+0000", 2024-09-21T15:00:48.854 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:48.854 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.854 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.854 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:48.854 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141977+0000", 2024-09-21T15:00:48.854 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:48.854 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.854 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.855 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:48.855 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141979+0000", 2024-09-21T15:00:48.855 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.511e-06 2024-09-21T15:00:48.855 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.855 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.855 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:48.855 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141982+0000", 2024-09-21T15:00:48.855 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4180000000000001e-06 2024-09-21T15:00:48.855 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.855 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.855 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:48.855 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141985+0000", 2024-09-21T15:00:48.855 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9450000000000002e-06 2024-09-21T15:00:48.855 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.855 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.856 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:48.856 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177139+0000", 2024-09-21T15:00:48.856 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035154409999999997 2024-09-21T15:00:48.856 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.856 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.856 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:48.856 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177147+0000", 2024-09-21T15:00:48.856 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9570000000000002e-06 2024-09-21T15:00:48.856 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:48.856 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:48.856 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:48.856 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.856 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.856 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45722 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:48.857 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141987+0000", 2024-09-21T15:00:48.857 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 98.638053679999999, 2024-09-21T15:00:48.857 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035188492000000002, 2024-09-21T15:00:48.857 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:48.857 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:48.857 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:48.857 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:48.857 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:48.857 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45722 2024-09-21T15:00:48.858 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.858 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:48.858 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.858 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:48.858 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141987+0000", 2024-09-21T15:00:48.858 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:48.858 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.858 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.858 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:48.859 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141987+0000", 2024-09-21T15:00:48.859 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:48.859 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.859 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.859 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:48.859 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141987+0000", 2024-09-21T15:00:48.859 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:48.859 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.859 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.860 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:48.860 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141988+0000", 2024-09-21T15:00:48.860 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.463e-06 2024-09-21T15:00:48.860 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.860 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.860 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:48.860 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141994+0000", 2024-09-21T15:00:48.861 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.361e-06 2024-09-21T15:00:48.861 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.861 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.861 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:48.861 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141997+0000", 2024-09-21T15:00:48.861 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0359999999999998e-06 2024-09-21T15:00:48.861 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.861 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.861 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:48.861 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177169+0000", 2024-09-21T15:00:48.861 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035171651999999998 2024-09-21T15:00:48.862 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.862 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.862 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:48.862 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177176+0000", 2024-09-21T15:00:48.862 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2189999999999998e-06 2024-09-21T15:00:48.862 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:48.862 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:48.862 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:48.862 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.862 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.862 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45723 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:48.862 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141999+0000", 2024-09-21T15:00:48.862 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 98.638041513000005, 2024-09-21T15:00:48.863 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035197701999999997, 2024-09-21T15:00:48.863 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:48.863 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:48.863 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:48.863 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:48.863 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:48.863 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45723 2024-09-21T15:00:48.863 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.863 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:48.863 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.863 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:48.863 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141999+0000", 2024-09-21T15:00:48.864 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:48.864 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.864 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.864 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:48.864 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141999+0000", 2024-09-21T15:00:48.864 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:48.864 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.864 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.864 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:48.864 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141999+0000", 2024-09-21T15:00:48.864 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:48.864 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.864 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.864 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:48.865 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142003+0000", 2024-09-21T15:00:48.865 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5049999999999999e-06 2024-09-21T15:00:48.865 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.865 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.865 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:48.865 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142008+0000", 2024-09-21T15:00:48.865 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6579999999999999e-06 2024-09-21T15:00:48.865 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.865 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.865 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:48.865 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142011+0000", 2024-09-21T15:00:48.865 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0529999999999998e-06 2024-09-21T15:00:48.865 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.865 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.865 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:48.866 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177194+0000", 2024-09-21T15:00:48.866 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035182221999999999 2024-09-21T15:00:48.866 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.866 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.866 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:48.866 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177197+0000", 2024-09-21T15:00:48.866 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5030000000000002e-06 2024-09-21T15:00:48.866 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:48.866 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:48.866 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:48.866 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.866 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.866 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45724 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:48.867 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.142015+0000", 2024-09-21T15:00:48.867 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 98.638025545999994, 2024-09-21T15:00:48.867 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035195345000000003, 2024-09-21T15:00:48.867 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:48.867 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:48.867 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:48.867 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:48.867 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:48.867 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45724 2024-09-21T15:00:48.867 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.867 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:48.867 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.867 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:48.867 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142015+0000", 2024-09-21T15:00:48.868 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:48.868 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.868 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.868 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:48.868 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142015+0000", 2024-09-21T15:00:48.868 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:48.868 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.868 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.868 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:48.868 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142015+0000", 2024-09-21T15:00:48.868 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:48.868 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.868 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.868 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:48.868 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142017+0000", 2024-09-21T15:00:48.869 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5540000000000001e-06 2024-09-21T15:00:48.869 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.869 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.869 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:48.869 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142022+0000", 2024-09-21T15:00:48.869 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4229999999999996e-06 2024-09-21T15:00:48.869 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.869 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.869 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:48.869 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142025+0000", 2024-09-21T15:00:48.869 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.056e-06 2024-09-21T15:00:48.869 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.869 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.869 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:48.870 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177207+0000", 2024-09-21T15:00:48.870 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035182089 2024-09-21T15:00:48.870 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.870 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.870 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:48.870 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177211+0000", 2024-09-21T15:00:48.870 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4649999999999999e-06 2024-09-21T15:00:48.870 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:48.870 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:48.870 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:48.870 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.870 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.870 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45725 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:48.870 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.142028+0000", 2024-09-21T15:00:48.871 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 98.638013267000005, 2024-09-21T15:00:48.871 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035195731000000001, 2024-09-21T15:00:48.871 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:48.871 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:48.871 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:48.871 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:48.871 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:48.871 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45725 2024-09-21T15:00:48.871 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.871 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:48.871 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.871 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:48.871 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142028+0000", 2024-09-21T15:00:48.871 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:48.872 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.872 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.872 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:48.872 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142028+0000", 2024-09-21T15:00:48.872 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:48.872 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.872 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.872 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:48.872 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142027+0000", 2024-09-21T15:00:48.872 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:48.872 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.872 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.872 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:48.872 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142029+0000", 2024-09-21T15:00:48.873 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5269999999999999e-06 2024-09-21T15:00:48.873 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.873 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.873 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:48.873 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142032+0000", 2024-09-21T15:00:48.873 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3249999999999999e-06 2024-09-21T15:00:48.873 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.873 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.873 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:48.873 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142035+0000", 2024-09-21T15:00:48.873 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.897e-06 2024-09-21T15:00:48.873 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.873 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.873 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:48.873 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177220+0000", 2024-09-21T15:00:48.874 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035184871 2024-09-21T15:00:48.874 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.874 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.874 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:48.874 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177223+0000", 2024-09-21T15:00:48.874 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3680000000000002e-06 2024-09-21T15:00:48.874 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:48.874 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:48.874 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:48.874 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.875 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.875 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45726 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:48.875 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.142037+0000", 2024-09-21T15:00:48.875 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 98.638003552000001, 2024-09-21T15:00:48.875 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035199273000000003, 2024-09-21T15:00:48.875 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:48.875 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:48.875 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:48.875 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:48.875 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:48.876 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45726 2024-09-21T15:00:48.876 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.876 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:48.876 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.876 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:48.876 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142037+0000", 2024-09-21T15:00:48.876 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:48.876 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.876 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.876 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:48.876 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142037+0000", 2024-09-21T15:00:48.876 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:48.877 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.877 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.877 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:48.877 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142037+0000", 2024-09-21T15:00:48.877 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:48.877 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.877 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.877 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:48.877 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142038+0000", 2024-09-21T15:00:48.877 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.344e-06 2024-09-21T15:00:48.877 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.877 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.877 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:48.877 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142044+0000", 2024-09-21T15:00:48.878 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1930000000000001e-06 2024-09-21T15:00:48.878 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.878 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.878 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:48.878 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142047+0000", 2024-09-21T15:00:48.878 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0280000000000001e-06 2024-09-21T15:00:48.878 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.878 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.878 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:48.878 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177233+0000", 2024-09-21T15:00:48.878 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035186394000000003 2024-09-21T15:00:48.878 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:48.878 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:48.878 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:48.879 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177237+0000", 2024-09-21T15:00:48.879 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5530000000000001e-06 2024-09-21T15:00:48.879 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:48.879 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:48.879 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:48.879 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:48.879 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:48.879 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:48.881 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T15:00:48.881 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:00:48.877+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:00:48.981 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:00:48.977+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:00:49.040 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:49.040 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:00:49.041 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T15:00:49.041 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:49.050 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T15:00:49.082 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:00:49.077+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:00:49.161 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:49.162 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:00:49.162 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T15:00:49.162 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T15:00:49.162 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:49.175 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T15:00:49.183 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:00:49.179+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:00:49.284 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:00:49.281+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:00:49.286 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:49.286 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T15:00:49.286 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T15:00:49.287 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T15:00:49.287 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.287 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4281.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-21T15:00:49.287 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:09.324416+0000", 2024-09-21T15:00:49.287 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 159.96147748300001, 2024-09-21T15:00:49.287 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.117837947, 2024-09-21T15:00:49.287 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:49.287 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:49.287 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:49.287 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4281", 2024-09-21T15:00:49.287 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/1518520303", 2024-09-21T15:00:49.287 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1 2024-09-21T15:00:49.287 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.287 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:49.288 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.288 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:49.288 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324416+0000", 2024-09-21T15:00:49.288 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:49.288 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.288 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.288 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:49.288 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324416+0000", 2024-09-21T15:00:49.288 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:49.288 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.288 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.288 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:49.288 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324414+0000", 2024-09-21T15:00:49.288 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T15:00:49.288 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.289 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.289 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:49.289 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324429+0000", 2024-09-21T15:00:49.289 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5637e-05 2024-09-21T15:00:49.289 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.289 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.289 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:49.289 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324448+0000", 2024-09-21T15:00:49.289 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9042000000000002e-05 2024-09-21T15:00:49.289 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.289 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.289 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:49.289 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324465+0000", 2024-09-21T15:00:49.289 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6767999999999999e-05 2024-09-21T15:00:49.289 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.289 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.290 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:49.290 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324571+0000", 2024-09-21T15:00:49.290 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000105714 2024-09-21T15:00:49.290 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.290 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.290 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op must wait for map", 2024-09-21T15:00:49.290 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324587+0000", 2024-09-21T15:00:49.290 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5753999999999999e-05 2024-09-21T15:00:49.290 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.290 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.290 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:49.290 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.325864+0000", 2024-09-21T15:00:49.290 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0012773350000000001 2024-09-21T15:00:49.290 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.291 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.291 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:00:49.291 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.325878+0000", 2024-09-21T15:00:49.291 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4498000000000001e-05 2024-09-21T15:00:49.291 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.291 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.291 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:49.291 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.439461+0000", 2024-09-21T15:00:49.291 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.11358212500000001 2024-09-21T15:00:49.291 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.291 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.291 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:49.291 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.439848+0000", 2024-09-21T15:00:49.291 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00038747800000000002 2024-09-21T15:00:49.291 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.292 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.292 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T15:00:49.292 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.440175+0000", 2024-09-21T15:00:49.292 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00032721900000000001 2024-09-21T15:00:49.292 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.292 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.292 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:49.292 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.441770+0000", 2024-09-21T15:00:49.292 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0015949409999999999 2024-09-21T15:00:49.292 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.292 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.292 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:49.292 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.442086+0000", 2024-09-21T15:00:49.292 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00031586299999999999 2024-09-21T15:00:49.292 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.292 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.292 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:49.293 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.442195+0000", 2024-09-21T15:00:49.293 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000108791 2024-09-21T15:00:49.293 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.293 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.293 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:49.293 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.442254+0000", 2024-09-21T15:00:49.293 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9038e-05 2024-09-21T15:00:49.293 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:49.293 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:49.293 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:49.293 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.293 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.293 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4281.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-21T15:00:49.293 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:09.447828+0000", 2024-09-21T15:00:49.293 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 159.83806548199999, 2024-09-21T15:00:49.294 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.32904393700000001, 2024-09-21T15:00:49.294 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:49.294 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:49.294 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:49.294 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4281", 2024-09-21T15:00:49.294 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/1518520303", 2024-09-21T15:00:49.294 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 2 2024-09-21T15:00:49.294 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.294 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:49.294 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.294 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:49.294 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447828+0000", 2024-09-21T15:00:49.294 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:49.295 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.295 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.295 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:49.295 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447828+0000", 2024-09-21T15:00:49.295 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:49.295 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.295 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.295 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:49.295 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447825+0000", 2024-09-21T15:00:49.295 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T15:00:49.295 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.295 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.295 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:49.295 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447841+0000", 2024-09-21T15:00:49.295 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5359999999999999e-05 2024-09-21T15:00:49.296 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.296 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.296 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:49.296 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447875+0000", 2024-09-21T15:00:49.296 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4388999999999999e-05 2024-09-21T15:00:49.296 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.296 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.296 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:49.296 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447903+0000", 2024-09-21T15:00:49.296 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7373e-05 2024-09-21T15:00:49.296 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.296 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.296 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:49.296 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447996+0000", 2024-09-21T15:00:49.296 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2954999999999993e-05 2024-09-21T15:00:49.297 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.297 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.297 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:00:49.297 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.448011+0000", 2024-09-21T15:00:49.297 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5038999999999999e-05 2024-09-21T15:00:49.297 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.297 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.297 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:49.297 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.776501+0000", 2024-09-21T15:00:49.297 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.32849012500000002 2024-09-21T15:00:49.297 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.297 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.297 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:49.297 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.776872+0000", 2024-09-21T15:00:49.298 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00037119600000000002 2024-09-21T15:00:49.298 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:49.298 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:49.298 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:49.298 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.298 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.298 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:97673 3.1d 3:bd91946e:::rbd_data.1136bfc79a70.0000000000000002:head [write 0~4718592 [fadvise_sequential+fadvise_nocache],write 5242880~32768 [fadvise_sequential+fadvise_nocache],write 5341184~147456 [fadvise_sequential+fadvise_nocache],write 5554176~147456 [fadvise_sequential+fadvise_nocache],write 5767168~16384 [fadvise_sequential+fadvise_nocache],write 5849088~147456 [fadvise_sequential+fadvise_nocache],write 6062080~147456 [fadvise_sequential+fadvise_nocache],write 6275072~16384 [fadvise_sequential+fadvise_nocache],write 6356992~147456 [fadvise_sequential+fadvise_nocache],write 6569984~147456 [fadvise_sequential+fadvise_nocache],write 6782976~32768 [fadvise_sequential+fadvise_nocache],write 6864896~147456 [fadvise_sequential+fadvise_nocache],write 7077888~147456 [fadvise_sequential+fadvise_nocache],write 7290880~49152 [fadvise_sequential+fadvise_nocache],write 7372800~147456 [fadvise_sequential+fadvise_nocache],write 7585792~147456 [fadvise_sequential+fadvise_nocache],write 7798784~65536 [fadvise_sequential+fadvise_nocache],write 7880704~147456 [fadvise_sequential+fadvise_nocache],write 8093696~147456 [fadvise_sequential+fadvise_nocache],write 8306688~81920 [fadvise_sequential+fadvise_nocache],write 8421376~114688 [fadvise_sequential+fadvise_nocache],write 8634368~114688 [fadvise_sequential+fadvise_nocache],write 8847360~65536 [fadvise_sequential+fadvise_nocache],write 9011200~32768 [fadvise_sequential+fadvise_nocache],write 9224192~32768 [fadvise_sequential+fadvise_nocache],write 9519104~32768 [fadvise_sequential+fadvise_nocache],write 9732096~32768 [fadvise_sequential+fadvise_nocache],write 9945088~16384 [fadvise_sequential+fadvise_nocache],write 9977856~196608 [fadvise_sequential+fadvise_nocache],write 10190848~196608 [fadvise_sequential+fadvise_nocache],write 10403840~278528 [fadvise_sequential+fadvise_nocache],write 10698752~196608 [fadvise_sequential+fadvise_nocache],write 10911744~278528 [fadvise_sequential+fadvise_nocache],write 11206656~196608 [fadvise_sequential+fadvise_nocache],write 11419648~196608 [fadvise_sequential+fadvise_nocache],write 11632640~65536 [fadvise_sequential+fadvise_nocache],write 11730944~98304 [fadvise_sequential+fadvise_nocache],write 11845632~65536 [fadvise_sequential+fadvise_nocache],write 11943936~98304 [fadvise_sequential+fadvise_nocache],write 12058624~65536 [fadvise_sequential+fadvise_nocache],write 12140544~65536 [fadvise_sequential+fadvise_nocache],write 12238848~98304 [fadvise_sequential+fadvise_nocache],write 12353536~65536 [fadvise_sequential+fadvise_nocache],write 12451840~98304 [fadvise_sequential+fadvise_nocache],write 12566528~65536 [fadvise_sequential+fadvise_nocache],write 12746752~98304 [fadvise_sequential+fadvise_nocache],write 12959744~98304 [fadvise_sequential+fadvise_nocache],write 13107200~32768 [fadvise_sequential+fadvise_nocache],write 13254656~98304 [fadvise_sequential+fadvise_nocache],write 13467648~98304 [fadvise_sequential+fadvise_nocache],write 13631488~16384 [fadvise_sequential+fadvise_nocache],write 13762560~98304 [fadvise_sequential+fadvise_nocache],write 13975552~98304 [fadvise_sequential+fadvise_nocache],write 15204352~49152 [fadvise_sequential+fadvise_nocache],write 15384576~81920 [fadvise_sequential+fadvise_nocache],write 15597568~81920 [fadvise_sequential+fadvise_nocache],write 15728640~32768 [fadvise_sequential+fadvise_nocache],write 15892480~81920 [fadvise_sequential+fadvise_nocache],write 16105472~81920 [fadvise_sequential+fadvise_nocache],write 16252928~16384 [fadvise_sequential+fadvise_nocache],write 16400384~81920 [fadvise_sequential+fadvise_nocache],write 16613376~81920 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:49.298 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.300719+0000", 2024-09-21T15:00:49.298 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 43.985174710000003, 2024-09-21T15:00:49.298 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.068947114000000004, 2024-09-21T15:00:49.298 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:49.298 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:49.298 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:49.298 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:49.298 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:49.299 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 97673 2024-09-21T15:00:49.299 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.299 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:49.299 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.299 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:49.299 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.300719+0000", 2024-09-21T15:00:49.299 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:49.299 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.299 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.299 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:49.299 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.300719+0000", 2024-09-21T15:00:49.299 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:49.299 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.299 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.299 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:49.300 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.300716+0000", 2024-09-21T15:00:49.300 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-21T15:00:49.300 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.300 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.300 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:49.300 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.308380+0000", 2024-09-21T15:00:49.300 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0076638519999999996 2024-09-21T15:00:49.300 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.300 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.300 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:49.300 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.308418+0000", 2024-09-21T15:00:49.300 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.8380000000000002e-05 2024-09-21T15:00:49.300 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.300 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.300 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:49.301 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.308431+0000", 2024-09-21T15:00:49.301 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3213999999999999e-05 2024-09-21T15:00:49.301 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.301 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.301 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:49.301 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.308562+0000", 2024-09-21T15:00:49.301 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013015999999999999 2024-09-21T15:00:49.301 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.301 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.301 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:49.301 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.309173+0000", 2024-09-21T15:00:49.301 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00061185500000000002 2024-09-21T15:00:49.301 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.301 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.301 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 0", 2024-09-21T15:00:49.302 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311612+0000", 2024-09-21T15:00:49.302 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0024383949999999999 2024-09-21T15:00:49.302 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.302 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.302 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:49.302 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.355482+0000", 2024-09-21T15:00:49.302 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.043869940000000003 2024-09-21T15:00:49.302 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.302 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.302 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:49.302 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.369306+0000", 2024-09-21T15:00:49.302 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013824279 2024-09-21T15:00:49.302 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.302 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.302 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:49.303 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.369575+0000", 2024-09-21T15:00:49.303 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000269404 2024-09-21T15:00:49.303 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.303 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.303 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:49.303 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.369666+0000", 2024-09-21T15:00:49.303 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0426000000000005e-05 2024-09-21T15:00:49.303 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:49.303 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:49.303 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:49.303 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.303 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.303 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:112719 3.13 3:cd56ff5a:::rbd_data.1136b1194d20.0000000000000002:58 [read 1998848~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:49.303 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.387056+0000", 2024-09-21T15:00:49.304 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 19.898837363999998, 2024-09-21T15:00:49.304 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055608623000000003, 2024-09-21T15:00:49.304 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:49.304 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:49.304 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:49.304 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:49.304 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:49.304 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 112719 2024-09-21T15:00:49.304 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.304 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:49.304 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.304 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:49.304 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.387056+0000", 2024-09-21T15:00:49.304 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:49.304 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.305 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.305 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:49.305 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.387056+0000", 2024-09-21T15:00:49.305 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:49.305 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.305 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.305 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:49.305 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.387056+0000", 2024-09-21T15:00:49.305 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:49.305 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.305 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.305 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:49.305 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.387058+0000", 2024-09-21T15:00:49.305 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.683e-06 2024-09-21T15:00:49.305 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.305 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.306 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:49.306 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.387064+0000", 2024-09-21T15:00:49.306 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6720000000000001e-06 2024-09-21T15:00:49.306 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.306 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.306 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:49.306 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.387069+0000", 2024-09-21T15:00:49.306 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8999999999999997e-06 2024-09-21T15:00:49.306 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.306 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.306 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:49.306 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.442514+0000", 2024-09-21T15:00:49.306 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055444881000000001 2024-09-21T15:00:49.306 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.306 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.307 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:49.307 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.442565+0000", 2024-09-21T15:00:49.307 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1010000000000001e-05 2024-09-21T15:00:49.307 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.307 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.307 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:49.307 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.442665+0000", 2024-09-21T15:00:49.307 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9840000000000006e-05 2024-09-21T15:00:49.307 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:49.307 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:49.307 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:49.307 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.307 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.307 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:112720 3.13 3:cd56ff5a:::rbd_data.1136b1194d20.0000000000000002:58 [read 2088960~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:49.307 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.387074+0000", 2024-09-21T15:00:49.307 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 19.898819848999999, 2024-09-21T15:00:49.308 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055824591999999999, 2024-09-21T15:00:49.308 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:49.308 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:49.308 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:49.308 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:49.308 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:49.308 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 112720 2024-09-21T15:00:49.308 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.308 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:49.308 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.308 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:49.308 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.387074+0000", 2024-09-21T15:00:49.308 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:49.308 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.308 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.308 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:49.309 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.387074+0000", 2024-09-21T15:00:49.309 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:49.309 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.309 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.309 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:49.309 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.387073+0000", 2024-09-21T15:00:49.309 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:49.309 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.309 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.309 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:49.309 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.387076+0000", 2024-09-21T15:00:49.309 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4710000000000001e-06 2024-09-21T15:00:49.309 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.309 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.309 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:49.309 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.387085+0000", 2024-09-21T15:00:49.310 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2110000000000004e-06 2024-09-21T15:00:49.310 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.310 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.310 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:49.310 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.387090+0000", 2024-09-21T15:00:49.310 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.904e-06 2024-09-21T15:00:49.310 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.310 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.310 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:49.310 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.442702+0000", 2024-09-21T15:00:49.310 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055612251000000001 2024-09-21T15:00:49.310 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.310 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.310 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:49.310 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.442780+0000", 2024-09-21T15:00:49.311 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8232999999999994e-05 2024-09-21T15:00:49.311 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.311 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.311 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:49.311 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.442898+0000", 2024-09-21T15:00:49.311 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000117889 2024-09-21T15:00:49.311 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:49.311 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:49.311 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:49.311 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.311 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.311 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:112721 3.13 3:cd56ff5a:::rbd_data.1136b1194d20.0000000000000002:58 [read 2105344~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:49.311 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.387095+0000", 2024-09-21T15:00:49.311 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 19.898798759000002, 2024-09-21T15:00:49.311 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.056000182000000003, 2024-09-21T15:00:49.311 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:49.312 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:49.312 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:49.312 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:49.312 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:49.312 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 112721 2024-09-21T15:00:49.312 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.312 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:49.312 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.312 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:49.312 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.387095+0000", 2024-09-21T15:00:49.312 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:49.312 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.312 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.312 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:49.312 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.387095+0000", 2024-09-21T15:00:49.312 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:49.313 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.313 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.313 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:49.313 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.387094+0000", 2024-09-21T15:00:49.313 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:49.313 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.313 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.313 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:49.313 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.387097+0000", 2024-09-21T15:00:49.313 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5940000000000001e-06 2024-09-21T15:00:49.313 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.313 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.313 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:49.314 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.387106+0000", 2024-09-21T15:00:49.314 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.9439999999999994e-06 2024-09-21T15:00:49.314 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.314 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.314 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:49.314 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.387111+0000", 2024-09-21T15:00:49.314 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3900000000000001e-06 2024-09-21T15:00:49.314 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.314 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.314 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:49.314 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.442934+0000", 2024-09-21T15:00:49.314 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055822996 2024-09-21T15:00:49.314 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.314 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.314 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:49.314 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.442998+0000", 2024-09-21T15:00:49.315 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3746999999999998e-05 2024-09-21T15:00:49.315 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.315 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.315 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:49.315 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.443095+0000", 2024-09-21T15:00:49.315 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6848999999999994e-05 2024-09-21T15:00:49.315 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:49.315 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:49.315 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:49.315 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.315 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.315 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:112733 3.13 3:cd56ff5a:::rbd_data.1136b1194d20.0000000000000002:58 [read 3186688~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:49.315 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.390744+0000", 2024-09-21T15:00:49.315 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 19.895149580999998, 2024-09-21T15:00:49.315 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055498697999999999, 2024-09-21T15:00:49.316 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:49.316 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:49.316 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:49.316 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:49.316 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:49.316 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 112733 2024-09-21T15:00:49.316 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.316 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:49.316 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.316 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:49.316 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390744+0000", 2024-09-21T15:00:49.316 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:49.316 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.316 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.316 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:49.316 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390744+0000", 2024-09-21T15:00:49.316 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:49.317 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.317 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.317 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:49.317 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390743+0000", 2024-09-21T15:00:49.317 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:49.317 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.317 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.317 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:49.317 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390746+0000", 2024-09-21T15:00:49.317 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9510000000000002e-06 2024-09-21T15:00:49.317 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.317 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.317 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:49.317 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390751+0000", 2024-09-21T15:00:49.317 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.357e-06 2024-09-21T15:00:49.317 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.318 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.318 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:49.318 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390753+0000", 2024-09-21T15:00:49.318 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7310000000000002e-06 2024-09-21T15:00:49.318 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.318 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.318 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:49.318 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.445994+0000", 2024-09-21T15:00:49.318 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055240247999999999 2024-09-21T15:00:49.318 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.318 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.318 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:49.318 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.446111+0000", 2024-09-21T15:00:49.318 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00011699799999999999 2024-09-21T15:00:49.318 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.318 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.319 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:49.319 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.446243+0000", 2024-09-21T15:00:49.319 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013196099999999999 2024-09-21T15:00:49.319 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:49.319 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:49.319 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:49.319 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.319 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.319 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:112734 3.13 3:cd56ff5a:::rbd_data.1136b1194d20.0000000000000002:58 [read 3276800~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:49.319 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.390755+0000", 2024-09-21T15:00:49.319 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 19.895137957999999, 2024-09-21T15:00:49.319 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.05569313, 2024-09-21T15:00:49.319 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:49.319 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:49.319 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:49.319 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:49.320 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:49.320 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 112734 2024-09-21T15:00:49.320 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.320 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:49.320 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.320 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:49.320 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390755+0000", 2024-09-21T15:00:49.320 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:49.320 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.320 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.320 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:49.320 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390755+0000", 2024-09-21T15:00:49.320 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:49.320 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.320 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.320 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:49.321 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390755+0000", 2024-09-21T15:00:49.321 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:00:49.321 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.321 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.321 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:49.321 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390757+0000", 2024-09-21T15:00:49.321 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5060000000000001e-06 2024-09-21T15:00:49.321 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.321 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.321 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:49.321 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390760+0000", 2024-09-21T15:00:49.321 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.79e-06 2024-09-21T15:00:49.321 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.321 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.321 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:49.321 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390762+0000", 2024-09-21T15:00:49.322 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.661e-06 2024-09-21T15:00:49.322 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.322 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.322 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:49.322 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.446280+0000", 2024-09-21T15:00:49.322 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055517761999999998 2024-09-21T15:00:49.322 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.322 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.322 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:49.322 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.446365+0000", 2024-09-21T15:00:49.322 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5208000000000001e-05 2024-09-21T15:00:49.322 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.322 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.322 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:49.322 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.446449+0000", 2024-09-21T15:00:49.323 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3417000000000004e-05 2024-09-21T15:00:49.323 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:49.323 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:49.323 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:49.323 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.323 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.323 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:112735 3.13 3:cd56ff5a:::rbd_data.1136b1194d20.0000000000000002:58 [read 3366912~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:49.323 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.390764+0000", 2024-09-21T15:00:49.323 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 19.895129147999999, 2024-09-21T15:00:49.323 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055846502999999999, 2024-09-21T15:00:49.323 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:49.323 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:49.323 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:49.323 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:49.323 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:49.324 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 112735 2024-09-21T15:00:49.324 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.324 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:49.324 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.324 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:49.324 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390764+0000", 2024-09-21T15:00:49.324 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:49.324 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.324 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.324 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:49.324 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390764+0000", 2024-09-21T15:00:49.324 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:49.324 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.324 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.324 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:49.325 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390764+0000", 2024-09-21T15:00:49.325 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:00:49.325 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.325 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.325 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:49.325 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390771+0000", 2024-09-21T15:00:49.325 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.1380000000000001e-06 2024-09-21T15:00:49.325 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.325 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.325 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:49.325 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390774+0000", 2024-09-21T15:00:49.325 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1370000000000002e-06 2024-09-21T15:00:49.325 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.325 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.325 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:49.326 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390777+0000", 2024-09-21T15:00:49.326 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7470000000000001e-06 2024-09-21T15:00:49.326 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.326 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.326 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:49.326 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.446488+0000", 2024-09-21T15:00:49.326 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055711016000000002 2024-09-21T15:00:49.326 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.326 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.326 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:49.326 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.446543+0000", 2024-09-21T15:00:49.326 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5148999999999997e-05 2024-09-21T15:00:49.326 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.327 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.327 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:49.327 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.446611+0000", 2024-09-21T15:00:49.327 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7533999999999994e-05 2024-09-21T15:00:49.327 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:49.327 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:49.327 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:49.327 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.327 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.327 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:112736 3.13 3:cd56ff5a:::rbd_data.1136b1194d20.0000000000000002:58 [read 3457024~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:49.327 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.390779+0000", 2024-09-21T15:00:49.327 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 19.895114203999999, 2024-09-21T15:00:49.327 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.056179989, 2024-09-21T15:00:49.327 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:49.327 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:49.328 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:49.328 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:49.328 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:49.328 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 112736 2024-09-21T15:00:49.328 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.328 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:49.328 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.328 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:49.328 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390779+0000", 2024-09-21T15:00:49.328 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:49.328 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.328 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.328 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:49.328 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390779+0000", 2024-09-21T15:00:49.328 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:49.328 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.329 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.329 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:49.329 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390779+0000", 2024-09-21T15:00:49.329 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:00:49.329 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.329 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.329 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:49.329 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390780+0000", 2024-09-21T15:00:49.329 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4360000000000001e-06 2024-09-21T15:00:49.329 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.329 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.329 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:49.329 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390783+0000", 2024-09-21T15:00:49.329 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.818e-06 2024-09-21T15:00:49.329 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.329 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.330 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:49.330 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390786+0000", 2024-09-21T15:00:49.330 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4439999999999998e-06 2024-09-21T15:00:49.330 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.330 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.330 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:49.330 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.446668+0000", 2024-09-21T15:00:49.330 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055881910999999999 2024-09-21T15:00:49.330 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.330 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.330 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:49.330 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.446797+0000", 2024-09-21T15:00:49.330 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012924599999999999 2024-09-21T15:00:49.330 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.330 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.330 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:49.331 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.446959+0000", 2024-09-21T15:00:49.331 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000162356 2024-09-21T15:00:49.331 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:49.331 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:49.331 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:49.331 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.331 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.331 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:118215 3.d 3:b5b5db2f:::rbd_data.1136b1194d20.0000000000000005:58 [read 5160960~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:49.331 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.833406+0000", 2024-09-21T15:00:49.331 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 19.452487906000002, 2024-09-21T15:00:49.331 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055458362999999997, 2024-09-21T15:00:49.331 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:49.331 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:49.331 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:49.331 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:49.332 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:49.332 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 118215 2024-09-21T15:00:49.332 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.332 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:49.332 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.332 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:49.332 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833406+0000", 2024-09-21T15:00:49.332 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:49.332 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.332 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.332 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:49.332 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833406+0000", 2024-09-21T15:00:49.332 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:49.332 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.332 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.332 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:49.333 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833405+0000", 2024-09-21T15:00:49.333 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:49.333 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.333 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.333 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:49.333 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833409+0000", 2024-09-21T15:00:49.333 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.8099999999999999e-06 2024-09-21T15:00:49.333 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.333 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.333 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:49.333 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833416+0000", 2024-09-21T15:00:49.333 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3989999999999998e-06 2024-09-21T15:00:49.333 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.333 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.334 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:49.334 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833420+0000", 2024-09-21T15:00:49.334 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7189999999999999e-06 2024-09-21T15:00:49.334 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.334 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.334 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:49.334 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.888444+0000", 2024-09-21T15:00:49.334 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055024311999999999 2024-09-21T15:00:49.334 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.334 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.334 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:49.334 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.888630+0000", 2024-09-21T15:00:49.334 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00018579800000000001 2024-09-21T15:00:49.334 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.335 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.335 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:49.335 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.888864+0000", 2024-09-21T15:00:49.335 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000234009 2024-09-21T15:00:49.335 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:49.335 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:49.335 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:49.335 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.335 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.335 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:118216 3.d 3:b5b5db2f:::rbd_data.1136b1194d20.0000000000000005:58 [read 5251072~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:49.335 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.833425+0000", 2024-09-21T15:00:49.335 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 19.452468474, 2024-09-21T15:00:49.335 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055891158000000003, 2024-09-21T15:00:49.335 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:49.335 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:49.336 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:49.336 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:49.336 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:49.336 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 118216 2024-09-21T15:00:49.336 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.336 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:49.336 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.336 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:49.336 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833425+0000", 2024-09-21T15:00:49.336 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:49.336 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.336 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.336 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:49.336 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833425+0000", 2024-09-21T15:00:49.336 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:49.337 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.337 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.337 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:49.337 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833425+0000", 2024-09-21T15:00:49.337 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:49.337 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.337 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.337 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:49.337 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833427+0000", 2024-09-21T15:00:49.337 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.137e-06 2024-09-21T15:00:49.337 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.337 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.337 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:49.337 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833430+0000", 2024-09-21T15:00:49.337 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5379999999999998e-06 2024-09-21T15:00:49.337 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.337 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.338 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:49.338 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833433+0000", 2024-09-21T15:00:49.338 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0599999999999999e-06 2024-09-21T15:00:49.338 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.338 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.338 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:49.338 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.888919+0000", 2024-09-21T15:00:49.338 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055485381 2024-09-21T15:00:49.338 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.338 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.338 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:49.338 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.889088+0000", 2024-09-21T15:00:49.338 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000169364 2024-09-21T15:00:49.338 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.338 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.339 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:49.339 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.889316+0000", 2024-09-21T15:00:49.339 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00022798600000000001 2024-09-21T15:00:49.339 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:49.339 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:49.339 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:49.339 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.339 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.339 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:118217 3.d 3:b5b5db2f:::rbd_data.1136b1194d20.0000000000000005:58 [read 5341184~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:49.339 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.833436+0000", 2024-09-21T15:00:49.339 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 19.452457046999999, 2024-09-21T15:00:49.339 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.056045230000000001, 2024-09-21T15:00:49.339 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:49.339 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:49.339 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:49.340 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:49.340 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:49.340 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 118217 2024-09-21T15:00:49.340 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.340 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:49.340 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.340 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:49.340 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833436+0000", 2024-09-21T15:00:49.340 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:49.340 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.340 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.340 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:49.340 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833436+0000", 2024-09-21T15:00:49.340 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:49.340 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.340 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.340 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:49.341 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833436+0000", 2024-09-21T15:00:49.341 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:49.341 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.341 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.341 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:49.341 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833438+0000", 2024-09-21T15:00:49.341 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.826e-06 2024-09-21T15:00:49.341 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.341 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.341 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:49.341 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833441+0000", 2024-09-21T15:00:49.341 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3280000000000002e-06 2024-09-21T15:00:49.341 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.341 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.341 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:49.341 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833447+0000", 2024-09-21T15:00:49.342 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2660000000000001e-06 2024-09-21T15:00:49.342 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.342 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.342 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:49.342 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.889362+0000", 2024-09-21T15:00:49.342 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055915323000000003 2024-09-21T15:00:49.342 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.342 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.342 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:49.342 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.889414+0000", 2024-09-21T15:00:49.342 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1711e-05 2024-09-21T15:00:49.342 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.342 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.342 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:49.342 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.889482+0000", 2024-09-21T15:00:49.343 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8056e-05 2024-09-21T15:00:49.343 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:49.343 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:49.343 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:49.343 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.343 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.343 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:118218 3.d 3:b5b5db2f:::rbd_data.1136b1194d20.0000000000000005:58 [read 5431296~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:49.343 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.833451+0000", 2024-09-21T15:00:49.343 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 19.452442011999999, 2024-09-21T15:00:49.343 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.056286765000000002, 2024-09-21T15:00:49.343 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:49.343 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:49.343 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:49.343 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:49.343 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:49.343 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 118218 2024-09-21T15:00:49.344 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.344 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:49.344 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.344 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:49.344 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833451+0000", 2024-09-21T15:00:49.344 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:49.344 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.344 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.344 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:49.344 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833451+0000", 2024-09-21T15:00:49.344 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:49.344 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.344 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.344 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:49.344 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833451+0000", 2024-09-21T15:00:49.345 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:49.345 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.345 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.345 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:49.345 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833454+0000", 2024-09-21T15:00:49.345 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9670000000000001e-06 2024-09-21T15:00:49.345 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.345 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.345 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:49.345 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833461+0000", 2024-09-21T15:00:49.345 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0489999999999998e-06 2024-09-21T15:00:49.345 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.345 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.345 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:49.345 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833468+0000", 2024-09-21T15:00:49.345 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0310000000000001e-06 2024-09-21T15:00:49.346 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.346 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.346 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:49.346 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.889539+0000", 2024-09-21T15:00:49.346 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.056071217 2024-09-21T15:00:49.346 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.346 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.346 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:49.346 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.889621+0000", 2024-09-21T15:00:49.346 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1528999999999998e-05 2024-09-21T15:00:49.346 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.346 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.346 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:49.346 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.889738+0000", 2024-09-21T15:00:49.346 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00011759799999999999 2024-09-21T15:00:49.346 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:49.347 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:49.347 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:49.347 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.347 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.347 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:118220 3.d 3:b5b5db2f:::rbd_data.1136b1194d20.0000000000000005:58 [read 5611520~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:49.347 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.834528+0000", 2024-09-21T15:00:49.347 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 19.451365023000001, 2024-09-21T15:00:49.347 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055660376999999997, 2024-09-21T15:00:49.347 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:49.347 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:49.347 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:49.347 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:49.347 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:49.347 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 118220 2024-09-21T15:00:49.347 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.347 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:49.348 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.348 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:49.348 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834528+0000", 2024-09-21T15:00:49.348 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:49.348 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.348 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.348 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:49.348 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834528+0000", 2024-09-21T15:00:49.348 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:49.348 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.348 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.348 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:49.348 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834528+0000", 2024-09-21T15:00:49.348 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:49.349 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.349 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.349 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:49.349 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834532+0000", 2024-09-21T15:00:49.349 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.9509999999999999e-06 2024-09-21T15:00:49.349 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.349 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.349 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:49.349 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834537+0000", 2024-09-21T15:00:49.349 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6899999999999997e-06 2024-09-21T15:00:49.349 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.349 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.349 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:49.349 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834541+0000", 2024-09-21T15:00:49.349 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.76e-06 2024-09-21T15:00:49.349 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.349 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.350 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:49.350 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.889962+0000", 2024-09-21T15:00:49.350 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055421249999999998 2024-09-21T15:00:49.350 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.350 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.350 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:49.350 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.890074+0000", 2024-09-21T15:00:49.350 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00011124300000000001 2024-09-21T15:00:49.350 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.350 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.350 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:49.350 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.890189+0000", 2024-09-21T15:00:49.350 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000115151 2024-09-21T15:00:49.350 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:49.350 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:49.351 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:49.351 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.351 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.351 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:118221 3.d 3:b5b5db2f:::rbd_data.1136b1194d20.0000000000000005:58 [read 5701632~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:49.351 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.834544+0000", 2024-09-21T15:00:49.351 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 19.451349101000002, 2024-09-21T15:00:49.351 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.056071875, 2024-09-21T15:00:49.351 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:49.351 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:49.351 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:49.351 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:49.351 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:49.351 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 118221 2024-09-21T15:00:49.351 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.351 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:49.351 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.352 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:49.352 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834544+0000", 2024-09-21T15:00:49.352 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:49.352 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.352 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.352 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:49.352 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834544+0000", 2024-09-21T15:00:49.352 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:49.352 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.352 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.352 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:49.352 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834544+0000", 2024-09-21T15:00:49.352 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:49.352 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.352 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.352 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:49.353 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834546+0000", 2024-09-21T15:00:49.353 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.458e-06 2024-09-21T15:00:49.353 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.353 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.353 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:49.353 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834550+0000", 2024-09-21T15:00:49.353 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.563e-06 2024-09-21T15:00:49.353 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.353 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.353 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:49.353 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834553+0000", 2024-09-21T15:00:49.353 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.253e-06 2024-09-21T15:00:49.353 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.353 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.353 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:49.354 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.890246+0000", 2024-09-21T15:00:49.354 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055692561000000002 2024-09-21T15:00:49.354 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.354 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.354 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:49.354 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.890397+0000", 2024-09-21T15:00:49.354 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015129999999999999 2024-09-21T15:00:49.354 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.354 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.354 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:49.354 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.890616+0000", 2024-09-21T15:00:49.354 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00021912799999999999 2024-09-21T15:00:49.354 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:49.354 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:49.354 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:49.355 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.355 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.355 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:118222 3.d 3:b5b5db2f:::rbd_data.1136b1194d20.0000000000000005:58 [read 5791744~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:49.355 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.834557+0000", 2024-09-21T15:00:49.355 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 19.45133629, 2024-09-21T15:00:49.355 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.056417907000000003, 2024-09-21T15:00:49.355 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:49.355 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:49.355 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:49.355 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:49.355 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:49.355 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 118222 2024-09-21T15:00:49.355 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.355 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:49.356 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.356 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:49.356 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834557+0000", 2024-09-21T15:00:49.356 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:49.356 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.356 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.356 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:49.356 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834557+0000", 2024-09-21T15:00:49.356 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:49.356 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.356 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.356 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:49.356 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834557+0000", 2024-09-21T15:00:49.356 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:49.356 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.357 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.357 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:49.357 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834559+0000", 2024-09-21T15:00:49.357 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.956e-06 2024-09-21T15:00:49.357 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.357 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.357 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:49.357 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834562+0000", 2024-09-21T15:00:49.357 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.518e-06 2024-09-21T15:00:49.357 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.357 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.357 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:49.357 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834566+0000", 2024-09-21T15:00:49.357 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.529e-06 2024-09-21T15:00:49.357 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.358 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.358 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:49.358 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.890671+0000", 2024-09-21T15:00:49.358 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.056105255 2024-09-21T15:00:49.358 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.358 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.358 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:49.358 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.890788+0000", 2024-09-21T15:00:49.358 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00011715899999999999 2024-09-21T15:00:49.358 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.358 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.358 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:49.358 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.890975+0000", 2024-09-21T15:00:49.358 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00018678100000000001 2024-09-21T15:00:49.359 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:49.359 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:49.359 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:49.359 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.359 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.359 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:118223 3.d 3:b5b5db2f:::rbd_data.1136b1194d20.0000000000000005:58 [read 5881856~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:49.359 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.834569+0000", 2024-09-21T15:00:49.359 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 19.451324733, 2024-09-21T15:00:49.359 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.056618784999999998, 2024-09-21T15:00:49.359 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:49.359 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:49.359 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:49.359 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:49.359 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:49.359 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 118223 2024-09-21T15:00:49.360 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.360 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:49.360 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.360 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:49.360 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834569+0000", 2024-09-21T15:00:49.360 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:49.360 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.360 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.360 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:49.360 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834569+0000", 2024-09-21T15:00:49.360 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:49.360 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.360 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.360 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:49.360 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834568+0000", 2024-09-21T15:00:49.360 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:49.361 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.361 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.361 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:49.361 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834570+0000", 2024-09-21T15:00:49.361 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8309999999999999e-06 2024-09-21T15:00:49.361 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.361 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.361 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:49.361 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834574+0000", 2024-09-21T15:00:49.361 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3720000000000001e-06 2024-09-21T15:00:49.361 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.361 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.361 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:49.361 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834577+0000", 2024-09-21T15:00:49.361 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3019999999999999e-06 2024-09-21T15:00:49.361 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.362 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.362 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:49.362 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.891014+0000", 2024-09-21T15:00:49.362 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.056437322999999998 2024-09-21T15:00:49.362 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.362 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.362 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:49.362 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.891101+0000", 2024-09-21T15:00:49.362 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6341999999999994e-05 2024-09-21T15:00:49.362 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.362 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.362 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:49.362 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.891187+0000", 2024-09-21T15:00:49.362 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6889000000000001e-05 2024-09-21T15:00:49.362 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:49.363 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:49.363 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:49.363 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.363 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.363 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:118224 3.d 3:b5b5db2f:::rbd_data.1136b1194d20.0000000000000005:58 [read 5971968~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:49.363 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.834580+0000", 2024-09-21T15:00:49.363 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 19.451313765999998, 2024-09-21T15:00:49.363 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.056770289000000002, 2024-09-21T15:00:49.363 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:49.363 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:49.363 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:49.363 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:49.363 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:49.364 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 118224 2024-09-21T15:00:49.364 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.364 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:49.364 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.364 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:49.364 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834580+0000", 2024-09-21T15:00:49.364 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:49.364 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.364 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.364 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:49.364 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834580+0000", 2024-09-21T15:00:49.364 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:49.364 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.364 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.364 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:49.365 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834579+0000", 2024-09-21T15:00:49.365 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:49.365 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.365 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.365 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:49.365 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834581+0000", 2024-09-21T15:00:49.365 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8160000000000001e-06 2024-09-21T15:00:49.365 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.365 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.365 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:49.365 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834585+0000", 2024-09-21T15:00:49.365 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.568e-06 2024-09-21T15:00:49.365 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.365 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.365 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:49.366 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834588+0000", 2024-09-21T15:00:49.366 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1810000000000001e-06 2024-09-21T15:00:49.366 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.366 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.366 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:49.366 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.891216+0000", 2024-09-21T15:00:49.366 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.056628127 2024-09-21T15:00:49.366 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.366 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.366 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:49.366 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.891273+0000", 2024-09-21T15:00:49.366 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7364000000000002e-05 2024-09-21T15:00:49.366 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.366 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.367 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:49.367 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.891350+0000", 2024-09-21T15:00:49.367 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6509999999999998e-05 2024-09-21T15:00:49.367 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:49.367 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:49.367 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:49.367 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.367 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.367 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:118225 3.d 3:b5b5db2f:::rbd_data.1136b1194d20.0000000000000005:58 [read 6062080~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:49.367 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.834591+0000", 2024-09-21T15:00:49.367 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 19.451302755, 2024-09-21T15:00:49.367 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.056883207999999998, 2024-09-21T15:00:49.367 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:49.367 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:49.368 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:49.368 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:49.368 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:49.368 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 118225 2024-09-21T15:00:49.368 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.368 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:49.368 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.368 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:49.368 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834591+0000", 2024-09-21T15:00:49.368 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:49.368 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.368 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.368 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:49.368 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834591+0000", 2024-09-21T15:00:49.368 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:49.368 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.369 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.369 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:49.369 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834590+0000", 2024-09-21T15:00:49.369 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:49.369 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.369 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.369 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:49.369 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834592+0000", 2024-09-21T15:00:49.369 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8810000000000001e-06 2024-09-21T15:00:49.369 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.369 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.369 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:49.369 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834596+0000", 2024-09-21T15:00:49.369 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.439e-06 2024-09-21T15:00:49.369 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.369 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.370 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:49.370 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834599+0000", 2024-09-21T15:00:49.370 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0740000000000001e-06 2024-09-21T15:00:49.370 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.370 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.370 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:49.370 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.891369+0000", 2024-09-21T15:00:49.370 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.056769872999999998 2024-09-21T15:00:49.370 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.370 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.370 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:49.370 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.891415+0000", 2024-09-21T15:00:49.370 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6257000000000001e-05 2024-09-21T15:00:49.370 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:49.370 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:49.371 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:49.371 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.891474+0000", 2024-09-21T15:00:49.371 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8964e-05 2024-09-21T15:00:49.371 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:49.371 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:49.371 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:49.371 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:49.371 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:49.371 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:49.372 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T15:00:49.419 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:00:49.417+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:00:49.453 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T15:00:49.485 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:00:49.481+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:00:49.494 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:49.494 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:00:49.494 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T15:00:49.494 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:49.506 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T15:00:49.586 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:00:49.585+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:00:49.610 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T15:00:49.610 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:49.610 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:00:49.610 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T15:00:49.611 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T15:00:49.611 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:49.620 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T15:00:49.687 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:00:49.685+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:00:49.735 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:49.735 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T15:00:49.735 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T15:00:49.735 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T15:00:49.735 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.735 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:63820 3.19 3.626bc079 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-21T15:00:49.735 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:27.241293+0000", 2024-09-21T15:00:49.736 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 82.489553772999997, 2024-09-21T15:00:49.736 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.98723808199999996, 2024-09-21T15:00:49.736 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:49.736 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:49.736 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:49.736 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:49.736 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:49.736 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 63820 2024-09-21T15:00:49.736 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.736 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:49.736 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.736 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:49.736 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241293+0000", 2024-09-21T15:00:49.736 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:49.737 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.737 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.737 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:49.737 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241293+0000", 2024-09-21T15:00:49.737 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:49.737 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.737 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.737 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:49.737 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241291+0000", 2024-09-21T15:00:49.737 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:00:49.737 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.737 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.737 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:49.738 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241299+0000", 2024-09-21T15:00:49.738 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1529999999999997e-06 2024-09-21T15:00:49.738 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.738 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.738 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:49.738 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241317+0000", 2024-09-21T15:00:49.738 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7764000000000001e-05 2024-09-21T15:00:49.738 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.738 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.738 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:49.738 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241328+0000", 2024-09-21T15:00:49.738 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1245e-05 2024-09-21T15:00:49.738 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.738 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.739 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:49.739 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241407+0000", 2024-09-21T15:00:49.739 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8762000000000002e-05 2024-09-21T15:00:49.739 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.739 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.739 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:00:49.739 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241418+0000", 2024-09-21T15:00:49.739 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1695000000000001e-05 2024-09-21T15:00:49.739 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.739 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.739 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:49.739 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:28.228510+0000", 2024-09-21T15:00:49.739 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.98709166100000001 2024-09-21T15:00:49.739 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.740 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.740 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:49.740 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:28.228531+0000", 2024-09-21T15:00:49.740 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0696e-05 2024-09-21T15:00:49.740 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:49.740 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:49.740 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:49.740 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.740 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.740 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:97734 3.19 3:9fcf754f:::rbd_data.1136bfc79a70.0000000000000009:head [write 0~4194304 [fadvise_sequential+fadvise_nocache],write 4784128~147456 [fadvise_sequential+fadvise_nocache],write 4997120~147456 [fadvise_sequential+fadvise_nocache],write 5210112~32768 [fadvise_sequential+fadvise_nocache],write 5292032~147456 [fadvise_sequential+fadvise_nocache],write 5505024~147456 [fadvise_sequential+fadvise_nocache],write 5718016~49152 [fadvise_sequential+fadvise_nocache],write 5799936~147456 [fadvise_sequential+fadvise_nocache],write 6012928~147456 [fadvise_sequential+fadvise_nocache],write 6225920~65536 [fadvise_sequential+fadvise_nocache],write 6307840~147456 [fadvise_sequential+fadvise_nocache],write 6520832~147456 [fadvise_sequential+fadvise_nocache],write 6733824~229376 [fadvise_sequential+fadvise_nocache],write 7028736~147456 [fadvise_sequential+fadvise_nocache],write 7241728~229376 [fadvise_sequential+fadvise_nocache],write 7536640~147456 [fadvise_sequential+fadvise_nocache],write 7749632~229376 [fadvise_sequential+fadvise_nocache],write 8077312~114688 [fadvise_sequential+fadvise_nocache],write 8290304~98304 [fadvise_sequential+fadvise_nocache],write 8404992~81920 [fadvise_sequential+fadvise_nocache],write 8617984~81920 [fadvise_sequential+fadvise_nocache],write 8830976~81920 [fadvise_sequential+fadvise_nocache],write 8962048~32768 [fadvise_sequential+fadvise_nocache],write 9175040~32768 [fadvise_sequential+fadvise_nocache],write 9388032~32768 [fadvise_sequential+fadvise_nocache],write 9469952~32768 [fadvise_sequential+fadvise_nocache],write 9682944~20092 [fadvise_sequential+fadvise_nocache],write 9961472~163840 [fadvise_sequential+fadvise_nocache],write 10141696~196608 [fadvise_sequential+fadvise_nocache],write 10354688~278528 [fadvise_sequential+fadvise_nocache],write 10649600~196608 [fadvise_sequential+fadvise_nocache],write 10862592~196608 [fadvise_sequential+fadvise_nocache],write 11075584~65536 [fadvise_sequential+fadvise_nocache],write 11173888~98304 [fadvise_sequential+fadvise_nocache],write 11288576~65536 [fadvise_sequential+fadvise_nocache],write 11386880~98304 [fadvise_sequential+fadvise_nocache],write 11501568~65536 [fadvise_sequential+fadvise_nocache],write 11583488~65536 [fadvise_sequential+fadvise_nocache],write 11681792~98304 [fadvise_sequential+fadvise_nocache],write 11796480~65536 [fadvise_sequential+fadvise_nocache],write 11894784~98304 [fadvise_sequential+fadvise_nocache],write 12009472~65536 [fadvise_sequential+fadvise_nocache],write 12189696~98304 [fadvise_sequential+fadvise_nocache],write 12402688~98304 [fadvise_sequential+fadvise_nocache],write 12697600~98304 [fadvise_sequential+fadvise_nocache],write 12910592~98304 [fadvise_sequential+fadvise_nocache],write 13205504~98304 [fadvise_sequential+fadvise_nocache],write 13418496~98304 [fadvise_sequential+fadvise_nocache],write 13713408~98304 [fadvise_sequential+fadvise_nocache],write 13926400~98304 [fadvise_sequential+fadvise_nocache],write 14139392~16384 [fadvise_sequential+fadvise_nocache],write 14680064~16384 [fadvise_sequential+fadvise_nocache],write 14827520~81920 [fadvise_sequential+fadvise_nocache],write 15040512~81920 [fadvise_sequential+fadvise_nocache],write 15335424~81920 [fadvise_sequential+fadvise_nocache],write 15548416~81920 [fadvise_sequential+fadvise_nocache],write 15843328~81920 [fadvise_sequential+fadvise_nocache],write 16056320~81920 [fadvise_sequential+fadvise_nocache],write 16351232~81920 [fadvise_sequential+fadvise_nocache],write 16564224~81920 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:49.740 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.354331+0000", 2024-09-21T15:00:49.740 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 44.376515726000001, 2024-09-21T15:00:49.740 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.086284870999999999, 2024-09-21T15:00:49.740 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:49.741 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:49.741 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:49.741 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:49.741 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:49.741 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97734 2024-09-21T15:00:49.741 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.741 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:49.741 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.741 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:49.741 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.354331+0000", 2024-09-21T15:00:49.741 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:49.741 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.741 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.741 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:49.742 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.354331+0000", 2024-09-21T15:00:49.742 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:49.742 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.742 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.742 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:49.742 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.354329+0000", 2024-09-21T15:00:49.742 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:00:49.742 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.742 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.742 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:49.742 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.370234+0000", 2024-09-21T15:00:49.742 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015905348999999999 2024-09-21T15:00:49.742 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.742 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.743 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:49.743 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.370286+0000", 2024-09-21T15:00:49.743 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1656999999999997e-05 2024-09-21T15:00:49.743 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.743 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.743 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:49.743 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.370315+0000", 2024-09-21T15:00:49.743 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9204e-05 2024-09-21T15:00:49.743 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.743 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.743 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:49.744 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.370395+0000", 2024-09-21T15:00:49.744 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0395999999999994e-05 2024-09-21T15:00:49.744 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.744 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.744 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:49.744 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.370604+0000", 2024-09-21T15:00:49.744 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000208684 2024-09-21T15:00:49.744 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.744 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.744 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 1", 2024-09-21T15:00:49.744 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.371701+0000", 2024-09-21T15:00:49.744 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0010973140000000001 2024-09-21T15:00:49.744 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.744 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.744 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:49.745 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.405485+0000", 2024-09-21T15:00:49.745 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.033783615000000003 2024-09-21T15:00:49.745 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.745 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.745 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:49.745 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.440211+0000", 2024-09-21T15:00:49.745 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034725757000000003 2024-09-21T15:00:49.745 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.745 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.745 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:49.745 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.440497+0000", 2024-09-21T15:00:49.745 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00028673099999999998 2024-09-21T15:00:49.746 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.746 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.746 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:49.746 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.440616+0000", 2024-09-21T15:00:49.746 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000118256 2024-09-21T15:00:49.746 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:49.746 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:49.746 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:49.746 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.746 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.746 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112831 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 1515520~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:49.746 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.406761+0000", 2024-09-21T15:00:49.746 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 20.324085027999999, 2024-09-21T15:00:49.746 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060265666000000002, 2024-09-21T15:00:49.746 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:49.747 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:49.747 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:49.747 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:49.747 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:49.747 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112831 2024-09-21T15:00:49.747 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.747 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:49.747 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.747 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:49.747 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406761+0000", 2024-09-21T15:00:49.747 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:49.747 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.747 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.747 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:49.748 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406761+0000", 2024-09-21T15:00:49.748 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:49.748 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.748 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.748 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:49.748 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406761+0000", 2024-09-21T15:00:49.748 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:49.748 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.748 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.748 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:49.748 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406764+0000", 2024-09-21T15:00:49.748 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0319999999999999e-06 2024-09-21T15:00:49.748 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.748 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.748 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:49.748 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406776+0000", 2024-09-21T15:00:49.749 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2357999999999999e-05 2024-09-21T15:00:49.749 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.749 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.749 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:49.749 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406784+0000", 2024-09-21T15:00:49.749 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6059999999999997e-06 2024-09-21T15:00:49.749 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.749 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.749 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:49.749 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.466595+0000", 2024-09-21T15:00:49.749 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059811062999999998 2024-09-21T15:00:49.749 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.749 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.749 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:49.749 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.466782+0000", 2024-09-21T15:00:49.750 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000186622 2024-09-21T15:00:49.750 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.750 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.750 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:49.750 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.467027+0000", 2024-09-21T15:00:49.750 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000245455 2024-09-21T15:00:49.750 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:49.750 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:49.750 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:49.750 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.750 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.750 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112832 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 1605632~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:49.750 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.406788+0000", 2024-09-21T15:00:49.751 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 20.324058438000002, 2024-09-21T15:00:49.751 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060452065999999999, 2024-09-21T15:00:49.751 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:49.751 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:49.751 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:49.751 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:49.751 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:49.751 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112832 2024-09-21T15:00:49.751 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.751 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:49.752 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.752 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:49.752 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406788+0000", 2024-09-21T15:00:49.752 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:49.752 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.752 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.752 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:49.752 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406788+0000", 2024-09-21T15:00:49.752 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:49.752 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.752 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.752 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:49.752 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406788+0000", 2024-09-21T15:00:49.752 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:49.753 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.753 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.753 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:49.753 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406798+0000", 2024-09-21T15:00:49.753 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0096e-05 2024-09-21T15:00:49.753 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.753 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.753 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:49.753 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406803+0000", 2024-09-21T15:00:49.753 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2020000000000003e-06 2024-09-21T15:00:49.753 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.753 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.753 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:49.753 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406808+0000", 2024-09-21T15:00:49.754 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8239999999999999e-06 2024-09-21T15:00:49.754 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.754 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.754 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:49.754 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.467049+0000", 2024-09-21T15:00:49.754 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060241754000000002 2024-09-21T15:00:49.754 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.754 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.754 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:49.754 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.467134+0000", 2024-09-21T15:00:49.754 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4216999999999997e-05 2024-09-21T15:00:49.754 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.754 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.754 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:49.754 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.467240+0000", 2024-09-21T15:00:49.755 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000106304 2024-09-21T15:00:49.755 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:49.755 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:49.755 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:49.755 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.755 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.755 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112843 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 2613248~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:49.755 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.410273+0000", 2024-09-21T15:00:49.755 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 20.320573843999998, 2024-09-21T15:00:49.755 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059854931, 2024-09-21T15:00:49.755 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:49.755 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:49.755 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:49.755 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:49.756 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:49.756 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112843 2024-09-21T15:00:49.756 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.756 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:49.756 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.756 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:49.756 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410273+0000", 2024-09-21T15:00:49.756 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:49.756 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.756 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.756 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:49.756 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410273+0000", 2024-09-21T15:00:49.756 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:49.756 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.757 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.757 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:49.757 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410272+0000", 2024-09-21T15:00:49.757 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:49.757 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.757 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.757 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:49.757 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410274+0000", 2024-09-21T15:00:49.757 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.905e-06 2024-09-21T15:00:49.757 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.757 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.757 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:49.757 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410283+0000", 2024-09-21T15:00:49.757 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3440000000000007e-06 2024-09-21T15:00:49.758 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.758 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.758 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:49.758 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410294+0000", 2024-09-21T15:00:49.758 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0271e-05 2024-09-21T15:00:49.758 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.758 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.758 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:49.758 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.469917+0000", 2024-09-21T15:00:49.758 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059623738000000003 2024-09-21T15:00:49.758 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.758 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.758 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:49.758 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470002+0000", 2024-09-21T15:00:49.759 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4270999999999993e-05 2024-09-21T15:00:49.759 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.759 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.759 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:49.759 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470127+0000", 2024-09-21T15:00:49.759 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000125711 2024-09-21T15:00:49.759 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:49.759 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:49.759 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:49.759 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.759 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.759 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112844 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 2703360~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:49.759 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.410326+0000", 2024-09-21T15:00:49.759 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 20.320520124000002, 2024-09-21T15:00:49.759 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060086056999999998, 2024-09-21T15:00:49.760 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:49.760 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:49.760 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:49.760 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:49.760 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:49.760 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112844 2024-09-21T15:00:49.760 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.760 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:49.760 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.760 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:49.760 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410326+0000", 2024-09-21T15:00:49.760 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:49.760 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.760 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.761 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:49.761 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410326+0000", 2024-09-21T15:00:49.761 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:49.761 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.761 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.761 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:49.761 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410326+0000", 2024-09-21T15:00:49.761 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:00:49.761 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.761 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.761 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:49.761 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410352+0000", 2024-09-21T15:00:49.761 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6872999999999998e-05 2024-09-21T15:00:49.761 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.762 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.762 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:49.762 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410374+0000", 2024-09-21T15:00:49.762 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1141e-05 2024-09-21T15:00:49.762 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.762 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.762 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:49.762 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410385+0000", 2024-09-21T15:00:49.762 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1456e-05 2024-09-21T15:00:49.762 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.762 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.762 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:49.762 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470174+0000", 2024-09-21T15:00:49.762 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059788574999999997 2024-09-21T15:00:49.763 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.763 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.763 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:49.763 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470277+0000", 2024-09-21T15:00:49.763 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000103838 2024-09-21T15:00:49.763 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.763 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.763 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:49.763 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470412+0000", 2024-09-21T15:00:49.763 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013489700000000001 2024-09-21T15:00:49.763 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:49.763 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:49.763 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:49.763 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.764 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.764 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112845 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 2793472~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:49.764 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.410394+0000", 2024-09-21T15:00:49.764 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 20.32045265, 2024-09-21T15:00:49.764 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060244173999999998, 2024-09-21T15:00:49.764 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:49.764 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:49.764 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:49.764 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:49.764 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:49.764 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112845 2024-09-21T15:00:49.764 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.764 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:49.764 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.764 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:49.765 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410394+0000", 2024-09-21T15:00:49.765 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:49.765 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.765 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.765 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:49.765 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410394+0000", 2024-09-21T15:00:49.765 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:49.765 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.765 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.765 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:49.765 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410393+0000", 2024-09-21T15:00:49.765 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:49.765 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.765 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.766 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:49.766 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410397+0000", 2024-09-21T15:00:49.766 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9430000000000002e-06 2024-09-21T15:00:49.766 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.766 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.766 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:49.766 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410406+0000", 2024-09-21T15:00:49.766 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.799e-06 2024-09-21T15:00:49.766 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.766 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.766 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:49.766 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410418+0000", 2024-09-21T15:00:49.766 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2225999999999999e-05 2024-09-21T15:00:49.766 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.766 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.767 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:49.767 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470444+0000", 2024-09-21T15:00:49.767 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060026311999999998 2024-09-21T15:00:49.767 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.767 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.767 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:49.767 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470535+0000", 2024-09-21T15:00:49.767 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0173999999999994e-05 2024-09-21T15:00:49.767 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.767 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.767 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:49.767 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470638+0000", 2024-09-21T15:00:49.767 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010333 2024-09-21T15:00:49.767 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:49.767 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:49.768 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:49.768 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.768 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.768 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112846 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 2883584~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:49.768 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.410425+0000", 2024-09-21T15:00:49.768 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 20.320421437, 2024-09-21T15:00:49.768 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060459247000000001, 2024-09-21T15:00:49.768 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:49.768 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:49.768 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:49.768 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:49.768 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:49.768 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112846 2024-09-21T15:00:49.768 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.768 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:49.768 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.769 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:49.769 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410425+0000", 2024-09-21T15:00:49.769 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:49.769 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.769 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.769 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:49.769 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410425+0000", 2024-09-21T15:00:49.769 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:49.769 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.769 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.769 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:49.769 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410424+0000", 2024-09-21T15:00:49.769 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:49.769 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.769 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.770 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:49.770 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410428+0000", 2024-09-21T15:00:49.770 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4709999999999999e-06 2024-09-21T15:00:49.770 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.770 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.770 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:49.770 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410446+0000", 2024-09-21T15:00:49.770 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8354000000000001e-05 2024-09-21T15:00:49.770 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.770 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.770 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:49.770 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410456+0000", 2024-09-21T15:00:49.770 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6709999999999994e-06 2024-09-21T15:00:49.770 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.770 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.771 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:49.771 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470661+0000", 2024-09-21T15:00:49.771 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060205303000000002 2024-09-21T15:00:49.771 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.771 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.771 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:49.771 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470756+0000", 2024-09-21T15:00:49.771 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4401999999999995e-05 2024-09-21T15:00:49.771 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.771 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.771 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:49.771 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470884+0000", 2024-09-21T15:00:49.771 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012860500000000001 2024-09-21T15:00:49.771 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:49.772 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:49.772 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:49.772 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.772 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.772 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112847 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 2973696~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:49.772 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.410462+0000", 2024-09-21T15:00:49.772 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 20.320384393000001, 2024-09-21T15:00:49.772 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060653836000000003, 2024-09-21T15:00:49.772 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:49.772 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:49.772 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:49.772 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:49.772 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:49.772 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112847 2024-09-21T15:00:49.772 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.773 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:49.773 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.773 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:49.773 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410462+0000", 2024-09-21T15:00:49.773 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:49.773 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.773 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.773 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:49.773 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410462+0000", 2024-09-21T15:00:49.773 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:49.773 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.773 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.773 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:49.773 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410461+0000", 2024-09-21T15:00:49.773 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:49.774 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.774 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.774 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:49.774 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410465+0000", 2024-09-21T15:00:49.774 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6899999999999998e-06 2024-09-21T15:00:49.774 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.774 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.774 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:49.774 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410474+0000", 2024-09-21T15:00:49.774 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4179999999999997e-06 2024-09-21T15:00:49.774 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.774 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.774 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:49.774 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410508+0000", 2024-09-21T15:00:49.774 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4282999999999997e-05 2024-09-21T15:00:49.774 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.775 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.775 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:49.775 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470922+0000", 2024-09-21T15:00:49.775 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060413820999999999 2024-09-21T15:00:49.775 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.775 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.775 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:49.775 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471008+0000", 2024-09-21T15:00:49.775 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6379999999999996e-05 2024-09-21T15:00:49.775 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.775 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.775 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:49.775 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471116+0000", 2024-09-21T15:00:49.775 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000107806 2024-09-21T15:00:49.775 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:49.776 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:49.776 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:49.776 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.776 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.776 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112848 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 3063808~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:49.776 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.410525+0000", 2024-09-21T15:00:49.776 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 20.320321109999998, 2024-09-21T15:00:49.776 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060880735999999998, 2024-09-21T15:00:49.776 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:49.776 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:49.776 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:49.776 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:49.776 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:49.776 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112848 2024-09-21T15:00:49.777 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.777 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:49.777 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.777 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:49.777 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410525+0000", 2024-09-21T15:00:49.777 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:49.777 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.777 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.777 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:49.777 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410525+0000", 2024-09-21T15:00:49.777 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:49.777 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.777 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.777 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:49.778 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410525+0000", 2024-09-21T15:00:49.778 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:49.778 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.778 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.778 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:49.778 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410529+0000", 2024-09-21T15:00:49.778 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1760000000000003e-06 2024-09-21T15:00:49.778 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.778 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.778 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:49.778 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410551+0000", 2024-09-21T15:00:49.778 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2099999999999998e-05 2024-09-21T15:00:49.778 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.779 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.779 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:49.779 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410571+0000", 2024-09-21T15:00:49.779 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0159000000000001e-05 2024-09-21T15:00:49.779 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.779 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.779 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:49.779 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471164+0000", 2024-09-21T15:00:49.779 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.06059287 2024-09-21T15:00:49.779 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.779 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.779 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:49.779 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471271+0000", 2024-09-21T15:00:49.779 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000106635 2024-09-21T15:00:49.779 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.780 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.780 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:49.780 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471406+0000", 2024-09-21T15:00:49.780 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000135399 2024-09-21T15:00:49.780 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:49.780 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:49.780 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:49.780 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.780 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.780 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112849 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 3153920~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:49.780 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.410579+0000", 2024-09-21T15:00:49.780 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 20.320267815000001, 2024-09-21T15:00:49.780 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.061057304999999999, 2024-09-21T15:00:49.780 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:49.780 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:49.781 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:49.781 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:49.781 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:49.781 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112849 2024-09-21T15:00:49.781 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.781 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:49.781 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.781 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:49.781 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410579+0000", 2024-09-21T15:00:49.781 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:49.781 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.781 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.782 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:49.782 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410579+0000", 2024-09-21T15:00:49.782 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:49.782 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.782 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.782 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:49.782 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410578+0000", 2024-09-21T15:00:49.782 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:49.782 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.782 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.782 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:49.783 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410582+0000", 2024-09-21T15:00:49.783 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.036e-06 2024-09-21T15:00:49.783 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.783 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.783 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:49.783 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410592+0000", 2024-09-21T15:00:49.783 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9850000000000001e-06 2024-09-21T15:00:49.783 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.783 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.783 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:49.783 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410604+0000", 2024-09-21T15:00:49.783 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2051e-05 2024-09-21T15:00:49.783 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.783 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.783 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:49.784 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471444+0000", 2024-09-21T15:00:49.784 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060839722999999998 2024-09-21T15:00:49.784 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.784 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.784 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:49.784 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471530+0000", 2024-09-21T15:00:49.784 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6359000000000004e-05 2024-09-21T15:00:49.784 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.784 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.784 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:49.784 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471636+0000", 2024-09-21T15:00:49.784 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010576700000000001 2024-09-21T15:00:49.784 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:49.784 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:49.785 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:49.785 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.785 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.785 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112850 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 3244032~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:49.785 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.410698+0000", 2024-09-21T15:00:49.785 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 20.320148473, 2024-09-21T15:00:49.785 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.061164118000000003, 2024-09-21T15:00:49.785 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:49.785 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:49.785 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:49.785 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:49.785 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:49.785 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112850 2024-09-21T15:00:49.785 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.786 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:49.786 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.786 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:49.786 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410698+0000", 2024-09-21T15:00:49.786 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:49.786 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.786 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.786 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:49.786 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410698+0000", 2024-09-21T15:00:49.786 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:49.786 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.786 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.786 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:49.786 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410610+0000", 2024-09-21T15:00:49.787 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999123 2024-09-21T15:00:49.787 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.787 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.787 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:49.787 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410707+0000", 2024-09-21T15:00:49.787 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7093e-05 2024-09-21T15:00:49.787 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.787 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.787 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:49.787 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410728+0000", 2024-09-21T15:00:49.787 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0275000000000002e-05 2024-09-21T15:00:49.787 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.787 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.788 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:49.788 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410740+0000", 2024-09-21T15:00:49.788 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2561999999999999e-05 2024-09-21T15:00:49.788 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.788 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.788 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:49.788 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471664+0000", 2024-09-21T15:00:49.788 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060923979000000003 2024-09-21T15:00:49.788 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.788 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.788 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:49.788 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471753+0000", 2024-09-21T15:00:49.788 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8265999999999998e-05 2024-09-21T15:00:49.788 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.789 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.789 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:49.789 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471862+0000", 2024-09-21T15:00:49.789 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000109474 2024-09-21T15:00:49.789 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:49.789 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:49.789 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:49.789 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.789 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.789 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112864 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 4521984~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:49.789 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.414789+0000", 2024-09-21T15:00:49.789 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 20.316057447999999, 2024-09-21T15:00:49.789 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059845903999999998, 2024-09-21T15:00:49.789 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:49.789 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:49.790 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:49.790 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:49.790 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:49.790 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112864 2024-09-21T15:00:49.790 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.790 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:49.790 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.790 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:49.790 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414789+0000", 2024-09-21T15:00:49.790 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:49.790 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.790 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.790 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:49.790 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414789+0000", 2024-09-21T15:00:49.791 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:49.791 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.791 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.791 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:49.791 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414788+0000", 2024-09-21T15:00:49.791 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:49.791 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.791 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.791 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:49.791 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414790+0000", 2024-09-21T15:00:49.791 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.995e-06 2024-09-21T15:00:49.791 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.791 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.791 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:49.791 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414801+0000", 2024-09-21T15:00:49.792 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.01e-05 2024-09-21T15:00:49.792 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.792 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.792 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:49.792 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414804+0000", 2024-09-21T15:00:49.792 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.563e-06 2024-09-21T15:00:49.792 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.792 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.792 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:49.792 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.474499+0000", 2024-09-21T15:00:49.792 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059695208 2024-09-21T15:00:49.792 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.792 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.792 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:49.793 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.474561+0000", 2024-09-21T15:00:49.793 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1672000000000002e-05 2024-09-21T15:00:49.793 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.793 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.793 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:49.793 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.474635+0000", 2024-09-21T15:00:49.793 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3792000000000001e-05 2024-09-21T15:00:49.793 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:49.793 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:49.793 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:49.793 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.793 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.793 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112865 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 4612096~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:49.793 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.414807+0000", 2024-09-21T15:00:49.793 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 20.316039052000001, 2024-09-21T15:00:49.794 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060003506999999998, 2024-09-21T15:00:49.794 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:49.794 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:49.794 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:49.794 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:49.794 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:49.794 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112865 2024-09-21T15:00:49.794 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.794 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:49.794 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.794 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:49.794 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414807+0000", 2024-09-21T15:00:49.794 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:49.794 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.795 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.795 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:49.795 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414807+0000", 2024-09-21T15:00:49.795 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:49.795 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.795 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.795 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:49.795 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414807+0000", 2024-09-21T15:00:49.795 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:49.795 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.795 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.795 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:49.795 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414809+0000", 2024-09-21T15:00:49.795 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0509999999999999e-06 2024-09-21T15:00:49.796 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.796 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.796 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:49.796 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414813+0000", 2024-09-21T15:00:49.796 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0960000000000003e-06 2024-09-21T15:00:49.796 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.796 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.796 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:49.796 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414817+0000", 2024-09-21T15:00:49.796 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.296e-06 2024-09-21T15:00:49.796 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.796 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.796 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:49.796 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.474671+0000", 2024-09-21T15:00:49.797 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059853663000000001 2024-09-21T15:00:49.797 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.797 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.797 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:49.797 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.474735+0000", 2024-09-21T15:00:49.797 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3653999999999997e-05 2024-09-21T15:00:49.797 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.797 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.797 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:49.797 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.474811+0000", 2024-09-21T15:00:49.797 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6074000000000004e-05 2024-09-21T15:00:49.797 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:49.797 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:49.797 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:49.797 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.798 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.798 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112866 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 4702208~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:49.798 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.414820+0000", 2024-09-21T15:00:49.798 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 20.316026132000001, 2024-09-21T15:00:49.798 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060161992999999997, 2024-09-21T15:00:49.798 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:49.798 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:49.798 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:49.798 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:49.798 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:49.798 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112866 2024-09-21T15:00:49.798 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.798 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:49.799 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.799 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:49.799 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414820+0000", 2024-09-21T15:00:49.799 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:49.799 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.799 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.799 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:49.799 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414820+0000", 2024-09-21T15:00:49.799 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:49.799 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.799 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.799 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:49.799 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414820+0000", 2024-09-21T15:00:49.799 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:49.799 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.800 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.800 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:49.800 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414822+0000", 2024-09-21T15:00:49.800 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8509999999999999e-06 2024-09-21T15:00:49.800 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.800 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.800 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:49.800 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414826+0000", 2024-09-21T15:00:49.800 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8190000000000002e-06 2024-09-21T15:00:49.800 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.800 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.800 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:49.800 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414837+0000", 2024-09-21T15:00:49.800 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1232e-05 2024-09-21T15:00:49.800 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.800 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.801 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:49.801 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.474846+0000", 2024-09-21T15:00:49.801 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060009506999999997 2024-09-21T15:00:49.801 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.801 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.801 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:49.801 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.474910+0000", 2024-09-21T15:00:49.801 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3918999999999996e-05 2024-09-21T15:00:49.801 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.801 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.801 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:49.801 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.474982+0000", 2024-09-21T15:00:49.801 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1958999999999993e-05 2024-09-21T15:00:49.801 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:49.801 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:49.802 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:49.802 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.802 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.802 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112867 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 4792320~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:49.802 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.414840+0000", 2024-09-21T15:00:49.802 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 20.316006637000001, 2024-09-21T15:00:49.802 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060308542999999999, 2024-09-21T15:00:49.802 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:49.802 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:49.802 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:49.802 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:49.802 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:49.802 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112867 2024-09-21T15:00:49.802 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.803 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:49.803 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.803 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:49.803 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414840+0000", 2024-09-21T15:00:49.803 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:49.803 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.803 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.803 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:49.803 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414840+0000", 2024-09-21T15:00:49.803 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:49.803 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.803 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.803 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:49.803 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414839+0000", 2024-09-21T15:00:49.804 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:49.804 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.804 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.804 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:49.804 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414841+0000", 2024-09-21T15:00:49.804 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8500000000000001e-06 2024-09-21T15:00:49.804 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.804 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.804 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:49.804 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414846+0000", 2024-09-21T15:00:49.804 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7700000000000001e-06 2024-09-21T15:00:49.804 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.804 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.804 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:49.805 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414850+0000", 2024-09-21T15:00:49.805 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1389999999999997e-06 2024-09-21T15:00:49.805 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.805 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.805 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:49.805 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.475016+0000", 2024-09-21T15:00:49.805 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.06016614 2024-09-21T15:00:49.805 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.805 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.805 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:49.805 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.475073+0000", 2024-09-21T15:00:49.805 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6295000000000002e-05 2024-09-21T15:00:49.805 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.805 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.805 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:49.806 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.475148+0000", 2024-09-21T15:00:49.806 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5647000000000003e-05 2024-09-21T15:00:49.806 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:49.806 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:49.806 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:49.806 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.806 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.806 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:113030 3.6 3:60420fc2:::rbd_data.1136b442e94a.0000000000000001:head [write 0~360448 [fadvise_sequential+fadvise_nocache],write 385024~573440 [fadvise_sequential+fadvise_nocache],write 1146880~573440 [fadvise_sequential+fadvise_nocache],write 1908736~188416 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:49.806 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.422420+0000", 2024-09-21T15:00:49.806 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 20.308425992, 2024-09-21T15:00:49.806 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.063268959999999999, 2024-09-21T15:00:49.806 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:49.806 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:49.806 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:49.807 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:49.807 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:49.807 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 113030 2024-09-21T15:00:49.807 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.807 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:49.807 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.807 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:49.807 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.422420+0000", 2024-09-21T15:00:49.807 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:49.807 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.807 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.807 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:49.807 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.422420+0000", 2024-09-21T15:00:49.807 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:49.807 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.808 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.808 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:49.808 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.422420+0000", 2024-09-21T15:00:49.808 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:49.808 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.808 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.808 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:49.808 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.424657+0000", 2024-09-21T15:00:49.808 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0022369780000000001 2024-09-21T15:00:49.808 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.808 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.808 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:49.808 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.424673+0000", 2024-09-21T15:00:49.808 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6141e-05 2024-09-21T15:00:49.809 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.809 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.809 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:49.809 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.424689+0000", 2024-09-21T15:00:49.809 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5897999999999999e-05 2024-09-21T15:00:49.809 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.809 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.809 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:49.809 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.478005+0000", 2024-09-21T15:00:49.809 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.05331636 2024-09-21T15:00:49.809 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.809 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.809 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:49.809 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.478133+0000", 2024-09-21T15:00:49.809 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012736799999999999 2024-09-21T15:00:49.810 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.810 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.810 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 7", 2024-09-21T15:00:49.810 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.478307+0000", 2024-09-21T15:00:49.810 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017491499999999999 2024-09-21T15:00:49.810 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.810 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.810 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:49.810 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.483611+0000", 2024-09-21T15:00:49.810 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0053039389999999997 2024-09-21T15:00:49.810 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.810 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.810 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:49.811 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.485553+0000", 2024-09-21T15:00:49.811 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0019418759999999999 2024-09-21T15:00:49.811 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.811 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.811 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:49.811 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.485645+0000", 2024-09-21T15:00:49.811 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2138000000000005e-05 2024-09-21T15:00:49.811 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.811 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.811 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:49.811 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.485689+0000", 2024-09-21T15:00:49.811 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3922999999999999e-05 2024-09-21T15:00:49.811 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:49.811 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:49.812 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:49.812 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.812 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.812 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:114290 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 6946816~8192 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:49.812 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.558087+0000", 2024-09-21T15:00:49.812 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 20.172759195000001, 2024-09-21T15:00:49.812 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059973091999999999, 2024-09-21T15:00:49.812 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:49.812 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:49.812 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:49.812 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:49.812 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:49.812 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 114290 2024-09-21T15:00:49.812 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.812 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:49.813 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.813 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:49.813 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.558087+0000", 2024-09-21T15:00:49.813 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:49.813 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.813 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.813 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:49.813 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.558087+0000", 2024-09-21T15:00:49.813 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:49.813 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.813 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.813 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:49.813 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.558087+0000", 2024-09-21T15:00:49.813 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:49.814 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.814 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.814 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:49.814 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.558092+0000", 2024-09-21T15:00:49.814 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2379999999999997e-06 2024-09-21T15:00:49.814 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.814 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.814 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:49.814 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.558095+0000", 2024-09-21T15:00:49.814 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.275e-06 2024-09-21T15:00:49.814 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.814 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.814 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:49.814 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.558099+0000", 2024-09-21T15:00:49.815 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6679999999999999e-06 2024-09-21T15:00:49.815 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.815 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.815 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:49.815 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.617757+0000", 2024-09-21T15:00:49.815 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059657531999999999 2024-09-21T15:00:49.815 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.815 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.815 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:49.815 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.617891+0000", 2024-09-21T15:00:49.815 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000134478 2024-09-21T15:00:49.815 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.815 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.815 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:49.816 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.618060+0000", 2024-09-21T15:00:49.816 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016925100000000001 2024-09-21T15:00:49.816 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:49.816 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:49.816 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:49.816 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.816 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.816 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:114312 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 8945664~8192 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:49.816 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.562396+0000", 2024-09-21T15:00:49.816 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 20.168450846999999, 2024-09-21T15:00:49.816 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059832396000000003, 2024-09-21T15:00:49.816 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:49.816 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:49.816 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:49.816 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:49.817 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:49.817 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 114312 2024-09-21T15:00:49.817 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.817 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:49.817 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.817 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:49.817 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562396+0000", 2024-09-21T15:00:49.817 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:49.817 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.817 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.817 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:49.817 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562396+0000", 2024-09-21T15:00:49.817 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:49.817 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.818 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.818 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:49.818 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562395+0000", 2024-09-21T15:00:49.818 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:49.818 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.818 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.818 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:49.818 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562397+0000", 2024-09-21T15:00:49.818 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9e-06 2024-09-21T15:00:49.818 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.818 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.818 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:49.818 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562401+0000", 2024-09-21T15:00:49.818 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8099999999999999e-06 2024-09-21T15:00:49.818 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.819 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.819 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:49.819 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562405+0000", 2024-09-21T15:00:49.819 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.5710000000000002e-06 2024-09-21T15:00:49.819 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.819 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.819 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:49.819 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.621997+0000", 2024-09-21T15:00:49.819 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059592118999999999 2024-09-21T15:00:49.819 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.819 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.819 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:49.819 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.622121+0000", 2024-09-21T15:00:49.819 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012411599999999999 2024-09-21T15:00:49.820 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.820 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.820 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:49.820 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.622228+0000", 2024-09-21T15:00:49.820 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000107184 2024-09-21T15:00:49.820 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:49.820 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:49.820 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:49.820 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.820 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.820 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:114313 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 9035776~8192 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:49.820 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.562408+0000", 2024-09-21T15:00:49.820 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 20.168438865999999, 2024-09-21T15:00:49.820 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059975560999999997, 2024-09-21T15:00:49.820 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:49.821 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:49.821 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:49.821 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:49.821 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:49.821 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 114313 2024-09-21T15:00:49.821 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.821 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:49.821 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.821 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:49.821 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562408+0000", 2024-09-21T15:00:49.821 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:49.821 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.821 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.821 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:49.821 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562408+0000", 2024-09-21T15:00:49.822 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:49.822 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.822 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.822 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:49.822 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562407+0000", 2024-09-21T15:00:49.822 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:49.822 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.822 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.822 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:49.822 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562409+0000", 2024-09-21T15:00:49.822 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9700000000000002e-06 2024-09-21T15:00:49.822 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.822 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.822 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:49.823 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562421+0000", 2024-09-21T15:00:49.823 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2094e-05 2024-09-21T15:00:49.823 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.823 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.823 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:49.823 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562427+0000", 2024-09-21T15:00:49.823 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7690000000000002e-06 2024-09-21T15:00:49.823 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.823 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.823 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:49.823 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.622264+0000", 2024-09-21T15:00:49.823 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059837488000000001 2024-09-21T15:00:49.823 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.823 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.824 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:49.824 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.622319+0000", 2024-09-21T15:00:49.824 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4064999999999998e-05 2024-09-21T15:00:49.824 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:49.824 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:49.824 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:49.824 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.622383+0000", 2024-09-21T15:00:49.824 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4506999999999998e-05 2024-09-21T15:00:49.824 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:49.824 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:49.824 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:49.824 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:49.824 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:49.824 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:49.826 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T15:00:49.826 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T15:00:49.826 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T15:00:49.826 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T15:00:49.827 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:00:49.823+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:00:49.827 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T15:00:49.829 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-21T15:00:49.856 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T15:00:49.880 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T15:00:49.902 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T15:00:49.904 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:00:49.904 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:00:49.904 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:00:49.904 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:00:49.905 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:00:49.905 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:00:49.905 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:00:49.905 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:00:49.927 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:00:49.925+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:00:49.945 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:49.945 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:49.955 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T15:00:50.028 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:00:50.023+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:00:50.064 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:50.064 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:50.074 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T15:00:50.129 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:00:50.127+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:00:50.164 INFO:teuthology.orchestra.run.smithi032.stderr:nodeep-scrub is unset 2024-09-21T15:00:50.183 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:50.184 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:50.194 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T15:00:50.230 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:00:50.225+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:00:50.247 INFO:teuthology.orchestra.run.smithi032.stdout:checking clone #31, image image_client.0-clone31 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent32 2024-09-21T15:00:50.304 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:50.304 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:50.315 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T15:00:50.359 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:00:50.357+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:00:50.419 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:50.420 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:50.429 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T15:00:50.432 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:00:50.429+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:00:50.533 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:00:50.527+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:00:50.537 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:50.537 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:50.547 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T15:00:50.615 INFO:teuthology.orchestra.run.smithi032.stdout:4925 writesame 0xb8b0c4a thru 0xb8bcb64 data_size 0x1d(0xbf1b bytes) 2024-09-21T15:00:50.615 INFO:teuthology.orchestra.run.smithi032.stdout:4926 read 0xa0380e4 thru 0xa042019 (0x9f36 bytes) 2024-09-21T15:00:50.615 INFO:teuthology.orchestra.run.smithi032.stdout:4927 punch from 0x5739df9 to 0x5747e9e, (0xe0a5 bytes) 2024-09-21T15:00:50.615 INFO:teuthology.orchestra.run.smithi032.stdout:4928 read 0x5b39300 thru 0x5b42684 (0x9385 bytes) 2024-09-21T15:00:50.615 INFO:teuthology.orchestra.run.smithi032.stdout:4929 punch from 0x3b0c8ec to 0x3b15d1c, (0x9430 bytes) 2024-09-21T15:00:50.621 INFO:teuthology.orchestra.run.smithi032.stdout:4930 trunc from 0xb8bcb65 to 0x7792d8e 2024-09-21T15:00:50.632 INFO:teuthology.orchestra.run.smithi032.stdout:4932 writesame 0x462ec8b thru 0x4637c62 data_size 0x4(0x8fd8 bytes) 2024-09-21T15:00:50.633 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:00:50.631+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:00:50.644 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:50.652 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:50.652 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:50.662 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T15:00:50.665 INFO:teuthology.orchestra.run.smithi032.stdout:4934 write 0xd2ed1b7 thru 0xd2f2ba7 (0x59f1 bytes) 2024-09-21T15:00:50.675 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:50.675 INFO:teuthology.orchestra.run.smithi032.stdout:4937 read 0x9132d74 thru 0x913e66d (0xb8fa bytes) 2024-09-21T15:00:50.675 INFO:teuthology.orchestra.run.smithi032.stdout:4939 read 0x93abb90 thru 0x93b5302 (0x9773 bytes) 2024-09-21T15:00:50.675 INFO:teuthology.orchestra.run.smithi032.stdout:4940 writesame 0x6f1fc7e thru 0x6f2ba1e data_size 0x5f(0xbda1 bytes) 2024-09-21T15:00:50.686 INFO:teuthology.orchestra.run.smithi032.stdout:4942 read 0x987d070 thru 0x98878d5 (0xa866 bytes) 2024-09-21T15:00:50.686 INFO:teuthology.orchestra.run.smithi032.stdout:4944 read 0xd17ea26 thru 0xd18510f (0x66ea bytes) 2024-09-21T15:00:50.711 INFO:teuthology.orchestra.run.smithi032.stdout:4945 trunc from 0xd2f2ba8 to 0x376b04 2024-09-21T15:00:50.734 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:00:50.729+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:00:50.761 INFO:teuthology.orchestra.run.smithi032.stdout:4946 trunc from 0x376b04 to 0xc83b5b7 2024-09-21T15:00:50.768 INFO:teuthology.orchestra.run.smithi032.stdout:4947 punch from 0x53fd8bd to 0x54002bc, (0x29ff bytes) 2024-09-21T15:00:50.769 INFO:teuthology.orchestra.run.smithi032.stdout:4948 writesame 0x28f0bd5 thru 0x28f9773 data_size 0x1(0x8b9f bytes) 2024-09-21T15:00:50.774 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:50.774 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:50.785 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T15:00:50.796 INFO:teuthology.orchestra.run.smithi032.stdout:4949 write 0x1ff7994 thru 0x1ffcff7 (0x5664 bytes) 2024-09-21T15:00:50.802 INFO:teuthology.orchestra.run.smithi032.stdout:4952 punch from 0x1e30ff5 to 0x1e3123b, (0x246 bytes) 2024-09-21T15:00:50.802 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:50.802 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:50.803 INFO:teuthology.orchestra.run.smithi032.stdout:4955 writesame 0xa67cafe thru 0xa68122f data_size 0xd(0x4732 bytes) 2024-09-21T15:00:50.811 INFO:teuthology.orchestra.run.smithi032.stdout:4956 write 0x6c9845 thru 0x6d5cf9 (0xc4b5 bytes) 2024-09-21T15:00:50.818 INFO:teuthology.orchestra.run.smithi032.stdout:4957 writesame 0x1bc8403 thru 0x1bcb829 data_size 0xd(0x3427 bytes) 2024-09-21T15:00:50.825 INFO:teuthology.orchestra.run.smithi032.stdout:4958 read 0x404ae4c thru 0x4057629 (0xc7de bytes) 2024-09-21T15:00:50.825 INFO:teuthology.orchestra.run.smithi032.stdout:4959 writesame 0x48e8f6c thru 0x48e9794 data_size 0x1(0x829 bytes) 2024-09-21T15:00:50.832 INFO:teuthology.orchestra.run.smithi032.stdout:4960 read 0x80f7fd1 thru 0x8101d0a (0x9d3a bytes) 2024-09-21T15:00:50.834 INFO:teuthology.orchestra.run.smithi032.stdout:4961 trunc from 0xc83b5b7 to 0xbbdfadd 2024-09-21T15:00:50.834 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:00:50.831+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:00:50.845 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:50.845 INFO:teuthology.orchestra.run.smithi032.stdout:4963 punch from 0x707c9e3 to 0x70807bf, (0x3ddc bytes) 2024-09-21T15:00:50.845 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:50.845 INFO:teuthology.orchestra.run.smithi032.stdout:4965 write 0xa484867 thru 0xa48e62e (0x9dc8 bytes) 2024-09-21T15:00:50.871 INFO:teuthology.orchestra.run.smithi032.stdout:4966 write 0xd641ca7 thru 0xd645aa8 (0x3e02 bytes) 2024-09-21T15:00:50.887 INFO:teuthology.orchestra.run.smithi032.stdout:4967 trunc from 0xd645aa9 to 0x8ee372e 2024-09-21T15:00:50.893 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:50.893 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:50.903 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T15:00:50.919 INFO:teuthology.orchestra.run.smithi032.stdout:4969 write 0xaed7ee4 thru 0xaede2ef (0x640c bytes) 2024-09-21T15:00:50.928 INFO:teuthology.orchestra.run.smithi032.stdout:4971 read 0xe4dd54 thru 0xe5b680 (0xd92d bytes) 2024-09-21T15:00:50.928 INFO:teuthology.orchestra.run.smithi032.stdout:4972 write 0x4ab0494 thru 0x4abdda4 (0xd911 bytes) 2024-09-21T15:00:50.935 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:00:50.931+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:00:50.944 INFO:teuthology.orchestra.run.smithi032.stdout:4973 trunc from 0xaede2f0 to 0x7ccdfca 2024-09-21T15:00:50.972 INFO:teuthology.orchestra.run.smithi032.stdout:4976 trunc from 0x7ccdfca to 0xd1ad800 2024-09-21T15:00:50.982 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:50.990 INFO:teuthology.orchestra.run.smithi032.stdout:4978 trunc from 0xd1ad800 to 0x931724f 2024-09-21T15:00:51.000 INFO:teuthology.orchestra.run.smithi032.stdout:4979 punch from 0x4cafd15 to 0x4cbcec7, (0xd1b2 bytes) 2024-09-21T15:00:51.000 INFO:teuthology.orchestra.run.smithi032.stdout:4980 read 0x8c112a thru 0x8c226e (0x1145 bytes) 2024-09-21T15:00:51.000 INFO:teuthology.orchestra.run.smithi032.stdout:4983 writesame 0x2e50998 thru 0x2e54114 data_size 0x5(0x377d bytes) 2024-09-21T15:00:51.008 INFO:teuthology.orchestra.run.smithi032.stdout:4984 read 0x55c8737 thru 0x55cb00e (0x28d8 bytes) 2024-09-21T15:00:51.008 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:51.008 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:51.008 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:51.008 INFO:teuthology.orchestra.run.smithi032.stdout:4991 punch from 0xff079e to 0xffb883, (0xb0e5 bytes) 2024-09-21T15:00:51.008 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:51.008 INFO:teuthology.orchestra.run.smithi032.stdout:4995 write 0x370a9b thru 0x37acd8 (0xa23e bytes) 2024-09-21T15:00:51.011 INFO:teuthology.orchestra.run.smithi032.stdout:4996 read 0x6abea6a thru 0x6ac3840 (0x4dd7 bytes) 2024-09-21T15:00:51.011 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:51.015 INFO:teuthology.orchestra.run.smithi032.stdout:4998 trunc from 0x931724f to 0x73d61ab 2024-09-21T15:00:51.025 INFO:teuthology.orchestra.run.smithi032.stdout:4999 writesame 0x4537927 thru 0x454125d data_size 0x1(0x9937 bytes) 2024-09-21T15:00:51.026 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:51.026 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:51.035 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:00:51.033+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:00:51.037 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T15:00:51.063 INFO:teuthology.orchestra.run.smithi032.stdout:5000 read 0x5e2477d thru 0x5e2d3da (0x8c5e bytes) 2024-09-21T15:00:51.063 INFO:teuthology.orchestra.run.smithi032.stdout:5002 trunc from 0x73d61ab to 0x77f7a3c 2024-09-21T15:00:51.084 INFO:teuthology.orchestra.run.smithi032.stdout:5004 trunc from 0x77f7a3c to 0xa5946a5 2024-09-21T15:00:51.093 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:51.094 INFO:teuthology.orchestra.run.smithi032.stdout:5006 read 0x1e1cd8 thru 0x1e28d4 (0xbfd bytes) 2024-09-21T15:00:51.094 INFO:teuthology.orchestra.run.smithi032.stdout:5007 writesame 0x7f87328 thru 0x7f880d8 data_size 0x1(0xdb1 bytes) 2024-09-21T15:00:51.103 INFO:teuthology.orchestra.run.smithi032.stdout:5008 writesame 0x992b3d0 thru 0x992f887 data_size 0xc(0x44b8 bytes) 2024-09-21T15:00:51.109 INFO:teuthology.orchestra.run.smithi032.stdout:5009 write 0x84302bc thru 0x8434253 (0x3f98 bytes) 2024-09-21T15:00:51.115 INFO:teuthology.orchestra.run.smithi032.stdout:5010 write 0x4b90ca7 thru 0x4b9d4ef (0xc849 bytes) 2024-09-21T15:00:51.123 INFO:teuthology.orchestra.run.smithi032.stdout:5011 punch from 0x21c750e to 0x21cd697, (0x6189 bytes) 2024-09-21T15:00:51.126 INFO:teuthology.orchestra.run.smithi032.stdout:5012 trunc from 0xa5946a5 to 0x8885a06 2024-09-21T15:00:51.136 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:00:51.131+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:00:51.144 INFO:teuthology.orchestra.run.smithi032.stdout:5014 punch from 0x2830cbb to 0x28374ea, (0x682f bytes) 2024-09-21T15:00:51.157 INFO:teuthology.orchestra.run.smithi032.stdout:5015 write 0xb6437cd thru 0xb6512b8 (0xdaec bytes) 2024-09-21T15:00:51.181 INFO:teuthology.orchestra.run.smithi032.stdout:5016 trunc from 0xb6512b9 to 0x34bb300 2024-09-21T15:00:51.186 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:51.186 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:51.197 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T15:00:51.201 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:51.201 INFO:teuthology.orchestra.run.smithi032.stdout:5019 punch from 0xa760cb to 0xa78d0b, (0x2c40 bytes) 2024-09-21T15:00:51.201 INFO:teuthology.orchestra.run.smithi032.stdout:5020 read 0x56f1c9 thru 0x570ac9 (0x1901 bytes) 2024-09-21T15:00:51.222 INFO:teuthology.orchestra.run.smithi032.stdout:5021 writesame 0xbed452c thru 0xbed8749 data_size 0x2a(0x421e bytes) 2024-09-21T15:00:51.222 INFO:teuthology.orchestra.run.smithi032.stdout:5022 read 0xb71f347 thru 0xb727c04 (0x88be bytes) 2024-09-21T15:00:51.222 INFO:teuthology.orchestra.run.smithi032.stdout:5023 read 0x4b67301 thru 0x4b71689 (0xa389 bytes) 2024-09-21T15:00:51.222 INFO:teuthology.orchestra.run.smithi032.stdout:5024 punch from 0x8c00c2b to 0x8c0611b, (0x54f0 bytes) 2024-09-21T15:00:51.222 INFO:teuthology.orchestra.run.smithi032.stdout:5025 read 0x33bd368 thru 0x33c77a6 (0xa43f bytes) 2024-09-21T15:00:51.232 INFO:teuthology.orchestra.run.smithi032.stdout:5026 writesame 0xc8ae691 thru 0xc8af89a data_size 0x1(0x120a bytes) 2024-09-21T15:00:51.232 INFO:teuthology.orchestra.run.smithi032.stdout:5028 read 0x9f0c8d9 thru 0x9f0e7e7 (0x1f0f bytes) 2024-09-21T15:00:51.234 INFO:teuthology.orchestra.run.smithi032.stdout:5030 trunc from 0xc8af89b to 0xdb52c71 2024-09-21T15:00:51.237 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:00:51.233+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:00:51.243 INFO:teuthology.orchestra.run.smithi032.stdout:5031 writesame 0x1e9e29 thru 0x1ed6f5 data_size 0x25(0x38cd bytes) 2024-09-21T15:00:51.247 INFO:teuthology.orchestra.run.smithi032.stdout:5033 read 0xc330800 thru 0xc337997 (0x7198 bytes) 2024-09-21T15:00:51.248 INFO:teuthology.orchestra.run.smithi032.stdout:5034 punch from 0xbb3612b to 0xbb40c27, (0xaafc bytes) 2024-09-21T15:00:51.248 INFO:teuthology.orchestra.run.smithi032.stdout:5036 punch from 0x2485e27 to 0x2492245, (0xc41e bytes) 2024-09-21T15:00:51.248 INFO:teuthology.orchestra.run.smithi032.stdout:5037 read 0x51fc29d thru 0x5200eb0 (0x4c14 bytes) 2024-09-21T15:00:51.265 INFO:teuthology.orchestra.run.smithi032.stdout:5040 trunc from 0xdb52c71 to 0x2fa7ee5 2024-09-21T15:00:51.280 INFO:teuthology.orchestra.run.smithi032.stdout:5041 trunc from 0x2fa7ee5 to 0x1100dfd 2024-09-21T15:00:51.302 INFO:teuthology.orchestra.run.smithi032.stdout:5042 punch from 0x131e08 to 0x1371ef, (0x53e7 bytes) 2024-09-21T15:00:51.308 INFO:teuthology.orchestra.run.smithi032.stdout:5043 trunc from 0x1100dfd to 0x46f918c 2024-09-21T15:00:51.312 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:51.312 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:51.316 INFO:teuthology.orchestra.run.smithi032.stdout:5044 read 0x20701b0 thru 0x2074f8b (0x4ddc bytes) 2024-09-21T15:00:51.326 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T15:00:51.326 INFO:teuthology.orchestra.run.smithi032.stdout:5045 trunc from 0x46f918c to 0x968fe10 2024-09-21T15:00:51.332 INFO:teuthology.orchestra.run.smithi032.stdout:5046 punch from 0x8307022 to 0x8316dd4, (0xfdb2 bytes) 2024-09-21T15:00:51.334 INFO:teuthology.orchestra.run.smithi032.stdout:5047 trunc from 0x968fe10 to 0x8146757 2024-09-21T15:00:51.337 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:00:51.335+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:00:51.344 INFO:teuthology.orchestra.run.smithi032.stdout:5048 read 0x515530b thru 0x515ca19 (0x770f bytes) 2024-09-21T15:00:51.364 INFO:teuthology.orchestra.run.smithi032.stdout:5049 write 0xd61d4a3 thru 0xd620fde (0x3b3c bytes) 2024-09-21T15:00:51.372 INFO:teuthology.orchestra.run.smithi032.stdout:5050 writesame 0x3faa7b2 thru 0x3fb131a data_size 0x1(0x6b69 bytes) 2024-09-21T15:00:51.399 INFO:teuthology.orchestra.run.smithi032.stdout:5051 punch from 0x1bc4b4f to 0x1bce7c7, (0x9c78 bytes) 2024-09-21T15:00:51.400 INFO:teuthology.orchestra.run.smithi032.stdout:5052 write 0x89e8340 thru 0x89f4829 (0xc4ea bytes) 2024-09-21T15:00:51.407 INFO:teuthology.orchestra.run.smithi032.stdout:5054 writesame 0x69c817 thru 0x6a2826 data_size 0x3a(0x6010 bytes) 2024-09-21T15:00:51.416 INFO:teuthology.orchestra.run.smithi032.stdout:5055 read 0x812ba46 thru 0x812f061 (0x361c bytes) 2024-09-21T15:00:51.416 INFO:teuthology.orchestra.run.smithi032.stdout:5056 write 0x8445176 thru 0x84516a6 (0xc531 bytes) 2024-09-21T15:00:51.426 INFO:teuthology.orchestra.run.smithi032.stdout:5058 read 0x6bd5b77 thru 0x6be2dd4 (0xd25e bytes) 2024-09-21T15:00:51.426 INFO:teuthology.orchestra.run.smithi032.stdout:5059 punch from 0xc43efe8 to 0xc4407fc, (0x1814 bytes) 2024-09-21T15:00:51.426 INFO:teuthology.orchestra.run.smithi032.stdout:5061 punch from 0x3dcd40e to 0x3dd8b41, (0xb733 bytes) 2024-09-21T15:00:51.426 INFO:teuthology.orchestra.run.smithi032.stdout:5063 writesame 0x5006cce thru 0x500e1a3 data_size 0xf(0x74d6 bytes) 2024-09-21T15:00:51.438 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:00:51.435+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:00:51.449 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:51.449 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:51.450 INFO:teuthology.orchestra.run.smithi032.stdout:5064 trunc from 0xd620fdf to 0x1a5d3fe 2024-09-21T15:00:51.462 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T15:00:51.489 INFO:teuthology.orchestra.run.smithi032.stdout:5065 write 0x7ed5b06 thru 0x7ee1018 (0xb513 bytes) 2024-09-21T15:00:51.500 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:51.500 INFO:teuthology.orchestra.run.smithi032.stdout:5067 read 0x7944851 thru 0x7951663 (0xce13 bytes) 2024-09-21T15:00:51.511 INFO:teuthology.orchestra.run.smithi032.stdout:5068 write 0x8af589f thru 0x8b00d41 (0xb4a3 bytes) 2024-09-21T15:00:51.531 INFO:teuthology.orchestra.run.smithi032.stdout:5070 write 0xa1e7e7a thru 0xa1f22a8 (0xa42f bytes) 2024-09-21T15:00:51.539 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:00:51.535+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:00:51.540 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:51.540 INFO:teuthology.orchestra.run.smithi032.stdout:5072 read 0xa3c899 thru 0xa4ab6b (0xe2d3 bytes) 2024-09-21T15:00:51.545 INFO:teuthology.orchestra.run.smithi032.stdout:5073 trunc from 0xa1f22a9 to 0x6ff42ad 2024-09-21T15:00:51.563 INFO:teuthology.orchestra.run.smithi032.stdout:5074 write 0x2041c88 thru 0x2046eb1 (0x522a bytes) 2024-09-21T15:00:51.580 INFO:teuthology.orchestra.run.smithi032.stdout:5075 trunc from 0x6ff42ad to 0xe2a4c1c 2024-09-21T15:00:51.588 INFO:teuthology.orchestra.run.smithi032.stdout:5076 trunc from 0xe2a4c1c to 0xe4b05a1 2024-09-21T15:00:51.598 INFO:teuthology.orchestra.run.smithi032.stdout:5077 writesame 0x23e40 thru 0x2a7ee data_size 0x7(0x69af bytes) 2024-09-21T15:00:51.610 INFO:teuthology.orchestra.run.smithi032.stdout:5079 trunc from 0xe4b05a1 to 0xa06845e 2024-09-21T15:00:51.615 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:51.615 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:51.627 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T15:00:51.632 INFO:teuthology.orchestra.run.smithi032.stdout:5080 write 0xab14b42 thru 0xab23e6a (0xf329 bytes) 2024-09-21T15:00:51.639 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:00:51.637+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:00:51.642 INFO:teuthology.orchestra.run.smithi032.stdout:5082 write 0x1732891 thru 0x174146f (0xebdf bytes) 2024-09-21T15:00:51.650 INFO:teuthology.orchestra.run.smithi032.stdout:5084 punch from 0x87e78e5 to 0x87f5f40, (0xe65b bytes) 2024-09-21T15:00:51.650 INFO:teuthology.orchestra.run.smithi032.stdout:5085 read 0x92f3c8b thru 0x9300468 (0xc7de bytes) 2024-09-21T15:00:51.650 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:51.651 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:51.651 INFO:teuthology.orchestra.run.smithi032.stdout:5090 punch from 0xeae870 to 0xeb7b5c, (0x92ec bytes) 2024-09-21T15:00:51.658 INFO:teuthology.orchestra.run.smithi032.stdout:5091 writesame 0xb048569 thru 0xb056b70 data_size 0x44(0xe608 bytes) 2024-09-21T15:00:51.674 INFO:teuthology.orchestra.run.smithi032.stdout:5092 trunc from 0xb056b71 to 0xa00e70 2024-09-21T15:00:51.692 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:51.692 INFO:teuthology.orchestra.run.smithi032.stdout:5095 read 0x71efc8 thru 0x7281a9 (0x91e2 bytes) 2024-09-21T15:00:51.693 INFO:teuthology.orchestra.run.smithi032.stdout:5096 read 0xae647 thru 0xaee98 (0x852 bytes) 2024-09-21T15:00:51.712 INFO:teuthology.orchestra.run.smithi032.stdout:5097 write 0x6c8c868 thru 0x6c9477a (0x7f13 bytes) 2024-09-21T15:00:51.722 INFO:teuthology.orchestra.run.smithi032.stdout:5101 read 0x5dd2ae0 thru 0x5dd52d7 (0x27f8 bytes) 2024-09-21T15:00:51.722 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:51.738 INFO:teuthology.orchestra.run.smithi032.stdout:5103 writesame 0xe08b56d thru 0xe09a22d data_size 0x59(0xecc1 bytes) 2024-09-21T15:00:51.739 INFO:teuthology.orchestra.run.smithi032.stdout:5104 read 0x9964dfb thru 0x997356f (0xe775 bytes) 2024-09-21T15:00:51.739 INFO:teuthology.orchestra.run.smithi032.stdout:5105 write 0xafea613 thru 0xaff5642 (0xb030 bytes) 2024-09-21T15:00:51.740 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:00:51.735+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:00:51.745 INFO:teuthology.orchestra.run.smithi032.stdout:5106 read 0x56a2f97 thru 0x56aeca4 (0xbd0e bytes) 2024-09-21T15:00:51.746 INFO:teuthology.orchestra.run.smithi032.stdout:5107 read 0x479e721 thru 0x479edfe (0x6de bytes) 2024-09-21T15:00:51.746 INFO:teuthology.orchestra.run.smithi032.stdout:5108 write 0xceb8bda thru 0xcec4298 (0xb6bf bytes) 2024-09-21T15:00:51.754 INFO:teuthology.orchestra.run.smithi032.stdout:5110 punch from 0x2f61205 to 0x2f6a45e, (0x9259 bytes) 2024-09-21T15:00:51.754 INFO:teuthology.orchestra.run.smithi032.stdout:5111 writesame 0x14d6a88 thru 0x14dd838 data_size 0x1(0x6db1 bytes) 2024-09-21T15:00:51.757 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:51.757 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:51.771 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T15:00:51.776 INFO:teuthology.orchestra.run.smithi032.stdout:5113 read 0x1d0f23a thru 0x1d13a81 (0x4848 bytes) 2024-09-21T15:00:51.776 INFO:teuthology.orchestra.run.smithi032.stdout:5114 writesame 0xdb8dfc6 thru 0xdb8f0b3 data_size 0x2(0x10ee bytes) 2024-09-21T15:00:51.783 INFO:teuthology.orchestra.run.smithi032.stdout:5115 writesame 0xd2b5d74 thru 0xd2bce16 data_size 0x49(0x70a3 bytes) 2024-09-21T15:00:51.792 INFO:teuthology.orchestra.run.smithi032.stdout:5116 writesame 0x33c7c1a thru 0x33cf4bb data_size 0x3(0x78a2 bytes) 2024-09-21T15:00:51.804 INFO:teuthology.orchestra.run.smithi032.stdout:5117 writesame 0xaa6068 thru 0xaa965c data_size 0x1(0x35f5 bytes) 2024-09-21T15:00:51.820 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:51.821 INFO:teuthology.orchestra.run.smithi032.stdout:5119 read 0x19a4d9 thru 0x1a26da (0x8202 bytes) 2024-09-21T15:00:51.822 INFO:teuthology.orchestra.run.smithi032.stdout:5120 write 0x1461fe9 thru 0x14695fd (0x7615 bytes) 2024-09-21T15:00:51.829 INFO:teuthology.orchestra.run.smithi032.stdout:5121 write 0xdf12d14 thru 0xdf1f402 (0xc6ef bytes) 2024-09-21T15:00:51.835 INFO:teuthology.orchestra.run.smithi032.stdout:5122 read 0x3ce070f thru 0x3cef06a (0xe95c bytes) 2024-09-21T15:00:51.837 INFO:teuthology.orchestra.run.smithi032.stdout:5123 trunc from 0xe09a22e to 0xea36c70 2024-09-21T15:00:51.841 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:00:51.839+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:00:51.845 INFO:teuthology.orchestra.run.smithi032.stdout:5125 write 0x63d3f36 thru 0x63d4a17 (0xae2 bytes) 2024-09-21T15:00:51.850 INFO:teuthology.orchestra.run.smithi032.stdout:5126 write 0xca59e93 thru 0xca67f76 (0xe0e4 bytes) 2024-09-21T15:00:51.858 INFO:teuthology.orchestra.run.smithi032.stdout:5128 write 0xa780ca9 thru 0xa790825 (0xfb7d bytes) 2024-09-21T15:00:51.870 INFO:teuthology.orchestra.run.smithi032.stdout:5131 read 0x609e92a thru 0x60ae4ae (0xfb85 bytes) 2024-09-21T15:00:51.870 INFO:teuthology.orchestra.run.smithi032.stdout:5133 write 0xd26f0 thru 0xde84b (0xc15c bytes) 2024-09-21T15:00:51.891 INFO:teuthology.orchestra.run.smithi032.stdout:5134 trunc from 0xea36c70 to 0x33f091d 2024-09-21T15:00:51.899 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:51.900 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:51.913 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T15:00:51.914 INFO:teuthology.orchestra.run.smithi032.stdout:5136 read 0x803a6b thru 0x809e73 (0x6409 bytes) 2024-09-21T15:00:51.916 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T15:00:51.919 INFO:teuthology.orchestra.run.smithi032.stdout:5137 trunc from 0x33f091d to 0x599bef8 2024-09-21T15:00:51.930 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:51.930 INFO:teuthology.orchestra.run.smithi032.stdout:5139 punch from 0x4145319 to 0x41480c5, (0x2dac bytes) 2024-09-21T15:00:51.930 INFO:teuthology.orchestra.run.smithi032.stdout:5140 read 0x255ddc1 thru 0x2565405 (0x7645 bytes) 2024-09-21T15:00:51.941 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:00:51.937+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:00:51.950 INFO:teuthology.orchestra.run.smithi032.stdout:5142 writesame 0xd0e2c04 thru 0xd0e4a40 data_size 0x1(0x1e3d bytes) 2024-09-21T15:00:51.951 INFO:teuthology.orchestra.run.smithi032.stdout:5145 write 0x7fcecb thru 0x8050bb (0x81f1 bytes) 2024-09-21T15:00:51.957 INFO:teuthology.orchestra.run.smithi032.stdout:5146 write 0x80e4a4f thru 0x80eba22 (0x6fd4 bytes) 2024-09-21T15:00:51.965 INFO:teuthology.orchestra.run.smithi032.stdout:5148 punch from 0x577db37 to 0x577fbc2, (0x208b bytes) 2024-09-21T15:00:51.975 INFO:teuthology.orchestra.run.smithi032.stdout:5150 trunc from 0xd0e4a41 to 0x6ba3e8a 2024-09-21T15:00:51.991 INFO:teuthology.orchestra.run.smithi032.stdout:5151 read 0x2e4cc86 thru 0x2e4cd2c (0xa7 bytes) 2024-09-21T15:00:52.008 INFO:teuthology.orchestra.run.smithi032.stdout:5152 writesame 0xbcdc4d9 thru 0xbce88d5 data_size 0x1(0xc3fd bytes) 2024-09-21T15:00:52.009 INFO:teuthology.orchestra.run.smithi032.stdout:5153 read 0x8ffdfad thru 0x8ffe581 (0x5d5 bytes) 2024-09-21T15:00:52.027 INFO:teuthology.orchestra.run.smithi032.stdout:5154 trunc from 0xbce88d6 to 0xc1da06 2024-09-21T15:00:52.040 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:52.040 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:52.043 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:00:52.041+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:00:52.053 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T15:00:52.057 INFO:teuthology.orchestra.run.smithi032.stdout:5155 write 0x1675ef1 thru 0x16853d0 (0xf4e0 bytes) 2024-09-21T15:00:52.064 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T15:00:52.066 INFO:teuthology.orchestra.run.smithi032.stdout:5156 read 0x909139 thru 0x9183dd (0xf2a5 bytes) 2024-09-21T15:00:52.080 INFO:teuthology.orchestra.run.smithi032.stdout:5157 trunc from 0x16853d1 to 0xab11c78 2024-09-21T15:00:52.161 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T15:00:52.162 INFO:teuthology.orchestra.run.smithi032.stdout:5158 write 0x805fe4e thru 0x806c280 (0xc433 bytes) 2024-09-21T15:00:52.162 INFO:teuthology.orchestra.run.smithi032.stdout:5159 read 0x24ae847 thru 0x24ba1d5 (0xb98f bytes) 2024-09-21T15:00:52.162 INFO:teuthology.orchestra.run.smithi032.stdout:5160 read 0xc4006d thru 0xc48a90 (0x8a24 bytes) 2024-09-21T15:00:52.162 INFO:teuthology.orchestra.run.smithi032.stdout:5161 clone 34 order 21 su 8192 sc 2 2024-09-21T15:00:52.162 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T15:00:52.162 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T15:00:52.162 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-21T15:00:52.163 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:00:52.157+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:00:52.165 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:52.166 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T15:00:52.166 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T15:00:52.166 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T15:00:52.166 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.166 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:15130 3.2 3:47f1bbdb:::rbd_data.11361bed1f78.0000000000000067:head [sparse-read 0~147456] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-21T15:00:52.166 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:40.288284+0000", 2024-09-21T15:00:52.166 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 131.87704579000001, 2024-09-21T15:00:52.166 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1142935039999999, 2024-09-21T15:00:52.166 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:52.166 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:52.166 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:52.166 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:52.167 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:52.167 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 15130 2024-09-21T15:00:52.167 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.167 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:52.167 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.167 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:52.167 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288284+0000", 2024-09-21T15:00:52.167 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:52.167 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.167 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.167 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:52.167 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288284+0000", 2024-09-21T15:00:52.167 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:52.167 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.168 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.168 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:52.168 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288284+0000", 2024-09-21T15:00:52.168 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:52.168 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.168 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.168 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:52.168 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288288+0000", 2024-09-21T15:00:52.168 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5829999999999998e-06 2024-09-21T15:00:52.168 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.168 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.168 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:52.168 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288297+0000", 2024-09-21T15:00:52.168 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0669999999999998e-06 2024-09-21T15:00:52.168 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.169 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.169 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:52.169 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288305+0000", 2024-09-21T15:00:52.169 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8860000000000004e-06 2024-09-21T15:00:52.169 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.169 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.169 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:52.169 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288417+0000", 2024-09-21T15:00:52.169 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000112952 2024-09-21T15:00:52.169 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.169 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.169 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:00:52.169 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288428+0000", 2024-09-21T15:00:52.169 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0631e-05 2024-09-21T15:00:52.170 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.170 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.170 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:52.170 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.401876+0000", 2024-09-21T15:00:52.170 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1134479610000001 2024-09-21T15:00:52.170 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.170 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.170 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:52.170 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.402076+0000", 2024-09-21T15:00:52.170 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00019944200000000001 2024-09-21T15:00:52.170 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.170 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.170 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:52.170 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.402578+0000", 2024-09-21T15:00:52.171 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00050242899999999998 2024-09-21T15:00:52.171 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:52.171 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:52.171 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:52.171 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.171 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.171 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45708 3.2 3:415db91a:::rbd_data.1136ecbe5879.0000000000000000:28 [read 140288~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:52.171 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141801+0000", 2024-09-21T15:00:52.171 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 102.023529596, 2024-09-21T15:00:52.171 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034274200999999997, 2024-09-21T15:00:52.171 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:52.171 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:52.171 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:52.171 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:52.172 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:52.172 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45708 2024-09-21T15:00:52.172 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.172 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:52.172 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.172 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:52.172 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141801+0000", 2024-09-21T15:00:52.172 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:52.172 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.172 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.172 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:52.172 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141801+0000", 2024-09-21T15:00:52.172 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:52.172 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.172 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.173 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:52.173 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141800+0000", 2024-09-21T15:00:52.173 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:52.173 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.173 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.173 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:52.173 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141803+0000", 2024-09-21T15:00:52.173 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4210000000000002e-06 2024-09-21T15:00:52.173 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.173 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.173 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:52.173 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141812+0000", 2024-09-21T15:00:52.173 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.9800000000000004e-06 2024-09-21T15:00:52.173 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.174 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.174 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:52.174 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141816+0000", 2024-09-21T15:00:52.174 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4769999999999997e-06 2024-09-21T15:00:52.174 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.174 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.174 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:52.174 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.175865+0000", 2024-09-21T15:00:52.174 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034048875999999999 2024-09-21T15:00:52.174 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.174 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.174 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:52.174 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.175943+0000", 2024-09-21T15:00:52.174 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7464000000000002e-05 2024-09-21T15:00:52.175 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.175 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.175 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:52.175 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176075+0000", 2024-09-21T15:00:52.175 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013230800000000001 2024-09-21T15:00:52.175 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:52.175 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:52.175 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:52.175 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.175 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.175 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45709 3.2 3:415db91a:::rbd_data.1136ecbe5879.0000000000000000:28 [read 143360~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:52.175 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141820+0000", 2024-09-21T15:00:52.175 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 102.023509749, 2024-09-21T15:00:52.175 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034537536000000001, 2024-09-21T15:00:52.176 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:52.176 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:52.176 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:52.176 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:52.176 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:52.176 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45709 2024-09-21T15:00:52.176 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.176 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:52.176 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.176 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:52.176 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141820+0000", 2024-09-21T15:00:52.176 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:52.176 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.176 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.176 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:52.177 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141820+0000", 2024-09-21T15:00:52.177 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:52.177 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.177 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.177 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:52.177 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141820+0000", 2024-09-21T15:00:52.177 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:52.177 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.177 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.177 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:52.177 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141823+0000", 2024-09-21T15:00:52.177 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5110000000000002e-06 2024-09-21T15:00:52.177 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.177 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.178 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:52.178 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141829+0000", 2024-09-21T15:00:52.178 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0850000000000002e-06 2024-09-21T15:00:52.178 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.178 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.178 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:52.178 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141833+0000", 2024-09-21T15:00:52.178 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7620000000000003e-06 2024-09-21T15:00:52.178 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.178 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.178 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:52.178 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176103+0000", 2024-09-21T15:00:52.178 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034269933000000002 2024-09-21T15:00:52.178 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.179 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.179 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:52.179 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176161+0000", 2024-09-21T15:00:52.179 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7658999999999998e-05 2024-09-21T15:00:52.179 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.179 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.179 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:52.179 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176358+0000", 2024-09-21T15:00:52.179 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000196937 2024-09-21T15:00:52.179 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:52.179 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:52.179 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:52.179 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.179 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.180 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45710 3.2 3:415db91a:::rbd_data.1136ecbe5879.0000000000000000:28 [read 146432~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:52.180 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141838+0000", 2024-09-21T15:00:52.180 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 102.023492586, 2024-09-21T15:00:52.180 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034953674999999997, 2024-09-21T15:00:52.180 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:52.180 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:52.180 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:52.180 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:52.180 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:52.180 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45710 2024-09-21T15:00:52.180 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.180 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:52.180 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.180 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:52.181 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141838+0000", 2024-09-21T15:00:52.181 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:52.181 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.181 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.181 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:52.181 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141838+0000", 2024-09-21T15:00:52.181 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:52.181 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.181 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.181 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:52.181 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141837+0000", 2024-09-21T15:00:52.181 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:52.181 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.181 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.182 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:52.182 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141843+0000", 2024-09-21T15:00:52.182 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1619999999999996e-06 2024-09-21T15:00:52.182 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.182 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.182 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:52.182 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141850+0000", 2024-09-21T15:00:52.182 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4799999999999998e-06 2024-09-21T15:00:52.182 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.182 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.182 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:52.182 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141853+0000", 2024-09-21T15:00:52.182 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3050000000000001e-06 2024-09-21T15:00:52.182 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.183 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.183 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:52.183 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176490+0000", 2024-09-21T15:00:52.183 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034636413999999997 2024-09-21T15:00:52.183 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.183 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.183 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:52.183 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176606+0000", 2024-09-21T15:00:52.183 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000115841 2024-09-21T15:00:52.183 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.183 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.183 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:52.183 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176791+0000", 2024-09-21T15:00:52.183 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00018581500000000001 2024-09-21T15:00:52.183 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:52.184 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:52.184 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:52.184 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.184 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.184 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45711 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:52.184 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141856+0000", 2024-09-21T15:00:52.184 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 102.023474249, 2024-09-21T15:00:52.184 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034986308000000001, 2024-09-21T15:00:52.184 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:52.184 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:52.184 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:52.184 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:52.184 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:52.185 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45711 2024-09-21T15:00:52.185 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.185 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:52.185 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.185 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:52.185 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141856+0000", 2024-09-21T15:00:52.185 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:52.185 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.185 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.185 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:52.185 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141856+0000", 2024-09-21T15:00:52.185 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:52.185 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.185 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.186 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:52.186 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141856+0000", 2024-09-21T15:00:52.186 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:52.186 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.186 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.186 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:52.186 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141862+0000", 2024-09-21T15:00:52.186 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.567e-06 2024-09-21T15:00:52.186 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.186 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.186 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:52.186 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141868+0000", 2024-09-21T15:00:52.186 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9569999999999999e-06 2024-09-21T15:00:52.186 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.186 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.187 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:52.187 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141871+0000", 2024-09-21T15:00:52.187 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.146e-06 2024-09-21T15:00:52.187 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.187 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.187 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:52.187 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176831+0000", 2024-09-21T15:00:52.187 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034959935999999997 2024-09-21T15:00:52.187 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.187 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.187 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:52.187 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176842+0000", 2024-09-21T15:00:52.187 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1008e-05 2024-09-21T15:00:52.188 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:52.188 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:52.188 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:52.188 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.188 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.188 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45712 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:52.188 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141874+0000", 2024-09-21T15:00:52.188 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 102.023456354, 2024-09-21T15:00:52.188 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034993511999999997, 2024-09-21T15:00:52.188 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:52.188 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:52.188 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:52.188 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:52.188 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:52.188 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45712 2024-09-21T15:00:52.189 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.189 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:52.189 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.189 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:52.189 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141874+0000", 2024-09-21T15:00:52.189 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:52.189 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.189 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.189 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:52.189 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141874+0000", 2024-09-21T15:00:52.189 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:52.189 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.189 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.189 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:52.190 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141874+0000", 2024-09-21T15:00:52.190 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:52.190 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.190 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.190 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:52.190 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141875+0000", 2024-09-21T15:00:52.190 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5430000000000001e-06 2024-09-21T15:00:52.190 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.190 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.190 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:52.190 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141878+0000", 2024-09-21T15:00:52.190 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1420000000000002e-06 2024-09-21T15:00:52.190 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.190 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.191 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:52.191 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141881+0000", 2024-09-21T15:00:52.191 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.807e-06 2024-09-21T15:00:52.191 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.191 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.191 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:52.191 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176860+0000", 2024-09-21T15:00:52.191 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034979243 2024-09-21T15:00:52.191 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.191 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.191 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:52.191 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176867+0000", 2024-09-21T15:00:52.191 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0169999999999999e-06 2024-09-21T15:00:52.191 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:52.191 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:52.192 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:52.192 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.192 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.192 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45713 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:52.192 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141883+0000", 2024-09-21T15:00:52.192 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 102.023446779, 2024-09-21T15:00:52.192 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035010516999999998, 2024-09-21T15:00:52.192 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:52.192 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:52.192 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:52.192 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:52.192 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:52.192 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45713 2024-09-21T15:00:52.192 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.193 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:52.193 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.193 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:52.193 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141883+0000", 2024-09-21T15:00:52.193 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:52.193 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.193 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.193 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:52.193 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141883+0000", 2024-09-21T15:00:52.193 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:52.193 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.193 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.193 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:52.193 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141883+0000", 2024-09-21T15:00:52.194 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:52.194 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.194 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.194 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:52.194 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141885+0000", 2024-09-21T15:00:52.194 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4649999999999999e-06 2024-09-21T15:00:52.194 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.194 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.194 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:52.194 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141888+0000", 2024-09-21T15:00:52.194 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3890000000000001e-06 2024-09-21T15:00:52.194 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.194 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.194 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:52.195 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141891+0000", 2024-09-21T15:00:52.195 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9239999999999999e-06 2024-09-21T15:00:52.195 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.195 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.195 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:52.195 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176887+0000", 2024-09-21T15:00:52.195 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034996370999999998 2024-09-21T15:00:52.195 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.195 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.195 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:52.195 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176894+0000", 2024-09-21T15:00:52.195 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6069999999999996e-06 2024-09-21T15:00:52.195 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:52.196 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:52.196 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:52.196 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.196 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.196 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45714 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:52.196 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141893+0000", 2024-09-21T15:00:52.196 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 102.023436864, 2024-09-21T15:00:52.196 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035048215000000001, 2024-09-21T15:00:52.196 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:52.196 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:52.196 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:52.196 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:52.196 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:52.197 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45714 2024-09-21T15:00:52.197 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.197 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:52.197 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.197 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:52.197 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141893+0000", 2024-09-21T15:00:52.197 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:52.197 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.197 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.197 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:52.197 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141893+0000", 2024-09-21T15:00:52.197 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:52.197 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.197 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.197 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:52.198 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141893+0000", 2024-09-21T15:00:52.198 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:52.198 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.198 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.198 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:52.198 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141895+0000", 2024-09-21T15:00:52.198 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.703e-06 2024-09-21T15:00:52.198 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.198 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.198 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:52.198 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141900+0000", 2024-09-21T15:00:52.198 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.242e-06 2024-09-21T15:00:52.198 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.198 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.199 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:52.199 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141903+0000", 2024-09-21T15:00:52.199 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.117e-06 2024-09-21T15:00:52.199 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.199 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.199 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:52.199 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176930+0000", 2024-09-21T15:00:52.199 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035026750000000002 2024-09-21T15:00:52.199 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.199 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.199 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:52.199 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176942+0000", 2024-09-21T15:00:52.199 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1668000000000001e-05 2024-09-21T15:00:52.199 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:52.199 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:52.200 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:52.200 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.200 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.200 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45715 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:52.200 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141906+0000", 2024-09-21T15:00:52.200 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 102.023424706, 2024-09-21T15:00:52.200 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035066057999999997, 2024-09-21T15:00:52.200 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:52.200 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:52.200 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:52.200 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:52.200 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:52.200 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45715 2024-09-21T15:00:52.200 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.201 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:52.201 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.201 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:52.201 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141906+0000", 2024-09-21T15:00:52.201 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:52.201 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.201 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.201 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:52.201 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141906+0000", 2024-09-21T15:00:52.201 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:52.201 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.201 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.201 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:52.201 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141905+0000", 2024-09-21T15:00:52.201 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:52.202 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.202 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.202 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:52.202 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141907+0000", 2024-09-21T15:00:52.202 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5400000000000001e-06 2024-09-21T15:00:52.202 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.202 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.202 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:52.202 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141912+0000", 2024-09-21T15:00:52.202 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.502e-06 2024-09-21T15:00:52.202 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.202 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.202 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:52.202 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141915+0000", 2024-09-21T15:00:52.203 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0170000000000001e-06 2024-09-21T15:00:52.203 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.203 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.203 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:52.203 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176964+0000", 2024-09-21T15:00:52.203 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035048587999999999 2024-09-21T15:00:52.203 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.203 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.203 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:52.203 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176972+0000", 2024-09-21T15:00:52.203 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6580000000000002e-06 2024-09-21T15:00:52.203 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:52.203 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:52.204 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:52.204 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.204 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.204 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45716 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:52.204 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141918+0000", 2024-09-21T15:00:52.204 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 102.02341253, 2024-09-21T15:00:52.204 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035086223, 2024-09-21T15:00:52.204 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:52.204 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:52.204 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:52.204 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:52.204 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:52.204 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45716 2024-09-21T15:00:52.204 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.205 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:52.205 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.205 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:52.205 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141918+0000", 2024-09-21T15:00:52.205 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:52.205 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.205 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.205 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:52.205 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141918+0000", 2024-09-21T15:00:52.205 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:52.205 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.205 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.205 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:52.205 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141917+0000", 2024-09-21T15:00:52.206 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:52.206 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.206 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.206 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:52.206 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141919+0000", 2024-09-21T15:00:52.206 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5680000000000001e-06 2024-09-21T15:00:52.206 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.206 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.206 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:52.206 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141922+0000", 2024-09-21T15:00:52.206 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2789999999999999e-06 2024-09-21T15:00:52.206 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.206 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.206 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:52.207 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141925+0000", 2024-09-21T15:00:52.207 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9469999999999999e-06 2024-09-21T15:00:52.207 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.207 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.207 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:52.207 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176997+0000", 2024-09-21T15:00:52.207 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035071366999999999 2024-09-21T15:00:52.207 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.207 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.207 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:52.207 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177004+0000", 2024-09-21T15:00:52.207 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3309999999999998e-06 2024-09-21T15:00:52.207 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:52.207 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:52.208 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:52.208 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.208 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.208 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45717 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:52.208 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141928+0000", 2024-09-21T15:00:52.208 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 102.023402584, 2024-09-21T15:00:52.208 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035102573999999998, 2024-09-21T15:00:52.208 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:52.208 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:52.208 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:52.208 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:52.208 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:52.208 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45717 2024-09-21T15:00:52.209 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.209 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:52.209 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.209 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:52.209 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141928+0000", 2024-09-21T15:00:52.209 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:52.209 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.209 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.209 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:52.209 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141928+0000", 2024-09-21T15:00:52.209 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:52.209 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.209 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.209 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:52.209 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141927+0000", 2024-09-21T15:00:52.210 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:00:52.210 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.210 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.210 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:52.210 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141929+0000", 2024-09-21T15:00:52.210 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.533e-06 2024-09-21T15:00:52.210 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.210 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.210 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:52.210 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141934+0000", 2024-09-21T15:00:52.210 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1660000000000002e-06 2024-09-21T15:00:52.210 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.210 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.210 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:52.210 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141937+0000", 2024-09-21T15:00:52.211 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0309999999999999e-06 2024-09-21T15:00:52.211 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.211 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.211 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:52.211 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177023+0000", 2024-09-21T15:00:52.211 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035085595999999997 2024-09-21T15:00:52.211 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.211 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.211 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:52.211 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177030+0000", 2024-09-21T15:00:52.211 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.486e-06 2024-09-21T15:00:52.211 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:52.211 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:52.211 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:52.212 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.212 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.212 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45718 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:52.212 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141940+0000", 2024-09-21T15:00:52.212 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 102.023390726, 2024-09-21T15:00:52.212 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035124956999999998, 2024-09-21T15:00:52.212 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:52.212 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:52.212 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:52.212 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:52.212 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:52.212 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45718 2024-09-21T15:00:52.212 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.212 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:52.212 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.212 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:52.213 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141940+0000", 2024-09-21T15:00:52.213 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:52.213 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.213 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.213 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:52.213 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141940+0000", 2024-09-21T15:00:52.213 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:52.213 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.213 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.213 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:52.213 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141939+0000", 2024-09-21T15:00:52.213 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:52.213 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.213 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.214 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:52.214 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141941+0000", 2024-09-21T15:00:52.214 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.502e-06 2024-09-21T15:00:52.214 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.214 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.214 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:52.214 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141946+0000", 2024-09-21T15:00:52.214 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3449999999999996e-06 2024-09-21T15:00:52.214 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.214 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.214 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:52.214 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141949+0000", 2024-09-21T15:00:52.214 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9749999999999999e-06 2024-09-21T15:00:52.214 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.215 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.215 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:52.215 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177057+0000", 2024-09-21T15:00:52.215 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035108385999999998 2024-09-21T15:00:52.215 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.215 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.215 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:52.215 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177064+0000", 2024-09-21T15:00:52.215 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0099999999999998e-06 2024-09-21T15:00:52.215 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:52.215 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:52.215 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:52.215 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.215 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.215 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45719 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:52.215 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141952+0000", 2024-09-21T15:00:52.216 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 102.02337845300001, 2024-09-21T15:00:52.216 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035139108000000002, 2024-09-21T15:00:52.216 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:52.216 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:52.216 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:52.216 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:52.216 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:52.216 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45719 2024-09-21T15:00:52.216 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.216 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:52.216 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.216 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:52.216 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141952+0000", 2024-09-21T15:00:52.216 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:52.216 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.217 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.217 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:52.217 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141952+0000", 2024-09-21T15:00:52.217 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:52.217 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.217 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.217 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:52.217 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141952+0000", 2024-09-21T15:00:52.217 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:52.217 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.217 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.217 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:52.217 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141953+0000", 2024-09-21T15:00:52.217 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.486e-06 2024-09-21T15:00:52.217 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.218 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.218 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:52.218 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141959+0000", 2024-09-21T15:00:52.218 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4940000000000003e-06 2024-09-21T15:00:52.218 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.218 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.218 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:52.218 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141963+0000", 2024-09-21T15:00:52.218 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3540000000000002e-06 2024-09-21T15:00:52.218 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.218 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.218 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:52.218 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177084+0000", 2024-09-21T15:00:52.218 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035120983000000001 2024-09-21T15:00:52.218 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.219 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.219 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:52.219 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177091+0000", 2024-09-21T15:00:52.219 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0350000000000004e-06 2024-09-21T15:00:52.219 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:52.219 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:52.219 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:52.219 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.219 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.219 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45720 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:52.219 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141965+0000", 2024-09-21T15:00:52.219 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 102.023364986, 2024-09-21T15:00:52.219 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035155268000000003, 2024-09-21T15:00:52.219 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:52.220 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:52.220 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:52.220 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:52.220 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:52.220 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45720 2024-09-21T15:00:52.220 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.220 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:52.220 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.220 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:52.220 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141965+0000", 2024-09-21T15:00:52.220 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:52.220 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.220 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.220 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:52.220 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141965+0000", 2024-09-21T15:00:52.221 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:52.221 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.221 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.221 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:52.221 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141965+0000", 2024-09-21T15:00:52.221 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:52.221 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.221 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.221 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:52.221 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141966+0000", 2024-09-21T15:00:52.221 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.46e-06 2024-09-21T15:00:52.221 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.221 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.221 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:52.222 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141972+0000", 2024-09-21T15:00:52.222 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4210000000000003e-06 2024-09-21T15:00:52.222 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.222 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.222 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:52.222 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141975+0000", 2024-09-21T15:00:52.222 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.948e-06 2024-09-21T15:00:52.222 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.222 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.222 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:52.222 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177113+0000", 2024-09-21T15:00:52.222 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035137855000000003 2024-09-21T15:00:52.222 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.222 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.223 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:52.223 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177121+0000", 2024-09-21T15:00:52.223 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8269999999999998e-06 2024-09-21T15:00:52.223 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:52.223 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:52.223 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:52.223 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.223 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.223 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45721 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:52.223 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141977+0000", 2024-09-21T15:00:52.223 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 102.02335296299999, 2024-09-21T15:00:52.223 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035169984000000001, 2024-09-21T15:00:52.223 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:52.224 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:52.224 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:52.224 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:52.224 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:52.224 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45721 2024-09-21T15:00:52.224 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.224 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:52.224 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.224 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:52.224 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141977+0000", 2024-09-21T15:00:52.224 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:52.224 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.224 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.224 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:52.225 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141977+0000", 2024-09-21T15:00:52.225 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:52.225 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.225 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.225 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:52.225 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141977+0000", 2024-09-21T15:00:52.225 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:52.225 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.225 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.225 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:52.225 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141979+0000", 2024-09-21T15:00:52.225 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.511e-06 2024-09-21T15:00:52.225 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.225 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.226 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:52.226 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141982+0000", 2024-09-21T15:00:52.226 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4180000000000001e-06 2024-09-21T15:00:52.226 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.226 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.226 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:52.226 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141985+0000", 2024-09-21T15:00:52.226 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9450000000000002e-06 2024-09-21T15:00:52.226 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.226 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.226 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:52.226 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177139+0000", 2024-09-21T15:00:52.226 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035154409999999997 2024-09-21T15:00:52.227 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.227 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.227 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:52.227 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177147+0000", 2024-09-21T15:00:52.227 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9570000000000002e-06 2024-09-21T15:00:52.227 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:52.227 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:52.227 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:52.227 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.227 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.227 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45722 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:52.227 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141987+0000", 2024-09-21T15:00:52.227 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 102.023342992, 2024-09-21T15:00:52.228 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035188492000000002, 2024-09-21T15:00:52.228 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:52.228 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:52.228 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:52.228 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:52.228 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:52.228 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45722 2024-09-21T15:00:52.228 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.228 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:52.228 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.228 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:52.228 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141987+0000", 2024-09-21T15:00:52.228 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:52.228 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.229 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.229 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:52.229 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141987+0000", 2024-09-21T15:00:52.229 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:52.229 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.229 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.229 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:52.229 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141987+0000", 2024-09-21T15:00:52.229 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:52.229 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.229 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.229 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:52.229 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141988+0000", 2024-09-21T15:00:52.229 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.463e-06 2024-09-21T15:00:52.229 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.230 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.230 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:52.230 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141994+0000", 2024-09-21T15:00:52.230 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.361e-06 2024-09-21T15:00:52.230 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.230 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.230 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:52.230 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141997+0000", 2024-09-21T15:00:52.230 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0359999999999998e-06 2024-09-21T15:00:52.230 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.230 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.230 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:52.230 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177169+0000", 2024-09-21T15:00:52.230 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035171651999999998 2024-09-21T15:00:52.231 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.231 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.231 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:52.231 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177176+0000", 2024-09-21T15:00:52.231 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2189999999999998e-06 2024-09-21T15:00:52.231 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:52.231 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:52.231 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:52.231 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.231 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.231 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45723 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:52.231 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141999+0000", 2024-09-21T15:00:52.231 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 102.023330825, 2024-09-21T15:00:52.231 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035197701999999997, 2024-09-21T15:00:52.231 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:52.232 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:52.232 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:52.232 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:52.232 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:52.232 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45723 2024-09-21T15:00:52.232 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.232 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:52.232 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.232 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:52.232 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141999+0000", 2024-09-21T15:00:52.232 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:52.232 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.232 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.233 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:52.233 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141999+0000", 2024-09-21T15:00:52.233 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:52.233 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.233 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.233 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:52.233 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141999+0000", 2024-09-21T15:00:52.233 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:52.233 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.233 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.233 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:52.233 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142003+0000", 2024-09-21T15:00:52.233 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5049999999999999e-06 2024-09-21T15:00:52.233 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.234 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.234 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:52.234 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142008+0000", 2024-09-21T15:00:52.234 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6579999999999999e-06 2024-09-21T15:00:52.234 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.234 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.234 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:52.234 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142011+0000", 2024-09-21T15:00:52.234 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0529999999999998e-06 2024-09-21T15:00:52.234 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.234 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.234 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:52.234 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177194+0000", 2024-09-21T15:00:52.234 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035182221999999999 2024-09-21T15:00:52.235 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.235 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.235 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:52.235 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177197+0000", 2024-09-21T15:00:52.235 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5030000000000002e-06 2024-09-21T15:00:52.235 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:52.235 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:52.235 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:52.235 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.235 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.235 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45724 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:52.235 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.142015+0000", 2024-09-21T15:00:52.235 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 102.02331485800001, 2024-09-21T15:00:52.235 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035195345000000003, 2024-09-21T15:00:52.236 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:52.236 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:52.236 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:52.236 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:52.236 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:52.236 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45724 2024-09-21T15:00:52.236 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.236 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:52.236 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.236 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:52.236 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142015+0000", 2024-09-21T15:00:52.236 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:52.236 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.236 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.237 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:52.237 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142015+0000", 2024-09-21T15:00:52.237 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:52.237 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.237 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.237 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:52.237 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142015+0000", 2024-09-21T15:00:52.237 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:52.237 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.237 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.237 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:52.237 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142017+0000", 2024-09-21T15:00:52.237 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5540000000000001e-06 2024-09-21T15:00:52.237 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.237 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.238 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:52.238 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142022+0000", 2024-09-21T15:00:52.238 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4229999999999996e-06 2024-09-21T15:00:52.238 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.238 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.238 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:52.238 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142025+0000", 2024-09-21T15:00:52.238 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.056e-06 2024-09-21T15:00:52.238 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.238 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.238 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:52.238 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177207+0000", 2024-09-21T15:00:52.238 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035182089 2024-09-21T15:00:52.238 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.239 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.239 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:52.239 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177211+0000", 2024-09-21T15:00:52.239 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4649999999999999e-06 2024-09-21T15:00:52.239 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:52.239 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:52.239 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:52.239 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.239 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.239 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45725 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:52.239 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.142028+0000", 2024-09-21T15:00:52.239 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 102.023302579, 2024-09-21T15:00:52.239 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035195731000000001, 2024-09-21T15:00:52.239 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:52.239 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:52.240 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:52.240 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:52.240 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:52.240 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45725 2024-09-21T15:00:52.240 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.240 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:52.240 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.240 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:52.240 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142028+0000", 2024-09-21T15:00:52.240 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:52.240 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.240 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.240 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:52.240 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142028+0000", 2024-09-21T15:00:52.240 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:52.241 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.241 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.241 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:52.241 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142027+0000", 2024-09-21T15:00:52.241 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:52.241 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.241 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.241 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:52.241 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142029+0000", 2024-09-21T15:00:52.241 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5269999999999999e-06 2024-09-21T15:00:52.241 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.241 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.241 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:52.241 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142032+0000", 2024-09-21T15:00:52.242 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3249999999999999e-06 2024-09-21T15:00:52.242 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.242 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.242 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:52.242 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142035+0000", 2024-09-21T15:00:52.242 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.897e-06 2024-09-21T15:00:52.242 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.242 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.242 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:52.242 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177220+0000", 2024-09-21T15:00:52.242 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035184871 2024-09-21T15:00:52.242 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.242 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.242 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:52.242 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177223+0000", 2024-09-21T15:00:52.243 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3680000000000002e-06 2024-09-21T15:00:52.243 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:52.243 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:52.243 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:52.243 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.243 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.243 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45726 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:52.243 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.142037+0000", 2024-09-21T15:00:52.243 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 102.023292864, 2024-09-21T15:00:52.243 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035199273000000003, 2024-09-21T15:00:52.243 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:52.243 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:52.243 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:52.243 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:52.243 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:52.244 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45726 2024-09-21T15:00:52.244 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.244 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:52.244 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.244 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:52.244 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142037+0000", 2024-09-21T15:00:52.244 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:52.244 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.244 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.244 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:52.244 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142037+0000", 2024-09-21T15:00:52.244 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:52.244 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.244 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.245 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:52.245 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142037+0000", 2024-09-21T15:00:52.245 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:52.245 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.245 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.245 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:52.245 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142038+0000", 2024-09-21T15:00:52.245 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.344e-06 2024-09-21T15:00:52.245 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.245 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.245 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:52.245 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142044+0000", 2024-09-21T15:00:52.245 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1930000000000001e-06 2024-09-21T15:00:52.245 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.245 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.246 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:52.246 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142047+0000", 2024-09-21T15:00:52.246 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0280000000000001e-06 2024-09-21T15:00:52.246 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.246 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.246 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:52.246 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177233+0000", 2024-09-21T15:00:52.246 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035186394000000003 2024-09-21T15:00:52.246 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.246 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.246 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:52.246 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177237+0000", 2024-09-21T15:00:52.246 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5530000000000001e-06 2024-09-21T15:00:52.246 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:52.247 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:52.247 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:52.247 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:52.247 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:52.247 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:52.247 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T15:00:52.249 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T15:00:52.249 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T15:00:52.249 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T15:00:52.249 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T15:00:52.250 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:00:52.250 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:00:52.250 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:00:52.250 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:00:52.250 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:00:52.250 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:00:52.251 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:00:52.251 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:00:52.291 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:00:52.287+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:00:52.363 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:00:52.359+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:00:52.415 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:52.415 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:00:52.415 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T15:00:52.415 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:52.425 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T15:00:52.469 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:00:52.467+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:00:52.542 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:52.542 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:00:52.542 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T15:00:52.542 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T15:00:52.542 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:52.552 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T15:00:52.564 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:00:52.561+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:00:52.659 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:52.659 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T15:00:52.659 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T15:00:52.659 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T15:00:52.660 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.660 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4281.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-21T15:00:52.660 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:09.324416+0000", 2024-09-21T15:00:52.660 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 163.33431279000001, 2024-09-21T15:00:52.660 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.117837947, 2024-09-21T15:00:52.660 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:52.660 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:52.660 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:52.660 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4281", 2024-09-21T15:00:52.660 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/1518520303", 2024-09-21T15:00:52.660 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1 2024-09-21T15:00:52.660 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.660 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:52.661 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.661 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:52.661 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324416+0000", 2024-09-21T15:00:52.661 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:52.661 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.661 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.661 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:52.661 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324416+0000", 2024-09-21T15:00:52.661 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:52.661 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.661 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.661 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:52.661 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324414+0000", 2024-09-21T15:00:52.661 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T15:00:52.662 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.662 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.662 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:52.662 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324429+0000", 2024-09-21T15:00:52.662 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5637e-05 2024-09-21T15:00:52.662 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.662 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.662 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:52.662 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324448+0000", 2024-09-21T15:00:52.662 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9042000000000002e-05 2024-09-21T15:00:52.662 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.662 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.662 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:52.663 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324465+0000", 2024-09-21T15:00:52.663 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6767999999999999e-05 2024-09-21T15:00:52.663 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.663 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.663 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:52.663 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324571+0000", 2024-09-21T15:00:52.663 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000105714 2024-09-21T15:00:52.663 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.663 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.663 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op must wait for map", 2024-09-21T15:00:52.663 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.324587+0000", 2024-09-21T15:00:52.663 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5753999999999999e-05 2024-09-21T15:00:52.663 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.664 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.664 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:52.664 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.325864+0000", 2024-09-21T15:00:52.664 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0012773350000000001 2024-09-21T15:00:52.664 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.664 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.664 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:00:52.664 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.325878+0000", 2024-09-21T15:00:52.664 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4498000000000001e-05 2024-09-21T15:00:52.664 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.664 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.664 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:52.664 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.439461+0000", 2024-09-21T15:00:52.664 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.11358212500000001 2024-09-21T15:00:52.665 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.665 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.665 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:52.665 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.439848+0000", 2024-09-21T15:00:52.665 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00038747800000000002 2024-09-21T15:00:52.665 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.665 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.665 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 2", 2024-09-21T15:00:52.665 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.440175+0000", 2024-09-21T15:00:52.665 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00032721900000000001 2024-09-21T15:00:52.665 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.665 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.665 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:52.665 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.441770+0000", 2024-09-21T15:00:52.666 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0015949409999999999 2024-09-21T15:00:52.666 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.666 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.666 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:52.666 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.442086+0000", 2024-09-21T15:00:52.666 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00031586299999999999 2024-09-21T15:00:52.666 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.666 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.666 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:52.666 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.442195+0000", 2024-09-21T15:00:52.666 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000108791 2024-09-21T15:00:52.666 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.666 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.667 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:52.667 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.442254+0000", 2024-09-21T15:00:52.667 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9038e-05 2024-09-21T15:00:52.667 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:52.667 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:52.667 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:52.667 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.667 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.667 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4281.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-21T15:00:52.667 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:09.447828+0000", 2024-09-21T15:00:52.667 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 163.21090078899999, 2024-09-21T15:00:52.667 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.32904393700000001, 2024-09-21T15:00:52.667 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:52.668 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:52.668 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:52.668 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4281", 2024-09-21T15:00:52.668 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/1518520303", 2024-09-21T15:00:52.668 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 2 2024-09-21T15:00:52.668 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.668 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:52.668 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.668 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:52.668 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447828+0000", 2024-09-21T15:00:52.668 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:52.668 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.668 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.668 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:52.668 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447828+0000", 2024-09-21T15:00:52.669 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:52.669 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.669 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.669 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:52.669 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447825+0000", 2024-09-21T15:00:52.669 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T15:00:52.669 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.669 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.669 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:52.669 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447841+0000", 2024-09-21T15:00:52.669 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5359999999999999e-05 2024-09-21T15:00:52.669 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.669 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.670 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:52.670 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447875+0000", 2024-09-21T15:00:52.670 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4388999999999999e-05 2024-09-21T15:00:52.670 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.670 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.670 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:52.670 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447903+0000", 2024-09-21T15:00:52.670 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7373e-05 2024-09-21T15:00:52.670 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.670 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.670 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:52.670 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.447996+0000", 2024-09-21T15:00:52.670 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2954999999999993e-05 2024-09-21T15:00:52.670 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.671 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.671 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:00:52.671 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.448011+0000", 2024-09-21T15:00:52.671 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5038999999999999e-05 2024-09-21T15:00:52.671 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.671 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.671 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:52.671 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.776501+0000", 2024-09-21T15:00:52.671 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.32849012500000002 2024-09-21T15:00:52.671 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.671 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.671 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:52.671 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:09.776872+0000", 2024-09-21T15:00:52.672 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00037119600000000002 2024-09-21T15:00:52.672 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:52.672 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:52.672 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:52.672 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.672 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.672 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:97673 3.1d 3:bd91946e:::rbd_data.1136bfc79a70.0000000000000002:head [write 0~4718592 [fadvise_sequential+fadvise_nocache],write 5242880~32768 [fadvise_sequential+fadvise_nocache],write 5341184~147456 [fadvise_sequential+fadvise_nocache],write 5554176~147456 [fadvise_sequential+fadvise_nocache],write 5767168~16384 [fadvise_sequential+fadvise_nocache],write 5849088~147456 [fadvise_sequential+fadvise_nocache],write 6062080~147456 [fadvise_sequential+fadvise_nocache],write 6275072~16384 [fadvise_sequential+fadvise_nocache],write 6356992~147456 [fadvise_sequential+fadvise_nocache],write 6569984~147456 [fadvise_sequential+fadvise_nocache],write 6782976~32768 [fadvise_sequential+fadvise_nocache],write 6864896~147456 [fadvise_sequential+fadvise_nocache],write 7077888~147456 [fadvise_sequential+fadvise_nocache],write 7290880~49152 [fadvise_sequential+fadvise_nocache],write 7372800~147456 [fadvise_sequential+fadvise_nocache],write 7585792~147456 [fadvise_sequential+fadvise_nocache],write 7798784~65536 [fadvise_sequential+fadvise_nocache],write 7880704~147456 [fadvise_sequential+fadvise_nocache],write 8093696~147456 [fadvise_sequential+fadvise_nocache],write 8306688~81920 [fadvise_sequential+fadvise_nocache],write 8421376~114688 [fadvise_sequential+fadvise_nocache],write 8634368~114688 [fadvise_sequential+fadvise_nocache],write 8847360~65536 [fadvise_sequential+fadvise_nocache],write 9011200~32768 [fadvise_sequential+fadvise_nocache],write 9224192~32768 [fadvise_sequential+fadvise_nocache],write 9519104~32768 [fadvise_sequential+fadvise_nocache],write 9732096~32768 [fadvise_sequential+fadvise_nocache],write 9945088~16384 [fadvise_sequential+fadvise_nocache],write 9977856~196608 [fadvise_sequential+fadvise_nocache],write 10190848~196608 [fadvise_sequential+fadvise_nocache],write 10403840~278528 [fadvise_sequential+fadvise_nocache],write 10698752~196608 [fadvise_sequential+fadvise_nocache],write 10911744~278528 [fadvise_sequential+fadvise_nocache],write 11206656~196608 [fadvise_sequential+fadvise_nocache],write 11419648~196608 [fadvise_sequential+fadvise_nocache],write 11632640~65536 [fadvise_sequential+fadvise_nocache],write 11730944~98304 [fadvise_sequential+fadvise_nocache],write 11845632~65536 [fadvise_sequential+fadvise_nocache],write 11943936~98304 [fadvise_sequential+fadvise_nocache],write 12058624~65536 [fadvise_sequential+fadvise_nocache],write 12140544~65536 [fadvise_sequential+fadvise_nocache],write 12238848~98304 [fadvise_sequential+fadvise_nocache],write 12353536~65536 [fadvise_sequential+fadvise_nocache],write 12451840~98304 [fadvise_sequential+fadvise_nocache],write 12566528~65536 [fadvise_sequential+fadvise_nocache],write 12746752~98304 [fadvise_sequential+fadvise_nocache],write 12959744~98304 [fadvise_sequential+fadvise_nocache],write 13107200~32768 [fadvise_sequential+fadvise_nocache],write 13254656~98304 [fadvise_sequential+fadvise_nocache],write 13467648~98304 [fadvise_sequential+fadvise_nocache],write 13631488~16384 [fadvise_sequential+fadvise_nocache],write 13762560~98304 [fadvise_sequential+fadvise_nocache],write 13975552~98304 [fadvise_sequential+fadvise_nocache],write 15204352~49152 [fadvise_sequential+fadvise_nocache],write 15384576~81920 [fadvise_sequential+fadvise_nocache],write 15597568~81920 [fadvise_sequential+fadvise_nocache],write 15728640~32768 [fadvise_sequential+fadvise_nocache],write 15892480~81920 [fadvise_sequential+fadvise_nocache],write 16105472~81920 [fadvise_sequential+fadvise_nocache],write 16252928~16384 [fadvise_sequential+fadvise_nocache],write 16400384~81920 [fadvise_sequential+fadvise_nocache],write 16613376~81920 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:52.672 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.300719+0000", 2024-09-21T15:00:52.672 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 47.358010016999998, 2024-09-21T15:00:52.672 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.068947114000000004, 2024-09-21T15:00:52.672 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:52.672 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:52.672 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:52.672 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:52.673 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:52.673 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 97673 2024-09-21T15:00:52.673 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.673 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:52.673 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.673 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:52.673 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.300719+0000", 2024-09-21T15:00:52.673 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:52.673 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.673 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.673 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:52.673 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.300719+0000", 2024-09-21T15:00:52.673 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:52.673 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.674 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.674 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:52.674 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.300716+0000", 2024-09-21T15:00:52.674 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-21T15:00:52.674 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.674 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.674 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:52.674 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.308380+0000", 2024-09-21T15:00:52.674 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0076638519999999996 2024-09-21T15:00:52.674 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.674 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.674 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:52.674 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.308418+0000", 2024-09-21T15:00:52.675 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.8380000000000002e-05 2024-09-21T15:00:52.675 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.675 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.675 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:52.675 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.308431+0000", 2024-09-21T15:00:52.675 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3213999999999999e-05 2024-09-21T15:00:52.675 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.675 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.675 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:52.675 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.308562+0000", 2024-09-21T15:00:52.675 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013015999999999999 2024-09-21T15:00:52.675 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.675 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.675 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:52.676 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.309173+0000", 2024-09-21T15:00:52.676 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00061185500000000002 2024-09-21T15:00:52.676 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.676 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.676 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 0", 2024-09-21T15:00:52.676 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311612+0000", 2024-09-21T15:00:52.676 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0024383949999999999 2024-09-21T15:00:52.676 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.676 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.676 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:52.676 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.355482+0000", 2024-09-21T15:00:52.676 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.043869940000000003 2024-09-21T15:00:52.676 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.676 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.677 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:52.677 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.369306+0000", 2024-09-21T15:00:52.677 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013824279 2024-09-21T15:00:52.677 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.677 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.677 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:52.677 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.369575+0000", 2024-09-21T15:00:52.677 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000269404 2024-09-21T15:00:52.677 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.677 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.677 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:52.677 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.369666+0000", 2024-09-21T15:00:52.677 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0426000000000005e-05 2024-09-21T15:00:52.677 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:52.677 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:52.677 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:52.678 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.678 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.678 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:112719 3.13 3:cd56ff5a:::rbd_data.1136b1194d20.0000000000000002:58 [read 1998848~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:52.678 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.387056+0000", 2024-09-21T15:00:52.678 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 23.271672671000001, 2024-09-21T15:00:52.678 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055608623000000003, 2024-09-21T15:00:52.678 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:52.678 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:52.678 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:52.678 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:52.678 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:52.678 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 112719 2024-09-21T15:00:52.678 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.678 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:52.679 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.679 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:52.679 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.387056+0000", 2024-09-21T15:00:52.679 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:52.679 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.679 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.679 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:52.679 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.387056+0000", 2024-09-21T15:00:52.679 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:52.679 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.679 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.679 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:52.679 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.387056+0000", 2024-09-21T15:00:52.680 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:52.680 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.680 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.680 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:52.680 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.387058+0000", 2024-09-21T15:00:52.680 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.683e-06 2024-09-21T15:00:52.680 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.680 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.680 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:52.680 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.387064+0000", 2024-09-21T15:00:52.680 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6720000000000001e-06 2024-09-21T15:00:52.680 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.680 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.680 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:52.680 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.387069+0000", 2024-09-21T15:00:52.681 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8999999999999997e-06 2024-09-21T15:00:52.681 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.681 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.681 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:52.681 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.442514+0000", 2024-09-21T15:00:52.681 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055444881000000001 2024-09-21T15:00:52.681 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.681 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.681 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:52.681 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.442565+0000", 2024-09-21T15:00:52.681 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1010000000000001e-05 2024-09-21T15:00:52.681 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.681 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.681 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:52.682 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.442665+0000", 2024-09-21T15:00:52.682 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9840000000000006e-05 2024-09-21T15:00:52.682 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:52.682 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:52.682 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:52.682 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.682 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.682 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:112720 3.13 3:cd56ff5a:::rbd_data.1136b1194d20.0000000000000002:58 [read 2088960~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:52.682 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.387074+0000", 2024-09-21T15:00:52.682 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 23.271655156000001, 2024-09-21T15:00:52.682 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055824591999999999, 2024-09-21T15:00:52.682 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:52.682 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:52.683 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:52.683 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:52.683 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:52.683 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 112720 2024-09-21T15:00:52.683 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.683 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:52.683 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.683 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:52.683 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.387074+0000", 2024-09-21T15:00:52.683 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:52.683 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.683 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.683 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:52.684 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.387074+0000", 2024-09-21T15:00:52.684 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:52.684 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.684 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.684 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:52.684 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.387073+0000", 2024-09-21T15:00:52.684 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:52.684 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.684 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.684 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:52.684 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.387076+0000", 2024-09-21T15:00:52.684 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4710000000000001e-06 2024-09-21T15:00:52.684 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.684 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.685 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:52.685 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.387085+0000", 2024-09-21T15:00:52.685 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2110000000000004e-06 2024-09-21T15:00:52.685 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.685 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.685 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:52.685 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.387090+0000", 2024-09-21T15:00:52.685 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.904e-06 2024-09-21T15:00:52.685 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.685 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.685 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:52.685 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.442702+0000", 2024-09-21T15:00:52.685 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055612251000000001 2024-09-21T15:00:52.685 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.685 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.686 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:52.686 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.442780+0000", 2024-09-21T15:00:52.686 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8232999999999994e-05 2024-09-21T15:00:52.686 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.686 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.686 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:52.686 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.442898+0000", 2024-09-21T15:00:52.686 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000117889 2024-09-21T15:00:52.686 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:52.686 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:52.686 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:52.686 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.686 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.686 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:112721 3.13 3:cd56ff5a:::rbd_data.1136b1194d20.0000000000000002:58 [read 2105344~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:52.687 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.387095+0000", 2024-09-21T15:00:52.687 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 23.271634066000001, 2024-09-21T15:00:52.687 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.056000182000000003, 2024-09-21T15:00:52.687 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:52.687 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:52.687 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:52.687 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:52.687 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:52.687 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 112721 2024-09-21T15:00:52.687 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.687 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:52.687 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.687 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:52.687 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.387095+0000", 2024-09-21T15:00:52.688 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:52.688 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.688 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.688 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:52.688 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.387095+0000", 2024-09-21T15:00:52.688 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:52.688 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.688 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.688 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:52.688 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.387094+0000", 2024-09-21T15:00:52.688 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:52.688 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.688 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.688 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:52.688 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.387097+0000", 2024-09-21T15:00:52.689 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5940000000000001e-06 2024-09-21T15:00:52.689 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.689 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.689 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:52.689 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.387106+0000", 2024-09-21T15:00:52.689 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.9439999999999994e-06 2024-09-21T15:00:52.689 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.689 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.689 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:52.689 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.387111+0000", 2024-09-21T15:00:52.689 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3900000000000001e-06 2024-09-21T15:00:52.689 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.689 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.689 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:52.690 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.442934+0000", 2024-09-21T15:00:52.690 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055822996 2024-09-21T15:00:52.690 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.690 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.690 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:52.690 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.442998+0000", 2024-09-21T15:00:52.690 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3746999999999998e-05 2024-09-21T15:00:52.690 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.690 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.690 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:52.690 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.443095+0000", 2024-09-21T15:00:52.690 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6848999999999994e-05 2024-09-21T15:00:52.690 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:52.690 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:52.691 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:52.691 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.691 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.691 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:112733 3.13 3:cd56ff5a:::rbd_data.1136b1194d20.0000000000000002:58 [read 3186688~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:52.691 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.390744+0000", 2024-09-21T15:00:52.691 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 23.267984888000001, 2024-09-21T15:00:52.691 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055498697999999999, 2024-09-21T15:00:52.691 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:52.691 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:52.691 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:52.691 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:52.691 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:52.691 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 112733 2024-09-21T15:00:52.691 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.692 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:52.692 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.692 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:52.692 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390744+0000", 2024-09-21T15:00:52.692 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:52.692 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.692 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.692 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:52.692 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390744+0000", 2024-09-21T15:00:52.692 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:52.692 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.692 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.692 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:52.692 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390743+0000", 2024-09-21T15:00:52.692 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:52.693 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.693 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.693 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:52.693 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390746+0000", 2024-09-21T15:00:52.693 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9510000000000002e-06 2024-09-21T15:00:52.693 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.693 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.693 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:52.693 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390751+0000", 2024-09-21T15:00:52.693 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.357e-06 2024-09-21T15:00:52.693 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.693 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.693 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:52.693 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390753+0000", 2024-09-21T15:00:52.694 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7310000000000002e-06 2024-09-21T15:00:52.694 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.694 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.694 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:52.694 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.445994+0000", 2024-09-21T15:00:52.694 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055240247999999999 2024-09-21T15:00:52.694 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.694 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.694 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:52.694 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.446111+0000", 2024-09-21T15:00:52.694 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00011699799999999999 2024-09-21T15:00:52.694 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.694 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.694 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:52.695 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.446243+0000", 2024-09-21T15:00:52.695 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013196099999999999 2024-09-21T15:00:52.695 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:52.695 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:52.695 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:52.695 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.695 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.695 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:112734 3.13 3:cd56ff5a:::rbd_data.1136b1194d20.0000000000000002:58 [read 3276800~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:52.695 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.390755+0000", 2024-09-21T15:00:52.695 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 23.267973264999998, 2024-09-21T15:00:52.695 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.05569313, 2024-09-21T15:00:52.695 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:52.695 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:52.695 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:52.696 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:52.696 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:52.696 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 112734 2024-09-21T15:00:52.696 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.696 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:52.696 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.696 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:52.696 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390755+0000", 2024-09-21T15:00:52.696 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:52.696 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.696 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.696 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:52.696 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390755+0000", 2024-09-21T15:00:52.696 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:52.697 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.697 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.697 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:52.697 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390755+0000", 2024-09-21T15:00:52.697 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:00:52.697 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.697 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.697 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:52.697 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390757+0000", 2024-09-21T15:00:52.697 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5060000000000001e-06 2024-09-21T15:00:52.697 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.697 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.697 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:52.697 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390760+0000", 2024-09-21T15:00:52.698 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.79e-06 2024-09-21T15:00:52.698 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.698 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.698 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:52.698 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390762+0000", 2024-09-21T15:00:52.698 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.661e-06 2024-09-21T15:00:52.698 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.698 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.698 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:52.698 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.446280+0000", 2024-09-21T15:00:52.698 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055517761999999998 2024-09-21T15:00:52.698 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.698 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.698 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:52.699 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.446365+0000", 2024-09-21T15:00:52.699 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5208000000000001e-05 2024-09-21T15:00:52.699 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.699 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.699 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:52.699 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.446449+0000", 2024-09-21T15:00:52.699 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3417000000000004e-05 2024-09-21T15:00:52.699 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:52.699 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:52.699 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:52.699 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.699 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.699 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:112735 3.13 3:cd56ff5a:::rbd_data.1136b1194d20.0000000000000002:58 [read 3366912~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:52.699 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.390764+0000", 2024-09-21T15:00:52.700 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 23.267964455000001, 2024-09-21T15:00:52.700 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055846502999999999, 2024-09-21T15:00:52.700 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:52.700 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:52.700 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:52.700 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:52.700 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:52.700 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 112735 2024-09-21T15:00:52.700 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.700 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:52.700 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.700 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:52.700 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390764+0000", 2024-09-21T15:00:52.700 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:52.701 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.701 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.701 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:52.701 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390764+0000", 2024-09-21T15:00:52.701 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:52.701 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.701 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.701 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:52.701 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390764+0000", 2024-09-21T15:00:52.701 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:00:52.701 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.701 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.701 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:52.701 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390771+0000", 2024-09-21T15:00:52.701 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.1380000000000001e-06 2024-09-21T15:00:52.702 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.702 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.702 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:52.702 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390774+0000", 2024-09-21T15:00:52.702 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1370000000000002e-06 2024-09-21T15:00:52.702 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.702 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.702 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:52.702 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390777+0000", 2024-09-21T15:00:52.702 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7470000000000001e-06 2024-09-21T15:00:52.702 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.702 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.702 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:52.702 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.446488+0000", 2024-09-21T15:00:52.703 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055711016000000002 2024-09-21T15:00:52.703 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.703 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.703 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:52.703 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.446543+0000", 2024-09-21T15:00:52.703 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5148999999999997e-05 2024-09-21T15:00:52.703 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.703 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.703 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:52.703 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.446611+0000", 2024-09-21T15:00:52.703 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7533999999999994e-05 2024-09-21T15:00:52.703 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:52.703 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:52.703 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:52.704 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.704 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.704 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:112736 3.13 3:cd56ff5a:::rbd_data.1136b1194d20.0000000000000002:58 [read 3457024~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:52.704 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.390779+0000", 2024-09-21T15:00:52.704 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 23.267949511000001, 2024-09-21T15:00:52.704 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.056179989, 2024-09-21T15:00:52.704 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:52.704 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:52.704 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:52.704 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:52.704 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:52.704 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 112736 2024-09-21T15:00:52.704 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.704 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:52.705 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.705 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:52.705 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390779+0000", 2024-09-21T15:00:52.705 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:52.705 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.705 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.705 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:52.705 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390779+0000", 2024-09-21T15:00:52.705 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:52.705 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.705 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.705 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:52.705 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390779+0000", 2024-09-21T15:00:52.705 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:00:52.705 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.706 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.706 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:52.706 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390780+0000", 2024-09-21T15:00:52.706 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4360000000000001e-06 2024-09-21T15:00:52.706 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.706 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.706 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:52.706 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390783+0000", 2024-09-21T15:00:52.706 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.818e-06 2024-09-21T15:00:52.706 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.706 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.706 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:52.706 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.390786+0000", 2024-09-21T15:00:52.706 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4439999999999998e-06 2024-09-21T15:00:52.707 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.707 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.707 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:52.707 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.446668+0000", 2024-09-21T15:00:52.707 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055881910999999999 2024-09-21T15:00:52.707 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.707 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.707 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:52.707 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.446797+0000", 2024-09-21T15:00:52.707 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012924599999999999 2024-09-21T15:00:52.707 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.707 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.707 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:52.707 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.446959+0000", 2024-09-21T15:00:52.707 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000162356 2024-09-21T15:00:52.708 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:52.708 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:52.708 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:52.708 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.708 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.708 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:118215 3.d 3:b5b5db2f:::rbd_data.1136b1194d20.0000000000000005:58 [read 5160960~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:52.708 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.833406+0000", 2024-09-21T15:00:52.708 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 22.825323213000001, 2024-09-21T15:00:52.708 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055458362999999997, 2024-09-21T15:00:52.708 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:52.708 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:52.708 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:52.708 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:52.708 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:52.709 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 118215 2024-09-21T15:00:52.709 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.709 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:52.709 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.709 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:52.709 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833406+0000", 2024-09-21T15:00:52.709 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:52.709 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.709 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.709 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:52.709 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833406+0000", 2024-09-21T15:00:52.709 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:52.709 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.709 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.710 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:52.710 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833405+0000", 2024-09-21T15:00:52.710 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:52.710 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.710 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.710 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:52.710 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833409+0000", 2024-09-21T15:00:52.710 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.8099999999999999e-06 2024-09-21T15:00:52.710 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.710 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.710 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:52.710 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833416+0000", 2024-09-21T15:00:52.710 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3989999999999998e-06 2024-09-21T15:00:52.710 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.711 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.711 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:52.711 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833420+0000", 2024-09-21T15:00:52.711 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7189999999999999e-06 2024-09-21T15:00:52.711 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.711 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.711 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:52.711 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.888444+0000", 2024-09-21T15:00:52.711 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055024311999999999 2024-09-21T15:00:52.711 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.711 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.711 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:52.711 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.888630+0000", 2024-09-21T15:00:52.711 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00018579800000000001 2024-09-21T15:00:52.711 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.712 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.712 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:52.712 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.888864+0000", 2024-09-21T15:00:52.712 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000234009 2024-09-21T15:00:52.712 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:52.712 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:52.712 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:52.712 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.712 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.712 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:118216 3.d 3:b5b5db2f:::rbd_data.1136b1194d20.0000000000000005:58 [read 5251072~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:52.712 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.833425+0000", 2024-09-21T15:00:52.712 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 22.825303780999999, 2024-09-21T15:00:52.712 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055891158000000003, 2024-09-21T15:00:52.712 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:52.713 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:52.713 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:52.713 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:52.713 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:52.713 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 118216 2024-09-21T15:00:52.713 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.713 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:52.713 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.713 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:52.713 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833425+0000", 2024-09-21T15:00:52.713 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:52.713 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.713 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.713 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:52.713 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833425+0000", 2024-09-21T15:00:52.714 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:52.714 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.714 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.714 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:52.714 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833425+0000", 2024-09-21T15:00:52.714 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:52.714 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.714 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.714 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:52.714 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833427+0000", 2024-09-21T15:00:52.714 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.137e-06 2024-09-21T15:00:52.714 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.714 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.714 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:52.715 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833430+0000", 2024-09-21T15:00:52.715 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5379999999999998e-06 2024-09-21T15:00:52.715 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.715 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.715 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:52.715 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833433+0000", 2024-09-21T15:00:52.715 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0599999999999999e-06 2024-09-21T15:00:52.715 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.715 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.715 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:52.715 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.888919+0000", 2024-09-21T15:00:52.715 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055485381 2024-09-21T15:00:52.715 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.715 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.715 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:52.716 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.889088+0000", 2024-09-21T15:00:52.716 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000169364 2024-09-21T15:00:52.716 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.716 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.716 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:52.716 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.889316+0000", 2024-09-21T15:00:52.716 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00022798600000000001 2024-09-21T15:00:52.716 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:52.716 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:52.716 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:52.716 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.716 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.716 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:118217 3.d 3:b5b5db2f:::rbd_data.1136b1194d20.0000000000000005:58 [read 5341184~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:52.716 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.833436+0000", 2024-09-21T15:00:52.717 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 22.825292353999998, 2024-09-21T15:00:52.717 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.056045230000000001, 2024-09-21T15:00:52.717 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:52.717 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:52.717 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:52.717 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:52.717 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:52.717 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 118217 2024-09-21T15:00:52.717 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.717 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:52.717 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.717 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:52.717 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833436+0000", 2024-09-21T15:00:52.717 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:52.717 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.718 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.718 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:52.718 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833436+0000", 2024-09-21T15:00:52.718 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:52.718 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.718 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.718 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:52.718 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833436+0000", 2024-09-21T15:00:52.718 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:52.718 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.718 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.718 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:52.718 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833438+0000", 2024-09-21T15:00:52.718 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.826e-06 2024-09-21T15:00:52.718 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.719 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.719 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:52.719 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833441+0000", 2024-09-21T15:00:52.719 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3280000000000002e-06 2024-09-21T15:00:52.719 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.719 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.719 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:52.719 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833447+0000", 2024-09-21T15:00:52.719 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2660000000000001e-06 2024-09-21T15:00:52.719 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.719 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.719 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:52.719 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.889362+0000", 2024-09-21T15:00:52.719 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055915323000000003 2024-09-21T15:00:52.719 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.720 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.720 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:52.720 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.889414+0000", 2024-09-21T15:00:52.720 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1711e-05 2024-09-21T15:00:52.720 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.720 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.720 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:52.720 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.889482+0000", 2024-09-21T15:00:52.720 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8056e-05 2024-09-21T15:00:52.720 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:52.720 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:52.720 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:52.720 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.720 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.721 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:118218 3.d 3:b5b5db2f:::rbd_data.1136b1194d20.0000000000000005:58 [read 5431296~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:52.721 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.833451+0000", 2024-09-21T15:00:52.721 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 22.825277319000001, 2024-09-21T15:00:52.721 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.056286765000000002, 2024-09-21T15:00:52.721 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:52.721 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:52.721 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:52.721 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:52.721 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:52.721 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 118218 2024-09-21T15:00:52.721 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.721 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:52.721 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.721 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:52.722 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833451+0000", 2024-09-21T15:00:52.722 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:52.722 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.722 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.722 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:52.722 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833451+0000", 2024-09-21T15:00:52.722 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:52.722 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.722 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.722 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:52.722 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833451+0000", 2024-09-21T15:00:52.722 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:52.722 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.722 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.723 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:52.723 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833454+0000", 2024-09-21T15:00:52.723 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9670000000000001e-06 2024-09-21T15:00:52.723 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.723 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.723 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:52.723 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833461+0000", 2024-09-21T15:00:52.723 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0489999999999998e-06 2024-09-21T15:00:52.723 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.723 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.723 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:52.723 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.833468+0000", 2024-09-21T15:00:52.723 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0310000000000001e-06 2024-09-21T15:00:52.723 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.724 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.724 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:52.724 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.889539+0000", 2024-09-21T15:00:52.724 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.056071217 2024-09-21T15:00:52.724 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.724 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.724 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:52.724 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.889621+0000", 2024-09-21T15:00:52.724 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1528999999999998e-05 2024-09-21T15:00:52.724 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.724 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.724 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:52.724 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.889738+0000", 2024-09-21T15:00:52.724 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00011759799999999999 2024-09-21T15:00:52.725 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:52.725 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:52.725 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:52.725 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.725 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.725 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:118220 3.d 3:b5b5db2f:::rbd_data.1136b1194d20.0000000000000005:58 [read 5611520~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:52.725 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.834528+0000", 2024-09-21T15:00:52.725 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 22.82420033, 2024-09-21T15:00:52.725 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055660376999999997, 2024-09-21T15:00:52.725 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:52.725 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:52.725 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:52.725 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:52.725 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:52.725 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 118220 2024-09-21T15:00:52.726 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.726 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:52.726 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.726 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:52.726 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834528+0000", 2024-09-21T15:00:52.726 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:52.726 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.726 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.726 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:52.726 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834528+0000", 2024-09-21T15:00:52.726 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:52.726 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.726 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.727 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:52.727 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834528+0000", 2024-09-21T15:00:52.727 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:52.727 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.727 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.727 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:52.727 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834532+0000", 2024-09-21T15:00:52.727 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.9509999999999999e-06 2024-09-21T15:00:52.727 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.727 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.727 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:52.727 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834537+0000", 2024-09-21T15:00:52.727 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6899999999999997e-06 2024-09-21T15:00:52.727 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.727 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.728 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:52.728 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834541+0000", 2024-09-21T15:00:52.728 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.76e-06 2024-09-21T15:00:52.728 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.728 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.728 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:52.728 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.889962+0000", 2024-09-21T15:00:52.728 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055421249999999998 2024-09-21T15:00:52.728 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.728 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.728 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:52.728 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.890074+0000", 2024-09-21T15:00:52.728 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00011124300000000001 2024-09-21T15:00:52.728 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.729 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.729 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:52.729 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.890189+0000", 2024-09-21T15:00:52.729 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000115151 2024-09-21T15:00:52.729 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:52.729 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:52.729 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:52.729 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.729 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.729 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:118221 3.d 3:b5b5db2f:::rbd_data.1136b1194d20.0000000000000005:58 [read 5701632~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:52.729 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.834544+0000", 2024-09-21T15:00:52.729 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 22.824184408000001, 2024-09-21T15:00:52.729 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.056071875, 2024-09-21T15:00:52.729 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:52.730 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:52.730 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:52.730 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:52.730 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:52.730 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 118221 2024-09-21T15:00:52.730 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.730 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:52.730 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.730 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:52.730 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834544+0000", 2024-09-21T15:00:52.730 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:52.730 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.730 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.730 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:52.730 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834544+0000", 2024-09-21T15:00:52.731 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:52.731 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.731 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.731 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:52.731 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834544+0000", 2024-09-21T15:00:52.731 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:52.731 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.731 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.731 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:52.731 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834546+0000", 2024-09-21T15:00:52.731 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.458e-06 2024-09-21T15:00:52.731 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.731 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.731 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:52.732 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834550+0000", 2024-09-21T15:00:52.732 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.563e-06 2024-09-21T15:00:52.732 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.732 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.732 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:52.732 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834553+0000", 2024-09-21T15:00:52.732 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.253e-06 2024-09-21T15:00:52.732 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.732 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.732 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:52.732 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.890246+0000", 2024-09-21T15:00:52.732 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055692561000000002 2024-09-21T15:00:52.732 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.732 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.732 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:52.733 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.890397+0000", 2024-09-21T15:00:52.733 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015129999999999999 2024-09-21T15:00:52.733 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.733 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.733 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:52.733 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.890616+0000", 2024-09-21T15:00:52.733 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00021912799999999999 2024-09-21T15:00:52.733 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:52.733 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:52.733 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:52.733 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.733 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.733 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:118222 3.d 3:b5b5db2f:::rbd_data.1136b1194d20.0000000000000005:58 [read 5791744~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:52.733 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.834557+0000", 2024-09-21T15:00:52.734 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 22.824171596999999, 2024-09-21T15:00:52.734 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.056417907000000003, 2024-09-21T15:00:52.734 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:52.734 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:52.734 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:52.734 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:52.734 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:52.734 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 118222 2024-09-21T15:00:52.734 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.734 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:52.734 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.734 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:52.734 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834557+0000", 2024-09-21T15:00:52.735 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:52.735 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.735 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.735 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:52.735 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834557+0000", 2024-09-21T15:00:52.735 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:52.735 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.735 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.735 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:52.735 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834557+0000", 2024-09-21T15:00:52.735 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:52.735 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.735 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.735 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:52.736 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834559+0000", 2024-09-21T15:00:52.736 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.956e-06 2024-09-21T15:00:52.736 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.736 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.736 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:52.736 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834562+0000", 2024-09-21T15:00:52.736 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.518e-06 2024-09-21T15:00:52.736 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.736 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.736 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:52.736 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834566+0000", 2024-09-21T15:00:52.736 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.529e-06 2024-09-21T15:00:52.736 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.736 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.736 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:52.737 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.890671+0000", 2024-09-21T15:00:52.737 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.056105255 2024-09-21T15:00:52.737 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.737 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.737 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:52.737 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.890788+0000", 2024-09-21T15:00:52.737 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00011715899999999999 2024-09-21T15:00:52.737 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.737 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.737 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:52.737 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.890975+0000", 2024-09-21T15:00:52.737 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00018678100000000001 2024-09-21T15:00:52.737 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:52.737 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:52.738 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:52.738 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.738 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.738 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:118223 3.d 3:b5b5db2f:::rbd_data.1136b1194d20.0000000000000005:58 [read 5881856~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:52.738 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.834569+0000", 2024-09-21T15:00:52.738 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 22.824160039999999, 2024-09-21T15:00:52.738 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.056618784999999998, 2024-09-21T15:00:52.738 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:52.738 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:52.738 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:52.738 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:52.738 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:52.738 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 118223 2024-09-21T15:00:52.738 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.739 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:52.739 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.739 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:52.739 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834569+0000", 2024-09-21T15:00:52.739 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:52.739 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.739 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.739 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:52.739 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834569+0000", 2024-09-21T15:00:52.739 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:52.739 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.739 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.739 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:52.739 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834568+0000", 2024-09-21T15:00:52.740 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:52.740 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.740 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.740 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:52.740 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834570+0000", 2024-09-21T15:00:52.740 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8309999999999999e-06 2024-09-21T15:00:52.740 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.740 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.740 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:52.740 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834574+0000", 2024-09-21T15:00:52.740 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3720000000000001e-06 2024-09-21T15:00:52.740 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.740 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.740 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:52.740 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834577+0000", 2024-09-21T15:00:52.741 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3019999999999999e-06 2024-09-21T15:00:52.741 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.741 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.741 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:52.741 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.891014+0000", 2024-09-21T15:00:52.741 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.056437322999999998 2024-09-21T15:00:52.741 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.741 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.741 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:52.741 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.891101+0000", 2024-09-21T15:00:52.741 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6341999999999994e-05 2024-09-21T15:00:52.741 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.741 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.741 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:52.742 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.891187+0000", 2024-09-21T15:00:52.742 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6889000000000001e-05 2024-09-21T15:00:52.742 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:52.742 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:52.742 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:52.742 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.742 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.742 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:118224 3.d 3:b5b5db2f:::rbd_data.1136b1194d20.0000000000000005:58 [read 5971968~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:52.742 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.834580+0000", 2024-09-21T15:00:52.742 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 22.824149073000001, 2024-09-21T15:00:52.742 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.056770289000000002, 2024-09-21T15:00:52.742 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:52.742 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:52.743 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:52.743 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:52.743 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:52.743 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 118224 2024-09-21T15:00:52.743 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.743 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:52.743 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.743 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:52.743 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834580+0000", 2024-09-21T15:00:52.743 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:52.743 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.744 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.744 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:52.744 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834580+0000", 2024-09-21T15:00:52.744 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:52.744 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.744 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.744 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:52.744 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834579+0000", 2024-09-21T15:00:52.744 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:52.744 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.744 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.744 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:52.744 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834581+0000", 2024-09-21T15:00:52.744 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8160000000000001e-06 2024-09-21T15:00:52.745 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.745 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.745 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:52.745 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834585+0000", 2024-09-21T15:00:52.745 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.568e-06 2024-09-21T15:00:52.745 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.745 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.745 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:52.745 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834588+0000", 2024-09-21T15:00:52.745 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1810000000000001e-06 2024-09-21T15:00:52.745 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.745 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.745 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:52.745 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.891216+0000", 2024-09-21T15:00:52.746 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.056628127 2024-09-21T15:00:52.746 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.746 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.746 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:52.746 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.891273+0000", 2024-09-21T15:00:52.746 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7364000000000002e-05 2024-09-21T15:00:52.746 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.746 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.746 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:52.746 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.891350+0000", 2024-09-21T15:00:52.746 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6509999999999998e-05 2024-09-21T15:00:52.746 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:52.746 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:52.746 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:52.747 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.747 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.747 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:118225 3.d 3:b5b5db2f:::rbd_data.1136b1194d20.0000000000000005:58 [read 6062080~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:52.747 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.834591+0000", 2024-09-21T15:00:52.747 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 22.824138061999999, 2024-09-21T15:00:52.747 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.056883207999999998, 2024-09-21T15:00:52.747 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:52.747 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:52.747 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:52.747 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:52.747 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:52.747 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 118225 2024-09-21T15:00:52.747 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.747 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:52.747 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.748 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:52.748 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834591+0000", 2024-09-21T15:00:52.748 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:52.748 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.748 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.748 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:52.748 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834591+0000", 2024-09-21T15:00:52.748 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:52.748 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.748 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.748 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:52.748 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834590+0000", 2024-09-21T15:00:52.748 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:52.748 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.748 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.749 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:52.749 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834592+0000", 2024-09-21T15:00:52.749 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8810000000000001e-06 2024-09-21T15:00:52.749 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.749 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.749 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:52.749 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834596+0000", 2024-09-21T15:00:52.749 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.439e-06 2024-09-21T15:00:52.749 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.749 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.749 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:52.749 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.834599+0000", 2024-09-21T15:00:52.749 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0740000000000001e-06 2024-09-21T15:00:52.749 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.750 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.750 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:52.750 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.891369+0000", 2024-09-21T15:00:52.750 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.056769872999999998 2024-09-21T15:00:52.750 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.750 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.750 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:52.750 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.891415+0000", 2024-09-21T15:00:52.750 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6257000000000001e-05 2024-09-21T15:00:52.750 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:52.750 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:52.750 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:52.750 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.891474+0000", 2024-09-21T15:00:52.750 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8964e-05 2024-09-21T15:00:52.750 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:52.751 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:52.751 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:52.751 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:52.751 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:52.751 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:52.752 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T15:00:52.753 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:00:52.747+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:00:52.853 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:00:52.849+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:00:52.857 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 5, 6, 7, 4] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T15:00:52.857 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T15:00:52.857 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T15:00:52.871 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:52.871 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:00:52.871 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T15:00:52.872 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:52.882 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T15:00:52.888 INFO:teuthology.orchestra.run.smithi032.stderr:noscrub is set 2024-09-21T15:00:52.954 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:00:52.949+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:00:52.990 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:52.991 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:00:52.991 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T15:00:52.991 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T15:00:52.991 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:53.001 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T15:00:53.055 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:00:53.051+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:00:53.153 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:53.153 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T15:00:53.153 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T15:00:53.153 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T15:00:53.153 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.153 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:63820 3.19 3.626bc079 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-21T15:00:53.153 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:27.241293+0000", 2024-09-21T15:00:53.153 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 85.908971147000003, 2024-09-21T15:00:53.153 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.98723808199999996, 2024-09-21T15:00:53.153 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:53.153 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:53.153 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:53.154 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:53.154 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:53.154 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 63820 2024-09-21T15:00:53.154 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.154 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:53.154 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.154 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:53.154 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241293+0000", 2024-09-21T15:00:53.154 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:53.154 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.154 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.154 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:53.154 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241293+0000", 2024-09-21T15:00:53.155 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:53.155 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.155 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.155 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:53.155 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241291+0000", 2024-09-21T15:00:53.155 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:00:53.155 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.155 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.155 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:53.155 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241299+0000", 2024-09-21T15:00:53.155 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1529999999999997e-06 2024-09-21T15:00:53.155 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.155 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.155 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:53.156 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241317+0000", 2024-09-21T15:00:53.156 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7764000000000001e-05 2024-09-21T15:00:53.156 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.156 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.156 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:53.156 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241328+0000", 2024-09-21T15:00:53.156 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1245e-05 2024-09-21T15:00:53.156 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.156 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.156 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:53.156 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241407+0000", 2024-09-21T15:00:53.156 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8762000000000002e-05 2024-09-21T15:00:53.156 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.157 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.157 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:00:53.157 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241418+0000", 2024-09-21T15:00:53.157 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1695000000000001e-05 2024-09-21T15:00:53.157 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.157 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.157 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:53.157 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:28.228510+0000", 2024-09-21T15:00:53.157 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.98709166100000001 2024-09-21T15:00:53.157 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.157 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.157 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:53.157 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:28.228531+0000", 2024-09-21T15:00:53.157 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0696e-05 2024-09-21T15:00:53.158 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:53.158 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:53.158 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:53.158 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.158 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.158 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:97734 3.19 3:9fcf754f:::rbd_data.1136bfc79a70.0000000000000009:head [write 0~4194304 [fadvise_sequential+fadvise_nocache],write 4784128~147456 [fadvise_sequential+fadvise_nocache],write 4997120~147456 [fadvise_sequential+fadvise_nocache],write 5210112~32768 [fadvise_sequential+fadvise_nocache],write 5292032~147456 [fadvise_sequential+fadvise_nocache],write 5505024~147456 [fadvise_sequential+fadvise_nocache],write 5718016~49152 [fadvise_sequential+fadvise_nocache],write 5799936~147456 [fadvise_sequential+fadvise_nocache],write 6012928~147456 [fadvise_sequential+fadvise_nocache],write 6225920~65536 [fadvise_sequential+fadvise_nocache],write 6307840~147456 [fadvise_sequential+fadvise_nocache],write 6520832~147456 [fadvise_sequential+fadvise_nocache],write 6733824~229376 [fadvise_sequential+fadvise_nocache],write 7028736~147456 [fadvise_sequential+fadvise_nocache],write 7241728~229376 [fadvise_sequential+fadvise_nocache],write 7536640~147456 [fadvise_sequential+fadvise_nocache],write 7749632~229376 [fadvise_sequential+fadvise_nocache],write 8077312~114688 [fadvise_sequential+fadvise_nocache],write 8290304~98304 [fadvise_sequential+fadvise_nocache],write 8404992~81920 [fadvise_sequential+fadvise_nocache],write 8617984~81920 [fadvise_sequential+fadvise_nocache],write 8830976~81920 [fadvise_sequential+fadvise_nocache],write 8962048~32768 [fadvise_sequential+fadvise_nocache],write 9175040~32768 [fadvise_sequential+fadvise_nocache],write 9388032~32768 [fadvise_sequential+fadvise_nocache],write 9469952~32768 [fadvise_sequential+fadvise_nocache],write 9682944~20092 [fadvise_sequential+fadvise_nocache],write 9961472~163840 [fadvise_sequential+fadvise_nocache],write 10141696~196608 [fadvise_sequential+fadvise_nocache],write 10354688~278528 [fadvise_sequential+fadvise_nocache],write 10649600~196608 [fadvise_sequential+fadvise_nocache],write 10862592~196608 [fadvise_sequential+fadvise_nocache],write 11075584~65536 [fadvise_sequential+fadvise_nocache],write 11173888~98304 [fadvise_sequential+fadvise_nocache],write 11288576~65536 [fadvise_sequential+fadvise_nocache],write 11386880~98304 [fadvise_sequential+fadvise_nocache],write 11501568~65536 [fadvise_sequential+fadvise_nocache],write 11583488~65536 [fadvise_sequential+fadvise_nocache],write 11681792~98304 [fadvise_sequential+fadvise_nocache],write 11796480~65536 [fadvise_sequential+fadvise_nocache],write 11894784~98304 [fadvise_sequential+fadvise_nocache],write 12009472~65536 [fadvise_sequential+fadvise_nocache],write 12189696~98304 [fadvise_sequential+fadvise_nocache],write 12402688~98304 [fadvise_sequential+fadvise_nocache],write 12697600~98304 [fadvise_sequential+fadvise_nocache],write 12910592~98304 [fadvise_sequential+fadvise_nocache],write 13205504~98304 [fadvise_sequential+fadvise_nocache],write 13418496~98304 [fadvise_sequential+fadvise_nocache],write 13713408~98304 [fadvise_sequential+fadvise_nocache],write 13926400~98304 [fadvise_sequential+fadvise_nocache],write 14139392~16384 [fadvise_sequential+fadvise_nocache],write 14680064~16384 [fadvise_sequential+fadvise_nocache],write 14827520~81920 [fadvise_sequential+fadvise_nocache],write 15040512~81920 [fadvise_sequential+fadvise_nocache],write 15335424~81920 [fadvise_sequential+fadvise_nocache],write 15548416~81920 [fadvise_sequential+fadvise_nocache],write 15843328~81920 [fadvise_sequential+fadvise_nocache],write 16056320~81920 [fadvise_sequential+fadvise_nocache],write 16351232~81920 [fadvise_sequential+fadvise_nocache],write 16564224~81920 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:53.158 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.354331+0000", 2024-09-21T15:00:53.158 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 47.795933099999999, 2024-09-21T15:00:53.158 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.086284870999999999, 2024-09-21T15:00:53.158 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:53.158 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:53.158 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:53.158 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:53.158 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:53.159 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97734 2024-09-21T15:00:53.159 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.159 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:53.159 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.159 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:53.159 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.354331+0000", 2024-09-21T15:00:53.159 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:53.159 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.159 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.159 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:53.159 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.354331+0000", 2024-09-21T15:00:53.159 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:53.159 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.159 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.159 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:53.160 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.354329+0000", 2024-09-21T15:00:53.160 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:00:53.160 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.160 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.160 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:53.160 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.370234+0000", 2024-09-21T15:00:53.160 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015905348999999999 2024-09-21T15:00:53.160 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.160 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.160 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:53.160 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.370286+0000", 2024-09-21T15:00:53.160 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1656999999999997e-05 2024-09-21T15:00:53.160 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.160 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.161 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:53.161 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.370315+0000", 2024-09-21T15:00:53.161 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9204e-05 2024-09-21T15:00:53.161 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.161 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.161 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:53.161 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.370395+0000", 2024-09-21T15:00:53.161 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0395999999999994e-05 2024-09-21T15:00:53.161 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.161 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.161 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:53.161 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.370604+0000", 2024-09-21T15:00:53.161 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000208684 2024-09-21T15:00:53.162 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.162 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.162 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 1", 2024-09-21T15:00:53.162 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.371701+0000", 2024-09-21T15:00:53.162 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0010973140000000001 2024-09-21T15:00:53.162 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.162 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.162 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:53.162 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.405485+0000", 2024-09-21T15:00:53.162 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.033783615000000003 2024-09-21T15:00:53.162 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.162 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.162 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:53.162 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.440211+0000", 2024-09-21T15:00:53.163 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034725757000000003 2024-09-21T15:00:53.163 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.163 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.163 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:53.163 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.440497+0000", 2024-09-21T15:00:53.163 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00028673099999999998 2024-09-21T15:00:53.163 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.163 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.163 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:53.163 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.440616+0000", 2024-09-21T15:00:53.163 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000118256 2024-09-21T15:00:53.163 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:53.163 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:53.163 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:53.164 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.164 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.164 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112831 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 1515520~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:53.164 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.406761+0000", 2024-09-21T15:00:53.164 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.743502402000001, 2024-09-21T15:00:53.164 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060265666000000002, 2024-09-21T15:00:53.164 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:53.164 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:53.164 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:53.164 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:53.164 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:53.164 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112831 2024-09-21T15:00:53.164 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.164 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:53.164 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.165 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:53.165 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406761+0000", 2024-09-21T15:00:53.165 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:53.165 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.165 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.165 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:53.165 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406761+0000", 2024-09-21T15:00:53.165 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:53.165 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.165 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.165 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:53.165 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406761+0000", 2024-09-21T15:00:53.165 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:53.166 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.166 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.166 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:53.166 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406764+0000", 2024-09-21T15:00:53.166 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0319999999999999e-06 2024-09-21T15:00:53.166 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.166 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.166 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:53.166 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406776+0000", 2024-09-21T15:00:53.166 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2357999999999999e-05 2024-09-21T15:00:53.166 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.166 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.166 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:53.167 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406784+0000", 2024-09-21T15:00:53.167 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6059999999999997e-06 2024-09-21T15:00:53.167 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.167 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.167 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:53.167 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.466595+0000", 2024-09-21T15:00:53.167 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059811062999999998 2024-09-21T15:00:53.167 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.167 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.167 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:53.167 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.466782+0000", 2024-09-21T15:00:53.167 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000186622 2024-09-21T15:00:53.167 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.167 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.168 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:53.168 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.467027+0000", 2024-09-21T15:00:53.168 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000245455 2024-09-21T15:00:53.168 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:53.168 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:53.168 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:53.168 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.168 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.168 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112832 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 1605632~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:53.168 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.406788+0000", 2024-09-21T15:00:53.168 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.743475812, 2024-09-21T15:00:53.168 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060452065999999999, 2024-09-21T15:00:53.168 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:53.168 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:53.169 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:53.169 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:53.169 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:53.169 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112832 2024-09-21T15:00:53.169 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.169 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:53.169 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.169 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:53.169 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406788+0000", 2024-09-21T15:00:53.169 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:53.169 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.169 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.169 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:53.169 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406788+0000", 2024-09-21T15:00:53.169 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:53.170 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.170 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.170 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:53.170 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406788+0000", 2024-09-21T15:00:53.170 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:53.170 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.170 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.170 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:53.170 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406798+0000", 2024-09-21T15:00:53.170 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0096e-05 2024-09-21T15:00:53.170 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.170 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.170 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:53.170 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406803+0000", 2024-09-21T15:00:53.171 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2020000000000003e-06 2024-09-21T15:00:53.171 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.171 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.171 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:53.171 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406808+0000", 2024-09-21T15:00:53.171 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8239999999999999e-06 2024-09-21T15:00:53.171 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.171 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.171 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:53.171 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.467049+0000", 2024-09-21T15:00:53.171 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060241754000000002 2024-09-21T15:00:53.171 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.171 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.171 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:53.171 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.467134+0000", 2024-09-21T15:00:53.172 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4216999999999997e-05 2024-09-21T15:00:53.172 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.172 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.172 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:53.172 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.467240+0000", 2024-09-21T15:00:53.172 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000106304 2024-09-21T15:00:53.172 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:53.172 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:53.172 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:53.172 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.172 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.172 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112843 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 2613248~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:53.172 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.410273+0000", 2024-09-21T15:00:53.172 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.739991218, 2024-09-21T15:00:53.173 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059854931, 2024-09-21T15:00:53.173 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:53.173 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:53.173 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:53.173 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:53.173 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:53.173 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112843 2024-09-21T15:00:53.173 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.173 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:53.173 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.173 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:53.173 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410273+0000", 2024-09-21T15:00:53.173 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:53.174 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.174 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.174 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:53.174 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410273+0000", 2024-09-21T15:00:53.174 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:53.174 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.174 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.174 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:53.174 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410272+0000", 2024-09-21T15:00:53.174 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:53.174 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.174 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.174 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:53.174 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410274+0000", 2024-09-21T15:00:53.175 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.905e-06 2024-09-21T15:00:53.175 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.175 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.175 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:53.175 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410283+0000", 2024-09-21T15:00:53.175 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3440000000000007e-06 2024-09-21T15:00:53.175 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.175 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.175 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:53.175 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410294+0000", 2024-09-21T15:00:53.175 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0271e-05 2024-09-21T15:00:53.175 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.175 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.175 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:53.175 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.469917+0000", 2024-09-21T15:00:53.176 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059623738000000003 2024-09-21T15:00:53.176 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.176 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.176 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:53.176 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470002+0000", 2024-09-21T15:00:53.176 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4270999999999993e-05 2024-09-21T15:00:53.176 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.176 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.176 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:53.176 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470127+0000", 2024-09-21T15:00:53.176 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000125711 2024-09-21T15:00:53.176 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:53.176 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:53.176 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:53.177 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.177 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.177 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112844 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 2703360~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:53.177 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.410326+0000", 2024-09-21T15:00:53.177 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.739937498, 2024-09-21T15:00:53.177 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060086056999999998, 2024-09-21T15:00:53.177 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:53.177 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:53.177 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:53.177 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:53.177 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:53.177 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112844 2024-09-21T15:00:53.177 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.177 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:53.177 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.178 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:53.178 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410326+0000", 2024-09-21T15:00:53.178 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:53.178 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.178 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.178 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:53.178 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410326+0000", 2024-09-21T15:00:53.178 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:53.178 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.178 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.178 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:53.178 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410326+0000", 2024-09-21T15:00:53.178 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:00:53.178 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.179 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.179 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:53.179 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410352+0000", 2024-09-21T15:00:53.179 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6872999999999998e-05 2024-09-21T15:00:53.179 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.179 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.179 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:53.179 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410374+0000", 2024-09-21T15:00:53.179 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1141e-05 2024-09-21T15:00:53.179 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.179 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.179 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:53.179 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410385+0000", 2024-09-21T15:00:53.180 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1456e-05 2024-09-21T15:00:53.180 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.180 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.180 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:53.180 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470174+0000", 2024-09-21T15:00:53.180 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059788574999999997 2024-09-21T15:00:53.180 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.180 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.180 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:53.180 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470277+0000", 2024-09-21T15:00:53.180 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000103838 2024-09-21T15:00:53.180 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.180 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.180 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:53.180 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470412+0000", 2024-09-21T15:00:53.181 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013489700000000001 2024-09-21T15:00:53.181 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:53.181 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:53.181 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:53.181 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.181 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.181 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112845 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 2793472~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:53.181 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.410394+0000", 2024-09-21T15:00:53.181 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.739870023999998, 2024-09-21T15:00:53.181 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060244173999999998, 2024-09-21T15:00:53.181 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:53.181 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:53.181 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:53.181 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:53.182 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:53.182 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112845 2024-09-21T15:00:53.182 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.182 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:53.182 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.182 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:53.182 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410394+0000", 2024-09-21T15:00:53.182 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:53.182 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.182 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.182 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:53.182 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410394+0000", 2024-09-21T15:00:53.182 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:53.183 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.183 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.183 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:53.183 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410393+0000", 2024-09-21T15:00:53.183 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:53.183 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.183 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.183 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:53.183 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410397+0000", 2024-09-21T15:00:53.183 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9430000000000002e-06 2024-09-21T15:00:53.183 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.183 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.183 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:53.183 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410406+0000", 2024-09-21T15:00:53.184 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.799e-06 2024-09-21T15:00:53.184 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.184 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.184 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:53.184 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410418+0000", 2024-09-21T15:00:53.184 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2225999999999999e-05 2024-09-21T15:00:53.184 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.184 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.184 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:53.184 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470444+0000", 2024-09-21T15:00:53.184 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060026311999999998 2024-09-21T15:00:53.184 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.184 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.185 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:53.185 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470535+0000", 2024-09-21T15:00:53.185 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0173999999999994e-05 2024-09-21T15:00:53.185 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.185 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.185 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:53.185 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470638+0000", 2024-09-21T15:00:53.185 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010333 2024-09-21T15:00:53.185 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:53.185 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:53.185 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:53.185 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.185 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.185 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112846 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 2883584~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:53.186 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.410425+0000", 2024-09-21T15:00:53.186 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.739838810999998, 2024-09-21T15:00:53.186 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060459247000000001, 2024-09-21T15:00:53.186 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:53.186 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:53.186 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:53.186 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:53.186 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:53.186 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112846 2024-09-21T15:00:53.186 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.186 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:53.186 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.186 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:53.186 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410425+0000", 2024-09-21T15:00:53.187 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:53.187 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.187 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.187 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:53.187 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410425+0000", 2024-09-21T15:00:53.187 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:53.187 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.187 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.187 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:53.187 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410424+0000", 2024-09-21T15:00:53.187 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:53.187 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.187 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.187 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:53.188 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410428+0000", 2024-09-21T15:00:53.188 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4709999999999999e-06 2024-09-21T15:00:53.188 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.188 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.188 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:53.188 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410446+0000", 2024-09-21T15:00:53.188 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8354000000000001e-05 2024-09-21T15:00:53.188 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.188 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.188 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:53.188 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410456+0000", 2024-09-21T15:00:53.188 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6709999999999994e-06 2024-09-21T15:00:53.188 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.188 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.188 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:53.189 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470661+0000", 2024-09-21T15:00:53.189 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060205303000000002 2024-09-21T15:00:53.189 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.189 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.189 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:53.189 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470756+0000", 2024-09-21T15:00:53.189 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4401999999999995e-05 2024-09-21T15:00:53.189 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.189 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.189 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:53.189 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470884+0000", 2024-09-21T15:00:53.189 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012860500000000001 2024-09-21T15:00:53.189 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:53.189 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:53.190 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:53.190 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.190 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.190 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112847 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 2973696~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:53.190 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.410462+0000", 2024-09-21T15:00:53.190 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.739801766999999, 2024-09-21T15:00:53.190 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060653836000000003, 2024-09-21T15:00:53.190 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:53.190 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:53.190 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:53.190 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:53.190 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:53.190 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112847 2024-09-21T15:00:53.190 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.191 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:53.191 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.191 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:53.191 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410462+0000", 2024-09-21T15:00:53.191 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:53.191 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.191 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.191 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:53.191 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410462+0000", 2024-09-21T15:00:53.191 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:53.192 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.192 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.192 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:53.192 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410461+0000", 2024-09-21T15:00:53.192 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:53.192 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.192 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.192 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:53.192 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410465+0000", 2024-09-21T15:00:53.192 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6899999999999998e-06 2024-09-21T15:00:53.192 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.192 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.192 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:53.192 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410474+0000", 2024-09-21T15:00:53.193 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4179999999999997e-06 2024-09-21T15:00:53.193 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.193 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.193 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:53.193 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410508+0000", 2024-09-21T15:00:53.193 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4282999999999997e-05 2024-09-21T15:00:53.193 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.193 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.193 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:53.193 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470922+0000", 2024-09-21T15:00:53.193 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060413820999999999 2024-09-21T15:00:53.193 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.193 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.193 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:53.194 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471008+0000", 2024-09-21T15:00:53.194 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6379999999999996e-05 2024-09-21T15:00:53.194 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.194 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.194 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:53.194 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471116+0000", 2024-09-21T15:00:53.194 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000107806 2024-09-21T15:00:53.194 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:53.194 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:53.194 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:53.194 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.194 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.195 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112848 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 3063808~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:53.195 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.410525+0000", 2024-09-21T15:00:53.195 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.739738484, 2024-09-21T15:00:53.195 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060880735999999998, 2024-09-21T15:00:53.195 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:53.195 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:53.195 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:53.195 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:53.195 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:53.195 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112848 2024-09-21T15:00:53.195 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.195 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:53.195 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.195 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:53.196 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410525+0000", 2024-09-21T15:00:53.196 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:53.196 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.196 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.196 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:53.196 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410525+0000", 2024-09-21T15:00:53.196 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:53.196 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.196 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.196 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:53.196 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410525+0000", 2024-09-21T15:00:53.196 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:53.196 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.196 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.196 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:53.196 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410529+0000", 2024-09-21T15:00:53.197 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1760000000000003e-06 2024-09-21T15:00:53.197 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.197 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.197 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:53.197 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410551+0000", 2024-09-21T15:00:53.197 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2099999999999998e-05 2024-09-21T15:00:53.197 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.197 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.197 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:53.197 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410571+0000", 2024-09-21T15:00:53.197 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0159000000000001e-05 2024-09-21T15:00:53.197 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.197 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.197 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:53.198 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471164+0000", 2024-09-21T15:00:53.198 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.06059287 2024-09-21T15:00:53.198 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.198 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.198 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:53.198 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471271+0000", 2024-09-21T15:00:53.198 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000106635 2024-09-21T15:00:53.198 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.198 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.198 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:53.198 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471406+0000", 2024-09-21T15:00:53.198 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000135399 2024-09-21T15:00:53.198 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:53.198 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:53.198 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:53.199 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.199 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.199 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112849 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 3153920~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:53.199 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.410579+0000", 2024-09-21T15:00:53.199 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.739685188999999, 2024-09-21T15:00:53.199 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.061057304999999999, 2024-09-21T15:00:53.199 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:53.199 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:53.199 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:53.199 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:53.199 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:53.199 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112849 2024-09-21T15:00:53.199 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.199 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:53.200 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.200 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:53.200 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410579+0000", 2024-09-21T15:00:53.200 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:53.200 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.200 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.200 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:53.200 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410579+0000", 2024-09-21T15:00:53.200 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:53.200 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.200 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.200 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:53.200 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410578+0000", 2024-09-21T15:00:53.200 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:53.200 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.201 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.201 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:53.201 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410582+0000", 2024-09-21T15:00:53.201 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.036e-06 2024-09-21T15:00:53.201 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.201 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.201 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:53.201 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410592+0000", 2024-09-21T15:00:53.201 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9850000000000001e-06 2024-09-21T15:00:53.201 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.201 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.201 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:53.201 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410604+0000", 2024-09-21T15:00:53.201 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2051e-05 2024-09-21T15:00:53.202 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.202 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.202 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:53.202 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471444+0000", 2024-09-21T15:00:53.202 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060839722999999998 2024-09-21T15:00:53.202 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.202 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.202 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:53.202 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471530+0000", 2024-09-21T15:00:53.202 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6359000000000004e-05 2024-09-21T15:00:53.202 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.202 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.202 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:53.203 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471636+0000", 2024-09-21T15:00:53.203 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010576700000000001 2024-09-21T15:00:53.203 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:53.203 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:53.203 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:53.203 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.203 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.203 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112850 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 3244032~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:53.203 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.410698+0000", 2024-09-21T15:00:53.203 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.739565847000001, 2024-09-21T15:00:53.203 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.061164118000000003, 2024-09-21T15:00:53.203 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:53.203 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:53.204 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:53.204 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:53.204 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:53.204 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112850 2024-09-21T15:00:53.204 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.204 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:53.204 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.204 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:53.204 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410698+0000", 2024-09-21T15:00:53.204 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:53.204 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.204 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.204 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:53.204 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410698+0000", 2024-09-21T15:00:53.205 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:53.205 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.205 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.205 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:53.205 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410610+0000", 2024-09-21T15:00:53.205 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999123 2024-09-21T15:00:53.205 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.205 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.205 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:53.205 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410707+0000", 2024-09-21T15:00:53.205 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7093e-05 2024-09-21T15:00:53.205 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.205 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.205 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:53.205 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410728+0000", 2024-09-21T15:00:53.206 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0275000000000002e-05 2024-09-21T15:00:53.206 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.206 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.206 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:53.206 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410740+0000", 2024-09-21T15:00:53.206 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2561999999999999e-05 2024-09-21T15:00:53.206 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.206 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.206 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:53.206 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471664+0000", 2024-09-21T15:00:53.206 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060923979000000003 2024-09-21T15:00:53.206 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.206 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.206 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:53.207 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471753+0000", 2024-09-21T15:00:53.207 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8265999999999998e-05 2024-09-21T15:00:53.207 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.207 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.207 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:53.207 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471862+0000", 2024-09-21T15:00:53.207 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000109474 2024-09-21T15:00:53.207 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:53.207 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:53.207 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:53.207 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.207 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.207 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112864 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 4521984~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:53.207 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.414789+0000", 2024-09-21T15:00:53.207 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.735474822, 2024-09-21T15:00:53.208 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059845903999999998, 2024-09-21T15:00:53.208 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:53.208 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:53.208 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:53.208 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:53.208 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:53.208 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112864 2024-09-21T15:00:53.208 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.208 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:53.208 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.208 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:53.208 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414789+0000", 2024-09-21T15:00:53.208 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:53.208 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.208 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.208 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:53.209 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414789+0000", 2024-09-21T15:00:53.209 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:53.209 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.209 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.209 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:53.209 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414788+0000", 2024-09-21T15:00:53.209 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:53.209 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.209 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.209 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:53.209 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414790+0000", 2024-09-21T15:00:53.209 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.995e-06 2024-09-21T15:00:53.209 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.209 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.209 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:53.209 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414801+0000", 2024-09-21T15:00:53.210 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.01e-05 2024-09-21T15:00:53.210 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.210 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.210 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:53.210 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414804+0000", 2024-09-21T15:00:53.210 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.563e-06 2024-09-21T15:00:53.210 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.210 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.210 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:53.210 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.474499+0000", 2024-09-21T15:00:53.210 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059695208 2024-09-21T15:00:53.210 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.210 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.210 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:53.210 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.474561+0000", 2024-09-21T15:00:53.211 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1672000000000002e-05 2024-09-21T15:00:53.211 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.211 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.211 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:53.211 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.474635+0000", 2024-09-21T15:00:53.211 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3792000000000001e-05 2024-09-21T15:00:53.211 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:53.211 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:53.211 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:53.211 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.211 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.211 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112865 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 4612096~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:53.211 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.414807+0000", 2024-09-21T15:00:53.211 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.735456425999999, 2024-09-21T15:00:53.211 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060003506999999998, 2024-09-21T15:00:53.212 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:53.212 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:53.212 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:53.212 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:53.212 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:53.212 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112865 2024-09-21T15:00:53.212 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.212 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:53.212 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.212 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:53.212 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414807+0000", 2024-09-21T15:00:53.212 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:53.212 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.212 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.212 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:53.213 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414807+0000", 2024-09-21T15:00:53.213 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:53.213 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.213 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.213 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:53.213 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414807+0000", 2024-09-21T15:00:53.213 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:53.213 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.213 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.213 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:53.213 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414809+0000", 2024-09-21T15:00:53.213 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0509999999999999e-06 2024-09-21T15:00:53.213 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.213 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.213 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:53.213 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414813+0000", 2024-09-21T15:00:53.214 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0960000000000003e-06 2024-09-21T15:00:53.214 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.214 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.214 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:53.214 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414817+0000", 2024-09-21T15:00:53.214 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.296e-06 2024-09-21T15:00:53.214 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.214 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.214 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:53.214 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.474671+0000", 2024-09-21T15:00:53.214 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059853663000000001 2024-09-21T15:00:53.214 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.214 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.214 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:53.215 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.474735+0000", 2024-09-21T15:00:53.215 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3653999999999997e-05 2024-09-21T15:00:53.215 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.215 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.215 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:53.215 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.474811+0000", 2024-09-21T15:00:53.215 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6074000000000004e-05 2024-09-21T15:00:53.215 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:53.215 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:53.215 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:53.215 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.215 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.215 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112866 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 4702208~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:53.215 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.414820+0000", 2024-09-21T15:00:53.215 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.735443505999999, 2024-09-21T15:00:53.216 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060161992999999997, 2024-09-21T15:00:53.216 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:53.216 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:53.216 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:53.216 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:53.216 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:53.216 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112866 2024-09-21T15:00:53.216 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.216 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:53.216 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.216 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:53.216 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414820+0000", 2024-09-21T15:00:53.216 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:53.216 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.217 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.217 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:53.217 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414820+0000", 2024-09-21T15:00:53.217 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:53.217 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.217 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.217 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:53.217 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414820+0000", 2024-09-21T15:00:53.217 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:53.217 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.217 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.217 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:53.217 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414822+0000", 2024-09-21T15:00:53.217 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8509999999999999e-06 2024-09-21T15:00:53.217 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.218 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.218 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:53.218 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414826+0000", 2024-09-21T15:00:53.218 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8190000000000002e-06 2024-09-21T15:00:53.218 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.218 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.218 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:53.218 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414837+0000", 2024-09-21T15:00:53.218 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1232e-05 2024-09-21T15:00:53.218 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.218 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.218 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:53.218 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.474846+0000", 2024-09-21T15:00:53.218 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060009506999999997 2024-09-21T15:00:53.218 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.219 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.219 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:53.219 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.474910+0000", 2024-09-21T15:00:53.219 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3918999999999996e-05 2024-09-21T15:00:53.219 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.219 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.219 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:53.219 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.474982+0000", 2024-09-21T15:00:53.219 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1958999999999993e-05 2024-09-21T15:00:53.219 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:53.219 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:53.219 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:53.219 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.219 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.219 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112867 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 4792320~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:53.220 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.414840+0000", 2024-09-21T15:00:53.220 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.735424010999999, 2024-09-21T15:00:53.220 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060308542999999999, 2024-09-21T15:00:53.220 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:53.220 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:53.220 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:53.220 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:53.220 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:53.220 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112867 2024-09-21T15:00:53.220 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.220 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:53.220 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.220 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:53.221 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414840+0000", 2024-09-21T15:00:53.221 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:53.221 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.221 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.221 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:53.221 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414840+0000", 2024-09-21T15:00:53.221 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:53.221 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.221 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.221 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:53.221 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414839+0000", 2024-09-21T15:00:53.221 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:53.221 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.221 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.222 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:53.222 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414841+0000", 2024-09-21T15:00:53.222 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8500000000000001e-06 2024-09-21T15:00:53.222 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.222 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.222 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:53.222 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414846+0000", 2024-09-21T15:00:53.222 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7700000000000001e-06 2024-09-21T15:00:53.222 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.222 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.222 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:53.222 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414850+0000", 2024-09-21T15:00:53.222 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1389999999999997e-06 2024-09-21T15:00:53.222 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.222 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.223 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:53.223 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.475016+0000", 2024-09-21T15:00:53.223 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.06016614 2024-09-21T15:00:53.223 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.223 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.223 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:53.223 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.475073+0000", 2024-09-21T15:00:53.223 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6295000000000002e-05 2024-09-21T15:00:53.223 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.223 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.223 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:53.223 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.475148+0000", 2024-09-21T15:00:53.223 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5647000000000003e-05 2024-09-21T15:00:53.224 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:53.224 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:53.224 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:53.224 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.224 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.224 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:113030 3.6 3:60420fc2:::rbd_data.1136b442e94a.0000000000000001:head [write 0~360448 [fadvise_sequential+fadvise_nocache],write 385024~573440 [fadvise_sequential+fadvise_nocache],write 1146880~573440 [fadvise_sequential+fadvise_nocache],write 1908736~188416 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:53.224 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.422420+0000", 2024-09-21T15:00:53.224 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.727843365999998, 2024-09-21T15:00:53.224 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.063268959999999999, 2024-09-21T15:00:53.224 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:53.224 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:53.224 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:53.224 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:53.224 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:53.224 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 113030 2024-09-21T15:00:53.225 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.225 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:53.225 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.225 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:53.225 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.422420+0000", 2024-09-21T15:00:53.225 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:53.225 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.225 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.225 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:53.225 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.422420+0000", 2024-09-21T15:00:53.225 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:53.225 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.225 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.225 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:53.225 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.422420+0000", 2024-09-21T15:00:53.226 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:53.226 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.226 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.226 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:53.226 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.424657+0000", 2024-09-21T15:00:53.226 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0022369780000000001 2024-09-21T15:00:53.226 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.226 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.226 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:53.226 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.424673+0000", 2024-09-21T15:00:53.226 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6141e-05 2024-09-21T15:00:53.226 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.226 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.227 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:53.227 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.424689+0000", 2024-09-21T15:00:53.227 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5897999999999999e-05 2024-09-21T15:00:53.227 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.227 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.227 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:53.227 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.478005+0000", 2024-09-21T15:00:53.227 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.05331636 2024-09-21T15:00:53.227 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.227 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.227 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:53.227 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.478133+0000", 2024-09-21T15:00:53.227 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012736799999999999 2024-09-21T15:00:53.227 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.228 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.228 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 7", 2024-09-21T15:00:53.228 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.478307+0000", 2024-09-21T15:00:53.228 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017491499999999999 2024-09-21T15:00:53.228 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.228 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.228 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:53.228 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.483611+0000", 2024-09-21T15:00:53.228 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0053039389999999997 2024-09-21T15:00:53.228 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.228 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.228 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:53.228 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.485553+0000", 2024-09-21T15:00:53.228 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0019418759999999999 2024-09-21T15:00:53.229 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.229 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.229 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:53.229 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.485645+0000", 2024-09-21T15:00:53.229 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2138000000000005e-05 2024-09-21T15:00:53.229 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.229 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.229 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:53.229 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.485689+0000", 2024-09-21T15:00:53.229 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3922999999999999e-05 2024-09-21T15:00:53.229 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:53.229 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:53.229 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:53.229 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.229 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.230 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:114290 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 6946816~8192 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:53.230 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.558087+0000", 2024-09-21T15:00:53.230 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.592176568999999, 2024-09-21T15:00:53.230 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059973091999999999, 2024-09-21T15:00:53.230 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:53.230 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:53.230 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:53.230 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:53.230 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:53.230 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 114290 2024-09-21T15:00:53.230 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.230 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:53.230 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.230 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:53.231 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.558087+0000", 2024-09-21T15:00:53.231 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:53.231 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.231 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.231 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:53.231 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.558087+0000", 2024-09-21T15:00:53.231 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:53.231 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.231 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.231 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:53.231 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.558087+0000", 2024-09-21T15:00:53.231 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:53.231 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.231 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.231 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:53.232 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.558092+0000", 2024-09-21T15:00:53.232 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2379999999999997e-06 2024-09-21T15:00:53.232 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.232 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.232 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:53.232 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.558095+0000", 2024-09-21T15:00:53.232 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.275e-06 2024-09-21T15:00:53.232 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.232 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.232 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:53.232 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.558099+0000", 2024-09-21T15:00:53.232 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6679999999999999e-06 2024-09-21T15:00:53.232 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.232 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.232 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:53.233 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.617757+0000", 2024-09-21T15:00:53.233 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059657531999999999 2024-09-21T15:00:53.233 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.233 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.233 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:53.233 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.617891+0000", 2024-09-21T15:00:53.233 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000134478 2024-09-21T15:00:53.233 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.233 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.233 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:53.233 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.618060+0000", 2024-09-21T15:00:53.233 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016925100000000001 2024-09-21T15:00:53.233 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:53.233 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:53.233 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:53.234 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.234 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.234 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:114312 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 8945664~8192 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:53.234 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.562396+0000", 2024-09-21T15:00:53.234 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.587868221000001, 2024-09-21T15:00:53.234 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059832396000000003, 2024-09-21T15:00:53.234 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:53.234 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:53.234 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:53.234 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:53.234 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:53.234 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 114312 2024-09-21T15:00:53.234 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.235 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:53.235 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.235 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:53.235 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562396+0000", 2024-09-21T15:00:53.235 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:53.235 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.235 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.235 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:53.235 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562396+0000", 2024-09-21T15:00:53.235 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:53.235 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.235 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.235 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:53.235 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562395+0000", 2024-09-21T15:00:53.235 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:53.235 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.235 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.236 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:53.236 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562397+0000", 2024-09-21T15:00:53.236 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9e-06 2024-09-21T15:00:53.236 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.236 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.236 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:53.236 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562401+0000", 2024-09-21T15:00:53.236 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8099999999999999e-06 2024-09-21T15:00:53.236 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.236 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.236 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:53.236 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562405+0000", 2024-09-21T15:00:53.236 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.5710000000000002e-06 2024-09-21T15:00:53.236 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.236 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.237 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:53.237 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.621997+0000", 2024-09-21T15:00:53.237 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059592118999999999 2024-09-21T15:00:53.237 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.237 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.237 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:53.237 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.622121+0000", 2024-09-21T15:00:53.237 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012411599999999999 2024-09-21T15:00:53.237 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.237 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.237 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:53.237 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.622228+0000", 2024-09-21T15:00:53.237 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000107184 2024-09-21T15:00:53.237 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:53.238 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:53.238 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:53.238 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.238 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.238 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:114313 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 9035776~8192 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:53.238 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.562408+0000", 2024-09-21T15:00:53.238 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.587856240000001, 2024-09-21T15:00:53.238 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059975560999999997, 2024-09-21T15:00:53.238 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:53.238 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:53.238 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:53.238 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:53.238 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:53.238 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 114313 2024-09-21T15:00:53.239 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.239 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:53.239 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.239 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:53.239 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562408+0000", 2024-09-21T15:00:53.239 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:53.239 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.239 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.239 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:53.239 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562408+0000", 2024-09-21T15:00:53.239 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:53.239 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.239 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.239 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:53.240 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562407+0000", 2024-09-21T15:00:53.240 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:53.240 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.240 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.240 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:53.240 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562409+0000", 2024-09-21T15:00:53.240 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9700000000000002e-06 2024-09-21T15:00:53.240 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.240 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.240 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:53.240 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562421+0000", 2024-09-21T15:00:53.240 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2094e-05 2024-09-21T15:00:53.240 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.240 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.240 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:53.240 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562427+0000", 2024-09-21T15:00:53.241 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7690000000000002e-06 2024-09-21T15:00:53.241 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.241 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.241 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:53.241 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.622264+0000", 2024-09-21T15:00:53.241 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059837488000000001 2024-09-21T15:00:53.241 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.241 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.241 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:53.241 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.622319+0000", 2024-09-21T15:00:53.241 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4064999999999998e-05 2024-09-21T15:00:53.241 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.241 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.241 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:53.241 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.622383+0000", 2024-09-21T15:00:53.242 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4506999999999998e-05 2024-09-21T15:00:53.242 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:53.242 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:53.242 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:53.242 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:53.242 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:53.242 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:53.245 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T15:00:53.245 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:00:53.241+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:00:53.246 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-21T15:00:53.247 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-09-21T15:00:53.247 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":243,"stamp":"2024-09-21T15:00:52.881087+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":944710440,"num_objects":1196,"num_object_clones":0,"num_object_copies":2392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1196,"num_whiteouts":0,"num_read":133976,"num_read_kb":1088370,"num_write":23953,"num_write_kb":838140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1381,"num_bytes_recovered":103685771,"num_keys_recovered":152,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15917,"ondisk_log_size":15917,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":434292,"kb_used_data":74616,"kb_used_omap":12,"kb_used_meta":359539,"kb_avail":754540428,"statfs":{"total":773094113280,"available":772649398272,"internally_reserved":0,"allocated":76406784,"data_stored":554352621,"data_compressed":12145944,"data_compressed_allocated":48709632,"data_compressed_original":540046961,"omap_allocated":12744,"internal_metadata":368168504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6758530,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":3205,"num_read_kb":18132,"num_write":924,"num_write_kb":12706,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.062868"},"pg_stats":[{"pgid":"3.1f","version":"163'442","reported_seq":3353,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-21T15:00:50.673906+0000","last_change":"2024-09-21T15:00:50.235803+0000","last_active":"2024-09-21T15:00:50.673906+0000","last_peered":"2024-09-21T15:00:50.673906+0000","last_clean":"2024-09-21T15:00:50.673906+0000","last_became_active":"2024-09-21T14:58:19.410055+0000","last_became_peered":"2024-09-21T14:58:19.410055+0000","last_unstale":"2024-09-21T15:00:50.673906+0000","last_undegraded":"2024-09-21T15:00:50.673906+0000","last_fullsized":"2024-09-21T15:00:50.673906+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":442,"log_dups_size":0,"ondisk_log_size":442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:49:13.944219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000946819,"stat_sum":{"num_bytes":31451550,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":3345,"num_read_kb":41422,"num_write":666,"num_write_kb":33492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"65","length":"1"}]},{"pgid":"3.1e","version":"160'291","reported_seq":2292,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-21T15:00:50.235267+0000","last_change":"2024-09-21T15:00:50.235872+0000","last_active":"2024-09-21T15:00:50.235267+0000","last_peered":"2024-09-21T15:00:50.235267+0000","last_clean":"2024-09-21T15:00:50.235267+0000","last_became_active":"2024-09-21T14:59:13.130091+0000","last_became_peered":"2024-09-21T14:59:13.130091+0000","last_unstale":"2024-09-21T15:00:50.235267+0000","last_undegraded":"2024-09-21T15:00:50.235267+0000","last_fullsized":"2024-09-21T15:00:50.235267+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":291,"log_dups_size":0,"ondisk_log_size":291,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:46:00.199136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001106281,"stat_sum":{"num_bytes":18252682,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":2122,"num_read_kb":20264,"num_write":381,"num_write_kb":15377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":3844512,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"65","length":"1"}]},{"pgid":"3.1d","version":"163'232","reported_seq":3208,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-21T15:00:50.878532+0000","last_change":"2024-09-21T15:00:50.163080+0000","last_active":"2024-09-21T15:00:50.878532+0000","last_peered":"2024-09-21T15:00:50.878532+0000","last_clean":"2024-09-21T15:00:50.878532+0000","last_became_active":"2024-09-21T14:58:19.411387+0000","last_became_peered":"2024-09-21T14:58:19.411387+0000","last_unstale":"2024-09-21T15:00:50.878532+0000","last_undegraded":"2024-09-21T15:00:50.878532+0000","last_fullsized":"2024-09-21T15:00:50.878532+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":232,"log_dups_size":0,"ondisk_log_size":232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:37:57.597069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084242500000000005,"stat_sum":{"num_bytes":13916871,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":2670,"num_read_kb":20096,"num_write":368,"num_write_kb":22186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"65","length":"1"}]},{"pgid":"3.1c","version":"163'405","reported_seq":2736,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-21T15:00:50.727578+0000","last_change":"2024-09-21T15:00:50.155675+0000","last_active":"2024-09-21T15:00:50.727578+0000","last_peered":"2024-09-21T15:00:50.727578+0000","last_clean":"2024-09-21T15:00:50.727578+0000","last_became_active":"2024-09-21T14:58:19.411306+0000","last_became_peered":"2024-09-21T14:58:19.411306+0000","last_unstale":"2024-09-21T15:00:50.727578+0000","last_undegraded":"2024-09-21T15:00:50.727578+0000","last_fullsized":"2024-09-21T15:00:50.727578+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":405,"log_dups_size":0,"ondisk_log_size":405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:20:27.079795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011898810000000001,"stat_sum":{"num_bytes":17294633,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":2724,"num_read_kb":20123,"num_write":579,"num_write_kb":22866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"65","length":"1"}]},{"pgid":"3.1b","version":"161'393","reported_seq":2317,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-21T15:00:50.219817+0000","last_change":"2024-09-21T15:00:50.153496+0000","last_active":"2024-09-21T15:00:50.219817+0000","last_peered":"2024-09-21T15:00:50.219817+0000","last_clean":"2024-09-21T15:00:50.219817+0000","last_became_active":"2024-09-21T14:59:31.131051+0000","last_became_peered":"2024-09-21T14:59:31.131051+0000","last_unstale":"2024-09-21T15:00:50.219817+0000","last_undegraded":"2024-09-21T15:00:50.219817+0000","last_fullsized":"2024-09-21T15:00:50.219817+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":393,"log_dups_size":0,"ondisk_log_size":393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:21:37.320774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001266098,"stat_sum":{"num_bytes":19383166,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":2196,"num_read_kb":25333,"num_write":493,"num_write_kb":17752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":117,"num_bytes_recovered":7757116,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"65","length":"1"}]},{"pgid":"3.1a","version":"163'330","reported_seq":3009,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-21T15:00:50.907071+0000","last_change":"2024-09-21T15:00:50.156920+0000","last_active":"2024-09-21T15:00:50.907071+0000","last_peered":"2024-09-21T15:00:50.907071+0000","last_clean":"2024-09-21T15:00:50.907071+0000","last_became_active":"2024-09-21T14:58:19.409415+0000","last_became_peered":"2024-09-21T14:58:19.409415+0000","last_unstale":"2024-09-21T15:00:50.907071+0000","last_undegraded":"2024-09-21T15:00:50.907071+0000","last_fullsized":"2024-09-21T15:00:50.907071+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":330,"log_dups_size":0,"ondisk_log_size":330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:28:46.336301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002468661,"stat_sum":{"num_bytes":20353386,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":2835,"num_read_kb":24254,"num_write":442,"num_write_kb":20570,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"65","length":"1"}]},{"pgid":"3.19","version":"163'464","reported_seq":3544,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-21T15:00:50.719042+0000","last_change":"2024-09-21T15:00:50.152699+0000","last_active":"2024-09-21T15:00:50.719042+0000","last_peered":"2024-09-21T15:00:50.719042+0000","last_clean":"2024-09-21T15:00:50.719042+0000","last_became_active":"2024-09-21T14:59:32.130624+0000","last_became_peered":"2024-09-21T14:59:32.130624+0000","last_unstale":"2024-09-21T15:00:50.719042+0000","last_undegraded":"2024-09-21T15:00:50.719042+0000","last_fullsized":"2024-09-21T15:00:50.719042+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":464,"log_dups_size":0,"ondisk_log_size":464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:37:53.312733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071355099999999999,"stat_sum":{"num_bytes":38376840,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":4027,"num_read_kb":50123,"num_write":716,"num_write_kb":25499,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":128,"num_bytes_recovered":11896992,"num_keys_recovered":44,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"65","length":"1"}]},{"pgid":"3.18","version":"163'508","reported_seq":7247,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-21T15:00:50.724433+0000","last_change":"2024-09-21T15:00:50.237349+0000","last_active":"2024-09-21T15:00:50.724433+0000","last_peered":"2024-09-21T15:00:50.724433+0000","last_clean":"2024-09-21T15:00:50.724433+0000","last_became_active":"2024-09-21T14:58:19.409371+0000","last_became_peered":"2024-09-21T14:58:19.409371+0000","last_unstale":"2024-09-21T15:00:50.724433+0000","last_undegraded":"2024-09-21T15:00:50.724433+0000","last_fullsized":"2024-09-21T15:00:50.724433+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":508,"log_dups_size":0,"ondisk_log_size":508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:35:27.143445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021509319999999999,"stat_sum":{"num_bytes":31620138,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":7903,"num_read_kb":39887,"num_write":739,"num_write_kb":26987,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"65","length":"1"}]},{"pgid":"3.17","version":"163'314","reported_seq":3576,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-21T15:00:50.725566+0000","last_change":"2024-09-21T15:00:50.237991+0000","last_active":"2024-09-21T15:00:50.725566+0000","last_peered":"2024-09-21T15:00:50.725566+0000","last_clean":"2024-09-21T15:00:50.725566+0000","last_became_active":"2024-09-21T14:58:19.410306+0000","last_became_peered":"2024-09-21T14:58:19.410306+0000","last_unstale":"2024-09-21T15:00:50.725566+0000","last_undegraded":"2024-09-21T15:00:50.725566+0000","last_fullsized":"2024-09-21T15:00:50.725566+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":314,"log_dups_size":0,"ondisk_log_size":314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:33:50.089165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019021369999999999,"stat_sum":{"num_bytes":24400336,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":3317,"num_read_kb":27042,"num_write":526,"num_write_kb":31198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"65","length":"1"}]},{"pgid":"3.16","version":"163'867","reported_seq":4571,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-21T15:00:50.729465+0000","last_change":"2024-09-21T15:00:50.164355+0000","last_active":"2024-09-21T15:00:50.729465+0000","last_peered":"2024-09-21T15:00:50.729465+0000","last_clean":"2024-09-21T15:00:50.729465+0000","last_became_active":"2024-09-21T14:59:32.132898+0000","last_became_peered":"2024-09-21T14:59:32.132898+0000","last_unstale":"2024-09-21T15:00:50.729465+0000","last_undegraded":"2024-09-21T15:00:50.729465+0000","last_fullsized":"2024-09-21T15:00:50.729465+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":867,"log_dups_size":0,"ondisk_log_size":867,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:09:47.566958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092800300000000005,"stat_sum":{"num_bytes":31226659,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":6682,"num_read_kb":39228,"num_write":1354,"num_write_kb":24570,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":153,"num_bytes_recovered":9621881,"num_keys_recovered":33,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"65","length":"1"}]},{"pgid":"3.15","version":"163'691","reported_seq":3103,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-21T15:00:50.724485+0000","last_change":"2024-09-21T15:00:50.158719+0000","last_active":"2024-09-21T15:00:50.724485+0000","last_peered":"2024-09-21T15:00:50.724485+0000","last_clean":"2024-09-21T15:00:50.724485+0000","last_became_active":"2024-09-21T14:58:19.410687+0000","last_became_peered":"2024-09-21T14:58:19.410687+0000","last_unstale":"2024-09-21T15:00:50.724485+0000","last_undegraded":"2024-09-21T15:00:50.724485+0000","last_fullsized":"2024-09-21T15:00:50.724485+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":691,"log_dups_size":0,"ondisk_log_size":691,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:21:42.226982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059067799999999997,"stat_sum":{"num_bytes":34019198,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":4223,"num_read_kb":44489,"num_write":1159,"num_write_kb":31411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"65","length":"1"}]},{"pgid":"3.14","version":"163'410","reported_seq":2619,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-21T15:00:50.734949+0000","last_change":"2024-09-21T15:00:50.162834+0000","last_active":"2024-09-21T15:00:50.734949+0000","last_peered":"2024-09-21T15:00:50.734949+0000","last_clean":"2024-09-21T15:00:50.734949+0000","last_became_active":"2024-09-21T14:58:19.407310+0000","last_became_peered":"2024-09-21T14:58:19.407310+0000","last_unstale":"2024-09-21T15:00:50.734949+0000","last_undegraded":"2024-09-21T15:00:50.734949+0000","last_fullsized":"2024-09-21T15:00:50.734949+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":410,"log_dups_size":0,"ondisk_log_size":410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:47:46.079115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083343000000000002,"stat_sum":{"num_bytes":31456336,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":2536,"num_read_kb":34994,"num_write":554,"num_write_kb":29671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"65","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":157,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-21T15:00:49.456060+0000","last_change":"2024-09-21T14:58:11.348177+0000","last_active":"2024-09-21T15:00:49.456060+0000","last_peered":"2024-09-21T15:00:49.456060+0000","last_clean":"2024-09-21T15:00:49.456060+0000","last_became_active":"2024-09-21T14:58:09.328417+0000","last_became_peered":"2024-09-21T14:58:09.328417+0000","last_unstale":"2024-09-21T15:00:49.456060+0000","last_undegraded":"2024-09-21T15:00:49.456060+0000","last_fullsized":"2024-09-21T15:00:49.456060+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:52:22.671185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018917490000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"160'350","reported_seq":5284,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-21T15:00:50.154486+0000","last_change":"2024-09-21T15:00:50.154707+0000","last_active":"2024-09-21T15:00:50.154486+0000","last_peered":"2024-09-21T15:00:50.154486+0000","last_clean":"2024-09-21T15:00:50.154486+0000","last_became_active":"2024-09-21T14:59:32.131232+0000","last_became_peered":"2024-09-21T14:59:32.131232+0000","last_unstale":"2024-09-21T15:00:50.154486+0000","last_undegraded":"2024-09-21T15:00:50.154486+0000","last_fullsized":"2024-09-21T15:00:50.154486+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":350,"log_dups_size":0,"ondisk_log_size":350,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:31:04.421782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016542729999999999,"stat_sum":{"num_bytes":49535968,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":5120,"num_read_kb":51027,"num_write":463,"num_write_kb":25212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":3585713,"num_keys_recovered":9,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"65","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":82,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-21T15:00:49.451064+0000","last_change":"2024-09-21T14:59:28.228883+0000","last_active":"2024-09-21T15:00:49.451064+0000","last_peered":"2024-09-21T15:00:49.451064+0000","last_clean":"2024-09-21T15:00:49.451064+0000","last_became_active":"2024-09-21T14:59:28.228464+0000","last_became_peered":"2024-09-21T14:59:28.228464+0000","last_unstale":"2024-09-21T15:00:49.451064+0000","last_undegraded":"2024-09-21T15:00:49.451064+0000","last_fullsized":"2024-09-21T15:00:49.451064+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:00:26.310864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"163'763","reported_seq":3097,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-21T15:00:50.723125+0000","last_change":"2024-09-21T15:00:50.154106+0000","last_active":"2024-09-21T15:00:50.723125+0000","last_peered":"2024-09-21T15:00:50.723125+0000","last_clean":"2024-09-21T15:00:50.723125+0000","last_became_active":"2024-09-21T14:59:31.132581+0000","last_became_peered":"2024-09-21T14:59:31.132581+0000","last_unstale":"2024-09-21T15:00:50.723125+0000","last_undegraded":"2024-09-21T15:00:50.723125+0000","last_fullsized":"2024-09-21T15:00:50.723125+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":763,"log_dups_size":0,"ondisk_log_size":763,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:36:45.467515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001158931,"stat_sum":{"num_bytes":27113438,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":4660,"num_read_kb":28503,"num_write":1125,"num_write_kb":23454,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":119,"num_bytes_recovered":6871924,"num_keys_recovered":11,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"65","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":173,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-21T15:00:49.456578+0000","last_change":"2024-09-21T14:59:28.231502+0000","last_active":"2024-09-21T15:00:49.456578+0000","last_peered":"2024-09-21T15:00:49.456578+0000","last_clean":"2024-09-21T15:00:49.456578+0000","last_became_active":"2024-09-21T14:59:28.231043+0000","last_became_peered":"2024-09-21T14:59:28.231043+0000","last_unstale":"2024-09-21T15:00:49.456578+0000","last_undegraded":"2024-09-21T15:00:49.456578+0000","last_fullsized":"2024-09-21T15:00:49.456578+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:09:46.773690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019338770000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"163'529","reported_seq":4981,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-21T15:00:50.340778+0000","last_change":"2024-09-21T15:00:50.154207+0000","last_active":"2024-09-21T15:00:50.340778+0000","last_peered":"2024-09-21T15:00:50.340778+0000","last_clean":"2024-09-21T15:00:50.340778+0000","last_became_active":"2024-09-21T15:00:34.347238+0000","last_became_peered":"2024-09-21T15:00:34.347238+0000","last_unstale":"2024-09-21T15:00:50.340778+0000","last_undegraded":"2024-09-21T15:00:50.340778+0000","last_fullsized":"2024-09-21T15:00:50.340778+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":529,"log_dups_size":0,"ondisk_log_size":529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:31:33.639268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00194477,"stat_sum":{"num_bytes":39366483,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":5020,"num_read_kb":37583,"num_write":914,"num_write_kb":38851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":14049436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"65","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":158,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-21T15:00:49.450320+0000","last_change":"2024-09-21T14:58:11.406176+0000","last_active":"2024-09-21T15:00:49.450320+0000","last_peered":"2024-09-21T15:00:49.450320+0000","last_clean":"2024-09-21T15:00:49.450320+0000","last_became_active":"2024-09-21T14:58:09.777465+0000","last_became_peered":"2024-09-21T14:58:09.777465+0000","last_unstale":"2024-09-21T15:00:49.450320+0000","last_undegraded":"2024-09-21T15:00:49.450320+0000","last_fullsized":"2024-09-21T15:00:49.450320+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:55:04.667816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095704000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"160'649","reported_seq":2478,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-21T15:00:50.342377+0000","last_change":"2024-09-21T15:00:50.168975+0000","last_active":"2024-09-21T15:00:50.342377+0000","last_peered":"2024-09-21T15:00:50.342377+0000","last_clean":"2024-09-21T15:00:50.342377+0000","last_became_active":"2024-09-21T14:58:19.408971+0000","last_became_peered":"2024-09-21T14:58:19.408971+0000","last_unstale":"2024-09-21T15:00:50.342377+0000","last_undegraded":"2024-09-21T15:00:50.342377+0000","last_fullsized":"2024-09-21T15:00:50.342377+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":649,"log_dups_size":0,"ondisk_log_size":649,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:29:52.642121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051265399999999997,"stat_sum":{"num_bytes":19212230,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":3541,"num_read_kb":18046,"num_write":1071,"num_write_kb":19458,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"65","length":"1"}]},{"pgid":"3.3","version":"163'712","reported_seq":2782,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-21T15:00:50.723386+0000","last_change":"2024-09-21T15:00:50.157709+0000","last_active":"2024-09-21T15:00:50.723386+0000","last_peered":"2024-09-21T15:00:50.723386+0000","last_clean":"2024-09-21T15:00:50.723386+0000","last_became_active":"2024-09-21T14:58:19.417453+0000","last_became_peered":"2024-09-21T14:58:19.417453+0000","last_unstale":"2024-09-21T15:00:50.723386+0000","last_undegraded":"2024-09-21T15:00:50.723386+0000","last_fullsized":"2024-09-21T15:00:50.723386+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":712,"log_dups_size":0,"ondisk_log_size":712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:19:27.087272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092717099999999996,"stat_sum":{"num_bytes":14896983,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":4291,"num_read_kb":22918,"num_write":1189,"num_write_kb":19281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"65","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":159,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-21T15:00:49.455795+0000","last_change":"2024-09-21T14:58:11.347504+0000","last_active":"2024-09-21T15:00:49.455795+0000","last_peered":"2024-09-21T15:00:49.455795+0000","last_clean":"2024-09-21T15:00:49.455795+0000","last_became_active":"2024-09-21T14:58:09.775981+0000","last_became_peered":"2024-09-21T14:58:09.775981+0000","last_unstale":"2024-09-21T15:00:49.455795+0000","last_undegraded":"2024-09-21T15:00:49.455795+0000","last_fullsized":"2024-09-21T15:00:49.455795+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:47:16.286652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012568880000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"163'503","reported_seq":2875,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-21T15:00:50.720844+0000","last_change":"2024-09-21T15:00:50.236995+0000","last_active":"2024-09-21T15:00:50.720844+0000","last_peered":"2024-09-21T15:00:50.720844+0000","last_clean":"2024-09-21T15:00:50.720844+0000","last_became_active":"2024-09-21T14:58:19.405811+0000","last_became_peered":"2024-09-21T14:58:19.405811+0000","last_unstale":"2024-09-21T15:00:50.720844+0000","last_undegraded":"2024-09-21T15:00:50.720844+0000","last_fullsized":"2024-09-21T15:00:50.720844+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":503,"log_dups_size":0,"ondisk_log_size":503,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:36:25.551607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021337389999999999,"stat_sum":{"num_bytes":20016089,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":3745,"num_read_kb":25728,"num_write":653,"num_write_kb":24233,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"65","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":158,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-21T15:00:49.505941+0000","last_change":"2024-09-21T14:58:11.342210+0000","last_active":"2024-09-21T15:00:49.505941+0000","last_peered":"2024-09-21T15:00:49.505941+0000","last_clean":"2024-09-21T15:00:49.505941+0000","last_became_active":"2024-09-21T14:58:09.330624+0000","last_became_peered":"2024-09-21T14:58:09.330624+0000","last_unstale":"2024-09-21T15:00:49.505941+0000","last_undegraded":"2024-09-21T15:00:49.505941+0000","last_fullsized":"2024-09-21T15:00:49.505941+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:29:43.229538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001051151,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"159'263","reported_seq":1721,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-21T15:00:50.184955+0000","last_change":"2024-09-21T15:00:50.185155+0000","last_active":"2024-09-21T15:00:50.184955+0000","last_peered":"2024-09-21T15:00:50.184955+0000","last_clean":"2024-09-21T15:00:50.184955+0000","last_became_active":"2024-09-21T15:00:07.158207+0000","last_became_peered":"2024-09-21T15:00:07.158207+0000","last_unstale":"2024-09-21T15:00:50.184955+0000","last_undegraded":"2024-09-21T15:00:50.184955+0000","last_fullsized":"2024-09-21T15:00:50.184955+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":263,"log_dups_size":0,"ondisk_log_size":263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:36:40.720592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052533899999999999,"stat_sum":{"num_bytes":34528667,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":2346,"num_read_kb":38794,"num_write":393,"num_write_kb":38660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":150,"num_bytes_recovered":8136552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"65","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":157,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-21T15:00:49.455561+0000","last_change":"2024-09-21T14:58:11.346766+0000","last_active":"2024-09-21T15:00:49.455561+0000","last_peered":"2024-09-21T15:00:49.455561+0000","last_clean":"2024-09-21T15:00:49.455561+0000","last_became_active":"2024-09-21T14:58:09.329748+0000","last_became_peered":"2024-09-21T14:58:09.329748+0000","last_unstale":"2024-09-21T15:00:49.455561+0000","last_undegraded":"2024-09-21T15:00:49.455561+0000","last_fullsized":"2024-09-21T15:00:49.455561+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:31:20.267675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052698000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"163'547","reported_seq":5483,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-21T15:00:50.852018+0000","last_change":"2024-09-21T15:00:50.169884+0000","last_active":"2024-09-21T15:00:50.852018+0000","last_peered":"2024-09-21T15:00:50.852018+0000","last_clean":"2024-09-21T15:00:50.852018+0000","last_became_active":"2024-09-21T14:58:41.401003+0000","last_became_peered":"2024-09-21T14:58:41.401003+0000","last_unstale":"2024-09-21T15:00:50.852018+0000","last_undegraded":"2024-09-21T15:00:50.852018+0000","last_fullsized":"2024-09-21T15:00:50.852018+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":547,"log_dups_size":0,"ondisk_log_size":547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:05:37.644835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001445212,"stat_sum":{"num_bytes":42903203,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":0,"num_read":6739,"num_read_kb":41929,"num_write":706,"num_write_kb":18685,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1403348,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"65","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":158,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-21T15:00:49.456025+0000","last_change":"2024-09-21T14:58:11.347700+0000","last_active":"2024-09-21T15:00:49.456025+0000","last_peered":"2024-09-21T15:00:49.456025+0000","last_clean":"2024-09-21T15:00:49.456025+0000","last_became_active":"2024-09-21T14:58:09.438842+0000","last_became_peered":"2024-09-21T14:58:09.438842+0000","last_unstale":"2024-09-21T15:00:49.456025+0000","last_undegraded":"2024-09-21T15:00:49.456025+0000","last_fullsized":"2024-09-21T15:00:49.456025+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:41:34.626249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001426194,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":306,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-21T15:00:49.460101+0000","last_change":"2024-09-21T14:58:03.616055+0000","last_active":"2024-09-21T15:00:49.460101+0000","last_peered":"2024-09-21T15:00:49.460101+0000","last_clean":"2024-09-21T15:00:49.460101+0000","last_became_active":"2024-09-21T14:58:03.613214+0000","last_became_peered":"2024-09-21T14:58:03.613214+0000","last_unstale":"2024-09-21T15:00:49.460101+0000","last_undegraded":"2024-09-21T15:00:49.460101+0000","last_fullsized":"2024-09-21T15:00:49.460101+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:02.180162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:02.180162+0000","last_clean_scrub_stamp":"2024-09-21T14:58:02.180162+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:29:05.860361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"163'575","reported_seq":3607,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-21T15:00:50.717302+0000","last_change":"2024-09-21T15:00:50.152942+0000","last_active":"2024-09-21T15:00:50.717302+0000","last_peered":"2024-09-21T15:00:50.717302+0000","last_clean":"2024-09-21T15:00:50.717302+0000","last_became_active":"2024-09-21T14:59:31.131460+0000","last_became_peered":"2024-09-21T14:59:31.131460+0000","last_unstale":"2024-09-21T15:00:50.717302+0000","last_undegraded":"2024-09-21T15:00:50.717302+0000","last_fullsized":"2024-09-21T15:00:50.717302+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:01:38.353476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078960200000000003,"stat_sum":{"num_bytes":34215823,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":3978,"num_read_kb":38216,"num_write":946,"num_write_kb":26667,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":119,"num_bytes_recovered":6097618,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"65","length":"1"}]},{"pgid":"3.9","version":"163'750","reported_seq":2771,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-21T15:00:50.734129+0000","last_change":"2024-09-21T15:00:50.163903+0000","last_active":"2024-09-21T15:00:50.734129+0000","last_peered":"2024-09-21T15:00:50.734129+0000","last_clean":"2024-09-21T15:00:50.734129+0000","last_became_active":"2024-09-21T14:59:28.233956+0000","last_became_peered":"2024-09-21T14:59:28.233956+0000","last_unstale":"2024-09-21T15:00:50.734129+0000","last_undegraded":"2024-09-21T15:00:50.734129+0000","last_fullsized":"2024-09-21T15:00:50.734129+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":750,"log_dups_size":0,"ondisk_log_size":750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:29:09.682792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050756599999999999,"stat_sum":{"num_bytes":22901701,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":4365,"num_read_kb":30534,"num_write":1207,"num_write_kb":29668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":91,"num_bytes_recovered":6925016,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"65","length":"1"}]},{"pgid":"3.a","version":"163'655","reported_seq":3631,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-21T15:00:50.735201+0000","last_change":"2024-09-21T15:00:50.156493+0000","last_active":"2024-09-21T15:00:50.735201+0000","last_peered":"2024-09-21T15:00:50.735201+0000","last_clean":"2024-09-21T15:00:50.735201+0000","last_became_active":"2024-09-21T15:00:27.280971+0000","last_became_peered":"2024-09-21T15:00:27.280971+0000","last_unstale":"2024-09-21T15:00:50.735201+0000","last_undegraded":"2024-09-21T15:00:50.735201+0000","last_fullsized":"2024-09-21T15:00:50.735201+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":655,"log_dups_size":0,"ondisk_log_size":655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:26:07.617059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020403349999999999,"stat_sum":{"num_bytes":19873924,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":5020,"num_read_kb":27343,"num_write":904,"num_write_kb":30508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":9228123,"num_keys_recovered":33,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"65","length":"1"}]},{"pgid":"3.b","version":"163'421","reported_seq":2536,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-21T15:00:51.753518+0000","last_change":"2024-09-21T15:00:50.247513+0000","last_active":"2024-09-21T15:00:51.753518+0000","last_peered":"2024-09-21T15:00:51.753518+0000","last_clean":"2024-09-21T15:00:51.753518+0000","last_became_active":"2024-09-21T14:58:19.410679+0000","last_became_peered":"2024-09-21T14:58:19.410679+0000","last_unstale":"2024-09-21T15:00:51.753518+0000","last_undegraded":"2024-09-21T15:00:51.753518+0000","last_fullsized":"2024-09-21T15:00:51.753518+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":421,"log_dups_size":0,"ondisk_log_size":421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:07:40.758932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050536899999999998,"stat_sum":{"num_bytes":29184808,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":2502,"num_read_kb":32796,"num_write":647,"num_write_kb":35870,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"65","length":"1"}]},{"pgid":"3.c","version":"160'395","reported_seq":3943,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-21T15:00:50.342038+0000","last_change":"2024-09-21T15:00:50.157959+0000","last_active":"2024-09-21T15:00:50.342038+0000","last_peered":"2024-09-21T15:00:50.342038+0000","last_clean":"2024-09-21T15:00:50.342038+0000","last_became_active":"2024-09-21T14:59:25.130057+0000","last_became_peered":"2024-09-21T14:59:25.130057+0000","last_unstale":"2024-09-21T15:00:50.342038+0000","last_undegraded":"2024-09-21T15:00:50.342038+0000","last_fullsized":"2024-09-21T15:00:50.342038+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":395,"log_dups_size":0,"ondisk_log_size":395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:53:33.752550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001244345,"stat_sum":{"num_bytes":20607929,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":4010,"num_read_kb":23674,"num_write":547,"num_write_kb":23279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":9059600,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"65","length":"1"}]},{"pgid":"3.d","version":"163'397","reported_seq":4572,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-21T15:00:50.672578+0000","last_change":"2024-09-21T15:00:50.163372+0000","last_active":"2024-09-21T15:00:50.672578+0000","last_peered":"2024-09-21T15:00:50.672578+0000","last_clean":"2024-09-21T15:00:50.672578+0000","last_became_active":"2024-09-21T14:58:19.407884+0000","last_became_peered":"2024-09-21T14:58:19.407884+0000","last_unstale":"2024-09-21T15:00:50.672578+0000","last_undegraded":"2024-09-21T15:00:50.672578+0000","last_fullsized":"2024-09-21T15:00:50.672578+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":397,"log_dups_size":0,"ondisk_log_size":397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:28:57.111615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083893699999999999,"stat_sum":{"num_bytes":46838850,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":4490,"num_read_kb":51439,"num_write":657,"num_write_kb":33184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"65","length":"1"}]},{"pgid":"3.e","version":"160'753","reported_seq":2795,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-21T15:00:50.156757+0000","last_change":"2024-09-21T15:00:50.157221+0000","last_active":"2024-09-21T15:00:50.156757+0000","last_peered":"2024-09-21T15:00:50.156757+0000","last_clean":"2024-09-21T15:00:50.156757+0000","last_became_active":"2024-09-21T14:59:13.131825+0000","last_became_peered":"2024-09-21T14:59:13.131825+0000","last_unstale":"2024-09-21T15:00:50.156757+0000","last_undegraded":"2024-09-21T15:00:50.156757+0000","last_fullsized":"2024-09-21T15:00:50.156757+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":753,"log_dups_size":0,"ondisk_log_size":753,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:20:52.475349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083415300000000002,"stat_sum":{"num_bytes":11242088,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":4101,"num_read_kb":15729,"num_write":1242,"num_write_kb":15566,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":2113894,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"65","length":"1"}]},{"pgid":"3.f","version":"163'408","reported_seq":4724,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-21T15:00:50.720972+0000","last_change":"2024-09-21T15:00:50.157993+0000","last_active":"2024-09-21T15:00:50.720972+0000","last_peered":"2024-09-21T15:00:50.720972+0000","last_clean":"2024-09-21T15:00:50.720972+0000","last_became_active":"2024-09-21T14:58:19.410406+0000","last_became_peered":"2024-09-21T14:58:19.410406+0000","last_unstale":"2024-09-21T15:00:50.720972+0000","last_undegraded":"2024-09-21T15:00:50.720972+0000","last_fullsized":"2024-09-21T15:00:50.720972+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:59:16.026939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065308100000000002,"stat_sum":{"num_bytes":44844309,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":4671,"num_read_kb":50280,"num_write":647,"num_write_kb":33317,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"65","length":"1"}]},{"pgid":"3.10","version":"163'398","reported_seq":4105,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-21T15:00:50.720376+0000","last_change":"2024-09-21T15:00:50.155724+0000","last_active":"2024-09-21T15:00:50.720376+0000","last_peered":"2024-09-21T15:00:50.720376+0000","last_clean":"2024-09-21T15:00:50.720376+0000","last_became_active":"2024-09-21T14:58:19.409935+0000","last_became_peered":"2024-09-21T14:58:19.409935+0000","last_unstale":"2024-09-21T15:00:50.720376+0000","last_undegraded":"2024-09-21T15:00:50.720376+0000","last_fullsized":"2024-09-21T15:00:50.720376+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":398,"log_dups_size":0,"ondisk_log_size":398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:10:36.252629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093791100000000004,"stat_sum":{"num_bytes":48724507,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":3793,"num_read_kb":56674,"num_write":623,"num_write_kb":34859,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"65","length":"1"}]},{"pgid":"3.11","version":"163'384","reported_seq":3832,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-21T15:00:50.870664+0000","last_change":"2024-09-21T15:00:50.164129+0000","last_active":"2024-09-21T15:00:50.870664+0000","last_peered":"2024-09-21T15:00:50.870664+0000","last_clean":"2024-09-21T15:00:50.870664+0000","last_became_active":"2024-09-21T14:59:10.128001+0000","last_became_peered":"2024-09-21T14:59:10.128001+0000","last_unstale":"2024-09-21T15:00:50.870664+0000","last_undegraded":"2024-09-21T15:00:50.870664+0000","last_fullsized":"2024-09-21T15:00:50.870664+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":384,"log_dups_size":0,"ondisk_log_size":384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:44:22.770961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023350369999999999,"stat_sum":{"num_bytes":22947443,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":4297,"num_read_kb":26385,"num_write":456,"num_write_kb":22351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":3094046,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"65","length":"1"}]},{"pgid":"3.12","version":"163'417","reported_seq":5895,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-21T15:00:50.810184+0000","last_change":"2024-09-21T15:00:50.185264+0000","last_active":"2024-09-21T15:00:50.810184+0000","last_peered":"2024-09-21T15:00:50.810184+0000","last_clean":"2024-09-21T15:00:50.810184+0000","last_became_active":"2024-09-21T15:00:07.155144+0000","last_became_peered":"2024-09-21T15:00:07.155144+0000","last_unstale":"2024-09-21T15:00:50.810184+0000","last_undegraded":"2024-09-21T15:00:50.810184+0000","last_fullsized":"2024-09-21T15:00:50.810184+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":417,"log_dups_size":0,"ondisk_log_size":417,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:58:00.133639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057172199999999999,"stat_sum":{"num_bytes":33941311,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":6289,"num_read_kb":29149,"num_write":658,"num_write_kb":14946,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"65","length":"1"}]},{"pgid":"3.13","version":"163'581","reported_seq":5012,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-21T15:00:50.723704+0000","last_change":"2024-09-21T15:00:50.162538+0000","last_active":"2024-09-21T15:00:50.723704+0000","last_peered":"2024-09-21T15:00:50.723704+0000","last_clean":"2024-09-21T15:00:50.723704+0000","last_became_active":"2024-09-21T14:58:19.412030+0000","last_became_peered":"2024-09-21T14:58:19.412030+0000","last_unstale":"2024-09-21T15:00:50.723704+0000","last_undegraded":"2024-09-21T15:00:50.723704+0000","last_fullsized":"2024-09-21T15:00:50.723704+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":581,"log_dups_size":0,"ondisk_log_size":581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:44:29.282391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060185200000000005,"stat_sum":{"num_bytes":49472504,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":6302,"num_read_kb":54268,"num_write":767,"num_write_kb":26094,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"65","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":944120053,"num_objects":1192,"num_object_clones":0,"num_object_copies":2384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1192,"num_whiteouts":0,"num_read":133860,"num_read_kb":1088270,"num_write":23792,"num_write_kb":835722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1381,"num_bytes_recovered":103685771,"num_keys_recovered":152,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":60338176,"data_stored":537602427,"data_compressed":12198699,"data_compressed_allocated":48209920,"data_compressed_original":530836601,"omap_allocated":0,"internal_metadata":0},"log_size":15797,"ondisk_log_size":15797,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":10404,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509478,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33748,"kb_used_data":6732,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94338092,"statfs":{"total":96636764160,"available":96602206208,"internally_reserved":0,"allocated":6893568,"data_stored":44146124,"data_compressed":1005063,"data_compressed_allocated":3895296,"data_compressed_original":42627675,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:00:03 2024","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.84499999999999997,"15min":0.83899999999999997},"min":{"1min":0.504,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.95099999999999996,"15min":0.95799999999999996},"min":{"1min":0.51400000000000001,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.4970000000000001,"5min":1.917,"15min":1.917},"last":0.99399999999999999}]},{"osd":1,"last update":"Sat Sep 21 15:00:03 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.78500000000000003,"15min":0.76900000000000002},"min":{"1min":0.626,"5min":0.312,"15min":0.312},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.879,"5min":0.83499999999999996,"15min":0.82699999999999996},"min":{"1min":0.53600000000000003,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.224,"5min":2.2149999999999999,"15min":2.2149999999999999},"last":0.61699999999999999}]},{"osd":2,"last update":"Sat Sep 21 15:00:03 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.81699999999999995,"15min":0.81100000000000005},"min":{"1min":0.495,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.1859999999999999,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.95099999999999996,"15min":0.94799999999999995},"min":{"1min":0.72199999999999998,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.3300000000000001,"5min":1.466,"15min":1.466},"last":0.55600000000000005}]},{"osd":3,"last update":"Sat Sep 21 15:00:03 2024","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.86199999999999999,"15min":0.85399999999999998},"min":{"1min":0.67200000000000004,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.3120000000000001,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":0.94799999999999995},{"interface":"front","average":{"1min":1.0429999999999999,"5min":0.97999999999999998,"15min":0.96999999999999997},"min":{"1min":0.68500000000000005,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.244,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.68899999999999995}]},{"osd":4,"last update":"Sat Sep 21 15:00:03 2024","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.93700000000000006,"15min":0.93799999999999994},"min":{"1min":0.64200000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.4510000000000001,"5min":2.7130000000000001,"15min":2.7130000000000001},"last":1.042},{"interface":"front","average":{"1min":1.0369999999999999,"5min":0.94099999999999995,"15min":0.92500000000000004},"min":{"1min":0.59999999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.6499999999999999,"5min":2.125,"15min":2.125},"last":1.1259999999999999}]},{"osd":5,"last update":"Sat Sep 21 15:00:03 2024","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.0409999999999999,"15min":1.0269999999999999},"min":{"1min":0.71099999999999997,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.4770000000000001,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":0.86099999999999999},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.0640000000000001,"15min":1.0580000000000001},"min":{"1min":0.70799999999999996,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.8169999999999999,"5min":3.1880000000000002,"15min":3.1880000000000002},"last":1.1779999999999999}]},{"osd":6,"last update":"Sat Sep 21 15:00:03 2024","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.163,"15min":1.159},"min":{"1min":0.71299999999999997,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.6579999999999999,"5min":3.2730000000000001,"15min":3.2730000000000001},"last":1.0660000000000001},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.0469999999999999,"15min":1.0249999999999999},"min":{"1min":0.79100000000000004,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.6180000000000001,"5min":2.2879999999999998,"15min":2.2879999999999998},"last":1.1020000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542181,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62048,"kb_used_data":8788,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94309792,"statfs":{"total":96636764160,"available":96573227008,"internally_reserved":0,"allocated":8998912,"data_stored":63020529,"data_compressed":1430221,"data_compressed_allocated":5623808,"data_compressed_original":61282265,"omap_allocated":1593,"internal_metadata":54524359},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:00:06 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.94399999999999995,"15min":0.95699999999999996},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.244,"5min":1.593,"15min":1.593},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.93600000000000005,"15min":0.95099999999999996},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.3360000000000001,"5min":1.587,"15min":1.587},"last":0.98499999999999999}]},{"osd":1,"last update":"Sat Sep 21 15:00:06 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.90400000000000003,"15min":0.91400000000000003},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.1120000000000001,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.95699999999999996,"15min":0.97999999999999998},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.1819999999999999,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.629}]},{"osd":2,"last update":"Sat Sep 21 15:00:06 2024","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.98499999999999999,"15min":0.99099999999999999},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.3560000000000001,"5min":1.758,"15min":1.758},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.89900000000000002,"15min":0.90900000000000003},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.3069999999999999,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":1.008}]},{"osd":3,"last update":"Sat Sep 21 15:00:06 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.94499999999999995,"15min":0.95799999999999996},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.169,"5min":1.835,"15min":1.835},"last":0.96099999999999997},{"interface":"front","average":{"1min":1.012,"5min":1.1180000000000001,"15min":1.1359999999999999},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.4430000000000001,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.876}]},{"osd":4,"last update":"Sat Sep 21 15:00:06 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.97699999999999998,"15min":1.0049999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.1379999999999999,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.93999999999999995,"5min":1.02,"15min":1.0329999999999999},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.411,"5min":1.897,"15min":1.897},"last":0.78200000000000003}]},{"osd":5,"last update":"Sat Sep 21 15:00:06 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":1.0860000000000001,"15min":1.1240000000000001},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.2589999999999999,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.75600000000000001},{"interface":"front","average":{"1min":1.01,"5min":1.0800000000000001,"15min":1.0920000000000001},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.3779999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":0.90600000000000003}]},{"osd":7,"last update":"Sat Sep 21 15:00:06 2024","interfaces":[{"interface":"back","average":{"1min":1.123,"5min":1.2350000000000001,"15min":1.2529999999999999},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.4530000000000001,"5min":1.8660000000000001,"15min":1.8660000000000001},"last":1.0309999999999999},{"interface":"front","average":{"1min":1.044,"5min":1.26,"15min":1.296},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.385,"5min":2.004,"15min":2.004},"last":0.83499999999999996}]}]},{"osd":5,"up_from":14,"seq":60129542182,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39972,"kb_used_data":8848,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94331868,"statfs":{"total":96636764160,"available":96595832832,"internally_reserved":0,"allocated":9060352,"data_stored":62282461,"data_compressed":1255227,"data_compressed_allocated":5472256,"data_compressed_original":60401636,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:00:01 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.96799999999999997,"15min":1.0109999999999999},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.98999999999999999,"5min":4.4800000000000004,"15min":4.4800000000000004},"last":1.337},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.80800000000000005,"15min":0.82099999999999995},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.0329999999999999,"5min":1.181,"15min":1.181},"last":0.67700000000000005}]},{"osd":1,"last update":"Sat Sep 21 15:00:01 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.91000000000000003,"15min":0.92800000000000005},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.165,"5min":1.75,"15min":1.75},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.91900000000000004,"15min":0.93500000000000005},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.216,"5min":2.2400000000000002,"15min":2.2400000000000002},"last":0.92300000000000004}]},{"osd":2,"last update":"Sat Sep 21 15:00:01 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.95599999999999996,"15min":0.98799999999999999},"min":{"1min":0.54100000000000004,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.016,"5min":3.419,"15min":3.419},"last":1.0760000000000001},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.88500000000000001,"15min":0.90100000000000002},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.075,"5min":1.423,"15min":1.423},"last":0.96299999999999997}]},{"osd":3,"last update":"Sat Sep 21 15:00:01 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.80100000000000005,"15min":0.80700000000000005},"min":{"1min":0.54800000000000004,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.016,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":1.0069999999999999,"5min":1.081,"15min":1.093},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":2.6379999999999999,"5min":2.6379999999999999,"15min":2.6379999999999999},"last":1.272}]},{"osd":4,"last update":"Sat Sep 21 15:00:01 2024","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":1.0129999999999999,"15min":1.024},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":2.5470000000000002,"5min":2.5470000000000002,"15min":2.5470000000000002},"last":1.2230000000000001},{"interface":"front","average":{"1min":0.93999999999999995,"5min":1.075,"15min":1.097},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":2.5099999999999998,"5min":2.5099999999999998,"15min":2.5099999999999998},"last":1.149}]},{"osd":6,"last update":"Sat Sep 21 15:00:01 2024","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.95699999999999996,"15min":0.97299999999999998},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.1539999999999999,"5min":1.244,"15min":1.244},"last":1.0269999999999999},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.325,"15min":1.379},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":2.6789999999999998,"5min":4.2469999999999999,"15min":4.2469999999999999},"last":1.4219999999999999}]},{"osd":7,"last update":"Sat Sep 21 15:00:05 2024","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.3200000000000001,"15min":1.363},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":2.7189999999999999,"5min":3.3010000000000002,"15min":3.3010000000000002},"last":1.383},{"interface":"front","average":{"1min":1.0740000000000001,"5min":1.1339999999999999,"15min":1.1439999999999999},"min":{"1min":0.64200000000000002,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":2.5880000000000001,"5min":2.5880000000000001,"15min":2.5880000000000001},"last":1.1839999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542181,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63396,"kb_used_data":10128,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94308444,"statfs":{"total":96636764160,"available":96571846656,"internally_reserved":0,"allocated":10371072,"data_stored":82901007,"data_compressed":1730744,"data_compressed_allocated":7057408,"data_compressed_original":81226539,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:00:04 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.76900000000000002,"15min":0.77900000000000003},"min":{"1min":0.36399999999999999,"5min":0.311,"15min":0.311},"max":{"1min":0.96799999999999997,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.92100000000000004,"15min":0.93799999999999994},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.486,"5min":1.486,"15min":1.486},"last":0.49299999999999999}]},{"osd":1,"last update":"Sat Sep 21 15:00:04 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.89000000000000001,"15min":0.91600000000000004},"min":{"1min":0.48099999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.92300000000000004,"5min":1.347,"15min":1.347},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.83899999999999997,"15min":0.84499999999999997},"min":{"1min":0.39600000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.0720000000000001,"5min":1.153,"15min":1.153},"last":0.52500000000000002}]},{"osd":2,"last update":"Sat Sep 21 15:00:04 2024","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.875,"15min":0.877},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.268,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.91600000000000004,"5min":1.0129999999999999,"15min":1.0289999999999999},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.258,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.86299999999999999}]},{"osd":3,"last update":"Sat Sep 21 15:00:04 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.89300000000000002,"15min":0.89000000000000001},"min":{"1min":0.68100000000000005,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.407,"5min":1.538,"15min":1.538},"last":0.61799999999999999},{"interface":"front","average":{"1min":1.095,"5min":1.004,"15min":0.98899999999999999},"min":{"1min":0.63100000000000001,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.365,"5min":1.365,"15min":1.365},"last":0.80600000000000005}]},{"osd":5,"last update":"Sat Sep 21 15:00:04 2024","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.97899999999999998,"15min":0.98499999999999999},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.2210000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.872,"5min":0.95199999999999996,"15min":0.96499999999999997},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.1619999999999999,"5min":1.337,"15min":1.337},"last":0.66300000000000003}]},{"osd":6,"last update":"Sat Sep 21 15:00:04 2024","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":1.0009999999999999,"15min":1.0129999999999999},"min":{"1min":0.51400000000000001,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.2210000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.55400000000000005},{"interface":"front","average":{"1min":1.077,"5min":1.091,"15min":1.093},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.5449999999999999,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":0.97099999999999997}]},{"osd":7,"last update":"Sat Sep 21 15:00:04 2024","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":1.022,"15min":1.034},"min":{"1min":0.46000000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.3859999999999999,"5min":1.7010000000000001,"15min":1.7010000000000001},"last":0.45200000000000001},{"interface":"front","average":{"1min":1.202,"5min":1.214,"15min":1.216},"min":{"1min":0.78800000000000003,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.4590000000000001,"5min":1.53,"15min":1.53},"last":0.627}]}]},{"osd":3,"up_from":14,"seq":60129542182,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68640,"kb_used_data":11280,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94303200,"statfs":{"total":96636764160,"available":96566476800,"internally_reserved":0,"allocated":11550720,"data_stored":85523600,"data_compressed":1879481,"data_compressed_allocated":7553024,"data_compressed_original":83394561,"omap_allocated":1592,"internal_metadata":58718664},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:00:05 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.378,"5min":0.315,"15min":0.315},"max":{"1min":1.0109999999999999,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.71199999999999997,"15min":0.71999999999999997},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.89200000000000002,"5min":1.919,"15min":1.919},"last":0.63300000000000001}]},{"osd":1,"last update":"Sat Sep 21 15:00:05 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.67500000000000004,"15min":0.67600000000000005},"min":{"1min":0.33400000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":1.1240000000000001,"5min":1.135,"15min":1.135},"last":1.0529999999999999},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.80700000000000005,"15min":0.81699999999999995},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.1599999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.59899999999999998}]},{"osd":2,"last update":"Sat Sep 21 15:00:05 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.72799999999999998,"15min":0.72399999999999998},"min":{"1min":0.40200000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.83399999999999996,"15min":0.83499999999999996},"min":{"1min":0.47499999999999998,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.1759999999999999,"5min":1.254,"15min":1.254},"last":0.83299999999999996}]},{"osd":4,"last update":"Sat Sep 21 15:00:05 2024","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.95699999999999996,"15min":0.96299999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.2709999999999999,"5min":1.7609999999999999,"15min":1.7609999999999999},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.91100000000000003,"5min":1.0189999999999999,"15min":1.0369999999999999},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.25,"5min":2.028,"15min":2.028},"last":1.079}]},{"osd":5,"last update":"Sat Sep 21 15:00:05 2024","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.052,"15min":1.0569999999999999},"min":{"1min":0.77700000000000002,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.246,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":0.89200000000000002},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.0649999999999999,"15min":1.0660000000000001},"min":{"1min":0.70799999999999996,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.3600000000000001,"5min":1.431,"15min":1.431},"last":0.97399999999999998}]},{"osd":6,"last update":"Sat Sep 21 15:00:05 2024","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.0860000000000001,"15min":1.0980000000000001},"min":{"1min":0.58599999999999997,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.3879999999999999,"5min":2.1219999999999999,"15min":2.1219999999999999},"last":0.95099999999999996},{"interface":"front","average":{"1min":1.046,"5min":1.097,"15min":1.105},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.7370000000000001,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":1.1180000000000001}]},{"osd":7,"last update":"Sat Sep 21 15:00:05 2024","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.1279999999999999,"15min":1.139},"min":{"1min":0.61799999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.4870000000000001,"5min":1.706,"15min":1.706},"last":1.161},{"interface":"front","average":{"1min":1.093,"5min":1.173,"15min":1.1859999999999999},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.516,"5min":2.3380000000000001,"15min":2.3380000000000001},"last":1.0289999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542180,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55696,"kb_used_data":6528,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94316144,"statfs":{"total":96636764160,"available":96579731456,"internally_reserved":0,"allocated":6684672,"data_stored":40644282,"data_compressed":1122861,"data_compressed_allocated":3788800,"data_compressed_original":39196227,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:00:00 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.62,"15min":0.62},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.97999999999999998,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.66100000000000003,"15min":0.65900000000000003},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":1.0429999999999999,"5min":1.244,"15min":1.244},"last":0.90000000000000002}]},{"osd":1,"last update":"Sat Sep 21 15:00:00 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.69699999999999995,"15min":0.68799999999999994},"min":{"1min":0.32400000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":2.6520000000000001,"5min":2.6520000000000001,"15min":2.6520000000000001},"last":0.63},{"interface":"front","average":{"1min":0.622,"5min":0.73999999999999999,"15min":0.76000000000000001},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.89300000000000002,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.76600000000000001}]},{"osd":3,"last update":"Sat Sep 21 15:00:00 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.72599999999999998,"15min":0.72999999999999998},"min":{"1min":0.38900000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.113,"5min":2.1259999999999999,"15min":2.1259999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":1.1359999999999999,"5min":0.88400000000000001,"15min":0.84199999999999997},"min":{"1min":0.52600000000000002,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":4.1440000000000001,"5min":4.1440000000000001,"15min":4.1440000000000001},"last":1.05}]},{"osd":4,"last update":"Sat Sep 21 15:00:00 2024","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.96199999999999997,"15min":0.96599999999999997},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":2.899,"5min":2.899,"15min":2.899},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.95199999999999996,"15min":0.95999999999999996},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.1679999999999999,"5min":1.675,"15min":1.675},"last":1.0229999999999999}]},{"osd":5,"last update":"Sat Sep 21 15:00:00 2024","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":1.0049999999999999,"15min":1.0069999999999999},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":2.8450000000000002,"5min":2.8450000000000002,"15min":2.8450000000000002},"last":1.0860000000000001},{"interface":"front","average":{"1min":1.002,"5min":1.0009999999999999,"15min":1.0009999999999999},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.502,"5min":1.546,"15min":1.546},"last":0.95199999999999996}]},{"osd":6,"last update":"Sat Sep 21 15:00:00 2024","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.97899999999999998,"15min":0.98399999999999999},"min":{"1min":0.40100000000000002,"5min":0.378,"15min":0.378},"max":{"1min":1.153,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":0.873},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.05,"15min":1.05},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.399,"5min":1.446,"15min":1.446},"last":1.1930000000000001}]},{"osd":7,"last update":"Sat Sep 21 15:00:07 2024","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":1.0029999999999999,"15min":0.995},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.99199999999999999},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.097,"15min":1.091},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.5489999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":1.1479999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542181,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46512,"kb_used_data":11296,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94325328,"statfs":{"total":96636764160,"available":96589135872,"internally_reserved":0,"allocated":11567104,"data_stored":90663235,"data_compressed":1823995,"data_compressed_allocated":7794688,"data_compressed_original":88693368,"omap_allocated":1595,"internal_metadata":36043205},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:00:05 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.60299999999999998,"15min":0.61099999999999999},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.83599999999999997,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.60099999999999998,"15min":0.60899999999999999},"min":{"1min":0.252,"5min":0.216,"15min":0.216},"max":{"1min":0.90800000000000003,"5min":1.107,"15min":1.107},"last":0.67600000000000005}]},{"osd":2,"last update":"Sat Sep 21 15:00:05 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.68999999999999995,"15min":0.70499999999999996},"min":{"1min":0.313,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":1.399,"5min":2.5529999999999999,"15min":2.5529999999999999},"last":11.303000000000001},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69599999999999995,"15min":0.69699999999999995},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.95499999999999996,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.47299999999999998}]},{"osd":3,"last update":"Sat Sep 21 15:00:05 2024","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.84399999999999997,"15min":0.84799999999999998},"min":{"1min":0.311,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":3.1469999999999998,"5min":3.1469999999999998,"15min":3.1469999999999998},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.88700000000000001,"15min":0.89300000000000002},"min":{"1min":0.47199999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":3.2290000000000001,"5min":3.2320000000000002,"15min":3.2320000000000002},"last":0.53300000000000003}]},{"osd":4,"last update":"Sat Sep 21 15:00:05 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.89400000000000002,"15min":0.91600000000000004},"min":{"1min":0.38600000000000001,"5min":0.309,"15min":0.309},"max":{"1min":1.1950000000000001,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":0.78900000000000003},{"interface":"front","average":{"1min":1.0409999999999999,"5min":0.93999999999999995,"15min":0.92300000000000004},"min":{"1min":0.54200000000000004,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":4.9409999999999998,"5min":4.9409999999999998,"15min":4.9409999999999998},"last":11.563000000000001}]},{"osd":5,"last update":"Sat Sep 21 15:00:05 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.90200000000000002,"15min":0.90100000000000002},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":11.534000000000001},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.95899999999999996,"15min":0.96099999999999997},"min":{"1min":0.629,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.63}]},{"osd":6,"last update":"Sat Sep 21 15:00:05 2024","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":1.026,"15min":1.054},"min":{"1min":0.42199999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.3959999999999999,"5min":2.3519999999999999,"15min":2.3519999999999999},"last":0.96799999999999997},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.94499999999999995,"15min":0.94699999999999995},"min":{"1min":0.58799999999999997,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.494,"5min":1.494,"15min":1.494},"last":11.601000000000001}]},{"osd":7,"last update":"Sat Sep 21 15:00:05 2024","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":1.0429999999999999,"15min":1.0629999999999999},"min":{"1min":0.48999999999999999,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.4590000000000001,"5min":2.2360000000000002,"15min":2.2360000000000002},"last":1.0449999999999999},{"interface":"front","average":{"1min":0.96399999999999997,"5min":1.028,"15min":1.0389999999999999},"min":{"1min":0.66500000000000004,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.5369999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":11.489000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542180,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64280,"kb_used_data":11016,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94307560,"statfs":{"total":96636764160,"available":96570941440,"internally_reserved":0,"allocated":11280384,"data_stored":85171383,"data_compressed":1898352,"data_compressed_allocated":7524352,"data_compressed_original":83224690,"omap_allocated":1593,"internal_metadata":54524359},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 21 15:00:04 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.69699999999999995,"15min":0.72499999999999998},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.97599999999999998,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.62,"15min":0.60699999999999998},"min":{"1min":0.42399999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":1.012}]},{"osd":2,"last update":"Sat Sep 21 15:00:04 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.61399999999999999,"15min":0.60599999999999998},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.82999999999999996,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.74199999999999999,"15min":0.76400000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.94699999999999995,"5min":1.204,"15min":1.204},"last":0.58399999999999996}]},{"osd":3,"last update":"Sat Sep 21 15:00:04 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.747,"15min":0.748},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.79500000000000004,"15min":0.81799999999999995},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.95699999999999996,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.81399999999999995}]},{"osd":4,"last update":"Sat Sep 21 15:00:04 2024","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":1.036,"15min":1.0549999999999999},"min":{"1min":0.56100000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.7,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":0.96099999999999997},{"interface":"front","average":{"1min":0.99299999999999999,"5min":1.054,"15min":1.0640000000000001},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.373,"5min":1.665,"15min":1.665},"last":1.446}]},{"osd":5,"last update":"Sat Sep 21 15:00:04 2024","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":1.012,"15min":1.028},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.204,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":1.3120000000000001},{"interface":"front","average":{"1min":0.875,"5min":1.05,"15min":1.079},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.319,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":1.0580000000000001}]},{"osd":6,"last update":"Sat Sep 21 15:00:04 2024","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":1.0980000000000001,"15min":1.1259999999999999},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.621,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":1.145},{"interface":"front","average":{"1min":0.99399999999999999,"5min":1.0549999999999999,"15min":1.0649999999999999},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.321,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":1.21}]},{"osd":7,"last update":"Sat Sep 21 15:00:04 2024","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":1.052,"15min":1.0640000000000001},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.2769999999999999,"5min":1.323,"15min":1.323},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.94699999999999995,"5min":1.1120000000000001,"15min":1.1399999999999999},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.6719999999999999,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":1.403}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5202,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5202,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":9252864,"data_stored":82464744,"data_compressed":1922920,"data_compressed_allocated":7434240,"data_compressed_original":81458038,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9408512,"data_stored":86954520,"data_compressed":1805675,"data_compressed_allocated":7606272,"data_compressed_original":85944952,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4685824,"data_stored":38858712,"data_compressed":1132706,"data_compressed_allocated":3751936,"data_compressed_original":38360643,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":9506816,"data_stored":82872996,"data_compressed":1857029,"data_compressed_allocated":7438336,"data_compressed_original":81677061,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8372224,"data_stored":80715498,"data_compressed":1756754,"data_compressed_allocated":7004160,"data_compressed_original":79977259,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":7073792,"data_stored":60795652,"data_compressed":1257803,"data_compressed_allocated":5447680,"data_compressed_original":59869156,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":7036928,"data_stored":61681985,"data_compressed":1429988,"data_compressed_allocated":5595136,"data_compressed_original":60872665,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":5001216,"data_stored":43258320,"data_compressed":1035824,"data_compressed_allocated":3932160,"data_compressed_original":42676827,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T15:00:53.344 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:00:53.339+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:00:53.364 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:53.364 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:00:53.364 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T15:00:53.364 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:53.374 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T15:00:53.445 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:00:53.441+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:00:53.483 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:53.483 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:00:53.483 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T15:00:53.483 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T15:00:53.483 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:53.493 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T15:00:53.546 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:00:53.541+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:00:53.605 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:53.605 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T15:00:53.605 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T15:00:53.606 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T15:00:53.606 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.606 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45839 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:53.606 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159313+0000", 2024-09-21T15:00:53.606 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 103.44253972, 2024-09-21T15:00:53.606 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068238699, 2024-09-21T15:00:53.606 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:53.606 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:53.606 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:53.606 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:53.606 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:53.606 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45839 2024-09-21T15:00:53.607 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.607 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:53.607 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.607 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:53.607 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159313+0000", 2024-09-21T15:00:53.607 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:53.607 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.607 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.607 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:53.607 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159313+0000", 2024-09-21T15:00:53.607 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:53.608 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.608 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.608 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:53.608 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159313+0000", 2024-09-21T15:00:53.608 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:53.608 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.608 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.608 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:53.608 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159316+0000", 2024-09-21T15:00:53.608 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.033e-06 2024-09-21T15:00:53.608 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.608 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.608 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:53.608 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159322+0000", 2024-09-21T15:00:53.609 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1820000000000003e-06 2024-09-21T15:00:53.609 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.609 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.609 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:53.609 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159327+0000", 2024-09-21T15:00:53.609 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.062e-06 2024-09-21T15:00:53.609 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.609 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.609 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:53.609 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227544+0000", 2024-09-21T15:00:53.609 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068217142999999994 2024-09-21T15:00:53.609 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.609 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.610 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:53.610 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227552+0000", 2024-09-21T15:00:53.610 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6260000000000003e-06 2024-09-21T15:00:53.610 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:53.610 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:53.610 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:53.610 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.610 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.610 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45840 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:53.610 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159332+0000", 2024-09-21T15:00:53.610 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 103.442521423, 2024-09-21T15:00:53.610 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068262046000000007, 2024-09-21T15:00:53.610 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:53.611 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:53.611 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:53.611 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:53.611 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:53.611 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45840 2024-09-21T15:00:53.611 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.611 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:53.611 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.611 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:53.611 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159332+0000", 2024-09-21T15:00:53.611 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:53.611 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.612 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.612 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:53.612 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159332+0000", 2024-09-21T15:00:53.612 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:53.612 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.612 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.612 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:53.612 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159331+0000", 2024-09-21T15:00:53.612 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:53.612 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.612 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.612 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:53.612 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159351+0000", 2024-09-21T15:00:53.612 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0064999999999998e-05 2024-09-21T15:00:53.613 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.613 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.613 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:53.613 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159367+0000", 2024-09-21T15:00:53.613 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5308e-05 2024-09-21T15:00:53.613 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.613 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.613 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:53.613 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159373+0000", 2024-09-21T15:00:53.613 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1840000000000004e-06 2024-09-21T15:00:53.613 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.613 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.613 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:53.613 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227584+0000", 2024-09-21T15:00:53.614 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068211401000000005 2024-09-21T15:00:53.614 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.614 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.614 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:53.614 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227594+0000", 2024-09-21T15:00:53.614 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4120000000000007e-06 2024-09-21T15:00:53.614 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:53.614 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:53.614 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:53.614 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.614 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.614 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45841 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:53.615 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159379+0000", 2024-09-21T15:00:53.615 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 103.44247426699999, 2024-09-21T15:00:53.615 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068249039999999997, 2024-09-21T15:00:53.615 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:53.615 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:53.615 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:53.615 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:53.615 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:53.615 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45841 2024-09-21T15:00:53.615 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.615 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:53.615 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.615 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:53.615 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159379+0000", 2024-09-21T15:00:53.616 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:53.616 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.616 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.616 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:53.616 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159379+0000", 2024-09-21T15:00:53.616 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:53.616 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.616 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.616 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:53.616 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159378+0000", 2024-09-21T15:00:53.616 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:53.616 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.616 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.616 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:53.616 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159382+0000", 2024-09-21T15:00:53.617 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.641e-06 2024-09-21T15:00:53.617 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.617 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.617 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:53.617 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159421+0000", 2024-09-21T15:00:53.617 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9439e-05 2024-09-21T15:00:53.617 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.617 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.617 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:53.617 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159430+0000", 2024-09-21T15:00:53.617 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8429999999999999e-06 2024-09-21T15:00:53.617 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.617 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.617 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:53.617 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227617+0000", 2024-09-21T15:00:53.618 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068187084999999995 2024-09-21T15:00:53.618 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.618 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.618 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:53.618 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227628+0000", 2024-09-21T15:00:53.618 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0507e-05 2024-09-21T15:00:53.618 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:53.618 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:53.618 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:53.618 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.618 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.619 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45844 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:53.619 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159499+0000", 2024-09-21T15:00:53.619 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 103.442353881, 2024-09-21T15:00:53.619 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068255415, 2024-09-21T15:00:53.619 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:53.619 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:53.619 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:53.619 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:53.619 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:53.619 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45844 2024-09-21T15:00:53.619 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.619 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:53.619 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.619 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:53.620 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159499+0000", 2024-09-21T15:00:53.620 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:53.620 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.620 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.620 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:53.620 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159499+0000", 2024-09-21T15:00:53.620 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:53.620 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.620 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.620 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:53.620 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159499+0000", 2024-09-21T15:00:53.620 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:53.620 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.620 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.620 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:53.621 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159503+0000", 2024-09-21T15:00:53.621 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0999999999999997e-06 2024-09-21T15:00:53.621 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.621 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.621 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:53.621 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159512+0000", 2024-09-21T15:00:53.621 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5850000000000006e-06 2024-09-21T15:00:53.621 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.621 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.621 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:53.621 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159518+0000", 2024-09-21T15:00:53.621 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0909999999999998e-06 2024-09-21T15:00:53.621 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.622 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.622 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:53.622 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227743+0000", 2024-09-21T15:00:53.622 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068224536000000002 2024-09-21T15:00:53.622 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.622 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.622 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:53.622 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227755+0000", 2024-09-21T15:00:53.622 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1659e-05 2024-09-21T15:00:53.622 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:53.622 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:53.622 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:53.623 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.623 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.623 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45845 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:53.623 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159524+0000", 2024-09-21T15:00:53.623 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 103.442329521, 2024-09-21T15:00:53.623 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068267194000000003, 2024-09-21T15:00:53.623 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:53.623 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:53.623 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:53.623 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:53.623 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:53.623 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45845 2024-09-21T15:00:53.623 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.623 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:53.624 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.624 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:53.624 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159524+0000", 2024-09-21T15:00:53.624 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:53.624 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.624 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.624 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:53.624 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159524+0000", 2024-09-21T15:00:53.624 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:53.624 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.624 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.624 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:53.624 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159523+0000", 2024-09-21T15:00:53.624 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:53.625 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.625 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.625 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:53.625 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159526+0000", 2024-09-21T15:00:53.625 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1980000000000001e-06 2024-09-21T15:00:53.625 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.625 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.625 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:53.625 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159533+0000", 2024-09-21T15:00:53.625 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.348e-06 2024-09-21T15:00:53.625 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.625 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.625 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:53.625 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159546+0000", 2024-09-21T15:00:53.625 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3464e-05 2024-09-21T15:00:53.626 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.626 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.626 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:53.626 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227783+0000", 2024-09-21T15:00:53.626 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068236926000000003 2024-09-21T15:00:53.626 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.626 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.626 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:53.626 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227791+0000", 2024-09-21T15:00:53.626 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6669999999999996e-06 2024-09-21T15:00:53.626 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:53.626 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:53.626 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:53.627 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.627 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.627 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45846 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:53.627 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159584+0000", 2024-09-21T15:00:53.627 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 103.442268879, 2024-09-21T15:00:53.627 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068254767999999993, 2024-09-21T15:00:53.627 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:53.627 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:53.627 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:53.627 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:53.627 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:53.627 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45846 2024-09-21T15:00:53.627 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.627 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:53.628 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.628 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:53.628 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159584+0000", 2024-09-21T15:00:53.628 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:53.628 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.628 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.628 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:53.628 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159584+0000", 2024-09-21T15:00:53.628 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:53.628 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.628 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.628 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:53.628 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159583+0000", 2024-09-21T15:00:53.628 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:53.628 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.629 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.629 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:53.629 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159588+0000", 2024-09-21T15:00:53.629 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1330000000000001e-06 2024-09-21T15:00:53.629 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.629 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.629 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:53.629 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159596+0000", 2024-09-21T15:00:53.629 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5469999999999997e-06 2024-09-21T15:00:53.629 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.629 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.629 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:53.629 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159609+0000", 2024-09-21T15:00:53.629 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2785999999999999e-05 2024-09-21T15:00:53.629 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.630 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.630 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:53.630 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227830+0000", 2024-09-21T15:00:53.630 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068220879999999998 2024-09-21T15:00:53.630 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.630 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.630 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:53.630 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227839+0000", 2024-09-21T15:00:53.630 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1239999999999994e-06 2024-09-21T15:00:53.630 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:53.630 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:53.630 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:53.630 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.630 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.631 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45847 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:53.631 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159614+0000", 2024-09-21T15:00:53.631 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 103.442238803, 2024-09-21T15:00:53.631 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068260639999999997, 2024-09-21T15:00:53.631 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:53.631 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:53.631 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:53.631 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:53.631 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:53.631 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45847 2024-09-21T15:00:53.631 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.631 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:53.631 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.631 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:53.632 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159614+0000", 2024-09-21T15:00:53.632 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:53.632 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.632 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.632 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:53.632 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159614+0000", 2024-09-21T15:00:53.632 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:53.632 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.632 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.632 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:53.632 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159614+0000", 2024-09-21T15:00:53.632 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:53.632 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.632 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.632 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:53.633 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159617+0000", 2024-09-21T15:00:53.633 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2909999999999999e-06 2024-09-21T15:00:53.633 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.633 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.633 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:53.633 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159623+0000", 2024-09-21T15:00:53.633 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.511e-06 2024-09-21T15:00:53.633 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.633 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.634 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:53.634 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159631+0000", 2024-09-21T15:00:53.634 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.571e-06 2024-09-21T15:00:53.634 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.634 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.634 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:53.635 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227866+0000", 2024-09-21T15:00:53.635 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068235136000000002 2024-09-21T15:00:53.635 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.635 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.635 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:53.635 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227875+0000", 2024-09-21T15:00:53.635 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7199999999999995e-06 2024-09-21T15:00:53.636 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:53.636 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:53.636 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:53.636 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.636 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.636 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45848 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:53.636 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159636+0000", 2024-09-21T15:00:53.636 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 103.442217023, 2024-09-21T15:00:53.637 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068267283999999998, 2024-09-21T15:00:53.637 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:53.637 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:53.637 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:53.637 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:53.637 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:53.637 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45848 2024-09-21T15:00:53.637 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.637 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:53.638 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.638 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:53.638 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159636+0000", 2024-09-21T15:00:53.638 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:53.638 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.638 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.638 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:53.639 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159636+0000", 2024-09-21T15:00:53.639 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:53.639 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.639 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.639 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:53.639 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159636+0000", 2024-09-21T15:00:53.639 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:53.639 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.640 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.640 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:53.640 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159638+0000", 2024-09-21T15:00:53.640 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6560000000000001e-06 2024-09-21T15:00:53.640 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.640 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.640 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:53.641 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159644+0000", 2024-09-21T15:00:53.641 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0839999999999997e-06 2024-09-21T15:00:53.641 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.641 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.641 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:53.641 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159649+0000", 2024-09-21T15:00:53.641 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7500000000000003e-06 2024-09-21T15:00:53.641 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.641 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.641 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:53.641 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227892+0000", 2024-09-21T15:00:53.642 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068243315999999998 2024-09-21T15:00:53.642 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.642 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.642 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:53.642 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227903+0000", 2024-09-21T15:00:53.642 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0835999999999999e-05 2024-09-21T15:00:53.642 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:53.642 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:53.642 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:53.642 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.642 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.642 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45849 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:53.642 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159667+0000", 2024-09-21T15:00:53.642 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 103.442186209, 2024-09-21T15:00:53.643 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068291051000000005, 2024-09-21T15:00:53.643 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:53.643 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:53.643 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:53.643 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:53.643 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:53.643 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45849 2024-09-21T15:00:53.643 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.643 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:53.643 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.643 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:53.643 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159667+0000", 2024-09-21T15:00:53.643 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:53.643 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.643 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.644 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:53.644 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159667+0000", 2024-09-21T15:00:53.644 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:53.644 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.644 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.644 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:53.644 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159666+0000", 2024-09-21T15:00:53.644 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:53.644 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.644 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.644 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:53.644 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159670+0000", 2024-09-21T15:00:53.644 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.427e-06 2024-09-21T15:00:53.644 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.644 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.645 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:53.645 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159678+0000", 2024-09-21T15:00:53.645 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9869999999999999e-06 2024-09-21T15:00:53.645 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.645 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.645 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:53.645 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159688+0000", 2024-09-21T15:00:53.645 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0631999999999999e-05 2024-09-21T15:00:53.645 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.645 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.645 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:53.645 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227950+0000", 2024-09-21T15:00:53.645 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068261798999999998 2024-09-21T15:00:53.645 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.645 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.646 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:53.646 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227958+0000", 2024-09-21T15:00:53.646 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7510000000000008e-06 2024-09-21T15:00:53.646 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:53.646 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:53.646 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:53.646 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.646 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.646 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45850 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:53.646 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159695+0000", 2024-09-21T15:00:53.646 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 103.442158498, 2024-09-21T15:00:53.646 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068302999000000003, 2024-09-21T15:00:53.646 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:53.646 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:53.647 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:53.647 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:53.647 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:53.647 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45850 2024-09-21T15:00:53.647 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.647 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:53.647 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.647 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:53.647 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159695+0000", 2024-09-21T15:00:53.647 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:53.647 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.647 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.647 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:53.647 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159695+0000", 2024-09-21T15:00:53.647 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:53.648 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.648 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.648 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:53.648 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159694+0000", 2024-09-21T15:00:53.648 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:53.648 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.648 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.648 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:53.648 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159697+0000", 2024-09-21T15:00:53.648 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.258e-06 2024-09-21T15:00:53.648 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.648 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.648 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:53.648 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159704+0000", 2024-09-21T15:00:53.648 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7660000000000001e-06 2024-09-21T15:00:53.648 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.649 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.649 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:53.649 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159713+0000", 2024-09-21T15:00:53.649 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1779999999999992e-06 2024-09-21T15:00:53.649 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.649 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.649 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:53.649 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227989+0000", 2024-09-21T15:00:53.649 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068276015999999995 2024-09-21T15:00:53.649 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.649 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.649 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:53.649 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227998+0000", 2024-09-21T15:00:53.649 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2050000000000002e-06 2024-09-21T15:00:53.649 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:53.650 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:53.650 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:53.650 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.650 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.650 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45851 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:53.650 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159719+0000", 2024-09-21T15:00:53.650 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 103.442134365, 2024-09-21T15:00:53.650 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068320327, 2024-09-21T15:00:53.650 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:53.650 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:53.650 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:53.650 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:53.650 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:53.650 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45851 2024-09-21T15:00:53.651 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.651 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:53.651 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.651 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:53.651 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159719+0000", 2024-09-21T15:00:53.651 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:53.651 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.651 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.651 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:53.651 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159719+0000", 2024-09-21T15:00:53.651 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:53.652 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.652 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.652 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:53.652 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159718+0000", 2024-09-21T15:00:53.652 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:53.652 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.652 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.652 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:53.652 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159721+0000", 2024-09-21T15:00:53.652 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2169999999999999e-06 2024-09-21T15:00:53.652 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.652 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.652 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:53.653 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159730+0000", 2024-09-21T15:00:53.653 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5720000000000003e-06 2024-09-21T15:00:53.653 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.653 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.653 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:53.653 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159735+0000", 2024-09-21T15:00:53.653 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1100000000000002e-06 2024-09-21T15:00:53.653 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.653 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.653 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:53.653 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228029+0000", 2024-09-21T15:00:53.653 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068293624999999997 2024-09-21T15:00:53.653 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.654 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.654 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:53.654 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228039+0000", 2024-09-21T15:00:53.654 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0254e-05 2024-09-21T15:00:53.654 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:53.654 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:53.654 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:53.654 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.654 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.654 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45852 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:53.654 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159740+0000", 2024-09-21T15:00:53.654 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 103.442113197, 2024-09-21T15:00:53.654 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068340260999999999, 2024-09-21T15:00:53.655 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:53.655 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:53.655 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:53.655 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:53.655 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:53.655 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45852 2024-09-21T15:00:53.655 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.655 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:53.655 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.655 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:53.655 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159740+0000", 2024-09-21T15:00:53.655 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:53.655 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.655 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.656 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:53.656 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159740+0000", 2024-09-21T15:00:53.656 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:53.656 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.656 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.656 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:53.656 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159739+0000", 2024-09-21T15:00:53.656 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:53.656 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.656 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.656 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:53.656 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159757+0000", 2024-09-21T15:00:53.657 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7314999999999999e-05 2024-09-21T15:00:53.657 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.657 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.657 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:53.657 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159769+0000", 2024-09-21T15:00:53.657 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2376e-05 2024-09-21T15:00:53.657 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.657 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.657 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:53.657 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159775+0000", 2024-09-21T15:00:53.657 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2179999999999996e-06 2024-09-21T15:00:53.657 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.657 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.658 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:53.658 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228071+0000", 2024-09-21T15:00:53.658 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068295344999999993 2024-09-21T15:00:53.658 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.658 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.658 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:53.658 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228080+0000", 2024-09-21T15:00:53.658 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3959999999999995e-06 2024-09-21T15:00:53.658 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:53.658 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:53.658 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:53.658 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.659 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.659 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45853 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:53.659 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159781+0000", 2024-09-21T15:00:53.659 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 103.44207193299999, 2024-09-21T15:00:53.659 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068337093000000002, 2024-09-21T15:00:53.659 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:53.659 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:53.659 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:53.659 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:53.659 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:53.659 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45853 2024-09-21T15:00:53.659 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.659 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:53.659 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.660 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:53.660 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159781+0000", 2024-09-21T15:00:53.660 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:53.660 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.660 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.660 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:53.660 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159781+0000", 2024-09-21T15:00:53.660 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:53.660 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.660 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.660 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:53.660 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159781+0000", 2024-09-21T15:00:53.660 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:53.660 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.661 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.661 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:53.661 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159784+0000", 2024-09-21T15:00:53.661 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4759999999999998e-06 2024-09-21T15:00:53.661 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.661 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.661 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:53.661 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159792+0000", 2024-09-21T15:00:53.661 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0569999999999993e-06 2024-09-21T15:00:53.661 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.661 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.662 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:53.662 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159797+0000", 2024-09-21T15:00:53.662 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9640000000000002e-06 2024-09-21T15:00:53.662 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.662 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.662 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:53.662 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228111+0000", 2024-09-21T15:00:53.662 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068313737999999999 2024-09-21T15:00:53.662 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.662 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.662 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:53.663 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228118+0000", 2024-09-21T15:00:53.663 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.43e-06 2024-09-21T15:00:53.663 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:53.663 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:53.663 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:53.663 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.663 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.663 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45854 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:53.663 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159802+0000", 2024-09-21T15:00:53.663 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 103.442051283, 2024-09-21T15:00:53.663 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068358938999999994, 2024-09-21T15:00:53.663 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:53.663 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:53.663 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:53.664 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:53.664 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:53.664 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45854 2024-09-21T15:00:53.664 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.664 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:53.664 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.664 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:53.664 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159802+0000", 2024-09-21T15:00:53.664 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:53.664 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.664 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.664 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:53.664 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159802+0000", 2024-09-21T15:00:53.664 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:53.664 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.665 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.665 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:53.665 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159801+0000", 2024-09-21T15:00:53.665 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:53.665 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.665 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.665 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:53.665 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159804+0000", 2024-09-21T15:00:53.665 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0970000000000002e-06 2024-09-21T15:00:53.665 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.665 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.665 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:53.665 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159823+0000", 2024-09-21T15:00:53.665 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8895000000000001e-05 2024-09-21T15:00:53.665 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.666 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.666 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:53.666 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159837+0000", 2024-09-21T15:00:53.666 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3247999999999999e-05 2024-09-21T15:00:53.666 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.666 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.666 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:53.666 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228151+0000", 2024-09-21T15:00:53.666 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068314642999999994 2024-09-21T15:00:53.666 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.666 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.666 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:53.666 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228161+0000", 2024-09-21T15:00:53.666 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4250000000000004e-06 2024-09-21T15:00:53.666 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:53.667 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:53.667 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:53.667 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.667 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.667 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45983 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:53.667 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159848+0000", 2024-09-21T15:00:53.667 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 103.442005024, 2024-09-21T15:00:53.667 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068333801, 2024-09-21T15:00:53.667 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:53.667 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:53.667 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:53.667 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:53.667 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:53.667 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45983 2024-09-21T15:00:53.668 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.668 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:53.668 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.668 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:53.668 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159848+0000", 2024-09-21T15:00:53.668 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:53.668 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.668 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.668 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:53.668 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159848+0000", 2024-09-21T15:00:53.668 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:53.668 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.668 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.668 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:53.669 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159847+0000", 2024-09-21T15:00:53.669 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:53.669 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.669 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.669 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:53.669 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159856+0000", 2024-09-21T15:00:53.669 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4349999999999997e-06 2024-09-21T15:00:53.669 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.669 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.669 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:53.669 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159866+0000", 2024-09-21T15:00:53.669 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0076e-05 2024-09-21T15:00:53.669 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.670 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.670 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:53.670 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159872+0000", 2024-09-21T15:00:53.670 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8309999999999997e-06 2024-09-21T15:00:53.670 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.670 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.670 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:53.670 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228175+0000", 2024-09-21T15:00:53.670 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068302680000000005 2024-09-21T15:00:53.670 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.670 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.670 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:53.670 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228182+0000", 2024-09-21T15:00:53.670 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3150000000000003e-06 2024-09-21T15:00:53.671 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:53.671 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:53.671 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:53.671 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.671 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.671 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45984 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:53.671 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159877+0000", 2024-09-21T15:00:53.671 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 103.44197560400001, 2024-09-21T15:00:53.671 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068328448, 2024-09-21T15:00:53.671 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:53.671 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:53.671 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:53.671 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:53.672 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:53.672 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45984 2024-09-21T15:00:53.672 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.672 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:53.672 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.672 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:53.672 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159877+0000", 2024-09-21T15:00:53.672 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:53.672 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.672 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.672 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:53.672 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159877+0000", 2024-09-21T15:00:53.672 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:53.672 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.672 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.673 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:53.673 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159877+0000", 2024-09-21T15:00:53.673 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:53.673 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.673 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.673 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:53.673 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159880+0000", 2024-09-21T15:00:53.673 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7610000000000001e-06 2024-09-21T15:00:53.673 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.673 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.673 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:53.673 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159888+0000", 2024-09-21T15:00:53.673 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0330000000000003e-06 2024-09-21T15:00:53.673 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.674 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.674 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:53.674 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159893+0000", 2024-09-21T15:00:53.674 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3550000000000004e-06 2024-09-21T15:00:53.674 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.674 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.674 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:53.674 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228199+0000", 2024-09-21T15:00:53.674 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068306438999999997 2024-09-21T15:00:53.674 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.674 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.674 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:53.674 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228206+0000", 2024-09-21T15:00:53.675 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5599999999999999e-06 2024-09-21T15:00:53.675 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:53.675 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:53.675 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:53.675 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.675 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.675 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45985 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:53.675 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159897+0000", 2024-09-21T15:00:53.675 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 103.441955781, 2024-09-21T15:00:53.675 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068345664, 2024-09-21T15:00:53.675 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:53.675 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:53.675 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:53.675 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:53.675 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:53.676 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45985 2024-09-21T15:00:53.676 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.676 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:53.676 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.676 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:53.676 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159897+0000", 2024-09-21T15:00:53.676 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:53.676 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.676 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.676 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:53.676 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159897+0000", 2024-09-21T15:00:53.676 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:53.676 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.676 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.676 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:53.677 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159897+0000", 2024-09-21T15:00:53.677 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:53.677 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.677 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.677 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:53.677 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159900+0000", 2024-09-21T15:00:53.677 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.954e-06 2024-09-21T15:00:53.677 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.677 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.677 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:53.677 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159913+0000", 2024-09-21T15:00:53.677 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3106e-05 2024-09-21T15:00:53.677 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.677 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.677 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:53.678 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159918+0000", 2024-09-21T15:00:53.678 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5199999999999997e-06 2024-09-21T15:00:53.678 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.678 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.678 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:53.678 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228236+0000", 2024-09-21T15:00:53.678 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068317040999999995 2024-09-21T15:00:53.678 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.678 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.678 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:53.678 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228243+0000", 2024-09-21T15:00:53.678 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3880000000000002e-06 2024-09-21T15:00:53.678 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:53.679 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:53.679 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:53.679 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.679 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.679 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45986 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:53.679 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159924+0000", 2024-09-21T15:00:53.679 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 103.44192948600001, 2024-09-21T15:00:53.679 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068364362999999997, 2024-09-21T15:00:53.679 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:53.679 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:53.679 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:53.679 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:53.679 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:53.679 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45986 2024-09-21T15:00:53.679 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.680 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:53.680 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.680 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:53.680 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159924+0000", 2024-09-21T15:00:53.680 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:53.680 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.680 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.680 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:53.680 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159924+0000", 2024-09-21T15:00:53.680 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:53.680 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.680 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.680 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:53.681 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159923+0000", 2024-09-21T15:00:53.681 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:53.681 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.681 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.681 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:53.681 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159926+0000", 2024-09-21T15:00:53.681 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0510000000000001e-06 2024-09-21T15:00:53.681 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.681 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.681 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:53.681 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159932+0000", 2024-09-21T15:00:53.681 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4160000000000001e-06 2024-09-21T15:00:53.681 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.681 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.682 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:53.682 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159938+0000", 2024-09-21T15:00:53.682 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.383e-06 2024-09-21T15:00:53.682 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.682 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.682 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:53.682 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228275+0000", 2024-09-21T15:00:53.682 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068337007000000005 2024-09-21T15:00:53.682 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.682 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.682 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:53.682 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228288+0000", 2024-09-21T15:00:53.682 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3052e-05 2024-09-21T15:00:53.683 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:53.683 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:53.683 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:53.683 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.683 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.683 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45987 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:53.683 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159944+0000", 2024-09-21T15:00:53.683 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 103.441909262, 2024-09-21T15:00:53.683 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068387232000000006, 2024-09-21T15:00:53.683 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:53.683 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:53.683 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:53.683 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:53.684 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:53.684 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45987 2024-09-21T15:00:53.684 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.684 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:53.684 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.684 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:53.684 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159944+0000", 2024-09-21T15:00:53.684 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:53.684 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.684 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.684 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:53.684 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159944+0000", 2024-09-21T15:00:53.684 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:53.685 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.685 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.685 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:53.685 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159943+0000", 2024-09-21T15:00:53.685 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:53.685 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.685 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.685 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:53.685 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159946+0000", 2024-09-21T15:00:53.685 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6309999999999999e-06 2024-09-21T15:00:53.685 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.685 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.685 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:53.685 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159952+0000", 2024-09-21T15:00:53.686 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6540000000000004e-06 2024-09-21T15:00:53.686 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.686 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.686 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:53.686 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159957+0000", 2024-09-21T15:00:53.686 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9280000000000001e-06 2024-09-21T15:00:53.686 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.686 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.686 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:53.686 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228322+0000", 2024-09-21T15:00:53.686 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068365067000000002 2024-09-21T15:00:53.686 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.686 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.686 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:53.687 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228331+0000", 2024-09-21T15:00:53.687 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3270000000000007e-06 2024-09-21T15:00:53.687 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:53.687 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:53.687 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:53.687 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.687 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.687 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:97672 3.4 3:26046d29:::rbd_data.1136bfc79a70.0000000000000001:head [write 0~4718592 [fadvise_sequential+fadvise_nocache],write 5242880~131072 [fadvise_sequential+fadvise_nocache],write 5439488~147456 [fadvise_sequential+fadvise_nocache],write 5652480~229376 [fadvise_sequential+fadvise_nocache],write 5947392~147456 [fadvise_sequential+fadvise_nocache],write 6160384~229376 [fadvise_sequential+fadvise_nocache],write 6455296~147456 [fadvise_sequential+fadvise_nocache],write 6668288~229376 [fadvise_sequential+fadvise_nocache],write 6963200~147456 [fadvise_sequential+fadvise_nocache],write 7176192~147456 [fadvise_sequential+fadvise_nocache],write 7340032~65536 [fadvise_sequential+fadvise_nocache],write 7471104~147456 [fadvise_sequential+fadvise_nocache],write 7684096~147456 [fadvise_sequential+fadvise_nocache],write 7864320~49152 [fadvise_sequential+fadvise_nocache],write 7979008~147456 [fadvise_sequential+fadvise_nocache],write 8192000~147456 [fadvise_sequential+fadvise_nocache],write 8388608~32768 [fadvise_sequential+fadvise_nocache],write 8519680~114688 [fadvise_sequential+fadvise_nocache],write 8732672~114688 [fadvise_sequential+fadvise_nocache],write 8912896~16384 [fadvise_sequential+fadvise_nocache],write 9109504~32768 [fadvise_sequential+fadvise_nocache],write 9322496~32768 [fadvise_sequential+fadvise_nocache],write 9617408~32768 [fadvise_sequential+fadvise_nocache],write 9830400~32768 [fadvise_sequential+fadvise_nocache],write 9961472~98304 [fadvise_sequential+fadvise_nocache],write 10076160~196608 [fadvise_sequential+fadvise_nocache],write 10289152~278528 [fadvise_sequential+fadvise_nocache],write 10584064~196608 [fadvise_sequential+fadvise_nocache],write 10797056~196608 [fadvise_sequential+fadvise_nocache],write 11010048~65536 [fadvise_sequential+fadvise_nocache],write 11091968~196608 [fadvise_sequential+fadvise_nocache],write 11304960~196608 [fadvise_sequential+fadvise_nocache],write 11517952~65536 [fadvise_sequential+fadvise_nocache],write 11616256~98304 [fadvise_sequential+fadvise_nocache],write 11730944~65536 [fadvise_sequential+fadvise_nocache],write 11829248~98304 [fadvise_sequential+fadvise_nocache],write 11943936~65536 [fadvise_sequential+fadvise_nocache],write 12042240~49152 [fadvise_sequential+fadvise_nocache],write 12124160~98304 [fadvise_sequential+fadvise_nocache],write 12238848~65536 [fadvise_sequential+fadvise_nocache],write 12337152~98304 [fadvise_sequential+fadvise_nocache],write 12451840~65536 [fadvise_sequential+fadvise_nocache],write 12550144~32768 [fadvise_sequential+fadvise_nocache],write 12632064~98304 [fadvise_sequential+fadvise_nocache],write 12845056~98304 [fadvise_sequential+fadvise_nocache],write 13058048~49152 [fadvise_sequential+fadvise_nocache],write 13139968~98304 [fadvise_sequential+fadvise_nocache],write 13352960~98304 [fadvise_sequential+fadvise_nocache],write 13565952~65536 [fadvise_sequential+fadvise_nocache],write 13647872~98304 [fadvise_sequential+fadvise_nocache],write 13860864~98304 [fadvise_sequential+fadvise_nocache],write 14073856~81920 [fadvise_sequential+fadvise_nocache],write 15269888~81920 [fadvise_sequential+fadvise_nocache],write 15482880~81920 [fadvise_sequential+fadvise_nocache],write 15695872~32768 [fadvise_sequential+fadvise_nocache],write 15777792~81920 [fadvise_sequential+fadvise_nocache],write 15990784~81920 [fadvise_sequential+fadvise_nocache],write 16203776~49152 [fadvise_sequential+fadvise_nocache],write 16285696~81920 [fadvise_sequential+fadvise_nocache],write 16498688~81920 [fadvise_sequential+fadvise_nocache],write 16711680~65536 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:53.687 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.298669+0000", 2024-09-21T15:00:53.687 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 48.303184195999997, 2024-09-21T15:00:53.687 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.10282358900000001, 2024-09-21T15:00:53.687 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:53.687 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:53.687 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:53.688 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:53.688 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:53.688 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97672 2024-09-21T15:00:53.688 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.688 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:53.688 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.688 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:53.688 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.298669+0000", 2024-09-21T15:00:53.688 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:53.688 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.688 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.688 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:53.689 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.298669+0000", 2024-09-21T15:00:53.689 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:53.689 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.689 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.689 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:53.689 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.298667+0000", 2024-09-21T15:00:53.689 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:00:53.689 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.689 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.689 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:53.689 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.327458+0000", 2024-09-21T15:00:53.689 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.028790754000000002 2024-09-21T15:00:53.689 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.689 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.689 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:53.689 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.327492+0000", 2024-09-21T15:00:53.690 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3806999999999997e-05 2024-09-21T15:00:53.690 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.690 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.690 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:53.690 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.327507+0000", 2024-09-21T15:00:53.690 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5112e-05 2024-09-21T15:00:53.690 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.690 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.690 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:53.690 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.327597+0000", 2024-09-21T15:00:53.690 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9938999999999994e-05 2024-09-21T15:00:53.690 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.690 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.690 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:53.691 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.327927+0000", 2024-09-21T15:00:53.691 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000329766 2024-09-21T15:00:53.691 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.691 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.691 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T15:00:53.691 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.330735+0000", 2024-09-21T15:00:53.691 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0028086869999999998 2024-09-21T15:00:53.691 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.691 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.691 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:53.691 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.378299+0000", 2024-09-21T15:00:53.691 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.047563495999999997 2024-09-21T15:00:53.691 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.691 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.691 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:53.691 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.401248+0000", 2024-09-21T15:00:53.692 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.022949311 2024-09-21T15:00:53.692 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.692 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.692 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:53.692 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.401446+0000", 2024-09-21T15:00:53.692 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019831199999999999 2024-09-21T15:00:53.692 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:53.692 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:53.692 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:53.692 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.401492+0000", 2024-09-21T15:00:53.692 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6109000000000002e-05 2024-09-21T15:00:53.692 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:53.692 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:53.692 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:53.692 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:53.693 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:53.693 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:53.694 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T15:00:53.695 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:00:53.689+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:00:53.795 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:00:53.795+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:00:53.856 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:53.856 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:00:53.856 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T15:00:53.856 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:53.866 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T15:00:53.896 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:00:53.891+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:00:53.985 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:53.985 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:00:53.986 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T15:00:53.986 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T15:00:53.986 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:53.997 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:00:53.993+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:00:53.998 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T15:00:54.098 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:00:54.095+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:00:54.149 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:54.149 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T15:00:54.149 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T15:00:54.149 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T15:00:54.149 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.150 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:97677 3.15 e122/24 3:a8e72133:::rbd_data.1136bfc79a70.0000000000000005:head v 122'567, mlcod=120'565)", 2024-09-21T15:00:54.150 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.320753+0000", 2024-09-21T15:00:54.150 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 48.824528794999999, 2024-09-21T15:00:54.150 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070639375000000004, 2024-09-21T15:00:54.150 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:54.150 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:54.150 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:54.150 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.150 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:54.150 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.320753+0000", 2024-09-21T15:00:54.150 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:54.150 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.150 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.150 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:54.151 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.320753+0000", 2024-09-21T15:00:54.151 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:54.151 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.151 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.151 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:54.151 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.320753+0000", 2024-09-21T15:00:54.151 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:54.151 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.151 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.151 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:54.151 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.375452+0000", 2024-09-21T15:00:54.151 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054698945999999998 2024-09-21T15:00:54.151 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.151 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.152 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:54.152 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.375476+0000", 2024-09-21T15:00:54.152 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4380999999999999e-05 2024-09-21T15:00:54.152 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.152 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.152 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:54.152 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.375485+0000", 2024-09-21T15:00:54.152 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0049999999999993e-06 2024-09-21T15:00:54.152 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.152 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.152 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:54.152 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.375552+0000", 2024-09-21T15:00:54.152 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6550999999999994e-05 2024-09-21T15:00:54.152 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.153 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.153 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:54.153 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.375578+0000", 2024-09-21T15:00:54.153 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6196000000000001e-05 2024-09-21T15:00:54.153 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.153 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.153 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:54.153 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.391319+0000", 2024-09-21T15:00:54.153 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015741271000000001 2024-09-21T15:00:54.153 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.153 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.153 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:54.153 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.391393+0000", 2024-09-21T15:00:54.154 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3287000000000005e-05 2024-09-21T15:00:54.154 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:54.154 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:54.154 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:54.154 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.154 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.154 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:97672 3.4 e122/51 3:26046d29:::rbd_data.1136bfc79a70.0000000000000001:head v 122'256, mlcod=120'254)", 2024-09-21T15:00:54.154 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.331696+0000", 2024-09-21T15:00:54.154 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 48.813585631999999, 2024-09-21T15:00:54.154 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041222507999999998, 2024-09-21T15:00:54.154 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:54.154 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:54.155 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:54.155 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.155 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:54.155 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.331696+0000", 2024-09-21T15:00:54.155 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:54.155 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.155 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.155 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:54.155 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.331696+0000", 2024-09-21T15:00:54.155 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:54.155 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.155 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.155 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:54.155 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.331695+0000", 2024-09-21T15:00:54.156 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:00:54.156 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.156 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.156 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:54.156 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.340175+0000", 2024-09-21T15:00:54.156 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0084799520000000007 2024-09-21T15:00:54.156 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.156 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.156 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:54.156 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.340208+0000", 2024-09-21T15:00:54.156 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2997999999999998e-05 2024-09-21T15:00:54.156 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.156 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.156 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:54.157 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.340223+0000", 2024-09-21T15:00:54.157 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5330999999999999e-05 2024-09-21T15:00:54.157 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.157 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.157 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:54.157 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.340305+0000", 2024-09-21T15:00:54.157 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2095999999999995e-05 2024-09-21T15:00:54.157 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.157 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.157 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:54.157 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.340337+0000", 2024-09-21T15:00:54.157 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2097000000000001e-05 2024-09-21T15:00:54.157 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.157 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.158 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:54.158 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.372863+0000", 2024-09-21T15:00:54.158 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.032525923999999998 2024-09-21T15:00:54.158 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.158 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.158 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:54.158 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.372919+0000", 2024-09-21T15:00:54.158 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5729000000000001e-05 2024-09-21T15:00:54.158 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:54.158 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:54.158 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:54.158 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.158 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.159 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:116177 3.12 3:4efbaa3f:::rbd_data.1136b1194d20.0000000000000006:58 [read 5791744~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:54.159 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.712449+0000", 2024-09-21T15:00:54.159 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 24.432832951000002, 2024-09-21T15:00:54.159 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.033824706000000003, 2024-09-21T15:00:54.159 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:54.159 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:54.159 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:54.159 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:54.159 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:54.159 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116177 2024-09-21T15:00:54.159 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.159 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:54.159 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.160 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:54.160 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712449+0000", 2024-09-21T15:00:54.160 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:54.160 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.160 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.160 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:54.160 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712449+0000", 2024-09-21T15:00:54.160 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:54.160 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.160 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.160 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:54.160 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712448+0000", 2024-09-21T15:00:54.160 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:54.160 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.160 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.161 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:54.161 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712451+0000", 2024-09-21T15:00:54.161 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5330000000000001e-06 2024-09-21T15:00:54.161 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.161 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.161 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:54.161 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712458+0000", 2024-09-21T15:00:54.161 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5239999999999997e-06 2024-09-21T15:00:54.161 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.161 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.161 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:54.161 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712467+0000", 2024-09-21T15:00:54.161 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7189999999999996e-06 2024-09-21T15:00:54.161 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.161 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.161 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:54.162 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.746189+0000", 2024-09-21T15:00:54.162 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.033722203999999999 2024-09-21T15:00:54.162 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.162 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.162 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:54.162 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.746228+0000", 2024-09-21T15:00:54.162 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8908000000000001e-05 2024-09-21T15:00:54.162 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.162 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.162 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:54.162 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.746274+0000", 2024-09-21T15:00:54.162 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5380000000000003e-05 2024-09-21T15:00:54.162 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:54.162 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:54.162 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:54.163 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.163 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.163 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:116178 3.12 3:4efbaa3f:::rbd_data.1136b1194d20.0000000000000006:58 [read 5881856~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:54.163 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.712472+0000", 2024-09-21T15:00:54.163 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 24.4328103, 2024-09-21T15:00:54.163 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.033958110999999999, 2024-09-21T15:00:54.163 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:54.163 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:54.163 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:54.163 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:54.163 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:54.163 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116178 2024-09-21T15:00:54.163 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.163 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:54.163 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.164 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:54.164 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712472+0000", 2024-09-21T15:00:54.164 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:54.164 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.164 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.164 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:54.164 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712472+0000", 2024-09-21T15:00:54.164 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:54.164 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.164 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.164 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:54.164 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712471+0000", 2024-09-21T15:00:54.164 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:54.165 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.165 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.165 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:54.165 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712474+0000", 2024-09-21T15:00:54.165 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4389999999999999e-06 2024-09-21T15:00:54.165 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.165 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.165 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:54.165 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712479+0000", 2024-09-21T15:00:54.165 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4269999999999999e-06 2024-09-21T15:00:54.165 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.165 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.165 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:54.165 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712483+0000", 2024-09-21T15:00:54.165 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8449999999999996e-06 2024-09-21T15:00:54.166 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.166 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.166 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:54.166 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.746287+0000", 2024-09-21T15:00:54.166 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.033804381000000001 2024-09-21T15:00:54.166 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.166 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.166 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:54.166 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.746337+0000", 2024-09-21T15:00:54.166 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9622e-05 2024-09-21T15:00:54.166 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.166 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.166 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:54.166 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.746430+0000", 2024-09-21T15:00:54.167 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2800000000000006e-05 2024-09-21T15:00:54.167 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:54.167 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:54.167 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:54.167 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.167 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.167 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:116179 3.12 3:4efbaa3f:::rbd_data.1136b1194d20.0000000000000006:58 [read 5971968~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:54.167 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.712487+0000", 2024-09-21T15:00:54.167 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 24.432795458000001, 2024-09-21T15:00:54.167 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034144503, 2024-09-21T15:00:54.167 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:54.167 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:54.167 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:54.167 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:54.167 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:54.168 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116179 2024-09-21T15:00:54.168 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.168 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:54.168 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.168 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:54.168 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712487+0000", 2024-09-21T15:00:54.168 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:54.168 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.168 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.168 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:54.168 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712487+0000", 2024-09-21T15:00:54.168 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:54.168 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.168 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.168 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:54.169 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712486+0000", 2024-09-21T15:00:54.169 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:54.169 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.169 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.169 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:54.169 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712488+0000", 2024-09-21T15:00:54.169 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2000000000000001e-06 2024-09-21T15:00:54.169 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.169 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.169 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:54.169 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712493+0000", 2024-09-21T15:00:54.169 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9429999999999999e-06 2024-09-21T15:00:54.169 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.169 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.169 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:54.169 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712497+0000", 2024-09-21T15:00:54.170 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9689999999999996e-06 2024-09-21T15:00:54.170 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.170 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.170 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:54.170 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.746459+0000", 2024-09-21T15:00:54.170 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.033961365 2024-09-21T15:00:54.170 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.170 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.170 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:54.170 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.746538+0000", 2024-09-21T15:00:54.170 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9073999999999995e-05 2024-09-21T15:00:54.170 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.170 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.171 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:54.171 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.746631+0000", 2024-09-21T15:00:54.171 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3375999999999995e-05 2024-09-21T15:00:54.171 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:54.171 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:54.171 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:54.171 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.171 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.171 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:116180 3.12 3:4efbaa3f:::rbd_data.1136b1194d20.0000000000000006:58 [read 6062080~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:54.171 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.712501+0000", 2024-09-21T15:00:54.171 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 24.432781259999999, 2024-09-21T15:00:54.171 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034331168000000002, 2024-09-21T15:00:54.171 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:54.171 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:54.171 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:54.172 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:54.172 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:54.172 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116180 2024-09-21T15:00:54.172 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.172 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:54.172 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.172 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:54.172 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712501+0000", 2024-09-21T15:00:54.172 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:54.172 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.172 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.172 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:54.172 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712501+0000", 2024-09-21T15:00:54.172 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:54.172 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.173 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.173 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:54.173 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712500+0000", 2024-09-21T15:00:54.173 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:54.173 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.173 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.173 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:54.173 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712502+0000", 2024-09-21T15:00:54.173 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.035e-06 2024-09-21T15:00:54.173 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.173 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.173 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:54.173 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712508+0000", 2024-09-21T15:00:54.173 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4720000000000003e-06 2024-09-21T15:00:54.174 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.174 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.174 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:54.174 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712512+0000", 2024-09-21T15:00:54.174 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7129999999999999e-06 2024-09-21T15:00:54.174 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.174 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.174 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:54.174 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.746662+0000", 2024-09-21T15:00:54.174 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034149837000000002 2024-09-21T15:00:54.174 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.174 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.174 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:54.174 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.746740+0000", 2024-09-21T15:00:54.175 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8350999999999996e-05 2024-09-21T15:00:54.175 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.175 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.175 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:54.175 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.746832+0000", 2024-09-21T15:00:54.175 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2046000000000006e-05 2024-09-21T15:00:54.175 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:54.175 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:54.175 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:54.175 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.175 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.175 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:116181 3.12 3:4efbaa3f:::rbd_data.1136b1194d20.0000000000000006:58 [read 6152192~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:54.175 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.712515+0000", 2024-09-21T15:00:54.175 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 24.432766913999998, 2024-09-21T15:00:54.176 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034525820999999998, 2024-09-21T15:00:54.176 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:54.176 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:54.176 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:54.176 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:54.176 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:54.176 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116181 2024-09-21T15:00:54.176 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.176 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:54.176 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.176 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:54.176 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712515+0000", 2024-09-21T15:00:54.176 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:54.176 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.176 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.177 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:54.177 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712515+0000", 2024-09-21T15:00:54.177 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:54.177 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.177 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.177 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:54.177 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712515+0000", 2024-09-21T15:00:54.177 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:54.177 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.177 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.177 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:54.177 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712522+0000", 2024-09-21T15:00:54.177 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8000000000000001e-06 2024-09-21T15:00:54.177 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.177 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.177 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:54.178 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712528+0000", 2024-09-21T15:00:54.178 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0650000000000004e-06 2024-09-21T15:00:54.178 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.178 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.178 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:54.178 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712533+0000", 2024-09-21T15:00:54.178 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.079e-06 2024-09-21T15:00:54.178 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.178 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.178 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:54.178 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.746864+0000", 2024-09-21T15:00:54.178 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034330830999999999 2024-09-21T15:00:54.178 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.178 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.179 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:54.179 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.746943+0000", 2024-09-21T15:00:54.179 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9221999999999995e-05 2024-09-21T15:00:54.179 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.179 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.179 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:54.179 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.747041+0000", 2024-09-21T15:00:54.179 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.8114000000000003e-05 2024-09-21T15:00:54.179 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:54.179 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:54.179 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:54.179 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.179 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.179 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:116182 3.12 3:4efbaa3f:::rbd_data.1136b1194d20.0000000000000006:58 [read 6242304~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:54.180 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.712538+0000", 2024-09-21T15:00:54.180 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 24.432743640999998, 2024-09-21T15:00:54.180 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034702127999999999, 2024-09-21T15:00:54.180 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:54.180 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:54.180 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:54.180 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:54.180 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:54.180 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116182 2024-09-21T15:00:54.180 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.180 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:54.180 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.180 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:54.180 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712538+0000", 2024-09-21T15:00:54.180 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:54.181 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.181 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.181 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:54.181 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712538+0000", 2024-09-21T15:00:54.181 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:54.181 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.181 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.181 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:54.181 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712538+0000", 2024-09-21T15:00:54.181 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:54.181 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.181 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.181 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:54.181 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712541+0000", 2024-09-21T15:00:54.181 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5330000000000001e-06 2024-09-21T15:00:54.182 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.182 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.182 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:54.182 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712547+0000", 2024-09-21T15:00:54.182 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1109999999999996e-06 2024-09-21T15:00:54.182 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.182 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.182 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:54.182 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712551+0000", 2024-09-21T15:00:54.182 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.685e-06 2024-09-21T15:00:54.182 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.182 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.182 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:54.182 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.747067+0000", 2024-09-21T15:00:54.182 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034516181 2024-09-21T15:00:54.183 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.183 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.183 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:54.183 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.747152+0000", 2024-09-21T15:00:54.183 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4075999999999999e-05 2024-09-21T15:00:54.183 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.183 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.183 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:54.183 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.747240+0000", 2024-09-21T15:00:54.183 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8924000000000004e-05 2024-09-21T15:00:54.183 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:54.183 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:54.183 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:54.183 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.183 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.183 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:116216 3.12 3:4efbaa3f:::rbd_data.1136b1194d20.0000000000000006:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:54.184 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.713202+0000", 2024-09-21T15:00:54.184 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 24.432080136, 2024-09-21T15:00:54.184 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034182227000000003, 2024-09-21T15:00:54.184 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:54.184 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:54.184 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:54.184 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:54.184 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:54.184 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116216 2024-09-21T15:00:54.184 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.184 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:54.184 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.184 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:54.184 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.713202+0000", 2024-09-21T15:00:54.184 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:54.185 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.185 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.185 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:54.185 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.713202+0000", 2024-09-21T15:00:54.185 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:54.185 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.185 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.185 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:54.185 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.713201+0000", 2024-09-21T15:00:54.185 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:54.185 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.185 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.185 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:54.185 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.713204+0000", 2024-09-21T15:00:54.185 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7410000000000001e-06 2024-09-21T15:00:54.185 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.185 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.186 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:54.186 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.713211+0000", 2024-09-21T15:00:54.186 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1110000000000002e-06 2024-09-21T15:00:54.186 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.186 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.186 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:54.186 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.713217+0000", 2024-09-21T15:00:54.186 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.446e-06 2024-09-21T15:00:54.186 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.186 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.186 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:54.186 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.747263+0000", 2024-09-21T15:00:54.186 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034046598999999997 2024-09-21T15:00:54.186 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.186 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.187 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:54.187 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.747321+0000", 2024-09-21T15:00:54.187 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7778999999999997e-05 2024-09-21T15:00:54.187 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.187 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.187 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:54.187 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.747384+0000", 2024-09-21T15:00:54.187 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3083000000000006e-05 2024-09-21T15:00:54.187 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:54.187 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:54.187 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:54.187 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.187 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.187 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:118512 3.12 3:4efbaa3f:::rbd_data.1136b1194d20.0000000000000006:58 [read 10043392~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:54.187 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.893240+0000", 2024-09-21T15:00:54.187 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 24.252041548000001, 2024-09-21T15:00:54.188 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.033811941999999998, 2024-09-21T15:00:54.188 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:54.188 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:54.188 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:54.188 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:54.188 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:54.188 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 118512 2024-09-21T15:00:54.188 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.188 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:54.188 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.188 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:54.188 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893240+0000", 2024-09-21T15:00:54.188 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:54.188 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.188 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.188 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:54.189 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893240+0000", 2024-09-21T15:00:54.189 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:54.189 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.189 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.189 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:54.189 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893240+0000", 2024-09-21T15:00:54.189 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:54.189 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.189 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.189 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:54.189 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893242+0000", 2024-09-21T15:00:54.189 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.717e-06 2024-09-21T15:00:54.189 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.189 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.189 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:54.190 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893245+0000", 2024-09-21T15:00:54.190 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3380000000000001e-06 2024-09-21T15:00:54.190 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.190 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.190 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:54.190 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893248+0000", 2024-09-21T15:00:54.190 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9670000000000001e-06 2024-09-21T15:00:54.190 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.190 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.190 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:54.190 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.926868+0000", 2024-09-21T15:00:54.190 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.033619938000000002 2024-09-21T15:00:54.190 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.190 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.191 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:54.191 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.926964+0000", 2024-09-21T15:00:54.191 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5624999999999993e-05 2024-09-21T15:00:54.191 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.191 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.191 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:54.191 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.927052+0000", 2024-09-21T15:00:54.191 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8632000000000001e-05 2024-09-21T15:00:54.191 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:54.191 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:54.191 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:54.191 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.191 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.191 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:118513 3.12 3:4efbaa3f:::rbd_data.1136b1194d20.0000000000000006:58 [read 10133504~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:54.191 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.893251+0000", 2024-09-21T15:00:54.192 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 24.252031301999999, 2024-09-21T15:00:54.192 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.033992324999999997, 2024-09-21T15:00:54.192 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:54.192 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:54.192 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:54.192 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:54.192 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:54.192 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 118513 2024-09-21T15:00:54.192 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.192 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:54.192 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.192 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:54.192 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893251+0000", 2024-09-21T15:00:54.192 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:54.192 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.192 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.192 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:54.193 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893251+0000", 2024-09-21T15:00:54.193 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:54.193 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.193 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.193 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:54.193 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893250+0000", 2024-09-21T15:00:54.193 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:54.193 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.193 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.193 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:54.193 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893252+0000", 2024-09-21T15:00:54.193 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.666e-06 2024-09-21T15:00:54.193 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.193 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.193 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:54.194 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893255+0000", 2024-09-21T15:00:54.194 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.287e-06 2024-09-21T15:00:54.194 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.194 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.194 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:54.194 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893258+0000", 2024-09-21T15:00:54.194 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.027e-06 2024-09-21T15:00:54.194 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.194 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.194 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:54.194 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.927083+0000", 2024-09-21T15:00:54.194 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.033824262000000001 2024-09-21T15:00:54.194 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.194 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.195 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:54.195 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.927154+0000", 2024-09-21T15:00:54.195 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1099000000000005e-05 2024-09-21T15:00:54.195 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.195 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.195 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:54.195 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.927243+0000", 2024-09-21T15:00:54.195 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9266999999999998e-05 2024-09-21T15:00:54.195 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:54.195 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:54.195 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:54.195 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.195 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.195 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:118514 3.12 3:4efbaa3f:::rbd_data.1136b1194d20.0000000000000006:58 [read 10223616~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:54.195 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.893261+0000", 2024-09-21T15:00:54.195 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 24.252020942000001, 2024-09-21T15:00:54.196 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034165500000000001, 2024-09-21T15:00:54.196 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:54.196 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:54.196 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:54.196 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:54.196 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:54.196 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 118514 2024-09-21T15:00:54.196 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.196 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:54.196 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.196 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:54.196 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893261+0000", 2024-09-21T15:00:54.196 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:54.196 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.196 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.196 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:54.197 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893261+0000", 2024-09-21T15:00:54.197 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:54.197 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.197 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.197 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:54.197 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893261+0000", 2024-09-21T15:00:54.197 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:54.197 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.197 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.197 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:54.197 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893262+0000", 2024-09-21T15:00:54.197 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.787e-06 2024-09-21T15:00:54.197 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.197 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.197 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:54.197 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893267+0000", 2024-09-21T15:00:54.198 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6399999999999996e-06 2024-09-21T15:00:54.198 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.198 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.198 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:54.198 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893270+0000", 2024-09-21T15:00:54.198 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9610000000000001e-06 2024-09-21T15:00:54.198 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.198 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.198 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:54.198 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.927271+0000", 2024-09-21T15:00:54.198 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034001253000000002 2024-09-21T15:00:54.198 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.198 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.198 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:54.198 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.927344+0000", 2024-09-21T15:00:54.198 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2962999999999999e-05 2024-09-21T15:00:54.199 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.199 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.199 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:54.199 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.927427+0000", 2024-09-21T15:00:54.199 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2292000000000004e-05 2024-09-21T15:00:54.199 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:54.199 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:54.199 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:54.199 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.199 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.199 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:118515 3.12 3:4efbaa3f:::rbd_data.1136b1194d20.0000000000000006:58 [read 10313728~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:54.199 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.893273+0000", 2024-09-21T15:00:54.199 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 24.252008567000001, 2024-09-21T15:00:54.199 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034333437000000001, 2024-09-21T15:00:54.199 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:54.199 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:54.200 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:54.200 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:54.200 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:54.200 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 118515 2024-09-21T15:00:54.200 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.200 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:54.200 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.200 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:54.200 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893273+0000", 2024-09-21T15:00:54.200 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:54.200 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.200 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.200 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:54.200 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893273+0000", 2024-09-21T15:00:54.200 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:54.200 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.201 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.201 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:54.201 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893273+0000", 2024-09-21T15:00:54.201 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:54.201 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.201 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.201 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:54.201 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893275+0000", 2024-09-21T15:00:54.201 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6789999999999999e-06 2024-09-21T15:00:54.201 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.201 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.201 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:54.201 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893278+0000", 2024-09-21T15:00:54.201 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2440000000000002e-06 2024-09-21T15:00:54.201 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.201 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.201 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:54.202 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893282+0000", 2024-09-21T15:00:54.202 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.89e-06 2024-09-21T15:00:54.202 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.202 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.202 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:54.202 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.927454+0000", 2024-09-21T15:00:54.202 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034172077000000002 2024-09-21T15:00:54.202 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.202 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.202 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:54.202 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.927525+0000", 2024-09-21T15:00:54.202 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0963999999999993e-05 2024-09-21T15:00:54.202 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.202 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.202 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:54.202 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.927607+0000", 2024-09-21T15:00:54.203 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1868999999999998e-05 2024-09-21T15:00:54.203 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:54.203 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:54.203 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:54.203 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.203 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.203 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:118516 3.12 3:4efbaa3f:::rbd_data.1136b1194d20.0000000000000006:58 [read 10403840~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:54.203 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.893284+0000", 2024-09-21T15:00:54.203 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 24.251997626000001, 2024-09-21T15:00:54.203 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034589559999999998, 2024-09-21T15:00:54.203 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:54.203 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:54.203 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:54.203 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:54.203 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:54.203 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 118516 2024-09-21T15:00:54.204 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.204 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:54.204 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.204 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:54.204 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893284+0000", 2024-09-21T15:00:54.204 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:54.204 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.204 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.204 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:54.204 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893284+0000", 2024-09-21T15:00:54.204 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:54.204 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.204 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.204 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:54.204 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893284+0000", 2024-09-21T15:00:54.204 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:54.205 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.205 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.205 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:54.205 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893291+0000", 2024-09-21T15:00:54.205 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6200000000000001e-06 2024-09-21T15:00:54.205 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.205 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.205 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:54.205 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893294+0000", 2024-09-21T15:00:54.205 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6729999999999998e-06 2024-09-21T15:00:54.205 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.205 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.205 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:54.205 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893297+0000", 2024-09-21T15:00:54.205 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0510000000000001e-06 2024-09-21T15:00:54.205 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.206 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.206 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:54.206 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.927694+0000", 2024-09-21T15:00:54.206 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034397018000000001 2024-09-21T15:00:54.206 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.206 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.206 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:54.206 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.927767+0000", 2024-09-21T15:00:54.206 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2379000000000006e-05 2024-09-21T15:00:54.206 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.206 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.206 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:54.206 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.927874+0000", 2024-09-21T15:00:54.206 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010709000000000001 2024-09-21T15:00:54.206 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:54.206 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:54.207 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:54.207 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.207 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.207 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:118696 3.12 3:4efbaa3f:::rbd_data.1136b1194d20.0000000000000006:58 [read 9961472~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:54.207 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.902583+0000", 2024-09-21T15:00:54.207 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 24.242698578999999, 2024-09-21T15:00:54.207 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.033756517999999999, 2024-09-21T15:00:54.207 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:54.207 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:54.207 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:54.207 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:54.207 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:54.207 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 118696 2024-09-21T15:00:54.207 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.207 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:54.207 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.207 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:54.208 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902583+0000", 2024-09-21T15:00:54.208 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:54.208 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.208 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.208 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:54.208 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902583+0000", 2024-09-21T15:00:54.208 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:54.208 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.208 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.208 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:54.208 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902583+0000", 2024-09-21T15:00:54.208 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:54.208 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.208 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.208 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:54.208 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902585+0000", 2024-09-21T15:00:54.209 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7060000000000001e-06 2024-09-21T15:00:54.209 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.209 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.209 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:54.209 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902588+0000", 2024-09-21T15:00:54.209 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.128e-06 2024-09-21T15:00:54.209 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.209 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.209 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:54.209 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902592+0000", 2024-09-21T15:00:54.209 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0990000000000001e-06 2024-09-21T15:00:54.209 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.209 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.209 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:54.209 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.936161+0000", 2024-09-21T15:00:54.209 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.033569103000000003 2024-09-21T15:00:54.210 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.210 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.210 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:54.210 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.936241+0000", 2024-09-21T15:00:54.210 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9437000000000005e-05 2024-09-21T15:00:54.210 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.210 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.210 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:54.210 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.936340+0000", 2024-09-21T15:00:54.210 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9302999999999994e-05 2024-09-21T15:00:54.210 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:54.210 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:54.210 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:54.210 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.210 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.210 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:118697 3.12 3:4efbaa3f:::rbd_data.1136b1194d20.0000000000000006:58 [read 10051584~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:54.211 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.902595+0000", 2024-09-21T15:00:54.211 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 24.242686642999999, 2024-09-21T15:00:54.211 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.033875518, 2024-09-21T15:00:54.211 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:54.211 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:54.211 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:54.211 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:54.211 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:54.211 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 118697 2024-09-21T15:00:54.211 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.211 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:54.211 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.211 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:54.211 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902595+0000", 2024-09-21T15:00:54.211 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:54.212 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.212 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.212 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:54.212 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902595+0000", 2024-09-21T15:00:54.212 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:54.212 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.212 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.212 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:54.212 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902595+0000", 2024-09-21T15:00:54.212 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:54.212 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.212 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.212 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:54.212 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902597+0000", 2024-09-21T15:00:54.212 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7889999999999999e-06 2024-09-21T15:00:54.213 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.213 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.213 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:54.213 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902601+0000", 2024-09-21T15:00:54.213 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.16e-06 2024-09-21T15:00:54.213 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.213 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.213 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:54.213 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902605+0000", 2024-09-21T15:00:54.213 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7560000000000001e-06 2024-09-21T15:00:54.213 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.213 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.213 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:54.213 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.936370+0000", 2024-09-21T15:00:54.213 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.033765114999999998 2024-09-21T15:00:54.214 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.214 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.214 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:54.214 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.936413+0000", 2024-09-21T15:00:54.214 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3399000000000002e-05 2024-09-21T15:00:54.214 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.214 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.214 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:54.214 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.936471+0000", 2024-09-21T15:00:54.214 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7726000000000003e-05 2024-09-21T15:00:54.214 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:54.214 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:54.214 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:54.214 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.215 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.215 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:118698 3.12 3:4efbaa3f:::rbd_data.1136b1194d20.0000000000000006:58 [read 10141696~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:54.215 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.902607+0000", 2024-09-21T15:00:54.215 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 24.242675048999999, 2024-09-21T15:00:54.215 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034062306000000001, 2024-09-21T15:00:54.215 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:54.215 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:54.215 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:54.215 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:54.215 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:54.215 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 118698 2024-09-21T15:00:54.215 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.215 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:54.216 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.216 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:54.216 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902607+0000", 2024-09-21T15:00:54.216 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:54.216 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.216 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.216 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:54.216 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902607+0000", 2024-09-21T15:00:54.216 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:54.216 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.216 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.216 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:54.216 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902607+0000", 2024-09-21T15:00:54.216 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:54.217 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.217 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.217 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:54.217 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902608+0000", 2024-09-21T15:00:54.217 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5910000000000001e-06 2024-09-21T15:00:54.217 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.217 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.217 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:54.217 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902611+0000", 2024-09-21T15:00:54.217 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.965e-06 2024-09-21T15:00:54.217 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.217 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.217 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:54.217 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902615+0000", 2024-09-21T15:00:54.217 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0049999999999998e-06 2024-09-21T15:00:54.218 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.218 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.218 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:54.218 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.936501+0000", 2024-09-21T15:00:54.218 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.033885884999999998 2024-09-21T15:00:54.218 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.218 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.218 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:54.218 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.936584+0000", 2024-09-21T15:00:54.218 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2650000000000003e-05 2024-09-21T15:00:54.218 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.218 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.218 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:54.218 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.936669+0000", 2024-09-21T15:00:54.219 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5461000000000001e-05 2024-09-21T15:00:54.219 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:54.219 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:54.219 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:54.219 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.219 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.219 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:118699 3.12 3:4efbaa3f:::rbd_data.1136b1194d20.0000000000000006:58 [read 10231808~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:54.219 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.902625+0000", 2024-09-21T15:00:54.219 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 24.242656918000002, 2024-09-21T15:00:54.219 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034245128999999999, 2024-09-21T15:00:54.219 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:54.219 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:54.219 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:54.219 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:54.220 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:54.220 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 118699 2024-09-21T15:00:54.220 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.220 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:54.220 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.220 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:54.220 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902625+0000", 2024-09-21T15:00:54.220 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:54.220 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.220 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.220 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:54.220 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902625+0000", 2024-09-21T15:00:54.220 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:54.220 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.220 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.220 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:54.221 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902625+0000", 2024-09-21T15:00:54.221 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:54.221 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.221 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.221 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:54.221 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902627+0000", 2024-09-21T15:00:54.221 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7269999999999999e-06 2024-09-21T15:00:54.221 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.221 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.221 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:54.221 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902630+0000", 2024-09-21T15:00:54.221 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.5200000000000002e-06 2024-09-21T15:00:54.221 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.221 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.221 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:54.222 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902633+0000", 2024-09-21T15:00:54.222 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8399999999999999e-06 2024-09-21T15:00:54.222 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.222 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.222 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:54.222 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.936700+0000", 2024-09-21T15:00:54.222 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034066723 2024-09-21T15:00:54.222 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.222 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.222 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:54.222 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.936778+0000", 2024-09-21T15:00:54.222 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8713000000000003e-05 2024-09-21T15:00:54.223 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.223 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.223 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:54.223 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.936870+0000", 2024-09-21T15:00:54.223 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1885000000000005e-05 2024-09-21T15:00:54.223 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:54.223 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:54.223 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:54.223 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.223 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.223 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:118700 3.12 3:4efbaa3f:::rbd_data.1136b1194d20.0000000000000006:58 [read 10321920~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:54.223 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.902635+0000", 2024-09-21T15:00:54.223 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 24.242646710999999, 2024-09-21T15:00:54.223 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034423665999999999, 2024-09-21T15:00:54.224 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:54.224 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:54.224 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:54.224 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:54.224 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:54.224 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 118700 2024-09-21T15:00:54.224 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.224 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:54.224 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.224 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:54.224 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902635+0000", 2024-09-21T15:00:54.224 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:54.224 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.224 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.225 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:54.225 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902635+0000", 2024-09-21T15:00:54.225 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:54.225 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.225 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.225 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:54.225 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902635+0000", 2024-09-21T15:00:54.225 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:54.225 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.225 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.225 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:54.225 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902637+0000", 2024-09-21T15:00:54.225 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.734e-06 2024-09-21T15:00:54.225 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.226 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.226 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:54.226 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902640+0000", 2024-09-21T15:00:54.226 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9720000000000001e-06 2024-09-21T15:00:54.226 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.226 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.226 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:54.226 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902642+0000", 2024-09-21T15:00:54.226 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.616e-06 2024-09-21T15:00:54.226 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.226 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.226 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:54.226 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.936904+0000", 2024-09-21T15:00:54.227 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034261982000000003 2024-09-21T15:00:54.227 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.227 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.227 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:54.227 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.936974+0000", 2024-09-21T15:00:54.227 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9406000000000006e-05 2024-09-21T15:00:54.227 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.227 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.227 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:54.227 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.937059+0000", 2024-09-21T15:00:54.227 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5315999999999994e-05 2024-09-21T15:00:54.227 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:54.227 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:54.227 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:54.228 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.228 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.228 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:118701 3.12 3:4efbaa3f:::rbd_data.1136b1194d20.0000000000000006:58 [read 10412032~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:54.228 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.902645+0000", 2024-09-21T15:00:54.228 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 24.242637266999999, 2024-09-21T15:00:54.228 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034600791999999998, 2024-09-21T15:00:54.228 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:54.228 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:54.228 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:54.228 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:54.228 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:54.228 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 118701 2024-09-21T15:00:54.228 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.228 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:54.228 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.229 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:54.229 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902645+0000", 2024-09-21T15:00:54.229 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:54.229 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.229 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.229 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:54.229 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902645+0000", 2024-09-21T15:00:54.229 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:54.229 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.229 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.229 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:54.229 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902644+0000", 2024-09-21T15:00:54.229 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:54.229 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.229 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.230 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:54.230 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902646+0000", 2024-09-21T15:00:54.230 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6169999999999999e-06 2024-09-21T15:00:54.230 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.230 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.230 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:54.230 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902649+0000", 2024-09-21T15:00:54.230 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9450000000000002e-06 2024-09-21T15:00:54.230 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.230 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.230 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:54.230 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902652+0000", 2024-09-21T15:00:54.230 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7889999999999999e-06 2024-09-21T15:00:54.231 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.231 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.231 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:54.231 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.937086+0000", 2024-09-21T15:00:54.231 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034433832999999997 2024-09-21T15:00:54.231 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.231 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.231 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:54.231 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.937152+0000", 2024-09-21T15:00:54.231 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5934999999999998e-05 2024-09-21T15:00:54.231 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:54.231 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:54.231 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:54.231 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.937246+0000", 2024-09-21T15:00:54.231 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3923000000000002e-05 2024-09-21T15:00:54.232 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:54.232 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:54.232 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:54.232 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:54.232 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:54.232 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:54.233 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T15:00:54.234 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:00:54.229+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:00:54.252 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T15:00:54.334 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:00:54.329+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:00:54.346 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:54.346 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:00:54.347 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T15:00:54.347 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:54.356 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T15:00:54.405 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T15:00:54.425 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T15:00:54.434 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:00:54.431+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:00:54.446 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T15:00:54.464 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:54.464 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:00:54.464 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T15:00:54.464 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T15:00:54.464 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:54.468 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T15:00:54.475 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T15:00:54.501 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-21T15:00:54.523 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T15:00:54.535 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:00:54.533+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:00:54.546 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T15:00:54.570 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T15:00:54.573 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:00:54.573 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:00:54.573 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:00:54.573 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:00:54.573 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:00:54.573 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:00:54.574 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:00:54.574 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:00:54.586 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:54.586 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:54.596 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T15:00:54.639 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:00:54.637+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:00:54.710 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:54.710 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:54.721 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T15:00:54.737 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:00:54.735+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:00:54.830 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:54.830 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:54.838 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:00:54.835+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:00:54.840 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T15:00:54.903 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T15:00:54.939 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:00:54.937+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:00:54.998 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:54.998 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:55.009 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T15:00:55.039 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:00:55.035+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:00:55.050 INFO:teuthology.orchestra.run.smithi032.stdout:checking clone #32, image image_client.0-clone32 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent33 2024-09-21T15:00:55.118 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:55.118 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:55.129 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T15:00:55.141 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:00:55.137+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:00:55.179 INFO:teuthology.orchestra.run.smithi032.stdout:5162 read 0x14b8cc4 thru 0x14bb6be (0x29fb bytes) 2024-09-21T15:00:55.180 INFO:teuthology.orchestra.run.smithi032.stdout:5163 punch from 0x63fd539 to 0x6409035, (0xbafc bytes) 2024-09-21T15:00:55.194 INFO:teuthology.orchestra.run.smithi032.stdout:5164 punch from 0x280759b to 0x28097dd, (0x2242 bytes) 2024-09-21T15:00:55.194 INFO:teuthology.orchestra.run.smithi032.stdout:5165 read 0x30c9938 thru 0x30d877a (0xee43 bytes) 2024-09-21T15:00:55.194 INFO:teuthology.orchestra.run.smithi032.stdout:5166 write 0x66d976 thru 0x67892c (0xafb7 bytes) 2024-09-21T15:00:55.198 INFO:teuthology.orchestra.run.smithi032.stdout:5167 punch from 0x3586418 to 0x358c9c6, (0x65ae bytes) 2024-09-21T15:00:55.198 INFO:teuthology.orchestra.run.smithi032.stdout:5168 read 0x960ef44 thru 0x960fcf0 (0xdad bytes) 2024-09-21T15:00:55.198 INFO:teuthology.orchestra.run.smithi032.stdout:5170 write 0x180159d thru 0x1807bae (0x6612 bytes) 2024-09-21T15:00:55.207 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:55.208 INFO:teuthology.orchestra.run.smithi032.stdout:5173 write 0x2c2252b thru 0x2c3020c (0xdce2 bytes) 2024-09-21T15:00:55.215 INFO:teuthology.orchestra.run.smithi032.stdout:5174 read 0x65c0136 thru 0x65c2b22 (0x29ed bytes) 2024-09-21T15:00:55.215 INFO:teuthology.orchestra.run.smithi032.stdout:5175 write 0x48a662 thru 0x48f65b (0x4ffa bytes) 2024-09-21T15:00:55.231 INFO:teuthology.orchestra.run.smithi032.stdout:5176 trunc from 0xab11c78 to 0x2cfee61 2024-09-21T15:00:55.241 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:00:55.239+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:00:55.254 INFO:teuthology.orchestra.run.smithi032.stdout:5177 read 0x14fba5e thru 0x14fde92 (0x2435 bytes) 2024-09-21T15:00:55.265 INFO:teuthology.orchestra.run.smithi032.stdout:5179 write 0x3265986 thru 0x3271572 (0xbbed bytes) 2024-09-21T15:00:55.275 INFO:teuthology.orchestra.run.smithi032.stdout:5180 read 0x27cb5c7 thru 0x27d6944 (0xb37e bytes) 2024-09-21T15:00:55.302 INFO:teuthology.orchestra.run.smithi032.stdout:5181 writesame 0xb5f40a3 thru 0xb5f9ede data_size 0x4a(0x5e3c bytes) 2024-09-21T15:00:55.306 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:55.306 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:55.313 INFO:teuthology.orchestra.run.smithi032.stdout:5182 trunc from 0xb5f9edf to 0x44b9732 2024-09-21T15:00:55.318 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T15:00:55.328 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:55.328 INFO:teuthology.orchestra.run.smithi032.stdout:5185 read 0xa7d002 thru 0xa83288 (0x6287 bytes) 2024-09-21T15:00:55.329 INFO:teuthology.orchestra.run.smithi032.stdout:5186 punch from 0x164528f to 0x1648846, (0x35b7 bytes) 2024-09-21T15:00:55.329 INFO:teuthology.orchestra.run.smithi032.stdout:5187 writesame 0xffecdc thru 0x1004bb8 data_size 0x5(0x5edd bytes) 2024-09-21T15:00:55.342 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:00:55.341+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:00:55.358 INFO:teuthology.orchestra.run.smithi032.stdout:5188 trunc from 0x44b9732 to 0xde769bf 2024-09-21T15:00:55.392 INFO:teuthology.orchestra.run.smithi032.stdout:5189 trunc from 0xde769bf to 0x188a193 2024-09-21T15:00:55.416 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:55.437 INFO:teuthology.orchestra.run.smithi032.stdout:5193 write 0x6d3dc72 thru 0x6d44a18 (0x6da7 bytes) 2024-09-21T15:00:55.443 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:55.443 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:55.443 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:00:55.441+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:00:55.445 INFO:teuthology.orchestra.run.smithi032.stdout:5195 write 0x317f937 thru 0x318013b (0x805 bytes) 2024-09-21T15:00:55.452 INFO:teuthology.orchestra.run.smithi032.stdout:5196 read 0x17c8959 thru 0x17cfe6a (0x7512 bytes) 2024-09-21T15:00:55.453 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:55.454 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T15:00:55.474 INFO:teuthology.orchestra.run.smithi032.stdout:5198 write 0xb9e35b6 thru 0xb9e6196 (0x2be1 bytes) 2024-09-21T15:00:55.481 INFO:teuthology.orchestra.run.smithi032.stdout:5200 punch from 0x1b16500 to 0x1b245fb, (0xe0fb bytes) 2024-09-21T15:00:55.486 INFO:teuthology.orchestra.run.smithi032.stdout:5201 trunc from 0xb9e6197 to 0x8e9769d 2024-09-21T15:00:55.503 INFO:teuthology.orchestra.run.smithi032.stdout:5202 punch from 0x16290da to 0x16383d9, (0xf2ff bytes) 2024-09-21T15:00:55.522 INFO:teuthology.orchestra.run.smithi032.stdout:5203 write 0xd1352e5 thru 0xd13b7b3 (0x64cf bytes) 2024-09-21T15:00:55.533 INFO:teuthology.orchestra.run.smithi032.stdout:5204 writesame 0x3cf2ff6 thru 0x3cf4ef2 data_size 0x1(0x1efd bytes) 2024-09-21T15:00:55.542 INFO:teuthology.orchestra.run.smithi032.stdout:5207 read 0x8fbbaf8 thru 0x8fc0275 (0x477e bytes) 2024-09-21T15:00:55.542 INFO:teuthology.orchestra.run.smithi032.stdout:5208 punch from 0xa387606 to 0xa3902c8, (0x8cc2 bytes) 2024-09-21T15:00:55.542 INFO:teuthology.orchestra.run.smithi032.stdout:5209 read 0xba9ab5f thru 0xbaa2ad8 (0x7f7a bytes) 2024-09-21T15:00:55.542 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:55.542 INFO:teuthology.orchestra.run.smithi032.stdout:5211 read 0x22e7c0a thru 0x22efff7 (0x83ee bytes) 2024-09-21T15:00:55.543 INFO:teuthology.orchestra.run.smithi032.stdout:5212 writesame 0x557cfbf thru 0x5587c27 data_size 0x17(0xac69 bytes) 2024-09-21T15:00:55.544 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:00:55.539+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:00:55.550 INFO:teuthology.orchestra.run.smithi032.stdout:5213 writesame 0x4643de2 thru 0x464f17a data_size 0x1(0xb399 bytes) 2024-09-21T15:00:55.570 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:55.571 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:55.579 INFO:teuthology.orchestra.run.smithi032.stdout:5214 punch from 0x1ea4ff0 to 0x1eb33f5, (0xe405 bytes) 2024-09-21T15:00:55.579 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:55.579 INFO:teuthology.orchestra.run.smithi032.stdout:5216 writesame 0xa7695a5 thru 0xa76a7e3 data_size 0x9(0x123f bytes) 2024-09-21T15:00:55.580 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T15:00:55.586 INFO:teuthology.orchestra.run.smithi032.stdout:5217 punch from 0xa0052ea to 0xa00fb2c, (0xa842 bytes) 2024-09-21T15:00:55.586 INFO:teuthology.orchestra.run.smithi032.stdout:5219 punch from 0x68606e2 to 0x6868950, (0x826e bytes) 2024-09-21T15:00:55.586 INFO:teuthology.orchestra.run.smithi032.stdout:5220 read 0xcffa818 thru 0xcffc217 (0x1a00 bytes) 2024-09-21T15:00:55.587 INFO:teuthology.orchestra.run.smithi032.stdout:5221 punch from 0x699b2f to 0x6a111d, (0x75ee bytes) 2024-09-21T15:00:55.587 INFO:teuthology.orchestra.run.smithi032.stdout:5223 write 0x896cb00 thru 0x89741cf (0x76d0 bytes) 2024-09-21T15:00:55.595 INFO:teuthology.orchestra.run.smithi032.stdout:5225 read 0x830628c thru 0x830a773 (0x44e8 bytes) 2024-09-21T15:00:55.596 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:55.596 INFO:teuthology.orchestra.run.smithi032.stdout:5227 writesame 0x1c55022 thru 0x1c58207 data_size 0x3(0x31e6 bytes) 2024-09-21T15:00:55.604 INFO:teuthology.orchestra.run.smithi032.stdout:5228 punch from 0x8ecc9e5 to 0x8ecf899, (0x2eb4 bytes) 2024-09-21T15:00:55.618 INFO:teuthology.orchestra.run.smithi032.stdout:5229 write 0xee33960 thru 0xee3547d (0x1b1e bytes) 2024-09-21T15:00:55.627 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:55.627 INFO:teuthology.orchestra.run.smithi032.stdout:5231 read 0x31e0f1b thru 0x31e28ea (0x19d0 bytes) 2024-09-21T15:00:55.628 INFO:teuthology.orchestra.run.smithi032.stdout:5232 write 0x8c437c6 thru 0x8c43ced (0x528 bytes) 2024-09-21T15:00:55.635 INFO:teuthology.orchestra.run.smithi032.stdout:5233 punch from 0x212b8a3 to 0x212db31, (0x228e bytes) 2024-09-21T15:00:55.636 INFO:teuthology.orchestra.run.smithi032.stdout:5234 read 0xd3d47ea thru 0xd3deff1 (0xa808 bytes) 2024-09-21T15:00:55.636 INFO:teuthology.orchestra.run.smithi032.stdout:5235 writesame 0xb4c4b47 thru 0xb4d1f26 data_size 0x10(0xd3e0 bytes) 2024-09-21T15:00:55.644 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:00:55.641+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:00:55.650 INFO:teuthology.orchestra.run.smithi032.stdout:5238 trunc from 0xee3547e to 0xd7f6ec0 2024-09-21T15:00:55.674 INFO:teuthology.orchestra.run.smithi032.stdout:5239 read 0x49f4889 thru 0x4a038ae (0xf026 bytes) 2024-09-21T15:00:55.674 INFO:teuthology.orchestra.run.smithi032.stdout:5240 read 0x2b29713 thru 0x2b340fe (0xa9ec bytes) 2024-09-21T15:00:55.674 INFO:teuthology.orchestra.run.smithi032.stdout:5241 write 0x2830870 thru 0x283a2c5 (0x9a56 bytes) 2024-09-21T15:00:55.683 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:55.683 INFO:teuthology.orchestra.run.smithi032.stdout:5243 write 0x151fb28 thru 0x15205ad (0xa86 bytes) 2024-09-21T15:00:55.686 INFO:teuthology.orchestra.run.smithi032.stdout:5246 write 0x6160064 thru 0x6160fee (0xf8b bytes) 2024-09-21T15:00:55.693 INFO:teuthology.orchestra.run.smithi032.stdout:5248 write 0xa46b0f1 thru 0xa46bf0e (0xe1e bytes) 2024-09-21T15:00:55.699 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:55.699 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:55.713 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T15:00:55.715 INFO:teuthology.orchestra.run.smithi032.stdout:5249 trunc from 0xd7f6ec0 to 0x2b2eda5 2024-09-21T15:00:55.740 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:55.741 INFO:teuthology.orchestra.run.smithi032.stdout:5251 read 0x8d75ca thru 0x8e0013 (0x8a4a bytes) 2024-09-21T15:00:55.741 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:55.745 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:00:55.743+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:00:55.752 INFO:teuthology.orchestra.run.smithi032.stdout:5255 writesame 0x3c77ac7 thru 0x3c81318 data_size 0x86(0x9852 bytes) 2024-09-21T15:00:55.764 INFO:teuthology.orchestra.run.smithi032.stdout:5259 trunc from 0x3c81319 to 0xb23d17b 2024-09-21T15:00:55.773 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:55.774 INFO:teuthology.orchestra.run.smithi032.stdout:5262 punch from 0x5442096 to 0x5447037, (0x4fa1 bytes) 2024-09-21T15:00:55.774 INFO:teuthology.orchestra.run.smithi032.stdout:5264 write 0x679dfe5 thru 0x67a2173 (0x418f bytes) 2024-09-21T15:00:55.785 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:55.785 INFO:teuthology.orchestra.run.smithi032.stdout:5266 write 0x657ba0c thru 0x657eee9 (0x34de bytes) 2024-09-21T15:00:55.789 INFO:teuthology.orchestra.run.smithi032.stdout:5268 writesame 0x452b642 thru 0x45363ea data_size 0x1d(0xada9 bytes) 2024-09-21T15:00:55.798 INFO:teuthology.orchestra.run.smithi032.stdout:5269 punch from 0x47faaa to 0x480021, (0x577 bytes) 2024-09-21T15:00:55.800 INFO:teuthology.orchestra.run.smithi032.stdout:5270 trunc from 0xb23d17b to 0xc560238 2024-09-21T15:00:55.813 INFO:teuthology.orchestra.run.smithi032.stdout:5271 writesame 0x5178caf thru 0x517d3ee data_size 0x40(0x4740 bytes) 2024-09-21T15:00:55.823 INFO:teuthology.orchestra.run.smithi032.stdout:5273 writesame 0x7cf1b60 thru 0x7cf7645 data_size 0x1a(0x5ae6 bytes) 2024-09-21T15:00:55.825 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:55.825 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:55.837 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T15:00:55.839 INFO:teuthology.orchestra.run.smithi032.stdout:5274 trunc from 0xc560238 to 0x6ecb59f 2024-09-21T15:00:55.846 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:00:55.841+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:00:55.859 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:55.859 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:55.859 INFO:teuthology.orchestra.run.smithi032.stdout:5279 read 0x2f69910 thru 0x2f6c2d6 (0x29c7 bytes) 2024-09-21T15:00:55.859 INFO:teuthology.orchestra.run.smithi032.stdout:5281 read 0x2550bb7 thru 0x25568c8 (0x5d12 bytes) 2024-09-21T15:00:55.859 INFO:teuthology.orchestra.run.smithi032.stdout:5282 punch from 0xf482dd to 0xf4de81, (0x5ba4 bytes) 2024-09-21T15:00:55.859 INFO:teuthology.orchestra.run.smithi032.stdout:5283 read 0x29fd09d thru 0x2a07dc0 (0xad24 bytes) 2024-09-21T15:00:55.860 INFO:teuthology.orchestra.run.smithi032.stdout:5286 read 0x43c4dec thru 0x43c7718 (0x292d bytes) 2024-09-21T15:00:55.870 INFO:teuthology.orchestra.run.smithi032.stdout:5287 trunc from 0x6ecb59f to 0x11adbaf 2024-09-21T15:00:55.896 INFO:teuthology.orchestra.run.smithi032.stdout:5288 read 0xab47b thru 0xb56d9 (0xa25f bytes) 2024-09-21T15:00:55.914 INFO:teuthology.orchestra.run.smithi032.stdout:5289 write 0x5b0f7f9 thru 0x5b180fb (0x8903 bytes) 2024-09-21T15:00:55.933 INFO:teuthology.orchestra.run.smithi032.stdout:5290 trunc from 0x5b180fc to 0xd1a70c2 2024-09-21T15:00:55.947 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:00:55.945+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:00:55.949 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:55.950 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:55.963 INFO:teuthology.orchestra.run.smithi032.stdout:5291 trunc from 0xd1a70c2 to 0x16afaa5 2024-09-21T15:00:55.964 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T15:00:56.008 INFO:teuthology.orchestra.run.smithi032.stdout:5292 write 0x4e19efb thru 0x4e277f5 (0xd8fb bytes) 2024-09-21T15:00:56.017 INFO:teuthology.orchestra.run.smithi032.stdout:5293 read 0x175c251 thru 0x1766d10 (0xaac0 bytes) 2024-09-21T15:00:56.025 INFO:teuthology.orchestra.run.smithi032.stdout:5294 trunc from 0x4e277f6 to 0x8a6dfa6 2024-09-21T15:00:56.047 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:00:56.043+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:00:56.056 INFO:teuthology.orchestra.run.smithi032.stdout:5296 write 0xcaee6db thru 0xcafe390 (0xfcb6 bytes) 2024-09-21T15:00:56.063 INFO:teuthology.orchestra.run.smithi032.stdout:5297 read 0x88c0429 thru 0x88c2fbd (0x2b95 bytes) 2024-09-21T15:00:56.063 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:56.077 INFO:teuthology.orchestra.run.smithi032.stdout:5300 write 0xdf3ce96 thru 0xdf4a611 (0xd77c bytes) 2024-09-21T15:00:56.098 INFO:teuthology.orchestra.run.smithi032.stdout:5301 trunc from 0xdf4a612 to 0x7e293e7 2024-09-21T15:00:56.119 INFO:teuthology.orchestra.run.smithi032.stdout:5302 write 0x64c9a71 thru 0x64d4beb (0xb17b bytes) 2024-09-21T15:00:56.126 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:56.126 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:56.127 INFO:teuthology.orchestra.run.smithi032.stdout:5303 read 0x59ad6f9 thru 0x59bb6e9 (0xdff1 bytes) 2024-09-21T15:00:56.127 INFO:teuthology.orchestra.run.smithi032.stdout:5304 punch from 0x2542cb9 to 0x2546585, (0x38cc bytes) 2024-09-21T15:00:56.138 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T15:00:56.148 INFO:teuthology.orchestra.run.smithi032.stdout:5306 write 0xc0beb2d thru 0xc0c2513 (0x39e7 bytes) 2024-09-21T15:00:56.148 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:00:56.145+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:00:56.159 INFO:teuthology.orchestra.run.smithi032.stdout:5307 write 0xbb8d090 thru 0xbb93fb4 (0x6f25 bytes) 2024-09-21T15:00:56.166 INFO:teuthology.orchestra.run.smithi032.stderr:nodeep-scrub is set 2024-09-21T15:00:56.188 INFO:teuthology.orchestra.run.smithi032.stdout:5309 write 0xe04fee2 thru 0xe05ac5f (0xad7e bytes) 2024-09-21T15:00:56.195 INFO:teuthology.orchestra.run.smithi032.stdout:5312 write 0x383aa1 thru 0x38d87a (0x9dda bytes) 2024-09-21T15:00:56.203 INFO:teuthology.orchestra.run.smithi032.stdout:5314 write 0x8c8e6a9 thru 0x8c93bc6 (0x551e bytes) 2024-09-21T15:00:56.211 INFO:teuthology.orchestra.run.smithi032.stdout:5317 read 0x9a86855 thru 0x9a8c152 (0x58fe bytes) 2024-09-21T15:00:56.211 INFO:teuthology.orchestra.run.smithi032.stdout:5318 read 0xafdc50e thru 0xafe1920 (0x5413 bytes) 2024-09-21T15:00:56.211 INFO:teuthology.orchestra.run.smithi032.stdout:5320 write 0x9660c44 thru 0x966de0d (0xd1ca bytes) 2024-09-21T15:00:56.220 INFO:teuthology.orchestra.run.smithi032.stdout:5321 trunc from 0xe05ac60 to 0xd83e3fd 2024-09-21T15:00:56.243 INFO:teuthology.orchestra.run.smithi032.stdout:5322 trunc from 0xd83e3fd to 0xc14c026 2024-09-21T15:00:56.249 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:00:56.245+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:00:56.259 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:56.259 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:56.272 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T15:00:56.274 INFO:teuthology.orchestra.run.smithi032.stdout:5323 trunc from 0xc14c026 to 0x40c25d5 2024-09-21T15:00:56.294 INFO:teuthology.orchestra.run.smithi032.stdout:5324 read 0xfcb311 thru 0xfd611e (0xae0e bytes) 2024-09-21T15:00:56.295 INFO:teuthology.orchestra.run.smithi032.stdout:5325 punch from 0x893a83 to 0x8949ff, (0xf7c bytes) 2024-09-21T15:00:56.320 INFO:teuthology.orchestra.run.smithi032.stdout:5326 write 0xb42ebc7 thru 0xb430234 (0x166e bytes) 2024-09-21T15:00:56.330 INFO:teuthology.orchestra.run.smithi032.stdout:5329 clone 35 order 18 su 16384 sc 14 2024-09-21T15:00:56.349 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:00:56.345+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:00:56.390 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:56.390 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:56.400 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T15:00:56.450 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:00:56.445+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:00:56.507 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:56.507 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:56.516 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T15:00:56.551 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:00:56.549+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:00:56.586 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T15:00:56.624 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:56.624 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:56.634 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T15:00:56.651 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:00:56.647+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:00:56.740 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T15:00:56.752 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:00:56.749+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:00:56.759 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T15:00:56.779 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T15:00:56.785 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:56.785 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:56.795 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T15:00:56.800 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T15:00:56.824 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-21T15:00:56.847 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T15:00:56.852 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:00:56.849+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:00:56.869 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T15:00:56.891 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T15:00:56.894 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:00:56.894 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:00:56.894 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:00:56.894 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:00:56.894 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:00:56.895 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:00:56.895 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:00:56.895 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:00:56.909 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:56.909 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:00:56.909 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T15:00:56.909 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T15:00:56.909 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:56.919 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T15:00:56.954 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:00:56.951+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:00:57.055 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:00:57.053+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:00:57.077 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:57.077 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T15:00:57.077 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T15:00:57.077 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T15:00:57.077 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.077 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45839 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:57.077 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159313+0000", 2024-09-21T15:00:57.078 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 106.914374022, 2024-09-21T15:00:57.078 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068238699, 2024-09-21T15:00:57.078 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:57.078 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:57.078 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:57.078 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:57.078 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:57.078 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45839 2024-09-21T15:00:57.078 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.078 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:57.078 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.078 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:57.078 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159313+0000", 2024-09-21T15:00:57.078 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:57.079 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.079 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.079 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:57.079 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159313+0000", 2024-09-21T15:00:57.079 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:57.079 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.079 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.079 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:57.079 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159313+0000", 2024-09-21T15:00:57.079 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:57.079 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.079 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.079 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:57.079 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159316+0000", 2024-09-21T15:00:57.080 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.033e-06 2024-09-21T15:00:57.080 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.080 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.080 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:57.080 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159322+0000", 2024-09-21T15:00:57.080 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1820000000000003e-06 2024-09-21T15:00:57.080 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.080 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.080 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:57.080 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159327+0000", 2024-09-21T15:00:57.080 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.062e-06 2024-09-21T15:00:57.080 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.080 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.080 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:57.081 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227544+0000", 2024-09-21T15:00:57.081 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068217142999999994 2024-09-21T15:00:57.081 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.081 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.081 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:57.081 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227552+0000", 2024-09-21T15:00:57.081 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6260000000000003e-06 2024-09-21T15:00:57.081 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:57.081 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:57.081 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:57.081 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.081 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.081 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45840 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:57.081 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159332+0000", 2024-09-21T15:00:57.081 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 106.91435572499999, 2024-09-21T15:00:57.082 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068262046000000007, 2024-09-21T15:00:57.082 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:57.082 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:57.082 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:57.082 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:57.082 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:57.082 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45840 2024-09-21T15:00:57.082 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.082 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:57.082 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.082 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:57.082 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159332+0000", 2024-09-21T15:00:57.082 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:57.082 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.083 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.083 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:57.083 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159332+0000", 2024-09-21T15:00:57.083 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:57.083 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.083 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.083 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:57.083 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159331+0000", 2024-09-21T15:00:57.083 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:57.083 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.083 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.083 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:57.083 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159351+0000", 2024-09-21T15:00:57.083 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0064999999999998e-05 2024-09-21T15:00:57.083 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.084 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.084 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:57.084 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159367+0000", 2024-09-21T15:00:57.084 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5308e-05 2024-09-21T15:00:57.084 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.084 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.084 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:57.084 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159373+0000", 2024-09-21T15:00:57.084 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1840000000000004e-06 2024-09-21T15:00:57.084 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.084 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.084 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:57.084 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227584+0000", 2024-09-21T15:00:57.084 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068211401000000005 2024-09-21T15:00:57.084 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.085 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.085 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:57.085 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227594+0000", 2024-09-21T15:00:57.085 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4120000000000007e-06 2024-09-21T15:00:57.085 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:57.085 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:57.085 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:57.085 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.085 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.085 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45841 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:57.085 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159379+0000", 2024-09-21T15:00:57.085 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 106.914308569, 2024-09-21T15:00:57.085 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068249039999999997, 2024-09-21T15:00:57.085 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:57.086 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:57.086 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:57.086 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:57.086 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:57.086 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45841 2024-09-21T15:00:57.086 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.086 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:57.086 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.086 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:57.086 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159379+0000", 2024-09-21T15:00:57.086 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:57.086 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.086 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.086 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:57.087 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159379+0000", 2024-09-21T15:00:57.087 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:57.087 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.087 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.087 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:57.087 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159378+0000", 2024-09-21T15:00:57.087 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:57.087 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.087 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.087 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:57.087 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159382+0000", 2024-09-21T15:00:57.087 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.641e-06 2024-09-21T15:00:57.087 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.087 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.088 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:57.088 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159421+0000", 2024-09-21T15:00:57.088 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9439e-05 2024-09-21T15:00:57.088 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.088 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.088 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:57.088 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159430+0000", 2024-09-21T15:00:57.088 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8429999999999999e-06 2024-09-21T15:00:57.088 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.088 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.088 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:57.088 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227617+0000", 2024-09-21T15:00:57.088 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068187084999999995 2024-09-21T15:00:57.088 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.089 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.089 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:57.089 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227628+0000", 2024-09-21T15:00:57.089 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0507e-05 2024-09-21T15:00:57.089 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:57.089 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:57.089 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:57.089 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.089 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.089 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45844 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:57.089 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159499+0000", 2024-09-21T15:00:57.089 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 106.91418818299999, 2024-09-21T15:00:57.089 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068255415, 2024-09-21T15:00:57.089 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:57.089 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:57.089 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:57.090 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:57.090 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:57.090 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45844 2024-09-21T15:00:57.090 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.090 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:57.090 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.090 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:57.090 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159499+0000", 2024-09-21T15:00:57.090 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:57.090 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.090 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.090 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:57.090 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159499+0000", 2024-09-21T15:00:57.090 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:57.090 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.091 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.091 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:57.091 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159499+0000", 2024-09-21T15:00:57.091 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:57.091 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.091 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.091 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:57.091 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159503+0000", 2024-09-21T15:00:57.091 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0999999999999997e-06 2024-09-21T15:00:57.091 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.091 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.091 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:57.091 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159512+0000", 2024-09-21T15:00:57.091 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5850000000000006e-06 2024-09-21T15:00:57.092 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.092 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.092 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:57.092 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159518+0000", 2024-09-21T15:00:57.092 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0909999999999998e-06 2024-09-21T15:00:57.092 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.092 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.092 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:57.092 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227743+0000", 2024-09-21T15:00:57.092 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068224536000000002 2024-09-21T15:00:57.092 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.092 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.092 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:57.092 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227755+0000", 2024-09-21T15:00:57.092 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1659e-05 2024-09-21T15:00:57.093 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:57.093 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:57.093 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:57.093 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.093 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.093 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45845 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:57.093 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159524+0000", 2024-09-21T15:00:57.093 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 106.914163823, 2024-09-21T15:00:57.093 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068267194000000003, 2024-09-21T15:00:57.093 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:57.093 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:57.093 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:57.093 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:57.093 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:57.093 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45845 2024-09-21T15:00:57.094 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.094 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:57.094 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.094 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:57.094 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159524+0000", 2024-09-21T15:00:57.094 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:57.094 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.094 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.094 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:57.094 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159524+0000", 2024-09-21T15:00:57.094 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:57.094 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.094 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.094 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:57.094 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159523+0000", 2024-09-21T15:00:57.094 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:57.095 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.095 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.095 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:57.095 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159526+0000", 2024-09-21T15:00:57.095 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1980000000000001e-06 2024-09-21T15:00:57.095 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.095 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.095 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:57.095 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159533+0000", 2024-09-21T15:00:57.095 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.348e-06 2024-09-21T15:00:57.095 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.095 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.095 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:57.095 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159546+0000", 2024-09-21T15:00:57.096 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3464e-05 2024-09-21T15:00:57.096 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.096 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.096 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:57.096 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227783+0000", 2024-09-21T15:00:57.096 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068236926000000003 2024-09-21T15:00:57.096 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.096 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.096 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:57.096 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227791+0000", 2024-09-21T15:00:57.096 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6669999999999996e-06 2024-09-21T15:00:57.096 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:57.096 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:57.096 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:57.096 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.097 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.097 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45846 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:57.097 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159584+0000", 2024-09-21T15:00:57.097 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 106.914103181, 2024-09-21T15:00:57.097 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068254767999999993, 2024-09-21T15:00:57.097 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:57.097 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:57.097 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:57.097 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:57.097 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:57.097 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45846 2024-09-21T15:00:57.097 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.097 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:57.097 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.097 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:57.098 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159584+0000", 2024-09-21T15:00:57.098 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:57.098 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.098 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.098 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:57.098 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159584+0000", 2024-09-21T15:00:57.098 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:57.098 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.098 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.098 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:57.098 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159583+0000", 2024-09-21T15:00:57.098 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:57.098 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.098 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.099 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:57.099 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159588+0000", 2024-09-21T15:00:57.099 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1330000000000001e-06 2024-09-21T15:00:57.099 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.099 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.099 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:57.099 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159596+0000", 2024-09-21T15:00:57.099 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5469999999999997e-06 2024-09-21T15:00:57.099 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.099 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.099 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:57.099 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159609+0000", 2024-09-21T15:00:57.099 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2785999999999999e-05 2024-09-21T15:00:57.099 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.099 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.100 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:57.100 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227830+0000", 2024-09-21T15:00:57.100 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068220879999999998 2024-09-21T15:00:57.100 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.100 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.100 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:57.100 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227839+0000", 2024-09-21T15:00:57.100 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1239999999999994e-06 2024-09-21T15:00:57.100 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:57.100 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:57.100 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:57.100 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.100 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.100 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45847 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:57.100 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159614+0000", 2024-09-21T15:00:57.100 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 106.914073105, 2024-09-21T15:00:57.101 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068260639999999997, 2024-09-21T15:00:57.101 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:57.101 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:57.101 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:57.101 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:57.101 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:57.101 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45847 2024-09-21T15:00:57.101 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.101 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:57.101 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.101 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:57.101 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159614+0000", 2024-09-21T15:00:57.101 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:57.101 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.101 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.102 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:57.102 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159614+0000", 2024-09-21T15:00:57.102 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:57.102 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.102 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.102 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:57.102 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159614+0000", 2024-09-21T15:00:57.102 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:57.102 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.102 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.102 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:57.102 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159617+0000", 2024-09-21T15:00:57.102 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2909999999999999e-06 2024-09-21T15:00:57.102 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.102 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.103 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:57.103 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159623+0000", 2024-09-21T15:00:57.103 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.511e-06 2024-09-21T15:00:57.103 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.103 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.103 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:57.103 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159631+0000", 2024-09-21T15:00:57.103 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.571e-06 2024-09-21T15:00:57.103 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.103 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.103 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:57.103 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227866+0000", 2024-09-21T15:00:57.103 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068235136000000002 2024-09-21T15:00:57.103 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.104 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.104 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:57.104 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227875+0000", 2024-09-21T15:00:57.104 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7199999999999995e-06 2024-09-21T15:00:57.104 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:57.104 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:57.104 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:57.104 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.104 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.104 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45848 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:57.104 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159636+0000", 2024-09-21T15:00:57.104 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 106.914051325, 2024-09-21T15:00:57.104 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068267283999999998, 2024-09-21T15:00:57.104 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:57.105 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:57.105 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:57.105 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:57.105 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:57.105 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45848 2024-09-21T15:00:57.105 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.105 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:57.105 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.105 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:57.105 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159636+0000", 2024-09-21T15:00:57.105 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:57.105 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.105 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.105 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:57.105 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159636+0000", 2024-09-21T15:00:57.106 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:57.106 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.106 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.106 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:57.106 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159636+0000", 2024-09-21T15:00:57.106 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:57.106 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.106 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.106 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:57.106 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159638+0000", 2024-09-21T15:00:57.106 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6560000000000001e-06 2024-09-21T15:00:57.106 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.106 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.106 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:57.106 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159644+0000", 2024-09-21T15:00:57.106 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0839999999999997e-06 2024-09-21T15:00:57.107 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.107 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.107 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:57.107 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159649+0000", 2024-09-21T15:00:57.107 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7500000000000003e-06 2024-09-21T15:00:57.107 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.107 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.107 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:57.107 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227892+0000", 2024-09-21T15:00:57.107 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068243315999999998 2024-09-21T15:00:57.107 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.107 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.107 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:57.107 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227903+0000", 2024-09-21T15:00:57.107 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0835999999999999e-05 2024-09-21T15:00:57.108 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:57.108 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:57.108 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:57.108 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.108 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.108 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45849 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:57.108 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159667+0000", 2024-09-21T15:00:57.108 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 106.914020511, 2024-09-21T15:00:57.108 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068291051000000005, 2024-09-21T15:00:57.108 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:57.108 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:57.108 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:57.108 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:57.108 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:57.109 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45849 2024-09-21T15:00:57.109 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.109 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:57.109 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.109 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:57.109 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159667+0000", 2024-09-21T15:00:57.109 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:57.109 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.109 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.109 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:57.109 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159667+0000", 2024-09-21T15:00:57.109 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:57.109 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.109 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.109 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:57.110 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159666+0000", 2024-09-21T15:00:57.110 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:57.110 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.110 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.110 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:57.110 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159670+0000", 2024-09-21T15:00:57.110 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.427e-06 2024-09-21T15:00:57.110 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.110 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.110 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:57.110 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159678+0000", 2024-09-21T15:00:57.110 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9869999999999999e-06 2024-09-21T15:00:57.110 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.110 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.111 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:57.111 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159688+0000", 2024-09-21T15:00:57.111 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0631999999999999e-05 2024-09-21T15:00:57.111 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.111 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.111 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:57.111 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227950+0000", 2024-09-21T15:00:57.111 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068261798999999998 2024-09-21T15:00:57.111 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.111 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.111 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:57.111 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227958+0000", 2024-09-21T15:00:57.111 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7510000000000008e-06 2024-09-21T15:00:57.111 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:57.111 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:57.112 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:57.112 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.112 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.112 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45850 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:57.112 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159695+0000", 2024-09-21T15:00:57.112 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 106.9139928, 2024-09-21T15:00:57.112 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068302999000000003, 2024-09-21T15:00:57.112 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:57.112 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:57.112 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:57.112 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:57.112 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:57.112 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45850 2024-09-21T15:00:57.112 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.113 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:57.113 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.113 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:57.113 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159695+0000", 2024-09-21T15:00:57.113 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:57.113 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.113 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.113 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:57.113 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159695+0000", 2024-09-21T15:00:57.113 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:57.113 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.113 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.113 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:57.113 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159694+0000", 2024-09-21T15:00:57.113 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:57.114 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.114 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.114 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:57.114 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159697+0000", 2024-09-21T15:00:57.114 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.258e-06 2024-09-21T15:00:57.114 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.114 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.114 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:57.114 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159704+0000", 2024-09-21T15:00:57.114 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7660000000000001e-06 2024-09-21T15:00:57.114 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.114 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.114 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:57.114 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159713+0000", 2024-09-21T15:00:57.115 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1779999999999992e-06 2024-09-21T15:00:57.115 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.115 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.115 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:57.115 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227989+0000", 2024-09-21T15:00:57.115 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068276015999999995 2024-09-21T15:00:57.115 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.115 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.115 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:57.115 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.227998+0000", 2024-09-21T15:00:57.115 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2050000000000002e-06 2024-09-21T15:00:57.115 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:57.115 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:57.115 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:57.115 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.116 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.116 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45851 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:57.116 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159719+0000", 2024-09-21T15:00:57.116 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 106.91396866700001, 2024-09-21T15:00:57.116 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068320327, 2024-09-21T15:00:57.116 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:57.116 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:57.116 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:57.116 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:57.116 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:57.116 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45851 2024-09-21T15:00:57.116 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.116 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:57.116 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.117 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:57.117 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159719+0000", 2024-09-21T15:00:57.117 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:57.117 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.117 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.117 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:57.117 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159719+0000", 2024-09-21T15:00:57.117 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:57.117 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.117 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.117 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:57.117 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159718+0000", 2024-09-21T15:00:57.117 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:57.117 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.118 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.118 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:57.118 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159721+0000", 2024-09-21T15:00:57.118 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2169999999999999e-06 2024-09-21T15:00:57.118 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.118 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.118 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:57.118 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159730+0000", 2024-09-21T15:00:57.118 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5720000000000003e-06 2024-09-21T15:00:57.118 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.118 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.118 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:57.118 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159735+0000", 2024-09-21T15:00:57.118 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1100000000000002e-06 2024-09-21T15:00:57.119 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.119 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.119 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:57.119 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228029+0000", 2024-09-21T15:00:57.119 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068293624999999997 2024-09-21T15:00:57.119 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.119 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.119 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:57.119 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228039+0000", 2024-09-21T15:00:57.119 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0254e-05 2024-09-21T15:00:57.119 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:57.119 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:57.119 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:57.119 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.120 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.120 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45852 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:57.120 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159740+0000", 2024-09-21T15:00:57.120 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 106.913947499, 2024-09-21T15:00:57.120 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068340260999999999, 2024-09-21T15:00:57.120 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:57.120 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:57.120 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:57.120 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:57.120 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:57.120 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45852 2024-09-21T15:00:57.120 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.120 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:57.120 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.120 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:57.121 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159740+0000", 2024-09-21T15:00:57.121 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:57.121 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.121 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.121 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:57.121 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159740+0000", 2024-09-21T15:00:57.121 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:57.121 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.121 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.121 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:57.121 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159739+0000", 2024-09-21T15:00:57.121 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:57.121 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.121 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.121 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:57.121 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159757+0000", 2024-09-21T15:00:57.122 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7314999999999999e-05 2024-09-21T15:00:57.122 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.122 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.122 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:57.122 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159769+0000", 2024-09-21T15:00:57.122 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2376e-05 2024-09-21T15:00:57.122 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.122 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.122 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:57.122 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159775+0000", 2024-09-21T15:00:57.122 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2179999999999996e-06 2024-09-21T15:00:57.122 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.122 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.122 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:57.122 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228071+0000", 2024-09-21T15:00:57.123 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068295344999999993 2024-09-21T15:00:57.123 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.123 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.123 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:57.123 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228080+0000", 2024-09-21T15:00:57.123 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3959999999999995e-06 2024-09-21T15:00:57.123 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:57.123 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:57.123 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:57.123 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.123 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.123 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45853 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:57.123 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159781+0000", 2024-09-21T15:00:57.123 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 106.913906235, 2024-09-21T15:00:57.123 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068337093000000002, 2024-09-21T15:00:57.124 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:57.124 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:57.124 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:57.124 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:57.124 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:57.124 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45853 2024-09-21T15:00:57.124 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.124 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:57.124 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.124 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:57.124 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159781+0000", 2024-09-21T15:00:57.124 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:57.124 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.124 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.125 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:57.125 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159781+0000", 2024-09-21T15:00:57.125 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:57.125 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.125 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.125 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:57.125 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159781+0000", 2024-09-21T15:00:57.125 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:57.125 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.125 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.125 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:57.125 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159784+0000", 2024-09-21T15:00:57.125 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4759999999999998e-06 2024-09-21T15:00:57.125 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.125 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.126 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:57.126 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159792+0000", 2024-09-21T15:00:57.126 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0569999999999993e-06 2024-09-21T15:00:57.126 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.126 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.126 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:57.126 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159797+0000", 2024-09-21T15:00:57.126 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9640000000000002e-06 2024-09-21T15:00:57.126 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.126 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.126 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:57.126 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228111+0000", 2024-09-21T15:00:57.126 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068313737999999999 2024-09-21T15:00:57.126 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.127 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.127 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:57.127 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228118+0000", 2024-09-21T15:00:57.127 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.43e-06 2024-09-21T15:00:57.127 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:57.127 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:57.127 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:57.127 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.127 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.127 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45854 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:57.127 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159802+0000", 2024-09-21T15:00:57.127 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 106.913885585, 2024-09-21T15:00:57.127 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068358938999999994, 2024-09-21T15:00:57.127 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:57.127 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:57.128 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:57.128 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:57.128 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:57.128 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45854 2024-09-21T15:00:57.128 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.128 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:57.128 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.128 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:57.128 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159802+0000", 2024-09-21T15:00:57.128 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:57.128 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.128 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.128 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:57.128 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159802+0000", 2024-09-21T15:00:57.128 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:57.129 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.129 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.129 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:57.129 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159801+0000", 2024-09-21T15:00:57.129 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:57.129 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.129 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.129 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:57.129 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159804+0000", 2024-09-21T15:00:57.129 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0970000000000002e-06 2024-09-21T15:00:57.129 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.129 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.129 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:57.129 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159823+0000", 2024-09-21T15:00:57.129 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8895000000000001e-05 2024-09-21T15:00:57.129 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.130 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.130 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:57.130 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159837+0000", 2024-09-21T15:00:57.130 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3247999999999999e-05 2024-09-21T15:00:57.130 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.130 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.130 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:57.130 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228151+0000", 2024-09-21T15:00:57.130 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068314642999999994 2024-09-21T15:00:57.130 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.130 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.130 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:57.130 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228161+0000", 2024-09-21T15:00:57.130 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4250000000000004e-06 2024-09-21T15:00:57.130 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:57.131 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:57.131 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:57.131 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.131 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.131 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45983 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:57.131 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159848+0000", 2024-09-21T15:00:57.131 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 106.913839326, 2024-09-21T15:00:57.131 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068333801, 2024-09-21T15:00:57.131 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:57.131 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:57.131 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:57.131 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:57.131 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:57.131 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45983 2024-09-21T15:00:57.131 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.132 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:57.132 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.132 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:57.132 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159848+0000", 2024-09-21T15:00:57.132 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:57.132 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.132 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.132 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:57.132 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159848+0000", 2024-09-21T15:00:57.132 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:57.132 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.132 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.132 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:57.132 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159847+0000", 2024-09-21T15:00:57.132 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:57.132 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.133 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.133 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:57.133 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159856+0000", 2024-09-21T15:00:57.133 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4349999999999997e-06 2024-09-21T15:00:57.133 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.133 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.133 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:57.133 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159866+0000", 2024-09-21T15:00:57.133 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0076e-05 2024-09-21T15:00:57.133 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.133 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.133 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:57.133 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159872+0000", 2024-09-21T15:00:57.133 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8309999999999997e-06 2024-09-21T15:00:57.133 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.134 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.134 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:57.134 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228175+0000", 2024-09-21T15:00:57.134 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068302680000000005 2024-09-21T15:00:57.134 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.134 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.134 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:57.134 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228182+0000", 2024-09-21T15:00:57.134 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3150000000000003e-06 2024-09-21T15:00:57.134 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:57.134 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:57.134 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:57.134 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.134 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.134 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45984 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:57.135 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159877+0000", 2024-09-21T15:00:57.135 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 106.913809906, 2024-09-21T15:00:57.135 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068328448, 2024-09-21T15:00:57.135 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:57.135 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:57.135 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:57.135 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:57.135 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:57.135 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45984 2024-09-21T15:00:57.135 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.135 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:57.135 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.135 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:57.135 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159877+0000", 2024-09-21T15:00:57.136 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:57.136 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.136 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.136 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:57.136 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159877+0000", 2024-09-21T15:00:57.136 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:57.136 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.136 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.136 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:57.136 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159877+0000", 2024-09-21T15:00:57.136 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:57.136 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.136 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.136 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:57.137 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159880+0000", 2024-09-21T15:00:57.137 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7610000000000001e-06 2024-09-21T15:00:57.137 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.137 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.137 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:57.137 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159888+0000", 2024-09-21T15:00:57.137 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0330000000000003e-06 2024-09-21T15:00:57.137 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.137 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.137 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:57.137 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159893+0000", 2024-09-21T15:00:57.137 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3550000000000004e-06 2024-09-21T15:00:57.137 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.137 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.137 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:57.138 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228199+0000", 2024-09-21T15:00:57.138 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068306438999999997 2024-09-21T15:00:57.138 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.138 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.138 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:57.138 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228206+0000", 2024-09-21T15:00:57.138 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5599999999999999e-06 2024-09-21T15:00:57.138 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:57.138 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:57.138 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:57.138 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.138 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.138 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45985 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:57.138 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159897+0000", 2024-09-21T15:00:57.138 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 106.91379008299999, 2024-09-21T15:00:57.139 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068345664, 2024-09-21T15:00:57.139 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:57.139 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:57.139 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:57.139 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:57.139 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:57.139 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45985 2024-09-21T15:00:57.139 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.139 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:57.139 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.139 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:57.139 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159897+0000", 2024-09-21T15:00:57.139 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:57.139 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.140 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.140 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:57.140 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159897+0000", 2024-09-21T15:00:57.140 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:57.140 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.140 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.140 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:57.140 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159897+0000", 2024-09-21T15:00:57.140 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:57.140 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.140 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.140 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:57.140 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159900+0000", 2024-09-21T15:00:57.140 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.954e-06 2024-09-21T15:00:57.140 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.141 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.141 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:57.141 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159913+0000", 2024-09-21T15:00:57.141 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3106e-05 2024-09-21T15:00:57.141 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.141 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.141 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:57.141 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159918+0000", 2024-09-21T15:00:57.141 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5199999999999997e-06 2024-09-21T15:00:57.141 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.141 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.141 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:57.141 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228236+0000", 2024-09-21T15:00:57.141 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068317040999999995 2024-09-21T15:00:57.141 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.142 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.142 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:57.142 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228243+0000", 2024-09-21T15:00:57.142 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3880000000000002e-06 2024-09-21T15:00:57.142 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:57.142 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:57.142 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:57.142 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.142 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.142 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45986 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:57.142 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159924+0000", 2024-09-21T15:00:57.142 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 106.913763788, 2024-09-21T15:00:57.142 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068364362999999997, 2024-09-21T15:00:57.142 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:57.143 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:57.143 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:57.143 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:57.143 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:57.143 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45986 2024-09-21T15:00:57.143 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.143 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:57.143 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.143 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:57.143 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159924+0000", 2024-09-21T15:00:57.143 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:57.143 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.143 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.144 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:57.144 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159924+0000", 2024-09-21T15:00:57.144 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:57.144 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.144 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.144 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:57.144 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159923+0000", 2024-09-21T15:00:57.144 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:57.144 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.144 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.144 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:57.144 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159926+0000", 2024-09-21T15:00:57.144 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0510000000000001e-06 2024-09-21T15:00:57.144 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.145 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.145 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:57.145 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159932+0000", 2024-09-21T15:00:57.145 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4160000000000001e-06 2024-09-21T15:00:57.145 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.145 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.145 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:57.145 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159938+0000", 2024-09-21T15:00:57.145 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.383e-06 2024-09-21T15:00:57.145 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.145 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.145 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:57.145 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228275+0000", 2024-09-21T15:00:57.145 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068337007000000005 2024-09-21T15:00:57.145 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.146 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.146 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:57.146 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228288+0000", 2024-09-21T15:00:57.146 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3052e-05 2024-09-21T15:00:57.146 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:57.146 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:57.146 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:57.146 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.146 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.146 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45987 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:00:57.146 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159944+0000", 2024-09-21T15:00:57.146 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 106.913743564, 2024-09-21T15:00:57.146 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068387232000000006, 2024-09-21T15:00:57.146 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:57.147 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:57.147 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:57.147 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:57.147 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:57.147 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45987 2024-09-21T15:00:57.147 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.147 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:57.147 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.147 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:57.147 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159944+0000", 2024-09-21T15:00:57.147 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:57.147 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.147 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.147 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:57.148 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159944+0000", 2024-09-21T15:00:57.148 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:57.148 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.148 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.148 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:57.148 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159943+0000", 2024-09-21T15:00:57.148 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:57.148 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.148 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.148 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:57.148 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159946+0000", 2024-09-21T15:00:57.148 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6309999999999999e-06 2024-09-21T15:00:57.148 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.148 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.148 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:57.149 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159952+0000", 2024-09-21T15:00:57.149 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6540000000000004e-06 2024-09-21T15:00:57.149 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.149 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.149 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:57.149 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159957+0000", 2024-09-21T15:00:57.149 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9280000000000001e-06 2024-09-21T15:00:57.149 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.149 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.149 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:57.149 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228322+0000", 2024-09-21T15:00:57.149 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068365067000000002 2024-09-21T15:00:57.149 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.149 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.149 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:57.150 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228331+0000", 2024-09-21T15:00:57.150 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3270000000000007e-06 2024-09-21T15:00:57.150 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:57.150 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:57.150 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:57.150 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.150 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.150 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:97672 3.4 3:26046d29:::rbd_data.1136bfc79a70.0000000000000001:head [write 0~4718592 [fadvise_sequential+fadvise_nocache],write 5242880~131072 [fadvise_sequential+fadvise_nocache],write 5439488~147456 [fadvise_sequential+fadvise_nocache],write 5652480~229376 [fadvise_sequential+fadvise_nocache],write 5947392~147456 [fadvise_sequential+fadvise_nocache],write 6160384~229376 [fadvise_sequential+fadvise_nocache],write 6455296~147456 [fadvise_sequential+fadvise_nocache],write 6668288~229376 [fadvise_sequential+fadvise_nocache],write 6963200~147456 [fadvise_sequential+fadvise_nocache],write 7176192~147456 [fadvise_sequential+fadvise_nocache],write 7340032~65536 [fadvise_sequential+fadvise_nocache],write 7471104~147456 [fadvise_sequential+fadvise_nocache],write 7684096~147456 [fadvise_sequential+fadvise_nocache],write 7864320~49152 [fadvise_sequential+fadvise_nocache],write 7979008~147456 [fadvise_sequential+fadvise_nocache],write 8192000~147456 [fadvise_sequential+fadvise_nocache],write 8388608~32768 [fadvise_sequential+fadvise_nocache],write 8519680~114688 [fadvise_sequential+fadvise_nocache],write 8732672~114688 [fadvise_sequential+fadvise_nocache],write 8912896~16384 [fadvise_sequential+fadvise_nocache],write 9109504~32768 [fadvise_sequential+fadvise_nocache],write 9322496~32768 [fadvise_sequential+fadvise_nocache],write 9617408~32768 [fadvise_sequential+fadvise_nocache],write 9830400~32768 [fadvise_sequential+fadvise_nocache],write 9961472~98304 [fadvise_sequential+fadvise_nocache],write 10076160~196608 [fadvise_sequential+fadvise_nocache],write 10289152~278528 [fadvise_sequential+fadvise_nocache],write 10584064~196608 [fadvise_sequential+fadvise_nocache],write 10797056~196608 [fadvise_sequential+fadvise_nocache],write 11010048~65536 [fadvise_sequential+fadvise_nocache],write 11091968~196608 [fadvise_sequential+fadvise_nocache],write 11304960~196608 [fadvise_sequential+fadvise_nocache],write 11517952~65536 [fadvise_sequential+fadvise_nocache],write 11616256~98304 [fadvise_sequential+fadvise_nocache],write 11730944~65536 [fadvise_sequential+fadvise_nocache],write 11829248~98304 [fadvise_sequential+fadvise_nocache],write 11943936~65536 [fadvise_sequential+fadvise_nocache],write 12042240~49152 [fadvise_sequential+fadvise_nocache],write 12124160~98304 [fadvise_sequential+fadvise_nocache],write 12238848~65536 [fadvise_sequential+fadvise_nocache],write 12337152~98304 [fadvise_sequential+fadvise_nocache],write 12451840~65536 [fadvise_sequential+fadvise_nocache],write 12550144~32768 [fadvise_sequential+fadvise_nocache],write 12632064~98304 [fadvise_sequential+fadvise_nocache],write 12845056~98304 [fadvise_sequential+fadvise_nocache],write 13058048~49152 [fadvise_sequential+fadvise_nocache],write 13139968~98304 [fadvise_sequential+fadvise_nocache],write 13352960~98304 [fadvise_sequential+fadvise_nocache],write 13565952~65536 [fadvise_sequential+fadvise_nocache],write 13647872~98304 [fadvise_sequential+fadvise_nocache],write 13860864~98304 [fadvise_sequential+fadvise_nocache],write 14073856~81920 [fadvise_sequential+fadvise_nocache],write 15269888~81920 [fadvise_sequential+fadvise_nocache],write 15482880~81920 [fadvise_sequential+fadvise_nocache],write 15695872~32768 [fadvise_sequential+fadvise_nocache],write 15777792~81920 [fadvise_sequential+fadvise_nocache],write 15990784~81920 [fadvise_sequential+fadvise_nocache],write 16203776~49152 [fadvise_sequential+fadvise_nocache],write 16285696~81920 [fadvise_sequential+fadvise_nocache],write 16498688~81920 [fadvise_sequential+fadvise_nocache],write 16711680~65536 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:57.150 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.298669+0000", 2024-09-21T15:00:57.150 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 51.775018498000001, 2024-09-21T15:00:57.150 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.10282358900000001, 2024-09-21T15:00:57.150 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:57.150 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:57.150 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:57.150 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:57.151 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:57.151 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97672 2024-09-21T15:00:57.151 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.151 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:57.151 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.151 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:57.151 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.298669+0000", 2024-09-21T15:00:57.151 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:57.151 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.151 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.151 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:57.151 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.298669+0000", 2024-09-21T15:00:57.151 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:57.151 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.151 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.152 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:57.152 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.298667+0000", 2024-09-21T15:00:57.152 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:00:57.152 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.152 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.152 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:57.152 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.327458+0000", 2024-09-21T15:00:57.152 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.028790754000000002 2024-09-21T15:00:57.152 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.152 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.152 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:57.152 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.327492+0000", 2024-09-21T15:00:57.152 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3806999999999997e-05 2024-09-21T15:00:57.152 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.152 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.152 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:57.153 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.327507+0000", 2024-09-21T15:00:57.153 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5112e-05 2024-09-21T15:00:57.153 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.153 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.153 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:57.153 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.327597+0000", 2024-09-21T15:00:57.153 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9938999999999994e-05 2024-09-21T15:00:57.153 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.153 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.153 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:57.153 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.327927+0000", 2024-09-21T15:00:57.153 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000329766 2024-09-21T15:00:57.153 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.153 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.153 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T15:00:57.153 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.330735+0000", 2024-09-21T15:00:57.154 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0028086869999999998 2024-09-21T15:00:57.154 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.154 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.154 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:57.154 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.378299+0000", 2024-09-21T15:00:57.154 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.047563495999999997 2024-09-21T15:00:57.154 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.154 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.154 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:57.154 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.401248+0000", 2024-09-21T15:00:57.154 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.022949311 2024-09-21T15:00:57.154 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.154 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.154 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:57.155 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.401446+0000", 2024-09-21T15:00:57.155 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019831199999999999 2024-09-21T15:00:57.155 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.155 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.155 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:57.155 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.401492+0000", 2024-09-21T15:00:57.155 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6109000000000002e-05 2024-09-21T15:00:57.155 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:57.155 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:57.155 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:57.155 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:57.155 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:57.155 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:57.157 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T15:00:57.157 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:00:57.155+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:00:57.257 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:00:57.253+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:00:57.267 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:57.267 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:00:57.267 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T15:00:57.267 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:57.277 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T15:00:57.359 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:00:57.357+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:00:57.383 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:57.383 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:00:57.383 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T15:00:57.383 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T15:00:57.383 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:57.393 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T15:00:57.459 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:00:57.457+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:00:57.550 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:57.550 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T15:00:57.550 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T15:00:57.550 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T15:00:57.550 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.551 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:97677 3.15 e122/24 3:a8e72133:::rbd_data.1136bfc79a70.0000000000000005:head v 122'567, mlcod=120'565)", 2024-09-21T15:00:57.551 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.320753+0000", 2024-09-21T15:00:57.551 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 52.225517674000002, 2024-09-21T15:00:57.551 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070639375000000004, 2024-09-21T15:00:57.551 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:57.551 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:57.551 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:57.551 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.551 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:57.551 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.320753+0000", 2024-09-21T15:00:57.551 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:57.551 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.551 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.551 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:57.551 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.320753+0000", 2024-09-21T15:00:57.552 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:57.552 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.552 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.552 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:57.552 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.320753+0000", 2024-09-21T15:00:57.552 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:57.552 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.552 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.552 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:57.552 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.375452+0000", 2024-09-21T15:00:57.552 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054698945999999998 2024-09-21T15:00:57.552 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.552 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.552 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:57.552 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.375476+0000", 2024-09-21T15:00:57.553 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4380999999999999e-05 2024-09-21T15:00:57.553 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.553 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.553 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:57.553 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.375485+0000", 2024-09-21T15:00:57.553 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0049999999999993e-06 2024-09-21T15:00:57.553 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.553 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.553 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:57.553 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.375552+0000", 2024-09-21T15:00:57.553 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6550999999999994e-05 2024-09-21T15:00:57.553 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.553 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.553 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:57.553 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.375578+0000", 2024-09-21T15:00:57.553 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6196000000000001e-05 2024-09-21T15:00:57.554 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.554 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.554 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:57.554 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.391319+0000", 2024-09-21T15:00:57.554 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015741271000000001 2024-09-21T15:00:57.554 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.554 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.554 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:57.554 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.391393+0000", 2024-09-21T15:00:57.554 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3287000000000005e-05 2024-09-21T15:00:57.554 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:57.554 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:57.554 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:57.554 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.554 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.554 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:97672 3.4 e122/51 3:26046d29:::rbd_data.1136bfc79a70.0000000000000001:head v 122'256, mlcod=120'254)", 2024-09-21T15:00:57.555 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.331696+0000", 2024-09-21T15:00:57.555 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 52.214574511000002, 2024-09-21T15:00:57.555 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041222507999999998, 2024-09-21T15:00:57.555 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:57.555 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:57.555 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:57.555 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.555 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:57.555 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.331696+0000", 2024-09-21T15:00:57.555 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:57.555 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.555 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.555 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:57.555 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.331696+0000", 2024-09-21T15:00:57.555 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:57.555 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.556 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.556 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:57.556 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.331695+0000", 2024-09-21T15:00:57.556 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:00:57.556 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.556 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.556 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:57.556 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.340175+0000", 2024-09-21T15:00:57.556 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0084799520000000007 2024-09-21T15:00:57.556 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.556 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.556 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:57.556 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.340208+0000", 2024-09-21T15:00:57.556 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2997999999999998e-05 2024-09-21T15:00:57.556 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.556 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.557 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:57.557 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.340223+0000", 2024-09-21T15:00:57.557 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5330999999999999e-05 2024-09-21T15:00:57.557 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.557 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.557 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:57.557 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.340305+0000", 2024-09-21T15:00:57.557 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2095999999999995e-05 2024-09-21T15:00:57.557 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.557 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.557 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:57.557 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.340337+0000", 2024-09-21T15:00:57.557 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2097000000000001e-05 2024-09-21T15:00:57.557 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.557 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.557 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:57.558 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.372863+0000", 2024-09-21T15:00:57.558 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.032525923999999998 2024-09-21T15:00:57.558 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.558 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.558 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:57.558 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.372919+0000", 2024-09-21T15:00:57.558 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5729000000000001e-05 2024-09-21T15:00:57.558 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:57.558 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:57.558 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:57.558 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.558 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.558 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:116177 3.12 3:4efbaa3f:::rbd_data.1136b1194d20.0000000000000006:58 [read 5791744~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:57.558 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.712449+0000", 2024-09-21T15:00:57.558 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 27.833821830000002, 2024-09-21T15:00:57.558 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.033824706000000003, 2024-09-21T15:00:57.559 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:57.559 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:57.559 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:57.559 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:57.559 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:57.559 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116177 2024-09-21T15:00:57.559 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.559 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:57.559 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.559 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:57.559 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712449+0000", 2024-09-21T15:00:57.559 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:57.559 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.559 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.559 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:57.559 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712449+0000", 2024-09-21T15:00:57.560 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:57.560 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.560 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.560 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:57.560 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712448+0000", 2024-09-21T15:00:57.560 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:57.560 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.560 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.560 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:57.560 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712451+0000", 2024-09-21T15:00:57.560 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5330000000000001e-06 2024-09-21T15:00:57.560 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.560 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.560 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:57.560 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712458+0000", 2024-09-21T15:00:57.560 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5239999999999997e-06 2024-09-21T15:00:57.561 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.561 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.561 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:57.561 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712467+0000", 2024-09-21T15:00:57.561 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7189999999999996e-06 2024-09-21T15:00:57.561 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.561 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.561 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:57.561 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.746189+0000", 2024-09-21T15:00:57.561 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.033722203999999999 2024-09-21T15:00:57.561 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.561 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.561 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:57.561 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.746228+0000", 2024-09-21T15:00:57.562 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8908000000000001e-05 2024-09-21T15:00:57.562 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.562 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.562 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:57.562 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.746274+0000", 2024-09-21T15:00:57.562 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5380000000000003e-05 2024-09-21T15:00:57.562 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:57.562 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:57.562 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:57.562 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.562 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.562 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:116178 3.12 3:4efbaa3f:::rbd_data.1136b1194d20.0000000000000006:58 [read 5881856~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:57.562 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.712472+0000", 2024-09-21T15:00:57.562 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 27.833799179, 2024-09-21T15:00:57.563 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.033958110999999999, 2024-09-21T15:00:57.563 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:57.563 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:57.563 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:57.563 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:57.563 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:57.563 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116178 2024-09-21T15:00:57.563 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.563 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:57.563 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.563 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:57.563 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712472+0000", 2024-09-21T15:00:57.563 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:57.563 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.563 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.564 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:57.564 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712472+0000", 2024-09-21T15:00:57.564 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:57.564 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.564 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.564 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:57.564 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712471+0000", 2024-09-21T15:00:57.564 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:57.564 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.564 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.564 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:57.564 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712474+0000", 2024-09-21T15:00:57.564 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4389999999999999e-06 2024-09-21T15:00:57.564 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.564 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.565 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:57.565 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712479+0000", 2024-09-21T15:00:57.565 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4269999999999999e-06 2024-09-21T15:00:57.565 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.565 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.565 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:57.565 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712483+0000", 2024-09-21T15:00:57.565 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8449999999999996e-06 2024-09-21T15:00:57.565 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.565 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.565 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:57.565 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.746287+0000", 2024-09-21T15:00:57.565 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.033804381000000001 2024-09-21T15:00:57.565 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.565 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.566 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:57.566 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.746337+0000", 2024-09-21T15:00:57.566 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9622e-05 2024-09-21T15:00:57.566 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.566 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.566 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:57.566 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.746430+0000", 2024-09-21T15:00:57.566 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2800000000000006e-05 2024-09-21T15:00:57.566 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:57.566 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:57.566 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:57.566 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.566 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.566 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:116179 3.12 3:4efbaa3f:::rbd_data.1136b1194d20.0000000000000006:58 [read 5971968~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:57.567 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.712487+0000", 2024-09-21T15:00:57.567 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 27.833784337000001, 2024-09-21T15:00:57.567 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034144503, 2024-09-21T15:00:57.567 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:57.567 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:57.567 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:57.567 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:57.567 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:57.567 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116179 2024-09-21T15:00:57.567 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.567 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:57.567 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.567 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:57.567 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712487+0000", 2024-09-21T15:00:57.567 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:57.568 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.568 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.568 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:57.568 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712487+0000", 2024-09-21T15:00:57.568 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:57.568 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.568 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.568 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:57.568 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712486+0000", 2024-09-21T15:00:57.568 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:57.568 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.568 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.568 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:57.568 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712488+0000", 2024-09-21T15:00:57.568 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2000000000000001e-06 2024-09-21T15:00:57.569 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.569 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.569 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:57.569 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712493+0000", 2024-09-21T15:00:57.569 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9429999999999999e-06 2024-09-21T15:00:57.569 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.569 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.569 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:57.569 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712497+0000", 2024-09-21T15:00:57.569 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9689999999999996e-06 2024-09-21T15:00:57.569 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.569 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.569 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:57.569 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.746459+0000", 2024-09-21T15:00:57.569 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.033961365 2024-09-21T15:00:57.570 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.570 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.570 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:57.570 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.746538+0000", 2024-09-21T15:00:57.570 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9073999999999995e-05 2024-09-21T15:00:57.570 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.570 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.570 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:57.570 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.746631+0000", 2024-09-21T15:00:57.570 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3375999999999995e-05 2024-09-21T15:00:57.570 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:57.570 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:57.570 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:57.570 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.571 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.571 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:116180 3.12 3:4efbaa3f:::rbd_data.1136b1194d20.0000000000000006:58 [read 6062080~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:57.571 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.712501+0000", 2024-09-21T15:00:57.571 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 27.833770138999999, 2024-09-21T15:00:57.571 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034331168000000002, 2024-09-21T15:00:57.571 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:57.571 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:57.571 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:57.571 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:57.571 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:57.571 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116180 2024-09-21T15:00:57.571 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.571 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:57.571 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.571 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:57.572 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712501+0000", 2024-09-21T15:00:57.572 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:57.572 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.572 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.572 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:57.572 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712501+0000", 2024-09-21T15:00:57.572 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:57.572 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.572 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.572 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:57.572 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712500+0000", 2024-09-21T15:00:57.572 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:57.572 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.572 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.572 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:57.572 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712502+0000", 2024-09-21T15:00:57.572 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.035e-06 2024-09-21T15:00:57.573 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.573 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.573 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:57.573 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712508+0000", 2024-09-21T15:00:57.573 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4720000000000003e-06 2024-09-21T15:00:57.573 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.573 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.573 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:57.573 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712512+0000", 2024-09-21T15:00:57.573 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7129999999999999e-06 2024-09-21T15:00:57.573 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.573 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.573 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:57.573 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.746662+0000", 2024-09-21T15:00:57.573 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034149837000000002 2024-09-21T15:00:57.573 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.573 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.574 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:57.574 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.746740+0000", 2024-09-21T15:00:57.574 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8350999999999996e-05 2024-09-21T15:00:57.574 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.574 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.574 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:57.574 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.746832+0000", 2024-09-21T15:00:57.574 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2046000000000006e-05 2024-09-21T15:00:57.574 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:57.574 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:57.574 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:57.574 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.574 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.574 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:116181 3.12 3:4efbaa3f:::rbd_data.1136b1194d20.0000000000000006:58 [read 6152192~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:57.575 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.712515+0000", 2024-09-21T15:00:57.575 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 27.833755793000002, 2024-09-21T15:00:57.575 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034525820999999998, 2024-09-21T15:00:57.575 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:57.575 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:57.575 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:57.575 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:57.575 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:57.575 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116181 2024-09-21T15:00:57.575 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.575 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:57.575 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.575 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:57.575 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712515+0000", 2024-09-21T15:00:57.575 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:57.575 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.576 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.576 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:57.576 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712515+0000", 2024-09-21T15:00:57.576 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:57.576 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.576 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.576 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:57.576 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712515+0000", 2024-09-21T15:00:57.576 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:57.576 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.576 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.576 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:57.576 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712522+0000", 2024-09-21T15:00:57.576 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8000000000000001e-06 2024-09-21T15:00:57.576 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.576 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.577 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:57.577 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712528+0000", 2024-09-21T15:00:57.577 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0650000000000004e-06 2024-09-21T15:00:57.577 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.577 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.577 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:57.577 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712533+0000", 2024-09-21T15:00:57.577 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.079e-06 2024-09-21T15:00:57.577 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.577 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.577 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:57.577 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.746864+0000", 2024-09-21T15:00:57.577 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034330830999999999 2024-09-21T15:00:57.577 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.577 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.577 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:57.578 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.746943+0000", 2024-09-21T15:00:57.578 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9221999999999995e-05 2024-09-21T15:00:57.578 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.578 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.578 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:57.578 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.747041+0000", 2024-09-21T15:00:57.578 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.8114000000000003e-05 2024-09-21T15:00:57.578 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:57.578 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:57.578 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:57.578 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.578 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.578 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:116182 3.12 3:4efbaa3f:::rbd_data.1136b1194d20.0000000000000006:58 [read 6242304~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:57.578 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.712538+0000", 2024-09-21T15:00:57.578 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 27.833732520000002, 2024-09-21T15:00:57.578 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034702127999999999, 2024-09-21T15:00:57.579 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:57.579 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:57.579 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:57.579 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:57.579 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:57.579 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116182 2024-09-21T15:00:57.579 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.579 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:57.579 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.579 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:57.579 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712538+0000", 2024-09-21T15:00:57.579 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:57.579 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.579 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.579 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:57.579 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712538+0000", 2024-09-21T15:00:57.580 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:57.580 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.580 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.580 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:57.580 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712538+0000", 2024-09-21T15:00:57.580 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:57.580 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.580 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.580 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:57.580 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712541+0000", 2024-09-21T15:00:57.580 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5330000000000001e-06 2024-09-21T15:00:57.580 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.580 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.580 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:57.580 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712547+0000", 2024-09-21T15:00:57.581 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1109999999999996e-06 2024-09-21T15:00:57.581 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.581 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.581 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:57.581 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712551+0000", 2024-09-21T15:00:57.581 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.685e-06 2024-09-21T15:00:57.581 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.581 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.581 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:57.581 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.747067+0000", 2024-09-21T15:00:57.581 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034516181 2024-09-21T15:00:57.581 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.581 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.581 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:57.581 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.747152+0000", 2024-09-21T15:00:57.581 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4075999999999999e-05 2024-09-21T15:00:57.582 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.582 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.582 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:57.582 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.747240+0000", 2024-09-21T15:00:57.582 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8924000000000004e-05 2024-09-21T15:00:57.582 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:57.582 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:57.582 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:57.582 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.582 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.582 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:116216 3.12 3:4efbaa3f:::rbd_data.1136b1194d20.0000000000000006:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:57.582 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.713202+0000", 2024-09-21T15:00:57.582 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 27.833069015, 2024-09-21T15:00:57.582 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034182227000000003, 2024-09-21T15:00:57.582 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:57.582 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:57.583 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:57.583 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:57.583 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:57.583 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116216 2024-09-21T15:00:57.583 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.583 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:57.583 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.583 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:57.583 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.713202+0000", 2024-09-21T15:00:57.583 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:57.583 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.583 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.583 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:57.583 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.713202+0000", 2024-09-21T15:00:57.583 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:57.584 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.584 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.584 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:57.584 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.713201+0000", 2024-09-21T15:00:57.584 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:57.584 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.584 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.584 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:57.584 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.713204+0000", 2024-09-21T15:00:57.584 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7410000000000001e-06 2024-09-21T15:00:57.584 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.584 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.584 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:57.584 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.713211+0000", 2024-09-21T15:00:57.584 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1110000000000002e-06 2024-09-21T15:00:57.585 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.585 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.585 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:57.585 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.713217+0000", 2024-09-21T15:00:57.585 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.446e-06 2024-09-21T15:00:57.585 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.585 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.585 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:57.585 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.747263+0000", 2024-09-21T15:00:57.585 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034046598999999997 2024-09-21T15:00:57.585 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.585 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.585 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:57.585 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.747321+0000", 2024-09-21T15:00:57.585 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7778999999999997e-05 2024-09-21T15:00:57.585 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.586 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.586 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:57.586 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.747384+0000", 2024-09-21T15:00:57.586 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3083000000000006e-05 2024-09-21T15:00:57.586 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:57.586 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:57.586 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:57.586 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.586 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.586 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:118512 3.12 3:4efbaa3f:::rbd_data.1136b1194d20.0000000000000006:58 [read 10043392~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:57.586 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.893240+0000", 2024-09-21T15:00:57.586 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 27.653030427000001, 2024-09-21T15:00:57.586 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.033811941999999998, 2024-09-21T15:00:57.586 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:57.586 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:57.587 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:57.587 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:57.587 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:57.587 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 118512 2024-09-21T15:00:57.587 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.587 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:57.587 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.587 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:57.587 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893240+0000", 2024-09-21T15:00:57.587 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:57.587 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.587 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.587 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:57.587 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893240+0000", 2024-09-21T15:00:57.587 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:57.587 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.588 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.588 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:57.588 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893240+0000", 2024-09-21T15:00:57.588 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:57.588 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.588 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.588 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:57.588 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893242+0000", 2024-09-21T15:00:57.588 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.717e-06 2024-09-21T15:00:57.588 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.588 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.588 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:57.588 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893245+0000", 2024-09-21T15:00:57.588 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3380000000000001e-06 2024-09-21T15:00:57.588 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.588 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.589 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:57.589 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893248+0000", 2024-09-21T15:00:57.589 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9670000000000001e-06 2024-09-21T15:00:57.589 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.589 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.589 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:57.589 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.926868+0000", 2024-09-21T15:00:57.589 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.033619938000000002 2024-09-21T15:00:57.589 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.589 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.589 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:57.589 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.926964+0000", 2024-09-21T15:00:57.589 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5624999999999993e-05 2024-09-21T15:00:57.589 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.589 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.590 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:57.590 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.927052+0000", 2024-09-21T15:00:57.590 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8632000000000001e-05 2024-09-21T15:00:57.590 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:57.590 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:57.590 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:57.590 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.590 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.590 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:118513 3.12 3:4efbaa3f:::rbd_data.1136b1194d20.0000000000000006:58 [read 10133504~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:57.590 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.893251+0000", 2024-09-21T15:00:57.590 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 27.653020180999999, 2024-09-21T15:00:57.590 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.033992324999999997, 2024-09-21T15:00:57.590 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:57.590 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:57.590 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:57.590 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:57.591 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:57.591 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 118513 2024-09-21T15:00:57.591 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.591 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:57.591 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.591 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:57.591 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893251+0000", 2024-09-21T15:00:57.591 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:57.591 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.591 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.591 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:57.591 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893251+0000", 2024-09-21T15:00:57.591 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:57.591 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.591 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.591 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:57.592 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893250+0000", 2024-09-21T15:00:57.592 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:57.592 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.592 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.592 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:57.592 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893252+0000", 2024-09-21T15:00:57.592 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.666e-06 2024-09-21T15:00:57.592 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.592 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.592 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:57.592 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893255+0000", 2024-09-21T15:00:57.592 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.287e-06 2024-09-21T15:00:57.592 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.592 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.592 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:57.592 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893258+0000", 2024-09-21T15:00:57.593 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.027e-06 2024-09-21T15:00:57.593 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.593 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.593 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:57.593 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.927083+0000", 2024-09-21T15:00:57.593 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.033824262000000001 2024-09-21T15:00:57.593 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.593 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.593 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:57.593 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.927154+0000", 2024-09-21T15:00:57.593 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1099000000000005e-05 2024-09-21T15:00:57.593 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.593 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.594 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:57.594 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.927243+0000", 2024-09-21T15:00:57.594 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9266999999999998e-05 2024-09-21T15:00:57.594 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:57.594 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:57.594 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:57.594 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.594 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.594 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:118514 3.12 3:4efbaa3f:::rbd_data.1136b1194d20.0000000000000006:58 [read 10223616~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:57.594 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.893261+0000", 2024-09-21T15:00:57.594 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 27.653009821000001, 2024-09-21T15:00:57.594 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034165500000000001, 2024-09-21T15:00:57.594 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:57.594 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:57.595 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:57.595 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:57.595 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:57.595 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 118514 2024-09-21T15:00:57.595 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.595 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:57.595 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.595 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:57.595 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893261+0000", 2024-09-21T15:00:57.595 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:57.595 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.595 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.595 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:57.595 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893261+0000", 2024-09-21T15:00:57.595 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:57.596 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.596 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.596 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:57.596 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893261+0000", 2024-09-21T15:00:57.596 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:57.596 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.596 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.596 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:57.596 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893262+0000", 2024-09-21T15:00:57.596 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.787e-06 2024-09-21T15:00:57.596 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.596 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.596 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:57.596 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893267+0000", 2024-09-21T15:00:57.596 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6399999999999996e-06 2024-09-21T15:00:57.597 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.597 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.597 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:57.597 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893270+0000", 2024-09-21T15:00:57.597 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9610000000000001e-06 2024-09-21T15:00:57.597 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.597 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.597 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:57.597 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.927271+0000", 2024-09-21T15:00:57.597 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034001253000000002 2024-09-21T15:00:57.597 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.597 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.597 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:57.597 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.927344+0000", 2024-09-21T15:00:57.597 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2962999999999999e-05 2024-09-21T15:00:57.597 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.598 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.598 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:57.598 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.927427+0000", 2024-09-21T15:00:57.598 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2292000000000004e-05 2024-09-21T15:00:57.598 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:57.598 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:57.598 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:57.598 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.598 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.598 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:118515 3.12 3:4efbaa3f:::rbd_data.1136b1194d20.0000000000000006:58 [read 10313728~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:57.598 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.893273+0000", 2024-09-21T15:00:57.598 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 27.652997446000001, 2024-09-21T15:00:57.598 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034333437000000001, 2024-09-21T15:00:57.598 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:57.598 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:57.598 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:57.599 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:57.599 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:57.599 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 118515 2024-09-21T15:00:57.599 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.599 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:57.599 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.599 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:57.599 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893273+0000", 2024-09-21T15:00:57.599 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:57.599 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.599 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.599 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:57.599 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893273+0000", 2024-09-21T15:00:57.599 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:57.599 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.600 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.600 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:57.600 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893273+0000", 2024-09-21T15:00:57.600 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:57.600 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.600 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.600 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:57.600 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893275+0000", 2024-09-21T15:00:57.600 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6789999999999999e-06 2024-09-21T15:00:57.600 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.600 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.600 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:57.600 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893278+0000", 2024-09-21T15:00:57.600 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2440000000000002e-06 2024-09-21T15:00:57.600 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.600 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.601 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:57.601 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893282+0000", 2024-09-21T15:00:57.601 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.89e-06 2024-09-21T15:00:57.601 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.601 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.601 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:57.601 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.927454+0000", 2024-09-21T15:00:57.601 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034172077000000002 2024-09-21T15:00:57.601 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.601 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.601 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:57.601 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.927525+0000", 2024-09-21T15:00:57.601 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0963999999999993e-05 2024-09-21T15:00:57.601 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.601 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.601 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:57.602 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.927607+0000", 2024-09-21T15:00:57.602 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1868999999999998e-05 2024-09-21T15:00:57.602 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:57.602 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:57.602 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:57.602 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.602 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.602 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:118516 3.12 3:4efbaa3f:::rbd_data.1136b1194d20.0000000000000006:58 [read 10403840~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:57.602 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.893284+0000", 2024-09-21T15:00:57.602 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 27.652986505000001, 2024-09-21T15:00:57.602 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034589559999999998, 2024-09-21T15:00:57.602 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:57.602 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:57.602 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:57.603 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:57.603 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:57.603 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 118516 2024-09-21T15:00:57.603 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.603 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:57.603 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.603 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:57.603 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893284+0000", 2024-09-21T15:00:57.603 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:57.603 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.603 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.603 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:57.603 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893284+0000", 2024-09-21T15:00:57.603 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:57.603 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.604 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.604 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:57.604 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893284+0000", 2024-09-21T15:00:57.604 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:57.604 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.604 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.604 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:57.604 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893291+0000", 2024-09-21T15:00:57.604 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6200000000000001e-06 2024-09-21T15:00:57.604 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.604 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.604 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:57.604 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893294+0000", 2024-09-21T15:00:57.604 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6729999999999998e-06 2024-09-21T15:00:57.605 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.605 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.605 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:57.605 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.893297+0000", 2024-09-21T15:00:57.605 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0510000000000001e-06 2024-09-21T15:00:57.605 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.605 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.605 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:57.605 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.927694+0000", 2024-09-21T15:00:57.605 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034397018000000001 2024-09-21T15:00:57.605 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.605 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.605 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:57.605 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.927767+0000", 2024-09-21T15:00:57.605 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2379000000000006e-05 2024-09-21T15:00:57.606 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.606 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.606 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:57.606 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.927874+0000", 2024-09-21T15:00:57.606 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010709000000000001 2024-09-21T15:00:57.606 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:57.606 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:57.606 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:57.606 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.606 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.606 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:118696 3.12 3:4efbaa3f:::rbd_data.1136b1194d20.0000000000000006:58 [read 9961472~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:57.606 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.902583+0000", 2024-09-21T15:00:57.606 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 27.643687457999999, 2024-09-21T15:00:57.606 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.033756517999999999, 2024-09-21T15:00:57.606 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:57.607 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:57.607 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:57.607 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:57.607 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:57.607 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 118696 2024-09-21T15:00:57.607 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.607 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:57.607 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.607 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:57.607 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902583+0000", 2024-09-21T15:00:57.607 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:57.607 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.607 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.607 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:57.607 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902583+0000", 2024-09-21T15:00:57.608 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:57.608 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.608 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.608 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:57.608 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902583+0000", 2024-09-21T15:00:57.608 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:57.608 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.608 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.608 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:57.608 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902585+0000", 2024-09-21T15:00:57.608 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7060000000000001e-06 2024-09-21T15:00:57.608 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.608 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.608 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:57.608 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902588+0000", 2024-09-21T15:00:57.608 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.128e-06 2024-09-21T15:00:57.609 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.609 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.609 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:57.609 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902592+0000", 2024-09-21T15:00:57.609 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0990000000000001e-06 2024-09-21T15:00:57.609 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.609 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.609 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:57.609 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.936161+0000", 2024-09-21T15:00:57.609 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.033569103000000003 2024-09-21T15:00:57.609 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.609 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.609 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:57.609 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.936241+0000", 2024-09-21T15:00:57.609 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9437000000000005e-05 2024-09-21T15:00:57.609 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.609 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.610 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:57.610 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.936340+0000", 2024-09-21T15:00:57.610 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9302999999999994e-05 2024-09-21T15:00:57.610 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:57.610 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:57.610 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:57.610 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.610 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.610 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:118697 3.12 3:4efbaa3f:::rbd_data.1136b1194d20.0000000000000006:58 [read 10051584~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:57.610 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.902595+0000", 2024-09-21T15:00:57.610 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 27.643675521999999, 2024-09-21T15:00:57.610 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.033875518, 2024-09-21T15:00:57.610 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:57.610 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:57.610 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:57.610 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:57.611 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:57.611 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 118697 2024-09-21T15:00:57.611 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.611 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:57.611 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.611 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:57.611 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902595+0000", 2024-09-21T15:00:57.611 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:57.611 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.611 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.611 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:57.611 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902595+0000", 2024-09-21T15:00:57.611 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:57.611 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.611 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.612 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:57.612 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902595+0000", 2024-09-21T15:00:57.612 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:57.612 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.612 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.612 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:57.612 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902597+0000", 2024-09-21T15:00:57.612 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7889999999999999e-06 2024-09-21T15:00:57.612 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.612 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.612 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:57.612 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902601+0000", 2024-09-21T15:00:57.612 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.16e-06 2024-09-21T15:00:57.612 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.612 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.612 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:57.613 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902605+0000", 2024-09-21T15:00:57.613 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7560000000000001e-06 2024-09-21T15:00:57.613 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.613 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.613 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:57.613 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.936370+0000", 2024-09-21T15:00:57.613 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.033765114999999998 2024-09-21T15:00:57.613 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.613 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.613 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:57.613 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.936413+0000", 2024-09-21T15:00:57.613 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3399000000000002e-05 2024-09-21T15:00:57.613 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.614 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.614 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:57.614 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.936471+0000", 2024-09-21T15:00:57.614 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7726000000000003e-05 2024-09-21T15:00:57.614 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:57.614 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:57.614 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:57.614 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.614 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.614 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:118698 3.12 3:4efbaa3f:::rbd_data.1136b1194d20.0000000000000006:58 [read 10141696~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:57.614 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.902607+0000", 2024-09-21T15:00:57.614 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 27.643663927999999, 2024-09-21T15:00:57.614 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034062306000000001, 2024-09-21T15:00:57.614 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:57.614 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:57.614 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:57.614 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:57.615 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:57.615 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 118698 2024-09-21T15:00:57.615 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.615 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:57.615 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.615 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:57.615 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902607+0000", 2024-09-21T15:00:57.615 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:57.615 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.615 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.615 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:57.615 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902607+0000", 2024-09-21T15:00:57.615 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:57.615 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.615 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.615 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:57.616 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902607+0000", 2024-09-21T15:00:57.616 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:57.616 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.616 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.616 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:57.616 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902608+0000", 2024-09-21T15:00:57.616 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5910000000000001e-06 2024-09-21T15:00:57.616 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.616 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.616 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:57.616 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902611+0000", 2024-09-21T15:00:57.616 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.965e-06 2024-09-21T15:00:57.616 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.616 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.616 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:57.616 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902615+0000", 2024-09-21T15:00:57.617 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0049999999999998e-06 2024-09-21T15:00:57.617 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.617 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.617 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:57.617 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.936501+0000", 2024-09-21T15:00:57.617 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.033885884999999998 2024-09-21T15:00:57.617 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.617 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.617 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:57.617 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.936584+0000", 2024-09-21T15:00:57.617 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2650000000000003e-05 2024-09-21T15:00:57.617 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.617 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.617 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:57.617 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.936669+0000", 2024-09-21T15:00:57.617 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5461000000000001e-05 2024-09-21T15:00:57.618 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:57.618 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:57.618 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:57.618 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.618 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.618 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:118699 3.12 3:4efbaa3f:::rbd_data.1136b1194d20.0000000000000006:58 [read 10231808~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:57.618 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.902625+0000", 2024-09-21T15:00:57.618 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 27.643645797000001, 2024-09-21T15:00:57.618 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034245128999999999, 2024-09-21T15:00:57.618 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:57.618 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:57.618 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:57.618 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:57.618 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:57.618 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 118699 2024-09-21T15:00:57.618 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.619 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:57.619 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.619 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:57.619 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902625+0000", 2024-09-21T15:00:57.619 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:57.619 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.619 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.619 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:57.619 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902625+0000", 2024-09-21T15:00:57.619 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:57.619 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.619 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.619 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:57.619 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902625+0000", 2024-09-21T15:00:57.619 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:57.619 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.620 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.620 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:57.620 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902627+0000", 2024-09-21T15:00:57.620 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7269999999999999e-06 2024-09-21T15:00:57.620 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.620 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.620 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:57.620 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902630+0000", 2024-09-21T15:00:57.620 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.5200000000000002e-06 2024-09-21T15:00:57.620 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.620 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.620 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:57.620 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902633+0000", 2024-09-21T15:00:57.620 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8399999999999999e-06 2024-09-21T15:00:57.620 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.620 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.621 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:57.621 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.936700+0000", 2024-09-21T15:00:57.621 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034066723 2024-09-21T15:00:57.621 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.621 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.621 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:57.621 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.936778+0000", 2024-09-21T15:00:57.621 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8713000000000003e-05 2024-09-21T15:00:57.621 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.621 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.621 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:57.621 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.936870+0000", 2024-09-21T15:00:57.621 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1885000000000005e-05 2024-09-21T15:00:57.621 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:57.621 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:57.622 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:57.622 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.622 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.622 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:118700 3.12 3:4efbaa3f:::rbd_data.1136b1194d20.0000000000000006:58 [read 10321920~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:57.622 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.902635+0000", 2024-09-21T15:00:57.622 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 27.643635589999999, 2024-09-21T15:00:57.622 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034423665999999999, 2024-09-21T15:00:57.622 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:57.622 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:57.622 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:57.622 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:57.622 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:57.622 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 118700 2024-09-21T15:00:57.622 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.622 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:57.622 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.623 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:57.623 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902635+0000", 2024-09-21T15:00:57.623 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:57.623 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.623 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.623 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:57.623 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902635+0000", 2024-09-21T15:00:57.623 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:57.623 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.623 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.623 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:57.623 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902635+0000", 2024-09-21T15:00:57.623 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:57.623 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.623 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.623 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:57.623 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902637+0000", 2024-09-21T15:00:57.624 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.734e-06 2024-09-21T15:00:57.624 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.624 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.624 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:57.624 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902640+0000", 2024-09-21T15:00:57.624 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9720000000000001e-06 2024-09-21T15:00:57.624 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.624 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.624 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:57.624 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902642+0000", 2024-09-21T15:00:57.624 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.616e-06 2024-09-21T15:00:57.624 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.624 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.624 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:57.624 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.936904+0000", 2024-09-21T15:00:57.624 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034261982000000003 2024-09-21T15:00:57.625 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.625 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.625 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:57.625 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.936974+0000", 2024-09-21T15:00:57.625 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9406000000000006e-05 2024-09-21T15:00:57.625 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.625 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.625 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:57.625 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.937059+0000", 2024-09-21T15:00:57.625 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5315999999999994e-05 2024-09-21T15:00:57.625 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:57.625 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:57.625 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:57.625 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.625 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.625 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:118701 3.12 3:4efbaa3f:::rbd_data.1136b1194d20.0000000000000006:58 [read 10412032~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:57.625 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.902645+0000", 2024-09-21T15:00:57.626 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 27.643626145999999, 2024-09-21T15:00:57.626 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034600791999999998, 2024-09-21T15:00:57.626 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:57.626 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:57.626 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:57.626 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:57.626 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:57.626 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 118701 2024-09-21T15:00:57.626 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.626 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:57.626 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.626 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:57.626 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902645+0000", 2024-09-21T15:00:57.626 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:57.626 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.627 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.627 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:57.627 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902645+0000", 2024-09-21T15:00:57.627 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:57.627 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.627 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.627 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:57.627 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902644+0000", 2024-09-21T15:00:57.627 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:57.627 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.627 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.627 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:57.627 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902646+0000", 2024-09-21T15:00:57.627 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6169999999999999e-06 2024-09-21T15:00:57.627 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.628 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.628 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:57.628 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902649+0000", 2024-09-21T15:00:57.628 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9450000000000002e-06 2024-09-21T15:00:57.628 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.628 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.628 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:57.628 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902652+0000", 2024-09-21T15:00:57.628 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7889999999999999e-06 2024-09-21T15:00:57.628 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.628 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.628 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:57.628 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.937086+0000", 2024-09-21T15:00:57.628 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034433832999999997 2024-09-21T15:00:57.628 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.628 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.628 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:57.629 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.937152+0000", 2024-09-21T15:00:57.629 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5934999999999998e-05 2024-09-21T15:00:57.629 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.629 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.629 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:57.629 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.937246+0000", 2024-09-21T15:00:57.629 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3923000000000002e-05 2024-09-21T15:00:57.629 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:57.629 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:57.629 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:57.629 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:57.629 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:57.629 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:57.630 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T15:00:57.631 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:00:57.631+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:00:57.731 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:00:57.729+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:00:57.746 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:57.746 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:00:57.747 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T15:00:57.747 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:57.756 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T15:00:57.832 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:00:57.829+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:00:57.863 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:57.863 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:00:57.863 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T15:00:57.863 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T15:00:57.863 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:57.872 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T15:00:57.933 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:00:57.929+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:00:57.984 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:00:57.984 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T15:00:57.984 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T15:00:57.984 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T15:00:57.984 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.984 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31476 3.c 3:35718b6b:::rbd_data.11367721c19.0000000000000014:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T15:00:57.984 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.505975+0000", 2024-09-21T15:00:57.984 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 119.473877085, 2024-09-21T15:00:57.984 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99515995899999998, 2024-09-21T15:00:57.984 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:57.984 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:57.985 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:57.985 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:57.985 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:57.985 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31476 2024-09-21T15:00:57.985 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.985 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:57.985 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.985 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:57.985 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.505975+0000", 2024-09-21T15:00:57.985 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:57.985 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.985 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.985 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:57.985 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.505975+0000", 2024-09-21T15:00:57.985 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:57.986 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.986 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.986 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:57.986 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.505971+0000", 2024-09-21T15:00:57.986 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999967 2024-09-21T15:00:57.986 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.986 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.986 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:57.986 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.505989+0000", 2024-09-21T15:00:57.986 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7985000000000001e-05 2024-09-21T15:00:57.986 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.986 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.986 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:57.986 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506050+0000", 2024-09-21T15:00:57.986 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0659000000000003e-05 2024-09-21T15:00:57.987 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.987 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.987 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:57.987 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506074+0000", 2024-09-21T15:00:57.987 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4156e-05 2024-09-21T15:00:57.987 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.987 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.987 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:57.987 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506225+0000", 2024-09-21T15:00:57.987 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015097 2024-09-21T15:00:57.987 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.987 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.987 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:00:57.987 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506256+0000", 2024-09-21T15:00:57.987 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0505999999999999e-05 2024-09-21T15:00:57.987 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.988 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.988 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:57.988 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.497974+0000", 2024-09-21T15:00:57.988 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99171825700000005 2024-09-21T15:00:57.988 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.988 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.988 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for missing object", 2024-09-21T15:00:57.988 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.498449+0000", 2024-09-21T15:00:57.988 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00047506399999999998 2024-09-21T15:00:57.988 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.988 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.988 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:57.988 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.501000+0000", 2024-09-21T15:00:57.988 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0025506919999999998 2024-09-21T15:00:57.988 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.989 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.989 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:57.989 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.501135+0000", 2024-09-21T15:00:57.989 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013524000000000001 2024-09-21T15:00:57.989 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:57.989 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:57.989 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:57.989 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.989 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.989 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31480 3.c 3:36ccd808:::rbd_data.11367721c19.0000000000000018:head [sparse-read 0~14821] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T15:00:57.989 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.506116+0000", 2024-09-21T15:00:57.989 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 119.473735742, 2024-09-21T15:00:57.989 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99316033599999998, 2024-09-21T15:00:57.989 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:57.989 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:57.990 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:57.990 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:57.990 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:57.990 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31480 2024-09-21T15:00:57.990 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.990 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:57.990 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.990 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:57.990 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506116+0000", 2024-09-21T15:00:57.990 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:57.990 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.990 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.990 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:57.990 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506116+0000", 2024-09-21T15:00:57.990 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:57.991 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.991 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.991 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:57.991 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506115+0000", 2024-09-21T15:00:57.991 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:00:57.991 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.991 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.991 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:57.991 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506124+0000", 2024-09-21T15:00:57.991 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0659999999999993e-06 2024-09-21T15:00:57.991 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.991 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.991 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:57.991 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506145+0000", 2024-09-21T15:00:57.991 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1447e-05 2024-09-21T15:00:57.992 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.992 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.992 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:57.992 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506161+0000", 2024-09-21T15:00:57.992 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5209e-05 2024-09-21T15:00:57.992 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.992 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.992 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:57.992 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506337+0000", 2024-09-21T15:00:57.992 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017642999999999999 2024-09-21T15:00:57.992 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.992 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.992 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:00:57.992 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506375+0000", 2024-09-21T15:00:57.992 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7536000000000001e-05 2024-09-21T15:00:57.993 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.993 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.993 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:57.993 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.498815+0000", 2024-09-21T15:00:57.993 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99244067999999996 2024-09-21T15:00:57.993 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.993 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.993 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:57.993 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.498950+0000", 2024-09-21T15:00:57.993 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013436700000000001 2024-09-21T15:00:57.993 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.993 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.993 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:57.993 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499277+0000", 2024-09-21T15:00:57.993 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00032696800000000002 2024-09-21T15:00:57.993 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:57.993 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:57.994 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:57.994 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.994 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.994 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31513 3.c 3:36de8523:::rbd_data.11367721c19.0000000000000039:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T15:00:57.994 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.509071+0000", 2024-09-21T15:00:57.994 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 119.470781542, 2024-09-21T15:00:57.994 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99038225099999999, 2024-09-21T15:00:57.994 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:57.994 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:57.994 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:57.994 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:57.994 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:57.994 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31513 2024-09-21T15:00:57.994 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.994 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:57.995 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.995 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:57.995 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509071+0000", 2024-09-21T15:00:57.995 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:57.995 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.995 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.995 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:57.995 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509071+0000", 2024-09-21T15:00:57.995 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:57.995 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.995 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.995 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:57.995 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509068+0000", 2024-09-21T15:00:57.995 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-21T15:00:57.996 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.996 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.996 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:57.996 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509082+0000", 2024-09-21T15:00:57.996 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3545e-05 2024-09-21T15:00:57.996 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.996 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.996 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:57.996 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509116+0000", 2024-09-21T15:00:57.996 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3942000000000001e-05 2024-09-21T15:00:57.996 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.996 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.996 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:57.996 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509134+0000", 2024-09-21T15:00:57.996 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8839999999999999e-05 2024-09-21T15:00:57.997 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.997 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.997 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:57.997 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509318+0000", 2024-09-21T15:00:57.997 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018326799999999999 2024-09-21T15:00:57.997 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.997 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.997 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:00:57.997 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509346+0000", 2024-09-21T15:00:57.997 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8019000000000001e-05 2024-09-21T15:00:57.997 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.997 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.997 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:57.997 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499330+0000", 2024-09-21T15:00:57.997 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.98998399299999995 2024-09-21T15:00:57.997 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.998 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.998 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:57.998 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499453+0000", 2024-09-21T15:00:57.998 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000123115 2024-09-21T15:00:57.998 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:57.998 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:57.998 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:57.998 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.998 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.998 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31515 3.c 3:35b36ddd:::rbd_data.11367721c19.000000000000003b:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T15:00:57.998 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.509177+0000", 2024-09-21T15:00:57.998 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 119.470675116, 2024-09-21T15:00:57.998 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99044870600000001, 2024-09-21T15:00:57.998 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:57.998 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:00:57.999 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:57.999 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:57.999 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:57.999 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31515 2024-09-21T15:00:57.999 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.999 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:57.999 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.999 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:57.999 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509177+0000", 2024-09-21T15:00:57.999 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:57.999 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:57.999 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:57.999 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:57.999 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509177+0000", 2024-09-21T15:00:57.999 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:58.000 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.000 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.000 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:58.000 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509176+0000", 2024-09-21T15:00:58.000 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:00:58.000 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.000 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.000 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:58.000 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509183+0000", 2024-09-21T15:00:58.000 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6720000000000004e-06 2024-09-21T15:00:58.000 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.000 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.000 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:58.000 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509269+0000", 2024-09-21T15:00:58.000 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5881e-05 2024-09-21T15:00:58.000 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.000 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.001 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:58.001 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509303+0000", 2024-09-21T15:00:58.001 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3367000000000001e-05 2024-09-21T15:00:58.001 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.001 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.001 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:58.001 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509425+0000", 2024-09-21T15:00:58.001 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000122199 2024-09-21T15:00:58.001 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.001 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.001 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:00:58.001 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509448+0000", 2024-09-21T15:00:58.001 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2688999999999999e-05 2024-09-21T15:00:58.001 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.002 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.002 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:58.002 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499501+0000", 2024-09-21T15:00:58.002 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99005306999999998 2024-09-21T15:00:58.002 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.002 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.002 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:58.002 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499626+0000", 2024-09-21T15:00:58.002 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012501400000000001 2024-09-21T15:00:58.002 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:58.002 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:58.002 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:58.002 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.002 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.002 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31531 3.c 3:314f0287:::rbd_data.11367721c19.000000000000004b:head [sparse-read 0~7106] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T15:00:58.003 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.509329+0000", 2024-09-21T15:00:58.003 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 119.47052309599999, 2024-09-21T15:00:58.003 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99103085300000004, 2024-09-21T15:00:58.003 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:58.003 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:58.003 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:58.003 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:58.003 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:58.003 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31531 2024-09-21T15:00:58.003 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.003 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:58.003 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.003 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:58.003 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509329+0000", 2024-09-21T15:00:58.003 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:58.004 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.004 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.004 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:58.004 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509329+0000", 2024-09-21T15:00:58.004 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:58.004 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.004 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.004 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:58.004 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509328+0000", 2024-09-21T15:00:58.004 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:00:58.004 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.004 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.004 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:58.004 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509336+0000", 2024-09-21T15:00:58.004 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3170000000000002e-06 2024-09-21T15:00:58.005 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.005 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.005 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:58.005 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509353+0000", 2024-09-21T15:00:58.005 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7214000000000002e-05 2024-09-21T15:00:58.005 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.005 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.005 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:58.005 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509366+0000", 2024-09-21T15:00:58.005 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.26e-05 2024-09-21T15:00:58.005 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.005 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.005 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:58.005 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509523+0000", 2024-09-21T15:00:58.005 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015708000000000001 2024-09-21T15:00:58.006 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.006 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.006 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:00:58.006 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509544+0000", 2024-09-21T15:00:58.006 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0783000000000001e-05 2024-09-21T15:00:58.006 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.006 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.006 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:58.006 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499727+0000", 2024-09-21T15:00:58.006 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99018345200000002 2024-09-21T15:00:58.006 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.006 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.006 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:58.006 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499949+0000", 2024-09-21T15:00:58.006 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000221358 2024-09-21T15:00:58.007 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.007 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.007 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:58.007 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.500360+0000", 2024-09-21T15:00:58.007 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00041130700000000002 2024-09-21T15:00:58.007 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:58.007 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:58.007 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:58.007 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.007 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.007 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:82545 3.b 3:d4eb1ee5:::rbd_data.11368296f791.0000000000000000:head [write 0~671744 [fadvise_sequential+fadvise_nocache],write 704512~65536 [fadvise_sequential+fadvise_nocache],write 786432~49152 [fadvise_sequential+fadvise_nocache],write 868352~49152 [fadvise_sequential+fadvise_nocache],write 933888~65536 [fadvise_sequential+fadvise_nocache],write 1032192~32768 [fadvise_sequential+fadvise_nocache],write 1097728~65536 [fadvise_sequential+fadvise_nocache],write 1179648~49152 [fadvise_sequential+fadvise_nocache],write 1261568~49152 [fadvise_sequential+fadvise_nocache],write 1327104~65536 [fadvise_sequential+fadvise_nocache],write 1425408~32768 [fadvise_sequential+fadvise_nocache],write 1490944~65536 [fadvise_sequential+fadvise_nocache],write 1572864~8192 [fadvise_sequential+fadvise_nocache],write 1630208~49152 [fadvise_sequential+fadvise_nocache],write 1703936~40960 [fadvise_sequential+fadvise_nocache],write 1794048~40960 [fadvise_sequential+fadvise_nocache],write 1859584~49152 [fadvise_sequential+fadvise_nocache],write 1957888~8192 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:00:58.007 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:47.416089+0000", 2024-09-21T15:00:58.007 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 70.563763370999993, 2024-09-21T15:00:58.007 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011896455, 2024-09-21T15:00:58.007 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:58.007 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:58.008 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:58.008 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:58.008 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:58.008 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82545 2024-09-21T15:00:58.008 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.008 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:58.008 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.008 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:58.008 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.416089+0000", 2024-09-21T15:00:58.008 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:58.008 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.008 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.008 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:58.008 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.416089+0000", 2024-09-21T15:00:58.009 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:58.009 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.009 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.009 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:58.009 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.416087+0000", 2024-09-21T15:00:58.009 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:00:58.009 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.009 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.009 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:58.009 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.419643+0000", 2024-09-21T15:00:58.009 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0035555550000000002 2024-09-21T15:00:58.009 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.009 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.009 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:58.009 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.419663+0000", 2024-09-21T15:00:58.009 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0202000000000001e-05 2024-09-21T15:00:58.010 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.010 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.010 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:58.010 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.419676+0000", 2024-09-21T15:00:58.010 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3207000000000001e-05 2024-09-21T15:00:58.010 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.010 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.010 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:58.010 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.419759+0000", 2024-09-21T15:00:58.010 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3521000000000002e-05 2024-09-21T15:00:58.010 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.010 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.010 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op must wait for map", 2024-09-21T15:00:58.010 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.419768+0000", 2024-09-21T15:00:58.011 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4479999999999994e-06 2024-09-21T15:00:58.011 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.011 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.011 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:58.011 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.422171+0000", 2024-09-21T15:00:58.011 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0024035469999999998 2024-09-21T15:00:58.011 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.011 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.011 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:58.011 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.422350+0000", 2024-09-21T15:00:58.011 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000179001 2024-09-21T15:00:58.011 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.011 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.011 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T15:00:58.012 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.422710+0000", 2024-09-21T15:00:58.012 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000359878 2024-09-21T15:00:58.012 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.012 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.012 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:58.012 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.426676+0000", 2024-09-21T15:00:58.012 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0039655790000000003 2024-09-21T15:00:58.012 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.012 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.012 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:58.012 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.427884+0000", 2024-09-21T15:00:58.012 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001208207 2024-09-21T15:00:58.012 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.012 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.012 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:58.013 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.427953+0000", 2024-09-21T15:00:58.013 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8533000000000003e-05 2024-09-21T15:00:58.013 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.013 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.013 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:58.013 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.427985+0000", 2024-09-21T15:00:58.013 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2481999999999998e-05 2024-09-21T15:00:58.013 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:58.013 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:58.013 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:58.013 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.013 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.013 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:97675 3.b 3:d321acb8:::rbd_data.1136bfc79a70.0000000000000003:head [write 0~4718592 [fadvise_sequential+fadvise_nocache],write 5242880~147456 [fadvise_sequential+fadvise_nocache],write 5455872~147456 [fadvise_sequential+fadvise_nocache],write 5668864~229376 [fadvise_sequential+fadvise_nocache],write 5963776~147456 [fadvise_sequential+fadvise_nocache],write 6176768~229376 [fadvise_sequential+fadvise_nocache],write 6471680~147456 [fadvise_sequential+fadvise_nocache],write 6684672~229376 [fadvise_sequential+fadvise_nocache],write 6979584~147456 [fadvise_sequential+fadvise_nocache],write 7192576~229376 [fadvise_sequential+fadvise_nocache],write 7487488~147456 [fadvise_sequential+fadvise_nocache],write 7700480~147456 [fadvise_sequential+fadvise_nocache],write 7864320~65536 [fadvise_sequential+fadvise_nocache],write 7995392~147456 [fadvise_sequential+fadvise_nocache],write 8208384~147456 [fadvise_sequential+fadvise_nocache],write 8388608~49152 [fadvise_sequential+fadvise_nocache],write 8536064~114688 [fadvise_sequential+fadvise_nocache],write 8749056~114688 [fadvise_sequential+fadvise_nocache],write 8912896~32768 [fadvise_sequential+fadvise_nocache],write 9125888~32768 [fadvise_sequential+fadvise_nocache],write 9338880~32768 [fadvise_sequential+fadvise_nocache],write 9437184~16384 [fadvise_sequential+fadvise_nocache],write 9633792~32768 [fadvise_sequential+fadvise_nocache],write 9846784~32768 [fadvise_sequential+fadvise_nocache],write 9961472~114688 [fadvise_sequential+fadvise_nocache],write 10092544~196608 [fadvise_sequential+fadvise_nocache],write 10305536~278528 [fadvise_sequential+fadvise_nocache],write 10600448~196608 [fadvise_sequential+fadvise_nocache],write 10813440~278528 [fadvise_sequential+fadvise_nocache],write 11108352~196608 [fadvise_sequential+fadvise_nocache],write 11321344~196608 [fadvise_sequential+fadvise_nocache],write 11534336~65536 [fadvise_sequential+fadvise_nocache],write 11632640~98304 [fadvise_sequential+fadvise_nocache],write 11747328~65536 [fadvise_sequential+fadvise_nocache],write 11845632~98304 [fadvise_sequential+fadvise_nocache],write 11960320~65536 [fadvise_sequential+fadvise_nocache],write 12058624~49152 [fadvise_sequential+fadvise_nocache],write 12140544~98304 [fadvise_sequential+fadvise_nocache],write 12255232~65536 [fadvise_sequential+fadvise_nocache],write 12353536~98304 [fadvise_sequential+fadvise_nocache],write 12468224~65536 [fadvise_sequential+fadvise_nocache],write 12566528~16384 [fadvise_sequential+fadvise_nocache],write 12648448~98304 [fadvise_sequential+fadvise_nocache],write 12861440~98304 [fadvise_sequential+fadvise_nocache],write 13074432~32768 [fadvise_sequential+fadvise_nocache],write 13156352~98304 [fadvise_sequential+fadvise_nocache],write 13369344~98304 [fadvise_sequential+fadvise_nocache],write 13582336~49152 [fadvise_sequential+fadvise_nocache],write 13664256~98304 [fadvise_sequential+fadvise_nocache],write 13877248~98304 [fadvise_sequential+fadvise_nocache],write 14090240~65536 [fadvise_sequential+fadvise_nocache],write 15286272~81920 [fadvise_sequential+fadvise_nocache],write 15499264~81920 [fadvise_sequential+fadvise_nocache],write 15712256~16384 [fadvise_sequential+fadvise_nocache],write 15794176~81920 [fadvise_sequential+fadvise_nocache],write 16007168~81920 [fadvise_sequential+fadvise_nocache],write 16220160~32768 [fadvise_sequential+fadvise_nocache],write 16302080~81920 [fadvise_sequential+fadvise_nocache],write 16515072~81920 [fadvise_sequential+fadvise_nocache],write 16728064~49152 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:58.013 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.302412+0000", 2024-09-21T15:00:58.013 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 52.677440582000003, 2024-09-21T15:00:58.014 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.067380532000000007, 2024-09-21T15:00:58.014 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:58.014 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:58.014 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:58.014 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:58.014 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:58.014 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97675 2024-09-21T15:00:58.014 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.014 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:58.014 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.014 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:58.014 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.302412+0000", 2024-09-21T15:00:58.014 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:58.014 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.014 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.015 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:58.015 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.302412+0000", 2024-09-21T15:00:58.015 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:58.015 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.015 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.015 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:58.015 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.302409+0000", 2024-09-21T15:00:58.015 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-21T15:00:58.015 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.015 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.015 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:58.015 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328550+0000", 2024-09-21T15:00:58.015 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.026140674999999999 2024-09-21T15:00:58.015 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.015 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.016 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:58.016 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328584+0000", 2024-09-21T15:00:58.016 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4165999999999998e-05 2024-09-21T15:00:58.016 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.016 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.016 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:58.016 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328599+0000", 2024-09-21T15:00:58.016 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5505000000000001e-05 2024-09-21T15:00:58.016 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.016 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.016 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:58.016 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328704+0000", 2024-09-21T15:00:58.016 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010487199999999999 2024-09-21T15:00:58.016 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.016 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.016 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:58.017 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.329082+0000", 2024-09-21T15:00:58.017 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00037804899999999998 2024-09-21T15:00:58.017 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.017 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.017 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T15:00:58.017 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.331158+0000", 2024-09-21T15:00:58.017 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0020755449999999998 2024-09-21T15:00:58.017 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.017 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.017 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:00:58.017 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.360553+0000", 2024-09-21T15:00:58.017 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.029395553000000001 2024-09-21T15:00:58.017 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.017 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.018 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:58.018 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.369496+0000", 2024-09-21T15:00:58.018 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0089428619999999993 2024-09-21T15:00:58.018 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.018 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.018 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:58.018 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.369686+0000", 2024-09-21T15:00:58.018 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019000399999999999 2024-09-21T15:00:58.018 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.018 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.018 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:58.018 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.369792+0000", 2024-09-21T15:00:58.018 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000105775 2024-09-21T15:00:58.018 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:58.018 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:58.019 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:58.019 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.019 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.019 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:97693 3.b 3:d7d7b39b:::rbd_data.1136b9c49b1c.000000000000000c:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:58.019 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.328643+0000", 2024-09-21T15:00:58.019 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 52.651208795999999, 2024-09-21T15:00:58.019 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.031062136000000001, 2024-09-21T15:00:58.019 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:58.019 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:58.019 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:58.019 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:58.019 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:58.019 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97693 2024-09-21T15:00:58.019 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.019 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:58.019 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.020 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:58.020 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328643+0000", 2024-09-21T15:00:58.020 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:58.020 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.020 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.020 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:58.020 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328643+0000", 2024-09-21T15:00:58.020 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:58.020 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.020 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.020 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:58.020 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328642+0000", 2024-09-21T15:00:58.020 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:00:58.020 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.020 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.021 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:58.021 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328650+0000", 2024-09-21T15:00:58.021 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7770000000000008e-06 2024-09-21T15:00:58.021 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.021 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.021 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:58.021 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328662+0000", 2024-09-21T15:00:58.021 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2026e-05 2024-09-21T15:00:58.021 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.021 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.021 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:58.021 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328671+0000", 2024-09-21T15:00:58.021 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.585e-06 2024-09-21T15:00:58.021 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.021 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.022 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:58.022 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.359498+0000", 2024-09-21T15:00:58.022 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.030827001 2024-09-21T15:00:58.022 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.022 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.022 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:58.022 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.359603+0000", 2024-09-21T15:00:58.022 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010512400000000001 2024-09-21T15:00:58.022 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.022 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.022 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:58.022 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.359705+0000", 2024-09-21T15:00:58.022 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000102327 2024-09-21T15:00:58.022 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:58.022 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:58.023 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:58.023 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.023 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.023 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:97724 3.b 3:d4bf6bcf:::rbd_data.1136b9c49b1c.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:58.023 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.328704+0000", 2024-09-21T15:00:58.023 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 52.651148524, 2024-09-21T15:00:58.023 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.031157957, 2024-09-21T15:00:58.023 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:58.023 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:58.023 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:58.023 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:58.023 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:58.023 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97724 2024-09-21T15:00:58.023 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.023 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:58.023 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.024 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:58.024 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328704+0000", 2024-09-21T15:00:58.024 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:58.024 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.024 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.024 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:58.024 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328704+0000", 2024-09-21T15:00:58.024 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:58.024 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.024 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.024 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:58.024 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328703+0000", 2024-09-21T15:00:58.024 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:00:58.024 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.024 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.024 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:58.025 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328708+0000", 2024-09-21T15:00:58.025 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0050000000000004e-06 2024-09-21T15:00:58.025 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.025 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.025 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:58.025 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328719+0000", 2024-09-21T15:00:58.025 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1052e-05 2024-09-21T15:00:58.025 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.025 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.025 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:58.025 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328727+0000", 2024-09-21T15:00:58.025 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4519999999999997e-06 2024-09-21T15:00:58.025 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.025 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.025 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:58.026 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.359737+0000", 2024-09-21T15:00:58.026 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.031009341999999999 2024-09-21T15:00:58.026 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.026 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.026 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:58.026 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.359806+0000", 2024-09-21T15:00:58.026 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9750000000000001e-05 2024-09-21T15:00:58.026 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.026 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.026 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:58.026 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.359862+0000", 2024-09-21T15:00:58.026 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5084999999999999e-05 2024-09-21T15:00:58.026 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:58.026 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:58.026 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:58.027 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.027 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.027 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:129970 3.c e158/87 3:33fa4d4e:::rbd_data.1136a12e4351.0000000000000000:head v 158'379, mlcod=156'377)", 2024-09-21T15:00:58.027 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:44.452766+0000", 2024-09-21T15:00:58.027 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 13.527085972, 2024-09-21T15:00:58.027 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011942062, 2024-09-21T15:00:58.027 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:58.027 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:58.027 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:58.027 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.027 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:58.027 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:44.452766+0000", 2024-09-21T15:00:58.027 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:58.027 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.027 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.028 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:58.028 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:44.452766+0000", 2024-09-21T15:00:58.028 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:58.028 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.028 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.028 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:58.028 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:44.452766+0000", 2024-09-21T15:00:58.028 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:58.028 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.028 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.028 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:58.028 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:44.458962+0000", 2024-09-21T15:00:58.028 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0061964799999999999 2024-09-21T15:00:58.028 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.028 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.029 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:58.029 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:44.459012+0000", 2024-09-21T15:00:58.029 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9427e-05 2024-09-21T15:00:58.029 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.029 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.029 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:58.029 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:44.459042+0000", 2024-09-21T15:00:58.029 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9838000000000001e-05 2024-09-21T15:00:58.029 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.029 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.029 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:58.029 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:44.459115+0000", 2024-09-21T15:00:58.029 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3430999999999995e-05 2024-09-21T15:00:58.029 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.029 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.030 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:58.030 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:44.459135+0000", 2024-09-21T15:00:58.030 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9701999999999999e-05 2024-09-21T15:00:58.030 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.030 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.030 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:00:58.030 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:44.464645+0000", 2024-09-21T15:00:58.030 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0055100820000000003 2024-09-21T15:00:58.030 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.030 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.030 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:58.030 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:44.464708+0000", 2024-09-21T15:00:58.030 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3399999999999996e-05 2024-09-21T15:00:58.030 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:58.030 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:58.031 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:58.031 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.031 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.031 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:135330 3.b 3:d3301c39:::rbd_data.11365ed419e3.0000000000000102:67 [read 270336~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:00:58.031 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:53.998760+0000", 2024-09-21T15:00:58.031 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.9810924399999998, 2024-09-21T15:00:58.031 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0097195630000000005, 2024-09-21T15:00:58.031 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:58.031 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:58.031 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:58.031 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:58.031 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:58.031 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 135330 2024-09-21T15:00:58.031 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.031 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:58.032 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.032 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:58.032 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998760+0000", 2024-09-21T15:00:58.032 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:58.032 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.032 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.032 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:58.032 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998760+0000", 2024-09-21T15:00:58.032 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:58.032 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.032 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.032 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:58.032 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998759+0000", 2024-09-21T15:00:58.032 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:58.033 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.033 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.033 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:58.033 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998762+0000", 2024-09-21T15:00:58.033 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.266e-06 2024-09-21T15:00:58.033 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.033 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.033 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:58.033 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998767+0000", 2024-09-21T15:00:58.033 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4219999999999999e-06 2024-09-21T15:00:58.033 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.033 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.033 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:58.033 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998772+0000", 2024-09-21T15:00:58.033 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5589999999999999e-06 2024-09-21T15:00:58.034 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.034 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.034 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:58.034 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:54.008368+0000", 2024-09-21T15:00:58.034 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0095967269999999993 2024-09-21T15:00:58.034 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.034 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.034 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:58.034 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:54.008413+0000", 2024-09-21T15:00:58.034 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4941000000000002e-05 2024-09-21T15:00:58.034 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.034 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.034 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:58.034 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:54.008479+0000", 2024-09-21T15:00:58.035 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5959999999999999e-05 2024-09-21T15:00:58.035 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:58.035 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:58.035 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:58.035 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.035 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.035 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:135331 3.b 3:d3301c39:::rbd_data.11365ed419e3.0000000000000102:67 [read 286720~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:00:58.035 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:53.998776+0000", 2024-09-21T15:00:58.035 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.9810765429999999, 2024-09-21T15:00:58.035 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0098308560000000007, 2024-09-21T15:00:58.035 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:58.035 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:58.035 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:58.035 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:58.036 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:58.036 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 135331 2024-09-21T15:00:58.036 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.036 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:58.036 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.036 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:58.036 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998776+0000", 2024-09-21T15:00:58.036 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:58.036 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.036 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.036 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:58.036 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998776+0000", 2024-09-21T15:00:58.036 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:58.036 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.036 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.036 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:58.037 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998775+0000", 2024-09-21T15:00:58.037 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:58.037 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.037 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.037 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:58.037 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998777+0000", 2024-09-21T15:00:58.037 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2120000000000002e-06 2024-09-21T15:00:58.037 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.037 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.037 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:58.037 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998783+0000", 2024-09-21T15:00:58.037 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5899999999999998e-06 2024-09-21T15:00:58.037 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.037 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.038 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:58.038 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998786+0000", 2024-09-21T15:00:58.038 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0060000000000001e-06 2024-09-21T15:00:58.038 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.038 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.038 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:58.038 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:54.008500+0000", 2024-09-21T15:00:58.038 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0097136150000000001 2024-09-21T15:00:58.038 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.038 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.038 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:58.038 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:54.008543+0000", 2024-09-21T15:00:58.038 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3464999999999998e-05 2024-09-21T15:00:58.038 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.039 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.039 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:58.039 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:54.008606+0000", 2024-09-21T15:00:58.039 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3287999999999994e-05 2024-09-21T15:00:58.039 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:58.039 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:58.039 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:58.039 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.039 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.039 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:135354 3.b 3:d3301c39:::rbd_data.11365ed419e3.0000000000000102:67 [read 294912~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:00:58.039 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:53.998789+0000", 2024-09-21T15:00:58.039 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.9810632889999997, 2024-09-21T15:00:58.039 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010057417000000001, 2024-09-21T15:00:58.039 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:58.039 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:58.040 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:58.040 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:58.040 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:58.040 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 135354 2024-09-21T15:00:58.040 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.040 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:58.040 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.040 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:58.040 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998789+0000", 2024-09-21T15:00:58.040 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:58.040 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.040 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.040 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:58.040 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998789+0000", 2024-09-21T15:00:58.040 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:58.041 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.041 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.041 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:58.041 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998788+0000", 2024-09-21T15:00:58.041 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:58.041 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.041 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.041 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:58.041 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998801+0000", 2024-09-21T15:00:58.041 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2088e-05 2024-09-21T15:00:58.041 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.041 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.041 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:58.041 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998807+0000", 2024-09-21T15:00:58.042 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6340000000000003e-06 2024-09-21T15:00:58.042 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.042 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.042 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:58.042 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998812+0000", 2024-09-21T15:00:58.042 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.215e-06 2024-09-21T15:00:58.042 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.042 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.042 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:58.042 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:54.008655+0000", 2024-09-21T15:00:58.042 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0098430540000000004 2024-09-21T15:00:58.042 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.042 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.043 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:58.043 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:54.008753+0000", 2024-09-21T15:00:58.043 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7026000000000002e-05 2024-09-21T15:00:58.043 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.043 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.043 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:58.043 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:54.008846+0000", 2024-09-21T15:00:58.043 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3696999999999994e-05 2024-09-21T15:00:58.043 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:58.043 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:58.043 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:58.043 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.043 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.043 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:135355 3.b 3:d3301c39:::rbd_data.11365ed419e3.0000000000000102:67 [read 311296~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:00:58.044 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:53.998817+0000", 2024-09-21T15:00:58.044 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.9810352089999999, 2024-09-21T15:00:58.044 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010208788999999999, 2024-09-21T15:00:58.044 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:58.044 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:58.044 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:58.044 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:58.044 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:58.044 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 135355 2024-09-21T15:00:58.044 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.044 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:58.044 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.044 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:58.044 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998817+0000", 2024-09-21T15:00:58.044 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:58.045 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.045 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.045 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:58.045 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998817+0000", 2024-09-21T15:00:58.045 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:58.045 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.045 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.045 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:58.045 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998817+0000", 2024-09-21T15:00:58.045 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:58.045 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.045 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.045 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:58.045 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998820+0000", 2024-09-21T15:00:58.046 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2229999999999999e-06 2024-09-21T15:00:58.046 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.046 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.046 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:58.046 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998825+0000", 2024-09-21T15:00:58.046 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5539999999999997e-06 2024-09-21T15:00:58.046 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.046 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.046 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:58.046 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998830+0000", 2024-09-21T15:00:58.046 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8590000000000004e-06 2024-09-21T15:00:58.046 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.046 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.046 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:58.046 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:54.008882+0000", 2024-09-21T15:00:58.047 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010052238999999999 2024-09-21T15:00:58.047 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.047 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.047 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:58.047 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:54.008954+0000", 2024-09-21T15:00:58.047 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1748000000000005e-05 2024-09-21T15:00:58.047 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.047 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.047 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:58.047 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:54.009026+0000", 2024-09-21T15:00:58.047 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1538999999999994e-05 2024-09-21T15:00:58.047 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:58.047 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:58.047 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:58.048 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.048 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.048 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:135356 3.b 3:d3301c39:::rbd_data.11365ed419e3.0000000000000102:67 [read 327680~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:00:58.048 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:53.998835+0000", 2024-09-21T15:00:58.048 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.9810175550000002, 2024-09-21T15:00:58.048 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010668557, 2024-09-21T15:00:58.048 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:58.048 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:58.048 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:58.048 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:58.048 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:58.048 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 135356 2024-09-21T15:00:58.048 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.048 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:58.049 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.049 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:58.049 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998835+0000", 2024-09-21T15:00:58.049 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:58.049 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.049 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.049 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:58.049 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998835+0000", 2024-09-21T15:00:58.049 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:58.049 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.049 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.049 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:58.049 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998834+0000", 2024-09-21T15:00:58.049 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:58.049 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.049 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.050 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:58.050 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998837+0000", 2024-09-21T15:00:58.050 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5689999999999999e-06 2024-09-21T15:00:58.050 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.050 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.050 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:58.050 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998842+0000", 2024-09-21T15:00:58.050 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6540000000000004e-06 2024-09-21T15:00:58.050 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.050 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.050 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:58.050 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998847+0000", 2024-09-21T15:00:58.050 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0069999999999997e-06 2024-09-21T15:00:58.050 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.050 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.051 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:58.051 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:54.009075+0000", 2024-09-21T15:00:58.051 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010227192 2024-09-21T15:00:58.051 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.051 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.051 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:58.051 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:54.009146+0000", 2024-09-21T15:00:58.051 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1230999999999996e-05 2024-09-21T15:00:58.051 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.051 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.051 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:58.051 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:54.009503+0000", 2024-09-21T15:00:58.051 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00035721599999999999 2024-09-21T15:00:58.051 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:58.052 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:58.052 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:58.052 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.052 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.052 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:135357 3.b 3:d3301c39:::rbd_data.11365ed419e3.0000000000000102:67 [read 344064~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:00:58.052 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:53.998852+0000", 2024-09-21T15:00:58.052 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.9810004700000001, 2024-09-21T15:00:58.052 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011570637, 2024-09-21T15:00:58.052 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:58.052 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:58.052 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:58.052 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:58.052 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:58.052 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 135357 2024-09-21T15:00:58.053 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.053 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:58.053 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.053 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:58.053 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998852+0000", 2024-09-21T15:00:58.053 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:58.053 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.053 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.053 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:58.053 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998852+0000", 2024-09-21T15:00:58.053 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:58.053 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.053 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.053 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:58.053 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998851+0000", 2024-09-21T15:00:58.054 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:58.054 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.054 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.054 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:58.054 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998854+0000", 2024-09-21T15:00:58.054 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3609999999999999e-06 2024-09-21T15:00:58.054 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.054 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.054 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:58.054 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998858+0000", 2024-09-21T15:00:58.054 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3619999999999999e-06 2024-09-21T15:00:58.054 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.054 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.055 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:58.055 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998863+0000", 2024-09-21T15:00:58.055 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5360000000000003e-06 2024-09-21T15:00:58.055 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.055 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.055 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:58.055 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:54.009531+0000", 2024-09-21T15:00:58.055 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.010667972 2024-09-21T15:00:58.055 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.055 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.055 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:58.055 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:54.009610+0000", 2024-09-21T15:00:58.055 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9874000000000001e-05 2024-09-21T15:00:58.055 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.055 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.056 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:58.056 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:54.010422+0000", 2024-09-21T15:00:58.056 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00081185000000000003 2024-09-21T15:00:58.056 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:58.056 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:58.056 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:58.056 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.056 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.056 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:135366 3.b 3:d3301c39:::rbd_data.11365ed419e3.0000000000000102:67 [read 303104~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:00:58.056 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:53.998866+0000", 2024-09-21T15:00:58.056 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.9809857439999998, 2024-09-21T15:00:58.056 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011198392, 2024-09-21T15:00:58.056 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:58.056 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:58.057 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:58.057 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:58.057 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:58.057 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 135366 2024-09-21T15:00:58.057 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.057 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:58.057 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.057 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:58.057 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998866+0000", 2024-09-21T15:00:58.057 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:58.057 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.057 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.057 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:58.057 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998866+0000", 2024-09-21T15:00:58.057 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:58.058 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.058 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.058 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:58.058 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998866+0000", 2024-09-21T15:00:58.058 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:58.058 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.058 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.058 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:58.058 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998868+0000", 2024-09-21T15:00:58.058 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4059999999999999e-06 2024-09-21T15:00:58.058 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.058 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.058 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:58.058 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998874+0000", 2024-09-21T15:00:58.059 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8289999999999996e-06 2024-09-21T15:00:58.059 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.059 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.059 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:58.059 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998879+0000", 2024-09-21T15:00:58.059 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8579999999999999e-06 2024-09-21T15:00:58.059 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.059 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.059 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:58.059 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:54.009885+0000", 2024-09-21T15:00:58.059 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011005716 2024-09-21T15:00:58.059 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.059 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.059 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:58.060 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:54.009973+0000", 2024-09-21T15:00:58.060 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8502000000000001e-05 2024-09-21T15:00:58.060 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.060 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.060 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:58.060 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:54.010065+0000", 2024-09-21T15:00:58.060 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.1391999999999995e-05 2024-09-21T15:00:58.060 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:58.060 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:58.060 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:58.060 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.060 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.060 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:135367 3.b 3:d3301c39:::rbd_data.11365ed419e3.0000000000000102:67 [read 319488~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:00:58.060 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:53.998883+0000", 2024-09-21T15:00:58.061 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.9809687609999997, 2024-09-21T15:00:58.061 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011368680000000001, 2024-09-21T15:00:58.061 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:58.061 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:58.061 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:58.061 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:58.061 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:58.061 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 135367 2024-09-21T15:00:58.061 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.061 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:58.061 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.061 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:58.061 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998883+0000", 2024-09-21T15:00:58.061 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:58.061 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.062 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.062 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:58.062 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998883+0000", 2024-09-21T15:00:58.062 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:58.062 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.062 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.062 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:58.062 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998883+0000", 2024-09-21T15:00:58.062 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:58.062 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.062 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.062 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:58.062 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998885+0000", 2024-09-21T15:00:58.062 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.418e-06 2024-09-21T15:00:58.063 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.063 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.063 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:58.063 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998891+0000", 2024-09-21T15:00:58.063 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7030000000000003e-06 2024-09-21T15:00:58.063 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.063 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.063 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:58.063 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998896+0000", 2024-09-21T15:00:58.063 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9470000000000002e-06 2024-09-21T15:00:58.063 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.063 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.063 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:58.063 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:54.010082+0000", 2024-09-21T15:00:58.063 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011185502999999999 2024-09-21T15:00:58.064 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.064 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.064 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:58.064 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:54.010151+0000", 2024-09-21T15:00:58.064 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9356999999999993e-05 2024-09-21T15:00:58.065 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.065 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.065 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:58.065 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:54.010252+0000", 2024-09-21T15:00:58.065 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010105 2024-09-21T15:00:58.066 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:58.066 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:58.066 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:58.066 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.066 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.066 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:135368 3.b 3:d3301c39:::rbd_data.11365ed419e3.0000000000000102:67 [read 335872~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:00:58.066 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:53.998901+0000", 2024-09-21T15:00:58.066 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.98095116, 2024-09-21T15:00:58.066 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011666174, 2024-09-21T15:00:58.066 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:58.066 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:58.066 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:58.066 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:58.066 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:58.067 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 135368 2024-09-21T15:00:58.067 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.067 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:58.067 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.067 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:58.067 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998901+0000", 2024-09-21T15:00:58.067 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:58.067 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.067 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.067 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:58.067 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998901+0000", 2024-09-21T15:00:58.067 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:58.067 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.067 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.067 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:58.067 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998901+0000", 2024-09-21T15:00:58.068 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:58.068 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.068 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.068 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:58.068 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998903+0000", 2024-09-21T15:00:58.068 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5500000000000001e-06 2024-09-21T15:00:58.068 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.068 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.068 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:58.068 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998913+0000", 2024-09-21T15:00:58.068 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6479999999999998e-06 2024-09-21T15:00:58.068 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.068 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.069 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:58.069 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998919+0000", 2024-09-21T15:00:58.069 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7930000000000003e-06 2024-09-21T15:00:58.069 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.069 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.069 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:58.069 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:54.010277+0000", 2024-09-21T15:00:58.069 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011358692 2024-09-21T15:00:58.069 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.069 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.069 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:58.069 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:54.010348+0000", 2024-09-21T15:00:58.069 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0247999999999996e-05 2024-09-21T15:00:58.069 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.069 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.069 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:58.070 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:54.010567+0000", 2024-09-21T15:00:58.070 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021953599999999999 2024-09-21T15:00:58.070 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:58.070 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:58.070 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:58.070 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.070 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.070 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:135369 3.b 3:d3301c39:::rbd_data.11365ed419e3.0000000000000102:67 [read 352256~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:00:58.070 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:53.998923+0000", 2024-09-21T15:00:58.070 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 3.980928859, 2024-09-21T15:00:58.070 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011990331999999999, 2024-09-21T15:00:58.070 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:00:58.070 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:00:58.070 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:00:58.070 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:00:58.071 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:58.071 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 135369 2024-09-21T15:00:58.071 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.071 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:00:58.071 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.071 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:00:58.071 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998923+0000", 2024-09-21T15:00:58.071 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:58.071 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.071 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.071 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:00:58.071 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998923+0000", 2024-09-21T15:00:58.071 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:00:58.071 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.071 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.071 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:00:58.072 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998923+0000", 2024-09-21T15:00:58.072 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:58.072 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.072 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.072 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:00:58.072 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998926+0000", 2024-09-21T15:00:58.072 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6749999999999998e-06 2024-09-21T15:00:58.072 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.072 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.072 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:00:58.072 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998935+0000", 2024-09-21T15:00:58.072 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3619999999999995e-06 2024-09-21T15:00:58.072 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.072 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.072 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:00:58.073 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998940+0000", 2024-09-21T15:00:58.073 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2379999999999997e-06 2024-09-21T15:00:58.073 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.073 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.073 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:00:58.073 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:54.010593+0000", 2024-09-21T15:00:58.073 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011652531000000001 2024-09-21T15:00:58.073 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.073 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.073 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:00:58.073 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:54.010679+0000", 2024-09-21T15:00:58.073 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6811000000000006e-05 2024-09-21T15:00:58.073 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:00:58.073 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:00:58.073 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:00:58.074 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:54.010914+0000", 2024-09-21T15:00:58.074 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000234071 2024-09-21T15:00:58.074 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:58.074 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:58.074 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:58.074 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:00:58.074 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:00:58.074 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:00:58.075 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T15:00:58.076 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:00:58.071+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:00:58.176 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:00:58.173+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:00:58.180 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T15:00:58.201 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:58.201 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:00:58.201 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T15:00:58.202 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:58.219 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T15:00:58.251 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 5, 6, 7, 4] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T15:00:58.251 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-21T15:00:58.277 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:00:58.273+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:00:58.378 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:00:58.373+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:00:58.386 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:58.386 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:00:58.386 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T15:00:58.386 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T15:00:58.386 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:58.402 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T15:00:58.479 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:00:58.477+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:00:58.579 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:00:58.577+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:00:58.588 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:58.588 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T15:00:58.588 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T15:00:58.588 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T15:00:58.588 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.588 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:97677 3.15 3:a8e72133:::rbd_data.1136bfc79a70.0000000000000005:head [write 0~4440027 [fadvise_sequential+fadvise_nocache],write 4751360~147456 [fadvise_sequential+fadvise_nocache],write 4964352~147456 [fadvise_sequential+fadvise_nocache],write 5177344~65536 [fadvise_sequential+fadvise_nocache],write 5259264~147456 [fadvise_sequential+fadvise_nocache],write 5472256~147456 [fadvise_sequential+fadvise_nocache],write 5685248~229376 [fadvise_sequential+fadvise_nocache],write 5980160~147456 [fadvise_sequential+fadvise_nocache],write 6193152~229376 [fadvise_sequential+fadvise_nocache],write 6488064~147456 [fadvise_sequential+fadvise_nocache],write 6701056~229376 [fadvise_sequential+fadvise_nocache],write 6995968~147456 [fadvise_sequential+fadvise_nocache],write 7208960~229376 [fadvise_sequential+fadvise_nocache],write 7503872~147456 [fadvise_sequential+fadvise_nocache],write 7716864~229376 [fadvise_sequential+fadvise_nocache],write 8044544~114688 [fadvise_sequential+fadvise_nocache],write 8257536~114688 [fadvise_sequential+fadvise_nocache],write 8388608~65536 [fadvise_sequential+fadvise_nocache],write 8552448~114688 [fadvise_sequential+fadvise_nocache],write 8765440~114688 [fadvise_sequential+fadvise_nocache],write 8929280~32768 [fadvise_sequential+fadvise_nocache],write 9142272~32768 [fadvise_sequential+fadvise_nocache],write 9355264~32768 [fadvise_sequential+fadvise_nocache],write 9437184~32768 [fadvise_sequential+fadvise_nocache],write 9650176~32768 [fadvise_sequential+fadvise_nocache],write 9863168~32768 [fadvise_sequential+fadvise_nocache],write 9961472~131072 [fadvise_sequential+fadvise_nocache],write 10108928~196608 [fadvise_sequential+fadvise_nocache],write 10321920~278528 [fadvise_sequential+fadvise_nocache],write 10616832~196608 [fadvise_sequential+fadvise_nocache],write 10829824~278528 [fadvise_sequential+fadvise_nocache],write 11124736~196608 [fadvise_sequential+fadvise_nocache],write 11337728~196608 [fadvise_sequential+fadvise_nocache],write 11550720~65536 [fadvise_sequential+fadvise_nocache],write 11649024~98304 [fadvise_sequential+fadvise_nocache],write 11763712~65536 [fadvise_sequential+fadvise_nocache],write 11862016~98304 [fadvise_sequential+fadvise_nocache],write 11976704~65536 [fadvise_sequential+fadvise_nocache],write 12058624~65536 [fadvise_sequential+fadvise_nocache],write 12156928~98304 [fadvise_sequential+fadvise_nocache],write 12271616~65536 [fadvise_sequential+fadvise_nocache],write 12369920~98304 [fadvise_sequential+fadvise_nocache],write 12484608~65536 [fadvise_sequential+fadvise_nocache],write 12664832~98304 [fadvise_sequential+fadvise_nocache],write 12877824~98304 [fadvise_sequential+fadvise_nocache],write 13090816~16384 [fadvise_sequential+fadvise_nocache],write 13172736~98304 [fadvise_sequential+fadvise_nocache],write 13385728~98304 [fadvise_sequential+fadvise_nocache],write 13598720~32768 [fadvise_sequential+fadvise_nocache],write 13680640~98304 [fadvise_sequential+fadvise_nocache],write 13893632~98304 [fadvise_sequential+fadvise_nocache],write 14106624~49152 [fadvise_sequential+fadvise_nocache],write 14794752~81920 [fadvise_sequential+fadvise_nocache],write 15007744~81920 [fadvise_sequential+fadvise_nocache],write 15302656~81920 [fadvise_sequential+fadvise_nocache],write 15515648~81920 [fadvise_sequential+fadvise_nocache],write 15810560~81920 [fadvise_sequential+fadvise_nocache],write 16023552~81920 [fadvise_sequential+fadvise_nocache],write 16236544~16384 [fadvise_sequential+fadvise_nocache],write 16318464~81920 [fadvise_sequential+fadvise_nocache],write 16531456~81920 [fadvise_sequential+fadvise_nocache],write 16744448~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:00:58.588 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.302946+0000", 2024-09-21T15:00:58.589 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 53.283824115999998, 2024-09-21T15:00:58.589 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.091231204999999996, 2024-09-21T15:00:58.589 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:58.589 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:58.589 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:58.589 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:58.589 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:58.589 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 97677 2024-09-21T15:00:58.589 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.589 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:58.589 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.589 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:58.589 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.302946+0000", 2024-09-21T15:00:58.589 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:58.589 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.590 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.590 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:58.590 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.302946+0000", 2024-09-21T15:00:58.590 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:58.590 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.590 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.590 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:58.590 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.302942+0000", 2024-09-21T15:00:58.590 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999962 2024-09-21T15:00:58.590 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.590 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.590 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:58.590 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311253+0000", 2024-09-21T15:00:58.590 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0083105360000000003 2024-09-21T15:00:58.590 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.591 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.591 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:58.591 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311283+0000", 2024-09-21T15:00:58.591 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0065000000000001e-05 2024-09-21T15:00:58.591 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.591 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.591 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:58.591 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311296+0000", 2024-09-21T15:00:58.591 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3206e-05 2024-09-21T15:00:58.591 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.591 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.591 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:58.591 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311404+0000", 2024-09-21T15:00:58.591 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000107514 2024-09-21T15:00:58.591 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.591 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.591 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:58.592 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311795+0000", 2024-09-21T15:00:58.592 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000390913 2024-09-21T15:00:58.592 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.592 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.592 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 6", 2024-09-21T15:00:58.592 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.314686+0000", 2024-09-21T15:00:58.592 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0028915659999999999 2024-09-21T15:00:58.592 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.592 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.592 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:58.592 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.368178+0000", 2024-09-21T15:00:58.592 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.053491706 2024-09-21T15:00:58.592 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.592 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.592 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:58.592 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.393925+0000", 2024-09-21T15:00:58.593 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025747289 2024-09-21T15:00:58.593 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.593 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.593 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:58.593 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.394127+0000", 2024-09-21T15:00:58.593 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000201444 2024-09-21T15:00:58.593 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.593 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.593 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:58.593 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.394177+0000", 2024-09-21T15:00:58.593 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0689000000000002e-05 2024-09-21T15:00:58.593 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:58.593 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:58.593 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:58.593 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.593 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.594 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:97673 3.1d e122/24 3:bd91946e:::rbd_data.1136bfc79a70.0000000000000002:head v 122'162, mlcod=120'160)", 2024-09-21T15:00:58.594 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.313730+0000", 2024-09-21T15:00:58.594 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 53.273039791000002, 2024-09-21T15:00:58.594 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055373462999999998, 2024-09-21T15:00:58.594 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:58.594 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:58.594 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:58.594 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.594 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:58.594 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.313730+0000", 2024-09-21T15:00:58.594 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:58.594 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.594 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.594 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:58.594 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.313730+0000", 2024-09-21T15:00:58.594 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:58.595 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.595 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.595 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:58.595 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.313730+0000", 2024-09-21T15:00:58.595 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:58.595 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.595 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.595 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:58.595 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.336113+0000", 2024-09-21T15:00:58.595 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022382962999999999 2024-09-21T15:00:58.595 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.595 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.595 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:58.595 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.336140+0000", 2024-09-21T15:00:58.595 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7248999999999999e-05 2024-09-21T15:00:58.596 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.596 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.596 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:58.596 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.336151+0000", 2024-09-21T15:00:58.596 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1015e-05 2024-09-21T15:00:58.596 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.596 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.596 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:58.596 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.355428+0000", 2024-09-21T15:00:58.596 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019276379999999999 2024-09-21T15:00:58.596 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.596 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.596 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:58.596 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.355484+0000", 2024-09-21T15:00:58.596 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6820999999999997e-05 2024-09-21T15:00:58.596 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.597 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.597 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:58.597 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.369040+0000", 2024-09-21T15:00:58.597 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013555466 2024-09-21T15:00:58.597 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.597 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.597 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:58.597 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.369104+0000", 2024-09-21T15:00:58.597 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3986999999999996e-05 2024-09-21T15:00:58.597 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:58.597 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:58.597 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:58.597 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.597 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.597 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:113889 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 5300224~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:58.597 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.501880+0000", 2024-09-21T15:00:58.598 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 29.084889877999998, 2024-09-21T15:00:58.598 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052644204999999999, 2024-09-21T15:00:58.598 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:58.598 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:58.598 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:58.598 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:58.598 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:58.598 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 113889 2024-09-21T15:00:58.598 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.598 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:58.598 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.598 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:58.598 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501880+0000", 2024-09-21T15:00:58.598 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:58.598 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.598 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.599 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:58.599 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501880+0000", 2024-09-21T15:00:58.599 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:58.599 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.599 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.599 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:58.599 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501880+0000", 2024-09-21T15:00:58.599 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:00:58.599 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.599 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.599 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:58.599 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501882+0000", 2024-09-21T15:00:58.599 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.364e-06 2024-09-21T15:00:58.600 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.600 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.600 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:58.600 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501885+0000", 2024-09-21T15:00:58.600 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4199999999999999e-06 2024-09-21T15:00:58.600 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.600 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.600 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:58.600 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501888+0000", 2024-09-21T15:00:58.600 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8669999999999998e-06 2024-09-21T15:00:58.600 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.600 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.600 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:58.600 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.554369+0000", 2024-09-21T15:00:58.601 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052481456000000003 2024-09-21T15:00:58.601 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.601 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.601 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:58.601 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.554449+0000", 2024-09-21T15:00:58.601 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9412000000000005e-05 2024-09-21T15:00:58.601 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.601 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.601 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:58.601 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.554525+0000", 2024-09-21T15:00:58.601 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5908999999999994e-05 2024-09-21T15:00:58.601 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:58.601 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:58.601 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:58.601 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.601 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.602 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:113890 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 5390336~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:58.602 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.501890+0000", 2024-09-21T15:00:58.602 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 29.084880163000001, 2024-09-21T15:00:58.602 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052785212999999997, 2024-09-21T15:00:58.602 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:58.602 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:58.602 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:58.602 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:58.602 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:58.602 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 113890 2024-09-21T15:00:58.602 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.602 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:58.602 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.602 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:58.603 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501890+0000", 2024-09-21T15:00:58.603 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:58.603 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.603 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.603 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:58.603 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501890+0000", 2024-09-21T15:00:58.603 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:58.603 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.603 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.603 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:58.603 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501890+0000", 2024-09-21T15:00:58.603 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:00:58.603 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.603 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.604 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:58.604 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501891+0000", 2024-09-21T15:00:58.604 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3599999999999999e-06 2024-09-21T15:00:58.604 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.604 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.604 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:58.604 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501894+0000", 2024-09-21T15:00:58.604 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.717e-06 2024-09-21T15:00:58.604 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.604 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.604 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:58.604 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501897+0000", 2024-09-21T15:00:58.604 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5790000000000002e-06 2024-09-21T15:00:58.604 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.604 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.605 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:58.605 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.554569+0000", 2024-09-21T15:00:58.605 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052672434999999997 2024-09-21T15:00:58.605 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.605 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.605 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:58.605 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.554623+0000", 2024-09-21T15:00:58.605 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4509999999999998e-05 2024-09-21T15:00:58.605 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.605 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.605 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:58.605 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.554675+0000", 2024-09-21T15:00:58.605 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1833999999999999e-05 2024-09-21T15:00:58.605 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:58.606 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:58.606 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:58.606 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.606 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.606 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:113891 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 5480448~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:58.606 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.501899+0000", 2024-09-21T15:00:58.606 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 29.084871676999999, 2024-09-21T15:00:58.606 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052991637000000001, 2024-09-21T15:00:58.606 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:58.606 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:58.606 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:58.606 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:58.606 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:58.606 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 113891 2024-09-21T15:00:58.607 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.607 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:58.607 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.607 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:58.607 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501899+0000", 2024-09-21T15:00:58.607 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:58.607 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.607 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.607 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:58.607 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501899+0000", 2024-09-21T15:00:58.607 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:58.607 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.607 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.607 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:58.607 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501898+0000", 2024-09-21T15:00:58.607 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:00:58.608 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.608 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.608 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:58.608 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501900+0000", 2024-09-21T15:00:58.608 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.356e-06 2024-09-21T15:00:58.608 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.608 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.608 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:58.608 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501902+0000", 2024-09-21T15:00:58.608 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6620000000000001e-06 2024-09-21T15:00:58.608 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.608 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.608 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:58.608 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501905+0000", 2024-09-21T15:00:58.608 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.492e-06 2024-09-21T15:00:58.608 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.609 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.609 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:58.609 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.554707+0000", 2024-09-21T15:00:58.609 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052802618000000003 2024-09-21T15:00:58.609 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.609 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.609 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:58.609 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.554798+0000", 2024-09-21T15:00:58.609 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0976000000000004e-05 2024-09-21T15:00:58.609 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.609 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.609 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:58.609 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.554890+0000", 2024-09-21T15:00:58.609 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1755999999999994e-05 2024-09-21T15:00:58.609 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:58.610 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:58.610 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:58.610 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.610 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.610 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:113892 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 5570560~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:58.610 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.501907+0000", 2024-09-21T15:00:58.610 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 29.084863245000001, 2024-09-21T15:00:58.610 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.053089242000000002, 2024-09-21T15:00:58.610 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:58.610 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:58.610 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:58.610 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:58.610 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:58.610 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 113892 2024-09-21T15:00:58.611 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.611 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:58.611 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.611 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:58.611 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501907+0000", 2024-09-21T15:00:58.611 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:58.611 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.611 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.611 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:58.611 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501907+0000", 2024-09-21T15:00:58.611 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:58.611 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.611 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.611 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:58.612 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501907+0000", 2024-09-21T15:00:58.612 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:00:58.612 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.612 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.612 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:58.612 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501908+0000", 2024-09-21T15:00:58.612 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2860000000000001e-06 2024-09-21T15:00:58.612 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.612 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.612 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:58.612 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501913+0000", 2024-09-21T15:00:58.612 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1320000000000002e-06 2024-09-21T15:00:58.612 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.612 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.612 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:58.612 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501916+0000", 2024-09-21T15:00:58.613 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5330000000000001e-06 2024-09-21T15:00:58.613 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.613 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.613 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:58.613 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.554904+0000", 2024-09-21T15:00:58.613 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052988631000000001 2024-09-21T15:00:58.613 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.613 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.613 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:58.613 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.554945+0000", 2024-09-21T15:00:58.613 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.0695999999999998e-05 2024-09-21T15:00:58.613 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.613 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.613 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:58.613 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.554996+0000", 2024-09-21T15:00:58.613 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1193999999999998e-05 2024-09-21T15:00:58.614 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:58.614 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:58.614 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:58.614 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.614 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.614 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:113893 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 5660672~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:58.614 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.501918+0000", 2024-09-21T15:00:58.614 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 29.084852267999999, 2024-09-21T15:00:58.614 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.053245595, 2024-09-21T15:00:58.614 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:58.614 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:58.614 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:58.614 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:58.614 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:58.614 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 113893 2024-09-21T15:00:58.615 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.615 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:58.615 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.615 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:58.615 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501918+0000", 2024-09-21T15:00:58.615 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:58.615 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.615 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.615 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:58.615 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501918+0000", 2024-09-21T15:00:58.615 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:58.615 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.615 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.615 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:58.616 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501918+0000", 2024-09-21T15:00:58.616 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:58.616 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.616 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.616 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:58.616 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501920+0000", 2024-09-21T15:00:58.616 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9800000000000001e-06 2024-09-21T15:00:58.616 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.616 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.616 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:58.616 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501928+0000", 2024-09-21T15:00:58.616 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7999999999999999e-06 2024-09-21T15:00:58.616 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.616 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.616 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:58.617 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501930+0000", 2024-09-21T15:00:58.617 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.785e-06 2024-09-21T15:00:58.617 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.617 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.617 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:58.617 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555019+0000", 2024-09-21T15:00:58.617 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.053089154 2024-09-21T15:00:58.617 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.617 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.617 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:58.617 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555065+0000", 2024-09-21T15:00:58.617 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5485000000000002e-05 2024-09-21T15:00:58.617 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.617 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.617 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:58.618 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555164+0000", 2024-09-21T15:00:58.618 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8629999999999996e-05 2024-09-21T15:00:58.618 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:58.618 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:58.618 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:58.618 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.618 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.618 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:113894 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 5750784~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:58.618 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.502384+0000", 2024-09-21T15:00:58.618 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 29.084386331000001, 2024-09-21T15:00:58.618 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052884821999999998, 2024-09-21T15:00:58.618 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:58.618 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:58.618 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:58.618 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:58.619 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:58.619 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 113894 2024-09-21T15:00:58.619 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.619 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:58.619 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.619 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:58.619 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502384+0000", 2024-09-21T15:00:58.619 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:58.619 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.619 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.619 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:58.619 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502384+0000", 2024-09-21T15:00:58.619 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:58.619 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.620 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.620 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:58.620 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501933+0000", 2024-09-21T15:00:58.620 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9995484 2024-09-21T15:00:58.620 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.620 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.620 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:58.620 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502390+0000", 2024-09-21T15:00:58.620 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00045714099999999999 2024-09-21T15:00:58.620 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.620 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.620 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:58.620 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502402+0000", 2024-09-21T15:00:58.620 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.236e-05 2024-09-21T15:00:58.620 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.621 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.621 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:58.621 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502411+0000", 2024-09-21T15:00:58.621 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8689999999999993e-06 2024-09-21T15:00:58.621 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.621 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.621 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:58.621 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555178+0000", 2024-09-21T15:00:58.621 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052767050000000003 2024-09-21T15:00:58.621 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.621 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.621 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:58.621 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555215+0000", 2024-09-21T15:00:58.621 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.667e-05 2024-09-21T15:00:58.621 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.622 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.622 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:58.622 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555269+0000", 2024-09-21T15:00:58.622 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4109e-05 2024-09-21T15:00:58.622 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:58.622 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:58.622 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:58.622 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.622 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.622 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:113895 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 5840896~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:58.622 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.502651+0000", 2024-09-21T15:00:58.622 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 29.084119718, 2024-09-21T15:00:58.622 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052770692000000001, 2024-09-21T15:00:58.622 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:58.623 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:58.623 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:58.623 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:58.623 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:58.623 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 113895 2024-09-21T15:00:58.623 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.623 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:58.623 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.623 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:58.623 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502651+0000", 2024-09-21T15:00:58.623 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:58.623 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.623 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.623 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:58.623 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502651+0000", 2024-09-21T15:00:58.624 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:58.624 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.624 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.624 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:58.624 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502421+0000", 2024-09-21T15:00:58.624 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9997702 2024-09-21T15:00:58.624 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.624 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.624 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:58.624 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502658+0000", 2024-09-21T15:00:58.624 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000237264 2024-09-21T15:00:58.624 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.624 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.624 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:58.624 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502673+0000", 2024-09-21T15:00:58.624 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4474999999999999e-05 2024-09-21T15:00:58.625 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.625 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.625 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:58.625 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502681+0000", 2024-09-21T15:00:58.625 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6119999999999999e-06 2024-09-21T15:00:58.625 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.625 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.625 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:58.625 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555285+0000", 2024-09-21T15:00:58.625 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052603860000000002 2024-09-21T15:00:58.625 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.625 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.625 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:58.625 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555348+0000", 2024-09-21T15:00:58.626 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3411999999999995e-05 2024-09-21T15:00:58.626 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.626 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.626 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:58.626 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555421+0000", 2024-09-21T15:00:58.626 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2809e-05 2024-09-21T15:00:58.626 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:58.626 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:58.626 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:58.626 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.626 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.626 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:113896 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 5931008~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:58.626 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.502862+0000", 2024-09-21T15:00:58.626 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 29.083908170000001, 2024-09-21T15:00:58.627 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052660208, 2024-09-21T15:00:58.627 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:58.627 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:58.627 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:58.627 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:58.627 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:58.627 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 113896 2024-09-21T15:00:58.627 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.627 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:58.627 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.627 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:58.627 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502862+0000", 2024-09-21T15:00:58.627 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:58.627 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.627 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.628 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:58.628 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502862+0000", 2024-09-21T15:00:58.628 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:58.628 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.628 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.628 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:58.628 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502689+0000", 2024-09-21T15:00:58.628 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9998264 2024-09-21T15:00:58.628 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.628 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.628 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:58.628 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502873+0000", 2024-09-21T15:00:58.628 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000184146 2024-09-21T15:00:58.628 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.628 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.628 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:58.628 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502884+0000", 2024-09-21T15:00:58.629 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1293e-05 2024-09-21T15:00:58.629 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.629 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.629 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:58.629 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502891+0000", 2024-09-21T15:00:58.629 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4259999999999997e-06 2024-09-21T15:00:58.629 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.629 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.629 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:58.629 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555435+0000", 2024-09-21T15:00:58.629 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052543053999999999 2024-09-21T15:00:58.629 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.629 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.629 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:58.629 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555471+0000", 2024-09-21T15:00:58.629 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6454e-05 2024-09-21T15:00:58.630 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.630 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.630 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:58.630 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555522+0000", 2024-09-21T15:00:58.630 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1314999999999999e-05 2024-09-21T15:00:58.630 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:58.630 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:58.630 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:58.630 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.630 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.630 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:113897 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 6021120~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:58.630 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.503004+0000", 2024-09-21T15:00:58.630 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 29.083766062999999, 2024-09-21T15:00:58.630 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052669499000000002, 2024-09-21T15:00:58.630 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:58.631 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:58.631 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:58.631 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:58.631 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:58.631 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 113897 2024-09-21T15:00:58.631 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.631 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:58.631 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.631 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:58.631 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503004+0000", 2024-09-21T15:00:58.631 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:58.631 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.631 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.631 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:58.631 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503004+0000", 2024-09-21T15:00:58.631 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:58.632 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.632 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.632 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:58.632 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502898+0000", 2024-09-21T15:00:58.632 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9998937 2024-09-21T15:00:58.632 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.632 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.632 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:58.632 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503010+0000", 2024-09-21T15:00:58.632 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000112396 2024-09-21T15:00:58.632 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.632 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.632 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:58.632 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503023+0000", 2024-09-21T15:00:58.632 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2206e-05 2024-09-21T15:00:58.632 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.633 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.633 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:58.633 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503031+0000", 2024-09-21T15:00:58.633 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3780000000000001e-06 2024-09-21T15:00:58.633 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.633 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.633 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:58.633 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555549+0000", 2024-09-21T15:00:58.633 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052517909000000002 2024-09-21T15:00:58.633 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.633 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.633 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:58.633 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555594+0000", 2024-09-21T15:00:58.633 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5385e-05 2024-09-21T15:00:58.633 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.633 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.633 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:58.634 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555674+0000", 2024-09-21T15:00:58.634 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9447e-05 2024-09-21T15:00:58.634 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:58.634 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:58.634 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:58.634 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.634 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.634 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:113898 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 6111232~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:58.634 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.503039+0000", 2024-09-21T15:00:58.634 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 29.083731277999998, 2024-09-21T15:00:58.634 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052754290000000002, 2024-09-21T15:00:58.634 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:58.634 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:58.634 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:58.634 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:58.635 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:58.635 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 113898 2024-09-21T15:00:58.635 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.635 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:58.635 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.635 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:58.635 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503039+0000", 2024-09-21T15:00:58.635 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:58.635 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.635 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.635 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:58.635 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503039+0000", 2024-09-21T15:00:58.635 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:58.635 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.635 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.636 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:58.636 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503038+0000", 2024-09-21T15:00:58.636 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:00:58.636 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.636 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.636 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:58.636 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503042+0000", 2024-09-21T15:00:58.636 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.0640000000000004e-06 2024-09-21T15:00:58.636 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.636 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.636 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:58.636 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503049+0000", 2024-09-21T15:00:58.636 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5579999999999997e-06 2024-09-21T15:00:58.636 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.636 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.636 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:58.637 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503054+0000", 2024-09-21T15:00:58.637 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6550000000000001e-06 2024-09-21T15:00:58.637 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.637 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.637 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:58.637 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555693+0000", 2024-09-21T15:00:58.637 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052638216000000002 2024-09-21T15:00:58.637 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.637 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.637 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:58.637 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555736+0000", 2024-09-21T15:00:58.637 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3619999999999999e-05 2024-09-21T15:00:58.637 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.637 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.637 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:58.637 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555793+0000", 2024-09-21T15:00:58.637 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6956999999999997e-05 2024-09-21T15:00:58.638 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:58.638 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:58.638 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:58.638 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.638 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.638 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:113899 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 6201344~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:58.638 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.503255+0000", 2024-09-21T15:00:58.638 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 29.083515004999999, 2024-09-21T15:00:58.638 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052648161999999998, 2024-09-21T15:00:58.638 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:58.638 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:58.638 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:58.638 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:58.638 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:58.639 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 113899 2024-09-21T15:00:58.639 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.639 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:58.639 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.639 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:58.639 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503255+0000", 2024-09-21T15:00:58.639 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:58.639 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.639 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.639 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:58.639 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503255+0000", 2024-09-21T15:00:58.639 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:58.639 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.639 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.639 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:58.639 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503058+0000", 2024-09-21T15:00:58.640 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9998021 2024-09-21T15:00:58.640 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.640 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.640 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:58.640 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503262+0000", 2024-09-21T15:00:58.640 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00020421299999999999 2024-09-21T15:00:58.640 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.640 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.640 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:58.640 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503274+0000", 2024-09-21T15:00:58.640 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2052000000000001e-05 2024-09-21T15:00:58.640 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.640 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.640 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:58.640 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503282+0000", 2024-09-21T15:00:58.641 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.977e-06 2024-09-21T15:00:58.641 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.641 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.641 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:58.641 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555808+0000", 2024-09-21T15:00:58.641 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052526150000000001 2024-09-21T15:00:58.641 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.641 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.641 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:58.641 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555851+0000", 2024-09-21T15:00:58.641 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2846999999999998e-05 2024-09-21T15:00:58.641 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.641 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.641 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:58.642 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555903+0000", 2024-09-21T15:00:58.642 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2607e-05 2024-09-21T15:00:58.642 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:58.642 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:58.642 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:58.642 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.642 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.642 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:113900 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 6307840~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:58.642 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.503445+0000", 2024-09-21T15:00:58.642 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 29.083325070000001, 2024-09-21T15:00:58.642 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052589546000000001, 2024-09-21T15:00:58.642 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:58.642 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:58.642 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:58.643 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:58.643 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:58.643 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 113900 2024-09-21T15:00:58.643 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.643 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:58.643 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.643 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:58.643 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503445+0000", 2024-09-21T15:00:58.643 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:58.643 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.643 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.643 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:58.643 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503445+0000", 2024-09-21T15:00:58.644 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:58.644 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.644 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.644 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:58.644 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503290+0000", 2024-09-21T15:00:58.644 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9998446 2024-09-21T15:00:58.644 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.644 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.644 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:58.644 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503451+0000", 2024-09-21T15:00:58.644 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00016111000000000001 2024-09-21T15:00:58.644 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.644 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.644 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:58.645 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503461+0000", 2024-09-21T15:00:58.645 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8740000000000007e-06 2024-09-21T15:00:58.645 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.645 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.645 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:58.645 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503468+0000", 2024-09-21T15:00:58.645 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.187e-06 2024-09-21T15:00:58.645 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.645 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.645 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:58.645 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555930+0000", 2024-09-21T15:00:58.645 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052462404999999997 2024-09-21T15:00:58.645 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.645 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.646 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:58.646 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555971+0000", 2024-09-21T15:00:58.646 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.0419e-05 2024-09-21T15:00:58.646 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.646 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.646 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:58.646 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.556035+0000", 2024-09-21T15:00:58.646 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3948999999999994e-05 2024-09-21T15:00:58.646 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:58.646 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:58.646 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:58.646 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.646 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.647 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:114050 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 4767744~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:58.647 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.514988+0000", 2024-09-21T15:00:58.647 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 29.071782558999999, 2024-09-21T15:00:58.647 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052614432000000003, 2024-09-21T15:00:58.647 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:58.647 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:58.647 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:58.647 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:58.647 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:58.647 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 114050 2024-09-21T15:00:58.647 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.647 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:58.647 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.647 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:58.648 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.514988+0000", 2024-09-21T15:00:58.648 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:58.648 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.648 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.648 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:58.648 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.514988+0000", 2024-09-21T15:00:58.648 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:58.648 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.648 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.648 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:58.648 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.514987+0000", 2024-09-21T15:00:58.648 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:58.648 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.648 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.649 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:58.649 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.514990+0000", 2024-09-21T15:00:58.649 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4739999999999999e-06 2024-09-21T15:00:58.649 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.649 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.649 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:58.649 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.514995+0000", 2024-09-21T15:00:58.649 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1939999999999997e-06 2024-09-21T15:00:58.649 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.649 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.649 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:58.649 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.514999+0000", 2024-09-21T15:00:58.649 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.245e-06 2024-09-21T15:00:58.649 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.649 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.650 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:58.650 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.567498+0000", 2024-09-21T15:00:58.650 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052498516000000002 2024-09-21T15:00:58.650 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.650 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.650 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:58.650 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.567541+0000", 2024-09-21T15:00:58.650 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3516000000000001e-05 2024-09-21T15:00:58.650 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.650 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.650 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:58.650 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.567602+0000", 2024-09-21T15:00:58.650 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0800000000000001e-05 2024-09-21T15:00:58.650 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:58.650 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:58.650 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:58.651 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.651 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.651 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:114051 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 4857856~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:58.651 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.515003+0000", 2024-09-21T15:00:58.651 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 29.071766843999999, 2024-09-21T15:00:58.651 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052709245000000002, 2024-09-21T15:00:58.651 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:58.651 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:58.651 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:58.651 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:58.651 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:58.651 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 114051 2024-09-21T15:00:58.651 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.651 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:58.651 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.652 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:58.652 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515003+0000", 2024-09-21T15:00:58.652 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:58.652 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.652 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.652 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:58.652 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515003+0000", 2024-09-21T15:00:58.652 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:58.652 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.652 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.652 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:58.652 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515003+0000", 2024-09-21T15:00:58.652 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:58.652 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.653 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.653 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:58.653 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515005+0000", 2024-09-21T15:00:58.653 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3499999999999999e-06 2024-09-21T15:00:58.653 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.653 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.653 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:58.653 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515011+0000", 2024-09-21T15:00:58.653 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2850000000000002e-06 2024-09-21T15:00:58.653 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.653 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.653 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:58.653 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515017+0000", 2024-09-21T15:00:58.653 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0059999999999998e-06 2024-09-21T15:00:58.653 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.653 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.654 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:58.654 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.567618+0000", 2024-09-21T15:00:58.654 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052600806999999999 2024-09-21T15:00:58.654 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.654 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.654 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:58.654 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.567655+0000", 2024-09-21T15:00:58.654 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7866e-05 2024-09-21T15:00:58.654 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.654 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.654 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:58.654 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.567713+0000", 2024-09-21T15:00:58.654 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7281999999999998e-05 2024-09-21T15:00:58.654 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:58.654 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:58.654 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:58.655 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.655 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.655 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:114052 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 4947968~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:58.655 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.515046+0000", 2024-09-21T15:00:58.655 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 29.071724185000001, 2024-09-21T15:00:58.655 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052795975000000002, 2024-09-21T15:00:58.655 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:58.655 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:58.655 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:58.655 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:58.655 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:58.655 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 114052 2024-09-21T15:00:58.655 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.655 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:58.655 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.656 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:58.656 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515046+0000", 2024-09-21T15:00:58.656 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:58.656 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.656 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.656 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:58.656 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515046+0000", 2024-09-21T15:00:58.656 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:58.656 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.656 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.656 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:58.656 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515046+0000", 2024-09-21T15:00:58.656 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:58.656 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.656 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.657 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:58.657 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515050+0000", 2024-09-21T15:00:58.657 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1359999999999999e-06 2024-09-21T15:00:58.657 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.657 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.657 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:58.657 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515060+0000", 2024-09-21T15:00:58.657 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0339e-05 2024-09-21T15:00:58.657 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.657 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.657 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:58.657 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515068+0000", 2024-09-21T15:00:58.657 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1149999999999994e-06 2024-09-21T15:00:58.657 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.657 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.657 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:58.658 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.567733+0000", 2024-09-21T15:00:58.658 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052664872000000001 2024-09-21T15:00:58.658 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.658 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.658 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:58.658 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.567778+0000", 2024-09-21T15:00:58.658 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4966999999999998e-05 2024-09-21T15:00:58.658 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.658 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.658 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:58.658 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.567842+0000", 2024-09-21T15:00:58.658 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4114000000000003e-05 2024-09-21T15:00:58.658 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:58.658 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:58.658 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:58.659 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.659 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.659 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:114053 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 5038080~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:58.659 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.515076+0000", 2024-09-21T15:00:58.659 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 29.071694324999999, 2024-09-21T15:00:58.659 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052886112999999998, 2024-09-21T15:00:58.659 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:58.659 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:58.659 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:58.659 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:58.659 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:58.659 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 114053 2024-09-21T15:00:58.659 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.659 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:58.659 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.659 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:58.660 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515076+0000", 2024-09-21T15:00:58.660 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:58.660 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.660 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.660 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:58.660 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515076+0000", 2024-09-21T15:00:58.660 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:58.660 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.660 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.660 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:58.660 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515075+0000", 2024-09-21T15:00:58.660 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:58.660 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.660 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.660 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:58.661 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515083+0000", 2024-09-21T15:00:58.661 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0520000000000002e-06 2024-09-21T15:00:58.661 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.661 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.661 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:58.661 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515092+0000", 2024-09-21T15:00:58.661 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1440000000000003e-06 2024-09-21T15:00:58.661 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.661 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.661 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:58.661 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515098+0000", 2024-09-21T15:00:58.661 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1090000000000003e-06 2024-09-21T15:00:58.661 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.661 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.661 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:58.661 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.567871+0000", 2024-09-21T15:00:58.661 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052772966999999997 2024-09-21T15:00:58.662 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.662 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.662 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:58.662 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.567909+0000", 2024-09-21T15:00:58.662 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.8380000000000002e-05 2024-09-21T15:00:58.662 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.662 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.662 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:58.662 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.567962+0000", 2024-09-21T15:00:58.662 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3068000000000001e-05 2024-09-21T15:00:58.662 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:58.662 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:58.662 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:58.662 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.662 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.662 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:114055 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 5218304~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:58.663 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.515526+0000", 2024-09-21T15:00:58.663 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 29.071243885000001, 2024-09-21T15:00:58.663 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052684278000000001, 2024-09-21T15:00:58.663 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:58.663 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:58.663 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:58.663 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:58.663 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:58.663 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 114055 2024-09-21T15:00:58.663 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.663 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:58.663 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.663 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:58.663 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515526+0000", 2024-09-21T15:00:58.663 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:58.663 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.664 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.664 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:58.664 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515526+0000", 2024-09-21T15:00:58.664 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:58.664 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.664 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.664 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:58.664 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515526+0000", 2024-09-21T15:00:58.664 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:58.664 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.664 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.664 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:58.664 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515529+0000", 2024-09-21T15:00:58.664 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.596e-06 2024-09-21T15:00:58.664 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.664 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.665 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:58.665 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515536+0000", 2024-09-21T15:00:58.665 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7399999999999998e-06 2024-09-21T15:00:58.665 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.665 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.665 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:58.665 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515542+0000", 2024-09-21T15:00:58.665 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5740000000000003e-06 2024-09-21T15:00:58.665 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.665 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.665 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:58.665 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.568116+0000", 2024-09-21T15:00:58.665 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052574152999999998 2024-09-21T15:00:58.665 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.665 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.666 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:58.666 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.568154+0000", 2024-09-21T15:00:58.666 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7926000000000002e-05 2024-09-21T15:00:58.666 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.666 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.666 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:58.666 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.568211+0000", 2024-09-21T15:00:58.666 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6858999999999999e-05 2024-09-21T15:00:58.666 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:58.666 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:58.666 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:58.666 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.666 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.666 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:114056 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 5308416~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:00:58.666 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.515547+0000", 2024-09-21T15:00:58.666 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 29.071222859999999, 2024-09-21T15:00:58.667 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052792795000000003, 2024-09-21T15:00:58.667 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:58.667 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:58.667 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:58.667 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:58.667 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:58.667 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 114056 2024-09-21T15:00:58.667 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.667 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:58.667 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.667 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:58.667 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515547+0000", 2024-09-21T15:00:58.667 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:58.668 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.668 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.668 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:58.668 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515547+0000", 2024-09-21T15:00:58.668 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:58.668 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.668 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.668 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:58.668 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515547+0000", 2024-09-21T15:00:58.668 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:58.668 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.668 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.668 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:58.668 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515550+0000", 2024-09-21T15:00:58.668 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2770000000000001e-06 2024-09-21T15:00:58.668 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.669 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.669 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:58.669 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515554+0000", 2024-09-21T15:00:58.669 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.087e-06 2024-09-21T15:00:58.669 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.669 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.669 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:58.669 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515558+0000", 2024-09-21T15:00:58.669 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1709999999999998e-06 2024-09-21T15:00:58.669 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.669 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.669 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:58.669 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.568234+0000", 2024-09-21T15:00:58.669 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052676675999999999 2024-09-21T15:00:58.669 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.670 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.670 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:58.670 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.568277+0000", 2024-09-21T15:00:58.670 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2400000000000001e-05 2024-09-21T15:00:58.670 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:58.670 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:58.670 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:58.670 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.568340+0000", 2024-09-21T15:00:58.670 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3581e-05 2024-09-21T15:00:58.670 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:58.670 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:58.670 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:58.670 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:58.670 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:58.671 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:58.671 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T15:00:58.681 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:00:58.677+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:00:58.773 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:58.773 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:00:58.773 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T15:00:58.773 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:58.780 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:00:58.777+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:00:58.782 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T15:00:58.882 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:00:58.879+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:00:58.890 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:58.891 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:00:58.891 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T15:00:58.891 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T15:00:58.891 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:58.900 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T15:00:58.907 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T15:00:58.982 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:00:58.981+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:00:59.052 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:00:59.052 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T15:00:59.052 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T15:00:59.052 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T15:00:59.052 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.052 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T15:00:59.052 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.205754+0000", 2024-09-21T15:00:59.052 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 175.845070975, 2024-09-21T15:00:59.052 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.41117094199999998, 2024-09-21T15:00:59.052 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:59.053 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:59.053 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:59.053 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T15:00:59.053 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T15:00:59.053 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1 2024-09-21T15:00:59.053 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.053 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:59.053 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.053 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:59.053 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205754+0000", 2024-09-21T15:00:59.053 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:59.053 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.053 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.053 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:59.053 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205754+0000", 2024-09-21T15:00:59.054 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:59.054 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.054 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.054 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:59.054 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205752+0000", 2024-09-21T15:00:59.054 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T15:00:59.054 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.054 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.054 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:59.054 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205765+0000", 2024-09-21T15:00:59.054 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3230999999999999e-05 2024-09-21T15:00:59.054 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.054 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.054 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:59.055 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205782+0000", 2024-09-21T15:00:59.055 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6634e-05 2024-09-21T15:00:59.055 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.055 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.055 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:59.055 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205793+0000", 2024-09-21T15:00:59.055 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1297e-05 2024-09-21T15:00:59.055 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.055 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.055 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:59.055 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205867+0000", 2024-09-21T15:00:59.055 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4269000000000003e-05 2024-09-21T15:00:59.055 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.055 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.055 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:00:59.056 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205877+0000", 2024-09-21T15:00:59.056 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2690000000000005e-06 2024-09-21T15:00:59.056 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.056 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.056 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:59.056 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.613700+0000", 2024-09-21T15:00:59.056 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.407823455 2024-09-21T15:00:59.056 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.056 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.056 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:59.056 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.614203+0000", 2024-09-21T15:00:59.056 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00050311500000000005 2024-09-21T15:00:59.056 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.056 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.057 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T15:00:59.057 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.614750+0000", 2024-09-21T15:00:59.057 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00054738699999999998 2024-09-21T15:00:59.057 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.057 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.057 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:59.057 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616367+0000", 2024-09-21T15:00:59.057 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001616634 2024-09-21T15:00:59.057 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.057 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.057 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:59.057 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616574+0000", 2024-09-21T15:00:59.057 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00020668499999999999 2024-09-21T15:00:59.057 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.058 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.058 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:59.058 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616802+0000", 2024-09-21T15:00:59.058 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00022817900000000001 2024-09-21T15:00:59.058 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.058 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.058 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:59.058 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616925+0000", 2024-09-21T15:00:59.058 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012309699999999999 2024-09-21T15:00:59.058 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:59.058 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:59.058 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:59.058 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.058 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.059 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:61214 3.7 3:e28aec10:::rbd_header.1136862f872f:head [watch reconnect cookie 139864671932080 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-21T15:00:59.059 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:18.147172+0000", 2024-09-21T15:00:59.059 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 100.903653332, 2024-09-21T15:00:59.059 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.019264956, 2024-09-21T15:00:59.059 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:59.059 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:59.059 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:59.059 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:59.059 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:59.059 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 61214 2024-09-21T15:00:59.059 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.059 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:59.059 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.059 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:59.060 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147172+0000", 2024-09-21T15:00:59.060 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:59.060 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.060 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.060 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:59.060 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147172+0000", 2024-09-21T15:00:59.060 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:59.060 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.060 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.060 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:59.060 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147168+0000", 2024-09-21T15:00:59.060 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999962 2024-09-21T15:00:59.060 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.060 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.061 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:59.061 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147188+0000", 2024-09-21T15:00:59.061 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9717000000000001e-05 2024-09-21T15:00:59.061 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.061 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.061 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:59.061 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147239+0000", 2024-09-21T15:00:59.061 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1493999999999999e-05 2024-09-21T15:00:59.061 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.061 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.061 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:59.061 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147265+0000", 2024-09-21T15:00:59.061 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5746999999999999e-05 2024-09-21T15:00:59.061 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.061 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.062 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:59.062 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147372+0000", 2024-09-21T15:00:59.062 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010681200000000001 2024-09-21T15:00:59.062 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.062 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.062 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op must wait for map", 2024-09-21T15:00:59.062 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147380+0000", 2024-09-21T15:00:59.062 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7919999999999993e-06 2024-09-21T15:00:59.062 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.062 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.062 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:59.062 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.154025+0000", 2024-09-21T15:00:59.062 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0066454909999999999 2024-09-21T15:00:59.062 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.063 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.063 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:00:59.063 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.154041+0000", 2024-09-21T15:00:59.063 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5894000000000001e-05 2024-09-21T15:00:59.063 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.063 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.063 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:59.063 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.164345+0000", 2024-09-21T15:00:59.063 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0103040459999999 2024-09-21T15:00:59.063 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.063 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.063 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:59.063 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.164590+0000", 2024-09-21T15:00:59.063 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00024492699999999998 2024-09-21T15:00:59.064 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.064 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.064 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:59.064 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.166437+0000", 2024-09-21T15:00:59.064 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0018466330000000001 2024-09-21T15:00:59.064 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:59.064 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:59.064 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:59.064 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.064 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.064 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:61215 3.7 3:e28aec10:::rbd_header.1136862f872f:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-21T15:00:59.064 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:18.152008+0000", 2024-09-21T15:00:59.064 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 100.89881688, 2024-09-21T15:00:59.064 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.015644958, 2024-09-21T15:00:59.064 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:59.065 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:59.065 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:59.065 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:59.065 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:59.065 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 61215 2024-09-21T15:00:59.065 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.065 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:59.065 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.065 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:59.065 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152008+0000", 2024-09-21T15:00:59.065 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:59.065 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.065 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.065 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:59.066 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152008+0000", 2024-09-21T15:00:59.066 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:59.066 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.066 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.066 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:59.066 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152005+0000", 2024-09-21T15:00:59.066 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-21T15:00:59.066 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.066 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.066 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:59.066 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152025+0000", 2024-09-21T15:00:59.066 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9306000000000001e-05 2024-09-21T15:00:59.066 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.066 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.066 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:59.067 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152054+0000", 2024-09-21T15:00:59.067 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9189000000000001e-05 2024-09-21T15:00:59.067 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.067 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.067 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:59.067 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152078+0000", 2024-09-21T15:00:59.067 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4317000000000001e-05 2024-09-21T15:00:59.067 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.067 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.067 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:59.067 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152243+0000", 2024-09-21T15:00:59.067 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000164802 2024-09-21T15:00:59.067 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.067 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.068 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting_for_map not empty", 2024-09-21T15:00:59.068 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152251+0000", 2024-09-21T15:00:59.068 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0600000000000008e-06 2024-09-21T15:00:59.068 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.068 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.068 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:59.068 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.154741+0000", 2024-09-21T15:00:59.068 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0024900069999999998 2024-09-21T15:00:59.068 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.068 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.068 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:00:59.068 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.154753+0000", 2024-09-21T15:00:59.068 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1588e-05 2024-09-21T15:00:59.068 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.068 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.068 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:59.069 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.165380+0000", 2024-09-21T15:00:59.069 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0106272249999999 2024-09-21T15:00:59.069 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.069 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.069 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:59.069 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.165517+0000", 2024-09-21T15:00:59.069 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013737999999999999 2024-09-21T15:00:59.069 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.069 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.069 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 6", 2024-09-21T15:00:59.069 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.165938+0000", 2024-09-21T15:00:59.069 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00042082399999999998 2024-09-21T15:00:59.069 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.069 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.069 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:59.069 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.166795+0000", 2024-09-21T15:00:59.070 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00085674899999999997 2024-09-21T15:00:59.070 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.070 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.070 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:59.070 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.167598+0000", 2024-09-21T15:00:59.070 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00080335599999999997 2024-09-21T15:00:59.070 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.070 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.070 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:59.070 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.167632+0000", 2024-09-21T15:00:59.070 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4122000000000003e-05 2024-09-21T15:00:59.070 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.070 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.070 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:59.070 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.167653+0000", 2024-09-21T15:00:59.071 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0862000000000001e-05 2024-09-21T15:00:59.071 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:59.071 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:59.071 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:59.071 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.071 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.071 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:97678 3.d e122/24 3:b2f34c6d:::rbd_data.1136bfc79a70.0000000000000006:head v 122'234, mlcod=120'232)", 2024-09-21T15:00:59.071 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.321655+0000", 2024-09-21T15:00:59.071 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 53.729169962999997, 2024-09-21T15:00:59.071 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024509388999999999, 2024-09-21T15:00:59.071 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:59.071 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:59.071 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:59.071 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.072 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:59.072 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.321655+0000", 2024-09-21T15:00:59.072 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:59.072 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.072 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.072 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:59.072 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.321655+0000", 2024-09-21T15:00:59.072 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:59.072 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.072 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.072 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:59.072 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.321655+0000", 2024-09-21T15:00:59.072 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:59.072 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.072 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.072 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:59.072 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.329453+0000", 2024-09-21T15:00:59.073 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0077980560000000003 2024-09-21T15:00:59.073 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.073 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.073 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:59.073 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.329482+0000", 2024-09-21T15:00:59.073 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9153999999999999e-05 2024-09-21T15:00:59.073 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.073 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.073 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:59.073 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.329493+0000", 2024-09-21T15:00:59.073 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1103e-05 2024-09-21T15:00:59.073 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.073 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.073 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:59.073 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.329555+0000", 2024-09-21T15:00:59.074 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2345e-05 2024-09-21T15:00:59.074 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.074 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.074 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:59.074 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.329589+0000", 2024-09-21T15:00:59.074 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3908999999999997e-05 2024-09-21T15:00:59.074 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.074 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.074 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:59.074 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.346083+0000", 2024-09-21T15:00:59.074 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016493383 2024-09-21T15:00:59.074 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.074 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.074 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:59.075 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.346165+0000", 2024-09-21T15:00:59.075 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1796000000000001e-05 2024-09-21T15:00:59.075 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:59.075 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:59.075 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:59.075 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.075 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.075 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:97679 3.f e122/24 3:f2a362b4:::rbd_data.1136bfc79a70.0000000000000007:head v 122'331, mlcod=120'329)", 2024-09-21T15:00:59.075 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.337145+0000", 2024-09-21T15:00:59.075 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 53.713679954, 2024-09-21T15:00:59.075 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021912411, 2024-09-21T15:00:59.075 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:59.075 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:59.075 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:59.075 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.075 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:59.076 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.337145+0000", 2024-09-21T15:00:59.076 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:59.076 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.076 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.076 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:59.076 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.337145+0000", 2024-09-21T15:00:59.076 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:59.076 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.076 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.076 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:59.076 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.337145+0000", 2024-09-21T15:00:59.076 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:59.076 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.076 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.076 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:59.076 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.343899+0000", 2024-09-21T15:00:59.077 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0067541420000000003 2024-09-21T15:00:59.077 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.077 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.077 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:59.077 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.343921+0000", 2024-09-21T15:00:59.077 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2255e-05 2024-09-21T15:00:59.077 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.077 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.077 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:59.077 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.343933+0000", 2024-09-21T15:00:59.077 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1528e-05 2024-09-21T15:00:59.077 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.077 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.078 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:59.078 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.344001+0000", 2024-09-21T15:00:59.078 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8269999999999995e-05 2024-09-21T15:00:59.078 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.078 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.078 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:59.078 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.344029+0000", 2024-09-21T15:00:59.078 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8254000000000001e-05 2024-09-21T15:00:59.078 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.078 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.078 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:59.078 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.358901+0000", 2024-09-21T15:00:59.078 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0148719 2024-09-21T15:00:59.078 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.079 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.079 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:59.079 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.359058+0000", 2024-09-21T15:00:59.079 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000156446 2024-09-21T15:00:59.079 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:59.079 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:59.079 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:59.079 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.079 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.079 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:97676 3.1f e122/24 3:ffaa836a:::rbd_data.1136bfc79a70.0000000000000004:head v 122'312, mlcod=120'310)", 2024-09-21T15:00:59.079 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.348683+0000", 2024-09-21T15:00:59.079 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 53.702141789000002, 2024-09-21T15:00:59.079 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.033768802000000001, 2024-09-21T15:00:59.079 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:59.080 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:59.080 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:59.080 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.080 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:59.080 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.348683+0000", 2024-09-21T15:00:59.080 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:59.080 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.080 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.080 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:59.080 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.348683+0000", 2024-09-21T15:00:59.080 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:59.080 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.080 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.080 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:59.080 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.348683+0000", 2024-09-21T15:00:59.080 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:00:59.081 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.081 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.081 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:59.081 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.366937+0000", 2024-09-21T15:00:59.081 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018254415 2024-09-21T15:00:59.081 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.081 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.081 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:59.081 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.366959+0000", 2024-09-21T15:00:59.081 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1634e-05 2024-09-21T15:00:59.081 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.081 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.081 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:59.081 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.366971+0000", 2024-09-21T15:00:59.081 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2305e-05 2024-09-21T15:00:59.082 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.082 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.082 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:59.082 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.367034+0000", 2024-09-21T15:00:59.082 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3211000000000002e-05 2024-09-21T15:00:59.082 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.082 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.082 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:59.082 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.367056+0000", 2024-09-21T15:00:59.082 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2098000000000001e-05 2024-09-21T15:00:59.082 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.082 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.082 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:59.082 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.382400+0000", 2024-09-21T15:00:59.083 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.015343867000000001 2024-09-21T15:00:59.083 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.083 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.083 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:59.083 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.382452+0000", 2024-09-21T15:00:59.083 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1992e-05 2024-09-21T15:00:59.083 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:59.083 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:59.083 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:59.083 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.083 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.083 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:97734 3.19 e122/94 3:9fcf754f:::rbd_data.1136bfc79a70.0000000000000009:head v 122'314, mlcod=120'312)", 2024-09-21T15:00:59.083 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.378547+0000", 2024-09-21T15:00:59.083 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 53.672277811000001, 2024-09-21T15:00:59.084 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.063380675999999997, 2024-09-21T15:00:59.084 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:59.084 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:59.084 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:59.084 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.084 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:59.084 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.378547+0000", 2024-09-21T15:00:59.084 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:59.084 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.084 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.084 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:59.084 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.378547+0000", 2024-09-21T15:00:59.084 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:59.084 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.085 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.085 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:59.085 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.378547+0000", 2024-09-21T15:00:59.085 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:00:59.085 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.085 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.085 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:59.085 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.425118+0000", 2024-09-21T15:00:59.085 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046571166999999997 2024-09-21T15:00:59.085 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.085 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.085 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:59.085 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.425150+0000", 2024-09-21T15:00:59.086 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2150999999999998e-05 2024-09-21T15:00:59.086 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.086 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.086 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:59.086 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.425164+0000", 2024-09-21T15:00:59.086 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3302000000000001e-05 2024-09-21T15:00:59.086 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.086 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.086 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:59.086 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.425231+0000", 2024-09-21T15:00:59.086 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7106000000000004e-05 2024-09-21T15:00:59.086 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.086 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.086 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:59.087 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.425259+0000", 2024-09-21T15:00:59.087 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8087000000000001e-05 2024-09-21T15:00:59.087 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.087 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.087 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:59.087 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.441890+0000", 2024-09-21T15:00:59.087 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016631328000000001 2024-09-21T15:00:59.087 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.087 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.087 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:59.087 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.441928+0000", 2024-09-21T15:00:59.087 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7701999999999999e-05 2024-09-21T15:00:59.087 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:59.087 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:59.087 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:59.088 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.089 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.090 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:100187 3.10 3:08941a97:::rbd_data.1136ab4d6ff1.000000000000000e:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-21T15:00:59.090 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:10.869845+0000", 2024-09-21T15:00:59.090 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 48.180979915000002, 2024-09-21T15:00:59.090 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029345237, 2024-09-21T15:00:59.090 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:59.090 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:59.090 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:59.090 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:59.090 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:59.090 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 100187 2024-09-21T15:00:59.090 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.091 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:59.091 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.091 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:59.091 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869845+0000", 2024-09-21T15:00:59.091 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:59.091 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.091 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.091 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:59.091 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869845+0000", 2024-09-21T15:00:59.091 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:59.091 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.091 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.091 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:59.092 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869844+0000", 2024-09-21T15:00:59.092 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T15:00:59.092 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.092 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.092 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:59.092 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869853+0000", 2024-09-21T15:00:59.092 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5719999999999992e-06 2024-09-21T15:00:59.092 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.092 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.092 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:59.092 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869874+0000", 2024-09-21T15:00:59.092 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0483e-05 2024-09-21T15:00:59.092 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.093 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.093 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:59.093 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869883+0000", 2024-09-21T15:00:59.093 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6080000000000002e-06 2024-09-21T15:00:59.093 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.093 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.093 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:59.093 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869991+0000", 2024-09-21T15:00:59.093 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000108013 2024-09-21T15:00:59.093 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.093 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.093 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:59.094 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.870258+0000", 2024-09-21T15:00:59.094 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00026629099999999999 2024-09-21T15:00:59.094 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.094 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.094 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T15:00:59.094 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.878656+0000", 2024-09-21T15:00:59.094 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0083989000000000008 2024-09-21T15:00:59.094 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.094 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.094 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:59.094 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.899081+0000", 2024-09-21T15:00:59.095 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.020424437 2024-09-21T15:00:59.095 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.095 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.095 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:59.095 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.899125+0000", 2024-09-21T15:00:59.095 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4440999999999997e-05 2024-09-21T15:00:59.095 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.095 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.095 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:59.095 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.899160+0000", 2024-09-21T15:00:59.095 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4518999999999999e-05 2024-09-21T15:00:59.095 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.095 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.096 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:59.096 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.899190+0000", 2024-09-21T15:00:59.096 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0574999999999998e-05 2024-09-21T15:00:59.096 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:59.096 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:59.096 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:59.096 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.096 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.096 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:100263 3.10 3:0ece142e:::rbd_data.1136ab4d6ff1.0000000000000008:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-21T15:00:59.096 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:11.052257+0000", 2024-09-21T15:00:59.096 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 47.998567856999998, 2024-09-21T15:00:59.096 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042678025000000001, 2024-09-21T15:00:59.096 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:59.096 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:00:59.097 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:59.097 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:59.097 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:59.097 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 100263 2024-09-21T15:00:59.097 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.097 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:59.097 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.097 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:59.097 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052257+0000", 2024-09-21T15:00:59.097 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:59.097 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.097 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.097 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:59.098 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052257+0000", 2024-09-21T15:00:59.098 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:59.098 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.098 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.098 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:59.098 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052256+0000", 2024-09-21T15:00:59.098 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:00:59.098 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.098 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.098 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:59.098 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052263+0000", 2024-09-21T15:00:59.098 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4799999999999998e-06 2024-09-21T15:00:59.098 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.099 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.099 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:59.099 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052275+0000", 2024-09-21T15:00:59.099 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2386e-05 2024-09-21T15:00:59.099 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.099 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.099 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:59.099 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052281+0000", 2024-09-21T15:00:59.099 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6699999999999999e-06 2024-09-21T15:00:59.099 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.099 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.099 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:59.099 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052350+0000", 2024-09-21T15:00:59.100 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9099999999999999e-05 2024-09-21T15:00:59.100 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.100 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.100 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:59.100 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052456+0000", 2024-09-21T15:00:59.100 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000106669 2024-09-21T15:00:59.100 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.100 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.100 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T15:00:59.100 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.059694+0000", 2024-09-21T15:00:59.100 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.007237566 2024-09-21T15:00:59.100 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.101 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.101 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:00:59.101 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.094721+0000", 2024-09-21T15:00:59.101 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035026977000000001 2024-09-21T15:00:59.101 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.101 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.101 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:00:59.101 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.094746+0000", 2024-09-21T15:00:59.101 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5329000000000001e-05 2024-09-21T15:00:59.101 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.101 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.101 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:00:59.102 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.094850+0000", 2024-09-21T15:00:59.102 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000103452 2024-09-21T15:00:59.102 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.102 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.102 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:59.102 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.094935+0000", 2024-09-21T15:00:59.102 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5550000000000006e-05 2024-09-21T15:00:59.102 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:59.102 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:59.102 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:59.102 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.103 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.103 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:134708 3.a 3:51eaadc0:::rbd_data.11365ed419e3.0000000000000007:67 [read 221184~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:00:59.103 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.938945+0000", 2024-09-21T15:00:59.103 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 5.1118805790000001, 2024-09-21T15:00:59.103 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021073784000000002, 2024-09-21T15:00:59.103 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:59.103 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:59.103 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:59.103 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:59.103 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:59.103 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 134708 2024-09-21T15:00:59.103 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.104 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:59.104 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.104 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:59.104 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938945+0000", 2024-09-21T15:00:59.104 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:59.104 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.104 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.104 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:59.104 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938945+0000", 2024-09-21T15:00:59.104 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:59.104 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.104 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.104 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:59.105 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938944+0000", 2024-09-21T15:00:59.105 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:59.105 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.105 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.105 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:59.105 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938946+0000", 2024-09-21T15:00:59.105 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.616e-06 2024-09-21T15:00:59.105 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.105 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.105 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:59.105 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938952+0000", 2024-09-21T15:00:59.105 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7309999999999998e-06 2024-09-21T15:00:59.106 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.106 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.106 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:59.106 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938957+0000", 2024-09-21T15:00:59.106 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8579999999999999e-06 2024-09-21T15:00:59.106 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.106 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.106 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:59.106 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.959730+0000", 2024-09-21T15:00:59.106 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.020773183000000001 2024-09-21T15:00:59.106 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.106 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.107 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:59.107 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.959810+0000", 2024-09-21T15:00:59.107 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0158000000000001e-05 2024-09-21T15:00:59.107 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.107 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.107 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:59.107 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.960018+0000", 2024-09-21T15:00:59.107 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000207951 2024-09-21T15:00:59.107 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:59.107 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:59.107 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:59.108 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.108 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.108 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:134709 3.a 3:51eaadc0:::rbd_data.11365ed419e3.0000000000000007:67 [read 237568~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:00:59.108 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.938996+0000", 2024-09-21T15:00:59.108 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 5.1118295869999999, 2024-09-21T15:00:59.108 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021382577, 2024-09-21T15:00:59.108 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:59.108 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:59.108 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:59.108 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:59.108 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:59.108 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 134709 2024-09-21T15:00:59.108 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.109 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:59.109 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.109 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:59.109 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938996+0000", 2024-09-21T15:00:59.109 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:59.109 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.109 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.109 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:59.109 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938996+0000", 2024-09-21T15:00:59.109 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:59.109 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.109 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.109 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:59.110 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938995+0000", 2024-09-21T15:00:59.110 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:00:59.110 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.110 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.110 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:59.110 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938999+0000", 2024-09-21T15:00:59.110 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7979999999999999e-06 2024-09-21T15:00:59.110 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.110 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.110 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:59.110 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939006+0000", 2024-09-21T15:00:59.110 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6669999999999996e-06 2024-09-21T15:00:59.111 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.111 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.111 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:59.111 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939012+0000", 2024-09-21T15:00:59.111 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9329999999999998e-06 2024-09-21T15:00:59.111 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.111 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.111 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:59.111 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.960080+0000", 2024-09-21T15:00:59.111 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021068303 2024-09-21T15:00:59.111 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.111 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.111 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:59.112 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.960154+0000", 2024-09-21T15:00:59.112 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3819000000000006e-05 2024-09-21T15:00:59.112 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.112 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.112 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:59.112 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.960378+0000", 2024-09-21T15:00:59.112 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00022385800000000001 2024-09-21T15:00:59.112 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:59.112 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:59.112 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:59.112 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.112 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.112 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:134710 3.a 3:51eaadc0:::rbd_data.11365ed419e3.0000000000000007:67 [read 253952~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:00:59.113 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.939017+0000", 2024-09-21T15:00:59.113 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 5.1118085759999996, 2024-09-21T15:00:59.113 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021721436, 2024-09-21T15:00:59.113 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:59.113 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:59.113 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:59.113 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:59.113 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:59.113 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 134710 2024-09-21T15:00:59.113 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.113 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:59.113 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.113 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:59.114 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939017+0000", 2024-09-21T15:00:59.114 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:59.114 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.114 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.114 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:59.114 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939017+0000", 2024-09-21T15:00:59.114 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:59.114 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.114 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.114 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:59.114 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939016+0000", 2024-09-21T15:00:59.114 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:59.115 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.115 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.115 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:59.115 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939019+0000", 2024-09-21T15:00:59.115 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.554e-06 2024-09-21T15:00:59.115 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.115 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.115 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:59.115 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939024+0000", 2024-09-21T15:00:59.115 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1569999999999999e-06 2024-09-21T15:00:59.115 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.115 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.115 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:59.116 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939034+0000", 2024-09-21T15:00:59.116 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9219999999999992e-06 2024-09-21T15:00:59.116 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.116 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.116 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:59.116 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.960408+0000", 2024-09-21T15:00:59.116 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021373792999999999 2024-09-21T15:00:59.116 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.116 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.116 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:59.116 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.960482+0000", 2024-09-21T15:00:59.116 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4331000000000004e-05 2024-09-21T15:00:59.117 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.117 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.117 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:59.117 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.960738+0000", 2024-09-21T15:00:59.117 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00025600500000000001 2024-09-21T15:00:59.117 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:59.117 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:59.117 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:59.117 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.117 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.117 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:134711 3.a 3:51eaadc0:::rbd_data.11365ed419e3.0000000000000007:67 [read 270336~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:00:59.117 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.939039+0000", 2024-09-21T15:00:59.117 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 5.1117865269999996, 2024-09-21T15:00:59.118 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022064698000000001, 2024-09-21T15:00:59.118 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:59.118 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:59.118 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:59.118 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:59.118 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:59.118 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 134711 2024-09-21T15:00:59.118 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.118 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:59.118 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.118 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:59.118 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939039+0000", 2024-09-21T15:00:59.119 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:59.119 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.119 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.119 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:59.119 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939039+0000", 2024-09-21T15:00:59.119 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:59.119 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.119 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.119 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:59.119 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939038+0000", 2024-09-21T15:00:59.119 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:59.119 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.120 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.120 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:59.120 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939041+0000", 2024-09-21T15:00:59.120 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9440000000000001e-06 2024-09-21T15:00:59.120 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.120 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.120 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:59.120 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939047+0000", 2024-09-21T15:00:59.120 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1260000000000003e-06 2024-09-21T15:00:59.121 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.121 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.121 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:59.121 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939052+0000", 2024-09-21T15:00:59.121 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1619999999999999e-06 2024-09-21T15:00:59.121 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.121 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.121 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:59.121 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.960768+0000", 2024-09-21T15:00:59.121 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021715140000000001 2024-09-21T15:00:59.121 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.121 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.122 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:59.122 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.960842+0000", 2024-09-21T15:00:59.122 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4698999999999997e-05 2024-09-21T15:00:59.122 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.122 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.122 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:59.122 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.961103+0000", 2024-09-21T15:00:59.122 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00026098499999999998 2024-09-21T15:00:59.122 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:59.122 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:59.122 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:59.122 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.122 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.123 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:134712 3.a 3:51eaadc0:::rbd_data.11365ed419e3.0000000000000007:67 [read 286720~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:00:59.123 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.939057+0000", 2024-09-21T15:00:59.123 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 5.111767994, 2024-09-21T15:00:59.123 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022387067, 2024-09-21T15:00:59.123 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:59.123 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:59.123 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:59.123 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:59.123 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:59.123 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 134712 2024-09-21T15:00:59.123 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.124 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:59.124 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.124 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:59.124 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939057+0000", 2024-09-21T15:00:59.124 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:59.124 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.124 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.124 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:59.124 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939057+0000", 2024-09-21T15:00:59.124 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:59.124 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.124 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.124 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:59.125 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939057+0000", 2024-09-21T15:00:59.125 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:59.125 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.125 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.125 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:59.125 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939059+0000", 2024-09-21T15:00:59.125 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.447e-06 2024-09-21T15:00:59.125 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.125 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.125 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:59.125 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939065+0000", 2024-09-21T15:00:59.125 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4670000000000003e-06 2024-09-21T15:00:59.126 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.126 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.126 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:59.126 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939069+0000", 2024-09-21T15:00:59.126 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2119999999999997e-06 2024-09-21T15:00:59.126 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.126 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.126 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:59.126 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.961141+0000", 2024-09-21T15:00:59.126 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022072268999999999 2024-09-21T15:00:59.126 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.126 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.127 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:59.127 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.961240+0000", 2024-09-21T15:00:59.127 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9147999999999993e-05 2024-09-21T15:00:59.127 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.127 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.127 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:59.127 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.961444+0000", 2024-09-21T15:00:59.127 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00020381899999999999 2024-09-21T15:00:59.127 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:59.127 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:59.127 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:59.127 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.127 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.128 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:134713 3.a 3:51eaadc0:::rbd_data.11365ed419e3.0000000000000007:67 [read 303104~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:00:59.128 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.939073+0000", 2024-09-21T15:00:59.128 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 5.1117520909999996, 2024-09-21T15:00:59.128 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022687394999999999, 2024-09-21T15:00:59.128 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:59.128 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:59.128 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:59.128 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:59.128 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:59.128 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 134713 2024-09-21T15:00:59.128 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.128 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:59.128 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.129 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:59.129 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939073+0000", 2024-09-21T15:00:59.129 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:59.129 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.129 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.129 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:59.129 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939073+0000", 2024-09-21T15:00:59.129 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:59.129 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.129 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.129 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:59.129 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939073+0000", 2024-09-21T15:00:59.129 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:59.129 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.130 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.130 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:59.130 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939075+0000", 2024-09-21T15:00:59.130 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5969999999999999e-06 2024-09-21T15:00:59.130 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.130 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.130 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:59.130 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939081+0000", 2024-09-21T15:00:59.130 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.609e-06 2024-09-21T15:00:59.130 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.130 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.130 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:59.130 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939086+0000", 2024-09-21T15:00:59.131 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.916e-06 2024-09-21T15:00:59.131 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.131 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.131 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:59.131 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.961474+0000", 2024-09-21T15:00:59.131 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022388512999999999 2024-09-21T15:00:59.131 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.131 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.131 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:59.131 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.961567+0000", 2024-09-21T15:00:59.131 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2267000000000003e-05 2024-09-21T15:00:59.131 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.131 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.132 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:59.132 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.961760+0000", 2024-09-21T15:00:59.132 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00019375999999999999 2024-09-21T15:00:59.132 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:59.132 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:59.132 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:59.132 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.132 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.132 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:134714 3.a 3:51eaadc0:::rbd_data.11365ed419e3.0000000000000007:67 [read 319488~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:00:59.132 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.939090+0000", 2024-09-21T15:00:59.132 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 5.111735264, 2024-09-21T15:00:59.132 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023043403, 2024-09-21T15:00:59.132 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:59.133 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:59.133 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:59.133 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:59.133 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:59.133 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 134714 2024-09-21T15:00:59.133 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.133 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:59.133 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.133 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:59.133 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939090+0000", 2024-09-21T15:00:59.133 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:59.133 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.133 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.134 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:59.134 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939090+0000", 2024-09-21T15:00:59.134 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:59.134 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.134 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.134 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:59.134 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939090+0000", 2024-09-21T15:00:59.134 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:59.134 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.134 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.134 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:59.134 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939092+0000", 2024-09-21T15:00:59.135 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2790000000000001e-06 2024-09-21T15:00:59.135 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.135 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.135 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:59.135 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939097+0000", 2024-09-21T15:00:59.135 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1410000000000004e-06 2024-09-21T15:00:59.135 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.135 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.135 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:59.135 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939102+0000", 2024-09-21T15:00:59.135 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7369999999999997e-06 2024-09-21T15:00:59.135 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.135 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.135 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:59.136 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.961791+0000", 2024-09-21T15:00:59.136 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022688974000000001 2024-09-21T15:00:59.136 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.136 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.136 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:59.136 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.961865+0000", 2024-09-21T15:00:59.136 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4651000000000001e-05 2024-09-21T15:00:59.136 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.136 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.136 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:59.136 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.962133+0000", 2024-09-21T15:00:59.136 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00026793999999999999 2024-09-21T15:00:59.136 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:59.137 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:59.137 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:59.137 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.137 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.137 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:134715 3.a 3:51eaadc0:::rbd_data.11365ed419e3.0000000000000007:67 [read 335872~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:00:59.137 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.939106+0000", 2024-09-21T15:00:59.137 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 5.1117186539999997, 2024-09-21T15:00:59.137 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023401142999999999, 2024-09-21T15:00:59.137 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:59.137 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:59.137 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:59.137 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:59.137 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:59.137 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 134715 2024-09-21T15:00:59.138 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.138 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:59.138 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.138 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:59.138 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939106+0000", 2024-09-21T15:00:59.138 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:59.138 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.138 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.138 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:59.138 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939106+0000", 2024-09-21T15:00:59.138 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:59.138 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.138 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.139 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:59.139 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939106+0000", 2024-09-21T15:00:59.139 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:59.139 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.139 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.139 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:59.139 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939114+0000", 2024-09-21T15:00:59.139 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1009999999999992e-06 2024-09-21T15:00:59.139 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.139 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.139 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:59.139 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939121+0000", 2024-09-21T15:00:59.139 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7710000000000001e-06 2024-09-21T15:00:59.140 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.140 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.140 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:59.140 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939126+0000", 2024-09-21T15:00:59.140 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3600000000000004e-06 2024-09-21T15:00:59.140 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.140 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.140 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:59.140 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.962162+0000", 2024-09-21T15:00:59.140 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023036148999999999 2024-09-21T15:00:59.140 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.140 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.140 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:59.141 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.962239+0000", 2024-09-21T15:00:59.141 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6874999999999998e-05 2024-09-21T15:00:59.141 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.141 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.141 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:59.141 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.962508+0000", 2024-09-21T15:00:59.141 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00026842199999999999 2024-09-21T15:00:59.141 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:59.141 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:59.141 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:59.141 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.142 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.142 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:134716 3.a 3:51eaadc0:::rbd_data.11365ed419e3.0000000000000007:67 [read 352256~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:00:59.142 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.939131+0000", 2024-09-21T15:00:59.142 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 5.1116945200000004, 2024-09-21T15:00:59.142 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023742441, 2024-09-21T15:00:59.142 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:59.142 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:00:59.142 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:59.142 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:59.142 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:59.142 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 134716 2024-09-21T15:00:59.142 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.142 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:59.143 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.143 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:59.143 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939131+0000", 2024-09-21T15:00:59.143 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:59.143 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.143 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.143 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:59.143 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939131+0000", 2024-09-21T15:00:59.143 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:59.143 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.143 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.143 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:59.143 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939130+0000", 2024-09-21T15:00:59.144 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:59.144 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.144 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.144 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:59.144 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939133+0000", 2024-09-21T15:00:59.144 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5620000000000002e-06 2024-09-21T15:00:59.144 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.144 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.144 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:59.144 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939138+0000", 2024-09-21T15:00:59.144 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.322e-06 2024-09-21T15:00:59.144 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.145 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.145 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:59.145 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939143+0000", 2024-09-21T15:00:59.145 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5510000000000001e-06 2024-09-21T15:00:59.145 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.145 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.145 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:59.145 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.962537+0000", 2024-09-21T15:00:59.145 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023394808 2024-09-21T15:00:59.145 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.145 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.145 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:00:59.146 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.962608+0000", 2024-09-21T15:00:59.146 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0699999999999997e-05 2024-09-21T15:00:59.146 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.146 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.146 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:59.146 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.962873+0000", 2024-09-21T15:00:59.146 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00026487 2024-09-21T15:00:59.146 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:59.146 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:59.146 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:59.146 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.147 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.147 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:135024 3.a 3:53261383:::rbd_data.11365ed419e3.0000000000000037:snapdir [list-snaps] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:00:59.147 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.940162+0000", 2024-09-21T15:00:59.147 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 5.1106635569999996, 2024-09-21T15:00:59.147 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022852172, 2024-09-21T15:00:59.147 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:59.147 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:59.147 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:59.147 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:59.147 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:59.147 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 135024 2024-09-21T15:00:59.148 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.148 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:59.148 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.148 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:59.148 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940162+0000", 2024-09-21T15:00:59.148 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:59.148 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.148 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.148 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:59.148 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940162+0000", 2024-09-21T15:00:59.148 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:59.148 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.149 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.149 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:59.149 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940161+0000", 2024-09-21T15:00:59.149 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:59.149 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.149 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.149 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:59.149 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940164+0000", 2024-09-21T15:00:59.149 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4099999999999998e-06 2024-09-21T15:00:59.149 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.149 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.149 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:59.150 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940169+0000", 2024-09-21T15:00:59.150 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3720000000000004e-06 2024-09-21T15:00:59.150 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.150 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.150 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:59.150 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940174+0000", 2024-09-21T15:00:59.150 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.724e-06 2024-09-21T15:00:59.150 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.150 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.150 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:59.150 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.962902+0000", 2024-09-21T15:00:59.150 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022728719000000001 2024-09-21T15:00:59.151 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.151 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.151 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:59.151 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.963014+0000", 2024-09-21T15:00:59.151 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000111245 2024-09-21T15:00:59.151 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:59.151 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:59.151 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:59.151 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.151 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.151 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:135044 3.a 3:53261383:::rbd_data.11365ed419e3.0000000000000037:snapdir [list-snaps] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:00:59.152 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.940484+0000", 2024-09-21T15:00:59.152 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 5.1103408249999998, 2024-09-21T15:00:59.152 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022619169000000001, 2024-09-21T15:00:59.152 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:00:59.152 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:00:59.152 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:00:59.152 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:00:59.152 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:00:59.152 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 135044 2024-09-21T15:00:59.152 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.152 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:00:59.152 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.153 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:00:59.153 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940484+0000", 2024-09-21T15:00:59.153 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:59.153 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.153 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.153 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:00:59.153 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940484+0000", 2024-09-21T15:00:59.153 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:00:59.153 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.153 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.153 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:00:59.154 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940484+0000", 2024-09-21T15:00:59.154 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:00:59.154 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.154 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.154 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:00:59.154 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940487+0000", 2024-09-21T15:00:59.154 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8449999999999999e-06 2024-09-21T15:00:59.154 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.154 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.154 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:00:59.154 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940493+0000", 2024-09-21T15:00:59.155 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9599999999999997e-06 2024-09-21T15:00:59.155 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.155 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.155 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:00:59.155 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940497+0000", 2024-09-21T15:00:59.155 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9620000000000001e-06 2024-09-21T15:00:59.155 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.155 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.155 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:00:59.155 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.963034+0000", 2024-09-21T15:00:59.155 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022536878 2024-09-21T15:00:59.155 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:00:59.155 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:00:59.156 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:00:59.156 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.963103+0000", 2024-09-21T15:00:59.156 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9073999999999996e-05 2024-09-21T15:00:59.156 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:59.156 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:59.156 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:59.156 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:00:59.156 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:00:59.156 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:00:59.158 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T15:00:59.158 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T15:00:59.159 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T15:00:59.159 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T15:00:59.159 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T15:00:59.159 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:00:59.155+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:00:59.164 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-21T15:00:59.193 INFO:teuthology.orchestra.run.smithi032.stderr:reweighted osd.0 to 1 (10000) 2024-09-21T15:00:59.193 INFO:teuthology.orchestra.run.smithi032.stderr:noscrub is unset 2024-09-21T15:00:59.193 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T15:00:59.210 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-21T15:00:59.234 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T15:00:59.258 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T15:00:59.258 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:00:59.253+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:00:59.260 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:00:59.260 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:00:59.260 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:00:59.260 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:00:59.260 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:00:59.260 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:00:59.261 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:00:59.261 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:00:59.289 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:59.289 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:59.300 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T15:00:59.343 INFO:teuthology.orchestra.run.smithi032.stdout:checking clone #33, image image_client.0-clone33 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent34 2024-09-21T15:00:59.359 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:00:59.355+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:00:59.460 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:00:59.455+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:00:59.485 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:59.485 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:59.501 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T15:00:59.561 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:00:59.559+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:00:59.653 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:59.653 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:59.663 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T15:00:59.663 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:00:59.657+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:00:59.668 INFO:teuthology.orchestra.run.smithi032.stdout:5330 read 0x7922e32 thru 0x792d758 (0xa927 bytes) 2024-09-21T15:00:59.669 INFO:teuthology.orchestra.run.smithi032.stdout:5331 write 0x30e57b4 thru 0x30f4274 (0xeac1 bytes) 2024-09-21T15:00:59.693 INFO:teuthology.orchestra.run.smithi032.stdout:5334 punch from 0x427c43e to 0x4288fe6, (0xcba8 bytes) 2024-09-21T15:00:59.693 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:59.693 INFO:teuthology.orchestra.run.smithi032.stdout:5336 read 0x4226b49 thru 0x4235950 (0xee08 bytes) 2024-09-21T15:00:59.694 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:59.694 INFO:teuthology.orchestra.run.smithi032.stdout:5339 write 0xb0b746c thru 0xb0c4ed1 (0xda66 bytes) 2024-09-21T15:00:59.706 INFO:teuthology.orchestra.run.smithi032.stdout:5340 writesame 0x1f99 thru 0x2721 data_size 0x1(0x789 bytes) 2024-09-21T15:00:59.709 INFO:teuthology.orchestra.run.smithi032.stdout:5343 read 0x4c356f8 thru 0x4c35f8e (0x897 bytes) 2024-09-21T15:00:59.710 INFO:teuthology.orchestra.run.smithi032.stdout:5344 read 0x400fd5c thru 0x401ec1f (0xeec4 bytes) 2024-09-21T15:00:59.710 INFO:teuthology.orchestra.run.smithi032.stdout:5345 read 0x103432c thru 0x10433c1 (0xf096 bytes) 2024-09-21T15:00:59.711 INFO:teuthology.orchestra.run.smithi032.stdout:5346 read 0xb101d51 thru 0xb10f391 (0xd641 bytes) 2024-09-21T15:00:59.711 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:59.711 INFO:teuthology.orchestra.run.smithi032.stdout:5351 write 0xb61f9f thru 0xb6510f (0x3171 bytes) 2024-09-21T15:00:59.714 INFO:teuthology.orchestra.run.smithi032.stdout:5352 punch from 0x2483835 to 0x2491bad, (0xe378 bytes) 2024-09-21T15:00:59.727 INFO:teuthology.orchestra.run.smithi032.stdout:5354 write 0xc236b25 thru 0xc23de33 (0x730f bytes) 2024-09-21T15:00:59.740 INFO:teuthology.orchestra.run.smithi032.stdout:5355 read 0xa6b3415 thru 0xa6b3cc1 (0x8ad bytes) 2024-09-21T15:00:59.740 INFO:teuthology.orchestra.run.smithi032.stdout:5357 writesame 0x83b5628 thru 0x83bbb57 data_size 0x8(0x6530 bytes) 2024-09-21T15:00:59.749 INFO:teuthology.orchestra.run.smithi032.stdout:5358 write 0x9883a15 thru 0x988f75b (0xbd47 bytes) 2024-09-21T15:00:59.761 INFO:teuthology.orchestra.run.smithi032.stdout:5361 read 0xaa165f8 thru 0xaa2287e (0xc287 bytes) 2024-09-21T15:00:59.763 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:00:59.759+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:00:59.773 INFO:teuthology.orchestra.run.smithi032.stdout:5362 trunc from 0xc23de34 to 0x515360c 2024-09-21T15:00:59.787 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:59.787 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:59.798 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T15:00:59.799 INFO:teuthology.orchestra.run.smithi032.stdout:5363 read 0x98263e thru 0x989c41 (0x7604 bytes) 2024-09-21T15:00:59.799 INFO:teuthology.orchestra.run.smithi032.stdout:5364 writesame 0x2d01ed4 thru 0x2d0da42 data_size 0x1(0xbb6f bytes) 2024-09-21T15:00:59.825 INFO:teuthology.orchestra.run.smithi032.stdout:5365 punch from 0x4d421f9 to 0x4d47860, (0x5667 bytes) 2024-09-21T15:00:59.831 INFO:teuthology.orchestra.run.smithi032.stdout:5368 trunc from 0x515360c to 0x2122bf5 2024-09-21T15:00:59.864 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:00:59.859+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:00:59.881 INFO:teuthology.orchestra.run.smithi032.stdout:5370 writesame 0xd3070f3 thru 0xd30e43a data_size 0x7c(0x7348 bytes) 2024-09-21T15:00:59.881 INFO:teuthology.orchestra.run.smithi032.stdout:5371 writesame 0x7b1c334 thru 0x7b27f2d data_size 0x1(0xbbfa bytes) 2024-09-21T15:00:59.904 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:00:59.904 INFO:teuthology.orchestra.run.smithi032.stdout:5374 punch from 0x819612a to 0x819c2d2, (0x61a8 bytes) 2024-09-21T15:00:59.904 INFO:teuthology.orchestra.run.smithi032.stdout:5375 punch from 0xd15913c to 0xd15b21d, (0x20e1 bytes) 2024-09-21T15:00:59.904 INFO:teuthology.orchestra.run.smithi032.stdout:5376 read 0x5cd78e5 thru 0x5cdd4f3 (0x5c0f bytes) 2024-09-21T15:00:59.904 INFO:teuthology.orchestra.run.smithi032.stdout:5377 punch from 0x9c2d681 to 0x9c3d4b7, (0xfe36 bytes) 2024-09-21T15:00:59.908 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:00:59.908 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:00:59.912 INFO:teuthology.orchestra.run.smithi032.stdout:5379 write 0xd41ccb8 thru 0xd42418d (0x74d6 bytes) 2024-09-21T15:00:59.918 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T15:00:59.921 INFO:teuthology.orchestra.run.smithi032.stdout:5381 read 0x4743c40 thru 0x47445dd (0x99e bytes) 2024-09-21T15:00:59.936 INFO:teuthology.orchestra.run.smithi032.stdout:5382 writesame 0xee31fa6 thru 0xee34e58 data_size 0x5(0x2eb3 bytes) 2024-09-21T15:00:59.958 INFO:teuthology.orchestra.run.smithi032.stdout:5383 trunc from 0xee34e59 to 0x2081d97 2024-09-21T15:00:59.965 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:00:59.961+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:01:00.007 INFO:teuthology.orchestra.run.smithi032.stdout:5384 writesame 0x6ef2e5a thru 0x6ef4179 data_size 0x4(0x1320 bytes) 2024-09-21T15:01:00.007 INFO:teuthology.orchestra.run.smithi032.stdout:5385 read 0x35e8d7f thru 0x35f7398 (0xe61a bytes) 2024-09-21T15:01:00.027 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:00.027 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:00.028 INFO:teuthology.orchestra.run.smithi032.stdout:5387 write 0xc464db2 thru 0xc4721c4 (0xd413 bytes) 2024-09-21T15:01:00.038 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T15:01:00.041 INFO:teuthology.orchestra.run.smithi032.stdout:5388 write 0x801f33e thru 0x802d8bd (0xe580 bytes) 2024-09-21T15:01:00.055 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:01:00.063 INFO:teuthology.orchestra.run.smithi032.stdout:5391 writesame 0xc5b560d thru 0xc5c371e data_size 0x63(0xe112 bytes) 2024-09-21T15:01:00.066 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:01:00.063+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:01:00.067 INFO:teuthology.orchestra.run.smithi032.stdout:5392 write 0x2e61820 thru 0x2e6e7b9 (0xcf9a bytes) 2024-09-21T15:01:00.079 INFO:teuthology.orchestra.run.smithi032.stdout:5393 write 0xbf28142 thru 0xbf3223d (0xa0fc bytes) 2024-09-21T15:01:00.089 INFO:teuthology.orchestra.run.smithi032.stdout:5396 read 0x66455a0 thru 0x664f1f6 (0x9c57 bytes) 2024-09-21T15:01:00.098 INFO:teuthology.orchestra.run.smithi032.stdout:5397 write 0xcb01fc1 thru 0xcb0f63a (0xd67a bytes) 2024-09-21T15:01:00.111 INFO:teuthology.orchestra.run.smithi032.stdout:5398 read 0x2d22bd0 thru 0x2d2b07a (0x84ab bytes) 2024-09-21T15:01:00.112 INFO:teuthology.orchestra.run.smithi032.stdout:5401 read 0x5a331ad thru 0x5a41c08 (0xea5c bytes) 2024-09-21T15:01:00.112 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:01:00.112 INFO:teuthology.orchestra.run.smithi032.stdout:5406 write 0x28339f3 thru 0x283d3bb (0x99c9 bytes) 2024-09-21T15:01:00.121 INFO:teuthology.orchestra.run.smithi032.stdout:5411 writesame 0x8d0e1ba thru 0x8d1e0fb data_size 0x3(0xff42 bytes) 2024-09-21T15:01:00.141 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:01:00.141 INFO:teuthology.orchestra.run.smithi032.stdout:5413 read 0x59f8a6c thru 0x5a0819b (0xf730 bytes) 2024-09-21T15:01:00.141 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:01:00.141 INFO:teuthology.orchestra.run.smithi032.stdout:5415 read 0x2580582 thru 0x25896e8 (0x9167 bytes) 2024-09-21T15:01:00.141 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:01:00.141 INFO:teuthology.orchestra.run.smithi032.stdout:5418 punch from 0x10d1ff1 to 0x10e19d6, (0xf9e5 bytes) 2024-09-21T15:01:00.141 INFO:teuthology.orchestra.run.smithi032.stdout:5419 read 0x9505edf thru 0x9507158 (0x127a bytes) 2024-09-21T15:01:00.141 INFO:teuthology.orchestra.run.smithi032.stdout:5422 read 0xc776fa7 thru 0xc7795e3 (0x263d bytes) 2024-09-21T15:01:00.142 INFO:teuthology.orchestra.run.smithi032.stdout:5423 punch from 0x94d7a6 to 0x94e863, (0x10bd bytes) 2024-09-21T15:01:00.142 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:01:00.152 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:00.152 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:00.158 INFO:teuthology.orchestra.run.smithi032.stdout:5425 write 0xe6f174d thru 0xe6f5e88 (0x473c bytes) 2024-09-21T15:01:00.160 INFO:teuthology.orchestra.run.smithi032.stderr:reweighted osd.1 to 1 (10000) 2024-09-21T15:01:00.164 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T15:01:00.166 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:01:00.163+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:01:00.173 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-21T15:01:00.177 INFO:teuthology.orchestra.run.smithi032.stdout:5426 trunc from 0xe6f5e89 to 0xc8cf3e3 2024-09-21T15:01:00.198 INFO:teuthology.orchestra.run.smithi032.stdout:5427 writesame 0x7f27bac thru 0x7f2888f data_size 0x32(0xce4 bytes) 2024-09-21T15:01:00.204 INFO:teuthology.orchestra.run.smithi032.stdout:5428 read 0x764aeb1 thru 0x764b72a (0x87a bytes) 2024-09-21T15:01:00.204 INFO:teuthology.orchestra.run.smithi032.stdout:5429 read 0x2774745 thru 0x277c264 (0x7b20 bytes) 2024-09-21T15:01:00.204 INFO:teuthology.orchestra.run.smithi032.stdout:5430 read 0xf248f2 thru 0xf25f71 (0x1680 bytes) 2024-09-21T15:01:00.213 INFO:teuthology.orchestra.run.smithi032.stdout:5431 trunc from 0xc8cf3e3 to 0x7cb5381 2024-09-21T15:01:00.242 INFO:teuthology.orchestra.run.smithi032.stdout:5432 trunc from 0x7cb5381 to 0x20bf5e6 2024-09-21T15:01:00.267 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:01:00.263+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:01:00.270 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:00.270 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:00.281 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T15:01:00.290 INFO:teuthology.orchestra.run.smithi032.stdout:5435 write 0xaf72bcd thru 0xaf7c3e5 (0x9819 bytes) 2024-09-21T15:01:00.298 INFO:teuthology.orchestra.run.smithi032.stdout:5436 write 0x81c0bdf thru 0x81c20ad (0x14cf bytes) 2024-09-21T15:01:00.303 INFO:teuthology.orchestra.run.smithi032.stdout:5437 write 0x1a5b8ae thru 0x1a682da (0xca2d bytes) 2024-09-21T15:01:00.317 INFO:teuthology.orchestra.run.smithi032.stdout:5438 trunc from 0xaf7c3e6 to 0xb955b8e 2024-09-21T15:01:00.326 INFO:teuthology.orchestra.run.smithi032.stdout:5439 read 0x130b86e thru 0x130c598 (0xd2b bytes) 2024-09-21T15:01:00.326 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:01:00.326 INFO:teuthology.orchestra.run.smithi032.stdout:5442 write 0xab79108 thru 0xab7e3d1 (0x52ca bytes) 2024-09-21T15:01:00.334 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:01:00.351 INFO:teuthology.orchestra.run.smithi032.stdout:5444 trunc from 0xb955b8e to 0x196467d 2024-09-21T15:01:00.368 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:01:00.365+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:01:00.397 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:00.397 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:00.400 INFO:teuthology.orchestra.run.smithi032.stdout:5445 write 0x8de0794 thru 0x8deefe8 (0xe855 bytes) 2024-09-21T15:01:00.408 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T15:01:00.414 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:01:00.414 INFO:teuthology.orchestra.run.smithi032.stdout:5447 write 0x629b7d1 thru 0x62a9ef3 (0xe723 bytes) 2024-09-21T15:01:00.426 INFO:teuthology.orchestra.run.smithi032.stdout:5448 read 0x52ab258 thru 0x52ac935 (0x16de bytes) 2024-09-21T15:01:00.426 INFO:teuthology.orchestra.run.smithi032.stdout:5449 punch from 0x346a11e to 0x347535d, (0xb23f bytes) 2024-09-21T15:01:00.436 INFO:teuthology.orchestra.run.smithi032.stdout:5450 write 0x90888e0 thru 0x9088c38 (0x359 bytes) 2024-09-21T15:01:00.442 INFO:teuthology.orchestra.run.smithi032.stdout:5452 read 0x91cb8a thru 0x921236 (0x46ad bytes) 2024-09-21T15:01:00.443 INFO:teuthology.orchestra.run.smithi032.stdout:5455 punch from 0x8aff285 to 0x8b07c94, (0x8a0f bytes) 2024-09-21T15:01:00.443 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:01:00.443 INFO:teuthology.orchestra.run.smithi032.stdout:5457 write 0x154efc5 thru 0x155a596 (0xb5d2 bytes) 2024-09-21T15:01:00.454 INFO:teuthology.orchestra.run.smithi032.stdout:5458 punch from 0x804c1b6 to 0x8056851, (0xa69b bytes) 2024-09-21T15:01:00.454 INFO:teuthology.orchestra.run.smithi032.stdout:5459 read 0x57753c3 thru 0x5779821 (0x445f bytes) 2024-09-21T15:01:00.454 INFO:teuthology.orchestra.run.smithi032.stdout:5460 read 0x1aa7b08 thru 0x1aad435 (0x592e bytes) 2024-09-21T15:01:00.469 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:01:00.463+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:01:00.474 INFO:teuthology.orchestra.run.smithi032.stdout:5463 writesame 0xdf8d7d4 thru 0xdf9d3a3 data_size 0xcc(0xfbd0 bytes) 2024-09-21T15:01:00.474 INFO:teuthology.orchestra.run.smithi032.stdout:5464 write 0x8b42360 thru 0x8b487c2 (0x6463 bytes) 2024-09-21T15:01:00.494 INFO:teuthology.orchestra.run.smithi032.stdout:5465 trunc from 0xdf9d3a4 to 0x96a618d 2024-09-21T15:01:00.505 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:01:00.505 INFO:teuthology.orchestra.run.smithi032.stdout:5468 read 0x7960b2b thru 0x796e5f2 (0xdac8 bytes) 2024-09-21T15:01:00.507 INFO:teuthology.orchestra.run.smithi032.stdout:5469 trunc from 0x96a618d to 0x7d3903e 2024-09-21T15:01:00.515 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:00.515 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:00.526 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T15:01:00.533 INFO:teuthology.orchestra.run.smithi032.stdout:5470 trunc from 0x7d3903e to 0xa55e40c 2024-09-21T15:01:00.546 INFO:teuthology.orchestra.run.smithi032.stdout:5471 write 0x3bcb299 thru 0x3bd4b6d (0x98d5 bytes) 2024-09-21T15:01:00.558 INFO:teuthology.orchestra.run.smithi032.stdout:5472 read 0x50ce65e thru 0x50dbcbe (0xd661 bytes) 2024-09-21T15:01:00.570 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:01:00.567+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:01:00.580 INFO:teuthology.orchestra.run.smithi032.stdout:5476 writesame 0xeadd363 thru 0xeaea7c1 data_size 0x1(0xd45f bytes) 2024-09-21T15:01:00.580 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:01:00.580 INFO:teuthology.orchestra.run.smithi032.stdout:5478 writesame 0x1ac5f27 thru 0x1ac856d data_size 0x1(0x2647 bytes) 2024-09-21T15:01:00.593 INFO:teuthology.orchestra.run.smithi032.stdout:5480 punch from 0xd0ecf4a to 0xd0f5f2f, (0x8fe5 bytes) 2024-09-21T15:01:00.593 INFO:teuthology.orchestra.run.smithi032.stdout:5481 punch from 0x94f2d4b to 0x94fff5b, (0xd210 bytes) 2024-09-21T15:01:00.593 INFO:teuthology.orchestra.run.smithi032.stdout:5482 punch from 0xad30d87 to 0xad32722, (0x199b bytes) 2024-09-21T15:01:00.593 INFO:teuthology.orchestra.run.smithi032.stdout:5483 write 0xcba1582 thru 0xcba18c9 (0x348 bytes) 2024-09-21T15:01:00.600 INFO:teuthology.orchestra.run.smithi032.stdout:5484 read 0x25aaed7 thru 0x25ac280 (0x13aa bytes) 2024-09-21T15:01:00.623 INFO:teuthology.orchestra.run.smithi032.stdout:5487 trunc from 0xeaea7c2 to 0x37cf4e6 2024-09-21T15:01:00.635 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:00.635 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:00.646 INFO:teuthology.orchestra.run.smithi032.stdout:5488 writesame 0x29b68c5 thru 0x29c4908 data_size 0x3e(0xe044 bytes) 2024-09-21T15:01:00.647 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T15:01:00.691 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:01:00.689+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:01:00.692 INFO:teuthology.orchestra.run.smithi032.stdout:5489 writesame 0xec88651 thru 0xec8b8fa data_size 0x5(0x32aa bytes) 2024-09-21T15:01:00.693 INFO:teuthology.orchestra.run.smithi032.stdout:5490 read 0x21d221a thru 0x21d5f06 (0x3ced bytes) 2024-09-21T15:01:00.693 INFO:teuthology.orchestra.run.smithi032.stdout:5491 write 0xd656b68 thru 0xd662128 (0xb5c1 bytes) 2024-09-21T15:01:00.707 INFO:teuthology.orchestra.run.smithi032.stdout:5492 read 0xe1fa4a8 thru 0xe206d61 (0xc8ba bytes) 2024-09-21T15:01:00.707 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:01:00.707 INFO:teuthology.orchestra.run.smithi032.stdout:5494 punch from 0xbe09152 to 0xbe0d9f0, (0x489e bytes) 2024-09-21T15:01:00.707 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:01:00.707 INFO:teuthology.orchestra.run.smithi032.stdout:5496 clone 36 order 23 su 32768 sc 14 2024-09-21T15:01:00.747 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:00.747 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:00.757 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T15:01:00.772 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:01:00.771+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:01:00.863 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:00.863 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:00.873 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:01:00.869+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:01:00.873 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T15:01:00.973 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:01:00.971+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:01:00.986 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:00.986 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:00.995 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T15:01:01.074 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:01:01.069+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:01:01.107 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:01.107 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:01.116 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T15:01:01.174 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:01:01.173+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:01:01.204 INFO:teuthology.orchestra.run.smithi032.stderr:reweighted osd.2 to 1 (10000) 2024-09-21T15:01:01.207 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T15:01:01.218 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-21T15:01:01.273 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T15:01:01.275 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:01:01.271+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:01:01.282 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:01.282 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:01.292 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T15:01:01.375 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:01:01.371+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:01:01.403 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T15:01:01.403 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:01.404 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:01.414 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T15:01:01.423 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T15:01:01.447 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T15:01:01.465 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T15:01:01.476 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:01:01.473+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:01:01.488 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-21T15:01:01.510 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T15:01:01.522 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:01.522 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:01.532 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T15:01:01.534 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T15:01:01.559 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T15:01:01.561 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:01:01.561 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:01:01.561 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:01:01.561 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:01:01.561 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:01:01.562 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:01:01.562 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:01:01.562 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:01:01.621 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:01:01.619+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:01:01.641 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:01.641 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:01:01.641 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T15:01:01.642 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:01.651 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T15:01:01.679 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:01:01.677+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:01:01.762 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:01.762 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:01:01.762 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T15:01:01.762 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T15:01:01.762 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:01.772 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T15:01:01.779 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:01:01.777+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:01:01.879 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:01.879 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T15:01:01.879 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T15:01:01.879 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T15:01:01.879 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.879 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:97677 3.15 3:a8e72133:::rbd_data.1136bfc79a70.0000000000000005:head [write 0~4440027 [fadvise_sequential+fadvise_nocache],write 4751360~147456 [fadvise_sequential+fadvise_nocache],write 4964352~147456 [fadvise_sequential+fadvise_nocache],write 5177344~65536 [fadvise_sequential+fadvise_nocache],write 5259264~147456 [fadvise_sequential+fadvise_nocache],write 5472256~147456 [fadvise_sequential+fadvise_nocache],write 5685248~229376 [fadvise_sequential+fadvise_nocache],write 5980160~147456 [fadvise_sequential+fadvise_nocache],write 6193152~229376 [fadvise_sequential+fadvise_nocache],write 6488064~147456 [fadvise_sequential+fadvise_nocache],write 6701056~229376 [fadvise_sequential+fadvise_nocache],write 6995968~147456 [fadvise_sequential+fadvise_nocache],write 7208960~229376 [fadvise_sequential+fadvise_nocache],write 7503872~147456 [fadvise_sequential+fadvise_nocache],write 7716864~229376 [fadvise_sequential+fadvise_nocache],write 8044544~114688 [fadvise_sequential+fadvise_nocache],write 8257536~114688 [fadvise_sequential+fadvise_nocache],write 8388608~65536 [fadvise_sequential+fadvise_nocache],write 8552448~114688 [fadvise_sequential+fadvise_nocache],write 8765440~114688 [fadvise_sequential+fadvise_nocache],write 8929280~32768 [fadvise_sequential+fadvise_nocache],write 9142272~32768 [fadvise_sequential+fadvise_nocache],write 9355264~32768 [fadvise_sequential+fadvise_nocache],write 9437184~32768 [fadvise_sequential+fadvise_nocache],write 9650176~32768 [fadvise_sequential+fadvise_nocache],write 9863168~32768 [fadvise_sequential+fadvise_nocache],write 9961472~131072 [fadvise_sequential+fadvise_nocache],write 10108928~196608 [fadvise_sequential+fadvise_nocache],write 10321920~278528 [fadvise_sequential+fadvise_nocache],write 10616832~196608 [fadvise_sequential+fadvise_nocache],write 10829824~278528 [fadvise_sequential+fadvise_nocache],write 11124736~196608 [fadvise_sequential+fadvise_nocache],write 11337728~196608 [fadvise_sequential+fadvise_nocache],write 11550720~65536 [fadvise_sequential+fadvise_nocache],write 11649024~98304 [fadvise_sequential+fadvise_nocache],write 11763712~65536 [fadvise_sequential+fadvise_nocache],write 11862016~98304 [fadvise_sequential+fadvise_nocache],write 11976704~65536 [fadvise_sequential+fadvise_nocache],write 12058624~65536 [fadvise_sequential+fadvise_nocache],write 12156928~98304 [fadvise_sequential+fadvise_nocache],write 12271616~65536 [fadvise_sequential+fadvise_nocache],write 12369920~98304 [fadvise_sequential+fadvise_nocache],write 12484608~65536 [fadvise_sequential+fadvise_nocache],write 12664832~98304 [fadvise_sequential+fadvise_nocache],write 12877824~98304 [fadvise_sequential+fadvise_nocache],write 13090816~16384 [fadvise_sequential+fadvise_nocache],write 13172736~98304 [fadvise_sequential+fadvise_nocache],write 13385728~98304 [fadvise_sequential+fadvise_nocache],write 13598720~32768 [fadvise_sequential+fadvise_nocache],write 13680640~98304 [fadvise_sequential+fadvise_nocache],write 13893632~98304 [fadvise_sequential+fadvise_nocache],write 14106624~49152 [fadvise_sequential+fadvise_nocache],write 14794752~81920 [fadvise_sequential+fadvise_nocache],write 15007744~81920 [fadvise_sequential+fadvise_nocache],write 15302656~81920 [fadvise_sequential+fadvise_nocache],write 15515648~81920 [fadvise_sequential+fadvise_nocache],write 15810560~81920 [fadvise_sequential+fadvise_nocache],write 16023552~81920 [fadvise_sequential+fadvise_nocache],write 16236544~16384 [fadvise_sequential+fadvise_nocache],write 16318464~81920 [fadvise_sequential+fadvise_nocache],write 16531456~81920 [fadvise_sequential+fadvise_nocache],write 16744448~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:01:01.879 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.302946+0000", 2024-09-21T15:01:01.880 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 56.573986298000001, 2024-09-21T15:01:01.880 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.091231204999999996, 2024-09-21T15:01:01.880 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:01.880 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:01.880 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:01.880 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:01.880 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:01.880 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 97677 2024-09-21T15:01:01.880 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.880 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:01.880 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.880 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:01.880 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.302946+0000", 2024-09-21T15:01:01.881 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:01.881 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.881 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.881 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:01.881 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.302946+0000", 2024-09-21T15:01:01.881 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:01.881 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.881 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.881 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:01.881 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.302942+0000", 2024-09-21T15:01:01.881 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999962 2024-09-21T15:01:01.881 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.881 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.881 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:01.882 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311253+0000", 2024-09-21T15:01:01.882 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0083105360000000003 2024-09-21T15:01:01.882 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.882 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.882 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:01.882 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311283+0000", 2024-09-21T15:01:01.882 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0065000000000001e-05 2024-09-21T15:01:01.882 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.882 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.882 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:01.882 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311296+0000", 2024-09-21T15:01:01.882 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3206e-05 2024-09-21T15:01:01.882 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.882 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.883 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:01.883 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311404+0000", 2024-09-21T15:01:01.883 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000107514 2024-09-21T15:01:01.883 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.883 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.883 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:01.883 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311795+0000", 2024-09-21T15:01:01.883 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000390913 2024-09-21T15:01:01.883 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.883 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.883 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 6", 2024-09-21T15:01:01.883 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.314686+0000", 2024-09-21T15:01:01.883 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0028915659999999999 2024-09-21T15:01:01.883 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.883 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.884 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:01:01.884 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.368178+0000", 2024-09-21T15:01:01.884 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.053491706 2024-09-21T15:01:01.884 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.884 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.884 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:01:01.884 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.393925+0000", 2024-09-21T15:01:01.884 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025747289 2024-09-21T15:01:01.884 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.884 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.884 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:01:01.884 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.394127+0000", 2024-09-21T15:01:01.884 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000201444 2024-09-21T15:01:01.884 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.885 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.885 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:01.885 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.394177+0000", 2024-09-21T15:01:01.885 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0689000000000002e-05 2024-09-21T15:01:01.885 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:01.885 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:01.885 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:01.885 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.885 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.885 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:97673 3.1d e122/24 3:bd91946e:::rbd_data.1136bfc79a70.0000000000000002:head v 122'162, mlcod=120'160)", 2024-09-21T15:01:01.885 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.313730+0000", 2024-09-21T15:01:01.885 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 56.563201972999998, 2024-09-21T15:01:01.885 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055373462999999998, 2024-09-21T15:01:01.885 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:01.886 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:01.886 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:01.886 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.886 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:01.886 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.313730+0000", 2024-09-21T15:01:01.886 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:01.886 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.886 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.886 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:01.886 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.313730+0000", 2024-09-21T15:01:01.886 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:01.886 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.886 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.886 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:01.887 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.313730+0000", 2024-09-21T15:01:01.887 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:01.887 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.887 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.887 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:01.887 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.336113+0000", 2024-09-21T15:01:01.887 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022382962999999999 2024-09-21T15:01:01.887 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.887 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.887 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:01.887 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.336140+0000", 2024-09-21T15:01:01.887 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7248999999999999e-05 2024-09-21T15:01:01.887 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.887 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.887 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:01.888 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.336151+0000", 2024-09-21T15:01:01.888 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1015e-05 2024-09-21T15:01:01.888 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.888 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.888 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:01.888 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.355428+0000", 2024-09-21T15:01:01.888 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019276379999999999 2024-09-21T15:01:01.888 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.888 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.888 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:01.888 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.355484+0000", 2024-09-21T15:01:01.888 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6820999999999997e-05 2024-09-21T15:01:01.888 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.889 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.889 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:01:01.889 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.369040+0000", 2024-09-21T15:01:01.889 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013555466 2024-09-21T15:01:01.889 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.889 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.889 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:01.889 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.369104+0000", 2024-09-21T15:01:01.889 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3986999999999996e-05 2024-09-21T15:01:01.889 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:01.889 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:01.889 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:01.889 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.889 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.889 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:113889 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 5300224~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:01.890 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.501880+0000", 2024-09-21T15:01:01.890 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 32.375052060000002, 2024-09-21T15:01:01.890 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052644204999999999, 2024-09-21T15:01:01.890 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:01.890 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:01.890 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:01.890 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:01.890 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:01.890 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 113889 2024-09-21T15:01:01.890 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.890 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:01.890 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.890 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:01.891 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501880+0000", 2024-09-21T15:01:01.891 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:01.891 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.891 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.891 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:01.891 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501880+0000", 2024-09-21T15:01:01.891 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:01.891 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.891 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.891 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:01.891 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501880+0000", 2024-09-21T15:01:01.891 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:01:01.891 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.891 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.891 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:01.892 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501882+0000", 2024-09-21T15:01:01.892 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.364e-06 2024-09-21T15:01:01.892 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.892 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.892 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:01.892 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501885+0000", 2024-09-21T15:01:01.892 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4199999999999999e-06 2024-09-21T15:01:01.892 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.892 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.892 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:01.892 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501888+0000", 2024-09-21T15:01:01.892 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8669999999999998e-06 2024-09-21T15:01:01.892 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.892 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.893 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:01.893 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.554369+0000", 2024-09-21T15:01:01.893 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052481456000000003 2024-09-21T15:01:01.893 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.893 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.893 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:01.893 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.554449+0000", 2024-09-21T15:01:01.893 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9412000000000005e-05 2024-09-21T15:01:01.893 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.893 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.893 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:01.893 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.554525+0000", 2024-09-21T15:01:01.893 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5908999999999994e-05 2024-09-21T15:01:01.893 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:01.894 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:01.894 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:01.894 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.894 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.894 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:113890 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 5390336~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:01.894 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.501890+0000", 2024-09-21T15:01:01.894 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 32.375042344999997, 2024-09-21T15:01:01.894 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052785212999999997, 2024-09-21T15:01:01.894 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:01.894 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:01.894 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:01.894 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:01.894 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:01.894 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 113890 2024-09-21T15:01:01.894 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.895 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:01.895 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.895 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:01.895 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501890+0000", 2024-09-21T15:01:01.895 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:01.895 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.895 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.895 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:01.895 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501890+0000", 2024-09-21T15:01:01.895 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:01.895 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.895 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.895 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:01.895 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501890+0000", 2024-09-21T15:01:01.895 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:01:01.896 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.896 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.896 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:01.896 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501891+0000", 2024-09-21T15:01:01.896 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3599999999999999e-06 2024-09-21T15:01:01.896 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.896 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.896 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:01.896 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501894+0000", 2024-09-21T15:01:01.896 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.717e-06 2024-09-21T15:01:01.896 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.896 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.896 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:01.896 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501897+0000", 2024-09-21T15:01:01.896 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5790000000000002e-06 2024-09-21T15:01:01.897 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.897 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.897 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:01.897 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.554569+0000", 2024-09-21T15:01:01.897 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052672434999999997 2024-09-21T15:01:01.897 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.897 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.897 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:01.897 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.554623+0000", 2024-09-21T15:01:01.897 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4509999999999998e-05 2024-09-21T15:01:01.897 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.897 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.897 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:01.897 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.554675+0000", 2024-09-21T15:01:01.898 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1833999999999999e-05 2024-09-21T15:01:01.898 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:01.898 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:01.898 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:01.898 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.898 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.898 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:113891 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 5480448~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:01.898 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.501899+0000", 2024-09-21T15:01:01.898 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 32.375033858999998, 2024-09-21T15:01:01.898 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052991637000000001, 2024-09-21T15:01:01.898 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:01.898 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:01.898 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:01.898 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:01.899 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:01.899 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 113891 2024-09-21T15:01:01.899 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.899 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:01.899 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.899 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:01.899 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501899+0000", 2024-09-21T15:01:01.899 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:01.899 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.899 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.899 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:01.899 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501899+0000", 2024-09-21T15:01:01.899 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:01.899 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.899 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.900 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:01.900 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501898+0000", 2024-09-21T15:01:01.900 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:01:01.900 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.900 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.900 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:01.900 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501900+0000", 2024-09-21T15:01:01.900 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.356e-06 2024-09-21T15:01:01.900 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.900 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.900 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:01.900 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501902+0000", 2024-09-21T15:01:01.900 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6620000000000001e-06 2024-09-21T15:01:01.900 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.901 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.901 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:01.901 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501905+0000", 2024-09-21T15:01:01.901 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.492e-06 2024-09-21T15:01:01.901 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.901 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.901 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:01.901 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.554707+0000", 2024-09-21T15:01:01.901 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052802618000000003 2024-09-21T15:01:01.901 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.901 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.901 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:01.901 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.554798+0000", 2024-09-21T15:01:01.901 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0976000000000004e-05 2024-09-21T15:01:01.901 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.902 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.902 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:01.902 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.554890+0000", 2024-09-21T15:01:01.902 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1755999999999994e-05 2024-09-21T15:01:01.902 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:01.902 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:01.902 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:01.902 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.902 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.902 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:113892 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 5570560~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:01.902 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.501907+0000", 2024-09-21T15:01:01.902 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 32.375025426999997, 2024-09-21T15:01:01.902 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.053089242000000002, 2024-09-21T15:01:01.902 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:01.903 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:01.903 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:01.903 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:01.903 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:01.903 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 113892 2024-09-21T15:01:01.903 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.903 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:01.903 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.903 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:01.903 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501907+0000", 2024-09-21T15:01:01.903 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:01.903 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.903 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.903 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:01.903 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501907+0000", 2024-09-21T15:01:01.904 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:01.904 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.904 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.904 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:01.904 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501907+0000", 2024-09-21T15:01:01.904 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:01:01.904 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.904 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.904 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:01.904 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501908+0000", 2024-09-21T15:01:01.904 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2860000000000001e-06 2024-09-21T15:01:01.904 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.904 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.904 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:01.904 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501913+0000", 2024-09-21T15:01:01.904 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1320000000000002e-06 2024-09-21T15:01:01.905 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.905 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.905 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:01.905 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501916+0000", 2024-09-21T15:01:01.905 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5330000000000001e-06 2024-09-21T15:01:01.905 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.905 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.905 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:01.905 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.554904+0000", 2024-09-21T15:01:01.905 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052988631000000001 2024-09-21T15:01:01.905 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.905 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.905 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:01.905 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.554945+0000", 2024-09-21T15:01:01.905 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.0695999999999998e-05 2024-09-21T15:01:01.906 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.906 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.906 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:01.906 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.554996+0000", 2024-09-21T15:01:01.906 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1193999999999998e-05 2024-09-21T15:01:01.906 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:01.906 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:01.906 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:01.906 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.906 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.906 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:113893 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 5660672~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:01.906 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.501918+0000", 2024-09-21T15:01:01.906 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 32.375014450000002, 2024-09-21T15:01:01.906 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.053245595, 2024-09-21T15:01:01.906 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:01.907 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:01.907 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:01.907 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:01.907 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:01.907 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 113893 2024-09-21T15:01:01.907 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.907 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:01.907 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.907 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:01.907 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501918+0000", 2024-09-21T15:01:01.907 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:01.907 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.907 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.907 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:01.907 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501918+0000", 2024-09-21T15:01:01.907 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:01.908 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.908 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.908 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:01.908 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501918+0000", 2024-09-21T15:01:01.908 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:01.908 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.908 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.908 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:01.908 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501920+0000", 2024-09-21T15:01:01.908 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9800000000000001e-06 2024-09-21T15:01:01.908 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.908 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.908 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:01.908 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501928+0000", 2024-09-21T15:01:01.908 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7999999999999999e-06 2024-09-21T15:01:01.909 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.909 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.909 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:01.909 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501930+0000", 2024-09-21T15:01:01.909 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.785e-06 2024-09-21T15:01:01.909 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.909 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.909 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:01.909 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555019+0000", 2024-09-21T15:01:01.909 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.053089154 2024-09-21T15:01:01.909 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.909 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.909 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:01.909 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555065+0000", 2024-09-21T15:01:01.909 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5485000000000002e-05 2024-09-21T15:01:01.910 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.910 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.910 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:01.910 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555164+0000", 2024-09-21T15:01:01.910 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8629999999999996e-05 2024-09-21T15:01:01.910 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:01.910 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:01.910 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:01.910 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.910 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.910 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:113894 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 5750784~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:01.910 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.502384+0000", 2024-09-21T15:01:01.910 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 32.374548513000001, 2024-09-21T15:01:01.910 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052884821999999998, 2024-09-21T15:01:01.910 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:01.910 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:01.911 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:01.911 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:01.911 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:01.911 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 113894 2024-09-21T15:01:01.911 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.911 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:01.911 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.911 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:01.911 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502384+0000", 2024-09-21T15:01:01.911 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:01.911 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.911 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.911 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:01.911 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502384+0000", 2024-09-21T15:01:01.911 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:01.912 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.912 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.912 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:01.912 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.501933+0000", 2024-09-21T15:01:01.912 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9995484 2024-09-21T15:01:01.912 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.912 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.912 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:01.912 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502390+0000", 2024-09-21T15:01:01.912 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00045714099999999999 2024-09-21T15:01:01.912 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.912 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.912 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:01.912 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502402+0000", 2024-09-21T15:01:01.912 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.236e-05 2024-09-21T15:01:01.913 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.913 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.913 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:01.913 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502411+0000", 2024-09-21T15:01:01.913 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8689999999999993e-06 2024-09-21T15:01:01.913 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.913 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.913 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:01.913 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555178+0000", 2024-09-21T15:01:01.913 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052767050000000003 2024-09-21T15:01:01.913 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.913 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.913 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:01.913 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555215+0000", 2024-09-21T15:01:01.913 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.667e-05 2024-09-21T15:01:01.913 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.914 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.914 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:01.914 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555269+0000", 2024-09-21T15:01:01.914 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4109e-05 2024-09-21T15:01:01.914 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:01.914 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:01.914 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:01.914 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.914 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.914 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:113895 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 5840896~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:01.914 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.502651+0000", 2024-09-21T15:01:01.914 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 32.3742819, 2024-09-21T15:01:01.914 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052770692000000001, 2024-09-21T15:01:01.914 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:01.915 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:01.915 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:01.915 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:01.915 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:01.915 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 113895 2024-09-21T15:01:01.915 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.915 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:01.915 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.915 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:01.915 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502651+0000", 2024-09-21T15:01:01.915 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:01.915 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.915 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.915 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:01.916 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502651+0000", 2024-09-21T15:01:01.916 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:01.916 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.916 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.916 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:01.916 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502421+0000", 2024-09-21T15:01:01.916 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9997702 2024-09-21T15:01:01.916 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.916 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.916 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:01.916 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502658+0000", 2024-09-21T15:01:01.916 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000237264 2024-09-21T15:01:01.916 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.916 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.916 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:01.916 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502673+0000", 2024-09-21T15:01:01.917 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4474999999999999e-05 2024-09-21T15:01:01.917 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.917 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.917 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:01.917 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502681+0000", 2024-09-21T15:01:01.917 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6119999999999999e-06 2024-09-21T15:01:01.917 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.917 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.917 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:01.917 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555285+0000", 2024-09-21T15:01:01.917 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052603860000000002 2024-09-21T15:01:01.917 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.917 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.917 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:01.917 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555348+0000", 2024-09-21T15:01:01.918 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3411999999999995e-05 2024-09-21T15:01:01.918 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.918 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.918 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:01.918 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555421+0000", 2024-09-21T15:01:01.918 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2809e-05 2024-09-21T15:01:01.918 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:01.918 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:01.918 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:01.918 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.918 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.918 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:113896 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 5931008~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:01.918 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.502862+0000", 2024-09-21T15:01:01.918 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 32.374070351999997, 2024-09-21T15:01:01.919 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052660208, 2024-09-21T15:01:01.919 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:01.919 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:01.919 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:01.919 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:01.919 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:01.919 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 113896 2024-09-21T15:01:01.919 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.919 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:01.919 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.919 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:01.919 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502862+0000", 2024-09-21T15:01:01.919 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:01.919 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.920 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.920 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:01.920 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502862+0000", 2024-09-21T15:01:01.920 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:01.920 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.920 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.920 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:01.920 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502689+0000", 2024-09-21T15:01:01.920 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9998264 2024-09-21T15:01:01.920 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.920 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.920 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:01.920 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502873+0000", 2024-09-21T15:01:01.920 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000184146 2024-09-21T15:01:01.920 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.921 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.921 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:01.921 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502884+0000", 2024-09-21T15:01:01.921 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1293e-05 2024-09-21T15:01:01.921 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.921 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.921 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:01.921 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502891+0000", 2024-09-21T15:01:01.921 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4259999999999997e-06 2024-09-21T15:01:01.921 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.921 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.921 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:01.921 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555435+0000", 2024-09-21T15:01:01.921 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052543053999999999 2024-09-21T15:01:01.921 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.922 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.922 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:01.922 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555471+0000", 2024-09-21T15:01:01.922 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6454e-05 2024-09-21T15:01:01.922 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.922 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.922 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:01.922 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555522+0000", 2024-09-21T15:01:01.922 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1314999999999999e-05 2024-09-21T15:01:01.922 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:01.922 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:01.922 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:01.922 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.922 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.923 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:113897 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 6021120~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:01.923 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.503004+0000", 2024-09-21T15:01:01.923 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 32.373928245000002, 2024-09-21T15:01:01.923 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052669499000000002, 2024-09-21T15:01:01.923 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:01.923 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:01.923 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:01.923 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:01.923 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:01.923 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 113897 2024-09-21T15:01:01.923 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.923 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:01.923 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.923 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:01.923 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503004+0000", 2024-09-21T15:01:01.923 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:01.924 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.924 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.924 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:01.924 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503004+0000", 2024-09-21T15:01:01.924 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:01.924 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.924 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.924 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:01.924 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.502898+0000", 2024-09-21T15:01:01.924 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9998937 2024-09-21T15:01:01.924 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.924 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.924 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:01.924 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503010+0000", 2024-09-21T15:01:01.925 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000112396 2024-09-21T15:01:01.925 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.925 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.925 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:01.925 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503023+0000", 2024-09-21T15:01:01.925 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2206e-05 2024-09-21T15:01:01.925 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.925 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.925 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:01.925 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503031+0000", 2024-09-21T15:01:01.925 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.3780000000000001e-06 2024-09-21T15:01:01.925 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.925 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.925 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:01.925 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555549+0000", 2024-09-21T15:01:01.926 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052517909000000002 2024-09-21T15:01:01.926 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.926 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.926 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:01.926 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555594+0000", 2024-09-21T15:01:01.926 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5385e-05 2024-09-21T15:01:01.926 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.926 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.926 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:01.926 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555674+0000", 2024-09-21T15:01:01.926 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9447e-05 2024-09-21T15:01:01.926 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:01.926 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:01.927 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:01.927 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.927 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.927 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:113898 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 6111232~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:01.927 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.503039+0000", 2024-09-21T15:01:01.927 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 32.373893459999998, 2024-09-21T15:01:01.927 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052754290000000002, 2024-09-21T15:01:01.927 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:01.927 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:01.927 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:01.927 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:01.927 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:01.927 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 113898 2024-09-21T15:01:01.927 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.928 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:01.928 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.928 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:01.928 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503039+0000", 2024-09-21T15:01:01.928 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:01.928 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.928 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.928 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:01.928 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503039+0000", 2024-09-21T15:01:01.928 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:01.928 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.928 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.928 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:01.928 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503038+0000", 2024-09-21T15:01:01.928 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:01.929 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.929 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.929 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:01.929 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503042+0000", 2024-09-21T15:01:01.929 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.0640000000000004e-06 2024-09-21T15:01:01.929 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.929 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.929 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:01.929 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503049+0000", 2024-09-21T15:01:01.929 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5579999999999997e-06 2024-09-21T15:01:01.929 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.929 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.929 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:01.930 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503054+0000", 2024-09-21T15:01:01.930 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6550000000000001e-06 2024-09-21T15:01:01.930 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.930 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.930 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:01.930 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555693+0000", 2024-09-21T15:01:01.930 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052638216000000002 2024-09-21T15:01:01.930 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.930 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.930 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:01.930 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555736+0000", 2024-09-21T15:01:01.930 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3619999999999999e-05 2024-09-21T15:01:01.930 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.931 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.931 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:01.931 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555793+0000", 2024-09-21T15:01:01.931 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6956999999999997e-05 2024-09-21T15:01:01.931 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:01.931 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:01.931 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:01.931 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.931 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.931 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:113899 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 6201344~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:01.931 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.503255+0000", 2024-09-21T15:01:01.931 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 32.373677186999998, 2024-09-21T15:01:01.931 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052648161999999998, 2024-09-21T15:01:01.931 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:01.932 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:01.932 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:01.932 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:01.932 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:01.932 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 113899 2024-09-21T15:01:01.932 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.932 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:01.932 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.932 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:01.932 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503255+0000", 2024-09-21T15:01:01.932 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:01.932 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.932 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.932 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:01.933 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503255+0000", 2024-09-21T15:01:01.933 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:01.933 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.933 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.933 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:01.933 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503058+0000", 2024-09-21T15:01:01.933 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9998021 2024-09-21T15:01:01.933 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.933 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.933 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:01.933 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503262+0000", 2024-09-21T15:01:01.933 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00020421299999999999 2024-09-21T15:01:01.933 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.933 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.933 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:01.933 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503274+0000", 2024-09-21T15:01:01.934 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2052000000000001e-05 2024-09-21T15:01:01.934 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.934 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.934 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:01.934 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503282+0000", 2024-09-21T15:01:01.934 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.977e-06 2024-09-21T15:01:01.934 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.934 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.934 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:01.934 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555808+0000", 2024-09-21T15:01:01.934 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052526150000000001 2024-09-21T15:01:01.934 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.934 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.934 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:01.934 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555851+0000", 2024-09-21T15:01:01.935 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2846999999999998e-05 2024-09-21T15:01:01.935 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.935 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.935 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:01.935 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555903+0000", 2024-09-21T15:01:01.935 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2607e-05 2024-09-21T15:01:01.935 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:01.935 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:01.935 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:01.935 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.935 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.935 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:113900 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 6307840~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:01.935 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.503445+0000", 2024-09-21T15:01:01.935 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 32.373487251999997, 2024-09-21T15:01:01.936 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052589546000000001, 2024-09-21T15:01:01.936 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:01.936 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:01.936 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:01.936 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:01.936 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:01.936 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 113900 2024-09-21T15:01:01.936 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.936 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:01.936 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.936 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:01.936 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503445+0000", 2024-09-21T15:01:01.936 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:01.936 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.936 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.937 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:01.937 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503445+0000", 2024-09-21T15:01:01.937 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:01.937 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.937 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.937 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:01.937 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503290+0000", 2024-09-21T15:01:01.937 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9998446 2024-09-21T15:01:01.937 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.937 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.937 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:01.937 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503451+0000", 2024-09-21T15:01:01.937 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00016111000000000001 2024-09-21T15:01:01.937 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.937 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.937 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:01.938 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503461+0000", 2024-09-21T15:01:01.938 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8740000000000007e-06 2024-09-21T15:01:01.938 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.938 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.938 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:01.938 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.503468+0000", 2024-09-21T15:01:01.938 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.187e-06 2024-09-21T15:01:01.938 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.938 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.938 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:01.938 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555930+0000", 2024-09-21T15:01:01.938 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052462404999999997 2024-09-21T15:01:01.938 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.938 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.939 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:01.939 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.555971+0000", 2024-09-21T15:01:01.939 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.0419e-05 2024-09-21T15:01:01.939 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.939 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.939 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:01.939 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.556035+0000", 2024-09-21T15:01:01.939 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3948999999999994e-05 2024-09-21T15:01:01.939 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:01.939 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:01.939 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:01.939 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.939 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.939 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:114050 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 4767744~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:01.939 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.514988+0000", 2024-09-21T15:01:01.939 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 32.361944741000002, 2024-09-21T15:01:01.940 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052614432000000003, 2024-09-21T15:01:01.940 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:01.940 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:01.940 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:01.940 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:01.940 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:01.940 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 114050 2024-09-21T15:01:01.940 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.940 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:01.940 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.940 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:01.940 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.514988+0000", 2024-09-21T15:01:01.940 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:01.940 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.940 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.941 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:01.941 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.514988+0000", 2024-09-21T15:01:01.941 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:01.941 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.941 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.941 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:01.941 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.514987+0000", 2024-09-21T15:01:01.941 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:01.941 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.941 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.941 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:01.941 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.514990+0000", 2024-09-21T15:01:01.941 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4739999999999999e-06 2024-09-21T15:01:01.941 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.942 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.942 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:01.942 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.514995+0000", 2024-09-21T15:01:01.942 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1939999999999997e-06 2024-09-21T15:01:01.942 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.942 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.942 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:01.942 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.514999+0000", 2024-09-21T15:01:01.942 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.245e-06 2024-09-21T15:01:01.942 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.942 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.942 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:01.942 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.567498+0000", 2024-09-21T15:01:01.942 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052498516000000002 2024-09-21T15:01:01.942 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.943 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.943 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:01.943 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.567541+0000", 2024-09-21T15:01:01.943 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3516000000000001e-05 2024-09-21T15:01:01.943 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.943 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.943 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:01.943 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.567602+0000", 2024-09-21T15:01:01.943 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0800000000000001e-05 2024-09-21T15:01:01.943 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:01.943 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:01.943 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:01.943 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.943 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.943 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:114051 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 4857856~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:01.943 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.515003+0000", 2024-09-21T15:01:01.944 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 32.361929025999999, 2024-09-21T15:01:01.944 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052709245000000002, 2024-09-21T15:01:01.944 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:01.944 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:01.944 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:01.944 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:01.944 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:01.944 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 114051 2024-09-21T15:01:01.944 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.944 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:01.944 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.944 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:01.944 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515003+0000", 2024-09-21T15:01:01.944 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:01.944 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.944 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.945 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:01.945 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515003+0000", 2024-09-21T15:01:01.945 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:01.945 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.945 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.945 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:01.945 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515003+0000", 2024-09-21T15:01:01.945 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:01.945 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.945 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.945 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:01.945 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515005+0000", 2024-09-21T15:01:01.945 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.3499999999999999e-06 2024-09-21T15:01:01.945 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.945 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.946 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:01.946 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515011+0000", 2024-09-21T15:01:01.946 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2850000000000002e-06 2024-09-21T15:01:01.946 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.946 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.946 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:01.946 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515017+0000", 2024-09-21T15:01:01.946 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0059999999999998e-06 2024-09-21T15:01:01.946 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.946 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.946 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:01.946 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.567618+0000", 2024-09-21T15:01:01.946 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052600806999999999 2024-09-21T15:01:01.946 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.947 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.947 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:01.947 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.567655+0000", 2024-09-21T15:01:01.947 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7866e-05 2024-09-21T15:01:01.947 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.947 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.947 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:01.947 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.567713+0000", 2024-09-21T15:01:01.947 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7281999999999998e-05 2024-09-21T15:01:01.947 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:01.947 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:01.947 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:01.947 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.947 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.948 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:114052 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 4947968~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:01.949 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.515046+0000", 2024-09-21T15:01:01.949 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 32.361886366999997, 2024-09-21T15:01:01.949 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052795975000000002, 2024-09-21T15:01:01.949 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:01.949 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:01.950 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:01.950 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:01.950 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:01.950 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 114052 2024-09-21T15:01:01.950 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.950 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:01.950 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.950 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:01.950 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515046+0000", 2024-09-21T15:01:01.950 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:01.950 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.950 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.950 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:01.951 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515046+0000", 2024-09-21T15:01:01.951 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:01.951 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.951 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.951 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:01.951 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515046+0000", 2024-09-21T15:01:01.951 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:01.951 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.951 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.951 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:01.951 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515050+0000", 2024-09-21T15:01:01.951 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1359999999999999e-06 2024-09-21T15:01:01.951 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.951 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.952 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:01.952 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515060+0000", 2024-09-21T15:01:01.952 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0339e-05 2024-09-21T15:01:01.952 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.952 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.952 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:01.952 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515068+0000", 2024-09-21T15:01:01.952 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1149999999999994e-06 2024-09-21T15:01:01.952 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.952 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.952 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:01.952 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.567733+0000", 2024-09-21T15:01:01.952 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052664872000000001 2024-09-21T15:01:01.952 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.952 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.953 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:01.953 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.567778+0000", 2024-09-21T15:01:01.953 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4966999999999998e-05 2024-09-21T15:01:01.953 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.953 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.953 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:01.953 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.567842+0000", 2024-09-21T15:01:01.953 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4114000000000003e-05 2024-09-21T15:01:01.953 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:01.953 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:01.953 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:01.953 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.953 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.953 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:114053 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 5038080~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:01.954 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.515076+0000", 2024-09-21T15:01:01.954 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 32.361856506999999, 2024-09-21T15:01:01.954 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052886112999999998, 2024-09-21T15:01:01.954 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:01.954 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:01.954 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:01.954 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:01.954 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:01.954 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 114053 2024-09-21T15:01:01.954 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.954 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:01.954 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.954 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:01.955 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515076+0000", 2024-09-21T15:01:01.955 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:01.955 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.955 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.955 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:01.955 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515076+0000", 2024-09-21T15:01:01.955 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:01.955 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.955 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.955 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:01.955 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515075+0000", 2024-09-21T15:01:01.955 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:01.955 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.955 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.956 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:01.956 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515083+0000", 2024-09-21T15:01:01.956 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0520000000000002e-06 2024-09-21T15:01:01.956 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.956 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.956 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:01.956 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515092+0000", 2024-09-21T15:01:01.956 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1440000000000003e-06 2024-09-21T15:01:01.956 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.956 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.956 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:01.956 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515098+0000", 2024-09-21T15:01:01.956 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1090000000000003e-06 2024-09-21T15:01:01.957 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.957 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.957 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:01.957 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.567871+0000", 2024-09-21T15:01:01.957 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052772966999999997 2024-09-21T15:01:01.957 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.957 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.957 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:01.957 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.567909+0000", 2024-09-21T15:01:01.957 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.8380000000000002e-05 2024-09-21T15:01:01.957 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.957 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.957 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:01.957 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.567962+0000", 2024-09-21T15:01:01.957 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3068000000000001e-05 2024-09-21T15:01:01.958 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:01.958 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:01.958 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:01.958 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.958 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.958 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:114055 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 5218304~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:01.958 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.515526+0000", 2024-09-21T15:01:01.958 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 32.361406066999997, 2024-09-21T15:01:01.958 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052684278000000001, 2024-09-21T15:01:01.958 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:01.958 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:01.958 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:01.958 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:01.958 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:01.959 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 114055 2024-09-21T15:01:01.959 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.959 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:01.959 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.959 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:01.959 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515526+0000", 2024-09-21T15:01:01.959 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:01.959 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.959 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.959 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:01.959 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515526+0000", 2024-09-21T15:01:01.959 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:01.959 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.959 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.960 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:01.960 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515526+0000", 2024-09-21T15:01:01.960 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:01.960 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.960 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.960 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:01.960 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515529+0000", 2024-09-21T15:01:01.960 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.596e-06 2024-09-21T15:01:01.960 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.960 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.960 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:01.960 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515536+0000", 2024-09-21T15:01:01.960 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7399999999999998e-06 2024-09-21T15:01:01.960 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.960 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.961 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:01.961 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515542+0000", 2024-09-21T15:01:01.961 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5740000000000003e-06 2024-09-21T15:01:01.961 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.961 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.961 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:01.961 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.568116+0000", 2024-09-21T15:01:01.961 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052574152999999998 2024-09-21T15:01:01.961 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.961 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.961 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:01.961 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.568154+0000", 2024-09-21T15:01:01.961 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7926000000000002e-05 2024-09-21T15:01:01.961 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.961 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.962 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:01.962 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.568211+0000", 2024-09-21T15:01:01.962 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6858999999999999e-05 2024-09-21T15:01:01.962 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:01.962 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:01.962 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:01.962 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.962 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.962 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:114056 3.f 3:f0f830c6:::rbd_data.1136b1194d20.0000000000000003:58 [read 5308416~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:01.962 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:29.515547+0000", 2024-09-21T15:01:01.962 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 32.361385042000002, 2024-09-21T15:01:01.962 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052792795000000003, 2024-09-21T15:01:01.962 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:01.963 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:01.963 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:01.963 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:01.963 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:01.963 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 114056 2024-09-21T15:01:01.963 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.963 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:01.963 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.963 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:01.963 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515547+0000", 2024-09-21T15:01:01.963 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:01.963 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.963 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.963 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:01.963 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515547+0000", 2024-09-21T15:01:01.964 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:01.964 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.964 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.964 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:01.964 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515547+0000", 2024-09-21T15:01:01.964 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:01.964 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.964 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.964 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:01.964 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515550+0000", 2024-09-21T15:01:01.964 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2770000000000001e-06 2024-09-21T15:01:01.964 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.964 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.964 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:01.964 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515554+0000", 2024-09-21T15:01:01.965 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.087e-06 2024-09-21T15:01:01.965 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.965 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.965 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:01.965 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.515558+0000", 2024-09-21T15:01:01.965 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1709999999999998e-06 2024-09-21T15:01:01.965 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.965 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.965 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:01.965 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.568234+0000", 2024-09-21T15:01:01.965 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.052676675999999999 2024-09-21T15:01:01.965 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.965 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.965 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:01.965 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.568277+0000", 2024-09-21T15:01:01.966 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2400000000000001e-05 2024-09-21T15:01:01.966 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:01.966 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:01.966 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:01.966 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:29.568340+0000", 2024-09-21T15:01:01.966 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3581e-05 2024-09-21T15:01:01.966 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:01.966 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:01.966 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:01.966 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:01.966 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:01.966 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:01.968 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T15:01:01.968 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:01:01.965+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:01:02.068 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:01:02.065+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:01:02.080 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:02.081 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:01:02.081 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T15:01:02.081 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:02.090 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T15:01:02.169 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:01:02.167+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:01:02.197 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:02.197 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:01:02.197 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T15:01:02.197 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T15:01:02.197 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:02.211 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T15:01:02.215 INFO:teuthology.orchestra.run.smithi032.stderr:reweighted osd.3 to 1 (10000) 2024-09-21T15:01:02.216 INFO:teuthology.orchestra.run.smithi032.stderr:nodeep-scrub is unset 2024-09-21T15:01:02.239 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-21T15:01:02.270 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:01:02.265+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:01:02.319 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:02.320 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T15:01:02.320 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T15:01:02.320 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T15:01:02.320 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.320 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T15:01:02.320 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.205754+0000", 2024-09-21T15:01:02.320 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 179.11210736999999, 2024-09-21T15:01:02.320 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.41117094199999998, 2024-09-21T15:01:02.320 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:02.320 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:02.320 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:02.320 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T15:01:02.320 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T15:01:02.321 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1 2024-09-21T15:01:02.321 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.321 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:02.321 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.321 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:02.321 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205754+0000", 2024-09-21T15:01:02.321 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:02.321 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.321 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.321 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:02.321 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205754+0000", 2024-09-21T15:01:02.321 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:02.321 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.321 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.321 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:02.322 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205752+0000", 2024-09-21T15:01:02.322 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T15:01:02.322 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.322 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.322 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:02.322 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205765+0000", 2024-09-21T15:01:02.322 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3230999999999999e-05 2024-09-21T15:01:02.322 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.322 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.322 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:02.322 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205782+0000", 2024-09-21T15:01:02.322 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6634e-05 2024-09-21T15:01:02.322 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.323 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.323 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:02.323 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205793+0000", 2024-09-21T15:01:02.323 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1297e-05 2024-09-21T15:01:02.323 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.323 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.323 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:02.323 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205867+0000", 2024-09-21T15:01:02.323 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4269000000000003e-05 2024-09-21T15:01:02.323 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.323 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.323 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:02.323 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205877+0000", 2024-09-21T15:01:02.323 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2690000000000005e-06 2024-09-21T15:01:02.324 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.324 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.324 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:02.324 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.613700+0000", 2024-09-21T15:01:02.324 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.407823455 2024-09-21T15:01:02.324 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.324 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.324 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:02.324 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.614203+0000", 2024-09-21T15:01:02.324 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00050311500000000005 2024-09-21T15:01:02.324 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.324 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.324 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T15:01:02.324 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.614750+0000", 2024-09-21T15:01:02.325 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00054738699999999998 2024-09-21T15:01:02.325 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.325 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.325 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:01:02.325 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616367+0000", 2024-09-21T15:01:02.325 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001616634 2024-09-21T15:01:02.325 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.325 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.325 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:01:02.325 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616574+0000", 2024-09-21T15:01:02.325 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00020668499999999999 2024-09-21T15:01:02.325 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.325 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.325 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:01:02.326 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616802+0000", 2024-09-21T15:01:02.326 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00022817900000000001 2024-09-21T15:01:02.326 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.326 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.326 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:02.326 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616925+0000", 2024-09-21T15:01:02.326 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012309699999999999 2024-09-21T15:01:02.326 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:02.326 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:02.326 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:02.326 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.326 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.326 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:61214 3.7 3:e28aec10:::rbd_header.1136862f872f:head [watch reconnect cookie 139864671932080 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-21T15:01:02.327 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:18.147172+0000", 2024-09-21T15:01:02.327 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 104.170689727, 2024-09-21T15:01:02.327 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.019264956, 2024-09-21T15:01:02.327 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:02.327 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:02.327 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:02.327 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:02.327 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:02.327 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 61214 2024-09-21T15:01:02.327 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.327 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:02.327 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.327 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:02.327 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147172+0000", 2024-09-21T15:01:02.328 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:02.328 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.328 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.328 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:02.328 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147172+0000", 2024-09-21T15:01:02.328 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:02.328 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.328 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.328 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:02.328 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147168+0000", 2024-09-21T15:01:02.328 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999962 2024-09-21T15:01:02.328 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.328 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.328 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:02.329 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147188+0000", 2024-09-21T15:01:02.329 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9717000000000001e-05 2024-09-21T15:01:02.329 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.329 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.329 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:02.329 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147239+0000", 2024-09-21T15:01:02.329 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1493999999999999e-05 2024-09-21T15:01:02.329 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.329 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.329 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:02.329 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147265+0000", 2024-09-21T15:01:02.329 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5746999999999999e-05 2024-09-21T15:01:02.329 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.329 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.330 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:02.330 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147372+0000", 2024-09-21T15:01:02.330 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010681200000000001 2024-09-21T15:01:02.330 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.330 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.330 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op must wait for map", 2024-09-21T15:01:02.330 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147380+0000", 2024-09-21T15:01:02.330 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7919999999999993e-06 2024-09-21T15:01:02.330 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.330 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.330 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:02.330 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.154025+0000", 2024-09-21T15:01:02.330 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0066454909999999999 2024-09-21T15:01:02.330 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.331 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.331 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:02.331 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.154041+0000", 2024-09-21T15:01:02.331 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5894000000000001e-05 2024-09-21T15:01:02.331 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.331 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.331 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:02.331 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.164345+0000", 2024-09-21T15:01:02.331 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0103040459999999 2024-09-21T15:01:02.331 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.331 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.331 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:02.331 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.164590+0000", 2024-09-21T15:01:02.331 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00024492699999999998 2024-09-21T15:01:02.331 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.332 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.332 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:02.332 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.166437+0000", 2024-09-21T15:01:02.332 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0018466330000000001 2024-09-21T15:01:02.332 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:02.332 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:02.332 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:02.332 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.332 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.332 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:61215 3.7 3:e28aec10:::rbd_header.1136862f872f:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-21T15:01:02.332 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:18.152008+0000", 2024-09-21T15:01:02.332 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 104.165853275, 2024-09-21T15:01:02.332 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.015644958, 2024-09-21T15:01:02.332 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:02.333 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:02.333 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:02.333 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:02.333 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:02.333 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 61215 2024-09-21T15:01:02.333 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.333 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:02.333 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.333 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:02.333 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152008+0000", 2024-09-21T15:01:02.333 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:02.333 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.333 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.334 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:02.334 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152008+0000", 2024-09-21T15:01:02.334 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:02.334 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.334 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.334 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:02.334 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152005+0000", 2024-09-21T15:01:02.334 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-21T15:01:02.334 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.334 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.334 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:02.334 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152025+0000", 2024-09-21T15:01:02.334 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9306000000000001e-05 2024-09-21T15:01:02.334 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.335 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.335 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:02.335 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152054+0000", 2024-09-21T15:01:02.335 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9189000000000001e-05 2024-09-21T15:01:02.335 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.335 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.335 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:02.335 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152078+0000", 2024-09-21T15:01:02.335 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4317000000000001e-05 2024-09-21T15:01:02.335 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.335 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.335 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:02.335 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152243+0000", 2024-09-21T15:01:02.335 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000164802 2024-09-21T15:01:02.336 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.336 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.336 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting_for_map not empty", 2024-09-21T15:01:02.336 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152251+0000", 2024-09-21T15:01:02.336 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0600000000000008e-06 2024-09-21T15:01:02.336 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.336 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.336 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:02.336 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.154741+0000", 2024-09-21T15:01:02.336 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0024900069999999998 2024-09-21T15:01:02.336 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.336 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.336 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:02.336 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.154753+0000", 2024-09-21T15:01:02.336 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1588e-05 2024-09-21T15:01:02.337 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.337 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.337 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:02.337 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.165380+0000", 2024-09-21T15:01:02.337 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0106272249999999 2024-09-21T15:01:02.337 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.337 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.337 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:02.337 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.165517+0000", 2024-09-21T15:01:02.337 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013737999999999999 2024-09-21T15:01:02.337 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.337 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.337 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 6", 2024-09-21T15:01:02.337 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.165938+0000", 2024-09-21T15:01:02.338 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00042082399999999998 2024-09-21T15:01:02.338 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.338 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.338 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:01:02.338 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.166795+0000", 2024-09-21T15:01:02.338 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00085674899999999997 2024-09-21T15:01:02.338 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.338 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.338 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:01:02.338 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.167598+0000", 2024-09-21T15:01:02.338 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00080335599999999997 2024-09-21T15:01:02.338 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.338 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.338 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:01:02.339 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.167632+0000", 2024-09-21T15:01:02.339 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4122000000000003e-05 2024-09-21T15:01:02.339 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.339 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.339 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:02.339 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.167653+0000", 2024-09-21T15:01:02.339 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0862000000000001e-05 2024-09-21T15:01:02.339 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:02.339 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:02.339 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:02.339 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.339 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.339 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:97678 3.d e122/24 3:b2f34c6d:::rbd_data.1136bfc79a70.0000000000000006:head v 122'234, mlcod=120'232)", 2024-09-21T15:01:02.339 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.321655+0000", 2024-09-21T15:01:02.339 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 56.996206358000002, 2024-09-21T15:01:02.339 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024509388999999999, 2024-09-21T15:01:02.340 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:02.340 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:02.340 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:02.340 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.340 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:02.340 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.321655+0000", 2024-09-21T15:01:02.340 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:02.340 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.340 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.340 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:02.340 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.321655+0000", 2024-09-21T15:01:02.340 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:02.340 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.340 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.340 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:02.340 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.321655+0000", 2024-09-21T15:01:02.341 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:02.341 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.341 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.341 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:02.341 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.329453+0000", 2024-09-21T15:01:02.341 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0077980560000000003 2024-09-21T15:01:02.341 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.341 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.341 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:02.341 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.329482+0000", 2024-09-21T15:01:02.341 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9153999999999999e-05 2024-09-21T15:01:02.341 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.341 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.341 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:02.342 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.329493+0000", 2024-09-21T15:01:02.342 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1103e-05 2024-09-21T15:01:02.342 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.342 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.342 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:02.342 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.329555+0000", 2024-09-21T15:01:02.342 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2345e-05 2024-09-21T15:01:02.342 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.342 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.342 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:02.342 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.329589+0000", 2024-09-21T15:01:02.342 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3908999999999997e-05 2024-09-21T15:01:02.342 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.342 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.342 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:01:02.343 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.346083+0000", 2024-09-21T15:01:02.343 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016493383 2024-09-21T15:01:02.343 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.343 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.343 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:02.343 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.346165+0000", 2024-09-21T15:01:02.343 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1796000000000001e-05 2024-09-21T15:01:02.343 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:02.343 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:02.343 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:02.343 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.343 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.343 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:97679 3.f e122/24 3:f2a362b4:::rbd_data.1136bfc79a70.0000000000000007:head v 122'331, mlcod=120'329)", 2024-09-21T15:01:02.343 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.337145+0000", 2024-09-21T15:01:02.344 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 56.980716348999998, 2024-09-21T15:01:02.344 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021912411, 2024-09-21T15:01:02.344 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:02.344 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:02.344 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:02.344 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.344 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:02.344 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.337145+0000", 2024-09-21T15:01:02.344 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:02.344 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.344 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.344 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:02.344 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.337145+0000", 2024-09-21T15:01:02.344 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:02.344 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.345 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.345 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:02.345 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.337145+0000", 2024-09-21T15:01:02.345 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:02.345 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.345 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.345 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:02.345 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.343899+0000", 2024-09-21T15:01:02.345 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0067541420000000003 2024-09-21T15:01:02.345 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.345 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.345 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:02.345 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.343921+0000", 2024-09-21T15:01:02.345 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2255e-05 2024-09-21T15:01:02.345 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.346 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.346 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:02.346 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.343933+0000", 2024-09-21T15:01:02.346 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1528e-05 2024-09-21T15:01:02.346 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.346 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.346 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:02.346 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.344001+0000", 2024-09-21T15:01:02.346 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8269999999999995e-05 2024-09-21T15:01:02.346 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.346 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.346 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:02.346 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.344029+0000", 2024-09-21T15:01:02.346 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8254000000000001e-05 2024-09-21T15:01:02.346 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.347 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.347 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:01:02.347 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.358901+0000", 2024-09-21T15:01:02.347 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0148719 2024-09-21T15:01:02.347 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.347 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.347 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:02.347 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.359058+0000", 2024-09-21T15:01:02.347 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000156446 2024-09-21T15:01:02.347 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:02.347 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:02.347 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:02.347 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.347 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.347 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:97676 3.1f e122/24 3:ffaa836a:::rbd_data.1136bfc79a70.0000000000000004:head v 122'312, mlcod=120'310)", 2024-09-21T15:01:02.348 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.348683+0000", 2024-09-21T15:01:02.348 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 56.969178184, 2024-09-21T15:01:02.348 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.033768802000000001, 2024-09-21T15:01:02.348 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:02.348 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:02.348 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:02.348 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.348 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:02.348 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.348683+0000", 2024-09-21T15:01:02.348 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:02.348 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.348 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.348 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:02.348 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.348683+0000", 2024-09-21T15:01:02.348 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:02.348 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.349 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.349 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:02.349 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.348683+0000", 2024-09-21T15:01:02.349 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:02.349 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.349 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.349 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:02.349 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.366937+0000", 2024-09-21T15:01:02.349 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018254415 2024-09-21T15:01:02.349 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.349 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.349 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:02.349 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.366959+0000", 2024-09-21T15:01:02.349 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1634e-05 2024-09-21T15:01:02.349 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.350 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.350 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:02.350 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.366971+0000", 2024-09-21T15:01:02.350 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2305e-05 2024-09-21T15:01:02.350 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.350 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.350 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:02.350 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.367034+0000", 2024-09-21T15:01:02.350 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3211000000000002e-05 2024-09-21T15:01:02.350 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.350 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.350 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:02.350 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.367056+0000", 2024-09-21T15:01:02.350 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2098000000000001e-05 2024-09-21T15:01:02.350 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.351 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.351 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:01:02.351 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.382400+0000", 2024-09-21T15:01:02.351 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.015343867000000001 2024-09-21T15:01:02.351 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.351 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.351 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:02.351 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.382452+0000", 2024-09-21T15:01:02.351 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1992e-05 2024-09-21T15:01:02.351 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:02.351 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:02.351 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:02.351 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.351 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.352 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:97734 3.19 e122/94 3:9fcf754f:::rbd_data.1136bfc79a70.0000000000000009:head v 122'314, mlcod=120'312)", 2024-09-21T15:01:02.352 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.378547+0000", 2024-09-21T15:01:02.352 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 56.939314205999999, 2024-09-21T15:01:02.352 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.063380675999999997, 2024-09-21T15:01:02.352 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:02.352 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:02.352 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:02.352 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.352 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:02.352 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.378547+0000", 2024-09-21T15:01:02.352 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:02.352 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.352 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.352 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:02.352 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.378547+0000", 2024-09-21T15:01:02.353 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:02.353 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.353 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.353 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:02.353 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.378547+0000", 2024-09-21T15:01:02.353 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:01:02.353 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.353 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.353 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:02.353 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.425118+0000", 2024-09-21T15:01:02.353 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046571166999999997 2024-09-21T15:01:02.353 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.353 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.353 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:02.354 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.425150+0000", 2024-09-21T15:01:02.354 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2150999999999998e-05 2024-09-21T15:01:02.354 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.354 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.354 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:02.354 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.425164+0000", 2024-09-21T15:01:02.354 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3302000000000001e-05 2024-09-21T15:01:02.354 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.354 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.354 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:02.354 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.425231+0000", 2024-09-21T15:01:02.354 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7106000000000004e-05 2024-09-21T15:01:02.354 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.354 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.355 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:02.355 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.425259+0000", 2024-09-21T15:01:02.355 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8087000000000001e-05 2024-09-21T15:01:02.355 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.355 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.355 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:01:02.355 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.441890+0000", 2024-09-21T15:01:02.355 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016631328000000001 2024-09-21T15:01:02.355 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.355 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.355 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:02.355 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.441928+0000", 2024-09-21T15:01:02.355 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7701999999999999e-05 2024-09-21T15:01:02.356 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:02.356 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:02.356 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:02.356 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.356 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.356 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:100187 3.10 3:08941a97:::rbd_data.1136ab4d6ff1.000000000000000e:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-21T15:01:02.356 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:10.869845+0000", 2024-09-21T15:01:02.356 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 51.44801631, 2024-09-21T15:01:02.356 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029345237, 2024-09-21T15:01:02.356 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:02.356 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:02.356 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:02.356 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:02.356 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:02.356 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 100187 2024-09-21T15:01:02.356 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.357 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:02.357 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.357 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:02.357 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869845+0000", 2024-09-21T15:01:02.357 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:02.357 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.357 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.357 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:02.357 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869845+0000", 2024-09-21T15:01:02.357 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:02.357 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.357 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.357 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:02.357 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869844+0000", 2024-09-21T15:01:02.358 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:02.358 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.358 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.358 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:02.358 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869853+0000", 2024-09-21T15:01:02.358 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5719999999999992e-06 2024-09-21T15:01:02.358 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.358 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.358 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:02.358 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869874+0000", 2024-09-21T15:01:02.358 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0483e-05 2024-09-21T15:01:02.358 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.358 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.358 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:02.359 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869883+0000", 2024-09-21T15:01:02.359 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6080000000000002e-06 2024-09-21T15:01:02.359 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.359 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.359 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:02.359 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869991+0000", 2024-09-21T15:01:02.359 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000108013 2024-09-21T15:01:02.359 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.359 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.359 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:02.361 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.870258+0000", 2024-09-21T15:01:02.361 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00026629099999999999 2024-09-21T15:01:02.361 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.362 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.362 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T15:01:02.362 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.878656+0000", 2024-09-21T15:01:02.362 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0083989000000000008 2024-09-21T15:01:02.362 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.362 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.362 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:01:02.362 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.899081+0000", 2024-09-21T15:01:02.362 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.020424437 2024-09-21T15:01:02.362 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.362 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.362 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:01:02.362 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.899125+0000", 2024-09-21T15:01:02.362 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4440999999999997e-05 2024-09-21T15:01:02.363 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.363 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.363 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:01:02.363 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.899160+0000", 2024-09-21T15:01:02.363 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4518999999999999e-05 2024-09-21T15:01:02.363 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.363 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.363 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:02.363 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.899190+0000", 2024-09-21T15:01:02.363 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0574999999999998e-05 2024-09-21T15:01:02.363 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:02.363 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:02.363 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:02.363 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.363 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.364 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:100263 3.10 3:0ece142e:::rbd_data.1136ab4d6ff1.0000000000000008:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-21T15:01:02.364 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:11.052257+0000", 2024-09-21T15:01:02.364 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 51.265604252000003, 2024-09-21T15:01:02.364 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042678025000000001, 2024-09-21T15:01:02.364 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:02.364 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:02.364 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:02.364 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:02.364 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:02.364 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 100263 2024-09-21T15:01:02.364 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.364 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:02.364 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.364 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:02.364 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052257+0000", 2024-09-21T15:01:02.365 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:02.365 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.365 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.365 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:02.365 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052257+0000", 2024-09-21T15:01:02.365 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:02.365 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.365 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.365 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:02.365 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052256+0000", 2024-09-21T15:01:02.365 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:02.365 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.365 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.365 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:02.365 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052263+0000", 2024-09-21T15:01:02.366 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4799999999999998e-06 2024-09-21T15:01:02.366 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.366 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.366 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:02.366 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052275+0000", 2024-09-21T15:01:02.366 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2386e-05 2024-09-21T15:01:02.366 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.366 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.366 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:02.366 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052281+0000", 2024-09-21T15:01:02.366 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6699999999999999e-06 2024-09-21T15:01:02.366 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.366 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.366 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:02.367 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052350+0000", 2024-09-21T15:01:02.367 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9099999999999999e-05 2024-09-21T15:01:02.367 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.367 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.367 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:02.367 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052456+0000", 2024-09-21T15:01:02.367 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000106669 2024-09-21T15:01:02.367 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.367 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.367 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T15:01:02.367 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.059694+0000", 2024-09-21T15:01:02.367 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.007237566 2024-09-21T15:01:02.367 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.367 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.367 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:01:02.368 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.094721+0000", 2024-09-21T15:01:02.368 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035026977000000001 2024-09-21T15:01:02.368 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.368 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.368 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:01:02.368 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.094746+0000", 2024-09-21T15:01:02.368 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5329000000000001e-05 2024-09-21T15:01:02.368 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.368 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.368 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:01:02.368 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.094850+0000", 2024-09-21T15:01:02.368 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000103452 2024-09-21T15:01:02.368 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.368 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.368 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:02.368 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.094935+0000", 2024-09-21T15:01:02.369 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5550000000000006e-05 2024-09-21T15:01:02.369 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:02.369 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:02.369 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:02.369 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.369 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.369 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:134708 3.a 3:51eaadc0:::rbd_data.11365ed419e3.0000000000000007:67 [read 221184~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:02.369 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.938945+0000", 2024-09-21T15:01:02.369 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 8.3789169739999991, 2024-09-21T15:01:02.369 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021073784000000002, 2024-09-21T15:01:02.369 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:02.369 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:02.369 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:02.369 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:02.369 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:02.369 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 134708 2024-09-21T15:01:02.370 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.370 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:02.370 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.370 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:02.370 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938945+0000", 2024-09-21T15:01:02.370 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:02.370 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.370 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.370 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:02.370 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938945+0000", 2024-09-21T15:01:02.370 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:02.370 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.370 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.370 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:02.370 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938944+0000", 2024-09-21T15:01:02.371 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:02.371 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.371 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.371 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:02.371 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938946+0000", 2024-09-21T15:01:02.371 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.616e-06 2024-09-21T15:01:02.371 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.371 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.371 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:02.371 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938952+0000", 2024-09-21T15:01:02.371 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7309999999999998e-06 2024-09-21T15:01:02.371 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.371 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.371 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:02.371 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938957+0000", 2024-09-21T15:01:02.372 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8579999999999999e-06 2024-09-21T15:01:02.372 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.372 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.372 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:02.372 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.959730+0000", 2024-09-21T15:01:02.372 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.020773183000000001 2024-09-21T15:01:02.372 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.372 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.372 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:02.372 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.959810+0000", 2024-09-21T15:01:02.372 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0158000000000001e-05 2024-09-21T15:01:02.372 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.372 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.372 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:02.372 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.960018+0000", 2024-09-21T15:01:02.372 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000207951 2024-09-21T15:01:02.373 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:02.373 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:02.373 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:02.373 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.373 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.373 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:134709 3.a 3:51eaadc0:::rbd_data.11365ed419e3.0000000000000007:67 [read 237568~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:02.373 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.938996+0000", 2024-09-21T15:01:02.373 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 8.3788659820000007, 2024-09-21T15:01:02.373 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021382577, 2024-09-21T15:01:02.373 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:02.373 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:02.373 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:02.373 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:02.373 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:02.373 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 134709 2024-09-21T15:01:02.373 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.374 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:02.374 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.374 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:02.374 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938996+0000", 2024-09-21T15:01:02.374 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:02.374 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.374 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.374 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:02.374 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938996+0000", 2024-09-21T15:01:02.374 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:02.374 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.374 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.374 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:02.374 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938995+0000", 2024-09-21T15:01:02.375 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:02.375 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.375 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.375 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:02.375 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938999+0000", 2024-09-21T15:01:02.375 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7979999999999999e-06 2024-09-21T15:01:02.375 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.375 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.375 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:02.375 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939006+0000", 2024-09-21T15:01:02.375 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6669999999999996e-06 2024-09-21T15:01:02.375 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.375 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.375 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:02.376 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939012+0000", 2024-09-21T15:01:02.376 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9329999999999998e-06 2024-09-21T15:01:02.376 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.376 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.376 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:02.376 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.960080+0000", 2024-09-21T15:01:02.376 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021068303 2024-09-21T15:01:02.376 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.376 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.376 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:02.376 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.960154+0000", 2024-09-21T15:01:02.376 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3819000000000006e-05 2024-09-21T15:01:02.376 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.376 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.377 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:02.377 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.960378+0000", 2024-09-21T15:01:02.377 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00022385800000000001 2024-09-21T15:01:02.377 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:02.377 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:02.377 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:02.377 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.377 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.377 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:134710 3.a 3:51eaadc0:::rbd_data.11365ed419e3.0000000000000007:67 [read 253952~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:02.377 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.939017+0000", 2024-09-21T15:01:02.377 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 8.3788449709999995, 2024-09-21T15:01:02.377 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021721436, 2024-09-21T15:01:02.377 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:02.377 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:02.377 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:02.378 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:02.378 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:02.378 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 134710 2024-09-21T15:01:02.378 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.378 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:02.378 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.378 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:02.378 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939017+0000", 2024-09-21T15:01:02.378 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:02.378 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.378 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.378 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:02.378 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939017+0000", 2024-09-21T15:01:02.378 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:02.379 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.379 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.379 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:02.379 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939016+0000", 2024-09-21T15:01:02.379 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:02.379 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.379 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.379 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:02.379 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939019+0000", 2024-09-21T15:01:02.379 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.554e-06 2024-09-21T15:01:02.379 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.379 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.379 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:02.379 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939024+0000", 2024-09-21T15:01:02.379 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1569999999999999e-06 2024-09-21T15:01:02.380 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.380 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.380 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:02.380 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939034+0000", 2024-09-21T15:01:02.380 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9219999999999992e-06 2024-09-21T15:01:02.380 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.380 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.380 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:02.380 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.960408+0000", 2024-09-21T15:01:02.380 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021373792999999999 2024-09-21T15:01:02.380 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.380 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.380 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:02.381 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.960482+0000", 2024-09-21T15:01:02.381 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4331000000000004e-05 2024-09-21T15:01:02.381 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.381 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.381 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:02.381 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.960738+0000", 2024-09-21T15:01:02.381 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00025600500000000001 2024-09-21T15:01:02.381 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:02.381 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:02.381 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:02.381 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.381 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.381 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:134711 3.a 3:51eaadc0:::rbd_data.11365ed419e3.0000000000000007:67 [read 270336~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:02.381 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.939039+0000", 2024-09-21T15:01:02.381 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 8.3788229219999995, 2024-09-21T15:01:02.382 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022064698000000001, 2024-09-21T15:01:02.382 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:02.382 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:02.382 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:02.382 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:02.382 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:02.382 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 134711 2024-09-21T15:01:02.382 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.382 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:02.382 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.382 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:02.382 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939039+0000", 2024-09-21T15:01:02.382 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:02.383 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.383 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.383 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:02.383 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939039+0000", 2024-09-21T15:01:02.383 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:02.383 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.383 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.383 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:02.383 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939038+0000", 2024-09-21T15:01:02.383 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:02.383 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.383 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.384 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:02.384 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939041+0000", 2024-09-21T15:01:02.384 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9440000000000001e-06 2024-09-21T15:01:02.384 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.384 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.384 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:02.384 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939047+0000", 2024-09-21T15:01:02.384 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1260000000000003e-06 2024-09-21T15:01:02.384 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.384 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.384 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:02.384 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939052+0000", 2024-09-21T15:01:02.384 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1619999999999999e-06 2024-09-21T15:01:02.385 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.385 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.385 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:02.385 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.960768+0000", 2024-09-21T15:01:02.385 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021715140000000001 2024-09-21T15:01:02.385 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.385 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.385 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:02.385 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.960842+0000", 2024-09-21T15:01:02.385 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4698999999999997e-05 2024-09-21T15:01:02.385 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.385 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.385 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:02.385 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.961103+0000", 2024-09-21T15:01:02.386 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00026098499999999998 2024-09-21T15:01:02.386 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:02.386 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:02.386 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:02.386 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.386 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.386 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:134712 3.a 3:51eaadc0:::rbd_data.11365ed419e3.0000000000000007:67 [read 286720~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:02.386 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.939057+0000", 2024-09-21T15:01:02.386 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 8.3788043890000008, 2024-09-21T15:01:02.386 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022387067, 2024-09-21T15:01:02.386 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:02.386 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:02.386 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:02.386 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:02.387 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:02.387 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 134712 2024-09-21T15:01:02.387 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.387 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:02.387 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.387 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:02.387 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939057+0000", 2024-09-21T15:01:02.387 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:02.387 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.387 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.387 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:02.387 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939057+0000", 2024-09-21T15:01:02.387 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:02.387 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.388 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.388 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:02.388 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939057+0000", 2024-09-21T15:01:02.388 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:02.388 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.388 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.388 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:02.388 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939059+0000", 2024-09-21T15:01:02.388 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.447e-06 2024-09-21T15:01:02.388 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.388 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.388 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:02.388 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939065+0000", 2024-09-21T15:01:02.389 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4670000000000003e-06 2024-09-21T15:01:02.389 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.389 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.389 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:02.389 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939069+0000", 2024-09-21T15:01:02.389 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2119999999999997e-06 2024-09-21T15:01:02.389 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.389 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.389 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:02.389 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.961141+0000", 2024-09-21T15:01:02.389 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022072268999999999 2024-09-21T15:01:02.389 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.389 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.389 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:02.389 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.961240+0000", 2024-09-21T15:01:02.389 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9147999999999993e-05 2024-09-21T15:01:02.390 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.390 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.390 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:02.390 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.961444+0000", 2024-09-21T15:01:02.390 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00020381899999999999 2024-09-21T15:01:02.390 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:02.390 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:02.390 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:02.390 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.390 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.390 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:134713 3.a 3:51eaadc0:::rbd_data.11365ed419e3.0000000000000007:67 [read 303104~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:02.390 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.939073+0000", 2024-09-21T15:01:02.390 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 8.3787884859999995, 2024-09-21T15:01:02.390 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022687394999999999, 2024-09-21T15:01:02.391 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:02.391 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:02.391 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:02.391 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:02.391 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:02.391 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 134713 2024-09-21T15:01:02.391 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.391 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:02.391 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.391 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:02.391 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939073+0000", 2024-09-21T15:01:02.391 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:02.391 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.391 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.392 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:02.392 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939073+0000", 2024-09-21T15:01:02.392 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:02.392 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.392 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.392 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:02.392 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939073+0000", 2024-09-21T15:01:02.392 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:02.392 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.392 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.392 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:02.392 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939075+0000", 2024-09-21T15:01:02.392 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5969999999999999e-06 2024-09-21T15:01:02.392 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.392 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.393 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:02.393 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939081+0000", 2024-09-21T15:01:02.393 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.609e-06 2024-09-21T15:01:02.393 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.393 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.393 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:02.393 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939086+0000", 2024-09-21T15:01:02.393 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.916e-06 2024-09-21T15:01:02.393 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.393 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.393 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:02.393 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.961474+0000", 2024-09-21T15:01:02.393 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022388512999999999 2024-09-21T15:01:02.393 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.393 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.394 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:02.394 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.961567+0000", 2024-09-21T15:01:02.394 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2267000000000003e-05 2024-09-21T15:01:02.394 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.394 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.394 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:02.394 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.961760+0000", 2024-09-21T15:01:02.394 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00019375999999999999 2024-09-21T15:01:02.394 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:02.394 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:02.394 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:02.394 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.394 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.394 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:134714 3.a 3:51eaadc0:::rbd_data.11365ed419e3.0000000000000007:67 [read 319488~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:02.395 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.939090+0000", 2024-09-21T15:01:02.395 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 8.3787716589999999, 2024-09-21T15:01:02.395 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023043403, 2024-09-21T15:01:02.395 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:02.395 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:02.395 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:02.395 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:02.395 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:02.395 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 134714 2024-09-21T15:01:02.395 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.395 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:02.395 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.395 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:02.395 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939090+0000", 2024-09-21T15:01:02.395 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:02.396 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.396 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.396 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:02.396 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939090+0000", 2024-09-21T15:01:02.396 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:02.396 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.396 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.396 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:02.396 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939090+0000", 2024-09-21T15:01:02.396 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:02.396 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.396 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.396 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:02.396 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939092+0000", 2024-09-21T15:01:02.396 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2790000000000001e-06 2024-09-21T15:01:02.397 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.397 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.397 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:02.397 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939097+0000", 2024-09-21T15:01:02.397 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1410000000000004e-06 2024-09-21T15:01:02.397 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.397 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.397 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:02.397 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939102+0000", 2024-09-21T15:01:02.397 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7369999999999997e-06 2024-09-21T15:01:02.397 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.397 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.397 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:02.397 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.961791+0000", 2024-09-21T15:01:02.397 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022688974000000001 2024-09-21T15:01:02.397 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.398 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.398 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:02.398 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.961865+0000", 2024-09-21T15:01:02.398 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4651000000000001e-05 2024-09-21T15:01:02.398 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.398 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.398 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:02.398 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.962133+0000", 2024-09-21T15:01:02.398 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00026793999999999999 2024-09-21T15:01:02.398 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:02.398 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:02.398 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:02.398 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.398 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.399 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:134715 3.a 3:51eaadc0:::rbd_data.11365ed419e3.0000000000000007:67 [read 335872~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:02.399 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.939106+0000", 2024-09-21T15:01:02.399 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 8.3787550490000005, 2024-09-21T15:01:02.399 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023401142999999999, 2024-09-21T15:01:02.399 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:02.399 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:02.399 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:02.399 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:02.399 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:02.399 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 134715 2024-09-21T15:01:02.399 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.399 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:02.399 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.399 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:02.399 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939106+0000", 2024-09-21T15:01:02.399 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:02.400 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.400 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.400 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:02.400 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939106+0000", 2024-09-21T15:01:02.400 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:02.400 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.400 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.400 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:02.400 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939106+0000", 2024-09-21T15:01:02.400 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:02.400 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.400 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.400 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:02.400 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939114+0000", 2024-09-21T15:01:02.400 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1009999999999992e-06 2024-09-21T15:01:02.400 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.401 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.401 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:02.401 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939121+0000", 2024-09-21T15:01:02.401 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7710000000000001e-06 2024-09-21T15:01:02.401 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.401 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.401 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:02.401 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939126+0000", 2024-09-21T15:01:02.401 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3600000000000004e-06 2024-09-21T15:01:02.401 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.401 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.401 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:02.401 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.962162+0000", 2024-09-21T15:01:02.401 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023036148999999999 2024-09-21T15:01:02.401 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.401 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.402 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:02.402 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.962239+0000", 2024-09-21T15:01:02.402 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6874999999999998e-05 2024-09-21T15:01:02.402 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.402 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.402 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:02.402 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.962508+0000", 2024-09-21T15:01:02.402 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00026842199999999999 2024-09-21T15:01:02.402 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:02.402 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:02.402 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:02.402 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.402 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.403 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:134716 3.a 3:51eaadc0:::rbd_data.11365ed419e3.0000000000000007:67 [read 352256~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:02.403 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.939131+0000", 2024-09-21T15:01:02.403 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 8.3787309150000002, 2024-09-21T15:01:02.403 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023742441, 2024-09-21T15:01:02.403 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:02.403 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:02.403 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:02.403 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:02.403 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:02.403 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 134716 2024-09-21T15:01:02.403 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.403 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:02.403 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.403 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:02.403 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939131+0000", 2024-09-21T15:01:02.403 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:02.404 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.404 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.404 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:02.404 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939131+0000", 2024-09-21T15:01:02.404 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:02.404 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.404 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.404 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:02.404 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939130+0000", 2024-09-21T15:01:02.404 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:02.404 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.404 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.404 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:02.404 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939133+0000", 2024-09-21T15:01:02.404 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5620000000000002e-06 2024-09-21T15:01:02.405 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.405 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.405 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:02.405 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939138+0000", 2024-09-21T15:01:02.405 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.322e-06 2024-09-21T15:01:02.405 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.405 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.405 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:02.405 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939143+0000", 2024-09-21T15:01:02.405 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5510000000000001e-06 2024-09-21T15:01:02.405 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.405 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.405 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:02.405 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.962537+0000", 2024-09-21T15:01:02.405 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023394808 2024-09-21T15:01:02.405 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.406 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.406 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:02.406 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.962608+0000", 2024-09-21T15:01:02.406 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0699999999999997e-05 2024-09-21T15:01:02.406 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.406 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.406 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:02.406 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.962873+0000", 2024-09-21T15:01:02.406 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00026487 2024-09-21T15:01:02.406 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:02.406 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:02.406 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:02.406 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.406 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.406 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:135024 3.a 3:53261383:::rbd_data.11365ed419e3.0000000000000037:snapdir [list-snaps] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:02.406 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.940162+0000", 2024-09-21T15:01:02.407 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 8.3776999520000004, 2024-09-21T15:01:02.407 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022852172, 2024-09-21T15:01:02.407 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:02.407 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:02.407 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:02.407 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:02.407 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:02.407 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 135024 2024-09-21T15:01:02.407 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.407 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:02.407 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.407 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:02.407 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940162+0000", 2024-09-21T15:01:02.407 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:02.407 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.407 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.408 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:02.408 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940162+0000", 2024-09-21T15:01:02.408 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:02.408 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.408 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.408 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:02.408 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940161+0000", 2024-09-21T15:01:02.408 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:02.408 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.408 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.408 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:02.408 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940164+0000", 2024-09-21T15:01:02.408 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4099999999999998e-06 2024-09-21T15:01:02.408 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.408 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.408 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:02.409 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940169+0000", 2024-09-21T15:01:02.409 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3720000000000004e-06 2024-09-21T15:01:02.409 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.409 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.409 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:02.409 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940174+0000", 2024-09-21T15:01:02.409 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.724e-06 2024-09-21T15:01:02.409 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.409 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.409 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:02.409 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.962902+0000", 2024-09-21T15:01:02.409 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022728719000000001 2024-09-21T15:01:02.409 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.409 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.409 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:02.409 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.963014+0000", 2024-09-21T15:01:02.410 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000111245 2024-09-21T15:01:02.410 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:02.410 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:02.410 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:02.410 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.410 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.410 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:135044 3.a 3:53261383:::rbd_data.11365ed419e3.0000000000000037:snapdir [list-snaps] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:02.410 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.940484+0000", 2024-09-21T15:01:02.410 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 8.3773772199999996, 2024-09-21T15:01:02.410 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022619169000000001, 2024-09-21T15:01:02.410 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:02.410 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:02.410 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:02.410 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:02.410 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:02.410 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 135044 2024-09-21T15:01:02.411 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.411 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:02.411 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.411 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:02.411 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940484+0000", 2024-09-21T15:01:02.411 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:02.411 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.411 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.411 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:02.411 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940484+0000", 2024-09-21T15:01:02.411 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:02.411 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.411 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.411 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:02.411 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940484+0000", 2024-09-21T15:01:02.411 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:02.412 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.412 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.412 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:02.412 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940487+0000", 2024-09-21T15:01:02.412 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8449999999999999e-06 2024-09-21T15:01:02.412 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.412 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.412 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:02.412 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940493+0000", 2024-09-21T15:01:02.412 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9599999999999997e-06 2024-09-21T15:01:02.412 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.412 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.412 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:02.412 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940497+0000", 2024-09-21T15:01:02.412 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9620000000000001e-06 2024-09-21T15:01:02.412 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.413 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.413 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:02.413 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.963034+0000", 2024-09-21T15:01:02.413 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022536878 2024-09-21T15:01:02.413 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.413 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.413 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:02.413 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.963103+0000", 2024-09-21T15:01:02.413 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9073999999999996e-05 2024-09-21T15:01:02.413 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:02.413 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:02.413 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:02.413 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:02.413 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:02.413 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:02.415 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T15:01:02.415 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:01:02.413+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:01:02.516 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:01:02.511+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:01:02.526 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:02.526 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:01:02.526 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T15:01:02.526 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:02.536 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T15:01:02.617 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:01:02.611+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:01:02.640 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:02.641 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:01:02.641 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T15:01:02.641 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T15:01:02.641 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:02.650 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T15:01:02.717 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:01:02.711+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:01:02.806 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:02.807 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T15:01:02.807 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T15:01:02.807 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T15:01:02.807 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.807 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:15130 3.2 3:47f1bbdb:::rbd_data.11361bed1f78.0000000000000067:head [sparse-read 0~147456] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-21T15:01:02.807 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:40.288284+0000", 2024-09-21T15:01:02.807 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 142.516753002, 2024-09-21T15:01:02.807 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1142935039999999, 2024-09-21T15:01:02.807 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:02.807 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:02.807 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:02.807 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:02.807 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:02.808 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 15130 2024-09-21T15:01:02.808 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.808 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:02.808 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.808 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:02.808 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288284+0000", 2024-09-21T15:01:02.808 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:02.808 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.808 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.808 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:02.808 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288284+0000", 2024-09-21T15:01:02.808 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:02.808 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.808 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.809 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:02.809 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288284+0000", 2024-09-21T15:01:02.809 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:02.809 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.809 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.809 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:02.809 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288288+0000", 2024-09-21T15:01:02.809 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5829999999999998e-06 2024-09-21T15:01:02.809 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.809 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.809 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:02.809 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288297+0000", 2024-09-21T15:01:02.809 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0669999999999998e-06 2024-09-21T15:01:02.809 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.810 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.810 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:02.810 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288305+0000", 2024-09-21T15:01:02.810 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8860000000000004e-06 2024-09-21T15:01:02.810 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.810 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.810 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:02.810 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288417+0000", 2024-09-21T15:01:02.810 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000112952 2024-09-21T15:01:02.810 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.810 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.810 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:02.811 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288428+0000", 2024-09-21T15:01:02.811 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0631e-05 2024-09-21T15:01:02.811 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.811 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.811 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:02.811 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.401876+0000", 2024-09-21T15:01:02.811 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1134479610000001 2024-09-21T15:01:02.811 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.811 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.811 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:02.811 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.402076+0000", 2024-09-21T15:01:02.811 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00019944200000000001 2024-09-21T15:01:02.811 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.811 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.812 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:02.812 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.402578+0000", 2024-09-21T15:01:02.812 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00050242899999999998 2024-09-21T15:01:02.812 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:02.812 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:02.812 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:02.812 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.812 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.812 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45708 3.2 3:415db91a:::rbd_data.1136ecbe5879.0000000000000000:28 [read 140288~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:02.812 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141801+0000", 2024-09-21T15:01:02.812 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 112.66323680799999, 2024-09-21T15:01:02.812 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034274200999999997, 2024-09-21T15:01:02.812 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:02.812 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:02.812 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:02.813 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:02.813 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:02.813 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45708 2024-09-21T15:01:02.813 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.813 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:02.813 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.813 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:02.813 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141801+0000", 2024-09-21T15:01:02.813 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:02.813 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.813 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.813 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:02.813 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141801+0000", 2024-09-21T15:01:02.814 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:02.814 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.814 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.814 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:02.814 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141800+0000", 2024-09-21T15:01:02.814 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:02.814 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.814 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.814 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:02.814 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141803+0000", 2024-09-21T15:01:02.814 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4210000000000002e-06 2024-09-21T15:01:02.814 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.814 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.814 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:02.815 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141812+0000", 2024-09-21T15:01:02.815 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.9800000000000004e-06 2024-09-21T15:01:02.815 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.815 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.815 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:02.815 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141816+0000", 2024-09-21T15:01:02.815 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4769999999999997e-06 2024-09-21T15:01:02.815 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.815 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.815 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:02.815 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.175865+0000", 2024-09-21T15:01:02.815 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034048875999999999 2024-09-21T15:01:02.815 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.815 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.815 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:02.816 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.175943+0000", 2024-09-21T15:01:02.816 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7464000000000002e-05 2024-09-21T15:01:02.816 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.816 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.816 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:02.816 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176075+0000", 2024-09-21T15:01:02.816 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013230800000000001 2024-09-21T15:01:02.816 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:02.816 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:02.816 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:02.816 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.816 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.816 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45709 3.2 3:415db91a:::rbd_data.1136ecbe5879.0000000000000000:28 [read 143360~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:02.816 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141820+0000", 2024-09-21T15:01:02.816 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 112.663216961, 2024-09-21T15:01:02.816 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034537536000000001, 2024-09-21T15:01:02.817 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:02.817 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:02.817 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:02.817 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:02.817 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:02.817 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45709 2024-09-21T15:01:02.817 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.817 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:02.817 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.817 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:02.817 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141820+0000", 2024-09-21T15:01:02.817 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:02.817 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.817 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.817 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:02.818 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141820+0000", 2024-09-21T15:01:02.818 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:02.818 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.818 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.818 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:02.818 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141820+0000", 2024-09-21T15:01:02.818 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:02.818 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.818 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.818 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:02.818 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141823+0000", 2024-09-21T15:01:02.818 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5110000000000002e-06 2024-09-21T15:01:02.818 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.818 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.818 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:02.818 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141829+0000", 2024-09-21T15:01:02.819 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0850000000000002e-06 2024-09-21T15:01:02.819 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.819 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.819 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:02.819 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141833+0000", 2024-09-21T15:01:02.819 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7620000000000003e-06 2024-09-21T15:01:02.819 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.819 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.819 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:02.819 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176103+0000", 2024-09-21T15:01:02.819 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034269933000000002 2024-09-21T15:01:02.819 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.819 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.819 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:02.819 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176161+0000", 2024-09-21T15:01:02.819 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7658999999999998e-05 2024-09-21T15:01:02.820 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.820 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.820 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:02.820 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176358+0000", 2024-09-21T15:01:02.820 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000196937 2024-09-21T15:01:02.820 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:02.820 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:02.820 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:02.820 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.820 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.820 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45710 3.2 3:415db91a:::rbd_data.1136ecbe5879.0000000000000000:28 [read 146432~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:02.820 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141838+0000", 2024-09-21T15:01:02.820 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 112.66319979799999, 2024-09-21T15:01:02.820 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034953674999999997, 2024-09-21T15:01:02.820 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:02.820 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:02.821 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:02.821 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:02.821 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:02.821 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45710 2024-09-21T15:01:02.821 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.821 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:02.821 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.821 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:02.821 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141838+0000", 2024-09-21T15:01:02.821 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:02.821 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.821 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.821 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:02.821 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141838+0000", 2024-09-21T15:01:02.821 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:02.821 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.822 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.822 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:02.822 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141837+0000", 2024-09-21T15:01:02.822 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:02.822 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.822 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.822 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:02.822 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141843+0000", 2024-09-21T15:01:02.822 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1619999999999996e-06 2024-09-21T15:01:02.822 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.822 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.822 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:02.822 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141850+0000", 2024-09-21T15:01:02.822 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4799999999999998e-06 2024-09-21T15:01:02.822 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.823 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.823 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:02.823 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141853+0000", 2024-09-21T15:01:02.823 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3050000000000001e-06 2024-09-21T15:01:02.823 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.823 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.823 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:02.823 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176490+0000", 2024-09-21T15:01:02.823 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034636413999999997 2024-09-21T15:01:02.823 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.823 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.823 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:02.823 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176606+0000", 2024-09-21T15:01:02.823 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000115841 2024-09-21T15:01:02.823 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.824 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.824 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:02.824 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176791+0000", 2024-09-21T15:01:02.824 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00018581500000000001 2024-09-21T15:01:02.824 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:02.824 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:02.824 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:02.824 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.824 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.824 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45711 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:02.824 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141856+0000", 2024-09-21T15:01:02.824 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 112.66318146099999, 2024-09-21T15:01:02.824 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034986308000000001, 2024-09-21T15:01:02.824 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:02.824 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:02.825 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:02.825 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:02.825 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:02.825 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45711 2024-09-21T15:01:02.825 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.825 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:02.825 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.825 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:02.825 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141856+0000", 2024-09-21T15:01:02.825 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:02.825 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.825 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.825 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:02.825 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141856+0000", 2024-09-21T15:01:02.825 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:02.825 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.826 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.826 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:02.826 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141856+0000", 2024-09-21T15:01:02.826 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:02.826 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.826 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.826 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:02.826 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141862+0000", 2024-09-21T15:01:02.826 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.567e-06 2024-09-21T15:01:02.826 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.826 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.826 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:02.826 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141868+0000", 2024-09-21T15:01:02.826 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9569999999999999e-06 2024-09-21T15:01:02.826 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.826 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.827 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:02.827 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141871+0000", 2024-09-21T15:01:02.827 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.146e-06 2024-09-21T15:01:02.827 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.827 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.827 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:02.827 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176831+0000", 2024-09-21T15:01:02.827 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034959935999999997 2024-09-21T15:01:02.827 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.827 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.827 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:02.827 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176842+0000", 2024-09-21T15:01:02.827 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1008e-05 2024-09-21T15:01:02.827 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:02.827 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:02.827 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:02.828 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.828 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.828 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45712 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:02.828 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141874+0000", 2024-09-21T15:01:02.828 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 112.66316356599999, 2024-09-21T15:01:02.828 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034993511999999997, 2024-09-21T15:01:02.828 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:02.828 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:02.828 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:02.828 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:02.828 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:02.828 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45712 2024-09-21T15:01:02.828 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.828 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:02.828 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.828 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:02.829 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141874+0000", 2024-09-21T15:01:02.829 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:02.829 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.829 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.829 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:02.829 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141874+0000", 2024-09-21T15:01:02.829 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:02.829 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.829 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.829 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:02.829 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141874+0000", 2024-09-21T15:01:02.829 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:02.829 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.829 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.829 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:02.829 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141875+0000", 2024-09-21T15:01:02.830 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5430000000000001e-06 2024-09-21T15:01:02.830 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.830 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.830 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:02.830 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141878+0000", 2024-09-21T15:01:02.830 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1420000000000002e-06 2024-09-21T15:01:02.830 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.830 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.830 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:02.830 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141881+0000", 2024-09-21T15:01:02.830 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.807e-06 2024-09-21T15:01:02.830 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.830 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.830 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:02.830 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176860+0000", 2024-09-21T15:01:02.831 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034979243 2024-09-21T15:01:02.831 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.831 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.831 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:02.831 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176867+0000", 2024-09-21T15:01:02.831 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0169999999999999e-06 2024-09-21T15:01:02.831 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:02.831 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:02.831 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:02.831 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.831 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.831 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45713 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:02.831 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141883+0000", 2024-09-21T15:01:02.831 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 112.663153991, 2024-09-21T15:01:02.832 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035010516999999998, 2024-09-21T15:01:02.832 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:02.832 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:02.832 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:02.832 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:02.832 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:02.832 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45713 2024-09-21T15:01:02.832 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.832 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:02.832 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.832 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:02.832 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141883+0000", 2024-09-21T15:01:02.832 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:02.832 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.832 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.832 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:02.833 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141883+0000", 2024-09-21T15:01:02.833 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:02.833 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.833 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.833 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:02.833 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141883+0000", 2024-09-21T15:01:02.833 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:02.833 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.833 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.833 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:02.833 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141885+0000", 2024-09-21T15:01:02.833 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4649999999999999e-06 2024-09-21T15:01:02.833 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.833 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.833 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:02.833 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141888+0000", 2024-09-21T15:01:02.834 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3890000000000001e-06 2024-09-21T15:01:02.834 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.834 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.834 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:02.834 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141891+0000", 2024-09-21T15:01:02.834 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9239999999999999e-06 2024-09-21T15:01:02.834 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.834 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.834 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:02.834 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176887+0000", 2024-09-21T15:01:02.834 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034996370999999998 2024-09-21T15:01:02.834 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.834 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.834 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:02.834 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176894+0000", 2024-09-21T15:01:02.835 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6069999999999996e-06 2024-09-21T15:01:02.835 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:02.835 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:02.835 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:02.835 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.835 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.835 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45714 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:02.835 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141893+0000", 2024-09-21T15:01:02.835 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 112.66314407599999, 2024-09-21T15:01:02.835 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035048215000000001, 2024-09-21T15:01:02.835 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:02.835 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:02.835 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:02.835 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:02.835 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:02.835 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45714 2024-09-21T15:01:02.835 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.836 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:02.836 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.836 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:02.836 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141893+0000", 2024-09-21T15:01:02.836 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:02.836 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.836 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.836 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:02.836 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141893+0000", 2024-09-21T15:01:02.836 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:02.836 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.836 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.836 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:02.836 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141893+0000", 2024-09-21T15:01:02.836 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:02.836 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.837 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.837 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:02.837 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141895+0000", 2024-09-21T15:01:02.837 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.703e-06 2024-09-21T15:01:02.837 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.837 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.837 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:02.837 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141900+0000", 2024-09-21T15:01:02.837 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.242e-06 2024-09-21T15:01:02.837 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.837 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.837 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:02.837 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141903+0000", 2024-09-21T15:01:02.837 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.117e-06 2024-09-21T15:01:02.837 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.837 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.838 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:02.838 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176930+0000", 2024-09-21T15:01:02.838 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035026750000000002 2024-09-21T15:01:02.838 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.838 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.838 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:02.838 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176942+0000", 2024-09-21T15:01:02.838 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1668000000000001e-05 2024-09-21T15:01:02.838 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:02.838 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:02.838 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:02.838 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.838 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.838 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45715 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:02.838 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141906+0000", 2024-09-21T15:01:02.838 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 112.663131918, 2024-09-21T15:01:02.839 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035066057999999997, 2024-09-21T15:01:02.839 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:02.839 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:02.839 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:02.839 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:02.839 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:02.839 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45715 2024-09-21T15:01:02.839 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.839 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:02.839 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.839 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:02.839 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141906+0000", 2024-09-21T15:01:02.839 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:02.839 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.839 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.839 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:02.840 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141906+0000", 2024-09-21T15:01:02.840 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:02.840 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.840 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.840 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:02.840 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141905+0000", 2024-09-21T15:01:02.840 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:02.840 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.840 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.840 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:02.840 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141907+0000", 2024-09-21T15:01:02.840 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5400000000000001e-06 2024-09-21T15:01:02.840 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.840 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.840 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:02.840 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141912+0000", 2024-09-21T15:01:02.841 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.502e-06 2024-09-21T15:01:02.841 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.841 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.841 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:02.841 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141915+0000", 2024-09-21T15:01:02.841 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0170000000000001e-06 2024-09-21T15:01:02.841 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.841 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.841 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:02.841 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176964+0000", 2024-09-21T15:01:02.841 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035048587999999999 2024-09-21T15:01:02.841 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.841 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.841 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:02.841 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176972+0000", 2024-09-21T15:01:02.841 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6580000000000002e-06 2024-09-21T15:01:02.842 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:02.842 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:02.842 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:02.842 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.842 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.842 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45716 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:02.842 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141918+0000", 2024-09-21T15:01:02.842 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 112.66311974200001, 2024-09-21T15:01:02.842 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035086223, 2024-09-21T15:01:02.842 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:02.842 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:02.842 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:02.842 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:02.842 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:02.842 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45716 2024-09-21T15:01:02.842 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.843 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:02.843 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.843 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:02.843 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141918+0000", 2024-09-21T15:01:02.843 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:02.843 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.843 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.843 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:02.843 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141918+0000", 2024-09-21T15:01:02.843 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:02.843 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.843 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.843 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:02.843 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141917+0000", 2024-09-21T15:01:02.843 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:02.844 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.844 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.844 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:02.844 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141919+0000", 2024-09-21T15:01:02.844 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5680000000000001e-06 2024-09-21T15:01:02.844 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.844 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.844 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:02.844 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141922+0000", 2024-09-21T15:01:02.844 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2789999999999999e-06 2024-09-21T15:01:02.844 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.844 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.844 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:02.844 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141925+0000", 2024-09-21T15:01:02.844 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9469999999999999e-06 2024-09-21T15:01:02.844 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.845 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.845 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:02.845 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176997+0000", 2024-09-21T15:01:02.845 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035071366999999999 2024-09-21T15:01:02.845 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.845 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.845 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:02.845 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177004+0000", 2024-09-21T15:01:02.845 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3309999999999998e-06 2024-09-21T15:01:02.845 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:02.845 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:02.845 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:02.845 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.845 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.845 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45717 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:02.846 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141928+0000", 2024-09-21T15:01:02.846 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 112.663109796, 2024-09-21T15:01:02.846 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035102573999999998, 2024-09-21T15:01:02.846 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:02.846 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:02.846 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:02.846 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:02.846 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:02.846 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45717 2024-09-21T15:01:02.846 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.846 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:02.846 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.846 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:02.846 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141928+0000", 2024-09-21T15:01:02.846 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:02.846 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.847 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.847 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:02.847 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141928+0000", 2024-09-21T15:01:02.847 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:02.847 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.847 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.847 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:02.847 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141927+0000", 2024-09-21T15:01:02.847 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:01:02.847 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.847 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.847 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:02.847 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141929+0000", 2024-09-21T15:01:02.847 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.533e-06 2024-09-21T15:01:02.847 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.847 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.848 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:02.848 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141934+0000", 2024-09-21T15:01:02.848 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1660000000000002e-06 2024-09-21T15:01:02.848 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.848 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.848 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:02.848 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141937+0000", 2024-09-21T15:01:02.848 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0309999999999999e-06 2024-09-21T15:01:02.848 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.848 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.848 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:02.848 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177023+0000", 2024-09-21T15:01:02.848 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035085595999999997 2024-09-21T15:01:02.848 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.848 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.849 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:02.849 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177030+0000", 2024-09-21T15:01:02.849 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.486e-06 2024-09-21T15:01:02.849 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:02.849 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:02.849 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:02.849 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.849 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.849 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45718 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:02.849 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141940+0000", 2024-09-21T15:01:02.849 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 112.66309793800001, 2024-09-21T15:01:02.849 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035124956999999998, 2024-09-21T15:01:02.849 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:02.849 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:02.850 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:02.850 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:02.850 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:02.850 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45718 2024-09-21T15:01:02.850 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.850 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:02.850 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.850 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:02.850 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141940+0000", 2024-09-21T15:01:02.850 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:02.850 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.850 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.850 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:02.850 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141940+0000", 2024-09-21T15:01:02.850 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:02.850 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.851 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.851 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:02.851 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141939+0000", 2024-09-21T15:01:02.851 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:02.851 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.851 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.851 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:02.851 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141941+0000", 2024-09-21T15:01:02.851 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.502e-06 2024-09-21T15:01:02.851 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.851 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.851 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:02.851 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141946+0000", 2024-09-21T15:01:02.851 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3449999999999996e-06 2024-09-21T15:01:02.852 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.852 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.852 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:02.852 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141949+0000", 2024-09-21T15:01:02.852 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9749999999999999e-06 2024-09-21T15:01:02.852 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.852 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.852 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:02.852 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177057+0000", 2024-09-21T15:01:02.852 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035108385999999998 2024-09-21T15:01:02.852 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.852 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.852 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:02.852 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177064+0000", 2024-09-21T15:01:02.852 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0099999999999998e-06 2024-09-21T15:01:02.853 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:02.853 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:02.853 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:02.853 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.853 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.853 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45719 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:02.853 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141952+0000", 2024-09-21T15:01:02.853 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 112.663085665, 2024-09-21T15:01:02.853 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035139108000000002, 2024-09-21T15:01:02.853 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:02.853 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:02.853 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:02.853 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:02.853 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:02.854 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45719 2024-09-21T15:01:02.854 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.854 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:02.854 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.854 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:02.854 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141952+0000", 2024-09-21T15:01:02.854 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:02.854 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.854 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.854 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:02.854 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141952+0000", 2024-09-21T15:01:02.854 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:02.854 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.854 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.855 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:02.855 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141952+0000", 2024-09-21T15:01:02.855 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:02.855 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.855 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.855 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:02.855 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141953+0000", 2024-09-21T15:01:02.855 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.486e-06 2024-09-21T15:01:02.855 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.855 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.855 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:02.855 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141959+0000", 2024-09-21T15:01:02.855 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4940000000000003e-06 2024-09-21T15:01:02.855 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.855 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.855 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:02.856 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141963+0000", 2024-09-21T15:01:02.856 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3540000000000002e-06 2024-09-21T15:01:02.856 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.856 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.856 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:02.856 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177084+0000", 2024-09-21T15:01:02.856 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035120983000000001 2024-09-21T15:01:02.856 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.856 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.856 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:02.856 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177091+0000", 2024-09-21T15:01:02.856 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0350000000000004e-06 2024-09-21T15:01:02.856 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:02.856 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:02.856 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:02.856 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.856 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.857 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45720 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:02.857 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141965+0000", 2024-09-21T15:01:02.857 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 112.66307219799999, 2024-09-21T15:01:02.857 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035155268000000003, 2024-09-21T15:01:02.857 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:02.857 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:02.857 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:02.857 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:02.857 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:02.857 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45720 2024-09-21T15:01:02.857 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.857 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:02.857 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.857 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:02.857 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141965+0000", 2024-09-21T15:01:02.858 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:02.858 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.858 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.858 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:02.858 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141965+0000", 2024-09-21T15:01:02.858 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:02.858 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.858 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.858 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:02.858 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141965+0000", 2024-09-21T15:01:02.858 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:02.858 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.858 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.858 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:02.858 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141966+0000", 2024-09-21T15:01:02.859 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.46e-06 2024-09-21T15:01:02.859 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.859 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.859 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:02.859 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141972+0000", 2024-09-21T15:01:02.859 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4210000000000003e-06 2024-09-21T15:01:02.859 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.859 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.859 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:02.859 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141975+0000", 2024-09-21T15:01:02.859 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.948e-06 2024-09-21T15:01:02.859 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.859 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.860 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:02.860 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177113+0000", 2024-09-21T15:01:02.860 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035137855000000003 2024-09-21T15:01:02.860 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.860 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.860 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:02.860 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177121+0000", 2024-09-21T15:01:02.860 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8269999999999998e-06 2024-09-21T15:01:02.860 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:02.860 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:02.860 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:02.860 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.860 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.860 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45721 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:02.860 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141977+0000", 2024-09-21T15:01:02.860 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 112.663060175, 2024-09-21T15:01:02.861 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035169984000000001, 2024-09-21T15:01:02.861 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:02.861 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:02.861 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:02.861 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:02.861 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:02.861 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45721 2024-09-21T15:01:02.861 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.861 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:02.861 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.861 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:02.861 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141977+0000", 2024-09-21T15:01:02.861 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:02.861 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.861 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.862 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:02.862 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141977+0000", 2024-09-21T15:01:02.862 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:02.862 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.862 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.862 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:02.862 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141977+0000", 2024-09-21T15:01:02.862 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:02.862 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.862 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.862 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:02.862 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141979+0000", 2024-09-21T15:01:02.862 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.511e-06 2024-09-21T15:01:02.862 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.863 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.863 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:02.863 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141982+0000", 2024-09-21T15:01:02.863 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4180000000000001e-06 2024-09-21T15:01:02.863 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.863 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.863 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:02.863 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141985+0000", 2024-09-21T15:01:02.863 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9450000000000002e-06 2024-09-21T15:01:02.863 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.863 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.863 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:02.863 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177139+0000", 2024-09-21T15:01:02.863 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035154409999999997 2024-09-21T15:01:02.863 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.864 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.864 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:02.864 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177147+0000", 2024-09-21T15:01:02.864 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9570000000000002e-06 2024-09-21T15:01:02.864 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:02.864 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:02.864 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:02.864 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.864 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.864 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45722 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:02.864 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141987+0000", 2024-09-21T15:01:02.864 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 112.663050204, 2024-09-21T15:01:02.864 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035188492000000002, 2024-09-21T15:01:02.864 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:02.865 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:02.865 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:02.865 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:02.865 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:02.865 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45722 2024-09-21T15:01:02.865 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.865 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:02.865 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.865 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:02.865 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141987+0000", 2024-09-21T15:01:02.865 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:02.865 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.865 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.865 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:02.865 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141987+0000", 2024-09-21T15:01:02.865 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:02.866 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.866 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.866 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:02.866 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141987+0000", 2024-09-21T15:01:02.866 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:02.866 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.866 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.866 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:02.866 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141988+0000", 2024-09-21T15:01:02.866 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.463e-06 2024-09-21T15:01:02.866 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.866 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.866 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:02.866 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141994+0000", 2024-09-21T15:01:02.867 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.361e-06 2024-09-21T15:01:02.867 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.867 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.867 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:02.867 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141997+0000", 2024-09-21T15:01:02.867 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0359999999999998e-06 2024-09-21T15:01:02.867 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.867 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.867 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:02.867 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177169+0000", 2024-09-21T15:01:02.867 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035171651999999998 2024-09-21T15:01:02.867 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.867 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.867 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:02.867 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177176+0000", 2024-09-21T15:01:02.867 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2189999999999998e-06 2024-09-21T15:01:02.868 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:02.868 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:02.868 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:02.868 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.868 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.868 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45723 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:02.868 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141999+0000", 2024-09-21T15:01:02.868 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 112.66303803700001, 2024-09-21T15:01:02.868 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035197701999999997, 2024-09-21T15:01:02.868 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:02.868 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:02.868 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:02.868 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:02.868 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:02.868 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45723 2024-09-21T15:01:02.868 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.869 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:02.869 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.869 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:02.869 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141999+0000", 2024-09-21T15:01:02.869 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:02.869 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.869 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.869 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:02.869 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141999+0000", 2024-09-21T15:01:02.869 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:02.869 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.869 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.869 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:02.869 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141999+0000", 2024-09-21T15:01:02.869 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:02.869 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.870 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.870 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:02.870 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142003+0000", 2024-09-21T15:01:02.870 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5049999999999999e-06 2024-09-21T15:01:02.870 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.870 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.870 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:02.870 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142008+0000", 2024-09-21T15:01:02.870 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6579999999999999e-06 2024-09-21T15:01:02.870 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.870 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.870 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:02.870 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142011+0000", 2024-09-21T15:01:02.870 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0529999999999998e-06 2024-09-21T15:01:02.870 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.871 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.871 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:02.871 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177194+0000", 2024-09-21T15:01:02.871 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035182221999999999 2024-09-21T15:01:02.871 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.871 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.871 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:02.871 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177197+0000", 2024-09-21T15:01:02.871 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5030000000000002e-06 2024-09-21T15:01:02.871 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:02.871 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:02.871 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:02.871 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.871 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.871 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45724 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:02.872 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.142015+0000", 2024-09-21T15:01:02.872 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 112.66302207, 2024-09-21T15:01:02.872 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035195345000000003, 2024-09-21T15:01:02.872 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:02.872 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:02.872 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:02.872 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:02.872 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:02.872 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45724 2024-09-21T15:01:02.872 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.872 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:02.872 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.872 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:02.872 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142015+0000", 2024-09-21T15:01:02.872 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:02.873 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.873 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.873 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:02.873 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142015+0000", 2024-09-21T15:01:02.873 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:02.873 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.873 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.873 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:02.873 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142015+0000", 2024-09-21T15:01:02.873 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:02.873 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.873 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.873 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:02.873 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142017+0000", 2024-09-21T15:01:02.873 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5540000000000001e-06 2024-09-21T15:01:02.874 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.874 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.874 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:02.874 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142022+0000", 2024-09-21T15:01:02.874 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4229999999999996e-06 2024-09-21T15:01:02.874 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.874 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.874 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:02.874 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142025+0000", 2024-09-21T15:01:02.874 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.056e-06 2024-09-21T15:01:02.874 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.874 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.874 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:02.875 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177207+0000", 2024-09-21T15:01:02.875 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035182089 2024-09-21T15:01:02.875 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.875 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.875 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:02.875 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177211+0000", 2024-09-21T15:01:02.875 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4649999999999999e-06 2024-09-21T15:01:02.875 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:02.875 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:02.875 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:02.875 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.875 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.875 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45725 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:02.875 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.142028+0000", 2024-09-21T15:01:02.875 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 112.66300979099999, 2024-09-21T15:01:02.876 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035195731000000001, 2024-09-21T15:01:02.876 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:02.876 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:02.876 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:02.876 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:02.876 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:02.876 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45725 2024-09-21T15:01:02.876 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.876 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:02.876 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.876 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:02.876 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142028+0000", 2024-09-21T15:01:02.876 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:02.876 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.876 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.876 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:02.877 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142028+0000", 2024-09-21T15:01:02.877 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:02.877 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.877 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.877 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:02.877 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142027+0000", 2024-09-21T15:01:02.877 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:02.877 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.877 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.877 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:02.877 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142029+0000", 2024-09-21T15:01:02.877 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5269999999999999e-06 2024-09-21T15:01:02.877 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.877 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.877 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:02.877 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142032+0000", 2024-09-21T15:01:02.878 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3249999999999999e-06 2024-09-21T15:01:02.878 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.878 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.878 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:02.878 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142035+0000", 2024-09-21T15:01:02.878 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.897e-06 2024-09-21T15:01:02.878 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.878 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.878 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:02.878 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177220+0000", 2024-09-21T15:01:02.878 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035184871 2024-09-21T15:01:02.878 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.878 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.878 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:02.878 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177223+0000", 2024-09-21T15:01:02.878 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3680000000000002e-06 2024-09-21T15:01:02.879 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:02.879 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:02.879 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:02.879 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.879 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.879 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45726 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:02.879 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.142037+0000", 2024-09-21T15:01:02.879 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 112.663000076, 2024-09-21T15:01:02.879 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035199273000000003, 2024-09-21T15:01:02.879 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:02.879 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:02.879 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:02.879 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:02.879 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:02.879 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45726 2024-09-21T15:01:02.879 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.880 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:02.880 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.880 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:02.880 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142037+0000", 2024-09-21T15:01:02.880 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:02.880 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.880 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.880 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:02.880 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142037+0000", 2024-09-21T15:01:02.880 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:02.880 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.880 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.880 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:02.880 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142037+0000", 2024-09-21T15:01:02.881 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:02.881 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.881 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.881 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:02.881 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142038+0000", 2024-09-21T15:01:02.881 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.344e-06 2024-09-21T15:01:02.881 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.881 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.881 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:02.881 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142044+0000", 2024-09-21T15:01:02.881 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1930000000000001e-06 2024-09-21T15:01:02.881 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.881 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.881 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:02.881 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142047+0000", 2024-09-21T15:01:02.881 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0280000000000001e-06 2024-09-21T15:01:02.882 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.882 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.882 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:02.882 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177233+0000", 2024-09-21T15:01:02.882 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035186394000000003 2024-09-21T15:01:02.882 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:02.882 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:02.882 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:02.882 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177237+0000", 2024-09-21T15:01:02.882 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5530000000000001e-06 2024-09-21T15:01:02.882 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:02.882 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:02.882 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:02.882 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:02.882 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:02.882 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:02.884 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T15:01:02.884 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:01:02.881+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:01:02.984 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:01:02.981+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:01:03.002 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:03.002 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T15:01:03.002 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.002 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139776 3.16 3.470b1c36 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:03.002 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.223442+0000", 2024-09-21T15:01:03.003 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.77426267100000001, 2024-09-21T15:01:03.003 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.77430219700000003, 2024-09-21T15:01:03.003 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:03.003 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "waiting for peered", 2024-09-21T15:01:03.003 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:03.003 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:03.003 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:03.003 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139776 2024-09-21T15:01:03.003 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.003 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:03.003 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.003 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:03.003 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223442+0000", 2024-09-21T15:01:03.004 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.004 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.004 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.004 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:03.004 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223442+0000", 2024-09-21T15:01:03.004 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.004 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.004 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.004 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:03.004 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223442+0000", 2024-09-21T15:01:03.004 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:03.004 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.004 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.004 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:03.005 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223446+0000", 2024-09-21T15:01:03.005 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2549999999999999e-06 2024-09-21T15:01:03.005 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.005 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.005 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:03.005 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223454+0000", 2024-09-21T15:01:03.005 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8199999999999997e-06 2024-09-21T15:01:03.005 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.005 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.005 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:03.005 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223460+0000", 2024-09-21T15:01:03.005 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6330000000000001e-06 2024-09-21T15:01:03.005 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.005 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.005 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:03.006 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223542+0000", 2024-09-21T15:01:03.006 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1929999999999997e-05 2024-09-21T15:01:03.006 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.006 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.006 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:03.006 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223552+0000", 2024-09-21T15:01:03.006 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0627e-05 2024-09-21T15:01:03.006 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.006 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:03.006 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.006 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.006 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.006 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139913 3.16 3.b62d81f6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:03.006 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.227685+0000", 2024-09-21T15:01:03.007 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.77002023399999997, 2024-09-21T15:01:03.007 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.77017217400000004, 2024-09-21T15:01:03.007 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:03.007 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "waiting for peered", 2024-09-21T15:01:03.007 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:03.007 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:03.007 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:03.007 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139913 2024-09-21T15:01:03.007 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.007 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:03.007 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.007 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:03.007 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227685+0000", 2024-09-21T15:01:03.007 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.008 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.008 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.008 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:03.008 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227685+0000", 2024-09-21T15:01:03.008 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.008 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.008 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.008 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:03.008 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227685+0000", 2024-09-21T15:01:03.008 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:03.008 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.008 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.008 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:03.008 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227687+0000", 2024-09-21T15:01:03.008 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2929999999999999e-06 2024-09-21T15:01:03.009 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.009 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.009 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:03.009 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227693+0000", 2024-09-21T15:01:03.009 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1070000000000001e-06 2024-09-21T15:01:03.009 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.009 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.009 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:03.009 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227698+0000", 2024-09-21T15:01:03.009 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9339999999999997e-06 2024-09-21T15:01:03.009 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.009 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.009 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:03.009 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227784+0000", 2024-09-21T15:01:03.009 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6096999999999999e-05 2024-09-21T15:01:03.009 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.010 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.010 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:03.010 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227791+0000", 2024-09-21T15:01:03.010 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9090000000000003e-06 2024-09-21T15:01:03.010 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.010 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:03.010 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.010 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.010 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.010 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140332 3.16 3.24c08f96 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:03.010 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.242728+0000", 2024-09-21T15:01:03.010 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.75497711499999998, 2024-09-21T15:01:03.010 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.755224809, 2024-09-21T15:01:03.010 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:03.011 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "waiting for peered", 2024-09-21T15:01:03.011 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:03.011 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:03.011 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:03.011 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140332 2024-09-21T15:01:03.011 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.011 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:03.011 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.011 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:03.011 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.242728+0000", 2024-09-21T15:01:03.011 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.011 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.011 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.011 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:03.011 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.242728+0000", 2024-09-21T15:01:03.012 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.012 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.012 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.012 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:03.012 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.242728+0000", 2024-09-21T15:01:03.012 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:03.012 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.012 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.012 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:03.012 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.242731+0000", 2024-09-21T15:01:03.012 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4989999999999999e-06 2024-09-21T15:01:03.012 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.012 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.012 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:03.013 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.242743+0000", 2024-09-21T15:01:03.013 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2081e-05 2024-09-21T15:01:03.013 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.013 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.013 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:03.013 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.242750+0000", 2024-09-21T15:01:03.013 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.002e-06 2024-09-21T15:01:03.013 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.013 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.013 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:03.013 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.242789+0000", 2024-09-21T15:01:03.013 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.9112000000000001e-05 2024-09-21T15:01:03.013 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.013 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.013 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:03.014 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.242797+0000", 2024-09-21T15:01:03.014 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0460000000000006e-06 2024-09-21T15:01:03.014 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.014 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:03.014 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.014 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.014 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.014 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139785 3.11 3.908ddd51 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:03.014 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.223531+0000", 2024-09-21T15:01:03.014 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.77417393000000001, 2024-09-21T15:01:03.014 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.77451472799999999, 2024-09-21T15:01:03.014 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:03.014 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "waiting for peered", 2024-09-21T15:01:03.014 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:03.014 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:03.015 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:03.015 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139785 2024-09-21T15:01:03.015 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.015 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:03.015 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.015 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:03.015 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223531+0000", 2024-09-21T15:01:03.015 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.015 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.015 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.015 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:03.015 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223531+0000", 2024-09-21T15:01:03.015 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.015 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.016 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.016 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:03.016 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223531+0000", 2024-09-21T15:01:03.016 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:03.016 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.016 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.016 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:03.016 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223534+0000", 2024-09-21T15:01:03.016 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2650000000000001e-06 2024-09-21T15:01:03.016 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.016 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.016 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:03.016 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223541+0000", 2024-09-21T15:01:03.017 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8990000000000004e-06 2024-09-21T15:01:03.017 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.017 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.017 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:03.017 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223546+0000", 2024-09-21T15:01:03.017 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3399999999999997e-06 2024-09-21T15:01:03.017 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.017 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.017 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:03.017 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223724+0000", 2024-09-21T15:01:03.017 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017745799999999999 2024-09-21T15:01:03.017 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.017 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.017 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:03.018 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223735+0000", 2024-09-21T15:01:03.018 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1610999999999999e-05 2024-09-21T15:01:03.018 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.018 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:03.018 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.018 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.018 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.018 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139920 3.11 3.5bc4fe31 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:03.018 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.227979+0000", 2024-09-21T15:01:03.018 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.76972582700000003, 2024-09-21T15:01:03.018 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.77016003600000005, 2024-09-21T15:01:03.018 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:03.018 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "waiting for peered", 2024-09-21T15:01:03.018 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:03.019 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:03.019 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:03.019 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139920 2024-09-21T15:01:03.019 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.019 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:03.019 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.019 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:03.019 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227979+0000", 2024-09-21T15:01:03.019 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.019 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.019 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.019 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:03.019 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227979+0000", 2024-09-21T15:01:03.019 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.020 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.020 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.020 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:03.020 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227979+0000", 2024-09-21T15:01:03.020 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:03.020 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.020 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.020 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:03.020 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227983+0000", 2024-09-21T15:01:03.020 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2529999999999998e-06 2024-09-21T15:01:03.020 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.020 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.020 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:03.020 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227992+0000", 2024-09-21T15:01:03.020 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1190000000000003e-06 2024-09-21T15:01:03.020 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.021 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.021 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:03.021 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227998+0000", 2024-09-21T15:01:03.021 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2809999999999997e-06 2024-09-21T15:01:03.021 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.021 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.021 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:03.021 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.228387+0000", 2024-09-21T15:01:03.021 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00038847000000000003 2024-09-21T15:01:03.021 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.021 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.021 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:03.021 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.228396+0000", 2024-09-21T15:01:03.021 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4870000000000008e-06 2024-09-21T15:01:03.022 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.022 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:03.022 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.022 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.022 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.022 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140076 3.11 3.1f6d09f1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:03.022 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.234754+0000", 2024-09-21T15:01:03.022 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.762951139, 2024-09-21T15:01:03.022 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.76347633000000004, 2024-09-21T15:01:03.022 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:03.022 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "waiting for peered", 2024-09-21T15:01:03.022 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:03.022 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:03.022 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:03.023 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140076 2024-09-21T15:01:03.023 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.023 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:03.023 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.023 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:03.023 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.234754+0000", 2024-09-21T15:01:03.023 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.023 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.023 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.023 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:03.023 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.234754+0000", 2024-09-21T15:01:03.023 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.023 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.023 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.023 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:03.024 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.234754+0000", 2024-09-21T15:01:03.024 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:03.024 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.024 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.024 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:03.024 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.234756+0000", 2024-09-21T15:01:03.024 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.785e-06 2024-09-21T15:01:03.024 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.024 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.024 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:03.024 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.234763+0000", 2024-09-21T15:01:03.024 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9870000000000002e-06 2024-09-21T15:01:03.024 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.024 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.024 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:03.025 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.234768+0000", 2024-09-21T15:01:03.025 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8300000000000003e-06 2024-09-21T15:01:03.025 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.025 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.025 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:03.025 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.234817+0000", 2024-09-21T15:01:03.025 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9119999999999997e-05 2024-09-21T15:01:03.025 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.025 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.025 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:03.025 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.234827+0000", 2024-09-21T15:01:03.025 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.7459999999999995e-06 2024-09-21T15:01:03.025 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.025 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:03.025 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.025 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.026 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.026 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139790 3.11 3.1be31851 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:03.026 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.223578+0000", 2024-09-21T15:01:03.026 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.77412693499999996, 2024-09-21T15:01:03.026 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.77474392000000003, 2024-09-21T15:01:03.026 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:03.026 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "waiting for peered", 2024-09-21T15:01:03.026 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:03.026 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:03.026 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:03.026 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139790 2024-09-21T15:01:03.026 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.026 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:03.026 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.026 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:03.027 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223578+0000", 2024-09-21T15:01:03.027 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.027 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.027 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.027 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:03.027 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223578+0000", 2024-09-21T15:01:03.027 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.027 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.027 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.027 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:03.027 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223578+0000", 2024-09-21T15:01:03.027 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:03.027 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.027 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.027 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:03.028 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223581+0000", 2024-09-21T15:01:03.028 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6879999999999999e-06 2024-09-21T15:01:03.028 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.028 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.028 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:03.028 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223586+0000", 2024-09-21T15:01:03.028 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7080000000000002e-06 2024-09-21T15:01:03.028 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.028 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.028 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:03.028 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223591+0000", 2024-09-21T15:01:03.028 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5639999999999998e-06 2024-09-21T15:01:03.028 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.028 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.028 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:03.028 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223761+0000", 2024-09-21T15:01:03.029 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000170213 2024-09-21T15:01:03.029 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.029 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.029 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:03.029 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223769+0000", 2024-09-21T15:01:03.029 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7049999999999999e-06 2024-09-21T15:01:03.029 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.029 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:03.029 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.029 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.029 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.029 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139792 3.16 3.cf53a736 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:03.029 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.223595+0000", 2024-09-21T15:01:03.029 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.77410998200000003, 2024-09-21T15:01:03.029 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.77482145499999999, 2024-09-21T15:01:03.029 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:03.030 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "waiting for peered", 2024-09-21T15:01:03.030 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:03.030 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:03.030 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:03.030 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139792 2024-09-21T15:01:03.030 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.030 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:03.030 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.030 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:03.030 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223595+0000", 2024-09-21T15:01:03.030 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.030 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.030 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.030 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:03.030 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223595+0000", 2024-09-21T15:01:03.031 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.031 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.031 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.031 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:03.031 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223595+0000", 2024-09-21T15:01:03.031 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:03.031 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.031 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.031 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:03.031 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223598+0000", 2024-09-21T15:01:03.031 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4620000000000001e-06 2024-09-21T15:01:03.031 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.031 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.032 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:03.032 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223604+0000", 2024-09-21T15:01:03.032 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6879999999999996e-06 2024-09-21T15:01:03.032 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.032 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.032 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:03.032 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223609+0000", 2024-09-21T15:01:03.032 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9490000000000004e-06 2024-09-21T15:01:03.032 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.032 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.032 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:03.032 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224067+0000", 2024-09-21T15:01:03.032 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00045780700000000002 2024-09-21T15:01:03.032 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.032 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.033 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:03.033 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224075+0000", 2024-09-21T15:01:03.033 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6610000000000006e-06 2024-09-21T15:01:03.033 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.033 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:03.033 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.033 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.033 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.033 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140224 3.16 3.470b1c36 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:03.033 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.241547+0000", 2024-09-21T15:01:03.033 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.75615787599999995, 2024-09-21T15:01:03.033 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.75697704600000004, 2024-09-21T15:01:03.033 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:03.033 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "waiting for peered", 2024-09-21T15:01:03.033 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:03.034 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:03.034 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:03.034 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140224 2024-09-21T15:01:03.034 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.034 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:03.034 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.034 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:03.034 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.241547+0000", 2024-09-21T15:01:03.034 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.034 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.034 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.034 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:03.034 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.241547+0000", 2024-09-21T15:01:03.034 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.035 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.035 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.035 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:03.035 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.241547+0000", 2024-09-21T15:01:03.035 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:03.035 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.035 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.035 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:03.035 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.241550+0000", 2024-09-21T15:01:03.035 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3249999999999999e-06 2024-09-21T15:01:03.035 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.035 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.035 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:03.035 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.241556+0000", 2024-09-21T15:01:03.035 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1340000000000001e-06 2024-09-21T15:01:03.036 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.036 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.036 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:03.036 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.241562+0000", 2024-09-21T15:01:03.036 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1249999999999998e-06 2024-09-21T15:01:03.036 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.036 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.036 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:03.036 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.241617+0000", 2024-09-21T15:01:03.036 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4976999999999999e-05 2024-09-21T15:01:03.036 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.036 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.036 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:03.036 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.241625+0000", 2024-09-21T15:01:03.037 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6969999999999993e-06 2024-09-21T15:01:03.037 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.037 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:03.037 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.037 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.037 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.037 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140361 3.16 3.b62d81f6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:03.037 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.243797+0000", 2024-09-21T15:01:03.037 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.753907671, 2024-09-21T15:01:03.037 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.75481783599999996, 2024-09-21T15:01:03.037 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:03.037 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "waiting for peered", 2024-09-21T15:01:03.037 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:03.037 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:03.037 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:03.037 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140361 2024-09-21T15:01:03.038 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.038 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:03.038 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.038 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:03.038 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243797+0000", 2024-09-21T15:01:03.038 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.038 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.038 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.038 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:03.038 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243797+0000", 2024-09-21T15:01:03.038 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.038 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.038 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.038 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:03.039 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243797+0000", 2024-09-21T15:01:03.039 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:03.039 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.039 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.039 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:03.039 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243801+0000", 2024-09-21T15:01:03.039 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1309999999999999e-06 2024-09-21T15:01:03.039 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.039 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.039 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:03.039 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243810+0000", 2024-09-21T15:01:03.039 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6459999999999999e-06 2024-09-21T15:01:03.039 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.039 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.039 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:03.039 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243816+0000", 2024-09-21T15:01:03.040 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2609999999999999e-06 2024-09-21T15:01:03.040 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.040 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.040 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:03.040 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243875+0000", 2024-09-21T15:01:03.040 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8604999999999999e-05 2024-09-21T15:01:03.040 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.040 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.040 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:03.040 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243883+0000", 2024-09-21T15:01:03.040 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5860000000000005e-06 2024-09-21T15:01:03.040 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.040 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:03.041 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.041 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.041 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.041 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139794 3.11 3.2f45cf31 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:03.041 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.224011+0000", 2024-09-21T15:01:03.041 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.77369395399999996, 2024-09-21T15:01:03.041 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.77469581300000001, 2024-09-21T15:01:03.041 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:03.041 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "waiting for peered", 2024-09-21T15:01:03.041 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:03.041 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:03.041 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:03.041 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139794 2024-09-21T15:01:03.041 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.041 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:03.041 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.042 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:03.042 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224011+0000", 2024-09-21T15:01:03.042 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.042 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.042 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.042 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:03.042 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224011+0000", 2024-09-21T15:01:03.042 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.042 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.042 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.042 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:03.042 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224010+0000", 2024-09-21T15:01:03.042 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:03.043 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.043 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.043 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:03.043 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224016+0000", 2024-09-21T15:01:03.043 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6550000000000001e-06 2024-09-21T15:01:03.043 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.043 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.043 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:03.043 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224028+0000", 2024-09-21T15:01:03.043 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2422999999999999e-05 2024-09-21T15:01:03.043 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.043 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.043 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:03.043 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224036+0000", 2024-09-21T15:01:03.043 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7749999999999993e-06 2024-09-21T15:01:03.044 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.044 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.044 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:03.044 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225176+0000", 2024-09-21T15:01:03.044 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0011401740000000001 2024-09-21T15:01:03.044 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.044 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.044 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:03.044 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225187+0000", 2024-09-21T15:01:03.044 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0951999999999999e-05 2024-09-21T15:01:03.044 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.044 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:03.044 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.045 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.045 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.045 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140233 3.11 3.908ddd51 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:03.045 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.241599+0000", 2024-09-21T15:01:03.045 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.75610609699999998, 2024-09-21T15:01:03.045 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.75719972099999999, 2024-09-21T15:01:03.045 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:03.045 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "waiting for peered", 2024-09-21T15:01:03.045 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:03.045 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:03.045 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:03.045 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140233 2024-09-21T15:01:03.045 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.045 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:03.045 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.046 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:03.046 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.241599+0000", 2024-09-21T15:01:03.046 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.046 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.046 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.046 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:03.046 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.241599+0000", 2024-09-21T15:01:03.046 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.046 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.046 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.046 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:03.046 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.241599+0000", 2024-09-21T15:01:03.046 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:03.046 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.047 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.047 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:03.047 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.241606+0000", 2024-09-21T15:01:03.047 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7440000000000007e-06 2024-09-21T15:01:03.047 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.047 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.047 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:03.047 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.241614+0000", 2024-09-21T15:01:03.047 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8509999999999999e-06 2024-09-21T15:01:03.047 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.047 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.047 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:03.047 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.241620+0000", 2024-09-21T15:01:03.047 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6289999999999998e-06 2024-09-21T15:01:03.048 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.048 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.048 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:03.048 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.241863+0000", 2024-09-21T15:01:03.048 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000243632 2024-09-21T15:01:03.048 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.048 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.048 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:03.048 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.241872+0000", 2024-09-21T15:01:03.048 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8389999999999996e-06 2024-09-21T15:01:03.048 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.048 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:03.048 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.048 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.049 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.049 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140368 3.11 3.5bc4fe31 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:03.049 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.244276+0000", 2024-09-21T15:01:03.049 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.75342922800000001, 2024-09-21T15:01:03.049 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.75461361299999996, 2024-09-21T15:01:03.049 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:03.049 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "waiting for peered", 2024-09-21T15:01:03.049 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:03.049 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:03.049 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:03.049 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140368 2024-09-21T15:01:03.049 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.049 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:03.049 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.050 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:03.050 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.244276+0000", 2024-09-21T15:01:03.050 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.050 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.050 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.050 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:03.050 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.244276+0000", 2024-09-21T15:01:03.050 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.050 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.050 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.050 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:03.050 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.244275+0000", 2024-09-21T15:01:03.050 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:03.050 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.050 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.051 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:03.051 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.244287+0000", 2024-09-21T15:01:03.051 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1671999999999999e-05 2024-09-21T15:01:03.051 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.051 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.051 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:03.051 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.244302+0000", 2024-09-21T15:01:03.051 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5474000000000001e-05 2024-09-21T15:01:03.051 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.051 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.051 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:03.051 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.244318+0000", 2024-09-21T15:01:03.051 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5821e-05 2024-09-21T15:01:03.051 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.051 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.052 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:03.052 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.244362+0000", 2024-09-21T15:01:03.052 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3968999999999999e-05 2024-09-21T15:01:03.052 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.052 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.052 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:03.052 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.244373+0000", 2024-09-21T15:01:03.052 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1171e-05 2024-09-21T15:01:03.052 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.052 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:03.052 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.052 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.052 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.052 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139800 3.16 3.b70a0d16 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:03.053 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.224579+0000", 2024-09-21T15:01:03.053 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.77312619500000002, 2024-09-21T15:01:03.053 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.77440239099999997, 2024-09-21T15:01:03.053 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:03.053 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "waiting for peered", 2024-09-21T15:01:03.053 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:03.053 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:03.053 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:03.053 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139800 2024-09-21T15:01:03.053 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.053 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:03.053 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.053 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:03.053 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224579+0000", 2024-09-21T15:01:03.053 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.054 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.054 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.054 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:03.054 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224579+0000", 2024-09-21T15:01:03.054 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.054 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.054 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.054 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:03.054 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224578+0000", 2024-09-21T15:01:03.054 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:03.054 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.054 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.054 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:03.054 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224582+0000", 2024-09-21T15:01:03.055 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0029999999999999e-06 2024-09-21T15:01:03.055 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.055 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.055 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:03.055 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224588+0000", 2024-09-21T15:01:03.055 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8120000000000002e-06 2024-09-21T15:01:03.055 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.055 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.055 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:03.055 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224593+0000", 2024-09-21T15:01:03.055 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7469999999999996e-06 2024-09-21T15:01:03.055 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.055 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.055 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:03.055 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224648+0000", 2024-09-21T15:01:03.056 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4693e-05 2024-09-21T15:01:03.056 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.056 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.056 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:03.056 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224656+0000", 2024-09-21T15:01:03.056 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8429999999999993e-06 2024-09-21T15:01:03.056 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.056 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:03.056 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.056 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.056 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.056 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140238 3.11 3.1be31851 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:03.056 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.241649+0000", 2024-09-21T15:01:03.056 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.75605566800000001, 2024-09-21T15:01:03.057 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.75742950499999995, 2024-09-21T15:01:03.057 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:03.057 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "waiting for peered", 2024-09-21T15:01:03.057 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:03.057 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:03.057 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:03.057 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140238 2024-09-21T15:01:03.057 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.057 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:03.057 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.057 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:03.057 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.241649+0000", 2024-09-21T15:01:03.057 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.057 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.058 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.058 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:03.058 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.241649+0000", 2024-09-21T15:01:03.058 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.058 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.058 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.058 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:03.058 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.241649+0000", 2024-09-21T15:01:03.058 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:03.058 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.058 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.058 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:03.058 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.241652+0000", 2024-09-21T15:01:03.058 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.57e-06 2024-09-21T15:01:03.059 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.059 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.059 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:03.059 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.241657+0000", 2024-09-21T15:01:03.059 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.2379999999999997e-06 2024-09-21T15:01:03.059 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.059 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.059 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:03.059 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.241662+0000", 2024-09-21T15:01:03.059 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7829999999999998e-06 2024-09-21T15:01:03.059 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.059 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.059 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:03.059 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.241895+0000", 2024-09-21T15:01:03.059 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000233666 2024-09-21T15:01:03.060 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.060 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.060 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:03.060 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.241904+0000", 2024-09-21T15:01:03.060 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4829999999999999e-06 2024-09-21T15:01:03.060 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.060 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:03.060 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.060 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.060 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.060 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140240 3.16 3.cf53a736 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:03.060 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.241669+0000", 2024-09-21T15:01:03.060 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.75603661600000005, 2024-09-21T15:01:03.060 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.75750148100000003, 2024-09-21T15:01:03.061 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:03.061 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "waiting for peered", 2024-09-21T15:01:03.061 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:03.061 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:03.061 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:03.061 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140240 2024-09-21T15:01:03.061 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.061 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:03.061 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.061 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:03.061 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.241669+0000", 2024-09-21T15:01:03.061 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.061 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.061 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.062 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:03.062 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.241669+0000", 2024-09-21T15:01:03.062 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.062 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.062 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.062 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:03.062 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.241668+0000", 2024-09-21T15:01:03.062 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:03.062 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.062 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.062 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:03.062 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.241682+0000", 2024-09-21T15:01:03.062 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4075e-05 2024-09-21T15:01:03.063 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.063 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.063 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:03.063 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.241689+0000", 2024-09-21T15:01:03.063 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.401e-06 2024-09-21T15:01:03.063 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.063 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.063 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:03.063 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.241695+0000", 2024-09-21T15:01:03.063 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6470000000000003e-06 2024-09-21T15:01:03.063 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.063 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.063 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:03.064 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.241726+0000", 2024-09-21T15:01:03.064 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1440999999999999e-05 2024-09-21T15:01:03.064 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.064 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.064 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:03.064 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.241733+0000", 2024-09-21T15:01:03.064 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1970000000000001e-06 2024-09-21T15:01:03.064 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.064 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:03.064 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.064 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.064 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.064 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139804 3.11 3.73bb9f11 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:03.064 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.224632+0000", 2024-09-21T15:01:03.065 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.77307321399999995, 2024-09-21T15:01:03.065 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.77462836999999996, 2024-09-21T15:01:03.065 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:03.065 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "waiting for peered", 2024-09-21T15:01:03.065 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:03.065 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:03.065 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:03.065 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139804 2024-09-21T15:01:03.065 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.065 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:03.065 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.065 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:03.065 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224632+0000", 2024-09-21T15:01:03.065 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.065 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.066 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.066 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:03.066 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224632+0000", 2024-09-21T15:01:03.066 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.066 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.066 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.066 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:03.066 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224632+0000", 2024-09-21T15:01:03.066 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:03.066 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.066 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.066 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:03.066 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224635+0000", 2024-09-21T15:01:03.066 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2710000000000001e-06 2024-09-21T15:01:03.066 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.067 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.067 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:03.067 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224640+0000", 2024-09-21T15:01:03.067 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6779999999999997e-06 2024-09-21T15:01:03.067 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.067 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.067 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:03.067 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224645+0000", 2024-09-21T15:01:03.067 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9339999999999997e-06 2024-09-21T15:01:03.067 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.067 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.067 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:03.067 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225307+0000", 2024-09-21T15:01:03.067 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00066201699999999999 2024-09-21T15:01:03.067 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.068 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.068 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:03.068 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225312+0000", 2024-09-21T15:01:03.068 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9799999999999998e-06 2024-09-21T15:01:03.068 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.068 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:03.068 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.068 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.068 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.068 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140242 3.11 3.2f45cf31 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:03.068 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.241708+0000", 2024-09-21T15:01:03.068 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.75599755599999996, 2024-09-21T15:01:03.068 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.75764376300000003, 2024-09-21T15:01:03.068 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:03.068 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "waiting for peered", 2024-09-21T15:01:03.068 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:03.068 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:03.069 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:03.069 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140242 2024-09-21T15:01:03.069 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.069 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:03.069 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.069 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:03.069 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.241708+0000", 2024-09-21T15:01:03.069 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.069 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.069 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.069 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:03.069 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.241708+0000", 2024-09-21T15:01:03.069 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.069 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.069 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.069 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:03.070 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.241707+0000", 2024-09-21T15:01:03.070 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:03.070 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.070 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.070 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:03.070 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.241711+0000", 2024-09-21T15:01:03.070 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5779999999999999e-06 2024-09-21T15:01:03.070 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.070 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.070 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:03.070 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.241719+0000", 2024-09-21T15:01:03.070 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.568e-06 2024-09-21T15:01:03.070 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.070 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.070 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:03.071 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.241728+0000", 2024-09-21T15:01:03.071 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0659999999999993e-06 2024-09-21T15:01:03.071 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.071 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.071 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:03.071 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.241934+0000", 2024-09-21T15:01:03.071 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000205773 2024-09-21T15:01:03.071 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.071 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.071 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:03.071 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.241941+0000", 2024-09-21T15:01:03.071 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4800000000000004e-06 2024-09-21T15:01:03.071 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.071 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:03.071 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.071 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.072 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.072 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140143 3.16 3.69ab7cb6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:03.072 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.239496+0000", 2024-09-21T15:01:03.072 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.75820884, 2024-09-21T15:01:03.072 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.75994601799999995, 2024-09-21T15:01:03.072 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:03.072 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "waiting for peered", 2024-09-21T15:01:03.072 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:03.072 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:03.072 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:03.072 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140143 2024-09-21T15:01:03.072 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.072 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:03.072 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.072 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:03.072 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239496+0000", 2024-09-21T15:01:03.073 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.073 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.073 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.073 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:03.073 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239496+0000", 2024-09-21T15:01:03.073 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.073 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.073 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.073 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:03.073 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239496+0000", 2024-09-21T15:01:03.073 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:03.073 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.073 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.073 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:03.073 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239498+0000", 2024-09-21T15:01:03.074 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.509e-06 2024-09-21T15:01:03.074 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.074 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.074 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:03.074 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239504+0000", 2024-09-21T15:01:03.074 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6869999999999999e-06 2024-09-21T15:01:03.074 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.074 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.074 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:03.074 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239509+0000", 2024-09-21T15:01:03.074 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8860000000000003e-06 2024-09-21T15:01:03.074 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.074 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.074 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:03.074 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239572+0000", 2024-09-21T15:01:03.075 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2908000000000001e-05 2024-09-21T15:01:03.075 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.075 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.075 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:03.075 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239580+0000", 2024-09-21T15:01:03.075 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4270000000000008e-06 2024-09-21T15:01:03.075 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.075 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:03.075 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.075 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.075 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.075 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139809 3.11 3.6295b211 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:03.075 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.224667+0000", 2024-09-21T15:01:03.075 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.77303816999999997, 2024-09-21T15:01:03.075 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.77486661999999995, 2024-09-21T15:01:03.076 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:03.076 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "waiting for peered", 2024-09-21T15:01:03.076 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:03.076 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:03.076 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:03.076 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139809 2024-09-21T15:01:03.076 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.076 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:03.076 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.076 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:03.076 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224667+0000", 2024-09-21T15:01:03.076 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.076 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.076 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.076 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:03.076 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224667+0000", 2024-09-21T15:01:03.076 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.077 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.077 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.077 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:03.077 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224667+0000", 2024-09-21T15:01:03.077 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:03.077 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.077 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.077 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:03.077 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224669+0000", 2024-09-21T15:01:03.077 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.452e-06 2024-09-21T15:01:03.077 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.077 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.077 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:03.077 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224675+0000", 2024-09-21T15:01:03.077 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.818e-06 2024-09-21T15:01:03.077 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.078 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.078 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:03.078 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224680+0000", 2024-09-21T15:01:03.078 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6319999999999999e-06 2024-09-21T15:01:03.078 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.078 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.078 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:03.078 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225400+0000", 2024-09-21T15:01:03.078 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00072012699999999999 2024-09-21T15:01:03.078 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.078 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.078 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:03.078 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225409+0000", 2024-09-21T15:01:03.078 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1039999999999996e-06 2024-09-21T15:01:03.078 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.079 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:03.079 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.079 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.079 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.079 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140248 3.16 3.b70a0d16 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:03.079 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.241778+0000", 2024-09-21T15:01:03.079 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.75592755899999997, 2024-09-21T15:01:03.079 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.75784718699999998, 2024-09-21T15:01:03.079 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:03.079 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "waiting for peered", 2024-09-21T15:01:03.079 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:03.079 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:03.079 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:03.079 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140248 2024-09-21T15:01:03.079 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.079 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:03.080 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.080 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:03.080 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.241778+0000", 2024-09-21T15:01:03.080 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.080 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.080 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.080 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:03.080 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.241778+0000", 2024-09-21T15:01:03.080 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.080 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.080 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.080 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:03.080 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.241777+0000", 2024-09-21T15:01:03.080 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:03.080 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.080 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.081 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:03.081 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.241780+0000", 2024-09-21T15:01:03.081 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8210000000000002e-06 2024-09-21T15:01:03.081 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.081 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.081 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:03.081 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.241814+0000", 2024-09-21T15:01:03.081 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4459000000000003e-05 2024-09-21T15:01:03.081 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.081 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.081 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:03.081 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.241833+0000", 2024-09-21T15:01:03.081 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9094999999999999e-05 2024-09-21T15:01:03.081 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.081 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.081 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:03.081 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.241884+0000", 2024-09-21T15:01:03.082 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0578000000000003e-05 2024-09-21T15:01:03.082 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.082 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.082 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:03.082 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.241892+0000", 2024-09-21T15:01:03.082 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7009999999999996e-06 2024-09-21T15:01:03.082 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.082 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:03.082 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.082 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.082 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.082 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139988 3.11 3.25f94511 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:03.082 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.229772+0000", 2024-09-21T15:01:03.082 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.76793273500000003, 2024-09-21T15:01:03.082 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.76994274100000004, 2024-09-21T15:01:03.083 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:03.083 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "waiting for peered", 2024-09-21T15:01:03.083 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:03.083 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:03.083 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:03.083 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139988 2024-09-21T15:01:03.083 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.083 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:03.083 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.083 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:03.083 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229772+0000", 2024-09-21T15:01:03.083 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.083 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.083 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.083 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:03.083 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229772+0000", 2024-09-21T15:01:03.084 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.084 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.084 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.084 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:03.084 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229772+0000", 2024-09-21T15:01:03.084 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:03.084 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.084 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.084 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:03.084 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229776+0000", 2024-09-21T15:01:03.084 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6890000000000002e-06 2024-09-21T15:01:03.084 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.084 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.084 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:03.085 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229783+0000", 2024-09-21T15:01:03.085 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2280000000000001e-06 2024-09-21T15:01:03.085 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.085 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.085 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:03.085 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229789+0000", 2024-09-21T15:01:03.085 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1700000000000002e-06 2024-09-21T15:01:03.085 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.085 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.085 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:03.085 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.230106+0000", 2024-09-21T15:01:03.085 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00031700800000000003 2024-09-21T15:01:03.085 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.085 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.085 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:03.085 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.230115+0000", 2024-09-21T15:01:03.086 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4190000000000008e-06 2024-09-21T15:01:03.086 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.086 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:03.086 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.086 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.086 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.086 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140252 3.11 3.73bb9f11 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:03.086 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.241890+0000", 2024-09-21T15:01:03.086 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.75581543299999998, 2024-09-21T15:01:03.086 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.75791626999999995, 2024-09-21T15:01:03.086 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:03.086 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "waiting for peered", 2024-09-21T15:01:03.086 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:03.086 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:03.086 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:03.087 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140252 2024-09-21T15:01:03.087 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.087 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:03.087 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.087 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:03.087 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.241890+0000", 2024-09-21T15:01:03.087 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.087 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.087 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.087 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:03.087 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.241890+0000", 2024-09-21T15:01:03.087 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.087 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.088 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.088 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:03.088 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.241889+0000", 2024-09-21T15:01:03.088 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:03.088 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.088 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.088 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:03.088 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.241892+0000", 2024-09-21T15:01:03.088 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.202e-06 2024-09-21T15:01:03.088 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.088 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.088 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:03.088 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.241899+0000", 2024-09-21T15:01:03.088 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7870000000000004e-06 2024-09-21T15:01:03.088 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.088 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.089 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:03.089 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.241905+0000", 2024-09-21T15:01:03.089 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4029999999999998e-06 2024-09-21T15:01:03.089 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.089 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.089 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:03.089 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.242039+0000", 2024-09-21T15:01:03.089 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000134838 2024-09-21T15:01:03.089 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.089 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.089 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:03.089 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.242044+0000", 2024-09-21T15:01:03.089 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.369e-06 2024-09-21T15:01:03.089 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.090 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:03.090 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.090 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.090 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.090 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139994 3.16 3.8b1fb9d6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:03.090 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.231370+0000", 2024-09-21T15:01:03.090 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.76633465700000003, 2024-09-21T15:01:03.090 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.76852637499999998, 2024-09-21T15:01:03.090 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:03.090 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "waiting for peered", 2024-09-21T15:01:03.090 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:03.090 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:03.090 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:03.090 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139994 2024-09-21T15:01:03.091 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.091 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:03.091 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.091 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:03.091 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231370+0000", 2024-09-21T15:01:03.091 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.091 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.091 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.091 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:03.091 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231370+0000", 2024-09-21T15:01:03.091 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.091 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.091 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.091 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:03.092 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231369+0000", 2024-09-21T15:01:03.092 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:03.092 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.092 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.092 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:03.092 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231376+0000", 2024-09-21T15:01:03.092 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5509999999999996e-06 2024-09-21T15:01:03.092 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.092 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.092 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:03.092 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231387+0000", 2024-09-21T15:01:03.092 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.061e-05 2024-09-21T15:01:03.092 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.092 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.092 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:03.093 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231394+0000", 2024-09-21T15:01:03.093 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3799999999999996e-06 2024-09-21T15:01:03.093 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.093 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.093 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:03.093 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231459+0000", 2024-09-21T15:01:03.093 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4716999999999997e-05 2024-09-21T15:01:03.093 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.093 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.093 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:03.093 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231468+0000", 2024-09-21T15:01:03.093 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1479999999999995e-06 2024-09-21T15:01:03.093 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.093 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:03.094 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.094 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.094 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.094 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140163 3.16 3.d7417296 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:03.094 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.240334+0000", 2024-09-21T15:01:03.094 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.75737112799999995, 2024-09-21T15:01:03.094 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.759653523, 2024-09-21T15:01:03.094 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:03.094 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "waiting for peered", 2024-09-21T15:01:03.094 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:03.094 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:03.094 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:03.094 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140163 2024-09-21T15:01:03.095 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.095 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:03.095 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.095 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:03.095 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240334+0000", 2024-09-21T15:01:03.095 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.095 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.095 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.095 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:03.095 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240334+0000", 2024-09-21T15:01:03.095 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.095 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.095 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.095 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:03.096 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240334+0000", 2024-09-21T15:01:03.096 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:03.096 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.096 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.096 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:03.096 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240337+0000", 2024-09-21T15:01:03.096 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.089e-06 2024-09-21T15:01:03.096 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.096 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.096 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:03.096 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240343+0000", 2024-09-21T15:01:03.096 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4420000000000003e-06 2024-09-21T15:01:03.096 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.096 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.097 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:03.097 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240348+0000", 2024-09-21T15:01:03.097 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1039999999999998e-06 2024-09-21T15:01:03.097 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.097 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.097 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:03.097 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240451+0000", 2024-09-21T15:01:03.097 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000102996 2024-09-21T15:01:03.097 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.097 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.097 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:03.097 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240460+0000", 2024-09-21T15:01:03.097 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5369999999999997e-06 2024-09-21T15:01:03.097 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.097 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:03.098 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.098 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.098 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.098 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140257 3.11 3.6295b211 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:03.098 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.242004+0000", 2024-09-21T15:01:03.098 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.75570129200000002, 2024-09-21T15:01:03.098 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.75807396500000002, 2024-09-21T15:01:03.098 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:03.098 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "waiting for peered", 2024-09-21T15:01:03.098 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:03.098 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:03.098 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:03.098 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140257 2024-09-21T15:01:03.098 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.098 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:03.099 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.099 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:03.099 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.242004+0000", 2024-09-21T15:01:03.099 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.099 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.099 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.099 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:03.099 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.242004+0000", 2024-09-21T15:01:03.099 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.099 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.099 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.099 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:03.099 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.242003+0000", 2024-09-21T15:01:03.099 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:03.100 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.100 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.100 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:03.100 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.242006+0000", 2024-09-21T15:01:03.100 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9440000000000001e-06 2024-09-21T15:01:03.100 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.100 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.100 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:03.100 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.242012+0000", 2024-09-21T15:01:03.100 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9619999999999998e-06 2024-09-21T15:01:03.100 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.100 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.100 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:03.100 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.242017+0000", 2024-09-21T15:01:03.100 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9749999999999998e-06 2024-09-21T15:01:03.101 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.101 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.101 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:03.101 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.242109+0000", 2024-09-21T15:01:03.101 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1787999999999996e-05 2024-09-21T15:01:03.101 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.101 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.101 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:03.101 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.242113+0000", 2024-09-21T15:01:03.101 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3669999999999999e-06 2024-09-21T15:01:03.101 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.101 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:03.101 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.101 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.101 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.102 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140164 3.16 3.ea3bfd16 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:03.102 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.240358+0000", 2024-09-21T15:01:03.102 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.75734743400000004, 2024-09-21T15:01:03.102 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.759811075, 2024-09-21T15:01:03.102 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:03.102 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "waiting for peered", 2024-09-21T15:01:03.102 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:03.102 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:03.102 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:03.102 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140164 2024-09-21T15:01:03.102 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.102 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:03.102 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.102 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:03.102 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240358+0000", 2024-09-21T15:01:03.103 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.103 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.103 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.103 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:03.103 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240358+0000", 2024-09-21T15:01:03.103 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.103 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.103 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.103 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:03.103 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240357+0000", 2024-09-21T15:01:03.103 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:03.103 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.103 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.103 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:03.103 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240360+0000", 2024-09-21T15:01:03.104 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7089999999999998e-06 2024-09-21T15:01:03.104 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.104 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.104 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:03.104 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240368+0000", 2024-09-21T15:01:03.104 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7270000000000007e-06 2024-09-21T15:01:03.104 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.104 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.104 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:03.104 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240373+0000", 2024-09-21T15:01:03.104 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4060000000000004e-06 2024-09-21T15:01:03.104 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.104 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.104 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:03.105 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240473+0000", 2024-09-21T15:01:03.105 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000100132 2024-09-21T15:01:03.105 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.105 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.105 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:03.105 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240476+0000", 2024-09-21T15:01:03.105 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.332e-06 2024-09-21T15:01:03.105 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.105 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:03.105 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.105 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.105 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.105 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140003 3.11 3.a88fc51 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:03.105 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.231435+0000", 2024-09-21T15:01:03.105 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.76627040800000001, 2024-09-21T15:01:03.105 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.768834664, 2024-09-21T15:01:03.106 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:03.106 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "waiting for peered", 2024-09-21T15:01:03.106 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:03.106 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:03.106 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:03.106 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140003 2024-09-21T15:01:03.106 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.106 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:03.106 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.106 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:03.106 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231435+0000", 2024-09-21T15:01:03.106 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.106 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.106 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.107 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:03.107 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231435+0000", 2024-09-21T15:01:03.107 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.107 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.107 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.107 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:03.107 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231434+0000", 2024-09-21T15:01:03.107 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:03.107 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.107 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.107 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:03.107 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231438+0000", 2024-09-21T15:01:03.107 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2090000000000001e-06 2024-09-21T15:01:03.107 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.107 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.108 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:03.108 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231444+0000", 2024-09-21T15:01:03.108 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2339999999999999e-06 2024-09-21T15:01:03.108 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.108 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.108 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:03.108 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231450+0000", 2024-09-21T15:01:03.108 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9089999999999997e-06 2024-09-21T15:01:03.108 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.108 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.108 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:03.108 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231530+0000", 2024-09-21T15:01:03.108 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0072000000000002e-05 2024-09-21T15:01:03.108 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.108 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.108 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:03.109 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231539+0000", 2024-09-21T15:01:03.109 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2550000000000003e-06 2024-09-21T15:01:03.109 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.109 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:03.109 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.109 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.109 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.109 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140012 3.16 3.6dfb6bd6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:03.109 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.231890+0000", 2024-09-21T15:01:03.109 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.76581489800000002, 2024-09-21T15:01:03.109 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.76854195300000006, 2024-09-21T15:01:03.109 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:03.109 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "waiting for peered", 2024-09-21T15:01:03.109 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:03.109 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:03.109 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:03.110 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140012 2024-09-21T15:01:03.110 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.110 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:03.110 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.110 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:03.110 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231890+0000", 2024-09-21T15:01:03.110 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.110 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.110 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.110 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:03.110 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231890+0000", 2024-09-21T15:01:03.110 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.110 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.110 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.110 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:03.110 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231889+0000", 2024-09-21T15:01:03.111 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:03.111 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.111 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.111 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:03.111 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231895+0000", 2024-09-21T15:01:03.111 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9569999999999999e-06 2024-09-21T15:01:03.111 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.111 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.111 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:03.111 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231910+0000", 2024-09-21T15:01:03.111 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4549000000000001e-05 2024-09-21T15:01:03.111 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.111 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.111 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:03.111 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231917+0000", 2024-09-21T15:01:03.111 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5730000000000002e-06 2024-09-21T15:01:03.112 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.112 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.112 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:03.112 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231988+0000", 2024-09-21T15:01:03.112 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0852999999999994e-05 2024-09-21T15:01:03.112 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.112 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.112 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:03.112 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.232000+0000", 2024-09-21T15:01:03.112 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2159e-05 2024-09-21T15:01:03.112 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.112 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:03.112 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.112 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.112 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.113 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139868 3.16 3.9041f96 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:03.113 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.225982+0000", 2024-09-21T15:01:03.113 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.77172267000000005, 2024-09-21T15:01:03.113 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.77454226299999995, 2024-09-21T15:01:03.113 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:03.113 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "waiting for peered", 2024-09-21T15:01:03.113 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:03.113 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:03.113 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:03.113 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139868 2024-09-21T15:01:03.113 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.113 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:03.113 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.113 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:03.113 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225982+0000", 2024-09-21T15:01:03.114 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.114 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.114 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.114 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:03.114 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225982+0000", 2024-09-21T15:01:03.114 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.114 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.114 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.114 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:03.114 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225982+0000", 2024-09-21T15:01:03.114 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:03.114 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.114 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.114 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:03.115 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225985+0000", 2024-09-21T15:01:03.115 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.74e-06 2024-09-21T15:01:03.115 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.115 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.115 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:03.115 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225990+0000", 2024-09-21T15:01:03.115 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7039999999999999e-06 2024-09-21T15:01:03.115 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.115 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.115 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:03.115 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225995+0000", 2024-09-21T15:01:03.115 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7249999999999997e-06 2024-09-21T15:01:03.115 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.115 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.115 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:03.115 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226299+0000", 2024-09-21T15:01:03.116 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00030383199999999997 2024-09-21T15:01:03.116 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.116 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.116 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:03.116 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226308+0000", 2024-09-21T15:01:03.116 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.9740000000000008e-06 2024-09-21T15:01:03.116 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.116 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:03.116 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.116 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.116 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.116 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140036 3.11 3.cec47e91 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:03.116 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.232498+0000", 2024-09-21T15:01:03.116 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.76520754700000004, 2024-09-21T15:01:03.116 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.76812160799999996, 2024-09-21T15:01:03.116 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:03.117 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "waiting for peered", 2024-09-21T15:01:03.117 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:03.117 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:03.117 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:03.117 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140036 2024-09-21T15:01:03.117 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.117 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:03.117 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.117 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:03.117 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.232498+0000", 2024-09-21T15:01:03.117 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.117 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.117 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.117 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:03.117 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.232498+0000", 2024-09-21T15:01:03.117 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.117 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.118 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.118 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:03.118 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.232497+0000", 2024-09-21T15:01:03.118 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:03.118 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.118 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.118 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:03.118 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.232500+0000", 2024-09-21T15:01:03.118 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3119999999999998e-06 2024-09-21T15:01:03.118 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.118 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.118 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:03.118 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.232509+0000", 2024-09-21T15:01:03.118 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.7669999999999992e-06 2024-09-21T15:01:03.118 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.119 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.119 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:03.119 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.232514+0000", 2024-09-21T15:01:03.119 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0660000000000003e-06 2024-09-21T15:01:03.119 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.119 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.119 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:03.119 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.232743+0000", 2024-09-21T15:01:03.119 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00022838200000000001 2024-09-21T15:01:03.119 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.119 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.119 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:03.119 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.232752+0000", 2024-09-21T15:01:03.119 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.7340000000000003e-06 2024-09-21T15:01:03.119 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.120 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:03.120 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.120 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.120 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.120 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139869 3.16 3.250936d6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:03.120 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.226006+0000", 2024-09-21T15:01:03.120 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.77169905400000005, 2024-09-21T15:01:03.120 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.77470414200000004, 2024-09-21T15:01:03.120 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:03.120 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "waiting for peered", 2024-09-21T15:01:03.120 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:03.120 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:03.120 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:03.120 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139869 2024-09-21T15:01:03.120 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.120 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:03.121 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.121 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:03.121 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226006+0000", 2024-09-21T15:01:03.121 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.121 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.121 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.121 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:03.121 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226006+0000", 2024-09-21T15:01:03.121 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.121 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.121 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.121 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:03.121 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226006+0000", 2024-09-21T15:01:03.121 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:03.121 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.121 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.122 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:03.122 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226008+0000", 2024-09-21T15:01:03.122 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5569999999999998e-06 2024-09-21T15:01:03.122 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.122 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.122 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:03.122 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226014+0000", 2024-09-21T15:01:03.122 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2079999999999997e-06 2024-09-21T15:01:03.122 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.122 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.122 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:03.122 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226020+0000", 2024-09-21T15:01:03.122 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7429999999999999e-06 2024-09-21T15:01:03.122 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.122 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.123 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:03.123 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226330+0000", 2024-09-21T15:01:03.123 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00030963300000000002 2024-09-21T15:01:03.123 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.123 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.123 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:03.123 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226336+0000", 2024-09-21T15:01:03.123 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6340000000000003e-06 2024-09-21T15:01:03.123 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.123 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:03.123 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.123 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.123 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.123 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140175 3.11 3.ef4f4951 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:03.123 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.240901+0000", 2024-09-21T15:01:03.123 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.75680398400000004, 2024-09-21T15:01:03.124 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.75990364399999999, 2024-09-21T15:01:03.124 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:03.124 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "waiting for peered", 2024-09-21T15:01:03.124 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:03.124 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:03.124 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:03.124 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140175 2024-09-21T15:01:03.124 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.124 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:03.124 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.124 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:03.124 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240901+0000", 2024-09-21T15:01:03.124 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.124 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.124 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.125 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:03.125 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240901+0000", 2024-09-21T15:01:03.125 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.125 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.125 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.125 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:03.125 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240901+0000", 2024-09-21T15:01:03.125 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:03.125 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.125 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.125 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:03.125 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240904+0000", 2024-09-21T15:01:03.125 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0029999999999999e-06 2024-09-21T15:01:03.125 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.125 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.126 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:03.126 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240910+0000", 2024-09-21T15:01:03.126 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4010000000000002e-06 2024-09-21T15:01:03.126 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.126 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.126 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:03.126 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240915+0000", 2024-09-21T15:01:03.126 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.792e-06 2024-09-21T15:01:03.126 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.126 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.126 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:03.126 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.241192+0000", 2024-09-21T15:01:03.126 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00027683699999999999 2024-09-21T15:01:03.126 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.126 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.127 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:03.127 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.241201+0000", 2024-09-21T15:01:03.127 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2699999999999993e-06 2024-09-21T15:01:03.127 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.127 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:03.127 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.127 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.127 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.127 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139884 3.16 3.24c08f96 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:03.127 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.227573+0000", 2024-09-21T15:01:03.127 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.77013241300000002, 2024-09-21T15:01:03.127 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.77332313100000005, 2024-09-21T15:01:03.127 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:03.127 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "waiting for peered", 2024-09-21T15:01:03.127 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:03.127 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:03.128 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:03.128 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139884 2024-09-21T15:01:03.128 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.128 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:03.128 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.128 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:03.128 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227573+0000", 2024-09-21T15:01:03.128 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.128 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.128 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.128 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:03.128 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227573+0000", 2024-09-21T15:01:03.128 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.128 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.128 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.128 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:03.129 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227572+0000", 2024-09-21T15:01:03.129 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:03.129 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.129 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.129 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:03.129 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227575+0000", 2024-09-21T15:01:03.129 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7769999999999998e-06 2024-09-21T15:01:03.129 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.129 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.129 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:03.129 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227581+0000", 2024-09-21T15:01:03.129 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7359999999999998e-06 2024-09-21T15:01:03.129 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.129 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.129 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:03.129 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227585+0000", 2024-09-21T15:01:03.130 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5090000000000004e-06 2024-09-21T15:01:03.130 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.130 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.130 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:03.130 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227753+0000", 2024-09-21T15:01:03.130 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00016761599999999999 2024-09-21T15:01:03.130 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.130 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.130 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:03.130 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227761+0000", 2024-09-21T15:01:03.130 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0579999999999998e-06 2024-09-21T15:01:03.130 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.130 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:03.130 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.130 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.130 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.131 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140188 3.16 3.2be846f6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:03.131 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.240978+0000", 2024-09-21T15:01:03.131 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.75672677499999996, 2024-09-21T15:01:03.131 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.76001289699999997, 2024-09-21T15:01:03.131 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:03.131 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "waiting for peered", 2024-09-21T15:01:03.131 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:03.131 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:03.131 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:03.131 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140188 2024-09-21T15:01:03.131 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.131 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:03.131 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.131 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:03.131 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240978+0000", 2024-09-21T15:01:03.131 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.131 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.132 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.132 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:03.132 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240978+0000", 2024-09-21T15:01:03.132 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.132 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.132 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.132 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:03.132 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240978+0000", 2024-09-21T15:01:03.132 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:03.132 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.132 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.132 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:03.132 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240981+0000", 2024-09-21T15:01:03.132 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1559999999999999e-06 2024-09-21T15:01:03.132 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.132 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.133 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:03.133 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240987+0000", 2024-09-21T15:01:03.133 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.229e-06 2024-09-21T15:01:03.133 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.133 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.133 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:03.133 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240992+0000", 2024-09-21T15:01:03.133 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9710000000000003e-06 2024-09-21T15:01:03.133 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.133 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.133 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:03.133 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.241361+0000", 2024-09-21T15:01:03.133 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000369236 2024-09-21T15:01:03.133 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.133 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.133 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:03.134 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.241369+0000", 2024-09-21T15:01:03.134 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6199999999999999e-06 2024-09-21T15:01:03.134 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.134 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:03.134 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.134 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.134 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.134 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140055 3.16 3.c019dd96 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:03.134 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.233922+0000", 2024-09-21T15:01:03.134 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.76378336899999999, 2024-09-21T15:01:03.134 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.76716327500000003, 2024-09-21T15:01:03.134 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:03.134 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "waiting for peered", 2024-09-21T15:01:03.134 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:03.134 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:03.135 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:03.135 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140055 2024-09-21T15:01:03.135 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.135 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:03.135 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.135 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:03.135 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233922+0000", 2024-09-21T15:01:03.135 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.135 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.135 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.135 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:03.135 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233922+0000", 2024-09-21T15:01:03.135 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.135 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.135 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.135 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:03.136 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233921+0000", 2024-09-21T15:01:03.136 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:03.136 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.136 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.136 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:03.136 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233927+0000", 2024-09-21T15:01:03.136 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0360000000000003e-06 2024-09-21T15:01:03.136 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.136 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.136 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:03.136 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233937+0000", 2024-09-21T15:01:03.136 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6439999999999995e-06 2024-09-21T15:01:03.136 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.136 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.136 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:03.136 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233944+0000", 2024-09-21T15:01:03.137 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7710000000000006e-06 2024-09-21T15:01:03.137 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.137 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.137 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:03.137 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233996+0000", 2024-09-21T15:01:03.137 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1597000000000001e-05 2024-09-21T15:01:03.137 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.137 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.137 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:03.137 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.234005+0000", 2024-09-21T15:01:03.137 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5960000000000004e-06 2024-09-21T15:01:03.137 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.137 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:03.137 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.137 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.137 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.138 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140316 3.16 3.9041f96 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:03.138 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.242355+0000", 2024-09-21T15:01:03.138 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.75534997599999998, 2024-09-21T15:01:03.138 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.75882011000000005, 2024-09-21T15:01:03.138 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:03.138 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "waiting for peered", 2024-09-21T15:01:03.138 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:03.138 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:03.138 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:03.138 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140316 2024-09-21T15:01:03.138 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.138 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:03.138 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.138 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:03.139 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.242355+0000", 2024-09-21T15:01:03.139 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.139 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.139 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.139 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:03.139 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.242355+0000", 2024-09-21T15:01:03.139 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.139 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.139 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.139 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:03.139 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.242355+0000", 2024-09-21T15:01:03.139 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:03.139 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.139 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.140 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:03.140 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.242357+0000", 2024-09-21T15:01:03.140 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6129999999999998e-06 2024-09-21T15:01:03.140 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.140 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.140 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:03.140 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.242363+0000", 2024-09-21T15:01:03.140 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6389999999999997e-06 2024-09-21T15:01:03.140 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.140 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.140 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:03.140 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.242368+0000", 2024-09-21T15:01:03.140 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6750000000000001e-06 2024-09-21T15:01:03.140 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.140 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.141 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:03.141 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.242408+0000", 2024-09-21T15:01:03.141 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.0130999999999999e-05 2024-09-21T15:01:03.141 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.141 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.141 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:03.141 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.242416+0000", 2024-09-21T15:01:03.141 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8699999999999992e-06 2024-09-21T15:01:03.141 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.141 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:03.141 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.141 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.141 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.141 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140068 3.11 3.46c84111 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:03.141 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.234128+0000", 2024-09-21T15:01:03.142 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.76357757100000001, 2024-09-21T15:01:03.142 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.76714181000000004, 2024-09-21T15:01:03.142 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:03.142 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "waiting for peered", 2024-09-21T15:01:03.142 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:03.142 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:03.142 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:03.142 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140068 2024-09-21T15:01:03.142 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.142 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:03.142 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.142 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:03.142 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.234128+0000", 2024-09-21T15:01:03.142 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.142 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.143 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.143 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:03.143 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.234128+0000", 2024-09-21T15:01:03.143 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.143 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.143 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.143 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:03.143 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.234127+0000", 2024-09-21T15:01:03.143 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:03.143 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.143 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.143 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:03.143 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.234133+0000", 2024-09-21T15:01:03.143 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1789999999999996e-06 2024-09-21T15:01:03.144 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.144 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.144 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:03.144 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.234147+0000", 2024-09-21T15:01:03.144 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4394e-05 2024-09-21T15:01:03.144 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.144 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.144 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:03.144 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.234155+0000", 2024-09-21T15:01:03.144 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7519999999999996e-06 2024-09-21T15:01:03.144 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.144 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.144 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:03.144 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.234227+0000", 2024-09-21T15:01:03.145 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2486999999999999e-05 2024-09-21T15:01:03.145 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.145 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.145 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:03.145 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.234237+0000", 2024-09-21T15:01:03.145 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4199999999999996e-06 2024-09-21T15:01:03.145 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.145 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:03.145 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.145 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.145 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.145 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140198 3.11 3.1de9ffd1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:03.145 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.241025+0000", 2024-09-21T15:01:03.145 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.75668055899999997, 2024-09-21T15:01:03.145 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.76033541900000001, 2024-09-21T15:01:03.146 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:03.146 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "waiting for peered", 2024-09-21T15:01:03.146 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:03.146 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:03.146 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:03.146 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140198 2024-09-21T15:01:03.146 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.146 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:03.146 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.146 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:03.146 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.241025+0000", 2024-09-21T15:01:03.146 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.146 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.146 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.146 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:03.146 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.241025+0000", 2024-09-21T15:01:03.147 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.147 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.147 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.147 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:03.147 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.241024+0000", 2024-09-21T15:01:03.147 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:03.147 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.147 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.147 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:03.147 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.241027+0000", 2024-09-21T15:01:03.147 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6989999999999999e-06 2024-09-21T15:01:03.147 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.147 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.147 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:03.147 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.241033+0000", 2024-09-21T15:01:03.148 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9089999999999997e-06 2024-09-21T15:01:03.148 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.148 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.148 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:03.148 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.241038+0000", 2024-09-21T15:01:03.148 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8280000000000002e-06 2024-09-21T15:01:03.148 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.148 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.148 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:03.148 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.241222+0000", 2024-09-21T15:01:03.148 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000184835 2024-09-21T15:01:03.148 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.148 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.148 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:03.148 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.241230+0000", 2024-09-21T15:01:03.148 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9140000000000008e-06 2024-09-21T15:01:03.149 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.149 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:03.149 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.149 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.149 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.149 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140317 3.16 3.250936d6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:03.149 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.242435+0000", 2024-09-21T15:01:03.149 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.75527059500000004, 2024-09-21T15:01:03.149 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.75901923599999999, 2024-09-21T15:01:03.149 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:03.149 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "waiting for peered", 2024-09-21T15:01:03.149 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:03.149 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:03.149 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:03.149 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140317 2024-09-21T15:01:03.149 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.149 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:03.150 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.150 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:03.150 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.242435+0000", 2024-09-21T15:01:03.150 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.150 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.150 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.150 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:03.150 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.242435+0000", 2024-09-21T15:01:03.150 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.150 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.150 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.150 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:03.150 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.242434+0000", 2024-09-21T15:01:03.150 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:03.150 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.151 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.151 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:03.151 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.242438+0000", 2024-09-21T15:01:03.151 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7349999999999998e-06 2024-09-21T15:01:03.151 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.151 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.151 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:03.151 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.242447+0000", 2024-09-21T15:01:03.151 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.3880000000000006e-06 2024-09-21T15:01:03.151 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.151 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.151 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:03.151 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.242452+0000", 2024-09-21T15:01:03.151 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1399999999999999e-06 2024-09-21T15:01:03.151 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.151 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.151 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:03.152 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.242680+0000", 2024-09-21T15:01:03.152 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00022838700000000001 2024-09-21T15:01:03.152 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.152 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.152 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:03.152 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.242689+0000", 2024-09-21T15:01:03.152 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4610000000000008e-06 2024-09-21T15:01:03.152 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.152 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:03.152 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.152 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.152 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-21T15:01:03.152 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 40 2024-09-21T15:01:03.152 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:03.154 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T15:01:03.154 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:01:03.151+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:01:03.227 INFO:teuthology.orchestra.run.smithi032.stderr:reweighted osd.5 to 1 (10000) 2024-09-21T15:01:03.240 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-21T15:01:03.254 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:01:03.251+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:01:03.327 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:03.327 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:01:03.327 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T15:01:03.327 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T15:01:03.327 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:03.343 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T15:01:03.355 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:01:03.353+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:01:03.456 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:01:03.451+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:01:03.488 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:03.488 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T15:01:03.488 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T15:01:03.489 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T15:01:03.489 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.489 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139776 3.16 3:6c38d0e2:::rbd_data.113664780c85.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:03.489 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.223442+0000", 2024-09-21T15:01:03.489 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.263689829, 2024-09-21T15:01:03.489 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0315488150000001, 2024-09-21T15:01:03.489 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:03.489 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:03.489 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:03.489 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:03.489 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:03.489 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139776 2024-09-21T15:01:03.489 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.489 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:03.489 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.490 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:03.490 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223442+0000", 2024-09-21T15:01:03.490 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.490 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.490 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.490 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:03.490 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223442+0000", 2024-09-21T15:01:03.490 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.490 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.490 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.490 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:03.490 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223442+0000", 2024-09-21T15:01:03.490 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:03.491 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.491 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.491 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:03.491 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223446+0000", 2024-09-21T15:01:03.491 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2549999999999999e-06 2024-09-21T15:01:03.491 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.491 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.491 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:03.491 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223454+0000", 2024-09-21T15:01:03.491 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8199999999999997e-06 2024-09-21T15:01:03.491 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.491 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.491 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:03.491 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223460+0000", 2024-09-21T15:01:03.491 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6330000000000001e-06 2024-09-21T15:01:03.492 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.492 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.492 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:03.492 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223542+0000", 2024-09-21T15:01:03.492 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1929999999999997e-05 2024-09-21T15:01:03.492 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.492 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.492 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:03.492 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223552+0000", 2024-09-21T15:01:03.492 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0627e-05 2024-09-21T15:01:03.492 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.492 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.492 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:03.492 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.254634+0000", 2024-09-21T15:01:03.492 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.031081972 2024-09-21T15:01:03.493 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.493 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.493 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:03.493 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.254796+0000", 2024-09-21T15:01:03.493 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000161879 2024-09-21T15:01:03.493 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.493 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.493 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:03.493 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.254991+0000", 2024-09-21T15:01:03.493 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00019526499999999999 2024-09-21T15:01:03.493 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.493 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:03.493 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.493 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.494 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.494 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139785 3.11 3:8abbb109:::rbd_data.113664780c85.0000000000000070:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:03.494 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.223531+0000", 2024-09-21T15:01:03.494 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.2636010879999999, 2024-09-21T15:01:03.494 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0187143510000001, 2024-09-21T15:01:03.494 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:03.494 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:03.494 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:03.494 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:03.494 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:03.494 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139785 2024-09-21T15:01:03.494 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.494 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:03.495 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.495 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:03.495 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223531+0000", 2024-09-21T15:01:03.495 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.495 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.495 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.495 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:03.495 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223531+0000", 2024-09-21T15:01:03.495 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.495 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.495 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.495 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:03.495 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223531+0000", 2024-09-21T15:01:03.496 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:03.496 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.496 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.496 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:03.496 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223534+0000", 2024-09-21T15:01:03.496 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2650000000000001e-06 2024-09-21T15:01:03.496 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.496 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.496 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:03.496 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223541+0000", 2024-09-21T15:01:03.496 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8990000000000004e-06 2024-09-21T15:01:03.496 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.496 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.496 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:03.497 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223546+0000", 2024-09-21T15:01:03.497 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3399999999999997e-06 2024-09-21T15:01:03.497 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.497 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.497 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:03.497 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223724+0000", 2024-09-21T15:01:03.497 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017745799999999999 2024-09-21T15:01:03.497 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.497 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.497 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:03.497 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223735+0000", 2024-09-21T15:01:03.497 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1610999999999999e-05 2024-09-21T15:01:03.497 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.498 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.498 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:03.498 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242113+0000", 2024-09-21T15:01:03.498 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.018377879 2024-09-21T15:01:03.498 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.498 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.498 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:03.498 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242246+0000", 2024-09-21T15:01:03.498 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013231500000000001 2024-09-21T15:01:03.498 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.498 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:03.498 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.498 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.498 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.499 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139790 3.11 3:8a18c7d8:::rbd_data.113664780c85.000000000000008d:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:03.499 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.223578+0000", 2024-09-21T15:01:03.499 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.263554093, 2024-09-21T15:01:03.499 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0187879719999999, 2024-09-21T15:01:03.499 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:03.499 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:03.499 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:03.499 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:03.499 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:03.499 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139790 2024-09-21T15:01:03.499 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.499 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:03.499 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.499 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:03.500 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223578+0000", 2024-09-21T15:01:03.500 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.500 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.500 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.500 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:03.500 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223578+0000", 2024-09-21T15:01:03.500 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.500 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.500 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.500 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:03.500 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223578+0000", 2024-09-21T15:01:03.500 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:03.500 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.501 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.501 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:03.501 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223581+0000", 2024-09-21T15:01:03.501 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6879999999999999e-06 2024-09-21T15:01:03.501 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.501 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.501 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:03.501 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223586+0000", 2024-09-21T15:01:03.501 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7080000000000002e-06 2024-09-21T15:01:03.501 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.501 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.501 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:03.501 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223591+0000", 2024-09-21T15:01:03.501 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5639999999999998e-06 2024-09-21T15:01:03.501 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.502 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.502 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:03.502 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223761+0000", 2024-09-21T15:01:03.502 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000170213 2024-09-21T15:01:03.502 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.502 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.502 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:03.502 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223769+0000", 2024-09-21T15:01:03.502 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7049999999999999e-06 2024-09-21T15:01:03.502 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.502 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.502 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:03.502 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242269+0000", 2024-09-21T15:01:03.502 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.018500288 2024-09-21T15:01:03.502 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.503 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.503 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:03.503 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242366+0000", 2024-09-21T15:01:03.503 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.7126000000000005e-05 2024-09-21T15:01:03.503 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.503 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:03.503 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.503 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.503 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.503 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139792 3.16 3:6ce5caf3:::rbd_data.113664780c85.000000000000009b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:03.503 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.223595+0000", 2024-09-21T15:01:03.503 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.2635371399999999, 2024-09-21T15:01:03.503 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0317517899999999, 2024-09-21T15:01:03.503 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:03.504 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:03.504 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:03.504 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:03.504 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:03.504 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139792 2024-09-21T15:01:03.504 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.504 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:03.504 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.504 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:03.504 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223595+0000", 2024-09-21T15:01:03.504 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.504 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.504 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.504 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:03.504 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223595+0000", 2024-09-21T15:01:03.505 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.505 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.505 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.505 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:03.505 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223595+0000", 2024-09-21T15:01:03.505 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:03.505 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.505 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.505 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:03.505 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223598+0000", 2024-09-21T15:01:03.505 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4620000000000001e-06 2024-09-21T15:01:03.505 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.505 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.505 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:03.505 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223604+0000", 2024-09-21T15:01:03.505 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6879999999999996e-06 2024-09-21T15:01:03.506 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.506 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.506 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:03.506 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223609+0000", 2024-09-21T15:01:03.506 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9490000000000004e-06 2024-09-21T15:01:03.506 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.506 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.506 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:03.506 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224067+0000", 2024-09-21T15:01:03.506 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00045780700000000002 2024-09-21T15:01:03.506 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.506 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.506 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:03.506 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224075+0000", 2024-09-21T15:01:03.506 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6610000000000006e-06 2024-09-21T15:01:03.507 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.507 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.507 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:03.507 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255141+0000", 2024-09-21T15:01:03.507 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0310657480000001 2024-09-21T15:01:03.507 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.507 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.507 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:03.507 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255347+0000", 2024-09-21T15:01:03.507 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000205837 2024-09-21T15:01:03.507 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.507 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:03.508 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.508 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.508 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.508 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139794 3.11 3:8cf3a2f4:::rbd_data.113664780c85.00000000000000a9:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:03.508 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.224011+0000", 2024-09-21T15:01:03.508 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.2631211119999999, 2024-09-21T15:01:03.508 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.018507791, 2024-09-21T15:01:03.508 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:03.508 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:03.508 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:03.508 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:03.508 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:03.508 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139794 2024-09-21T15:01:03.508 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.508 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:03.508 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.509 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:03.509 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224011+0000", 2024-09-21T15:01:03.509 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.509 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.509 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.509 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:03.509 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224011+0000", 2024-09-21T15:01:03.509 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.509 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.509 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.509 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:03.509 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224010+0000", 2024-09-21T15:01:03.509 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:03.509 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.509 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.509 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:03.509 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224016+0000", 2024-09-21T15:01:03.510 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6550000000000001e-06 2024-09-21T15:01:03.510 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.510 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.510 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:03.510 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224028+0000", 2024-09-21T15:01:03.510 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2422999999999999e-05 2024-09-21T15:01:03.510 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.510 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.510 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:03.510 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224036+0000", 2024-09-21T15:01:03.510 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7749999999999993e-06 2024-09-21T15:01:03.510 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.510 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.510 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:03.510 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225176+0000", 2024-09-21T15:01:03.511 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0011401740000000001 2024-09-21T15:01:03.511 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.511 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.511 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:03.511 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225187+0000", 2024-09-21T15:01:03.511 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0951999999999999e-05 2024-09-21T15:01:03.511 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.511 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.511 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:03.511 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242392+0000", 2024-09-21T15:01:03.511 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.017204905 2024-09-21T15:01:03.511 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.511 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.511 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:03.511 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242519+0000", 2024-09-21T15:01:03.511 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012680499999999999 2024-09-21T15:01:03.512 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.512 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:03.512 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.512 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.512 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.512 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139800 3.16 3:68b050ed:::rbd_data.113664780c85.00000000000000d3:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:03.512 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.224579+0000", 2024-09-21T15:01:03.512 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.2625533529999999, 2024-09-21T15:01:03.512 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.030932637, 2024-09-21T15:01:03.512 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:03.512 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:03.512 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:03.512 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:03.512 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:03.512 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139800 2024-09-21T15:01:03.512 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.513 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:03.513 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.513 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:03.513 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224579+0000", 2024-09-21T15:01:03.513 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.513 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.513 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.513 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:03.513 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224579+0000", 2024-09-21T15:01:03.513 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.513 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.513 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.513 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:03.513 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224578+0000", 2024-09-21T15:01:03.513 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:03.513 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.514 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.514 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:03.514 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224582+0000", 2024-09-21T15:01:03.514 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0029999999999999e-06 2024-09-21T15:01:03.514 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.514 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.514 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:03.514 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224588+0000", 2024-09-21T15:01:03.514 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8120000000000002e-06 2024-09-21T15:01:03.514 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.514 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.514 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:03.514 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224593+0000", 2024-09-21T15:01:03.514 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7469999999999996e-06 2024-09-21T15:01:03.515 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.515 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.515 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:03.515 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224648+0000", 2024-09-21T15:01:03.515 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4693e-05 2024-09-21T15:01:03.515 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.515 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.515 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:03.515 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224656+0000", 2024-09-21T15:01:03.515 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8429999999999993e-06 2024-09-21T15:01:03.515 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.515 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.515 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:03.515 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255382+0000", 2024-09-21T15:01:03.515 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0307266070000001 2024-09-21T15:01:03.515 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.516 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.516 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:03.516 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255512+0000", 2024-09-21T15:01:03.516 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000129367 2024-09-21T15:01:03.516 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.516 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:03.516 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.516 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.516 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.516 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139804 3.11 3:88f9ddce:::rbd_data.113664780c85.00000000000000ef:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:03.516 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.224632+0000", 2024-09-21T15:01:03.516 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.2625003719999999, 2024-09-21T15:01:03.516 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.018021302, 2024-09-21T15:01:03.516 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:03.516 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:03.516 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:03.517 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:03.517 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:03.517 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139804 2024-09-21T15:01:03.517 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.517 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:03.517 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.517 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:03.517 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224632+0000", 2024-09-21T15:01:03.517 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.517 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.517 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.517 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:03.517 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224632+0000", 2024-09-21T15:01:03.517 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.518 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.518 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.518 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:03.518 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224632+0000", 2024-09-21T15:01:03.518 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:03.518 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.518 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.518 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:03.518 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224635+0000", 2024-09-21T15:01:03.518 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2710000000000001e-06 2024-09-21T15:01:03.518 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.519 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.519 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:03.519 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224640+0000", 2024-09-21T15:01:03.519 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6779999999999997e-06 2024-09-21T15:01:03.519 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.519 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.519 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:03.519 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224645+0000", 2024-09-21T15:01:03.519 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9339999999999997e-06 2024-09-21T15:01:03.519 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.519 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.519 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:03.519 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225307+0000", 2024-09-21T15:01:03.519 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00066201699999999999 2024-09-21T15:01:03.520 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.520 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.520 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:03.520 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225312+0000", 2024-09-21T15:01:03.520 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9799999999999998e-06 2024-09-21T15:01:03.520 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.520 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.520 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:03.520 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242548+0000", 2024-09-21T15:01:03.520 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.017235603 2024-09-21T15:01:03.520 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.520 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.520 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:03.521 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242653+0000", 2024-09-21T15:01:03.521 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010522 2024-09-21T15:01:03.521 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.521 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:03.521 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.521 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.521 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.521 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139809 3.11 3:884da946:::rbd_data.113664780c85.0000000000000118:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:03.521 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.224667+0000", 2024-09-21T15:01:03.521 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.262465328, 2024-09-21T15:01:03.521 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0181101960000001, 2024-09-21T15:01:03.521 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:03.521 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:03.521 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:03.522 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:03.522 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:03.522 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139809 2024-09-21T15:01:03.522 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.522 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:03.522 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.522 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:03.522 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224667+0000", 2024-09-21T15:01:03.522 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.522 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.522 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.522 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:03.522 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224667+0000", 2024-09-21T15:01:03.523 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.523 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.523 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.523 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:03.523 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224667+0000", 2024-09-21T15:01:03.523 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:03.523 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.523 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.523 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:03.523 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224669+0000", 2024-09-21T15:01:03.523 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.452e-06 2024-09-21T15:01:03.523 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.523 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.523 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:03.524 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224675+0000", 2024-09-21T15:01:03.524 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.818e-06 2024-09-21T15:01:03.524 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.524 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.524 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:03.524 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224680+0000", 2024-09-21T15:01:03.524 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6319999999999999e-06 2024-09-21T15:01:03.524 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.524 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.524 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:03.524 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225400+0000", 2024-09-21T15:01:03.524 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00072012699999999999 2024-09-21T15:01:03.524 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.524 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.525 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:03.525 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225409+0000", 2024-09-21T15:01:03.525 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1039999999999996e-06 2024-09-21T15:01:03.525 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.525 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.525 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:03.525 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242675+0000", 2024-09-21T15:01:03.525 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0172663319999999 2024-09-21T15:01:03.525 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.525 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.525 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:03.525 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242777+0000", 2024-09-21T15:01:03.525 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010197899999999999 2024-09-21T15:01:03.526 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.526 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:03.526 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.526 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.526 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.526 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139868 3.16 3:69f82090:::rbd_data.113664780c85.00000000000000f1:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:03.526 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.225982+0000", 2024-09-21T15:01:03.526 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.261149828, 2024-09-21T15:01:03.526 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0296791729999999, 2024-09-21T15:01:03.526 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:03.526 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:03.526 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:03.526 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:03.526 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:03.527 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139868 2024-09-21T15:01:03.527 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.527 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:03.527 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.527 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:03.527 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225982+0000", 2024-09-21T15:01:03.527 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.527 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.527 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.527 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:03.527 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225982+0000", 2024-09-21T15:01:03.527 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.527 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.527 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.528 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:03.528 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225982+0000", 2024-09-21T15:01:03.528 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:03.528 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.528 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.528 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:03.528 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225985+0000", 2024-09-21T15:01:03.528 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.74e-06 2024-09-21T15:01:03.528 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.528 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.528 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:03.528 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225990+0000", 2024-09-21T15:01:03.528 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7039999999999999e-06 2024-09-21T15:01:03.528 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.528 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.529 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:03.529 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225995+0000", 2024-09-21T15:01:03.529 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7249999999999997e-06 2024-09-21T15:01:03.529 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.529 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.529 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:03.529 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226299+0000", 2024-09-21T15:01:03.529 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00030383199999999997 2024-09-21T15:01:03.529 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.529 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.529 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:03.529 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226308+0000", 2024-09-21T15:01:03.529 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.9740000000000008e-06 2024-09-21T15:01:03.529 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.530 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.530 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:03.530 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255539+0000", 2024-09-21T15:01:03.530 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0292305960000001 2024-09-21T15:01:03.530 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.530 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.530 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:03.530 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255662+0000", 2024-09-21T15:01:03.530 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012307400000000001 2024-09-21T15:01:03.530 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.530 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:03.530 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.530 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.530 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.531 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139869 3.16 3:6b6c90a4:::rbd_data.113664780c85.00000000000000fe:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:03.531 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.226006+0000", 2024-09-21T15:01:03.531 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.261126212, 2024-09-21T15:01:03.531 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.029790518, 2024-09-21T15:01:03.531 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:03.531 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:03.531 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:03.531 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:03.531 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:03.531 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139869 2024-09-21T15:01:03.531 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.531 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:03.531 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.532 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:03.532 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226006+0000", 2024-09-21T15:01:03.532 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.532 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.532 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.532 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:03.532 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226006+0000", 2024-09-21T15:01:03.532 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.532 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.532 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.532 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:03.532 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226006+0000", 2024-09-21T15:01:03.532 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:03.532 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.533 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.533 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:03.533 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226008+0000", 2024-09-21T15:01:03.533 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5569999999999998e-06 2024-09-21T15:01:03.533 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.533 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.533 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:03.533 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226014+0000", 2024-09-21T15:01:03.533 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2079999999999997e-06 2024-09-21T15:01:03.533 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.533 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.533 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:03.533 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226020+0000", 2024-09-21T15:01:03.533 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7429999999999999e-06 2024-09-21T15:01:03.533 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.534 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.534 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:03.534 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226330+0000", 2024-09-21T15:01:03.534 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00030963300000000002 2024-09-21T15:01:03.534 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.534 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.534 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:03.534 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226336+0000", 2024-09-21T15:01:03.534 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6340000000000003e-06 2024-09-21T15:01:03.534 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.534 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.534 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:03.534 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255687+0000", 2024-09-21T15:01:03.535 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0293507150000001 2024-09-21T15:01:03.535 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.535 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.535 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:03.535 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255797+0000", 2024-09-21T15:01:03.535 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000109471 2024-09-21T15:01:03.535 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.535 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:03.535 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.535 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.535 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.535 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139884 3.16 3:69f10324:::rbd_data.113664780c85.0000000000000161:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:03.535 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.227573+0000", 2024-09-21T15:01:03.535 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.259559571, 2024-09-21T15:01:03.536 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0283568759999999, 2024-09-21T15:01:03.536 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:03.536 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:03.536 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:03.536 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:03.536 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:03.536 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139884 2024-09-21T15:01:03.536 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.536 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:03.536 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.536 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:03.536 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227573+0000", 2024-09-21T15:01:03.536 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.536 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.536 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.537 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:03.537 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227573+0000", 2024-09-21T15:01:03.537 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.537 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.537 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.537 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:03.537 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227572+0000", 2024-09-21T15:01:03.537 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:03.537 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.537 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.537 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:03.537 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227575+0000", 2024-09-21T15:01:03.537 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7769999999999998e-06 2024-09-21T15:01:03.537 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.538 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.538 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:03.538 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227581+0000", 2024-09-21T15:01:03.538 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7359999999999998e-06 2024-09-21T15:01:03.538 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.538 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.538 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:03.538 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227585+0000", 2024-09-21T15:01:03.538 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5090000000000004e-06 2024-09-21T15:01:03.538 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.538 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.538 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:03.538 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227753+0000", 2024-09-21T15:01:03.538 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00016761599999999999 2024-09-21T15:01:03.539 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.539 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.539 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:03.539 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227761+0000", 2024-09-21T15:01:03.539 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0579999999999998e-06 2024-09-21T15:01:03.539 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.539 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.539 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:03.539 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255828+0000", 2024-09-21T15:01:03.539 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0280671349999999 2024-09-21T15:01:03.539 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.539 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.539 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:03.540 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255930+0000", 2024-09-21T15:01:03.540 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000101372 2024-09-21T15:01:03.540 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.540 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:03.540 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.540 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.540 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.540 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139913 3.16 3:6f81b46d:::rbd_data.113664780c85.0000000000000074:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:03.540 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.227685+0000", 2024-09-21T15:01:03.540 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.259447392, 2024-09-21T15:01:03.540 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0283644750000001, 2024-09-21T15:01:03.540 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:03.540 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:03.540 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:03.540 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:03.541 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:03.541 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139913 2024-09-21T15:01:03.541 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.541 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:03.541 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.541 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:03.541 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227685+0000", 2024-09-21T15:01:03.541 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.541 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.541 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.541 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:03.541 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227685+0000", 2024-09-21T15:01:03.541 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.541 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.542 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.542 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:03.542 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227685+0000", 2024-09-21T15:01:03.542 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:03.542 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.542 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.542 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:03.542 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227687+0000", 2024-09-21T15:01:03.542 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2929999999999999e-06 2024-09-21T15:01:03.542 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.542 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.542 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:03.542 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227693+0000", 2024-09-21T15:01:03.542 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1070000000000001e-06 2024-09-21T15:01:03.543 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.543 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.543 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:03.543 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227698+0000", 2024-09-21T15:01:03.543 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9339999999999997e-06 2024-09-21T15:01:03.543 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.543 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.543 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:03.543 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227784+0000", 2024-09-21T15:01:03.543 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6096999999999999e-05 2024-09-21T15:01:03.543 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.543 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.543 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:03.543 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227791+0000", 2024-09-21T15:01:03.544 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9090000000000003e-06 2024-09-21T15:01:03.544 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.544 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.544 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:03.544 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255955+0000", 2024-09-21T15:01:03.544 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0281643579999999 2024-09-21T15:01:03.544 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.544 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.544 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:03.544 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256049+0000", 2024-09-21T15:01:03.544 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4047000000000003e-05 2024-09-21T15:01:03.544 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.544 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:03.544 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.544 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.545 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.545 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139988 3.11 3:88a29fa4:::rbd_data.113664780c85.00000000000000bd:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:03.545 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.229772+0000", 2024-09-21T15:01:03.545 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.2573598930000001, 2024-09-21T15:01:03.545 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.017967668, 2024-09-21T15:01:03.545 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:03.545 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:03.545 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:03.545 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:03.545 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:03.545 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139988 2024-09-21T15:01:03.545 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.545 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:03.546 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.546 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:03.546 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229772+0000", 2024-09-21T15:01:03.546 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.546 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.546 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.546 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:03.546 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229772+0000", 2024-09-21T15:01:03.546 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.546 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.546 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.546 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:03.546 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229772+0000", 2024-09-21T15:01:03.546 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:03.547 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.547 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.547 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:03.547 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229776+0000", 2024-09-21T15:01:03.547 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6890000000000002e-06 2024-09-21T15:01:03.547 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.547 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.547 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:03.547 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229783+0000", 2024-09-21T15:01:03.547 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2280000000000001e-06 2024-09-21T15:01:03.547 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.547 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.547 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:03.547 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229789+0000", 2024-09-21T15:01:03.548 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1700000000000002e-06 2024-09-21T15:01:03.548 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.548 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.548 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:03.548 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.230106+0000", 2024-09-21T15:01:03.548 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00031700800000000003 2024-09-21T15:01:03.548 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.548 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.548 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:03.548 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.230115+0000", 2024-09-21T15:01:03.548 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4190000000000008e-06 2024-09-21T15:01:03.548 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.548 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.549 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:03.549 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242935+0000", 2024-09-21T15:01:03.549 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0128191449999999 2024-09-21T15:01:03.549 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.549 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.549 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for missing object", 2024-09-21T15:01:03.549 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.243078+0000", 2024-09-21T15:01:03.549 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000142984 2024-09-21T15:01:03.549 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.549 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.549 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:03.549 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.247659+0000", 2024-09-21T15:01:03.549 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0045813030000000001 2024-09-21T15:01:03.549 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.550 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.550 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:03.550 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.247740+0000", 2024-09-21T15:01:03.550 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1216000000000003e-05 2024-09-21T15:01:03.550 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.550 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:03.550 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.550 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.550 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.550 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139994 3.16 3:6b9df8d1:::rbd_data.113664780c85.00000000000000e7:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:03.550 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.231370+0000", 2024-09-21T15:01:03.550 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.2557618150000001, 2024-09-21T15:01:03.550 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.024795576, 2024-09-21T15:01:03.550 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:03.551 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:03.551 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:03.551 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:03.551 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:03.551 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139994 2024-09-21T15:01:03.551 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.551 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:03.551 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.551 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:03.551 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231370+0000", 2024-09-21T15:01:03.551 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.551 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.551 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.551 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:03.552 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231370+0000", 2024-09-21T15:01:03.552 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.552 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.552 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.552 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:03.552 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231369+0000", 2024-09-21T15:01:03.552 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:03.552 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.552 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.552 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:03.552 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231376+0000", 2024-09-21T15:01:03.552 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5509999999999996e-06 2024-09-21T15:01:03.552 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.552 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.553 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:03.553 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231387+0000", 2024-09-21T15:01:03.553 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.061e-05 2024-09-21T15:01:03.553 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.553 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.553 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:03.553 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231394+0000", 2024-09-21T15:01:03.553 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3799999999999996e-06 2024-09-21T15:01:03.553 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.553 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.553 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:03.553 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231459+0000", 2024-09-21T15:01:03.553 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4716999999999997e-05 2024-09-21T15:01:03.553 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.553 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.554 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:03.554 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231468+0000", 2024-09-21T15:01:03.554 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1479999999999995e-06 2024-09-21T15:01:03.554 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.554 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.554 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:03.554 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256074+0000", 2024-09-21T15:01:03.554 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0246064180000001 2024-09-21T15:01:03.554 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.554 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.554 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:03.554 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256166+0000", 2024-09-21T15:01:03.554 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1862999999999998e-05 2024-09-21T15:01:03.554 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.555 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:03.555 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.555 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.555 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.555 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140012 3.16 3:6bd6dfb6:::rbd_data.113664780c85.0000000000000165:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:03.555 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.231890+0000", 2024-09-21T15:01:03.555 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.2552420559999999, 2024-09-21T15:01:03.555 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.024408881, 2024-09-21T15:01:03.555 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:03.555 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:03.555 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:03.555 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:03.555 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:03.555 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140012 2024-09-21T15:01:03.556 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.556 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:03.556 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.556 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:03.556 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231890+0000", 2024-09-21T15:01:03.556 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.556 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.556 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.556 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:03.556 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231890+0000", 2024-09-21T15:01:03.556 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.556 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.556 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.556 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:03.557 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231889+0000", 2024-09-21T15:01:03.557 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:03.557 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.557 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.557 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:03.557 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231895+0000", 2024-09-21T15:01:03.557 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9569999999999999e-06 2024-09-21T15:01:03.557 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.557 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.557 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:03.557 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231910+0000", 2024-09-21T15:01:03.557 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4549000000000001e-05 2024-09-21T15:01:03.557 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.557 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.557 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:03.558 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231917+0000", 2024-09-21T15:01:03.558 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5730000000000002e-06 2024-09-21T15:01:03.558 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.558 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.558 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:03.558 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231988+0000", 2024-09-21T15:01:03.558 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0852999999999994e-05 2024-09-21T15:01:03.558 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.558 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.558 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:03.558 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.232000+0000", 2024-09-21T15:01:03.558 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2159e-05 2024-09-21T15:01:03.558 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.559 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.559 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:03.559 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256192+0000", 2024-09-21T15:01:03.559 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.024192073 2024-09-21T15:01:03.559 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.559 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.559 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:03.559 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256299+0000", 2024-09-21T15:01:03.559 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000106733 2024-09-21T15:01:03.559 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.559 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:03.559 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.559 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.559 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.560 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140055 3.16 3:69bb9803:::rbd_data.113664780c85.00000000000000da:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:03.560 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.233922+0000", 2024-09-21T15:01:03.560 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.253210527, 2024-09-21T15:01:03.560 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.022511768, 2024-09-21T15:01:03.560 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:03.560 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:03.560 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:03.560 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:03.560 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:03.560 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140055 2024-09-21T15:01:03.560 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.560 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:03.560 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.560 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:03.561 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233922+0000", 2024-09-21T15:01:03.561 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.561 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.561 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.561 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:03.561 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233922+0000", 2024-09-21T15:01:03.561 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.561 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.561 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.561 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:03.561 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233921+0000", 2024-09-21T15:01:03.561 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:03.561 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.561 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.562 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:03.562 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233927+0000", 2024-09-21T15:01:03.562 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0360000000000003e-06 2024-09-21T15:01:03.562 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.562 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.562 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:03.562 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233937+0000", 2024-09-21T15:01:03.562 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6439999999999995e-06 2024-09-21T15:01:03.562 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.562 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.562 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:03.562 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233944+0000", 2024-09-21T15:01:03.562 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7710000000000006e-06 2024-09-21T15:01:03.562 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.563 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.563 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:03.563 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233996+0000", 2024-09-21T15:01:03.563 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1597000000000001e-05 2024-09-21T15:01:03.563 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.563 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.563 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:03.563 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.234005+0000", 2024-09-21T15:01:03.563 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5960000000000004e-06 2024-09-21T15:01:03.563 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.563 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.563 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:03.563 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256332+0000", 2024-09-21T15:01:03.563 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0223271949999999 2024-09-21T15:01:03.564 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.564 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.564 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:03.564 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256434+0000", 2024-09-21T15:01:03.564 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010180599999999999 2024-09-21T15:01:03.564 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.564 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:03.564 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.564 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.564 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.564 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140143 3.16 3:6d3ed596:::rbd_data.113664780c85.0000000000000184:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:03.564 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.239496+0000", 2024-09-21T15:01:03.564 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.247635998, 2024-09-21T15:01:03.564 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.017076576, 2024-09-21T15:01:03.565 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:03.565 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:03.565 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:03.565 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:03.565 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:03.565 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140143 2024-09-21T15:01:03.565 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.565 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:03.565 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.565 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:03.565 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239496+0000", 2024-09-21T15:01:03.565 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.565 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.565 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.566 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:03.566 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239496+0000", 2024-09-21T15:01:03.566 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.566 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.566 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.566 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:03.566 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239496+0000", 2024-09-21T15:01:03.566 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:03.566 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.566 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.566 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:03.566 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239498+0000", 2024-09-21T15:01:03.566 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.509e-06 2024-09-21T15:01:03.566 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.567 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.567 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:03.567 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239504+0000", 2024-09-21T15:01:03.567 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6869999999999999e-06 2024-09-21T15:01:03.567 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.567 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.567 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:03.567 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239509+0000", 2024-09-21T15:01:03.567 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8860000000000003e-06 2024-09-21T15:01:03.567 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.567 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.567 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:03.567 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239572+0000", 2024-09-21T15:01:03.567 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2908000000000001e-05 2024-09-21T15:01:03.568 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.568 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.568 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:03.568 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239580+0000", 2024-09-21T15:01:03.568 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4270000000000008e-06 2024-09-21T15:01:03.568 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.568 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.568 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:03.568 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256457+0000", 2024-09-21T15:01:03.568 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0168770979999999 2024-09-21T15:01:03.568 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.568 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.568 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:03.568 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256573+0000", 2024-09-21T15:01:03.569 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00011538400000000001 2024-09-21T15:01:03.569 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.569 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:03.569 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.569 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.569 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.569 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140163 3.16 3:694e82eb:::rbd_data.113664780c85.0000000000000052:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:03.569 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.240334+0000", 2024-09-21T15:01:03.569 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.246798286, 2024-09-21T15:01:03.569 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.016365537, 2024-09-21T15:01:03.569 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:03.569 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:03.569 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:03.569 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:03.569 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:03.570 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140163 2024-09-21T15:01:03.570 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.570 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:03.570 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.570 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:03.570 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240334+0000", 2024-09-21T15:01:03.570 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.570 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.570 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.570 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:03.570 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240334+0000", 2024-09-21T15:01:03.570 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.570 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.570 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.571 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:03.571 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240334+0000", 2024-09-21T15:01:03.571 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:03.571 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.571 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.571 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:03.571 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240337+0000", 2024-09-21T15:01:03.571 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.089e-06 2024-09-21T15:01:03.571 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.571 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.571 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:03.571 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240343+0000", 2024-09-21T15:01:03.571 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4420000000000003e-06 2024-09-21T15:01:03.571 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.572 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.572 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:03.572 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240348+0000", 2024-09-21T15:01:03.572 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1039999999999998e-06 2024-09-21T15:01:03.572 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.572 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.572 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:03.572 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240451+0000", 2024-09-21T15:01:03.572 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000102996 2024-09-21T15:01:03.572 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.572 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.572 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:03.572 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240460+0000", 2024-09-21T15:01:03.572 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5369999999999997e-06 2024-09-21T15:01:03.572 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.573 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.573 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:03.573 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256600+0000", 2024-09-21T15:01:03.573 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.016139884 2024-09-21T15:01:03.573 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.573 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.573 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:03.573 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256700+0000", 2024-09-21T15:01:03.573 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9844000000000002e-05 2024-09-21T15:01:03.573 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.573 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:03.573 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.573 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.573 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.574 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140164 3.16 3:68bfdc57:::rbd_data.113664780c85.0000000000000053:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:03.574 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.240358+0000", 2024-09-21T15:01:03.574 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.246774592, 2024-09-21T15:01:03.574 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.01646094, 2024-09-21T15:01:03.574 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:03.574 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:03.574 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:03.574 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:03.574 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:03.574 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140164 2024-09-21T15:01:03.574 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.574 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:03.574 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.574 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:03.575 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240358+0000", 2024-09-21T15:01:03.575 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.575 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.575 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.575 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:03.575 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240358+0000", 2024-09-21T15:01:03.575 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.575 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.575 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.575 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:03.575 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240357+0000", 2024-09-21T15:01:03.575 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:03.575 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.575 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.576 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:03.576 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240360+0000", 2024-09-21T15:01:03.576 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7089999999999998e-06 2024-09-21T15:01:03.576 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.576 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.576 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:03.576 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240368+0000", 2024-09-21T15:01:03.576 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7270000000000007e-06 2024-09-21T15:01:03.576 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.576 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.576 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:03.576 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240373+0000", 2024-09-21T15:01:03.576 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4060000000000004e-06 2024-09-21T15:01:03.576 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.576 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.577 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:03.577 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240473+0000", 2024-09-21T15:01:03.577 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000100132 2024-09-21T15:01:03.577 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.577 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.577 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:03.577 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240476+0000", 2024-09-21T15:01:03.577 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.332e-06 2024-09-21T15:01:03.577 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.577 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.577 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:03.577 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256724+0000", 2024-09-21T15:01:03.577 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0162471930000001 2024-09-21T15:01:03.577 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.577 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.578 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:03.578 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256819+0000", 2024-09-21T15:01:03.578 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5007999999999996e-05 2024-09-21T15:01:03.578 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.578 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:03.578 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.578 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.578 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.578 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140188 3.16 3:6f6217d4:::rbd_data.113664780c85.00000000000000fb:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:03.578 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.240978+0000", 2024-09-21T15:01:03.578 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 1.246153933, 2024-09-21T15:01:03.578 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.015983809, 2024-09-21T15:01:03.578 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:03.579 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:03.579 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:03.579 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:03.579 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:03.579 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140188 2024-09-21T15:01:03.579 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.579 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:03.579 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.579 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:03.579 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240978+0000", 2024-09-21T15:01:03.579 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.579 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.579 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.579 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:03.579 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240978+0000", 2024-09-21T15:01:03.580 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:03.580 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.580 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.580 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:03.580 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240978+0000", 2024-09-21T15:01:03.580 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:03.580 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.580 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.580 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:03.580 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240981+0000", 2024-09-21T15:01:03.580 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1559999999999999e-06 2024-09-21T15:01:03.580 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.580 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.580 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:03.581 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240987+0000", 2024-09-21T15:01:03.581 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.229e-06 2024-09-21T15:01:03.581 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.581 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.581 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:03.581 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240992+0000", 2024-09-21T15:01:03.581 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9710000000000003e-06 2024-09-21T15:01:03.581 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.581 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.581 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:03.581 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.241361+0000", 2024-09-21T15:01:03.581 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000369236 2024-09-21T15:01:03.581 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.581 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.581 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:03.582 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.241369+0000", 2024-09-21T15:01:03.582 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6199999999999999e-06 2024-09-21T15:01:03.582 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.582 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.582 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:03.582 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256844+0000", 2024-09-21T15:01:03.582 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0154751609999999 2024-09-21T15:01:03.582 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:03.582 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:03.582 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:03.582 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256962+0000", 2024-09-21T15:01:03.582 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000117964 2024-09-21T15:01:03.582 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.583 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:03.583 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.583 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:03.583 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:03.583 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:03.585 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T15:01:03.586 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:01:03.581+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:01:03.587 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T15:01:03.685 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:01:03.681+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:01:03.702 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:01:03.702 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:01:03.702 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T15:01:03.702 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:01:03.712 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T15:01:03.776 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T15:01:03.785 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:01:03.783+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:01:03.796 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T15:01:03.822 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:01:03.822 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:01:03.823 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T15:01:03.823 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T15:01:03.823 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:01:03.825 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T15:01:03.833 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T15:01:03.851 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T15:01:03.874 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-21T15:01:03.886 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:01:03.883+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:01:03.895 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T15:01:03.926 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T15:01:03.945 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:03.945 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:03.955 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T15:01:03.955 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T15:01:03.958 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:01:03.958 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:01:03.958 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:01:03.958 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:01:03.958 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:01:03.958 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:01:03.958 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:01:03.959 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:01:04.013 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:01:04.011+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:01:04.061 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:04.062 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:04.071 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T15:01:04.087 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:01:04.083+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:01:04.183 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:04.183 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:04.187 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:01:04.183+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:01:04.193 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T15:01:04.233 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T15:01:04.236 INFO:teuthology.orchestra.run.smithi032.stderr:reweighted osd.6 to 1 (10000) 2024-09-21T15:01:04.252 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-21T15:01:04.288 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:01:04.285+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:01:04.313 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:04.313 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:04.324 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T15:01:04.389 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:01:04.383+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:01:04.426 INFO:teuthology.orchestra.run.smithi032.stdout:checking clone #34, image image_client.0-clone34 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent35 2024-09-21T15:01:04.429 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:04.429 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:04.439 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T15:01:04.490 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:01:04.485+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:01:04.545 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:04.545 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:04.555 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T15:01:04.590 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:01:04.587+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:01:04.691 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:01:04.689+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:01:04.715 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:04.715 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:04.725 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T15:01:04.793 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:01:04.786+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:01:04.833 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:04.833 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:04.844 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T15:01:04.893 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:01:04.889+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:01:04.950 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:04.950 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:04.960 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T15:01:04.994 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:01:04.989+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:01:05.068 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:05.069 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:05.078 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T15:01:05.095 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:01:05.093+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:01:05.173 INFO:teuthology.orchestra.run.smithi032.stderr:reweighted osd.7 to 1 (10000) 2024-09-21T15:01:05.173 INFO:teuthology.orchestra.run.smithi032.stderr:noscrub is set 2024-09-21T15:01:05.185 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-21T15:01:05.196 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:01:05.190+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:01:05.225 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:05.226 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:05.240 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T15:01:05.297 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:01:05.293+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:01:05.355 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:05.355 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:05.365 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T15:01:05.398 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:01:05.393+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:01:05.467 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:05.467 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:05.476 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T15:01:05.499 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:01:05.494+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:01:05.581 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:05.581 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:05.590 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T15:01:05.633 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:01:05.630+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:01:05.701 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:01:05.694+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:01:05.743 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:05.743 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:05.753 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T15:01:05.801 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:01:05.798+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:01:05.862 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:05.862 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:05.872 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T15:01:05.902 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:01:05.898+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:01:05.973 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T15:01:05.983 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:05.983 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:05.993 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T15:01:06.041 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:01:06.038+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:01:06.104 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:01:06.101+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:01:06.115 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T15:01:06.148 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T15:01:06.167 INFO:teuthology.orchestra.run.smithi032.stderr:reweighted osd.4 to 1 (10000) 2024-09-21T15:01:06.170 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:06.170 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:06.173 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T15:01:06.180 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-21T15:01:06.180 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T15:01:06.184 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T15:01:06.197 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T15:01:06.205 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:01:06.201+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:01:06.221 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-21T15:01:06.250 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T15:01:06.272 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T15:01:06.288 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:06.288 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T15:01:06.288 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T15:01:06.288 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T15:01:06.288 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.288 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:15130 3.2 3:47f1bbdb:::rbd_data.11361bed1f78.0000000000000067:head [sparse-read 0~147456] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-21T15:01:06.288 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:40.288284+0000", 2024-09-21T15:01:06.288 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 145.99833932799999, 2024-09-21T15:01:06.288 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1142935039999999, 2024-09-21T15:01:06.288 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:06.289 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:06.289 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:06.289 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:06.289 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:06.289 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 15130 2024-09-21T15:01:06.289 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.289 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:06.289 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.289 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:06.289 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288284+0000", 2024-09-21T15:01:06.289 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:06.289 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.289 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.289 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:06.290 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288284+0000", 2024-09-21T15:01:06.290 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:06.290 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.290 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.290 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:06.290 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288284+0000", 2024-09-21T15:01:06.290 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:06.290 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.290 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.290 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:06.290 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288288+0000", 2024-09-21T15:01:06.290 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5829999999999998e-06 2024-09-21T15:01:06.290 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.290 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.291 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:06.291 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288297+0000", 2024-09-21T15:01:06.291 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0669999999999998e-06 2024-09-21T15:01:06.291 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.291 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.291 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:06.291 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288305+0000", 2024-09-21T15:01:06.291 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8860000000000004e-06 2024-09-21T15:01:06.291 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.291 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.291 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:06.291 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288417+0000", 2024-09-21T15:01:06.291 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000112952 2024-09-21T15:01:06.291 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.292 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.292 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:06.292 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288428+0000", 2024-09-21T15:01:06.292 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0631e-05 2024-09-21T15:01:06.292 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.292 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.292 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:06.292 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.401876+0000", 2024-09-21T15:01:06.292 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1134479610000001 2024-09-21T15:01:06.292 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.292 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.292 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:06.292 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.402076+0000", 2024-09-21T15:01:06.292 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00019944200000000001 2024-09-21T15:01:06.293 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.293 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.293 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:06.293 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.402578+0000", 2024-09-21T15:01:06.293 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00050242899999999998 2024-09-21T15:01:06.293 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:06.293 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:06.293 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:06.293 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.293 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.293 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45708 3.2 3:415db91a:::rbd_data.1136ecbe5879.0000000000000000:28 [read 140288~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:06.293 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141801+0000", 2024-09-21T15:01:06.293 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 116.14482313400001, 2024-09-21T15:01:06.293 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034274200999999997, 2024-09-21T15:01:06.293 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:06.294 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:06.294 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:06.294 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:06.294 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:06.294 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45708 2024-09-21T15:01:06.294 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.294 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:06.294 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.294 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:06.294 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141801+0000", 2024-09-21T15:01:06.294 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:06.294 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.294 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.295 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:06.295 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141801+0000", 2024-09-21T15:01:06.295 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:06.295 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.295 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.295 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:06.295 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141800+0000", 2024-09-21T15:01:06.295 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:06.295 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.295 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.295 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:06.295 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141803+0000", 2024-09-21T15:01:06.295 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4210000000000002e-06 2024-09-21T15:01:06.295 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.296 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.296 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:06.296 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141812+0000", 2024-09-21T15:01:06.296 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.9800000000000004e-06 2024-09-21T15:01:06.296 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.296 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.296 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:06.296 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141816+0000", 2024-09-21T15:01:06.296 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4769999999999997e-06 2024-09-21T15:01:06.296 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.296 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.296 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:06.296 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.175865+0000", 2024-09-21T15:01:06.296 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034048875999999999 2024-09-21T15:01:06.296 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.297 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.297 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:06.297 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.175943+0000", 2024-09-21T15:01:06.297 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7464000000000002e-05 2024-09-21T15:01:06.297 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.297 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.297 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:06.297 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176075+0000", 2024-09-21T15:01:06.297 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013230800000000001 2024-09-21T15:01:06.297 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:06.297 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:06.297 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:06.297 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.297 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.297 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45709 3.2 3:415db91a:::rbd_data.1136ecbe5879.0000000000000000:28 [read 143360~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:06.298 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141820+0000", 2024-09-21T15:01:06.298 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 116.144803287, 2024-09-21T15:01:06.298 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034537536000000001, 2024-09-21T15:01:06.298 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:06.298 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:06.298 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:06.298 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:06.298 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:06.298 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45709 2024-09-21T15:01:06.298 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.298 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:06.298 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.298 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:06.299 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141820+0000", 2024-09-21T15:01:06.299 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:06.299 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.299 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.299 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:06.299 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141820+0000", 2024-09-21T15:01:06.299 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:06.299 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.299 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.299 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:06.299 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141820+0000", 2024-09-21T15:01:06.299 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:06.299 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.299 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.299 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:06.300 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141823+0000", 2024-09-21T15:01:06.300 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5110000000000002e-06 2024-09-21T15:01:06.300 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.300 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.300 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:06.300 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141829+0000", 2024-09-21T15:01:06.300 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0850000000000002e-06 2024-09-21T15:01:06.300 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.300 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.300 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:06.300 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141833+0000", 2024-09-21T15:01:06.300 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7620000000000003e-06 2024-09-21T15:01:06.300 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.300 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.301 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:06.301 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176103+0000", 2024-09-21T15:01:06.301 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034269933000000002 2024-09-21T15:01:06.301 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.301 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.301 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:06.301 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176161+0000", 2024-09-21T15:01:06.301 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7658999999999998e-05 2024-09-21T15:01:06.301 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.301 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.301 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:06.301 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176358+0000", 2024-09-21T15:01:06.301 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000196937 2024-09-21T15:01:06.302 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:06.302 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:06.302 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:06.302 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.302 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.302 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45710 3.2 3:415db91a:::rbd_data.1136ecbe5879.0000000000000000:28 [read 146432~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:06.302 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141838+0000", 2024-09-21T15:01:06.302 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 116.14478612400001, 2024-09-21T15:01:06.302 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034953674999999997, 2024-09-21T15:01:06.302 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:06.302 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:06.302 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:06.302 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:06.302 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:06.303 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45710 2024-09-21T15:01:06.303 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.303 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:06.303 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.303 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:06.303 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141838+0000", 2024-09-21T15:01:06.303 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:06.303 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.303 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.303 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:06.303 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141838+0000", 2024-09-21T15:01:06.303 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:06.303 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.303 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.303 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:06.303 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141837+0000", 2024-09-21T15:01:06.304 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:06.304 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.304 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.304 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:06.304 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141843+0000", 2024-09-21T15:01:06.304 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1619999999999996e-06 2024-09-21T15:01:06.304 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.304 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.304 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:06.304 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141850+0000", 2024-09-21T15:01:06.304 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4799999999999998e-06 2024-09-21T15:01:06.304 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.304 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.304 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:06.304 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141853+0000", 2024-09-21T15:01:06.305 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3050000000000001e-06 2024-09-21T15:01:06.305 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.305 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.305 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:06.305 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176490+0000", 2024-09-21T15:01:06.305 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034636413999999997 2024-09-21T15:01:06.305 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.305 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.305 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:06.305 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176606+0000", 2024-09-21T15:01:06.305 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000115841 2024-09-21T15:01:06.305 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.305 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.305 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:06.306 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176791+0000", 2024-09-21T15:01:06.306 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00018581500000000001 2024-09-21T15:01:06.306 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:06.306 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:06.306 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:06.306 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.306 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.306 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45711 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:06.306 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141856+0000", 2024-09-21T15:01:06.306 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 116.14476778700001, 2024-09-21T15:01:06.306 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034986308000000001, 2024-09-21T15:01:06.306 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:06.306 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:06.306 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:06.306 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:06.307 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:06.307 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45711 2024-09-21T15:01:06.307 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.307 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:06.307 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.307 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:06.307 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141856+0000", 2024-09-21T15:01:06.307 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:06.307 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.307 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.307 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:06.307 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141856+0000", 2024-09-21T15:01:06.307 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:06.307 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.307 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.308 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:06.308 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141856+0000", 2024-09-21T15:01:06.308 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:06.308 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.308 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.308 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:06.308 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141862+0000", 2024-09-21T15:01:06.308 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.567e-06 2024-09-21T15:01:06.308 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.308 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.308 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:06.308 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141868+0000", 2024-09-21T15:01:06.308 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9569999999999999e-06 2024-09-21T15:01:06.308 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.308 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.308 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:06.309 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141871+0000", 2024-09-21T15:01:06.309 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.146e-06 2024-09-21T15:01:06.309 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.309 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.309 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:06.309 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176831+0000", 2024-09-21T15:01:06.309 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034959935999999997 2024-09-21T15:01:06.309 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.309 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.309 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:06.309 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176842+0000", 2024-09-21T15:01:06.309 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1008e-05 2024-09-21T15:01:06.309 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:06.309 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:06.309 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:06.309 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.310 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.310 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45712 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:06.310 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141874+0000", 2024-09-21T15:01:06.310 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 116.14474989199999, 2024-09-21T15:01:06.310 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034993511999999997, 2024-09-21T15:01:06.310 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:06.310 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:06.310 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:06.310 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:06.310 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:06.310 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45712 2024-09-21T15:01:06.310 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.310 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:06.310 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.311 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:06.311 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141874+0000", 2024-09-21T15:01:06.311 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:06.311 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.311 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.311 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:06.311 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141874+0000", 2024-09-21T15:01:06.311 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:06.311 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.311 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.311 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:06.311 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141874+0000", 2024-09-21T15:01:06.311 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:06.311 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.311 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.312 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:06.312 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141875+0000", 2024-09-21T15:01:06.312 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5430000000000001e-06 2024-09-21T15:01:06.312 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.312 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.312 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:06.312 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141878+0000", 2024-09-21T15:01:06.312 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1420000000000002e-06 2024-09-21T15:01:06.312 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.312 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.312 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:06.312 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141881+0000", 2024-09-21T15:01:06.312 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.807e-06 2024-09-21T15:01:06.312 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.313 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.313 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:06.313 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176860+0000", 2024-09-21T15:01:06.313 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034979243 2024-09-21T15:01:06.313 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.313 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.313 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:06.313 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176867+0000", 2024-09-21T15:01:06.313 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0169999999999999e-06 2024-09-21T15:01:06.313 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:06.313 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:06.313 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:06.313 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.313 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.313 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45713 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:06.313 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141883+0000", 2024-09-21T15:01:06.314 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 116.144740317, 2024-09-21T15:01:06.314 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035010516999999998, 2024-09-21T15:01:06.314 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:06.314 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:06.314 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:06.314 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:06.314 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:06.314 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45713 2024-09-21T15:01:06.314 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.314 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:06.314 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.314 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:06.314 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141883+0000", 2024-09-21T15:01:06.314 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:06.314 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.315 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.315 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:06.315 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141883+0000", 2024-09-21T15:01:06.315 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:06.315 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.315 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.315 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:06.315 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141883+0000", 2024-09-21T15:01:06.315 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:06.315 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.315 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.315 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:06.315 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141885+0000", 2024-09-21T15:01:06.315 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4649999999999999e-06 2024-09-21T15:01:06.316 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.316 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.316 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:06.316 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141888+0000", 2024-09-21T15:01:06.316 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3890000000000001e-06 2024-09-21T15:01:06.316 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.316 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.316 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:06.316 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141891+0000", 2024-09-21T15:01:06.316 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9239999999999999e-06 2024-09-21T15:01:06.316 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.316 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.316 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:06.316 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176887+0000", 2024-09-21T15:01:06.317 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034996370999999998 2024-09-21T15:01:06.317 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.317 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.317 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:06.317 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176894+0000", 2024-09-21T15:01:06.317 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6069999999999996e-06 2024-09-21T15:01:06.317 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:06.317 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:06.317 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:06.317 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.317 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.317 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45714 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:06.317 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141893+0000", 2024-09-21T15:01:06.317 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 116.14473040199999, 2024-09-21T15:01:06.317 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035048215000000001, 2024-09-21T15:01:06.318 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:06.318 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:06.318 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:06.318 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:06.318 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:06.318 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45714 2024-09-21T15:01:06.318 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.318 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:06.318 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.318 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:06.318 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141893+0000", 2024-09-21T15:01:06.318 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:06.318 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.318 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.319 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:06.319 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141893+0000", 2024-09-21T15:01:06.319 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:06.319 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.319 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.319 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:06.319 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141893+0000", 2024-09-21T15:01:06.319 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:06.319 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.319 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.319 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:06.319 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141895+0000", 2024-09-21T15:01:06.319 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.703e-06 2024-09-21T15:01:06.319 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.320 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.320 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:06.320 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141900+0000", 2024-09-21T15:01:06.320 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.242e-06 2024-09-21T15:01:06.320 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.320 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.320 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:06.320 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141903+0000", 2024-09-21T15:01:06.320 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.117e-06 2024-09-21T15:01:06.320 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.320 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.320 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:06.320 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176930+0000", 2024-09-21T15:01:06.320 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035026750000000002 2024-09-21T15:01:06.320 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.320 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.321 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:06.321 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176942+0000", 2024-09-21T15:01:06.321 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1668000000000001e-05 2024-09-21T15:01:06.321 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:06.321 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:06.321 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:06.321 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.321 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.321 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45715 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:06.321 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141906+0000", 2024-09-21T15:01:06.321 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 116.144718244, 2024-09-21T15:01:06.321 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035066057999999997, 2024-09-21T15:01:06.321 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:06.321 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:06.321 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:06.321 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:06.322 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:06.322 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45715 2024-09-21T15:01:06.322 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.322 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:06.322 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.322 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:06.322 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141906+0000", 2024-09-21T15:01:06.322 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:06.322 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.322 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.322 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:06.322 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141906+0000", 2024-09-21T15:01:06.322 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:06.322 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.323 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.323 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:06.323 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141905+0000", 2024-09-21T15:01:06.323 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:06.323 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.323 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.323 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:06.323 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141907+0000", 2024-09-21T15:01:06.323 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5400000000000001e-06 2024-09-21T15:01:06.323 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.323 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.323 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:06.323 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141912+0000", 2024-09-21T15:01:06.323 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.502e-06 2024-09-21T15:01:06.324 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.324 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.324 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:06.324 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141915+0000", 2024-09-21T15:01:06.324 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0170000000000001e-06 2024-09-21T15:01:06.324 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.324 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.324 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:06.324 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176964+0000", 2024-09-21T15:01:06.324 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035048587999999999 2024-09-21T15:01:06.324 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.324 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.324 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:06.324 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176972+0000", 2024-09-21T15:01:06.324 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6580000000000002e-06 2024-09-21T15:01:06.325 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:06.325 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:06.325 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:06.325 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.325 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.325 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45716 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:06.325 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141918+0000", 2024-09-21T15:01:06.325 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 116.144706068, 2024-09-21T15:01:06.325 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035086223, 2024-09-21T15:01:06.325 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:06.325 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:06.325 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:06.325 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:06.325 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:06.326 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45716 2024-09-21T15:01:06.326 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.326 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:06.326 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.326 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:06.326 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141918+0000", 2024-09-21T15:01:06.326 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:06.326 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.326 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.326 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:06.326 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141918+0000", 2024-09-21T15:01:06.326 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:06.326 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.326 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.326 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:06.327 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141917+0000", 2024-09-21T15:01:06.327 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:06.327 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.327 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.327 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:06.327 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141919+0000", 2024-09-21T15:01:06.327 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5680000000000001e-06 2024-09-21T15:01:06.327 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.327 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.327 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:06.327 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141922+0000", 2024-09-21T15:01:06.327 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2789999999999999e-06 2024-09-21T15:01:06.327 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.327 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.327 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:06.327 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141925+0000", 2024-09-21T15:01:06.328 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9469999999999999e-06 2024-09-21T15:01:06.328 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.328 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.328 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:06.328 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176997+0000", 2024-09-21T15:01:06.328 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035071366999999999 2024-09-21T15:01:06.328 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.328 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.328 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:06.328 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177004+0000", 2024-09-21T15:01:06.328 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3309999999999998e-06 2024-09-21T15:01:06.328 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:06.328 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:06.328 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:06.328 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.329 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.329 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45717 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:06.329 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141928+0000", 2024-09-21T15:01:06.329 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 116.144696122, 2024-09-21T15:01:06.329 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035102573999999998, 2024-09-21T15:01:06.329 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:06.329 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:06.329 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:06.329 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:06.329 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:06.329 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45717 2024-09-21T15:01:06.329 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.329 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:06.329 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.329 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:06.329 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141928+0000", 2024-09-21T15:01:06.330 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:06.330 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.330 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.330 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:06.330 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141928+0000", 2024-09-21T15:01:06.330 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:06.330 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.330 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.330 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:06.330 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141927+0000", 2024-09-21T15:01:06.330 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:01:06.330 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.330 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.330 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:06.331 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141929+0000", 2024-09-21T15:01:06.331 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.533e-06 2024-09-21T15:01:06.331 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.331 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.331 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:06.331 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141934+0000", 2024-09-21T15:01:06.331 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1660000000000002e-06 2024-09-21T15:01:06.331 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.331 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.331 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:06.331 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141937+0000", 2024-09-21T15:01:06.331 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0309999999999999e-06 2024-09-21T15:01:06.331 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.331 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.331 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:06.332 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177023+0000", 2024-09-21T15:01:06.332 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035085595999999997 2024-09-21T15:01:06.332 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.332 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.332 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:06.332 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177030+0000", 2024-09-21T15:01:06.332 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.486e-06 2024-09-21T15:01:06.332 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:06.332 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:06.332 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:06.332 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.332 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.332 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45718 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:06.332 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141940+0000", 2024-09-21T15:01:06.332 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 116.14468426400001, 2024-09-21T15:01:06.332 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035124956999999998, 2024-09-21T15:01:06.333 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:06.333 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:06.333 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:06.333 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:06.333 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:06.333 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45718 2024-09-21T15:01:06.333 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.333 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:06.333 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.333 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:06.333 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141940+0000", 2024-09-21T15:01:06.333 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:06.333 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.333 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.333 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:06.333 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141940+0000", 2024-09-21T15:01:06.334 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:06.334 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.334 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.334 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:06.334 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141939+0000", 2024-09-21T15:01:06.334 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:06.334 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.334 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.334 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:06.334 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141941+0000", 2024-09-21T15:01:06.334 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.502e-06 2024-09-21T15:01:06.334 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.334 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.334 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:06.335 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141946+0000", 2024-09-21T15:01:06.335 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3449999999999996e-06 2024-09-21T15:01:06.335 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.335 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.335 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:06.335 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141949+0000", 2024-09-21T15:01:06.335 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9749999999999999e-06 2024-09-21T15:01:06.335 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.335 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.335 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:06.335 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177057+0000", 2024-09-21T15:01:06.335 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035108385999999998 2024-09-21T15:01:06.335 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.335 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.336 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:06.336 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177064+0000", 2024-09-21T15:01:06.336 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0099999999999998e-06 2024-09-21T15:01:06.336 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:06.336 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:06.336 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:06.336 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.336 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.336 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45719 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:06.336 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141952+0000", 2024-09-21T15:01:06.336 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 116.144671991, 2024-09-21T15:01:06.336 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035139108000000002, 2024-09-21T15:01:06.336 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:06.336 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:06.336 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:06.337 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:06.337 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:06.337 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45719 2024-09-21T15:01:06.337 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.337 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:06.337 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.337 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:06.337 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141952+0000", 2024-09-21T15:01:06.337 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:06.337 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.337 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.337 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:06.337 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141952+0000", 2024-09-21T15:01:06.337 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:06.337 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.338 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.338 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:06.338 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141952+0000", 2024-09-21T15:01:06.338 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:06.338 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.338 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.338 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:06.338 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141953+0000", 2024-09-21T15:01:06.338 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.486e-06 2024-09-21T15:01:06.338 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.338 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.338 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:06.338 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141959+0000", 2024-09-21T15:01:06.338 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4940000000000003e-06 2024-09-21T15:01:06.338 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.339 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.339 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:06.339 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141963+0000", 2024-09-21T15:01:06.339 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3540000000000002e-06 2024-09-21T15:01:06.339 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.339 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.339 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:06.339 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177084+0000", 2024-09-21T15:01:06.339 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035120983000000001 2024-09-21T15:01:06.339 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.339 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.339 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:06.339 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177091+0000", 2024-09-21T15:01:06.340 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0350000000000004e-06 2024-09-21T15:01:06.340 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:06.340 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:06.340 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:06.340 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.340 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.340 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45720 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:06.340 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141965+0000", 2024-09-21T15:01:06.340 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 116.14465852399999, 2024-09-21T15:01:06.340 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035155268000000003, 2024-09-21T15:01:06.340 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:06.340 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:06.340 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:06.340 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:06.340 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:06.341 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45720 2024-09-21T15:01:06.341 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.341 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:06.341 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.341 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:06.341 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141965+0000", 2024-09-21T15:01:06.341 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:06.341 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.341 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.341 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:06.341 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141965+0000", 2024-09-21T15:01:06.341 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:06.341 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.341 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.341 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:06.342 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141965+0000", 2024-09-21T15:01:06.342 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:06.342 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.342 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.342 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:06.342 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141966+0000", 2024-09-21T15:01:06.342 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.46e-06 2024-09-21T15:01:06.342 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.342 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.342 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:06.342 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141972+0000", 2024-09-21T15:01:06.342 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4210000000000003e-06 2024-09-21T15:01:06.342 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.342 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.343 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:06.343 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141975+0000", 2024-09-21T15:01:06.343 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.948e-06 2024-09-21T15:01:06.343 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.343 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.343 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:06.343 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177113+0000", 2024-09-21T15:01:06.343 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035137855000000003 2024-09-21T15:01:06.343 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.343 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.343 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:06.343 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177121+0000", 2024-09-21T15:01:06.343 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8269999999999998e-06 2024-09-21T15:01:06.343 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:06.343 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:06.344 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:06.344 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.344 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.344 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45721 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:06.344 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141977+0000", 2024-09-21T15:01:06.344 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 116.144646501, 2024-09-21T15:01:06.344 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035169984000000001, 2024-09-21T15:01:06.344 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:06.344 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:06.344 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:06.344 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:06.344 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:06.344 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45721 2024-09-21T15:01:06.344 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.345 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:06.345 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.345 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:06.345 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141977+0000", 2024-09-21T15:01:06.345 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:06.345 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.345 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.345 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:06.345 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141977+0000", 2024-09-21T15:01:06.345 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:06.345 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.345 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.345 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:06.345 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141977+0000", 2024-09-21T15:01:06.345 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:06.345 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.346 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.346 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:06.346 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141979+0000", 2024-09-21T15:01:06.346 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.511e-06 2024-09-21T15:01:06.346 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.346 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.346 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:06.346 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141982+0000", 2024-09-21T15:01:06.346 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4180000000000001e-06 2024-09-21T15:01:06.346 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.346 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.346 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:06.346 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141985+0000", 2024-09-21T15:01:06.346 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9450000000000002e-06 2024-09-21T15:01:06.346 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.347 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.347 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:06.347 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177139+0000", 2024-09-21T15:01:06.347 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035154409999999997 2024-09-21T15:01:06.347 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.347 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.347 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:06.347 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177147+0000", 2024-09-21T15:01:06.347 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9570000000000002e-06 2024-09-21T15:01:06.347 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:06.347 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:06.347 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:06.347 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.347 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.348 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45722 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:06.348 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141987+0000", 2024-09-21T15:01:06.348 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 116.14463653, 2024-09-21T15:01:06.348 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035188492000000002, 2024-09-21T15:01:06.348 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:06.348 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:06.348 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:06.348 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:06.348 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:06.348 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45722 2024-09-21T15:01:06.348 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.348 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:06.348 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.348 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:06.349 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141987+0000", 2024-09-21T15:01:06.349 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:06.349 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.349 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.349 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:06.349 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141987+0000", 2024-09-21T15:01:06.349 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:06.349 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.349 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.349 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:06.349 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141987+0000", 2024-09-21T15:01:06.349 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:06.349 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.349 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.350 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:06.350 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141988+0000", 2024-09-21T15:01:06.350 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.463e-06 2024-09-21T15:01:06.350 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.350 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.350 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:06.350 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141994+0000", 2024-09-21T15:01:06.350 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.361e-06 2024-09-21T15:01:06.350 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.350 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.350 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:06.350 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141997+0000", 2024-09-21T15:01:06.350 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0359999999999998e-06 2024-09-21T15:01:06.350 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.351 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.351 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:06.351 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177169+0000", 2024-09-21T15:01:06.351 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035171651999999998 2024-09-21T15:01:06.351 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.351 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.351 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:06.351 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177176+0000", 2024-09-21T15:01:06.351 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2189999999999998e-06 2024-09-21T15:01:06.351 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:06.351 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:06.351 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:06.351 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.351 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.352 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45723 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:06.352 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141999+0000", 2024-09-21T15:01:06.352 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 116.14462436300001, 2024-09-21T15:01:06.352 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035197701999999997, 2024-09-21T15:01:06.352 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:06.352 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:06.352 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:06.352 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:06.352 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:06.352 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45723 2024-09-21T15:01:06.352 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.352 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:06.352 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.352 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:06.352 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141999+0000", 2024-09-21T15:01:06.353 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:06.353 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.353 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.353 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:06.353 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141999+0000", 2024-09-21T15:01:06.353 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:06.353 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.353 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.353 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:06.353 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141999+0000", 2024-09-21T15:01:06.353 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:06.353 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.353 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.353 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:06.353 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142003+0000", 2024-09-21T15:01:06.354 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5049999999999999e-06 2024-09-21T15:01:06.354 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.354 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.354 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:06.354 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142008+0000", 2024-09-21T15:01:06.354 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6579999999999999e-06 2024-09-21T15:01:06.354 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.354 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.354 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:06.354 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142011+0000", 2024-09-21T15:01:06.354 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0529999999999998e-06 2024-09-21T15:01:06.354 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.354 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.354 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:06.355 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177194+0000", 2024-09-21T15:01:06.355 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035182221999999999 2024-09-21T15:01:06.355 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.355 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.355 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:06.355 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177197+0000", 2024-09-21T15:01:06.355 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5030000000000002e-06 2024-09-21T15:01:06.355 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:06.355 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:06.355 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:06.355 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.355 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.355 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45724 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:06.355 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.142015+0000", 2024-09-21T15:01:06.356 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 116.144608396, 2024-09-21T15:01:06.356 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035195345000000003, 2024-09-21T15:01:06.356 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:06.356 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:06.356 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:06.356 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:06.356 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:06.356 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45724 2024-09-21T15:01:06.356 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.356 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:06.356 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.356 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:06.356 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142015+0000", 2024-09-21T15:01:06.356 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:06.356 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.357 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.357 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:06.357 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142015+0000", 2024-09-21T15:01:06.357 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:06.357 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.357 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.357 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:06.357 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142015+0000", 2024-09-21T15:01:06.357 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:06.357 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.357 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.357 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:06.357 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142017+0000", 2024-09-21T15:01:06.357 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5540000000000001e-06 2024-09-21T15:01:06.357 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.358 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.358 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:06.358 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142022+0000", 2024-09-21T15:01:06.358 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4229999999999996e-06 2024-09-21T15:01:06.358 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.358 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.358 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:06.358 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142025+0000", 2024-09-21T15:01:06.358 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.056e-06 2024-09-21T15:01:06.358 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.358 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.358 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:06.358 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177207+0000", 2024-09-21T15:01:06.358 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035182089 2024-09-21T15:01:06.359 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.359 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.359 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:06.359 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177211+0000", 2024-09-21T15:01:06.359 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4649999999999999e-06 2024-09-21T15:01:06.359 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:06.359 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:06.359 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:06.359 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.359 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.359 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45725 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:06.359 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.142028+0000", 2024-09-21T15:01:06.359 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 116.14459611700001, 2024-09-21T15:01:06.359 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035195731000000001, 2024-09-21T15:01:06.360 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:06.360 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:06.360 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:06.360 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:06.360 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:06.360 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45725 2024-09-21T15:01:06.360 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.360 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:06.360 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.360 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:06.360 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142028+0000", 2024-09-21T15:01:06.360 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:06.360 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.360 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.361 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:06.361 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142028+0000", 2024-09-21T15:01:06.361 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:06.361 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.361 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.361 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:06.361 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142027+0000", 2024-09-21T15:01:06.361 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:06.361 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.361 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.361 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:06.361 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142029+0000", 2024-09-21T15:01:06.361 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5269999999999999e-06 2024-09-21T15:01:06.361 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.361 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.362 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:06.362 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142032+0000", 2024-09-21T15:01:06.362 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3249999999999999e-06 2024-09-21T15:01:06.362 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.362 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.362 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:06.362 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142035+0000", 2024-09-21T15:01:06.362 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.897e-06 2024-09-21T15:01:06.362 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.362 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.362 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:06.362 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177220+0000", 2024-09-21T15:01:06.362 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035184871 2024-09-21T15:01:06.362 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.363 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.363 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:06.363 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177223+0000", 2024-09-21T15:01:06.363 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3680000000000002e-06 2024-09-21T15:01:06.363 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:06.363 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:06.363 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:06.363 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.363 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.363 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45726 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:06.363 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.142037+0000", 2024-09-21T15:01:06.363 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 116.144586402, 2024-09-21T15:01:06.363 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035199273000000003, 2024-09-21T15:01:06.363 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:06.364 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:06.364 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:06.364 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:06.364 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:06.364 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45726 2024-09-21T15:01:06.364 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.364 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:06.364 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.364 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:06.364 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142037+0000", 2024-09-21T15:01:06.364 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:06.364 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.364 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.364 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:06.364 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142037+0000", 2024-09-21T15:01:06.365 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:06.365 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.365 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.365 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:06.365 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142037+0000", 2024-09-21T15:01:06.365 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:06.365 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.365 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.365 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:06.365 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142038+0000", 2024-09-21T15:01:06.365 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.344e-06 2024-09-21T15:01:06.365 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.365 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.365 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:06.366 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142044+0000", 2024-09-21T15:01:06.366 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1930000000000001e-06 2024-09-21T15:01:06.366 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.366 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.366 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:06.366 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142047+0000", 2024-09-21T15:01:06.366 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0280000000000001e-06 2024-09-21T15:01:06.366 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.366 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.366 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:06.366 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177233+0000", 2024-09-21T15:01:06.366 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035186394000000003 2024-09-21T15:01:06.366 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.366 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.366 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:06.367 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177237+0000", 2024-09-21T15:01:06.367 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5530000000000001e-06 2024-09-21T15:01:06.367 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:06.367 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:06.367 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:06.367 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:06.367 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:06.367 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:06.368 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T15:01:06.369 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T15:01:06.369 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:01:06.370 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:01:06.370 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:01:06.370 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:01:06.370 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:01:06.370 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:01:06.370 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:01:06.370 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:01:06.370 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:01:06.365+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:01:06.378 INFO:teuthology.orchestra.run.smithi032.stdout:5497 write 0x106a360 thru 0x1075e91 (0xbb32 bytes) 2024-09-21T15:01:06.392 INFO:teuthology.orchestra.run.smithi032.stdout:5498 writesame 0x6f49709 thru 0x6f54bd6 data_size 0x1(0xb4ce bytes) 2024-09-21T15:01:06.412 INFO:teuthology.orchestra.run.smithi032.stdout:5499 writesame 0x7bacdf1 thru 0x7badc4d data_size 0x1(0xe5d bytes) 2024-09-21T15:01:06.420 INFO:teuthology.orchestra.run.smithi032.stdout:5500 write 0x9f6f398 thru 0x9f7b052 (0xbcbb bytes) 2024-09-21T15:01:06.423 INFO:teuthology.orchestra.run.smithi032.stdout:5501 read 0x956cd26 thru 0x957983c (0xcb17 bytes) 2024-09-21T15:01:06.426 INFO:teuthology.orchestra.run.smithi032.stdout:5502 trunc from 0xec8b8fb to 0xdb4bc0c 2024-09-21T15:01:06.438 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:01:06.438 INFO:teuthology.orchestra.run.smithi032.stdout:5504 writesame 0x5957869 thru 0x595f6b9 data_size 0x3(0x7e51 bytes) 2024-09-21T15:01:06.445 INFO:teuthology.orchestra.run.smithi032.stdout:5506 read 0xc0c0066 thru 0xc0ce090 (0xe02b bytes) 2024-09-21T15:01:06.446 INFO:teuthology.orchestra.run.smithi032.stdout:5507 write 0x3d98916 thru 0x3d9d7d5 (0x4ec0 bytes) 2024-09-21T15:01:06.451 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:01:06.451 INFO:teuthology.orchestra.run.smithi032.stdout:5509 punch from 0x4fdce5f to 0x4fe12fd, (0x449e bytes) 2024-09-21T15:01:06.451 INFO:teuthology.orchestra.run.smithi032.stdout:5510 write 0x219239c thru 0x21963e7 (0x404c bytes) 2024-09-21T15:01:06.454 INFO:teuthology.orchestra.run.smithi032.stdout:5511 write 0x820a8d thru 0x8230e9 (0x265d bytes) 2024-09-21T15:01:06.457 INFO:teuthology.orchestra.run.smithi032.stdout:5513 write 0x4d3a1fa thru 0x4d42ddd (0x8be4 bytes) 2024-09-21T15:01:06.460 INFO:teuthology.orchestra.run.smithi032.stdout:5514 read 0x991e4dd thru 0x991fb8a (0x16ae bytes) 2024-09-21T15:01:06.461 INFO:teuthology.orchestra.run.smithi032.stdout:5515 write 0x9608cbc thru 0x960dfa3 (0x52e8 bytes) 2024-09-21T15:01:06.466 INFO:teuthology.orchestra.run.smithi032.stdout:5516 write 0xba40261 thru 0xba44096 (0x3e36 bytes) 2024-09-21T15:01:06.469 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:01:06.466+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:01:06.473 INFO:teuthology.orchestra.run.smithi032.stdout:5519 writesame 0x2baf596 thru 0x2bb2df7 data_size 0x7(0x3862 bytes) 2024-09-21T15:01:06.477 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:01:06.477 INFO:teuthology.orchestra.run.smithi032.stdout:5521 write 0x51662b7 thru 0x5174d53 (0xea9d bytes) 2024-09-21T15:01:06.481 INFO:teuthology.orchestra.run.smithi032.stdout:5523 punch from 0x795354 to 0x7976fe, (0x23aa bytes) 2024-09-21T15:01:06.493 INFO:teuthology.orchestra.run.smithi032.stdout:5525 trunc from 0xdb4bc0c to 0x5e476a3 2024-09-21T15:01:06.495 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:06.495 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:01:06.495 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T15:01:06.495 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:06.506 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T15:01:06.519 INFO:teuthology.orchestra.run.smithi032.stdout:5526 punch from 0x4af0db6 to 0x4afe915, (0xdb5f bytes) 2024-09-21T15:01:06.528 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-21T15:01:06.531 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-09-21T15:01:06.536 INFO:teuthology.orchestra.run.smithi032.stdout:5527 writesame 0xbf7e826 thru 0xbf88469 data_size 0x49(0x9c44 bytes) 2024-09-21T15:01:06.537 INFO:teuthology.orchestra.run.smithi032.stdout:5530 read 0x48dee32 thru 0x48e54c2 (0x6691 bytes) 2024-09-21T15:01:06.537 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:01:06.537 INFO:teuthology.orchestra.run.smithi032.stdout:5532 write 0x120e1eb thru 0x121d04f (0xee65 bytes) 2024-09-21T15:01:06.541 INFO:teuthology.orchestra.run.smithi032.stdout:5535 write 0x32dddbf thru 0x32e3eae (0x60f0 bytes) 2024-09-21T15:01:06.544 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":264,"stamp":"2024-09-21T15:01:06.158077+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1052048706,"num_objects":1338,"num_object_clones":0,"num_object_copies":2676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1338,"num_whiteouts":0,"num_read":144511,"num_read_kb":1152527,"num_write":26194,"num_write_kb":886220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1920,"num_bytes_recovered":145354624,"num_keys_recovered":229,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17355,"ondisk_log_size":17355,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":85,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":464376,"kb_used_data":86648,"kb_used_omap":12,"kb_used_meta":377587,"kb_avail":754510344,"statfs":{"total":773094113280,"available":772618592256,"internally_reserved":0,"allocated":88727552,"data_stored":646871178,"data_compressed":14204054,"data_compressed_allocated":58728448,"data_compressed_original":631348116,"omap_allocated":12744,"internal_metadata":386649656},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,31],"upper_bound":1024},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":70414573,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1851,"num_read_kb":25577,"num_write":148,"num_write_kb":20269,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":539,"num_bytes_recovered":41668853,"num_keys_recovered":77,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.954975"},"pg_stats":[{"pgid":"3.1f","version":"172'452","reported_seq":3450,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.157805+0000","last_change":"2024-09-21T15:01:04.223131+0000","last_active":"2024-09-21T15:01:05.157805+0000","last_peered":"2024-09-21T15:01:05.157805+0000","last_clean":"2024-09-21T15:01:05.157805+0000","last_became_active":"2024-09-21T14:58:19.410055+0000","last_became_peered":"2024-09-21T14:58:19.410055+0000","last_unstale":"2024-09-21T15:01:05.157805+0000","last_undegraded":"2024-09-21T15:01:05.157805+0000","last_fullsized":"2024-09-21T15:01:05.157805+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":452,"log_dups_size":0,"ondisk_log_size":452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:49:13.944219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063336100000000004,"stat_sum":{"num_bytes":32245069,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":3403,"num_read_kb":42213,"num_write":684,"num_write_kb":34241,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.1e","version":"176'366","reported_seq":2962,"reported_epoch":177,"state":"peering","last_fresh":"2024-09-21T15:01:05.160058+0000","last_change":"2024-09-21T15:01:05.160420+0000","last_active":"2024-09-21T15:01:05.160058+0000","last_peered":"2024-09-21T15:01:04.425541+0000","last_clean":"2024-09-21T15:01:00.453521+0000","last_became_active":"2024-09-21T15:01:03.253387+0000","last_became_peered":"2024-09-21T15:01:03.253387+0000","last_unstale":"2024-09-21T15:01:05.160058+0000","last_undegraded":"2024-09-21T15:01:05.160058+0000","last_fullsized":"2024-09-21T15:01:05.160058+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":366,"log_dups_size":0,"ondisk_log_size":366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:27:07.402534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021611780000000001,"stat_sum":{"num_bytes":19327265,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":0,"num_read":2999,"num_read_kb":22037,"num_write":453,"num_write_kb":16509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":148,"num_bytes_recovered":11635695,"num_keys_recovered":22,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,3],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"6b","length":"1"}]},{"pgid":"3.1d","version":"175'260","reported_seq":3611,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.164700+0000","last_change":"2024-09-21T15:01:04.229475+0000","last_active":"2024-09-21T15:01:05.164700+0000","last_peered":"2024-09-21T15:01:05.164700+0000","last_clean":"2024-09-21T15:01:05.164700+0000","last_became_active":"2024-09-21T14:58:19.411387+0000","last_became_peered":"2024-09-21T14:58:19.411387+0000","last_unstale":"2024-09-21T15:01:05.164700+0000","last_undegraded":"2024-09-21T15:01:05.164700+0000","last_fullsized":"2024-09-21T15:01:05.164700+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":260,"log_dups_size":0,"ondisk_log_size":260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:37:57.597069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054775500000000003,"stat_sum":{"num_bytes":18081929,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":3016,"num_read_kb":23636,"num_write":434,"num_write_kb":24977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.1c","version":"175'430","reported_seq":2874,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.187232+0000","last_change":"2024-09-21T15:01:04.234927+0000","last_active":"2024-09-21T15:01:05.187232+0000","last_peered":"2024-09-21T15:01:05.187232+0000","last_clean":"2024-09-21T15:01:05.187232+0000","last_became_active":"2024-09-21T14:58:19.411306+0000","last_became_peered":"2024-09-21T14:58:19.411306+0000","last_unstale":"2024-09-21T15:01:05.187232+0000","last_undegraded":"2024-09-21T15:01:05.187232+0000","last_fullsized":"2024-09-21T15:01:05.187232+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":430,"log_dups_size":0,"ondisk_log_size":430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:20:27.079795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010334350000000001,"stat_sum":{"num_bytes":19507114,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":2814,"num_read_kb":20995,"num_write":622,"num_write_kb":24708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.1b","version":"175'422","reported_seq":2730,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.158957+0000","last_change":"2024-09-21T15:01:04.227802+0000","last_active":"2024-09-21T15:01:05.158957+0000","last_peered":"2024-09-21T15:01:05.158957+0000","last_clean":"2024-09-21T15:01:05.158957+0000","last_became_active":"2024-09-21T14:59:31.131051+0000","last_became_peered":"2024-09-21T14:59:31.131051+0000","last_unstale":"2024-09-21T15:01:05.158957+0000","last_undegraded":"2024-09-21T15:01:05.158957+0000","last_fullsized":"2024-09-21T15:01:05.158957+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":422,"log_dups_size":0,"ondisk_log_size":422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:21:37.320774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001254673,"stat_sum":{"num_bytes":25447559,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":2614,"num_read_kb":28976,"num_write":559,"num_write_kb":20747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":117,"num_bytes_recovered":7757116,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.1a","version":"175'352","reported_seq":3233,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.187249+0000","last_change":"2024-09-21T15:01:04.234672+0000","last_active":"2024-09-21T15:01:05.187249+0000","last_peered":"2024-09-21T15:01:05.187249+0000","last_clean":"2024-09-21T15:01:05.187249+0000","last_became_active":"2024-09-21T14:58:19.409415+0000","last_became_peered":"2024-09-21T14:58:19.409415+0000","last_unstale":"2024-09-21T15:01:05.187249+0000","last_undegraded":"2024-09-21T15:01:05.187249+0000","last_fullsized":"2024-09-21T15:01:05.187249+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":352,"log_dups_size":0,"ondisk_log_size":352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:28:46.336301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060307399999999999,"stat_sum":{"num_bytes":31396202,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":3008,"num_read_kb":24746,"num_write":475,"num_write_kb":22526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.19","version":"175'648","reported_seq":3837,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.159011+0000","last_change":"2024-09-21T15:01:04.228157+0000","last_active":"2024-09-21T15:01:05.159011+0000","last_peered":"2024-09-21T15:01:05.159011+0000","last_clean":"2024-09-21T15:01:05.159011+0000","last_became_active":"2024-09-21T14:59:32.130624+0000","last_became_peered":"2024-09-21T14:59:32.130624+0000","last_unstale":"2024-09-21T15:01:05.159011+0000","last_undegraded":"2024-09-21T15:01:05.159011+0000","last_fullsized":"2024-09-21T15:01:05.159011+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":648,"log_dups_size":0,"ondisk_log_size":648,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:37:53.312733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001540409,"stat_sum":{"num_bytes":47896216,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":4950,"num_read_kb":51571,"num_write":1030,"num_write_kb":28173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":128,"num_bytes_recovered":11896992,"num_keys_recovered":44,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.18","version":"175'525","reported_seq":7399,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.157243+0000","last_change":"2024-09-21T15:01:04.223390+0000","last_active":"2024-09-21T15:01:05.157243+0000","last_peered":"2024-09-21T15:01:05.157243+0000","last_clean":"2024-09-21T15:01:05.157243+0000","last_became_active":"2024-09-21T14:58:19.409371+0000","last_became_peered":"2024-09-21T14:58:19.409371+0000","last_unstale":"2024-09-21T15:01:05.157243+0000","last_undegraded":"2024-09-21T15:01:05.157243+0000","last_fullsized":"2024-09-21T15:01:05.157243+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":525,"log_dups_size":0,"ondisk_log_size":525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:35:27.143445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076656300000000005,"stat_sum":{"num_bytes":36011050,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":8009,"num_read_kb":41151,"num_write":765,"num_write_kb":29537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.17","version":"172'334","reported_seq":3678,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.158315+0000","last_change":"2024-09-21T15:01:04.225804+0000","last_active":"2024-09-21T15:01:05.158315+0000","last_peered":"2024-09-21T15:01:05.158315+0000","last_clean":"2024-09-21T15:01:05.158315+0000","last_became_active":"2024-09-21T14:58:19.410306+0000","last_became_peered":"2024-09-21T14:58:19.410306+0000","last_unstale":"2024-09-21T15:01:05.158315+0000","last_undegraded":"2024-09-21T15:01:05.158315+0000","last_fullsized":"2024-09-21T15:01:05.158315+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":334,"log_dups_size":0,"ondisk_log_size":334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:33:50.089165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030273969999999998,"stat_sum":{"num_bytes":25072080,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":3370,"num_read_kb":27721,"num_write":557,"num_write_kb":31849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.16","version":"175'887","reported_seq":4751,"reported_epoch":177,"state":"creating+peering","last_fresh":"2024-09-21T15:01:04.608919+0000","last_change":"2024-09-21T15:01:05.877830+0000","last_active":"2024-09-21T15:01:05.877830+0000","last_peered":"2024-09-21T15:01:04.608919+0000","last_clean":"2024-09-21T15:01:00.518248+0000","last_became_active":"2024-09-21T15:01:03.252683+0000","last_became_peered":"2024-09-21T15:01:03.252683+0000","last_unstale":"2024-09-21T15:01:04.608919+0000","last_undegraded":"2024-09-21T15:01:04.608919+0000","last_fullsized":"2024-09-21T15:01:03.215921+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":887,"log_dups_size":0,"ondisk_log_size":887,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:19:04.988784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001973472,"stat_sum":{"num_bytes":33667875,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":6806,"num_read_kb":40610,"num_write":1387,"num_write_kb":26337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":271,"num_bytes_recovered":16821266,"num_keys_recovered":55,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"6b","length":"1"}]},{"pgid":"3.15","version":"175'717","reported_seq":3617,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.169730+0000","last_change":"2024-09-21T15:01:04.231959+0000","last_active":"2024-09-21T15:01:05.169730+0000","last_peered":"2024-09-21T15:01:05.169730+0000","last_clean":"2024-09-21T15:01:05.169730+0000","last_became_active":"2024-09-21T14:58:19.410687+0000","last_became_peered":"2024-09-21T14:58:19.410687+0000","last_unstale":"2024-09-21T15:01:05.169730+0000","last_undegraded":"2024-09-21T15:01:05.169730+0000","last_fullsized":"2024-09-21T15:01:05.169730+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":717,"log_dups_size":0,"ondisk_log_size":717,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:21:42.226982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033742220000000001,"stat_sum":{"num_bytes":45321568,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":4682,"num_read_kb":48927,"num_write":1212,"num_write_kb":32331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.14","version":"175'433","reported_seq":2993,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.164396+0000","last_change":"2024-09-21T15:01:04.229559+0000","last_active":"2024-09-21T15:01:05.164396+0000","last_peered":"2024-09-21T15:01:05.164396+0000","last_clean":"2024-09-21T15:01:05.164396+0000","last_became_active":"2024-09-21T14:58:19.407310+0000","last_became_peered":"2024-09-21T14:58:19.407310+0000","last_unstale":"2024-09-21T15:01:05.164396+0000","last_undegraded":"2024-09-21T15:01:05.164396+0000","last_fullsized":"2024-09-21T15:01:05.164396+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:47:46.079115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052455500000000001,"stat_sum":{"num_bytes":43744336,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":2858,"num_read_kb":38772,"num_write":581,"num_write_kb":32120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":171,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-21T15:01:04.229713+0000","last_change":"2024-09-21T14:58:11.348177+0000","last_active":"2024-09-21T15:01:04.229713+0000","last_peered":"2024-09-21T15:01:04.229713+0000","last_clean":"2024-09-21T15:01:04.229713+0000","last_became_active":"2024-09-21T14:58:09.328417+0000","last_became_peered":"2024-09-21T14:58:09.328417+0000","last_unstale":"2024-09-21T15:01:04.229713+0000","last_undegraded":"2024-09-21T15:01:04.229713+0000","last_fullsized":"2024-09-21T15:01:04.229713+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:52:22.671185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018917490000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"172'367","reported_seq":5379,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.158972+0000","last_change":"2024-09-21T15:01:04.228240+0000","last_active":"2024-09-21T15:01:05.158972+0000","last_peered":"2024-09-21T15:01:05.158972+0000","last_clean":"2024-09-21T15:01:05.158972+0000","last_became_active":"2024-09-21T14:59:32.131232+0000","last_became_peered":"2024-09-21T14:59:32.131232+0000","last_unstale":"2024-09-21T15:01:05.158972+0000","last_undegraded":"2024-09-21T15:01:05.158972+0000","last_fullsized":"2024-09-21T15:01:05.158972+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":367,"log_dups_size":0,"ondisk_log_size":367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:31:04.421782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001438581,"stat_sum":{"num_bytes":50093146,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":5172,"num_read_kb":51612,"num_write":497,"num_write_kb":25669,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":3585713,"num_keys_recovered":9,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":96,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-21T15:01:04.227015+0000","last_change":"2024-09-21T14:59:28.228883+0000","last_active":"2024-09-21T15:01:04.227015+0000","last_peered":"2024-09-21T15:01:04.227015+0000","last_clean":"2024-09-21T15:01:04.227015+0000","last_became_active":"2024-09-21T14:59:28.228464+0000","last_became_peered":"2024-09-21T14:59:28.228464+0000","last_unstale":"2024-09-21T15:01:04.227015+0000","last_undegraded":"2024-09-21T15:01:04.227015+0000","last_fullsized":"2024-09-21T15:01:04.227015+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:00:26.310864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"175'809","reported_seq":3401,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.159556+0000","last_change":"2024-09-21T15:01:04.228668+0000","last_active":"2024-09-21T15:01:05.159556+0000","last_peered":"2024-09-21T15:01:05.159556+0000","last_clean":"2024-09-21T15:01:05.159556+0000","last_became_active":"2024-09-21T14:59:31.132581+0000","last_became_peered":"2024-09-21T14:59:31.132581+0000","last_unstale":"2024-09-21T15:01:05.159556+0000","last_undegraded":"2024-09-21T15:01:05.159556+0000","last_fullsized":"2024-09-21T15:01:05.159556+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":809,"log_dups_size":0,"ondisk_log_size":809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:36:45.467515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016620179999999999,"stat_sum":{"num_bytes":28736355,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":4991,"num_read_kb":30499,"num_write":1191,"num_write_kb":23913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":119,"num_bytes_recovered":6871924,"num_keys_recovered":11,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":187,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-21T15:01:04.229145+0000","last_change":"2024-09-21T14:59:28.231502+0000","last_active":"2024-09-21T15:01:04.229145+0000","last_peered":"2024-09-21T15:01:04.229145+0000","last_clean":"2024-09-21T15:01:04.229145+0000","last_became_active":"2024-09-21T14:59:28.231043+0000","last_became_peered":"2024-09-21T14:59:28.231043+0000","last_unstale":"2024-09-21T15:01:04.229145+0000","last_undegraded":"2024-09-21T15:01:04.229145+0000","last_fullsized":"2024-09-21T15:01:04.229145+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:09:46.773690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019338770000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"172'553","reported_seq":5095,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.158909+0000","last_change":"2024-09-21T15:01:04.227811+0000","last_active":"2024-09-21T15:01:05.158909+0000","last_peered":"2024-09-21T15:01:05.158909+0000","last_clean":"2024-09-21T15:01:05.158909+0000","last_became_active":"2024-09-21T15:00:34.347238+0000","last_became_peered":"2024-09-21T15:00:34.347238+0000","last_unstale":"2024-09-21T15:01:05.158909+0000","last_undegraded":"2024-09-21T15:01:05.158909+0000","last_fullsized":"2024-09-21T15:01:05.158909+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":553,"log_dups_size":0,"ondisk_log_size":553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:31:33.639268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001277607,"stat_sum":{"num_bytes":39912228,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":5081,"num_read_kb":38156,"num_write":939,"num_write_kb":39278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":14049436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":172,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-21T15:01:04.222407+0000","last_change":"2024-09-21T14:58:11.406176+0000","last_active":"2024-09-21T15:01:04.222407+0000","last_peered":"2024-09-21T15:01:04.222407+0000","last_clean":"2024-09-21T15:01:04.222407+0000","last_became_active":"2024-09-21T14:58:09.777465+0000","last_became_peered":"2024-09-21T14:58:09.777465+0000","last_unstale":"2024-09-21T15:01:04.222407+0000","last_undegraded":"2024-09-21T15:01:04.222407+0000","last_fullsized":"2024-09-21T15:01:04.222407+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:55:04.667816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095704000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"172'662","reported_seq":2540,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.162165+0000","last_change":"2024-09-21T15:01:04.224786+0000","last_active":"2024-09-21T15:01:05.162165+0000","last_peered":"2024-09-21T15:01:05.162165+0000","last_clean":"2024-09-21T15:01:05.162165+0000","last_became_active":"2024-09-21T14:58:19.408971+0000","last_became_peered":"2024-09-21T14:58:19.408971+0000","last_unstale":"2024-09-21T15:01:05.162165+0000","last_undegraded":"2024-09-21T15:01:05.162165+0000","last_fullsized":"2024-09-21T15:01:05.162165+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":662,"log_dups_size":0,"ondisk_log_size":662,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:29:52.642121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000380054,"stat_sum":{"num_bytes":19474374,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":3561,"num_read_kb":20372,"num_write":1084,"num_write_kb":19812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.3","version":"172'738","reported_seq":2939,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.169226+0000","last_change":"2024-09-21T15:01:04.231096+0000","last_active":"2024-09-21T15:01:05.169226+0000","last_peered":"2024-09-21T15:01:05.169226+0000","last_clean":"2024-09-21T15:01:05.169226+0000","last_became_active":"2024-09-21T14:58:19.417453+0000","last_became_peered":"2024-09-21T14:58:19.417453+0000","last_unstale":"2024-09-21T15:01:05.169226+0000","last_undegraded":"2024-09-21T15:01:05.169226+0000","last_fullsized":"2024-09-21T15:01:05.169226+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":738,"log_dups_size":0,"ondisk_log_size":738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:19:27.087272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002785708,"stat_sum":{"num_bytes":15820678,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":4393,"num_read_kb":24172,"num_write":1224,"num_write_kb":20113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":173,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-21T15:01:04.232215+0000","last_change":"2024-09-21T14:58:11.347504+0000","last_active":"2024-09-21T15:01:04.232215+0000","last_peered":"2024-09-21T15:01:04.232215+0000","last_clean":"2024-09-21T15:01:04.232215+0000","last_became_active":"2024-09-21T14:58:09.775981+0000","last_became_peered":"2024-09-21T14:58:09.775981+0000","last_unstale":"2024-09-21T15:01:04.232215+0000","last_undegraded":"2024-09-21T15:01:04.232215+0000","last_fullsized":"2024-09-21T15:01:04.232215+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:47:16.286652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012568880000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"176'534","reported_seq":2983,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.157022+0000","last_change":"2024-09-21T15:01:04.223338+0000","last_active":"2024-09-21T15:01:05.157022+0000","last_peered":"2024-09-21T15:01:05.157022+0000","last_clean":"2024-09-21T15:01:05.157022+0000","last_became_active":"2024-09-21T14:58:19.405811+0000","last_became_peered":"2024-09-21T14:58:19.405811+0000","last_unstale":"2024-09-21T15:01:05.157022+0000","last_undegraded":"2024-09-21T15:01:05.157022+0000","last_fullsized":"2024-09-21T15:01:05.157022+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":534,"log_dups_size":0,"ondisk_log_size":534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:36:25.551607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083259800000000004,"stat_sum":{"num_bytes":20501650,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":3805,"num_read_kb":26326,"num_write":700,"num_write_kb":25158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":168,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-21T15:01:00.171240+0000","last_change":"2024-09-21T14:58:11.342210+0000","last_active":"2024-09-21T15:01:00.171240+0000","last_peered":"2024-09-21T15:01:00.171240+0000","last_clean":"2024-09-21T15:01:00.171240+0000","last_became_active":"2024-09-21T14:58:09.330624+0000","last_became_peered":"2024-09-21T14:58:09.330624+0000","last_unstale":"2024-09-21T15:01:00.171240+0000","last_undegraded":"2024-09-21T15:01:00.171240+0000","last_fullsized":"2024-09-21T15:01:00.171240+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:29:43.229538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001051151,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"175'276","reported_seq":2007,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.166575+0000","last_change":"2024-09-21T15:01:05.166575+0000","last_active":"2024-09-21T15:01:05.166575+0000","last_peered":"2024-09-21T15:01:05.166575+0000","last_clean":"2024-09-21T15:01:05.166575+0000","last_became_active":"2024-09-21T15:01:05.164862+0000","last_became_peered":"2024-09-21T15:01:05.164862+0000","last_unstale":"2024-09-21T15:01:05.166575+0000","last_undegraded":"2024-09-21T15:01:05.166575+0000","last_fullsized":"2024-09-21T15:01:05.166575+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":276,"log_dups_size":0,"ondisk_log_size":276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:57:46.675335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046340299999999998,"stat_sum":{"num_bytes":42467012,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":2366,"num_read_kb":39964,"num_write":407,"num_write_kb":39026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":254,"num_bytes_recovered":20571253,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":171,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-21T15:01:04.227843+0000","last_change":"2024-09-21T14:58:11.346766+0000","last_active":"2024-09-21T15:01:04.227843+0000","last_peered":"2024-09-21T15:01:04.227843+0000","last_clean":"2024-09-21T15:01:04.227843+0000","last_became_active":"2024-09-21T14:58:09.329748+0000","last_became_peered":"2024-09-21T14:58:09.329748+0000","last_unstale":"2024-09-21T15:01:04.227843+0000","last_undegraded":"2024-09-21T15:01:04.227843+0000","last_fullsized":"2024-09-21T15:01:04.227843+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:31:20.267675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052698000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"175'715","reported_seq":5700,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.160742+0000","last_change":"2024-09-21T15:01:04.224483+0000","last_active":"2024-09-21T15:01:05.160742+0000","last_peered":"2024-09-21T15:01:05.160742+0000","last_clean":"2024-09-21T15:01:05.160742+0000","last_became_active":"2024-09-21T14:58:41.401003+0000","last_became_peered":"2024-09-21T14:58:41.401003+0000","last_unstale":"2024-09-21T15:01:05.160742+0000","last_undegraded":"2024-09-21T15:01:05.160742+0000","last_fullsized":"2024-09-21T15:01:05.160742+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":715,"log_dups_size":0,"ondisk_log_size":715,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:05:37.644835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046693700000000001,"stat_sum":{"num_bytes":45819535,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":7513,"num_read_kb":42793,"num_write":987,"num_write_kb":21873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1403348,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":172,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-21T15:01:04.228858+0000","last_change":"2024-09-21T14:58:11.347700+0000","last_active":"2024-09-21T15:01:04.228858+0000","last_peered":"2024-09-21T15:01:04.228858+0000","last_clean":"2024-09-21T15:01:04.228858+0000","last_became_active":"2024-09-21T14:58:09.438842+0000","last_became_peered":"2024-09-21T14:58:09.438842+0000","last_unstale":"2024-09-21T15:01:04.228858+0000","last_undegraded":"2024-09-21T15:01:04.228858+0000","last_fullsized":"2024-09-21T15:01:04.228858+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:41:34.626249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001426194,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":320,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-21T15:01:04.233768+0000","last_change":"2024-09-21T14:58:03.616055+0000","last_active":"2024-09-21T15:01:04.233768+0000","last_peered":"2024-09-21T15:01:04.233768+0000","last_clean":"2024-09-21T15:01:04.233768+0000","last_became_active":"2024-09-21T14:58:03.613214+0000","last_became_peered":"2024-09-21T14:58:03.613214+0000","last_unstale":"2024-09-21T15:01:04.233768+0000","last_undegraded":"2024-09-21T15:01:04.233768+0000","last_fullsized":"2024-09-21T15:01:04.233768+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:02.180162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:02.180162+0000","last_clean_scrub_stamp":"2024-09-21T14:58:02.180162+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:29:05.860361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"172'602","reported_seq":4041,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.157772+0000","last_change":"2024-09-21T15:01:04.227318+0000","last_active":"2024-09-21T15:01:05.157772+0000","last_peered":"2024-09-21T15:01:05.157772+0000","last_clean":"2024-09-21T15:01:05.157772+0000","last_became_active":"2024-09-21T14:59:31.131460+0000","last_became_peered":"2024-09-21T14:59:31.131460+0000","last_unstale":"2024-09-21T15:01:05.157772+0000","last_undegraded":"2024-09-21T15:01:05.157772+0000","last_fullsized":"2024-09-21T15:01:05.157772+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":602,"log_dups_size":0,"ondisk_log_size":602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:01:38.353476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098975100000000009,"stat_sum":{"num_bytes":37289213,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":4356,"num_read_kb":43318,"num_write":990,"num_write_kb":29471,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":119,"num_bytes_recovered":6097618,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.9","version":"173'786","reported_seq":2989,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.164259+0000","last_change":"2024-09-21T15:01:04.232710+0000","last_active":"2024-09-21T15:01:05.164259+0000","last_peered":"2024-09-21T15:01:05.164259+0000","last_clean":"2024-09-21T15:01:05.164259+0000","last_became_active":"2024-09-21T14:59:28.233956+0000","last_became_peered":"2024-09-21T14:59:28.233956+0000","last_unstale":"2024-09-21T15:01:05.164259+0000","last_undegraded":"2024-09-21T15:01:05.164259+0000","last_fullsized":"2024-09-21T15:01:05.164259+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":786,"log_dups_size":0,"ondisk_log_size":786,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:29:09.682792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027159770000000001,"stat_sum":{"num_bytes":23720917,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":4523,"num_read_kb":33253,"num_write":1276,"num_write_kb":30634,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":91,"num_bytes_recovered":6925016,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.a","version":"176'786","reported_seq":3964,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.187273+0000","last_change":"2024-09-21T15:01:04.234662+0000","last_active":"2024-09-21T15:01:05.187273+0000","last_peered":"2024-09-21T15:01:05.187273+0000","last_clean":"2024-09-21T15:01:05.187273+0000","last_became_active":"2024-09-21T15:00:27.280971+0000","last_became_peered":"2024-09-21T15:00:27.280971+0000","last_unstale":"2024-09-21T15:01:05.187273+0000","last_undegraded":"2024-09-21T15:01:05.187273+0000","last_fullsized":"2024-09-21T15:01:05.187273+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":786,"log_dups_size":0,"ondisk_log_size":786,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:26:07.617059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076445899999999999,"stat_sum":{"num_bytes":20136068,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":5765,"num_read_kb":29533,"num_write":1129,"num_write_kb":31048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":9228123,"num_keys_recovered":33,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.b","version":"172'448","reported_seq":2919,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-21T15:01:00.700977+0000","last_change":"2024-09-21T15:00:59.287225+0000","last_active":"2024-09-21T15:01:00.700977+0000","last_peered":"2024-09-21T15:01:00.700977+0000","last_clean":"2024-09-21T15:01:00.700977+0000","last_became_active":"2024-09-21T14:58:19.410679+0000","last_became_peered":"2024-09-21T14:58:19.410679+0000","last_unstale":"2024-09-21T15:01:00.700977+0000","last_undegraded":"2024-09-21T15:01:00.700977+0000","last_fullsized":"2024-09-21T15:01:00.700977+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":448,"log_dups_size":0,"ondisk_log_size":448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:07:40.758932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0009774880000000001,"stat_sum":{"num_bytes":31436179,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":2840,"num_read_kb":35254,"num_write":697,"num_write_kb":37108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"6b","length":"1"}]},{"pgid":"3.c","version":"172'408","reported_seq":4312,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.169078+0000","last_change":"2024-09-21T15:01:04.231357+0000","last_active":"2024-09-21T15:01:05.169078+0000","last_peered":"2024-09-21T15:01:05.169078+0000","last_clean":"2024-09-21T15:01:05.169078+0000","last_became_active":"2024-09-21T14:59:25.130057+0000","last_became_peered":"2024-09-21T14:59:25.130057+0000","last_unstale":"2024-09-21T15:01:05.169078+0000","last_undegraded":"2024-09-21T15:01:05.169078+0000","last_fullsized":"2024-09-21T15:01:05.169078+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:53:33.752550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003377793,"stat_sum":{"num_bytes":22999993,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":4337,"num_read_kb":28452,"num_write":576,"num_write_kb":23757,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":9059600,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.d","version":"172'422","reported_seq":4914,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.164463+0000","last_change":"2024-09-21T15:01:04.233218+0000","last_active":"2024-09-21T15:01:05.164463+0000","last_peered":"2024-09-21T15:01:05.164463+0000","last_clean":"2024-09-21T15:01:05.164463+0000","last_became_active":"2024-09-21T14:58:19.407884+0000","last_became_peered":"2024-09-21T14:58:19.407884+0000","last_unstale":"2024-09-21T15:01:05.164463+0000","last_undegraded":"2024-09-21T15:01:05.164463+0000","last_fullsized":"2024-09-21T15:01:05.164463+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":422,"log_dups_size":0,"ondisk_log_size":422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:28:57.111615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034473960000000001,"stat_sum":{"num_bytes":49032397,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":4783,"num_read_kb":55414,"num_write":689,"num_write_kb":35805,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.e","version":"171'774","reported_seq":2852,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.176277+0000","last_change":"2024-09-21T15:01:05.176277+0000","last_active":"2024-09-21T15:01:05.176277+0000","last_peered":"2024-09-21T15:01:05.176277+0000","last_clean":"2024-09-21T15:01:05.176277+0000","last_became_active":"2024-09-21T15:01:05.172427+0000","last_became_peered":"2024-09-21T15:01:05.172427+0000","last_unstale":"2024-09-21T15:01:05.176277+0000","last_undegraded":"2024-09-21T15:01:05.176277+0000","last_fullsized":"2024-09-21T15:01:05.176277+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":774,"log_dups_size":0,"ondisk_log_size":774,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:28:18.710621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093917399999999998,"stat_sum":{"num_bytes":12028520,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":4155,"num_read_kb":16497,"num_write":1272,"num_write_kb":16419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":138,"num_bytes_recovered":7486496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.f","version":"175'427","reported_seq":5010,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.169470+0000","last_change":"2024-09-21T15:01:04.229660+0000","last_active":"2024-09-21T15:01:05.169470+0000","last_peered":"2024-09-21T15:01:05.169470+0000","last_clean":"2024-09-21T15:01:05.169470+0000","last_became_active":"2024-09-21T14:58:19.410406+0000","last_became_peered":"2024-09-21T14:58:19.410406+0000","last_unstale":"2024-09-21T15:01:05.169470+0000","last_undegraded":"2024-09-21T15:01:05.169470+0000","last_fullsized":"2024-09-21T15:01:05.169470+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":427,"log_dups_size":0,"ondisk_log_size":427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:59:16.026939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014556770000000001,"stat_sum":{"num_bytes":49430534,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":4917,"num_read_kb":52382,"num_write":667,"num_write_kb":35936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.10","version":"172'427","reported_seq":4297,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.186765+0000","last_change":"2024-09-21T15:01:04.233986+0000","last_active":"2024-09-21T15:01:05.186765+0000","last_peered":"2024-09-21T15:01:05.186765+0000","last_clean":"2024-09-21T15:01:05.186765+0000","last_became_active":"2024-09-21T14:58:19.409935+0000","last_became_peered":"2024-09-21T14:58:19.409935+0000","last_unstale":"2024-09-21T15:01:05.186765+0000","last_undegraded":"2024-09-21T15:01:05.186765+0000","last_fullsized":"2024-09-21T15:01:05.186765+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":427,"log_dups_size":0,"ondisk_log_size":427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:10:36.252629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001679137,"stat_sum":{"num_bytes":49580759,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":3927,"num_read_kb":58252,"num_write":670,"num_write_kb":35408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.11","version":"172'457","reported_seq":4563,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.168682+0000","last_change":"2024-09-21T15:01:05.168682+0000","last_active":"2024-09-21T15:01:05.168682+0000","last_peered":"2024-09-21T15:01:05.168682+0000","last_clean":"2024-09-21T15:01:05.168682+0000","last_became_active":"2024-09-21T15:01:05.165955+0000","last_became_peered":"2024-09-21T15:01:05.165955+0000","last_unstale":"2024-09-21T15:01:05.168682+0000","last_undegraded":"2024-09-21T15:01:05.168682+0000","last_fullsized":"2024-09-21T15:01:05.168682+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":457,"log_dups_size":0,"ondisk_log_size":457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:34:54.111119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010096230000000001,"stat_sum":{"num_bytes":23766643,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":5218,"num_read_kb":27936,"num_write":524,"num_write_kb":22561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":166,"num_bytes_recovered":11965028,"num_keys_recovered":33,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.12","version":"176'480","reported_seq":6284,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.163311+0000","last_change":"2024-09-21T15:01:04.224935+0000","last_active":"2024-09-21T15:01:05.163311+0000","last_peered":"2024-09-21T15:01:05.163311+0000","last_clean":"2024-09-21T15:01:05.163311+0000","last_became_active":"2024-09-21T15:00:07.155144+0000","last_became_peered":"2024-09-21T15:00:07.155144+0000","last_unstale":"2024-09-21T15:01:05.163311+0000","last_undegraded":"2024-09-21T15:01:05.163311+0000","last_fullsized":"2024-09-21T15:01:05.163311+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":480,"log_dups_size":0,"ondisk_log_size":480,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:58:00.133639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010497799999999999,"stat_sum":{"num_bytes":38476413,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":7022,"num_read_kb":30524,"num_write":743,"num_write_kb":18554,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.13","version":"175'738","reported_seq":5364,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.164109+0000","last_change":"2024-09-21T15:01:04.229229+0000","last_active":"2024-09-21T15:01:05.164109+0000","last_peered":"2024-09-21T15:01:05.164109+0000","last_clean":"2024-09-21T15:01:05.164109+0000","last_became_active":"2024-09-21T14:58:19.412030+0000","last_became_peered":"2024-09-21T14:58:19.412030+0000","last_unstale":"2024-09-21T15:01:05.164109+0000","last_undegraded":"2024-09-21T15:01:05.164109+0000","last_fullsized":"2024-09-21T15:01:05.164109+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":738,"log_dups_size":0,"ondisk_log_size":738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:44:29.282391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046048699999999998,"stat_sum":{"num_bytes":53017442,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":7141,"num_read_kb":56363,"num_write":1012,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"6e","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1051458319,"num_objects":1334,"num_object_clones":0,"num_object_copies":2668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1334,"num_whiteouts":0,"num_read":144395,"num_read_kb":1152427,"num_write":26033,"num_write_kb":883802,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1920,"num_bytes_recovered":145354624,"num_keys_recovered":229,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":71651328,"data_stored":637465472,"data_compressed":14193650,"data_compressed_allocated":58654720,"data_compressed_original":630201236,"omap_allocated":0,"internal_metadata":0},"log_size":17235,"ondisk_log_size":17235,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":10404,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509480,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34572,"kb_used_data":7552,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94337268,"statfs":{"total":96636764160,"available":96601362432,"internally_reserved":0,"allocated":7733248,"data_stored":48875988,"data_compressed":1141276,"data_compressed_allocated":4558848,"data_compressed_original":47268443,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:00:03 2024","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.84499999999999997,"15min":0.83899999999999997},"min":{"1min":0.504,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.95099999999999996,"15min":0.95799999999999996},"min":{"1min":0.51400000000000001,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.4970000000000001,"5min":1.917,"15min":1.917},"last":0.80200000000000005}]},{"osd":1,"last update":"Sat Sep 21 15:00:03 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.78500000000000003,"15min":0.76900000000000002},"min":{"1min":0.626,"5min":0.312,"15min":0.312},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.879,"5min":0.83499999999999996,"15min":0.82699999999999996},"min":{"1min":0.53600000000000003,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.224,"5min":2.2149999999999999,"15min":2.2149999999999999},"last":0.70199999999999996}]},{"osd":2,"last update":"Sat Sep 21 15:00:03 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.81699999999999995,"15min":0.81100000000000005},"min":{"1min":0.495,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.1859999999999999,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.95099999999999996,"15min":0.94799999999999995},"min":{"1min":0.72199999999999998,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.3300000000000001,"5min":1.466,"15min":1.466},"last":0.77800000000000002}]},{"osd":3,"last update":"Sat Sep 21 15:00:03 2024","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.86199999999999999,"15min":0.85399999999999998},"min":{"1min":0.67200000000000004,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.3120000000000001,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":0.82099999999999995},{"interface":"front","average":{"1min":1.0429999999999999,"5min":0.97999999999999998,"15min":0.96999999999999997},"min":{"1min":0.68500000000000005,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.244,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.98399999999999999}]},{"osd":4,"last update":"Sat Sep 21 15:00:03 2024","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.93700000000000006,"15min":0.93799999999999994},"min":{"1min":0.64200000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.4510000000000001,"5min":2.7130000000000001,"15min":2.7130000000000001},"last":0.59699999999999998},{"interface":"front","average":{"1min":1.0369999999999999,"5min":0.94099999999999995,"15min":0.92500000000000004},"min":{"1min":0.59999999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.6499999999999999,"5min":2.125,"15min":2.125},"last":0.75700000000000001}]},{"osd":5,"last update":"Sat Sep 21 15:00:03 2024","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.0409999999999999,"15min":1.0269999999999999},"min":{"1min":0.71099999999999997,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.4770000000000001,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":1.079},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.0640000000000001,"15min":1.0580000000000001},"min":{"1min":0.70799999999999996,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.8169999999999999,"5min":3.1880000000000002,"15min":3.1880000000000002},"last":0.84299999999999997}]},{"osd":6,"last update":"Sat Sep 21 15:00:03 2024","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.163,"15min":1.159},"min":{"1min":0.71299999999999997,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.6579999999999999,"5min":3.2730000000000001,"15min":3.2730000000000001},"last":1.036},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.0469999999999999,"15min":1.0249999999999999},"min":{"1min":0.79100000000000004,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.6180000000000001,"5min":2.2879999999999998,"15min":2.2879999999999998},"last":1.052}]}]},{"osd":6,"up_from":14,"seq":60129542184,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63192,"kb_used_data":9928,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94308648,"statfs":{"total":96636764160,"available":96572055552,"internally_reserved":0,"allocated":10166272,"data_stored":71351473,"data_compressed":1591208,"data_compressed_allocated":6516736,"data_compressed_original":69451532,"omap_allocated":1593,"internal_metadata":54524359},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,16],"upper_bound":1024},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:00:06 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.94399999999999995,"15min":0.95699999999999996},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.244,"5min":1.593,"15min":1.593},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.93600000000000005,"15min":0.95099999999999996},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.3360000000000001,"5min":1.587,"15min":1.587},"last":1.089}]},{"osd":1,"last update":"Sat Sep 21 15:00:06 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.90400000000000003,"15min":0.91400000000000003},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.1120000000000001,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.98299999999999998},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.95699999999999996,"15min":0.97999999999999998},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.1819999999999999,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.89100000000000001}]},{"osd":2,"last update":"Sat Sep 21 15:00:06 2024","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.98499999999999999,"15min":0.99099999999999999},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.3560000000000001,"5min":1.758,"15min":1.758},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.89900000000000002,"15min":0.90900000000000003},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.3069999999999999,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.93799999999999994}]},{"osd":3,"last update":"Sat Sep 21 15:00:06 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.94499999999999995,"15min":0.95799999999999996},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.169,"5min":1.835,"15min":1.835},"last":0.64200000000000002},{"interface":"front","average":{"1min":1.012,"5min":1.1180000000000001,"15min":1.1359999999999999},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.4430000000000001,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":1.2210000000000001}]},{"osd":4,"last update":"Sat Sep 21 15:00:06 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.97699999999999998,"15min":1.0049999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.1379999999999999,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.754},{"interface":"front","average":{"1min":0.93999999999999995,"5min":1.02,"15min":1.0329999999999999},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.411,"5min":1.897,"15min":1.897},"last":0.72399999999999998}]},{"osd":5,"last update":"Sat Sep 21 15:00:06 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":1.0860000000000001,"15min":1.1240000000000001},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.2589999999999999,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":1.0209999999999999},{"interface":"front","average":{"1min":1.01,"5min":1.0800000000000001,"15min":1.0920000000000001},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.3779999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":0.82299999999999995}]},{"osd":7,"last update":"Sat Sep 21 15:00:06 2024","interfaces":[{"interface":"back","average":{"1min":1.123,"5min":1.2350000000000001,"15min":1.2529999999999999},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.4530000000000001,"5min":1.8660000000000001,"15min":1.8660000000000001},"last":1.163},{"interface":"front","average":{"1min":1.044,"5min":1.26,"15min":1.296},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.385,"5min":2.004,"15min":2.004},"last":1.1140000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542185,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40508,"kb_used_data":9388,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94331332,"statfs":{"total":96636764160,"available":96595283968,"internally_reserved":0,"allocated":9613312,"data_stored":65131679,"data_compressed":1272182,"data_compressed_allocated":5857280,"data_compressed_original":63146092,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,15],"upper_bound":1024},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:00:01 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.96799999999999997,"15min":1.0109999999999999},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.98999999999999999,"5min":4.4800000000000004,"15min":4.4800000000000004},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.80800000000000005,"15min":0.82099999999999995},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.0329999999999999,"5min":1.181,"15min":1.181},"last":0.57499999999999996}]},{"osd":1,"last update":"Sat Sep 21 15:01:02 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.89600000000000002,"15min":0.92400000000000004},"min":{"1min":0.47699999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":2.7709999999999999,"5min":2.7709999999999999,"15min":2.7709999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.91400000000000003,"15min":0.93400000000000005},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":2.823,"5min":2.823,"15min":2.823},"last":0.84699999999999998}]},{"osd":2,"last update":"Sat Sep 21 15:01:02 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.91200000000000003,"15min":0.97399999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.159,"5min":3.419,"15min":3.419},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.87,"15min":0.89600000000000002},"min":{"1min":0.53400000000000003,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0780000000000001,"5min":1.423,"15min":1.423},"last":0.96299999999999997}]},{"osd":3,"last update":"Sat Sep 21 15:01:02 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.80300000000000005,"15min":0.80700000000000005},"min":{"1min":0.58499999999999996,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":1.258},{"interface":"front","average":{"1min":0.96699999999999997,"5min":1.054,"15min":1.0840000000000001},"min":{"1min":0.59899999999999998,"5min":0.435,"15min":0.435},"max":{"1min":1.272,"5min":2.6379999999999999,"15min":2.6379999999999999},"last":1.1399999999999999}]},{"osd":4,"last update":"Sat Sep 21 15:01:02 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.98899999999999999,"15min":1.016},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.5609999999999999,"5min":2.5470000000000002,"15min":2.5470000000000002},"last":1.038},{"interface":"front","average":{"1min":0.82099999999999995,"5min":1.0169999999999999,"15min":1.0780000000000001},"min":{"1min":0.39300000000000002,"5min":0.376,"15min":0.376},"max":{"1min":1.302,"5min":2.5099999999999998,"15min":2.5099999999999998},"last":1.302}]},{"osd":6,"last update":"Sat Sep 21 15:01:02 2024","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.94299999999999995,"15min":0.96899999999999997},"min":{"1min":0.63,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":1.0840000000000001},{"interface":"front","average":{"1min":1.004,"5min":1.2450000000000001,"15min":1.3520000000000001},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.425,"5min":4.2469999999999999,"15min":4.2469999999999999},"last":1.214}]},{"osd":7,"last update":"Sat Sep 21 15:00:05 2024","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.3200000000000001,"15min":1.363},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":2.7189999999999999,"5min":3.3010000000000002,"15min":3.3010000000000002},"last":1.345},{"interface":"front","average":{"1min":1.0740000000000001,"5min":1.1339999999999999,"15min":1.1439999999999999},"min":{"1min":0.64200000000000002,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":2.5880000000000001,"5min":2.5880000000000001,"15min":2.5880000000000001},"last":1.181}]}]},{"osd":4,"up_from":14,"seq":60129542184,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":67704,"kb_used_data":10340,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94304136,"statfs":{"total":96636764160,"available":96567435264,"internally_reserved":0,"allocated":10588160,"data_stored":79055540,"data_compressed":1750025,"data_compressed_allocated":7159808,"data_compressed_original":77331143,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.76000000000000001,"15min":0.77600000000000002},"min":{"1min":0.47499999999999998,"5min":0.311,"15min":0.311},"max":{"1min":1.1020000000000001,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":1.5860000000000001,"5min":1.0489999999999999,"15min":0.98099999999999998},"min":{"1min":0.49299999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":13.324,"5min":13.324,"15min":13.324},"last":1.1659999999999999}]},{"osd":1,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.85899999999999999,"15min":0.90600000000000003},"min":{"1min":0.46800000000000003,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.0609999999999999,"5min":1.347,"15min":1.347},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.82199999999999995,"15min":0.83899999999999997},"min":{"1min":0.502,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.58499999999999996}]},{"osd":2,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.88200000000000001,"15min":0.88},"min":{"1min":0.55100000000000005,"5min":0.501,"15min":0.501},"max":{"1min":1.47,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.98099999999999998,"5min":1.002,"15min":1.026},"min":{"1min":0.60699999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.3740000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":1.056}]},{"osd":3,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.90700000000000003,"15min":0.89500000000000002},"min":{"1min":0.61799999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.482,"5min":1.538,"15min":1.538},"last":1.121},{"interface":"front","average":{"1min":1.024,"5min":1.0129999999999999,"15min":0.99199999999999999},"min":{"1min":0.56599999999999995,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.528,"5min":1.528,"15min":1.528},"last":1.01}]},{"osd":5,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.97499999999999998,"15min":0.98399999999999999},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.579,"5min":1.579,"15min":1.579},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.94799999999999995,"15min":0.96399999999999997},"min":{"1min":0.61899999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":0.97299999999999998}]},{"osd":6,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.996,"15min":1.0109999999999999},"min":{"1min":0.55400000000000005,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.349,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.86599999999999999},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.081,"15min":1.0900000000000001},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.665,"5min":1.665,"15min":1.665},"last":0.71899999999999997}]},{"osd":7,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.995,"15min":1.0249999999999999},"min":{"1min":0.45200000000000001,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.226,"5min":1.7010000000000001,"15min":1.7010000000000001},"last":0.748},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.2050000000000001,"15min":1.2130000000000001},"min":{"1min":0.627,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.91800000000000004}]}]},{"osd":3,"up_from":14,"seq":60129542185,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58544,"kb_used_data":13472,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94313296,"statfs":{"total":96636764160,"available":96576815104,"internally_reserved":0,"allocated":13795328,"data_stored":103378431,"data_compressed":2311080,"data_compressed_allocated":9502720,"data_compressed_original":101103314,"omap_allocated":1592,"internal_metadata":46135752},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:00:05 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.378,"5min":0.315,"15min":0.315},"max":{"1min":1.0109999999999999,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.71199999999999997,"15min":0.71999999999999997},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.89200000000000002,"5min":1.919,"15min":1.919},"last":0.66100000000000003}]},{"osd":1,"last update":"Sat Sep 21 15:00:05 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.67500000000000004,"15min":0.67600000000000005},"min":{"1min":0.33400000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":1.1240000000000001,"5min":1.135,"15min":1.135},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.80700000000000005,"15min":0.81699999999999995},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.1599999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.753}]},{"osd":2,"last update":"Sat Sep 21 15:00:05 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.72799999999999998,"15min":0.72399999999999998},"min":{"1min":0.40200000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.83399999999999996,"15min":0.83499999999999996},"min":{"1min":0.47499999999999998,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.1759999999999999,"5min":1.254,"15min":1.254},"last":0.82299999999999995}]},{"osd":4,"last update":"Sat Sep 21 15:00:05 2024","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.95699999999999996,"15min":0.96299999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.2709999999999999,"5min":1.7609999999999999,"15min":1.7609999999999999},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.91100000000000003,"5min":1.0189999999999999,"15min":1.0369999999999999},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.25,"5min":2.028,"15min":2.028},"last":0.85799999999999998}]},{"osd":5,"last update":"Sat Sep 21 15:00:05 2024","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.052,"15min":1.0569999999999999},"min":{"1min":0.77700000000000002,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.246,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.022},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.0649999999999999,"15min":1.0660000000000001},"min":{"1min":0.70799999999999996,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.3600000000000001,"5min":1.431,"15min":1.431},"last":0.95699999999999996}]},{"osd":6,"last update":"Sat Sep 21 15:00:05 2024","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.0860000000000001,"15min":1.0980000000000001},"min":{"1min":0.58599999999999997,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.3879999999999999,"5min":2.1219999999999999,"15min":2.1219999999999999},"last":0.97999999999999998},{"interface":"front","average":{"1min":1.046,"5min":1.097,"15min":1.105},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.7370000000000001,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":1.1100000000000001}]},{"osd":7,"last update":"Sat Sep 21 15:00:05 2024","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.1279999999999999,"15min":1.139},"min":{"1min":0.61799999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.4870000000000001,"5min":1.706,"15min":1.706},"last":1.0469999999999999},{"interface":"front","average":{"1min":1.093,"5min":1.173,"15min":1.1859999999999999},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.516,"5min":2.3380000000000001,"15min":2.3380000000000001},"last":1.073}]}]},{"osd":2,"up_from":14,"seq":60129542183,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64844,"kb_used_data":11584,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94306996,"statfs":{"total":96636764160,"available":96570363904,"internally_reserved":0,"allocated":11862016,"data_stored":89535081,"data_compressed":2014763,"data_compressed_allocated":8007680,"data_compressed_original":87553058,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.62},"min":{"1min":0.26900000000000002,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.92800000000000005,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.69399999999999995,"15min":0.67000000000000004},"min":{"1min":0.34799999999999998,"5min":0.223,"15min":0.223},"max":{"1min":2.9169999999999998,"5min":2.9169999999999998,"15min":2.9169999999999998},"last":0.41299999999999998}]},{"osd":1,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.32700000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.0529999999999999,"5min":2.6520000000000001,"15min":2.6520000000000001},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.746,"5min":0.73499999999999999,"15min":0.75800000000000001},"min":{"1min":0.40500000000000003,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.349,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.69699999999999995}]},{"osd":3,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.73999999999999999,"15min":0.73499999999999999},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":3.0110000000000001,"5min":3.0110000000000001,"15min":3.0110000000000001},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.877,"15min":0.83899999999999997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.05,"5min":4.1440000000000001,"15min":4.1440000000000001},"last":0.47599999999999998}]},{"osd":4,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.93200000000000005,"15min":0.95599999999999996},"min":{"1min":0.42399999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.222,"5min":2.899,"15min":2.899},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.94399999999999995,"15min":0.95799999999999996},"min":{"1min":0.47499999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.4379999999999999,"5min":1.675,"15min":1.675},"last":0.73599999999999999}]},{"osd":5,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.0129999999999999,"15min":1.0089999999999999},"min":{"1min":0.45500000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":3.1080000000000001,"5min":3.1080000000000001,"15min":3.1080000000000001},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.98199999999999998,"15min":0.99399999999999999},"min":{"1min":0.51700000000000002,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.268,"5min":1.546,"15min":1.546},"last":0.51700000000000002}]},{"osd":6,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":0.98299999999999998,"15min":0.98499999999999999},"min":{"1min":0.56200000000000006,"5min":0.378,"15min":0.378},"max":{"1min":1.345,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":0.77700000000000002},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.0649999999999999,"15min":1.0549999999999999},"min":{"1min":0.53200000000000003,"5min":0.502,"15min":0.502},"max":{"1min":3.0739999999999998,"5min":3.0739999999999998,"15min":3.0739999999999998},"last":0.83799999999999997}]},{"osd":7,"last update":"Sat Sep 21 15:00:07 2024","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":1.0029999999999999,"15min":0.995},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.88700000000000001},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.097,"15min":1.091},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.5489999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":0.66000000000000003}]}]},{"osd":1,"up_from":14,"seq":60129542184,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64836,"kb_used_data":11572,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94307004,"statfs":{"total":96636764160,"available":96570372096,"internally_reserved":0,"allocated":11849728,"data_stored":91303543,"data_compressed":1932797,"data_compressed_allocated":8060928,"data_compressed_original":89352649,"omap_allocated":1595,"internal_metadata":54524357},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.59599999999999997,"15min":0.60899999999999999},"min":{"1min":0.36399999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.78200000000000003,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59699999999999998,"15min":0.60799999999999998},"min":{"1min":0.39500000000000002,"5min":0.216,"15min":0.216},"max":{"1min":0.80100000000000005,"5min":1.107,"15min":1.107},"last":0.39500000000000002}]},{"osd":2,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":0.79600000000000004,"15min":0.73999999999999999},"min":{"1min":0.38700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":11.303000000000001,"5min":11.303000000000001,"15min":11.303000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.71299999999999997,"15min":0.70299999999999996},"min":{"1min":0.47299999999999998,"5min":0.435,"15min":0.435},"max":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.81599999999999995}]},{"osd":3,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.81799999999999995,"15min":0.83899999999999997},"min":{"1min":0.27800000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.0049999999999999,"5min":3.1469999999999998,"15min":3.1469999999999998},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.86399999999999999,"15min":0.88600000000000001},"min":{"1min":0.52800000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.306,"5min":3.2320000000000002,"15min":3.2320000000000002},"last":0.94799999999999995}]},{"osd":4,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.89000000000000001,"15min":0.91500000000000004},"min":{"1min":0.59599999999999997,"5min":0.309,"15min":0.309},"max":{"1min":1.45,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":1.5600000000000001,"5min":1.069,"15min":0.96599999999999997},"min":{"1min":0.504,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":11.563000000000001,"5min":11.563000000000001,"15min":11.563000000000001},"last":1.0629999999999999}]},{"osd":5,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":1.625,"5min":1.0469999999999999,"15min":0.94899999999999995},"min":{"1min":0.73399999999999999,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":11.534000000000001,"5min":11.534000000000001,"15min":11.534000000000001},"last":1.121},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95799999999999996,"15min":0.96099999999999997},"min":{"1min":0.63,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.256,"5min":1.484,"15min":1.484},"last":1.256}]},{"osd":6,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":1.0109999999999999,"15min":1.0489999999999999},"min":{"1min":0.60399999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.4059999999999999,"5min":2.3519999999999999,"15min":2.3519999999999999},"last":0.85099999999999998},{"interface":"front","average":{"1min":1.631,"5min":1.081,"15min":0.99299999999999999},"min":{"1min":0.57399999999999995,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":11.601000000000001,"5min":11.601000000000001,"15min":11.601000000000001},"last":1.1639999999999999}]},{"osd":7,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":1.046,"15min":1.0640000000000001},"min":{"1min":0.79300000000000004,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.5820000000000001,"5min":2.2360000000000002,"15min":2.2360000000000002},"last":0.79300000000000004},{"interface":"front","average":{"1min":1.76,"5min":1.171,"15min":1.087},"min":{"1min":0.877,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":11.489000000000001,"5min":11.489000000000001,"15min":11.489000000000001},"last":1.23}]}]},{"osd":0,"up_from":14,"seq":60129542183,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70176,"kb_used_data":12812,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94301664,"statfs":{"total":96636764160,"available":96564903936,"internally_reserved":0,"allocated":13119488,"data_stored":98239443,"data_compressed":2190723,"data_compressed_allocated":9064448,"data_compressed_original":96141885,"omap_allocated":1593,"internal_metadata":58718663},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 21 15:01:04 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.67500000000000004,"15min":0.71699999999999997},"min":{"1min":0.40999999999999998,"5min":0.309,"15min":0.309},"max":{"1min":1.1299999999999999,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.68100000000000005,"15min":0.627},"min":{"1min":0.58999999999999997,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.67800000000000005}]},{"osd":2,"last update":"Sat Sep 21 15:01:04 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.65200000000000002,"15min":0.61799999999999999},"min":{"1min":0.59599999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.72499999999999998,"15min":0.75800000000000001},"min":{"1min":0.39300000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.79200000000000004}]},{"osd":3,"last update":"Sat Sep 21 15:01:04 2024","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.76600000000000001,"15min":0.754},"min":{"1min":0.50900000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.78700000000000003,"15min":0.81499999999999995},"min":{"1min":0.50900000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.50900000000000001}]},{"osd":4,"last update":"Sat Sep 21 15:01:04 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.98399999999999999,"15min":1.038},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.1140000000000001,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":1.036,"5min":1.0469999999999999,"15min":1.0620000000000001},"min":{"1min":0.67800000000000005,"5min":0.379,"15min":0.379},"max":{"1min":1.446,"5min":1.665,"15min":1.665},"last":0.71199999999999997}]},{"osd":5,"last update":"Sat Sep 21 15:01:04 2024","interfaces":[{"interface":"back","average":{"1min":1.0960000000000001,"5min":1.024,"15min":1.032},"min":{"1min":0.79300000000000004,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.4450000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.83899999999999997},{"interface":"front","average":{"1min":1.0589999999999999,"5min":1.0429999999999999,"15min":1.077},"min":{"1min":0.82399999999999995,"5min":0.435,"15min":0.435},"max":{"1min":1.2609999999999999,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":0.95499999999999996}]},{"osd":6,"last update":"Sat Sep 21 15:01:04 2024","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":1.0580000000000001,"15min":1.1120000000000001},"min":{"1min":0.68400000000000005,"5min":0.628,"15min":0.628},"max":{"1min":1.167,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":0.85799999999999998},{"interface":"front","average":{"1min":1.109,"5min":1.0629999999999999,"15min":1.0680000000000001},"min":{"1min":0.91000000000000003,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.5029999999999999,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.91900000000000004}]},{"osd":7,"last update":"Sat Sep 21 15:01:04 2024","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.0469999999999999,"15min":1.0620000000000001},"min":{"1min":0.78700000000000003,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.4770000000000001,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":0.88500000000000001},{"interface":"front","average":{"1min":1.02,"5min":1.085,"15min":1.131},"min":{"1min":0.84799999999999998,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.403,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":0.93600000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5202,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5202,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":11001856,"data_stored":97211967,"data_compressed":2190723,"data_compressed_allocated":9064448,"data_compressed_original":96141885,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9662464,"data_stored":89679641,"data_compressed":1927595,"data_compressed_allocated":8024064,"data_compressed_original":88779209,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":9732096,"data_stored":88501547,"data_compressed":2014763,"data_compressed_allocated":8007680,"data_compressed_original":87553058,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":11673600,"data_stored":102350936,"data_compressed":2311080,"data_compressed_allocated":9502720,"data_compressed_original":101103314,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8458240,"data_stored":78022006,"data_compressed":1750025,"data_compressed_allocated":7159808,"data_compressed_original":77331143,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":7426048,"data_stored":63507777,"data_compressed":1266980,"data_compressed_allocated":5820416,"data_compressed_original":62572652,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8044544,"data_stored":70323978,"data_compressed":1591208,"data_compressed_allocated":6516736,"data_compressed_original":69451532,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":5652480,"data_stored":47867620,"data_compressed":1141276,"data_compressed_allocated":4558848,"data_compressed_original":47268443,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T15:01:06.547 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T15:01:06.548 INFO:teuthology.orchestra.run.smithi032.stdout:5536 write 0x135f21b thru 0x13660ab (0x6e91 bytes) 2024-09-21T15:01:06.548 INFO:teuthology.orchestra.run.smithi032.stdout:5539 punch from 0x3b6c55c to 0x3b730dd, (0x6b81 bytes) 2024-09-21T15:01:06.548 INFO:teuthology.orchestra.run.smithi032.stdout:5541 write 0x1ffc30a thru 0x2009f6d (0xdc64 bytes) 2024-09-21T15:01:06.552 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:01:06.552 INFO:teuthology.orchestra.run.smithi032.stdout:5544 punch from 0x71d1c2e to 0x71d40fd, (0x24cf bytes) 2024-09-21T15:01:06.552 INFO:teuthology.orchestra.run.smithi032.stdout:5545 read 0x37a73b5 thru 0x37aba85 (0x46d1 bytes) 2024-09-21T15:01:06.569 INFO:teuthology.orchestra.run.smithi032.stdout:5546 trunc from 0xbf8846a to 0x255fee9 2024-09-21T15:01:06.570 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:01:06.565+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:01:06.595 INFO:teuthology.orchestra.run.smithi032.stdout:5547 trunc from 0x255fee9 to 0x82925cf 2024-09-21T15:01:06.623 INFO:teuthology.orchestra.run.smithi032.stdout:5548 write 0xbf1c409 thru 0xbf28985 (0xc57d bytes) 2024-09-21T15:01:06.636 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:01:06.637 INFO:teuthology.orchestra.run.smithi032.stdout:5550 read 0x2529794 thru 0x25379fc (0xe269 bytes) 2024-09-21T15:01:06.641 INFO:teuthology.orchestra.run.smithi032.stdout:5552 trunc from 0xbf28986 to 0xa87dbbe 2024-09-21T15:01:06.654 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:01:06.654 INFO:teuthology.orchestra.run.smithi032.stdout:5554 read 0x2e7f531 thru 0x2e80361 (0xe31 bytes) 2024-09-21T15:01:06.654 INFO:teuthology.orchestra.run.smithi032.stdout:5555 read 0x6301a11 thru 0x630ff6f (0xe55f bytes) 2024-09-21T15:01:06.658 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:06.658 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:01:06.658 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T15:01:06.658 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T15:01:06.658 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:06.661 INFO:teuthology.orchestra.run.smithi032.stdout:5556 trunc from 0xa87dbbe to 0xe7bf4de 2024-09-21T15:01:06.670 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:01:06.666+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:01:06.671 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T15:01:06.671 INFO:teuthology.orchestra.run.smithi032.stdout:5559 read 0x8c3d700 thru 0x8c3fc36 (0x2537 bytes) 2024-09-21T15:01:06.672 INFO:teuthology.orchestra.run.smithi032.stdout:5560 writesame 0xa7b76a7 thru 0xa7bf7e9 data_size 0x1(0x8143 bytes) 2024-09-21T15:01:06.717 INFO:teuthology.orchestra.run.smithi032.stdout:5563 trunc from 0xe7bf4de to 0x2818230 2024-09-21T15:01:06.744 INFO:teuthology.orchestra.run.smithi032.stdout:5564 read 0x220f233 thru 0x2212843 (0x3611 bytes) 2024-09-21T15:01:06.744 INFO:teuthology.orchestra.run.smithi032.stdout:5565 read 0x217b475 thru 0x2185896 (0xa422 bytes) 2024-09-21T15:01:06.771 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:01:06.769+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:01:06.773 INFO:teuthology.orchestra.run.smithi032.stdout:5566 write 0xdfd350c thru 0xdfe1b47 (0xe63c bytes) 2024-09-21T15:01:06.785 INFO:teuthology.orchestra.run.smithi032.stdout:5567 write 0x806357c thru 0x8069b16 (0x659b bytes) 2024-09-21T15:01:06.792 INFO:teuthology.orchestra.run.smithi032.stdout:5568 write 0xd4ee133 thru 0xd4efd56 (0x1c24 bytes) 2024-09-21T15:01:06.795 INFO:teuthology.orchestra.run.smithi032.stdout:5569 write 0x4323685 thru 0x4330aa6 (0xd422 bytes) 2024-09-21T15:01:06.798 INFO:teuthology.orchestra.run.smithi032.stdout:5570 write 0xc1e8149 thru 0xc1f2c5b (0xab13 bytes) 2024-09-21T15:01:06.818 INFO:teuthology.orchestra.run.smithi032.stdout:5571 trunc from 0xdfe1b48 to 0x712894a 2024-09-21T15:01:06.832 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:01:06.832 INFO:teuthology.orchestra.run.smithi032.stdout:5573 read 0x317d637 thru 0x317fe54 (0x281e bytes) 2024-09-21T15:01:06.832 INFO:teuthology.orchestra.run.smithi032.stdout:5574 read 0x4ec9374 thru 0x4ecb5f0 (0x227d bytes) 2024-09-21T15:01:06.846 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:06.846 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T15:01:06.846 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T15:01:06.846 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T15:01:06.846 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.846 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139776 3.16 3:6c38d0e2:::rbd_data.113664780c85.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:06.846 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.223442+0000", 2024-09-21T15:01:06.846 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.6206574539999998, 2024-09-21T15:01:06.847 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0315488150000001, 2024-09-21T15:01:06.847 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:06.847 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:06.847 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:06.847 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:06.847 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:06.847 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139776 2024-09-21T15:01:06.847 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.847 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:06.847 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.847 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:06.847 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223442+0000", 2024-09-21T15:01:06.847 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:06.847 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.847 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.848 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:06.848 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223442+0000", 2024-09-21T15:01:06.848 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:06.848 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.848 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.848 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:06.848 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223442+0000", 2024-09-21T15:01:06.848 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:06.848 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.848 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.848 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:06.848 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223446+0000", 2024-09-21T15:01:06.848 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2549999999999999e-06 2024-09-21T15:01:06.848 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.848 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.849 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:06.849 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223454+0000", 2024-09-21T15:01:06.849 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8199999999999997e-06 2024-09-21T15:01:06.849 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.849 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.849 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:06.849 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223460+0000", 2024-09-21T15:01:06.849 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6330000000000001e-06 2024-09-21T15:01:06.849 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.849 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.849 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:06.849 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223542+0000", 2024-09-21T15:01:06.849 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1929999999999997e-05 2024-09-21T15:01:06.849 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.850 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.850 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:06.850 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223552+0000", 2024-09-21T15:01:06.850 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0627e-05 2024-09-21T15:01:06.850 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.850 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.850 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:06.850 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.254634+0000", 2024-09-21T15:01:06.850 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.031081972 2024-09-21T15:01:06.850 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.850 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.850 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:06.850 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.254796+0000", 2024-09-21T15:01:06.850 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000161879 2024-09-21T15:01:06.850 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.851 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.851 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:06.851 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.254991+0000", 2024-09-21T15:01:06.851 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00019526499999999999 2024-09-21T15:01:06.851 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:06.851 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:06.851 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:06.851 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.851 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.851 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139785 3.11 3:8abbb109:::rbd_data.113664780c85.0000000000000070:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:06.851 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.223531+0000", 2024-09-21T15:01:06.851 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.6205687129999999, 2024-09-21T15:01:06.851 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0187143510000001, 2024-09-21T15:01:06.851 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:06.852 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:06.852 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:06.852 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:06.852 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:06.852 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139785 2024-09-21T15:01:06.852 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.852 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:06.852 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.852 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:06.852 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223531+0000", 2024-09-21T15:01:06.852 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:06.852 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.852 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.852 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:06.852 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223531+0000", 2024-09-21T15:01:06.853 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:06.853 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.853 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.853 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:06.853 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223531+0000", 2024-09-21T15:01:06.853 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:06.853 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.853 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.853 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:06.853 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223534+0000", 2024-09-21T15:01:06.853 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2650000000000001e-06 2024-09-21T15:01:06.853 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.853 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.853 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:06.853 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223541+0000", 2024-09-21T15:01:06.853 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8990000000000004e-06 2024-09-21T15:01:06.854 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.854 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.854 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:06.854 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223546+0000", 2024-09-21T15:01:06.854 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3399999999999997e-06 2024-09-21T15:01:06.854 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.854 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.854 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:06.854 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223724+0000", 2024-09-21T15:01:06.854 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017745799999999999 2024-09-21T15:01:06.854 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.854 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.854 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:06.855 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223735+0000", 2024-09-21T15:01:06.855 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1610999999999999e-05 2024-09-21T15:01:06.855 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.855 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.855 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:06.855 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242113+0000", 2024-09-21T15:01:06.855 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.018377879 2024-09-21T15:01:06.855 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.855 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.855 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:06.855 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242246+0000", 2024-09-21T15:01:06.855 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013231500000000001 2024-09-21T15:01:06.855 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:06.855 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:06.855 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:06.856 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.856 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.856 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139790 3.11 3:8a18c7d8:::rbd_data.113664780c85.000000000000008d:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:06.856 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.223578+0000", 2024-09-21T15:01:06.856 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.620521718, 2024-09-21T15:01:06.856 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0187879719999999, 2024-09-21T15:01:06.856 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:06.856 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:06.856 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:06.856 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:06.856 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:06.856 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139790 2024-09-21T15:01:06.856 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.856 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:06.856 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.857 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:06.857 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223578+0000", 2024-09-21T15:01:06.857 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:06.857 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.857 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.857 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:06.857 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223578+0000", 2024-09-21T15:01:06.857 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:06.857 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.857 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.857 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:06.857 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223578+0000", 2024-09-21T15:01:06.857 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:06.857 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.857 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.857 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:06.858 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223581+0000", 2024-09-21T15:01:06.858 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6879999999999999e-06 2024-09-21T15:01:06.858 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.858 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.858 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:06.858 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223586+0000", 2024-09-21T15:01:06.858 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7080000000000002e-06 2024-09-21T15:01:06.858 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.858 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.858 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:06.858 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223591+0000", 2024-09-21T15:01:06.858 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5639999999999998e-06 2024-09-21T15:01:06.858 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.858 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.859 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:06.859 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223761+0000", 2024-09-21T15:01:06.859 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000170213 2024-09-21T15:01:06.859 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.859 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.859 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:06.859 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223769+0000", 2024-09-21T15:01:06.859 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7049999999999999e-06 2024-09-21T15:01:06.859 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.859 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.859 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:06.859 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242269+0000", 2024-09-21T15:01:06.859 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.018500288 2024-09-21T15:01:06.860 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.860 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.860 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:06.860 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242366+0000", 2024-09-21T15:01:06.860 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.7126000000000005e-05 2024-09-21T15:01:06.860 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:06.860 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:06.860 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:06.860 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.860 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.860 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139792 3.16 3:6ce5caf3:::rbd_data.113664780c85.000000000000009b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:06.860 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.223595+0000", 2024-09-21T15:01:06.860 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.6205047649999997, 2024-09-21T15:01:06.860 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0317517899999999, 2024-09-21T15:01:06.860 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:06.861 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:06.861 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:06.861 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:06.861 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:06.861 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139792 2024-09-21T15:01:06.861 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.861 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:06.861 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.861 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:06.861 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223595+0000", 2024-09-21T15:01:06.861 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:06.861 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.861 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.861 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:06.862 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223595+0000", 2024-09-21T15:01:06.862 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:06.862 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.862 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.862 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:06.862 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223595+0000", 2024-09-21T15:01:06.862 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:06.862 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.862 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.862 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:06.862 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223598+0000", 2024-09-21T15:01:06.862 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4620000000000001e-06 2024-09-21T15:01:06.862 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.862 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.863 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:06.863 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223604+0000", 2024-09-21T15:01:06.863 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6879999999999996e-06 2024-09-21T15:01:06.863 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.863 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.863 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:06.863 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223609+0000", 2024-09-21T15:01:06.863 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9490000000000004e-06 2024-09-21T15:01:06.863 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.863 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.863 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:06.863 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224067+0000", 2024-09-21T15:01:06.863 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00045780700000000002 2024-09-21T15:01:06.864 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.864 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.864 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:06.864 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224075+0000", 2024-09-21T15:01:06.864 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6610000000000006e-06 2024-09-21T15:01:06.864 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.864 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.864 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:06.864 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255141+0000", 2024-09-21T15:01:06.864 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0310657480000001 2024-09-21T15:01:06.864 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.864 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.864 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:06.864 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255347+0000", 2024-09-21T15:01:06.865 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000205837 2024-09-21T15:01:06.865 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:06.865 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:06.865 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:06.865 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.865 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.865 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139794 3.11 3:8cf3a2f4:::rbd_data.113664780c85.00000000000000a9:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:06.865 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.224011+0000", 2024-09-21T15:01:06.865 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.6200887369999997, 2024-09-21T15:01:06.865 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.018507791, 2024-09-21T15:01:06.865 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:06.865 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:06.865 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:06.865 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:06.865 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:06.866 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139794 2024-09-21T15:01:06.866 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.866 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:06.866 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.866 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:06.866 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224011+0000", 2024-09-21T15:01:06.866 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:06.866 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.866 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.866 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:06.866 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224011+0000", 2024-09-21T15:01:06.866 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:06.866 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.866 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.867 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:06.867 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224010+0000", 2024-09-21T15:01:06.867 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:06.867 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.867 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.867 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:06.867 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224016+0000", 2024-09-21T15:01:06.867 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6550000000000001e-06 2024-09-21T15:01:06.867 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.867 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.867 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:06.867 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224028+0000", 2024-09-21T15:01:06.867 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2422999999999999e-05 2024-09-21T15:01:06.867 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.868 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.868 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:06.868 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224036+0000", 2024-09-21T15:01:06.868 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7749999999999993e-06 2024-09-21T15:01:06.868 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.868 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.868 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:06.868 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225176+0000", 2024-09-21T15:01:06.868 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0011401740000000001 2024-09-21T15:01:06.868 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.868 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.868 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:06.868 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225187+0000", 2024-09-21T15:01:06.868 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0951999999999999e-05 2024-09-21T15:01:06.869 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.869 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.869 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:06.869 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242392+0000", 2024-09-21T15:01:06.869 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.017204905 2024-09-21T15:01:06.869 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.869 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.869 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:06.869 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242519+0000", 2024-09-21T15:01:06.869 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012680499999999999 2024-09-21T15:01:06.869 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:06.869 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:06.869 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:06.869 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.870 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.870 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139800 3.16 3:68b050ed:::rbd_data.113664780c85.00000000000000d3:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:06.870 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.224579+0000", 2024-09-21T15:01:06.870 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.6195209779999997, 2024-09-21T15:01:06.870 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.030932637, 2024-09-21T15:01:06.870 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:06.870 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:06.870 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:06.870 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:06.870 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:06.870 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139800 2024-09-21T15:01:06.870 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.870 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:06.870 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.871 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:06.871 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224579+0000", 2024-09-21T15:01:06.871 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:06.871 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.871 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.871 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:06.871 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224579+0000", 2024-09-21T15:01:06.871 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:06.871 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.871 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.871 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:06.871 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224578+0000", 2024-09-21T15:01:06.871 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:06.871 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.872 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.872 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:06.872 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224582+0000", 2024-09-21T15:01:06.872 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0029999999999999e-06 2024-09-21T15:01:06.872 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.872 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.872 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:06.872 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224588+0000", 2024-09-21T15:01:06.872 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8120000000000002e-06 2024-09-21T15:01:06.872 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.872 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.872 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:06.872 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224593+0000", 2024-09-21T15:01:06.872 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7469999999999996e-06 2024-09-21T15:01:06.872 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.873 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.873 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:06.873 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224648+0000", 2024-09-21T15:01:06.873 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4693e-05 2024-09-21T15:01:06.873 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.873 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.873 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:06.873 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224656+0000", 2024-09-21T15:01:06.873 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8429999999999993e-06 2024-09-21T15:01:06.873 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.873 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.873 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:06.873 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255382+0000", 2024-09-21T15:01:06.873 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0307266070000001 2024-09-21T15:01:06.874 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.874 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.874 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:06.874 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255512+0000", 2024-09-21T15:01:06.874 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000129367 2024-09-21T15:01:06.874 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:06.874 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:06.874 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:06.874 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.874 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.874 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139804 3.11 3:88f9ddce:::rbd_data.113664780c85.00000000000000ef:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:06.874 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.224632+0000", 2024-09-21T15:01:06.874 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.6194679970000001, 2024-09-21T15:01:06.874 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.018021302, 2024-09-21T15:01:06.875 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:06.875 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:06.875 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:06.875 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:06.875 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:06.875 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139804 2024-09-21T15:01:06.875 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.875 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:06.875 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.875 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:06.875 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224632+0000", 2024-09-21T15:01:06.875 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:06.875 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.875 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.876 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:06.876 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224632+0000", 2024-09-21T15:01:06.876 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:06.876 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.876 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.876 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:06.876 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224632+0000", 2024-09-21T15:01:06.876 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:06.876 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.876 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.876 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:06.876 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224635+0000", 2024-09-21T15:01:06.876 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2710000000000001e-06 2024-09-21T15:01:06.876 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.877 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.877 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:06.877 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224640+0000", 2024-09-21T15:01:06.877 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6779999999999997e-06 2024-09-21T15:01:06.877 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.877 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.877 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:06.877 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224645+0000", 2024-09-21T15:01:06.877 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9339999999999997e-06 2024-09-21T15:01:06.877 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.877 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.877 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:06.877 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225307+0000", 2024-09-21T15:01:06.877 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00066201699999999999 2024-09-21T15:01:06.877 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.878 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.878 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:06.878 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225312+0000", 2024-09-21T15:01:06.878 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9799999999999998e-06 2024-09-21T15:01:06.878 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.878 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.878 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:06.878 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242548+0000", 2024-09-21T15:01:06.878 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.017235603 2024-09-21T15:01:06.878 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.878 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.878 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:06.878 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242653+0000", 2024-09-21T15:01:06.878 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010522 2024-09-21T15:01:06.879 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:06.879 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:06.879 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:06.879 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.879 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.879 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139809 3.11 3:884da946:::rbd_data.113664780c85.0000000000000118:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:06.879 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.224667+0000", 2024-09-21T15:01:06.879 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.6194329530000005, 2024-09-21T15:01:06.879 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0181101960000001, 2024-09-21T15:01:06.879 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:06.879 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:06.879 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:06.879 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:06.879 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:06.880 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139809 2024-09-21T15:01:06.880 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.880 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:06.880 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.880 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:06.880 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224667+0000", 2024-09-21T15:01:06.880 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:06.880 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.880 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.880 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:06.880 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224667+0000", 2024-09-21T15:01:06.880 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:06.880 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.880 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.880 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:06.881 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224667+0000", 2024-09-21T15:01:06.881 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:06.881 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.881 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.881 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:06.881 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224669+0000", 2024-09-21T15:01:06.881 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.452e-06 2024-09-21T15:01:06.881 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.881 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.881 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:06.881 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224675+0000", 2024-09-21T15:01:06.881 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.818e-06 2024-09-21T15:01:06.881 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.881 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.881 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:06.882 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224680+0000", 2024-09-21T15:01:06.882 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6319999999999999e-06 2024-09-21T15:01:06.882 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.882 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.882 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:06.882 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225400+0000", 2024-09-21T15:01:06.882 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00072012699999999999 2024-09-21T15:01:06.882 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.882 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.882 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:06.882 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225409+0000", 2024-09-21T15:01:06.882 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1039999999999996e-06 2024-09-21T15:01:06.882 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.882 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.883 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:06.883 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242675+0000", 2024-09-21T15:01:06.883 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0172663319999999 2024-09-21T15:01:06.883 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.883 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.883 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:06.883 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242777+0000", 2024-09-21T15:01:06.883 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010197899999999999 2024-09-21T15:01:06.883 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:06.883 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:06.883 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:06.883 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.883 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.883 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139868 3.16 3:69f82090:::rbd_data.113664780c85.00000000000000f1:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:06.883 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.225982+0000", 2024-09-21T15:01:06.884 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.618117453, 2024-09-21T15:01:06.884 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0296791729999999, 2024-09-21T15:01:06.884 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:06.884 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:06.884 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:06.884 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:06.884 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:06.884 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139868 2024-09-21T15:01:06.884 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.884 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:06.884 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.884 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:06.884 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225982+0000", 2024-09-21T15:01:06.885 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:06.885 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.885 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.885 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:06.885 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225982+0000", 2024-09-21T15:01:06.885 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:06.885 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.885 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.885 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:06.885 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225982+0000", 2024-09-21T15:01:06.885 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:06.885 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.885 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.885 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:06.885 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225985+0000", 2024-09-21T15:01:06.885 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.74e-06 2024-09-21T15:01:06.886 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.886 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.886 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:06.886 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225990+0000", 2024-09-21T15:01:06.886 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7039999999999999e-06 2024-09-21T15:01:06.886 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.886 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.886 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:06.886 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225995+0000", 2024-09-21T15:01:06.886 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7249999999999997e-06 2024-09-21T15:01:06.886 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.886 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.886 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:06.886 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226299+0000", 2024-09-21T15:01:06.887 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00030383199999999997 2024-09-21T15:01:06.887 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.887 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.887 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:06.887 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226308+0000", 2024-09-21T15:01:06.887 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.9740000000000008e-06 2024-09-21T15:01:06.887 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.887 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.887 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:06.887 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255539+0000", 2024-09-21T15:01:06.887 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0292305960000001 2024-09-21T15:01:06.887 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.887 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.887 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:06.888 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255662+0000", 2024-09-21T15:01:06.888 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012307400000000001 2024-09-21T15:01:06.888 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:06.888 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:06.888 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:06.888 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.888 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.888 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139869 3.16 3:6b6c90a4:::rbd_data.113664780c85.00000000000000fe:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:06.888 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.226006+0000", 2024-09-21T15:01:06.888 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.618093837, 2024-09-21T15:01:06.888 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.029790518, 2024-09-21T15:01:06.888 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:06.888 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:06.888 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:06.888 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:06.889 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:06.889 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139869 2024-09-21T15:01:06.889 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.889 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:06.889 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.889 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:06.889 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226006+0000", 2024-09-21T15:01:06.889 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:06.889 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.889 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.889 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:06.889 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226006+0000", 2024-09-21T15:01:06.889 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:06.889 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.889 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.889 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:06.890 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226006+0000", 2024-09-21T15:01:06.890 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:06.890 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.890 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.890 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:06.890 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226008+0000", 2024-09-21T15:01:06.890 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5569999999999998e-06 2024-09-21T15:01:06.890 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.890 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.890 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:06.890 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226014+0000", 2024-09-21T15:01:06.890 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2079999999999997e-06 2024-09-21T15:01:06.890 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.890 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.891 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:06.891 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226020+0000", 2024-09-21T15:01:06.891 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7429999999999999e-06 2024-09-21T15:01:06.891 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.891 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.891 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:06.891 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226330+0000", 2024-09-21T15:01:06.891 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00030963300000000002 2024-09-21T15:01:06.891 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.891 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.891 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:06.891 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226336+0000", 2024-09-21T15:01:06.891 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6340000000000003e-06 2024-09-21T15:01:06.891 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.891 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.892 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:06.892 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255687+0000", 2024-09-21T15:01:06.892 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0293507150000001 2024-09-21T15:01:06.892 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.892 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.892 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:06.892 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255797+0000", 2024-09-21T15:01:06.892 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000109471 2024-09-21T15:01:06.892 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:06.892 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:06.892 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:06.892 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.892 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.892 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139884 3.16 3:69f10324:::rbd_data.113664780c85.0000000000000161:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:06.893 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.227573+0000", 2024-09-21T15:01:06.893 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.6165271959999998, 2024-09-21T15:01:06.893 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0283568759999999, 2024-09-21T15:01:06.893 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:06.893 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:06.893 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:06.893 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:06.893 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:06.893 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139884 2024-09-21T15:01:06.893 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.893 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:06.893 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.893 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:06.893 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227573+0000", 2024-09-21T15:01:06.893 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:06.894 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.894 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.894 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:06.894 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227573+0000", 2024-09-21T15:01:06.894 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:06.894 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.894 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.894 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:06.894 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227572+0000", 2024-09-21T15:01:06.894 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:06.894 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.894 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.894 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:06.894 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227575+0000", 2024-09-21T15:01:06.895 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7769999999999998e-06 2024-09-21T15:01:06.895 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.895 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.895 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:06.895 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227581+0000", 2024-09-21T15:01:06.895 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7359999999999998e-06 2024-09-21T15:01:06.895 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.895 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.895 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:06.895 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227585+0000", 2024-09-21T15:01:06.895 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5090000000000004e-06 2024-09-21T15:01:06.895 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.895 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.895 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:06.896 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227753+0000", 2024-09-21T15:01:06.896 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00016761599999999999 2024-09-21T15:01:06.896 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.896 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.896 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:06.896 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227761+0000", 2024-09-21T15:01:06.896 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0579999999999998e-06 2024-09-21T15:01:06.896 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.896 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.896 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:06.896 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255828+0000", 2024-09-21T15:01:06.896 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0280671349999999 2024-09-21T15:01:06.896 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.896 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.897 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:06.897 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255930+0000", 2024-09-21T15:01:06.897 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000101372 2024-09-21T15:01:06.897 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:06.897 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:06.897 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:06.897 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.897 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.897 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139913 3.16 3:6f81b46d:::rbd_data.113664780c85.0000000000000074:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:06.897 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.227685+0000", 2024-09-21T15:01:06.897 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.6164150169999996, 2024-09-21T15:01:06.897 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0283644750000001, 2024-09-21T15:01:06.897 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:06.897 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:06.898 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:06.898 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:06.898 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:06.898 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139913 2024-09-21T15:01:06.898 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.898 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:06.898 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.898 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:06.898 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227685+0000", 2024-09-21T15:01:06.898 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:06.898 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.898 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.898 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:06.898 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227685+0000", 2024-09-21T15:01:06.898 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:06.899 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.899 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.899 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:06.899 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227685+0000", 2024-09-21T15:01:06.899 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:06.899 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.899 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.899 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:06.899 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227687+0000", 2024-09-21T15:01:06.899 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2929999999999999e-06 2024-09-21T15:01:06.899 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.899 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.899 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:06.899 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227693+0000", 2024-09-21T15:01:06.900 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1070000000000001e-06 2024-09-21T15:01:06.900 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.900 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.900 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:06.900 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227698+0000", 2024-09-21T15:01:06.900 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9339999999999997e-06 2024-09-21T15:01:06.900 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.900 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.900 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:06.900 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227784+0000", 2024-09-21T15:01:06.900 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6096999999999999e-05 2024-09-21T15:01:06.900 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.900 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.901 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:06.901 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227791+0000", 2024-09-21T15:01:06.901 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9090000000000003e-06 2024-09-21T15:01:06.901 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.901 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.901 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:06.901 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255955+0000", 2024-09-21T15:01:06.901 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0281643579999999 2024-09-21T15:01:06.901 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.901 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.901 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:06.901 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256049+0000", 2024-09-21T15:01:06.901 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4047000000000003e-05 2024-09-21T15:01:06.901 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:06.901 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:06.902 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:06.902 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.902 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.902 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139988 3.11 3:88a29fa4:::rbd_data.113664780c85.00000000000000bd:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:06.902 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.229772+0000", 2024-09-21T15:01:06.902 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.6143275179999996, 2024-09-21T15:01:06.902 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.017967668, 2024-09-21T15:01:06.902 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:06.902 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:06.902 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:06.902 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:06.902 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:06.902 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139988 2024-09-21T15:01:06.903 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.903 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:06.903 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.903 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:06.903 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229772+0000", 2024-09-21T15:01:06.903 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:06.903 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.903 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.903 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:06.903 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229772+0000", 2024-09-21T15:01:06.903 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:06.903 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.903 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.903 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:06.904 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229772+0000", 2024-09-21T15:01:06.904 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:06.904 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.904 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.904 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:06.904 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229776+0000", 2024-09-21T15:01:06.904 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6890000000000002e-06 2024-09-21T15:01:06.904 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.904 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.904 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:06.904 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229783+0000", 2024-09-21T15:01:06.904 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2280000000000001e-06 2024-09-21T15:01:06.904 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.904 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.905 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:06.905 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229789+0000", 2024-09-21T15:01:06.905 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1700000000000002e-06 2024-09-21T15:01:06.905 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.905 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.905 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:06.905 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.230106+0000", 2024-09-21T15:01:06.905 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00031700800000000003 2024-09-21T15:01:06.905 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.905 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.905 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:06.905 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.230115+0000", 2024-09-21T15:01:06.905 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4190000000000008e-06 2024-09-21T15:01:06.905 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.906 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.906 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:06.906 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242935+0000", 2024-09-21T15:01:06.906 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0128191449999999 2024-09-21T15:01:06.906 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.906 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.906 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for missing object", 2024-09-21T15:01:06.906 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.243078+0000", 2024-09-21T15:01:06.906 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000142984 2024-09-21T15:01:06.906 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.906 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.906 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:06.906 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.247659+0000", 2024-09-21T15:01:06.906 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0045813030000000001 2024-09-21T15:01:06.907 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.907 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.907 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:06.907 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.247740+0000", 2024-09-21T15:01:06.907 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1216000000000003e-05 2024-09-21T15:01:06.907 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:06.907 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:06.907 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:06.907 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.907 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.907 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139994 3.16 3:6b9df8d1:::rbd_data.113664780c85.00000000000000e7:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:06.907 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.231370+0000", 2024-09-21T15:01:06.907 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.6127294399999998, 2024-09-21T15:01:06.908 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.024795576, 2024-09-21T15:01:06.908 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:06.908 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:06.908 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:06.908 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:06.908 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:06.908 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139994 2024-09-21T15:01:06.908 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.908 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:06.908 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.908 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:06.908 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231370+0000", 2024-09-21T15:01:06.908 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:06.908 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.909 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.909 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:06.909 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231370+0000", 2024-09-21T15:01:06.909 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:06.909 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.909 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.909 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:06.909 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231369+0000", 2024-09-21T15:01:06.909 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:06.909 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.909 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.909 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:06.909 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231376+0000", 2024-09-21T15:01:06.909 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5509999999999996e-06 2024-09-21T15:01:06.910 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.910 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.910 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:06.910 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231387+0000", 2024-09-21T15:01:06.910 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.061e-05 2024-09-21T15:01:06.910 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.910 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.910 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:06.910 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231394+0000", 2024-09-21T15:01:06.910 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3799999999999996e-06 2024-09-21T15:01:06.910 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.910 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.910 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:06.911 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231459+0000", 2024-09-21T15:01:06.911 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4716999999999997e-05 2024-09-21T15:01:06.911 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.911 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.911 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:06.911 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231468+0000", 2024-09-21T15:01:06.911 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1479999999999995e-06 2024-09-21T15:01:06.911 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.911 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.911 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:06.911 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256074+0000", 2024-09-21T15:01:06.911 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0246064180000001 2024-09-21T15:01:06.911 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.911 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.912 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:06.912 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256166+0000", 2024-09-21T15:01:06.912 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1862999999999998e-05 2024-09-21T15:01:06.912 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:06.912 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:06.912 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:06.912 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.912 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.912 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140012 3.16 3:6bd6dfb6:::rbd_data.113664780c85.0000000000000165:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:06.912 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.231890+0000", 2024-09-21T15:01:06.912 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.6122096809999995, 2024-09-21T15:01:06.912 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.024408881, 2024-09-21T15:01:06.912 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:06.912 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:06.913 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:06.913 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:06.913 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:06.913 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140012 2024-09-21T15:01:06.913 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.913 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:06.913 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.913 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:06.913 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231890+0000", 2024-09-21T15:01:06.913 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:06.913 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.913 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.913 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:06.913 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231890+0000", 2024-09-21T15:01:06.913 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:06.913 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.914 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.914 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:06.914 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231889+0000", 2024-09-21T15:01:06.914 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:06.914 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.914 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.914 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:06.914 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231895+0000", 2024-09-21T15:01:06.914 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9569999999999999e-06 2024-09-21T15:01:06.914 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.914 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.914 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:06.914 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231910+0000", 2024-09-21T15:01:06.914 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4549000000000001e-05 2024-09-21T15:01:06.915 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.915 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.915 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:06.915 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231917+0000", 2024-09-21T15:01:06.915 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5730000000000002e-06 2024-09-21T15:01:06.915 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.915 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.915 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:06.915 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231988+0000", 2024-09-21T15:01:06.915 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0852999999999994e-05 2024-09-21T15:01:06.915 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.915 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.915 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:06.915 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.232000+0000", 2024-09-21T15:01:06.916 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2159e-05 2024-09-21T15:01:06.916 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.916 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.916 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:06.916 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256192+0000", 2024-09-21T15:01:06.916 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.024192073 2024-09-21T15:01:06.916 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.916 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.916 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:06.916 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256299+0000", 2024-09-21T15:01:06.916 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000106733 2024-09-21T15:01:06.916 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:06.916 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:06.916 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:06.917 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.917 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.917 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140055 3.16 3:69bb9803:::rbd_data.113664780c85.00000000000000da:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:06.917 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.233922+0000", 2024-09-21T15:01:06.917 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.6101781519999996, 2024-09-21T15:01:06.917 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.022511768, 2024-09-21T15:01:06.917 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:06.917 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:06.917 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:06.917 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:06.917 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:06.917 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140055 2024-09-21T15:01:06.917 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.917 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:06.917 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.918 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:06.918 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233922+0000", 2024-09-21T15:01:06.918 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:06.918 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.918 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.918 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:06.918 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233922+0000", 2024-09-21T15:01:06.918 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:06.918 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.918 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.918 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:06.918 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233921+0000", 2024-09-21T15:01:06.918 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:06.918 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.919 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.919 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:06.919 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233927+0000", 2024-09-21T15:01:06.919 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0360000000000003e-06 2024-09-21T15:01:06.919 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.919 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.919 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:06.919 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233937+0000", 2024-09-21T15:01:06.919 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6439999999999995e-06 2024-09-21T15:01:06.919 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.919 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.919 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:06.919 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233944+0000", 2024-09-21T15:01:06.919 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7710000000000006e-06 2024-09-21T15:01:06.920 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.920 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.920 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:06.920 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233996+0000", 2024-09-21T15:01:06.920 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1597000000000001e-05 2024-09-21T15:01:06.920 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.920 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.920 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:06.920 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.234005+0000", 2024-09-21T15:01:06.920 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5960000000000004e-06 2024-09-21T15:01:06.920 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.920 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.920 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:06.920 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256332+0000", 2024-09-21T15:01:06.920 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0223271949999999 2024-09-21T15:01:06.921 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.921 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.921 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:06.921 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256434+0000", 2024-09-21T15:01:06.921 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010180599999999999 2024-09-21T15:01:06.921 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:06.921 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:06.921 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:06.921 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.921 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.921 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140143 3.16 3:6d3ed596:::rbd_data.113664780c85.0000000000000184:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:06.921 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.239496+0000", 2024-09-21T15:01:06.921 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.604603623, 2024-09-21T15:01:06.921 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.017076576, 2024-09-21T15:01:06.922 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:06.922 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:06.922 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:06.922 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:06.922 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:06.922 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140143 2024-09-21T15:01:06.922 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.922 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:06.922 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.922 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:06.922 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239496+0000", 2024-09-21T15:01:06.922 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:06.922 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.922 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.923 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:06.923 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239496+0000", 2024-09-21T15:01:06.923 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:06.923 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.923 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.923 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:06.923 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239496+0000", 2024-09-21T15:01:06.923 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:06.923 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.923 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.923 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:06.923 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239498+0000", 2024-09-21T15:01:06.923 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.509e-06 2024-09-21T15:01:06.923 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.924 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.924 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:06.924 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239504+0000", 2024-09-21T15:01:06.924 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6869999999999999e-06 2024-09-21T15:01:06.924 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.924 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.924 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:06.924 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239509+0000", 2024-09-21T15:01:06.924 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8860000000000003e-06 2024-09-21T15:01:06.924 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.924 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.924 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:06.924 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239572+0000", 2024-09-21T15:01:06.924 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2908000000000001e-05 2024-09-21T15:01:06.924 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.925 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.925 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:06.925 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239580+0000", 2024-09-21T15:01:06.925 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4270000000000008e-06 2024-09-21T15:01:06.925 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.925 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.925 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:06.925 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256457+0000", 2024-09-21T15:01:06.925 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0168770979999999 2024-09-21T15:01:06.925 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.925 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.925 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:06.925 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256573+0000", 2024-09-21T15:01:06.925 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00011538400000000001 2024-09-21T15:01:06.925 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:06.926 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:06.926 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:06.926 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.926 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.926 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140163 3.16 3:694e82eb:::rbd_data.113664780c85.0000000000000052:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:06.926 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.240334+0000", 2024-09-21T15:01:06.926 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.603765911, 2024-09-21T15:01:06.926 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.016365537, 2024-09-21T15:01:06.926 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:06.926 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:06.926 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:06.926 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:06.926 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:06.926 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140163 2024-09-21T15:01:06.926 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.927 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:06.927 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.927 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:06.927 INFO:teuthology.orchestra.run.smithi032.stdout:5575 write 0x80eb88d thru 0x80f3941 (0x80b5 bytes) 2024-09-21T15:01:06.927 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240334+0000", 2024-09-21T15:01:06.927 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:06.928 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.928 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.928 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:06.928 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240334+0000", 2024-09-21T15:01:06.928 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:06.928 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.928 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.928 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:06.928 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240334+0000", 2024-09-21T15:01:06.928 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:06.928 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.928 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.928 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:06.929 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240337+0000", 2024-09-21T15:01:06.929 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.089e-06 2024-09-21T15:01:06.929 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.929 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.929 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:06.929 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240343+0000", 2024-09-21T15:01:06.929 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4420000000000003e-06 2024-09-21T15:01:06.929 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.929 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.929 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:06.929 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240348+0000", 2024-09-21T15:01:06.929 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1039999999999998e-06 2024-09-21T15:01:06.929 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.929 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.930 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:06.930 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240451+0000", 2024-09-21T15:01:06.930 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000102996 2024-09-21T15:01:06.930 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.930 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.930 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:06.930 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240460+0000", 2024-09-21T15:01:06.930 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5369999999999997e-06 2024-09-21T15:01:06.930 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.930 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.930 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:06.930 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256600+0000", 2024-09-21T15:01:06.930 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.016139884 2024-09-21T15:01:06.930 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.930 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.931 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:06.931 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256700+0000", 2024-09-21T15:01:06.931 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9844000000000002e-05 2024-09-21T15:01:06.931 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:06.931 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:06.931 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:06.931 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.931 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.931 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140164 3.16 3:68bfdc57:::rbd_data.113664780c85.0000000000000053:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:06.931 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.240358+0000", 2024-09-21T15:01:06.931 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.6037422169999997, 2024-09-21T15:01:06.931 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.01646094, 2024-09-21T15:01:06.931 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:06.931 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:06.931 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:06.932 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:06.932 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:06.932 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140164 2024-09-21T15:01:06.932 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.932 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:06.932 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.932 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:06.932 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240358+0000", 2024-09-21T15:01:06.932 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:06.932 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.932 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.932 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:06.932 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240358+0000", 2024-09-21T15:01:06.932 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:06.932 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.933 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.933 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:06.933 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240357+0000", 2024-09-21T15:01:06.933 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:06.933 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.933 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.933 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:06.933 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240360+0000", 2024-09-21T15:01:06.933 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7089999999999998e-06 2024-09-21T15:01:06.933 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.933 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.933 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:06.933 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240368+0000", 2024-09-21T15:01:06.933 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7270000000000007e-06 2024-09-21T15:01:06.934 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.934 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.934 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:06.934 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240373+0000", 2024-09-21T15:01:06.934 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4060000000000004e-06 2024-09-21T15:01:06.934 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.934 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.934 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:06.934 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240473+0000", 2024-09-21T15:01:06.934 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000100132 2024-09-21T15:01:06.934 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.934 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.934 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:06.934 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240476+0000", 2024-09-21T15:01:06.935 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.332e-06 2024-09-21T15:01:06.935 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.935 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.935 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:06.935 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256724+0000", 2024-09-21T15:01:06.935 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0162471930000001 2024-09-21T15:01:06.935 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.935 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.935 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:06.935 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256819+0000", 2024-09-21T15:01:06.935 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5007999999999996e-05 2024-09-21T15:01:06.935 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:06.935 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:06.935 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:06.935 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.936 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.936 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140188 3.16 3:6f6217d4:::rbd_data.113664780c85.00000000000000fb:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:06.936 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.240978+0000", 2024-09-21T15:01:06.936 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 4.6031215579999998, 2024-09-21T15:01:06.936 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.015983809, 2024-09-21T15:01:06.936 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:06.936 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:06.936 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:06.936 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:06.936 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:06.936 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140188 2024-09-21T15:01:06.936 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.936 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:06.936 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.936 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:06.937 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240978+0000", 2024-09-21T15:01:06.937 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:06.937 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.937 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.937 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:06.937 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240978+0000", 2024-09-21T15:01:06.937 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:06.937 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.937 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.937 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:06.937 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240978+0000", 2024-09-21T15:01:06.937 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:06.937 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.937 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.937 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:06.938 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240981+0000", 2024-09-21T15:01:06.938 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1559999999999999e-06 2024-09-21T15:01:06.938 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.938 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.938 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:06.938 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240987+0000", 2024-09-21T15:01:06.938 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.229e-06 2024-09-21T15:01:06.938 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.938 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.938 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:06.938 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240992+0000", 2024-09-21T15:01:06.938 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9710000000000003e-06 2024-09-21T15:01:06.938 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.938 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.938 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:06.939 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.241361+0000", 2024-09-21T15:01:06.939 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000369236 2024-09-21T15:01:06.939 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.939 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.939 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:06.939 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.241369+0000", 2024-09-21T15:01:06.939 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6199999999999999e-06 2024-09-21T15:01:06.939 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.939 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.939 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:06.939 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256844+0000", 2024-09-21T15:01:06.939 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0154751609999999 2024-09-21T15:01:06.939 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:06.939 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:06.940 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:06.940 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256962+0000", 2024-09-21T15:01:06.940 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000117964 2024-09-21T15:01:06.940 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:06.940 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:06.940 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:06.940 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:06.940 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:06.940 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:06.940 INFO:teuthology.orchestra.run.smithi032.stdout:5576 writesame 0x236b3bd thru 0x236d040 data_size 0xa(0x1c84 bytes) 2024-09-21T15:01:06.941 INFO:teuthology.orchestra.run.smithi032.stdout:5577 read 0x578005e thru 0x578b62e (0xb5d1 bytes) 2024-09-21T15:01:06.942 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T15:01:06.942 INFO:teuthology.orchestra.run.smithi032.stdout:5578 trunc from 0x80f3942 to 0x36f335e 2024-09-21T15:01:06.944 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-21T15:01:06.945 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-09-21T15:01:06.945 INFO:teuthology.orchestra.run.smithi032.stdout:5579 write 0x9523ef1 thru 0x952cfc1 (0x90d1 bytes) 2024-09-21T15:01:06.946 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":264,"stamp":"2024-09-21T15:01:06.158077+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1052048706,"num_objects":1338,"num_object_clones":0,"num_object_copies":2676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1338,"num_whiteouts":0,"num_read":144511,"num_read_kb":1152527,"num_write":26194,"num_write_kb":886220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1920,"num_bytes_recovered":145354624,"num_keys_recovered":229,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17355,"ondisk_log_size":17355,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":85,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":464376,"kb_used_data":86648,"kb_used_omap":12,"kb_used_meta":377587,"kb_avail":754510344,"statfs":{"total":773094113280,"available":772618592256,"internally_reserved":0,"allocated":88727552,"data_stored":646871178,"data_compressed":14204054,"data_compressed_allocated":58728448,"data_compressed_original":631348116,"omap_allocated":12744,"internal_metadata":386649656},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,31],"upper_bound":1024},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":70414573,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1851,"num_read_kb":25577,"num_write":148,"num_write_kb":20269,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":539,"num_bytes_recovered":41668853,"num_keys_recovered":77,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.954975"},"pg_stats":[{"pgid":"3.1f","version":"172'452","reported_seq":3450,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.157805+0000","last_change":"2024-09-21T15:01:04.223131+0000","last_active":"2024-09-21T15:01:05.157805+0000","last_peered":"2024-09-21T15:01:05.157805+0000","last_clean":"2024-09-21T15:01:05.157805+0000","last_became_active":"2024-09-21T14:58:19.410055+0000","last_became_peered":"2024-09-21T14:58:19.410055+0000","last_unstale":"2024-09-21T15:01:05.157805+0000","last_undegraded":"2024-09-21T15:01:05.157805+0000","last_fullsized":"2024-09-21T15:01:05.157805+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":452,"log_dups_size":0,"ondisk_log_size":452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:49:13.944219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063336100000000004,"stat_sum":{"num_bytes":32245069,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":3403,"num_read_kb":42213,"num_write":684,"num_write_kb":34241,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.1e","version":"176'366","reported_seq":2962,"reported_epoch":177,"state":"peering","last_fresh":"2024-09-21T15:01:05.160058+0000","last_change":"2024-09-21T15:01:05.160420+0000","last_active":"2024-09-21T15:01:05.160058+0000","last_peered":"2024-09-21T15:01:04.425541+0000","last_clean":"2024-09-21T15:01:00.453521+0000","last_became_active":"2024-09-21T15:01:03.253387+0000","last_became_peered":"2024-09-21T15:01:03.253387+0000","last_unstale":"2024-09-21T15:01:05.160058+0000","last_undegraded":"2024-09-21T15:01:05.160058+0000","last_fullsized":"2024-09-21T15:01:05.160058+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":366,"log_dups_size":0,"ondisk_log_size":366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:27:07.402534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021611780000000001,"stat_sum":{"num_bytes":19327265,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":0,"num_read":2999,"num_read_kb":22037,"num_write":453,"num_write_kb":16509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":148,"num_bytes_recovered":11635695,"num_keys_recovered":22,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,3],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"6b","length":"1"}]},{"pgid":"3.1d","version":"175'260","reported_seq":3611,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.164700+0000","last_change":"2024-09-21T15:01:04.229475+0000","last_active":"2024-09-21T15:01:05.164700+0000","last_peered":"2024-09-21T15:01:05.164700+0000","last_clean":"2024-09-21T15:01:05.164700+0000","last_became_active":"2024-09-21T14:58:19.411387+0000","last_became_peered":"2024-09-21T14:58:19.411387+0000","last_unstale":"2024-09-21T15:01:05.164700+0000","last_undegraded":"2024-09-21T15:01:05.164700+0000","last_fullsized":"2024-09-21T15:01:05.164700+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":260,"log_dups_size":0,"ondisk_log_size":260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:37:57.597069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054775500000000003,"stat_sum":{"num_bytes":18081929,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":3016,"num_read_kb":23636,"num_write":434,"num_write_kb":24977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.1c","version":"175'430","reported_seq":2874,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.187232+0000","last_change":"2024-09-21T15:01:04.234927+0000","last_active":"2024-09-21T15:01:05.187232+0000","last_peered":"2024-09-21T15:01:05.187232+0000","last_clean":"2024-09-21T15:01:05.187232+0000","last_became_active":"2024-09-21T14:58:19.411306+0000","last_became_peered":"2024-09-21T14:58:19.411306+0000","last_unstale":"2024-09-21T15:01:05.187232+0000","last_undegraded":"2024-09-21T15:01:05.187232+0000","last_fullsized":"2024-09-21T15:01:05.187232+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":430,"log_dups_size":0,"ondisk_log_size":430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:20:27.079795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010334350000000001,"stat_sum":{"num_bytes":19507114,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":2814,"num_read_kb":20995,"num_write":622,"num_write_kb":24708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.1b","version":"175'422","reported_seq":2730,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.158957+0000","last_change":"2024-09-21T15:01:04.227802+0000","last_active":"2024-09-21T15:01:05.158957+0000","last_peered":"2024-09-21T15:01:05.158957+0000","last_clean":"2024-09-21T15:01:05.158957+0000","last_became_active":"2024-09-21T14:59:31.131051+0000","last_became_peered":"2024-09-21T14:59:31.131051+0000","last_unstale":"2024-09-21T15:01:05.158957+0000","last_undegraded":"2024-09-21T15:01:05.158957+0000","last_fullsized":"2024-09-21T15:01:05.158957+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":422,"log_dups_size":0,"ondisk_log_size":422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:21:37.320774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001254673,"stat_sum":{"num_bytes":25447559,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":2614,"num_read_kb":28976,"num_write":559,"num_write_kb":20747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":117,"num_bytes_recovered":7757116,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.1a","version":"175'352","reported_seq":3233,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.187249+0000","last_change":"2024-09-21T15:01:04.234672+0000","last_active":"2024-09-21T15:01:05.187249+0000","last_peered":"2024-09-21T15:01:05.187249+0000","last_clean":"2024-09-21T15:01:05.187249+0000","last_became_active":"2024-09-21T14:58:19.409415+0000","last_became_peered":"2024-09-21T14:58:19.409415+0000","last_unstale":"2024-09-21T15:01:05.187249+0000","last_undegraded":"2024-09-21T15:01:05.187249+0000","last_fullsized":"2024-09-21T15:01:05.187249+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":352,"log_dups_size":0,"ondisk_log_size":352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:28:46.336301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060307399999999999,"stat_sum":{"num_bytes":31396202,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":3008,"num_read_kb":24746,"num_write":475,"num_write_kb":22526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.19","version":"175'648","reported_seq":3837,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.159011+0000","last_change":"2024-09-21T15:01:04.228157+0000","last_active":"2024-09-21T15:01:05.159011+0000","last_peered":"2024-09-21T15:01:05.159011+0000","last_clean":"2024-09-21T15:01:05.159011+0000","last_became_active":"2024-09-21T14:59:32.130624+0000","last_became_peered":"2024-09-21T14:59:32.130624+0000","last_unstale":"2024-09-21T15:01:05.159011+0000","last_undegraded":"2024-09-21T15:01:05.159011+0000","last_fullsized":"2024-09-21T15:01:05.159011+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":648,"log_dups_size":0,"ondisk_log_size":648,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:37:53.312733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001540409,"stat_sum":{"num_bytes":47896216,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":4950,"num_read_kb":51571,"num_write":1030,"num_write_kb":28173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":128,"num_bytes_recovered":11896992,"num_keys_recovered":44,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.18","version":"175'525","reported_seq":7399,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.157243+0000","last_change":"2024-09-21T15:01:04.223390+0000","last_active":"2024-09-21T15:01:05.157243+0000","last_peered":"2024-09-21T15:01:05.157243+0000","last_clean":"2024-09-21T15:01:05.157243+0000","last_became_active":"2024-09-21T14:58:19.409371+0000","last_became_peered":"2024-09-21T14:58:19.409371+0000","last_unstale":"2024-09-21T15:01:05.157243+0000","last_undegraded":"2024-09-21T15:01:05.157243+0000","last_fullsized":"2024-09-21T15:01:05.157243+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":525,"log_dups_size":0,"ondisk_log_size":525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:35:27.143445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076656300000000005,"stat_sum":{"num_bytes":36011050,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":8009,"num_read_kb":41151,"num_write":765,"num_write_kb":29537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.17","version":"172'334","reported_seq":3678,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.158315+0000","last_change":"2024-09-21T15:01:04.225804+0000","last_active":"2024-09-21T15:01:05.158315+0000","last_peered":"2024-09-21T15:01:05.158315+0000","last_clean":"2024-09-21T15:01:05.158315+0000","last_became_active":"2024-09-21T14:58:19.410306+0000","last_became_peered":"2024-09-21T14:58:19.410306+0000","last_unstale":"2024-09-21T15:01:05.158315+0000","last_undegraded":"2024-09-21T15:01:05.158315+0000","last_fullsized":"2024-09-21T15:01:05.158315+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":334,"log_dups_size":0,"ondisk_log_size":334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:33:50.089165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030273969999999998,"stat_sum":{"num_bytes":25072080,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":3370,"num_read_kb":27721,"num_write":557,"num_write_kb":31849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.16","version":"175'887","reported_seq":4751,"reported_epoch":177,"state":"creating+peering","last_fresh":"2024-09-21T15:01:04.608919+0000","last_change":"2024-09-21T15:01:05.877830+0000","last_active":"2024-09-21T15:01:05.877830+0000","last_peered":"2024-09-21T15:01:04.608919+0000","last_clean":"2024-09-21T15:01:00.518248+0000","last_became_active":"2024-09-21T15:01:03.252683+0000","last_became_peered":"2024-09-21T15:01:03.252683+0000","last_unstale":"2024-09-21T15:01:04.608919+0000","last_undegraded":"2024-09-21T15:01:04.608919+0000","last_fullsized":"2024-09-21T15:01:03.215921+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":887,"log_dups_size":0,"ondisk_log_size":887,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:19:04.988784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001973472,"stat_sum":{"num_bytes":33667875,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":6806,"num_read_kb":40610,"num_write":1387,"num_write_kb":26337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":271,"num_bytes_recovered":16821266,"num_keys_recovered":55,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"6b","length":"1"}]},{"pgid":"3.15","version":"175'717","reported_seq":3617,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.169730+0000","last_change":"2024-09-21T15:01:04.231959+0000","last_active":"2024-09-21T15:01:05.169730+0000","last_peered":"2024-09-21T15:01:05.169730+0000","last_clean":"2024-09-21T15:01:05.169730+0000","last_became_active":"2024-09-21T14:58:19.410687+0000","last_became_peered":"2024-09-21T14:58:19.410687+0000","last_unstale":"2024-09-21T15:01:05.169730+0000","last_undegraded":"2024-09-21T15:01:05.169730+0000","last_fullsized":"2024-09-21T15:01:05.169730+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":717,"log_dups_size":0,"ondisk_log_size":717,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:21:42.226982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033742220000000001,"stat_sum":{"num_bytes":45321568,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":4682,"num_read_kb":48927,"num_write":1212,"num_write_kb":32331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.14","version":"175'433","reported_seq":2993,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.164396+0000","last_change":"2024-09-21T15:01:04.229559+0000","last_active":"2024-09-21T15:01:05.164396+0000","last_peered":"2024-09-21T15:01:05.164396+0000","last_clean":"2024-09-21T15:01:05.164396+0000","last_became_active":"2024-09-21T14:58:19.407310+0000","last_became_peered":"2024-09-21T14:58:19.407310+0000","last_unstale":"2024-09-21T15:01:05.164396+0000","last_undegraded":"2024-09-21T15:01:05.164396+0000","last_fullsized":"2024-09-21T15:01:05.164396+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:47:46.079115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052455500000000001,"stat_sum":{"num_bytes":43744336,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":2858,"num_read_kb":38772,"num_write":581,"num_write_kb":32120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":171,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-21T15:01:04.229713+0000","last_change":"2024-09-21T14:58:11.348177+0000","last_active":"2024-09-21T15:01:04.229713+0000","last_peered":"2024-09-21T15:01:04.229713+0000","last_clean":"2024-09-21T15:01:04.229713+0000","last_became_active":"2024-09-21T14:58:09.328417+0000","last_became_peered":"2024-09-21T14:58:09.328417+0000","last_unstale":"2024-09-21T15:01:04.229713+0000","last_undegraded":"2024-09-21T15:01:04.229713+0000","last_fullsized":"2024-09-21T15:01:04.229713+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:52:22.671185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018917490000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"172'367","reported_seq":5379,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.158972+0000","last_change":"2024-09-21T15:01:04.228240+0000","last_active":"2024-09-21T15:01:05.158972+0000","last_peered":"2024-09-21T15:01:05.158972+0000","last_clean":"2024-09-21T15:01:05.158972+0000","last_became_active":"2024-09-21T14:59:32.131232+0000","last_became_peered":"2024-09-21T14:59:32.131232+0000","last_unstale":"2024-09-21T15:01:05.158972+0000","last_undegraded":"2024-09-21T15:01:05.158972+0000","last_fullsized":"2024-09-21T15:01:05.158972+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":367,"log_dups_size":0,"ondisk_log_size":367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:31:04.421782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001438581,"stat_sum":{"num_bytes":50093146,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":5172,"num_read_kb":51612,"num_write":497,"num_write_kb":25669,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":3585713,"num_keys_recovered":9,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":96,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-21T15:01:04.227015+0000","last_change":"2024-09-21T14:59:28.228883+0000","last_active":"2024-09-21T15:01:04.227015+0000","last_peered":"2024-09-21T15:01:04.227015+0000","last_clean":"2024-09-21T15:01:04.227015+0000","last_became_active":"2024-09-21T14:59:28.228464+0000","last_became_peered":"2024-09-21T14:59:28.228464+0000","last_unstale":"2024-09-21T15:01:04.227015+0000","last_undegraded":"2024-09-21T15:01:04.227015+0000","last_fullsized":"2024-09-21T15:01:04.227015+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:00:26.310864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"175'809","reported_seq":3401,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.159556+0000","last_change":"2024-09-21T15:01:04.228668+0000","last_active":"2024-09-21T15:01:05.159556+0000","last_peered":"2024-09-21T15:01:05.159556+0000","last_clean":"2024-09-21T15:01:05.159556+0000","last_became_active":"2024-09-21T14:59:31.132581+0000","last_became_peered":"2024-09-21T14:59:31.132581+0000","last_unstale":"2024-09-21T15:01:05.159556+0000","last_undegraded":"2024-09-21T15:01:05.159556+0000","last_fullsized":"2024-09-21T15:01:05.159556+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":809,"log_dups_size":0,"ondisk_log_size":809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:36:45.467515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016620179999999999,"stat_sum":{"num_bytes":28736355,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":4991,"num_read_kb":30499,"num_write":1191,"num_write_kb":23913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":119,"num_bytes_recovered":6871924,"num_keys_recovered":11,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":187,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-21T15:01:04.229145+0000","last_change":"2024-09-21T14:59:28.231502+0000","last_active":"2024-09-21T15:01:04.229145+0000","last_peered":"2024-09-21T15:01:04.229145+0000","last_clean":"2024-09-21T15:01:04.229145+0000","last_became_active":"2024-09-21T14:59:28.231043+0000","last_became_peered":"2024-09-21T14:59:28.231043+0000","last_unstale":"2024-09-21T15:01:04.229145+0000","last_undegraded":"2024-09-21T15:01:04.229145+0000","last_fullsized":"2024-09-21T15:01:04.229145+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:09:46.773690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019338770000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"172'553","reported_seq":5095,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.158909+0000","last_change":"2024-09-21T15:01:04.227811+0000","last_active":"2024-09-21T15:01:05.158909+0000","last_peered":"2024-09-21T15:01:05.158909+0000","last_clean":"2024-09-21T15:01:05.158909+0000","last_became_active":"2024-09-21T15:00:34.347238+0000","last_became_peered":"2024-09-21T15:00:34.347238+0000","last_unstale":"2024-09-21T15:01:05.158909+0000","last_undegraded":"2024-09-21T15:01:05.158909+0000","last_fullsized":"2024-09-21T15:01:05.158909+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":553,"log_dups_size":0,"ondisk_log_size":553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:31:33.639268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001277607,"stat_sum":{"num_bytes":39912228,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":5081,"num_read_kb":38156,"num_write":939,"num_write_kb":39278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":14049436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":172,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-21T15:01:04.222407+0000","last_change":"2024-09-21T14:58:11.406176+0000","last_active":"2024-09-21T15:01:04.222407+0000","last_peered":"2024-09-21T15:01:04.222407+0000","last_clean":"2024-09-21T15:01:04.222407+0000","last_became_active":"2024-09-21T14:58:09.777465+0000","last_became_peered":"2024-09-21T14:58:09.777465+0000","last_unstale":"2024-09-21T15:01:04.222407+0000","last_undegraded":"2024-09-21T15:01:04.222407+0000","last_fullsized":"2024-09-21T15:01:04.222407+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:55:04.667816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095704000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"172'662","reported_seq":2540,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.162165+0000","last_change":"2024-09-21T15:01:04.224786+0000","last_active":"2024-09-21T15:01:05.162165+0000","last_peered":"2024-09-21T15:01:05.162165+0000","last_clean":"2024-09-21T15:01:05.162165+0000","last_became_active":"2024-09-21T14:58:19.408971+0000","last_became_peered":"2024-09-21T14:58:19.408971+0000","last_unstale":"2024-09-21T15:01:05.162165+0000","last_undegraded":"2024-09-21T15:01:05.162165+0000","last_fullsized":"2024-09-21T15:01:05.162165+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":662,"log_dups_size":0,"ondisk_log_size":662,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:29:52.642121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000380054,"stat_sum":{"num_bytes":19474374,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":3561,"num_read_kb":20372,"num_write":1084,"num_write_kb":19812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.3","version":"172'738","reported_seq":2939,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.169226+0000","last_change":"2024-09-21T15:01:04.231096+0000","last_active":"2024-09-21T15:01:05.169226+0000","last_peered":"2024-09-21T15:01:05.169226+0000","last_clean":"2024-09-21T15:01:05.169226+0000","last_became_active":"2024-09-21T14:58:19.417453+0000","last_became_peered":"2024-09-21T14:58:19.417453+0000","last_unstale":"2024-09-21T15:01:05.169226+0000","last_undegraded":"2024-09-21T15:01:05.169226+0000","last_fullsized":"2024-09-21T15:01:05.169226+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":738,"log_dups_size":0,"ondisk_log_size":738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:19:27.087272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002785708,"stat_sum":{"num_bytes":15820678,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":4393,"num_read_kb":24172,"num_write":1224,"num_write_kb":20113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":173,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-21T15:01:04.232215+0000","last_change":"2024-09-21T14:58:11.347504+0000","last_active":"2024-09-21T15:01:04.232215+0000","last_peered":"2024-09-21T15:01:04.232215+0000","last_clean":"2024-09-21T15:01:04.232215+0000","last_became_active":"2024-09-21T14:58:09.775981+0000","last_became_peered":"2024-09-21T14:58:09.775981+0000","last_unstale":"2024-09-21T15:01:04.232215+0000","last_undegraded":"2024-09-21T15:01:04.232215+0000","last_fullsized":"2024-09-21T15:01:04.232215+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:47:16.286652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012568880000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"176'534","reported_seq":2983,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.157022+0000","last_change":"2024-09-21T15:01:04.223338+0000","last_active":"2024-09-21T15:01:05.157022+0000","last_peered":"2024-09-21T15:01:05.157022+0000","last_clean":"2024-09-21T15:01:05.157022+0000","last_became_active":"2024-09-21T14:58:19.405811+0000","last_became_peered":"2024-09-21T14:58:19.405811+0000","last_unstale":"2024-09-21T15:01:05.157022+0000","last_undegraded":"2024-09-21T15:01:05.157022+0000","last_fullsized":"2024-09-21T15:01:05.157022+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":534,"log_dups_size":0,"ondisk_log_size":534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:36:25.551607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083259800000000004,"stat_sum":{"num_bytes":20501650,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":3805,"num_read_kb":26326,"num_write":700,"num_write_kb":25158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":168,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-21T15:01:00.171240+0000","last_change":"2024-09-21T14:58:11.342210+0000","last_active":"2024-09-21T15:01:00.171240+0000","last_peered":"2024-09-21T15:01:00.171240+0000","last_clean":"2024-09-21T15:01:00.171240+0000","last_became_active":"2024-09-21T14:58:09.330624+0000","last_became_peered":"2024-09-21T14:58:09.330624+0000","last_unstale":"2024-09-21T15:01:00.171240+0000","last_undegraded":"2024-09-21T15:01:00.171240+0000","last_fullsized":"2024-09-21T15:01:00.171240+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:29:43.229538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001051151,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"175'276","reported_seq":2007,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.166575+0000","last_change":"2024-09-21T15:01:05.166575+0000","last_active":"2024-09-21T15:01:05.166575+0000","last_peered":"2024-09-21T15:01:05.166575+0000","last_clean":"2024-09-21T15:01:05.166575+0000","last_became_active":"2024-09-21T15:01:05.164862+0000","last_became_peered":"2024-09-21T15:01:05.164862+0000","last_unstale":"2024-09-21T15:01:05.166575+0000","last_undegraded":"2024-09-21T15:01:05.166575+0000","last_fullsized":"2024-09-21T15:01:05.166575+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":276,"log_dups_size":0,"ondisk_log_size":276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:57:46.675335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046340299999999998,"stat_sum":{"num_bytes":42467012,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":2366,"num_read_kb":39964,"num_write":407,"num_write_kb":39026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":254,"num_bytes_recovered":20571253,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":171,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-21T15:01:04.227843+0000","last_change":"2024-09-21T14:58:11.346766+0000","last_active":"2024-09-21T15:01:04.227843+0000","last_peered":"2024-09-21T15:01:04.227843+0000","last_clean":"2024-09-21T15:01:04.227843+0000","last_became_active":"2024-09-21T14:58:09.329748+0000","last_became_peered":"2024-09-21T14:58:09.329748+0000","last_unstale":"2024-09-21T15:01:04.227843+0000","last_undegraded":"2024-09-21T15:01:04.227843+0000","last_fullsized":"2024-09-21T15:01:04.227843+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:31:20.267675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052698000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"175'715","reported_seq":5700,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.160742+0000","last_change":"2024-09-21T15:01:04.224483+0000","last_active":"2024-09-21T15:01:05.160742+0000","last_peered":"2024-09-21T15:01:05.160742+0000","last_clean":"2024-09-21T15:01:05.160742+0000","last_became_active":"2024-09-21T14:58:41.401003+0000","last_became_peered":"2024-09-21T14:58:41.401003+0000","last_unstale":"2024-09-21T15:01:05.160742+0000","last_undegraded":"2024-09-21T15:01:05.160742+0000","last_fullsized":"2024-09-21T15:01:05.160742+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":715,"log_dups_size":0,"ondisk_log_size":715,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:05:37.644835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046693700000000001,"stat_sum":{"num_bytes":45819535,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":7513,"num_read_kb":42793,"num_write":987,"num_write_kb":21873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1403348,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":172,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-21T15:01:04.228858+0000","last_change":"2024-09-21T14:58:11.347700+0000","last_active":"2024-09-21T15:01:04.228858+0000","last_peered":"2024-09-21T15:01:04.228858+0000","last_clean":"2024-09-21T15:01:04.228858+0000","last_became_active":"2024-09-21T14:58:09.438842+0000","last_became_peered":"2024-09-21T14:58:09.438842+0000","last_unstale":"2024-09-21T15:01:04.228858+0000","last_undegraded":"2024-09-21T15:01:04.228858+0000","last_fullsized":"2024-09-21T15:01:04.228858+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:41:34.626249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001426194,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":320,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-21T15:01:04.233768+0000","last_change":"2024-09-21T14:58:03.616055+0000","last_active":"2024-09-21T15:01:04.233768+0000","last_peered":"2024-09-21T15:01:04.233768+0000","last_clean":"2024-09-21T15:01:04.233768+0000","last_became_active":"2024-09-21T14:58:03.613214+0000","last_became_peered":"2024-09-21T14:58:03.613214+0000","last_unstale":"2024-09-21T15:01:04.233768+0000","last_undegraded":"2024-09-21T15:01:04.233768+0000","last_fullsized":"2024-09-21T15:01:04.233768+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:02.180162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:02.180162+0000","last_clean_scrub_stamp":"2024-09-21T14:58:02.180162+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:29:05.860361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"172'602","reported_seq":4041,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.157772+0000","last_change":"2024-09-21T15:01:04.227318+0000","last_active":"2024-09-21T15:01:05.157772+0000","last_peered":"2024-09-21T15:01:05.157772+0000","last_clean":"2024-09-21T15:01:05.157772+0000","last_became_active":"2024-09-21T14:59:31.131460+0000","last_became_peered":"2024-09-21T14:59:31.131460+0000","last_unstale":"2024-09-21T15:01:05.157772+0000","last_undegraded":"2024-09-21T15:01:05.157772+0000","last_fullsized":"2024-09-21T15:01:05.157772+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":602,"log_dups_size":0,"ondisk_log_size":602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:01:38.353476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098975100000000009,"stat_sum":{"num_bytes":37289213,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":4356,"num_read_kb":43318,"num_write":990,"num_write_kb":29471,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":119,"num_bytes_recovered":6097618,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.9","version":"173'786","reported_seq":2989,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.164259+0000","last_change":"2024-09-21T15:01:04.232710+0000","last_active":"2024-09-21T15:01:05.164259+0000","last_peered":"2024-09-21T15:01:05.164259+0000","last_clean":"2024-09-21T15:01:05.164259+0000","last_became_active":"2024-09-21T14:59:28.233956+0000","last_became_peered":"2024-09-21T14:59:28.233956+0000","last_unstale":"2024-09-21T15:01:05.164259+0000","last_undegraded":"2024-09-21T15:01:05.164259+0000","last_fullsized":"2024-09-21T15:01:05.164259+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":786,"log_dups_size":0,"ondisk_log_size":786,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:29:09.682792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027159770000000001,"stat_sum":{"num_bytes":23720917,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":4523,"num_read_kb":33253,"num_write":1276,"num_write_kb":30634,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":91,"num_bytes_recovered":6925016,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.a","version":"176'786","reported_seq":3964,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.187273+0000","last_change":"2024-09-21T15:01:04.234662+0000","last_active":"2024-09-21T15:01:05.187273+0000","last_peered":"2024-09-21T15:01:05.187273+0000","last_clean":"2024-09-21T15:01:05.187273+0000","last_became_active":"2024-09-21T15:00:27.280971+0000","last_became_peered":"2024-09-21T15:00:27.280971+0000","last_unstale":"2024-09-21T15:01:05.187273+0000","last_undegraded":"2024-09-21T15:01:05.187273+0000","last_fullsized":"2024-09-21T15:01:05.187273+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":786,"log_dups_size":0,"ondisk_log_size":786,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:26:07.617059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076445899999999999,"stat_sum":{"num_bytes":20136068,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":5765,"num_read_kb":29533,"num_write":1129,"num_write_kb":31048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":9228123,"num_keys_recovered":33,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.b","version":"172'448","reported_seq":2919,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-21T15:01:00.700977+0000","last_change":"2024-09-21T15:00:59.287225+0000","last_active":"2024-09-21T15:01:00.700977+0000","last_peered":"2024-09-21T15:01:00.700977+0000","last_clean":"2024-09-21T15:01:00.700977+0000","last_became_active":"2024-09-21T14:58:19.410679+0000","last_became_peered":"2024-09-21T14:58:19.410679+0000","last_unstale":"2024-09-21T15:01:00.700977+0000","last_undegraded":"2024-09-21T15:01:00.700977+0000","last_fullsized":"2024-09-21T15:01:00.700977+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":448,"log_dups_size":0,"ondisk_log_size":448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:07:40.758932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0009774880000000001,"stat_sum":{"num_bytes":31436179,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":2840,"num_read_kb":35254,"num_write":697,"num_write_kb":37108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"6b","length":"1"}]},{"pgid":"3.c","version":"172'408","reported_seq":4312,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.169078+0000","last_change":"2024-09-21T15:01:04.231357+0000","last_active":"2024-09-21T15:01:05.169078+0000","last_peered":"2024-09-21T15:01:05.169078+0000","last_clean":"2024-09-21T15:01:05.169078+0000","last_became_active":"2024-09-21T14:59:25.130057+0000","last_became_peered":"2024-09-21T14:59:25.130057+0000","last_unstale":"2024-09-21T15:01:05.169078+0000","last_undegraded":"2024-09-21T15:01:05.169078+0000","last_fullsized":"2024-09-21T15:01:05.169078+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:53:33.752550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003377793,"stat_sum":{"num_bytes":22999993,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":4337,"num_read_kb":28452,"num_write":576,"num_write_kb":23757,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":9059600,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.d","version":"172'422","reported_seq":4914,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.164463+0000","last_change":"2024-09-21T15:01:04.233218+0000","last_active":"2024-09-21T15:01:05.164463+0000","last_peered":"2024-09-21T15:01:05.164463+0000","last_clean":"2024-09-21T15:01:05.164463+0000","last_became_active":"2024-09-21T14:58:19.407884+0000","last_became_peered":"2024-09-21T14:58:19.407884+0000","last_unstale":"2024-09-21T15:01:05.164463+0000","last_undegraded":"2024-09-21T15:01:05.164463+0000","last_fullsized":"2024-09-21T15:01:05.164463+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":422,"log_dups_size":0,"ondisk_log_size":422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:28:57.111615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034473960000000001,"stat_sum":{"num_bytes":49032397,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":4783,"num_read_kb":55414,"num_write":689,"num_write_kb":35805,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.e","version":"171'774","reported_seq":2852,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.176277+0000","last_change":"2024-09-21T15:01:05.176277+0000","last_active":"2024-09-21T15:01:05.176277+0000","last_peered":"2024-09-21T15:01:05.176277+0000","last_clean":"2024-09-21T15:01:05.176277+0000","last_became_active":"2024-09-21T15:01:05.172427+0000","last_became_peered":"2024-09-21T15:01:05.172427+0000","last_unstale":"2024-09-21T15:01:05.176277+0000","last_undegraded":"2024-09-21T15:01:05.176277+0000","last_fullsized":"2024-09-21T15:01:05.176277+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":774,"log_dups_size":0,"ondisk_log_size":774,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:28:18.710621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093917399999999998,"stat_sum":{"num_bytes":12028520,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":4155,"num_read_kb":16497,"num_write":1272,"num_write_kb":16419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":138,"num_bytes_recovered":7486496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.f","version":"175'427","reported_seq":5010,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.169470+0000","last_change":"2024-09-21T15:01:04.229660+0000","last_active":"2024-09-21T15:01:05.169470+0000","last_peered":"2024-09-21T15:01:05.169470+0000","last_clean":"2024-09-21T15:01:05.169470+0000","last_became_active":"2024-09-21T14:58:19.410406+0000","last_became_peered":"2024-09-21T14:58:19.410406+0000","last_unstale":"2024-09-21T15:01:05.169470+0000","last_undegraded":"2024-09-21T15:01:05.169470+0000","last_fullsized":"2024-09-21T15:01:05.169470+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":427,"log_dups_size":0,"ondisk_log_size":427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:59:16.026939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014556770000000001,"stat_sum":{"num_bytes":49430534,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":4917,"num_read_kb":52382,"num_write":667,"num_write_kb":35936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.10","version":"172'427","reported_seq":4297,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.186765+0000","last_change":"2024-09-21T15:01:04.233986+0000","last_active":"2024-09-21T15:01:05.186765+0000","last_peered":"2024-09-21T15:01:05.186765+0000","last_clean":"2024-09-21T15:01:05.186765+0000","last_became_active":"2024-09-21T14:58:19.409935+0000","last_became_peered":"2024-09-21T14:58:19.409935+0000","last_unstale":"2024-09-21T15:01:05.186765+0000","last_undegraded":"2024-09-21T15:01:05.186765+0000","last_fullsized":"2024-09-21T15:01:05.186765+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":427,"log_dups_size":0,"ondisk_log_size":427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:10:36.252629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001679137,"stat_sum":{"num_bytes":49580759,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":3927,"num_read_kb":58252,"num_write":670,"num_write_kb":35408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.11","version":"172'457","reported_seq":4563,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.168682+0000","last_change":"2024-09-21T15:01:05.168682+0000","last_active":"2024-09-21T15:01:05.168682+0000","last_peered":"2024-09-21T15:01:05.168682+0000","last_clean":"2024-09-21T15:01:05.168682+0000","last_became_active":"2024-09-21T15:01:05.165955+0000","last_became_peered":"2024-09-21T15:01:05.165955+0000","last_unstale":"2024-09-21T15:01:05.168682+0000","last_undegraded":"2024-09-21T15:01:05.168682+0000","last_fullsized":"2024-09-21T15:01:05.168682+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":457,"log_dups_size":0,"ondisk_log_size":457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:34:54.111119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010096230000000001,"stat_sum":{"num_bytes":23766643,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":5218,"num_read_kb":27936,"num_write":524,"num_write_kb":22561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":166,"num_bytes_recovered":11965028,"num_keys_recovered":33,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.12","version":"176'480","reported_seq":6284,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.163311+0000","last_change":"2024-09-21T15:01:04.224935+0000","last_active":"2024-09-21T15:01:05.163311+0000","last_peered":"2024-09-21T15:01:05.163311+0000","last_clean":"2024-09-21T15:01:05.163311+0000","last_became_active":"2024-09-21T15:00:07.155144+0000","last_became_peered":"2024-09-21T15:00:07.155144+0000","last_unstale":"2024-09-21T15:01:05.163311+0000","last_undegraded":"2024-09-21T15:01:05.163311+0000","last_fullsized":"2024-09-21T15:01:05.163311+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":480,"log_dups_size":0,"ondisk_log_size":480,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:58:00.133639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010497799999999999,"stat_sum":{"num_bytes":38476413,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":7022,"num_read_kb":30524,"num_write":743,"num_write_kb":18554,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.13","version":"175'738","reported_seq":5364,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.164109+0000","last_change":"2024-09-21T15:01:04.229229+0000","last_active":"2024-09-21T15:01:05.164109+0000","last_peered":"2024-09-21T15:01:05.164109+0000","last_clean":"2024-09-21T15:01:05.164109+0000","last_became_active":"2024-09-21T14:58:19.412030+0000","last_became_peered":"2024-09-21T14:58:19.412030+0000","last_unstale":"2024-09-21T15:01:05.164109+0000","last_undegraded":"2024-09-21T15:01:05.164109+0000","last_fullsized":"2024-09-21T15:01:05.164109+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":738,"log_dups_size":0,"ondisk_log_size":738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:44:29.282391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046048699999999998,"stat_sum":{"num_bytes":53017442,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":7141,"num_read_kb":56363,"num_write":1012,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"6e","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1051458319,"num_objects":1334,"num_object_clones":0,"num_object_copies":2668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1334,"num_whiteouts":0,"num_read":144395,"num_read_kb":1152427,"num_write":26033,"num_write_kb":883802,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1920,"num_bytes_recovered":145354624,"num_keys_recovered":229,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":71651328,"data_stored":637465472,"data_compressed":14193650,"data_compressed_allocated":58654720,"data_compressed_original":630201236,"omap_allocated":0,"internal_metadata":0},"log_size":17235,"ondisk_log_size":17235,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":10404,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509480,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34572,"kb_used_data":7552,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94337268,"statfs":{"total":96636764160,"available":96601362432,"internally_reserved":0,"allocated":7733248,"data_stored":48875988,"data_compressed":1141276,"data_compressed_allocated":4558848,"data_compressed_original":47268443,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:00:03 2024","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.84499999999999997,"15min":0.83899999999999997},"min":{"1min":0.504,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.95099999999999996,"15min":0.95799999999999996},"min":{"1min":0.51400000000000001,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.4970000000000001,"5min":1.917,"15min":1.917},"last":0.80200000000000005}]},{"osd":1,"last update":"Sat Sep 21 15:00:03 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.78500000000000003,"15min":0.76900000000000002},"min":{"1min":0.626,"5min":0.312,"15min":0.312},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.879,"5min":0.83499999999999996,"15min":0.82699999999999996},"min":{"1min":0.53600000000000003,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.224,"5min":2.2149999999999999,"15min":2.2149999999999999},"last":0.70199999999999996}]},{"osd":2,"last update":"Sat Sep 21 15:00:03 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.81699999999999995,"15min":0.81100000000000005},"min":{"1min":0.495,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.1859999999999999,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.95099999999999996,"15min":0.94799999999999995},"min":{"1min":0.72199999999999998,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.3300000000000001,"5min":1.466,"15min":1.466},"last":0.77800000000000002}]},{"osd":3,"last update":"Sat Sep 21 15:00:03 2024","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.86199999999999999,"15min":0.85399999999999998},"min":{"1min":0.67200000000000004,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.3120000000000001,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":0.82099999999999995},{"interface":"front","average":{"1min":1.0429999999999999,"5min":0.97999999999999998,"15min":0.96999999999999997},"min":{"1min":0.68500000000000005,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.244,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.98399999999999999}]},{"osd":4,"last update":"Sat Sep 21 15:00:03 2024","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.93700000000000006,"15min":0.93799999999999994},"min":{"1min":0.64200000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.4510000000000001,"5min":2.7130000000000001,"15min":2.7130000000000001},"last":0.59699999999999998},{"interface":"front","average":{"1min":1.0369999999999999,"5min":0.94099999999999995,"15min":0.92500000000000004},"min":{"1min":0.59999999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.6499999999999999,"5min":2.125,"15min":2.125},"last":0.75700000000000001}]},{"osd":5,"last update":"Sat Sep 21 15:00:03 2024","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.0409999999999999,"15min":1.0269999999999999},"min":{"1min":0.71099999999999997,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.4770000000000001,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":1.079},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.0640000000000001,"15min":1.0580000000000001},"min":{"1min":0.70799999999999996,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.8169999999999999,"5min":3.1880000000000002,"15min":3.1880000000000002},"last":0.84299999999999997}]},{"osd":6,"last update":"Sat Sep 21 15:00:03 2024","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.163,"15min":1.159},"min":{"1min":0.71299999999999997,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.6579999999999999,"5min":3.2730000000000001,"15min":3.2730000000000001},"last":1.036},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.0469999999999999,"15min":1.0249999999999999},"min":{"1min":0.79100000000000004,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.6180000000000001,"5min":2.2879999999999998,"15min":2.2879999999999998},"last":1.052}]}]},{"osd":6,"up_from":14,"seq":60129542184,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63192,"kb_used_data":9928,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94308648,"statfs":{"total":96636764160,"available":96572055552,"internally_reserved":0,"allocated":10166272,"data_stored":71351473,"data_compressed":1591208,"data_compressed_allocated":6516736,"data_compressed_original":69451532,"omap_allocated":1593,"internal_metadata":54524359},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,16],"upper_bound":1024},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:00:06 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.94399999999999995,"15min":0.95699999999999996},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.244,"5min":1.593,"15min":1.593},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.93600000000000005,"15min":0.95099999999999996},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.3360000000000001,"5min":1.587,"15min":1.587},"last":1.089}]},{"osd":1,"last update":"Sat Sep 21 15:00:06 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.90400000000000003,"15min":0.91400000000000003},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.1120000000000001,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.98299999999999998},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.95699999999999996,"15min":0.97999999999999998},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.1819999999999999,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.89100000000000001}]},{"osd":2,"last update":"Sat Sep 21 15:00:06 2024","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.98499999999999999,"15min":0.99099999999999999},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.3560000000000001,"5min":1.758,"15min":1.758},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.89900000000000002,"15min":0.90900000000000003},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.3069999999999999,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.93799999999999994}]},{"osd":3,"last update":"Sat Sep 21 15:00:06 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.94499999999999995,"15min":0.95799999999999996},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.169,"5min":1.835,"15min":1.835},"last":0.64200000000000002},{"interface":"front","average":{"1min":1.012,"5min":1.1180000000000001,"15min":1.1359999999999999},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.4430000000000001,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":1.2210000000000001}]},{"osd":4,"last update":"Sat Sep 21 15:00:06 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.97699999999999998,"15min":1.0049999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.1379999999999999,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.754},{"interface":"front","average":{"1min":0.93999999999999995,"5min":1.02,"15min":1.0329999999999999},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.411,"5min":1.897,"15min":1.897},"last":0.72399999999999998}]},{"osd":5,"last update":"Sat Sep 21 15:00:06 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":1.0860000000000001,"15min":1.1240000000000001},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.2589999999999999,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":1.0209999999999999},{"interface":"front","average":{"1min":1.01,"5min":1.0800000000000001,"15min":1.0920000000000001},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.3779999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":0.82299999999999995}]},{"osd":7,"last update":"Sat Sep 21 15:00:06 2024","interfaces":[{"interface":"back","average":{"1min":1.123,"5min":1.2350000000000001,"15min":1.2529999999999999},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.4530000000000001,"5min":1.8660000000000001,"15min":1.8660000000000001},"last":1.163},{"interface":"front","average":{"1min":1.044,"5min":1.26,"15min":1.296},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.385,"5min":2.004,"15min":2.004},"last":1.1140000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542185,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40508,"kb_used_data":9388,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94331332,"statfs":{"total":96636764160,"available":96595283968,"internally_reserved":0,"allocated":9613312,"data_stored":65131679,"data_compressed":1272182,"data_compressed_allocated":5857280,"data_compressed_original":63146092,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,15],"upper_bound":1024},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:00:01 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.96799999999999997,"15min":1.0109999999999999},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.98999999999999999,"5min":4.4800000000000004,"15min":4.4800000000000004},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.80800000000000005,"15min":0.82099999999999995},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.0329999999999999,"5min":1.181,"15min":1.181},"last":0.57499999999999996}]},{"osd":1,"last update":"Sat Sep 21 15:01:02 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.89600000000000002,"15min":0.92400000000000004},"min":{"1min":0.47699999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":2.7709999999999999,"5min":2.7709999999999999,"15min":2.7709999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.91400000000000003,"15min":0.93400000000000005},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":2.823,"5min":2.823,"15min":2.823},"last":0.84699999999999998}]},{"osd":2,"last update":"Sat Sep 21 15:01:02 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.91200000000000003,"15min":0.97399999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.159,"5min":3.419,"15min":3.419},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.87,"15min":0.89600000000000002},"min":{"1min":0.53400000000000003,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0780000000000001,"5min":1.423,"15min":1.423},"last":0.96299999999999997}]},{"osd":3,"last update":"Sat Sep 21 15:01:02 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.80300000000000005,"15min":0.80700000000000005},"min":{"1min":0.58499999999999996,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":1.258},{"interface":"front","average":{"1min":0.96699999999999997,"5min":1.054,"15min":1.0840000000000001},"min":{"1min":0.59899999999999998,"5min":0.435,"15min":0.435},"max":{"1min":1.272,"5min":2.6379999999999999,"15min":2.6379999999999999},"last":1.1399999999999999}]},{"osd":4,"last update":"Sat Sep 21 15:01:02 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.98899999999999999,"15min":1.016},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.5609999999999999,"5min":2.5470000000000002,"15min":2.5470000000000002},"last":1.038},{"interface":"front","average":{"1min":0.82099999999999995,"5min":1.0169999999999999,"15min":1.0780000000000001},"min":{"1min":0.39300000000000002,"5min":0.376,"15min":0.376},"max":{"1min":1.302,"5min":2.5099999999999998,"15min":2.5099999999999998},"last":1.302}]},{"osd":6,"last update":"Sat Sep 21 15:01:02 2024","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.94299999999999995,"15min":0.96899999999999997},"min":{"1min":0.63,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":1.0840000000000001},{"interface":"front","average":{"1min":1.004,"5min":1.2450000000000001,"15min":1.3520000000000001},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.425,"5min":4.2469999999999999,"15min":4.2469999999999999},"last":1.214}]},{"osd":7,"last update":"Sat Sep 21 15:00:05 2024","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.3200000000000001,"15min":1.363},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":2.7189999999999999,"5min":3.3010000000000002,"15min":3.3010000000000002},"last":1.345},{"interface":"front","average":{"1min":1.0740000000000001,"5min":1.1339999999999999,"15min":1.1439999999999999},"min":{"1min":0.64200000000000002,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":2.5880000000000001,"5min":2.5880000000000001,"15min":2.5880000000000001},"last":1.181}]}]},{"osd":4,"up_from":14,"seq":60129542184,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":67704,"kb_used_data":10340,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94304136,"statfs":{"total":96636764160,"available":96567435264,"internally_reserved":0,"allocated":10588160,"data_stored":79055540,"data_compressed":1750025,"data_compressed_allocated":7159808,"data_compressed_original":77331143,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.76000000000000001,"15min":0.77600000000000002},"min":{"1min":0.47499999999999998,"5min":0.311,"15min":0.311},"max":{"1min":1.1020000000000001,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":1.5860000000000001,"5min":1.0489999999999999,"15min":0.98099999999999998},"min":{"1min":0.49299999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":13.324,"5min":13.324,"15min":13.324},"last":1.1659999999999999}]},{"osd":1,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.85899999999999999,"15min":0.90600000000000003},"min":{"1min":0.46800000000000003,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.0609999999999999,"5min":1.347,"15min":1.347},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.82199999999999995,"15min":0.83899999999999997},"min":{"1min":0.502,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.58499999999999996}]},{"osd":2,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.88200000000000001,"15min":0.88},"min":{"1min":0.55100000000000005,"5min":0.501,"15min":0.501},"max":{"1min":1.47,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.98099999999999998,"5min":1.002,"15min":1.026},"min":{"1min":0.60699999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.3740000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":1.056}]},{"osd":3,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.90700000000000003,"15min":0.89500000000000002},"min":{"1min":0.61799999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.482,"5min":1.538,"15min":1.538},"last":1.121},{"interface":"front","average":{"1min":1.024,"5min":1.0129999999999999,"15min":0.99199999999999999},"min":{"1min":0.56599999999999995,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.528,"5min":1.528,"15min":1.528},"last":1.01}]},{"osd":5,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.97499999999999998,"15min":0.98399999999999999},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.579,"5min":1.579,"15min":1.579},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.94799999999999995,"15min":0.96399999999999997},"min":{"1min":0.61899999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":0.97299999999999998}]},{"osd":6,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.996,"15min":1.0109999999999999},"min":{"1min":0.55400000000000005,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.349,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.86599999999999999},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.081,"15min":1.0900000000000001},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.665,"5min":1.665,"15min":1.665},"last":0.71899999999999997}]},{"osd":7,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.995,"15min":1.0249999999999999},"min":{"1min":0.45200000000000001,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.226,"5min":1.7010000000000001,"15min":1.7010000000000001},"last":0.748},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.2050000000000001,"15min":1.2130000000000001},"min":{"1min":0.627,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.91800000000000004}]}]},{"osd":3,"up_from":14,"seq":60129542185,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58544,"kb_used_data":13472,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94313296,"statfs":{"total":96636764160,"available":96576815104,"internally_reserved":0,"allocated":13795328,"data_stored":103378431,"data_compressed":2311080,"data_compressed_allocated":9502720,"data_compressed_original":101103314,"omap_allocated":1592,"internal_metadata":46135752},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:00:05 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.378,"5min":0.315,"15min":0.315},"max":{"1min":1.0109999999999999,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.71199999999999997,"15min":0.71999999999999997},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.89200000000000002,"5min":1.919,"15min":1.919},"last":0.66100000000000003}]},{"osd":1,"last update":"Sat Sep 21 15:00:05 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.67500000000000004,"15min":0.67600000000000005},"min":{"1min":0.33400000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":1.1240000000000001,"5min":1.135,"15min":1.135},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.80700000000000005,"15min":0.81699999999999995},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.1599999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.753}]},{"osd":2,"last update":"Sat Sep 21 15:00:05 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.72799999999999998,"15min":0.72399999999999998},"min":{"1min":0.40200000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.83399999999999996,"15min":0.83499999999999996},"min":{"1min":0.47499999999999998,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.1759999999999999,"5min":1.254,"15min":1.254},"last":0.82299999999999995}]},{"osd":4,"last update":"Sat Sep 21 15:00:05 2024","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.95699999999999996,"15min":0.96299999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.2709999999999999,"5min":1.7609999999999999,"15min":1.7609999999999999},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.91100000000000003,"5min":1.0189999999999999,"15min":1.0369999999999999},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.25,"5min":2.028,"15min":2.028},"last":0.85799999999999998}]},{"osd":5,"last update":"Sat Sep 21 15:00:05 2024","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.052,"15min":1.0569999999999999},"min":{"1min":0.77700000000000002,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.246,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.022},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.0649999999999999,"15min":1.0660000000000001},"min":{"1min":0.70799999999999996,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.3600000000000001,"5min":1.431,"15min":1.431},"last":0.95699999999999996}]},{"osd":6,"last update":"Sat Sep 21 15:00:05 2024","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.0860000000000001,"15min":1.0980000000000001},"min":{"1min":0.58599999999999997,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.3879999999999999,"5min":2.1219999999999999,"15min":2.1219999999999999},"last":0.97999999999999998},{"interface":"front","average":{"1min":1.046,"5min":1.097,"15min":1.105},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.7370000000000001,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":1.1100000000000001}]},{"osd":7,"last update":"Sat Sep 21 15:00:05 2024","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.1279999999999999,"15min":1.139},"min":{"1min":0.61799999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.4870000000000001,"5min":1.706,"15min":1.706},"last":1.0469999999999999},{"interface":"front","average":{"1min":1.093,"5min":1.173,"15min":1.1859999999999999},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.516,"5min":2.3380000000000001,"15min":2.3380000000000001},"last":1.073}]}]},{"osd":2,"up_from":14,"seq":60129542183,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64844,"kb_used_data":11584,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94306996,"statfs":{"total":96636764160,"available":96570363904,"internally_reserved":0,"allocated":11862016,"data_stored":89535081,"data_compressed":2014763,"data_compressed_allocated":8007680,"data_compressed_original":87553058,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.62},"min":{"1min":0.26900000000000002,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.92800000000000005,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.69399999999999995,"15min":0.67000000000000004},"min":{"1min":0.34799999999999998,"5min":0.223,"15min":0.223},"max":{"1min":2.9169999999999998,"5min":2.9169999999999998,"15min":2.9169999999999998},"last":0.41299999999999998}]},{"osd":1,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.32700000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.0529999999999999,"5min":2.6520000000000001,"15min":2.6520000000000001},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.746,"5min":0.73499999999999999,"15min":0.75800000000000001},"min":{"1min":0.40500000000000003,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.349,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.69699999999999995}]},{"osd":3,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.73999999999999999,"15min":0.73499999999999999},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":3.0110000000000001,"5min":3.0110000000000001,"15min":3.0110000000000001},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.877,"15min":0.83899999999999997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.05,"5min":4.1440000000000001,"15min":4.1440000000000001},"last":0.47599999999999998}]},{"osd":4,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.93200000000000005,"15min":0.95599999999999996},"min":{"1min":0.42399999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.222,"5min":2.899,"15min":2.899},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.94399999999999995,"15min":0.95799999999999996},"min":{"1min":0.47499999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.4379999999999999,"5min":1.675,"15min":1.675},"last":0.73599999999999999}]},{"osd":5,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.0129999999999999,"15min":1.0089999999999999},"min":{"1min":0.45500000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":3.1080000000000001,"5min":3.1080000000000001,"15min":3.1080000000000001},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.98199999999999998,"15min":0.99399999999999999},"min":{"1min":0.51700000000000002,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.268,"5min":1.546,"15min":1.546},"last":0.51700000000000002}]},{"osd":6,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":0.98299999999999998,"15min":0.98499999999999999},"min":{"1min":0.56200000000000006,"5min":0.378,"15min":0.378},"max":{"1min":1.345,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":0.77700000000000002},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.0649999999999999,"15min":1.0549999999999999},"min":{"1min":0.53200000000000003,"5min":0.502,"15min":0.502},"max":{"1min":3.0739999999999998,"5min":3.0739999999999998,"15min":3.0739999999999998},"last":0.83799999999999997}]},{"osd":7,"last update":"Sat Sep 21 15:00:07 2024","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":1.0029999999999999,"15min":0.995},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.88700000000000001},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.097,"15min":1.091},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.5489999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":0.66000000000000003}]}]},{"osd":1,"up_from":14,"seq":60129542184,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64836,"kb_used_data":11572,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94307004,"statfs":{"total":96636764160,"available":96570372096,"internally_reserved":0,"allocated":11849728,"data_stored":91303543,"data_compressed":1932797,"data_compressed_allocated":8060928,"data_compressed_original":89352649,"omap_allocated":1595,"internal_metadata":54524357},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.59599999999999997,"15min":0.60899999999999999},"min":{"1min":0.36399999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.78200000000000003,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59699999999999998,"15min":0.60799999999999998},"min":{"1min":0.39500000000000002,"5min":0.216,"15min":0.216},"max":{"1min":0.80100000000000005,"5min":1.107,"15min":1.107},"last":0.39500000000000002}]},{"osd":2,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":0.79600000000000004,"15min":0.73999999999999999},"min":{"1min":0.38700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":11.303000000000001,"5min":11.303000000000001,"15min":11.303000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.71299999999999997,"15min":0.70299999999999996},"min":{"1min":0.47299999999999998,"5min":0.435,"15min":0.435},"max":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.81599999999999995}]},{"osd":3,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.81799999999999995,"15min":0.83899999999999997},"min":{"1min":0.27800000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.0049999999999999,"5min":3.1469999999999998,"15min":3.1469999999999998},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.86399999999999999,"15min":0.88600000000000001},"min":{"1min":0.52800000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.306,"5min":3.2320000000000002,"15min":3.2320000000000002},"last":0.94799999999999995}]},{"osd":4,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.89000000000000001,"15min":0.91500000000000004},"min":{"1min":0.59599999999999997,"5min":0.309,"15min":0.309},"max":{"1min":1.45,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":1.5600000000000001,"5min":1.069,"15min":0.96599999999999997},"min":{"1min":0.504,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":11.563000000000001,"5min":11.563000000000001,"15min":11.563000000000001},"last":1.0629999999999999}]},{"osd":5,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":1.625,"5min":1.0469999999999999,"15min":0.94899999999999995},"min":{"1min":0.73399999999999999,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":11.534000000000001,"5min":11.534000000000001,"15min":11.534000000000001},"last":1.121},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95799999999999996,"15min":0.96099999999999997},"min":{"1min":0.63,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.256,"5min":1.484,"15min":1.484},"last":1.256}]},{"osd":6,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":1.0109999999999999,"15min":1.0489999999999999},"min":{"1min":0.60399999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.4059999999999999,"5min":2.3519999999999999,"15min":2.3519999999999999},"last":0.85099999999999998},{"interface":"front","average":{"1min":1.631,"5min":1.081,"15min":0.99299999999999999},"min":{"1min":0.57399999999999995,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":11.601000000000001,"5min":11.601000000000001,"15min":11.601000000000001},"last":1.1639999999999999}]},{"osd":7,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":1.046,"15min":1.0640000000000001},"min":{"1min":0.79300000000000004,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.5820000000000001,"5min":2.2360000000000002,"15min":2.2360000000000002},"last":0.79300000000000004},{"interface":"front","average":{"1min":1.76,"5min":1.171,"15min":1.087},"min":{"1min":0.877,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":11.489000000000001,"5min":11.489000000000001,"15min":11.489000000000001},"last":1.23}]}]},{"osd":0,"up_from":14,"seq":60129542183,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70176,"kb_used_data":12812,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94301664,"statfs":{"total":96636764160,"available":96564903936,"internally_reserved":0,"allocated":13119488,"data_stored":98239443,"data_compressed":2190723,"data_compressed_allocated":9064448,"data_compressed_original":96141885,"omap_allocated":1593,"internal_metadata":58718663},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 21 15:01:04 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.67500000000000004,"15min":0.71699999999999997},"min":{"1min":0.40999999999999998,"5min":0.309,"15min":0.309},"max":{"1min":1.1299999999999999,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.68100000000000005,"15min":0.627},"min":{"1min":0.58999999999999997,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.67800000000000005}]},{"osd":2,"last update":"Sat Sep 21 15:01:04 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.65200000000000002,"15min":0.61799999999999999},"min":{"1min":0.59599999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.72499999999999998,"15min":0.75800000000000001},"min":{"1min":0.39300000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.79200000000000004}]},{"osd":3,"last update":"Sat Sep 21 15:01:04 2024","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.76600000000000001,"15min":0.754},"min":{"1min":0.50900000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.78700000000000003,"15min":0.81499999999999995},"min":{"1min":0.50900000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.50900000000000001}]},{"osd":4,"last update":"Sat Sep 21 15:01:04 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.98399999999999999,"15min":1.038},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.1140000000000001,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":1.036,"5min":1.0469999999999999,"15min":1.0620000000000001},"min":{"1min":0.67800000000000005,"5min":0.379,"15min":0.379},"max":{"1min":1.446,"5min":1.665,"15min":1.665},"last":0.71199999999999997}]},{"osd":5,"last update":"Sat Sep 21 15:01:04 2024","interfaces":[{"interface":"back","average":{"1min":1.0960000000000001,"5min":1.024,"15min":1.032},"min":{"1min":0.79300000000000004,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.4450000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.83899999999999997},{"interface":"front","average":{"1min":1.0589999999999999,"5min":1.0429999999999999,"15min":1.077},"min":{"1min":0.82399999999999995,"5min":0.435,"15min":0.435},"max":{"1min":1.2609999999999999,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":0.95499999999999996}]},{"osd":6,"last update":"Sat Sep 21 15:01:04 2024","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":1.0580000000000001,"15min":1.1120000000000001},"min":{"1min":0.68400000000000005,"5min":0.628,"15min":0.628},"max":{"1min":1.167,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":0.85799999999999998},{"interface":"front","average":{"1min":1.109,"5min":1.0629999999999999,"15min":1.0680000000000001},"min":{"1min":0.91000000000000003,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.5029999999999999,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.91900000000000004}]},{"osd":7,"last update":"Sat Sep 21 15:01:04 2024","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.0469999999999999,"15min":1.0620000000000001},"min":{"1min":0.78700000000000003,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.4770000000000001,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":0.88500000000000001},{"interface":"front","average":{"1min":1.02,"5min":1.085,"15min":1.131},"min":{"1min":0.84799999999999998,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.403,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":0.93600000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5202,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5202,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":11001856,"data_stored":97211967,"data_compressed":2190723,"data_compressed_allocated":9064448,"data_compressed_original":96141885,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9662464,"data_stored":89679641,"data_compressed":1927595,"data_compressed_allocated":8024064,"data_compressed_original":88779209,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":9732096,"data_stored":88501547,"data_compressed":2014763,"data_compressed_allocated":8007680,"data_compressed_original":87553058,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":11673600,"data_stored":102350936,"data_compressed":2311080,"data_compressed_allocated":9502720,"data_compressed_original":101103314,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8458240,"data_stored":78022006,"data_compressed":1750025,"data_compressed_allocated":7159808,"data_compressed_original":77331143,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":7426048,"data_stored":63507777,"data_compressed":1266980,"data_compressed_allocated":5820416,"data_compressed_original":62572652,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8044544,"data_stored":70323978,"data_compressed":1591208,"data_compressed_allocated":6516736,"data_compressed_original":69451532,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":5652480,"data_stored":47867620,"data_compressed":1141276,"data_compressed_allocated":4558848,"data_compressed_original":47268443,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T15:01:06.949 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-21T15:01:06.950 INFO:teuthology.orchestra.run.smithi032.stdout:5582 read 0x235b15a thru 0x2364e27 (0x9cce bytes) 2024-09-21T15:01:06.950 INFO:teuthology.orchestra.run.smithi032.stdout:5583 punch from 0x378eec0 to 0x379993d, (0xaa7d bytes) 2024-09-21T15:01:06.950 INFO:teuthology.orchestra.run.smithi032.stdout:5584 write 0x6343297 thru 0x634dcfa (0xaa64 bytes) 2024-09-21T15:01:06.950 INFO:teuthology.orchestra.run.smithi032.stdout:5585 write 0x6b8830e thru 0x6b93f03 (0xbbf6 bytes) 2024-09-21T15:01:06.950 INFO:teuthology.orchestra.run.smithi032.stdout:5586 write 0x432669c thru 0x432b72a (0x508f bytes) 2024-09-21T15:01:06.950 INFO:teuthology.orchestra.run.smithi032.stdout:5588 read 0x4e7f616 thru 0x4e86c08 (0x75f3 bytes) 2024-09-21T15:01:06.950 INFO:teuthology.orchestra.run.smithi032.stdout:5589 trunc from 0x952cfc2 to 0x58c045f 2024-09-21T15:01:06.951 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:01:06.938+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:01:06.969 INFO:teuthology.orchestra.run.smithi032.stdout:5591 read 0x24555b thru 0x24cd9d (0x7843 bytes) 2024-09-21T15:01:06.981 INFO:teuthology.orchestra.run.smithi032.stdout:5593 trunc from 0x58c045f to 0xac29e38 2024-09-21T15:01:07.001 INFO:teuthology.orchestra.run.smithi032.stdout:5596 write 0xc3e9047 thru 0xc3f4053 (0xb00d bytes) 2024-09-21T15:01:07.015 INFO:teuthology.orchestra.run.smithi032.stdout:5599 trunc from 0xc3f4054 to 0x9ff19eb 2024-09-21T15:01:07.028 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:01:07.037 INFO:teuthology.orchestra.run.smithi032.stdout:5601 trunc from 0x9ff19eb to 0x444a5af 2024-09-21T15:01:07.045 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:01:07.042+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:01:07.060 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:01:07.060 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:01:07.060 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T15:01:07.060 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:01:07.063 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:01:07.063 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:01:07.071 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T15:01:07.076 INFO:teuthology.orchestra.run.smithi032.stdout:5607 write 0x7f42978 thru 0x7f46204 (0x388d bytes) 2024-09-21T15:01:07.082 INFO:teuthology.orchestra.run.smithi032.stdout:5608 trunc from 0x7f46205 to 0x796ae9c 2024-09-21T15:01:07.096 INFO:teuthology.orchestra.run.smithi032.stdout:5609 write 0x941cd thru 0x9bd23 (0x7b57 bytes) 2024-09-21T15:01:07.106 INFO:teuthology.orchestra.run.smithi032.stdout:5610 trunc from 0x796ae9c to 0x3d9ab2f 2024-09-21T15:01:07.132 INFO:teuthology.orchestra.run.smithi032.stdout:5613 write 0x1110c86 thru 0x111f8a4 (0xec1f bytes) 2024-09-21T15:01:07.141 INFO:teuthology.orchestra.run.smithi032.stdout:5614 trunc from 0x3d9ab2f to 0x3942a 2024-09-21T15:01:07.147 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:01:07.141+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:01:07.181 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:01:07.181 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:01:07.181 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T15:01:07.181 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T15:01:07.181 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:01:07.191 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T15:01:07.191 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T15:01:07.197 INFO:teuthology.orchestra.run.smithi032.stdout:5615 write 0xd357cb6 thru 0xd358f6d (0x12b8 bytes) 2024-09-21T15:01:07.211 INFO:teuthology.orchestra.run.smithi032.stdout:5616 trunc from 0xd358f6e to 0x93a64ab 2024-09-21T15:01:07.223 INFO:teuthology.orchestra.run.smithi032.stdout:5617 punch from 0x3151bc6 to 0x315591b, (0x3d55 bytes) 2024-09-21T15:01:07.227 INFO:teuthology.orchestra.run.smithi032.stdout:5620 trunc from 0x93a64ab to 0xb46d7e5 2024-09-21T15:01:07.238 INFO:teuthology.orchestra.run.smithi032.stdout:5621 write 0x6ec75a8 thru 0x6ecd6bd (0x6116 bytes) 2024-09-21T15:01:07.242 INFO:teuthology.orchestra.run.smithi032.stdout:5622 punch from 0x2d0b356 to 0x2d0b573, (0x21d bytes) 2024-09-21T15:01:07.242 INFO:teuthology.orchestra.run.smithi032.stdout:5624 read 0x837bd92 thru 0x8381e4d (0x60bc bytes) 2024-09-21T15:01:07.242 INFO:teuthology.orchestra.run.smithi032.stdout:5626 read 0x6603925 thru 0x6610776 (0xce52 bytes) 2024-09-21T15:01:07.243 INFO:teuthology.orchestra.run.smithi032.stdout:5628 write 0xaa7f32d thru 0xaa8186f (0x2543 bytes) 2024-09-21T15:01:07.247 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:01:07.245+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:01:07.251 INFO:teuthology.orchestra.run.smithi032.stdout:5630 read 0x73675bd thru 0x736b55a (0x3f9e bytes) 2024-09-21T15:01:07.252 INFO:teuthology.orchestra.run.smithi032.stdout:5631 punch from 0x405f686 to 0x406a99b, (0xb315 bytes) 2024-09-21T15:01:07.265 INFO:teuthology.orchestra.run.smithi032.stdout:5632 writesame 0xce346b4 thru 0xce38d7c data_size 0x1(0x46c9 bytes) 2024-09-21T15:01:07.265 INFO:teuthology.orchestra.run.smithi032.stdout:5635 read 0xa360a82 thru 0xa36cc14 (0xc193 bytes) 2024-09-21T15:01:07.265 INFO:teuthology.orchestra.run.smithi032.stdout:5637 punch from 0x3364220 to 0x336cfc9, (0x8da9 bytes) 2024-09-21T15:01:07.265 INFO:teuthology.orchestra.run.smithi032.stdout:5638 write 0x33856a9 thru 0x3385b45 (0x49d bytes) 2024-09-21T15:01:07.271 INFO:teuthology.orchestra.run.smithi032.stdout:5639 punch from 0x42856e5 to 0x428fe3d, (0xa758 bytes) 2024-09-21T15:01:07.271 INFO:teuthology.orchestra.run.smithi032.stdout:5641 read 0x381aa04 thru 0x381f421 (0x4a1e bytes) 2024-09-21T15:01:07.286 INFO:teuthology.orchestra.run.smithi032.stdout:5642 trunc from 0xce38d7d to 0x4278a24 2024-09-21T15:01:07.308 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:01:07.308 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T15:01:07.308 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T15:01:07.308 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T15:01:07.308 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.308 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:63820 3.19 3.626bc079 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-21T15:01:07.308 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:27.241293+0000", 2024-09-21T15:01:07.308 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 100.06370572100001, 2024-09-21T15:01:07.308 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.98723808199999996, 2024-09-21T15:01:07.309 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:07.309 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:07.309 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:07.309 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:07.309 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:07.309 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 63820 2024-09-21T15:01:07.309 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.309 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:07.309 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.309 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:07.309 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241293+0000", 2024-09-21T15:01:07.309 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:07.309 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.309 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.309 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:07.310 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241293+0000", 2024-09-21T15:01:07.310 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:07.310 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.310 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.310 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:07.310 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241291+0000", 2024-09-21T15:01:07.310 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:01:07.310 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.310 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.310 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:07.310 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241299+0000", 2024-09-21T15:01:07.310 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1529999999999997e-06 2024-09-21T15:01:07.311 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.311 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.311 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:07.311 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241317+0000", 2024-09-21T15:01:07.311 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7764000000000001e-05 2024-09-21T15:01:07.313 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.313 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.313 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:07.313 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241328+0000", 2024-09-21T15:01:07.313 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1245e-05 2024-09-21T15:01:07.313 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.313 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.314 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:07.314 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241407+0000", 2024-09-21T15:01:07.314 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8762000000000002e-05 2024-09-21T15:01:07.314 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.314 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.314 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:07.314 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241418+0000", 2024-09-21T15:01:07.314 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1695000000000001e-05 2024-09-21T15:01:07.314 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.314 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.314 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:07.314 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:28.228510+0000", 2024-09-21T15:01:07.314 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.98709166100000001 2024-09-21T15:01:07.314 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.314 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.315 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:07.315 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:28.228531+0000", 2024-09-21T15:01:07.315 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0696e-05 2024-09-21T15:01:07.315 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:07.315 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:07.315 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:07.315 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.315 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.315 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:97734 3.19 3:9fcf754f:::rbd_data.1136bfc79a70.0000000000000009:head [write 0~4194304 [fadvise_sequential+fadvise_nocache],write 4784128~147456 [fadvise_sequential+fadvise_nocache],write 4997120~147456 [fadvise_sequential+fadvise_nocache],write 5210112~32768 [fadvise_sequential+fadvise_nocache],write 5292032~147456 [fadvise_sequential+fadvise_nocache],write 5505024~147456 [fadvise_sequential+fadvise_nocache],write 5718016~49152 [fadvise_sequential+fadvise_nocache],write 5799936~147456 [fadvise_sequential+fadvise_nocache],write 6012928~147456 [fadvise_sequential+fadvise_nocache],write 6225920~65536 [fadvise_sequential+fadvise_nocache],write 6307840~147456 [fadvise_sequential+fadvise_nocache],write 6520832~147456 [fadvise_sequential+fadvise_nocache],write 6733824~229376 [fadvise_sequential+fadvise_nocache],write 7028736~147456 [fadvise_sequential+fadvise_nocache],write 7241728~229376 [fadvise_sequential+fadvise_nocache],write 7536640~147456 [fadvise_sequential+fadvise_nocache],write 7749632~229376 [fadvise_sequential+fadvise_nocache],write 8077312~114688 [fadvise_sequential+fadvise_nocache],write 8290304~98304 [fadvise_sequential+fadvise_nocache],write 8404992~81920 [fadvise_sequential+fadvise_nocache],write 8617984~81920 [fadvise_sequential+fadvise_nocache],write 8830976~81920 [fadvise_sequential+fadvise_nocache],write 8962048~32768 [fadvise_sequential+fadvise_nocache],write 9175040~32768 [fadvise_sequential+fadvise_nocache],write 9388032~32768 [fadvise_sequential+fadvise_nocache],write 9469952~32768 [fadvise_sequential+fadvise_nocache],write 9682944~20092 [fadvise_sequential+fadvise_nocache],write 9961472~163840 [fadvise_sequential+fadvise_nocache],write 10141696~196608 [fadvise_sequential+fadvise_nocache],write 10354688~278528 [fadvise_sequential+fadvise_nocache],write 10649600~196608 [fadvise_sequential+fadvise_nocache],write 10862592~196608 [fadvise_sequential+fadvise_nocache],write 11075584~65536 [fadvise_sequential+fadvise_nocache],write 11173888~98304 [fadvise_sequential+fadvise_nocache],write 11288576~65536 [fadvise_sequential+fadvise_nocache],write 11386880~98304 [fadvise_sequential+fadvise_nocache],write 11501568~65536 [fadvise_sequential+fadvise_nocache],write 11583488~65536 [fadvise_sequential+fadvise_nocache],write 11681792~98304 [fadvise_sequential+fadvise_nocache],write 11796480~65536 [fadvise_sequential+fadvise_nocache],write 11894784~98304 [fadvise_sequential+fadvise_nocache],write 12009472~65536 [fadvise_sequential+fadvise_nocache],write 12189696~98304 [fadvise_sequential+fadvise_nocache],write 12402688~98304 [fadvise_sequential+fadvise_nocache],write 12697600~98304 [fadvise_sequential+fadvise_nocache],write 12910592~98304 [fadvise_sequential+fadvise_nocache],write 13205504~98304 [fadvise_sequential+fadvise_nocache],write 13418496~98304 [fadvise_sequential+fadvise_nocache],write 13713408~98304 [fadvise_sequential+fadvise_nocache],write 13926400~98304 [fadvise_sequential+fadvise_nocache],write 14139392~16384 [fadvise_sequential+fadvise_nocache],write 14680064~16384 [fadvise_sequential+fadvise_nocache],write 14827520~81920 [fadvise_sequential+fadvise_nocache],write 15040512~81920 [fadvise_sequential+fadvise_nocache],write 15335424~81920 [fadvise_sequential+fadvise_nocache],write 15548416~81920 [fadvise_sequential+fadvise_nocache],write 15843328~81920 [fadvise_sequential+fadvise_nocache],write 16056320~81920 [fadvise_sequential+fadvise_nocache],write 16351232~81920 [fadvise_sequential+fadvise_nocache],write 16564224~81920 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:01:07.315 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.354331+0000", 2024-09-21T15:01:07.315 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 61.950667674000002, 2024-09-21T15:01:07.315 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.086284870999999999, 2024-09-21T15:01:07.315 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:07.315 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:07.316 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:07.316 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:07.316 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:07.316 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97734 2024-09-21T15:01:07.316 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.316 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:07.316 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.316 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:07.316 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.354331+0000", 2024-09-21T15:01:07.316 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:07.316 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.316 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.316 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:07.316 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.354331+0000", 2024-09-21T15:01:07.316 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:07.316 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.317 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.317 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:07.317 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.354329+0000", 2024-09-21T15:01:07.317 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:01:07.317 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.317 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.317 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:07.317 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.370234+0000", 2024-09-21T15:01:07.317 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015905348999999999 2024-09-21T15:01:07.317 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.317 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.317 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:07.317 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.370286+0000", 2024-09-21T15:01:07.317 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1656999999999997e-05 2024-09-21T15:01:07.317 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.317 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.318 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:07.318 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.370315+0000", 2024-09-21T15:01:07.318 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9204e-05 2024-09-21T15:01:07.318 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.318 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.318 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:07.318 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.370395+0000", 2024-09-21T15:01:07.318 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0395999999999994e-05 2024-09-21T15:01:07.318 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.318 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.318 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:07.318 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.370604+0000", 2024-09-21T15:01:07.318 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000208684 2024-09-21T15:01:07.318 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.319 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.319 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 1", 2024-09-21T15:01:07.319 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.371701+0000", 2024-09-21T15:01:07.319 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0010973140000000001 2024-09-21T15:01:07.319 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.319 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.319 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:01:07.319 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.405485+0000", 2024-09-21T15:01:07.319 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.033783615000000003 2024-09-21T15:01:07.319 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.319 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.319 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:01:07.319 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.440211+0000", 2024-09-21T15:01:07.319 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034725757000000003 2024-09-21T15:01:07.319 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.319 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.320 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:01:07.320 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.440497+0000", 2024-09-21T15:01:07.320 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00028673099999999998 2024-09-21T15:01:07.320 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.320 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.320 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:07.320 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.440616+0000", 2024-09-21T15:01:07.320 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000118256 2024-09-21T15:01:07.320 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:07.320 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:07.320 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:07.320 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.320 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.320 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112831 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 1515520~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:07.321 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.406761+0000", 2024-09-21T15:01:07.321 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 37.898236976, 2024-09-21T15:01:07.321 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060265666000000002, 2024-09-21T15:01:07.321 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:07.321 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:07.321 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:07.321 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:07.321 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:07.321 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112831 2024-09-21T15:01:07.321 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.321 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:07.321 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.321 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:07.321 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406761+0000", 2024-09-21T15:01:07.321 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:07.322 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.322 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.322 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:07.322 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406761+0000", 2024-09-21T15:01:07.322 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:07.322 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.322 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.322 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:07.322 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406761+0000", 2024-09-21T15:01:07.322 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:07.322 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.322 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.322 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:07.322 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406764+0000", 2024-09-21T15:01:07.322 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0319999999999999e-06 2024-09-21T15:01:07.322 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.323 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.323 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:07.323 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406776+0000", 2024-09-21T15:01:07.323 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2357999999999999e-05 2024-09-21T15:01:07.323 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.323 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.323 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:07.323 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406784+0000", 2024-09-21T15:01:07.323 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6059999999999997e-06 2024-09-21T15:01:07.323 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.323 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.323 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:07.323 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.466595+0000", 2024-09-21T15:01:07.323 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059811062999999998 2024-09-21T15:01:07.323 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.324 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.324 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:07.324 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.466782+0000", 2024-09-21T15:01:07.324 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000186622 2024-09-21T15:01:07.324 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.324 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.324 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:07.324 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.467027+0000", 2024-09-21T15:01:07.324 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000245455 2024-09-21T15:01:07.324 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:07.324 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:07.324 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:07.324 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.324 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.324 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112832 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 1605632~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:07.325 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.406788+0000", 2024-09-21T15:01:07.325 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 37.898210386000002, 2024-09-21T15:01:07.325 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060452065999999999, 2024-09-21T15:01:07.325 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:07.325 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:07.325 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:07.325 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:07.325 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:07.325 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112832 2024-09-21T15:01:07.325 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.325 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:07.325 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.325 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:07.325 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406788+0000", 2024-09-21T15:01:07.326 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:07.326 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.326 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.326 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:07.326 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406788+0000", 2024-09-21T15:01:07.326 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:07.326 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.326 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.326 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:07.326 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406788+0000", 2024-09-21T15:01:07.326 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:07.326 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.326 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.326 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:07.327 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406798+0000", 2024-09-21T15:01:07.327 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0096e-05 2024-09-21T15:01:07.327 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.327 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.327 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:07.327 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406803+0000", 2024-09-21T15:01:07.327 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2020000000000003e-06 2024-09-21T15:01:07.327 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.327 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.327 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:07.327 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406808+0000", 2024-09-21T15:01:07.327 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8239999999999999e-06 2024-09-21T15:01:07.327 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.327 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.328 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:07.328 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.467049+0000", 2024-09-21T15:01:07.328 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060241754000000002 2024-09-21T15:01:07.328 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.328 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.328 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:07.328 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.467134+0000", 2024-09-21T15:01:07.328 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4216999999999997e-05 2024-09-21T15:01:07.328 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.328 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.328 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:07.328 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.467240+0000", 2024-09-21T15:01:07.328 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000106304 2024-09-21T15:01:07.328 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:07.329 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:07.329 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:07.329 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.329 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.329 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112843 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 2613248~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:07.329 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.410273+0000", 2024-09-21T15:01:07.329 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 37.894725792000003, 2024-09-21T15:01:07.329 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059854931, 2024-09-21T15:01:07.329 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:07.329 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:07.329 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:07.329 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:07.329 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:07.329 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112843 2024-09-21T15:01:07.329 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.330 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:07.330 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.330 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:07.330 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410273+0000", 2024-09-21T15:01:07.330 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:07.330 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.330 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.330 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:07.330 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410273+0000", 2024-09-21T15:01:07.330 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:07.330 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.330 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.330 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:07.330 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410272+0000", 2024-09-21T15:01:07.331 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:07.331 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.331 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.331 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:07.331 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410274+0000", 2024-09-21T15:01:07.331 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.905e-06 2024-09-21T15:01:07.331 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.331 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.331 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:07.331 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410283+0000", 2024-09-21T15:01:07.331 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3440000000000007e-06 2024-09-21T15:01:07.331 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.332 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.332 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:07.332 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410294+0000", 2024-09-21T15:01:07.332 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0271e-05 2024-09-21T15:01:07.332 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.332 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.332 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:07.332 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.469917+0000", 2024-09-21T15:01:07.332 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059623738000000003 2024-09-21T15:01:07.332 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.332 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.332 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:07.332 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470002+0000", 2024-09-21T15:01:07.332 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4270999999999993e-05 2024-09-21T15:01:07.333 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.333 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.333 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:07.333 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470127+0000", 2024-09-21T15:01:07.333 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000125711 2024-09-21T15:01:07.333 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:07.333 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:07.333 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:07.333 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.333 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.333 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112844 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 2703360~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:07.333 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.410326+0000", 2024-09-21T15:01:07.333 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 37.894672071999999, 2024-09-21T15:01:07.333 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060086056999999998, 2024-09-21T15:01:07.333 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:07.334 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:07.334 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:07.334 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:07.334 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:07.334 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112844 2024-09-21T15:01:07.334 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.334 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:07.334 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.334 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:07.334 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410326+0000", 2024-09-21T15:01:07.334 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:07.334 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.335 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.335 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:07.335 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410326+0000", 2024-09-21T15:01:07.335 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:07.335 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.335 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.335 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:07.335 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410326+0000", 2024-09-21T15:01:07.335 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:07.335 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.335 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.335 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:07.335 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410352+0000", 2024-09-21T15:01:07.335 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6872999999999998e-05 2024-09-21T15:01:07.335 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.335 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.336 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:07.336 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410374+0000", 2024-09-21T15:01:07.336 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1141e-05 2024-09-21T15:01:07.336 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.336 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.336 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:07.336 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410385+0000", 2024-09-21T15:01:07.336 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1456e-05 2024-09-21T15:01:07.336 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.336 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.336 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:07.336 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470174+0000", 2024-09-21T15:01:07.336 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059788574999999997 2024-09-21T15:01:07.336 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.336 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.336 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:07.336 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470277+0000", 2024-09-21T15:01:07.337 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000103838 2024-09-21T15:01:07.337 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.337 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.337 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:07.337 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470412+0000", 2024-09-21T15:01:07.337 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013489700000000001 2024-09-21T15:01:07.337 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:07.337 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:07.337 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:07.337 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.337 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.337 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112845 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 2793472~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:07.337 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.410394+0000", 2024-09-21T15:01:07.337 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 37.894604598000001, 2024-09-21T15:01:07.337 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060244173999999998, 2024-09-21T15:01:07.337 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:07.338 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:07.338 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:07.338 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:07.338 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:07.338 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112845 2024-09-21T15:01:07.338 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.338 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:07.338 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.338 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:07.338 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410394+0000", 2024-09-21T15:01:07.338 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:07.338 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.338 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.338 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:07.338 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410394+0000", 2024-09-21T15:01:07.338 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:07.339 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.339 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.339 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:07.339 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410393+0000", 2024-09-21T15:01:07.339 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:07.339 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.339 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.339 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:07.339 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410397+0000", 2024-09-21T15:01:07.339 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9430000000000002e-06 2024-09-21T15:01:07.339 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.339 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.339 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:07.339 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410406+0000", 2024-09-21T15:01:07.339 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.799e-06 2024-09-21T15:01:07.339 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.340 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.340 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:07.340 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410418+0000", 2024-09-21T15:01:07.340 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2225999999999999e-05 2024-09-21T15:01:07.340 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.340 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.340 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:07.340 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470444+0000", 2024-09-21T15:01:07.340 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060026311999999998 2024-09-21T15:01:07.340 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.340 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.340 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:07.340 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470535+0000", 2024-09-21T15:01:07.340 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0173999999999994e-05 2024-09-21T15:01:07.340 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.340 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.341 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:07.341 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470638+0000", 2024-09-21T15:01:07.341 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010333 2024-09-21T15:01:07.341 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:07.341 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:07.341 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:07.341 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.341 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.341 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112846 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 2883584~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:07.341 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.410425+0000", 2024-09-21T15:01:07.341 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 37.894573385000001, 2024-09-21T15:01:07.341 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060459247000000001, 2024-09-21T15:01:07.341 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:07.341 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:07.341 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:07.342 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:07.342 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:07.342 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112846 2024-09-21T15:01:07.342 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.342 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:07.342 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.342 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:07.342 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410425+0000", 2024-09-21T15:01:07.342 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:07.342 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.342 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.342 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:07.342 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410425+0000", 2024-09-21T15:01:07.342 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:07.343 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.343 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.343 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:07.343 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410424+0000", 2024-09-21T15:01:07.343 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:07.343 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.343 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.343 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:07.343 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410428+0000", 2024-09-21T15:01:07.343 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4709999999999999e-06 2024-09-21T15:01:07.343 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.343 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.343 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:07.343 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410446+0000", 2024-09-21T15:01:07.344 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8354000000000001e-05 2024-09-21T15:01:07.344 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.344 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.344 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:07.344 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410456+0000", 2024-09-21T15:01:07.344 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6709999999999994e-06 2024-09-21T15:01:07.344 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.344 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.344 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:07.344 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470661+0000", 2024-09-21T15:01:07.344 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060205303000000002 2024-09-21T15:01:07.344 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.344 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.344 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:07.345 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470756+0000", 2024-09-21T15:01:07.345 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4401999999999995e-05 2024-09-21T15:01:07.345 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.345 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.345 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:07.345 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470884+0000", 2024-09-21T15:01:07.345 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012860500000000001 2024-09-21T15:01:07.345 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:07.345 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:07.345 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:07.345 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.345 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.345 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112847 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 2973696~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:07.345 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.410462+0000", 2024-09-21T15:01:07.345 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 37.894536340999998, 2024-09-21T15:01:07.345 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060653836000000003, 2024-09-21T15:01:07.346 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:07.346 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:07.346 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:07.346 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:07.346 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:07.346 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112847 2024-09-21T15:01:07.346 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.346 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:07.346 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.346 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:07.346 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410462+0000", 2024-09-21T15:01:07.346 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:07.346 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.346 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.347 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:07.347 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410462+0000", 2024-09-21T15:01:07.347 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:07.347 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.347 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.347 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:07.347 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410461+0000", 2024-09-21T15:01:07.347 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:07.347 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.347 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.347 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:07.347 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410465+0000", 2024-09-21T15:01:07.347 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6899999999999998e-06 2024-09-21T15:01:07.347 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.347 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.348 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:07.348 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410474+0000", 2024-09-21T15:01:07.348 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4179999999999997e-06 2024-09-21T15:01:07.348 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.348 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.348 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:07.348 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410508+0000", 2024-09-21T15:01:07.348 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4282999999999997e-05 2024-09-21T15:01:07.348 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.348 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.348 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:07.348 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470922+0000", 2024-09-21T15:01:07.348 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060413820999999999 2024-09-21T15:01:07.348 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.348 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.349 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:07.349 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471008+0000", 2024-09-21T15:01:07.349 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6379999999999996e-05 2024-09-21T15:01:07.349 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.349 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.349 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:07.349 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471116+0000", 2024-09-21T15:01:07.349 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000107806 2024-09-21T15:01:07.349 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:07.349 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:07.349 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:07.349 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.349 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.349 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112848 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 3063808~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:07.349 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.410525+0000", 2024-09-21T15:01:07.350 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 37.894473058000003, 2024-09-21T15:01:07.350 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060880735999999998, 2024-09-21T15:01:07.350 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:07.350 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:07.350 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:07.350 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:07.350 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:07.350 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112848 2024-09-21T15:01:07.350 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.350 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:07.350 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.350 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:07.350 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410525+0000", 2024-09-21T15:01:07.350 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:07.350 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.351 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.351 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:07.351 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410525+0000", 2024-09-21T15:01:07.351 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:07.351 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.351 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.351 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:07.351 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410525+0000", 2024-09-21T15:01:07.351 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:07.351 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.351 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.351 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:07.351 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410529+0000", 2024-09-21T15:01:07.351 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1760000000000003e-06 2024-09-21T15:01:07.352 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.352 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.352 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:07.352 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410551+0000", 2024-09-21T15:01:07.352 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2099999999999998e-05 2024-09-21T15:01:07.352 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.352 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.352 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:07.352 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410571+0000", 2024-09-21T15:01:07.352 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0159000000000001e-05 2024-09-21T15:01:07.352 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.352 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.352 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:07.352 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471164+0000", 2024-09-21T15:01:07.353 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.06059287 2024-09-21T15:01:07.353 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.353 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.353 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:07.353 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471271+0000", 2024-09-21T15:01:07.353 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000106635 2024-09-21T15:01:07.353 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.353 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.353 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:07.353 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471406+0000", 2024-09-21T15:01:07.353 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000135399 2024-09-21T15:01:07.353 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:07.353 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:07.353 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:07.353 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.353 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.354 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112849 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 3153920~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:07.354 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.410579+0000", 2024-09-21T15:01:07.354 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 37.894419763000002, 2024-09-21T15:01:07.354 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.061057304999999999, 2024-09-21T15:01:07.354 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:07.354 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:07.354 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:07.354 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:07.354 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:07.354 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112849 2024-09-21T15:01:07.354 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.354 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:07.354 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.355 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:07.355 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410579+0000", 2024-09-21T15:01:07.355 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:07.355 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.355 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.355 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:07.355 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410579+0000", 2024-09-21T15:01:07.355 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:07.355 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.355 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.355 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:07.355 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410578+0000", 2024-09-21T15:01:07.355 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:07.355 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.355 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.356 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:07.356 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410582+0000", 2024-09-21T15:01:07.356 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.036e-06 2024-09-21T15:01:07.356 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.356 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.356 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:07.356 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410592+0000", 2024-09-21T15:01:07.356 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9850000000000001e-06 2024-09-21T15:01:07.356 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.356 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.356 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:07.356 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410604+0000", 2024-09-21T15:01:07.356 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2051e-05 2024-09-21T15:01:07.356 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.356 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.357 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:07.357 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471444+0000", 2024-09-21T15:01:07.357 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060839722999999998 2024-09-21T15:01:07.357 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.357 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.357 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:07.357 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471530+0000", 2024-09-21T15:01:07.357 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6359000000000004e-05 2024-09-21T15:01:07.357 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.357 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.357 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:07.357 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471636+0000", 2024-09-21T15:01:07.357 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010576700000000001 2024-09-21T15:01:07.357 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:07.358 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:07.358 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:07.358 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.358 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.358 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112850 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 3244032~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:07.358 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.410698+0000", 2024-09-21T15:01:07.358 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 37.894300420999997, 2024-09-21T15:01:07.358 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.061164118000000003, 2024-09-21T15:01:07.358 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:07.358 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:07.358 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:07.358 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:07.358 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:07.358 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112850 2024-09-21T15:01:07.358 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.359 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:07.359 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.359 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:07.359 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410698+0000", 2024-09-21T15:01:07.359 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:07.359 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.359 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.359 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:07.359 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410698+0000", 2024-09-21T15:01:07.359 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:07.359 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.359 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.359 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:07.359 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410610+0000", 2024-09-21T15:01:07.359 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999123 2024-09-21T15:01:07.360 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.360 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.360 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:07.360 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410707+0000", 2024-09-21T15:01:07.360 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7093e-05 2024-09-21T15:01:07.360 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.360 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.360 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:07.360 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410728+0000", 2024-09-21T15:01:07.360 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0275000000000002e-05 2024-09-21T15:01:07.360 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.360 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.360 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:07.360 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410740+0000", 2024-09-21T15:01:07.360 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2561999999999999e-05 2024-09-21T15:01:07.360 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.361 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.361 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:07.361 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471664+0000", 2024-09-21T15:01:07.361 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060923979000000003 2024-09-21T15:01:07.361 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.361 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.361 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:07.361 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471753+0000", 2024-09-21T15:01:07.361 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8265999999999998e-05 2024-09-21T15:01:07.361 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.361 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.361 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:07.361 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471862+0000", 2024-09-21T15:01:07.361 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000109474 2024-09-21T15:01:07.361 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:07.362 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:07.362 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:07.362 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.362 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.362 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112864 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 4521984~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:07.362 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.414789+0000", 2024-09-21T15:01:07.362 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 37.890209396000003, 2024-09-21T15:01:07.362 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059845903999999998, 2024-09-21T15:01:07.362 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:07.362 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:07.362 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:07.362 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:07.362 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:07.362 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112864 2024-09-21T15:01:07.362 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.363 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:07.363 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.363 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:07.363 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414789+0000", 2024-09-21T15:01:07.363 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:07.363 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.363 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.363 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:07.363 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414789+0000", 2024-09-21T15:01:07.363 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:07.363 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.363 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.363 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:07.363 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414788+0000", 2024-09-21T15:01:07.363 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:07.363 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.364 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.364 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:07.364 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414790+0000", 2024-09-21T15:01:07.364 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.995e-06 2024-09-21T15:01:07.364 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.364 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.364 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:07.364 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414801+0000", 2024-09-21T15:01:07.364 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.01e-05 2024-09-21T15:01:07.364 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.364 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.364 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:07.364 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414804+0000", 2024-09-21T15:01:07.364 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.563e-06 2024-09-21T15:01:07.365 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.365 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.365 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:07.365 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.474499+0000", 2024-09-21T15:01:07.365 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059695208 2024-09-21T15:01:07.365 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.365 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.365 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:07.365 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.474561+0000", 2024-09-21T15:01:07.365 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1672000000000002e-05 2024-09-21T15:01:07.365 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.365 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.365 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:07.365 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.474635+0000", 2024-09-21T15:01:07.365 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3792000000000001e-05 2024-09-21T15:01:07.366 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:07.366 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:07.366 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:07.366 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.366 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.366 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112865 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 4612096~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:07.366 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.414807+0000", 2024-09-21T15:01:07.366 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 37.890191000000002, 2024-09-21T15:01:07.366 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060003506999999998, 2024-09-21T15:01:07.366 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:07.366 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:07.366 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:07.366 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:07.367 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:07.367 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112865 2024-09-21T15:01:07.367 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.367 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:07.367 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.367 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:07.367 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414807+0000", 2024-09-21T15:01:07.367 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:07.367 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.367 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.367 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:07.367 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414807+0000", 2024-09-21T15:01:07.367 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:07.367 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.367 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.367 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:07.368 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414807+0000", 2024-09-21T15:01:07.368 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:07.368 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.368 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.368 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:07.368 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414809+0000", 2024-09-21T15:01:07.368 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0509999999999999e-06 2024-09-21T15:01:07.368 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.368 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.368 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:07.368 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414813+0000", 2024-09-21T15:01:07.368 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0960000000000003e-06 2024-09-21T15:01:07.368 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.368 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.368 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:07.368 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414817+0000", 2024-09-21T15:01:07.369 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.296e-06 2024-09-21T15:01:07.369 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.369 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.369 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:07.369 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.474671+0000", 2024-09-21T15:01:07.369 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059853663000000001 2024-09-21T15:01:07.369 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.369 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.369 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:07.369 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.474735+0000", 2024-09-21T15:01:07.369 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3653999999999997e-05 2024-09-21T15:01:07.369 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.369 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.369 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:07.369 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.474811+0000", 2024-09-21T15:01:07.369 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6074000000000004e-05 2024-09-21T15:01:07.370 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:07.370 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:07.370 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:07.370 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.370 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.370 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112866 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 4702208~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:07.370 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.414820+0000", 2024-09-21T15:01:07.370 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 37.890178079999998, 2024-09-21T15:01:07.370 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060161992999999997, 2024-09-21T15:01:07.370 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:07.370 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:07.370 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:07.370 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:07.370 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:07.370 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112866 2024-09-21T15:01:07.370 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.371 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:07.371 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.371 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:07.371 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414820+0000", 2024-09-21T15:01:07.371 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:07.371 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.371 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.371 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:07.371 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414820+0000", 2024-09-21T15:01:07.371 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:07.371 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.371 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.371 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:07.371 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414820+0000", 2024-09-21T15:01:07.371 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:07.372 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.372 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.372 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:07.372 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414822+0000", 2024-09-21T15:01:07.372 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8509999999999999e-06 2024-09-21T15:01:07.372 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.372 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.372 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:07.372 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414826+0000", 2024-09-21T15:01:07.372 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8190000000000002e-06 2024-09-21T15:01:07.372 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.372 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.372 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:07.372 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414837+0000", 2024-09-21T15:01:07.372 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1232e-05 2024-09-21T15:01:07.373 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.373 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.373 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:07.373 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.474846+0000", 2024-09-21T15:01:07.373 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060009506999999997 2024-09-21T15:01:07.373 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.373 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.373 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:07.373 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.474910+0000", 2024-09-21T15:01:07.373 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3918999999999996e-05 2024-09-21T15:01:07.373 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.373 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.373 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:07.373 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.474982+0000", 2024-09-21T15:01:07.373 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1958999999999993e-05 2024-09-21T15:01:07.374 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:07.374 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:07.374 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:07.374 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.374 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.374 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112867 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 4792320~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:07.374 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.414840+0000", 2024-09-21T15:01:07.374 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 37.890158585000002, 2024-09-21T15:01:07.374 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060308542999999999, 2024-09-21T15:01:07.374 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:07.374 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:07.374 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:07.374 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:07.374 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:07.375 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112867 2024-09-21T15:01:07.375 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.375 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:07.375 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.375 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:07.375 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414840+0000", 2024-09-21T15:01:07.375 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:07.375 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.375 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.375 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:07.375 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414840+0000", 2024-09-21T15:01:07.375 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:07.375 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.375 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.376 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:07.376 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414839+0000", 2024-09-21T15:01:07.376 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:07.376 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.376 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.376 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:07.376 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414841+0000", 2024-09-21T15:01:07.376 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8500000000000001e-06 2024-09-21T15:01:07.376 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.376 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.376 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:07.376 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414846+0000", 2024-09-21T15:01:07.376 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7700000000000001e-06 2024-09-21T15:01:07.376 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.376 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.376 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:07.377 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414850+0000", 2024-09-21T15:01:07.377 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1389999999999997e-06 2024-09-21T15:01:07.377 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.377 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.377 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:07.377 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.475016+0000", 2024-09-21T15:01:07.377 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.06016614 2024-09-21T15:01:07.377 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.377 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.377 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:07.377 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.475073+0000", 2024-09-21T15:01:07.377 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6295000000000002e-05 2024-09-21T15:01:07.377 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.377 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.377 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:07.378 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.475148+0000", 2024-09-21T15:01:07.378 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5647000000000003e-05 2024-09-21T15:01:07.378 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:07.378 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:07.378 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:07.378 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.378 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.378 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:113030 3.6 3:60420fc2:::rbd_data.1136b442e94a.0000000000000001:head [write 0~360448 [fadvise_sequential+fadvise_nocache],write 385024~573440 [fadvise_sequential+fadvise_nocache],write 1146880~573440 [fadvise_sequential+fadvise_nocache],write 1908736~188416 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:07.378 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.422420+0000", 2024-09-21T15:01:07.378 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 37.882577939999997, 2024-09-21T15:01:07.378 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.063268959999999999, 2024-09-21T15:01:07.378 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:07.378 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:07.378 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:07.378 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:07.379 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:07.379 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 113030 2024-09-21T15:01:07.379 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.379 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:07.379 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.379 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:07.379 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.422420+0000", 2024-09-21T15:01:07.379 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:07.379 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.379 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.379 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:07.379 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.422420+0000", 2024-09-21T15:01:07.379 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:07.379 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.379 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.379 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:07.380 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.422420+0000", 2024-09-21T15:01:07.380 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:07.380 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.380 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.380 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:07.380 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.424657+0000", 2024-09-21T15:01:07.380 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0022369780000000001 2024-09-21T15:01:07.380 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.380 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.380 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:07.380 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.424673+0000", 2024-09-21T15:01:07.380 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6141e-05 2024-09-21T15:01:07.380 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.380 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.380 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:07.381 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.424689+0000", 2024-09-21T15:01:07.381 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5897999999999999e-05 2024-09-21T15:01:07.381 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.381 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.381 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:07.381 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.478005+0000", 2024-09-21T15:01:07.381 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.05331636 2024-09-21T15:01:07.381 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.381 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.381 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:07.381 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.478133+0000", 2024-09-21T15:01:07.381 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012736799999999999 2024-09-21T15:01:07.381 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.381 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.381 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 7", 2024-09-21T15:01:07.382 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.478307+0000", 2024-09-21T15:01:07.382 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017491499999999999 2024-09-21T15:01:07.382 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.382 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.382 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:01:07.382 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.483611+0000", 2024-09-21T15:01:07.382 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0053039389999999997 2024-09-21T15:01:07.382 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.382 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.382 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:01:07.382 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.485553+0000", 2024-09-21T15:01:07.382 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0019418759999999999 2024-09-21T15:01:07.382 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.382 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.382 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:01:07.382 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.485645+0000", 2024-09-21T15:01:07.383 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2138000000000005e-05 2024-09-21T15:01:07.383 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.383 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.383 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:07.383 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.485689+0000", 2024-09-21T15:01:07.383 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3922999999999999e-05 2024-09-21T15:01:07.383 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:07.383 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:07.383 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:07.383 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.383 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.383 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:114290 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 6946816~8192 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:07.383 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.558087+0000", 2024-09-21T15:01:07.384 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 37.746911142999998, 2024-09-21T15:01:07.384 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059973091999999999, 2024-09-21T15:01:07.384 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:07.384 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:07.384 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:07.384 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:07.384 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:07.384 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 114290 2024-09-21T15:01:07.384 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.384 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:07.384 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.384 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:07.384 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.558087+0000", 2024-09-21T15:01:07.384 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:07.385 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.385 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.385 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:07.385 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.558087+0000", 2024-09-21T15:01:07.385 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:07.385 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.385 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.385 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:07.385 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.558087+0000", 2024-09-21T15:01:07.385 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:07.385 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.385 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.385 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:07.385 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.558092+0000", 2024-09-21T15:01:07.386 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2379999999999997e-06 2024-09-21T15:01:07.386 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.386 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.386 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:07.386 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.558095+0000", 2024-09-21T15:01:07.386 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.275e-06 2024-09-21T15:01:07.386 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.386 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.386 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:07.386 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.558099+0000", 2024-09-21T15:01:07.386 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6679999999999999e-06 2024-09-21T15:01:07.386 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.386 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.387 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:07.387 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.617757+0000", 2024-09-21T15:01:07.387 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059657531999999999 2024-09-21T15:01:07.387 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.387 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.387 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:07.387 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.617891+0000", 2024-09-21T15:01:07.387 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000134478 2024-09-21T15:01:07.387 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.387 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.387 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:07.387 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.618060+0000", 2024-09-21T15:01:07.387 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016925100000000001 2024-09-21T15:01:07.387 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:07.387 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:07.388 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:07.388 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.388 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.388 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:114312 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 8945664~8192 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:07.388 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.562396+0000", 2024-09-21T15:01:07.388 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 37.742602795000003, 2024-09-21T15:01:07.388 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059832396000000003, 2024-09-21T15:01:07.388 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:07.388 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:07.388 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:07.388 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:07.388 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:07.388 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 114312 2024-09-21T15:01:07.388 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.388 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:07.389 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.389 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:07.389 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562396+0000", 2024-09-21T15:01:07.389 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:07.389 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.389 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.389 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:07.389 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562396+0000", 2024-09-21T15:01:07.389 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:07.389 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.389 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.389 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:07.389 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562395+0000", 2024-09-21T15:01:07.389 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:07.389 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.389 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.390 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:07.390 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562397+0000", 2024-09-21T15:01:07.390 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9e-06 2024-09-21T15:01:07.390 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.390 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.390 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:07.390 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562401+0000", 2024-09-21T15:01:07.390 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8099999999999999e-06 2024-09-21T15:01:07.390 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.390 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.390 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:07.390 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562405+0000", 2024-09-21T15:01:07.390 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.5710000000000002e-06 2024-09-21T15:01:07.390 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.391 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.391 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:07.391 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.621997+0000", 2024-09-21T15:01:07.391 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059592118999999999 2024-09-21T15:01:07.391 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.391 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.391 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:07.391 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.622121+0000", 2024-09-21T15:01:07.391 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012411599999999999 2024-09-21T15:01:07.391 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.391 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.391 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:07.391 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.622228+0000", 2024-09-21T15:01:07.391 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000107184 2024-09-21T15:01:07.392 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:07.392 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:07.392 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:07.392 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.392 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.392 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:114313 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 9035776~8192 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:07.392 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.562408+0000", 2024-09-21T15:01:07.392 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 37.742590814000003, 2024-09-21T15:01:07.392 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059975560999999997, 2024-09-21T15:01:07.392 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:07.392 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:07.392 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:07.392 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:07.392 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:07.392 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 114313 2024-09-21T15:01:07.393 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.393 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:07.393 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.393 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:07.393 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562408+0000", 2024-09-21T15:01:07.393 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:07.393 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.393 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.393 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:07.393 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562408+0000", 2024-09-21T15:01:07.393 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:07.393 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.393 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.393 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:07.393 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562407+0000", 2024-09-21T15:01:07.394 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:07.394 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.394 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.394 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:07.394 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562409+0000", 2024-09-21T15:01:07.394 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9700000000000002e-06 2024-09-21T15:01:07.394 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.394 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.394 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:07.394 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562421+0000", 2024-09-21T15:01:07.394 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2094e-05 2024-09-21T15:01:07.394 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.394 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.395 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:07.395 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562427+0000", 2024-09-21T15:01:07.395 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7690000000000002e-06 2024-09-21T15:01:07.395 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.395 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.395 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:07.395 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.622264+0000", 2024-09-21T15:01:07.395 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059837488000000001 2024-09-21T15:01:07.395 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.395 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.395 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:07.395 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.622319+0000", 2024-09-21T15:01:07.395 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4064999999999998e-05 2024-09-21T15:01:07.395 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.395 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.396 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:07.396 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.622383+0000", 2024-09-21T15:01:07.396 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4506999999999998e-05 2024-09-21T15:01:07.396 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:07.396 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:07.396 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:07.396 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:07.396 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:07.396 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:01:07.398 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T15:01:07.399 INFO:teuthology.orchestra.run.smithi032.stdout:5643 write 0x237b724 thru 0x2381495 (0x5d72 bytes) 2024-09-21T15:01:07.399 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:01:07.399 INFO:teuthology.orchestra.run.smithi032.stdout:5645 write 0x213c985 thru 0x2148122 (0xb79e bytes) 2024-09-21T15:01:07.399 INFO:teuthology.orchestra.run.smithi032.stdout:5647 read 0x87949d thru 0x888952 (0xf4b6 bytes) 2024-09-21T15:01:07.399 INFO:teuthology.orchestra.run.smithi032.stdout:5648 punch from 0x3250856 to 0x325ed2e, (0xe4d8 bytes) 2024-09-21T15:01:07.399 INFO:teuthology.orchestra.run.smithi032.stdout:5649 write 0x6daa5cf thru 0x6dadb28 (0x355a bytes) 2024-09-21T15:01:07.400 INFO:teuthology.orchestra.run.smithi032.stdout:5651 write 0x2797302 thru 0x27a5ac3 (0xe7c2 bytes) 2024-09-21T15:01:07.400 INFO:teuthology.orchestra.run.smithi032.stdout:5652 read 0x574573d thru 0x5749990 (0x4254 bytes) 2024-09-21T15:01:07.400 INFO:teuthology.orchestra.run.smithi032.stdout:5654 read 0x1cece32 thru 0x1cee192 (0x1361 bytes) 2024-09-21T15:01:07.400 INFO:teuthology.orchestra.run.smithi032.stdout:5655 punch from 0x57baaab to 0x57bdc94, (0x31e9 bytes) 2024-09-21T15:01:07.400 INFO:teuthology.orchestra.run.smithi032.stdout:5656 read 0x553daa8 thru 0x5542bb2 (0x510b bytes) 2024-09-21T15:01:07.400 INFO:teuthology.orchestra.run.smithi032.stdout:5657 punch from 0x5133932 to 0x513f6f4, (0xbdc2 bytes) 2024-09-21T15:01:07.400 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:01:07.400 INFO:teuthology.orchestra.run.smithi032.stdout:5660 read 0x2617e00 thru 0x261c82a (0x4a2b bytes) 2024-09-21T15:01:07.400 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:01:07.400 INFO:teuthology.orchestra.run.smithi032.stdout:5664 read 0x2973462 thru 0x298026e (0xce0d bytes) 2024-09-21T15:01:07.400 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-21T15:01:07.401 INFO:teuthology.orchestra.run.smithi032.stdout:{"fsid":"caf51e55-9234-4a26-bcd2-d60811962560","health":{"status":"HEALTH_WARN","checks":{"OSDMAP_FLAGS":{"severity":"HEALTH_WARN","summary":{"message":"noscrub flag(s) set","count":7},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":197,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":178,"num_osds":8,"num_up_osds":8,"osd_up_since":1726930681,"num_in_osds":8,"osd_in_since":1726930767,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":41}],"num_pgs":41,"num_pools":3,"num_objects":1315,"data_bytes":981634133,"bytes_used":447287296,"bytes_avail":772646825984,"bytes_total":773094113280,"read_bytes_sec":530813,"write_bytes_sec":123617,"read_op_per_sec":71,"write_op_per_sec":4},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":4,"modified":"2024-09-21T14:59:03.791262+0000","services":{}},"progress_events":{}} 2024-09-21T15:01:07.401 INFO:teuthology.orchestra.run.smithi032.stdout:5665 write 0xe750d65 thru 0xe75927b (0x8517 bytes) 2024-09-21T15:01:07.402 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-21T15:01:07.402 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T15:01:07.403 INFO:teuthology.orchestra.run.smithi032.stdout:5667 punch from 0x1692119 to 0x169a749, (0x8630 bytes) 2024-09-21T15:01:07.403 INFO:teuthology.orchestra.run.smithi032.stdout:5670 writesame 0x5541446 thru 0x5544780 data_size 0x2b(0x333b bytes) 2024-09-21T15:01:07.403 INFO:teuthology.orchestra.run.smithi032.stdout:5671 read 0x2178f55 thru 0x218080f (0x78bb bytes) 2024-09-21T15:01:07.403 INFO:teuthology.orchestra.run.smithi032.stdout:5674 write 0x473efe0 thru 0x4748d2f (0x9d50 bytes) 2024-09-21T15:01:07.403 INFO:teuthology.orchestra.run.smithi032.stdout:5675 writesame 0xcfeedff thru 0xcff2b84 data_size 0x3(0x3d86 bytes) 2024-09-21T15:01:07.403 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:01:07.394+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:01:07.405 INFO:teuthology.orchestra.run.smithi032.stdout:5677 trunc from 0xe75927c to 0xb9f1537 2024-09-21T15:01:07.428 INFO:teuthology.orchestra.run.smithi032.stdout:5678 read 0x22e1340 thru 0x22eee03 (0xdac4 bytes) 2024-09-21T15:01:07.429 INFO:teuthology.orchestra.run.smithi032.stdout:5679 read 0x35690fa thru 0x356f5bf (0x64c6 bytes) 2024-09-21T15:01:07.432 INFO:teuthology.orchestra.run.smithi032.stdout:5680 trunc from 0xb9f1537 to 0xcfd6eed 2024-09-21T15:01:07.442 INFO:teuthology.orchestra.run.smithi032.stdout:5681 write 0x27b1926 thru 0x27bfb2f (0xe20a bytes) 2024-09-21T15:01:07.544 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:01:07.544 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:01:07.544 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T15:01:07.544 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:01:07.544 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T15:01:07.546 INFO:teuthology.orchestra.run.smithi032.stdout:5682 write 0x7e65b70 thru 0x7e726ea (0xcb7b bytes) 2024-09-21T15:01:07.546 INFO:teuthology.orchestra.run.smithi032.stdout:5683 read 0x1cdd8cd thru 0x1ce9bb0 (0xc2e4 bytes) 2024-09-21T15:01:07.546 INFO:teuthology.orchestra.run.smithi032.stdout:5684 punch from 0x38b001a to 0x38b7b6e, (0x7b54 bytes) 2024-09-21T15:01:07.547 INFO:teuthology.orchestra.run.smithi032.stdout:5685 write 0x7835112 thru 0x7844b23 (0xfa12 bytes) 2024-09-21T15:01:07.547 INFO:teuthology.orchestra.run.smithi032.stdout:5686 read 0x8df0ee3 thru 0x8dfee9b (0xdfb9 bytes) 2024-09-21T15:01:07.547 INFO:teuthology.orchestra.run.smithi032.stdout:5688 write 0x187a53c thru 0x188889e (0xe363 bytes) 2024-09-21T15:01:07.547 INFO:teuthology.orchestra.run.smithi032.stdout:5691 write 0xac0f566 thru 0xac1d5ad (0xe048 bytes) 2024-09-21T15:01:07.547 INFO:teuthology.orchestra.run.smithi032.stdout:5692 writesame 0x11f1486 thru 0x11fe0b8 data_size 0x33(0xcc33 bytes) 2024-09-21T15:01:07.547 INFO:teuthology.orchestra.run.smithi032.stdout:5693 punch from 0xa8d0edc to 0xa8d4730, (0x3854 bytes) 2024-09-21T15:01:07.547 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:01:07.547 INFO:teuthology.orchestra.run.smithi032.stdout:5696 punch from 0x8007212 to 0x800925c, (0x204a bytes) 2024-09-21T15:01:07.547 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:01:07.547 INFO:teuthology.orchestra.run.smithi032.stdout:5698 punch from 0x860f42f to 0x8614b65, (0x5736 bytes) 2024-09-21T15:01:07.547 INFO:teuthology.orchestra.run.smithi032.stdout:5699 read 0x24db59f thru 0x24e254d (0x6faf bytes) 2024-09-21T15:01:07.548 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:01:07.548 INFO:teuthology.orchestra.run.smithi032.stdout:5701 read 0x773f02b thru 0x7749164 (0xa13a bytes) 2024-09-21T15:01:07.548 INFO:teuthology.orchestra.run.smithi032.stdout:5702 writesame 0xd8d7d7d thru 0xd8e1115 data_size 0x37(0x9399 bytes) 2024-09-21T15:01:07.548 INFO:teuthology.orchestra.run.smithi032.stdout:5704 write 0x9c51ce7 thru 0x9c57f85 (0x629f bytes) 2024-09-21T15:01:07.548 INFO:teuthology.orchestra.run.smithi032.stdout:5706 punch from 0x6dd9270 to 0x6de506a, (0xbdfa bytes) 2024-09-21T15:01:07.548 INFO:teuthology.orchestra.run.smithi032.stdout:5708 punch from 0x39972ea to 0x399954c, (0x2262 bytes) 2024-09-21T15:01:07.548 INFO:teuthology.orchestra.run.smithi032.stdout:5709 writesame 0xa9e2bea thru 0xa9ed6ff data_size 0x3d(0xab16 bytes) 2024-09-21T15:01:07.548 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:01:07.548 INFO:teuthology.orchestra.run.smithi032.stdout:5711 trunc from 0xd8e1116 to 0x1cbcdea 2024-09-21T15:01:07.549 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:01:07.538+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:01:07.568 INFO:teuthology.orchestra.run.smithi032.stdout:5712 writesame 0xb04ff64 thru 0xb05593d data_size 0xe(0x59da bytes) 2024-09-21T15:01:07.568 INFO:teuthology.orchestra.run.smithi032.stdout:5713 write 0x6f19d48 thru 0x6f1c03a (0x22f3 bytes) 2024-09-21T15:01:07.570 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:01:07.570 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:01:07.570 INFO:teuthology.orchestra.run.smithi032.stdout:5720 writesame 0x3838338 thru 0x3841842 data_size 0xd(0x950b bytes) 2024-09-21T15:01:07.574 INFO:teuthology.orchestra.run.smithi032.stdout:5721 write 0x14a03b8 thru 0x14a0548 (0x191 bytes) 2024-09-21T15:01:07.592 INFO:teuthology.orchestra.run.smithi032.stdout:5722 writesame 0xe1e0a2e thru 0xe1ea4e6 data_size 0x51(0x9ab9 bytes) 2024-09-21T15:01:07.592 INFO:teuthology.orchestra.run.smithi032.stdout:5724 read 0x78b88e3 thru 0x78bd317 (0x4a35 bytes) 2024-09-21T15:01:07.592 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:01:07.592 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:01:07.592 INFO:teuthology.orchestra.run.smithi032.stdout:5728 read 0xabcde7e thru 0xabdbfc7 (0xe14a bytes) 2024-09-21T15:01:07.595 INFO:teuthology.orchestra.run.smithi032.stdout:5729 trunc from 0xe1ea4e7 to 0xcb8e15e 2024-09-21T15:01:07.623 INFO:teuthology.orchestra.run.smithi032.stdout:5730 trunc from 0xcb8e15e to 0x29e9746 2024-09-21T15:01:07.638 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:01:07.638 INFO:teuthology.orchestra.run.smithi032.stdout:5732 read 0x1fe2ed8 thru 0x1fee2b3 (0xb3dc bytes) 2024-09-21T15:01:07.639 INFO:teuthology.orchestra.run.smithi032.stdout:5735 read 0x1fe0b06 thru 0x1fe256e (0x1a69 bytes) 2024-09-21T15:01:07.639 INFO:teuthology.orchestra.run.smithi032.stdout:5736 punch from 0x12459c to 0x127b3d, (0x35a1 bytes) 2024-09-21T15:01:07.639 INFO:teuthology.orchestra.run.smithi032.stdout:5739 read 0x24d48a4 thru 0x24d5592 (0xcef bytes) 2024-09-21T15:01:07.640 INFO:teuthology.orchestra.run.smithi032.stdout:5740 punch from 0x198c695 to 0x198ccb7, (0x622 bytes) 2024-09-21T15:01:07.640 INFO:teuthology.orchestra.run.smithi032.stdout:5742 punch from 0x1077cdf to 0x107b355, (0x3676 bytes) 2024-09-21T15:01:07.641 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:01:07.637+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:01:07.654 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:01:07.654 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:01:07.654 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T15:01:07.654 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T15:01:07.654 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:01:07.655 INFO:teuthology.orchestra.run.smithi032.stdout:5743 trunc from 0x29e9746 to 0xbd3bf60 2024-09-21T15:01:07.664 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T15:01:07.664 INFO:teuthology.orchestra.run.smithi032.stdout:5744 writesame 0x5ef4224 thru 0x5efdb5e data_size 0x1(0x993b bytes) 2024-09-21T15:01:07.681 INFO:teuthology.orchestra.run.smithi032.stdout:5745 write 0x66e3469 thru 0x66e5cc1 (0x2859 bytes) 2024-09-21T15:01:07.684 INFO:teuthology.orchestra.run.smithi032.stdout:5746 writesame 0xa7356df thru 0xa73a02d data_size 0x7(0x494f bytes) 2024-09-21T15:01:07.706 INFO:teuthology.orchestra.run.smithi032.stdout:5747 write 0xc8e8c16 thru 0xc8f6c60 (0xe04b bytes) 2024-09-21T15:01:07.712 INFO:teuthology.orchestra.run.smithi032.stdout:5749 read 0x6508f3f thru 0x6512381 (0x9443 bytes) 2024-09-21T15:01:07.713 INFO:teuthology.orchestra.run.smithi032.stdout:5750 write 0x3ad6702 thru 0x3ae4c4b (0xe54a bytes) 2024-09-21T15:01:07.716 INFO:teuthology.orchestra.run.smithi032.stdout:5755 writesame 0x8117047 thru 0x8122d4c data_size 0xf(0xbd06 bytes) 2024-09-21T15:01:07.726 INFO:teuthology.orchestra.run.smithi032.stdout:5756 read 0xb6b8c73 thru 0xb6b95b4 (0x942 bytes) 2024-09-21T15:01:07.726 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:01:07.726 INFO:teuthology.orchestra.run.smithi032.stdout:5759 read 0xb912452 thru 0xb913f5f (0x1b0e bytes) 2024-09-21T15:01:07.736 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-21T15:01:07.740 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-09-21T15:01:07.742 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:01:07.737+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:01:07.744 INFO:teuthology.orchestra.run.smithi032.stdout:5760 trunc from 0xc8f6c61 to 0x258854e 2024-09-21T15:01:07.753 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":264,"stamp":"2024-09-21T15:01:06.158077+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1052048706,"num_objects":1338,"num_object_clones":0,"num_object_copies":2676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1338,"num_whiteouts":0,"num_read":144511,"num_read_kb":1152527,"num_write":26194,"num_write_kb":886220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1920,"num_bytes_recovered":145354624,"num_keys_recovered":229,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17355,"ondisk_log_size":17355,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":85,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":464376,"kb_used_data":86648,"kb_used_omap":12,"kb_used_meta":377587,"kb_avail":754510344,"statfs":{"total":773094113280,"available":772618592256,"internally_reserved":0,"allocated":88727552,"data_stored":646871178,"data_compressed":14204054,"data_compressed_allocated":58728448,"data_compressed_original":631348116,"omap_allocated":12744,"internal_metadata":386649656},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,31],"upper_bound":1024},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":70414573,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1851,"num_read_kb":25577,"num_write":148,"num_write_kb":20269,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":539,"num_bytes_recovered":41668853,"num_keys_recovered":77,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.954975"},"pg_stats":[{"pgid":"3.1f","version":"172'452","reported_seq":3450,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.157805+0000","last_change":"2024-09-21T15:01:04.223131+0000","last_active":"2024-09-21T15:01:05.157805+0000","last_peered":"2024-09-21T15:01:05.157805+0000","last_clean":"2024-09-21T15:01:05.157805+0000","last_became_active":"2024-09-21T14:58:19.410055+0000","last_became_peered":"2024-09-21T14:58:19.410055+0000","last_unstale":"2024-09-21T15:01:05.157805+0000","last_undegraded":"2024-09-21T15:01:05.157805+0000","last_fullsized":"2024-09-21T15:01:05.157805+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":452,"log_dups_size":0,"ondisk_log_size":452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:49:13.944219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063336100000000004,"stat_sum":{"num_bytes":32245069,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":3403,"num_read_kb":42213,"num_write":684,"num_write_kb":34241,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.1e","version":"176'366","reported_seq":2962,"reported_epoch":177,"state":"peering","last_fresh":"2024-09-21T15:01:05.160058+0000","last_change":"2024-09-21T15:01:05.160420+0000","last_active":"2024-09-21T15:01:05.160058+0000","last_peered":"2024-09-21T15:01:04.425541+0000","last_clean":"2024-09-21T15:01:00.453521+0000","last_became_active":"2024-09-21T15:01:03.253387+0000","last_became_peered":"2024-09-21T15:01:03.253387+0000","last_unstale":"2024-09-21T15:01:05.160058+0000","last_undegraded":"2024-09-21T15:01:05.160058+0000","last_fullsized":"2024-09-21T15:01:05.160058+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":366,"log_dups_size":0,"ondisk_log_size":366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:27:07.402534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021611780000000001,"stat_sum":{"num_bytes":19327265,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":0,"num_read":2999,"num_read_kb":22037,"num_write":453,"num_write_kb":16509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":148,"num_bytes_recovered":11635695,"num_keys_recovered":22,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,3],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"6b","length":"1"}]},{"pgid":"3.1d","version":"175'260","reported_seq":3611,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.164700+0000","last_change":"2024-09-21T15:01:04.229475+0000","last_active":"2024-09-21T15:01:05.164700+0000","last_peered":"2024-09-21T15:01:05.164700+0000","last_clean":"2024-09-21T15:01:05.164700+0000","last_became_active":"2024-09-21T14:58:19.411387+0000","last_became_peered":"2024-09-21T14:58:19.411387+0000","last_unstale":"2024-09-21T15:01:05.164700+0000","last_undegraded":"2024-09-21T15:01:05.164700+0000","last_fullsized":"2024-09-21T15:01:05.164700+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":260,"log_dups_size":0,"ondisk_log_size":260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:37:57.597069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054775500000000003,"stat_sum":{"num_bytes":18081929,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":3016,"num_read_kb":23636,"num_write":434,"num_write_kb":24977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.1c","version":"175'430","reported_seq":2874,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.187232+0000","last_change":"2024-09-21T15:01:04.234927+0000","last_active":"2024-09-21T15:01:05.187232+0000","last_peered":"2024-09-21T15:01:05.187232+0000","last_clean":"2024-09-21T15:01:05.187232+0000","last_became_active":"2024-09-21T14:58:19.411306+0000","last_became_peered":"2024-09-21T14:58:19.411306+0000","last_unstale":"2024-09-21T15:01:05.187232+0000","last_undegraded":"2024-09-21T15:01:05.187232+0000","last_fullsized":"2024-09-21T15:01:05.187232+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":430,"log_dups_size":0,"ondisk_log_size":430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:20:27.079795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010334350000000001,"stat_sum":{"num_bytes":19507114,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":2814,"num_read_kb":20995,"num_write":622,"num_write_kb":24708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.1b","version":"175'422","reported_seq":2730,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.158957+0000","last_change":"2024-09-21T15:01:04.227802+0000","last_active":"2024-09-21T15:01:05.158957+0000","last_peered":"2024-09-21T15:01:05.158957+0000","last_clean":"2024-09-21T15:01:05.158957+0000","last_became_active":"2024-09-21T14:59:31.131051+0000","last_became_peered":"2024-09-21T14:59:31.131051+0000","last_unstale":"2024-09-21T15:01:05.158957+0000","last_undegraded":"2024-09-21T15:01:05.158957+0000","last_fullsized":"2024-09-21T15:01:05.158957+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":422,"log_dups_size":0,"ondisk_log_size":422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:21:37.320774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001254673,"stat_sum":{"num_bytes":25447559,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":2614,"num_read_kb":28976,"num_write":559,"num_write_kb":20747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":117,"num_bytes_recovered":7757116,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.1a","version":"175'352","reported_seq":3233,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.187249+0000","last_change":"2024-09-21T15:01:04.234672+0000","last_active":"2024-09-21T15:01:05.187249+0000","last_peered":"2024-09-21T15:01:05.187249+0000","last_clean":"2024-09-21T15:01:05.187249+0000","last_became_active":"2024-09-21T14:58:19.409415+0000","last_became_peered":"2024-09-21T14:58:19.409415+0000","last_unstale":"2024-09-21T15:01:05.187249+0000","last_undegraded":"2024-09-21T15:01:05.187249+0000","last_fullsized":"2024-09-21T15:01:05.187249+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":352,"log_dups_size":0,"ondisk_log_size":352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:28:46.336301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060307399999999999,"stat_sum":{"num_bytes":31396202,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":3008,"num_read_kb":24746,"num_write":475,"num_write_kb":22526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.19","version":"175'648","reported_seq":3837,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.159011+0000","last_change":"2024-09-21T15:01:04.228157+0000","last_active":"2024-09-21T15:01:05.159011+0000","last_peered":"2024-09-21T15:01:05.159011+0000","last_clean":"2024-09-21T15:01:05.159011+0000","last_became_active":"2024-09-21T14:59:32.130624+0000","last_became_peered":"2024-09-21T14:59:32.130624+0000","last_unstale":"2024-09-21T15:01:05.159011+0000","last_undegraded":"2024-09-21T15:01:05.159011+0000","last_fullsized":"2024-09-21T15:01:05.159011+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":648,"log_dups_size":0,"ondisk_log_size":648,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:37:53.312733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001540409,"stat_sum":{"num_bytes":47896216,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":4950,"num_read_kb":51571,"num_write":1030,"num_write_kb":28173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":128,"num_bytes_recovered":11896992,"num_keys_recovered":44,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.18","version":"175'525","reported_seq":7399,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.157243+0000","last_change":"2024-09-21T15:01:04.223390+0000","last_active":"2024-09-21T15:01:05.157243+0000","last_peered":"2024-09-21T15:01:05.157243+0000","last_clean":"2024-09-21T15:01:05.157243+0000","last_became_active":"2024-09-21T14:58:19.409371+0000","last_became_peered":"2024-09-21T14:58:19.409371+0000","last_unstale":"2024-09-21T15:01:05.157243+0000","last_undegraded":"2024-09-21T15:01:05.157243+0000","last_fullsized":"2024-09-21T15:01:05.157243+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":525,"log_dups_size":0,"ondisk_log_size":525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:35:27.143445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076656300000000005,"stat_sum":{"num_bytes":36011050,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":8009,"num_read_kb":41151,"num_write":765,"num_write_kb":29537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.17","version":"172'334","reported_seq":3678,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.158315+0000","last_change":"2024-09-21T15:01:04.225804+0000","last_active":"2024-09-21T15:01:05.158315+0000","last_peered":"2024-09-21T15:01:05.158315+0000","last_clean":"2024-09-21T15:01:05.158315+0000","last_became_active":"2024-09-21T14:58:19.410306+0000","last_became_peered":"2024-09-21T14:58:19.410306+0000","last_unstale":"2024-09-21T15:01:05.158315+0000","last_undegraded":"2024-09-21T15:01:05.158315+0000","last_fullsized":"2024-09-21T15:01:05.158315+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":334,"log_dups_size":0,"ondisk_log_size":334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:33:50.089165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030273969999999998,"stat_sum":{"num_bytes":25072080,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":3370,"num_read_kb":27721,"num_write":557,"num_write_kb":31849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.16","version":"175'887","reported_seq":4751,"reported_epoch":177,"state":"creating+peering","last_fresh":"2024-09-21T15:01:04.608919+0000","last_change":"2024-09-21T15:01:05.877830+0000","last_active":"2024-09-21T15:01:05.877830+0000","last_peered":"2024-09-21T15:01:04.608919+0000","last_clean":"2024-09-21T15:01:00.518248+0000","last_became_active":"2024-09-21T15:01:03.252683+0000","last_became_peered":"2024-09-21T15:01:03.252683+0000","last_unstale":"2024-09-21T15:01:04.608919+0000","last_undegraded":"2024-09-21T15:01:04.608919+0000","last_fullsized":"2024-09-21T15:01:03.215921+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":887,"log_dups_size":0,"ondisk_log_size":887,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:19:04.988784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001973472,"stat_sum":{"num_bytes":33667875,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":6806,"num_read_kb":40610,"num_write":1387,"num_write_kb":26337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":271,"num_bytes_recovered":16821266,"num_keys_recovered":55,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"6b","length":"1"}]},{"pgid":"3.15","version":"175'717","reported_seq":3617,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.169730+0000","last_change":"2024-09-21T15:01:04.231959+0000","last_active":"2024-09-21T15:01:05.169730+0000","last_peered":"2024-09-21T15:01:05.169730+0000","last_clean":"2024-09-21T15:01:05.169730+0000","last_became_active":"2024-09-21T14:58:19.410687+0000","last_became_peered":"2024-09-21T14:58:19.410687+0000","last_unstale":"2024-09-21T15:01:05.169730+0000","last_undegraded":"2024-09-21T15:01:05.169730+0000","last_fullsized":"2024-09-21T15:01:05.169730+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":717,"log_dups_size":0,"ondisk_log_size":717,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:21:42.226982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033742220000000001,"stat_sum":{"num_bytes":45321568,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":4682,"num_read_kb":48927,"num_write":1212,"num_write_kb":32331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.14","version":"175'433","reported_seq":2993,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.164396+0000","last_change":"2024-09-21T15:01:04.229559+0000","last_active":"2024-09-21T15:01:05.164396+0000","last_peered":"2024-09-21T15:01:05.164396+0000","last_clean":"2024-09-21T15:01:05.164396+0000","last_became_active":"2024-09-21T14:58:19.407310+0000","last_became_peered":"2024-09-21T14:58:19.407310+0000","last_unstale":"2024-09-21T15:01:05.164396+0000","last_undegraded":"2024-09-21T15:01:05.164396+0000","last_fullsized":"2024-09-21T15:01:05.164396+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:47:46.079115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052455500000000001,"stat_sum":{"num_bytes":43744336,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":2858,"num_read_kb":38772,"num_write":581,"num_write_kb":32120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":171,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-21T15:01:04.229713+0000","last_change":"2024-09-21T14:58:11.348177+0000","last_active":"2024-09-21T15:01:04.229713+0000","last_peered":"2024-09-21T15:01:04.229713+0000","last_clean":"2024-09-21T15:01:04.229713+0000","last_became_active":"2024-09-21T14:58:09.328417+0000","last_became_peered":"2024-09-21T14:58:09.328417+0000","last_unstale":"2024-09-21T15:01:04.229713+0000","last_undegraded":"2024-09-21T15:01:04.229713+0000","last_fullsized":"2024-09-21T15:01:04.229713+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:52:22.671185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018917490000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"172'367","reported_seq":5379,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.158972+0000","last_change":"2024-09-21T15:01:04.228240+0000","last_active":"2024-09-21T15:01:05.158972+0000","last_peered":"2024-09-21T15:01:05.158972+0000","last_clean":"2024-09-21T15:01:05.158972+0000","last_became_active":"2024-09-21T14:59:32.131232+0000","last_became_peered":"2024-09-21T14:59:32.131232+0000","last_unstale":"2024-09-21T15:01:05.158972+0000","last_undegraded":"2024-09-21T15:01:05.158972+0000","last_fullsized":"2024-09-21T15:01:05.158972+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":367,"log_dups_size":0,"ondisk_log_size":367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:31:04.421782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001438581,"stat_sum":{"num_bytes":50093146,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":5172,"num_read_kb":51612,"num_write":497,"num_write_kb":25669,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":3585713,"num_keys_recovered":9,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":96,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-21T15:01:04.227015+0000","last_change":"2024-09-21T14:59:28.228883+0000","last_active":"2024-09-21T15:01:04.227015+0000","last_peered":"2024-09-21T15:01:04.227015+0000","last_clean":"2024-09-21T15:01:04.227015+0000","last_became_active":"2024-09-21T14:59:28.228464+0000","last_became_peered":"2024-09-21T14:59:28.228464+0000","last_unstale":"2024-09-21T15:01:04.227015+0000","last_undegraded":"2024-09-21T15:01:04.227015+0000","last_fullsized":"2024-09-21T15:01:04.227015+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:00:26.310864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"175'809","reported_seq":3401,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.159556+0000","last_change":"2024-09-21T15:01:04.228668+0000","last_active":"2024-09-21T15:01:05.159556+0000","last_peered":"2024-09-21T15:01:05.159556+0000","last_clean":"2024-09-21T15:01:05.159556+0000","last_became_active":"2024-09-21T14:59:31.132581+0000","last_became_peered":"2024-09-21T14:59:31.132581+0000","last_unstale":"2024-09-21T15:01:05.159556+0000","last_undegraded":"2024-09-21T15:01:05.159556+0000","last_fullsized":"2024-09-21T15:01:05.159556+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":809,"log_dups_size":0,"ondisk_log_size":809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:36:45.467515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016620179999999999,"stat_sum":{"num_bytes":28736355,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":4991,"num_read_kb":30499,"num_write":1191,"num_write_kb":23913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":119,"num_bytes_recovered":6871924,"num_keys_recovered":11,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":187,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-21T15:01:04.229145+0000","last_change":"2024-09-21T14:59:28.231502+0000","last_active":"2024-09-21T15:01:04.229145+0000","last_peered":"2024-09-21T15:01:04.229145+0000","last_clean":"2024-09-21T15:01:04.229145+0000","last_became_active":"2024-09-21T14:59:28.231043+0000","last_became_peered":"2024-09-21T14:59:28.231043+0000","last_unstale":"2024-09-21T15:01:04.229145+0000","last_undegraded":"2024-09-21T15:01:04.229145+0000","last_fullsized":"2024-09-21T15:01:04.229145+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:09:46.773690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019338770000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"172'553","reported_seq":5095,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.158909+0000","last_change":"2024-09-21T15:01:04.227811+0000","last_active":"2024-09-21T15:01:05.158909+0000","last_peered":"2024-09-21T15:01:05.158909+0000","last_clean":"2024-09-21T15:01:05.158909+0000","last_became_active":"2024-09-21T15:00:34.347238+0000","last_became_peered":"2024-09-21T15:00:34.347238+0000","last_unstale":"2024-09-21T15:01:05.158909+0000","last_undegraded":"2024-09-21T15:01:05.158909+0000","last_fullsized":"2024-09-21T15:01:05.158909+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":553,"log_dups_size":0,"ondisk_log_size":553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:31:33.639268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001277607,"stat_sum":{"num_bytes":39912228,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":5081,"num_read_kb":38156,"num_write":939,"num_write_kb":39278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":14049436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":172,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-21T15:01:04.222407+0000","last_change":"2024-09-21T14:58:11.406176+0000","last_active":"2024-09-21T15:01:04.222407+0000","last_peered":"2024-09-21T15:01:04.222407+0000","last_clean":"2024-09-21T15:01:04.222407+0000","last_became_active":"2024-09-21T14:58:09.777465+0000","last_became_peered":"2024-09-21T14:58:09.777465+0000","last_unstale":"2024-09-21T15:01:04.222407+0000","last_undegraded":"2024-09-21T15:01:04.222407+0000","last_fullsized":"2024-09-21T15:01:04.222407+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:55:04.667816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095704000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"172'662","reported_seq":2540,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.162165+0000","last_change":"2024-09-21T15:01:04.224786+0000","last_active":"2024-09-21T15:01:05.162165+0000","last_peered":"2024-09-21T15:01:05.162165+0000","last_clean":"2024-09-21T15:01:05.162165+0000","last_became_active":"2024-09-21T14:58:19.408971+0000","last_became_peered":"2024-09-21T14:58:19.408971+0000","last_unstale":"2024-09-21T15:01:05.162165+0000","last_undegraded":"2024-09-21T15:01:05.162165+0000","last_fullsized":"2024-09-21T15:01:05.162165+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":662,"log_dups_size":0,"ondisk_log_size":662,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:29:52.642121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000380054,"stat_sum":{"num_bytes":19474374,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":3561,"num_read_kb":20372,"num_write":1084,"num_write_kb":19812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.3","version":"172'738","reported_seq":2939,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.169226+0000","last_change":"2024-09-21T15:01:04.231096+0000","last_active":"2024-09-21T15:01:05.169226+0000","last_peered":"2024-09-21T15:01:05.169226+0000","last_clean":"2024-09-21T15:01:05.169226+0000","last_became_active":"2024-09-21T14:58:19.417453+0000","last_became_peered":"2024-09-21T14:58:19.417453+0000","last_unstale":"2024-09-21T15:01:05.169226+0000","last_undegraded":"2024-09-21T15:01:05.169226+0000","last_fullsized":"2024-09-21T15:01:05.169226+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":738,"log_dups_size":0,"ondisk_log_size":738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:19:27.087272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002785708,"stat_sum":{"num_bytes":15820678,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":4393,"num_read_kb":24172,"num_write":1224,"num_write_kb":20113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":173,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-21T15:01:04.232215+0000","last_change":"2024-09-21T14:58:11.347504+0000","last_active":"2024-09-21T15:01:04.232215+0000","last_peered":"2024-09-21T15:01:04.232215+0000","last_clean":"2024-09-21T15:01:04.232215+0000","last_became_active":"2024-09-21T14:58:09.775981+0000","last_became_peered":"2024-09-21T14:58:09.775981+0000","last_unstale":"2024-09-21T15:01:04.232215+0000","last_undegraded":"2024-09-21T15:01:04.232215+0000","last_fullsized":"2024-09-21T15:01:04.232215+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:47:16.286652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012568880000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"176'534","reported_seq":2983,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.157022+0000","last_change":"2024-09-21T15:01:04.223338+0000","last_active":"2024-09-21T15:01:05.157022+0000","last_peered":"2024-09-21T15:01:05.157022+0000","last_clean":"2024-09-21T15:01:05.157022+0000","last_became_active":"2024-09-21T14:58:19.405811+0000","last_became_peered":"2024-09-21T14:58:19.405811+0000","last_unstale":"2024-09-21T15:01:05.157022+0000","last_undegraded":"2024-09-21T15:01:05.157022+0000","last_fullsized":"2024-09-21T15:01:05.157022+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":534,"log_dups_size":0,"ondisk_log_size":534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:36:25.551607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083259800000000004,"stat_sum":{"num_bytes":20501650,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":3805,"num_read_kb":26326,"num_write":700,"num_write_kb":25158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":168,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-21T15:01:00.171240+0000","last_change":"2024-09-21T14:58:11.342210+0000","last_active":"2024-09-21T15:01:00.171240+0000","last_peered":"2024-09-21T15:01:00.171240+0000","last_clean":"2024-09-21T15:01:00.171240+0000","last_became_active":"2024-09-21T14:58:09.330624+0000","last_became_peered":"2024-09-21T14:58:09.330624+0000","last_unstale":"2024-09-21T15:01:00.171240+0000","last_undegraded":"2024-09-21T15:01:00.171240+0000","last_fullsized":"2024-09-21T15:01:00.171240+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:29:43.229538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001051151,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"175'276","reported_seq":2007,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.166575+0000","last_change":"2024-09-21T15:01:05.166575+0000","last_active":"2024-09-21T15:01:05.166575+0000","last_peered":"2024-09-21T15:01:05.166575+0000","last_clean":"2024-09-21T15:01:05.166575+0000","last_became_active":"2024-09-21T15:01:05.164862+0000","last_became_peered":"2024-09-21T15:01:05.164862+0000","last_unstale":"2024-09-21T15:01:05.166575+0000","last_undegraded":"2024-09-21T15:01:05.166575+0000","last_fullsized":"2024-09-21T15:01:05.166575+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":276,"log_dups_size":0,"ondisk_log_size":276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:57:46.675335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046340299999999998,"stat_sum":{"num_bytes":42467012,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":2366,"num_read_kb":39964,"num_write":407,"num_write_kb":39026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":254,"num_bytes_recovered":20571253,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":171,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-21T15:01:04.227843+0000","last_change":"2024-09-21T14:58:11.346766+0000","last_active":"2024-09-21T15:01:04.227843+0000","last_peered":"2024-09-21T15:01:04.227843+0000","last_clean":"2024-09-21T15:01:04.227843+0000","last_became_active":"2024-09-21T14:58:09.329748+0000","last_became_peered":"2024-09-21T14:58:09.329748+0000","last_unstale":"2024-09-21T15:01:04.227843+0000","last_undegraded":"2024-09-21T15:01:04.227843+0000","last_fullsized":"2024-09-21T15:01:04.227843+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:31:20.267675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052698000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"175'715","reported_seq":5700,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.160742+0000","last_change":"2024-09-21T15:01:04.224483+0000","last_active":"2024-09-21T15:01:05.160742+0000","last_peered":"2024-09-21T15:01:05.160742+0000","last_clean":"2024-09-21T15:01:05.160742+0000","last_became_active":"2024-09-21T14:58:41.401003+0000","last_became_peered":"2024-09-21T14:58:41.401003+0000","last_unstale":"2024-09-21T15:01:05.160742+0000","last_undegraded":"2024-09-21T15:01:05.160742+0000","last_fullsized":"2024-09-21T15:01:05.160742+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":715,"log_dups_size":0,"ondisk_log_size":715,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:05:37.644835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046693700000000001,"stat_sum":{"num_bytes":45819535,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":7513,"num_read_kb":42793,"num_write":987,"num_write_kb":21873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1403348,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":172,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-21T15:01:04.228858+0000","last_change":"2024-09-21T14:58:11.347700+0000","last_active":"2024-09-21T15:01:04.228858+0000","last_peered":"2024-09-21T15:01:04.228858+0000","last_clean":"2024-09-21T15:01:04.228858+0000","last_became_active":"2024-09-21T14:58:09.438842+0000","last_became_peered":"2024-09-21T14:58:09.438842+0000","last_unstale":"2024-09-21T15:01:04.228858+0000","last_undegraded":"2024-09-21T15:01:04.228858+0000","last_fullsized":"2024-09-21T15:01:04.228858+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:41:34.626249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001426194,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":320,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-21T15:01:04.233768+0000","last_change":"2024-09-21T14:58:03.616055+0000","last_active":"2024-09-21T15:01:04.233768+0000","last_peered":"2024-09-21T15:01:04.233768+0000","last_clean":"2024-09-21T15:01:04.233768+0000","last_became_active":"2024-09-21T14:58:03.613214+0000","last_became_peered":"2024-09-21T14:58:03.613214+0000","last_unstale":"2024-09-21T15:01:04.233768+0000","last_undegraded":"2024-09-21T15:01:04.233768+0000","last_fullsized":"2024-09-21T15:01:04.233768+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:02.180162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:02.180162+0000","last_clean_scrub_stamp":"2024-09-21T14:58:02.180162+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:29:05.860361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"172'602","reported_seq":4041,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.157772+0000","last_change":"2024-09-21T15:01:04.227318+0000","last_active":"2024-09-21T15:01:05.157772+0000","last_peered":"2024-09-21T15:01:05.157772+0000","last_clean":"2024-09-21T15:01:05.157772+0000","last_became_active":"2024-09-21T14:59:31.131460+0000","last_became_peered":"2024-09-21T14:59:31.131460+0000","last_unstale":"2024-09-21T15:01:05.157772+0000","last_undegraded":"2024-09-21T15:01:05.157772+0000","last_fullsized":"2024-09-21T15:01:05.157772+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":602,"log_dups_size":0,"ondisk_log_size":602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:01:38.353476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098975100000000009,"stat_sum":{"num_bytes":37289213,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":4356,"num_read_kb":43318,"num_write":990,"num_write_kb":29471,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":119,"num_bytes_recovered":6097618,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.9","version":"173'786","reported_seq":2989,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.164259+0000","last_change":"2024-09-21T15:01:04.232710+0000","last_active":"2024-09-21T15:01:05.164259+0000","last_peered":"2024-09-21T15:01:05.164259+0000","last_clean":"2024-09-21T15:01:05.164259+0000","last_became_active":"2024-09-21T14:59:28.233956+0000","last_became_peered":"2024-09-21T14:59:28.233956+0000","last_unstale":"2024-09-21T15:01:05.164259+0000","last_undegraded":"2024-09-21T15:01:05.164259+0000","last_fullsized":"2024-09-21T15:01:05.164259+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":786,"log_dups_size":0,"ondisk_log_size":786,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:29:09.682792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027159770000000001,"stat_sum":{"num_bytes":23720917,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":4523,"num_read_kb":33253,"num_write":1276,"num_write_kb":30634,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":91,"num_bytes_recovered":6925016,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.a","version":"176'786","reported_seq":3964,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.187273+0000","last_change":"2024-09-21T15:01:04.234662+0000","last_active":"2024-09-21T15:01:05.187273+0000","last_peered":"2024-09-21T15:01:05.187273+0000","last_clean":"2024-09-21T15:01:05.187273+0000","last_became_active":"2024-09-21T15:00:27.280971+0000","last_became_peered":"2024-09-21T15:00:27.280971+0000","last_unstale":"2024-09-21T15:01:05.187273+0000","last_undegraded":"2024-09-21T15:01:05.187273+0000","last_fullsized":"2024-09-21T15:01:05.187273+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":786,"log_dups_size":0,"ondisk_log_size":786,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:26:07.617059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076445899999999999,"stat_sum":{"num_bytes":20136068,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":5765,"num_read_kb":29533,"num_write":1129,"num_write_kb":31048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":9228123,"num_keys_recovered":33,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.b","version":"172'448","reported_seq":2919,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-21T15:01:00.700977+0000","last_change":"2024-09-21T15:00:59.287225+0000","last_active":"2024-09-21T15:01:00.700977+0000","last_peered":"2024-09-21T15:01:00.700977+0000","last_clean":"2024-09-21T15:01:00.700977+0000","last_became_active":"2024-09-21T14:58:19.410679+0000","last_became_peered":"2024-09-21T14:58:19.410679+0000","last_unstale":"2024-09-21T15:01:00.700977+0000","last_undegraded":"2024-09-21T15:01:00.700977+0000","last_fullsized":"2024-09-21T15:01:00.700977+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":448,"log_dups_size":0,"ondisk_log_size":448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:07:40.758932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0009774880000000001,"stat_sum":{"num_bytes":31436179,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":2840,"num_read_kb":35254,"num_write":697,"num_write_kb":37108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"6b","length":"1"}]},{"pgid":"3.c","version":"172'408","reported_seq":4312,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.169078+0000","last_change":"2024-09-21T15:01:04.231357+0000","last_active":"2024-09-21T15:01:05.169078+0000","last_peered":"2024-09-21T15:01:05.169078+0000","last_clean":"2024-09-21T15:01:05.169078+0000","last_became_active":"2024-09-21T14:59:25.130057+0000","last_became_peered":"2024-09-21T14:59:25.130057+0000","last_unstale":"2024-09-21T15:01:05.169078+0000","last_undegraded":"2024-09-21T15:01:05.169078+0000","last_fullsized":"2024-09-21T15:01:05.169078+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:53:33.752550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003377793,"stat_sum":{"num_bytes":22999993,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":4337,"num_read_kb":28452,"num_write":576,"num_write_kb":23757,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":9059600,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.d","version":"172'422","reported_seq":4914,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.164463+0000","last_change":"2024-09-21T15:01:04.233218+0000","last_active":"2024-09-21T15:01:05.164463+0000","last_peered":"2024-09-21T15:01:05.164463+0000","last_clean":"2024-09-21T15:01:05.164463+0000","last_became_active":"2024-09-21T14:58:19.407884+0000","last_became_peered":"2024-09-21T14:58:19.407884+0000","last_unstale":"2024-09-21T15:01:05.164463+0000","last_undegraded":"2024-09-21T15:01:05.164463+0000","last_fullsized":"2024-09-21T15:01:05.164463+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":422,"log_dups_size":0,"ondisk_log_size":422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:28:57.111615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034473960000000001,"stat_sum":{"num_bytes":49032397,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":4783,"num_read_kb":55414,"num_write":689,"num_write_kb":35805,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.e","version":"171'774","reported_seq":2852,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.176277+0000","last_change":"2024-09-21T15:01:05.176277+0000","last_active":"2024-09-21T15:01:05.176277+0000","last_peered":"2024-09-21T15:01:05.176277+0000","last_clean":"2024-09-21T15:01:05.176277+0000","last_became_active":"2024-09-21T15:01:05.172427+0000","last_became_peered":"2024-09-21T15:01:05.172427+0000","last_unstale":"2024-09-21T15:01:05.176277+0000","last_undegraded":"2024-09-21T15:01:05.176277+0000","last_fullsized":"2024-09-21T15:01:05.176277+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":774,"log_dups_size":0,"ondisk_log_size":774,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:28:18.710621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093917399999999998,"stat_sum":{"num_bytes":12028520,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":4155,"num_read_kb":16497,"num_write":1272,"num_write_kb":16419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":138,"num_bytes_recovered":7486496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.f","version":"175'427","reported_seq":5010,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.169470+0000","last_change":"2024-09-21T15:01:04.229660+0000","last_active":"2024-09-21T15:01:05.169470+0000","last_peered":"2024-09-21T15:01:05.169470+0000","last_clean":"2024-09-21T15:01:05.169470+0000","last_became_active":"2024-09-21T14:58:19.410406+0000","last_became_peered":"2024-09-21T14:58:19.410406+0000","last_unstale":"2024-09-21T15:01:05.169470+0000","last_undegraded":"2024-09-21T15:01:05.169470+0000","last_fullsized":"2024-09-21T15:01:05.169470+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":427,"log_dups_size":0,"ondisk_log_size":427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:59:16.026939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014556770000000001,"stat_sum":{"num_bytes":49430534,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":4917,"num_read_kb":52382,"num_write":667,"num_write_kb":35936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.10","version":"172'427","reported_seq":4297,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.186765+0000","last_change":"2024-09-21T15:01:04.233986+0000","last_active":"2024-09-21T15:01:05.186765+0000","last_peered":"2024-09-21T15:01:05.186765+0000","last_clean":"2024-09-21T15:01:05.186765+0000","last_became_active":"2024-09-21T14:58:19.409935+0000","last_became_peered":"2024-09-21T14:58:19.409935+0000","last_unstale":"2024-09-21T15:01:05.186765+0000","last_undegraded":"2024-09-21T15:01:05.186765+0000","last_fullsized":"2024-09-21T15:01:05.186765+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":427,"log_dups_size":0,"ondisk_log_size":427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:10:36.252629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001679137,"stat_sum":{"num_bytes":49580759,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":3927,"num_read_kb":58252,"num_write":670,"num_write_kb":35408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.11","version":"172'457","reported_seq":4563,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.168682+0000","last_change":"2024-09-21T15:01:05.168682+0000","last_active":"2024-09-21T15:01:05.168682+0000","last_peered":"2024-09-21T15:01:05.168682+0000","last_clean":"2024-09-21T15:01:05.168682+0000","last_became_active":"2024-09-21T15:01:05.165955+0000","last_became_peered":"2024-09-21T15:01:05.165955+0000","last_unstale":"2024-09-21T15:01:05.168682+0000","last_undegraded":"2024-09-21T15:01:05.168682+0000","last_fullsized":"2024-09-21T15:01:05.168682+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":457,"log_dups_size":0,"ondisk_log_size":457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:34:54.111119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010096230000000001,"stat_sum":{"num_bytes":23766643,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":5218,"num_read_kb":27936,"num_write":524,"num_write_kb":22561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":166,"num_bytes_recovered":11965028,"num_keys_recovered":33,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.12","version":"176'480","reported_seq":6284,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.163311+0000","last_change":"2024-09-21T15:01:04.224935+0000","last_active":"2024-09-21T15:01:05.163311+0000","last_peered":"2024-09-21T15:01:05.163311+0000","last_clean":"2024-09-21T15:01:05.163311+0000","last_became_active":"2024-09-21T15:00:07.155144+0000","last_became_peered":"2024-09-21T15:00:07.155144+0000","last_unstale":"2024-09-21T15:01:05.163311+0000","last_undegraded":"2024-09-21T15:01:05.163311+0000","last_fullsized":"2024-09-21T15:01:05.163311+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":480,"log_dups_size":0,"ondisk_log_size":480,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:58:00.133639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010497799999999999,"stat_sum":{"num_bytes":38476413,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":7022,"num_read_kb":30524,"num_write":743,"num_write_kb":18554,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.13","version":"175'738","reported_seq":5364,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.164109+0000","last_change":"2024-09-21T15:01:04.229229+0000","last_active":"2024-09-21T15:01:05.164109+0000","last_peered":"2024-09-21T15:01:05.164109+0000","last_clean":"2024-09-21T15:01:05.164109+0000","last_became_active":"2024-09-21T14:58:19.412030+0000","last_became_peered":"2024-09-21T14:58:19.412030+0000","last_unstale":"2024-09-21T15:01:05.164109+0000","last_undegraded":"2024-09-21T15:01:05.164109+0000","last_fullsized":"2024-09-21T15:01:05.164109+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":738,"log_dups_size":0,"ondisk_log_size":738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:44:29.282391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046048699999999998,"stat_sum":{"num_bytes":53017442,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":7141,"num_read_kb":56363,"num_write":1012,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"6e","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1051458319,"num_objects":1334,"num_object_clones":0,"num_object_copies":2668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1334,"num_whiteouts":0,"num_read":144395,"num_read_kb":1152427,"num_write":26033,"num_write_kb":883802,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1920,"num_bytes_recovered":145354624,"num_keys_recovered":229,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":71651328,"data_stored":637465472,"data_compressed":14193650,"data_compressed_allocated":58654720,"data_compressed_original":630201236,"omap_allocated":0,"internal_metadata":0},"log_size":17235,"ondisk_log_size":17235,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":10404,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509480,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34572,"kb_used_data":7552,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94337268,"statfs":{"total":96636764160,"available":96601362432,"internally_reserved":0,"allocated":7733248,"data_stored":48875988,"data_compressed":1141276,"data_compressed_allocated":4558848,"data_compressed_original":47268443,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:00:03 2024","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.84499999999999997,"15min":0.83899999999999997},"min":{"1min":0.504,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.95099999999999996,"15min":0.95799999999999996},"min":{"1min":0.51400000000000001,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.4970000000000001,"5min":1.917,"15min":1.917},"last":0.80200000000000005}]},{"osd":1,"last update":"Sat Sep 21 15:00:03 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.78500000000000003,"15min":0.76900000000000002},"min":{"1min":0.626,"5min":0.312,"15min":0.312},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.879,"5min":0.83499999999999996,"15min":0.82699999999999996},"min":{"1min":0.53600000000000003,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.224,"5min":2.2149999999999999,"15min":2.2149999999999999},"last":0.70199999999999996}]},{"osd":2,"last update":"Sat Sep 21 15:00:03 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.81699999999999995,"15min":0.81100000000000005},"min":{"1min":0.495,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.1859999999999999,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.95099999999999996,"15min":0.94799999999999995},"min":{"1min":0.72199999999999998,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.3300000000000001,"5min":1.466,"15min":1.466},"last":0.77800000000000002}]},{"osd":3,"last update":"Sat Sep 21 15:00:03 2024","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.86199999999999999,"15min":0.85399999999999998},"min":{"1min":0.67200000000000004,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.3120000000000001,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":0.82099999999999995},{"interface":"front","average":{"1min":1.0429999999999999,"5min":0.97999999999999998,"15min":0.96999999999999997},"min":{"1min":0.68500000000000005,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.244,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.98399999999999999}]},{"osd":4,"last update":"Sat Sep 21 15:00:03 2024","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.93700000000000006,"15min":0.93799999999999994},"min":{"1min":0.64200000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.4510000000000001,"5min":2.7130000000000001,"15min":2.7130000000000001},"last":0.59699999999999998},{"interface":"front","average":{"1min":1.0369999999999999,"5min":0.94099999999999995,"15min":0.92500000000000004},"min":{"1min":0.59999999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.6499999999999999,"5min":2.125,"15min":2.125},"last":0.75700000000000001}]},{"osd":5,"last update":"Sat Sep 21 15:00:03 2024","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.0409999999999999,"15min":1.0269999999999999},"min":{"1min":0.71099999999999997,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.4770000000000001,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":1.079},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.0640000000000001,"15min":1.0580000000000001},"min":{"1min":0.70799999999999996,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.8169999999999999,"5min":3.1880000000000002,"15min":3.1880000000000002},"last":0.84299999999999997}]},{"osd":6,"last update":"Sat Sep 21 15:00:03 2024","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.163,"15min":1.159},"min":{"1min":0.71299999999999997,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.6579999999999999,"5min":3.2730000000000001,"15min":3.2730000000000001},"last":1.036},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.0469999999999999,"15min":1.0249999999999999},"min":{"1min":0.79100000000000004,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.6180000000000001,"5min":2.2879999999999998,"15min":2.2879999999999998},"last":1.052}]}]},{"osd":6,"up_from":14,"seq":60129542184,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63192,"kb_used_data":9928,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94308648,"statfs":{"total":96636764160,"available":96572055552,"internally_reserved":0,"allocated":10166272,"data_stored":71351473,"data_compressed":1591208,"data_compressed_allocated":6516736,"data_compressed_original":69451532,"omap_allocated":1593,"internal_metadata":54524359},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,16],"upper_bound":1024},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:00:06 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.94399999999999995,"15min":0.95699999999999996},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.244,"5min":1.593,"15min":1.593},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.93600000000000005,"15min":0.95099999999999996},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.3360000000000001,"5min":1.587,"15min":1.587},"last":1.089}]},{"osd":1,"last update":"Sat Sep 21 15:00:06 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.90400000000000003,"15min":0.91400000000000003},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.1120000000000001,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.98299999999999998},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.95699999999999996,"15min":0.97999999999999998},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.1819999999999999,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.89100000000000001}]},{"osd":2,"last update":"Sat Sep 21 15:00:06 2024","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.98499999999999999,"15min":0.99099999999999999},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.3560000000000001,"5min":1.758,"15min":1.758},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.89900000000000002,"15min":0.90900000000000003},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.3069999999999999,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.93799999999999994}]},{"osd":3,"last update":"Sat Sep 21 15:00:06 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.94499999999999995,"15min":0.95799999999999996},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.169,"5min":1.835,"15min":1.835},"last":0.64200000000000002},{"interface":"front","average":{"1min":1.012,"5min":1.1180000000000001,"15min":1.1359999999999999},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.4430000000000001,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":1.2210000000000001}]},{"osd":4,"last update":"Sat Sep 21 15:00:06 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.97699999999999998,"15min":1.0049999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.1379999999999999,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.754},{"interface":"front","average":{"1min":0.93999999999999995,"5min":1.02,"15min":1.0329999999999999},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.411,"5min":1.897,"15min":1.897},"last":0.72399999999999998}]},{"osd":5,"last update":"Sat Sep 21 15:00:06 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":1.0860000000000001,"15min":1.1240000000000001},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.2589999999999999,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":1.0209999999999999},{"interface":"front","average":{"1min":1.01,"5min":1.0800000000000001,"15min":1.0920000000000001},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.3779999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":0.82299999999999995}]},{"osd":7,"last update":"Sat Sep 21 15:00:06 2024","interfaces":[{"interface":"back","average":{"1min":1.123,"5min":1.2350000000000001,"15min":1.2529999999999999},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.4530000000000001,"5min":1.8660000000000001,"15min":1.8660000000000001},"last":1.163},{"interface":"front","average":{"1min":1.044,"5min":1.26,"15min":1.296},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.385,"5min":2.004,"15min":2.004},"last":1.1140000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542185,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40508,"kb_used_data":9388,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94331332,"statfs":{"total":96636764160,"available":96595283968,"internally_reserved":0,"allocated":9613312,"data_stored":65131679,"data_compressed":1272182,"data_compressed_allocated":5857280,"data_compressed_original":63146092,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,15],"upper_bound":1024},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:00:01 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.96799999999999997,"15min":1.0109999999999999},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.98999999999999999,"5min":4.4800000000000004,"15min":4.4800000000000004},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.80800000000000005,"15min":0.82099999999999995},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.0329999999999999,"5min":1.181,"15min":1.181},"last":0.57499999999999996}]},{"osd":1,"last update":"Sat Sep 21 15:01:02 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.89600000000000002,"15min":0.92400000000000004},"min":{"1min":0.47699999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":2.7709999999999999,"5min":2.7709999999999999,"15min":2.7709999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.91400000000000003,"15min":0.93400000000000005},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":2.823,"5min":2.823,"15min":2.823},"last":0.84699999999999998}]},{"osd":2,"last update":"Sat Sep 21 15:01:02 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.91200000000000003,"15min":0.97399999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.159,"5min":3.419,"15min":3.419},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.87,"15min":0.89600000000000002},"min":{"1min":0.53400000000000003,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0780000000000001,"5min":1.423,"15min":1.423},"last":0.96299999999999997}]},{"osd":3,"last update":"Sat Sep 21 15:01:02 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.80300000000000005,"15min":0.80700000000000005},"min":{"1min":0.58499999999999996,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":1.258},{"interface":"front","average":{"1min":0.96699999999999997,"5min":1.054,"15min":1.0840000000000001},"min":{"1min":0.59899999999999998,"5min":0.435,"15min":0.435},"max":{"1min":1.272,"5min":2.6379999999999999,"15min":2.6379999999999999},"last":1.1399999999999999}]},{"osd":4,"last update":"Sat Sep 21 15:01:02 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.98899999999999999,"15min":1.016},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.5609999999999999,"5min":2.5470000000000002,"15min":2.5470000000000002},"last":1.038},{"interface":"front","average":{"1min":0.82099999999999995,"5min":1.0169999999999999,"15min":1.0780000000000001},"min":{"1min":0.39300000000000002,"5min":0.376,"15min":0.376},"max":{"1min":1.302,"5min":2.5099999999999998,"15min":2.5099999999999998},"last":1.302}]},{"osd":6,"last update":"Sat Sep 21 15:01:02 2024","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.94299999999999995,"15min":0.96899999999999997},"min":{"1min":0.63,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":1.0840000000000001},{"interface":"front","average":{"1min":1.004,"5min":1.2450000000000001,"15min":1.3520000000000001},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.425,"5min":4.2469999999999999,"15min":4.2469999999999999},"last":1.214}]},{"osd":7,"last update":"Sat Sep 21 15:00:05 2024","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.3200000000000001,"15min":1.363},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":2.7189999999999999,"5min":3.3010000000000002,"15min":3.3010000000000002},"last":1.345},{"interface":"front","average":{"1min":1.0740000000000001,"5min":1.1339999999999999,"15min":1.1439999999999999},"min":{"1min":0.64200000000000002,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":2.5880000000000001,"5min":2.5880000000000001,"15min":2.5880000000000001},"last":1.181}]}]},{"osd":4,"up_from":14,"seq":60129542184,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":67704,"kb_used_data":10340,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94304136,"statfs":{"total":96636764160,"available":96567435264,"internally_reserved":0,"allocated":10588160,"data_stored":79055540,"data_compressed":1750025,"data_compressed_allocated":7159808,"data_compressed_original":77331143,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.76000000000000001,"15min":0.77600000000000002},"min":{"1min":0.47499999999999998,"5min":0.311,"15min":0.311},"max":{"1min":1.1020000000000001,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":1.5860000000000001,"5min":1.0489999999999999,"15min":0.98099999999999998},"min":{"1min":0.49299999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":13.324,"5min":13.324,"15min":13.324},"last":1.1659999999999999}]},{"osd":1,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.85899999999999999,"15min":0.90600000000000003},"min":{"1min":0.46800000000000003,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.0609999999999999,"5min":1.347,"15min":1.347},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.82199999999999995,"15min":0.83899999999999997},"min":{"1min":0.502,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.58499999999999996}]},{"osd":2,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.88200000000000001,"15min":0.88},"min":{"1min":0.55100000000000005,"5min":0.501,"15min":0.501},"max":{"1min":1.47,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.98099999999999998,"5min":1.002,"15min":1.026},"min":{"1min":0.60699999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.3740000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":1.056}]},{"osd":3,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.90700000000000003,"15min":0.89500000000000002},"min":{"1min":0.61799999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.482,"5min":1.538,"15min":1.538},"last":1.121},{"interface":"front","average":{"1min":1.024,"5min":1.0129999999999999,"15min":0.99199999999999999},"min":{"1min":0.56599999999999995,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.528,"5min":1.528,"15min":1.528},"last":1.01}]},{"osd":5,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.97499999999999998,"15min":0.98399999999999999},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.579,"5min":1.579,"15min":1.579},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.94799999999999995,"15min":0.96399999999999997},"min":{"1min":0.61899999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":0.97299999999999998}]},{"osd":6,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.996,"15min":1.0109999999999999},"min":{"1min":0.55400000000000005,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.349,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.86599999999999999},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.081,"15min":1.0900000000000001},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.665,"5min":1.665,"15min":1.665},"last":0.71899999999999997}]},{"osd":7,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.995,"15min":1.0249999999999999},"min":{"1min":0.45200000000000001,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.226,"5min":1.7010000000000001,"15min":1.7010000000000001},"last":0.748},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.2050000000000001,"15min":1.2130000000000001},"min":{"1min":0.627,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.91800000000000004}]}]},{"osd":3,"up_from":14,"seq":60129542185,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58544,"kb_used_data":13472,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94313296,"statfs":{"total":96636764160,"available":96576815104,"internally_reserved":0,"allocated":13795328,"data_stored":103378431,"data_compressed":2311080,"data_compressed_allocated":9502720,"data_compressed_original":101103314,"omap_allocated":1592,"internal_metadata":46135752},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:00:05 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.378,"5min":0.315,"15min":0.315},"max":{"1min":1.0109999999999999,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.71199999999999997,"15min":0.71999999999999997},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.89200000000000002,"5min":1.919,"15min":1.919},"last":0.66100000000000003}]},{"osd":1,"last update":"Sat Sep 21 15:00:05 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.67500000000000004,"15min":0.67600000000000005},"min":{"1min":0.33400000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":1.1240000000000001,"5min":1.135,"15min":1.135},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.80700000000000005,"15min":0.81699999999999995},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.1599999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.753}]},{"osd":2,"last update":"Sat Sep 21 15:00:05 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.72799999999999998,"15min":0.72399999999999998},"min":{"1min":0.40200000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.83399999999999996,"15min":0.83499999999999996},"min":{"1min":0.47499999999999998,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.1759999999999999,"5min":1.254,"15min":1.254},"last":0.82299999999999995}]},{"osd":4,"last update":"Sat Sep 21 15:00:05 2024","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.95699999999999996,"15min":0.96299999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.2709999999999999,"5min":1.7609999999999999,"15min":1.7609999999999999},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.91100000000000003,"5min":1.0189999999999999,"15min":1.0369999999999999},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.25,"5min":2.028,"15min":2.028},"last":0.85799999999999998}]},{"osd":5,"last update":"Sat Sep 21 15:00:05 2024","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.052,"15min":1.0569999999999999},"min":{"1min":0.77700000000000002,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.246,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.022},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.0649999999999999,"15min":1.0660000000000001},"min":{"1min":0.70799999999999996,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.3600000000000001,"5min":1.431,"15min":1.431},"last":0.95699999999999996}]},{"osd":6,"last update":"Sat Sep 21 15:00:05 2024","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.0860000000000001,"15min":1.0980000000000001},"min":{"1min":0.58599999999999997,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.3879999999999999,"5min":2.1219999999999999,"15min":2.1219999999999999},"last":0.97999999999999998},{"interface":"front","average":{"1min":1.046,"5min":1.097,"15min":1.105},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.7370000000000001,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":1.1100000000000001}]},{"osd":7,"last update":"Sat Sep 21 15:00:05 2024","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.1279999999999999,"15min":1.139},"min":{"1min":0.61799999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.4870000000000001,"5min":1.706,"15min":1.706},"last":1.0469999999999999},{"interface":"front","average":{"1min":1.093,"5min":1.173,"15min":1.1859999999999999},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.516,"5min":2.3380000000000001,"15min":2.3380000000000001},"last":1.073}]}]},{"osd":2,"up_from":14,"seq":60129542183,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64844,"kb_used_data":11584,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94306996,"statfs":{"total":96636764160,"available":96570363904,"internally_reserved":0,"allocated":11862016,"data_stored":89535081,"data_compressed":2014763,"data_compressed_allocated":8007680,"data_compressed_original":87553058,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.62},"min":{"1min":0.26900000000000002,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.92800000000000005,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.69399999999999995,"15min":0.67000000000000004},"min":{"1min":0.34799999999999998,"5min":0.223,"15min":0.223},"max":{"1min":2.9169999999999998,"5min":2.9169999999999998,"15min":2.9169999999999998},"last":0.41299999999999998}]},{"osd":1,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.32700000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.0529999999999999,"5min":2.6520000000000001,"15min":2.6520000000000001},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.746,"5min":0.73499999999999999,"15min":0.75800000000000001},"min":{"1min":0.40500000000000003,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.349,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.69699999999999995}]},{"osd":3,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.73999999999999999,"15min":0.73499999999999999},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":3.0110000000000001,"5min":3.0110000000000001,"15min":3.0110000000000001},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.877,"15min":0.83899999999999997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.05,"5min":4.1440000000000001,"15min":4.1440000000000001},"last":0.47599999999999998}]},{"osd":4,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.93200000000000005,"15min":0.95599999999999996},"min":{"1min":0.42399999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.222,"5min":2.899,"15min":2.899},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.94399999999999995,"15min":0.95799999999999996},"min":{"1min":0.47499999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.4379999999999999,"5min":1.675,"15min":1.675},"last":0.73599999999999999}]},{"osd":5,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.0129999999999999,"15min":1.0089999999999999},"min":{"1min":0.45500000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":3.1080000000000001,"5min":3.1080000000000001,"15min":3.1080000000000001},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.98199999999999998,"15min":0.99399999999999999},"min":{"1min":0.51700000000000002,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.268,"5min":1.546,"15min":1.546},"last":0.51700000000000002}]},{"osd":6,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":0.98299999999999998,"15min":0.98499999999999999},"min":{"1min":0.56200000000000006,"5min":0.378,"15min":0.378},"max":{"1min":1.345,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":0.77700000000000002},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.0649999999999999,"15min":1.0549999999999999},"min":{"1min":0.53200000000000003,"5min":0.502,"15min":0.502},"max":{"1min":3.0739999999999998,"5min":3.0739999999999998,"15min":3.0739999999999998},"last":0.83799999999999997}]},{"osd":7,"last update":"Sat Sep 21 15:00:07 2024","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":1.0029999999999999,"15min":0.995},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.88700000000000001},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.097,"15min":1.091},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.5489999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":0.66000000000000003}]}]},{"osd":1,"up_from":14,"seq":60129542184,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64836,"kb_used_data":11572,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94307004,"statfs":{"total":96636764160,"available":96570372096,"internally_reserved":0,"allocated":11849728,"data_stored":91303543,"data_compressed":1932797,"data_compressed_allocated":8060928,"data_compressed_original":89352649,"omap_allocated":1595,"internal_metadata":54524357},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.59599999999999997,"15min":0.60899999999999999},"min":{"1min":0.36399999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.78200000000000003,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59699999999999998,"15min":0.60799999999999998},"min":{"1min":0.39500000000000002,"5min":0.216,"15min":0.216},"max":{"1min":0.80100000000000005,"5min":1.107,"15min":1.107},"last":0.39500000000000002}]},{"osd":2,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":0.79600000000000004,"15min":0.73999999999999999},"min":{"1min":0.38700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":11.303000000000001,"5min":11.303000000000001,"15min":11.303000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.71299999999999997,"15min":0.70299999999999996},"min":{"1min":0.47299999999999998,"5min":0.435,"15min":0.435},"max":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.81599999999999995}]},{"osd":3,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.81799999999999995,"15min":0.83899999999999997},"min":{"1min":0.27800000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.0049999999999999,"5min":3.1469999999999998,"15min":3.1469999999999998},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.86399999999999999,"15min":0.88600000000000001},"min":{"1min":0.52800000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.306,"5min":3.2320000000000002,"15min":3.2320000000000002},"last":0.94799999999999995}]},{"osd":4,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.89000000000000001,"15min":0.91500000000000004},"min":{"1min":0.59599999999999997,"5min":0.309,"15min":0.309},"max":{"1min":1.45,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":1.5600000000000001,"5min":1.069,"15min":0.96599999999999997},"min":{"1min":0.504,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":11.563000000000001,"5min":11.563000000000001,"15min":11.563000000000001},"last":1.0629999999999999}]},{"osd":5,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":1.625,"5min":1.0469999999999999,"15min":0.94899999999999995},"min":{"1min":0.73399999999999999,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":11.534000000000001,"5min":11.534000000000001,"15min":11.534000000000001},"last":1.121},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95799999999999996,"15min":0.96099999999999997},"min":{"1min":0.63,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.256,"5min":1.484,"15min":1.484},"last":1.256}]},{"osd":6,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":1.0109999999999999,"15min":1.0489999999999999},"min":{"1min":0.60399999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.4059999999999999,"5min":2.3519999999999999,"15min":2.3519999999999999},"last":0.85099999999999998},{"interface":"front","average":{"1min":1.631,"5min":1.081,"15min":0.99299999999999999},"min":{"1min":0.57399999999999995,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":11.601000000000001,"5min":11.601000000000001,"15min":11.601000000000001},"last":1.1639999999999999}]},{"osd":7,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":1.046,"15min":1.0640000000000001},"min":{"1min":0.79300000000000004,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.5820000000000001,"5min":2.2360000000000002,"15min":2.2360000000000002},"last":0.79300000000000004},{"interface":"front","average":{"1min":1.76,"5min":1.171,"15min":1.087},"min":{"1min":0.877,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":11.489000000000001,"5min":11.489000000000001,"15min":11.489000000000001},"last":1.23}]}]},{"osd":0,"up_from":14,"seq":60129542183,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70176,"kb_used_data":12812,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94301664,"statfs":{"total":96636764160,"available":96564903936,"internally_reserved":0,"allocated":13119488,"data_stored":98239443,"data_compressed":2190723,"data_compressed_allocated":9064448,"data_compressed_original":96141885,"omap_allocated":1593,"internal_metadata":58718663},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 21 15:01:04 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.67500000000000004,"15min":0.71699999999999997},"min":{"1min":0.40999999999999998,"5min":0.309,"15min":0.309},"max":{"1min":1.1299999999999999,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.68100000000000005,"15min":0.627},"min":{"1min":0.58999999999999997,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.67800000000000005}]},{"osd":2,"last update":"Sat Sep 21 15:01:04 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.65200000000000002,"15min":0.61799999999999999},"min":{"1min":0.59599999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.72499999999999998,"15min":0.75800000000000001},"min":{"1min":0.39300000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.79200000000000004}]},{"osd":3,"last update":"Sat Sep 21 15:01:04 2024","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.76600000000000001,"15min":0.754},"min":{"1min":0.50900000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.78700000000000003,"15min":0.81499999999999995},"min":{"1min":0.50900000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.50900000000000001}]},{"osd":4,"last update":"Sat Sep 21 15:01:04 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.98399999999999999,"15min":1.038},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.1140000000000001,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":1.036,"5min":1.0469999999999999,"15min":1.0620000000000001},"min":{"1min":0.67800000000000005,"5min":0.379,"15min":0.379},"max":{"1min":1.446,"5min":1.665,"15min":1.665},"last":0.71199999999999997}]},{"osd":5,"last update":"Sat Sep 21 15:01:04 2024","interfaces":[{"interface":"back","average":{"1min":1.0960000000000001,"5min":1.024,"15min":1.032},"min":{"1min":0.79300000000000004,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.4450000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.83899999999999997},{"interface":"front","average":{"1min":1.0589999999999999,"5min":1.0429999999999999,"15min":1.077},"min":{"1min":0.82399999999999995,"5min":0.435,"15min":0.435},"max":{"1min":1.2609999999999999,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":0.95499999999999996}]},{"osd":6,"last update":"Sat Sep 21 15:01:04 2024","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":1.0580000000000001,"15min":1.1120000000000001},"min":{"1min":0.68400000000000005,"5min":0.628,"15min":0.628},"max":{"1min":1.167,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":0.85799999999999998},{"interface":"front","average":{"1min":1.109,"5min":1.0629999999999999,"15min":1.0680000000000001},"min":{"1min":0.91000000000000003,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.5029999999999999,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.91900000000000004}]},{"osd":7,"last update":"Sat Sep 21 15:01:04 2024","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.0469999999999999,"15min":1.0620000000000001},"min":{"1min":0.78700000000000003,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.4770000000000001,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":0.88500000000000001},{"interface":"front","average":{"1min":1.02,"5min":1.085,"15min":1.131},"min":{"1min":0.84799999999999998,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.403,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":0.93600000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5202,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5202,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":11001856,"data_stored":97211967,"data_compressed":2190723,"data_compressed_allocated":9064448,"data_compressed_original":96141885,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9662464,"data_stored":89679641,"data_compressed":1927595,"data_compressed_allocated":8024064,"data_compressed_original":88779209,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":9732096,"data_stored":88501547,"data_compressed":2014763,"data_compressed_allocated":8007680,"data_compressed_original":87553058,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":11673600,"data_stored":102350936,"data_compressed":2311080,"data_compressed_allocated":9502720,"data_compressed_original":101103314,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8458240,"data_stored":78022006,"data_compressed":1750025,"data_compressed_allocated":7159808,"data_compressed_original":77331143,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":7426048,"data_stored":63507777,"data_compressed":1266980,"data_compressed_allocated":5820416,"data_compressed_original":62572652,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8044544,"data_stored":70323978,"data_compressed":1591208,"data_compressed_allocated":6516736,"data_compressed_original":69451532,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":5652480,"data_stored":47867620,"data_compressed":1141276,"data_compressed_allocated":4558848,"data_compressed_original":47268443,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T15:01:07.796 INFO:teuthology.orchestra.run.smithi032.stdout:5763 write 0xb5b5a68 thru 0xb5b5cc1 (0x25a bytes) 2024-09-21T15:01:07.802 INFO:teuthology.orchestra.run.smithi032.stdout:5764 punch from 0x3b89101 to 0x3b92dd2, (0x9cd1 bytes) 2024-09-21T15:01:07.821 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:01:07.821 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T15:01:07.821 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T15:01:07.821 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T15:01:07.821 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.821 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45854 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:07.821 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159802+0000", 2024-09-21T15:01:07.821 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 117.65698645800001, 2024-09-21T15:01:07.821 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068358938999999994, 2024-09-21T15:01:07.821 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:07.822 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:07.822 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:07.822 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:07.822 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:07.822 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45854 2024-09-21T15:01:07.822 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.822 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:07.822 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.822 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:07.822 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159802+0000", 2024-09-21T15:01:07.822 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:07.822 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.822 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.823 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:07.823 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159802+0000", 2024-09-21T15:01:07.823 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:07.823 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.823 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.823 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:07.823 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159801+0000", 2024-09-21T15:01:07.823 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:07.823 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.823 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.823 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:07.823 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159804+0000", 2024-09-21T15:01:07.823 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0970000000000002e-06 2024-09-21T15:01:07.823 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.823 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.824 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:07.824 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159823+0000", 2024-09-21T15:01:07.824 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8895000000000001e-05 2024-09-21T15:01:07.824 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.824 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.824 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:07.824 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159837+0000", 2024-09-21T15:01:07.824 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3247999999999999e-05 2024-09-21T15:01:07.824 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.824 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.824 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:07.824 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228151+0000", 2024-09-21T15:01:07.824 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068314642999999994 2024-09-21T15:01:07.824 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.825 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.825 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:07.825 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228161+0000", 2024-09-21T15:01:07.825 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4250000000000004e-06 2024-09-21T15:01:07.825 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:07.825 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:07.825 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:07.825 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.825 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.825 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45985 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:07.825 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159897+0000", 2024-09-21T15:01:07.825 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 117.656890956, 2024-09-21T15:01:07.825 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068345664, 2024-09-21T15:01:07.825 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:07.826 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:07.826 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:07.826 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:07.826 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:07.826 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45985 2024-09-21T15:01:07.826 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.826 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:07.826 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.826 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:07.826 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159897+0000", 2024-09-21T15:01:07.826 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:07.826 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.826 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.826 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:07.826 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159897+0000", 2024-09-21T15:01:07.827 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:07.827 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.827 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.827 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:07.827 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159897+0000", 2024-09-21T15:01:07.827 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:07.827 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.827 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.827 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:07.827 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159900+0000", 2024-09-21T15:01:07.827 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.954e-06 2024-09-21T15:01:07.827 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.827 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.827 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:07.827 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159913+0000", 2024-09-21T15:01:07.828 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3106e-05 2024-09-21T15:01:07.828 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.828 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.828 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:07.828 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159918+0000", 2024-09-21T15:01:07.828 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5199999999999997e-06 2024-09-21T15:01:07.828 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.828 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.828 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:07.828 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228236+0000", 2024-09-21T15:01:07.828 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068317040999999995 2024-09-21T15:01:07.828 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.828 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.828 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:07.828 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228243+0000", 2024-09-21T15:01:07.828 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3880000000000002e-06 2024-09-21T15:01:07.829 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:07.829 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:07.829 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:07.829 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.829 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.829 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45986 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:07.829 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159924+0000", 2024-09-21T15:01:07.829 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 117.656864661, 2024-09-21T15:01:07.829 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068364362999999997, 2024-09-21T15:01:07.829 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:07.829 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:07.829 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:07.829 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:07.829 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:07.830 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45986 2024-09-21T15:01:07.830 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.830 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:07.830 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.830 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:07.830 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159924+0000", 2024-09-21T15:01:07.830 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:07.830 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.830 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.830 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:07.830 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159924+0000", 2024-09-21T15:01:07.830 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:07.830 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.830 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.830 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:07.831 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159923+0000", 2024-09-21T15:01:07.831 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:07.831 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.831 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.831 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:07.831 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159926+0000", 2024-09-21T15:01:07.831 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0510000000000001e-06 2024-09-21T15:01:07.831 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.831 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.831 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:07.831 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159932+0000", 2024-09-21T15:01:07.831 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4160000000000001e-06 2024-09-21T15:01:07.831 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.831 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.832 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:07.832 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159938+0000", 2024-09-21T15:01:07.832 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.383e-06 2024-09-21T15:01:07.832 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.832 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.832 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:07.832 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228275+0000", 2024-09-21T15:01:07.832 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068337007000000005 2024-09-21T15:01:07.832 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.832 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.832 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:07.832 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228288+0000", 2024-09-21T15:01:07.832 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3052e-05 2024-09-21T15:01:07.833 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:07.833 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:07.833 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:07.833 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.833 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.833 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45987 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:07.833 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159944+0000", 2024-09-21T15:01:07.833 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 117.656844437, 2024-09-21T15:01:07.833 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068387232000000006, 2024-09-21T15:01:07.833 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:07.833 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:07.833 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:07.833 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:07.833 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:07.833 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45987 2024-09-21T15:01:07.834 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.834 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:07.834 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.834 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:07.834 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159944+0000", 2024-09-21T15:01:07.834 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:07.834 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.834 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.834 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:07.834 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159944+0000", 2024-09-21T15:01:07.834 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:07.834 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.834 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.834 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:07.835 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159943+0000", 2024-09-21T15:01:07.835 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:07.835 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.835 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.835 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:07.835 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159946+0000", 2024-09-21T15:01:07.835 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6309999999999999e-06 2024-09-21T15:01:07.835 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.835 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.835 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:07.835 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159952+0000", 2024-09-21T15:01:07.835 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6540000000000004e-06 2024-09-21T15:01:07.835 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.835 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.836 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:07.836 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159957+0000", 2024-09-21T15:01:07.836 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9280000000000001e-06 2024-09-21T15:01:07.836 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.836 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.836 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:07.836 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228322+0000", 2024-09-21T15:01:07.836 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068365067000000002 2024-09-21T15:01:07.836 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.836 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.836 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:07.836 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228331+0000", 2024-09-21T15:01:07.836 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3270000000000007e-06 2024-09-21T15:01:07.836 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:07.836 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:07.836 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:07.837 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.837 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.837 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:97672 3.4 3:26046d29:::rbd_data.1136bfc79a70.0000000000000001:head [write 0~4718592 [fadvise_sequential+fadvise_nocache],write 5242880~131072 [fadvise_sequential+fadvise_nocache],write 5439488~147456 [fadvise_sequential+fadvise_nocache],write 5652480~229376 [fadvise_sequential+fadvise_nocache],write 5947392~147456 [fadvise_sequential+fadvise_nocache],write 6160384~229376 [fadvise_sequential+fadvise_nocache],write 6455296~147456 [fadvise_sequential+fadvise_nocache],write 6668288~229376 [fadvise_sequential+fadvise_nocache],write 6963200~147456 [fadvise_sequential+fadvise_nocache],write 7176192~147456 [fadvise_sequential+fadvise_nocache],write 7340032~65536 [fadvise_sequential+fadvise_nocache],write 7471104~147456 [fadvise_sequential+fadvise_nocache],write 7684096~147456 [fadvise_sequential+fadvise_nocache],write 7864320~49152 [fadvise_sequential+fadvise_nocache],write 7979008~147456 [fadvise_sequential+fadvise_nocache],write 8192000~147456 [fadvise_sequential+fadvise_nocache],write 8388608~32768 [fadvise_sequential+fadvise_nocache],write 8519680~114688 [fadvise_sequential+fadvise_nocache],write 8732672~114688 [fadvise_sequential+fadvise_nocache],write 8912896~16384 [fadvise_sequential+fadvise_nocache],write 9109504~32768 [fadvise_sequential+fadvise_nocache],write 9322496~32768 [fadvise_sequential+fadvise_nocache],write 9617408~32768 [fadvise_sequential+fadvise_nocache],write 9830400~32768 [fadvise_sequential+fadvise_nocache],write 9961472~98304 [fadvise_sequential+fadvise_nocache],write 10076160~196608 [fadvise_sequential+fadvise_nocache],write 10289152~278528 [fadvise_sequential+fadvise_nocache],write 10584064~196608 [fadvise_sequential+fadvise_nocache],write 10797056~196608 [fadvise_sequential+fadvise_nocache],write 11010048~65536 [fadvise_sequential+fadvise_nocache],write 11091968~196608 [fadvise_sequential+fadvise_nocache],write 11304960~196608 [fadvise_sequential+fadvise_nocache],write 11517952~65536 [fadvise_sequential+fadvise_nocache],write 11616256~98304 [fadvise_sequential+fadvise_nocache],write 11730944~65536 [fadvise_sequential+fadvise_nocache],write 11829248~98304 [fadvise_sequential+fadvise_nocache],write 11943936~65536 [fadvise_sequential+fadvise_nocache],write 12042240~49152 [fadvise_sequential+fadvise_nocache],write 12124160~98304 [fadvise_sequential+fadvise_nocache],write 12238848~65536 [fadvise_sequential+fadvise_nocache],write 12337152~98304 [fadvise_sequential+fadvise_nocache],write 12451840~65536 [fadvise_sequential+fadvise_nocache],write 12550144~32768 [fadvise_sequential+fadvise_nocache],write 12632064~98304 [fadvise_sequential+fadvise_nocache],write 12845056~98304 [fadvise_sequential+fadvise_nocache],write 13058048~49152 [fadvise_sequential+fadvise_nocache],write 13139968~98304 [fadvise_sequential+fadvise_nocache],write 13352960~98304 [fadvise_sequential+fadvise_nocache],write 13565952~65536 [fadvise_sequential+fadvise_nocache],write 13647872~98304 [fadvise_sequential+fadvise_nocache],write 13860864~98304 [fadvise_sequential+fadvise_nocache],write 14073856~81920 [fadvise_sequential+fadvise_nocache],write 15269888~81920 [fadvise_sequential+fadvise_nocache],write 15482880~81920 [fadvise_sequential+fadvise_nocache],write 15695872~32768 [fadvise_sequential+fadvise_nocache],write 15777792~81920 [fadvise_sequential+fadvise_nocache],write 15990784~81920 [fadvise_sequential+fadvise_nocache],write 16203776~49152 [fadvise_sequential+fadvise_nocache],write 16285696~81920 [fadvise_sequential+fadvise_nocache],write 16498688~81920 [fadvise_sequential+fadvise_nocache],write 16711680~65536 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:01:07.837 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.298669+0000", 2024-09-21T15:01:07.837 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 62.518119370999997, 2024-09-21T15:01:07.837 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.10282358900000001, 2024-09-21T15:01:07.837 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:07.837 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:07.837 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:07.837 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:07.837 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:07.837 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97672 2024-09-21T15:01:07.837 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.837 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:07.837 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.837 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:07.838 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.298669+0000", 2024-09-21T15:01:07.838 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:07.838 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.838 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.838 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:07.838 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.298669+0000", 2024-09-21T15:01:07.838 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:07.838 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.838 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.838 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:07.838 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.298667+0000", 2024-09-21T15:01:07.838 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:01:07.838 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.838 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.838 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:07.839 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.327458+0000", 2024-09-21T15:01:07.839 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.028790754000000002 2024-09-21T15:01:07.839 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.839 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.839 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:07.839 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.327492+0000", 2024-09-21T15:01:07.839 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3806999999999997e-05 2024-09-21T15:01:07.839 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.839 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.839 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:07.839 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.327507+0000", 2024-09-21T15:01:07.839 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5112e-05 2024-09-21T15:01:07.839 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.839 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.839 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:07.839 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.327597+0000", 2024-09-21T15:01:07.840 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9938999999999994e-05 2024-09-21T15:01:07.840 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.840 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.840 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:07.840 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.327927+0000", 2024-09-21T15:01:07.840 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000329766 2024-09-21T15:01:07.840 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.840 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.840 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T15:01:07.840 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.330735+0000", 2024-09-21T15:01:07.840 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0028086869999999998 2024-09-21T15:01:07.840 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.840 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.840 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:01:07.840 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.378299+0000", 2024-09-21T15:01:07.840 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.047563495999999997 2024-09-21T15:01:07.841 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.841 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.841 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:01:07.841 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.401248+0000", 2024-09-21T15:01:07.841 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.022949311 2024-09-21T15:01:07.841 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.841 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.841 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:01:07.841 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.401446+0000", 2024-09-21T15:01:07.841 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019831199999999999 2024-09-21T15:01:07.841 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.841 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.841 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:07.841 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.401492+0000", 2024-09-21T15:01:07.841 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6109000000000002e-05 2024-09-21T15:01:07.841 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:07.841 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:07.842 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:07.842 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.842 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.842 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139769 3.1e 3:793dcf44:::rbd_data.113664780c85.0000000000000000:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:07.842 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.221432+0000", 2024-09-21T15:01:07.842 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.5953565019999996, 2024-09-21T15:01:07.842 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.035363362, 2024-09-21T15:01:07.842 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:07.842 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:07.842 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:07.842 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:07.842 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:07.842 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139769 2024-09-21T15:01:07.842 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.842 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:07.843 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.843 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:07.843 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221432+0000", 2024-09-21T15:01:07.843 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:07.843 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.843 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.843 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:07.843 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221432+0000", 2024-09-21T15:01:07.843 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:07.843 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.843 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.843 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:07.843 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221430+0000", 2024-09-21T15:01:07.843 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:01:07.844 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.844 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.844 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:07.844 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221438+0000", 2024-09-21T15:01:07.844 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2630000000000003e-06 2024-09-21T15:01:07.844 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.844 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.844 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:07.844 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221458+0000", 2024-09-21T15:01:07.844 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0071000000000001e-05 2024-09-21T15:01:07.844 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.844 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.844 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:07.844 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221467+0000", 2024-09-21T15:01:07.845 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.816e-06 2024-09-21T15:01:07.845 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.845 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.845 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:07.845 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221572+0000", 2024-09-21T15:01:07.845 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000105166 2024-09-21T15:01:07.845 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.845 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.845 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:07.845 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221589+0000", 2024-09-21T15:01:07.845 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7391e-05 2024-09-21T15:01:07.845 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.845 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.845 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:07.846 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.255741+0000", 2024-09-21T15:01:07.846 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0341519690000001 2024-09-21T15:01:07.846 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.846 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.846 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:07.846 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.256353+0000", 2024-09-21T15:01:07.846 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000611544 2024-09-21T15:01:07.846 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.846 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.846 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:07.846 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.256795+0000", 2024-09-21T15:01:07.846 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00044208700000000002 2024-09-21T15:01:07.846 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:07.846 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:07.847 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:07.847 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.847 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.847 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139773 3.1e 3:7fa3054f:::rbd_data.113664780c85.000000000000001c:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:07.847 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.221517+0000", 2024-09-21T15:01:07.847 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.595271458, 2024-09-21T15:01:07.847 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0367423140000001, 2024-09-21T15:01:07.847 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:07.847 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:07.847 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:07.847 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:07.847 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:07.847 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139773 2024-09-21T15:01:07.847 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.847 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:07.848 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.848 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:07.848 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221517+0000", 2024-09-21T15:01:07.848 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:07.848 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.848 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.848 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:07.848 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221517+0000", 2024-09-21T15:01:07.848 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:07.848 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.848 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.848 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:07.848 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221516+0000", 2024-09-21T15:01:07.848 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:07.848 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.849 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.849 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:07.849 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221520+0000", 2024-09-21T15:01:07.849 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1030000000000004e-06 2024-09-21T15:01:07.849 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.849 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.849 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:07.849 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221566+0000", 2024-09-21T15:01:07.849 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5512000000000001e-05 2024-09-21T15:01:07.849 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.849 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.849 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:07.849 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221575+0000", 2024-09-21T15:01:07.849 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9630000000000004e-06 2024-09-21T15:01:07.849 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.849 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.850 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:07.850 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221650+0000", 2024-09-21T15:01:07.850 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5531999999999994e-05 2024-09-21T15:01:07.850 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.850 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.850 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:07.850 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221661+0000", 2024-09-21T15:01:07.850 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0878e-05 2024-09-21T15:01:07.850 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.850 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.850 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:07.850 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.257513+0000", 2024-09-21T15:01:07.850 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0358513089999999 2024-09-21T15:01:07.850 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.850 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.851 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:07.851 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.257885+0000", 2024-09-21T15:01:07.851 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00037274299999999998 2024-09-21T15:01:07.851 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.851 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.851 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:07.851 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.258259+0000", 2024-09-21T15:01:07.851 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000373829 2024-09-21T15:01:07.851 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:07.851 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:07.851 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:07.851 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.851 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.851 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139789 3.1e 3:7c803367:::rbd_data.113664780c85.000000000000008c:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:07.851 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.221656+0000", 2024-09-21T15:01:07.852 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.5951318270000003, 2024-09-21T15:01:07.852 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.037139595, 2024-09-21T15:01:07.852 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:07.852 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:07.852 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:07.852 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:07.852 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:07.852 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139789 2024-09-21T15:01:07.852 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.852 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:07.852 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.852 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:07.852 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221656+0000", 2024-09-21T15:01:07.852 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:07.853 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.853 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.853 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:07.853 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221656+0000", 2024-09-21T15:01:07.853 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:07.853 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.853 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.853 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:07.853 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221656+0000", 2024-09-21T15:01:07.853 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:07.853 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.853 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.853 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:07.853 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221660+0000", 2024-09-21T15:01:07.854 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0369999999999996e-06 2024-09-21T15:01:07.854 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.854 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.854 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:07.854 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221668+0000", 2024-09-21T15:01:07.854 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3440000000000001e-06 2024-09-21T15:01:07.854 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.854 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.854 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:07.854 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221674+0000", 2024-09-21T15:01:07.854 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0920000000000003e-06 2024-09-21T15:01:07.854 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.854 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.855 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:07.855 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221787+0000", 2024-09-21T15:01:07.855 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000112289 2024-09-21T15:01:07.855 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.855 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.855 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:07.855 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221798+0000", 2024-09-21T15:01:07.855 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1419e-05 2024-09-21T15:01:07.855 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.855 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.855 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:07.855 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.258642+0000", 2024-09-21T15:01:07.855 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.036843851 2024-09-21T15:01:07.855 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.856 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.856 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:07.856 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.258796+0000", 2024-09-21T15:01:07.856 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015414199999999999 2024-09-21T15:01:07.856 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:07.856 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:07.856 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:07.856 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.856 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.856 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139832 3.1e 3:7b675eeb:::rbd_data.113664780c85.00000000000001b3:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:07.856 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.223976+0000", 2024-09-21T15:01:07.856 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.5928120940000001, 2024-09-21T15:01:07.856 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0350267980000001, 2024-09-21T15:01:07.856 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:07.856 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:07.857 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:07.857 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:07.857 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:07.857 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139832 2024-09-21T15:01:07.857 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.857 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:07.857 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.857 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:07.857 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223976+0000", 2024-09-21T15:01:07.857 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:07.857 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.857 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.857 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:07.857 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223976+0000", 2024-09-21T15:01:07.858 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:07.858 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.858 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.858 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:07.858 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223976+0000", 2024-09-21T15:01:07.858 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:07.858 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.858 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.858 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:07.858 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223981+0000", 2024-09-21T15:01:07.858 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1560000000000003e-06 2024-09-21T15:01:07.858 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.858 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.858 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:07.859 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223999+0000", 2024-09-21T15:01:07.859 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8808e-05 2024-09-21T15:01:07.859 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.859 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.859 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:07.859 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224008+0000", 2024-09-21T15:01:07.859 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7269999999999996e-06 2024-09-21T15:01:07.859 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.859 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.859 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:07.859 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224208+0000", 2024-09-21T15:01:07.859 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000199614 2024-09-21T15:01:07.859 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.859 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.860 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:07.860 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224218+0000", 2024-09-21T15:01:07.860 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0209e-05 2024-09-21T15:01:07.860 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.860 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.860 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:07.860 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.258839+0000", 2024-09-21T15:01:07.860 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.034621083 2024-09-21T15:01:07.860 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.860 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.860 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:07.860 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.259003+0000", 2024-09-21T15:01:07.860 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016379 2024-09-21T15:01:07.860 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:07.861 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:07.861 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:07.861 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.861 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.861 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139900 3.1e 3:7b440264:::rbd_data.113664780c85.0000000000000013:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:07.861 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.225748+0000", 2024-09-21T15:01:07.861 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.5910400930000002, 2024-09-21T15:01:07.861 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0339966009999999, 2024-09-21T15:01:07.861 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:07.861 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:07.861 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:07.861 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:07.861 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:07.861 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139900 2024-09-21T15:01:07.862 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.862 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:07.862 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.862 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:07.862 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.225748+0000", 2024-09-21T15:01:07.862 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:07.862 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.862 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.862 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:07.862 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.225748+0000", 2024-09-21T15:01:07.862 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:07.862 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.862 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.862 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:07.863 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.225746+0000", 2024-09-21T15:01:07.863 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:07.863 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.863 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.863 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:07.863 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.225757+0000", 2024-09-21T15:01:07.863 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.066e-05 2024-09-21T15:01:07.863 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.863 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.863 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:07.863 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.225775+0000", 2024-09-21T15:01:07.863 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7631000000000001e-05 2024-09-21T15:01:07.863 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.863 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.863 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:07.864 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.225788+0000", 2024-09-21T15:01:07.864 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3046e-05 2024-09-21T15:01:07.864 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.864 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.864 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:07.864 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.225881+0000", 2024-09-21T15:01:07.864 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2987999999999998e-05 2024-09-21T15:01:07.864 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.864 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.864 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:07.864 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.225893+0000", 2024-09-21T15:01:07.864 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2398999999999999e-05 2024-09-21T15:01:07.864 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.864 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.865 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:07.865 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.259207+0000", 2024-09-21T15:01:07.865 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.033313696 2024-09-21T15:01:07.865 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.865 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.865 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:07.865 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.259452+0000", 2024-09-21T15:01:07.865 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00024495700000000002 2024-09-21T15:01:07.865 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.865 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.865 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:07.865 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.259745+0000", 2024-09-21T15:01:07.865 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000292868 2024-09-21T15:01:07.865 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:07.865 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:07.866 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:07.866 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.866 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.866 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140008 3.1e 3:7cd8b476:::rbd_data.113664780c85.0000000000000149:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:07.866 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.229555+0000", 2024-09-21T15:01:07.866 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.5872332289999997, 2024-09-21T15:01:07.866 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0304914199999999, 2024-09-21T15:01:07.866 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:07.866 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:07.866 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:07.866 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:07.866 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:07.866 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140008 2024-09-21T15:01:07.866 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.867 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:07.867 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.867 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:07.867 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.229555+0000", 2024-09-21T15:01:07.867 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:07.867 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.867 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.867 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:07.867 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.229555+0000", 2024-09-21T15:01:07.867 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:07.867 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.867 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.867 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:07.867 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.229554+0000", 2024-09-21T15:01:07.868 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:07.868 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.868 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.868 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:07.868 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.229559+0000", 2024-09-21T15:01:07.868 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1059999999999999e-06 2024-09-21T15:01:07.868 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.868 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.868 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:07.868 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.229570+0000", 2024-09-21T15:01:07.868 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0465000000000001e-05 2024-09-21T15:01:07.868 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.868 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.868 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:07.868 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.229581+0000", 2024-09-21T15:01:07.869 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0958000000000001e-05 2024-09-21T15:01:07.869 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.869 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.869 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:07.869 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.229660+0000", 2024-09-21T15:01:07.869 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9103999999999993e-05 2024-09-21T15:01:07.869 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.869 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.869 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:07.869 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.229675+0000", 2024-09-21T15:01:07.869 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5622999999999999e-05 2024-09-21T15:01:07.869 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.869 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.869 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:07.869 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.259816+0000", 2024-09-21T15:01:07.870 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.030140595 2024-09-21T15:01:07.870 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.870 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.870 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:07.870 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.260046+0000", 2024-09-21T15:01:07.870 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00023045300000000001 2024-09-21T15:01:07.870 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:07.870 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:07.870 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:07.870 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.870 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.870 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140037 3.1e 3:7a4585d9:::rbd_data.113664780c85.000000000000005c:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:07.870 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.230439+0000", 2024-09-21T15:01:07.870 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.5863487410000001, 2024-09-21T15:01:07.871 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.030277613, 2024-09-21T15:01:07.871 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:07.871 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:07.871 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:07.871 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:07.871 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:07.871 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140037 2024-09-21T15:01:07.871 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.871 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:07.871 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.871 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:07.871 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.230439+0000", 2024-09-21T15:01:07.871 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:07.871 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.872 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.872 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:07.872 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.230439+0000", 2024-09-21T15:01:07.872 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:07.872 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.872 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.872 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:07.872 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.230438+0000", 2024-09-21T15:01:07.872 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:07.872 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.872 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.872 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:07.872 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.230447+0000", 2024-09-21T15:01:07.872 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6270000000000006e-06 2024-09-21T15:01:07.872 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.873 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.873 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:07.873 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.230464+0000", 2024-09-21T15:01:07.873 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7394e-05 2024-09-21T15:01:07.873 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.873 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.873 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:07.873 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.230472+0000", 2024-09-21T15:01:07.873 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0360000000000006e-06 2024-09-21T15:01:07.873 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.873 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.873 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:07.873 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.230567+0000", 2024-09-21T15:01:07.873 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4648000000000006e-05 2024-09-21T15:01:07.873 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.874 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.874 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:07.874 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.230581+0000", 2024-09-21T15:01:07.874 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3799e-05 2024-09-21T15:01:07.874 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.874 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.874 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:07.874 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.260095+0000", 2024-09-21T15:01:07.874 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.029514153 2024-09-21T15:01:07.874 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.874 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.874 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:07.874 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.260356+0000", 2024-09-21T15:01:07.874 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00026118400000000003 2024-09-21T15:01:07.875 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.875 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.875 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:07.875 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.260717+0000", 2024-09-21T15:01:07.875 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00036124799999999999 2024-09-21T15:01:07.875 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:07.875 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:07.875 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:07.875 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.875 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.875 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140061 3.1e 3:7fdd08f8:::rbd_data.113664780c85.0000000000000104:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:07.875 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.232026+0000", 2024-09-21T15:01:07.875 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.5847624109999998, 2024-09-21T15:01:07.875 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0290620450000001, 2024-09-21T15:01:07.875 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:07.875 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:07.876 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:07.876 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:07.876 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:07.876 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140061 2024-09-21T15:01:07.876 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.876 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:07.876 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.876 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:07.876 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232026+0000", 2024-09-21T15:01:07.876 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:07.876 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.876 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.876 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:07.876 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232026+0000", 2024-09-21T15:01:07.877 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:07.877 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.877 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.877 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:07.877 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232024+0000", 2024-09-21T15:01:07.877 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:01:07.877 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.877 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.877 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:07.877 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232034+0000", 2024-09-21T15:01:07.877 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0421000000000001e-05 2024-09-21T15:01:07.877 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.877 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.877 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:07.877 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232058+0000", 2024-09-21T15:01:07.877 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3944e-05 2024-09-21T15:01:07.878 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.878 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.878 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:07.878 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232071+0000", 2024-09-21T15:01:07.878 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2816999999999999e-05 2024-09-21T15:01:07.878 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.878 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.878 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:07.878 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232180+0000", 2024-09-21T15:01:07.878 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000109323 2024-09-21T15:01:07.878 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.878 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.878 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:07.878 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232199+0000", 2024-09-21T15:01:07.878 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8606000000000001e-05 2024-09-21T15:01:07.879 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.879 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.879 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:07.879 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.260821+0000", 2024-09-21T15:01:07.879 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0286222270000001 2024-09-21T15:01:07.879 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.879 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.879 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:07.879 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.261088+0000", 2024-09-21T15:01:07.879 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00026663799999999998 2024-09-21T15:01:07.879 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:07.879 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:07.879 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:07.879 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.879 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.880 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140090 3.1e 3:79f99893:::rbd_data.113664780c85.000000000000000b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:07.881 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.233668+0000", 2024-09-21T15:01:07.881 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.583120546, 2024-09-21T15:01:07.881 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0281047409999999, 2024-09-21T15:01:07.881 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:07.881 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:07.881 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:07.882 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:07.882 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:07.882 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140090 2024-09-21T15:01:07.882 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.882 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:07.882 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.882 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:07.882 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233668+0000", 2024-09-21T15:01:07.882 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:07.882 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.882 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.882 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:07.882 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233668+0000", 2024-09-21T15:01:07.882 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:07.882 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.883 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.883 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:07.883 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233666+0000", 2024-09-21T15:01:07.883 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:07.883 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.883 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.883 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:07.883 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233673+0000", 2024-09-21T15:01:07.883 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1840000000000002e-06 2024-09-21T15:01:07.883 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.883 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.883 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:07.883 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233689+0000", 2024-09-21T15:01:07.883 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5366000000000002e-05 2024-09-21T15:01:07.883 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.884 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.884 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:07.884 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233697+0000", 2024-09-21T15:01:07.884 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7679999999999997e-06 2024-09-21T15:01:07.884 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.884 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.884 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:07.884 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233772+0000", 2024-09-21T15:01:07.884 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4832000000000005e-05 2024-09-21T15:01:07.884 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.884 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.884 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:07.884 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233790+0000", 2024-09-21T15:01:07.884 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7297999999999999e-05 2024-09-21T15:01:07.885 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.885 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.885 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:07.885 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.261145+0000", 2024-09-21T15:01:07.885 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.027355381 2024-09-21T15:01:07.885 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.885 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.885 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:07.885 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.261466+0000", 2024-09-21T15:01:07.885 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00032105300000000002 2024-09-21T15:01:07.885 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.885 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.885 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:07.885 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.261772+0000", 2024-09-21T15:01:07.885 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00030635100000000001 2024-09-21T15:01:07.886 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:07.886 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:07.886 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:07.886 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.886 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.886 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140128 3.1e 3:78baf82e:::rbd_data.113664780c85.0000000000000115:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:07.886 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.235029+0000", 2024-09-21T15:01:07.886 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.5817592920000001, 2024-09-21T15:01:07.886 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.02698942, 2024-09-21T15:01:07.886 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:07.886 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:07.886 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:07.886 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:07.886 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:07.887 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140128 2024-09-21T15:01:07.887 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.887 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:07.887 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.887 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:07.887 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235029+0000", 2024-09-21T15:01:07.887 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:07.887 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.887 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.887 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:07.887 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235029+0000", 2024-09-21T15:01:07.887 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:07.887 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.887 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.888 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:07.888 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235028+0000", 2024-09-21T15:01:07.888 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:07.888 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.888 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.888 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:07.888 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235032+0000", 2024-09-21T15:01:07.888 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0579999999999999e-06 2024-09-21T15:01:07.888 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.888 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.888 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:07.888 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235040+0000", 2024-09-21T15:01:07.888 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2099999999999993e-06 2024-09-21T15:01:07.888 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.888 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.888 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:07.889 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235047+0000", 2024-09-21T15:01:07.889 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1099999999999999e-06 2024-09-21T15:01:07.889 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.889 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.889 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:07.889 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235123+0000", 2024-09-21T15:01:07.889 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6835999999999994e-05 2024-09-21T15:01:07.889 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.889 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.889 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:07.889 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235136+0000", 2024-09-21T15:01:07.889 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2962000000000001e-05 2024-09-21T15:01:07.889 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.889 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.890 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:07.890 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.261827+0000", 2024-09-21T15:01:07.890 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0266906149999999 2024-09-21T15:01:07.890 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.890 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.890 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:07.890 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.262018+0000", 2024-09-21T15:01:07.890 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019132400000000001 2024-09-21T15:01:07.890 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:07.890 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:07.890 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:07.890 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.890 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.890 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140217 3.1e 3:793dcf44:::rbd_data.113664780c85.0000000000000000:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:07.891 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.235497+0000", 2024-09-21T15:01:07.891 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.5812909929999996, 2024-09-21T15:01:07.891 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0269053020000001, 2024-09-21T15:01:07.891 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:07.891 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:07.891 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:07.891 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:07.891 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:07.891 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140217 2024-09-21T15:01:07.891 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.891 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:07.891 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.891 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:07.891 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235497+0000", 2024-09-21T15:01:07.891 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:07.892 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.892 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.892 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:07.892 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235497+0000", 2024-09-21T15:01:07.892 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:07.892 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.892 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.892 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:07.892 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235496+0000", 2024-09-21T15:01:07.892 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:07.892 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.892 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.892 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:07.892 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235503+0000", 2024-09-21T15:01:07.892 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8460000000000002e-06 2024-09-21T15:01:07.892 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.893 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.893 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:07.893 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235514+0000", 2024-09-21T15:01:07.893 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0576e-05 2024-09-21T15:01:07.893 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.893 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.893 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:07.893 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235522+0000", 2024-09-21T15:01:07.893 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2640000000000008e-06 2024-09-21T15:01:07.893 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.893 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.893 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:07.893 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235579+0000", 2024-09-21T15:01:07.893 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6842999999999998e-05 2024-09-21T15:01:07.894 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.894 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.894 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:07.894 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235590+0000", 2024-09-21T15:01:07.894 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1194e-05 2024-09-21T15:01:07.894 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.894 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.894 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:07.894 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.262119+0000", 2024-09-21T15:01:07.894 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0265294540000001 2024-09-21T15:01:07.894 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.894 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.894 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:07.894 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.262241+0000", 2024-09-21T15:01:07.895 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012182 2024-09-21T15:01:07.895 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.895 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.895 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:07.895 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.262403+0000", 2024-09-21T15:01:07.895 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000161442 2024-09-21T15:01:07.895 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:07.895 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:07.895 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:07.895 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.895 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.895 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140221 3.1e 3:7fa3054f:::rbd_data.113664780c85.000000000000001c:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:07.895 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.235592+0000", 2024-09-21T15:01:07.895 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.5811961319999996, 2024-09-21T15:01:07.895 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.027322012, 2024-09-21T15:01:07.896 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:07.896 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:07.896 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:07.896 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:07.896 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:07.896 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140221 2024-09-21T15:01:07.896 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.896 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:07.896 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.896 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:07.896 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235592+0000", 2024-09-21T15:01:07.896 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:07.896 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.896 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.896 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:07.897 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235592+0000", 2024-09-21T15:01:07.897 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:07.897 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.897 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.897 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:07.897 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235591+0000", 2024-09-21T15:01:07.897 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:07.897 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.897 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.897 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:07.897 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235596+0000", 2024-09-21T15:01:07.897 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4209999999999997e-06 2024-09-21T15:01:07.897 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.897 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.897 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:07.898 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235603+0000", 2024-09-21T15:01:07.898 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.909e-06 2024-09-21T15:01:07.898 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.898 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.898 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:07.898 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235609+0000", 2024-09-21T15:01:07.898 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3539999999999999e-06 2024-09-21T15:01:07.898 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.898 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.898 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:07.898 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236013+0000", 2024-09-21T15:01:07.898 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00040399799999999998 2024-09-21T15:01:07.898 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.899 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.899 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:07.899 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236020+0000", 2024-09-21T15:01:07.899 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6699999999999994e-06 2024-09-21T15:01:07.899 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.899 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.899 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:07.899 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.262452+0000", 2024-09-21T15:01:07.899 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.02643145 2024-09-21T15:01:07.899 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.899 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.899 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:07.899 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.262624+0000", 2024-09-21T15:01:07.899 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000172206 2024-09-21T15:01:07.899 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.900 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.900 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:07.900 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.262914+0000", 2024-09-21T15:01:07.900 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00028994600000000002 2024-09-21T15:01:07.900 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:07.900 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:07.900 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:07.900 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.900 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.900 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140237 3.1e 3:7c803367:::rbd_data.113664780c85.000000000000008c:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:07.900 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.237113+0000", 2024-09-21T15:01:07.900 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.5796756250000001, 2024-09-21T15:01:07.900 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0260336299999999, 2024-09-21T15:01:07.900 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:07.900 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:07.901 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:07.901 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:07.901 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:07.901 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140237 2024-09-21T15:01:07.901 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.901 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:07.901 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.901 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:07.901 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237113+0000", 2024-09-21T15:01:07.901 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:07.901 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.901 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.901 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:07.901 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237113+0000", 2024-09-21T15:01:07.901 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:07.901 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.902 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.902 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:07.902 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237112+0000", 2024-09-21T15:01:07.902 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:07.902 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.902 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.902 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:07.902 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237118+0000", 2024-09-21T15:01:07.902 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8930000000000002e-06 2024-09-21T15:01:07.902 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.902 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.902 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:07.902 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237126+0000", 2024-09-21T15:01:07.902 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6729999999999998e-06 2024-09-21T15:01:07.902 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.903 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.903 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:07.903 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237135+0000", 2024-09-21T15:01:07.903 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0929999999999992e-06 2024-09-21T15:01:07.903 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.903 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.903 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:07.903 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237205+0000", 2024-09-21T15:01:07.903 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9697999999999996e-05 2024-09-21T15:01:07.903 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.903 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.903 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:07.903 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237218+0000", 2024-09-21T15:01:07.903 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2524000000000001e-05 2024-09-21T15:01:07.904 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.904 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.904 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:07.904 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.262965+0000", 2024-09-21T15:01:07.904 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0257478630000001 2024-09-21T15:01:07.904 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.904 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.904 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:07.904 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.263146+0000", 2024-09-21T15:01:07.904 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018074599999999999 2024-09-21T15:01:07.904 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:07.904 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:07.904 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:07.905 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.905 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.905 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140280 3.1e 3:7b675eeb:::rbd_data.113664780c85.00000000000001b3:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:07.905 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.237477+0000", 2024-09-21T15:01:07.905 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.5793113630000004, 2024-09-21T15:01:07.905 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.026437329, 2024-09-21T15:01:07.905 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:07.905 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:07.905 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:07.905 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:07.905 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:07.905 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140280 2024-09-21T15:01:07.905 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.905 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:07.905 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.906 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:07.906 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237477+0000", 2024-09-21T15:01:07.906 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:07.906 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.906 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.906 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:07.906 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237477+0000", 2024-09-21T15:01:07.906 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:07.906 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.906 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.906 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:07.906 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237476+0000", 2024-09-21T15:01:07.906 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:07.907 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.907 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.907 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:07.907 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237480+0000", 2024-09-21T15:01:07.907 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7570000000000002e-06 2024-09-21T15:01:07.907 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.907 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.907 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:07.907 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237490+0000", 2024-09-21T15:01:07.907 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3170000000000008e-06 2024-09-21T15:01:07.907 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.907 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.907 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:07.907 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237496+0000", 2024-09-21T15:01:07.908 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7150000000000001e-06 2024-09-21T15:01:07.908 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.908 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.908 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:07.908 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237559+0000", 2024-09-21T15:01:07.908 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2844999999999999e-05 2024-09-21T15:01:07.908 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.908 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.908 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:07.908 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237570+0000", 2024-09-21T15:01:07.908 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1253000000000001e-05 2024-09-21T15:01:07.908 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.908 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.908 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:07.909 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.263203+0000", 2024-09-21T15:01:07.909 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0256330339999999 2024-09-21T15:01:07.909 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.909 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.909 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:07.909 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.263914+0000", 2024-09-21T15:01:07.909 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00071080899999999996 2024-09-21T15:01:07.909 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:07.909 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:07.909 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:07.909 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.909 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.909 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140348 3.1e 3:7b440264:::rbd_data.113664780c85.0000000000000013:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:07.909 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.241762+0000", 2024-09-21T15:01:07.910 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 5.5750260919999999, 2024-09-21T15:01:07.910 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0221355759999999, 2024-09-21T15:01:07.910 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:07.910 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:07.910 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:07.910 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:07.910 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:07.910 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140348 2024-09-21T15:01:07.910 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.910 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:07.910 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.910 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:07.910 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.241762+0000", 2024-09-21T15:01:07.910 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:07.911 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.911 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.911 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:07.911 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.241762+0000", 2024-09-21T15:01:07.911 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:07.911 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.911 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.911 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:07.911 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.241761+0000", 2024-09-21T15:01:07.911 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:07.911 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.911 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.911 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:07.911 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.241767+0000", 2024-09-21T15:01:07.912 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4219999999999997e-06 2024-09-21T15:01:07.912 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.912 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.912 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:07.912 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.241783+0000", 2024-09-21T15:01:07.912 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5959000000000001e-05 2024-09-21T15:01:07.912 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.912 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.912 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:07.912 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.241793+0000", 2024-09-21T15:01:07.912 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3999999999999998e-06 2024-09-21T15:01:07.912 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.912 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.912 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:07.913 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.241894+0000", 2024-09-21T15:01:07.913 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000100976 2024-09-21T15:01:07.913 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.913 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.913 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:07.913 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.241907+0000", 2024-09-21T15:01:07.913 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3281e-05 2024-09-21T15:01:07.913 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.913 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.913 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:07.913 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.263430+0000", 2024-09-21T15:01:07.913 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0215229379999999 2024-09-21T15:01:07.913 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.913 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.913 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:07.914 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.263604+0000", 2024-09-21T15:01:07.914 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017368700000000001 2024-09-21T15:01:07.914 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:07.914 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:07.914 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:07.914 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.263898+0000", 2024-09-21T15:01:07.914 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00029406899999999999 2024-09-21T15:01:07.914 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:07.914 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:07.914 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:07.914 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:07.914 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:07.914 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:01:07.916 INFO:teuthology.orchestra.run.smithi032.stdout:5765 writesame 0xde36162 thru 0xde456ae data_size 0x1(0xf54d bytes) 2024-09-21T15:01:07.916 INFO:teuthology.orchestra.run.smithi032.stdout:5766 writesame 0xd074a85 thru 0xd07636f data_size 0x1(0x18eb bytes) 2024-09-21T15:01:07.916 INFO:teuthology.orchestra.run.smithi032.stdout:5767 punch from 0x7da6b89 to 0x7daa8bb, (0x3d32 bytes) 2024-09-21T15:01:07.916 INFO:teuthology.orchestra.run.smithi032.stdout:5768 clone 37 order 19 su 4096 sc 7 2024-09-21T15:01:07.916 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:01:07.914+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:01:07.917 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T15:01:08.016 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:01:08.014+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:01:08.030 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:01:08.031 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:01:08.031 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T15:01:08.031 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:01:08.040 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T15:01:08.117 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:01:08.110+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:01:08.152 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:01:08.152 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:01:08.152 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T15:01:08.152 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T15:01:08.152 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:01:08.162 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T15:01:08.218 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:01:08.218+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:01:08.248 INFO:teuthology.orchestra.run.smithi032.stderr:nodeep-scrub is set 2024-09-21T15:01:08.281 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:01:08.281 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T15:01:08.281 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T15:01:08.281 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T15:01:08.281 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.281 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:97677 3.15 e122/24 3:a8e72133:::rbd_data.1136bfc79a70.0000000000000005:head v 122'567, mlcod=120'565)", 2024-09-21T15:01:08.281 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.320753+0000", 2024-09-21T15:01:08.281 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 62.955922461999997, 2024-09-21T15:01:08.281 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070639375000000004, 2024-09-21T15:01:08.281 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:08.281 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:08.281 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:08.281 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.282 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:08.282 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.320753+0000", 2024-09-21T15:01:08.282 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:08.282 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.282 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.282 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:08.282 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.320753+0000", 2024-09-21T15:01:08.282 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:08.282 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.282 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.282 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:08.282 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.320753+0000", 2024-09-21T15:01:08.282 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:08.282 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.282 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.283 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:08.283 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.375452+0000", 2024-09-21T15:01:08.283 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054698945999999998 2024-09-21T15:01:08.283 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.283 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.283 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:08.283 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.375476+0000", 2024-09-21T15:01:08.283 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4380999999999999e-05 2024-09-21T15:01:08.283 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.283 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.283 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:08.283 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.375485+0000", 2024-09-21T15:01:08.283 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0049999999999993e-06 2024-09-21T15:01:08.283 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.284 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.284 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:08.284 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.375552+0000", 2024-09-21T15:01:08.284 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6550999999999994e-05 2024-09-21T15:01:08.284 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.284 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.284 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:08.284 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.375578+0000", 2024-09-21T15:01:08.284 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6196000000000001e-05 2024-09-21T15:01:08.284 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.284 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.284 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:01:08.284 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.391319+0000", 2024-09-21T15:01:08.285 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015741271000000001 2024-09-21T15:01:08.285 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.285 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.285 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:08.285 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.391393+0000", 2024-09-21T15:01:08.285 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3287000000000005e-05 2024-09-21T15:01:08.285 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:08.285 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:08.285 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:08.285 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.285 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.285 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:97672 3.4 e122/51 3:26046d29:::rbd_data.1136bfc79a70.0000000000000001:head v 122'256, mlcod=120'254)", 2024-09-21T15:01:08.285 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.331696+0000", 2024-09-21T15:01:08.286 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 62.944979299000003, 2024-09-21T15:01:08.286 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041222507999999998, 2024-09-21T15:01:08.286 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:08.286 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:08.286 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:08.286 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.286 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:08.286 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.331696+0000", 2024-09-21T15:01:08.286 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:08.286 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.286 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.286 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:08.286 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.331696+0000", 2024-09-21T15:01:08.286 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:08.287 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.287 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.287 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:08.287 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.331695+0000", 2024-09-21T15:01:08.287 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:08.287 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.287 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.287 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:08.287 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.340175+0000", 2024-09-21T15:01:08.287 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0084799520000000007 2024-09-21T15:01:08.287 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.287 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.287 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:08.287 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.340208+0000", 2024-09-21T15:01:08.288 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2997999999999998e-05 2024-09-21T15:01:08.288 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.288 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.288 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:08.289 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.340223+0000", 2024-09-21T15:01:08.289 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5330999999999999e-05 2024-09-21T15:01:08.289 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.290 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.290 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:08.290 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.340305+0000", 2024-09-21T15:01:08.290 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2095999999999995e-05 2024-09-21T15:01:08.290 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.290 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.290 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:08.290 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.340337+0000", 2024-09-21T15:01:08.290 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2097000000000001e-05 2024-09-21T15:01:08.290 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.290 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.290 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:01:08.290 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.372863+0000", 2024-09-21T15:01:08.291 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.032525923999999998 2024-09-21T15:01:08.291 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.291 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.291 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:08.291 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.372919+0000", 2024-09-21T15:01:08.291 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5729000000000001e-05 2024-09-21T15:01:08.291 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:08.291 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:08.291 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:08.291 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.291 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.291 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:116182 3.12 3:4efbaa3f:::rbd_data.1136b1194d20.0000000000000006:58 [read 6242304~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:08.291 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.712538+0000", 2024-09-21T15:01:08.291 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 38.564137307999999, 2024-09-21T15:01:08.292 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034702127999999999, 2024-09-21T15:01:08.292 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:08.292 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:08.292 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:08.292 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:08.292 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:08.292 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116182 2024-09-21T15:01:08.292 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.292 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:08.292 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.292 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:08.292 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712538+0000", 2024-09-21T15:01:08.292 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:08.292 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.293 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.293 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:08.293 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712538+0000", 2024-09-21T15:01:08.293 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:08.293 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.293 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.293 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:08.293 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712538+0000", 2024-09-21T15:01:08.293 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:08.293 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.293 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.293 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:08.293 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712541+0000", 2024-09-21T15:01:08.294 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5330000000000001e-06 2024-09-21T15:01:08.294 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.294 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.294 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:08.294 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712547+0000", 2024-09-21T15:01:08.294 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1109999999999996e-06 2024-09-21T15:01:08.294 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.294 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.294 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:08.294 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712551+0000", 2024-09-21T15:01:08.294 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.685e-06 2024-09-21T15:01:08.294 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.294 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.294 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:08.295 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.747067+0000", 2024-09-21T15:01:08.295 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034516181 2024-09-21T15:01:08.295 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.295 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.295 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:08.295 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.747152+0000", 2024-09-21T15:01:08.295 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4075999999999999e-05 2024-09-21T15:01:08.295 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.295 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.295 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:08.295 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.747240+0000", 2024-09-21T15:01:08.295 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8924000000000004e-05 2024-09-21T15:01:08.295 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:08.295 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:08.295 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:08.296 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.296 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.296 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:118701 3.12 3:4efbaa3f:::rbd_data.1136b1194d20.0000000000000006:58 [read 10412032~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:08.296 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.902645+0000", 2024-09-21T15:01:08.296 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 38.374030933999997, 2024-09-21T15:01:08.296 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034600791999999998, 2024-09-21T15:01:08.296 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:08.296 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:08.296 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:08.296 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:08.296 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:08.296 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 118701 2024-09-21T15:01:08.296 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.296 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:08.296 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.297 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:08.297 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902645+0000", 2024-09-21T15:01:08.297 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:08.297 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.297 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.297 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:08.297 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902645+0000", 2024-09-21T15:01:08.297 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:08.297 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.297 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.297 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:08.297 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902644+0000", 2024-09-21T15:01:08.297 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:08.297 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.298 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.298 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:08.298 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902646+0000", 2024-09-21T15:01:08.298 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6169999999999999e-06 2024-09-21T15:01:08.298 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.298 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.298 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:08.298 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902649+0000", 2024-09-21T15:01:08.298 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9450000000000002e-06 2024-09-21T15:01:08.298 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.298 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.298 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:08.298 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902652+0000", 2024-09-21T15:01:08.298 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7889999999999999e-06 2024-09-21T15:01:08.299 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.299 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.299 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:08.299 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.937086+0000", 2024-09-21T15:01:08.299 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034433832999999997 2024-09-21T15:01:08.299 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.299 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.299 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:08.299 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.937152+0000", 2024-09-21T15:01:08.299 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5934999999999998e-05 2024-09-21T15:01:08.299 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.299 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.299 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:08.299 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.937246+0000", 2024-09-21T15:01:08.299 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3923000000000002e-05 2024-09-21T15:01:08.300 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:08.300 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:08.300 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:08.300 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.300 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.300 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139784 3.1 3:862e0e65:::rbd_data.113664780c85.0000000000000063:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:08.300 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.222486+0000", 2024-09-21T15:01:08.300 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 6.0541894410000001, 2024-09-21T15:01:08.300 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0141793059999999, 2024-09-21T15:01:08.300 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:08.300 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:08.300 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:08.300 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:08.300 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:08.300 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139784 2024-09-21T15:01:08.300 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.301 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:08.301 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.301 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:08.301 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222486+0000", 2024-09-21T15:01:08.301 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:08.301 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.301 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.301 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:08.301 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222486+0000", 2024-09-21T15:01:08.301 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:08.301 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.301 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.301 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:08.301 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222484+0000", 2024-09-21T15:01:08.301 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-21T15:01:08.301 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.302 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.302 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:08.302 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222494+0000", 2024-09-21T15:01:08.302 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0676000000000001e-05 2024-09-21T15:01:08.302 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.302 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.302 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:08.302 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222517+0000", 2024-09-21T15:01:08.302 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2371e-05 2024-09-21T15:01:08.302 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.302 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.302 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:08.302 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222527+0000", 2024-09-21T15:01:08.302 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0461e-05 2024-09-21T15:01:08.303 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.303 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.303 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:08.303 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222625+0000", 2024-09-21T15:01:08.303 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.8160999999999997e-05 2024-09-21T15:01:08.303 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.303 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.303 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:08.303 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222641+0000", 2024-09-21T15:01:08.303 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5899000000000001e-05 2024-09-21T15:01:08.303 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.303 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.303 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:08.303 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.236479+0000", 2024-09-21T15:01:08.304 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0138377709999999 2024-09-21T15:01:08.304 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.304 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.304 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:08.304 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.236666+0000", 2024-09-21T15:01:08.304 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018648199999999999 2024-09-21T15:01:08.304 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:08.304 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:08.304 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:08.304 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.304 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.304 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139812 3.1 3:8407fb9b:::rbd_data.113664780c85.0000000000000127:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:08.304 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.222547+0000", 2024-09-21T15:01:08.304 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 6.0541285169999997, 2024-09-21T15:01:08.305 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.014796059, 2024-09-21T15:01:08.305 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:08.305 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:08.305 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:08.305 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:08.305 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:08.305 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139812 2024-09-21T15:01:08.305 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.305 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:08.305 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.305 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:08.305 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222547+0000", 2024-09-21T15:01:08.305 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:08.305 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.305 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.305 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:08.306 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222547+0000", 2024-09-21T15:01:08.306 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:08.306 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.306 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.306 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:08.306 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222546+0000", 2024-09-21T15:01:08.306 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:08.306 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.306 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.306 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:08.306 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222552+0000", 2024-09-21T15:01:08.306 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2739999999999998e-06 2024-09-21T15:01:08.306 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.306 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.306 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:08.306 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222578+0000", 2024-09-21T15:01:08.307 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6594999999999999e-05 2024-09-21T15:01:08.307 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.307 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.307 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:08.307 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222596+0000", 2024-09-21T15:01:08.307 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7838000000000001e-05 2024-09-21T15:01:08.307 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.307 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.307 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:08.307 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222701+0000", 2024-09-21T15:01:08.307 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000104714 2024-09-21T15:01:08.307 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.307 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.307 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:08.307 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222714+0000", 2024-09-21T15:01:08.308 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2998e-05 2024-09-21T15:01:08.308 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.308 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.308 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:08.308 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.237124+0000", 2024-09-21T15:01:08.308 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.014409806 2024-09-21T15:01:08.308 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.308 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.308 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:08.308 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.237343+0000", 2024-09-21T15:01:08.308 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021950499999999999 2024-09-21T15:01:08.308 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:08.308 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:08.308 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:08.308 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.308 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.309 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139826 3.1 3:854f6f9b:::rbd_data.113664780c85.0000000000000189:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:08.309 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.222611+0000", 2024-09-21T15:01:08.309 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 6.054064425, 2024-09-21T15:01:08.309 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.014916828, 2024-09-21T15:01:08.309 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:08.309 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:08.309 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:08.309 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:08.309 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:08.309 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139826 2024-09-21T15:01:08.309 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.309 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:08.309 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.309 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:08.309 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222611+0000", 2024-09-21T15:01:08.310 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:08.310 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.310 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.310 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:08.310 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222611+0000", 2024-09-21T15:01:08.310 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:08.310 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.310 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.310 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:08.310 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222610+0000", 2024-09-21T15:01:08.310 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:08.310 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.310 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.310 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:08.310 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222618+0000", 2024-09-21T15:01:08.311 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6690000000000006e-06 2024-09-21T15:01:08.311 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.311 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.311 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:08.311 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222632+0000", 2024-09-21T15:01:08.311 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4433999999999999e-05 2024-09-21T15:01:08.311 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.311 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.311 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:08.311 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222640+0000", 2024-09-21T15:01:08.311 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6260000000000003e-06 2024-09-21T15:01:08.311 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.311 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.311 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:08.311 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222758+0000", 2024-09-21T15:01:08.311 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011826499999999999 2024-09-21T15:01:08.312 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.312 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.312 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:08.312 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222776+0000", 2024-09-21T15:01:08.312 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7564999999999999e-05 2024-09-21T15:01:08.312 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.312 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.312 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:08.312 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.237402+0000", 2024-09-21T15:01:08.312 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0146261889999999 2024-09-21T15:01:08.312 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.312 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.312 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:08.312 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.237528+0000", 2024-09-21T15:01:08.312 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000126212 2024-09-21T15:01:08.312 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:08.313 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:08.313 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:08.313 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.313 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.313 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139864 3.1 3:86b14cb7:::rbd_data.113664780c85.00000000000000d5:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:08.313 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.223966+0000", 2024-09-21T15:01:08.313 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 6.0527096890000003, 2024-09-21T15:01:08.313 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.013859007, 2024-09-21T15:01:08.313 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:08.313 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:08.313 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:08.313 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:08.313 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:08.313 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139864 2024-09-21T15:01:08.313 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.314 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:08.314 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.314 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:08.314 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223966+0000", 2024-09-21T15:01:08.314 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:08.314 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.314 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.314 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:08.314 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223966+0000", 2024-09-21T15:01:08.314 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:08.314 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.314 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.314 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:08.314 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223965+0000", 2024-09-21T15:01:08.314 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:08.315 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.315 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.315 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:08.315 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223970+0000", 2024-09-21T15:01:08.315 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3769999999999998e-06 2024-09-21T15:01:08.315 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.315 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.315 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:08.315 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223979+0000", 2024-09-21T15:01:08.315 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5899999999999997e-06 2024-09-21T15:01:08.315 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.315 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.315 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:08.315 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223985+0000", 2024-09-21T15:01:08.315 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5790000000000003e-06 2024-09-21T15:01:08.315 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.316 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.316 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:08.316 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224073+0000", 2024-09-21T15:01:08.316 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7781000000000005e-05 2024-09-21T15:01:08.316 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.316 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.316 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:08.316 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224110+0000", 2024-09-21T15:01:08.316 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7060000000000001e-05 2024-09-21T15:01:08.316 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.316 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.316 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:08.316 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.237599+0000", 2024-09-21T15:01:08.316 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.01348953 2024-09-21T15:01:08.316 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.316 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.317 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:08.317 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.237825+0000", 2024-09-21T15:01:08.317 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000225758 2024-09-21T15:01:08.317 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:08.317 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:08.317 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:08.317 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.317 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.317 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139877 3.1 3:87277f92:::rbd_data.113664780c85.0000000000000136:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:08.317 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.224489+0000", 2024-09-21T15:01:08.317 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 6.0521868310000002, 2024-09-21T15:01:08.317 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0137916330000001, 2024-09-21T15:01:08.317 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:08.317 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:08.317 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:08.317 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:08.318 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:08.318 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139877 2024-09-21T15:01:08.318 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.318 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:08.318 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.318 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:08.318 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224489+0000", 2024-09-21T15:01:08.318 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:08.318 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.318 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.318 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:08.318 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224489+0000", 2024-09-21T15:01:08.318 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:08.318 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.318 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.319 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:08.319 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224488+0000", 2024-09-21T15:01:08.319 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:08.319 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.319 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.319 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:08.319 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224502+0000", 2024-09-21T15:01:08.319 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4194e-05 2024-09-21T15:01:08.319 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.319 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.319 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:08.319 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224522+0000", 2024-09-21T15:01:08.319 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0038e-05 2024-09-21T15:01:08.319 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.319 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.319 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:08.320 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224531+0000", 2024-09-21T15:01:08.320 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6900000000000004e-06 2024-09-21T15:01:08.320 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.320 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.320 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:08.320 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224621+0000", 2024-09-21T15:01:08.320 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9654000000000006e-05 2024-09-21T15:01:08.320 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.320 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.320 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:08.320 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224641+0000", 2024-09-21T15:01:08.320 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9375e-05 2024-09-21T15:01:08.320 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.320 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.320 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:08.321 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.238027+0000", 2024-09-21T15:01:08.321 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0133866840000001 2024-09-21T15:01:08.321 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.321 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.321 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:08.321 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.238280+0000", 2024-09-21T15:01:08.321 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000253281 2024-09-21T15:01:08.321 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:08.321 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:08.321 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:08.321 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.321 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.321 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139887 3.1 3:80f1e8f1:::rbd_data.113664780c85.000000000000017c:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:08.321 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.224549+0000", 2024-09-21T15:01:08.321 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 6.0521265409999998, 2024-09-21T15:01:08.321 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0139776060000001, 2024-09-21T15:01:08.322 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:08.322 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:08.322 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:08.322 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:08.322 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:08.322 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139887 2024-09-21T15:01:08.322 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.322 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:08.322 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.322 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:08.322 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224549+0000", 2024-09-21T15:01:08.322 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:08.322 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.322 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.322 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:08.323 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224549+0000", 2024-09-21T15:01:08.323 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:08.323 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.323 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.323 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:08.323 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224548+0000", 2024-09-21T15:01:08.323 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:08.323 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.323 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.323 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:08.323 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224554+0000", 2024-09-21T15:01:08.323 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4049999999999999e-06 2024-09-21T15:01:08.323 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.323 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.324 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:08.324 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224565+0000", 2024-09-21T15:01:08.324 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0879e-05 2024-09-21T15:01:08.324 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.324 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.324 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:08.324 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224572+0000", 2024-09-21T15:01:08.324 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0380000000000002e-06 2024-09-21T15:01:08.324 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.324 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.324 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:08.324 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224684+0000", 2024-09-21T15:01:08.324 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011229 2024-09-21T15:01:08.324 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.325 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.325 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:08.325 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224698+0000", 2024-09-21T15:01:08.325 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4448e-05 2024-09-21T15:01:08.325 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.325 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.325 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:08.325 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.238359+0000", 2024-09-21T15:01:08.325 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0136610660000001 2024-09-21T15:01:08.325 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.325 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.325 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:08.325 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.238527+0000", 2024-09-21T15:01:08.325 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016731100000000001 2024-09-21T15:01:08.325 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:08.325 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:08.326 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:08.326 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.326 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.326 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140077 3.1 3:85642daf:::rbd_data.113664780c85.0000000000000174:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:08.326 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.232797+0000", 2024-09-21T15:01:08.326 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 6.0438788260000003, 2024-09-21T15:01:08.326 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.006029955, 2024-09-21T15:01:08.326 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:08.326 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:08.326 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:08.326 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:08.326 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:08.326 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140077 2024-09-21T15:01:08.326 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.327 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:08.327 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.327 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:08.327 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232797+0000", 2024-09-21T15:01:08.327 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:08.327 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.327 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.327 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:08.327 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232797+0000", 2024-09-21T15:01:08.327 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:08.327 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.327 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.327 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:08.327 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232795+0000", 2024-09-21T15:01:08.327 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:08.327 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.328 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.328 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:08.328 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232804+0000", 2024-09-21T15:01:08.328 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9670000000000007e-06 2024-09-21T15:01:08.328 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.328 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.328 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:08.328 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232824+0000", 2024-09-21T15:01:08.328 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9238000000000001e-05 2024-09-21T15:01:08.328 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.328 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.328 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:08.328 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232843+0000", 2024-09-21T15:01:08.328 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9256e-05 2024-09-21T15:01:08.329 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.329 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.329 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:08.329 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232912+0000", 2024-09-21T15:01:08.329 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9589000000000001e-05 2024-09-21T15:01:08.329 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.329 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.329 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:08.329 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232925+0000", 2024-09-21T15:01:08.329 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2585999999999999e-05 2024-09-21T15:01:08.329 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.329 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.329 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:08.329 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.238591+0000", 2024-09-21T15:01:08.329 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0056659939999999 2024-09-21T15:01:08.329 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.330 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.330 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:08.330 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.238827+0000", 2024-09-21T15:01:08.330 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00023583999999999999 2024-09-21T15:01:08.330 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:08.330 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:08.330 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:08.330 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.330 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.330 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140091 3.1 3:8358f0db:::rbd_data.113664780c85.0000000000000018:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:08.330 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.233637+0000", 2024-09-21T15:01:08.330 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 6.0430385299999996, 2024-09-21T15:01:08.330 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.006272847, 2024-09-21T15:01:08.331 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:08.331 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:08.331 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:08.331 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:08.331 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:08.331 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140091 2024-09-21T15:01:08.331 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.331 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:08.331 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.331 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:08.331 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233637+0000", 2024-09-21T15:01:08.331 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:08.331 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.331 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.331 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:08.331 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233637+0000", 2024-09-21T15:01:08.332 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:08.332 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.332 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.332 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:08.332 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233636+0000", 2024-09-21T15:01:08.332 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:08.332 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.332 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.332 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:08.332 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233644+0000", 2024-09-21T15:01:08.332 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1380000000000007e-06 2024-09-21T15:01:08.332 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.332 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.332 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:08.332 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233662+0000", 2024-09-21T15:01:08.332 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8224999999999999e-05 2024-09-21T15:01:08.333 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.333 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.333 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:08.333 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233674+0000", 2024-09-21T15:01:08.333 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2081e-05 2024-09-21T15:01:08.333 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.333 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.333 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:08.333 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233750+0000", 2024-09-21T15:01:08.333 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6106000000000006e-05 2024-09-21T15:01:08.333 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.333 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.333 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:08.333 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233768+0000", 2024-09-21T15:01:08.333 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7285000000000001e-05 2024-09-21T15:01:08.334 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.334 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.334 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:08.334 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.239196+0000", 2024-09-21T15:01:08.334 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.005428223 2024-09-21T15:01:08.334 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.334 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.334 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:08.334 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.239528+0000", 2024-09-21T15:01:08.334 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00033262199999999997 2024-09-21T15:01:08.334 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.334 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.334 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:08.334 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.239910+0000", 2024-09-21T15:01:08.335 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00038151100000000001 2024-09-21T15:01:08.335 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:08.335 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:08.335 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:08.335 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.335 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.335 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140109 3.1 3:8586e9d9:::rbd_data.113664780c85.0000000000000096:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:08.335 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.234171+0000", 2024-09-21T15:01:08.335 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 6.042504696, 2024-09-21T15:01:08.335 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.006019896, 2024-09-21T15:01:08.335 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:08.335 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:08.335 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:08.335 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:08.335 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:08.335 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140109 2024-09-21T15:01:08.336 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.336 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:08.336 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.336 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:08.336 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.234171+0000", 2024-09-21T15:01:08.336 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:08.336 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.336 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.336 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:08.336 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.234171+0000", 2024-09-21T15:01:08.336 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:08.336 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.336 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.336 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:08.336 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.234170+0000", 2024-09-21T15:01:08.336 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:08.337 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.337 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.337 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:08.337 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.234178+0000", 2024-09-21T15:01:08.337 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.464e-06 2024-09-21T15:01:08.337 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.337 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.337 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:08.337 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.234191+0000", 2024-09-21T15:01:08.337 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3489000000000001e-05 2024-09-21T15:01:08.337 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.337 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.337 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:08.337 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.234200+0000", 2024-09-21T15:01:08.337 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2779999999999999e-06 2024-09-21T15:01:08.337 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.338 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.338 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:08.338 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.234279+0000", 2024-09-21T15:01:08.338 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8813000000000006e-05 2024-09-21T15:01:08.338 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.338 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.338 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:08.338 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.234298+0000", 2024-09-21T15:01:08.338 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8947999999999999e-05 2024-09-21T15:01:08.338 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.338 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.338 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:08.338 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.239974+0000", 2024-09-21T15:01:08.338 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0056760259999999 2024-09-21T15:01:08.338 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.339 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.339 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:08.339 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.240191+0000", 2024-09-21T15:01:08.339 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000216708 2024-09-21T15:01:08.339 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:08.339 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:08.339 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:08.339 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.339 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.339 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140149 3.1 3:86b1e433:::rbd_data.113664780c85.00000000000001ae:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:08.339 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.235545+0000", 2024-09-21T15:01:08.339 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 6.0411304110000001, 2024-09-21T15:01:08.339 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0049193430000001, 2024-09-21T15:01:08.339 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:08.339 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:08.339 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:08.340 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:08.340 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:08.340 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140149 2024-09-21T15:01:08.340 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.340 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:08.340 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.340 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:08.340 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235545+0000", 2024-09-21T15:01:08.340 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:08.340 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.340 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.340 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:08.340 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235545+0000", 2024-09-21T15:01:08.340 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:08.340 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.340 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.341 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:08.341 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235544+0000", 2024-09-21T15:01:08.341 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:08.341 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.341 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.341 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:08.341 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235554+0000", 2024-09-21T15:01:08.341 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.8870000000000004e-06 2024-09-21T15:01:08.341 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.341 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.341 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:08.341 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235577+0000", 2024-09-21T15:01:08.341 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2946999999999999e-05 2024-09-21T15:01:08.341 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.341 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.341 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:08.342 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235587+0000", 2024-09-21T15:01:08.342 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0326000000000001e-05 2024-09-21T15:01:08.342 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.342 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.342 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:08.342 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235658+0000", 2024-09-21T15:01:08.342 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1360999999999997e-05 2024-09-21T15:01:08.342 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.342 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.342 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:08.342 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235670+0000", 2024-09-21T15:01:08.342 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1735e-05 2024-09-21T15:01:08.342 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.342 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.342 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:08.343 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.240246+0000", 2024-09-21T15:01:08.343 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0045757790000001 2024-09-21T15:01:08.343 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.343 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.343 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:08.343 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.240465+0000", 2024-09-21T15:01:08.343 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021870799999999999 2024-09-21T15:01:08.343 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:08.343 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:08.343 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:08.343 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.343 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.343 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140232 3.1 3:862e0e65:::rbd_data.113664780c85.0000000000000063:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:08.343 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.235760+0000", 2024-09-21T15:01:08.343 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 6.0409156929999996, 2024-09-21T15:01:08.343 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.004979812, 2024-09-21T15:01:08.344 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:08.344 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:08.344 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:08.344 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:08.344 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:08.344 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140232 2024-09-21T15:01:08.344 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.344 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:08.344 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.344 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:08.344 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235760+0000", 2024-09-21T15:01:08.344 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:08.344 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.344 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.344 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:08.345 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235760+0000", 2024-09-21T15:01:08.345 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:08.345 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.345 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.345 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:08.345 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235759+0000", 2024-09-21T15:01:08.345 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:08.345 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.345 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.345 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:08.345 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235777+0000", 2024-09-21T15:01:08.345 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8215000000000001e-05 2024-09-21T15:01:08.345 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.345 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.345 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:08.345 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235793+0000", 2024-09-21T15:01:08.346 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6461e-05 2024-09-21T15:01:08.346 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.346 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.346 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:08.346 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235803+0000", 2024-09-21T15:01:08.346 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5300000000000002e-06 2024-09-21T15:01:08.346 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.346 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.346 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:08.346 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235875+0000", 2024-09-21T15:01:08.346 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2113000000000006e-05 2024-09-21T15:01:08.346 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.346 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.346 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:08.346 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235887+0000", 2024-09-21T15:01:08.346 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2160999999999999e-05 2024-09-21T15:01:08.347 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.347 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.347 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:08.347 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.240521+0000", 2024-09-21T15:01:08.347 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.004633428 2024-09-21T15:01:08.347 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.347 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.347 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:08.347 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.240740+0000", 2024-09-21T15:01:08.347 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000219228 2024-09-21T15:01:08.347 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:08.347 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:08.347 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:08.347 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.347 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.348 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140260 3.1 3:8407fb9b:::rbd_data.113664780c85.0000000000000127:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:08.348 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.235839+0000", 2024-09-21T15:01:08.348 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 6.0408365240000004, 2024-09-21T15:01:08.348 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.005157981, 2024-09-21T15:01:08.348 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:08.348 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:08.348 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:08.348 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:08.348 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:08.348 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140260 2024-09-21T15:01:08.348 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.348 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:08.348 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.348 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:08.348 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235839+0000", 2024-09-21T15:01:08.348 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:08.349 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.349 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.349 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:08.349 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235839+0000", 2024-09-21T15:01:08.349 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:08.349 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.349 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.349 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:08.349 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235838+0000", 2024-09-21T15:01:08.349 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:08.349 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.349 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.349 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:08.349 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235843+0000", 2024-09-21T15:01:08.349 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.481e-06 2024-09-21T15:01:08.349 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.350 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.350 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:08.350 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235849+0000", 2024-09-21T15:01:08.350 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4880000000000004e-06 2024-09-21T15:01:08.350 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.350 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.350 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:08.350 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235856+0000", 2024-09-21T15:01:08.350 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7649999999999996e-06 2024-09-21T15:01:08.350 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.350 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.350 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:08.350 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235933+0000", 2024-09-21T15:01:08.350 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6987e-05 2024-09-21T15:01:08.350 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.351 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.351 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:08.351 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235949+0000", 2024-09-21T15:01:08.351 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6696000000000001e-05 2024-09-21T15:01:08.351 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.351 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.351 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:08.351 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.240823+0000", 2024-09-21T15:01:08.351 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.004873259 2024-09-21T15:01:08.351 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.351 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.351 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:08.351 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.240997+0000", 2024-09-21T15:01:08.351 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000174365 2024-09-21T15:01:08.351 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:08.352 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:08.352 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:08.352 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.352 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.352 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140274 3.1 3:854f6f9b:::rbd_data.113664780c85.0000000000000189:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:08.352 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.235884+0000", 2024-09-21T15:01:08.352 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 6.0407915760000002, 2024-09-21T15:01:08.352 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.005397366, 2024-09-21T15:01:08.352 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:08.352 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:08.352 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:08.352 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:08.352 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:08.352 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140274 2024-09-21T15:01:08.352 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.353 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:08.353 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.353 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:08.353 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235884+0000", 2024-09-21T15:01:08.353 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:08.353 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.353 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.353 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:08.353 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235884+0000", 2024-09-21T15:01:08.353 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:08.353 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.353 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.353 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:08.353 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235883+0000", 2024-09-21T15:01:08.353 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:08.353 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.354 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.354 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:08.354 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235891+0000", 2024-09-21T15:01:08.354 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9740000000000002e-06 2024-09-21T15:01:08.354 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.354 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.354 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:08.354 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235900+0000", 2024-09-21T15:01:08.354 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6659999999999997e-06 2024-09-21T15:01:08.354 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.354 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.354 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:08.354 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235909+0000", 2024-09-21T15:01:08.354 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3400000000000004e-06 2024-09-21T15:01:08.354 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.355 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.355 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:08.355 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236012+0000", 2024-09-21T15:01:08.355 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010240500000000001 2024-09-21T15:01:08.355 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.355 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.355 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:08.355 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236027+0000", 2024-09-21T15:01:08.355 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5418e-05 2024-09-21T15:01:08.355 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.355 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.355 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:08.355 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.241068+0000", 2024-09-21T15:01:08.355 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0050410729999999 2024-09-21T15:01:08.355 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.355 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.356 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:08.356 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.241281+0000", 2024-09-21T15:01:08.356 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021321900000000001 2024-09-21T15:01:08.356 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:08.356 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:08.356 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:08.356 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.356 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.356 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140312 3.1 3:86b14cb7:::rbd_data.113664780c85.00000000000000d5:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:08.356 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.236769+0000", 2024-09-21T15:01:08.356 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 6.0399071119999999, 2024-09-21T15:01:08.356 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.004717198, 2024-09-21T15:01:08.356 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:08.356 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:08.356 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:08.357 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:08.357 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:08.357 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140312 2024-09-21T15:01:08.357 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.357 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:08.357 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.357 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:08.357 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236769+0000", 2024-09-21T15:01:08.357 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:08.357 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.357 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.357 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:08.357 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236769+0000", 2024-09-21T15:01:08.357 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:08.357 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.357 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.358 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:08.358 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236768+0000", 2024-09-21T15:01:08.358 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:08.358 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.358 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.358 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:08.358 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236771+0000", 2024-09-21T15:01:08.358 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.331e-06 2024-09-21T15:01:08.358 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.358 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.358 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:08.358 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236777+0000", 2024-09-21T15:01:08.358 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6949999999999997e-06 2024-09-21T15:01:08.358 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.358 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.359 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:08.359 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236787+0000", 2024-09-21T15:01:08.359 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6700000000000006e-06 2024-09-21T15:01:08.359 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.359 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.359 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:08.359 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236846+0000", 2024-09-21T15:01:08.359 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.944e-05 2024-09-21T15:01:08.359 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.359 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.359 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:08.359 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236861+0000", 2024-09-21T15:01:08.359 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4611e-05 2024-09-21T15:01:08.359 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.359 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.360 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:08.360 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.241349+0000", 2024-09-21T15:01:08.360 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0044879250000001 2024-09-21T15:01:08.360 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.360 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.360 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:08.360 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.241486+0000", 2024-09-21T15:01:08.360 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013713000000000001 2024-09-21T15:01:08.360 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:08.360 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:08.360 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:08.360 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.360 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.360 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140325 3.1 3:87277f92:::rbd_data.113664780c85.0000000000000136:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:08.360 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.239351+0000", 2024-09-21T15:01:08.361 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 6.0373249729999996, 2024-09-21T15:01:08.361 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0024334690000001, 2024-09-21T15:01:08.361 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:08.361 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:08.361 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:08.361 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:08.361 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:08.361 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140325 2024-09-21T15:01:08.361 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.361 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:08.361 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.361 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:08.361 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239351+0000", 2024-09-21T15:01:08.361 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:08.361 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.361 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.362 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:08.362 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239351+0000", 2024-09-21T15:01:08.362 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:08.362 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.362 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.362 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:08.362 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239349+0000", 2024-09-21T15:01:08.362 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:08.362 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.362 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.362 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:08.362 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239358+0000", 2024-09-21T15:01:08.362 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2830000000000001e-06 2024-09-21T15:01:08.362 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.362 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.363 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:08.363 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239379+0000", 2024-09-21T15:01:08.363 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1780000000000002e-05 2024-09-21T15:01:08.363 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.363 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.363 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:08.363 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239389+0000", 2024-09-21T15:01:08.363 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0056e-05 2024-09-21T15:01:08.363 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.363 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.363 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:08.363 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239484+0000", 2024-09-21T15:01:08.363 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4611000000000006e-05 2024-09-21T15:01:08.363 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.364 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.364 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:08.364 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239497+0000", 2024-09-21T15:01:08.364 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2909e-05 2024-09-21T15:01:08.364 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.364 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.364 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:08.364 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.241559+0000", 2024-09-21T15:01:08.364 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0020622459999999 2024-09-21T15:01:08.364 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.364 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.364 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:08.364 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.241784+0000", 2024-09-21T15:01:08.364 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000225009 2024-09-21T15:01:08.364 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:08.364 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:08.365 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:08.365 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.365 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.365 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140335 3.1 3:80f1e8f1:::rbd_data.113664780c85.000000000000017c:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:08.365 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.239409+0000", 2024-09-21T15:01:08.365 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 6.0372662540000004, 2024-09-21T15:01:08.365 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0025803150000001, 2024-09-21T15:01:08.365 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:08.365 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:08.365 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:08.365 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:08.365 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:08.365 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140335 2024-09-21T15:01:08.365 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.365 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:08.365 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.366 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:08.366 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239409+0000", 2024-09-21T15:01:08.366 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:08.366 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.366 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.366 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:08.366 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239409+0000", 2024-09-21T15:01:08.366 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:08.366 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.366 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.366 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:08.366 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239409+0000", 2024-09-21T15:01:08.366 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:08.366 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.366 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.367 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:08.367 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239413+0000", 2024-09-21T15:01:08.367 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7620000000000003e-06 2024-09-21T15:01:08.367 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.367 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.367 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:08.367 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239436+0000", 2024-09-21T15:01:08.367 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2889000000000001e-05 2024-09-21T15:01:08.367 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.367 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.367 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:08.367 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239444+0000", 2024-09-21T15:01:08.367 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6809999999999998e-06 2024-09-21T15:01:08.367 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.367 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.368 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:08.368 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239542+0000", 2024-09-21T15:01:08.368 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7868999999999994e-05 2024-09-21T15:01:08.368 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.368 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.368 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:08.368 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239558+0000", 2024-09-21T15:01:08.368 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6257e-05 2024-09-21T15:01:08.368 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.368 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.368 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:08.368 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.241847+0000", 2024-09-21T15:01:08.368 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.002288533 2024-09-21T15:01:08.368 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:08.369 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:08.369 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:08.369 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.241990+0000", 2024-09-21T15:01:08.369 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014308300000000001 2024-09-21T15:01:08.369 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:08.369 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:08.369 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:08.369 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:08.369 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:08.369 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:01:08.370 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T15:01:08.371 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T15:01:08.371 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:01:08.366+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:01:08.472 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:01:08.466+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:01:08.489 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:01:08.490 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:01:08.490 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T15:01:08.490 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:01:08.500 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T15:01:08.526 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T15:01:08.549 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T15:01:08.571 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T15:01:08.572 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:01:08.569+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:01:08.590 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T15:01:08.609 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:01:08.610 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:01:08.610 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T15:01:08.610 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T15:01:08.610 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:01:08.612 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-21T15:01:08.619 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T15:01:08.635 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T15:01:08.654 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T15:01:08.673 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:01:08.669+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:01:08.675 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T15:01:08.678 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:01:08.678 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:01:08.678 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:01:08.678 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:01:08.679 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:01:08.679 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:01:08.679 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:01:08.679 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:01:08.730 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:08.730 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:08.740 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T15:01:08.787 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:01:08.785+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:01:08.855 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:08.855 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:08.865 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T15:01:08.874 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:01:08.869+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:01:08.975 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:01:08.973+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:01:08.976 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:08.976 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:08.986 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T15:01:09.075 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:01:09.070+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:01:09.096 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:09.096 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:09.106 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T15:01:09.176 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:01:09.173+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:01:09.213 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:09.213 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:09.223 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T15:01:09.277 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:01:09.273+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:01:09.367 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:09.367 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:09.377 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T15:01:09.392 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:01:09.374+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:01:09.477 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:01:09.473+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:01:09.577 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:09.578 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:09.578 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:01:09.573+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:01:09.594 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T15:01:09.678 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:01:09.677+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:01:09.779 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:01:09.777+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:01:09.809 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:09.809 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:09.822 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T15:01:09.879 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:01:09.877+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:01:09.944 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:09.944 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:09.954 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T15:01:09.980 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:01:09.974+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:01:10.073 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:10.073 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:10.081 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:01:10.077+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:01:10.084 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T15:01:10.182 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:01:10.178+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:01:10.197 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:10.197 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:10.208 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T15:01:10.264 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T15:01:10.283 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:01:10.278+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:01:10.378 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:10.378 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:10.384 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:01:10.381+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:01:10.390 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T15:01:10.484 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:01:10.481+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:01:10.559 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:10.560 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:10.570 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T15:01:10.586 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:01:10.582+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:01:10.682 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:10.682 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:10.686 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:01:10.682+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:01:10.692 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T15:01:10.693 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T15:01:10.757 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T15:01:10.787 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:01:10.782+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:01:10.804 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:10.804 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:10.813 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T15:01:10.832 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T15:01:10.856 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T15:01:10.878 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T15:01:10.888 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:01:10.885+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:01:10.901 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T15:01:10.922 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-21T15:01:10.925 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:01:10.925 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T15:01:10.926 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T15:01:10.926 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T15:01:10.926 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.926 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:63820 3.19 3.626bc079 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-21T15:01:10.926 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:27.241293+0000", 2024-09-21T15:01:10.926 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 103.679999479, 2024-09-21T15:01:10.926 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.98723808199999996, 2024-09-21T15:01:10.926 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:10.926 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:10.926 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:10.926 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:10.926 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:10.926 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 63820 2024-09-21T15:01:10.926 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.927 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:10.927 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.927 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:10.927 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241293+0000", 2024-09-21T15:01:10.927 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:10.927 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.927 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.927 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:10.927 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241293+0000", 2024-09-21T15:01:10.927 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:10.927 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.927 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.927 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:10.927 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241291+0000", 2024-09-21T15:01:10.928 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:01:10.928 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.928 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.928 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:10.928 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241299+0000", 2024-09-21T15:01:10.928 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1529999999999997e-06 2024-09-21T15:01:10.928 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.928 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.928 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:10.928 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241317+0000", 2024-09-21T15:01:10.928 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7764000000000001e-05 2024-09-21T15:01:10.928 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.928 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.928 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:10.928 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241328+0000", 2024-09-21T15:01:10.929 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1245e-05 2024-09-21T15:01:10.929 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.929 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.929 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:10.929 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241407+0000", 2024-09-21T15:01:10.929 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8762000000000002e-05 2024-09-21T15:01:10.929 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.929 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.929 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:10.929 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241418+0000", 2024-09-21T15:01:10.929 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1695000000000001e-05 2024-09-21T15:01:10.929 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.929 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.929 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:10.929 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:28.228510+0000", 2024-09-21T15:01:10.929 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.98709166100000001 2024-09-21T15:01:10.930 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.930 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.930 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:10.930 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:28.228531+0000", 2024-09-21T15:01:10.930 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0696e-05 2024-09-21T15:01:10.930 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:10.930 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:10.930 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:10.930 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.930 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.930 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:97734 3.19 3:9fcf754f:::rbd_data.1136bfc79a70.0000000000000009:head [write 0~4194304 [fadvise_sequential+fadvise_nocache],write 4784128~147456 [fadvise_sequential+fadvise_nocache],write 4997120~147456 [fadvise_sequential+fadvise_nocache],write 5210112~32768 [fadvise_sequential+fadvise_nocache],write 5292032~147456 [fadvise_sequential+fadvise_nocache],write 5505024~147456 [fadvise_sequential+fadvise_nocache],write 5718016~49152 [fadvise_sequential+fadvise_nocache],write 5799936~147456 [fadvise_sequential+fadvise_nocache],write 6012928~147456 [fadvise_sequential+fadvise_nocache],write 6225920~65536 [fadvise_sequential+fadvise_nocache],write 6307840~147456 [fadvise_sequential+fadvise_nocache],write 6520832~147456 [fadvise_sequential+fadvise_nocache],write 6733824~229376 [fadvise_sequential+fadvise_nocache],write 7028736~147456 [fadvise_sequential+fadvise_nocache],write 7241728~229376 [fadvise_sequential+fadvise_nocache],write 7536640~147456 [fadvise_sequential+fadvise_nocache],write 7749632~229376 [fadvise_sequential+fadvise_nocache],write 8077312~114688 [fadvise_sequential+fadvise_nocache],write 8290304~98304 [fadvise_sequential+fadvise_nocache],write 8404992~81920 [fadvise_sequential+fadvise_nocache],write 8617984~81920 [fadvise_sequential+fadvise_nocache],write 8830976~81920 [fadvise_sequential+fadvise_nocache],write 8962048~32768 [fadvise_sequential+fadvise_nocache],write 9175040~32768 [fadvise_sequential+fadvise_nocache],write 9388032~32768 [fadvise_sequential+fadvise_nocache],write 9469952~32768 [fadvise_sequential+fadvise_nocache],write 9682944~20092 [fadvise_sequential+fadvise_nocache],write 9961472~163840 [fadvise_sequential+fadvise_nocache],write 10141696~196608 [fadvise_sequential+fadvise_nocache],write 10354688~278528 [fadvise_sequential+fadvise_nocache],write 10649600~196608 [fadvise_sequential+fadvise_nocache],write 10862592~196608 [fadvise_sequential+fadvise_nocache],write 11075584~65536 [fadvise_sequential+fadvise_nocache],write 11173888~98304 [fadvise_sequential+fadvise_nocache],write 11288576~65536 [fadvise_sequential+fadvise_nocache],write 11386880~98304 [fadvise_sequential+fadvise_nocache],write 11501568~65536 [fadvise_sequential+fadvise_nocache],write 11583488~65536 [fadvise_sequential+fadvise_nocache],write 11681792~98304 [fadvise_sequential+fadvise_nocache],write 11796480~65536 [fadvise_sequential+fadvise_nocache],write 11894784~98304 [fadvise_sequential+fadvise_nocache],write 12009472~65536 [fadvise_sequential+fadvise_nocache],write 12189696~98304 [fadvise_sequential+fadvise_nocache],write 12402688~98304 [fadvise_sequential+fadvise_nocache],write 12697600~98304 [fadvise_sequential+fadvise_nocache],write 12910592~98304 [fadvise_sequential+fadvise_nocache],write 13205504~98304 [fadvise_sequential+fadvise_nocache],write 13418496~98304 [fadvise_sequential+fadvise_nocache],write 13713408~98304 [fadvise_sequential+fadvise_nocache],write 13926400~98304 [fadvise_sequential+fadvise_nocache],write 14139392~16384 [fadvise_sequential+fadvise_nocache],write 14680064~16384 [fadvise_sequential+fadvise_nocache],write 14827520~81920 [fadvise_sequential+fadvise_nocache],write 15040512~81920 [fadvise_sequential+fadvise_nocache],write 15335424~81920 [fadvise_sequential+fadvise_nocache],write 15548416~81920 [fadvise_sequential+fadvise_nocache],write 15843328~81920 [fadvise_sequential+fadvise_nocache],write 16056320~81920 [fadvise_sequential+fadvise_nocache],write 16351232~81920 [fadvise_sequential+fadvise_nocache],write 16564224~81920 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:01:10.930 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.354331+0000", 2024-09-21T15:01:10.930 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 65.566961431999999, 2024-09-21T15:01:10.930 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.086284870999999999, 2024-09-21T15:01:10.931 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:10.931 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:10.931 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:10.931 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:10.931 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:10.931 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97734 2024-09-21T15:01:10.931 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.931 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:10.931 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.931 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:10.931 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.354331+0000", 2024-09-21T15:01:10.931 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:10.931 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.932 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.932 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:10.932 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.354331+0000", 2024-09-21T15:01:10.932 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:10.932 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.932 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.932 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:10.932 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.354329+0000", 2024-09-21T15:01:10.932 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:01:10.932 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.932 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.932 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:10.932 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.370234+0000", 2024-09-21T15:01:10.932 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015905348999999999 2024-09-21T15:01:10.933 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.933 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.933 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:10.933 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.370286+0000", 2024-09-21T15:01:10.933 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1656999999999997e-05 2024-09-21T15:01:10.933 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.933 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.933 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:10.933 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.370315+0000", 2024-09-21T15:01:10.933 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9204e-05 2024-09-21T15:01:10.933 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.933 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.933 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:10.933 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.370395+0000", 2024-09-21T15:01:10.934 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0395999999999994e-05 2024-09-21T15:01:10.934 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.934 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.934 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:10.934 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.370604+0000", 2024-09-21T15:01:10.934 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000208684 2024-09-21T15:01:10.934 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.934 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.934 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 1", 2024-09-21T15:01:10.934 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.371701+0000", 2024-09-21T15:01:10.934 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0010973140000000001 2024-09-21T15:01:10.934 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.934 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.934 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:01:10.935 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.405485+0000", 2024-09-21T15:01:10.935 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.033783615000000003 2024-09-21T15:01:10.935 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.935 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.935 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:01:10.935 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.440211+0000", 2024-09-21T15:01:10.935 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034725757000000003 2024-09-21T15:01:10.935 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.935 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.935 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:01:10.935 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.440497+0000", 2024-09-21T15:01:10.935 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00028673099999999998 2024-09-21T15:01:10.935 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.935 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.936 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:10.936 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.440616+0000", 2024-09-21T15:01:10.936 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000118256 2024-09-21T15:01:10.936 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:10.936 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:10.936 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:10.936 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.936 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.936 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112831 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 1515520~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:10.936 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.406761+0000", 2024-09-21T15:01:10.936 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 41.514530733999997, 2024-09-21T15:01:10.936 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060265666000000002, 2024-09-21T15:01:10.936 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:10.936 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:10.937 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:10.937 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:10.937 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:10.937 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112831 2024-09-21T15:01:10.937 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.937 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:10.937 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.937 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:10.937 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406761+0000", 2024-09-21T15:01:10.937 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:10.937 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.937 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.937 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:10.937 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406761+0000", 2024-09-21T15:01:10.937 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:10.938 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.938 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.938 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:10.938 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406761+0000", 2024-09-21T15:01:10.938 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:10.938 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.938 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.938 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:10.938 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406764+0000", 2024-09-21T15:01:10.938 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0319999999999999e-06 2024-09-21T15:01:10.938 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.938 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.938 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:10.938 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406776+0000", 2024-09-21T15:01:10.939 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2357999999999999e-05 2024-09-21T15:01:10.939 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.939 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.939 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:10.939 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406784+0000", 2024-09-21T15:01:10.939 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6059999999999997e-06 2024-09-21T15:01:10.939 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.939 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.939 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:10.939 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.466595+0000", 2024-09-21T15:01:10.939 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059811062999999998 2024-09-21T15:01:10.939 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.939 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.939 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:10.940 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.466782+0000", 2024-09-21T15:01:10.940 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000186622 2024-09-21T15:01:10.940 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.940 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.940 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:10.940 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.467027+0000", 2024-09-21T15:01:10.940 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000245455 2024-09-21T15:01:10.940 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:10.940 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:10.940 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:10.940 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.940 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.940 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112832 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 1605632~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:10.940 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.406788+0000", 2024-09-21T15:01:10.940 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 41.514504144, 2024-09-21T15:01:10.941 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060452065999999999, 2024-09-21T15:01:10.941 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:10.941 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:10.941 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:10.941 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:10.941 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:10.941 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112832 2024-09-21T15:01:10.941 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.941 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:10.941 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.941 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:10.941 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406788+0000", 2024-09-21T15:01:10.941 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:10.941 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.941 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.942 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:10.942 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406788+0000", 2024-09-21T15:01:10.942 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:10.942 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.942 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.942 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:10.942 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406788+0000", 2024-09-21T15:01:10.942 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:10.942 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.942 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.942 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:10.942 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406798+0000", 2024-09-21T15:01:10.942 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0096e-05 2024-09-21T15:01:10.942 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.942 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.943 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:10.943 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406803+0000", 2024-09-21T15:01:10.943 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2020000000000003e-06 2024-09-21T15:01:10.943 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.943 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.943 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:10.943 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406808+0000", 2024-09-21T15:01:10.943 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8239999999999999e-06 2024-09-21T15:01:10.943 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.943 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.943 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:10.943 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.467049+0000", 2024-09-21T15:01:10.943 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060241754000000002 2024-09-21T15:01:10.943 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.943 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.945 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:10.945 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.467134+0000", 2024-09-21T15:01:10.945 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4216999999999997e-05 2024-09-21T15:01:10.945 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.945 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.946 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:10.946 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.467240+0000", 2024-09-21T15:01:10.946 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000106304 2024-09-21T15:01:10.946 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:10.946 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:10.946 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:10.946 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.946 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.946 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112843 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 2613248~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:10.946 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.410273+0000", 2024-09-21T15:01:10.946 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 41.51101955, 2024-09-21T15:01:10.946 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059854931, 2024-09-21T15:01:10.946 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:10.947 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:10.947 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:10.947 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:10.947 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:10.947 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112843 2024-09-21T15:01:10.947 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.947 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:10.947 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.947 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:10.947 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410273+0000", 2024-09-21T15:01:10.947 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:10.947 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.947 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.947 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:10.948 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410273+0000", 2024-09-21T15:01:10.948 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:10.948 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.948 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.948 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:10.948 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410272+0000", 2024-09-21T15:01:10.948 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:10.948 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.948 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.948 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:10.948 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410274+0000", 2024-09-21T15:01:10.948 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.905e-06 2024-09-21T15:01:10.948 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.948 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.949 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:10.949 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410283+0000", 2024-09-21T15:01:10.949 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3440000000000007e-06 2024-09-21T15:01:10.949 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.949 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.949 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:10.949 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410294+0000", 2024-09-21T15:01:10.949 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0271e-05 2024-09-21T15:01:10.949 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.949 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.949 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:10.949 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.469917+0000", 2024-09-21T15:01:10.949 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059623738000000003 2024-09-21T15:01:10.949 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.950 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.950 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:10.950 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470002+0000", 2024-09-21T15:01:10.950 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4270999999999993e-05 2024-09-21T15:01:10.950 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.950 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.950 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:10.950 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470127+0000", 2024-09-21T15:01:10.950 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000125711 2024-09-21T15:01:10.950 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:10.950 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:10.950 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:10.950 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.951 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.951 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112844 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 2703360~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:10.951 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.410326+0000", 2024-09-21T15:01:10.951 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 41.510965830000004, 2024-09-21T15:01:10.951 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060086056999999998, 2024-09-21T15:01:10.951 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:10.951 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:10.951 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:10.951 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:10.951 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:10.951 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112844 2024-09-21T15:01:10.951 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.951 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:10.951 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.952 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:10.952 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410326+0000", 2024-09-21T15:01:10.952 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:10.952 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.952 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.952 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:10.952 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410326+0000", 2024-09-21T15:01:10.952 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:10.952 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.952 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.952 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:10.952 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410326+0000", 2024-09-21T15:01:10.952 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:10.952 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.953 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.953 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:10.953 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410352+0000", 2024-09-21T15:01:10.953 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6872999999999998e-05 2024-09-21T15:01:10.953 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.953 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.953 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:10.953 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410374+0000", 2024-09-21T15:01:10.953 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1141e-05 2024-09-21T15:01:10.953 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.953 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.953 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:10.953 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410385+0000", 2024-09-21T15:01:10.954 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1456e-05 2024-09-21T15:01:10.954 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.954 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.954 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:10.954 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470174+0000", 2024-09-21T15:01:10.954 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059788574999999997 2024-09-21T15:01:10.954 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.954 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.954 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:10.954 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470277+0000", 2024-09-21T15:01:10.954 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000103838 2024-09-21T15:01:10.954 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.954 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.955 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:10.955 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470412+0000", 2024-09-21T15:01:10.955 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013489700000000001 2024-09-21T15:01:10.955 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:10.955 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:10.955 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:10.955 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.955 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.955 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112845 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 2793472~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:10.955 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.410394+0000", 2024-09-21T15:01:10.955 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 41.510898355999998, 2024-09-21T15:01:10.955 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060244173999999998, 2024-09-21T15:01:10.955 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:10.955 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:10.955 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:10.956 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:10.956 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:10.956 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112845 2024-09-21T15:01:10.956 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.956 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:10.956 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.956 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:10.956 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410394+0000", 2024-09-21T15:01:10.956 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:10.956 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.956 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.956 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:10.956 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410394+0000", 2024-09-21T15:01:10.956 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:10.956 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.956 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.957 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:10.957 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410393+0000", 2024-09-21T15:01:10.957 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:10.957 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.957 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.957 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:10.957 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410397+0000", 2024-09-21T15:01:10.957 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9430000000000002e-06 2024-09-21T15:01:10.957 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.957 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.957 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:10.957 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410406+0000", 2024-09-21T15:01:10.957 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.799e-06 2024-09-21T15:01:10.957 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.957 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.957 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:10.958 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410418+0000", 2024-09-21T15:01:10.958 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2225999999999999e-05 2024-09-21T15:01:10.958 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.958 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.958 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:10.958 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470444+0000", 2024-09-21T15:01:10.958 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060026311999999998 2024-09-21T15:01:10.958 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.958 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.958 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:10.958 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470535+0000", 2024-09-21T15:01:10.958 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0173999999999994e-05 2024-09-21T15:01:10.958 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.958 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.959 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:10.959 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470638+0000", 2024-09-21T15:01:10.959 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010333 2024-09-21T15:01:10.959 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:10.959 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:10.959 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:10.959 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.959 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.959 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112846 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 2883584~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:10.959 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.410425+0000", 2024-09-21T15:01:10.959 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 41.510867142999999, 2024-09-21T15:01:10.959 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060459247000000001, 2024-09-21T15:01:10.959 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:10.959 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:10.959 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:10.960 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:10.960 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:10.960 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112846 2024-09-21T15:01:10.960 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.960 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:10.960 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.960 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:10.960 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410425+0000", 2024-09-21T15:01:10.960 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:10.960 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.960 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.960 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:10.960 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410425+0000", 2024-09-21T15:01:10.960 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:10.961 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.961 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.961 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:10.961 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410424+0000", 2024-09-21T15:01:10.961 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:10.961 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.961 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.961 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:10.961 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410428+0000", 2024-09-21T15:01:10.961 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4709999999999999e-06 2024-09-21T15:01:10.961 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.961 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.961 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:10.961 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410446+0000", 2024-09-21T15:01:10.961 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8354000000000001e-05 2024-09-21T15:01:10.961 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.962 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.962 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:10.962 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410456+0000", 2024-09-21T15:01:10.962 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6709999999999994e-06 2024-09-21T15:01:10.962 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.962 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.962 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:10.962 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470661+0000", 2024-09-21T15:01:10.962 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060205303000000002 2024-09-21T15:01:10.962 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.962 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.962 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:10.962 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470756+0000", 2024-09-21T15:01:10.962 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4401999999999995e-05 2024-09-21T15:01:10.963 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.963 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.963 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:10.963 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470884+0000", 2024-09-21T15:01:10.963 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012860500000000001 2024-09-21T15:01:10.963 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:10.963 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:10.963 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:10.963 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.963 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.963 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112847 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 2973696~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:10.963 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.410462+0000", 2024-09-21T15:01:10.963 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 41.510830099000003, 2024-09-21T15:01:10.963 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060653836000000003, 2024-09-21T15:01:10.963 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:10.964 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:10.964 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:10.964 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:10.964 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:10.964 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112847 2024-09-21T15:01:10.964 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.964 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:10.964 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.964 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:10.964 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410462+0000", 2024-09-21T15:01:10.964 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:10.964 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.964 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.964 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:10.964 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410462+0000", 2024-09-21T15:01:10.965 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:10.965 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.965 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.965 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:10.965 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410461+0000", 2024-09-21T15:01:10.965 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:10.965 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.965 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.965 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:10.965 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410465+0000", 2024-09-21T15:01:10.965 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6899999999999998e-06 2024-09-21T15:01:10.965 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.965 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.965 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:10.965 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410474+0000", 2024-09-21T15:01:10.966 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4179999999999997e-06 2024-09-21T15:01:10.966 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.966 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.966 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:10.966 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410508+0000", 2024-09-21T15:01:10.966 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4282999999999997e-05 2024-09-21T15:01:10.966 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.966 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.966 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:10.966 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470922+0000", 2024-09-21T15:01:10.966 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060413820999999999 2024-09-21T15:01:10.966 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.966 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.966 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:10.967 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471008+0000", 2024-09-21T15:01:10.967 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6379999999999996e-05 2024-09-21T15:01:10.967 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.967 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.967 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:10.967 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471116+0000", 2024-09-21T15:01:10.967 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000107806 2024-09-21T15:01:10.967 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:10.967 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:10.967 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:10.967 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.967 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.967 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112848 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 3063808~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:10.967 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.410525+0000", 2024-09-21T15:01:10.967 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 41.510766816, 2024-09-21T15:01:10.968 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060880735999999998, 2024-09-21T15:01:10.968 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:10.968 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:10.968 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:10.968 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:10.968 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:10.968 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112848 2024-09-21T15:01:10.968 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.968 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:10.968 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.968 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:10.968 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410525+0000", 2024-09-21T15:01:10.968 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:10.968 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.968 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.969 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:10.969 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410525+0000", 2024-09-21T15:01:10.969 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:10.969 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.969 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.969 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:10.969 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410525+0000", 2024-09-21T15:01:10.969 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:10.969 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.969 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.969 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:10.969 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410529+0000", 2024-09-21T15:01:10.969 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1760000000000003e-06 2024-09-21T15:01:10.969 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.969 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.969 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:10.970 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410551+0000", 2024-09-21T15:01:10.970 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2099999999999998e-05 2024-09-21T15:01:10.970 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.970 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.970 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:10.970 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410571+0000", 2024-09-21T15:01:10.970 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0159000000000001e-05 2024-09-21T15:01:10.970 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.970 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.970 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:10.970 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471164+0000", 2024-09-21T15:01:10.970 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.06059287 2024-09-21T15:01:10.970 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.970 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.971 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:10.971 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471271+0000", 2024-09-21T15:01:10.971 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000106635 2024-09-21T15:01:10.971 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.971 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.971 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:10.971 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471406+0000", 2024-09-21T15:01:10.971 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000135399 2024-09-21T15:01:10.971 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:10.971 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:10.971 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:10.971 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.971 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.971 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112849 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 3153920~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:10.972 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.410579+0000", 2024-09-21T15:01:10.972 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 41.510713521, 2024-09-21T15:01:10.972 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.061057304999999999, 2024-09-21T15:01:10.972 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:10.972 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:10.972 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:10.972 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:10.972 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:10.972 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112849 2024-09-21T15:01:10.972 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.972 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:10.972 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.972 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:10.972 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410579+0000", 2024-09-21T15:01:10.973 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:10.973 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.973 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.973 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:10.973 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410579+0000", 2024-09-21T15:01:10.973 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:10.973 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.973 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.973 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:10.973 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410578+0000", 2024-09-21T15:01:10.973 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:10.973 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.973 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.973 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:10.973 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410582+0000", 2024-09-21T15:01:10.974 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.036e-06 2024-09-21T15:01:10.974 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.974 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.974 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:10.974 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410592+0000", 2024-09-21T15:01:10.974 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9850000000000001e-06 2024-09-21T15:01:10.974 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.974 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.974 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:10.974 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410604+0000", 2024-09-21T15:01:10.974 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2051e-05 2024-09-21T15:01:10.974 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.974 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.974 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:10.975 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471444+0000", 2024-09-21T15:01:10.975 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060839722999999998 2024-09-21T15:01:10.975 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.975 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.975 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:10.975 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471530+0000", 2024-09-21T15:01:10.975 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6359000000000004e-05 2024-09-21T15:01:10.975 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.975 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.975 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:10.975 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471636+0000", 2024-09-21T15:01:10.975 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010576700000000001 2024-09-21T15:01:10.975 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:10.975 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:10.976 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:10.976 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.976 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.976 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112850 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 3244032~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:10.976 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.410698+0000", 2024-09-21T15:01:10.976 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 41.510594179000002, 2024-09-21T15:01:10.976 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.061164118000000003, 2024-09-21T15:01:10.976 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:10.976 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:10.976 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:10.976 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:10.976 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:10.976 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112850 2024-09-21T15:01:10.977 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.977 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:10.977 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.977 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:10.977 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410698+0000", 2024-09-21T15:01:10.977 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:10.977 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.977 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.977 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:10.977 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410698+0000", 2024-09-21T15:01:10.977 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:10.977 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.977 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.977 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:10.978 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410610+0000", 2024-09-21T15:01:10.978 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999123 2024-09-21T15:01:10.978 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.978 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.978 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:10.978 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410707+0000", 2024-09-21T15:01:10.978 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7093e-05 2024-09-21T15:01:10.978 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.978 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.978 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:10.978 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410728+0000", 2024-09-21T15:01:10.978 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0275000000000002e-05 2024-09-21T15:01:10.978 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.978 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.979 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:10.979 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410740+0000", 2024-09-21T15:01:10.979 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2561999999999999e-05 2024-09-21T15:01:10.979 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.979 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.979 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:10.979 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471664+0000", 2024-09-21T15:01:10.979 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060923979000000003 2024-09-21T15:01:10.979 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.979 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.979 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:10.979 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471753+0000", 2024-09-21T15:01:10.979 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8265999999999998e-05 2024-09-21T15:01:10.979 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.979 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.980 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:10.980 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471862+0000", 2024-09-21T15:01:10.980 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000109474 2024-09-21T15:01:10.980 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:10.980 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:10.980 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:10.980 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.980 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.980 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112864 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 4521984~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:10.980 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.414789+0000", 2024-09-21T15:01:10.980 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 41.506503154000001, 2024-09-21T15:01:10.980 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059845903999999998, 2024-09-21T15:01:10.980 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:10.981 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:10.981 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:10.981 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:10.981 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:10.981 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112864 2024-09-21T15:01:10.981 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.981 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:10.981 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.981 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:10.981 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414789+0000", 2024-09-21T15:01:10.981 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:10.981 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.981 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.981 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:10.981 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414789+0000", 2024-09-21T15:01:10.982 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:10.982 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.982 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.982 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:10.982 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414788+0000", 2024-09-21T15:01:10.982 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:10.982 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.982 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.982 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:10.982 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414790+0000", 2024-09-21T15:01:10.982 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.995e-06 2024-09-21T15:01:10.982 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.982 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.982 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:10.983 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414801+0000", 2024-09-21T15:01:10.983 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.01e-05 2024-09-21T15:01:10.983 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.983 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.983 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:10.983 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414804+0000", 2024-09-21T15:01:10.983 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.563e-06 2024-09-21T15:01:10.983 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.983 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.983 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:10.983 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.474499+0000", 2024-09-21T15:01:10.983 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059695208 2024-09-21T15:01:10.983 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.983 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.983 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:10.984 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.474561+0000", 2024-09-21T15:01:10.984 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1672000000000002e-05 2024-09-21T15:01:10.984 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.984 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.984 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:10.984 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.474635+0000", 2024-09-21T15:01:10.984 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3792000000000001e-05 2024-09-21T15:01:10.984 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:10.984 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:10.984 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:10.984 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.984 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.984 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112865 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 4612096~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:10.984 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.414807+0000", 2024-09-21T15:01:10.985 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 41.506484757999999, 2024-09-21T15:01:10.985 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060003506999999998, 2024-09-21T15:01:10.985 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:10.985 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:10.985 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:10.985 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:10.985 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:10.985 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112865 2024-09-21T15:01:10.985 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.985 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:10.985 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.985 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:10.985 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414807+0000", 2024-09-21T15:01:10.985 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:10.985 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.986 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.986 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:10.986 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414807+0000", 2024-09-21T15:01:10.986 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:10.986 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.986 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.986 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:10.986 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414807+0000", 2024-09-21T15:01:10.986 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:10.986 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.986 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.986 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:10.986 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414809+0000", 2024-09-21T15:01:10.986 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0509999999999999e-06 2024-09-21T15:01:10.987 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.987 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.987 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:10.987 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414813+0000", 2024-09-21T15:01:10.987 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0960000000000003e-06 2024-09-21T15:01:10.987 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.987 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.987 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:10.987 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414817+0000", 2024-09-21T15:01:10.987 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.296e-06 2024-09-21T15:01:10.987 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.987 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.987 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:10.987 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.474671+0000", 2024-09-21T15:01:10.987 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059853663000000001 2024-09-21T15:01:10.988 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.988 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.988 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:10.988 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.474735+0000", 2024-09-21T15:01:10.988 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3653999999999997e-05 2024-09-21T15:01:10.988 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.988 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.988 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:10.988 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.474811+0000", 2024-09-21T15:01:10.988 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6074000000000004e-05 2024-09-21T15:01:10.988 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:10.988 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:10.988 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:10.988 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.988 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.989 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112866 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 4702208~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:10.989 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.414820+0000", 2024-09-21T15:01:10.989 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 41.506471838000003, 2024-09-21T15:01:10.989 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060161992999999997, 2024-09-21T15:01:10.989 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:10.989 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:10.989 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:10.989 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:10.989 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:10.989 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112866 2024-09-21T15:01:10.989 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.989 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:10.989 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.989 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:10.989 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414820+0000", 2024-09-21T15:01:10.990 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:10.990 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.990 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.990 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:10.990 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414820+0000", 2024-09-21T15:01:10.990 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:10.990 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.990 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.990 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:10.990 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414820+0000", 2024-09-21T15:01:10.990 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:10.990 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.990 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.991 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:10.991 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414822+0000", 2024-09-21T15:01:10.991 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8509999999999999e-06 2024-09-21T15:01:10.991 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.991 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.991 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:10.991 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414826+0000", 2024-09-21T15:01:10.991 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8190000000000002e-06 2024-09-21T15:01:10.991 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.991 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.991 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:10.991 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414837+0000", 2024-09-21T15:01:10.991 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1232e-05 2024-09-21T15:01:10.991 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.991 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.992 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:10.992 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.474846+0000", 2024-09-21T15:01:10.992 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060009506999999997 2024-09-21T15:01:10.992 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.992 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.992 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:10.992 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.474910+0000", 2024-09-21T15:01:10.992 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3918999999999996e-05 2024-09-21T15:01:10.992 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.992 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.992 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:10.992 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.474982+0000", 2024-09-21T15:01:10.992 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1958999999999993e-05 2024-09-21T15:01:10.993 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:10.993 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:10.993 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:10.993 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.993 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.993 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112867 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 4792320~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:10.993 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.414840+0000", 2024-09-21T15:01:10.993 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 41.506452342999999, 2024-09-21T15:01:10.993 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060308542999999999, 2024-09-21T15:01:10.993 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:10.993 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:10.993 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:10.993 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:10.993 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:10.994 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112867 2024-09-21T15:01:10.994 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.994 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:10.994 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.994 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:10.994 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414840+0000", 2024-09-21T15:01:10.994 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:10.994 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.994 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.994 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:10.994 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414840+0000", 2024-09-21T15:01:10.994 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:10.994 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.994 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.995 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:10.995 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414839+0000", 2024-09-21T15:01:10.995 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:10.995 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.995 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.995 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:10.995 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414841+0000", 2024-09-21T15:01:10.995 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8500000000000001e-06 2024-09-21T15:01:10.995 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.995 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.995 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:10.995 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414846+0000", 2024-09-21T15:01:10.995 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7700000000000001e-06 2024-09-21T15:01:10.995 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.995 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.996 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:10.996 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414850+0000", 2024-09-21T15:01:10.996 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1389999999999997e-06 2024-09-21T15:01:10.996 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.996 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.996 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:10.996 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.475016+0000", 2024-09-21T15:01:10.996 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.06016614 2024-09-21T15:01:10.996 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.996 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.996 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:10.996 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.475073+0000", 2024-09-21T15:01:10.996 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6295000000000002e-05 2024-09-21T15:01:10.996 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.997 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.997 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:10.997 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.475148+0000", 2024-09-21T15:01:10.997 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5647000000000003e-05 2024-09-21T15:01:10.997 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:10.997 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:10.997 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:10.997 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.997 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.997 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:113030 3.6 3:60420fc2:::rbd_data.1136b442e94a.0000000000000001:head [write 0~360448 [fadvise_sequential+fadvise_nocache],write 385024~573440 [fadvise_sequential+fadvise_nocache],write 1146880~573440 [fadvise_sequential+fadvise_nocache],write 1908736~188416 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:10.997 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.422420+0000", 2024-09-21T15:01:10.997 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 41.498871698000002, 2024-09-21T15:01:10.997 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.063268959999999999, 2024-09-21T15:01:10.997 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:10.997 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:10.998 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:10.998 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:10.998 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:10.998 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 113030 2024-09-21T15:01:10.998 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.998 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:10.998 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.998 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:10.998 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.422420+0000", 2024-09-21T15:01:10.998 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:10.998 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.998 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.998 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:10.998 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.422420+0000", 2024-09-21T15:01:10.999 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:10.999 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.999 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.999 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:10.999 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.422420+0000", 2024-09-21T15:01:10.999 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:10.999 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.999 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.999 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:10.999 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.424657+0000", 2024-09-21T15:01:10.999 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0022369780000000001 2024-09-21T15:01:10.999 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:10.999 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:10.999 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:11.000 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.424673+0000", 2024-09-21T15:01:11.000 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6141e-05 2024-09-21T15:01:11.000 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.000 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.000 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:11.000 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.424689+0000", 2024-09-21T15:01:11.000 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5897999999999999e-05 2024-09-21T15:01:11.000 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.000 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.000 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:11.000 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.478005+0000", 2024-09-21T15:01:11.000 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.05331636 2024-09-21T15:01:11.000 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.000 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.001 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:11.001 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.478133+0000", 2024-09-21T15:01:11.001 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012736799999999999 2024-09-21T15:01:11.001 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.001 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.001 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 7", 2024-09-21T15:01:11.001 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.478307+0000", 2024-09-21T15:01:11.001 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017491499999999999 2024-09-21T15:01:11.001 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.001 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.001 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:01:11.001 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.483611+0000", 2024-09-21T15:01:11.001 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0053039389999999997 2024-09-21T15:01:11.001 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.001 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.001 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:01:11.002 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.485553+0000", 2024-09-21T15:01:11.002 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0019418759999999999 2024-09-21T15:01:11.002 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.002 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.002 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:01:11.002 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.485645+0000", 2024-09-21T15:01:11.002 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2138000000000005e-05 2024-09-21T15:01:11.002 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.002 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.002 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:11.002 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.485689+0000", 2024-09-21T15:01:11.002 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3922999999999999e-05 2024-09-21T15:01:11.002 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.003 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:11.003 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.003 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.003 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.003 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:114290 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 6946816~8192 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:11.003 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.558087+0000", 2024-09-21T15:01:11.003 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 41.363204901000003, 2024-09-21T15:01:11.003 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059973091999999999, 2024-09-21T15:01:11.003 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:11.003 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:11.003 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:11.003 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:11.003 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:11.003 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 114290 2024-09-21T15:01:11.003 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.004 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:11.004 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.004 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:11.004 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.558087+0000", 2024-09-21T15:01:11.004 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:11.004 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.004 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.004 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:11.004 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.558087+0000", 2024-09-21T15:01:11.004 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:11.004 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.004 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.004 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:11.004 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.558087+0000", 2024-09-21T15:01:11.005 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:11.005 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.005 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.005 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:11.005 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.558092+0000", 2024-09-21T15:01:11.005 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2379999999999997e-06 2024-09-21T15:01:11.005 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.005 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.005 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:11.005 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.558095+0000", 2024-09-21T15:01:11.005 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.275e-06 2024-09-21T15:01:11.005 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.005 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.005 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:11.006 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.558099+0000", 2024-09-21T15:01:11.006 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6679999999999999e-06 2024-09-21T15:01:11.006 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.006 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.006 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:11.006 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.617757+0000", 2024-09-21T15:01:11.006 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059657531999999999 2024-09-21T15:01:11.006 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.006 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.006 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:11.006 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.617891+0000", 2024-09-21T15:01:11.006 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000134478 2024-09-21T15:01:11.006 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.007 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.007 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:11.007 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.618060+0000", 2024-09-21T15:01:11.007 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016925100000000001 2024-09-21T15:01:11.007 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.007 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:11.007 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.007 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.007 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.007 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:114312 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 8945664~8192 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:11.007 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.562396+0000", 2024-09-21T15:01:11.007 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 41.358896553000001, 2024-09-21T15:01:11.007 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059832396000000003, 2024-09-21T15:01:11.007 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:11.007 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:11.008 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:11.008 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:11.008 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:11.008 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 114312 2024-09-21T15:01:11.008 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.008 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:11.008 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.008 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:11.008 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562396+0000", 2024-09-21T15:01:11.008 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:11.008 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.008 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.008 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:11.008 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562396+0000", 2024-09-21T15:01:11.008 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:11.009 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.009 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.009 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:11.009 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562395+0000", 2024-09-21T15:01:11.009 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:11.009 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.009 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.009 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:11.009 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562397+0000", 2024-09-21T15:01:11.009 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9e-06 2024-09-21T15:01:11.009 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.009 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.009 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:11.009 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562401+0000", 2024-09-21T15:01:11.010 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8099999999999999e-06 2024-09-21T15:01:11.010 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.010 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.010 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:11.010 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562405+0000", 2024-09-21T15:01:11.010 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.5710000000000002e-06 2024-09-21T15:01:11.010 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.010 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.010 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:11.010 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.621997+0000", 2024-09-21T15:01:11.010 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059592118999999999 2024-09-21T15:01:11.010 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.010 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.011 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:11.011 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.622121+0000", 2024-09-21T15:01:11.011 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012411599999999999 2024-09-21T15:01:11.011 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.011 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.011 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:11.011 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.622228+0000", 2024-09-21T15:01:11.011 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000107184 2024-09-21T15:01:11.011 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.011 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:11.011 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.011 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.011 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.011 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:114313 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 9035776~8192 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:11.012 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.562408+0000", 2024-09-21T15:01:11.012 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 41.358884572000001, 2024-09-21T15:01:11.012 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059975560999999997, 2024-09-21T15:01:11.012 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:11.012 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:11.012 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:11.012 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:11.012 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:11.012 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 114313 2024-09-21T15:01:11.012 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.012 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:11.012 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.012 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:11.012 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562408+0000", 2024-09-21T15:01:11.012 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:11.013 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.013 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.013 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:11.013 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562408+0000", 2024-09-21T15:01:11.013 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:11.013 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.013 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.013 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:11.013 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562407+0000", 2024-09-21T15:01:11.013 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:11.013 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.013 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.013 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:11.013 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562409+0000", 2024-09-21T15:01:11.014 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9700000000000002e-06 2024-09-21T15:01:11.014 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.014 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.014 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:11.014 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562421+0000", 2024-09-21T15:01:11.014 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2094e-05 2024-09-21T15:01:11.014 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.014 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.014 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:11.014 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562427+0000", 2024-09-21T15:01:11.014 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7690000000000002e-06 2024-09-21T15:01:11.014 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.014 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.014 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:11.015 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.622264+0000", 2024-09-21T15:01:11.015 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059837488000000001 2024-09-21T15:01:11.015 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.015 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.015 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:11.015 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.622319+0000", 2024-09-21T15:01:11.015 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4064999999999998e-05 2024-09-21T15:01:11.015 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.015 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.015 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:11.015 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.622383+0000", 2024-09-21T15:01:11.015 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4506999999999998e-05 2024-09-21T15:01:11.015 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.015 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:11.015 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.016 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.016 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:11.016 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:01:11.017 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T15:01:11.017 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T15:01:11.017 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T15:01:11.018 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T15:01:11.019 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:01:11.019 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:01:11.019 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:01:11.019 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:01:11.019 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:01:11.019 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:01:11.019 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:01:11.019 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:01:11.019 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:01:11.014+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:01:11.070 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-21T15:01:11.073 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-09-21T15:01:11.089 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":268,"stamp":"2024-09-21T15:01:09.824020+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1052048706,"num_objects":1338,"num_object_clones":0,"num_object_copies":2676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1338,"num_whiteouts":0,"num_read":144560,"num_read_kb":1153382,"num_write":26201,"num_write_kb":886308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1920,"num_bytes_recovered":145354624,"num_keys_recovered":229,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17362,"ondisk_log_size":17362,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":85,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":464432,"kb_used_data":86708,"kb_used_omap":12,"kb_used_meta":377587,"kb_avail":754510288,"statfs":{"total":773094113280,"available":772618534912,"internally_reserved":0,"allocated":88788992,"data_stored":646902284,"data_compressed":14204054,"data_compressed_allocated":58728448,"data_compressed_original":631348116,"omap_allocated":12744,"internal_metadata":386649656},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,31],"upper_bound":1024},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":70414573,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1900,"num_read_kb":26432,"num_write":155,"num_write_kb":20357,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":539,"num_bytes_recovered":41668853,"num_keys_recovered":77,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.667873"},"pg_stats":[{"pgid":"3.1f","version":"172'452","reported_seq":3450,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.157805+0000","last_change":"2024-09-21T15:01:04.223131+0000","last_active":"2024-09-21T15:01:05.157805+0000","last_peered":"2024-09-21T15:01:05.157805+0000","last_clean":"2024-09-21T15:01:05.157805+0000","last_became_active":"2024-09-21T14:58:19.410055+0000","last_became_peered":"2024-09-21T14:58:19.410055+0000","last_unstale":"2024-09-21T15:01:05.157805+0000","last_undegraded":"2024-09-21T15:01:05.157805+0000","last_fullsized":"2024-09-21T15:01:05.157805+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":452,"log_dups_size":0,"ondisk_log_size":452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:49:13.944219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063336100000000004,"stat_sum":{"num_bytes":32245069,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":3403,"num_read_kb":42213,"num_write":684,"num_write_kb":34241,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.1e","version":"176'366","reported_seq":2962,"reported_epoch":177,"state":"peering","last_fresh":"2024-09-21T15:01:05.160058+0000","last_change":"2024-09-21T15:01:05.160420+0000","last_active":"2024-09-21T15:01:05.160058+0000","last_peered":"2024-09-21T15:01:04.425541+0000","last_clean":"2024-09-21T15:01:00.453521+0000","last_became_active":"2024-09-21T15:01:03.253387+0000","last_became_peered":"2024-09-21T15:01:03.253387+0000","last_unstale":"2024-09-21T15:01:05.160058+0000","last_undegraded":"2024-09-21T15:01:05.160058+0000","last_fullsized":"2024-09-21T15:01:05.160058+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":366,"log_dups_size":0,"ondisk_log_size":366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:27:07.402534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021611780000000001,"stat_sum":{"num_bytes":19327265,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":0,"num_read":2999,"num_read_kb":22037,"num_write":453,"num_write_kb":16509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":148,"num_bytes_recovered":11635695,"num_keys_recovered":22,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,3],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"6b","length":"1"}]},{"pgid":"3.1d","version":"175'260","reported_seq":3611,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.164700+0000","last_change":"2024-09-21T15:01:04.229475+0000","last_active":"2024-09-21T15:01:05.164700+0000","last_peered":"2024-09-21T15:01:05.164700+0000","last_clean":"2024-09-21T15:01:05.164700+0000","last_became_active":"2024-09-21T14:58:19.411387+0000","last_became_peered":"2024-09-21T14:58:19.411387+0000","last_unstale":"2024-09-21T15:01:05.164700+0000","last_undegraded":"2024-09-21T15:01:05.164700+0000","last_fullsized":"2024-09-21T15:01:05.164700+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":260,"log_dups_size":0,"ondisk_log_size":260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:37:57.597069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054775500000000003,"stat_sum":{"num_bytes":18081929,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":3016,"num_read_kb":23636,"num_write":434,"num_write_kb":24977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.1c","version":"175'430","reported_seq":2874,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.187232+0000","last_change":"2024-09-21T15:01:04.234927+0000","last_active":"2024-09-21T15:01:05.187232+0000","last_peered":"2024-09-21T15:01:05.187232+0000","last_clean":"2024-09-21T15:01:05.187232+0000","last_became_active":"2024-09-21T14:58:19.411306+0000","last_became_peered":"2024-09-21T14:58:19.411306+0000","last_unstale":"2024-09-21T15:01:05.187232+0000","last_undegraded":"2024-09-21T15:01:05.187232+0000","last_fullsized":"2024-09-21T15:01:05.187232+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":430,"log_dups_size":0,"ondisk_log_size":430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:20:27.079795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010334350000000001,"stat_sum":{"num_bytes":19507114,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":2814,"num_read_kb":20995,"num_write":622,"num_write_kb":24708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.1b","version":"175'422","reported_seq":2730,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.158957+0000","last_change":"2024-09-21T15:01:04.227802+0000","last_active":"2024-09-21T15:01:05.158957+0000","last_peered":"2024-09-21T15:01:05.158957+0000","last_clean":"2024-09-21T15:01:05.158957+0000","last_became_active":"2024-09-21T14:59:31.131051+0000","last_became_peered":"2024-09-21T14:59:31.131051+0000","last_unstale":"2024-09-21T15:01:05.158957+0000","last_undegraded":"2024-09-21T15:01:05.158957+0000","last_fullsized":"2024-09-21T15:01:05.158957+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":422,"log_dups_size":0,"ondisk_log_size":422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:21:37.320774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001254673,"stat_sum":{"num_bytes":25447559,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":2614,"num_read_kb":28976,"num_write":559,"num_write_kb":20747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":117,"num_bytes_recovered":7757116,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.1a","version":"175'352","reported_seq":3233,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.187249+0000","last_change":"2024-09-21T15:01:04.234672+0000","last_active":"2024-09-21T15:01:05.187249+0000","last_peered":"2024-09-21T15:01:05.187249+0000","last_clean":"2024-09-21T15:01:05.187249+0000","last_became_active":"2024-09-21T14:58:19.409415+0000","last_became_peered":"2024-09-21T14:58:19.409415+0000","last_unstale":"2024-09-21T15:01:05.187249+0000","last_undegraded":"2024-09-21T15:01:05.187249+0000","last_fullsized":"2024-09-21T15:01:05.187249+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":352,"log_dups_size":0,"ondisk_log_size":352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:28:46.336301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060307399999999999,"stat_sum":{"num_bytes":31396202,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":3008,"num_read_kb":24746,"num_write":475,"num_write_kb":22526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.19","version":"175'648","reported_seq":3837,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.159011+0000","last_change":"2024-09-21T15:01:04.228157+0000","last_active":"2024-09-21T15:01:05.159011+0000","last_peered":"2024-09-21T15:01:05.159011+0000","last_clean":"2024-09-21T15:01:05.159011+0000","last_became_active":"2024-09-21T14:59:32.130624+0000","last_became_peered":"2024-09-21T14:59:32.130624+0000","last_unstale":"2024-09-21T15:01:05.159011+0000","last_undegraded":"2024-09-21T15:01:05.159011+0000","last_fullsized":"2024-09-21T15:01:05.159011+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":648,"log_dups_size":0,"ondisk_log_size":648,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:37:53.312733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001540409,"stat_sum":{"num_bytes":47896216,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":4950,"num_read_kb":51571,"num_write":1030,"num_write_kb":28173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":128,"num_bytes_recovered":11896992,"num_keys_recovered":44,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.18","version":"175'525","reported_seq":7399,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.157243+0000","last_change":"2024-09-21T15:01:04.223390+0000","last_active":"2024-09-21T15:01:05.157243+0000","last_peered":"2024-09-21T15:01:05.157243+0000","last_clean":"2024-09-21T15:01:05.157243+0000","last_became_active":"2024-09-21T14:58:19.409371+0000","last_became_peered":"2024-09-21T14:58:19.409371+0000","last_unstale":"2024-09-21T15:01:05.157243+0000","last_undegraded":"2024-09-21T15:01:05.157243+0000","last_fullsized":"2024-09-21T15:01:05.157243+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":525,"log_dups_size":0,"ondisk_log_size":525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:35:27.143445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076656300000000005,"stat_sum":{"num_bytes":36011050,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":8009,"num_read_kb":41151,"num_write":765,"num_write_kb":29537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.17","version":"172'334","reported_seq":3678,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.158315+0000","last_change":"2024-09-21T15:01:04.225804+0000","last_active":"2024-09-21T15:01:05.158315+0000","last_peered":"2024-09-21T15:01:05.158315+0000","last_clean":"2024-09-21T15:01:05.158315+0000","last_became_active":"2024-09-21T14:58:19.410306+0000","last_became_peered":"2024-09-21T14:58:19.410306+0000","last_unstale":"2024-09-21T15:01:05.158315+0000","last_undegraded":"2024-09-21T15:01:05.158315+0000","last_fullsized":"2024-09-21T15:01:05.158315+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":334,"log_dups_size":0,"ondisk_log_size":334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:33:50.089165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030273969999999998,"stat_sum":{"num_bytes":25072080,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":3370,"num_read_kb":27721,"num_write":557,"num_write_kb":31849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.16","version":"175'887","reported_seq":4751,"reported_epoch":177,"state":"creating+peering","last_fresh":"2024-09-21T15:01:04.608919+0000","last_change":"2024-09-21T15:01:05.877830+0000","last_active":"2024-09-21T15:01:05.877830+0000","last_peered":"2024-09-21T15:01:04.608919+0000","last_clean":"2024-09-21T15:01:00.518248+0000","last_became_active":"2024-09-21T15:01:03.252683+0000","last_became_peered":"2024-09-21T15:01:03.252683+0000","last_unstale":"2024-09-21T15:01:04.608919+0000","last_undegraded":"2024-09-21T15:01:04.608919+0000","last_fullsized":"2024-09-21T15:01:03.215921+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":887,"log_dups_size":0,"ondisk_log_size":887,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:19:04.988784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001973472,"stat_sum":{"num_bytes":33667875,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":6806,"num_read_kb":40610,"num_write":1387,"num_write_kb":26337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":271,"num_bytes_recovered":16821266,"num_keys_recovered":55,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"6b","length":"1"}]},{"pgid":"3.15","version":"175'717","reported_seq":3617,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.169730+0000","last_change":"2024-09-21T15:01:04.231959+0000","last_active":"2024-09-21T15:01:05.169730+0000","last_peered":"2024-09-21T15:01:05.169730+0000","last_clean":"2024-09-21T15:01:05.169730+0000","last_became_active":"2024-09-21T14:58:19.410687+0000","last_became_peered":"2024-09-21T14:58:19.410687+0000","last_unstale":"2024-09-21T15:01:05.169730+0000","last_undegraded":"2024-09-21T15:01:05.169730+0000","last_fullsized":"2024-09-21T15:01:05.169730+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":717,"log_dups_size":0,"ondisk_log_size":717,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:21:42.226982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033742220000000001,"stat_sum":{"num_bytes":45321568,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":4682,"num_read_kb":48927,"num_write":1212,"num_write_kb":32331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.14","version":"175'433","reported_seq":2993,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.164396+0000","last_change":"2024-09-21T15:01:04.229559+0000","last_active":"2024-09-21T15:01:05.164396+0000","last_peered":"2024-09-21T15:01:05.164396+0000","last_clean":"2024-09-21T15:01:05.164396+0000","last_became_active":"2024-09-21T14:58:19.407310+0000","last_became_peered":"2024-09-21T14:58:19.407310+0000","last_unstale":"2024-09-21T15:01:05.164396+0000","last_undegraded":"2024-09-21T15:01:05.164396+0000","last_fullsized":"2024-09-21T15:01:05.164396+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:47:46.079115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052455500000000001,"stat_sum":{"num_bytes":43744336,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":2858,"num_read_kb":38772,"num_write":581,"num_write_kb":32120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":171,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-21T15:01:04.229713+0000","last_change":"2024-09-21T14:58:11.348177+0000","last_active":"2024-09-21T15:01:04.229713+0000","last_peered":"2024-09-21T15:01:04.229713+0000","last_clean":"2024-09-21T15:01:04.229713+0000","last_became_active":"2024-09-21T14:58:09.328417+0000","last_became_peered":"2024-09-21T14:58:09.328417+0000","last_unstale":"2024-09-21T15:01:04.229713+0000","last_undegraded":"2024-09-21T15:01:04.229713+0000","last_fullsized":"2024-09-21T15:01:04.229713+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:52:22.671185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018917490000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"172'367","reported_seq":5379,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.158972+0000","last_change":"2024-09-21T15:01:04.228240+0000","last_active":"2024-09-21T15:01:05.158972+0000","last_peered":"2024-09-21T15:01:05.158972+0000","last_clean":"2024-09-21T15:01:05.158972+0000","last_became_active":"2024-09-21T14:59:32.131232+0000","last_became_peered":"2024-09-21T14:59:32.131232+0000","last_unstale":"2024-09-21T15:01:05.158972+0000","last_undegraded":"2024-09-21T15:01:05.158972+0000","last_fullsized":"2024-09-21T15:01:05.158972+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":367,"log_dups_size":0,"ondisk_log_size":367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:31:04.421782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001438581,"stat_sum":{"num_bytes":50093146,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":5172,"num_read_kb":51612,"num_write":497,"num_write_kb":25669,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":3585713,"num_keys_recovered":9,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":96,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-21T15:01:04.227015+0000","last_change":"2024-09-21T14:59:28.228883+0000","last_active":"2024-09-21T15:01:04.227015+0000","last_peered":"2024-09-21T15:01:04.227015+0000","last_clean":"2024-09-21T15:01:04.227015+0000","last_became_active":"2024-09-21T14:59:28.228464+0000","last_became_peered":"2024-09-21T14:59:28.228464+0000","last_unstale":"2024-09-21T15:01:04.227015+0000","last_undegraded":"2024-09-21T15:01:04.227015+0000","last_fullsized":"2024-09-21T15:01:04.227015+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:00:26.310864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"175'809","reported_seq":3401,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.159556+0000","last_change":"2024-09-21T15:01:04.228668+0000","last_active":"2024-09-21T15:01:05.159556+0000","last_peered":"2024-09-21T15:01:05.159556+0000","last_clean":"2024-09-21T15:01:05.159556+0000","last_became_active":"2024-09-21T14:59:31.132581+0000","last_became_peered":"2024-09-21T14:59:31.132581+0000","last_unstale":"2024-09-21T15:01:05.159556+0000","last_undegraded":"2024-09-21T15:01:05.159556+0000","last_fullsized":"2024-09-21T15:01:05.159556+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":809,"log_dups_size":0,"ondisk_log_size":809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:36:45.467515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016620179999999999,"stat_sum":{"num_bytes":28736355,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":4991,"num_read_kb":30499,"num_write":1191,"num_write_kb":23913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":119,"num_bytes_recovered":6871924,"num_keys_recovered":11,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":187,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-21T15:01:04.229145+0000","last_change":"2024-09-21T14:59:28.231502+0000","last_active":"2024-09-21T15:01:04.229145+0000","last_peered":"2024-09-21T15:01:04.229145+0000","last_clean":"2024-09-21T15:01:04.229145+0000","last_became_active":"2024-09-21T14:59:28.231043+0000","last_became_peered":"2024-09-21T14:59:28.231043+0000","last_unstale":"2024-09-21T15:01:04.229145+0000","last_undegraded":"2024-09-21T15:01:04.229145+0000","last_fullsized":"2024-09-21T15:01:04.229145+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:09:46.773690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019338770000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"172'553","reported_seq":5095,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.158909+0000","last_change":"2024-09-21T15:01:04.227811+0000","last_active":"2024-09-21T15:01:05.158909+0000","last_peered":"2024-09-21T15:01:05.158909+0000","last_clean":"2024-09-21T15:01:05.158909+0000","last_became_active":"2024-09-21T15:00:34.347238+0000","last_became_peered":"2024-09-21T15:00:34.347238+0000","last_unstale":"2024-09-21T15:01:05.158909+0000","last_undegraded":"2024-09-21T15:01:05.158909+0000","last_fullsized":"2024-09-21T15:01:05.158909+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":553,"log_dups_size":0,"ondisk_log_size":553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:31:33.639268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001277607,"stat_sum":{"num_bytes":39912228,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":5081,"num_read_kb":38156,"num_write":939,"num_write_kb":39278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":14049436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":172,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-21T15:01:04.222407+0000","last_change":"2024-09-21T14:58:11.406176+0000","last_active":"2024-09-21T15:01:04.222407+0000","last_peered":"2024-09-21T15:01:04.222407+0000","last_clean":"2024-09-21T15:01:04.222407+0000","last_became_active":"2024-09-21T14:58:09.777465+0000","last_became_peered":"2024-09-21T14:58:09.777465+0000","last_unstale":"2024-09-21T15:01:04.222407+0000","last_undegraded":"2024-09-21T15:01:04.222407+0000","last_fullsized":"2024-09-21T15:01:04.222407+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:55:04.667816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095704000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"172'662","reported_seq":2540,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.162165+0000","last_change":"2024-09-21T15:01:04.224786+0000","last_active":"2024-09-21T15:01:05.162165+0000","last_peered":"2024-09-21T15:01:05.162165+0000","last_clean":"2024-09-21T15:01:05.162165+0000","last_became_active":"2024-09-21T14:58:19.408971+0000","last_became_peered":"2024-09-21T14:58:19.408971+0000","last_unstale":"2024-09-21T15:01:05.162165+0000","last_undegraded":"2024-09-21T15:01:05.162165+0000","last_fullsized":"2024-09-21T15:01:05.162165+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":662,"log_dups_size":0,"ondisk_log_size":662,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:29:52.642121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000380054,"stat_sum":{"num_bytes":19474374,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":3561,"num_read_kb":20372,"num_write":1084,"num_write_kb":19812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.3","version":"172'738","reported_seq":2939,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.169226+0000","last_change":"2024-09-21T15:01:04.231096+0000","last_active":"2024-09-21T15:01:05.169226+0000","last_peered":"2024-09-21T15:01:05.169226+0000","last_clean":"2024-09-21T15:01:05.169226+0000","last_became_active":"2024-09-21T14:58:19.417453+0000","last_became_peered":"2024-09-21T14:58:19.417453+0000","last_unstale":"2024-09-21T15:01:05.169226+0000","last_undegraded":"2024-09-21T15:01:05.169226+0000","last_fullsized":"2024-09-21T15:01:05.169226+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":738,"log_dups_size":0,"ondisk_log_size":738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:19:27.087272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002785708,"stat_sum":{"num_bytes":15820678,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":4393,"num_read_kb":24172,"num_write":1224,"num_write_kb":20113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":173,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-21T15:01:04.232215+0000","last_change":"2024-09-21T14:58:11.347504+0000","last_active":"2024-09-21T15:01:04.232215+0000","last_peered":"2024-09-21T15:01:04.232215+0000","last_clean":"2024-09-21T15:01:04.232215+0000","last_became_active":"2024-09-21T14:58:09.775981+0000","last_became_peered":"2024-09-21T14:58:09.775981+0000","last_unstale":"2024-09-21T15:01:04.232215+0000","last_undegraded":"2024-09-21T15:01:04.232215+0000","last_fullsized":"2024-09-21T15:01:04.232215+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:47:16.286652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012568880000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"176'534","reported_seq":2983,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.157022+0000","last_change":"2024-09-21T15:01:04.223338+0000","last_active":"2024-09-21T15:01:05.157022+0000","last_peered":"2024-09-21T15:01:05.157022+0000","last_clean":"2024-09-21T15:01:05.157022+0000","last_became_active":"2024-09-21T14:58:19.405811+0000","last_became_peered":"2024-09-21T14:58:19.405811+0000","last_unstale":"2024-09-21T15:01:05.157022+0000","last_undegraded":"2024-09-21T15:01:05.157022+0000","last_fullsized":"2024-09-21T15:01:05.157022+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":534,"log_dups_size":0,"ondisk_log_size":534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:36:25.551607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083259800000000004,"stat_sum":{"num_bytes":20501650,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":3805,"num_read_kb":26326,"num_write":700,"num_write_kb":25158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":173,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.184226+0000","last_change":"2024-09-21T14:58:11.342210+0000","last_active":"2024-09-21T15:01:05.184226+0000","last_peered":"2024-09-21T15:01:05.184226+0000","last_clean":"2024-09-21T15:01:05.184226+0000","last_became_active":"2024-09-21T14:58:09.330624+0000","last_became_peered":"2024-09-21T14:58:09.330624+0000","last_unstale":"2024-09-21T15:01:05.184226+0000","last_undegraded":"2024-09-21T15:01:05.184226+0000","last_fullsized":"2024-09-21T15:01:05.184226+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:29:43.229538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001051151,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"175'276","reported_seq":2007,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.166575+0000","last_change":"2024-09-21T15:01:05.166575+0000","last_active":"2024-09-21T15:01:05.166575+0000","last_peered":"2024-09-21T15:01:05.166575+0000","last_clean":"2024-09-21T15:01:05.166575+0000","last_became_active":"2024-09-21T15:01:05.164862+0000","last_became_peered":"2024-09-21T15:01:05.164862+0000","last_unstale":"2024-09-21T15:01:05.166575+0000","last_undegraded":"2024-09-21T15:01:05.166575+0000","last_fullsized":"2024-09-21T15:01:05.166575+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":276,"log_dups_size":0,"ondisk_log_size":276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:57:46.675335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046340299999999998,"stat_sum":{"num_bytes":42467012,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":2366,"num_read_kb":39964,"num_write":407,"num_write_kb":39026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":254,"num_bytes_recovered":20571253,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":171,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-21T15:01:04.227843+0000","last_change":"2024-09-21T14:58:11.346766+0000","last_active":"2024-09-21T15:01:04.227843+0000","last_peered":"2024-09-21T15:01:04.227843+0000","last_clean":"2024-09-21T15:01:04.227843+0000","last_became_active":"2024-09-21T14:58:09.329748+0000","last_became_peered":"2024-09-21T14:58:09.329748+0000","last_unstale":"2024-09-21T15:01:04.227843+0000","last_undegraded":"2024-09-21T15:01:04.227843+0000","last_fullsized":"2024-09-21T15:01:04.227843+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:31:20.267675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052698000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"175'715","reported_seq":5700,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.160742+0000","last_change":"2024-09-21T15:01:04.224483+0000","last_active":"2024-09-21T15:01:05.160742+0000","last_peered":"2024-09-21T15:01:05.160742+0000","last_clean":"2024-09-21T15:01:05.160742+0000","last_became_active":"2024-09-21T14:58:41.401003+0000","last_became_peered":"2024-09-21T14:58:41.401003+0000","last_unstale":"2024-09-21T15:01:05.160742+0000","last_undegraded":"2024-09-21T15:01:05.160742+0000","last_fullsized":"2024-09-21T15:01:05.160742+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":715,"log_dups_size":0,"ondisk_log_size":715,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:05:37.644835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046693700000000001,"stat_sum":{"num_bytes":45819535,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":7513,"num_read_kb":42793,"num_write":987,"num_write_kb":21873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1403348,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":172,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-21T15:01:04.228858+0000","last_change":"2024-09-21T14:58:11.347700+0000","last_active":"2024-09-21T15:01:04.228858+0000","last_peered":"2024-09-21T15:01:04.228858+0000","last_clean":"2024-09-21T15:01:04.228858+0000","last_became_active":"2024-09-21T14:58:09.438842+0000","last_became_peered":"2024-09-21T14:58:09.438842+0000","last_unstale":"2024-09-21T15:01:04.228858+0000","last_undegraded":"2024-09-21T15:01:04.228858+0000","last_fullsized":"2024-09-21T15:01:04.228858+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:41:34.626249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001426194,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":320,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-21T15:01:04.233768+0000","last_change":"2024-09-21T14:58:03.616055+0000","last_active":"2024-09-21T15:01:04.233768+0000","last_peered":"2024-09-21T15:01:04.233768+0000","last_clean":"2024-09-21T15:01:04.233768+0000","last_became_active":"2024-09-21T14:58:03.613214+0000","last_became_peered":"2024-09-21T14:58:03.613214+0000","last_unstale":"2024-09-21T15:01:04.233768+0000","last_undegraded":"2024-09-21T15:01:04.233768+0000","last_fullsized":"2024-09-21T15:01:04.233768+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:02.180162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:02.180162+0000","last_clean_scrub_stamp":"2024-09-21T14:58:02.180162+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:29:05.860361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"172'602","reported_seq":4041,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.157772+0000","last_change":"2024-09-21T15:01:04.227318+0000","last_active":"2024-09-21T15:01:05.157772+0000","last_peered":"2024-09-21T15:01:05.157772+0000","last_clean":"2024-09-21T15:01:05.157772+0000","last_became_active":"2024-09-21T14:59:31.131460+0000","last_became_peered":"2024-09-21T14:59:31.131460+0000","last_unstale":"2024-09-21T15:01:05.157772+0000","last_undegraded":"2024-09-21T15:01:05.157772+0000","last_fullsized":"2024-09-21T15:01:05.157772+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":602,"log_dups_size":0,"ondisk_log_size":602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:01:38.353476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098975100000000009,"stat_sum":{"num_bytes":37289213,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":4356,"num_read_kb":43318,"num_write":990,"num_write_kb":29471,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":119,"num_bytes_recovered":6097618,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.9","version":"173'786","reported_seq":2989,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.164259+0000","last_change":"2024-09-21T15:01:04.232710+0000","last_active":"2024-09-21T15:01:05.164259+0000","last_peered":"2024-09-21T15:01:05.164259+0000","last_clean":"2024-09-21T15:01:05.164259+0000","last_became_active":"2024-09-21T14:59:28.233956+0000","last_became_peered":"2024-09-21T14:59:28.233956+0000","last_unstale":"2024-09-21T15:01:05.164259+0000","last_undegraded":"2024-09-21T15:01:05.164259+0000","last_fullsized":"2024-09-21T15:01:05.164259+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":786,"log_dups_size":0,"ondisk_log_size":786,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:29:09.682792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027159770000000001,"stat_sum":{"num_bytes":23720917,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":4523,"num_read_kb":33253,"num_write":1276,"num_write_kb":30634,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":91,"num_bytes_recovered":6925016,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.a","version":"176'786","reported_seq":3964,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.187273+0000","last_change":"2024-09-21T15:01:04.234662+0000","last_active":"2024-09-21T15:01:05.187273+0000","last_peered":"2024-09-21T15:01:05.187273+0000","last_clean":"2024-09-21T15:01:05.187273+0000","last_became_active":"2024-09-21T15:00:27.280971+0000","last_became_peered":"2024-09-21T15:00:27.280971+0000","last_unstale":"2024-09-21T15:01:05.187273+0000","last_undegraded":"2024-09-21T15:01:05.187273+0000","last_fullsized":"2024-09-21T15:01:05.187273+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":786,"log_dups_size":0,"ondisk_log_size":786,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:26:07.617059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076445899999999999,"stat_sum":{"num_bytes":20136068,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":5765,"num_read_kb":29533,"num_write":1129,"num_write_kb":31048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":9228123,"num_keys_recovered":33,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.b","version":"178'455","reported_seq":2985,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-21T15:01:06.643684+0000","last_change":"2024-09-21T15:01:04.239682+0000","last_active":"2024-09-21T15:01:06.643684+0000","last_peered":"2024-09-21T15:01:06.643684+0000","last_clean":"2024-09-21T15:01:06.643684+0000","last_became_active":"2024-09-21T14:58:19.410679+0000","last_became_peered":"2024-09-21T14:58:19.410679+0000","last_unstale":"2024-09-21T15:01:06.643684+0000","last_undegraded":"2024-09-21T15:01:06.643684+0000","last_fullsized":"2024-09-21T15:01:06.643684+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:07:40.758932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039592400000000003,"stat_sum":{"num_bytes":31436179,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":2889,"num_read_kb":36109,"num_write":704,"num_write_kb":37196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.c","version":"172'408","reported_seq":4312,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.169078+0000","last_change":"2024-09-21T15:01:04.231357+0000","last_active":"2024-09-21T15:01:05.169078+0000","last_peered":"2024-09-21T15:01:05.169078+0000","last_clean":"2024-09-21T15:01:05.169078+0000","last_became_active":"2024-09-21T14:59:25.130057+0000","last_became_peered":"2024-09-21T14:59:25.130057+0000","last_unstale":"2024-09-21T15:01:05.169078+0000","last_undegraded":"2024-09-21T15:01:05.169078+0000","last_fullsized":"2024-09-21T15:01:05.169078+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:53:33.752550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003377793,"stat_sum":{"num_bytes":22999993,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":4337,"num_read_kb":28452,"num_write":576,"num_write_kb":23757,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":9059600,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.d","version":"172'422","reported_seq":4914,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.164463+0000","last_change":"2024-09-21T15:01:04.233218+0000","last_active":"2024-09-21T15:01:05.164463+0000","last_peered":"2024-09-21T15:01:05.164463+0000","last_clean":"2024-09-21T15:01:05.164463+0000","last_became_active":"2024-09-21T14:58:19.407884+0000","last_became_peered":"2024-09-21T14:58:19.407884+0000","last_unstale":"2024-09-21T15:01:05.164463+0000","last_undegraded":"2024-09-21T15:01:05.164463+0000","last_fullsized":"2024-09-21T15:01:05.164463+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":422,"log_dups_size":0,"ondisk_log_size":422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:28:57.111615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034473960000000001,"stat_sum":{"num_bytes":49032397,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":4783,"num_read_kb":55414,"num_write":689,"num_write_kb":35805,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.e","version":"171'774","reported_seq":2852,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.176277+0000","last_change":"2024-09-21T15:01:05.176277+0000","last_active":"2024-09-21T15:01:05.176277+0000","last_peered":"2024-09-21T15:01:05.176277+0000","last_clean":"2024-09-21T15:01:05.176277+0000","last_became_active":"2024-09-21T15:01:05.172427+0000","last_became_peered":"2024-09-21T15:01:05.172427+0000","last_unstale":"2024-09-21T15:01:05.176277+0000","last_undegraded":"2024-09-21T15:01:05.176277+0000","last_fullsized":"2024-09-21T15:01:05.176277+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":774,"log_dups_size":0,"ondisk_log_size":774,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:28:18.710621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093917399999999998,"stat_sum":{"num_bytes":12028520,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":4155,"num_read_kb":16497,"num_write":1272,"num_write_kb":16419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":138,"num_bytes_recovered":7486496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.f","version":"175'427","reported_seq":5010,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.169470+0000","last_change":"2024-09-21T15:01:04.229660+0000","last_active":"2024-09-21T15:01:05.169470+0000","last_peered":"2024-09-21T15:01:05.169470+0000","last_clean":"2024-09-21T15:01:05.169470+0000","last_became_active":"2024-09-21T14:58:19.410406+0000","last_became_peered":"2024-09-21T14:58:19.410406+0000","last_unstale":"2024-09-21T15:01:05.169470+0000","last_undegraded":"2024-09-21T15:01:05.169470+0000","last_fullsized":"2024-09-21T15:01:05.169470+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":427,"log_dups_size":0,"ondisk_log_size":427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:59:16.026939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014556770000000001,"stat_sum":{"num_bytes":49430534,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":4917,"num_read_kb":52382,"num_write":667,"num_write_kb":35936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.10","version":"172'427","reported_seq":4297,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.186765+0000","last_change":"2024-09-21T15:01:04.233986+0000","last_active":"2024-09-21T15:01:05.186765+0000","last_peered":"2024-09-21T15:01:05.186765+0000","last_clean":"2024-09-21T15:01:05.186765+0000","last_became_active":"2024-09-21T14:58:19.409935+0000","last_became_peered":"2024-09-21T14:58:19.409935+0000","last_unstale":"2024-09-21T15:01:05.186765+0000","last_undegraded":"2024-09-21T15:01:05.186765+0000","last_fullsized":"2024-09-21T15:01:05.186765+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":427,"log_dups_size":0,"ondisk_log_size":427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:10:36.252629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001679137,"stat_sum":{"num_bytes":49580759,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":3927,"num_read_kb":58252,"num_write":670,"num_write_kb":35408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.11","version":"172'457","reported_seq":4563,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.168682+0000","last_change":"2024-09-21T15:01:05.168682+0000","last_active":"2024-09-21T15:01:05.168682+0000","last_peered":"2024-09-21T15:01:05.168682+0000","last_clean":"2024-09-21T15:01:05.168682+0000","last_became_active":"2024-09-21T15:01:05.165955+0000","last_became_peered":"2024-09-21T15:01:05.165955+0000","last_unstale":"2024-09-21T15:01:05.168682+0000","last_undegraded":"2024-09-21T15:01:05.168682+0000","last_fullsized":"2024-09-21T15:01:05.168682+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":457,"log_dups_size":0,"ondisk_log_size":457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:34:54.111119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010096230000000001,"stat_sum":{"num_bytes":23766643,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":5218,"num_read_kb":27936,"num_write":524,"num_write_kb":22561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":166,"num_bytes_recovered":11965028,"num_keys_recovered":33,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.12","version":"176'480","reported_seq":6284,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.163311+0000","last_change":"2024-09-21T15:01:04.224935+0000","last_active":"2024-09-21T15:01:05.163311+0000","last_peered":"2024-09-21T15:01:05.163311+0000","last_clean":"2024-09-21T15:01:05.163311+0000","last_became_active":"2024-09-21T15:00:07.155144+0000","last_became_peered":"2024-09-21T15:00:07.155144+0000","last_unstale":"2024-09-21T15:01:05.163311+0000","last_undegraded":"2024-09-21T15:01:05.163311+0000","last_fullsized":"2024-09-21T15:01:05.163311+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":480,"log_dups_size":0,"ondisk_log_size":480,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:58:00.133639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010497799999999999,"stat_sum":{"num_bytes":38476413,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":7022,"num_read_kb":30524,"num_write":743,"num_write_kb":18554,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.13","version":"175'738","reported_seq":5364,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.164109+0000","last_change":"2024-09-21T15:01:04.229229+0000","last_active":"2024-09-21T15:01:05.164109+0000","last_peered":"2024-09-21T15:01:05.164109+0000","last_clean":"2024-09-21T15:01:05.164109+0000","last_became_active":"2024-09-21T14:58:19.412030+0000","last_became_peered":"2024-09-21T14:58:19.412030+0000","last_unstale":"2024-09-21T15:01:05.164109+0000","last_undegraded":"2024-09-21T15:01:05.164109+0000","last_fullsized":"2024-09-21T15:01:05.164109+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":738,"log_dups_size":0,"ondisk_log_size":738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:44:29.282391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046048699999999998,"stat_sum":{"num_bytes":53017442,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":7141,"num_read_kb":56363,"num_write":1012,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"6e","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1051458319,"num_objects":1334,"num_object_clones":0,"num_object_copies":2668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1334,"num_whiteouts":0,"num_read":144444,"num_read_kb":1153282,"num_write":26040,"num_write_kb":883890,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1920,"num_bytes_recovered":145354624,"num_keys_recovered":229,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":71651328,"data_stored":637465465,"data_compressed":14193650,"data_compressed_allocated":58654720,"data_compressed_original":630201236,"omap_allocated":0,"internal_metadata":0},"log_size":17242,"ondisk_log_size":17242,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":10404,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509481,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34628,"kb_used_data":7612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94337212,"statfs":{"total":96636764160,"available":96601305088,"internally_reserved":0,"allocated":7794688,"data_stored":48907094,"data_compressed":1141276,"data_compressed_allocated":4558848,"data_compressed_original":47268443,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.85499999999999998,"15min":0.84199999999999997},"min":{"1min":0.46500000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.872,"5min":0.93300000000000005,"15min":0.95199999999999996},"min":{"1min":0.54900000000000004,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.3839999999999999,"5min":1.917,"15min":1.917},"last":0.86499999999999999}]},{"osd":1,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.81000000000000005,"15min":0.77700000000000002},"min":{"1min":0.35499999999999998,"5min":0.312,"15min":0.312},"max":{"1min":1.2789999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.83199999999999996,"15min":0.82599999999999996},"min":{"1min":0.57399999999999995,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.208,"5min":2.2149999999999999,"15min":2.2149999999999999},"last":1.002}]},{"osd":2,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.82199999999999995,"15min":0.81200000000000006},"min":{"1min":0.50900000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.169,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.93500000000000005,"15min":0.94299999999999995},"min":{"1min":0.55600000000000005,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.355,"5min":1.466,"15min":1.466},"last":0.90500000000000003}]},{"osd":3,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.88500000000000001,"15min":0.86099999999999999},"min":{"1min":0.60999999999999999,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.401,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.97799999999999998,"15min":0.96899999999999997},"min":{"1min":0.68200000000000005,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.3420000000000001,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.81899999999999995}]},{"osd":4,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.92200000000000004,"15min":0.93300000000000005},"min":{"1min":0.42199999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.3089999999999999,"5min":2.7130000000000001,"15min":2.7130000000000001},"last":1.048},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.94499999999999995,"15min":0.92600000000000005},"min":{"1min":0.39000000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.425,"5min":2.125,"15min":2.125},"last":0.97699999999999998}]},{"osd":5,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":1.026,"15min":1.022},"min":{"1min":0.51300000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.5800000000000001,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.94299999999999995,"5min":1.0409999999999999,"15min":1.05},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.3029999999999999,"5min":3.1880000000000002,"15min":3.1880000000000002},"last":1.0700000000000001}]},{"osd":6,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.133,"15min":1.149},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.5329999999999999,"5min":3.2730000000000001,"15min":3.2730000000000001},"last":1.0269999999999999},{"interface":"front","average":{"1min":1.095,"5min":1.0629999999999999,"15min":1.03},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.6990000000000001,"5min":2.2879999999999998,"15min":2.2879999999999998},"last":1.109}]}]},{"osd":6,"up_from":14,"seq":60129542184,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63192,"kb_used_data":9928,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94308648,"statfs":{"total":96636764160,"available":96572055552,"internally_reserved":0,"allocated":10166272,"data_stored":71351473,"data_compressed":1591208,"data_compressed_allocated":6516736,"data_compressed_original":69451532,"omap_allocated":1593,"internal_metadata":54524359},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,16],"upper_bound":1024},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:00:06 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.94399999999999995,"15min":0.95699999999999996},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.244,"5min":1.593,"15min":1.593},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.93600000000000005,"15min":0.95099999999999996},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.3360000000000001,"5min":1.587,"15min":1.587},"last":1.089}]},{"osd":1,"last update":"Sat Sep 21 15:00:06 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.90400000000000003,"15min":0.91400000000000003},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.1120000000000001,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.98299999999999998},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.95699999999999996,"15min":0.97999999999999998},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.1819999999999999,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.89100000000000001}]},{"osd":2,"last update":"Sat Sep 21 15:00:06 2024","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.98499999999999999,"15min":0.99099999999999999},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.3560000000000001,"5min":1.758,"15min":1.758},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.89900000000000002,"15min":0.90900000000000003},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.3069999999999999,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.93799999999999994}]},{"osd":3,"last update":"Sat Sep 21 15:00:06 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.94499999999999995,"15min":0.95799999999999996},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.169,"5min":1.835,"15min":1.835},"last":0.64200000000000002},{"interface":"front","average":{"1min":1.012,"5min":1.1180000000000001,"15min":1.1359999999999999},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.4430000000000001,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":1.2210000000000001}]},{"osd":4,"last update":"Sat Sep 21 15:00:06 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.97699999999999998,"15min":1.0049999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.1379999999999999,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.754},{"interface":"front","average":{"1min":0.93999999999999995,"5min":1.02,"15min":1.0329999999999999},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.411,"5min":1.897,"15min":1.897},"last":0.72399999999999998}]},{"osd":5,"last update":"Sat Sep 21 15:00:06 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":1.0860000000000001,"15min":1.1240000000000001},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.2589999999999999,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":1.0209999999999999},{"interface":"front","average":{"1min":1.01,"5min":1.0800000000000001,"15min":1.0920000000000001},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.3779999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":0.82299999999999995}]},{"osd":7,"last update":"Sat Sep 21 15:00:06 2024","interfaces":[{"interface":"back","average":{"1min":1.123,"5min":1.2350000000000001,"15min":1.2529999999999999},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.4530000000000001,"5min":1.8660000000000001,"15min":1.8660000000000001},"last":1.163},{"interface":"front","average":{"1min":1.044,"5min":1.26,"15min":1.296},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.385,"5min":2.004,"15min":2.004},"last":1.1140000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542185,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40508,"kb_used_data":9388,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94331332,"statfs":{"total":96636764160,"available":96595283968,"internally_reserved":0,"allocated":9613312,"data_stored":65131679,"data_compressed":1272182,"data_compressed_allocated":5857280,"data_compressed_original":63146092,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,15],"upper_bound":1024},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:00:01 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.96799999999999997,"15min":1.0109999999999999},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.98999999999999999,"5min":4.4800000000000004,"15min":4.4800000000000004},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.80800000000000005,"15min":0.82099999999999995},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.0329999999999999,"5min":1.181,"15min":1.181},"last":0.57499999999999996}]},{"osd":1,"last update":"Sat Sep 21 15:01:02 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.89600000000000002,"15min":0.92400000000000004},"min":{"1min":0.47699999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":2.7709999999999999,"5min":2.7709999999999999,"15min":2.7709999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.91400000000000003,"15min":0.93400000000000005},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":2.823,"5min":2.823,"15min":2.823},"last":0.84699999999999998}]},{"osd":2,"last update":"Sat Sep 21 15:01:02 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.91200000000000003,"15min":0.97399999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.159,"5min":3.419,"15min":3.419},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.87,"15min":0.89600000000000002},"min":{"1min":0.53400000000000003,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0780000000000001,"5min":1.423,"15min":1.423},"last":0.96299999999999997}]},{"osd":3,"last update":"Sat Sep 21 15:01:02 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.80300000000000005,"15min":0.80700000000000005},"min":{"1min":0.58499999999999996,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":1.258},{"interface":"front","average":{"1min":0.96699999999999997,"5min":1.054,"15min":1.0840000000000001},"min":{"1min":0.59899999999999998,"5min":0.435,"15min":0.435},"max":{"1min":1.272,"5min":2.6379999999999999,"15min":2.6379999999999999},"last":1.1399999999999999}]},{"osd":4,"last update":"Sat Sep 21 15:01:02 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.98899999999999999,"15min":1.016},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.5609999999999999,"5min":2.5470000000000002,"15min":2.5470000000000002},"last":1.038},{"interface":"front","average":{"1min":0.82099999999999995,"5min":1.0169999999999999,"15min":1.0780000000000001},"min":{"1min":0.39300000000000002,"5min":0.376,"15min":0.376},"max":{"1min":1.302,"5min":2.5099999999999998,"15min":2.5099999999999998},"last":1.302}]},{"osd":6,"last update":"Sat Sep 21 15:01:02 2024","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.94299999999999995,"15min":0.96899999999999997},"min":{"1min":0.63,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":1.0840000000000001},{"interface":"front","average":{"1min":1.004,"5min":1.2450000000000001,"15min":1.3520000000000001},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.425,"5min":4.2469999999999999,"15min":4.2469999999999999},"last":1.214}]},{"osd":7,"last update":"Sat Sep 21 15:00:05 2024","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.3200000000000001,"15min":1.363},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":2.7189999999999999,"5min":3.3010000000000002,"15min":3.3010000000000002},"last":1.345},{"interface":"front","average":{"1min":1.0740000000000001,"5min":1.1339999999999999,"15min":1.1439999999999999},"min":{"1min":0.64200000000000002,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":2.5880000000000001,"5min":2.5880000000000001,"15min":2.5880000000000001},"last":1.181}]}]},{"osd":4,"up_from":14,"seq":60129542184,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":67704,"kb_used_data":10340,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94304136,"statfs":{"total":96636764160,"available":96567435264,"internally_reserved":0,"allocated":10588160,"data_stored":79055540,"data_compressed":1750025,"data_compressed_allocated":7159808,"data_compressed_original":77331143,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.76000000000000001,"15min":0.77600000000000002},"min":{"1min":0.47499999999999998,"5min":0.311,"15min":0.311},"max":{"1min":1.1020000000000001,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":1.5860000000000001,"5min":1.0489999999999999,"15min":0.98099999999999998},"min":{"1min":0.49299999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":13.324,"5min":13.324,"15min":13.324},"last":1.1659999999999999}]},{"osd":1,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.85899999999999999,"15min":0.90600000000000003},"min":{"1min":0.46800000000000003,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.0609999999999999,"5min":1.347,"15min":1.347},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.82199999999999995,"15min":0.83899999999999997},"min":{"1min":0.502,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.58499999999999996}]},{"osd":2,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.88200000000000001,"15min":0.88},"min":{"1min":0.55100000000000005,"5min":0.501,"15min":0.501},"max":{"1min":1.47,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.98099999999999998,"5min":1.002,"15min":1.026},"min":{"1min":0.60699999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.3740000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":1.056}]},{"osd":3,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.90700000000000003,"15min":0.89500000000000002},"min":{"1min":0.61799999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.482,"5min":1.538,"15min":1.538},"last":1.121},{"interface":"front","average":{"1min":1.024,"5min":1.0129999999999999,"15min":0.99199999999999999},"min":{"1min":0.56599999999999995,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.528,"5min":1.528,"15min":1.528},"last":1.01}]},{"osd":5,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.97499999999999998,"15min":0.98399999999999999},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.579,"5min":1.579,"15min":1.579},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.94799999999999995,"15min":0.96399999999999997},"min":{"1min":0.61899999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":0.97299999999999998}]},{"osd":6,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.996,"15min":1.0109999999999999},"min":{"1min":0.55400000000000005,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.349,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.86599999999999999},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.081,"15min":1.0900000000000001},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.665,"5min":1.665,"15min":1.665},"last":0.71899999999999997}]},{"osd":7,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.995,"15min":1.0249999999999999},"min":{"1min":0.45200000000000001,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.226,"5min":1.7010000000000001,"15min":1.7010000000000001},"last":0.748},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.2050000000000001,"15min":1.2130000000000001},"min":{"1min":0.627,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.91800000000000004}]}]},{"osd":3,"up_from":14,"seq":60129542185,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58544,"kb_used_data":13472,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94313296,"statfs":{"total":96636764160,"available":96576815104,"internally_reserved":0,"allocated":13795328,"data_stored":103378431,"data_compressed":2311080,"data_compressed_allocated":9502720,"data_compressed_original":101103314,"omap_allocated":1592,"internal_metadata":46135752},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:00:05 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.378,"5min":0.315,"15min":0.315},"max":{"1min":1.0109999999999999,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.71199999999999997,"15min":0.71999999999999997},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.89200000000000002,"5min":1.919,"15min":1.919},"last":0.66100000000000003}]},{"osd":1,"last update":"Sat Sep 21 15:00:05 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.67500000000000004,"15min":0.67600000000000005},"min":{"1min":0.33400000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":1.1240000000000001,"5min":1.135,"15min":1.135},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.80700000000000005,"15min":0.81699999999999995},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.1599999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.753}]},{"osd":2,"last update":"Sat Sep 21 15:00:05 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.72799999999999998,"15min":0.72399999999999998},"min":{"1min":0.40200000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.83399999999999996,"15min":0.83499999999999996},"min":{"1min":0.47499999999999998,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.1759999999999999,"5min":1.254,"15min":1.254},"last":0.82299999999999995}]},{"osd":4,"last update":"Sat Sep 21 15:00:05 2024","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.95699999999999996,"15min":0.96299999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.2709999999999999,"5min":1.7609999999999999,"15min":1.7609999999999999},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.91100000000000003,"5min":1.0189999999999999,"15min":1.0369999999999999},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.25,"5min":2.028,"15min":2.028},"last":0.85799999999999998}]},{"osd":5,"last update":"Sat Sep 21 15:00:05 2024","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.052,"15min":1.0569999999999999},"min":{"1min":0.77700000000000002,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.246,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.022},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.0649999999999999,"15min":1.0660000000000001},"min":{"1min":0.70799999999999996,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.3600000000000001,"5min":1.431,"15min":1.431},"last":0.95699999999999996}]},{"osd":6,"last update":"Sat Sep 21 15:00:05 2024","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.0860000000000001,"15min":1.0980000000000001},"min":{"1min":0.58599999999999997,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.3879999999999999,"5min":2.1219999999999999,"15min":2.1219999999999999},"last":0.97999999999999998},{"interface":"front","average":{"1min":1.046,"5min":1.097,"15min":1.105},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.7370000000000001,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":1.1100000000000001}]},{"osd":7,"last update":"Sat Sep 21 15:00:05 2024","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.1279999999999999,"15min":1.139},"min":{"1min":0.61799999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.4870000000000001,"5min":1.706,"15min":1.706},"last":1.0469999999999999},{"interface":"front","average":{"1min":1.093,"5min":1.173,"15min":1.1859999999999999},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.516,"5min":2.3380000000000001,"15min":2.3380000000000001},"last":1.073}]}]},{"osd":2,"up_from":14,"seq":60129542183,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64844,"kb_used_data":11584,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94306996,"statfs":{"total":96636764160,"available":96570363904,"internally_reserved":0,"allocated":11862016,"data_stored":89535081,"data_compressed":2014763,"data_compressed_allocated":8007680,"data_compressed_original":87553058,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.62},"min":{"1min":0.26900000000000002,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.92800000000000005,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.69399999999999995,"15min":0.67000000000000004},"min":{"1min":0.34799999999999998,"5min":0.223,"15min":0.223},"max":{"1min":2.9169999999999998,"5min":2.9169999999999998,"15min":2.9169999999999998},"last":0.41299999999999998}]},{"osd":1,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.32700000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.0529999999999999,"5min":2.6520000000000001,"15min":2.6520000000000001},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.746,"5min":0.73499999999999999,"15min":0.75800000000000001},"min":{"1min":0.40500000000000003,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.349,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.69699999999999995}]},{"osd":3,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.73999999999999999,"15min":0.73499999999999999},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":3.0110000000000001,"5min":3.0110000000000001,"15min":3.0110000000000001},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.877,"15min":0.83899999999999997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.05,"5min":4.1440000000000001,"15min":4.1440000000000001},"last":0.47599999999999998}]},{"osd":4,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.93200000000000005,"15min":0.95599999999999996},"min":{"1min":0.42399999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.222,"5min":2.899,"15min":2.899},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.94399999999999995,"15min":0.95799999999999996},"min":{"1min":0.47499999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.4379999999999999,"5min":1.675,"15min":1.675},"last":0.73599999999999999}]},{"osd":5,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.0129999999999999,"15min":1.0089999999999999},"min":{"1min":0.45500000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":3.1080000000000001,"5min":3.1080000000000001,"15min":3.1080000000000001},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.98199999999999998,"15min":0.99399999999999999},"min":{"1min":0.51700000000000002,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.268,"5min":1.546,"15min":1.546},"last":0.51700000000000002}]},{"osd":6,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":0.98299999999999998,"15min":0.98499999999999999},"min":{"1min":0.56200000000000006,"5min":0.378,"15min":0.378},"max":{"1min":1.345,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":0.77700000000000002},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.0649999999999999,"15min":1.0549999999999999},"min":{"1min":0.53200000000000003,"5min":0.502,"15min":0.502},"max":{"1min":3.0739999999999998,"5min":3.0739999999999998,"15min":3.0739999999999998},"last":0.83799999999999997}]},{"osd":7,"last update":"Sat Sep 21 15:00:07 2024","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":1.0029999999999999,"15min":0.995},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.88700000000000001},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.097,"15min":1.091},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.5489999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":0.66000000000000003}]}]},{"osd":1,"up_from":14,"seq":60129542184,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64836,"kb_used_data":11572,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94307004,"statfs":{"total":96636764160,"available":96570372096,"internally_reserved":0,"allocated":11849728,"data_stored":91303543,"data_compressed":1932797,"data_compressed_allocated":8060928,"data_compressed_original":89352649,"omap_allocated":1595,"internal_metadata":54524357},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.59599999999999997,"15min":0.60899999999999999},"min":{"1min":0.36399999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.78200000000000003,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59699999999999998,"15min":0.60799999999999998},"min":{"1min":0.39500000000000002,"5min":0.216,"15min":0.216},"max":{"1min":0.80100000000000005,"5min":1.107,"15min":1.107},"last":0.39500000000000002}]},{"osd":2,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":0.79600000000000004,"15min":0.73999999999999999},"min":{"1min":0.38700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":11.303000000000001,"5min":11.303000000000001,"15min":11.303000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.71299999999999997,"15min":0.70299999999999996},"min":{"1min":0.47299999999999998,"5min":0.435,"15min":0.435},"max":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.81599999999999995}]},{"osd":3,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.81799999999999995,"15min":0.83899999999999997},"min":{"1min":0.27800000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.0049999999999999,"5min":3.1469999999999998,"15min":3.1469999999999998},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.86399999999999999,"15min":0.88600000000000001},"min":{"1min":0.52800000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.306,"5min":3.2320000000000002,"15min":3.2320000000000002},"last":0.94799999999999995}]},{"osd":4,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.89000000000000001,"15min":0.91500000000000004},"min":{"1min":0.59599999999999997,"5min":0.309,"15min":0.309},"max":{"1min":1.45,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":1.5600000000000001,"5min":1.069,"15min":0.96599999999999997},"min":{"1min":0.504,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":11.563000000000001,"5min":11.563000000000001,"15min":11.563000000000001},"last":1.0629999999999999}]},{"osd":5,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":1.625,"5min":1.0469999999999999,"15min":0.94899999999999995},"min":{"1min":0.73399999999999999,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":11.534000000000001,"5min":11.534000000000001,"15min":11.534000000000001},"last":1.121},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95799999999999996,"15min":0.96099999999999997},"min":{"1min":0.63,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.256,"5min":1.484,"15min":1.484},"last":1.256}]},{"osd":6,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":1.0109999999999999,"15min":1.0489999999999999},"min":{"1min":0.60399999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.4059999999999999,"5min":2.3519999999999999,"15min":2.3519999999999999},"last":0.85099999999999998},{"interface":"front","average":{"1min":1.631,"5min":1.081,"15min":0.99299999999999999},"min":{"1min":0.57399999999999995,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":11.601000000000001,"5min":11.601000000000001,"15min":11.601000000000001},"last":1.1639999999999999}]},{"osd":7,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":1.046,"15min":1.0640000000000001},"min":{"1min":0.79300000000000004,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.5820000000000001,"5min":2.2360000000000002,"15min":2.2360000000000002},"last":0.79300000000000004},{"interface":"front","average":{"1min":1.76,"5min":1.171,"15min":1.087},"min":{"1min":0.877,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":11.489000000000001,"5min":11.489000000000001,"15min":11.489000000000001},"last":1.23}]}]},{"osd":0,"up_from":14,"seq":60129542183,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70176,"kb_used_data":12812,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94301664,"statfs":{"total":96636764160,"available":96564903936,"internally_reserved":0,"allocated":13119488,"data_stored":98239443,"data_compressed":2190723,"data_compressed_allocated":9064448,"data_compressed_original":96141885,"omap_allocated":1593,"internal_metadata":58718663},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 21 15:01:04 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.67500000000000004,"15min":0.71699999999999997},"min":{"1min":0.40999999999999998,"5min":0.309,"15min":0.309},"max":{"1min":1.1299999999999999,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.68100000000000005,"15min":0.627},"min":{"1min":0.58999999999999997,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.67800000000000005}]},{"osd":2,"last update":"Sat Sep 21 15:01:04 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.65200000000000002,"15min":0.61799999999999999},"min":{"1min":0.59599999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.72499999999999998,"15min":0.75800000000000001},"min":{"1min":0.39300000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.79200000000000004}]},{"osd":3,"last update":"Sat Sep 21 15:01:04 2024","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.76600000000000001,"15min":0.754},"min":{"1min":0.50900000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.78700000000000003,"15min":0.81499999999999995},"min":{"1min":0.50900000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.50900000000000001}]},{"osd":4,"last update":"Sat Sep 21 15:01:04 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.98399999999999999,"15min":1.038},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.1140000000000001,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":1.036,"5min":1.0469999999999999,"15min":1.0620000000000001},"min":{"1min":0.67800000000000005,"5min":0.379,"15min":0.379},"max":{"1min":1.446,"5min":1.665,"15min":1.665},"last":0.71199999999999997}]},{"osd":5,"last update":"Sat Sep 21 15:01:04 2024","interfaces":[{"interface":"back","average":{"1min":1.0960000000000001,"5min":1.024,"15min":1.032},"min":{"1min":0.79300000000000004,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.4450000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.83899999999999997},{"interface":"front","average":{"1min":1.0589999999999999,"5min":1.0429999999999999,"15min":1.077},"min":{"1min":0.82399999999999995,"5min":0.435,"15min":0.435},"max":{"1min":1.2609999999999999,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":0.95499999999999996}]},{"osd":6,"last update":"Sat Sep 21 15:01:04 2024","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":1.0580000000000001,"15min":1.1120000000000001},"min":{"1min":0.68400000000000005,"5min":0.628,"15min":0.628},"max":{"1min":1.167,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":0.85799999999999998},{"interface":"front","average":{"1min":1.109,"5min":1.0629999999999999,"15min":1.0680000000000001},"min":{"1min":0.91000000000000003,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.5029999999999999,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.91900000000000004}]},{"osd":7,"last update":"Sat Sep 21 15:01:04 2024","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.0469999999999999,"15min":1.0620000000000001},"min":{"1min":0.78700000000000003,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.4770000000000001,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":0.88500000000000001},{"interface":"front","average":{"1min":1.02,"5min":1.085,"15min":1.131},"min":{"1min":0.84799999999999998,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.403,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":0.93600000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5202,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5202,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":11001856,"data_stored":97211967,"data_compressed":2190723,"data_compressed_allocated":9064448,"data_compressed_original":96141885,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9662464,"data_stored":89679641,"data_compressed":1927595,"data_compressed_allocated":8024064,"data_compressed_original":88779209,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":9732096,"data_stored":88501547,"data_compressed":2014763,"data_compressed_allocated":8007680,"data_compressed_original":87553058,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":11673600,"data_stored":102350936,"data_compressed":2311080,"data_compressed_allocated":9502720,"data_compressed_original":101103314,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8458240,"data_stored":78022006,"data_compressed":1750025,"data_compressed_allocated":7159808,"data_compressed_original":77331143,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":7426048,"data_stored":63507777,"data_compressed":1266980,"data_compressed_allocated":5820416,"data_compressed_original":62572652,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8044544,"data_stored":70323978,"data_compressed":1591208,"data_compressed_allocated":6516736,"data_compressed_original":69451532,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":5652480,"data_stored":47867613,"data_compressed":1141276,"data_compressed_allocated":4558848,"data_compressed_original":47268443,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T15:01:11.092 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-21T15:01:11.118 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:01:11.113+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:01:11.135 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:01:11.136 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:01:11.136 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T15:01:11.136 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:01:11.146 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T15:01:11.219 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:01:11.217+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:01:11.253 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:01:11.253 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:01:11.253 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T15:01:11.253 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T15:01:11.253 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:01:11.262 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T15:01:11.273 INFO:teuthology.orchestra.run.smithi032.stderr:noscrub is unset 2024-09-21T15:01:11.320 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:01:11.314+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:01:11.368 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:01:11.369 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T15:01:11.369 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T15:01:11.369 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T15:01:11.369 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.369 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45854 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:11.369 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159802+0000", 2024-09-21T15:01:11.369 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 121.20468052699999, 2024-09-21T15:01:11.369 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068358938999999994, 2024-09-21T15:01:11.369 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:11.369 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:11.369 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:11.369 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:11.369 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:11.370 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45854 2024-09-21T15:01:11.370 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.370 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:11.370 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.370 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:11.370 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159802+0000", 2024-09-21T15:01:11.370 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:11.370 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.370 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.370 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:11.370 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159802+0000", 2024-09-21T15:01:11.370 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:11.370 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.370 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.371 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:11.371 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159801+0000", 2024-09-21T15:01:11.371 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:11.371 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.371 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.371 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:11.371 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159804+0000", 2024-09-21T15:01:11.371 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0970000000000002e-06 2024-09-21T15:01:11.371 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.371 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.371 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:11.371 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159823+0000", 2024-09-21T15:01:11.371 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8895000000000001e-05 2024-09-21T15:01:11.372 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.372 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.372 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:11.372 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159837+0000", 2024-09-21T15:01:11.372 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3247999999999999e-05 2024-09-21T15:01:11.372 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.372 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.372 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:11.372 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228151+0000", 2024-09-21T15:01:11.372 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068314642999999994 2024-09-21T15:01:11.372 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.372 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.372 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:11.372 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228161+0000", 2024-09-21T15:01:11.373 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4250000000000004e-06 2024-09-21T15:01:11.373 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.373 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:11.373 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.373 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.373 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.373 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45985 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:11.373 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159897+0000", 2024-09-21T15:01:11.373 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 121.204585025, 2024-09-21T15:01:11.373 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068345664, 2024-09-21T15:01:11.373 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:11.373 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:11.373 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:11.374 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:11.374 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:11.374 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45985 2024-09-21T15:01:11.374 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.374 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:11.374 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.374 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:11.374 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159897+0000", 2024-09-21T15:01:11.374 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:11.374 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.374 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.374 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:11.374 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159897+0000", 2024-09-21T15:01:11.374 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:11.375 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.375 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.375 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:11.375 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159897+0000", 2024-09-21T15:01:11.375 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:11.375 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.375 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.375 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:11.375 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159900+0000", 2024-09-21T15:01:11.375 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.954e-06 2024-09-21T15:01:11.375 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.375 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.375 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:11.375 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159913+0000", 2024-09-21T15:01:11.376 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3106e-05 2024-09-21T15:01:11.376 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.376 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.376 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:11.376 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159918+0000", 2024-09-21T15:01:11.376 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5199999999999997e-06 2024-09-21T15:01:11.376 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.376 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.376 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:11.376 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228236+0000", 2024-09-21T15:01:11.376 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068317040999999995 2024-09-21T15:01:11.376 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.376 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.376 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:11.376 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228243+0000", 2024-09-21T15:01:11.377 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3880000000000002e-06 2024-09-21T15:01:11.377 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.377 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:11.377 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.377 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.377 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.377 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45986 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:11.377 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159924+0000", 2024-09-21T15:01:11.377 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 121.20455873, 2024-09-21T15:01:11.377 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068364362999999997, 2024-09-21T15:01:11.377 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:11.377 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:11.377 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:11.377 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:11.378 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:11.378 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45986 2024-09-21T15:01:11.378 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.378 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:11.378 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.378 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:11.378 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159924+0000", 2024-09-21T15:01:11.378 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:11.378 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.378 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.378 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:11.378 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159924+0000", 2024-09-21T15:01:11.378 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:11.378 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.379 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.379 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:11.379 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159923+0000", 2024-09-21T15:01:11.379 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:11.379 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.379 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.379 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:11.379 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159926+0000", 2024-09-21T15:01:11.379 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0510000000000001e-06 2024-09-21T15:01:11.379 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.379 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.379 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:11.379 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159932+0000", 2024-09-21T15:01:11.379 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4160000000000001e-06 2024-09-21T15:01:11.380 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.380 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.380 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:11.380 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159938+0000", 2024-09-21T15:01:11.380 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.383e-06 2024-09-21T15:01:11.380 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.380 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.380 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:11.380 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228275+0000", 2024-09-21T15:01:11.380 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068337007000000005 2024-09-21T15:01:11.380 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.380 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.380 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:11.380 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228288+0000", 2024-09-21T15:01:11.380 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3052e-05 2024-09-21T15:01:11.381 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.381 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:11.381 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.381 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.381 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.381 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45987 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:11.381 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159944+0000", 2024-09-21T15:01:11.381 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 121.20453850600001, 2024-09-21T15:01:11.381 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068387232000000006, 2024-09-21T15:01:11.381 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:11.381 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:11.381 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:11.381 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:11.381 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:11.381 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45987 2024-09-21T15:01:11.382 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.382 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:11.382 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.382 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:11.382 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159944+0000", 2024-09-21T15:01:11.382 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:11.382 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.382 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.382 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:11.382 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159944+0000", 2024-09-21T15:01:11.382 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:11.382 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.382 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.382 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:11.382 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159943+0000", 2024-09-21T15:01:11.383 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:11.383 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.383 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.383 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:11.383 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159946+0000", 2024-09-21T15:01:11.383 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6309999999999999e-06 2024-09-21T15:01:11.383 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.383 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.383 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:11.383 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159952+0000", 2024-09-21T15:01:11.383 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6540000000000004e-06 2024-09-21T15:01:11.383 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.383 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.384 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:11.384 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159957+0000", 2024-09-21T15:01:11.384 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9280000000000001e-06 2024-09-21T15:01:11.384 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.384 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.384 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:11.384 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228322+0000", 2024-09-21T15:01:11.384 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068365067000000002 2024-09-21T15:01:11.384 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.384 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.384 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:11.384 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228331+0000", 2024-09-21T15:01:11.384 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3270000000000007e-06 2024-09-21T15:01:11.384 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.384 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:11.385 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.385 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.385 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.385 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:97672 3.4 3:26046d29:::rbd_data.1136bfc79a70.0000000000000001:head [write 0~4718592 [fadvise_sequential+fadvise_nocache],write 5242880~131072 [fadvise_sequential+fadvise_nocache],write 5439488~147456 [fadvise_sequential+fadvise_nocache],write 5652480~229376 [fadvise_sequential+fadvise_nocache],write 5947392~147456 [fadvise_sequential+fadvise_nocache],write 6160384~229376 [fadvise_sequential+fadvise_nocache],write 6455296~147456 [fadvise_sequential+fadvise_nocache],write 6668288~229376 [fadvise_sequential+fadvise_nocache],write 6963200~147456 [fadvise_sequential+fadvise_nocache],write 7176192~147456 [fadvise_sequential+fadvise_nocache],write 7340032~65536 [fadvise_sequential+fadvise_nocache],write 7471104~147456 [fadvise_sequential+fadvise_nocache],write 7684096~147456 [fadvise_sequential+fadvise_nocache],write 7864320~49152 [fadvise_sequential+fadvise_nocache],write 7979008~147456 [fadvise_sequential+fadvise_nocache],write 8192000~147456 [fadvise_sequential+fadvise_nocache],write 8388608~32768 [fadvise_sequential+fadvise_nocache],write 8519680~114688 [fadvise_sequential+fadvise_nocache],write 8732672~114688 [fadvise_sequential+fadvise_nocache],write 8912896~16384 [fadvise_sequential+fadvise_nocache],write 9109504~32768 [fadvise_sequential+fadvise_nocache],write 9322496~32768 [fadvise_sequential+fadvise_nocache],write 9617408~32768 [fadvise_sequential+fadvise_nocache],write 9830400~32768 [fadvise_sequential+fadvise_nocache],write 9961472~98304 [fadvise_sequential+fadvise_nocache],write 10076160~196608 [fadvise_sequential+fadvise_nocache],write 10289152~278528 [fadvise_sequential+fadvise_nocache],write 10584064~196608 [fadvise_sequential+fadvise_nocache],write 10797056~196608 [fadvise_sequential+fadvise_nocache],write 11010048~65536 [fadvise_sequential+fadvise_nocache],write 11091968~196608 [fadvise_sequential+fadvise_nocache],write 11304960~196608 [fadvise_sequential+fadvise_nocache],write 11517952~65536 [fadvise_sequential+fadvise_nocache],write 11616256~98304 [fadvise_sequential+fadvise_nocache],write 11730944~65536 [fadvise_sequential+fadvise_nocache],write 11829248~98304 [fadvise_sequential+fadvise_nocache],write 11943936~65536 [fadvise_sequential+fadvise_nocache],write 12042240~49152 [fadvise_sequential+fadvise_nocache],write 12124160~98304 [fadvise_sequential+fadvise_nocache],write 12238848~65536 [fadvise_sequential+fadvise_nocache],write 12337152~98304 [fadvise_sequential+fadvise_nocache],write 12451840~65536 [fadvise_sequential+fadvise_nocache],write 12550144~32768 [fadvise_sequential+fadvise_nocache],write 12632064~98304 [fadvise_sequential+fadvise_nocache],write 12845056~98304 [fadvise_sequential+fadvise_nocache],write 13058048~49152 [fadvise_sequential+fadvise_nocache],write 13139968~98304 [fadvise_sequential+fadvise_nocache],write 13352960~98304 [fadvise_sequential+fadvise_nocache],write 13565952~65536 [fadvise_sequential+fadvise_nocache],write 13647872~98304 [fadvise_sequential+fadvise_nocache],write 13860864~98304 [fadvise_sequential+fadvise_nocache],write 14073856~81920 [fadvise_sequential+fadvise_nocache],write 15269888~81920 [fadvise_sequential+fadvise_nocache],write 15482880~81920 [fadvise_sequential+fadvise_nocache],write 15695872~32768 [fadvise_sequential+fadvise_nocache],write 15777792~81920 [fadvise_sequential+fadvise_nocache],write 15990784~81920 [fadvise_sequential+fadvise_nocache],write 16203776~49152 [fadvise_sequential+fadvise_nocache],write 16285696~81920 [fadvise_sequential+fadvise_nocache],write 16498688~81920 [fadvise_sequential+fadvise_nocache],write 16711680~65536 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:01:11.385 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.298669+0000", 2024-09-21T15:01:11.385 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 66.065813439999999, 2024-09-21T15:01:11.385 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.10282358900000001, 2024-09-21T15:01:11.385 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:11.385 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:11.385 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:11.385 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:11.385 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:11.385 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97672 2024-09-21T15:01:11.385 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.386 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:11.386 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.386 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:11.386 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.298669+0000", 2024-09-21T15:01:11.386 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:11.386 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.386 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.386 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:11.386 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.298669+0000", 2024-09-21T15:01:11.386 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:11.386 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.386 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.386 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:11.386 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.298667+0000", 2024-09-21T15:01:11.387 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:01:11.387 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.387 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.387 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:11.387 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.327458+0000", 2024-09-21T15:01:11.387 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.028790754000000002 2024-09-21T15:01:11.387 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.387 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.387 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:11.387 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.327492+0000", 2024-09-21T15:01:11.387 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3806999999999997e-05 2024-09-21T15:01:11.387 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.387 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.387 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:11.388 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.327507+0000", 2024-09-21T15:01:11.388 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5112e-05 2024-09-21T15:01:11.388 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.388 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.388 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:11.388 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.327597+0000", 2024-09-21T15:01:11.388 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9938999999999994e-05 2024-09-21T15:01:11.388 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.388 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.388 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:11.388 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.327927+0000", 2024-09-21T15:01:11.388 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000329766 2024-09-21T15:01:11.388 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.388 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.388 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T15:01:11.388 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.330735+0000", 2024-09-21T15:01:11.389 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0028086869999999998 2024-09-21T15:01:11.389 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.389 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.389 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:01:11.389 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.378299+0000", 2024-09-21T15:01:11.389 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.047563495999999997 2024-09-21T15:01:11.389 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.389 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.389 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:01:11.389 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.401248+0000", 2024-09-21T15:01:11.389 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.022949311 2024-09-21T15:01:11.389 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.389 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.389 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:01:11.389 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.401446+0000", 2024-09-21T15:01:11.390 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019831199999999999 2024-09-21T15:01:11.390 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.390 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.390 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:11.390 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.401492+0000", 2024-09-21T15:01:11.390 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6109000000000002e-05 2024-09-21T15:01:11.390 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.390 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:11.390 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.390 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.390 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.390 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139769 3.1e 3:793dcf44:::rbd_data.113664780c85.0000000000000000:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:11.390 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.221432+0000", 2024-09-21T15:01:11.390 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.1430505709999998, 2024-09-21T15:01:11.390 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.035363362, 2024-09-21T15:01:11.391 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:11.391 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:11.391 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:11.391 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:11.391 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:11.391 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139769 2024-09-21T15:01:11.391 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.391 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:11.391 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.391 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:11.391 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221432+0000", 2024-09-21T15:01:11.391 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:11.391 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.391 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.392 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:11.392 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221432+0000", 2024-09-21T15:01:11.392 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:11.392 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.392 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.392 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:11.392 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221430+0000", 2024-09-21T15:01:11.392 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:01:11.392 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.392 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.392 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:11.392 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221438+0000", 2024-09-21T15:01:11.392 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2630000000000003e-06 2024-09-21T15:01:11.393 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.393 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.393 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:11.393 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221458+0000", 2024-09-21T15:01:11.393 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0071000000000001e-05 2024-09-21T15:01:11.393 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.393 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.393 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:11.393 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221467+0000", 2024-09-21T15:01:11.393 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.816e-06 2024-09-21T15:01:11.393 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.393 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.393 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:11.393 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221572+0000", 2024-09-21T15:01:11.393 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000105166 2024-09-21T15:01:11.394 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.394 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.394 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:11.394 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221589+0000", 2024-09-21T15:01:11.394 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7391e-05 2024-09-21T15:01:11.394 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.394 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.394 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:11.394 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.255741+0000", 2024-09-21T15:01:11.394 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0341519690000001 2024-09-21T15:01:11.394 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.394 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.394 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:11.394 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.256353+0000", 2024-09-21T15:01:11.395 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000611544 2024-09-21T15:01:11.395 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.395 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.395 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:11.395 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.256795+0000", 2024-09-21T15:01:11.395 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00044208700000000002 2024-09-21T15:01:11.395 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.395 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:11.395 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.395 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.395 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.395 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139773 3.1e 3:7fa3054f:::rbd_data.113664780c85.000000000000001c:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:11.395 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.221517+0000", 2024-09-21T15:01:11.395 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.1429655269999994, 2024-09-21T15:01:11.395 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0367423140000001, 2024-09-21T15:01:11.396 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:11.396 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:11.396 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:11.396 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:11.396 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:11.396 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139773 2024-09-21T15:01:11.396 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.396 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:11.396 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.396 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:11.396 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221517+0000", 2024-09-21T15:01:11.396 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:11.396 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.396 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.397 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:11.397 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221517+0000", 2024-09-21T15:01:11.397 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:11.397 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.397 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.397 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:11.397 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221516+0000", 2024-09-21T15:01:11.397 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:11.397 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.397 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.397 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:11.397 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221520+0000", 2024-09-21T15:01:11.397 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1030000000000004e-06 2024-09-21T15:01:11.397 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.397 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.397 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:11.398 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221566+0000", 2024-09-21T15:01:11.398 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5512000000000001e-05 2024-09-21T15:01:11.398 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.398 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.398 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:11.398 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221575+0000", 2024-09-21T15:01:11.398 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9630000000000004e-06 2024-09-21T15:01:11.398 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.398 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.398 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:11.398 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221650+0000", 2024-09-21T15:01:11.398 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5531999999999994e-05 2024-09-21T15:01:11.398 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.398 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.398 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:11.399 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221661+0000", 2024-09-21T15:01:11.399 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0878e-05 2024-09-21T15:01:11.399 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.399 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.399 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:11.399 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.257513+0000", 2024-09-21T15:01:11.399 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0358513089999999 2024-09-21T15:01:11.399 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.399 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.399 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:11.399 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.257885+0000", 2024-09-21T15:01:11.399 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00037274299999999998 2024-09-21T15:01:11.399 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.399 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.399 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:11.400 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.258259+0000", 2024-09-21T15:01:11.400 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000373829 2024-09-21T15:01:11.400 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.400 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:11.400 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.400 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.400 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.400 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139789 3.1e 3:7c803367:::rbd_data.113664780c85.000000000000008c:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:11.400 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.221656+0000", 2024-09-21T15:01:11.400 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.1428258959999997, 2024-09-21T15:01:11.400 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.037139595, 2024-09-21T15:01:11.400 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:11.400 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:11.401 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:11.401 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:11.401 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:11.401 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139789 2024-09-21T15:01:11.401 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.401 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:11.401 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.401 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:11.401 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221656+0000", 2024-09-21T15:01:11.401 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:11.401 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.401 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.401 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:11.401 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221656+0000", 2024-09-21T15:01:11.401 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:11.402 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.402 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.402 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:11.402 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221656+0000", 2024-09-21T15:01:11.402 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:11.402 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.402 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.402 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:11.402 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221660+0000", 2024-09-21T15:01:11.402 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0369999999999996e-06 2024-09-21T15:01:11.402 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.402 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.402 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:11.402 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221668+0000", 2024-09-21T15:01:11.402 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3440000000000001e-06 2024-09-21T15:01:11.402 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.403 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.403 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:11.403 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221674+0000", 2024-09-21T15:01:11.403 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0920000000000003e-06 2024-09-21T15:01:11.403 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.403 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.403 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:11.403 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221787+0000", 2024-09-21T15:01:11.403 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000112289 2024-09-21T15:01:11.403 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.403 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.403 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:11.403 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221798+0000", 2024-09-21T15:01:11.403 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1419e-05 2024-09-21T15:01:11.403 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.404 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.404 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:11.404 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.258642+0000", 2024-09-21T15:01:11.404 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.036843851 2024-09-21T15:01:11.404 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.404 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.404 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:11.404 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.258796+0000", 2024-09-21T15:01:11.404 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015414199999999999 2024-09-21T15:01:11.404 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.404 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:11.404 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.404 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.404 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.404 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139832 3.1e 3:7b675eeb:::rbd_data.113664780c85.00000000000001b3:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:11.405 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.223976+0000", 2024-09-21T15:01:11.405 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.1405061629999995, 2024-09-21T15:01:11.405 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0350267980000001, 2024-09-21T15:01:11.405 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:11.405 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:11.405 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:11.405 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:11.405 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:11.405 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139832 2024-09-21T15:01:11.405 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.405 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:11.405 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.405 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:11.405 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223976+0000", 2024-09-21T15:01:11.406 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:11.406 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.406 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.406 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:11.406 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223976+0000", 2024-09-21T15:01:11.406 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:11.406 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.406 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.406 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:11.406 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223976+0000", 2024-09-21T15:01:11.406 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:11.406 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.406 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.406 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:11.406 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223981+0000", 2024-09-21T15:01:11.407 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1560000000000003e-06 2024-09-21T15:01:11.407 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.407 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.407 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:11.407 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223999+0000", 2024-09-21T15:01:11.407 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8808e-05 2024-09-21T15:01:11.407 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.407 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.407 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:11.407 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224008+0000", 2024-09-21T15:01:11.407 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7269999999999996e-06 2024-09-21T15:01:11.407 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.407 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.407 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:11.407 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224208+0000", 2024-09-21T15:01:11.408 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000199614 2024-09-21T15:01:11.408 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.408 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.408 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:11.408 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224218+0000", 2024-09-21T15:01:11.408 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0209e-05 2024-09-21T15:01:11.408 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.408 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.408 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:11.408 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.258839+0000", 2024-09-21T15:01:11.408 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.034621083 2024-09-21T15:01:11.408 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.408 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.408 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:11.408 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.259003+0000", 2024-09-21T15:01:11.408 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016379 2024-09-21T15:01:11.409 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.409 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:11.409 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.409 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.409 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.409 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139900 3.1e 3:7b440264:::rbd_data.113664780c85.0000000000000013:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:11.409 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.225748+0000", 2024-09-21T15:01:11.409 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.1387341620000004, 2024-09-21T15:01:11.409 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0339966009999999, 2024-09-21T15:01:11.409 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:11.409 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:11.409 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:11.409 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:11.409 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:11.409 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139900 2024-09-21T15:01:11.410 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.410 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:11.410 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.410 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:11.410 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.225748+0000", 2024-09-21T15:01:11.410 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:11.410 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.410 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.410 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:11.410 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.225748+0000", 2024-09-21T15:01:11.410 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:11.410 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.410 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.410 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:11.411 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.225746+0000", 2024-09-21T15:01:11.411 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:11.411 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.411 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.411 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:11.411 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.225757+0000", 2024-09-21T15:01:11.411 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.066e-05 2024-09-21T15:01:11.411 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.411 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.411 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:11.411 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.225775+0000", 2024-09-21T15:01:11.411 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7631000000000001e-05 2024-09-21T15:01:11.411 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.411 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.411 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:11.412 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.225788+0000", 2024-09-21T15:01:11.412 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3046e-05 2024-09-21T15:01:11.412 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.412 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.412 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:11.412 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.225881+0000", 2024-09-21T15:01:11.412 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2987999999999998e-05 2024-09-21T15:01:11.412 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.412 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.412 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:11.412 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.225893+0000", 2024-09-21T15:01:11.412 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2398999999999999e-05 2024-09-21T15:01:11.412 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.412 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.412 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:11.413 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.259207+0000", 2024-09-21T15:01:11.413 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.033313696 2024-09-21T15:01:11.413 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.413 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.413 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:11.413 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.259452+0000", 2024-09-21T15:01:11.413 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00024495700000000002 2024-09-21T15:01:11.413 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.413 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.413 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:11.413 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.259745+0000", 2024-09-21T15:01:11.413 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000292868 2024-09-21T15:01:11.413 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.413 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:11.413 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.413 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.414 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.414 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140008 3.1e 3:7cd8b476:::rbd_data.113664780c85.0000000000000149:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:11.414 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.229555+0000", 2024-09-21T15:01:11.414 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.1349272979999991, 2024-09-21T15:01:11.414 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0304914199999999, 2024-09-21T15:01:11.414 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:11.414 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:11.414 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:11.414 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:11.414 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:11.414 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140008 2024-09-21T15:01:11.414 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.414 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:11.414 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.415 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:11.415 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.229555+0000", 2024-09-21T15:01:11.415 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:11.415 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.415 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.415 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:11.415 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.229555+0000", 2024-09-21T15:01:11.415 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:11.415 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.415 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.415 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:11.415 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.229554+0000", 2024-09-21T15:01:11.415 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:11.415 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.415 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.415 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:11.416 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.229559+0000", 2024-09-21T15:01:11.416 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1059999999999999e-06 2024-09-21T15:01:11.416 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.416 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.416 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:11.416 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.229570+0000", 2024-09-21T15:01:11.416 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0465000000000001e-05 2024-09-21T15:01:11.416 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.416 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.416 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:11.416 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.229581+0000", 2024-09-21T15:01:11.416 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0958000000000001e-05 2024-09-21T15:01:11.416 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.416 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.416 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:11.417 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.229660+0000", 2024-09-21T15:01:11.417 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9103999999999993e-05 2024-09-21T15:01:11.417 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.417 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.417 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:11.417 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.229675+0000", 2024-09-21T15:01:11.417 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5622999999999999e-05 2024-09-21T15:01:11.417 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.417 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.417 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:11.417 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.259816+0000", 2024-09-21T15:01:11.417 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.030140595 2024-09-21T15:01:11.417 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.417 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.417 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:11.418 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.260046+0000", 2024-09-21T15:01:11.418 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00023045300000000001 2024-09-21T15:01:11.418 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.418 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:11.418 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.418 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.418 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.418 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140037 3.1e 3:7a4585d9:::rbd_data.113664780c85.000000000000005c:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:11.418 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.230439+0000", 2024-09-21T15:01:11.418 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.1340428100000004, 2024-09-21T15:01:11.418 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.030277613, 2024-09-21T15:01:11.418 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:11.418 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:11.418 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:11.419 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:11.419 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:11.419 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140037 2024-09-21T15:01:11.419 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.419 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:11.419 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.419 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:11.419 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.230439+0000", 2024-09-21T15:01:11.419 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:11.419 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.419 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.419 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:11.419 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.230439+0000", 2024-09-21T15:01:11.419 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:11.420 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.420 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.420 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:11.420 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.230438+0000", 2024-09-21T15:01:11.420 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:11.420 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.420 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.420 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:11.420 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.230447+0000", 2024-09-21T15:01:11.420 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6270000000000006e-06 2024-09-21T15:01:11.420 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.420 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.420 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:11.420 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.230464+0000", 2024-09-21T15:01:11.420 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7394e-05 2024-09-21T15:01:11.420 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.421 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.421 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:11.421 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.230472+0000", 2024-09-21T15:01:11.421 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0360000000000006e-06 2024-09-21T15:01:11.421 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.421 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.421 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:11.421 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.230567+0000", 2024-09-21T15:01:11.421 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4648000000000006e-05 2024-09-21T15:01:11.421 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.421 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.421 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:11.421 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.230581+0000", 2024-09-21T15:01:11.421 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3799e-05 2024-09-21T15:01:11.421 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.421 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.422 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:11.422 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.260095+0000", 2024-09-21T15:01:11.422 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.029514153 2024-09-21T15:01:11.422 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.422 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.422 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:11.422 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.260356+0000", 2024-09-21T15:01:11.422 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00026118400000000003 2024-09-21T15:01:11.422 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.422 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.422 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:11.422 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.260717+0000", 2024-09-21T15:01:11.422 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00036124799999999999 2024-09-21T15:01:11.422 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.423 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:11.423 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.423 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.423 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.423 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140061 3.1e 3:7fdd08f8:::rbd_data.113664780c85.0000000000000104:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:11.423 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.232026+0000", 2024-09-21T15:01:11.423 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.1324564800000001, 2024-09-21T15:01:11.423 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0290620450000001, 2024-09-21T15:01:11.423 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:11.423 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:11.423 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:11.423 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:11.423 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:11.423 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140061 2024-09-21T15:01:11.424 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.424 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:11.424 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.424 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:11.424 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232026+0000", 2024-09-21T15:01:11.424 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:11.424 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.424 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.424 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:11.424 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232026+0000", 2024-09-21T15:01:11.424 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:11.424 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.424 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.424 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:11.424 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232024+0000", 2024-09-21T15:01:11.425 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:01:11.425 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.425 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.425 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:11.425 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232034+0000", 2024-09-21T15:01:11.425 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0421000000000001e-05 2024-09-21T15:01:11.425 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.425 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.425 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:11.425 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232058+0000", 2024-09-21T15:01:11.425 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3944e-05 2024-09-21T15:01:11.425 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.425 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.425 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:11.425 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232071+0000", 2024-09-21T15:01:11.425 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2816999999999999e-05 2024-09-21T15:01:11.426 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.426 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.426 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:11.426 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232180+0000", 2024-09-21T15:01:11.426 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000109323 2024-09-21T15:01:11.426 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.426 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.426 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:11.426 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232199+0000", 2024-09-21T15:01:11.426 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8606000000000001e-05 2024-09-21T15:01:11.426 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.426 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.426 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:11.426 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.260821+0000", 2024-09-21T15:01:11.427 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0286222270000001 2024-09-21T15:01:11.427 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.427 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.427 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:11.427 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.261088+0000", 2024-09-21T15:01:11.427 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00026663799999999998 2024-09-21T15:01:11.427 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.427 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:11.427 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.427 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.427 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.427 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140090 3.1e 3:79f99893:::rbd_data.113664780c85.000000000000000b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:11.427 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.233668+0000", 2024-09-21T15:01:11.427 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.1308146150000002, 2024-09-21T15:01:11.428 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0281047409999999, 2024-09-21T15:01:11.428 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:11.428 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:11.428 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:11.428 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:11.428 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:11.428 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140090 2024-09-21T15:01:11.428 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.428 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:11.428 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.428 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:11.428 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233668+0000", 2024-09-21T15:01:11.428 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:11.428 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.428 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.428 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:11.429 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233668+0000", 2024-09-21T15:01:11.429 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:11.429 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.429 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.429 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:11.429 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233666+0000", 2024-09-21T15:01:11.429 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:11.429 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.429 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.429 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:11.429 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233673+0000", 2024-09-21T15:01:11.429 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1840000000000002e-06 2024-09-21T15:01:11.429 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.429 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.429 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:11.430 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233689+0000", 2024-09-21T15:01:11.430 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5366000000000002e-05 2024-09-21T15:01:11.430 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.430 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.430 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:11.430 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233697+0000", 2024-09-21T15:01:11.430 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7679999999999997e-06 2024-09-21T15:01:11.430 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.430 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.430 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:11.430 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233772+0000", 2024-09-21T15:01:11.430 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4832000000000005e-05 2024-09-21T15:01:11.430 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.431 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.431 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:11.431 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233790+0000", 2024-09-21T15:01:11.431 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7297999999999999e-05 2024-09-21T15:01:11.431 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.431 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.431 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:11.431 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.261145+0000", 2024-09-21T15:01:11.431 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.027355381 2024-09-21T15:01:11.431 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.431 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.431 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:11.431 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.261466+0000", 2024-09-21T15:01:11.432 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00032105300000000002 2024-09-21T15:01:11.432 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.432 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.432 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:11.432 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.261772+0000", 2024-09-21T15:01:11.432 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00030635100000000001 2024-09-21T15:01:11.432 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.432 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:11.432 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.432 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.432 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.432 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140128 3.1e 3:78baf82e:::rbd_data.113664780c85.0000000000000115:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:11.432 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.235029+0000", 2024-09-21T15:01:11.432 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.1294533609999995, 2024-09-21T15:01:11.432 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.02698942, 2024-09-21T15:01:11.432 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:11.433 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:11.433 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:11.433 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:11.433 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:11.433 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140128 2024-09-21T15:01:11.433 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.433 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:11.433 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.433 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:11.433 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235029+0000", 2024-09-21T15:01:11.433 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:11.433 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.433 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.433 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:11.433 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235029+0000", 2024-09-21T15:01:11.434 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:11.434 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.434 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.434 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:11.434 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235028+0000", 2024-09-21T15:01:11.434 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:11.434 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.434 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.434 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:11.434 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235032+0000", 2024-09-21T15:01:11.434 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0579999999999999e-06 2024-09-21T15:01:11.434 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.434 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.434 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:11.435 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235040+0000", 2024-09-21T15:01:11.435 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2099999999999993e-06 2024-09-21T15:01:11.435 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.435 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.435 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:11.435 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235047+0000", 2024-09-21T15:01:11.435 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1099999999999999e-06 2024-09-21T15:01:11.435 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.435 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.435 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:11.435 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235123+0000", 2024-09-21T15:01:11.435 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6835999999999994e-05 2024-09-21T15:01:11.435 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.435 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.435 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:11.436 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235136+0000", 2024-09-21T15:01:11.436 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2962000000000001e-05 2024-09-21T15:01:11.436 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.436 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.436 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:11.436 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.261827+0000", 2024-09-21T15:01:11.436 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0266906149999999 2024-09-21T15:01:11.436 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.436 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.436 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:11.436 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.262018+0000", 2024-09-21T15:01:11.436 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019132400000000001 2024-09-21T15:01:11.436 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.436 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:11.436 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.436 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.437 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.437 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140217 3.1e 3:793dcf44:::rbd_data.113664780c85.0000000000000000:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:11.437 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.235497+0000", 2024-09-21T15:01:11.437 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.1289850619999999, 2024-09-21T15:01:11.437 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0269053020000001, 2024-09-21T15:01:11.437 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:11.437 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:11.437 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:11.437 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:11.437 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:11.437 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140217 2024-09-21T15:01:11.437 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.437 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:11.437 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.438 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:11.438 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235497+0000", 2024-09-21T15:01:11.438 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:11.438 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.438 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.438 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:11.438 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235497+0000", 2024-09-21T15:01:11.438 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:11.438 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.438 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.438 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:11.438 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235496+0000", 2024-09-21T15:01:11.438 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:11.438 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.438 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.439 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:11.439 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235503+0000", 2024-09-21T15:01:11.439 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8460000000000002e-06 2024-09-21T15:01:11.439 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.439 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.439 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:11.439 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235514+0000", 2024-09-21T15:01:11.439 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0576e-05 2024-09-21T15:01:11.439 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.439 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.439 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:11.439 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235522+0000", 2024-09-21T15:01:11.439 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2640000000000008e-06 2024-09-21T15:01:11.439 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.439 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.440 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:11.440 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235579+0000", 2024-09-21T15:01:11.440 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6842999999999998e-05 2024-09-21T15:01:11.440 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.440 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.440 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:11.440 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235590+0000", 2024-09-21T15:01:11.440 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1194e-05 2024-09-21T15:01:11.440 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.440 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.440 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:11.440 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.262119+0000", 2024-09-21T15:01:11.440 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0265294540000001 2024-09-21T15:01:11.440 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.440 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.440 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:11.441 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.262241+0000", 2024-09-21T15:01:11.441 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012182 2024-09-21T15:01:11.441 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.441 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.441 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:11.441 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.262403+0000", 2024-09-21T15:01:11.441 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000161442 2024-09-21T15:01:11.441 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.441 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:11.441 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.441 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.441 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.441 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140221 3.1e 3:7fa3054f:::rbd_data.113664780c85.000000000000001c:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:11.441 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.235592+0000", 2024-09-21T15:01:11.441 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.1288902010000008, 2024-09-21T15:01:11.442 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.027322012, 2024-09-21T15:01:11.442 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:11.442 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:11.442 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:11.442 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:11.442 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:11.442 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140221 2024-09-21T15:01:11.442 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.442 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:11.442 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.442 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:11.442 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235592+0000", 2024-09-21T15:01:11.442 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:11.443 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.443 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.443 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:11.443 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235592+0000", 2024-09-21T15:01:11.443 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:11.443 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.443 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.443 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:11.443 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235591+0000", 2024-09-21T15:01:11.443 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:11.443 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.443 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.443 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:11.444 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235596+0000", 2024-09-21T15:01:11.444 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4209999999999997e-06 2024-09-21T15:01:11.444 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.444 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.444 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:11.444 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235603+0000", 2024-09-21T15:01:11.444 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.909e-06 2024-09-21T15:01:11.444 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.444 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.444 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:11.444 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235609+0000", 2024-09-21T15:01:11.444 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3539999999999999e-06 2024-09-21T15:01:11.444 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.444 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.444 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:11.445 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236013+0000", 2024-09-21T15:01:11.445 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00040399799999999998 2024-09-21T15:01:11.445 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.445 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.445 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:11.445 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236020+0000", 2024-09-21T15:01:11.445 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6699999999999994e-06 2024-09-21T15:01:11.445 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.445 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.445 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:11.445 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.262452+0000", 2024-09-21T15:01:11.445 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.02643145 2024-09-21T15:01:11.445 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.445 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.445 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:11.446 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.262624+0000", 2024-09-21T15:01:11.446 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000172206 2024-09-21T15:01:11.446 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.446 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.446 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:11.446 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.262914+0000", 2024-09-21T15:01:11.446 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00028994600000000002 2024-09-21T15:01:11.446 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.446 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:11.446 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.446 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.446 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.446 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140237 3.1e 3:7c803367:::rbd_data.113664780c85.000000000000008c:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:11.446 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.237113+0000", 2024-09-21T15:01:11.447 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.1273696940000004, 2024-09-21T15:01:11.447 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0260336299999999, 2024-09-21T15:01:11.447 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:11.447 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:11.447 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:11.447 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:11.447 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:11.447 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140237 2024-09-21T15:01:11.447 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.447 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:11.447 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.447 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:11.447 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237113+0000", 2024-09-21T15:01:11.447 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:11.448 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.448 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.448 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:11.448 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237113+0000", 2024-09-21T15:01:11.448 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:11.448 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.448 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.448 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:11.448 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237112+0000", 2024-09-21T15:01:11.448 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:11.448 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.448 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.448 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:11.448 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237118+0000", 2024-09-21T15:01:11.449 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8930000000000002e-06 2024-09-21T15:01:11.449 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.449 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.449 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:11.449 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237126+0000", 2024-09-21T15:01:11.449 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6729999999999998e-06 2024-09-21T15:01:11.449 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.449 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.449 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:11.449 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237135+0000", 2024-09-21T15:01:11.449 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0929999999999992e-06 2024-09-21T15:01:11.449 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.449 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.449 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:11.450 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237205+0000", 2024-09-21T15:01:11.450 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9697999999999996e-05 2024-09-21T15:01:11.450 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.450 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.450 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:11.450 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237218+0000", 2024-09-21T15:01:11.450 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2524000000000001e-05 2024-09-21T15:01:11.450 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.450 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.450 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:11.450 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.262965+0000", 2024-09-21T15:01:11.450 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0257478630000001 2024-09-21T15:01:11.450 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.450 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.451 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:11.451 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.263146+0000", 2024-09-21T15:01:11.451 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018074599999999999 2024-09-21T15:01:11.451 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.451 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:11.451 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.451 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.451 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.451 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140280 3.1e 3:7b675eeb:::rbd_data.113664780c85.00000000000001b3:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:11.451 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.237477+0000", 2024-09-21T15:01:11.451 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.1270054320000007, 2024-09-21T15:01:11.451 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.026437329, 2024-09-21T15:01:11.451 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:11.451 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:11.451 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:11.452 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:11.452 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:11.452 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140280 2024-09-21T15:01:11.452 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.452 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:11.452 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.452 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:11.452 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237477+0000", 2024-09-21T15:01:11.452 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:11.452 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.452 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.452 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:11.452 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237477+0000", 2024-09-21T15:01:11.452 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:11.452 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.453 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.453 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:11.453 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237476+0000", 2024-09-21T15:01:11.453 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:11.453 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.453 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.453 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:11.453 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237480+0000", 2024-09-21T15:01:11.453 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7570000000000002e-06 2024-09-21T15:01:11.453 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.453 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.453 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:11.453 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237490+0000", 2024-09-21T15:01:11.453 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3170000000000008e-06 2024-09-21T15:01:11.453 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.454 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.454 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:11.454 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237496+0000", 2024-09-21T15:01:11.454 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7150000000000001e-06 2024-09-21T15:01:11.454 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.454 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.454 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:11.454 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237559+0000", 2024-09-21T15:01:11.454 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2844999999999999e-05 2024-09-21T15:01:11.454 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.454 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.454 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:11.454 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237570+0000", 2024-09-21T15:01:11.454 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1253000000000001e-05 2024-09-21T15:01:11.455 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.455 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.455 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:11.455 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.263203+0000", 2024-09-21T15:01:11.455 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0256330339999999 2024-09-21T15:01:11.455 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.455 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.455 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:11.455 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.263914+0000", 2024-09-21T15:01:11.455 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00071080899999999996 2024-09-21T15:01:11.455 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.455 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:11.455 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.456 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.456 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.456 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140348 3.1e 3:7b440264:::rbd_data.113664780c85.0000000000000013:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:11.456 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.241762+0000", 2024-09-21T15:01:11.456 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.1227201610000002, 2024-09-21T15:01:11.456 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0221355759999999, 2024-09-21T15:01:11.456 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:11.456 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:11.456 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:11.456 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:11.456 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:11.456 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140348 2024-09-21T15:01:11.456 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.456 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:11.456 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.457 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:11.457 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.241762+0000", 2024-09-21T15:01:11.457 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:11.457 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.457 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.457 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:11.457 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.241762+0000", 2024-09-21T15:01:11.457 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:11.457 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.457 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.457 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:11.457 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.241761+0000", 2024-09-21T15:01:11.457 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:11.457 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.457 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.458 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:11.458 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.241767+0000", 2024-09-21T15:01:11.458 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4219999999999997e-06 2024-09-21T15:01:11.458 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.458 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.458 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:11.458 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.241783+0000", 2024-09-21T15:01:11.458 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5959000000000001e-05 2024-09-21T15:01:11.458 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.458 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.458 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:11.458 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.241793+0000", 2024-09-21T15:01:11.458 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3999999999999998e-06 2024-09-21T15:01:11.458 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.459 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.459 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:11.459 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.241894+0000", 2024-09-21T15:01:11.459 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000100976 2024-09-21T15:01:11.459 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.459 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.459 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:11.459 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.241907+0000", 2024-09-21T15:01:11.459 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3281e-05 2024-09-21T15:01:11.459 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.459 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.459 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:11.459 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.263430+0000", 2024-09-21T15:01:11.459 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0215229379999999 2024-09-21T15:01:11.459 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.460 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.460 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:11.460 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.263604+0000", 2024-09-21T15:01:11.460 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017368700000000001 2024-09-21T15:01:11.460 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.460 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.460 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:11.460 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.263898+0000", 2024-09-21T15:01:11.460 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00029406899999999999 2024-09-21T15:01:11.460 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.460 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:11.460 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.460 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.460 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:11.461 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:01:11.461 INFO:teuthology.orchestra.run.smithi032.stdout:checking clone #35, image image_client.0-clone35 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent36 2024-09-21T15:01:11.462 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T15:01:11.462 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:01:11.457+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:01:11.474 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-21T15:01:11.474 INFO:teuthology.orchestra.run.smithi032.stdout:{"fsid":"caf51e55-9234-4a26-bcd2-d60811962560","health":{"status":"HEALTH_WARN","checks":{"OSDMAP_FLAGS":{"severity":"HEALTH_WARN","summary":{"message":"nodeep-scrub flag(s) set","count":12},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":201,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":181,"num_osds":8,"num_up_osds":8,"osd_up_since":1726930681,"num_in_osds":8,"osd_in_since":1726930767,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":39},{"state_name":"creating+peering","count":1},{"state_name":"peering","count":1}],"num_pgs":41,"num_pools":3,"num_objects":1338,"data_bytes":1052048706,"bytes_used":475578368,"bytes_avail":772618534912,"bytes_total":773094113280,"inactive_pgs_ratio":0.048780485987663269,"recovering_objects_per_sec":115,"recovering_bytes_per_sec":8926730,"recovering_keys_per_sec":16,"num_objects_recovered":539,"num_bytes_recovered":41668853,"num_keys_recovered":77,"read_bytes_sec":5798436,"write_bytes_sec":4465752,"read_op_per_sec":407,"write_op_per_sec":33},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":4,"modified":"2024-09-21T14:59:03.791262+0000","services":{}},"progress_events":{}} 2024-09-21T15:01:11.490 INFO:tasks.ceph.ceph_manager.ceph:making progress, resetting timeout 2024-09-21T15:01:11.490 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T15:01:11.565 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:01:11.562+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:01:11.585 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:01:11.585 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T15:01:11.585 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.585 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:163288 3.12 3:4feb2de0:::rbd_data.113664780c85.00000000000002bb:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e181)", 2024-09-21T15:01:11.585 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:11.583263+0000", 2024-09-21T15:01:11.585 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.00058667599999999995, 2024-09-21T15:01:11.585 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00061764600000000002, 2024-09-21T15:01:11.585 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:11.585 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:11.585 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:11.586 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:11.586 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:11.586 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 163288 2024-09-21T15:01:11.586 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.586 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:11.586 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.586 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:11.586 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:11.583263+0000", 2024-09-21T15:01:11.586 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:11.586 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.586 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.586 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:11.586 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:11.583263+0000", 2024-09-21T15:01:11.586 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:11.587 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.587 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.587 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:11.587 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:11.583263+0000", 2024-09-21T15:01:11.587 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:11.587 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.587 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.587 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:11.587 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:11.583266+0000", 2024-09-21T15:01:11.587 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8490000000000002e-06 2024-09-21T15:01:11.587 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.587 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.587 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:11.587 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:11.583272+0000", 2024-09-21T15:01:11.587 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6270000000000002e-06 2024-09-21T15:01:11.588 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.588 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.588 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:11.588 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:11.583280+0000", 2024-09-21T15:01:11.588 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9750000000000007e-06 2024-09-21T15:01:11.588 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.588 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.588 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:11.588 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:11.583818+0000", 2024-09-21T15:01:11.588 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00053793799999999996 2024-09-21T15:01:11.588 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.588 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:11.588 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.588 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.588 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-21T15:01:11.589 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 1 2024-09-21T15:01:11.589 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:01:11.597 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T15:01:11.664 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:01:11.658+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:01:11.695 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:01:11.695 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:01:11.695 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T15:01:11.695 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T15:01:11.695 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:01:11.704 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T15:01:11.766 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:01:11.762+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:01:11.809 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:01:11.809 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T15:01:11.809 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T15:01:11.809 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T15:01:11.809 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.809 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:97677 3.15 e122/24 3:a8e72133:::rbd_data.1136bfc79a70.0000000000000005:head v 122'567, mlcod=120'565)", 2024-09-21T15:01:11.809 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.320753+0000", 2024-09-21T15:01:11.809 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 66.485492914999995, 2024-09-21T15:01:11.809 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070639375000000004, 2024-09-21T15:01:11.809 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:11.809 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:11.810 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:11.810 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.810 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:11.810 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.320753+0000", 2024-09-21T15:01:11.810 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:11.810 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.810 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.810 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:11.810 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.320753+0000", 2024-09-21T15:01:11.810 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:11.810 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.810 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.810 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:11.811 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.320753+0000", 2024-09-21T15:01:11.811 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:11.811 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.811 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.811 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:11.811 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.375452+0000", 2024-09-21T15:01:11.811 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054698945999999998 2024-09-21T15:01:11.811 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.811 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.811 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:11.811 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.375476+0000", 2024-09-21T15:01:11.811 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4380999999999999e-05 2024-09-21T15:01:11.811 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.811 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.812 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:11.812 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.375485+0000", 2024-09-21T15:01:11.812 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0049999999999993e-06 2024-09-21T15:01:11.812 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.812 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.812 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:11.812 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.375552+0000", 2024-09-21T15:01:11.812 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6550999999999994e-05 2024-09-21T15:01:11.812 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.812 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.812 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:11.812 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.375578+0000", 2024-09-21T15:01:11.812 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6196000000000001e-05 2024-09-21T15:01:11.812 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.812 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.813 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:01:11.813 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.391319+0000", 2024-09-21T15:01:11.813 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015741271000000001 2024-09-21T15:01:11.813 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.813 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.813 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:11.813 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.391393+0000", 2024-09-21T15:01:11.813 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3287000000000005e-05 2024-09-21T15:01:11.813 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.813 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:11.813 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.813 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.813 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.813 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:97672 3.4 e122/51 3:26046d29:::rbd_data.1136bfc79a70.0000000000000001:head v 122'256, mlcod=120'254)", 2024-09-21T15:01:11.814 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.331696+0000", 2024-09-21T15:01:11.814 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 66.474549752000001, 2024-09-21T15:01:11.814 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041222507999999998, 2024-09-21T15:01:11.814 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:11.814 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:11.814 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:11.814 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.814 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:11.814 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.331696+0000", 2024-09-21T15:01:11.814 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:11.814 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.814 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.814 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:11.815 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.331696+0000", 2024-09-21T15:01:11.815 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:11.815 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.815 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.815 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:11.815 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.331695+0000", 2024-09-21T15:01:11.815 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:11.815 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.815 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.815 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:11.815 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.340175+0000", 2024-09-21T15:01:11.815 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0084799520000000007 2024-09-21T15:01:11.815 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.816 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.816 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:11.816 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.340208+0000", 2024-09-21T15:01:11.816 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2997999999999998e-05 2024-09-21T15:01:11.816 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.816 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.816 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:11.816 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.340223+0000", 2024-09-21T15:01:11.816 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5330999999999999e-05 2024-09-21T15:01:11.816 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.816 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.816 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:11.816 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.340305+0000", 2024-09-21T15:01:11.816 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2095999999999995e-05 2024-09-21T15:01:11.817 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.817 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.817 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:11.817 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.340337+0000", 2024-09-21T15:01:11.817 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2097000000000001e-05 2024-09-21T15:01:11.817 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.817 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.817 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:01:11.817 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.372863+0000", 2024-09-21T15:01:11.817 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.032525923999999998 2024-09-21T15:01:11.817 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.817 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.817 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:11.817 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.372919+0000", 2024-09-21T15:01:11.818 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5729000000000001e-05 2024-09-21T15:01:11.818 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.818 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:11.818 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.818 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.818 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.818 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:116182 3.12 3:4efbaa3f:::rbd_data.1136b1194d20.0000000000000006:58 [read 6242304~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:11.818 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.712538+0000", 2024-09-21T15:01:11.818 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.093707760999997, 2024-09-21T15:01:11.818 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034702127999999999, 2024-09-21T15:01:11.818 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:11.818 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:11.818 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:11.819 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:11.819 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:11.819 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116182 2024-09-21T15:01:11.819 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.819 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:11.819 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.819 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:11.819 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712538+0000", 2024-09-21T15:01:11.819 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:11.819 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.819 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.819 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:11.819 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712538+0000", 2024-09-21T15:01:11.819 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:11.820 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.820 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.820 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:11.820 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712538+0000", 2024-09-21T15:01:11.820 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:11.820 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.820 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.820 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:11.820 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712541+0000", 2024-09-21T15:01:11.820 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5330000000000001e-06 2024-09-21T15:01:11.820 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.820 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.820 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:11.820 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712547+0000", 2024-09-21T15:01:11.820 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1109999999999996e-06 2024-09-21T15:01:11.821 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.821 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.821 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:11.821 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712551+0000", 2024-09-21T15:01:11.821 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.685e-06 2024-09-21T15:01:11.821 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.821 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.821 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:11.821 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.747067+0000", 2024-09-21T15:01:11.821 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034516181 2024-09-21T15:01:11.821 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.821 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.821 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:11.821 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.747152+0000", 2024-09-21T15:01:11.822 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4075999999999999e-05 2024-09-21T15:01:11.822 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.822 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.822 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:11.822 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.747240+0000", 2024-09-21T15:01:11.822 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8924000000000004e-05 2024-09-21T15:01:11.822 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.822 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:11.822 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.822 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.822 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.822 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:118701 3.12 3:4efbaa3f:::rbd_data.1136b1194d20.0000000000000006:58 [read 10412032~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:11.822 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.902645+0000", 2024-09-21T15:01:11.822 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 41.903601387000002, 2024-09-21T15:01:11.823 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034600791999999998, 2024-09-21T15:01:11.823 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:11.823 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:11.823 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:11.823 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:11.823 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:11.823 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 118701 2024-09-21T15:01:11.823 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.823 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:11.823 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.823 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:11.823 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902645+0000", 2024-09-21T15:01:11.823 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:11.823 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.824 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.824 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:11.824 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902645+0000", 2024-09-21T15:01:11.824 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:11.824 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.824 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.824 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:11.824 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902644+0000", 2024-09-21T15:01:11.824 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:11.824 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.824 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.824 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:11.824 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902646+0000", 2024-09-21T15:01:11.824 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6169999999999999e-06 2024-09-21T15:01:11.825 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.825 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.825 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:11.825 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902649+0000", 2024-09-21T15:01:11.825 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9450000000000002e-06 2024-09-21T15:01:11.825 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.825 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.825 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:11.825 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902652+0000", 2024-09-21T15:01:11.825 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7889999999999999e-06 2024-09-21T15:01:11.825 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.825 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.825 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:11.825 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.937086+0000", 2024-09-21T15:01:11.826 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034433832999999997 2024-09-21T15:01:11.826 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.826 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.826 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:11.826 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.937152+0000", 2024-09-21T15:01:11.826 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5934999999999998e-05 2024-09-21T15:01:11.826 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.826 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.826 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:11.826 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.937246+0000", 2024-09-21T15:01:11.826 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3923000000000002e-05 2024-09-21T15:01:11.826 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.826 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:11.826 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.827 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.827 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.827 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139784 3.1 3:862e0e65:::rbd_data.113664780c85.0000000000000063:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:11.827 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.222486+0000", 2024-09-21T15:01:11.827 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.5837598939999999, 2024-09-21T15:01:11.827 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0141793059999999, 2024-09-21T15:01:11.827 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:11.827 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:11.827 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:11.827 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:11.827 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:11.827 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139784 2024-09-21T15:01:11.827 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.827 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:11.828 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.828 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:11.828 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222486+0000", 2024-09-21T15:01:11.828 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:11.828 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.828 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.828 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:11.828 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222486+0000", 2024-09-21T15:01:11.828 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:11.828 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.828 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.828 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:11.828 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222484+0000", 2024-09-21T15:01:11.828 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-21T15:01:11.829 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.829 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.829 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:11.829 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222494+0000", 2024-09-21T15:01:11.829 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0676000000000001e-05 2024-09-21T15:01:11.829 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.829 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.829 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:11.829 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222517+0000", 2024-09-21T15:01:11.829 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2371e-05 2024-09-21T15:01:11.829 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.829 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.829 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:11.829 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222527+0000", 2024-09-21T15:01:11.830 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0461e-05 2024-09-21T15:01:11.830 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.830 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.830 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:11.830 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222625+0000", 2024-09-21T15:01:11.830 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.8160999999999997e-05 2024-09-21T15:01:11.830 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.830 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.830 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:11.830 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222641+0000", 2024-09-21T15:01:11.830 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5899000000000001e-05 2024-09-21T15:01:11.830 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.830 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.831 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:11.831 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.236479+0000", 2024-09-21T15:01:11.831 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0138377709999999 2024-09-21T15:01:11.831 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.831 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.831 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:11.831 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.236666+0000", 2024-09-21T15:01:11.831 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018648199999999999 2024-09-21T15:01:11.831 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.831 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:11.831 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.831 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.831 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.831 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139812 3.1 3:8407fb9b:::rbd_data.113664780c85.0000000000000127:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:11.831 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.222547+0000", 2024-09-21T15:01:11.832 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.5836989700000004, 2024-09-21T15:01:11.832 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.014796059, 2024-09-21T15:01:11.832 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:11.832 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:11.832 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:11.832 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:11.832 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:11.832 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139812 2024-09-21T15:01:11.832 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.832 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:11.832 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.832 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:11.832 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222547+0000", 2024-09-21T15:01:11.832 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:11.833 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.833 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.833 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:11.833 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222547+0000", 2024-09-21T15:01:11.833 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:11.833 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.833 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.833 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:11.833 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222546+0000", 2024-09-21T15:01:11.833 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:11.833 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.833 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.833 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:11.833 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222552+0000", 2024-09-21T15:01:11.834 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2739999999999998e-06 2024-09-21T15:01:11.834 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.834 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.834 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:11.834 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222578+0000", 2024-09-21T15:01:11.834 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6594999999999999e-05 2024-09-21T15:01:11.834 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.834 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.834 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:11.834 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222596+0000", 2024-09-21T15:01:11.834 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7838000000000001e-05 2024-09-21T15:01:11.834 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.834 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.834 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:11.835 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222701+0000", 2024-09-21T15:01:11.835 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000104714 2024-09-21T15:01:11.835 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.835 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.835 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:11.835 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222714+0000", 2024-09-21T15:01:11.835 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2998e-05 2024-09-21T15:01:11.835 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.835 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.835 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:11.835 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.237124+0000", 2024-09-21T15:01:11.835 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.014409806 2024-09-21T15:01:11.835 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.835 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.836 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:11.836 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.237343+0000", 2024-09-21T15:01:11.836 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021950499999999999 2024-09-21T15:01:11.836 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.836 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:11.836 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.836 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.836 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.836 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139826 3.1 3:854f6f9b:::rbd_data.113664780c85.0000000000000189:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:11.836 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.222611+0000", 2024-09-21T15:01:11.836 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.5836348779999998, 2024-09-21T15:01:11.836 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.014916828, 2024-09-21T15:01:11.836 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:11.837 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:11.837 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:11.837 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:11.837 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:11.837 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139826 2024-09-21T15:01:11.837 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.837 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:11.837 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.837 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:11.837 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222611+0000", 2024-09-21T15:01:11.837 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:11.837 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.837 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.837 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:11.838 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222611+0000", 2024-09-21T15:01:11.838 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:11.838 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.838 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.838 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:11.838 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222610+0000", 2024-09-21T15:01:11.838 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:11.838 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.838 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.838 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:11.838 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222618+0000", 2024-09-21T15:01:11.838 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6690000000000006e-06 2024-09-21T15:01:11.838 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.839 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.839 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:11.839 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222632+0000", 2024-09-21T15:01:11.839 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4433999999999999e-05 2024-09-21T15:01:11.839 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.839 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.839 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:11.839 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222640+0000", 2024-09-21T15:01:11.839 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6260000000000003e-06 2024-09-21T15:01:11.839 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.839 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.839 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:11.839 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222758+0000", 2024-09-21T15:01:11.839 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011826499999999999 2024-09-21T15:01:11.840 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.840 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.840 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:11.840 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222776+0000", 2024-09-21T15:01:11.840 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7564999999999999e-05 2024-09-21T15:01:11.840 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.840 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.840 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:11.840 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.237402+0000", 2024-09-21T15:01:11.840 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0146261889999999 2024-09-21T15:01:11.840 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.840 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.840 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:11.840 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.237528+0000", 2024-09-21T15:01:11.840 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000126212 2024-09-21T15:01:11.841 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.841 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:11.841 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.841 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.841 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.841 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139864 3.1 3:86b14cb7:::rbd_data.113664780c85.00000000000000d5:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:11.841 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.223966+0000", 2024-09-21T15:01:11.841 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.5822801420000001, 2024-09-21T15:01:11.841 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.013859007, 2024-09-21T15:01:11.841 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:11.841 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:11.841 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:11.841 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:11.841 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:11.841 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139864 2024-09-21T15:01:11.841 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.842 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:11.842 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.842 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:11.842 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223966+0000", 2024-09-21T15:01:11.842 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:11.842 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.842 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.842 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:11.842 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223966+0000", 2024-09-21T15:01:11.842 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:11.842 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.842 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.842 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:11.842 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223965+0000", 2024-09-21T15:01:11.843 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:11.843 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.843 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.843 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:11.843 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223970+0000", 2024-09-21T15:01:11.843 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3769999999999998e-06 2024-09-21T15:01:11.843 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.843 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.843 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:11.843 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223979+0000", 2024-09-21T15:01:11.843 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5899999999999997e-06 2024-09-21T15:01:11.843 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.843 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.844 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:11.844 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223985+0000", 2024-09-21T15:01:11.844 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5790000000000003e-06 2024-09-21T15:01:11.844 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.844 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.844 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:11.844 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224073+0000", 2024-09-21T15:01:11.844 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7781000000000005e-05 2024-09-21T15:01:11.844 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.844 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.844 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:11.844 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224110+0000", 2024-09-21T15:01:11.844 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7060000000000001e-05 2024-09-21T15:01:11.844 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.845 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.845 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:11.845 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.237599+0000", 2024-09-21T15:01:11.845 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.01348953 2024-09-21T15:01:11.845 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.845 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.845 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:11.845 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.237825+0000", 2024-09-21T15:01:11.845 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000225758 2024-09-21T15:01:11.845 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.845 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:11.845 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.845 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.845 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.845 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139877 3.1 3:87277f92:::rbd_data.113664780c85.0000000000000136:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:11.845 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.224489+0000", 2024-09-21T15:01:11.846 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.581757284, 2024-09-21T15:01:11.846 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0137916330000001, 2024-09-21T15:01:11.846 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:11.846 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:11.846 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:11.846 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:11.846 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:11.846 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139877 2024-09-21T15:01:11.846 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.846 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:11.846 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.846 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:11.846 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224489+0000", 2024-09-21T15:01:11.846 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:11.846 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.847 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.847 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:11.847 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224489+0000", 2024-09-21T15:01:11.847 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:11.847 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.847 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.847 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:11.847 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224488+0000", 2024-09-21T15:01:11.847 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:11.847 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.847 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.847 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:11.847 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224502+0000", 2024-09-21T15:01:11.847 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4194e-05 2024-09-21T15:01:11.847 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.848 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.848 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:11.848 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224522+0000", 2024-09-21T15:01:11.848 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0038e-05 2024-09-21T15:01:11.848 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.848 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.848 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:11.848 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224531+0000", 2024-09-21T15:01:11.848 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6900000000000004e-06 2024-09-21T15:01:11.848 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.848 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.848 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:11.848 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224621+0000", 2024-09-21T15:01:11.848 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9654000000000006e-05 2024-09-21T15:01:11.848 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.849 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.849 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:11.849 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224641+0000", 2024-09-21T15:01:11.849 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9375e-05 2024-09-21T15:01:11.849 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.849 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.849 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:11.849 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.238027+0000", 2024-09-21T15:01:11.849 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0133866840000001 2024-09-21T15:01:11.849 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.849 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.849 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:11.849 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.238280+0000", 2024-09-21T15:01:11.850 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000253281 2024-09-21T15:01:11.850 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.850 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:11.850 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.850 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.850 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.850 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139887 3.1 3:80f1e8f1:::rbd_data.113664780c85.000000000000017c:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:11.850 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.224549+0000", 2024-09-21T15:01:11.850 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.5816969939999996, 2024-09-21T15:01:11.850 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0139776060000001, 2024-09-21T15:01:11.850 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:11.850 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:11.850 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:11.850 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:11.850 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:11.851 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139887 2024-09-21T15:01:11.851 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.851 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:11.851 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.851 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:11.851 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224549+0000", 2024-09-21T15:01:11.851 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:11.851 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.851 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.851 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:11.851 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224549+0000", 2024-09-21T15:01:11.851 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:11.851 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.851 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.852 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:11.852 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224548+0000", 2024-09-21T15:01:11.852 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:11.852 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.852 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.852 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:11.852 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224554+0000", 2024-09-21T15:01:11.852 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4049999999999999e-06 2024-09-21T15:01:11.852 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.852 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.852 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:11.852 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224565+0000", 2024-09-21T15:01:11.852 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0879e-05 2024-09-21T15:01:11.852 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.852 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.853 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:11.853 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224572+0000", 2024-09-21T15:01:11.853 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0380000000000002e-06 2024-09-21T15:01:11.853 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.853 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.853 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:11.853 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224684+0000", 2024-09-21T15:01:11.853 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011229 2024-09-21T15:01:11.853 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.853 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.853 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:11.853 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224698+0000", 2024-09-21T15:01:11.853 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4448e-05 2024-09-21T15:01:11.853 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.853 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.853 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:11.854 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.238359+0000", 2024-09-21T15:01:11.854 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0136610660000001 2024-09-21T15:01:11.854 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.854 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.854 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:11.854 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.238527+0000", 2024-09-21T15:01:11.854 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016731100000000001 2024-09-21T15:01:11.854 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.854 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:11.854 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.854 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.854 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.854 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140077 3.1 3:85642daf:::rbd_data.113664780c85.0000000000000174:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:11.854 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.232797+0000", 2024-09-21T15:01:11.855 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.5734492790000001, 2024-09-21T15:01:11.855 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.006029955, 2024-09-21T15:01:11.855 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:11.855 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:11.855 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:11.855 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:11.855 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:11.855 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140077 2024-09-21T15:01:11.855 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.855 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:11.855 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.855 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:11.855 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232797+0000", 2024-09-21T15:01:11.855 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:11.856 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.856 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.856 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:11.856 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232797+0000", 2024-09-21T15:01:11.856 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:11.856 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.856 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.856 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:11.856 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232795+0000", 2024-09-21T15:01:11.856 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:11.856 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.856 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.856 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:11.856 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232804+0000", 2024-09-21T15:01:11.857 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9670000000000007e-06 2024-09-21T15:01:11.857 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.857 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.857 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:11.857 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232824+0000", 2024-09-21T15:01:11.857 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9238000000000001e-05 2024-09-21T15:01:11.857 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.857 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.857 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:11.857 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232843+0000", 2024-09-21T15:01:11.857 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9256e-05 2024-09-21T15:01:11.857 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.857 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.857 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:11.858 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232912+0000", 2024-09-21T15:01:11.858 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9589000000000001e-05 2024-09-21T15:01:11.858 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.858 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.858 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:11.858 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232925+0000", 2024-09-21T15:01:11.858 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2585999999999999e-05 2024-09-21T15:01:11.858 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.858 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.858 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:11.858 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.238591+0000", 2024-09-21T15:01:11.858 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0056659939999999 2024-09-21T15:01:11.858 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.858 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.859 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:11.859 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.238827+0000", 2024-09-21T15:01:11.859 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00023583999999999999 2024-09-21T15:01:11.859 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.859 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:11.859 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.859 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.859 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.859 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140091 3.1 3:8358f0db:::rbd_data.113664780c85.0000000000000018:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:11.859 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.233637+0000", 2024-09-21T15:01:11.859 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.5726089830000003, 2024-09-21T15:01:11.859 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.006272847, 2024-09-21T15:01:11.859 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:11.859 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:11.860 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:11.860 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:11.860 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:11.860 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140091 2024-09-21T15:01:11.860 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.860 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:11.860 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.860 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:11.860 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233637+0000", 2024-09-21T15:01:11.860 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:11.860 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.860 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.860 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:11.860 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233637+0000", 2024-09-21T15:01:11.861 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:11.861 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.861 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.861 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:11.861 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233636+0000", 2024-09-21T15:01:11.861 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:11.861 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.861 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.861 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:11.861 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233644+0000", 2024-09-21T15:01:11.861 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1380000000000007e-06 2024-09-21T15:01:11.861 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.861 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.861 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:11.862 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233662+0000", 2024-09-21T15:01:11.862 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8224999999999999e-05 2024-09-21T15:01:11.862 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.862 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.862 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:11.862 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233674+0000", 2024-09-21T15:01:11.862 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2081e-05 2024-09-21T15:01:11.862 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.862 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.862 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:11.862 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233750+0000", 2024-09-21T15:01:11.862 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6106000000000006e-05 2024-09-21T15:01:11.862 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.862 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.863 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:11.863 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233768+0000", 2024-09-21T15:01:11.863 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7285000000000001e-05 2024-09-21T15:01:11.863 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.863 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.863 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:11.863 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.239196+0000", 2024-09-21T15:01:11.863 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.005428223 2024-09-21T15:01:11.863 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.863 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.863 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:11.863 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.239528+0000", 2024-09-21T15:01:11.863 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00033262199999999997 2024-09-21T15:01:11.863 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.864 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.864 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:11.864 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.239910+0000", 2024-09-21T15:01:11.864 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00038151100000000001 2024-09-21T15:01:11.864 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.864 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:11.864 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.864 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.864 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.864 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140109 3.1 3:8586e9d9:::rbd_data.113664780c85.0000000000000096:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:11.864 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.234171+0000", 2024-09-21T15:01:11.864 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.5720751489999998, 2024-09-21T15:01:11.864 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.006019896, 2024-09-21T15:01:11.865 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:11.865 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:11.865 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:11.865 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:11.865 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:11.865 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140109 2024-09-21T15:01:11.865 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.865 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:11.865 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.865 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:11.865 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.234171+0000", 2024-09-21T15:01:11.865 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:11.865 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.865 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.866 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:11.866 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.234171+0000", 2024-09-21T15:01:11.866 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:11.866 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.866 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.866 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:11.866 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.234170+0000", 2024-09-21T15:01:11.866 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:11.866 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.866 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.866 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:11.866 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.234178+0000", 2024-09-21T15:01:11.866 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.464e-06 2024-09-21T15:01:11.866 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.867 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.867 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:11.867 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.234191+0000", 2024-09-21T15:01:11.867 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3489000000000001e-05 2024-09-21T15:01:11.867 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.867 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.867 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:11.867 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.234200+0000", 2024-09-21T15:01:11.867 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2779999999999999e-06 2024-09-21T15:01:11.867 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.867 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.867 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:11.867 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.234279+0000", 2024-09-21T15:01:11.867 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8813000000000006e-05 2024-09-21T15:01:11.868 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.868 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.868 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:11.868 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.234298+0000", 2024-09-21T15:01:11.868 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8947999999999999e-05 2024-09-21T15:01:11.868 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.868 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.868 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:11.868 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.239974+0000", 2024-09-21T15:01:11.868 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0056760259999999 2024-09-21T15:01:11.868 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.868 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.868 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:11.868 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.240191+0000", 2024-09-21T15:01:11.869 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000216708 2024-09-21T15:01:11.869 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.869 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:11.869 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.869 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.869 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.869 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140149 3.1 3:86b1e433:::rbd_data.113664780c85.00000000000001ae:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:11.869 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.235545+0000", 2024-09-21T15:01:11.869 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.5707008639999991, 2024-09-21T15:01:11.869 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0049193430000001, 2024-09-21T15:01:11.869 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:11.869 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:11.869 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:11.869 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:11.870 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:11.870 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140149 2024-09-21T15:01:11.870 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.870 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:11.870 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.870 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:11.870 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235545+0000", 2024-09-21T15:01:11.870 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:11.870 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.870 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.870 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:11.870 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235545+0000", 2024-09-21T15:01:11.870 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:11.870 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.871 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.871 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:11.871 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235544+0000", 2024-09-21T15:01:11.871 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:11.871 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.871 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.871 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:11.871 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235554+0000", 2024-09-21T15:01:11.871 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.8870000000000004e-06 2024-09-21T15:01:11.871 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.871 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.871 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:11.871 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235577+0000", 2024-09-21T15:01:11.872 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2946999999999999e-05 2024-09-21T15:01:11.872 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.872 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.872 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:11.872 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235587+0000", 2024-09-21T15:01:11.872 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0326000000000001e-05 2024-09-21T15:01:11.872 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.872 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.872 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:11.872 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235658+0000", 2024-09-21T15:01:11.872 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1360999999999997e-05 2024-09-21T15:01:11.872 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.872 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.872 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:11.873 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235670+0000", 2024-09-21T15:01:11.873 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1735e-05 2024-09-21T15:01:11.873 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.873 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.873 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:11.873 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.240246+0000", 2024-09-21T15:01:11.873 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0045757790000001 2024-09-21T15:01:11.873 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.873 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.873 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:11.873 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.240465+0000", 2024-09-21T15:01:11.873 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021870799999999999 2024-09-21T15:01:11.873 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.873 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:11.873 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.874 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.874 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.874 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140232 3.1 3:862e0e65:::rbd_data.113664780c85.0000000000000063:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:11.874 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.235760+0000", 2024-09-21T15:01:11.874 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.5704861460000004, 2024-09-21T15:01:11.874 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.004979812, 2024-09-21T15:01:11.874 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:11.874 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:11.874 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:11.874 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:11.874 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:11.874 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140232 2024-09-21T15:01:11.874 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.874 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:11.875 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.875 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:11.875 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235760+0000", 2024-09-21T15:01:11.875 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:11.875 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.875 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.875 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:11.875 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235760+0000", 2024-09-21T15:01:11.875 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:11.875 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.875 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.875 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:11.875 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235759+0000", 2024-09-21T15:01:11.875 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:11.876 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.876 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.876 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:11.876 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235777+0000", 2024-09-21T15:01:11.876 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8215000000000001e-05 2024-09-21T15:01:11.876 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.876 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.876 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:11.876 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235793+0000", 2024-09-21T15:01:11.876 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6461e-05 2024-09-21T15:01:11.876 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.876 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.876 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:11.876 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235803+0000", 2024-09-21T15:01:11.877 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5300000000000002e-06 2024-09-21T15:01:11.877 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.877 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.877 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:11.877 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235875+0000", 2024-09-21T15:01:11.877 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2113000000000006e-05 2024-09-21T15:01:11.877 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.877 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.877 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:11.877 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235887+0000", 2024-09-21T15:01:11.877 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2160999999999999e-05 2024-09-21T15:01:11.877 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.877 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.877 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:11.878 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.240521+0000", 2024-09-21T15:01:11.878 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.004633428 2024-09-21T15:01:11.878 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.878 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.878 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:11.878 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.240740+0000", 2024-09-21T15:01:11.878 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000219228 2024-09-21T15:01:11.878 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.878 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:11.878 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.878 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.878 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.878 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140260 3.1 3:8407fb9b:::rbd_data.113664780c85.0000000000000127:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:11.878 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.235839+0000", 2024-09-21T15:01:11.879 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.5704069769999993, 2024-09-21T15:01:11.879 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.005157981, 2024-09-21T15:01:11.879 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:11.879 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:11.879 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:11.879 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:11.879 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:11.879 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140260 2024-09-21T15:01:11.879 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.879 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:11.879 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.879 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:11.879 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235839+0000", 2024-09-21T15:01:11.880 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:11.880 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.880 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.880 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:11.880 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235839+0000", 2024-09-21T15:01:11.880 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:11.880 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.880 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.880 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:11.880 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235838+0000", 2024-09-21T15:01:11.880 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:11.880 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.880 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.880 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:11.881 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235843+0000", 2024-09-21T15:01:11.881 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.481e-06 2024-09-21T15:01:11.881 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.881 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.881 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:11.881 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235849+0000", 2024-09-21T15:01:11.881 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4880000000000004e-06 2024-09-21T15:01:11.881 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.881 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.881 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:11.881 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235856+0000", 2024-09-21T15:01:11.881 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7649999999999996e-06 2024-09-21T15:01:11.881 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.881 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.882 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:11.882 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235933+0000", 2024-09-21T15:01:11.882 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6987e-05 2024-09-21T15:01:11.882 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.882 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.882 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:11.882 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235949+0000", 2024-09-21T15:01:11.882 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6696000000000001e-05 2024-09-21T15:01:11.882 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.882 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.882 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:11.882 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.240823+0000", 2024-09-21T15:01:11.882 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.004873259 2024-09-21T15:01:11.883 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.883 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.883 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:11.883 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.240997+0000", 2024-09-21T15:01:11.883 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000174365 2024-09-21T15:01:11.883 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.883 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:11.883 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.883 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.883 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.883 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140274 3.1 3:854f6f9b:::rbd_data.113664780c85.0000000000000189:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:11.883 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.235884+0000", 2024-09-21T15:01:11.883 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.570362029, 2024-09-21T15:01:11.884 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.005397366, 2024-09-21T15:01:11.884 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:11.884 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:11.884 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:11.884 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:11.884 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:11.884 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140274 2024-09-21T15:01:11.884 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.884 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:11.884 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.884 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:11.884 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235884+0000", 2024-09-21T15:01:11.884 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:11.885 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.885 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.885 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:11.885 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235884+0000", 2024-09-21T15:01:11.885 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:11.885 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.885 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.885 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:11.885 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235883+0000", 2024-09-21T15:01:11.885 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:11.885 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.885 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.885 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:11.885 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235891+0000", 2024-09-21T15:01:11.886 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9740000000000002e-06 2024-09-21T15:01:11.886 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.886 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.886 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:11.886 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235900+0000", 2024-09-21T15:01:11.886 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6659999999999997e-06 2024-09-21T15:01:11.886 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.886 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.886 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:11.886 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235909+0000", 2024-09-21T15:01:11.886 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3400000000000004e-06 2024-09-21T15:01:11.886 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.886 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.887 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:11.887 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236012+0000", 2024-09-21T15:01:11.887 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010240500000000001 2024-09-21T15:01:11.887 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.887 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.887 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:11.887 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236027+0000", 2024-09-21T15:01:11.887 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5418e-05 2024-09-21T15:01:11.887 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.887 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.887 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:11.887 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.241068+0000", 2024-09-21T15:01:11.887 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0050410729999999 2024-09-21T15:01:11.887 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.888 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.888 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:11.888 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.241281+0000", 2024-09-21T15:01:11.888 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021321900000000001 2024-09-21T15:01:11.888 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.888 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:11.888 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.888 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.888 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.888 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140312 3.1 3:86b14cb7:::rbd_data.113664780c85.00000000000000d5:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:11.888 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.236769+0000", 2024-09-21T15:01:11.888 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.5694775649999997, 2024-09-21T15:01:11.888 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.004717198, 2024-09-21T15:01:11.888 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:11.889 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:11.889 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:11.889 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:11.889 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:11.889 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140312 2024-09-21T15:01:11.889 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.889 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:11.889 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.889 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:11.889 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236769+0000", 2024-09-21T15:01:11.889 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:11.889 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.889 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.889 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:11.890 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236769+0000", 2024-09-21T15:01:11.890 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:11.890 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.890 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.890 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:11.890 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236768+0000", 2024-09-21T15:01:11.890 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:11.890 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.890 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.890 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:11.890 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236771+0000", 2024-09-21T15:01:11.890 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.331e-06 2024-09-21T15:01:11.890 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.891 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.891 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:11.891 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236777+0000", 2024-09-21T15:01:11.891 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6949999999999997e-06 2024-09-21T15:01:11.891 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.891 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.891 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:11.891 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236787+0000", 2024-09-21T15:01:11.891 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6700000000000006e-06 2024-09-21T15:01:11.891 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.891 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.891 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:11.891 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236846+0000", 2024-09-21T15:01:11.891 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.944e-05 2024-09-21T15:01:11.892 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.892 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.892 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:11.892 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236861+0000", 2024-09-21T15:01:11.892 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4611e-05 2024-09-21T15:01:11.892 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.892 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.892 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:11.892 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.241349+0000", 2024-09-21T15:01:11.892 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0044879250000001 2024-09-21T15:01:11.892 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.892 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.892 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:11.892 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.241486+0000", 2024-09-21T15:01:11.892 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013713000000000001 2024-09-21T15:01:11.893 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.893 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:11.893 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.893 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.893 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.893 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140325 3.1 3:87277f92:::rbd_data.113664780c85.0000000000000136:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:11.893 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.239351+0000", 2024-09-21T15:01:11.893 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.5668954260000003, 2024-09-21T15:01:11.893 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0024334690000001, 2024-09-21T15:01:11.893 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:11.893 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:11.893 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:11.893 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:11.893 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:11.894 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140325 2024-09-21T15:01:11.894 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.894 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:11.894 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.894 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:11.894 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239351+0000", 2024-09-21T15:01:11.894 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:11.894 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.894 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.894 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:11.894 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239351+0000", 2024-09-21T15:01:11.894 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:11.894 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.895 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.895 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:11.895 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239349+0000", 2024-09-21T15:01:11.895 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:11.895 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.895 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.895 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:11.895 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239358+0000", 2024-09-21T15:01:11.895 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2830000000000001e-06 2024-09-21T15:01:11.895 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.895 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.895 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:11.895 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239379+0000", 2024-09-21T15:01:11.895 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1780000000000002e-05 2024-09-21T15:01:11.896 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.896 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.896 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:11.896 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239389+0000", 2024-09-21T15:01:11.896 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0056e-05 2024-09-21T15:01:11.896 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.896 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.896 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:11.896 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239484+0000", 2024-09-21T15:01:11.896 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4611000000000006e-05 2024-09-21T15:01:11.896 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.896 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.896 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:11.896 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239497+0000", 2024-09-21T15:01:11.897 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2909e-05 2024-09-21T15:01:11.897 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.897 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.897 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:11.897 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.241559+0000", 2024-09-21T15:01:11.897 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0020622459999999 2024-09-21T15:01:11.897 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.897 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.897 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:11.897 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.241784+0000", 2024-09-21T15:01:11.897 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000225009 2024-09-21T15:01:11.897 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.897 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:11.897 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.898 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.898 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.898 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140335 3.1 3:80f1e8f1:::rbd_data.113664780c85.000000000000017c:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:11.898 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.239409+0000", 2024-09-21T15:01:11.898 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 9.5668367070000002, 2024-09-21T15:01:11.898 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0025803150000001, 2024-09-21T15:01:11.898 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:11.898 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:11.898 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:11.898 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:11.898 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:11.898 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140335 2024-09-21T15:01:11.898 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.899 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:11.899 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.899 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:11.899 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239409+0000", 2024-09-21T15:01:11.899 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:11.899 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.899 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.899 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:11.899 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239409+0000", 2024-09-21T15:01:11.899 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:11.899 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.899 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.899 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:11.899 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239409+0000", 2024-09-21T15:01:11.900 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:11.900 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.900 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.900 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:11.900 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239413+0000", 2024-09-21T15:01:11.900 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7620000000000003e-06 2024-09-21T15:01:11.900 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.900 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.900 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:11.900 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239436+0000", 2024-09-21T15:01:11.900 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2889000000000001e-05 2024-09-21T15:01:11.900 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.900 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.901 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:11.901 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239444+0000", 2024-09-21T15:01:11.901 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6809999999999998e-06 2024-09-21T15:01:11.901 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.901 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.901 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:11.901 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239542+0000", 2024-09-21T15:01:11.901 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7868999999999994e-05 2024-09-21T15:01:11.901 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.901 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.901 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:11.901 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239558+0000", 2024-09-21T15:01:11.901 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6257e-05 2024-09-21T15:01:11.901 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.901 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.902 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:11.902 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.241847+0000", 2024-09-21T15:01:11.902 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.002288533 2024-09-21T15:01:11.902 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:11.902 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:11.902 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:11.902 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.241990+0000", 2024-09-21T15:01:11.902 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014308300000000001 2024-09-21T15:01:11.902 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.902 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:11.902 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.902 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:11.902 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:11.903 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:01:11.905 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T15:01:11.906 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-21T15:01:11.907 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-09-21T15:01:11.907 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":269,"stamp":"2024-09-21T15:01:11.262086+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1163705511,"num_objects":1604,"num_object_clones":0,"num_object_copies":3208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1604,"num_whiteouts":0,"num_read":162590,"num_read_kb":1207640,"num_write":30053,"num_write_kb":937500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1920,"num_bytes_recovered":145354624,"num_keys_recovered":229,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18464,"ondisk_log_size":18464,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":507808,"kb_used_data":99776,"kb_used_omap":12,"kb_used_meta":407923,"kb_avail":754466912,"statfs":{"total":773094113280,"available":772574117888,"internally_reserved":0,"allocated":102170624,"data_stored":679620724,"data_compressed":13538801,"data_compressed_allocated":60862464,"data_compressed_original":652893313,"omap_allocated":12744,"internal_metadata":417713720},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":182071378,"num_objects":289,"num_object_clones":0,"num_object_copies":578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":289,"num_whiteouts":0,"num_read":19930,"num_read_kb":80690,"num_write":4007,"num_write_kb":71549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":539,"num_bytes_recovered":41668853,"num_keys_recovered":77,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.439224"},"pg_stats":[{"pgid":"3.1f","version":"180'467","reported_seq":3468,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:01:10.439967+0000","last_change":"2024-09-21T15:01:04.223131+0000","last_active":"2024-09-21T15:01:10.439967+0000","last_peered":"2024-09-21T15:01:10.439967+0000","last_clean":"2024-09-21T15:01:10.439967+0000","last_became_active":"2024-09-21T14:58:19.410055+0000","last_became_peered":"2024-09-21T14:58:19.410055+0000","last_unstale":"2024-09-21T15:01:10.439967+0000","last_undegraded":"2024-09-21T15:01:10.439967+0000","last_fullsized":"2024-09-21T15:01:10.439967+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":467,"log_dups_size":0,"ondisk_log_size":467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:49:13.944219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063336100000000004,"stat_sum":{"num_bytes":39568717,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":3403,"num_read_kb":42213,"num_write":855,"num_write_kb":36337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.1e","version":"180'378","reported_seq":2988,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:01:10.438472+0000","last_change":"2024-09-21T15:01:06.173953+0000","last_active":"2024-09-21T15:01:10.438472+0000","last_peered":"2024-09-21T15:01:10.438472+0000","last_clean":"2024-09-21T15:01:10.438472+0000","last_became_active":"2024-09-21T15:01:06.168726+0000","last_became_peered":"2024-09-21T15:01:06.168726+0000","last_unstale":"2024-09-21T15:01:10.438472+0000","last_undegraded":"2024-09-21T15:01:10.438472+0000","last_fullsized":"2024-09-21T15:01:10.438472+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":378,"log_dups_size":0,"ondisk_log_size":378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:29:01.079772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021250790000000002,"stat_sum":{"num_bytes":23357729,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":2999,"num_read_kb":22037,"num_write":522,"num_write_kb":18370,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":148,"num_bytes_recovered":11635695,"num_keys_recovered":22,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.1d","version":"180'288","reported_seq":4485,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:01:10.388302+0000","last_change":"2024-09-21T15:01:04.229475+0000","last_active":"2024-09-21T15:01:10.388302+0000","last_peered":"2024-09-21T15:01:10.388302+0000","last_clean":"2024-09-21T15:01:10.388302+0000","last_became_active":"2024-09-21T14:58:19.411387+0000","last_became_peered":"2024-09-21T14:58:19.411387+0000","last_unstale":"2024-09-21T15:01:10.388302+0000","last_undegraded":"2024-09-21T15:01:10.388302+0000","last_fullsized":"2024-09-21T15:01:10.388302+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":288,"log_dups_size":0,"ondisk_log_size":288,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:37:57.597069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054775500000000003,"stat_sum":{"num_bytes":21313673,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":3859,"num_read_kb":26411,"num_write":483,"num_write_kb":26568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.1c","version":"180'455","reported_seq":3762,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:01:10.434175+0000","last_change":"2024-09-21T15:01:04.234927+0000","last_active":"2024-09-21T15:01:10.434175+0000","last_peered":"2024-09-21T15:01:10.434175+0000","last_clean":"2024-09-21T15:01:10.434175+0000","last_became_active":"2024-09-21T14:58:19.411306+0000","last_became_peered":"2024-09-21T14:58:19.411306+0000","last_unstale":"2024-09-21T15:01:10.434175+0000","last_undegraded":"2024-09-21T15:01:10.434175+0000","last_fullsized":"2024-09-21T15:01:10.434175+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:20:27.079795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010334350000000001,"stat_sum":{"num_bytes":23930794,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":3676,"num_read_kb":23881,"num_write":713,"num_write_kb":25173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.1b","version":"180'445","reported_seq":3596,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:01:10.374329+0000","last_change":"2024-09-21T15:01:04.227802+0000","last_active":"2024-09-21T15:01:10.374329+0000","last_peered":"2024-09-21T15:01:10.374329+0000","last_clean":"2024-09-21T15:01:10.374329+0000","last_became_active":"2024-09-21T14:59:31.131051+0000","last_became_peered":"2024-09-21T14:59:31.131051+0000","last_unstale":"2024-09-21T15:01:10.374329+0000","last_undegraded":"2024-09-21T15:01:10.374329+0000","last_fullsized":"2024-09-21T15:01:10.374329+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":445,"log_dups_size":0,"ondisk_log_size":445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:21:37.320774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001254673,"stat_sum":{"num_bytes":28181374,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":3454,"num_read_kb":31760,"num_write":619,"num_write_kb":22499,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":117,"num_bytes_recovered":7757116,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.1a","version":"180'392","reported_seq":4180,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:01:10.539507+0000","last_change":"2024-09-21T15:01:04.234672+0000","last_active":"2024-09-21T15:01:10.539507+0000","last_peered":"2024-09-21T15:01:10.539507+0000","last_clean":"2024-09-21T15:01:10.539507+0000","last_became_active":"2024-09-21T14:58:19.409415+0000","last_became_peered":"2024-09-21T14:58:19.409415+0000","last_unstale":"2024-09-21T15:01:10.539507+0000","last_undegraded":"2024-09-21T15:01:10.539507+0000","last_fullsized":"2024-09-21T15:01:10.539507+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":392,"log_dups_size":0,"ondisk_log_size":392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:28:46.336301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060307399999999999,"stat_sum":{"num_bytes":27365738,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":3912,"num_read_kb":27610,"num_write":612,"num_write_kb":23813,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.19","version":"180'695","reported_seq":7919,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:01:10.517130+0000","last_change":"2024-09-21T15:01:04.228157+0000","last_active":"2024-09-21T15:01:10.517130+0000","last_peered":"2024-09-21T15:01:10.517130+0000","last_clean":"2024-09-21T15:01:10.517130+0000","last_became_active":"2024-09-21T14:59:32.130624+0000","last_became_peered":"2024-09-21T14:59:32.130624+0000","last_unstale":"2024-09-21T15:01:10.517130+0000","last_undegraded":"2024-09-21T15:01:10.517130+0000","last_fullsized":"2024-09-21T15:01:10.517130+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":695,"log_dups_size":0,"ondisk_log_size":695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:37:53.312733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001540409,"stat_sum":{"num_bytes":57317066,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":8982,"num_read_kb":66656,"num_write":1130,"num_write_kb":29843,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":128,"num_bytes_recovered":11896992,"num_keys_recovered":44,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.18","version":"180'549","reported_seq":8261,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:01:10.485156+0000","last_change":"2024-09-21T15:01:04.223390+0000","last_active":"2024-09-21T15:01:10.485156+0000","last_peered":"2024-09-21T15:01:10.485156+0000","last_clean":"2024-09-21T15:01:10.485156+0000","last_became_active":"2024-09-21T14:58:19.409371+0000","last_became_peered":"2024-09-21T14:58:19.409371+0000","last_unstale":"2024-09-21T15:01:10.485156+0000","last_undegraded":"2024-09-21T15:01:10.485156+0000","last_fullsized":"2024-09-21T15:01:10.485156+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":549,"log_dups_size":0,"ondisk_log_size":549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:35:27.143445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076656300000000005,"stat_sum":{"num_bytes":39453048,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":8844,"num_read_kb":43937,"num_write":867,"num_write_kb":30512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.17","version":"180'348","reported_seq":3695,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:01:10.399384+0000","last_change":"2024-09-21T15:01:04.225804+0000","last_active":"2024-09-21T15:01:10.399384+0000","last_peered":"2024-09-21T15:01:10.399384+0000","last_clean":"2024-09-21T15:01:10.399384+0000","last_became_active":"2024-09-21T14:58:19.410306+0000","last_became_peered":"2024-09-21T14:58:19.410306+0000","last_unstale":"2024-09-21T15:01:10.399384+0000","last_undegraded":"2024-09-21T15:01:10.399384+0000","last_fullsized":"2024-09-21T15:01:10.399384+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:33:50.089165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030273969999999998,"stat_sum":{"num_bytes":32084432,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":3370,"num_read_kb":27721,"num_write":705,"num_write_kb":34569,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.16","version":"180'911","reported_seq":5643,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:01:10.462549+0000","last_change":"2024-09-21T15:01:06.169652+0000","last_active":"2024-09-21T15:01:10.462549+0000","last_peered":"2024-09-21T15:01:10.462549+0000","last_clean":"2024-09-21T15:01:10.462549+0000","last_became_active":"2024-09-21T15:01:06.166756+0000","last_became_peered":"2024-09-21T15:01:06.166756+0000","last_unstale":"2024-09-21T15:01:10.462549+0000","last_undegraded":"2024-09-21T15:01:10.462549+0000","last_fullsized":"2024-09-21T15:01:10.462549+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":911,"log_dups_size":0,"ondisk_log_size":911,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:19:04.988784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000477617,"stat_sum":{"num_bytes":39025443,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":7660,"num_read_kb":43453,"num_write":1485,"num_write_kb":28270,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":271,"num_bytes_recovered":16821266,"num_keys_recovered":55,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.15","version":"180'730","reported_seq":3633,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:01:10.489670+0000","last_change":"2024-09-21T15:01:04.231959+0000","last_active":"2024-09-21T15:01:10.489670+0000","last_peered":"2024-09-21T15:01:10.489670+0000","last_clean":"2024-09-21T15:01:10.489670+0000","last_became_active":"2024-09-21T14:58:19.410687+0000","last_became_peered":"2024-09-21T14:58:19.410687+0000","last_unstale":"2024-09-21T15:01:10.489670+0000","last_undegraded":"2024-09-21T15:01:10.489670+0000","last_fullsized":"2024-09-21T15:01:10.489670+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":730,"log_dups_size":0,"ondisk_log_size":730,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:21:42.226982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033742220000000001,"stat_sum":{"num_bytes":39686352,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":4682,"num_read_kb":48927,"num_write":1251,"num_write_kb":33262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.14","version":"180'460","reported_seq":3873,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:01:10.434207+0000","last_change":"2024-09-21T15:01:04.229559+0000","last_active":"2024-09-21T15:01:10.434207+0000","last_peered":"2024-09-21T15:01:10.434207+0000","last_clean":"2024-09-21T15:01:10.434207+0000","last_became_active":"2024-09-21T14:58:19.407310+0000","last_became_peered":"2024-09-21T14:58:19.407310+0000","last_unstale":"2024-09-21T15:01:10.434207+0000","last_undegraded":"2024-09-21T15:01:10.434207+0000","last_fullsized":"2024-09-21T15:01:10.434207+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":460,"log_dups_size":0,"ondisk_log_size":460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:47:46.079115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052455500000000001,"stat_sum":{"num_bytes":38779984,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":3708,"num_read_kb":41600,"num_write":644,"num_write_kb":33420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":174,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-21T15:01:08.256323+0000","last_change":"2024-09-21T14:58:11.348177+0000","last_active":"2024-09-21T15:01:08.256323+0000","last_peered":"2024-09-21T15:01:08.256323+0000","last_clean":"2024-09-21T15:01:08.256323+0000","last_became_active":"2024-09-21T14:58:09.328417+0000","last_became_peered":"2024-09-21T14:58:09.328417+0000","last_unstale":"2024-09-21T15:01:08.256323+0000","last_undegraded":"2024-09-21T15:01:08.256323+0000","last_fullsized":"2024-09-21T15:01:08.256323+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:52:22.671185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018917490000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"180'398","reported_seq":5452,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:01:10.562506+0000","last_change":"2024-09-21T15:01:04.228240+0000","last_active":"2024-09-21T15:01:10.562506+0000","last_peered":"2024-09-21T15:01:10.562506+0000","last_clean":"2024-09-21T15:01:10.562506+0000","last_became_active":"2024-09-21T14:59:32.131232+0000","last_became_peered":"2024-09-21T14:59:32.131232+0000","last_unstale":"2024-09-21T15:01:10.562506+0000","last_undegraded":"2024-09-21T15:01:10.562506+0000","last_fullsized":"2024-09-21T15:01:10.562506+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":398,"log_dups_size":0,"ondisk_log_size":398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:31:04.421782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001438581,"stat_sum":{"num_bytes":53173338,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":5286,"num_read_kb":51707,"num_write":539,"num_write_kb":27824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":3585713,"num_keys_recovered":9,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":99,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-21T15:01:08.242215+0000","last_change":"2024-09-21T14:59:28.228883+0000","last_active":"2024-09-21T15:01:08.242215+0000","last_peered":"2024-09-21T15:01:08.242215+0000","last_clean":"2024-09-21T15:01:08.242215+0000","last_became_active":"2024-09-21T14:59:28.228464+0000","last_became_peered":"2024-09-21T14:59:28.228464+0000","last_unstale":"2024-09-21T15:01:08.242215+0000","last_undegraded":"2024-09-21T15:01:08.242215+0000","last_fullsized":"2024-09-21T15:01:08.242215+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:00:26.310864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"180'818","reported_seq":3413,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:01:10.447321+0000","last_change":"2024-09-21T15:01:04.228668+0000","last_active":"2024-09-21T15:01:10.447321+0000","last_peered":"2024-09-21T15:01:10.447321+0000","last_clean":"2024-09-21T15:01:10.447321+0000","last_became_active":"2024-09-21T14:59:31.132581+0000","last_became_peered":"2024-09-21T14:59:31.132581+0000","last_unstale":"2024-09-21T15:01:10.447321+0000","last_undegraded":"2024-09-21T15:01:10.447321+0000","last_fullsized":"2024-09-21T15:01:10.447321+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":818,"log_dups_size":0,"ondisk_log_size":818,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:36:45.467515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016620179999999999,"stat_sum":{"num_bytes":31751011,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":4991,"num_read_kb":30499,"num_write":1244,"num_write_kb":25152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":119,"num_bytes_recovered":6871924,"num_keys_recovered":11,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":190,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-21T15:01:08.256889+0000","last_change":"2024-09-21T14:59:28.231502+0000","last_active":"2024-09-21T15:01:08.256889+0000","last_peered":"2024-09-21T15:01:08.256889+0000","last_clean":"2024-09-21T15:01:08.256889+0000","last_became_active":"2024-09-21T14:59:28.231043+0000","last_became_peered":"2024-09-21T14:59:28.231043+0000","last_unstale":"2024-09-21T15:01:08.256889+0000","last_undegraded":"2024-09-21T15:01:08.256889+0000","last_fullsized":"2024-09-21T15:01:08.256889+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:09:46.773690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019338770000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"180'562","reported_seq":5107,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:01:10.455452+0000","last_change":"2024-09-21T15:01:04.227811+0000","last_active":"2024-09-21T15:01:10.455452+0000","last_peered":"2024-09-21T15:01:10.455452+0000","last_clean":"2024-09-21T15:01:10.455452+0000","last_became_active":"2024-09-21T15:00:34.347238+0000","last_became_peered":"2024-09-21T15:00:34.347238+0000","last_unstale":"2024-09-21T15:01:10.455452+0000","last_undegraded":"2024-09-21T15:01:10.455452+0000","last_fullsized":"2024-09-21T15:01:10.455452+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":562,"log_dups_size":0,"ondisk_log_size":562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:31:33.639268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001277607,"stat_sum":{"num_bytes":44434212,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":5081,"num_read_kb":38156,"num_write":1014,"num_write_kb":41198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":14049436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":175,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-21T15:01:08.250709+0000","last_change":"2024-09-21T14:58:11.406176+0000","last_active":"2024-09-21T15:01:08.250709+0000","last_peered":"2024-09-21T15:01:08.250709+0000","last_clean":"2024-09-21T15:01:08.250709+0000","last_became_active":"2024-09-21T14:58:09.777465+0000","last_became_peered":"2024-09-21T14:58:09.777465+0000","last_unstale":"2024-09-21T15:01:08.250709+0000","last_undegraded":"2024-09-21T15:01:08.250709+0000","last_fullsized":"2024-09-21T15:01:08.250709+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:55:04.667816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095704000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"180'668","reported_seq":2549,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:01:10.244392+0000","last_change":"2024-09-21T15:01:04.224786+0000","last_active":"2024-09-21T15:01:10.244392+0000","last_peered":"2024-09-21T15:01:10.244392+0000","last_clean":"2024-09-21T15:01:10.244392+0000","last_became_active":"2024-09-21T14:58:19.408971+0000","last_became_peered":"2024-09-21T14:58:19.408971+0000","last_unstale":"2024-09-21T15:01:10.244392+0000","last_undegraded":"2024-09-21T15:01:10.244392+0000","last_fullsized":"2024-09-21T15:01:10.244392+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":668,"log_dups_size":0,"ondisk_log_size":668,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:29:52.642121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000380054,"stat_sum":{"num_bytes":22456262,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":3561,"num_read_kb":20372,"num_write":1165,"num_write_kb":20644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.3","version":"180'747","reported_seq":2955,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:01:10.533772+0000","last_change":"2024-09-21T15:01:04.231096+0000","last_active":"2024-09-21T15:01:10.533772+0000","last_peered":"2024-09-21T15:01:10.533772+0000","last_clean":"2024-09-21T15:01:10.533772+0000","last_became_active":"2024-09-21T14:58:19.417453+0000","last_became_peered":"2024-09-21T14:58:19.417453+0000","last_unstale":"2024-09-21T15:01:10.533772+0000","last_undegraded":"2024-09-21T15:01:10.533772+0000","last_fullsized":"2024-09-21T15:01:10.533772+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":747,"log_dups_size":0,"ondisk_log_size":747,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:19:27.087272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002785708,"stat_sum":{"num_bytes":19818390,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":4401,"num_read_kb":24174,"num_write":1307,"num_write_kb":21522,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":176,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-21T15:01:08.255930+0000","last_change":"2024-09-21T14:58:11.347504+0000","last_active":"2024-09-21T15:01:08.255930+0000","last_peered":"2024-09-21T15:01:08.255930+0000","last_clean":"2024-09-21T15:01:08.255930+0000","last_became_active":"2024-09-21T14:58:09.775981+0000","last_became_peered":"2024-09-21T14:58:09.775981+0000","last_unstale":"2024-09-21T15:01:08.255930+0000","last_undegraded":"2024-09-21T15:01:08.255930+0000","last_fullsized":"2024-09-21T15:01:08.255930+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:47:16.286652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012568880000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"180'945","reported_seq":3401,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:01:10.563901+0000","last_change":"2024-09-21T15:01:04.223338+0000","last_active":"2024-09-21T15:01:10.563901+0000","last_peered":"2024-09-21T15:01:10.563901+0000","last_clean":"2024-09-21T15:01:10.563901+0000","last_became_active":"2024-09-21T14:58:19.405811+0000","last_became_peered":"2024-09-21T14:58:19.405811+0000","last_unstale":"2024-09-21T15:01:10.563901+0000","last_undegraded":"2024-09-21T15:01:10.563901+0000","last_fullsized":"2024-09-21T15:01:10.563901+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":945,"log_dups_size":0,"ondisk_log_size":945,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:36:25.551607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083259800000000004,"stat_sum":{"num_bytes":23975054,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":5779,"num_read_kb":27902,"num_write":1544,"num_write_kb":26901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":173,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-21T15:01:05.184226+0000","last_change":"2024-09-21T14:58:11.342210+0000","last_active":"2024-09-21T15:01:05.184226+0000","last_peered":"2024-09-21T15:01:05.184226+0000","last_clean":"2024-09-21T15:01:05.184226+0000","last_became_active":"2024-09-21T14:58:09.330624+0000","last_became_peered":"2024-09-21T14:58:09.330624+0000","last_unstale":"2024-09-21T15:01:05.184226+0000","last_undegraded":"2024-09-21T15:01:05.184226+0000","last_fullsized":"2024-09-21T15:01:05.184226+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:29:43.229538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001051151,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"180'290","reported_seq":2025,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:01:10.451444+0000","last_change":"2024-09-21T15:01:05.166575+0000","last_active":"2024-09-21T15:01:10.451444+0000","last_peered":"2024-09-21T15:01:10.451444+0000","last_clean":"2024-09-21T15:01:10.451444+0000","last_became_active":"2024-09-21T15:01:05.164862+0000","last_became_peered":"2024-09-21T15:01:05.164862+0000","last_unstale":"2024-09-21T15:01:10.451444+0000","last_undegraded":"2024-09-21T15:01:10.451444+0000","last_fullsized":"2024-09-21T15:01:10.451444+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":290,"log_dups_size":0,"ondisk_log_size":290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:57:46.675335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046340299999999998,"stat_sum":{"num_bytes":37248411,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":0,"num_read":2367,"num_read_kb":39964,"num_write":480,"num_write_kb":39361,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":254,"num_bytes_recovered":20571253,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":174,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-21T15:01:08.255787+0000","last_change":"2024-09-21T14:58:11.346766+0000","last_active":"2024-09-21T15:01:08.255787+0000","last_peered":"2024-09-21T15:01:08.255787+0000","last_clean":"2024-09-21T15:01:08.255787+0000","last_became_active":"2024-09-21T14:58:09.329748+0000","last_became_peered":"2024-09-21T14:58:09.329748+0000","last_unstale":"2024-09-21T15:01:08.255787+0000","last_undegraded":"2024-09-21T15:01:08.255787+0000","last_fullsized":"2024-09-21T15:01:08.255787+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:31:20.267675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052698000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"180'770","reported_seq":7475,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:01:10.560976+0000","last_change":"2024-09-21T15:01:04.224483+0000","last_active":"2024-09-21T15:01:10.560976+0000","last_peered":"2024-09-21T15:01:10.560976+0000","last_clean":"2024-09-21T15:01:10.560976+0000","last_became_active":"2024-09-21T14:58:41.401003+0000","last_became_peered":"2024-09-21T14:58:41.401003+0000","last_unstale":"2024-09-21T15:01:10.560976+0000","last_undegraded":"2024-09-21T15:01:10.560976+0000","last_fullsized":"2024-09-21T15:01:10.560976+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":770,"log_dups_size":0,"ondisk_log_size":770,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:05:37.644835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046693700000000001,"stat_sum":{"num_bytes":53815073,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":9241,"num_read_kb":48522,"num_write":1137,"num_write_kb":24140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1403348,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":175,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-21T15:01:08.256169+0000","last_change":"2024-09-21T14:58:11.347700+0000","last_active":"2024-09-21T15:01:08.256169+0000","last_peered":"2024-09-21T15:01:08.256169+0000","last_clean":"2024-09-21T15:01:08.256169+0000","last_became_active":"2024-09-21T14:58:09.438842+0000","last_became_peered":"2024-09-21T14:58:09.438842+0000","last_unstale":"2024-09-21T15:01:08.256169+0000","last_undegraded":"2024-09-21T15:01:08.256169+0000","last_fullsized":"2024-09-21T15:01:08.256169+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:41:34.626249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001426194,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":323,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-21T15:01:08.243225+0000","last_change":"2024-09-21T14:58:03.616055+0000","last_active":"2024-09-21T15:01:08.243225+0000","last_peered":"2024-09-21T15:01:08.243225+0000","last_clean":"2024-09-21T15:01:08.243225+0000","last_became_active":"2024-09-21T14:58:03.613214+0000","last_became_peered":"2024-09-21T14:58:03.613214+0000","last_unstale":"2024-09-21T15:01:08.243225+0000","last_undegraded":"2024-09-21T15:01:08.243225+0000","last_fullsized":"2024-09-21T15:01:08.243225+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:02.180162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:02.180162+0000","last_clean_scrub_stamp":"2024-09-21T14:58:02.180162+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:29:05.860361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"180'613","reported_seq":4055,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:01:10.409385+0000","last_change":"2024-09-21T15:01:04.227318+0000","last_active":"2024-09-21T15:01:10.409385+0000","last_peered":"2024-09-21T15:01:10.409385+0000","last_clean":"2024-09-21T15:01:10.409385+0000","last_became_active":"2024-09-21T14:59:31.131460+0000","last_became_peered":"2024-09-21T14:59:31.131460+0000","last_unstale":"2024-09-21T15:01:10.409385+0000","last_undegraded":"2024-09-21T15:01:10.409385+0000","last_fullsized":"2024-09-21T15:01:10.409385+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":613,"log_dups_size":0,"ondisk_log_size":613,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:01:38.353476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098975100000000009,"stat_sum":{"num_bytes":41254141,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":4356,"num_read_kb":43318,"num_write":1090,"num_write_kb":30450,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":119,"num_bytes_recovered":6097618,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.9","version":"180'812","reported_seq":3018,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:01:10.447535+0000","last_change":"2024-09-21T15:01:04.232710+0000","last_active":"2024-09-21T15:01:10.447535+0000","last_peered":"2024-09-21T15:01:10.447535+0000","last_clean":"2024-09-21T15:01:10.447535+0000","last_became_active":"2024-09-21T14:59:28.233956+0000","last_became_peered":"2024-09-21T14:59:28.233956+0000","last_unstale":"2024-09-21T15:01:10.447535+0000","last_undegraded":"2024-09-21T15:01:10.447535+0000","last_fullsized":"2024-09-21T15:01:10.447535+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":812,"log_dups_size":0,"ondisk_log_size":812,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:29:09.682792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027159770000000001,"stat_sum":{"num_bytes":31459619,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":4523,"num_read_kb":33253,"num_write":1434,"num_write_kb":33626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":91,"num_bytes_recovered":6925016,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.a","version":"180'848","reported_seq":4312,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:01:10.504216+0000","last_change":"2024-09-21T15:01:04.234662+0000","last_active":"2024-09-21T15:01:10.504216+0000","last_peered":"2024-09-21T15:01:10.504216+0000","last_clean":"2024-09-21T15:01:10.504216+0000","last_became_active":"2024-09-21T15:00:27.280971+0000","last_became_peered":"2024-09-21T15:00:27.280971+0000","last_unstale":"2024-09-21T15:01:10.504216+0000","last_undegraded":"2024-09-21T15:01:10.504216+0000","last_fullsized":"2024-09-21T15:01:10.504216+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":848,"log_dups_size":0,"ondisk_log_size":848,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:26:07.617059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076445899999999999,"stat_sum":{"num_bytes":24461444,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":6575,"num_read_kb":30248,"num_write":1267,"num_write_kb":33063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":9228123,"num_keys_recovered":33,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.b","version":"178'455","reported_seq":2985,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-21T15:01:06.643684+0000","last_change":"2024-09-21T15:01:04.239682+0000","last_active":"2024-09-21T15:01:06.643684+0000","last_peered":"2024-09-21T15:01:06.643684+0000","last_clean":"2024-09-21T15:01:06.643684+0000","last_became_active":"2024-09-21T14:58:19.410679+0000","last_became_peered":"2024-09-21T14:58:19.410679+0000","last_unstale":"2024-09-21T15:01:06.643684+0000","last_undegraded":"2024-09-21T15:01:06.643684+0000","last_fullsized":"2024-09-21T15:01:06.643684+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:07:40.758932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039592400000000003,"stat_sum":{"num_bytes":31436179,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":2889,"num_read_kb":36109,"num_write":704,"num_write_kb":37196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.c","version":"180'415","reported_seq":4322,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:01:10.472448+0000","last_change":"2024-09-21T15:01:04.231357+0000","last_active":"2024-09-21T15:01:10.472448+0000","last_peered":"2024-09-21T15:01:10.472448+0000","last_clean":"2024-09-21T15:01:10.472448+0000","last_became_active":"2024-09-21T14:59:25.130057+0000","last_became_peered":"2024-09-21T14:59:25.130057+0000","last_unstale":"2024-09-21T15:01:10.472448+0000","last_undegraded":"2024-09-21T15:01:10.472448+0000","last_fullsized":"2024-09-21T15:01:10.472448+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":415,"log_dups_size":0,"ondisk_log_size":415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:53:33.752550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003377793,"stat_sum":{"num_bytes":26473401,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":4337,"num_read_kb":28452,"num_write":657,"num_write_kb":24589,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":9059600,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.d","version":"180'436","reported_seq":4931,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:01:10.344978+0000","last_change":"2024-09-21T15:01:04.233218+0000","last_active":"2024-09-21T15:01:10.344978+0000","last_peered":"2024-09-21T15:01:10.344978+0000","last_clean":"2024-09-21T15:01:10.344978+0000","last_became_active":"2024-09-21T14:58:19.407884+0000","last_became_peered":"2024-09-21T14:58:19.407884+0000","last_unstale":"2024-09-21T15:01:10.344978+0000","last_undegraded":"2024-09-21T15:01:10.344978+0000","last_fullsized":"2024-09-21T15:01:10.344978+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":436,"log_dups_size":0,"ondisk_log_size":436,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:28:57.111615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034473960000000001,"stat_sum":{"num_bytes":55848141,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":4783,"num_read_kb":55414,"num_write":842,"num_write_kb":38545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.e","version":"180'785","reported_seq":2866,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:01:10.395014+0000","last_change":"2024-09-21T15:01:05.176277+0000","last_active":"2024-09-21T15:01:10.395014+0000","last_peered":"2024-09-21T15:01:10.395014+0000","last_clean":"2024-09-21T15:01:10.395014+0000","last_became_active":"2024-09-21T15:01:05.172427+0000","last_became_peered":"2024-09-21T15:01:05.172427+0000","last_unstale":"2024-09-21T15:01:10.395014+0000","last_undegraded":"2024-09-21T15:01:10.395014+0000","last_fullsized":"2024-09-21T15:01:10.395014+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":785,"log_dups_size":0,"ondisk_log_size":785,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:28:18.710621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093917399999999998,"stat_sum":{"num_bytes":17357416,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":4155,"num_read_kb":16497,"num_write":1410,"num_write_kb":17911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":138,"num_bytes_recovered":7486496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.f","version":"180'449","reported_seq":5872,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:01:10.462551+0000","last_change":"2024-09-21T15:01:04.229660+0000","last_active":"2024-09-21T15:01:10.462551+0000","last_peered":"2024-09-21T15:01:10.462551+0000","last_clean":"2024-09-21T15:01:10.462551+0000","last_became_active":"2024-09-21T14:58:19.410406+0000","last_became_peered":"2024-09-21T14:58:19.410406+0000","last_unstale":"2024-09-21T15:01:10.462551+0000","last_undegraded":"2024-09-21T15:01:10.462551+0000","last_fullsized":"2024-09-21T15:01:10.462551+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:59:16.026939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014556770000000001,"stat_sum":{"num_bytes":51593222,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":5754,"num_read_kb":55172,"num_write":733,"num_write_kb":36722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.10","version":"180'444","reported_seq":4319,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:01:10.439141+0000","last_change":"2024-09-21T15:01:04.233986+0000","last_active":"2024-09-21T15:01:10.439141+0000","last_peered":"2024-09-21T15:01:10.439141+0000","last_clean":"2024-09-21T15:01:10.439141+0000","last_became_active":"2024-09-21T14:58:19.409935+0000","last_became_peered":"2024-09-21T14:58:19.409935+0000","last_unstale":"2024-09-21T15:01:10.439141+0000","last_undegraded":"2024-09-21T15:01:10.439141+0000","last_fullsized":"2024-09-21T15:01:10.439141+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":444,"log_dups_size":0,"ondisk_log_size":444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:10:36.252629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001679137,"stat_sum":{"num_bytes":53119703,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":3929,"num_read_kb":58277,"num_write":731,"num_write_kb":37234,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.11","version":"180'468","reported_seq":4579,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:01:10.428493+0000","last_change":"2024-09-21T15:01:05.168682+0000","last_active":"2024-09-21T15:01:10.428493+0000","last_peered":"2024-09-21T15:01:10.428493+0000","last_clean":"2024-09-21T15:01:10.428493+0000","last_became_active":"2024-09-21T15:01:05.165955+0000","last_became_peered":"2024-09-21T15:01:05.165955+0000","last_unstale":"2024-09-21T15:01:10.428493+0000","last_undegraded":"2024-09-21T15:01:10.428493+0000","last_fullsized":"2024-09-21T15:01:10.428493+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":468,"log_dups_size":0,"ondisk_log_size":468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:34:54.111119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010096230000000001,"stat_sum":{"num_bytes":28747420,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":5222,"num_read_kb":27938,"num_write":639,"num_write_kb":24066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":166,"num_bytes_recovered":11965028,"num_keys_recovered":33,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.12","version":"180'528","reported_seq":8023,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:01:10.331152+0000","last_change":"2024-09-21T15:01:04.224935+0000","last_active":"2024-09-21T15:01:10.331152+0000","last_peered":"2024-09-21T15:01:10.331152+0000","last_clean":"2024-09-21T15:01:10.331152+0000","last_became_active":"2024-09-21T15:00:07.155144+0000","last_became_peered":"2024-09-21T15:00:07.155144+0000","last_unstale":"2024-09-21T15:01:10.331152+0000","last_undegraded":"2024-09-21T15:01:10.331152+0000","last_fullsized":"2024-09-21T15:01:10.331152+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":528,"log_dups_size":0,"ondisk_log_size":528,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:58:00.133639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010497799999999999,"stat_sum":{"num_bytes":45585679,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":8710,"num_read_kb":36186,"num_write":911,"num_write_kb":20822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.13","version":"180'775","reported_seq":6248,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:01:10.313496+0000","last_change":"2024-09-21T15:01:04.229229+0000","last_active":"2024-09-21T15:01:10.313496+0000","last_peered":"2024-09-21T15:01:10.313496+0000","last_clean":"2024-09-21T15:01:10.313496+0000","last_became_active":"2024-09-21T14:58:19.412030+0000","last_became_peered":"2024-09-21T14:58:19.412030+0000","last_unstale":"2024-09-21T15:01:10.313496+0000","last_undegraded":"2024-09-21T15:01:10.313496+0000","last_fullsized":"2024-09-21T15:01:10.313496+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":775,"log_dups_size":0,"ondisk_log_size":775,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:44:29.282391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046048699999999998,"stat_sum":{"num_bytes":59042658,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":7985,"num_read_kb":59174,"num_write":1158,"num_write_kb":31480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"6e","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1163115124,"num_objects":1600,"num_object_clones":0,"num_object_copies":3200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1600,"num_whiteouts":0,"num_read":162474,"num_read_kb":1207540,"num_write":29892,"num_write_kb":935082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1920,"num_bytes_recovered":145354624,"num_keys_recovered":229,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":91877376,"data_stored":678011523,"data_compressed":13535607,"data_compressed_allocated":61632512,"data_compressed_original":653433985,"omap_allocated":0,"internal_metadata":0},"log_size":18344,"ondisk_log_size":18344,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":10404,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509481,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34628,"kb_used_data":7612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94337212,"statfs":{"total":96636764160,"available":96601305088,"internally_reserved":0,"allocated":7794688,"data_stored":48907094,"data_compressed":1141276,"data_compressed_allocated":4558848,"data_compressed_original":47268443,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.85499999999999998,"15min":0.84199999999999997},"min":{"1min":0.46500000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.872,"5min":0.93300000000000005,"15min":0.95199999999999996},"min":{"1min":0.54900000000000004,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.3839999999999999,"5min":1.917,"15min":1.917},"last":0.86499999999999999}]},{"osd":1,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.81000000000000005,"15min":0.77700000000000002},"min":{"1min":0.35499999999999998,"5min":0.312,"15min":0.312},"max":{"1min":1.2789999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.83199999999999996,"15min":0.82599999999999996},"min":{"1min":0.57399999999999995,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.208,"5min":2.2149999999999999,"15min":2.2149999999999999},"last":1.002}]},{"osd":2,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.82199999999999995,"15min":0.81200000000000006},"min":{"1min":0.50900000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.169,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.93500000000000005,"15min":0.94299999999999995},"min":{"1min":0.55600000000000005,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.355,"5min":1.466,"15min":1.466},"last":0.90500000000000003}]},{"osd":3,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.88500000000000001,"15min":0.86099999999999999},"min":{"1min":0.60999999999999999,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.401,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.97799999999999998,"15min":0.96899999999999997},"min":{"1min":0.68200000000000005,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.3420000000000001,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.81899999999999995}]},{"osd":4,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.92200000000000004,"15min":0.93300000000000005},"min":{"1min":0.42199999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.3089999999999999,"5min":2.7130000000000001,"15min":2.7130000000000001},"last":1.048},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.94499999999999995,"15min":0.92600000000000005},"min":{"1min":0.39000000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.425,"5min":2.125,"15min":2.125},"last":0.97699999999999998}]},{"osd":5,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":1.026,"15min":1.022},"min":{"1min":0.51300000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.5800000000000001,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.94299999999999995,"5min":1.0409999999999999,"15min":1.05},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.3029999999999999,"5min":3.1880000000000002,"15min":3.1880000000000002},"last":1.0700000000000001}]},{"osd":6,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.133,"15min":1.149},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.5329999999999999,"5min":3.2730000000000001,"15min":3.2730000000000001},"last":1.0269999999999999},{"interface":"front","average":{"1min":1.095,"5min":1.0629999999999999,"15min":1.03},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.6990000000000001,"5min":2.2879999999999998,"15min":2.2879999999999998},"last":1.109}]}]},{"osd":6,"up_from":14,"seq":60129542185,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64140,"kb_used_data":10876,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94307700,"statfs":{"total":96636764160,"available":96571084800,"internally_reserved":0,"allocated":11137024,"data_stored":74424794,"data_compressed":1553596,"data_compressed_allocated":6676480,"data_compressed_original":71733004,"omap_allocated":1593,"internal_metadata":54524359},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:01:08 2024","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.92500000000000004,"15min":0.95099999999999996},"min":{"1min":0.51900000000000002,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.2010000000000001,"5min":1.593,"15min":1.593},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.93300000000000005,"15min":0.94999999999999996},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.226,"5min":1.587,"15min":1.587},"last":0.55900000000000005}]},{"osd":1,"last update":"Sat Sep 21 15:01:08 2024","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.89400000000000002,"15min":0.91000000000000003},"min":{"1min":0.51700000000000002,"5min":0.502,"15min":0.502},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.90200000000000002,"15min":0.96199999999999997},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.70499999999999996}]},{"osd":2,"last update":"Sat Sep 21 15:01:08 2024","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.96199999999999997,"15min":0.98399999999999999},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.389,"5min":1.758,"15min":1.758},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.90900000000000003,"15min":0.91300000000000003},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.444,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.61599999999999999}]},{"osd":3,"last update":"Sat Sep 21 15:01:08 2024","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.94799999999999995,"15min":0.95899999999999996},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.5329999999999999,"5min":1.835,"15min":1.835},"last":0.92300000000000004},{"interface":"front","average":{"1min":1.016,"5min":1.0920000000000001,"15min":1.127},"min":{"1min":0.64600000000000002,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.4670000000000001,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.96099999999999997}]},{"osd":4,"last update":"Sat Sep 21 15:01:08 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.93500000000000005,"15min":0.99099999999999999},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":1.2,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.99099999999999999},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.98499999999999999,"15min":1.0209999999999999},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.3759999999999999,"5min":1.897,"15min":1.897},"last":0.86899999999999999}]},{"osd":5,"last update":"Sat Sep 21 15:01:08 2024","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":1.0449999999999999,"15min":1.1100000000000001},"min":{"1min":0.63500000000000001,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.256,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.63600000000000001},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0660000000000001,"15min":1.087},"min":{"1min":0.65400000000000003,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.609,"5min":1.609,"15min":1.609},"last":0.77400000000000002}]},{"osd":7,"last update":"Sat Sep 21 15:01:08 2024","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.206,"15min":1.244},"min":{"1min":0.78900000000000003,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.3109999999999999,"5min":1.8660000000000001,"15min":1.8660000000000001},"last":0.90000000000000002},{"interface":"front","average":{"1min":1,"5min":1.1970000000000001,"15min":1.2749999999999999},"min":{"1min":0.69499999999999995,"5min":0.63,"15min":0.63},"max":{"1min":1.298,"5min":2.004,"15min":2.004},"last":0.81200000000000006}]}]},{"osd":5,"up_from":14,"seq":60129542186,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65712,"kb_used_data":12452,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94306128,"statfs":{"total":96636764160,"available":96569475072,"internally_reserved":0,"allocated":12750848,"data_stored":73729824,"data_compressed":1254811,"data_compressed_allocated":6623232,"data_compressed_original":69388396,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:01:07 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.89900000000000002,"15min":0.98799999999999999},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.337,"5min":4.4800000000000004,"15min":4.4800000000000004},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.627,"5min":0.76800000000000002,"15min":0.80800000000000005},"min":{"1min":0.36099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.94999999999999996,"5min":1.181,"15min":1.181},"last":0.56999999999999995}]},{"osd":1,"last update":"Sat Sep 21 15:01:02 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.89600000000000002,"15min":0.92400000000000004},"min":{"1min":0.47699999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":2.7709999999999999,"5min":2.7709999999999999,"15min":2.7709999999999999},"last":0.753},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.91400000000000003,"15min":0.93400000000000005},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":2.823,"5min":2.823,"15min":2.823},"last":0.77700000000000002}]},{"osd":2,"last update":"Sat Sep 21 15:01:02 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.91200000000000003,"15min":0.97399999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.159,"5min":3.419,"15min":3.419},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.87,"15min":0.89600000000000002},"min":{"1min":0.53400000000000003,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0780000000000001,"5min":1.423,"15min":1.423},"last":0.58999999999999997}]},{"osd":3,"last update":"Sat Sep 21 15:01:02 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.80300000000000005,"15min":0.80700000000000005},"min":{"1min":0.58499999999999996,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.96699999999999997,"5min":1.054,"15min":1.0840000000000001},"min":{"1min":0.59899999999999998,"5min":0.435,"15min":0.435},"max":{"1min":1.272,"5min":2.6379999999999999,"15min":2.6379999999999999},"last":0.71899999999999997}]},{"osd":4,"last update":"Sat Sep 21 15:01:02 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.98899999999999999,"15min":1.016},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.5609999999999999,"5min":2.5470000000000002,"15min":2.5470000000000002},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.82099999999999995,"5min":1.0169999999999999,"15min":1.0780000000000001},"min":{"1min":0.39300000000000002,"5min":0.376,"15min":0.376},"max":{"1min":1.302,"5min":2.5099999999999998,"15min":2.5099999999999998},"last":0.46700000000000003}]},{"osd":6,"last update":"Sat Sep 21 15:01:02 2024","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.94299999999999995,"15min":0.96899999999999997},"min":{"1min":0.63,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":1.004,"5min":1.2450000000000001,"15min":1.3520000000000001},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.425,"5min":4.2469999999999999,"15min":4.2469999999999999},"last":0.84499999999999997}]},{"osd":7,"last update":"Sat Sep 21 15:01:07 2024","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.2569999999999999,"15min":1.3420000000000001},"min":{"1min":0.52500000000000002,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.383,"5min":3.3010000000000002,"15min":3.3010000000000002},"last":0.79300000000000004},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.107,"15min":1.135},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.462,"5min":2.5880000000000001,"15min":2.5880000000000001},"last":0.64300000000000002}]}]},{"osd":4,"up_from":14,"seq":60129542185,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68696,"kb_used_data":11344,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94303144,"statfs":{"total":96636764160,"available":96566419456,"internally_reserved":0,"allocated":11616256,"data_stored":80720875,"data_compressed":1548190,"data_compressed_allocated":7086080,"data_compressed_original":77864961,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.76000000000000001,"15min":0.77600000000000002},"min":{"1min":0.47499999999999998,"5min":0.311,"15min":0.311},"max":{"1min":1.1020000000000001,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.98899999999999999},{"interface":"front","average":{"1min":1.5860000000000001,"5min":1.0489999999999999,"15min":0.98099999999999998},"min":{"1min":0.49299999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":13.324,"5min":13.324,"15min":13.324},"last":1.2330000000000001}]},{"osd":1,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.85899999999999999,"15min":0.90600000000000003},"min":{"1min":0.46800000000000003,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.0609999999999999,"5min":1.347,"15min":1.347},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.82199999999999995,"15min":0.83899999999999997},"min":{"1min":0.502,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.92700000000000005}]},{"osd":2,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.88200000000000001,"15min":0.88},"min":{"1min":0.55100000000000005,"5min":0.501,"15min":0.501},"max":{"1min":1.47,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":1.1379999999999999},{"interface":"front","average":{"1min":0.98099999999999998,"5min":1.002,"15min":1.026},"min":{"1min":0.60699999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.3740000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.872}]},{"osd":3,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.90700000000000003,"15min":0.89500000000000002},"min":{"1min":0.61799999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.482,"5min":1.538,"15min":1.538},"last":0.76700000000000002},{"interface":"front","average":{"1min":1.024,"5min":1.0129999999999999,"15min":0.99199999999999999},"min":{"1min":0.56599999999999995,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.528,"5min":1.528,"15min":1.528},"last":1.0309999999999999}]},{"osd":5,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.97499999999999998,"15min":0.98399999999999999},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.579,"5min":1.579,"15min":1.579},"last":1.1899999999999999},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.94799999999999995,"15min":0.96399999999999997},"min":{"1min":0.61899999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":1.1619999999999999}]},{"osd":6,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.996,"15min":1.0109999999999999},"min":{"1min":0.55400000000000005,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.349,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.081,"15min":1.0900000000000001},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.665,"5min":1.665,"15min":1.665},"last":1.274}]},{"osd":7,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.995,"15min":1.0249999999999999},"min":{"1min":0.45200000000000001,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.226,"5min":1.7010000000000001,"15min":1.7010000000000001},"last":1.091},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.2050000000000001,"15min":1.2130000000000001},"min":{"1min":0.627,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":1.321}]}]},{"osd":3,"up_from":14,"seq":60129542186,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59424,"kb_used_data":14356,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94312416,"statfs":{"total":96636764160,"available":96575913984,"internally_reserved":0,"allocated":14700544,"data_stored":103127883,"data_compressed":2074542,"data_compressed_allocated":9273344,"data_compressed_original":99696219,"omap_allocated":1592,"internal_metadata":46135752},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:01:09 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.625,"15min":0.63400000000000001},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.171,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.68200000000000005,"15min":0.70999999999999996},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.0700000000000001,"5min":1.919,"15min":1.919},"last":0.29199999999999998}]},{"osd":1,"last update":"Sat Sep 21 15:01:09 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.66400000000000003,"15min":0.67200000000000004},"min":{"1min":0.29499999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":1.722,"5min":1.722,"15min":1.722},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.76500000000000001,"15min":0.80300000000000005},"min":{"1min":0.33300000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.89500000000000002,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.33400000000000002}]},{"osd":2,"last update":"Sat Sep 21 15:01:09 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.71999999999999997,"15min":0.72199999999999998},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.92400000000000004,"5min":1.157,"15min":1.157},"last":0.311},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.81000000000000005,"15min":0.82699999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.1539999999999999,"5min":1.254,"15min":1.254},"last":0.39400000000000002}]},{"osd":4,"last update":"Sat Sep 21 15:01:09 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.93799999999999994,"15min":0.95599999999999996},"min":{"1min":0.54300000000000004,"5min":0.442,"15min":0.442},"max":{"1min":1.2629999999999999,"5min":1.7609999999999999,"15min":1.7609999999999999},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.98899999999999999,"15min":1.0269999999999999},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.234,"5min":2.028,"15min":2.028},"last":0.73999999999999999}]},{"osd":5,"last update":"Sat Sep 21 15:01:09 2024","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":1.0369999999999999,"15min":1.052},"min":{"1min":0.70499999999999996,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.3440000000000001,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":0.76400000000000001},{"interface":"front","average":{"1min":1.0249999999999999,"5min":1.056,"15min":1.0629999999999999},"min":{"1min":0.57999999999999996,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.548,"5min":1.548,"15min":1.548},"last":0.82499999999999996}]},{"osd":6,"last update":"Sat Sep 21 15:01:09 2024","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":1.0669999999999999,"15min":1.0920000000000001},"min":{"1min":0.59099999999999997,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.6419999999999999,"5min":2.1219999999999999,"15min":2.1219999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.091,"15min":1.103},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.6839999999999999,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":0.48799999999999999}]},{"osd":7,"last update":"Sat Sep 21 15:01:09 2024","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":1.103,"15min":1.131},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.498,"5min":1.706,"15min":1.706},"last":0.55900000000000005},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.147,"15min":1.177},"min":{"1min":0.59699999999999998,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.417,"5min":2.3380000000000001,"15min":2.3380000000000001},"last":0.78600000000000003}]}]},{"osd":2,"up_from":14,"seq":60129542184,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72532,"kb_used_data":15172,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94299308,"statfs":{"total":96636764160,"available":96562491392,"internally_reserved":0,"allocated":15536128,"data_stored":98629584,"data_compressed":1955612,"data_compressed_allocated":8757248,"data_compressed_original":93754402,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.62},"min":{"1min":0.26900000000000002,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.92800000000000005,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":1.383},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.69399999999999995,"15min":0.67000000000000004},"min":{"1min":0.34799999999999998,"5min":0.223,"15min":0.223},"max":{"1min":2.9169999999999998,"5min":2.9169999999999998,"15min":2.9169999999999998},"last":0.42899999999999999}]},{"osd":1,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.32700000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.0529999999999999,"5min":2.6520000000000001,"15min":2.6520000000000001},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.746,"5min":0.73499999999999999,"15min":0.75800000000000001},"min":{"1min":0.40500000000000003,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.349,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":1.4550000000000001}]},{"osd":3,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.73999999999999999,"15min":0.73499999999999999},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":3.0110000000000001,"5min":3.0110000000000001,"15min":3.0110000000000001},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.877,"15min":0.83899999999999997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.05,"5min":4.1440000000000001,"15min":4.1440000000000001},"last":0.86399999999999999}]},{"osd":4,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.93200000000000005,"15min":0.95599999999999996},"min":{"1min":0.42399999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.222,"5min":2.899,"15min":2.899},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.94399999999999995,"15min":0.95799999999999996},"min":{"1min":0.47499999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.4379999999999999,"5min":1.675,"15min":1.675},"last":1.5489999999999999}]},{"osd":5,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.0129999999999999,"15min":1.0089999999999999},"min":{"1min":0.45500000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":3.1080000000000001,"5min":3.1080000000000001,"15min":3.1080000000000001},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.98199999999999998,"15min":0.99399999999999999},"min":{"1min":0.51700000000000002,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.268,"5min":1.546,"15min":1.546},"last":0.55200000000000005}]},{"osd":6,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":0.98299999999999998,"15min":0.98499999999999999},"min":{"1min":0.56200000000000006,"5min":0.378,"15min":0.378},"max":{"1min":1.345,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":1.516},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.0649999999999999,"15min":1.0549999999999999},"min":{"1min":0.53200000000000003,"5min":0.502,"15min":0.502},"max":{"1min":3.0739999999999998,"5min":3.0739999999999998,"15min":3.0739999999999998},"last":0.48699999999999999}]},{"osd":7,"last update":"Sat Sep 21 15:01:08 2024","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.0449999999999999,"15min":1.0089999999999999},"min":{"1min":0.59499999999999997,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":3.1589999999999998,"5min":3.1589999999999998,"15min":3.1589999999999998},"last":0.61399999999999999},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.121,"15min":1.099},"min":{"1min":0.66000000000000003,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":3.1400000000000001,"5min":3.1400000000000001,"15min":3.1400000000000001},"last":0.58799999999999997}]}]},{"osd":1,"up_from":14,"seq":60129542185,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":71084,"kb_used_data":13732,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94300756,"statfs":{"total":96636764160,"available":96563974144,"internally_reserved":0,"allocated":14061568,"data_stored":96758848,"data_compressed":1883265,"data_compressed_allocated":8507392,"data_compressed_original":93072883,"omap_allocated":1595,"internal_metadata":58718661},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.59599999999999997,"15min":0.60899999999999999},"min":{"1min":0.36399999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.78200000000000003,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59699999999999998,"15min":0.60799999999999998},"min":{"1min":0.39500000000000002,"5min":0.216,"15min":0.216},"max":{"1min":0.80100000000000005,"5min":1.107,"15min":1.107},"last":0.57399999999999995}]},{"osd":2,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":0.79600000000000004,"15min":0.73999999999999999},"min":{"1min":0.38700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":11.303000000000001,"5min":11.303000000000001,"15min":11.303000000000001},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.71299999999999997,"15min":0.70299999999999996},"min":{"1min":0.47299999999999998,"5min":0.435,"15min":0.435},"max":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.74099999999999999}]},{"osd":3,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.81799999999999995,"15min":0.83899999999999997},"min":{"1min":0.27800000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.0049999999999999,"5min":3.1469999999999998,"15min":3.1469999999999998},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.86399999999999999,"15min":0.88600000000000001},"min":{"1min":0.52800000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.306,"5min":3.2320000000000002,"15min":3.2320000000000002},"last":0.86599999999999999}]},{"osd":4,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.89000000000000001,"15min":0.91500000000000004},"min":{"1min":0.59599999999999997,"5min":0.309,"15min":0.309},"max":{"1min":1.45,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":0.68200000000000005},{"interface":"front","average":{"1min":1.5600000000000001,"5min":1.069,"15min":0.96599999999999997},"min":{"1min":0.504,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":11.563000000000001,"5min":11.563000000000001,"15min":11.563000000000001},"last":1.0580000000000001}]},{"osd":5,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":1.625,"5min":1.0469999999999999,"15min":0.94899999999999995},"min":{"1min":0.73399999999999999,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":11.534000000000001,"5min":11.534000000000001,"15min":11.534000000000001},"last":1.1599999999999999},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95799999999999996,"15min":0.96099999999999997},"min":{"1min":0.63,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.256,"5min":1.484,"15min":1.484},"last":0.91500000000000004}]},{"osd":6,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":1.0109999999999999,"15min":1.0489999999999999},"min":{"1min":0.60399999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.4059999999999999,"5min":2.3519999999999999,"15min":2.3519999999999999},"last":0.84699999999999998},{"interface":"front","average":{"1min":1.631,"5min":1.081,"15min":0.99299999999999999},"min":{"1min":0.57399999999999995,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":11.601000000000001,"5min":11.601000000000001,"15min":11.601000000000001},"last":1.117}]},{"osd":7,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":1.046,"15min":1.0640000000000001},"min":{"1min":0.79300000000000004,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.5820000000000001,"5min":2.2360000000000002,"15min":2.2360000000000002},"last":0.96299999999999997},{"interface":"front","average":{"1min":1.76,"5min":1.171,"15min":1.087},"min":{"1min":0.877,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":11.489000000000001,"5min":11.489000000000001,"15min":11.489000000000001},"last":1.2010000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542184,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":71592,"kb_used_data":14232,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94300248,"statfs":{"total":96636764160,"available":96563453952,"internally_reserved":0,"allocated":14573568,"data_stored":103321822,"data_compressed":2127509,"data_compressed_allocated":9379840,"data_compressed_original":100115005,"omap_allocated":1593,"internal_metadata":58718663},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 21 15:01:04 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.67500000000000004,"15min":0.71699999999999997},"min":{"1min":0.40999999999999998,"5min":0.309,"15min":0.309},"max":{"1min":1.1299999999999999,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":1.002},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.68100000000000005,"15min":0.627},"min":{"1min":0.58999999999999997,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.88100000000000001}]},{"osd":2,"last update":"Sat Sep 21 15:01:04 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.65200000000000002,"15min":0.61799999999999999},"min":{"1min":0.59599999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.72499999999999998,"15min":0.75800000000000001},"min":{"1min":0.39300000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.28799999999999998}]},{"osd":3,"last update":"Sat Sep 21 15:01:04 2024","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.76600000000000001,"15min":0.754},"min":{"1min":0.50900000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.78700000000000003,"15min":0.81499999999999995},"min":{"1min":0.50900000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.58599999999999997}]},{"osd":4,"last update":"Sat Sep 21 15:01:04 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.98399999999999999,"15min":1.038},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.1140000000000001,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":0.95199999999999996},{"interface":"front","average":{"1min":1.036,"5min":1.0469999999999999,"15min":1.0620000000000001},"min":{"1min":0.67800000000000005,"5min":0.379,"15min":0.379},"max":{"1min":1.446,"5min":1.665,"15min":1.665},"last":0.44}]},{"osd":5,"last update":"Sat Sep 21 15:01:04 2024","interfaces":[{"interface":"back","average":{"1min":1.0960000000000001,"5min":1.024,"15min":1.032},"min":{"1min":0.79300000000000004,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.4450000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.45800000000000002},{"interface":"front","average":{"1min":1.0589999999999999,"5min":1.0429999999999999,"15min":1.077},"min":{"1min":0.82399999999999995,"5min":0.435,"15min":0.435},"max":{"1min":1.2609999999999999,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":0.378}]},{"osd":6,"last update":"Sat Sep 21 15:01:04 2024","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":1.0580000000000001,"15min":1.1120000000000001},"min":{"1min":0.68400000000000005,"5min":0.628,"15min":0.628},"max":{"1min":1.167,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":0.89400000000000002},{"interface":"front","average":{"1min":1.109,"5min":1.0629999999999999,"15min":1.0680000000000001},"min":{"1min":0.91000000000000003,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.5029999999999999,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.47399999999999998}]},{"osd":7,"last update":"Sat Sep 21 15:01:04 2024","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.0469999999999999,"15min":1.0620000000000001},"min":{"1min":0.78700000000000003,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.4770000000000001,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":0.34399999999999997},{"interface":"front","average":{"1min":1.02,"5min":1.085,"15min":1.131},"min":{"1min":0.84799999999999998,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.403,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":0.92600000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5202,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5202,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":14073856,"data_stored":104108982,"data_compressed":2129014,"data_compressed_allocated":9555968,"data_compressed_original":100467261,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12804096,"data_stored":96214214,"data_compressed":1879183,"data_compressed_allocated":8601600,"data_compressed_original":92761587,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":13369344,"data_stored":97577590,"data_compressed":1955612,"data_compressed_allocated":8757248,"data_compressed_original":93754402,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":15351808,"data_stored":105270750,"data_compressed":2077727,"data_compressed_allocated":9646080,"data_compressed_original":100441691,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":10043392,"data_stored":80360275,"data_compressed":1549030,"data_compressed_allocated":7184384,"data_compressed_original":78061569,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":10526720,"data_stored":72087462,"data_compressed":1249609,"data_compressed_allocated":6586368,"data_compressed_original":68814956,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":10055680,"data_stored":74524637,"data_compressed":1554156,"data_compressed_allocated":6742016,"data_compressed_original":71864076,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":5652480,"data_stored":47867613,"data_compressed":1141276,"data_compressed_allocated":4558848,"data_compressed_original":47268443,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T15:01:11.910 INFO:teuthology.orchestra.run.smithi032.stdout:5769 write 0x1380a85 thru 0x1381f3e (0x14ba bytes) 2024-09-21T15:01:11.910 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:01:11.898+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:01:11.936 INFO:teuthology.orchestra.run.smithi032.stdout:5770 trunc from 0xde456af to 0x14ef09e 2024-09-21T15:01:12.005 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:01:12.002+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:01:12.047 INFO:teuthology.orchestra.run.smithi032.stdout:5771 write 0x669b8a7 thru 0x669fb37 (0x4291 bytes) 2024-09-21T15:01:12.051 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:01:12.051 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:01:12.051 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T15:01:12.051 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:01:12.061 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T15:01:12.062 INFO:teuthology.orchestra.run.smithi032.stdout:5772 punch from 0x48bef13 to 0x48c4a3d, (0x5b2a bytes) 2024-09-21T15:01:12.071 INFO:teuthology.orchestra.run.smithi032.stdout:5773 trunc from 0x669fb38 to 0xb5bb8f4 2024-09-21T15:01:12.084 INFO:teuthology.orchestra.run.smithi032.stdout:5776 write 0x640e7 thru 0x6d17d (0x9097 bytes) 2024-09-21T15:01:12.089 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:01:12.094 INFO:teuthology.orchestra.run.smithi032.stdout:5778 trunc from 0xb5bb8f4 to 0x911efc0 2024-09-21T15:01:12.106 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:01:12.106 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:01:12.102+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:01:12.123 INFO:teuthology.orchestra.run.smithi032.stdout:5780 write 0xa122b83 thru 0xa1284e4 (0x5962 bytes) 2024-09-21T15:01:12.163 INFO:teuthology.orchestra.run.smithi032.stdout:5782 write 0xd94d4e5 thru 0xd95778d (0xa2a9 bytes) 2024-09-21T15:01:12.174 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:01:12.174 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:01:12.175 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T15:01:12.175 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T15:01:12.177 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:01:12.184 INFO:teuthology.orchestra.run.smithi032.stdout:5785 trunc from 0xd95778e to 0xb45b9d7 2024-09-21T15:01:12.185 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T15:01:12.205 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:01:12.205 INFO:teuthology.orchestra.run.smithi032.stdout:5788 punch from 0x1b594c1 to 0x1b5d5ca, (0x4109 bytes) 2024-09-21T15:01:12.205 INFO:teuthology.orchestra.run.smithi032.stdout:5789 punch from 0x83bf063 to 0x83c99c3, (0xa960 bytes) 2024-09-21T15:01:12.205 INFO:teuthology.orchestra.run.smithi032.stdout:5790 read 0x497a4f3 thru 0x4981076 (0x6b84 bytes) 2024-09-21T15:01:12.205 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:01:12.205 INFO:teuthology.orchestra.run.smithi032.stdout:5793 read 0x49953c4 thru 0x499c20b (0x6e48 bytes) 2024-09-21T15:01:12.205 INFO:teuthology.orchestra.run.smithi032.stdout:5795 read 0x16fe776 thru 0x17097b4 (0xb03f bytes) 2024-09-21T15:01:12.206 INFO:teuthology.orchestra.run.smithi032.stdout:5798 read 0x7eea2d7 thru 0x7ef2e93 (0x8bbd bytes) 2024-09-21T15:01:12.206 INFO:teuthology.orchestra.run.smithi032.stdout:5799 read 0x1689ee8 thru 0x168d0fb (0x3214 bytes) 2024-09-21T15:01:12.206 INFO:teuthology.orchestra.run.smithi032.stdout:5800 write 0x4ffe67e thru 0x5009cd6 (0xb659 bytes) 2024-09-21T15:01:12.207 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:01:12.205+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:01:12.224 INFO:teuthology.orchestra.run.smithi032.stdout:5801 read 0x82a711f thru 0x82b1ea2 (0xad84 bytes) 2024-09-21T15:01:12.228 INFO:teuthology.orchestra.run.smithi032.stdout:5803 trunc from 0xb45b9d7 to 0x9421bac 2024-09-21T15:01:12.249 INFO:teuthology.orchestra.run.smithi032.stdout:5805 read 0x648d13d thru 0x6496c56 (0x9b1a bytes) 2024-09-21T15:01:12.266 INFO:teuthology.orchestra.run.smithi032.stdout:5806 trunc from 0x9421bac to 0x168dff2 2024-09-21T15:01:12.299 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:01:12.299 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T15:01:12.299 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T15:01:12.299 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T15:01:12.299 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.299 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31476 3.c 3:35718b6b:::rbd_data.11367721c19.0000000000000014:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T15:01:12.299 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.505975+0000", 2024-09-21T15:01:12.300 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 133.78982611199999, 2024-09-21T15:01:12.300 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99515995899999998, 2024-09-21T15:01:12.300 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:12.300 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:12.300 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:12.300 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:12.300 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:12.300 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31476 2024-09-21T15:01:12.300 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.300 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:12.300 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.300 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:12.300 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.505975+0000", 2024-09-21T15:01:12.300 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:12.300 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.301 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.301 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:12.301 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.505975+0000", 2024-09-21T15:01:12.301 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:12.301 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.301 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.301 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:12.301 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.505971+0000", 2024-09-21T15:01:12.301 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999967 2024-09-21T15:01:12.301 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.301 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.301 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:12.301 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.505989+0000", 2024-09-21T15:01:12.301 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7985000000000001e-05 2024-09-21T15:01:12.301 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.302 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.302 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:12.302 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506050+0000", 2024-09-21T15:01:12.302 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0659000000000003e-05 2024-09-21T15:01:12.302 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.302 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.302 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:12.302 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506074+0000", 2024-09-21T15:01:12.302 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4156e-05 2024-09-21T15:01:12.302 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.302 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.302 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:12.302 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506225+0000", 2024-09-21T15:01:12.302 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015097 2024-09-21T15:01:12.303 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.303 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.303 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:12.303 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506256+0000", 2024-09-21T15:01:12.303 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0505999999999999e-05 2024-09-21T15:01:12.303 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.303 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.303 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:12.303 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.497974+0000", 2024-09-21T15:01:12.303 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99171825700000005 2024-09-21T15:01:12.303 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.303 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.303 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for missing object", 2024-09-21T15:01:12.303 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.498449+0000", 2024-09-21T15:01:12.304 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00047506399999999998 2024-09-21T15:01:12.304 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.304 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.304 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:12.304 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.501000+0000", 2024-09-21T15:01:12.304 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0025506919999999998 2024-09-21T15:01:12.304 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.304 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.304 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:12.304 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.501135+0000", 2024-09-21T15:01:12.304 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013524000000000001 2024-09-21T15:01:12.304 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:12.304 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:12.304 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:12.305 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.305 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.305 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31480 3.c 3:36ccd808:::rbd_data.11367721c19.0000000000000018:head [sparse-read 0~14821] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T15:01:12.305 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.506116+0000", 2024-09-21T15:01:12.305 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 133.78968476899999, 2024-09-21T15:01:12.305 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99316033599999998, 2024-09-21T15:01:12.305 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:12.305 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:12.305 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:12.305 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:12.305 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:12.305 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31480 2024-09-21T15:01:12.305 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.306 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:12.306 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.306 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:12.306 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506116+0000", 2024-09-21T15:01:12.306 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:12.306 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.306 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.306 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:12.306 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506116+0000", 2024-09-21T15:01:12.306 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:12.306 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.306 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.306 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:12.306 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506115+0000", 2024-09-21T15:01:12.307 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:12.307 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.307 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.307 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:12.307 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506124+0000", 2024-09-21T15:01:12.307 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0659999999999993e-06 2024-09-21T15:01:12.307 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.307 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.307 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:12.307 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506145+0000", 2024-09-21T15:01:12.307 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1447e-05 2024-09-21T15:01:12.307 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.307 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.308 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:12.308 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506161+0000", 2024-09-21T15:01:12.308 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5209e-05 2024-09-21T15:01:12.308 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.308 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.308 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:12.308 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506337+0000", 2024-09-21T15:01:12.308 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017642999999999999 2024-09-21T15:01:12.308 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.308 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.308 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:12.308 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506375+0000", 2024-09-21T15:01:12.308 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7536000000000001e-05 2024-09-21T15:01:12.308 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.309 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.309 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:12.309 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.498815+0000", 2024-09-21T15:01:12.309 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99244067999999996 2024-09-21T15:01:12.309 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.309 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.309 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:12.309 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.498950+0000", 2024-09-21T15:01:12.309 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013436700000000001 2024-09-21T15:01:12.309 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.309 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.309 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:12.309 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499277+0000", 2024-09-21T15:01:12.310 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00032696800000000002 2024-09-21T15:01:12.310 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:12.310 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:12.310 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:12.310 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.310 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.310 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31513 3.c 3:36de8523:::rbd_data.11367721c19.0000000000000039:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T15:01:12.310 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.509071+0000", 2024-09-21T15:01:12.310 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 133.78673056900001, 2024-09-21T15:01:12.310 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99038225099999999, 2024-09-21T15:01:12.310 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:12.310 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:12.310 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:12.311 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:12.311 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:12.311 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31513 2024-09-21T15:01:12.311 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.311 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:12.311 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.311 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:12.311 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509071+0000", 2024-09-21T15:01:12.311 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:12.311 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.311 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.311 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:12.311 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509071+0000", 2024-09-21T15:01:12.311 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:12.312 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.312 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.312 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:12.312 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509068+0000", 2024-09-21T15:01:12.312 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-21T15:01:12.312 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.312 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.312 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:12.312 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509082+0000", 2024-09-21T15:01:12.312 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3545e-05 2024-09-21T15:01:12.312 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.312 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.312 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:12.312 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509116+0000", 2024-09-21T15:01:12.312 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3942000000000001e-05 2024-09-21T15:01:12.313 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.313 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.313 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:12.313 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509134+0000", 2024-09-21T15:01:12.313 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8839999999999999e-05 2024-09-21T15:01:12.313 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.313 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.313 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:12.313 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509318+0000", 2024-09-21T15:01:12.313 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018326799999999999 2024-09-21T15:01:12.313 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.313 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.313 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:12.314 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509346+0000", 2024-09-21T15:01:12.314 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8019000000000001e-05 2024-09-21T15:01:12.314 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.314 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.314 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:12.314 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499330+0000", 2024-09-21T15:01:12.314 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.98998399299999995 2024-09-21T15:01:12.314 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.314 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.314 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:12.314 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499453+0000", 2024-09-21T15:01:12.314 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000123115 2024-09-21T15:01:12.314 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:12.314 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:12.315 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:12.315 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.315 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.315 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31515 3.c 3:35b36ddd:::rbd_data.11367721c19.000000000000003b:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T15:01:12.315 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.509177+0000", 2024-09-21T15:01:12.315 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 133.78662414300001, 2024-09-21T15:01:12.315 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99044870600000001, 2024-09-21T15:01:12.315 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:12.315 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:12.315 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:12.315 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:12.315 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:12.315 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31515 2024-09-21T15:01:12.316 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.316 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:12.316 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.316 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:12.316 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509177+0000", 2024-09-21T15:01:12.316 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:12.316 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.316 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.316 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:12.316 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509177+0000", 2024-09-21T15:01:12.316 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:12.316 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.316 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.316 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:12.316 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509176+0000", 2024-09-21T15:01:12.317 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:12.317 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.317 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.317 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:12.317 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509183+0000", 2024-09-21T15:01:12.317 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6720000000000004e-06 2024-09-21T15:01:12.317 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.317 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.317 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:12.317 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509269+0000", 2024-09-21T15:01:12.317 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5881e-05 2024-09-21T15:01:12.317 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.317 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.317 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:12.317 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509303+0000", 2024-09-21T15:01:12.317 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3367000000000001e-05 2024-09-21T15:01:12.318 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.318 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.318 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:12.318 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509425+0000", 2024-09-21T15:01:12.318 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000122199 2024-09-21T15:01:12.318 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.318 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.318 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:12.318 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509448+0000", 2024-09-21T15:01:12.318 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2688999999999999e-05 2024-09-21T15:01:12.318 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.318 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.318 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:12.318 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499501+0000", 2024-09-21T15:01:12.318 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99005306999999998 2024-09-21T15:01:12.319 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.319 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.319 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:12.319 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499626+0000", 2024-09-21T15:01:12.319 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012501400000000001 2024-09-21T15:01:12.319 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:12.319 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:12.319 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:12.319 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.319 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.319 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31531 3.c 3:314f0287:::rbd_data.11367721c19.000000000000004b:head [sparse-read 0~7106] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T15:01:12.319 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.509329+0000", 2024-09-21T15:01:12.319 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 133.78647212300001, 2024-09-21T15:01:12.319 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99103085300000004, 2024-09-21T15:01:12.319 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:12.320 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:12.320 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:12.320 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:12.320 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:12.320 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31531 2024-09-21T15:01:12.320 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.320 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:12.320 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.320 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:12.320 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509329+0000", 2024-09-21T15:01:12.320 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:12.320 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.320 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.320 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:12.320 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509329+0000", 2024-09-21T15:01:12.321 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:12.321 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.321 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.321 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:12.321 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509328+0000", 2024-09-21T15:01:12.321 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:12.321 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.321 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.321 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:12.321 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509336+0000", 2024-09-21T15:01:12.321 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3170000000000002e-06 2024-09-21T15:01:12.321 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.321 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.321 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:12.321 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509353+0000", 2024-09-21T15:01:12.321 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7214000000000002e-05 2024-09-21T15:01:12.322 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.322 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.322 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:12.322 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509366+0000", 2024-09-21T15:01:12.322 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.26e-05 2024-09-21T15:01:12.322 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.322 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.322 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:12.322 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509523+0000", 2024-09-21T15:01:12.322 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015708000000000001 2024-09-21T15:01:12.322 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.322 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.322 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:12.322 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509544+0000", 2024-09-21T15:01:12.322 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0783000000000001e-05 2024-09-21T15:01:12.323 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.323 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.323 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:12.323 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499727+0000", 2024-09-21T15:01:12.323 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99018345200000002 2024-09-21T15:01:12.323 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.323 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.323 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:12.323 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499949+0000", 2024-09-21T15:01:12.323 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000221358 2024-09-21T15:01:12.323 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.323 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.323 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:12.323 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.500360+0000", 2024-09-21T15:01:12.323 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00041130700000000002 2024-09-21T15:01:12.324 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:12.324 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:12.324 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:12.324 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.324 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.324 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:82545 3.b 3:d4eb1ee5:::rbd_data.11368296f791.0000000000000000:head [write 0~671744 [fadvise_sequential+fadvise_nocache],write 704512~65536 [fadvise_sequential+fadvise_nocache],write 786432~49152 [fadvise_sequential+fadvise_nocache],write 868352~49152 [fadvise_sequential+fadvise_nocache],write 933888~65536 [fadvise_sequential+fadvise_nocache],write 1032192~32768 [fadvise_sequential+fadvise_nocache],write 1097728~65536 [fadvise_sequential+fadvise_nocache],write 1179648~49152 [fadvise_sequential+fadvise_nocache],write 1261568~49152 [fadvise_sequential+fadvise_nocache],write 1327104~65536 [fadvise_sequential+fadvise_nocache],write 1425408~32768 [fadvise_sequential+fadvise_nocache],write 1490944~65536 [fadvise_sequential+fadvise_nocache],write 1572864~8192 [fadvise_sequential+fadvise_nocache],write 1630208~49152 [fadvise_sequential+fadvise_nocache],write 1703936~40960 [fadvise_sequential+fadvise_nocache],write 1794048~40960 [fadvise_sequential+fadvise_nocache],write 1859584~49152 [fadvise_sequential+fadvise_nocache],write 1957888~8192 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:01:12.324 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:47.416089+0000", 2024-09-21T15:01:12.324 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 84.879712397999995, 2024-09-21T15:01:12.324 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011896455, 2024-09-21T15:01:12.324 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:12.324 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:12.324 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:12.324 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:12.324 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:12.324 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82545 2024-09-21T15:01:12.325 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.325 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:12.325 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.325 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:12.325 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.416089+0000", 2024-09-21T15:01:12.325 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:12.325 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.325 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.325 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:12.325 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.416089+0000", 2024-09-21T15:01:12.325 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:12.325 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.325 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.325 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:12.325 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.416087+0000", 2024-09-21T15:01:12.326 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:01:12.326 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.326 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.326 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:12.326 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.419643+0000", 2024-09-21T15:01:12.326 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0035555550000000002 2024-09-21T15:01:12.326 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.326 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.326 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:12.326 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.419663+0000", 2024-09-21T15:01:12.326 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0202000000000001e-05 2024-09-21T15:01:12.326 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.326 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.326 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:12.326 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.419676+0000", 2024-09-21T15:01:12.327 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3207000000000001e-05 2024-09-21T15:01:12.327 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.327 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.327 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:12.327 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.419759+0000", 2024-09-21T15:01:12.327 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3521000000000002e-05 2024-09-21T15:01:12.327 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.327 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.327 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op must wait for map", 2024-09-21T15:01:12.327 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.419768+0000", 2024-09-21T15:01:12.327 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4479999999999994e-06 2024-09-21T15:01:12.327 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.327 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.327 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:12.327 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.422171+0000", 2024-09-21T15:01:12.327 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0024035469999999998 2024-09-21T15:01:12.328 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.328 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.328 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:12.328 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.422350+0000", 2024-09-21T15:01:12.328 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000179001 2024-09-21T15:01:12.328 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.328 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.328 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T15:01:12.328 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.422710+0000", 2024-09-21T15:01:12.328 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000359878 2024-09-21T15:01:12.328 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.328 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.328 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:01:12.328 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.426676+0000", 2024-09-21T15:01:12.328 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0039655790000000003 2024-09-21T15:01:12.328 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.329 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.329 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:01:12.329 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.427884+0000", 2024-09-21T15:01:12.329 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001208207 2024-09-21T15:01:12.329 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.329 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.329 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:01:12.329 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.427953+0000", 2024-09-21T15:01:12.329 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8533000000000003e-05 2024-09-21T15:01:12.329 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.329 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.329 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:12.329 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.427985+0000", 2024-09-21T15:01:12.329 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2481999999999998e-05 2024-09-21T15:01:12.330 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:12.330 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:12.330 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:12.330 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.330 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.330 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:97675 3.b 3:d321acb8:::rbd_data.1136bfc79a70.0000000000000003:head [write 0~4718592 [fadvise_sequential+fadvise_nocache],write 5242880~147456 [fadvise_sequential+fadvise_nocache],write 5455872~147456 [fadvise_sequential+fadvise_nocache],write 5668864~229376 [fadvise_sequential+fadvise_nocache],write 5963776~147456 [fadvise_sequential+fadvise_nocache],write 6176768~229376 [fadvise_sequential+fadvise_nocache],write 6471680~147456 [fadvise_sequential+fadvise_nocache],write 6684672~229376 [fadvise_sequential+fadvise_nocache],write 6979584~147456 [fadvise_sequential+fadvise_nocache],write 7192576~229376 [fadvise_sequential+fadvise_nocache],write 7487488~147456 [fadvise_sequential+fadvise_nocache],write 7700480~147456 [fadvise_sequential+fadvise_nocache],write 7864320~65536 [fadvise_sequential+fadvise_nocache],write 7995392~147456 [fadvise_sequential+fadvise_nocache],write 8208384~147456 [fadvise_sequential+fadvise_nocache],write 8388608~49152 [fadvise_sequential+fadvise_nocache],write 8536064~114688 [fadvise_sequential+fadvise_nocache],write 8749056~114688 [fadvise_sequential+fadvise_nocache],write 8912896~32768 [fadvise_sequential+fadvise_nocache],write 9125888~32768 [fadvise_sequential+fadvise_nocache],write 9338880~32768 [fadvise_sequential+fadvise_nocache],write 9437184~16384 [fadvise_sequential+fadvise_nocache],write 9633792~32768 [fadvise_sequential+fadvise_nocache],write 9846784~32768 [fadvise_sequential+fadvise_nocache],write 9961472~114688 [fadvise_sequential+fadvise_nocache],write 10092544~196608 [fadvise_sequential+fadvise_nocache],write 10305536~278528 [fadvise_sequential+fadvise_nocache],write 10600448~196608 [fadvise_sequential+fadvise_nocache],write 10813440~278528 [fadvise_sequential+fadvise_nocache],write 11108352~196608 [fadvise_sequential+fadvise_nocache],write 11321344~196608 [fadvise_sequential+fadvise_nocache],write 11534336~65536 [fadvise_sequential+fadvise_nocache],write 11632640~98304 [fadvise_sequential+fadvise_nocache],write 11747328~65536 [fadvise_sequential+fadvise_nocache],write 11845632~98304 [fadvise_sequential+fadvise_nocache],write 11960320~65536 [fadvise_sequential+fadvise_nocache],write 12058624~49152 [fadvise_sequential+fadvise_nocache],write 12140544~98304 [fadvise_sequential+fadvise_nocache],write 12255232~65536 [fadvise_sequential+fadvise_nocache],write 12353536~98304 [fadvise_sequential+fadvise_nocache],write 12468224~65536 [fadvise_sequential+fadvise_nocache],write 12566528~16384 [fadvise_sequential+fadvise_nocache],write 12648448~98304 [fadvise_sequential+fadvise_nocache],write 12861440~98304 [fadvise_sequential+fadvise_nocache],write 13074432~32768 [fadvise_sequential+fadvise_nocache],write 13156352~98304 [fadvise_sequential+fadvise_nocache],write 13369344~98304 [fadvise_sequential+fadvise_nocache],write 13582336~49152 [fadvise_sequential+fadvise_nocache],write 13664256~98304 [fadvise_sequential+fadvise_nocache],write 13877248~98304 [fadvise_sequential+fadvise_nocache],write 14090240~65536 [fadvise_sequential+fadvise_nocache],write 15286272~81920 [fadvise_sequential+fadvise_nocache],write 15499264~81920 [fadvise_sequential+fadvise_nocache],write 15712256~16384 [fadvise_sequential+fadvise_nocache],write 15794176~81920 [fadvise_sequential+fadvise_nocache],write 16007168~81920 [fadvise_sequential+fadvise_nocache],write 16220160~32768 [fadvise_sequential+fadvise_nocache],write 16302080~81920 [fadvise_sequential+fadvise_nocache],write 16515072~81920 [fadvise_sequential+fadvise_nocache],write 16728064~49152 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:01:12.330 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.302412+0000", 2024-09-21T15:01:12.330 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 66.993389609000005, 2024-09-21T15:01:12.330 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.067380532000000007, 2024-09-21T15:01:12.330 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:12.330 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:12.330 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:12.330 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:12.330 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:12.331 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97675 2024-09-21T15:01:12.331 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.331 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:12.331 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.331 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:12.331 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.302412+0000", 2024-09-21T15:01:12.331 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:12.331 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.331 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.331 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:12.331 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.302412+0000", 2024-09-21T15:01:12.331 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:12.331 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.331 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.331 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:12.332 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.302409+0000", 2024-09-21T15:01:12.332 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-21T15:01:12.332 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.332 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.332 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:12.332 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328550+0000", 2024-09-21T15:01:12.332 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.026140674999999999 2024-09-21T15:01:12.332 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.332 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.332 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:12.332 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328584+0000", 2024-09-21T15:01:12.332 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4165999999999998e-05 2024-09-21T15:01:12.332 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.332 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.332 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:12.332 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328599+0000", 2024-09-21T15:01:12.333 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5505000000000001e-05 2024-09-21T15:01:12.333 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.333 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.333 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:12.333 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328704+0000", 2024-09-21T15:01:12.333 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010487199999999999 2024-09-21T15:01:12.333 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.333 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.333 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:12.333 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.329082+0000", 2024-09-21T15:01:12.333 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00037804899999999998 2024-09-21T15:01:12.333 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.333 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.334 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T15:01:12.334 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.331158+0000", 2024-09-21T15:01:12.334 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0020755449999999998 2024-09-21T15:01:12.334 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.334 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.334 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:01:12.334 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.360553+0000", 2024-09-21T15:01:12.334 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.029395553000000001 2024-09-21T15:01:12.334 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.334 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.334 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:01:12.334 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.369496+0000", 2024-09-21T15:01:12.334 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0089428619999999993 2024-09-21T15:01:12.334 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.335 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.335 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:01:12.335 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.369686+0000", 2024-09-21T15:01:12.335 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019000399999999999 2024-09-21T15:01:12.335 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.335 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.335 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:12.335 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.369792+0000", 2024-09-21T15:01:12.335 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000105775 2024-09-21T15:01:12.335 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:12.335 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:12.335 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:12.335 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.335 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.335 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:97693 3.b 3:d7d7b39b:::rbd_data.1136b9c49b1c.000000000000000c:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:01:12.336 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.328643+0000", 2024-09-21T15:01:12.336 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 66.967157822999994, 2024-09-21T15:01:12.336 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.031062136000000001, 2024-09-21T15:01:12.336 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:12.336 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:12.336 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:12.336 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:12.336 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:12.336 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97693 2024-09-21T15:01:12.336 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.336 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:12.336 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.336 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:12.336 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328643+0000", 2024-09-21T15:01:12.336 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:12.336 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.337 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.337 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:12.337 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328643+0000", 2024-09-21T15:01:12.337 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:12.337 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.337 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.337 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:12.337 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328642+0000", 2024-09-21T15:01:12.337 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:01:12.337 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.337 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.337 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:12.337 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328650+0000", 2024-09-21T15:01:12.337 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7770000000000008e-06 2024-09-21T15:01:12.337 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.338 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.338 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:12.338 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328662+0000", 2024-09-21T15:01:12.338 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2026e-05 2024-09-21T15:01:12.338 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.338 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.338 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:12.338 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328671+0000", 2024-09-21T15:01:12.338 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.585e-06 2024-09-21T15:01:12.338 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.338 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.338 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:12.338 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.359498+0000", 2024-09-21T15:01:12.338 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.030827001 2024-09-21T15:01:12.338 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.339 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.339 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:12.339 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.359603+0000", 2024-09-21T15:01:12.339 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010512400000000001 2024-09-21T15:01:12.339 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.339 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.339 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:12.339 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.359705+0000", 2024-09-21T15:01:12.339 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000102327 2024-09-21T15:01:12.339 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:12.339 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:12.339 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:12.339 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.339 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.339 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:97724 3.b 3:d4bf6bcf:::rbd_data.1136b9c49b1c.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:01:12.340 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.328704+0000", 2024-09-21T15:01:12.340 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 66.967097550999995, 2024-09-21T15:01:12.340 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.031157957, 2024-09-21T15:01:12.340 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:12.340 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:12.340 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:12.340 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:12.340 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:12.340 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97724 2024-09-21T15:01:12.340 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.340 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:12.340 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.340 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:12.340 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328704+0000", 2024-09-21T15:01:12.340 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:12.341 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.341 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.341 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:12.341 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328704+0000", 2024-09-21T15:01:12.341 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:12.341 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.341 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.341 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:12.341 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328703+0000", 2024-09-21T15:01:12.341 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:12.341 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.341 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.341 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:12.341 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328708+0000", 2024-09-21T15:01:12.341 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0050000000000004e-06 2024-09-21T15:01:12.341 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.342 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.342 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:12.342 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328719+0000", 2024-09-21T15:01:12.342 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1052e-05 2024-09-21T15:01:12.342 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.342 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.342 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:12.342 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328727+0000", 2024-09-21T15:01:12.342 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4519999999999997e-06 2024-09-21T15:01:12.342 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.342 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.342 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:12.342 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.359737+0000", 2024-09-21T15:01:12.342 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.031009341999999999 2024-09-21T15:01:12.343 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.343 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.343 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:12.343 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.359806+0000", 2024-09-21T15:01:12.343 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9750000000000001e-05 2024-09-21T15:01:12.343 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.343 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.343 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:12.343 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.359862+0000", 2024-09-21T15:01:12.343 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5084999999999999e-05 2024-09-21T15:01:12.343 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:12.343 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:12.343 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:12.343 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.343 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.343 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:129970 3.c e158/87 3:33fa4d4e:::rbd_data.1136a12e4351.0000000000000000:head v 158'379, mlcod=156'377)", 2024-09-21T15:01:12.344 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:44.452766+0000", 2024-09-21T15:01:12.344 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 27.843034999, 2024-09-21T15:01:12.344 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011942062, 2024-09-21T15:01:12.344 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:12.344 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:12.344 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:12.344 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.344 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:12.344 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:44.452766+0000", 2024-09-21T15:01:12.344 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:12.344 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.344 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.344 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:12.344 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:44.452766+0000", 2024-09-21T15:01:12.345 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:12.345 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.345 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.345 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:12.345 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:44.452766+0000", 2024-09-21T15:01:12.345 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:12.345 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.345 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.345 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:12.345 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:44.458962+0000", 2024-09-21T15:01:12.345 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0061964799999999999 2024-09-21T15:01:12.345 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.345 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.345 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:12.345 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:44.459012+0000", 2024-09-21T15:01:12.346 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9427e-05 2024-09-21T15:01:12.346 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.346 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.346 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:12.346 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:44.459042+0000", 2024-09-21T15:01:12.346 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9838000000000001e-05 2024-09-21T15:01:12.346 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.346 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.346 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:12.346 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:44.459115+0000", 2024-09-21T15:01:12.346 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3430999999999995e-05 2024-09-21T15:01:12.346 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.346 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.346 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:12.347 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:44.459135+0000", 2024-09-21T15:01:12.347 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9701999999999999e-05 2024-09-21T15:01:12.347 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.347 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.347 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:01:12.347 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:44.464645+0000", 2024-09-21T15:01:12.347 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0055100820000000003 2024-09-21T15:01:12.347 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.347 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.347 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:12.347 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:44.464708+0000", 2024-09-21T15:01:12.347 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3399999999999996e-05 2024-09-21T15:01:12.347 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:12.347 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:12.348 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:12.348 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.348 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.348 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:135368 3.b 3:d3301c39:::rbd_data.11365ed419e3.0000000000000102:67 [read 335872~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:12.348 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:53.998901+0000", 2024-09-21T15:01:12.348 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 18.296900186999999, 2024-09-21T15:01:12.348 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011666174, 2024-09-21T15:01:12.348 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:12.348 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:12.348 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:12.348 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:12.348 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:12.348 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 135368 2024-09-21T15:01:12.348 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.348 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:12.349 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.349 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:12.349 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998901+0000", 2024-09-21T15:01:12.349 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:12.349 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.349 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.349 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:12.349 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998901+0000", 2024-09-21T15:01:12.349 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:12.349 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.349 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.349 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:12.349 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998901+0000", 2024-09-21T15:01:12.349 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:12.349 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.349 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.350 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:12.350 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998903+0000", 2024-09-21T15:01:12.350 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5500000000000001e-06 2024-09-21T15:01:12.350 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.350 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.350 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:12.350 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998913+0000", 2024-09-21T15:01:12.350 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6479999999999998e-06 2024-09-21T15:01:12.350 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.350 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.350 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:12.350 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998919+0000", 2024-09-21T15:01:12.350 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7930000000000003e-06 2024-09-21T15:01:12.350 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.351 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.351 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:12.351 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:54.010277+0000", 2024-09-21T15:01:12.351 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011358692 2024-09-21T15:01:12.351 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.351 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.351 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:12.351 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:54.010348+0000", 2024-09-21T15:01:12.351 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0247999999999996e-05 2024-09-21T15:01:12.351 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.351 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.351 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:12.351 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:54.010567+0000", 2024-09-21T15:01:12.351 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021953599999999999 2024-09-21T15:01:12.351 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:12.352 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:12.352 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:12.352 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.352 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.352 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:135369 3.b 3:d3301c39:::rbd_data.11365ed419e3.0000000000000102:67 [read 352256~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:12.352 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:53.998923+0000", 2024-09-21T15:01:12.352 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 18.296877886000001, 2024-09-21T15:01:12.352 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011990331999999999, 2024-09-21T15:01:12.352 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:12.352 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:12.352 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:12.352 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:12.352 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:12.352 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 135369 2024-09-21T15:01:12.352 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.353 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:12.353 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.353 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:12.353 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998923+0000", 2024-09-21T15:01:12.353 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:12.353 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.353 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.353 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:12.353 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998923+0000", 2024-09-21T15:01:12.353 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:12.353 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.353 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.353 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:12.353 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998923+0000", 2024-09-21T15:01:12.353 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:12.353 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.354 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.354 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:12.354 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998926+0000", 2024-09-21T15:01:12.354 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6749999999999998e-06 2024-09-21T15:01:12.354 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.354 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.354 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:12.354 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998935+0000", 2024-09-21T15:01:12.354 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3619999999999995e-06 2024-09-21T15:01:12.354 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.354 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.354 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:12.354 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998940+0000", 2024-09-21T15:01:12.354 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2379999999999997e-06 2024-09-21T15:01:12.354 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.355 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.355 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:12.355 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:54.010593+0000", 2024-09-21T15:01:12.355 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011652531000000001 2024-09-21T15:01:12.355 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.355 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.355 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:12.355 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:54.010679+0000", 2024-09-21T15:01:12.355 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6811000000000006e-05 2024-09-21T15:01:12.355 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.355 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.355 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:12.355 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:54.010914+0000", 2024-09-21T15:01:12.355 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000234071 2024-09-21T15:01:12.356 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:12.356 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:12.356 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:12.356 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.356 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.356 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:137705 3.b 3:d0cb966d:::rbd_data.1136f64ddcfe.0000000000000005:6a [read 1327104~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-21T15:01:12.356 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:58.376355+0000", 2024-09-21T15:01:12.356 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 13.919446268, 2024-09-21T15:01:12.356 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011637611000000001, 2024-09-21T15:01:12.356 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:12.356 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:12.356 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:12.356 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:12.356 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:12.356 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 137705 2024-09-21T15:01:12.356 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.357 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:12.357 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.357 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:12.357 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376355+0000", 2024-09-21T15:01:12.357 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:12.357 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.357 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.357 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:12.357 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376355+0000", 2024-09-21T15:01:12.357 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:12.357 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.357 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.357 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:12.357 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376354+0000", 2024-09-21T15:01:12.358 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:12.358 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.358 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.358 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:12.358 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376358+0000", 2024-09-21T15:01:12.358 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.179e-06 2024-09-21T15:01:12.358 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.358 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.358 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:12.358 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376369+0000", 2024-09-21T15:01:12.358 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1766000000000001e-05 2024-09-21T15:01:12.358 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.358 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.358 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:12.358 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376376+0000", 2024-09-21T15:01:12.359 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6019999999999996e-06 2024-09-21T15:01:12.359 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.359 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.359 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:12.359 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.387882+0000", 2024-09-21T15:01:12.359 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011506126 2024-09-21T15:01:12.359 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.359 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.359 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:12.359 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.387929+0000", 2024-09-21T15:01:12.359 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6742000000000001e-05 2024-09-21T15:01:12.359 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.359 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.359 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:12.359 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.387992+0000", 2024-09-21T15:01:12.359 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3656000000000002e-05 2024-09-21T15:01:12.360 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:12.360 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:12.360 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:12.360 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.360 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.360 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:137706 3.b 3:d0cb966d:::rbd_data.1136f64ddcfe.0000000000000005:6a [read 1441792~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-21T15:01:12.360 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:58.376385+0000", 2024-09-21T15:01:12.360 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 13.919416299, 2024-09-21T15:01:12.360 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011783609, 2024-09-21T15:01:12.360 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:12.360 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:12.360 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:12.360 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:12.360 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:12.360 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 137706 2024-09-21T15:01:12.361 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.361 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:12.361 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.361 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:12.361 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376385+0000", 2024-09-21T15:01:12.361 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:12.361 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.361 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.361 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:12.361 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376385+0000", 2024-09-21T15:01:12.361 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:12.361 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.361 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.361 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:12.361 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376384+0000", 2024-09-21T15:01:12.361 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:12.362 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.362 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.362 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:12.362 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376387+0000", 2024-09-21T15:01:12.362 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9809999999999999e-06 2024-09-21T15:01:12.362 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.362 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.362 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:12.362 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376398+0000", 2024-09-21T15:01:12.362 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0319000000000001e-05 2024-09-21T15:01:12.362 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.362 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.362 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:12.362 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376404+0000", 2024-09-21T15:01:12.363 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5339999999999996e-06 2024-09-21T15:01:12.363 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.363 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.363 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:12.363 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.388025+0000", 2024-09-21T15:01:12.363 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011620449999999999 2024-09-21T15:01:12.363 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.363 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.363 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:12.363 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.388082+0000", 2024-09-21T15:01:12.363 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7065999999999999e-05 2024-09-21T15:01:12.363 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.363 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.363 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:12.363 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.388168+0000", 2024-09-21T15:01:12.364 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6689999999999998e-05 2024-09-21T15:01:12.364 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:12.364 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:12.364 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:12.364 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.364 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.364 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:137707 3.b 3:d0cb966d:::rbd_data.1136f64ddcfe.0000000000000005:6a [read 1556480~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-21T15:01:12.364 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:58.376428+0000", 2024-09-21T15:01:12.364 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 13.919372807, 2024-09-21T15:01:12.364 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012004865999999999, 2024-09-21T15:01:12.364 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:12.364 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:12.364 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:12.364 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:12.364 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:12.365 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 137707 2024-09-21T15:01:12.365 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.365 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:12.365 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.365 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:12.365 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376428+0000", 2024-09-21T15:01:12.365 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:12.365 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.365 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.365 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:12.365 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376428+0000", 2024-09-21T15:01:12.365 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:12.365 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.365 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.365 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:12.366 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376428+0000", 2024-09-21T15:01:12.366 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:12.366 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.366 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.366 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:12.366 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376432+0000", 2024-09-21T15:01:12.366 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2119999999999997e-06 2024-09-21T15:01:12.366 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.366 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.366 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:12.366 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376448+0000", 2024-09-21T15:01:12.366 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5913000000000002e-05 2024-09-21T15:01:12.366 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.367 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.367 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:12.367 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376455+0000", 2024-09-21T15:01:12.367 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6359999999999994e-06 2024-09-21T15:01:12.367 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.367 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.367 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:12.367 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.388197+0000", 2024-09-21T15:01:12.367 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011741843 2024-09-21T15:01:12.367 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.367 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.367 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:12.367 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.388249+0000", 2024-09-21T15:01:12.367 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1549999999999999e-05 2024-09-21T15:01:12.367 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.368 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.368 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:12.368 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.388433+0000", 2024-09-21T15:01:12.368 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018437999999999999 2024-09-21T15:01:12.368 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:12.368 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:12.368 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:12.368 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.368 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.368 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:137708 3.b 3:d0cb966d:::rbd_data.1136f64ddcfe.0000000000000005:6a [read 1671168~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-21T15:01:12.368 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:58.376462+0000", 2024-09-21T15:01:12.368 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 13.919339019000001, 2024-09-21T15:01:12.368 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012315491, 2024-09-21T15:01:12.368 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:12.368 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:12.369 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:12.369 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:12.369 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:12.369 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 137708 2024-09-21T15:01:12.369 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.369 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:12.369 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.369 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:12.369 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376462+0000", 2024-09-21T15:01:12.369 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:12.369 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.369 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.369 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:12.369 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376462+0000", 2024-09-21T15:01:12.369 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:12.369 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.370 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.370 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:12.370 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376461+0000", 2024-09-21T15:01:12.370 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:12.370 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.370 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.370 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:12.370 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376465+0000", 2024-09-21T15:01:12.370 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7069999999999998e-06 2024-09-21T15:01:12.370 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.370 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.370 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:12.370 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376473+0000", 2024-09-21T15:01:12.370 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6669999999999996e-06 2024-09-21T15:01:12.370 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.371 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.371 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:12.371 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376479+0000", 2024-09-21T15:01:12.371 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.263e-06 2024-09-21T15:01:12.371 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.371 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.371 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:12.371 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.388489+0000", 2024-09-21T15:01:12.371 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012009953 2024-09-21T15:01:12.371 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.371 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.371 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:12.371 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.388625+0000", 2024-09-21T15:01:12.371 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013548799999999999 2024-09-21T15:01:12.371 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.372 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.372 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:12.372 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.388778+0000", 2024-09-21T15:01:12.372 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015302299999999999 2024-09-21T15:01:12.372 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:12.372 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:12.372 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:12.372 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.372 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.372 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:137709 3.b 3:d0cb966d:::rbd_data.1136f64ddcfe.0000000000000005:6a [read 1785856~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-21T15:01:12.372 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:58.376485+0000", 2024-09-21T15:01:12.372 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 13.919315686999999, 2024-09-21T15:01:12.372 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012466549, 2024-09-21T15:01:12.372 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:12.373 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:12.373 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:12.373 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:12.373 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:12.373 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 137709 2024-09-21T15:01:12.373 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.373 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:12.373 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.373 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:12.373 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376485+0000", 2024-09-21T15:01:12.373 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:12.373 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.373 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.373 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:12.373 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376485+0000", 2024-09-21T15:01:12.373 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:12.374 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.374 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.374 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:12.374 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376485+0000", 2024-09-21T15:01:12.374 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:12.374 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.374 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.374 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:12.374 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376492+0000", 2024-09-21T15:01:12.374 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8929999999999999e-06 2024-09-21T15:01:12.374 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.374 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.374 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:12.375 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376500+0000", 2024-09-21T15:01:12.375 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7989999999999994e-06 2024-09-21T15:01:12.375 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.375 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.375 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:12.375 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376506+0000", 2024-09-21T15:01:12.375 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4799999999999998e-06 2024-09-21T15:01:12.375 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.375 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.375 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:12.375 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.388810+0000", 2024-09-21T15:01:12.375 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012303727 2024-09-21T15:01:12.375 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.375 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.375 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:12.376 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.388871+0000", 2024-09-21T15:01:12.376 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1278000000000005e-05 2024-09-21T15:01:12.376 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.376 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.376 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:12.376 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.388952+0000", 2024-09-21T15:01:12.376 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0770000000000001e-05 2024-09-21T15:01:12.376 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:12.376 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:12.376 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:12.376 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.376 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.376 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:137710 3.b 3:d0cb966d:::rbd_data.1136f64ddcfe.0000000000000005:6a [read 1900544~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-21T15:01:12.376 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:58.376512+0000", 2024-09-21T15:01:12.376 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 13.919289484, 2024-09-21T15:01:12.377 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012746351, 2024-09-21T15:01:12.377 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:12.377 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:12.377 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:12.377 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:12.377 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:12.377 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 137710 2024-09-21T15:01:12.377 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.377 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:12.377 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.377 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:12.377 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376512+0000", 2024-09-21T15:01:12.377 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:12.377 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.377 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.377 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:12.378 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376512+0000", 2024-09-21T15:01:12.378 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:12.378 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.378 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.378 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:12.378 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376511+0000", 2024-09-21T15:01:12.378 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:12.378 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.378 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.378 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:12.378 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376514+0000", 2024-09-21T15:01:12.378 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1719999999999999e-06 2024-09-21T15:01:12.378 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.378 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.379 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:12.379 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376522+0000", 2024-09-21T15:01:12.379 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1389999999999998e-06 2024-09-21T15:01:12.379 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.379 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.379 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:12.379 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376535+0000", 2024-09-21T15:01:12.379 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3658999999999999e-05 2024-09-21T15:01:12.379 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.379 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.379 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:12.379 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.389005+0000", 2024-09-21T15:01:12.379 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012470261 2024-09-21T15:01:12.379 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.379 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.380 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:12.380 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.389119+0000", 2024-09-21T15:01:12.380 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000113346 2024-09-21T15:01:12.380 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.380 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.380 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:12.380 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.389258+0000", 2024-09-21T15:01:12.380 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013920100000000001 2024-09-21T15:01:12.380 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:12.380 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:12.380 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:12.380 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.380 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.380 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:137711 3.b 3:d0cb966d:::rbd_data.1136f64ddcfe.0000000000000005:6a [read 2015232~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-21T15:01:12.381 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:58.376541+0000", 2024-09-21T15:01:12.381 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 13.919260122000001, 2024-09-21T15:01:12.381 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012849955999999999, 2024-09-21T15:01:12.381 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:12.381 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:12.381 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:12.381 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:12.381 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:12.381 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 137711 2024-09-21T15:01:12.381 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.381 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:12.381 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.381 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:12.381 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376541+0000", 2024-09-21T15:01:12.382 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:12.382 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.382 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.382 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:12.382 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376541+0000", 2024-09-21T15:01:12.382 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:12.382 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.382 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.382 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:12.382 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376540+0000", 2024-09-21T15:01:12.382 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:12.382 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.382 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.382 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:12.383 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376544+0000", 2024-09-21T15:01:12.383 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3249999999999999e-06 2024-09-21T15:01:12.383 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.383 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.383 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:12.383 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376551+0000", 2024-09-21T15:01:12.383 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2189999999999998e-06 2024-09-21T15:01:12.383 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.383 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.383 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:12.383 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376557+0000", 2024-09-21T15:01:12.383 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8869999999999997e-06 2024-09-21T15:01:12.383 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.383 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.383 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:12.384 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.389277+0000", 2024-09-21T15:01:12.384 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012720555999999999 2024-09-21T15:01:12.384 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.384 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.384 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:12.384 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.389325+0000", 2024-09-21T15:01:12.384 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7367000000000002e-05 2024-09-21T15:01:12.384 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.384 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.384 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:12.384 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.389391+0000", 2024-09-21T15:01:12.384 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6095999999999999e-05 2024-09-21T15:01:12.384 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:12.384 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:12.384 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:12.384 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.385 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.385 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:137723 3.b 3:d0cb966d:::rbd_data.1136f64ddcfe.0000000000000005:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-21T15:01:12.385 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:58.377091+0000", 2024-09-21T15:01:12.385 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 13.918709801, 2024-09-21T15:01:12.385 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012594424, 2024-09-21T15:01:12.385 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:12.385 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:12.385 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:12.385 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:12.385 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:12.385 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 137723 2024-09-21T15:01:12.385 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.385 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:12.385 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.386 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:12.386 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.377091+0000", 2024-09-21T15:01:12.386 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:12.386 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.386 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.386 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:12.386 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.377091+0000", 2024-09-21T15:01:12.386 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:12.386 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.386 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.386 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:12.386 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.377090+0000", 2024-09-21T15:01:12.386 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:12.386 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.386 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.387 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:12.387 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.377097+0000", 2024-09-21T15:01:12.387 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7100000000000001e-06 2024-09-21T15:01:12.387 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.387 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.387 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:12.387 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.377120+0000", 2024-09-21T15:01:12.387 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3189000000000001e-05 2024-09-21T15:01:12.387 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.387 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.387 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:12.387 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.377132+0000", 2024-09-21T15:01:12.387 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2041999999999999e-05 2024-09-21T15:01:12.387 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.387 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.388 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:12.388 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.389439+0000", 2024-09-21T15:01:12.388 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.01230698 2024-09-21T15:01:12.388 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.388 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.388 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:12.388 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.389556+0000", 2024-09-21T15:01:12.388 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000116446 2024-09-21T15:01:12.388 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:12.388 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:12.388 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:12.388 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.389686+0000", 2024-09-21T15:01:12.388 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000129966 2024-09-21T15:01:12.388 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:12.388 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:12.388 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:12.389 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:12.389 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:12.389 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:01:12.390 INFO:teuthology.orchestra.run.smithi032.stdout:5807 writesame 0xc5a0086 thru 0xc5a18aa data_size 0x1(0x1825 bytes) 2024-09-21T15:01:12.390 INFO:teuthology.orchestra.run.smithi032.stdout:5808 trunc from 0xc5a18ab to 0x5496170 2024-09-21T15:01:12.390 INFO:teuthology.orchestra.run.smithi032.stdout:5809 punch from 0x10397e2 to 0x1040015, (0x6833 bytes) 2024-09-21T15:01:12.390 INFO:teuthology.orchestra.run.smithi032.stdout:5810 write 0xe63e7da thru 0xe642ef3 (0x471a bytes) 2024-09-21T15:01:12.390 INFO:teuthology.orchestra.run.smithi032.stdout:5811 read 0x58bc4e6 thru 0x58bc7b6 (0x2d1 bytes) 2024-09-21T15:01:12.390 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:01:12.386+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:01:12.391 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T15:01:12.395 INFO:teuthology.orchestra.run.smithi032.stdout:5812 trunc from 0xe642ef4 to 0xacf44b0 2024-09-21T15:01:12.415 INFO:teuthology.orchestra.run.smithi032.stdout:5813 read 0x6070e26 thru 0x607cb11 (0xbcec bytes) 2024-09-21T15:01:12.415 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:01:12.429 INFO:teuthology.orchestra.run.smithi032.stdout:5815 write 0xb2567ef thru 0xb26473d (0xdf4f bytes) 2024-09-21T15:01:12.465 INFO:teuthology.orchestra.run.smithi032.stdout:5816 write 0xec0a5fb thru 0xec0f29c (0x4ca2 bytes) 2024-09-21T15:01:12.481 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:01:12.481 INFO:teuthology.orchestra.run.smithi032.stdout:5819 writesame 0x2d8ae1e thru 0x2d934ce data_size 0x1(0x86b1 bytes) 2024-09-21T15:01:12.490 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:01:12.485+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:01:12.501 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:01:12.501 INFO:teuthology.orchestra.run.smithi032.stdout:5821 read 0x17c16bb thru 0x17c8e39 (0x777f bytes) 2024-09-21T15:01:12.501 INFO:teuthology.orchestra.run.smithi032.stdout:5823 read 0xb92087c thru 0xb92762f (0x6db4 bytes) 2024-09-21T15:01:12.502 INFO:teuthology.orchestra.run.smithi032.stdout:5824 writesame 0x7094ec7 thru 0x7097872 data_size 0x2a(0x29ac bytes) 2024-09-21T15:01:12.511 INFO:teuthology.orchestra.run.smithi032.stdout:5825 punch from 0x10d2727 to 0x10d9bc4, (0x749d bytes) 2024-09-21T15:01:12.511 INFO:teuthology.orchestra.run.smithi032.stdout:5826 read 0x1e1a8fd thru 0x1e20575 (0x5c79 bytes) 2024-09-21T15:01:12.511 INFO:teuthology.orchestra.run.smithi032.stdout:5827 read 0x60d5a9e thru 0x60e3321 (0xd884 bytes) 2024-09-21T15:01:12.511 INFO:teuthology.orchestra.run.smithi032.stdout:5828 write 0x61955bc thru 0x619c1a3 (0x6be8 bytes) 2024-09-21T15:01:12.515 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:12.515 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:01:12.515 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T15:01:12.515 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:12.526 INFO:teuthology.orchestra.run.smithi032.stdout:5830 read 0x2ef4fde thru 0x2f0431c (0xf33f bytes) 2024-09-21T15:01:12.527 INFO:teuthology.orchestra.run.smithi032.stdout:5831 writesame 0x46b10be thru 0x46c0c15 data_size 0x54(0xfb58 bytes) 2024-09-21T15:01:12.528 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T15:01:12.542 INFO:teuthology.orchestra.run.smithi032.stdout:5833 read 0xbb4cb43 thru 0xbb55edc (0x939a bytes) 2024-09-21T15:01:12.542 INFO:teuthology.orchestra.run.smithi032.stdout:5834 read 0xacd4f5 thru 0xad6240 (0x8d4c bytes) 2024-09-21T15:01:12.543 INFO:teuthology.orchestra.run.smithi032.stdout:5835 punch from 0x9461592 to 0x946dd9e, (0xc80c bytes) 2024-09-21T15:01:12.543 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:01:12.553 INFO:teuthology.orchestra.run.smithi032.stdout:5837 trunc from 0xec0f29d to 0x8750ffb 2024-09-21T15:01:12.575 INFO:teuthology.orchestra.run.smithi032.stdout:5838 write 0x50beae8 thru 0x50c03c3 (0x18dc bytes) 2024-09-21T15:01:12.585 INFO:teuthology.orchestra.run.smithi032.stdout:5839 read 0x4bc7fb0 thru 0x4bd0431 (0x8482 bytes) 2024-09-21T15:01:12.585 INFO:teuthology.orchestra.run.smithi032.stdout:5840 punch from 0x76c3702 to 0x76cfa7d, (0xc37b bytes) 2024-09-21T15:01:12.585 INFO:teuthology.orchestra.run.smithi032.stdout:5841 writesame 0x2044031 thru 0x204e999 data_size 0x1(0xa969 bytes) 2024-09-21T15:01:12.591 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:01:12.589+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:01:12.611 INFO:teuthology.orchestra.run.smithi032.stdout:5842 read 0x28fbc0d thru 0x29035c0 (0x79b4 bytes) 2024-09-21T15:01:12.611 INFO:teuthology.orchestra.run.smithi032.stdout:5843 read 0x367ce79 thru 0x36841dc (0x7364 bytes) 2024-09-21T15:01:12.611 INFO:teuthology.orchestra.run.smithi032.stdout:5845 write 0x3be4fa thru 0x3c2cb5 (0x47bc bytes) 2024-09-21T15:01:12.615 INFO:teuthology.orchestra.run.smithi032.stdout:5846 punch from 0x42768ab to 0x42769c0, (0x115 bytes) 2024-09-21T15:01:12.632 INFO:teuthology.orchestra.run.smithi032.stdout:5847 write 0xde83ac3 thru 0xde8de09 (0xa347 bytes) 2024-09-21T15:01:12.652 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:01:12.652 INFO:teuthology.orchestra.run.smithi032.stdout:5849 writesame 0xaa173ec thru 0xaa217ec data_size 0x2d(0xa401 bytes) 2024-09-21T15:01:12.662 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:12.662 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:01:12.662 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T15:01:12.662 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T15:01:12.662 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:12.666 INFO:teuthology.orchestra.run.smithi032.stdout:5850 read 0xa85013b thru 0xa85af20 (0xade6 bytes) 2024-09-21T15:01:12.667 INFO:teuthology.orchestra.run.smithi032.stdout:5853 punch from 0x5350850 to 0x535e603, (0xddb3 bytes) 2024-09-21T15:01:12.667 INFO:teuthology.orchestra.run.smithi032.stdout:5856 punch from 0x36f0a11 to 0x36fa179, (0x9768 bytes) 2024-09-21T15:01:12.667 INFO:teuthology.orchestra.run.smithi032.stdout:5857 read 0xa1954ae thru 0xa1a4070 (0xebc3 bytes) 2024-09-21T15:01:12.668 INFO:teuthology.orchestra.run.smithi032.stdout:5859 read 0x4f1cc15 thru 0x4f2b638 (0xea24 bytes) 2024-09-21T15:01:12.668 INFO:teuthology.orchestra.run.smithi032.stdout:5860 writesame 0x72202f0 thru 0x7227f45 data_size 0xf(0x7c56 bytes) 2024-09-21T15:01:12.674 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T15:01:12.678 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:01:12.678 INFO:teuthology.orchestra.run.smithi032.stdout:5862 read 0xc430357 thru 0xc433659 (0x3303 bytes) 2024-09-21T15:01:12.678 INFO:teuthology.orchestra.run.smithi032.stdout:5863 read 0x9537985 thru 0x9539473 (0x1aef bytes) 2024-09-21T15:01:12.678 INFO:teuthology.orchestra.run.smithi032.stdout:5864 writesame 0x1fe57d2 thru 0x1fed247 data_size 0x4b(0x7a76 bytes) 2024-09-21T15:01:12.682 INFO:teuthology.orchestra.run.smithi032.stdout:5865 write 0x187fb10 thru 0x188ad58 (0xb249 bytes) 2024-09-21T15:01:12.692 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:01:12.689+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:01:12.699 INFO:teuthology.orchestra.run.smithi032.stdout:5866 write 0x98b219 thru 0x98e772 (0x355a bytes) 2024-09-21T15:01:12.717 INFO:teuthology.orchestra.run.smithi032.stdout:5867 trunc from 0xde8de0a to 0x4c65ccd 2024-09-21T15:01:12.743 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:01:12.743 INFO:teuthology.orchestra.run.smithi032.stdout:5869 write 0x899597 thru 0x8a0ff3 (0x7a5d bytes) 2024-09-21T15:01:12.746 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:01:12.752 INFO:teuthology.orchestra.run.smithi032.stdout:5871 trunc from 0x4c65ccd to 0x80d2161 2024-09-21T15:01:12.763 INFO:teuthology.orchestra.run.smithi032.stdout:5872 read 0x5c2567b thru 0x5c271c0 (0x1b46 bytes) 2024-09-21T15:01:12.763 INFO:teuthology.orchestra.run.smithi032.stdout:5873 punch from 0x1b0caad to 0x1b12473, (0x59c6 bytes) 2024-09-21T15:01:12.763 INFO:teuthology.orchestra.run.smithi032.stdout:5876 read 0x5e56ba9 thru 0x5e5b86d (0x4cc5 bytes) 2024-09-21T15:01:12.764 INFO:teuthology.orchestra.run.smithi032.stdout:5877 read 0x240a728 thru 0x2414461 (0x9d3a bytes) 2024-09-21T15:01:12.764 INFO:teuthology.orchestra.run.smithi032.stdout:5878 punch from 0x4a0cdc8 to 0x4a0d9a8, (0xbe0 bytes) 2024-09-21T15:01:12.764 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:01:12.764 INFO:teuthology.orchestra.run.smithi032.stdout:5881 read 0x4389f03 thru 0x439909e (0xf19c bytes) 2024-09-21T15:01:12.764 INFO:teuthology.orchestra.run.smithi032.stdout:5882 writesame 0x11255e7 thru 0x1134cee data_size 0xaa(0xf708 bytes) 2024-09-21T15:01:12.768 INFO:teuthology.orchestra.run.smithi032.stdout:5884 punch from 0x1ec8956 to 0x1ed6142, (0xd7ec bytes) 2024-09-21T15:01:12.768 INFO:teuthology.orchestra.run.smithi032.stdout:5885 read 0x10e6995 thru 0x10f1e65 (0xb4d1 bytes) 2024-09-21T15:01:12.769 INFO:teuthology.orchestra.run.smithi032.stdout:5886 read 0x2ce749e thru 0x2cf574f (0xe2b2 bytes) 2024-09-21T15:01:12.772 INFO:teuthology.orchestra.run.smithi032.stdout:5888 trunc from 0x80d2161 to 0x6a4cb61 2024-09-21T15:01:12.784 INFO:teuthology.orchestra.run.smithi032.stdout:5889 write 0x57889ad thru 0x5797b55 (0xf1a9 bytes) 2024-09-21T15:01:12.792 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:01:12.786+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:01:12.801 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:12.801 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T15:01:12.802 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T15:01:12.802 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T15:01:12.802 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.802 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:97677 3.15 3:a8e72133:::rbd_data.1136bfc79a70.0000000000000005:head [write 0~4440027 [fadvise_sequential+fadvise_nocache],write 4751360~147456 [fadvise_sequential+fadvise_nocache],write 4964352~147456 [fadvise_sequential+fadvise_nocache],write 5177344~65536 [fadvise_sequential+fadvise_nocache],write 5259264~147456 [fadvise_sequential+fadvise_nocache],write 5472256~147456 [fadvise_sequential+fadvise_nocache],write 5685248~229376 [fadvise_sequential+fadvise_nocache],write 5980160~147456 [fadvise_sequential+fadvise_nocache],write 6193152~229376 [fadvise_sequential+fadvise_nocache],write 6488064~147456 [fadvise_sequential+fadvise_nocache],write 6701056~229376 [fadvise_sequential+fadvise_nocache],write 6995968~147456 [fadvise_sequential+fadvise_nocache],write 7208960~229376 [fadvise_sequential+fadvise_nocache],write 7503872~147456 [fadvise_sequential+fadvise_nocache],write 7716864~229376 [fadvise_sequential+fadvise_nocache],write 8044544~114688 [fadvise_sequential+fadvise_nocache],write 8257536~114688 [fadvise_sequential+fadvise_nocache],write 8388608~65536 [fadvise_sequential+fadvise_nocache],write 8552448~114688 [fadvise_sequential+fadvise_nocache],write 8765440~114688 [fadvise_sequential+fadvise_nocache],write 8929280~32768 [fadvise_sequential+fadvise_nocache],write 9142272~32768 [fadvise_sequential+fadvise_nocache],write 9355264~32768 [fadvise_sequential+fadvise_nocache],write 9437184~32768 [fadvise_sequential+fadvise_nocache],write 9650176~32768 [fadvise_sequential+fadvise_nocache],write 9863168~32768 [fadvise_sequential+fadvise_nocache],write 9961472~131072 [fadvise_sequential+fadvise_nocache],write 10108928~196608 [fadvise_sequential+fadvise_nocache],write 10321920~278528 [fadvise_sequential+fadvise_nocache],write 10616832~196608 [fadvise_sequential+fadvise_nocache],write 10829824~278528 [fadvise_sequential+fadvise_nocache],write 11124736~196608 [fadvise_sequential+fadvise_nocache],write 11337728~196608 [fadvise_sequential+fadvise_nocache],write 11550720~65536 [fadvise_sequential+fadvise_nocache],write 11649024~98304 [fadvise_sequential+fadvise_nocache],write 11763712~65536 [fadvise_sequential+fadvise_nocache],write 11862016~98304 [fadvise_sequential+fadvise_nocache],write 11976704~65536 [fadvise_sequential+fadvise_nocache],write 12058624~65536 [fadvise_sequential+fadvise_nocache],write 12156928~98304 [fadvise_sequential+fadvise_nocache],write 12271616~65536 [fadvise_sequential+fadvise_nocache],write 12369920~98304 [fadvise_sequential+fadvise_nocache],write 12484608~65536 [fadvise_sequential+fadvise_nocache],write 12664832~98304 [fadvise_sequential+fadvise_nocache],write 12877824~98304 [fadvise_sequential+fadvise_nocache],write 13090816~16384 [fadvise_sequential+fadvise_nocache],write 13172736~98304 [fadvise_sequential+fadvise_nocache],write 13385728~98304 [fadvise_sequential+fadvise_nocache],write 13598720~32768 [fadvise_sequential+fadvise_nocache],write 13680640~98304 [fadvise_sequential+fadvise_nocache],write 13893632~98304 [fadvise_sequential+fadvise_nocache],write 14106624~49152 [fadvise_sequential+fadvise_nocache],write 14794752~81920 [fadvise_sequential+fadvise_nocache],write 15007744~81920 [fadvise_sequential+fadvise_nocache],write 15302656~81920 [fadvise_sequential+fadvise_nocache],write 15515648~81920 [fadvise_sequential+fadvise_nocache],write 15810560~81920 [fadvise_sequential+fadvise_nocache],write 16023552~81920 [fadvise_sequential+fadvise_nocache],write 16236544~16384 [fadvise_sequential+fadvise_nocache],write 16318464~81920 [fadvise_sequential+fadvise_nocache],write 16531456~81920 [fadvise_sequential+fadvise_nocache],write 16744448~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:01:12.802 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.302946+0000", 2024-09-21T15:01:12.802 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 67.497168646000006, 2024-09-21T15:01:12.802 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.091231204999999996, 2024-09-21T15:01:12.802 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:12.802 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:12.802 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:12.802 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:12.802 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:12.802 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 97677 2024-09-21T15:01:12.803 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.803 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:12.803 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.803 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:12.803 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.302946+0000", 2024-09-21T15:01:12.803 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:12.803 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.803 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.803 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:12.803 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.302946+0000", 2024-09-21T15:01:12.803 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:12.803 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.803 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.803 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:12.804 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.302942+0000", 2024-09-21T15:01:12.804 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999962 2024-09-21T15:01:12.804 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.804 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.804 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:12.804 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311253+0000", 2024-09-21T15:01:12.804 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0083105360000000003 2024-09-21T15:01:12.804 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.804 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.804 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:12.804 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311283+0000", 2024-09-21T15:01:12.804 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0065000000000001e-05 2024-09-21T15:01:12.804 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.805 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.805 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:12.805 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311296+0000", 2024-09-21T15:01:12.805 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3206e-05 2024-09-21T15:01:12.805 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.805 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.805 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:12.805 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311404+0000", 2024-09-21T15:01:12.805 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000107514 2024-09-21T15:01:12.805 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.805 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.805 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:12.805 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311795+0000", 2024-09-21T15:01:12.805 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000390913 2024-09-21T15:01:12.805 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.806 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.806 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 6", 2024-09-21T15:01:12.806 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.314686+0000", 2024-09-21T15:01:12.806 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0028915659999999999 2024-09-21T15:01:12.806 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.806 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.806 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:01:12.806 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.368178+0000", 2024-09-21T15:01:12.806 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.053491706 2024-09-21T15:01:12.806 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.806 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.806 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:01:12.806 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.393925+0000", 2024-09-21T15:01:12.807 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025747289 2024-09-21T15:01:12.807 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.807 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.807 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:01:12.807 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.394127+0000", 2024-09-21T15:01:12.807 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000201444 2024-09-21T15:01:12.807 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.807 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.807 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:12.807 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.394177+0000", 2024-09-21T15:01:12.807 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0689000000000002e-05 2024-09-21T15:01:12.807 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:12.807 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:12.807 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:12.807 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.808 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.808 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:97673 3.1d e122/24 3:bd91946e:::rbd_data.1136bfc79a70.0000000000000002:head v 122'162, mlcod=120'160)", 2024-09-21T15:01:12.808 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.313730+0000", 2024-09-21T15:01:12.808 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 67.486384321000003, 2024-09-21T15:01:12.808 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055373462999999998, 2024-09-21T15:01:12.808 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:12.808 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:12.808 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:12.808 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.808 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:12.808 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.313730+0000", 2024-09-21T15:01:12.808 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:12.808 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.809 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.809 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:12.809 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.313730+0000", 2024-09-21T15:01:12.809 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:12.809 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.809 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.809 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:12.809 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.313730+0000", 2024-09-21T15:01:12.809 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:12.809 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.809 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.809 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:12.809 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.336113+0000", 2024-09-21T15:01:12.809 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022382962999999999 2024-09-21T15:01:12.809 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.810 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.810 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:12.810 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.336140+0000", 2024-09-21T15:01:12.810 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7248999999999999e-05 2024-09-21T15:01:12.810 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.810 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.810 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:12.810 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.336151+0000", 2024-09-21T15:01:12.810 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1015e-05 2024-09-21T15:01:12.810 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.810 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.810 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:12.810 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.355428+0000", 2024-09-21T15:01:12.811 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019276379999999999 2024-09-21T15:01:12.811 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.811 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.811 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:12.811 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.355484+0000", 2024-09-21T15:01:12.811 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6820999999999997e-05 2024-09-21T15:01:12.811 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.811 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.811 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:01:12.811 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.369040+0000", 2024-09-21T15:01:12.811 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013555466 2024-09-21T15:01:12.811 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.811 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.811 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:12.812 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.369104+0000", 2024-09-21T15:01:12.812 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3986999999999996e-05 2024-09-21T15:01:12.812 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:12.812 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:12.812 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:12.812 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.812 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.812 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139890 3.e 3:71dba096:::rbd_data.113664780c85.000000000000018b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:12.812 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.226444+0000", 2024-09-21T15:01:12.812 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.573670678999999, 2024-09-21T15:01:12.812 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.028209567, 2024-09-21T15:01:12.812 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:12.812 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:12.812 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:12.812 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:12.812 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:12.813 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139890 2024-09-21T15:01:12.813 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.813 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:12.813 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.813 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:12.813 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226444+0000", 2024-09-21T15:01:12.813 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:12.813 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.813 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.813 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:12.813 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226444+0000", 2024-09-21T15:01:12.813 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:12.813 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.813 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.814 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:12.814 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226443+0000", 2024-09-21T15:01:12.814 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:12.814 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.814 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.814 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:12.814 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226448+0000", 2024-09-21T15:01:12.814 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5499999999999996e-06 2024-09-21T15:01:12.814 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.814 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.814 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:12.814 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226456+0000", 2024-09-21T15:01:12.814 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8679999999999999e-06 2024-09-21T15:01:12.814 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.815 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.815 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:12.815 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226462+0000", 2024-09-21T15:01:12.815 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1800000000000001e-06 2024-09-21T15:01:12.815 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.815 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.815 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:12.815 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226545+0000", 2024-09-21T15:01:12.815 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2527000000000005e-05 2024-09-21T15:01:12.815 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.815 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.815 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:12.815 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226553+0000", 2024-09-21T15:01:12.815 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8019999999999998e-06 2024-09-21T15:01:12.816 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.816 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.816 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:12.816 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.254491+0000", 2024-09-21T15:01:12.816 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.02793785 2024-09-21T15:01:12.816 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.816 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.816 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:12.816 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.254654+0000", 2024-09-21T15:01:12.816 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00016245800000000001 2024-09-21T15:01:12.816 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:12.816 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:12.816 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:12.816 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.817 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.817 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139897 3.e 3:76ddeb1d:::rbd_data.113664780c85.0000000000000004:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:12.817 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.227198+0000", 2024-09-21T15:01:12.817 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.572917073999999, 2024-09-21T15:01:12.817 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.027799822, 2024-09-21T15:01:12.817 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:12.817 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:12.817 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:12.817 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:12.817 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:12.817 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139897 2024-09-21T15:01:12.817 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.817 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:12.817 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.818 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:12.818 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227198+0000", 2024-09-21T15:01:12.818 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:12.818 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.818 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.818 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:12.818 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227198+0000", 2024-09-21T15:01:12.818 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:12.818 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.818 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.818 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:12.818 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227197+0000", 2024-09-21T15:01:12.818 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:12.818 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.819 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.819 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:12.819 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227203+0000", 2024-09-21T15:01:12.819 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4819999999999999e-06 2024-09-21T15:01:12.819 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.819 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.819 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:12.819 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227220+0000", 2024-09-21T15:01:12.819 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6671e-05 2024-09-21T15:01:12.819 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.819 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.819 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:12.819 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227229+0000", 2024-09-21T15:01:12.819 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0499999999999997e-06 2024-09-21T15:01:12.819 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.820 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.820 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:12.820 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227293+0000", 2024-09-21T15:01:12.820 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4434e-05 2024-09-21T15:01:12.820 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.820 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.820 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:12.820 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227302+0000", 2024-09-21T15:01:12.820 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0929999999999992e-06 2024-09-21T15:01:12.820 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.820 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.820 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:12.820 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.254720+0000", 2024-09-21T15:01:12.820 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0274177550000001 2024-09-21T15:01:12.820 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.821 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.821 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:12.821 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.254854+0000", 2024-09-21T15:01:12.821 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000133416 2024-09-21T15:01:12.821 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.821 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.821 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:12.821 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.254998+0000", 2024-09-21T15:01:12.821 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014400099999999999 2024-09-21T15:01:12.821 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:12.821 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:12.821 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:12.821 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.821 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.822 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139902 3.e 3:76b9b799:::rbd_data.113664780c85.0000000000000021:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:12.822 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.227319+0000", 2024-09-21T15:01:12.822 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.572795693, 2024-09-21T15:01:12.822 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0279584639999999, 2024-09-21T15:01:12.822 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:12.822 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:12.822 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:12.822 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:12.822 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:12.822 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139902 2024-09-21T15:01:12.822 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.822 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:12.822 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.822 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:12.822 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227319+0000", 2024-09-21T15:01:12.823 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:12.823 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.823 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.823 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:12.823 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227319+0000", 2024-09-21T15:01:12.823 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:12.823 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.823 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.823 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:12.823 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227319+0000", 2024-09-21T15:01:12.823 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:12.823 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.823 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.823 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:12.824 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227322+0000", 2024-09-21T15:01:12.824 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1980000000000001e-06 2024-09-21T15:01:12.824 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.824 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.824 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:12.824 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227328+0000", 2024-09-21T15:01:12.824 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4230000000000002e-06 2024-09-21T15:01:12.824 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.824 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.824 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:12.824 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227333+0000", 2024-09-21T15:01:12.824 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0560000000000004e-06 2024-09-21T15:01:12.824 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.824 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.824 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:12.825 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227729+0000", 2024-09-21T15:01:12.825 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00039591799999999999 2024-09-21T15:01:12.825 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.825 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.825 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:12.825 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227738+0000", 2024-09-21T15:01:12.825 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8119999999999997e-06 2024-09-21T15:01:12.825 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.825 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.825 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:12.825 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255021+0000", 2024-09-21T15:01:12.825 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0272826779999999 2024-09-21T15:01:12.825 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.825 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.825 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:12.825 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255147+0000", 2024-09-21T15:01:12.826 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012595600000000001 2024-09-21T15:01:12.826 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.826 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.826 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:12.826 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255278+0000", 2024-09-21T15:01:12.826 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013081400000000001 2024-09-21T15:01:12.826 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:12.826 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:12.826 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:12.826 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.826 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.826 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139945 3.e 3:7756a55d:::rbd_data.113664780c85.0000000000000154:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:12.826 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.229015+0000", 2024-09-21T15:01:12.826 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.571099889999999, 2024-09-21T15:01:12.827 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.026468825, 2024-09-21T15:01:12.827 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:12.827 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:12.827 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:12.827 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:12.827 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:12.827 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139945 2024-09-21T15:01:12.827 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.827 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:12.827 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.827 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:12.827 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229015+0000", 2024-09-21T15:01:12.827 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:12.827 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.827 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.828 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:12.828 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229015+0000", 2024-09-21T15:01:12.828 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:12.828 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.828 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.828 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:12.828 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229014+0000", 2024-09-21T15:01:12.828 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:12.828 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.828 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.828 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:12.828 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229017+0000", 2024-09-21T15:01:12.828 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7310000000000002e-06 2024-09-21T15:01:12.828 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.828 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.829 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:12.829 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229023+0000", 2024-09-21T15:01:12.829 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8150000000000002e-06 2024-09-21T15:01:12.829 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.829 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.829 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:12.829 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229028+0000", 2024-09-21T15:01:12.829 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7650000000000001e-06 2024-09-21T15:01:12.829 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.829 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.829 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:12.829 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229082+0000", 2024-09-21T15:01:12.829 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4521999999999997e-05 2024-09-21T15:01:12.829 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.829 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.830 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:12.830 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229092+0000", 2024-09-21T15:01:12.830 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.3940000000000002e-06 2024-09-21T15:01:12.830 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.830 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.830 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:12.830 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255382+0000", 2024-09-21T15:01:12.830 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0262900930000001 2024-09-21T15:01:12.830 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.830 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.830 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:12.830 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255484+0000", 2024-09-21T15:01:12.830 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000101932 2024-09-21T15:01:12.830 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:12.831 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:12.831 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:12.831 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.831 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.831 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139967 3.e 3:74e90d85:::rbd_data.113664780c85.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:12.831 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.229124+0000", 2024-09-21T15:01:12.831 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.570990950000001, 2024-09-21T15:01:12.831 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.026724381, 2024-09-21T15:01:12.831 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:12.831 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:12.831 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:12.831 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:12.831 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:12.831 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139967 2024-09-21T15:01:12.831 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.832 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:12.832 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.832 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:12.832 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229124+0000", 2024-09-21T15:01:12.832 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:12.832 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.832 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.832 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:12.832 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229124+0000", 2024-09-21T15:01:12.832 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:12.832 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.832 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.832 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:12.832 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229123+0000", 2024-09-21T15:01:12.833 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:12.833 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.833 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.833 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:12.833 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229126+0000", 2024-09-21T15:01:12.833 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4779999999999998e-06 2024-09-21T15:01:12.833 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.833 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.833 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:12.833 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229131+0000", 2024-09-21T15:01:12.833 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.729e-06 2024-09-21T15:01:12.833 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.833 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.833 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:12.833 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229135+0000", 2024-09-21T15:01:12.834 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5859999999999998e-06 2024-09-21T15:01:12.834 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.834 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.834 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:12.834 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229558+0000", 2024-09-21T15:01:12.834 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00042262499999999999 2024-09-21T15:01:12.834 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.834 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.834 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:12.834 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229569+0000", 2024-09-21T15:01:12.834 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0985000000000001e-05 2024-09-21T15:01:12.834 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.834 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.834 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:12.834 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255509+0000", 2024-09-21T15:01:12.835 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.025940246 2024-09-21T15:01:12.835 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.835 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.835 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:12.835 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255662+0000", 2024-09-21T15:01:12.835 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015324199999999999 2024-09-21T15:01:12.835 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.835 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.835 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:12.835 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255848+0000", 2024-09-21T15:01:12.835 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00018583599999999999 2024-09-21T15:01:12.835 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:12.835 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:12.835 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:12.835 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.836 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.836 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140010 3.e 3:735c06f2:::rbd_data.113664780c85.0000000000000157:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:12.836 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.231531+0000", 2024-09-21T15:01:12.836 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.568584084999999, 2024-09-21T15:01:12.836 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0244773149999999, 2024-09-21T15:01:12.836 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:12.836 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:12.836 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:12.836 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:12.836 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:12.836 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140010 2024-09-21T15:01:12.836 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.836 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:12.836 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.836 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:12.837 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231531+0000", 2024-09-21T15:01:12.837 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:12.837 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.837 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.837 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:12.837 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231531+0000", 2024-09-21T15:01:12.837 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:12.837 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.837 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.837 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:12.837 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231530+0000", 2024-09-21T15:01:12.837 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:12.837 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.837 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.837 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:12.838 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231535+0000", 2024-09-21T15:01:12.838 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3970000000000002e-06 2024-09-21T15:01:12.838 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.838 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.838 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:12.838 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231544+0000", 2024-09-21T15:01:12.838 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5909999999999996e-06 2024-09-21T15:01:12.838 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.838 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.838 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:12.838 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231551+0000", 2024-09-21T15:01:12.838 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7129999999999999e-06 2024-09-21T15:01:12.838 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.838 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.838 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:12.839 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231632+0000", 2024-09-21T15:01:12.839 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1181999999999996e-05 2024-09-21T15:01:12.839 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.839 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.839 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:12.839 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231640+0000", 2024-09-21T15:01:12.839 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.551e-06 2024-09-21T15:01:12.839 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.839 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.839 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:12.839 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255884+0000", 2024-09-21T15:01:12.839 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.024244057 2024-09-21T15:01:12.839 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.839 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.839 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:12.840 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256008+0000", 2024-09-21T15:01:12.840 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012351699999999999 2024-09-21T15:01:12.840 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:12.840 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:12.840 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:12.840 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.840 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.840 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140043 3.e 3:74a37859:::rbd_data.113664780c85.0000000000000086:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:12.840 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.232623+0000", 2024-09-21T15:01:12.840 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.567491610999999, 2024-09-21T15:01:12.840 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.023518634, 2024-09-21T15:01:12.840 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:12.840 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:12.840 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:12.841 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:12.841 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:12.841 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140043 2024-09-21T15:01:12.841 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.841 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:12.841 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.841 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:12.841 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.232623+0000", 2024-09-21T15:01:12.841 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:12.841 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.841 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.841 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:12.841 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.232623+0000", 2024-09-21T15:01:12.841 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:12.842 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.842 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.842 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:12.842 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.232623+0000", 2024-09-21T15:01:12.842 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:12.842 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.842 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.842 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:12.842 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.232626+0000", 2024-09-21T15:01:12.842 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0019999999999998e-06 2024-09-21T15:01:12.842 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.842 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.842 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:12.843 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.232632+0000", 2024-09-21T15:01:12.843 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3260000000000001e-06 2024-09-21T15:01:12.843 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.843 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.843 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:12.843 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.232637+0000", 2024-09-21T15:01:12.843 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9899999999999997e-06 2024-09-21T15:01:12.843 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.843 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.843 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:12.843 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.232678+0000", 2024-09-21T15:01:12.843 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.0423999999999998e-05 2024-09-21T15:01:12.843 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.843 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.844 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:12.844 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.232686+0000", 2024-09-21T15:01:12.844 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1419999999999993e-06 2024-09-21T15:01:12.844 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.844 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.844 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:12.844 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256037+0000", 2024-09-21T15:01:12.844 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.023351736 2024-09-21T15:01:12.844 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.844 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.844 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:12.844 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256142+0000", 2024-09-21T15:01:12.844 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010443 2024-09-21T15:01:12.844 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:12.844 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:12.845 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:12.845 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.845 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.845 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140063 3.e 3:771cae23:::rbd_data.113664780c85.0000000000000112:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:12.845 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.233948+0000", 2024-09-21T15:01:12.845 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.566166544, 2024-09-21T15:01:12.845 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0223114710000001, 2024-09-21T15:01:12.845 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:12.845 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:12.845 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:12.845 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:12.845 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:12.845 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140063 2024-09-21T15:01:12.846 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.846 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:12.846 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.846 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:12.846 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233948+0000", 2024-09-21T15:01:12.846 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:12.846 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.846 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.846 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:12.846 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233948+0000", 2024-09-21T15:01:12.846 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:12.846 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.846 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.846 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:12.847 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233948+0000", 2024-09-21T15:01:12.847 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:12.847 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.847 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.847 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:12.847 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233951+0000", 2024-09-21T15:01:12.847 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3380000000000001e-06 2024-09-21T15:01:12.847 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.847 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.847 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:12.847 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233958+0000", 2024-09-21T15:01:12.847 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0990000000000001e-06 2024-09-21T15:01:12.847 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.847 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.848 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:12.848 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233964+0000", 2024-09-21T15:01:12.848 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.852e-06 2024-09-21T15:01:12.848 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.848 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.848 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:12.848 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.234056+0000", 2024-09-21T15:01:12.848 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2392000000000006e-05 2024-09-21T15:01:12.848 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.848 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.848 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:12.848 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.234066+0000", 2024-09-21T15:01:12.848 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1360000000000003e-06 2024-09-21T15:01:12.849 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.849 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.849 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:12.849 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256169+0000", 2024-09-21T15:01:12.849 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0221031949999999 2024-09-21T15:01:12.849 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.849 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.849 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:12.849 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256260+0000", 2024-09-21T15:01:12.849 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0939000000000005e-05 2024-09-21T15:01:12.849 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:12.849 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:12.849 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:12.849 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.850 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.850 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140086 3.e 3:758134b7:::rbd_data.113664780c85.00000000000001ad:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:12.850 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.235110+0000", 2024-09-21T15:01:12.850 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.565004845000001, 2024-09-21T15:01:12.850 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.021276217, 2024-09-21T15:01:12.850 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:12.850 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:12.850 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:12.850 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:12.850 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:12.850 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140086 2024-09-21T15:01:12.850 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.850 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:12.851 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.851 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:12.851 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.235110+0000", 2024-09-21T15:01:12.851 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:12.851 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.851 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.851 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:12.851 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.235110+0000", 2024-09-21T15:01:12.851 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:12.851 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.851 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.851 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:12.851 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.235109+0000", 2024-09-21T15:01:12.851 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:12.851 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.852 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.852 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:12.852 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.235114+0000", 2024-09-21T15:01:12.852 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1659999999999996e-06 2024-09-21T15:01:12.852 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.852 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.852 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:12.852 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.235122+0000", 2024-09-21T15:01:12.852 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5590000000000006e-06 2024-09-21T15:01:12.852 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.852 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.852 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:12.852 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.235128+0000", 2024-09-21T15:01:12.852 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8579999999999996e-06 2024-09-21T15:01:12.852 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.852 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.853 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:12.853 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.235169+0000", 2024-09-21T15:01:12.853 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1304000000000003e-05 2024-09-21T15:01:12.853 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.853 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.853 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:12.853 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.235178+0000", 2024-09-21T15:01:12.853 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8610000000000004e-06 2024-09-21T15:01:12.853 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.853 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.853 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:12.853 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256286+0000", 2024-09-21T15:01:12.853 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.021108162 2024-09-21T15:01:12.853 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.854 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.854 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:12.854 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256386+0000", 2024-09-21T15:01:12.854 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9871000000000007e-05 2024-09-21T15:01:12.854 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:12.854 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:12.854 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:12.854 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.854 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.854 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140168 3.e 3:76f4e81d:::rbd_data.113664780c85.000000000000006f:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:12.854 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.236992+0000", 2024-09-21T15:01:12.854 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.563122782000001, 2024-09-21T15:01:12.854 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.019513699, 2024-09-21T15:01:12.854 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:12.854 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:12.854 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:12.855 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:12.855 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:12.855 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140168 2024-09-21T15:01:12.855 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.855 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:12.855 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.855 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:12.855 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.236992+0000", 2024-09-21T15:01:12.855 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:12.855 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.855 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.855 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:12.855 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.236992+0000", 2024-09-21T15:01:12.855 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:12.855 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.856 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.856 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:12.856 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.236991+0000", 2024-09-21T15:01:12.856 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:12.856 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.856 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.856 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:12.856 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.236995+0000", 2024-09-21T15:01:12.856 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3100000000000001e-06 2024-09-21T15:01:12.856 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.856 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.856 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:12.856 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237002+0000", 2024-09-21T15:01:12.856 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9929999999999998e-06 2024-09-21T15:01:12.857 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.857 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.857 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:12.857 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237008+0000", 2024-09-21T15:01:12.857 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7559999999999996e-06 2024-09-21T15:01:12.857 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.857 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.857 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:12.857 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237050+0000", 2024-09-21T15:01:12.857 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2852999999999998e-05 2024-09-21T15:01:12.857 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.857 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.857 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:12.857 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237060+0000", 2024-09-21T15:01:12.858 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2399999999999996e-06 2024-09-21T15:01:12.858 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.858 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.858 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:12.858 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256413+0000", 2024-09-21T15:01:12.858 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0193528999999999 2024-09-21T15:01:12.858 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.858 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.858 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:12.858 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256506+0000", 2024-09-21T15:01:12.858 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.3154999999999998e-05 2024-09-21T15:01:12.858 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:12.858 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:12.858 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:12.859 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.859 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.859 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140186 3.e 3:75504345:::rbd_data.113664780c85.00000000000000ed:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:12.859 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.237050+0000", 2024-09-21T15:01:12.859 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.563064811, 2024-09-21T15:01:12.859 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0195720150000001, 2024-09-21T15:01:12.859 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:12.859 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:12.859 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:12.859 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:12.859 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:12.859 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140186 2024-09-21T15:01:12.859 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.859 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:12.860 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.860 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:12.860 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237050+0000", 2024-09-21T15:01:12.860 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:12.860 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.860 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.860 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:12.860 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237050+0000", 2024-09-21T15:01:12.860 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:12.860 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.860 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.860 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:12.860 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237049+0000", 2024-09-21T15:01:12.860 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:12.860 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.861 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.861 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:12.861 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237052+0000", 2024-09-21T15:01:12.861 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.016e-06 2024-09-21T15:01:12.861 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.861 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.861 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:12.861 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237070+0000", 2024-09-21T15:01:12.861 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7266999999999999e-05 2024-09-21T15:01:12.861 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.861 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.861 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:12.861 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237075+0000", 2024-09-21T15:01:12.861 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6910000000000002e-06 2024-09-21T15:01:12.861 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.862 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.862 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:12.862 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237232+0000", 2024-09-21T15:01:12.862 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015683599999999999 2024-09-21T15:01:12.862 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.862 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.862 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:12.862 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237237+0000", 2024-09-21T15:01:12.862 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6439999999999999e-06 2024-09-21T15:01:12.862 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.862 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.862 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:12.862 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256534+0000", 2024-09-21T15:01:12.862 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.019296736 2024-09-21T15:01:12.863 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.863 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.863 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:12.863 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256622+0000", 2024-09-21T15:01:12.863 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8386000000000004e-05 2024-09-21T15:01:12.863 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:12.863 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:12.863 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:12.863 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.863 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.863 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140197 3.e 3:71ef5b14:::rbd_data.113664780c85.0000000000000140:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:12.863 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.237251+0000", 2024-09-21T15:01:12.863 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.562863782999999, 2024-09-21T15:01:12.864 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.01968386, 2024-09-21T15:01:12.864 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:12.864 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:12.864 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:12.864 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:12.864 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:12.864 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140197 2024-09-21T15:01:12.864 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.864 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:12.864 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.864 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:12.864 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237251+0000", 2024-09-21T15:01:12.864 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:12.864 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.865 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.865 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:12.865 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237251+0000", 2024-09-21T15:01:12.865 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:12.865 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.865 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.865 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:12.865 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237250+0000", 2024-09-21T15:01:12.865 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:12.865 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.865 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.865 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:12.865 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237254+0000", 2024-09-21T15:01:12.865 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.0180000000000003e-06 2024-09-21T15:01:12.866 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.866 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.866 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:12.866 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237262+0000", 2024-09-21T15:01:12.866 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0669999999999992e-06 2024-09-21T15:01:12.866 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.866 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.866 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:12.866 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237269+0000", 2024-09-21T15:01:12.866 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6000000000000003e-06 2024-09-21T15:01:12.866 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.866 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.866 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:12.866 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237319+0000", 2024-09-21T15:01:12.866 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9932000000000002e-05 2024-09-21T15:01:12.867 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.867 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.867 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:12.867 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237326+0000", 2024-09-21T15:01:12.867 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5360000000000004e-06 2024-09-21T15:01:12.867 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.867 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.867 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:12.867 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256657+0000", 2024-09-21T15:01:12.867 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.019330756 2024-09-21T15:01:12.867 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.867 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.867 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:12.867 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256935+0000", 2024-09-21T15:01:12.867 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00027762099999999998 2024-09-21T15:01:12.867 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:12.868 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:12.868 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:12.868 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.868 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.868 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140201 3.e 3:732ea575:::rbd_data.113664780c85.000000000000015c:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:12.868 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.237281+0000", 2024-09-21T15:01:12.868 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.562833945, 2024-09-21T15:01:12.868 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0198284529999999, 2024-09-21T15:01:12.868 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:12.868 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:12.868 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:12.868 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:12.868 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:12.868 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140201 2024-09-21T15:01:12.868 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.869 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:12.869 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.869 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:12.869 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237281+0000", 2024-09-21T15:01:12.869 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:12.869 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.869 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.869 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:12.869 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237281+0000", 2024-09-21T15:01:12.869 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:12.869 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.869 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.869 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:12.869 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237280+0000", 2024-09-21T15:01:12.869 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:12.870 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.870 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.870 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:12.870 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237284+0000", 2024-09-21T15:01:12.870 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7400000000000002e-06 2024-09-21T15:01:12.870 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.870 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.870 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:12.870 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237292+0000", 2024-09-21T15:01:12.870 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8939999999999993e-06 2024-09-21T15:01:12.870 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.870 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.870 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:12.870 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237298+0000", 2024-09-21T15:01:12.871 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8000000000000004e-06 2024-09-21T15:01:12.871 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.871 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.871 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:12.871 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237521+0000", 2024-09-21T15:01:12.871 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00022326 2024-09-21T15:01:12.871 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.871 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.871 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:12.871 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237529+0000", 2024-09-21T15:01:12.871 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6170000000000001e-06 2024-09-21T15:01:12.871 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.871 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.871 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:12.871 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256968+0000", 2024-09-21T15:01:12.872 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.019439113 2024-09-21T15:01:12.872 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.872 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.872 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:12.872 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.257109+0000", 2024-09-21T15:01:12.872 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014148399999999999 2024-09-21T15:01:12.872 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:12.872 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:12.872 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:12.872 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.872 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.872 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140206 3.e 3:76d38029:::rbd_data.113664780c85.0000000000000179:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:12.872 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.237520+0000", 2024-09-21T15:01:12.872 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.562594831, 2024-09-21T15:01:12.872 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0210658699999999, 2024-09-21T15:01:12.872 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:12.873 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:12.873 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:12.873 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:12.873 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:12.873 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140206 2024-09-21T15:01:12.873 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.873 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:12.873 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.873 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:12.873 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237520+0000", 2024-09-21T15:01:12.873 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:12.873 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.873 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.873 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:12.874 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237520+0000", 2024-09-21T15:01:12.874 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:12.874 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.874 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.874 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:12.874 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237519+0000", 2024-09-21T15:01:12.874 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:12.874 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.874 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.874 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:12.874 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237523+0000", 2024-09-21T15:01:12.874 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.9009999999999996e-06 2024-09-21T15:01:12.874 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.874 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.874 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:12.875 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237531+0000", 2024-09-21T15:01:12.875 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7069999999999992e-06 2024-09-21T15:01:12.875 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.875 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.875 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:12.875 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237537+0000", 2024-09-21T15:01:12.875 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4559999999999997e-06 2024-09-21T15:01:12.875 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.875 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.875 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:12.875 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237583+0000", 2024-09-21T15:01:12.875 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5532999999999999e-05 2024-09-21T15:01:12.875 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.875 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.876 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:12.876 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237592+0000", 2024-09-21T15:01:12.876 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6009999999999995e-06 2024-09-21T15:01:12.876 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.876 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.876 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:12.876 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.257157+0000", 2024-09-21T15:01:12.876 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0195657279999999 2024-09-21T15:01:12.876 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.876 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.876 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:12.876 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.258586+0000", 2024-09-21T15:01:12.876 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00142847 2024-09-21T15:01:12.876 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:12.876 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:12.877 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:12.877 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.877 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.877 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140338 3.e 3:71dba096:::rbd_data.113664780c85.000000000000018b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:12.877 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.243856+0000", 2024-09-21T15:01:12.877 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.556258998000001, 2024-09-21T15:01:12.877 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0138226889999999, 2024-09-21T15:01:12.877 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:12.877 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:12.877 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:12.877 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:12.877 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:12.877 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140338 2024-09-21T15:01:12.877 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.877 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:12.877 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.878 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:12.878 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243856+0000", 2024-09-21T15:01:12.878 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:12.878 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.878 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.878 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:12.878 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243856+0000", 2024-09-21T15:01:12.878 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:12.878 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.878 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.878 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:12.878 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243854+0000", 2024-09-21T15:01:12.878 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:12.878 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.878 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.879 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:12.879 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243862+0000", 2024-09-21T15:01:12.879 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2570000000000001e-06 2024-09-21T15:01:12.879 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.879 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.879 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:12.879 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243877+0000", 2024-09-21T15:01:12.879 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.571e-05 2024-09-21T15:01:12.879 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.879 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.879 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:12.879 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243884+0000", 2024-09-21T15:01:12.879 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.1269999999999997e-06 2024-09-21T15:01:12.879 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.880 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.880 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:12.880 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243932+0000", 2024-09-21T15:01:12.880 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7655999999999999e-05 2024-09-21T15:01:12.880 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.880 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.880 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:12.880 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243942+0000", 2024-09-21T15:01:12.880 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6679999999999996e-06 2024-09-21T15:01:12.880 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.880 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.880 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:12.880 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.257568+0000", 2024-09-21T15:01:12.880 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0136267670000001 2024-09-21T15:01:12.880 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.881 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.881 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:12.881 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.257678+0000", 2024-09-21T15:01:12.881 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000110011 2024-09-21T15:01:12.881 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:12.881 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:12.881 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:12.881 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.881 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.881 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140345 3.e 3:76ddeb1d:::rbd_data.113664780c85.0000000000000004:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:12.881 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.243927+0000", 2024-09-21T15:01:12.881 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.556187414, 2024-09-21T15:01:12.881 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0140328439999999, 2024-09-21T15:01:12.881 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:12.881 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:12.882 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:12.882 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:12.882 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:12.882 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140345 2024-09-21T15:01:12.882 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.882 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:12.882 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.882 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:12.882 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243927+0000", 2024-09-21T15:01:12.882 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:12.882 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.882 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.882 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:12.882 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243927+0000", 2024-09-21T15:01:12.883 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:12.883 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.883 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.883 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:12.883 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243927+0000", 2024-09-21T15:01:12.883 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:12.883 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.883 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.883 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:12.883 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243930+0000", 2024-09-21T15:01:12.883 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1319999999999998e-06 2024-09-21T15:01:12.883 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.883 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.884 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:12.884 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243937+0000", 2024-09-21T15:01:12.884 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4330000000000001e-06 2024-09-21T15:01:12.884 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.884 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.884 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:12.884 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243942+0000", 2024-09-21T15:01:12.884 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5090000000000001e-06 2024-09-21T15:01:12.884 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.884 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.884 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:12.884 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243964+0000", 2024-09-21T15:01:12.884 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1838999999999999e-05 2024-09-21T15:01:12.884 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.884 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.885 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:12.885 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243971+0000", 2024-09-21T15:01:12.885 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.131e-06 2024-09-21T15:01:12.885 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.885 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.885 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:12.885 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.257704+0000", 2024-09-21T15:01:12.885 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0137329310000001 2024-09-21T15:01:12.885 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.885 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.885 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:12.885 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.257805+0000", 2024-09-21T15:01:12.885 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000101121 2024-09-21T15:01:12.885 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.885 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.886 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:12.886 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.257960+0000", 2024-09-21T15:01:12.886 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015518500000000001 2024-09-21T15:01:12.886 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:12.886 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:12.886 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:12.886 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.886 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.886 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140350 3.e 3:76b9b799:::rbd_data.113664780c85.0000000000000021:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:12.886 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.243989+0000", 2024-09-21T15:01:12.886 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.556125663, 2024-09-21T15:01:12.886 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0142022340000001, 2024-09-21T15:01:12.886 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:12.886 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:12.886 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:12.887 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:12.887 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:12.887 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140350 2024-09-21T15:01:12.887 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.887 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:12.887 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.887 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:12.887 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243989+0000", 2024-09-21T15:01:12.887 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:12.887 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.887 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.887 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:12.887 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243989+0000", 2024-09-21T15:01:12.887 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:12.888 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.888 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.888 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:12.888 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243989+0000", 2024-09-21T15:01:12.888 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:12.888 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.888 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.888 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:12.888 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243992+0000", 2024-09-21T15:01:12.888 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.236e-06 2024-09-21T15:01:12.888 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.888 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.888 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:12.888 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243998+0000", 2024-09-21T15:01:12.888 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2500000000000003e-06 2024-09-21T15:01:12.888 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.889 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.889 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:12.889 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.244003+0000", 2024-09-21T15:01:12.889 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7740000000000004e-06 2024-09-21T15:01:12.889 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.889 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.889 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:12.889 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.244047+0000", 2024-09-21T15:01:12.889 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3665999999999998e-05 2024-09-21T15:01:12.889 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.889 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.889 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:12.889 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.244055+0000", 2024-09-21T15:01:12.889 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0639999999999994e-06 2024-09-21T15:01:12.889 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.890 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.890 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:12.890 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.257985+0000", 2024-09-21T15:01:12.890 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0139306079999999 2024-09-21T15:01:12.890 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.890 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.890 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:12.890 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.258084+0000", 2024-09-21T15:01:12.890 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8610000000000006e-05 2024-09-21T15:01:12.890 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.890 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.890 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:12.890 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.258191+0000", 2024-09-21T15:01:12.891 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000107518 2024-09-21T15:01:12.891 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:12.891 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:12.891 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:12.891 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.891 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.891 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140394 3.e 3:7756a55d:::rbd_data.113664780c85.0000000000000154:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:12.891 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.245038+0000", 2024-09-21T15:01:12.891 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 10.555076445999999, 2024-09-21T15:01:12.891 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.013225182, 2024-09-21T15:01:12.891 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:12.892 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:12.892 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:12.892 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:12.892 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:12.892 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140394 2024-09-21T15:01:12.892 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.892 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:12.892 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.892 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:12.892 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.245038+0000", 2024-09-21T15:01:12.892 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:12.892 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.892 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.892 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:12.892 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.245038+0000", 2024-09-21T15:01:12.893 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:12.893 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.893 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.893 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:12.893 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.245037+0000", 2024-09-21T15:01:12.893 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:12.893 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.893 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.893 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:12.893 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.245044+0000", 2024-09-21T15:01:12.893 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3239999999999999e-06 2024-09-21T15:01:12.893 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.893 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.893 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:12.894 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.245061+0000", 2024-09-21T15:01:12.894 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7166000000000001e-05 2024-09-21T15:01:12.894 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.894 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.894 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:12.894 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.245069+0000", 2024-09-21T15:01:12.894 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0779999999999996e-06 2024-09-21T15:01:12.894 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.894 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.894 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:12.894 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.245118+0000", 2024-09-21T15:01:12.894 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8887000000000001e-05 2024-09-21T15:01:12.894 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.895 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.895 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:12.895 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.245126+0000", 2024-09-21T15:01:12.895 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0409999999999998e-06 2024-09-21T15:01:12.895 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.895 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.895 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:12.895 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.258206+0000", 2024-09-21T15:01:12.895 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.013080451 2024-09-21T15:01:12.895 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:12.895 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:12.895 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:12.895 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.258264+0000", 2024-09-21T15:01:12.895 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7294999999999999e-05 2024-09-21T15:01:12.896 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:12.896 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:12.896 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:12.896 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:12.896 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:12.896 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:12.896 INFO:teuthology.orchestra.run.smithi032.stdout:5890 punch from 0x14b5db9 to 0x14c0f10, (0xb157 bytes) 2024-09-21T15:01:12.897 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T15:01:12.898 INFO:teuthology.orchestra.run.smithi032.stdout:5891 writesame 0xa1bd3f9 thru 0xa1ca4d7 data_size 0x1(0xd0df bytes) 2024-09-21T15:01:12.898 INFO:teuthology.orchestra.run.smithi032.stdout:5892 read 0x3beea55 thru 0x3bfcf1a (0xe4c6 bytes) 2024-09-21T15:01:12.898 INFO:teuthology.orchestra.run.smithi032.stdout:5893 writesame 0x143eca5 thru 0x1447570 data_size 0x67(0x88cc bytes) 2024-09-21T15:01:12.898 INFO:teuthology.orchestra.run.smithi032.stdout:5895 read 0x75952af thru 0x759981d (0x456f bytes) 2024-09-21T15:01:12.898 INFO:teuthology.orchestra.run.smithi032.stdout:5897 trunc from 0xa1ca4d8 to 0xaafaed8 2024-09-21T15:01:12.899 INFO:teuthology.orchestra.run.smithi032.stdout:5898 writesame 0x48d93c3 thru 0x48de5fd data_size 0x3(0x523b bytes) 2024-09-21T15:01:12.899 INFO:teuthology.orchestra.run.smithi032.stdout:5899 trunc from 0xaafaed8 to 0x26f5a9d 2024-09-21T15:01:12.900 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:01:12.897+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:01:12.900 INFO:teuthology.orchestra.run.smithi032.stdout:5900 write 0x6000e39 thru 0x600db0b (0xccd3 bytes) 2024-09-21T15:01:12.919 INFO:teuthology.orchestra.run.smithi032.stdout:5901 writesame 0x5267fab thru 0x5270137 data_size 0xf(0x818d bytes) 2024-09-21T15:01:12.950 INFO:teuthology.orchestra.run.smithi032.stdout:5902 write 0x8bba855 thru 0x8bc137e (0x6b2a bytes) 2024-09-21T15:01:12.967 INFO:teuthology.orchestra.run.smithi032.stdout:5903 write 0x65446ac thru 0x654ea93 (0xa3e8 bytes) 2024-09-21T15:01:12.996 INFO:teuthology.orchestra.run.smithi032.stdout:5904 trunc from 0x8bc137f to 0x88b0b7 2024-09-21T15:01:13.001 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:01:12.998+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:01:13.021 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T15:01:13.030 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:13.030 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:01:13.030 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T15:01:13.030 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:13.034 INFO:teuthology.orchestra.run.smithi032.stdout:5905 read 0x281697 thru 0x2890f1 (0x7a5b bytes) 2024-09-21T15:01:13.035 INFO:teuthology.orchestra.run.smithi032.stdout:5906 read 0x5f5061 thru 0x5f8634 (0x35d4 bytes) 2024-09-21T15:01:13.036 INFO:teuthology.orchestra.run.smithi032.stdout:5908 punch from 0x1a8f04 to 0x1b34cb, (0xa5c7 bytes) 2024-09-21T15:01:13.041 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T15:01:13.065 INFO:teuthology.orchestra.run.smithi032.stdout:5909 write 0xcc018af thru 0xcc1038a (0xeadc bytes) 2024-09-21T15:01:13.083 INFO:teuthology.orchestra.run.smithi032.stdout:5910 writesame 0x22834d3 thru 0x228bc83 data_size 0x1(0x87b1 bytes) 2024-09-21T15:01:13.101 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:01:13.098+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:01:13.109 INFO:teuthology.orchestra.run.smithi032.stdout:5911 write 0x31e375f thru 0x31ee601 (0xaea3 bytes) 2024-09-21T15:01:13.127 INFO:teuthology.orchestra.run.smithi032.stdout:5913 write 0x7c996fc thru 0x7c9ca68 (0x336d bytes) 2024-09-21T15:01:13.135 INFO:teuthology.orchestra.run.smithi032.stdout:5914 writesame 0x836107b thru 0x8365706 data_size 0x46(0x468c bytes) 2024-09-21T15:01:13.147 INFO:teuthology.orchestra.run.smithi032.stdout:5915 read 0x4a2ca6a thru 0x4a311ed (0x4784 bytes) 2024-09-21T15:01:13.147 INFO:teuthology.orchestra.run.smithi032.stdout:5916 writesame 0x2e72c7 thru 0x2ede57 data_size 0x43(0x6b91 bytes) 2024-09-21T15:01:13.153 INFO:teuthology.orchestra.run.smithi032.stdout:5917 punch from 0xcb6f62d to 0xcb77993, (0x8366 bytes) 2024-09-21T15:01:13.153 INFO:teuthology.orchestra.run.smithi032.stdout:5918 read 0x5e702df thru 0x5e749d6 (0x46f8 bytes) 2024-09-21T15:01:13.153 INFO:teuthology.orchestra.run.smithi032.stdout:5920 read 0x6ac0b16 thru 0x6ac1172 (0x65d bytes) 2024-09-21T15:01:13.153 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:01:13.153 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:01:13.164 INFO:teuthology.orchestra.run.smithi032.stdout:5923 trunc from 0xcc1038b to 0x50e6652 2024-09-21T15:01:13.183 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T15:01:13.187 INFO:teuthology.orchestra.run.smithi032.stdout:5925 write 0xdba19f thru 0xdc4b20 (0xa982 bytes) 2024-09-21T15:01:13.203 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:01:13.201+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:01:13.209 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T15:01:13.209 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:13.209 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:13.210 INFO:teuthology.orchestra.run.smithi032.stdout:5926 trunc from 0x50e6652 to 0x8a74614 2024-09-21T15:01:13.220 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T15:01:13.229 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T15:01:13.241 INFO:teuthology.orchestra.run.smithi032.stdout:5927 write 0xd44dbb9 thru 0xd45ae2e (0xd276 bytes) 2024-09-21T15:01:13.255 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T15:01:13.262 INFO:teuthology.orchestra.run.smithi032.stdout:5928 punch from 0x7e7b758 to 0x7e7e2c6, (0x2b6e bytes) 2024-09-21T15:01:13.262 INFO:teuthology.orchestra.run.smithi032.stdout:5929 punch from 0xb396df1 to 0xb39b3fb, (0x460a bytes) 2024-09-21T15:01:13.262 INFO:teuthology.orchestra.run.smithi032.stdout:5930 punch from 0xb0e9806 to 0xb0e9b53, (0x34d bytes) 2024-09-21T15:01:13.262 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:01:13.262 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:01:13.262 INFO:teuthology.orchestra.run.smithi032.stdout:5933 read 0x4f04801 thru 0x4f0bfae (0x77ae bytes) 2024-09-21T15:01:13.262 INFO:teuthology.orchestra.run.smithi032.stdout:5934 read 0x5ad73e4 thru 0x5adce9c (0x5ab9 bytes) 2024-09-21T15:01:13.262 INFO:teuthology.orchestra.run.smithi032.stdout:5935 read 0xd35b17c thru 0xd365e82 (0xad07 bytes) 2024-09-21T15:01:13.263 INFO:teuthology.orchestra.run.smithi032.stdout:5936 writesame 0x651baa5 thru 0x6526aae data_size 0x6f(0xb00a bytes) 2024-09-21T15:01:13.281 INFO:teuthology.orchestra.run.smithi032.stdout:5937 write 0x9ad8d63 thru 0x9adaba4 (0x1e42 bytes) 2024-09-21T15:01:13.286 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-21T15:01:13.288 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T15:01:13.290 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:01:13.290 INFO:teuthology.orchestra.run.smithi032.stdout:5941 writesame 0x84c0bdd thru 0x84c5aa2 data_size 0x3(0x4ec6 bytes) 2024-09-21T15:01:13.303 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:01:13.301+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:01:13.304 INFO:teuthology.orchestra.run.smithi032.stdout:5943 writesame 0x82e68fe thru 0x82f02a8 data_size 0x5d(0x99ab bytes) 2024-09-21T15:01:13.311 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:01:13.311 INFO:teuthology.orchestra.run.smithi032.stdout:5947 read 0xd3c295c thru 0xd3d28a5 (0xff4a bytes) 2024-09-21T15:01:13.313 INFO:teuthology.orchestra.run.smithi032.stdout:5948 punch from 0x1415cb5 to 0x141f84d, (0x9b98 bytes) 2024-09-21T15:01:13.313 INFO:teuthology.orchestra.run.smithi032.stdout:5949 write 0x15173d8 thru 0x15196c0 (0x22e9 bytes) 2024-09-21T15:01:13.317 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T15:01:13.321 INFO:teuthology.orchestra.run.smithi032.stdout:5950 read 0x2aa2281 thru 0x2aabbb7 (0x9937 bytes) 2024-09-21T15:01:13.321 INFO:teuthology.orchestra.run.smithi032.stdout:5952 read 0x854c9d5 thru 0x8552b8e (0x61ba bytes) 2024-09-21T15:01:13.329 INFO:teuthology.orchestra.run.smithi032.stdout:5953 trunc from 0xd45ae2f to 0x80ee6d0 2024-09-21T15:01:13.349 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T15:01:13.352 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:13.352 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:13.354 INFO:teuthology.orchestra.run.smithi032.stdout:5954 punch from 0x4ab869 to 0x4b18e4, (0x607b bytes) 2024-09-21T15:01:13.354 INFO:teuthology.orchestra.run.smithi032.stdout:5955 write 0x6e8acc thru 0x6eb581 (0x2ab6 bytes) 2024-09-21T15:01:13.362 INFO:teuthology.orchestra.run.smithi032.stdout:5958 writesame 0x65b29ef thru 0x65bfbba data_size 0x3a(0xd1cc bytes) 2024-09-21T15:01:13.367 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T15:01:13.380 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T15:01:13.383 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:01:13.383 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:01:13.383 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:01:13.383 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:01:13.383 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:01:13.383 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:01:13.383 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:01:13.384 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:01:13.392 INFO:teuthology.orchestra.run.smithi032.stdout:5959 write 0x9fe0c38 thru 0x9feedba (0xe183 bytes) 2024-09-21T15:01:13.404 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:01:13.401+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:01:13.409 INFO:teuthology.orchestra.run.smithi032.stdout:5960 read 0x60770c thru 0x611327 (0x9c1c bytes) 2024-09-21T15:01:13.410 INFO:teuthology.orchestra.run.smithi032.stdout:compare and write disabled 2024-09-21T15:01:13.410 INFO:teuthology.orchestra.run.smithi032.stdout:5962 write 0x44a835a thru 0x44b1e59 (0x9b00 bytes) 2024-09-21T15:01:13.425 INFO:teuthology.orchestra.run.smithi032.stdout:5963 write 0x9464aad thru 0x9469a4d (0x4fa1 bytes) 2024-09-21T15:01:13.440 INFO:teuthology.orchestra.run.smithi032.stdout:5964 punch from 0x9f74da3 to 0x9f82cae, (0xdf0b bytes) 2024-09-21T15:01:13.440 INFO:teuthology.orchestra.run.smithi032.stdout:5965 read 0x468b815 thru 0x4692631 (0x6e1d bytes) 2024-09-21T15:01:13.441 INFO:teuthology.orchestra.run.smithi032.stdout:5966 write 0x24622c9 thru 0x2464b1f (0x2857 bytes) 2024-09-21T15:01:13.466 INFO:teuthology.orchestra.run.smithi032.stdout:5968 write 0xc2664c0 thru 0xc26fc46 (0x9787 bytes) 2024-09-21T15:01:13.483 INFO:teuthology.orchestra.run.smithi032.stdout:5969 writesame 0x6dc2ba5 thru 0x6dcb5c9 data_size 0xb(0x8a25 bytes) 2024-09-21T15:01:13.501 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:13.501 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:13.504 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:01:13.498+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:01:13.513 INFO:teuthology.orchestra.run.smithi032.stdout:5972 trunc from 0xc26fc47 to 0x21824f7 2024-09-21T15:01:13.515 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T15:01:13.548 INFO:teuthology.orchestra.run.smithi032.stdout:5973 writesame 0xdd8a62 thru 0xde59ab data_size 0x9d(0xcf4a bytes) 2024-09-21T15:01:13.570 INFO:teuthology.orchestra.run.smithi032.stdout:5975 writesame 0x6d8f4c4 thru 0x6d96ef7 data_size 0x42(0x7a34 bytes) 2024-09-21T15:01:13.570 INFO:teuthology.orchestra.run.smithi032.stdout:5976 write 0x43b352 thru 0x442dc0 (0x7a6f bytes) 2024-09-21T15:01:13.575 INFO:teuthology.orchestra.run.smithi032.stdout:5977 read 0x6606ff3 thru 0x6612f7a (0xbf88 bytes) 2024-09-21T15:01:13.575 INFO:teuthology.orchestra.run.smithi032.stdout:5978 write 0x5990258 thru 0x599f7ef (0xf598 bytes) 2024-09-21T15:01:13.595 INFO:teuthology.orchestra.run.smithi032.stdout:5980 write 0x2a8df45 thru 0x2a9b4b3 (0xd56f bytes) 2024-09-21T15:01:13.605 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:01:13.601+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:01:13.634 INFO:teuthology.orchestra.run.smithi032.stdout:5981 writesame 0xcbca357 thru 0xcbd332c data_size 0x39(0x8fd6 bytes) 2024-09-21T15:01:13.634 INFO:teuthology.orchestra.run.smithi032.stdout:5982 punch from 0xa054f88 to 0xa0587d3, (0x384b bytes) 2024-09-21T15:01:13.634 INFO:teuthology.orchestra.run.smithi032.stdout:5983 punch from 0xb3c025 to 0xb49e11, (0xddec bytes) 2024-09-21T15:01:13.634 INFO:teuthology.orchestra.run.smithi032.stdout:5984 writesame 0x2a3f624 thru 0x2a41590 data_size 0x1(0x1f6d bytes) 2024-09-21T15:01:13.640 INFO:teuthology.orchestra.run.smithi032.stdout:5985 read 0xb04c95a thru 0xb0594a9 (0xcb50 bytes) 2024-09-21T15:01:13.640 INFO:teuthology.orchestra.run.smithi032.stdout:5986 write 0x12a6715 thru 0x12b3e7c (0xd768 bytes) 2024-09-21T15:01:13.660 INFO:teuthology.orchestra.run.smithi032.stdout:5987 read 0x7a843ab thru 0x7a86d5b (0x29b1 bytes) 2024-09-21T15:01:13.660 INFO:teuthology.orchestra.run.smithi032.stdout:5988 read 0x794775f thru 0x7956c20 (0xf4c2 bytes) 2024-09-21T15:01:13.660 INFO:teuthology.orchestra.run.smithi032.stdout:5991 write 0x5bf6bd5 thru 0x5bfc94c (0x5d78 bytes) 2024-09-21T15:01:13.689 INFO:teuthology.orchestra.run.smithi032.stdout:5992 write 0xe769bcb thru 0xe77449a (0xa8d0 bytes) 2024-09-21T15:01:13.695 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:13.695 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:13.706 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:01:13.701+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:01:13.708 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T15:01:13.709 INFO:teuthology.orchestra.run.smithi032.stdout:5993 punch from 0x9121779 to 0x912c8d3, (0xb15a bytes) 2024-09-21T15:01:13.709 INFO:teuthology.orchestra.run.smithi032.stdout:5994 writesame 0xd83deb6 thru 0xd849ab9 data_size 0x7e(0xbc04 bytes) 2024-09-21T15:01:13.723 INFO:teuthology.orchestra.run.smithi032.stdout:5995 read 0xa56d7fc thru 0xa57180b (0x4010 bytes) 2024-09-21T15:01:13.723 INFO:teuthology.orchestra.run.smithi032.stdout:5997 write 0xaace4ba thru 0xaacf444 (0xf8b bytes) 2024-09-21T15:01:13.728 INFO:teuthology.orchestra.run.smithi032.stdout:5998 write 0xe74a236 thru 0xe74fa08 (0x57d3 bytes) 2024-09-21T15:01:13.732 INFO:teuthology.orchestra.run.smithi032.stdout:6000 write 0x17a6dad thru 0x17aa688 (0x38dc bytes) 2024-09-21T15:01:13.762 INFO:teuthology.orchestra.run.smithi032.stdout:checking clone #36, image image_client.0-clone36 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent37 2024-09-21T15:01:13.807 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:01:13.802+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:01:13.814 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:13.814 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:13.826 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T15:01:13.909 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:01:13.905+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:01:13.989 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:13.989 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:13.999 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T15:01:14.010 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:01:14.005+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:01:14.105 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:14.105 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:14.111 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:01:14.109+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:01:14.115 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T15:01:14.211 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:01:14.206+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:01:14.229 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:14.229 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:14.240 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T15:01:14.286 INFO:teuthology.orchestra.run.smithi032.stderr:nodeep-scrub is unset 2024-09-21T15:01:14.312 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:01:14.309+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:01:14.372 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:14.372 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:14.383 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T15:01:14.413 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:01:14.409+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:01:14.487 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:14.487 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:14.497 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T15:01:14.539 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:01:14.537+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:01:14.615 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:01:14.610+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:01:14.651 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:14.651 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:14.662 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T15:01:14.715 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:01:14.714+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:01:14.772 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:14.772 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:14.783 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T15:01:14.827 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:01:14.825+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:01:14.891 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:14.891 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:14.901 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T15:01:14.911 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T15:01:14.917 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:01:14.913+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:01:15.014 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:15.014 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:15.017 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:01:15.014+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:01:15.023 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T15:01:15.118 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:01:15.113+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:01:15.130 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:15.130 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:15.140 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T15:01:15.219 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:01:15.214+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:01:15.248 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-21T15:01:15.251 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-09-21T15:01:15.263 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":273,"stamp":"2024-09-21T15:01:15.159108+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1167768743,"num_objects":1612,"num_object_clones":0,"num_object_copies":3224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1612,"num_whiteouts":0,"num_read":162594,"num_read_kb":1208105,"num_write":30117,"num_write_kb":939856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1920,"num_bytes_recovered":145354624,"num_keys_recovered":229,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18476,"ondisk_log_size":18476,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":513676,"kb_used_data":101540,"kb_used_omap":12,"kb_used_meta":412019,"kb_avail":754461044,"statfs":{"total":773094113280,"available":772568109056,"internally_reserved":0,"allocated":103976960,"data_stored":687012370,"data_compressed":13548995,"data_compressed_allocated":61419520,"data_compressed_original":659053697,"omap_allocated":12744,"internal_metadata":421908024},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":115720037,"num_objects":274,"num_object_clones":0,"num_object_copies":548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":274,"num_whiteouts":0,"num_read":18034,"num_read_kb":54723,"num_write":3916,"num_write_kb":53548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.910235"},"pg_stats":[{"pgid":"3.1f","version":"180'467","reported_seq":3468,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:01:10.439967+0000","last_change":"2024-09-21T15:01:04.223131+0000","last_active":"2024-09-21T15:01:10.439967+0000","last_peered":"2024-09-21T15:01:10.439967+0000","last_clean":"2024-09-21T15:01:10.439967+0000","last_became_active":"2024-09-21T14:58:19.410055+0000","last_became_peered":"2024-09-21T14:58:19.410055+0000","last_unstale":"2024-09-21T15:01:10.439967+0000","last_undegraded":"2024-09-21T15:01:10.439967+0000","last_fullsized":"2024-09-21T15:01:10.439967+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":467,"log_dups_size":0,"ondisk_log_size":467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:49:13.944219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063336100000000004,"stat_sum":{"num_bytes":39568717,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":3403,"num_read_kb":42213,"num_write":855,"num_write_kb":36337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.1e","version":"180'378","reported_seq":2988,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:01:10.438472+0000","last_change":"2024-09-21T15:01:06.173953+0000","last_active":"2024-09-21T15:01:10.438472+0000","last_peered":"2024-09-21T15:01:10.438472+0000","last_clean":"2024-09-21T15:01:10.438472+0000","last_became_active":"2024-09-21T15:01:06.168726+0000","last_became_peered":"2024-09-21T15:01:06.168726+0000","last_unstale":"2024-09-21T15:01:10.438472+0000","last_undegraded":"2024-09-21T15:01:10.438472+0000","last_fullsized":"2024-09-21T15:01:10.438472+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":378,"log_dups_size":0,"ondisk_log_size":378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:29:01.079772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021250790000000002,"stat_sum":{"num_bytes":23357729,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":2999,"num_read_kb":22037,"num_write":522,"num_write_kb":18370,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":148,"num_bytes_recovered":11635695,"num_keys_recovered":22,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.1d","version":"180'288","reported_seq":4485,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:01:10.388302+0000","last_change":"2024-09-21T15:01:04.229475+0000","last_active":"2024-09-21T15:01:10.388302+0000","last_peered":"2024-09-21T15:01:10.388302+0000","last_clean":"2024-09-21T15:01:10.388302+0000","last_became_active":"2024-09-21T14:58:19.411387+0000","last_became_peered":"2024-09-21T14:58:19.411387+0000","last_unstale":"2024-09-21T15:01:10.388302+0000","last_undegraded":"2024-09-21T15:01:10.388302+0000","last_fullsized":"2024-09-21T15:01:10.388302+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":288,"log_dups_size":0,"ondisk_log_size":288,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:37:57.597069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054775500000000003,"stat_sum":{"num_bytes":21313673,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":3859,"num_read_kb":26411,"num_write":483,"num_write_kb":26568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.1c","version":"180'455","reported_seq":3762,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:01:10.434175+0000","last_change":"2024-09-21T15:01:04.234927+0000","last_active":"2024-09-21T15:01:10.434175+0000","last_peered":"2024-09-21T15:01:10.434175+0000","last_clean":"2024-09-21T15:01:10.434175+0000","last_became_active":"2024-09-21T14:58:19.411306+0000","last_became_peered":"2024-09-21T14:58:19.411306+0000","last_unstale":"2024-09-21T15:01:10.434175+0000","last_undegraded":"2024-09-21T15:01:10.434175+0000","last_fullsized":"2024-09-21T15:01:10.434175+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:20:27.079795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010334350000000001,"stat_sum":{"num_bytes":23930794,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":3676,"num_read_kb":23881,"num_write":713,"num_write_kb":25173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.1b","version":"180'445","reported_seq":3596,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:01:10.374329+0000","last_change":"2024-09-21T15:01:04.227802+0000","last_active":"2024-09-21T15:01:10.374329+0000","last_peered":"2024-09-21T15:01:10.374329+0000","last_clean":"2024-09-21T15:01:10.374329+0000","last_became_active":"2024-09-21T14:59:31.131051+0000","last_became_peered":"2024-09-21T14:59:31.131051+0000","last_unstale":"2024-09-21T15:01:10.374329+0000","last_undegraded":"2024-09-21T15:01:10.374329+0000","last_fullsized":"2024-09-21T15:01:10.374329+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":445,"log_dups_size":0,"ondisk_log_size":445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:21:37.320774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001254673,"stat_sum":{"num_bytes":28181374,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":3454,"num_read_kb":31760,"num_write":619,"num_write_kb":22499,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":117,"num_bytes_recovered":7757116,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.1a","version":"180'392","reported_seq":4180,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:01:10.539507+0000","last_change":"2024-09-21T15:01:04.234672+0000","last_active":"2024-09-21T15:01:10.539507+0000","last_peered":"2024-09-21T15:01:10.539507+0000","last_clean":"2024-09-21T15:01:10.539507+0000","last_became_active":"2024-09-21T14:58:19.409415+0000","last_became_peered":"2024-09-21T14:58:19.409415+0000","last_unstale":"2024-09-21T15:01:10.539507+0000","last_undegraded":"2024-09-21T15:01:10.539507+0000","last_fullsized":"2024-09-21T15:01:10.539507+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":392,"log_dups_size":0,"ondisk_log_size":392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:28:46.336301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060307399999999999,"stat_sum":{"num_bytes":27365738,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":3912,"num_read_kb":27610,"num_write":612,"num_write_kb":23813,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.19","version":"180'695","reported_seq":7919,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:01:10.517130+0000","last_change":"2024-09-21T15:01:04.228157+0000","last_active":"2024-09-21T15:01:10.517130+0000","last_peered":"2024-09-21T15:01:10.517130+0000","last_clean":"2024-09-21T15:01:10.517130+0000","last_became_active":"2024-09-21T14:59:32.130624+0000","last_became_peered":"2024-09-21T14:59:32.130624+0000","last_unstale":"2024-09-21T15:01:10.517130+0000","last_undegraded":"2024-09-21T15:01:10.517130+0000","last_fullsized":"2024-09-21T15:01:10.517130+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":695,"log_dups_size":0,"ondisk_log_size":695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:37:53.312733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001540409,"stat_sum":{"num_bytes":57317066,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":8982,"num_read_kb":66656,"num_write":1130,"num_write_kb":29843,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":128,"num_bytes_recovered":11896992,"num_keys_recovered":44,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.18","version":"180'549","reported_seq":8261,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:01:10.485156+0000","last_change":"2024-09-21T15:01:04.223390+0000","last_active":"2024-09-21T15:01:10.485156+0000","last_peered":"2024-09-21T15:01:10.485156+0000","last_clean":"2024-09-21T15:01:10.485156+0000","last_became_active":"2024-09-21T14:58:19.409371+0000","last_became_peered":"2024-09-21T14:58:19.409371+0000","last_unstale":"2024-09-21T15:01:10.485156+0000","last_undegraded":"2024-09-21T15:01:10.485156+0000","last_fullsized":"2024-09-21T15:01:10.485156+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":549,"log_dups_size":0,"ondisk_log_size":549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:35:27.143445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076656300000000005,"stat_sum":{"num_bytes":39453048,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":8844,"num_read_kb":43937,"num_write":867,"num_write_kb":30512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.17","version":"180'348","reported_seq":3695,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:01:10.399384+0000","last_change":"2024-09-21T15:01:04.225804+0000","last_active":"2024-09-21T15:01:10.399384+0000","last_peered":"2024-09-21T15:01:10.399384+0000","last_clean":"2024-09-21T15:01:10.399384+0000","last_became_active":"2024-09-21T14:58:19.410306+0000","last_became_peered":"2024-09-21T14:58:19.410306+0000","last_unstale":"2024-09-21T15:01:10.399384+0000","last_undegraded":"2024-09-21T15:01:10.399384+0000","last_fullsized":"2024-09-21T15:01:10.399384+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:33:50.089165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030273969999999998,"stat_sum":{"num_bytes":32084432,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":3370,"num_read_kb":27721,"num_write":705,"num_write_kb":34569,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.16","version":"180'911","reported_seq":5643,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:01:10.462549+0000","last_change":"2024-09-21T15:01:06.169652+0000","last_active":"2024-09-21T15:01:10.462549+0000","last_peered":"2024-09-21T15:01:10.462549+0000","last_clean":"2024-09-21T15:01:10.462549+0000","last_became_active":"2024-09-21T15:01:06.166756+0000","last_became_peered":"2024-09-21T15:01:06.166756+0000","last_unstale":"2024-09-21T15:01:10.462549+0000","last_undegraded":"2024-09-21T15:01:10.462549+0000","last_fullsized":"2024-09-21T15:01:10.462549+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":911,"log_dups_size":0,"ondisk_log_size":911,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:19:04.988784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000477617,"stat_sum":{"num_bytes":39025443,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":7660,"num_read_kb":43453,"num_write":1485,"num_write_kb":28270,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":271,"num_bytes_recovered":16821266,"num_keys_recovered":55,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.15","version":"180'730","reported_seq":3633,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:01:10.489670+0000","last_change":"2024-09-21T15:01:04.231959+0000","last_active":"2024-09-21T15:01:10.489670+0000","last_peered":"2024-09-21T15:01:10.489670+0000","last_clean":"2024-09-21T15:01:10.489670+0000","last_became_active":"2024-09-21T14:58:19.410687+0000","last_became_peered":"2024-09-21T14:58:19.410687+0000","last_unstale":"2024-09-21T15:01:10.489670+0000","last_undegraded":"2024-09-21T15:01:10.489670+0000","last_fullsized":"2024-09-21T15:01:10.489670+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":730,"log_dups_size":0,"ondisk_log_size":730,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:21:42.226982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033742220000000001,"stat_sum":{"num_bytes":39686352,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":4682,"num_read_kb":48927,"num_write":1251,"num_write_kb":33262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.14","version":"180'460","reported_seq":3873,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:01:10.434207+0000","last_change":"2024-09-21T15:01:04.229559+0000","last_active":"2024-09-21T15:01:10.434207+0000","last_peered":"2024-09-21T15:01:10.434207+0000","last_clean":"2024-09-21T15:01:10.434207+0000","last_became_active":"2024-09-21T14:58:19.407310+0000","last_became_peered":"2024-09-21T14:58:19.407310+0000","last_unstale":"2024-09-21T15:01:10.434207+0000","last_undegraded":"2024-09-21T15:01:10.434207+0000","last_fullsized":"2024-09-21T15:01:10.434207+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":460,"log_dups_size":0,"ondisk_log_size":460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:47:46.079115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052455500000000001,"stat_sum":{"num_bytes":38779984,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":3708,"num_read_kb":41600,"num_write":644,"num_write_kb":33420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":174,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-21T15:01:08.256323+0000","last_change":"2024-09-21T14:58:11.348177+0000","last_active":"2024-09-21T15:01:08.256323+0000","last_peered":"2024-09-21T15:01:08.256323+0000","last_clean":"2024-09-21T15:01:08.256323+0000","last_became_active":"2024-09-21T14:58:09.328417+0000","last_became_peered":"2024-09-21T14:58:09.328417+0000","last_unstale":"2024-09-21T15:01:08.256323+0000","last_undegraded":"2024-09-21T15:01:08.256323+0000","last_fullsized":"2024-09-21T15:01:08.256323+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:52:22.671185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018917490000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"180'398","reported_seq":5452,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:01:10.562506+0000","last_change":"2024-09-21T15:01:04.228240+0000","last_active":"2024-09-21T15:01:10.562506+0000","last_peered":"2024-09-21T15:01:10.562506+0000","last_clean":"2024-09-21T15:01:10.562506+0000","last_became_active":"2024-09-21T14:59:32.131232+0000","last_became_peered":"2024-09-21T14:59:32.131232+0000","last_unstale":"2024-09-21T15:01:10.562506+0000","last_undegraded":"2024-09-21T15:01:10.562506+0000","last_fullsized":"2024-09-21T15:01:10.562506+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":398,"log_dups_size":0,"ondisk_log_size":398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:31:04.421782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001438581,"stat_sum":{"num_bytes":53173338,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":5286,"num_read_kb":51707,"num_write":539,"num_write_kb":27824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":3585713,"num_keys_recovered":9,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":99,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-21T15:01:08.242215+0000","last_change":"2024-09-21T14:59:28.228883+0000","last_active":"2024-09-21T15:01:08.242215+0000","last_peered":"2024-09-21T15:01:08.242215+0000","last_clean":"2024-09-21T15:01:08.242215+0000","last_became_active":"2024-09-21T14:59:28.228464+0000","last_became_peered":"2024-09-21T14:59:28.228464+0000","last_unstale":"2024-09-21T15:01:08.242215+0000","last_undegraded":"2024-09-21T15:01:08.242215+0000","last_fullsized":"2024-09-21T15:01:08.242215+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:00:26.310864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"180'818","reported_seq":3413,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:01:10.447321+0000","last_change":"2024-09-21T15:01:04.228668+0000","last_active":"2024-09-21T15:01:10.447321+0000","last_peered":"2024-09-21T15:01:10.447321+0000","last_clean":"2024-09-21T15:01:10.447321+0000","last_became_active":"2024-09-21T14:59:31.132581+0000","last_became_peered":"2024-09-21T14:59:31.132581+0000","last_unstale":"2024-09-21T15:01:10.447321+0000","last_undegraded":"2024-09-21T15:01:10.447321+0000","last_fullsized":"2024-09-21T15:01:10.447321+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":818,"log_dups_size":0,"ondisk_log_size":818,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:36:45.467515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016620179999999999,"stat_sum":{"num_bytes":31751011,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":4991,"num_read_kb":30499,"num_write":1244,"num_write_kb":25152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":119,"num_bytes_recovered":6871924,"num_keys_recovered":11,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":190,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-21T15:01:08.256889+0000","last_change":"2024-09-21T14:59:28.231502+0000","last_active":"2024-09-21T15:01:08.256889+0000","last_peered":"2024-09-21T15:01:08.256889+0000","last_clean":"2024-09-21T15:01:08.256889+0000","last_became_active":"2024-09-21T14:59:28.231043+0000","last_became_peered":"2024-09-21T14:59:28.231043+0000","last_unstale":"2024-09-21T15:01:08.256889+0000","last_undegraded":"2024-09-21T15:01:08.256889+0000","last_fullsized":"2024-09-21T15:01:08.256889+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:09:46.773690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019338770000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"180'562","reported_seq":5107,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:01:10.455452+0000","last_change":"2024-09-21T15:01:04.227811+0000","last_active":"2024-09-21T15:01:10.455452+0000","last_peered":"2024-09-21T15:01:10.455452+0000","last_clean":"2024-09-21T15:01:10.455452+0000","last_became_active":"2024-09-21T15:00:34.347238+0000","last_became_peered":"2024-09-21T15:00:34.347238+0000","last_unstale":"2024-09-21T15:01:10.455452+0000","last_undegraded":"2024-09-21T15:01:10.455452+0000","last_fullsized":"2024-09-21T15:01:10.455452+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":562,"log_dups_size":0,"ondisk_log_size":562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:31:33.639268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001277607,"stat_sum":{"num_bytes":44434212,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":5081,"num_read_kb":38156,"num_write":1014,"num_write_kb":41198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":14049436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":175,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-21T15:01:08.250709+0000","last_change":"2024-09-21T14:58:11.406176+0000","last_active":"2024-09-21T15:01:08.250709+0000","last_peered":"2024-09-21T15:01:08.250709+0000","last_clean":"2024-09-21T15:01:08.250709+0000","last_became_active":"2024-09-21T14:58:09.777465+0000","last_became_peered":"2024-09-21T14:58:09.777465+0000","last_unstale":"2024-09-21T15:01:08.250709+0000","last_undegraded":"2024-09-21T15:01:08.250709+0000","last_fullsized":"2024-09-21T15:01:08.250709+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:55:04.667816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095704000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"180'668","reported_seq":2549,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:01:10.244392+0000","last_change":"2024-09-21T15:01:04.224786+0000","last_active":"2024-09-21T15:01:10.244392+0000","last_peered":"2024-09-21T15:01:10.244392+0000","last_clean":"2024-09-21T15:01:10.244392+0000","last_became_active":"2024-09-21T14:58:19.408971+0000","last_became_peered":"2024-09-21T14:58:19.408971+0000","last_unstale":"2024-09-21T15:01:10.244392+0000","last_undegraded":"2024-09-21T15:01:10.244392+0000","last_fullsized":"2024-09-21T15:01:10.244392+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":668,"log_dups_size":0,"ondisk_log_size":668,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:29:52.642121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000380054,"stat_sum":{"num_bytes":22456262,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":3561,"num_read_kb":20372,"num_write":1165,"num_write_kb":20644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.3","version":"180'747","reported_seq":2955,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:01:10.533772+0000","last_change":"2024-09-21T15:01:04.231096+0000","last_active":"2024-09-21T15:01:10.533772+0000","last_peered":"2024-09-21T15:01:10.533772+0000","last_clean":"2024-09-21T15:01:10.533772+0000","last_became_active":"2024-09-21T14:58:19.417453+0000","last_became_peered":"2024-09-21T14:58:19.417453+0000","last_unstale":"2024-09-21T15:01:10.533772+0000","last_undegraded":"2024-09-21T15:01:10.533772+0000","last_fullsized":"2024-09-21T15:01:10.533772+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":747,"log_dups_size":0,"ondisk_log_size":747,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:19:27.087272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002785708,"stat_sum":{"num_bytes":19818390,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":4401,"num_read_kb":24174,"num_write":1307,"num_write_kb":21522,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":176,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-21T15:01:08.255930+0000","last_change":"2024-09-21T14:58:11.347504+0000","last_active":"2024-09-21T15:01:08.255930+0000","last_peered":"2024-09-21T15:01:08.255930+0000","last_clean":"2024-09-21T15:01:08.255930+0000","last_became_active":"2024-09-21T14:58:09.775981+0000","last_became_peered":"2024-09-21T14:58:09.775981+0000","last_unstale":"2024-09-21T15:01:08.255930+0000","last_undegraded":"2024-09-21T15:01:08.255930+0000","last_fullsized":"2024-09-21T15:01:08.255930+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:47:16.286652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012568880000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"180'945","reported_seq":3401,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:01:10.563901+0000","last_change":"2024-09-21T15:01:04.223338+0000","last_active":"2024-09-21T15:01:10.563901+0000","last_peered":"2024-09-21T15:01:10.563901+0000","last_clean":"2024-09-21T15:01:10.563901+0000","last_became_active":"2024-09-21T14:58:19.405811+0000","last_became_peered":"2024-09-21T14:58:19.405811+0000","last_unstale":"2024-09-21T15:01:10.563901+0000","last_undegraded":"2024-09-21T15:01:10.563901+0000","last_fullsized":"2024-09-21T15:01:10.563901+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":945,"log_dups_size":0,"ondisk_log_size":945,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:36:25.551607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083259800000000004,"stat_sum":{"num_bytes":23975054,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":5779,"num_read_kb":27902,"num_write":1544,"num_write_kb":26901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":176,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:01:09.256536+0000","last_change":"2024-09-21T14:58:11.342210+0000","last_active":"2024-09-21T15:01:09.256536+0000","last_peered":"2024-09-21T15:01:09.256536+0000","last_clean":"2024-09-21T15:01:09.256536+0000","last_became_active":"2024-09-21T14:58:09.330624+0000","last_became_peered":"2024-09-21T14:58:09.330624+0000","last_unstale":"2024-09-21T15:01:09.256536+0000","last_undegraded":"2024-09-21T15:01:09.256536+0000","last_fullsized":"2024-09-21T15:01:09.256536+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:29:43.229538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001051151,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"180'290","reported_seq":2025,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:01:10.451444+0000","last_change":"2024-09-21T15:01:05.166575+0000","last_active":"2024-09-21T15:01:10.451444+0000","last_peered":"2024-09-21T15:01:10.451444+0000","last_clean":"2024-09-21T15:01:10.451444+0000","last_became_active":"2024-09-21T15:01:05.164862+0000","last_became_peered":"2024-09-21T15:01:05.164862+0000","last_unstale":"2024-09-21T15:01:10.451444+0000","last_undegraded":"2024-09-21T15:01:10.451444+0000","last_fullsized":"2024-09-21T15:01:10.451444+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":290,"log_dups_size":0,"ondisk_log_size":290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:57:46.675335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046340299999999998,"stat_sum":{"num_bytes":37248411,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":0,"num_read":2367,"num_read_kb":39964,"num_write":480,"num_write_kb":39361,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":254,"num_bytes_recovered":20571253,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":174,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-21T15:01:08.255787+0000","last_change":"2024-09-21T14:58:11.346766+0000","last_active":"2024-09-21T15:01:08.255787+0000","last_peered":"2024-09-21T15:01:08.255787+0000","last_clean":"2024-09-21T15:01:08.255787+0000","last_became_active":"2024-09-21T14:58:09.329748+0000","last_became_peered":"2024-09-21T14:58:09.329748+0000","last_unstale":"2024-09-21T15:01:08.255787+0000","last_undegraded":"2024-09-21T15:01:08.255787+0000","last_fullsized":"2024-09-21T15:01:08.255787+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:31:20.267675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052698000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"180'770","reported_seq":7475,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:01:10.560976+0000","last_change":"2024-09-21T15:01:04.224483+0000","last_active":"2024-09-21T15:01:10.560976+0000","last_peered":"2024-09-21T15:01:10.560976+0000","last_clean":"2024-09-21T15:01:10.560976+0000","last_became_active":"2024-09-21T14:58:41.401003+0000","last_became_peered":"2024-09-21T14:58:41.401003+0000","last_unstale":"2024-09-21T15:01:10.560976+0000","last_undegraded":"2024-09-21T15:01:10.560976+0000","last_fullsized":"2024-09-21T15:01:10.560976+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":770,"log_dups_size":0,"ondisk_log_size":770,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:05:37.644835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046693700000000001,"stat_sum":{"num_bytes":53815073,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":9241,"num_read_kb":48522,"num_write":1137,"num_write_kb":24140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1403348,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":175,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-21T15:01:08.256169+0000","last_change":"2024-09-21T14:58:11.347700+0000","last_active":"2024-09-21T15:01:08.256169+0000","last_peered":"2024-09-21T15:01:08.256169+0000","last_clean":"2024-09-21T15:01:08.256169+0000","last_became_active":"2024-09-21T14:58:09.438842+0000","last_became_peered":"2024-09-21T14:58:09.438842+0000","last_unstale":"2024-09-21T15:01:08.256169+0000","last_undegraded":"2024-09-21T15:01:08.256169+0000","last_fullsized":"2024-09-21T15:01:08.256169+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:41:34.626249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001426194,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":323,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-21T15:01:08.243225+0000","last_change":"2024-09-21T14:58:03.616055+0000","last_active":"2024-09-21T15:01:08.243225+0000","last_peered":"2024-09-21T15:01:08.243225+0000","last_clean":"2024-09-21T15:01:08.243225+0000","last_became_active":"2024-09-21T14:58:03.613214+0000","last_became_peered":"2024-09-21T14:58:03.613214+0000","last_unstale":"2024-09-21T15:01:08.243225+0000","last_undegraded":"2024-09-21T15:01:08.243225+0000","last_fullsized":"2024-09-21T15:01:08.243225+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:02.180162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:02.180162+0000","last_clean_scrub_stamp":"2024-09-21T14:58:02.180162+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:29:05.860361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"180'613","reported_seq":4055,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:01:10.409385+0000","last_change":"2024-09-21T15:01:04.227318+0000","last_active":"2024-09-21T15:01:10.409385+0000","last_peered":"2024-09-21T15:01:10.409385+0000","last_clean":"2024-09-21T15:01:10.409385+0000","last_became_active":"2024-09-21T14:59:31.131460+0000","last_became_peered":"2024-09-21T14:59:31.131460+0000","last_unstale":"2024-09-21T15:01:10.409385+0000","last_undegraded":"2024-09-21T15:01:10.409385+0000","last_fullsized":"2024-09-21T15:01:10.409385+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":613,"log_dups_size":0,"ondisk_log_size":613,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:01:38.353476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098975100000000009,"stat_sum":{"num_bytes":41254141,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":4356,"num_read_kb":43318,"num_write":1090,"num_write_kb":30450,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":119,"num_bytes_recovered":6097618,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.9","version":"180'812","reported_seq":3018,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:01:10.447535+0000","last_change":"2024-09-21T15:01:04.232710+0000","last_active":"2024-09-21T15:01:10.447535+0000","last_peered":"2024-09-21T15:01:10.447535+0000","last_clean":"2024-09-21T15:01:10.447535+0000","last_became_active":"2024-09-21T14:59:28.233956+0000","last_became_peered":"2024-09-21T14:59:28.233956+0000","last_unstale":"2024-09-21T15:01:10.447535+0000","last_undegraded":"2024-09-21T15:01:10.447535+0000","last_fullsized":"2024-09-21T15:01:10.447535+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":812,"log_dups_size":0,"ondisk_log_size":812,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:29:09.682792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027159770000000001,"stat_sum":{"num_bytes":31459619,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":4523,"num_read_kb":33253,"num_write":1434,"num_write_kb":33626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":91,"num_bytes_recovered":6925016,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.a","version":"180'848","reported_seq":4312,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:01:10.504216+0000","last_change":"2024-09-21T15:01:04.234662+0000","last_active":"2024-09-21T15:01:10.504216+0000","last_peered":"2024-09-21T15:01:10.504216+0000","last_clean":"2024-09-21T15:01:10.504216+0000","last_became_active":"2024-09-21T15:00:27.280971+0000","last_became_peered":"2024-09-21T15:00:27.280971+0000","last_unstale":"2024-09-21T15:01:10.504216+0000","last_undegraded":"2024-09-21T15:01:10.504216+0000","last_fullsized":"2024-09-21T15:01:10.504216+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":848,"log_dups_size":0,"ondisk_log_size":848,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:26:07.617059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076445899999999999,"stat_sum":{"num_bytes":24461444,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":6575,"num_read_kb":30248,"num_write":1267,"num_write_kb":33063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":9228123,"num_keys_recovered":33,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.b","version":"180'467","reported_seq":3008,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:01:11.566836+0000","last_change":"2024-09-21T15:01:11.273095+0000","last_active":"2024-09-21T15:01:11.566836+0000","last_peered":"2024-09-21T15:01:11.566836+0000","last_clean":"2024-09-21T15:01:11.566836+0000","last_became_active":"2024-09-21T14:58:19.410679+0000","last_became_peered":"2024-09-21T14:58:19.410679+0000","last_unstale":"2024-09-21T15:01:11.566836+0000","last_undegraded":"2024-09-21T15:01:11.566836+0000","last_fullsized":"2024-09-21T15:01:11.566836+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":467,"log_dups_size":0,"ondisk_log_size":467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:07:40.758932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078806500000000001,"stat_sum":{"num_bytes":35499411,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":2893,"num_read_kb":36574,"num_write":768,"num_write_kb":39552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"6e","length":"1"},{"start":"71","length":"1"}]},{"pgid":"3.c","version":"180'415","reported_seq":4322,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:01:10.472448+0000","last_change":"2024-09-21T15:01:04.231357+0000","last_active":"2024-09-21T15:01:10.472448+0000","last_peered":"2024-09-21T15:01:10.472448+0000","last_clean":"2024-09-21T15:01:10.472448+0000","last_became_active":"2024-09-21T14:59:25.130057+0000","last_became_peered":"2024-09-21T14:59:25.130057+0000","last_unstale":"2024-09-21T15:01:10.472448+0000","last_undegraded":"2024-09-21T15:01:10.472448+0000","last_fullsized":"2024-09-21T15:01:10.472448+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":415,"log_dups_size":0,"ondisk_log_size":415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:53:33.752550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003377793,"stat_sum":{"num_bytes":26473401,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":4337,"num_read_kb":28452,"num_write":657,"num_write_kb":24589,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":9059600,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.d","version":"180'436","reported_seq":4931,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:01:10.344978+0000","last_change":"2024-09-21T15:01:04.233218+0000","last_active":"2024-09-21T15:01:10.344978+0000","last_peered":"2024-09-21T15:01:10.344978+0000","last_clean":"2024-09-21T15:01:10.344978+0000","last_became_active":"2024-09-21T14:58:19.407884+0000","last_became_peered":"2024-09-21T14:58:19.407884+0000","last_unstale":"2024-09-21T15:01:10.344978+0000","last_undegraded":"2024-09-21T15:01:10.344978+0000","last_fullsized":"2024-09-21T15:01:10.344978+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":436,"log_dups_size":0,"ondisk_log_size":436,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:28:57.111615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034473960000000001,"stat_sum":{"num_bytes":55848141,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":4783,"num_read_kb":55414,"num_write":842,"num_write_kb":38545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.e","version":"180'785","reported_seq":2866,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:01:10.395014+0000","last_change":"2024-09-21T15:01:05.176277+0000","last_active":"2024-09-21T15:01:10.395014+0000","last_peered":"2024-09-21T15:01:10.395014+0000","last_clean":"2024-09-21T15:01:10.395014+0000","last_became_active":"2024-09-21T15:01:05.172427+0000","last_became_peered":"2024-09-21T15:01:05.172427+0000","last_unstale":"2024-09-21T15:01:10.395014+0000","last_undegraded":"2024-09-21T15:01:10.395014+0000","last_fullsized":"2024-09-21T15:01:10.395014+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":785,"log_dups_size":0,"ondisk_log_size":785,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:28:18.710621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093917399999999998,"stat_sum":{"num_bytes":17357416,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":4155,"num_read_kb":16497,"num_write":1410,"num_write_kb":17911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":138,"num_bytes_recovered":7486496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.f","version":"180'449","reported_seq":5872,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:01:10.462551+0000","last_change":"2024-09-21T15:01:04.229660+0000","last_active":"2024-09-21T15:01:10.462551+0000","last_peered":"2024-09-21T15:01:10.462551+0000","last_clean":"2024-09-21T15:01:10.462551+0000","last_became_active":"2024-09-21T14:58:19.410406+0000","last_became_peered":"2024-09-21T14:58:19.410406+0000","last_unstale":"2024-09-21T15:01:10.462551+0000","last_undegraded":"2024-09-21T15:01:10.462551+0000","last_fullsized":"2024-09-21T15:01:10.462551+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:59:16.026939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014556770000000001,"stat_sum":{"num_bytes":51593222,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":5754,"num_read_kb":55172,"num_write":733,"num_write_kb":36722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.10","version":"180'444","reported_seq":4319,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:01:10.439141+0000","last_change":"2024-09-21T15:01:04.233986+0000","last_active":"2024-09-21T15:01:10.439141+0000","last_peered":"2024-09-21T15:01:10.439141+0000","last_clean":"2024-09-21T15:01:10.439141+0000","last_became_active":"2024-09-21T14:58:19.409935+0000","last_became_peered":"2024-09-21T14:58:19.409935+0000","last_unstale":"2024-09-21T15:01:10.439141+0000","last_undegraded":"2024-09-21T15:01:10.439141+0000","last_fullsized":"2024-09-21T15:01:10.439141+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":444,"log_dups_size":0,"ondisk_log_size":444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:10:36.252629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001679137,"stat_sum":{"num_bytes":53119703,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":3929,"num_read_kb":58277,"num_write":731,"num_write_kb":37234,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.11","version":"180'468","reported_seq":4579,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:01:10.428493+0000","last_change":"2024-09-21T15:01:05.168682+0000","last_active":"2024-09-21T15:01:10.428493+0000","last_peered":"2024-09-21T15:01:10.428493+0000","last_clean":"2024-09-21T15:01:10.428493+0000","last_became_active":"2024-09-21T15:01:05.165955+0000","last_became_peered":"2024-09-21T15:01:05.165955+0000","last_unstale":"2024-09-21T15:01:10.428493+0000","last_undegraded":"2024-09-21T15:01:10.428493+0000","last_fullsized":"2024-09-21T15:01:10.428493+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":468,"log_dups_size":0,"ondisk_log_size":468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:34:54.111119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010096230000000001,"stat_sum":{"num_bytes":28747420,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":5222,"num_read_kb":27938,"num_write":639,"num_write_kb":24066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":166,"num_bytes_recovered":11965028,"num_keys_recovered":33,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.12","version":"180'528","reported_seq":8023,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:01:10.331152+0000","last_change":"2024-09-21T15:01:04.224935+0000","last_active":"2024-09-21T15:01:10.331152+0000","last_peered":"2024-09-21T15:01:10.331152+0000","last_clean":"2024-09-21T15:01:10.331152+0000","last_became_active":"2024-09-21T15:00:07.155144+0000","last_became_peered":"2024-09-21T15:00:07.155144+0000","last_unstale":"2024-09-21T15:01:10.331152+0000","last_undegraded":"2024-09-21T15:01:10.331152+0000","last_fullsized":"2024-09-21T15:01:10.331152+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":528,"log_dups_size":0,"ondisk_log_size":528,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:58:00.133639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010497799999999999,"stat_sum":{"num_bytes":45585679,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":8710,"num_read_kb":36186,"num_write":911,"num_write_kb":20822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"6e","length":"1"}]},{"pgid":"3.13","version":"180'775","reported_seq":6248,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:01:10.313496+0000","last_change":"2024-09-21T15:01:04.229229+0000","last_active":"2024-09-21T15:01:10.313496+0000","last_peered":"2024-09-21T15:01:10.313496+0000","last_clean":"2024-09-21T15:01:10.313496+0000","last_became_active":"2024-09-21T14:58:19.412030+0000","last_became_peered":"2024-09-21T14:58:19.412030+0000","last_unstale":"2024-09-21T15:01:10.313496+0000","last_undegraded":"2024-09-21T15:01:10.313496+0000","last_fullsized":"2024-09-21T15:01:10.313496+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":775,"log_dups_size":0,"ondisk_log_size":775,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:44:29.282391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046048699999999998,"stat_sum":{"num_bytes":59042658,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":7985,"num_read_kb":59174,"num_write":1158,"num_write_kb":31480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"6e","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1167178356,"num_objects":1608,"num_object_clones":0,"num_object_copies":3216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1608,"num_whiteouts":0,"num_read":162478,"num_read_kb":1208005,"num_write":29956,"num_write_kb":937438,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1920,"num_bytes_recovered":145354624,"num_keys_recovered":229,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":93646848,"data_stored":685384326,"data_compressed":13545801,"data_compressed_allocated":62189568,"data_compressed_original":659594369,"omap_allocated":0,"internal_metadata":0},"log_size":18356,"ondisk_log_size":18356,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":10404,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509482,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40496,"kb_used_data":9376,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94331344,"statfs":{"total":96636764160,"available":96595296256,"internally_reserved":0,"allocated":9601024,"data_stored":56298740,"data_compressed":1151470,"data_compressed_allocated":5115904,"data_compressed_original":53428827,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.85499999999999998,"15min":0.84199999999999997},"min":{"1min":0.46500000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.872,"5min":0.93300000000000005,"15min":0.95199999999999996},"min":{"1min":0.54900000000000004,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.3839999999999999,"5min":1.917,"15min":1.917},"last":0.48099999999999998}]},{"osd":1,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.81000000000000005,"15min":0.77700000000000002},"min":{"1min":0.35499999999999998,"5min":0.312,"15min":0.312},"max":{"1min":1.2789999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.307},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.83199999999999996,"15min":0.82599999999999996},"min":{"1min":0.57399999999999995,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.208,"5min":2.2149999999999999,"15min":2.2149999999999999},"last":0.40500000000000003}]},{"osd":2,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.82199999999999995,"15min":0.81200000000000006},"min":{"1min":0.50900000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.169,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.93500000000000005,"15min":0.94299999999999995},"min":{"1min":0.55600000000000005,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.355,"5min":1.466,"15min":1.466},"last":0.56499999999999995}]},{"osd":3,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.88500000000000001,"15min":0.86099999999999999},"min":{"1min":0.60999999999999999,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.401,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":0.37},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.97799999999999998,"15min":0.96899999999999997},"min":{"1min":0.68200000000000005,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.3420000000000001,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.54200000000000004}]},{"osd":4,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.92200000000000004,"15min":0.93300000000000005},"min":{"1min":0.42199999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.3089999999999999,"5min":2.7130000000000001,"15min":2.7130000000000001},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.94499999999999995,"15min":0.92600000000000005},"min":{"1min":0.39000000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.425,"5min":2.125,"15min":2.125},"last":0.42499999999999999}]},{"osd":5,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":1.026,"15min":1.022},"min":{"1min":0.51300000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.5800000000000001,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":0.505},{"interface":"front","average":{"1min":0.94299999999999995,"5min":1.0409999999999999,"15min":1.05},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.3029999999999999,"5min":3.1880000000000002,"15min":3.1880000000000002},"last":0.78600000000000003}]},{"osd":6,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.133,"15min":1.149},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.5329999999999999,"5min":3.2730000000000001,"15min":3.2730000000000001},"last":0.52900000000000003},{"interface":"front","average":{"1min":1.095,"5min":1.0629999999999999,"15min":1.03},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.6990000000000001,"5min":2.2879999999999998,"15min":2.2879999999999998},"last":0.46600000000000003}]}]},{"osd":6,"up_from":14,"seq":60129542185,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64140,"kb_used_data":10876,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94307700,"statfs":{"total":96636764160,"available":96571084800,"internally_reserved":0,"allocated":11137024,"data_stored":74424794,"data_compressed":1553596,"data_compressed_allocated":6676480,"data_compressed_original":71733004,"omap_allocated":1593,"internal_metadata":54524359},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:01:08 2024","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.92500000000000004,"15min":0.95099999999999996},"min":{"1min":0.51900000000000002,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.2010000000000001,"5min":1.593,"15min":1.593},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.93300000000000005,"15min":0.94999999999999996},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.226,"5min":1.587,"15min":1.587},"last":0.55900000000000005}]},{"osd":1,"last update":"Sat Sep 21 15:01:08 2024","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.89400000000000002,"15min":0.91000000000000003},"min":{"1min":0.51700000000000002,"5min":0.502,"15min":0.502},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.90200000000000002,"15min":0.96199999999999997},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.70499999999999996}]},{"osd":2,"last update":"Sat Sep 21 15:01:08 2024","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.96199999999999997,"15min":0.98399999999999999},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.389,"5min":1.758,"15min":1.758},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.90900000000000003,"15min":0.91300000000000003},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.444,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.61599999999999999}]},{"osd":3,"last update":"Sat Sep 21 15:01:08 2024","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.94799999999999995,"15min":0.95899999999999996},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.5329999999999999,"5min":1.835,"15min":1.835},"last":0.92300000000000004},{"interface":"front","average":{"1min":1.016,"5min":1.0920000000000001,"15min":1.127},"min":{"1min":0.64600000000000002,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.4670000000000001,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.96099999999999997}]},{"osd":4,"last update":"Sat Sep 21 15:01:08 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.93500000000000005,"15min":0.99099999999999999},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":1.2,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.99099999999999999},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.98499999999999999,"15min":1.0209999999999999},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.3759999999999999,"5min":1.897,"15min":1.897},"last":0.86899999999999999}]},{"osd":5,"last update":"Sat Sep 21 15:01:08 2024","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":1.0449999999999999,"15min":1.1100000000000001},"min":{"1min":0.63500000000000001,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.256,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.63600000000000001},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0660000000000001,"15min":1.087},"min":{"1min":0.65400000000000003,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.609,"5min":1.609,"15min":1.609},"last":0.77400000000000002}]},{"osd":7,"last update":"Sat Sep 21 15:01:08 2024","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.206,"15min":1.244},"min":{"1min":0.78900000000000003,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.3109999999999999,"5min":1.8660000000000001,"15min":1.8660000000000001},"last":0.90000000000000002},{"interface":"front","average":{"1min":1,"5min":1.1970000000000001,"15min":1.2749999999999999},"min":{"1min":0.69499999999999995,"5min":0.63,"15min":0.63},"max":{"1min":1.298,"5min":2.004,"15min":2.004},"last":0.81200000000000006}]}]},{"osd":5,"up_from":14,"seq":60129542186,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65712,"kb_used_data":12452,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94306128,"statfs":{"total":96636764160,"available":96569475072,"internally_reserved":0,"allocated":12750848,"data_stored":73729824,"data_compressed":1254811,"data_compressed_allocated":6623232,"data_compressed_original":69388396,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:01:07 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.89900000000000002,"15min":0.98799999999999999},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.337,"5min":4.4800000000000004,"15min":4.4800000000000004},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.627,"5min":0.76800000000000002,"15min":0.80800000000000005},"min":{"1min":0.36099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.94999999999999996,"5min":1.181,"15min":1.181},"last":0.56999999999999995}]},{"osd":1,"last update":"Sat Sep 21 15:01:02 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.89600000000000002,"15min":0.92400000000000004},"min":{"1min":0.47699999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":2.7709999999999999,"5min":2.7709999999999999,"15min":2.7709999999999999},"last":0.753},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.91400000000000003,"15min":0.93400000000000005},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":2.823,"5min":2.823,"15min":2.823},"last":0.77700000000000002}]},{"osd":2,"last update":"Sat Sep 21 15:01:02 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.91200000000000003,"15min":0.97399999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.159,"5min":3.419,"15min":3.419},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.87,"15min":0.89600000000000002},"min":{"1min":0.53400000000000003,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0780000000000001,"5min":1.423,"15min":1.423},"last":0.58999999999999997}]},{"osd":3,"last update":"Sat Sep 21 15:01:02 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.80300000000000005,"15min":0.80700000000000005},"min":{"1min":0.58499999999999996,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.96699999999999997,"5min":1.054,"15min":1.0840000000000001},"min":{"1min":0.59899999999999998,"5min":0.435,"15min":0.435},"max":{"1min":1.272,"5min":2.6379999999999999,"15min":2.6379999999999999},"last":0.71899999999999997}]},{"osd":4,"last update":"Sat Sep 21 15:01:02 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.98899999999999999,"15min":1.016},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.5609999999999999,"5min":2.5470000000000002,"15min":2.5470000000000002},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.82099999999999995,"5min":1.0169999999999999,"15min":1.0780000000000001},"min":{"1min":0.39300000000000002,"5min":0.376,"15min":0.376},"max":{"1min":1.302,"5min":2.5099999999999998,"15min":2.5099999999999998},"last":0.46700000000000003}]},{"osd":6,"last update":"Sat Sep 21 15:01:02 2024","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.94299999999999995,"15min":0.96899999999999997},"min":{"1min":0.63,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":1.004,"5min":1.2450000000000001,"15min":1.3520000000000001},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.425,"5min":4.2469999999999999,"15min":4.2469999999999999},"last":0.84499999999999997}]},{"osd":7,"last update":"Sat Sep 21 15:01:07 2024","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.2569999999999999,"15min":1.3420000000000001},"min":{"1min":0.52500000000000002,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.383,"5min":3.3010000000000002,"15min":3.3010000000000002},"last":0.79300000000000004},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.107,"15min":1.135},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.462,"5min":2.5880000000000001,"15min":2.5880000000000001},"last":0.64300000000000002}]}]},{"osd":4,"up_from":14,"seq":60129542185,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68696,"kb_used_data":11344,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94303144,"statfs":{"total":96636764160,"available":96566419456,"internally_reserved":0,"allocated":11616256,"data_stored":80720875,"data_compressed":1548190,"data_compressed_allocated":7086080,"data_compressed_original":77864961,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.76000000000000001,"15min":0.77600000000000002},"min":{"1min":0.47499999999999998,"5min":0.311,"15min":0.311},"max":{"1min":1.1020000000000001,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.98899999999999999},{"interface":"front","average":{"1min":1.5860000000000001,"5min":1.0489999999999999,"15min":0.98099999999999998},"min":{"1min":0.49299999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":13.324,"5min":13.324,"15min":13.324},"last":1.2330000000000001}]},{"osd":1,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.85899999999999999,"15min":0.90600000000000003},"min":{"1min":0.46800000000000003,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.0609999999999999,"5min":1.347,"15min":1.347},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.82199999999999995,"15min":0.83899999999999997},"min":{"1min":0.502,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.92700000000000005}]},{"osd":2,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.88200000000000001,"15min":0.88},"min":{"1min":0.55100000000000005,"5min":0.501,"15min":0.501},"max":{"1min":1.47,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":1.1379999999999999},{"interface":"front","average":{"1min":0.98099999999999998,"5min":1.002,"15min":1.026},"min":{"1min":0.60699999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.3740000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.872}]},{"osd":3,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.90700000000000003,"15min":0.89500000000000002},"min":{"1min":0.61799999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.482,"5min":1.538,"15min":1.538},"last":0.76700000000000002},{"interface":"front","average":{"1min":1.024,"5min":1.0129999999999999,"15min":0.99199999999999999},"min":{"1min":0.56599999999999995,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.528,"5min":1.528,"15min":1.528},"last":1.0309999999999999}]},{"osd":5,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.97499999999999998,"15min":0.98399999999999999},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.579,"5min":1.579,"15min":1.579},"last":1.1899999999999999},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.94799999999999995,"15min":0.96399999999999997},"min":{"1min":0.61899999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":1.1619999999999999}]},{"osd":6,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.996,"15min":1.0109999999999999},"min":{"1min":0.55400000000000005,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.349,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.081,"15min":1.0900000000000001},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.665,"5min":1.665,"15min":1.665},"last":1.274}]},{"osd":7,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.995,"15min":1.0249999999999999},"min":{"1min":0.45200000000000001,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.226,"5min":1.7010000000000001,"15min":1.7010000000000001},"last":1.091},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.2050000000000001,"15min":1.2130000000000001},"min":{"1min":0.627,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":1.321}]}]},{"osd":3,"up_from":14,"seq":60129542186,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59424,"kb_used_data":14356,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94312416,"statfs":{"total":96636764160,"available":96575913984,"internally_reserved":0,"allocated":14700544,"data_stored":103127883,"data_compressed":2074542,"data_compressed_allocated":9273344,"data_compressed_original":99696219,"omap_allocated":1592,"internal_metadata":46135752},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:01:09 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.625,"15min":0.63400000000000001},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.171,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.68200000000000005,"15min":0.70999999999999996},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.0700000000000001,"5min":1.919,"15min":1.919},"last":0.29199999999999998}]},{"osd":1,"last update":"Sat Sep 21 15:01:09 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.66400000000000003,"15min":0.67200000000000004},"min":{"1min":0.29499999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":1.722,"5min":1.722,"15min":1.722},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.76500000000000001,"15min":0.80300000000000005},"min":{"1min":0.33300000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.89500000000000002,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.33400000000000002}]},{"osd":2,"last update":"Sat Sep 21 15:01:09 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.71999999999999997,"15min":0.72199999999999998},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.92400000000000004,"5min":1.157,"15min":1.157},"last":0.311},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.81000000000000005,"15min":0.82699999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.1539999999999999,"5min":1.254,"15min":1.254},"last":0.39400000000000002}]},{"osd":4,"last update":"Sat Sep 21 15:01:09 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.93799999999999994,"15min":0.95599999999999996},"min":{"1min":0.54300000000000004,"5min":0.442,"15min":0.442},"max":{"1min":1.2629999999999999,"5min":1.7609999999999999,"15min":1.7609999999999999},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.98899999999999999,"15min":1.0269999999999999},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.234,"5min":2.028,"15min":2.028},"last":0.73999999999999999}]},{"osd":5,"last update":"Sat Sep 21 15:01:09 2024","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":1.0369999999999999,"15min":1.052},"min":{"1min":0.70499999999999996,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.3440000000000001,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":0.76400000000000001},{"interface":"front","average":{"1min":1.0249999999999999,"5min":1.056,"15min":1.0629999999999999},"min":{"1min":0.57999999999999996,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.548,"5min":1.548,"15min":1.548},"last":0.82499999999999996}]},{"osd":6,"last update":"Sat Sep 21 15:01:09 2024","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":1.0669999999999999,"15min":1.0920000000000001},"min":{"1min":0.59099999999999997,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.6419999999999999,"5min":2.1219999999999999,"15min":2.1219999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.091,"15min":1.103},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.6839999999999999,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":0.48799999999999999}]},{"osd":7,"last update":"Sat Sep 21 15:01:09 2024","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":1.103,"15min":1.131},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.498,"5min":1.706,"15min":1.706},"last":0.55900000000000005},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.147,"15min":1.177},"min":{"1min":0.59699999999999998,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.417,"5min":2.3380000000000001,"15min":2.3380000000000001},"last":0.78600000000000003}]}]},{"osd":2,"up_from":14,"seq":60129542184,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72532,"kb_used_data":15172,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94299308,"statfs":{"total":96636764160,"available":96562491392,"internally_reserved":0,"allocated":15536128,"data_stored":98629584,"data_compressed":1955612,"data_compressed_allocated":8757248,"data_compressed_original":93754402,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.62},"min":{"1min":0.26900000000000002,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.92800000000000005,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":1.383},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.69399999999999995,"15min":0.67000000000000004},"min":{"1min":0.34799999999999998,"5min":0.223,"15min":0.223},"max":{"1min":2.9169999999999998,"5min":2.9169999999999998,"15min":2.9169999999999998},"last":0.42899999999999999}]},{"osd":1,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.32700000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.0529999999999999,"5min":2.6520000000000001,"15min":2.6520000000000001},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.746,"5min":0.73499999999999999,"15min":0.75800000000000001},"min":{"1min":0.40500000000000003,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.349,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":1.4550000000000001}]},{"osd":3,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.73999999999999999,"15min":0.73499999999999999},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":3.0110000000000001,"5min":3.0110000000000001,"15min":3.0110000000000001},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.877,"15min":0.83899999999999997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.05,"5min":4.1440000000000001,"15min":4.1440000000000001},"last":0.86399999999999999}]},{"osd":4,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.93200000000000005,"15min":0.95599999999999996},"min":{"1min":0.42399999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.222,"5min":2.899,"15min":2.899},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.94399999999999995,"15min":0.95799999999999996},"min":{"1min":0.47499999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.4379999999999999,"5min":1.675,"15min":1.675},"last":1.5489999999999999}]},{"osd":5,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.0129999999999999,"15min":1.0089999999999999},"min":{"1min":0.45500000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":3.1080000000000001,"5min":3.1080000000000001,"15min":3.1080000000000001},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.98199999999999998,"15min":0.99399999999999999},"min":{"1min":0.51700000000000002,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.268,"5min":1.546,"15min":1.546},"last":0.55200000000000005}]},{"osd":6,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":0.98299999999999998,"15min":0.98499999999999999},"min":{"1min":0.56200000000000006,"5min":0.378,"15min":0.378},"max":{"1min":1.345,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":1.516},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.0649999999999999,"15min":1.0549999999999999},"min":{"1min":0.53200000000000003,"5min":0.502,"15min":0.502},"max":{"1min":3.0739999999999998,"5min":3.0739999999999998,"15min":3.0739999999999998},"last":0.48699999999999999}]},{"osd":7,"last update":"Sat Sep 21 15:01:08 2024","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.0449999999999999,"15min":1.0089999999999999},"min":{"1min":0.59499999999999997,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":3.1589999999999998,"5min":3.1589999999999998,"15min":3.1589999999999998},"last":0.61399999999999999},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.121,"15min":1.099},"min":{"1min":0.66000000000000003,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":3.1400000000000001,"5min":3.1400000000000001,"15min":3.1400000000000001},"last":0.58799999999999997}]}]},{"osd":1,"up_from":14,"seq":60129542185,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":71084,"kb_used_data":13732,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94300756,"statfs":{"total":96636764160,"available":96563974144,"internally_reserved":0,"allocated":14061568,"data_stored":96758848,"data_compressed":1883265,"data_compressed_allocated":8507392,"data_compressed_original":93072883,"omap_allocated":1595,"internal_metadata":58718661},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.59599999999999997,"15min":0.60899999999999999},"min":{"1min":0.36399999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.78200000000000003,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59699999999999998,"15min":0.60799999999999998},"min":{"1min":0.39500000000000002,"5min":0.216,"15min":0.216},"max":{"1min":0.80100000000000005,"5min":1.107,"15min":1.107},"last":0.57399999999999995}]},{"osd":2,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":0.79600000000000004,"15min":0.73999999999999999},"min":{"1min":0.38700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":11.303000000000001,"5min":11.303000000000001,"15min":11.303000000000001},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.71299999999999997,"15min":0.70299999999999996},"min":{"1min":0.47299999999999998,"5min":0.435,"15min":0.435},"max":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.74099999999999999}]},{"osd":3,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.81799999999999995,"15min":0.83899999999999997},"min":{"1min":0.27800000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.0049999999999999,"5min":3.1469999999999998,"15min":3.1469999999999998},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.86399999999999999,"15min":0.88600000000000001},"min":{"1min":0.52800000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.306,"5min":3.2320000000000002,"15min":3.2320000000000002},"last":0.86599999999999999}]},{"osd":4,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.89000000000000001,"15min":0.91500000000000004},"min":{"1min":0.59599999999999997,"5min":0.309,"15min":0.309},"max":{"1min":1.45,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":0.68200000000000005},{"interface":"front","average":{"1min":1.5600000000000001,"5min":1.069,"15min":0.96599999999999997},"min":{"1min":0.504,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":11.563000000000001,"5min":11.563000000000001,"15min":11.563000000000001},"last":1.0580000000000001}]},{"osd":5,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":1.625,"5min":1.0469999999999999,"15min":0.94899999999999995},"min":{"1min":0.73399999999999999,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":11.534000000000001,"5min":11.534000000000001,"15min":11.534000000000001},"last":1.1599999999999999},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95799999999999996,"15min":0.96099999999999997},"min":{"1min":0.63,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.256,"5min":1.484,"15min":1.484},"last":0.91500000000000004}]},{"osd":6,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":1.0109999999999999,"15min":1.0489999999999999},"min":{"1min":0.60399999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.4059999999999999,"5min":2.3519999999999999,"15min":2.3519999999999999},"last":0.84699999999999998},{"interface":"front","average":{"1min":1.631,"5min":1.081,"15min":0.99299999999999999},"min":{"1min":0.57399999999999995,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":11.601000000000001,"5min":11.601000000000001,"15min":11.601000000000001},"last":1.117}]},{"osd":7,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":1.046,"15min":1.0640000000000001},"min":{"1min":0.79300000000000004,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.5820000000000001,"5min":2.2360000000000002,"15min":2.2360000000000002},"last":0.96299999999999997},{"interface":"front","average":{"1min":1.76,"5min":1.171,"15min":1.087},"min":{"1min":0.877,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":11.489000000000001,"5min":11.489000000000001,"15min":11.489000000000001},"last":1.2010000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542184,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":71592,"kb_used_data":14232,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94300248,"statfs":{"total":96636764160,"available":96563453952,"internally_reserved":0,"allocated":14573568,"data_stored":103321822,"data_compressed":2127509,"data_compressed_allocated":9379840,"data_compressed_original":100115005,"omap_allocated":1593,"internal_metadata":58718663},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 21 15:01:04 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.67500000000000004,"15min":0.71699999999999997},"min":{"1min":0.40999999999999998,"5min":0.309,"15min":0.309},"max":{"1min":1.1299999999999999,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":1.002},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.68100000000000005,"15min":0.627},"min":{"1min":0.58999999999999997,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.88100000000000001}]},{"osd":2,"last update":"Sat Sep 21 15:01:04 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.65200000000000002,"15min":0.61799999999999999},"min":{"1min":0.59599999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.72499999999999998,"15min":0.75800000000000001},"min":{"1min":0.39300000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.28799999999999998}]},{"osd":3,"last update":"Sat Sep 21 15:01:04 2024","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.76600000000000001,"15min":0.754},"min":{"1min":0.50900000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.78700000000000003,"15min":0.81499999999999995},"min":{"1min":0.50900000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.58599999999999997}]},{"osd":4,"last update":"Sat Sep 21 15:01:04 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.98399999999999999,"15min":1.038},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.1140000000000001,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":0.95199999999999996},{"interface":"front","average":{"1min":1.036,"5min":1.0469999999999999,"15min":1.0620000000000001},"min":{"1min":0.67800000000000005,"5min":0.379,"15min":0.379},"max":{"1min":1.446,"5min":1.665,"15min":1.665},"last":0.44}]},{"osd":5,"last update":"Sat Sep 21 15:01:04 2024","interfaces":[{"interface":"back","average":{"1min":1.0960000000000001,"5min":1.024,"15min":1.032},"min":{"1min":0.79300000000000004,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.4450000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.45800000000000002},{"interface":"front","average":{"1min":1.0589999999999999,"5min":1.0429999999999999,"15min":1.077},"min":{"1min":0.82399999999999995,"5min":0.435,"15min":0.435},"max":{"1min":1.2609999999999999,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":0.378}]},{"osd":6,"last update":"Sat Sep 21 15:01:04 2024","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":1.0580000000000001,"15min":1.1120000000000001},"min":{"1min":0.68400000000000005,"5min":0.628,"15min":0.628},"max":{"1min":1.167,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":0.89400000000000002},{"interface":"front","average":{"1min":1.109,"5min":1.0629999999999999,"15min":1.0680000000000001},"min":{"1min":0.91000000000000003,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.5029999999999999,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.47399999999999998}]},{"osd":7,"last update":"Sat Sep 21 15:01:04 2024","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.0469999999999999,"15min":1.0620000000000001},"min":{"1min":0.78700000000000003,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.4770000000000001,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":0.34399999999999997},{"interface":"front","average":{"1min":1.02,"5min":1.085,"15min":1.131},"min":{"1min":0.84799999999999998,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.403,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":0.92600000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5202,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5202,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":14073856,"data_stored":104108982,"data_compressed":2129014,"data_compressed_allocated":9555968,"data_compressed_original":100467261,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12804096,"data_stored":96214214,"data_compressed":1879183,"data_compressed_allocated":8601600,"data_compressed_original":92761587,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":13369344,"data_stored":97577590,"data_compressed":1955612,"data_compressed_allocated":8757248,"data_compressed_original":93754402,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":15351808,"data_stored":105270750,"data_compressed":2077727,"data_compressed_allocated":9646080,"data_compressed_original":100441691,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":10043392,"data_stored":80360275,"data_compressed":1549030,"data_compressed_allocated":7184384,"data_compressed_original":78061569,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":10526720,"data_stored":72087462,"data_compressed":1249609,"data_compressed_allocated":6586368,"data_compressed_original":68814956,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":10055680,"data_stored":74524637,"data_compressed":1554156,"data_compressed_allocated":6742016,"data_compressed_original":71864076,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":7421952,"data_stored":55240416,"data_compressed":1151470,"data_compressed_allocated":5115904,"data_compressed_original":53428827,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T15:01:15.265 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-21T15:01:15.266 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T15:01:15.266 INFO:tasks.thrashosds.thrasher:inject_pause on osd.4 2024-09-21T15:01:15.266 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-21T15:01:15.266 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-21T15:01:15.266 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok config set filestore_inject_stall 3 2024-09-21T15:01:15.305 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:15.305 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:15.315 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T15:01:15.320 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:01:15.317+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:01:15.373 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:01:15.373 INFO:teuthology.orchestra.run.smithi046.stdout: "success": "" 2024-09-21T15:01:15.373 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:01:15.397 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T15:01:15.416 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:15.416 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:15.421 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:01:15.417+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:01:15.425 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T15:01:15.522 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:01:15.518+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:01:15.549 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T15:01:15.573 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T15:01:15.575 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:15.575 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:15.584 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T15:01:15.601 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T15:01:15.620 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T15:01:15.623 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:01:15.621+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:01:15.640 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-21T15:01:15.662 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T15:01:15.685 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T15:01:15.695 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:15.695 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:15.705 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T15:01:15.709 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T15:01:15.712 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:01:15.712 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:01:15.712 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:01:15.713 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:01:15.713 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:01:15.713 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:01:15.713 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:01:15.713 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:01:15.724 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:01:15.718+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:01:15.810 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:01:15.810 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T15:01:15.811 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T15:01:15.811 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T15:01:15.811 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.811 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31476 3.c 3:35718b6b:::rbd_data.11367721c19.0000000000000014:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T15:01:15.811 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.505975+0000", 2024-09-21T15:01:15.811 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 137.30186166499999, 2024-09-21T15:01:15.811 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99515995899999998, 2024-09-21T15:01:15.811 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:15.811 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:15.811 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:15.811 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:15.811 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:15.811 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31476 2024-09-21T15:01:15.811 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.811 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:15.812 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.812 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:15.812 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.505975+0000", 2024-09-21T15:01:15.812 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:15.812 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.812 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.812 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:15.812 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.505975+0000", 2024-09-21T15:01:15.812 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:15.812 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.812 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.812 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:15.812 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.505971+0000", 2024-09-21T15:01:15.812 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999967 2024-09-21T15:01:15.813 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.813 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.813 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:15.813 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.505989+0000", 2024-09-21T15:01:15.813 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7985000000000001e-05 2024-09-21T15:01:15.813 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.813 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.813 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:15.813 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506050+0000", 2024-09-21T15:01:15.813 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0659000000000003e-05 2024-09-21T15:01:15.813 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.813 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.813 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:15.813 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506074+0000", 2024-09-21T15:01:15.813 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4156e-05 2024-09-21T15:01:15.814 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.814 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.814 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:15.814 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506225+0000", 2024-09-21T15:01:15.814 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015097 2024-09-21T15:01:15.814 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.814 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.814 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:15.814 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506256+0000", 2024-09-21T15:01:15.814 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0505999999999999e-05 2024-09-21T15:01:15.814 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.814 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.814 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:15.814 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.497974+0000", 2024-09-21T15:01:15.815 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99171825700000005 2024-09-21T15:01:15.815 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.815 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.815 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for missing object", 2024-09-21T15:01:15.815 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.498449+0000", 2024-09-21T15:01:15.815 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00047506399999999998 2024-09-21T15:01:15.815 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.815 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.815 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:15.815 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.501000+0000", 2024-09-21T15:01:15.815 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0025506919999999998 2024-09-21T15:01:15.815 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.815 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.815 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:15.815 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.501135+0000", 2024-09-21T15:01:15.816 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013524000000000001 2024-09-21T15:01:15.816 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:15.816 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:15.816 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:15.816 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.816 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.816 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31480 3.c 3:36ccd808:::rbd_data.11367721c19.0000000000000018:head [sparse-read 0~14821] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T15:01:15.816 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.506116+0000", 2024-09-21T15:01:15.816 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 137.30172032199999, 2024-09-21T15:01:15.816 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99316033599999998, 2024-09-21T15:01:15.816 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:15.816 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:15.816 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:15.816 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:15.816 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:15.817 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31480 2024-09-21T15:01:15.817 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.817 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:15.817 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.817 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:15.817 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506116+0000", 2024-09-21T15:01:15.817 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:15.817 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.817 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.817 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:15.817 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506116+0000", 2024-09-21T15:01:15.817 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:15.817 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.817 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.817 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:15.818 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506115+0000", 2024-09-21T15:01:15.818 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:15.818 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.818 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.818 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:15.818 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506124+0000", 2024-09-21T15:01:15.818 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0659999999999993e-06 2024-09-21T15:01:15.818 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.818 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.818 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:15.818 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506145+0000", 2024-09-21T15:01:15.818 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1447e-05 2024-09-21T15:01:15.818 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.818 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.818 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:15.819 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506161+0000", 2024-09-21T15:01:15.819 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5209e-05 2024-09-21T15:01:15.819 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.819 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.819 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:15.819 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506337+0000", 2024-09-21T15:01:15.819 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017642999999999999 2024-09-21T15:01:15.819 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.819 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.819 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:15.819 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506375+0000", 2024-09-21T15:01:15.819 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7536000000000001e-05 2024-09-21T15:01:15.819 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.819 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.819 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:15.820 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.498815+0000", 2024-09-21T15:01:15.820 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99244067999999996 2024-09-21T15:01:15.820 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.820 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.820 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:15.820 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.498950+0000", 2024-09-21T15:01:15.820 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013436700000000001 2024-09-21T15:01:15.820 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.820 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.820 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:15.820 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499277+0000", 2024-09-21T15:01:15.820 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00032696800000000002 2024-09-21T15:01:15.820 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:15.820 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:15.820 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:15.821 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.821 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.821 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31513 3.c 3:36de8523:::rbd_data.11367721c19.0000000000000039:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T15:01:15.821 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.509071+0000", 2024-09-21T15:01:15.821 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 137.29876612199999, 2024-09-21T15:01:15.821 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99038225099999999, 2024-09-21T15:01:15.821 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:15.821 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:15.821 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:15.821 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:15.821 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:15.821 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31513 2024-09-21T15:01:15.821 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.821 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:15.821 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.822 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:15.822 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509071+0000", 2024-09-21T15:01:15.822 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:15.822 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.822 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.822 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:15.822 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509071+0000", 2024-09-21T15:01:15.822 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:15.822 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.822 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.822 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:15.822 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509068+0000", 2024-09-21T15:01:15.822 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-21T15:01:15.822 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.822 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.823 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:15.823 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509082+0000", 2024-09-21T15:01:15.823 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3545e-05 2024-09-21T15:01:15.823 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.823 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.823 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:15.823 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509116+0000", 2024-09-21T15:01:15.823 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3942000000000001e-05 2024-09-21T15:01:15.823 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.823 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.823 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:15.823 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509134+0000", 2024-09-21T15:01:15.823 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8839999999999999e-05 2024-09-21T15:01:15.823 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.824 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.824 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:15.824 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509318+0000", 2024-09-21T15:01:15.824 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018326799999999999 2024-09-21T15:01:15.824 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.824 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.824 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:15.824 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509346+0000", 2024-09-21T15:01:15.824 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8019000000000001e-05 2024-09-21T15:01:15.824 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.824 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.824 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:15.824 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499330+0000", 2024-09-21T15:01:15.824 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.98998399299999995 2024-09-21T15:01:15.824 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.824 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.825 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:15.825 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499453+0000", 2024-09-21T15:01:15.825 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000123115 2024-09-21T15:01:15.825 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:15.825 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:15.825 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:15.825 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.825 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.825 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31515 3.c 3:35b36ddd:::rbd_data.11367721c19.000000000000003b:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T15:01:15.825 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.509177+0000", 2024-09-21T15:01:15.825 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 137.29865969599999, 2024-09-21T15:01:15.825 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99044870600000001, 2024-09-21T15:01:15.825 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:15.825 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:15.825 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:15.825 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:15.826 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:15.826 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31515 2024-09-21T15:01:15.826 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.826 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:15.826 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.826 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:15.826 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509177+0000", 2024-09-21T15:01:15.826 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:15.826 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.826 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.826 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:15.826 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509177+0000", 2024-09-21T15:01:15.826 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:15.826 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.827 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.827 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:15.827 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509176+0000", 2024-09-21T15:01:15.827 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:15.827 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.827 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.827 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:15.827 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509183+0000", 2024-09-21T15:01:15.827 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6720000000000004e-06 2024-09-21T15:01:15.827 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.827 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.827 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:15.827 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509269+0000", 2024-09-21T15:01:15.827 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5881e-05 2024-09-21T15:01:15.827 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.828 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.828 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:15.828 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509303+0000", 2024-09-21T15:01:15.828 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3367000000000001e-05 2024-09-21T15:01:15.828 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.828 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.828 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:15.828 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509425+0000", 2024-09-21T15:01:15.828 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000122199 2024-09-21T15:01:15.828 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.828 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.828 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:15.828 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509448+0000", 2024-09-21T15:01:15.828 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2688999999999999e-05 2024-09-21T15:01:15.828 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.829 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.829 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:15.829 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499501+0000", 2024-09-21T15:01:15.829 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99005306999999998 2024-09-21T15:01:15.829 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.829 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.829 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:15.829 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499626+0000", 2024-09-21T15:01:15.829 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012501400000000001 2024-09-21T15:01:15.829 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:15.829 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:15.829 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:15.829 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.829 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.829 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31531 3.c 3:314f0287:::rbd_data.11367721c19.000000000000004b:head [sparse-read 0~7106] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T15:01:15.830 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.509329+0000", 2024-09-21T15:01:15.830 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 137.29850767600001, 2024-09-21T15:01:15.830 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99103085300000004, 2024-09-21T15:01:15.830 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:15.830 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:15.830 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:15.830 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:15.830 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:15.830 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31531 2024-09-21T15:01:15.830 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.830 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:15.830 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.830 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:15.830 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509329+0000", 2024-09-21T15:01:15.830 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:15.831 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.831 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.831 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:15.831 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509329+0000", 2024-09-21T15:01:15.831 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:15.831 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.831 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.831 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:15.831 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509328+0000", 2024-09-21T15:01:15.831 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:15.831 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.831 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.831 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:15.831 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509336+0000", 2024-09-21T15:01:15.831 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3170000000000002e-06 2024-09-21T15:01:15.832 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.832 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.832 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:15.832 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509353+0000", 2024-09-21T15:01:15.832 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7214000000000002e-05 2024-09-21T15:01:15.832 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.832 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.832 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:15.832 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509366+0000", 2024-09-21T15:01:15.832 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.26e-05 2024-09-21T15:01:15.832 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.832 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.832 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:15.832 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509523+0000", 2024-09-21T15:01:15.832 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015708000000000001 2024-09-21T15:01:15.832 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.833 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.833 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:15.833 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509544+0000", 2024-09-21T15:01:15.833 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0783000000000001e-05 2024-09-21T15:01:15.833 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.833 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.833 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:15.833 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499727+0000", 2024-09-21T15:01:15.833 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99018345200000002 2024-09-21T15:01:15.833 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.833 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.833 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:15.833 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499949+0000", 2024-09-21T15:01:15.833 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000221358 2024-09-21T15:01:15.833 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.833 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.834 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:15.834 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.500360+0000", 2024-09-21T15:01:15.834 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00041130700000000002 2024-09-21T15:01:15.834 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:15.834 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:15.834 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:15.834 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.834 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.834 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:82545 3.b 3:d4eb1ee5:::rbd_data.11368296f791.0000000000000000:head [write 0~671744 [fadvise_sequential+fadvise_nocache],write 704512~65536 [fadvise_sequential+fadvise_nocache],write 786432~49152 [fadvise_sequential+fadvise_nocache],write 868352~49152 [fadvise_sequential+fadvise_nocache],write 933888~65536 [fadvise_sequential+fadvise_nocache],write 1032192~32768 [fadvise_sequential+fadvise_nocache],write 1097728~65536 [fadvise_sequential+fadvise_nocache],write 1179648~49152 [fadvise_sequential+fadvise_nocache],write 1261568~49152 [fadvise_sequential+fadvise_nocache],write 1327104~65536 [fadvise_sequential+fadvise_nocache],write 1425408~32768 [fadvise_sequential+fadvise_nocache],write 1490944~65536 [fadvise_sequential+fadvise_nocache],write 1572864~8192 [fadvise_sequential+fadvise_nocache],write 1630208~49152 [fadvise_sequential+fadvise_nocache],write 1703936~40960 [fadvise_sequential+fadvise_nocache],write 1794048~40960 [fadvise_sequential+fadvise_nocache],write 1859584~49152 [fadvise_sequential+fadvise_nocache],write 1957888~8192 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:01:15.834 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:47.416089+0000", 2024-09-21T15:01:15.834 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 88.391747950999999, 2024-09-21T15:01:15.834 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011896455, 2024-09-21T15:01:15.834 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:15.834 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:15.834 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:15.835 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:15.835 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:15.835 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82545 2024-09-21T15:01:15.835 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.835 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:15.835 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.835 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:15.835 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.416089+0000", 2024-09-21T15:01:15.835 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:15.835 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.835 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.835 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:15.835 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.416089+0000", 2024-09-21T15:01:15.835 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:15.836 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.836 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.836 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:15.836 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.416087+0000", 2024-09-21T15:01:15.836 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:01:15.836 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.836 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.836 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:15.836 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.419643+0000", 2024-09-21T15:01:15.836 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0035555550000000002 2024-09-21T15:01:15.836 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.836 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.836 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:15.836 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.419663+0000", 2024-09-21T15:01:15.836 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0202000000000001e-05 2024-09-21T15:01:15.837 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.837 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.837 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:15.837 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.419676+0000", 2024-09-21T15:01:15.837 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3207000000000001e-05 2024-09-21T15:01:15.837 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.837 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.837 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:15.837 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.419759+0000", 2024-09-21T15:01:15.837 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3521000000000002e-05 2024-09-21T15:01:15.837 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.837 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.837 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op must wait for map", 2024-09-21T15:01:15.837 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.419768+0000", 2024-09-21T15:01:15.837 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4479999999999994e-06 2024-09-21T15:01:15.838 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.838 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.838 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:15.838 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.422171+0000", 2024-09-21T15:01:15.838 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0024035469999999998 2024-09-21T15:01:15.838 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.838 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.838 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:15.838 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.422350+0000", 2024-09-21T15:01:15.838 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000179001 2024-09-21T15:01:15.838 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.838 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.838 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T15:01:15.838 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.422710+0000", 2024-09-21T15:01:15.839 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000359878 2024-09-21T15:01:15.839 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.839 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.839 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:01:15.839 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.426676+0000", 2024-09-21T15:01:15.839 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0039655790000000003 2024-09-21T15:01:15.839 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.839 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.839 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:01:15.839 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.427884+0000", 2024-09-21T15:01:15.839 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001208207 2024-09-21T15:01:15.839 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.839 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.839 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:01:15.839 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.427953+0000", 2024-09-21T15:01:15.840 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8533000000000003e-05 2024-09-21T15:01:15.840 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.840 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.840 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:15.840 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.427985+0000", 2024-09-21T15:01:15.840 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2481999999999998e-05 2024-09-21T15:01:15.840 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:15.840 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:15.840 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:15.840 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.840 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.840 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:97675 3.b 3:d321acb8:::rbd_data.1136bfc79a70.0000000000000003:head [write 0~4718592 [fadvise_sequential+fadvise_nocache],write 5242880~147456 [fadvise_sequential+fadvise_nocache],write 5455872~147456 [fadvise_sequential+fadvise_nocache],write 5668864~229376 [fadvise_sequential+fadvise_nocache],write 5963776~147456 [fadvise_sequential+fadvise_nocache],write 6176768~229376 [fadvise_sequential+fadvise_nocache],write 6471680~147456 [fadvise_sequential+fadvise_nocache],write 6684672~229376 [fadvise_sequential+fadvise_nocache],write 6979584~147456 [fadvise_sequential+fadvise_nocache],write 7192576~229376 [fadvise_sequential+fadvise_nocache],write 7487488~147456 [fadvise_sequential+fadvise_nocache],write 7700480~147456 [fadvise_sequential+fadvise_nocache],write 7864320~65536 [fadvise_sequential+fadvise_nocache],write 7995392~147456 [fadvise_sequential+fadvise_nocache],write 8208384~147456 [fadvise_sequential+fadvise_nocache],write 8388608~49152 [fadvise_sequential+fadvise_nocache],write 8536064~114688 [fadvise_sequential+fadvise_nocache],write 8749056~114688 [fadvise_sequential+fadvise_nocache],write 8912896~32768 [fadvise_sequential+fadvise_nocache],write 9125888~32768 [fadvise_sequential+fadvise_nocache],write 9338880~32768 [fadvise_sequential+fadvise_nocache],write 9437184~16384 [fadvise_sequential+fadvise_nocache],write 9633792~32768 [fadvise_sequential+fadvise_nocache],write 9846784~32768 [fadvise_sequential+fadvise_nocache],write 9961472~114688 [fadvise_sequential+fadvise_nocache],write 10092544~196608 [fadvise_sequential+fadvise_nocache],write 10305536~278528 [fadvise_sequential+fadvise_nocache],write 10600448~196608 [fadvise_sequential+fadvise_nocache],write 10813440~278528 [fadvise_sequential+fadvise_nocache],write 11108352~196608 [fadvise_sequential+fadvise_nocache],write 11321344~196608 [fadvise_sequential+fadvise_nocache],write 11534336~65536 [fadvise_sequential+fadvise_nocache],write 11632640~98304 [fadvise_sequential+fadvise_nocache],write 11747328~65536 [fadvise_sequential+fadvise_nocache],write 11845632~98304 [fadvise_sequential+fadvise_nocache],write 11960320~65536 [fadvise_sequential+fadvise_nocache],write 12058624~49152 [fadvise_sequential+fadvise_nocache],write 12140544~98304 [fadvise_sequential+fadvise_nocache],write 12255232~65536 [fadvise_sequential+fadvise_nocache],write 12353536~98304 [fadvise_sequential+fadvise_nocache],write 12468224~65536 [fadvise_sequential+fadvise_nocache],write 12566528~16384 [fadvise_sequential+fadvise_nocache],write 12648448~98304 [fadvise_sequential+fadvise_nocache],write 12861440~98304 [fadvise_sequential+fadvise_nocache],write 13074432~32768 [fadvise_sequential+fadvise_nocache],write 13156352~98304 [fadvise_sequential+fadvise_nocache],write 13369344~98304 [fadvise_sequential+fadvise_nocache],write 13582336~49152 [fadvise_sequential+fadvise_nocache],write 13664256~98304 [fadvise_sequential+fadvise_nocache],write 13877248~98304 [fadvise_sequential+fadvise_nocache],write 14090240~65536 [fadvise_sequential+fadvise_nocache],write 15286272~81920 [fadvise_sequential+fadvise_nocache],write 15499264~81920 [fadvise_sequential+fadvise_nocache],write 15712256~16384 [fadvise_sequential+fadvise_nocache],write 15794176~81920 [fadvise_sequential+fadvise_nocache],write 16007168~81920 [fadvise_sequential+fadvise_nocache],write 16220160~32768 [fadvise_sequential+fadvise_nocache],write 16302080~81920 [fadvise_sequential+fadvise_nocache],write 16515072~81920 [fadvise_sequential+fadvise_nocache],write 16728064~49152 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:01:15.840 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.302412+0000", 2024-09-21T15:01:15.840 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 70.505425161999995, 2024-09-21T15:01:15.841 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.067380532000000007, 2024-09-21T15:01:15.841 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:15.841 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:15.841 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:15.841 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:15.841 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:15.841 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97675 2024-09-21T15:01:15.841 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.841 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:15.841 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.841 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:15.841 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.302412+0000", 2024-09-21T15:01:15.841 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:15.841 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.842 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.842 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:15.842 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.302412+0000", 2024-09-21T15:01:15.842 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:15.842 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.842 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.842 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:15.842 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.302409+0000", 2024-09-21T15:01:15.842 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-21T15:01:15.842 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.842 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.842 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:15.842 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328550+0000", 2024-09-21T15:01:15.842 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.026140674999999999 2024-09-21T15:01:15.843 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.843 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.843 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:15.843 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328584+0000", 2024-09-21T15:01:15.843 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4165999999999998e-05 2024-09-21T15:01:15.843 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.843 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.843 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:15.843 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328599+0000", 2024-09-21T15:01:15.843 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5505000000000001e-05 2024-09-21T15:01:15.843 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.843 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.843 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:15.843 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328704+0000", 2024-09-21T15:01:15.844 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010487199999999999 2024-09-21T15:01:15.844 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.844 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.844 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:15.844 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.329082+0000", 2024-09-21T15:01:15.844 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00037804899999999998 2024-09-21T15:01:15.844 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.844 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.844 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T15:01:15.844 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.331158+0000", 2024-09-21T15:01:15.844 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0020755449999999998 2024-09-21T15:01:15.844 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.844 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.844 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:01:15.844 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.360553+0000", 2024-09-21T15:01:15.845 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.029395553000000001 2024-09-21T15:01:15.845 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.845 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.845 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:01:15.845 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.369496+0000", 2024-09-21T15:01:15.845 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0089428619999999993 2024-09-21T15:01:15.845 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.845 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.845 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:01:15.845 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.369686+0000", 2024-09-21T15:01:15.845 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019000399999999999 2024-09-21T15:01:15.845 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.845 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.845 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:15.845 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.369792+0000", 2024-09-21T15:01:15.845 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000105775 2024-09-21T15:01:15.846 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:15.846 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:15.846 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:15.846 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.846 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.846 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:97693 3.b 3:d7d7b39b:::rbd_data.1136b9c49b1c.000000000000000c:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:01:15.846 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.328643+0000", 2024-09-21T15:01:15.846 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 70.479193375999998, 2024-09-21T15:01:15.846 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.031062136000000001, 2024-09-21T15:01:15.846 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:15.846 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:15.846 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:15.846 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:15.846 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:15.846 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97693 2024-09-21T15:01:15.847 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.847 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:15.847 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.847 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:15.847 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328643+0000", 2024-09-21T15:01:15.847 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:15.847 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.847 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.847 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:15.847 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328643+0000", 2024-09-21T15:01:15.847 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:15.847 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.847 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.847 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:15.847 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328642+0000", 2024-09-21T15:01:15.848 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:01:15.848 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.848 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.848 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:15.848 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328650+0000", 2024-09-21T15:01:15.848 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7770000000000008e-06 2024-09-21T15:01:15.848 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.848 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.848 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:15.848 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328662+0000", 2024-09-21T15:01:15.848 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2026e-05 2024-09-21T15:01:15.848 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.848 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.848 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:15.848 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328671+0000", 2024-09-21T15:01:15.849 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.585e-06 2024-09-21T15:01:15.849 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.849 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.849 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:15.849 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.359498+0000", 2024-09-21T15:01:15.849 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.030827001 2024-09-21T15:01:15.849 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.849 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.849 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:15.849 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.359603+0000", 2024-09-21T15:01:15.849 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010512400000000001 2024-09-21T15:01:15.849 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.849 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.849 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:15.849 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.359705+0000", 2024-09-21T15:01:15.849 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000102327 2024-09-21T15:01:15.850 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:15.850 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:15.850 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:15.850 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.850 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.850 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:97724 3.b 3:d4bf6bcf:::rbd_data.1136b9c49b1c.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:01:15.850 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.328704+0000", 2024-09-21T15:01:15.850 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 70.479133103999999, 2024-09-21T15:01:15.850 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.031157957, 2024-09-21T15:01:15.850 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:15.850 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:15.850 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:15.850 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:15.850 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:15.850 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97724 2024-09-21T15:01:15.851 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.851 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:15.851 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.851 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:15.851 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328704+0000", 2024-09-21T15:01:15.851 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:15.851 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.851 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.851 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:15.851 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328704+0000", 2024-09-21T15:01:15.851 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:15.851 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.851 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.851 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:15.851 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328703+0000", 2024-09-21T15:01:15.852 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:15.852 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.852 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.852 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:15.852 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328708+0000", 2024-09-21T15:01:15.852 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0050000000000004e-06 2024-09-21T15:01:15.852 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.852 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.852 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:15.852 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328719+0000", 2024-09-21T15:01:15.852 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1052e-05 2024-09-21T15:01:15.852 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.852 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.852 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:15.852 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328727+0000", 2024-09-21T15:01:15.852 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4519999999999997e-06 2024-09-21T15:01:15.853 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.853 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.853 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:15.853 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.359737+0000", 2024-09-21T15:01:15.853 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.031009341999999999 2024-09-21T15:01:15.853 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.853 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.853 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:15.853 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.359806+0000", 2024-09-21T15:01:15.853 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9750000000000001e-05 2024-09-21T15:01:15.853 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.853 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.853 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:15.853 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.359862+0000", 2024-09-21T15:01:15.853 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5084999999999999e-05 2024-09-21T15:01:15.853 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:15.854 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:15.854 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:15.854 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.854 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.854 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:129970 3.c e158/87 3:33fa4d4e:::rbd_data.1136a12e4351.0000000000000000:head v 158'379, mlcod=156'377)", 2024-09-21T15:01:15.854 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:44.452766+0000", 2024-09-21T15:01:15.854 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 31.355070552000001, 2024-09-21T15:01:15.854 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011942062, 2024-09-21T15:01:15.854 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:15.854 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:15.854 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:15.854 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.854 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:15.854 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:44.452766+0000", 2024-09-21T15:01:15.854 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:15.854 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.855 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.855 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:15.855 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:44.452766+0000", 2024-09-21T15:01:15.855 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:15.855 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.855 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.855 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:15.855 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:44.452766+0000", 2024-09-21T15:01:15.855 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:15.855 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.855 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.855 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:15.855 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:44.458962+0000", 2024-09-21T15:01:15.855 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0061964799999999999 2024-09-21T15:01:15.855 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.856 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.856 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:15.856 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:44.459012+0000", 2024-09-21T15:01:15.856 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9427e-05 2024-09-21T15:01:15.856 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.856 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.856 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:15.856 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:44.459042+0000", 2024-09-21T15:01:15.856 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9838000000000001e-05 2024-09-21T15:01:15.856 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.856 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.856 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:15.856 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:44.459115+0000", 2024-09-21T15:01:15.856 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3430999999999995e-05 2024-09-21T15:01:15.856 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.857 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.857 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:15.857 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:44.459135+0000", 2024-09-21T15:01:15.857 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9701999999999999e-05 2024-09-21T15:01:15.857 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.857 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.857 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:01:15.857 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:44.464645+0000", 2024-09-21T15:01:15.857 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0055100820000000003 2024-09-21T15:01:15.857 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.857 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.857 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:15.857 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:44.464708+0000", 2024-09-21T15:01:15.857 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3399999999999996e-05 2024-09-21T15:01:15.857 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:15.857 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:15.858 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:15.858 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.858 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.858 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:135368 3.b 3:d3301c39:::rbd_data.11365ed419e3.0000000000000102:67 [read 335872~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:15.858 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:53.998901+0000", 2024-09-21T15:01:15.858 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 21.808935739999999, 2024-09-21T15:01:15.858 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011666174, 2024-09-21T15:01:15.858 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:15.858 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:15.858 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:15.858 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:15.858 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:15.858 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 135368 2024-09-21T15:01:15.858 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.858 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:15.859 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.859 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:15.859 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998901+0000", 2024-09-21T15:01:15.859 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:15.859 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.859 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.859 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:15.859 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998901+0000", 2024-09-21T15:01:15.859 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:15.859 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.859 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.859 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:15.859 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998901+0000", 2024-09-21T15:01:15.859 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:15.859 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.860 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.860 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:15.860 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998903+0000", 2024-09-21T15:01:15.860 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5500000000000001e-06 2024-09-21T15:01:15.860 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.860 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.860 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:15.860 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998913+0000", 2024-09-21T15:01:15.860 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6479999999999998e-06 2024-09-21T15:01:15.860 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.860 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.860 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:15.860 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998919+0000", 2024-09-21T15:01:15.860 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7930000000000003e-06 2024-09-21T15:01:15.860 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.860 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.861 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:15.861 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:54.010277+0000", 2024-09-21T15:01:15.861 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011358692 2024-09-21T15:01:15.861 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.861 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.861 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:15.861 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:54.010348+0000", 2024-09-21T15:01:15.861 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0247999999999996e-05 2024-09-21T15:01:15.861 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.861 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.861 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:15.861 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:54.010567+0000", 2024-09-21T15:01:15.861 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021953599999999999 2024-09-21T15:01:15.861 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:15.861 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:15.861 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:15.862 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.862 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.862 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:135369 3.b 3:d3301c39:::rbd_data.11365ed419e3.0000000000000102:67 [read 352256~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:15.862 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:53.998923+0000", 2024-09-21T15:01:15.862 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 21.808913439000001, 2024-09-21T15:01:15.862 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011990331999999999, 2024-09-21T15:01:15.862 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:15.862 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:15.862 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:15.862 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:15.862 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:15.862 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 135369 2024-09-21T15:01:15.862 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.862 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:15.863 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.863 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:15.863 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998923+0000", 2024-09-21T15:01:15.863 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:15.863 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.863 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.863 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:15.863 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998923+0000", 2024-09-21T15:01:15.863 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:15.863 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.863 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.863 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:15.863 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998923+0000", 2024-09-21T15:01:15.863 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:15.863 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.863 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.864 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:15.864 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998926+0000", 2024-09-21T15:01:15.864 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6749999999999998e-06 2024-09-21T15:01:15.864 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.864 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.864 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:15.864 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998935+0000", 2024-09-21T15:01:15.864 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3619999999999995e-06 2024-09-21T15:01:15.864 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.864 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.864 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:15.864 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998940+0000", 2024-09-21T15:01:15.864 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2379999999999997e-06 2024-09-21T15:01:15.864 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.864 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.864 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:15.865 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:54.010593+0000", 2024-09-21T15:01:15.865 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011652531000000001 2024-09-21T15:01:15.865 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.865 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.865 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:15.865 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:54.010679+0000", 2024-09-21T15:01:15.865 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6811000000000006e-05 2024-09-21T15:01:15.865 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.865 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.865 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:15.865 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:54.010914+0000", 2024-09-21T15:01:15.865 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000234071 2024-09-21T15:01:15.865 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:15.865 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:15.865 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:15.865 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.866 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.866 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:137705 3.b 3:d0cb966d:::rbd_data.1136f64ddcfe.0000000000000005:6a [read 1327104~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-21T15:01:15.866 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:58.376355+0000", 2024-09-21T15:01:15.866 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 17.431481820999998, 2024-09-21T15:01:15.866 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011637611000000001, 2024-09-21T15:01:15.866 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:15.866 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:15.866 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:15.866 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:15.866 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:15.866 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 137705 2024-09-21T15:01:15.866 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.866 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:15.866 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.866 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:15.867 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376355+0000", 2024-09-21T15:01:15.867 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:15.867 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.867 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.867 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:15.867 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376355+0000", 2024-09-21T15:01:15.867 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:15.867 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.867 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.867 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:15.867 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376354+0000", 2024-09-21T15:01:15.867 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:15.867 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.867 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.868 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:15.868 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376358+0000", 2024-09-21T15:01:15.868 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.179e-06 2024-09-21T15:01:15.868 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.868 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.868 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:15.868 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376369+0000", 2024-09-21T15:01:15.868 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1766000000000001e-05 2024-09-21T15:01:15.868 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.868 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.868 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:15.868 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376376+0000", 2024-09-21T15:01:15.868 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6019999999999996e-06 2024-09-21T15:01:15.868 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.868 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.869 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:15.869 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.387882+0000", 2024-09-21T15:01:15.869 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011506126 2024-09-21T15:01:15.869 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.869 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.869 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:15.869 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.387929+0000", 2024-09-21T15:01:15.869 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6742000000000001e-05 2024-09-21T15:01:15.869 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.869 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.869 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:15.869 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.387992+0000", 2024-09-21T15:01:15.869 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3656000000000002e-05 2024-09-21T15:01:15.869 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:15.870 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:15.870 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:15.870 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.870 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.870 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:137706 3.b 3:d0cb966d:::rbd_data.1136f64ddcfe.0000000000000005:6a [read 1441792~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-21T15:01:15.870 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:58.376385+0000", 2024-09-21T15:01:15.870 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 17.431451851999999, 2024-09-21T15:01:15.870 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011783609, 2024-09-21T15:01:15.870 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:15.870 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:15.870 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:15.870 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:15.870 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:15.870 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 137706 2024-09-21T15:01:15.870 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.871 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:15.871 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.871 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:15.871 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376385+0000", 2024-09-21T15:01:15.871 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:15.871 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.871 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.871 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:15.871 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376385+0000", 2024-09-21T15:01:15.871 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:15.871 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.871 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.871 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:15.871 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376384+0000", 2024-09-21T15:01:15.872 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:15.872 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.872 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.872 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:15.872 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376387+0000", 2024-09-21T15:01:15.872 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9809999999999999e-06 2024-09-21T15:01:15.872 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.872 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.872 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:15.872 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376398+0000", 2024-09-21T15:01:15.872 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0319000000000001e-05 2024-09-21T15:01:15.872 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.872 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.872 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:15.872 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376404+0000", 2024-09-21T15:01:15.872 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5339999999999996e-06 2024-09-21T15:01:15.873 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.873 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.873 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:15.873 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.388025+0000", 2024-09-21T15:01:15.873 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011620449999999999 2024-09-21T15:01:15.873 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.873 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.873 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:15.873 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.388082+0000", 2024-09-21T15:01:15.873 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7065999999999999e-05 2024-09-21T15:01:15.873 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.873 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.873 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:15.873 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.388168+0000", 2024-09-21T15:01:15.874 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6689999999999998e-05 2024-09-21T15:01:15.874 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:15.874 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:15.874 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:15.874 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.874 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.874 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:137707 3.b 3:d0cb966d:::rbd_data.1136f64ddcfe.0000000000000005:6a [read 1556480~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-21T15:01:15.874 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:58.376428+0000", 2024-09-21T15:01:15.874 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 17.431408359999999, 2024-09-21T15:01:15.874 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012004865999999999, 2024-09-21T15:01:15.874 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:15.874 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:15.874 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:15.874 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:15.874 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:15.875 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 137707 2024-09-21T15:01:15.875 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.875 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:15.875 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.875 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:15.875 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376428+0000", 2024-09-21T15:01:15.875 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:15.875 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.875 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.875 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:15.875 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376428+0000", 2024-09-21T15:01:15.875 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:15.875 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.876 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.876 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:15.876 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376428+0000", 2024-09-21T15:01:15.876 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:15.876 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.876 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.876 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:15.876 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376432+0000", 2024-09-21T15:01:15.876 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2119999999999997e-06 2024-09-21T15:01:15.876 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.876 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.876 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:15.876 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376448+0000", 2024-09-21T15:01:15.876 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5913000000000002e-05 2024-09-21T15:01:15.876 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.877 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.877 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:15.877 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376455+0000", 2024-09-21T15:01:15.877 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6359999999999994e-06 2024-09-21T15:01:15.877 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.877 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.877 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:15.877 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.388197+0000", 2024-09-21T15:01:15.877 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011741843 2024-09-21T15:01:15.877 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.877 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.877 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:15.877 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.388249+0000", 2024-09-21T15:01:15.877 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1549999999999999e-05 2024-09-21T15:01:15.877 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.878 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.878 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:15.878 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.388433+0000", 2024-09-21T15:01:15.878 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018437999999999999 2024-09-21T15:01:15.878 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:15.878 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:15.878 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:15.878 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.878 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.878 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:137708 3.b 3:d0cb966d:::rbd_data.1136f64ddcfe.0000000000000005:6a [read 1671168~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-21T15:01:15.878 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:58.376462+0000", 2024-09-21T15:01:15.878 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 17.431374571999999, 2024-09-21T15:01:15.878 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012315491, 2024-09-21T15:01:15.878 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:15.878 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:15.879 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:15.879 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:15.879 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:15.879 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 137708 2024-09-21T15:01:15.879 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.879 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:15.879 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.879 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:15.879 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376462+0000", 2024-09-21T15:01:15.879 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:15.879 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.879 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.879 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:15.879 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376462+0000", 2024-09-21T15:01:15.879 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:15.880 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.880 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.880 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:15.880 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376461+0000", 2024-09-21T15:01:15.880 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:15.880 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.880 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.880 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:15.880 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376465+0000", 2024-09-21T15:01:15.880 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7069999999999998e-06 2024-09-21T15:01:15.880 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.880 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.880 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:15.880 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376473+0000", 2024-09-21T15:01:15.881 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6669999999999996e-06 2024-09-21T15:01:15.881 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.881 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.881 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:15.881 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376479+0000", 2024-09-21T15:01:15.881 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.263e-06 2024-09-21T15:01:15.881 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.881 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.881 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:15.881 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.388489+0000", 2024-09-21T15:01:15.881 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012009953 2024-09-21T15:01:15.881 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.881 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.881 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:15.881 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.388625+0000", 2024-09-21T15:01:15.882 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013548799999999999 2024-09-21T15:01:15.882 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.882 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.882 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:15.882 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.388778+0000", 2024-09-21T15:01:15.882 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015302299999999999 2024-09-21T15:01:15.882 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:15.882 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:15.882 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:15.882 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.882 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.882 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:137709 3.b 3:d0cb966d:::rbd_data.1136f64ddcfe.0000000000000005:6a [read 1785856~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-21T15:01:15.882 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:58.376485+0000", 2024-09-21T15:01:15.882 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 17.431351240000001, 2024-09-21T15:01:15.882 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012466549, 2024-09-21T15:01:15.883 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:15.883 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:15.883 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:15.883 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:15.883 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:15.883 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 137709 2024-09-21T15:01:15.883 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.883 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:15.883 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.883 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:15.883 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376485+0000", 2024-09-21T15:01:15.883 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:15.883 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.883 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.883 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:15.883 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376485+0000", 2024-09-21T15:01:15.884 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:15.884 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.884 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.884 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:15.884 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376485+0000", 2024-09-21T15:01:15.884 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:15.884 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.884 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.884 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:15.884 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376492+0000", 2024-09-21T15:01:15.884 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8929999999999999e-06 2024-09-21T15:01:15.884 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.884 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.884 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:15.884 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376500+0000", 2024-09-21T15:01:15.885 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7989999999999994e-06 2024-09-21T15:01:15.885 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.885 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.885 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:15.885 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376506+0000", 2024-09-21T15:01:15.885 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4799999999999998e-06 2024-09-21T15:01:15.885 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.885 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.885 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:15.885 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.388810+0000", 2024-09-21T15:01:15.885 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012303727 2024-09-21T15:01:15.885 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.885 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.885 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:15.885 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.388871+0000", 2024-09-21T15:01:15.886 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1278000000000005e-05 2024-09-21T15:01:15.886 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.886 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.886 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:15.886 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.388952+0000", 2024-09-21T15:01:15.886 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0770000000000001e-05 2024-09-21T15:01:15.886 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:15.886 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:15.886 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:15.886 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.886 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.886 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:137710 3.b 3:d0cb966d:::rbd_data.1136f64ddcfe.0000000000000005:6a [read 1900544~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-21T15:01:15.886 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:58.376512+0000", 2024-09-21T15:01:15.886 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 17.431325037000001, 2024-09-21T15:01:15.887 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012746351, 2024-09-21T15:01:15.887 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:15.887 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:15.887 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:15.887 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:15.887 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:15.887 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 137710 2024-09-21T15:01:15.887 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.887 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:15.887 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.887 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:15.887 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376512+0000", 2024-09-21T15:01:15.887 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:15.887 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.888 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.888 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:15.888 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376512+0000", 2024-09-21T15:01:15.888 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:15.888 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.888 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.888 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:15.888 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376511+0000", 2024-09-21T15:01:15.888 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:15.888 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.888 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.888 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:15.888 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376514+0000", 2024-09-21T15:01:15.888 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1719999999999999e-06 2024-09-21T15:01:15.888 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.889 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.889 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:15.889 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376522+0000", 2024-09-21T15:01:15.889 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1389999999999998e-06 2024-09-21T15:01:15.889 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.889 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.889 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:15.889 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376535+0000", 2024-09-21T15:01:15.889 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3658999999999999e-05 2024-09-21T15:01:15.889 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.889 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.889 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:15.889 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.389005+0000", 2024-09-21T15:01:15.889 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012470261 2024-09-21T15:01:15.889 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.889 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.890 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:15.890 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.389119+0000", 2024-09-21T15:01:15.890 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000113346 2024-09-21T15:01:15.890 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.890 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.890 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:15.890 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.389258+0000", 2024-09-21T15:01:15.890 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013920100000000001 2024-09-21T15:01:15.890 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:15.890 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:15.890 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:15.890 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.890 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.890 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:137711 3.b 3:d0cb966d:::rbd_data.1136f64ddcfe.0000000000000005:6a [read 2015232~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-21T15:01:15.890 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:58.376541+0000", 2024-09-21T15:01:15.890 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 17.431295675000001, 2024-09-21T15:01:15.891 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012849955999999999, 2024-09-21T15:01:15.891 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:15.891 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:15.891 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:15.891 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:15.891 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:15.891 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 137711 2024-09-21T15:01:15.891 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.891 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:15.891 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.891 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:15.891 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376541+0000", 2024-09-21T15:01:15.891 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:15.891 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.891 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.891 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:15.892 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376541+0000", 2024-09-21T15:01:15.892 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:15.892 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.892 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.892 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:15.892 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376540+0000", 2024-09-21T15:01:15.892 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:15.892 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.892 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.892 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:15.892 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376544+0000", 2024-09-21T15:01:15.892 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3249999999999999e-06 2024-09-21T15:01:15.892 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.892 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.892 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:15.892 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376551+0000", 2024-09-21T15:01:15.893 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2189999999999998e-06 2024-09-21T15:01:15.893 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.893 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.893 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:15.893 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376557+0000", 2024-09-21T15:01:15.893 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8869999999999997e-06 2024-09-21T15:01:15.893 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.893 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.893 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:15.893 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.389277+0000", 2024-09-21T15:01:15.893 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012720555999999999 2024-09-21T15:01:15.893 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.893 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.893 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:15.894 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.389325+0000", 2024-09-21T15:01:15.894 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7367000000000002e-05 2024-09-21T15:01:15.894 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.894 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.894 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:15.894 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.389391+0000", 2024-09-21T15:01:15.894 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6095999999999999e-05 2024-09-21T15:01:15.894 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:15.894 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:15.894 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:15.894 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.894 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.894 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:137723 3.b 3:d0cb966d:::rbd_data.1136f64ddcfe.0000000000000005:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-21T15:01:15.894 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:58.377091+0000", 2024-09-21T15:01:15.894 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 17.430745353999999, 2024-09-21T15:01:15.895 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012594424, 2024-09-21T15:01:15.895 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:15.895 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:15.895 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:15.895 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:15.895 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:15.895 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 137723 2024-09-21T15:01:15.895 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.895 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:15.895 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.895 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:15.895 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.377091+0000", 2024-09-21T15:01:15.895 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:15.895 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.895 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.895 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:15.896 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.377091+0000", 2024-09-21T15:01:15.896 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:15.896 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.896 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.896 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:15.896 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.377090+0000", 2024-09-21T15:01:15.896 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:15.896 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.896 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.896 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:15.896 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.377097+0000", 2024-09-21T15:01:15.896 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7100000000000001e-06 2024-09-21T15:01:15.896 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.896 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.896 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:15.897 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.377120+0000", 2024-09-21T15:01:15.897 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3189000000000001e-05 2024-09-21T15:01:15.897 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.897 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.897 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:15.897 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.377132+0000", 2024-09-21T15:01:15.897 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2041999999999999e-05 2024-09-21T15:01:15.897 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.897 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.897 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:15.897 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.389439+0000", 2024-09-21T15:01:15.897 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.01230698 2024-09-21T15:01:15.897 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.897 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.897 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:15.897 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.389556+0000", 2024-09-21T15:01:15.898 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000116446 2024-09-21T15:01:15.898 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:15.898 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:15.898 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:15.898 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.389686+0000", 2024-09-21T15:01:15.898 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000129966 2024-09-21T15:01:15.898 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:15.898 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:15.898 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:15.898 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:15.898 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:15.898 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:01:15.899 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T15:01:15.900 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:01:15.897+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:01:16.000 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:01:15.994+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:01:16.019 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:16.020 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:01:16.020 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T15:01:16.020 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:16.030 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T15:01:16.101 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:01:16.098+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:01:16.141 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:16.141 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:01:16.141 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T15:01:16.141 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T15:01:16.141 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:16.151 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T15:01:16.203 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:01:16.198+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:01:16.259 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:16.259 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T15:01:16.259 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T15:01:16.259 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T15:01:16.259 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.259 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:97677 3.15 3:a8e72133:::rbd_data.1136bfc79a70.0000000000000005:head [write 0~4440027 [fadvise_sequential+fadvise_nocache],write 4751360~147456 [fadvise_sequential+fadvise_nocache],write 4964352~147456 [fadvise_sequential+fadvise_nocache],write 5177344~65536 [fadvise_sequential+fadvise_nocache],write 5259264~147456 [fadvise_sequential+fadvise_nocache],write 5472256~147456 [fadvise_sequential+fadvise_nocache],write 5685248~229376 [fadvise_sequential+fadvise_nocache],write 5980160~147456 [fadvise_sequential+fadvise_nocache],write 6193152~229376 [fadvise_sequential+fadvise_nocache],write 6488064~147456 [fadvise_sequential+fadvise_nocache],write 6701056~229376 [fadvise_sequential+fadvise_nocache],write 6995968~147456 [fadvise_sequential+fadvise_nocache],write 7208960~229376 [fadvise_sequential+fadvise_nocache],write 7503872~147456 [fadvise_sequential+fadvise_nocache],write 7716864~229376 [fadvise_sequential+fadvise_nocache],write 8044544~114688 [fadvise_sequential+fadvise_nocache],write 8257536~114688 [fadvise_sequential+fadvise_nocache],write 8388608~65536 [fadvise_sequential+fadvise_nocache],write 8552448~114688 [fadvise_sequential+fadvise_nocache],write 8765440~114688 [fadvise_sequential+fadvise_nocache],write 8929280~32768 [fadvise_sequential+fadvise_nocache],write 9142272~32768 [fadvise_sequential+fadvise_nocache],write 9355264~32768 [fadvise_sequential+fadvise_nocache],write 9437184~32768 [fadvise_sequential+fadvise_nocache],write 9650176~32768 [fadvise_sequential+fadvise_nocache],write 9863168~32768 [fadvise_sequential+fadvise_nocache],write 9961472~131072 [fadvise_sequential+fadvise_nocache],write 10108928~196608 [fadvise_sequential+fadvise_nocache],write 10321920~278528 [fadvise_sequential+fadvise_nocache],write 10616832~196608 [fadvise_sequential+fadvise_nocache],write 10829824~278528 [fadvise_sequential+fadvise_nocache],write 11124736~196608 [fadvise_sequential+fadvise_nocache],write 11337728~196608 [fadvise_sequential+fadvise_nocache],write 11550720~65536 [fadvise_sequential+fadvise_nocache],write 11649024~98304 [fadvise_sequential+fadvise_nocache],write 11763712~65536 [fadvise_sequential+fadvise_nocache],write 11862016~98304 [fadvise_sequential+fadvise_nocache],write 11976704~65536 [fadvise_sequential+fadvise_nocache],write 12058624~65536 [fadvise_sequential+fadvise_nocache],write 12156928~98304 [fadvise_sequential+fadvise_nocache],write 12271616~65536 [fadvise_sequential+fadvise_nocache],write 12369920~98304 [fadvise_sequential+fadvise_nocache],write 12484608~65536 [fadvise_sequential+fadvise_nocache],write 12664832~98304 [fadvise_sequential+fadvise_nocache],write 12877824~98304 [fadvise_sequential+fadvise_nocache],write 13090816~16384 [fadvise_sequential+fadvise_nocache],write 13172736~98304 [fadvise_sequential+fadvise_nocache],write 13385728~98304 [fadvise_sequential+fadvise_nocache],write 13598720~32768 [fadvise_sequential+fadvise_nocache],write 13680640~98304 [fadvise_sequential+fadvise_nocache],write 13893632~98304 [fadvise_sequential+fadvise_nocache],write 14106624~49152 [fadvise_sequential+fadvise_nocache],write 14794752~81920 [fadvise_sequential+fadvise_nocache],write 15007744~81920 [fadvise_sequential+fadvise_nocache],write 15302656~81920 [fadvise_sequential+fadvise_nocache],write 15515648~81920 [fadvise_sequential+fadvise_nocache],write 15810560~81920 [fadvise_sequential+fadvise_nocache],write 16023552~81920 [fadvise_sequential+fadvise_nocache],write 16236544~16384 [fadvise_sequential+fadvise_nocache],write 16318464~81920 [fadvise_sequential+fadvise_nocache],write 16531456~81920 [fadvise_sequential+fadvise_nocache],write 16744448~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:01:16.259 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.302946+0000", 2024-09-21T15:01:16.259 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 70.953082094999999, 2024-09-21T15:01:16.259 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.091231204999999996, 2024-09-21T15:01:16.259 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:16.259 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:16.259 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:16.260 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:16.260 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:16.260 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 97677 2024-09-21T15:01:16.260 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.260 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:16.260 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.260 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:16.260 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.302946+0000", 2024-09-21T15:01:16.260 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:16.260 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.260 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.260 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:16.260 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.302946+0000", 2024-09-21T15:01:16.260 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:16.261 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.261 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.261 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:16.261 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.302942+0000", 2024-09-21T15:01:16.261 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999962 2024-09-21T15:01:16.261 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.261 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.261 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:16.261 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311253+0000", 2024-09-21T15:01:16.261 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0083105360000000003 2024-09-21T15:01:16.261 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.261 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.261 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:16.261 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311283+0000", 2024-09-21T15:01:16.262 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0065000000000001e-05 2024-09-21T15:01:16.262 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.262 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.262 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:16.262 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311296+0000", 2024-09-21T15:01:16.262 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3206e-05 2024-09-21T15:01:16.262 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.262 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.262 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:16.262 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311404+0000", 2024-09-21T15:01:16.262 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000107514 2024-09-21T15:01:16.262 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.262 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.262 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:16.263 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311795+0000", 2024-09-21T15:01:16.263 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000390913 2024-09-21T15:01:16.263 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.263 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.263 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 6", 2024-09-21T15:01:16.263 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.314686+0000", 2024-09-21T15:01:16.263 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0028915659999999999 2024-09-21T15:01:16.263 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.263 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.263 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:01:16.263 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.368178+0000", 2024-09-21T15:01:16.263 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.053491706 2024-09-21T15:01:16.264 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.264 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.264 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:01:16.264 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.393925+0000", 2024-09-21T15:01:16.264 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025747289 2024-09-21T15:01:16.264 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.264 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.264 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:01:16.264 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.394127+0000", 2024-09-21T15:01:16.264 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000201444 2024-09-21T15:01:16.264 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.264 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.264 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:16.264 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.394177+0000", 2024-09-21T15:01:16.265 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0689000000000002e-05 2024-09-21T15:01:16.265 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:16.265 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:16.265 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:16.265 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.265 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.265 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:97673 3.1d e122/24 3:bd91946e:::rbd_data.1136bfc79a70.0000000000000002:head v 122'162, mlcod=120'160)", 2024-09-21T15:01:16.265 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.313730+0000", 2024-09-21T15:01:16.265 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 70.942297769999996, 2024-09-21T15:01:16.265 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055373462999999998, 2024-09-21T15:01:16.265 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:16.265 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:16.265 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:16.265 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.266 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:16.266 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.313730+0000", 2024-09-21T15:01:16.266 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:16.266 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.266 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.266 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:16.266 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.313730+0000", 2024-09-21T15:01:16.266 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:16.266 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.266 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.266 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:16.266 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.313730+0000", 2024-09-21T15:01:16.266 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:16.266 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.267 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.267 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:16.267 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.336113+0000", 2024-09-21T15:01:16.267 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022382962999999999 2024-09-21T15:01:16.267 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.267 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.267 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:16.267 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.336140+0000", 2024-09-21T15:01:16.267 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7248999999999999e-05 2024-09-21T15:01:16.267 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.267 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.267 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:16.267 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.336151+0000", 2024-09-21T15:01:16.268 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1015e-05 2024-09-21T15:01:16.268 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.268 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.268 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:16.268 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.355428+0000", 2024-09-21T15:01:16.268 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019276379999999999 2024-09-21T15:01:16.268 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.268 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.268 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:16.268 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.355484+0000", 2024-09-21T15:01:16.268 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6820999999999997e-05 2024-09-21T15:01:16.268 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.268 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.268 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:01:16.268 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.369040+0000", 2024-09-21T15:01:16.268 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013555466 2024-09-21T15:01:16.269 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.269 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.269 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:16.269 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.369104+0000", 2024-09-21T15:01:16.269 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3986999999999996e-05 2024-09-21T15:01:16.269 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:16.269 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:16.269 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:16.269 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.269 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.269 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139890 3.e 3:71dba096:::rbd_data.113664780c85.000000000000018b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:16.269 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.226444+0000", 2024-09-21T15:01:16.270 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.029584128, 2024-09-21T15:01:16.270 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.028209567, 2024-09-21T15:01:16.270 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:16.270 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:16.270 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:16.270 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:16.270 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:16.270 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139890 2024-09-21T15:01:16.270 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.270 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:16.270 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.270 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:16.270 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226444+0000", 2024-09-21T15:01:16.270 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:16.271 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.271 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.271 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:16.271 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226444+0000", 2024-09-21T15:01:16.271 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:16.271 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.271 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.271 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:16.271 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226443+0000", 2024-09-21T15:01:16.271 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:16.271 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.271 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.271 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:16.271 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226448+0000", 2024-09-21T15:01:16.272 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5499999999999996e-06 2024-09-21T15:01:16.272 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.272 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.272 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:16.272 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226456+0000", 2024-09-21T15:01:16.272 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8679999999999999e-06 2024-09-21T15:01:16.272 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.272 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.272 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:16.272 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226462+0000", 2024-09-21T15:01:16.272 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1800000000000001e-06 2024-09-21T15:01:16.272 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.272 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.272 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:16.273 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226545+0000", 2024-09-21T15:01:16.273 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2527000000000005e-05 2024-09-21T15:01:16.273 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.273 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.273 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:16.273 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226553+0000", 2024-09-21T15:01:16.273 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8019999999999998e-06 2024-09-21T15:01:16.273 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.273 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.273 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:16.273 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.254491+0000", 2024-09-21T15:01:16.273 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.02793785 2024-09-21T15:01:16.273 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.273 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.273 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:16.273 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.254654+0000", 2024-09-21T15:01:16.274 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00016245800000000001 2024-09-21T15:01:16.274 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:16.274 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:16.274 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:16.274 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.274 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.274 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139897 3.e 3:76ddeb1d:::rbd_data.113664780c85.0000000000000004:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:16.274 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.227198+0000", 2024-09-21T15:01:16.274 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.028830523, 2024-09-21T15:01:16.274 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.027799822, 2024-09-21T15:01:16.274 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:16.274 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:16.274 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:16.275 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:16.275 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:16.275 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139897 2024-09-21T15:01:16.275 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.275 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:16.275 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.275 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:16.275 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227198+0000", 2024-09-21T15:01:16.275 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:16.275 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.275 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.275 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:16.275 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227198+0000", 2024-09-21T15:01:16.275 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:16.276 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.276 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.276 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:16.276 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227197+0000", 2024-09-21T15:01:16.276 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:16.276 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.276 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.276 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:16.276 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227203+0000", 2024-09-21T15:01:16.276 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4819999999999999e-06 2024-09-21T15:01:16.276 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.276 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.276 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:16.276 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227220+0000", 2024-09-21T15:01:16.276 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6671e-05 2024-09-21T15:01:16.277 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.277 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.277 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:16.277 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227229+0000", 2024-09-21T15:01:16.277 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0499999999999997e-06 2024-09-21T15:01:16.277 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.277 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.277 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:16.277 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227293+0000", 2024-09-21T15:01:16.277 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4434e-05 2024-09-21T15:01:16.277 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.277 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.277 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:16.277 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227302+0000", 2024-09-21T15:01:16.277 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0929999999999992e-06 2024-09-21T15:01:16.278 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.278 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.278 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:16.278 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.254720+0000", 2024-09-21T15:01:16.278 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0274177550000001 2024-09-21T15:01:16.278 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.278 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.278 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:16.278 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.254854+0000", 2024-09-21T15:01:16.278 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000133416 2024-09-21T15:01:16.278 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.278 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.278 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:16.278 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.254998+0000", 2024-09-21T15:01:16.278 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014400099999999999 2024-09-21T15:01:16.278 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:16.279 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:16.279 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:16.279 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.279 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.279 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139902 3.e 3:76b9b799:::rbd_data.113664780c85.0000000000000021:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:16.279 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.227319+0000", 2024-09-21T15:01:16.279 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.028709142, 2024-09-21T15:01:16.279 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0279584639999999, 2024-09-21T15:01:16.279 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:16.279 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:16.279 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:16.279 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:16.279 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:16.279 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139902 2024-09-21T15:01:16.280 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.280 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:16.280 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.280 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:16.280 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227319+0000", 2024-09-21T15:01:16.280 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:16.280 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.280 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.280 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:16.280 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227319+0000", 2024-09-21T15:01:16.280 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:16.280 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.280 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.280 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:16.280 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227319+0000", 2024-09-21T15:01:16.281 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:16.281 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.281 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.281 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:16.281 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227322+0000", 2024-09-21T15:01:16.281 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1980000000000001e-06 2024-09-21T15:01:16.281 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.281 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.281 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:16.281 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227328+0000", 2024-09-21T15:01:16.281 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4230000000000002e-06 2024-09-21T15:01:16.281 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.281 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.281 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:16.281 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227333+0000", 2024-09-21T15:01:16.282 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0560000000000004e-06 2024-09-21T15:01:16.282 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.282 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.282 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:16.282 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227729+0000", 2024-09-21T15:01:16.282 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00039591799999999999 2024-09-21T15:01:16.282 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.282 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.282 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:16.282 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227738+0000", 2024-09-21T15:01:16.282 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8119999999999997e-06 2024-09-21T15:01:16.282 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.282 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.282 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:16.283 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255021+0000", 2024-09-21T15:01:16.283 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0272826779999999 2024-09-21T15:01:16.283 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.283 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.283 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:16.283 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255147+0000", 2024-09-21T15:01:16.283 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012595600000000001 2024-09-21T15:01:16.283 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.283 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.283 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:16.283 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255278+0000", 2024-09-21T15:01:16.283 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013081400000000001 2024-09-21T15:01:16.283 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:16.284 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:16.284 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:16.284 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.284 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.284 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139945 3.e 3:7756a55d:::rbd_data.113664780c85.0000000000000154:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:16.284 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.229015+0000", 2024-09-21T15:01:16.284 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.027013339, 2024-09-21T15:01:16.284 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.026468825, 2024-09-21T15:01:16.284 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:16.284 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:16.284 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:16.284 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:16.284 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:16.284 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139945 2024-09-21T15:01:16.284 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.285 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:16.285 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.285 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:16.285 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229015+0000", 2024-09-21T15:01:16.285 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:16.285 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.285 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.285 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:16.285 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229015+0000", 2024-09-21T15:01:16.285 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:16.285 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.285 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.285 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:16.285 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229014+0000", 2024-09-21T15:01:16.285 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:16.286 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.286 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.286 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:16.286 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229017+0000", 2024-09-21T15:01:16.286 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7310000000000002e-06 2024-09-21T15:01:16.286 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.286 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.286 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:16.286 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229023+0000", 2024-09-21T15:01:16.286 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8150000000000002e-06 2024-09-21T15:01:16.286 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.286 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.286 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:16.287 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229028+0000", 2024-09-21T15:01:16.287 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7650000000000001e-06 2024-09-21T15:01:16.287 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.287 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.287 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:16.287 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229082+0000", 2024-09-21T15:01:16.287 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4521999999999997e-05 2024-09-21T15:01:16.287 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.287 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.287 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:16.287 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229092+0000", 2024-09-21T15:01:16.287 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.3940000000000002e-06 2024-09-21T15:01:16.287 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.287 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.287 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:16.288 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255382+0000", 2024-09-21T15:01:16.288 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0262900930000001 2024-09-21T15:01:16.288 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.288 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.288 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:16.288 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255484+0000", 2024-09-21T15:01:16.288 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000101932 2024-09-21T15:01:16.288 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:16.288 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:16.288 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:16.288 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.288 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.288 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139967 3.e 3:74e90d85:::rbd_data.113664780c85.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:16.288 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.229124+0000", 2024-09-21T15:01:16.288 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.026904398999999, 2024-09-21T15:01:16.289 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.026724381, 2024-09-21T15:01:16.289 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:16.289 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:16.289 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:16.289 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:16.289 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:16.289 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139967 2024-09-21T15:01:16.289 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.289 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:16.289 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.289 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:16.289 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229124+0000", 2024-09-21T15:01:16.289 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:16.289 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.289 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.290 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:16.290 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229124+0000", 2024-09-21T15:01:16.290 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:16.290 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.290 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.290 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:16.290 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229123+0000", 2024-09-21T15:01:16.290 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:16.290 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.290 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.290 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:16.290 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229126+0000", 2024-09-21T15:01:16.290 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4779999999999998e-06 2024-09-21T15:01:16.291 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.291 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.291 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:16.291 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229131+0000", 2024-09-21T15:01:16.291 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.729e-06 2024-09-21T15:01:16.291 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.291 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.291 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:16.291 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229135+0000", 2024-09-21T15:01:16.291 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5859999999999998e-06 2024-09-21T15:01:16.291 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.292 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.292 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:16.292 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229558+0000", 2024-09-21T15:01:16.292 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00042262499999999999 2024-09-21T15:01:16.292 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.292 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.292 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:16.292 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229569+0000", 2024-09-21T15:01:16.292 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0985000000000001e-05 2024-09-21T15:01:16.292 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.292 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.292 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:16.292 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255509+0000", 2024-09-21T15:01:16.292 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.025940246 2024-09-21T15:01:16.293 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.293 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.293 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:16.293 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255662+0000", 2024-09-21T15:01:16.293 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015324199999999999 2024-09-21T15:01:16.293 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.293 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.293 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:16.293 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255848+0000", 2024-09-21T15:01:16.293 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00018583599999999999 2024-09-21T15:01:16.293 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:16.293 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:16.293 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:16.293 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.293 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.294 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140010 3.e 3:735c06f2:::rbd_data.113664780c85.0000000000000157:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:16.294 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.231531+0000", 2024-09-21T15:01:16.294 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.024497534, 2024-09-21T15:01:16.294 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0244773149999999, 2024-09-21T15:01:16.294 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:16.294 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:16.294 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:16.294 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:16.294 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:16.294 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140010 2024-09-21T15:01:16.294 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.294 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:16.295 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.295 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:16.295 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231531+0000", 2024-09-21T15:01:16.295 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:16.295 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.295 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.295 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:16.295 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231531+0000", 2024-09-21T15:01:16.295 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:16.295 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.295 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.296 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:16.296 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231530+0000", 2024-09-21T15:01:16.296 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:16.296 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.296 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.296 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:16.296 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231535+0000", 2024-09-21T15:01:16.296 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3970000000000002e-06 2024-09-21T15:01:16.296 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.296 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.296 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:16.296 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231544+0000", 2024-09-21T15:01:16.296 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5909999999999996e-06 2024-09-21T15:01:16.296 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.297 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.297 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:16.297 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231551+0000", 2024-09-21T15:01:16.297 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7129999999999999e-06 2024-09-21T15:01:16.297 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.297 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.297 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:16.297 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231632+0000", 2024-09-21T15:01:16.297 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1181999999999996e-05 2024-09-21T15:01:16.297 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.297 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.297 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:16.297 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231640+0000", 2024-09-21T15:01:16.297 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.551e-06 2024-09-21T15:01:16.297 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.298 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.298 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:16.298 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255884+0000", 2024-09-21T15:01:16.298 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.024244057 2024-09-21T15:01:16.298 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.298 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.298 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:16.298 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256008+0000", 2024-09-21T15:01:16.298 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012351699999999999 2024-09-21T15:01:16.298 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:16.298 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:16.298 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:16.298 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.298 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.299 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140043 3.e 3:74a37859:::rbd_data.113664780c85.0000000000000086:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:16.299 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.232623+0000", 2024-09-21T15:01:16.299 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.02340506, 2024-09-21T15:01:16.299 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.023518634, 2024-09-21T15:01:16.299 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:16.299 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:16.299 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:16.299 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:16.299 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:16.299 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140043 2024-09-21T15:01:16.299 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.299 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:16.299 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.299 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:16.299 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.232623+0000", 2024-09-21T15:01:16.300 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:16.300 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.300 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.300 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:16.300 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.232623+0000", 2024-09-21T15:01:16.300 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:16.300 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.300 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.300 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:16.300 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.232623+0000", 2024-09-21T15:01:16.300 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:16.300 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.300 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.300 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:16.300 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.232626+0000", 2024-09-21T15:01:16.300 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0019999999999998e-06 2024-09-21T15:01:16.301 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.301 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.301 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:16.301 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.232632+0000", 2024-09-21T15:01:16.301 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3260000000000001e-06 2024-09-21T15:01:16.301 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.301 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.301 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:16.301 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.232637+0000", 2024-09-21T15:01:16.301 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9899999999999997e-06 2024-09-21T15:01:16.301 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.301 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.301 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:16.301 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.232678+0000", 2024-09-21T15:01:16.302 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.0423999999999998e-05 2024-09-21T15:01:16.302 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.302 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.302 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:16.302 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.232686+0000", 2024-09-21T15:01:16.302 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1419999999999993e-06 2024-09-21T15:01:16.302 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.302 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.302 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:16.302 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256037+0000", 2024-09-21T15:01:16.302 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.023351736 2024-09-21T15:01:16.302 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.302 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.303 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:16.303 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256142+0000", 2024-09-21T15:01:16.303 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010443 2024-09-21T15:01:16.303 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:16.303 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:16.303 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:16.303 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.303 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.303 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140063 3.e 3:771cae23:::rbd_data.113664780c85.0000000000000112:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:16.303 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.233948+0000", 2024-09-21T15:01:16.303 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.022079993, 2024-09-21T15:01:16.303 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0223114710000001, 2024-09-21T15:01:16.303 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:16.303 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:16.303 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:16.304 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:16.304 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:16.304 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140063 2024-09-21T15:01:16.304 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.304 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:16.304 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.304 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:16.304 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233948+0000", 2024-09-21T15:01:16.304 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:16.304 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.304 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.304 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:16.304 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233948+0000", 2024-09-21T15:01:16.304 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:16.304 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.305 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.305 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:16.305 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233948+0000", 2024-09-21T15:01:16.305 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:16.305 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.305 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.305 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:16.305 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233951+0000", 2024-09-21T15:01:16.305 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3380000000000001e-06 2024-09-21T15:01:16.305 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.305 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.305 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:16.305 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233958+0000", 2024-09-21T15:01:16.305 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0990000000000001e-06 2024-09-21T15:01:16.306 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.306 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.306 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:16.306 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233964+0000", 2024-09-21T15:01:16.306 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.852e-06 2024-09-21T15:01:16.306 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.306 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.306 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:16.306 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.234056+0000", 2024-09-21T15:01:16.306 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2392000000000006e-05 2024-09-21T15:01:16.306 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.306 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.306 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:16.306 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.234066+0000", 2024-09-21T15:01:16.307 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1360000000000003e-06 2024-09-21T15:01:16.307 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.307 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.307 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:16.307 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256169+0000", 2024-09-21T15:01:16.307 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0221031949999999 2024-09-21T15:01:16.307 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.307 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.307 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:16.307 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256260+0000", 2024-09-21T15:01:16.307 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0939000000000005e-05 2024-09-21T15:01:16.307 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:16.307 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:16.307 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:16.307 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.307 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.308 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140086 3.e 3:758134b7:::rbd_data.113664780c85.00000000000001ad:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:16.308 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.235110+0000", 2024-09-21T15:01:16.308 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.020918293999999, 2024-09-21T15:01:16.308 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.021276217, 2024-09-21T15:01:16.308 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:16.308 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:16.308 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:16.308 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:16.308 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:16.308 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140086 2024-09-21T15:01:16.308 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.308 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:16.308 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.308 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:16.309 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.235110+0000", 2024-09-21T15:01:16.309 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:16.309 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.309 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.309 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:16.309 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.235110+0000", 2024-09-21T15:01:16.309 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:16.309 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.309 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.309 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:16.309 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.235109+0000", 2024-09-21T15:01:16.309 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:16.309 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.309 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.309 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:16.310 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.235114+0000", 2024-09-21T15:01:16.310 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1659999999999996e-06 2024-09-21T15:01:16.310 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.310 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.310 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:16.310 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.235122+0000", 2024-09-21T15:01:16.310 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5590000000000006e-06 2024-09-21T15:01:16.310 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.310 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.310 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:16.310 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.235128+0000", 2024-09-21T15:01:16.310 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8579999999999996e-06 2024-09-21T15:01:16.310 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.311 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.311 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:16.311 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.235169+0000", 2024-09-21T15:01:16.311 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1304000000000003e-05 2024-09-21T15:01:16.311 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.311 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.311 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:16.311 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.235178+0000", 2024-09-21T15:01:16.311 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8610000000000004e-06 2024-09-21T15:01:16.311 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.311 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.311 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:16.311 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256286+0000", 2024-09-21T15:01:16.311 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.021108162 2024-09-21T15:01:16.311 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.312 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.312 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:16.312 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256386+0000", 2024-09-21T15:01:16.312 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9871000000000007e-05 2024-09-21T15:01:16.312 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:16.312 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:16.312 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:16.312 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.312 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.312 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140168 3.e 3:76f4e81d:::rbd_data.113664780c85.000000000000006f:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:16.312 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.236992+0000", 2024-09-21T15:01:16.312 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.019036230999999, 2024-09-21T15:01:16.312 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.019513699, 2024-09-21T15:01:16.312 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:16.313 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:16.313 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:16.313 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:16.313 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:16.313 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140168 2024-09-21T15:01:16.313 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.313 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:16.313 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.313 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:16.313 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.236992+0000", 2024-09-21T15:01:16.313 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:16.313 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.314 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.314 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:16.314 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.236992+0000", 2024-09-21T15:01:16.314 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:16.314 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.314 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.314 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:16.314 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.236991+0000", 2024-09-21T15:01:16.314 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:16.314 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.314 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.314 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:16.314 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.236995+0000", 2024-09-21T15:01:16.314 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3100000000000001e-06 2024-09-21T15:01:16.315 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.315 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.315 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:16.315 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237002+0000", 2024-09-21T15:01:16.315 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9929999999999998e-06 2024-09-21T15:01:16.315 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.315 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.315 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:16.315 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237008+0000", 2024-09-21T15:01:16.315 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7559999999999996e-06 2024-09-21T15:01:16.315 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.315 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.315 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:16.315 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237050+0000", 2024-09-21T15:01:16.315 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2852999999999998e-05 2024-09-21T15:01:16.315 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.315 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.316 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:16.316 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237060+0000", 2024-09-21T15:01:16.316 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2399999999999996e-06 2024-09-21T15:01:16.316 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.316 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.316 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:16.316 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256413+0000", 2024-09-21T15:01:16.316 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0193528999999999 2024-09-21T15:01:16.316 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.316 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.316 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:16.316 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256506+0000", 2024-09-21T15:01:16.316 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.3154999999999998e-05 2024-09-21T15:01:16.316 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:16.317 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:16.317 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:16.317 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.317 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.317 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140186 3.e 3:75504345:::rbd_data.113664780c85.00000000000000ed:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:16.317 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.237050+0000", 2024-09-21T15:01:16.317 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.018978260000001, 2024-09-21T15:01:16.317 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0195720150000001, 2024-09-21T15:01:16.317 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:16.317 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:16.317 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:16.317 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:16.317 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:16.317 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140186 2024-09-21T15:01:16.318 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.318 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:16.318 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.318 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:16.318 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237050+0000", 2024-09-21T15:01:16.318 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:16.318 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.318 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.318 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:16.318 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237050+0000", 2024-09-21T15:01:16.318 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:16.318 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.318 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.318 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:16.318 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237049+0000", 2024-09-21T15:01:16.319 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:16.319 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.319 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.319 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:16.319 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237052+0000", 2024-09-21T15:01:16.319 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.016e-06 2024-09-21T15:01:16.319 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.319 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.319 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:16.319 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237070+0000", 2024-09-21T15:01:16.319 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7266999999999999e-05 2024-09-21T15:01:16.319 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.319 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.319 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:16.320 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237075+0000", 2024-09-21T15:01:16.320 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6910000000000002e-06 2024-09-21T15:01:16.320 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.320 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.320 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:16.320 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237232+0000", 2024-09-21T15:01:16.320 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015683599999999999 2024-09-21T15:01:16.320 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.320 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.320 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:16.320 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237237+0000", 2024-09-21T15:01:16.320 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6439999999999999e-06 2024-09-21T15:01:16.320 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.320 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.320 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:16.321 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256534+0000", 2024-09-21T15:01:16.321 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.019296736 2024-09-21T15:01:16.321 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.321 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.321 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:16.321 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256622+0000", 2024-09-21T15:01:16.321 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8386000000000004e-05 2024-09-21T15:01:16.321 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:16.321 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:16.321 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:16.321 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.321 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.321 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140197 3.e 3:71ef5b14:::rbd_data.113664780c85.0000000000000140:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:16.321 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.237251+0000", 2024-09-21T15:01:16.321 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.018777232, 2024-09-21T15:01:16.322 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.01968386, 2024-09-21T15:01:16.322 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:16.322 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:16.322 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:16.322 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:16.322 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:16.322 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140197 2024-09-21T15:01:16.322 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.322 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:16.322 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.322 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:16.322 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237251+0000", 2024-09-21T15:01:16.322 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:16.322 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.322 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.323 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:16.323 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237251+0000", 2024-09-21T15:01:16.323 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:16.323 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.323 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.323 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:16.323 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237250+0000", 2024-09-21T15:01:16.323 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:16.323 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.323 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.323 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:16.323 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237254+0000", 2024-09-21T15:01:16.323 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.0180000000000003e-06 2024-09-21T15:01:16.323 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.323 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.324 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:16.324 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237262+0000", 2024-09-21T15:01:16.324 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0669999999999992e-06 2024-09-21T15:01:16.324 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.324 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.324 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:16.324 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237269+0000", 2024-09-21T15:01:16.324 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6000000000000003e-06 2024-09-21T15:01:16.324 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.324 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.324 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:16.324 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237319+0000", 2024-09-21T15:01:16.324 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9932000000000002e-05 2024-09-21T15:01:16.324 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.325 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.325 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:16.325 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237326+0000", 2024-09-21T15:01:16.325 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5360000000000004e-06 2024-09-21T15:01:16.325 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.325 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.325 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:16.325 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256657+0000", 2024-09-21T15:01:16.325 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.019330756 2024-09-21T15:01:16.325 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.325 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.325 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:16.325 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256935+0000", 2024-09-21T15:01:16.325 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00027762099999999998 2024-09-21T15:01:16.325 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:16.326 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:16.326 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:16.326 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.326 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.326 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140201 3.e 3:732ea575:::rbd_data.113664780c85.000000000000015c:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:16.326 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.237281+0000", 2024-09-21T15:01:16.326 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.018747394, 2024-09-21T15:01:16.326 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0198284529999999, 2024-09-21T15:01:16.326 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:16.326 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:16.326 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:16.326 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:16.326 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:16.326 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140201 2024-09-21T15:01:16.327 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.327 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:16.327 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.327 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:16.327 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237281+0000", 2024-09-21T15:01:16.327 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:16.327 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.327 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.327 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:16.327 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237281+0000", 2024-09-21T15:01:16.327 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:16.327 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.327 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.327 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:16.328 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237280+0000", 2024-09-21T15:01:16.328 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:16.328 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.328 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.328 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:16.328 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237284+0000", 2024-09-21T15:01:16.328 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7400000000000002e-06 2024-09-21T15:01:16.328 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.328 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.328 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:16.328 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237292+0000", 2024-09-21T15:01:16.328 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8939999999999993e-06 2024-09-21T15:01:16.328 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.328 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.328 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:16.329 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237298+0000", 2024-09-21T15:01:16.329 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8000000000000004e-06 2024-09-21T15:01:16.329 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.329 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.329 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:16.329 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237521+0000", 2024-09-21T15:01:16.329 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00022326 2024-09-21T15:01:16.329 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.329 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.329 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:16.329 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237529+0000", 2024-09-21T15:01:16.329 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6170000000000001e-06 2024-09-21T15:01:16.329 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.329 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.330 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:16.330 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256968+0000", 2024-09-21T15:01:16.330 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.019439113 2024-09-21T15:01:16.330 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.330 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.330 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:16.330 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.257109+0000", 2024-09-21T15:01:16.330 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014148399999999999 2024-09-21T15:01:16.330 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:16.330 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:16.330 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:16.330 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.330 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.330 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140206 3.e 3:76d38029:::rbd_data.113664780c85.0000000000000179:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:16.331 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.237520+0000", 2024-09-21T15:01:16.331 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.018508280000001, 2024-09-21T15:01:16.331 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0210658699999999, 2024-09-21T15:01:16.331 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:16.331 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:16.331 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:16.331 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:16.331 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:16.331 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140206 2024-09-21T15:01:16.331 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.331 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:16.331 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.331 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:16.331 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237520+0000", 2024-09-21T15:01:16.332 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:16.332 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.332 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.332 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:16.332 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237520+0000", 2024-09-21T15:01:16.332 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:16.332 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.332 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.332 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:16.332 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237519+0000", 2024-09-21T15:01:16.332 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:16.332 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.332 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.332 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:16.332 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237523+0000", 2024-09-21T15:01:16.333 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.9009999999999996e-06 2024-09-21T15:01:16.333 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.333 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.333 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:16.333 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237531+0000", 2024-09-21T15:01:16.333 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7069999999999992e-06 2024-09-21T15:01:16.333 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.333 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.333 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:16.333 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237537+0000", 2024-09-21T15:01:16.333 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4559999999999997e-06 2024-09-21T15:01:16.333 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.333 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.333 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:16.333 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237583+0000", 2024-09-21T15:01:16.334 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5532999999999999e-05 2024-09-21T15:01:16.334 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.334 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.334 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:16.334 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237592+0000", 2024-09-21T15:01:16.334 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6009999999999995e-06 2024-09-21T15:01:16.334 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.334 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.334 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:16.334 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.257157+0000", 2024-09-21T15:01:16.334 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0195657279999999 2024-09-21T15:01:16.334 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.335 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.335 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:16.335 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.258586+0000", 2024-09-21T15:01:16.335 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00142847 2024-09-21T15:01:16.335 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:16.335 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:16.335 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:16.335 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.335 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.335 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140338 3.e 3:71dba096:::rbd_data.113664780c85.000000000000018b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:16.335 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.243856+0000", 2024-09-21T15:01:16.335 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.012172446999999, 2024-09-21T15:01:16.335 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0138226889999999, 2024-09-21T15:01:16.335 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:16.336 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:16.336 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:16.336 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:16.336 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:16.336 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140338 2024-09-21T15:01:16.336 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.336 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:16.336 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.336 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:16.336 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243856+0000", 2024-09-21T15:01:16.336 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:16.336 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.336 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.336 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:16.336 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243856+0000", 2024-09-21T15:01:16.337 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:16.337 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.337 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.337 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:16.337 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243854+0000", 2024-09-21T15:01:16.337 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:16.337 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.337 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.337 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:16.337 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243862+0000", 2024-09-21T15:01:16.337 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2570000000000001e-06 2024-09-21T15:01:16.337 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.337 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.337 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:16.337 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243877+0000", 2024-09-21T15:01:16.338 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.571e-05 2024-09-21T15:01:16.338 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.338 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.338 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:16.338 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243884+0000", 2024-09-21T15:01:16.338 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.1269999999999997e-06 2024-09-21T15:01:16.338 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.338 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.338 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:16.338 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243932+0000", 2024-09-21T15:01:16.338 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7655999999999999e-05 2024-09-21T15:01:16.338 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.338 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.338 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:16.339 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243942+0000", 2024-09-21T15:01:16.339 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6679999999999996e-06 2024-09-21T15:01:16.339 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.339 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.339 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:16.339 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.257568+0000", 2024-09-21T15:01:16.339 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0136267670000001 2024-09-21T15:01:16.339 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.339 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.339 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:16.339 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.257678+0000", 2024-09-21T15:01:16.339 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000110011 2024-09-21T15:01:16.339 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:16.339 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:16.339 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:16.339 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.340 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.340 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140345 3.e 3:76ddeb1d:::rbd_data.113664780c85.0000000000000004:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:16.340 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.243927+0000", 2024-09-21T15:01:16.340 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.012100863000001, 2024-09-21T15:01:16.340 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0140328439999999, 2024-09-21T15:01:16.340 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:16.340 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:16.340 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:16.340 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:16.340 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:16.340 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140345 2024-09-21T15:01:16.340 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.340 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:16.340 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.341 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:16.341 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243927+0000", 2024-09-21T15:01:16.341 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:16.341 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.341 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.341 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:16.341 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243927+0000", 2024-09-21T15:01:16.341 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:16.341 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.341 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.341 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:16.341 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243927+0000", 2024-09-21T15:01:16.341 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:16.341 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.341 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.342 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:16.342 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243930+0000", 2024-09-21T15:01:16.342 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1319999999999998e-06 2024-09-21T15:01:16.342 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.342 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.342 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:16.342 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243937+0000", 2024-09-21T15:01:16.342 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4330000000000001e-06 2024-09-21T15:01:16.342 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.342 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.342 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:16.342 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243942+0000", 2024-09-21T15:01:16.342 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5090000000000001e-06 2024-09-21T15:01:16.342 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.343 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.343 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:16.343 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243964+0000", 2024-09-21T15:01:16.343 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1838999999999999e-05 2024-09-21T15:01:16.343 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.343 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.343 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:16.343 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243971+0000", 2024-09-21T15:01:16.343 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.131e-06 2024-09-21T15:01:16.343 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.343 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.343 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:16.343 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.257704+0000", 2024-09-21T15:01:16.343 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0137329310000001 2024-09-21T15:01:16.343 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.344 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.344 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:16.344 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.257805+0000", 2024-09-21T15:01:16.344 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000101121 2024-09-21T15:01:16.344 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.344 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.344 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:16.344 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.257960+0000", 2024-09-21T15:01:16.344 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015518500000000001 2024-09-21T15:01:16.344 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:16.344 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:16.344 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:16.344 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.345 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.345 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140350 3.e 3:76b9b799:::rbd_data.113664780c85.0000000000000021:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:16.345 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.243989+0000", 2024-09-21T15:01:16.345 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.012039112, 2024-09-21T15:01:16.345 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0142022340000001, 2024-09-21T15:01:16.345 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:16.345 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:16.345 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:16.345 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:16.345 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:16.345 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140350 2024-09-21T15:01:16.345 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.345 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:16.346 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.346 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:16.346 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243989+0000", 2024-09-21T15:01:16.346 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:16.346 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.346 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.346 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:16.346 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243989+0000", 2024-09-21T15:01:16.346 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:16.346 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.346 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.346 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:16.346 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243989+0000", 2024-09-21T15:01:16.346 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:16.347 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.347 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.347 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:16.347 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243992+0000", 2024-09-21T15:01:16.347 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.236e-06 2024-09-21T15:01:16.347 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.347 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.347 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:16.347 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243998+0000", 2024-09-21T15:01:16.347 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2500000000000003e-06 2024-09-21T15:01:16.347 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.347 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.347 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:16.347 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.244003+0000", 2024-09-21T15:01:16.348 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7740000000000004e-06 2024-09-21T15:01:16.348 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.348 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.348 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:16.348 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.244047+0000", 2024-09-21T15:01:16.348 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3665999999999998e-05 2024-09-21T15:01:16.348 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.348 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.348 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:16.348 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.244055+0000", 2024-09-21T15:01:16.348 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0639999999999994e-06 2024-09-21T15:01:16.348 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.348 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.348 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:16.348 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.257985+0000", 2024-09-21T15:01:16.349 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0139306079999999 2024-09-21T15:01:16.349 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.349 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.349 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:16.349 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.258084+0000", 2024-09-21T15:01:16.349 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8610000000000006e-05 2024-09-21T15:01:16.349 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.349 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.349 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:16.349 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.258191+0000", 2024-09-21T15:01:16.349 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000107518 2024-09-21T15:01:16.349 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:16.349 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:16.349 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:16.349 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.349 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.350 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140394 3.e 3:7756a55d:::rbd_data.113664780c85.0000000000000154:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:16.350 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.245038+0000", 2024-09-21T15:01:16.350 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 14.010989895, 2024-09-21T15:01:16.350 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.013225182, 2024-09-21T15:01:16.350 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:16.350 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:16.350 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:16.350 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:16.350 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:16.350 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140394 2024-09-21T15:01:16.350 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.350 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:16.350 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.350 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:16.351 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.245038+0000", 2024-09-21T15:01:16.351 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:16.351 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.351 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.351 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:16.351 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.245038+0000", 2024-09-21T15:01:16.351 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:16.351 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.351 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.351 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:16.351 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.245037+0000", 2024-09-21T15:01:16.351 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:16.351 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.351 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.351 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:16.352 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.245044+0000", 2024-09-21T15:01:16.352 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3239999999999999e-06 2024-09-21T15:01:16.352 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.352 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.352 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:16.352 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.245061+0000", 2024-09-21T15:01:16.352 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7166000000000001e-05 2024-09-21T15:01:16.352 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.352 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.352 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:16.352 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.245069+0000", 2024-09-21T15:01:16.352 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0779999999999996e-06 2024-09-21T15:01:16.352 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.352 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.352 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:16.352 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.245118+0000", 2024-09-21T15:01:16.353 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8887000000000001e-05 2024-09-21T15:01:16.353 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.353 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.353 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:16.353 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.245126+0000", 2024-09-21T15:01:16.353 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0409999999999998e-06 2024-09-21T15:01:16.353 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.353 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.353 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:16.353 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.258206+0000", 2024-09-21T15:01:16.353 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.013080451 2024-09-21T15:01:16.353 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.353 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.353 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:16.353 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.258264+0000", 2024-09-21T15:01:16.354 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7294999999999999e-05 2024-09-21T15:01:16.354 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:16.354 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:16.354 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:16.354 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:16.354 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:16.354 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:16.354 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T15:01:16.356 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T15:01:16.356 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:01:16.350+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:01:16.456 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:01:16.450+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:01:16.539 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:16.539 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:01:16.539 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T15:01:16.539 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:16.549 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T15:01:16.557 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:01:16.553+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:01:16.646 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:16.646 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:01:16.646 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T15:01:16.646 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T15:01:16.646 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:16.656 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T15:01:16.657 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:01:16.654+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:01:16.758 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:01:16.754+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:01:16.773 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:16.773 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T15:01:16.773 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T15:01:16.774 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T15:01:16.774 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.774 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T15:01:16.774 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.205754+0000", 2024-09-21T15:01:16.774 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 193.56621831300001, 2024-09-21T15:01:16.774 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.41117094199999998, 2024-09-21T15:01:16.774 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:16.774 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:16.774 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:16.774 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T15:01:16.774 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T15:01:16.775 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1 2024-09-21T15:01:16.775 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.775 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:16.775 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.775 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:16.775 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205754+0000", 2024-09-21T15:01:16.775 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:16.775 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.775 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.775 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:16.775 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205754+0000", 2024-09-21T15:01:16.775 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:16.776 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.776 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.776 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:16.776 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205752+0000", 2024-09-21T15:01:16.776 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T15:01:16.776 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.776 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.776 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:16.776 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205765+0000", 2024-09-21T15:01:16.776 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3230999999999999e-05 2024-09-21T15:01:16.776 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.776 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.776 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:16.776 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205782+0000", 2024-09-21T15:01:16.777 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6634e-05 2024-09-21T15:01:16.777 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.777 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.777 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:16.777 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205793+0000", 2024-09-21T15:01:16.777 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1297e-05 2024-09-21T15:01:16.777 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.777 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.777 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:16.777 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205867+0000", 2024-09-21T15:01:16.777 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4269000000000003e-05 2024-09-21T15:01:16.777 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.777 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.778 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:16.778 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205877+0000", 2024-09-21T15:01:16.778 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2690000000000005e-06 2024-09-21T15:01:16.778 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.778 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.778 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:16.778 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.613700+0000", 2024-09-21T15:01:16.778 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.407823455 2024-09-21T15:01:16.778 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.778 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.778 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:16.779 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.614203+0000", 2024-09-21T15:01:16.779 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00050311500000000005 2024-09-21T15:01:16.779 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.779 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.779 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T15:01:16.779 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.614750+0000", 2024-09-21T15:01:16.779 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00054738699999999998 2024-09-21T15:01:16.779 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.779 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.779 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:01:16.779 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616367+0000", 2024-09-21T15:01:16.779 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001616634 2024-09-21T15:01:16.779 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.779 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.780 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:01:16.780 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616574+0000", 2024-09-21T15:01:16.780 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00020668499999999999 2024-09-21T15:01:16.780 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.780 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.780 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:01:16.780 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616802+0000", 2024-09-21T15:01:16.780 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00022817900000000001 2024-09-21T15:01:16.780 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.780 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.780 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:16.780 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616925+0000", 2024-09-21T15:01:16.781 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012309699999999999 2024-09-21T15:01:16.781 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:16.781 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:16.781 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:16.781 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.781 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.781 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:61214 3.7 3:e28aec10:::rbd_header.1136862f872f:head [watch reconnect cookie 139864671932080 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-21T15:01:16.781 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:18.147172+0000", 2024-09-21T15:01:16.781 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 118.62480067, 2024-09-21T15:01:16.781 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.019264956, 2024-09-21T15:01:16.781 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:16.781 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:16.781 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:16.781 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:16.782 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:16.782 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 61214 2024-09-21T15:01:16.782 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.782 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:16.782 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.782 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:16.782 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147172+0000", 2024-09-21T15:01:16.782 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:16.782 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.783 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.783 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:16.783 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147172+0000", 2024-09-21T15:01:16.783 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:16.783 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.783 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.784 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:16.784 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147168+0000", 2024-09-21T15:01:16.784 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999962 2024-09-21T15:01:16.784 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.784 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.784 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:16.784 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147188+0000", 2024-09-21T15:01:16.784 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9717000000000001e-05 2024-09-21T15:01:16.785 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.785 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.785 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:16.785 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147239+0000", 2024-09-21T15:01:16.785 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1493999999999999e-05 2024-09-21T15:01:16.785 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.785 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.785 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:16.785 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147265+0000", 2024-09-21T15:01:16.785 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5746999999999999e-05 2024-09-21T15:01:16.785 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.785 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.785 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:16.785 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147372+0000", 2024-09-21T15:01:16.786 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010681200000000001 2024-09-21T15:01:16.786 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.786 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.786 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op must wait for map", 2024-09-21T15:01:16.786 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147380+0000", 2024-09-21T15:01:16.786 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7919999999999993e-06 2024-09-21T15:01:16.786 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.786 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.786 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:16.786 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.154025+0000", 2024-09-21T15:01:16.786 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0066454909999999999 2024-09-21T15:01:16.787 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.787 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.787 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:16.787 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.154041+0000", 2024-09-21T15:01:16.787 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5894000000000001e-05 2024-09-21T15:01:16.787 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.787 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.787 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:16.787 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.164345+0000", 2024-09-21T15:01:16.787 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0103040459999999 2024-09-21T15:01:16.787 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.787 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.788 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:16.788 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.164590+0000", 2024-09-21T15:01:16.788 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00024492699999999998 2024-09-21T15:01:16.788 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.788 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.788 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:16.788 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.166437+0000", 2024-09-21T15:01:16.788 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0018466330000000001 2024-09-21T15:01:16.788 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:16.788 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:16.788 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:16.788 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.788 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.788 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:61215 3.7 3:e28aec10:::rbd_header.1136862f872f:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-21T15:01:16.789 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:18.152008+0000", 2024-09-21T15:01:16.789 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 118.61996421800001, 2024-09-21T15:01:16.789 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.015644958, 2024-09-21T15:01:16.789 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:16.789 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:16.789 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:16.789 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:16.789 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:16.789 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 61215 2024-09-21T15:01:16.789 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.789 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:16.789 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.790 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:16.790 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152008+0000", 2024-09-21T15:01:16.790 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:16.790 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.790 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.790 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:16.790 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152008+0000", 2024-09-21T15:01:16.790 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:16.790 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.790 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.790 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:16.790 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152005+0000", 2024-09-21T15:01:16.790 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-21T15:01:16.791 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.791 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.791 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:16.791 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152025+0000", 2024-09-21T15:01:16.791 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9306000000000001e-05 2024-09-21T15:01:16.791 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.791 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.791 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:16.791 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152054+0000", 2024-09-21T15:01:16.791 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9189000000000001e-05 2024-09-21T15:01:16.791 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.791 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.792 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:16.792 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152078+0000", 2024-09-21T15:01:16.792 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4317000000000001e-05 2024-09-21T15:01:16.792 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.792 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.792 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:16.792 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152243+0000", 2024-09-21T15:01:16.792 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000164802 2024-09-21T15:01:16.792 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.792 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.792 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting_for_map not empty", 2024-09-21T15:01:16.792 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152251+0000", 2024-09-21T15:01:16.792 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0600000000000008e-06 2024-09-21T15:01:16.792 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.793 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.793 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:16.793 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.154741+0000", 2024-09-21T15:01:16.793 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0024900069999999998 2024-09-21T15:01:16.793 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.793 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.793 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:16.794 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.154753+0000", 2024-09-21T15:01:16.794 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1588e-05 2024-09-21T15:01:16.794 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.794 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.794 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:16.794 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.165380+0000", 2024-09-21T15:01:16.794 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0106272249999999 2024-09-21T15:01:16.795 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.795 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.795 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:16.795 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.165517+0000", 2024-09-21T15:01:16.795 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013737999999999999 2024-09-21T15:01:16.795 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.795 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.795 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 6", 2024-09-21T15:01:16.795 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.165938+0000", 2024-09-21T15:01:16.795 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00042082399999999998 2024-09-21T15:01:16.795 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.796 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.796 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:01:16.796 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.166795+0000", 2024-09-21T15:01:16.796 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00085674899999999997 2024-09-21T15:01:16.796 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.796 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.796 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:01:16.796 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.167598+0000", 2024-09-21T15:01:16.796 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00080335599999999997 2024-09-21T15:01:16.796 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.796 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.796 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:01:16.796 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.167632+0000", 2024-09-21T15:01:16.796 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4122000000000003e-05 2024-09-21T15:01:16.796 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.796 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.797 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:16.797 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.167653+0000", 2024-09-21T15:01:16.797 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0862000000000001e-05 2024-09-21T15:01:16.797 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:16.797 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:16.797 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:16.797 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.797 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.797 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:97678 3.d e122/24 3:b2f34c6d:::rbd_data.1136bfc79a70.0000000000000006:head v 122'234, mlcod=120'232)", 2024-09-21T15:01:16.797 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.321655+0000", 2024-09-21T15:01:16.797 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 71.450317300999998, 2024-09-21T15:01:16.797 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024509388999999999, 2024-09-21T15:01:16.797 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:16.797 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:16.797 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:16.798 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.798 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:16.798 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.321655+0000", 2024-09-21T15:01:16.798 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:16.798 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.798 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.798 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:16.798 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.321655+0000", 2024-09-21T15:01:16.798 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:16.798 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.798 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.798 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:16.798 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.321655+0000", 2024-09-21T15:01:16.798 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:16.798 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.798 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.799 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:16.799 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.329453+0000", 2024-09-21T15:01:16.799 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0077980560000000003 2024-09-21T15:01:16.799 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.799 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.799 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:16.799 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.329482+0000", 2024-09-21T15:01:16.799 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9153999999999999e-05 2024-09-21T15:01:16.799 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.799 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.799 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:16.799 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.329493+0000", 2024-09-21T15:01:16.799 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1103e-05 2024-09-21T15:01:16.800 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.800 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.800 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:16.800 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.329555+0000", 2024-09-21T15:01:16.800 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2345e-05 2024-09-21T15:01:16.800 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.800 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.800 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:16.800 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.329589+0000", 2024-09-21T15:01:16.800 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3908999999999997e-05 2024-09-21T15:01:16.800 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.800 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.800 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:01:16.800 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.346083+0000", 2024-09-21T15:01:16.800 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016493383 2024-09-21T15:01:16.800 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.801 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.801 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:16.801 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.346165+0000", 2024-09-21T15:01:16.801 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1796000000000001e-05 2024-09-21T15:01:16.801 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:16.801 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:16.801 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:16.801 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.801 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.801 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:97679 3.f e122/24 3:f2a362b4:::rbd_data.1136bfc79a70.0000000000000007:head v 122'331, mlcod=120'329)", 2024-09-21T15:01:16.801 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.337145+0000", 2024-09-21T15:01:16.801 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 71.434827291999994, 2024-09-21T15:01:16.801 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021912411, 2024-09-21T15:01:16.801 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:16.801 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:16.802 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:16.802 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.802 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:16.802 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.337145+0000", 2024-09-21T15:01:16.802 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:16.802 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.802 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.802 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:16.802 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.337145+0000", 2024-09-21T15:01:16.802 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:16.802 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.802 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.802 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:16.802 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.337145+0000", 2024-09-21T15:01:16.802 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:16.803 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.803 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.803 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:16.803 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.343899+0000", 2024-09-21T15:01:16.803 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0067541420000000003 2024-09-21T15:01:16.803 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.803 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.803 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:16.803 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.343921+0000", 2024-09-21T15:01:16.803 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2255e-05 2024-09-21T15:01:16.803 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.803 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.804 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:16.804 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.343933+0000", 2024-09-21T15:01:16.804 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1528e-05 2024-09-21T15:01:16.804 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.804 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.804 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:16.804 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.344001+0000", 2024-09-21T15:01:16.804 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8269999999999995e-05 2024-09-21T15:01:16.804 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.804 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.804 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:16.804 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.344029+0000", 2024-09-21T15:01:16.804 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8254000000000001e-05 2024-09-21T15:01:16.804 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.804 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.805 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:01:16.805 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.358901+0000", 2024-09-21T15:01:16.805 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0148719 2024-09-21T15:01:16.805 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.805 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.805 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:16.805 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.359058+0000", 2024-09-21T15:01:16.805 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000156446 2024-09-21T15:01:16.805 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:16.805 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:16.805 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:16.805 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.805 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.805 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:97676 3.1f e122/24 3:ffaa836a:::rbd_data.1136bfc79a70.0000000000000004:head v 122'312, mlcod=120'310)", 2024-09-21T15:01:16.805 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.348683+0000", 2024-09-21T15:01:16.806 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 71.423289127000004, 2024-09-21T15:01:16.806 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.033768802000000001, 2024-09-21T15:01:16.806 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:16.806 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:16.806 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:16.806 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.806 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:16.806 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.348683+0000", 2024-09-21T15:01:16.806 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:16.806 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.806 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.806 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:16.806 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.348683+0000", 2024-09-21T15:01:16.806 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:16.806 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.806 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.807 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:16.807 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.348683+0000", 2024-09-21T15:01:16.807 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:16.807 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.807 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.807 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:16.807 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.366937+0000", 2024-09-21T15:01:16.807 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018254415 2024-09-21T15:01:16.807 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.807 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.807 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:16.807 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.366959+0000", 2024-09-21T15:01:16.807 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1634e-05 2024-09-21T15:01:16.807 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.807 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.808 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:16.808 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.366971+0000", 2024-09-21T15:01:16.808 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2305e-05 2024-09-21T15:01:16.808 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.808 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.808 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:16.808 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.367034+0000", 2024-09-21T15:01:16.808 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3211000000000002e-05 2024-09-21T15:01:16.808 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.808 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.808 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:16.808 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.367056+0000", 2024-09-21T15:01:16.808 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2098000000000001e-05 2024-09-21T15:01:16.809 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.809 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.809 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:01:16.809 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.382400+0000", 2024-09-21T15:01:16.809 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.015343867000000001 2024-09-21T15:01:16.809 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.809 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.809 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:16.809 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.382452+0000", 2024-09-21T15:01:16.809 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1992e-05 2024-09-21T15:01:16.809 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:16.809 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:16.809 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:16.809 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.809 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.809 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:97734 3.19 e122/94 3:9fcf754f:::rbd_data.1136bfc79a70.0000000000000009:head v 122'314, mlcod=120'312)", 2024-09-21T15:01:16.810 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.378547+0000", 2024-09-21T15:01:16.810 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 71.393425148999995, 2024-09-21T15:01:16.810 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.063380675999999997, 2024-09-21T15:01:16.810 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:16.810 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:16.810 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:16.810 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.810 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:16.810 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.378547+0000", 2024-09-21T15:01:16.810 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:16.810 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.810 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.810 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:16.810 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.378547+0000", 2024-09-21T15:01:16.811 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:16.811 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.811 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.811 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:16.811 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.378547+0000", 2024-09-21T15:01:16.811 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:01:16.811 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.811 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.811 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:16.811 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.425118+0000", 2024-09-21T15:01:16.811 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046571166999999997 2024-09-21T15:01:16.811 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.811 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.811 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:16.811 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.425150+0000", 2024-09-21T15:01:16.812 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2150999999999998e-05 2024-09-21T15:01:16.812 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.812 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.812 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:16.812 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.425164+0000", 2024-09-21T15:01:16.812 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3302000000000001e-05 2024-09-21T15:01:16.812 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.812 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.812 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:16.812 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.425231+0000", 2024-09-21T15:01:16.812 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7106000000000004e-05 2024-09-21T15:01:16.812 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.812 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.812 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:16.812 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.425259+0000", 2024-09-21T15:01:16.812 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8087000000000001e-05 2024-09-21T15:01:16.813 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.813 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.813 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:01:16.813 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.441890+0000", 2024-09-21T15:01:16.813 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016631328000000001 2024-09-21T15:01:16.813 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.813 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.813 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:16.813 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.441928+0000", 2024-09-21T15:01:16.813 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7701999999999999e-05 2024-09-21T15:01:16.813 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:16.813 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:16.813 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:16.813 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.813 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.814 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:100187 3.10 3:08941a97:::rbd_data.1136ab4d6ff1.000000000000000e:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-21T15:01:16.814 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:10.869845+0000", 2024-09-21T15:01:16.814 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 65.902127253000003, 2024-09-21T15:01:16.814 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029345237, 2024-09-21T15:01:16.814 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:16.814 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:16.814 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:16.814 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:16.814 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:16.814 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 100187 2024-09-21T15:01:16.814 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.814 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:16.814 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.814 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:16.815 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869845+0000", 2024-09-21T15:01:16.815 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:16.815 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.815 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.815 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:16.815 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869845+0000", 2024-09-21T15:01:16.815 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:16.815 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.815 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.815 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:16.815 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869844+0000", 2024-09-21T15:01:16.815 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:16.815 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.815 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.816 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:16.816 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869853+0000", 2024-09-21T15:01:16.816 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5719999999999992e-06 2024-09-21T15:01:16.816 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.816 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.816 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:16.816 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869874+0000", 2024-09-21T15:01:16.816 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0483e-05 2024-09-21T15:01:16.816 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.816 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.816 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:16.816 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869883+0000", 2024-09-21T15:01:16.816 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6080000000000002e-06 2024-09-21T15:01:16.816 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.816 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.817 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:16.817 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869991+0000", 2024-09-21T15:01:16.817 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000108013 2024-09-21T15:01:16.817 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.817 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.817 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:16.817 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.870258+0000", 2024-09-21T15:01:16.817 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00026629099999999999 2024-09-21T15:01:16.817 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.817 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.817 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T15:01:16.817 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.878656+0000", 2024-09-21T15:01:16.817 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0083989000000000008 2024-09-21T15:01:16.817 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.818 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.818 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:01:16.818 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.899081+0000", 2024-09-21T15:01:16.818 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.020424437 2024-09-21T15:01:16.818 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.818 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.818 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:01:16.818 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.899125+0000", 2024-09-21T15:01:16.818 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4440999999999997e-05 2024-09-21T15:01:16.818 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.818 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.818 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:01:16.818 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.899160+0000", 2024-09-21T15:01:16.819 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4518999999999999e-05 2024-09-21T15:01:16.819 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.819 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.819 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:16.819 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.899190+0000", 2024-09-21T15:01:16.819 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0574999999999998e-05 2024-09-21T15:01:16.819 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:16.819 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:16.819 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:16.819 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.819 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.819 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:100263 3.10 3:0ece142e:::rbd_data.1136ab4d6ff1.0000000000000008:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-21T15:01:16.819 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:11.052257+0000", 2024-09-21T15:01:16.820 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 65.719715195000006, 2024-09-21T15:01:16.820 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042678025000000001, 2024-09-21T15:01:16.820 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:16.820 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:16.820 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:16.820 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:16.820 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:16.820 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 100263 2024-09-21T15:01:16.820 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.820 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:16.820 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.820 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:16.820 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052257+0000", 2024-09-21T15:01:16.820 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:16.821 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.821 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.821 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:16.821 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052257+0000", 2024-09-21T15:01:16.821 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:16.821 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.821 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.821 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:16.821 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052256+0000", 2024-09-21T15:01:16.821 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:16.821 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.821 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.821 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:16.821 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052263+0000", 2024-09-21T15:01:16.822 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4799999999999998e-06 2024-09-21T15:01:16.822 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.822 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.822 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:16.822 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052275+0000", 2024-09-21T15:01:16.822 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2386e-05 2024-09-21T15:01:16.822 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.822 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.822 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:16.822 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052281+0000", 2024-09-21T15:01:16.822 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6699999999999999e-06 2024-09-21T15:01:16.822 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.823 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.823 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:16.823 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052350+0000", 2024-09-21T15:01:16.823 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9099999999999999e-05 2024-09-21T15:01:16.823 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.823 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.823 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:16.823 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052456+0000", 2024-09-21T15:01:16.823 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000106669 2024-09-21T15:01:16.823 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.823 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.823 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T15:01:16.823 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.059694+0000", 2024-09-21T15:01:16.823 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.007237566 2024-09-21T15:01:16.823 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.824 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.824 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:01:16.824 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.094721+0000", 2024-09-21T15:01:16.824 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035026977000000001 2024-09-21T15:01:16.824 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.824 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.824 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:01:16.824 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.094746+0000", 2024-09-21T15:01:16.824 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5329000000000001e-05 2024-09-21T15:01:16.824 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.824 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.824 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:01:16.824 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.094850+0000", 2024-09-21T15:01:16.824 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000103452 2024-09-21T15:01:16.825 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.825 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.825 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:16.825 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.094935+0000", 2024-09-21T15:01:16.825 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5550000000000006e-05 2024-09-21T15:01:16.825 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:16.825 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:16.825 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:16.825 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.825 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.825 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:134708 3.a 3:51eaadc0:::rbd_data.11365ed419e3.0000000000000007:67 [read 221184~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:16.825 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.938945+0000", 2024-09-21T15:01:16.825 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 22.833027916999999, 2024-09-21T15:01:16.826 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021073784000000002, 2024-09-21T15:01:16.826 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:16.826 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:16.826 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:16.826 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:16.826 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:16.826 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 134708 2024-09-21T15:01:16.826 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.826 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:16.826 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.826 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:16.826 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938945+0000", 2024-09-21T15:01:16.826 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:16.827 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.827 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.827 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:16.827 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938945+0000", 2024-09-21T15:01:16.827 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:16.827 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.827 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.827 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:16.827 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938944+0000", 2024-09-21T15:01:16.827 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:16.827 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.827 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.827 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:16.827 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938946+0000", 2024-09-21T15:01:16.827 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.616e-06 2024-09-21T15:01:16.827 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.828 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.828 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:16.828 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938952+0000", 2024-09-21T15:01:16.828 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7309999999999998e-06 2024-09-21T15:01:16.828 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.828 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.828 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:16.828 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938957+0000", 2024-09-21T15:01:16.828 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8579999999999999e-06 2024-09-21T15:01:16.828 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.828 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.828 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:16.828 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.959730+0000", 2024-09-21T15:01:16.828 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.020773183000000001 2024-09-21T15:01:16.828 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.828 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.829 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:16.829 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.959810+0000", 2024-09-21T15:01:16.829 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0158000000000001e-05 2024-09-21T15:01:16.829 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.829 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.829 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:16.829 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.960018+0000", 2024-09-21T15:01:16.829 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000207951 2024-09-21T15:01:16.829 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:16.829 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:16.829 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:16.829 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.829 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.829 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:134709 3.a 3:51eaadc0:::rbd_data.11365ed419e3.0000000000000007:67 [read 237568~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:16.829 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.938996+0000", 2024-09-21T15:01:16.829 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 22.832976925000001, 2024-09-21T15:01:16.830 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021382577, 2024-09-21T15:01:16.830 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:16.830 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:16.830 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:16.830 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:16.830 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:16.830 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 134709 2024-09-21T15:01:16.830 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.830 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:16.830 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.830 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:16.830 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938996+0000", 2024-09-21T15:01:16.830 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:16.830 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.830 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.831 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:16.831 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938996+0000", 2024-09-21T15:01:16.831 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:16.831 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.831 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.831 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:16.831 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938995+0000", 2024-09-21T15:01:16.831 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:16.831 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.831 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.831 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:16.831 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938999+0000", 2024-09-21T15:01:16.831 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7979999999999999e-06 2024-09-21T15:01:16.831 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.831 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.832 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:16.832 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939006+0000", 2024-09-21T15:01:16.832 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6669999999999996e-06 2024-09-21T15:01:16.832 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.832 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.832 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:16.832 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939012+0000", 2024-09-21T15:01:16.832 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9329999999999998e-06 2024-09-21T15:01:16.832 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.832 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.832 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:16.832 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.960080+0000", 2024-09-21T15:01:16.832 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021068303 2024-09-21T15:01:16.832 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.832 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.832 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:16.833 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.960154+0000", 2024-09-21T15:01:16.833 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3819000000000006e-05 2024-09-21T15:01:16.833 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.833 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.833 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:16.833 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.960378+0000", 2024-09-21T15:01:16.833 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00022385800000000001 2024-09-21T15:01:16.833 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:16.833 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:16.833 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:16.833 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.833 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.833 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:134710 3.a 3:51eaadc0:::rbd_data.11365ed419e3.0000000000000007:67 [read 253952~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:16.833 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.939017+0000", 2024-09-21T15:01:16.833 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 22.832955913999999, 2024-09-21T15:01:16.834 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021721436, 2024-09-21T15:01:16.834 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:16.834 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:16.834 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:16.834 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:16.834 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:16.834 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 134710 2024-09-21T15:01:16.834 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.834 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:16.834 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.834 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:16.834 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939017+0000", 2024-09-21T15:01:16.834 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:16.835 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.835 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.835 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:16.835 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939017+0000", 2024-09-21T15:01:16.835 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:16.835 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.835 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.835 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:16.835 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939016+0000", 2024-09-21T15:01:16.835 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:16.835 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.835 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.835 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:16.835 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939019+0000", 2024-09-21T15:01:16.835 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.554e-06 2024-09-21T15:01:16.835 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.836 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.836 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:16.836 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939024+0000", 2024-09-21T15:01:16.836 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1569999999999999e-06 2024-09-21T15:01:16.836 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.836 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.836 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:16.836 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939034+0000", 2024-09-21T15:01:16.836 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9219999999999992e-06 2024-09-21T15:01:16.836 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.836 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.836 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:16.836 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.960408+0000", 2024-09-21T15:01:16.836 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021373792999999999 2024-09-21T15:01:16.836 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.837 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.837 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:16.837 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.960482+0000", 2024-09-21T15:01:16.837 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4331000000000004e-05 2024-09-21T15:01:16.837 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.837 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.837 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:16.837 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.960738+0000", 2024-09-21T15:01:16.837 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00025600500000000001 2024-09-21T15:01:16.837 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:16.837 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:16.837 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:16.837 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.837 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.837 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:134711 3.a 3:51eaadc0:::rbd_data.11365ed419e3.0000000000000007:67 [read 270336~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:16.838 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.939039+0000", 2024-09-21T15:01:16.838 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 22.832933865000001, 2024-09-21T15:01:16.838 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022064698000000001, 2024-09-21T15:01:16.838 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:16.838 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:16.838 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:16.838 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:16.838 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:16.838 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 134711 2024-09-21T15:01:16.838 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.838 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:16.838 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.838 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:16.838 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939039+0000", 2024-09-21T15:01:16.838 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:16.839 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.839 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.839 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:16.839 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939039+0000", 2024-09-21T15:01:16.839 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:16.839 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.839 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.839 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:16.839 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939038+0000", 2024-09-21T15:01:16.839 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:16.839 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.839 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.839 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:16.839 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939041+0000", 2024-09-21T15:01:16.839 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9440000000000001e-06 2024-09-21T15:01:16.840 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.840 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.840 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:16.840 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939047+0000", 2024-09-21T15:01:16.840 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1260000000000003e-06 2024-09-21T15:01:16.840 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.840 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.840 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:16.840 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939052+0000", 2024-09-21T15:01:16.840 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1619999999999999e-06 2024-09-21T15:01:16.840 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.840 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.840 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:16.840 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.960768+0000", 2024-09-21T15:01:16.840 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021715140000000001 2024-09-21T15:01:16.841 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.841 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.841 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:16.841 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.960842+0000", 2024-09-21T15:01:16.841 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4698999999999997e-05 2024-09-21T15:01:16.841 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.841 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.841 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:16.841 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.961103+0000", 2024-09-21T15:01:16.841 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00026098499999999998 2024-09-21T15:01:16.841 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:16.841 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:16.841 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:16.841 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.841 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.841 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:134712 3.a 3:51eaadc0:::rbd_data.11365ed419e3.0000000000000007:67 [read 286720~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:16.841 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.939057+0000", 2024-09-21T15:01:16.842 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 22.832915331999999, 2024-09-21T15:01:16.842 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022387067, 2024-09-21T15:01:16.842 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:16.842 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:16.842 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:16.842 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:16.842 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:16.842 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 134712 2024-09-21T15:01:16.842 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.842 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:16.842 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.842 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:16.842 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939057+0000", 2024-09-21T15:01:16.842 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:16.843 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.843 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.843 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:16.843 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939057+0000", 2024-09-21T15:01:16.843 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:16.843 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.843 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.843 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:16.843 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939057+0000", 2024-09-21T15:01:16.843 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:16.843 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.843 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.843 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:16.843 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939059+0000", 2024-09-21T15:01:16.844 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.447e-06 2024-09-21T15:01:16.844 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.844 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.844 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:16.844 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939065+0000", 2024-09-21T15:01:16.844 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4670000000000003e-06 2024-09-21T15:01:16.844 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.844 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.844 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:16.844 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939069+0000", 2024-09-21T15:01:16.844 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2119999999999997e-06 2024-09-21T15:01:16.844 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.844 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.844 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:16.845 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.961141+0000", 2024-09-21T15:01:16.845 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022072268999999999 2024-09-21T15:01:16.845 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.845 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.845 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:16.845 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.961240+0000", 2024-09-21T15:01:16.845 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9147999999999993e-05 2024-09-21T15:01:16.845 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.845 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.845 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:16.845 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.961444+0000", 2024-09-21T15:01:16.845 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00020381899999999999 2024-09-21T15:01:16.845 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:16.845 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:16.845 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:16.846 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.846 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.846 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:134713 3.a 3:51eaadc0:::rbd_data.11365ed419e3.0000000000000007:67 [read 303104~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:16.846 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.939073+0000", 2024-09-21T15:01:16.846 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 22.832899429000001, 2024-09-21T15:01:16.846 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022687394999999999, 2024-09-21T15:01:16.846 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:16.846 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:16.846 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:16.846 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:16.846 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:16.846 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 134713 2024-09-21T15:01:16.846 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.846 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:16.846 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.847 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:16.847 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939073+0000", 2024-09-21T15:01:16.847 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:16.847 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.847 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.847 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:16.847 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939073+0000", 2024-09-21T15:01:16.847 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:16.847 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.847 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.847 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:16.847 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939073+0000", 2024-09-21T15:01:16.847 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:16.847 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.848 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.848 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:16.848 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939075+0000", 2024-09-21T15:01:16.848 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5969999999999999e-06 2024-09-21T15:01:16.848 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.848 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.848 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:16.848 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939081+0000", 2024-09-21T15:01:16.848 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.609e-06 2024-09-21T15:01:16.848 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.848 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.848 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:16.848 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939086+0000", 2024-09-21T15:01:16.848 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.916e-06 2024-09-21T15:01:16.849 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.849 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.849 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:16.849 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.961474+0000", 2024-09-21T15:01:16.849 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022388512999999999 2024-09-21T15:01:16.849 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.849 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.849 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:16.849 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.961567+0000", 2024-09-21T15:01:16.849 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2267000000000003e-05 2024-09-21T15:01:16.849 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.849 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.849 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:16.849 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.961760+0000", 2024-09-21T15:01:16.849 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00019375999999999999 2024-09-21T15:01:16.850 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:16.850 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:16.850 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:16.850 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.850 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.850 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:134714 3.a 3:51eaadc0:::rbd_data.11365ed419e3.0000000000000007:67 [read 319488~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:16.850 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.939090+0000", 2024-09-21T15:01:16.850 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 22.832882602000002, 2024-09-21T15:01:16.850 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023043403, 2024-09-21T15:01:16.850 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:16.850 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:16.850 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:16.850 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:16.850 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:16.850 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 134714 2024-09-21T15:01:16.850 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.851 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:16.851 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.851 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:16.851 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939090+0000", 2024-09-21T15:01:16.851 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:16.851 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.851 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.851 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:16.851 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939090+0000", 2024-09-21T15:01:16.851 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:16.851 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.851 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.851 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:16.851 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939090+0000", 2024-09-21T15:01:16.851 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:16.852 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.852 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.852 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:16.852 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939092+0000", 2024-09-21T15:01:16.852 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2790000000000001e-06 2024-09-21T15:01:16.852 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.852 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.852 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:16.852 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939097+0000", 2024-09-21T15:01:16.852 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1410000000000004e-06 2024-09-21T15:01:16.852 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.852 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.852 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:16.852 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939102+0000", 2024-09-21T15:01:16.852 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7369999999999997e-06 2024-09-21T15:01:16.852 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.853 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.853 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:16.853 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.961791+0000", 2024-09-21T15:01:16.853 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022688974000000001 2024-09-21T15:01:16.853 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.853 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.853 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:16.853 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.961865+0000", 2024-09-21T15:01:16.853 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4651000000000001e-05 2024-09-21T15:01:16.853 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.853 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.853 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:16.853 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.962133+0000", 2024-09-21T15:01:16.853 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00026793999999999999 2024-09-21T15:01:16.854 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:16.854 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:16.854 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:16.854 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.854 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.854 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:134715 3.a 3:51eaadc0:::rbd_data.11365ed419e3.0000000000000007:67 [read 335872~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:16.854 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.939106+0000", 2024-09-21T15:01:16.854 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 22.832865991999999, 2024-09-21T15:01:16.854 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023401142999999999, 2024-09-21T15:01:16.854 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:16.854 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:16.854 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:16.854 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:16.854 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:16.855 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 134715 2024-09-21T15:01:16.855 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.855 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:16.855 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.855 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:16.855 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939106+0000", 2024-09-21T15:01:16.855 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:16.855 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.855 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.855 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:16.855 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939106+0000", 2024-09-21T15:01:16.855 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:16.855 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.855 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.855 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:16.855 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939106+0000", 2024-09-21T15:01:16.856 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:16.856 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.856 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.856 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:16.856 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939114+0000", 2024-09-21T15:01:16.856 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1009999999999992e-06 2024-09-21T15:01:16.856 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.856 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.856 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:16.856 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939121+0000", 2024-09-21T15:01:16.856 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7710000000000001e-06 2024-09-21T15:01:16.856 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.856 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.856 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:16.856 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939126+0000", 2024-09-21T15:01:16.857 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3600000000000004e-06 2024-09-21T15:01:16.857 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.857 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.857 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:16.857 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.962162+0000", 2024-09-21T15:01:16.857 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023036148999999999 2024-09-21T15:01:16.857 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.857 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.857 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:16.857 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.962239+0000", 2024-09-21T15:01:16.857 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6874999999999998e-05 2024-09-21T15:01:16.857 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.857 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.857 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:16.857 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.962508+0000", 2024-09-21T15:01:16.857 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00026842199999999999 2024-09-21T15:01:16.858 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:16.858 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:16.858 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:16.858 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.858 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.858 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:134716 3.a 3:51eaadc0:::rbd_data.11365ed419e3.0000000000000007:67 [read 352256~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:16.858 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.939131+0000", 2024-09-21T15:01:16.858 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 22.832841857999998, 2024-09-21T15:01:16.858 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023742441, 2024-09-21T15:01:16.858 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:16.858 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:16.858 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:16.858 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:16.858 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:16.858 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 134716 2024-09-21T15:01:16.859 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.859 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:16.859 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.859 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:16.859 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939131+0000", 2024-09-21T15:01:16.859 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:16.859 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.859 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.859 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:16.859 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939131+0000", 2024-09-21T15:01:16.859 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:16.859 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.859 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.859 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:16.859 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939130+0000", 2024-09-21T15:01:16.859 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:16.860 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.860 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.860 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:16.860 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939133+0000", 2024-09-21T15:01:16.860 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5620000000000002e-06 2024-09-21T15:01:16.860 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.860 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.860 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:16.860 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939138+0000", 2024-09-21T15:01:16.860 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.322e-06 2024-09-21T15:01:16.860 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.860 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.860 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:16.860 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939143+0000", 2024-09-21T15:01:16.860 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5510000000000001e-06 2024-09-21T15:01:16.861 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.861 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.861 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:16.861 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.962537+0000", 2024-09-21T15:01:16.861 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023394808 2024-09-21T15:01:16.861 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.861 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.861 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:16.861 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.962608+0000", 2024-09-21T15:01:16.861 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0699999999999997e-05 2024-09-21T15:01:16.861 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.861 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.861 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:16.861 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.962873+0000", 2024-09-21T15:01:16.861 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00026487 2024-09-21T15:01:16.862 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:16.862 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:16.862 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:16.862 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.862 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.862 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:135024 3.a 3:53261383:::rbd_data.11365ed419e3.0000000000000037:snapdir [list-snaps] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:16.862 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.940162+0000", 2024-09-21T15:01:16.862 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 22.831810895, 2024-09-21T15:01:16.862 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022852172, 2024-09-21T15:01:16.862 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:16.862 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:16.862 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:16.862 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:16.862 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:16.862 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 135024 2024-09-21T15:01:16.862 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.863 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:16.863 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.863 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:16.863 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940162+0000", 2024-09-21T15:01:16.863 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:16.863 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.863 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.863 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:16.863 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940162+0000", 2024-09-21T15:01:16.863 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:16.863 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.863 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.863 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:16.863 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940161+0000", 2024-09-21T15:01:16.863 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:16.863 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.864 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.864 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:16.864 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940164+0000", 2024-09-21T15:01:16.864 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4099999999999998e-06 2024-09-21T15:01:16.864 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.864 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.864 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:16.864 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940169+0000", 2024-09-21T15:01:16.864 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3720000000000004e-06 2024-09-21T15:01:16.864 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.864 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.864 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:16.864 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940174+0000", 2024-09-21T15:01:16.864 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.724e-06 2024-09-21T15:01:16.865 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.865 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.865 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:16.865 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.962902+0000", 2024-09-21T15:01:16.865 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022728719000000001 2024-09-21T15:01:16.865 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.865 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.865 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:16.865 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.963014+0000", 2024-09-21T15:01:16.865 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000111245 2024-09-21T15:01:16.865 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:16.865 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:16.865 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:16.865 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.865 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.866 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:135044 3.a 3:53261383:::rbd_data.11365ed419e3.0000000000000037:snapdir [list-snaps] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:16.866 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.940484+0000", 2024-09-21T15:01:16.866 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 22.831488163, 2024-09-21T15:01:16.866 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022619169000000001, 2024-09-21T15:01:16.866 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:16.866 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:16.866 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:16.866 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:16.866 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:16.866 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 135044 2024-09-21T15:01:16.866 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.866 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:16.866 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.866 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:16.867 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940484+0000", 2024-09-21T15:01:16.867 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:16.867 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.867 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.867 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:16.867 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940484+0000", 2024-09-21T15:01:16.867 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:16.867 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.867 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.867 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:16.867 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940484+0000", 2024-09-21T15:01:16.867 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:16.867 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.867 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.867 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:16.867 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940487+0000", 2024-09-21T15:01:16.868 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8449999999999999e-06 2024-09-21T15:01:16.868 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.868 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.868 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:16.868 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940493+0000", 2024-09-21T15:01:16.868 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9599999999999997e-06 2024-09-21T15:01:16.868 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.868 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.868 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:16.868 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940497+0000", 2024-09-21T15:01:16.868 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9620000000000001e-06 2024-09-21T15:01:16.868 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.868 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.868 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:16.868 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.963034+0000", 2024-09-21T15:01:16.868 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022536878 2024-09-21T15:01:16.869 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:16.869 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:16.869 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:16.869 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.963103+0000", 2024-09-21T15:01:16.869 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9073999999999996e-05 2024-09-21T15:01:16.869 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:16.869 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:16.869 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:16.869 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:16.869 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:16.869 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:16.871 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T15:01:16.871 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:01:16.865+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:01:16.971 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:01:16.969+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:01:16.988 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:16.988 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:01:16.988 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T15:01:16.988 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:16.998 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T15:01:17.072 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:01:17.066+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:01:17.106 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:17.106 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:01:17.106 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T15:01:17.106 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T15:01:17.106 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:17.116 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T15:01:17.173 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:01:17.166+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:01:17.269 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:17.269 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T15:01:17.269 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T15:01:17.269 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T15:01:17.270 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.270 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:15130 3.2 3:47f1bbdb:::rbd_data.11361bed1f78.0000000000000067:head [sparse-read 0~147456] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-21T15:01:17.270 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:40.288284+0000", 2024-09-21T15:01:17.270 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 156.97944312499999, 2024-09-21T15:01:17.270 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1142935039999999, 2024-09-21T15:01:17.270 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:17.270 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:17.270 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:17.270 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:17.270 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:17.270 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 15130 2024-09-21T15:01:17.270 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.270 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:17.270 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.271 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:17.271 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288284+0000", 2024-09-21T15:01:17.271 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:17.271 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.271 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.271 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:17.271 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288284+0000", 2024-09-21T15:01:17.271 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:17.271 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.271 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.271 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:17.271 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288284+0000", 2024-09-21T15:01:17.271 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:17.272 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.272 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.272 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:17.272 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288288+0000", 2024-09-21T15:01:17.272 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5829999999999998e-06 2024-09-21T15:01:17.272 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.272 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.272 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:17.272 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288297+0000", 2024-09-21T15:01:17.272 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0669999999999998e-06 2024-09-21T15:01:17.272 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.272 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.272 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:17.272 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288305+0000", 2024-09-21T15:01:17.273 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8860000000000004e-06 2024-09-21T15:01:17.273 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.273 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.273 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:17.273 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288417+0000", 2024-09-21T15:01:17.273 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000112952 2024-09-21T15:01:17.273 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.273 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.273 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:17.273 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288428+0000", 2024-09-21T15:01:17.273 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0631e-05 2024-09-21T15:01:17.273 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.273 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.273 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:17.274 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.401876+0000", 2024-09-21T15:01:17.274 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1134479610000001 2024-09-21T15:01:17.274 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.274 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.274 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:17.274 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.402076+0000", 2024-09-21T15:01:17.274 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00019944200000000001 2024-09-21T15:01:17.274 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.274 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.274 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:17.274 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.402578+0000", 2024-09-21T15:01:17.274 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00050242899999999998 2024-09-21T15:01:17.274 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:17.274 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:17.275 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:17.275 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.275 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.275 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45708 3.2 3:415db91a:::rbd_data.1136ecbe5879.0000000000000000:28 [read 140288~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:17.275 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141801+0000", 2024-09-21T15:01:17.275 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 127.125926931, 2024-09-21T15:01:17.275 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034274200999999997, 2024-09-21T15:01:17.275 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:17.275 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:17.275 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:17.275 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:17.275 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:17.275 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45708 2024-09-21T15:01:17.275 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.276 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:17.276 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.276 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:17.276 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141801+0000", 2024-09-21T15:01:17.276 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:17.276 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.276 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.276 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:17.276 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141801+0000", 2024-09-21T15:01:17.276 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:17.276 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.276 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.276 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:17.276 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141800+0000", 2024-09-21T15:01:17.277 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:17.277 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.277 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.277 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:17.277 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141803+0000", 2024-09-21T15:01:17.277 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4210000000000002e-06 2024-09-21T15:01:17.277 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.277 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.277 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:17.277 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141812+0000", 2024-09-21T15:01:17.277 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.9800000000000004e-06 2024-09-21T15:01:17.277 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.277 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.277 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:17.277 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141816+0000", 2024-09-21T15:01:17.278 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4769999999999997e-06 2024-09-21T15:01:17.278 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.278 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.278 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:17.278 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.175865+0000", 2024-09-21T15:01:17.278 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034048875999999999 2024-09-21T15:01:17.278 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.278 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.278 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:17.278 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.175943+0000", 2024-09-21T15:01:17.278 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7464000000000002e-05 2024-09-21T15:01:17.278 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.278 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.279 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:17.279 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176075+0000", 2024-09-21T15:01:17.279 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013230800000000001 2024-09-21T15:01:17.279 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:17.279 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:17.279 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:17.279 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.279 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.279 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45709 3.2 3:415db91a:::rbd_data.1136ecbe5879.0000000000000000:28 [read 143360~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:17.279 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141820+0000", 2024-09-21T15:01:17.279 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 127.125907084, 2024-09-21T15:01:17.279 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034537536000000001, 2024-09-21T15:01:17.279 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:17.279 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:17.279 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:17.280 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:17.280 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:17.280 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45709 2024-09-21T15:01:17.280 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.280 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:17.280 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.280 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:17.280 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141820+0000", 2024-09-21T15:01:17.280 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:17.280 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.280 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.280 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:17.280 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141820+0000", 2024-09-21T15:01:17.280 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:17.280 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.281 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.281 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:17.281 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141820+0000", 2024-09-21T15:01:17.281 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:17.281 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.281 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.281 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:17.281 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141823+0000", 2024-09-21T15:01:17.281 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5110000000000002e-06 2024-09-21T15:01:17.281 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.281 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.281 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:17.281 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141829+0000", 2024-09-21T15:01:17.281 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0850000000000002e-06 2024-09-21T15:01:17.281 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.282 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.282 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:17.282 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141833+0000", 2024-09-21T15:01:17.282 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7620000000000003e-06 2024-09-21T15:01:17.282 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.282 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.282 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:17.282 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176103+0000", 2024-09-21T15:01:17.282 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034269933000000002 2024-09-21T15:01:17.282 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.282 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.282 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:17.282 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176161+0000", 2024-09-21T15:01:17.282 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7658999999999998e-05 2024-09-21T15:01:17.282 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.283 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.283 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:17.283 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176358+0000", 2024-09-21T15:01:17.283 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000196937 2024-09-21T15:01:17.283 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:17.283 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:17.283 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:17.283 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.283 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.283 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45710 3.2 3:415db91a:::rbd_data.1136ecbe5879.0000000000000000:28 [read 146432~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:17.283 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141838+0000", 2024-09-21T15:01:17.283 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 127.125889921, 2024-09-21T15:01:17.283 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034953674999999997, 2024-09-21T15:01:17.284 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:17.284 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:17.284 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:17.284 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:17.284 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:17.284 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45710 2024-09-21T15:01:17.284 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.284 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:17.284 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.284 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:17.284 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141838+0000", 2024-09-21T15:01:17.284 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:17.284 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.284 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.285 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:17.285 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141838+0000", 2024-09-21T15:01:17.285 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:17.285 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.285 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.285 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:17.285 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141837+0000", 2024-09-21T15:01:17.285 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:17.285 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.285 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.285 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:17.285 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141843+0000", 2024-09-21T15:01:17.285 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1619999999999996e-06 2024-09-21T15:01:17.285 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.286 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.286 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:17.286 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141850+0000", 2024-09-21T15:01:17.286 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4799999999999998e-06 2024-09-21T15:01:17.286 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.286 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.286 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:17.286 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141853+0000", 2024-09-21T15:01:17.286 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3050000000000001e-06 2024-09-21T15:01:17.286 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.286 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.286 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:17.286 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176490+0000", 2024-09-21T15:01:17.286 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034636413999999997 2024-09-21T15:01:17.287 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.287 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.287 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:17.287 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176606+0000", 2024-09-21T15:01:17.287 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000115841 2024-09-21T15:01:17.287 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.287 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.287 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:17.287 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176791+0000", 2024-09-21T15:01:17.287 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00018581500000000001 2024-09-21T15:01:17.287 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:17.287 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:17.287 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:17.287 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.288 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.288 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45711 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:17.288 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141856+0000", 2024-09-21T15:01:17.288 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 127.125871584, 2024-09-21T15:01:17.288 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034986308000000001, 2024-09-21T15:01:17.288 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:17.288 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:17.288 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:17.288 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:17.288 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:17.288 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45711 2024-09-21T15:01:17.288 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.288 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:17.288 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.289 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:17.289 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141856+0000", 2024-09-21T15:01:17.289 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:17.289 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.289 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.289 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:17.289 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141856+0000", 2024-09-21T15:01:17.289 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:17.289 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.289 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.289 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:17.289 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141856+0000", 2024-09-21T15:01:17.289 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:17.289 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.290 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.290 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:17.290 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141862+0000", 2024-09-21T15:01:17.290 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.567e-06 2024-09-21T15:01:17.290 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.290 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.290 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:17.290 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141868+0000", 2024-09-21T15:01:17.290 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9569999999999999e-06 2024-09-21T15:01:17.290 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.290 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.290 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:17.290 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141871+0000", 2024-09-21T15:01:17.290 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.146e-06 2024-09-21T15:01:17.291 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.291 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.291 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:17.291 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176831+0000", 2024-09-21T15:01:17.291 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034959935999999997 2024-09-21T15:01:17.291 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.291 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.291 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:17.291 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176842+0000", 2024-09-21T15:01:17.291 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1008e-05 2024-09-21T15:01:17.291 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:17.291 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:17.291 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:17.291 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.291 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.292 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45712 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:17.292 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141874+0000", 2024-09-21T15:01:17.292 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 127.125853689, 2024-09-21T15:01:17.292 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034993511999999997, 2024-09-21T15:01:17.292 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:17.292 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:17.292 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:17.292 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:17.292 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:17.292 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45712 2024-09-21T15:01:17.292 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.292 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:17.292 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.292 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:17.293 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141874+0000", 2024-09-21T15:01:17.293 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:17.293 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.293 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.293 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:17.293 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141874+0000", 2024-09-21T15:01:17.293 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:17.293 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.293 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.293 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:17.293 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141874+0000", 2024-09-21T15:01:17.293 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:17.293 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.293 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.293 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:17.294 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141875+0000", 2024-09-21T15:01:17.294 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5430000000000001e-06 2024-09-21T15:01:17.294 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.294 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.294 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:17.294 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141878+0000", 2024-09-21T15:01:17.294 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1420000000000002e-06 2024-09-21T15:01:17.294 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.294 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.294 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:17.294 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141881+0000", 2024-09-21T15:01:17.294 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.807e-06 2024-09-21T15:01:17.294 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.294 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.295 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:17.295 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176860+0000", 2024-09-21T15:01:17.295 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034979243 2024-09-21T15:01:17.295 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.295 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.295 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:17.295 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176867+0000", 2024-09-21T15:01:17.295 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0169999999999999e-06 2024-09-21T15:01:17.295 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:17.295 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:17.295 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:17.295 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.295 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.295 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45713 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:17.295 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141883+0000", 2024-09-21T15:01:17.296 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 127.125844114, 2024-09-21T15:01:17.296 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035010516999999998, 2024-09-21T15:01:17.296 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:17.296 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:17.296 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:17.296 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:17.296 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:17.296 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45713 2024-09-21T15:01:17.296 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.296 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:17.296 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.296 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:17.296 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141883+0000", 2024-09-21T15:01:17.296 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:17.296 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.297 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.297 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:17.297 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141883+0000", 2024-09-21T15:01:17.297 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:17.297 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.297 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.297 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:17.297 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141883+0000", 2024-09-21T15:01:17.297 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:17.297 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.297 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.297 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:17.297 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141885+0000", 2024-09-21T15:01:17.297 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4649999999999999e-06 2024-09-21T15:01:17.298 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.298 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.298 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:17.298 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141888+0000", 2024-09-21T15:01:17.298 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3890000000000001e-06 2024-09-21T15:01:17.298 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.298 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.298 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:17.298 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141891+0000", 2024-09-21T15:01:17.298 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9239999999999999e-06 2024-09-21T15:01:17.298 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.298 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.298 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:17.298 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176887+0000", 2024-09-21T15:01:17.299 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034996370999999998 2024-09-21T15:01:17.299 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.299 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.299 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:17.299 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176894+0000", 2024-09-21T15:01:17.299 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6069999999999996e-06 2024-09-21T15:01:17.299 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:17.299 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:17.299 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:17.299 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.299 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.299 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45714 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:17.299 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141893+0000", 2024-09-21T15:01:17.299 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 127.125834199, 2024-09-21T15:01:17.299 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035048215000000001, 2024-09-21T15:01:17.300 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:17.300 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:17.300 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:17.300 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:17.300 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:17.300 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45714 2024-09-21T15:01:17.300 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.300 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:17.300 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.300 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:17.300 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141893+0000", 2024-09-21T15:01:17.300 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:17.300 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.300 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.300 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:17.301 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141893+0000", 2024-09-21T15:01:17.301 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:17.301 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.301 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.301 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:17.301 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141893+0000", 2024-09-21T15:01:17.301 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:17.301 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.301 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.301 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:17.301 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141895+0000", 2024-09-21T15:01:17.301 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.703e-06 2024-09-21T15:01:17.301 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.301 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.301 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:17.302 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141900+0000", 2024-09-21T15:01:17.302 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.242e-06 2024-09-21T15:01:17.302 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.302 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.302 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:17.302 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141903+0000", 2024-09-21T15:01:17.302 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.117e-06 2024-09-21T15:01:17.302 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.302 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.302 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:17.302 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176930+0000", 2024-09-21T15:01:17.302 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035026750000000002 2024-09-21T15:01:17.302 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.302 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.302 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:17.303 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176942+0000", 2024-09-21T15:01:17.303 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1668000000000001e-05 2024-09-21T15:01:17.303 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:17.303 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:17.303 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:17.303 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.303 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.303 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45715 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:17.303 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141906+0000", 2024-09-21T15:01:17.303 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 127.12582204100001, 2024-09-21T15:01:17.303 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035066057999999997, 2024-09-21T15:01:17.303 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:17.303 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:17.303 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:17.303 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:17.304 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:17.304 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45715 2024-09-21T15:01:17.304 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.304 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:17.304 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.304 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:17.304 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141906+0000", 2024-09-21T15:01:17.304 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:17.304 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.304 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.304 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:17.304 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141906+0000", 2024-09-21T15:01:17.304 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:17.304 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.304 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.305 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:17.305 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141905+0000", 2024-09-21T15:01:17.305 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:17.305 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.305 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.305 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:17.305 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141907+0000", 2024-09-21T15:01:17.305 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5400000000000001e-06 2024-09-21T15:01:17.305 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.305 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.305 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:17.305 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141912+0000", 2024-09-21T15:01:17.305 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.502e-06 2024-09-21T15:01:17.305 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.306 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.306 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:17.306 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141915+0000", 2024-09-21T15:01:17.306 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0170000000000001e-06 2024-09-21T15:01:17.306 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.306 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.306 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:17.306 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176964+0000", 2024-09-21T15:01:17.306 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035048587999999999 2024-09-21T15:01:17.306 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.306 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.306 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:17.306 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176972+0000", 2024-09-21T15:01:17.306 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6580000000000002e-06 2024-09-21T15:01:17.307 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:17.307 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:17.307 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:17.307 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.307 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.307 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45716 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:17.307 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141918+0000", 2024-09-21T15:01:17.307 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 127.12580986499999, 2024-09-21T15:01:17.307 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035086223, 2024-09-21T15:01:17.307 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:17.307 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:17.307 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:17.307 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:17.307 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:17.308 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45716 2024-09-21T15:01:17.308 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.308 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:17.308 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.308 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:17.308 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141918+0000", 2024-09-21T15:01:17.308 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:17.308 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.308 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.308 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:17.308 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141918+0000", 2024-09-21T15:01:17.308 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:17.308 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.308 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.308 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:17.309 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141917+0000", 2024-09-21T15:01:17.309 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:17.309 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.309 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.309 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:17.309 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141919+0000", 2024-09-21T15:01:17.309 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5680000000000001e-06 2024-09-21T15:01:17.309 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.309 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.309 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:17.309 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141922+0000", 2024-09-21T15:01:17.309 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2789999999999999e-06 2024-09-21T15:01:17.309 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.309 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.309 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:17.310 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141925+0000", 2024-09-21T15:01:17.310 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9469999999999999e-06 2024-09-21T15:01:17.310 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.310 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.310 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:17.310 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176997+0000", 2024-09-21T15:01:17.310 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035071366999999999 2024-09-21T15:01:17.310 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.310 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.310 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:17.310 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177004+0000", 2024-09-21T15:01:17.310 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3309999999999998e-06 2024-09-21T15:01:17.310 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:17.310 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:17.310 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:17.311 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.311 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.311 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45717 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:17.311 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141928+0000", 2024-09-21T15:01:17.311 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 127.125799919, 2024-09-21T15:01:17.311 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035102573999999998, 2024-09-21T15:01:17.311 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:17.311 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:17.311 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:17.311 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:17.311 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:17.311 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45717 2024-09-21T15:01:17.311 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.311 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:17.312 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.312 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:17.312 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141928+0000", 2024-09-21T15:01:17.312 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:17.312 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.312 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.312 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:17.312 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141928+0000", 2024-09-21T15:01:17.312 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:17.312 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.312 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.312 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:17.312 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141927+0000", 2024-09-21T15:01:17.312 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:01:17.312 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.313 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.313 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:17.313 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141929+0000", 2024-09-21T15:01:17.313 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.533e-06 2024-09-21T15:01:17.313 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.313 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.313 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:17.313 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141934+0000", 2024-09-21T15:01:17.313 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1660000000000002e-06 2024-09-21T15:01:17.313 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.313 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.313 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:17.313 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141937+0000", 2024-09-21T15:01:17.313 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0309999999999999e-06 2024-09-21T15:01:17.314 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.314 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.314 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:17.314 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177023+0000", 2024-09-21T15:01:17.314 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035085595999999997 2024-09-21T15:01:17.314 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.314 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.314 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:17.314 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177030+0000", 2024-09-21T15:01:17.314 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.486e-06 2024-09-21T15:01:17.314 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:17.314 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:17.314 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:17.314 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.314 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.315 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45718 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:17.315 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141940+0000", 2024-09-21T15:01:17.315 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 127.12578806099999, 2024-09-21T15:01:17.315 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035124956999999998, 2024-09-21T15:01:17.315 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:17.315 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:17.315 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:17.315 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:17.315 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:17.315 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45718 2024-09-21T15:01:17.315 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.315 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:17.315 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.315 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:17.315 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141940+0000", 2024-09-21T15:01:17.315 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:17.316 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.316 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.316 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:17.316 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141940+0000", 2024-09-21T15:01:17.316 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:17.316 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.316 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.316 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:17.316 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141939+0000", 2024-09-21T15:01:17.316 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:17.316 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.316 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.316 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:17.316 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141941+0000", 2024-09-21T15:01:17.316 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.502e-06 2024-09-21T15:01:17.316 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.317 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.317 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:17.317 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141946+0000", 2024-09-21T15:01:17.317 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3449999999999996e-06 2024-09-21T15:01:17.317 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.317 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.317 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:17.317 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141949+0000", 2024-09-21T15:01:17.317 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9749999999999999e-06 2024-09-21T15:01:17.317 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.317 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.317 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:17.317 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177057+0000", 2024-09-21T15:01:17.317 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035108385999999998 2024-09-21T15:01:17.317 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.317 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.318 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:17.318 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177064+0000", 2024-09-21T15:01:17.318 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0099999999999998e-06 2024-09-21T15:01:17.318 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:17.318 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:17.318 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:17.318 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.318 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.318 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45719 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:17.318 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141952+0000", 2024-09-21T15:01:17.318 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 127.125775788, 2024-09-21T15:01:17.318 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035139108000000002, 2024-09-21T15:01:17.318 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:17.318 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:17.318 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:17.318 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:17.319 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:17.319 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45719 2024-09-21T15:01:17.319 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.319 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:17.319 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.319 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:17.319 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141952+0000", 2024-09-21T15:01:17.319 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:17.319 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.319 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.319 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:17.319 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141952+0000", 2024-09-21T15:01:17.319 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:17.319 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.319 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.320 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:17.320 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141952+0000", 2024-09-21T15:01:17.320 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:17.320 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.320 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.320 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:17.320 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141953+0000", 2024-09-21T15:01:17.320 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.486e-06 2024-09-21T15:01:17.320 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.320 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.320 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:17.320 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141959+0000", 2024-09-21T15:01:17.320 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4940000000000003e-06 2024-09-21T15:01:17.320 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.320 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.321 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:17.321 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141963+0000", 2024-09-21T15:01:17.321 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3540000000000002e-06 2024-09-21T15:01:17.321 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.321 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.321 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:17.321 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177084+0000", 2024-09-21T15:01:17.321 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035120983000000001 2024-09-21T15:01:17.321 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.321 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.321 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:17.321 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177091+0000", 2024-09-21T15:01:17.321 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0350000000000004e-06 2024-09-21T15:01:17.321 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:17.321 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:17.321 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:17.322 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.322 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.322 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45720 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:17.322 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141965+0000", 2024-09-21T15:01:17.322 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 127.125762321, 2024-09-21T15:01:17.322 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035155268000000003, 2024-09-21T15:01:17.322 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:17.322 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:17.322 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:17.322 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:17.322 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:17.322 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45720 2024-09-21T15:01:17.322 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.322 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:17.322 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.322 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:17.323 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141965+0000", 2024-09-21T15:01:17.323 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:17.323 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.323 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.323 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:17.323 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141965+0000", 2024-09-21T15:01:17.323 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:17.323 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.323 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.323 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:17.323 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141965+0000", 2024-09-21T15:01:17.323 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:17.323 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.323 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.323 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:17.324 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141966+0000", 2024-09-21T15:01:17.324 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.46e-06 2024-09-21T15:01:17.324 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.324 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.324 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:17.324 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141972+0000", 2024-09-21T15:01:17.324 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4210000000000003e-06 2024-09-21T15:01:17.324 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.324 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.324 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:17.324 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141975+0000", 2024-09-21T15:01:17.324 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.948e-06 2024-09-21T15:01:17.324 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.324 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.324 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:17.325 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177113+0000", 2024-09-21T15:01:17.325 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035137855000000003 2024-09-21T15:01:17.325 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.325 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.325 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:17.325 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177121+0000", 2024-09-21T15:01:17.325 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8269999999999998e-06 2024-09-21T15:01:17.325 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:17.325 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:17.325 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:17.325 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.325 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.325 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45721 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:17.325 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141977+0000", 2024-09-21T15:01:17.325 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 127.125750298, 2024-09-21T15:01:17.325 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035169984000000001, 2024-09-21T15:01:17.326 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:17.326 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:17.326 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:17.326 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:17.326 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:17.326 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45721 2024-09-21T15:01:17.326 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.326 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:17.326 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.326 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:17.326 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141977+0000", 2024-09-21T15:01:17.326 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:17.326 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.326 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.326 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:17.326 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141977+0000", 2024-09-21T15:01:17.327 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:17.327 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.327 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.327 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:17.327 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141977+0000", 2024-09-21T15:01:17.327 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:17.327 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.327 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.327 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:17.327 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141979+0000", 2024-09-21T15:01:17.327 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.511e-06 2024-09-21T15:01:17.327 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.327 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.327 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:17.328 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141982+0000", 2024-09-21T15:01:17.328 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4180000000000001e-06 2024-09-21T15:01:17.328 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.328 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.328 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:17.328 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141985+0000", 2024-09-21T15:01:17.328 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9450000000000002e-06 2024-09-21T15:01:17.328 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.328 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.328 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:17.328 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177139+0000", 2024-09-21T15:01:17.328 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035154409999999997 2024-09-21T15:01:17.328 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.328 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.329 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:17.329 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177147+0000", 2024-09-21T15:01:17.329 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9570000000000002e-06 2024-09-21T15:01:17.329 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:17.329 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:17.329 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:17.329 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.329 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.329 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45722 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:17.329 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141987+0000", 2024-09-21T15:01:17.329 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 127.125740327, 2024-09-21T15:01:17.329 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035188492000000002, 2024-09-21T15:01:17.329 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:17.329 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:17.330 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:17.330 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:17.330 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:17.330 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45722 2024-09-21T15:01:17.330 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.330 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:17.330 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.330 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:17.330 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141987+0000", 2024-09-21T15:01:17.330 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:17.330 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.330 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.330 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:17.330 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141987+0000", 2024-09-21T15:01:17.331 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:17.331 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.331 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.331 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:17.331 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141987+0000", 2024-09-21T15:01:17.331 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:17.331 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.331 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.331 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:17.331 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141988+0000", 2024-09-21T15:01:17.331 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.463e-06 2024-09-21T15:01:17.331 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.331 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.331 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:17.331 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141994+0000", 2024-09-21T15:01:17.332 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.361e-06 2024-09-21T15:01:17.332 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.332 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.332 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:17.332 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141997+0000", 2024-09-21T15:01:17.332 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0359999999999998e-06 2024-09-21T15:01:17.332 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.332 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.332 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:17.332 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177169+0000", 2024-09-21T15:01:17.332 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035171651999999998 2024-09-21T15:01:17.332 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.332 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.332 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:17.332 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177176+0000", 2024-09-21T15:01:17.332 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2189999999999998e-06 2024-09-21T15:01:17.333 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:17.333 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:17.333 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:17.333 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.333 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.333 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45723 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:17.333 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141999+0000", 2024-09-21T15:01:17.333 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 127.12572815999999, 2024-09-21T15:01:17.333 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035197701999999997, 2024-09-21T15:01:17.333 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:17.333 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:17.333 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:17.333 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:17.333 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:17.333 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45723 2024-09-21T15:01:17.334 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.334 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:17.334 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.334 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:17.334 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141999+0000", 2024-09-21T15:01:17.334 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:17.334 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.334 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.334 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:17.334 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141999+0000", 2024-09-21T15:01:17.334 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:17.334 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.334 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.334 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:17.335 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141999+0000", 2024-09-21T15:01:17.335 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:17.335 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.335 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.335 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:17.335 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142003+0000", 2024-09-21T15:01:17.335 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5049999999999999e-06 2024-09-21T15:01:17.335 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.335 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.335 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:17.335 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142008+0000", 2024-09-21T15:01:17.335 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6579999999999999e-06 2024-09-21T15:01:17.335 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.335 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.335 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:17.335 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142011+0000", 2024-09-21T15:01:17.336 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0529999999999998e-06 2024-09-21T15:01:17.336 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.336 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.336 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:17.336 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177194+0000", 2024-09-21T15:01:17.336 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035182221999999999 2024-09-21T15:01:17.336 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.336 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.336 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:17.336 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177197+0000", 2024-09-21T15:01:17.336 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5030000000000002e-06 2024-09-21T15:01:17.336 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:17.336 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:17.336 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:17.336 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.336 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.337 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45724 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:17.337 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.142015+0000", 2024-09-21T15:01:17.337 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 127.125712193, 2024-09-21T15:01:17.337 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035195345000000003, 2024-09-21T15:01:17.337 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:17.337 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:17.337 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:17.337 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:17.337 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:17.337 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45724 2024-09-21T15:01:17.337 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.337 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:17.337 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.337 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:17.337 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142015+0000", 2024-09-21T15:01:17.337 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:17.338 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.338 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.338 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:17.338 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142015+0000", 2024-09-21T15:01:17.338 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:17.338 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.338 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.338 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:17.338 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142015+0000", 2024-09-21T15:01:17.338 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:17.338 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.338 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.338 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:17.339 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142017+0000", 2024-09-21T15:01:17.339 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5540000000000001e-06 2024-09-21T15:01:17.339 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.339 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.339 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:17.339 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142022+0000", 2024-09-21T15:01:17.339 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4229999999999996e-06 2024-09-21T15:01:17.339 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.339 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.339 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:17.339 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142025+0000", 2024-09-21T15:01:17.339 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.056e-06 2024-09-21T15:01:17.339 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.339 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.340 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:17.340 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177207+0000", 2024-09-21T15:01:17.340 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035182089 2024-09-21T15:01:17.340 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.340 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.340 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:17.340 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177211+0000", 2024-09-21T15:01:17.340 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4649999999999999e-06 2024-09-21T15:01:17.340 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:17.340 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:17.340 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:17.340 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.340 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.340 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45725 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:17.340 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.142028+0000", 2024-09-21T15:01:17.341 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 127.12569991399999, 2024-09-21T15:01:17.341 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035195731000000001, 2024-09-21T15:01:17.341 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:17.341 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:17.341 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:17.341 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:17.341 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:17.341 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45725 2024-09-21T15:01:17.341 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.341 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:17.341 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.341 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:17.341 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142028+0000", 2024-09-21T15:01:17.341 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:17.342 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.342 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.342 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:17.342 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142028+0000", 2024-09-21T15:01:17.342 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:17.342 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.342 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.342 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:17.342 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142027+0000", 2024-09-21T15:01:17.342 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:17.342 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.342 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.342 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:17.342 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142029+0000", 2024-09-21T15:01:17.343 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5269999999999999e-06 2024-09-21T15:01:17.343 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.343 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.343 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:17.343 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142032+0000", 2024-09-21T15:01:17.343 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3249999999999999e-06 2024-09-21T15:01:17.343 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.343 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.343 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:17.343 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142035+0000", 2024-09-21T15:01:17.343 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.897e-06 2024-09-21T15:01:17.343 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.343 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.343 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:17.343 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177220+0000", 2024-09-21T15:01:17.343 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035184871 2024-09-21T15:01:17.344 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.344 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.344 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:17.344 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177223+0000", 2024-09-21T15:01:17.344 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3680000000000002e-06 2024-09-21T15:01:17.344 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:17.344 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:17.344 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:17.344 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.344 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.344 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45726 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:17.344 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.142037+0000", 2024-09-21T15:01:17.344 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 127.125690199, 2024-09-21T15:01:17.344 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035199273000000003, 2024-09-21T15:01:17.345 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:17.345 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:17.345 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:17.345 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:17.345 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:17.345 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45726 2024-09-21T15:01:17.345 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.345 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:17.345 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.345 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:17.345 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142037+0000", 2024-09-21T15:01:17.345 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:17.345 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.345 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.345 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:17.345 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142037+0000", 2024-09-21T15:01:17.346 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:17.346 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.346 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.346 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:17.346 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142037+0000", 2024-09-21T15:01:17.346 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:17.346 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.346 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.346 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:17.346 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142038+0000", 2024-09-21T15:01:17.346 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.344e-06 2024-09-21T15:01:17.346 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.346 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.346 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:17.346 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142044+0000", 2024-09-21T15:01:17.346 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1930000000000001e-06 2024-09-21T15:01:17.347 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.347 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.347 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:17.347 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142047+0000", 2024-09-21T15:01:17.347 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0280000000000001e-06 2024-09-21T15:01:17.347 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.347 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.347 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:17.347 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177233+0000", 2024-09-21T15:01:17.347 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035186394000000003 2024-09-21T15:01:17.347 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.347 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.347 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:17.347 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177237+0000", 2024-09-21T15:01:17.347 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5530000000000001e-06 2024-09-21T15:01:17.347 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:17.348 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:17.348 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:17.348 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:17.348 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:17.348 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:17.349 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T15:01:17.350 INFO:teuthology.orchestra.run.smithi032.stderr:noscrub is set 2024-09-21T15:01:17.350 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:01:17.346+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:01:17.450 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:01:17.445+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:01:17.524 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:17.524 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:01:17.524 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T15:01:17.524 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:17.534 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T15:01:17.550 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:01:17.546+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:01:17.640 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:17.640 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:01:17.640 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T15:01:17.640 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T15:01:17.640 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:17.649 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T15:01:17.652 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:01:17.646+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:01:17.729 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T15:01:17.772 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:17.772 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T15:01:17.772 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T15:01:17.772 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T15:01:17.772 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.772 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139776 3.16 3:6c38d0e2:::rbd_data.113664780c85.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:17.772 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.223442+0000", 2024-09-21T15:01:17.772 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 15.546932174, 2024-09-21T15:01:17.772 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0315488150000001, 2024-09-21T15:01:17.772 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:17.773 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:17.773 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:17.773 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:17.773 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:17.773 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139776 2024-09-21T15:01:17.773 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.773 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:17.773 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.773 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:17.773 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223442+0000", 2024-09-21T15:01:17.773 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:17.773 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.773 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.774 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:17.774 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223442+0000", 2024-09-21T15:01:17.774 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:17.774 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.774 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.774 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:17.774 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223442+0000", 2024-09-21T15:01:17.774 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:17.774 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.774 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.774 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:17.774 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223446+0000", 2024-09-21T15:01:17.775 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2549999999999999e-06 2024-09-21T15:01:17.775 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.775 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.775 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:17.775 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223454+0000", 2024-09-21T15:01:17.775 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8199999999999997e-06 2024-09-21T15:01:17.775 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.775 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.775 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:17.775 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223460+0000", 2024-09-21T15:01:17.775 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6330000000000001e-06 2024-09-21T15:01:17.775 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.775 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.775 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:17.776 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223542+0000", 2024-09-21T15:01:17.776 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1929999999999997e-05 2024-09-21T15:01:17.776 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.776 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.776 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:17.776 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223552+0000", 2024-09-21T15:01:17.776 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0627e-05 2024-09-21T15:01:17.776 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.776 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.776 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:17.776 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.254634+0000", 2024-09-21T15:01:17.776 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.031081972 2024-09-21T15:01:17.776 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.776 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.776 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:17.777 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.254796+0000", 2024-09-21T15:01:17.777 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000161879 2024-09-21T15:01:17.777 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.777 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.777 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:17.777 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.254991+0000", 2024-09-21T15:01:17.777 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00019526499999999999 2024-09-21T15:01:17.777 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:17.777 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:17.777 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:17.777 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.777 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.777 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139785 3.11 3:8abbb109:::rbd_data.113664780c85.0000000000000070:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:17.777 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.223531+0000", 2024-09-21T15:01:17.777 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 15.546843432999999, 2024-09-21T15:01:17.778 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0187143510000001, 2024-09-21T15:01:17.778 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:17.778 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:17.778 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:17.778 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:17.778 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:17.778 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139785 2024-09-21T15:01:17.778 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.778 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:17.778 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.778 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:17.778 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223531+0000", 2024-09-21T15:01:17.778 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:17.779 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.779 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.779 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:17.779 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223531+0000", 2024-09-21T15:01:17.779 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:17.779 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.779 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.779 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:17.779 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223531+0000", 2024-09-21T15:01:17.779 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:17.779 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.779 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.779 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:17.779 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223534+0000", 2024-09-21T15:01:17.780 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2650000000000001e-06 2024-09-21T15:01:17.780 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.780 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.780 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:17.780 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223541+0000", 2024-09-21T15:01:17.780 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8990000000000004e-06 2024-09-21T15:01:17.780 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.780 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.780 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:17.780 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223546+0000", 2024-09-21T15:01:17.780 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3399999999999997e-06 2024-09-21T15:01:17.780 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.780 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.780 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:17.780 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223724+0000", 2024-09-21T15:01:17.781 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017745799999999999 2024-09-21T15:01:17.781 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.781 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.781 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:17.781 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223735+0000", 2024-09-21T15:01:17.781 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1610999999999999e-05 2024-09-21T15:01:17.781 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.781 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.781 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:17.781 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242113+0000", 2024-09-21T15:01:17.781 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.018377879 2024-09-21T15:01:17.781 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.781 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.781 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:17.782 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242246+0000", 2024-09-21T15:01:17.782 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013231500000000001 2024-09-21T15:01:17.782 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:17.782 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:17.782 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:17.782 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.782 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.782 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139790 3.11 3:8a18c7d8:::rbd_data.113664780c85.000000000000008d:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:17.782 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.223578+0000", 2024-09-21T15:01:17.782 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 15.546796437999999, 2024-09-21T15:01:17.782 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0187879719999999, 2024-09-21T15:01:17.782 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:17.782 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:17.782 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:17.782 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:17.783 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:17.783 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139790 2024-09-21T15:01:17.783 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.783 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:17.783 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.783 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:17.783 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223578+0000", 2024-09-21T15:01:17.783 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:17.783 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.783 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.783 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:17.783 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223578+0000", 2024-09-21T15:01:17.783 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:17.783 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.784 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.784 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:17.784 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223578+0000", 2024-09-21T15:01:17.784 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:17.784 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.784 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.784 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:17.784 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223581+0000", 2024-09-21T15:01:17.784 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6879999999999999e-06 2024-09-21T15:01:17.784 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.784 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.784 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:17.784 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223586+0000", 2024-09-21T15:01:17.784 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7080000000000002e-06 2024-09-21T15:01:17.785 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.785 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.785 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:17.785 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223591+0000", 2024-09-21T15:01:17.785 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5639999999999998e-06 2024-09-21T15:01:17.785 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.785 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.785 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:17.785 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223761+0000", 2024-09-21T15:01:17.785 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000170213 2024-09-21T15:01:17.785 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.785 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.785 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:17.785 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223769+0000", 2024-09-21T15:01:17.786 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7049999999999999e-06 2024-09-21T15:01:17.786 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.786 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.786 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:17.786 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242269+0000", 2024-09-21T15:01:17.786 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.018500288 2024-09-21T15:01:17.786 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.786 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.786 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:17.786 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242366+0000", 2024-09-21T15:01:17.786 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.7126000000000005e-05 2024-09-21T15:01:17.786 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:17.786 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:17.787 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:17.787 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.787 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.787 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139792 3.16 3:6ce5caf3:::rbd_data.113664780c85.000000000000009b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:17.787 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.223595+0000", 2024-09-21T15:01:17.787 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 15.546779485, 2024-09-21T15:01:17.787 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0317517899999999, 2024-09-21T15:01:17.787 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:17.787 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:17.787 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:17.787 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:17.787 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:17.787 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139792 2024-09-21T15:01:17.787 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.788 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:17.788 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.788 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:17.788 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223595+0000", 2024-09-21T15:01:17.788 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:17.788 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.788 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.788 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:17.788 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223595+0000", 2024-09-21T15:01:17.788 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:17.788 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.788 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.788 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:17.788 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223595+0000", 2024-09-21T15:01:17.789 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:17.789 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.789 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.789 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:17.789 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223598+0000", 2024-09-21T15:01:17.789 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4620000000000001e-06 2024-09-21T15:01:17.789 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.789 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.789 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:17.789 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223604+0000", 2024-09-21T15:01:17.789 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6879999999999996e-06 2024-09-21T15:01:17.789 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.789 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.789 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:17.790 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223609+0000", 2024-09-21T15:01:17.790 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9490000000000004e-06 2024-09-21T15:01:17.790 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.790 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.790 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:17.790 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224067+0000", 2024-09-21T15:01:17.790 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00045780700000000002 2024-09-21T15:01:17.790 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.790 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.790 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:17.790 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224075+0000", 2024-09-21T15:01:17.790 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6610000000000006e-06 2024-09-21T15:01:17.790 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.790 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.791 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:17.791 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255141+0000", 2024-09-21T15:01:17.791 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0310657480000001 2024-09-21T15:01:17.791 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.791 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.791 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:17.791 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255347+0000", 2024-09-21T15:01:17.791 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000205837 2024-09-21T15:01:17.791 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:17.791 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:17.791 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:17.791 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.791 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.792 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139794 3.11 3:8cf3a2f4:::rbd_data.113664780c85.00000000000000a9:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:17.792 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.224011+0000", 2024-09-21T15:01:17.792 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 15.546363457, 2024-09-21T15:01:17.792 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.018507791, 2024-09-21T15:01:17.792 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:17.792 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:17.792 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:17.792 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:17.792 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:17.792 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139794 2024-09-21T15:01:17.792 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.792 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:17.792 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.792 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:17.792 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224011+0000", 2024-09-21T15:01:17.793 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:17.793 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.793 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.793 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:17.793 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224011+0000", 2024-09-21T15:01:17.793 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:17.793 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.793 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.793 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:17.793 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224010+0000", 2024-09-21T15:01:17.793 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:17.793 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.793 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.793 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:17.794 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224016+0000", 2024-09-21T15:01:17.794 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6550000000000001e-06 2024-09-21T15:01:17.794 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.794 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.794 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:17.794 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224028+0000", 2024-09-21T15:01:17.794 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2422999999999999e-05 2024-09-21T15:01:17.794 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.794 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.794 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:17.794 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224036+0000", 2024-09-21T15:01:17.794 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7749999999999993e-06 2024-09-21T15:01:17.794 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.794 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.794 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:17.795 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225176+0000", 2024-09-21T15:01:17.795 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0011401740000000001 2024-09-21T15:01:17.795 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.795 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.795 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:17.795 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225187+0000", 2024-09-21T15:01:17.795 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0951999999999999e-05 2024-09-21T15:01:17.795 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.795 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.795 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:17.795 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242392+0000", 2024-09-21T15:01:17.795 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.017204905 2024-09-21T15:01:17.795 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.795 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.796 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:17.796 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242519+0000", 2024-09-21T15:01:17.796 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012680499999999999 2024-09-21T15:01:17.796 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:17.796 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:17.796 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:17.796 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.796 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.796 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139800 3.16 3:68b050ed:::rbd_data.113664780c85.00000000000000d3:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:17.796 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.224579+0000", 2024-09-21T15:01:17.796 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 15.545795697999999, 2024-09-21T15:01:17.796 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.030932637, 2024-09-21T15:01:17.796 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:17.796 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:17.797 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:17.797 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:17.797 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:17.797 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139800 2024-09-21T15:01:17.797 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.797 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:17.797 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.797 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:17.797 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224579+0000", 2024-09-21T15:01:17.797 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:17.797 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.797 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.797 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:17.797 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224579+0000", 2024-09-21T15:01:17.797 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:17.798 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.798 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.798 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:17.798 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224578+0000", 2024-09-21T15:01:17.798 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:17.798 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.798 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.798 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:17.798 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224582+0000", 2024-09-21T15:01:17.798 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0029999999999999e-06 2024-09-21T15:01:17.798 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.798 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.798 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:17.798 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224588+0000", 2024-09-21T15:01:17.799 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8120000000000002e-06 2024-09-21T15:01:17.799 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.799 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.799 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:17.799 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224593+0000", 2024-09-21T15:01:17.799 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7469999999999996e-06 2024-09-21T15:01:17.799 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.799 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.799 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:17.799 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224648+0000", 2024-09-21T15:01:17.799 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4693e-05 2024-09-21T15:01:17.799 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.799 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.799 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:17.800 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224656+0000", 2024-09-21T15:01:17.800 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8429999999999993e-06 2024-09-21T15:01:17.800 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.800 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.800 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:17.800 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255382+0000", 2024-09-21T15:01:17.800 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0307266070000001 2024-09-21T15:01:17.800 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.800 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.800 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:17.800 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255512+0000", 2024-09-21T15:01:17.800 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000129367 2024-09-21T15:01:17.800 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:17.800 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:17.801 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:17.801 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.801 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.801 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139804 3.11 3:88f9ddce:::rbd_data.113664780c85.00000000000000ef:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:17.801 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.224632+0000", 2024-09-21T15:01:17.801 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 15.545742717, 2024-09-21T15:01:17.801 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.018021302, 2024-09-21T15:01:17.801 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:17.801 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:17.801 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:17.801 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:17.801 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:17.801 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139804 2024-09-21T15:01:17.801 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.801 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:17.802 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.802 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:17.802 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224632+0000", 2024-09-21T15:01:17.802 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:17.802 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.802 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.802 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:17.802 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224632+0000", 2024-09-21T15:01:17.802 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:17.802 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.802 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.802 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:17.802 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224632+0000", 2024-09-21T15:01:17.802 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:17.803 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.803 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.803 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:17.803 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224635+0000", 2024-09-21T15:01:17.803 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2710000000000001e-06 2024-09-21T15:01:17.803 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.803 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.803 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:17.803 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224640+0000", 2024-09-21T15:01:17.803 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6779999999999997e-06 2024-09-21T15:01:17.803 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.803 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.803 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:17.803 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224645+0000", 2024-09-21T15:01:17.804 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9339999999999997e-06 2024-09-21T15:01:17.804 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.804 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.804 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:17.804 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225307+0000", 2024-09-21T15:01:17.804 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00066201699999999999 2024-09-21T15:01:17.804 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.804 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.804 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:17.804 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225312+0000", 2024-09-21T15:01:17.804 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9799999999999998e-06 2024-09-21T15:01:17.804 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.804 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.804 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:17.805 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242548+0000", 2024-09-21T15:01:17.805 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.017235603 2024-09-21T15:01:17.805 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.805 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.805 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:17.805 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242653+0000", 2024-09-21T15:01:17.805 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010522 2024-09-21T15:01:17.805 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:17.805 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:17.805 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:17.805 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.805 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.805 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139809 3.11 3:884da946:::rbd_data.113664780c85.0000000000000118:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:17.805 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.224667+0000", 2024-09-21T15:01:17.805 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 15.545707673000001, 2024-09-21T15:01:17.806 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0181101960000001, 2024-09-21T15:01:17.806 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:17.806 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:17.806 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:17.806 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:17.806 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:17.806 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139809 2024-09-21T15:01:17.806 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.806 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:17.806 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.806 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:17.806 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224667+0000", 2024-09-21T15:01:17.806 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:17.806 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.807 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.807 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:17.807 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224667+0000", 2024-09-21T15:01:17.807 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:17.807 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.807 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.807 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:17.807 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224667+0000", 2024-09-21T15:01:17.807 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:17.807 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.807 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.807 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:17.807 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224669+0000", 2024-09-21T15:01:17.807 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.452e-06 2024-09-21T15:01:17.807 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.808 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.808 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:17.808 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224675+0000", 2024-09-21T15:01:17.808 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.818e-06 2024-09-21T15:01:17.808 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.808 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.808 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:17.808 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224680+0000", 2024-09-21T15:01:17.808 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6319999999999999e-06 2024-09-21T15:01:17.808 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.808 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.808 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:17.808 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225400+0000", 2024-09-21T15:01:17.808 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00072012699999999999 2024-09-21T15:01:17.808 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.809 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.809 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:17.809 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225409+0000", 2024-09-21T15:01:17.809 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1039999999999996e-06 2024-09-21T15:01:17.809 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.809 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.809 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:17.809 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242675+0000", 2024-09-21T15:01:17.809 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0172663319999999 2024-09-21T15:01:17.809 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.809 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.809 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:17.809 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242777+0000", 2024-09-21T15:01:17.809 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010197899999999999 2024-09-21T15:01:17.810 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:17.810 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:17.810 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:17.810 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.810 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.810 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139868 3.16 3:69f82090:::rbd_data.113664780c85.00000000000000f1:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:17.810 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.225982+0000", 2024-09-21T15:01:17.810 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 15.544392173, 2024-09-21T15:01:17.810 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0296791729999999, 2024-09-21T15:01:17.810 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:17.810 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:17.810 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:17.810 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:17.810 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:17.810 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139868 2024-09-21T15:01:17.811 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.811 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:17.811 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.811 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:17.811 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225982+0000", 2024-09-21T15:01:17.811 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:17.811 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.811 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.811 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:17.811 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225982+0000", 2024-09-21T15:01:17.811 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:17.811 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.811 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.811 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:17.812 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225982+0000", 2024-09-21T15:01:17.812 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:17.812 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.812 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.812 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:17.812 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225985+0000", 2024-09-21T15:01:17.812 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.74e-06 2024-09-21T15:01:17.812 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.812 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.812 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:17.812 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225990+0000", 2024-09-21T15:01:17.812 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7039999999999999e-06 2024-09-21T15:01:17.812 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.812 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.813 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:17.813 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225995+0000", 2024-09-21T15:01:17.813 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7249999999999997e-06 2024-09-21T15:01:17.813 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.813 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.813 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:17.813 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226299+0000", 2024-09-21T15:01:17.813 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00030383199999999997 2024-09-21T15:01:17.813 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.813 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.813 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:17.813 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226308+0000", 2024-09-21T15:01:17.813 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.9740000000000008e-06 2024-09-21T15:01:17.813 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.813 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.814 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:17.814 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255539+0000", 2024-09-21T15:01:17.814 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0292305960000001 2024-09-21T15:01:17.814 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.814 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.814 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:17.814 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255662+0000", 2024-09-21T15:01:17.814 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012307400000000001 2024-09-21T15:01:17.814 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:17.814 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:17.814 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:17.814 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.814 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.815 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139869 3.16 3:6b6c90a4:::rbd_data.113664780c85.00000000000000fe:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:17.815 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.226006+0000", 2024-09-21T15:01:17.815 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 15.544368557, 2024-09-21T15:01:17.815 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.029790518, 2024-09-21T15:01:17.815 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:17.815 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:17.815 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:17.815 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:17.815 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:17.815 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139869 2024-09-21T15:01:17.815 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.815 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:17.815 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.815 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:17.816 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226006+0000", 2024-09-21T15:01:17.816 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:17.816 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.816 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.816 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:17.816 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226006+0000", 2024-09-21T15:01:17.816 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:17.816 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.816 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.816 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:17.816 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226006+0000", 2024-09-21T15:01:17.816 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:17.816 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.816 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.816 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:17.817 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226008+0000", 2024-09-21T15:01:17.817 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5569999999999998e-06 2024-09-21T15:01:17.817 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.817 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.817 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:17.817 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226014+0000", 2024-09-21T15:01:17.817 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2079999999999997e-06 2024-09-21T15:01:17.817 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.817 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.817 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:17.817 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226020+0000", 2024-09-21T15:01:17.817 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7429999999999999e-06 2024-09-21T15:01:17.817 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.817 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.818 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:17.818 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226330+0000", 2024-09-21T15:01:17.818 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00030963300000000002 2024-09-21T15:01:17.818 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.818 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.818 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:17.818 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226336+0000", 2024-09-21T15:01:17.818 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6340000000000003e-06 2024-09-21T15:01:17.818 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.818 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.818 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:17.818 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255687+0000", 2024-09-21T15:01:17.818 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0293507150000001 2024-09-21T15:01:17.818 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.819 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.819 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:17.819 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255797+0000", 2024-09-21T15:01:17.819 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000109471 2024-09-21T15:01:17.819 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:17.819 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:17.819 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:17.819 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.819 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.819 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139884 3.16 3:69f10324:::rbd_data.113664780c85.0000000000000161:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:17.819 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.227573+0000", 2024-09-21T15:01:17.819 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 15.542801916, 2024-09-21T15:01:17.819 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0283568759999999, 2024-09-21T15:01:17.819 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:17.820 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:17.820 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:17.820 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:17.820 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:17.820 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139884 2024-09-21T15:01:17.820 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.820 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:17.820 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.820 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:17.820 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227573+0000", 2024-09-21T15:01:17.820 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:17.820 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.820 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.820 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:17.820 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227573+0000", 2024-09-21T15:01:17.821 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:17.821 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.821 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.821 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:17.821 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227572+0000", 2024-09-21T15:01:17.821 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:17.821 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.821 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.821 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:17.821 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227575+0000", 2024-09-21T15:01:17.821 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7769999999999998e-06 2024-09-21T15:01:17.821 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.821 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.821 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:17.822 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227581+0000", 2024-09-21T15:01:17.822 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7359999999999998e-06 2024-09-21T15:01:17.822 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.822 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.822 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:17.822 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227585+0000", 2024-09-21T15:01:17.822 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5090000000000004e-06 2024-09-21T15:01:17.822 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.822 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.822 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:17.822 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227753+0000", 2024-09-21T15:01:17.822 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00016761599999999999 2024-09-21T15:01:17.822 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.823 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.823 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:17.823 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227761+0000", 2024-09-21T15:01:17.823 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0579999999999998e-06 2024-09-21T15:01:17.823 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.823 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.823 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:17.823 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255828+0000", 2024-09-21T15:01:17.823 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0280671349999999 2024-09-21T15:01:17.823 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.823 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.823 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:17.823 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255930+0000", 2024-09-21T15:01:17.823 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000101372 2024-09-21T15:01:17.824 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:17.824 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:17.824 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:17.824 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.824 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.824 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139913 3.16 3:6f81b46d:::rbd_data.113664780c85.0000000000000074:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:17.824 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.227685+0000", 2024-09-21T15:01:17.824 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 15.542689737, 2024-09-21T15:01:17.824 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0283644750000001, 2024-09-21T15:01:17.824 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:17.824 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:17.824 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:17.824 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:17.824 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:17.825 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139913 2024-09-21T15:01:17.825 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.825 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:17.825 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.825 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:17.825 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227685+0000", 2024-09-21T15:01:17.825 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:17.825 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.825 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.825 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:17.825 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227685+0000", 2024-09-21T15:01:17.825 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:17.825 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.825 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.825 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:17.826 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227685+0000", 2024-09-21T15:01:17.826 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:17.826 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.826 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.826 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:17.826 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227687+0000", 2024-09-21T15:01:17.826 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2929999999999999e-06 2024-09-21T15:01:17.826 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.826 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.826 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:17.826 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227693+0000", 2024-09-21T15:01:17.826 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1070000000000001e-06 2024-09-21T15:01:17.826 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.826 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.827 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:17.827 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227698+0000", 2024-09-21T15:01:17.827 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9339999999999997e-06 2024-09-21T15:01:17.827 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.827 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.827 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:17.827 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227784+0000", 2024-09-21T15:01:17.827 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6096999999999999e-05 2024-09-21T15:01:17.827 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.827 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.827 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:17.827 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227791+0000", 2024-09-21T15:01:17.827 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9090000000000003e-06 2024-09-21T15:01:17.828 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.828 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.828 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:17.828 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255955+0000", 2024-09-21T15:01:17.828 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0281643579999999 2024-09-21T15:01:17.828 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.828 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.828 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:17.828 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256049+0000", 2024-09-21T15:01:17.828 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4047000000000003e-05 2024-09-21T15:01:17.828 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:17.828 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:17.828 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:17.828 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.828 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.828 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139988 3.11 3:88a29fa4:::rbd_data.113664780c85.00000000000000bd:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:17.829 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.229772+0000", 2024-09-21T15:01:17.829 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 15.540602238, 2024-09-21T15:01:17.829 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.017967668, 2024-09-21T15:01:17.829 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:17.829 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:17.829 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:17.829 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:17.829 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:17.829 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139988 2024-09-21T15:01:17.829 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.829 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:17.829 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.829 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:17.829 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229772+0000", 2024-09-21T15:01:17.830 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:17.830 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.830 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.830 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:17.830 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229772+0000", 2024-09-21T15:01:17.830 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:17.830 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.830 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.830 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:17.830 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229772+0000", 2024-09-21T15:01:17.830 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:17.830 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.830 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.830 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:17.830 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229776+0000", 2024-09-21T15:01:17.831 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6890000000000002e-06 2024-09-21T15:01:17.831 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.831 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.831 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:17.831 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229783+0000", 2024-09-21T15:01:17.831 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2280000000000001e-06 2024-09-21T15:01:17.831 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.831 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.831 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:17.831 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229789+0000", 2024-09-21T15:01:17.831 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1700000000000002e-06 2024-09-21T15:01:17.831 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.831 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.831 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:17.832 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.230106+0000", 2024-09-21T15:01:17.832 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00031700800000000003 2024-09-21T15:01:17.832 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.832 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.832 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:17.832 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.230115+0000", 2024-09-21T15:01:17.832 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4190000000000008e-06 2024-09-21T15:01:17.832 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.832 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.832 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:17.832 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242935+0000", 2024-09-21T15:01:17.832 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0128191449999999 2024-09-21T15:01:17.832 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.832 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.832 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for missing object", 2024-09-21T15:01:17.833 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.243078+0000", 2024-09-21T15:01:17.833 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000142984 2024-09-21T15:01:17.833 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.833 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.833 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:17.833 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.247659+0000", 2024-09-21T15:01:17.833 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0045813030000000001 2024-09-21T15:01:17.833 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.833 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.833 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:17.833 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.247740+0000", 2024-09-21T15:01:17.833 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1216000000000003e-05 2024-09-21T15:01:17.833 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:17.833 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:17.834 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:17.834 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.834 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.834 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139994 3.16 3:6b9df8d1:::rbd_data.113664780c85.00000000000000e7:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:17.834 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.231370+0000", 2024-09-21T15:01:17.834 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 15.539004159999999, 2024-09-21T15:01:17.834 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.024795576, 2024-09-21T15:01:17.834 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:17.834 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:17.834 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:17.834 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:17.834 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:17.834 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139994 2024-09-21T15:01:17.834 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.835 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:17.835 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.835 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:17.835 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231370+0000", 2024-09-21T15:01:17.835 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:17.835 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.835 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.835 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:17.835 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231370+0000", 2024-09-21T15:01:17.835 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:17.835 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.835 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.835 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:17.835 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231369+0000", 2024-09-21T15:01:17.835 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:17.836 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.836 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.836 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:17.836 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231376+0000", 2024-09-21T15:01:17.836 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5509999999999996e-06 2024-09-21T15:01:17.836 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.836 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.836 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:17.836 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231387+0000", 2024-09-21T15:01:17.836 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.061e-05 2024-09-21T15:01:17.836 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.836 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.836 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:17.836 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231394+0000", 2024-09-21T15:01:17.836 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3799999999999996e-06 2024-09-21T15:01:17.837 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.837 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.837 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:17.837 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231459+0000", 2024-09-21T15:01:17.837 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4716999999999997e-05 2024-09-21T15:01:17.837 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.837 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.837 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:17.837 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231468+0000", 2024-09-21T15:01:17.837 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1479999999999995e-06 2024-09-21T15:01:17.837 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.837 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.837 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:17.837 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256074+0000", 2024-09-21T15:01:17.837 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0246064180000001 2024-09-21T15:01:17.838 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.838 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.838 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:17.838 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256166+0000", 2024-09-21T15:01:17.838 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1862999999999998e-05 2024-09-21T15:01:17.838 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:17.838 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:17.838 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:17.838 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.838 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.838 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140012 3.16 3:6bd6dfb6:::rbd_data.113664780c85.0000000000000165:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:17.838 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.231890+0000", 2024-09-21T15:01:17.838 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 15.538484401, 2024-09-21T15:01:17.838 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.024408881, 2024-09-21T15:01:17.838 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:17.839 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:17.839 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:17.839 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:17.839 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:17.839 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140012 2024-09-21T15:01:17.839 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.839 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:17.839 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.839 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:17.839 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231890+0000", 2024-09-21T15:01:17.839 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:17.839 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.839 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.839 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:17.840 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231890+0000", 2024-09-21T15:01:17.840 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:17.840 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.840 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.840 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:17.840 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231889+0000", 2024-09-21T15:01:17.840 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:17.840 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.840 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.840 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:17.840 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231895+0000", 2024-09-21T15:01:17.840 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9569999999999999e-06 2024-09-21T15:01:17.840 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.840 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.841 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:17.841 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231910+0000", 2024-09-21T15:01:17.841 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4549000000000001e-05 2024-09-21T15:01:17.841 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.841 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.841 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:17.841 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231917+0000", 2024-09-21T15:01:17.841 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5730000000000002e-06 2024-09-21T15:01:17.841 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.841 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.841 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:17.841 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231988+0000", 2024-09-21T15:01:17.841 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0852999999999994e-05 2024-09-21T15:01:17.841 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.841 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.842 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:17.842 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.232000+0000", 2024-09-21T15:01:17.842 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2159e-05 2024-09-21T15:01:17.842 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.842 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.842 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:17.842 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256192+0000", 2024-09-21T15:01:17.842 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.024192073 2024-09-21T15:01:17.842 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.842 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.842 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:17.842 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256299+0000", 2024-09-21T15:01:17.842 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000106733 2024-09-21T15:01:17.842 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:17.843 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:17.843 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:17.843 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.843 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.843 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140055 3.16 3:69bb9803:::rbd_data.113664780c85.00000000000000da:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:17.843 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.233922+0000", 2024-09-21T15:01:17.843 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 15.536452872, 2024-09-21T15:01:17.843 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.022511768, 2024-09-21T15:01:17.843 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:17.843 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:17.843 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:17.843 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:17.843 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:17.843 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140055 2024-09-21T15:01:17.844 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.844 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:17.844 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.844 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:17.844 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233922+0000", 2024-09-21T15:01:17.844 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:17.844 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.844 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.844 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:17.844 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233922+0000", 2024-09-21T15:01:17.844 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:17.844 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.844 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.844 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:17.844 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233921+0000", 2024-09-21T15:01:17.845 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:17.845 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.845 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.845 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:17.845 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233927+0000", 2024-09-21T15:01:17.845 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0360000000000003e-06 2024-09-21T15:01:17.845 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.845 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.845 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:17.845 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233937+0000", 2024-09-21T15:01:17.845 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6439999999999995e-06 2024-09-21T15:01:17.845 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.845 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.845 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:17.845 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233944+0000", 2024-09-21T15:01:17.846 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7710000000000006e-06 2024-09-21T15:01:17.846 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.846 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.846 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:17.846 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233996+0000", 2024-09-21T15:01:17.846 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1597000000000001e-05 2024-09-21T15:01:17.846 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.846 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.846 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:17.846 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.234005+0000", 2024-09-21T15:01:17.846 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5960000000000004e-06 2024-09-21T15:01:17.846 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.846 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.846 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:17.846 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256332+0000", 2024-09-21T15:01:17.846 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0223271949999999 2024-09-21T15:01:17.847 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.847 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.847 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:17.847 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256434+0000", 2024-09-21T15:01:17.847 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010180599999999999 2024-09-21T15:01:17.847 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:17.847 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:17.847 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:17.847 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.847 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.847 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140143 3.16 3:6d3ed596:::rbd_data.113664780c85.0000000000000184:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:17.847 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.239496+0000", 2024-09-21T15:01:17.847 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 15.530878342999999, 2024-09-21T15:01:17.847 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.017076576, 2024-09-21T15:01:17.847 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:17.848 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:17.848 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:17.848 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:17.848 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:17.848 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140143 2024-09-21T15:01:17.848 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.848 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:17.848 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.848 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:17.848 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239496+0000", 2024-09-21T15:01:17.848 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:17.848 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.848 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.848 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:17.848 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239496+0000", 2024-09-21T15:01:17.849 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:17.849 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.849 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.849 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:17.849 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239496+0000", 2024-09-21T15:01:17.849 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:17.849 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.849 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.849 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:17.849 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239498+0000", 2024-09-21T15:01:17.849 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.509e-06 2024-09-21T15:01:17.849 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.849 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.849 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:17.849 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239504+0000", 2024-09-21T15:01:17.849 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6869999999999999e-06 2024-09-21T15:01:17.850 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.850 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.850 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:17.850 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239509+0000", 2024-09-21T15:01:17.850 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8860000000000003e-06 2024-09-21T15:01:17.850 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.850 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.850 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:17.850 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239572+0000", 2024-09-21T15:01:17.850 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2908000000000001e-05 2024-09-21T15:01:17.850 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.850 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.850 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:17.850 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239580+0000", 2024-09-21T15:01:17.851 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4270000000000008e-06 2024-09-21T15:01:17.851 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.851 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.851 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:17.851 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256457+0000", 2024-09-21T15:01:17.851 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0168770979999999 2024-09-21T15:01:17.851 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.851 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.851 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:17.851 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256573+0000", 2024-09-21T15:01:17.851 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00011538400000000001 2024-09-21T15:01:17.851 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:17.851 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:17.851 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:17.851 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.851 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.852 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140163 3.16 3:694e82eb:::rbd_data.113664780c85.0000000000000052:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:17.852 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.240334+0000", 2024-09-21T15:01:17.852 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 15.530040631, 2024-09-21T15:01:17.852 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.016365537, 2024-09-21T15:01:17.852 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:17.852 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:17.852 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:17.852 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:17.852 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:17.852 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140163 2024-09-21T15:01:17.852 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.852 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:17.852 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.852 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:17.852 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240334+0000", 2024-09-21T15:01:17.852 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:17.853 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.853 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.853 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:17.853 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240334+0000", 2024-09-21T15:01:17.853 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:17.853 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.853 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.853 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:17.853 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240334+0000", 2024-09-21T15:01:17.853 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:17.853 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.853 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.853 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:17.853 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240337+0000", 2024-09-21T15:01:17.853 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.089e-06 2024-09-21T15:01:17.853 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.854 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.854 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:17.854 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240343+0000", 2024-09-21T15:01:17.854 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4420000000000003e-06 2024-09-21T15:01:17.854 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.854 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.854 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:17.854 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240348+0000", 2024-09-21T15:01:17.854 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1039999999999998e-06 2024-09-21T15:01:17.854 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.854 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.854 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:17.854 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240451+0000", 2024-09-21T15:01:17.854 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000102996 2024-09-21T15:01:17.855 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.855 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.855 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:17.855 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240460+0000", 2024-09-21T15:01:17.855 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5369999999999997e-06 2024-09-21T15:01:17.855 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.855 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.855 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:17.855 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256600+0000", 2024-09-21T15:01:17.855 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.016139884 2024-09-21T15:01:17.855 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.855 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.855 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:17.855 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256700+0000", 2024-09-21T15:01:17.855 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9844000000000002e-05 2024-09-21T15:01:17.855 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:17.856 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:17.856 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:17.856 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.856 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.856 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140164 3.16 3:68bfdc57:::rbd_data.113664780c85.0000000000000053:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:17.856 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.240358+0000", 2024-09-21T15:01:17.856 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 15.530016936999999, 2024-09-21T15:01:17.856 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.01646094, 2024-09-21T15:01:17.856 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:17.856 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:17.856 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:17.856 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:17.856 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:17.856 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140164 2024-09-21T15:01:17.856 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.856 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:17.857 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.857 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:17.857 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240358+0000", 2024-09-21T15:01:17.857 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:17.857 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.857 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.857 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:17.857 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240358+0000", 2024-09-21T15:01:17.857 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:17.857 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.857 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.857 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:17.857 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240357+0000", 2024-09-21T15:01:17.857 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:17.857 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.857 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.858 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:17.858 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240360+0000", 2024-09-21T15:01:17.858 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7089999999999998e-06 2024-09-21T15:01:17.858 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.858 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.858 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:17.858 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240368+0000", 2024-09-21T15:01:17.858 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7270000000000007e-06 2024-09-21T15:01:17.858 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.858 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.858 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:17.858 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240373+0000", 2024-09-21T15:01:17.858 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4060000000000004e-06 2024-09-21T15:01:17.858 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.859 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.859 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:17.859 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240473+0000", 2024-09-21T15:01:17.859 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000100132 2024-09-21T15:01:17.859 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.859 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.859 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:17.859 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240476+0000", 2024-09-21T15:01:17.859 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.332e-06 2024-09-21T15:01:17.859 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.859 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.859 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:17.859 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256724+0000", 2024-09-21T15:01:17.859 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0162471930000001 2024-09-21T15:01:17.859 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.859 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.860 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:17.860 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256819+0000", 2024-09-21T15:01:17.860 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5007999999999996e-05 2024-09-21T15:01:17.860 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:17.860 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:17.860 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:17.860 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.860 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.860 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140188 3.16 3:6f6217d4:::rbd_data.113664780c85.00000000000000fb:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:17.860 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.240978+0000", 2024-09-21T15:01:17.860 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 15.529396278, 2024-09-21T15:01:17.860 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.015983809, 2024-09-21T15:01:17.860 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:17.860 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:17.860 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:17.860 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:17.861 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:17.861 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140188 2024-09-21T15:01:17.861 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.861 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:17.861 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.861 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:17.861 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240978+0000", 2024-09-21T15:01:17.861 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:17.861 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.861 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.861 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:17.861 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240978+0000", 2024-09-21T15:01:17.861 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:17.861 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.861 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.862 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:17.862 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240978+0000", 2024-09-21T15:01:17.862 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:17.862 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.862 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.862 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:17.862 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240981+0000", 2024-09-21T15:01:17.862 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1559999999999999e-06 2024-09-21T15:01:17.862 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.862 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.862 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:17.862 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240987+0000", 2024-09-21T15:01:17.862 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.229e-06 2024-09-21T15:01:17.862 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.862 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.863 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:17.863 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240992+0000", 2024-09-21T15:01:17.863 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9710000000000003e-06 2024-09-21T15:01:17.863 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.863 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.863 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:17.863 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.241361+0000", 2024-09-21T15:01:17.863 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000369236 2024-09-21T15:01:17.863 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.863 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.863 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:17.863 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.241369+0000", 2024-09-21T15:01:17.863 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6199999999999999e-06 2024-09-21T15:01:17.863 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.864 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.864 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:17.864 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256844+0000", 2024-09-21T15:01:17.864 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0154751609999999 2024-09-21T15:01:17.864 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:17.864 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:17.864 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:17.864 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256962+0000", 2024-09-21T15:01:17.864 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000117964 2024-09-21T15:01:17.864 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:17.864 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:17.864 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:17.864 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:17.864 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:17.865 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:17.865 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:01:17.766+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:01:17.866 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T15:01:17.868 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T15:01:17.869 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:01:17.862+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:01:17.890 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T15:01:17.912 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T15:01:17.935 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T15:01:17.961 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-21T15:01:17.968 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:01:17.965+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:01:17.984 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:17.985 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:17.986 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T15:01:17.995 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T15:01:18.009 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T15:01:18.034 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T15:01:18.037 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:01:18.037 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:01:18.037 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:01:18.037 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:01:18.037 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:01:18.037 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:01:18.037 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:01:18.038 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:01:18.093 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:01:18.090+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:01:18.104 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:18.104 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:18.114 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T15:01:18.169 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:01:18.166+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:01:18.222 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:18.222 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:18.232 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T15:01:18.270 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:01:18.265+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:01:18.342 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:18.343 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:18.352 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T15:01:18.371 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:01:18.369+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:01:18.463 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:18.464 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:18.472 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:01:18.469+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:01:18.474 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T15:01:18.573 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:01:18.566+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:01:18.625 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:18.625 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:18.635 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T15:01:18.673 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:01:18.670+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:01:18.745 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:18.745 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:18.755 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T15:01:18.775 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:01:18.773+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:01:18.862 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:18.863 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:18.873 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T15:01:18.875 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:01:18.870+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:01:18.976 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:01:18.972+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:01:19.024 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:19.025 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:19.034 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T15:01:19.076 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:01:19.070+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:01:19.140 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:19.140 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:19.151 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T15:01:19.177 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:01:19.172+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:01:19.257 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:19.257 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:19.267 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T15:01:19.279 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:01:19.278+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:01:19.351 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T15:01:19.380 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:19.380 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:19.391 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T15:01:19.397 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:01:19.394+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:01:19.480 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:01:19.474+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:01:19.516 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:19.516 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:19.526 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T15:01:19.581 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:01:19.576+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:01:19.628 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:19.628 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:19.638 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T15:01:19.682 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:01:19.678+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:01:19.741 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:19.741 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:19.752 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T15:01:19.783 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:01:19.780+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:01:19.884 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:01:19.880+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:01:19.902 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:19.903 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:19.913 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T15:01:19.984 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:01:19.980+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:01:20.017 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:20.017 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:20.027 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T15:01:20.051 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T15:01:20.085 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:01:20.080+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:01:20.132 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:20.132 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:20.142 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T15:01:20.168 INFO:teuthology.orchestra.run.smithi032.stderr:nodeep-scrub is set 2024-09-21T15:01:20.185 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:01:20.180+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:01:20.197 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T15:01:20.218 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T15:01:20.239 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T15:01:20.254 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:20.254 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:01:20.254 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T15:01:20.254 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:20.260 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T15:01:20.265 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T15:01:20.286 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:01:20.280+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:01:20.291 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-21T15:01:20.327 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T15:01:20.347 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T15:01:20.376 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T15:01:20.378 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:01:20.378 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:01:20.379 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:01:20.379 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:01:20.379 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:01:20.379 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:01:20.379 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:01:20.379 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:01:20.384 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 5, 6, 7, 4] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T15:01:20.384 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T15:01:20.385 INFO:tasks.thrashosds.thrasher:Removing osd 4, in_osds are: [0, 1, 2, 3, 5, 6, 7, 4] 2024-09-21T15:01:20.385 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 4 2024-09-21T15:01:20.387 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:01:20.382+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:01:20.387 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:20.387 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:01:20.388 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T15:01:20.388 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T15:01:20.388 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:20.398 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T15:01:20.488 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:01:20.482+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:01:20.509 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:20.509 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T15:01:20.509 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T15:01:20.509 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T15:01:20.509 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.509 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:15130 3.2 3:47f1bbdb:::rbd_data.11361bed1f78.0000000000000067:head [sparse-read 0~147456] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-21T15:01:20.509 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:40.288284+0000", 2024-09-21T15:01:20.509 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 160.21970909300001, 2024-09-21T15:01:20.509 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1142935039999999, 2024-09-21T15:01:20.510 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:20.510 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:20.510 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:20.510 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:20.510 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:20.510 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 15130 2024-09-21T15:01:20.510 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.510 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:20.510 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.510 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:20.510 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288284+0000", 2024-09-21T15:01:20.510 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:20.510 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.511 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.511 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:20.511 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288284+0000", 2024-09-21T15:01:20.511 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:20.511 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.511 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.511 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:20.511 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288284+0000", 2024-09-21T15:01:20.511 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:20.511 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.511 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.511 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:20.511 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288288+0000", 2024-09-21T15:01:20.511 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5829999999999998e-06 2024-09-21T15:01:20.512 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.512 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.512 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:20.512 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288297+0000", 2024-09-21T15:01:20.512 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0669999999999998e-06 2024-09-21T15:01:20.512 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.512 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.512 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:20.512 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288305+0000", 2024-09-21T15:01:20.512 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8860000000000004e-06 2024-09-21T15:01:20.512 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.512 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.512 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:20.513 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288417+0000", 2024-09-21T15:01:20.513 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000112952 2024-09-21T15:01:20.513 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.513 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.513 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:20.513 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288428+0000", 2024-09-21T15:01:20.513 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0631e-05 2024-09-21T15:01:20.513 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.513 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.513 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:20.513 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.401876+0000", 2024-09-21T15:01:20.513 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1134479610000001 2024-09-21T15:01:20.513 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.513 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.514 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:20.514 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.402076+0000", 2024-09-21T15:01:20.514 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00019944200000000001 2024-09-21T15:01:20.514 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.514 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.514 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:20.514 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.402578+0000", 2024-09-21T15:01:20.514 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00050242899999999998 2024-09-21T15:01:20.514 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:20.514 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:20.514 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:20.514 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.514 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.514 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45708 3.2 3:415db91a:::rbd_data.1136ecbe5879.0000000000000000:28 [read 140288~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:20.515 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141801+0000", 2024-09-21T15:01:20.515 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 130.366192899, 2024-09-21T15:01:20.515 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034274200999999997, 2024-09-21T15:01:20.515 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:20.515 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:20.515 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:20.515 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:20.515 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:20.515 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45708 2024-09-21T15:01:20.515 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.515 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:20.515 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.515 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:20.515 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141801+0000", 2024-09-21T15:01:20.515 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:20.516 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.516 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.516 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:20.516 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141801+0000", 2024-09-21T15:01:20.516 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:20.516 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.516 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.516 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:20.516 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141800+0000", 2024-09-21T15:01:20.516 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:20.516 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.516 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.516 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:20.516 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141803+0000", 2024-09-21T15:01:20.517 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4210000000000002e-06 2024-09-21T15:01:20.517 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.517 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.517 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:20.517 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141812+0000", 2024-09-21T15:01:20.517 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.9800000000000004e-06 2024-09-21T15:01:20.517 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.517 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.517 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:20.517 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141816+0000", 2024-09-21T15:01:20.517 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4769999999999997e-06 2024-09-21T15:01:20.517 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.517 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.517 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:20.518 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.175865+0000", 2024-09-21T15:01:20.518 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034048875999999999 2024-09-21T15:01:20.518 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.518 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.518 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:20.518 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.175943+0000", 2024-09-21T15:01:20.518 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7464000000000002e-05 2024-09-21T15:01:20.518 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.518 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.518 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:20.518 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176075+0000", 2024-09-21T15:01:20.518 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013230800000000001 2024-09-21T15:01:20.518 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:20.519 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:20.519 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:20.519 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.519 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.519 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45709 3.2 3:415db91a:::rbd_data.1136ecbe5879.0000000000000000:28 [read 143360~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:20.519 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141820+0000", 2024-09-21T15:01:20.519 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 130.36617305199999, 2024-09-21T15:01:20.519 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034537536000000001, 2024-09-21T15:01:20.519 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:20.519 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:20.519 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:20.519 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:20.520 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:20.520 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45709 2024-09-21T15:01:20.520 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.520 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:20.520 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.520 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:20.520 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141820+0000", 2024-09-21T15:01:20.520 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:20.520 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.520 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.520 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:20.520 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141820+0000", 2024-09-21T15:01:20.520 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:20.520 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.520 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.521 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:20.521 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141820+0000", 2024-09-21T15:01:20.521 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:20.521 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.521 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.521 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:20.521 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141823+0000", 2024-09-21T15:01:20.521 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5110000000000002e-06 2024-09-21T15:01:20.521 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.521 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.521 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:20.521 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141829+0000", 2024-09-21T15:01:20.521 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0850000000000002e-06 2024-09-21T15:01:20.521 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.521 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.522 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:20.522 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141833+0000", 2024-09-21T15:01:20.522 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7620000000000003e-06 2024-09-21T15:01:20.522 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.522 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.522 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:20.522 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176103+0000", 2024-09-21T15:01:20.522 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034269933000000002 2024-09-21T15:01:20.522 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.522 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.522 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:20.522 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176161+0000", 2024-09-21T15:01:20.522 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7658999999999998e-05 2024-09-21T15:01:20.522 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.523 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.523 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:20.523 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176358+0000", 2024-09-21T15:01:20.523 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000196937 2024-09-21T15:01:20.523 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:20.523 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:20.523 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:20.523 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.523 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.523 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45710 3.2 3:415db91a:::rbd_data.1136ecbe5879.0000000000000000:28 [read 146432~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:20.523 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141838+0000", 2024-09-21T15:01:20.523 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 130.366155889, 2024-09-21T15:01:20.523 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034953674999999997, 2024-09-21T15:01:20.523 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:20.524 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:20.524 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:20.524 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:20.524 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:20.524 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45710 2024-09-21T15:01:20.524 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.524 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:20.524 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.524 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:20.524 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141838+0000", 2024-09-21T15:01:20.524 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:20.524 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.524 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.524 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:20.525 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141838+0000", 2024-09-21T15:01:20.525 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:20.525 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.525 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.525 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:20.525 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141837+0000", 2024-09-21T15:01:20.525 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:20.525 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.525 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.525 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:20.525 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141843+0000", 2024-09-21T15:01:20.525 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1619999999999996e-06 2024-09-21T15:01:20.525 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.525 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.525 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:20.526 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141850+0000", 2024-09-21T15:01:20.526 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4799999999999998e-06 2024-09-21T15:01:20.526 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.526 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.526 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:20.526 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141853+0000", 2024-09-21T15:01:20.526 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3050000000000001e-06 2024-09-21T15:01:20.526 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.526 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.526 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:20.526 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176490+0000", 2024-09-21T15:01:20.526 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034636413999999997 2024-09-21T15:01:20.526 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.526 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.527 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:20.527 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176606+0000", 2024-09-21T15:01:20.527 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000115841 2024-09-21T15:01:20.527 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.527 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.527 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:20.527 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176791+0000", 2024-09-21T15:01:20.527 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00018581500000000001 2024-09-21T15:01:20.527 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:20.527 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:20.527 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:20.527 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.527 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.527 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45711 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:20.528 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141856+0000", 2024-09-21T15:01:20.528 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 130.366137552, 2024-09-21T15:01:20.528 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034986308000000001, 2024-09-21T15:01:20.528 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:20.528 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:20.528 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:20.528 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:20.528 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:20.528 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45711 2024-09-21T15:01:20.528 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.528 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:20.528 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.528 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:20.528 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141856+0000", 2024-09-21T15:01:20.528 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:20.529 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.529 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.529 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:20.529 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141856+0000", 2024-09-21T15:01:20.529 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:20.529 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.529 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.529 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:20.529 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141856+0000", 2024-09-21T15:01:20.529 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:20.529 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.529 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.529 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:20.529 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141862+0000", 2024-09-21T15:01:20.530 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.567e-06 2024-09-21T15:01:20.530 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.530 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.530 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:20.530 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141868+0000", 2024-09-21T15:01:20.530 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9569999999999999e-06 2024-09-21T15:01:20.530 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.530 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.530 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:20.530 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141871+0000", 2024-09-21T15:01:20.530 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.146e-06 2024-09-21T15:01:20.530 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.530 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.530 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:20.531 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176831+0000", 2024-09-21T15:01:20.531 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034959935999999997 2024-09-21T15:01:20.531 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.531 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.531 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:20.531 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176842+0000", 2024-09-21T15:01:20.531 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1008e-05 2024-09-21T15:01:20.531 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:20.531 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:20.531 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:20.531 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.531 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.531 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45712 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:20.532 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141874+0000", 2024-09-21T15:01:20.532 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 130.36611965700001, 2024-09-21T15:01:20.532 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034993511999999997, 2024-09-21T15:01:20.532 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:20.532 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:20.532 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:20.532 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:20.532 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:20.532 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45712 2024-09-21T15:01:20.532 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.532 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:20.532 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.532 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:20.532 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141874+0000", 2024-09-21T15:01:20.532 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:20.533 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.533 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.533 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:20.533 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141874+0000", 2024-09-21T15:01:20.533 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:20.533 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.533 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.533 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:20.533 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141874+0000", 2024-09-21T15:01:20.533 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:20.533 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.533 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.533 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:20.533 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141875+0000", 2024-09-21T15:01:20.534 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5430000000000001e-06 2024-09-21T15:01:20.534 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.534 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.534 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:20.534 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141878+0000", 2024-09-21T15:01:20.534 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1420000000000002e-06 2024-09-21T15:01:20.534 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.534 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.534 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:20.534 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141881+0000", 2024-09-21T15:01:20.534 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.807e-06 2024-09-21T15:01:20.534 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.534 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.534 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:20.535 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176860+0000", 2024-09-21T15:01:20.535 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034979243 2024-09-21T15:01:20.535 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.535 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.535 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:20.535 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176867+0000", 2024-09-21T15:01:20.535 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0169999999999999e-06 2024-09-21T15:01:20.535 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:20.535 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:20.535 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:20.535 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.535 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.535 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45713 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:20.535 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141883+0000", 2024-09-21T15:01:20.535 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 130.36611008200001, 2024-09-21T15:01:20.536 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035010516999999998, 2024-09-21T15:01:20.536 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:20.536 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:20.536 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:20.536 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:20.536 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:20.536 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45713 2024-09-21T15:01:20.536 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.536 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:20.536 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.536 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:20.536 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141883+0000", 2024-09-21T15:01:20.536 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:20.536 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.537 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.537 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:20.537 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141883+0000", 2024-09-21T15:01:20.537 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:20.537 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.537 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.537 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:20.537 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141883+0000", 2024-09-21T15:01:20.537 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:20.537 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.537 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.537 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:20.537 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141885+0000", 2024-09-21T15:01:20.537 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4649999999999999e-06 2024-09-21T15:01:20.538 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.538 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.538 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:20.538 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141888+0000", 2024-09-21T15:01:20.538 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3890000000000001e-06 2024-09-21T15:01:20.538 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.538 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.538 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:20.538 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141891+0000", 2024-09-21T15:01:20.538 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9239999999999999e-06 2024-09-21T15:01:20.538 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.538 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.538 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:20.538 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176887+0000", 2024-09-21T15:01:20.538 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034996370999999998 2024-09-21T15:01:20.539 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.539 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.539 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:20.539 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176894+0000", 2024-09-21T15:01:20.539 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6069999999999996e-06 2024-09-21T15:01:20.539 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:20.539 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:20.539 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:20.539 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.539 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.539 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45714 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:20.539 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141893+0000", 2024-09-21T15:01:20.539 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 130.36610016700001, 2024-09-21T15:01:20.539 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035048215000000001, 2024-09-21T15:01:20.540 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:20.540 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:20.540 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:20.540 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:20.540 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:20.540 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45714 2024-09-21T15:01:20.540 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.540 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:20.540 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.540 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:20.540 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141893+0000", 2024-09-21T15:01:20.540 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:20.540 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.540 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.540 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:20.541 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141893+0000", 2024-09-21T15:01:20.541 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:20.541 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.541 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.541 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:20.541 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141893+0000", 2024-09-21T15:01:20.541 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:20.541 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.541 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.541 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:20.541 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141895+0000", 2024-09-21T15:01:20.541 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.703e-06 2024-09-21T15:01:20.541 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.541 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.542 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:20.542 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141900+0000", 2024-09-21T15:01:20.542 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.242e-06 2024-09-21T15:01:20.542 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.542 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.542 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:20.542 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141903+0000", 2024-09-21T15:01:20.542 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.117e-06 2024-09-21T15:01:20.542 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.542 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.542 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:20.542 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176930+0000", 2024-09-21T15:01:20.542 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035026750000000002 2024-09-21T15:01:20.543 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.543 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.543 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:20.543 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176942+0000", 2024-09-21T15:01:20.543 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1668000000000001e-05 2024-09-21T15:01:20.543 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:20.543 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:20.543 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:20.543 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.543 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.543 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45715 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:20.543 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141906+0000", 2024-09-21T15:01:20.543 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 130.36608800900001, 2024-09-21T15:01:20.543 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035066057999999997, 2024-09-21T15:01:20.543 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:20.544 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:20.544 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:20.544 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:20.544 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:20.544 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45715 2024-09-21T15:01:20.544 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.544 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:20.544 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.544 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:20.544 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141906+0000", 2024-09-21T15:01:20.544 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:20.544 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.544 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.545 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:20.545 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141906+0000", 2024-09-21T15:01:20.545 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:20.545 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.545 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.545 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:20.545 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141905+0000", 2024-09-21T15:01:20.545 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:20.545 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.545 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.545 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:20.545 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141907+0000", 2024-09-21T15:01:20.545 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5400000000000001e-06 2024-09-21T15:01:20.545 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.546 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.546 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:20.546 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141912+0000", 2024-09-21T15:01:20.546 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.502e-06 2024-09-21T15:01:20.546 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.546 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.546 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:20.546 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141915+0000", 2024-09-21T15:01:20.546 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0170000000000001e-06 2024-09-21T15:01:20.546 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.546 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.546 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:20.546 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176964+0000", 2024-09-21T15:01:20.546 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035048587999999999 2024-09-21T15:01:20.546 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.547 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.547 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:20.547 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176972+0000", 2024-09-21T15:01:20.547 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6580000000000002e-06 2024-09-21T15:01:20.547 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:20.547 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:20.547 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:20.547 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.547 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.547 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45716 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:20.547 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141918+0000", 2024-09-21T15:01:20.547 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 130.366075833, 2024-09-21T15:01:20.547 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035086223, 2024-09-21T15:01:20.549 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:20.549 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:20.549 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:20.549 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:20.549 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:20.549 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45716 2024-09-21T15:01:20.549 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.549 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:20.549 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.549 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:20.550 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141918+0000", 2024-09-21T15:01:20.550 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:20.550 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.550 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.550 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:20.550 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141918+0000", 2024-09-21T15:01:20.550 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:20.550 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.550 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.550 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:20.550 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141917+0000", 2024-09-21T15:01:20.550 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:20.550 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.550 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.550 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:20.550 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141919+0000", 2024-09-21T15:01:20.551 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5680000000000001e-06 2024-09-21T15:01:20.551 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.551 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.551 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:20.551 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141922+0000", 2024-09-21T15:01:20.551 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2789999999999999e-06 2024-09-21T15:01:20.551 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.551 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.551 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:20.551 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141925+0000", 2024-09-21T15:01:20.551 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9469999999999999e-06 2024-09-21T15:01:20.551 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.551 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.552 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:20.552 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176997+0000", 2024-09-21T15:01:20.552 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035071366999999999 2024-09-21T15:01:20.552 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.552 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.552 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:20.552 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177004+0000", 2024-09-21T15:01:20.552 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3309999999999998e-06 2024-09-21T15:01:20.552 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:20.552 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:20.552 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:20.552 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.552 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.552 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45717 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:20.552 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141928+0000", 2024-09-21T15:01:20.553 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 130.36606588699999, 2024-09-21T15:01:20.553 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035102573999999998, 2024-09-21T15:01:20.553 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:20.553 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:20.553 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:20.553 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:20.553 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:20.553 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45717 2024-09-21T15:01:20.553 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.553 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:20.553 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.553 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:20.553 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141928+0000", 2024-09-21T15:01:20.553 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:20.554 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.554 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.554 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:20.554 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141928+0000", 2024-09-21T15:01:20.554 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:20.554 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.554 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.554 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:20.554 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141927+0000", 2024-09-21T15:01:20.554 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:01:20.554 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.554 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.554 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:20.554 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141929+0000", 2024-09-21T15:01:20.555 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.533e-06 2024-09-21T15:01:20.555 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.555 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.555 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:20.555 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141934+0000", 2024-09-21T15:01:20.555 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1660000000000002e-06 2024-09-21T15:01:20.555 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.555 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.555 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:20.555 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141937+0000", 2024-09-21T15:01:20.555 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0309999999999999e-06 2024-09-21T15:01:20.555 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.555 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.555 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:20.556 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177023+0000", 2024-09-21T15:01:20.556 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035085595999999997 2024-09-21T15:01:20.556 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.556 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.556 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:20.556 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177030+0000", 2024-09-21T15:01:20.556 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.486e-06 2024-09-21T15:01:20.556 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:20.556 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:20.556 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:20.556 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.556 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.556 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45718 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:20.556 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141940+0000", 2024-09-21T15:01:20.556 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 130.366054029, 2024-09-21T15:01:20.557 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035124956999999998, 2024-09-21T15:01:20.557 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:20.557 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:20.557 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:20.557 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:20.557 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:20.557 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45718 2024-09-21T15:01:20.557 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.557 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:20.557 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.557 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:20.557 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141940+0000", 2024-09-21T15:01:20.557 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:20.557 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.557 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.558 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:20.558 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141940+0000", 2024-09-21T15:01:20.558 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:20.558 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.558 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.558 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:20.558 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141939+0000", 2024-09-21T15:01:20.558 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:20.558 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.558 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.558 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:20.558 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141941+0000", 2024-09-21T15:01:20.558 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.502e-06 2024-09-21T15:01:20.558 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.559 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.559 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:20.559 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141946+0000", 2024-09-21T15:01:20.559 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3449999999999996e-06 2024-09-21T15:01:20.559 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.559 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.559 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:20.559 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141949+0000", 2024-09-21T15:01:20.559 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9749999999999999e-06 2024-09-21T15:01:20.559 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.559 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.559 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:20.559 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177057+0000", 2024-09-21T15:01:20.559 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035108385999999998 2024-09-21T15:01:20.559 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.560 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.560 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:20.560 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177064+0000", 2024-09-21T15:01:20.560 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0099999999999998e-06 2024-09-21T15:01:20.560 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:20.560 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:20.560 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:20.560 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.560 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.560 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45719 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:20.560 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141952+0000", 2024-09-21T15:01:20.560 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 130.36604175599999, 2024-09-21T15:01:20.560 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035139108000000002, 2024-09-21T15:01:20.560 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:20.561 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:20.561 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:20.561 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:20.561 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:20.561 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45719 2024-09-21T15:01:20.561 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.561 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:20.561 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.561 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:20.561 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141952+0000", 2024-09-21T15:01:20.561 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:20.561 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.561 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.561 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:20.561 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141952+0000", 2024-09-21T15:01:20.562 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:20.562 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.562 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.562 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:20.562 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141952+0000", 2024-09-21T15:01:20.562 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:20.562 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.562 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.562 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:20.562 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141953+0000", 2024-09-21T15:01:20.562 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.486e-06 2024-09-21T15:01:20.562 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.562 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.562 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:20.562 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141959+0000", 2024-09-21T15:01:20.562 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4940000000000003e-06 2024-09-21T15:01:20.563 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.563 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.563 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:20.563 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141963+0000", 2024-09-21T15:01:20.563 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3540000000000002e-06 2024-09-21T15:01:20.563 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.563 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.563 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:20.563 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177084+0000", 2024-09-21T15:01:20.563 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035120983000000001 2024-09-21T15:01:20.563 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.563 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.563 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:20.563 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177091+0000", 2024-09-21T15:01:20.564 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0350000000000004e-06 2024-09-21T15:01:20.564 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:20.564 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:20.564 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:20.564 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.564 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.564 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45720 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:20.564 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141965+0000", 2024-09-21T15:01:20.564 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 130.36602828900001, 2024-09-21T15:01:20.564 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035155268000000003, 2024-09-21T15:01:20.564 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:20.564 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:20.564 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:20.564 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:20.565 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:20.565 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45720 2024-09-21T15:01:20.565 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.565 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:20.565 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.565 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:20.565 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141965+0000", 2024-09-21T15:01:20.565 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:20.565 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.565 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.565 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:20.565 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141965+0000", 2024-09-21T15:01:20.565 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:20.565 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.565 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.565 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:20.565 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141965+0000", 2024-09-21T15:01:20.566 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:20.566 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.567 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.567 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:20.567 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141966+0000", 2024-09-21T15:01:20.567 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.46e-06 2024-09-21T15:01:20.567 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.567 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.567 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:20.567 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141972+0000", 2024-09-21T15:01:20.567 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4210000000000003e-06 2024-09-21T15:01:20.567 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.567 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.567 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:20.567 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141975+0000", 2024-09-21T15:01:20.568 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.948e-06 2024-09-21T15:01:20.568 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.568 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.568 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:20.568 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177113+0000", 2024-09-21T15:01:20.568 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035137855000000003 2024-09-21T15:01:20.568 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.568 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.568 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:20.568 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177121+0000", 2024-09-21T15:01:20.568 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8269999999999998e-06 2024-09-21T15:01:20.568 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:20.568 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:20.568 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:20.568 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.569 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.569 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45721 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:20.569 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141977+0000", 2024-09-21T15:01:20.569 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 130.366016266, 2024-09-21T15:01:20.569 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035169984000000001, 2024-09-21T15:01:20.569 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:20.569 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:20.569 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:20.569 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:20.569 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:20.569 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45721 2024-09-21T15:01:20.569 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.569 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:20.569 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.570 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:20.570 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141977+0000", 2024-09-21T15:01:20.570 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:20.570 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.570 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.570 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:20.570 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141977+0000", 2024-09-21T15:01:20.570 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:20.570 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.570 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.570 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:20.570 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141977+0000", 2024-09-21T15:01:20.570 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:20.570 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.571 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.571 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:20.571 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141979+0000", 2024-09-21T15:01:20.571 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.511e-06 2024-09-21T15:01:20.571 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.571 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.571 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:20.571 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141982+0000", 2024-09-21T15:01:20.571 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4180000000000001e-06 2024-09-21T15:01:20.571 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.571 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.571 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:20.571 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141985+0000", 2024-09-21T15:01:20.571 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9450000000000002e-06 2024-09-21T15:01:20.571 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.572 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.572 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:20.572 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177139+0000", 2024-09-21T15:01:20.572 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035154409999999997 2024-09-21T15:01:20.572 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.572 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.572 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:20.572 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177147+0000", 2024-09-21T15:01:20.572 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9570000000000002e-06 2024-09-21T15:01:20.572 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:20.572 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:20.572 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:20.572 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.572 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.573 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45722 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:20.573 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141987+0000", 2024-09-21T15:01:20.573 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 130.36600629500001, 2024-09-21T15:01:20.573 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035188492000000002, 2024-09-21T15:01:20.573 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:20.573 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:20.573 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:20.573 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:20.573 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:20.573 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45722 2024-09-21T15:01:20.573 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.573 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:20.573 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.573 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:20.574 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141987+0000", 2024-09-21T15:01:20.574 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:20.574 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.574 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.574 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:20.574 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141987+0000", 2024-09-21T15:01:20.574 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:20.574 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.574 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.574 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:20.574 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141987+0000", 2024-09-21T15:01:20.574 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:20.574 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.574 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.575 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:20.575 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141988+0000", 2024-09-21T15:01:20.575 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.463e-06 2024-09-21T15:01:20.575 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.575 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.575 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:20.575 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141994+0000", 2024-09-21T15:01:20.575 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.361e-06 2024-09-21T15:01:20.575 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.575 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.575 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:20.575 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141997+0000", 2024-09-21T15:01:20.575 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0359999999999998e-06 2024-09-21T15:01:20.575 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.575 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.576 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:20.576 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177169+0000", 2024-09-21T15:01:20.576 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035171651999999998 2024-09-21T15:01:20.576 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.576 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.576 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:20.576 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177176+0000", 2024-09-21T15:01:20.576 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2189999999999998e-06 2024-09-21T15:01:20.576 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:20.576 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:20.576 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:20.576 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.576 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.576 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45723 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:20.576 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141999+0000", 2024-09-21T15:01:20.577 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 130.36599412800001, 2024-09-21T15:01:20.577 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035197701999999997, 2024-09-21T15:01:20.577 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:20.577 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:20.577 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:20.577 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:20.577 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:20.577 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45723 2024-09-21T15:01:20.577 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.577 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:20.577 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.577 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:20.577 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141999+0000", 2024-09-21T15:01:20.577 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:20.577 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.577 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.578 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:20.578 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141999+0000", 2024-09-21T15:01:20.578 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:20.578 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.578 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.578 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:20.578 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141999+0000", 2024-09-21T15:01:20.578 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:20.578 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.578 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.578 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:20.578 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142003+0000", 2024-09-21T15:01:20.578 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5049999999999999e-06 2024-09-21T15:01:20.579 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.579 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.579 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:20.579 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142008+0000", 2024-09-21T15:01:20.579 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6579999999999999e-06 2024-09-21T15:01:20.579 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.579 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.579 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:20.579 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142011+0000", 2024-09-21T15:01:20.579 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0529999999999998e-06 2024-09-21T15:01:20.579 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.579 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.579 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:20.579 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177194+0000", 2024-09-21T15:01:20.579 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035182221999999999 2024-09-21T15:01:20.579 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.580 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.580 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:20.580 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177197+0000", 2024-09-21T15:01:20.580 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5030000000000002e-06 2024-09-21T15:01:20.580 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:20.580 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:20.580 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:20.580 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.580 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.580 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45724 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:20.580 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.142015+0000", 2024-09-21T15:01:20.580 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 130.36597816099999, 2024-09-21T15:01:20.580 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035195345000000003, 2024-09-21T15:01:20.580 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:20.580 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:20.581 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:20.581 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:20.581 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:20.581 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45724 2024-09-21T15:01:20.581 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.581 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:20.581 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.581 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:20.581 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142015+0000", 2024-09-21T15:01:20.581 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:20.581 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.581 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.581 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:20.581 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142015+0000", 2024-09-21T15:01:20.581 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:20.582 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.582 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.582 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:20.582 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142015+0000", 2024-09-21T15:01:20.582 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:20.582 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.582 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.582 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:20.582 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142017+0000", 2024-09-21T15:01:20.582 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5540000000000001e-06 2024-09-21T15:01:20.582 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.582 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.582 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:20.582 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142022+0000", 2024-09-21T15:01:20.583 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4229999999999996e-06 2024-09-21T15:01:20.583 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.583 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.583 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:20.583 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142025+0000", 2024-09-21T15:01:20.583 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.056e-06 2024-09-21T15:01:20.583 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.583 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.583 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:20.583 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177207+0000", 2024-09-21T15:01:20.583 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035182089 2024-09-21T15:01:20.583 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.583 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.584 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:20.584 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177211+0000", 2024-09-21T15:01:20.584 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4649999999999999e-06 2024-09-21T15:01:20.584 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:20.584 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:20.584 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:20.584 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.584 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.584 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45725 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:20.584 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.142028+0000", 2024-09-21T15:01:20.584 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 130.36596588200001, 2024-09-21T15:01:20.584 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035195731000000001, 2024-09-21T15:01:20.584 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:20.585 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:20.585 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:20.585 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:20.585 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:20.585 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45725 2024-09-21T15:01:20.585 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.585 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:20.585 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.585 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:20.585 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142028+0000", 2024-09-21T15:01:20.585 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:20.585 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.585 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.585 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:20.586 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142028+0000", 2024-09-21T15:01:20.586 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:20.586 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.586 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.586 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:20.586 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142027+0000", 2024-09-21T15:01:20.586 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:20.586 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.586 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.586 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:20.586 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142029+0000", 2024-09-21T15:01:20.586 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5269999999999999e-06 2024-09-21T15:01:20.586 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.586 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.586 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:20.587 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142032+0000", 2024-09-21T15:01:20.587 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3249999999999999e-06 2024-09-21T15:01:20.587 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.587 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.587 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:20.587 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142035+0000", 2024-09-21T15:01:20.587 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.897e-06 2024-09-21T15:01:20.587 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.587 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.587 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:20.587 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177220+0000", 2024-09-21T15:01:20.587 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035184871 2024-09-21T15:01:20.587 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.587 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.588 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:20.588 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177223+0000", 2024-09-21T15:01:20.588 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3680000000000002e-06 2024-09-21T15:01:20.588 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:20.588 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:20.588 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:20.588 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.588 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.588 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45726 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:20.588 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.142037+0000", 2024-09-21T15:01:20.588 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 130.36595616700001, 2024-09-21T15:01:20.588 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035199273000000003, 2024-09-21T15:01:20.588 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:20.588 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:20.588 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:20.589 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:20.589 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:20.589 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45726 2024-09-21T15:01:20.589 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.589 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:20.589 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.589 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:20.589 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142037+0000", 2024-09-21T15:01:20.589 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:20.589 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.589 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.589 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:20.589 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142037+0000", 2024-09-21T15:01:20.589 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:20.589 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.590 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.590 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:20.590 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142037+0000", 2024-09-21T15:01:20.590 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:20.590 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.590 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.590 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:20.590 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142038+0000", 2024-09-21T15:01:20.590 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.344e-06 2024-09-21T15:01:20.590 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.590 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.590 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:20.590 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142044+0000", 2024-09-21T15:01:20.591 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1930000000000001e-06 2024-09-21T15:01:20.591 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.591 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.591 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:20.591 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142047+0000", 2024-09-21T15:01:20.591 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0280000000000001e-06 2024-09-21T15:01:20.591 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.591 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.591 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:20.591 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177233+0000", 2024-09-21T15:01:20.591 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035186394000000003 2024-09-21T15:01:20.591 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:20.591 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:20.591 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:20.592 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177237+0000", 2024-09-21T15:01:20.592 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5530000000000001e-06 2024-09-21T15:01:20.592 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:20.592 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:20.592 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:20.592 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:20.592 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:20.592 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:20.592 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T15:01:20.609 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:01:20.606+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:01:20.695 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:01:20.690+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:01:20.712 INFO:tasks.ceph.mon.a.smithi032.stderr:2024-09-21T15:01:20.706+0000 7fdb7159c700 -1 mon.a@0(leader).osd e188 definitely_dead 0 2024-09-21T15:01:20.719 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:20.720 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:01:20.720 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T15:01:20.720 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:20.730 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T15:01:20.796 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:01:20.790+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:01:20.833 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:20.833 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:01:20.833 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T15:01:20.833 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T15:01:20.833 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:20.843 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T15:01:20.897 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:01:20.890+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:01:20.998 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:01:20.994+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:01:20.999 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:20.999 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T15:01:20.999 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T15:01:21.000 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T15:01:21.000 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.000 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139776 3.16 3:6c38d0e2:::rbd_data.113664780c85.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:21.000 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.223442+0000", 2024-09-21T15:01:21.000 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.773733918000001, 2024-09-21T15:01:21.000 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0315488150000001, 2024-09-21T15:01:21.000 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:21.000 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:21.000 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:21.000 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:21.000 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:21.000 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139776 2024-09-21T15:01:21.000 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.000 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:21.001 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.001 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:21.001 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223442+0000", 2024-09-21T15:01:21.001 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:21.001 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.001 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.001 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:21.001 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223442+0000", 2024-09-21T15:01:21.001 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:21.001 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.001 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.001 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:21.001 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223442+0000", 2024-09-21T15:01:21.002 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:21.002 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.002 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.002 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:21.002 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223446+0000", 2024-09-21T15:01:21.002 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2549999999999999e-06 2024-09-21T15:01:21.002 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.002 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.002 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:21.002 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223454+0000", 2024-09-21T15:01:21.002 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8199999999999997e-06 2024-09-21T15:01:21.002 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.002 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.002 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:21.003 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223460+0000", 2024-09-21T15:01:21.003 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6330000000000001e-06 2024-09-21T15:01:21.003 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.003 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.003 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:21.003 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223542+0000", 2024-09-21T15:01:21.003 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1929999999999997e-05 2024-09-21T15:01:21.003 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.003 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.003 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:21.003 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223552+0000", 2024-09-21T15:01:21.003 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0627e-05 2024-09-21T15:01:21.003 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.004 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.004 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:21.004 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.254634+0000", 2024-09-21T15:01:21.004 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.031081972 2024-09-21T15:01:21.004 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.004 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.004 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:21.004 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.254796+0000", 2024-09-21T15:01:21.004 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000161879 2024-09-21T15:01:21.004 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.004 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.004 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:21.004 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.254991+0000", 2024-09-21T15:01:21.004 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00019526499999999999 2024-09-21T15:01:21.004 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:21.005 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:21.005 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:21.005 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.005 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.005 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139785 3.11 3:8abbb109:::rbd_data.113664780c85.0000000000000070:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:21.005 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.223531+0000", 2024-09-21T15:01:21.005 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.773645176999999, 2024-09-21T15:01:21.005 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0187143510000001, 2024-09-21T15:01:21.005 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:21.005 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:21.005 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:21.005 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:21.005 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:21.005 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139785 2024-09-21T15:01:21.006 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.006 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:21.006 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.006 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:21.006 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223531+0000", 2024-09-21T15:01:21.006 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:21.006 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.006 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.006 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:21.006 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223531+0000", 2024-09-21T15:01:21.006 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:21.006 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.006 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.007 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:21.007 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223531+0000", 2024-09-21T15:01:21.007 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:21.007 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.007 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.007 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:21.007 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223534+0000", 2024-09-21T15:01:21.007 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2650000000000001e-06 2024-09-21T15:01:21.007 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.007 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.007 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:21.007 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223541+0000", 2024-09-21T15:01:21.007 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8990000000000004e-06 2024-09-21T15:01:21.007 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.007 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.008 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:21.008 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223546+0000", 2024-09-21T15:01:21.008 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3399999999999997e-06 2024-09-21T15:01:21.008 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.008 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.008 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:21.008 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223724+0000", 2024-09-21T15:01:21.008 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017745799999999999 2024-09-21T15:01:21.008 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.008 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.008 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:21.008 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223735+0000", 2024-09-21T15:01:21.008 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1610999999999999e-05 2024-09-21T15:01:21.008 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.009 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.009 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:21.009 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242113+0000", 2024-09-21T15:01:21.009 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.018377879 2024-09-21T15:01:21.009 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.009 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.009 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:21.009 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242246+0000", 2024-09-21T15:01:21.009 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013231500000000001 2024-09-21T15:01:21.009 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:21.009 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:21.009 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:21.009 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.009 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.010 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139790 3.11 3:8a18c7d8:::rbd_data.113664780c85.000000000000008d:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:21.010 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.223578+0000", 2024-09-21T15:01:21.010 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.773598182000001, 2024-09-21T15:01:21.010 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0187879719999999, 2024-09-21T15:01:21.010 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:21.010 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:21.010 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:21.010 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:21.010 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:21.010 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139790 2024-09-21T15:01:21.010 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.010 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:21.010 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.010 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:21.011 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223578+0000", 2024-09-21T15:01:21.011 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:21.011 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.011 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.011 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:21.011 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223578+0000", 2024-09-21T15:01:21.011 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:21.011 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.011 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.011 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:21.011 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223578+0000", 2024-09-21T15:01:21.011 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:21.011 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.012 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.012 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:21.012 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223581+0000", 2024-09-21T15:01:21.012 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6879999999999999e-06 2024-09-21T15:01:21.012 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.012 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.012 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:21.012 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223586+0000", 2024-09-21T15:01:21.012 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7080000000000002e-06 2024-09-21T15:01:21.012 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.012 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.012 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:21.012 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223591+0000", 2024-09-21T15:01:21.012 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5639999999999998e-06 2024-09-21T15:01:21.013 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.013 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.013 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:21.013 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223761+0000", 2024-09-21T15:01:21.013 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000170213 2024-09-21T15:01:21.013 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.013 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.013 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:21.013 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223769+0000", 2024-09-21T15:01:21.013 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7049999999999999e-06 2024-09-21T15:01:21.013 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.013 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.013 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:21.013 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242269+0000", 2024-09-21T15:01:21.013 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.018500288 2024-09-21T15:01:21.014 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.014 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.014 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:21.014 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242366+0000", 2024-09-21T15:01:21.014 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.7126000000000005e-05 2024-09-21T15:01:21.014 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:21.014 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:21.014 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:21.014 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.014 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.014 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139792 3.16 3:6ce5caf3:::rbd_data.113664780c85.000000000000009b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:21.014 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.223595+0000", 2024-09-21T15:01:21.014 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.773581229000001, 2024-09-21T15:01:21.015 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0317517899999999, 2024-09-21T15:01:21.015 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:21.015 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:21.015 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:21.015 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:21.015 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:21.015 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139792 2024-09-21T15:01:21.015 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.015 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:21.015 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.015 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:21.015 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223595+0000", 2024-09-21T15:01:21.015 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:21.015 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.015 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.016 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:21.016 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223595+0000", 2024-09-21T15:01:21.016 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:21.016 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.016 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.016 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:21.016 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223595+0000", 2024-09-21T15:01:21.016 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:21.016 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.016 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.016 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:21.016 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223598+0000", 2024-09-21T15:01:21.016 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4620000000000001e-06 2024-09-21T15:01:21.016 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.017 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.017 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:21.017 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223604+0000", 2024-09-21T15:01:21.017 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6879999999999996e-06 2024-09-21T15:01:21.017 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.017 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.017 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:21.017 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223609+0000", 2024-09-21T15:01:21.017 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9490000000000004e-06 2024-09-21T15:01:21.017 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.017 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.017 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:21.017 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224067+0000", 2024-09-21T15:01:21.017 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00045780700000000002 2024-09-21T15:01:21.017 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.018 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.018 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:21.018 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224075+0000", 2024-09-21T15:01:21.018 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6610000000000006e-06 2024-09-21T15:01:21.018 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.018 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.018 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:21.018 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255141+0000", 2024-09-21T15:01:21.018 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0310657480000001 2024-09-21T15:01:21.018 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.018 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.018 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:21.018 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255347+0000", 2024-09-21T15:01:21.018 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000205837 2024-09-21T15:01:21.019 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:21.019 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:21.019 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:21.019 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.019 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.019 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139794 3.11 3:8cf3a2f4:::rbd_data.113664780c85.00000000000000a9:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:21.019 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.224011+0000", 2024-09-21T15:01:21.019 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.773165201000001, 2024-09-21T15:01:21.019 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.018507791, 2024-09-21T15:01:21.019 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:21.019 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:21.019 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:21.019 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:21.020 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:21.020 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139794 2024-09-21T15:01:21.020 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.020 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:21.020 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.020 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:21.020 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224011+0000", 2024-09-21T15:01:21.020 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:21.020 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.020 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.020 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:21.020 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224011+0000", 2024-09-21T15:01:21.020 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:21.020 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.020 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.021 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:21.021 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224010+0000", 2024-09-21T15:01:21.021 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:21.021 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.021 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.021 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:21.021 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224016+0000", 2024-09-21T15:01:21.021 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6550000000000001e-06 2024-09-21T15:01:21.021 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.021 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.021 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:21.021 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224028+0000", 2024-09-21T15:01:21.021 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2422999999999999e-05 2024-09-21T15:01:21.021 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.022 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.022 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:21.022 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224036+0000", 2024-09-21T15:01:21.022 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7749999999999993e-06 2024-09-21T15:01:21.022 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.022 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.022 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:21.022 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225176+0000", 2024-09-21T15:01:21.022 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0011401740000000001 2024-09-21T15:01:21.022 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.022 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.022 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:21.022 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225187+0000", 2024-09-21T15:01:21.023 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0951999999999999e-05 2024-09-21T15:01:21.023 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.023 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.023 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:21.023 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242392+0000", 2024-09-21T15:01:21.023 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.017204905 2024-09-21T15:01:21.023 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.023 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.023 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:21.023 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242519+0000", 2024-09-21T15:01:21.023 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012680499999999999 2024-09-21T15:01:21.023 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:21.023 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:21.023 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:21.024 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.024 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.024 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139800 3.16 3:68b050ed:::rbd_data.113664780c85.00000000000000d3:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:21.024 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.224579+0000", 2024-09-21T15:01:21.024 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.772597441999999, 2024-09-21T15:01:21.024 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.030932637, 2024-09-21T15:01:21.024 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:21.024 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:21.024 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:21.024 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:21.024 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:21.024 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139800 2024-09-21T15:01:21.024 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.024 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:21.025 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.025 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:21.025 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224579+0000", 2024-09-21T15:01:21.025 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:21.025 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.025 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.025 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:21.025 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224579+0000", 2024-09-21T15:01:21.025 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:21.025 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.025 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.025 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:21.026 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224578+0000", 2024-09-21T15:01:21.026 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:21.026 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.026 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.026 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:21.026 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224582+0000", 2024-09-21T15:01:21.026 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0029999999999999e-06 2024-09-21T15:01:21.026 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.026 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.026 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:21.026 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224588+0000", 2024-09-21T15:01:21.026 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8120000000000002e-06 2024-09-21T15:01:21.027 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.027 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.027 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:21.027 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224593+0000", 2024-09-21T15:01:21.027 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7469999999999996e-06 2024-09-21T15:01:21.027 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.027 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.027 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:21.027 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224648+0000", 2024-09-21T15:01:21.027 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4693e-05 2024-09-21T15:01:21.027 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.027 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.027 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:21.027 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224656+0000", 2024-09-21T15:01:21.028 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8429999999999993e-06 2024-09-21T15:01:21.028 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.028 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.028 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:21.028 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255382+0000", 2024-09-21T15:01:21.028 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0307266070000001 2024-09-21T15:01:21.028 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.028 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.028 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:21.028 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255512+0000", 2024-09-21T15:01:21.028 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000129367 2024-09-21T15:01:21.028 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:21.028 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:21.028 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:21.029 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.029 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.029 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139804 3.11 3:88f9ddce:::rbd_data.113664780c85.00000000000000ef:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:21.029 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.224632+0000", 2024-09-21T15:01:21.029 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.772544460999999, 2024-09-21T15:01:21.029 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.018021302, 2024-09-21T15:01:21.029 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:21.029 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:21.029 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:21.029 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:21.029 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:21.029 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139804 2024-09-21T15:01:21.029 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.029 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:21.029 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.030 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:21.030 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224632+0000", 2024-09-21T15:01:21.030 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:21.030 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.030 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.030 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:21.030 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224632+0000", 2024-09-21T15:01:21.030 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:21.030 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.030 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.030 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:21.030 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224632+0000", 2024-09-21T15:01:21.030 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:21.030 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.031 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.031 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:21.031 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224635+0000", 2024-09-21T15:01:21.031 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2710000000000001e-06 2024-09-21T15:01:21.031 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.031 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.031 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:21.031 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224640+0000", 2024-09-21T15:01:21.031 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6779999999999997e-06 2024-09-21T15:01:21.031 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.031 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.031 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:21.031 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224645+0000", 2024-09-21T15:01:21.031 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9339999999999997e-06 2024-09-21T15:01:21.031 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.032 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.032 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:21.032 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225307+0000", 2024-09-21T15:01:21.032 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00066201699999999999 2024-09-21T15:01:21.032 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.032 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.032 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:21.032 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225312+0000", 2024-09-21T15:01:21.032 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9799999999999998e-06 2024-09-21T15:01:21.032 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.032 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.032 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:21.032 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242548+0000", 2024-09-21T15:01:21.033 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.017235603 2024-09-21T15:01:21.033 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.033 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.033 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:21.033 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242653+0000", 2024-09-21T15:01:21.033 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010522 2024-09-21T15:01:21.033 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:21.033 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:21.033 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:21.033 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.033 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.033 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139809 3.11 3:884da946:::rbd_data.113664780c85.0000000000000118:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:21.033 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.224667+0000", 2024-09-21T15:01:21.033 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.772509416999998, 2024-09-21T15:01:21.034 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0181101960000001, 2024-09-21T15:01:21.034 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:21.034 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:21.034 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:21.034 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:21.034 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:21.034 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139809 2024-09-21T15:01:21.034 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.034 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:21.034 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.034 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:21.034 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224667+0000", 2024-09-21T15:01:21.034 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:21.034 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.035 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.035 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:21.035 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224667+0000", 2024-09-21T15:01:21.035 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:21.035 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.035 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.035 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:21.035 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224667+0000", 2024-09-21T15:01:21.035 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:21.035 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.035 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.035 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:21.035 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224669+0000", 2024-09-21T15:01:21.035 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.452e-06 2024-09-21T15:01:21.035 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.036 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.036 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:21.036 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224675+0000", 2024-09-21T15:01:21.036 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.818e-06 2024-09-21T15:01:21.036 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.036 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.036 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:21.036 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224680+0000", 2024-09-21T15:01:21.036 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6319999999999999e-06 2024-09-21T15:01:21.036 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.036 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.036 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:21.036 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225400+0000", 2024-09-21T15:01:21.036 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00072012699999999999 2024-09-21T15:01:21.037 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.037 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.037 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:21.037 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225409+0000", 2024-09-21T15:01:21.037 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1039999999999996e-06 2024-09-21T15:01:21.037 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.037 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.037 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:21.037 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242675+0000", 2024-09-21T15:01:21.037 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0172663319999999 2024-09-21T15:01:21.037 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.037 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.037 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:21.037 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242777+0000", 2024-09-21T15:01:21.037 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010197899999999999 2024-09-21T15:01:21.038 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:21.038 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:21.038 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:21.038 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.038 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.038 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139868 3.16 3:69f82090:::rbd_data.113664780c85.00000000000000f1:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:21.038 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.225982+0000", 2024-09-21T15:01:21.038 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.771193917000002, 2024-09-21T15:01:21.038 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0296791729999999, 2024-09-21T15:01:21.038 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:21.038 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:21.038 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:21.038 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:21.038 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:21.039 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139868 2024-09-21T15:01:21.039 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.039 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:21.039 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.039 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:21.039 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225982+0000", 2024-09-21T15:01:21.039 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:21.039 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.039 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.039 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:21.039 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225982+0000", 2024-09-21T15:01:21.039 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:21.039 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.040 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.040 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:21.040 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225982+0000", 2024-09-21T15:01:21.040 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:21.040 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.040 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.040 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:21.040 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225985+0000", 2024-09-21T15:01:21.040 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.74e-06 2024-09-21T15:01:21.040 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.040 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.041 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:21.041 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225990+0000", 2024-09-21T15:01:21.041 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7039999999999999e-06 2024-09-21T15:01:21.041 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.041 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.041 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:21.041 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225995+0000", 2024-09-21T15:01:21.041 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7249999999999997e-06 2024-09-21T15:01:21.041 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.041 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.041 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:21.041 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226299+0000", 2024-09-21T15:01:21.041 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00030383199999999997 2024-09-21T15:01:21.041 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.042 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.042 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:21.042 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226308+0000", 2024-09-21T15:01:21.042 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.9740000000000008e-06 2024-09-21T15:01:21.042 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.042 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.042 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:21.042 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255539+0000", 2024-09-21T15:01:21.042 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0292305960000001 2024-09-21T15:01:21.042 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.042 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.042 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:21.042 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255662+0000", 2024-09-21T15:01:21.042 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012307400000000001 2024-09-21T15:01:21.043 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:21.043 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:21.043 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:21.043 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.043 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.043 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139869 3.16 3:6b6c90a4:::rbd_data.113664780c85.00000000000000fe:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:21.043 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.226006+0000", 2024-09-21T15:01:21.043 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.771170301000002, 2024-09-21T15:01:21.043 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.029790518, 2024-09-21T15:01:21.043 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:21.043 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:21.043 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:21.043 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:21.043 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:21.044 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139869 2024-09-21T15:01:21.044 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.044 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:21.044 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.044 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:21.044 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226006+0000", 2024-09-21T15:01:21.044 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:21.044 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.044 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.044 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:21.044 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226006+0000", 2024-09-21T15:01:21.044 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:21.044 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.044 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.045 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:21.045 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226006+0000", 2024-09-21T15:01:21.045 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:21.045 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.045 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.045 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:21.045 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226008+0000", 2024-09-21T15:01:21.045 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5569999999999998e-06 2024-09-21T15:01:21.045 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.045 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.045 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:21.045 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226014+0000", 2024-09-21T15:01:21.045 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2079999999999997e-06 2024-09-21T15:01:21.045 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.046 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.046 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:21.046 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226020+0000", 2024-09-21T15:01:21.046 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7429999999999999e-06 2024-09-21T15:01:21.046 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.046 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.046 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:21.046 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226330+0000", 2024-09-21T15:01:21.046 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00030963300000000002 2024-09-21T15:01:21.046 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.046 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.046 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:21.046 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226336+0000", 2024-09-21T15:01:21.046 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6340000000000003e-06 2024-09-21T15:01:21.047 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.047 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.047 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:21.047 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255687+0000", 2024-09-21T15:01:21.047 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0293507150000001 2024-09-21T15:01:21.047 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.047 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.047 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:21.047 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255797+0000", 2024-09-21T15:01:21.047 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000109471 2024-09-21T15:01:21.047 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:21.047 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:21.047 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:21.047 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.048 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.048 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139884 3.16 3:69f10324:::rbd_data.113664780c85.0000000000000161:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:21.048 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.227573+0000", 2024-09-21T15:01:21.048 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.769603660000001, 2024-09-21T15:01:21.048 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0283568759999999, 2024-09-21T15:01:21.048 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:21.048 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:21.048 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:21.048 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:21.048 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:21.048 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139884 2024-09-21T15:01:21.048 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.048 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:21.048 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.049 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:21.049 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227573+0000", 2024-09-21T15:01:21.049 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:21.049 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.049 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.049 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:21.049 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227573+0000", 2024-09-21T15:01:21.049 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:21.049 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.049 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.049 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:21.049 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227572+0000", 2024-09-21T15:01:21.049 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:21.049 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.049 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.049 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:21.050 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227575+0000", 2024-09-21T15:01:21.050 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7769999999999998e-06 2024-09-21T15:01:21.050 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.050 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.050 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:21.050 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227581+0000", 2024-09-21T15:01:21.050 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7359999999999998e-06 2024-09-21T15:01:21.050 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.050 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.050 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:21.050 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227585+0000", 2024-09-21T15:01:21.050 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5090000000000004e-06 2024-09-21T15:01:21.050 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.050 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.051 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:21.051 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227753+0000", 2024-09-21T15:01:21.051 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00016761599999999999 2024-09-21T15:01:21.051 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.051 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.051 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:21.051 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227761+0000", 2024-09-21T15:01:21.051 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0579999999999998e-06 2024-09-21T15:01:21.051 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.051 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.051 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:21.051 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255828+0000", 2024-09-21T15:01:21.051 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0280671349999999 2024-09-21T15:01:21.051 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.052 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.052 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:21.052 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255930+0000", 2024-09-21T15:01:21.052 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000101372 2024-09-21T15:01:21.052 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:21.052 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:21.052 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:21.052 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.052 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.052 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139913 3.16 3:6f81b46d:::rbd_data.113664780c85.0000000000000074:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:21.052 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.227685+0000", 2024-09-21T15:01:21.052 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.769491480999999, 2024-09-21T15:01:21.052 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0283644750000001, 2024-09-21T15:01:21.052 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:21.052 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:21.053 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:21.053 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:21.053 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:21.053 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139913 2024-09-21T15:01:21.053 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.053 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:21.053 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.053 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:21.053 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227685+0000", 2024-09-21T15:01:21.053 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:21.053 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.053 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.053 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:21.054 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227685+0000", 2024-09-21T15:01:21.054 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:21.054 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.054 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.054 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:21.054 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227685+0000", 2024-09-21T15:01:21.054 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:21.054 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.054 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.054 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:21.054 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227687+0000", 2024-09-21T15:01:21.054 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2929999999999999e-06 2024-09-21T15:01:21.055 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.055 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.055 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:21.055 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227693+0000", 2024-09-21T15:01:21.055 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1070000000000001e-06 2024-09-21T15:01:21.055 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.055 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.055 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:21.055 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227698+0000", 2024-09-21T15:01:21.055 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9339999999999997e-06 2024-09-21T15:01:21.055 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.055 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.055 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:21.056 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227784+0000", 2024-09-21T15:01:21.056 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6096999999999999e-05 2024-09-21T15:01:21.056 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.056 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.056 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:21.056 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227791+0000", 2024-09-21T15:01:21.056 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9090000000000003e-06 2024-09-21T15:01:21.056 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.056 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.056 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:21.056 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255955+0000", 2024-09-21T15:01:21.056 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0281643579999999 2024-09-21T15:01:21.056 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.056 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.056 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:21.057 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256049+0000", 2024-09-21T15:01:21.057 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4047000000000003e-05 2024-09-21T15:01:21.057 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:21.057 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:21.057 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:21.057 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.057 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.057 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139988 3.11 3:88a29fa4:::rbd_data.113664780c85.00000000000000bd:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:21.057 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.229772+0000", 2024-09-21T15:01:21.057 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.767403982000001, 2024-09-21T15:01:21.057 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.017967668, 2024-09-21T15:01:21.057 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:21.057 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:21.057 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:21.058 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:21.058 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:21.058 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139988 2024-09-21T15:01:21.058 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.058 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:21.058 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.058 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:21.058 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229772+0000", 2024-09-21T15:01:21.058 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:21.058 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.058 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.058 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:21.058 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229772+0000", 2024-09-21T15:01:21.058 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:21.058 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.059 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.059 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:21.059 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229772+0000", 2024-09-21T15:01:21.059 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:21.059 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.059 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.059 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:21.059 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229776+0000", 2024-09-21T15:01:21.059 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6890000000000002e-06 2024-09-21T15:01:21.059 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.059 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.059 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:21.059 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229783+0000", 2024-09-21T15:01:21.059 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2280000000000001e-06 2024-09-21T15:01:21.060 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.060 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.060 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:21.060 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229789+0000", 2024-09-21T15:01:21.060 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1700000000000002e-06 2024-09-21T15:01:21.060 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.060 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.060 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:21.060 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.230106+0000", 2024-09-21T15:01:21.060 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00031700800000000003 2024-09-21T15:01:21.060 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.060 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.060 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:21.060 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.230115+0000", 2024-09-21T15:01:21.060 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4190000000000008e-06 2024-09-21T15:01:21.061 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.061 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.061 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:21.061 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242935+0000", 2024-09-21T15:01:21.061 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0128191449999999 2024-09-21T15:01:21.061 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.061 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.061 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for missing object", 2024-09-21T15:01:21.061 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.243078+0000", 2024-09-21T15:01:21.061 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000142984 2024-09-21T15:01:21.061 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.061 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.061 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:21.061 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.247659+0000", 2024-09-21T15:01:21.062 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0045813030000000001 2024-09-21T15:01:21.062 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.062 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.062 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:21.062 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.247740+0000", 2024-09-21T15:01:21.062 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1216000000000003e-05 2024-09-21T15:01:21.062 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:21.062 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:21.062 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:21.062 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.062 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.062 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139994 3.16 3:6b9df8d1:::rbd_data.113664780c85.00000000000000e7:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:21.062 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.231370+0000", 2024-09-21T15:01:21.062 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.765805904, 2024-09-21T15:01:21.063 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.024795576, 2024-09-21T15:01:21.063 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:21.063 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:21.063 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:21.063 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:21.063 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:21.063 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139994 2024-09-21T15:01:21.063 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.063 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:21.063 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.063 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:21.063 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231370+0000", 2024-09-21T15:01:21.063 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:21.063 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.063 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.064 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:21.064 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231370+0000", 2024-09-21T15:01:21.064 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:21.064 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.064 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.064 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:21.064 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231369+0000", 2024-09-21T15:01:21.064 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:21.064 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.064 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.064 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:21.064 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231376+0000", 2024-09-21T15:01:21.064 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5509999999999996e-06 2024-09-21T15:01:21.064 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.064 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.065 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:21.065 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231387+0000", 2024-09-21T15:01:21.065 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.061e-05 2024-09-21T15:01:21.065 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.065 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.065 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:21.065 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231394+0000", 2024-09-21T15:01:21.065 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3799999999999996e-06 2024-09-21T15:01:21.065 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.065 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.065 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:21.065 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231459+0000", 2024-09-21T15:01:21.065 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4716999999999997e-05 2024-09-21T15:01:21.065 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.066 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.066 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:21.066 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231468+0000", 2024-09-21T15:01:21.066 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1479999999999995e-06 2024-09-21T15:01:21.066 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.066 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.066 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:21.066 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256074+0000", 2024-09-21T15:01:21.066 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0246064180000001 2024-09-21T15:01:21.066 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.066 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.066 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:21.066 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256166+0000", 2024-09-21T15:01:21.066 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1862999999999998e-05 2024-09-21T15:01:21.066 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:21.067 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:21.067 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:21.067 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.067 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.067 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140012 3.16 3:6bd6dfb6:::rbd_data.113664780c85.0000000000000165:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:21.067 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.231890+0000", 2024-09-21T15:01:21.067 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.765286145000001, 2024-09-21T15:01:21.067 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.024408881, 2024-09-21T15:01:21.067 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:21.067 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:21.067 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:21.067 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:21.067 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:21.067 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140012 2024-09-21T15:01:21.068 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.068 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:21.068 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.068 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:21.068 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231890+0000", 2024-09-21T15:01:21.068 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:21.068 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.068 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.068 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:21.068 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231890+0000", 2024-09-21T15:01:21.068 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:21.068 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.068 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.068 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:21.069 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231889+0000", 2024-09-21T15:01:21.069 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:21.069 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.069 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.069 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:21.069 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231895+0000", 2024-09-21T15:01:21.069 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9569999999999999e-06 2024-09-21T15:01:21.069 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.069 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.069 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:21.069 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231910+0000", 2024-09-21T15:01:21.069 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4549000000000001e-05 2024-09-21T15:01:21.069 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.069 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.070 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:21.070 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231917+0000", 2024-09-21T15:01:21.070 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5730000000000002e-06 2024-09-21T15:01:21.070 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.070 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.070 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:21.070 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231988+0000", 2024-09-21T15:01:21.070 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0852999999999994e-05 2024-09-21T15:01:21.070 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.070 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.070 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:21.070 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.232000+0000", 2024-09-21T15:01:21.070 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2159e-05 2024-09-21T15:01:21.070 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.070 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.071 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:21.071 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256192+0000", 2024-09-21T15:01:21.071 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.024192073 2024-09-21T15:01:21.071 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.071 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.071 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:21.071 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256299+0000", 2024-09-21T15:01:21.071 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000106733 2024-09-21T15:01:21.071 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:21.071 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:21.071 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:21.071 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.071 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.071 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140055 3.16 3:69bb9803:::rbd_data.113664780c85.00000000000000da:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:21.072 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.233922+0000", 2024-09-21T15:01:21.072 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.763254616000001, 2024-09-21T15:01:21.072 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.022511768, 2024-09-21T15:01:21.072 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:21.072 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:21.072 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:21.072 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:21.072 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:21.072 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140055 2024-09-21T15:01:21.072 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.072 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:21.072 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.072 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:21.072 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233922+0000", 2024-09-21T15:01:21.072 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:21.073 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.073 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.073 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:21.073 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233922+0000", 2024-09-21T15:01:21.073 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:21.073 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.073 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.073 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:21.073 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233921+0000", 2024-09-21T15:01:21.073 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:21.073 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.073 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.073 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:21.073 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233927+0000", 2024-09-21T15:01:21.074 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0360000000000003e-06 2024-09-21T15:01:21.074 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.074 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.074 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:21.074 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233937+0000", 2024-09-21T15:01:21.074 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6439999999999995e-06 2024-09-21T15:01:21.074 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.074 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.074 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:21.074 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233944+0000", 2024-09-21T15:01:21.074 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7710000000000006e-06 2024-09-21T15:01:21.074 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.074 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.074 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:21.075 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233996+0000", 2024-09-21T15:01:21.075 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1597000000000001e-05 2024-09-21T15:01:21.075 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.075 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.075 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:21.075 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.234005+0000", 2024-09-21T15:01:21.075 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5960000000000004e-06 2024-09-21T15:01:21.075 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.075 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.075 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:21.075 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256332+0000", 2024-09-21T15:01:21.076 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0223271949999999 2024-09-21T15:01:21.076 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.076 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.076 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:21.076 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256434+0000", 2024-09-21T15:01:21.076 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010180599999999999 2024-09-21T15:01:21.076 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:21.076 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:21.076 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:21.076 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.076 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.076 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140143 3.16 3:6d3ed596:::rbd_data.113664780c85.0000000000000184:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:21.076 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.239496+0000", 2024-09-21T15:01:21.076 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.757680087000001, 2024-09-21T15:01:21.076 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.017076576, 2024-09-21T15:01:21.077 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:21.077 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:21.077 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:21.077 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:21.077 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:21.077 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140143 2024-09-21T15:01:21.077 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.077 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:21.077 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.077 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:21.077 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239496+0000", 2024-09-21T15:01:21.077 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:21.077 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.077 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.077 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:21.078 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239496+0000", 2024-09-21T15:01:21.078 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:21.078 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.078 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.078 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:21.078 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239496+0000", 2024-09-21T15:01:21.078 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:21.078 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.078 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.078 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:21.078 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239498+0000", 2024-09-21T15:01:21.078 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.509e-06 2024-09-21T15:01:21.078 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.078 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.079 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:21.079 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239504+0000", 2024-09-21T15:01:21.079 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6869999999999999e-06 2024-09-21T15:01:21.079 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.079 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.079 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:21.079 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239509+0000", 2024-09-21T15:01:21.079 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8860000000000003e-06 2024-09-21T15:01:21.079 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.079 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.079 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:21.079 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239572+0000", 2024-09-21T15:01:21.079 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2908000000000001e-05 2024-09-21T15:01:21.079 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.079 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.080 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:21.080 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239580+0000", 2024-09-21T15:01:21.080 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4270000000000008e-06 2024-09-21T15:01:21.080 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.080 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.080 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:21.080 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256457+0000", 2024-09-21T15:01:21.080 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0168770979999999 2024-09-21T15:01:21.080 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.080 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.080 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:21.080 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256573+0000", 2024-09-21T15:01:21.080 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00011538400000000001 2024-09-21T15:01:21.080 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:21.081 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:21.081 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:21.081 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.081 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.081 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140163 3.16 3:694e82eb:::rbd_data.113664780c85.0000000000000052:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:21.081 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.240334+0000", 2024-09-21T15:01:21.081 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.756842375000002, 2024-09-21T15:01:21.081 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.016365537, 2024-09-21T15:01:21.081 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:21.081 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:21.081 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:21.081 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:21.081 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:21.081 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140163 2024-09-21T15:01:21.081 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.082 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:21.082 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.082 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:21.082 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240334+0000", 2024-09-21T15:01:21.082 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:21.082 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.082 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.082 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:21.082 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240334+0000", 2024-09-21T15:01:21.082 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:21.082 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.082 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.082 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:21.082 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240334+0000", 2024-09-21T15:01:21.083 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:21.083 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.083 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.083 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:21.083 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240337+0000", 2024-09-21T15:01:21.083 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.089e-06 2024-09-21T15:01:21.083 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.083 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.083 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:21.083 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240343+0000", 2024-09-21T15:01:21.083 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4420000000000003e-06 2024-09-21T15:01:21.083 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.083 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.083 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:21.084 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240348+0000", 2024-09-21T15:01:21.084 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1039999999999998e-06 2024-09-21T15:01:21.084 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.084 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.084 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:21.084 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240451+0000", 2024-09-21T15:01:21.084 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000102996 2024-09-21T15:01:21.084 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.084 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.084 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:21.084 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240460+0000", 2024-09-21T15:01:21.084 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5369999999999997e-06 2024-09-21T15:01:21.084 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.084 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.085 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:21.085 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256600+0000", 2024-09-21T15:01:21.085 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.016139884 2024-09-21T15:01:21.085 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.085 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.085 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:21.085 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256700+0000", 2024-09-21T15:01:21.085 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9844000000000002e-05 2024-09-21T15:01:21.085 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:21.085 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:21.085 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:21.085 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.085 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.085 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140164 3.16 3:68bfdc57:::rbd_data.113664780c85.0000000000000053:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:21.085 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.240358+0000", 2024-09-21T15:01:21.086 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.756818680999999, 2024-09-21T15:01:21.086 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.01646094, 2024-09-21T15:01:21.086 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:21.086 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:21.086 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:21.086 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:21.086 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:21.086 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140164 2024-09-21T15:01:21.086 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.086 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:21.086 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.086 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:21.086 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240358+0000", 2024-09-21T15:01:21.087 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:21.087 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.087 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.087 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:21.087 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240358+0000", 2024-09-21T15:01:21.087 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:21.087 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.087 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.087 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:21.087 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240357+0000", 2024-09-21T15:01:21.087 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:21.087 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.087 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.087 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:21.087 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240360+0000", 2024-09-21T15:01:21.088 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7089999999999998e-06 2024-09-21T15:01:21.088 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.088 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.088 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:21.088 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240368+0000", 2024-09-21T15:01:21.088 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7270000000000007e-06 2024-09-21T15:01:21.088 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.088 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.088 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:21.088 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240373+0000", 2024-09-21T15:01:21.088 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4060000000000004e-06 2024-09-21T15:01:21.088 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.088 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.088 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:21.089 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240473+0000", 2024-09-21T15:01:21.089 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000100132 2024-09-21T15:01:21.089 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.089 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.089 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:21.089 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240476+0000", 2024-09-21T15:01:21.089 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.332e-06 2024-09-21T15:01:21.089 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.089 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.089 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:21.089 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256724+0000", 2024-09-21T15:01:21.089 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0162471930000001 2024-09-21T15:01:21.089 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.089 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.090 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:21.090 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256819+0000", 2024-09-21T15:01:21.090 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5007999999999996e-05 2024-09-21T15:01:21.090 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:21.090 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:21.090 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:21.090 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.090 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.090 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140188 3.16 3:6f6217d4:::rbd_data.113664780c85.00000000000000fb:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:21.090 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.240978+0000", 2024-09-21T15:01:21.090 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 18.756198022, 2024-09-21T15:01:21.090 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.015983809, 2024-09-21T15:01:21.090 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:21.090 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:21.091 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:21.091 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:21.091 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:21.091 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140188 2024-09-21T15:01:21.091 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.091 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:21.091 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.091 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:21.091 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240978+0000", 2024-09-21T15:01:21.091 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:21.091 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.091 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.091 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:21.091 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240978+0000", 2024-09-21T15:01:21.091 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:21.092 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.092 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.092 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:21.092 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240978+0000", 2024-09-21T15:01:21.092 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:21.092 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.092 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.092 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:21.092 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240981+0000", 2024-09-21T15:01:21.092 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1559999999999999e-06 2024-09-21T15:01:21.092 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.092 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.092 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:21.092 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240987+0000", 2024-09-21T15:01:21.092 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.229e-06 2024-09-21T15:01:21.093 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.093 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.093 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:21.093 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240992+0000", 2024-09-21T15:01:21.093 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9710000000000003e-06 2024-09-21T15:01:21.093 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.093 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.093 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:21.093 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.241361+0000", 2024-09-21T15:01:21.093 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000369236 2024-09-21T15:01:21.093 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.093 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.093 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:21.093 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.241369+0000", 2024-09-21T15:01:21.094 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6199999999999999e-06 2024-09-21T15:01:21.094 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.094 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.094 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:21.094 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256844+0000", 2024-09-21T15:01:21.094 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0154751609999999 2024-09-21T15:01:21.094 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:21.094 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:21.094 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:21.094 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256962+0000", 2024-09-21T15:01:21.094 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000117964 2024-09-21T15:01:21.094 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:21.094 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:21.095 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:21.095 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:21.095 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:21.095 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:21.096 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T15:01:21.098 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:01:21.092+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:01:21.199 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:01:21.196+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:01:21.213 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:01:21.213 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:01:21.213 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T15:01:21.213 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:01:21.223 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T15:01:21.300 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:01:21.296+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:01:21.341 INFO:teuthology.orchestra.run.smithi032.stderr:marked out osd.4. 2024-09-21T15:01:21.342 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:01:21.342 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:01:21.342 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T15:01:21.343 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T15:01:21.343 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:01:21.352 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T15:01:21.401 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:01:21.398+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:01:21.453 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:01:21.453 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T15:01:21.453 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T15:01:21.454 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T15:01:21.454 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.454 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:63820 3.19 3.626bc079 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-21T15:01:21.454 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:27.241293+0000", 2024-09-21T15:01:21.454 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 114.20911270000001, 2024-09-21T15:01:21.454 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.98723808199999996, 2024-09-21T15:01:21.454 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:21.454 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:21.454 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:21.454 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:21.454 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:21.454 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 63820 2024-09-21T15:01:21.454 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.454 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:21.455 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.455 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:21.455 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241293+0000", 2024-09-21T15:01:21.455 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:21.455 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.455 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.455 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:21.455 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241293+0000", 2024-09-21T15:01:21.455 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:21.455 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.455 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.455 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:21.455 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241291+0000", 2024-09-21T15:01:21.455 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:01:21.456 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.456 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.456 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:21.456 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241299+0000", 2024-09-21T15:01:21.456 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1529999999999997e-06 2024-09-21T15:01:21.456 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.456 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.456 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:21.456 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241317+0000", 2024-09-21T15:01:21.456 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7764000000000001e-05 2024-09-21T15:01:21.456 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.456 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.456 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:21.456 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241328+0000", 2024-09-21T15:01:21.457 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1245e-05 2024-09-21T15:01:21.457 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.457 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.457 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:21.457 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241407+0000", 2024-09-21T15:01:21.457 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8762000000000002e-05 2024-09-21T15:01:21.457 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.457 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.457 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:21.457 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241418+0000", 2024-09-21T15:01:21.457 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1695000000000001e-05 2024-09-21T15:01:21.457 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.457 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.457 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:21.458 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:28.228510+0000", 2024-09-21T15:01:21.458 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.98709166100000001 2024-09-21T15:01:21.458 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.458 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.458 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:21.458 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:28.228531+0000", 2024-09-21T15:01:21.458 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0696e-05 2024-09-21T15:01:21.458 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:21.458 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:21.458 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:21.458 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.458 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.458 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:97734 3.19 3:9fcf754f:::rbd_data.1136bfc79a70.0000000000000009:head [write 0~4194304 [fadvise_sequential+fadvise_nocache],write 4784128~147456 [fadvise_sequential+fadvise_nocache],write 4997120~147456 [fadvise_sequential+fadvise_nocache],write 5210112~32768 [fadvise_sequential+fadvise_nocache],write 5292032~147456 [fadvise_sequential+fadvise_nocache],write 5505024~147456 [fadvise_sequential+fadvise_nocache],write 5718016~49152 [fadvise_sequential+fadvise_nocache],write 5799936~147456 [fadvise_sequential+fadvise_nocache],write 6012928~147456 [fadvise_sequential+fadvise_nocache],write 6225920~65536 [fadvise_sequential+fadvise_nocache],write 6307840~147456 [fadvise_sequential+fadvise_nocache],write 6520832~147456 [fadvise_sequential+fadvise_nocache],write 6733824~229376 [fadvise_sequential+fadvise_nocache],write 7028736~147456 [fadvise_sequential+fadvise_nocache],write 7241728~229376 [fadvise_sequential+fadvise_nocache],write 7536640~147456 [fadvise_sequential+fadvise_nocache],write 7749632~229376 [fadvise_sequential+fadvise_nocache],write 8077312~114688 [fadvise_sequential+fadvise_nocache],write 8290304~98304 [fadvise_sequential+fadvise_nocache],write 8404992~81920 [fadvise_sequential+fadvise_nocache],write 8617984~81920 [fadvise_sequential+fadvise_nocache],write 8830976~81920 [fadvise_sequential+fadvise_nocache],write 8962048~32768 [fadvise_sequential+fadvise_nocache],write 9175040~32768 [fadvise_sequential+fadvise_nocache],write 9388032~32768 [fadvise_sequential+fadvise_nocache],write 9469952~32768 [fadvise_sequential+fadvise_nocache],write 9682944~20092 [fadvise_sequential+fadvise_nocache],write 9961472~163840 [fadvise_sequential+fadvise_nocache],write 10141696~196608 [fadvise_sequential+fadvise_nocache],write 10354688~278528 [fadvise_sequential+fadvise_nocache],write 10649600~196608 [fadvise_sequential+fadvise_nocache],write 10862592~196608 [fadvise_sequential+fadvise_nocache],write 11075584~65536 [fadvise_sequential+fadvise_nocache],write 11173888~98304 [fadvise_sequential+fadvise_nocache],write 11288576~65536 [fadvise_sequential+fadvise_nocache],write 11386880~98304 [fadvise_sequential+fadvise_nocache],write 11501568~65536 [fadvise_sequential+fadvise_nocache],write 11583488~65536 [fadvise_sequential+fadvise_nocache],write 11681792~98304 [fadvise_sequential+fadvise_nocache],write 11796480~65536 [fadvise_sequential+fadvise_nocache],write 11894784~98304 [fadvise_sequential+fadvise_nocache],write 12009472~65536 [fadvise_sequential+fadvise_nocache],write 12189696~98304 [fadvise_sequential+fadvise_nocache],write 12402688~98304 [fadvise_sequential+fadvise_nocache],write 12697600~98304 [fadvise_sequential+fadvise_nocache],write 12910592~98304 [fadvise_sequential+fadvise_nocache],write 13205504~98304 [fadvise_sequential+fadvise_nocache],write 13418496~98304 [fadvise_sequential+fadvise_nocache],write 13713408~98304 [fadvise_sequential+fadvise_nocache],write 13926400~98304 [fadvise_sequential+fadvise_nocache],write 14139392~16384 [fadvise_sequential+fadvise_nocache],write 14680064~16384 [fadvise_sequential+fadvise_nocache],write 14827520~81920 [fadvise_sequential+fadvise_nocache],write 15040512~81920 [fadvise_sequential+fadvise_nocache],write 15335424~81920 [fadvise_sequential+fadvise_nocache],write 15548416~81920 [fadvise_sequential+fadvise_nocache],write 15843328~81920 [fadvise_sequential+fadvise_nocache],write 16056320~81920 [fadvise_sequential+fadvise_nocache],write 16351232~81920 [fadvise_sequential+fadvise_nocache],write 16564224~81920 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:01:21.459 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.354331+0000", 2024-09-21T15:01:21.459 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 76.096074653000002, 2024-09-21T15:01:21.459 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.086284870999999999, 2024-09-21T15:01:21.459 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:21.459 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:21.459 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:21.459 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:21.459 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:21.459 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97734 2024-09-21T15:01:21.459 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.459 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:21.459 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.459 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:21.459 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.354331+0000", 2024-09-21T15:01:21.460 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:21.460 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.460 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.460 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:21.460 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.354331+0000", 2024-09-21T15:01:21.460 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:21.460 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.460 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.460 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:21.460 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.354329+0000", 2024-09-21T15:01:21.460 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:01:21.460 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.460 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.460 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:21.461 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.370234+0000", 2024-09-21T15:01:21.461 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015905348999999999 2024-09-21T15:01:21.461 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.461 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.461 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:21.461 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.370286+0000", 2024-09-21T15:01:21.461 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1656999999999997e-05 2024-09-21T15:01:21.461 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.461 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.461 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:21.461 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.370315+0000", 2024-09-21T15:01:21.461 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9204e-05 2024-09-21T15:01:21.461 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.461 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.462 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:21.462 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.370395+0000", 2024-09-21T15:01:21.462 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0395999999999994e-05 2024-09-21T15:01:21.462 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.462 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.462 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:21.462 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.370604+0000", 2024-09-21T15:01:21.462 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000208684 2024-09-21T15:01:21.462 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.462 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.462 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 1", 2024-09-21T15:01:21.462 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.371701+0000", 2024-09-21T15:01:21.462 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0010973140000000001 2024-09-21T15:01:21.462 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.463 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.463 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:01:21.463 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.405485+0000", 2024-09-21T15:01:21.463 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.033783615000000003 2024-09-21T15:01:21.463 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.463 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.463 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:01:21.463 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.440211+0000", 2024-09-21T15:01:21.463 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034725757000000003 2024-09-21T15:01:21.463 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.463 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.463 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:01:21.463 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.440497+0000", 2024-09-21T15:01:21.463 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00028673099999999998 2024-09-21T15:01:21.464 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.464 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.464 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:21.464 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.440616+0000", 2024-09-21T15:01:21.464 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000118256 2024-09-21T15:01:21.464 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:21.464 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:21.464 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:21.464 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.464 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.464 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112831 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 1515520~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:21.464 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.406761+0000", 2024-09-21T15:01:21.464 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 52.043643955, 2024-09-21T15:01:21.464 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060265666000000002, 2024-09-21T15:01:21.465 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:21.465 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:21.465 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:21.465 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:21.465 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:21.465 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112831 2024-09-21T15:01:21.465 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.465 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:21.465 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.465 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:21.465 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406761+0000", 2024-09-21T15:01:21.465 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:21.465 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.465 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.466 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:21.466 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406761+0000", 2024-09-21T15:01:21.466 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:21.466 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.466 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.466 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:21.466 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406761+0000", 2024-09-21T15:01:21.466 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:21.466 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.466 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.466 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:21.466 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406764+0000", 2024-09-21T15:01:21.466 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0319999999999999e-06 2024-09-21T15:01:21.466 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.467 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.467 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:21.467 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406776+0000", 2024-09-21T15:01:21.467 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2357999999999999e-05 2024-09-21T15:01:21.467 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.467 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.467 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:21.467 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406784+0000", 2024-09-21T15:01:21.467 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6059999999999997e-06 2024-09-21T15:01:21.467 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.467 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.467 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:21.467 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.466595+0000", 2024-09-21T15:01:21.467 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059811062999999998 2024-09-21T15:01:21.468 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.468 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.468 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:21.468 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.466782+0000", 2024-09-21T15:01:21.468 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000186622 2024-09-21T15:01:21.468 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.468 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.468 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:21.468 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.467027+0000", 2024-09-21T15:01:21.468 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000245455 2024-09-21T15:01:21.468 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:21.468 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:21.468 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:21.468 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.469 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.469 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112832 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 1605632~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:21.469 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.406788+0000", 2024-09-21T15:01:21.469 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 52.043617365000003, 2024-09-21T15:01:21.469 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060452065999999999, 2024-09-21T15:01:21.469 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:21.469 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:21.469 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:21.469 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:21.469 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:21.469 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112832 2024-09-21T15:01:21.469 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.469 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:21.469 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.470 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:21.470 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406788+0000", 2024-09-21T15:01:21.470 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:21.470 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.470 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.470 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:21.470 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406788+0000", 2024-09-21T15:01:21.470 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:21.470 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.470 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.470 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:21.470 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406788+0000", 2024-09-21T15:01:21.470 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:21.470 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.471 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.471 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:21.471 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406798+0000", 2024-09-21T15:01:21.471 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0096e-05 2024-09-21T15:01:21.471 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.471 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.471 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:21.471 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406803+0000", 2024-09-21T15:01:21.471 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2020000000000003e-06 2024-09-21T15:01:21.471 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.471 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.471 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:21.471 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406808+0000", 2024-09-21T15:01:21.471 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8239999999999999e-06 2024-09-21T15:01:21.471 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.472 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.472 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:21.472 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.467049+0000", 2024-09-21T15:01:21.472 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060241754000000002 2024-09-21T15:01:21.472 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.472 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.472 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:21.472 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.467134+0000", 2024-09-21T15:01:21.472 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4216999999999997e-05 2024-09-21T15:01:21.472 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.472 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.472 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:21.472 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.467240+0000", 2024-09-21T15:01:21.472 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000106304 2024-09-21T15:01:21.472 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:21.473 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:21.473 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:21.473 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.473 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.473 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112843 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 2613248~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:21.473 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.410273+0000", 2024-09-21T15:01:21.473 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 52.040132771000003, 2024-09-21T15:01:21.473 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059854931, 2024-09-21T15:01:21.473 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:21.473 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:21.473 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:21.473 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:21.473 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:21.473 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112843 2024-09-21T15:01:21.474 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.474 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:21.474 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.474 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:21.474 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410273+0000", 2024-09-21T15:01:21.474 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:21.474 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.474 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.474 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:21.474 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410273+0000", 2024-09-21T15:01:21.474 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:21.474 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.474 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.475 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:21.475 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410272+0000", 2024-09-21T15:01:21.475 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:21.475 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.475 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.475 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:21.475 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410274+0000", 2024-09-21T15:01:21.475 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.905e-06 2024-09-21T15:01:21.475 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.475 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.475 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:21.475 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410283+0000", 2024-09-21T15:01:21.475 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3440000000000007e-06 2024-09-21T15:01:21.475 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.475 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.476 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:21.476 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410294+0000", 2024-09-21T15:01:21.476 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0271e-05 2024-09-21T15:01:21.476 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.476 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.476 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:21.476 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.469917+0000", 2024-09-21T15:01:21.476 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059623738000000003 2024-09-21T15:01:21.476 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.476 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.476 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:21.476 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470002+0000", 2024-09-21T15:01:21.476 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4270999999999993e-05 2024-09-21T15:01:21.476 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.476 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.477 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:21.477 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470127+0000", 2024-09-21T15:01:21.477 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000125711 2024-09-21T15:01:21.477 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:21.477 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:21.477 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:21.477 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.477 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.477 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112844 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 2703360~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:21.477 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.410326+0000", 2024-09-21T15:01:21.477 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 52.040079050999999, 2024-09-21T15:01:21.477 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060086056999999998, 2024-09-21T15:01:21.477 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:21.477 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:21.478 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:21.478 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:21.478 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:21.478 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112844 2024-09-21T15:01:21.478 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.478 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:21.478 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.478 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:21.478 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410326+0000", 2024-09-21T15:01:21.478 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:21.478 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.478 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.478 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:21.478 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410326+0000", 2024-09-21T15:01:21.478 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:21.479 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.479 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.479 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:21.479 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410326+0000", 2024-09-21T15:01:21.479 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:21.479 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.479 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.479 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:21.479 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410352+0000", 2024-09-21T15:01:21.479 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6872999999999998e-05 2024-09-21T15:01:21.479 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.479 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.479 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:21.479 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410374+0000", 2024-09-21T15:01:21.479 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1141e-05 2024-09-21T15:01:21.480 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.480 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.480 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:21.480 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410385+0000", 2024-09-21T15:01:21.480 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1456e-05 2024-09-21T15:01:21.480 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.480 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.480 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:21.480 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470174+0000", 2024-09-21T15:01:21.480 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059788574999999997 2024-09-21T15:01:21.480 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.480 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.480 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:21.480 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470277+0000", 2024-09-21T15:01:21.480 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000103838 2024-09-21T15:01:21.481 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.481 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.481 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:21.481 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470412+0000", 2024-09-21T15:01:21.481 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013489700000000001 2024-09-21T15:01:21.481 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:21.481 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:21.481 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:21.481 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.481 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.481 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112845 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 2793472~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:21.481 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.410394+0000", 2024-09-21T15:01:21.481 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 52.040011577000001, 2024-09-21T15:01:21.481 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060244173999999998, 2024-09-21T15:01:21.481 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:21.482 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:21.482 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:21.482 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:21.482 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:21.482 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112845 2024-09-21T15:01:21.482 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.482 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:21.482 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.482 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:21.482 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410394+0000", 2024-09-21T15:01:21.482 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:21.482 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.482 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.482 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:21.483 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410394+0000", 2024-09-21T15:01:21.483 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:21.483 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.483 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.483 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:21.483 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410393+0000", 2024-09-21T15:01:21.483 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:21.483 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.483 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.483 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:21.483 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410397+0000", 2024-09-21T15:01:21.483 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9430000000000002e-06 2024-09-21T15:01:21.483 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.483 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.483 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:21.484 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410406+0000", 2024-09-21T15:01:21.484 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.799e-06 2024-09-21T15:01:21.484 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.484 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.484 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:21.484 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410418+0000", 2024-09-21T15:01:21.484 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2225999999999999e-05 2024-09-21T15:01:21.484 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.484 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.484 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:21.484 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470444+0000", 2024-09-21T15:01:21.484 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060026311999999998 2024-09-21T15:01:21.484 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.484 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.485 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:21.485 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470535+0000", 2024-09-21T15:01:21.485 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0173999999999994e-05 2024-09-21T15:01:21.485 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.485 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.485 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:21.485 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470638+0000", 2024-09-21T15:01:21.485 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010333 2024-09-21T15:01:21.485 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:21.485 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:21.485 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:21.485 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.485 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.485 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112846 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 2883584~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:21.486 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.410425+0000", 2024-09-21T15:01:21.486 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 52.039980364000002, 2024-09-21T15:01:21.486 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060459247000000001, 2024-09-21T15:01:21.486 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:21.486 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:21.486 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:21.486 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:21.486 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:21.486 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112846 2024-09-21T15:01:21.486 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.486 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:21.486 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.486 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:21.486 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410425+0000", 2024-09-21T15:01:21.487 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:21.487 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.487 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.487 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:21.487 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410425+0000", 2024-09-21T15:01:21.487 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:21.487 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.487 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.487 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:21.487 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410424+0000", 2024-09-21T15:01:21.487 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:21.487 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.487 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.487 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:21.488 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410428+0000", 2024-09-21T15:01:21.488 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4709999999999999e-06 2024-09-21T15:01:21.488 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.488 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.488 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:21.488 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410446+0000", 2024-09-21T15:01:21.488 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8354000000000001e-05 2024-09-21T15:01:21.488 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.488 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.488 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:21.488 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410456+0000", 2024-09-21T15:01:21.488 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6709999999999994e-06 2024-09-21T15:01:21.488 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.488 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.488 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:21.489 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470661+0000", 2024-09-21T15:01:21.489 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060205303000000002 2024-09-21T15:01:21.489 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.489 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.489 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:21.489 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470756+0000", 2024-09-21T15:01:21.489 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4401999999999995e-05 2024-09-21T15:01:21.489 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.489 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.489 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:21.489 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470884+0000", 2024-09-21T15:01:21.489 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012860500000000001 2024-09-21T15:01:21.489 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:21.489 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:21.490 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:21.490 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.490 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.490 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112847 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 2973696~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:21.490 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.410462+0000", 2024-09-21T15:01:21.490 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 52.039943319999999, 2024-09-21T15:01:21.490 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060653836000000003, 2024-09-21T15:01:21.490 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:21.490 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:21.490 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:21.490 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:21.490 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:21.490 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112847 2024-09-21T15:01:21.490 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.491 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:21.491 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.491 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:21.491 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410462+0000", 2024-09-21T15:01:21.491 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:21.491 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.491 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.491 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:21.491 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410462+0000", 2024-09-21T15:01:21.491 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:21.491 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.491 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.491 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:21.491 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410461+0000", 2024-09-21T15:01:21.492 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:21.492 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.492 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.492 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:21.492 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410465+0000", 2024-09-21T15:01:21.492 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6899999999999998e-06 2024-09-21T15:01:21.492 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.492 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.492 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:21.492 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410474+0000", 2024-09-21T15:01:21.492 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4179999999999997e-06 2024-09-21T15:01:21.492 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.492 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.492 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:21.493 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410508+0000", 2024-09-21T15:01:21.493 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4282999999999997e-05 2024-09-21T15:01:21.493 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.493 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.493 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:21.493 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470922+0000", 2024-09-21T15:01:21.493 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060413820999999999 2024-09-21T15:01:21.493 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.493 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.493 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:21.493 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471008+0000", 2024-09-21T15:01:21.493 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6379999999999996e-05 2024-09-21T15:01:21.493 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.493 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.493 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:21.494 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471116+0000", 2024-09-21T15:01:21.494 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000107806 2024-09-21T15:01:21.494 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:21.494 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:21.494 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:21.494 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.494 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.494 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112848 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 3063808~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:21.494 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.410525+0000", 2024-09-21T15:01:21.494 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 52.039880037000003, 2024-09-21T15:01:21.494 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060880735999999998, 2024-09-21T15:01:21.494 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:21.494 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:21.494 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:21.495 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:21.495 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:21.495 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112848 2024-09-21T15:01:21.495 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.495 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:21.495 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.495 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:21.495 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410525+0000", 2024-09-21T15:01:21.495 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:21.495 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.495 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.495 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:21.495 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410525+0000", 2024-09-21T15:01:21.495 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:21.496 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.496 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.496 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:21.496 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410525+0000", 2024-09-21T15:01:21.496 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:21.496 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.496 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.496 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:21.496 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410529+0000", 2024-09-21T15:01:21.496 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1760000000000003e-06 2024-09-21T15:01:21.496 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.496 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.496 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:21.496 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410551+0000", 2024-09-21T15:01:21.496 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2099999999999998e-05 2024-09-21T15:01:21.497 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.497 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.497 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:21.497 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410571+0000", 2024-09-21T15:01:21.497 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0159000000000001e-05 2024-09-21T15:01:21.497 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.497 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.497 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:21.497 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471164+0000", 2024-09-21T15:01:21.497 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.06059287 2024-09-21T15:01:21.497 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.497 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.497 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:21.497 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471271+0000", 2024-09-21T15:01:21.497 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000106635 2024-09-21T15:01:21.498 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.498 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.498 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:21.498 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471406+0000", 2024-09-21T15:01:21.498 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000135399 2024-09-21T15:01:21.498 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:21.498 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:21.498 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:21.498 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.498 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.498 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112849 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 3153920~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:21.498 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.410579+0000", 2024-09-21T15:01:21.498 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 52.039826742000002, 2024-09-21T15:01:21.498 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.061057304999999999, 2024-09-21T15:01:21.499 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:21.499 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:21.499 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:21.499 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:21.499 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:21.499 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112849 2024-09-21T15:01:21.499 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.499 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:21.499 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.499 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:21.499 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410579+0000", 2024-09-21T15:01:21.499 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:21.499 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.499 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.500 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:21.500 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410579+0000", 2024-09-21T15:01:21.500 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:21.500 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.500 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.500 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:21.500 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410578+0000", 2024-09-21T15:01:21.500 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:21.500 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.500 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.500 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:21.500 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410582+0000", 2024-09-21T15:01:21.500 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.036e-06 2024-09-21T15:01:21.500 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.500 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.501 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:21.501 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410592+0000", 2024-09-21T15:01:21.501 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9850000000000001e-06 2024-09-21T15:01:21.501 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.501 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.501 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:21.501 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410604+0000", 2024-09-21T15:01:21.501 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2051e-05 2024-09-21T15:01:21.501 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.501 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.501 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:21.501 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471444+0000", 2024-09-21T15:01:21.501 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060839722999999998 2024-09-21T15:01:21.501 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.502 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.502 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:21.502 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471530+0000", 2024-09-21T15:01:21.502 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6359000000000004e-05 2024-09-21T15:01:21.502 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.502 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.502 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:21.502 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471636+0000", 2024-09-21T15:01:21.502 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010576700000000001 2024-09-21T15:01:21.502 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:21.502 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:21.502 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:21.502 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.502 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.503 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112850 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 3244032~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:21.503 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.410698+0000", 2024-09-21T15:01:21.503 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 52.039707399999998, 2024-09-21T15:01:21.503 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.061164118000000003, 2024-09-21T15:01:21.503 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:21.503 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:21.503 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:21.503 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:21.503 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:21.503 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112850 2024-09-21T15:01:21.503 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.503 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:21.503 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.503 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:21.504 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410698+0000", 2024-09-21T15:01:21.504 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:21.504 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.504 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.504 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:21.504 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410698+0000", 2024-09-21T15:01:21.504 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:21.504 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.504 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.504 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:21.504 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410610+0000", 2024-09-21T15:01:21.504 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999123 2024-09-21T15:01:21.504 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.504 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.505 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:21.505 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410707+0000", 2024-09-21T15:01:21.505 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7093e-05 2024-09-21T15:01:21.505 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.505 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.505 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:21.505 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410728+0000", 2024-09-21T15:01:21.505 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0275000000000002e-05 2024-09-21T15:01:21.505 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.505 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.505 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:21.505 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410740+0000", 2024-09-21T15:01:21.505 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2561999999999999e-05 2024-09-21T15:01:21.505 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.506 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.506 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:21.506 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471664+0000", 2024-09-21T15:01:21.506 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060923979000000003 2024-09-21T15:01:21.506 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.506 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.506 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:21.506 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471753+0000", 2024-09-21T15:01:21.506 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8265999999999998e-05 2024-09-21T15:01:21.506 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.506 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.506 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:21.506 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471862+0000", 2024-09-21T15:01:21.506 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000109474 2024-09-21T15:01:21.506 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:21.507 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:21.507 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:21.507 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.507 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.507 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112864 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 4521984~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:21.507 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.414789+0000", 2024-09-21T15:01:21.507 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 52.035616374999996, 2024-09-21T15:01:21.507 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059845903999999998, 2024-09-21T15:01:21.507 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:21.507 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:21.507 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:21.507 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:21.507 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:21.507 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112864 2024-09-21T15:01:21.508 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.508 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:21.508 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.508 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:21.508 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414789+0000", 2024-09-21T15:01:21.508 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:21.508 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.508 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.508 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:21.508 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414789+0000", 2024-09-21T15:01:21.508 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:21.508 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.508 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.508 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:21.508 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414788+0000", 2024-09-21T15:01:21.509 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:21.509 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.509 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.509 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:21.509 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414790+0000", 2024-09-21T15:01:21.509 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.995e-06 2024-09-21T15:01:21.509 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.509 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.509 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:21.509 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414801+0000", 2024-09-21T15:01:21.509 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.01e-05 2024-09-21T15:01:21.509 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.509 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.509 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:21.509 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414804+0000", 2024-09-21T15:01:21.510 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.563e-06 2024-09-21T15:01:21.510 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.510 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.510 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:21.510 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.474499+0000", 2024-09-21T15:01:21.510 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059695208 2024-09-21T15:01:21.510 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.510 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.510 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:21.510 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.474561+0000", 2024-09-21T15:01:21.510 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1672000000000002e-05 2024-09-21T15:01:21.510 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.510 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.510 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:21.510 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.474635+0000", 2024-09-21T15:01:21.511 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3792000000000001e-05 2024-09-21T15:01:21.511 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:21.511 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:21.511 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:21.511 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.511 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.511 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112865 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 4612096~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:21.511 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.414807+0000", 2024-09-21T15:01:21.511 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 52.035597979000002, 2024-09-21T15:01:21.511 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060003506999999998, 2024-09-21T15:01:21.511 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:21.511 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:21.511 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:21.511 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:21.512 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:21.512 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112865 2024-09-21T15:01:21.512 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.512 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:21.512 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.512 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:21.512 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414807+0000", 2024-09-21T15:01:21.512 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:21.512 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.512 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.512 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:21.512 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414807+0000", 2024-09-21T15:01:21.512 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:21.512 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.512 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.513 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:21.513 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414807+0000", 2024-09-21T15:01:21.513 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:21.513 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.513 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.513 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:21.513 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414809+0000", 2024-09-21T15:01:21.513 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0509999999999999e-06 2024-09-21T15:01:21.513 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.513 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.513 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:21.513 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414813+0000", 2024-09-21T15:01:21.513 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0960000000000003e-06 2024-09-21T15:01:21.513 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.513 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.514 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:21.514 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414817+0000", 2024-09-21T15:01:21.514 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.296e-06 2024-09-21T15:01:21.514 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.514 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.514 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:21.514 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.474671+0000", 2024-09-21T15:01:21.514 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059853663000000001 2024-09-21T15:01:21.514 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.514 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.514 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:21.514 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.474735+0000", 2024-09-21T15:01:21.514 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3653999999999997e-05 2024-09-21T15:01:21.514 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.515 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.515 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:21.515 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.474811+0000", 2024-09-21T15:01:21.515 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6074000000000004e-05 2024-09-21T15:01:21.515 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:21.515 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:21.515 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:21.515 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.515 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.515 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112866 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 4702208~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:21.515 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.414820+0000", 2024-09-21T15:01:21.515 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 52.035585058999999, 2024-09-21T15:01:21.515 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060161992999999997, 2024-09-21T15:01:21.515 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:21.516 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:21.516 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:21.516 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:21.516 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:21.516 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112866 2024-09-21T15:01:21.516 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.516 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:21.516 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.516 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:21.516 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414820+0000", 2024-09-21T15:01:21.516 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:21.516 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.516 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.516 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:21.516 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414820+0000", 2024-09-21T15:01:21.517 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:21.517 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.517 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.517 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:21.517 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414820+0000", 2024-09-21T15:01:21.517 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:21.517 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.517 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.517 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:21.517 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414822+0000", 2024-09-21T15:01:21.517 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8509999999999999e-06 2024-09-21T15:01:21.517 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.517 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.517 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:21.518 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414826+0000", 2024-09-21T15:01:21.518 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8190000000000002e-06 2024-09-21T15:01:21.518 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.518 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.518 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:21.518 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414837+0000", 2024-09-21T15:01:21.518 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1232e-05 2024-09-21T15:01:21.518 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.518 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.518 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:21.518 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.474846+0000", 2024-09-21T15:01:21.518 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060009506999999997 2024-09-21T15:01:21.518 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.518 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.519 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:21.519 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.474910+0000", 2024-09-21T15:01:21.519 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3918999999999996e-05 2024-09-21T15:01:21.519 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.519 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.519 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:21.519 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.474982+0000", 2024-09-21T15:01:21.519 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1958999999999993e-05 2024-09-21T15:01:21.519 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:21.519 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:21.519 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:21.519 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.519 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.519 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112867 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 4792320~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:21.520 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.414840+0000", 2024-09-21T15:01:21.520 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 52.035565564000002, 2024-09-21T15:01:21.520 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060308542999999999, 2024-09-21T15:01:21.520 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:21.520 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:21.520 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:21.520 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:21.520 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:21.520 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112867 2024-09-21T15:01:21.520 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.520 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:21.520 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.520 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:21.520 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414840+0000", 2024-09-21T15:01:21.520 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:21.521 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.521 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.521 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:21.521 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414840+0000", 2024-09-21T15:01:21.521 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:21.521 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.521 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.521 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:21.521 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414839+0000", 2024-09-21T15:01:21.521 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:21.521 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.521 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.521 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:21.521 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414841+0000", 2024-09-21T15:01:21.521 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8500000000000001e-06 2024-09-21T15:01:21.522 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.522 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.522 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:21.522 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414846+0000", 2024-09-21T15:01:21.522 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7700000000000001e-06 2024-09-21T15:01:21.522 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.522 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.522 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:21.522 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414850+0000", 2024-09-21T15:01:21.522 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1389999999999997e-06 2024-09-21T15:01:21.522 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.522 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.522 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:21.522 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.475016+0000", 2024-09-21T15:01:21.522 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.06016614 2024-09-21T15:01:21.523 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.523 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.523 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:21.523 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.475073+0000", 2024-09-21T15:01:21.523 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6295000000000002e-05 2024-09-21T15:01:21.523 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.523 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.523 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:21.523 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.475148+0000", 2024-09-21T15:01:21.523 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5647000000000003e-05 2024-09-21T15:01:21.523 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:21.523 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:21.523 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:21.523 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.523 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.524 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:113030 3.6 3:60420fc2:::rbd_data.1136b442e94a.0000000000000001:head [write 0~360448 [fadvise_sequential+fadvise_nocache],write 385024~573440 [fadvise_sequential+fadvise_nocache],write 1146880~573440 [fadvise_sequential+fadvise_nocache],write 1908736~188416 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:21.524 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.422420+0000", 2024-09-21T15:01:21.524 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 52.027984918999998, 2024-09-21T15:01:21.524 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.063268959999999999, 2024-09-21T15:01:21.524 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:21.524 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:21.524 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:21.524 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:21.524 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:21.524 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 113030 2024-09-21T15:01:21.524 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.524 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:21.524 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.525 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:21.525 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.422420+0000", 2024-09-21T15:01:21.525 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:21.525 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.525 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.525 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:21.525 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.422420+0000", 2024-09-21T15:01:21.525 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:21.525 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.525 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.525 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:21.525 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.422420+0000", 2024-09-21T15:01:21.525 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:21.525 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.525 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.526 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:21.526 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.424657+0000", 2024-09-21T15:01:21.526 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0022369780000000001 2024-09-21T15:01:21.526 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.526 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.526 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:21.526 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.424673+0000", 2024-09-21T15:01:21.526 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6141e-05 2024-09-21T15:01:21.526 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.526 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.526 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:21.526 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.424689+0000", 2024-09-21T15:01:21.526 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5897999999999999e-05 2024-09-21T15:01:21.526 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.526 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.527 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:21.527 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.478005+0000", 2024-09-21T15:01:21.527 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.05331636 2024-09-21T15:01:21.527 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.527 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.527 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:21.527 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.478133+0000", 2024-09-21T15:01:21.527 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012736799999999999 2024-09-21T15:01:21.527 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.527 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.527 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 7", 2024-09-21T15:01:21.527 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.478307+0000", 2024-09-21T15:01:21.527 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017491499999999999 2024-09-21T15:01:21.527 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.527 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.528 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:01:21.528 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.483611+0000", 2024-09-21T15:01:21.528 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0053039389999999997 2024-09-21T15:01:21.528 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.528 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.528 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:01:21.528 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.485553+0000", 2024-09-21T15:01:21.528 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0019418759999999999 2024-09-21T15:01:21.528 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.528 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.528 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:01:21.528 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.485645+0000", 2024-09-21T15:01:21.528 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2138000000000005e-05 2024-09-21T15:01:21.528 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.529 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.529 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:21.529 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.485689+0000", 2024-09-21T15:01:21.529 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3922999999999999e-05 2024-09-21T15:01:21.529 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:21.529 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:21.529 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:21.529 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.529 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.529 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:114290 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 6946816~8192 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:21.529 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.558087+0000", 2024-09-21T15:01:21.529 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 51.892318121999999, 2024-09-21T15:01:21.529 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059973091999999999, 2024-09-21T15:01:21.529 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:21.530 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:21.530 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:21.530 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:21.530 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:21.530 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 114290 2024-09-21T15:01:21.530 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.530 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:21.530 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.530 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:21.530 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.558087+0000", 2024-09-21T15:01:21.530 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:21.530 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.530 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.530 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:21.530 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.558087+0000", 2024-09-21T15:01:21.531 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:21.531 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.531 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.531 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:21.531 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.558087+0000", 2024-09-21T15:01:21.531 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:21.531 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.531 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.531 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:21.531 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.558092+0000", 2024-09-21T15:01:21.531 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2379999999999997e-06 2024-09-21T15:01:21.531 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.531 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.531 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:21.531 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.558095+0000", 2024-09-21T15:01:21.532 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.275e-06 2024-09-21T15:01:21.532 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.532 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.532 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:21.532 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.558099+0000", 2024-09-21T15:01:21.532 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6679999999999999e-06 2024-09-21T15:01:21.532 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.532 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.532 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:21.532 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.617757+0000", 2024-09-21T15:01:21.532 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059657531999999999 2024-09-21T15:01:21.532 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.532 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.533 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:21.533 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.617891+0000", 2024-09-21T15:01:21.533 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000134478 2024-09-21T15:01:21.533 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.533 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.533 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:21.533 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.618060+0000", 2024-09-21T15:01:21.533 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016925100000000001 2024-09-21T15:01:21.533 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:21.533 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:21.533 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:21.533 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.533 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.533 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:114312 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 8945664~8192 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:21.533 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.562396+0000", 2024-09-21T15:01:21.534 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 51.888009773999997, 2024-09-21T15:01:21.534 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059832396000000003, 2024-09-21T15:01:21.534 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:21.534 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:21.534 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:21.534 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:21.534 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:21.534 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 114312 2024-09-21T15:01:21.534 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.534 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:21.534 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.534 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:21.535 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562396+0000", 2024-09-21T15:01:21.535 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:21.535 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.535 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.535 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:21.535 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562396+0000", 2024-09-21T15:01:21.535 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:21.535 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.535 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.535 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:21.535 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562395+0000", 2024-09-21T15:01:21.535 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:21.535 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.535 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.535 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:21.535 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562397+0000", 2024-09-21T15:01:21.536 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9e-06 2024-09-21T15:01:21.536 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.536 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.536 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:21.536 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562401+0000", 2024-09-21T15:01:21.536 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8099999999999999e-06 2024-09-21T15:01:21.536 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.536 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.536 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:21.536 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562405+0000", 2024-09-21T15:01:21.536 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.5710000000000002e-06 2024-09-21T15:01:21.536 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.536 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.536 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:21.536 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.621997+0000", 2024-09-21T15:01:21.537 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059592118999999999 2024-09-21T15:01:21.537 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.537 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.537 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:21.537 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.622121+0000", 2024-09-21T15:01:21.537 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012411599999999999 2024-09-21T15:01:21.537 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.537 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.537 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:21.537 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.622228+0000", 2024-09-21T15:01:21.537 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000107184 2024-09-21T15:01:21.537 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:21.537 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:21.537 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:21.537 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.538 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.538 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:114313 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 9035776~8192 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:21.538 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.562408+0000", 2024-09-21T15:01:21.538 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 51.887997792999997, 2024-09-21T15:01:21.538 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059975560999999997, 2024-09-21T15:01:21.538 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:21.538 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:21.538 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:21.538 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:21.538 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:21.538 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 114313 2024-09-21T15:01:21.538 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.538 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:21.538 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.538 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:21.539 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562408+0000", 2024-09-21T15:01:21.539 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:21.539 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.539 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.539 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:21.539 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562408+0000", 2024-09-21T15:01:21.539 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:21.539 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.539 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.539 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:21.539 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562407+0000", 2024-09-21T15:01:21.539 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:21.539 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.539 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.539 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:21.539 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562409+0000", 2024-09-21T15:01:21.539 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9700000000000002e-06 2024-09-21T15:01:21.540 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.540 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.540 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:21.540 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562421+0000", 2024-09-21T15:01:21.540 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2094e-05 2024-09-21T15:01:21.540 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.540 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.540 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:21.540 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562427+0000", 2024-09-21T15:01:21.540 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7690000000000002e-06 2024-09-21T15:01:21.540 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.540 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.540 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:21.540 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.622264+0000", 2024-09-21T15:01:21.541 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059837488000000001 2024-09-21T15:01:21.541 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.541 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.541 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:21.541 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.622319+0000", 2024-09-21T15:01:21.541 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4064999999999998e-05 2024-09-21T15:01:21.541 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.541 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.541 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:21.541 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.622383+0000", 2024-09-21T15:01:21.541 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4506999999999998e-05 2024-09-21T15:01:21.541 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:21.541 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:21.541 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:21.541 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:21.541 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:21.542 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:01:21.543 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T15:01:21.543 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:01:21.540+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:01:21.643 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:01:21.640+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:01:21.701 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:01:21.701 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:01:21.701 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T15:01:21.702 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:01:21.712 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T15:01:21.744 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:01:21.740+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:01:21.819 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:01:21.819 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:01:21.819 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T15:01:21.819 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T15:01:21.819 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:01:21.829 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T15:01:21.844 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:01:21.838+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:01:21.946 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:01:21.946+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:01:21.985 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:01:21.985 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T15:01:21.985 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T15:01:21.985 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T15:01:21.985 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.985 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45854 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:21.985 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159802+0000", 2024-09-21T15:01:21.985 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 131.82053063500001, 2024-09-21T15:01:21.986 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068358938999999994, 2024-09-21T15:01:21.986 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:21.986 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:21.986 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:21.986 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:21.986 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:21.986 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45854 2024-09-21T15:01:21.986 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.986 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:21.986 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.986 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:21.986 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159802+0000", 2024-09-21T15:01:21.987 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:21.987 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.987 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.987 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:21.987 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159802+0000", 2024-09-21T15:01:21.987 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:21.987 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.987 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.987 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:21.987 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159801+0000", 2024-09-21T15:01:21.987 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:21.987 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.988 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.988 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:21.988 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159804+0000", 2024-09-21T15:01:21.988 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0970000000000002e-06 2024-09-21T15:01:21.988 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.988 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.988 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:21.988 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159823+0000", 2024-09-21T15:01:21.988 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8895000000000001e-05 2024-09-21T15:01:21.988 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.988 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.988 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:21.988 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159837+0000", 2024-09-21T15:01:21.989 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3247999999999999e-05 2024-09-21T15:01:21.989 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.989 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.989 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:21.989 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228151+0000", 2024-09-21T15:01:21.989 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068314642999999994 2024-09-21T15:01:21.989 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.989 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.989 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:21.989 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228161+0000", 2024-09-21T15:01:21.989 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4250000000000004e-06 2024-09-21T15:01:21.989 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:21.990 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:21.990 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:21.990 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.990 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.990 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45985 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:21.990 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159897+0000", 2024-09-21T15:01:21.990 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 131.82043513299999, 2024-09-21T15:01:21.990 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068345664, 2024-09-21T15:01:21.990 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:21.990 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:21.990 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:21.990 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:21.991 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:21.991 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45985 2024-09-21T15:01:21.991 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.991 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:21.991 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.991 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:21.991 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159897+0000", 2024-09-21T15:01:21.991 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:21.991 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.991 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.991 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:21.992 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159897+0000", 2024-09-21T15:01:21.992 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:21.992 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.992 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.992 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:21.992 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159897+0000", 2024-09-21T15:01:21.992 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:21.992 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.992 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.992 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:21.992 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159900+0000", 2024-09-21T15:01:21.992 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.954e-06 2024-09-21T15:01:21.993 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.993 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.993 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:21.993 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159913+0000", 2024-09-21T15:01:21.993 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3106e-05 2024-09-21T15:01:21.993 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.993 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.993 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:21.993 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159918+0000", 2024-09-21T15:01:21.993 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5199999999999997e-06 2024-09-21T15:01:21.993 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.994 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.994 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:21.994 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228236+0000", 2024-09-21T15:01:21.994 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068317040999999995 2024-09-21T15:01:21.994 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.994 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.994 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:21.994 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228243+0000", 2024-09-21T15:01:21.994 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3880000000000002e-06 2024-09-21T15:01:21.995 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:21.995 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:21.995 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:21.995 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.995 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.995 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45986 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:21.995 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159924+0000", 2024-09-21T15:01:21.995 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 131.82040883799999, 2024-09-21T15:01:21.995 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068364362999999997, 2024-09-21T15:01:21.995 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:21.995 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:21.996 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:21.996 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:21.996 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:21.996 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45986 2024-09-21T15:01:21.996 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.996 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:21.996 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.996 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:21.996 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159924+0000", 2024-09-21T15:01:21.996 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:21.996 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.997 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.997 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:21.997 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159924+0000", 2024-09-21T15:01:21.997 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:21.997 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.997 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.997 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:21.997 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159923+0000", 2024-09-21T15:01:21.997 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:21.997 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.997 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.998 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:21.998 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159926+0000", 2024-09-21T15:01:21.998 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0510000000000001e-06 2024-09-21T15:01:21.998 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.998 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.998 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:21.998 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159932+0000", 2024-09-21T15:01:21.998 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4160000000000001e-06 2024-09-21T15:01:21.998 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.998 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.998 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:21.998 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159938+0000", 2024-09-21T15:01:21.999 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.383e-06 2024-09-21T15:01:21.999 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.999 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.999 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:21.999 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228275+0000", 2024-09-21T15:01:21.999 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068337007000000005 2024-09-21T15:01:21.999 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:21.999 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:21.999 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:21.999 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228288+0000", 2024-09-21T15:01:21.999 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3052e-05 2024-09-21T15:01:22.000 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:22.000 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:22.000 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:22.000 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.000 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.000 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45987 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:22.000 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159944+0000", 2024-09-21T15:01:22.000 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 131.820388614, 2024-09-21T15:01:22.000 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068387232000000006, 2024-09-21T15:01:22.000 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:22.000 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:22.000 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:22.000 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:22.000 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:22.001 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45987 2024-09-21T15:01:22.001 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.001 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:22.001 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.001 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:22.001 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159944+0000", 2024-09-21T15:01:22.001 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:22.001 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.001 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.001 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:22.001 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159944+0000", 2024-09-21T15:01:22.001 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:22.001 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.002 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.002 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:22.002 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159943+0000", 2024-09-21T15:01:22.002 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:22.002 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.002 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.002 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:22.002 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159946+0000", 2024-09-21T15:01:22.002 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6309999999999999e-06 2024-09-21T15:01:22.002 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.002 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.002 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:22.003 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159952+0000", 2024-09-21T15:01:22.003 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6540000000000004e-06 2024-09-21T15:01:22.003 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.003 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.003 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:22.003 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159957+0000", 2024-09-21T15:01:22.003 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9280000000000001e-06 2024-09-21T15:01:22.003 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.003 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.003 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:22.003 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228322+0000", 2024-09-21T15:01:22.003 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068365067000000002 2024-09-21T15:01:22.004 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.004 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.004 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:22.004 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228331+0000", 2024-09-21T15:01:22.004 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3270000000000007e-06 2024-09-21T15:01:22.004 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:22.004 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:22.004 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:22.004 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.004 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.004 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:97672 3.4 3:26046d29:::rbd_data.1136bfc79a70.0000000000000001:head [write 0~4718592 [fadvise_sequential+fadvise_nocache],write 5242880~131072 [fadvise_sequential+fadvise_nocache],write 5439488~147456 [fadvise_sequential+fadvise_nocache],write 5652480~229376 [fadvise_sequential+fadvise_nocache],write 5947392~147456 [fadvise_sequential+fadvise_nocache],write 6160384~229376 [fadvise_sequential+fadvise_nocache],write 6455296~147456 [fadvise_sequential+fadvise_nocache],write 6668288~229376 [fadvise_sequential+fadvise_nocache],write 6963200~147456 [fadvise_sequential+fadvise_nocache],write 7176192~147456 [fadvise_sequential+fadvise_nocache],write 7340032~65536 [fadvise_sequential+fadvise_nocache],write 7471104~147456 [fadvise_sequential+fadvise_nocache],write 7684096~147456 [fadvise_sequential+fadvise_nocache],write 7864320~49152 [fadvise_sequential+fadvise_nocache],write 7979008~147456 [fadvise_sequential+fadvise_nocache],write 8192000~147456 [fadvise_sequential+fadvise_nocache],write 8388608~32768 [fadvise_sequential+fadvise_nocache],write 8519680~114688 [fadvise_sequential+fadvise_nocache],write 8732672~114688 [fadvise_sequential+fadvise_nocache],write 8912896~16384 [fadvise_sequential+fadvise_nocache],write 9109504~32768 [fadvise_sequential+fadvise_nocache],write 9322496~32768 [fadvise_sequential+fadvise_nocache],write 9617408~32768 [fadvise_sequential+fadvise_nocache],write 9830400~32768 [fadvise_sequential+fadvise_nocache],write 9961472~98304 [fadvise_sequential+fadvise_nocache],write 10076160~196608 [fadvise_sequential+fadvise_nocache],write 10289152~278528 [fadvise_sequential+fadvise_nocache],write 10584064~196608 [fadvise_sequential+fadvise_nocache],write 10797056~196608 [fadvise_sequential+fadvise_nocache],write 11010048~65536 [fadvise_sequential+fadvise_nocache],write 11091968~196608 [fadvise_sequential+fadvise_nocache],write 11304960~196608 [fadvise_sequential+fadvise_nocache],write 11517952~65536 [fadvise_sequential+fadvise_nocache],write 11616256~98304 [fadvise_sequential+fadvise_nocache],write 11730944~65536 [fadvise_sequential+fadvise_nocache],write 11829248~98304 [fadvise_sequential+fadvise_nocache],write 11943936~65536 [fadvise_sequential+fadvise_nocache],write 12042240~49152 [fadvise_sequential+fadvise_nocache],write 12124160~98304 [fadvise_sequential+fadvise_nocache],write 12238848~65536 [fadvise_sequential+fadvise_nocache],write 12337152~98304 [fadvise_sequential+fadvise_nocache],write 12451840~65536 [fadvise_sequential+fadvise_nocache],write 12550144~32768 [fadvise_sequential+fadvise_nocache],write 12632064~98304 [fadvise_sequential+fadvise_nocache],write 12845056~98304 [fadvise_sequential+fadvise_nocache],write 13058048~49152 [fadvise_sequential+fadvise_nocache],write 13139968~98304 [fadvise_sequential+fadvise_nocache],write 13352960~98304 [fadvise_sequential+fadvise_nocache],write 13565952~65536 [fadvise_sequential+fadvise_nocache],write 13647872~98304 [fadvise_sequential+fadvise_nocache],write 13860864~98304 [fadvise_sequential+fadvise_nocache],write 14073856~81920 [fadvise_sequential+fadvise_nocache],write 15269888~81920 [fadvise_sequential+fadvise_nocache],write 15482880~81920 [fadvise_sequential+fadvise_nocache],write 15695872~32768 [fadvise_sequential+fadvise_nocache],write 15777792~81920 [fadvise_sequential+fadvise_nocache],write 15990784~81920 [fadvise_sequential+fadvise_nocache],write 16203776~49152 [fadvise_sequential+fadvise_nocache],write 16285696~81920 [fadvise_sequential+fadvise_nocache],write 16498688~81920 [fadvise_sequential+fadvise_nocache],write 16711680~65536 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:01:22.005 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.298669+0000", 2024-09-21T15:01:22.005 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 76.681663548000003, 2024-09-21T15:01:22.005 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.10282358900000001, 2024-09-21T15:01:22.005 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:22.005 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:22.005 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:22.005 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:22.005 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:22.005 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97672 2024-09-21T15:01:22.005 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.005 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:22.005 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.005 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:22.006 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.298669+0000", 2024-09-21T15:01:22.006 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:22.006 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.006 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.006 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:22.006 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.298669+0000", 2024-09-21T15:01:22.006 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:22.006 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.006 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.006 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:22.006 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.298667+0000", 2024-09-21T15:01:22.006 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:01:22.007 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.007 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.007 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:22.007 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.327458+0000", 2024-09-21T15:01:22.007 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.028790754000000002 2024-09-21T15:01:22.007 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.007 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.007 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:22.007 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.327492+0000", 2024-09-21T15:01:22.007 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3806999999999997e-05 2024-09-21T15:01:22.007 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.008 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.008 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:22.008 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.327507+0000", 2024-09-21T15:01:22.008 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5112e-05 2024-09-21T15:01:22.008 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.008 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.008 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:22.008 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.327597+0000", 2024-09-21T15:01:22.008 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9938999999999994e-05 2024-09-21T15:01:22.008 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.008 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.008 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:22.008 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.327927+0000", 2024-09-21T15:01:22.008 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000329766 2024-09-21T15:01:22.009 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.009 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.009 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T15:01:22.009 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.330735+0000", 2024-09-21T15:01:22.009 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0028086869999999998 2024-09-21T15:01:22.009 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.009 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.009 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:01:22.009 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.378299+0000", 2024-09-21T15:01:22.009 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.047563495999999997 2024-09-21T15:01:22.009 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.009 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.009 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:01:22.010 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.401248+0000", 2024-09-21T15:01:22.010 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.022949311 2024-09-21T15:01:22.010 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.010 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.010 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:01:22.010 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.401446+0000", 2024-09-21T15:01:22.010 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019831199999999999 2024-09-21T15:01:22.010 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.010 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.010 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:22.010 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.401492+0000", 2024-09-21T15:01:22.010 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6109000000000002e-05 2024-09-21T15:01:22.010 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:22.011 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:22.011 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:22.011 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.011 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.011 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139769 3.1e 3:793dcf44:::rbd_data.113664780c85.0000000000000000:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:22.011 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.221432+0000", 2024-09-21T15:01:22.011 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 19.758900679, 2024-09-21T15:01:22.011 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.035363362, 2024-09-21T15:01:22.011 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:22.011 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:22.011 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:22.011 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:22.011 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:22.012 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139769 2024-09-21T15:01:22.012 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.012 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:22.012 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.012 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:22.012 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221432+0000", 2024-09-21T15:01:22.012 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:22.012 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.012 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.012 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:22.012 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221432+0000", 2024-09-21T15:01:22.012 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:22.012 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.012 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.013 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:22.013 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221430+0000", 2024-09-21T15:01:22.013 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:01:22.013 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.013 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.013 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:22.013 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221438+0000", 2024-09-21T15:01:22.013 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2630000000000003e-06 2024-09-21T15:01:22.013 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.013 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.013 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:22.013 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221458+0000", 2024-09-21T15:01:22.013 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0071000000000001e-05 2024-09-21T15:01:22.014 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.014 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.014 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:22.014 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221467+0000", 2024-09-21T15:01:22.014 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.816e-06 2024-09-21T15:01:22.014 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.014 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.014 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:22.014 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221572+0000", 2024-09-21T15:01:22.014 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000105166 2024-09-21T15:01:22.014 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.014 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.015 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:22.015 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221589+0000", 2024-09-21T15:01:22.015 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7391e-05 2024-09-21T15:01:22.015 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.015 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.015 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:22.015 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.255741+0000", 2024-09-21T15:01:22.015 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0341519690000001 2024-09-21T15:01:22.015 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.015 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.015 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:22.015 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.256353+0000", 2024-09-21T15:01:22.015 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000611544 2024-09-21T15:01:22.016 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.016 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.016 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:22.016 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.256795+0000", 2024-09-21T15:01:22.016 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00044208700000000002 2024-09-21T15:01:22.016 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:22.016 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:22.016 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:22.016 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.016 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.016 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139773 3.1e 3:7fa3054f:::rbd_data.113664780c85.000000000000001c:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:22.016 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.221517+0000", 2024-09-21T15:01:22.016 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 19.758815635000001, 2024-09-21T15:01:22.016 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0367423140000001, 2024-09-21T15:01:22.017 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:22.017 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:22.017 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:22.017 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:22.017 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:22.017 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139773 2024-09-21T15:01:22.017 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.017 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:22.017 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.017 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:22.017 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221517+0000", 2024-09-21T15:01:22.017 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:22.017 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.018 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.018 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:22.018 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221517+0000", 2024-09-21T15:01:22.018 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:22.018 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.018 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.018 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:22.018 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221516+0000", 2024-09-21T15:01:22.018 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:22.018 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.018 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.018 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:22.018 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221520+0000", 2024-09-21T15:01:22.019 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1030000000000004e-06 2024-09-21T15:01:22.019 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.019 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.019 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:22.019 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221566+0000", 2024-09-21T15:01:22.019 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5512000000000001e-05 2024-09-21T15:01:22.019 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.019 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.019 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:22.019 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221575+0000", 2024-09-21T15:01:22.019 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9630000000000004e-06 2024-09-21T15:01:22.019 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.019 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.020 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:22.020 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221650+0000", 2024-09-21T15:01:22.020 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5531999999999994e-05 2024-09-21T15:01:22.020 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.020 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.020 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:22.020 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221661+0000", 2024-09-21T15:01:22.020 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0878e-05 2024-09-21T15:01:22.020 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.020 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.020 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:22.020 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.257513+0000", 2024-09-21T15:01:22.020 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0358513089999999 2024-09-21T15:01:22.020 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.021 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.021 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:22.021 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.257885+0000", 2024-09-21T15:01:22.021 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00037274299999999998 2024-09-21T15:01:22.021 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.021 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.021 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:22.021 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.258259+0000", 2024-09-21T15:01:22.021 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000373829 2024-09-21T15:01:22.021 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:22.021 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:22.021 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:22.022 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.022 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.022 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139789 3.1e 3:7c803367:::rbd_data.113664780c85.000000000000008c:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:22.022 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.221656+0000", 2024-09-21T15:01:22.022 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 19.758676004000002, 2024-09-21T15:01:22.022 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.037139595, 2024-09-21T15:01:22.022 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:22.022 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:22.022 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:22.022 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:22.022 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:22.023 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139789 2024-09-21T15:01:22.023 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.023 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:22.023 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.023 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:22.023 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221656+0000", 2024-09-21T15:01:22.023 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:22.023 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.023 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.023 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:22.023 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221656+0000", 2024-09-21T15:01:22.023 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:22.024 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.024 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.024 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:22.024 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221656+0000", 2024-09-21T15:01:22.024 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:22.024 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.024 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.024 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:22.024 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221660+0000", 2024-09-21T15:01:22.024 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0369999999999996e-06 2024-09-21T15:01:22.024 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.025 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.025 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:22.025 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221668+0000", 2024-09-21T15:01:22.025 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3440000000000001e-06 2024-09-21T15:01:22.025 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.025 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.025 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:22.025 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221674+0000", 2024-09-21T15:01:22.025 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0920000000000003e-06 2024-09-21T15:01:22.025 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.025 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.026 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:22.026 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221787+0000", 2024-09-21T15:01:22.026 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000112289 2024-09-21T15:01:22.026 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.026 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.026 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:22.026 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221798+0000", 2024-09-21T15:01:22.026 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1419e-05 2024-09-21T15:01:22.026 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.026 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.027 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:22.027 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.258642+0000", 2024-09-21T15:01:22.027 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.036843851 2024-09-21T15:01:22.027 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.027 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.027 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:22.027 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.258796+0000", 2024-09-21T15:01:22.027 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015414199999999999 2024-09-21T15:01:22.027 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:22.027 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:22.027 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:22.028 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.028 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.028 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139832 3.1e 3:7b675eeb:::rbd_data.113664780c85.00000000000001b3:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:22.028 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.223976+0000", 2024-09-21T15:01:22.028 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 19.756356271000001, 2024-09-21T15:01:22.028 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0350267980000001, 2024-09-21T15:01:22.028 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:22.028 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:22.028 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:22.028 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:22.029 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:22.029 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139832 2024-09-21T15:01:22.029 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.029 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:22.029 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.029 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:22.029 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223976+0000", 2024-09-21T15:01:22.029 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:22.029 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.029 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.029 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:22.029 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223976+0000", 2024-09-21T15:01:22.029 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:22.030 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.030 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.030 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:22.030 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223976+0000", 2024-09-21T15:01:22.030 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:22.030 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.030 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.030 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:22.030 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223981+0000", 2024-09-21T15:01:22.030 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1560000000000003e-06 2024-09-21T15:01:22.030 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.030 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.031 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:22.031 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223999+0000", 2024-09-21T15:01:22.031 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8808e-05 2024-09-21T15:01:22.031 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.031 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.031 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:22.031 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224008+0000", 2024-09-21T15:01:22.031 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7269999999999996e-06 2024-09-21T15:01:22.031 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.031 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.031 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:22.031 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224208+0000", 2024-09-21T15:01:22.031 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000199614 2024-09-21T15:01:22.032 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.032 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.032 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:22.032 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224218+0000", 2024-09-21T15:01:22.032 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0209e-05 2024-09-21T15:01:22.032 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.032 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.032 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:22.032 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.258839+0000", 2024-09-21T15:01:22.032 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.034621083 2024-09-21T15:01:22.032 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.032 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.032 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:22.032 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.259003+0000", 2024-09-21T15:01:22.033 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016379 2024-09-21T15:01:22.033 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:22.033 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:22.033 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:22.033 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.033 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.033 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139900 3.1e 3:7b440264:::rbd_data.113664780c85.0000000000000013:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:22.033 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.225748+0000", 2024-09-21T15:01:22.033 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 19.754584269999999, 2024-09-21T15:01:22.033 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0339966009999999, 2024-09-21T15:01:22.033 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:22.033 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:22.033 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:22.034 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:22.034 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:22.034 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139900 2024-09-21T15:01:22.034 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.034 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:22.034 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.034 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:22.034 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.225748+0000", 2024-09-21T15:01:22.034 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:22.034 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.034 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.035 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:22.035 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.225748+0000", 2024-09-21T15:01:22.035 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:22.035 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.035 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.035 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:22.035 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.225746+0000", 2024-09-21T15:01:22.035 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:22.035 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.035 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.035 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:22.035 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.225757+0000", 2024-09-21T15:01:22.035 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.066e-05 2024-09-21T15:01:22.036 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.036 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.036 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:22.036 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.225775+0000", 2024-09-21T15:01:22.036 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7631000000000001e-05 2024-09-21T15:01:22.036 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.036 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.036 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:22.036 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.225788+0000", 2024-09-21T15:01:22.036 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3046e-05 2024-09-21T15:01:22.036 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.036 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.036 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:22.036 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.225881+0000", 2024-09-21T15:01:22.037 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2987999999999998e-05 2024-09-21T15:01:22.037 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.037 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.037 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:22.037 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.225893+0000", 2024-09-21T15:01:22.037 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2398999999999999e-05 2024-09-21T15:01:22.037 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.037 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.037 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:22.037 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.259207+0000", 2024-09-21T15:01:22.037 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.033313696 2024-09-21T15:01:22.037 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.037 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.038 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:22.038 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.259452+0000", 2024-09-21T15:01:22.038 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00024495700000000002 2024-09-21T15:01:22.038 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.038 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.038 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:22.038 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.259745+0000", 2024-09-21T15:01:22.038 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000292868 2024-09-21T15:01:22.038 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:22.038 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:22.038 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:22.038 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.038 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.039 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140008 3.1e 3:7cd8b476:::rbd_data.113664780c85.0000000000000149:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:22.039 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.229555+0000", 2024-09-21T15:01:22.039 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 19.750777406000001, 2024-09-21T15:01:22.039 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0304914199999999, 2024-09-21T15:01:22.039 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:22.039 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:22.039 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:22.039 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:22.039 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:22.039 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140008 2024-09-21T15:01:22.039 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.039 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:22.039 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.040 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:22.040 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.229555+0000", 2024-09-21T15:01:22.040 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:22.040 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.040 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.040 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:22.040 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.229555+0000", 2024-09-21T15:01:22.040 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:22.040 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.040 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.040 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:22.040 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.229554+0000", 2024-09-21T15:01:22.040 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:22.040 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.041 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.041 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:22.041 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.229559+0000", 2024-09-21T15:01:22.041 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1059999999999999e-06 2024-09-21T15:01:22.041 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.041 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.041 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:22.041 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.229570+0000", 2024-09-21T15:01:22.041 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0465000000000001e-05 2024-09-21T15:01:22.041 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.041 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.041 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:22.041 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.229581+0000", 2024-09-21T15:01:22.041 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0958000000000001e-05 2024-09-21T15:01:22.042 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.042 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.042 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:22.042 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.229660+0000", 2024-09-21T15:01:22.042 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9103999999999993e-05 2024-09-21T15:01:22.042 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.042 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.042 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:22.042 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.229675+0000", 2024-09-21T15:01:22.042 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5622999999999999e-05 2024-09-21T15:01:22.042 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.042 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.043 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:22.043 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.259816+0000", 2024-09-21T15:01:22.043 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.030140595 2024-09-21T15:01:22.043 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.043 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.043 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:22.043 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.260046+0000", 2024-09-21T15:01:22.043 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00023045300000000001 2024-09-21T15:01:22.043 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:22.043 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:22.043 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:22.043 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.043 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.044 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140037 3.1e 3:7a4585d9:::rbd_data.113664780c85.000000000000005c:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:22.044 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.230439+0000", 2024-09-21T15:01:22.044 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 19.749892918, 2024-09-21T15:01:22.044 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.030277613, 2024-09-21T15:01:22.044 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:22.044 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:22.044 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:22.044 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:22.044 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:22.044 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140037 2024-09-21T15:01:22.044 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.044 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:22.044 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.045 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:22.045 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.230439+0000", 2024-09-21T15:01:22.045 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:22.045 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.045 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.045 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:22.045 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.230439+0000", 2024-09-21T15:01:22.045 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:22.045 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.045 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.045 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:22.045 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.230438+0000", 2024-09-21T15:01:22.045 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:22.046 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.046 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.046 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:22.046 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.230447+0000", 2024-09-21T15:01:22.046 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6270000000000006e-06 2024-09-21T15:01:22.046 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.046 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.046 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:22.046 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.230464+0000", 2024-09-21T15:01:22.046 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7394e-05 2024-09-21T15:01:22.046 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.046 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.046 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:22.047 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.230472+0000", 2024-09-21T15:01:22.047 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0360000000000006e-06 2024-09-21T15:01:22.047 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.047 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.047 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:22.047 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.230567+0000", 2024-09-21T15:01:22.047 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4648000000000006e-05 2024-09-21T15:01:22.047 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.047 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.047 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:22.047 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.230581+0000", 2024-09-21T15:01:22.047 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3799e-05 2024-09-21T15:01:22.047 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.047 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.048 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:22.048 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.260095+0000", 2024-09-21T15:01:22.048 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.029514153 2024-09-21T15:01:22.048 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.048 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.048 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:22.048 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.260356+0000", 2024-09-21T15:01:22.048 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00026118400000000003 2024-09-21T15:01:22.048 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.048 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.048 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:22.048 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.260717+0000", 2024-09-21T15:01:22.048 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00036124799999999999 2024-09-21T15:01:22.049 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:22.049 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:22.049 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:22.049 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.049 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.049 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140061 3.1e 3:7fdd08f8:::rbd_data.113664780c85.0000000000000104:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:22.049 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.232026+0000", 2024-09-21T15:01:22.049 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 19.748306587999998, 2024-09-21T15:01:22.049 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0290620450000001, 2024-09-21T15:01:22.049 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:22.049 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:22.049 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:22.049 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:22.049 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:22.050 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140061 2024-09-21T15:01:22.050 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.050 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:22.050 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.050 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:22.050 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232026+0000", 2024-09-21T15:01:22.050 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:22.050 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.050 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.050 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:22.050 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232026+0000", 2024-09-21T15:01:22.050 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:22.051 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.051 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.051 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:22.051 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232024+0000", 2024-09-21T15:01:22.051 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:01:22.051 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.051 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.051 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:22.051 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232034+0000", 2024-09-21T15:01:22.051 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0421000000000001e-05 2024-09-21T15:01:22.051 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.051 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.051 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:22.052 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232058+0000", 2024-09-21T15:01:22.052 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3944e-05 2024-09-21T15:01:22.052 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.052 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.052 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:22.052 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232071+0000", 2024-09-21T15:01:22.052 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2816999999999999e-05 2024-09-21T15:01:22.052 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.052 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.052 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:22.052 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232180+0000", 2024-09-21T15:01:22.052 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000109323 2024-09-21T15:01:22.052 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.052 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.053 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:22.053 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232199+0000", 2024-09-21T15:01:22.053 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8606000000000001e-05 2024-09-21T15:01:22.053 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.053 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.053 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:22.053 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.260821+0000", 2024-09-21T15:01:22.053 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0286222270000001 2024-09-21T15:01:22.053 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.053 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.053 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:22.053 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.261088+0000", 2024-09-21T15:01:22.053 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00026663799999999998 2024-09-21T15:01:22.053 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:22.054 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:22.054 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:22.054 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.054 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.054 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140090 3.1e 3:79f99893:::rbd_data.113664780c85.000000000000000b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:22.054 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.233668+0000", 2024-09-21T15:01:22.054 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 19.746664722999999, 2024-09-21T15:01:22.054 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0281047409999999, 2024-09-21T15:01:22.054 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:22.054 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:22.054 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:22.054 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:22.055 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:22.055 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140090 2024-09-21T15:01:22.055 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.055 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:22.055 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.055 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:22.055 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233668+0000", 2024-09-21T15:01:22.055 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:22.055 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.055 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.055 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:22.056 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233668+0000", 2024-09-21T15:01:22.056 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:22.056 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.056 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.056 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:22.056 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233666+0000", 2024-09-21T15:01:22.056 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:22.056 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.056 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.056 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:22.056 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233673+0000", 2024-09-21T15:01:22.056 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1840000000000002e-06 2024-09-21T15:01:22.056 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.056 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.057 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:22.057 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233689+0000", 2024-09-21T15:01:22.057 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5366000000000002e-05 2024-09-21T15:01:22.057 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.057 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.057 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:22.057 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233697+0000", 2024-09-21T15:01:22.057 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7679999999999997e-06 2024-09-21T15:01:22.057 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.057 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.057 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:22.057 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233772+0000", 2024-09-21T15:01:22.058 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4832000000000005e-05 2024-09-21T15:01:22.058 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.058 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.058 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:22.058 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233790+0000", 2024-09-21T15:01:22.058 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7297999999999999e-05 2024-09-21T15:01:22.058 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.058 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.058 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:22.058 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.261145+0000", 2024-09-21T15:01:22.058 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.027355381 2024-09-21T15:01:22.058 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.058 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.059 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:22.059 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.261466+0000", 2024-09-21T15:01:22.059 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00032105300000000002 2024-09-21T15:01:22.059 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.059 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.059 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:22.059 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.261772+0000", 2024-09-21T15:01:22.059 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00030635100000000001 2024-09-21T15:01:22.059 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:22.059 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:22.059 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:22.059 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.059 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.059 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140128 3.1e 3:78baf82e:::rbd_data.113664780c85.0000000000000115:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:22.060 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.235029+0000", 2024-09-21T15:01:22.060 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 19.745303469, 2024-09-21T15:01:22.060 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.02698942, 2024-09-21T15:01:22.060 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:22.060 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:22.060 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:22.060 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:22.060 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:22.060 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140128 2024-09-21T15:01:22.060 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.060 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:22.060 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.060 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:22.061 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235029+0000", 2024-09-21T15:01:22.061 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:22.061 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.061 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.061 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:22.061 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235029+0000", 2024-09-21T15:01:22.061 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:22.061 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.061 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.061 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:22.061 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235028+0000", 2024-09-21T15:01:22.061 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:22.061 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.061 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.062 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:22.062 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235032+0000", 2024-09-21T15:01:22.062 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0579999999999999e-06 2024-09-21T15:01:22.062 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.062 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.062 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:22.062 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235040+0000", 2024-09-21T15:01:22.062 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2099999999999993e-06 2024-09-21T15:01:22.062 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.062 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.062 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:22.062 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235047+0000", 2024-09-21T15:01:22.062 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1099999999999999e-06 2024-09-21T15:01:22.063 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.063 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.063 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:22.063 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235123+0000", 2024-09-21T15:01:22.063 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6835999999999994e-05 2024-09-21T15:01:22.063 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.063 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.063 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:22.063 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235136+0000", 2024-09-21T15:01:22.063 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2962000000000001e-05 2024-09-21T15:01:22.063 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.063 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.063 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:22.064 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.261827+0000", 2024-09-21T15:01:22.064 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0266906149999999 2024-09-21T15:01:22.064 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.064 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.064 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:22.064 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.262018+0000", 2024-09-21T15:01:22.064 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019132400000000001 2024-09-21T15:01:22.064 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:22.064 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:22.064 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:22.064 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.064 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.064 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140217 3.1e 3:793dcf44:::rbd_data.113664780c85.0000000000000000:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:22.065 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.235497+0000", 2024-09-21T15:01:22.065 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 19.744835170000002, 2024-09-21T15:01:22.065 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0269053020000001, 2024-09-21T15:01:22.065 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:22.065 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:22.065 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:22.065 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:22.065 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:22.065 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140217 2024-09-21T15:01:22.065 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.065 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:22.065 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.065 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:22.066 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235497+0000", 2024-09-21T15:01:22.066 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:22.066 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.066 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.066 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:22.066 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235497+0000", 2024-09-21T15:01:22.066 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:22.066 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.066 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.066 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:22.066 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235496+0000", 2024-09-21T15:01:22.066 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:22.067 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.067 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.067 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:22.067 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235503+0000", 2024-09-21T15:01:22.067 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8460000000000002e-06 2024-09-21T15:01:22.067 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.067 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.067 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:22.067 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235514+0000", 2024-09-21T15:01:22.067 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0576e-05 2024-09-21T15:01:22.067 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.067 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.067 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:22.067 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235522+0000", 2024-09-21T15:01:22.068 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2640000000000008e-06 2024-09-21T15:01:22.068 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.068 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.068 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:22.068 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235579+0000", 2024-09-21T15:01:22.068 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6842999999999998e-05 2024-09-21T15:01:22.068 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.068 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.068 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:22.068 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235590+0000", 2024-09-21T15:01:22.068 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1194e-05 2024-09-21T15:01:22.068 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.069 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.069 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:22.069 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.262119+0000", 2024-09-21T15:01:22.069 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0265294540000001 2024-09-21T15:01:22.069 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.069 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.069 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:22.069 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.262241+0000", 2024-09-21T15:01:22.069 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012182 2024-09-21T15:01:22.069 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.069 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.069 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:22.069 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.262403+0000", 2024-09-21T15:01:22.069 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000161442 2024-09-21T15:01:22.070 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:22.070 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:22.070 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:22.070 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.070 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.070 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140221 3.1e 3:7fa3054f:::rbd_data.113664780c85.000000000000001c:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:22.070 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.235592+0000", 2024-09-21T15:01:22.070 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 19.744740309000001, 2024-09-21T15:01:22.070 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.027322012, 2024-09-21T15:01:22.070 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:22.070 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:22.070 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:22.070 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:22.071 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:22.071 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140221 2024-09-21T15:01:22.071 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.071 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:22.071 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.071 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:22.071 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235592+0000", 2024-09-21T15:01:22.071 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:22.071 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.071 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.071 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:22.071 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235592+0000", 2024-09-21T15:01:22.071 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:22.072 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.072 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.072 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:22.072 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235591+0000", 2024-09-21T15:01:22.072 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:22.072 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.072 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.072 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:22.072 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235596+0000", 2024-09-21T15:01:22.072 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4209999999999997e-06 2024-09-21T15:01:22.072 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.072 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.072 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:22.072 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235603+0000", 2024-09-21T15:01:22.073 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.909e-06 2024-09-21T15:01:22.073 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.073 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.073 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:22.073 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235609+0000", 2024-09-21T15:01:22.073 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3539999999999999e-06 2024-09-21T15:01:22.073 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.073 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.073 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:22.073 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236013+0000", 2024-09-21T15:01:22.073 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00040399799999999998 2024-09-21T15:01:22.073 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.073 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.074 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:22.074 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236020+0000", 2024-09-21T15:01:22.074 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6699999999999994e-06 2024-09-21T15:01:22.074 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.074 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.074 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:22.074 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.262452+0000", 2024-09-21T15:01:22.074 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.02643145 2024-09-21T15:01:22.074 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.074 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.074 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:22.074 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.262624+0000", 2024-09-21T15:01:22.074 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000172206 2024-09-21T15:01:22.075 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.075 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.075 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:22.075 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.262914+0000", 2024-09-21T15:01:22.075 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00028994600000000002 2024-09-21T15:01:22.075 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:22.075 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:22.075 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:22.075 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.075 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.075 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140237 3.1e 3:7c803367:::rbd_data.113664780c85.000000000000008c:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:22.075 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.237113+0000", 2024-09-21T15:01:22.075 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 19.743219801999999, 2024-09-21T15:01:22.076 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0260336299999999, 2024-09-21T15:01:22.076 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:22.076 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:22.076 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:22.076 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:22.076 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:22.076 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140237 2024-09-21T15:01:22.076 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.076 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:22.076 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.076 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:22.076 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237113+0000", 2024-09-21T15:01:22.076 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:22.077 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.077 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.077 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:22.077 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237113+0000", 2024-09-21T15:01:22.077 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:22.077 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.077 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.077 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:22.077 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237112+0000", 2024-09-21T15:01:22.077 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:22.077 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.077 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.077 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:22.078 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237118+0000", 2024-09-21T15:01:22.078 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8930000000000002e-06 2024-09-21T15:01:22.078 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.078 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.078 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:22.078 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237126+0000", 2024-09-21T15:01:22.078 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6729999999999998e-06 2024-09-21T15:01:22.078 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.078 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.078 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:22.078 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237135+0000", 2024-09-21T15:01:22.078 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0929999999999992e-06 2024-09-21T15:01:22.079 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.079 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.079 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:22.079 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237205+0000", 2024-09-21T15:01:22.079 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9697999999999996e-05 2024-09-21T15:01:22.079 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.079 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.079 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:22.079 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237218+0000", 2024-09-21T15:01:22.079 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2524000000000001e-05 2024-09-21T15:01:22.079 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.080 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.080 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:22.080 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.262965+0000", 2024-09-21T15:01:22.080 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0257478630000001 2024-09-21T15:01:22.080 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.080 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.080 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:22.080 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.263146+0000", 2024-09-21T15:01:22.080 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018074599999999999 2024-09-21T15:01:22.080 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:22.080 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:22.080 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:22.080 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.081 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.081 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140280 3.1e 3:7b675eeb:::rbd_data.113664780c85.00000000000001b3:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:22.081 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.237477+0000", 2024-09-21T15:01:22.081 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 19.742855540000001, 2024-09-21T15:01:22.081 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.026437329, 2024-09-21T15:01:22.081 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:22.081 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:22.081 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:22.081 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:22.081 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:22.081 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140280 2024-09-21T15:01:22.081 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.081 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:22.082 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.082 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:22.082 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237477+0000", 2024-09-21T15:01:22.082 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:22.082 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.082 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.082 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:22.082 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237477+0000", 2024-09-21T15:01:22.082 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:22.082 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.082 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.082 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:22.083 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237476+0000", 2024-09-21T15:01:22.083 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:22.083 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.083 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.083 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:22.083 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237480+0000", 2024-09-21T15:01:22.083 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7570000000000002e-06 2024-09-21T15:01:22.083 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.083 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.083 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:22.083 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237490+0000", 2024-09-21T15:01:22.083 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3170000000000008e-06 2024-09-21T15:01:22.084 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.084 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.084 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:22.084 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237496+0000", 2024-09-21T15:01:22.084 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7150000000000001e-06 2024-09-21T15:01:22.084 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.084 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.084 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:22.084 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237559+0000", 2024-09-21T15:01:22.084 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2844999999999999e-05 2024-09-21T15:01:22.084 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.084 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.085 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:22.085 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237570+0000", 2024-09-21T15:01:22.085 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1253000000000001e-05 2024-09-21T15:01:22.085 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.085 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.085 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:22.085 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.263203+0000", 2024-09-21T15:01:22.085 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0256330339999999 2024-09-21T15:01:22.085 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.085 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.085 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:22.085 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.263914+0000", 2024-09-21T15:01:22.085 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00071080899999999996 2024-09-21T15:01:22.086 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:22.086 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:22.086 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:22.086 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.086 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.086 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140348 3.1e 3:7b440264:::rbd_data.113664780c85.0000000000000013:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:22.086 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.241762+0000", 2024-09-21T15:01:22.086 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 19.738570269, 2024-09-21T15:01:22.086 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0221355759999999, 2024-09-21T15:01:22.086 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:22.086 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:22.086 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:22.087 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:22.087 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:22.087 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140348 2024-09-21T15:01:22.087 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.087 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:22.087 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.087 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:22.087 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.241762+0000", 2024-09-21T15:01:22.087 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:22.087 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.087 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.088 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:22.088 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.241762+0000", 2024-09-21T15:01:22.088 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:22.088 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.088 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.088 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:22.088 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.241761+0000", 2024-09-21T15:01:22.088 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:22.088 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.088 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.088 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:22.088 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.241767+0000", 2024-09-21T15:01:22.088 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4219999999999997e-06 2024-09-21T15:01:22.089 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.089 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.089 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:22.089 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.241783+0000", 2024-09-21T15:01:22.089 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5959000000000001e-05 2024-09-21T15:01:22.089 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.089 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.089 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:22.089 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.241793+0000", 2024-09-21T15:01:22.089 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3999999999999998e-06 2024-09-21T15:01:22.089 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.089 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.089 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:22.090 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.241894+0000", 2024-09-21T15:01:22.090 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000100976 2024-09-21T15:01:22.090 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.090 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.090 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:22.090 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.241907+0000", 2024-09-21T15:01:22.090 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3281e-05 2024-09-21T15:01:22.090 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.090 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.090 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:22.090 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.263430+0000", 2024-09-21T15:01:22.090 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0215229379999999 2024-09-21T15:01:22.091 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.091 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.091 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:22.091 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.263604+0000", 2024-09-21T15:01:22.091 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017368700000000001 2024-09-21T15:01:22.091 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.091 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.091 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:22.091 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.263898+0000", 2024-09-21T15:01:22.091 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00029406899999999999 2024-09-21T15:01:22.091 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:22.091 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:22.091 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:22.092 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:22.092 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:22.092 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:01:22.093 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T15:01:22.094 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:01:22.090+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:01:22.183 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T15:01:22.211 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:01:22.211 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T15:01:22.212 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.212 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:166488 3.4 3.7bcf3e84 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e189)", 2024-09-21T15:01:22.212 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:21.342023+0000", 2024-09-21T15:01:22.212 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.86820077600000001, 2024-09-21T15:01:22.212 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.86824571800000006, 2024-09-21T15:01:22.212 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:22.212 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "waiting for peered", 2024-09-21T15:01:22.212 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:22.212 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:22.212 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:22.213 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 166488 2024-09-21T15:01:22.213 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.213 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:22.213 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.213 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:22.213 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:21.342023+0000", 2024-09-21T15:01:22.213 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:22.213 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.213 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.213 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:22.213 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:21.342023+0000", 2024-09-21T15:01:22.214 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:22.214 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.214 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.214 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:22.214 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:21.342022+0000", 2024-09-21T15:01:22.214 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:22.214 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.214 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.214 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:22.214 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:21.342029+0000", 2024-09-21T15:01:22.214 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1420000000000004e-06 2024-09-21T15:01:22.215 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.215 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.215 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:22.215 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:21.342051+0000", 2024-09-21T15:01:22.215 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2549999999999999e-05 2024-09-21T15:01:22.215 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.215 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.215 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:22.215 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:21.342059+0000", 2024-09-21T15:01:22.215 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8490000000000006e-06 2024-09-21T15:01:22.215 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.216 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.216 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:22.216 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:21.342105+0000", 2024-09-21T15:01:22.216 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5488999999999998e-05 2024-09-21T15:01:22.216 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.216 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.216 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:22.216 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:21.342115+0000", 2024-09-21T15:01:22.216 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0633e-05 2024-09-21T15:01:22.216 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:22.217 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:22.217 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:22.217 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:22.217 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-21T15:01:22.217 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 1 2024-09-21T15:01:22.217 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:01:22.222 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T15:01:22.229 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:01:22.226+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:01:22.295 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:01:22.290+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:01:22.330 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:01:22.331 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:01:22.331 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T15:01:22.331 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T15:01:22.331 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:01:22.343 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T15:01:22.390 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T15:01:22.396 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:01:22.392+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:01:22.443 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:01:22.443 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T15:01:22.443 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T15:01:22.443 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T15:01:22.443 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.443 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:97677 3.15 e122/24 3:a8e72133:::rbd_data.1136bfc79a70.0000000000000005:head v 122'567, mlcod=120'565)", 2024-09-21T15:01:22.443 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.320753+0000", 2024-09-21T15:01:22.443 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 77.118123447000002, 2024-09-21T15:01:22.444 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070639375000000004, 2024-09-21T15:01:22.444 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:22.444 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:22.444 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:22.444 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.444 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:22.444 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.320753+0000", 2024-09-21T15:01:22.444 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:22.444 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.444 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.444 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:22.444 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.320753+0000", 2024-09-21T15:01:22.444 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:22.444 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.444 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.445 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:22.445 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.320753+0000", 2024-09-21T15:01:22.445 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:22.445 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.445 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.445 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:22.445 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.375452+0000", 2024-09-21T15:01:22.445 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054698945999999998 2024-09-21T15:01:22.445 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.445 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.445 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:22.445 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.375476+0000", 2024-09-21T15:01:22.445 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4380999999999999e-05 2024-09-21T15:01:22.445 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.445 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.445 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:22.446 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.375485+0000", 2024-09-21T15:01:22.446 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0049999999999993e-06 2024-09-21T15:01:22.446 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.446 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.446 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:22.446 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.375552+0000", 2024-09-21T15:01:22.446 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6550999999999994e-05 2024-09-21T15:01:22.446 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.446 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.446 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:22.446 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.375578+0000", 2024-09-21T15:01:22.446 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6196000000000001e-05 2024-09-21T15:01:22.446 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.446 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.446 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:01:22.447 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.391319+0000", 2024-09-21T15:01:22.447 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015741271000000001 2024-09-21T15:01:22.447 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.447 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.447 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:22.447 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.391393+0000", 2024-09-21T15:01:22.447 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3287000000000005e-05 2024-09-21T15:01:22.447 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:22.447 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:22.447 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:22.447 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.447 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.447 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:97672 3.4 e122/51 3:26046d29:::rbd_data.1136bfc79a70.0000000000000001:head v 122'256, mlcod=120'254)", 2024-09-21T15:01:22.447 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.331696+0000", 2024-09-21T15:01:22.447 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 77.107180283999995, 2024-09-21T15:01:22.447 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041222507999999998, 2024-09-21T15:01:22.448 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:22.448 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:22.448 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:22.448 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.448 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:22.448 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.331696+0000", 2024-09-21T15:01:22.448 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:22.448 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.448 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.448 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:22.448 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.331696+0000", 2024-09-21T15:01:22.448 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:22.448 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.448 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.448 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:22.449 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.331695+0000", 2024-09-21T15:01:22.449 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:22.449 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.449 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.449 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:22.449 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.340175+0000", 2024-09-21T15:01:22.449 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0084799520000000007 2024-09-21T15:01:22.449 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.449 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.449 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:22.449 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.340208+0000", 2024-09-21T15:01:22.449 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2997999999999998e-05 2024-09-21T15:01:22.449 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.449 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.449 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:22.449 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.340223+0000", 2024-09-21T15:01:22.450 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5330999999999999e-05 2024-09-21T15:01:22.450 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.450 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.450 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:22.450 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.340305+0000", 2024-09-21T15:01:22.450 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2095999999999995e-05 2024-09-21T15:01:22.450 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.450 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.450 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:22.450 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.340337+0000", 2024-09-21T15:01:22.450 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2097000000000001e-05 2024-09-21T15:01:22.450 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.450 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.450 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:01:22.450 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.372863+0000", 2024-09-21T15:01:22.450 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.032525923999999998 2024-09-21T15:01:22.451 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.451 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.451 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:22.451 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.372919+0000", 2024-09-21T15:01:22.451 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5729000000000001e-05 2024-09-21T15:01:22.451 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:22.451 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:22.451 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:22.451 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.451 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.451 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:116182 3.12 3:4efbaa3f:::rbd_data.1136b1194d20.0000000000000006:58 [read 6242304~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:22.451 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.712538+0000", 2024-09-21T15:01:22.451 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 52.726338292999998, 2024-09-21T15:01:22.451 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034702127999999999, 2024-09-21T15:01:22.452 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:22.452 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:22.452 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:22.452 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:22.452 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:22.452 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116182 2024-09-21T15:01:22.452 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.452 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:22.452 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.452 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:22.452 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712538+0000", 2024-09-21T15:01:22.452 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:22.452 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.452 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.452 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:22.452 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712538+0000", 2024-09-21T15:01:22.453 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:22.453 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.453 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.453 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:22.453 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712538+0000", 2024-09-21T15:01:22.453 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:22.453 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.453 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.453 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:22.453 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712541+0000", 2024-09-21T15:01:22.453 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5330000000000001e-06 2024-09-21T15:01:22.453 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.453 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.453 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:22.453 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712547+0000", 2024-09-21T15:01:22.454 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1109999999999996e-06 2024-09-21T15:01:22.454 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.454 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.454 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:22.454 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712551+0000", 2024-09-21T15:01:22.454 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.685e-06 2024-09-21T15:01:22.454 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.454 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.454 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:22.454 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.747067+0000", 2024-09-21T15:01:22.454 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034516181 2024-09-21T15:01:22.454 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.454 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.454 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:22.454 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.747152+0000", 2024-09-21T15:01:22.455 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4075999999999999e-05 2024-09-21T15:01:22.455 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.455 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.455 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:22.455 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.747240+0000", 2024-09-21T15:01:22.455 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8924000000000004e-05 2024-09-21T15:01:22.455 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:22.455 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:22.455 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:22.455 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.455 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.455 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:118701 3.12 3:4efbaa3f:::rbd_data.1136b1194d20.0000000000000006:58 [read 10412032~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:22.455 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.902645+0000", 2024-09-21T15:01:22.455 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 52.536231919000002, 2024-09-21T15:01:22.455 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034600791999999998, 2024-09-21T15:01:22.456 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:22.456 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:22.456 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:22.456 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:22.456 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:22.456 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 118701 2024-09-21T15:01:22.456 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.456 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:22.456 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.456 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:22.456 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902645+0000", 2024-09-21T15:01:22.456 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:22.456 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.456 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.456 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:22.456 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902645+0000", 2024-09-21T15:01:22.457 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:22.457 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.457 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.457 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:22.457 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902644+0000", 2024-09-21T15:01:22.457 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:22.457 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.457 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.457 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:22.457 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902646+0000", 2024-09-21T15:01:22.457 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6169999999999999e-06 2024-09-21T15:01:22.457 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.457 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.457 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:22.457 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902649+0000", 2024-09-21T15:01:22.458 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9450000000000002e-06 2024-09-21T15:01:22.458 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.458 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.458 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:22.458 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.902652+0000", 2024-09-21T15:01:22.458 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.7889999999999999e-06 2024-09-21T15:01:22.458 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.458 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.458 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:22.458 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.937086+0000", 2024-09-21T15:01:22.458 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034433832999999997 2024-09-21T15:01:22.458 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.458 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.458 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:22.458 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.937152+0000", 2024-09-21T15:01:22.459 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5934999999999998e-05 2024-09-21T15:01:22.459 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.459 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.459 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:22.459 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.937246+0000", 2024-09-21T15:01:22.459 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3923000000000002e-05 2024-09-21T15:01:22.459 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:22.459 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:22.459 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:22.459 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.459 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.459 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139784 3.1 3:862e0e65:::rbd_data.113664780c85.0000000000000063:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:22.459 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.222486+0000", 2024-09-21T15:01:22.459 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 20.216390426, 2024-09-21T15:01:22.459 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0141793059999999, 2024-09-21T15:01:22.459 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:22.460 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:22.460 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:22.460 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:22.460 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:22.460 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139784 2024-09-21T15:01:22.460 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.460 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:22.460 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.460 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:22.460 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222486+0000", 2024-09-21T15:01:22.460 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:22.460 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.460 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.460 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:22.460 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222486+0000", 2024-09-21T15:01:22.460 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:22.461 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.461 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.461 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:22.461 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222484+0000", 2024-09-21T15:01:22.461 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-21T15:01:22.461 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.461 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.461 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:22.461 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222494+0000", 2024-09-21T15:01:22.461 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0676000000000001e-05 2024-09-21T15:01:22.461 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.461 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.461 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:22.461 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222517+0000", 2024-09-21T15:01:22.461 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2371e-05 2024-09-21T15:01:22.462 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.462 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.462 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:22.462 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222527+0000", 2024-09-21T15:01:22.462 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0461e-05 2024-09-21T15:01:22.462 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.462 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.462 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:22.462 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222625+0000", 2024-09-21T15:01:22.462 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.8160999999999997e-05 2024-09-21T15:01:22.462 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.462 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.462 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:22.462 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222641+0000", 2024-09-21T15:01:22.462 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5899000000000001e-05 2024-09-21T15:01:22.463 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.463 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.463 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:22.463 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.236479+0000", 2024-09-21T15:01:22.463 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0138377709999999 2024-09-21T15:01:22.463 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.463 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.463 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:22.463 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.236666+0000", 2024-09-21T15:01:22.463 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018648199999999999 2024-09-21T15:01:22.463 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:22.463 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:22.463 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:22.463 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.463 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.464 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139812 3.1 3:8407fb9b:::rbd_data.113664780c85.0000000000000127:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:22.464 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.222547+0000", 2024-09-21T15:01:22.464 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 20.216329502000001, 2024-09-21T15:01:22.464 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.014796059, 2024-09-21T15:01:22.464 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:22.464 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:22.464 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:22.464 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:22.464 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:22.464 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139812 2024-09-21T15:01:22.464 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.464 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:22.464 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.464 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:22.464 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222547+0000", 2024-09-21T15:01:22.465 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:22.465 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.465 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.465 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:22.465 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222547+0000", 2024-09-21T15:01:22.465 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:22.465 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.465 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.465 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:22.465 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222546+0000", 2024-09-21T15:01:22.465 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:22.465 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.465 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.465 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:22.465 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222552+0000", 2024-09-21T15:01:22.465 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2739999999999998e-06 2024-09-21T15:01:22.466 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.466 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.466 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:22.466 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222578+0000", 2024-09-21T15:01:22.466 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6594999999999999e-05 2024-09-21T15:01:22.466 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.466 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.466 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:22.466 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222596+0000", 2024-09-21T15:01:22.466 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7838000000000001e-05 2024-09-21T15:01:22.466 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.466 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.466 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:22.466 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222701+0000", 2024-09-21T15:01:22.466 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000104714 2024-09-21T15:01:22.466 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.467 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.467 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:22.467 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222714+0000", 2024-09-21T15:01:22.467 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2998e-05 2024-09-21T15:01:22.467 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.467 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.467 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:22.467 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.237124+0000", 2024-09-21T15:01:22.467 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.014409806 2024-09-21T15:01:22.467 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.467 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.467 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:22.467 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.237343+0000", 2024-09-21T15:01:22.467 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021950499999999999 2024-09-21T15:01:22.467 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:22.467 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:22.468 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:22.468 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.468 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.468 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139826 3.1 3:854f6f9b:::rbd_data.113664780c85.0000000000000189:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:22.468 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.222611+0000", 2024-09-21T15:01:22.468 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 20.216265409999998, 2024-09-21T15:01:22.468 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.014916828, 2024-09-21T15:01:22.468 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:22.468 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:22.468 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:22.468 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:22.468 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:22.468 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139826 2024-09-21T15:01:22.468 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.468 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:22.469 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.469 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:22.469 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222611+0000", 2024-09-21T15:01:22.469 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:22.469 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.469 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.469 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:22.469 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222611+0000", 2024-09-21T15:01:22.469 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:22.469 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.469 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.469 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:22.469 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222610+0000", 2024-09-21T15:01:22.469 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:22.469 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.469 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.470 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:22.470 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222618+0000", 2024-09-21T15:01:22.470 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6690000000000006e-06 2024-09-21T15:01:22.470 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.470 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.470 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:22.470 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222632+0000", 2024-09-21T15:01:22.470 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4433999999999999e-05 2024-09-21T15:01:22.470 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.470 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.470 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:22.470 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222640+0000", 2024-09-21T15:01:22.470 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6260000000000003e-06 2024-09-21T15:01:22.470 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.470 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.471 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:22.471 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222758+0000", 2024-09-21T15:01:22.471 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011826499999999999 2024-09-21T15:01:22.471 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.471 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.471 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:22.471 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222776+0000", 2024-09-21T15:01:22.471 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7564999999999999e-05 2024-09-21T15:01:22.471 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.471 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.471 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:22.471 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.237402+0000", 2024-09-21T15:01:22.471 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0146261889999999 2024-09-21T15:01:22.471 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.471 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.471 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:22.472 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.237528+0000", 2024-09-21T15:01:22.472 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000126212 2024-09-21T15:01:22.472 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:22.472 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:22.472 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:22.472 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.472 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.472 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139864 3.1 3:86b14cb7:::rbd_data.113664780c85.00000000000000d5:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:22.472 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.223966+0000", 2024-09-21T15:01:22.472 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 20.214910673999999, 2024-09-21T15:01:22.472 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.013859007, 2024-09-21T15:01:22.472 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:22.472 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:22.472 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:22.472 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:22.472 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:22.473 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139864 2024-09-21T15:01:22.473 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.473 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:22.473 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.473 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:22.473 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223966+0000", 2024-09-21T15:01:22.473 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:22.473 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.473 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.473 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:22.473 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223966+0000", 2024-09-21T15:01:22.473 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:22.473 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.473 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.473 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:22.474 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223965+0000", 2024-09-21T15:01:22.474 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:22.474 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.474 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.474 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:22.474 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223970+0000", 2024-09-21T15:01:22.474 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3769999999999998e-06 2024-09-21T15:01:22.474 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.474 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.474 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:22.474 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223979+0000", 2024-09-21T15:01:22.474 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5899999999999997e-06 2024-09-21T15:01:22.474 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.474 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.474 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:22.475 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223985+0000", 2024-09-21T15:01:22.475 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5790000000000003e-06 2024-09-21T15:01:22.475 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.475 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.475 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:22.475 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224073+0000", 2024-09-21T15:01:22.475 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7781000000000005e-05 2024-09-21T15:01:22.475 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.475 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.475 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:22.475 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224110+0000", 2024-09-21T15:01:22.475 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7060000000000001e-05 2024-09-21T15:01:22.475 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.475 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.475 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:22.476 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.237599+0000", 2024-09-21T15:01:22.476 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.01348953 2024-09-21T15:01:22.476 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.476 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.476 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:22.476 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.237825+0000", 2024-09-21T15:01:22.476 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000225758 2024-09-21T15:01:22.476 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:22.476 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:22.476 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:22.476 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.476 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.476 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139877 3.1 3:87277f92:::rbd_data.113664780c85.0000000000000136:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:22.476 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.224489+0000", 2024-09-21T15:01:22.476 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 20.214387815999999, 2024-09-21T15:01:22.476 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0137916330000001, 2024-09-21T15:01:22.477 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:22.477 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:22.477 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:22.477 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:22.477 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:22.477 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139877 2024-09-21T15:01:22.477 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.477 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:22.477 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.477 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:22.477 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224489+0000", 2024-09-21T15:01:22.477 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:22.477 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.477 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.477 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:22.477 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224489+0000", 2024-09-21T15:01:22.478 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:22.478 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.478 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.478 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:22.478 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224488+0000", 2024-09-21T15:01:22.478 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:22.478 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.478 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.478 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:22.478 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224502+0000", 2024-09-21T15:01:22.478 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4194e-05 2024-09-21T15:01:22.478 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.478 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.478 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:22.478 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224522+0000", 2024-09-21T15:01:22.478 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0038e-05 2024-09-21T15:01:22.479 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.479 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.479 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:22.479 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224531+0000", 2024-09-21T15:01:22.479 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6900000000000004e-06 2024-09-21T15:01:22.479 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.479 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.479 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:22.479 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224621+0000", 2024-09-21T15:01:22.479 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9654000000000006e-05 2024-09-21T15:01:22.479 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.479 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.479 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:22.479 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224641+0000", 2024-09-21T15:01:22.480 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9375e-05 2024-09-21T15:01:22.480 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.480 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.480 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:22.480 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.238027+0000", 2024-09-21T15:01:22.480 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0133866840000001 2024-09-21T15:01:22.480 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.480 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.480 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:22.480 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.238280+0000", 2024-09-21T15:01:22.480 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000253281 2024-09-21T15:01:22.480 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:22.480 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:22.480 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:22.480 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.480 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.481 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139887 3.1 3:80f1e8f1:::rbd_data.113664780c85.000000000000017c:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:22.481 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.224549+0000", 2024-09-21T15:01:22.481 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 20.214327526000002, 2024-09-21T15:01:22.481 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0139776060000001, 2024-09-21T15:01:22.481 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:22.481 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:22.481 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:22.481 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:22.481 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:22.481 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139887 2024-09-21T15:01:22.481 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.481 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:22.481 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.481 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:22.481 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224549+0000", 2024-09-21T15:01:22.482 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:22.482 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.482 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.482 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:22.482 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224549+0000", 2024-09-21T15:01:22.482 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:22.482 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.482 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.482 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:22.482 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224548+0000", 2024-09-21T15:01:22.482 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:22.482 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.482 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.482 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:22.482 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224554+0000", 2024-09-21T15:01:22.483 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4049999999999999e-06 2024-09-21T15:01:22.483 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.483 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.483 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:22.483 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224565+0000", 2024-09-21T15:01:22.483 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0879e-05 2024-09-21T15:01:22.483 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.483 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.483 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:22.483 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224572+0000", 2024-09-21T15:01:22.483 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0380000000000002e-06 2024-09-21T15:01:22.483 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.483 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.483 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:22.484 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224684+0000", 2024-09-21T15:01:22.484 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011229 2024-09-21T15:01:22.484 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.484 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.484 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:22.484 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224698+0000", 2024-09-21T15:01:22.484 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4448e-05 2024-09-21T15:01:22.484 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.484 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.484 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:22.484 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.238359+0000", 2024-09-21T15:01:22.484 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0136610660000001 2024-09-21T15:01:22.484 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.484 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.484 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:22.485 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.238527+0000", 2024-09-21T15:01:22.485 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016731100000000001 2024-09-21T15:01:22.485 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:22.485 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:22.485 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:22.485 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.485 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.485 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140077 3.1 3:85642daf:::rbd_data.113664780c85.0000000000000174:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:22.485 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.232797+0000", 2024-09-21T15:01:22.485 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 20.206079810999999, 2024-09-21T15:01:22.485 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.006029955, 2024-09-21T15:01:22.485 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:22.485 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:22.485 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:22.485 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:22.485 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:22.486 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140077 2024-09-21T15:01:22.486 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.486 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:22.486 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.486 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:22.486 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232797+0000", 2024-09-21T15:01:22.486 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:22.486 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.486 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.486 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:22.486 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232797+0000", 2024-09-21T15:01:22.486 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:22.486 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.486 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.486 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:22.487 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232795+0000", 2024-09-21T15:01:22.487 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:22.487 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.487 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.487 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:22.487 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232804+0000", 2024-09-21T15:01:22.487 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9670000000000007e-06 2024-09-21T15:01:22.487 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.487 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.487 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:22.487 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232824+0000", 2024-09-21T15:01:22.487 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9238000000000001e-05 2024-09-21T15:01:22.487 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.487 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.487 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:22.488 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232843+0000", 2024-09-21T15:01:22.488 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9256e-05 2024-09-21T15:01:22.488 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.488 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.488 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:22.488 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232912+0000", 2024-09-21T15:01:22.488 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9589000000000001e-05 2024-09-21T15:01:22.488 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.488 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.488 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:22.488 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232925+0000", 2024-09-21T15:01:22.488 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2585999999999999e-05 2024-09-21T15:01:22.488 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.488 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.488 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:22.488 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.238591+0000", 2024-09-21T15:01:22.489 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0056659939999999 2024-09-21T15:01:22.489 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.489 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.489 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:22.489 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.238827+0000", 2024-09-21T15:01:22.489 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00023583999999999999 2024-09-21T15:01:22.489 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:22.489 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:22.489 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:22.489 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.489 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.489 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140091 3.1 3:8358f0db:::rbd_data.113664780c85.0000000000000018:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:22.489 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.233637+0000", 2024-09-21T15:01:22.489 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 20.205239514999999, 2024-09-21T15:01:22.489 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.006272847, 2024-09-21T15:01:22.489 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:22.490 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:22.490 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:22.490 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:22.490 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:22.490 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140091 2024-09-21T15:01:22.490 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.490 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:22.490 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.490 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:22.490 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233637+0000", 2024-09-21T15:01:22.490 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:22.490 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.490 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.490 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:22.490 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233637+0000", 2024-09-21T15:01:22.491 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:22.491 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.491 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.491 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:22.491 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233636+0000", 2024-09-21T15:01:22.491 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:22.491 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.491 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.491 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:22.491 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233644+0000", 2024-09-21T15:01:22.491 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1380000000000007e-06 2024-09-21T15:01:22.491 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.491 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.491 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:22.492 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233662+0000", 2024-09-21T15:01:22.492 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8224999999999999e-05 2024-09-21T15:01:22.492 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.492 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.492 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:22.492 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233674+0000", 2024-09-21T15:01:22.492 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2081e-05 2024-09-21T15:01:22.492 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.492 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.492 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:22.492 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233750+0000", 2024-09-21T15:01:22.492 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6106000000000006e-05 2024-09-21T15:01:22.492 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.492 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.493 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:22.493 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233768+0000", 2024-09-21T15:01:22.493 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7285000000000001e-05 2024-09-21T15:01:22.493 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.493 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.493 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:22.493 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.239196+0000", 2024-09-21T15:01:22.493 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.005428223 2024-09-21T15:01:22.493 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.493 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.493 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:22.493 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.239528+0000", 2024-09-21T15:01:22.493 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00033262199999999997 2024-09-21T15:01:22.493 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.493 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.493 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:22.494 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.239910+0000", 2024-09-21T15:01:22.494 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00038151100000000001 2024-09-21T15:01:22.494 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:22.494 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:22.494 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:22.494 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.494 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.494 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140109 3.1 3:8586e9d9:::rbd_data.113664780c85.0000000000000096:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:22.494 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.234171+0000", 2024-09-21T15:01:22.494 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 20.204705681, 2024-09-21T15:01:22.494 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.006019896, 2024-09-21T15:01:22.494 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:22.494 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:22.495 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:22.495 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:22.495 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:22.495 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140109 2024-09-21T15:01:22.495 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.495 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:22.495 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.495 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:22.495 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.234171+0000", 2024-09-21T15:01:22.495 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:22.495 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.495 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.495 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:22.495 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.234171+0000", 2024-09-21T15:01:22.495 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:22.496 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.496 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.496 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:22.496 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.234170+0000", 2024-09-21T15:01:22.496 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:22.496 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.496 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.496 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:22.496 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.234178+0000", 2024-09-21T15:01:22.496 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.464e-06 2024-09-21T15:01:22.496 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.496 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.496 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:22.496 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.234191+0000", 2024-09-21T15:01:22.496 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3489000000000001e-05 2024-09-21T15:01:22.497 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.497 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.497 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:22.497 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.234200+0000", 2024-09-21T15:01:22.497 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2779999999999999e-06 2024-09-21T15:01:22.497 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.497 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.497 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:22.497 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.234279+0000", 2024-09-21T15:01:22.497 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8813000000000006e-05 2024-09-21T15:01:22.497 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.497 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.497 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:22.497 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.234298+0000", 2024-09-21T15:01:22.497 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8947999999999999e-05 2024-09-21T15:01:22.498 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.498 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.498 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:22.498 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.239974+0000", 2024-09-21T15:01:22.498 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0056760259999999 2024-09-21T15:01:22.498 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.498 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.498 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:22.498 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.240191+0000", 2024-09-21T15:01:22.498 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000216708 2024-09-21T15:01:22.498 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:22.498 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:22.498 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:22.498 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.498 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.499 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140149 3.1 3:86b1e433:::rbd_data.113664780c85.00000000000001ae:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:22.499 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.235545+0000", 2024-09-21T15:01:22.499 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 20.203331395999999, 2024-09-21T15:01:22.499 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0049193430000001, 2024-09-21T15:01:22.499 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:22.499 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:22.499 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:22.499 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:22.499 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:22.499 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140149 2024-09-21T15:01:22.499 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.499 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:22.499 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.499 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:22.499 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235545+0000", 2024-09-21T15:01:22.499 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:22.500 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.500 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.500 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:22.500 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235545+0000", 2024-09-21T15:01:22.500 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:22.500 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.500 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.500 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:22.500 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235544+0000", 2024-09-21T15:01:22.500 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:22.500 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.500 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.500 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:22.500 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235554+0000", 2024-09-21T15:01:22.500 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.8870000000000004e-06 2024-09-21T15:01:22.501 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.501 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.501 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:22.501 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235577+0000", 2024-09-21T15:01:22.501 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2946999999999999e-05 2024-09-21T15:01:22.501 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.501 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.501 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:22.501 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235587+0000", 2024-09-21T15:01:22.501 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0326000000000001e-05 2024-09-21T15:01:22.501 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.501 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.501 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:22.501 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235658+0000", 2024-09-21T15:01:22.501 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1360999999999997e-05 2024-09-21T15:01:22.501 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.502 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.502 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:22.502 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235670+0000", 2024-09-21T15:01:22.502 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1735e-05 2024-09-21T15:01:22.502 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.502 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.502 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:22.502 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.240246+0000", 2024-09-21T15:01:22.502 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0045757790000001 2024-09-21T15:01:22.502 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.502 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.502 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:22.502 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.240465+0000", 2024-09-21T15:01:22.502 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021870799999999999 2024-09-21T15:01:22.502 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:22.502 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:22.503 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:22.503 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.503 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.503 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140232 3.1 3:862e0e65:::rbd_data.113664780c85.0000000000000063:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:22.503 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.235760+0000", 2024-09-21T15:01:22.503 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 20.203116678000001, 2024-09-21T15:01:22.503 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.004979812, 2024-09-21T15:01:22.503 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:22.503 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:22.503 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:22.503 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:22.503 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:22.503 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140232 2024-09-21T15:01:22.503 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.503 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:22.504 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.504 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:22.504 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235760+0000", 2024-09-21T15:01:22.504 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:22.504 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.504 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.504 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:22.504 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235760+0000", 2024-09-21T15:01:22.504 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:22.504 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.504 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.504 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:22.504 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235759+0000", 2024-09-21T15:01:22.504 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:22.505 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.505 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.505 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:22.505 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235777+0000", 2024-09-21T15:01:22.505 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8215000000000001e-05 2024-09-21T15:01:22.505 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.505 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.505 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:22.505 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235793+0000", 2024-09-21T15:01:22.505 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6461e-05 2024-09-21T15:01:22.505 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.505 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.505 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:22.505 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235803+0000", 2024-09-21T15:01:22.505 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5300000000000002e-06 2024-09-21T15:01:22.505 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.506 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.506 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:22.506 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235875+0000", 2024-09-21T15:01:22.506 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2113000000000006e-05 2024-09-21T15:01:22.506 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.506 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.506 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:22.506 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235887+0000", 2024-09-21T15:01:22.506 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2160999999999999e-05 2024-09-21T15:01:22.506 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.506 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.506 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:22.506 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.240521+0000", 2024-09-21T15:01:22.506 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.004633428 2024-09-21T15:01:22.506 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.507 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.507 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:22.507 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.240740+0000", 2024-09-21T15:01:22.507 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000219228 2024-09-21T15:01:22.507 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:22.507 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:22.507 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:22.507 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.507 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.507 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140260 3.1 3:8407fb9b:::rbd_data.113664780c85.0000000000000127:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:22.507 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.235839+0000", 2024-09-21T15:01:22.507 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 20.203037509000001, 2024-09-21T15:01:22.507 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.005157981, 2024-09-21T15:01:22.507 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:22.507 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:22.508 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:22.508 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:22.508 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:22.508 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140260 2024-09-21T15:01:22.508 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.508 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:22.508 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.508 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:22.508 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235839+0000", 2024-09-21T15:01:22.508 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:22.508 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.508 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.508 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:22.508 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235839+0000", 2024-09-21T15:01:22.508 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:22.509 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.509 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.509 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:22.509 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235838+0000", 2024-09-21T15:01:22.509 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:22.509 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.509 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.509 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:22.509 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235843+0000", 2024-09-21T15:01:22.509 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.481e-06 2024-09-21T15:01:22.509 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.509 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.509 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:22.509 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235849+0000", 2024-09-21T15:01:22.509 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4880000000000004e-06 2024-09-21T15:01:22.509 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.510 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.510 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:22.510 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235856+0000", 2024-09-21T15:01:22.510 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7649999999999996e-06 2024-09-21T15:01:22.510 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.510 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.510 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:22.510 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235933+0000", 2024-09-21T15:01:22.510 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6987e-05 2024-09-21T15:01:22.510 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.510 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.510 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:22.510 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235949+0000", 2024-09-21T15:01:22.510 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6696000000000001e-05 2024-09-21T15:01:22.510 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.511 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.511 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:22.511 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.240823+0000", 2024-09-21T15:01:22.511 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.004873259 2024-09-21T15:01:22.511 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.511 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.511 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:22.511 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.240997+0000", 2024-09-21T15:01:22.511 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000174365 2024-09-21T15:01:22.511 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:22.511 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:22.511 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:22.511 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.511 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.511 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140274 3.1 3:854f6f9b:::rbd_data.113664780c85.0000000000000189:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:22.511 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.235884+0000", 2024-09-21T15:01:22.512 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 20.202992560999999, 2024-09-21T15:01:22.512 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.005397366, 2024-09-21T15:01:22.512 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:22.512 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:22.512 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:22.512 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:22.512 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:22.512 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140274 2024-09-21T15:01:22.512 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.512 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:22.512 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.512 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:22.512 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235884+0000", 2024-09-21T15:01:22.512 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:22.512 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.512 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.513 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:22.513 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235884+0000", 2024-09-21T15:01:22.513 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:22.513 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.513 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.513 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:22.513 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235883+0000", 2024-09-21T15:01:22.513 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:22.513 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.513 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.513 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:22.513 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235891+0000", 2024-09-21T15:01:22.513 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9740000000000002e-06 2024-09-21T15:01:22.513 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.513 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.514 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:22.514 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235900+0000", 2024-09-21T15:01:22.514 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6659999999999997e-06 2024-09-21T15:01:22.514 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.514 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.514 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:22.514 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235909+0000", 2024-09-21T15:01:22.514 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3400000000000004e-06 2024-09-21T15:01:22.514 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.514 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.514 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:22.514 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236012+0000", 2024-09-21T15:01:22.514 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010240500000000001 2024-09-21T15:01:22.514 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.515 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.515 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:22.515 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236027+0000", 2024-09-21T15:01:22.515 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5418e-05 2024-09-21T15:01:22.515 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.515 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.515 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:22.515 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.241068+0000", 2024-09-21T15:01:22.515 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0050410729999999 2024-09-21T15:01:22.515 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.515 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.515 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:22.515 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.241281+0000", 2024-09-21T15:01:22.515 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021321900000000001 2024-09-21T15:01:22.516 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:22.516 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:22.516 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:22.516 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.516 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.516 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140312 3.1 3:86b14cb7:::rbd_data.113664780c85.00000000000000d5:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:22.516 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.236769+0000", 2024-09-21T15:01:22.516 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 20.202108097, 2024-09-21T15:01:22.516 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.004717198, 2024-09-21T15:01:22.516 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:22.516 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:22.516 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:22.516 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:22.516 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:22.516 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140312 2024-09-21T15:01:22.516 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.517 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:22.517 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.517 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:22.517 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236769+0000", 2024-09-21T15:01:22.517 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:22.517 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.517 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.517 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:22.517 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236769+0000", 2024-09-21T15:01:22.517 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:22.517 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.517 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.517 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:22.517 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236768+0000", 2024-09-21T15:01:22.517 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:22.517 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.517 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.518 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:22.518 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236771+0000", 2024-09-21T15:01:22.518 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.331e-06 2024-09-21T15:01:22.518 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.518 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.518 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:22.518 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236777+0000", 2024-09-21T15:01:22.518 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6949999999999997e-06 2024-09-21T15:01:22.518 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.518 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.518 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:22.518 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236787+0000", 2024-09-21T15:01:22.518 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6700000000000006e-06 2024-09-21T15:01:22.518 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.518 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.519 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:22.519 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236846+0000", 2024-09-21T15:01:22.519 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.944e-05 2024-09-21T15:01:22.519 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.519 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.519 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:22.519 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236861+0000", 2024-09-21T15:01:22.519 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4611e-05 2024-09-21T15:01:22.519 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.519 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.519 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:22.519 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.241349+0000", 2024-09-21T15:01:22.519 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0044879250000001 2024-09-21T15:01:22.519 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.519 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.519 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:22.520 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.241486+0000", 2024-09-21T15:01:22.520 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013713000000000001 2024-09-21T15:01:22.520 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:22.520 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:22.520 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:22.520 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.520 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.520 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140325 3.1 3:87277f92:::rbd_data.113664780c85.0000000000000136:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:22.520 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.239351+0000", 2024-09-21T15:01:22.520 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 20.199525957999999, 2024-09-21T15:01:22.520 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0024334690000001, 2024-09-21T15:01:22.520 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:22.520 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:22.520 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:22.520 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:22.521 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:22.521 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140325 2024-09-21T15:01:22.521 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.521 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:22.521 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.521 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:22.521 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239351+0000", 2024-09-21T15:01:22.521 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:22.521 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.521 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.521 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:22.521 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239351+0000", 2024-09-21T15:01:22.521 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:22.521 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.521 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.521 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:22.522 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239349+0000", 2024-09-21T15:01:22.522 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:22.522 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.522 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.522 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:22.522 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239358+0000", 2024-09-21T15:01:22.522 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2830000000000001e-06 2024-09-21T15:01:22.522 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.522 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.522 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:22.522 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239379+0000", 2024-09-21T15:01:22.522 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1780000000000002e-05 2024-09-21T15:01:22.522 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.522 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.522 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:22.523 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239389+0000", 2024-09-21T15:01:22.523 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0056e-05 2024-09-21T15:01:22.523 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.523 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.523 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:22.523 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239484+0000", 2024-09-21T15:01:22.523 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4611000000000006e-05 2024-09-21T15:01:22.523 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.523 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.523 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:22.523 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239497+0000", 2024-09-21T15:01:22.523 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2909e-05 2024-09-21T15:01:22.523 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.523 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.523 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:22.524 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.241559+0000", 2024-09-21T15:01:22.524 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0020622459999999 2024-09-21T15:01:22.524 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.524 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.524 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:22.524 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.241784+0000", 2024-09-21T15:01:22.524 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000225009 2024-09-21T15:01:22.524 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:22.524 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:22.524 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:22.524 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.524 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.524 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140335 3.1 3:80f1e8f1:::rbd_data.113664780c85.000000000000017c:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:22.524 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.239409+0000", 2024-09-21T15:01:22.524 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 20.199467239000001, 2024-09-21T15:01:22.525 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0025803150000001, 2024-09-21T15:01:22.525 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:22.525 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:22.525 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:22.525 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:22.525 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:22.525 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140335 2024-09-21T15:01:22.525 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.525 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:22.525 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.525 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:22.525 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239409+0000", 2024-09-21T15:01:22.525 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:22.525 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.525 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.525 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:22.526 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239409+0000", 2024-09-21T15:01:22.526 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:22.526 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.526 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.526 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:22.526 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239409+0000", 2024-09-21T15:01:22.526 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:22.526 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.526 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.526 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:22.526 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239413+0000", 2024-09-21T15:01:22.526 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7620000000000003e-06 2024-09-21T15:01:22.526 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.526 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.526 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:22.527 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239436+0000", 2024-09-21T15:01:22.527 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2889000000000001e-05 2024-09-21T15:01:22.527 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.527 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.527 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:22.527 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239444+0000", 2024-09-21T15:01:22.527 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6809999999999998e-06 2024-09-21T15:01:22.527 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.527 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.527 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:22.527 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239542+0000", 2024-09-21T15:01:22.527 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7868999999999994e-05 2024-09-21T15:01:22.527 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.527 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.527 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:22.527 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239558+0000", 2024-09-21T15:01:22.528 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6257e-05 2024-09-21T15:01:22.528 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.528 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.528 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:22.528 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.241847+0000", 2024-09-21T15:01:22.528 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.002288533 2024-09-21T15:01:22.528 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:22.528 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:22.528 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:22.528 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.241990+0000", 2024-09-21T15:01:22.528 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014308300000000001 2024-09-21T15:01:22.528 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:22.528 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:22.528 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:22.528 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:22.529 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:22.529 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:01:22.530 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T15:01:22.530 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:01:22.524+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:01:22.536 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T15:01:22.555 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T15:01:22.574 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T15:01:22.598 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T15:01:22.620 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-21T15:01:22.631 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:01:22.626+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:01:22.645 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T15:01:22.669 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T15:01:22.684 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:01:22.684 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:01:22.685 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T15:01:22.685 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:01:22.695 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T15:01:22.696 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T15:01:22.699 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:01:22.699 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:01:22.700 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:01:22.700 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:01:22.700 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:01:22.700 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:01:22.700 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:01:22.700 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:01:22.732 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:01:22.726+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:01:22.805 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:22.805 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:22.815 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T15:01:22.833 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:01:22.826+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:01:22.924 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:22.924 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:22.934 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:01:22.930+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:01:22.935 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T15:01:23.035 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:01:23.032+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:01:23.058 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:23.058 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:23.068 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T15:01:23.135 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:01:23.130+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:01:23.171 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:23.172 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:23.181 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T15:01:23.236 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:01:23.232+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:01:23.293 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:23.294 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:23.303 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T15:01:23.337 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:01:23.332+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:01:23.360 INFO:teuthology.orchestra.run.smithi032.stderr:noscrub is unset 2024-09-21T15:01:23.438 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:01:23.434+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:01:23.455 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:23.455 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:23.468 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T15:01:23.538 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:01:23.532+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:01:23.596 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:23.596 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:23.610 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T15:01:23.639 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:01:23.634+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:01:23.721 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:23.721 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:23.734 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T15:01:23.739 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:01:23.734+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:01:23.841 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:01:23.836+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:01:23.874 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:23.874 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:23.887 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T15:01:23.942 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:01:23.936+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:01:24.043 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:01:24.036+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:01:24.061 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:24.061 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:24.072 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T15:01:24.144 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:01:24.138+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:01:24.187 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:24.187 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:24.197 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T15:01:24.245 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:01:24.240+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:01:24.346 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:01:24.342+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:01:24.368 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:24.369 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:24.381 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T15:01:24.447 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:01:24.442+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:01:24.483 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:24.484 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:24.493 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T15:01:24.548 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:01:24.542+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:01:24.597 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:24.597 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:24.607 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T15:01:24.648 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:01:24.642+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:01:24.712 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T15:01:24.724 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:24.724 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:24.734 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T15:01:24.749 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:01:24.744+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:01:24.840 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:24.840 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:24.851 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T15:01:24.851 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:01:24.846+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:01:24.858 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T15:01:24.879 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T15:01:24.899 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T15:01:24.919 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T15:01:24.940 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-21T15:01:24.951 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:01:24.948+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:01:24.963 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:01:24.963 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T15:01:24.963 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T15:01:24.963 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T15:01:24.963 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.963 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:63820 3.19 3.626bc079 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-21T15:01:24.963 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:27.241293+0000", 2024-09-21T15:01:24.963 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 117.71738313500001, 2024-09-21T15:01:24.964 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.98723808199999996, 2024-09-21T15:01:24.964 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:24.964 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:24.964 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:24.964 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:24.964 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:24.964 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 63820 2024-09-21T15:01:24.964 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.964 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:24.964 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.964 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:24.964 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241293+0000", 2024-09-21T15:01:24.964 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:24.964 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.964 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.965 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:24.965 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241293+0000", 2024-09-21T15:01:24.965 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:24.965 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.965 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.965 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:24.965 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241291+0000", 2024-09-21T15:01:24.965 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:01:24.965 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.965 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.965 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:24.965 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241299+0000", 2024-09-21T15:01:24.965 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1529999999999997e-06 2024-09-21T15:01:24.965 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.965 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.965 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:24.966 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241317+0000", 2024-09-21T15:01:24.966 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7764000000000001e-05 2024-09-21T15:01:24.966 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.966 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.966 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:24.966 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241328+0000", 2024-09-21T15:01:24.966 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1245e-05 2024-09-21T15:01:24.966 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.966 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.966 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:24.966 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241407+0000", 2024-09-21T15:01:24.966 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8762000000000002e-05 2024-09-21T15:01:24.966 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.967 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.967 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:24.967 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241418+0000", 2024-09-21T15:01:24.967 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1695000000000001e-05 2024-09-21T15:01:24.967 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.967 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.967 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:24.967 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:28.228510+0000", 2024-09-21T15:01:24.967 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.98709166100000001 2024-09-21T15:01:24.967 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.967 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.967 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:24.967 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:28.228531+0000", 2024-09-21T15:01:24.967 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0696e-05 2024-09-21T15:01:24.967 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:24.967 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:24.968 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:24.968 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.968 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.968 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:97734 3.19 3:9fcf754f:::rbd_data.1136bfc79a70.0000000000000009:head [write 0~4194304 [fadvise_sequential+fadvise_nocache],write 4784128~147456 [fadvise_sequential+fadvise_nocache],write 4997120~147456 [fadvise_sequential+fadvise_nocache],write 5210112~32768 [fadvise_sequential+fadvise_nocache],write 5292032~147456 [fadvise_sequential+fadvise_nocache],write 5505024~147456 [fadvise_sequential+fadvise_nocache],write 5718016~49152 [fadvise_sequential+fadvise_nocache],write 5799936~147456 [fadvise_sequential+fadvise_nocache],write 6012928~147456 [fadvise_sequential+fadvise_nocache],write 6225920~65536 [fadvise_sequential+fadvise_nocache],write 6307840~147456 [fadvise_sequential+fadvise_nocache],write 6520832~147456 [fadvise_sequential+fadvise_nocache],write 6733824~229376 [fadvise_sequential+fadvise_nocache],write 7028736~147456 [fadvise_sequential+fadvise_nocache],write 7241728~229376 [fadvise_sequential+fadvise_nocache],write 7536640~147456 [fadvise_sequential+fadvise_nocache],write 7749632~229376 [fadvise_sequential+fadvise_nocache],write 8077312~114688 [fadvise_sequential+fadvise_nocache],write 8290304~98304 [fadvise_sequential+fadvise_nocache],write 8404992~81920 [fadvise_sequential+fadvise_nocache],write 8617984~81920 [fadvise_sequential+fadvise_nocache],write 8830976~81920 [fadvise_sequential+fadvise_nocache],write 8962048~32768 [fadvise_sequential+fadvise_nocache],write 9175040~32768 [fadvise_sequential+fadvise_nocache],write 9388032~32768 [fadvise_sequential+fadvise_nocache],write 9469952~32768 [fadvise_sequential+fadvise_nocache],write 9682944~20092 [fadvise_sequential+fadvise_nocache],write 9961472~163840 [fadvise_sequential+fadvise_nocache],write 10141696~196608 [fadvise_sequential+fadvise_nocache],write 10354688~278528 [fadvise_sequential+fadvise_nocache],write 10649600~196608 [fadvise_sequential+fadvise_nocache],write 10862592~196608 [fadvise_sequential+fadvise_nocache],write 11075584~65536 [fadvise_sequential+fadvise_nocache],write 11173888~98304 [fadvise_sequential+fadvise_nocache],write 11288576~65536 [fadvise_sequential+fadvise_nocache],write 11386880~98304 [fadvise_sequential+fadvise_nocache],write 11501568~65536 [fadvise_sequential+fadvise_nocache],write 11583488~65536 [fadvise_sequential+fadvise_nocache],write 11681792~98304 [fadvise_sequential+fadvise_nocache],write 11796480~65536 [fadvise_sequential+fadvise_nocache],write 11894784~98304 [fadvise_sequential+fadvise_nocache],write 12009472~65536 [fadvise_sequential+fadvise_nocache],write 12189696~98304 [fadvise_sequential+fadvise_nocache],write 12402688~98304 [fadvise_sequential+fadvise_nocache],write 12697600~98304 [fadvise_sequential+fadvise_nocache],write 12910592~98304 [fadvise_sequential+fadvise_nocache],write 13205504~98304 [fadvise_sequential+fadvise_nocache],write 13418496~98304 [fadvise_sequential+fadvise_nocache],write 13713408~98304 [fadvise_sequential+fadvise_nocache],write 13926400~98304 [fadvise_sequential+fadvise_nocache],write 14139392~16384 [fadvise_sequential+fadvise_nocache],write 14680064~16384 [fadvise_sequential+fadvise_nocache],write 14827520~81920 [fadvise_sequential+fadvise_nocache],write 15040512~81920 [fadvise_sequential+fadvise_nocache],write 15335424~81920 [fadvise_sequential+fadvise_nocache],write 15548416~81920 [fadvise_sequential+fadvise_nocache],write 15843328~81920 [fadvise_sequential+fadvise_nocache],write 16056320~81920 [fadvise_sequential+fadvise_nocache],write 16351232~81920 [fadvise_sequential+fadvise_nocache],write 16564224~81920 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:01:24.968 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.354331+0000", 2024-09-21T15:01:24.968 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 79.604345088000002, 2024-09-21T15:01:24.968 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.086284870999999999, 2024-09-21T15:01:24.968 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:24.968 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:24.968 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:24.968 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:24.968 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:24.968 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97734 2024-09-21T15:01:24.968 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.968 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:24.969 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.969 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:24.969 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.354331+0000", 2024-09-21T15:01:24.969 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:24.969 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.969 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.969 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:24.969 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.354331+0000", 2024-09-21T15:01:24.969 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:24.969 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.969 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.969 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:24.969 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.354329+0000", 2024-09-21T15:01:24.969 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:01:24.969 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.970 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.970 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:24.970 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.370234+0000", 2024-09-21T15:01:24.970 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015905348999999999 2024-09-21T15:01:24.970 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.970 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.970 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:24.970 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.370286+0000", 2024-09-21T15:01:24.970 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1656999999999997e-05 2024-09-21T15:01:24.970 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.970 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.970 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:24.970 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.370315+0000", 2024-09-21T15:01:24.970 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9204e-05 2024-09-21T15:01:24.970 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.970 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.971 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:24.971 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.370395+0000", 2024-09-21T15:01:24.971 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0395999999999994e-05 2024-09-21T15:01:24.971 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.971 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.971 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:24.971 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.370604+0000", 2024-09-21T15:01:24.971 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000208684 2024-09-21T15:01:24.971 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.971 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.971 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 1", 2024-09-21T15:01:24.971 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.371701+0000", 2024-09-21T15:01:24.971 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0010973140000000001 2024-09-21T15:01:24.971 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.971 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.971 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:01:24.972 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.405485+0000", 2024-09-21T15:01:24.972 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.033783615000000003 2024-09-21T15:01:24.972 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.972 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.972 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:01:24.972 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.440211+0000", 2024-09-21T15:01:24.972 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034725757000000003 2024-09-21T15:01:24.972 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.972 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.972 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:01:24.972 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.440497+0000", 2024-09-21T15:01:24.972 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00028673099999999998 2024-09-21T15:01:24.972 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.972 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.972 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:24.973 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.440616+0000", 2024-09-21T15:01:24.973 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000118256 2024-09-21T15:01:24.973 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:24.973 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:24.973 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:24.973 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.973 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.973 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112831 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 1515520~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:24.973 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.406761+0000", 2024-09-21T15:01:24.973 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 55.55191439, 2024-09-21T15:01:24.973 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060265666000000002, 2024-09-21T15:01:24.973 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:24.973 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:24.973 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:24.974 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:24.974 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:24.974 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112831 2024-09-21T15:01:24.974 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.974 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:24.974 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.974 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:24.974 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406761+0000", 2024-09-21T15:01:24.974 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:24.974 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.974 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.974 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:24.974 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406761+0000", 2024-09-21T15:01:24.974 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:24.974 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.975 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.975 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:24.975 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406761+0000", 2024-09-21T15:01:24.975 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:24.975 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.975 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.975 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:24.975 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406764+0000", 2024-09-21T15:01:24.975 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0319999999999999e-06 2024-09-21T15:01:24.975 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.975 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.975 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:24.975 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406776+0000", 2024-09-21T15:01:24.975 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2357999999999999e-05 2024-09-21T15:01:24.975 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.976 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.976 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:24.976 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406784+0000", 2024-09-21T15:01:24.976 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6059999999999997e-06 2024-09-21T15:01:24.976 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.976 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.976 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:24.976 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.466595+0000", 2024-09-21T15:01:24.976 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059811062999999998 2024-09-21T15:01:24.976 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.976 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.976 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:24.976 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.466782+0000", 2024-09-21T15:01:24.976 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000186622 2024-09-21T15:01:24.976 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.976 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.977 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:24.977 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.467027+0000", 2024-09-21T15:01:24.977 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000245455 2024-09-21T15:01:24.977 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:24.977 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:24.977 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:24.977 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.977 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.977 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112832 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 1605632~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:24.977 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.406788+0000", 2024-09-21T15:01:24.977 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 55.551887800000003, 2024-09-21T15:01:24.977 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060452065999999999, 2024-09-21T15:01:24.977 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:24.977 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:24.977 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:24.977 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:24.978 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:24.978 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112832 2024-09-21T15:01:24.978 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.978 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:24.978 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.978 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:24.978 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406788+0000", 2024-09-21T15:01:24.978 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:24.978 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.978 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.978 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:24.978 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406788+0000", 2024-09-21T15:01:24.978 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:24.978 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.978 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.979 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:24.979 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406788+0000", 2024-09-21T15:01:24.979 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:24.979 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.979 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.979 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:24.979 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406798+0000", 2024-09-21T15:01:24.979 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0096e-05 2024-09-21T15:01:24.979 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.979 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.979 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:24.979 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406803+0000", 2024-09-21T15:01:24.979 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2020000000000003e-06 2024-09-21T15:01:24.979 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.979 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.979 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:24.979 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406808+0000", 2024-09-21T15:01:24.980 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8239999999999999e-06 2024-09-21T15:01:24.980 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.980 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.980 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:24.980 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.467049+0000", 2024-09-21T15:01:24.980 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060241754000000002 2024-09-21T15:01:24.980 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.980 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.980 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:24.980 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.467134+0000", 2024-09-21T15:01:24.980 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4216999999999997e-05 2024-09-21T15:01:24.980 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.980 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.980 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:24.980 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.467240+0000", 2024-09-21T15:01:24.980 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000106304 2024-09-21T15:01:24.981 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:24.981 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:24.981 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:24.981 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.981 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.981 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112843 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 2613248~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:24.981 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.410273+0000", 2024-09-21T15:01:24.981 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 55.548403206000003, 2024-09-21T15:01:24.981 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059854931, 2024-09-21T15:01:24.981 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:24.981 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:24.981 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:24.981 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:24.981 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:24.982 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112843 2024-09-21T15:01:24.982 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.982 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:24.982 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.982 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:24.982 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410273+0000", 2024-09-21T15:01:24.982 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:24.982 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.982 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.982 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:24.982 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410273+0000", 2024-09-21T15:01:24.982 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:24.982 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.982 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.982 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:24.982 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410272+0000", 2024-09-21T15:01:24.983 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:24.983 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.983 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.983 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:24.983 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410274+0000", 2024-09-21T15:01:24.983 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.905e-06 2024-09-21T15:01:24.983 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.983 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.983 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:24.983 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410283+0000", 2024-09-21T15:01:24.983 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3440000000000007e-06 2024-09-21T15:01:24.983 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.983 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.983 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:24.983 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410294+0000", 2024-09-21T15:01:24.983 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0271e-05 2024-09-21T15:01:24.984 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.984 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.984 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:24.984 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.469917+0000", 2024-09-21T15:01:24.984 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059623738000000003 2024-09-21T15:01:24.984 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.984 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.984 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:24.984 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470002+0000", 2024-09-21T15:01:24.984 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4270999999999993e-05 2024-09-21T15:01:24.984 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.984 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.984 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:24.984 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470127+0000", 2024-09-21T15:01:24.984 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000125711 2024-09-21T15:01:24.985 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:24.985 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:24.985 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:24.985 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.985 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.985 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112844 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 2703360~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:24.985 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.410326+0000", 2024-09-21T15:01:24.985 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 55.548349485999999, 2024-09-21T15:01:24.985 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060086056999999998, 2024-09-21T15:01:24.985 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:24.985 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:24.985 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:24.985 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:24.985 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:24.985 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112844 2024-09-21T15:01:24.985 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.986 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:24.986 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.986 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:24.986 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410326+0000", 2024-09-21T15:01:24.986 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:24.986 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.986 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.986 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:24.986 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410326+0000", 2024-09-21T15:01:24.986 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:24.986 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.986 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.986 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:24.986 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410326+0000", 2024-09-21T15:01:24.987 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:24.987 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.987 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.987 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:24.987 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410352+0000", 2024-09-21T15:01:24.987 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6872999999999998e-05 2024-09-21T15:01:24.987 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.987 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.987 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:24.987 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410374+0000", 2024-09-21T15:01:24.987 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1141e-05 2024-09-21T15:01:24.987 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.987 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.987 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:24.988 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410385+0000", 2024-09-21T15:01:24.988 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1456e-05 2024-09-21T15:01:24.988 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.988 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.988 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:24.988 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470174+0000", 2024-09-21T15:01:24.988 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059788574999999997 2024-09-21T15:01:24.988 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.988 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.988 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:24.988 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470277+0000", 2024-09-21T15:01:24.988 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000103838 2024-09-21T15:01:24.988 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.988 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.988 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:24.988 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470412+0000", 2024-09-21T15:01:24.989 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013489700000000001 2024-09-21T15:01:24.989 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:24.989 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:24.989 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:24.989 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.989 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.989 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112845 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 2793472~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:24.989 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.410394+0000", 2024-09-21T15:01:24.989 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 55.548282012000001, 2024-09-21T15:01:24.989 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060244173999999998, 2024-09-21T15:01:24.989 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:24.989 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:24.989 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:24.989 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:24.989 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:24.990 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112845 2024-09-21T15:01:24.990 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.990 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:24.990 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.990 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:24.990 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410394+0000", 2024-09-21T15:01:24.990 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:24.990 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.990 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.990 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:24.990 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410394+0000", 2024-09-21T15:01:24.990 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:24.990 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.990 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.990 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:24.991 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410393+0000", 2024-09-21T15:01:24.991 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:24.991 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.991 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.991 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:24.991 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410397+0000", 2024-09-21T15:01:24.991 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9430000000000002e-06 2024-09-21T15:01:24.991 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.991 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.991 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:24.991 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410406+0000", 2024-09-21T15:01:24.991 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.799e-06 2024-09-21T15:01:24.991 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.991 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.991 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:24.991 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410418+0000", 2024-09-21T15:01:24.992 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2225999999999999e-05 2024-09-21T15:01:24.992 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.992 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.992 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:24.992 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470444+0000", 2024-09-21T15:01:24.992 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060026311999999998 2024-09-21T15:01:24.992 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.992 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.992 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:24.992 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470535+0000", 2024-09-21T15:01:24.992 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0173999999999994e-05 2024-09-21T15:01:24.992 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.992 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.992 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:24.992 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470638+0000", 2024-09-21T15:01:24.993 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010333 2024-09-21T15:01:24.993 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:24.993 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:24.993 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:24.993 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.993 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.993 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112846 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 2883584~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:24.993 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.410425+0000", 2024-09-21T15:01:24.993 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 55.548250799000002, 2024-09-21T15:01:24.993 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060459247000000001, 2024-09-21T15:01:24.993 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:24.993 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:24.993 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:24.993 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:24.993 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:24.994 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112846 2024-09-21T15:01:24.994 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.994 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:24.994 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.994 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:24.994 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410425+0000", 2024-09-21T15:01:24.994 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:24.994 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.994 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.994 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:24.994 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410425+0000", 2024-09-21T15:01:24.994 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:24.994 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.994 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.994 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:24.995 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410424+0000", 2024-09-21T15:01:24.995 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:24.995 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.995 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.995 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:24.995 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410428+0000", 2024-09-21T15:01:24.995 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4709999999999999e-06 2024-09-21T15:01:24.995 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.995 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.995 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:24.995 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410446+0000", 2024-09-21T15:01:24.995 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8354000000000001e-05 2024-09-21T15:01:24.995 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.995 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.995 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:24.995 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410456+0000", 2024-09-21T15:01:24.996 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6709999999999994e-06 2024-09-21T15:01:24.996 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.996 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.996 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:24.996 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470661+0000", 2024-09-21T15:01:24.996 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060205303000000002 2024-09-21T15:01:24.996 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.996 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.996 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:24.996 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470756+0000", 2024-09-21T15:01:24.996 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4401999999999995e-05 2024-09-21T15:01:24.996 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.996 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.996 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:24.996 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470884+0000", 2024-09-21T15:01:24.997 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012860500000000001 2024-09-21T15:01:24.997 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:24.997 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:24.997 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:24.997 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.997 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.997 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112847 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 2973696~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:24.997 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.410462+0000", 2024-09-21T15:01:24.997 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 55.548213754999999, 2024-09-21T15:01:24.997 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060653836000000003, 2024-09-21T15:01:24.997 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:24.997 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:24.997 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:24.997 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:24.998 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:24.998 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112847 2024-09-21T15:01:24.998 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.998 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:24.998 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.998 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:24.998 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410462+0000", 2024-09-21T15:01:24.998 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:24.998 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.998 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.998 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:24.998 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410462+0000", 2024-09-21T15:01:24.998 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:24.998 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.998 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.998 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:24.999 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410461+0000", 2024-09-21T15:01:24.999 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:24.999 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.999 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.999 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:24.999 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410465+0000", 2024-09-21T15:01:24.999 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6899999999999998e-06 2024-09-21T15:01:24.999 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.999 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:24.999 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:24.999 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410474+0000", 2024-09-21T15:01:24.999 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4179999999999997e-06 2024-09-21T15:01:24.999 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:24.999 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.000 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:25.000 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410508+0000", 2024-09-21T15:01:25.000 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4282999999999997e-05 2024-09-21T15:01:25.000 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.000 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.000 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:25.000 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470922+0000", 2024-09-21T15:01:25.000 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060413820999999999 2024-09-21T15:01:25.000 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.000 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.000 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:25.000 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471008+0000", 2024-09-21T15:01:25.000 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6379999999999996e-05 2024-09-21T15:01:25.000 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.000 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.000 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:25.001 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471116+0000", 2024-09-21T15:01:25.001 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000107806 2024-09-21T15:01:25.001 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.001 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:25.001 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.001 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.001 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.001 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112848 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 3063808~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:25.001 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.410525+0000", 2024-09-21T15:01:25.001 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 55.548150472000003, 2024-09-21T15:01:25.001 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060880735999999998, 2024-09-21T15:01:25.001 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:25.001 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:25.001 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:25.001 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:25.001 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:25.002 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112848 2024-09-21T15:01:25.002 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.002 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:25.002 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.002 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:25.002 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410525+0000", 2024-09-21T15:01:25.002 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.002 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.002 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.002 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:25.002 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410525+0000", 2024-09-21T15:01:25.002 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.002 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.002 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.002 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:25.002 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410525+0000", 2024-09-21T15:01:25.003 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:25.003 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.003 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.003 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:25.003 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410529+0000", 2024-09-21T15:01:25.003 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1760000000000003e-06 2024-09-21T15:01:25.003 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.003 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.003 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:25.003 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410551+0000", 2024-09-21T15:01:25.003 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2099999999999998e-05 2024-09-21T15:01:25.003 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.003 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.003 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:25.004 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410571+0000", 2024-09-21T15:01:25.004 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0159000000000001e-05 2024-09-21T15:01:25.004 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.004 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.004 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:25.004 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471164+0000", 2024-09-21T15:01:25.004 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.06059287 2024-09-21T15:01:25.004 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.004 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.004 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:25.004 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471271+0000", 2024-09-21T15:01:25.004 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000106635 2024-09-21T15:01:25.004 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.004 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.004 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:25.005 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471406+0000", 2024-09-21T15:01:25.005 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000135399 2024-09-21T15:01:25.005 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.005 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:25.005 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.005 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.005 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.005 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112849 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 3153920~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:25.005 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.410579+0000", 2024-09-21T15:01:25.005 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 55.548097177000002, 2024-09-21T15:01:25.005 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.061057304999999999, 2024-09-21T15:01:25.005 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:25.005 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:25.005 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:25.005 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:25.005 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:25.006 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112849 2024-09-21T15:01:25.006 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.006 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:25.006 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.006 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:25.006 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410579+0000", 2024-09-21T15:01:25.006 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.006 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.006 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.006 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:25.006 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410579+0000", 2024-09-21T15:01:25.006 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.006 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.006 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.006 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:25.007 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410578+0000", 2024-09-21T15:01:25.007 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:25.007 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.007 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.007 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:25.007 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410582+0000", 2024-09-21T15:01:25.007 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.036e-06 2024-09-21T15:01:25.007 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.007 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.007 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:25.007 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410592+0000", 2024-09-21T15:01:25.007 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9850000000000001e-06 2024-09-21T15:01:25.007 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.007 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.007 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:25.007 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410604+0000", 2024-09-21T15:01:25.008 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2051e-05 2024-09-21T15:01:25.008 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.008 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.008 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:25.008 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471444+0000", 2024-09-21T15:01:25.008 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060839722999999998 2024-09-21T15:01:25.008 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.008 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.008 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:25.008 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471530+0000", 2024-09-21T15:01:25.008 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6359000000000004e-05 2024-09-21T15:01:25.008 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.008 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.008 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:25.009 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471636+0000", 2024-09-21T15:01:25.009 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010576700000000001 2024-09-21T15:01:25.009 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.009 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:25.009 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.009 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.009 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.009 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112850 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 3244032~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:25.009 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.410698+0000", 2024-09-21T15:01:25.009 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 55.547977834999998, 2024-09-21T15:01:25.009 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.061164118000000003, 2024-09-21T15:01:25.009 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:25.009 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:25.010 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:25.010 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:25.010 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:25.010 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112850 2024-09-21T15:01:25.010 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.010 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:25.010 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.010 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:25.010 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410698+0000", 2024-09-21T15:01:25.010 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.010 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.010 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.010 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:25.011 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410698+0000", 2024-09-21T15:01:25.011 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.011 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.011 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.011 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:25.011 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410610+0000", 2024-09-21T15:01:25.011 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999123 2024-09-21T15:01:25.011 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.011 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.011 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:25.011 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410707+0000", 2024-09-21T15:01:25.011 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7093e-05 2024-09-21T15:01:25.011 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.011 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.012 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:25.012 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410728+0000", 2024-09-21T15:01:25.012 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0275000000000002e-05 2024-09-21T15:01:25.012 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.012 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.012 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:25.012 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410740+0000", 2024-09-21T15:01:25.012 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2561999999999999e-05 2024-09-21T15:01:25.012 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.012 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.012 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:25.012 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471664+0000", 2024-09-21T15:01:25.012 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060923979000000003 2024-09-21T15:01:25.012 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.012 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.013 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:25.013 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471753+0000", 2024-09-21T15:01:25.013 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8265999999999998e-05 2024-09-21T15:01:25.013 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.013 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.013 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:25.013 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471862+0000", 2024-09-21T15:01:25.013 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000109474 2024-09-21T15:01:25.013 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.013 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:25.013 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.013 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.013 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.013 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112864 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 4521984~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:25.013 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.414789+0000", 2024-09-21T15:01:25.014 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 55.543886810000004, 2024-09-21T15:01:25.014 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059845903999999998, 2024-09-21T15:01:25.014 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:25.014 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:25.014 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:25.014 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:25.014 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:25.014 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112864 2024-09-21T15:01:25.014 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.014 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:25.014 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.014 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:25.014 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414789+0000", 2024-09-21T15:01:25.015 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.015 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.015 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.015 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:25.015 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414789+0000", 2024-09-21T15:01:25.015 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.015 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.015 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.015 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:25.015 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414788+0000", 2024-09-21T15:01:25.015 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:25.015 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.015 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.015 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:25.015 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414790+0000", 2024-09-21T15:01:25.015 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.995e-06 2024-09-21T15:01:25.016 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.016 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.016 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:25.016 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414801+0000", 2024-09-21T15:01:25.016 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.01e-05 2024-09-21T15:01:25.016 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.016 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.016 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:25.016 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414804+0000", 2024-09-21T15:01:25.016 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.563e-06 2024-09-21T15:01:25.016 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.016 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.016 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:25.016 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.474499+0000", 2024-09-21T15:01:25.016 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059695208 2024-09-21T15:01:25.017 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.017 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.017 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:25.017 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.474561+0000", 2024-09-21T15:01:25.017 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1672000000000002e-05 2024-09-21T15:01:25.017 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.017 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.017 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:25.017 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.474635+0000", 2024-09-21T15:01:25.017 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3792000000000001e-05 2024-09-21T15:01:25.017 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.017 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:25.017 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.017 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.017 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.018 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112865 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 4612096~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:25.018 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.414807+0000", 2024-09-21T15:01:25.018 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 55.543868414000002, 2024-09-21T15:01:25.018 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060003506999999998, 2024-09-21T15:01:25.018 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:25.018 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:25.018 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:25.018 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:25.018 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:25.018 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112865 2024-09-21T15:01:25.018 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.018 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:25.018 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.019 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:25.019 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414807+0000", 2024-09-21T15:01:25.019 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.019 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.019 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.019 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:25.019 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414807+0000", 2024-09-21T15:01:25.019 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.019 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.019 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.019 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:25.019 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414807+0000", 2024-09-21T15:01:25.019 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:25.019 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.019 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.019 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:25.020 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414809+0000", 2024-09-21T15:01:25.020 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0509999999999999e-06 2024-09-21T15:01:25.020 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.020 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.020 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:25.020 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414813+0000", 2024-09-21T15:01:25.020 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0960000000000003e-06 2024-09-21T15:01:25.020 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.020 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.020 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:25.020 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414817+0000", 2024-09-21T15:01:25.020 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.296e-06 2024-09-21T15:01:25.020 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.020 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.020 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:25.020 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.474671+0000", 2024-09-21T15:01:25.021 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059853663000000001 2024-09-21T15:01:25.021 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.021 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.021 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:25.021 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.474735+0000", 2024-09-21T15:01:25.021 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3653999999999997e-05 2024-09-21T15:01:25.021 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.021 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.021 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:25.021 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.474811+0000", 2024-09-21T15:01:25.021 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6074000000000004e-05 2024-09-21T15:01:25.021 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.021 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:25.021 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.021 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.021 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.021 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112866 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 4702208~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:25.022 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.414820+0000", 2024-09-21T15:01:25.022 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 55.543855493999999, 2024-09-21T15:01:25.022 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060161992999999997, 2024-09-21T15:01:25.022 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:25.022 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:25.022 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:25.022 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:25.022 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:25.022 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112866 2024-09-21T15:01:25.022 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.022 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:25.022 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.022 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:25.022 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414820+0000", 2024-09-21T15:01:25.022 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.023 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.023 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.023 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:25.023 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414820+0000", 2024-09-21T15:01:25.023 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.023 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.023 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.023 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:25.023 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414820+0000", 2024-09-21T15:01:25.023 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:25.023 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.023 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.023 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:25.023 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414822+0000", 2024-09-21T15:01:25.024 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8509999999999999e-06 2024-09-21T15:01:25.024 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.024 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.024 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:25.024 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414826+0000", 2024-09-21T15:01:25.024 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8190000000000002e-06 2024-09-21T15:01:25.024 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.024 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.024 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:25.024 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414837+0000", 2024-09-21T15:01:25.024 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1232e-05 2024-09-21T15:01:25.024 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.024 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.024 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:25.025 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.474846+0000", 2024-09-21T15:01:25.025 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060009506999999997 2024-09-21T15:01:25.025 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.025 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.025 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:25.025 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.474910+0000", 2024-09-21T15:01:25.025 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3918999999999996e-05 2024-09-21T15:01:25.025 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.025 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.025 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:25.025 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.474982+0000", 2024-09-21T15:01:25.025 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1958999999999993e-05 2024-09-21T15:01:25.025 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.025 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:25.025 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.025 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.026 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.026 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112867 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 4792320~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:25.026 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.414840+0000", 2024-09-21T15:01:25.026 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 55.543835999000002, 2024-09-21T15:01:25.026 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060308542999999999, 2024-09-21T15:01:25.026 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:25.026 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:25.026 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:25.026 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:25.026 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:25.026 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112867 2024-09-21T15:01:25.026 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.026 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:25.026 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.026 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:25.027 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414840+0000", 2024-09-21T15:01:25.027 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.027 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.027 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.027 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:25.027 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414840+0000", 2024-09-21T15:01:25.027 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.027 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.027 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.027 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:25.027 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414839+0000", 2024-09-21T15:01:25.027 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:25.027 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.027 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.028 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:25.028 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414841+0000", 2024-09-21T15:01:25.028 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8500000000000001e-06 2024-09-21T15:01:25.028 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.028 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.028 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:25.028 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414846+0000", 2024-09-21T15:01:25.028 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7700000000000001e-06 2024-09-21T15:01:25.028 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.028 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.028 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:25.028 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414850+0000", 2024-09-21T15:01:25.028 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1389999999999997e-06 2024-09-21T15:01:25.028 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.028 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.028 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:25.029 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.475016+0000", 2024-09-21T15:01:25.029 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.06016614 2024-09-21T15:01:25.029 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.029 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.029 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:25.029 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.475073+0000", 2024-09-21T15:01:25.029 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6295000000000002e-05 2024-09-21T15:01:25.029 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.029 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.029 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:25.029 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.475148+0000", 2024-09-21T15:01:25.029 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5647000000000003e-05 2024-09-21T15:01:25.029 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.029 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:25.029 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.030 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.030 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.030 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:113030 3.6 3:60420fc2:::rbd_data.1136b442e94a.0000000000000001:head [write 0~360448 [fadvise_sequential+fadvise_nocache],write 385024~573440 [fadvise_sequential+fadvise_nocache],write 1146880~573440 [fadvise_sequential+fadvise_nocache],write 1908736~188416 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:25.030 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.422420+0000", 2024-09-21T15:01:25.030 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 55.536255353999998, 2024-09-21T15:01:25.030 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.063268959999999999, 2024-09-21T15:01:25.030 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:25.030 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:25.030 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:25.030 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:25.030 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:25.030 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 113030 2024-09-21T15:01:25.030 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.030 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:25.030 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.031 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:25.031 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.422420+0000", 2024-09-21T15:01:25.031 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.031 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.031 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.031 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:25.031 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.422420+0000", 2024-09-21T15:01:25.031 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.031 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.031 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.031 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:25.031 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.422420+0000", 2024-09-21T15:01:25.031 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:25.032 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.032 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.032 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:25.032 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.424657+0000", 2024-09-21T15:01:25.032 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0022369780000000001 2024-09-21T15:01:25.032 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.032 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.032 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:25.032 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.424673+0000", 2024-09-21T15:01:25.032 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6141e-05 2024-09-21T15:01:25.032 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.032 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.032 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:25.032 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.424689+0000", 2024-09-21T15:01:25.033 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5897999999999999e-05 2024-09-21T15:01:25.033 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.033 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.033 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:25.033 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.478005+0000", 2024-09-21T15:01:25.033 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.05331636 2024-09-21T15:01:25.033 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.033 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.033 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:25.033 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.478133+0000", 2024-09-21T15:01:25.033 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012736799999999999 2024-09-21T15:01:25.033 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.033 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.033 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 7", 2024-09-21T15:01:25.033 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.478307+0000", 2024-09-21T15:01:25.033 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017491499999999999 2024-09-21T15:01:25.034 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.034 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.034 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:01:25.034 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.483611+0000", 2024-09-21T15:01:25.034 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0053039389999999997 2024-09-21T15:01:25.034 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.034 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.034 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:01:25.034 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.485553+0000", 2024-09-21T15:01:25.034 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0019418759999999999 2024-09-21T15:01:25.034 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.034 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.034 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:01:25.034 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.485645+0000", 2024-09-21T15:01:25.035 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2138000000000005e-05 2024-09-21T15:01:25.035 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.035 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.035 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:25.035 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.485689+0000", 2024-09-21T15:01:25.035 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3922999999999999e-05 2024-09-21T15:01:25.035 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.035 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:25.035 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.035 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.035 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.035 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:114290 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 6946816~8192 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:25.035 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.558087+0000", 2024-09-21T15:01:25.035 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 55.400588556999999, 2024-09-21T15:01:25.035 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059973091999999999, 2024-09-21T15:01:25.036 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:25.036 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:25.036 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:25.036 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:25.036 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:25.036 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 114290 2024-09-21T15:01:25.036 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.036 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:25.036 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.036 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:25.036 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.558087+0000", 2024-09-21T15:01:25.036 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.036 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.036 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.036 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:25.036 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.558087+0000", 2024-09-21T15:01:25.037 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.037 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.037 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.037 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:25.037 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.558087+0000", 2024-09-21T15:01:25.037 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:25.037 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.037 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.037 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:25.037 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.558092+0000", 2024-09-21T15:01:25.037 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2379999999999997e-06 2024-09-21T15:01:25.037 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.037 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.037 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:25.038 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.558095+0000", 2024-09-21T15:01:25.038 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.275e-06 2024-09-21T15:01:25.038 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.038 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.038 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:25.038 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.558099+0000", 2024-09-21T15:01:25.038 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6679999999999999e-06 2024-09-21T15:01:25.038 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.038 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.038 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:25.038 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.617757+0000", 2024-09-21T15:01:25.038 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059657531999999999 2024-09-21T15:01:25.038 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.039 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.039 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:25.039 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.617891+0000", 2024-09-21T15:01:25.039 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000134478 2024-09-21T15:01:25.039 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.039 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.039 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:25.039 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.618060+0000", 2024-09-21T15:01:25.039 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016925100000000001 2024-09-21T15:01:25.039 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.039 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:25.039 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.039 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.039 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.040 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:114312 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 8945664~8192 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:25.040 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.562396+0000", 2024-09-21T15:01:25.040 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 55.396280208999997, 2024-09-21T15:01:25.040 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059832396000000003, 2024-09-21T15:01:25.040 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:25.040 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:25.040 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:25.040 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:25.040 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:25.040 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 114312 2024-09-21T15:01:25.040 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.040 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:25.040 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.040 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:25.040 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562396+0000", 2024-09-21T15:01:25.041 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.041 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.041 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.041 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:25.041 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562396+0000", 2024-09-21T15:01:25.041 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.041 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.041 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.041 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:25.041 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562395+0000", 2024-09-21T15:01:25.041 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:25.041 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.041 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.041 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:25.042 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562397+0000", 2024-09-21T15:01:25.042 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9e-06 2024-09-21T15:01:25.042 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.042 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.042 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:25.042 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562401+0000", 2024-09-21T15:01:25.042 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8099999999999999e-06 2024-09-21T15:01:25.042 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.042 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.042 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:25.042 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562405+0000", 2024-09-21T15:01:25.042 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.5710000000000002e-06 2024-09-21T15:01:25.042 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.042 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.043 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:25.043 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.621997+0000", 2024-09-21T15:01:25.043 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059592118999999999 2024-09-21T15:01:25.043 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.043 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.043 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:25.043 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.622121+0000", 2024-09-21T15:01:25.043 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012411599999999999 2024-09-21T15:01:25.043 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.043 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.043 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:25.043 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.622228+0000", 2024-09-21T15:01:25.043 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000107184 2024-09-21T15:01:25.043 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.044 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:25.044 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.044 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.044 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.044 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:114313 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 9035776~8192 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:25.044 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.562408+0000", 2024-09-21T15:01:25.044 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 55.396268227999997, 2024-09-21T15:01:25.044 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059975560999999997, 2024-09-21T15:01:25.044 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:25.044 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:25.044 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:25.044 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:25.044 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:25.044 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 114313 2024-09-21T15:01:25.045 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.045 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:25.045 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.045 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:25.045 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562408+0000", 2024-09-21T15:01:25.045 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.045 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.045 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.045 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:25.045 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562408+0000", 2024-09-21T15:01:25.045 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.045 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.045 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.045 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:25.046 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562407+0000", 2024-09-21T15:01:25.046 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:25.046 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.046 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.046 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:25.046 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562409+0000", 2024-09-21T15:01:25.046 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9700000000000002e-06 2024-09-21T15:01:25.046 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.046 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.046 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:25.046 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562421+0000", 2024-09-21T15:01:25.046 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2094e-05 2024-09-21T15:01:25.046 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.046 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.047 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:25.047 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562427+0000", 2024-09-21T15:01:25.047 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7690000000000002e-06 2024-09-21T15:01:25.047 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.047 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.047 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:25.047 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.622264+0000", 2024-09-21T15:01:25.047 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059837488000000001 2024-09-21T15:01:25.047 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.047 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.047 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:25.047 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.622319+0000", 2024-09-21T15:01:25.047 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4064999999999998e-05 2024-09-21T15:01:25.048 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.048 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.048 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:25.048 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.622383+0000", 2024-09-21T15:01:25.048 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4506999999999998e-05 2024-09-21T15:01:25.048 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.048 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:25.048 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.048 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.048 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:25.048 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:01:25.049 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T15:01:25.049 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T15:01:25.049 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T15:01:25.050 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T15:01:25.051 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:01:25.051 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:01:25.051 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:01:25.051 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:01:25.051 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:01:25.051 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:01:25.051 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:01:25.051 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:01:25.053 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:01:25.046+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:01:25.154 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:01:25.150+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:01:25.186 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:01:25.186 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:01:25.186 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T15:01:25.186 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:01:25.198 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T15:01:25.255 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:01:25.250+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:01:25.300 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:01:25.300 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:01:25.300 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T15:01:25.300 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T15:01:25.300 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:01:25.310 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T15:01:25.355 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:01:25.350+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:01:25.383 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T15:01:25.417 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:01:25.417 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T15:01:25.417 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T15:01:25.417 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T15:01:25.417 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.417 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45854 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:25.417 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159802+0000", 2024-09-21T15:01:25.417 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 135.25252891900001, 2024-09-21T15:01:25.417 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068358938999999994, 2024-09-21T15:01:25.417 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:25.417 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:25.417 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:25.418 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:25.418 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:25.418 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45854 2024-09-21T15:01:25.418 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.418 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:25.418 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.418 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:25.418 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159802+0000", 2024-09-21T15:01:25.418 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.418 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.418 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.418 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:25.418 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159802+0000", 2024-09-21T15:01:25.418 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.419 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.419 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.419 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:25.419 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159801+0000", 2024-09-21T15:01:25.419 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:25.419 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.419 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.419 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:25.419 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159804+0000", 2024-09-21T15:01:25.419 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0970000000000002e-06 2024-09-21T15:01:25.419 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.419 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.419 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:25.419 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159823+0000", 2024-09-21T15:01:25.419 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8895000000000001e-05 2024-09-21T15:01:25.420 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.420 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.420 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:25.420 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159837+0000", 2024-09-21T15:01:25.420 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3247999999999999e-05 2024-09-21T15:01:25.420 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.420 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.420 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:25.420 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228151+0000", 2024-09-21T15:01:25.420 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068314642999999994 2024-09-21T15:01:25.420 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.420 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.420 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:25.420 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228161+0000", 2024-09-21T15:01:25.421 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4250000000000004e-06 2024-09-21T15:01:25.421 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.421 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:25.421 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.421 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.421 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.421 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45985 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:25.421 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159897+0000", 2024-09-21T15:01:25.421 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 135.25243341699999, 2024-09-21T15:01:25.421 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068345664, 2024-09-21T15:01:25.421 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:25.421 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:25.421 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:25.421 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:25.421 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:25.421 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45985 2024-09-21T15:01:25.422 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.422 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:25.422 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.422 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:25.422 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159897+0000", 2024-09-21T15:01:25.422 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.422 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.422 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.422 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:25.422 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159897+0000", 2024-09-21T15:01:25.422 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.422 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.422 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.423 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:25.423 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159897+0000", 2024-09-21T15:01:25.423 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:25.423 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.423 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.423 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:25.423 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159900+0000", 2024-09-21T15:01:25.423 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.954e-06 2024-09-21T15:01:25.423 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.423 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.423 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:25.423 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159913+0000", 2024-09-21T15:01:25.423 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3106e-05 2024-09-21T15:01:25.423 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.424 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.424 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:25.424 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159918+0000", 2024-09-21T15:01:25.424 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5199999999999997e-06 2024-09-21T15:01:25.424 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.424 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.424 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:25.424 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228236+0000", 2024-09-21T15:01:25.424 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068317040999999995 2024-09-21T15:01:25.424 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.424 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.424 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:25.424 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228243+0000", 2024-09-21T15:01:25.424 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3880000000000002e-06 2024-09-21T15:01:25.425 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.425 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:25.425 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.425 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.425 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.425 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45986 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:25.425 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159924+0000", 2024-09-21T15:01:25.425 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 135.25240712199999, 2024-09-21T15:01:25.425 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068364362999999997, 2024-09-21T15:01:25.425 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:25.425 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:25.425 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:25.425 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:25.426 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:25.426 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45986 2024-09-21T15:01:25.426 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.426 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:25.426 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.426 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:25.426 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159924+0000", 2024-09-21T15:01:25.426 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.426 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.426 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.426 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:25.426 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159924+0000", 2024-09-21T15:01:25.426 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.427 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.427 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.427 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:25.427 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159923+0000", 2024-09-21T15:01:25.427 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:25.427 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.427 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.427 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:25.427 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159926+0000", 2024-09-21T15:01:25.427 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0510000000000001e-06 2024-09-21T15:01:25.427 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.427 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.427 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:25.428 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159932+0000", 2024-09-21T15:01:25.428 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4160000000000001e-06 2024-09-21T15:01:25.428 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.428 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.428 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:25.428 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159938+0000", 2024-09-21T15:01:25.428 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.383e-06 2024-09-21T15:01:25.428 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.428 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.428 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:25.428 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228275+0000", 2024-09-21T15:01:25.428 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068337007000000005 2024-09-21T15:01:25.428 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.428 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.429 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:25.429 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228288+0000", 2024-09-21T15:01:25.429 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3052e-05 2024-09-21T15:01:25.429 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.429 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:25.429 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.429 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.429 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.429 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45987 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:25.429 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159944+0000", 2024-09-21T15:01:25.429 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 135.252386898, 2024-09-21T15:01:25.429 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068387232000000006, 2024-09-21T15:01:25.429 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:25.429 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:25.430 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:25.430 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:25.430 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:25.430 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45987 2024-09-21T15:01:25.430 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.430 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:25.430 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.430 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:25.430 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159944+0000", 2024-09-21T15:01:25.430 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.430 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.430 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.430 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:25.430 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159944+0000", 2024-09-21T15:01:25.431 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.431 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.431 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.431 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:25.431 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159943+0000", 2024-09-21T15:01:25.431 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:25.431 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.431 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.431 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:25.431 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159946+0000", 2024-09-21T15:01:25.431 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6309999999999999e-06 2024-09-21T15:01:25.431 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.431 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.431 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:25.432 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159952+0000", 2024-09-21T15:01:25.432 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6540000000000004e-06 2024-09-21T15:01:25.432 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.432 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.432 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:25.432 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159957+0000", 2024-09-21T15:01:25.432 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9280000000000001e-06 2024-09-21T15:01:25.432 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.432 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.432 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:25.432 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228322+0000", 2024-09-21T15:01:25.432 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068365067000000002 2024-09-21T15:01:25.432 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.432 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.433 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:25.433 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228331+0000", 2024-09-21T15:01:25.433 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3270000000000007e-06 2024-09-21T15:01:25.433 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.433 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:25.433 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.433 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.433 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.433 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:97672 3.4 3:26046d29:::rbd_data.1136bfc79a70.0000000000000001:head [write 0~4718592 [fadvise_sequential+fadvise_nocache],write 5242880~131072 [fadvise_sequential+fadvise_nocache],write 5439488~147456 [fadvise_sequential+fadvise_nocache],write 5652480~229376 [fadvise_sequential+fadvise_nocache],write 5947392~147456 [fadvise_sequential+fadvise_nocache],write 6160384~229376 [fadvise_sequential+fadvise_nocache],write 6455296~147456 [fadvise_sequential+fadvise_nocache],write 6668288~229376 [fadvise_sequential+fadvise_nocache],write 6963200~147456 [fadvise_sequential+fadvise_nocache],write 7176192~147456 [fadvise_sequential+fadvise_nocache],write 7340032~65536 [fadvise_sequential+fadvise_nocache],write 7471104~147456 [fadvise_sequential+fadvise_nocache],write 7684096~147456 [fadvise_sequential+fadvise_nocache],write 7864320~49152 [fadvise_sequential+fadvise_nocache],write 7979008~147456 [fadvise_sequential+fadvise_nocache],write 8192000~147456 [fadvise_sequential+fadvise_nocache],write 8388608~32768 [fadvise_sequential+fadvise_nocache],write 8519680~114688 [fadvise_sequential+fadvise_nocache],write 8732672~114688 [fadvise_sequential+fadvise_nocache],write 8912896~16384 [fadvise_sequential+fadvise_nocache],write 9109504~32768 [fadvise_sequential+fadvise_nocache],write 9322496~32768 [fadvise_sequential+fadvise_nocache],write 9617408~32768 [fadvise_sequential+fadvise_nocache],write 9830400~32768 [fadvise_sequential+fadvise_nocache],write 9961472~98304 [fadvise_sequential+fadvise_nocache],write 10076160~196608 [fadvise_sequential+fadvise_nocache],write 10289152~278528 [fadvise_sequential+fadvise_nocache],write 10584064~196608 [fadvise_sequential+fadvise_nocache],write 10797056~196608 [fadvise_sequential+fadvise_nocache],write 11010048~65536 [fadvise_sequential+fadvise_nocache],write 11091968~196608 [fadvise_sequential+fadvise_nocache],write 11304960~196608 [fadvise_sequential+fadvise_nocache],write 11517952~65536 [fadvise_sequential+fadvise_nocache],write 11616256~98304 [fadvise_sequential+fadvise_nocache],write 11730944~65536 [fadvise_sequential+fadvise_nocache],write 11829248~98304 [fadvise_sequential+fadvise_nocache],write 11943936~65536 [fadvise_sequential+fadvise_nocache],write 12042240~49152 [fadvise_sequential+fadvise_nocache],write 12124160~98304 [fadvise_sequential+fadvise_nocache],write 12238848~65536 [fadvise_sequential+fadvise_nocache],write 12337152~98304 [fadvise_sequential+fadvise_nocache],write 12451840~65536 [fadvise_sequential+fadvise_nocache],write 12550144~32768 [fadvise_sequential+fadvise_nocache],write 12632064~98304 [fadvise_sequential+fadvise_nocache],write 12845056~98304 [fadvise_sequential+fadvise_nocache],write 13058048~49152 [fadvise_sequential+fadvise_nocache],write 13139968~98304 [fadvise_sequential+fadvise_nocache],write 13352960~98304 [fadvise_sequential+fadvise_nocache],write 13565952~65536 [fadvise_sequential+fadvise_nocache],write 13647872~98304 [fadvise_sequential+fadvise_nocache],write 13860864~98304 [fadvise_sequential+fadvise_nocache],write 14073856~81920 [fadvise_sequential+fadvise_nocache],write 15269888~81920 [fadvise_sequential+fadvise_nocache],write 15482880~81920 [fadvise_sequential+fadvise_nocache],write 15695872~32768 [fadvise_sequential+fadvise_nocache],write 15777792~81920 [fadvise_sequential+fadvise_nocache],write 15990784~81920 [fadvise_sequential+fadvise_nocache],write 16203776~49152 [fadvise_sequential+fadvise_nocache],write 16285696~81920 [fadvise_sequential+fadvise_nocache],write 16498688~81920 [fadvise_sequential+fadvise_nocache],write 16711680~65536 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:01:25.433 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.298669+0000", 2024-09-21T15:01:25.433 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 80.113661832000005, 2024-09-21T15:01:25.433 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.10282358900000001, 2024-09-21T15:01:25.433 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:25.433 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:25.433 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:25.434 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:25.434 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:25.434 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97672 2024-09-21T15:01:25.434 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.434 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:25.434 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.434 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:25.434 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.298669+0000", 2024-09-21T15:01:25.434 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.434 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.434 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.434 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:25.434 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.298669+0000", 2024-09-21T15:01:25.434 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.435 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.435 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.435 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:25.435 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.298667+0000", 2024-09-21T15:01:25.435 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:01:25.435 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.435 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.435 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:25.435 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.327458+0000", 2024-09-21T15:01:25.435 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.028790754000000002 2024-09-21T15:01:25.435 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.435 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.435 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:25.435 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.327492+0000", 2024-09-21T15:01:25.435 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3806999999999997e-05 2024-09-21T15:01:25.436 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.436 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.436 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:25.436 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.327507+0000", 2024-09-21T15:01:25.436 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5112e-05 2024-09-21T15:01:25.436 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.436 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.436 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:25.436 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.327597+0000", 2024-09-21T15:01:25.436 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9938999999999994e-05 2024-09-21T15:01:25.436 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.436 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.436 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:25.436 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.327927+0000", 2024-09-21T15:01:25.437 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000329766 2024-09-21T15:01:25.437 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.437 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.437 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T15:01:25.437 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.330735+0000", 2024-09-21T15:01:25.437 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0028086869999999998 2024-09-21T15:01:25.437 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.437 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.437 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:01:25.437 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.378299+0000", 2024-09-21T15:01:25.437 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.047563495999999997 2024-09-21T15:01:25.437 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.437 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.438 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:01:25.438 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.401248+0000", 2024-09-21T15:01:25.438 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.022949311 2024-09-21T15:01:25.438 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.438 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.438 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:01:25.438 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.401446+0000", 2024-09-21T15:01:25.438 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019831199999999999 2024-09-21T15:01:25.438 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.438 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.438 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:25.438 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.401492+0000", 2024-09-21T15:01:25.438 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6109000000000002e-05 2024-09-21T15:01:25.438 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.439 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:25.439 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.439 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.439 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.439 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139769 3.1e 3:793dcf44:::rbd_data.113664780c85.0000000000000000:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:25.439 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.221432+0000", 2024-09-21T15:01:25.439 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.190898962999999, 2024-09-21T15:01:25.439 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.035363362, 2024-09-21T15:01:25.439 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:25.439 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:25.439 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:25.439 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:25.439 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:25.440 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139769 2024-09-21T15:01:25.440 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.440 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:25.440 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.440 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:25.440 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221432+0000", 2024-09-21T15:01:25.440 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.440 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.440 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.440 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:25.440 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221432+0000", 2024-09-21T15:01:25.440 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.440 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.440 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.440 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:25.440 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221430+0000", 2024-09-21T15:01:25.441 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:01:25.441 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.441 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.441 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:25.441 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221438+0000", 2024-09-21T15:01:25.441 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2630000000000003e-06 2024-09-21T15:01:25.441 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.441 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.441 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:25.441 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221458+0000", 2024-09-21T15:01:25.441 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0071000000000001e-05 2024-09-21T15:01:25.441 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.441 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.441 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:25.441 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221467+0000", 2024-09-21T15:01:25.442 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.816e-06 2024-09-21T15:01:25.442 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.442 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.442 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:25.442 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221572+0000", 2024-09-21T15:01:25.442 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000105166 2024-09-21T15:01:25.442 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.442 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.442 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:25.442 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221589+0000", 2024-09-21T15:01:25.442 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7391e-05 2024-09-21T15:01:25.442 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.442 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.442 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:25.442 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.255741+0000", 2024-09-21T15:01:25.443 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0341519690000001 2024-09-21T15:01:25.443 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.443 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.443 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:25.443 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.256353+0000", 2024-09-21T15:01:25.443 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000611544 2024-09-21T15:01:25.443 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.443 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.443 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:25.443 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.256795+0000", 2024-09-21T15:01:25.443 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00044208700000000002 2024-09-21T15:01:25.443 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.443 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:25.443 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.443 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.444 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.444 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139773 3.1e 3:7fa3054f:::rbd_data.113664780c85.000000000000001c:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:25.444 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.221517+0000", 2024-09-21T15:01:25.444 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.190813919, 2024-09-21T15:01:25.444 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0367423140000001, 2024-09-21T15:01:25.444 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:25.444 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:25.444 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:25.444 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:25.444 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:25.444 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139773 2024-09-21T15:01:25.444 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.444 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:25.444 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.444 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:25.445 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221517+0000", 2024-09-21T15:01:25.445 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.445 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.445 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.445 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:25.445 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221517+0000", 2024-09-21T15:01:25.445 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.445 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.445 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.445 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:25.445 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221516+0000", 2024-09-21T15:01:25.445 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:25.445 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.445 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.445 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:25.445 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221520+0000", 2024-09-21T15:01:25.446 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1030000000000004e-06 2024-09-21T15:01:25.446 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.446 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.446 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:25.446 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221566+0000", 2024-09-21T15:01:25.446 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5512000000000001e-05 2024-09-21T15:01:25.446 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.446 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.446 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:25.446 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221575+0000", 2024-09-21T15:01:25.446 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9630000000000004e-06 2024-09-21T15:01:25.446 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.446 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.446 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:25.446 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221650+0000", 2024-09-21T15:01:25.447 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5531999999999994e-05 2024-09-21T15:01:25.447 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.447 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.447 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:25.447 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221661+0000", 2024-09-21T15:01:25.447 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0878e-05 2024-09-21T15:01:25.447 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.447 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.447 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:25.447 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.257513+0000", 2024-09-21T15:01:25.447 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0358513089999999 2024-09-21T15:01:25.447 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.447 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.447 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:25.447 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.257885+0000", 2024-09-21T15:01:25.448 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00037274299999999998 2024-09-21T15:01:25.448 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.448 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.448 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:25.448 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.258259+0000", 2024-09-21T15:01:25.448 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000373829 2024-09-21T15:01:25.448 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.448 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:25.448 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.448 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.448 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.448 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139789 3.1e 3:7c803367:::rbd_data.113664780c85.000000000000008c:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:25.448 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.221656+0000", 2024-09-21T15:01:25.448 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.190674288, 2024-09-21T15:01:25.448 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.037139595, 2024-09-21T15:01:25.449 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:25.449 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:25.449 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:25.449 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:25.449 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:25.449 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139789 2024-09-21T15:01:25.449 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.449 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:25.449 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.449 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:25.449 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221656+0000", 2024-09-21T15:01:25.449 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.449 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.449 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.449 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:25.450 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221656+0000", 2024-09-21T15:01:25.450 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.450 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.450 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.450 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:25.450 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221656+0000", 2024-09-21T15:01:25.450 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:25.450 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.450 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.450 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:25.450 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221660+0000", 2024-09-21T15:01:25.450 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0369999999999996e-06 2024-09-21T15:01:25.450 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.450 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.450 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:25.451 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221668+0000", 2024-09-21T15:01:25.451 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3440000000000001e-06 2024-09-21T15:01:25.451 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.451 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.451 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:25.451 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221674+0000", 2024-09-21T15:01:25.451 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0920000000000003e-06 2024-09-21T15:01:25.451 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.451 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.451 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:25.451 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221787+0000", 2024-09-21T15:01:25.451 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000112289 2024-09-21T15:01:25.451 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.451 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.451 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:25.452 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221798+0000", 2024-09-21T15:01:25.452 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1419e-05 2024-09-21T15:01:25.452 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.452 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.452 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:25.452 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.258642+0000", 2024-09-21T15:01:25.452 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.036843851 2024-09-21T15:01:25.452 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.452 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.452 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:25.452 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.258796+0000", 2024-09-21T15:01:25.452 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015414199999999999 2024-09-21T15:01:25.452 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.452 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:25.452 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.453 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.453 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.453 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139832 3.1e 3:7b675eeb:::rbd_data.113664780c85.00000000000001b3:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:25.453 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.223976+0000", 2024-09-21T15:01:25.453 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.188354555, 2024-09-21T15:01:25.453 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0350267980000001, 2024-09-21T15:01:25.453 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:25.453 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:25.453 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:25.453 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:25.453 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:25.453 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139832 2024-09-21T15:01:25.453 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.453 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:25.453 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.454 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:25.454 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223976+0000", 2024-09-21T15:01:25.454 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.454 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.454 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.454 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:25.454 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223976+0000", 2024-09-21T15:01:25.454 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.454 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.454 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.454 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:25.454 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223976+0000", 2024-09-21T15:01:25.454 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:25.454 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.454 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.455 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:25.455 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223981+0000", 2024-09-21T15:01:25.455 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1560000000000003e-06 2024-09-21T15:01:25.455 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.455 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.455 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:25.455 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223999+0000", 2024-09-21T15:01:25.455 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8808e-05 2024-09-21T15:01:25.455 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.455 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.455 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:25.455 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224008+0000", 2024-09-21T15:01:25.455 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7269999999999996e-06 2024-09-21T15:01:25.455 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.456 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.456 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:25.456 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224208+0000", 2024-09-21T15:01:25.457 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000199614 2024-09-21T15:01:25.457 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.457 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.457 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:25.457 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224218+0000", 2024-09-21T15:01:25.457 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0209e-05 2024-09-21T15:01:25.458 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.458 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.458 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:25.458 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.258839+0000", 2024-09-21T15:01:25.458 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.034621083 2024-09-21T15:01:25.458 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.458 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.458 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:25.458 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.259003+0000", 2024-09-21T15:01:25.458 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016379 2024-09-21T15:01:25.458 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.458 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:25.458 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.458 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.459 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.459 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139900 3.1e 3:7b440264:::rbd_data.113664780c85.0000000000000013:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:25.459 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.225748+0000", 2024-09-21T15:01:25.459 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.186582554000001, 2024-09-21T15:01:25.459 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0339966009999999, 2024-09-21T15:01:25.459 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:25.459 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:25.459 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:25.459 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:25.459 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:25.459 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139900 2024-09-21T15:01:25.459 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.459 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:25.459 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.459 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:25.460 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.225748+0000", 2024-09-21T15:01:25.460 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.460 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.460 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.460 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:25.460 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.225748+0000", 2024-09-21T15:01:25.460 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.460 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.460 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.460 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:25.460 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.225746+0000", 2024-09-21T15:01:25.460 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:25.460 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.460 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.460 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:25.460 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.225757+0000", 2024-09-21T15:01:25.461 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.066e-05 2024-09-21T15:01:25.461 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.461 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.461 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:25.461 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.225775+0000", 2024-09-21T15:01:25.461 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7631000000000001e-05 2024-09-21T15:01:25.461 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.461 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.461 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:25.461 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.225788+0000", 2024-09-21T15:01:25.461 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3046e-05 2024-09-21T15:01:25.461 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.461 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.461 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:25.461 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.225881+0000", 2024-09-21T15:01:25.461 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2987999999999998e-05 2024-09-21T15:01:25.462 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.462 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.462 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:25.462 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.225893+0000", 2024-09-21T15:01:25.462 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2398999999999999e-05 2024-09-21T15:01:25.462 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.462 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.462 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:25.462 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.259207+0000", 2024-09-21T15:01:25.462 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.033313696 2024-09-21T15:01:25.462 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.462 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.462 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:25.463 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.259452+0000", 2024-09-21T15:01:25.463 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00024495700000000002 2024-09-21T15:01:25.463 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.463 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.463 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:25.463 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.259745+0000", 2024-09-21T15:01:25.463 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000292868 2024-09-21T15:01:25.463 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.463 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:25.463 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.463 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.463 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.463 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140008 3.1e 3:7cd8b476:::rbd_data.113664780c85.0000000000000149:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:25.464 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.229555+0000", 2024-09-21T15:01:25.464 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.18277569, 2024-09-21T15:01:25.464 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0304914199999999, 2024-09-21T15:01:25.464 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:25.464 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:25.464 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:25.464 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:25.464 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:25.464 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140008 2024-09-21T15:01:25.464 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.464 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:25.464 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.464 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:25.464 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.229555+0000", 2024-09-21T15:01:25.465 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.465 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.465 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.465 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:25.465 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.229555+0000", 2024-09-21T15:01:25.465 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.465 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.465 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.465 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:25.465 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.229554+0000", 2024-09-21T15:01:25.465 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:25.465 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.465 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.465 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:25.465 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.229559+0000", 2024-09-21T15:01:25.466 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1059999999999999e-06 2024-09-21T15:01:25.466 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.466 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.466 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:25.466 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.229570+0000", 2024-09-21T15:01:25.466 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0465000000000001e-05 2024-09-21T15:01:25.466 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.466 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.466 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:25.466 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.229581+0000", 2024-09-21T15:01:25.466 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0958000000000001e-05 2024-09-21T15:01:25.467 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.467 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.467 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:25.467 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.229660+0000", 2024-09-21T15:01:25.467 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9103999999999993e-05 2024-09-21T15:01:25.467 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.467 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.467 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:25.467 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.229675+0000", 2024-09-21T15:01:25.467 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5622999999999999e-05 2024-09-21T15:01:25.467 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.468 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.468 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:25.468 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.259816+0000", 2024-09-21T15:01:25.468 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.030140595 2024-09-21T15:01:25.468 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.468 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.468 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:25.468 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.260046+0000", 2024-09-21T15:01:25.468 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00023045300000000001 2024-09-21T15:01:25.468 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.468 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:25.468 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.468 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.468 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.469 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140037 3.1e 3:7a4585d9:::rbd_data.113664780c85.000000000000005c:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:25.469 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.230439+0000", 2024-09-21T15:01:25.469 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.181891201999999, 2024-09-21T15:01:25.469 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.030277613, 2024-09-21T15:01:25.469 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:25.469 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:25.469 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:25.469 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:25.469 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:25.469 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140037 2024-09-21T15:01:25.469 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.469 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:25.469 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.470 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:25.470 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.230439+0000", 2024-09-21T15:01:25.470 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.470 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.470 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.470 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:25.470 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.230439+0000", 2024-09-21T15:01:25.470 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.470 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.470 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.470 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:25.470 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.230438+0000", 2024-09-21T15:01:25.470 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:25.471 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.471 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.471 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:25.471 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.230447+0000", 2024-09-21T15:01:25.471 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6270000000000006e-06 2024-09-21T15:01:25.471 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.471 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.471 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:25.471 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.230464+0000", 2024-09-21T15:01:25.471 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7394e-05 2024-09-21T15:01:25.471 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.471 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.471 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:25.471 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.230472+0000", 2024-09-21T15:01:25.471 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0360000000000006e-06 2024-09-21T15:01:25.472 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.472 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.472 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:25.472 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.230567+0000", 2024-09-21T15:01:25.472 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4648000000000006e-05 2024-09-21T15:01:25.472 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.472 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.472 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:25.472 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.230581+0000", 2024-09-21T15:01:25.472 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3799e-05 2024-09-21T15:01:25.472 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.472 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.472 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:25.472 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.260095+0000", 2024-09-21T15:01:25.473 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.029514153 2024-09-21T15:01:25.473 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.473 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.473 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:25.473 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.260356+0000", 2024-09-21T15:01:25.473 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00026118400000000003 2024-09-21T15:01:25.473 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.473 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.473 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:25.473 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.260717+0000", 2024-09-21T15:01:25.473 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00036124799999999999 2024-09-21T15:01:25.473 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.473 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:25.473 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.473 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.473 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.474 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140061 3.1e 3:7fdd08f8:::rbd_data.113664780c85.0000000000000104:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:25.474 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.232026+0000", 2024-09-21T15:01:25.474 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.180304872000001, 2024-09-21T15:01:25.474 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0290620450000001, 2024-09-21T15:01:25.474 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:25.474 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:25.474 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:25.474 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:25.474 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:25.474 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140061 2024-09-21T15:01:25.474 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.474 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:25.474 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.474 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:25.475 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232026+0000", 2024-09-21T15:01:25.475 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.475 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.475 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.475 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:25.475 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232026+0000", 2024-09-21T15:01:25.475 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.475 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.475 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.475 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:25.475 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232024+0000", 2024-09-21T15:01:25.475 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:01:25.475 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.475 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.475 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:25.476 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232034+0000", 2024-09-21T15:01:25.476 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0421000000000001e-05 2024-09-21T15:01:25.476 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.476 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.476 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:25.476 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232058+0000", 2024-09-21T15:01:25.476 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3944e-05 2024-09-21T15:01:25.476 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.476 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.476 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:25.476 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232071+0000", 2024-09-21T15:01:25.476 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2816999999999999e-05 2024-09-21T15:01:25.476 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.476 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.476 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:25.476 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232180+0000", 2024-09-21T15:01:25.477 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000109323 2024-09-21T15:01:25.477 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.477 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.477 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:25.477 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232199+0000", 2024-09-21T15:01:25.477 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8606000000000001e-05 2024-09-21T15:01:25.477 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.477 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.477 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:25.477 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.260821+0000", 2024-09-21T15:01:25.477 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0286222270000001 2024-09-21T15:01:25.477 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.477 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.477 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:25.477 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.261088+0000", 2024-09-21T15:01:25.477 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00026663799999999998 2024-09-21T15:01:25.478 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.478 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:25.478 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.478 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.478 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.478 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140090 3.1e 3:79f99893:::rbd_data.113664780c85.000000000000000b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:25.478 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.233668+0000", 2024-09-21T15:01:25.478 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.178663007000001, 2024-09-21T15:01:25.478 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0281047409999999, 2024-09-21T15:01:25.478 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:25.478 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:25.478 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:25.478 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:25.478 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:25.478 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140090 2024-09-21T15:01:25.479 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.479 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:25.479 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.479 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:25.479 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233668+0000", 2024-09-21T15:01:25.479 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.479 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.479 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.479 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:25.479 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233668+0000", 2024-09-21T15:01:25.479 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.479 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.479 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.479 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:25.480 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233666+0000", 2024-09-21T15:01:25.480 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:25.480 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.480 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.480 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:25.480 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233673+0000", 2024-09-21T15:01:25.480 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1840000000000002e-06 2024-09-21T15:01:25.480 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.480 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.480 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:25.480 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233689+0000", 2024-09-21T15:01:25.480 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5366000000000002e-05 2024-09-21T15:01:25.480 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.480 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.481 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:25.481 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233697+0000", 2024-09-21T15:01:25.481 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7679999999999997e-06 2024-09-21T15:01:25.481 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.481 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.481 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:25.481 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233772+0000", 2024-09-21T15:01:25.481 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4832000000000005e-05 2024-09-21T15:01:25.481 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.481 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.481 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:25.481 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233790+0000", 2024-09-21T15:01:25.481 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7297999999999999e-05 2024-09-21T15:01:25.481 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.482 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.482 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:25.482 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.261145+0000", 2024-09-21T15:01:25.482 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.027355381 2024-09-21T15:01:25.482 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.482 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.482 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:25.482 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.261466+0000", 2024-09-21T15:01:25.482 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00032105300000000002 2024-09-21T15:01:25.482 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.482 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.482 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:25.482 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.261772+0000", 2024-09-21T15:01:25.482 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00030635100000000001 2024-09-21T15:01:25.483 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.483 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:25.483 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.483 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.483 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.483 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140128 3.1e 3:78baf82e:::rbd_data.113664780c85.0000000000000115:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:25.483 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.235029+0000", 2024-09-21T15:01:25.483 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.177301752999998, 2024-09-21T15:01:25.483 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.02698942, 2024-09-21T15:01:25.483 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:25.483 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:25.483 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:25.483 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:25.483 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:25.484 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140128 2024-09-21T15:01:25.484 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.484 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:25.484 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.484 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:25.484 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235029+0000", 2024-09-21T15:01:25.484 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.484 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.484 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.484 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:25.484 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235029+0000", 2024-09-21T15:01:25.484 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.484 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.484 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.484 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:25.484 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235028+0000", 2024-09-21T15:01:25.485 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:25.485 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.485 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.485 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:25.485 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235032+0000", 2024-09-21T15:01:25.485 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0579999999999999e-06 2024-09-21T15:01:25.485 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.485 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.485 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:25.485 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235040+0000", 2024-09-21T15:01:25.485 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2099999999999993e-06 2024-09-21T15:01:25.485 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.485 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.485 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:25.485 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235047+0000", 2024-09-21T15:01:25.486 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1099999999999999e-06 2024-09-21T15:01:25.486 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.486 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.486 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:25.486 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235123+0000", 2024-09-21T15:01:25.486 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6835999999999994e-05 2024-09-21T15:01:25.486 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.486 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.486 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:25.486 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235136+0000", 2024-09-21T15:01:25.486 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2962000000000001e-05 2024-09-21T15:01:25.486 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.486 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.486 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:25.486 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.261827+0000", 2024-09-21T15:01:25.487 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0266906149999999 2024-09-21T15:01:25.487 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.487 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.487 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:25.487 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.262018+0000", 2024-09-21T15:01:25.487 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019132400000000001 2024-09-21T15:01:25.487 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.487 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:25.487 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.487 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.487 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.487 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140217 3.1e 3:793dcf44:::rbd_data.113664780c85.0000000000000000:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:25.487 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.235497+0000", 2024-09-21T15:01:25.487 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.176833454000001, 2024-09-21T15:01:25.488 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0269053020000001, 2024-09-21T15:01:25.488 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:25.488 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:25.488 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:25.488 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:25.488 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:25.488 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140217 2024-09-21T15:01:25.488 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.488 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:25.488 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.488 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:25.488 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235497+0000", 2024-09-21T15:01:25.488 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.488 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.488 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.488 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:25.489 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235497+0000", 2024-09-21T15:01:25.489 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.489 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.489 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.489 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:25.489 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235496+0000", 2024-09-21T15:01:25.489 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:25.489 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.489 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.489 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:25.489 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235503+0000", 2024-09-21T15:01:25.489 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8460000000000002e-06 2024-09-21T15:01:25.489 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.489 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.490 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:25.490 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235514+0000", 2024-09-21T15:01:25.490 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0576e-05 2024-09-21T15:01:25.490 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.490 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.490 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:25.490 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235522+0000", 2024-09-21T15:01:25.490 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2640000000000008e-06 2024-09-21T15:01:25.490 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.490 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.490 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:25.490 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235579+0000", 2024-09-21T15:01:25.490 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6842999999999998e-05 2024-09-21T15:01:25.490 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.490 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.491 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:25.491 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235590+0000", 2024-09-21T15:01:25.491 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1194e-05 2024-09-21T15:01:25.491 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.491 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.491 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:25.491 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.262119+0000", 2024-09-21T15:01:25.491 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0265294540000001 2024-09-21T15:01:25.491 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.491 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.491 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:25.491 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.262241+0000", 2024-09-21T15:01:25.491 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012182 2024-09-21T15:01:25.491 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.491 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.491 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:25.492 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.262403+0000", 2024-09-21T15:01:25.492 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000161442 2024-09-21T15:01:25.492 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.492 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:25.492 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.492 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.492 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.492 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140221 3.1e 3:7fa3054f:::rbd_data.113664780c85.000000000000001c:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:25.492 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.235592+0000", 2024-09-21T15:01:25.492 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.176738593, 2024-09-21T15:01:25.492 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.027322012, 2024-09-21T15:01:25.492 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:25.492 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:25.492 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:25.492 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:25.493 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:25.493 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140221 2024-09-21T15:01:25.493 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.493 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:25.493 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.493 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:25.493 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235592+0000", 2024-09-21T15:01:25.493 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.493 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.493 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.493 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:25.493 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235592+0000", 2024-09-21T15:01:25.493 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.493 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.493 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.493 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:25.494 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235591+0000", 2024-09-21T15:01:25.494 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:25.494 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.494 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.494 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:25.494 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235596+0000", 2024-09-21T15:01:25.494 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4209999999999997e-06 2024-09-21T15:01:25.494 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.494 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.494 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:25.494 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235603+0000", 2024-09-21T15:01:25.494 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.909e-06 2024-09-21T15:01:25.494 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.494 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.495 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:25.495 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235609+0000", 2024-09-21T15:01:25.495 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3539999999999999e-06 2024-09-21T15:01:25.495 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.495 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.495 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:25.495 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236013+0000", 2024-09-21T15:01:25.495 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00040399799999999998 2024-09-21T15:01:25.495 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.495 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.495 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:25.495 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236020+0000", 2024-09-21T15:01:25.495 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6699999999999994e-06 2024-09-21T15:01:25.495 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.495 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.496 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:25.496 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.262452+0000", 2024-09-21T15:01:25.496 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.02643145 2024-09-21T15:01:25.496 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.496 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.496 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:25.496 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.262624+0000", 2024-09-21T15:01:25.496 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000172206 2024-09-21T15:01:25.496 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.496 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.496 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:25.496 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.262914+0000", 2024-09-21T15:01:25.496 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00028994600000000002 2024-09-21T15:01:25.496 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.496 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:25.496 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.497 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.497 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.497 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140237 3.1e 3:7c803367:::rbd_data.113664780c85.000000000000008c:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:25.497 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.237113+0000", 2024-09-21T15:01:25.497 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.175218086000001, 2024-09-21T15:01:25.497 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0260336299999999, 2024-09-21T15:01:25.497 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:25.497 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:25.497 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:25.497 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:25.497 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:25.497 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140237 2024-09-21T15:01:25.497 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.497 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:25.497 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.498 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:25.498 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237113+0000", 2024-09-21T15:01:25.498 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.498 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.498 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.498 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:25.498 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237113+0000", 2024-09-21T15:01:25.498 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.498 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.498 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.498 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:25.498 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237112+0000", 2024-09-21T15:01:25.498 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:25.499 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.499 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.499 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:25.499 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237118+0000", 2024-09-21T15:01:25.499 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8930000000000002e-06 2024-09-21T15:01:25.499 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.499 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.499 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:25.499 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237126+0000", 2024-09-21T15:01:25.499 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6729999999999998e-06 2024-09-21T15:01:25.499 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.499 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.499 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:25.499 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237135+0000", 2024-09-21T15:01:25.500 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0929999999999992e-06 2024-09-21T15:01:25.500 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.500 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.500 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:25.500 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237205+0000", 2024-09-21T15:01:25.500 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9697999999999996e-05 2024-09-21T15:01:25.500 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.500 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.500 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:25.500 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237218+0000", 2024-09-21T15:01:25.500 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2524000000000001e-05 2024-09-21T15:01:25.500 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.500 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.500 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:25.500 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.262965+0000", 2024-09-21T15:01:25.501 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0257478630000001 2024-09-21T15:01:25.501 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.501 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.501 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:25.501 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.263146+0000", 2024-09-21T15:01:25.501 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018074599999999999 2024-09-21T15:01:25.501 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.501 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:25.501 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.501 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.501 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.501 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140280 3.1e 3:7b675eeb:::rbd_data.113664780c85.00000000000001b3:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:25.501 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.237477+0000", 2024-09-21T15:01:25.501 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.174853823999999, 2024-09-21T15:01:25.501 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.026437329, 2024-09-21T15:01:25.501 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:25.502 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:25.502 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:25.502 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:25.502 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:25.502 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140280 2024-09-21T15:01:25.502 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.502 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:25.502 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.502 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:25.502 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237477+0000", 2024-09-21T15:01:25.502 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.502 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.502 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.502 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:25.503 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237477+0000", 2024-09-21T15:01:25.503 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.503 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.503 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.503 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:25.503 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237476+0000", 2024-09-21T15:01:25.503 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:25.503 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.503 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.503 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:25.503 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237480+0000", 2024-09-21T15:01:25.503 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7570000000000002e-06 2024-09-21T15:01:25.503 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.504 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.504 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:25.504 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237490+0000", 2024-09-21T15:01:25.504 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3170000000000008e-06 2024-09-21T15:01:25.504 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.504 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.504 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:25.504 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237496+0000", 2024-09-21T15:01:25.504 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7150000000000001e-06 2024-09-21T15:01:25.504 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.505 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.505 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:25.505 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237559+0000", 2024-09-21T15:01:25.505 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2844999999999999e-05 2024-09-21T15:01:25.505 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.505 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.505 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:25.505 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237570+0000", 2024-09-21T15:01:25.505 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1253000000000001e-05 2024-09-21T15:01:25.505 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.505 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.505 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:25.505 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.263203+0000", 2024-09-21T15:01:25.505 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0256330339999999 2024-09-21T15:01:25.505 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.506 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.506 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:25.506 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.263914+0000", 2024-09-21T15:01:25.506 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00071080899999999996 2024-09-21T15:01:25.506 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.506 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:25.506 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.506 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.506 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.506 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140348 3.1e 3:7b440264:::rbd_data.113664780c85.0000000000000013:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:25.506 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.241762+0000", 2024-09-21T15:01:25.506 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.170568552999999, 2024-09-21T15:01:25.506 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0221355759999999, 2024-09-21T15:01:25.506 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:25.506 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:25.507 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:25.507 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:25.507 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:25.507 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140348 2024-09-21T15:01:25.507 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.507 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:25.507 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.507 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:25.507 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.241762+0000", 2024-09-21T15:01:25.507 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.507 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.507 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.508 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:25.508 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.241762+0000", 2024-09-21T15:01:25.508 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.508 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.508 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.508 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:25.508 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.241761+0000", 2024-09-21T15:01:25.508 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:25.508 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.508 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.508 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:25.508 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.241767+0000", 2024-09-21T15:01:25.508 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4219999999999997e-06 2024-09-21T15:01:25.508 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.509 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.509 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:25.509 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.241783+0000", 2024-09-21T15:01:25.509 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5959000000000001e-05 2024-09-21T15:01:25.509 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.509 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.509 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:25.509 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.241793+0000", 2024-09-21T15:01:25.509 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3999999999999998e-06 2024-09-21T15:01:25.509 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.509 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.509 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:25.509 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.241894+0000", 2024-09-21T15:01:25.509 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000100976 2024-09-21T15:01:25.509 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.509 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.510 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:25.510 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.241907+0000", 2024-09-21T15:01:25.510 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3281e-05 2024-09-21T15:01:25.510 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.510 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.510 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:25.510 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.263430+0000", 2024-09-21T15:01:25.510 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0215229379999999 2024-09-21T15:01:25.510 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.510 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.510 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:25.510 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.263604+0000", 2024-09-21T15:01:25.510 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017368700000000001 2024-09-21T15:01:25.510 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.511 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.511 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:25.511 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.263898+0000", 2024-09-21T15:01:25.511 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00029406899999999999 2024-09-21T15:01:25.511 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.511 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:25.511 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.511 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.511 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:25.511 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:01:25.512 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T15:01:25.513 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:01:25.508+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:01:25.613 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:01:25.608+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:01:25.627 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:01:25.627 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:01:25.627 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T15:01:25.627 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:01:25.637 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T15:01:25.714 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:01:25.710+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:01:25.746 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:01:25.746 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:01:25.747 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T15:01:25.747 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T15:01:25.747 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:01:25.756 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T15:01:25.815 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:01:25.812+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:01:25.866 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:01:25.866 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T15:01:25.866 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T15:01:25.866 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T15:01:25.867 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.867 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:97677 3.15 e122/24 3:a8e72133:::rbd_data.1136bfc79a70.0000000000000005:head v 122'567, mlcod=120'565)", 2024-09-21T15:01:25.867 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.320753+0000", 2024-09-21T15:01:25.867 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 80.542623160999995, 2024-09-21T15:01:25.867 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070639375000000004, 2024-09-21T15:01:25.867 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:25.867 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:25.867 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:25.867 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.867 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:25.867 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.320753+0000", 2024-09-21T15:01:25.867 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.867 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.867 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.868 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:25.868 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.320753+0000", 2024-09-21T15:01:25.868 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.868 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.868 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.868 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:25.868 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.320753+0000", 2024-09-21T15:01:25.868 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:25.868 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.868 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.868 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:25.868 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.375452+0000", 2024-09-21T15:01:25.868 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054698945999999998 2024-09-21T15:01:25.869 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.869 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.869 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:25.869 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.375476+0000", 2024-09-21T15:01:25.869 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4380999999999999e-05 2024-09-21T15:01:25.869 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.869 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.869 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:25.869 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.375485+0000", 2024-09-21T15:01:25.869 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0049999999999993e-06 2024-09-21T15:01:25.869 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.869 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.869 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:25.869 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.375552+0000", 2024-09-21T15:01:25.869 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6550999999999994e-05 2024-09-21T15:01:25.870 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.870 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.870 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:25.870 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.375578+0000", 2024-09-21T15:01:25.870 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6196000000000001e-05 2024-09-21T15:01:25.870 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.870 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.870 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:01:25.870 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.391319+0000", 2024-09-21T15:01:25.870 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015741271000000001 2024-09-21T15:01:25.871 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.871 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.871 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:25.871 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.391393+0000", 2024-09-21T15:01:25.871 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3287000000000005e-05 2024-09-21T15:01:25.871 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.871 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:25.871 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.871 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.871 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.871 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:97672 3.4 e122/51 3:26046d29:::rbd_data.1136bfc79a70.0000000000000001:head v 122'256, mlcod=120'254)", 2024-09-21T15:01:25.871 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.331696+0000", 2024-09-21T15:01:25.872 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 80.531679998000001, 2024-09-21T15:01:25.872 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041222507999999998, 2024-09-21T15:01:25.872 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:25.872 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:25.872 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:25.872 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.873 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:25.873 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.331696+0000", 2024-09-21T15:01:25.873 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.873 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.873 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.873 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:25.873 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.331696+0000", 2024-09-21T15:01:25.873 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.873 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.873 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.873 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:25.873 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.331695+0000", 2024-09-21T15:01:25.873 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:25.874 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.874 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.874 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:25.874 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.340175+0000", 2024-09-21T15:01:25.874 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0084799520000000007 2024-09-21T15:01:25.874 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.874 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.874 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:25.874 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.340208+0000", 2024-09-21T15:01:25.875 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2997999999999998e-05 2024-09-21T15:01:25.875 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.875 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.875 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:25.875 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.340223+0000", 2024-09-21T15:01:25.875 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5330999999999999e-05 2024-09-21T15:01:25.875 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.875 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.875 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:25.875 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.340305+0000", 2024-09-21T15:01:25.875 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2095999999999995e-05 2024-09-21T15:01:25.875 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.875 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.875 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:25.876 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.340337+0000", 2024-09-21T15:01:25.876 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2097000000000001e-05 2024-09-21T15:01:25.876 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.876 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.876 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:01:25.876 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.372863+0000", 2024-09-21T15:01:25.876 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.032525923999999998 2024-09-21T15:01:25.876 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.876 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.876 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:25.876 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.372919+0000", 2024-09-21T15:01:25.876 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5729000000000001e-05 2024-09-21T15:01:25.876 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.876 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:25.876 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.876 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.877 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.877 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:116182 3.12 3:4efbaa3f:::rbd_data.1136b1194d20.0000000000000006:58 [read 6242304~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:25.877 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.712538+0000", 2024-09-21T15:01:25.877 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 56.150838006999997, 2024-09-21T15:01:25.877 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034702127999999999, 2024-09-21T15:01:25.877 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:25.877 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:25.877 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:25.877 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:25.877 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:25.877 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116182 2024-09-21T15:01:25.877 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.877 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:25.877 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.877 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:25.877 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712538+0000", 2024-09-21T15:01:25.878 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.878 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.878 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.878 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:25.878 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712538+0000", 2024-09-21T15:01:25.878 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.878 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.878 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.878 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:25.878 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712538+0000", 2024-09-21T15:01:25.879 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:25.879 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.879 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.879 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:25.879 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712541+0000", 2024-09-21T15:01:25.879 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5330000000000001e-06 2024-09-21T15:01:25.879 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.879 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.879 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:25.879 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712547+0000", 2024-09-21T15:01:25.879 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1109999999999996e-06 2024-09-21T15:01:25.879 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.879 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.880 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:25.880 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712551+0000", 2024-09-21T15:01:25.880 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.685e-06 2024-09-21T15:01:25.880 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.880 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.880 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:25.880 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.747067+0000", 2024-09-21T15:01:25.880 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034516181 2024-09-21T15:01:25.880 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.880 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.880 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:25.880 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.747152+0000", 2024-09-21T15:01:25.880 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4075999999999999e-05 2024-09-21T15:01:25.880 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.881 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.881 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:25.881 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.747240+0000", 2024-09-21T15:01:25.881 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8924000000000004e-05 2024-09-21T15:01:25.881 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.881 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:25.881 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.881 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.881 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.881 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139784 3.1 3:862e0e65:::rbd_data.113664780c85.0000000000000063:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:25.881 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.222486+0000", 2024-09-21T15:01:25.881 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.64089014, 2024-09-21T15:01:25.881 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0141793059999999, 2024-09-21T15:01:25.881 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:25.881 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:25.882 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:25.882 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:25.882 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:25.882 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139784 2024-09-21T15:01:25.882 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.882 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:25.882 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.882 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:25.882 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222486+0000", 2024-09-21T15:01:25.882 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.882 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.882 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.882 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:25.882 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222486+0000", 2024-09-21T15:01:25.882 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.883 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.883 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.883 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:25.883 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222484+0000", 2024-09-21T15:01:25.883 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-21T15:01:25.883 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.883 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.883 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:25.883 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222494+0000", 2024-09-21T15:01:25.883 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0676000000000001e-05 2024-09-21T15:01:25.883 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.883 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.883 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:25.883 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222517+0000", 2024-09-21T15:01:25.883 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2371e-05 2024-09-21T15:01:25.884 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.884 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.884 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:25.884 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222527+0000", 2024-09-21T15:01:25.884 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0461e-05 2024-09-21T15:01:25.884 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.884 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.884 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:25.884 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222625+0000", 2024-09-21T15:01:25.884 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.8160999999999997e-05 2024-09-21T15:01:25.884 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.885 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.885 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:25.885 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222641+0000", 2024-09-21T15:01:25.885 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5899000000000001e-05 2024-09-21T15:01:25.885 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.885 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.885 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:25.885 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.236479+0000", 2024-09-21T15:01:25.885 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0138377709999999 2024-09-21T15:01:25.885 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.885 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.885 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:25.885 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.236666+0000", 2024-09-21T15:01:25.885 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018648199999999999 2024-09-21T15:01:25.885 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.886 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:25.886 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.886 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.886 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.886 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139812 3.1 3:8407fb9b:::rbd_data.113664780c85.0000000000000127:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:25.886 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.222547+0000", 2024-09-21T15:01:25.886 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.640829216, 2024-09-21T15:01:25.886 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.014796059, 2024-09-21T15:01:25.886 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:25.886 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:25.886 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:25.886 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:25.886 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:25.886 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139812 2024-09-21T15:01:25.887 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.887 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:25.887 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.887 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:25.887 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222547+0000", 2024-09-21T15:01:25.887 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.887 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.887 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.887 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:25.887 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222547+0000", 2024-09-21T15:01:25.887 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.887 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.887 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.887 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:25.887 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222546+0000", 2024-09-21T15:01:25.888 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:25.888 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.888 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.888 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:25.888 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222552+0000", 2024-09-21T15:01:25.888 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2739999999999998e-06 2024-09-21T15:01:25.888 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.888 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.888 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:25.888 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222578+0000", 2024-09-21T15:01:25.888 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6594999999999999e-05 2024-09-21T15:01:25.888 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.888 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.888 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:25.888 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222596+0000", 2024-09-21T15:01:25.888 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7838000000000001e-05 2024-09-21T15:01:25.889 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.889 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.889 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:25.889 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222701+0000", 2024-09-21T15:01:25.889 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000104714 2024-09-21T15:01:25.889 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.889 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.889 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:25.889 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222714+0000", 2024-09-21T15:01:25.889 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2998e-05 2024-09-21T15:01:25.889 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.889 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.890 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:25.890 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.237124+0000", 2024-09-21T15:01:25.890 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.014409806 2024-09-21T15:01:25.890 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.890 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.890 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:25.890 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.237343+0000", 2024-09-21T15:01:25.890 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021950499999999999 2024-09-21T15:01:25.890 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.890 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:25.890 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.890 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.890 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.890 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139826 3.1 3:854f6f9b:::rbd_data.113664780c85.0000000000000189:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:25.890 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.222611+0000", 2024-09-21T15:01:25.891 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.640765124000001, 2024-09-21T15:01:25.891 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.014916828, 2024-09-21T15:01:25.891 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:25.891 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:25.891 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:25.891 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:25.891 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:25.891 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139826 2024-09-21T15:01:25.891 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.891 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:25.891 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.891 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:25.891 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222611+0000", 2024-09-21T15:01:25.891 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.892 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.892 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.892 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:25.892 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222611+0000", 2024-09-21T15:01:25.892 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.892 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.892 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.892 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:25.892 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222610+0000", 2024-09-21T15:01:25.892 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:25.892 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.892 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.892 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:25.892 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222618+0000", 2024-09-21T15:01:25.892 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6690000000000006e-06 2024-09-21T15:01:25.893 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.893 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.893 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:25.893 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222632+0000", 2024-09-21T15:01:25.893 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4433999999999999e-05 2024-09-21T15:01:25.893 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.893 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.893 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:25.893 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222640+0000", 2024-09-21T15:01:25.893 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6260000000000003e-06 2024-09-21T15:01:25.893 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.893 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.893 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:25.893 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222758+0000", 2024-09-21T15:01:25.894 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011826499999999999 2024-09-21T15:01:25.894 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.894 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.894 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:25.894 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222776+0000", 2024-09-21T15:01:25.894 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7564999999999999e-05 2024-09-21T15:01:25.894 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.894 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.894 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:25.894 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.237402+0000", 2024-09-21T15:01:25.894 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0146261889999999 2024-09-21T15:01:25.894 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.894 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.894 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:25.895 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.237528+0000", 2024-09-21T15:01:25.895 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000126212 2024-09-21T15:01:25.895 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.895 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:25.895 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.895 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.895 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.895 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139864 3.1 3:86b14cb7:::rbd_data.113664780c85.00000000000000d5:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:25.895 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.223966+0000", 2024-09-21T15:01:25.895 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.639410388000002, 2024-09-21T15:01:25.895 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.013859007, 2024-09-21T15:01:25.895 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:25.896 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:25.896 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:25.896 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:25.896 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:25.896 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139864 2024-09-21T15:01:25.896 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.896 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:25.896 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.896 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:25.896 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223966+0000", 2024-09-21T15:01:25.896 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.896 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.896 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.896 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:25.897 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223966+0000", 2024-09-21T15:01:25.897 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.897 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.897 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.897 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:25.897 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223965+0000", 2024-09-21T15:01:25.897 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:25.897 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.897 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.898 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:25.898 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223970+0000", 2024-09-21T15:01:25.898 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3769999999999998e-06 2024-09-21T15:01:25.898 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.898 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.898 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:25.898 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223979+0000", 2024-09-21T15:01:25.898 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5899999999999997e-06 2024-09-21T15:01:25.898 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.898 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.899 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:25.899 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223985+0000", 2024-09-21T15:01:25.899 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5790000000000003e-06 2024-09-21T15:01:25.899 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.899 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.899 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:25.899 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224073+0000", 2024-09-21T15:01:25.899 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7781000000000005e-05 2024-09-21T15:01:25.899 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.899 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.899 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:25.899 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224110+0000", 2024-09-21T15:01:25.899 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7060000000000001e-05 2024-09-21T15:01:25.900 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.900 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.900 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:25.900 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.237599+0000", 2024-09-21T15:01:25.900 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.01348953 2024-09-21T15:01:25.900 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.900 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.900 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:25.900 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.237825+0000", 2024-09-21T15:01:25.900 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000225758 2024-09-21T15:01:25.900 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.900 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:25.900 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.900 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.901 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.901 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139877 3.1 3:87277f92:::rbd_data.113664780c85.0000000000000136:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:25.901 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.224489+0000", 2024-09-21T15:01:25.901 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.638887530000002, 2024-09-21T15:01:25.901 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0137916330000001, 2024-09-21T15:01:25.901 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:25.901 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:25.901 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:25.901 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:25.901 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:25.901 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139877 2024-09-21T15:01:25.902 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.902 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:25.902 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.902 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:25.902 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224489+0000", 2024-09-21T15:01:25.902 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.902 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.902 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.902 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:25.902 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224489+0000", 2024-09-21T15:01:25.902 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.902 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.902 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.902 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:25.903 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224488+0000", 2024-09-21T15:01:25.903 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:25.903 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.903 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.903 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:25.903 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224502+0000", 2024-09-21T15:01:25.903 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4194e-05 2024-09-21T15:01:25.903 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.903 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.903 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:25.903 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224522+0000", 2024-09-21T15:01:25.903 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0038e-05 2024-09-21T15:01:25.903 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.903 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.903 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:25.904 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224531+0000", 2024-09-21T15:01:25.904 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6900000000000004e-06 2024-09-21T15:01:25.904 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.904 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.904 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:25.904 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224621+0000", 2024-09-21T15:01:25.904 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9654000000000006e-05 2024-09-21T15:01:25.904 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.904 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.904 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:25.904 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224641+0000", 2024-09-21T15:01:25.904 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9375e-05 2024-09-21T15:01:25.904 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.904 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.904 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:25.905 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.238027+0000", 2024-09-21T15:01:25.905 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0133866840000001 2024-09-21T15:01:25.905 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.905 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.905 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:25.905 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.238280+0000", 2024-09-21T15:01:25.905 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000253281 2024-09-21T15:01:25.905 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.905 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:25.905 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.905 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.905 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.906 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139887 3.1 3:80f1e8f1:::rbd_data.113664780c85.000000000000017c:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:25.906 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.224549+0000", 2024-09-21T15:01:25.906 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.638827240000001, 2024-09-21T15:01:25.906 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0139776060000001, 2024-09-21T15:01:25.906 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:25.906 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:25.906 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:25.906 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:25.906 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:25.906 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139887 2024-09-21T15:01:25.906 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.906 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:25.906 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.906 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:25.907 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224549+0000", 2024-09-21T15:01:25.907 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.907 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.907 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.907 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:25.907 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224549+0000", 2024-09-21T15:01:25.907 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.907 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.907 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.907 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:25.907 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224548+0000", 2024-09-21T15:01:25.907 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:25.907 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.907 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.908 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:25.908 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224554+0000", 2024-09-21T15:01:25.908 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4049999999999999e-06 2024-09-21T15:01:25.908 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.908 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.908 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:25.908 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224565+0000", 2024-09-21T15:01:25.908 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0879e-05 2024-09-21T15:01:25.908 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.908 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.908 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:25.908 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224572+0000", 2024-09-21T15:01:25.908 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0380000000000002e-06 2024-09-21T15:01:25.909 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.909 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.909 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:25.909 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224684+0000", 2024-09-21T15:01:25.909 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011229 2024-09-21T15:01:25.909 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.909 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.909 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:25.909 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224698+0000", 2024-09-21T15:01:25.909 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4448e-05 2024-09-21T15:01:25.909 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.910 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.910 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:25.910 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.238359+0000", 2024-09-21T15:01:25.910 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0136610660000001 2024-09-21T15:01:25.910 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.910 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.910 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:25.910 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.238527+0000", 2024-09-21T15:01:25.910 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016731100000000001 2024-09-21T15:01:25.910 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.910 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:25.910 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.911 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.911 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.911 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140077 3.1 3:85642daf:::rbd_data.113664780c85.0000000000000174:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:25.911 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.232797+0000", 2024-09-21T15:01:25.911 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.630579525000002, 2024-09-21T15:01:25.911 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.006029955, 2024-09-21T15:01:25.911 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:25.911 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:25.911 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:25.911 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:25.911 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:25.911 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140077 2024-09-21T15:01:25.912 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.912 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:25.912 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.912 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:25.912 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232797+0000", 2024-09-21T15:01:25.912 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.912 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.912 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.912 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:25.912 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232797+0000", 2024-09-21T15:01:25.912 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.913 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.913 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.913 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:25.913 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232795+0000", 2024-09-21T15:01:25.913 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:25.913 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.913 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.913 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:25.913 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232804+0000", 2024-09-21T15:01:25.913 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9670000000000007e-06 2024-09-21T15:01:25.913 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.914 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.914 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:25.914 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232824+0000", 2024-09-21T15:01:25.914 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9238000000000001e-05 2024-09-21T15:01:25.914 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.914 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.914 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:25.914 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232843+0000", 2024-09-21T15:01:25.914 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9256e-05 2024-09-21T15:01:25.914 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.914 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.914 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:25.914 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232912+0000", 2024-09-21T15:01:25.915 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9589000000000001e-05 2024-09-21T15:01:25.915 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.915 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.915 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:25.915 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232925+0000", 2024-09-21T15:01:25.915 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2585999999999999e-05 2024-09-21T15:01:25.915 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.915 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.915 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:25.915 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.238591+0000", 2024-09-21T15:01:25.915 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0056659939999999 2024-09-21T15:01:25.916 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.916 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.916 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:25.916 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.238827+0000", 2024-09-21T15:01:25.916 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00023583999999999999 2024-09-21T15:01:25.916 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.916 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:25.916 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.916 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.916 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.916 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140091 3.1 3:8358f0db:::rbd_data.113664780c85.0000000000000018:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:25.916 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.233637+0000", 2024-09-21T15:01:25.916 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.629739228999998, 2024-09-21T15:01:25.917 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.006272847, 2024-09-21T15:01:25.917 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:25.917 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:25.917 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:25.917 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:25.917 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:25.917 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140091 2024-09-21T15:01:25.917 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.917 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:25.917 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.917 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:25.917 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233637+0000", 2024-09-21T15:01:25.917 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.918 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.918 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.918 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:25.918 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233637+0000", 2024-09-21T15:01:25.918 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.918 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.918 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.918 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:25.918 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233636+0000", 2024-09-21T15:01:25.918 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:25.918 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.918 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.918 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:25.918 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233644+0000", 2024-09-21T15:01:25.918 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1380000000000007e-06 2024-09-21T15:01:25.919 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.919 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.919 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:25.919 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233662+0000", 2024-09-21T15:01:25.919 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8224999999999999e-05 2024-09-21T15:01:25.919 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.919 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.919 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:25.919 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233674+0000", 2024-09-21T15:01:25.919 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2081e-05 2024-09-21T15:01:25.919 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.919 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.919 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:25.919 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233750+0000", 2024-09-21T15:01:25.920 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6106000000000006e-05 2024-09-21T15:01:25.920 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.920 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.920 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:25.920 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233768+0000", 2024-09-21T15:01:25.920 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7285000000000001e-05 2024-09-21T15:01:25.920 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.920 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.920 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:25.920 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.239196+0000", 2024-09-21T15:01:25.920 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.005428223 2024-09-21T15:01:25.920 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.920 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.920 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:25.921 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.239528+0000", 2024-09-21T15:01:25.921 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00033262199999999997 2024-09-21T15:01:25.921 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.921 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.921 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:25.921 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.239910+0000", 2024-09-21T15:01:25.921 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00038151100000000001 2024-09-21T15:01:25.921 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.921 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:25.921 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.921 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.921 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.921 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140109 3.1 3:8586e9d9:::rbd_data.113664780c85.0000000000000096:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:25.921 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.234171+0000", 2024-09-21T15:01:25.921 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.629205395, 2024-09-21T15:01:25.921 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.006019896, 2024-09-21T15:01:25.922 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:25.922 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:25.922 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:25.922 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:25.922 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:25.922 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140109 2024-09-21T15:01:25.922 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.922 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:25.922 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.922 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:25.922 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.234171+0000", 2024-09-21T15:01:25.922 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.922 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.922 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.922 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:25.922 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.234171+0000", 2024-09-21T15:01:25.923 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.923 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.923 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.923 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:25.923 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.234170+0000", 2024-09-21T15:01:25.923 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:25.923 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.923 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.923 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:25.923 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.234178+0000", 2024-09-21T15:01:25.923 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.464e-06 2024-09-21T15:01:25.923 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.923 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.923 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:25.923 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.234191+0000", 2024-09-21T15:01:25.923 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3489000000000001e-05 2024-09-21T15:01:25.924 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.924 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.924 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:25.924 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.234200+0000", 2024-09-21T15:01:25.924 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2779999999999999e-06 2024-09-21T15:01:25.924 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.924 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.924 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:25.924 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.234279+0000", 2024-09-21T15:01:25.924 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8813000000000006e-05 2024-09-21T15:01:25.924 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.924 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.924 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:25.924 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.234298+0000", 2024-09-21T15:01:25.924 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8947999999999999e-05 2024-09-21T15:01:25.924 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.925 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.925 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:25.925 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.239974+0000", 2024-09-21T15:01:25.925 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0056760259999999 2024-09-21T15:01:25.925 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.925 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.925 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:25.925 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.240191+0000", 2024-09-21T15:01:25.925 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000216708 2024-09-21T15:01:25.925 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.925 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:25.925 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.925 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.925 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.925 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140149 3.1 3:86b1e433:::rbd_data.113664780c85.00000000000001ae:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:25.926 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.235545+0000", 2024-09-21T15:01:25.926 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.627831109999999, 2024-09-21T15:01:25.926 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0049193430000001, 2024-09-21T15:01:25.926 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:25.926 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:25.926 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:25.926 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:25.926 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:25.926 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140149 2024-09-21T15:01:25.926 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.926 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:25.926 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.926 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:25.927 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235545+0000", 2024-09-21T15:01:25.927 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.927 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.927 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.927 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:25.927 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235545+0000", 2024-09-21T15:01:25.927 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.927 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.927 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.927 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:25.927 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235544+0000", 2024-09-21T15:01:25.927 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:25.927 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.927 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.928 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:25.928 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235554+0000", 2024-09-21T15:01:25.928 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.8870000000000004e-06 2024-09-21T15:01:25.928 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.928 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.928 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:25.928 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235577+0000", 2024-09-21T15:01:25.928 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2946999999999999e-05 2024-09-21T15:01:25.928 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.928 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.928 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:25.928 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235587+0000", 2024-09-21T15:01:25.928 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0326000000000001e-05 2024-09-21T15:01:25.928 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.928 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.929 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:25.929 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235658+0000", 2024-09-21T15:01:25.929 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1360999999999997e-05 2024-09-21T15:01:25.929 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.929 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.929 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:25.929 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235670+0000", 2024-09-21T15:01:25.929 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1735e-05 2024-09-21T15:01:25.929 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.929 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.929 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:25.929 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.240246+0000", 2024-09-21T15:01:25.929 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0045757790000001 2024-09-21T15:01:25.929 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.929 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.929 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:25.930 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.240465+0000", 2024-09-21T15:01:25.930 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021870799999999999 2024-09-21T15:01:25.930 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.930 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:25.930 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.930 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.930 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.930 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140232 3.1 3:862e0e65:::rbd_data.113664780c85.0000000000000063:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:25.930 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.235760+0000", 2024-09-21T15:01:25.930 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.627616392, 2024-09-21T15:01:25.930 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.004979812, 2024-09-21T15:01:25.930 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:25.930 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:25.931 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:25.931 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:25.931 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:25.931 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140232 2024-09-21T15:01:25.931 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.931 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:25.931 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.931 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:25.931 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235760+0000", 2024-09-21T15:01:25.931 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.931 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.931 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.931 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:25.931 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235760+0000", 2024-09-21T15:01:25.931 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.932 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.932 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.932 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:25.932 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235759+0000", 2024-09-21T15:01:25.932 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:25.932 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.932 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.932 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:25.932 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235777+0000", 2024-09-21T15:01:25.932 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8215000000000001e-05 2024-09-21T15:01:25.932 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.932 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.932 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:25.932 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235793+0000", 2024-09-21T15:01:25.933 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6461e-05 2024-09-21T15:01:25.933 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.933 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.933 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:25.933 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235803+0000", 2024-09-21T15:01:25.933 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5300000000000002e-06 2024-09-21T15:01:25.933 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.933 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.933 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:25.933 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235875+0000", 2024-09-21T15:01:25.933 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2113000000000006e-05 2024-09-21T15:01:25.933 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.933 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.933 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:25.933 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235887+0000", 2024-09-21T15:01:25.934 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2160999999999999e-05 2024-09-21T15:01:25.934 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.934 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.934 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:25.934 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.240521+0000", 2024-09-21T15:01:25.934 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.004633428 2024-09-21T15:01:25.934 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.934 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.934 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:25.934 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.240740+0000", 2024-09-21T15:01:25.934 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000219228 2024-09-21T15:01:25.934 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.934 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:25.934 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.934 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.935 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.935 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140260 3.1 3:8407fb9b:::rbd_data.113664780c85.0000000000000127:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:25.935 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.235839+0000", 2024-09-21T15:01:25.935 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.627537223000001, 2024-09-21T15:01:25.935 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.005157981, 2024-09-21T15:01:25.935 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:25.935 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:25.935 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:25.935 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:25.935 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:25.935 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140260 2024-09-21T15:01:25.935 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.935 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:25.935 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.935 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:25.935 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235839+0000", 2024-09-21T15:01:25.936 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.936 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.936 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.936 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:25.936 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235839+0000", 2024-09-21T15:01:25.936 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.936 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.936 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.936 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:25.936 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235838+0000", 2024-09-21T15:01:25.936 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:25.936 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.936 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.937 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:25.937 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235843+0000", 2024-09-21T15:01:25.937 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.481e-06 2024-09-21T15:01:25.937 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.937 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.937 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:25.937 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235849+0000", 2024-09-21T15:01:25.937 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4880000000000004e-06 2024-09-21T15:01:25.937 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.937 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.937 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:25.937 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235856+0000", 2024-09-21T15:01:25.937 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7649999999999996e-06 2024-09-21T15:01:25.937 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.937 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.937 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:25.938 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235933+0000", 2024-09-21T15:01:25.938 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6987e-05 2024-09-21T15:01:25.938 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.938 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.938 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:25.938 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235949+0000", 2024-09-21T15:01:25.938 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6696000000000001e-05 2024-09-21T15:01:25.938 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.938 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.938 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:25.938 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.240823+0000", 2024-09-21T15:01:25.938 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.004873259 2024-09-21T15:01:25.938 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.938 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.939 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:25.939 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.240997+0000", 2024-09-21T15:01:25.939 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000174365 2024-09-21T15:01:25.939 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.939 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:25.939 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.939 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.939 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.939 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140274 3.1 3:854f6f9b:::rbd_data.113664780c85.0000000000000189:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:25.939 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.235884+0000", 2024-09-21T15:01:25.939 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.627492275000002, 2024-09-21T15:01:25.939 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.005397366, 2024-09-21T15:01:25.939 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:25.939 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:25.939 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:25.940 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:25.940 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:25.940 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140274 2024-09-21T15:01:25.940 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.940 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:25.940 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.940 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:25.940 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235884+0000", 2024-09-21T15:01:25.940 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.940 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.940 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.940 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:25.940 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235884+0000", 2024-09-21T15:01:25.941 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.941 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.941 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.941 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:25.941 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235883+0000", 2024-09-21T15:01:25.941 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:25.941 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.941 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.941 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:25.941 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235891+0000", 2024-09-21T15:01:25.941 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9740000000000002e-06 2024-09-21T15:01:25.941 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.941 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.941 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:25.941 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235900+0000", 2024-09-21T15:01:25.942 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6659999999999997e-06 2024-09-21T15:01:25.942 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.942 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.942 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:25.942 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235909+0000", 2024-09-21T15:01:25.942 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3400000000000004e-06 2024-09-21T15:01:25.942 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.942 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.942 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:25.942 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236012+0000", 2024-09-21T15:01:25.942 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010240500000000001 2024-09-21T15:01:25.942 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.942 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.942 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:25.942 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236027+0000", 2024-09-21T15:01:25.942 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5418e-05 2024-09-21T15:01:25.943 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.943 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.943 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:25.943 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.241068+0000", 2024-09-21T15:01:25.943 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0050410729999999 2024-09-21T15:01:25.943 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.943 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.943 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:25.943 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.241281+0000", 2024-09-21T15:01:25.943 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021321900000000001 2024-09-21T15:01:25.943 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.943 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:25.943 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.943 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.943 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.944 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140312 3.1 3:86b14cb7:::rbd_data.113664780c85.00000000000000d5:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:25.944 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.236769+0000", 2024-09-21T15:01:25.944 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.626607811, 2024-09-21T15:01:25.944 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.004717198, 2024-09-21T15:01:25.944 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:25.944 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:25.944 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:25.944 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:25.944 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:25.944 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140312 2024-09-21T15:01:25.944 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.944 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:25.944 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.944 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:25.944 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236769+0000", 2024-09-21T15:01:25.944 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.945 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.945 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.945 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:25.945 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236769+0000", 2024-09-21T15:01:25.945 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.945 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.945 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.945 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:25.945 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236768+0000", 2024-09-21T15:01:25.945 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:25.945 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.945 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.945 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:25.945 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236771+0000", 2024-09-21T15:01:25.945 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.331e-06 2024-09-21T15:01:25.945 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.946 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.946 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:25.946 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236777+0000", 2024-09-21T15:01:25.946 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6949999999999997e-06 2024-09-21T15:01:25.946 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.946 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.946 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:25.946 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236787+0000", 2024-09-21T15:01:25.946 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6700000000000006e-06 2024-09-21T15:01:25.946 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.946 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.946 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:25.946 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236846+0000", 2024-09-21T15:01:25.947 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.944e-05 2024-09-21T15:01:25.947 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.947 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.947 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:25.947 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236861+0000", 2024-09-21T15:01:25.947 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4611e-05 2024-09-21T15:01:25.947 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.947 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.947 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:25.947 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.241349+0000", 2024-09-21T15:01:25.947 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0044879250000001 2024-09-21T15:01:25.947 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.947 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.948 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:25.948 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.241486+0000", 2024-09-21T15:01:25.948 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013713000000000001 2024-09-21T15:01:25.948 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.948 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:25.948 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.948 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.948 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.948 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140325 3.1 3:87277f92:::rbd_data.113664780c85.0000000000000136:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:25.948 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.239351+0000", 2024-09-21T15:01:25.948 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.624025671999998, 2024-09-21T15:01:25.948 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0024334690000001, 2024-09-21T15:01:25.948 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:25.948 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:25.948 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:25.949 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:25.949 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:25.949 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140325 2024-09-21T15:01:25.949 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.949 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:25.949 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.949 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:25.949 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239351+0000", 2024-09-21T15:01:25.949 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.949 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.949 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.949 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:25.949 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239351+0000", 2024-09-21T15:01:25.949 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.949 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.949 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.949 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:25.950 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239349+0000", 2024-09-21T15:01:25.950 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:25.950 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.950 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.950 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:25.950 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239358+0000", 2024-09-21T15:01:25.950 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2830000000000001e-06 2024-09-21T15:01:25.950 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.950 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.950 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:25.950 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239379+0000", 2024-09-21T15:01:25.951 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1780000000000002e-05 2024-09-21T15:01:25.951 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.951 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.951 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:25.951 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239389+0000", 2024-09-21T15:01:25.951 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0056e-05 2024-09-21T15:01:25.951 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.951 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.951 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:25.951 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239484+0000", 2024-09-21T15:01:25.951 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4611000000000006e-05 2024-09-21T15:01:25.951 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.951 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.951 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:25.952 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239497+0000", 2024-09-21T15:01:25.952 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2909e-05 2024-09-21T15:01:25.952 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.952 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.952 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:25.952 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.241559+0000", 2024-09-21T15:01:25.952 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0020622459999999 2024-09-21T15:01:25.952 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.952 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.952 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:25.952 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.241784+0000", 2024-09-21T15:01:25.952 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000225009 2024-09-21T15:01:25.952 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.952 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:25.953 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.953 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.953 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.953 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140335 3.1 3:80f1e8f1:::rbd_data.113664780c85.000000000000017c:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:25.953 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.239409+0000", 2024-09-21T15:01:25.953 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.623966953, 2024-09-21T15:01:25.953 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0025803150000001, 2024-09-21T15:01:25.953 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:25.953 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:25.953 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:25.953 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:25.953 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:25.953 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140335 2024-09-21T15:01:25.953 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.954 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:25.954 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.954 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:25.954 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239409+0000", 2024-09-21T15:01:25.954 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.954 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.954 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.954 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:25.954 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239409+0000", 2024-09-21T15:01:25.954 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.954 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.954 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.954 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:25.954 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239409+0000", 2024-09-21T15:01:25.954 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:25.955 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.955 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.955 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:25.955 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239413+0000", 2024-09-21T15:01:25.955 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7620000000000003e-06 2024-09-21T15:01:25.955 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.955 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.955 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:25.955 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239436+0000", 2024-09-21T15:01:25.955 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2889000000000001e-05 2024-09-21T15:01:25.955 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.955 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.955 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:25.955 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239444+0000", 2024-09-21T15:01:25.956 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6809999999999998e-06 2024-09-21T15:01:25.956 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.956 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.956 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:25.956 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239542+0000", 2024-09-21T15:01:25.956 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7868999999999994e-05 2024-09-21T15:01:25.956 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.956 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.956 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:25.956 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239558+0000", 2024-09-21T15:01:25.956 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6257e-05 2024-09-21T15:01:25.956 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.956 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.956 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:25.957 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.241847+0000", 2024-09-21T15:01:25.957 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.002288533 2024-09-21T15:01:25.957 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.957 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.957 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:25.957 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.241990+0000", 2024-09-21T15:01:25.957 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014308300000000001 2024-09-21T15:01:25.957 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.957 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:25.957 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.957 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.957 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.958 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:166488 3.4 3.7bcf3e84 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e189)", 2024-09-21T15:01:25.958 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:21.342023+0000", 2024-09-21T15:01:25.958 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 4.5213531209999998, 2024-09-21T15:01:25.958 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0015098469999999, 2024-09-21T15:01:25.958 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:25.958 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:25.958 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:25.958 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:25.958 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:25.958 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 166488 2024-09-21T15:01:25.958 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.958 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:25.958 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.958 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:25.959 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:21.342023+0000", 2024-09-21T15:01:25.959 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.959 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.959 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.959 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:25.959 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:21.342023+0000", 2024-09-21T15:01:25.959 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:25.959 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.959 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.959 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:25.959 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:21.342022+0000", 2024-09-21T15:01:25.959 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:25.959 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.959 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.959 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:25.960 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:21.342029+0000", 2024-09-21T15:01:25.960 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1420000000000004e-06 2024-09-21T15:01:25.960 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.960 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.960 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:25.960 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:21.342051+0000", 2024-09-21T15:01:25.960 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2549999999999999e-05 2024-09-21T15:01:25.960 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.960 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.960 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:25.960 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:21.342059+0000", 2024-09-21T15:01:25.960 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8490000000000006e-06 2024-09-21T15:01:25.960 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.960 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.961 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:25.961 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:21.342105+0000", 2024-09-21T15:01:25.961 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5488999999999998e-05 2024-09-21T15:01:25.961 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.961 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.961 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:25.961 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:21.342115+0000", 2024-09-21T15:01:25.962 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0633e-05 2024-09-21T15:01:25.962 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.962 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.962 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:25.962 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:22.343521+0000", 2024-09-21T15:01:25.962 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0014057270000001 2024-09-21T15:01:25.962 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:25.963 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:25.963 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:25.963 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:22.343533+0000", 2024-09-21T15:01:25.963 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2097e-05 2024-09-21T15:01:25.963 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.963 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:25.963 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.963 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:25.964 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:25.964 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:01:25.966 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T15:01:25.966 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:01:25.960+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:01:26.066 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:01:26.062+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:01:26.121 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:01:26.121 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T15:01:26.121 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.121 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:166795 3.8 3.6e679928 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e193)", 2024-09-21T15:01:26.121 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:25.321589+0000", 2024-09-21T15:01:26.121 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 0.79795098499999995, 2024-09-21T15:01:26.121 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.79799012800000002, 2024-09-21T15:01:26.121 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:26.121 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "waiting for peered", 2024-09-21T15:01:26.122 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:26.122 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:26.122 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:26.122 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 166795 2024-09-21T15:01:26.122 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.122 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:26.122 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.122 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:26.122 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:25.321589+0000", 2024-09-21T15:01:26.122 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:26.122 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.122 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.122 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:26.122 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:25.321589+0000", 2024-09-21T15:01:26.123 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:26.123 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.123 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.123 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:26.123 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:25.321588+0000", 2024-09-21T15:01:26.123 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:26.123 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.123 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.123 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:26.123 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:25.321598+0000", 2024-09-21T15:01:26.123 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7429999999999997e-06 2024-09-21T15:01:26.123 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.123 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.123 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:26.123 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:25.321624+0000", 2024-09-21T15:01:26.123 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6752000000000001e-05 2024-09-21T15:01:26.124 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.124 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.124 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:26.124 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:25.321637+0000", 2024-09-21T15:01:26.124 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2809999999999999e-05 2024-09-21T15:01:26.124 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.124 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.124 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:26.124 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:25.321721+0000", 2024-09-21T15:01:26.124 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3314999999999998e-05 2024-09-21T15:01:26.124 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.124 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.124 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:26.124 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:25.321736+0000", 2024-09-21T15:01:26.125 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5262000000000001e-05 2024-09-21T15:01:26.125 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:26.125 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:26.125 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:26.125 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:26.125 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-21T15:01:26.125 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 1 2024-09-21T15:01:26.125 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:01:26.132 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T15:01:26.167 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:01:26.162+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:01:26.177 INFO:teuthology.orchestra.run.smithi032.stderr:nodeep-scrub is unset 2024-09-21T15:01:26.242 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:01:26.243 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:01:26.243 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T15:01:26.243 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T15:01:26.243 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:01:26.252 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T15:01:26.268 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:01:26.266+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:01:26.354 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 5, 6, 7] out_osds: [4] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T15:01:26.354 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T15:01:26.354 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-21T15:01:26.355 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-21T15:01:26.356 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:01:26.356 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T15:01:26.356 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T15:01:26.356 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T15:01:26.356 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.356 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31476 3.c 3:35718b6b:::rbd_data.11367721c19.0000000000000014:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T15:01:26.356 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.505975+0000", 2024-09-21T15:01:26.357 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 147.84748901899999, 2024-09-21T15:01:26.357 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99515995899999998, 2024-09-21T15:01:26.357 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:26.357 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:26.357 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:26.357 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:26.357 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:26.357 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31476 2024-09-21T15:01:26.357 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.357 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:26.357 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.357 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:26.357 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.505975+0000", 2024-09-21T15:01:26.357 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:26.357 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.357 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.358 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:26.358 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.505975+0000", 2024-09-21T15:01:26.358 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:26.358 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.358 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.358 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:26.358 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.505971+0000", 2024-09-21T15:01:26.358 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999967 2024-09-21T15:01:26.358 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.358 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.358 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:26.358 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.505989+0000", 2024-09-21T15:01:26.358 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7985000000000001e-05 2024-09-21T15:01:26.358 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.359 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.359 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:26.359 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506050+0000", 2024-09-21T15:01:26.359 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0659000000000003e-05 2024-09-21T15:01:26.359 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.359 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.359 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:26.359 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506074+0000", 2024-09-21T15:01:26.359 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4156e-05 2024-09-21T15:01:26.359 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.359 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.359 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:26.359 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506225+0000", 2024-09-21T15:01:26.359 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015097 2024-09-21T15:01:26.360 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.360 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.360 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:26.360 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506256+0000", 2024-09-21T15:01:26.360 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0505999999999999e-05 2024-09-21T15:01:26.360 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.360 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.360 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:26.360 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.497974+0000", 2024-09-21T15:01:26.360 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99171825700000005 2024-09-21T15:01:26.360 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.360 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.360 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for missing object", 2024-09-21T15:01:26.360 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.498449+0000", 2024-09-21T15:01:26.360 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00047506399999999998 2024-09-21T15:01:26.360 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.361 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.361 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:26.361 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.501000+0000", 2024-09-21T15:01:26.361 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0025506919999999998 2024-09-21T15:01:26.361 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.361 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.361 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:26.361 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.501135+0000", 2024-09-21T15:01:26.361 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013524000000000001 2024-09-21T15:01:26.361 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:26.361 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:26.361 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:26.361 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.361 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.361 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31480 3.c 3:36ccd808:::rbd_data.11367721c19.0000000000000018:head [sparse-read 0~14821] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T15:01:26.361 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.506116+0000", 2024-09-21T15:01:26.362 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 147.847347676, 2024-09-21T15:01:26.362 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99316033599999998, 2024-09-21T15:01:26.362 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:26.362 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:26.362 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:26.362 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:26.362 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:26.362 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31480 2024-09-21T15:01:26.362 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.362 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:26.362 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.362 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:26.362 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506116+0000", 2024-09-21T15:01:26.362 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:26.362 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.363 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.363 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:26.363 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506116+0000", 2024-09-21T15:01:26.363 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:26.363 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.363 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.363 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:26.363 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506115+0000", 2024-09-21T15:01:26.363 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:26.363 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.363 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.363 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:26.363 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506124+0000", 2024-09-21T15:01:26.363 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0659999999999993e-06 2024-09-21T15:01:26.363 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.363 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.364 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:26.364 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506145+0000", 2024-09-21T15:01:26.364 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1447e-05 2024-09-21T15:01:26.364 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.364 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.364 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:26.364 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506161+0000", 2024-09-21T15:01:26.364 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5209e-05 2024-09-21T15:01:26.364 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.364 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.364 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:26.364 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506337+0000", 2024-09-21T15:01:26.364 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017642999999999999 2024-09-21T15:01:26.364 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.364 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.365 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:26.365 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506375+0000", 2024-09-21T15:01:26.365 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7536000000000001e-05 2024-09-21T15:01:26.365 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.365 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.365 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:26.365 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.498815+0000", 2024-09-21T15:01:26.365 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99244067999999996 2024-09-21T15:01:26.365 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.365 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.365 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:26.365 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.498950+0000", 2024-09-21T15:01:26.365 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013436700000000001 2024-09-21T15:01:26.365 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.365 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.366 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:26.366 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499277+0000", 2024-09-21T15:01:26.366 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00032696800000000002 2024-09-21T15:01:26.366 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:26.366 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:26.366 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:26.366 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.366 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.366 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31513 3.c 3:36de8523:::rbd_data.11367721c19.0000000000000039:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T15:01:26.366 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.509071+0000", 2024-09-21T15:01:26.366 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 147.84439347599999, 2024-09-21T15:01:26.366 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99038225099999999, 2024-09-21T15:01:26.366 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:26.366 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:26.366 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:26.367 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:26.367 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:26.367 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31513 2024-09-21T15:01:26.367 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.367 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:26.367 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.367 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:26.367 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509071+0000", 2024-09-21T15:01:26.367 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:26.367 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.367 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.367 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:26.367 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509071+0000", 2024-09-21T15:01:26.367 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:26.367 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.367 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.368 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:26.368 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509068+0000", 2024-09-21T15:01:26.368 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-21T15:01:26.368 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.368 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.368 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:26.368 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509082+0000", 2024-09-21T15:01:26.368 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3545e-05 2024-09-21T15:01:26.368 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.368 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.368 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:26.368 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509116+0000", 2024-09-21T15:01:26.368 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3942000000000001e-05 2024-09-21T15:01:26.368 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.368 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.368 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:26.369 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509134+0000", 2024-09-21T15:01:26.369 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8839999999999999e-05 2024-09-21T15:01:26.369 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.369 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.369 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:26.369 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509318+0000", 2024-09-21T15:01:26.369 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018326799999999999 2024-09-21T15:01:26.369 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.369 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.369 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:26.369 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509346+0000", 2024-09-21T15:01:26.369 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8019000000000001e-05 2024-09-21T15:01:26.369 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.369 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.369 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:26.369 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499330+0000", 2024-09-21T15:01:26.370 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.98998399299999995 2024-09-21T15:01:26.370 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.370 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.370 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:26.370 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499453+0000", 2024-09-21T15:01:26.370 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000123115 2024-09-21T15:01:26.370 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:26.370 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:26.370 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:26.370 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.370 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.370 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31515 3.c 3:35b36ddd:::rbd_data.11367721c19.000000000000003b:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T15:01:26.370 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.509177+0000", 2024-09-21T15:01:26.371 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 147.84428704999999, 2024-09-21T15:01:26.371 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99044870600000001, 2024-09-21T15:01:26.371 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:26.371 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:26.371 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:26.371 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:26.371 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:26.371 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31515 2024-09-21T15:01:26.371 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.371 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:26.371 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.371 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:26.371 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509177+0000", 2024-09-21T15:01:26.371 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:26.371 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.372 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.372 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:26.372 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509177+0000", 2024-09-21T15:01:26.372 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:26.372 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.372 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.372 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:26.372 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509176+0000", 2024-09-21T15:01:26.372 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:26.372 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.372 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.372 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:26.372 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509183+0000", 2024-09-21T15:01:26.372 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6720000000000004e-06 2024-09-21T15:01:26.372 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.372 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.373 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:26.373 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509269+0000", 2024-09-21T15:01:26.373 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5881e-05 2024-09-21T15:01:26.373 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.373 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.373 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:26.373 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509303+0000", 2024-09-21T15:01:26.373 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3367000000000001e-05 2024-09-21T15:01:26.373 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.373 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.373 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:26.373 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509425+0000", 2024-09-21T15:01:26.373 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000122199 2024-09-21T15:01:26.374 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.374 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.374 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:26.374 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509448+0000", 2024-09-21T15:01:26.374 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2688999999999999e-05 2024-09-21T15:01:26.374 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.374 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.374 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:26.374 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499501+0000", 2024-09-21T15:01:26.374 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99005306999999998 2024-09-21T15:01:26.374 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.374 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.374 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:26.375 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499626+0000", 2024-09-21T15:01:26.375 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012501400000000001 2024-09-21T15:01:26.375 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:26.375 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:26.375 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:26.375 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.375 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.375 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31531 3.c 3:314f0287:::rbd_data.11367721c19.000000000000004b:head [sparse-read 0~7106] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T15:01:26.375 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.509329+0000", 2024-09-21T15:01:26.375 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 147.84413502999999, 2024-09-21T15:01:26.375 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99103085300000004, 2024-09-21T15:01:26.375 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:26.375 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:26.376 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:26.376 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:26.376 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:26.376 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31531 2024-09-21T15:01:26.376 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.376 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:26.376 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.376 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:26.376 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509329+0000", 2024-09-21T15:01:26.376 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:26.376 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.376 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.376 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:26.376 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509329+0000", 2024-09-21T15:01:26.377 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:26.377 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.377 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.377 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:26.377 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509328+0000", 2024-09-21T15:01:26.377 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:26.377 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.377 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.377 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:26.377 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509336+0000", 2024-09-21T15:01:26.377 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3170000000000002e-06 2024-09-21T15:01:26.377 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.377 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.377 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:26.378 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509353+0000", 2024-09-21T15:01:26.378 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7214000000000002e-05 2024-09-21T15:01:26.378 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.378 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.378 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:26.378 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509366+0000", 2024-09-21T15:01:26.378 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.26e-05 2024-09-21T15:01:26.378 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.378 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.378 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:26.378 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509523+0000", 2024-09-21T15:01:26.378 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015708000000000001 2024-09-21T15:01:26.378 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.378 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.379 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:26.379 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509544+0000", 2024-09-21T15:01:26.379 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0783000000000001e-05 2024-09-21T15:01:26.379 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.379 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.379 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:26.379 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499727+0000", 2024-09-21T15:01:26.379 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99018345200000002 2024-09-21T15:01:26.379 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.379 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.379 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:26.379 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499949+0000", 2024-09-21T15:01:26.379 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000221358 2024-09-21T15:01:26.379 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.379 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.380 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:26.380 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.500360+0000", 2024-09-21T15:01:26.380 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00041130700000000002 2024-09-21T15:01:26.380 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:26.380 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:26.380 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:26.380 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.380 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.380 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:82545 3.b 3:d4eb1ee5:::rbd_data.11368296f791.0000000000000000:head [write 0~671744 [fadvise_sequential+fadvise_nocache],write 704512~65536 [fadvise_sequential+fadvise_nocache],write 786432~49152 [fadvise_sequential+fadvise_nocache],write 868352~49152 [fadvise_sequential+fadvise_nocache],write 933888~65536 [fadvise_sequential+fadvise_nocache],write 1032192~32768 [fadvise_sequential+fadvise_nocache],write 1097728~65536 [fadvise_sequential+fadvise_nocache],write 1179648~49152 [fadvise_sequential+fadvise_nocache],write 1261568~49152 [fadvise_sequential+fadvise_nocache],write 1327104~65536 [fadvise_sequential+fadvise_nocache],write 1425408~32768 [fadvise_sequential+fadvise_nocache],write 1490944~65536 [fadvise_sequential+fadvise_nocache],write 1572864~8192 [fadvise_sequential+fadvise_nocache],write 1630208~49152 [fadvise_sequential+fadvise_nocache],write 1703936~40960 [fadvise_sequential+fadvise_nocache],write 1794048~40960 [fadvise_sequential+fadvise_nocache],write 1859584~49152 [fadvise_sequential+fadvise_nocache],write 1957888~8192 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:01:26.380 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:47.416089+0000", 2024-09-21T15:01:26.380 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 98.937375305000003, 2024-09-21T15:01:26.380 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011896455, 2024-09-21T15:01:26.380 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:26.380 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:26.381 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:26.381 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:26.381 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:26.381 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82545 2024-09-21T15:01:26.381 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.381 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:26.381 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.381 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:26.381 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.416089+0000", 2024-09-21T15:01:26.381 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:26.381 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.381 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.381 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:26.381 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.416089+0000", 2024-09-21T15:01:26.381 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:26.381 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.382 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.382 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:26.382 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.416087+0000", 2024-09-21T15:01:26.382 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:01:26.382 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.382 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.382 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:26.382 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.419643+0000", 2024-09-21T15:01:26.382 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0035555550000000002 2024-09-21T15:01:26.382 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.382 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.382 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:26.382 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.419663+0000", 2024-09-21T15:01:26.382 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0202000000000001e-05 2024-09-21T15:01:26.382 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.383 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.383 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:26.383 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.419676+0000", 2024-09-21T15:01:26.383 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3207000000000001e-05 2024-09-21T15:01:26.383 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.383 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.383 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:26.383 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.419759+0000", 2024-09-21T15:01:26.383 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3521000000000002e-05 2024-09-21T15:01:26.383 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.383 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.383 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op must wait for map", 2024-09-21T15:01:26.383 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.419768+0000", 2024-09-21T15:01:26.383 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4479999999999994e-06 2024-09-21T15:01:26.383 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.384 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.384 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:26.384 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.422171+0000", 2024-09-21T15:01:26.384 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0024035469999999998 2024-09-21T15:01:26.384 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.384 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.384 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:26.384 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.422350+0000", 2024-09-21T15:01:26.384 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000179001 2024-09-21T15:01:26.384 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.384 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.384 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T15:01:26.384 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.422710+0000", 2024-09-21T15:01:26.384 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000359878 2024-09-21T15:01:26.384 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.385 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.385 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:01:26.385 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.426676+0000", 2024-09-21T15:01:26.385 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0039655790000000003 2024-09-21T15:01:26.385 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.385 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.385 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:01:26.385 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.427884+0000", 2024-09-21T15:01:26.385 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001208207 2024-09-21T15:01:26.385 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.385 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.385 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:01:26.385 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.427953+0000", 2024-09-21T15:01:26.385 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8533000000000003e-05 2024-09-21T15:01:26.386 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.386 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.386 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:26.386 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.427985+0000", 2024-09-21T15:01:26.386 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2481999999999998e-05 2024-09-21T15:01:26.386 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:26.386 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:26.386 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:26.386 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.386 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.386 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:97675 3.b 3:d321acb8:::rbd_data.1136bfc79a70.0000000000000003:head [write 0~4718592 [fadvise_sequential+fadvise_nocache],write 5242880~147456 [fadvise_sequential+fadvise_nocache],write 5455872~147456 [fadvise_sequential+fadvise_nocache],write 5668864~229376 [fadvise_sequential+fadvise_nocache],write 5963776~147456 [fadvise_sequential+fadvise_nocache],write 6176768~229376 [fadvise_sequential+fadvise_nocache],write 6471680~147456 [fadvise_sequential+fadvise_nocache],write 6684672~229376 [fadvise_sequential+fadvise_nocache],write 6979584~147456 [fadvise_sequential+fadvise_nocache],write 7192576~229376 [fadvise_sequential+fadvise_nocache],write 7487488~147456 [fadvise_sequential+fadvise_nocache],write 7700480~147456 [fadvise_sequential+fadvise_nocache],write 7864320~65536 [fadvise_sequential+fadvise_nocache],write 7995392~147456 [fadvise_sequential+fadvise_nocache],write 8208384~147456 [fadvise_sequential+fadvise_nocache],write 8388608~49152 [fadvise_sequential+fadvise_nocache],write 8536064~114688 [fadvise_sequential+fadvise_nocache],write 8749056~114688 [fadvise_sequential+fadvise_nocache],write 8912896~32768 [fadvise_sequential+fadvise_nocache],write 9125888~32768 [fadvise_sequential+fadvise_nocache],write 9338880~32768 [fadvise_sequential+fadvise_nocache],write 9437184~16384 [fadvise_sequential+fadvise_nocache],write 9633792~32768 [fadvise_sequential+fadvise_nocache],write 9846784~32768 [fadvise_sequential+fadvise_nocache],write 9961472~114688 [fadvise_sequential+fadvise_nocache],write 10092544~196608 [fadvise_sequential+fadvise_nocache],write 10305536~278528 [fadvise_sequential+fadvise_nocache],write 10600448~196608 [fadvise_sequential+fadvise_nocache],write 10813440~278528 [fadvise_sequential+fadvise_nocache],write 11108352~196608 [fadvise_sequential+fadvise_nocache],write 11321344~196608 [fadvise_sequential+fadvise_nocache],write 11534336~65536 [fadvise_sequential+fadvise_nocache],write 11632640~98304 [fadvise_sequential+fadvise_nocache],write 11747328~65536 [fadvise_sequential+fadvise_nocache],write 11845632~98304 [fadvise_sequential+fadvise_nocache],write 11960320~65536 [fadvise_sequential+fadvise_nocache],write 12058624~49152 [fadvise_sequential+fadvise_nocache],write 12140544~98304 [fadvise_sequential+fadvise_nocache],write 12255232~65536 [fadvise_sequential+fadvise_nocache],write 12353536~98304 [fadvise_sequential+fadvise_nocache],write 12468224~65536 [fadvise_sequential+fadvise_nocache],write 12566528~16384 [fadvise_sequential+fadvise_nocache],write 12648448~98304 [fadvise_sequential+fadvise_nocache],write 12861440~98304 [fadvise_sequential+fadvise_nocache],write 13074432~32768 [fadvise_sequential+fadvise_nocache],write 13156352~98304 [fadvise_sequential+fadvise_nocache],write 13369344~98304 [fadvise_sequential+fadvise_nocache],write 13582336~49152 [fadvise_sequential+fadvise_nocache],write 13664256~98304 [fadvise_sequential+fadvise_nocache],write 13877248~98304 [fadvise_sequential+fadvise_nocache],write 14090240~65536 [fadvise_sequential+fadvise_nocache],write 15286272~81920 [fadvise_sequential+fadvise_nocache],write 15499264~81920 [fadvise_sequential+fadvise_nocache],write 15712256~16384 [fadvise_sequential+fadvise_nocache],write 15794176~81920 [fadvise_sequential+fadvise_nocache],write 16007168~81920 [fadvise_sequential+fadvise_nocache],write 16220160~32768 [fadvise_sequential+fadvise_nocache],write 16302080~81920 [fadvise_sequential+fadvise_nocache],write 16515072~81920 [fadvise_sequential+fadvise_nocache],write 16728064~49152 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:01:26.386 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.302412+0000", 2024-09-21T15:01:26.386 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 81.051052515999999, 2024-09-21T15:01:26.386 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.067380532000000007, 2024-09-21T15:01:26.387 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:26.387 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:26.387 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:26.387 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:26.387 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:26.387 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97675 2024-09-21T15:01:26.387 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.387 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:26.387 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.387 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:26.387 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.302412+0000", 2024-09-21T15:01:26.387 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:26.387 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.387 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.387 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:26.388 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.302412+0000", 2024-09-21T15:01:26.388 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:26.388 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.388 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.388 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:26.388 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.302409+0000", 2024-09-21T15:01:26.388 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-21T15:01:26.388 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.388 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.388 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:26.388 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328550+0000", 2024-09-21T15:01:26.388 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.026140674999999999 2024-09-21T15:01:26.388 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.388 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.388 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:26.389 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328584+0000", 2024-09-21T15:01:26.389 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4165999999999998e-05 2024-09-21T15:01:26.389 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.389 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.389 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:26.389 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328599+0000", 2024-09-21T15:01:26.389 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5505000000000001e-05 2024-09-21T15:01:26.389 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.389 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.389 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:26.389 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328704+0000", 2024-09-21T15:01:26.389 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010487199999999999 2024-09-21T15:01:26.389 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.389 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.389 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:26.390 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.329082+0000", 2024-09-21T15:01:26.390 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00037804899999999998 2024-09-21T15:01:26.390 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.390 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.390 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T15:01:26.390 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.331158+0000", 2024-09-21T15:01:26.390 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0020755449999999998 2024-09-21T15:01:26.390 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.390 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.390 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:01:26.390 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.360553+0000", 2024-09-21T15:01:26.390 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.029395553000000001 2024-09-21T15:01:26.390 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.390 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.390 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:01:26.391 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.369496+0000", 2024-09-21T15:01:26.391 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0089428619999999993 2024-09-21T15:01:26.391 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.391 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.391 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:01:26.391 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.369686+0000", 2024-09-21T15:01:26.391 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019000399999999999 2024-09-21T15:01:26.391 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.391 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.391 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:26.391 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.369792+0000", 2024-09-21T15:01:26.391 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000105775 2024-09-21T15:01:26.391 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:26.391 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:26.392 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:26.392 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.392 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.392 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:97693 3.b 3:d7d7b39b:::rbd_data.1136b9c49b1c.000000000000000c:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:01:26.392 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.328643+0000", 2024-09-21T15:01:26.392 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 81.024820730000002, 2024-09-21T15:01:26.392 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.031062136000000001, 2024-09-21T15:01:26.392 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:26.392 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:26.392 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:26.392 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:26.392 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:26.392 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97693 2024-09-21T15:01:26.392 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.392 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:26.392 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.392 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:26.393 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328643+0000", 2024-09-21T15:01:26.393 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:26.393 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.393 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.393 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:26.393 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328643+0000", 2024-09-21T15:01:26.393 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:26.393 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.393 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.393 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:26.393 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328642+0000", 2024-09-21T15:01:26.393 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:01:26.393 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.393 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.393 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:26.394 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328650+0000", 2024-09-21T15:01:26.394 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7770000000000008e-06 2024-09-21T15:01:26.394 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.394 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.394 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:26.394 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328662+0000", 2024-09-21T15:01:26.394 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2026e-05 2024-09-21T15:01:26.394 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.394 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.394 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:26.394 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328671+0000", 2024-09-21T15:01:26.394 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.585e-06 2024-09-21T15:01:26.394 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.394 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.395 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:26.395 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.359498+0000", 2024-09-21T15:01:26.395 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.030827001 2024-09-21T15:01:26.395 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.395 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.395 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:26.395 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.359603+0000", 2024-09-21T15:01:26.395 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010512400000000001 2024-09-21T15:01:26.395 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.395 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.396 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:26.396 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.359705+0000", 2024-09-21T15:01:26.396 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000102327 2024-09-21T15:01:26.396 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:26.396 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:26.396 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:26.396 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.396 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.396 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:97724 3.b 3:d4bf6bcf:::rbd_data.1136b9c49b1c.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:01:26.396 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.328704+0000", 2024-09-21T15:01:26.396 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 81.024760458000003, 2024-09-21T15:01:26.396 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.031157957, 2024-09-21T15:01:26.396 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:26.397 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:26.397 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:26.397 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:26.397 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:26.397 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97724 2024-09-21T15:01:26.397 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.397 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:26.397 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.397 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:26.397 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328704+0000", 2024-09-21T15:01:26.397 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:26.397 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.397 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.398 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:26.398 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328704+0000", 2024-09-21T15:01:26.398 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:26.398 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.398 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.398 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:26.398 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328703+0000", 2024-09-21T15:01:26.398 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:26.398 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.398 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.398 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:26.398 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328708+0000", 2024-09-21T15:01:26.398 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0050000000000004e-06 2024-09-21T15:01:26.398 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.398 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.399 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:26.399 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328719+0000", 2024-09-21T15:01:26.399 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1052e-05 2024-09-21T15:01:26.399 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.399 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.399 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:26.399 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328727+0000", 2024-09-21T15:01:26.399 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4519999999999997e-06 2024-09-21T15:01:26.399 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.399 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.399 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:26.399 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.359737+0000", 2024-09-21T15:01:26.399 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.031009341999999999 2024-09-21T15:01:26.399 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.399 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.400 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:26.400 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.359806+0000", 2024-09-21T15:01:26.400 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9750000000000001e-05 2024-09-21T15:01:26.400 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.400 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.400 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:26.400 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.359862+0000", 2024-09-21T15:01:26.400 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5084999999999999e-05 2024-09-21T15:01:26.400 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:26.400 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:26.400 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:26.400 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.400 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.400 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:129970 3.c e158/87 3:33fa4d4e:::rbd_data.1136a12e4351.0000000000000000:head v 158'379, mlcod=156'377)", 2024-09-21T15:01:26.400 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:44.452766+0000", 2024-09-21T15:01:26.401 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 41.900697905999998, 2024-09-21T15:01:26.401 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011942062, 2024-09-21T15:01:26.401 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:26.401 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:26.401 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:26.401 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.401 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:26.401 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:44.452766+0000", 2024-09-21T15:01:26.401 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:26.401 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.401 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.401 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:26.401 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:44.452766+0000", 2024-09-21T15:01:26.401 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:26.401 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.401 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.402 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:26.402 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:44.452766+0000", 2024-09-21T15:01:26.402 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:26.402 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.402 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.402 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:26.402 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:44.458962+0000", 2024-09-21T15:01:26.402 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0061964799999999999 2024-09-21T15:01:26.402 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.402 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.402 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:26.402 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:44.459012+0000", 2024-09-21T15:01:26.402 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9427e-05 2024-09-21T15:01:26.402 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.403 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.403 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:26.403 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:44.459042+0000", 2024-09-21T15:01:26.403 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9838000000000001e-05 2024-09-21T15:01:26.403 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.403 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.403 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:26.403 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:44.459115+0000", 2024-09-21T15:01:26.403 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3430999999999995e-05 2024-09-21T15:01:26.403 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.403 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.403 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:26.403 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:44.459135+0000", 2024-09-21T15:01:26.403 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9701999999999999e-05 2024-09-21T15:01:26.404 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.404 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.404 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:01:26.404 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:44.464645+0000", 2024-09-21T15:01:26.404 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0055100820000000003 2024-09-21T15:01:26.404 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.404 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.404 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:26.404 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:44.464708+0000", 2024-09-21T15:01:26.404 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3399999999999996e-05 2024-09-21T15:01:26.404 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:26.404 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:26.404 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:26.404 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.405 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.405 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:135368 3.b 3:d3301c39:::rbd_data.11365ed419e3.0000000000000102:67 [read 335872~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:26.405 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:53.998901+0000", 2024-09-21T15:01:26.405 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 32.354563094, 2024-09-21T15:01:26.405 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011666174, 2024-09-21T15:01:26.405 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:26.405 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:26.405 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:26.405 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:26.405 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:26.405 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 135368 2024-09-21T15:01:26.405 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.405 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:26.405 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.405 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:26.405 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998901+0000", 2024-09-21T15:01:26.406 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:26.406 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.406 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.406 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:26.406 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998901+0000", 2024-09-21T15:01:26.406 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:26.406 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.406 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.406 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:26.406 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998901+0000", 2024-09-21T15:01:26.406 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:26.406 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.406 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.406 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:26.407 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998903+0000", 2024-09-21T15:01:26.407 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5500000000000001e-06 2024-09-21T15:01:26.407 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.407 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.407 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:26.407 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998913+0000", 2024-09-21T15:01:26.407 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6479999999999998e-06 2024-09-21T15:01:26.407 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.407 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.407 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:26.407 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998919+0000", 2024-09-21T15:01:26.407 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7930000000000003e-06 2024-09-21T15:01:26.407 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.407 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.407 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:26.408 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:54.010277+0000", 2024-09-21T15:01:26.408 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011358692 2024-09-21T15:01:26.408 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.408 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.408 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:26.408 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:54.010348+0000", 2024-09-21T15:01:26.408 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0247999999999996e-05 2024-09-21T15:01:26.408 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.408 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.408 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:26.408 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:54.010567+0000", 2024-09-21T15:01:26.408 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021953599999999999 2024-09-21T15:01:26.408 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:26.408 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:26.408 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:26.409 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.409 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.409 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:135369 3.b 3:d3301c39:::rbd_data.11365ed419e3.0000000000000102:67 [read 352256~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:26.409 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:53.998923+0000", 2024-09-21T15:01:26.409 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 32.354540792999998, 2024-09-21T15:01:26.409 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011990331999999999, 2024-09-21T15:01:26.409 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:26.409 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:26.409 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:26.409 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:26.409 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:26.409 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 135369 2024-09-21T15:01:26.409 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.409 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:26.409 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.409 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:26.410 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998923+0000", 2024-09-21T15:01:26.410 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:26.410 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.410 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.410 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:26.410 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998923+0000", 2024-09-21T15:01:26.410 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:26.410 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.410 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.410 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:26.410 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998923+0000", 2024-09-21T15:01:26.410 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:26.410 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.411 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.411 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:26.411 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998926+0000", 2024-09-21T15:01:26.411 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6749999999999998e-06 2024-09-21T15:01:26.411 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.411 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.411 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:26.411 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998935+0000", 2024-09-21T15:01:26.411 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3619999999999995e-06 2024-09-21T15:01:26.411 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.411 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.411 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:26.411 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998940+0000", 2024-09-21T15:01:26.411 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2379999999999997e-06 2024-09-21T15:01:26.411 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.412 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.412 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:26.412 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:54.010593+0000", 2024-09-21T15:01:26.412 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011652531000000001 2024-09-21T15:01:26.412 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.412 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.412 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:26.412 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:54.010679+0000", 2024-09-21T15:01:26.412 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6811000000000006e-05 2024-09-21T15:01:26.412 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.412 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.412 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:26.412 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:54.010914+0000", 2024-09-21T15:01:26.412 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000234071 2024-09-21T15:01:26.413 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:26.413 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:26.413 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:26.413 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.413 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.413 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:137705 3.b 3:d0cb966d:::rbd_data.1136f64ddcfe.0000000000000005:6a [read 1327104~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-21T15:01:26.413 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:58.376355+0000", 2024-09-21T15:01:26.413 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 27.977109174999999, 2024-09-21T15:01:26.413 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011637611000000001, 2024-09-21T15:01:26.413 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:26.413 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:26.413 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:26.413 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:26.413 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:26.413 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 137705 2024-09-21T15:01:26.414 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.414 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:26.414 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.414 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:26.414 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376355+0000", 2024-09-21T15:01:26.414 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:26.414 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.414 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.414 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:26.414 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376355+0000", 2024-09-21T15:01:26.414 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:26.414 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.414 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.414 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:26.415 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376354+0000", 2024-09-21T15:01:26.415 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:26.415 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.415 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.415 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:26.415 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376358+0000", 2024-09-21T15:01:26.415 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.179e-06 2024-09-21T15:01:26.415 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.415 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.415 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:26.415 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376369+0000", 2024-09-21T15:01:26.415 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1766000000000001e-05 2024-09-21T15:01:26.415 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.416 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.416 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:26.416 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376376+0000", 2024-09-21T15:01:26.416 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6019999999999996e-06 2024-09-21T15:01:26.416 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.416 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.416 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:26.416 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.387882+0000", 2024-09-21T15:01:26.416 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011506126 2024-09-21T15:01:26.416 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.416 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.416 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:26.416 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.387929+0000", 2024-09-21T15:01:26.416 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6742000000000001e-05 2024-09-21T15:01:26.417 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.417 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.417 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:26.417 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.387992+0000", 2024-09-21T15:01:26.417 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3656000000000002e-05 2024-09-21T15:01:26.417 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:26.417 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:26.417 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:26.417 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.417 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.417 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:137706 3.b 3:d0cb966d:::rbd_data.1136f64ddcfe.0000000000000005:6a [read 1441792~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-21T15:01:26.417 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:58.376385+0000", 2024-09-21T15:01:26.417 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 27.977079205999999, 2024-09-21T15:01:26.417 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011783609, 2024-09-21T15:01:26.418 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:26.418 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:26.418 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:26.418 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:26.418 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:26.418 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 137706 2024-09-21T15:01:26.418 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.418 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:26.418 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.418 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:26.418 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376385+0000", 2024-09-21T15:01:26.418 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:26.418 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.418 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.419 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:26.419 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376385+0000", 2024-09-21T15:01:26.419 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:26.419 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.419 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.419 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:26.419 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376384+0000", 2024-09-21T15:01:26.419 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:26.419 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.419 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.419 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:26.419 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376387+0000", 2024-09-21T15:01:26.419 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9809999999999999e-06 2024-09-21T15:01:26.419 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.420 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.420 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:26.420 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376398+0000", 2024-09-21T15:01:26.420 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0319000000000001e-05 2024-09-21T15:01:26.420 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.420 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.420 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:26.420 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376404+0000", 2024-09-21T15:01:26.420 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5339999999999996e-06 2024-09-21T15:01:26.420 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.420 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.420 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:26.420 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.388025+0000", 2024-09-21T15:01:26.420 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011620449999999999 2024-09-21T15:01:26.420 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.421 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.421 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:26.421 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.388082+0000", 2024-09-21T15:01:26.421 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7065999999999999e-05 2024-09-21T15:01:26.421 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.421 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.421 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:26.421 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.388168+0000", 2024-09-21T15:01:26.421 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6689999999999998e-05 2024-09-21T15:01:26.421 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:26.421 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:26.421 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:26.421 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.421 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.421 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:137707 3.b 3:d0cb966d:::rbd_data.1136f64ddcfe.0000000000000005:6a [read 1556480~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-21T15:01:26.422 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:58.376428+0000", 2024-09-21T15:01:26.422 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 27.977035713999999, 2024-09-21T15:01:26.422 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012004865999999999, 2024-09-21T15:01:26.422 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:26.422 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:26.422 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:26.422 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:26.422 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:26.422 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 137707 2024-09-21T15:01:26.422 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.422 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:26.422 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.422 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:26.422 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376428+0000", 2024-09-21T15:01:26.422 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:26.423 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.423 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.423 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:26.423 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376428+0000", 2024-09-21T15:01:26.423 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:26.423 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.423 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.423 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:26.423 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376428+0000", 2024-09-21T15:01:26.423 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:26.423 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.423 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.423 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:26.423 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376432+0000", 2024-09-21T15:01:26.423 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2119999999999997e-06 2024-09-21T15:01:26.423 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.424 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.424 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:26.424 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376448+0000", 2024-09-21T15:01:26.424 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5913000000000002e-05 2024-09-21T15:01:26.424 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.424 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.424 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:26.424 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376455+0000", 2024-09-21T15:01:26.424 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6359999999999994e-06 2024-09-21T15:01:26.424 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.424 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.424 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:26.424 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.388197+0000", 2024-09-21T15:01:26.424 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011741843 2024-09-21T15:01:26.424 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.425 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.425 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:26.425 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.388249+0000", 2024-09-21T15:01:26.425 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1549999999999999e-05 2024-09-21T15:01:26.425 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.425 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.425 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:26.425 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.388433+0000", 2024-09-21T15:01:26.425 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018437999999999999 2024-09-21T15:01:26.425 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:26.425 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:26.425 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:26.425 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.425 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.425 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:137708 3.b 3:d0cb966d:::rbd_data.1136f64ddcfe.0000000000000005:6a [read 1671168~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-21T15:01:26.425 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:58.376462+0000", 2024-09-21T15:01:26.426 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 27.977001926, 2024-09-21T15:01:26.426 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012315491, 2024-09-21T15:01:26.426 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:26.426 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:26.426 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:26.426 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:26.426 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:26.426 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 137708 2024-09-21T15:01:26.426 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.426 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:26.426 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.426 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:26.426 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376462+0000", 2024-09-21T15:01:26.426 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:26.426 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.426 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.427 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:26.427 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376462+0000", 2024-09-21T15:01:26.427 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:26.427 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.427 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.427 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:26.427 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376461+0000", 2024-09-21T15:01:26.427 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:26.427 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.427 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.427 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:26.427 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376465+0000", 2024-09-21T15:01:26.427 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7069999999999998e-06 2024-09-21T15:01:26.427 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.427 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.428 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:26.428 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376473+0000", 2024-09-21T15:01:26.428 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6669999999999996e-06 2024-09-21T15:01:26.428 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.428 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.428 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:26.428 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376479+0000", 2024-09-21T15:01:26.428 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.263e-06 2024-09-21T15:01:26.428 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.428 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.428 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:26.428 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.388489+0000", 2024-09-21T15:01:26.428 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012009953 2024-09-21T15:01:26.428 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.428 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.428 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:26.429 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.388625+0000", 2024-09-21T15:01:26.429 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013548799999999999 2024-09-21T15:01:26.429 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.429 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.429 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:26.429 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.388778+0000", 2024-09-21T15:01:26.429 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015302299999999999 2024-09-21T15:01:26.429 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:26.429 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:26.429 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:26.429 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.429 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.429 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:137709 3.b 3:d0cb966d:::rbd_data.1136f64ddcfe.0000000000000005:6a [read 1785856~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-21T15:01:26.429 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:58.376485+0000", 2024-09-21T15:01:26.429 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 27.976978593999998, 2024-09-21T15:01:26.429 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012466549, 2024-09-21T15:01:26.430 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:26.430 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:26.430 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:26.430 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:26.430 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:26.430 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 137709 2024-09-21T15:01:26.430 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.430 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:26.430 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.430 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:26.430 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376485+0000", 2024-09-21T15:01:26.430 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:26.430 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.430 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.430 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:26.431 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376485+0000", 2024-09-21T15:01:26.431 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:26.431 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.431 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.431 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:26.431 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376485+0000", 2024-09-21T15:01:26.431 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:26.431 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.431 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.431 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:26.431 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376492+0000", 2024-09-21T15:01:26.431 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8929999999999999e-06 2024-09-21T15:01:26.431 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.431 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.432 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:26.432 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376500+0000", 2024-09-21T15:01:26.432 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7989999999999994e-06 2024-09-21T15:01:26.432 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.432 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.432 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:26.432 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376506+0000", 2024-09-21T15:01:26.432 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4799999999999998e-06 2024-09-21T15:01:26.432 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.432 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.432 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:26.432 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.388810+0000", 2024-09-21T15:01:26.432 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012303727 2024-09-21T15:01:26.432 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.432 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.433 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:26.433 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.388871+0000", 2024-09-21T15:01:26.433 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1278000000000005e-05 2024-09-21T15:01:26.433 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.433 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.433 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:26.433 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.388952+0000", 2024-09-21T15:01:26.433 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0770000000000001e-05 2024-09-21T15:01:26.433 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:26.433 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:26.433 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:26.433 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.433 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.433 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:137710 3.b 3:d0cb966d:::rbd_data.1136f64ddcfe.0000000000000005:6a [read 1900544~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-21T15:01:26.433 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:58.376512+0000", 2024-09-21T15:01:26.434 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 27.976952391000001, 2024-09-21T15:01:26.434 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012746351, 2024-09-21T15:01:26.434 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:26.434 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:26.434 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:26.434 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:26.434 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:26.434 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 137710 2024-09-21T15:01:26.434 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.434 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:26.434 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.434 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:26.434 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376512+0000", 2024-09-21T15:01:26.434 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:26.435 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.435 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.435 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:26.435 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376512+0000", 2024-09-21T15:01:26.435 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:26.435 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.435 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.435 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:26.435 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376511+0000", 2024-09-21T15:01:26.435 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:26.435 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.435 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.435 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:26.435 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376514+0000", 2024-09-21T15:01:26.436 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1719999999999999e-06 2024-09-21T15:01:26.436 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.436 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.436 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:26.436 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376522+0000", 2024-09-21T15:01:26.436 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1389999999999998e-06 2024-09-21T15:01:26.436 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.436 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.436 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:26.436 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376535+0000", 2024-09-21T15:01:26.436 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3658999999999999e-05 2024-09-21T15:01:26.436 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.436 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.436 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:26.436 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.389005+0000", 2024-09-21T15:01:26.437 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012470261 2024-09-21T15:01:26.437 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.437 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.437 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:26.437 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.389119+0000", 2024-09-21T15:01:26.437 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000113346 2024-09-21T15:01:26.437 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.437 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.437 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:26.437 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.389258+0000", 2024-09-21T15:01:26.437 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013920100000000001 2024-09-21T15:01:26.437 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:26.437 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:26.437 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:26.437 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.438 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.438 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:137711 3.b 3:d0cb966d:::rbd_data.1136f64ddcfe.0000000000000005:6a [read 2015232~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-21T15:01:26.438 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:58.376541+0000", 2024-09-21T15:01:26.438 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 27.976923029000002, 2024-09-21T15:01:26.438 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012849955999999999, 2024-09-21T15:01:26.438 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:26.438 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:26.438 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:26.438 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:26.438 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:26.438 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 137711 2024-09-21T15:01:26.438 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.438 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:26.438 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.438 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:26.438 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376541+0000", 2024-09-21T15:01:26.439 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:26.439 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.439 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.439 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:26.439 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376541+0000", 2024-09-21T15:01:26.439 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:26.439 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.439 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.439 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:26.439 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376540+0000", 2024-09-21T15:01:26.439 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:26.439 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.439 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.439 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:26.439 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376544+0000", 2024-09-21T15:01:26.440 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3249999999999999e-06 2024-09-21T15:01:26.440 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.440 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.440 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:26.440 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376551+0000", 2024-09-21T15:01:26.440 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2189999999999998e-06 2024-09-21T15:01:26.440 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.440 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.440 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:26.440 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376557+0000", 2024-09-21T15:01:26.440 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8869999999999997e-06 2024-09-21T15:01:26.440 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.440 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.440 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:26.441 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.389277+0000", 2024-09-21T15:01:26.441 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012720555999999999 2024-09-21T15:01:26.441 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.441 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.441 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:26.441 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.389325+0000", 2024-09-21T15:01:26.441 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7367000000000002e-05 2024-09-21T15:01:26.441 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.441 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.441 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:26.441 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.389391+0000", 2024-09-21T15:01:26.441 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6095999999999999e-05 2024-09-21T15:01:26.441 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:26.441 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:26.441 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:26.441 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.442 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.442 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:137723 3.b 3:d0cb966d:::rbd_data.1136f64ddcfe.0000000000000005:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-21T15:01:26.442 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:58.377091+0000", 2024-09-21T15:01:26.442 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 27.976372708, 2024-09-21T15:01:26.442 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012594424, 2024-09-21T15:01:26.442 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:26.442 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:26.442 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:26.442 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:26.442 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:26.442 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 137723 2024-09-21T15:01:26.442 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.442 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:26.442 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.442 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:26.442 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.377091+0000", 2024-09-21T15:01:26.443 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:26.443 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.443 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.443 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:26.443 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.377091+0000", 2024-09-21T15:01:26.443 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:26.443 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.443 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.443 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:26.443 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.377090+0000", 2024-09-21T15:01:26.443 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:26.443 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.443 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.443 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:26.443 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.377097+0000", 2024-09-21T15:01:26.444 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7100000000000001e-06 2024-09-21T15:01:26.444 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.444 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.444 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:26.444 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.377120+0000", 2024-09-21T15:01:26.444 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3189000000000001e-05 2024-09-21T15:01:26.444 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.444 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.444 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:26.444 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.377132+0000", 2024-09-21T15:01:26.444 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2041999999999999e-05 2024-09-21T15:01:26.444 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.444 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.444 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:26.444 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.389439+0000", 2024-09-21T15:01:26.444 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.01230698 2024-09-21T15:01:26.445 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.445 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.445 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:26.445 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.389556+0000", 2024-09-21T15:01:26.445 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000116446 2024-09-21T15:01:26.445 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:26.445 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:26.445 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:26.445 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.389686+0000", 2024-09-21T15:01:26.445 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000129966 2024-09-21T15:01:26.445 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:26.445 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:26.445 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:26.445 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:26.445 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:26.446 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:01:26.447 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:01:26.444+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:01:26.447 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T15:01:26.547 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:01:26.544+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:01:26.562 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:26.562 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:01:26.562 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T15:01:26.562 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:26.572 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T15:01:26.649 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:01:26.642+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:01:26.674 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:26.674 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:01:26.674 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T15:01:26.674 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T15:01:26.674 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:26.684 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T15:01:26.749 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:01:26.744+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:01:26.770 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-21T15:01:26.770 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:26.771 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 194, 2024-09-21T15:01:26.771 INFO:teuthology.orchestra.run.smithi032.stdout: "fsid": "caf51e55-9234-4a26-bcd2-d60811962560", 2024-09-21T15:01:26.771 INFO:teuthology.orchestra.run.smithi032.stdout: "created": "2024-09-21T14:57:50.338822+0000", 2024-09-21T15:01:26.771 INFO:teuthology.orchestra.run.smithi032.stdout: "modified": "2024-09-21T15:01:26.162868+0000", 2024-09-21T15:01:26.771 INFO:teuthology.orchestra.run.smithi032.stdout: "last_up_change": "2024-09-21T14:58:01.177141+0000", 2024-09-21T15:01:26.771 INFO:teuthology.orchestra.run.smithi032.stdout: "last_in_change": "2024-09-21T15:01:21.319966+0000", 2024-09-21T15:01:26.771 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-21T15:01:26.771 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_num": 5799936, 2024-09-21T15:01:26.771 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_set": [ 2024-09-21T15:01:26.771 INFO:teuthology.orchestra.run.smithi032.stdout: "pglog_hardlimit", 2024-09-21T15:01:26.771 INFO:teuthology.orchestra.run.smithi032.stdout: "purged_snapdirs", 2024-09-21T15:01:26.772 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_deletes", 2024-09-21T15:01:26.772 INFO:teuthology.orchestra.run.smithi032.stdout: "sortbitwise" 2024-09-21T15:01:26.772 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-21T15:01:26.772 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_version": 5, 2024-09-21T15:01:26.772 INFO:teuthology.orchestra.run.smithi032.stdout: "full_ratio": 0.94999998807907104, 2024-09-21T15:01:26.772 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-21T15:01:26.772 INFO:teuthology.orchestra.run.smithi032.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-21T15:01:26.772 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_snapshot": "", 2024-09-21T15:01:26.772 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_max": 3, 2024-09-21T15:01:26.772 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd": 8, 2024-09-21T15:01:26.772 INFO:teuthology.orchestra.run.smithi032.stdout: "require_min_compat_client": "luminous", 2024-09-21T15:01:26.772 INFO:teuthology.orchestra.run.smithi032.stdout: "min_compat_client": "luminous", 2024-09-21T15:01:26.772 INFO:teuthology.orchestra.run.smithi032.stdout: "require_osd_release": "reef", 2024-09-21T15:01:26.772 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_crimson": false, 2024-09-21T15:01:26.773 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2024-09-21T15:01:26.773 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.773 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 1, 2024-09-21T15:01:26.773 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": ".mgr", 2024-09-21T15:01:26.773 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-09-21T14:58:01.842810+0000", 2024-09-21T15:01:26.773 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-09-21T15:01:26.773 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2024-09-21T15:01:26.773 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-09-21T15:01:26.773 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-09-21T15:01:26.773 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-09-21T15:01:26.773 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-09-21T15:01:26.773 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-09-21T15:01:26.773 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-09-21T15:01:26.773 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T15:01:26.773 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T15:01:26.774 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-09-21T15:01:26.774 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-09-21T15:01:26.774 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 1, 2024-09-21T15:01:26.774 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 1, 2024-09-21T15:01:26.774 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 1, 2024-09-21T15:01:26.774 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 1, 2024-09-21T15:01:26.774 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 1, 2024-09-21T15:01:26.774 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-09-21T15:01:26.774 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-09-21T15:01:26.774 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-09-21T15:01:26.775 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-09-21T15:01:26.775 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-09-21T15:01:26.775 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-09-21T15:01:26.775 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-09-21T15:01:26.775 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.775 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "18", 2024-09-21T15:01:26.775 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-09-21T15:01:26.775 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T15:01:26.775 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T15:01:26.775 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-09-21T15:01:26.775 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-09-21T15:01:26.775 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-09-21T15:01:26.775 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-09-21T15:01:26.776 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-09-21T15:01:26.776 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-09-21T15:01:26.776 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-09-21T15:01:26.776 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-09-21T15:01:26.776 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-09-21T15:01:26.776 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-09-21T15:01:26.776 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-09-21T15:01:26.776 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-09-21T15:01:26.776 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-09-21T15:01:26.776 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-09-21T15:01:26.776 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-09-21T15:01:26.776 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T15:01:26.777 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T15:01:26.777 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T15:01:26.777 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-09-21T15:01:26.777 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-09-21T15:01:26.777 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-09-21T15:01:26.777 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-09-21T15:01:26.777 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-09-21T15:01:26.777 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.777 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-09-21T15:01:26.777 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-09-21T15:01:26.777 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-09-21T15:01:26.777 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-09-21T15:01:26.777 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-09-21T15:01:26.777 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T15:01:26.777 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-09-21T15:01:26.778 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-09-21T15:01:26.778 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-09-21T15:01:26.778 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-09-21T15:01:26.778 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-09-21T15:01:26.778 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-09-21T15:01:26.778 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_max": 32, 2024-09-21T15:01:26.778 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 1 2024-09-21T15:01:26.778 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.778 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-09-21T15:01:26.778 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": {} 2024-09-21T15:01:26.778 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.778 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-09-21T15:01:26.778 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 8, 2024-09-21T15:01:26.779 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 8, 2024-09-21T15:01:26.779 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 0.25, 2024-09-21T15:01:26.779 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 2, 2024-09-21T15:01:26.779 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 2, 2024-09-21T15:01:26.779 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 0.95859527587890625, 2024-09-21T15:01:26.779 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 0.95999997854232788, 2024-09-21T15:01:26.779 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 0.95999997854232788 2024-09-21T15:01:26.779 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:26.779 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.779 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.779 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 2, 2024-09-21T15:01:26.779 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "rbd", 2024-09-21T15:01:26.779 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-09-21T14:58:08.284925+0000", 2024-09-21T15:01:26.779 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 8193, 2024-09-21T15:01:26.779 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T15:01:26.780 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-09-21T15:01:26.780 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-09-21T15:01:26.780 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-09-21T15:01:26.780 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-09-21T15:01:26.780 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-09-21T15:01:26.780 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-09-21T15:01:26.780 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T15:01:26.780 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T15:01:26.780 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-09-21T15:01:26.780 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-09-21T15:01:26.780 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2024-09-21T15:01:26.780 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2024-09-21T15:01:26.780 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2024-09-21T15:01:26.780 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2024-09-21T15:01:26.780 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2024-09-21T15:01:26.781 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-09-21T15:01:26.781 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-09-21T15:01:26.781 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-09-21T15:01:26.781 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-09-21T15:01:26.781 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-09-21T15:01:26.781 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-09-21T15:01:26.781 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-09-21T15:01:26.781 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.781 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "22", 2024-09-21T15:01:26.781 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-09-21T15:01:26.781 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T15:01:26.781 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T15:01:26.781 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-09-21T15:01:26.781 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-09-21T15:01:26.781 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 3, 2024-09-21T15:01:26.782 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 22, 2024-09-21T15:01:26.782 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-09-21T15:01:26.782 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-09-21T15:01:26.782 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-09-21T15:01:26.782 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-09-21T15:01:26.782 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-09-21T15:01:26.782 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-09-21T15:01:26.782 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-09-21T15:01:26.782 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-09-21T15:01:26.782 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-09-21T15:01:26.782 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-09-21T15:01:26.782 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-09-21T15:01:26.782 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T15:01:26.783 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T15:01:26.783 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T15:01:26.783 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-09-21T15:01:26.783 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-09-21T15:01:26.783 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-09-21T15:01:26.783 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-09-21T15:01:26.783 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-09-21T15:01:26.783 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.783 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-09-21T15:01:26.783 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-09-21T15:01:26.783 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-09-21T15:01:26.783 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-09-21T15:01:26.783 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-09-21T15:01:26.783 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T15:01:26.783 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-09-21T15:01:26.784 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-09-21T15:01:26.784 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-09-21T15:01:26.784 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-09-21T15:01:26.784 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-09-21T15:01:26.784 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2024-09-21T15:01:26.784 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-09-21T15:01:26.784 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd": {} 2024-09-21T15:01:26.784 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.784 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-09-21T15:01:26.784 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 5, 2024-09-21T15:01:26.784 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 5, 2024-09-21T15:01:26.784 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 0.75, 2024-09-21T15:01:26.784 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 3.75, 2024-09-21T15:01:26.785 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 3.75, 2024-09-21T15:01:26.785 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 0.95859527587890625, 2024-09-21T15:01:26.785 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 0.95999997854232788, 2024-09-21T15:01:26.785 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 0.95999997854232788 2024-09-21T15:01:26.785 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:26.785 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.785 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.785 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 3, 2024-09-21T15:01:26.785 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "pool_client.0", 2024-09-21T15:01:26.785 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-09-21T14:58:17.510709+0000", 2024-09-21T15:01:26.785 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 8193, 2024-09-21T15:01:26.785 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T15:01:26.786 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-09-21T15:01:26.786 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-09-21T15:01:26.786 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-09-21T15:01:26.786 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-09-21T15:01:26.786 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-09-21T15:01:26.786 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-09-21T15:01:26.786 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T15:01:26.786 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T15:01:26.786 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-09-21T15:01:26.786 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-09-21T15:01:26.786 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 32, 2024-09-21T15:01:26.786 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 32, 2024-09-21T15:01:26.786 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 32, 2024-09-21T15:01:26.787 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 32, 2024-09-21T15:01:26.787 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 32, 2024-09-21T15:01:26.787 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-09-21T15:01:26.787 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-09-21T15:01:26.787 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-09-21T15:01:26.787 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-09-21T15:01:26.787 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-09-21T15:01:26.787 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-09-21T15:01:26.787 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-09-21T15:01:26.787 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.787 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "192", 2024-09-21T15:01:26.787 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-09-21T15:01:26.787 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T15:01:26.787 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T15:01:26.788 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-09-21T15:01:26.788 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-09-21T15:01:26.788 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 122, 2024-09-21T15:01:26.788 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 192, 2024-09-21T15:01:26.788 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-09-21T15:01:26.788 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-09-21T15:01:26.788 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-09-21T15:01:26.788 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-09-21T15:01:26.788 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-09-21T15:01:26.788 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-09-21T15:01:26.788 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-09-21T15:01:26.788 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-09-21T15:01:26.788 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-09-21T15:01:26.788 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-09-21T15:01:26.788 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-09-21T15:01:26.789 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T15:01:26.789 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T15:01:26.789 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T15:01:26.789 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-09-21T15:01:26.789 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-09-21T15:01:26.789 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-09-21T15:01:26.789 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-09-21T15:01:26.789 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-09-21T15:01:26.789 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.789 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-09-21T15:01:26.789 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-09-21T15:01:26.789 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-09-21T15:01:26.790 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-09-21T15:01:26.790 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-09-21T15:01:26.790 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T15:01:26.790 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-09-21T15:01:26.790 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-09-21T15:01:26.790 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-09-21T15:01:26.790 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-09-21T15:01:26.790 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-09-21T15:01:26.790 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2024-09-21T15:01:26.790 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-09-21T15:01:26.790 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd": {} 2024-09-21T15:01:26.790 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.790 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-09-21T15:01:26.790 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 1.4900000095367432, 2024-09-21T15:01:26.791 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 1.4900000095367432, 2024-09-21T15:01:26.791 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 0.87999999523162842, 2024-09-21T15:01:26.791 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 1.309999942779541, 2024-09-21T15:01:26.791 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 1.309999942779541, 2024-09-21T15:01:26.791 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 0.95859527587890625, 2024-09-21T15:01:26.791 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 0.95999997854232788, 2024-09-21T15:01:26.791 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 0.95999997854232788 2024-09-21T15:01:26.791 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:26.791 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:26.791 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-21T15:01:26.791 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2024-09-21T15:01:26.791 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.791 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-09-21T15:01:26.792 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "a4a73316-3a94-42e9-9a51-deb46a66bb06", 2024-09-21T15:01:26.792 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-21T15:01:26.792 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-21T15:01:26.792 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-21T15:01:26.792 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-21T15:01:26.792 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-21T15:01:26.792 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-21T15:01:26.792 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2024-09-21T15:01:26.792 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 192, 2024-09-21T15:01:26.792 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-21T15:01:26.792 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-21T15:01:26.792 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-21T15:01:26.792 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:01:26.792 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.792 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:01:26.792 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6808", 2024-09-21T15:01:26.793 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4016672243 2024-09-21T15:01:26.793 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.793 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.793 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:01:26.793 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6809", 2024-09-21T15:01:26.793 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4016672243 2024-09-21T15:01:26.793 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:26.793 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:26.793 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.793 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-21T15:01:26.793 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:01:26.793 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.794 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:01:26.794 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6810", 2024-09-21T15:01:26.794 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4016672243 2024-09-21T15:01:26.794 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.794 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.794 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:01:26.794 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6811", 2024-09-21T15:01:26.794 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4016672243 2024-09-21T15:01:26.794 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:26.794 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:26.794 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.794 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-21T15:01:26.794 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:01:26.795 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.795 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:01:26.795 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6814", 2024-09-21T15:01:26.795 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4016672243 2024-09-21T15:01:26.795 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.795 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.795 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:01:26.795 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6815", 2024-09-21T15:01:26.795 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4016672243 2024-09-21T15:01:26.795 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:26.796 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:26.796 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.796 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-21T15:01:26.796 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:01:26.796 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.796 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:01:26.796 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6812", 2024-09-21T15:01:26.796 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4016672243 2024-09-21T15:01:26.796 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.796 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.796 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:01:26.796 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6813", 2024-09-21T15:01:26.796 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4016672243 2024-09-21T15:01:26.797 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:26.797 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:26.797 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.797 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6809/4016672243", 2024-09-21T15:01:26.797 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6811/4016672243", 2024-09-21T15:01:26.797 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6815/4016672243", 2024-09-21T15:01:26.797 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6813/4016672243", 2024-09-21T15:01:26.797 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-21T15:01:26.797 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-21T15:01:26.797 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-21T15:01:26.797 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:26.797 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.797 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.797 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-09-21T15:01:26.798 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "006757a0-94d1-4a9b-aef8-73634cc5828e", 2024-09-21T15:01:26.798 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-21T15:01:26.798 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-21T15:01:26.798 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-21T15:01:26.798 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-21T15:01:26.798 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-21T15:01:26.798 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-21T15:01:26.798 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2024-09-21T15:01:26.798 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 192, 2024-09-21T15:01:26.798 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-21T15:01:26.798 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-21T15:01:26.798 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-21T15:01:26.798 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:01:26.799 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.799 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:01:26.799 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6824", 2024-09-21T15:01:26.799 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 641090577 2024-09-21T15:01:26.799 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.799 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.799 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:01:26.799 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6825", 2024-09-21T15:01:26.799 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 641090577 2024-09-21T15:01:26.799 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:26.799 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:26.799 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.799 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-21T15:01:26.800 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:01:26.800 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.800 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:01:26.800 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6826", 2024-09-21T15:01:26.800 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 641090577 2024-09-21T15:01:26.800 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.800 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.800 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:01:26.800 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6827", 2024-09-21T15:01:26.800 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 641090577 2024-09-21T15:01:26.800 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:26.800 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:26.800 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.800 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-21T15:01:26.801 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:01:26.801 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.801 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:01:26.801 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6830", 2024-09-21T15:01:26.801 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 641090577 2024-09-21T15:01:26.801 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.801 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.801 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:01:26.801 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6831", 2024-09-21T15:01:26.801 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 641090577 2024-09-21T15:01:26.801 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:26.801 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:26.801 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.802 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-21T15:01:26.802 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:01:26.802 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.802 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:01:26.802 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6828", 2024-09-21T15:01:26.802 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 641090577 2024-09-21T15:01:26.802 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.802 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.802 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:01:26.802 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6829", 2024-09-21T15:01:26.802 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 641090577 2024-09-21T15:01:26.802 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:26.802 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:26.802 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.803 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6825/641090577", 2024-09-21T15:01:26.803 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6827/641090577", 2024-09-21T15:01:26.803 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6831/641090577", 2024-09-21T15:01:26.803 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6829/641090577", 2024-09-21T15:01:26.803 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-21T15:01:26.803 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-21T15:01:26.803 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-21T15:01:26.803 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:26.803 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.803 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.803 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-09-21T15:01:26.803 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "0b499a23-820e-4e6c-a558-adbf88b7014f", 2024-09-21T15:01:26.803 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-21T15:01:26.803 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-21T15:01:26.803 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-21T15:01:26.804 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-21T15:01:26.804 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-21T15:01:26.804 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-21T15:01:26.804 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2024-09-21T15:01:26.804 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 191, 2024-09-21T15:01:26.804 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-21T15:01:26.804 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-21T15:01:26.804 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-21T15:01:26.804 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:01:26.804 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.804 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:01:26.804 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6800", 2024-09-21T15:01:26.804 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2609154414 2024-09-21T15:01:26.804 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.805 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.805 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:01:26.805 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6801", 2024-09-21T15:01:26.805 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2609154414 2024-09-21T15:01:26.805 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:26.805 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:26.805 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.805 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-21T15:01:26.805 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:01:26.805 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.805 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:01:26.805 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6802", 2024-09-21T15:01:26.806 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2609154414 2024-09-21T15:01:26.806 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.806 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.806 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:01:26.806 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6803", 2024-09-21T15:01:26.806 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2609154414 2024-09-21T15:01:26.806 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:26.806 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:26.806 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.806 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-21T15:01:26.806 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:01:26.806 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.806 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:01:26.806 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6806", 2024-09-21T15:01:26.807 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2609154414 2024-09-21T15:01:26.807 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.807 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.807 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:01:26.807 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6807", 2024-09-21T15:01:26.807 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2609154414 2024-09-21T15:01:26.807 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:26.807 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:26.807 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.807 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-21T15:01:26.807 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:01:26.807 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.807 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:01:26.807 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6804", 2024-09-21T15:01:26.808 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2609154414 2024-09-21T15:01:26.808 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.808 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.808 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:01:26.808 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6805", 2024-09-21T15:01:26.808 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2609154414 2024-09-21T15:01:26.808 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:26.808 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:26.808 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.808 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6801/2609154414", 2024-09-21T15:01:26.808 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6803/2609154414", 2024-09-21T15:01:26.808 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6807/2609154414", 2024-09-21T15:01:26.808 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6805/2609154414", 2024-09-21T15:01:26.808 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-21T15:01:26.808 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-21T15:01:26.808 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-21T15:01:26.809 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:26.809 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.809 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.809 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-09-21T15:01:26.809 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "5a57b3dd-4a07-45b7-b159-3cac4c7eb262", 2024-09-21T15:01:26.809 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-21T15:01:26.809 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-21T15:01:26.809 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-21T15:01:26.809 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 0.833831787109375, 2024-09-21T15:01:26.809 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-21T15:01:26.809 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-21T15:01:26.809 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2024-09-21T15:01:26.809 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 192, 2024-09-21T15:01:26.809 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-21T15:01:26.809 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-21T15:01:26.810 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-21T15:01:26.810 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:01:26.810 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.810 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:01:26.810 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6816", 2024-09-21T15:01:26.810 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1462563376 2024-09-21T15:01:26.810 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.810 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.810 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:01:26.810 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6817", 2024-09-21T15:01:26.810 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1462563376 2024-09-21T15:01:26.810 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:26.810 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:26.810 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.811 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-21T15:01:26.811 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:01:26.811 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.811 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:01:26.811 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6818", 2024-09-21T15:01:26.811 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1462563376 2024-09-21T15:01:26.811 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.811 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.811 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:01:26.811 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6819", 2024-09-21T15:01:26.811 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1462563376 2024-09-21T15:01:26.811 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:26.811 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:26.812 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.812 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-21T15:01:26.812 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:01:26.812 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.812 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:01:26.812 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6822", 2024-09-21T15:01:26.812 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1462563376 2024-09-21T15:01:26.812 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.812 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.812 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:01:26.812 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6823", 2024-09-21T15:01:26.812 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1462563376 2024-09-21T15:01:26.812 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:26.813 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:26.813 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.813 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-21T15:01:26.813 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:01:26.813 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.813 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:01:26.813 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6820", 2024-09-21T15:01:26.813 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1462563376 2024-09-21T15:01:26.813 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.813 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.813 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:01:26.813 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6821", 2024-09-21T15:01:26.813 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1462563376 2024-09-21T15:01:26.813 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:26.813 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:26.814 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.814 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6817/1462563376", 2024-09-21T15:01:26.814 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6819/1462563376", 2024-09-21T15:01:26.814 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6823/1462563376", 2024-09-21T15:01:26.814 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6821/1462563376", 2024-09-21T15:01:26.814 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-21T15:01:26.814 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-21T15:01:26.814 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-21T15:01:26.814 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:26.814 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.814 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.814 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-09-21T15:01:26.814 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "39a00807-7517-474e-8842-623dd7093cc5", 2024-09-21T15:01:26.815 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-21T15:01:26.815 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 0, 2024-09-21T15:01:26.815 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 0, 2024-09-21T15:01:26.815 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-21T15:01:26.815 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-21T15:01:26.815 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-21T15:01:26.815 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2024-09-21T15:01:26.815 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 147, 2024-09-21T15:01:26.815 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-21T15:01:26.815 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-21T15:01:26.816 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-21T15:01:26.816 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:01:26.816 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.816 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:01:26.816 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6801", 2024-09-21T15:01:26.816 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 144412042 2024-09-21T15:01:26.816 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.817 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.817 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:01:26.817 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6803", 2024-09-21T15:01:26.817 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 144412042 2024-09-21T15:01:26.817 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:26.817 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:26.817 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.818 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-21T15:01:26.818 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:01:26.818 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.818 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:01:26.818 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6805", 2024-09-21T15:01:26.818 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 144412042 2024-09-21T15:01:26.818 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.818 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.819 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:01:26.819 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6807", 2024-09-21T15:01:26.819 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 144412042 2024-09-21T15:01:26.819 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:26.819 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:26.819 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.819 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-21T15:01:26.819 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:01:26.820 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.820 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:01:26.820 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6812", 2024-09-21T15:01:26.820 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 144412042 2024-09-21T15:01:26.820 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.820 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.820 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:01:26.820 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6814", 2024-09-21T15:01:26.820 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 144412042 2024-09-21T15:01:26.821 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:26.821 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:26.821 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.821 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-21T15:01:26.821 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:01:26.821 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.821 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:01:26.822 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6809", 2024-09-21T15:01:26.822 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 144412042 2024-09-21T15:01:26.822 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.822 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.822 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:01:26.822 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6811", 2024-09-21T15:01:26.823 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 144412042 2024-09-21T15:01:26.823 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:26.823 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:26.823 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.823 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.46:6803/144412042", 2024-09-21T15:01:26.823 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.46:6807/144412042", 2024-09-21T15:01:26.823 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.46:6814/144412042", 2024-09-21T15:01:26.824 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.46:6811/144412042", 2024-09-21T15:01:26.824 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-21T15:01:26.824 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-21T15:01:26.824 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-21T15:01:26.824 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:26.824 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.824 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.825 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-09-21T15:01:26.825 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "edf9f045-ddeb-4504-a0fc-1a0e4e853550", 2024-09-21T15:01:26.825 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-21T15:01:26.825 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-21T15:01:26.825 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-21T15:01:26.825 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-21T15:01:26.825 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-21T15:01:26.826 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-21T15:01:26.826 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2024-09-21T15:01:26.826 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 177, 2024-09-21T15:01:26.826 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-21T15:01:26.826 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-21T15:01:26.826 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-21T15:01:26.826 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:01:26.827 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.827 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:01:26.827 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6800", 2024-09-21T15:01:26.827 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 849101442 2024-09-21T15:01:26.827 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.827 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.827 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:01:26.827 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6802", 2024-09-21T15:01:26.828 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 849101442 2024-09-21T15:01:26.828 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:26.828 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:26.828 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.828 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-21T15:01:26.828 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:01:26.828 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.829 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:01:26.829 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6804", 2024-09-21T15:01:26.829 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 849101442 2024-09-21T15:01:26.829 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.829 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.829 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:01:26.829 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6806", 2024-09-21T15:01:26.830 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 849101442 2024-09-21T15:01:26.830 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:26.830 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:26.830 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.830 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-21T15:01:26.830 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:01:26.830 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.831 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:01:26.831 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6813", 2024-09-21T15:01:26.831 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 849101442 2024-09-21T15:01:26.831 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.831 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.831 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:01:26.832 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6815", 2024-09-21T15:01:26.832 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 849101442 2024-09-21T15:01:26.832 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:26.832 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:26.832 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.832 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-21T15:01:26.832 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:01:26.833 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.833 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:01:26.833 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6808", 2024-09-21T15:01:26.833 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 849101442 2024-09-21T15:01:26.833 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.833 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.833 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:01:26.834 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6810", 2024-09-21T15:01:26.834 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 849101442 2024-09-21T15:01:26.834 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:26.834 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:26.834 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.834 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.46:6802/849101442", 2024-09-21T15:01:26.834 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.46:6806/849101442", 2024-09-21T15:01:26.835 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.46:6815/849101442", 2024-09-21T15:01:26.835 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.46:6810/849101442", 2024-09-21T15:01:26.835 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-21T15:01:26.835 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-21T15:01:26.835 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-21T15:01:26.835 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:26.835 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.835 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.836 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-09-21T15:01:26.836 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "c820ea6b-88fe-4421-83ec-17c466691fea", 2024-09-21T15:01:26.836 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-21T15:01:26.836 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-21T15:01:26.836 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-21T15:01:26.836 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 0.834930419921875, 2024-09-21T15:01:26.836 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-21T15:01:26.836 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-21T15:01:26.836 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2024-09-21T15:01:26.836 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 192, 2024-09-21T15:01:26.837 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-21T15:01:26.837 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-21T15:01:26.837 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-21T15:01:26.837 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:01:26.837 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.837 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:01:26.837 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6817", 2024-09-21T15:01:26.837 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 847189172 2024-09-21T15:01:26.837 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.837 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.837 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:01:26.837 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6818", 2024-09-21T15:01:26.838 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 847189172 2024-09-21T15:01:26.838 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:26.838 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:26.838 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.838 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-21T15:01:26.838 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:01:26.838 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.838 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:01:26.838 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6820", 2024-09-21T15:01:26.838 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 847189172 2024-09-21T15:01:26.838 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.838 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.838 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:01:26.838 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6822", 2024-09-21T15:01:26.839 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 847189172 2024-09-21T15:01:26.839 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:26.839 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:26.839 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.839 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-21T15:01:26.839 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:01:26.839 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.839 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:01:26.839 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6827", 2024-09-21T15:01:26.839 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 847189172 2024-09-21T15:01:26.839 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.839 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.840 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:01:26.840 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6828", 2024-09-21T15:01:26.840 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 847189172 2024-09-21T15:01:26.840 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:26.840 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:26.840 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.840 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-21T15:01:26.840 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:01:26.840 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.840 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:01:26.840 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6823", 2024-09-21T15:01:26.840 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 847189172 2024-09-21T15:01:26.840 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.840 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.840 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:01:26.841 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6825", 2024-09-21T15:01:26.841 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 847189172 2024-09-21T15:01:26.841 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:26.841 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:26.841 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.841 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.46:6818/847189172", 2024-09-21T15:01:26.841 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.46:6822/847189172", 2024-09-21T15:01:26.841 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.46:6828/847189172", 2024-09-21T15:01:26.841 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.46:6825/847189172", 2024-09-21T15:01:26.841 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-21T15:01:26.841 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-21T15:01:26.841 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-21T15:01:26.841 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:26.841 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.842 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.842 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-09-21T15:01:26.842 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "3a2f82a4-28a0-4fcb-9d6b-2285fa4d07ac", 2024-09-21T15:01:26.842 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-21T15:01:26.842 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-21T15:01:26.842 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-21T15:01:26.842 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-21T15:01:26.842 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-21T15:01:26.842 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-21T15:01:26.842 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 15, 2024-09-21T15:01:26.842 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 193, 2024-09-21T15:01:26.842 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-21T15:01:26.843 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-21T15:01:26.843 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-21T15:01:26.843 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:01:26.843 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.843 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:01:26.843 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6816", 2024-09-21T15:01:26.843 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3894963553 2024-09-21T15:01:26.843 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.843 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.843 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:01:26.843 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6819", 2024-09-21T15:01:26.843 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3894963553 2024-09-21T15:01:26.843 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:26.843 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:26.844 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.844 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-21T15:01:26.844 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:01:26.844 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.844 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:01:26.844 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6821", 2024-09-21T15:01:26.844 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3894963553 2024-09-21T15:01:26.844 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.844 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.844 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:01:26.844 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6824", 2024-09-21T15:01:26.844 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3894963553 2024-09-21T15:01:26.844 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:26.844 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:26.845 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.845 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-21T15:01:26.845 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:01:26.845 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.845 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:01:26.845 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6830", 2024-09-21T15:01:26.845 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3894963553 2024-09-21T15:01:26.845 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.845 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.845 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:01:26.845 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6831", 2024-09-21T15:01:26.845 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3894963553 2024-09-21T15:01:26.845 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:26.845 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:26.845 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.846 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-21T15:01:26.846 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:01:26.846 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.846 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:01:26.846 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6826", 2024-09-21T15:01:26.846 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3894963553 2024-09-21T15:01:26.846 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.846 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.846 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:01:26.846 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6829", 2024-09-21T15:01:26.846 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3894963553 2024-09-21T15:01:26.846 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:26.846 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:26.847 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.847 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.46:6819/3894963553", 2024-09-21T15:01:26.847 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.46:6824/3894963553", 2024-09-21T15:01:26.847 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.46:6831/3894963553", 2024-09-21T15:01:26.847 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.46:6829/3894963553", 2024-09-21T15:01:26.847 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-21T15:01:26.847 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-21T15:01:26.847 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-21T15:01:26.847 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:26.847 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:26.847 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-21T15:01:26.847 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_xinfo": [ 2024-09-21T15:01:26.847 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.847 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-09-21T15:01:26.847 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-21T15:01:26.848 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-21T15:01:26.848 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-21T15:01:26.848 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-21T15:01:26.848 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-21T15:01:26.848 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-21T14:57:57.458317+0000", 2024-09-21T15:01:26.848 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-21T15:01:26.848 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.848 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.848 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-09-21T15:01:26.848 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-21T15:01:26.848 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-21T15:01:26.848 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-21T15:01:26.848 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-21T15:01:26.848 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-21T15:01:26.849 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-21T14:57:57.907624+0000", 2024-09-21T15:01:26.849 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-21T15:01:26.849 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.849 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.849 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-09-21T15:01:26.849 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-21T15:01:26.849 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-21T15:01:26.849 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-21T15:01:26.849 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-21T15:01:26.849 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-21T15:01:26.849 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-21T14:57:57.278817+0000", 2024-09-21T15:01:26.849 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-21T15:01:26.849 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.849 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.849 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-09-21T15:01:26.849 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-21T15:01:26.850 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-21T15:01:26.850 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-21T15:01:26.850 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-21T15:01:26.850 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-21T15:01:26.850 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-21T14:57:57.500449+0000", 2024-09-21T15:01:26.850 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-21T15:01:26.850 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.850 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.850 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-09-21T15:01:26.850 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-21T15:01:26.850 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-21T15:01:26.850 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-21T15:01:26.850 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-21T15:01:26.850 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 65536, 2024-09-21T15:01:26.851 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-21T14:57:57.504325+0000", 2024-09-21T15:01:26.851 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-21T15:01:26.851 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.851 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.851 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-09-21T15:01:26.851 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-21T15:01:26.851 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-21T15:01:26.851 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-21T15:01:26.851 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-21T15:01:26.851 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-21T15:01:26.851 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-21T14:57:57.777525+0000", 2024-09-21T15:01:26.851 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-21T15:01:26.851 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.851 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.851 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-09-21T15:01:26.852 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-21T15:01:26.852 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-21T15:01:26.852 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-21T15:01:26.852 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-21T15:01:26.852 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-21T15:01:26.852 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-21T14:57:57.698961+0000", 2024-09-21T15:01:26.852 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-21T15:01:26.852 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.852 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.852 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-09-21T15:01:26.852 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-21T15:01:26.852 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-21T15:01:26.852 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-21T15:01:26.852 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-21T15:01:26.852 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-21T15:01:26.853 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-21T14:57:57.932625+0000", 2024-09-21T15:01:26.853 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-21T15:01:26.853 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:26.853 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-21T15:01:26.853 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap": [ 2024-09-21T15:01:26.853 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.853 INFO:teuthology.orchestra.run.smithi032.stdout: "pgid": "3.2", 2024-09-21T15:01:26.853 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2024-09-21T15:01:26.853 INFO:teuthology.orchestra.run.smithi032.stdout: 2, 2024-09-21T15:01:26.853 INFO:teuthology.orchestra.run.smithi032.stdout: 3 2024-09-21T15:01:26.853 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:26.853 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.853 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.853 INFO:teuthology.orchestra.run.smithi032.stdout: "pgid": "3.4", 2024-09-21T15:01:26.853 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2024-09-21T15:01:26.854 INFO:teuthology.orchestra.run.smithi032.stdout: 4, 2024-09-21T15:01:26.854 INFO:teuthology.orchestra.run.smithi032.stdout: 2 2024-09-21T15:01:26.854 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:26.854 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.854 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.854 INFO:teuthology.orchestra.run.smithi032.stdout: "pgid": "3.a", 2024-09-21T15:01:26.854 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2024-09-21T15:01:26.854 INFO:teuthology.orchestra.run.smithi032.stdout: 1, 2024-09-21T15:01:26.854 INFO:teuthology.orchestra.run.smithi032.stdout: 0 2024-09-21T15:01:26.854 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:26.854 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:26.854 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-21T15:01:26.854 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_items": [], 2024-09-21T15:01:26.854 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_primaries": [], 2024-09-21T15:01:26.855 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_temp": [], 2024-09-21T15:01:26.855 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_temp": [], 2024-09-21T15:01:26.855 INFO:teuthology.orchestra.run.smithi032.stdout: "blocklist": {}, 2024-09-21T15:01:26.855 INFO:teuthology.orchestra.run.smithi032.stdout: "range_blocklist": {}, 2024-09-21T15:01:26.855 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profiles": { 2024-09-21T15:01:26.855 INFO:teuthology.orchestra.run.smithi032.stdout: "default": { 2024-09-21T15:01:26.855 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2024-09-21T15:01:26.855 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2024-09-21T15:01:26.855 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "1", 2024-09-21T15:01:26.855 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2024-09-21T15:01:26.855 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van" 2024-09-21T15:01:26.855 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:26.855 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.855 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps_queue": [ 2024-09-21T15:01:26.855 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.856 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 3, 2024-09-21T15:01:26.856 INFO:teuthology.orchestra.run.smithi032.stdout: "snaps": [ 2024-09-21T15:01:26.856 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.856 INFO:teuthology.orchestra.run.smithi032.stdout: "begin": 91, 2024-09-21T15:01:26.856 INFO:teuthology.orchestra.run.smithi032.stdout: "length": 1 2024-09-21T15:01:26.856 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.856 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.856 INFO:teuthology.orchestra.run.smithi032.stdout: "begin": 94, 2024-09-21T15:01:26.856 INFO:teuthology.orchestra.run.smithi032.stdout: "length": 1 2024-09-21T15:01:26.856 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:26.856 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:26.856 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:26.856 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-21T15:01:26.856 INFO:teuthology.orchestra.run.smithi032.stdout: "new_removed_snaps": [], 2024-09-21T15:01:26.856 INFO:teuthology.orchestra.run.smithi032.stdout: "new_purged_snaps": [], 2024-09-21T15:01:26.857 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_node_flags": {}, 2024-09-21T15:01:26.857 INFO:teuthology.orchestra.run.smithi032.stdout: "device_class_flags": {}, 2024-09-21T15:01:26.857 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode": { 2024-09-21T15:01:26.857 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_enabled": false, 2024-09-21T15:01:26.857 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_bucket_count": 0, 2024-09-21T15:01:26.857 INFO:teuthology.orchestra.run.smithi032.stdout: "degraded_stretch_mode": 0, 2024-09-21T15:01:26.857 INFO:teuthology.orchestra.run.smithi032.stdout: "recovering_stretch_mode": 0, 2024-09-21T15:01:26.857 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_bucket": 0 2024-09-21T15:01:26.857 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:26.857 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:26.859 INFO:tasks.thrashosds.thrasher:j is {'epoch': 194, 'fsid': 'caf51e55-9234-4a26-bcd2-d60811962560', 'created': '2024-09-21T14:57:50.338822+0000', 'modified': '2024-09-21T15:01:26.162868+0000', 'last_up_change': '2024-09-21T14:58:01.177141+0000', 'last_in_change': '2024-09-21T15:01:21.319966+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T14:58:01.842810+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.9585952758789062, 'average_primary_affinity': 0.9599999785423279, 'average_primary_affinity_weighted': 0.9599999785423279}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-21T14:58:08.284925+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 22, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 5, 'score_stable': 5, 'optimal_score': 0.75, 'raw_score_acting': 3.75, 'raw_score_stable': 3.75, 'primary_affinity_weighted': 0.9585952758789062, 'average_primary_affinity': 0.9599999785423279, 'average_primary_affinity_weighted': 0.9599999785423279}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-21T14:58:17.510709+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '192', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 122, 'snap_epoch': 192, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.4900000095367432, 'score_stable': 1.4900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.309999942779541, 'raw_score_stable': 1.309999942779541, 'primary_affinity_weighted': 0.9585952758789062, 'average_primary_affinity': 0.9599999785423279, 'average_primary_affinity_weighted': 0.9599999785423279}}], 'osds': [{'osd': 0, 'uuid': 'a4a73316-3a94-42e9-9a51-deb46a66bb06', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 192, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6808', 'nonce': 4016672243}, {'type': 'v1', 'addr': '172.21.15.32:6809', 'nonce': 4016672243}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6810', 'nonce': 4016672243}, {'type': 'v1', 'addr': '172.21.15.32:6811', 'nonce': 4016672243}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6814', 'nonce': 4016672243}, {'type': 'v1', 'addr': '172.21.15.32:6815', 'nonce': 4016672243}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6812', 'nonce': 4016672243}, {'type': 'v1', 'addr': '172.21.15.32:6813', 'nonce': 4016672243}]}, 'public_addr': '172.21.15.32:6809/4016672243', 'cluster_addr': '172.21.15.32:6811/4016672243', 'heartbeat_back_addr': '172.21.15.32:6815/4016672243', 'heartbeat_front_addr': '172.21.15.32:6813/4016672243', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '006757a0-94d1-4a9b-aef8-73634cc5828e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 192, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6824', 'nonce': 641090577}, {'type': 'v1', 'addr': '172.21.15.32:6825', 'nonce': 641090577}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6826', 'nonce': 641090577}, {'type': 'v1', 'addr': '172.21.15.32:6827', 'nonce': 641090577}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6830', 'nonce': 641090577}, {'type': 'v1', 'addr': '172.21.15.32:6831', 'nonce': 641090577}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6828', 'nonce': 641090577}, {'type': 'v1', 'addr': '172.21.15.32:6829', 'nonce': 641090577}]}, 'public_addr': '172.21.15.32:6825/641090577', 'cluster_addr': '172.21.15.32:6827/641090577', 'heartbeat_back_addr': '172.21.15.32:6831/641090577', 'heartbeat_front_addr': '172.21.15.32:6829/641090577', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '0b499a23-820e-4e6c-a558-adbf88b7014f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 191, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6800', 'nonce': 2609154414}, {'type': 'v1', 'addr': '172.21.15.32:6801', 'nonce': 2609154414}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6802', 'nonce': 2609154414}, {'type': 'v1', 'addr': '172.21.15.32:6803', 'nonce': 2609154414}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6806', 'nonce': 2609154414}, {'type': 'v1', 'addr': '172.21.15.32:6807', 'nonce': 2609154414}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6804', 'nonce': 2609154414}, {'type': 'v1', 'addr': '172.21.15.32:6805', 'nonce': 2609154414}]}, 'public_addr': '172.21.15.32:6801/2609154414', 'cluster_addr': '172.21.15.32:6803/2609154414', 'heartbeat_back_addr': '172.21.15.32:6807/2609154414', 'heartbeat_front_addr': '172.21.15.32:6805/2609154414', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '5a57b3dd-4a07-45b7-b159-3cac4c7eb262', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.833831787109375, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 192, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6816', 'nonce': 1462563376}, {'type': 'v1', 'addr': '172.21.15.32:6817', 'nonce': 1462563376}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6818', 'nonce': 1462563376}, {'type': 'v1', 'addr': '172.21.15.32:6819', 'nonce': 1462563376}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6822', 'nonce': 1462563376}, {'type': 'v1', 'addr': '172.21.15.32:6823', 'nonce': 1462563376}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6820', 'nonce': 1462563376}, {'type': 'v1', 'addr': '172.21.15.32:6821', 'nonce': 1462563376}]}, 'public_addr': '172.21.15.32:6817/1462563376', 'cluster_addr': '172.21.15.32:6819/1462563376', 'heartbeat_back_addr': '172.21.15.32:6823/1462563376', 'heartbeat_front_addr': '172.21.15.32:6821/1462563376', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '39a00807-7517-474e-8842-623dd7093cc5', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 147, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6801', 'nonce': 144412042}, {'type': 'v1', 'addr': '172.21.15.46:6803', 'nonce': 144412042}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6805', 'nonce': 144412042}, {'type': 'v1', 'addr': '172.21.15.46:6807', 'nonce': 144412042}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6812', 'nonce': 144412042}, {'type': 'v1', 'addr': '172.21.15.46:6814', 'nonce': 144412042}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6809', 'nonce': 144412042}, {'type': 'v1', 'addr': '172.21.15.46:6811', 'nonce': 144412042}]}, 'public_addr': '172.21.15.46:6803/144412042', 'cluster_addr': '172.21.15.46:6807/144412042', 'heartbeat_back_addr': '172.21.15.46:6814/144412042', 'heartbeat_front_addr': '172.21.15.46:6811/144412042', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'edf9f045-ddeb-4504-a0fc-1a0e4e853550', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 177, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6800', 'nonce': 849101442}, {'type': 'v1', 'addr': '172.21.15.46:6802', 'nonce': 849101442}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6804', 'nonce': 849101442}, {'type': 'v1', 'addr': '172.21.15.46:6806', 'nonce': 849101442}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6813', 'nonce': 849101442}, {'type': 'v1', 'addr': '172.21.15.46:6815', 'nonce': 849101442}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6808', 'nonce': 849101442}, {'type': 'v1', 'addr': '172.21.15.46:6810', 'nonce': 849101442}]}, 'public_addr': '172.21.15.46:6802/849101442', 'cluster_addr': '172.21.15.46:6806/849101442', 'heartbeat_back_addr': '172.21.15.46:6815/849101442', 'heartbeat_front_addr': '172.21.15.46:6810/849101442', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'c820ea6b-88fe-4421-83ec-17c466691fea', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.834930419921875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 192, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6817', 'nonce': 847189172}, {'type': 'v1', 'addr': '172.21.15.46:6818', 'nonce': 847189172}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6820', 'nonce': 847189172}, {'type': 'v1', 'addr': '172.21.15.46:6822', 'nonce': 847189172}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6827', 'nonce': 847189172}, {'type': 'v1', 'addr': '172.21.15.46:6828', 'nonce': 847189172}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6823', 'nonce': 847189172}, {'type': 'v1', 'addr': '172.21.15.46:6825', 'nonce': 847189172}]}, 'public_addr': '172.21.15.46:6818/847189172', 'cluster_addr': '172.21.15.46:6822/847189172', 'heartbeat_back_addr': '172.21.15.46:6828/847189172', 'heartbeat_front_addr': '172.21.15.46:6825/847189172', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '3a2f82a4-28a0-4fcb-9d6b-2285fa4d07ac', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 193, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6816', 'nonce': 3894963553}, {'type': 'v1', 'addr': '172.21.15.46:6819', 'nonce': 3894963553}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6821', 'nonce': 3894963553}, {'type': 'v1', 'addr': '172.21.15.46:6824', 'nonce': 3894963553}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6830', 'nonce': 3894963553}, {'type': 'v1', 'addr': '172.21.15.46:6831', 'nonce': 3894963553}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6826', 'nonce': 3894963553}, {'type': 'v1', 'addr': '172.21.15.46:6829', 'nonce': 3894963553}]}, 'public_addr': '172.21.15.46:6819/3894963553', 'cluster_addr': '172.21.15.46:6824/3894963553', 'heartbeat_back_addr': '172.21.15.46:6831/3894963553', 'heartbeat_front_addr': '172.21.15.46:6829/3894963553', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:57:57.458317+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:57:57.907624+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:57:57.278817+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:57:57.500449+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-21T14:57:57.504325+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:57:57.777525+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:57:57.698961+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:57:57.932625+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.2', 'osds': [2, 3]}, {'pgid': '3.4', 'osds': [4, 2]}, {'pgid': '3.a', 'osds': [1, 0]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 91, 'length': 1}, {'begin': 94, 'length': 1}]}], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-21T15:01:26.859 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T15:01:26.860 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:26.861 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T15:01:26.861 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T15:01:26.861 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T15:01:26.861 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.861 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:97677 3.15 3:a8e72133:::rbd_data.1136bfc79a70.0000000000000005:head [write 0~4440027 [fadvise_sequential+fadvise_nocache],write 4751360~147456 [fadvise_sequential+fadvise_nocache],write 4964352~147456 [fadvise_sequential+fadvise_nocache],write 5177344~65536 [fadvise_sequential+fadvise_nocache],write 5259264~147456 [fadvise_sequential+fadvise_nocache],write 5472256~147456 [fadvise_sequential+fadvise_nocache],write 5685248~229376 [fadvise_sequential+fadvise_nocache],write 5980160~147456 [fadvise_sequential+fadvise_nocache],write 6193152~229376 [fadvise_sequential+fadvise_nocache],write 6488064~147456 [fadvise_sequential+fadvise_nocache],write 6701056~229376 [fadvise_sequential+fadvise_nocache],write 6995968~147456 [fadvise_sequential+fadvise_nocache],write 7208960~229376 [fadvise_sequential+fadvise_nocache],write 7503872~147456 [fadvise_sequential+fadvise_nocache],write 7716864~229376 [fadvise_sequential+fadvise_nocache],write 8044544~114688 [fadvise_sequential+fadvise_nocache],write 8257536~114688 [fadvise_sequential+fadvise_nocache],write 8388608~65536 [fadvise_sequential+fadvise_nocache],write 8552448~114688 [fadvise_sequential+fadvise_nocache],write 8765440~114688 [fadvise_sequential+fadvise_nocache],write 8929280~32768 [fadvise_sequential+fadvise_nocache],write 9142272~32768 [fadvise_sequential+fadvise_nocache],write 9355264~32768 [fadvise_sequential+fadvise_nocache],write 9437184~32768 [fadvise_sequential+fadvise_nocache],write 9650176~32768 [fadvise_sequential+fadvise_nocache],write 9863168~32768 [fadvise_sequential+fadvise_nocache],write 9961472~131072 [fadvise_sequential+fadvise_nocache],write 10108928~196608 [fadvise_sequential+fadvise_nocache],write 10321920~278528 [fadvise_sequential+fadvise_nocache],write 10616832~196608 [fadvise_sequential+fadvise_nocache],write 10829824~278528 [fadvise_sequential+fadvise_nocache],write 11124736~196608 [fadvise_sequential+fadvise_nocache],write 11337728~196608 [fadvise_sequential+fadvise_nocache],write 11550720~65536 [fadvise_sequential+fadvise_nocache],write 11649024~98304 [fadvise_sequential+fadvise_nocache],write 11763712~65536 [fadvise_sequential+fadvise_nocache],write 11862016~98304 [fadvise_sequential+fadvise_nocache],write 11976704~65536 [fadvise_sequential+fadvise_nocache],write 12058624~65536 [fadvise_sequential+fadvise_nocache],write 12156928~98304 [fadvise_sequential+fadvise_nocache],write 12271616~65536 [fadvise_sequential+fadvise_nocache],write 12369920~98304 [fadvise_sequential+fadvise_nocache],write 12484608~65536 [fadvise_sequential+fadvise_nocache],write 12664832~98304 [fadvise_sequential+fadvise_nocache],write 12877824~98304 [fadvise_sequential+fadvise_nocache],write 13090816~16384 [fadvise_sequential+fadvise_nocache],write 13172736~98304 [fadvise_sequential+fadvise_nocache],write 13385728~98304 [fadvise_sequential+fadvise_nocache],write 13598720~32768 [fadvise_sequential+fadvise_nocache],write 13680640~98304 [fadvise_sequential+fadvise_nocache],write 13893632~98304 [fadvise_sequential+fadvise_nocache],write 14106624~49152 [fadvise_sequential+fadvise_nocache],write 14794752~81920 [fadvise_sequential+fadvise_nocache],write 15007744~81920 [fadvise_sequential+fadvise_nocache],write 15302656~81920 [fadvise_sequential+fadvise_nocache],write 15515648~81920 [fadvise_sequential+fadvise_nocache],write 15810560~81920 [fadvise_sequential+fadvise_nocache],write 16023552~81920 [fadvise_sequential+fadvise_nocache],write 16236544~16384 [fadvise_sequential+fadvise_nocache],write 16318464~81920 [fadvise_sequential+fadvise_nocache],write 16531456~81920 [fadvise_sequential+fadvise_nocache],write 16744448~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:01:26.861 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.302946+0000", 2024-09-21T15:01:26.861 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 81.487045977999998, 2024-09-21T15:01:26.861 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.091231204999999996, 2024-09-21T15:01:26.861 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:26.861 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:26.861 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:26.861 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:26.861 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:26.862 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 97677 2024-09-21T15:01:26.862 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.862 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:26.862 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.862 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:26.862 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.302946+0000", 2024-09-21T15:01:26.862 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:26.862 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.862 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.862 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:26.862 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.302946+0000", 2024-09-21T15:01:26.862 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:26.863 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.863 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.863 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:26.863 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.302942+0000", 2024-09-21T15:01:26.863 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999962 2024-09-21T15:01:26.863 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.863 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.863 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:26.863 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311253+0000", 2024-09-21T15:01:26.863 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0083105360000000003 2024-09-21T15:01:26.863 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.863 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.863 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:26.863 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311283+0000", 2024-09-21T15:01:26.864 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0065000000000001e-05 2024-09-21T15:01:26.864 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.864 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.864 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:26.864 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311296+0000", 2024-09-21T15:01:26.864 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3206e-05 2024-09-21T15:01:26.864 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.864 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.864 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:26.864 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311404+0000", 2024-09-21T15:01:26.864 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000107514 2024-09-21T15:01:26.864 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.864 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.864 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:26.864 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311795+0000", 2024-09-21T15:01:26.865 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000390913 2024-09-21T15:01:26.865 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.865 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.865 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 6", 2024-09-21T15:01:26.865 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.314686+0000", 2024-09-21T15:01:26.865 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0028915659999999999 2024-09-21T15:01:26.865 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.865 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.865 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:01:26.865 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.368178+0000", 2024-09-21T15:01:26.865 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.053491706 2024-09-21T15:01:26.865 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.865 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.865 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:01:26.866 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.393925+0000", 2024-09-21T15:01:26.866 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025747289 2024-09-21T15:01:26.866 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.866 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.866 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:01:26.866 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.394127+0000", 2024-09-21T15:01:26.866 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000201444 2024-09-21T15:01:26.866 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.866 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.866 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:26.866 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.394177+0000", 2024-09-21T15:01:26.866 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0689000000000002e-05 2024-09-21T15:01:26.866 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:26.867 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:26.867 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:26.867 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.867 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.867 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:97673 3.1d e122/24 3:bd91946e:::rbd_data.1136bfc79a70.0000000000000002:head v 122'162, mlcod=120'160)", 2024-09-21T15:01:26.867 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.313730+0000", 2024-09-21T15:01:26.867 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 81.476261652999995, 2024-09-21T15:01:26.867 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055373462999999998, 2024-09-21T15:01:26.868 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:26.868 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:26.868 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:26.868 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.868 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:26.868 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.313730+0000", 2024-09-21T15:01:26.868 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:26.869 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.869 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.869 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:26.869 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.313730+0000", 2024-09-21T15:01:26.869 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:26.869 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.869 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.870 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:26.870 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.313730+0000", 2024-09-21T15:01:26.870 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:26.870 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.870 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.870 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:26.870 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.336113+0000", 2024-09-21T15:01:26.871 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022382962999999999 2024-09-21T15:01:26.871 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.871 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.871 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:26.871 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.336140+0000", 2024-09-21T15:01:26.871 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7248999999999999e-05 2024-09-21T15:01:26.871 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.871 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.872 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:26.872 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.336151+0000", 2024-09-21T15:01:26.872 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1015e-05 2024-09-21T15:01:26.872 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.872 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.872 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:26.873 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.355428+0000", 2024-09-21T15:01:26.873 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019276379999999999 2024-09-21T15:01:26.873 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.873 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.873 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:26.873 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.355484+0000", 2024-09-21T15:01:26.873 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6820999999999997e-05 2024-09-21T15:01:26.874 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.874 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.874 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:01:26.874 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.369040+0000", 2024-09-21T15:01:26.874 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013555466 2024-09-21T15:01:26.874 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.875 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.875 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:26.875 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.369104+0000", 2024-09-21T15:01:26.875 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3986999999999996e-05 2024-09-21T15:01:26.875 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:26.875 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:26.875 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:26.876 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.876 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.876 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139890 3.e 3:71dba096:::rbd_data.113664780c85.000000000000018b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:26.876 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.226444+0000", 2024-09-21T15:01:26.876 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 24.563548011000002, 2024-09-21T15:01:26.876 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.028209567, 2024-09-21T15:01:26.876 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:26.877 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:26.877 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:26.877 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:26.877 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:26.877 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139890 2024-09-21T15:01:26.877 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.877 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:26.878 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.878 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:26.878 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226444+0000", 2024-09-21T15:01:26.878 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:26.878 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.878 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.879 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:26.879 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226444+0000", 2024-09-21T15:01:26.879 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:26.879 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.879 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.879 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:26.881 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226443+0000", 2024-09-21T15:01:26.881 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:26.881 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.881 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.882 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:26.882 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226448+0000", 2024-09-21T15:01:26.882 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5499999999999996e-06 2024-09-21T15:01:26.882 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.882 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.882 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:26.882 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226456+0000", 2024-09-21T15:01:26.883 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8679999999999999e-06 2024-09-21T15:01:26.883 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.883 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.883 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:26.883 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226462+0000", 2024-09-21T15:01:26.883 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1800000000000001e-06 2024-09-21T15:01:26.884 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.884 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.884 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:26.884 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226545+0000", 2024-09-21T15:01:26.884 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2527000000000005e-05 2024-09-21T15:01:26.884 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.884 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.885 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:26.885 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226553+0000", 2024-09-21T15:01:26.885 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8019999999999998e-06 2024-09-21T15:01:26.885 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.885 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.885 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:26.885 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.254491+0000", 2024-09-21T15:01:26.886 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.02793785 2024-09-21T15:01:26.886 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.886 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.886 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:26.886 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.254654+0000", 2024-09-21T15:01:26.886 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00016245800000000001 2024-09-21T15:01:26.886 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:26.887 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:26.887 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:26.887 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.887 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.887 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139897 3.e 3:76ddeb1d:::rbd_data.113664780c85.0000000000000004:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:26.887 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.227198+0000", 2024-09-21T15:01:26.888 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 24.562794405999998, 2024-09-21T15:01:26.888 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.027799822, 2024-09-21T15:01:26.888 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:26.888 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:26.888 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:26.888 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:26.888 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:26.889 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139897 2024-09-21T15:01:26.889 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.889 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:26.889 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.889 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:26.889 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227198+0000", 2024-09-21T15:01:26.889 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:26.890 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.890 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.890 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:26.890 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227198+0000", 2024-09-21T15:01:26.890 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:26.890 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.891 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.891 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:26.891 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227197+0000", 2024-09-21T15:01:26.891 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:26.891 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.891 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.891 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:26.892 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227203+0000", 2024-09-21T15:01:26.892 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4819999999999999e-06 2024-09-21T15:01:26.892 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.892 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.892 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:26.892 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227220+0000", 2024-09-21T15:01:26.892 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6671e-05 2024-09-21T15:01:26.893 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.893 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.893 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:26.893 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227229+0000", 2024-09-21T15:01:26.893 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0499999999999997e-06 2024-09-21T15:01:26.893 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.894 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.894 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:26.894 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227293+0000", 2024-09-21T15:01:26.894 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4434e-05 2024-09-21T15:01:26.894 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.894 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.895 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:26.895 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227302+0000", 2024-09-21T15:01:26.895 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0929999999999992e-06 2024-09-21T15:01:26.895 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.895 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.895 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:26.896 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.254720+0000", 2024-09-21T15:01:26.896 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0274177550000001 2024-09-21T15:01:26.896 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.896 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.896 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:26.896 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.254854+0000", 2024-09-21T15:01:26.896 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000133416 2024-09-21T15:01:26.897 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.897 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.897 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:26.897 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.254998+0000", 2024-09-21T15:01:26.897 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014400099999999999 2024-09-21T15:01:26.897 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:26.897 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:26.898 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:26.898 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.898 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.898 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139902 3.e 3:76b9b799:::rbd_data.113664780c85.0000000000000021:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:26.898 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.227319+0000", 2024-09-21T15:01:26.898 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 24.562673024999999, 2024-09-21T15:01:26.898 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0279584639999999, 2024-09-21T15:01:26.898 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:26.898 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:26.899 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:26.899 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:26.899 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:26.899 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139902 2024-09-21T15:01:26.899 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.899 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:26.899 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.899 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:26.900 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227319+0000", 2024-09-21T15:01:26.900 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:26.900 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.900 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.900 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:26.900 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227319+0000", 2024-09-21T15:01:26.900 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:26.900 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.900 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.900 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:26.900 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227319+0000", 2024-09-21T15:01:26.901 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:26.901 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.901 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.901 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:26.901 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227322+0000", 2024-09-21T15:01:26.901 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1980000000000001e-06 2024-09-21T15:01:26.901 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.901 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.901 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:26.901 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227328+0000", 2024-09-21T15:01:26.901 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4230000000000002e-06 2024-09-21T15:01:26.901 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.901 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.902 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:26.902 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227333+0000", 2024-09-21T15:01:26.902 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0560000000000004e-06 2024-09-21T15:01:26.902 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.902 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.902 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:26.902 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227729+0000", 2024-09-21T15:01:26.902 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00039591799999999999 2024-09-21T15:01:26.902 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.902 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.902 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:26.902 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227738+0000", 2024-09-21T15:01:26.902 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8119999999999997e-06 2024-09-21T15:01:26.902 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.903 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.903 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:26.903 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255021+0000", 2024-09-21T15:01:26.903 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0272826779999999 2024-09-21T15:01:26.903 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.903 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.903 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:26.903 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255147+0000", 2024-09-21T15:01:26.903 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012595600000000001 2024-09-21T15:01:26.903 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.903 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.903 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:26.903 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255278+0000", 2024-09-21T15:01:26.904 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013081400000000001 2024-09-21T15:01:26.904 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:26.904 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:26.904 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:26.904 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.904 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.904 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139945 3.e 3:7756a55d:::rbd_data.113664780c85.0000000000000154:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:26.904 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.229015+0000", 2024-09-21T15:01:26.904 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 24.560977221999998, 2024-09-21T15:01:26.904 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.026468825, 2024-09-21T15:01:26.904 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:26.904 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:26.904 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:26.904 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:26.905 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:26.905 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139945 2024-09-21T15:01:26.905 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.905 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:26.905 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.905 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:26.905 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229015+0000", 2024-09-21T15:01:26.905 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:26.905 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.905 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.905 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:26.905 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229015+0000", 2024-09-21T15:01:26.905 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:26.905 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.905 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.906 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:26.906 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229014+0000", 2024-09-21T15:01:26.906 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:26.906 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.906 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.906 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:26.906 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229017+0000", 2024-09-21T15:01:26.906 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7310000000000002e-06 2024-09-21T15:01:26.906 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.906 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.906 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:26.906 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229023+0000", 2024-09-21T15:01:26.906 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8150000000000002e-06 2024-09-21T15:01:26.907 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.907 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.907 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:26.907 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229028+0000", 2024-09-21T15:01:26.907 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7650000000000001e-06 2024-09-21T15:01:26.907 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.907 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.907 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:26.907 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229082+0000", 2024-09-21T15:01:26.907 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4521999999999997e-05 2024-09-21T15:01:26.907 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.908 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.908 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:26.908 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229092+0000", 2024-09-21T15:01:26.908 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.3940000000000002e-06 2024-09-21T15:01:26.908 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.908 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.908 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:26.908 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255382+0000", 2024-09-21T15:01:26.908 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0262900930000001 2024-09-21T15:01:26.908 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.908 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.908 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:26.908 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255484+0000", 2024-09-21T15:01:26.908 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000101932 2024-09-21T15:01:26.908 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:26.909 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:26.909 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:26.909 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.909 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.909 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139967 3.e 3:74e90d85:::rbd_data.113664780c85.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:26.909 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.229124+0000", 2024-09-21T15:01:26.909 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 24.560868282000001, 2024-09-21T15:01:26.909 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.026724381, 2024-09-21T15:01:26.909 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:26.909 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:26.909 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:26.909 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:26.909 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:26.910 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139967 2024-09-21T15:01:26.910 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.910 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:26.910 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.910 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:26.910 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229124+0000", 2024-09-21T15:01:26.910 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:26.910 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.910 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.910 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:26.910 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229124+0000", 2024-09-21T15:01:26.910 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:26.910 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.910 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.911 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:26.911 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229123+0000", 2024-09-21T15:01:26.911 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:26.911 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.911 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.911 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:26.911 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229126+0000", 2024-09-21T15:01:26.911 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4779999999999998e-06 2024-09-21T15:01:26.911 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.911 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.911 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:26.911 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229131+0000", 2024-09-21T15:01:26.911 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.729e-06 2024-09-21T15:01:26.911 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.912 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.912 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:26.912 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229135+0000", 2024-09-21T15:01:26.912 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5859999999999998e-06 2024-09-21T15:01:26.912 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.912 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.912 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:26.912 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229558+0000", 2024-09-21T15:01:26.912 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00042262499999999999 2024-09-21T15:01:26.912 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.912 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.912 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:26.912 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229569+0000", 2024-09-21T15:01:26.912 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0985000000000001e-05 2024-09-21T15:01:26.913 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.913 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.913 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:26.913 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255509+0000", 2024-09-21T15:01:26.913 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.025940246 2024-09-21T15:01:26.913 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.913 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.913 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:26.913 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255662+0000", 2024-09-21T15:01:26.913 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015324199999999999 2024-09-21T15:01:26.913 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.913 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.913 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:26.913 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255848+0000", 2024-09-21T15:01:26.914 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00018583599999999999 2024-09-21T15:01:26.914 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:26.914 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:26.914 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:26.914 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.914 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.914 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140010 3.e 3:735c06f2:::rbd_data.113664780c85.0000000000000157:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:26.914 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.231531+0000", 2024-09-21T15:01:26.914 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 24.558461417, 2024-09-21T15:01:26.914 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0244773149999999, 2024-09-21T15:01:26.914 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:26.914 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:26.914 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:26.915 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:26.915 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:26.915 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140010 2024-09-21T15:01:26.915 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.915 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:26.915 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.915 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:26.915 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231531+0000", 2024-09-21T15:01:26.915 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:26.915 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.915 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.915 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:26.915 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231531+0000", 2024-09-21T15:01:26.915 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:26.915 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.916 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.916 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:26.916 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231530+0000", 2024-09-21T15:01:26.916 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:26.916 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.916 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.916 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:26.916 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231535+0000", 2024-09-21T15:01:26.916 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3970000000000002e-06 2024-09-21T15:01:26.916 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.916 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.916 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:26.916 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231544+0000", 2024-09-21T15:01:26.916 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5909999999999996e-06 2024-09-21T15:01:26.917 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.917 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.917 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:26.917 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231551+0000", 2024-09-21T15:01:26.917 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7129999999999999e-06 2024-09-21T15:01:26.917 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.917 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.917 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:26.917 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231632+0000", 2024-09-21T15:01:26.917 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1181999999999996e-05 2024-09-21T15:01:26.917 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.917 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.917 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:26.917 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231640+0000", 2024-09-21T15:01:26.918 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.551e-06 2024-09-21T15:01:26.918 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.918 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.918 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:26.918 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255884+0000", 2024-09-21T15:01:26.918 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.024244057 2024-09-21T15:01:26.918 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.918 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.918 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:26.918 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256008+0000", 2024-09-21T15:01:26.918 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012351699999999999 2024-09-21T15:01:26.918 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:26.918 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:26.918 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:26.919 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.919 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.919 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140043 3.e 3:74a37859:::rbd_data.113664780c85.0000000000000086:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:26.919 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.232623+0000", 2024-09-21T15:01:26.919 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 24.557368943, 2024-09-21T15:01:26.919 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.023518634, 2024-09-21T15:01:26.919 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:26.919 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:26.919 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:26.919 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:26.919 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:26.919 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140043 2024-09-21T15:01:26.919 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.919 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:26.920 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.920 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:26.920 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.232623+0000", 2024-09-21T15:01:26.920 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:26.920 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.920 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.920 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:26.920 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.232623+0000", 2024-09-21T15:01:26.920 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:26.920 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.920 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.920 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:26.920 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.232623+0000", 2024-09-21T15:01:26.920 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:26.920 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.921 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.921 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:26.921 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.232626+0000", 2024-09-21T15:01:26.921 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0019999999999998e-06 2024-09-21T15:01:26.921 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.921 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.921 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:26.921 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.232632+0000", 2024-09-21T15:01:26.921 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3260000000000001e-06 2024-09-21T15:01:26.921 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.921 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.921 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:26.921 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.232637+0000", 2024-09-21T15:01:26.921 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9899999999999997e-06 2024-09-21T15:01:26.922 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.922 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.922 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:26.922 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.232678+0000", 2024-09-21T15:01:26.922 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.0423999999999998e-05 2024-09-21T15:01:26.922 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.922 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.922 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:26.922 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.232686+0000", 2024-09-21T15:01:26.922 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1419999999999993e-06 2024-09-21T15:01:26.922 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.922 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.922 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:26.923 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256037+0000", 2024-09-21T15:01:26.923 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.023351736 2024-09-21T15:01:26.923 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.923 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.923 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:26.923 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256142+0000", 2024-09-21T15:01:26.923 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010443 2024-09-21T15:01:26.923 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:26.923 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:26.923 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:26.923 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.923 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.923 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140063 3.e 3:771cae23:::rbd_data.113664780c85.0000000000000112:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:26.923 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.233948+0000", 2024-09-21T15:01:26.924 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 24.556043876, 2024-09-21T15:01:26.924 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0223114710000001, 2024-09-21T15:01:26.924 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:26.924 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:26.924 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:26.924 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:26.924 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:26.924 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140063 2024-09-21T15:01:26.924 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.924 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:26.924 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.924 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:26.924 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233948+0000", 2024-09-21T15:01:26.924 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:26.925 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.925 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.925 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:26.925 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233948+0000", 2024-09-21T15:01:26.925 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:26.925 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.925 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.925 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:26.925 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233948+0000", 2024-09-21T15:01:26.925 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:26.925 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.925 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.925 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:26.925 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233951+0000", 2024-09-21T15:01:26.926 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3380000000000001e-06 2024-09-21T15:01:26.926 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.926 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.926 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:26.926 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233958+0000", 2024-09-21T15:01:26.926 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0990000000000001e-06 2024-09-21T15:01:26.926 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.926 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.926 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:26.926 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233964+0000", 2024-09-21T15:01:26.926 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.852e-06 2024-09-21T15:01:26.926 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.926 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.926 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:26.926 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.234056+0000", 2024-09-21T15:01:26.927 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2392000000000006e-05 2024-09-21T15:01:26.927 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.927 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.927 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:26.927 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.234066+0000", 2024-09-21T15:01:26.927 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1360000000000003e-06 2024-09-21T15:01:26.927 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.927 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.927 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:26.927 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256169+0000", 2024-09-21T15:01:26.927 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0221031949999999 2024-09-21T15:01:26.927 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.927 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.927 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:26.928 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256260+0000", 2024-09-21T15:01:26.928 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0939000000000005e-05 2024-09-21T15:01:26.928 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:26.928 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:26.928 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:26.928 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.928 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.928 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140086 3.e 3:758134b7:::rbd_data.113664780c85.00000000000001ad:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:26.928 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.235110+0000", 2024-09-21T15:01:26.928 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 24.554882177, 2024-09-21T15:01:26.928 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.021276217, 2024-09-21T15:01:26.928 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:26.928 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:26.928 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:26.928 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:26.929 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:26.929 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140086 2024-09-21T15:01:26.929 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.929 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:26.929 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.929 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:26.929 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.235110+0000", 2024-09-21T15:01:26.929 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:26.929 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.929 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.929 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:26.929 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.235110+0000", 2024-09-21T15:01:26.929 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:26.929 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.929 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.930 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:26.930 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.235109+0000", 2024-09-21T15:01:26.930 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:26.930 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.930 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.930 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:26.930 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.235114+0000", 2024-09-21T15:01:26.930 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1659999999999996e-06 2024-09-21T15:01:26.930 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.930 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.930 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:26.930 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.235122+0000", 2024-09-21T15:01:26.930 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5590000000000006e-06 2024-09-21T15:01:26.930 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.930 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.931 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:26.931 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.235128+0000", 2024-09-21T15:01:26.931 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8579999999999996e-06 2024-09-21T15:01:26.931 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.931 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.931 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:26.931 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.235169+0000", 2024-09-21T15:01:26.931 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1304000000000003e-05 2024-09-21T15:01:26.931 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.931 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.931 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:26.931 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.235178+0000", 2024-09-21T15:01:26.931 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8610000000000004e-06 2024-09-21T15:01:26.931 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.932 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.932 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:26.932 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256286+0000", 2024-09-21T15:01:26.932 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.021108162 2024-09-21T15:01:26.932 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.932 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.932 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:26.932 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256386+0000", 2024-09-21T15:01:26.932 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9871000000000007e-05 2024-09-21T15:01:26.932 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:26.932 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:26.932 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:26.932 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.932 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.933 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140168 3.e 3:76f4e81d:::rbd_data.113664780c85.000000000000006f:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:26.933 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.236992+0000", 2024-09-21T15:01:26.933 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 24.553000114, 2024-09-21T15:01:26.933 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.019513699, 2024-09-21T15:01:26.933 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:26.933 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:26.933 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:26.933 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:26.933 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:26.933 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140168 2024-09-21T15:01:26.933 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.933 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:26.933 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.933 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:26.934 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.236992+0000", 2024-09-21T15:01:26.934 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:26.934 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.934 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.934 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:26.934 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.236992+0000", 2024-09-21T15:01:26.934 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:26.934 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.934 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.934 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:26.934 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.236991+0000", 2024-09-21T15:01:26.935 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:26.935 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.935 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.935 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:26.935 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.236995+0000", 2024-09-21T15:01:26.935 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3100000000000001e-06 2024-09-21T15:01:26.935 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.935 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.935 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:26.935 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237002+0000", 2024-09-21T15:01:26.935 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9929999999999998e-06 2024-09-21T15:01:26.935 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.935 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.935 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:26.935 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237008+0000", 2024-09-21T15:01:26.936 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7559999999999996e-06 2024-09-21T15:01:26.936 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.936 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.936 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:26.936 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237050+0000", 2024-09-21T15:01:26.936 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2852999999999998e-05 2024-09-21T15:01:26.936 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.936 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.936 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:26.936 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237060+0000", 2024-09-21T15:01:26.936 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2399999999999996e-06 2024-09-21T15:01:26.936 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.936 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.936 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:26.936 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256413+0000", 2024-09-21T15:01:26.937 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0193528999999999 2024-09-21T15:01:26.937 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.937 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.937 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:26.937 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256506+0000", 2024-09-21T15:01:26.937 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.3154999999999998e-05 2024-09-21T15:01:26.937 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:26.937 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:26.937 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:26.937 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.937 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.937 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140186 3.e 3:75504345:::rbd_data.113664780c85.00000000000000ed:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:26.937 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.237050+0000", 2024-09-21T15:01:26.937 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 24.552942142999999, 2024-09-21T15:01:26.937 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0195720150000001, 2024-09-21T15:01:26.938 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:26.938 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:26.938 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:26.938 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:26.938 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:26.938 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140186 2024-09-21T15:01:26.938 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.938 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:26.938 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.938 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:26.938 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237050+0000", 2024-09-21T15:01:26.938 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:26.938 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.938 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.938 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:26.939 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237050+0000", 2024-09-21T15:01:26.939 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:26.939 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.939 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.939 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:26.939 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237049+0000", 2024-09-21T15:01:26.939 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:26.939 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.939 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.939 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:26.939 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237052+0000", 2024-09-21T15:01:26.939 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.016e-06 2024-09-21T15:01:26.939 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.939 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.939 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:26.940 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237070+0000", 2024-09-21T15:01:26.940 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7266999999999999e-05 2024-09-21T15:01:26.940 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.940 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.940 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:26.940 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237075+0000", 2024-09-21T15:01:26.940 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6910000000000002e-06 2024-09-21T15:01:26.940 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.940 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.940 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:26.940 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237232+0000", 2024-09-21T15:01:26.940 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015683599999999999 2024-09-21T15:01:26.940 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.941 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.941 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:26.941 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237237+0000", 2024-09-21T15:01:26.941 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6439999999999999e-06 2024-09-21T15:01:26.941 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.941 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.941 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:26.941 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256534+0000", 2024-09-21T15:01:26.941 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.019296736 2024-09-21T15:01:26.941 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.941 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.941 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:26.941 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256622+0000", 2024-09-21T15:01:26.941 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8386000000000004e-05 2024-09-21T15:01:26.942 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:26.942 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:26.942 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:26.942 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.942 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.942 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140197 3.e 3:71ef5b14:::rbd_data.113664780c85.0000000000000140:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:26.942 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.237251+0000", 2024-09-21T15:01:26.942 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 24.552741115, 2024-09-21T15:01:26.942 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.01968386, 2024-09-21T15:01:26.942 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:26.942 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:26.942 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:26.942 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:26.943 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:26.943 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140197 2024-09-21T15:01:26.943 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.943 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:26.943 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.943 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:26.943 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237251+0000", 2024-09-21T15:01:26.943 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:26.943 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.943 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.943 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:26.943 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237251+0000", 2024-09-21T15:01:26.943 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:26.943 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.944 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.944 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:26.944 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237250+0000", 2024-09-21T15:01:26.944 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:26.944 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.944 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.944 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:26.944 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237254+0000", 2024-09-21T15:01:26.944 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.0180000000000003e-06 2024-09-21T15:01:26.944 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.944 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.944 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:26.944 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237262+0000", 2024-09-21T15:01:26.945 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0669999999999992e-06 2024-09-21T15:01:26.945 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.945 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.945 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:26.945 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237269+0000", 2024-09-21T15:01:26.945 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6000000000000003e-06 2024-09-21T15:01:26.945 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.945 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.945 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:26.945 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237319+0000", 2024-09-21T15:01:26.945 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9932000000000002e-05 2024-09-21T15:01:26.945 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.945 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.945 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:26.946 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237326+0000", 2024-09-21T15:01:26.946 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5360000000000004e-06 2024-09-21T15:01:26.946 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.946 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.946 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:26.946 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256657+0000", 2024-09-21T15:01:26.946 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.019330756 2024-09-21T15:01:26.946 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.946 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.946 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:26.946 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256935+0000", 2024-09-21T15:01:26.946 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00027762099999999998 2024-09-21T15:01:26.946 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:26.946 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:26.947 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:26.947 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.947 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.947 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140201 3.e 3:732ea575:::rbd_data.113664780c85.000000000000015c:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:26.947 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.237281+0000", 2024-09-21T15:01:26.947 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 24.552711277, 2024-09-21T15:01:26.947 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0198284529999999, 2024-09-21T15:01:26.947 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:26.947 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:26.947 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:26.947 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:26.947 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:26.947 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140201 2024-09-21T15:01:26.947 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.948 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:26.948 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.948 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:26.948 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237281+0000", 2024-09-21T15:01:26.948 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:26.948 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.948 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.948 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:26.948 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237281+0000", 2024-09-21T15:01:26.948 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:26.948 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.948 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.948 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:26.949 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237280+0000", 2024-09-21T15:01:26.949 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:26.949 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.949 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.949 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:26.949 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237284+0000", 2024-09-21T15:01:26.949 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7400000000000002e-06 2024-09-21T15:01:26.949 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.949 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.949 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:26.949 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237292+0000", 2024-09-21T15:01:26.949 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8939999999999993e-06 2024-09-21T15:01:26.949 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.949 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.950 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:26.950 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237298+0000", 2024-09-21T15:01:26.950 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8000000000000004e-06 2024-09-21T15:01:26.950 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.950 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.950 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:26.950 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237521+0000", 2024-09-21T15:01:26.950 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00022326 2024-09-21T15:01:26.950 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.950 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.950 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:26.950 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237529+0000", 2024-09-21T15:01:26.950 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6170000000000001e-06 2024-09-21T15:01:26.950 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.951 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.951 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:26.951 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256968+0000", 2024-09-21T15:01:26.951 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.019439113 2024-09-21T15:01:26.951 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.951 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.951 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:26.951 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.257109+0000", 2024-09-21T15:01:26.951 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014148399999999999 2024-09-21T15:01:26.951 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:26.951 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:26.951 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:26.951 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.951 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.951 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140206 3.e 3:76d38029:::rbd_data.113664780c85.0000000000000179:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:26.952 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.237520+0000", 2024-09-21T15:01:26.952 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 24.552472163000001, 2024-09-21T15:01:26.952 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0210658699999999, 2024-09-21T15:01:26.952 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:26.952 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:26.952 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:26.952 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:26.952 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:26.952 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140206 2024-09-21T15:01:26.952 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.952 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:26.952 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.952 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:26.952 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237520+0000", 2024-09-21T15:01:26.953 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:26.953 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.953 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.953 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:26.953 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237520+0000", 2024-09-21T15:01:26.953 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:26.953 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.953 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.953 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:26.953 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237519+0000", 2024-09-21T15:01:26.953 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:26.953 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.953 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.953 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:26.953 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237523+0000", 2024-09-21T15:01:26.954 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.9009999999999996e-06 2024-09-21T15:01:26.954 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.954 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.954 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:26.954 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237531+0000", 2024-09-21T15:01:26.954 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7069999999999992e-06 2024-09-21T15:01:26.954 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.954 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.954 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:26.954 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237537+0000", 2024-09-21T15:01:26.954 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4559999999999997e-06 2024-09-21T15:01:26.954 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.954 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.955 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:26.955 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237583+0000", 2024-09-21T15:01:26.955 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5532999999999999e-05 2024-09-21T15:01:26.955 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.955 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.955 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:26.955 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237592+0000", 2024-09-21T15:01:26.955 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6009999999999995e-06 2024-09-21T15:01:26.955 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.955 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.955 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:26.955 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.257157+0000", 2024-09-21T15:01:26.955 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0195657279999999 2024-09-21T15:01:26.955 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.955 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.955 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:26.956 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.258586+0000", 2024-09-21T15:01:26.956 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00142847 2024-09-21T15:01:26.956 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:26.956 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:26.956 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:26.956 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.956 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.956 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140338 3.e 3:71dba096:::rbd_data.113664780c85.000000000000018b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:26.956 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.243856+0000", 2024-09-21T15:01:26.956 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 24.54613633, 2024-09-21T15:01:26.956 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0138226889999999, 2024-09-21T15:01:26.956 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:26.956 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:26.956 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:26.956 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:26.956 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:26.957 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140338 2024-09-21T15:01:26.957 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.957 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:26.957 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.957 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:26.957 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243856+0000", 2024-09-21T15:01:26.957 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:26.957 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.957 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.957 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:26.957 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243856+0000", 2024-09-21T15:01:26.957 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:26.957 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.957 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.957 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:26.958 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243854+0000", 2024-09-21T15:01:26.958 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:26.958 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.958 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.958 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:26.958 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243862+0000", 2024-09-21T15:01:26.958 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2570000000000001e-06 2024-09-21T15:01:26.958 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.958 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.958 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:26.958 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243877+0000", 2024-09-21T15:01:26.958 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.571e-05 2024-09-21T15:01:26.958 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.958 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.958 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:26.959 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243884+0000", 2024-09-21T15:01:26.959 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.1269999999999997e-06 2024-09-21T15:01:26.959 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.959 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.959 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:26.959 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243932+0000", 2024-09-21T15:01:26.959 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7655999999999999e-05 2024-09-21T15:01:26.959 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.959 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.959 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:26.959 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243942+0000", 2024-09-21T15:01:26.959 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6679999999999996e-06 2024-09-21T15:01:26.959 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.959 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.959 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:26.960 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.257568+0000", 2024-09-21T15:01:26.960 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0136267670000001 2024-09-21T15:01:26.960 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.960 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.960 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:26.960 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.257678+0000", 2024-09-21T15:01:26.960 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000110011 2024-09-21T15:01:26.960 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:26.960 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:26.960 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:26.960 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.960 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.960 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140345 3.e 3:76ddeb1d:::rbd_data.113664780c85.0000000000000004:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:26.960 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.243927+0000", 2024-09-21T15:01:26.960 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 24.546064745999999, 2024-09-21T15:01:26.960 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0140328439999999, 2024-09-21T15:01:26.961 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:26.961 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:26.961 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:26.961 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:26.961 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:26.961 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140345 2024-09-21T15:01:26.961 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.961 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:26.961 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.961 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:26.961 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243927+0000", 2024-09-21T15:01:26.961 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:26.961 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.961 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.961 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:26.961 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243927+0000", 2024-09-21T15:01:26.962 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:26.962 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.962 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.962 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:26.962 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243927+0000", 2024-09-21T15:01:26.962 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:26.962 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.962 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.962 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:26.962 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243930+0000", 2024-09-21T15:01:26.962 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1319999999999998e-06 2024-09-21T15:01:26.962 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.962 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.962 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:26.962 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243937+0000", 2024-09-21T15:01:26.963 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4330000000000001e-06 2024-09-21T15:01:26.963 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.963 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.963 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:26.963 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243942+0000", 2024-09-21T15:01:26.963 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5090000000000001e-06 2024-09-21T15:01:26.963 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.963 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.963 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:26.963 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243964+0000", 2024-09-21T15:01:26.963 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1838999999999999e-05 2024-09-21T15:01:26.963 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.963 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.963 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:26.964 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243971+0000", 2024-09-21T15:01:26.964 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.131e-06 2024-09-21T15:01:26.964 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.964 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.964 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:26.964 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.257704+0000", 2024-09-21T15:01:26.964 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0137329310000001 2024-09-21T15:01:26.964 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.964 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.964 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:26.964 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.257805+0000", 2024-09-21T15:01:26.964 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000101121 2024-09-21T15:01:26.964 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.964 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.964 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:26.965 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.257960+0000", 2024-09-21T15:01:26.965 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015518500000000001 2024-09-21T15:01:26.965 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:26.965 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:26.965 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:26.965 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.965 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.965 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140350 3.e 3:76b9b799:::rbd_data.113664780c85.0000000000000021:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:26.965 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.243989+0000", 2024-09-21T15:01:26.965 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 24.546002994999998, 2024-09-21T15:01:26.965 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0142022340000001, 2024-09-21T15:01:26.965 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:26.965 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:26.965 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:26.965 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:26.965 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:26.966 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140350 2024-09-21T15:01:26.966 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.966 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:26.966 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.966 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:26.966 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243989+0000", 2024-09-21T15:01:26.966 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:26.966 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.966 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.966 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:26.966 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243989+0000", 2024-09-21T15:01:26.966 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:26.966 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.966 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.966 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:26.967 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243989+0000", 2024-09-21T15:01:26.967 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:26.967 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.967 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.967 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:26.967 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243992+0000", 2024-09-21T15:01:26.967 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.236e-06 2024-09-21T15:01:26.967 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.967 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.967 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:26.967 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243998+0000", 2024-09-21T15:01:26.967 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2500000000000003e-06 2024-09-21T15:01:26.967 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.967 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.968 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:26.968 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.244003+0000", 2024-09-21T15:01:26.968 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7740000000000004e-06 2024-09-21T15:01:26.968 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.968 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.968 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:26.968 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.244047+0000", 2024-09-21T15:01:26.968 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3665999999999998e-05 2024-09-21T15:01:26.968 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.968 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.968 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:26.968 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.244055+0000", 2024-09-21T15:01:26.968 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0639999999999994e-06 2024-09-21T15:01:26.968 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.968 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.969 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:26.969 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.257985+0000", 2024-09-21T15:01:26.969 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0139306079999999 2024-09-21T15:01:26.969 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.969 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.969 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:26.969 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.258084+0000", 2024-09-21T15:01:26.969 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8610000000000006e-05 2024-09-21T15:01:26.969 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.969 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.969 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:26.969 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.258191+0000", 2024-09-21T15:01:26.969 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000107518 2024-09-21T15:01:26.969 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:26.969 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:26.969 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:26.970 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.970 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.970 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140394 3.e 3:7756a55d:::rbd_data.113664780c85.0000000000000154:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:26.970 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.245038+0000", 2024-09-21T15:01:26.970 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 24.544953778, 2024-09-21T15:01:26.970 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.013225182, 2024-09-21T15:01:26.970 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:26.970 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:26.970 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:26.970 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:26.970 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:26.970 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140394 2024-09-21T15:01:26.970 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.970 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:26.970 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.971 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:26.971 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.245038+0000", 2024-09-21T15:01:26.971 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:26.971 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.971 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.971 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:26.971 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.245038+0000", 2024-09-21T15:01:26.971 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:26.971 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.971 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.971 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:26.971 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.245037+0000", 2024-09-21T15:01:26.971 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:26.971 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.971 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.972 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:26.972 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.245044+0000", 2024-09-21T15:01:26.972 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3239999999999999e-06 2024-09-21T15:01:26.972 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.972 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.972 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:26.972 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.245061+0000", 2024-09-21T15:01:26.972 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7166000000000001e-05 2024-09-21T15:01:26.972 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.972 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.972 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:26.972 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.245069+0000", 2024-09-21T15:01:26.972 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0779999999999996e-06 2024-09-21T15:01:26.972 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.972 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.972 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:26.973 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.245118+0000", 2024-09-21T15:01:26.973 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8887000000000001e-05 2024-09-21T15:01:26.973 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.973 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.973 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:26.973 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.245126+0000", 2024-09-21T15:01:26.973 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0409999999999998e-06 2024-09-21T15:01:26.973 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.973 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.973 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:26.973 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.258206+0000", 2024-09-21T15:01:26.973 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.013080451 2024-09-21T15:01:26.973 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:26.973 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:26.973 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:26.973 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.258264+0000", 2024-09-21T15:01:26.974 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7294999999999999e-05 2024-09-21T15:01:26.974 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:26.974 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:26.974 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:26.974 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:26.974 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:26.974 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:26.975 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T15:01:26.976 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:01:26.852+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:01:26.977 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:01:26.972+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:01:27.053 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T15:01:27.077 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:01:27.072+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:01:27.097 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:27.097 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:01:27.097 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T15:01:27.097 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:27.108 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T15:01:27.178 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:01:27.172+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:01:27.204 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T15:01:27.220 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:27.220 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:01:27.220 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T15:01:27.220 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T15:01:27.220 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:27.226 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T15:01:27.230 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T15:01:27.249 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T15:01:27.272 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T15:01:27.279 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:01:27.276+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:01:27.295 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-21T15:01:27.321 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T15:01:27.346 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T15:01:27.362 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-21T15:01:27.365 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-09-21T15:01:27.374 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:27.374 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:27.375 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T15:01:27.377 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:01:27.377 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:01:27.377 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:01:27.377 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:01:27.377 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:01:27.378 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:01:27.378 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:01:27.378 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:01:27.380 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":291,"stamp":"2024-09-21T15:01:27.169341+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":859797958,"num_objects":1057,"num_object_clones":0,"num_object_copies":2114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1057,"num_whiteouts":0,"num_read":167608,"num_read_kb":1232714,"num_write":32401,"num_write_kb":942847,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2882,"num_bytes_recovered":203475064,"num_keys_recovered":305,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20482,"ondisk_log_size":20482,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":81,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":460684,"kb_used_data":69768,"kb_used_omap":10,"kb_used_meta":390773,"kb_avail":660142196,"statfs":{"total":676457349120,"available":675985608704,"internally_reserved":0,"allocated":71442432,"data_stored":501823609,"data_compressed":11072673,"data_compressed_allocated":44191744,"data_compressed_original":487776902,"omap_allocated":11150,"internal_metadata":400151666},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-32552054,"num_objects":-69,"num_object_clones":0,"num_object_copies":-138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-69,"num_whiteouts":0,"num_read":370,"num_read_kb":321,"num_write":131,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":962,"num_bytes_recovered":58120440,"num_keys_recovered":76,"num_objects_omap":-2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.833385"},"pg_stats":[{"pgid":"3.1f","version":"191'524","reported_seq":3581,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-21T15:01:25.171061+0000","last_change":"2024-09-21T15:01:24.372647+0000","last_active":"2024-09-21T15:01:25.171061+0000","last_peered":"2024-09-21T15:01:25.171061+0000","last_clean":"2024-09-21T15:01:25.171061+0000","last_became_active":"2024-09-21T14:58:19.410055+0000","last_became_peered":"2024-09-21T14:58:19.410055+0000","last_unstale":"2024-09-21T15:01:25.171061+0000","last_undegraded":"2024-09-21T15:01:25.171061+0000","last_fullsized":"2024-09-21T15:01:25.171061+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":524,"log_dups_size":0,"ondisk_log_size":524,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:49:13.944219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013586609999999999,"stat_sum":{"num_bytes":31189406,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":3407,"num_read_kb":42916,"num_write":912,"num_write_kb":36424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"5b","length":"1"},{"start":"5e","length":"1"}]},{"pgid":"3.1e","version":"191'430","reported_seq":3170,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-21T15:01:25.171220+0000","last_change":"2024-09-21T15:01:24.372967+0000","last_active":"2024-09-21T15:01:25.171220+0000","last_peered":"2024-09-21T15:01:25.171220+0000","last_clean":"2024-09-21T15:01:25.171220+0000","last_became_active":"2024-09-21T15:01:06.168726+0000","last_became_peered":"2024-09-21T15:01:06.168726+0000","last_unstale":"2024-09-21T15:01:25.171220+0000","last_undegraded":"2024-09-21T15:01:25.171220+0000","last_fullsized":"2024-09-21T15:01:25.171220+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":430,"log_dups_size":0,"ondisk_log_size":430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:29:01.079772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001720584,"stat_sum":{"num_bytes":17990538,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":3179,"num_read_kb":23206,"num_write":569,"num_write_kb":18429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":148,"num_bytes_recovered":11635695,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"5b","length":"1"},{"start":"5e","length":"1"}]},{"pgid":"3.1d","version":"185'314","reported_seq":4580,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-21T15:01:25.168875+0000","last_change":"2024-09-21T15:01:24.369819+0000","last_active":"2024-09-21T15:01:25.168875+0000","last_peered":"2024-09-21T15:01:25.168875+0000","last_clean":"2024-09-21T15:01:25.168875+0000","last_became_active":"2024-09-21T14:58:19.411387+0000","last_became_peered":"2024-09-21T14:58:19.411387+0000","last_unstale":"2024-09-21T15:01:25.168875+0000","last_undegraded":"2024-09-21T15:01:25.168875+0000","last_fullsized":"2024-09-21T15:01:25.168875+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":314,"log_dups_size":0,"ondisk_log_size":314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:37:57.597069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001333159,"stat_sum":{"num_bytes":13166729,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":3876,"num_read_kb":28287,"num_write":509,"num_write_kb":26606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"5b","length":"1"},{"start":"5e","length":"1"}]},{"pgid":"3.1c","version":"193'495","reported_seq":3886,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-21T15:01:25.256332+0000","last_change":"2024-09-21T15:01:24.366633+0000","last_active":"2024-09-21T15:01:25.256332+0000","last_peered":"2024-09-21T15:01:25.256332+0000","last_clean":"2024-09-21T15:01:25.256332+0000","last_became_active":"2024-09-21T14:58:19.411306+0000","last_became_peered":"2024-09-21T14:58:19.411306+0000","last_unstale":"2024-09-21T15:01:25.256332+0000","last_undegraded":"2024-09-21T15:01:25.256332+0000","last_fullsized":"2024-09-21T15:01:25.256332+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":495,"log_dups_size":0,"ondisk_log_size":495,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:20:27.079795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001749682,"stat_sum":{"num_bytes":12624495,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":3727,"num_read_kb":24378,"num_write":762,"num_write_kb":25222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"5b","length":"1"},{"start":"5e","length":"1"}]},{"pgid":"3.1b","version":"193'480","reported_seq":3970,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-21T15:01:25.179388+0000","last_change":"2024-09-21T15:01:25.179213+0000","last_active":"2024-09-21T15:01:25.179388+0000","last_peered":"2024-09-21T15:01:25.179388+0000","last_clean":"2024-09-21T15:01:25.179388+0000","last_became_active":"2024-09-21T15:01:25.176460+0000","last_became_peered":"2024-09-21T15:01:25.176460+0000","last_unstale":"2024-09-21T15:01:25.179388+0000","last_undegraded":"2024-09-21T15:01:25.179388+0000","last_fullsized":"2024-09-21T15:01:25.179388+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":480,"log_dups_size":0,"ondisk_log_size":480,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:57.431201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084393900000000004,"stat_sum":{"num_bytes":17949492,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":3466,"num_read_kb":32522,"num_write":654,"num_write_kb":22571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":248,"num_bytes_recovered":13616631,"num_keys_recovered":33,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"5b","length":"1"},{"start":"5e","length":"1"}]},{"pgid":"3.1a","version":"191'424","reported_seq":4360,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-21T15:01:25.228350+0000","last_change":"2024-09-21T15:01:24.366098+0000","last_active":"2024-09-21T15:01:25.228350+0000","last_peered":"2024-09-21T15:01:25.228350+0000","last_clean":"2024-09-21T15:01:25.228350+0000","last_became_active":"2024-09-21T14:58:19.409415+0000","last_became_peered":"2024-09-21T14:58:19.409415+0000","last_unstale":"2024-09-21T15:01:25.228350+0000","last_undegraded":"2024-09-21T15:01:25.228350+0000","last_fullsized":"2024-09-21T15:01:25.228350+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":424,"log_dups_size":0,"ondisk_log_size":424,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:28:46.336301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001447285,"stat_sum":{"num_bytes":18340703,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":4008,"num_read_kb":27999,"num_write":644,"num_write_kb":23865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"5b","length":"1"},{"start":"5e","length":"1"}]},{"pgid":"3.19","version":"193'744","reported_seq":8313,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-21T15:01:25.182644+0000","last_change":"2024-09-21T15:01:25.182531+0000","last_active":"2024-09-21T15:01:25.182644+0000","last_peered":"2024-09-21T15:01:25.182644+0000","last_clean":"2024-09-21T15:01:25.182644+0000","last_became_active":"2024-09-21T15:01:25.179729+0000","last_became_peered":"2024-09-21T15:01:25.179729+0000","last_unstale":"2024-09-21T15:01:25.182644+0000","last_undegraded":"2024-09-21T15:01:25.182644+0000","last_fullsized":"2024-09-21T15:01:25.182644+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":744,"log_dups_size":0,"ondisk_log_size":744,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:15:24.357933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098270400000000004,"stat_sum":{"num_bytes":34682398,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":9024,"num_read_kb":67751,"num_write":1181,"num_write_kb":29909,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":261,"num_bytes_recovered":23509073,"num_keys_recovered":66,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"5b","length":"1"},{"start":"5e","length":"1"}]},{"pgid":"3.18","version":"191'596","reported_seq":8368,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-21T15:01:25.171267+0000","last_change":"2024-09-21T15:01:24.371840+0000","last_active":"2024-09-21T15:01:25.171267+0000","last_peered":"2024-09-21T15:01:25.171267+0000","last_clean":"2024-09-21T15:01:25.171267+0000","last_became_active":"2024-09-21T14:58:19.409371+0000","last_became_peered":"2024-09-21T14:58:19.409371+0000","last_unstale":"2024-09-21T15:01:25.171267+0000","last_undegraded":"2024-09-21T15:01:25.171267+0000","last_fullsized":"2024-09-21T15:01:25.171267+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":596,"log_dups_size":0,"ondisk_log_size":596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:35:27.143445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044436699999999998,"stat_sum":{"num_bytes":31290330,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":0,"num_read":8852,"num_read_kb":44937,"num_write":914,"num_write_kb":30599,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"5b","length":"1"},{"start":"5e","length":"1"}]},{"pgid":"3.17","version":"193'393","reported_seq":3802,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-21T15:01:25.171498+0000","last_change":"2024-09-21T15:01:24.373022+0000","last_active":"2024-09-21T15:01:25.171498+0000","last_peered":"2024-09-21T15:01:25.171498+0000","last_clean":"2024-09-21T15:01:25.171498+0000","last_became_active":"2024-09-21T14:58:19.410306+0000","last_became_peered":"2024-09-21T14:58:19.410306+0000","last_unstale":"2024-09-21T15:01:25.171498+0000","last_undegraded":"2024-09-21T15:01:25.171498+0000","last_fullsized":"2024-09-21T15:01:25.171498+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":393,"log_dups_size":0,"ondisk_log_size":393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:33:50.089165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001495323,"stat_sum":{"num_bytes":21853084,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":3380,"num_read_kb":28207,"num_write":750,"num_write_kb":34633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"5b","length":"1"},{"start":"5e","length":"1"}]},{"pgid":"3.16","version":"193'965","reported_seq":5758,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-21T15:01:25.305767+0000","last_change":"2024-09-21T15:01:24.372771+0000","last_active":"2024-09-21T15:01:25.305767+0000","last_peered":"2024-09-21T15:01:25.305767+0000","last_clean":"2024-09-21T15:01:25.305767+0000","last_became_active":"2024-09-21T15:01:06.166756+0000","last_became_peered":"2024-09-21T15:01:06.166756+0000","last_unstale":"2024-09-21T15:01:25.305767+0000","last_undegraded":"2024-09-21T15:01:25.305767+0000","last_fullsized":"2024-09-21T15:01:25.305767+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":965,"log_dups_size":0,"ondisk_log_size":965,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:19:04.988784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085890300000000005,"stat_sum":{"num_bytes":30085978,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":7696,"num_read_kb":43673,"num_write":1541,"num_write_kb":28357,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":271,"num_bytes_recovered":16821266,"num_keys_recovered":55,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"5b","length":"1"},{"start":"5e","length":"1"}]},{"pgid":"3.15","version":"191'763","reported_seq":3727,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-21T15:01:25.171548+0000","last_change":"2024-09-21T15:01:24.370231+0000","last_active":"2024-09-21T15:01:25.171548+0000","last_peered":"2024-09-21T15:01:25.171548+0000","last_clean":"2024-09-21T15:01:25.171548+0000","last_became_active":"2024-09-21T14:58:19.410687+0000","last_became_peered":"2024-09-21T14:58:19.410687+0000","last_unstale":"2024-09-21T15:01:25.171548+0000","last_undegraded":"2024-09-21T15:01:25.171548+0000","last_fullsized":"2024-09-21T15:01:25.171548+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":763,"log_dups_size":0,"ondisk_log_size":763,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:21:42.226982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020408679999999999,"stat_sum":{"num_bytes":33396606,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":4691,"num_read_kb":49720,"num_write":1284,"num_write_kb":33329,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"5b","length":"1"},{"start":"5e","length":"1"}]},{"pgid":"3.14","version":"193'497","reported_seq":3974,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-21T15:01:25.297662+0000","last_change":"2024-09-21T15:01:24.369509+0000","last_active":"2024-09-21T15:01:25.297662+0000","last_peered":"2024-09-21T15:01:25.297662+0000","last_clean":"2024-09-21T15:01:25.297662+0000","last_became_active":"2024-09-21T14:58:19.407310+0000","last_became_peered":"2024-09-21T14:58:19.407310+0000","last_unstale":"2024-09-21T15:01:25.297662+0000","last_undegraded":"2024-09-21T15:01:25.297662+0000","last_fullsized":"2024-09-21T15:01:25.297662+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":497,"log_dups_size":0,"ondisk_log_size":497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:47:46.079115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096169599999999995,"stat_sum":{"num_bytes":27524176,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":3720,"num_read_kb":43296,"num_write":681,"num_write_kb":33466,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"5b","length":"1"},{"start":"5e","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":187,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-09-21T15:01:24.371756+0000","last_change":"2024-09-21T14:58:11.348177+0000","last_active":"2024-09-21T15:01:24.371756+0000","last_peered":"2024-09-21T15:01:24.371756+0000","last_clean":"2024-09-21T15:01:24.371756+0000","last_became_active":"2024-09-21T14:58:09.328417+0000","last_became_peered":"2024-09-21T14:58:09.328417+0000","last_unstale":"2024-09-21T15:01:24.371756+0000","last_undegraded":"2024-09-21T15:01:24.371756+0000","last_fullsized":"2024-09-21T15:01:24.371756+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:52:22.671185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018917490000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"193'508","reported_seq":6214,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T15:01:26.172111+0000","last_change":"2024-09-21T15:01:25.176001+0000","last_active":"2024-09-21T15:01:26.172111+0000","last_peered":"2024-09-21T15:01:26.172111+0000","last_clean":"2024-09-21T15:01:26.172111+0000","last_became_active":"2024-09-21T15:01:25.173568+0000","last_became_peered":"2024-09-21T15:01:25.173568+0000","last_unstale":"2024-09-21T15:01:26.172111+0000","last_undegraded":"2024-09-21T15:01:26.172111+0000","last_fullsized":"2024-09-21T15:01:26.172111+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":508,"log_dups_size":0,"ondisk_log_size":508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:53:08.599143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082603600000000004,"stat_sum":{"num_bytes":46283728,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":6107,"num_read_kb":52873,"num_write":628,"num_write_kb":27950,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":202,"num_bytes_recovered":11818795,"num_keys_recovered":18,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"5b","length":"1"},{"start":"5e","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":34,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-21T15:01:25.171457+0000","last_change":"2024-09-21T15:01:22.344040+0000","last_active":"2024-09-21T15:01:25.171457+0000","last_peered":"2024-09-21T15:01:25.171457+0000","last_clean":"2024-09-21T15:01:25.171457+0000","last_became_active":"2024-09-21T15:01:22.343688+0000","last_became_peered":"2024-09-21T15:01:22.343688+0000","last_unstale":"2024-09-21T15:01:25.171457+0000","last_undegraded":"2024-09-21T15:01:25.171457+0000","last_fullsized":"2024-09-21T15:01:25.171457+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:11:18.042935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"188'862","reported_seq":3817,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-21T15:01:25.175886+0000","last_change":"2024-09-21T15:01:25.175707+0000","last_active":"2024-09-21T15:01:25.175886+0000","last_peered":"2024-09-21T15:01:25.175886+0000","last_clean":"2024-09-21T15:01:25.175886+0000","last_became_active":"2024-09-21T15:01:25.173798+0000","last_became_peered":"2024-09-21T15:01:25.173798+0000","last_unstale":"2024-09-21T15:01:25.175886+0000","last_undegraded":"2024-09-21T15:01:25.175886+0000","last_fullsized":"2024-09-21T15:01:25.175886+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:03:31.905691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000688756,"stat_sum":{"num_bytes":26157686,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":5002,"num_read_kb":30818,"num_write":1288,"num_write_kb":25254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":260,"num_bytes_recovered":12566413,"num_keys_recovered":33,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"5b","length":"1"},{"start":"5e","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":211,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-09-21T15:01:24.369286+0000","last_change":"2024-09-21T15:01:22.346021+0000","last_active":"2024-09-21T15:01:24.369286+0000","last_peered":"2024-09-21T15:01:24.369286+0000","last_clean":"2024-09-21T15:01:24.369286+0000","last_became_active":"2024-09-21T15:01:22.345643+0000","last_became_peered":"2024-09-21T15:01:22.345643+0000","last_unstale":"2024-09-21T15:01:24.369286+0000","last_undegraded":"2024-09-21T15:01:24.369286+0000","last_fullsized":"2024-09-21T15:01:24.369286+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:55:55.681542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019338770000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"191'620","reported_seq":5333,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-21T15:01:25.175857+0000","last_change":"2024-09-21T15:01:25.175213+0000","last_active":"2024-09-21T15:01:25.175857+0000","last_peered":"2024-09-21T15:01:25.175857+0000","last_clean":"2024-09-21T15:01:25.175857+0000","last_became_active":"2024-09-21T15:01:25.172697+0000","last_became_peered":"2024-09-21T15:01:25.172697+0000","last_unstale":"2024-09-21T15:01:25.175857+0000","last_undegraded":"2024-09-21T15:01:25.175857+0000","last_fullsized":"2024-09-21T15:01:25.175857+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":620,"log_dups_size":0,"ondisk_log_size":620,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:34:57.331662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076528200000000005,"stat_sum":{"num_bytes":37292257,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":5112,"num_read_kb":38409,"num_write":1074,"num_write_kb":41296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":203,"num_bytes_recovered":25796920,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"5b","length":"1"},{"start":"5e","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":188,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-09-21T15:01:24.373872+0000","last_change":"2024-09-21T14:58:11.406176+0000","last_active":"2024-09-21T15:01:24.373872+0000","last_peered":"2024-09-21T15:01:24.373872+0000","last_clean":"2024-09-21T15:01:24.373872+0000","last_became_active":"2024-09-21T14:58:09.777465+0000","last_became_peered":"2024-09-21T14:58:09.777465+0000","last_unstale":"2024-09-21T15:01:24.373872+0000","last_undegraded":"2024-09-21T15:01:24.373872+0000","last_fullsized":"2024-09-21T15:01:24.373872+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:55:04.667816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095704000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"188'690","reported_seq":2624,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-21T15:01:25.171488+0000","last_change":"2024-09-21T15:01:24.372554+0000","last_active":"2024-09-21T15:01:25.171488+0000","last_peered":"2024-09-21T15:01:25.171488+0000","last_clean":"2024-09-21T15:01:25.171488+0000","last_became_active":"2024-09-21T14:58:19.408971+0000","last_became_peered":"2024-09-21T14:58:19.408971+0000","last_unstale":"2024-09-21T15:01:25.171488+0000","last_undegraded":"2024-09-21T15:01:25.171488+0000","last_fullsized":"2024-09-21T15:01:25.171488+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":690,"log_dups_size":0,"ondisk_log_size":690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:29:52.642121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076141900000000005,"stat_sum":{"num_bytes":12010717,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":3562,"num_read_kb":20628,"num_write":1187,"num_write_kb":20674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"5b","length":"1"},{"start":"5e","length":"1"}]},{"pgid":"3.3","version":"193'861","reported_seq":3243,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-21T15:01:25.319565+0000","last_change":"2024-09-21T15:01:24.369847+0000","last_active":"2024-09-21T15:01:25.319565+0000","last_peered":"2024-09-21T15:01:25.319565+0000","last_clean":"2024-09-21T15:01:25.319565+0000","last_became_active":"2024-09-21T14:58:19.417453+0000","last_became_peered":"2024-09-21T14:58:19.417453+0000","last_unstale":"2024-09-21T15:01:25.319565+0000","last_undegraded":"2024-09-21T15:01:25.319565+0000","last_fullsized":"2024-09-21T15:01:25.319565+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":861,"log_dups_size":0,"ondisk_log_size":861,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:19:27.087272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001663655,"stat_sum":{"num_bytes":13295449,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":4671,"num_read_kb":25110,"num_write":1418,"num_write_kb":21648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"5b","length":"1"},{"start":"5e","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":189,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-09-21T15:01:24.367813+0000","last_change":"2024-09-21T14:58:11.347504+0000","last_active":"2024-09-21T15:01:24.367813+0000","last_peered":"2024-09-21T15:01:24.367813+0000","last_clean":"2024-09-21T15:01:24.367813+0000","last_became_active":"2024-09-21T14:58:09.775981+0000","last_became_peered":"2024-09-21T14:58:09.775981+0000","last_unstale":"2024-09-21T15:01:24.367813+0000","last_undegraded":"2024-09-21T15:01:24.367813+0000","last_fullsized":"2024-09-21T15:01:24.367813+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:47:16.286652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012568880000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"191'1013","reported_seq":3600,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-21T15:01:25.170926+0000","last_change":"2024-09-21T15:01:24.373224+0000","last_active":"2024-09-21T15:01:25.170926+0000","last_peered":"2024-09-21T15:01:25.170926+0000","last_clean":"2024-09-21T15:01:25.170926+0000","last_became_active":"2024-09-21T14:58:19.405811+0000","last_became_peered":"2024-09-21T14:58:19.405811+0000","last_unstale":"2024-09-21T15:01:25.170926+0000","last_undegraded":"2024-09-21T15:01:25.170926+0000","last_fullsized":"2024-09-21T15:01:25.170926+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":1013,"log_dups_size":0,"ondisk_log_size":1013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:36:25.551607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000468775,"stat_sum":{"num_bytes":19237016,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":5987,"num_read_kb":28491,"num_write":1609,"num_write_kb":26962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"5b","length":"1"},{"start":"5e","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":189,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-21T15:01:25.168175+0000","last_change":"2024-09-21T14:58:11.342210+0000","last_active":"2024-09-21T15:01:25.168175+0000","last_peered":"2024-09-21T15:01:25.168175+0000","last_clean":"2024-09-21T15:01:25.168175+0000","last_became_active":"2024-09-21T14:58:09.330624+0000","last_became_peered":"2024-09-21T14:58:09.330624+0000","last_unstale":"2024-09-21T15:01:25.168175+0000","last_undegraded":"2024-09-21T15:01:25.168175+0000","last_fullsized":"2024-09-21T15:01:25.168175+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:29:43.229538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001051151,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"193'311","reported_seq":2100,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-21T15:01:25.296078+0000","last_change":"2024-09-21T15:01:24.367809+0000","last_active":"2024-09-21T15:01:25.296078+0000","last_peered":"2024-09-21T15:01:25.296078+0000","last_clean":"2024-09-21T15:01:25.296078+0000","last_became_active":"2024-09-21T15:01:05.164862+0000","last_became_peered":"2024-09-21T15:01:05.164862+0000","last_unstale":"2024-09-21T15:01:25.296078+0000","last_undegraded":"2024-09-21T15:01:25.296078+0000","last_fullsized":"2024-09-21T15:01:25.296078+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":311,"log_dups_size":0,"ondisk_log_size":311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:57:46.675335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001677394,"stat_sum":{"num_bytes":29238822,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":2369,"num_read_kb":40192,"num_write":501,"num_write_kb":39390,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":254,"num_bytes_recovered":20571253,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"5b","length":"1"},{"start":"5e","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":187,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-09-21T15:01:24.367517+0000","last_change":"2024-09-21T14:58:11.346766+0000","last_active":"2024-09-21T15:01:24.367517+0000","last_peered":"2024-09-21T15:01:24.367517+0000","last_clean":"2024-09-21T15:01:24.367517+0000","last_became_active":"2024-09-21T14:58:09.329748+0000","last_became_peered":"2024-09-21T14:58:09.329748+0000","last_unstale":"2024-09-21T15:01:24.367517+0000","last_undegraded":"2024-09-21T15:01:24.367517+0000","last_fullsized":"2024-09-21T15:01:24.367517+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:31:20.267675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052698000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"191'1197","reported_seq":7971,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-21T15:01:25.171119+0000","last_change":"2024-09-21T15:01:24.372275+0000","last_active":"2024-09-21T15:01:25.171119+0000","last_peered":"2024-09-21T15:01:25.171119+0000","last_clean":"2024-09-21T15:01:25.171119+0000","last_became_active":"2024-09-21T14:58:41.401003+0000","last_became_peered":"2024-09-21T14:58:41.401003+0000","last_unstale":"2024-09-21T15:01:25.171119+0000","last_undegraded":"2024-09-21T15:01:25.171119+0000","last_fullsized":"2024-09-21T15:01:25.171119+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":1197,"log_dups_size":0,"ondisk_log_size":1197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:05:37.644835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054559699999999999,"stat_sum":{"num_bytes":40989545,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":11356,"num_read_kb":50512,"num_write":1888,"num_write_kb":24901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1403348,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"5b","length":"1"},{"start":"5e","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":188,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-09-21T15:01:24.368388+0000","last_change":"2024-09-21T14:58:11.347700+0000","last_active":"2024-09-21T15:01:24.368388+0000","last_peered":"2024-09-21T15:01:24.368388+0000","last_clean":"2024-09-21T15:01:24.368388+0000","last_became_active":"2024-09-21T14:58:09.438842+0000","last_became_peered":"2024-09-21T14:58:09.438842+0000","last_unstale":"2024-09-21T15:01:24.368388+0000","last_undegraded":"2024-09-21T15:01:24.368388+0000","last_fullsized":"2024-09-21T15:01:24.368388+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:41:34.626249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001426194,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":336,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-09-21T15:01:24.365102+0000","last_change":"2024-09-21T14:58:03.616055+0000","last_active":"2024-09-21T15:01:24.365102+0000","last_peered":"2024-09-21T15:01:24.365102+0000","last_clean":"2024-09-21T15:01:24.365102+0000","last_became_active":"2024-09-21T14:58:03.613214+0000","last_became_peered":"2024-09-21T14:58:03.613214+0000","last_unstale":"2024-09-21T15:01:24.365102+0000","last_undegraded":"2024-09-21T15:01:24.365102+0000","last_fullsized":"2024-09-21T15:01:24.365102+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:02.180162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:02.180162+0000","last_clean_scrub_stamp":"2024-09-21T14:58:02.180162+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:29:05.860361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"194'642","reported_seq":4458,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T15:01:26.225637+0000","last_change":"2024-09-21T15:01:26.178574+0000","last_active":"2024-09-21T15:01:26.225637+0000","last_peered":"2024-09-21T15:01:26.225637+0000","last_clean":"2024-09-21T15:01:26.225637+0000","last_became_active":"2024-09-21T15:01:26.174222+0000","last_became_peered":"2024-09-21T15:01:26.174222+0000","last_unstale":"2024-09-21T15:01:26.225637+0000","last_undegraded":"2024-09-21T15:01:26.225637+0000","last_fullsized":"2024-09-21T15:01:26.225637+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":642,"log_dups_size":0,"ondisk_log_size":642,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:22:55.525805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014425460000000001,"stat_sum":{"num_bytes":31944196,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":4370,"num_read_kb":44512,"num_write":1119,"num_write_kb":30497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":13253584,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"5b","length":"1"},{"start":"5e","length":"1"}]},{"pgid":"3.9","version":"193'913","reported_seq":3562,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-21T15:01:25.311107+0000","last_change":"2024-09-21T15:01:25.181352+0000","last_active":"2024-09-21T15:01:25.311107+0000","last_peered":"2024-09-21T15:01:25.311107+0000","last_clean":"2024-09-21T15:01:25.311107+0000","last_became_active":"2024-09-21T15:01:25.176267+0000","last_became_peered":"2024-09-21T15:01:25.176267+0000","last_unstale":"2024-09-21T15:01:25.311107+0000","last_undegraded":"2024-09-21T15:01:25.311107+0000","last_fullsized":"2024-09-21T15:01:25.311107+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":913,"log_dups_size":0,"ondisk_log_size":913,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:21:28.802799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00331964,"stat_sum":{"num_bytes":18356978,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":4719,"num_read_kb":35627,"num_write":1529,"num_write_kb":33787,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":233,"num_bytes_recovered":14742839,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"5b","length":"1"},{"start":"5e","length":"1"}]},{"pgid":"3.a","version":"191'909","reported_seq":4509,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-21T15:01:25.173456+0000","last_change":"2024-09-21T15:01:24.365869+0000","last_active":"2024-09-21T15:01:25.173456+0000","last_peered":"2024-09-21T15:01:25.173456+0000","last_clean":"2024-09-21T15:01:25.173456+0000","last_became_active":"2024-09-21T15:00:27.280971+0000","last_became_peered":"2024-09-21T15:00:27.280971+0000","last_unstale":"2024-09-21T15:01:25.173456+0000","last_undegraded":"2024-09-21T15:01:25.173456+0000","last_fullsized":"2024-09-21T15:01:25.173456+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":909,"log_dups_size":0,"ondisk_log_size":909,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:26:07.617059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014355100000000001,"stat_sum":{"num_bytes":18890884,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":6786,"num_read_kb":30662,"num_write":1320,"num_write_kb":33137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":9228123,"num_keys_recovered":33,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"5b","length":"1"},{"start":"5e","length":"1"}]},{"pgid":"3.b","version":"188'513","reported_seq":3106,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-21T15:01:25.168188+0000","last_change":"2024-09-21T15:01:24.368798+0000","last_active":"2024-09-21T15:01:25.168188+0000","last_peered":"2024-09-21T15:01:25.168188+0000","last_clean":"2024-09-21T15:01:25.168188+0000","last_became_active":"2024-09-21T14:58:19.410679+0000","last_became_peered":"2024-09-21T14:58:19.410679+0000","last_unstale":"2024-09-21T15:01:25.168188+0000","last_undegraded":"2024-09-21T15:01:25.168188+0000","last_fullsized":"2024-09-21T15:01:25.168188+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":513,"log_dups_size":0,"ondisk_log_size":513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:07:40.758932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00166442,"stat_sum":{"num_bytes":26807287,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":2899,"num_read_kb":36584,"num_write":814,"num_write_kb":39638,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"5b","length":"1"},{"start":"5e","length":"1"}]},{"pgid":"3.c","version":"191'437","reported_seq":4399,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-21T15:01:25.172158+0000","last_change":"2024-09-21T15:01:24.370284+0000","last_active":"2024-09-21T15:01:25.172158+0000","last_peered":"2024-09-21T15:01:25.172158+0000","last_clean":"2024-09-21T15:01:25.172158+0000","last_became_active":"2024-09-21T14:59:25.130057+0000","last_became_peered":"2024-09-21T14:59:25.130057+0000","last_unstale":"2024-09-21T15:01:25.172158+0000","last_undegraded":"2024-09-21T15:01:25.172158+0000","last_fullsized":"2024-09-21T15:01:25.172158+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":437,"log_dups_size":0,"ondisk_log_size":437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:53:33.752550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016162819999999999,"stat_sum":{"num_bytes":15284342,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":4340,"num_read_kb":28635,"num_write":679,"num_write_kb":24610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":9059600,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"5b","length":"1"},{"start":"5e","length":"1"}]},{"pgid":"3.d","version":"191'490","reported_seq":5045,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-21T15:01:25.169338+0000","last_change":"2024-09-21T15:01:24.371113+0000","last_active":"2024-09-21T15:01:25.169338+0000","last_peered":"2024-09-21T15:01:25.169338+0000","last_clean":"2024-09-21T15:01:25.169338+0000","last_became_active":"2024-09-21T14:58:19.407884+0000","last_became_peered":"2024-09-21T14:58:19.407884+0000","last_unstale":"2024-09-21T15:01:25.169338+0000","last_undegraded":"2024-09-21T15:01:25.169338+0000","last_fullsized":"2024-09-21T15:01:25.169338+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":490,"log_dups_size":0,"ondisk_log_size":490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:28:57.111615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021831239999999998,"stat_sum":{"num_bytes":44883751,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":4793,"num_read_kb":55676,"num_write":896,"num_write_kb":38644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"5b","length":"1"},{"start":"5e","length":"1"}]},{"pgid":"3.e","version":"191'817","reported_seq":2957,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-21T15:01:25.171261+0000","last_change":"2024-09-21T15:01:24.372561+0000","last_active":"2024-09-21T15:01:25.171261+0000","last_peered":"2024-09-21T15:01:25.171261+0000","last_clean":"2024-09-21T15:01:25.171261+0000","last_became_active":"2024-09-21T15:01:05.172427+0000","last_became_peered":"2024-09-21T15:01:05.172427+0000","last_unstale":"2024-09-21T15:01:25.171261+0000","last_undegraded":"2024-09-21T15:01:25.171261+0000","last_fullsized":"2024-09-21T15:01:25.171261+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":817,"log_dups_size":0,"ondisk_log_size":817,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:28:18.710621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000688818,"stat_sum":{"num_bytes":10512224,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":4162,"num_read_kb":17213,"num_write":1442,"num_write_kb":17955,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":138,"num_bytes_recovered":7486496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"5b","length":"1"},{"start":"5e","length":"1"}]},{"pgid":"3.f","version":"191'479","reported_seq":5964,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-21T15:01:25.171831+0000","last_change":"2024-09-21T15:01:24.369556+0000","last_active":"2024-09-21T15:01:25.171831+0000","last_peered":"2024-09-21T15:01:25.171831+0000","last_clean":"2024-09-21T15:01:25.171831+0000","last_became_active":"2024-09-21T14:58:19.410406+0000","last_became_peered":"2024-09-21T14:58:19.410406+0000","last_unstale":"2024-09-21T15:01:25.171831+0000","last_undegraded":"2024-09-21T15:01:25.171831+0000","last_fullsized":"2024-09-21T15:01:25.171831+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":479,"log_dups_size":0,"ondisk_log_size":479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:59:16.026939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015815639999999999,"stat_sum":{"num_bytes":40642936,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":5766,"num_read_kb":56245,"num_write":763,"num_write_kb":36768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"5b","length":"1"},{"start":"5e","length":"1"}]},{"pgid":"3.10","version":"193'479","reported_seq":4416,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-21T15:01:25.294436+0000","last_change":"2024-09-21T15:01:24.364013+0000","last_active":"2024-09-21T15:01:25.294436+0000","last_peered":"2024-09-21T15:01:25.294436+0000","last_clean":"2024-09-21T15:01:25.294436+0000","last_became_active":"2024-09-21T14:58:19.409935+0000","last_became_peered":"2024-09-21T14:58:19.409935+0000","last_unstale":"2024-09-21T15:01:25.294436+0000","last_undegraded":"2024-09-21T15:01:25.294436+0000","last_fullsized":"2024-09-21T15:01:25.294436+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":479,"log_dups_size":0,"ondisk_log_size":479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:10:36.252629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067573199999999996,"stat_sum":{"num_bytes":44857867,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":3940,"num_read_kb":58689,"num_write":766,"num_write_kb":37279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"5b","length":"1"},{"start":"5e","length":"1"}]},{"pgid":"3.11","version":"193'549","reported_seq":4792,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-21T15:01:25.168484+0000","last_change":"2024-09-21T15:01:24.369207+0000","last_active":"2024-09-21T15:01:25.168484+0000","last_peered":"2024-09-21T15:01:25.168484+0000","last_clean":"2024-09-21T15:01:25.168484+0000","last_became_active":"2024-09-21T15:01:05.165955+0000","last_became_peered":"2024-09-21T15:01:05.165955+0000","last_unstale":"2024-09-21T15:01:25.168484+0000","last_undegraded":"2024-09-21T15:01:25.168484+0000","last_fullsized":"2024-09-21T15:01:25.168484+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":549,"log_dups_size":0,"ondisk_log_size":549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:34:54.111119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017085360000000001,"stat_sum":{"num_bytes":15225199,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":5404,"num_read_kb":28178,"num_write":715,"num_write_kb":24173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":166,"num_bytes_recovered":11965028,"num_keys_recovered":33,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"5b","length":"1"},{"start":"5e","length":"1"}]},{"pgid":"3.12","version":"193'592","reported_seq":8227,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-21T15:01:25.298624+0000","last_change":"2024-09-21T15:01:24.368110+0000","last_active":"2024-09-21T15:01:25.298624+0000","last_peered":"2024-09-21T15:01:25.298624+0000","last_clean":"2024-09-21T15:01:25.298624+0000","last_became_active":"2024-09-21T15:00:07.155144+0000","last_became_peered":"2024-09-21T15:00:07.155144+0000","last_unstale":"2024-09-21T15:01:25.298624+0000","last_undegraded":"2024-09-21T15:01:25.298624+0000","last_fullsized":"2024-09-21T15:01:25.298624+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":592,"log_dups_size":0,"ondisk_log_size":592,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:58:00.133639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017840219999999999,"stat_sum":{"num_bytes":30010444,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":8925,"num_read_kb":37100,"num_write":968,"num_write_kb":20890,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"5b","length":"1"},{"start":"5e","length":"1"}]},{"pgid":"3.13","version":"193'855","reported_seq":6462,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-21T15:01:25.295318+0000","last_change":"2024-09-21T15:01:24.369162+0000","last_active":"2024-09-21T15:01:25.295318+0000","last_peered":"2024-09-21T15:01:25.295318+0000","last_clean":"2024-09-21T15:01:25.295318+0000","last_became_active":"2024-09-21T14:58:19.412030+0000","last_became_peered":"2024-09-21T14:58:19.412030+0000","last_unstale":"2024-09-21T15:01:25.295318+0000","last_undegraded":"2024-09-21T15:01:25.295318+0000","last_fullsized":"2024-09-21T15:01:25.295318+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":855,"log_dups_size":0,"ondisk_log_size":855,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:44:29.282391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088511000000000004,"stat_sum":{"num_bytes":47192308,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":8196,"num_read_kb":59568,"num_write":1235,"num_write_kb":31566,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"5b","length":"1"},{"start":"5e","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":859207571,"num_objects":1053,"num_object_clones":0,"num_object_copies":2106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1053,"num_whiteouts":0,"num_read":167492,"num_read_kb":1232614,"num_write":32240,"num_write_kb":940429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2882,"num_bytes_recovered":203475064,"num_keys_recovered":305,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":53821440,"data_stored":477605455,"data_compressed":10958047,"data_compressed_allocated":42999808,"data_compressed_original":471590831,"omap_allocated":0,"internal_metadata":0},"log_size":20362,"ondisk_log_size":20362,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":10404,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509485,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37744,"kb_used_data":6628,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94334096,"statfs":{"total":96636764160,"available":96598114304,"internally_reserved":0,"allocated":6787072,"data_stored":40618945,"data_compressed":899038,"data_compressed_allocated":3547136,"data_compressed_original":38996771,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.85499999999999998,"15min":0.84199999999999997},"min":{"1min":0.46500000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.872,"5min":0.93300000000000005,"15min":0.95199999999999996},"min":{"1min":0.54900000000000004,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.3839999999999999,"5min":1.917,"15min":1.917},"last":0.81999999999999995}]},{"osd":1,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.81000000000000005,"15min":0.77700000000000002},"min":{"1min":0.35499999999999998,"5min":0.312,"15min":0.312},"max":{"1min":1.2789999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":1.0409999999999999},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.83199999999999996,"15min":0.82599999999999996},"min":{"1min":0.57399999999999995,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.208,"5min":2.2149999999999999,"15min":2.2149999999999999},"last":0.75700000000000001}]},{"osd":2,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.82199999999999995,"15min":0.81200000000000006},"min":{"1min":0.50900000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.169,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.93500000000000005,"15min":0.94299999999999995},"min":{"1min":0.55600000000000005,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.355,"5min":1.466,"15min":1.466},"last":0.91300000000000003}]},{"osd":3,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.88500000000000001,"15min":0.86099999999999999},"min":{"1min":0.60999999999999999,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.401,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":1.012},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.97799999999999998,"15min":0.96899999999999997},"min":{"1min":0.68200000000000005,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.3420000000000001,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.66400000000000003}]},{"osd":4,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.92200000000000004,"15min":0.93300000000000005},"min":{"1min":0.42199999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.3089999999999999,"5min":2.7130000000000001,"15min":2.7130000000000001},"last":1.1699999999999999},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.94499999999999995,"15min":0.92600000000000005},"min":{"1min":0.39000000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.425,"5min":2.125,"15min":2.125},"last":1.1950000000000001}]},{"osd":5,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":1.026,"15min":1.022},"min":{"1min":0.51300000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.5800000000000001,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.94299999999999995,"5min":1.0409999999999999,"15min":1.05},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.3029999999999999,"5min":3.1880000000000002,"15min":3.1880000000000002},"last":0.96099999999999997}]},{"osd":6,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.133,"15min":1.149},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.5329999999999999,"5min":3.2730000000000001,"15min":3.2730000000000001},"last":1.113},{"interface":"front","average":{"1min":1.095,"5min":1.0629999999999999,"15min":1.03},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.6990000000000001,"5min":2.2879999999999998,"15min":2.2879999999999998},"last":1.0860000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542188,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63000,"kb_used_data":9732,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94308840,"statfs":{"total":96636764160,"available":96572252160,"internally_reserved":0,"allocated":9965568,"data_stored":70094136,"data_compressed":1582838,"data_compressed_allocated":6189056,"data_compressed_original":68147580,"omap_allocated":1593,"internal_metadata":54524359},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:01:08 2024","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.92500000000000004,"15min":0.95099999999999996},"min":{"1min":0.51900000000000002,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.2010000000000001,"5min":1.593,"15min":1.593},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.93300000000000005,"15min":0.94999999999999996},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.226,"5min":1.587,"15min":1.587},"last":0.51600000000000001}]},{"osd":1,"last update":"Sat Sep 21 15:01:08 2024","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.89400000000000002,"15min":0.91000000000000003},"min":{"1min":0.51700000000000002,"5min":0.502,"15min":0.502},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.90200000000000002,"15min":0.96199999999999997},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.47199999999999998}]},{"osd":2,"last update":"Sat Sep 21 15:01:08 2024","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.96199999999999997,"15min":0.98399999999999999},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.389,"5min":1.758,"15min":1.758},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.90900000000000003,"15min":0.91300000000000003},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.444,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.84899999999999998}]},{"osd":3,"last update":"Sat Sep 21 15:01:08 2024","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.94799999999999995,"15min":0.95899999999999996},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.5329999999999999,"5min":1.835,"15min":1.835},"last":0.70899999999999996},{"interface":"front","average":{"1min":1.016,"5min":1.0920000000000001,"15min":1.127},"min":{"1min":0.64600000000000002,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.4670000000000001,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.68300000000000005}]},{"osd":4,"last update":"Sat Sep 21 15:01:08 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.93500000000000005,"15min":0.99099999999999999},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":1.2,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.98499999999999999,"15min":1.0209999999999999},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.3759999999999999,"5min":1.897,"15min":1.897},"last":0.79200000000000004}]},{"osd":5,"last update":"Sat Sep 21 15:01:08 2024","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":1.0449999999999999,"15min":1.1100000000000001},"min":{"1min":0.63500000000000001,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.256,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.60199999999999998},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0660000000000001,"15min":1.087},"min":{"1min":0.65400000000000003,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.609,"5min":1.609,"15min":1.609},"last":0.99199999999999999}]},{"osd":7,"last update":"Sat Sep 21 15:01:08 2024","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.206,"15min":1.244},"min":{"1min":0.78900000000000003,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.3109999999999999,"5min":1.8660000000000001,"15min":1.8660000000000001},"last":0.57399999999999995},{"interface":"front","average":{"1min":1,"5min":1.1970000000000001,"15min":1.2749999999999999},"min":{"1min":0.69499999999999995,"5min":0.63,"15min":0.63},"max":{"1min":1.298,"5min":2.004,"15min":2.004},"last":0.90300000000000002}]}]},{"osd":5,"up_from":14,"seq":60129542189,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66808,"kb_used_data":9444,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94305032,"statfs":{"total":96636764160,"available":96568352768,"internally_reserved":0,"allocated":9670656,"data_stored":66479509,"data_compressed":1286479,"data_compressed_allocated":5734400,"data_compressed_original":64428004,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:01:07 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.89900000000000002,"15min":0.98799999999999999},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.337,"5min":4.4800000000000004,"15min":4.4800000000000004},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.627,"5min":0.76800000000000002,"15min":0.80800000000000005},"min":{"1min":0.36099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.94999999999999996,"5min":1.181,"15min":1.181},"last":1.0620000000000001}]},{"osd":1,"last update":"Sat Sep 21 15:01:02 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.89600000000000002,"15min":0.92400000000000004},"min":{"1min":0.47699999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":2.7709999999999999,"5min":2.7709999999999999,"15min":2.7709999999999999},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.91400000000000003,"15min":0.93400000000000005},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":2.823,"5min":2.823,"15min":2.823},"last":0.84799999999999998}]},{"osd":2,"last update":"Sat Sep 21 15:01:02 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.91200000000000003,"15min":0.97399999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.159,"5min":3.419,"15min":3.419},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.87,"15min":0.89600000000000002},"min":{"1min":0.53400000000000003,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0780000000000001,"5min":1.423,"15min":1.423},"last":0.95899999999999996}]},{"osd":3,"last update":"Sat Sep 21 15:01:02 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.80300000000000005,"15min":0.80700000000000005},"min":{"1min":0.58499999999999996,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":1.022},{"interface":"front","average":{"1min":0.96699999999999997,"5min":1.054,"15min":1.0840000000000001},"min":{"1min":0.59899999999999998,"5min":0.435,"15min":0.435},"max":{"1min":1.272,"5min":2.6379999999999999,"15min":2.6379999999999999},"last":0.88700000000000001}]},{"osd":4,"last update":"Sat Sep 21 15:01:02 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.98899999999999999,"15min":1.016},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.5609999999999999,"5min":2.5470000000000002,"15min":2.5470000000000002},"last":0.748},{"interface":"front","average":{"1min":0.82099999999999995,"5min":1.0169999999999999,"15min":1.0780000000000001},"min":{"1min":0.39300000000000002,"5min":0.376,"15min":0.376},"max":{"1min":1.302,"5min":2.5099999999999998,"15min":2.5099999999999998},"last":0.90900000000000003}]},{"osd":6,"last update":"Sat Sep 21 15:01:02 2024","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.94299999999999995,"15min":0.96899999999999997},"min":{"1min":0.63,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.97799999999999998},{"interface":"front","average":{"1min":1.004,"5min":1.2450000000000001,"15min":1.3520000000000001},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.425,"5min":4.2469999999999999,"15min":4.2469999999999999},"last":0.77800000000000002}]},{"osd":7,"last update":"Sat Sep 21 15:01:07 2024","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.2569999999999999,"15min":1.3420000000000001},"min":{"1min":0.52500000000000002,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.383,"5min":3.3010000000000002,"15min":3.3010000000000002},"last":0.71099999999999997},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.107,"15min":1.135},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.462,"5min":2.5880000000000001,"15min":2.5880000000000001},"last":0.93000000000000005}]}]},{"osd":4,"up_from":0,"seq":60129542188,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":3,"up_from":14,"seq":60129542189,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77372,"kb_used_data":10148,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94294468,"statfs":{"total":96636764160,"available":96557535232,"internally_reserved":0,"allocated":10391552,"data_stored":69812887,"data_compressed":1651860,"data_compressed_allocated":6279168,"data_compressed_original":67643778,"omap_allocated":1592,"internal_metadata":68811208},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:01:09 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.625,"15min":0.63400000000000001},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.171,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.68200000000000005,"15min":0.70999999999999996},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.0700000000000001,"5min":1.919,"15min":1.919},"last":0.40500000000000003}]},{"osd":1,"last update":"Sat Sep 21 15:01:09 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.66400000000000003,"15min":0.67200000000000004},"min":{"1min":0.29499999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":1.722,"5min":1.722,"15min":1.722},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.76500000000000001,"15min":0.80300000000000005},"min":{"1min":0.33300000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.89500000000000002,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.45300000000000001}]},{"osd":2,"last update":"Sat Sep 21 15:01:09 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.71999999999999997,"15min":0.72199999999999998},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.92400000000000004,"5min":1.157,"15min":1.157},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.81000000000000005,"15min":0.82699999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.1539999999999999,"5min":1.254,"15min":1.254},"last":0.89900000000000002}]},{"osd":4,"last update":"Sat Sep 21 15:01:09 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.93799999999999994,"15min":0.95599999999999996},"min":{"1min":0.54300000000000004,"5min":0.442,"15min":0.442},"max":{"1min":1.2629999999999999,"5min":1.7609999999999999,"15min":1.7609999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.98899999999999999,"15min":1.0269999999999999},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.234,"5min":2.028,"15min":2.028},"last":0.67200000000000004}]},{"osd":5,"last update":"Sat Sep 21 15:01:09 2024","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":1.0369999999999999,"15min":1.052},"min":{"1min":0.70499999999999996,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.3440000000000001,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":0.72599999999999998},{"interface":"front","average":{"1min":1.0249999999999999,"5min":1.056,"15min":1.0629999999999999},"min":{"1min":0.57999999999999996,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.548,"5min":1.548,"15min":1.548},"last":0.88100000000000001}]},{"osd":6,"last update":"Sat Sep 21 15:01:09 2024","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":1.0669999999999999,"15min":1.0920000000000001},"min":{"1min":0.59099999999999997,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.6419999999999999,"5min":2.1219999999999999,"15min":2.1219999999999999},"last":0.97699999999999998},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.091,"15min":1.103},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.6839999999999999,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":1.105}]},{"osd":7,"last update":"Sat Sep 21 15:01:09 2024","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":1.103,"15min":1.131},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.498,"5min":1.706,"15min":1.706},"last":1.046},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.147,"15min":1.177},"min":{"1min":0.59699999999999998,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.417,"5min":2.3380000000000001,"15min":2.3380000000000001},"last":0.76600000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542187,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":75372,"kb_used_data":12248,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94296468,"statfs":{"total":96636764160,"available":96559583232,"internally_reserved":0,"allocated":12541952,"data_stored":96037545,"data_compressed":2214647,"data_compressed_allocated":8470528,"data_compressed_original":93943782,"omap_allocated":1594,"internal_metadata":64616902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.62},"min":{"1min":0.26900000000000002,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.92800000000000005,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.69399999999999995,"15min":0.67000000000000004},"min":{"1min":0.34799999999999998,"5min":0.223,"15min":0.223},"max":{"1min":2.9169999999999998,"5min":2.9169999999999998,"15min":2.9169999999999998},"last":0.89600000000000002}]},{"osd":1,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.32700000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.0529999999999999,"5min":2.6520000000000001,"15min":2.6520000000000001},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.746,"5min":0.73499999999999999,"15min":0.75800000000000001},"min":{"1min":0.40500000000000003,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.349,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.95399999999999996}]},{"osd":3,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.73999999999999999,"15min":0.73499999999999999},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":3.0110000000000001,"5min":3.0110000000000001,"15min":3.0110000000000001},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.877,"15min":0.83899999999999997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.05,"5min":4.1440000000000001,"15min":4.1440000000000001},"last":0.85199999999999998}]},{"osd":4,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.93200000000000005,"15min":0.95599999999999996},"min":{"1min":0.42399999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.222,"5min":2.899,"15min":2.899},"last":0.99299999999999999},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.94399999999999995,"15min":0.95799999999999996},"min":{"1min":0.47499999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.4379999999999999,"5min":1.675,"15min":1.675},"last":1.0640000000000001}]},{"osd":5,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.0129999999999999,"15min":1.0089999999999999},"min":{"1min":0.45500000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":3.1080000000000001,"5min":3.1080000000000001,"15min":3.1080000000000001},"last":1.137},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.98199999999999998,"15min":0.99399999999999999},"min":{"1min":0.51700000000000002,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.268,"5min":1.546,"15min":1.546},"last":1.097}]},{"osd":6,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":0.98299999999999998,"15min":0.98499999999999999},"min":{"1min":0.56200000000000006,"5min":0.378,"15min":0.378},"max":{"1min":1.345,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":1.1799999999999999},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.0649999999999999,"15min":1.0549999999999999},"min":{"1min":0.53200000000000003,"5min":0.502,"15min":0.502},"max":{"1min":3.0739999999999998,"5min":3.0739999999999998,"15min":3.0739999999999998},"last":1.403}]},{"osd":7,"last update":"Sat Sep 21 15:01:08 2024","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.0449999999999999,"15min":1.0089999999999999},"min":{"1min":0.59499999999999997,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":3.1589999999999998,"5min":3.1589999999999998,"15min":3.1589999999999998},"last":1.3600000000000001},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.121,"15min":1.099},"min":{"1min":0.66000000000000003,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":3.1400000000000001,"5min":3.1400000000000001,"15min":3.1400000000000001},"last":1.272}]}]},{"osd":1,"up_from":14,"seq":60129542188,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68680,"kb_used_data":11320,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94303160,"statfs":{"total":96636764160,"available":96566435840,"internally_reserved":0,"allocated":11591680,"data_stored":84904208,"data_compressed":1773279,"data_compressed_allocated":7413760,"data_compressed_original":82761176,"omap_allocated":1595,"internal_metadata":58718661},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.59599999999999997,"15min":0.60899999999999999},"min":{"1min":0.36399999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.78200000000000003,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59699999999999998,"15min":0.60799999999999998},"min":{"1min":0.39500000000000002,"5min":0.216,"15min":0.216},"max":{"1min":0.80100000000000005,"5min":1.107,"15min":1.107},"last":0.26300000000000001}]},{"osd":2,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":0.79600000000000004,"15min":0.73999999999999999},"min":{"1min":0.38700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":11.303000000000001,"5min":11.303000000000001,"15min":11.303000000000001},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.71299999999999997,"15min":0.70299999999999996},"min":{"1min":0.47299999999999998,"5min":0.435,"15min":0.435},"max":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.68000000000000005}]},{"osd":3,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.81799999999999995,"15min":0.83899999999999997},"min":{"1min":0.27800000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.0049999999999999,"5min":3.1469999999999998,"15min":3.1469999999999998},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.86399999999999999,"15min":0.88600000000000001},"min":{"1min":0.52800000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.306,"5min":3.2320000000000002,"15min":3.2320000000000002},"last":0.42499999999999999}]},{"osd":4,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.89000000000000001,"15min":0.91500000000000004},"min":{"1min":0.59599999999999997,"5min":0.309,"15min":0.309},"max":{"1min":1.45,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":0.443},{"interface":"front","average":{"1min":1.5600000000000001,"5min":1.069,"15min":0.96599999999999997},"min":{"1min":0.504,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":11.563000000000001,"5min":11.563000000000001,"15min":11.563000000000001},"last":0.70799999999999996}]},{"osd":5,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":1.625,"5min":1.0469999999999999,"15min":0.94899999999999995},"min":{"1min":0.73399999999999999,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":11.534000000000001,"5min":11.534000000000001,"15min":11.534000000000001},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95799999999999996,"15min":0.96099999999999997},"min":{"1min":0.63,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.256,"5min":1.484,"15min":1.484},"last":0.58599999999999997}]},{"osd":6,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":1.0109999999999999,"15min":1.0489999999999999},"min":{"1min":0.60399999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.4059999999999999,"5min":2.3519999999999999,"15min":2.3519999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":1.631,"5min":1.081,"15min":0.99299999999999999},"min":{"1min":0.57399999999999995,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":11.601000000000001,"5min":11.601000000000001,"15min":11.601000000000001},"last":0.66400000000000003}]},{"osd":7,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":1.046,"15min":1.0640000000000001},"min":{"1min":0.79300000000000004,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.5820000000000001,"5min":2.2360000000000002,"15min":2.2360000000000002},"last":0.61099999999999999},{"interface":"front","average":{"1min":1.76,"5min":1.171,"15min":1.087},"min":{"1min":0.877,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":11.489000000000001,"5min":11.489000000000001,"15min":11.489000000000001},"last":0.754}]}]},{"osd":0,"up_from":14,"seq":60129542187,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":71708,"kb_used_data":10248,"kb_used_omap":1,"kb_used_meta":61438,"kb_avail":94300132,"statfs":{"total":96636764160,"available":96563335168,"internally_reserved":0,"allocated":10493952,"data_stored":73876379,"data_compressed":1664532,"data_compressed_allocated":6557696,"data_compressed_original":71855811,"omap_allocated":1593,"internal_metadata":62912967},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 21 15:01:04 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.67500000000000004,"15min":0.71699999999999997},"min":{"1min":0.40999999999999998,"5min":0.309,"15min":0.309},"max":{"1min":1.1299999999999999,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.68100000000000005,"15min":0.627},"min":{"1min":0.58999999999999997,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.81999999999999995}]},{"osd":2,"last update":"Sat Sep 21 15:01:04 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.65200000000000002,"15min":0.61799999999999999},"min":{"1min":0.59599999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.72499999999999998,"15min":0.75800000000000001},"min":{"1min":0.39300000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.91100000000000003}]},{"osd":3,"last update":"Sat Sep 21 15:01:04 2024","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.76600000000000001,"15min":0.754},"min":{"1min":0.50900000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.621},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.78700000000000003,"15min":0.81499999999999995},"min":{"1min":0.50900000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.65800000000000003}]},{"osd":4,"last update":"Sat Sep 21 15:01:04 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.98399999999999999,"15min":1.038},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.1140000000000001,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":0.79400000000000004},{"interface":"front","average":{"1min":1.036,"5min":1.0469999999999999,"15min":1.0620000000000001},"min":{"1min":0.67800000000000005,"5min":0.379,"15min":0.379},"max":{"1min":1.446,"5min":1.665,"15min":1.665},"last":1.129}]},{"osd":5,"last update":"Sat Sep 21 15:01:04 2024","interfaces":[{"interface":"back","average":{"1min":1.0960000000000001,"5min":1.024,"15min":1.032},"min":{"1min":0.79300000000000004,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.4450000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":1.2609999999999999},{"interface":"front","average":{"1min":1.0589999999999999,"5min":1.0429999999999999,"15min":1.077},"min":{"1min":0.82399999999999995,"5min":0.435,"15min":0.435},"max":{"1min":1.2609999999999999,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":1.0629999999999999}]},{"osd":6,"last update":"Sat Sep 21 15:01:04 2024","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":1.0580000000000001,"15min":1.1120000000000001},"min":{"1min":0.68400000000000005,"5min":0.628,"15min":0.628},"max":{"1min":1.167,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":1.085},{"interface":"front","average":{"1min":1.109,"5min":1.0629999999999999,"15min":1.0680000000000001},"min":{"1min":0.91000000000000003,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.5029999999999999,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":1.3340000000000001}]},{"osd":7,"last update":"Sat Sep 21 15:01:04 2024","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.0469999999999999,"15min":1.0620000000000001},"min":{"1min":0.78700000000000003,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.4770000000000001,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":1.0129999999999999},{"interface":"front","average":{"1min":1.02,"5min":1.085,"15min":1.131},"min":{"1min":0.84799999999999998,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.403,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":0.86799999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5202,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5202,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8167424,"data_stored":72741855,"data_compressed":1664532,"data_compressed_allocated":6557696,"data_compressed_original":71855811,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9207808,"data_stored":83179316,"data_compressed":1768077,"data_compressed_allocated":7376896,"data_compressed_original":82187736,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8560640,"data_stored":75535861,"data_compressed":1978769,"data_compressed_allocated":6971392,"data_compressed_original":74654367,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8060928,"data_stored":68678344,"data_compressed":1651860,"data_compressed_allocated":6279168,"data_compressed_original":67643778,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":675840,"data_stored":7155966,"data_compressed":134408,"data_compressed_allocated":589824,"data_compressed_original":7119101,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":7286784,"data_stored":64754617,"data_compressed":1281277,"data_compressed_allocated":5697536,"data_compressed_original":63854564,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":7413760,"data_stored":66081019,"data_compressed":1580086,"data_compressed_allocated":5980160,"data_compressed_original":65278703,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4448256,"data_stored":39478477,"data_compressed":899038,"data_compressed_allocated":3547136,"data_compressed_original":38996771,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T15:01:27.383 INFO:tasks.thrashosds.thrasher:Setting 3.8 to [6, 3] 2024-09-21T15:01:27.383 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.8', '6', '3'] 2024-09-21T15:01:27.383 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.8 6 3 2024-09-21T15:01:27.384 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:01:27.376+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:01:27.386 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T15:01:27.481 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:01:27.478+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:01:27.489 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:27.489 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:27.500 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T15:01:27.582 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:01:27.578+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:01:27.602 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:27.602 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:27.612 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T15:01:27.683 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:01:27.680+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:01:27.766 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:27.766 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:27.775 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T15:01:27.784 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:01:27.782+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:01:27.885 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:01:27.880+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:01:27.888 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:27.888 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:27.898 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T15:01:27.986 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:01:27.982+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:01:28.053 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:28.053 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:28.063 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T15:01:28.087 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:01:28.084+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:01:28.170 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:28.170 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:28.179 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T15:01:28.191 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T15:01:28.223 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:01:28.220+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:01:28.289 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:01:28.290+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:01:28.297 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:28.297 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:28.308 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T15:01:28.390 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:01:28.384+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:01:28.392 INFO:teuthology.orchestra.run.smithi032.stderr:set 3.8 pg_upmap mapping to [6,3] 2024-09-21T15:01:28.415 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:28.415 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:28.425 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T15:01:28.491 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:01:28.490+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:01:28.533 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:28.533 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:28.543 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T15:01:28.592 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:01:28.588+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:01:28.649 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:28.649 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:28.659 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T15:01:28.692 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:01:28.690+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:01:28.771 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:28.771 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:28.782 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T15:01:28.794 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:01:28.794+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:01:28.887 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:28.887 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:28.894 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:01:28.894+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:01:28.897 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T15:01:28.995 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:01:28.992+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:01:29.009 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:29.009 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:29.019 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T15:01:29.096 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:01:29.094+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:01:29.130 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:29.130 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:29.140 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T15:01:29.197 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:01:29.198+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:01:29.250 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:29.250 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:29.260 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T15:01:29.298 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:01:29.298+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:01:29.366 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:29.366 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:29.376 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T15:01:29.390 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T15:01:29.399 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:01:29.396+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:01:29.400 INFO:teuthology.orchestra.run.smithi032.stderr:noscrub is set 2024-09-21T15:01:29.479 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:29.479 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:29.489 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T15:01:29.531 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:01:29.528+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:01:29.537 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T15:01:29.560 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T15:01:29.583 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T15:01:29.601 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:01:29.596+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:01:29.605 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T15:01:29.629 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-21T15:01:29.645 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:29.645 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:29.653 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T15:01:29.655 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T15:01:29.676 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T15:01:29.699 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T15:01:29.702 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:01:29.702 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:01:29.702 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:01:29.702 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:01:29.702 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:01:29.702 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:01:29.702 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:01:29.702 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:01:29.703 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:01:29.696+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:01:29.767 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:29.767 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:01:29.768 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T15:01:29.768 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:29.777 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T15:01:29.803 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:01:29.802+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:01:29.888 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:29.888 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:01:29.888 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T15:01:29.888 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T15:01:29.888 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:29.898 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T15:01:29.903 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:01:29.902+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:01:30.004 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:01:30.002+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:01:30.011 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:30.012 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T15:01:30.012 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T15:01:30.012 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T15:01:30.012 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.012 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:97677 3.15 3:a8e72133:::rbd_data.1136bfc79a70.0000000000000005:head [write 0~4440027 [fadvise_sequential+fadvise_nocache],write 4751360~147456 [fadvise_sequential+fadvise_nocache],write 4964352~147456 [fadvise_sequential+fadvise_nocache],write 5177344~65536 [fadvise_sequential+fadvise_nocache],write 5259264~147456 [fadvise_sequential+fadvise_nocache],write 5472256~147456 [fadvise_sequential+fadvise_nocache],write 5685248~229376 [fadvise_sequential+fadvise_nocache],write 5980160~147456 [fadvise_sequential+fadvise_nocache],write 6193152~229376 [fadvise_sequential+fadvise_nocache],write 6488064~147456 [fadvise_sequential+fadvise_nocache],write 6701056~229376 [fadvise_sequential+fadvise_nocache],write 6995968~147456 [fadvise_sequential+fadvise_nocache],write 7208960~229376 [fadvise_sequential+fadvise_nocache],write 7503872~147456 [fadvise_sequential+fadvise_nocache],write 7716864~229376 [fadvise_sequential+fadvise_nocache],write 8044544~114688 [fadvise_sequential+fadvise_nocache],write 8257536~114688 [fadvise_sequential+fadvise_nocache],write 8388608~65536 [fadvise_sequential+fadvise_nocache],write 8552448~114688 [fadvise_sequential+fadvise_nocache],write 8765440~114688 [fadvise_sequential+fadvise_nocache],write 8929280~32768 [fadvise_sequential+fadvise_nocache],write 9142272~32768 [fadvise_sequential+fadvise_nocache],write 9355264~32768 [fadvise_sequential+fadvise_nocache],write 9437184~32768 [fadvise_sequential+fadvise_nocache],write 9650176~32768 [fadvise_sequential+fadvise_nocache],write 9863168~32768 [fadvise_sequential+fadvise_nocache],write 9961472~131072 [fadvise_sequential+fadvise_nocache],write 10108928~196608 [fadvise_sequential+fadvise_nocache],write 10321920~278528 [fadvise_sequential+fadvise_nocache],write 10616832~196608 [fadvise_sequential+fadvise_nocache],write 10829824~278528 [fadvise_sequential+fadvise_nocache],write 11124736~196608 [fadvise_sequential+fadvise_nocache],write 11337728~196608 [fadvise_sequential+fadvise_nocache],write 11550720~65536 [fadvise_sequential+fadvise_nocache],write 11649024~98304 [fadvise_sequential+fadvise_nocache],write 11763712~65536 [fadvise_sequential+fadvise_nocache],write 11862016~98304 [fadvise_sequential+fadvise_nocache],write 11976704~65536 [fadvise_sequential+fadvise_nocache],write 12058624~65536 [fadvise_sequential+fadvise_nocache],write 12156928~98304 [fadvise_sequential+fadvise_nocache],write 12271616~65536 [fadvise_sequential+fadvise_nocache],write 12369920~98304 [fadvise_sequential+fadvise_nocache],write 12484608~65536 [fadvise_sequential+fadvise_nocache],write 12664832~98304 [fadvise_sequential+fadvise_nocache],write 12877824~98304 [fadvise_sequential+fadvise_nocache],write 13090816~16384 [fadvise_sequential+fadvise_nocache],write 13172736~98304 [fadvise_sequential+fadvise_nocache],write 13385728~98304 [fadvise_sequential+fadvise_nocache],write 13598720~32768 [fadvise_sequential+fadvise_nocache],write 13680640~98304 [fadvise_sequential+fadvise_nocache],write 13893632~98304 [fadvise_sequential+fadvise_nocache],write 14106624~49152 [fadvise_sequential+fadvise_nocache],write 14794752~81920 [fadvise_sequential+fadvise_nocache],write 15007744~81920 [fadvise_sequential+fadvise_nocache],write 15302656~81920 [fadvise_sequential+fadvise_nocache],write 15515648~81920 [fadvise_sequential+fadvise_nocache],write 15810560~81920 [fadvise_sequential+fadvise_nocache],write 16023552~81920 [fadvise_sequential+fadvise_nocache],write 16236544~16384 [fadvise_sequential+fadvise_nocache],write 16318464~81920 [fadvise_sequential+fadvise_nocache],write 16531456~81920 [fadvise_sequential+fadvise_nocache],write 16744448~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:01:30.012 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.302946+0000", 2024-09-21T15:01:30.012 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 84.707992450000006, 2024-09-21T15:01:30.012 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.091231204999999996, 2024-09-21T15:01:30.012 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:30.012 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:30.012 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:30.012 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:30.012 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:30.013 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 97677 2024-09-21T15:01:30.013 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.013 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:30.013 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.013 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:30.013 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.302946+0000", 2024-09-21T15:01:30.013 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:30.013 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.013 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.013 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:30.013 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.302946+0000", 2024-09-21T15:01:30.013 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:30.013 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.013 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.013 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:30.014 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.302942+0000", 2024-09-21T15:01:30.014 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999962 2024-09-21T15:01:30.014 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.014 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.014 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:30.014 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311253+0000", 2024-09-21T15:01:30.014 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0083105360000000003 2024-09-21T15:01:30.014 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.014 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.014 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:30.014 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311283+0000", 2024-09-21T15:01:30.014 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0065000000000001e-05 2024-09-21T15:01:30.014 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.014 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.015 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:30.015 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311296+0000", 2024-09-21T15:01:30.015 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3206e-05 2024-09-21T15:01:30.015 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.015 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.015 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:30.015 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311404+0000", 2024-09-21T15:01:30.015 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000107514 2024-09-21T15:01:30.015 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.015 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.015 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:30.015 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311795+0000", 2024-09-21T15:01:30.015 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000390913 2024-09-21T15:01:30.015 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.015 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.016 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 6", 2024-09-21T15:01:30.016 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.314686+0000", 2024-09-21T15:01:30.016 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0028915659999999999 2024-09-21T15:01:30.016 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.016 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.016 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:01:30.016 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.368178+0000", 2024-09-21T15:01:30.016 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.053491706 2024-09-21T15:01:30.016 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.016 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.016 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:01:30.016 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.393925+0000", 2024-09-21T15:01:30.016 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025747289 2024-09-21T15:01:30.016 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.016 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.017 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:01:30.017 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.394127+0000", 2024-09-21T15:01:30.017 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000201444 2024-09-21T15:01:30.017 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.017 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.017 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:30.017 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.394177+0000", 2024-09-21T15:01:30.017 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0689000000000002e-05 2024-09-21T15:01:30.017 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:30.017 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:30.017 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:30.017 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.017 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.018 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:97673 3.1d e122/24 3:bd91946e:::rbd_data.1136bfc79a70.0000000000000002:head v 122'162, mlcod=120'160)", 2024-09-21T15:01:30.018 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.313730+0000", 2024-09-21T15:01:30.018 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 84.697208125000003, 2024-09-21T15:01:30.018 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055373462999999998, 2024-09-21T15:01:30.018 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:30.018 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:30.018 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:30.018 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.018 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:30.018 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.313730+0000", 2024-09-21T15:01:30.018 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:30.018 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.018 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.018 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:30.019 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.313730+0000", 2024-09-21T15:01:30.019 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:30.019 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.019 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.019 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:30.019 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.313730+0000", 2024-09-21T15:01:30.019 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:30.019 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.019 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.019 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:30.019 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.336113+0000", 2024-09-21T15:01:30.019 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022382962999999999 2024-09-21T15:01:30.019 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.020 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.020 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:30.020 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.336140+0000", 2024-09-21T15:01:30.020 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7248999999999999e-05 2024-09-21T15:01:30.020 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.020 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.020 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:30.020 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.336151+0000", 2024-09-21T15:01:30.020 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1015e-05 2024-09-21T15:01:30.020 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.020 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.020 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:30.020 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.355428+0000", 2024-09-21T15:01:30.020 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019276379999999999 2024-09-21T15:01:30.021 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.021 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.021 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:30.021 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.355484+0000", 2024-09-21T15:01:30.021 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6820999999999997e-05 2024-09-21T15:01:30.021 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.021 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.021 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:01:30.021 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.369040+0000", 2024-09-21T15:01:30.021 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013555466 2024-09-21T15:01:30.021 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.021 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.021 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:30.021 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.369104+0000", 2024-09-21T15:01:30.022 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3986999999999996e-05 2024-09-21T15:01:30.022 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:30.022 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:30.022 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:30.022 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.022 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.022 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139890 3.e 3:71dba096:::rbd_data.113664780c85.000000000000018b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:30.022 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.226444+0000", 2024-09-21T15:01:30.022 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 27.784494483, 2024-09-21T15:01:30.022 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.028209567, 2024-09-21T15:01:30.022 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:30.022 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:30.022 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:30.022 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:30.023 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:30.023 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139890 2024-09-21T15:01:30.023 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.023 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:30.023 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.023 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:30.023 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226444+0000", 2024-09-21T15:01:30.023 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:30.023 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.023 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.023 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:30.023 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226444+0000", 2024-09-21T15:01:30.023 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:30.023 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.024 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.024 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:30.024 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226443+0000", 2024-09-21T15:01:30.024 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:30.024 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.024 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.024 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:30.024 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226448+0000", 2024-09-21T15:01:30.024 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5499999999999996e-06 2024-09-21T15:01:30.024 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.024 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.024 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:30.024 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226456+0000", 2024-09-21T15:01:30.024 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8679999999999999e-06 2024-09-21T15:01:30.024 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.025 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.025 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:30.025 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226462+0000", 2024-09-21T15:01:30.025 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1800000000000001e-06 2024-09-21T15:01:30.025 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.025 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.025 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:30.025 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226545+0000", 2024-09-21T15:01:30.025 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2527000000000005e-05 2024-09-21T15:01:30.025 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.025 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.025 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:30.025 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226553+0000", 2024-09-21T15:01:30.025 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8019999999999998e-06 2024-09-21T15:01:30.025 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.026 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.026 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:30.026 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.254491+0000", 2024-09-21T15:01:30.026 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.02793785 2024-09-21T15:01:30.026 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.026 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.026 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:30.026 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.254654+0000", 2024-09-21T15:01:30.026 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00016245800000000001 2024-09-21T15:01:30.026 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:30.026 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:30.026 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:30.026 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.026 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.027 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139897 3.e 3:76ddeb1d:::rbd_data.113664780c85.0000000000000004:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:30.027 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.227198+0000", 2024-09-21T15:01:30.027 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 27.783740878, 2024-09-21T15:01:30.027 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.027799822, 2024-09-21T15:01:30.027 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:30.027 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:30.027 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:30.027 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:30.027 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:30.027 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139897 2024-09-21T15:01:30.027 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.027 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:30.027 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.027 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:30.027 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227198+0000", 2024-09-21T15:01:30.027 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:30.028 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.028 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.028 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:30.028 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227198+0000", 2024-09-21T15:01:30.028 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:30.028 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.028 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.028 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:30.028 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227197+0000", 2024-09-21T15:01:30.028 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:30.028 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.028 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.028 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:30.028 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227203+0000", 2024-09-21T15:01:30.028 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4819999999999999e-06 2024-09-21T15:01:30.028 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.029 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.029 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:30.029 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227220+0000", 2024-09-21T15:01:30.029 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6671e-05 2024-09-21T15:01:30.029 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.029 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.029 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:30.029 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227229+0000", 2024-09-21T15:01:30.029 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0499999999999997e-06 2024-09-21T15:01:30.029 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.029 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.029 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:30.029 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227293+0000", 2024-09-21T15:01:30.029 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4434e-05 2024-09-21T15:01:30.029 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.029 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.030 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:30.030 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227302+0000", 2024-09-21T15:01:30.030 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0929999999999992e-06 2024-09-21T15:01:30.030 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.030 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.030 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:30.030 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.254720+0000", 2024-09-21T15:01:30.030 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0274177550000001 2024-09-21T15:01:30.030 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.030 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.030 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:30.030 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.254854+0000", 2024-09-21T15:01:30.030 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000133416 2024-09-21T15:01:30.030 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.030 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.030 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:30.031 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.254998+0000", 2024-09-21T15:01:30.031 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014400099999999999 2024-09-21T15:01:30.031 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:30.031 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:30.031 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:30.031 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.031 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.031 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139902 3.e 3:76b9b799:::rbd_data.113664780c85.0000000000000021:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:30.031 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.227319+0000", 2024-09-21T15:01:30.031 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 27.783619497, 2024-09-21T15:01:30.031 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0279584639999999, 2024-09-21T15:01:30.031 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:30.031 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:30.031 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:30.031 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:30.032 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:30.032 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139902 2024-09-21T15:01:30.032 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.032 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:30.032 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.032 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:30.032 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227319+0000", 2024-09-21T15:01:30.032 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:30.032 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.032 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.032 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:30.032 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227319+0000", 2024-09-21T15:01:30.032 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:30.032 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.032 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.032 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:30.033 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227319+0000", 2024-09-21T15:01:30.033 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:30.033 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.033 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.033 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:30.033 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227322+0000", 2024-09-21T15:01:30.033 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1980000000000001e-06 2024-09-21T15:01:30.033 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.033 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.033 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:30.033 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227328+0000", 2024-09-21T15:01:30.033 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4230000000000002e-06 2024-09-21T15:01:30.033 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.033 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.033 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:30.034 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227333+0000", 2024-09-21T15:01:30.034 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0560000000000004e-06 2024-09-21T15:01:30.034 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.034 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.034 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:30.034 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227729+0000", 2024-09-21T15:01:30.034 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00039591799999999999 2024-09-21T15:01:30.034 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.034 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.034 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:30.034 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227738+0000", 2024-09-21T15:01:30.034 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8119999999999997e-06 2024-09-21T15:01:30.034 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.034 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.034 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:30.034 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255021+0000", 2024-09-21T15:01:30.035 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0272826779999999 2024-09-21T15:01:30.035 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.035 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.035 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:30.035 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255147+0000", 2024-09-21T15:01:30.035 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012595600000000001 2024-09-21T15:01:30.035 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.035 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.035 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:30.035 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255278+0000", 2024-09-21T15:01:30.035 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013081400000000001 2024-09-21T15:01:30.035 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:30.035 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:30.035 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:30.035 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.036 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.036 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139945 3.e 3:7756a55d:::rbd_data.113664780c85.0000000000000154:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:30.036 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.229015+0000", 2024-09-21T15:01:30.036 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 27.781923694, 2024-09-21T15:01:30.036 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.026468825, 2024-09-21T15:01:30.036 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:30.036 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:30.036 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:30.036 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:30.036 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:30.036 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139945 2024-09-21T15:01:30.036 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.036 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:30.036 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.036 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:30.037 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229015+0000", 2024-09-21T15:01:30.037 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:30.037 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.037 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.037 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:30.037 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229015+0000", 2024-09-21T15:01:30.037 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:30.037 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.037 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.037 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:30.037 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229014+0000", 2024-09-21T15:01:30.037 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:30.037 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.037 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.037 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:30.037 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229017+0000", 2024-09-21T15:01:30.038 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7310000000000002e-06 2024-09-21T15:01:30.038 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.038 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.038 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:30.038 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229023+0000", 2024-09-21T15:01:30.038 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8150000000000002e-06 2024-09-21T15:01:30.038 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.038 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.038 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:30.038 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229028+0000", 2024-09-21T15:01:30.038 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7650000000000001e-06 2024-09-21T15:01:30.038 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.038 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.038 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:30.039 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229082+0000", 2024-09-21T15:01:30.039 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4521999999999997e-05 2024-09-21T15:01:30.039 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.039 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.039 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:30.039 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229092+0000", 2024-09-21T15:01:30.039 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.3940000000000002e-06 2024-09-21T15:01:30.039 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.039 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.039 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:30.039 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255382+0000", 2024-09-21T15:01:30.039 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0262900930000001 2024-09-21T15:01:30.039 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.039 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.039 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:30.039 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255484+0000", 2024-09-21T15:01:30.040 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000101932 2024-09-21T15:01:30.040 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:30.040 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:30.040 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:30.040 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.040 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.040 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139967 3.e 3:74e90d85:::rbd_data.113664780c85.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:30.040 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.229124+0000", 2024-09-21T15:01:30.040 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 27.781814753999999, 2024-09-21T15:01:30.040 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.026724381, 2024-09-21T15:01:30.040 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:30.040 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:30.040 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:30.040 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:30.040 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:30.040 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139967 2024-09-21T15:01:30.041 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.041 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:30.041 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.041 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:30.041 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229124+0000", 2024-09-21T15:01:30.041 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:30.041 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.041 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.041 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:30.041 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229124+0000", 2024-09-21T15:01:30.041 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:30.041 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.041 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.041 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:30.041 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229123+0000", 2024-09-21T15:01:30.041 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:30.042 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.042 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.042 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:30.042 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229126+0000", 2024-09-21T15:01:30.042 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4779999999999998e-06 2024-09-21T15:01:30.042 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.042 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.042 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:30.042 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229131+0000", 2024-09-21T15:01:30.042 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.729e-06 2024-09-21T15:01:30.042 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.042 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.042 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:30.042 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229135+0000", 2024-09-21T15:01:30.042 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5859999999999998e-06 2024-09-21T15:01:30.043 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.043 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.043 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:30.043 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229558+0000", 2024-09-21T15:01:30.043 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00042262499999999999 2024-09-21T15:01:30.043 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.043 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.043 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:30.043 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229569+0000", 2024-09-21T15:01:30.043 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0985000000000001e-05 2024-09-21T15:01:30.043 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.043 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.043 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:30.043 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255509+0000", 2024-09-21T15:01:30.043 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.025940246 2024-09-21T15:01:30.043 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.044 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.044 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:30.044 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255662+0000", 2024-09-21T15:01:30.044 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015324199999999999 2024-09-21T15:01:30.044 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.044 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.044 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:30.044 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255848+0000", 2024-09-21T15:01:30.044 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00018583599999999999 2024-09-21T15:01:30.044 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:30.044 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:30.044 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:30.044 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.044 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.044 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140010 3.e 3:735c06f2:::rbd_data.113664780c85.0000000000000157:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:30.044 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.231531+0000", 2024-09-21T15:01:30.045 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 27.779407889000002, 2024-09-21T15:01:30.045 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0244773149999999, 2024-09-21T15:01:30.045 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:30.045 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:30.045 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:30.045 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:30.045 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:30.045 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140010 2024-09-21T15:01:30.045 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.045 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:30.045 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.045 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:30.045 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231531+0000", 2024-09-21T15:01:30.045 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:30.045 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.046 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.046 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:30.046 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231531+0000", 2024-09-21T15:01:30.046 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:30.046 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.046 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.046 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:30.046 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231530+0000", 2024-09-21T15:01:30.046 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:30.046 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.046 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.046 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:30.046 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231535+0000", 2024-09-21T15:01:30.046 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3970000000000002e-06 2024-09-21T15:01:30.047 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.047 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.047 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:30.047 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231544+0000", 2024-09-21T15:01:30.047 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5909999999999996e-06 2024-09-21T15:01:30.047 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.047 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.047 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:30.047 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231551+0000", 2024-09-21T15:01:30.047 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7129999999999999e-06 2024-09-21T15:01:30.047 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.047 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.047 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:30.048 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231632+0000", 2024-09-21T15:01:30.048 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1181999999999996e-05 2024-09-21T15:01:30.048 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.048 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.048 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:30.048 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231640+0000", 2024-09-21T15:01:30.048 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.551e-06 2024-09-21T15:01:30.048 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.048 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.048 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:30.048 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255884+0000", 2024-09-21T15:01:30.048 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.024244057 2024-09-21T15:01:30.048 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.048 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.048 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:30.049 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256008+0000", 2024-09-21T15:01:30.049 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012351699999999999 2024-09-21T15:01:30.049 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:30.049 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:30.049 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:30.049 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.049 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.049 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140043 3.e 3:74a37859:::rbd_data.113664780c85.0000000000000086:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:30.049 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.232623+0000", 2024-09-21T15:01:30.049 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 27.778315415000002, 2024-09-21T15:01:30.049 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.023518634, 2024-09-21T15:01:30.049 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:30.049 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:30.049 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:30.049 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:30.050 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:30.050 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140043 2024-09-21T15:01:30.050 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.050 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:30.050 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.050 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:30.050 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.232623+0000", 2024-09-21T15:01:30.050 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:30.050 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.050 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.050 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:30.050 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.232623+0000", 2024-09-21T15:01:30.050 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:30.051 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.051 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.051 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:30.051 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.232623+0000", 2024-09-21T15:01:30.051 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:30.051 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.051 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.051 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:30.051 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.232626+0000", 2024-09-21T15:01:30.051 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0019999999999998e-06 2024-09-21T15:01:30.051 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.051 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.051 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:30.051 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.232632+0000", 2024-09-21T15:01:30.052 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3260000000000001e-06 2024-09-21T15:01:30.052 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.052 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.052 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:30.052 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.232637+0000", 2024-09-21T15:01:30.052 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9899999999999997e-06 2024-09-21T15:01:30.052 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.052 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.052 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:30.052 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.232678+0000", 2024-09-21T15:01:30.052 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.0423999999999998e-05 2024-09-21T15:01:30.052 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.052 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.052 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:30.052 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.232686+0000", 2024-09-21T15:01:30.053 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1419999999999993e-06 2024-09-21T15:01:30.053 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.053 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.053 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:30.053 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256037+0000", 2024-09-21T15:01:30.053 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.023351736 2024-09-21T15:01:30.053 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.053 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.053 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:30.053 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256142+0000", 2024-09-21T15:01:30.053 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010443 2024-09-21T15:01:30.053 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:30.053 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:30.053 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:30.053 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.054 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.054 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140063 3.e 3:771cae23:::rbd_data.113664780c85.0000000000000112:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:30.054 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.233948+0000", 2024-09-21T15:01:30.054 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 27.776990347999998, 2024-09-21T15:01:30.054 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0223114710000001, 2024-09-21T15:01:30.054 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:30.054 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:30.054 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:30.054 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:30.054 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:30.054 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140063 2024-09-21T15:01:30.054 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.054 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:30.054 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.055 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:30.055 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233948+0000", 2024-09-21T15:01:30.055 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:30.055 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.055 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.055 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:30.055 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233948+0000", 2024-09-21T15:01:30.055 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:30.055 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.055 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.055 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:30.055 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233948+0000", 2024-09-21T15:01:30.055 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:30.055 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.056 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.056 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:30.056 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233951+0000", 2024-09-21T15:01:30.056 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3380000000000001e-06 2024-09-21T15:01:30.056 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.056 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.056 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:30.056 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233958+0000", 2024-09-21T15:01:30.056 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0990000000000001e-06 2024-09-21T15:01:30.056 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.056 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.056 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:30.056 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233964+0000", 2024-09-21T15:01:30.056 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.852e-06 2024-09-21T15:01:30.056 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.057 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.057 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:30.057 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.234056+0000", 2024-09-21T15:01:30.057 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2392000000000006e-05 2024-09-21T15:01:30.057 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.057 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.057 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:30.057 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.234066+0000", 2024-09-21T15:01:30.057 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1360000000000003e-06 2024-09-21T15:01:30.057 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.057 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.057 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:30.057 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256169+0000", 2024-09-21T15:01:30.057 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0221031949999999 2024-09-21T15:01:30.057 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.057 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.058 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:30.058 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256260+0000", 2024-09-21T15:01:30.058 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0939000000000005e-05 2024-09-21T15:01:30.058 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:30.058 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:30.058 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:30.058 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.058 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.058 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140086 3.e 3:758134b7:::rbd_data.113664780c85.00000000000001ad:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:30.058 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.235110+0000", 2024-09-21T15:01:30.058 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 27.775828649000001, 2024-09-21T15:01:30.058 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.021276217, 2024-09-21T15:01:30.058 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:30.058 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:30.058 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:30.058 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:30.059 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:30.059 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140086 2024-09-21T15:01:30.059 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.059 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:30.059 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.059 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:30.059 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.235110+0000", 2024-09-21T15:01:30.059 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:30.059 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.059 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.059 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:30.059 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.235110+0000", 2024-09-21T15:01:30.059 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:30.059 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.059 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.059 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:30.060 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.235109+0000", 2024-09-21T15:01:30.060 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:30.060 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.060 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.060 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:30.060 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.235114+0000", 2024-09-21T15:01:30.060 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1659999999999996e-06 2024-09-21T15:01:30.060 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.060 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.060 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:30.060 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.235122+0000", 2024-09-21T15:01:30.060 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5590000000000006e-06 2024-09-21T15:01:30.060 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.060 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.060 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:30.060 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.235128+0000", 2024-09-21T15:01:30.061 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8579999999999996e-06 2024-09-21T15:01:30.061 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.061 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.061 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:30.061 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.235169+0000", 2024-09-21T15:01:30.061 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1304000000000003e-05 2024-09-21T15:01:30.061 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.061 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.061 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:30.061 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.235178+0000", 2024-09-21T15:01:30.061 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8610000000000004e-06 2024-09-21T15:01:30.061 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.061 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.061 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:30.061 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256286+0000", 2024-09-21T15:01:30.061 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.021108162 2024-09-21T15:01:30.062 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.062 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.062 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:30.062 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256386+0000", 2024-09-21T15:01:30.062 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9871000000000007e-05 2024-09-21T15:01:30.062 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:30.062 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:30.062 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:30.062 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.062 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.062 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140168 3.e 3:76f4e81d:::rbd_data.113664780c85.000000000000006f:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:30.062 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.236992+0000", 2024-09-21T15:01:30.062 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 27.773946586000001, 2024-09-21T15:01:30.062 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.019513699, 2024-09-21T15:01:30.063 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:30.063 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:30.063 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:30.063 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:30.063 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:30.063 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140168 2024-09-21T15:01:30.063 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.063 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:30.063 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.063 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:30.063 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.236992+0000", 2024-09-21T15:01:30.063 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:30.063 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.063 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.063 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:30.063 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.236992+0000", 2024-09-21T15:01:30.064 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:30.064 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.064 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.064 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:30.064 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.236991+0000", 2024-09-21T15:01:30.064 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:30.064 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.064 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.064 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:30.064 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.236995+0000", 2024-09-21T15:01:30.064 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3100000000000001e-06 2024-09-21T15:01:30.064 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.064 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.064 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:30.064 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237002+0000", 2024-09-21T15:01:30.064 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9929999999999998e-06 2024-09-21T15:01:30.065 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.065 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.065 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:30.065 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237008+0000", 2024-09-21T15:01:30.065 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7559999999999996e-06 2024-09-21T15:01:30.065 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.065 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.065 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:30.065 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237050+0000", 2024-09-21T15:01:30.065 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2852999999999998e-05 2024-09-21T15:01:30.065 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.065 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.065 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:30.065 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237060+0000", 2024-09-21T15:01:30.065 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2399999999999996e-06 2024-09-21T15:01:30.065 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.066 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.066 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:30.066 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256413+0000", 2024-09-21T15:01:30.066 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0193528999999999 2024-09-21T15:01:30.066 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.066 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.066 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:30.066 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256506+0000", 2024-09-21T15:01:30.066 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.3154999999999998e-05 2024-09-21T15:01:30.066 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:30.066 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:30.066 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:30.066 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.066 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.066 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140186 3.e 3:75504345:::rbd_data.113664780c85.00000000000000ed:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:30.067 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.237050+0000", 2024-09-21T15:01:30.067 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 27.773888615000001, 2024-09-21T15:01:30.067 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0195720150000001, 2024-09-21T15:01:30.067 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:30.067 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:30.067 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:30.067 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:30.067 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:30.067 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140186 2024-09-21T15:01:30.067 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.067 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:30.067 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.067 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:30.067 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237050+0000", 2024-09-21T15:01:30.067 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:30.067 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.068 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.068 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:30.068 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237050+0000", 2024-09-21T15:01:30.068 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:30.068 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.068 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.068 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:30.068 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237049+0000", 2024-09-21T15:01:30.068 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:30.068 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.068 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.068 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:30.068 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237052+0000", 2024-09-21T15:01:30.068 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.016e-06 2024-09-21T15:01:30.068 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.068 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.069 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:30.069 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237070+0000", 2024-09-21T15:01:30.069 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7266999999999999e-05 2024-09-21T15:01:30.069 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.069 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.069 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:30.069 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237075+0000", 2024-09-21T15:01:30.069 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6910000000000002e-06 2024-09-21T15:01:30.069 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.069 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.069 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:30.069 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237232+0000", 2024-09-21T15:01:30.069 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015683599999999999 2024-09-21T15:01:30.069 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.069 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.069 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:30.070 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237237+0000", 2024-09-21T15:01:30.070 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6439999999999999e-06 2024-09-21T15:01:30.070 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.070 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.070 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:30.070 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256534+0000", 2024-09-21T15:01:30.070 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.019296736 2024-09-21T15:01:30.070 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.070 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.070 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:30.070 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256622+0000", 2024-09-21T15:01:30.070 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8386000000000004e-05 2024-09-21T15:01:30.070 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:30.070 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:30.070 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:30.071 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.071 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.071 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140197 3.e 3:71ef5b14:::rbd_data.113664780c85.0000000000000140:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:30.071 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.237251+0000", 2024-09-21T15:01:30.071 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 27.773687587000001, 2024-09-21T15:01:30.071 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.01968386, 2024-09-21T15:01:30.071 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:30.071 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:30.071 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:30.071 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:30.071 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:30.071 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140197 2024-09-21T15:01:30.071 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.071 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:30.071 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.072 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:30.072 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237251+0000", 2024-09-21T15:01:30.072 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:30.072 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.072 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.072 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:30.072 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237251+0000", 2024-09-21T15:01:30.072 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:30.072 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.072 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.072 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:30.072 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237250+0000", 2024-09-21T15:01:30.072 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:30.072 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.073 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.073 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:30.073 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237254+0000", 2024-09-21T15:01:30.073 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.0180000000000003e-06 2024-09-21T15:01:30.073 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.073 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.073 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:30.073 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237262+0000", 2024-09-21T15:01:30.073 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0669999999999992e-06 2024-09-21T15:01:30.073 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.073 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.073 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:30.073 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237269+0000", 2024-09-21T15:01:30.073 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6000000000000003e-06 2024-09-21T15:01:30.073 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.073 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.074 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:30.074 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237319+0000", 2024-09-21T15:01:30.074 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9932000000000002e-05 2024-09-21T15:01:30.074 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.074 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.074 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:30.074 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237326+0000", 2024-09-21T15:01:30.074 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5360000000000004e-06 2024-09-21T15:01:30.074 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.074 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.074 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:30.074 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256657+0000", 2024-09-21T15:01:30.074 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.019330756 2024-09-21T15:01:30.074 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.074 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.075 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:30.075 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256935+0000", 2024-09-21T15:01:30.075 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00027762099999999998 2024-09-21T15:01:30.075 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:30.075 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:30.075 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:30.075 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.075 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.075 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140201 3.e 3:732ea575:::rbd_data.113664780c85.000000000000015c:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:30.075 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.237281+0000", 2024-09-21T15:01:30.075 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 27.773657749000002, 2024-09-21T15:01:30.075 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0198284529999999, 2024-09-21T15:01:30.075 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:30.075 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:30.075 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:30.075 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:30.075 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:30.076 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140201 2024-09-21T15:01:30.076 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.076 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:30.076 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.076 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:30.076 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237281+0000", 2024-09-21T15:01:30.076 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:30.076 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.076 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.076 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:30.076 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237281+0000", 2024-09-21T15:01:30.076 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:30.076 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.076 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.076 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:30.076 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237280+0000", 2024-09-21T15:01:30.077 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:30.077 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.077 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.077 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:30.077 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237284+0000", 2024-09-21T15:01:30.077 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7400000000000002e-06 2024-09-21T15:01:30.077 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.077 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.077 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:30.077 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237292+0000", 2024-09-21T15:01:30.077 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8939999999999993e-06 2024-09-21T15:01:30.077 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.077 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.077 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:30.077 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237298+0000", 2024-09-21T15:01:30.077 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8000000000000004e-06 2024-09-21T15:01:30.078 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.078 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.078 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:30.078 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237521+0000", 2024-09-21T15:01:30.078 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00022326 2024-09-21T15:01:30.078 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.078 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.078 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:30.078 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237529+0000", 2024-09-21T15:01:30.078 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6170000000000001e-06 2024-09-21T15:01:30.078 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.078 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.078 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:30.078 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256968+0000", 2024-09-21T15:01:30.078 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.019439113 2024-09-21T15:01:30.078 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.079 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.079 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:30.079 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.257109+0000", 2024-09-21T15:01:30.079 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014148399999999999 2024-09-21T15:01:30.079 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:30.079 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:30.079 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:30.079 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.079 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.079 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140206 3.e 3:76d38029:::rbd_data.113664780c85.0000000000000179:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:30.079 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.237520+0000", 2024-09-21T15:01:30.079 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 27.773418634999999, 2024-09-21T15:01:30.079 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0210658699999999, 2024-09-21T15:01:30.079 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:30.079 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:30.079 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:30.080 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:30.080 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:30.080 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140206 2024-09-21T15:01:30.080 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.080 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:30.080 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.080 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:30.080 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237520+0000", 2024-09-21T15:01:30.080 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:30.080 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.080 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.080 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:30.080 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237520+0000", 2024-09-21T15:01:30.080 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:30.080 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.080 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.081 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:30.081 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237519+0000", 2024-09-21T15:01:30.081 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:30.081 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.081 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.081 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:30.081 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237523+0000", 2024-09-21T15:01:30.081 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.9009999999999996e-06 2024-09-21T15:01:30.081 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.081 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.081 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:30.081 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237531+0000", 2024-09-21T15:01:30.081 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7069999999999992e-06 2024-09-21T15:01:30.081 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.081 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.081 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:30.082 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237537+0000", 2024-09-21T15:01:30.082 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4559999999999997e-06 2024-09-21T15:01:30.082 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.082 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.082 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:30.082 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237583+0000", 2024-09-21T15:01:30.082 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5532999999999999e-05 2024-09-21T15:01:30.082 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.082 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.082 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:30.082 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237592+0000", 2024-09-21T15:01:30.082 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6009999999999995e-06 2024-09-21T15:01:30.082 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.082 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.082 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:30.082 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.257157+0000", 2024-09-21T15:01:30.083 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0195657279999999 2024-09-21T15:01:30.083 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.083 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.083 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:30.083 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.258586+0000", 2024-09-21T15:01:30.083 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00142847 2024-09-21T15:01:30.083 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:30.083 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:30.083 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:30.083 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.083 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.083 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140338 3.e 3:71dba096:::rbd_data.113664780c85.000000000000018b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:30.083 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.243856+0000", 2024-09-21T15:01:30.083 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 27.767082802000001, 2024-09-21T15:01:30.083 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0138226889999999, 2024-09-21T15:01:30.083 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:30.084 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:30.084 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:30.084 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:30.084 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:30.084 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140338 2024-09-21T15:01:30.084 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.084 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:30.084 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.084 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:30.084 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243856+0000", 2024-09-21T15:01:30.084 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:30.084 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.084 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.084 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:30.084 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243856+0000", 2024-09-21T15:01:30.084 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:30.085 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.085 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.085 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:30.085 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243854+0000", 2024-09-21T15:01:30.085 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:30.085 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.085 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.085 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:30.085 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243862+0000", 2024-09-21T15:01:30.085 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2570000000000001e-06 2024-09-21T15:01:30.085 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.085 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.085 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:30.085 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243877+0000", 2024-09-21T15:01:30.085 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.571e-05 2024-09-21T15:01:30.085 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.086 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.086 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:30.086 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243884+0000", 2024-09-21T15:01:30.086 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.1269999999999997e-06 2024-09-21T15:01:30.086 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.086 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.086 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:30.086 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243932+0000", 2024-09-21T15:01:30.086 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7655999999999999e-05 2024-09-21T15:01:30.086 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.086 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.086 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:30.086 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243942+0000", 2024-09-21T15:01:30.086 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6679999999999996e-06 2024-09-21T15:01:30.087 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.087 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.087 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:30.087 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.257568+0000", 2024-09-21T15:01:30.087 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0136267670000001 2024-09-21T15:01:30.087 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.087 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.087 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:30.087 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.257678+0000", 2024-09-21T15:01:30.087 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000110011 2024-09-21T15:01:30.087 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:30.087 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:30.087 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:30.087 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.087 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.087 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140345 3.e 3:76ddeb1d:::rbd_data.113664780c85.0000000000000004:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:30.088 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.243927+0000", 2024-09-21T15:01:30.088 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 27.767011218, 2024-09-21T15:01:30.088 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0140328439999999, 2024-09-21T15:01:30.088 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:30.088 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:30.088 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:30.088 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:30.088 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:30.088 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140345 2024-09-21T15:01:30.088 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.088 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:30.088 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.088 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:30.088 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243927+0000", 2024-09-21T15:01:30.088 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:30.088 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.089 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.089 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:30.089 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243927+0000", 2024-09-21T15:01:30.089 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:30.089 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.089 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.089 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:30.089 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243927+0000", 2024-09-21T15:01:30.089 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:30.089 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.089 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.089 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:30.089 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243930+0000", 2024-09-21T15:01:30.089 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1319999999999998e-06 2024-09-21T15:01:30.089 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.089 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.090 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:30.090 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243937+0000", 2024-09-21T15:01:30.090 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4330000000000001e-06 2024-09-21T15:01:30.090 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.090 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.090 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:30.090 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243942+0000", 2024-09-21T15:01:30.090 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5090000000000001e-06 2024-09-21T15:01:30.090 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.090 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.090 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:30.090 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243964+0000", 2024-09-21T15:01:30.090 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1838999999999999e-05 2024-09-21T15:01:30.090 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.090 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.091 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:30.091 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243971+0000", 2024-09-21T15:01:30.091 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.131e-06 2024-09-21T15:01:30.091 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.091 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.091 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:30.091 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.257704+0000", 2024-09-21T15:01:30.091 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0137329310000001 2024-09-21T15:01:30.091 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.091 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.091 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:30.091 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.257805+0000", 2024-09-21T15:01:30.091 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000101121 2024-09-21T15:01:30.091 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.091 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.092 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:30.092 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.257960+0000", 2024-09-21T15:01:30.092 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015518500000000001 2024-09-21T15:01:30.092 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:30.092 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:30.092 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:30.092 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.092 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.092 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140350 3.e 3:76b9b799:::rbd_data.113664780c85.0000000000000021:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:30.092 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.243989+0000", 2024-09-21T15:01:30.092 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 27.766949467, 2024-09-21T15:01:30.092 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0142022340000001, 2024-09-21T15:01:30.092 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:30.092 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:30.092 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:30.092 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:30.092 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:30.093 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140350 2024-09-21T15:01:30.093 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.093 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:30.093 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.093 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:30.093 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243989+0000", 2024-09-21T15:01:30.093 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:30.093 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.093 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.093 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:30.093 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243989+0000", 2024-09-21T15:01:30.093 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:30.093 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.093 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.093 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:30.093 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243989+0000", 2024-09-21T15:01:30.094 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:30.094 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.094 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.094 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:30.094 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243992+0000", 2024-09-21T15:01:30.094 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.236e-06 2024-09-21T15:01:30.094 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.094 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.094 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:30.094 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243998+0000", 2024-09-21T15:01:30.094 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2500000000000003e-06 2024-09-21T15:01:30.094 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.094 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.094 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:30.094 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.244003+0000", 2024-09-21T15:01:30.094 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7740000000000004e-06 2024-09-21T15:01:30.095 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.095 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.095 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:30.095 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.244047+0000", 2024-09-21T15:01:30.095 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3665999999999998e-05 2024-09-21T15:01:30.095 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.095 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.095 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:30.095 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.244055+0000", 2024-09-21T15:01:30.095 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0639999999999994e-06 2024-09-21T15:01:30.095 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.095 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.095 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:30.095 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.257985+0000", 2024-09-21T15:01:30.095 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0139306079999999 2024-09-21T15:01:30.095 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.096 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.096 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:30.096 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.258084+0000", 2024-09-21T15:01:30.096 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8610000000000006e-05 2024-09-21T15:01:30.096 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.096 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.096 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:30.096 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.258191+0000", 2024-09-21T15:01:30.096 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000107518 2024-09-21T15:01:30.096 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:30.096 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:30.096 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:30.096 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.096 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.096 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140394 3.e 3:7756a55d:::rbd_data.113664780c85.0000000000000154:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:30.097 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.245038+0000", 2024-09-21T15:01:30.097 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 27.765900250000001, 2024-09-21T15:01:30.097 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.013225182, 2024-09-21T15:01:30.097 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:30.097 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:30.097 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:30.097 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:30.097 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:30.097 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140394 2024-09-21T15:01:30.097 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.097 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:30.097 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.097 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:30.097 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.245038+0000", 2024-09-21T15:01:30.097 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:30.097 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.098 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.098 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:30.098 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.245038+0000", 2024-09-21T15:01:30.098 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:30.098 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.098 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.098 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:30.098 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.245037+0000", 2024-09-21T15:01:30.098 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:30.098 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.098 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.098 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:30.098 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.245044+0000", 2024-09-21T15:01:30.098 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3239999999999999e-06 2024-09-21T15:01:30.098 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.098 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.099 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:30.099 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.245061+0000", 2024-09-21T15:01:30.099 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7166000000000001e-05 2024-09-21T15:01:30.099 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.099 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.099 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:30.099 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.245069+0000", 2024-09-21T15:01:30.099 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0779999999999996e-06 2024-09-21T15:01:30.099 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.099 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.099 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:30.099 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.245118+0000", 2024-09-21T15:01:30.099 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8887000000000001e-05 2024-09-21T15:01:30.099 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.099 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.099 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:30.100 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.245126+0000", 2024-09-21T15:01:30.100 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0409999999999998e-06 2024-09-21T15:01:30.100 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.100 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.100 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:30.100 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.258206+0000", 2024-09-21T15:01:30.100 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.013080451 2024-09-21T15:01:30.100 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.100 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.100 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:30.100 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.258264+0000", 2024-09-21T15:01:30.100 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7294999999999999e-05 2024-09-21T15:01:30.100 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:30.100 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:30.100 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:30.100 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:30.101 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:30.101 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:30.101 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T15:01:30.104 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:01:30.100+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:01:30.205 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:01:30.206+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:01:30.267 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:30.267 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:01:30.267 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T15:01:30.267 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:30.280 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T15:01:30.325 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:01:30.326+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:01:30.385 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:30.385 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:01:30.385 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T15:01:30.386 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T15:01:30.386 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:30.397 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T15:01:30.406 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:01:30.406+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:01:30.505 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:30.505 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T15:01:30.506 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T15:01:30.506 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T15:01:30.506 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.506 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T15:01:30.506 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.205754+0000", 2024-09-21T15:01:30.506 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 207.29889544299999, 2024-09-21T15:01:30.506 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.41117094199999998, 2024-09-21T15:01:30.506 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:30.506 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:30.506 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:30.506 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T15:01:30.506 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T15:01:30.506 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1 2024-09-21T15:01:30.506 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.507 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:30.507 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.507 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:30.507 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205754+0000", 2024-09-21T15:01:30.507 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:30.507 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.507 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.507 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:30.507 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205754+0000", 2024-09-21T15:01:30.507 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:30.507 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.507 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.507 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:30.507 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205752+0000", 2024-09-21T15:01:30.508 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T15:01:30.508 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.508 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.508 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:30.508 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205765+0000", 2024-09-21T15:01:30.508 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3230999999999999e-05 2024-09-21T15:01:30.508 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.508 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.508 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:30.508 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205782+0000", 2024-09-21T15:01:30.508 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6634e-05 2024-09-21T15:01:30.508 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.508 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.508 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:30.509 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205793+0000", 2024-09-21T15:01:30.509 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1297e-05 2024-09-21T15:01:30.509 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.509 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.509 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:30.509 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205867+0000", 2024-09-21T15:01:30.509 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4269000000000003e-05 2024-09-21T15:01:30.509 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.509 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.509 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:30.509 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205877+0000", 2024-09-21T15:01:30.509 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2690000000000005e-06 2024-09-21T15:01:30.509 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.509 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.510 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:30.510 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.613700+0000", 2024-09-21T15:01:30.510 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.407823455 2024-09-21T15:01:30.510 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.510 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.510 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:30.510 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.614203+0000", 2024-09-21T15:01:30.510 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00050311500000000005 2024-09-21T15:01:30.510 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.510 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.510 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T15:01:30.510 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.614750+0000", 2024-09-21T15:01:30.510 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00054738699999999998 2024-09-21T15:01:30.510 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.511 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.511 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:01:30.511 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616367+0000", 2024-09-21T15:01:30.511 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001616634 2024-09-21T15:01:30.511 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.511 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.511 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:01:30.511 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616574+0000", 2024-09-21T15:01:30.511 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00020668499999999999 2024-09-21T15:01:30.511 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.511 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.511 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:01:30.511 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616802+0000", 2024-09-21T15:01:30.511 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00022817900000000001 2024-09-21T15:01:30.512 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.512 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.512 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:30.512 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616925+0000", 2024-09-21T15:01:30.512 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012309699999999999 2024-09-21T15:01:30.512 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:30.512 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:30.512 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:30.512 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.512 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.512 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:61214 3.7 3:e28aec10:::rbd_header.1136862f872f:head [watch reconnect cookie 139864671932080 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-21T15:01:30.512 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:18.147172+0000", 2024-09-21T15:01:30.512 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 132.3574778, 2024-09-21T15:01:30.512 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.019264956, 2024-09-21T15:01:30.512 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:30.513 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:30.513 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:30.513 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:30.513 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:30.513 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 61214 2024-09-21T15:01:30.513 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.513 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:30.513 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.513 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:30.513 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147172+0000", 2024-09-21T15:01:30.513 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:30.513 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.513 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.513 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:30.513 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147172+0000", 2024-09-21T15:01:30.513 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:30.514 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.514 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.514 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:30.514 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147168+0000", 2024-09-21T15:01:30.514 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999962 2024-09-21T15:01:30.514 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.514 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.514 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:30.514 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147188+0000", 2024-09-21T15:01:30.514 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9717000000000001e-05 2024-09-21T15:01:30.514 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.514 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.514 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:30.514 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147239+0000", 2024-09-21T15:01:30.515 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1493999999999999e-05 2024-09-21T15:01:30.515 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.515 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.515 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:30.515 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147265+0000", 2024-09-21T15:01:30.515 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5746999999999999e-05 2024-09-21T15:01:30.515 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.515 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.515 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:30.515 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147372+0000", 2024-09-21T15:01:30.515 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010681200000000001 2024-09-21T15:01:30.515 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.515 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.515 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op must wait for map", 2024-09-21T15:01:30.515 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147380+0000", 2024-09-21T15:01:30.515 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7919999999999993e-06 2024-09-21T15:01:30.516 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.516 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.516 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:30.516 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.154025+0000", 2024-09-21T15:01:30.516 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0066454909999999999 2024-09-21T15:01:30.516 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.516 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.516 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:30.516 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.154041+0000", 2024-09-21T15:01:30.516 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5894000000000001e-05 2024-09-21T15:01:30.516 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.516 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.516 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:30.516 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.164345+0000", 2024-09-21T15:01:30.516 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0103040459999999 2024-09-21T15:01:30.517 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.517 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.517 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:30.517 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.164590+0000", 2024-09-21T15:01:30.517 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00024492699999999998 2024-09-21T15:01:30.517 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.517 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.517 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:30.517 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.166437+0000", 2024-09-21T15:01:30.517 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0018466330000000001 2024-09-21T15:01:30.517 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:30.517 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:30.517 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:30.517 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.517 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.517 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:61215 3.7 3:e28aec10:::rbd_header.1136862f872f:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-21T15:01:30.518 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:18.152008+0000", 2024-09-21T15:01:30.518 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 132.35264134799999, 2024-09-21T15:01:30.518 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.015644958, 2024-09-21T15:01:30.518 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:30.518 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:30.518 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:30.518 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:30.518 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:30.518 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 61215 2024-09-21T15:01:30.518 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.518 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:30.518 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.518 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:30.518 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152008+0000", 2024-09-21T15:01:30.519 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:30.519 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.519 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.519 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:30.519 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152008+0000", 2024-09-21T15:01:30.519 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:30.519 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.519 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.519 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:30.519 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152005+0000", 2024-09-21T15:01:30.519 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-21T15:01:30.519 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.519 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.519 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:30.520 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152025+0000", 2024-09-21T15:01:30.520 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9306000000000001e-05 2024-09-21T15:01:30.520 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.520 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.520 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:30.520 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152054+0000", 2024-09-21T15:01:30.520 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9189000000000001e-05 2024-09-21T15:01:30.520 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.520 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.520 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:30.520 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152078+0000", 2024-09-21T15:01:30.520 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4317000000000001e-05 2024-09-21T15:01:30.520 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.520 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.520 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:30.520 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152243+0000", 2024-09-21T15:01:30.521 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000164802 2024-09-21T15:01:30.521 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.521 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.521 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting_for_map not empty", 2024-09-21T15:01:30.521 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152251+0000", 2024-09-21T15:01:30.521 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0600000000000008e-06 2024-09-21T15:01:30.521 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.521 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.521 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:30.521 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.154741+0000", 2024-09-21T15:01:30.521 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0024900069999999998 2024-09-21T15:01:30.521 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.521 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.521 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:30.521 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.154753+0000", 2024-09-21T15:01:30.522 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1588e-05 2024-09-21T15:01:30.522 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.522 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.522 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:30.522 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.165380+0000", 2024-09-21T15:01:30.522 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0106272249999999 2024-09-21T15:01:30.522 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.522 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.522 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:30.522 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.165517+0000", 2024-09-21T15:01:30.522 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013737999999999999 2024-09-21T15:01:30.522 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.522 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.522 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 6", 2024-09-21T15:01:30.522 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.165938+0000", 2024-09-21T15:01:30.522 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00042082399999999998 2024-09-21T15:01:30.523 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.523 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.523 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:01:30.523 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.166795+0000", 2024-09-21T15:01:30.523 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00085674899999999997 2024-09-21T15:01:30.523 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.523 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.523 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:01:30.523 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.167598+0000", 2024-09-21T15:01:30.523 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00080335599999999997 2024-09-21T15:01:30.523 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.523 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.523 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:01:30.523 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.167632+0000", 2024-09-21T15:01:30.524 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4122000000000003e-05 2024-09-21T15:01:30.524 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.524 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.524 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:30.524 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.167653+0000", 2024-09-21T15:01:30.524 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0862000000000001e-05 2024-09-21T15:01:30.524 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:30.524 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:30.524 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:30.524 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.524 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.524 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:97678 3.d e122/24 3:b2f34c6d:::rbd_data.1136bfc79a70.0000000000000006:head v 122'234, mlcod=120'232)", 2024-09-21T15:01:30.524 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.321655+0000", 2024-09-21T15:01:30.524 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 85.182994430999997, 2024-09-21T15:01:30.525 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024509388999999999, 2024-09-21T15:01:30.525 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:30.525 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:30.525 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:30.525 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.525 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:30.525 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.321655+0000", 2024-09-21T15:01:30.525 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:30.525 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.525 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.525 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:30.525 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.321655+0000", 2024-09-21T15:01:30.525 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:30.525 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.525 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.525 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:30.526 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.321655+0000", 2024-09-21T15:01:30.526 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:30.526 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.526 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.526 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:30.526 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.329453+0000", 2024-09-21T15:01:30.526 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0077980560000000003 2024-09-21T15:01:30.526 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.526 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.526 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:30.526 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.329482+0000", 2024-09-21T15:01:30.526 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9153999999999999e-05 2024-09-21T15:01:30.526 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.526 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.526 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:30.527 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.329493+0000", 2024-09-21T15:01:30.527 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1103e-05 2024-09-21T15:01:30.527 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.527 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.527 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:30.527 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.329555+0000", 2024-09-21T15:01:30.527 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2345e-05 2024-09-21T15:01:30.527 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.527 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.527 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:30.527 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.329589+0000", 2024-09-21T15:01:30.527 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3908999999999997e-05 2024-09-21T15:01:30.527 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.527 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.527 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:01:30.527 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.346083+0000", 2024-09-21T15:01:30.528 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016493383 2024-09-21T15:01:30.528 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.528 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.528 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:30.528 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.346165+0000", 2024-09-21T15:01:30.528 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1796000000000001e-05 2024-09-21T15:01:30.528 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:30.528 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:30.528 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:30.528 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.528 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.528 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:97679 3.f e122/24 3:f2a362b4:::rbd_data.1136bfc79a70.0000000000000007:head v 122'331, mlcod=120'329)", 2024-09-21T15:01:30.528 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.337145+0000", 2024-09-21T15:01:30.528 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 85.167504421999993, 2024-09-21T15:01:30.528 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021912411, 2024-09-21T15:01:30.528 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:30.528 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:30.529 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:30.529 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.529 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:30.529 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.337145+0000", 2024-09-21T15:01:30.529 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:30.529 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.529 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.529 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:30.529 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.337145+0000", 2024-09-21T15:01:30.529 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:30.529 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.529 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.529 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:30.529 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.337145+0000", 2024-09-21T15:01:30.529 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:30.530 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.530 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.530 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:30.530 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.343899+0000", 2024-09-21T15:01:30.530 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0067541420000000003 2024-09-21T15:01:30.530 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.530 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.530 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:30.530 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.343921+0000", 2024-09-21T15:01:30.530 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2255e-05 2024-09-21T15:01:30.530 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.530 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.530 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:30.530 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.343933+0000", 2024-09-21T15:01:30.530 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1528e-05 2024-09-21T15:01:30.530 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.531 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.531 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:30.531 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.344001+0000", 2024-09-21T15:01:30.531 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8269999999999995e-05 2024-09-21T15:01:30.531 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.531 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.531 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:30.531 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.344029+0000", 2024-09-21T15:01:30.531 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8254000000000001e-05 2024-09-21T15:01:30.531 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.531 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.531 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:01:30.531 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.358901+0000", 2024-09-21T15:01:30.531 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0148719 2024-09-21T15:01:30.531 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.532 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.532 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:30.532 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.359058+0000", 2024-09-21T15:01:30.532 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000156446 2024-09-21T15:01:30.532 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:30.532 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:30.532 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:30.532 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.532 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.532 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:97676 3.1f e122/24 3:ffaa836a:::rbd_data.1136bfc79a70.0000000000000004:head v 122'312, mlcod=120'310)", 2024-09-21T15:01:30.532 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.348683+0000", 2024-09-21T15:01:30.532 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 85.155966257000003, 2024-09-21T15:01:30.532 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.033768802000000001, 2024-09-21T15:01:30.532 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:30.533 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:30.533 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:30.533 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.533 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:30.533 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.348683+0000", 2024-09-21T15:01:30.533 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:30.533 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.533 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.533 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:30.533 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.348683+0000", 2024-09-21T15:01:30.533 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:30.533 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.533 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.533 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:30.533 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.348683+0000", 2024-09-21T15:01:30.534 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:30.534 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.534 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.534 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:30.534 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.366937+0000", 2024-09-21T15:01:30.534 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018254415 2024-09-21T15:01:30.534 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.534 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.534 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:30.534 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.366959+0000", 2024-09-21T15:01:30.534 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1634e-05 2024-09-21T15:01:30.534 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.534 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.535 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:30.535 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.366971+0000", 2024-09-21T15:01:30.535 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2305e-05 2024-09-21T15:01:30.535 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.535 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.535 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:30.535 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.367034+0000", 2024-09-21T15:01:30.535 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3211000000000002e-05 2024-09-21T15:01:30.535 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.535 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.535 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:30.535 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.367056+0000", 2024-09-21T15:01:30.535 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2098000000000001e-05 2024-09-21T15:01:30.535 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.535 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.536 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:01:30.536 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.382400+0000", 2024-09-21T15:01:30.536 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.015343867000000001 2024-09-21T15:01:30.536 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.536 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.536 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:30.536 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.382452+0000", 2024-09-21T15:01:30.536 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1992e-05 2024-09-21T15:01:30.536 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:30.536 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:30.536 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:30.536 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.536 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.536 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:97734 3.19 e122/94 3:9fcf754f:::rbd_data.1136bfc79a70.0000000000000009:head v 122'314, mlcod=120'312)", 2024-09-21T15:01:30.536 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.378547+0000", 2024-09-21T15:01:30.537 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 85.126102278999994, 2024-09-21T15:01:30.537 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.063380675999999997, 2024-09-21T15:01:30.537 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:30.537 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:30.537 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:30.537 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.537 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:30.537 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.378547+0000", 2024-09-21T15:01:30.537 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:30.537 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.537 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.537 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:30.537 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.378547+0000", 2024-09-21T15:01:30.537 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:30.537 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.537 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.538 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:30.538 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.378547+0000", 2024-09-21T15:01:30.538 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:01:30.538 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.538 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.538 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:30.538 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.425118+0000", 2024-09-21T15:01:30.538 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046571166999999997 2024-09-21T15:01:30.538 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.538 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.538 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:30.538 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.425150+0000", 2024-09-21T15:01:30.538 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2150999999999998e-05 2024-09-21T15:01:30.538 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.539 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.539 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:30.539 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.425164+0000", 2024-09-21T15:01:30.539 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3302000000000001e-05 2024-09-21T15:01:30.539 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.539 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.539 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:30.539 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.425231+0000", 2024-09-21T15:01:30.539 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7106000000000004e-05 2024-09-21T15:01:30.539 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.539 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.539 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:30.539 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.425259+0000", 2024-09-21T15:01:30.539 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8087000000000001e-05 2024-09-21T15:01:30.540 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.540 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.540 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:01:30.540 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.441890+0000", 2024-09-21T15:01:30.540 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016631328000000001 2024-09-21T15:01:30.540 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.540 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.540 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:30.540 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.441928+0000", 2024-09-21T15:01:30.540 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7701999999999999e-05 2024-09-21T15:01:30.540 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:30.540 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:30.540 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:30.540 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.540 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.541 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:100187 3.10 3:08941a97:::rbd_data.1136ab4d6ff1.000000000000000e:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-21T15:01:30.541 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:10.869845+0000", 2024-09-21T15:01:30.541 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 79.634804383000002, 2024-09-21T15:01:30.541 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029345237, 2024-09-21T15:01:30.541 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:30.541 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:30.541 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:30.541 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:30.541 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:30.541 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 100187 2024-09-21T15:01:30.541 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.541 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:30.541 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.541 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:30.541 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869845+0000", 2024-09-21T15:01:30.541 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:30.542 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.542 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.542 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:30.542 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869845+0000", 2024-09-21T15:01:30.542 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:30.542 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.542 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.542 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:30.542 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869844+0000", 2024-09-21T15:01:30.542 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:30.542 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.542 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.542 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:30.542 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869853+0000", 2024-09-21T15:01:30.542 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5719999999999992e-06 2024-09-21T15:01:30.543 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.543 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.543 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:30.543 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869874+0000", 2024-09-21T15:01:30.543 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0483e-05 2024-09-21T15:01:30.543 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.543 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.543 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:30.543 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869883+0000", 2024-09-21T15:01:30.543 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6080000000000002e-06 2024-09-21T15:01:30.543 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.543 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.543 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:30.543 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869991+0000", 2024-09-21T15:01:30.544 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000108013 2024-09-21T15:01:30.544 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.544 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.544 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:30.544 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.870258+0000", 2024-09-21T15:01:30.544 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00026629099999999999 2024-09-21T15:01:30.544 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.544 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.544 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T15:01:30.544 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.878656+0000", 2024-09-21T15:01:30.544 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0083989000000000008 2024-09-21T15:01:30.544 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.544 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.544 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:01:30.545 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.899081+0000", 2024-09-21T15:01:30.545 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.020424437 2024-09-21T15:01:30.545 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.545 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.545 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:01:30.545 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.899125+0000", 2024-09-21T15:01:30.545 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4440999999999997e-05 2024-09-21T15:01:30.545 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.545 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.545 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:01:30.545 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.899160+0000", 2024-09-21T15:01:30.545 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4518999999999999e-05 2024-09-21T15:01:30.546 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.546 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.546 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:30.546 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.899190+0000", 2024-09-21T15:01:30.546 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0574999999999998e-05 2024-09-21T15:01:30.546 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:30.546 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:30.546 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:30.546 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.546 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.546 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:100263 3.10 3:0ece142e:::rbd_data.1136ab4d6ff1.0000000000000008:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-21T15:01:30.546 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:11.052257+0000", 2024-09-21T15:01:30.546 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 79.452392325000005, 2024-09-21T15:01:30.546 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042678025000000001, 2024-09-21T15:01:30.546 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:30.547 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:30.547 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:30.547 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:30.547 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:30.547 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 100263 2024-09-21T15:01:30.547 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.547 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:30.547 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.547 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:30.547 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052257+0000", 2024-09-21T15:01:30.547 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:30.547 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.547 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.547 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:30.548 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052257+0000", 2024-09-21T15:01:30.548 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:30.548 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.548 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.548 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:30.548 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052256+0000", 2024-09-21T15:01:30.548 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:30.548 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.548 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.548 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:30.548 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052263+0000", 2024-09-21T15:01:30.548 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4799999999999998e-06 2024-09-21T15:01:30.548 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.548 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.548 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:30.548 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052275+0000", 2024-09-21T15:01:30.549 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2386e-05 2024-09-21T15:01:30.549 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.549 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.549 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:30.549 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052281+0000", 2024-09-21T15:01:30.549 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6699999999999999e-06 2024-09-21T15:01:30.549 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.549 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.549 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:30.549 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052350+0000", 2024-09-21T15:01:30.549 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9099999999999999e-05 2024-09-21T15:01:30.549 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.549 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.549 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:30.549 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052456+0000", 2024-09-21T15:01:30.550 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000106669 2024-09-21T15:01:30.550 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.550 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.550 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T15:01:30.550 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.059694+0000", 2024-09-21T15:01:30.550 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.007237566 2024-09-21T15:01:30.550 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.550 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.550 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:01:30.550 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.094721+0000", 2024-09-21T15:01:30.550 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035026977000000001 2024-09-21T15:01:30.550 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.550 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.550 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:01:30.551 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.094746+0000", 2024-09-21T15:01:30.551 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5329000000000001e-05 2024-09-21T15:01:30.551 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.551 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.551 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:01:30.551 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.094850+0000", 2024-09-21T15:01:30.551 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000103452 2024-09-21T15:01:30.551 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.551 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.551 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:30.551 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.094935+0000", 2024-09-21T15:01:30.551 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5550000000000006e-05 2024-09-21T15:01:30.551 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:30.551 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:30.551 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:30.551 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.552 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.552 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:134708 3.a 3:51eaadc0:::rbd_data.11365ed419e3.0000000000000007:67 [read 221184~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:30.552 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.938945+0000", 2024-09-21T15:01:30.552 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 36.565705047000002, 2024-09-21T15:01:30.552 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021073784000000002, 2024-09-21T15:01:30.552 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:30.552 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:30.552 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:30.552 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:30.552 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:30.552 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 134708 2024-09-21T15:01:30.552 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.552 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:30.552 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.552 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:30.552 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938945+0000", 2024-09-21T15:01:30.553 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:30.553 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.553 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.553 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:30.553 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938945+0000", 2024-09-21T15:01:30.553 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:30.553 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.553 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.553 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:30.553 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938944+0000", 2024-09-21T15:01:30.553 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:30.553 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.553 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.553 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:30.553 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938946+0000", 2024-09-21T15:01:30.554 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.616e-06 2024-09-21T15:01:30.554 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.554 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.554 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:30.554 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938952+0000", 2024-09-21T15:01:30.554 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7309999999999998e-06 2024-09-21T15:01:30.554 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.554 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.554 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:30.554 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938957+0000", 2024-09-21T15:01:30.554 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8579999999999999e-06 2024-09-21T15:01:30.554 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.554 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.554 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:30.555 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.959730+0000", 2024-09-21T15:01:30.555 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.020773183000000001 2024-09-21T15:01:30.555 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.555 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.555 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:30.555 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.959810+0000", 2024-09-21T15:01:30.555 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0158000000000001e-05 2024-09-21T15:01:30.555 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.555 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.555 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:30.555 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.960018+0000", 2024-09-21T15:01:30.555 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000207951 2024-09-21T15:01:30.555 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:30.555 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:30.555 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:30.556 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.556 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.556 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:134709 3.a 3:51eaadc0:::rbd_data.11365ed419e3.0000000000000007:67 [read 237568~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:30.556 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.938996+0000", 2024-09-21T15:01:30.556 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 36.565654055000003, 2024-09-21T15:01:30.556 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021382577, 2024-09-21T15:01:30.556 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:30.556 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:30.556 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:30.556 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:30.556 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:30.556 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 134709 2024-09-21T15:01:30.556 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.556 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:30.556 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.557 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:30.557 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938996+0000", 2024-09-21T15:01:30.557 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:30.557 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.557 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.557 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:30.557 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938996+0000", 2024-09-21T15:01:30.557 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:30.557 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.557 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.557 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:30.557 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938995+0000", 2024-09-21T15:01:30.557 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:30.557 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.557 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.558 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:30.558 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938999+0000", 2024-09-21T15:01:30.558 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7979999999999999e-06 2024-09-21T15:01:30.558 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.558 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.558 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:30.558 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939006+0000", 2024-09-21T15:01:30.558 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6669999999999996e-06 2024-09-21T15:01:30.558 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.558 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.558 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:30.558 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939012+0000", 2024-09-21T15:01:30.558 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9329999999999998e-06 2024-09-21T15:01:30.558 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.559 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.559 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:30.559 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.960080+0000", 2024-09-21T15:01:30.559 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021068303 2024-09-21T15:01:30.559 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.559 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.559 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:30.559 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.960154+0000", 2024-09-21T15:01:30.559 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3819000000000006e-05 2024-09-21T15:01:30.559 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.559 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.559 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:30.559 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.960378+0000", 2024-09-21T15:01:30.559 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00022385800000000001 2024-09-21T15:01:30.560 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:30.560 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:30.560 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:30.560 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.560 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.560 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:134710 3.a 3:51eaadc0:::rbd_data.11365ed419e3.0000000000000007:67 [read 253952~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:30.560 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.939017+0000", 2024-09-21T15:01:30.560 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 36.565633044000002, 2024-09-21T15:01:30.560 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021721436, 2024-09-21T15:01:30.560 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:30.560 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:30.560 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:30.560 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:30.560 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:30.560 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 134710 2024-09-21T15:01:30.561 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.561 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:30.561 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.561 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:30.561 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939017+0000", 2024-09-21T15:01:30.561 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:30.561 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.561 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.561 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:30.561 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939017+0000", 2024-09-21T15:01:30.561 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:30.561 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.561 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.561 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:30.562 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939016+0000", 2024-09-21T15:01:30.562 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:30.562 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.562 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.562 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:30.562 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939019+0000", 2024-09-21T15:01:30.562 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.554e-06 2024-09-21T15:01:30.562 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.562 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.562 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:30.562 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939024+0000", 2024-09-21T15:01:30.562 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1569999999999999e-06 2024-09-21T15:01:30.562 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.562 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.562 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:30.563 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939034+0000", 2024-09-21T15:01:30.563 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9219999999999992e-06 2024-09-21T15:01:30.563 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.563 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.563 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:30.563 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.960408+0000", 2024-09-21T15:01:30.563 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021373792999999999 2024-09-21T15:01:30.563 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.563 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.563 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:30.563 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.960482+0000", 2024-09-21T15:01:30.563 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4331000000000004e-05 2024-09-21T15:01:30.563 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.563 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.564 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:30.564 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.960738+0000", 2024-09-21T15:01:30.564 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00025600500000000001 2024-09-21T15:01:30.564 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:30.564 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:30.564 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:30.564 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.564 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.564 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:134711 3.a 3:51eaadc0:::rbd_data.11365ed419e3.0000000000000007:67 [read 270336~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:30.564 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.939039+0000", 2024-09-21T15:01:30.564 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 36.565610995, 2024-09-21T15:01:30.564 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022064698000000001, 2024-09-21T15:01:30.564 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:30.564 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:30.564 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:30.565 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:30.565 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:30.565 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 134711 2024-09-21T15:01:30.565 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.565 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:30.565 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.565 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:30.565 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939039+0000", 2024-09-21T15:01:30.565 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:30.565 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.565 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.565 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:30.565 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939039+0000", 2024-09-21T15:01:30.565 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:30.565 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.566 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.566 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:30.566 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939038+0000", 2024-09-21T15:01:30.566 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:30.566 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.566 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.566 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:30.566 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939041+0000", 2024-09-21T15:01:30.566 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9440000000000001e-06 2024-09-21T15:01:30.566 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.566 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.566 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:30.566 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939047+0000", 2024-09-21T15:01:30.566 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1260000000000003e-06 2024-09-21T15:01:30.566 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.567 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.567 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:30.567 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939052+0000", 2024-09-21T15:01:30.567 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1619999999999999e-06 2024-09-21T15:01:30.567 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.567 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.567 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:30.567 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.960768+0000", 2024-09-21T15:01:30.567 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021715140000000001 2024-09-21T15:01:30.567 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.567 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.567 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:30.567 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.960842+0000", 2024-09-21T15:01:30.567 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4698999999999997e-05 2024-09-21T15:01:30.567 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.568 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.568 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:30.568 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.961103+0000", 2024-09-21T15:01:30.568 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00026098499999999998 2024-09-21T15:01:30.568 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:30.568 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:30.568 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:30.568 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.568 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.568 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:134712 3.a 3:51eaadc0:::rbd_data.11365ed419e3.0000000000000007:67 [read 286720~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:30.568 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.939057+0000", 2024-09-21T15:01:30.568 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 36.565592461999998, 2024-09-21T15:01:30.568 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022387067, 2024-09-21T15:01:30.568 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:30.568 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:30.568 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:30.569 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:30.569 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:30.569 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 134712 2024-09-21T15:01:30.569 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.569 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:30.569 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.569 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:30.569 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939057+0000", 2024-09-21T15:01:30.569 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:30.569 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.569 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.569 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:30.569 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939057+0000", 2024-09-21T15:01:30.569 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:30.569 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.569 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.570 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:30.570 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939057+0000", 2024-09-21T15:01:30.570 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:30.570 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.570 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.570 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:30.570 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939059+0000", 2024-09-21T15:01:30.570 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.447e-06 2024-09-21T15:01:30.570 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.570 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.570 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:30.570 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939065+0000", 2024-09-21T15:01:30.570 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4670000000000003e-06 2024-09-21T15:01:30.570 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.571 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.571 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:30.571 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939069+0000", 2024-09-21T15:01:30.571 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2119999999999997e-06 2024-09-21T15:01:30.571 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.571 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.571 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:30.571 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.961141+0000", 2024-09-21T15:01:30.571 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022072268999999999 2024-09-21T15:01:30.571 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.571 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.571 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:30.571 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.961240+0000", 2024-09-21T15:01:30.571 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9147999999999993e-05 2024-09-21T15:01:30.572 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.572 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.572 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:30.572 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.961444+0000", 2024-09-21T15:01:30.572 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00020381899999999999 2024-09-21T15:01:30.572 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:30.572 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:30.572 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:30.572 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.572 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.572 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:134713 3.a 3:51eaadc0:::rbd_data.11365ed419e3.0000000000000007:67 [read 303104~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:30.572 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.939073+0000", 2024-09-21T15:01:30.572 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 36.565576559, 2024-09-21T15:01:30.572 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022687394999999999, 2024-09-21T15:01:30.573 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:30.573 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:30.573 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:30.573 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:30.573 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:30.573 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 134713 2024-09-21T15:01:30.573 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.573 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:30.573 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.573 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:30.573 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939073+0000", 2024-09-21T15:01:30.573 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:30.573 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.573 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.573 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:30.574 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939073+0000", 2024-09-21T15:01:30.574 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:30.574 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.574 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.574 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:30.574 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939073+0000", 2024-09-21T15:01:30.574 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:30.574 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.574 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.574 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:30.574 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939075+0000", 2024-09-21T15:01:30.574 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5969999999999999e-06 2024-09-21T15:01:30.574 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.575 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.575 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:30.575 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939081+0000", 2024-09-21T15:01:30.575 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.609e-06 2024-09-21T15:01:30.575 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.575 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.575 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:30.575 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939086+0000", 2024-09-21T15:01:30.575 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.916e-06 2024-09-21T15:01:30.575 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.575 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.575 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:30.575 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.961474+0000", 2024-09-21T15:01:30.575 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022388512999999999 2024-09-21T15:01:30.576 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.576 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.576 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:30.576 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.961567+0000", 2024-09-21T15:01:30.576 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2267000000000003e-05 2024-09-21T15:01:30.576 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.576 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.576 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:30.576 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.961760+0000", 2024-09-21T15:01:30.576 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00019375999999999999 2024-09-21T15:01:30.576 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:30.576 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:30.576 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:30.576 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.576 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.576 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:134714 3.a 3:51eaadc0:::rbd_data.11365ed419e3.0000000000000007:67 [read 319488~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:30.577 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.939090+0000", 2024-09-21T15:01:30.577 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 36.565559731999997, 2024-09-21T15:01:30.577 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023043403, 2024-09-21T15:01:30.577 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:30.577 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:30.577 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:30.577 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:30.577 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:30.577 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 134714 2024-09-21T15:01:30.577 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.577 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:30.577 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.577 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:30.577 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939090+0000", 2024-09-21T15:01:30.577 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:30.578 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.578 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.578 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:30.578 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939090+0000", 2024-09-21T15:01:30.578 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:30.578 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.578 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.578 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:30.578 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939090+0000", 2024-09-21T15:01:30.578 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:30.578 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.578 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.578 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:30.578 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939092+0000", 2024-09-21T15:01:30.579 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2790000000000001e-06 2024-09-21T15:01:30.579 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.579 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.579 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:30.579 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939097+0000", 2024-09-21T15:01:30.579 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1410000000000004e-06 2024-09-21T15:01:30.579 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.579 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.579 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:30.579 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939102+0000", 2024-09-21T15:01:30.579 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7369999999999997e-06 2024-09-21T15:01:30.579 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.579 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.579 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:30.579 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.961791+0000", 2024-09-21T15:01:30.579 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022688974000000001 2024-09-21T15:01:30.580 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.580 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.580 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:30.580 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.961865+0000", 2024-09-21T15:01:30.580 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4651000000000001e-05 2024-09-21T15:01:30.580 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.580 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.580 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:30.580 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.962133+0000", 2024-09-21T15:01:30.580 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00026793999999999999 2024-09-21T15:01:30.580 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:30.580 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:30.580 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:30.580 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.580 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.580 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:134715 3.a 3:51eaadc0:::rbd_data.11365ed419e3.0000000000000007:67 [read 335872~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:30.581 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.939106+0000", 2024-09-21T15:01:30.581 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 36.565543122000001, 2024-09-21T15:01:30.581 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023401142999999999, 2024-09-21T15:01:30.581 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:30.581 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:30.581 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:30.581 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:30.581 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:30.581 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 134715 2024-09-21T15:01:30.581 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.581 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:30.581 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.581 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:30.581 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939106+0000", 2024-09-21T15:01:30.581 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:30.581 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.581 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.582 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:30.582 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939106+0000", 2024-09-21T15:01:30.582 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:30.582 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.582 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.582 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:30.582 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939106+0000", 2024-09-21T15:01:30.582 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:30.582 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.582 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.582 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:30.582 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939114+0000", 2024-09-21T15:01:30.582 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1009999999999992e-06 2024-09-21T15:01:30.582 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.583 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.583 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:30.583 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939121+0000", 2024-09-21T15:01:30.583 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7710000000000001e-06 2024-09-21T15:01:30.583 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.583 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.583 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:30.583 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939126+0000", 2024-09-21T15:01:30.583 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3600000000000004e-06 2024-09-21T15:01:30.583 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.583 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.583 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:30.583 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.962162+0000", 2024-09-21T15:01:30.583 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023036148999999999 2024-09-21T15:01:30.583 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.584 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.584 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:30.584 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.962239+0000", 2024-09-21T15:01:30.584 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6874999999999998e-05 2024-09-21T15:01:30.584 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.584 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.584 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:30.584 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.962508+0000", 2024-09-21T15:01:30.584 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00026842199999999999 2024-09-21T15:01:30.584 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:30.584 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:30.584 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:30.584 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.584 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.584 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:134716 3.a 3:51eaadc0:::rbd_data.11365ed419e3.0000000000000007:67 [read 352256~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:30.585 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.939131+0000", 2024-09-21T15:01:30.585 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 36.565518988000001, 2024-09-21T15:01:30.585 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023742441, 2024-09-21T15:01:30.585 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:30.585 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:30.585 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:30.585 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:30.585 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:30.585 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 134716 2024-09-21T15:01:30.585 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.585 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:30.585 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.585 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:30.585 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939131+0000", 2024-09-21T15:01:30.585 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:30.586 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.586 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.586 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:30.586 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939131+0000", 2024-09-21T15:01:30.586 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:30.586 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.586 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.586 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:30.586 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939130+0000", 2024-09-21T15:01:30.586 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:30.586 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.586 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.586 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:30.586 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939133+0000", 2024-09-21T15:01:30.586 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5620000000000002e-06 2024-09-21T15:01:30.587 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.587 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.587 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:30.587 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939138+0000", 2024-09-21T15:01:30.587 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.322e-06 2024-09-21T15:01:30.587 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.587 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.587 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:30.587 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939143+0000", 2024-09-21T15:01:30.587 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5510000000000001e-06 2024-09-21T15:01:30.587 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.587 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.587 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:30.587 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.962537+0000", 2024-09-21T15:01:30.587 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023394808 2024-09-21T15:01:30.588 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.588 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.588 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:30.588 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.962608+0000", 2024-09-21T15:01:30.588 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0699999999999997e-05 2024-09-21T15:01:30.588 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.588 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.588 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:30.588 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.962873+0000", 2024-09-21T15:01:30.588 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00026487 2024-09-21T15:01:30.588 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:30.588 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:30.588 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:30.588 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.588 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.589 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:135024 3.a 3:53261383:::rbd_data.11365ed419e3.0000000000000037:snapdir [list-snaps] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:30.589 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.940162+0000", 2024-09-21T15:01:30.589 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 36.564488025000003, 2024-09-21T15:01:30.589 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022852172, 2024-09-21T15:01:30.589 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:30.589 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:30.589 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:30.589 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:30.589 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:30.589 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 135024 2024-09-21T15:01:30.589 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.589 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:30.589 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.589 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:30.589 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940162+0000", 2024-09-21T15:01:30.589 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:30.590 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.590 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.590 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:30.590 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940162+0000", 2024-09-21T15:01:30.590 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:30.590 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.590 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.590 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:30.590 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940161+0000", 2024-09-21T15:01:30.590 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:30.590 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.590 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.590 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:30.590 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940164+0000", 2024-09-21T15:01:30.591 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4099999999999998e-06 2024-09-21T15:01:30.591 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.591 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.591 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:30.591 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940169+0000", 2024-09-21T15:01:30.591 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3720000000000004e-06 2024-09-21T15:01:30.591 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.591 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.591 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:30.591 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940174+0000", 2024-09-21T15:01:30.591 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.724e-06 2024-09-21T15:01:30.591 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.591 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.591 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:30.591 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.962902+0000", 2024-09-21T15:01:30.591 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022728719000000001 2024-09-21T15:01:30.592 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.592 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.592 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:30.592 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.963014+0000", 2024-09-21T15:01:30.592 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000111245 2024-09-21T15:01:30.592 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:30.592 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:30.592 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:30.592 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.592 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.592 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:135044 3.a 3:53261383:::rbd_data.11365ed419e3.0000000000000037:snapdir [list-snaps] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:30.592 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.940484+0000", 2024-09-21T15:01:30.592 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 36.564165293000002, 2024-09-21T15:01:30.592 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022619169000000001, 2024-09-21T15:01:30.592 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:30.592 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:30.593 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:30.593 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:30.593 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:30.593 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 135044 2024-09-21T15:01:30.593 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.593 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:30.593 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.593 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:30.593 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940484+0000", 2024-09-21T15:01:30.593 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:30.593 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.593 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.593 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:30.593 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940484+0000", 2024-09-21T15:01:30.593 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:30.593 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.594 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.594 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:30.594 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940484+0000", 2024-09-21T15:01:30.594 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:30.594 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.594 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.594 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:30.594 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940487+0000", 2024-09-21T15:01:30.594 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8449999999999999e-06 2024-09-21T15:01:30.594 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.594 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.594 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:30.594 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940493+0000", 2024-09-21T15:01:30.594 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9599999999999997e-06 2024-09-21T15:01:30.594 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.595 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.595 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:30.595 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940497+0000", 2024-09-21T15:01:30.595 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9620000000000001e-06 2024-09-21T15:01:30.595 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.595 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.595 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:30.595 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.963034+0000", 2024-09-21T15:01:30.595 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022536878 2024-09-21T15:01:30.595 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.595 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.595 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:30.595 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.963103+0000", 2024-09-21T15:01:30.595 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9073999999999996e-05 2024-09-21T15:01:30.596 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:30.596 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:30.596 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:30.596 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:30.596 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:30.596 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:30.597 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T15:01:30.597 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:01:30.594+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:01:30.698 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:01:30.698+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:01:30.762 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:30.762 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:01:30.762 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T15:01:30.762 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:30.772 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T15:01:30.799 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:01:30.796+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:01:30.876 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:30.876 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:01:30.876 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T15:01:30.876 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T15:01:30.876 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:30.886 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T15:01:30.900 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:01:30.898+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:01:30.993 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:30.994 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T15:01:30.994 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T15:01:30.994 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T15:01:30.994 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.994 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:15130 3.2 3:47f1bbdb:::rbd_data.11361bed1f78.0000000000000067:head [sparse-read 0~147456] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-21T15:01:30.994 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:40.288284+0000", 2024-09-21T15:01:30.994 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 170.70358878600001, 2024-09-21T15:01:30.994 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1142935039999999, 2024-09-21T15:01:30.994 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:30.994 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:30.994 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:30.994 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:30.995 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:30.995 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 15130 2024-09-21T15:01:30.995 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.995 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:30.995 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.995 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:30.995 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288284+0000", 2024-09-21T15:01:30.995 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:30.995 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.995 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.995 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:30.995 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288284+0000", 2024-09-21T15:01:30.995 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:30.995 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.995 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.996 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:30.996 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288284+0000", 2024-09-21T15:01:30.996 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:30.996 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.996 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.996 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:30.996 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288288+0000", 2024-09-21T15:01:30.996 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5829999999999998e-06 2024-09-21T15:01:30.996 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.996 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.996 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:30.996 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288297+0000", 2024-09-21T15:01:30.996 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0669999999999998e-06 2024-09-21T15:01:30.996 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.996 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.996 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:30.997 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288305+0000", 2024-09-21T15:01:30.997 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8860000000000004e-06 2024-09-21T15:01:30.997 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.997 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.997 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:30.997 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288417+0000", 2024-09-21T15:01:30.997 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000112952 2024-09-21T15:01:30.997 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.997 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.997 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:30.997 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288428+0000", 2024-09-21T15:01:30.997 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0631e-05 2024-09-21T15:01:30.997 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.997 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.997 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:30.997 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.401876+0000", 2024-09-21T15:01:30.998 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1134479610000001 2024-09-21T15:01:30.998 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.998 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.998 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:30.998 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.402076+0000", 2024-09-21T15:01:30.998 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00019944200000000001 2024-09-21T15:01:30.998 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.998 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.998 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:30.998 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.402578+0000", 2024-09-21T15:01:30.998 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00050242899999999998 2024-09-21T15:01:30.998 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:30.998 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:30.998 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:30.998 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.999 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:30.999 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45708 3.2 3:415db91a:::rbd_data.1136ecbe5879.0000000000000000:28 [read 140288~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:30.999 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141801+0000", 2024-09-21T15:01:30.999 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 140.850072592, 2024-09-21T15:01:30.999 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034274200999999997, 2024-09-21T15:01:30.999 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:30.999 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:30.999 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:30.999 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:30.999 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:30.999 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45708 2024-09-21T15:01:30.999 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:30.999 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:30.999 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.000 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:31.000 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141801+0000", 2024-09-21T15:01:31.000 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:31.000 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.000 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.000 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:31.000 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141801+0000", 2024-09-21T15:01:31.000 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:31.000 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.000 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.000 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:31.000 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141800+0000", 2024-09-21T15:01:31.000 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:31.000 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.001 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.001 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:31.001 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141803+0000", 2024-09-21T15:01:31.001 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4210000000000002e-06 2024-09-21T15:01:31.001 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.001 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.001 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:31.001 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141812+0000", 2024-09-21T15:01:31.001 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.9800000000000004e-06 2024-09-21T15:01:31.001 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.001 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.001 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:31.001 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141816+0000", 2024-09-21T15:01:31.002 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4769999999999997e-06 2024-09-21T15:01:31.002 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.002 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.002 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:31.002 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.175865+0000", 2024-09-21T15:01:31.002 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034048875999999999 2024-09-21T15:01:31.002 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.002 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.002 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:31.002 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.175943+0000", 2024-09-21T15:01:31.002 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7464000000000002e-05 2024-09-21T15:01:31.002 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.002 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.002 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:31.003 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176075+0000", 2024-09-21T15:01:31.003 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013230800000000001 2024-09-21T15:01:31.003 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:31.003 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:31.003 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:31.003 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.003 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.003 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45709 3.2 3:415db91a:::rbd_data.1136ecbe5879.0000000000000000:28 [read 143360~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:31.003 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141820+0000", 2024-09-21T15:01:31.003 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 140.850052745, 2024-09-21T15:01:31.003 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034537536000000001, 2024-09-21T15:01:31.003 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:31.003 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:31.004 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:31.004 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:31.004 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:31.004 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45709 2024-09-21T15:01:31.004 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.004 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:31.004 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.004 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:31.004 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141820+0000", 2024-09-21T15:01:31.004 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:31.004 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.004 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.004 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:31.005 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141820+0000", 2024-09-21T15:01:31.005 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:31.005 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.005 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.005 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:31.005 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141820+0000", 2024-09-21T15:01:31.005 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:31.005 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.005 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.005 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:31.005 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141823+0000", 2024-09-21T15:01:31.005 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5110000000000002e-06 2024-09-21T15:01:31.005 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.005 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.006 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:31.006 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141829+0000", 2024-09-21T15:01:31.006 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0850000000000002e-06 2024-09-21T15:01:31.006 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.006 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.006 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:31.006 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141833+0000", 2024-09-21T15:01:31.006 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7620000000000003e-06 2024-09-21T15:01:31.006 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.006 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.006 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:31.006 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176103+0000", 2024-09-21T15:01:31.006 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034269933000000002 2024-09-21T15:01:31.006 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.007 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.007 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:31.007 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176161+0000", 2024-09-21T15:01:31.007 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7658999999999998e-05 2024-09-21T15:01:31.007 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.007 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.007 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:31.007 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176358+0000", 2024-09-21T15:01:31.007 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000196937 2024-09-21T15:01:31.007 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:31.007 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:31.007 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:31.007 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.007 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.008 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45710 3.2 3:415db91a:::rbd_data.1136ecbe5879.0000000000000000:28 [read 146432~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:31.008 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141838+0000", 2024-09-21T15:01:31.008 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 140.850035582, 2024-09-21T15:01:31.008 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034953674999999997, 2024-09-21T15:01:31.008 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:31.008 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:31.008 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:31.008 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:31.008 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:31.008 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45710 2024-09-21T15:01:31.008 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.008 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:31.008 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.008 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:31.008 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141838+0000", 2024-09-21T15:01:31.009 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:31.009 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.009 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.009 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:31.009 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141838+0000", 2024-09-21T15:01:31.009 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:31.009 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.009 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.009 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:31.009 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141837+0000", 2024-09-21T15:01:31.009 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:31.009 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.009 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.009 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:31.009 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141843+0000", 2024-09-21T15:01:31.010 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1619999999999996e-06 2024-09-21T15:01:31.010 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.010 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.010 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:31.010 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141850+0000", 2024-09-21T15:01:31.010 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4799999999999998e-06 2024-09-21T15:01:31.010 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.010 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.010 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:31.010 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141853+0000", 2024-09-21T15:01:31.010 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3050000000000001e-06 2024-09-21T15:01:31.010 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.010 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.010 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:31.010 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176490+0000", 2024-09-21T15:01:31.011 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034636413999999997 2024-09-21T15:01:31.011 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.011 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.011 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:31.011 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176606+0000", 2024-09-21T15:01:31.011 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000115841 2024-09-21T15:01:31.011 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.011 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.011 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:31.011 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176791+0000", 2024-09-21T15:01:31.011 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00018581500000000001 2024-09-21T15:01:31.011 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:31.011 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:31.011 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:31.011 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.012 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.012 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45711 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:31.012 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141856+0000", 2024-09-21T15:01:31.012 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 140.850017245, 2024-09-21T15:01:31.012 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034986308000000001, 2024-09-21T15:01:31.012 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:31.012 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:31.012 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:31.012 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:31.012 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:31.012 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45711 2024-09-21T15:01:31.012 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.012 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:31.012 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.012 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:31.013 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141856+0000", 2024-09-21T15:01:31.013 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:31.013 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.013 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.013 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:31.013 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141856+0000", 2024-09-21T15:01:31.013 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:31.013 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.013 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.013 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:31.013 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141856+0000", 2024-09-21T15:01:31.013 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:31.013 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.013 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.013 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:31.014 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141862+0000", 2024-09-21T15:01:31.014 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.567e-06 2024-09-21T15:01:31.014 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.014 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.014 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:31.014 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141868+0000", 2024-09-21T15:01:31.014 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9569999999999999e-06 2024-09-21T15:01:31.014 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.014 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.014 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:31.014 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141871+0000", 2024-09-21T15:01:31.014 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.146e-06 2024-09-21T15:01:31.014 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.014 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.014 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:31.014 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176831+0000", 2024-09-21T15:01:31.015 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034959935999999997 2024-09-21T15:01:31.015 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.015 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.015 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:31.015 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176842+0000", 2024-09-21T15:01:31.015 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1008e-05 2024-09-21T15:01:31.015 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:31.015 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:31.015 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:31.015 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.015 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.015 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45712 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:31.015 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141874+0000", 2024-09-21T15:01:31.015 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 140.84999934999999, 2024-09-21T15:01:31.015 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034993511999999997, 2024-09-21T15:01:31.015 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:31.016 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:31.016 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:31.016 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:31.016 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:31.016 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45712 2024-09-21T15:01:31.016 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.016 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:31.016 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.016 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:31.016 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141874+0000", 2024-09-21T15:01:31.016 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:31.016 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.016 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.016 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:31.016 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141874+0000", 2024-09-21T15:01:31.016 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:31.017 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.017 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.017 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:31.017 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141874+0000", 2024-09-21T15:01:31.017 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:31.017 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.017 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.017 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:31.017 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141875+0000", 2024-09-21T15:01:31.017 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5430000000000001e-06 2024-09-21T15:01:31.017 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.017 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.017 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:31.017 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141878+0000", 2024-09-21T15:01:31.017 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1420000000000002e-06 2024-09-21T15:01:31.017 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.018 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.018 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:31.018 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141881+0000", 2024-09-21T15:01:31.018 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.807e-06 2024-09-21T15:01:31.018 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.018 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.018 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:31.018 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176860+0000", 2024-09-21T15:01:31.018 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034979243 2024-09-21T15:01:31.018 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.018 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.018 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:31.018 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176867+0000", 2024-09-21T15:01:31.018 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0169999999999999e-06 2024-09-21T15:01:31.018 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:31.018 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:31.019 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:31.019 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.019 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.019 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45713 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:31.019 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141883+0000", 2024-09-21T15:01:31.019 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 140.84998977500001, 2024-09-21T15:01:31.019 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035010516999999998, 2024-09-21T15:01:31.019 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:31.019 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:31.019 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:31.019 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:31.019 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:31.019 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45713 2024-09-21T15:01:31.019 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.019 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:31.020 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.020 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:31.020 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141883+0000", 2024-09-21T15:01:31.020 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:31.020 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.020 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.020 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:31.020 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141883+0000", 2024-09-21T15:01:31.020 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:31.020 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.020 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.020 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:31.020 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141883+0000", 2024-09-21T15:01:31.020 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:31.021 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.021 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.021 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:31.021 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141885+0000", 2024-09-21T15:01:31.021 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4649999999999999e-06 2024-09-21T15:01:31.021 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.021 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.021 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:31.021 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141888+0000", 2024-09-21T15:01:31.021 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3890000000000001e-06 2024-09-21T15:01:31.021 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.021 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.021 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:31.021 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141891+0000", 2024-09-21T15:01:31.021 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9239999999999999e-06 2024-09-21T15:01:31.021 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.022 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.022 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:31.022 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176887+0000", 2024-09-21T15:01:31.022 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034996370999999998 2024-09-21T15:01:31.022 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.022 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.022 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:31.022 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176894+0000", 2024-09-21T15:01:31.022 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6069999999999996e-06 2024-09-21T15:01:31.022 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:31.022 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:31.022 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:31.022 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.022 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.022 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45714 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:31.022 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141893+0000", 2024-09-21T15:01:31.023 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 140.84997985999999, 2024-09-21T15:01:31.023 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035048215000000001, 2024-09-21T15:01:31.023 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:31.023 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:31.023 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:31.023 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:31.023 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:31.023 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45714 2024-09-21T15:01:31.023 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.023 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:31.023 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.023 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:31.023 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141893+0000", 2024-09-21T15:01:31.023 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:31.023 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.023 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.024 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:31.024 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141893+0000", 2024-09-21T15:01:31.024 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:31.024 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.024 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.024 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:31.024 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141893+0000", 2024-09-21T15:01:31.024 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:31.024 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.024 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.024 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:31.024 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141895+0000", 2024-09-21T15:01:31.024 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.703e-06 2024-09-21T15:01:31.024 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.024 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.025 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:31.025 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141900+0000", 2024-09-21T15:01:31.025 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.242e-06 2024-09-21T15:01:31.025 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.025 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.025 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:31.025 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141903+0000", 2024-09-21T15:01:31.025 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.117e-06 2024-09-21T15:01:31.025 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.025 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.025 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:31.025 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176930+0000", 2024-09-21T15:01:31.025 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035026750000000002 2024-09-21T15:01:31.025 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.025 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.025 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:31.026 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176942+0000", 2024-09-21T15:01:31.026 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1668000000000001e-05 2024-09-21T15:01:31.026 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:31.026 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:31.026 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:31.026 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.026 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.026 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45715 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:31.026 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141906+0000", 2024-09-21T15:01:31.026 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 140.84996770199999, 2024-09-21T15:01:31.026 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035066057999999997, 2024-09-21T15:01:31.026 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:31.026 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:31.026 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:31.026 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:31.027 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:31.027 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45715 2024-09-21T15:01:31.027 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.027 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:31.027 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.027 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:31.027 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141906+0000", 2024-09-21T15:01:31.027 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:31.027 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.027 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.027 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:31.027 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141906+0000", 2024-09-21T15:01:31.027 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:31.027 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.027 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.028 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:31.028 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141905+0000", 2024-09-21T15:01:31.028 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:31.028 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.028 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.028 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:31.028 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141907+0000", 2024-09-21T15:01:31.028 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5400000000000001e-06 2024-09-21T15:01:31.028 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.028 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.028 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:31.028 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141912+0000", 2024-09-21T15:01:31.028 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.502e-06 2024-09-21T15:01:31.028 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.028 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.028 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:31.028 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141915+0000", 2024-09-21T15:01:31.029 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0170000000000001e-06 2024-09-21T15:01:31.029 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.029 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.029 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:31.029 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176964+0000", 2024-09-21T15:01:31.029 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035048587999999999 2024-09-21T15:01:31.029 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.029 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.029 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:31.029 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176972+0000", 2024-09-21T15:01:31.029 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6580000000000002e-06 2024-09-21T15:01:31.029 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:31.029 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:31.029 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:31.029 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.029 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.030 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45716 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:31.030 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141918+0000", 2024-09-21T15:01:31.030 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 140.849955526, 2024-09-21T15:01:31.030 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035086223, 2024-09-21T15:01:31.030 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:31.030 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:31.030 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:31.030 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:31.030 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:31.030 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45716 2024-09-21T15:01:31.030 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.030 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:31.030 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.030 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:31.030 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141918+0000", 2024-09-21T15:01:31.030 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:31.031 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.031 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.031 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:31.031 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141918+0000", 2024-09-21T15:01:31.031 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:31.031 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.031 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.031 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:31.031 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141917+0000", 2024-09-21T15:01:31.031 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:31.031 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.031 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.031 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:31.031 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141919+0000", 2024-09-21T15:01:31.031 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5680000000000001e-06 2024-09-21T15:01:31.031 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.032 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.032 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:31.032 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141922+0000", 2024-09-21T15:01:31.032 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2789999999999999e-06 2024-09-21T15:01:31.032 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.032 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.032 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:31.032 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141925+0000", 2024-09-21T15:01:31.032 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9469999999999999e-06 2024-09-21T15:01:31.032 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.032 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.032 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:31.032 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176997+0000", 2024-09-21T15:01:31.032 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035071366999999999 2024-09-21T15:01:31.032 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.033 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.033 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:31.033 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177004+0000", 2024-09-21T15:01:31.033 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3309999999999998e-06 2024-09-21T15:01:31.033 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:31.033 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:31.033 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:31.033 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.033 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.033 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45717 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:31.033 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141928+0000", 2024-09-21T15:01:31.033 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 140.84994558, 2024-09-21T15:01:31.033 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035102573999999998, 2024-09-21T15:01:31.034 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:31.034 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:31.034 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:31.034 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:31.034 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:31.034 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45717 2024-09-21T15:01:31.034 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.034 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:31.034 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.034 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:31.034 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141928+0000", 2024-09-21T15:01:31.034 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:31.034 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.034 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.035 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:31.035 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141928+0000", 2024-09-21T15:01:31.035 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:31.035 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.035 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.035 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:31.035 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141927+0000", 2024-09-21T15:01:31.035 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:01:31.035 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.035 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.035 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:31.035 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141929+0000", 2024-09-21T15:01:31.035 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.533e-06 2024-09-21T15:01:31.035 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.035 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.035 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:31.036 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141934+0000", 2024-09-21T15:01:31.036 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1660000000000002e-06 2024-09-21T15:01:31.036 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.036 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.036 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:31.036 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141937+0000", 2024-09-21T15:01:31.036 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0309999999999999e-06 2024-09-21T15:01:31.036 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.036 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.036 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:31.036 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177023+0000", 2024-09-21T15:01:31.036 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035085595999999997 2024-09-21T15:01:31.036 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.036 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.036 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:31.036 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177030+0000", 2024-09-21T15:01:31.037 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.486e-06 2024-09-21T15:01:31.037 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:31.037 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:31.037 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:31.037 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.037 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.037 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45718 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:31.037 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141940+0000", 2024-09-21T15:01:31.037 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 140.849933722, 2024-09-21T15:01:31.037 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035124956999999998, 2024-09-21T15:01:31.037 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:31.037 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:31.037 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:31.037 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:31.037 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:31.037 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45718 2024-09-21T15:01:31.038 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.038 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:31.038 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.038 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:31.038 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141940+0000", 2024-09-21T15:01:31.038 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:31.038 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.038 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.038 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:31.038 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141940+0000", 2024-09-21T15:01:31.038 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:31.038 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.038 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.038 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:31.038 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141939+0000", 2024-09-21T15:01:31.038 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:31.039 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.039 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.039 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:31.039 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141941+0000", 2024-09-21T15:01:31.039 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.502e-06 2024-09-21T15:01:31.039 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.039 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.039 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:31.039 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141946+0000", 2024-09-21T15:01:31.039 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3449999999999996e-06 2024-09-21T15:01:31.039 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.039 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.039 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:31.039 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141949+0000", 2024-09-21T15:01:31.039 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9749999999999999e-06 2024-09-21T15:01:31.039 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.040 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.040 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:31.040 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177057+0000", 2024-09-21T15:01:31.040 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035108385999999998 2024-09-21T15:01:31.040 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.040 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.040 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:31.040 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177064+0000", 2024-09-21T15:01:31.040 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0099999999999998e-06 2024-09-21T15:01:31.040 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:31.040 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:31.040 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:31.040 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.040 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.041 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45719 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:31.041 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141952+0000", 2024-09-21T15:01:31.041 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 140.84992144899999, 2024-09-21T15:01:31.041 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035139108000000002, 2024-09-21T15:01:31.041 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:31.041 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:31.041 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:31.041 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:31.041 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:31.041 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45719 2024-09-21T15:01:31.041 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.041 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:31.041 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.041 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:31.042 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141952+0000", 2024-09-21T15:01:31.042 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:31.042 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.042 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.042 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:31.042 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141952+0000", 2024-09-21T15:01:31.042 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:31.042 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.042 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.042 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:31.042 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141952+0000", 2024-09-21T15:01:31.043 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:31.043 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.043 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.043 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:31.043 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141953+0000", 2024-09-21T15:01:31.043 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.486e-06 2024-09-21T15:01:31.043 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.043 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.043 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:31.043 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141959+0000", 2024-09-21T15:01:31.043 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4940000000000003e-06 2024-09-21T15:01:31.043 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.043 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.043 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:31.043 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141963+0000", 2024-09-21T15:01:31.044 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3540000000000002e-06 2024-09-21T15:01:31.044 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.044 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.044 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:31.044 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177084+0000", 2024-09-21T15:01:31.044 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035120983000000001 2024-09-21T15:01:31.044 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.044 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.044 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:31.044 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177091+0000", 2024-09-21T15:01:31.044 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0350000000000004e-06 2024-09-21T15:01:31.044 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:31.044 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:31.044 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:31.045 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.045 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.045 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45720 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:31.045 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141965+0000", 2024-09-21T15:01:31.045 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 140.84990798199999, 2024-09-21T15:01:31.045 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035155268000000003, 2024-09-21T15:01:31.045 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:31.045 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:31.045 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:31.045 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:31.045 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:31.045 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45720 2024-09-21T15:01:31.046 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.046 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:31.046 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.046 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:31.046 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141965+0000", 2024-09-21T15:01:31.046 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:31.046 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.046 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.046 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:31.046 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141965+0000", 2024-09-21T15:01:31.046 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:31.046 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.046 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.046 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:31.046 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141965+0000", 2024-09-21T15:01:31.046 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:31.047 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.047 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.047 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:31.047 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141966+0000", 2024-09-21T15:01:31.047 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.46e-06 2024-09-21T15:01:31.047 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.047 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.047 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:31.047 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141972+0000", 2024-09-21T15:01:31.047 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4210000000000003e-06 2024-09-21T15:01:31.047 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.047 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.047 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:31.047 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141975+0000", 2024-09-21T15:01:31.048 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.948e-06 2024-09-21T15:01:31.048 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.048 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.048 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:31.048 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177113+0000", 2024-09-21T15:01:31.048 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035137855000000003 2024-09-21T15:01:31.048 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.048 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.048 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:31.048 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177121+0000", 2024-09-21T15:01:31.048 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8269999999999998e-06 2024-09-21T15:01:31.049 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:31.049 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:31.049 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:31.049 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.049 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.049 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45721 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:31.049 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141977+0000", 2024-09-21T15:01:31.049 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 140.84989595900001, 2024-09-21T15:01:31.049 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035169984000000001, 2024-09-21T15:01:31.049 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:31.049 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:31.049 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:31.049 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:31.049 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:31.050 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45721 2024-09-21T15:01:31.050 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.050 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:31.050 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.050 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:31.050 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141977+0000", 2024-09-21T15:01:31.050 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:31.050 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.050 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.050 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:31.050 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141977+0000", 2024-09-21T15:01:31.050 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:31.050 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.050 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.051 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:31.051 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141977+0000", 2024-09-21T15:01:31.051 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:31.051 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.051 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.051 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:31.051 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141979+0000", 2024-09-21T15:01:31.051 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.511e-06 2024-09-21T15:01:31.051 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.051 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.051 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:31.051 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141982+0000", 2024-09-21T15:01:31.051 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4180000000000001e-06 2024-09-21T15:01:31.051 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.052 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.052 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:31.052 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141985+0000", 2024-09-21T15:01:31.052 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9450000000000002e-06 2024-09-21T15:01:31.052 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.052 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.052 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:31.052 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177139+0000", 2024-09-21T15:01:31.052 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035154409999999997 2024-09-21T15:01:31.052 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.052 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.052 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:31.052 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177147+0000", 2024-09-21T15:01:31.052 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9570000000000002e-06 2024-09-21T15:01:31.053 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:31.053 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:31.053 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:31.053 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.053 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.053 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45722 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:31.053 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141987+0000", 2024-09-21T15:01:31.053 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 140.84988598800001, 2024-09-21T15:01:31.053 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035188492000000002, 2024-09-21T15:01:31.053 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:31.053 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:31.053 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:31.053 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:31.053 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:31.053 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45722 2024-09-21T15:01:31.054 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.054 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:31.054 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.054 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:31.054 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141987+0000", 2024-09-21T15:01:31.054 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:31.054 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.054 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.054 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:31.054 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141987+0000", 2024-09-21T15:01:31.054 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:31.054 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.055 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.055 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:31.055 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141987+0000", 2024-09-21T15:01:31.055 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:31.055 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.055 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.055 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:31.055 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141988+0000", 2024-09-21T15:01:31.055 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.463e-06 2024-09-21T15:01:31.055 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.055 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.055 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:31.055 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141994+0000", 2024-09-21T15:01:31.055 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.361e-06 2024-09-21T15:01:31.055 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.056 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.056 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:31.056 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141997+0000", 2024-09-21T15:01:31.056 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0359999999999998e-06 2024-09-21T15:01:31.056 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.056 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.056 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:31.056 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177169+0000", 2024-09-21T15:01:31.056 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035171651999999998 2024-09-21T15:01:31.056 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.056 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.056 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:31.056 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177176+0000", 2024-09-21T15:01:31.056 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2189999999999998e-06 2024-09-21T15:01:31.057 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:31.057 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:31.057 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:31.057 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.057 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.057 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45723 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:31.057 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141999+0000", 2024-09-21T15:01:31.057 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 140.84987382099999, 2024-09-21T15:01:31.057 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035197701999999997, 2024-09-21T15:01:31.057 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:31.057 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:31.057 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:31.057 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:31.058 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:31.058 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45723 2024-09-21T15:01:31.058 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.058 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:31.058 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.058 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:31.058 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141999+0000", 2024-09-21T15:01:31.058 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:31.058 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.058 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.058 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:31.058 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141999+0000", 2024-09-21T15:01:31.059 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:31.059 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.059 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.059 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:31.059 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141999+0000", 2024-09-21T15:01:31.059 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:31.059 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.059 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.059 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:31.059 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142003+0000", 2024-09-21T15:01:31.059 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5049999999999999e-06 2024-09-21T15:01:31.059 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.060 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.060 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:31.060 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142008+0000", 2024-09-21T15:01:31.060 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6579999999999999e-06 2024-09-21T15:01:31.060 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.060 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.060 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:31.060 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142011+0000", 2024-09-21T15:01:31.060 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0529999999999998e-06 2024-09-21T15:01:31.060 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.060 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.061 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:31.061 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177194+0000", 2024-09-21T15:01:31.061 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035182221999999999 2024-09-21T15:01:31.061 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.061 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.061 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:31.061 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177197+0000", 2024-09-21T15:01:31.061 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5030000000000002e-06 2024-09-21T15:01:31.061 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:31.061 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:31.061 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:31.061 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.061 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.061 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45724 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:31.062 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.142015+0000", 2024-09-21T15:01:31.062 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 140.84985785399999, 2024-09-21T15:01:31.062 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035195345000000003, 2024-09-21T15:01:31.062 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:31.062 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:31.062 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:31.062 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:31.062 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:31.062 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45724 2024-09-21T15:01:31.062 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.062 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:31.062 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.062 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:31.062 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142015+0000", 2024-09-21T15:01:31.063 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:31.063 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.063 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.063 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:31.063 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142015+0000", 2024-09-21T15:01:31.063 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:31.063 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.063 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.063 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:31.063 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142015+0000", 2024-09-21T15:01:31.063 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:31.063 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.063 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.063 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:31.063 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142017+0000", 2024-09-21T15:01:31.064 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5540000000000001e-06 2024-09-21T15:01:31.064 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.064 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.064 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:31.064 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142022+0000", 2024-09-21T15:01:31.064 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4229999999999996e-06 2024-09-21T15:01:31.064 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.064 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.064 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:31.064 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142025+0000", 2024-09-21T15:01:31.064 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.056e-06 2024-09-21T15:01:31.064 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.064 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.065 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:31.065 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177207+0000", 2024-09-21T15:01:31.065 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035182089 2024-09-21T15:01:31.065 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.065 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.065 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:31.065 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177211+0000", 2024-09-21T15:01:31.065 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4649999999999999e-06 2024-09-21T15:01:31.065 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:31.065 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:31.065 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:31.065 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.065 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.066 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45725 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:31.066 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.142028+0000", 2024-09-21T15:01:31.066 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 140.84984557499999, 2024-09-21T15:01:31.066 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035195731000000001, 2024-09-21T15:01:31.066 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:31.066 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:31.066 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:31.066 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:31.066 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:31.066 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45725 2024-09-21T15:01:31.066 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.066 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:31.066 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.066 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:31.067 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142028+0000", 2024-09-21T15:01:31.067 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:31.067 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.067 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.067 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:31.067 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142028+0000", 2024-09-21T15:01:31.067 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:31.067 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.067 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.067 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:31.067 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142027+0000", 2024-09-21T15:01:31.067 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:31.067 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.067 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.067 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:31.068 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142029+0000", 2024-09-21T15:01:31.068 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5269999999999999e-06 2024-09-21T15:01:31.068 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.068 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.068 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:31.068 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142032+0000", 2024-09-21T15:01:31.068 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3249999999999999e-06 2024-09-21T15:01:31.068 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.068 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.068 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:31.068 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142035+0000", 2024-09-21T15:01:31.068 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.897e-06 2024-09-21T15:01:31.068 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.068 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.068 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:31.069 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177220+0000", 2024-09-21T15:01:31.069 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035184871 2024-09-21T15:01:31.069 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.069 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.069 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:31.069 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177223+0000", 2024-09-21T15:01:31.069 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3680000000000002e-06 2024-09-21T15:01:31.069 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:31.069 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:31.069 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:31.069 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.069 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.069 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45726 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:31.069 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.142037+0000", 2024-09-21T15:01:31.069 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 140.84983586000001, 2024-09-21T15:01:31.070 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035199273000000003, 2024-09-21T15:01:31.070 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:31.070 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:31.070 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:31.070 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:31.070 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:31.070 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45726 2024-09-21T15:01:31.070 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.070 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:31.070 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.070 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:31.070 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142037+0000", 2024-09-21T15:01:31.070 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:31.070 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.070 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.070 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:31.071 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142037+0000", 2024-09-21T15:01:31.071 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:31.071 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.071 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.071 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:31.071 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142037+0000", 2024-09-21T15:01:31.071 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:31.071 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.071 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.071 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:31.071 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142038+0000", 2024-09-21T15:01:31.071 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.344e-06 2024-09-21T15:01:31.071 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.071 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.071 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:31.071 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142044+0000", 2024-09-21T15:01:31.072 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1930000000000001e-06 2024-09-21T15:01:31.072 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.072 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.072 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:31.072 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142047+0000", 2024-09-21T15:01:31.072 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0280000000000001e-06 2024-09-21T15:01:31.072 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.072 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.072 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:31.072 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177233+0000", 2024-09-21T15:01:31.072 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035186394000000003 2024-09-21T15:01:31.072 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.072 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.072 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:31.072 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177237+0000", 2024-09-21T15:01:31.072 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5530000000000001e-06 2024-09-21T15:01:31.072 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:31.073 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:31.073 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:31.073 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:31.073 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:31.073 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:31.074 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T15:01:31.075 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:01:31.074+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:01:31.175 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:01:31.172+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:01:31.195 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:31.195 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:01:31.195 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T15:01:31.195 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:31.206 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T15:01:31.275 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:01:31.274+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:01:31.314 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:31.314 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:01:31.314 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T15:01:31.314 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T15:01:31.314 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:31.323 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T15:01:31.376 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:01:31.374+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:01:31.414 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T15:01:31.477 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:01:31.472+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:01:31.497 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:31.497 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T15:01:31.497 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T15:01:31.497 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T15:01:31.497 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.497 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139776 3.16 3:6c38d0e2:::rbd_data.113664780c85.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:31.497 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.223442+0000", 2024-09-21T15:01:31.497 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 29.272389450999999, 2024-09-21T15:01:31.497 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0315488150000001, 2024-09-21T15:01:31.497 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:31.497 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:31.497 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:31.497 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:31.497 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:31.498 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139776 2024-09-21T15:01:31.498 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.498 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:31.498 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.498 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:31.498 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223442+0000", 2024-09-21T15:01:31.498 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:31.498 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.498 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.498 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:31.498 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223442+0000", 2024-09-21T15:01:31.498 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:31.498 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.499 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.499 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:31.499 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223442+0000", 2024-09-21T15:01:31.499 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:31.499 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.499 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.499 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:31.499 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223446+0000", 2024-09-21T15:01:31.499 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2549999999999999e-06 2024-09-21T15:01:31.499 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.499 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.499 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:31.499 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223454+0000", 2024-09-21T15:01:31.499 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8199999999999997e-06 2024-09-21T15:01:31.499 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.500 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.500 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:31.500 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223460+0000", 2024-09-21T15:01:31.500 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6330000000000001e-06 2024-09-21T15:01:31.500 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.500 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.500 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:31.500 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223542+0000", 2024-09-21T15:01:31.500 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1929999999999997e-05 2024-09-21T15:01:31.500 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.500 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.500 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:31.500 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223552+0000", 2024-09-21T15:01:31.501 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0627e-05 2024-09-21T15:01:31.501 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.501 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.501 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:31.501 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.254634+0000", 2024-09-21T15:01:31.501 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.031081972 2024-09-21T15:01:31.501 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.501 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.501 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:31.501 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.254796+0000", 2024-09-21T15:01:31.501 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000161879 2024-09-21T15:01:31.501 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.501 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.501 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:31.502 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.254991+0000", 2024-09-21T15:01:31.502 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00019526499999999999 2024-09-21T15:01:31.502 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:31.502 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:31.502 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:31.502 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.502 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.502 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139785 3.11 3:8abbb109:::rbd_data.113664780c85.0000000000000070:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:31.502 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.223531+0000", 2024-09-21T15:01:31.502 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 29.27230071, 2024-09-21T15:01:31.502 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0187143510000001, 2024-09-21T15:01:31.502 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:31.502 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:31.503 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:31.503 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:31.503 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:31.503 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139785 2024-09-21T15:01:31.503 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.503 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:31.503 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.503 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:31.503 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223531+0000", 2024-09-21T15:01:31.503 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:31.503 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.503 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.503 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:31.503 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223531+0000", 2024-09-21T15:01:31.504 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:31.504 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.504 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.504 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:31.504 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223531+0000", 2024-09-21T15:01:31.504 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:31.504 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.504 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.504 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:31.504 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223534+0000", 2024-09-21T15:01:31.504 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2650000000000001e-06 2024-09-21T15:01:31.504 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.504 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.504 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:31.504 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223541+0000", 2024-09-21T15:01:31.505 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8990000000000004e-06 2024-09-21T15:01:31.505 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.505 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.505 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:31.505 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223546+0000", 2024-09-21T15:01:31.505 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3399999999999997e-06 2024-09-21T15:01:31.505 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.505 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.505 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:31.505 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223724+0000", 2024-09-21T15:01:31.505 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017745799999999999 2024-09-21T15:01:31.505 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.505 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.505 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:31.506 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223735+0000", 2024-09-21T15:01:31.506 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1610999999999999e-05 2024-09-21T15:01:31.506 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.506 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.506 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:31.506 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242113+0000", 2024-09-21T15:01:31.506 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.018377879 2024-09-21T15:01:31.506 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.506 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.506 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:31.506 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242246+0000", 2024-09-21T15:01:31.506 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013231500000000001 2024-09-21T15:01:31.506 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:31.506 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:31.506 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:31.506 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.507 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.507 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139790 3.11 3:8a18c7d8:::rbd_data.113664780c85.000000000000008d:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:31.507 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.223578+0000", 2024-09-21T15:01:31.507 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 29.272253715000002, 2024-09-21T15:01:31.507 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0187879719999999, 2024-09-21T15:01:31.507 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:31.507 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:31.507 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:31.507 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:31.507 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:31.507 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139790 2024-09-21T15:01:31.507 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.507 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:31.507 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.507 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:31.508 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223578+0000", 2024-09-21T15:01:31.508 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:31.508 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.508 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.508 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:31.508 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223578+0000", 2024-09-21T15:01:31.508 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:31.508 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.508 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.508 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:31.508 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223578+0000", 2024-09-21T15:01:31.508 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:31.508 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.508 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.509 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:31.509 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223581+0000", 2024-09-21T15:01:31.509 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6879999999999999e-06 2024-09-21T15:01:31.509 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.509 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.509 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:31.509 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223586+0000", 2024-09-21T15:01:31.509 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7080000000000002e-06 2024-09-21T15:01:31.509 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.509 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.509 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:31.509 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223591+0000", 2024-09-21T15:01:31.509 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5639999999999998e-06 2024-09-21T15:01:31.509 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.509 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.509 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:31.510 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223761+0000", 2024-09-21T15:01:31.510 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000170213 2024-09-21T15:01:31.510 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.510 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.510 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:31.510 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223769+0000", 2024-09-21T15:01:31.510 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7049999999999999e-06 2024-09-21T15:01:31.510 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.510 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.510 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:31.510 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242269+0000", 2024-09-21T15:01:31.510 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.018500288 2024-09-21T15:01:31.511 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.511 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.511 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:31.511 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242366+0000", 2024-09-21T15:01:31.511 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.7126000000000005e-05 2024-09-21T15:01:31.511 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:31.511 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:31.511 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:31.511 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.511 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.511 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139792 3.16 3:6ce5caf3:::rbd_data.113664780c85.000000000000009b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:31.511 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.223595+0000", 2024-09-21T15:01:31.511 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 29.272236761999999, 2024-09-21T15:01:31.511 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0317517899999999, 2024-09-21T15:01:31.512 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:31.512 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:31.512 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:31.512 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:31.512 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:31.512 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139792 2024-09-21T15:01:31.512 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.512 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:31.512 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.512 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:31.512 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223595+0000", 2024-09-21T15:01:31.512 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:31.512 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.512 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.512 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:31.513 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223595+0000", 2024-09-21T15:01:31.513 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:31.513 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.513 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.513 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:31.513 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223595+0000", 2024-09-21T15:01:31.513 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:31.513 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.513 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.513 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:31.513 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223598+0000", 2024-09-21T15:01:31.513 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4620000000000001e-06 2024-09-21T15:01:31.513 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.514 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.514 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:31.514 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223604+0000", 2024-09-21T15:01:31.514 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6879999999999996e-06 2024-09-21T15:01:31.514 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.514 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.514 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:31.514 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223609+0000", 2024-09-21T15:01:31.514 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9490000000000004e-06 2024-09-21T15:01:31.514 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.514 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.514 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:31.514 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224067+0000", 2024-09-21T15:01:31.514 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00045780700000000002 2024-09-21T15:01:31.514 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.515 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.515 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:31.515 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224075+0000", 2024-09-21T15:01:31.515 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6610000000000006e-06 2024-09-21T15:01:31.515 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.515 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.515 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:31.515 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255141+0000", 2024-09-21T15:01:31.515 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0310657480000001 2024-09-21T15:01:31.515 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.515 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.515 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:31.515 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255347+0000", 2024-09-21T15:01:31.515 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000205837 2024-09-21T15:01:31.515 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:31.515 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:31.516 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:31.516 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.516 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.516 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139794 3.11 3:8cf3a2f4:::rbd_data.113664780c85.00000000000000a9:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:31.516 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.224011+0000", 2024-09-21T15:01:31.516 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 29.271820733999999, 2024-09-21T15:01:31.516 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.018507791, 2024-09-21T15:01:31.516 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:31.516 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:31.516 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:31.516 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:31.516 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:31.516 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139794 2024-09-21T15:01:31.516 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.516 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:31.516 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.517 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:31.517 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224011+0000", 2024-09-21T15:01:31.517 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:31.517 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.517 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.517 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:31.517 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224011+0000", 2024-09-21T15:01:31.517 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:31.517 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.517 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.517 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:31.517 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224010+0000", 2024-09-21T15:01:31.517 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:31.517 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.517 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.518 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:31.518 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224016+0000", 2024-09-21T15:01:31.518 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6550000000000001e-06 2024-09-21T15:01:31.518 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.518 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.518 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:31.518 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224028+0000", 2024-09-21T15:01:31.518 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2422999999999999e-05 2024-09-21T15:01:31.518 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.518 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.518 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:31.518 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224036+0000", 2024-09-21T15:01:31.518 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7749999999999993e-06 2024-09-21T15:01:31.518 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.519 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.519 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:31.519 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225176+0000", 2024-09-21T15:01:31.519 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0011401740000000001 2024-09-21T15:01:31.519 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.519 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.519 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:31.519 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225187+0000", 2024-09-21T15:01:31.519 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0951999999999999e-05 2024-09-21T15:01:31.519 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.519 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.519 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:31.519 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242392+0000", 2024-09-21T15:01:31.519 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.017204905 2024-09-21T15:01:31.519 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.519 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.519 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:31.520 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242519+0000", 2024-09-21T15:01:31.520 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012680499999999999 2024-09-21T15:01:31.520 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:31.520 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:31.520 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:31.520 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.520 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.520 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139800 3.16 3:68b050ed:::rbd_data.113664780c85.00000000000000d3:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:31.520 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.224579+0000", 2024-09-21T15:01:31.520 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 29.271252974999999, 2024-09-21T15:01:31.520 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.030932637, 2024-09-21T15:01:31.520 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:31.520 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:31.520 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:31.520 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:31.520 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:31.521 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139800 2024-09-21T15:01:31.521 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.521 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:31.521 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.521 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:31.521 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224579+0000", 2024-09-21T15:01:31.521 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:31.521 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.521 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.521 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:31.521 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224579+0000", 2024-09-21T15:01:31.521 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:31.522 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.522 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.522 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:31.522 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224578+0000", 2024-09-21T15:01:31.522 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:31.522 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.522 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.522 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:31.522 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224582+0000", 2024-09-21T15:01:31.522 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0029999999999999e-06 2024-09-21T15:01:31.522 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.522 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.522 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:31.522 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224588+0000", 2024-09-21T15:01:31.523 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8120000000000002e-06 2024-09-21T15:01:31.523 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.523 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.523 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:31.523 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224593+0000", 2024-09-21T15:01:31.523 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7469999999999996e-06 2024-09-21T15:01:31.523 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.523 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.523 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:31.523 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224648+0000", 2024-09-21T15:01:31.523 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4693e-05 2024-09-21T15:01:31.523 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.523 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.523 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:31.523 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224656+0000", 2024-09-21T15:01:31.524 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8429999999999993e-06 2024-09-21T15:01:31.524 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.524 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.524 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:31.524 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255382+0000", 2024-09-21T15:01:31.524 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0307266070000001 2024-09-21T15:01:31.524 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.524 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.524 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:31.524 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255512+0000", 2024-09-21T15:01:31.524 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000129367 2024-09-21T15:01:31.524 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:31.524 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:31.525 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:31.525 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.525 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.525 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139804 3.11 3:88f9ddce:::rbd_data.113664780c85.00000000000000ef:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:31.525 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.224632+0000", 2024-09-21T15:01:31.525 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 29.271199994, 2024-09-21T15:01:31.525 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.018021302, 2024-09-21T15:01:31.525 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:31.525 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:31.525 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:31.525 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:31.525 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:31.525 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139804 2024-09-21T15:01:31.525 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.526 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:31.526 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.526 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:31.526 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224632+0000", 2024-09-21T15:01:31.526 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:31.526 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.526 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.526 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:31.526 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224632+0000", 2024-09-21T15:01:31.526 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:31.526 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.526 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.526 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:31.526 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224632+0000", 2024-09-21T15:01:31.527 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:31.527 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.527 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.527 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:31.527 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224635+0000", 2024-09-21T15:01:31.527 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2710000000000001e-06 2024-09-21T15:01:31.527 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.527 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.527 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:31.527 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224640+0000", 2024-09-21T15:01:31.527 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6779999999999997e-06 2024-09-21T15:01:31.527 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.527 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.528 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:31.528 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224645+0000", 2024-09-21T15:01:31.528 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9339999999999997e-06 2024-09-21T15:01:31.528 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.528 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.528 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:31.528 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225307+0000", 2024-09-21T15:01:31.528 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00066201699999999999 2024-09-21T15:01:31.528 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.528 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.528 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:31.528 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225312+0000", 2024-09-21T15:01:31.528 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9799999999999998e-06 2024-09-21T15:01:31.529 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.529 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.529 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:31.529 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242548+0000", 2024-09-21T15:01:31.529 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.017235603 2024-09-21T15:01:31.529 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.529 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.529 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:31.529 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242653+0000", 2024-09-21T15:01:31.529 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010522 2024-09-21T15:01:31.529 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:31.529 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:31.529 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:31.530 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.530 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.530 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139809 3.11 3:884da946:::rbd_data.113664780c85.0000000000000118:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:31.530 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.224667+0000", 2024-09-21T15:01:31.530 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 29.271164949999999, 2024-09-21T15:01:31.530 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0181101960000001, 2024-09-21T15:01:31.530 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:31.530 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:31.530 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:31.530 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:31.530 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:31.530 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139809 2024-09-21T15:01:31.530 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.531 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:31.531 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.531 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:31.531 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224667+0000", 2024-09-21T15:01:31.531 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:31.531 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.531 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.531 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:31.531 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224667+0000", 2024-09-21T15:01:31.531 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:31.531 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.532 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.532 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:31.532 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224667+0000", 2024-09-21T15:01:31.532 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:31.532 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.532 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.532 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:31.532 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224669+0000", 2024-09-21T15:01:31.532 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.452e-06 2024-09-21T15:01:31.532 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.532 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.532 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:31.532 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224675+0000", 2024-09-21T15:01:31.532 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.818e-06 2024-09-21T15:01:31.533 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.533 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.533 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:31.533 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224680+0000", 2024-09-21T15:01:31.533 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6319999999999999e-06 2024-09-21T15:01:31.533 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.533 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.533 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:31.533 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225400+0000", 2024-09-21T15:01:31.533 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00072012699999999999 2024-09-21T15:01:31.533 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.533 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.533 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:31.533 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225409+0000", 2024-09-21T15:01:31.534 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1039999999999996e-06 2024-09-21T15:01:31.534 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.534 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.534 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:31.534 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242675+0000", 2024-09-21T15:01:31.534 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0172663319999999 2024-09-21T15:01:31.534 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.534 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.534 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:31.534 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242777+0000", 2024-09-21T15:01:31.534 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010197899999999999 2024-09-21T15:01:31.534 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:31.534 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:31.535 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:31.535 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.535 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.535 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139868 3.16 3:69f82090:::rbd_data.113664780c85.00000000000000f1:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:31.535 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.225982+0000", 2024-09-21T15:01:31.535 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 29.269849449999999, 2024-09-21T15:01:31.535 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0296791729999999, 2024-09-21T15:01:31.535 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:31.535 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:31.535 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:31.535 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:31.535 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:31.535 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139868 2024-09-21T15:01:31.535 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.536 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:31.536 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.536 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:31.536 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225982+0000", 2024-09-21T15:01:31.536 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:31.536 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.536 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.536 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:31.536 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225982+0000", 2024-09-21T15:01:31.536 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:31.536 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.536 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.536 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:31.536 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225982+0000", 2024-09-21T15:01:31.536 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:31.537 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.537 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.537 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:31.537 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225985+0000", 2024-09-21T15:01:31.537 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.74e-06 2024-09-21T15:01:31.537 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.537 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.537 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:31.537 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225990+0000", 2024-09-21T15:01:31.537 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7039999999999999e-06 2024-09-21T15:01:31.537 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.537 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.537 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:31.537 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225995+0000", 2024-09-21T15:01:31.538 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7249999999999997e-06 2024-09-21T15:01:31.538 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.538 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.538 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:31.538 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226299+0000", 2024-09-21T15:01:31.538 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00030383199999999997 2024-09-21T15:01:31.538 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.538 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.538 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:31.538 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226308+0000", 2024-09-21T15:01:31.538 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.9740000000000008e-06 2024-09-21T15:01:31.538 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.538 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.539 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:31.539 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255539+0000", 2024-09-21T15:01:31.539 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0292305960000001 2024-09-21T15:01:31.539 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.539 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.539 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:31.539 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255662+0000", 2024-09-21T15:01:31.539 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012307400000000001 2024-09-21T15:01:31.539 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:31.539 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:31.539 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:31.539 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.539 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.539 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139869 3.16 3:6b6c90a4:::rbd_data.113664780c85.00000000000000fe:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:31.539 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.226006+0000", 2024-09-21T15:01:31.540 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 29.269825833999999, 2024-09-21T15:01:31.540 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.029790518, 2024-09-21T15:01:31.540 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:31.540 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:31.540 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:31.540 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:31.540 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:31.540 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139869 2024-09-21T15:01:31.540 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.540 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:31.540 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.540 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:31.540 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226006+0000", 2024-09-21T15:01:31.540 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:31.541 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.541 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.541 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:31.541 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226006+0000", 2024-09-21T15:01:31.541 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:31.541 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.541 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.541 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:31.541 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226006+0000", 2024-09-21T15:01:31.541 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:31.541 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.541 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.541 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:31.542 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226008+0000", 2024-09-21T15:01:31.542 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5569999999999998e-06 2024-09-21T15:01:31.542 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.542 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.542 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:31.542 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226014+0000", 2024-09-21T15:01:31.542 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2079999999999997e-06 2024-09-21T15:01:31.542 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.542 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.542 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:31.542 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226020+0000", 2024-09-21T15:01:31.542 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7429999999999999e-06 2024-09-21T15:01:31.543 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.543 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.543 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:31.543 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226330+0000", 2024-09-21T15:01:31.543 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00030963300000000002 2024-09-21T15:01:31.543 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.543 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.543 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:31.543 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226336+0000", 2024-09-21T15:01:31.543 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6340000000000003e-06 2024-09-21T15:01:31.543 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.543 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.543 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:31.543 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255687+0000", 2024-09-21T15:01:31.544 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0293507150000001 2024-09-21T15:01:31.544 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.544 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.544 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:31.544 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255797+0000", 2024-09-21T15:01:31.544 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000109471 2024-09-21T15:01:31.544 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:31.544 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:31.544 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:31.544 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.544 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.544 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139884 3.16 3:69f10324:::rbd_data.113664780c85.0000000000000161:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:31.544 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.227573+0000", 2024-09-21T15:01:31.544 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 29.268259192999999, 2024-09-21T15:01:31.544 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0283568759999999, 2024-09-21T15:01:31.545 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:31.545 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:31.545 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:31.545 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:31.545 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:31.545 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139884 2024-09-21T15:01:31.545 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.545 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:31.545 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.545 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:31.545 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227573+0000", 2024-09-21T15:01:31.545 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:31.545 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.545 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.546 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:31.546 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227573+0000", 2024-09-21T15:01:31.546 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:31.546 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.546 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.546 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:31.546 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227572+0000", 2024-09-21T15:01:31.546 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:31.546 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.546 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.546 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:31.546 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227575+0000", 2024-09-21T15:01:31.546 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7769999999999998e-06 2024-09-21T15:01:31.546 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.547 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.547 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:31.547 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227581+0000", 2024-09-21T15:01:31.547 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7359999999999998e-06 2024-09-21T15:01:31.547 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.547 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.547 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:31.547 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227585+0000", 2024-09-21T15:01:31.547 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5090000000000004e-06 2024-09-21T15:01:31.547 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.547 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.547 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:31.547 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227753+0000", 2024-09-21T15:01:31.547 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00016761599999999999 2024-09-21T15:01:31.548 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.548 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.548 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:31.548 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227761+0000", 2024-09-21T15:01:31.548 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0579999999999998e-06 2024-09-21T15:01:31.548 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.548 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.548 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:31.548 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255828+0000", 2024-09-21T15:01:31.548 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0280671349999999 2024-09-21T15:01:31.548 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.548 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.548 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:31.549 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255930+0000", 2024-09-21T15:01:31.549 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000101372 2024-09-21T15:01:31.549 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:31.549 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:31.549 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:31.549 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.549 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.549 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139913 3.16 3:6f81b46d:::rbd_data.113664780c85.0000000000000074:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:31.549 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.227685+0000", 2024-09-21T15:01:31.549 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 29.268147014, 2024-09-21T15:01:31.549 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0283644750000001, 2024-09-21T15:01:31.549 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:31.549 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:31.549 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:31.550 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:31.550 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:31.550 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139913 2024-09-21T15:01:31.550 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.550 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:31.550 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.550 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:31.550 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227685+0000", 2024-09-21T15:01:31.550 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:31.550 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.550 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.550 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:31.550 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227685+0000", 2024-09-21T15:01:31.551 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:31.551 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.551 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.551 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:31.551 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227685+0000", 2024-09-21T15:01:31.551 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:31.551 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.551 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.551 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:31.551 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227687+0000", 2024-09-21T15:01:31.551 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2929999999999999e-06 2024-09-21T15:01:31.551 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.551 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.551 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:31.552 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227693+0000", 2024-09-21T15:01:31.552 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1070000000000001e-06 2024-09-21T15:01:31.552 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.552 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.552 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:31.552 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227698+0000", 2024-09-21T15:01:31.552 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9339999999999997e-06 2024-09-21T15:01:31.552 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.552 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.552 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:31.552 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227784+0000", 2024-09-21T15:01:31.553 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6096999999999999e-05 2024-09-21T15:01:31.553 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.553 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.553 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:31.553 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227791+0000", 2024-09-21T15:01:31.553 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9090000000000003e-06 2024-09-21T15:01:31.553 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.553 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.553 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:31.553 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255955+0000", 2024-09-21T15:01:31.553 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0281643579999999 2024-09-21T15:01:31.553 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.553 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.553 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:31.554 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256049+0000", 2024-09-21T15:01:31.554 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4047000000000003e-05 2024-09-21T15:01:31.554 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:31.554 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:31.554 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:31.554 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.554 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.554 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139988 3.11 3:88a29fa4:::rbd_data.113664780c85.00000000000000bd:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:31.554 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.229772+0000", 2024-09-21T15:01:31.554 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 29.266059514999998, 2024-09-21T15:01:31.554 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.017967668, 2024-09-21T15:01:31.554 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:31.554 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:31.555 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:31.555 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:31.555 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:31.555 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139988 2024-09-21T15:01:31.555 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.555 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:31.555 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.555 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:31.555 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229772+0000", 2024-09-21T15:01:31.555 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:31.555 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.555 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.555 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:31.555 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229772+0000", 2024-09-21T15:01:31.556 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:31.556 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.556 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.556 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:31.556 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229772+0000", 2024-09-21T15:01:31.556 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:31.556 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.556 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.556 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:31.556 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229776+0000", 2024-09-21T15:01:31.556 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6890000000000002e-06 2024-09-21T15:01:31.556 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.556 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.556 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:31.556 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229783+0000", 2024-09-21T15:01:31.557 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2280000000000001e-06 2024-09-21T15:01:31.557 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.557 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.557 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:31.557 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229789+0000", 2024-09-21T15:01:31.557 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1700000000000002e-06 2024-09-21T15:01:31.557 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.557 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.557 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:31.557 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.230106+0000", 2024-09-21T15:01:31.558 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00031700800000000003 2024-09-21T15:01:31.558 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.558 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.558 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:31.558 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.230115+0000", 2024-09-21T15:01:31.558 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4190000000000008e-06 2024-09-21T15:01:31.558 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.558 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.558 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:31.558 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242935+0000", 2024-09-21T15:01:31.558 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0128191449999999 2024-09-21T15:01:31.558 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.558 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.559 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for missing object", 2024-09-21T15:01:31.559 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.243078+0000", 2024-09-21T15:01:31.559 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000142984 2024-09-21T15:01:31.559 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.559 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.559 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:31.559 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.247659+0000", 2024-09-21T15:01:31.559 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0045813030000000001 2024-09-21T15:01:31.559 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.559 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.559 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:31.559 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.247740+0000", 2024-09-21T15:01:31.559 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1216000000000003e-05 2024-09-21T15:01:31.559 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:31.560 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:31.560 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:31.560 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.560 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.560 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139994 3.16 3:6b9df8d1:::rbd_data.113664780c85.00000000000000e7:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:31.560 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.231370+0000", 2024-09-21T15:01:31.560 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 29.264461437000001, 2024-09-21T15:01:31.560 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.024795576, 2024-09-21T15:01:31.560 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:31.560 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:31.560 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:31.560 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:31.560 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:31.560 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139994 2024-09-21T15:01:31.560 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.561 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:31.561 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.561 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:31.561 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231370+0000", 2024-09-21T15:01:31.561 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:31.561 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.561 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.561 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:31.561 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231370+0000", 2024-09-21T15:01:31.561 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:31.561 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.561 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.561 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:31.561 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231369+0000", 2024-09-21T15:01:31.561 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:31.561 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.561 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.562 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:31.562 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231376+0000", 2024-09-21T15:01:31.562 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5509999999999996e-06 2024-09-21T15:01:31.562 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.562 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.562 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:31.562 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231387+0000", 2024-09-21T15:01:31.562 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.061e-05 2024-09-21T15:01:31.562 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.562 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.562 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:31.563 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231394+0000", 2024-09-21T15:01:31.563 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3799999999999996e-06 2024-09-21T15:01:31.563 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.563 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.563 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:31.563 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231459+0000", 2024-09-21T15:01:31.563 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4716999999999997e-05 2024-09-21T15:01:31.563 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.563 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.563 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:31.563 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231468+0000", 2024-09-21T15:01:31.563 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1479999999999995e-06 2024-09-21T15:01:31.563 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.563 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.563 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:31.564 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256074+0000", 2024-09-21T15:01:31.564 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0246064180000001 2024-09-21T15:01:31.564 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.564 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.564 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:31.564 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256166+0000", 2024-09-21T15:01:31.564 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1862999999999998e-05 2024-09-21T15:01:31.564 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:31.564 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:31.564 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:31.564 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.564 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.564 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140012 3.16 3:6bd6dfb6:::rbd_data.113664780c85.0000000000000165:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:31.564 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.231890+0000", 2024-09-21T15:01:31.564 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 29.263941677999998, 2024-09-21T15:01:31.565 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.024408881, 2024-09-21T15:01:31.565 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:31.565 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:31.565 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:31.565 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:31.565 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:31.565 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140012 2024-09-21T15:01:31.565 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.565 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:31.565 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.565 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:31.565 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231890+0000", 2024-09-21T15:01:31.565 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:31.565 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.565 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.565 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:31.566 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231890+0000", 2024-09-21T15:01:31.566 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:31.566 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.566 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.566 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:31.566 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231889+0000", 2024-09-21T15:01:31.566 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:31.566 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.566 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.566 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:31.566 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231895+0000", 2024-09-21T15:01:31.566 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9569999999999999e-06 2024-09-21T15:01:31.566 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.566 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.567 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:31.567 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231910+0000", 2024-09-21T15:01:31.567 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4549000000000001e-05 2024-09-21T15:01:31.567 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.567 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.567 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:31.567 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231917+0000", 2024-09-21T15:01:31.567 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5730000000000002e-06 2024-09-21T15:01:31.567 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.567 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.567 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:31.567 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231988+0000", 2024-09-21T15:01:31.567 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0852999999999994e-05 2024-09-21T15:01:31.567 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.567 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.567 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:31.568 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.232000+0000", 2024-09-21T15:01:31.568 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2159e-05 2024-09-21T15:01:31.568 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.568 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.568 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:31.568 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256192+0000", 2024-09-21T15:01:31.568 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.024192073 2024-09-21T15:01:31.568 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.568 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.568 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:31.568 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256299+0000", 2024-09-21T15:01:31.568 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000106733 2024-09-21T15:01:31.568 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:31.568 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:31.568 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:31.568 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.569 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.569 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140055 3.16 3:69bb9803:::rbd_data.113664780c85.00000000000000da:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:31.569 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.233922+0000", 2024-09-21T15:01:31.569 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 29.261910148999998, 2024-09-21T15:01:31.569 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.022511768, 2024-09-21T15:01:31.569 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:31.569 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:31.569 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:31.569 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:31.569 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:31.569 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140055 2024-09-21T15:01:31.569 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.569 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:31.569 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.569 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:31.569 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233922+0000", 2024-09-21T15:01:31.570 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:31.570 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.570 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.570 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:31.570 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233922+0000", 2024-09-21T15:01:31.570 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:31.570 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.570 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.570 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:31.570 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233921+0000", 2024-09-21T15:01:31.570 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:31.570 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.570 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.570 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:31.571 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233927+0000", 2024-09-21T15:01:31.571 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0360000000000003e-06 2024-09-21T15:01:31.571 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.571 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.571 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:31.571 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233937+0000", 2024-09-21T15:01:31.571 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6439999999999995e-06 2024-09-21T15:01:31.571 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.571 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.571 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:31.571 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233944+0000", 2024-09-21T15:01:31.571 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7710000000000006e-06 2024-09-21T15:01:31.571 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.571 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.572 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:31.572 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233996+0000", 2024-09-21T15:01:31.572 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1597000000000001e-05 2024-09-21T15:01:31.572 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.572 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.572 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:31.572 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.234005+0000", 2024-09-21T15:01:31.572 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5960000000000004e-06 2024-09-21T15:01:31.572 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.572 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.572 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:31.573 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256332+0000", 2024-09-21T15:01:31.573 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0223271949999999 2024-09-21T15:01:31.573 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.573 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.573 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:31.573 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256434+0000", 2024-09-21T15:01:31.573 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010180599999999999 2024-09-21T15:01:31.573 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:31.573 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:31.573 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:31.573 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.573 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.573 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140143 3.16 3:6d3ed596:::rbd_data.113664780c85.0000000000000184:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:31.574 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.239496+0000", 2024-09-21T15:01:31.574 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 29.256335620000002, 2024-09-21T15:01:31.574 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.017076576, 2024-09-21T15:01:31.574 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:31.574 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:31.574 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:31.574 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:31.574 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:31.574 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140143 2024-09-21T15:01:31.574 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.574 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:31.574 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.574 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:31.575 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239496+0000", 2024-09-21T15:01:31.575 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:31.575 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.575 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.575 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:31.575 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239496+0000", 2024-09-21T15:01:31.575 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:31.575 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.575 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.575 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:31.575 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239496+0000", 2024-09-21T15:01:31.575 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:31.575 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.576 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.576 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:31.576 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239498+0000", 2024-09-21T15:01:31.576 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.509e-06 2024-09-21T15:01:31.576 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.576 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.576 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:31.576 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239504+0000", 2024-09-21T15:01:31.576 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6869999999999999e-06 2024-09-21T15:01:31.576 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.576 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.576 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:31.576 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239509+0000", 2024-09-21T15:01:31.576 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8860000000000003e-06 2024-09-21T15:01:31.577 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.577 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.577 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:31.577 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239572+0000", 2024-09-21T15:01:31.577 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2908000000000001e-05 2024-09-21T15:01:31.577 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.577 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.577 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:31.577 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239580+0000", 2024-09-21T15:01:31.577 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4270000000000008e-06 2024-09-21T15:01:31.577 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.577 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.577 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:31.577 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256457+0000", 2024-09-21T15:01:31.578 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0168770979999999 2024-09-21T15:01:31.578 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.578 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.578 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:31.578 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256573+0000", 2024-09-21T15:01:31.578 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00011538400000000001 2024-09-21T15:01:31.578 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:31.578 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:31.578 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:31.578 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.578 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.579 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140163 3.16 3:694e82eb:::rbd_data.113664780c85.0000000000000052:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:31.579 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.240334+0000", 2024-09-21T15:01:31.579 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 29.255497907999999, 2024-09-21T15:01:31.579 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.016365537, 2024-09-21T15:01:31.579 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:31.579 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:31.579 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:31.579 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:31.579 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:31.579 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140163 2024-09-21T15:01:31.579 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.579 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:31.579 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.580 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:31.580 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240334+0000", 2024-09-21T15:01:31.580 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:31.580 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.580 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.580 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:31.580 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240334+0000", 2024-09-21T15:01:31.580 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:31.580 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.580 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.580 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:31.580 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240334+0000", 2024-09-21T15:01:31.580 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:31.580 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.581 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.581 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:31.581 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240337+0000", 2024-09-21T15:01:31.581 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.089e-06 2024-09-21T15:01:31.581 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.581 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.581 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:31.581 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240343+0000", 2024-09-21T15:01:31.581 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4420000000000003e-06 2024-09-21T15:01:31.581 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.581 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.581 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:31.581 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240348+0000", 2024-09-21T15:01:31.581 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1039999999999998e-06 2024-09-21T15:01:31.581 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.581 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.582 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:31.582 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240451+0000", 2024-09-21T15:01:31.582 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000102996 2024-09-21T15:01:31.582 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.582 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.582 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:31.582 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240460+0000", 2024-09-21T15:01:31.582 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5369999999999997e-06 2024-09-21T15:01:31.582 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.582 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.582 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:31.582 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256600+0000", 2024-09-21T15:01:31.583 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.016139884 2024-09-21T15:01:31.583 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.583 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.583 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:31.583 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256700+0000", 2024-09-21T15:01:31.583 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9844000000000002e-05 2024-09-21T15:01:31.583 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:31.583 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:31.583 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:31.583 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.583 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.583 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140164 3.16 3:68bfdc57:::rbd_data.113664780c85.0000000000000053:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:31.583 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.240358+0000", 2024-09-21T15:01:31.583 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 29.255474213999999, 2024-09-21T15:01:31.584 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.01646094, 2024-09-21T15:01:31.584 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:31.584 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:31.584 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:31.584 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:31.584 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:31.584 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140164 2024-09-21T15:01:31.584 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.584 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:31.584 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.584 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:31.584 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240358+0000", 2024-09-21T15:01:31.584 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:31.584 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.584 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.584 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:31.585 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240358+0000", 2024-09-21T15:01:31.585 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:31.585 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.585 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.585 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:31.585 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240357+0000", 2024-09-21T15:01:31.585 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:31.585 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.585 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.585 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:31.585 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240360+0000", 2024-09-21T15:01:31.585 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7089999999999998e-06 2024-09-21T15:01:31.585 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.585 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.585 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:31.586 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240368+0000", 2024-09-21T15:01:31.586 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7270000000000007e-06 2024-09-21T15:01:31.586 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.586 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.586 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:31.586 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240373+0000", 2024-09-21T15:01:31.586 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4060000000000004e-06 2024-09-21T15:01:31.586 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.586 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.586 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:31.586 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240473+0000", 2024-09-21T15:01:31.586 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000100132 2024-09-21T15:01:31.586 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.586 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.586 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:31.587 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240476+0000", 2024-09-21T15:01:31.587 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.332e-06 2024-09-21T15:01:31.587 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.587 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.587 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:31.587 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256724+0000", 2024-09-21T15:01:31.587 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0162471930000001 2024-09-21T15:01:31.587 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.587 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.587 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:31.587 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256819+0000", 2024-09-21T15:01:31.587 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5007999999999996e-05 2024-09-21T15:01:31.587 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:31.587 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:31.588 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:31.588 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.588 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.588 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140188 3.16 3:6f6217d4:::rbd_data.113664780c85.00000000000000fb:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:31.588 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.240978+0000", 2024-09-21T15:01:31.588 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 29.254853555, 2024-09-21T15:01:31.588 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.015983809, 2024-09-21T15:01:31.588 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:31.588 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:31.588 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:31.588 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:31.588 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:31.588 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140188 2024-09-21T15:01:31.588 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.588 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:31.588 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.589 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:31.589 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240978+0000", 2024-09-21T15:01:31.589 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:31.589 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.589 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.589 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:31.589 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240978+0000", 2024-09-21T15:01:31.589 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:31.589 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.589 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.589 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:31.589 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240978+0000", 2024-09-21T15:01:31.589 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:31.589 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.589 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.590 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:31.590 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240981+0000", 2024-09-21T15:01:31.590 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1559999999999999e-06 2024-09-21T15:01:31.590 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.590 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.590 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:31.590 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240987+0000", 2024-09-21T15:01:31.590 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.229e-06 2024-09-21T15:01:31.590 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.590 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.590 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:31.590 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240992+0000", 2024-09-21T15:01:31.590 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9710000000000003e-06 2024-09-21T15:01:31.590 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.590 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.591 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:31.591 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.241361+0000", 2024-09-21T15:01:31.591 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000369236 2024-09-21T15:01:31.591 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.591 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.591 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:31.591 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.241369+0000", 2024-09-21T15:01:31.591 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6199999999999999e-06 2024-09-21T15:01:31.591 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.591 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.591 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:31.591 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256844+0000", 2024-09-21T15:01:31.592 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0154751609999999 2024-09-21T15:01:31.592 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:31.592 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:31.592 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:31.592 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256962+0000", 2024-09-21T15:01:31.592 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000117964 2024-09-21T15:01:31.592 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:31.592 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:31.592 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:31.592 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:31.592 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:31.592 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:31.594 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T15:01:31.594 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:01:31.588+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:01:31.694 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:01:31.688+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:01:31.709 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:01:31.709 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:01:31.709 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T15:01:31.710 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:01:31.714 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T15:01:31.719 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T15:01:31.795 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:01:31.792+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:01:31.825 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:01:31.825 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:01:31.825 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T15:01:31.825 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T15:01:31.825 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:01:31.835 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T15:01:31.857 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T15:01:31.880 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T15:01:31.896 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:01:31.892+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:01:31.903 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T15:01:31.923 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T15:01:31.945 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:01:31.945 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T15:01:31.945 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T15:01:31.945 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T15:01:31.945 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.946 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:63820 3.19 3.626bc079 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-21T15:01:31.946 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:27.241293+0000", 2024-09-21T15:01:31.946 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 124.699281075, 2024-09-21T15:01:31.946 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.98723808199999996, 2024-09-21T15:01:31.946 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:31.946 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:31.946 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:31.946 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:31.946 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:31.946 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 63820 2024-09-21T15:01:31.946 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.946 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:31.946 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.946 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:31.947 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241293+0000", 2024-09-21T15:01:31.947 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:31.947 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.947 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.947 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:31.947 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241293+0000", 2024-09-21T15:01:31.947 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:31.947 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.947 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.947 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:31.947 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241291+0000", 2024-09-21T15:01:31.947 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:01:31.947 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.947 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.947 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:31.948 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241299+0000", 2024-09-21T15:01:31.948 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1529999999999997e-06 2024-09-21T15:01:31.948 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.948 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.948 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:31.948 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241317+0000", 2024-09-21T15:01:31.948 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7764000000000001e-05 2024-09-21T15:01:31.948 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.948 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.948 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:31.948 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241328+0000", 2024-09-21T15:01:31.948 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1245e-05 2024-09-21T15:01:31.948 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.948 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.949 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:31.949 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241407+0000", 2024-09-21T15:01:31.949 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8762000000000002e-05 2024-09-21T15:01:31.949 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.949 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.949 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:31.949 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:27.241418+0000", 2024-09-21T15:01:31.949 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1695000000000001e-05 2024-09-21T15:01:31.949 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.949 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.949 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:31.949 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:28.228510+0000", 2024-09-21T15:01:31.949 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.98709166100000001 2024-09-21T15:01:31.949 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.949 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.950 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:31.950 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:28.228531+0000", 2024-09-21T15:01:31.950 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0696e-05 2024-09-21T15:01:31.950 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:31.950 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:31.950 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:31.950 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.950 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.950 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:97734 3.19 3:9fcf754f:::rbd_data.1136bfc79a70.0000000000000009:head [write 0~4194304 [fadvise_sequential+fadvise_nocache],write 4784128~147456 [fadvise_sequential+fadvise_nocache],write 4997120~147456 [fadvise_sequential+fadvise_nocache],write 5210112~32768 [fadvise_sequential+fadvise_nocache],write 5292032~147456 [fadvise_sequential+fadvise_nocache],write 5505024~147456 [fadvise_sequential+fadvise_nocache],write 5718016~49152 [fadvise_sequential+fadvise_nocache],write 5799936~147456 [fadvise_sequential+fadvise_nocache],write 6012928~147456 [fadvise_sequential+fadvise_nocache],write 6225920~65536 [fadvise_sequential+fadvise_nocache],write 6307840~147456 [fadvise_sequential+fadvise_nocache],write 6520832~147456 [fadvise_sequential+fadvise_nocache],write 6733824~229376 [fadvise_sequential+fadvise_nocache],write 7028736~147456 [fadvise_sequential+fadvise_nocache],write 7241728~229376 [fadvise_sequential+fadvise_nocache],write 7536640~147456 [fadvise_sequential+fadvise_nocache],write 7749632~229376 [fadvise_sequential+fadvise_nocache],write 8077312~114688 [fadvise_sequential+fadvise_nocache],write 8290304~98304 [fadvise_sequential+fadvise_nocache],write 8404992~81920 [fadvise_sequential+fadvise_nocache],write 8617984~81920 [fadvise_sequential+fadvise_nocache],write 8830976~81920 [fadvise_sequential+fadvise_nocache],write 8962048~32768 [fadvise_sequential+fadvise_nocache],write 9175040~32768 [fadvise_sequential+fadvise_nocache],write 9388032~32768 [fadvise_sequential+fadvise_nocache],write 9469952~32768 [fadvise_sequential+fadvise_nocache],write 9682944~20092 [fadvise_sequential+fadvise_nocache],write 9961472~163840 [fadvise_sequential+fadvise_nocache],write 10141696~196608 [fadvise_sequential+fadvise_nocache],write 10354688~278528 [fadvise_sequential+fadvise_nocache],write 10649600~196608 [fadvise_sequential+fadvise_nocache],write 10862592~196608 [fadvise_sequential+fadvise_nocache],write 11075584~65536 [fadvise_sequential+fadvise_nocache],write 11173888~98304 [fadvise_sequential+fadvise_nocache],write 11288576~65536 [fadvise_sequential+fadvise_nocache],write 11386880~98304 [fadvise_sequential+fadvise_nocache],write 11501568~65536 [fadvise_sequential+fadvise_nocache],write 11583488~65536 [fadvise_sequential+fadvise_nocache],write 11681792~98304 [fadvise_sequential+fadvise_nocache],write 11796480~65536 [fadvise_sequential+fadvise_nocache],write 11894784~98304 [fadvise_sequential+fadvise_nocache],write 12009472~65536 [fadvise_sequential+fadvise_nocache],write 12189696~98304 [fadvise_sequential+fadvise_nocache],write 12402688~98304 [fadvise_sequential+fadvise_nocache],write 12697600~98304 [fadvise_sequential+fadvise_nocache],write 12910592~98304 [fadvise_sequential+fadvise_nocache],write 13205504~98304 [fadvise_sequential+fadvise_nocache],write 13418496~98304 [fadvise_sequential+fadvise_nocache],write 13713408~98304 [fadvise_sequential+fadvise_nocache],write 13926400~98304 [fadvise_sequential+fadvise_nocache],write 14139392~16384 [fadvise_sequential+fadvise_nocache],write 14680064~16384 [fadvise_sequential+fadvise_nocache],write 14827520~81920 [fadvise_sequential+fadvise_nocache],write 15040512~81920 [fadvise_sequential+fadvise_nocache],write 15335424~81920 [fadvise_sequential+fadvise_nocache],write 15548416~81920 [fadvise_sequential+fadvise_nocache],write 15843328~81920 [fadvise_sequential+fadvise_nocache],write 16056320~81920 [fadvise_sequential+fadvise_nocache],write 16351232~81920 [fadvise_sequential+fadvise_nocache],write 16564224~81920 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:01:31.950 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.354331+0000", 2024-09-21T15:01:31.950 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 86.586243027999998, 2024-09-21T15:01:31.950 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.086284870999999999, 2024-09-21T15:01:31.950 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:31.951 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:31.951 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:31.951 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:31.951 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:31.951 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97734 2024-09-21T15:01:31.951 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.951 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:31.951 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.951 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:31.951 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.354331+0000", 2024-09-21T15:01:31.951 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:31.951 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.951 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.952 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:31.952 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.354331+0000", 2024-09-21T15:01:31.952 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:31.952 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.952 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.952 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:31.952 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.354329+0000", 2024-09-21T15:01:31.952 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:01:31.952 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.952 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.952 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:31.952 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.370234+0000", 2024-09-21T15:01:31.952 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015905348999999999 2024-09-21T15:01:31.952 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.952 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.953 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:31.953 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.370286+0000", 2024-09-21T15:01:31.953 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1656999999999997e-05 2024-09-21T15:01:31.953 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.953 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.953 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:31.953 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.370315+0000", 2024-09-21T15:01:31.953 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9204e-05 2024-09-21T15:01:31.953 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.953 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.953 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:31.954 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.370395+0000", 2024-09-21T15:01:31.954 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0395999999999994e-05 2024-09-21T15:01:31.954 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.954 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.954 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:31.954 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.370604+0000", 2024-09-21T15:01:31.954 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000208684 2024-09-21T15:01:31.954 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.954 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.955 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 1", 2024-09-21T15:01:31.955 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.371701+0000", 2024-09-21T15:01:31.955 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0010973140000000001 2024-09-21T15:01:31.955 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.955 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.955 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:01:31.955 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.405485+0000", 2024-09-21T15:01:31.955 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.033783615000000003 2024-09-21T15:01:31.955 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.956 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.956 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:01:31.956 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.440211+0000", 2024-09-21T15:01:31.956 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034725757000000003 2024-09-21T15:01:31.956 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.956 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.956 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:01:31.956 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.440497+0000", 2024-09-21T15:01:31.957 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00028673099999999998 2024-09-21T15:01:31.957 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.957 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.957 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:31.957 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.440616+0000", 2024-09-21T15:01:31.957 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000118256 2024-09-21T15:01:31.957 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:31.957 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:31.957 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:31.958 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.958 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.958 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112831 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 1515520~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:31.958 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.406761+0000", 2024-09-21T15:01:31.958 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 62.533812330000003, 2024-09-21T15:01:31.958 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060265666000000002, 2024-09-21T15:01:31.958 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:31.958 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:31.959 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:31.959 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:31.959 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:31.959 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112831 2024-09-21T15:01:31.959 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.959 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:31.959 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.959 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:31.959 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406761+0000", 2024-09-21T15:01:31.960 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:31.960 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.960 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.960 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:31.960 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406761+0000", 2024-09-21T15:01:31.960 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:31.960 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.960 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.961 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:31.961 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406761+0000", 2024-09-21T15:01:31.961 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:31.961 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.961 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.961 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:31.961 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406764+0000", 2024-09-21T15:01:31.961 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0319999999999999e-06 2024-09-21T15:01:31.962 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.962 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.962 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:31.962 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406776+0000", 2024-09-21T15:01:31.962 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2357999999999999e-05 2024-09-21T15:01:31.962 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.962 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.962 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:31.962 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406784+0000", 2024-09-21T15:01:31.963 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6059999999999997e-06 2024-09-21T15:01:31.963 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.963 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.963 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:31.963 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.466595+0000", 2024-09-21T15:01:31.963 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059811062999999998 2024-09-21T15:01:31.963 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.963 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.964 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:31.964 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.466782+0000", 2024-09-21T15:01:31.964 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000186622 2024-09-21T15:01:31.964 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.964 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.964 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:31.964 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.467027+0000", 2024-09-21T15:01:31.964 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000245455 2024-09-21T15:01:31.964 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:31.965 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:31.965 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:31.965 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.965 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.965 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112832 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 1605632~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:31.965 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.406788+0000", 2024-09-21T15:01:31.965 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 62.533785739999999, 2024-09-21T15:01:31.965 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060452065999999999, 2024-09-21T15:01:31.965 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:31.966 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:31.966 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:31.966 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:31.966 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:31.966 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112832 2024-09-21T15:01:31.966 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.966 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:31.966 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.966 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:31.967 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406788+0000", 2024-09-21T15:01:31.967 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:31.967 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.967 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.967 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:31.967 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406788+0000", 2024-09-21T15:01:31.967 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:31.967 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.967 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.968 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:31.968 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406788+0000", 2024-09-21T15:01:31.968 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:31.968 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.968 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.968 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:31.968 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406798+0000", 2024-09-21T15:01:31.968 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0096e-05 2024-09-21T15:01:31.968 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.968 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.969 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:31.969 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406803+0000", 2024-09-21T15:01:31.969 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2020000000000003e-06 2024-09-21T15:01:31.969 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.969 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.969 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:31.969 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.406808+0000", 2024-09-21T15:01:31.969 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.8239999999999999e-06 2024-09-21T15:01:31.969 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.970 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.970 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:31.970 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.467049+0000", 2024-09-21T15:01:31.970 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060241754000000002 2024-09-21T15:01:31.970 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.970 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.970 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:31.970 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.467134+0000", 2024-09-21T15:01:31.970 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4216999999999997e-05 2024-09-21T15:01:31.971 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.971 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.971 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:31.971 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.467240+0000", 2024-09-21T15:01:31.971 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000106304 2024-09-21T15:01:31.971 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:31.971 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:31.971 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:31.971 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.971 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.971 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112843 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 2613248~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:31.971 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.410273+0000", 2024-09-21T15:01:31.971 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 62.530301145999999, 2024-09-21T15:01:31.972 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059854931, 2024-09-21T15:01:31.972 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:31.972 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:31.972 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:31.972 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:31.972 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:31.972 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112843 2024-09-21T15:01:31.972 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.972 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:31.972 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.972 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:31.972 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410273+0000", 2024-09-21T15:01:31.972 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:31.973 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.973 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.973 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:31.973 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410273+0000", 2024-09-21T15:01:31.973 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:31.973 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.973 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.973 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:31.973 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410272+0000", 2024-09-21T15:01:31.973 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:31.973 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.973 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.973 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:31.973 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410274+0000", 2024-09-21T15:01:31.973 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.905e-06 2024-09-21T15:01:31.974 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.974 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.974 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:31.974 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410283+0000", 2024-09-21T15:01:31.974 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3440000000000007e-06 2024-09-21T15:01:31.974 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.974 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.974 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:31.974 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410294+0000", 2024-09-21T15:01:31.974 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0271e-05 2024-09-21T15:01:31.974 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.974 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.974 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:31.975 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.469917+0000", 2024-09-21T15:01:31.975 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059623738000000003 2024-09-21T15:01:31.975 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.975 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.975 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:31.975 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470002+0000", 2024-09-21T15:01:31.975 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4270999999999993e-05 2024-09-21T15:01:31.975 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.975 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.975 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:31.975 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470127+0000", 2024-09-21T15:01:31.975 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000125711 2024-09-21T15:01:31.975 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:31.975 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:31.975 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:31.976 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.976 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.976 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112844 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 2703360~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:31.976 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.410326+0000", 2024-09-21T15:01:31.976 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 62.530247426000003, 2024-09-21T15:01:31.976 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060086056999999998, 2024-09-21T15:01:31.976 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:31.976 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:31.976 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:31.976 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:31.976 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:31.976 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112844 2024-09-21T15:01:31.976 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.976 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:31.977 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.977 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:31.977 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410326+0000", 2024-09-21T15:01:31.977 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:31.977 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.977 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.977 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:31.977 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410326+0000", 2024-09-21T15:01:31.977 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:31.977 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.977 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.977 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:31.977 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410326+0000", 2024-09-21T15:01:31.977 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:31.977 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.977 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.978 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:31.978 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410352+0000", 2024-09-21T15:01:31.978 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6872999999999998e-05 2024-09-21T15:01:31.978 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.978 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.978 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:31.978 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410374+0000", 2024-09-21T15:01:31.978 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1141e-05 2024-09-21T15:01:31.978 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.978 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.978 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:31.978 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410385+0000", 2024-09-21T15:01:31.978 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1456e-05 2024-09-21T15:01:31.978 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.979 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.979 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:31.979 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470174+0000", 2024-09-21T15:01:31.979 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059788574999999997 2024-09-21T15:01:31.979 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.979 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.979 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:31.979 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470277+0000", 2024-09-21T15:01:31.979 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000103838 2024-09-21T15:01:31.979 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.979 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.979 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:31.979 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470412+0000", 2024-09-21T15:01:31.979 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013489700000000001 2024-09-21T15:01:31.980 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:31.980 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:31.980 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:31.980 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.980 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.980 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112845 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 2793472~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:31.980 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.410394+0000", 2024-09-21T15:01:31.980 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 62.530179951999997, 2024-09-21T15:01:31.980 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060244173999999998, 2024-09-21T15:01:31.980 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:31.980 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:31.980 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:31.980 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:31.980 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:31.980 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112845 2024-09-21T15:01:31.981 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.981 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:31.981 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.981 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:31.981 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410394+0000", 2024-09-21T15:01:31.981 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:31.981 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.981 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.981 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:31.981 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410394+0000", 2024-09-21T15:01:31.981 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:31.981 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.981 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.981 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:31.981 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410393+0000", 2024-09-21T15:01:31.981 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:31.981 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.982 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.982 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:31.982 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410397+0000", 2024-09-21T15:01:31.982 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.9430000000000002e-06 2024-09-21T15:01:31.982 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.982 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.982 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:31.982 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410406+0000", 2024-09-21T15:01:31.982 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.799e-06 2024-09-21T15:01:31.982 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.982 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.982 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:31.982 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410418+0000", 2024-09-21T15:01:31.982 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2225999999999999e-05 2024-09-21T15:01:31.983 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.983 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.983 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:31.983 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470444+0000", 2024-09-21T15:01:31.983 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060026311999999998 2024-09-21T15:01:31.983 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.983 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.983 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:31.983 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470535+0000", 2024-09-21T15:01:31.983 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0173999999999994e-05 2024-09-21T15:01:31.983 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.983 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.983 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:31.984 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470638+0000", 2024-09-21T15:01:31.984 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010333 2024-09-21T15:01:31.984 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:31.984 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:31.984 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:31.984 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.984 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.984 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112846 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 2883584~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:31.984 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.410425+0000", 2024-09-21T15:01:31.984 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 62.530148738999998, 2024-09-21T15:01:31.984 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060459247000000001, 2024-09-21T15:01:31.984 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:31.984 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:31.984 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:31.985 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:31.985 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:31.985 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112846 2024-09-21T15:01:31.985 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.985 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:31.985 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.985 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:31.985 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410425+0000", 2024-09-21T15:01:31.985 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:31.985 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.985 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.985 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:31.985 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410425+0000", 2024-09-21T15:01:31.985 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:31.985 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.986 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.986 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:31.986 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410424+0000", 2024-09-21T15:01:31.986 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:31.986 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.986 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.986 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:31.986 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410428+0000", 2024-09-21T15:01:31.986 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4709999999999999e-06 2024-09-21T15:01:31.986 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.986 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.986 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:31.986 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410446+0000", 2024-09-21T15:01:31.987 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8354000000000001e-05 2024-09-21T15:01:31.987 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.987 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.987 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:31.987 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410456+0000", 2024-09-21T15:01:31.987 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6709999999999994e-06 2024-09-21T15:01:31.987 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.987 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.987 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:31.987 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470661+0000", 2024-09-21T15:01:31.987 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060205303000000002 2024-09-21T15:01:31.987 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.987 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.987 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:31.987 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470756+0000", 2024-09-21T15:01:31.988 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4401999999999995e-05 2024-09-21T15:01:31.988 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.988 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.988 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:31.988 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470884+0000", 2024-09-21T15:01:31.988 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012860500000000001 2024-09-21T15:01:31.988 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:31.988 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:31.988 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:31.988 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.988 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.988 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112847 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 2973696~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:31.988 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.410462+0000", 2024-09-21T15:01:31.988 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 62.530111695000002, 2024-09-21T15:01:31.989 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060653836000000003, 2024-09-21T15:01:31.989 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:31.989 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:31.989 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:31.989 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:31.989 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:31.989 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112847 2024-09-21T15:01:31.989 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.989 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:31.989 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.989 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:31.989 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410462+0000", 2024-09-21T15:01:31.989 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:31.989 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.989 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.990 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:31.990 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410462+0000", 2024-09-21T15:01:31.990 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:31.990 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.990 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.990 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:31.990 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410461+0000", 2024-09-21T15:01:31.990 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:31.990 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.990 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.990 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:31.990 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410465+0000", 2024-09-21T15:01:31.990 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6899999999999998e-06 2024-09-21T15:01:31.990 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.990 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.991 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:31.991 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410474+0000", 2024-09-21T15:01:31.991 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4179999999999997e-06 2024-09-21T15:01:31.991 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.991 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.991 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:31.991 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410508+0000", 2024-09-21T15:01:31.991 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4282999999999997e-05 2024-09-21T15:01:31.991 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.991 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.991 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:31.991 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.470922+0000", 2024-09-21T15:01:31.991 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060413820999999999 2024-09-21T15:01:31.991 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.991 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.992 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:31.992 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471008+0000", 2024-09-21T15:01:31.992 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6379999999999996e-05 2024-09-21T15:01:31.992 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.992 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.992 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:31.992 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471116+0000", 2024-09-21T15:01:31.992 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000107806 2024-09-21T15:01:31.992 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:31.992 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:31.992 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:31.992 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.992 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.992 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112848 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 3063808~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:31.993 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.410525+0000", 2024-09-21T15:01:31.993 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 62.530048411999999, 2024-09-21T15:01:31.993 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060880735999999998, 2024-09-21T15:01:31.993 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:31.993 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:31.993 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:31.993 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:31.993 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:31.993 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112848 2024-09-21T15:01:31.993 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.993 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:31.993 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.993 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:31.993 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410525+0000", 2024-09-21T15:01:31.994 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:31.994 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.994 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.994 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:31.994 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410525+0000", 2024-09-21T15:01:31.994 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:31.994 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.994 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.994 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:31.994 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410525+0000", 2024-09-21T15:01:31.994 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:31.994 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.994 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.994 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:31.995 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410529+0000", 2024-09-21T15:01:31.995 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1760000000000003e-06 2024-09-21T15:01:31.995 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.995 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.995 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:31.995 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410551+0000", 2024-09-21T15:01:31.995 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2099999999999998e-05 2024-09-21T15:01:31.995 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.995 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.995 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:31.995 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410571+0000", 2024-09-21T15:01:31.995 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0159000000000001e-05 2024-09-21T15:01:31.996 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.996 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.996 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:31.996 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471164+0000", 2024-09-21T15:01:31.996 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.06059287 2024-09-21T15:01:31.996 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.996 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.996 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:31.996 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471271+0000", 2024-09-21T15:01:31.996 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000106635 2024-09-21T15:01:31.996 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.996 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.996 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:31.996 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471406+0000", 2024-09-21T15:01:31.996 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000135399 2024-09-21T15:01:31.997 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:31.997 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:31.997 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:31.997 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.997 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.997 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112849 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 3153920~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:31.997 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.410579+0000", 2024-09-21T15:01:31.997 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 62.529995116999999, 2024-09-21T15:01:31.997 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.061057304999999999, 2024-09-21T15:01:31.997 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:31.997 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:31.997 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:31.997 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:31.997 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:31.998 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112849 2024-09-21T15:01:31.998 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.998 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:31.998 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.998 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:31.998 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410579+0000", 2024-09-21T15:01:31.998 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:31.998 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.998 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.998 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:31.998 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410579+0000", 2024-09-21T15:01:31.998 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:31.999 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.999 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.999 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:31.999 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410578+0000", 2024-09-21T15:01:31.999 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:31.999 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.999 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.999 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:31.999 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410582+0000", 2024-09-21T15:01:31.999 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.036e-06 2024-09-21T15:01:31.999 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:31.999 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:31.999 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:31.999 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410592+0000", 2024-09-21T15:01:31.999 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.9850000000000001e-06 2024-09-21T15:01:32.000 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.000 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.000 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:32.000 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410604+0000", 2024-09-21T15:01:32.000 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2051e-05 2024-09-21T15:01:32.000 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.000 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.000 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:32.000 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471444+0000", 2024-09-21T15:01:32.000 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060839722999999998 2024-09-21T15:01:32.000 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.000 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.000 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:32.001 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471530+0000", 2024-09-21T15:01:32.001 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6359000000000004e-05 2024-09-21T15:01:32.001 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.001 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.001 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:32.001 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471636+0000", 2024-09-21T15:01:32.001 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010576700000000001 2024-09-21T15:01:32.001 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:32.001 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:32.001 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:32.001 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.001 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.001 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112850 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 3244032~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:32.001 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.410698+0000", 2024-09-21T15:01:32.002 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 62.529875775000001, 2024-09-21T15:01:32.002 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.061164118000000003, 2024-09-21T15:01:32.002 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:32.002 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:32.002 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:32.002 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:32.002 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:32.002 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112850 2024-09-21T15:01:32.002 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.002 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:32.002 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.002 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:32.002 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410698+0000", 2024-09-21T15:01:32.002 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:32.002 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.003 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.003 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:32.003 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410698+0000", 2024-09-21T15:01:32.003 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:32.003 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.003 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.003 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:32.003 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410610+0000", 2024-09-21T15:01:32.003 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999123 2024-09-21T15:01:32.003 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.003 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.003 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:32.003 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410707+0000", 2024-09-21T15:01:32.003 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7093e-05 2024-09-21T15:01:32.003 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.003 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.004 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:32.004 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410728+0000", 2024-09-21T15:01:32.004 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0275000000000002e-05 2024-09-21T15:01:32.004 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.004 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.004 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:32.004 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.410740+0000", 2024-09-21T15:01:32.004 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2561999999999999e-05 2024-09-21T15:01:32.004 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.004 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.004 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:32.004 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471664+0000", 2024-09-21T15:01:32.004 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060923979000000003 2024-09-21T15:01:32.004 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.005 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.005 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:32.005 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471753+0000", 2024-09-21T15:01:32.005 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8265999999999998e-05 2024-09-21T15:01:32.005 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.005 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.005 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:32.005 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.471862+0000", 2024-09-21T15:01:32.005 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000109474 2024-09-21T15:01:32.005 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:32.005 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:32.005 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:32.005 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.005 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.005 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112864 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 4521984~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:32.005 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.414789+0000", 2024-09-21T15:01:32.006 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 62.52578475, 2024-09-21T15:01:32.006 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059845903999999998, 2024-09-21T15:01:32.006 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:32.006 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:32.006 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:32.006 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:32.006 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:32.006 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112864 2024-09-21T15:01:32.006 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.006 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:32.006 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.006 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:32.006 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414789+0000", 2024-09-21T15:01:32.006 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:32.007 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.007 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.007 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:32.007 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414789+0000", 2024-09-21T15:01:32.007 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:32.007 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.007 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.007 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:32.007 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414788+0000", 2024-09-21T15:01:32.007 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:32.007 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.007 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.007 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:32.007 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414790+0000", 2024-09-21T15:01:32.007 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.995e-06 2024-09-21T15:01:32.007 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.008 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.008 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:32.008 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414801+0000", 2024-09-21T15:01:32.008 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.01e-05 2024-09-21T15:01:32.008 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.008 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.008 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:32.008 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414804+0000", 2024-09-21T15:01:32.008 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.563e-06 2024-09-21T15:01:32.008 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.008 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.008 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:32.008 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.474499+0000", 2024-09-21T15:01:32.008 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059695208 2024-09-21T15:01:32.008 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.008 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.008 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:32.009 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.474561+0000", 2024-09-21T15:01:32.009 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1672000000000002e-05 2024-09-21T15:01:32.009 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.009 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.009 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:32.009 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.474635+0000", 2024-09-21T15:01:32.009 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3792000000000001e-05 2024-09-21T15:01:32.009 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:32.009 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:32.009 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:32.009 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.009 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.009 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112865 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 4612096~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:32.009 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.414807+0000", 2024-09-21T15:01:32.009 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 62.525766353999998, 2024-09-21T15:01:32.009 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060003506999999998, 2024-09-21T15:01:32.010 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:32.010 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:32.010 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:32.010 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:32.010 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:32.010 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112865 2024-09-21T15:01:32.010 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.010 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:32.010 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.010 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:32.010 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414807+0000", 2024-09-21T15:01:32.010 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:32.010 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.010 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.010 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:32.011 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414807+0000", 2024-09-21T15:01:32.011 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:32.011 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.011 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.011 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:32.011 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414807+0000", 2024-09-21T15:01:32.011 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:32.011 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.011 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.011 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:32.011 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414809+0000", 2024-09-21T15:01:32.011 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0509999999999999e-06 2024-09-21T15:01:32.011 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.012 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.012 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:32.012 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414813+0000", 2024-09-21T15:01:32.012 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0960000000000003e-06 2024-09-21T15:01:32.012 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.012 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.012 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:32.012 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414817+0000", 2024-09-21T15:01:32.012 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.296e-06 2024-09-21T15:01:32.012 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.012 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.012 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:32.012 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.474671+0000", 2024-09-21T15:01:32.012 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059853663000000001 2024-09-21T15:01:32.012 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.013 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.013 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:32.013 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.474735+0000", 2024-09-21T15:01:32.013 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3653999999999997e-05 2024-09-21T15:01:32.013 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.013 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.013 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:32.013 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.474811+0000", 2024-09-21T15:01:32.013 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6074000000000004e-05 2024-09-21T15:01:32.013 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:32.013 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:32.013 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:32.013 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.013 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.013 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112866 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 4702208~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:32.013 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.414820+0000", 2024-09-21T15:01:32.014 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 62.525753434000002, 2024-09-21T15:01:32.014 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060161992999999997, 2024-09-21T15:01:32.014 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:32.014 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:32.014 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:32.014 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:32.014 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:32.014 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112866 2024-09-21T15:01:32.014 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.014 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:32.014 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.014 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:32.015 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414820+0000", 2024-09-21T15:01:32.015 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:32.015 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.015 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.015 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:32.015 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414820+0000", 2024-09-21T15:01:32.015 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:32.015 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.015 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.015 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:32.016 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414820+0000", 2024-09-21T15:01:32.016 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:32.016 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.016 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.016 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:32.016 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414822+0000", 2024-09-21T15:01:32.016 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8509999999999999e-06 2024-09-21T15:01:32.016 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.016 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.016 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:32.016 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414826+0000", 2024-09-21T15:01:32.016 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8190000000000002e-06 2024-09-21T15:01:32.016 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.016 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.017 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:32.017 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414837+0000", 2024-09-21T15:01:32.017 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1232e-05 2024-09-21T15:01:32.017 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.017 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.017 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:32.017 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.474846+0000", 2024-09-21T15:01:32.017 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060009506999999997 2024-09-21T15:01:32.017 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.017 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.017 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:32.017 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.474910+0000", 2024-09-21T15:01:32.017 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3918999999999996e-05 2024-09-21T15:01:32.017 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.018 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.018 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:32.018 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.474982+0000", 2024-09-21T15:01:32.018 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1958999999999993e-05 2024-09-21T15:01:32.018 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:32.018 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:32.018 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:32.018 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.018 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.018 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:112867 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 4792320~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:32.018 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.414840+0000", 2024-09-21T15:01:32.018 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 62.525733938999998, 2024-09-21T15:01:32.018 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.060308542999999999, 2024-09-21T15:01:32.018 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:32.018 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:32.019 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:32.019 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:32.019 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:32.019 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 112867 2024-09-21T15:01:32.019 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.019 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:32.019 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.019 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:32.019 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414840+0000", 2024-09-21T15:01:32.019 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:32.019 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.019 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.019 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:32.019 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414840+0000", 2024-09-21T15:01:32.019 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:32.020 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.020 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.020 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:32.020 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414839+0000", 2024-09-21T15:01:32.020 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:32.020 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.020 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.020 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:32.020 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414841+0000", 2024-09-21T15:01:32.020 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8500000000000001e-06 2024-09-21T15:01:32.020 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.020 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.020 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:32.021 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414846+0000", 2024-09-21T15:01:32.021 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7700000000000001e-06 2024-09-21T15:01:32.021 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.021 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.021 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:32.021 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.414850+0000", 2024-09-21T15:01:32.021 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1389999999999997e-06 2024-09-21T15:01:32.021 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.021 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.021 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:32.021 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.475016+0000", 2024-09-21T15:01:32.021 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.06016614 2024-09-21T15:01:32.021 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.021 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.021 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:32.022 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.475073+0000", 2024-09-21T15:01:32.022 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6295000000000002e-05 2024-09-21T15:01:32.022 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.022 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.022 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:32.022 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.475148+0000", 2024-09-21T15:01:32.022 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5647000000000003e-05 2024-09-21T15:01:32.022 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:32.022 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:32.022 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:32.022 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.022 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.022 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:113030 3.6 3:60420fc2:::rbd_data.1136b442e94a.0000000000000001:head [write 0~360448 [fadvise_sequential+fadvise_nocache],write 385024~573440 [fadvise_sequential+fadvise_nocache],write 1146880~573440 [fadvise_sequential+fadvise_nocache],write 1908736~188416 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:32.022 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.422420+0000", 2024-09-21T15:01:32.023 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 62.518153294000001, 2024-09-21T15:01:32.023 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.063268959999999999, 2024-09-21T15:01:32.023 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:32.023 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:32.023 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:32.023 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:32.023 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:32.023 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 113030 2024-09-21T15:01:32.023 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.023 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:32.023 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.024 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:32.024 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.422420+0000", 2024-09-21T15:01:32.024 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:32.024 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.024 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.024 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:32.024 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.422420+0000", 2024-09-21T15:01:32.024 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:32.024 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.024 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.024 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:32.024 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.422420+0000", 2024-09-21T15:01:32.024 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:32.024 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.024 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.025 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:32.025 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.424657+0000", 2024-09-21T15:01:32.025 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0022369780000000001 2024-09-21T15:01:32.025 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.025 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.025 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:32.025 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.424673+0000", 2024-09-21T15:01:32.025 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6141e-05 2024-09-21T15:01:32.025 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.025 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.025 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:32.025 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.424689+0000", 2024-09-21T15:01:32.025 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5897999999999999e-05 2024-09-21T15:01:32.025 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.025 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.025 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:32.026 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.478005+0000", 2024-09-21T15:01:32.026 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.05331636 2024-09-21T15:01:32.026 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.026 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.026 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:32.026 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.478133+0000", 2024-09-21T15:01:32.026 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012736799999999999 2024-09-21T15:01:32.026 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.026 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.026 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 7", 2024-09-21T15:01:32.026 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.478307+0000", 2024-09-21T15:01:32.026 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017491499999999999 2024-09-21T15:01:32.026 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.026 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.027 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:01:32.027 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.483611+0000", 2024-09-21T15:01:32.027 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0053039389999999997 2024-09-21T15:01:32.027 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.027 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.027 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:01:32.027 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.485553+0000", 2024-09-21T15:01:32.027 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0019418759999999999 2024-09-21T15:01:32.027 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.027 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.027 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:01:32.027 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.485645+0000", 2024-09-21T15:01:32.027 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2138000000000005e-05 2024-09-21T15:01:32.027 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.027 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.028 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:32.028 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.485689+0000", 2024-09-21T15:01:32.028 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3922999999999999e-05 2024-09-21T15:01:32.028 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:32.028 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:32.028 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:32.028 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.028 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.028 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:114290 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 6946816~8192 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:32.028 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.558087+0000", 2024-09-21T15:01:32.028 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 62.382486497000002, 2024-09-21T15:01:32.028 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059973091999999999, 2024-09-21T15:01:32.028 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:32.028 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:32.028 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:32.029 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:32.029 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:32.029 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 114290 2024-09-21T15:01:32.029 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.029 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:32.029 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.029 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:32.029 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.558087+0000", 2024-09-21T15:01:32.029 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:32.029 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.029 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.029 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:32.029 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.558087+0000", 2024-09-21T15:01:32.029 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:32.029 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.030 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.030 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:32.030 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.558087+0000", 2024-09-21T15:01:32.030 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:32.030 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.030 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.030 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:32.030 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.558092+0000", 2024-09-21T15:01:32.030 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2379999999999997e-06 2024-09-21T15:01:32.030 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.030 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.030 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:32.030 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.558095+0000", 2024-09-21T15:01:32.030 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.275e-06 2024-09-21T15:01:32.030 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.031 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.031 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:32.031 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.558099+0000", 2024-09-21T15:01:32.031 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.6679999999999999e-06 2024-09-21T15:01:32.031 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.031 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.031 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:32.031 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.617757+0000", 2024-09-21T15:01:32.031 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059657531999999999 2024-09-21T15:01:32.031 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.031 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.031 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:32.031 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.617891+0000", 2024-09-21T15:01:32.031 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000134478 2024-09-21T15:01:32.032 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.032 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.032 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:32.032 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.618060+0000", 2024-09-21T15:01:32.032 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016925100000000001 2024-09-21T15:01:32.032 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:32.032 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:32.032 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:32.032 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.032 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.032 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:114312 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 8945664~8192 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:32.032 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.562396+0000", 2024-09-21T15:01:32.032 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 62.378178149, 2024-09-21T15:01:32.032 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059832396000000003, 2024-09-21T15:01:32.032 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:32.033 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:32.033 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:32.033 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:32.033 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:32.033 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 114312 2024-09-21T15:01:32.033 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.033 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:32.033 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.033 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:32.033 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562396+0000", 2024-09-21T15:01:32.033 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:32.033 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.034 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.034 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:32.034 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562396+0000", 2024-09-21T15:01:32.034 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:32.034 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.034 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.034 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:32.034 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562395+0000", 2024-09-21T15:01:32.034 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:32.034 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.034 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.034 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:32.034 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562397+0000", 2024-09-21T15:01:32.034 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9e-06 2024-09-21T15:01:32.035 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.035 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.035 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:32.035 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562401+0000", 2024-09-21T15:01:32.035 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.8099999999999999e-06 2024-09-21T15:01:32.035 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.035 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.035 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:32.035 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562405+0000", 2024-09-21T15:01:32.035 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.5710000000000002e-06 2024-09-21T15:01:32.035 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.035 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.035 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:32.035 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.621997+0000", 2024-09-21T15:01:32.035 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059592118999999999 2024-09-21T15:01:32.036 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.036 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.036 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:32.036 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.622121+0000", 2024-09-21T15:01:32.036 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012411599999999999 2024-09-21T15:01:32.036 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.036 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.036 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:32.036 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.622228+0000", 2024-09-21T15:01:32.036 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000107184 2024-09-21T15:01:32.036 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:32.036 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:32.036 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:32.036 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.036 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.037 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:114313 3.6 3:66bbbf5b:::rbd_data.1136b1194d20.0000000000000000:58 [read 9035776~8192 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:32.037 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.562408+0000", 2024-09-21T15:01:32.037 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 62.378166168, 2024-09-21T15:01:32.037 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059975560999999997, 2024-09-21T15:01:32.037 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:32.037 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:32.037 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:32.037 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:32.037 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:32.037 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 114313 2024-09-21T15:01:32.037 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.037 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:32.037 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.037 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:32.038 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562408+0000", 2024-09-21T15:01:32.038 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:32.038 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.038 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.038 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:32.038 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562408+0000", 2024-09-21T15:01:32.038 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:32.038 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.038 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.038 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:32.038 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562407+0000", 2024-09-21T15:01:32.038 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:32.038 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.038 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.038 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:32.038 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562409+0000", 2024-09-21T15:01:32.039 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9700000000000002e-06 2024-09-21T15:01:32.039 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.039 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.039 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:32.039 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562421+0000", 2024-09-21T15:01:32.039 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2094e-05 2024-09-21T15:01:32.039 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.039 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.039 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:32.039 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.562427+0000", 2024-09-21T15:01:32.039 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7690000000000002e-06 2024-09-21T15:01:32.039 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.039 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.039 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:32.039 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.622264+0000", 2024-09-21T15:01:32.040 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.059837488000000001 2024-09-21T15:01:32.040 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.040 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.040 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:32.040 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.622319+0000", 2024-09-21T15:01:32.040 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4064999999999998e-05 2024-09-21T15:01:32.040 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:32.040 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:32.040 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:32.040 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.622383+0000", 2024-09-21T15:01:32.040 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4506999999999998e-05 2024-09-21T15:01:32.040 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:32.040 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:32.040 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:32.040 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:32.040 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:32.041 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:01:32.042 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4: {} 2024-09-21T15:01:32.042 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T15:01:32.042 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T15:01:32.042 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T15:01:32.043 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T15:01:32.044 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:01:32.044 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:01:32.044 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:01:32.044 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:01:32.044 INFO:teuthology.orchestra.run.smithi032.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:01:32.044 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:01:32.044 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:01:32.044 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:01:32.044 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:01:32.042+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:01:32.142 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:01:32.136+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:01:32.162 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:32.162 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:32.173 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T15:01:32.182 INFO:teuthology.orchestra.run.smithi032.stderr:nodeep-scrub is set 2024-09-21T15:01:32.243 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:01:32.242+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:01:32.327 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:32.327 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:32.337 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T15:01:32.344 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:01:32.342+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:01:32.440 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:32.441 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:32.445 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:01:32.440+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:01:32.451 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T15:01:32.546 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:01:32.540+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:01:32.567 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:32.567 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:32.577 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T15:01:32.646 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:01:32.640+0000 7f315a3d0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22238) UID: 0 2024-09-21T15:01:32.684 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:32.684 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:32.694 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T15:01:32.747 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:01:32.746+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:01:32.804 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:32.804 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:32.815 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T15:01:32.848 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:01:32.846+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:01:32.924 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:32.924 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:32.934 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T15:01:32.949 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:01:32.950+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:01:33.043 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:33.112 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:33.114 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T15:01:33.114 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:01:33.114+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:01:33.214 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:01:33.208+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:01:33.228 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:33.228 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:33.238 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T15:01:33.315 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:01:33.312+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:01:33.363 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:33.363 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:33.374 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T15:01:33.404 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 5, 6, 7] out_osds: [4] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T15:01:33.404 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T15:01:33.404 INFO:tasks.thrashosds.thrasher:Killing osd 4, live_osds are [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T15:01:33.404 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-21T15:01:33.405 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T15:01:33.416 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:01:33.414+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:01:33.479 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:33.479 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:33.489 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T15:01:33.517 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:01:33.518+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:01:33.598 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:33.599 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:33.608 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T15:01:33.618 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:01:33.616+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:01:33.718 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:01:33.718+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:01:33.720 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:33.720 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:33.730 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T15:01:33.819 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:01:33.816+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:01:33.833 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:33.833 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:33.843 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T15:01:33.920 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:01:33.918+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:01:33.946 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:33.947 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:33.956 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T15:01:34.021 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:01:34.016+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:01:34.045 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T15:01:34.060 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:34.061 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:34.071 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T15:01:34.121 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:01:34.122+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:01:34.186 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:34.186 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:34.198 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T15:01:34.201 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T15:01:34.202 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T15:01:34.222 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:01:34.216+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:01:34.240 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T15:01:34.273 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T15:01:34.296 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T15:01:34.323 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:01:34.322+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:01:34.326 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:34.326 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T15:01:34.326 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T15:01:34.326 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T15:01:34.326 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.326 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:15130 3.2 3:47f1bbdb:::rbd_data.11361bed1f78.0000000000000067:head [sparse-read 0~147456] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-21T15:01:34.326 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:40.288284+0000", 2024-09-21T15:01:34.327 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 174.03753767800001, 2024-09-21T15:01:34.327 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1142935039999999, 2024-09-21T15:01:34.327 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:34.327 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:34.327 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:34.327 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:34.327 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:34.327 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 15130 2024-09-21T15:01:34.327 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.327 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:34.327 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.327 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:34.328 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288284+0000", 2024-09-21T15:01:34.328 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:34.328 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.328 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.328 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:34.328 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288284+0000", 2024-09-21T15:01:34.328 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:34.328 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.328 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.328 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:34.328 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288284+0000", 2024-09-21T15:01:34.328 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:34.328 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.329 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.329 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:34.329 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288288+0000", 2024-09-21T15:01:34.329 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5829999999999998e-06 2024-09-21T15:01:34.329 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.329 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.329 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:34.329 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288297+0000", 2024-09-21T15:01:34.329 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0669999999999998e-06 2024-09-21T15:01:34.329 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.329 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.329 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:34.329 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288305+0000", 2024-09-21T15:01:34.329 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8860000000000004e-06 2024-09-21T15:01:34.330 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.330 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.330 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:34.330 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288417+0000", 2024-09-21T15:01:34.330 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000112952 2024-09-21T15:01:34.330 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.330 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.330 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:34.330 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288428+0000", 2024-09-21T15:01:34.330 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0631e-05 2024-09-21T15:01:34.330 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.330 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.330 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:34.330 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.401876+0000", 2024-09-21T15:01:34.331 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1134479610000001 2024-09-21T15:01:34.331 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.331 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.331 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:34.331 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.402076+0000", 2024-09-21T15:01:34.331 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00019944200000000001 2024-09-21T15:01:34.331 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.331 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.331 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:34.331 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.402578+0000", 2024-09-21T15:01:34.331 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00050242899999999998 2024-09-21T15:01:34.331 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:34.331 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:34.331 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:34.332 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.332 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.332 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45708 3.2 3:415db91a:::rbd_data.1136ecbe5879.0000000000000000:28 [read 140288~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:34.332 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141801+0000", 2024-09-21T15:01:34.332 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 144.184021484, 2024-09-21T15:01:34.332 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034274200999999997, 2024-09-21T15:01:34.332 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:34.332 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:34.332 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:34.332 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:34.332 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:34.332 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45708 2024-09-21T15:01:34.332 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.332 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:34.333 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.333 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:34.333 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141801+0000", 2024-09-21T15:01:34.333 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:34.333 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.333 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.333 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:34.333 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141801+0000", 2024-09-21T15:01:34.333 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:34.333 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.333 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.333 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:34.333 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141800+0000", 2024-09-21T15:01:34.333 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:34.334 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.334 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.334 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:34.334 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141803+0000", 2024-09-21T15:01:34.334 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4210000000000002e-06 2024-09-21T15:01:34.334 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.334 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.334 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:34.334 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141812+0000", 2024-09-21T15:01:34.334 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.9800000000000004e-06 2024-09-21T15:01:34.334 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.334 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.334 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:34.334 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141816+0000", 2024-09-21T15:01:34.335 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4769999999999997e-06 2024-09-21T15:01:34.335 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.335 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.335 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:34.335 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.175865+0000", 2024-09-21T15:01:34.335 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034048875999999999 2024-09-21T15:01:34.335 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.335 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.335 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:34.335 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.175943+0000", 2024-09-21T15:01:34.335 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7464000000000002e-05 2024-09-21T15:01:34.335 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.335 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.335 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:34.336 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176075+0000", 2024-09-21T15:01:34.336 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013230800000000001 2024-09-21T15:01:34.336 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:34.336 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:34.336 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:34.336 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.336 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.336 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45709 3.2 3:415db91a:::rbd_data.1136ecbe5879.0000000000000000:28 [read 143360~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:34.336 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141820+0000", 2024-09-21T15:01:34.336 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 144.18400163699999, 2024-09-21T15:01:34.336 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034537536000000001, 2024-09-21T15:01:34.336 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:34.336 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:34.337 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:34.337 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:34.337 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:34.337 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45709 2024-09-21T15:01:34.337 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.337 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:34.337 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.337 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:34.337 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141820+0000", 2024-09-21T15:01:34.337 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:34.337 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.337 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.337 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:34.337 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141820+0000", 2024-09-21T15:01:34.337 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:34.338 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.338 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.338 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:34.338 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141820+0000", 2024-09-21T15:01:34.338 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:34.338 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.338 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.338 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:34.338 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141823+0000", 2024-09-21T15:01:34.338 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5110000000000002e-06 2024-09-21T15:01:34.338 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.338 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.338 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:34.339 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141829+0000", 2024-09-21T15:01:34.339 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0850000000000002e-06 2024-09-21T15:01:34.339 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.339 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.339 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:34.339 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141833+0000", 2024-09-21T15:01:34.339 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7620000000000003e-06 2024-09-21T15:01:34.339 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.339 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.339 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:34.339 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176103+0000", 2024-09-21T15:01:34.339 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034269933000000002 2024-09-21T15:01:34.339 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.339 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.339 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:34.340 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176161+0000", 2024-09-21T15:01:34.340 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7658999999999998e-05 2024-09-21T15:01:34.340 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.340 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.340 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:34.340 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176358+0000", 2024-09-21T15:01:34.340 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000196937 2024-09-21T15:01:34.340 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:34.340 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:34.340 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:34.340 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.340 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.340 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45710 3.2 3:415db91a:::rbd_data.1136ecbe5879.0000000000000000:28 [read 146432~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:34.341 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141838+0000", 2024-09-21T15:01:34.341 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 144.183984474, 2024-09-21T15:01:34.341 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034953674999999997, 2024-09-21T15:01:34.341 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:34.341 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:34.341 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:34.341 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:34.341 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:34.341 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45710 2024-09-21T15:01:34.341 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.341 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:34.341 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.341 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:34.341 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141838+0000", 2024-09-21T15:01:34.342 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:34.342 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.342 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.342 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:34.342 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141838+0000", 2024-09-21T15:01:34.342 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:34.342 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.342 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.342 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:34.342 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141837+0000", 2024-09-21T15:01:34.342 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:34.342 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.342 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.343 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:34.343 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141843+0000", 2024-09-21T15:01:34.343 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1619999999999996e-06 2024-09-21T15:01:34.343 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.343 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.343 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:34.343 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141850+0000", 2024-09-21T15:01:34.343 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4799999999999998e-06 2024-09-21T15:01:34.343 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.343 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.343 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:34.343 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141853+0000", 2024-09-21T15:01:34.343 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3050000000000001e-06 2024-09-21T15:01:34.343 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.344 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.344 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:34.344 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176490+0000", 2024-09-21T15:01:34.344 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034636413999999997 2024-09-21T15:01:34.344 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.344 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.344 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:34.344 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176606+0000", 2024-09-21T15:01:34.344 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000115841 2024-09-21T15:01:34.344 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.344 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.344 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:34.344 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176791+0000", 2024-09-21T15:01:34.344 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00018581500000000001 2024-09-21T15:01:34.345 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:34.345 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:34.345 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:34.345 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.345 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.345 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45711 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:34.345 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141856+0000", 2024-09-21T15:01:34.345 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 144.183966137, 2024-09-21T15:01:34.345 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034986308000000001, 2024-09-21T15:01:34.345 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:34.345 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:34.345 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:34.345 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:34.345 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:34.346 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45711 2024-09-21T15:01:34.346 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.346 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:34.346 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.346 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:34.346 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141856+0000", 2024-09-21T15:01:34.346 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:34.346 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.346 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.346 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:34.346 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141856+0000", 2024-09-21T15:01:34.346 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:34.346 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.347 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.347 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:34.347 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141856+0000", 2024-09-21T15:01:34.347 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:34.347 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.347 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.347 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:34.347 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141862+0000", 2024-09-21T15:01:34.347 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.567e-06 2024-09-21T15:01:34.347 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.347 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.347 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:34.347 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141868+0000", 2024-09-21T15:01:34.347 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9569999999999999e-06 2024-09-21T15:01:34.348 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.348 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.348 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:34.348 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141871+0000", 2024-09-21T15:01:34.348 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.146e-06 2024-09-21T15:01:34.348 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.348 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.348 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:34.348 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176831+0000", 2024-09-21T15:01:34.348 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034959935999999997 2024-09-21T15:01:34.348 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.348 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.348 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:34.348 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176842+0000", 2024-09-21T15:01:34.349 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1008e-05 2024-09-21T15:01:34.349 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:34.349 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:34.349 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:34.349 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.349 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.349 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45712 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:34.349 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141874+0000", 2024-09-21T15:01:34.349 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 144.18394824200001, 2024-09-21T15:01:34.349 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034993511999999997, 2024-09-21T15:01:34.349 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:34.349 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:34.349 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:34.349 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:34.350 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:34.350 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45712 2024-09-21T15:01:34.350 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.350 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:34.350 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.350 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:34.350 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141874+0000", 2024-09-21T15:01:34.350 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:34.350 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.350 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.350 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:34.350 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141874+0000", 2024-09-21T15:01:34.350 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:34.350 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.351 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.351 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:34.351 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141874+0000", 2024-09-21T15:01:34.351 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:34.351 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.351 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.351 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:34.351 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141875+0000", 2024-09-21T15:01:34.351 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5430000000000001e-06 2024-09-21T15:01:34.351 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.351 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.351 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:34.351 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141878+0000", 2024-09-21T15:01:34.351 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1420000000000002e-06 2024-09-21T15:01:34.352 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.352 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.352 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:34.352 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141881+0000", 2024-09-21T15:01:34.352 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.807e-06 2024-09-21T15:01:34.352 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.352 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.352 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:34.352 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176860+0000", 2024-09-21T15:01:34.352 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034979243 2024-09-21T15:01:34.352 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.352 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.352 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:34.352 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176867+0000", 2024-09-21T15:01:34.353 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0169999999999999e-06 2024-09-21T15:01:34.353 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:34.353 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:34.353 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:34.353 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.353 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.353 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45713 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:34.353 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141883+0000", 2024-09-21T15:01:34.353 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 144.18393866700001, 2024-09-21T15:01:34.353 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035010516999999998, 2024-09-21T15:01:34.353 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:34.353 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:34.353 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:34.353 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:34.354 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:34.354 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45713 2024-09-21T15:01:34.354 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.354 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:34.354 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.354 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:34.354 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141883+0000", 2024-09-21T15:01:34.354 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:34.354 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.354 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.354 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:34.354 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141883+0000", 2024-09-21T15:01:34.354 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:34.354 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.355 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.355 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:34.355 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141883+0000", 2024-09-21T15:01:34.355 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:34.355 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.355 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.355 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:34.355 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141885+0000", 2024-09-21T15:01:34.355 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4649999999999999e-06 2024-09-21T15:01:34.355 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.355 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.355 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:34.355 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141888+0000", 2024-09-21T15:01:34.356 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3890000000000001e-06 2024-09-21T15:01:34.356 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.356 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.356 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:34.356 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141891+0000", 2024-09-21T15:01:34.356 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9239999999999999e-06 2024-09-21T15:01:34.356 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.356 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.356 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:34.356 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176887+0000", 2024-09-21T15:01:34.356 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034996370999999998 2024-09-21T15:01:34.356 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.356 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.356 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:34.357 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176894+0000", 2024-09-21T15:01:34.357 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6069999999999996e-06 2024-09-21T15:01:34.357 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:34.357 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:34.357 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:34.357 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.357 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.357 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45714 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:34.357 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141893+0000", 2024-09-21T15:01:34.357 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 144.18392875200001, 2024-09-21T15:01:34.358 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035048215000000001, 2024-09-21T15:01:34.358 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:34.358 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:34.358 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:34.358 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:34.358 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:34.358 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45714 2024-09-21T15:01:34.358 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.358 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:34.358 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.358 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:34.358 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141893+0000", 2024-09-21T15:01:34.358 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:34.359 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.359 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.359 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:34.359 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141893+0000", 2024-09-21T15:01:34.359 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:34.359 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.359 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.359 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:34.359 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141893+0000", 2024-09-21T15:01:34.359 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:34.359 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.359 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.359 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:34.360 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141895+0000", 2024-09-21T15:01:34.360 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.703e-06 2024-09-21T15:01:34.360 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.360 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.360 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:34.360 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141900+0000", 2024-09-21T15:01:34.360 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.242e-06 2024-09-21T15:01:34.360 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.360 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.360 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:34.360 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141903+0000", 2024-09-21T15:01:34.360 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.117e-06 2024-09-21T15:01:34.360 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.360 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.361 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:34.361 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176930+0000", 2024-09-21T15:01:34.361 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035026750000000002 2024-09-21T15:01:34.361 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.361 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.361 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:34.361 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176942+0000", 2024-09-21T15:01:34.361 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1668000000000001e-05 2024-09-21T15:01:34.361 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:34.361 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:34.361 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:34.361 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.361 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.361 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45715 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:34.361 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141906+0000", 2024-09-21T15:01:34.362 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 144.18391659400001, 2024-09-21T15:01:34.362 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035066057999999997, 2024-09-21T15:01:34.362 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:34.362 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:34.362 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:34.362 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:34.362 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:34.362 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45715 2024-09-21T15:01:34.362 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.362 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:34.362 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.362 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:34.362 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141906+0000", 2024-09-21T15:01:34.362 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:34.363 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.363 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.363 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:34.363 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141906+0000", 2024-09-21T15:01:34.363 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:34.363 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.363 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.363 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:34.363 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141905+0000", 2024-09-21T15:01:34.363 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:34.363 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.363 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.363 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:34.363 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141907+0000", 2024-09-21T15:01:34.364 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5400000000000001e-06 2024-09-21T15:01:34.364 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.364 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.364 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:34.364 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141912+0000", 2024-09-21T15:01:34.364 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.502e-06 2024-09-21T15:01:34.364 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.364 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.364 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:34.364 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141915+0000", 2024-09-21T15:01:34.364 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0170000000000001e-06 2024-09-21T15:01:34.364 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.364 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.364 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:34.365 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176964+0000", 2024-09-21T15:01:34.365 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035048587999999999 2024-09-21T15:01:34.365 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.365 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.365 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:34.365 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176972+0000", 2024-09-21T15:01:34.365 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6580000000000002e-06 2024-09-21T15:01:34.365 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:34.365 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:34.365 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:34.365 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.365 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.365 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45716 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:34.365 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141918+0000", 2024-09-21T15:01:34.366 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 144.183904418, 2024-09-21T15:01:34.366 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035086223, 2024-09-21T15:01:34.366 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:34.366 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:34.366 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:34.366 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:34.366 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:34.366 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45716 2024-09-21T15:01:34.366 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.366 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:34.366 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.366 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:34.366 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141918+0000", 2024-09-21T15:01:34.367 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:34.367 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.367 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.367 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:34.367 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141918+0000", 2024-09-21T15:01:34.367 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:34.367 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.367 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.367 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:34.367 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141917+0000", 2024-09-21T15:01:34.367 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:34.367 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.367 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.367 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:34.367 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141919+0000", 2024-09-21T15:01:34.368 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5680000000000001e-06 2024-09-21T15:01:34.368 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.368 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.368 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:34.368 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141922+0000", 2024-09-21T15:01:34.368 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2789999999999999e-06 2024-09-21T15:01:34.368 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.368 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.368 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:34.368 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141925+0000", 2024-09-21T15:01:34.368 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9469999999999999e-06 2024-09-21T15:01:34.368 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.368 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.368 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:34.369 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176997+0000", 2024-09-21T15:01:34.369 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035071366999999999 2024-09-21T15:01:34.369 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.369 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.369 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:34.369 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177004+0000", 2024-09-21T15:01:34.369 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3309999999999998e-06 2024-09-21T15:01:34.369 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:34.369 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:34.369 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:34.369 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.369 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.369 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45717 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:34.369 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141928+0000", 2024-09-21T15:01:34.369 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 144.18389447199999, 2024-09-21T15:01:34.370 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035102573999999998, 2024-09-21T15:01:34.370 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:34.370 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:34.370 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:34.370 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:34.370 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:34.370 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45717 2024-09-21T15:01:34.370 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.370 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:34.370 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.370 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:34.370 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141928+0000", 2024-09-21T15:01:34.370 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:34.370 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.371 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.371 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:34.371 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141928+0000", 2024-09-21T15:01:34.371 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:34.371 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.371 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.371 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:34.371 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141927+0000", 2024-09-21T15:01:34.371 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:01:34.371 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.371 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.371 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:34.371 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141929+0000", 2024-09-21T15:01:34.371 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.533e-06 2024-09-21T15:01:34.372 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.372 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.372 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:34.372 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141934+0000", 2024-09-21T15:01:34.372 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1660000000000002e-06 2024-09-21T15:01:34.372 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.372 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.372 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:34.372 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141937+0000", 2024-09-21T15:01:34.372 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0309999999999999e-06 2024-09-21T15:01:34.372 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.372 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.372 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:34.372 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177023+0000", 2024-09-21T15:01:34.372 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035085595999999997 2024-09-21T15:01:34.373 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.373 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.373 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:34.373 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177030+0000", 2024-09-21T15:01:34.373 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.486e-06 2024-09-21T15:01:34.373 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:34.373 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:34.373 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:34.373 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.373 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.373 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45718 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:34.373 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141940+0000", 2024-09-21T15:01:34.373 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 144.183882614, 2024-09-21T15:01:34.373 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035124956999999998, 2024-09-21T15:01:34.374 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:34.374 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:34.374 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:34.374 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:34.374 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:34.374 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45718 2024-09-21T15:01:34.374 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.374 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:34.374 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.374 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:34.374 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141940+0000", 2024-09-21T15:01:34.374 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:34.374 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.374 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.375 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:34.375 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141940+0000", 2024-09-21T15:01:34.375 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:34.375 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.375 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.375 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:34.375 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141939+0000", 2024-09-21T15:01:34.375 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:34.375 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.375 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.375 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:34.375 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141941+0000", 2024-09-21T15:01:34.375 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.502e-06 2024-09-21T15:01:34.376 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.376 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.376 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:34.376 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141946+0000", 2024-09-21T15:01:34.376 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3449999999999996e-06 2024-09-21T15:01:34.376 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.376 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.376 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:34.376 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141949+0000", 2024-09-21T15:01:34.376 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9749999999999999e-06 2024-09-21T15:01:34.376 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.376 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.376 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:34.376 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177057+0000", 2024-09-21T15:01:34.376 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035108385999999998 2024-09-21T15:01:34.377 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.377 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.377 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:34.377 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177064+0000", 2024-09-21T15:01:34.377 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0099999999999998e-06 2024-09-21T15:01:34.377 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:34.377 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:34.377 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:34.377 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.377 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.377 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45719 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:34.377 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141952+0000", 2024-09-21T15:01:34.377 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 144.18387034099999, 2024-09-21T15:01:34.377 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035139108000000002, 2024-09-21T15:01:34.377 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:34.377 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:34.378 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:34.378 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:34.378 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:34.378 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45719 2024-09-21T15:01:34.378 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.378 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:34.378 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.378 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:34.378 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141952+0000", 2024-09-21T15:01:34.378 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:34.378 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.378 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.378 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:34.378 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141952+0000", 2024-09-21T15:01:34.378 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:34.379 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.379 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.379 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:34.379 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141952+0000", 2024-09-21T15:01:34.379 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:34.379 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.379 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.379 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:34.379 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141953+0000", 2024-09-21T15:01:34.379 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.486e-06 2024-09-21T15:01:34.379 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.379 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.379 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:34.379 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141959+0000", 2024-09-21T15:01:34.379 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4940000000000003e-06 2024-09-21T15:01:34.380 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.380 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.380 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:34.380 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141963+0000", 2024-09-21T15:01:34.380 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3540000000000002e-06 2024-09-21T15:01:34.380 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.380 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.380 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:34.380 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177084+0000", 2024-09-21T15:01:34.380 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035120983000000001 2024-09-21T15:01:34.380 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.380 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.380 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:34.380 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177091+0000", 2024-09-21T15:01:34.380 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0350000000000004e-06 2024-09-21T15:01:34.380 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:34.381 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:34.381 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:34.381 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.381 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.381 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45720 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:34.381 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141965+0000", 2024-09-21T15:01:34.381 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 144.18385687400001, 2024-09-21T15:01:34.381 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035155268000000003, 2024-09-21T15:01:34.381 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:34.381 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:34.381 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:34.381 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:34.381 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:34.381 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45720 2024-09-21T15:01:34.381 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.381 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:34.382 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.382 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:34.382 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141965+0000", 2024-09-21T15:01:34.382 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:34.382 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.382 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.382 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:34.382 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141965+0000", 2024-09-21T15:01:34.382 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:34.382 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.382 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.382 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:34.382 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141965+0000", 2024-09-21T15:01:34.382 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:34.382 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.383 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.383 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:34.383 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141966+0000", 2024-09-21T15:01:34.383 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.46e-06 2024-09-21T15:01:34.383 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.383 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.383 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:34.383 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141972+0000", 2024-09-21T15:01:34.383 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4210000000000003e-06 2024-09-21T15:01:34.383 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.383 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.383 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:34.383 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141975+0000", 2024-09-21T15:01:34.384 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.948e-06 2024-09-21T15:01:34.384 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.384 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.384 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:34.384 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177113+0000", 2024-09-21T15:01:34.384 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035137855000000003 2024-09-21T15:01:34.384 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.384 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.384 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:34.384 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177121+0000", 2024-09-21T15:01:34.384 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8269999999999998e-06 2024-09-21T15:01:34.384 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:34.384 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:34.384 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:34.385 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.385 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.385 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45721 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:34.385 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141977+0000", 2024-09-21T15:01:34.385 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 144.183844851, 2024-09-21T15:01:34.385 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035169984000000001, 2024-09-21T15:01:34.385 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:34.385 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:34.385 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:34.385 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:34.385 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:34.385 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45721 2024-09-21T15:01:34.385 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.385 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:34.386 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.386 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:34.386 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141977+0000", 2024-09-21T15:01:34.386 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:34.386 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.386 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.386 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:34.386 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141977+0000", 2024-09-21T15:01:34.386 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:34.386 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.386 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.386 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:34.386 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141977+0000", 2024-09-21T15:01:34.387 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:34.387 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.387 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.387 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:34.387 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141979+0000", 2024-09-21T15:01:34.387 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.511e-06 2024-09-21T15:01:34.387 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.387 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.387 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:34.387 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141982+0000", 2024-09-21T15:01:34.387 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4180000000000001e-06 2024-09-21T15:01:34.387 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.387 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.388 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:34.388 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141985+0000", 2024-09-21T15:01:34.388 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9450000000000002e-06 2024-09-21T15:01:34.388 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.388 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.388 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:34.388 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177139+0000", 2024-09-21T15:01:34.388 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035154409999999997 2024-09-21T15:01:34.388 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.388 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.388 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:34.388 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177147+0000", 2024-09-21T15:01:34.388 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9570000000000002e-06 2024-09-21T15:01:34.388 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:34.389 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:34.389 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:34.389 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.389 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.389 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45722 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:34.389 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141987+0000", 2024-09-21T15:01:34.389 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 144.18383488000001, 2024-09-21T15:01:34.389 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035188492000000002, 2024-09-21T15:01:34.389 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:34.389 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:34.389 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:34.389 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:34.389 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:34.390 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45722 2024-09-21T15:01:34.390 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.390 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:34.390 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.390 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:34.390 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141987+0000", 2024-09-21T15:01:34.390 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:34.390 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.390 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.390 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:34.390 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141987+0000", 2024-09-21T15:01:34.390 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:34.390 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.391 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.391 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:34.391 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141987+0000", 2024-09-21T15:01:34.391 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:34.391 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.391 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.391 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:34.391 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141988+0000", 2024-09-21T15:01:34.391 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.463e-06 2024-09-21T15:01:34.391 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.391 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.391 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:34.391 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141994+0000", 2024-09-21T15:01:34.392 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.361e-06 2024-09-21T15:01:34.392 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.392 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.392 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:34.392 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141997+0000", 2024-09-21T15:01:34.392 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0359999999999998e-06 2024-09-21T15:01:34.392 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.392 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.392 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:34.392 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177169+0000", 2024-09-21T15:01:34.392 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035171651999999998 2024-09-21T15:01:34.392 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.392 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.392 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:34.393 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177176+0000", 2024-09-21T15:01:34.393 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2189999999999998e-06 2024-09-21T15:01:34.393 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:34.393 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:34.393 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:34.393 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.393 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.393 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45723 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:34.393 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141999+0000", 2024-09-21T15:01:34.393 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 144.18382271300001, 2024-09-21T15:01:34.393 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035197701999999997, 2024-09-21T15:01:34.393 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:34.393 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:34.393 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:34.394 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:34.394 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:34.394 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45723 2024-09-21T15:01:34.394 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.394 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:34.394 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.394 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:34.394 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141999+0000", 2024-09-21T15:01:34.394 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:34.394 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.394 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.394 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:34.395 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141999+0000", 2024-09-21T15:01:34.395 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:34.395 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.395 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.395 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:34.395 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141999+0000", 2024-09-21T15:01:34.395 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:34.395 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.395 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.395 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:34.395 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142003+0000", 2024-09-21T15:01:34.395 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5049999999999999e-06 2024-09-21T15:01:34.395 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.395 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.396 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:34.396 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142008+0000", 2024-09-21T15:01:34.396 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6579999999999999e-06 2024-09-21T15:01:34.396 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.396 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.396 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:34.396 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142011+0000", 2024-09-21T15:01:34.396 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0529999999999998e-06 2024-09-21T15:01:34.396 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.396 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.396 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:34.396 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177194+0000", 2024-09-21T15:01:34.396 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035182221999999999 2024-09-21T15:01:34.397 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.397 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.397 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:34.397 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177197+0000", 2024-09-21T15:01:34.397 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5030000000000002e-06 2024-09-21T15:01:34.397 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:34.397 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:34.397 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:34.397 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.397 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.397 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45724 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:34.397 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.142015+0000", 2024-09-21T15:01:34.398 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 144.18380674599999, 2024-09-21T15:01:34.398 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035195345000000003, 2024-09-21T15:01:34.398 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:34.398 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:34.398 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:34.398 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:34.398 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:34.398 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45724 2024-09-21T15:01:34.398 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.398 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:34.398 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.398 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:34.398 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142015+0000", 2024-09-21T15:01:34.398 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:34.399 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.399 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.399 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:34.399 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142015+0000", 2024-09-21T15:01:34.399 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:34.399 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.399 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.399 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:34.399 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142015+0000", 2024-09-21T15:01:34.399 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:34.399 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.399 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.399 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:34.399 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142017+0000", 2024-09-21T15:01:34.400 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5540000000000001e-06 2024-09-21T15:01:34.400 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.400 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.400 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:34.400 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142022+0000", 2024-09-21T15:01:34.400 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4229999999999996e-06 2024-09-21T15:01:34.400 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.400 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.400 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:34.400 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142025+0000", 2024-09-21T15:01:34.400 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.056e-06 2024-09-21T15:01:34.400 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.400 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.400 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:34.400 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177207+0000", 2024-09-21T15:01:34.400 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035182089 2024-09-21T15:01:34.401 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.401 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.401 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:34.401 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177211+0000", 2024-09-21T15:01:34.401 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4649999999999999e-06 2024-09-21T15:01:34.401 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:34.401 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:34.401 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:34.401 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.401 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.401 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45725 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:34.401 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.142028+0000", 2024-09-21T15:01:34.401 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 144.18379446700001, 2024-09-21T15:01:34.401 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035195731000000001, 2024-09-21T15:01:34.401 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:34.401 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:34.402 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:34.402 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:34.402 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:34.402 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45725 2024-09-21T15:01:34.402 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.402 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:34.402 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.402 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:34.402 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142028+0000", 2024-09-21T15:01:34.402 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:34.402 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.402 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.402 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:34.402 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142028+0000", 2024-09-21T15:01:34.402 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:34.403 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.403 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.403 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:34.403 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142027+0000", 2024-09-21T15:01:34.403 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:34.403 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.403 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.403 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:34.403 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142029+0000", 2024-09-21T15:01:34.403 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5269999999999999e-06 2024-09-21T15:01:34.403 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.403 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.403 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:34.403 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142032+0000", 2024-09-21T15:01:34.403 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3249999999999999e-06 2024-09-21T15:01:34.404 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.404 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.404 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:34.404 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142035+0000", 2024-09-21T15:01:34.404 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.897e-06 2024-09-21T15:01:34.404 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.404 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.404 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:34.404 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177220+0000", 2024-09-21T15:01:34.404 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035184871 2024-09-21T15:01:34.404 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.404 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.404 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:34.404 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177223+0000", 2024-09-21T15:01:34.404 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3680000000000002e-06 2024-09-21T15:01:34.405 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:34.405 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:34.405 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:34.405 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.405 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.405 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45726 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:34.405 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.142037+0000", 2024-09-21T15:01:34.405 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 144.18378475200001, 2024-09-21T15:01:34.405 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035199273000000003, 2024-09-21T15:01:34.405 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:34.405 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:34.405 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:34.405 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:34.405 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:34.405 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45726 2024-09-21T15:01:34.405 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.406 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:34.406 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.406 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:34.406 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142037+0000", 2024-09-21T15:01:34.406 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:34.406 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.406 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.406 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:34.406 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142037+0000", 2024-09-21T15:01:34.406 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:34.406 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.406 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.406 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:34.406 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142037+0000", 2024-09-21T15:01:34.407 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:34.407 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.407 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.407 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:34.407 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142038+0000", 2024-09-21T15:01:34.407 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.344e-06 2024-09-21T15:01:34.407 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.407 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.407 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:34.407 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142044+0000", 2024-09-21T15:01:34.407 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1930000000000001e-06 2024-09-21T15:01:34.407 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.407 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.407 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:34.407 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142047+0000", 2024-09-21T15:01:34.408 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0280000000000001e-06 2024-09-21T15:01:34.408 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.408 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.408 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:34.408 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177233+0000", 2024-09-21T15:01:34.408 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035186394000000003 2024-09-21T15:01:34.408 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.408 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.408 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:34.408 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177237+0000", 2024-09-21T15:01:34.408 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5530000000000001e-06 2024-09-21T15:01:34.408 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:34.408 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:34.408 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:34.408 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:34.409 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:34.409 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:34.410 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T15:01:34.453 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:01:34.454+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:01:34.525 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:01:34.520+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:01:34.560 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:34.560 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:01:34.560 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T15:01:34.560 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:34.570 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T15:01:34.626 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:01:34.626+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:01:34.680 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:34.681 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:01:34.681 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T15:01:34.681 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T15:01:34.681 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:34.690 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T15:01:34.727 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:01:34.724+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:01:34.799 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:34.799 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T15:01:34.799 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T15:01:34.799 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T15:01:34.800 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.800 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139776 3.16 3:6c38d0e2:::rbd_data.113664780c85.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:34.800 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.223442+0000", 2024-09-21T15:01:34.800 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 32.573795998999998, 2024-09-21T15:01:34.800 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0315488150000001, 2024-09-21T15:01:34.800 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:34.800 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:34.800 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:34.800 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:34.800 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:34.800 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139776 2024-09-21T15:01:34.800 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.800 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:34.800 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.801 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:34.801 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223442+0000", 2024-09-21T15:01:34.801 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:34.801 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.801 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.801 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:34.801 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223442+0000", 2024-09-21T15:01:34.801 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:34.801 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.801 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.801 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:34.801 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223442+0000", 2024-09-21T15:01:34.801 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:34.801 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.802 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.802 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:34.802 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223446+0000", 2024-09-21T15:01:34.802 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2549999999999999e-06 2024-09-21T15:01:34.802 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.802 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.802 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:34.802 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223454+0000", 2024-09-21T15:01:34.802 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8199999999999997e-06 2024-09-21T15:01:34.802 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.802 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.802 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:34.802 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223460+0000", 2024-09-21T15:01:34.802 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6330000000000001e-06 2024-09-21T15:01:34.803 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.803 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.803 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:34.803 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223542+0000", 2024-09-21T15:01:34.803 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1929999999999997e-05 2024-09-21T15:01:34.803 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.803 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.803 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:34.803 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223552+0000", 2024-09-21T15:01:34.803 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0627e-05 2024-09-21T15:01:34.803 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.803 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.803 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:34.803 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.254634+0000", 2024-09-21T15:01:34.804 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.031081972 2024-09-21T15:01:34.804 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.804 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.804 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:34.804 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.254796+0000", 2024-09-21T15:01:34.804 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000161879 2024-09-21T15:01:34.804 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.804 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.804 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:34.804 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.254991+0000", 2024-09-21T15:01:34.804 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00019526499999999999 2024-09-21T15:01:34.804 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:34.804 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:34.805 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:34.805 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.805 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.805 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139785 3.11 3:8abbb109:::rbd_data.113664780c85.0000000000000070:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:34.805 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.223531+0000", 2024-09-21T15:01:34.805 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 32.573707257999999, 2024-09-21T15:01:34.805 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0187143510000001, 2024-09-21T15:01:34.805 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:34.805 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:34.805 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:34.805 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:34.805 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:34.805 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139785 2024-09-21T15:01:34.805 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.805 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:34.806 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.806 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:34.806 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223531+0000", 2024-09-21T15:01:34.806 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:34.806 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.806 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.806 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:34.806 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223531+0000", 2024-09-21T15:01:34.806 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:34.806 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.806 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.806 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:34.806 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223531+0000", 2024-09-21T15:01:34.807 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:34.807 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.807 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.807 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:34.807 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223534+0000", 2024-09-21T15:01:34.807 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2650000000000001e-06 2024-09-21T15:01:34.807 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.807 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.807 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:34.807 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223541+0000", 2024-09-21T15:01:34.807 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8990000000000004e-06 2024-09-21T15:01:34.807 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.807 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.807 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:34.808 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223546+0000", 2024-09-21T15:01:34.808 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3399999999999997e-06 2024-09-21T15:01:34.808 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.808 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.808 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:34.808 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223724+0000", 2024-09-21T15:01:34.808 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017745799999999999 2024-09-21T15:01:34.808 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.808 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.808 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:34.808 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223735+0000", 2024-09-21T15:01:34.808 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1610999999999999e-05 2024-09-21T15:01:34.808 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.808 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.808 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:34.809 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242113+0000", 2024-09-21T15:01:34.809 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.018377879 2024-09-21T15:01:34.809 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.809 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.809 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:34.809 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242246+0000", 2024-09-21T15:01:34.809 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013231500000000001 2024-09-21T15:01:34.809 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:34.809 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:34.809 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:34.809 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.809 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.809 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139790 3.11 3:8a18c7d8:::rbd_data.113664780c85.000000000000008d:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:34.809 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.223578+0000", 2024-09-21T15:01:34.810 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 32.573660263000001, 2024-09-21T15:01:34.810 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0187879719999999, 2024-09-21T15:01:34.810 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:34.810 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:34.810 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:34.810 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:34.810 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:34.810 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139790 2024-09-21T15:01:34.810 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.810 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:34.810 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.810 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:34.810 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223578+0000", 2024-09-21T15:01:34.810 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:34.810 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.811 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.811 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:34.811 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223578+0000", 2024-09-21T15:01:34.811 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:34.811 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.811 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.811 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:34.811 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223578+0000", 2024-09-21T15:01:34.811 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:34.811 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.811 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.811 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:34.811 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223581+0000", 2024-09-21T15:01:34.811 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6879999999999999e-06 2024-09-21T15:01:34.811 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.811 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.812 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:34.812 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223586+0000", 2024-09-21T15:01:34.812 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7080000000000002e-06 2024-09-21T15:01:34.812 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.812 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.812 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:34.812 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223591+0000", 2024-09-21T15:01:34.812 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5639999999999998e-06 2024-09-21T15:01:34.812 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.812 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.812 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:34.812 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223761+0000", 2024-09-21T15:01:34.812 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000170213 2024-09-21T15:01:34.812 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.812 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.813 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:34.813 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223769+0000", 2024-09-21T15:01:34.813 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7049999999999999e-06 2024-09-21T15:01:34.813 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.813 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.813 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:34.813 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242269+0000", 2024-09-21T15:01:34.813 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.018500288 2024-09-21T15:01:34.813 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.813 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.813 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:34.813 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242366+0000", 2024-09-21T15:01:34.813 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.7126000000000005e-05 2024-09-21T15:01:34.813 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:34.813 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:34.814 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:34.814 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.814 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.814 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139792 3.16 3:6ce5caf3:::rbd_data.113664780c85.000000000000009b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:34.814 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.223595+0000", 2024-09-21T15:01:34.814 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 32.573643310000001, 2024-09-21T15:01:34.814 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0317517899999999, 2024-09-21T15:01:34.814 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:34.814 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:34.814 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:34.814 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:34.815 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:34.815 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139792 2024-09-21T15:01:34.815 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.815 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:34.815 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.815 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:34.815 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223595+0000", 2024-09-21T15:01:34.815 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:34.815 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.815 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.815 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:34.815 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223595+0000", 2024-09-21T15:01:34.815 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:34.815 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.815 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.816 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:34.816 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223595+0000", 2024-09-21T15:01:34.816 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:34.816 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.816 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.816 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:34.816 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223598+0000", 2024-09-21T15:01:34.816 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4620000000000001e-06 2024-09-21T15:01:34.816 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.816 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.816 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:34.816 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223604+0000", 2024-09-21T15:01:34.816 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6879999999999996e-06 2024-09-21T15:01:34.816 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.817 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.817 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:34.817 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223609+0000", 2024-09-21T15:01:34.817 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9490000000000004e-06 2024-09-21T15:01:34.817 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.817 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.817 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:34.817 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224067+0000", 2024-09-21T15:01:34.817 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00045780700000000002 2024-09-21T15:01:34.817 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.817 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.817 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:34.817 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224075+0000", 2024-09-21T15:01:34.817 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6610000000000006e-06 2024-09-21T15:01:34.817 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.818 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.818 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:34.818 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255141+0000", 2024-09-21T15:01:34.818 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0310657480000001 2024-09-21T15:01:34.818 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.818 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.818 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:34.818 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255347+0000", 2024-09-21T15:01:34.818 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000205837 2024-09-21T15:01:34.818 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:34.818 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:34.818 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:34.818 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.819 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.819 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139794 3.11 3:8cf3a2f4:::rbd_data.113664780c85.00000000000000a9:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:34.819 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.224011+0000", 2024-09-21T15:01:34.819 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 32.573227281999998, 2024-09-21T15:01:34.819 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.018507791, 2024-09-21T15:01:34.819 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:34.819 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:34.819 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:34.819 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:34.819 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:34.819 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139794 2024-09-21T15:01:34.819 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.819 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:34.819 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.820 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:34.820 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224011+0000", 2024-09-21T15:01:34.820 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:34.820 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.820 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.820 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:34.820 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224011+0000", 2024-09-21T15:01:34.820 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:34.820 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.820 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.820 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:34.820 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224010+0000", 2024-09-21T15:01:34.820 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:34.820 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.820 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.820 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:34.821 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224016+0000", 2024-09-21T15:01:34.821 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6550000000000001e-06 2024-09-21T15:01:34.821 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.821 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.821 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:34.821 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224028+0000", 2024-09-21T15:01:34.821 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2422999999999999e-05 2024-09-21T15:01:34.821 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.821 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.821 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:34.821 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224036+0000", 2024-09-21T15:01:34.821 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7749999999999993e-06 2024-09-21T15:01:34.821 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.821 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.821 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:34.821 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225176+0000", 2024-09-21T15:01:34.822 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0011401740000000001 2024-09-21T15:01:34.822 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.822 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.822 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:34.822 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225187+0000", 2024-09-21T15:01:34.822 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0951999999999999e-05 2024-09-21T15:01:34.822 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.822 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.822 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:34.822 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242392+0000", 2024-09-21T15:01:34.822 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.017204905 2024-09-21T15:01:34.822 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.822 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.822 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:34.822 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242519+0000", 2024-09-21T15:01:34.823 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012680499999999999 2024-09-21T15:01:34.823 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:34.823 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:34.823 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:34.823 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.823 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.823 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139800 3.16 3:68b050ed:::rbd_data.113664780c85.00000000000000d3:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:34.823 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.224579+0000", 2024-09-21T15:01:34.823 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 32.572659522999999, 2024-09-21T15:01:34.823 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.030932637, 2024-09-21T15:01:34.823 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:34.823 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:34.823 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:34.823 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:34.824 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:34.824 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139800 2024-09-21T15:01:34.824 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.824 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:34.824 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.824 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:34.824 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224579+0000", 2024-09-21T15:01:34.824 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:34.824 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.824 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.824 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:34.824 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224579+0000", 2024-09-21T15:01:34.824 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:34.824 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.824 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.825 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:34.825 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224578+0000", 2024-09-21T15:01:34.825 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:34.825 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.825 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.825 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:34.825 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224582+0000", 2024-09-21T15:01:34.825 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0029999999999999e-06 2024-09-21T15:01:34.825 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.825 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.825 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:34.825 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224588+0000", 2024-09-21T15:01:34.825 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8120000000000002e-06 2024-09-21T15:01:34.825 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.825 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.826 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:34.826 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224593+0000", 2024-09-21T15:01:34.826 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7469999999999996e-06 2024-09-21T15:01:34.826 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.826 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.826 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:34.826 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224648+0000", 2024-09-21T15:01:34.826 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4693e-05 2024-09-21T15:01:34.826 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.826 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.826 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:34.826 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224656+0000", 2024-09-21T15:01:34.826 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8429999999999993e-06 2024-09-21T15:01:34.826 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.826 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.827 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:34.827 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255382+0000", 2024-09-21T15:01:34.827 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0307266070000001 2024-09-21T15:01:34.827 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.827 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.827 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:34.827 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255512+0000", 2024-09-21T15:01:34.827 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000129367 2024-09-21T15:01:34.827 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:34.827 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:34.827 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:34.827 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.827 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.827 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139804 3.11 3:88f9ddce:::rbd_data.113664780c85.00000000000000ef:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:34.828 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.224632+0000", 2024-09-21T15:01:34.828 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 32.572606542000003, 2024-09-21T15:01:34.828 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.018021302, 2024-09-21T15:01:34.828 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:34.828 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:34.828 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:34.828 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:34.828 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:34.828 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139804 2024-09-21T15:01:34.828 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.828 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:34.828 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.828 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:34.828 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224632+0000", 2024-09-21T15:01:34.828 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:34.829 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.829 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.829 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:34.829 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224632+0000", 2024-09-21T15:01:34.829 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:34.829 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.829 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.829 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:34.829 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224632+0000", 2024-09-21T15:01:34.829 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:34.829 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.829 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.829 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:34.829 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224635+0000", 2024-09-21T15:01:34.830 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2710000000000001e-06 2024-09-21T15:01:34.830 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.830 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.830 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:34.830 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224640+0000", 2024-09-21T15:01:34.830 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6779999999999997e-06 2024-09-21T15:01:34.830 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.830 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.830 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:34.830 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224645+0000", 2024-09-21T15:01:34.830 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9339999999999997e-06 2024-09-21T15:01:34.830 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.830 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.831 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:34.831 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225307+0000", 2024-09-21T15:01:34.831 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00066201699999999999 2024-09-21T15:01:34.831 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.831 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.831 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:34.831 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225312+0000", 2024-09-21T15:01:34.831 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9799999999999998e-06 2024-09-21T15:01:34.831 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.831 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.831 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:34.831 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242548+0000", 2024-09-21T15:01:34.831 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.017235603 2024-09-21T15:01:34.831 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.832 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.832 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:34.832 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242653+0000", 2024-09-21T15:01:34.832 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010522 2024-09-21T15:01:34.832 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:34.832 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:34.832 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:34.832 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.832 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.832 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139809 3.11 3:884da946:::rbd_data.113664780c85.0000000000000118:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:34.832 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.224667+0000", 2024-09-21T15:01:34.832 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 32.572571498000002, 2024-09-21T15:01:34.832 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0181101960000001, 2024-09-21T15:01:34.832 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:34.832 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:34.833 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:34.833 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:34.833 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:34.833 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139809 2024-09-21T15:01:34.833 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.833 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:34.833 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.833 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:34.833 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224667+0000", 2024-09-21T15:01:34.833 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:34.833 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.833 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.833 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:34.833 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224667+0000", 2024-09-21T15:01:34.834 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:34.834 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.834 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.834 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:34.834 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224667+0000", 2024-09-21T15:01:34.834 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:34.834 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.834 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.834 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:34.834 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224669+0000", 2024-09-21T15:01:34.834 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.452e-06 2024-09-21T15:01:34.834 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.834 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.834 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:34.835 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224675+0000", 2024-09-21T15:01:34.835 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.818e-06 2024-09-21T15:01:34.835 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.835 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.835 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:34.835 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224680+0000", 2024-09-21T15:01:34.835 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6319999999999999e-06 2024-09-21T15:01:34.835 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.835 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.835 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:34.835 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225400+0000", 2024-09-21T15:01:34.835 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00072012699999999999 2024-09-21T15:01:34.835 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.835 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.836 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:34.836 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225409+0000", 2024-09-21T15:01:34.836 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1039999999999996e-06 2024-09-21T15:01:34.836 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.836 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.836 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:34.836 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242675+0000", 2024-09-21T15:01:34.836 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0172663319999999 2024-09-21T15:01:34.836 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.836 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.836 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:34.836 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242777+0000", 2024-09-21T15:01:34.836 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010197899999999999 2024-09-21T15:01:34.836 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:34.837 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:34.837 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:34.837 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.837 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.837 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139868 3.16 3:69f82090:::rbd_data.113664780c85.00000000000000f1:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:34.837 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.225982+0000", 2024-09-21T15:01:34.837 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 32.571255997999998, 2024-09-21T15:01:34.837 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0296791729999999, 2024-09-21T15:01:34.837 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:34.837 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:34.837 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:34.837 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:34.837 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:34.837 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139868 2024-09-21T15:01:34.838 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.838 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:34.838 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.838 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:34.838 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225982+0000", 2024-09-21T15:01:34.838 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:34.838 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.838 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.838 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:34.838 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225982+0000", 2024-09-21T15:01:34.838 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:34.838 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.838 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.838 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:34.839 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225982+0000", 2024-09-21T15:01:34.839 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:34.839 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.839 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.839 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:34.839 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225985+0000", 2024-09-21T15:01:34.839 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.74e-06 2024-09-21T15:01:34.839 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.839 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.839 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:34.839 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225990+0000", 2024-09-21T15:01:34.839 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7039999999999999e-06 2024-09-21T15:01:34.839 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.840 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.840 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:34.840 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225995+0000", 2024-09-21T15:01:34.840 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7249999999999997e-06 2024-09-21T15:01:34.840 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.840 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.840 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:34.840 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226299+0000", 2024-09-21T15:01:34.840 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00030383199999999997 2024-09-21T15:01:34.840 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.840 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.840 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:34.841 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226308+0000", 2024-09-21T15:01:34.841 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.9740000000000008e-06 2024-09-21T15:01:34.841 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.841 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.841 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:34.841 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255539+0000", 2024-09-21T15:01:34.841 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0292305960000001 2024-09-21T15:01:34.841 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.841 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.841 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:34.841 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255662+0000", 2024-09-21T15:01:34.841 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012307400000000001 2024-09-21T15:01:34.841 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:34.841 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:34.842 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:34.842 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.842 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.842 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139869 3.16 3:6b6c90a4:::rbd_data.113664780c85.00000000000000fe:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:34.842 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.226006+0000", 2024-09-21T15:01:34.842 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 32.571232381999998, 2024-09-21T15:01:34.842 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.029790518, 2024-09-21T15:01:34.842 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:34.842 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:34.842 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:34.842 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:34.842 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:34.842 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139869 2024-09-21T15:01:34.843 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.843 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:34.843 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.843 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:34.843 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226006+0000", 2024-09-21T15:01:34.843 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:34.843 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.843 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.843 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:34.843 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226006+0000", 2024-09-21T15:01:34.843 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:34.843 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.844 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.844 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:34.844 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226006+0000", 2024-09-21T15:01:34.844 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:34.844 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.844 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.844 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:34.844 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226008+0000", 2024-09-21T15:01:34.844 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5569999999999998e-06 2024-09-21T15:01:34.844 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.844 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.844 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:34.845 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226014+0000", 2024-09-21T15:01:34.845 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2079999999999997e-06 2024-09-21T15:01:34.845 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.845 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.845 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:34.845 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226020+0000", 2024-09-21T15:01:34.845 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7429999999999999e-06 2024-09-21T15:01:34.845 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.845 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.845 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:34.845 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226330+0000", 2024-09-21T15:01:34.845 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00030963300000000002 2024-09-21T15:01:34.845 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.845 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.846 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:34.846 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226336+0000", 2024-09-21T15:01:34.846 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6340000000000003e-06 2024-09-21T15:01:34.846 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.846 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.846 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:34.846 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255687+0000", 2024-09-21T15:01:34.846 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0293507150000001 2024-09-21T15:01:34.846 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.846 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.846 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:34.846 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255797+0000", 2024-09-21T15:01:34.846 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000109471 2024-09-21T15:01:34.847 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:34.847 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:34.847 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:34.847 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.847 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.847 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139884 3.16 3:69f10324:::rbd_data.113664780c85.0000000000000161:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:34.847 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.227573+0000", 2024-09-21T15:01:34.847 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 32.569665741000001, 2024-09-21T15:01:34.847 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0283568759999999, 2024-09-21T15:01:34.847 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:34.847 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:34.847 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:34.847 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:34.848 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:34.848 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139884 2024-09-21T15:01:34.848 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.848 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:34.848 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.848 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:34.848 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227573+0000", 2024-09-21T15:01:34.848 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:34.848 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.848 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.848 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:34.848 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227573+0000", 2024-09-21T15:01:34.848 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:34.848 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.848 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.849 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:34.849 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227572+0000", 2024-09-21T15:01:34.849 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:34.849 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.849 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.849 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:34.849 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227575+0000", 2024-09-21T15:01:34.849 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7769999999999998e-06 2024-09-21T15:01:34.849 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.849 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.849 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:34.849 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227581+0000", 2024-09-21T15:01:34.849 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7359999999999998e-06 2024-09-21T15:01:34.849 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.850 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.850 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:34.850 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227585+0000", 2024-09-21T15:01:34.850 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5090000000000004e-06 2024-09-21T15:01:34.850 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.850 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.850 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:34.850 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227753+0000", 2024-09-21T15:01:34.850 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00016761599999999999 2024-09-21T15:01:34.850 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.850 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.850 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:34.850 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227761+0000", 2024-09-21T15:01:34.850 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0579999999999998e-06 2024-09-21T15:01:34.851 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.851 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.851 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:34.851 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255828+0000", 2024-09-21T15:01:34.851 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0280671349999999 2024-09-21T15:01:34.851 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.851 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.851 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:34.851 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255930+0000", 2024-09-21T15:01:34.851 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000101372 2024-09-21T15:01:34.851 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:34.851 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:34.851 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:34.851 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.852 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.852 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139913 3.16 3:6f81b46d:::rbd_data.113664780c85.0000000000000074:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:34.852 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.227685+0000", 2024-09-21T15:01:34.852 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 32.569553562000003, 2024-09-21T15:01:34.852 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0283644750000001, 2024-09-21T15:01:34.852 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:34.852 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:34.852 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:34.852 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:34.852 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:34.852 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139913 2024-09-21T15:01:34.852 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.852 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:34.852 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.852 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:34.853 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227685+0000", 2024-09-21T15:01:34.853 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:34.853 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.853 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.853 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:34.853 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227685+0000", 2024-09-21T15:01:34.853 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:34.853 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.853 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.853 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:34.853 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227685+0000", 2024-09-21T15:01:34.853 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:34.853 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.853 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.854 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:34.854 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227687+0000", 2024-09-21T15:01:34.854 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2929999999999999e-06 2024-09-21T15:01:34.854 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.854 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.854 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:34.854 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227693+0000", 2024-09-21T15:01:34.854 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1070000000000001e-06 2024-09-21T15:01:34.854 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.854 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.854 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:34.854 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227698+0000", 2024-09-21T15:01:34.854 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9339999999999997e-06 2024-09-21T15:01:34.855 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.855 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.855 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:34.855 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227784+0000", 2024-09-21T15:01:34.855 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6096999999999999e-05 2024-09-21T15:01:34.855 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.855 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.855 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:34.855 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227791+0000", 2024-09-21T15:01:34.855 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9090000000000003e-06 2024-09-21T15:01:34.855 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.855 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.855 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:34.855 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255955+0000", 2024-09-21T15:01:34.856 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0281643579999999 2024-09-21T15:01:34.856 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.856 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.856 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:34.856 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256049+0000", 2024-09-21T15:01:34.856 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4047000000000003e-05 2024-09-21T15:01:34.856 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:34.856 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:34.856 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:34.856 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.856 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.856 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139988 3.11 3:88a29fa4:::rbd_data.113664780c85.00000000000000bd:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:34.856 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.229772+0000", 2024-09-21T15:01:34.856 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 32.567466062999998, 2024-09-21T15:01:34.856 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.017967668, 2024-09-21T15:01:34.857 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:34.857 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:34.857 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:34.857 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:34.857 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:34.857 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139988 2024-09-21T15:01:34.857 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.857 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:34.857 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.857 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:34.857 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229772+0000", 2024-09-21T15:01:34.857 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:34.857 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.857 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.858 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:34.858 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229772+0000", 2024-09-21T15:01:34.858 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:34.858 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.858 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.858 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:34.858 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229772+0000", 2024-09-21T15:01:34.858 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:34.858 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.858 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.858 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:34.858 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229776+0000", 2024-09-21T15:01:34.858 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6890000000000002e-06 2024-09-21T15:01:34.859 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.859 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.859 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:34.859 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229783+0000", 2024-09-21T15:01:34.859 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2280000000000001e-06 2024-09-21T15:01:34.859 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.859 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.859 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:34.859 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229789+0000", 2024-09-21T15:01:34.859 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1700000000000002e-06 2024-09-21T15:01:34.859 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.859 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.859 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:34.859 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.230106+0000", 2024-09-21T15:01:34.859 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00031700800000000003 2024-09-21T15:01:34.860 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.860 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.860 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:34.860 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.230115+0000", 2024-09-21T15:01:34.860 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4190000000000008e-06 2024-09-21T15:01:34.860 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.860 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.860 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:34.860 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242935+0000", 2024-09-21T15:01:34.860 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0128191449999999 2024-09-21T15:01:34.860 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.860 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.860 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for missing object", 2024-09-21T15:01:34.860 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.243078+0000", 2024-09-21T15:01:34.861 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000142984 2024-09-21T15:01:34.861 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.861 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.861 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:34.861 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.247659+0000", 2024-09-21T15:01:34.861 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0045813030000000001 2024-09-21T15:01:34.861 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.861 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.861 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:34.861 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.247740+0000", 2024-09-21T15:01:34.861 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1216000000000003e-05 2024-09-21T15:01:34.861 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:34.861 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:34.861 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:34.861 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.862 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.862 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139994 3.16 3:6b9df8d1:::rbd_data.113664780c85.00000000000000e7:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:34.862 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.231370+0000", 2024-09-21T15:01:34.862 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 32.565867984999997, 2024-09-21T15:01:34.862 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.024795576, 2024-09-21T15:01:34.862 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:34.862 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:34.862 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:34.862 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:34.862 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:34.862 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139994 2024-09-21T15:01:34.862 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.862 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:34.862 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.863 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:34.863 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231370+0000", 2024-09-21T15:01:34.863 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:34.863 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.863 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.863 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:34.863 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231370+0000", 2024-09-21T15:01:34.863 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:34.863 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.863 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.863 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:34.863 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231369+0000", 2024-09-21T15:01:34.863 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:34.863 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.864 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.864 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:34.864 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231376+0000", 2024-09-21T15:01:34.864 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5509999999999996e-06 2024-09-21T15:01:34.864 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.864 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.864 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:34.864 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231387+0000", 2024-09-21T15:01:34.864 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.061e-05 2024-09-21T15:01:34.864 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.864 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.864 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:34.864 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231394+0000", 2024-09-21T15:01:34.864 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3799999999999996e-06 2024-09-21T15:01:34.865 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.865 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.865 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:34.865 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231459+0000", 2024-09-21T15:01:34.865 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4716999999999997e-05 2024-09-21T15:01:34.865 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.865 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.865 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:34.865 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231468+0000", 2024-09-21T15:01:34.865 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1479999999999995e-06 2024-09-21T15:01:34.865 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.865 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.865 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:34.865 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256074+0000", 2024-09-21T15:01:34.865 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0246064180000001 2024-09-21T15:01:34.866 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.866 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.866 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:34.866 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256166+0000", 2024-09-21T15:01:34.866 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1862999999999998e-05 2024-09-21T15:01:34.866 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:34.866 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:34.866 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:34.866 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.866 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.866 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140012 3.16 3:6bd6dfb6:::rbd_data.113664780c85.0000000000000165:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:34.866 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.231890+0000", 2024-09-21T15:01:34.866 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 32.565348225999998, 2024-09-21T15:01:34.866 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.024408881, 2024-09-21T15:01:34.866 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:34.867 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:34.867 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:34.867 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:34.867 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:34.867 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140012 2024-09-21T15:01:34.867 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.867 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:34.867 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.867 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:34.867 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231890+0000", 2024-09-21T15:01:34.867 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:34.867 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.867 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.867 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:34.867 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231890+0000", 2024-09-21T15:01:34.868 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:34.868 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.868 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.868 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:34.868 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231889+0000", 2024-09-21T15:01:34.868 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:34.868 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.868 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.868 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:34.868 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231895+0000", 2024-09-21T15:01:34.868 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9569999999999999e-06 2024-09-21T15:01:34.868 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.868 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.868 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:34.869 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231910+0000", 2024-09-21T15:01:34.869 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4549000000000001e-05 2024-09-21T15:01:34.869 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.869 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.869 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:34.869 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231917+0000", 2024-09-21T15:01:34.869 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5730000000000002e-06 2024-09-21T15:01:34.869 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.869 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.869 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:34.869 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231988+0000", 2024-09-21T15:01:34.869 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0852999999999994e-05 2024-09-21T15:01:34.869 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.869 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.869 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:34.870 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.232000+0000", 2024-09-21T15:01:34.870 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2159e-05 2024-09-21T15:01:34.870 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.870 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.870 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:34.870 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256192+0000", 2024-09-21T15:01:34.870 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.024192073 2024-09-21T15:01:34.870 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.870 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.870 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:34.870 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256299+0000", 2024-09-21T15:01:34.870 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000106733 2024-09-21T15:01:34.870 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:34.870 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:34.871 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:34.871 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.871 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.871 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140055 3.16 3:69bb9803:::rbd_data.113664780c85.00000000000000da:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:34.871 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.233922+0000", 2024-09-21T15:01:34.871 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 32.563316696999998, 2024-09-21T15:01:34.871 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.022511768, 2024-09-21T15:01:34.871 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:34.871 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:34.871 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:34.871 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:34.871 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:34.871 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140055 2024-09-21T15:01:34.871 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.871 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:34.872 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.872 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:34.872 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233922+0000", 2024-09-21T15:01:34.872 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:34.872 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.872 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.872 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:34.872 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233922+0000", 2024-09-21T15:01:34.872 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:34.872 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.872 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.872 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:34.872 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233921+0000", 2024-09-21T15:01:34.872 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:34.873 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.873 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.873 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:34.873 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233927+0000", 2024-09-21T15:01:34.873 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0360000000000003e-06 2024-09-21T15:01:34.873 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.873 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.873 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:34.873 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233937+0000", 2024-09-21T15:01:34.873 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6439999999999995e-06 2024-09-21T15:01:34.873 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.873 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.873 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:34.873 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233944+0000", 2024-09-21T15:01:34.873 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7710000000000006e-06 2024-09-21T15:01:34.874 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.874 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.874 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:34.874 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233996+0000", 2024-09-21T15:01:34.874 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1597000000000001e-05 2024-09-21T15:01:34.874 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.874 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.874 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:34.874 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.234005+0000", 2024-09-21T15:01:34.874 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5960000000000004e-06 2024-09-21T15:01:34.874 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.874 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.874 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:34.874 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256332+0000", 2024-09-21T15:01:34.875 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0223271949999999 2024-09-21T15:01:34.875 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.875 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.875 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:34.875 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256434+0000", 2024-09-21T15:01:34.875 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010180599999999999 2024-09-21T15:01:34.875 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:34.875 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:34.875 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:34.875 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.875 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.875 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140143 3.16 3:6d3ed596:::rbd_data.113664780c85.0000000000000184:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:34.875 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.239496+0000", 2024-09-21T15:01:34.875 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 32.557742167999997, 2024-09-21T15:01:34.875 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.017076576, 2024-09-21T15:01:34.876 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:34.876 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:34.876 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:34.876 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:34.876 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:34.876 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140143 2024-09-21T15:01:34.876 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.876 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:34.876 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.876 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:34.876 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239496+0000", 2024-09-21T15:01:34.876 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:34.876 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.876 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.876 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:34.876 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239496+0000", 2024-09-21T15:01:34.877 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:34.877 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.877 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.877 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:34.877 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239496+0000", 2024-09-21T15:01:34.877 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:34.877 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.877 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.877 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:34.877 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239498+0000", 2024-09-21T15:01:34.877 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.509e-06 2024-09-21T15:01:34.877 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.877 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.877 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:34.877 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239504+0000", 2024-09-21T15:01:34.878 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6869999999999999e-06 2024-09-21T15:01:34.878 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.878 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.878 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:34.878 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239509+0000", 2024-09-21T15:01:34.878 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8860000000000003e-06 2024-09-21T15:01:34.878 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.878 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.878 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:34.878 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239572+0000", 2024-09-21T15:01:34.878 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2908000000000001e-05 2024-09-21T15:01:34.878 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.878 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.878 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:34.879 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239580+0000", 2024-09-21T15:01:34.879 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4270000000000008e-06 2024-09-21T15:01:34.879 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.879 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.879 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:34.879 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256457+0000", 2024-09-21T15:01:34.879 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0168770979999999 2024-09-21T15:01:34.879 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.879 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.879 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:34.879 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256573+0000", 2024-09-21T15:01:34.879 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00011538400000000001 2024-09-21T15:01:34.879 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:34.879 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:34.879 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:34.880 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.880 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.880 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140163 3.16 3:694e82eb:::rbd_data.113664780c85.0000000000000052:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:34.880 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.240334+0000", 2024-09-21T15:01:34.880 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 32.556904455999998, 2024-09-21T15:01:34.880 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.016365537, 2024-09-21T15:01:34.880 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:34.880 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:34.880 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:34.880 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:34.880 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:34.880 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140163 2024-09-21T15:01:34.880 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.880 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:34.880 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.881 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:34.881 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240334+0000", 2024-09-21T15:01:34.881 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:34.881 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.881 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.881 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:34.881 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240334+0000", 2024-09-21T15:01:34.881 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:34.881 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.881 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.881 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:34.881 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240334+0000", 2024-09-21T15:01:34.881 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:34.881 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.881 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.882 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:34.882 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240337+0000", 2024-09-21T15:01:34.882 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.089e-06 2024-09-21T15:01:34.882 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.882 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.882 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:34.882 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240343+0000", 2024-09-21T15:01:34.882 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4420000000000003e-06 2024-09-21T15:01:34.882 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.882 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.882 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:34.882 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240348+0000", 2024-09-21T15:01:34.882 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1039999999999998e-06 2024-09-21T15:01:34.882 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.883 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.883 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:34.883 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240451+0000", 2024-09-21T15:01:34.883 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000102996 2024-09-21T15:01:34.883 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.883 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.883 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:34.883 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240460+0000", 2024-09-21T15:01:34.883 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5369999999999997e-06 2024-09-21T15:01:34.883 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.883 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.883 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:34.883 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256600+0000", 2024-09-21T15:01:34.883 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.016139884 2024-09-21T15:01:34.883 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.884 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.884 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:34.884 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256700+0000", 2024-09-21T15:01:34.884 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9844000000000002e-05 2024-09-21T15:01:34.884 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:34.884 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:34.884 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:34.884 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.884 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.884 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140164 3.16 3:68bfdc57:::rbd_data.113664780c85.0000000000000053:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:34.884 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.240358+0000", 2024-09-21T15:01:34.884 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 32.556880761999999, 2024-09-21T15:01:34.884 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.01646094, 2024-09-21T15:01:34.884 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:34.885 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:34.885 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:34.885 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:34.885 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:34.885 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140164 2024-09-21T15:01:34.885 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.885 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:34.885 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.885 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:34.885 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240358+0000", 2024-09-21T15:01:34.885 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:34.885 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.885 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.885 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:34.885 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240358+0000", 2024-09-21T15:01:34.886 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:34.886 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.886 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.886 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:34.886 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240357+0000", 2024-09-21T15:01:34.886 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:34.886 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.886 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.886 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:34.886 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240360+0000", 2024-09-21T15:01:34.886 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7089999999999998e-06 2024-09-21T15:01:34.886 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.886 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.886 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:34.887 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240368+0000", 2024-09-21T15:01:34.887 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7270000000000007e-06 2024-09-21T15:01:34.887 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.887 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.887 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:34.887 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240373+0000", 2024-09-21T15:01:34.887 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4060000000000004e-06 2024-09-21T15:01:34.887 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.887 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.887 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:34.887 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240473+0000", 2024-09-21T15:01:34.887 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000100132 2024-09-21T15:01:34.887 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.887 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.888 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:34.888 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240476+0000", 2024-09-21T15:01:34.888 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.332e-06 2024-09-21T15:01:34.888 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.888 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.888 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:34.888 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256724+0000", 2024-09-21T15:01:34.888 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0162471930000001 2024-09-21T15:01:34.888 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.888 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.888 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:34.888 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256819+0000", 2024-09-21T15:01:34.888 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5007999999999996e-05 2024-09-21T15:01:34.888 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:34.888 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:34.889 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:34.889 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.889 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.889 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140188 3.16 3:6f6217d4:::rbd_data.113664780c85.00000000000000fb:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:34.889 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.240978+0000", 2024-09-21T15:01:34.889 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 32.556260103, 2024-09-21T15:01:34.889 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.015983809, 2024-09-21T15:01:34.889 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:34.889 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:34.889 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:34.889 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:34.889 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:34.889 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140188 2024-09-21T15:01:34.889 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.890 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:34.890 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.890 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:34.890 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240978+0000", 2024-09-21T15:01:34.890 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:34.890 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.890 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.890 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:34.890 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240978+0000", 2024-09-21T15:01:34.890 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:34.890 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.890 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.890 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:34.890 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240978+0000", 2024-09-21T15:01:34.891 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:34.891 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.891 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.891 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:34.891 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240981+0000", 2024-09-21T15:01:34.891 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1559999999999999e-06 2024-09-21T15:01:34.891 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.891 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.891 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:34.891 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240987+0000", 2024-09-21T15:01:34.891 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.229e-06 2024-09-21T15:01:34.891 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.891 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.891 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:34.891 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240992+0000", 2024-09-21T15:01:34.892 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9710000000000003e-06 2024-09-21T15:01:34.892 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.892 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.892 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:34.892 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.241361+0000", 2024-09-21T15:01:34.892 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000369236 2024-09-21T15:01:34.892 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.892 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.892 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:34.892 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.241369+0000", 2024-09-21T15:01:34.892 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6199999999999999e-06 2024-09-21T15:01:34.892 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.892 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.892 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:34.892 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256844+0000", 2024-09-21T15:01:34.892 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0154751609999999 2024-09-21T15:01:34.893 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:34.893 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:34.893 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:34.893 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256962+0000", 2024-09-21T15:01:34.893 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000117964 2024-09-21T15:01:34.893 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:34.893 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:34.893 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:34.893 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:34.893 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:34.893 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:34.895 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T15:01:34.895 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:01:34.894+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:01:34.996 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:01:34.992+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:01:35.015 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:35.015 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:35.026 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T15:01:35.097 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:01:35.098+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:01:35.139 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:35.139 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:35.149 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T15:01:35.181 INFO:teuthology.orchestra.run.smithi032.stderr:noscrub is unset 2024-09-21T15:01:35.197 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:01:35.192+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:01:35.263 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:35.263 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:35.273 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T15:01:35.316 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:01:35.312+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:01:35.399 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:01:35.398+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:01:35.421 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:35.422 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:35.432 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T15:01:35.500 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:01:35.498+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:01:35.536 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:35.536 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:35.546 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T15:01:35.601 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:01:35.596+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:01:35.653 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:35.653 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:35.663 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T15:01:35.707 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:01:35.704+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:01:35.802 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:01:35.802+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:01:35.817 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:35.817 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:35.827 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T15:01:35.903 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:01:35.902+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:01:35.934 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:35.934 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:35.945 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T15:01:36.003 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:01:36.002+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:01:36.050 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:36.050 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:36.060 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T15:01:36.110 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:01:36.110+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:01:36.182 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:36.182 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:01:36.182 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T15:01:36.183 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:36.193 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T15:01:36.205 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:01:36.200+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:01:36.300 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:36.300 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:01:36.301 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T15:01:36.301 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T15:01:36.301 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:36.306 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:01:36.306+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:01:36.310 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T15:01:36.407 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:01:36.400+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:01:36.410 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~0s 2024-09-21T15:01:36.459 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:36.459 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T15:01:36.460 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T15:01:36.460 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T15:01:36.460 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.460 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:97677 3.15 3:a8e72133:::rbd_data.1136bfc79a70.0000000000000005:head [write 0~4440027 [fadvise_sequential+fadvise_nocache],write 4751360~147456 [fadvise_sequential+fadvise_nocache],write 4964352~147456 [fadvise_sequential+fadvise_nocache],write 5177344~65536 [fadvise_sequential+fadvise_nocache],write 5259264~147456 [fadvise_sequential+fadvise_nocache],write 5472256~147456 [fadvise_sequential+fadvise_nocache],write 5685248~229376 [fadvise_sequential+fadvise_nocache],write 5980160~147456 [fadvise_sequential+fadvise_nocache],write 6193152~229376 [fadvise_sequential+fadvise_nocache],write 6488064~147456 [fadvise_sequential+fadvise_nocache],write 6701056~229376 [fadvise_sequential+fadvise_nocache],write 6995968~147456 [fadvise_sequential+fadvise_nocache],write 7208960~229376 [fadvise_sequential+fadvise_nocache],write 7503872~147456 [fadvise_sequential+fadvise_nocache],write 7716864~229376 [fadvise_sequential+fadvise_nocache],write 8044544~114688 [fadvise_sequential+fadvise_nocache],write 8257536~114688 [fadvise_sequential+fadvise_nocache],write 8388608~65536 [fadvise_sequential+fadvise_nocache],write 8552448~114688 [fadvise_sequential+fadvise_nocache],write 8765440~114688 [fadvise_sequential+fadvise_nocache],write 8929280~32768 [fadvise_sequential+fadvise_nocache],write 9142272~32768 [fadvise_sequential+fadvise_nocache],write 9355264~32768 [fadvise_sequential+fadvise_nocache],write 9437184~32768 [fadvise_sequential+fadvise_nocache],write 9650176~32768 [fadvise_sequential+fadvise_nocache],write 9863168~32768 [fadvise_sequential+fadvise_nocache],write 9961472~131072 [fadvise_sequential+fadvise_nocache],write 10108928~196608 [fadvise_sequential+fadvise_nocache],write 10321920~278528 [fadvise_sequential+fadvise_nocache],write 10616832~196608 [fadvise_sequential+fadvise_nocache],write 10829824~278528 [fadvise_sequential+fadvise_nocache],write 11124736~196608 [fadvise_sequential+fadvise_nocache],write 11337728~196608 [fadvise_sequential+fadvise_nocache],write 11550720~65536 [fadvise_sequential+fadvise_nocache],write 11649024~98304 [fadvise_sequential+fadvise_nocache],write 11763712~65536 [fadvise_sequential+fadvise_nocache],write 11862016~98304 [fadvise_sequential+fadvise_nocache],write 11976704~65536 [fadvise_sequential+fadvise_nocache],write 12058624~65536 [fadvise_sequential+fadvise_nocache],write 12156928~98304 [fadvise_sequential+fadvise_nocache],write 12271616~65536 [fadvise_sequential+fadvise_nocache],write 12369920~98304 [fadvise_sequential+fadvise_nocache],write 12484608~65536 [fadvise_sequential+fadvise_nocache],write 12664832~98304 [fadvise_sequential+fadvise_nocache],write 12877824~98304 [fadvise_sequential+fadvise_nocache],write 13090816~16384 [fadvise_sequential+fadvise_nocache],write 13172736~98304 [fadvise_sequential+fadvise_nocache],write 13385728~98304 [fadvise_sequential+fadvise_nocache],write 13598720~32768 [fadvise_sequential+fadvise_nocache],write 13680640~98304 [fadvise_sequential+fadvise_nocache],write 13893632~98304 [fadvise_sequential+fadvise_nocache],write 14106624~49152 [fadvise_sequential+fadvise_nocache],write 14794752~81920 [fadvise_sequential+fadvise_nocache],write 15007744~81920 [fadvise_sequential+fadvise_nocache],write 15302656~81920 [fadvise_sequential+fadvise_nocache],write 15515648~81920 [fadvise_sequential+fadvise_nocache],write 15810560~81920 [fadvise_sequential+fadvise_nocache],write 16023552~81920 [fadvise_sequential+fadvise_nocache],write 16236544~16384 [fadvise_sequential+fadvise_nocache],write 16318464~81920 [fadvise_sequential+fadvise_nocache],write 16531456~81920 [fadvise_sequential+fadvise_nocache],write 16744448~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:01:36.460 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.302946+0000", 2024-09-21T15:01:36.460 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 91.155916392999998, 2024-09-21T15:01:36.460 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.091231204999999996, 2024-09-21T15:01:36.460 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:36.460 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:36.460 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:36.460 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:36.460 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:36.460 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 97677 2024-09-21T15:01:36.460 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.460 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:36.461 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.461 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:36.461 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.302946+0000", 2024-09-21T15:01:36.461 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:36.461 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.461 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.461 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:36.461 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.302946+0000", 2024-09-21T15:01:36.461 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:36.461 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.461 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.461 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:36.461 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.302942+0000", 2024-09-21T15:01:36.461 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999962 2024-09-21T15:01:36.461 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.461 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.462 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:36.462 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311253+0000", 2024-09-21T15:01:36.462 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0083105360000000003 2024-09-21T15:01:36.462 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.462 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.462 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:36.462 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311283+0000", 2024-09-21T15:01:36.462 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0065000000000001e-05 2024-09-21T15:01:36.462 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.462 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.462 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:36.462 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311296+0000", 2024-09-21T15:01:36.462 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3206e-05 2024-09-21T15:01:36.462 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.463 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.463 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:36.463 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311404+0000", 2024-09-21T15:01:36.463 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000107514 2024-09-21T15:01:36.463 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.463 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.463 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:36.463 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311795+0000", 2024-09-21T15:01:36.463 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000390913 2024-09-21T15:01:36.463 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.463 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.463 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 6", 2024-09-21T15:01:36.463 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.314686+0000", 2024-09-21T15:01:36.463 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0028915659999999999 2024-09-21T15:01:36.464 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.464 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.464 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:01:36.464 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.368178+0000", 2024-09-21T15:01:36.464 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.053491706 2024-09-21T15:01:36.464 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.464 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.464 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:01:36.464 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.393925+0000", 2024-09-21T15:01:36.464 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025747289 2024-09-21T15:01:36.464 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.464 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.464 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:01:36.465 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.394127+0000", 2024-09-21T15:01:36.465 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000201444 2024-09-21T15:01:36.465 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.465 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.465 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:36.465 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.394177+0000", 2024-09-21T15:01:36.465 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0689000000000002e-05 2024-09-21T15:01:36.465 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:36.465 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:36.465 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:36.465 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.465 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.465 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:97673 3.1d e122/24 3:bd91946e:::rbd_data.1136bfc79a70.0000000000000002:head v 122'162, mlcod=120'160)", 2024-09-21T15:01:36.465 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.313730+0000", 2024-09-21T15:01:36.465 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 91.145132067999995, 2024-09-21T15:01:36.466 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055373462999999998, 2024-09-21T15:01:36.466 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:36.466 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:36.466 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:36.466 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.466 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:36.466 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.313730+0000", 2024-09-21T15:01:36.466 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:36.466 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.466 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.466 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:36.466 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.313730+0000", 2024-09-21T15:01:36.466 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:36.466 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.467 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.467 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:36.467 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.313730+0000", 2024-09-21T15:01:36.467 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:36.467 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.467 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.467 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:36.467 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.336113+0000", 2024-09-21T15:01:36.467 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022382962999999999 2024-09-21T15:01:36.467 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.467 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.467 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:36.467 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.336140+0000", 2024-09-21T15:01:36.468 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7248999999999999e-05 2024-09-21T15:01:36.468 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.468 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.468 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:36.468 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.336151+0000", 2024-09-21T15:01:36.468 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1015e-05 2024-09-21T15:01:36.468 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.468 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.468 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:36.468 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.355428+0000", 2024-09-21T15:01:36.468 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019276379999999999 2024-09-21T15:01:36.468 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.468 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.468 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:36.468 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.355484+0000", 2024-09-21T15:01:36.469 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6820999999999997e-05 2024-09-21T15:01:36.469 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.469 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.469 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:01:36.469 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.369040+0000", 2024-09-21T15:01:36.469 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013555466 2024-09-21T15:01:36.469 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.469 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.469 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:36.469 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.369104+0000", 2024-09-21T15:01:36.469 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3986999999999996e-05 2024-09-21T15:01:36.469 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:36.469 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:36.469 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:36.470 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.470 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.470 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139890 3.e 3:71dba096:::rbd_data.113664780c85.000000000000018b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:36.470 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.226444+0000", 2024-09-21T15:01:36.470 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 34.232418426000002, 2024-09-21T15:01:36.470 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.028209567, 2024-09-21T15:01:36.470 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:36.470 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:36.470 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:36.470 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:36.470 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:36.470 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139890 2024-09-21T15:01:36.470 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.470 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:36.470 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.471 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:36.471 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226444+0000", 2024-09-21T15:01:36.471 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:36.471 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.471 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.471 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:36.471 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226444+0000", 2024-09-21T15:01:36.471 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:36.471 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.471 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.471 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:36.471 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226443+0000", 2024-09-21T15:01:36.471 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:36.471 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.471 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.472 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:36.472 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226448+0000", 2024-09-21T15:01:36.472 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5499999999999996e-06 2024-09-21T15:01:36.472 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.472 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.472 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:36.472 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226456+0000", 2024-09-21T15:01:36.472 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8679999999999999e-06 2024-09-21T15:01:36.472 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.472 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.472 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:36.472 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226462+0000", 2024-09-21T15:01:36.472 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1800000000000001e-06 2024-09-21T15:01:36.472 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.472 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.472 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:36.473 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226545+0000", 2024-09-21T15:01:36.473 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2527000000000005e-05 2024-09-21T15:01:36.473 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.473 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.473 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:36.473 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226553+0000", 2024-09-21T15:01:36.473 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8019999999999998e-06 2024-09-21T15:01:36.473 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.473 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.473 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:36.473 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.254491+0000", 2024-09-21T15:01:36.473 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.02793785 2024-09-21T15:01:36.473 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.473 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.473 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:36.474 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.254654+0000", 2024-09-21T15:01:36.474 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00016245800000000001 2024-09-21T15:01:36.474 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:36.474 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:36.474 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:36.474 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.474 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.474 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139897 3.e 3:76ddeb1d:::rbd_data.113664780c85.0000000000000004:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:36.474 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.227198+0000", 2024-09-21T15:01:36.474 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 34.231664821000003, 2024-09-21T15:01:36.474 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.027799822, 2024-09-21T15:01:36.474 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:36.474 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:36.474 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:36.474 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:36.475 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:36.475 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139897 2024-09-21T15:01:36.475 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.475 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:36.475 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.475 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:36.475 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227198+0000", 2024-09-21T15:01:36.475 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:36.475 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.475 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.475 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:36.475 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227198+0000", 2024-09-21T15:01:36.475 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:36.475 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.476 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.476 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:36.476 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227197+0000", 2024-09-21T15:01:36.476 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:36.476 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.476 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.476 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:36.476 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227203+0000", 2024-09-21T15:01:36.476 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4819999999999999e-06 2024-09-21T15:01:36.476 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.476 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.476 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:36.476 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227220+0000", 2024-09-21T15:01:36.476 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6671e-05 2024-09-21T15:01:36.476 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.476 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.476 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:36.477 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227229+0000", 2024-09-21T15:01:36.477 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0499999999999997e-06 2024-09-21T15:01:36.477 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.477 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.477 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:36.477 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227293+0000", 2024-09-21T15:01:36.477 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4434e-05 2024-09-21T15:01:36.477 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.477 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.477 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:36.477 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227302+0000", 2024-09-21T15:01:36.477 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0929999999999992e-06 2024-09-21T15:01:36.477 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.477 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.477 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:36.477 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.254720+0000", 2024-09-21T15:01:36.478 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0274177550000001 2024-09-21T15:01:36.478 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.478 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.478 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:36.478 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.254854+0000", 2024-09-21T15:01:36.478 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000133416 2024-09-21T15:01:36.478 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.478 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.478 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:36.478 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.254998+0000", 2024-09-21T15:01:36.478 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014400099999999999 2024-09-21T15:01:36.478 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:36.478 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:36.478 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:36.478 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.479 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.479 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139902 3.e 3:76b9b799:::rbd_data.113664780c85.0000000000000021:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:36.479 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.227319+0000", 2024-09-21T15:01:36.479 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 34.231543440000003, 2024-09-21T15:01:36.479 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0279584639999999, 2024-09-21T15:01:36.479 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:36.479 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:36.479 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:36.479 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:36.479 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:36.479 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139902 2024-09-21T15:01:36.479 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.479 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:36.479 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.479 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:36.479 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227319+0000", 2024-09-21T15:01:36.480 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:36.480 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.480 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.480 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:36.480 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227319+0000", 2024-09-21T15:01:36.480 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:36.480 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.480 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.480 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:36.480 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227319+0000", 2024-09-21T15:01:36.480 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:36.480 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.480 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.480 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:36.481 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227322+0000", 2024-09-21T15:01:36.481 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1980000000000001e-06 2024-09-21T15:01:36.481 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.481 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.481 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:36.481 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227328+0000", 2024-09-21T15:01:36.481 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4230000000000002e-06 2024-09-21T15:01:36.481 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.481 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.481 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:36.481 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227333+0000", 2024-09-21T15:01:36.481 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0560000000000004e-06 2024-09-21T15:01:36.481 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.482 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.482 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:36.482 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227729+0000", 2024-09-21T15:01:36.482 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00039591799999999999 2024-09-21T15:01:36.482 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.482 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.482 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:36.482 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227738+0000", 2024-09-21T15:01:36.482 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8119999999999997e-06 2024-09-21T15:01:36.482 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.482 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.482 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:36.482 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255021+0000", 2024-09-21T15:01:36.482 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0272826779999999 2024-09-21T15:01:36.483 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.483 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.483 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:36.483 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255147+0000", 2024-09-21T15:01:36.483 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012595600000000001 2024-09-21T15:01:36.483 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.483 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.483 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:36.483 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255278+0000", 2024-09-21T15:01:36.483 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013081400000000001 2024-09-21T15:01:36.483 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:36.483 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:36.483 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:36.483 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.484 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.484 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139945 3.e 3:7756a55d:::rbd_data.113664780c85.0000000000000154:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:36.484 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.229015+0000", 2024-09-21T15:01:36.484 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 34.229847636999999, 2024-09-21T15:01:36.484 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.026468825, 2024-09-21T15:01:36.484 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:36.484 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:36.484 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:36.484 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:36.484 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:36.484 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139945 2024-09-21T15:01:36.484 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.484 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:36.484 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.485 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:36.485 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229015+0000", 2024-09-21T15:01:36.485 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:36.485 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.485 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.485 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:36.485 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229015+0000", 2024-09-21T15:01:36.485 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:36.485 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.485 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.485 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:36.485 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229014+0000", 2024-09-21T15:01:36.485 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:36.485 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.486 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.486 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:36.486 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229017+0000", 2024-09-21T15:01:36.486 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7310000000000002e-06 2024-09-21T15:01:36.486 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.486 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.486 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:36.486 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229023+0000", 2024-09-21T15:01:36.486 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8150000000000002e-06 2024-09-21T15:01:36.486 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.486 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.486 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:36.486 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229028+0000", 2024-09-21T15:01:36.486 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7650000000000001e-06 2024-09-21T15:01:36.487 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.487 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.487 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:36.487 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229082+0000", 2024-09-21T15:01:36.487 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4521999999999997e-05 2024-09-21T15:01:36.487 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.487 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.487 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:36.487 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229092+0000", 2024-09-21T15:01:36.487 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.3940000000000002e-06 2024-09-21T15:01:36.487 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.487 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.487 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:36.487 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255382+0000", 2024-09-21T15:01:36.488 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0262900930000001 2024-09-21T15:01:36.488 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.488 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.488 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:36.488 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255484+0000", 2024-09-21T15:01:36.488 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000101932 2024-09-21T15:01:36.488 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:36.488 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:36.488 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:36.488 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.488 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.488 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139967 3.e 3:74e90d85:::rbd_data.113664780c85.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:36.488 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.229124+0000", 2024-09-21T15:01:36.489 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 34.229738697000002, 2024-09-21T15:01:36.489 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.026724381, 2024-09-21T15:01:36.489 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:36.489 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:36.489 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:36.489 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:36.489 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:36.489 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139967 2024-09-21T15:01:36.489 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.489 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:36.489 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.489 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:36.489 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229124+0000", 2024-09-21T15:01:36.489 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:36.489 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.490 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.490 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:36.490 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229124+0000", 2024-09-21T15:01:36.490 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:36.490 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.490 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.490 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:36.490 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229123+0000", 2024-09-21T15:01:36.490 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:36.490 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.490 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.490 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:36.490 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229126+0000", 2024-09-21T15:01:36.490 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4779999999999998e-06 2024-09-21T15:01:36.490 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.491 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.491 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:36.491 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229131+0000", 2024-09-21T15:01:36.491 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.729e-06 2024-09-21T15:01:36.491 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.491 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.491 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:36.491 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229135+0000", 2024-09-21T15:01:36.491 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5859999999999998e-06 2024-09-21T15:01:36.491 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.491 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.491 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:36.491 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229558+0000", 2024-09-21T15:01:36.491 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00042262499999999999 2024-09-21T15:01:36.492 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.492 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.492 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:36.492 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229569+0000", 2024-09-21T15:01:36.492 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0985000000000001e-05 2024-09-21T15:01:36.492 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.492 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.492 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:36.492 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255509+0000", 2024-09-21T15:01:36.492 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.025940246 2024-09-21T15:01:36.492 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.492 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.492 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:36.492 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255662+0000", 2024-09-21T15:01:36.493 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015324199999999999 2024-09-21T15:01:36.493 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.493 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.493 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:36.493 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255848+0000", 2024-09-21T15:01:36.493 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00018583599999999999 2024-09-21T15:01:36.493 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:36.493 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:36.493 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:36.493 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.493 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.493 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140010 3.e 3:735c06f2:::rbd_data.113664780c85.0000000000000157:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:36.493 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.231531+0000", 2024-09-21T15:01:36.493 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 34.227331831999997, 2024-09-21T15:01:36.493 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0244773149999999, 2024-09-21T15:01:36.493 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:36.494 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:36.494 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:36.494 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:36.494 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:36.494 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140010 2024-09-21T15:01:36.494 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.494 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:36.494 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.494 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:36.494 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231531+0000", 2024-09-21T15:01:36.494 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:36.494 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.494 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.494 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:36.495 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231531+0000", 2024-09-21T15:01:36.495 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:36.495 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.495 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.495 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:36.495 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231530+0000", 2024-09-21T15:01:36.495 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:36.495 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.495 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.495 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:36.495 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231535+0000", 2024-09-21T15:01:36.495 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3970000000000002e-06 2024-09-21T15:01:36.495 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.495 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.495 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:36.495 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231544+0000", 2024-09-21T15:01:36.496 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5909999999999996e-06 2024-09-21T15:01:36.496 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.496 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.496 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:36.496 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231551+0000", 2024-09-21T15:01:36.496 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7129999999999999e-06 2024-09-21T15:01:36.496 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.496 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.496 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:36.496 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231632+0000", 2024-09-21T15:01:36.496 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1181999999999996e-05 2024-09-21T15:01:36.496 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.496 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.496 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:36.497 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231640+0000", 2024-09-21T15:01:36.497 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.551e-06 2024-09-21T15:01:36.497 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.497 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.497 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:36.497 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255884+0000", 2024-09-21T15:01:36.497 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.024244057 2024-09-21T15:01:36.497 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.497 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.497 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:36.497 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256008+0000", 2024-09-21T15:01:36.497 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012351699999999999 2024-09-21T15:01:36.497 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:36.497 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:36.497 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:36.497 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.498 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.498 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140043 3.e 3:74a37859:::rbd_data.113664780c85.0000000000000086:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:36.498 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.232623+0000", 2024-09-21T15:01:36.498 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 34.226239358000001, 2024-09-21T15:01:36.498 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.023518634, 2024-09-21T15:01:36.498 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:36.498 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:36.498 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:36.498 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:36.498 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:36.498 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140043 2024-09-21T15:01:36.498 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.498 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:36.498 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.498 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:36.499 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.232623+0000", 2024-09-21T15:01:36.499 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:36.499 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.499 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.499 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:36.499 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.232623+0000", 2024-09-21T15:01:36.499 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:36.499 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.499 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.499 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:36.499 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.232623+0000", 2024-09-21T15:01:36.499 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:36.499 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.499 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.500 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:36.500 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.232626+0000", 2024-09-21T15:01:36.500 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0019999999999998e-06 2024-09-21T15:01:36.500 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.500 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.500 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:36.500 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.232632+0000", 2024-09-21T15:01:36.500 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3260000000000001e-06 2024-09-21T15:01:36.500 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.500 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.500 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:36.500 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.232637+0000", 2024-09-21T15:01:36.500 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9899999999999997e-06 2024-09-21T15:01:36.500 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.500 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.500 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:36.500 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.232678+0000", 2024-09-21T15:01:36.501 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.0423999999999998e-05 2024-09-21T15:01:36.501 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.501 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.501 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:36.501 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.232686+0000", 2024-09-21T15:01:36.501 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1419999999999993e-06 2024-09-21T15:01:36.501 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.501 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.501 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:36.501 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256037+0000", 2024-09-21T15:01:36.501 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.023351736 2024-09-21T15:01:36.501 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.501 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.501 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:36.501 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256142+0000", 2024-09-21T15:01:36.502 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010443 2024-09-21T15:01:36.502 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:36.502 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:36.502 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:36.502 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.502 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.502 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140063 3.e 3:771cae23:::rbd_data.113664780c85.0000000000000112:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:36.502 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.233948+0000", 2024-09-21T15:01:36.502 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 34.224914290999997, 2024-09-21T15:01:36.502 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0223114710000001, 2024-09-21T15:01:36.502 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:36.502 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:36.502 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:36.502 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:36.502 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:36.503 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140063 2024-09-21T15:01:36.503 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.503 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:36.503 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.503 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:36.503 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233948+0000", 2024-09-21T15:01:36.503 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:36.503 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.503 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.503 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:36.503 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233948+0000", 2024-09-21T15:01:36.503 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:36.503 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.503 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.503 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:36.504 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233948+0000", 2024-09-21T15:01:36.504 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:36.504 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.504 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.504 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:36.504 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233951+0000", 2024-09-21T15:01:36.504 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3380000000000001e-06 2024-09-21T15:01:36.504 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.504 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.504 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:36.504 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233958+0000", 2024-09-21T15:01:36.504 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0990000000000001e-06 2024-09-21T15:01:36.504 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.504 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.504 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:36.504 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233964+0000", 2024-09-21T15:01:36.504 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.852e-06 2024-09-21T15:01:36.505 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.505 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.505 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:36.505 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.234056+0000", 2024-09-21T15:01:36.505 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2392000000000006e-05 2024-09-21T15:01:36.505 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.505 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.505 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:36.505 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.234066+0000", 2024-09-21T15:01:36.505 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1360000000000003e-06 2024-09-21T15:01:36.505 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.505 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.505 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:36.505 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256169+0000", 2024-09-21T15:01:36.505 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0221031949999999 2024-09-21T15:01:36.505 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.506 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.506 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:36.506 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256260+0000", 2024-09-21T15:01:36.506 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0939000000000005e-05 2024-09-21T15:01:36.506 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:36.506 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:36.506 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:36.506 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.506 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.506 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140086 3.e 3:758134b7:::rbd_data.113664780c85.00000000000001ad:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:36.506 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.235110+0000", 2024-09-21T15:01:36.506 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 34.223752591999997, 2024-09-21T15:01:36.506 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.021276217, 2024-09-21T15:01:36.506 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:36.506 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:36.507 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:36.507 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:36.507 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:36.507 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140086 2024-09-21T15:01:36.507 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.507 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:36.507 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.507 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:36.507 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.235110+0000", 2024-09-21T15:01:36.507 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:36.507 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.507 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.507 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:36.507 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.235110+0000", 2024-09-21T15:01:36.508 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:36.508 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.508 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.508 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:36.508 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.235109+0000", 2024-09-21T15:01:36.508 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:36.508 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.508 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.508 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:36.508 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.235114+0000", 2024-09-21T15:01:36.508 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1659999999999996e-06 2024-09-21T15:01:36.508 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.508 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.508 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:36.508 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.235122+0000", 2024-09-21T15:01:36.508 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5590000000000006e-06 2024-09-21T15:01:36.509 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.509 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.509 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:36.509 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.235128+0000", 2024-09-21T15:01:36.509 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8579999999999996e-06 2024-09-21T15:01:36.509 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.509 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.509 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:36.509 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.235169+0000", 2024-09-21T15:01:36.509 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1304000000000003e-05 2024-09-21T15:01:36.509 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.509 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.509 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:36.509 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.235178+0000", 2024-09-21T15:01:36.510 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8610000000000004e-06 2024-09-21T15:01:36.510 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.510 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.510 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:36.510 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256286+0000", 2024-09-21T15:01:36.510 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.021108162 2024-09-21T15:01:36.510 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.510 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.510 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:36.510 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256386+0000", 2024-09-21T15:01:36.510 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9871000000000007e-05 2024-09-21T15:01:36.510 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:36.510 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:36.510 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:36.511 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.511 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.511 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140168 3.e 3:76f4e81d:::rbd_data.113664780c85.000000000000006f:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:36.511 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.236992+0000", 2024-09-21T15:01:36.511 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 34.221870529, 2024-09-21T15:01:36.511 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.019513699, 2024-09-21T15:01:36.511 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:36.511 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:36.511 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:36.511 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:36.511 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:36.511 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140168 2024-09-21T15:01:36.511 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.511 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:36.512 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.512 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:36.512 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.236992+0000", 2024-09-21T15:01:36.512 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:36.512 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.512 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.512 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:36.512 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.236992+0000", 2024-09-21T15:01:36.512 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:36.512 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.512 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.512 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:36.512 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.236991+0000", 2024-09-21T15:01:36.512 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:36.512 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.512 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.513 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:36.513 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.236995+0000", 2024-09-21T15:01:36.513 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3100000000000001e-06 2024-09-21T15:01:36.513 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.513 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.513 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:36.513 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237002+0000", 2024-09-21T15:01:36.513 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9929999999999998e-06 2024-09-21T15:01:36.513 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.513 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.513 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:36.513 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237008+0000", 2024-09-21T15:01:36.513 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7559999999999996e-06 2024-09-21T15:01:36.513 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.513 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.514 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:36.514 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237050+0000", 2024-09-21T15:01:36.514 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2852999999999998e-05 2024-09-21T15:01:36.514 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.514 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.514 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:36.514 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237060+0000", 2024-09-21T15:01:36.514 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2399999999999996e-06 2024-09-21T15:01:36.514 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.514 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.514 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:36.514 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256413+0000", 2024-09-21T15:01:36.514 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0193528999999999 2024-09-21T15:01:36.514 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.514 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.515 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:36.515 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256506+0000", 2024-09-21T15:01:36.515 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.3154999999999998e-05 2024-09-21T15:01:36.515 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:36.515 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:36.515 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:36.515 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.515 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.515 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140186 3.e 3:75504345:::rbd_data.113664780c85.00000000000000ed:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:36.515 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.237050+0000", 2024-09-21T15:01:36.515 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 34.221812558000003, 2024-09-21T15:01:36.515 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0195720150000001, 2024-09-21T15:01:36.515 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:36.515 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:36.515 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:36.515 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:36.516 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:36.516 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140186 2024-09-21T15:01:36.516 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.516 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:36.516 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.516 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:36.516 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237050+0000", 2024-09-21T15:01:36.516 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:36.516 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.516 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.516 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:36.516 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237050+0000", 2024-09-21T15:01:36.516 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:36.516 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.516 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.516 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:36.517 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237049+0000", 2024-09-21T15:01:36.517 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:36.517 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.517 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.517 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:36.517 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237052+0000", 2024-09-21T15:01:36.517 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.016e-06 2024-09-21T15:01:36.517 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.517 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.517 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:36.517 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237070+0000", 2024-09-21T15:01:36.517 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7266999999999999e-05 2024-09-21T15:01:36.517 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.517 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.517 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:36.517 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237075+0000", 2024-09-21T15:01:36.518 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6910000000000002e-06 2024-09-21T15:01:36.518 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.518 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.518 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:36.518 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237232+0000", 2024-09-21T15:01:36.518 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015683599999999999 2024-09-21T15:01:36.518 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.518 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.518 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:36.518 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237237+0000", 2024-09-21T15:01:36.518 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6439999999999999e-06 2024-09-21T15:01:36.518 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.518 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.518 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:36.519 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256534+0000", 2024-09-21T15:01:36.519 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.019296736 2024-09-21T15:01:36.519 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.519 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.519 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:36.519 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256622+0000", 2024-09-21T15:01:36.519 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8386000000000004e-05 2024-09-21T15:01:36.519 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:36.519 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:36.519 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:36.519 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.519 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.519 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140197 3.e 3:71ef5b14:::rbd_data.113664780c85.0000000000000140:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:36.519 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.237251+0000", 2024-09-21T15:01:36.519 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 34.221611529999997, 2024-09-21T15:01:36.519 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.01968386, 2024-09-21T15:01:36.520 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:36.520 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:36.520 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:36.520 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:36.520 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:36.520 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140197 2024-09-21T15:01:36.520 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.520 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:36.520 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.520 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:36.520 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237251+0000", 2024-09-21T15:01:36.520 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:36.520 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.520 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.520 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:36.520 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237251+0000", 2024-09-21T15:01:36.521 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:36.521 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.521 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.521 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:36.521 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237250+0000", 2024-09-21T15:01:36.521 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:36.521 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.521 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.521 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:36.521 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237254+0000", 2024-09-21T15:01:36.521 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.0180000000000003e-06 2024-09-21T15:01:36.521 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.521 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.521 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:36.521 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237262+0000", 2024-09-21T15:01:36.521 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0669999999999992e-06 2024-09-21T15:01:36.522 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.522 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.522 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:36.522 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237269+0000", 2024-09-21T15:01:36.522 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6000000000000003e-06 2024-09-21T15:01:36.522 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.522 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.522 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:36.522 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237319+0000", 2024-09-21T15:01:36.522 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9932000000000002e-05 2024-09-21T15:01:36.522 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.522 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.522 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:36.522 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237326+0000", 2024-09-21T15:01:36.523 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5360000000000004e-06 2024-09-21T15:01:36.523 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.523 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.523 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:36.523 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256657+0000", 2024-09-21T15:01:36.523 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.019330756 2024-09-21T15:01:36.523 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.523 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.523 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:36.523 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256935+0000", 2024-09-21T15:01:36.523 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00027762099999999998 2024-09-21T15:01:36.523 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:36.523 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:36.523 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:36.523 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.524 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.524 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140201 3.e 3:732ea575:::rbd_data.113664780c85.000000000000015c:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:36.524 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.237281+0000", 2024-09-21T15:01:36.524 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 34.221581692000001, 2024-09-21T15:01:36.524 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0198284529999999, 2024-09-21T15:01:36.524 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:36.524 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:36.524 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:36.524 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:36.524 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:36.524 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140201 2024-09-21T15:01:36.524 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.524 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:36.524 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.524 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:36.525 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237281+0000", 2024-09-21T15:01:36.525 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:36.525 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.525 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.525 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:36.525 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237281+0000", 2024-09-21T15:01:36.525 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:36.525 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.525 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.525 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:36.525 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237280+0000", 2024-09-21T15:01:36.525 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:36.525 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.525 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.526 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:36.526 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237284+0000", 2024-09-21T15:01:36.526 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7400000000000002e-06 2024-09-21T15:01:36.526 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.526 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.526 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:36.526 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237292+0000", 2024-09-21T15:01:36.526 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8939999999999993e-06 2024-09-21T15:01:36.526 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.526 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.526 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:36.526 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237298+0000", 2024-09-21T15:01:36.526 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8000000000000004e-06 2024-09-21T15:01:36.526 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.526 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.527 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:36.527 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237521+0000", 2024-09-21T15:01:36.527 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00022326 2024-09-21T15:01:36.527 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.527 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.527 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:36.527 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237529+0000", 2024-09-21T15:01:36.527 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6170000000000001e-06 2024-09-21T15:01:36.527 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.527 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.527 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:36.527 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256968+0000", 2024-09-21T15:01:36.527 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.019439113 2024-09-21T15:01:36.527 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.527 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.527 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:36.528 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.257109+0000", 2024-09-21T15:01:36.528 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014148399999999999 2024-09-21T15:01:36.528 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:36.528 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:36.528 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:36.528 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.528 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.528 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140206 3.e 3:76d38029:::rbd_data.113664780c85.0000000000000179:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:36.528 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.237520+0000", 2024-09-21T15:01:36.528 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 34.221342577999998, 2024-09-21T15:01:36.528 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0210658699999999, 2024-09-21T15:01:36.528 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:36.528 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:36.528 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:36.528 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:36.528 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:36.529 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140206 2024-09-21T15:01:36.529 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.529 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:36.529 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.529 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:36.529 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237520+0000", 2024-09-21T15:01:36.529 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:36.529 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.529 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.529 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:36.529 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237520+0000", 2024-09-21T15:01:36.529 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:36.529 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.529 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.529 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:36.530 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237519+0000", 2024-09-21T15:01:36.530 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:36.530 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.530 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.530 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:36.530 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237523+0000", 2024-09-21T15:01:36.530 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.9009999999999996e-06 2024-09-21T15:01:36.530 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.530 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.530 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:36.530 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237531+0000", 2024-09-21T15:01:36.530 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7069999999999992e-06 2024-09-21T15:01:36.530 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.531 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.531 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:36.531 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237537+0000", 2024-09-21T15:01:36.531 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4559999999999997e-06 2024-09-21T15:01:36.531 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.531 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.531 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:36.531 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237583+0000", 2024-09-21T15:01:36.531 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5532999999999999e-05 2024-09-21T15:01:36.531 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.531 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.531 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:36.531 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237592+0000", 2024-09-21T15:01:36.531 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6009999999999995e-06 2024-09-21T15:01:36.531 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.532 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.532 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:36.532 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.257157+0000", 2024-09-21T15:01:36.532 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0195657279999999 2024-09-21T15:01:36.532 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.532 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.532 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:36.532 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.258586+0000", 2024-09-21T15:01:36.532 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00142847 2024-09-21T15:01:36.532 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:36.532 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:36.532 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:36.532 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.532 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.532 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140338 3.e 3:71dba096:::rbd_data.113664780c85.000000000000018b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:36.533 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.243856+0000", 2024-09-21T15:01:36.533 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 34.215006744999997, 2024-09-21T15:01:36.533 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0138226889999999, 2024-09-21T15:01:36.533 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:36.533 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:36.533 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:36.533 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:36.533 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:36.533 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140338 2024-09-21T15:01:36.533 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.533 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:36.533 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.533 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:36.533 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243856+0000", 2024-09-21T15:01:36.534 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:36.534 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.534 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.534 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:36.534 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243856+0000", 2024-09-21T15:01:36.534 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:36.534 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.534 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.534 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:36.534 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243854+0000", 2024-09-21T15:01:36.534 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:36.534 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.534 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.534 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:36.534 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243862+0000", 2024-09-21T15:01:36.535 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2570000000000001e-06 2024-09-21T15:01:36.535 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.535 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.535 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:36.535 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243877+0000", 2024-09-21T15:01:36.535 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.571e-05 2024-09-21T15:01:36.535 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.535 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.535 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:36.535 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243884+0000", 2024-09-21T15:01:36.535 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.1269999999999997e-06 2024-09-21T15:01:36.535 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.535 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.535 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:36.536 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243932+0000", 2024-09-21T15:01:36.536 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7655999999999999e-05 2024-09-21T15:01:36.536 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.536 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.536 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:36.536 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243942+0000", 2024-09-21T15:01:36.536 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6679999999999996e-06 2024-09-21T15:01:36.536 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.536 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.536 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:36.536 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.257568+0000", 2024-09-21T15:01:36.536 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0136267670000001 2024-09-21T15:01:36.536 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.536 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.536 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:36.537 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.257678+0000", 2024-09-21T15:01:36.537 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000110011 2024-09-21T15:01:36.537 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:36.537 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:36.537 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:36.537 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.537 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.537 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140345 3.e 3:76ddeb1d:::rbd_data.113664780c85.0000000000000004:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:36.537 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.243927+0000", 2024-09-21T15:01:36.537 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 34.214935161, 2024-09-21T15:01:36.537 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0140328439999999, 2024-09-21T15:01:36.537 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:36.537 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:36.537 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:36.537 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:36.537 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:36.538 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140345 2024-09-21T15:01:36.538 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.538 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:36.538 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.538 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:36.538 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243927+0000", 2024-09-21T15:01:36.538 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:36.538 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.538 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.538 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:36.538 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243927+0000", 2024-09-21T15:01:36.538 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:36.538 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.538 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.539 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:36.539 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243927+0000", 2024-09-21T15:01:36.539 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:36.539 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.539 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.539 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:36.539 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243930+0000", 2024-09-21T15:01:36.539 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1319999999999998e-06 2024-09-21T15:01:36.539 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.539 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.539 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:36.539 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243937+0000", 2024-09-21T15:01:36.539 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4330000000000001e-06 2024-09-21T15:01:36.539 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.539 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.539 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:36.540 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243942+0000", 2024-09-21T15:01:36.540 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5090000000000001e-06 2024-09-21T15:01:36.540 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.540 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.540 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:36.540 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243964+0000", 2024-09-21T15:01:36.540 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1838999999999999e-05 2024-09-21T15:01:36.540 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.540 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.540 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:36.540 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243971+0000", 2024-09-21T15:01:36.540 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.131e-06 2024-09-21T15:01:36.540 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.540 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.540 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:36.540 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.257704+0000", 2024-09-21T15:01:36.541 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0137329310000001 2024-09-21T15:01:36.541 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.541 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.541 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:36.541 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.257805+0000", 2024-09-21T15:01:36.541 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000101121 2024-09-21T15:01:36.541 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.541 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.541 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:36.541 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.257960+0000", 2024-09-21T15:01:36.541 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015518500000000001 2024-09-21T15:01:36.541 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:36.541 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:36.541 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:36.541 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.541 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.542 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140350 3.e 3:76b9b799:::rbd_data.113664780c85.0000000000000021:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:36.542 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.243989+0000", 2024-09-21T15:01:36.542 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 34.214873410000003, 2024-09-21T15:01:36.542 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0142022340000001, 2024-09-21T15:01:36.542 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:36.542 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:36.542 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:36.542 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:36.542 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:36.542 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140350 2024-09-21T15:01:36.542 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.542 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:36.542 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.542 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:36.542 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243989+0000", 2024-09-21T15:01:36.543 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:36.543 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.543 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.543 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:36.543 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243989+0000", 2024-09-21T15:01:36.543 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:36.543 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.543 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.543 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:36.543 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243989+0000", 2024-09-21T15:01:36.543 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:36.543 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.543 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.543 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:36.543 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243992+0000", 2024-09-21T15:01:36.544 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.236e-06 2024-09-21T15:01:36.544 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.544 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.544 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:36.544 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243998+0000", 2024-09-21T15:01:36.544 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2500000000000003e-06 2024-09-21T15:01:36.544 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.544 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.544 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:36.544 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.244003+0000", 2024-09-21T15:01:36.544 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7740000000000004e-06 2024-09-21T15:01:36.544 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.544 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.544 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:36.544 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.244047+0000", 2024-09-21T15:01:36.544 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3665999999999998e-05 2024-09-21T15:01:36.545 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.545 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.545 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:36.545 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.244055+0000", 2024-09-21T15:01:36.545 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0639999999999994e-06 2024-09-21T15:01:36.545 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.545 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.545 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:36.545 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.257985+0000", 2024-09-21T15:01:36.545 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0139306079999999 2024-09-21T15:01:36.545 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.545 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.545 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:36.545 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.258084+0000", 2024-09-21T15:01:36.545 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8610000000000006e-05 2024-09-21T15:01:36.546 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.546 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.546 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:36.546 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.258191+0000", 2024-09-21T15:01:36.546 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000107518 2024-09-21T15:01:36.546 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:36.546 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:36.546 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:36.546 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.546 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.546 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140394 3.e 3:7756a55d:::rbd_data.113664780c85.0000000000000154:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:36.546 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.245038+0000", 2024-09-21T15:01:36.546 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 34.213824193000001, 2024-09-21T15:01:36.546 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.013225182, 2024-09-21T15:01:36.546 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:36.547 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:36.547 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:36.547 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:36.547 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:36.547 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140394 2024-09-21T15:01:36.547 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.547 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:36.547 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.547 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:36.547 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.245038+0000", 2024-09-21T15:01:36.547 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:36.547 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.547 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.547 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:36.547 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.245038+0000", 2024-09-21T15:01:36.548 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:36.548 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.548 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.548 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:36.548 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.245037+0000", 2024-09-21T15:01:36.548 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:36.548 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.548 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.548 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:36.548 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.245044+0000", 2024-09-21T15:01:36.548 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3239999999999999e-06 2024-09-21T15:01:36.548 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.548 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.548 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:36.549 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.245061+0000", 2024-09-21T15:01:36.549 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7166000000000001e-05 2024-09-21T15:01:36.549 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.549 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.549 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:36.549 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.245069+0000", 2024-09-21T15:01:36.549 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0779999999999996e-06 2024-09-21T15:01:36.549 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.549 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.549 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:36.549 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.245118+0000", 2024-09-21T15:01:36.549 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8887000000000001e-05 2024-09-21T15:01:36.549 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.549 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.549 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:36.549 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.245126+0000", 2024-09-21T15:01:36.550 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0409999999999998e-06 2024-09-21T15:01:36.550 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.550 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.550 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:36.550 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.258206+0000", 2024-09-21T15:01:36.550 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.013080451 2024-09-21T15:01:36.550 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.550 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.550 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:36.550 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.258264+0000", 2024-09-21T15:01:36.550 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7294999999999999e-05 2024-09-21T15:01:36.550 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:36.550 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:36.550 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:36.550 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:36.551 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:36.551 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:36.552 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T15:01:36.552 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:01:36.548+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:01:36.652 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:01:36.650+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:01:36.665 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:36.665 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:01:36.665 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T15:01:36.665 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:36.675 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T15:01:36.753 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:01:36.748+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:01:36.787 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:36.787 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:01:36.787 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T15:01:36.787 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T15:01:36.787 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:36.797 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T15:01:36.854 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:01:36.854+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:01:36.906 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:36.906 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T15:01:36.906 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T15:01:36.906 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T15:01:36.906 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.906 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T15:01:36.906 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.205754+0000", 2024-09-21T15:01:36.906 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 213.69768199999999, 2024-09-21T15:01:36.906 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.41117094199999998, 2024-09-21T15:01:36.906 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:36.907 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:36.907 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:36.907 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T15:01:36.907 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T15:01:36.907 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1 2024-09-21T15:01:36.907 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.907 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:36.907 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.907 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:36.907 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205754+0000", 2024-09-21T15:01:36.907 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:36.907 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.907 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.907 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:36.907 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205754+0000", 2024-09-21T15:01:36.908 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:36.908 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.908 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.908 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:36.908 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205752+0000", 2024-09-21T15:01:36.908 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T15:01:36.908 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.908 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.908 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:36.908 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205765+0000", 2024-09-21T15:01:36.908 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3230999999999999e-05 2024-09-21T15:01:36.908 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.908 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.908 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:36.908 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205782+0000", 2024-09-21T15:01:36.909 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6634e-05 2024-09-21T15:01:36.909 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.909 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.909 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:36.909 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205793+0000", 2024-09-21T15:01:36.909 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1297e-05 2024-09-21T15:01:36.909 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.909 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.909 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:36.909 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205867+0000", 2024-09-21T15:01:36.909 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4269000000000003e-05 2024-09-21T15:01:36.909 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.909 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.909 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:36.909 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205877+0000", 2024-09-21T15:01:36.910 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2690000000000005e-06 2024-09-21T15:01:36.910 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.910 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.910 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:36.910 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.613700+0000", 2024-09-21T15:01:36.910 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.407823455 2024-09-21T15:01:36.910 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.910 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.910 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:36.910 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.614203+0000", 2024-09-21T15:01:36.910 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00050311500000000005 2024-09-21T15:01:36.910 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.910 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.910 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T15:01:36.911 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.614750+0000", 2024-09-21T15:01:36.911 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00054738699999999998 2024-09-21T15:01:36.911 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.911 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.911 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:01:36.911 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616367+0000", 2024-09-21T15:01:36.911 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001616634 2024-09-21T15:01:36.911 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.911 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.911 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:01:36.911 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616574+0000", 2024-09-21T15:01:36.911 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00020668499999999999 2024-09-21T15:01:36.911 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.911 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.911 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:01:36.912 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616802+0000", 2024-09-21T15:01:36.912 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00022817900000000001 2024-09-21T15:01:36.912 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.912 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.912 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:36.912 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616925+0000", 2024-09-21T15:01:36.912 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012309699999999999 2024-09-21T15:01:36.912 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:36.912 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:36.912 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:36.912 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.912 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.912 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:61214 3.7 3:e28aec10:::rbd_header.1136862f872f:head [watch reconnect cookie 139864671932080 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-21T15:01:36.912 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:18.147172+0000", 2024-09-21T15:01:36.912 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 138.75626435699999, 2024-09-21T15:01:36.913 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.019264956, 2024-09-21T15:01:36.913 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:36.913 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:36.913 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:36.913 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:36.913 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:36.913 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 61214 2024-09-21T15:01:36.913 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.913 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:36.913 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.913 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:36.913 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147172+0000", 2024-09-21T15:01:36.913 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:36.913 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.913 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.913 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:36.914 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147172+0000", 2024-09-21T15:01:36.914 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:36.914 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.914 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.914 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:36.914 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147168+0000", 2024-09-21T15:01:36.914 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999962 2024-09-21T15:01:36.914 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.914 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.914 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:36.914 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147188+0000", 2024-09-21T15:01:36.914 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9717000000000001e-05 2024-09-21T15:01:36.914 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.914 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.914 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:36.915 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147239+0000", 2024-09-21T15:01:36.915 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1493999999999999e-05 2024-09-21T15:01:36.915 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.915 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.915 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:36.915 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147265+0000", 2024-09-21T15:01:36.915 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5746999999999999e-05 2024-09-21T15:01:36.915 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.915 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.915 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:36.915 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147372+0000", 2024-09-21T15:01:36.915 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010681200000000001 2024-09-21T15:01:36.915 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.915 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.915 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op must wait for map", 2024-09-21T15:01:36.916 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147380+0000", 2024-09-21T15:01:36.916 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7919999999999993e-06 2024-09-21T15:01:36.916 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.916 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.916 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:36.916 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.154025+0000", 2024-09-21T15:01:36.916 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0066454909999999999 2024-09-21T15:01:36.916 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.916 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.916 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:36.916 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.154041+0000", 2024-09-21T15:01:36.916 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5894000000000001e-05 2024-09-21T15:01:36.916 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.916 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.916 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:36.916 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.164345+0000", 2024-09-21T15:01:36.917 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0103040459999999 2024-09-21T15:01:36.917 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.917 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.917 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:36.917 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.164590+0000", 2024-09-21T15:01:36.917 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00024492699999999998 2024-09-21T15:01:36.917 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.917 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.917 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:36.917 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.166437+0000", 2024-09-21T15:01:36.917 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0018466330000000001 2024-09-21T15:01:36.917 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:36.917 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:36.917 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:36.917 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.917 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.918 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:61215 3.7 3:e28aec10:::rbd_header.1136862f872f:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-21T15:01:36.918 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:18.152008+0000", 2024-09-21T15:01:36.918 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 138.75142790500001, 2024-09-21T15:01:36.918 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.015644958, 2024-09-21T15:01:36.918 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:36.918 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:36.918 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:36.918 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:36.918 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:36.918 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 61215 2024-09-21T15:01:36.918 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.918 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:36.918 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.918 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:36.919 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152008+0000", 2024-09-21T15:01:36.919 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:36.919 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.919 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.919 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:36.919 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152008+0000", 2024-09-21T15:01:36.919 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:36.919 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.919 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.919 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:36.919 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152005+0000", 2024-09-21T15:01:36.919 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-21T15:01:36.919 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.919 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.919 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:36.919 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152025+0000", 2024-09-21T15:01:36.920 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9306000000000001e-05 2024-09-21T15:01:36.920 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.920 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.920 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:36.920 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152054+0000", 2024-09-21T15:01:36.920 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9189000000000001e-05 2024-09-21T15:01:36.920 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.920 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.920 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:36.920 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152078+0000", 2024-09-21T15:01:36.920 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4317000000000001e-05 2024-09-21T15:01:36.920 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.920 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.920 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:36.921 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152243+0000", 2024-09-21T15:01:36.921 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000164802 2024-09-21T15:01:36.921 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.921 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.921 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting_for_map not empty", 2024-09-21T15:01:36.921 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152251+0000", 2024-09-21T15:01:36.921 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0600000000000008e-06 2024-09-21T15:01:36.921 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.921 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.921 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:36.921 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.154741+0000", 2024-09-21T15:01:36.921 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0024900069999999998 2024-09-21T15:01:36.921 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.921 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.922 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:36.922 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.154753+0000", 2024-09-21T15:01:36.922 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1588e-05 2024-09-21T15:01:36.922 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.922 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.922 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:36.922 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.165380+0000", 2024-09-21T15:01:36.922 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0106272249999999 2024-09-21T15:01:36.922 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.922 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.922 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:36.922 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.165517+0000", 2024-09-21T15:01:36.922 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013737999999999999 2024-09-21T15:01:36.922 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.922 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.923 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 6", 2024-09-21T15:01:36.923 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.165938+0000", 2024-09-21T15:01:36.923 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00042082399999999998 2024-09-21T15:01:36.923 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.923 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.923 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:01:36.923 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.166795+0000", 2024-09-21T15:01:36.923 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00085674899999999997 2024-09-21T15:01:36.923 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.923 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.923 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:01:36.923 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.167598+0000", 2024-09-21T15:01:36.923 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00080335599999999997 2024-09-21T15:01:36.923 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.924 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.924 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:01:36.924 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.167632+0000", 2024-09-21T15:01:36.924 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4122000000000003e-05 2024-09-21T15:01:36.924 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.924 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.924 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:36.924 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.167653+0000", 2024-09-21T15:01:36.924 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0862000000000001e-05 2024-09-21T15:01:36.924 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:36.924 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:36.924 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:36.924 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.924 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.924 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:97678 3.d e122/24 3:b2f34c6d:::rbd_data.1136bfc79a70.0000000000000006:head v 122'234, mlcod=120'232)", 2024-09-21T15:01:36.924 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.321655+0000", 2024-09-21T15:01:36.925 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 91.581780988000006, 2024-09-21T15:01:36.925 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024509388999999999, 2024-09-21T15:01:36.925 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:36.925 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:36.925 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:36.925 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.925 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:36.925 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.321655+0000", 2024-09-21T15:01:36.925 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:36.925 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.925 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.925 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:36.925 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.321655+0000", 2024-09-21T15:01:36.925 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:36.926 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.926 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.926 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:36.926 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.321655+0000", 2024-09-21T15:01:36.926 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:36.926 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.926 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.926 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:36.926 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.329453+0000", 2024-09-21T15:01:36.926 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0077980560000000003 2024-09-21T15:01:36.926 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.926 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.926 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:36.926 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.329482+0000", 2024-09-21T15:01:36.927 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9153999999999999e-05 2024-09-21T15:01:36.927 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.927 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.927 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:36.927 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.329493+0000", 2024-09-21T15:01:36.927 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1103e-05 2024-09-21T15:01:36.927 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.927 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.927 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:36.927 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.329555+0000", 2024-09-21T15:01:36.927 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2345e-05 2024-09-21T15:01:36.927 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.928 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.928 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:36.928 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.329589+0000", 2024-09-21T15:01:36.928 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3908999999999997e-05 2024-09-21T15:01:36.928 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.928 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.928 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:01:36.928 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.346083+0000", 2024-09-21T15:01:36.928 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016493383 2024-09-21T15:01:36.928 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.928 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.928 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:36.928 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.346165+0000", 2024-09-21T15:01:36.929 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1796000000000001e-05 2024-09-21T15:01:36.929 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:36.929 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:36.929 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:36.929 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.929 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.929 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:97679 3.f e122/24 3:f2a362b4:::rbd_data.1136bfc79a70.0000000000000007:head v 122'331, mlcod=120'329)", 2024-09-21T15:01:36.929 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.337145+0000", 2024-09-21T15:01:36.929 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 91.566290979000001, 2024-09-21T15:01:36.929 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021912411, 2024-09-21T15:01:36.929 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:36.929 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:36.929 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:36.929 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.930 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:36.930 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.337145+0000", 2024-09-21T15:01:36.930 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:36.930 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.930 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.930 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:36.930 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.337145+0000", 2024-09-21T15:01:36.930 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:36.930 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.930 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.930 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:36.930 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.337145+0000", 2024-09-21T15:01:36.930 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:36.930 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.931 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.931 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:36.931 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.343899+0000", 2024-09-21T15:01:36.931 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0067541420000000003 2024-09-21T15:01:36.931 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.931 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.931 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:36.931 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.343921+0000", 2024-09-21T15:01:36.931 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2255e-05 2024-09-21T15:01:36.931 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.931 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.931 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:36.931 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.343933+0000", 2024-09-21T15:01:36.931 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1528e-05 2024-09-21T15:01:36.931 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.932 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.932 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:36.932 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.344001+0000", 2024-09-21T15:01:36.932 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8269999999999995e-05 2024-09-21T15:01:36.932 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.932 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.932 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:36.932 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.344029+0000", 2024-09-21T15:01:36.932 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8254000000000001e-05 2024-09-21T15:01:36.932 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.932 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.933 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:01:36.933 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.358901+0000", 2024-09-21T15:01:36.933 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0148719 2024-09-21T15:01:36.933 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.933 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.933 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:36.933 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.359058+0000", 2024-09-21T15:01:36.933 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000156446 2024-09-21T15:01:36.933 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:36.933 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:36.933 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:36.933 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.933 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.933 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:97676 3.1f e122/24 3:ffaa836a:::rbd_data.1136bfc79a70.0000000000000004:head v 122'312, mlcod=120'310)", 2024-09-21T15:01:36.934 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.348683+0000", 2024-09-21T15:01:36.934 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 91.554752813999997, 2024-09-21T15:01:36.934 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.033768802000000001, 2024-09-21T15:01:36.934 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:36.934 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:36.934 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:36.934 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.934 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:36.934 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.348683+0000", 2024-09-21T15:01:36.934 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:36.934 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.934 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.934 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:36.934 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.348683+0000", 2024-09-21T15:01:36.935 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:36.935 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.935 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.935 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:36.935 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.348683+0000", 2024-09-21T15:01:36.935 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:36.935 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.935 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.935 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:36.935 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.366937+0000", 2024-09-21T15:01:36.935 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018254415 2024-09-21T15:01:36.935 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.935 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.936 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:36.936 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.366959+0000", 2024-09-21T15:01:36.936 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1634e-05 2024-09-21T15:01:36.936 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.936 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.936 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:36.936 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.366971+0000", 2024-09-21T15:01:36.936 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2305e-05 2024-09-21T15:01:36.936 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.936 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.936 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:36.936 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.367034+0000", 2024-09-21T15:01:36.936 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3211000000000002e-05 2024-09-21T15:01:36.936 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.937 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.937 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:36.937 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.367056+0000", 2024-09-21T15:01:36.937 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2098000000000001e-05 2024-09-21T15:01:36.937 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.937 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.937 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:01:36.937 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.382400+0000", 2024-09-21T15:01:36.937 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.015343867000000001 2024-09-21T15:01:36.937 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.937 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.937 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:36.937 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.382452+0000", 2024-09-21T15:01:36.937 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1992e-05 2024-09-21T15:01:36.937 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:36.938 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:36.938 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:36.938 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.938 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.938 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:97734 3.19 e122/94 3:9fcf754f:::rbd_data.1136bfc79a70.0000000000000009:head v 122'314, mlcod=120'312)", 2024-09-21T15:01:36.938 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.378547+0000", 2024-09-21T15:01:36.938 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 91.524888836000002, 2024-09-21T15:01:36.938 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.063380675999999997, 2024-09-21T15:01:36.938 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:36.938 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:36.938 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:36.938 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.938 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:36.938 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.378547+0000", 2024-09-21T15:01:36.938 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:36.939 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.939 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.939 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:36.939 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.378547+0000", 2024-09-21T15:01:36.939 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:36.939 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.939 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.939 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:36.939 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.378547+0000", 2024-09-21T15:01:36.939 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:01:36.939 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.939 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.939 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:36.939 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.425118+0000", 2024-09-21T15:01:36.940 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046571166999999997 2024-09-21T15:01:36.940 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.940 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.940 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:36.940 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.425150+0000", 2024-09-21T15:01:36.940 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2150999999999998e-05 2024-09-21T15:01:36.940 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.940 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.940 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:36.940 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.425164+0000", 2024-09-21T15:01:36.940 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3302000000000001e-05 2024-09-21T15:01:36.940 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.940 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.940 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:36.940 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.425231+0000", 2024-09-21T15:01:36.940 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7106000000000004e-05 2024-09-21T15:01:36.941 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.941 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.941 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:36.941 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.425259+0000", 2024-09-21T15:01:36.941 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8087000000000001e-05 2024-09-21T15:01:36.941 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.941 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.941 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:01:36.941 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.441890+0000", 2024-09-21T15:01:36.941 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016631328000000001 2024-09-21T15:01:36.941 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.941 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.941 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:36.941 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.441928+0000", 2024-09-21T15:01:36.941 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7701999999999999e-05 2024-09-21T15:01:36.941 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:36.942 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:36.942 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:36.942 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.942 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.942 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:100187 3.10 3:08941a97:::rbd_data.1136ab4d6ff1.000000000000000e:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-21T15:01:36.942 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:10.869845+0000", 2024-09-21T15:01:36.942 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 86.033590939999996, 2024-09-21T15:01:36.942 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029345237, 2024-09-21T15:01:36.942 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:36.942 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:36.942 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:36.942 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:36.942 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:36.942 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 100187 2024-09-21T15:01:36.942 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.943 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:36.943 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.943 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:36.943 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869845+0000", 2024-09-21T15:01:36.943 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:36.943 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.943 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.943 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:36.943 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869845+0000", 2024-09-21T15:01:36.943 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:36.943 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.943 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.943 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:36.943 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869844+0000", 2024-09-21T15:01:36.943 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:36.943 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.944 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.944 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:36.944 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869853+0000", 2024-09-21T15:01:36.944 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5719999999999992e-06 2024-09-21T15:01:36.944 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.944 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.944 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:36.944 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869874+0000", 2024-09-21T15:01:36.944 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0483e-05 2024-09-21T15:01:36.944 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.944 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.944 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:36.944 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869883+0000", 2024-09-21T15:01:36.944 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6080000000000002e-06 2024-09-21T15:01:36.945 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.945 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.945 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:36.945 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869991+0000", 2024-09-21T15:01:36.945 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000108013 2024-09-21T15:01:36.945 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.945 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.945 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:36.945 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.870258+0000", 2024-09-21T15:01:36.945 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00026629099999999999 2024-09-21T15:01:36.945 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.945 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.945 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T15:01:36.945 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.878656+0000", 2024-09-21T15:01:36.945 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0083989000000000008 2024-09-21T15:01:36.945 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.946 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.946 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:01:36.946 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.899081+0000", 2024-09-21T15:01:36.946 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.020424437 2024-09-21T15:01:36.946 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.946 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.946 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:01:36.946 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.899125+0000", 2024-09-21T15:01:36.946 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4440999999999997e-05 2024-09-21T15:01:36.946 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.946 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.946 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:01:36.946 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.899160+0000", 2024-09-21T15:01:36.946 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4518999999999999e-05 2024-09-21T15:01:36.947 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.947 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.947 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:36.947 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.899190+0000", 2024-09-21T15:01:36.947 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0574999999999998e-05 2024-09-21T15:01:36.947 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:36.947 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:36.947 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:36.947 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.947 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.947 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:100263 3.10 3:0ece142e:::rbd_data.1136ab4d6ff1.0000000000000008:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-21T15:01:36.947 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:11.052257+0000", 2024-09-21T15:01:36.947 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 85.851178881999999, 2024-09-21T15:01:36.948 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042678025000000001, 2024-09-21T15:01:36.948 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:36.948 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:36.948 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:36.948 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:36.948 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:36.948 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 100263 2024-09-21T15:01:36.948 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.948 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:36.948 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.948 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:36.948 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052257+0000", 2024-09-21T15:01:36.948 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:36.948 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.948 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.949 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:36.949 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052257+0000", 2024-09-21T15:01:36.949 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:36.949 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.949 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.949 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:36.949 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052256+0000", 2024-09-21T15:01:36.949 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:36.949 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.949 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.949 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:36.949 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052263+0000", 2024-09-21T15:01:36.949 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4799999999999998e-06 2024-09-21T15:01:36.949 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.949 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.949 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:36.950 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052275+0000", 2024-09-21T15:01:36.950 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2386e-05 2024-09-21T15:01:36.950 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.950 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.950 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:36.950 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052281+0000", 2024-09-21T15:01:36.950 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6699999999999999e-06 2024-09-21T15:01:36.950 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.950 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.950 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:36.950 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052350+0000", 2024-09-21T15:01:36.950 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9099999999999999e-05 2024-09-21T15:01:36.950 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.950 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.950 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:36.951 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052456+0000", 2024-09-21T15:01:36.951 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000106669 2024-09-21T15:01:36.951 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.951 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.951 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T15:01:36.951 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.059694+0000", 2024-09-21T15:01:36.951 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.007237566 2024-09-21T15:01:36.951 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.951 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.951 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:01:36.951 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.094721+0000", 2024-09-21T15:01:36.951 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035026977000000001 2024-09-21T15:01:36.951 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.951 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.951 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:01:36.952 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.094746+0000", 2024-09-21T15:01:36.952 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5329000000000001e-05 2024-09-21T15:01:36.952 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.952 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.952 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:01:36.952 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.094850+0000", 2024-09-21T15:01:36.952 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000103452 2024-09-21T15:01:36.952 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.952 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.952 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:36.952 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.094935+0000", 2024-09-21T15:01:36.952 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5550000000000006e-05 2024-09-21T15:01:36.952 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:36.952 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:36.952 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:36.953 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.953 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.953 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:134708 3.a 3:51eaadc0:::rbd_data.11365ed419e3.0000000000000007:67 [read 221184~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:36.953 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.938945+0000", 2024-09-21T15:01:36.953 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 42.964491604000003, 2024-09-21T15:01:36.953 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021073784000000002, 2024-09-21T15:01:36.953 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:36.953 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:36.953 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:36.953 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:36.953 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:36.953 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 134708 2024-09-21T15:01:36.953 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.953 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:36.953 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.953 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:36.954 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938945+0000", 2024-09-21T15:01:36.954 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:36.954 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.954 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.954 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:36.954 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938945+0000", 2024-09-21T15:01:36.954 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:36.954 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.954 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.954 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:36.954 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938944+0000", 2024-09-21T15:01:36.954 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:36.954 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.954 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.954 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:36.954 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938946+0000", 2024-09-21T15:01:36.955 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.616e-06 2024-09-21T15:01:36.955 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.955 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.955 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:36.955 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938952+0000", 2024-09-21T15:01:36.955 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7309999999999998e-06 2024-09-21T15:01:36.955 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.955 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.955 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:36.955 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938957+0000", 2024-09-21T15:01:36.955 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8579999999999999e-06 2024-09-21T15:01:36.955 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.955 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.955 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:36.955 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.959730+0000", 2024-09-21T15:01:36.955 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.020773183000000001 2024-09-21T15:01:36.956 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.956 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.956 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:36.956 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.959810+0000", 2024-09-21T15:01:36.956 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0158000000000001e-05 2024-09-21T15:01:36.956 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.956 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.956 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:36.956 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.960018+0000", 2024-09-21T15:01:36.956 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000207951 2024-09-21T15:01:36.956 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:36.956 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:36.956 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:36.956 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.957 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.957 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:134709 3.a 3:51eaadc0:::rbd_data.11365ed419e3.0000000000000007:67 [read 237568~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:36.957 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.938996+0000", 2024-09-21T15:01:36.957 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 42.964440611999997, 2024-09-21T15:01:36.957 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021382577, 2024-09-21T15:01:36.957 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:36.957 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:36.957 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:36.957 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:36.957 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:36.957 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 134709 2024-09-21T15:01:36.957 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.957 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:36.957 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.957 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:36.957 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938996+0000", 2024-09-21T15:01:36.958 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:36.958 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.958 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.958 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:36.958 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938996+0000", 2024-09-21T15:01:36.958 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:36.958 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.958 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.958 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:36.958 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938995+0000", 2024-09-21T15:01:36.958 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:36.958 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.958 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.958 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:36.959 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938999+0000", 2024-09-21T15:01:36.959 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7979999999999999e-06 2024-09-21T15:01:36.959 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.959 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.959 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:36.959 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939006+0000", 2024-09-21T15:01:36.959 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6669999999999996e-06 2024-09-21T15:01:36.959 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.959 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.959 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:36.959 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939012+0000", 2024-09-21T15:01:36.959 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9329999999999998e-06 2024-09-21T15:01:36.959 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.959 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.959 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:36.960 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.960080+0000", 2024-09-21T15:01:36.960 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021068303 2024-09-21T15:01:36.960 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.960 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.960 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:36.960 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.960154+0000", 2024-09-21T15:01:36.960 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3819000000000006e-05 2024-09-21T15:01:36.960 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.960 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.960 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:36.960 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.960378+0000", 2024-09-21T15:01:36.960 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00022385800000000001 2024-09-21T15:01:36.960 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:36.960 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:36.960 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:36.960 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.961 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.961 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:134710 3.a 3:51eaadc0:::rbd_data.11365ed419e3.0000000000000007:67 [read 253952~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:36.961 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.939017+0000", 2024-09-21T15:01:36.961 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 42.964419601000003, 2024-09-21T15:01:36.961 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021721436, 2024-09-21T15:01:36.961 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:36.961 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:36.961 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:36.961 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:36.961 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:36.961 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 134710 2024-09-21T15:01:36.961 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.961 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:36.961 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.961 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:36.962 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939017+0000", 2024-09-21T15:01:36.962 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:36.962 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.962 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.962 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:36.962 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939017+0000", 2024-09-21T15:01:36.962 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:36.962 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.962 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.962 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:36.962 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939016+0000", 2024-09-21T15:01:36.962 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:36.962 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.962 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.962 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:36.963 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939019+0000", 2024-09-21T15:01:36.963 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.554e-06 2024-09-21T15:01:36.963 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.963 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.963 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:36.963 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939024+0000", 2024-09-21T15:01:36.963 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1569999999999999e-06 2024-09-21T15:01:36.963 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.963 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.963 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:36.963 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939034+0000", 2024-09-21T15:01:36.963 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9219999999999992e-06 2024-09-21T15:01:36.963 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.963 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.963 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:36.964 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.960408+0000", 2024-09-21T15:01:36.964 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021373792999999999 2024-09-21T15:01:36.964 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.964 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.964 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:36.964 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.960482+0000", 2024-09-21T15:01:36.964 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4331000000000004e-05 2024-09-21T15:01:36.964 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.964 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.964 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:36.964 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.960738+0000", 2024-09-21T15:01:36.964 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00025600500000000001 2024-09-21T15:01:36.964 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:36.964 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:36.964 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:36.964 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.965 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.965 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:134711 3.a 3:51eaadc0:::rbd_data.11365ed419e3.0000000000000007:67 [read 270336~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:36.965 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.939039+0000", 2024-09-21T15:01:36.965 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 42.964397552000001, 2024-09-21T15:01:36.965 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022064698000000001, 2024-09-21T15:01:36.965 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:36.965 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:36.965 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:36.965 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:36.965 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:36.965 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 134711 2024-09-21T15:01:36.965 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.965 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:36.965 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.966 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:36.966 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939039+0000", 2024-09-21T15:01:36.966 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:36.966 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.966 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.966 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:36.966 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939039+0000", 2024-09-21T15:01:36.966 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:36.966 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.966 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.966 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:36.966 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939038+0000", 2024-09-21T15:01:36.966 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:36.966 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.967 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.967 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:36.967 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939041+0000", 2024-09-21T15:01:36.967 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9440000000000001e-06 2024-09-21T15:01:36.967 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.967 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.967 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:36.967 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939047+0000", 2024-09-21T15:01:36.967 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1260000000000003e-06 2024-09-21T15:01:36.967 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.967 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.967 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:36.967 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939052+0000", 2024-09-21T15:01:36.967 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1619999999999999e-06 2024-09-21T15:01:36.967 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.968 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.968 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:36.968 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.960768+0000", 2024-09-21T15:01:36.968 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021715140000000001 2024-09-21T15:01:36.968 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.968 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.968 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:36.968 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.960842+0000", 2024-09-21T15:01:36.968 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4698999999999997e-05 2024-09-21T15:01:36.968 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.968 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.968 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:36.968 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.961103+0000", 2024-09-21T15:01:36.968 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00026098499999999998 2024-09-21T15:01:36.968 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:36.968 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:36.969 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:36.969 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.969 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.969 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:134712 3.a 3:51eaadc0:::rbd_data.11365ed419e3.0000000000000007:67 [read 286720~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:36.969 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.939057+0000", 2024-09-21T15:01:36.969 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 42.964379018999999, 2024-09-21T15:01:36.969 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022387067, 2024-09-21T15:01:36.969 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:36.969 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:36.969 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:36.969 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:36.969 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:36.969 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 134712 2024-09-21T15:01:36.969 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.969 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:36.970 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.970 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:36.970 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939057+0000", 2024-09-21T15:01:36.970 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:36.970 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.970 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.970 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:36.970 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939057+0000", 2024-09-21T15:01:36.970 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:36.970 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.970 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.970 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:36.970 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939057+0000", 2024-09-21T15:01:36.970 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:36.970 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.971 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.971 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:36.971 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939059+0000", 2024-09-21T15:01:36.971 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.447e-06 2024-09-21T15:01:36.971 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.971 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.971 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:36.971 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939065+0000", 2024-09-21T15:01:36.971 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4670000000000003e-06 2024-09-21T15:01:36.971 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.971 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.971 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:36.971 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939069+0000", 2024-09-21T15:01:36.971 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2119999999999997e-06 2024-09-21T15:01:36.971 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.972 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.972 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:36.972 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.961141+0000", 2024-09-21T15:01:36.972 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022072268999999999 2024-09-21T15:01:36.972 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.972 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.972 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:36.972 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.961240+0000", 2024-09-21T15:01:36.972 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9147999999999993e-05 2024-09-21T15:01:36.972 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.972 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.972 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:36.972 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.961444+0000", 2024-09-21T15:01:36.972 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00020381899999999999 2024-09-21T15:01:36.972 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:36.972 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:36.973 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:36.973 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.973 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.973 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:134713 3.a 3:51eaadc0:::rbd_data.11365ed419e3.0000000000000007:67 [read 303104~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:36.973 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.939073+0000", 2024-09-21T15:01:36.973 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 42.964363116000001, 2024-09-21T15:01:36.973 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022687394999999999, 2024-09-21T15:01:36.973 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:36.973 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:36.973 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:36.973 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:36.973 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:36.973 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 134713 2024-09-21T15:01:36.973 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.973 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:36.974 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.974 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:36.974 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939073+0000", 2024-09-21T15:01:36.974 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:36.974 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.974 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.974 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:36.974 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939073+0000", 2024-09-21T15:01:36.974 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:36.974 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.974 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.974 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:36.974 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939073+0000", 2024-09-21T15:01:36.974 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:36.974 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.975 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.975 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:36.975 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939075+0000", 2024-09-21T15:01:36.975 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5969999999999999e-06 2024-09-21T15:01:36.975 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.975 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.975 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:36.975 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939081+0000", 2024-09-21T15:01:36.975 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.609e-06 2024-09-21T15:01:36.975 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.975 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.975 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:36.975 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939086+0000", 2024-09-21T15:01:36.976 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.916e-06 2024-09-21T15:01:36.976 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.976 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.976 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:36.976 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.961474+0000", 2024-09-21T15:01:36.976 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022388512999999999 2024-09-21T15:01:36.976 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.976 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.976 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:36.976 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.961567+0000", 2024-09-21T15:01:36.976 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2267000000000003e-05 2024-09-21T15:01:36.976 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.976 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.976 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:36.976 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.961760+0000", 2024-09-21T15:01:36.976 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00019375999999999999 2024-09-21T15:01:36.977 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:36.977 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:36.977 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:36.977 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.977 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.977 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:134714 3.a 3:51eaadc0:::rbd_data.11365ed419e3.0000000000000007:67 [read 319488~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:36.977 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.939090+0000", 2024-09-21T15:01:36.977 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 42.964346288999998, 2024-09-21T15:01:36.977 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023043403, 2024-09-21T15:01:36.977 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:36.977 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:36.977 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:36.978 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:36.978 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:36.978 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 134714 2024-09-21T15:01:36.978 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.978 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:36.978 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.978 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:36.978 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939090+0000", 2024-09-21T15:01:36.978 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:36.978 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.978 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.978 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:36.978 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939090+0000", 2024-09-21T15:01:36.978 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:36.979 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.979 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.979 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:36.979 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939090+0000", 2024-09-21T15:01:36.979 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:36.979 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.979 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.979 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:36.979 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939092+0000", 2024-09-21T15:01:36.979 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2790000000000001e-06 2024-09-21T15:01:36.979 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.979 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.979 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:36.979 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939097+0000", 2024-09-21T15:01:36.979 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1410000000000004e-06 2024-09-21T15:01:36.979 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.980 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.980 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:36.980 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939102+0000", 2024-09-21T15:01:36.980 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7369999999999997e-06 2024-09-21T15:01:36.980 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.980 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.980 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:36.980 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.961791+0000", 2024-09-21T15:01:36.980 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022688974000000001 2024-09-21T15:01:36.980 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.980 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.980 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:36.980 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.961865+0000", 2024-09-21T15:01:36.980 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4651000000000001e-05 2024-09-21T15:01:36.980 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.980 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.981 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:36.981 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.962133+0000", 2024-09-21T15:01:36.981 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00026793999999999999 2024-09-21T15:01:36.981 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:36.981 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:36.981 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:36.981 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.981 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.981 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:134715 3.a 3:51eaadc0:::rbd_data.11365ed419e3.0000000000000007:67 [read 335872~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:36.981 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.939106+0000", 2024-09-21T15:01:36.981 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 42.964329679000002, 2024-09-21T15:01:36.981 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023401142999999999, 2024-09-21T15:01:36.981 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:36.981 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:36.981 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:36.981 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:36.982 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:36.982 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 134715 2024-09-21T15:01:36.982 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.982 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:36.982 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.982 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:36.982 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939106+0000", 2024-09-21T15:01:36.982 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:36.982 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.982 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.982 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:36.982 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939106+0000", 2024-09-21T15:01:36.982 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:36.982 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.982 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.983 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:36.983 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939106+0000", 2024-09-21T15:01:36.983 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:36.983 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.983 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.983 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:36.983 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939114+0000", 2024-09-21T15:01:36.983 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1009999999999992e-06 2024-09-21T15:01:36.983 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.983 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.983 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:36.983 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939121+0000", 2024-09-21T15:01:36.983 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7710000000000001e-06 2024-09-21T15:01:36.983 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.983 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.984 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:36.984 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939126+0000", 2024-09-21T15:01:36.984 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3600000000000004e-06 2024-09-21T15:01:36.984 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.984 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.984 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:36.984 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.962162+0000", 2024-09-21T15:01:36.984 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023036148999999999 2024-09-21T15:01:36.984 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.984 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.984 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:36.984 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.962239+0000", 2024-09-21T15:01:36.984 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6874999999999998e-05 2024-09-21T15:01:36.984 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.984 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.984 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:36.985 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.962508+0000", 2024-09-21T15:01:36.985 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00026842199999999999 2024-09-21T15:01:36.985 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:36.985 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:36.985 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:36.985 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.985 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.985 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:134716 3.a 3:51eaadc0:::rbd_data.11365ed419e3.0000000000000007:67 [read 352256~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:36.985 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.939131+0000", 2024-09-21T15:01:36.985 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 42.964305545000002, 2024-09-21T15:01:36.985 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023742441, 2024-09-21T15:01:36.985 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:36.985 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:36.985 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:36.985 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:36.985 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:36.986 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 134716 2024-09-21T15:01:36.986 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.986 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:36.986 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.986 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:36.986 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939131+0000", 2024-09-21T15:01:36.986 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:36.986 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.986 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.986 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:36.986 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939131+0000", 2024-09-21T15:01:36.986 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:36.986 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.986 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.986 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:36.987 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939130+0000", 2024-09-21T15:01:36.987 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:36.987 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.987 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.987 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:36.987 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939133+0000", 2024-09-21T15:01:36.987 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5620000000000002e-06 2024-09-21T15:01:36.987 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.987 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.987 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:36.987 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939138+0000", 2024-09-21T15:01:36.987 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.322e-06 2024-09-21T15:01:36.987 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.987 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.987 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:36.988 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939143+0000", 2024-09-21T15:01:36.988 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5510000000000001e-06 2024-09-21T15:01:36.988 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.988 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.988 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:36.988 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.962537+0000", 2024-09-21T15:01:36.988 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023394808 2024-09-21T15:01:36.988 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.988 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.988 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:36.988 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.962608+0000", 2024-09-21T15:01:36.988 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0699999999999997e-05 2024-09-21T15:01:36.988 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.988 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.988 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:36.988 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.962873+0000", 2024-09-21T15:01:36.989 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00026487 2024-09-21T15:01:36.989 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:36.989 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:36.989 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:36.989 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.989 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.989 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:135024 3.a 3:53261383:::rbd_data.11365ed419e3.0000000000000037:snapdir [list-snaps] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:36.989 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.940162+0000", 2024-09-21T15:01:36.989 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 42.963274581999997, 2024-09-21T15:01:36.989 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022852172, 2024-09-21T15:01:36.989 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:36.989 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:36.989 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:36.989 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:36.989 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:36.989 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 135024 2024-09-21T15:01:36.990 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.990 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:36.990 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.990 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:36.990 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940162+0000", 2024-09-21T15:01:36.990 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:36.990 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.990 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.990 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:36.990 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940162+0000", 2024-09-21T15:01:36.990 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:36.990 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.990 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.990 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:36.991 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940161+0000", 2024-09-21T15:01:36.991 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:36.991 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.991 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.991 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:36.991 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940164+0000", 2024-09-21T15:01:36.991 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4099999999999998e-06 2024-09-21T15:01:36.991 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.991 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.991 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:36.991 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940169+0000", 2024-09-21T15:01:36.991 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3720000000000004e-06 2024-09-21T15:01:36.991 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.991 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.991 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:36.991 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940174+0000", 2024-09-21T15:01:36.992 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.724e-06 2024-09-21T15:01:36.992 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.992 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.992 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:36.992 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.962902+0000", 2024-09-21T15:01:36.992 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022728719000000001 2024-09-21T15:01:36.992 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.992 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.992 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:36.992 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.963014+0000", 2024-09-21T15:01:36.992 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000111245 2024-09-21T15:01:36.992 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:36.992 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:36.992 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:36.992 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.993 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.993 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:135044 3.a 3:53261383:::rbd_data.11365ed419e3.0000000000000037:snapdir [list-snaps] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:36.993 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.940484+0000", 2024-09-21T15:01:36.993 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 42.962951850000003, 2024-09-21T15:01:36.993 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022619169000000001, 2024-09-21T15:01:36.993 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:36.993 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:36.993 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:36.993 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:36.993 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:36.993 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 135044 2024-09-21T15:01:36.993 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.993 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:36.993 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.994 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:36.994 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940484+0000", 2024-09-21T15:01:36.994 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:36.994 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.994 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.994 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:36.994 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940484+0000", 2024-09-21T15:01:36.994 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:36.994 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.994 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.994 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:36.994 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940484+0000", 2024-09-21T15:01:36.994 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:36.994 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.994 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.994 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:36.995 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940487+0000", 2024-09-21T15:01:36.995 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8449999999999999e-06 2024-09-21T15:01:36.995 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.995 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.995 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:36.995 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940493+0000", 2024-09-21T15:01:36.995 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9599999999999997e-06 2024-09-21T15:01:36.995 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.995 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.995 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:36.995 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940497+0000", 2024-09-21T15:01:36.995 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9620000000000001e-06 2024-09-21T15:01:36.995 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.995 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.995 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:36.996 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.963034+0000", 2024-09-21T15:01:36.996 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022536878 2024-09-21T15:01:36.996 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:36.996 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:36.996 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:36.996 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.963103+0000", 2024-09-21T15:01:36.996 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9073999999999996e-05 2024-09-21T15:01:36.996 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:36.996 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:36.996 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:36.996 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:36.996 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:36.996 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:36.998 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T15:01:36.998 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:01:36.998+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:01:37.098 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:01:37.098+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:01:37.159 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:37.159 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:01:37.159 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T15:01:37.159 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:37.170 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T15:01:37.195 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T15:01:37.241 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:01:37.242+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:01:37.299 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:01:37.298+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:01:37.303 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:37.303 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:01:37.303 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T15:01:37.303 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T15:01:37.303 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:37.315 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T15:01:37.400 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:01:37.398+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:01:37.418 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:37.418 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T15:01:37.418 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T15:01:37.418 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T15:01:37.418 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.418 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:15130 3.2 3:47f1bbdb:::rbd_data.11361bed1f78.0000000000000067:head [sparse-read 0~147456] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-21T15:01:37.418 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:40.288284+0000", 2024-09-21T15:01:37.418 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 177.12882965399999, 2024-09-21T15:01:37.419 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1142935039999999, 2024-09-21T15:01:37.419 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:37.419 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:37.419 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:37.419 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:37.419 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:37.419 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 15130 2024-09-21T15:01:37.419 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.419 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:37.419 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.419 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:37.419 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288284+0000", 2024-09-21T15:01:37.419 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:37.419 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.420 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.420 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:37.420 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288284+0000", 2024-09-21T15:01:37.420 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:37.420 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.420 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.420 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:37.420 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288284+0000", 2024-09-21T15:01:37.420 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:37.420 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.420 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.420 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:37.420 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288288+0000", 2024-09-21T15:01:37.420 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5829999999999998e-06 2024-09-21T15:01:37.420 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.421 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.421 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:37.421 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288297+0000", 2024-09-21T15:01:37.421 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0669999999999998e-06 2024-09-21T15:01:37.421 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.421 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.421 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:37.421 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288305+0000", 2024-09-21T15:01:37.421 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8860000000000004e-06 2024-09-21T15:01:37.421 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.421 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.421 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:37.421 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288417+0000", 2024-09-21T15:01:37.421 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000112952 2024-09-21T15:01:37.422 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.422 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.422 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:37.422 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288428+0000", 2024-09-21T15:01:37.422 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0631e-05 2024-09-21T15:01:37.422 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.422 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.422 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:37.422 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.401876+0000", 2024-09-21T15:01:37.422 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1134479610000001 2024-09-21T15:01:37.422 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.422 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.422 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:37.422 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.402076+0000", 2024-09-21T15:01:37.423 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00019944200000000001 2024-09-21T15:01:37.423 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.423 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.423 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:37.423 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.402578+0000", 2024-09-21T15:01:37.423 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00050242899999999998 2024-09-21T15:01:37.423 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:37.423 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:37.423 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:37.423 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.423 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.423 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45708 3.2 3:415db91a:::rbd_data.1136ecbe5879.0000000000000000:28 [read 140288~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:37.423 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141801+0000", 2024-09-21T15:01:37.423 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 147.27531346000001, 2024-09-21T15:01:37.424 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034274200999999997, 2024-09-21T15:01:37.424 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:37.424 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:37.424 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:37.424 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:37.424 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:37.424 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45708 2024-09-21T15:01:37.424 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.424 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:37.424 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.424 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:37.424 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141801+0000", 2024-09-21T15:01:37.424 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:37.424 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.424 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.425 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:37.425 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141801+0000", 2024-09-21T15:01:37.425 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:37.425 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.425 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.425 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:37.425 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141800+0000", 2024-09-21T15:01:37.425 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:37.425 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.425 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.425 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:37.425 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141803+0000", 2024-09-21T15:01:37.425 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4210000000000002e-06 2024-09-21T15:01:37.425 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.425 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.426 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:37.426 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141812+0000", 2024-09-21T15:01:37.426 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.9800000000000004e-06 2024-09-21T15:01:37.426 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.426 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.426 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:37.426 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141816+0000", 2024-09-21T15:01:37.426 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4769999999999997e-06 2024-09-21T15:01:37.426 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.426 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.426 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:37.426 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.175865+0000", 2024-09-21T15:01:37.426 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034048875999999999 2024-09-21T15:01:37.427 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.427 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.427 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:37.427 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.175943+0000", 2024-09-21T15:01:37.427 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7464000000000002e-05 2024-09-21T15:01:37.427 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.427 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.427 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:37.427 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176075+0000", 2024-09-21T15:01:37.427 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013230800000000001 2024-09-21T15:01:37.427 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:37.427 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:37.427 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:37.428 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.428 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.428 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45709 3.2 3:415db91a:::rbd_data.1136ecbe5879.0000000000000000:28 [read 143360~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:37.428 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141820+0000", 2024-09-21T15:01:37.428 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 147.275293613, 2024-09-21T15:01:37.428 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034537536000000001, 2024-09-21T15:01:37.428 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:37.428 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:37.428 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:37.428 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:37.428 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:37.428 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45709 2024-09-21T15:01:37.428 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.428 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:37.428 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.429 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:37.429 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141820+0000", 2024-09-21T15:01:37.429 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:37.429 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.429 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.429 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:37.429 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141820+0000", 2024-09-21T15:01:37.429 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:37.429 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.429 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.429 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:37.429 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141820+0000", 2024-09-21T15:01:37.429 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:37.429 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.430 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.430 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:37.430 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141823+0000", 2024-09-21T15:01:37.430 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5110000000000002e-06 2024-09-21T15:01:37.430 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.430 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.430 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:37.430 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141829+0000", 2024-09-21T15:01:37.430 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0850000000000002e-06 2024-09-21T15:01:37.430 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.430 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.430 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:37.430 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141833+0000", 2024-09-21T15:01:37.431 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7620000000000003e-06 2024-09-21T15:01:37.431 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.431 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.431 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:37.431 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176103+0000", 2024-09-21T15:01:37.431 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034269933000000002 2024-09-21T15:01:37.431 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.431 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.431 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:37.431 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176161+0000", 2024-09-21T15:01:37.431 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7658999999999998e-05 2024-09-21T15:01:37.431 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.431 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.431 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:37.432 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176358+0000", 2024-09-21T15:01:37.432 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000196937 2024-09-21T15:01:37.432 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:37.432 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:37.432 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:37.432 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.432 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.432 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45710 3.2 3:415db91a:::rbd_data.1136ecbe5879.0000000000000000:28 [read 146432~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:37.432 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141838+0000", 2024-09-21T15:01:37.432 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 147.27527645000001, 2024-09-21T15:01:37.432 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034953674999999997, 2024-09-21T15:01:37.432 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:37.432 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:37.432 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:37.433 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:37.433 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:37.433 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45710 2024-09-21T15:01:37.433 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.433 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:37.433 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.433 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:37.433 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141838+0000", 2024-09-21T15:01:37.433 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:37.433 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.433 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.433 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:37.433 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141838+0000", 2024-09-21T15:01:37.433 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:37.434 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.434 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.434 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:37.434 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141837+0000", 2024-09-21T15:01:37.434 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:37.434 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.434 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.434 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:37.434 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141843+0000", 2024-09-21T15:01:37.434 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1619999999999996e-06 2024-09-21T15:01:37.434 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.434 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.435 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:37.435 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141850+0000", 2024-09-21T15:01:37.435 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4799999999999998e-06 2024-09-21T15:01:37.435 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.435 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.435 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:37.435 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141853+0000", 2024-09-21T15:01:37.435 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3050000000000001e-06 2024-09-21T15:01:37.435 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.435 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.435 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:37.435 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176490+0000", 2024-09-21T15:01:37.435 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034636413999999997 2024-09-21T15:01:37.436 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.436 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.436 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:37.436 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176606+0000", 2024-09-21T15:01:37.436 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000115841 2024-09-21T15:01:37.436 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.436 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.436 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:37.436 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176791+0000", 2024-09-21T15:01:37.436 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00018581500000000001 2024-09-21T15:01:37.436 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:37.436 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:37.436 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:37.437 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.437 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.437 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45711 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:37.437 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141856+0000", 2024-09-21T15:01:37.437 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 147.27525811300001, 2024-09-21T15:01:37.437 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034986308000000001, 2024-09-21T15:01:37.437 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:37.437 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:37.437 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:37.437 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:37.437 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:37.437 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45711 2024-09-21T15:01:37.437 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.438 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:37.438 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.438 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:37.438 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141856+0000", 2024-09-21T15:01:37.438 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:37.438 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.438 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.438 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:37.438 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141856+0000", 2024-09-21T15:01:37.438 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:37.438 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.438 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.438 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:37.439 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141856+0000", 2024-09-21T15:01:37.439 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:37.439 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.439 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.439 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:37.439 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141862+0000", 2024-09-21T15:01:37.439 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.567e-06 2024-09-21T15:01:37.439 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.439 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.439 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:37.439 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141868+0000", 2024-09-21T15:01:37.439 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9569999999999999e-06 2024-09-21T15:01:37.439 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.440 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.440 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:37.440 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141871+0000", 2024-09-21T15:01:37.440 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.146e-06 2024-09-21T15:01:37.440 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.440 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.440 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:37.440 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176831+0000", 2024-09-21T15:01:37.440 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034959935999999997 2024-09-21T15:01:37.440 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.440 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.440 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:37.440 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176842+0000", 2024-09-21T15:01:37.440 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1008e-05 2024-09-21T15:01:37.440 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:37.441 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:37.441 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:37.441 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.441 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.441 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45712 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:37.441 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141874+0000", 2024-09-21T15:01:37.441 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 147.27524021799999, 2024-09-21T15:01:37.441 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034993511999999997, 2024-09-21T15:01:37.441 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:37.441 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:37.441 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:37.441 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:37.441 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:37.441 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45712 2024-09-21T15:01:37.441 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.442 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:37.442 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.442 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:37.442 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141874+0000", 2024-09-21T15:01:37.442 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:37.442 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.442 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.442 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:37.442 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141874+0000", 2024-09-21T15:01:37.442 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:37.442 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.442 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.442 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:37.442 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141874+0000", 2024-09-21T15:01:37.443 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:37.443 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.443 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.443 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:37.443 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141875+0000", 2024-09-21T15:01:37.443 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5430000000000001e-06 2024-09-21T15:01:37.443 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.443 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.443 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:37.443 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141878+0000", 2024-09-21T15:01:37.443 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1420000000000002e-06 2024-09-21T15:01:37.443 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.443 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.443 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:37.444 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141881+0000", 2024-09-21T15:01:37.444 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.807e-06 2024-09-21T15:01:37.444 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.444 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.444 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:37.444 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176860+0000", 2024-09-21T15:01:37.444 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034979243 2024-09-21T15:01:37.444 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.444 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.444 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:37.444 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176867+0000", 2024-09-21T15:01:37.444 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0169999999999999e-06 2024-09-21T15:01:37.444 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:37.444 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:37.444 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:37.445 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.445 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.445 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45713 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:37.445 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141883+0000", 2024-09-21T15:01:37.445 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 147.27523064299999, 2024-09-21T15:01:37.445 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035010516999999998, 2024-09-21T15:01:37.445 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:37.445 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:37.445 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:37.445 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:37.445 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:37.445 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45713 2024-09-21T15:01:37.445 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.445 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:37.445 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.446 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:37.446 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141883+0000", 2024-09-21T15:01:37.446 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:37.446 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.446 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.446 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:37.446 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141883+0000", 2024-09-21T15:01:37.446 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:37.446 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.446 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.446 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:37.446 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141883+0000", 2024-09-21T15:01:37.446 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:37.447 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.447 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.447 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:37.447 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141885+0000", 2024-09-21T15:01:37.447 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4649999999999999e-06 2024-09-21T15:01:37.447 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.447 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.447 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:37.447 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141888+0000", 2024-09-21T15:01:37.447 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3890000000000001e-06 2024-09-21T15:01:37.447 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.448 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.448 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:37.448 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141891+0000", 2024-09-21T15:01:37.448 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9239999999999999e-06 2024-09-21T15:01:37.448 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.448 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.448 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:37.448 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176887+0000", 2024-09-21T15:01:37.448 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034996370999999998 2024-09-21T15:01:37.448 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.449 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.449 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:37.449 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176894+0000", 2024-09-21T15:01:37.449 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6069999999999996e-06 2024-09-21T15:01:37.449 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:37.449 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:37.449 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:37.449 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.449 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.449 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45714 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:37.449 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141893+0000", 2024-09-21T15:01:37.449 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 147.27522072799999, 2024-09-21T15:01:37.449 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035048215000000001, 2024-09-21T15:01:37.450 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:37.450 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:37.450 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:37.450 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:37.450 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:37.450 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45714 2024-09-21T15:01:37.450 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.450 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:37.450 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.450 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:37.450 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141893+0000", 2024-09-21T15:01:37.450 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:37.450 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.450 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.451 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:37.451 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141893+0000", 2024-09-21T15:01:37.451 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:37.451 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.451 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.451 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:37.451 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141893+0000", 2024-09-21T15:01:37.451 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:37.451 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.451 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.451 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:37.451 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141895+0000", 2024-09-21T15:01:37.451 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.703e-06 2024-09-21T15:01:37.451 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.451 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.452 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:37.452 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141900+0000", 2024-09-21T15:01:37.452 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.242e-06 2024-09-21T15:01:37.452 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.452 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.452 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:37.452 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141903+0000", 2024-09-21T15:01:37.452 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.117e-06 2024-09-21T15:01:37.452 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.452 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.452 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:37.452 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176930+0000", 2024-09-21T15:01:37.452 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035026750000000002 2024-09-21T15:01:37.452 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.452 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.453 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:37.453 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176942+0000", 2024-09-21T15:01:37.453 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1668000000000001e-05 2024-09-21T15:01:37.453 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:37.453 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:37.453 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:37.453 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.453 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.453 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45715 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:37.453 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141906+0000", 2024-09-21T15:01:37.453 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 147.27520856999999, 2024-09-21T15:01:37.453 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035066057999999997, 2024-09-21T15:01:37.453 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:37.453 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:37.453 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:37.453 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:37.454 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:37.454 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45715 2024-09-21T15:01:37.454 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.454 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:37.454 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.454 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:37.454 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141906+0000", 2024-09-21T15:01:37.454 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:37.454 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.454 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.454 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:37.454 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141906+0000", 2024-09-21T15:01:37.454 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:37.454 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.454 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.455 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:37.455 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141905+0000", 2024-09-21T15:01:37.455 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:37.455 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.455 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.455 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:37.455 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141907+0000", 2024-09-21T15:01:37.455 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5400000000000001e-06 2024-09-21T15:01:37.455 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.455 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.455 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:37.455 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141912+0000", 2024-09-21T15:01:37.455 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.502e-06 2024-09-21T15:01:37.455 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.455 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.456 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:37.456 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141915+0000", 2024-09-21T15:01:37.456 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0170000000000001e-06 2024-09-21T15:01:37.456 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.456 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.456 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:37.456 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176964+0000", 2024-09-21T15:01:37.456 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035048587999999999 2024-09-21T15:01:37.456 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.456 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.456 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:37.456 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176972+0000", 2024-09-21T15:01:37.456 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6580000000000002e-06 2024-09-21T15:01:37.456 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:37.456 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:37.456 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:37.457 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.457 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.457 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45716 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:37.457 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141918+0000", 2024-09-21T15:01:37.457 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 147.27519639400001, 2024-09-21T15:01:37.457 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035086223, 2024-09-21T15:01:37.457 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:37.457 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:37.457 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:37.457 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:37.457 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:37.457 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45716 2024-09-21T15:01:37.457 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.457 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:37.457 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.457 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:37.458 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141918+0000", 2024-09-21T15:01:37.458 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:37.458 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.458 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.458 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:37.458 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141918+0000", 2024-09-21T15:01:37.458 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:37.458 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.458 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.458 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:37.458 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141917+0000", 2024-09-21T15:01:37.458 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:37.458 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.458 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.458 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:37.459 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141919+0000", 2024-09-21T15:01:37.459 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5680000000000001e-06 2024-09-21T15:01:37.459 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.459 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.459 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:37.459 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141922+0000", 2024-09-21T15:01:37.459 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2789999999999999e-06 2024-09-21T15:01:37.459 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.459 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.459 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:37.459 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141925+0000", 2024-09-21T15:01:37.459 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9469999999999999e-06 2024-09-21T15:01:37.459 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.459 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.459 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:37.460 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176997+0000", 2024-09-21T15:01:37.460 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035071366999999999 2024-09-21T15:01:37.460 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.460 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.460 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:37.460 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177004+0000", 2024-09-21T15:01:37.460 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3309999999999998e-06 2024-09-21T15:01:37.460 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:37.460 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:37.460 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:37.460 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.460 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.460 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45717 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:37.461 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141928+0000", 2024-09-21T15:01:37.461 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 147.275186448, 2024-09-21T15:01:37.461 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035102573999999998, 2024-09-21T15:01:37.461 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:37.461 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:37.461 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:37.461 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:37.461 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:37.461 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45717 2024-09-21T15:01:37.461 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.461 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:37.461 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.461 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:37.461 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141928+0000", 2024-09-21T15:01:37.461 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:37.461 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.462 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.462 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:37.462 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141928+0000", 2024-09-21T15:01:37.462 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:37.462 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.462 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.462 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:37.462 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141927+0000", 2024-09-21T15:01:37.462 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:01:37.462 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.462 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.462 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:37.462 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141929+0000", 2024-09-21T15:01:37.463 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.533e-06 2024-09-21T15:01:37.463 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.463 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.463 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:37.463 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141934+0000", 2024-09-21T15:01:37.463 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1660000000000002e-06 2024-09-21T15:01:37.463 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.463 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.463 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:37.463 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141937+0000", 2024-09-21T15:01:37.463 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0309999999999999e-06 2024-09-21T15:01:37.463 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.463 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.463 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:37.464 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177023+0000", 2024-09-21T15:01:37.464 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035085595999999997 2024-09-21T15:01:37.464 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.464 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.464 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:37.464 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177030+0000", 2024-09-21T15:01:37.464 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.486e-06 2024-09-21T15:01:37.464 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:37.464 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:37.464 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:37.464 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.464 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.464 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45718 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:37.464 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141940+0000", 2024-09-21T15:01:37.465 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 147.27517459000001, 2024-09-21T15:01:37.465 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035124956999999998, 2024-09-21T15:01:37.465 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:37.465 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:37.465 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:37.465 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:37.465 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:37.465 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45718 2024-09-21T15:01:37.465 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.465 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:37.465 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.465 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:37.465 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141940+0000", 2024-09-21T15:01:37.465 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:37.465 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.466 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.466 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:37.466 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141940+0000", 2024-09-21T15:01:37.466 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:37.466 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.466 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.466 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:37.466 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141939+0000", 2024-09-21T15:01:37.466 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:37.466 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.466 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.467 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:37.467 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141941+0000", 2024-09-21T15:01:37.467 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.502e-06 2024-09-21T15:01:37.467 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.467 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.467 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:37.467 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141946+0000", 2024-09-21T15:01:37.467 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3449999999999996e-06 2024-09-21T15:01:37.467 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.467 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.467 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:37.467 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141949+0000", 2024-09-21T15:01:37.467 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9749999999999999e-06 2024-09-21T15:01:37.467 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.468 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.468 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:37.468 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177057+0000", 2024-09-21T15:01:37.468 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035108385999999998 2024-09-21T15:01:37.468 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.468 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.468 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:37.468 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177064+0000", 2024-09-21T15:01:37.468 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0099999999999998e-06 2024-09-21T15:01:37.468 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:37.468 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:37.468 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:37.468 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.469 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.469 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45719 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:37.469 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141952+0000", 2024-09-21T15:01:37.469 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 147.275162317, 2024-09-21T15:01:37.469 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035139108000000002, 2024-09-21T15:01:37.469 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:37.469 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:37.469 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:37.469 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:37.469 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:37.469 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45719 2024-09-21T15:01:37.469 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.469 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:37.469 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.470 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:37.470 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141952+0000", 2024-09-21T15:01:37.470 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:37.470 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.470 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.470 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:37.470 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141952+0000", 2024-09-21T15:01:37.470 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:37.470 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.470 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.470 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:37.470 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141952+0000", 2024-09-21T15:01:37.470 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:37.470 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.471 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.471 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:37.471 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141953+0000", 2024-09-21T15:01:37.471 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.486e-06 2024-09-21T15:01:37.471 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.471 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.471 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:37.471 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141959+0000", 2024-09-21T15:01:37.471 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4940000000000003e-06 2024-09-21T15:01:37.471 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.471 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.471 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:37.471 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141963+0000", 2024-09-21T15:01:37.471 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3540000000000002e-06 2024-09-21T15:01:37.471 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.472 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.472 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:37.472 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177084+0000", 2024-09-21T15:01:37.472 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035120983000000001 2024-09-21T15:01:37.472 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.472 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.472 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:37.472 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177091+0000", 2024-09-21T15:01:37.472 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0350000000000004e-06 2024-09-21T15:01:37.472 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:37.472 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:37.472 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:37.472 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.472 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.473 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45720 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:37.473 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141965+0000", 2024-09-21T15:01:37.473 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 147.27514884999999, 2024-09-21T15:01:37.473 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035155268000000003, 2024-09-21T15:01:37.473 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:37.473 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:37.473 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:37.473 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:37.473 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:37.473 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45720 2024-09-21T15:01:37.473 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.473 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:37.473 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.473 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:37.473 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141965+0000", 2024-09-21T15:01:37.473 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:37.473 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.474 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.474 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:37.474 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141965+0000", 2024-09-21T15:01:37.474 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:37.474 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.474 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.474 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:37.474 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141965+0000", 2024-09-21T15:01:37.474 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:37.474 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.474 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.474 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:37.475 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141966+0000", 2024-09-21T15:01:37.475 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.46e-06 2024-09-21T15:01:37.475 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.475 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.475 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:37.475 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141972+0000", 2024-09-21T15:01:37.475 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4210000000000003e-06 2024-09-21T15:01:37.475 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.475 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.475 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:37.475 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141975+0000", 2024-09-21T15:01:37.475 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.948e-06 2024-09-21T15:01:37.475 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.476 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.476 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:37.476 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177113+0000", 2024-09-21T15:01:37.476 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035137855000000003 2024-09-21T15:01:37.476 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.476 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.476 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:37.476 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177121+0000", 2024-09-21T15:01:37.476 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8269999999999998e-06 2024-09-21T15:01:37.476 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:37.476 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:37.476 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:37.476 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.476 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.476 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45721 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:37.477 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141977+0000", 2024-09-21T15:01:37.477 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 147.27513682700001, 2024-09-21T15:01:37.477 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035169984000000001, 2024-09-21T15:01:37.477 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:37.477 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:37.477 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:37.477 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:37.477 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:37.477 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45721 2024-09-21T15:01:37.477 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.477 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:37.477 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.477 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:37.477 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141977+0000", 2024-09-21T15:01:37.477 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:37.477 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.478 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.478 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:37.478 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141977+0000", 2024-09-21T15:01:37.478 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:37.478 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.478 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.478 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:37.478 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141977+0000", 2024-09-21T15:01:37.478 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:37.478 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.478 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.478 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:37.478 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141979+0000", 2024-09-21T15:01:37.478 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.511e-06 2024-09-21T15:01:37.479 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.479 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.479 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:37.479 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141982+0000", 2024-09-21T15:01:37.479 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4180000000000001e-06 2024-09-21T15:01:37.479 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.479 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.479 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:37.479 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141985+0000", 2024-09-21T15:01:37.479 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9450000000000002e-06 2024-09-21T15:01:37.479 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.480 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.480 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:37.480 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177139+0000", 2024-09-21T15:01:37.480 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035154409999999997 2024-09-21T15:01:37.480 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.480 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.480 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:37.480 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177147+0000", 2024-09-21T15:01:37.480 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9570000000000002e-06 2024-09-21T15:01:37.480 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:37.480 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:37.480 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:37.480 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.481 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.481 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45722 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:37.481 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141987+0000", 2024-09-21T15:01:37.481 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 147.27512685599999, 2024-09-21T15:01:37.481 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035188492000000002, 2024-09-21T15:01:37.481 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:37.481 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:37.481 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:37.481 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:37.481 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:37.481 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45722 2024-09-21T15:01:37.481 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.481 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:37.481 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.481 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:37.482 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141987+0000", 2024-09-21T15:01:37.482 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:37.482 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.482 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.482 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:37.482 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141987+0000", 2024-09-21T15:01:37.482 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:37.482 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.482 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.482 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:37.482 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141987+0000", 2024-09-21T15:01:37.482 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:37.482 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.482 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.482 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:37.483 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141988+0000", 2024-09-21T15:01:37.483 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.463e-06 2024-09-21T15:01:37.483 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.483 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.483 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:37.483 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141994+0000", 2024-09-21T15:01:37.483 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.361e-06 2024-09-21T15:01:37.483 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.483 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.483 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:37.483 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141997+0000", 2024-09-21T15:01:37.483 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0359999999999998e-06 2024-09-21T15:01:37.483 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.483 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.484 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:37.484 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177169+0000", 2024-09-21T15:01:37.484 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035171651999999998 2024-09-21T15:01:37.484 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.484 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.484 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:37.484 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177176+0000", 2024-09-21T15:01:37.484 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2189999999999998e-06 2024-09-21T15:01:37.484 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:37.484 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:37.484 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:37.484 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.484 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.484 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45723 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:37.484 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141999+0000", 2024-09-21T15:01:37.485 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 147.27511468899999, 2024-09-21T15:01:37.485 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035197701999999997, 2024-09-21T15:01:37.485 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:37.485 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:37.485 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:37.485 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:37.485 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:37.485 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45723 2024-09-21T15:01:37.485 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.485 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:37.485 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.485 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:37.485 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141999+0000", 2024-09-21T15:01:37.486 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:37.486 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.486 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.486 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:37.486 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141999+0000", 2024-09-21T15:01:37.486 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:37.486 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.486 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.486 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:37.486 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141999+0000", 2024-09-21T15:01:37.486 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:37.486 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.486 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.486 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:37.486 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142003+0000", 2024-09-21T15:01:37.487 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5049999999999999e-06 2024-09-21T15:01:37.487 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.487 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.487 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:37.487 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142008+0000", 2024-09-21T15:01:37.487 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6579999999999999e-06 2024-09-21T15:01:37.487 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.487 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.487 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:37.487 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142011+0000", 2024-09-21T15:01:37.487 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0529999999999998e-06 2024-09-21T15:01:37.487 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.487 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.487 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:37.487 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177194+0000", 2024-09-21T15:01:37.488 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035182221999999999 2024-09-21T15:01:37.488 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.488 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.488 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:37.488 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177197+0000", 2024-09-21T15:01:37.488 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5030000000000002e-06 2024-09-21T15:01:37.488 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:37.488 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:37.488 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:37.488 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.488 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.488 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45724 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:37.488 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.142015+0000", 2024-09-21T15:01:37.488 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 147.275098722, 2024-09-21T15:01:37.489 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035195345000000003, 2024-09-21T15:01:37.489 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:37.489 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:37.489 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:37.489 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:37.489 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:37.489 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45724 2024-09-21T15:01:37.489 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.489 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:37.489 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.489 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:37.489 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142015+0000", 2024-09-21T15:01:37.489 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:37.489 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.489 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.489 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:37.490 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142015+0000", 2024-09-21T15:01:37.490 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:37.490 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.490 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.490 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:37.490 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142015+0000", 2024-09-21T15:01:37.490 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:37.490 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.490 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.490 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:37.490 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142017+0000", 2024-09-21T15:01:37.490 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5540000000000001e-06 2024-09-21T15:01:37.490 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.490 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.491 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:37.491 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142022+0000", 2024-09-21T15:01:37.491 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4229999999999996e-06 2024-09-21T15:01:37.491 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.491 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.491 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:37.491 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142025+0000", 2024-09-21T15:01:37.491 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.056e-06 2024-09-21T15:01:37.491 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.491 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.491 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:37.491 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177207+0000", 2024-09-21T15:01:37.491 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035182089 2024-09-21T15:01:37.491 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.491 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.491 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:37.492 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177211+0000", 2024-09-21T15:01:37.492 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4649999999999999e-06 2024-09-21T15:01:37.492 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:37.492 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:37.492 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:37.492 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.492 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.492 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45725 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:37.492 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.142028+0000", 2024-09-21T15:01:37.492 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 147.27508644299999, 2024-09-21T15:01:37.492 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035195731000000001, 2024-09-21T15:01:37.492 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:37.492 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:37.492 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:37.492 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:37.493 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:37.493 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45725 2024-09-21T15:01:37.493 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.493 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:37.493 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.493 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:37.493 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142028+0000", 2024-09-21T15:01:37.493 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:37.493 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.493 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.493 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:37.493 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142028+0000", 2024-09-21T15:01:37.493 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:37.493 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.493 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.493 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:37.494 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142027+0000", 2024-09-21T15:01:37.494 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:37.494 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.494 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.494 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:37.494 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142029+0000", 2024-09-21T15:01:37.494 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5269999999999999e-06 2024-09-21T15:01:37.494 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.494 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.494 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:37.494 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142032+0000", 2024-09-21T15:01:37.494 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3249999999999999e-06 2024-09-21T15:01:37.494 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.494 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.495 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:37.495 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142035+0000", 2024-09-21T15:01:37.495 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.897e-06 2024-09-21T15:01:37.495 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.495 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.495 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:37.495 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177220+0000", 2024-09-21T15:01:37.495 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035184871 2024-09-21T15:01:37.495 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.495 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.495 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:37.495 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177223+0000", 2024-09-21T15:01:37.496 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3680000000000002e-06 2024-09-21T15:01:37.496 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:37.496 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:37.496 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:37.496 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.496 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.496 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45726 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:37.496 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.142037+0000", 2024-09-21T15:01:37.496 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 147.27507672799999, 2024-09-21T15:01:37.496 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035199273000000003, 2024-09-21T15:01:37.496 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:37.496 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:37.496 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:37.496 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:37.496 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:37.497 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45726 2024-09-21T15:01:37.497 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.497 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:37.497 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.497 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:37.497 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142037+0000", 2024-09-21T15:01:37.497 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:37.497 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.497 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.497 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:37.497 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142037+0000", 2024-09-21T15:01:37.497 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:37.497 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.497 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.497 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:37.497 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142037+0000", 2024-09-21T15:01:37.498 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:37.498 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.498 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.498 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:37.498 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142038+0000", 2024-09-21T15:01:37.498 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.344e-06 2024-09-21T15:01:37.498 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.498 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.498 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:37.498 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142044+0000", 2024-09-21T15:01:37.498 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1930000000000001e-06 2024-09-21T15:01:37.498 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.498 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.499 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:37.499 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142047+0000", 2024-09-21T15:01:37.499 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0280000000000001e-06 2024-09-21T15:01:37.499 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.499 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.499 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:37.499 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177233+0000", 2024-09-21T15:01:37.499 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035186394000000003 2024-09-21T15:01:37.499 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.499 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.499 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:37.499 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177237+0000", 2024-09-21T15:01:37.499 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5530000000000001e-06 2024-09-21T15:01:37.499 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:37.499 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:37.500 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:37.500 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:37.500 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:37.500 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:37.501 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T15:01:37.501 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:01:37.498+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:01:37.602 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:01:37.602+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:01:37.657 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:37.658 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:01:37.658 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T15:01:37.658 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:37.667 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T15:01:37.703 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:01:37.700+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:01:37.777 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:37.777 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:01:37.777 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T15:01:37.777 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T15:01:37.777 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:37.787 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T15:01:37.804 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:01:37.802+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:01:37.896 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:37.896 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T15:01:37.896 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T15:01:37.896 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T15:01:37.896 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.896 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139776 3.16 3:6c38d0e2:::rbd_data.113664780c85.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:37.896 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.223442+0000", 2024-09-21T15:01:37.896 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 35.670127299000001, 2024-09-21T15:01:37.897 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0315488150000001, 2024-09-21T15:01:37.897 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:37.897 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:37.897 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:37.897 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:37.897 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:37.897 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139776 2024-09-21T15:01:37.897 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.897 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:37.897 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.897 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:37.897 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223442+0000", 2024-09-21T15:01:37.897 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:37.897 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.897 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.898 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:37.898 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223442+0000", 2024-09-21T15:01:37.898 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:37.898 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.898 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.898 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:37.898 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223442+0000", 2024-09-21T15:01:37.898 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:37.898 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.898 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.898 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:37.898 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223446+0000", 2024-09-21T15:01:37.898 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2549999999999999e-06 2024-09-21T15:01:37.898 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.898 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.899 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:37.899 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223454+0000", 2024-09-21T15:01:37.899 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8199999999999997e-06 2024-09-21T15:01:37.899 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.899 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.899 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:37.899 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223460+0000", 2024-09-21T15:01:37.899 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6330000000000001e-06 2024-09-21T15:01:37.899 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.899 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.899 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:37.899 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223542+0000", 2024-09-21T15:01:37.899 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1929999999999997e-05 2024-09-21T15:01:37.899 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.899 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.899 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:37.900 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223552+0000", 2024-09-21T15:01:37.900 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0627e-05 2024-09-21T15:01:37.900 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.900 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.900 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:37.900 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.254634+0000", 2024-09-21T15:01:37.900 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.031081972 2024-09-21T15:01:37.900 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.900 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.900 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:37.900 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.254796+0000", 2024-09-21T15:01:37.900 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000161879 2024-09-21T15:01:37.900 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.900 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.901 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:37.901 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.254991+0000", 2024-09-21T15:01:37.901 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00019526499999999999 2024-09-21T15:01:37.901 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:37.901 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:37.901 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:37.901 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.901 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.901 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139785 3.11 3:8abbb109:::rbd_data.113664780c85.0000000000000070:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:37.901 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.223531+0000", 2024-09-21T15:01:37.901 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 35.670038558000002, 2024-09-21T15:01:37.901 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0187143510000001, 2024-09-21T15:01:37.901 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:37.901 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:37.901 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:37.901 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:37.902 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:37.902 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139785 2024-09-21T15:01:37.902 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.902 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:37.902 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.902 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:37.902 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223531+0000", 2024-09-21T15:01:37.902 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:37.902 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.902 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.902 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:37.902 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223531+0000", 2024-09-21T15:01:37.902 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:37.902 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.903 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.903 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:37.903 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223531+0000", 2024-09-21T15:01:37.903 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:37.903 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.903 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.903 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:37.903 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223534+0000", 2024-09-21T15:01:37.903 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2650000000000001e-06 2024-09-21T15:01:37.903 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.903 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.903 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:37.903 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223541+0000", 2024-09-21T15:01:37.903 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8990000000000004e-06 2024-09-21T15:01:37.903 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.904 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.904 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:37.904 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223546+0000", 2024-09-21T15:01:37.904 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3399999999999997e-06 2024-09-21T15:01:37.904 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.904 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.904 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:37.904 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223724+0000", 2024-09-21T15:01:37.904 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017745799999999999 2024-09-21T15:01:37.904 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.904 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.904 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:37.904 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223735+0000", 2024-09-21T15:01:37.904 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1610999999999999e-05 2024-09-21T15:01:37.904 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.904 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.905 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:37.905 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242113+0000", 2024-09-21T15:01:37.905 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.018377879 2024-09-21T15:01:37.905 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.905 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.905 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:37.905 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242246+0000", 2024-09-21T15:01:37.905 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013231500000000001 2024-09-21T15:01:37.905 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:37.905 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:37.905 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:37.905 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.905 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.905 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139790 3.11 3:8a18c7d8:::rbd_data.113664780c85.000000000000008d:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:37.905 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.223578+0000", 2024-09-21T15:01:37.905 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 35.669991562999996, 2024-09-21T15:01:37.906 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0187879719999999, 2024-09-21T15:01:37.906 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:37.906 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:37.906 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:37.906 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:37.906 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:37.906 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139790 2024-09-21T15:01:37.906 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.906 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:37.906 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.906 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:37.906 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223578+0000", 2024-09-21T15:01:37.906 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:37.906 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.906 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.907 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:37.907 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223578+0000", 2024-09-21T15:01:37.907 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:37.907 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.907 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.907 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:37.907 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223578+0000", 2024-09-21T15:01:37.907 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:37.907 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.907 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.907 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:37.907 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223581+0000", 2024-09-21T15:01:37.907 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6879999999999999e-06 2024-09-21T15:01:37.907 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.907 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.907 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:37.908 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223586+0000", 2024-09-21T15:01:37.908 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7080000000000002e-06 2024-09-21T15:01:37.908 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.908 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.908 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:37.908 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223591+0000", 2024-09-21T15:01:37.908 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5639999999999998e-06 2024-09-21T15:01:37.908 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.908 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.908 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:37.908 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223761+0000", 2024-09-21T15:01:37.908 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000170213 2024-09-21T15:01:37.908 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.908 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.908 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:37.908 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223769+0000", 2024-09-21T15:01:37.909 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7049999999999999e-06 2024-09-21T15:01:37.909 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.909 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.909 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:37.909 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242269+0000", 2024-09-21T15:01:37.909 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.018500288 2024-09-21T15:01:37.909 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.909 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.909 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:37.909 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242366+0000", 2024-09-21T15:01:37.909 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.7126000000000005e-05 2024-09-21T15:01:37.909 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:37.909 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:37.910 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:37.910 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.910 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.910 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139792 3.16 3:6ce5caf3:::rbd_data.113664780c85.000000000000009b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:37.910 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.223595+0000", 2024-09-21T15:01:37.910 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 35.669974609999997, 2024-09-21T15:01:37.910 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0317517899999999, 2024-09-21T15:01:37.910 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:37.910 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:37.910 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:37.910 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:37.910 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:37.910 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139792 2024-09-21T15:01:37.910 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.910 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:37.911 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.911 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:37.911 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223595+0000", 2024-09-21T15:01:37.911 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:37.911 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.911 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.911 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:37.911 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223595+0000", 2024-09-21T15:01:37.911 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:37.911 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.911 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.911 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:37.911 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223595+0000", 2024-09-21T15:01:37.911 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:37.911 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.911 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.912 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:37.912 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223598+0000", 2024-09-21T15:01:37.912 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4620000000000001e-06 2024-09-21T15:01:37.912 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.912 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.912 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:37.912 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223604+0000", 2024-09-21T15:01:37.912 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6879999999999996e-06 2024-09-21T15:01:37.912 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.912 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.912 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:37.912 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223609+0000", 2024-09-21T15:01:37.912 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9490000000000004e-06 2024-09-21T15:01:37.912 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.912 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.912 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:37.913 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224067+0000", 2024-09-21T15:01:37.913 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00045780700000000002 2024-09-21T15:01:37.913 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.913 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.913 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:37.913 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224075+0000", 2024-09-21T15:01:37.913 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6610000000000006e-06 2024-09-21T15:01:37.913 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.913 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.913 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:37.913 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255141+0000", 2024-09-21T15:01:37.913 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0310657480000001 2024-09-21T15:01:37.913 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.913 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.913 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:37.914 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255347+0000", 2024-09-21T15:01:37.914 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000205837 2024-09-21T15:01:37.914 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:37.914 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:37.914 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:37.914 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.914 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.914 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139794 3.11 3:8cf3a2f4:::rbd_data.113664780c85.00000000000000a9:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:37.914 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.224011+0000", 2024-09-21T15:01:37.914 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 35.669558582000001, 2024-09-21T15:01:37.914 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.018507791, 2024-09-21T15:01:37.914 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:37.914 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:37.914 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:37.914 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:37.914 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:37.915 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139794 2024-09-21T15:01:37.915 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.915 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:37.915 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.915 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:37.915 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224011+0000", 2024-09-21T15:01:37.915 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:37.915 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.915 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.915 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:37.915 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224011+0000", 2024-09-21T15:01:37.915 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:37.915 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.915 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.916 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:37.916 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224010+0000", 2024-09-21T15:01:37.916 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:37.916 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.916 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.916 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:37.916 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224016+0000", 2024-09-21T15:01:37.916 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6550000000000001e-06 2024-09-21T15:01:37.916 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.916 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.916 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:37.916 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224028+0000", 2024-09-21T15:01:37.916 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2422999999999999e-05 2024-09-21T15:01:37.916 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.916 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.916 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:37.917 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224036+0000", 2024-09-21T15:01:37.917 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7749999999999993e-06 2024-09-21T15:01:37.917 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.917 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.917 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:37.917 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225176+0000", 2024-09-21T15:01:37.917 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0011401740000000001 2024-09-21T15:01:37.917 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.917 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.917 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:37.917 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225187+0000", 2024-09-21T15:01:37.917 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0951999999999999e-05 2024-09-21T15:01:37.917 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.917 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.917 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:37.918 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242392+0000", 2024-09-21T15:01:37.918 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.017204905 2024-09-21T15:01:37.918 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.918 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.918 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:37.918 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242519+0000", 2024-09-21T15:01:37.918 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012680499999999999 2024-09-21T15:01:37.918 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:37.918 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:37.918 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:37.918 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.918 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.918 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139800 3.16 3:68b050ed:::rbd_data.113664780c85.00000000000000d3:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:37.918 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.224579+0000", 2024-09-21T15:01:37.918 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 35.668990823000001, 2024-09-21T15:01:37.919 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.030932637, 2024-09-21T15:01:37.919 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:37.919 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:37.919 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:37.919 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:37.919 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:37.919 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139800 2024-09-21T15:01:37.919 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.919 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:37.919 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.919 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:37.919 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224579+0000", 2024-09-21T15:01:37.919 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:37.919 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.919 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.919 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:37.920 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224579+0000", 2024-09-21T15:01:37.920 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:37.920 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.920 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.920 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:37.920 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224578+0000", 2024-09-21T15:01:37.920 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:37.920 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.920 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.920 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:37.920 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224582+0000", 2024-09-21T15:01:37.920 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0029999999999999e-06 2024-09-21T15:01:37.920 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.920 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.920 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:37.920 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224588+0000", 2024-09-21T15:01:37.921 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8120000000000002e-06 2024-09-21T15:01:37.921 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.921 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.921 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:37.921 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224593+0000", 2024-09-21T15:01:37.921 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7469999999999996e-06 2024-09-21T15:01:37.921 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.921 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.921 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:37.921 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224648+0000", 2024-09-21T15:01:37.921 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4693e-05 2024-09-21T15:01:37.921 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.921 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.921 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:37.921 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224656+0000", 2024-09-21T15:01:37.922 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8429999999999993e-06 2024-09-21T15:01:37.922 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.922 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.922 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:37.922 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255382+0000", 2024-09-21T15:01:37.922 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0307266070000001 2024-09-21T15:01:37.922 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.922 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.922 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:37.922 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255512+0000", 2024-09-21T15:01:37.922 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000129367 2024-09-21T15:01:37.922 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:37.922 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:37.922 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:37.922 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.923 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.923 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139804 3.11 3:88f9ddce:::rbd_data.113664780c85.00000000000000ef:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:37.923 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.224632+0000", 2024-09-21T15:01:37.923 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 35.668937841999998, 2024-09-21T15:01:37.923 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.018021302, 2024-09-21T15:01:37.923 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:37.923 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:37.923 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:37.923 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:37.923 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:37.923 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139804 2024-09-21T15:01:37.923 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.923 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:37.923 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.923 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:37.924 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224632+0000", 2024-09-21T15:01:37.924 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:37.924 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.924 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.924 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:37.924 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224632+0000", 2024-09-21T15:01:37.924 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:37.924 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.924 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.924 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:37.924 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224632+0000", 2024-09-21T15:01:37.924 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:37.924 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.924 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.924 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:37.924 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224635+0000", 2024-09-21T15:01:37.925 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2710000000000001e-06 2024-09-21T15:01:37.925 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.925 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.925 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:37.925 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224640+0000", 2024-09-21T15:01:37.925 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6779999999999997e-06 2024-09-21T15:01:37.925 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.925 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.925 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:37.925 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224645+0000", 2024-09-21T15:01:37.925 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9339999999999997e-06 2024-09-21T15:01:37.925 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.925 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.925 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:37.925 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225307+0000", 2024-09-21T15:01:37.925 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00066201699999999999 2024-09-21T15:01:37.926 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.926 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.926 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:37.926 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225312+0000", 2024-09-21T15:01:37.926 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9799999999999998e-06 2024-09-21T15:01:37.926 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.926 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.926 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:37.926 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242548+0000", 2024-09-21T15:01:37.926 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.017235603 2024-09-21T15:01:37.926 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.926 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.926 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:37.927 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242653+0000", 2024-09-21T15:01:37.927 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010522 2024-09-21T15:01:37.927 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:37.927 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:37.927 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:37.927 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.927 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.927 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139809 3.11 3:884da946:::rbd_data.113664780c85.0000000000000118:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:37.927 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.224667+0000", 2024-09-21T15:01:37.927 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 35.668902797999998, 2024-09-21T15:01:37.927 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0181101960000001, 2024-09-21T15:01:37.927 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:37.927 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:37.927 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:37.927 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:37.928 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:37.928 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139809 2024-09-21T15:01:37.928 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.928 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:37.928 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.928 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:37.928 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224667+0000", 2024-09-21T15:01:37.928 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:37.928 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.928 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.928 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:37.928 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224667+0000", 2024-09-21T15:01:37.928 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:37.928 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.929 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.929 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:37.929 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224667+0000", 2024-09-21T15:01:37.929 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:37.929 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.929 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.929 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:37.929 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224669+0000", 2024-09-21T15:01:37.929 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.452e-06 2024-09-21T15:01:37.929 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.929 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.929 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:37.929 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224675+0000", 2024-09-21T15:01:37.929 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.818e-06 2024-09-21T15:01:37.929 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.929 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.930 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:37.930 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224680+0000", 2024-09-21T15:01:37.930 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6319999999999999e-06 2024-09-21T15:01:37.930 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.930 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.930 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:37.930 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225400+0000", 2024-09-21T15:01:37.930 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00072012699999999999 2024-09-21T15:01:37.930 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.930 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.930 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:37.930 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225409+0000", 2024-09-21T15:01:37.930 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1039999999999996e-06 2024-09-21T15:01:37.930 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.931 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.931 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:37.931 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242675+0000", 2024-09-21T15:01:37.931 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0172663319999999 2024-09-21T15:01:37.931 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.931 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.931 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:37.931 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242777+0000", 2024-09-21T15:01:37.931 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010197899999999999 2024-09-21T15:01:37.931 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:37.931 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:37.931 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:37.931 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.931 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.931 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139868 3.16 3:69f82090:::rbd_data.113664780c85.00000000000000f1:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:37.931 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.225982+0000", 2024-09-21T15:01:37.932 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 35.667587298000001, 2024-09-21T15:01:37.932 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0296791729999999, 2024-09-21T15:01:37.932 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:37.932 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:37.932 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:37.932 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:37.932 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:37.932 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139868 2024-09-21T15:01:37.932 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.932 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:37.932 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.932 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:37.932 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225982+0000", 2024-09-21T15:01:37.932 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:37.932 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.932 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.933 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:37.933 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225982+0000", 2024-09-21T15:01:37.933 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:37.933 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.933 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.933 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:37.933 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225982+0000", 2024-09-21T15:01:37.933 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:37.933 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.933 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.933 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:37.933 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225985+0000", 2024-09-21T15:01:37.933 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.74e-06 2024-09-21T15:01:37.933 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.933 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.934 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:37.934 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225990+0000", 2024-09-21T15:01:37.934 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7039999999999999e-06 2024-09-21T15:01:37.934 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.934 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.934 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:37.934 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225995+0000", 2024-09-21T15:01:37.934 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7249999999999997e-06 2024-09-21T15:01:37.934 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.934 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.934 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:37.934 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226299+0000", 2024-09-21T15:01:37.934 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00030383199999999997 2024-09-21T15:01:37.934 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.935 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.935 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:37.935 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226308+0000", 2024-09-21T15:01:37.935 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.9740000000000008e-06 2024-09-21T15:01:37.935 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.935 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.935 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:37.935 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255539+0000", 2024-09-21T15:01:37.935 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0292305960000001 2024-09-21T15:01:37.935 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.935 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.935 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:37.935 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255662+0000", 2024-09-21T15:01:37.936 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012307400000000001 2024-09-21T15:01:37.936 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:37.936 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:37.936 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:37.936 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.936 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.936 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139869 3.16 3:6b6c90a4:::rbd_data.113664780c85.00000000000000fe:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:37.936 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.226006+0000", 2024-09-21T15:01:37.936 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 35.667563682000001, 2024-09-21T15:01:37.936 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.029790518, 2024-09-21T15:01:37.936 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:37.936 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:37.936 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:37.936 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:37.936 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:37.937 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139869 2024-09-21T15:01:37.937 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.937 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:37.937 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.937 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:37.937 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226006+0000", 2024-09-21T15:01:37.937 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:37.937 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.937 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.937 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:37.937 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226006+0000", 2024-09-21T15:01:37.937 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:37.937 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.937 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.937 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:37.938 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226006+0000", 2024-09-21T15:01:37.938 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:37.938 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.938 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.938 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:37.938 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226008+0000", 2024-09-21T15:01:37.938 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5569999999999998e-06 2024-09-21T15:01:37.938 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.938 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.938 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:37.938 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226014+0000", 2024-09-21T15:01:37.938 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2079999999999997e-06 2024-09-21T15:01:37.938 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.938 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.939 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:37.939 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226020+0000", 2024-09-21T15:01:37.939 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7429999999999999e-06 2024-09-21T15:01:37.939 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.939 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.939 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:37.939 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226330+0000", 2024-09-21T15:01:37.939 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00030963300000000002 2024-09-21T15:01:37.939 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.939 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.939 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:37.939 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226336+0000", 2024-09-21T15:01:37.939 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6340000000000003e-06 2024-09-21T15:01:37.939 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.939 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.939 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:37.940 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255687+0000", 2024-09-21T15:01:37.940 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0293507150000001 2024-09-21T15:01:37.940 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.940 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.940 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:37.940 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255797+0000", 2024-09-21T15:01:37.940 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000109471 2024-09-21T15:01:37.940 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:37.940 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:37.940 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:37.940 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.940 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.940 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139884 3.16 3:69f10324:::rbd_data.113664780c85.0000000000000161:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:37.940 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.227573+0000", 2024-09-21T15:01:37.940 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 35.665997040999997, 2024-09-21T15:01:37.940 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0283568759999999, 2024-09-21T15:01:37.941 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:37.941 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:37.941 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:37.941 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:37.941 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:37.941 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139884 2024-09-21T15:01:37.941 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.941 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:37.941 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.941 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:37.941 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227573+0000", 2024-09-21T15:01:37.941 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:37.941 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.941 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.941 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:37.941 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227573+0000", 2024-09-21T15:01:37.942 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:37.942 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.942 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.942 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:37.942 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227572+0000", 2024-09-21T15:01:37.942 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:37.942 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.942 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.942 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:37.942 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227575+0000", 2024-09-21T15:01:37.942 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7769999999999998e-06 2024-09-21T15:01:37.942 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.942 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.942 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:37.943 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227581+0000", 2024-09-21T15:01:37.943 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7359999999999998e-06 2024-09-21T15:01:37.943 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.943 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.943 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:37.943 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227585+0000", 2024-09-21T15:01:37.943 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5090000000000004e-06 2024-09-21T15:01:37.943 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.943 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.943 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:37.943 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227753+0000", 2024-09-21T15:01:37.943 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00016761599999999999 2024-09-21T15:01:37.943 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.943 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.943 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:37.943 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227761+0000", 2024-09-21T15:01:37.944 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0579999999999998e-06 2024-09-21T15:01:37.944 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.944 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.944 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:37.944 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255828+0000", 2024-09-21T15:01:37.944 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0280671349999999 2024-09-21T15:01:37.944 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.944 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.944 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:37.944 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255930+0000", 2024-09-21T15:01:37.944 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000101372 2024-09-21T15:01:37.944 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:37.944 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:37.944 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:37.944 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.944 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.945 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139913 3.16 3:6f81b46d:::rbd_data.113664780c85.0000000000000074:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:37.945 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.227685+0000", 2024-09-21T15:01:37.945 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 35.665884861999999, 2024-09-21T15:01:37.945 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0283644750000001, 2024-09-21T15:01:37.945 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:37.945 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:37.945 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:37.945 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:37.945 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:37.945 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139913 2024-09-21T15:01:37.945 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.945 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:37.945 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.945 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:37.945 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227685+0000", 2024-09-21T15:01:37.945 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:37.946 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.946 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.946 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:37.946 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227685+0000", 2024-09-21T15:01:37.946 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:37.946 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.946 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.946 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:37.946 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227685+0000", 2024-09-21T15:01:37.946 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:37.946 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.946 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.946 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:37.946 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227687+0000", 2024-09-21T15:01:37.946 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2929999999999999e-06 2024-09-21T15:01:37.946 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.947 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.947 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:37.947 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227693+0000", 2024-09-21T15:01:37.947 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1070000000000001e-06 2024-09-21T15:01:37.947 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.947 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.947 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:37.947 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227698+0000", 2024-09-21T15:01:37.947 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9339999999999997e-06 2024-09-21T15:01:37.947 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.947 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.947 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:37.947 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227784+0000", 2024-09-21T15:01:37.947 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6096999999999999e-05 2024-09-21T15:01:37.947 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.948 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.948 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:37.948 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227791+0000", 2024-09-21T15:01:37.948 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9090000000000003e-06 2024-09-21T15:01:37.948 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.948 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.948 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:37.948 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255955+0000", 2024-09-21T15:01:37.948 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0281643579999999 2024-09-21T15:01:37.948 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.948 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.948 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:37.948 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256049+0000", 2024-09-21T15:01:37.948 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4047000000000003e-05 2024-09-21T15:01:37.948 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:37.948 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:37.949 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:37.949 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.949 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.949 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139988 3.11 3:88a29fa4:::rbd_data.113664780c85.00000000000000bd:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:37.949 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.229772+0000", 2024-09-21T15:01:37.949 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 35.663797363, 2024-09-21T15:01:37.949 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.017967668, 2024-09-21T15:01:37.949 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:37.949 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:37.949 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:37.949 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:37.949 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:37.949 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139988 2024-09-21T15:01:37.949 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.949 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:37.950 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.950 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:37.950 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229772+0000", 2024-09-21T15:01:37.950 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:37.950 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.950 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.950 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:37.950 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229772+0000", 2024-09-21T15:01:37.950 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:37.950 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.950 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.950 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:37.950 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229772+0000", 2024-09-21T15:01:37.950 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:37.950 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.951 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.951 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:37.951 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229776+0000", 2024-09-21T15:01:37.951 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6890000000000002e-06 2024-09-21T15:01:37.951 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.951 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.951 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:37.951 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229783+0000", 2024-09-21T15:01:37.951 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2280000000000001e-06 2024-09-21T15:01:37.951 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.951 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.951 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:37.951 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229789+0000", 2024-09-21T15:01:37.951 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1700000000000002e-06 2024-09-21T15:01:37.951 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.951 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.952 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:37.952 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.230106+0000", 2024-09-21T15:01:37.952 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00031700800000000003 2024-09-21T15:01:37.952 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.952 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.952 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:37.952 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.230115+0000", 2024-09-21T15:01:37.952 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4190000000000008e-06 2024-09-21T15:01:37.952 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.952 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.952 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:37.952 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242935+0000", 2024-09-21T15:01:37.952 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0128191449999999 2024-09-21T15:01:37.952 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.952 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.952 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for missing object", 2024-09-21T15:01:37.953 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.243078+0000", 2024-09-21T15:01:37.953 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000142984 2024-09-21T15:01:37.953 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.953 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.953 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:37.953 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.247659+0000", 2024-09-21T15:01:37.953 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0045813030000000001 2024-09-21T15:01:37.953 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.953 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.953 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:37.953 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.247740+0000", 2024-09-21T15:01:37.953 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1216000000000003e-05 2024-09-21T15:01:37.953 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:37.953 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:37.953 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:37.953 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.954 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.954 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139994 3.16 3:6b9df8d1:::rbd_data.113664780c85.00000000000000e7:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:37.954 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.231370+0000", 2024-09-21T15:01:37.954 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 35.662199285, 2024-09-21T15:01:37.954 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.024795576, 2024-09-21T15:01:37.954 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:37.954 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:37.954 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:37.954 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:37.954 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:37.954 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139994 2024-09-21T15:01:37.954 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.954 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:37.954 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.954 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:37.954 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231370+0000", 2024-09-21T15:01:37.955 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:37.955 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.955 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.955 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:37.955 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231370+0000", 2024-09-21T15:01:37.955 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:37.955 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.955 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.955 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:37.955 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231369+0000", 2024-09-21T15:01:37.955 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:37.955 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.955 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.955 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:37.955 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231376+0000", 2024-09-21T15:01:37.955 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5509999999999996e-06 2024-09-21T15:01:37.956 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.956 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.956 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:37.956 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231387+0000", 2024-09-21T15:01:37.956 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.061e-05 2024-09-21T15:01:37.956 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.956 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.956 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:37.956 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231394+0000", 2024-09-21T15:01:37.956 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3799999999999996e-06 2024-09-21T15:01:37.956 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.956 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.956 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:37.956 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231459+0000", 2024-09-21T15:01:37.956 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4716999999999997e-05 2024-09-21T15:01:37.957 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.957 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.957 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:37.957 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231468+0000", 2024-09-21T15:01:37.957 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1479999999999995e-06 2024-09-21T15:01:37.957 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.957 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.957 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:37.957 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256074+0000", 2024-09-21T15:01:37.957 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0246064180000001 2024-09-21T15:01:37.957 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.957 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.957 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:37.957 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256166+0000", 2024-09-21T15:01:37.957 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1862999999999998e-05 2024-09-21T15:01:37.957 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:37.958 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:37.958 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:37.958 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.958 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.958 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140012 3.16 3:6bd6dfb6:::rbd_data.113664780c85.0000000000000165:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:37.958 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.231890+0000", 2024-09-21T15:01:37.958 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 35.661679526, 2024-09-21T15:01:37.958 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.024408881, 2024-09-21T15:01:37.958 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:37.958 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:37.958 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:37.958 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:37.958 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:37.958 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140012 2024-09-21T15:01:37.958 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.959 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:37.959 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.959 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:37.959 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231890+0000", 2024-09-21T15:01:37.959 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:37.959 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.959 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.959 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:37.959 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231890+0000", 2024-09-21T15:01:37.959 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:37.959 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.959 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.959 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:37.959 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231889+0000", 2024-09-21T15:01:37.959 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:37.960 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.960 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.960 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:37.960 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231895+0000", 2024-09-21T15:01:37.960 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9569999999999999e-06 2024-09-21T15:01:37.960 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.960 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.960 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:37.960 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231910+0000", 2024-09-21T15:01:37.960 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4549000000000001e-05 2024-09-21T15:01:37.960 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.960 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.960 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:37.960 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231917+0000", 2024-09-21T15:01:37.960 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5730000000000002e-06 2024-09-21T15:01:37.960 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.961 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.961 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:37.961 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231988+0000", 2024-09-21T15:01:37.961 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0852999999999994e-05 2024-09-21T15:01:37.961 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.961 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.961 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:37.961 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.232000+0000", 2024-09-21T15:01:37.961 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2159e-05 2024-09-21T15:01:37.961 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.961 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.961 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:37.961 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256192+0000", 2024-09-21T15:01:37.961 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.024192073 2024-09-21T15:01:37.961 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.962 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.962 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:37.962 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256299+0000", 2024-09-21T15:01:37.962 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000106733 2024-09-21T15:01:37.962 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:37.962 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:37.962 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:37.962 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.962 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.962 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140055 3.16 3:69bb9803:::rbd_data.113664780c85.00000000000000da:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:37.962 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.233922+0000", 2024-09-21T15:01:37.962 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 35.659647997, 2024-09-21T15:01:37.962 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.022511768, 2024-09-21T15:01:37.962 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:37.962 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:37.962 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:37.963 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:37.963 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:37.963 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140055 2024-09-21T15:01:37.963 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.963 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:37.963 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.963 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:37.963 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233922+0000", 2024-09-21T15:01:37.963 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:37.963 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.963 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.963 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:37.963 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233922+0000", 2024-09-21T15:01:37.963 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:37.963 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.964 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.964 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:37.964 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233921+0000", 2024-09-21T15:01:37.964 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:37.964 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.964 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.964 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:37.964 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233927+0000", 2024-09-21T15:01:37.964 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0360000000000003e-06 2024-09-21T15:01:37.964 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.964 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.964 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:37.964 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233937+0000", 2024-09-21T15:01:37.964 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6439999999999995e-06 2024-09-21T15:01:37.964 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.965 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.965 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:37.965 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233944+0000", 2024-09-21T15:01:37.965 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7710000000000006e-06 2024-09-21T15:01:37.965 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.965 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.965 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:37.965 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233996+0000", 2024-09-21T15:01:37.965 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1597000000000001e-05 2024-09-21T15:01:37.965 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.965 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.965 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:37.965 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.234005+0000", 2024-09-21T15:01:37.965 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5960000000000004e-06 2024-09-21T15:01:37.965 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.966 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.966 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:37.966 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256332+0000", 2024-09-21T15:01:37.966 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0223271949999999 2024-09-21T15:01:37.966 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.966 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.966 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:37.966 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256434+0000", 2024-09-21T15:01:37.966 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010180599999999999 2024-09-21T15:01:37.966 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:37.966 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:37.966 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:37.966 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.967 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.967 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140143 3.16 3:6d3ed596:::rbd_data.113664780c85.0000000000000184:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:37.967 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.239496+0000", 2024-09-21T15:01:37.967 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 35.654073468, 2024-09-21T15:01:37.967 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.017076576, 2024-09-21T15:01:37.967 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:37.967 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:37.967 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:37.967 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:37.967 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:37.967 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140143 2024-09-21T15:01:37.967 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.967 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:37.967 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.967 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:37.967 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239496+0000", 2024-09-21T15:01:37.968 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:37.968 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.968 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.968 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:37.968 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239496+0000", 2024-09-21T15:01:37.968 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:37.968 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.968 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.968 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:37.968 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239496+0000", 2024-09-21T15:01:37.968 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:37.968 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.968 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.968 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:37.968 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239498+0000", 2024-09-21T15:01:37.969 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.509e-06 2024-09-21T15:01:37.969 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.969 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.969 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:37.969 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239504+0000", 2024-09-21T15:01:37.969 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6869999999999999e-06 2024-09-21T15:01:37.969 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.969 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.969 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:37.969 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239509+0000", 2024-09-21T15:01:37.969 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8860000000000003e-06 2024-09-21T15:01:37.969 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.969 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.969 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:37.969 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239572+0000", 2024-09-21T15:01:37.970 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2908000000000001e-05 2024-09-21T15:01:37.970 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.970 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.970 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:37.970 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239580+0000", 2024-09-21T15:01:37.970 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4270000000000008e-06 2024-09-21T15:01:37.970 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.970 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.970 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:37.970 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256457+0000", 2024-09-21T15:01:37.970 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0168770979999999 2024-09-21T15:01:37.970 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.970 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.970 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:37.970 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256573+0000", 2024-09-21T15:01:37.971 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00011538400000000001 2024-09-21T15:01:37.971 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:37.971 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:37.971 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:37.971 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.971 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.971 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140163 3.16 3:694e82eb:::rbd_data.113664780c85.0000000000000052:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:37.971 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.240334+0000", 2024-09-21T15:01:37.971 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 35.653235756000001, 2024-09-21T15:01:37.971 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.016365537, 2024-09-21T15:01:37.971 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:37.971 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:37.971 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:37.971 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:37.972 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:37.972 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140163 2024-09-21T15:01:37.972 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.972 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:37.972 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.972 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:37.972 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240334+0000", 2024-09-21T15:01:37.972 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:37.972 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.972 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.972 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:37.972 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240334+0000", 2024-09-21T15:01:37.972 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:37.972 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.972 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.972 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:37.973 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240334+0000", 2024-09-21T15:01:37.973 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:37.973 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.973 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.973 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:37.973 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240337+0000", 2024-09-21T15:01:37.973 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.089e-06 2024-09-21T15:01:37.973 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.973 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.973 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:37.973 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240343+0000", 2024-09-21T15:01:37.973 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4420000000000003e-06 2024-09-21T15:01:37.973 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.973 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.973 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:37.974 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240348+0000", 2024-09-21T15:01:37.974 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1039999999999998e-06 2024-09-21T15:01:37.974 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.974 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.974 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:37.974 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240451+0000", 2024-09-21T15:01:37.974 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000102996 2024-09-21T15:01:37.974 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.974 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.974 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:37.974 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240460+0000", 2024-09-21T15:01:37.974 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5369999999999997e-06 2024-09-21T15:01:37.974 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.975 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.975 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:37.975 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256600+0000", 2024-09-21T15:01:37.975 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.016139884 2024-09-21T15:01:37.975 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.975 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.975 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:37.975 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256700+0000", 2024-09-21T15:01:37.975 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9844000000000002e-05 2024-09-21T15:01:37.975 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:37.975 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:37.975 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:37.975 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.975 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.976 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140164 3.16 3:68bfdc57:::rbd_data.113664780c85.0000000000000053:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:37.976 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.240358+0000", 2024-09-21T15:01:37.976 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 35.653212062000001, 2024-09-21T15:01:37.976 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.01646094, 2024-09-21T15:01:37.976 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:37.976 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:37.976 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:37.976 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:37.976 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:37.976 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140164 2024-09-21T15:01:37.976 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.976 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:37.976 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.976 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:37.976 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240358+0000", 2024-09-21T15:01:37.976 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:37.977 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.977 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.977 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:37.977 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240358+0000", 2024-09-21T15:01:37.977 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:37.977 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.977 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.977 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:37.977 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240357+0000", 2024-09-21T15:01:37.977 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:37.977 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.977 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.977 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:37.977 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240360+0000", 2024-09-21T15:01:37.978 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7089999999999998e-06 2024-09-21T15:01:37.978 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.978 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.978 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:37.978 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240368+0000", 2024-09-21T15:01:37.978 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7270000000000007e-06 2024-09-21T15:01:37.978 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.978 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.978 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:37.978 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240373+0000", 2024-09-21T15:01:37.978 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4060000000000004e-06 2024-09-21T15:01:37.978 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.978 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.978 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:37.979 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240473+0000", 2024-09-21T15:01:37.979 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000100132 2024-09-21T15:01:37.979 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.979 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.979 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:37.979 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240476+0000", 2024-09-21T15:01:37.979 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.332e-06 2024-09-21T15:01:37.979 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.979 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.979 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:37.979 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256724+0000", 2024-09-21T15:01:37.979 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0162471930000001 2024-09-21T15:01:37.979 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.979 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.979 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:37.980 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256819+0000", 2024-09-21T15:01:37.980 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5007999999999996e-05 2024-09-21T15:01:37.980 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:37.980 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:37.980 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:37.980 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.980 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.980 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140188 3.16 3:6f6217d4:::rbd_data.113664780c85.00000000000000fb:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:37.980 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.240978+0000", 2024-09-21T15:01:37.980 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 35.652591403000002, 2024-09-21T15:01:37.980 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.015983809, 2024-09-21T15:01:37.980 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:37.980 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:37.980 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:37.980 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:37.981 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:37.981 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140188 2024-09-21T15:01:37.981 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.981 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:37.981 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.981 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:37.981 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240978+0000", 2024-09-21T15:01:37.981 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:37.981 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.981 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.981 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:37.981 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240978+0000", 2024-09-21T15:01:37.981 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:37.981 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.981 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.982 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:37.982 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240978+0000", 2024-09-21T15:01:37.982 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:37.982 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.982 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.982 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:37.982 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240981+0000", 2024-09-21T15:01:37.982 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1559999999999999e-06 2024-09-21T15:01:37.982 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.982 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.982 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:37.982 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240987+0000", 2024-09-21T15:01:37.982 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.229e-06 2024-09-21T15:01:37.982 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.983 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.983 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:37.983 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240992+0000", 2024-09-21T15:01:37.983 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9710000000000003e-06 2024-09-21T15:01:37.983 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.983 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.983 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:37.983 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.241361+0000", 2024-09-21T15:01:37.983 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000369236 2024-09-21T15:01:37.983 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.983 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.983 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:37.983 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.241369+0000", 2024-09-21T15:01:37.983 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6199999999999999e-06 2024-09-21T15:01:37.983 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.984 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.984 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:37.984 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256844+0000", 2024-09-21T15:01:37.984 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0154751609999999 2024-09-21T15:01:37.984 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:37.984 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:37.984 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:37.984 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256962+0000", 2024-09-21T15:01:37.984 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000117964 2024-09-21T15:01:37.984 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:37.984 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:37.984 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:37.984 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:37.984 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:37.984 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:37.985 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T15:01:37.986 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:01:37.986+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:01:38.086 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:01:38.086+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:01:38.103 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:38.103 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:38.113 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T15:01:38.187 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:01:38.186+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:01:38.199 INFO:teuthology.orchestra.run.smithi032.stderr:nodeep-scrub is unset 2024-09-21T15:01:38.226 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:38.226 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:38.236 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T15:01:38.287 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:01:38.280+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:01:38.342 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:38.342 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:38.352 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T15:01:38.387 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:01:38.384+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:01:38.458 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:38.458 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:38.468 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T15:01:38.489 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:01:38.486+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:01:38.572 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:38.572 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:38.583 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T15:01:38.590 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:01:38.590+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:01:38.689 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:38.689 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:38.691 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:01:38.690+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:01:38.699 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T15:01:38.792 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:01:38.790+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:01:38.811 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:38.812 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:38.822 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T15:01:38.893 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:01:38.888+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:01:38.928 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:38.928 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:38.938 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T15:01:38.994 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:01:38.988+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:01:39.090 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:39.091 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:39.095 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:01:39.094+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:01:39.101 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T15:01:39.197 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:01:39.194+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:01:39.203 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T15:01:39.226 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:39.226 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T15:01:39.226 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.226 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:168331 3.c 3.ed1185ac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e207)", 2024-09-21T15:01:39.226 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:39.226831+0000", 2024-09-21T15:01:39.226 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 8.1114999999999998e-05, 2024-09-21T15:01:39.226 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.7670000000000005e-05, 2024-09-21T15:01:39.226 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:39.226 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:39.227 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:39.227 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:39.227 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:39.227 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 168331 2024-09-21T15:01:39.227 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.227 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:39.227 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.227 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:39.227 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:39.226831+0000", 2024-09-21T15:01:39.227 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:39.227 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.227 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.227 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:39.227 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:39.226831+0000", 2024-09-21T15:01:39.228 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:39.228 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.228 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.228 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:39.228 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:39.226828+0000", 2024-09-21T15:01:39.228 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T15:01:39.228 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.228 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.228 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:39.228 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:39.226839+0000", 2024-09-21T15:01:39.228 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0535e-05 2024-09-21T15:01:39.228 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.228 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.228 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:39.228 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:39.226858+0000", 2024-09-21T15:01:39.229 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.8924e-05 2024-09-21T15:01:39.229 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.229 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.229 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:39.229 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:39.226867+0000", 2024-09-21T15:01:39.229 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8100000000000004e-06 2024-09-21T15:01:39.229 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.229 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.229 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:39.229 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:39.226914+0000", 2024-09-21T15:01:39.229 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.693e-05 2024-09-21T15:01:39.229 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:39.229 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:39.229 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:39.229 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:39.230 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-21T15:01:39.230 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 1 2024-09-21T15:01:39.230 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:39.230 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T15:01:39.237 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T15:01:39.251 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T15:01:39.252 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:01:39.252 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:01:39.252 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:01:39.252 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:01:39.252 INFO:teuthology.orchestra.run.smithi032.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-21T15:01:39.252 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:01:39.252 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:01:39.252 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:01:39.255 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-21T15:01:39.255 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-21T15:01:39.297 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:01:39.292+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:01:39.345 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:39.346 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:01:39.346 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T15:01:39.346 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T15:01:39.346 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:39.356 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T15:01:39.405 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:01:39.406+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:01:39.465 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:39.465 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T15:01:39.466 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T15:01:39.466 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T15:01:39.466 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.466 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:97677 3.15 3:a8e72133:::rbd_data.1136bfc79a70.0000000000000005:head [write 0~4440027 [fadvise_sequential+fadvise_nocache],write 4751360~147456 [fadvise_sequential+fadvise_nocache],write 4964352~147456 [fadvise_sequential+fadvise_nocache],write 5177344~65536 [fadvise_sequential+fadvise_nocache],write 5259264~147456 [fadvise_sequential+fadvise_nocache],write 5472256~147456 [fadvise_sequential+fadvise_nocache],write 5685248~229376 [fadvise_sequential+fadvise_nocache],write 5980160~147456 [fadvise_sequential+fadvise_nocache],write 6193152~229376 [fadvise_sequential+fadvise_nocache],write 6488064~147456 [fadvise_sequential+fadvise_nocache],write 6701056~229376 [fadvise_sequential+fadvise_nocache],write 6995968~147456 [fadvise_sequential+fadvise_nocache],write 7208960~229376 [fadvise_sequential+fadvise_nocache],write 7503872~147456 [fadvise_sequential+fadvise_nocache],write 7716864~229376 [fadvise_sequential+fadvise_nocache],write 8044544~114688 [fadvise_sequential+fadvise_nocache],write 8257536~114688 [fadvise_sequential+fadvise_nocache],write 8388608~65536 [fadvise_sequential+fadvise_nocache],write 8552448~114688 [fadvise_sequential+fadvise_nocache],write 8765440~114688 [fadvise_sequential+fadvise_nocache],write 8929280~32768 [fadvise_sequential+fadvise_nocache],write 9142272~32768 [fadvise_sequential+fadvise_nocache],write 9355264~32768 [fadvise_sequential+fadvise_nocache],write 9437184~32768 [fadvise_sequential+fadvise_nocache],write 9650176~32768 [fadvise_sequential+fadvise_nocache],write 9863168~32768 [fadvise_sequential+fadvise_nocache],write 9961472~131072 [fadvise_sequential+fadvise_nocache],write 10108928~196608 [fadvise_sequential+fadvise_nocache],write 10321920~278528 [fadvise_sequential+fadvise_nocache],write 10616832~196608 [fadvise_sequential+fadvise_nocache],write 10829824~278528 [fadvise_sequential+fadvise_nocache],write 11124736~196608 [fadvise_sequential+fadvise_nocache],write 11337728~196608 [fadvise_sequential+fadvise_nocache],write 11550720~65536 [fadvise_sequential+fadvise_nocache],write 11649024~98304 [fadvise_sequential+fadvise_nocache],write 11763712~65536 [fadvise_sequential+fadvise_nocache],write 11862016~98304 [fadvise_sequential+fadvise_nocache],write 11976704~65536 [fadvise_sequential+fadvise_nocache],write 12058624~65536 [fadvise_sequential+fadvise_nocache],write 12156928~98304 [fadvise_sequential+fadvise_nocache],write 12271616~65536 [fadvise_sequential+fadvise_nocache],write 12369920~98304 [fadvise_sequential+fadvise_nocache],write 12484608~65536 [fadvise_sequential+fadvise_nocache],write 12664832~98304 [fadvise_sequential+fadvise_nocache],write 12877824~98304 [fadvise_sequential+fadvise_nocache],write 13090816~16384 [fadvise_sequential+fadvise_nocache],write 13172736~98304 [fadvise_sequential+fadvise_nocache],write 13385728~98304 [fadvise_sequential+fadvise_nocache],write 13598720~32768 [fadvise_sequential+fadvise_nocache],write 13680640~98304 [fadvise_sequential+fadvise_nocache],write 13893632~98304 [fadvise_sequential+fadvise_nocache],write 14106624~49152 [fadvise_sequential+fadvise_nocache],write 14794752~81920 [fadvise_sequential+fadvise_nocache],write 15007744~81920 [fadvise_sequential+fadvise_nocache],write 15302656~81920 [fadvise_sequential+fadvise_nocache],write 15515648~81920 [fadvise_sequential+fadvise_nocache],write 15810560~81920 [fadvise_sequential+fadvise_nocache],write 16023552~81920 [fadvise_sequential+fadvise_nocache],write 16236544~16384 [fadvise_sequential+fadvise_nocache],write 16318464~81920 [fadvise_sequential+fadvise_nocache],write 16531456~81920 [fadvise_sequential+fadvise_nocache],write 16744448~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:01:39.466 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.302946+0000", 2024-09-21T15:01:39.466 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 94.160077260999998, 2024-09-21T15:01:39.466 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.091231204999999996, 2024-09-21T15:01:39.466 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:39.466 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:39.466 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:39.466 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:39.466 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:39.466 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 97677 2024-09-21T15:01:39.466 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.467 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:39.467 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.467 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:39.467 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.302946+0000", 2024-09-21T15:01:39.467 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:39.467 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.467 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.467 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:39.467 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.302946+0000", 2024-09-21T15:01:39.467 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:39.467 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.467 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.467 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:39.468 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.302942+0000", 2024-09-21T15:01:39.468 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999962 2024-09-21T15:01:39.468 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.468 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.468 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:39.468 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311253+0000", 2024-09-21T15:01:39.468 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0083105360000000003 2024-09-21T15:01:39.468 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.468 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.468 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:39.468 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311283+0000", 2024-09-21T15:01:39.468 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0065000000000001e-05 2024-09-21T15:01:39.468 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.468 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.468 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:39.469 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311296+0000", 2024-09-21T15:01:39.469 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3206e-05 2024-09-21T15:01:39.469 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.469 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.469 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:39.469 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311404+0000", 2024-09-21T15:01:39.469 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000107514 2024-09-21T15:01:39.469 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.469 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.469 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:39.469 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311795+0000", 2024-09-21T15:01:39.469 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000390913 2024-09-21T15:01:39.469 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.469 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.470 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 6", 2024-09-21T15:01:39.470 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.314686+0000", 2024-09-21T15:01:39.470 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0028915659999999999 2024-09-21T15:01:39.470 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.470 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.470 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:01:39.470 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.368178+0000", 2024-09-21T15:01:39.470 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.053491706 2024-09-21T15:01:39.470 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.470 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.470 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:01:39.470 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.393925+0000", 2024-09-21T15:01:39.470 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025747289 2024-09-21T15:01:39.470 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.471 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.471 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:01:39.471 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.394127+0000", 2024-09-21T15:01:39.471 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000201444 2024-09-21T15:01:39.471 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.471 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.471 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:39.471 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.394177+0000", 2024-09-21T15:01:39.471 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0689000000000002e-05 2024-09-21T15:01:39.471 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:39.471 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:39.471 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:39.471 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.471 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.471 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:97673 3.1d e122/24 3:bd91946e:::rbd_data.1136bfc79a70.0000000000000002:head v 122'162, mlcod=120'160)", 2024-09-21T15:01:39.472 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.313730+0000", 2024-09-21T15:01:39.472 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 94.149292935999995, 2024-09-21T15:01:39.472 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055373462999999998, 2024-09-21T15:01:39.472 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:39.472 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:39.472 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:39.472 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.472 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:39.472 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.313730+0000", 2024-09-21T15:01:39.472 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:39.472 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.472 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.472 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:39.472 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.313730+0000", 2024-09-21T15:01:39.473 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:39.473 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.473 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.473 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:39.473 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.313730+0000", 2024-09-21T15:01:39.473 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:39.473 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.473 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.473 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:39.473 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.336113+0000", 2024-09-21T15:01:39.473 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022382962999999999 2024-09-21T15:01:39.473 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.473 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.473 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:39.473 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.336140+0000", 2024-09-21T15:01:39.474 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7248999999999999e-05 2024-09-21T15:01:39.474 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.474 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.474 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:39.474 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.336151+0000", 2024-09-21T15:01:39.474 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1015e-05 2024-09-21T15:01:39.474 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.474 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.474 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:39.474 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.355428+0000", 2024-09-21T15:01:39.474 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019276379999999999 2024-09-21T15:01:39.474 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.474 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.474 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:39.475 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.355484+0000", 2024-09-21T15:01:39.475 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6820999999999997e-05 2024-09-21T15:01:39.475 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.475 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.475 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:01:39.475 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.369040+0000", 2024-09-21T15:01:39.475 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013555466 2024-09-21T15:01:39.475 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.475 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.475 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:39.475 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.369104+0000", 2024-09-21T15:01:39.475 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3986999999999996e-05 2024-09-21T15:01:39.475 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:39.475 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:39.475 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:39.476 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.476 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.476 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139890 3.e 3:71dba096:::rbd_data.113664780c85.000000000000018b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:39.476 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.226444+0000", 2024-09-21T15:01:39.476 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 37.236579294000002, 2024-09-21T15:01:39.476 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.028209567, 2024-09-21T15:01:39.476 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:39.476 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:39.476 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:39.476 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:39.476 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:39.476 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139890 2024-09-21T15:01:39.476 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.476 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:39.476 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.477 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:39.477 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226444+0000", 2024-09-21T15:01:39.477 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:39.477 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.477 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.477 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:39.477 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226444+0000", 2024-09-21T15:01:39.477 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:39.477 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.477 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.477 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:39.477 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226443+0000", 2024-09-21T15:01:39.477 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:39.477 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.477 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.477 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:39.478 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226448+0000", 2024-09-21T15:01:39.478 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5499999999999996e-06 2024-09-21T15:01:39.478 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.478 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.478 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:39.478 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226456+0000", 2024-09-21T15:01:39.478 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8679999999999999e-06 2024-09-21T15:01:39.478 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.478 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.478 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:39.478 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226462+0000", 2024-09-21T15:01:39.478 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1800000000000001e-06 2024-09-21T15:01:39.478 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.478 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.479 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:39.479 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226545+0000", 2024-09-21T15:01:39.479 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2527000000000005e-05 2024-09-21T15:01:39.479 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.479 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.479 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:39.479 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226553+0000", 2024-09-21T15:01:39.479 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8019999999999998e-06 2024-09-21T15:01:39.479 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.479 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.479 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:39.479 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.254491+0000", 2024-09-21T15:01:39.479 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.02793785 2024-09-21T15:01:39.479 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.479 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.480 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:39.480 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.254654+0000", 2024-09-21T15:01:39.480 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00016245800000000001 2024-09-21T15:01:39.480 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:39.480 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:39.480 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:39.480 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.480 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.480 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139897 3.e 3:76ddeb1d:::rbd_data.113664780c85.0000000000000004:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:39.480 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.227198+0000", 2024-09-21T15:01:39.480 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 37.235825689000002, 2024-09-21T15:01:39.480 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.027799822, 2024-09-21T15:01:39.480 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:39.480 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:39.481 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:39.481 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:39.481 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:39.481 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139897 2024-09-21T15:01:39.481 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.481 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:39.481 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.481 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:39.481 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227198+0000", 2024-09-21T15:01:39.481 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:39.481 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.481 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.481 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:39.481 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227198+0000", 2024-09-21T15:01:39.481 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:39.482 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.482 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.482 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:39.482 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227197+0000", 2024-09-21T15:01:39.482 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:39.482 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.482 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.482 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:39.482 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227203+0000", 2024-09-21T15:01:39.482 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4819999999999999e-06 2024-09-21T15:01:39.482 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.482 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.482 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:39.482 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227220+0000", 2024-09-21T15:01:39.483 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6671e-05 2024-09-21T15:01:39.483 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.483 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.483 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:39.483 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227229+0000", 2024-09-21T15:01:39.483 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0499999999999997e-06 2024-09-21T15:01:39.483 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.483 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.483 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:39.483 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227293+0000", 2024-09-21T15:01:39.483 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4434e-05 2024-09-21T15:01:39.483 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.483 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.483 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:39.484 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227302+0000", 2024-09-21T15:01:39.484 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0929999999999992e-06 2024-09-21T15:01:39.484 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.484 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.484 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:39.484 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.254720+0000", 2024-09-21T15:01:39.484 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0274177550000001 2024-09-21T15:01:39.484 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.484 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.484 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:39.484 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.254854+0000", 2024-09-21T15:01:39.484 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000133416 2024-09-21T15:01:39.484 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.484 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.484 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:39.485 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.254998+0000", 2024-09-21T15:01:39.485 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014400099999999999 2024-09-21T15:01:39.485 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:39.485 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:39.485 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:39.485 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.485 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.485 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139902 3.e 3:76b9b799:::rbd_data.113664780c85.0000000000000021:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:39.485 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.227319+0000", 2024-09-21T15:01:39.485 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 37.235704308000003, 2024-09-21T15:01:39.485 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0279584639999999, 2024-09-21T15:01:39.485 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:39.485 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:39.485 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:39.485 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:39.486 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:39.486 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139902 2024-09-21T15:01:39.486 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.486 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:39.486 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.486 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:39.486 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227319+0000", 2024-09-21T15:01:39.486 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:39.486 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.486 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.486 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:39.486 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227319+0000", 2024-09-21T15:01:39.486 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:39.486 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.487 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.487 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:39.487 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227319+0000", 2024-09-21T15:01:39.487 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:39.487 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.487 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.487 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:39.487 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227322+0000", 2024-09-21T15:01:39.487 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1980000000000001e-06 2024-09-21T15:01:39.487 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.487 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.487 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:39.487 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227328+0000", 2024-09-21T15:01:39.487 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4230000000000002e-06 2024-09-21T15:01:39.487 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.488 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.488 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:39.488 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227333+0000", 2024-09-21T15:01:39.488 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0560000000000004e-06 2024-09-21T15:01:39.488 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.488 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.488 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:39.488 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227729+0000", 2024-09-21T15:01:39.488 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00039591799999999999 2024-09-21T15:01:39.488 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.488 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.488 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:39.488 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227738+0000", 2024-09-21T15:01:39.488 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8119999999999997e-06 2024-09-21T15:01:39.489 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.489 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.489 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:39.489 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255021+0000", 2024-09-21T15:01:39.489 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0272826779999999 2024-09-21T15:01:39.489 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.489 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.489 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:39.489 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255147+0000", 2024-09-21T15:01:39.489 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012595600000000001 2024-09-21T15:01:39.489 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.489 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.489 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:39.489 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255278+0000", 2024-09-21T15:01:39.490 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013081400000000001 2024-09-21T15:01:39.490 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:39.490 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:39.490 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:39.490 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.490 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.490 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139945 3.e 3:7756a55d:::rbd_data.113664780c85.0000000000000154:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:39.490 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.229015+0000", 2024-09-21T15:01:39.490 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 37.234008504999998, 2024-09-21T15:01:39.490 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.026468825, 2024-09-21T15:01:39.490 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:39.490 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:39.490 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:39.490 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:39.491 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:39.491 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139945 2024-09-21T15:01:39.491 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.491 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:39.491 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.491 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:39.491 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229015+0000", 2024-09-21T15:01:39.491 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:39.491 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.491 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.491 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:39.491 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229015+0000", 2024-09-21T15:01:39.491 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:39.491 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.491 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.492 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:39.492 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229014+0000", 2024-09-21T15:01:39.492 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:39.492 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.492 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.492 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:39.492 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229017+0000", 2024-09-21T15:01:39.492 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7310000000000002e-06 2024-09-21T15:01:39.492 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.492 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.492 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:39.492 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229023+0000", 2024-09-21T15:01:39.492 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8150000000000002e-06 2024-09-21T15:01:39.492 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.492 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.493 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:39.493 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229028+0000", 2024-09-21T15:01:39.493 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7650000000000001e-06 2024-09-21T15:01:39.493 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.493 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.493 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:39.493 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229082+0000", 2024-09-21T15:01:39.493 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4521999999999997e-05 2024-09-21T15:01:39.493 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.493 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.493 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:39.493 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229092+0000", 2024-09-21T15:01:39.493 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.3940000000000002e-06 2024-09-21T15:01:39.493 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.494 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.494 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:39.494 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255382+0000", 2024-09-21T15:01:39.494 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0262900930000001 2024-09-21T15:01:39.494 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.494 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.494 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:39.494 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255484+0000", 2024-09-21T15:01:39.494 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000101932 2024-09-21T15:01:39.494 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:39.494 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:39.494 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:39.494 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.494 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.495 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139967 3.e 3:74e90d85:::rbd_data.113664780c85.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:39.495 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.229124+0000", 2024-09-21T15:01:39.495 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 37.233899565000002, 2024-09-21T15:01:39.495 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.026724381, 2024-09-21T15:01:39.495 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:39.495 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:39.495 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:39.495 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:39.495 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:39.495 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139967 2024-09-21T15:01:39.495 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.495 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:39.495 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.495 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:39.496 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229124+0000", 2024-09-21T15:01:39.496 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:39.496 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.496 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.496 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:39.496 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229124+0000", 2024-09-21T15:01:39.496 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:39.496 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.496 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.496 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:39.496 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229123+0000", 2024-09-21T15:01:39.496 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:39.496 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.496 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.497 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:39.497 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229126+0000", 2024-09-21T15:01:39.497 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4779999999999998e-06 2024-09-21T15:01:39.497 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.497 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.497 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:39.497 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229131+0000", 2024-09-21T15:01:39.497 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.729e-06 2024-09-21T15:01:39.497 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.497 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.497 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:39.497 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229135+0000", 2024-09-21T15:01:39.497 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5859999999999998e-06 2024-09-21T15:01:39.497 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.498 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.498 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:39.498 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229558+0000", 2024-09-21T15:01:39.498 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00042262499999999999 2024-09-21T15:01:39.498 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.498 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.498 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:39.498 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229569+0000", 2024-09-21T15:01:39.498 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0985000000000001e-05 2024-09-21T15:01:39.498 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.498 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.498 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:39.498 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255509+0000", 2024-09-21T15:01:39.499 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.025940246 2024-09-21T15:01:39.499 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.499 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.499 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:39.499 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255662+0000", 2024-09-21T15:01:39.499 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015324199999999999 2024-09-21T15:01:39.499 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.499 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.499 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:39.499 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255848+0000", 2024-09-21T15:01:39.499 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00018583599999999999 2024-09-21T15:01:39.499 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:39.499 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:39.499 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:39.499 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.500 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.500 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140010 3.e 3:735c06f2:::rbd_data.113664780c85.0000000000000157:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:39.500 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.231531+0000", 2024-09-21T15:01:39.500 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 37.231492699999997, 2024-09-21T15:01:39.500 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0244773149999999, 2024-09-21T15:01:39.500 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:39.500 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:39.500 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:39.500 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:39.500 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:39.500 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140010 2024-09-21T15:01:39.500 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.500 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:39.501 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.501 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:39.501 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231531+0000", 2024-09-21T15:01:39.501 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:39.501 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.501 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.501 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:39.501 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231531+0000", 2024-09-21T15:01:39.501 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:39.501 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.501 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.501 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:39.501 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231530+0000", 2024-09-21T15:01:39.501 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:39.502 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.502 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.502 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:39.502 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231535+0000", 2024-09-21T15:01:39.502 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3970000000000002e-06 2024-09-21T15:01:39.502 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.502 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.502 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:39.502 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231544+0000", 2024-09-21T15:01:39.502 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5909999999999996e-06 2024-09-21T15:01:39.502 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.502 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.502 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:39.502 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231551+0000", 2024-09-21T15:01:39.503 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7129999999999999e-06 2024-09-21T15:01:39.503 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.503 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.503 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:39.503 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231632+0000", 2024-09-21T15:01:39.503 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1181999999999996e-05 2024-09-21T15:01:39.503 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.503 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.503 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:39.503 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231640+0000", 2024-09-21T15:01:39.503 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.551e-06 2024-09-21T15:01:39.503 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.503 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.503 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:39.503 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255884+0000", 2024-09-21T15:01:39.504 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.024244057 2024-09-21T15:01:39.504 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.504 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.504 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:39.504 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256008+0000", 2024-09-21T15:01:39.504 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012351699999999999 2024-09-21T15:01:39.504 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:39.504 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:39.504 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:39.504 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.504 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.504 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140043 3.e 3:74a37859:::rbd_data.113664780c85.0000000000000086:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:39.504 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.232623+0000", 2024-09-21T15:01:39.504 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 37.230400226, 2024-09-21T15:01:39.504 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.023518634, 2024-09-21T15:01:39.505 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:39.505 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:39.505 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:39.505 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:39.505 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:39.505 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140043 2024-09-21T15:01:39.505 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.505 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:39.505 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.505 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:39.505 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.232623+0000", 2024-09-21T15:01:39.505 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:39.505 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.505 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.506 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:39.506 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.232623+0000", 2024-09-21T15:01:39.506 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:39.506 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.506 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.506 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:39.506 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.232623+0000", 2024-09-21T15:01:39.506 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:39.506 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.506 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.506 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:39.506 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.232626+0000", 2024-09-21T15:01:39.506 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0019999999999998e-06 2024-09-21T15:01:39.507 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.507 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.507 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:39.507 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.232632+0000", 2024-09-21T15:01:39.507 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3260000000000001e-06 2024-09-21T15:01:39.507 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.507 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.507 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:39.507 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.232637+0000", 2024-09-21T15:01:39.507 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9899999999999997e-06 2024-09-21T15:01:39.507 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.507 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.507 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:39.507 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.232678+0000", 2024-09-21T15:01:39.507 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.0423999999999998e-05 2024-09-21T15:01:39.508 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.508 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.508 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:39.508 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.232686+0000", 2024-09-21T15:01:39.508 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1419999999999993e-06 2024-09-21T15:01:39.508 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.508 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.508 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:39.508 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256037+0000", 2024-09-21T15:01:39.508 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.023351736 2024-09-21T15:01:39.508 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.508 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.508 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:39.509 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256142+0000", 2024-09-21T15:01:39.509 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010443 2024-09-21T15:01:39.509 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:39.509 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:39.509 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:39.509 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.509 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.509 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140063 3.e 3:771cae23:::rbd_data.113664780c85.0000000000000112:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:39.509 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.233948+0000", 2024-09-21T15:01:39.509 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 37.229075158999997, 2024-09-21T15:01:39.509 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0223114710000001, 2024-09-21T15:01:39.509 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:39.509 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:39.509 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:39.510 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:39.510 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:39.510 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140063 2024-09-21T15:01:39.510 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.510 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:39.510 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.510 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:39.510 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233948+0000", 2024-09-21T15:01:39.510 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:39.510 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.510 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.510 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:39.510 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233948+0000", 2024-09-21T15:01:39.511 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:39.511 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.511 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.511 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:39.511 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233948+0000", 2024-09-21T15:01:39.511 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:39.511 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.511 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.511 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:39.511 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233951+0000", 2024-09-21T15:01:39.511 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3380000000000001e-06 2024-09-21T15:01:39.511 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.511 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.511 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:39.512 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233958+0000", 2024-09-21T15:01:39.512 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0990000000000001e-06 2024-09-21T15:01:39.512 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.512 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.512 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:39.512 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233964+0000", 2024-09-21T15:01:39.512 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.852e-06 2024-09-21T15:01:39.512 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.512 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.512 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:39.512 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.234056+0000", 2024-09-21T15:01:39.512 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2392000000000006e-05 2024-09-21T15:01:39.512 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.512 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.513 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:39.513 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.234066+0000", 2024-09-21T15:01:39.513 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1360000000000003e-06 2024-09-21T15:01:39.513 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.513 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.513 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:39.513 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256169+0000", 2024-09-21T15:01:39.513 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0221031949999999 2024-09-21T15:01:39.513 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.513 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.513 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:39.513 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256260+0000", 2024-09-21T15:01:39.513 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0939000000000005e-05 2024-09-21T15:01:39.513 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:39.513 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:39.514 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:39.514 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.514 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.514 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140086 3.e 3:758134b7:::rbd_data.113664780c85.00000000000001ad:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:39.514 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.235110+0000", 2024-09-21T15:01:39.514 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 37.227913460000003, 2024-09-21T15:01:39.514 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.021276217, 2024-09-21T15:01:39.514 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:39.514 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:39.514 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:39.514 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:39.514 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:39.514 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140086 2024-09-21T15:01:39.515 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.515 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:39.515 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.515 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:39.515 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.235110+0000", 2024-09-21T15:01:39.515 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:39.515 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.515 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.515 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:39.515 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.235110+0000", 2024-09-21T15:01:39.515 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:39.515 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.515 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.515 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:39.516 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.235109+0000", 2024-09-21T15:01:39.516 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:39.516 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.516 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.516 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:39.516 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.235114+0000", 2024-09-21T15:01:39.516 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1659999999999996e-06 2024-09-21T15:01:39.516 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.516 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.516 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:39.516 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.235122+0000", 2024-09-21T15:01:39.516 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5590000000000006e-06 2024-09-21T15:01:39.516 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.516 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.517 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:39.517 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.235128+0000", 2024-09-21T15:01:39.517 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8579999999999996e-06 2024-09-21T15:01:39.517 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.517 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.517 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:39.517 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.235169+0000", 2024-09-21T15:01:39.517 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1304000000000003e-05 2024-09-21T15:01:39.517 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.517 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.517 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:39.517 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.235178+0000", 2024-09-21T15:01:39.517 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8610000000000004e-06 2024-09-21T15:01:39.517 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.518 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.518 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:39.518 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256286+0000", 2024-09-21T15:01:39.518 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.021108162 2024-09-21T15:01:39.518 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.518 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.518 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:39.518 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256386+0000", 2024-09-21T15:01:39.518 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9871000000000007e-05 2024-09-21T15:01:39.518 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:39.518 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:39.518 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:39.518 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.518 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.519 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140168 3.e 3:76f4e81d:::rbd_data.113664780c85.000000000000006f:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:39.519 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.236992+0000", 2024-09-21T15:01:39.519 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 37.226031397, 2024-09-21T15:01:39.519 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.019513699, 2024-09-21T15:01:39.519 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:39.519 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:39.519 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:39.519 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:39.519 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:39.519 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140168 2024-09-21T15:01:39.519 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.519 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:39.519 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.520 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:39.520 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.236992+0000", 2024-09-21T15:01:39.520 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:39.520 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.520 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.520 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:39.520 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.236992+0000", 2024-09-21T15:01:39.520 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:39.520 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.520 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.520 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:39.520 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.236991+0000", 2024-09-21T15:01:39.520 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:39.520 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.520 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.521 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:39.521 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.236995+0000", 2024-09-21T15:01:39.521 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3100000000000001e-06 2024-09-21T15:01:39.521 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.521 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.521 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:39.521 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237002+0000", 2024-09-21T15:01:39.521 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9929999999999998e-06 2024-09-21T15:01:39.521 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.521 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.521 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:39.521 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237008+0000", 2024-09-21T15:01:39.521 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7559999999999996e-06 2024-09-21T15:01:39.521 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.522 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.522 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:39.522 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237050+0000", 2024-09-21T15:01:39.522 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2852999999999998e-05 2024-09-21T15:01:39.522 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.522 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.522 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:39.522 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237060+0000", 2024-09-21T15:01:39.522 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2399999999999996e-06 2024-09-21T15:01:39.522 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.522 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.522 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:39.522 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256413+0000", 2024-09-21T15:01:39.523 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0193528999999999 2024-09-21T15:01:39.523 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.523 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.523 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:39.523 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256506+0000", 2024-09-21T15:01:39.523 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.3154999999999998e-05 2024-09-21T15:01:39.523 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:39.523 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:39.523 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:39.523 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.523 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.523 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140186 3.e 3:75504345:::rbd_data.113664780c85.00000000000000ed:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:39.523 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.237050+0000", 2024-09-21T15:01:39.523 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 37.225973426000003, 2024-09-21T15:01:39.524 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0195720150000001, 2024-09-21T15:01:39.524 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:39.524 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:39.524 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:39.524 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:39.524 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:39.524 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140186 2024-09-21T15:01:39.524 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.524 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:39.524 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.524 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:39.524 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237050+0000", 2024-09-21T15:01:39.524 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:39.524 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.525 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.525 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:39.525 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237050+0000", 2024-09-21T15:01:39.525 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:39.525 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.525 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.525 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:39.525 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237049+0000", 2024-09-21T15:01:39.525 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:39.525 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.525 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.525 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:39.525 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237052+0000", 2024-09-21T15:01:39.525 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.016e-06 2024-09-21T15:01:39.525 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.526 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.526 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:39.526 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237070+0000", 2024-09-21T15:01:39.526 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7266999999999999e-05 2024-09-21T15:01:39.526 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.526 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.526 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:39.526 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237075+0000", 2024-09-21T15:01:39.526 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6910000000000002e-06 2024-09-21T15:01:39.526 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.526 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.526 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:39.526 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237232+0000", 2024-09-21T15:01:39.526 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015683599999999999 2024-09-21T15:01:39.527 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.527 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.527 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:39.527 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237237+0000", 2024-09-21T15:01:39.527 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6439999999999999e-06 2024-09-21T15:01:39.527 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.527 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.527 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:39.527 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256534+0000", 2024-09-21T15:01:39.527 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.019296736 2024-09-21T15:01:39.527 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.527 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.527 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:39.528 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256622+0000", 2024-09-21T15:01:39.528 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8386000000000004e-05 2024-09-21T15:01:39.528 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:39.528 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:39.528 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:39.528 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.528 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.528 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140197 3.e 3:71ef5b14:::rbd_data.113664780c85.0000000000000140:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:39.528 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.237251+0000", 2024-09-21T15:01:39.528 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 37.225772397999997, 2024-09-21T15:01:39.528 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.01968386, 2024-09-21T15:01:39.528 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:39.528 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:39.528 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:39.529 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:39.529 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:39.529 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140197 2024-09-21T15:01:39.529 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.529 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:39.529 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.529 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:39.529 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237251+0000", 2024-09-21T15:01:39.529 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:39.529 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.529 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.529 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:39.529 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237251+0000", 2024-09-21T15:01:39.529 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:39.530 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.530 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.530 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:39.530 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237250+0000", 2024-09-21T15:01:39.530 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:39.530 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.530 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.530 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:39.530 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237254+0000", 2024-09-21T15:01:39.530 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.0180000000000003e-06 2024-09-21T15:01:39.530 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.530 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.530 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:39.530 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237262+0000", 2024-09-21T15:01:39.531 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0669999999999992e-06 2024-09-21T15:01:39.531 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.531 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.531 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:39.531 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237269+0000", 2024-09-21T15:01:39.531 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6000000000000003e-06 2024-09-21T15:01:39.531 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.531 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.531 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:39.531 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237319+0000", 2024-09-21T15:01:39.531 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9932000000000002e-05 2024-09-21T15:01:39.531 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.531 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.531 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:39.532 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237326+0000", 2024-09-21T15:01:39.532 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5360000000000004e-06 2024-09-21T15:01:39.532 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.532 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.532 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:39.532 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256657+0000", 2024-09-21T15:01:39.532 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.019330756 2024-09-21T15:01:39.532 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.532 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.532 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:39.532 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256935+0000", 2024-09-21T15:01:39.532 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00027762099999999998 2024-09-21T15:01:39.532 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:39.532 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:39.532 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:39.533 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.533 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.533 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140201 3.e 3:732ea575:::rbd_data.113664780c85.000000000000015c:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:39.533 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.237281+0000", 2024-09-21T15:01:39.533 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 37.22574256, 2024-09-21T15:01:39.533 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0198284529999999, 2024-09-21T15:01:39.533 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:39.533 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:39.533 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:39.533 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:39.533 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:39.533 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140201 2024-09-21T15:01:39.533 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.533 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:39.533 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.534 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:39.534 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237281+0000", 2024-09-21T15:01:39.534 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:39.534 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.534 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.534 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:39.534 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237281+0000", 2024-09-21T15:01:39.534 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:39.534 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.534 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.534 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:39.534 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237280+0000", 2024-09-21T15:01:39.534 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:39.535 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.535 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.535 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:39.535 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237284+0000", 2024-09-21T15:01:39.535 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7400000000000002e-06 2024-09-21T15:01:39.535 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.535 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.535 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:39.535 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237292+0000", 2024-09-21T15:01:39.535 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8939999999999993e-06 2024-09-21T15:01:39.535 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.535 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.535 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:39.535 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237298+0000", 2024-09-21T15:01:39.536 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8000000000000004e-06 2024-09-21T15:01:39.536 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.536 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.536 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:39.536 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237521+0000", 2024-09-21T15:01:39.536 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00022326 2024-09-21T15:01:39.536 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.536 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.536 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:39.536 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237529+0000", 2024-09-21T15:01:39.536 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6170000000000001e-06 2024-09-21T15:01:39.536 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.536 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.537 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:39.537 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256968+0000", 2024-09-21T15:01:39.537 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.019439113 2024-09-21T15:01:39.537 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.537 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.537 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:39.537 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.257109+0000", 2024-09-21T15:01:39.537 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014148399999999999 2024-09-21T15:01:39.537 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:39.537 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:39.537 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:39.537 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.537 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.537 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140206 3.e 3:76d38029:::rbd_data.113664780c85.0000000000000179:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:39.538 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.237520+0000", 2024-09-21T15:01:39.538 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 37.225503445999998, 2024-09-21T15:01:39.538 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0210658699999999, 2024-09-21T15:01:39.538 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:39.538 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:39.538 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:39.538 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:39.538 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:39.538 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140206 2024-09-21T15:01:39.538 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.538 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:39.538 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.538 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:39.538 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237520+0000", 2024-09-21T15:01:39.539 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:39.539 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.539 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.539 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:39.539 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237520+0000", 2024-09-21T15:01:39.539 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:39.539 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.539 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.539 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:39.539 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237519+0000", 2024-09-21T15:01:39.539 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:39.539 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.539 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.539 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:39.540 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237523+0000", 2024-09-21T15:01:39.540 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.9009999999999996e-06 2024-09-21T15:01:39.540 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.540 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.540 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:39.540 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237531+0000", 2024-09-21T15:01:39.540 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7069999999999992e-06 2024-09-21T15:01:39.540 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.540 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.540 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:39.540 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237537+0000", 2024-09-21T15:01:39.540 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4559999999999997e-06 2024-09-21T15:01:39.540 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.540 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.541 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:39.541 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237583+0000", 2024-09-21T15:01:39.541 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5532999999999999e-05 2024-09-21T15:01:39.541 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.541 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.541 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:39.541 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237592+0000", 2024-09-21T15:01:39.541 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6009999999999995e-06 2024-09-21T15:01:39.541 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.541 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.541 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:39.541 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.257157+0000", 2024-09-21T15:01:39.541 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0195657279999999 2024-09-21T15:01:39.541 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.542 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.542 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:39.542 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.258586+0000", 2024-09-21T15:01:39.542 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00142847 2024-09-21T15:01:39.542 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:39.542 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:39.542 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:39.542 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.542 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.542 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140338 3.e 3:71dba096:::rbd_data.113664780c85.000000000000018b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:39.542 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.243856+0000", 2024-09-21T15:01:39.542 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 37.219167613000003, 2024-09-21T15:01:39.542 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0138226889999999, 2024-09-21T15:01:39.543 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:39.543 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:39.543 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:39.543 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:39.543 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:39.543 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140338 2024-09-21T15:01:39.543 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.543 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:39.543 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.543 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:39.543 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243856+0000", 2024-09-21T15:01:39.543 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:39.543 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.543 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.544 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:39.544 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243856+0000", 2024-09-21T15:01:39.544 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:39.544 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.544 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.544 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:39.544 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243854+0000", 2024-09-21T15:01:39.544 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:39.544 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.544 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.544 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:39.544 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243862+0000", 2024-09-21T15:01:39.544 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2570000000000001e-06 2024-09-21T15:01:39.545 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.545 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.545 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:39.545 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243877+0000", 2024-09-21T15:01:39.545 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.571e-05 2024-09-21T15:01:39.545 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.545 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.545 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:39.545 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243884+0000", 2024-09-21T15:01:39.545 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.1269999999999997e-06 2024-09-21T15:01:39.545 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.545 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.545 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:39.546 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243932+0000", 2024-09-21T15:01:39.546 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7655999999999999e-05 2024-09-21T15:01:39.546 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.546 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.546 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:39.546 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243942+0000", 2024-09-21T15:01:39.546 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6679999999999996e-06 2024-09-21T15:01:39.546 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.546 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.546 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:39.546 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.257568+0000", 2024-09-21T15:01:39.546 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0136267670000001 2024-09-21T15:01:39.546 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.546 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.547 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:39.547 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.257678+0000", 2024-09-21T15:01:39.547 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000110011 2024-09-21T15:01:39.547 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:39.547 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:39.547 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:39.547 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.547 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.547 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140345 3.e 3:76ddeb1d:::rbd_data.113664780c85.0000000000000004:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:39.547 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.243927+0000", 2024-09-21T15:01:39.547 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 37.219096028999999, 2024-09-21T15:01:39.547 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0140328439999999, 2024-09-21T15:01:39.547 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:39.547 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:39.548 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:39.548 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:39.548 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:39.548 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140345 2024-09-21T15:01:39.548 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.548 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:39.548 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.548 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:39.548 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243927+0000", 2024-09-21T15:01:39.548 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:39.548 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.548 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.548 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:39.548 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243927+0000", 2024-09-21T15:01:39.548 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:39.549 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.549 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.549 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:39.549 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243927+0000", 2024-09-21T15:01:39.549 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:39.549 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.549 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.549 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:39.549 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243930+0000", 2024-09-21T15:01:39.549 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1319999999999998e-06 2024-09-21T15:01:39.549 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.549 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.549 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:39.549 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243937+0000", 2024-09-21T15:01:39.550 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4330000000000001e-06 2024-09-21T15:01:39.550 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.550 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.550 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:39.550 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243942+0000", 2024-09-21T15:01:39.550 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5090000000000001e-06 2024-09-21T15:01:39.550 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.550 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.550 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:39.550 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243964+0000", 2024-09-21T15:01:39.550 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1838999999999999e-05 2024-09-21T15:01:39.550 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.551 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.551 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:39.551 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243971+0000", 2024-09-21T15:01:39.551 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.131e-06 2024-09-21T15:01:39.551 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.551 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.551 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:39.551 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.257704+0000", 2024-09-21T15:01:39.551 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0137329310000001 2024-09-21T15:01:39.551 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.551 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.551 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:39.551 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.257805+0000", 2024-09-21T15:01:39.551 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000101121 2024-09-21T15:01:39.551 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.552 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.552 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:39.552 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.257960+0000", 2024-09-21T15:01:39.552 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015518500000000001 2024-09-21T15:01:39.552 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:39.552 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:39.552 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:39.552 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.552 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.552 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140350 3.e 3:76b9b799:::rbd_data.113664780c85.0000000000000021:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:39.552 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.243989+0000", 2024-09-21T15:01:39.552 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 37.219034278000002, 2024-09-21T15:01:39.552 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0142022340000001, 2024-09-21T15:01:39.552 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:39.552 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:39.553 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:39.553 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:39.553 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:39.553 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140350 2024-09-21T15:01:39.553 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.553 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:39.553 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.553 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:39.553 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243989+0000", 2024-09-21T15:01:39.553 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:39.553 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.553 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.553 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:39.553 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243989+0000", 2024-09-21T15:01:39.554 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:39.554 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.554 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.554 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:39.554 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243989+0000", 2024-09-21T15:01:39.554 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:39.554 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.554 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.554 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:39.554 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243992+0000", 2024-09-21T15:01:39.554 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.236e-06 2024-09-21T15:01:39.554 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.554 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.555 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:39.555 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243998+0000", 2024-09-21T15:01:39.555 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2500000000000003e-06 2024-09-21T15:01:39.555 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.555 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.555 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:39.555 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.244003+0000", 2024-09-21T15:01:39.555 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7740000000000004e-06 2024-09-21T15:01:39.555 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.555 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.555 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:39.555 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.244047+0000", 2024-09-21T15:01:39.555 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3665999999999998e-05 2024-09-21T15:01:39.555 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.556 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.556 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:39.556 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.244055+0000", 2024-09-21T15:01:39.557 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0639999999999994e-06 2024-09-21T15:01:39.557 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.557 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.557 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:39.557 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.257985+0000", 2024-09-21T15:01:39.558 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0139306079999999 2024-09-21T15:01:39.558 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.558 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.558 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:39.558 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.258084+0000", 2024-09-21T15:01:39.558 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8610000000000006e-05 2024-09-21T15:01:39.558 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.558 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.558 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:39.558 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.258191+0000", 2024-09-21T15:01:39.558 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000107518 2024-09-21T15:01:39.558 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:39.558 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:39.558 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:39.559 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.559 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.559 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140394 3.e 3:7756a55d:::rbd_data.113664780c85.0000000000000154:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:39.559 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.245038+0000", 2024-09-21T15:01:39.559 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 37.217985061, 2024-09-21T15:01:39.559 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.013225182, 2024-09-21T15:01:39.559 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:39.559 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:39.559 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:39.559 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:39.559 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:39.559 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140394 2024-09-21T15:01:39.559 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.559 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:39.560 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.560 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:39.560 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.245038+0000", 2024-09-21T15:01:39.560 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:39.560 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.560 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.560 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:39.560 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.245038+0000", 2024-09-21T15:01:39.560 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:39.560 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.560 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.560 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:39.560 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.245037+0000", 2024-09-21T15:01:39.560 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:39.561 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.561 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.561 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:39.561 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.245044+0000", 2024-09-21T15:01:39.561 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3239999999999999e-06 2024-09-21T15:01:39.561 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.561 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.561 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:39.561 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.245061+0000", 2024-09-21T15:01:39.561 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7166000000000001e-05 2024-09-21T15:01:39.561 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.561 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.561 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:39.561 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.245069+0000", 2024-09-21T15:01:39.562 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0779999999999996e-06 2024-09-21T15:01:39.562 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.562 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.562 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:39.562 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.245118+0000", 2024-09-21T15:01:39.562 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8887000000000001e-05 2024-09-21T15:01:39.562 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.562 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.562 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:39.562 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.245126+0000", 2024-09-21T15:01:39.562 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0409999999999998e-06 2024-09-21T15:01:39.562 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.562 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.562 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:39.563 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.258206+0000", 2024-09-21T15:01:39.563 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.013080451 2024-09-21T15:01:39.563 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.563 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.563 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:39.563 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.258264+0000", 2024-09-21T15:01:39.563 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7294999999999999e-05 2024-09-21T15:01:39.563 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:39.563 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:39.563 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:39.563 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:39.563 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:39.563 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:39.564 INFO:tasks.ceph.osd.4:Stopped 2024-09-21T15:01:39.564 INFO:tasks.thrashosds.thrasher:Testing ceph-objectstore-tool on down osd.4 2024-09-21T15:01:39.565 INFO:tasks.thrashosds.thrasher:remote for osd 4 is ubuntu@smithi046.front.sepia.ceph.com 2024-09-21T15:01:39.565 DEBUG:teuthology.orchestra.run.smithi046:> type ceph-objectstore-tool 2024-09-21T15:01:39.566 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T15:01:39.566 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:01:39.560+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:01:39.572 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-21T15:01:39.572 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-4 --journal-path /var/lib/ceph/osd/ceph-4/journal --op list-pgs 2024-09-21T15:01:39.666 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:01:39.660+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:01:39.681 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:39.682 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:01:39.682 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T15:01:39.682 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:39.692 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T15:01:39.767 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:01:39.766+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:01:39.798 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:39.798 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:01:39.798 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T15:01:39.798 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T15:01:39.798 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:39.808 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T15:01:39.868 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:01:39.866+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:01:39.922 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:39.922 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T15:01:39.922 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T15:01:39.922 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T15:01:39.922 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.922 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T15:01:39.922 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.205754+0000", 2024-09-21T15:01:39.922 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 216.714010879, 2024-09-21T15:01:39.923 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.41117094199999998, 2024-09-21T15:01:39.923 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:39.923 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:39.923 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:39.923 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T15:01:39.923 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T15:01:39.923 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1 2024-09-21T15:01:39.923 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.923 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:39.923 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.923 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:39.923 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205754+0000", 2024-09-21T15:01:39.923 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:39.924 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.924 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.924 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:39.924 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205754+0000", 2024-09-21T15:01:39.924 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:39.924 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.924 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.924 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:39.924 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205752+0000", 2024-09-21T15:01:39.924 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T15:01:39.924 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.924 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.925 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:39.925 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205765+0000", 2024-09-21T15:01:39.925 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3230999999999999e-05 2024-09-21T15:01:39.925 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.925 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.925 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:39.925 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205782+0000", 2024-09-21T15:01:39.925 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6634e-05 2024-09-21T15:01:39.925 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.925 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.925 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:39.925 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205793+0000", 2024-09-21T15:01:39.925 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1297e-05 2024-09-21T15:01:39.926 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.926 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.926 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:39.926 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205867+0000", 2024-09-21T15:01:39.926 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4269000000000003e-05 2024-09-21T15:01:39.926 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.926 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.926 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:39.926 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205877+0000", 2024-09-21T15:01:39.926 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2690000000000005e-06 2024-09-21T15:01:39.926 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.926 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.926 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:39.927 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.613700+0000", 2024-09-21T15:01:39.927 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.407823455 2024-09-21T15:01:39.927 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.927 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.927 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:39.927 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.614203+0000", 2024-09-21T15:01:39.927 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00050311500000000005 2024-09-21T15:01:39.927 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.927 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.927 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T15:01:39.927 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.614750+0000", 2024-09-21T15:01:39.927 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00054738699999999998 2024-09-21T15:01:39.927 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.928 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.928 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:01:39.928 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616367+0000", 2024-09-21T15:01:39.928 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001616634 2024-09-21T15:01:39.928 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.928 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.928 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:01:39.928 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616574+0000", 2024-09-21T15:01:39.928 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00020668499999999999 2024-09-21T15:01:39.928 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.928 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.928 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:01:39.928 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616802+0000", 2024-09-21T15:01:39.929 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00022817900000000001 2024-09-21T15:01:39.929 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.929 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.929 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:39.929 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616925+0000", 2024-09-21T15:01:39.929 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012309699999999999 2024-09-21T15:01:39.929 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:39.929 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:39.929 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:39.929 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.929 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.929 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:61214 3.7 3:e28aec10:::rbd_header.1136862f872f:head [watch reconnect cookie 139864671932080 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-21T15:01:39.929 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:18.147172+0000", 2024-09-21T15:01:39.929 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 141.77259323600001, 2024-09-21T15:01:39.930 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.019264956, 2024-09-21T15:01:39.930 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:39.930 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:39.930 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:39.930 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:39.930 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:39.930 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 61214 2024-09-21T15:01:39.930 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.930 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:39.930 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.930 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:39.930 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147172+0000", 2024-09-21T15:01:39.930 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:39.930 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.930 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.931 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:39.931 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147172+0000", 2024-09-21T15:01:39.931 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:39.931 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.931 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.931 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:39.931 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147168+0000", 2024-09-21T15:01:39.931 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999962 2024-09-21T15:01:39.931 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.931 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.931 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:39.931 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147188+0000", 2024-09-21T15:01:39.931 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9717000000000001e-05 2024-09-21T15:01:39.931 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.932 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.932 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:39.932 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147239+0000", 2024-09-21T15:01:39.932 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1493999999999999e-05 2024-09-21T15:01:39.932 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.932 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.932 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:39.932 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147265+0000", 2024-09-21T15:01:39.932 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5746999999999999e-05 2024-09-21T15:01:39.932 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.932 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.932 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:39.932 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147372+0000", 2024-09-21T15:01:39.932 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010681200000000001 2024-09-21T15:01:39.932 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.932 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.933 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op must wait for map", 2024-09-21T15:01:39.933 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147380+0000", 2024-09-21T15:01:39.933 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7919999999999993e-06 2024-09-21T15:01:39.933 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.933 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.933 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:39.933 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.154025+0000", 2024-09-21T15:01:39.933 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0066454909999999999 2024-09-21T15:01:39.933 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.933 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.933 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:39.933 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.154041+0000", 2024-09-21T15:01:39.933 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5894000000000001e-05 2024-09-21T15:01:39.933 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.933 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.933 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:39.934 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.164345+0000", 2024-09-21T15:01:39.934 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0103040459999999 2024-09-21T15:01:39.934 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.934 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.934 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:39.934 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.164590+0000", 2024-09-21T15:01:39.934 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00024492699999999998 2024-09-21T15:01:39.934 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.934 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.934 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:39.934 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.166437+0000", 2024-09-21T15:01:39.934 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0018466330000000001 2024-09-21T15:01:39.934 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:39.934 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:39.935 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:39.935 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.935 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.935 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:61215 3.7 3:e28aec10:::rbd_header.1136862f872f:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-21T15:01:39.935 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:18.152008+0000", 2024-09-21T15:01:39.935 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 141.767756784, 2024-09-21T15:01:39.935 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.015644958, 2024-09-21T15:01:39.935 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:39.935 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:39.935 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:39.935 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:39.935 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:39.935 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 61215 2024-09-21T15:01:39.936 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.936 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:39.936 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.936 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:39.936 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152008+0000", 2024-09-21T15:01:39.936 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:39.936 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.936 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.936 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:39.936 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152008+0000", 2024-09-21T15:01:39.936 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:39.937 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.937 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.937 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:39.937 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152005+0000", 2024-09-21T15:01:39.937 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-21T15:01:39.937 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.937 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.937 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:39.937 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152025+0000", 2024-09-21T15:01:39.937 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9306000000000001e-05 2024-09-21T15:01:39.937 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.937 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.937 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:39.937 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152054+0000", 2024-09-21T15:01:39.938 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9189000000000001e-05 2024-09-21T15:01:39.938 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.938 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.938 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:39.938 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152078+0000", 2024-09-21T15:01:39.938 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4317000000000001e-05 2024-09-21T15:01:39.938 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.938 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.938 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:39.938 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152243+0000", 2024-09-21T15:01:39.938 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000164802 2024-09-21T15:01:39.938 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.938 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.939 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting_for_map not empty", 2024-09-21T15:01:39.939 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152251+0000", 2024-09-21T15:01:39.939 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0600000000000008e-06 2024-09-21T15:01:39.939 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.939 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.939 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:39.939 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.154741+0000", 2024-09-21T15:01:39.939 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0024900069999999998 2024-09-21T15:01:39.939 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.939 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.939 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:39.939 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.154753+0000", 2024-09-21T15:01:39.939 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1588e-05 2024-09-21T15:01:39.939 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.940 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.940 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:39.940 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.165380+0000", 2024-09-21T15:01:39.940 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0106272249999999 2024-09-21T15:01:39.940 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.940 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.940 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:39.940 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.165517+0000", 2024-09-21T15:01:39.940 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013737999999999999 2024-09-21T15:01:39.940 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.940 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.940 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 6", 2024-09-21T15:01:39.940 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.165938+0000", 2024-09-21T15:01:39.940 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00042082399999999998 2024-09-21T15:01:39.941 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.941 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.941 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:01:39.941 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.166795+0000", 2024-09-21T15:01:39.941 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00085674899999999997 2024-09-21T15:01:39.941 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.941 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.941 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:01:39.941 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.167598+0000", 2024-09-21T15:01:39.941 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00080335599999999997 2024-09-21T15:01:39.941 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.941 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.942 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:01:39.942 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.167632+0000", 2024-09-21T15:01:39.942 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4122000000000003e-05 2024-09-21T15:01:39.942 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.942 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.942 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:39.942 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.167653+0000", 2024-09-21T15:01:39.942 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0862000000000001e-05 2024-09-21T15:01:39.942 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:39.942 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:39.942 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:39.942 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.943 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.943 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:97678 3.d e122/24 3:b2f34c6d:::rbd_data.1136bfc79a70.0000000000000006:head v 122'234, mlcod=120'232)", 2024-09-21T15:01:39.943 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.321655+0000", 2024-09-21T15:01:39.943 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 94.598109867000005, 2024-09-21T15:01:39.943 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024509388999999999, 2024-09-21T15:01:39.943 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:39.943 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:39.943 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:39.943 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.943 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:39.943 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.321655+0000", 2024-09-21T15:01:39.943 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:39.943 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.943 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.944 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:39.944 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.321655+0000", 2024-09-21T15:01:39.944 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:39.944 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.944 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.944 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:39.944 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.321655+0000", 2024-09-21T15:01:39.944 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:39.944 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.944 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.944 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:39.944 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.329453+0000", 2024-09-21T15:01:39.944 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0077980560000000003 2024-09-21T15:01:39.944 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.945 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.945 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:39.945 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.329482+0000", 2024-09-21T15:01:39.945 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9153999999999999e-05 2024-09-21T15:01:39.945 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.945 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.945 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:39.945 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.329493+0000", 2024-09-21T15:01:39.945 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1103e-05 2024-09-21T15:01:39.945 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.945 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.945 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:39.945 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.329555+0000", 2024-09-21T15:01:39.946 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2345e-05 2024-09-21T15:01:39.946 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.946 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.946 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:39.946 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.329589+0000", 2024-09-21T15:01:39.946 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3908999999999997e-05 2024-09-21T15:01:39.946 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.946 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.946 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:01:39.946 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.346083+0000", 2024-09-21T15:01:39.946 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016493383 2024-09-21T15:01:39.946 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.946 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.947 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:39.947 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.346165+0000", 2024-09-21T15:01:39.947 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1796000000000001e-05 2024-09-21T15:01:39.947 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:39.947 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:39.947 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:39.947 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.947 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.947 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:97679 3.f e122/24 3:f2a362b4:::rbd_data.1136bfc79a70.0000000000000007:head v 122'331, mlcod=120'329)", 2024-09-21T15:01:39.947 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.337145+0000", 2024-09-21T15:01:39.947 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 94.582619858000001, 2024-09-21T15:01:39.947 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021912411, 2024-09-21T15:01:39.947 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:39.947 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:39.948 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:39.948 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.948 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:39.948 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.337145+0000", 2024-09-21T15:01:39.948 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:39.948 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.948 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.948 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:39.948 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.337145+0000", 2024-09-21T15:01:39.949 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:39.949 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.949 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.949 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:39.949 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.337145+0000", 2024-09-21T15:01:39.949 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:39.949 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.949 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.949 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:39.949 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.343899+0000", 2024-09-21T15:01:39.949 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0067541420000000003 2024-09-21T15:01:39.949 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.949 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.949 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:39.950 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.343921+0000", 2024-09-21T15:01:39.950 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2255e-05 2024-09-21T15:01:39.950 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.950 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.950 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:39.950 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.343933+0000", 2024-09-21T15:01:39.950 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1528e-05 2024-09-21T15:01:39.950 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.950 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.950 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:39.950 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.344001+0000", 2024-09-21T15:01:39.951 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8269999999999995e-05 2024-09-21T15:01:39.951 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.951 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.951 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:39.951 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.344029+0000", 2024-09-21T15:01:39.951 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8254000000000001e-05 2024-09-21T15:01:39.951 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.951 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.951 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:01:39.951 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.358901+0000", 2024-09-21T15:01:39.951 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0148719 2024-09-21T15:01:39.951 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.951 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.952 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:39.952 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.359058+0000", 2024-09-21T15:01:39.952 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000156446 2024-09-21T15:01:39.952 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:39.952 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:39.952 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:39.952 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.952 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.952 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:97676 3.1f e122/24 3:ffaa836a:::rbd_data.1136bfc79a70.0000000000000004:head v 122'312, mlcod=120'310)", 2024-09-21T15:01:39.952 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.348683+0000", 2024-09-21T15:01:39.952 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 94.571081692999996, 2024-09-21T15:01:39.952 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.033768802000000001, 2024-09-21T15:01:39.952 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:39.952 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:39.953 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:39.953 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.953 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:39.953 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.348683+0000", 2024-09-21T15:01:39.953 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:39.953 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.953 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.953 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:39.953 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.348683+0000", 2024-09-21T15:01:39.953 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:39.953 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.953 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.953 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:39.953 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.348683+0000", 2024-09-21T15:01:39.953 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:39.954 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.954 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.954 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:39.954 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.366937+0000", 2024-09-21T15:01:39.954 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018254415 2024-09-21T15:01:39.954 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.954 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.954 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:39.954 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.366959+0000", 2024-09-21T15:01:39.954 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1634e-05 2024-09-21T15:01:39.954 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.954 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.954 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:39.954 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.366971+0000", 2024-09-21T15:01:39.955 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2305e-05 2024-09-21T15:01:39.955 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.955 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.955 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:39.955 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.367034+0000", 2024-09-21T15:01:39.955 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3211000000000002e-05 2024-09-21T15:01:39.955 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.955 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.955 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:39.955 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.367056+0000", 2024-09-21T15:01:39.955 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2098000000000001e-05 2024-09-21T15:01:39.955 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.955 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.955 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:01:39.955 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.382400+0000", 2024-09-21T15:01:39.956 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.015343867000000001 2024-09-21T15:01:39.956 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.956 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.956 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:39.956 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.382452+0000", 2024-09-21T15:01:39.956 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1992e-05 2024-09-21T15:01:39.956 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:39.956 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:39.956 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:39.956 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.956 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.956 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:97734 3.19 e122/94 3:9fcf754f:::rbd_data.1136bfc79a70.0000000000000009:head v 122'314, mlcod=120'312)", 2024-09-21T15:01:39.956 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.378547+0000", 2024-09-21T15:01:39.956 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 94.541217715000002, 2024-09-21T15:01:39.957 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.063380675999999997, 2024-09-21T15:01:39.957 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:39.957 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:39.957 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:39.957 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.957 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:39.957 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.378547+0000", 2024-09-21T15:01:39.957 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:39.957 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.957 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.957 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:39.957 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.378547+0000", 2024-09-21T15:01:39.957 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:39.957 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.957 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.957 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:39.958 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.378547+0000", 2024-09-21T15:01:39.958 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:01:39.958 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.958 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.958 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:39.958 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.425118+0000", 2024-09-21T15:01:39.958 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046571166999999997 2024-09-21T15:01:39.958 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.958 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.958 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:39.958 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.425150+0000", 2024-09-21T15:01:39.958 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2150999999999998e-05 2024-09-21T15:01:39.958 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.959 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.959 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:39.959 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.425164+0000", 2024-09-21T15:01:39.959 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3302000000000001e-05 2024-09-21T15:01:39.959 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.959 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.959 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:39.959 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.425231+0000", 2024-09-21T15:01:39.959 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7106000000000004e-05 2024-09-21T15:01:39.959 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.959 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.959 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:39.959 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.425259+0000", 2024-09-21T15:01:39.959 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8087000000000001e-05 2024-09-21T15:01:39.960 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.960 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.960 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:01:39.960 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.441890+0000", 2024-09-21T15:01:39.960 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016631328000000001 2024-09-21T15:01:39.960 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.960 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.960 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:39.960 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.441928+0000", 2024-09-21T15:01:39.960 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7701999999999999e-05 2024-09-21T15:01:39.960 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:39.960 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:39.960 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:39.960 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.960 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.961 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:100187 3.10 3:08941a97:::rbd_data.1136ab4d6ff1.000000000000000e:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-21T15:01:39.961 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:10.869845+0000", 2024-09-21T15:01:39.961 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 89.049919818999996, 2024-09-21T15:01:39.961 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029345237, 2024-09-21T15:01:39.961 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:39.961 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:39.961 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:39.961 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:39.961 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:39.961 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 100187 2024-09-21T15:01:39.961 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.961 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:39.961 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.961 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:39.962 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869845+0000", 2024-09-21T15:01:39.962 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:39.962 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.962 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.962 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:39.962 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869845+0000", 2024-09-21T15:01:39.962 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:39.962 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.962 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.962 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:39.962 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869844+0000", 2024-09-21T15:01:39.962 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:39.962 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.962 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.962 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:39.963 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869853+0000", 2024-09-21T15:01:39.963 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5719999999999992e-06 2024-09-21T15:01:39.963 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.963 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.963 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:39.963 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869874+0000", 2024-09-21T15:01:39.963 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0483e-05 2024-09-21T15:01:39.963 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.963 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.963 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:39.963 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869883+0000", 2024-09-21T15:01:39.963 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6080000000000002e-06 2024-09-21T15:01:39.963 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.963 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.963 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:39.964 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869991+0000", 2024-09-21T15:01:39.964 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000108013 2024-09-21T15:01:39.964 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.964 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.964 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:39.964 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.870258+0000", 2024-09-21T15:01:39.964 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00026629099999999999 2024-09-21T15:01:39.964 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.964 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.964 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T15:01:39.964 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.878656+0000", 2024-09-21T15:01:39.964 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0083989000000000008 2024-09-21T15:01:39.964 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.964 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.965 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:01:39.965 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.899081+0000", 2024-09-21T15:01:39.965 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.020424437 2024-09-21T15:01:39.965 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.965 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.965 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:01:39.965 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.899125+0000", 2024-09-21T15:01:39.965 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4440999999999997e-05 2024-09-21T15:01:39.965 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.965 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.965 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:01:39.965 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.899160+0000", 2024-09-21T15:01:39.965 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4518999999999999e-05 2024-09-21T15:01:39.966 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.966 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.966 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:39.966 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.899190+0000", 2024-09-21T15:01:39.966 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0574999999999998e-05 2024-09-21T15:01:39.966 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:39.966 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:39.966 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:39.966 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.966 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.966 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:100263 3.10 3:0ece142e:::rbd_data.1136ab4d6ff1.0000000000000008:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-21T15:01:39.966 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:11.052257+0000", 2024-09-21T15:01:39.966 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 88.867507760999999, 2024-09-21T15:01:39.967 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042678025000000001, 2024-09-21T15:01:39.967 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:39.967 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:39.967 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:39.967 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:39.967 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:39.967 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 100263 2024-09-21T15:01:39.967 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.967 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:39.967 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.967 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:39.967 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052257+0000", 2024-09-21T15:01:39.968 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:39.968 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.968 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.968 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:39.968 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052257+0000", 2024-09-21T15:01:39.968 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:39.968 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.968 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.968 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:39.968 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052256+0000", 2024-09-21T15:01:39.968 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:39.968 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.968 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.968 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:39.969 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052263+0000", 2024-09-21T15:01:39.969 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4799999999999998e-06 2024-09-21T15:01:39.969 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.969 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.969 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:39.969 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052275+0000", 2024-09-21T15:01:39.969 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2386e-05 2024-09-21T15:01:39.969 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.969 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.969 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:39.969 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052281+0000", 2024-09-21T15:01:39.969 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6699999999999999e-06 2024-09-21T15:01:39.969 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.969 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.969 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:39.970 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052350+0000", 2024-09-21T15:01:39.970 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9099999999999999e-05 2024-09-21T15:01:39.970 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.970 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.970 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:39.970 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052456+0000", 2024-09-21T15:01:39.970 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000106669 2024-09-21T15:01:39.970 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.970 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.970 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T15:01:39.970 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.059694+0000", 2024-09-21T15:01:39.970 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.007237566 2024-09-21T15:01:39.970 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.970 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.970 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:01:39.971 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.094721+0000", 2024-09-21T15:01:39.971 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035026977000000001 2024-09-21T15:01:39.971 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.971 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.971 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:01:39.971 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.094746+0000", 2024-09-21T15:01:39.971 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5329000000000001e-05 2024-09-21T15:01:39.971 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.971 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.971 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:01:39.971 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.094850+0000", 2024-09-21T15:01:39.971 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000103452 2024-09-21T15:01:39.971 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.972 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.972 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:39.972 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.094935+0000", 2024-09-21T15:01:39.972 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5550000000000006e-05 2024-09-21T15:01:39.972 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:39.972 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:39.972 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:39.972 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.972 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.972 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:134708 3.a 3:51eaadc0:::rbd_data.11365ed419e3.0000000000000007:67 [read 221184~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:39.972 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.938945+0000", 2024-09-21T15:01:39.972 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 45.980820483000002, 2024-09-21T15:01:39.972 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021073784000000002, 2024-09-21T15:01:39.972 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:39.972 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:39.972 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:39.973 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:39.973 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:39.973 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 134708 2024-09-21T15:01:39.973 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.973 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:39.973 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.973 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:39.973 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938945+0000", 2024-09-21T15:01:39.973 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:39.973 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.973 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.973 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:39.973 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938945+0000", 2024-09-21T15:01:39.973 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:39.973 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.974 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.974 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:39.974 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938944+0000", 2024-09-21T15:01:39.974 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:39.974 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.974 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.974 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:39.974 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938946+0000", 2024-09-21T15:01:39.974 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.616e-06 2024-09-21T15:01:39.974 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.974 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.974 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:39.974 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938952+0000", 2024-09-21T15:01:39.974 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7309999999999998e-06 2024-09-21T15:01:39.975 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.975 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.975 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:39.975 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938957+0000", 2024-09-21T15:01:39.975 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8579999999999999e-06 2024-09-21T15:01:39.975 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.975 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.975 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:39.975 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.959730+0000", 2024-09-21T15:01:39.975 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.020773183000000001 2024-09-21T15:01:39.975 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.975 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.975 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:39.976 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.959810+0000", 2024-09-21T15:01:39.976 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0158000000000001e-05 2024-09-21T15:01:39.976 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.976 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.976 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:39.976 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.960018+0000", 2024-09-21T15:01:39.976 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000207951 2024-09-21T15:01:39.976 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:39.976 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:39.977 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:39.977 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.977 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.977 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:134709 3.a 3:51eaadc0:::rbd_data.11365ed419e3.0000000000000007:67 [read 237568~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:39.977 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.938996+0000", 2024-09-21T15:01:39.977 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 45.980769490999997, 2024-09-21T15:01:39.977 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021382577, 2024-09-21T15:01:39.977 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:39.977 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:39.978 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:39.978 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:39.978 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:39.978 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 134709 2024-09-21T15:01:39.978 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.978 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:39.978 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.978 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:39.978 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938996+0000", 2024-09-21T15:01:39.979 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:39.979 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.979 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.979 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:39.979 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938996+0000", 2024-09-21T15:01:39.979 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:39.979 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.979 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.980 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:39.980 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938995+0000", 2024-09-21T15:01:39.980 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:39.980 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.980 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.980 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:39.980 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938999+0000", 2024-09-21T15:01:39.980 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7979999999999999e-06 2024-09-21T15:01:39.980 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.981 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.981 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:39.981 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939006+0000", 2024-09-21T15:01:39.981 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6669999999999996e-06 2024-09-21T15:01:39.981 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.981 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.981 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:39.981 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939012+0000", 2024-09-21T15:01:39.981 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9329999999999998e-06 2024-09-21T15:01:39.982 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.982 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.982 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:39.982 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.960080+0000", 2024-09-21T15:01:39.982 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021068303 2024-09-21T15:01:39.982 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.982 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.982 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:39.983 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.960154+0000", 2024-09-21T15:01:39.983 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3819000000000006e-05 2024-09-21T15:01:39.983 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.983 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.983 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:39.983 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.960378+0000", 2024-09-21T15:01:39.983 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00022385800000000001 2024-09-21T15:01:39.983 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:39.983 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:39.984 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:39.984 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.984 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.984 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:134710 3.a 3:51eaadc0:::rbd_data.11365ed419e3.0000000000000007:67 [read 253952~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:39.984 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.939017+0000", 2024-09-21T15:01:39.984 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 45.980748480000003, 2024-09-21T15:01:39.984 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021721436, 2024-09-21T15:01:39.984 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:39.984 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:39.985 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:39.985 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:39.985 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:39.985 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 134710 2024-09-21T15:01:39.985 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.985 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:39.985 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.985 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:39.985 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939017+0000", 2024-09-21T15:01:39.986 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:39.986 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.986 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.986 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:39.986 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939017+0000", 2024-09-21T15:01:39.986 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:39.986 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.986 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.986 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:39.987 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939016+0000", 2024-09-21T15:01:39.987 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:39.987 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.987 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.987 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:39.987 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939019+0000", 2024-09-21T15:01:39.987 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.554e-06 2024-09-21T15:01:39.987 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.988 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.988 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:39.988 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939024+0000", 2024-09-21T15:01:39.988 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1569999999999999e-06 2024-09-21T15:01:39.988 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.988 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.988 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:39.988 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939034+0000", 2024-09-21T15:01:39.988 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9219999999999992e-06 2024-09-21T15:01:39.989 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.989 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.989 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:39.989 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.960408+0000", 2024-09-21T15:01:39.989 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021373792999999999 2024-09-21T15:01:39.989 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.989 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.989 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:39.989 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.960482+0000", 2024-09-21T15:01:39.989 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4331000000000004e-05 2024-09-21T15:01:39.990 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.990 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.990 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:39.990 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.960738+0000", 2024-09-21T15:01:39.990 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00025600500000000001 2024-09-21T15:01:39.990 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:39.990 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:39.990 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:39.990 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.990 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.990 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:134711 3.a 3:51eaadc0:::rbd_data.11365ed419e3.0000000000000007:67 [read 270336~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:39.990 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.939039+0000", 2024-09-21T15:01:39.990 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 45.980726431000001, 2024-09-21T15:01:39.990 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022064698000000001, 2024-09-21T15:01:39.991 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:39.991 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:39.991 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:39.991 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:39.991 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:39.991 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 134711 2024-09-21T15:01:39.991 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.991 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:39.991 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.991 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:39.991 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939039+0000", 2024-09-21T15:01:39.991 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:39.991 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.991 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.992 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:39.992 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939039+0000", 2024-09-21T15:01:39.992 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:39.992 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.992 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.992 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:39.992 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939038+0000", 2024-09-21T15:01:39.992 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:39.992 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.992 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.992 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:39.992 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939041+0000", 2024-09-21T15:01:39.992 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9440000000000001e-06 2024-09-21T15:01:39.992 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.992 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.992 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:39.993 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939047+0000", 2024-09-21T15:01:39.993 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1260000000000003e-06 2024-09-21T15:01:39.993 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.993 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.993 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:39.993 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939052+0000", 2024-09-21T15:01:39.993 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1619999999999999e-06 2024-09-21T15:01:39.993 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.993 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.993 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:39.993 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.960768+0000", 2024-09-21T15:01:39.993 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021715140000000001 2024-09-21T15:01:39.993 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.993 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.993 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:39.993 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.960842+0000", 2024-09-21T15:01:39.994 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4698999999999997e-05 2024-09-21T15:01:39.994 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.994 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.994 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:39.994 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.961103+0000", 2024-09-21T15:01:39.994 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00026098499999999998 2024-09-21T15:01:39.994 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:39.994 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:39.994 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:39.994 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.994 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.994 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:134712 3.a 3:51eaadc0:::rbd_data.11365ed419e3.0000000000000007:67 [read 286720~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:39.995 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.939057+0000", 2024-09-21T15:01:39.995 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 45.980707897999999, 2024-09-21T15:01:39.995 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022387067, 2024-09-21T15:01:39.995 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:39.995 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:39.995 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:39.995 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:39.995 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:39.995 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 134712 2024-09-21T15:01:39.995 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.995 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:39.996 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.996 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:39.996 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939057+0000", 2024-09-21T15:01:39.996 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:39.996 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.996 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.996 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:39.996 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939057+0000", 2024-09-21T15:01:39.996 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:39.996 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.996 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.996 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:39.996 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939057+0000", 2024-09-21T15:01:39.996 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:39.997 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.997 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.997 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:39.997 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939059+0000", 2024-09-21T15:01:39.997 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.447e-06 2024-09-21T15:01:39.997 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.997 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.997 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:39.997 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939065+0000", 2024-09-21T15:01:39.997 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4670000000000003e-06 2024-09-21T15:01:39.997 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.997 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.997 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:39.997 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939069+0000", 2024-09-21T15:01:39.997 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2119999999999997e-06 2024-09-21T15:01:39.998 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.998 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.998 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:39.998 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.961141+0000", 2024-09-21T15:01:39.998 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022072268999999999 2024-09-21T15:01:39.998 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.998 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.998 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:39.998 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.961240+0000", 2024-09-21T15:01:39.998 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9147999999999993e-05 2024-09-21T15:01:39.998 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.998 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.998 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:39.998 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.961444+0000", 2024-09-21T15:01:39.998 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00020381899999999999 2024-09-21T15:01:39.999 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:39.999 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:39.999 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:39.999 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:39.999 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:39.999 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:134713 3.a 3:51eaadc0:::rbd_data.11365ed419e3.0000000000000007:67 [read 303104~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:39.999 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.939073+0000", 2024-09-21T15:01:39.999 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 45.980691995000001, 2024-09-21T15:01:39.999 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022687394999999999, 2024-09-21T15:01:39.999 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:39.999 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:39.999 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:39.999 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:39.999 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:39.999 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 134713 2024-09-21T15:01:40.000 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.000 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:40.000 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.000 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:40.000 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939073+0000", 2024-09-21T15:01:40.000 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.000 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.000 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.000 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:40.000 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939073+0000", 2024-09-21T15:01:40.000 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.000 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.000 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.000 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:40.000 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939073+0000", 2024-09-21T15:01:40.001 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:40.001 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.001 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.001 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:40.001 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939075+0000", 2024-09-21T15:01:40.001 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5969999999999999e-06 2024-09-21T15:01:40.001 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.001 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.001 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:40.001 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939081+0000", 2024-09-21T15:01:40.001 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.609e-06 2024-09-21T15:01:40.001 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.001 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.001 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:40.001 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939086+0000", 2024-09-21T15:01:40.001 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.916e-06 2024-09-21T15:01:40.002 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.002 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.002 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:40.002 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.961474+0000", 2024-09-21T15:01:40.002 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022388512999999999 2024-09-21T15:01:40.002 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.002 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.002 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:40.002 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.961567+0000", 2024-09-21T15:01:40.002 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2267000000000003e-05 2024-09-21T15:01:40.002 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.002 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.002 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:40.003 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.961760+0000", 2024-09-21T15:01:40.003 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00019375999999999999 2024-09-21T15:01:40.003 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.003 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:40.003 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.003 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.003 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.003 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:134714 3.a 3:51eaadc0:::rbd_data.11365ed419e3.0000000000000007:67 [read 319488~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:40.003 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.939090+0000", 2024-09-21T15:01:40.003 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 45.980675167999998, 2024-09-21T15:01:40.003 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023043403, 2024-09-21T15:01:40.003 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:40.003 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:40.003 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:40.003 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:40.004 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:40.004 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 134714 2024-09-21T15:01:40.004 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.004 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:40.004 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.004 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:40.004 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939090+0000", 2024-09-21T15:01:40.004 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.004 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.004 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.004 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:40.004 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939090+0000", 2024-09-21T15:01:40.004 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.004 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.005 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.005 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:40.005 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939090+0000", 2024-09-21T15:01:40.005 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:40.005 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.005 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.005 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:40.005 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939092+0000", 2024-09-21T15:01:40.005 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2790000000000001e-06 2024-09-21T15:01:40.005 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.005 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.005 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:40.005 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939097+0000", 2024-09-21T15:01:40.005 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1410000000000004e-06 2024-09-21T15:01:40.005 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.006 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.006 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:40.006 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939102+0000", 2024-09-21T15:01:40.006 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7369999999999997e-06 2024-09-21T15:01:40.006 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.006 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.006 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:40.006 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.961791+0000", 2024-09-21T15:01:40.006 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022688974000000001 2024-09-21T15:01:40.006 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.006 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.006 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:40.006 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.961865+0000", 2024-09-21T15:01:40.006 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4651000000000001e-05 2024-09-21T15:01:40.007 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.007 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.007 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:40.007 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.962133+0000", 2024-09-21T15:01:40.007 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00026793999999999999 2024-09-21T15:01:40.007 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.007 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:40.007 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.007 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.007 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.007 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:134715 3.a 3:51eaadc0:::rbd_data.11365ed419e3.0000000000000007:67 [read 335872~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:40.007 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.939106+0000", 2024-09-21T15:01:40.007 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 45.980658558000002, 2024-09-21T15:01:40.007 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023401142999999999, 2024-09-21T15:01:40.008 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:40.008 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:40.008 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:40.008 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:40.008 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:40.008 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 134715 2024-09-21T15:01:40.008 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.008 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:40.008 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.008 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:40.008 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939106+0000", 2024-09-21T15:01:40.008 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.008 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.008 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.008 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:40.009 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939106+0000", 2024-09-21T15:01:40.009 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.009 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.009 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.009 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:40.009 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939106+0000", 2024-09-21T15:01:40.009 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:40.009 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.009 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.009 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:40.009 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939114+0000", 2024-09-21T15:01:40.009 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1009999999999992e-06 2024-09-21T15:01:40.009 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.009 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.010 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:40.010 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939121+0000", 2024-09-21T15:01:40.010 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7710000000000001e-06 2024-09-21T15:01:40.010 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.010 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.010 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:40.010 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939126+0000", 2024-09-21T15:01:40.010 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3600000000000004e-06 2024-09-21T15:01:40.010 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.010 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.010 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:40.010 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.962162+0000", 2024-09-21T15:01:40.010 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023036148999999999 2024-09-21T15:01:40.010 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.011 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.011 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:40.011 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.962239+0000", 2024-09-21T15:01:40.011 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6874999999999998e-05 2024-09-21T15:01:40.011 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.011 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.011 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:40.011 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.962508+0000", 2024-09-21T15:01:40.011 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00026842199999999999 2024-09-21T15:01:40.011 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.011 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:40.011 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.011 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.012 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.012 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:134716 3.a 3:51eaadc0:::rbd_data.11365ed419e3.0000000000000007:67 [read 352256~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:40.012 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.939131+0000", 2024-09-21T15:01:40.012 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 45.980634424000002, 2024-09-21T15:01:40.012 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023742441, 2024-09-21T15:01:40.012 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:40.012 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:40.012 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:40.012 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:40.012 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:40.012 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 134716 2024-09-21T15:01:40.012 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.012 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:40.012 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.012 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:40.013 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939131+0000", 2024-09-21T15:01:40.013 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.013 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.013 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.013 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:40.013 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939131+0000", 2024-09-21T15:01:40.013 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.013 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.013 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.013 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:40.013 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939130+0000", 2024-09-21T15:01:40.013 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:40.013 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.013 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.014 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:40.014 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939133+0000", 2024-09-21T15:01:40.014 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5620000000000002e-06 2024-09-21T15:01:40.014 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.014 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.014 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:40.014 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939138+0000", 2024-09-21T15:01:40.014 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.322e-06 2024-09-21T15:01:40.014 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.014 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.014 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:40.014 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939143+0000", 2024-09-21T15:01:40.014 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5510000000000001e-06 2024-09-21T15:01:40.015 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.015 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.015 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:40.015 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.962537+0000", 2024-09-21T15:01:40.015 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023394808 2024-09-21T15:01:40.015 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.015 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.015 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:40.015 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.962608+0000", 2024-09-21T15:01:40.015 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0699999999999997e-05 2024-09-21T15:01:40.015 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.015 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.016 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:40.016 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.962873+0000", 2024-09-21T15:01:40.016 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00026487 2024-09-21T15:01:40.016 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.016 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:40.016 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.016 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.016 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.016 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:135024 3.a 3:53261383:::rbd_data.11365ed419e3.0000000000000037:snapdir [list-snaps] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:40.016 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.940162+0000", 2024-09-21T15:01:40.016 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 45.979603460999996, 2024-09-21T15:01:40.016 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022852172, 2024-09-21T15:01:40.016 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:40.017 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:40.017 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:40.017 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:40.017 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:40.017 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 135024 2024-09-21T15:01:40.017 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.017 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:40.017 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.017 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:40.017 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940162+0000", 2024-09-21T15:01:40.017 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.017 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.017 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.018 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:40.018 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940162+0000", 2024-09-21T15:01:40.018 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.018 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.018 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.018 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:40.018 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940161+0000", 2024-09-21T15:01:40.018 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:40.018 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.018 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.018 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:40.018 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940164+0000", 2024-09-21T15:01:40.019 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4099999999999998e-06 2024-09-21T15:01:40.019 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.019 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.019 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:40.019 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940169+0000", 2024-09-21T15:01:40.019 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3720000000000004e-06 2024-09-21T15:01:40.019 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.019 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.019 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:40.019 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940174+0000", 2024-09-21T15:01:40.019 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.724e-06 2024-09-21T15:01:40.019 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.019 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.019 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:40.020 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.962902+0000", 2024-09-21T15:01:40.020 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022728719000000001 2024-09-21T15:01:40.020 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.020 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.020 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:40.020 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.963014+0000", 2024-09-21T15:01:40.020 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000111245 2024-09-21T15:01:40.020 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.020 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:40.020 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.020 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.020 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.020 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:135044 3.a 3:53261383:::rbd_data.11365ed419e3.0000000000000037:snapdir [list-snaps] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:40.020 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.940484+0000", 2024-09-21T15:01:40.021 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 45.979280729000003, 2024-09-21T15:01:40.021 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022619169000000001, 2024-09-21T15:01:40.021 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:40.021 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:40.021 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:40.021 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:40.021 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:40.021 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 135044 2024-09-21T15:01:40.021 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.021 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:40.021 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.021 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:40.021 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940484+0000", 2024-09-21T15:01:40.021 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.021 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.022 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.022 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:40.022 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940484+0000", 2024-09-21T15:01:40.022 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.022 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.022 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.022 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:40.022 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940484+0000", 2024-09-21T15:01:40.022 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:40.022 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.022 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.022 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:40.022 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940487+0000", 2024-09-21T15:01:40.022 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8449999999999999e-06 2024-09-21T15:01:40.022 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.023 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.023 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:40.023 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940493+0000", 2024-09-21T15:01:40.023 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9599999999999997e-06 2024-09-21T15:01:40.023 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.023 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.023 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:40.023 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940497+0000", 2024-09-21T15:01:40.023 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9620000000000001e-06 2024-09-21T15:01:40.023 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.023 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.023 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:40.023 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.963034+0000", 2024-09-21T15:01:40.023 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022536878 2024-09-21T15:01:40.023 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.023 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.024 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:40.024 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.963103+0000", 2024-09-21T15:01:40.024 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9073999999999996e-05 2024-09-21T15:01:40.024 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.024 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:40.024 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.024 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.024 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:40.024 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:40.025 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T15:01:40.026 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:01:40.020+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:01:40.126 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:01:40.126+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:01:40.195 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:40.195 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:01:40.195 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T15:01:40.195 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:40.207 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T15:01:40.214 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T15:01:40.257 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:01:40.258+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:01:40.322 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:40.322 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:01:40.322 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T15:01:40.322 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T15:01:40.322 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:40.328 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:01:40.326+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:01:40.334 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T15:01:40.429 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:01:40.426+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:01:40.438 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:40.438 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T15:01:40.438 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T15:01:40.438 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T15:01:40.439 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.439 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:15130 3.2 3:47f1bbdb:::rbd_data.11361bed1f78.0000000000000067:head [sparse-read 0~147456] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-21T15:01:40.439 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:40.288284+0000", 2024-09-21T15:01:40.439 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 180.14840255199999, 2024-09-21T15:01:40.439 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1142935039999999, 2024-09-21T15:01:40.439 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:40.439 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:40.439 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:40.439 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:40.439 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:40.439 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 15130 2024-09-21T15:01:40.439 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.439 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:40.439 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.440 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:40.440 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288284+0000", 2024-09-21T15:01:40.440 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.440 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.440 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.440 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:40.440 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288284+0000", 2024-09-21T15:01:40.440 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.440 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.440 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.440 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:40.440 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288284+0000", 2024-09-21T15:01:40.440 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:40.440 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.441 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.441 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:40.441 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288288+0000", 2024-09-21T15:01:40.441 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5829999999999998e-06 2024-09-21T15:01:40.441 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.441 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.441 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:40.441 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288297+0000", 2024-09-21T15:01:40.441 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0669999999999998e-06 2024-09-21T15:01:40.441 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.441 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.441 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:40.441 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288305+0000", 2024-09-21T15:01:40.441 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8860000000000004e-06 2024-09-21T15:01:40.441 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.442 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.442 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:40.442 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288417+0000", 2024-09-21T15:01:40.442 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000112952 2024-09-21T15:01:40.442 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.442 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.442 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:40.442 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288428+0000", 2024-09-21T15:01:40.442 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0631e-05 2024-09-21T15:01:40.442 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.442 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.442 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:40.442 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.401876+0000", 2024-09-21T15:01:40.442 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1134479610000001 2024-09-21T15:01:40.443 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.443 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.443 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:40.443 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.402076+0000", 2024-09-21T15:01:40.443 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00019944200000000001 2024-09-21T15:01:40.443 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.443 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.443 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:40.443 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.402578+0000", 2024-09-21T15:01:40.443 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00050242899999999998 2024-09-21T15:01:40.443 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.443 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:40.443 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.444 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.444 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.444 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45708 3.2 3:415db91a:::rbd_data.1136ecbe5879.0000000000000000:28 [read 140288~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:40.444 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141801+0000", 2024-09-21T15:01:40.444 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 150.29488635800001, 2024-09-21T15:01:40.444 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034274200999999997, 2024-09-21T15:01:40.444 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:40.444 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:40.444 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:40.444 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:40.444 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:40.444 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45708 2024-09-21T15:01:40.444 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.444 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:40.445 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.445 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:40.445 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141801+0000", 2024-09-21T15:01:40.445 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.445 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.445 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.445 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:40.445 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141801+0000", 2024-09-21T15:01:40.445 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.445 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.445 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.445 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:40.445 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141800+0000", 2024-09-21T15:01:40.445 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:40.445 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.446 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.446 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:40.446 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141803+0000", 2024-09-21T15:01:40.446 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4210000000000002e-06 2024-09-21T15:01:40.446 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.446 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.446 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:40.446 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141812+0000", 2024-09-21T15:01:40.446 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.9800000000000004e-06 2024-09-21T15:01:40.446 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.446 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.446 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:40.446 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141816+0000", 2024-09-21T15:01:40.446 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4769999999999997e-06 2024-09-21T15:01:40.447 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.447 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.447 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:40.447 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.175865+0000", 2024-09-21T15:01:40.447 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034048875999999999 2024-09-21T15:01:40.447 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.447 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.447 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:40.447 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.175943+0000", 2024-09-21T15:01:40.447 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7464000000000002e-05 2024-09-21T15:01:40.447 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.447 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.447 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:40.447 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176075+0000", 2024-09-21T15:01:40.447 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013230800000000001 2024-09-21T15:01:40.448 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.448 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:40.448 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.448 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.448 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.448 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45709 3.2 3:415db91a:::rbd_data.1136ecbe5879.0000000000000000:28 [read 143360~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:40.448 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141820+0000", 2024-09-21T15:01:40.448 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 150.29486651100001, 2024-09-21T15:01:40.448 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034537536000000001, 2024-09-21T15:01:40.448 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:40.448 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:40.448 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:40.448 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:40.448 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:40.448 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45709 2024-09-21T15:01:40.449 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.449 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:40.449 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.449 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:40.449 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141820+0000", 2024-09-21T15:01:40.449 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.449 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.449 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.449 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:40.449 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141820+0000", 2024-09-21T15:01:40.449 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.449 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.449 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.449 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:40.449 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141820+0000", 2024-09-21T15:01:40.450 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:40.450 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.450 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.450 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:40.450 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141823+0000", 2024-09-21T15:01:40.450 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5110000000000002e-06 2024-09-21T15:01:40.450 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.450 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.450 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:40.450 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141829+0000", 2024-09-21T15:01:40.450 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0850000000000002e-06 2024-09-21T15:01:40.450 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.450 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.450 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:40.450 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141833+0000", 2024-09-21T15:01:40.451 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7620000000000003e-06 2024-09-21T15:01:40.451 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.451 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.451 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:40.451 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176103+0000", 2024-09-21T15:01:40.451 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034269933000000002 2024-09-21T15:01:40.451 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.451 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.451 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:40.451 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176161+0000", 2024-09-21T15:01:40.451 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7658999999999998e-05 2024-09-21T15:01:40.451 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.451 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.451 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:40.452 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176358+0000", 2024-09-21T15:01:40.452 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000196937 2024-09-21T15:01:40.452 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.452 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:40.452 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.452 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.452 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.452 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45710 3.2 3:415db91a:::rbd_data.1136ecbe5879.0000000000000000:28 [read 146432~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:40.452 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141838+0000", 2024-09-21T15:01:40.452 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 150.29484934800001, 2024-09-21T15:01:40.452 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034953674999999997, 2024-09-21T15:01:40.452 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:40.452 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:40.452 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:40.453 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:40.453 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:40.453 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45710 2024-09-21T15:01:40.453 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.453 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:40.453 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.453 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:40.453 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141838+0000", 2024-09-21T15:01:40.453 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.453 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.453 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.453 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:40.453 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141838+0000", 2024-09-21T15:01:40.453 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.454 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.454 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.454 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:40.454 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141837+0000", 2024-09-21T15:01:40.454 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:40.454 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.454 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.454 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:40.454 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141843+0000", 2024-09-21T15:01:40.454 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1619999999999996e-06 2024-09-21T15:01:40.454 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.454 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.454 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:40.455 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141850+0000", 2024-09-21T15:01:40.455 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4799999999999998e-06 2024-09-21T15:01:40.455 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.455 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.455 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:40.455 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141853+0000", 2024-09-21T15:01:40.455 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3050000000000001e-06 2024-09-21T15:01:40.455 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.455 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.455 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:40.455 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176490+0000", 2024-09-21T15:01:40.455 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034636413999999997 2024-09-21T15:01:40.455 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.455 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.456 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:40.456 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176606+0000", 2024-09-21T15:01:40.456 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000115841 2024-09-21T15:01:40.456 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.456 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.456 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:40.456 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176791+0000", 2024-09-21T15:01:40.456 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00018581500000000001 2024-09-21T15:01:40.456 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.456 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:40.456 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.456 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.456 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.456 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45711 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:40.457 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141856+0000", 2024-09-21T15:01:40.457 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 150.29483101100001, 2024-09-21T15:01:40.457 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034986308000000001, 2024-09-21T15:01:40.457 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:40.457 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:40.457 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:40.457 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:40.457 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:40.457 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45711 2024-09-21T15:01:40.457 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.457 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:40.457 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.457 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:40.457 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141856+0000", 2024-09-21T15:01:40.458 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.458 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.458 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.458 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:40.458 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141856+0000", 2024-09-21T15:01:40.458 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.458 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.458 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.458 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:40.458 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141856+0000", 2024-09-21T15:01:40.458 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:40.458 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.458 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.459 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:40.459 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141862+0000", 2024-09-21T15:01:40.459 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.567e-06 2024-09-21T15:01:40.459 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.459 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.459 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:40.459 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141868+0000", 2024-09-21T15:01:40.459 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9569999999999999e-06 2024-09-21T15:01:40.459 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.459 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.459 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:40.459 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141871+0000", 2024-09-21T15:01:40.459 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.146e-06 2024-09-21T15:01:40.459 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.459 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.460 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:40.460 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176831+0000", 2024-09-21T15:01:40.460 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034959935999999997 2024-09-21T15:01:40.460 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.460 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.460 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:40.460 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176842+0000", 2024-09-21T15:01:40.460 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1008e-05 2024-09-21T15:01:40.460 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.460 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:40.460 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.460 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.460 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.460 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45712 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:40.460 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141874+0000", 2024-09-21T15:01:40.461 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 150.294813116, 2024-09-21T15:01:40.461 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034993511999999997, 2024-09-21T15:01:40.461 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:40.461 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:40.461 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:40.461 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:40.461 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:40.461 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45712 2024-09-21T15:01:40.461 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.461 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:40.461 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.461 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:40.461 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141874+0000", 2024-09-21T15:01:40.461 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.461 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.462 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.462 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:40.462 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141874+0000", 2024-09-21T15:01:40.462 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.462 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.462 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.462 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:40.462 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141874+0000", 2024-09-21T15:01:40.462 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:40.462 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.462 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.462 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:40.462 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141875+0000", 2024-09-21T15:01:40.463 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5430000000000001e-06 2024-09-21T15:01:40.463 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.463 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.463 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:40.463 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141878+0000", 2024-09-21T15:01:40.463 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1420000000000002e-06 2024-09-21T15:01:40.463 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.463 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.463 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:40.463 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141881+0000", 2024-09-21T15:01:40.463 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.807e-06 2024-09-21T15:01:40.463 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.463 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.463 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:40.464 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176860+0000", 2024-09-21T15:01:40.464 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034979243 2024-09-21T15:01:40.464 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.464 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.464 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:40.464 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176867+0000", 2024-09-21T15:01:40.464 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0169999999999999e-06 2024-09-21T15:01:40.464 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.464 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:40.464 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.464 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.464 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.464 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45713 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:40.464 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141883+0000", 2024-09-21T15:01:40.464 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 150.29480354099999, 2024-09-21T15:01:40.465 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035010516999999998, 2024-09-21T15:01:40.465 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:40.465 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:40.465 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:40.465 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:40.465 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:40.465 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45713 2024-09-21T15:01:40.465 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.465 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:40.465 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.465 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:40.465 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141883+0000", 2024-09-21T15:01:40.465 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.465 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.466 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.466 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:40.466 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141883+0000", 2024-09-21T15:01:40.466 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.466 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.466 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.466 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:40.466 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141883+0000", 2024-09-21T15:01:40.466 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:40.466 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.466 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.466 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:40.466 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141885+0000", 2024-09-21T15:01:40.466 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4649999999999999e-06 2024-09-21T15:01:40.467 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.467 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.467 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:40.467 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141888+0000", 2024-09-21T15:01:40.467 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3890000000000001e-06 2024-09-21T15:01:40.467 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.467 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.467 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:40.467 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141891+0000", 2024-09-21T15:01:40.467 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9239999999999999e-06 2024-09-21T15:01:40.467 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.467 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.467 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:40.467 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176887+0000", 2024-09-21T15:01:40.468 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034996370999999998 2024-09-21T15:01:40.468 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.468 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.468 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:40.468 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176894+0000", 2024-09-21T15:01:40.468 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6069999999999996e-06 2024-09-21T15:01:40.468 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.468 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:40.468 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.468 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.468 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.468 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45714 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:40.468 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141893+0000", 2024-09-21T15:01:40.468 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 150.294793626, 2024-09-21T15:01:40.468 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035048215000000001, 2024-09-21T15:01:40.469 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:40.469 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:40.469 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:40.469 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:40.469 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:40.469 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45714 2024-09-21T15:01:40.469 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.469 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:40.469 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.469 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:40.469 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141893+0000", 2024-09-21T15:01:40.469 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.469 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.470 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.470 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:40.470 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141893+0000", 2024-09-21T15:01:40.470 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.470 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.470 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.470 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:40.470 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141893+0000", 2024-09-21T15:01:40.470 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:40.470 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.470 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.470 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:40.470 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141895+0000", 2024-09-21T15:01:40.470 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.703e-06 2024-09-21T15:01:40.471 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.471 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.471 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:40.471 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141900+0000", 2024-09-21T15:01:40.471 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.242e-06 2024-09-21T15:01:40.471 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.471 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.471 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:40.471 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141903+0000", 2024-09-21T15:01:40.471 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.117e-06 2024-09-21T15:01:40.471 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.471 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.471 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:40.471 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176930+0000", 2024-09-21T15:01:40.471 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035026750000000002 2024-09-21T15:01:40.472 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.472 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.472 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:40.472 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176942+0000", 2024-09-21T15:01:40.472 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1668000000000001e-05 2024-09-21T15:01:40.472 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.472 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:40.472 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.472 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.472 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.472 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45715 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:40.472 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141906+0000", 2024-09-21T15:01:40.472 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 150.294781468, 2024-09-21T15:01:40.472 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035066057999999997, 2024-09-21T15:01:40.472 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:40.473 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:40.473 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:40.473 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:40.473 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:40.473 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45715 2024-09-21T15:01:40.473 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.473 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:40.473 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.473 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:40.473 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141906+0000", 2024-09-21T15:01:40.473 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.473 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.473 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.473 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:40.473 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141906+0000", 2024-09-21T15:01:40.474 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.474 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.474 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.474 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:40.474 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141905+0000", 2024-09-21T15:01:40.474 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:40.474 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.474 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.474 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:40.474 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141907+0000", 2024-09-21T15:01:40.474 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5400000000000001e-06 2024-09-21T15:01:40.474 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.474 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.474 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:40.475 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141912+0000", 2024-09-21T15:01:40.475 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.502e-06 2024-09-21T15:01:40.475 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.475 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.475 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:40.475 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141915+0000", 2024-09-21T15:01:40.475 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0170000000000001e-06 2024-09-21T15:01:40.475 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.475 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.475 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:40.475 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176964+0000", 2024-09-21T15:01:40.475 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035048587999999999 2024-09-21T15:01:40.475 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.475 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.475 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:40.476 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176972+0000", 2024-09-21T15:01:40.476 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6580000000000002e-06 2024-09-21T15:01:40.476 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.476 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:40.476 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.476 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.476 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.476 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45716 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:40.476 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141918+0000", 2024-09-21T15:01:40.476 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 150.29476929200001, 2024-09-21T15:01:40.476 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035086223, 2024-09-21T15:01:40.476 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:40.476 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:40.476 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:40.476 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:40.477 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:40.477 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45716 2024-09-21T15:01:40.477 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.477 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:40.477 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.477 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:40.477 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141918+0000", 2024-09-21T15:01:40.477 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.477 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.477 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.477 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:40.477 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141918+0000", 2024-09-21T15:01:40.477 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.477 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.477 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.477 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:40.478 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141917+0000", 2024-09-21T15:01:40.478 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:40.478 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.478 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.478 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:40.478 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141919+0000", 2024-09-21T15:01:40.478 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5680000000000001e-06 2024-09-21T15:01:40.478 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.478 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.478 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:40.478 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141922+0000", 2024-09-21T15:01:40.478 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2789999999999999e-06 2024-09-21T15:01:40.478 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.478 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.479 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:40.479 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141925+0000", 2024-09-21T15:01:40.479 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9469999999999999e-06 2024-09-21T15:01:40.479 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.479 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.479 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:40.479 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176997+0000", 2024-09-21T15:01:40.479 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035071366999999999 2024-09-21T15:01:40.479 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.479 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.479 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:40.479 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177004+0000", 2024-09-21T15:01:40.479 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3309999999999998e-06 2024-09-21T15:01:40.479 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.480 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:40.480 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.480 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.480 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.480 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45717 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:40.480 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141928+0000", 2024-09-21T15:01:40.480 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 150.29475934600001, 2024-09-21T15:01:40.480 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035102573999999998, 2024-09-21T15:01:40.480 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:40.480 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:40.480 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:40.480 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:40.480 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:40.480 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45717 2024-09-21T15:01:40.480 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.480 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:40.481 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.481 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:40.481 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141928+0000", 2024-09-21T15:01:40.481 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.481 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.481 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.481 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:40.481 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141928+0000", 2024-09-21T15:01:40.481 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.481 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.481 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.481 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:40.481 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141927+0000", 2024-09-21T15:01:40.481 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:01:40.481 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.482 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.482 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:40.482 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141929+0000", 2024-09-21T15:01:40.482 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.533e-06 2024-09-21T15:01:40.482 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.482 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.482 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:40.482 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141934+0000", 2024-09-21T15:01:40.482 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1660000000000002e-06 2024-09-21T15:01:40.482 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.482 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.482 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:40.482 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141937+0000", 2024-09-21T15:01:40.482 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0309999999999999e-06 2024-09-21T15:01:40.483 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.483 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.483 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:40.483 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177023+0000", 2024-09-21T15:01:40.483 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035085595999999997 2024-09-21T15:01:40.483 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.483 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.483 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:40.483 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177030+0000", 2024-09-21T15:01:40.483 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.486e-06 2024-09-21T15:01:40.483 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.483 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:40.483 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.483 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.483 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.484 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45718 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:40.484 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141940+0000", 2024-09-21T15:01:40.484 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 150.29474748800001, 2024-09-21T15:01:40.484 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035124956999999998, 2024-09-21T15:01:40.484 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:40.484 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:40.484 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:40.484 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:40.484 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:40.484 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45718 2024-09-21T15:01:40.484 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.484 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:40.484 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.484 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:40.484 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141940+0000", 2024-09-21T15:01:40.485 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.485 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.485 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.485 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:40.485 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141940+0000", 2024-09-21T15:01:40.485 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.485 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.485 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.485 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:40.485 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141939+0000", 2024-09-21T15:01:40.485 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:40.485 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.485 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.485 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:40.485 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141941+0000", 2024-09-21T15:01:40.485 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.502e-06 2024-09-21T15:01:40.486 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.486 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.486 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:40.486 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141946+0000", 2024-09-21T15:01:40.486 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3449999999999996e-06 2024-09-21T15:01:40.486 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.486 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.486 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:40.486 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141949+0000", 2024-09-21T15:01:40.486 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9749999999999999e-06 2024-09-21T15:01:40.486 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.486 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.486 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:40.486 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177057+0000", 2024-09-21T15:01:40.486 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035108385999999998 2024-09-21T15:01:40.487 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.487 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.487 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:40.487 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177064+0000", 2024-09-21T15:01:40.487 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0099999999999998e-06 2024-09-21T15:01:40.487 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.487 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:40.487 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.487 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.487 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.487 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45719 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:40.487 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141952+0000", 2024-09-21T15:01:40.487 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 150.294735215, 2024-09-21T15:01:40.487 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035139108000000002, 2024-09-21T15:01:40.488 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:40.488 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:40.488 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:40.488 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:40.488 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:40.488 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45719 2024-09-21T15:01:40.488 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.488 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:40.488 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.488 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:40.488 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141952+0000", 2024-09-21T15:01:40.488 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.488 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.488 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.488 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:40.489 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141952+0000", 2024-09-21T15:01:40.489 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.489 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.489 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.489 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:40.489 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141952+0000", 2024-09-21T15:01:40.489 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:40.489 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.489 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.489 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:40.489 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141953+0000", 2024-09-21T15:01:40.489 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.486e-06 2024-09-21T15:01:40.489 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.489 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.489 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:40.489 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141959+0000", 2024-09-21T15:01:40.490 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4940000000000003e-06 2024-09-21T15:01:40.490 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.490 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.490 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:40.490 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141963+0000", 2024-09-21T15:01:40.490 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3540000000000002e-06 2024-09-21T15:01:40.490 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.490 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.490 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:40.490 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177084+0000", 2024-09-21T15:01:40.490 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035120983000000001 2024-09-21T15:01:40.490 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.490 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.490 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:40.491 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177091+0000", 2024-09-21T15:01:40.491 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0350000000000004e-06 2024-09-21T15:01:40.491 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.491 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:40.491 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.491 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.491 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.491 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45720 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:40.491 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141965+0000", 2024-09-21T15:01:40.491 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 150.294721748, 2024-09-21T15:01:40.491 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035155268000000003, 2024-09-21T15:01:40.491 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:40.491 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:40.491 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:40.492 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:40.492 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:40.492 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45720 2024-09-21T15:01:40.492 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.492 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:40.492 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.492 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:40.492 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141965+0000", 2024-09-21T15:01:40.492 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.492 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.492 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.492 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:40.492 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141965+0000", 2024-09-21T15:01:40.492 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.492 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.492 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.493 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:40.493 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141965+0000", 2024-09-21T15:01:40.493 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:40.493 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.493 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.493 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:40.493 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141966+0000", 2024-09-21T15:01:40.493 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.46e-06 2024-09-21T15:01:40.493 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.493 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.493 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:40.493 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141972+0000", 2024-09-21T15:01:40.493 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4210000000000003e-06 2024-09-21T15:01:40.494 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.494 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.494 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:40.494 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141975+0000", 2024-09-21T15:01:40.494 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.948e-06 2024-09-21T15:01:40.494 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.494 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.494 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:40.494 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177113+0000", 2024-09-21T15:01:40.494 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035137855000000003 2024-09-21T15:01:40.494 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.494 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.494 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:40.494 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177121+0000", 2024-09-21T15:01:40.495 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8269999999999998e-06 2024-09-21T15:01:40.495 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.495 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:40.495 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.495 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.495 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.495 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45721 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:40.495 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141977+0000", 2024-09-21T15:01:40.495 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 150.29470972499999, 2024-09-21T15:01:40.495 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035169984000000001, 2024-09-21T15:01:40.495 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:40.495 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:40.495 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:40.496 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:40.496 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:40.496 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45721 2024-09-21T15:01:40.496 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.496 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:40.496 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.496 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:40.496 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141977+0000", 2024-09-21T15:01:40.496 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.496 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.496 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.496 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:40.496 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141977+0000", 2024-09-21T15:01:40.496 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.497 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.497 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.497 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:40.497 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141977+0000", 2024-09-21T15:01:40.497 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:40.497 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.497 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.497 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:40.497 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141979+0000", 2024-09-21T15:01:40.497 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.511e-06 2024-09-21T15:01:40.497 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.497 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.497 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:40.497 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141982+0000", 2024-09-21T15:01:40.498 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4180000000000001e-06 2024-09-21T15:01:40.498 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.498 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.498 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:40.498 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141985+0000", 2024-09-21T15:01:40.498 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9450000000000002e-06 2024-09-21T15:01:40.498 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.498 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.498 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:40.498 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177139+0000", 2024-09-21T15:01:40.498 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035154409999999997 2024-09-21T15:01:40.498 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.498 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.498 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:40.499 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177147+0000", 2024-09-21T15:01:40.499 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9570000000000002e-06 2024-09-21T15:01:40.499 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.499 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:40.499 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.499 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.499 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.499 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45722 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:40.499 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141987+0000", 2024-09-21T15:01:40.499 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 150.29469975399999, 2024-09-21T15:01:40.499 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035188492000000002, 2024-09-21T15:01:40.499 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:40.499 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:40.499 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:40.500 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:40.500 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:40.500 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45722 2024-09-21T15:01:40.500 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.500 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:40.500 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.500 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:40.500 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141987+0000", 2024-09-21T15:01:40.500 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.500 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.500 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.500 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:40.500 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141987+0000", 2024-09-21T15:01:40.500 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.501 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.501 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.501 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:40.501 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141987+0000", 2024-09-21T15:01:40.501 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:40.501 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.501 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.501 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:40.501 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141988+0000", 2024-09-21T15:01:40.501 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.463e-06 2024-09-21T15:01:40.501 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.501 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.501 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:40.502 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141994+0000", 2024-09-21T15:01:40.502 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.361e-06 2024-09-21T15:01:40.502 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.502 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.502 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:40.502 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141997+0000", 2024-09-21T15:01:40.502 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0359999999999998e-06 2024-09-21T15:01:40.502 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.502 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.502 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:40.502 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177169+0000", 2024-09-21T15:01:40.502 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035171651999999998 2024-09-21T15:01:40.502 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.502 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.503 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:40.503 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177176+0000", 2024-09-21T15:01:40.503 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2189999999999998e-06 2024-09-21T15:01:40.503 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.503 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:40.503 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.503 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.503 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.503 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45723 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:40.503 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141999+0000", 2024-09-21T15:01:40.503 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 150.294687587, 2024-09-21T15:01:40.503 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035197701999999997, 2024-09-21T15:01:40.503 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:40.503 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:40.504 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:40.504 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:40.504 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:40.504 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45723 2024-09-21T15:01:40.504 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.504 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:40.504 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.504 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:40.504 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141999+0000", 2024-09-21T15:01:40.504 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.504 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.504 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.504 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:40.504 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141999+0000", 2024-09-21T15:01:40.505 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.505 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.505 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.505 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:40.505 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141999+0000", 2024-09-21T15:01:40.505 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:40.505 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.505 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.505 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:40.505 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142003+0000", 2024-09-21T15:01:40.505 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5049999999999999e-06 2024-09-21T15:01:40.505 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.505 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.505 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:40.505 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142008+0000", 2024-09-21T15:01:40.506 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6579999999999999e-06 2024-09-21T15:01:40.506 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.506 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.506 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:40.506 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142011+0000", 2024-09-21T15:01:40.506 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0529999999999998e-06 2024-09-21T15:01:40.506 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.506 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.506 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:40.506 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177194+0000", 2024-09-21T15:01:40.506 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035182221999999999 2024-09-21T15:01:40.506 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.506 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.506 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:40.507 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177197+0000", 2024-09-21T15:01:40.507 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5030000000000002e-06 2024-09-21T15:01:40.507 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.507 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:40.507 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.507 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.507 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.507 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45724 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:40.507 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.142015+0000", 2024-09-21T15:01:40.507 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 150.29467162, 2024-09-21T15:01:40.507 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035195345000000003, 2024-09-21T15:01:40.507 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:40.507 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:40.507 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:40.508 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:40.508 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:40.508 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45724 2024-09-21T15:01:40.508 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.508 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:40.508 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.508 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:40.508 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142015+0000", 2024-09-21T15:01:40.508 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.508 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.508 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.508 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:40.508 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142015+0000", 2024-09-21T15:01:40.508 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.508 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.509 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.509 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:40.509 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142015+0000", 2024-09-21T15:01:40.509 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:40.509 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.509 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.509 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:40.509 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142017+0000", 2024-09-21T15:01:40.509 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5540000000000001e-06 2024-09-21T15:01:40.509 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.509 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.509 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:40.509 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142022+0000", 2024-09-21T15:01:40.510 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4229999999999996e-06 2024-09-21T15:01:40.510 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.510 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.510 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:40.510 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142025+0000", 2024-09-21T15:01:40.510 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.056e-06 2024-09-21T15:01:40.510 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.510 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.510 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:40.510 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177207+0000", 2024-09-21T15:01:40.510 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035182089 2024-09-21T15:01:40.510 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.510 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.510 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:40.511 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177211+0000", 2024-09-21T15:01:40.511 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4649999999999999e-06 2024-09-21T15:01:40.511 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.511 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:40.511 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.511 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.511 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.511 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45725 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:40.511 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.142028+0000", 2024-09-21T15:01:40.511 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 150.294659341, 2024-09-21T15:01:40.511 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035195731000000001, 2024-09-21T15:01:40.511 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:40.511 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:40.511 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:40.512 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:40.512 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:40.512 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45725 2024-09-21T15:01:40.512 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.512 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:40.512 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.512 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:40.512 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142028+0000", 2024-09-21T15:01:40.512 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.512 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.512 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.512 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:40.512 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142028+0000", 2024-09-21T15:01:40.512 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.512 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.512 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.513 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:40.513 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142027+0000", 2024-09-21T15:01:40.513 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:40.513 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.513 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.513 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:40.513 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142029+0000", 2024-09-21T15:01:40.513 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5269999999999999e-06 2024-09-21T15:01:40.513 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.513 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.513 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:40.513 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142032+0000", 2024-09-21T15:01:40.513 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3249999999999999e-06 2024-09-21T15:01:40.513 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.513 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.513 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:40.514 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142035+0000", 2024-09-21T15:01:40.514 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.897e-06 2024-09-21T15:01:40.514 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.514 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.514 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:40.514 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177220+0000", 2024-09-21T15:01:40.514 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035184871 2024-09-21T15:01:40.514 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.514 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.514 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:40.514 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177223+0000", 2024-09-21T15:01:40.514 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3680000000000002e-06 2024-09-21T15:01:40.514 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.515 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:40.515 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.515 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.515 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.515 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45726 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:40.515 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.142037+0000", 2024-09-21T15:01:40.515 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 150.29464962599999, 2024-09-21T15:01:40.515 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035199273000000003, 2024-09-21T15:01:40.515 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:40.515 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:40.515 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:40.515 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:40.515 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:40.515 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45726 2024-09-21T15:01:40.516 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.516 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:40.516 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.516 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:40.516 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142037+0000", 2024-09-21T15:01:40.516 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.516 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.516 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.516 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:40.516 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142037+0000", 2024-09-21T15:01:40.516 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.516 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.516 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.516 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:40.517 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142037+0000", 2024-09-21T15:01:40.517 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:40.517 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.517 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.517 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:40.517 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142038+0000", 2024-09-21T15:01:40.517 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.344e-06 2024-09-21T15:01:40.517 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.517 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.517 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:40.517 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142044+0000", 2024-09-21T15:01:40.517 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1930000000000001e-06 2024-09-21T15:01:40.517 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.517 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.517 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:40.518 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142047+0000", 2024-09-21T15:01:40.518 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0280000000000001e-06 2024-09-21T15:01:40.518 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.518 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.518 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:40.518 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177233+0000", 2024-09-21T15:01:40.518 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035186394000000003 2024-09-21T15:01:40.518 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.518 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.518 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:40.518 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177237+0000", 2024-09-21T15:01:40.518 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5530000000000001e-06 2024-09-21T15:01:40.518 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.518 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:40.519 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.519 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.519 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:40.519 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:40.520 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T15:01:40.561 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:01:40.562+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:01:40.630 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:01:40.630+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:01:40.671 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:40.671 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:01:40.671 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T15:01:40.671 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:40.681 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T15:01:40.730 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:01:40.724+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:01:40.787 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:40.787 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:01:40.787 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T15:01:40.787 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T15:01:40.787 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:40.796 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T15:01:40.831 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:01:40.824+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:01:40.910 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:40.910 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T15:01:40.910 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T15:01:40.911 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T15:01:40.911 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.911 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139776 3.16 3:6c38d0e2:::rbd_data.113664780c85.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:40.911 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.223442+0000", 2024-09-21T15:01:40.911 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 38.684859725000003, 2024-09-21T15:01:40.911 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0315488150000001, 2024-09-21T15:01:40.911 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:40.911 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:40.911 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:40.911 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:40.911 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:40.911 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139776 2024-09-21T15:01:40.911 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.912 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:40.912 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.912 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:40.912 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223442+0000", 2024-09-21T15:01:40.912 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.912 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.912 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.912 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:40.912 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223442+0000", 2024-09-21T15:01:40.912 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.912 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.912 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.912 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:40.912 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223442+0000", 2024-09-21T15:01:40.912 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:40.913 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.913 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.913 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:40.913 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223446+0000", 2024-09-21T15:01:40.913 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2549999999999999e-06 2024-09-21T15:01:40.913 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.913 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.913 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:40.913 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223454+0000", 2024-09-21T15:01:40.913 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8199999999999997e-06 2024-09-21T15:01:40.913 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.913 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.913 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:40.913 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223460+0000", 2024-09-21T15:01:40.914 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6330000000000001e-06 2024-09-21T15:01:40.914 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.914 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.914 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:40.914 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223542+0000", 2024-09-21T15:01:40.914 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1929999999999997e-05 2024-09-21T15:01:40.914 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.914 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.914 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:40.914 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223552+0000", 2024-09-21T15:01:40.914 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0627e-05 2024-09-21T15:01:40.914 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.914 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.914 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:40.915 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.254634+0000", 2024-09-21T15:01:40.915 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.031081972 2024-09-21T15:01:40.915 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.915 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.915 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:40.915 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.254796+0000", 2024-09-21T15:01:40.915 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000161879 2024-09-21T15:01:40.915 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.915 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.915 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:40.915 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.254991+0000", 2024-09-21T15:01:40.915 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00019526499999999999 2024-09-21T15:01:40.915 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.915 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:40.915 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.916 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.916 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.916 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139785 3.11 3:8abbb109:::rbd_data.113664780c85.0000000000000070:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:40.916 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.223531+0000", 2024-09-21T15:01:40.916 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 38.684770983999996, 2024-09-21T15:01:40.916 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0187143510000001, 2024-09-21T15:01:40.916 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:40.916 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:40.916 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:40.916 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:40.916 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:40.916 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139785 2024-09-21T15:01:40.916 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.916 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:40.916 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.917 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:40.917 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223531+0000", 2024-09-21T15:01:40.917 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.917 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.917 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.917 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:40.917 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223531+0000", 2024-09-21T15:01:40.917 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.917 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.917 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.917 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:40.917 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223531+0000", 2024-09-21T15:01:40.917 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:40.917 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.917 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.917 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:40.918 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223534+0000", 2024-09-21T15:01:40.918 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2650000000000001e-06 2024-09-21T15:01:40.918 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.918 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.918 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:40.918 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223541+0000", 2024-09-21T15:01:40.918 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8990000000000004e-06 2024-09-21T15:01:40.918 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.918 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.918 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:40.918 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223546+0000", 2024-09-21T15:01:40.918 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3399999999999997e-06 2024-09-21T15:01:40.918 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.918 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.919 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:40.919 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223724+0000", 2024-09-21T15:01:40.919 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017745799999999999 2024-09-21T15:01:40.919 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.919 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.919 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:40.919 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223735+0000", 2024-09-21T15:01:40.919 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1610999999999999e-05 2024-09-21T15:01:40.919 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.919 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.919 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:40.919 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242113+0000", 2024-09-21T15:01:40.919 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.018377879 2024-09-21T15:01:40.919 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.920 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.920 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:40.920 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242246+0000", 2024-09-21T15:01:40.920 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013231500000000001 2024-09-21T15:01:40.920 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.920 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:40.920 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.920 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.920 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.920 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139790 3.11 3:8a18c7d8:::rbd_data.113664780c85.000000000000008d:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:40.920 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.223578+0000", 2024-09-21T15:01:40.920 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 38.684723988999998, 2024-09-21T15:01:40.920 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0187879719999999, 2024-09-21T15:01:40.920 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:40.920 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:40.920 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:40.921 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:40.921 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:40.921 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139790 2024-09-21T15:01:40.921 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.921 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:40.921 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.921 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:40.921 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223578+0000", 2024-09-21T15:01:40.921 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.921 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.921 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.921 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:40.921 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223578+0000", 2024-09-21T15:01:40.921 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.921 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.922 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.922 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:40.922 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223578+0000", 2024-09-21T15:01:40.922 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:40.922 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.922 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.922 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:40.922 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223581+0000", 2024-09-21T15:01:40.922 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6879999999999999e-06 2024-09-21T15:01:40.922 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.922 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.922 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:40.922 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223586+0000", 2024-09-21T15:01:40.922 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7080000000000002e-06 2024-09-21T15:01:40.922 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.923 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.923 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:40.923 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223591+0000", 2024-09-21T15:01:40.923 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5639999999999998e-06 2024-09-21T15:01:40.923 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.923 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.923 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:40.923 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223761+0000", 2024-09-21T15:01:40.923 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000170213 2024-09-21T15:01:40.923 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.923 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.923 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:40.923 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223769+0000", 2024-09-21T15:01:40.923 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7049999999999999e-06 2024-09-21T15:01:40.924 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.924 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.924 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:40.924 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242269+0000", 2024-09-21T15:01:40.924 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.018500288 2024-09-21T15:01:40.924 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.924 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.924 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:40.924 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242366+0000", 2024-09-21T15:01:40.924 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.7126000000000005e-05 2024-09-21T15:01:40.924 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.924 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:40.924 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.924 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.925 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.925 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139792 3.16 3:6ce5caf3:::rbd_data.113664780c85.000000000000009b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:40.925 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.223595+0000", 2024-09-21T15:01:40.925 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 38.684707035999999, 2024-09-21T15:01:40.925 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0317517899999999, 2024-09-21T15:01:40.925 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:40.925 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:40.925 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:40.925 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:40.925 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:40.925 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139792 2024-09-21T15:01:40.925 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.925 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:40.925 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.925 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:40.926 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223595+0000", 2024-09-21T15:01:40.926 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.926 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.926 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.926 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:40.926 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223595+0000", 2024-09-21T15:01:40.926 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.926 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.926 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.926 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:40.926 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223595+0000", 2024-09-21T15:01:40.926 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:40.926 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.926 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.926 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:40.927 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223598+0000", 2024-09-21T15:01:40.927 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4620000000000001e-06 2024-09-21T15:01:40.927 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.927 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.927 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:40.927 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223604+0000", 2024-09-21T15:01:40.927 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6879999999999996e-06 2024-09-21T15:01:40.927 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.927 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.927 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:40.927 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223609+0000", 2024-09-21T15:01:40.927 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9490000000000004e-06 2024-09-21T15:01:40.927 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.927 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.928 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:40.928 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224067+0000", 2024-09-21T15:01:40.928 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00045780700000000002 2024-09-21T15:01:40.928 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.928 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.928 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:40.928 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224075+0000", 2024-09-21T15:01:40.928 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6610000000000006e-06 2024-09-21T15:01:40.928 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.928 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.928 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:40.928 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255141+0000", 2024-09-21T15:01:40.928 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0310657480000001 2024-09-21T15:01:40.928 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.928 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.929 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:40.929 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255347+0000", 2024-09-21T15:01:40.929 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000205837 2024-09-21T15:01:40.929 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.929 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:40.929 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.929 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.929 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.929 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139794 3.11 3:8cf3a2f4:::rbd_data.113664780c85.00000000000000a9:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:40.929 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.224011+0000", 2024-09-21T15:01:40.929 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 38.684291008000002, 2024-09-21T15:01:40.929 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.018507791, 2024-09-21T15:01:40.929 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:40.929 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:40.929 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:40.929 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:40.930 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:40.930 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139794 2024-09-21T15:01:40.930 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.930 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:40.930 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.930 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:40.930 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224011+0000", 2024-09-21T15:01:40.930 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.930 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.930 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.930 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:40.930 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224011+0000", 2024-09-21T15:01:40.930 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.930 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.931 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.931 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:40.931 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224010+0000", 2024-09-21T15:01:40.931 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:40.931 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.931 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.931 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:40.931 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224016+0000", 2024-09-21T15:01:40.931 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6550000000000001e-06 2024-09-21T15:01:40.931 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.931 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.931 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:40.931 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224028+0000", 2024-09-21T15:01:40.931 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2422999999999999e-05 2024-09-21T15:01:40.932 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.932 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.932 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:40.932 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224036+0000", 2024-09-21T15:01:40.932 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7749999999999993e-06 2024-09-21T15:01:40.932 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.932 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.932 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:40.932 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225176+0000", 2024-09-21T15:01:40.932 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0011401740000000001 2024-09-21T15:01:40.932 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.932 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.932 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:40.932 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225187+0000", 2024-09-21T15:01:40.932 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0951999999999999e-05 2024-09-21T15:01:40.933 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.933 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.933 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:40.933 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242392+0000", 2024-09-21T15:01:40.933 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.017204905 2024-09-21T15:01:40.933 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.933 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.933 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:40.933 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242519+0000", 2024-09-21T15:01:40.933 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012680499999999999 2024-09-21T15:01:40.933 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.933 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:40.933 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.933 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.934 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.934 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139800 3.16 3:68b050ed:::rbd_data.113664780c85.00000000000000d3:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:40.934 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.224579+0000", 2024-09-21T15:01:40.934 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 38.683723249000003, 2024-09-21T15:01:40.934 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.030932637, 2024-09-21T15:01:40.934 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:40.934 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:40.934 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:40.934 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:40.934 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:40.934 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139800 2024-09-21T15:01:40.934 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.934 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:40.934 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.935 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:40.935 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224579+0000", 2024-09-21T15:01:40.935 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.935 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.935 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.935 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:40.935 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224579+0000", 2024-09-21T15:01:40.935 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.935 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.935 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.935 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:40.935 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224578+0000", 2024-09-21T15:01:40.935 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:40.935 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.935 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.936 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:40.936 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224582+0000", 2024-09-21T15:01:40.936 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0029999999999999e-06 2024-09-21T15:01:40.936 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.936 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.936 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:40.936 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224588+0000", 2024-09-21T15:01:40.936 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8120000000000002e-06 2024-09-21T15:01:40.936 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.936 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.936 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:40.936 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224593+0000", 2024-09-21T15:01:40.936 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7469999999999996e-06 2024-09-21T15:01:40.937 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.937 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.937 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:40.937 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224648+0000", 2024-09-21T15:01:40.937 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4693e-05 2024-09-21T15:01:40.937 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.937 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.937 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:40.937 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224656+0000", 2024-09-21T15:01:40.937 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8429999999999993e-06 2024-09-21T15:01:40.937 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.937 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.937 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:40.937 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255382+0000", 2024-09-21T15:01:40.938 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0307266070000001 2024-09-21T15:01:40.938 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.938 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.938 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:40.938 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255512+0000", 2024-09-21T15:01:40.938 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000129367 2024-09-21T15:01:40.938 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.938 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:40.938 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.938 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.938 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.938 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139804 3.11 3:88f9ddce:::rbd_data.113664780c85.00000000000000ef:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:40.938 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.224632+0000", 2024-09-21T15:01:40.938 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 38.683670268, 2024-09-21T15:01:40.939 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.018021302, 2024-09-21T15:01:40.939 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:40.939 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:40.939 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:40.939 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:40.939 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:40.939 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139804 2024-09-21T15:01:40.939 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.939 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:40.939 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.939 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:40.939 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224632+0000", 2024-09-21T15:01:40.939 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.940 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.940 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.940 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:40.940 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224632+0000", 2024-09-21T15:01:40.940 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.940 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.940 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.940 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:40.940 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224632+0000", 2024-09-21T15:01:40.940 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:40.940 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.940 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.940 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:40.940 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224635+0000", 2024-09-21T15:01:40.940 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2710000000000001e-06 2024-09-21T15:01:40.941 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.941 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.941 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:40.941 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224640+0000", 2024-09-21T15:01:40.941 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6779999999999997e-06 2024-09-21T15:01:40.941 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.941 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.941 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:40.941 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224645+0000", 2024-09-21T15:01:40.941 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9339999999999997e-06 2024-09-21T15:01:40.941 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.941 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.941 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:40.942 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225307+0000", 2024-09-21T15:01:40.942 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00066201699999999999 2024-09-21T15:01:40.942 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.942 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.942 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:40.942 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225312+0000", 2024-09-21T15:01:40.942 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9799999999999998e-06 2024-09-21T15:01:40.942 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.942 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.942 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:40.942 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242548+0000", 2024-09-21T15:01:40.942 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.017235603 2024-09-21T15:01:40.942 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.943 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.943 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:40.943 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242653+0000", 2024-09-21T15:01:40.943 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010522 2024-09-21T15:01:40.943 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.943 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:40.943 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.943 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.943 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.943 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139809 3.11 3:884da946:::rbd_data.113664780c85.0000000000000118:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:40.943 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.224667+0000", 2024-09-21T15:01:40.943 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 38.683635224, 2024-09-21T15:01:40.943 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0181101960000001, 2024-09-21T15:01:40.943 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:40.944 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:40.944 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:40.944 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:40.944 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:40.944 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139809 2024-09-21T15:01:40.944 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.944 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:40.944 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.944 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:40.944 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224667+0000", 2024-09-21T15:01:40.944 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.944 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.944 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.944 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:40.945 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224667+0000", 2024-09-21T15:01:40.945 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.945 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.945 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.945 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:40.945 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224667+0000", 2024-09-21T15:01:40.945 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:40.945 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.945 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.945 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:40.945 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224669+0000", 2024-09-21T15:01:40.945 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.452e-06 2024-09-21T15:01:40.945 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.945 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.946 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:40.946 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224675+0000", 2024-09-21T15:01:40.946 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.818e-06 2024-09-21T15:01:40.946 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.946 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.946 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:40.946 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224680+0000", 2024-09-21T15:01:40.946 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6319999999999999e-06 2024-09-21T15:01:40.946 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.946 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.946 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:40.946 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225400+0000", 2024-09-21T15:01:40.946 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00072012699999999999 2024-09-21T15:01:40.947 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.947 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.947 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:40.947 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225409+0000", 2024-09-21T15:01:40.947 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1039999999999996e-06 2024-09-21T15:01:40.947 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.947 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.947 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:40.947 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242675+0000", 2024-09-21T15:01:40.947 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0172663319999999 2024-09-21T15:01:40.947 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.947 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.947 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:40.947 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242777+0000", 2024-09-21T15:01:40.947 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010197899999999999 2024-09-21T15:01:40.948 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.948 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:40.948 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.948 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.948 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.948 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139868 3.16 3:69f82090:::rbd_data.113664780c85.00000000000000f1:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:40.948 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.225982+0000", 2024-09-21T15:01:40.948 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 38.682319724000003, 2024-09-21T15:01:40.948 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0296791729999999, 2024-09-21T15:01:40.948 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:40.948 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:40.948 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:40.948 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:40.948 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:40.949 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139868 2024-09-21T15:01:40.949 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.949 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:40.949 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.949 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:40.949 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225982+0000", 2024-09-21T15:01:40.949 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.949 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.949 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.949 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:40.949 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225982+0000", 2024-09-21T15:01:40.949 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.949 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.949 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.949 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:40.949 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225982+0000", 2024-09-21T15:01:40.950 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:40.950 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.950 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.950 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:40.950 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225985+0000", 2024-09-21T15:01:40.950 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.74e-06 2024-09-21T15:01:40.950 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.950 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.950 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:40.950 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225990+0000", 2024-09-21T15:01:40.950 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7039999999999999e-06 2024-09-21T15:01:40.950 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.950 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.950 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:40.950 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225995+0000", 2024-09-21T15:01:40.951 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7249999999999997e-06 2024-09-21T15:01:40.951 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.951 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.951 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:40.951 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226299+0000", 2024-09-21T15:01:40.951 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00030383199999999997 2024-09-21T15:01:40.951 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.951 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.951 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:40.951 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226308+0000", 2024-09-21T15:01:40.951 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.9740000000000008e-06 2024-09-21T15:01:40.951 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.951 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.951 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:40.952 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255539+0000", 2024-09-21T15:01:40.952 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0292305960000001 2024-09-21T15:01:40.952 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.952 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.952 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:40.952 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255662+0000", 2024-09-21T15:01:40.952 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012307400000000001 2024-09-21T15:01:40.952 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.952 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:40.952 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.952 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.952 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.952 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139869 3.16 3:6b6c90a4:::rbd_data.113664780c85.00000000000000fe:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:40.952 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.226006+0000", 2024-09-21T15:01:40.952 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 38.682296108000003, 2024-09-21T15:01:40.953 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.029790518, 2024-09-21T15:01:40.953 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:40.953 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:40.953 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:40.953 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:40.953 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:40.953 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139869 2024-09-21T15:01:40.953 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.953 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:40.953 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.953 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:40.953 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226006+0000", 2024-09-21T15:01:40.953 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.953 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.953 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.954 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:40.954 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226006+0000", 2024-09-21T15:01:40.954 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.954 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.954 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.954 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:40.954 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226006+0000", 2024-09-21T15:01:40.954 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:40.954 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.954 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.954 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:40.954 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226008+0000", 2024-09-21T15:01:40.954 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5569999999999998e-06 2024-09-21T15:01:40.954 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.955 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.955 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:40.955 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226014+0000", 2024-09-21T15:01:40.955 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2079999999999997e-06 2024-09-21T15:01:40.955 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.955 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.955 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:40.955 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226020+0000", 2024-09-21T15:01:40.955 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7429999999999999e-06 2024-09-21T15:01:40.955 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.955 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.955 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:40.955 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226330+0000", 2024-09-21T15:01:40.955 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00030963300000000002 2024-09-21T15:01:40.956 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.956 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.956 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:40.956 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226336+0000", 2024-09-21T15:01:40.956 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6340000000000003e-06 2024-09-21T15:01:40.956 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.956 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.956 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:40.956 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255687+0000", 2024-09-21T15:01:40.956 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0293507150000001 2024-09-21T15:01:40.956 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.956 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.956 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:40.956 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255797+0000", 2024-09-21T15:01:40.956 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000109471 2024-09-21T15:01:40.957 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.957 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:40.957 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.957 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.957 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.957 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139884 3.16 3:69f10324:::rbd_data.113664780c85.0000000000000161:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:40.957 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.227573+0000", 2024-09-21T15:01:40.957 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 38.680729466999999, 2024-09-21T15:01:40.957 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0283568759999999, 2024-09-21T15:01:40.957 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:40.957 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:40.957 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:40.957 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:40.957 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:40.957 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139884 2024-09-21T15:01:40.958 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.958 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:40.958 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.958 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:40.958 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227573+0000", 2024-09-21T15:01:40.958 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.958 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.958 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.958 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:40.958 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227573+0000", 2024-09-21T15:01:40.958 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.958 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.958 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.959 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:40.959 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227572+0000", 2024-09-21T15:01:40.959 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:40.959 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.959 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.959 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:40.959 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227575+0000", 2024-09-21T15:01:40.959 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7769999999999998e-06 2024-09-21T15:01:40.959 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.959 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.959 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:40.959 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227581+0000", 2024-09-21T15:01:40.959 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7359999999999998e-06 2024-09-21T15:01:40.959 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.959 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.960 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:40.960 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227585+0000", 2024-09-21T15:01:40.960 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5090000000000004e-06 2024-09-21T15:01:40.960 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.960 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.960 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:40.960 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227753+0000", 2024-09-21T15:01:40.960 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00016761599999999999 2024-09-21T15:01:40.960 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.960 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.960 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:40.960 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227761+0000", 2024-09-21T15:01:40.960 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0579999999999998e-06 2024-09-21T15:01:40.960 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.960 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.960 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:40.961 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255828+0000", 2024-09-21T15:01:40.961 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0280671349999999 2024-09-21T15:01:40.961 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.961 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.961 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:40.961 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255930+0000", 2024-09-21T15:01:40.961 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000101372 2024-09-21T15:01:40.961 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.961 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:40.961 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.961 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.961 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.961 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139913 3.16 3:6f81b46d:::rbd_data.113664780c85.0000000000000074:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:40.961 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.227685+0000", 2024-09-21T15:01:40.961 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 38.680617288000001, 2024-09-21T15:01:40.961 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0283644750000001, 2024-09-21T15:01:40.962 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:40.962 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:40.962 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:40.962 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:40.962 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:40.962 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139913 2024-09-21T15:01:40.962 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.962 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:40.962 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.962 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:40.962 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227685+0000", 2024-09-21T15:01:40.962 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.962 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.962 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.962 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:40.963 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227685+0000", 2024-09-21T15:01:40.963 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.963 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.963 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.963 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:40.963 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227685+0000", 2024-09-21T15:01:40.963 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:40.963 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.963 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.963 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:40.963 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227687+0000", 2024-09-21T15:01:40.963 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2929999999999999e-06 2024-09-21T15:01:40.963 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.963 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.964 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:40.964 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227693+0000", 2024-09-21T15:01:40.964 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1070000000000001e-06 2024-09-21T15:01:40.964 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.964 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.964 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:40.964 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227698+0000", 2024-09-21T15:01:40.964 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9339999999999997e-06 2024-09-21T15:01:40.964 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.964 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.964 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:40.964 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227784+0000", 2024-09-21T15:01:40.964 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6096999999999999e-05 2024-09-21T15:01:40.964 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.965 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.965 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:40.965 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227791+0000", 2024-09-21T15:01:40.965 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9090000000000003e-06 2024-09-21T15:01:40.965 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.965 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.965 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:40.965 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255955+0000", 2024-09-21T15:01:40.965 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0281643579999999 2024-09-21T15:01:40.965 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.965 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.965 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:40.965 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256049+0000", 2024-09-21T15:01:40.965 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4047000000000003e-05 2024-09-21T15:01:40.966 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.966 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:40.966 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.966 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.966 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.966 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139988 3.11 3:88a29fa4:::rbd_data.113664780c85.00000000000000bd:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:40.966 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.229772+0000", 2024-09-21T15:01:40.966 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 38.678529789000002, 2024-09-21T15:01:40.966 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.017967668, 2024-09-21T15:01:40.966 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:40.966 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:40.966 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:40.966 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:40.967 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:40.967 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139988 2024-09-21T15:01:40.967 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.967 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:40.967 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.967 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:40.967 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229772+0000", 2024-09-21T15:01:40.967 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.967 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.967 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.967 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:40.967 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229772+0000", 2024-09-21T15:01:40.967 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.967 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.967 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.968 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:40.968 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229772+0000", 2024-09-21T15:01:40.968 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:40.968 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.968 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.968 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:40.968 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229776+0000", 2024-09-21T15:01:40.968 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6890000000000002e-06 2024-09-21T15:01:40.968 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.968 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.968 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:40.968 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229783+0000", 2024-09-21T15:01:40.968 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2280000000000001e-06 2024-09-21T15:01:40.968 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.968 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.969 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:40.969 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229789+0000", 2024-09-21T15:01:40.969 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1700000000000002e-06 2024-09-21T15:01:40.969 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.969 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.969 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:40.969 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.230106+0000", 2024-09-21T15:01:40.969 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00031700800000000003 2024-09-21T15:01:40.969 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.969 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.969 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:40.969 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.230115+0000", 2024-09-21T15:01:40.969 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4190000000000008e-06 2024-09-21T15:01:40.969 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.969 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.970 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:40.970 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242935+0000", 2024-09-21T15:01:40.970 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0128191449999999 2024-09-21T15:01:40.970 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.970 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.970 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for missing object", 2024-09-21T15:01:40.970 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.243078+0000", 2024-09-21T15:01:40.970 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000142984 2024-09-21T15:01:40.970 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.970 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.970 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:40.970 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.247659+0000", 2024-09-21T15:01:40.970 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0045813030000000001 2024-09-21T15:01:40.970 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.970 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.971 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:40.971 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.247740+0000", 2024-09-21T15:01:40.971 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1216000000000003e-05 2024-09-21T15:01:40.971 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.971 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:40.971 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.971 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.971 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.971 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139994 3.16 3:6b9df8d1:::rbd_data.113664780c85.00000000000000e7:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:40.971 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.231370+0000", 2024-09-21T15:01:40.971 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 38.676931711000002, 2024-09-21T15:01:40.971 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.024795576, 2024-09-21T15:01:40.971 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:40.971 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:40.972 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:40.972 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:40.972 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:40.972 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139994 2024-09-21T15:01:40.972 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.972 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:40.972 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.972 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:40.972 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231370+0000", 2024-09-21T15:01:40.972 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.972 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.972 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.972 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:40.972 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231370+0000", 2024-09-21T15:01:40.972 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.973 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.973 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.973 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:40.973 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231369+0000", 2024-09-21T15:01:40.973 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:40.973 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.973 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.973 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:40.973 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231376+0000", 2024-09-21T15:01:40.973 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5509999999999996e-06 2024-09-21T15:01:40.973 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.973 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.973 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:40.973 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231387+0000", 2024-09-21T15:01:40.973 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.061e-05 2024-09-21T15:01:40.974 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.974 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.974 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:40.974 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231394+0000", 2024-09-21T15:01:40.974 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3799999999999996e-06 2024-09-21T15:01:40.974 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.974 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.974 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:40.974 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231459+0000", 2024-09-21T15:01:40.974 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4716999999999997e-05 2024-09-21T15:01:40.974 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.974 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.974 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:40.974 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231468+0000", 2024-09-21T15:01:40.975 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1479999999999995e-06 2024-09-21T15:01:40.975 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.975 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.975 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:40.975 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256074+0000", 2024-09-21T15:01:40.975 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0246064180000001 2024-09-21T15:01:40.975 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.975 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.975 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:40.975 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256166+0000", 2024-09-21T15:01:40.975 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1862999999999998e-05 2024-09-21T15:01:40.975 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.975 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:40.975 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.975 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.976 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.976 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140012 3.16 3:6bd6dfb6:::rbd_data.113664780c85.0000000000000165:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:40.976 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.231890+0000", 2024-09-21T15:01:40.976 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 38.676411952000002, 2024-09-21T15:01:40.976 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.024408881, 2024-09-21T15:01:40.976 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:40.976 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:40.976 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:40.976 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:40.976 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:40.976 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140012 2024-09-21T15:01:40.976 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.976 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:40.976 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.976 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:40.976 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231890+0000", 2024-09-21T15:01:40.977 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.977 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.977 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.977 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:40.977 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231890+0000", 2024-09-21T15:01:40.977 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.977 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.977 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.977 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:40.977 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231889+0000", 2024-09-21T15:01:40.977 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:40.977 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.977 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.977 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:40.978 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231895+0000", 2024-09-21T15:01:40.978 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9569999999999999e-06 2024-09-21T15:01:40.978 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.978 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.978 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:40.978 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231910+0000", 2024-09-21T15:01:40.978 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4549000000000001e-05 2024-09-21T15:01:40.978 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.978 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.978 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:40.978 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231917+0000", 2024-09-21T15:01:40.978 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5730000000000002e-06 2024-09-21T15:01:40.978 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.978 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.978 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:40.979 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231988+0000", 2024-09-21T15:01:40.979 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0852999999999994e-05 2024-09-21T15:01:40.979 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.979 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.979 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:40.979 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.232000+0000", 2024-09-21T15:01:40.979 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2159e-05 2024-09-21T15:01:40.979 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.979 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.979 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:40.979 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256192+0000", 2024-09-21T15:01:40.979 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.024192073 2024-09-21T15:01:40.979 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.979 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.979 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:40.980 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256299+0000", 2024-09-21T15:01:40.980 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000106733 2024-09-21T15:01:40.980 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.980 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:40.980 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.980 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.980 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.980 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140055 3.16 3:69bb9803:::rbd_data.113664780c85.00000000000000da:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:40.980 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.233922+0000", 2024-09-21T15:01:40.980 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 38.674380423000002, 2024-09-21T15:01:40.980 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.022511768, 2024-09-21T15:01:40.980 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:40.980 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:40.980 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:40.980 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:40.980 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:40.981 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140055 2024-09-21T15:01:40.981 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.981 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:40.981 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.981 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:40.981 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233922+0000", 2024-09-21T15:01:40.981 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.981 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.981 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.981 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:40.981 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233922+0000", 2024-09-21T15:01:40.981 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.981 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.981 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.981 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:40.982 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233921+0000", 2024-09-21T15:01:40.982 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:40.982 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.982 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.982 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:40.982 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233927+0000", 2024-09-21T15:01:40.982 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0360000000000003e-06 2024-09-21T15:01:40.982 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.982 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.982 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:40.982 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233937+0000", 2024-09-21T15:01:40.982 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6439999999999995e-06 2024-09-21T15:01:40.982 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.982 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.983 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:40.983 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233944+0000", 2024-09-21T15:01:40.983 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7710000000000006e-06 2024-09-21T15:01:40.983 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.983 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.983 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:40.983 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233996+0000", 2024-09-21T15:01:40.983 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1597000000000001e-05 2024-09-21T15:01:40.983 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.983 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.983 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:40.983 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.234005+0000", 2024-09-21T15:01:40.983 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5960000000000004e-06 2024-09-21T15:01:40.983 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.983 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.983 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:40.984 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256332+0000", 2024-09-21T15:01:40.984 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0223271949999999 2024-09-21T15:01:40.984 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.984 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.984 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:40.984 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256434+0000", 2024-09-21T15:01:40.984 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010180599999999999 2024-09-21T15:01:40.984 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.984 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:40.984 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.984 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.984 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.984 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140143 3.16 3:6d3ed596:::rbd_data.113664780c85.0000000000000184:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:40.984 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.239496+0000", 2024-09-21T15:01:40.985 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 38.668805894000002, 2024-09-21T15:01:40.985 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.017076576, 2024-09-21T15:01:40.985 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:40.985 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:40.985 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:40.985 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:40.985 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:40.985 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140143 2024-09-21T15:01:40.985 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.985 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:40.985 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.985 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:40.985 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239496+0000", 2024-09-21T15:01:40.985 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.985 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.986 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.986 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:40.986 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239496+0000", 2024-09-21T15:01:40.986 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.986 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.986 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.986 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:40.986 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239496+0000", 2024-09-21T15:01:40.986 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:40.986 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.986 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.986 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:40.986 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239498+0000", 2024-09-21T15:01:40.986 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.509e-06 2024-09-21T15:01:40.987 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.987 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.987 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:40.987 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239504+0000", 2024-09-21T15:01:40.987 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6869999999999999e-06 2024-09-21T15:01:40.987 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.987 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.987 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:40.987 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239509+0000", 2024-09-21T15:01:40.987 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8860000000000003e-06 2024-09-21T15:01:40.987 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.987 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.987 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:40.987 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239572+0000", 2024-09-21T15:01:40.987 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2908000000000001e-05 2024-09-21T15:01:40.988 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.988 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.988 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:40.988 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239580+0000", 2024-09-21T15:01:40.988 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4270000000000008e-06 2024-09-21T15:01:40.988 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.988 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.988 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:40.988 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256457+0000", 2024-09-21T15:01:40.988 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0168770979999999 2024-09-21T15:01:40.988 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.988 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.988 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:40.988 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256573+0000", 2024-09-21T15:01:40.988 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00011538400000000001 2024-09-21T15:01:40.989 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.989 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:40.989 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.989 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.989 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.989 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140163 3.16 3:694e82eb:::rbd_data.113664780c85.0000000000000052:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:40.989 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.240334+0000", 2024-09-21T15:01:40.989 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 38.667968182000003, 2024-09-21T15:01:40.989 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.016365537, 2024-09-21T15:01:40.989 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:40.989 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:40.989 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:40.989 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:40.989 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:40.989 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140163 2024-09-21T15:01:40.990 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.990 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:40.990 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.990 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:40.990 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240334+0000", 2024-09-21T15:01:40.990 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.990 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.990 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.990 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:40.990 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240334+0000", 2024-09-21T15:01:40.990 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.990 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.990 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.990 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:40.990 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240334+0000", 2024-09-21T15:01:40.991 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:40.991 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.991 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.991 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:40.991 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240337+0000", 2024-09-21T15:01:40.991 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.089e-06 2024-09-21T15:01:40.991 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.991 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.991 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:40.991 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240343+0000", 2024-09-21T15:01:40.991 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4420000000000003e-06 2024-09-21T15:01:40.991 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.991 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.991 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:40.992 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240348+0000", 2024-09-21T15:01:40.992 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1039999999999998e-06 2024-09-21T15:01:40.992 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.992 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.992 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:40.992 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240451+0000", 2024-09-21T15:01:40.992 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000102996 2024-09-21T15:01:40.992 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.992 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.992 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:40.992 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240460+0000", 2024-09-21T15:01:40.992 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5369999999999997e-06 2024-09-21T15:01:40.992 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.992 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.992 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:40.992 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256600+0000", 2024-09-21T15:01:40.993 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.016139884 2024-09-21T15:01:40.993 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.993 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.993 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:40.993 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256700+0000", 2024-09-21T15:01:40.993 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9844000000000002e-05 2024-09-21T15:01:40.993 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.993 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:40.993 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.993 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.993 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.993 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140164 3.16 3:68bfdc57:::rbd_data.113664780c85.0000000000000053:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:40.993 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.240358+0000", 2024-09-21T15:01:40.993 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 38.667944488000003, 2024-09-21T15:01:40.994 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.01646094, 2024-09-21T15:01:40.994 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:40.994 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:40.994 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:40.994 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:40.994 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:40.994 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140164 2024-09-21T15:01:40.994 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.994 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:40.994 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.994 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:40.994 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240358+0000", 2024-09-21T15:01:40.994 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.994 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.995 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.995 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:40.995 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240358+0000", 2024-09-21T15:01:40.995 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.995 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.995 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.995 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:40.995 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240357+0000", 2024-09-21T15:01:40.995 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:40.995 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.995 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.995 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:40.995 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240360+0000", 2024-09-21T15:01:40.995 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7089999999999998e-06 2024-09-21T15:01:40.996 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.996 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.996 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:40.996 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240368+0000", 2024-09-21T15:01:40.996 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7270000000000007e-06 2024-09-21T15:01:40.996 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.996 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.996 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:40.996 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240373+0000", 2024-09-21T15:01:40.996 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4060000000000004e-06 2024-09-21T15:01:40.996 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.996 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.996 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:40.996 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240473+0000", 2024-09-21T15:01:40.996 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000100132 2024-09-21T15:01:40.997 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.997 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.997 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:40.997 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240476+0000", 2024-09-21T15:01:40.997 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.332e-06 2024-09-21T15:01:40.997 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.997 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.997 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:40.997 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256724+0000", 2024-09-21T15:01:40.997 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0162471930000001 2024-09-21T15:01:40.997 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.997 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.997 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:40.997 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256819+0000", 2024-09-21T15:01:40.997 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5007999999999996e-05 2024-09-21T15:01:40.998 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.998 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:40.998 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:40.998 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.998 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.998 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140188 3.16 3:6f6217d4:::rbd_data.113664780c85.00000000000000fb:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:40.998 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.240978+0000", 2024-09-21T15:01:40.998 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 38.667323828999997, 2024-09-21T15:01:40.998 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.015983809, 2024-09-21T15:01:40.998 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:40.998 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:40.998 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:40.998 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:40.998 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:40.998 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140188 2024-09-21T15:01:40.999 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.999 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:40.999 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.999 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:40.999 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240978+0000", 2024-09-21T15:01:40.999 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.999 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.999 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.999 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:40.999 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240978+0000", 2024-09-21T15:01:40.999 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:40.999 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:40.999 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:40.999 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:40.999 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240978+0000", 2024-09-21T15:01:40.999 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:41.000 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:41.000 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:41.000 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:41.000 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240981+0000", 2024-09-21T15:01:41.000 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1559999999999999e-06 2024-09-21T15:01:41.000 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:41.000 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:41.000 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:41.000 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240987+0000", 2024-09-21T15:01:41.000 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.229e-06 2024-09-21T15:01:41.000 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:41.000 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:41.000 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:41.000 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240992+0000", 2024-09-21T15:01:41.000 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9710000000000003e-06 2024-09-21T15:01:41.001 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:41.001 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:41.001 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:41.001 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.241361+0000", 2024-09-21T15:01:41.001 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000369236 2024-09-21T15:01:41.001 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:41.001 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:41.001 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:41.001 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.241369+0000", 2024-09-21T15:01:41.001 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6199999999999999e-06 2024-09-21T15:01:41.001 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:41.001 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:41.001 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:41.001 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256844+0000", 2024-09-21T15:01:41.002 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0154751609999999 2024-09-21T15:01:41.002 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:41.002 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:41.002 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:41.002 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256962+0000", 2024-09-21T15:01:41.002 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000117964 2024-09-21T15:01:41.002 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:41.002 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:41.002 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:41.002 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:41.002 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:41.002 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:41.003 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T15:01:41.004 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:01:41.000+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:01:41.104 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:01:41.100+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:01:41.115 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:01:41.115 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:01:41.115 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T15:01:41.116 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:01:41.125 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T15:01:41.205 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:01:41.200+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:01:41.222 INFO:teuthology.orchestra.run.smithi032.stderr:noscrub is set 2024-09-21T15:01:41.233 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:01:41.234 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:01:41.234 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T15:01:41.234 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T15:01:41.234 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:01:41.244 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T15:01:41.256 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T15:01:41.306 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:01:41.306+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:01:41.354 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:01:41.354 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T15:01:41.354 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T15:01:41.354 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T15:01:41.354 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.355 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45854 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:41.355 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159802+0000", 2024-09-21T15:01:41.355 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 151.19113029100001, 2024-09-21T15:01:41.355 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068358938999999994, 2024-09-21T15:01:41.355 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:41.355 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:41.355 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:41.355 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:41.355 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:41.355 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45854 2024-09-21T15:01:41.355 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.355 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:41.355 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.356 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:41.356 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159802+0000", 2024-09-21T15:01:41.356 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:41.356 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.356 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.356 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:41.356 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159802+0000", 2024-09-21T15:01:41.356 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:41.356 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.356 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.356 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:41.356 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159801+0000", 2024-09-21T15:01:41.356 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:41.356 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.357 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.357 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:41.357 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159804+0000", 2024-09-21T15:01:41.357 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0970000000000002e-06 2024-09-21T15:01:41.357 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.357 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.357 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:41.357 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159823+0000", 2024-09-21T15:01:41.357 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8895000000000001e-05 2024-09-21T15:01:41.357 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.357 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.357 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:41.357 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159837+0000", 2024-09-21T15:01:41.357 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3247999999999999e-05 2024-09-21T15:01:41.357 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.358 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.358 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:41.358 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228151+0000", 2024-09-21T15:01:41.358 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068314642999999994 2024-09-21T15:01:41.358 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.358 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.358 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:41.358 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228161+0000", 2024-09-21T15:01:41.358 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4250000000000004e-06 2024-09-21T15:01:41.358 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:41.358 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:41.358 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:41.358 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.359 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.359 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45985 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:41.359 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159897+0000", 2024-09-21T15:01:41.359 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 151.19103478900001, 2024-09-21T15:01:41.359 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068345664, 2024-09-21T15:01:41.359 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:41.359 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:41.359 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:41.359 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:41.359 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:41.359 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45985 2024-09-21T15:01:41.359 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.359 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:41.359 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.359 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:41.360 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159897+0000", 2024-09-21T15:01:41.360 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:41.360 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.360 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.360 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:41.360 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159897+0000", 2024-09-21T15:01:41.360 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:41.360 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.360 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.360 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:41.360 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159897+0000", 2024-09-21T15:01:41.360 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:41.360 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.360 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.361 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:41.361 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159900+0000", 2024-09-21T15:01:41.361 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.954e-06 2024-09-21T15:01:41.361 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.361 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.361 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:41.361 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159913+0000", 2024-09-21T15:01:41.361 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3106e-05 2024-09-21T15:01:41.361 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.361 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.361 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:41.361 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159918+0000", 2024-09-21T15:01:41.361 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5199999999999997e-06 2024-09-21T15:01:41.361 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.362 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.362 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:41.362 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228236+0000", 2024-09-21T15:01:41.362 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068317040999999995 2024-09-21T15:01:41.362 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.362 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.362 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:41.362 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228243+0000", 2024-09-21T15:01:41.362 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3880000000000002e-06 2024-09-21T15:01:41.362 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:41.362 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:41.362 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:41.362 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.362 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.362 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45986 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:41.362 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159924+0000", 2024-09-21T15:01:41.363 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 151.19100849399999, 2024-09-21T15:01:41.363 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068364362999999997, 2024-09-21T15:01:41.363 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:41.363 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:41.363 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:41.363 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:41.363 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:41.363 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45986 2024-09-21T15:01:41.363 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.363 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:41.363 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.363 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:41.363 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159924+0000", 2024-09-21T15:01:41.363 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:41.364 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.364 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.364 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:41.364 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159924+0000", 2024-09-21T15:01:41.364 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:41.364 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.364 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.364 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:41.364 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159923+0000", 2024-09-21T15:01:41.364 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:41.364 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.364 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.364 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:41.364 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159926+0000", 2024-09-21T15:01:41.364 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0510000000000001e-06 2024-09-21T15:01:41.365 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.365 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.365 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:41.365 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159932+0000", 2024-09-21T15:01:41.365 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4160000000000001e-06 2024-09-21T15:01:41.365 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.365 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.365 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:41.365 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159938+0000", 2024-09-21T15:01:41.365 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.383e-06 2024-09-21T15:01:41.365 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.365 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.365 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:41.365 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228275+0000", 2024-09-21T15:01:41.366 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068337007000000005 2024-09-21T15:01:41.366 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.366 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.366 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:41.366 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228288+0000", 2024-09-21T15:01:41.366 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3052e-05 2024-09-21T15:01:41.366 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:41.366 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:41.366 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:41.366 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.366 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.366 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45987 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:41.366 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159944+0000", 2024-09-21T15:01:41.366 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 151.19098826999999, 2024-09-21T15:01:41.367 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068387232000000006, 2024-09-21T15:01:41.367 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:41.367 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:41.367 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:41.367 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:41.367 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:41.367 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45987 2024-09-21T15:01:41.367 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.367 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:41.367 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.367 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:41.367 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159944+0000", 2024-09-21T15:01:41.367 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:41.367 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.367 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.368 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:41.368 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159944+0000", 2024-09-21T15:01:41.368 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:41.368 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.368 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.368 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:41.368 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159943+0000", 2024-09-21T15:01:41.368 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:41.368 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.368 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.368 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:41.368 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159946+0000", 2024-09-21T15:01:41.368 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6309999999999999e-06 2024-09-21T15:01:41.369 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.369 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.369 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:41.369 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159952+0000", 2024-09-21T15:01:41.369 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6540000000000004e-06 2024-09-21T15:01:41.369 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.369 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.369 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:41.369 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159957+0000", 2024-09-21T15:01:41.369 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9280000000000001e-06 2024-09-21T15:01:41.369 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.369 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.369 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:41.369 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228322+0000", 2024-09-21T15:01:41.370 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068365067000000002 2024-09-21T15:01:41.370 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.370 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.370 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:41.370 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228331+0000", 2024-09-21T15:01:41.370 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3270000000000007e-06 2024-09-21T15:01:41.370 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:41.370 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:41.370 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:41.370 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.370 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.370 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:97672 3.4 3:26046d29:::rbd_data.1136bfc79a70.0000000000000001:head [write 0~4718592 [fadvise_sequential+fadvise_nocache],write 5242880~131072 [fadvise_sequential+fadvise_nocache],write 5439488~147456 [fadvise_sequential+fadvise_nocache],write 5652480~229376 [fadvise_sequential+fadvise_nocache],write 5947392~147456 [fadvise_sequential+fadvise_nocache],write 6160384~229376 [fadvise_sequential+fadvise_nocache],write 6455296~147456 [fadvise_sequential+fadvise_nocache],write 6668288~229376 [fadvise_sequential+fadvise_nocache],write 6963200~147456 [fadvise_sequential+fadvise_nocache],write 7176192~147456 [fadvise_sequential+fadvise_nocache],write 7340032~65536 [fadvise_sequential+fadvise_nocache],write 7471104~147456 [fadvise_sequential+fadvise_nocache],write 7684096~147456 [fadvise_sequential+fadvise_nocache],write 7864320~49152 [fadvise_sequential+fadvise_nocache],write 7979008~147456 [fadvise_sequential+fadvise_nocache],write 8192000~147456 [fadvise_sequential+fadvise_nocache],write 8388608~32768 [fadvise_sequential+fadvise_nocache],write 8519680~114688 [fadvise_sequential+fadvise_nocache],write 8732672~114688 [fadvise_sequential+fadvise_nocache],write 8912896~16384 [fadvise_sequential+fadvise_nocache],write 9109504~32768 [fadvise_sequential+fadvise_nocache],write 9322496~32768 [fadvise_sequential+fadvise_nocache],write 9617408~32768 [fadvise_sequential+fadvise_nocache],write 9830400~32768 [fadvise_sequential+fadvise_nocache],write 9961472~98304 [fadvise_sequential+fadvise_nocache],write 10076160~196608 [fadvise_sequential+fadvise_nocache],write 10289152~278528 [fadvise_sequential+fadvise_nocache],write 10584064~196608 [fadvise_sequential+fadvise_nocache],write 10797056~196608 [fadvise_sequential+fadvise_nocache],write 11010048~65536 [fadvise_sequential+fadvise_nocache],write 11091968~196608 [fadvise_sequential+fadvise_nocache],write 11304960~196608 [fadvise_sequential+fadvise_nocache],write 11517952~65536 [fadvise_sequential+fadvise_nocache],write 11616256~98304 [fadvise_sequential+fadvise_nocache],write 11730944~65536 [fadvise_sequential+fadvise_nocache],write 11829248~98304 [fadvise_sequential+fadvise_nocache],write 11943936~65536 [fadvise_sequential+fadvise_nocache],write 12042240~49152 [fadvise_sequential+fadvise_nocache],write 12124160~98304 [fadvise_sequential+fadvise_nocache],write 12238848~65536 [fadvise_sequential+fadvise_nocache],write 12337152~98304 [fadvise_sequential+fadvise_nocache],write 12451840~65536 [fadvise_sequential+fadvise_nocache],write 12550144~32768 [fadvise_sequential+fadvise_nocache],write 12632064~98304 [fadvise_sequential+fadvise_nocache],write 12845056~98304 [fadvise_sequential+fadvise_nocache],write 13058048~49152 [fadvise_sequential+fadvise_nocache],write 13139968~98304 [fadvise_sequential+fadvise_nocache],write 13352960~98304 [fadvise_sequential+fadvise_nocache],write 13565952~65536 [fadvise_sequential+fadvise_nocache],write 13647872~98304 [fadvise_sequential+fadvise_nocache],write 13860864~98304 [fadvise_sequential+fadvise_nocache],write 14073856~81920 [fadvise_sequential+fadvise_nocache],write 15269888~81920 [fadvise_sequential+fadvise_nocache],write 15482880~81920 [fadvise_sequential+fadvise_nocache],write 15695872~32768 [fadvise_sequential+fadvise_nocache],write 15777792~81920 [fadvise_sequential+fadvise_nocache],write 15990784~81920 [fadvise_sequential+fadvise_nocache],write 16203776~49152 [fadvise_sequential+fadvise_nocache],write 16285696~81920 [fadvise_sequential+fadvise_nocache],write 16498688~81920 [fadvise_sequential+fadvise_nocache],write 16711680~65536 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:01:41.370 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.298669+0000", 2024-09-21T15:01:41.371 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 96.052263203999999, 2024-09-21T15:01:41.371 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.10282358900000001, 2024-09-21T15:01:41.371 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:41.371 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:41.371 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:41.371 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:41.371 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:41.371 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97672 2024-09-21T15:01:41.371 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.371 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:41.371 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.371 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:41.371 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.298669+0000", 2024-09-21T15:01:41.371 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:41.372 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.372 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.372 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:41.372 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.298669+0000", 2024-09-21T15:01:41.372 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:41.372 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.372 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.372 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:41.372 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.298667+0000", 2024-09-21T15:01:41.372 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:01:41.372 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.372 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.372 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:41.372 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.327458+0000", 2024-09-21T15:01:41.372 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.028790754000000002 2024-09-21T15:01:41.373 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.373 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.373 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:41.373 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.327492+0000", 2024-09-21T15:01:41.373 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3806999999999997e-05 2024-09-21T15:01:41.373 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.373 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.373 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:41.373 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.327507+0000", 2024-09-21T15:01:41.373 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5112e-05 2024-09-21T15:01:41.373 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.373 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.373 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:41.373 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.327597+0000", 2024-09-21T15:01:41.374 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9938999999999994e-05 2024-09-21T15:01:41.374 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.374 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.374 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:41.374 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.327927+0000", 2024-09-21T15:01:41.374 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000329766 2024-09-21T15:01:41.374 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.374 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.374 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T15:01:41.374 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.330735+0000", 2024-09-21T15:01:41.374 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0028086869999999998 2024-09-21T15:01:41.374 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.374 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.374 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:01:41.375 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.378299+0000", 2024-09-21T15:01:41.375 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.047563495999999997 2024-09-21T15:01:41.375 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.375 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.375 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:01:41.375 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.401248+0000", 2024-09-21T15:01:41.375 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.022949311 2024-09-21T15:01:41.375 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.375 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.375 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:01:41.375 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.401446+0000", 2024-09-21T15:01:41.375 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019831199999999999 2024-09-21T15:01:41.375 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.375 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.375 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:41.376 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.401492+0000", 2024-09-21T15:01:41.376 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6109000000000002e-05 2024-09-21T15:01:41.376 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:41.376 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:41.376 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:41.376 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.376 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.376 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139769 3.1e 3:793dcf44:::rbd_data.113664780c85.0000000000000000:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:41.376 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.221432+0000", 2024-09-21T15:01:41.376 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 39.129500335000003, 2024-09-21T15:01:41.376 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.035363362, 2024-09-21T15:01:41.376 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:41.376 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:41.376 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:41.377 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:41.377 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:41.377 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139769 2024-09-21T15:01:41.377 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.377 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:41.377 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.377 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:41.377 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221432+0000", 2024-09-21T15:01:41.377 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:41.377 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.377 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.377 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:41.377 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221432+0000", 2024-09-21T15:01:41.377 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:41.378 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.378 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.378 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:41.378 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221430+0000", 2024-09-21T15:01:41.378 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:01:41.378 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.378 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.378 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:41.378 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221438+0000", 2024-09-21T15:01:41.378 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2630000000000003e-06 2024-09-21T15:01:41.378 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.378 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.378 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:41.378 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221458+0000", 2024-09-21T15:01:41.379 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0071000000000001e-05 2024-09-21T15:01:41.379 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.379 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.379 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:41.379 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221467+0000", 2024-09-21T15:01:41.379 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.816e-06 2024-09-21T15:01:41.379 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.379 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.379 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:41.379 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221572+0000", 2024-09-21T15:01:41.379 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000105166 2024-09-21T15:01:41.379 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.379 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.379 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:41.380 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221589+0000", 2024-09-21T15:01:41.380 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7391e-05 2024-09-21T15:01:41.380 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.380 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.380 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:41.380 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.255741+0000", 2024-09-21T15:01:41.380 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0341519690000001 2024-09-21T15:01:41.380 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.380 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.380 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:41.380 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.256353+0000", 2024-09-21T15:01:41.380 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000611544 2024-09-21T15:01:41.380 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.380 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.380 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:41.381 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.256795+0000", 2024-09-21T15:01:41.381 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00044208700000000002 2024-09-21T15:01:41.381 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:41.381 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:41.381 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:41.381 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.381 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.381 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139773 3.1e 3:7fa3054f:::rbd_data.113664780c85.000000000000001c:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:41.381 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.221517+0000", 2024-09-21T15:01:41.381 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 39.129415291000001, 2024-09-21T15:01:41.381 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0367423140000001, 2024-09-21T15:01:41.381 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:41.381 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:41.382 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:41.382 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:41.382 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:41.382 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139773 2024-09-21T15:01:41.382 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.382 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:41.382 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.382 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:41.382 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221517+0000", 2024-09-21T15:01:41.382 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:41.382 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.382 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.382 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:41.382 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221517+0000", 2024-09-21T15:01:41.382 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:41.383 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.383 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.383 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:41.383 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221516+0000", 2024-09-21T15:01:41.383 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:41.383 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.383 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.383 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:41.383 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221520+0000", 2024-09-21T15:01:41.383 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1030000000000004e-06 2024-09-21T15:01:41.383 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.383 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.383 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:41.383 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221566+0000", 2024-09-21T15:01:41.384 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5512000000000001e-05 2024-09-21T15:01:41.384 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.384 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.384 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:41.384 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221575+0000", 2024-09-21T15:01:41.384 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9630000000000004e-06 2024-09-21T15:01:41.384 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.384 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.384 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:41.384 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221650+0000", 2024-09-21T15:01:41.384 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5531999999999994e-05 2024-09-21T15:01:41.384 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.384 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.384 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:41.385 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221661+0000", 2024-09-21T15:01:41.385 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0878e-05 2024-09-21T15:01:41.385 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.385 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.385 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:41.385 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.257513+0000", 2024-09-21T15:01:41.385 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0358513089999999 2024-09-21T15:01:41.385 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.385 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.385 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:41.385 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.257885+0000", 2024-09-21T15:01:41.385 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00037274299999999998 2024-09-21T15:01:41.385 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.385 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.386 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:41.386 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.258259+0000", 2024-09-21T15:01:41.386 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000373829 2024-09-21T15:01:41.386 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:41.386 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:41.386 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:41.386 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.386 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.386 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139789 3.1e 3:7c803367:::rbd_data.113664780c85.000000000000008c:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:41.386 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.221656+0000", 2024-09-21T15:01:41.386 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 39.129275659999998, 2024-09-21T15:01:41.386 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.037139595, 2024-09-21T15:01:41.387 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:41.387 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:41.387 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:41.387 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:41.387 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:41.387 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139789 2024-09-21T15:01:41.387 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.387 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:41.387 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.387 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:41.387 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221656+0000", 2024-09-21T15:01:41.387 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:41.387 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.387 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.388 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:41.388 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221656+0000", 2024-09-21T15:01:41.388 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:41.388 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.388 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.388 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:41.388 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221656+0000", 2024-09-21T15:01:41.388 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:41.388 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.388 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.388 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:41.388 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221660+0000", 2024-09-21T15:01:41.388 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0369999999999996e-06 2024-09-21T15:01:41.388 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.388 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.389 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:41.389 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221668+0000", 2024-09-21T15:01:41.389 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3440000000000001e-06 2024-09-21T15:01:41.389 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.389 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.389 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:41.389 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221674+0000", 2024-09-21T15:01:41.389 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0920000000000003e-06 2024-09-21T15:01:41.389 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.389 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.389 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:41.389 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221787+0000", 2024-09-21T15:01:41.389 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000112289 2024-09-21T15:01:41.389 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.390 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.390 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:41.390 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221798+0000", 2024-09-21T15:01:41.390 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1419e-05 2024-09-21T15:01:41.390 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.390 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.390 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:41.390 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.258642+0000", 2024-09-21T15:01:41.390 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.036843851 2024-09-21T15:01:41.390 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.390 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.390 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:41.390 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.258796+0000", 2024-09-21T15:01:41.390 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015414199999999999 2024-09-21T15:01:41.391 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:41.391 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:41.391 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:41.391 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.391 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.391 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139832 3.1e 3:7b675eeb:::rbd_data.113664780c85.00000000000001b3:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:41.391 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.223976+0000", 2024-09-21T15:01:41.391 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 39.126955926999997, 2024-09-21T15:01:41.391 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0350267980000001, 2024-09-21T15:01:41.391 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:41.391 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:41.391 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:41.391 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:41.391 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:41.391 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139832 2024-09-21T15:01:41.392 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.392 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:41.392 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.392 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:41.392 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223976+0000", 2024-09-21T15:01:41.392 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:41.392 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.392 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.392 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:41.392 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223976+0000", 2024-09-21T15:01:41.392 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:41.392 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.392 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.392 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:41.393 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223976+0000", 2024-09-21T15:01:41.393 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:41.393 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.393 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.393 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:41.393 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223981+0000", 2024-09-21T15:01:41.393 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1560000000000003e-06 2024-09-21T15:01:41.393 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.393 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.393 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:41.393 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223999+0000", 2024-09-21T15:01:41.393 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8808e-05 2024-09-21T15:01:41.393 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.393 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.393 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:41.394 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224008+0000", 2024-09-21T15:01:41.394 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7269999999999996e-06 2024-09-21T15:01:41.394 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.394 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.394 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:41.394 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224208+0000", 2024-09-21T15:01:41.394 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000199614 2024-09-21T15:01:41.394 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.394 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.394 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:41.394 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224218+0000", 2024-09-21T15:01:41.394 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0209e-05 2024-09-21T15:01:41.394 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.395 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.395 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:41.395 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.258839+0000", 2024-09-21T15:01:41.395 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.034621083 2024-09-21T15:01:41.395 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.395 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.395 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:41.395 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.259003+0000", 2024-09-21T15:01:41.395 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016379 2024-09-21T15:01:41.395 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:41.395 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:41.395 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:41.395 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.395 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.395 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139900 3.1e 3:7b440264:::rbd_data.113664780c85.0000000000000013:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:41.396 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.225748+0000", 2024-09-21T15:01:41.396 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 39.125183925999998, 2024-09-21T15:01:41.396 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0339966009999999, 2024-09-21T15:01:41.396 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:41.396 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:41.396 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:41.396 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:41.396 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:41.396 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139900 2024-09-21T15:01:41.396 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.396 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:41.396 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.396 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:41.396 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.225748+0000", 2024-09-21T15:01:41.397 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:41.397 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.397 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.397 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:41.397 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.225748+0000", 2024-09-21T15:01:41.397 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:41.397 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.397 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.397 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:41.397 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.225746+0000", 2024-09-21T15:01:41.397 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:41.397 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.397 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.397 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:41.398 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.225757+0000", 2024-09-21T15:01:41.398 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.066e-05 2024-09-21T15:01:41.398 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.398 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.398 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:41.398 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.225775+0000", 2024-09-21T15:01:41.398 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7631000000000001e-05 2024-09-21T15:01:41.398 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.398 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.398 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:41.398 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.225788+0000", 2024-09-21T15:01:41.398 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3046e-05 2024-09-21T15:01:41.398 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.398 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.399 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:41.399 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.225881+0000", 2024-09-21T15:01:41.399 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2987999999999998e-05 2024-09-21T15:01:41.399 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.399 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.399 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:41.399 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.225893+0000", 2024-09-21T15:01:41.399 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2398999999999999e-05 2024-09-21T15:01:41.399 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.399 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.399 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:41.399 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.259207+0000", 2024-09-21T15:01:41.399 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.033313696 2024-09-21T15:01:41.399 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.399 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.400 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:41.400 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.259452+0000", 2024-09-21T15:01:41.400 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00024495700000000002 2024-09-21T15:01:41.400 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.400 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.400 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:41.400 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.259745+0000", 2024-09-21T15:01:41.400 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000292868 2024-09-21T15:01:41.400 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:41.400 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:41.400 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:41.400 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.400 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.400 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140008 3.1e 3:7cd8b476:::rbd_data.113664780c85.0000000000000149:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:41.401 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.229555+0000", 2024-09-21T15:01:41.401 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 39.121377062000001, 2024-09-21T15:01:41.401 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0304914199999999, 2024-09-21T15:01:41.401 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:41.401 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:41.401 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:41.401 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:41.401 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:41.401 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140008 2024-09-21T15:01:41.401 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.401 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:41.401 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.401 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:41.401 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.229555+0000", 2024-09-21T15:01:41.401 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:41.402 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.402 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.402 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:41.402 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.229555+0000", 2024-09-21T15:01:41.402 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:41.402 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.402 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.402 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:41.402 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.229554+0000", 2024-09-21T15:01:41.402 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:41.402 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.402 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.402 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:41.403 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.229559+0000", 2024-09-21T15:01:41.403 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1059999999999999e-06 2024-09-21T15:01:41.403 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.403 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.403 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:41.403 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.229570+0000", 2024-09-21T15:01:41.403 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0465000000000001e-05 2024-09-21T15:01:41.403 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.403 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.403 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:41.403 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.229581+0000", 2024-09-21T15:01:41.403 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0958000000000001e-05 2024-09-21T15:01:41.403 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.403 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.403 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:41.404 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.229660+0000", 2024-09-21T15:01:41.404 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9103999999999993e-05 2024-09-21T15:01:41.404 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.404 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.404 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:41.404 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.229675+0000", 2024-09-21T15:01:41.404 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5622999999999999e-05 2024-09-21T15:01:41.404 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.404 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.404 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:41.404 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.259816+0000", 2024-09-21T15:01:41.404 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.030140595 2024-09-21T15:01:41.404 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.404 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.405 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:41.405 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.260046+0000", 2024-09-21T15:01:41.405 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00023045300000000001 2024-09-21T15:01:41.405 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:41.405 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:41.405 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:41.405 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.405 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.405 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140037 3.1e 3:7a4585d9:::rbd_data.113664780c85.000000000000005c:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:41.405 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.230439+0000", 2024-09-21T15:01:41.405 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 39.120492573999996, 2024-09-21T15:01:41.405 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.030277613, 2024-09-21T15:01:41.405 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:41.406 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:41.406 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:41.406 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:41.406 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:41.406 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140037 2024-09-21T15:01:41.406 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.406 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:41.406 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.406 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:41.406 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.230439+0000", 2024-09-21T15:01:41.406 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:41.406 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.406 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.406 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:41.407 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.230439+0000", 2024-09-21T15:01:41.407 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:41.407 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.407 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.407 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:41.407 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.230438+0000", 2024-09-21T15:01:41.407 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:41.407 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.407 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.407 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:41.407 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.230447+0000", 2024-09-21T15:01:41.407 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6270000000000006e-06 2024-09-21T15:01:41.407 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.407 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.408 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:41.408 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.230464+0000", 2024-09-21T15:01:41.408 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7394e-05 2024-09-21T15:01:41.408 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.408 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.408 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:41.408 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.230472+0000", 2024-09-21T15:01:41.408 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0360000000000006e-06 2024-09-21T15:01:41.408 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.408 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.408 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:41.408 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.230567+0000", 2024-09-21T15:01:41.408 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4648000000000006e-05 2024-09-21T15:01:41.408 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.409 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.409 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:41.409 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.230581+0000", 2024-09-21T15:01:41.409 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3799e-05 2024-09-21T15:01:41.409 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.409 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.409 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:41.409 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.260095+0000", 2024-09-21T15:01:41.409 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.029514153 2024-09-21T15:01:41.409 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.409 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.409 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:41.409 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.260356+0000", 2024-09-21T15:01:41.409 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00026118400000000003 2024-09-21T15:01:41.410 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.410 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.410 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:41.410 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.260717+0000", 2024-09-21T15:01:41.410 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00036124799999999999 2024-09-21T15:01:41.410 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:41.410 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:41.410 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:41.410 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.410 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.410 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140061 3.1e 3:7fdd08f8:::rbd_data.113664780c85.0000000000000104:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:41.410 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.232026+0000", 2024-09-21T15:01:41.410 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 39.118906244000001, 2024-09-21T15:01:41.410 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0290620450000001, 2024-09-21T15:01:41.411 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:41.411 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:41.411 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:41.411 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:41.411 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:41.411 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140061 2024-09-21T15:01:41.411 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.411 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:41.411 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.411 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:41.411 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232026+0000", 2024-09-21T15:01:41.411 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:41.411 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.411 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.412 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:41.412 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232026+0000", 2024-09-21T15:01:41.412 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:41.412 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.412 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.412 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:41.412 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232024+0000", 2024-09-21T15:01:41.412 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:01:41.412 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.412 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.412 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:41.412 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232034+0000", 2024-09-21T15:01:41.412 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0421000000000001e-05 2024-09-21T15:01:41.412 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.412 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.413 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:41.413 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232058+0000", 2024-09-21T15:01:41.413 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3944e-05 2024-09-21T15:01:41.413 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.413 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.413 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:41.413 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232071+0000", 2024-09-21T15:01:41.413 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2816999999999999e-05 2024-09-21T15:01:41.413 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.413 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.413 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:41.413 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232180+0000", 2024-09-21T15:01:41.413 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000109323 2024-09-21T15:01:41.413 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.413 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.414 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:41.414 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232199+0000", 2024-09-21T15:01:41.414 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8606000000000001e-05 2024-09-21T15:01:41.414 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.414 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.414 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:41.414 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.260821+0000", 2024-09-21T15:01:41.414 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0286222270000001 2024-09-21T15:01:41.414 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.414 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.414 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:41.414 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.261088+0000", 2024-09-21T15:01:41.414 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00026663799999999998 2024-09-21T15:01:41.414 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:41.415 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:41.415 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:41.415 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.415 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.415 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140090 3.1e 3:79f99893:::rbd_data.113664780c85.000000000000000b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:41.415 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.233668+0000", 2024-09-21T15:01:41.415 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 39.117264378999998, 2024-09-21T15:01:41.415 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0281047409999999, 2024-09-21T15:01:41.415 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:41.415 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:41.415 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:41.415 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:41.415 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:41.415 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140090 2024-09-21T15:01:41.416 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.416 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:41.416 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.416 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:41.416 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233668+0000", 2024-09-21T15:01:41.416 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:41.416 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.416 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.416 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:41.416 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233668+0000", 2024-09-21T15:01:41.416 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:41.416 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.416 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.416 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:41.417 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233666+0000", 2024-09-21T15:01:41.417 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:41.417 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.417 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.417 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:41.417 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233673+0000", 2024-09-21T15:01:41.417 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1840000000000002e-06 2024-09-21T15:01:41.417 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.417 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.417 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:41.417 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233689+0000", 2024-09-21T15:01:41.417 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5366000000000002e-05 2024-09-21T15:01:41.417 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.417 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.418 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:41.418 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233697+0000", 2024-09-21T15:01:41.418 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7679999999999997e-06 2024-09-21T15:01:41.418 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.418 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.418 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:41.418 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233772+0000", 2024-09-21T15:01:41.418 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4832000000000005e-05 2024-09-21T15:01:41.418 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.418 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.418 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:41.418 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233790+0000", 2024-09-21T15:01:41.418 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7297999999999999e-05 2024-09-21T15:01:41.419 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.419 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.419 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:41.419 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.261145+0000", 2024-09-21T15:01:41.419 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.027355381 2024-09-21T15:01:41.419 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.419 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.419 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:41.419 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.261466+0000", 2024-09-21T15:01:41.419 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00032105300000000002 2024-09-21T15:01:41.419 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.419 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.419 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:41.419 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.261772+0000", 2024-09-21T15:01:41.420 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00030635100000000001 2024-09-21T15:01:41.420 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:41.420 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:41.420 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:41.420 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.420 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.420 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140128 3.1e 3:78baf82e:::rbd_data.113664780c85.0000000000000115:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:41.420 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.235029+0000", 2024-09-21T15:01:41.420 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 39.115903125000003, 2024-09-21T15:01:41.420 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.02698942, 2024-09-21T15:01:41.420 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:41.420 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:41.420 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:41.421 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:41.421 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:41.421 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140128 2024-09-21T15:01:41.421 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.421 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:41.421 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.421 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:41.421 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235029+0000", 2024-09-21T15:01:41.421 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:41.421 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.421 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.421 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:41.421 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235029+0000", 2024-09-21T15:01:41.421 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:41.421 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.422 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.422 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:41.422 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235028+0000", 2024-09-21T15:01:41.422 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:41.422 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.422 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.422 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:41.422 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235032+0000", 2024-09-21T15:01:41.422 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0579999999999999e-06 2024-09-21T15:01:41.422 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.422 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.422 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:41.422 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235040+0000", 2024-09-21T15:01:41.423 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2099999999999993e-06 2024-09-21T15:01:41.423 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.423 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.423 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:41.423 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235047+0000", 2024-09-21T15:01:41.423 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1099999999999999e-06 2024-09-21T15:01:41.423 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.423 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.423 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:41.423 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235123+0000", 2024-09-21T15:01:41.423 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6835999999999994e-05 2024-09-21T15:01:41.423 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.423 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.423 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:41.423 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235136+0000", 2024-09-21T15:01:41.424 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2962000000000001e-05 2024-09-21T15:01:41.424 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.424 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.424 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:41.424 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.261827+0000", 2024-09-21T15:01:41.424 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0266906149999999 2024-09-21T15:01:41.424 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.424 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.424 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:41.424 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.262018+0000", 2024-09-21T15:01:41.424 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019132400000000001 2024-09-21T15:01:41.424 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:41.424 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:41.424 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:41.425 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.425 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.425 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140217 3.1e 3:793dcf44:::rbd_data.113664780c85.0000000000000000:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:41.425 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.235497+0000", 2024-09-21T15:01:41.425 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 39.115434825999998, 2024-09-21T15:01:41.425 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0269053020000001, 2024-09-21T15:01:41.425 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:41.425 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:41.425 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:41.425 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:41.425 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:41.425 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140217 2024-09-21T15:01:41.425 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.425 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:41.425 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.426 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:41.426 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235497+0000", 2024-09-21T15:01:41.426 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:41.426 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.426 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.426 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:41.426 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235497+0000", 2024-09-21T15:01:41.426 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:41.426 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.426 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.426 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:41.426 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235496+0000", 2024-09-21T15:01:41.426 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:41.426 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.427 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.427 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:41.427 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235503+0000", 2024-09-21T15:01:41.427 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8460000000000002e-06 2024-09-21T15:01:41.427 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.427 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.427 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:41.427 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235514+0000", 2024-09-21T15:01:41.427 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0576e-05 2024-09-21T15:01:41.427 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.427 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.427 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:41.427 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235522+0000", 2024-09-21T15:01:41.427 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2640000000000008e-06 2024-09-21T15:01:41.428 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.428 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.428 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:41.428 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235579+0000", 2024-09-21T15:01:41.428 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6842999999999998e-05 2024-09-21T15:01:41.428 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.428 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.428 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:41.428 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235590+0000", 2024-09-21T15:01:41.428 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1194e-05 2024-09-21T15:01:41.428 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.428 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.428 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:41.428 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.262119+0000", 2024-09-21T15:01:41.428 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0265294540000001 2024-09-21T15:01:41.429 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.429 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.429 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:41.429 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.262241+0000", 2024-09-21T15:01:41.429 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012182 2024-09-21T15:01:41.429 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.429 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.429 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:41.429 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.262403+0000", 2024-09-21T15:01:41.429 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000161442 2024-09-21T15:01:41.429 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:41.429 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:41.429 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:41.429 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.429 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.430 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140221 3.1e 3:7fa3054f:::rbd_data.113664780c85.000000000000001c:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:41.430 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.235592+0000", 2024-09-21T15:01:41.430 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 39.115339964999997, 2024-09-21T15:01:41.430 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.027322012, 2024-09-21T15:01:41.430 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:41.430 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:41.430 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:41.430 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:41.430 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:41.430 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140221 2024-09-21T15:01:41.430 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.430 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:41.430 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.430 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:41.431 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235592+0000", 2024-09-21T15:01:41.431 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:41.431 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.431 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.431 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:41.431 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235592+0000", 2024-09-21T15:01:41.431 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:41.431 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.431 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.431 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:41.431 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235591+0000", 2024-09-21T15:01:41.431 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:41.431 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.431 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.432 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:41.432 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235596+0000", 2024-09-21T15:01:41.432 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4209999999999997e-06 2024-09-21T15:01:41.432 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.432 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.432 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:41.432 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235603+0000", 2024-09-21T15:01:41.432 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.909e-06 2024-09-21T15:01:41.432 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.432 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.432 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:41.432 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235609+0000", 2024-09-21T15:01:41.432 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3539999999999999e-06 2024-09-21T15:01:41.432 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.433 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.433 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:41.433 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236013+0000", 2024-09-21T15:01:41.433 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00040399799999999998 2024-09-21T15:01:41.433 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.433 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.433 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:41.433 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236020+0000", 2024-09-21T15:01:41.433 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6699999999999994e-06 2024-09-21T15:01:41.433 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.433 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.433 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:41.433 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.262452+0000", 2024-09-21T15:01:41.433 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.02643145 2024-09-21T15:01:41.433 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.434 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.434 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:41.434 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.262624+0000", 2024-09-21T15:01:41.434 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000172206 2024-09-21T15:01:41.434 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.434 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.434 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:41.434 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.262914+0000", 2024-09-21T15:01:41.434 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00028994600000000002 2024-09-21T15:01:41.434 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:41.434 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:41.434 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:41.434 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.435 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.435 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140237 3.1e 3:7c803367:::rbd_data.113664780c85.000000000000008c:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:41.435 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.237113+0000", 2024-09-21T15:01:41.435 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 39.113819458000002, 2024-09-21T15:01:41.435 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0260336299999999, 2024-09-21T15:01:41.435 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:41.435 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:41.435 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:41.435 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:41.435 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:41.435 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140237 2024-09-21T15:01:41.435 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.435 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:41.435 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.436 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:41.436 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237113+0000", 2024-09-21T15:01:41.436 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:41.436 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.436 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.436 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:41.436 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237113+0000", 2024-09-21T15:01:41.436 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:41.436 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.436 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.436 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:41.436 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237112+0000", 2024-09-21T15:01:41.436 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:41.436 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.437 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.437 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:41.437 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237118+0000", 2024-09-21T15:01:41.437 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8930000000000002e-06 2024-09-21T15:01:41.437 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.437 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.437 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:41.437 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237126+0000", 2024-09-21T15:01:41.437 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6729999999999998e-06 2024-09-21T15:01:41.437 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.437 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.437 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:41.437 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237135+0000", 2024-09-21T15:01:41.437 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0929999999999992e-06 2024-09-21T15:01:41.437 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.438 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.438 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:41.438 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237205+0000", 2024-09-21T15:01:41.438 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9697999999999996e-05 2024-09-21T15:01:41.438 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.438 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.438 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:41.438 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237218+0000", 2024-09-21T15:01:41.438 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2524000000000001e-05 2024-09-21T15:01:41.438 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.438 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.438 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:41.438 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.262965+0000", 2024-09-21T15:01:41.439 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0257478630000001 2024-09-21T15:01:41.439 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.439 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.439 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:41.439 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.263146+0000", 2024-09-21T15:01:41.439 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018074599999999999 2024-09-21T15:01:41.439 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:41.439 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:41.439 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:41.439 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.439 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.439 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140280 3.1e 3:7b675eeb:::rbd_data.113664780c85.00000000000001b3:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:41.439 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.237477+0000", 2024-09-21T15:01:41.439 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 39.113455195999997, 2024-09-21T15:01:41.439 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.026437329, 2024-09-21T15:01:41.440 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:41.440 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:41.440 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:41.440 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:41.440 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:41.440 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140280 2024-09-21T15:01:41.440 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.440 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:41.440 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.440 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:41.440 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237477+0000", 2024-09-21T15:01:41.440 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:41.440 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.440 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.441 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:41.441 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237477+0000", 2024-09-21T15:01:41.441 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:41.441 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.441 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.441 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:41.441 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237476+0000", 2024-09-21T15:01:41.441 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:41.441 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.441 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.441 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:41.441 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237480+0000", 2024-09-21T15:01:41.441 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7570000000000002e-06 2024-09-21T15:01:41.441 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.441 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.442 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:41.442 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237490+0000", 2024-09-21T15:01:41.442 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3170000000000008e-06 2024-09-21T15:01:41.442 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.442 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.442 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:41.442 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237496+0000", 2024-09-21T15:01:41.442 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7150000000000001e-06 2024-09-21T15:01:41.442 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.442 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.442 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:41.442 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237559+0000", 2024-09-21T15:01:41.442 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2844999999999999e-05 2024-09-21T15:01:41.442 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.443 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.443 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:41.443 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237570+0000", 2024-09-21T15:01:41.443 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1253000000000001e-05 2024-09-21T15:01:41.443 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.443 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.443 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:41.443 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.263203+0000", 2024-09-21T15:01:41.443 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0256330339999999 2024-09-21T15:01:41.443 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.443 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.443 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:41.443 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.263914+0000", 2024-09-21T15:01:41.444 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00071080899999999996 2024-09-21T15:01:41.444 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:41.444 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:41.444 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:41.444 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.444 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.444 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140348 3.1e 3:7b440264:::rbd_data.113664780c85.0000000000000013:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:41.444 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.241762+0000", 2024-09-21T15:01:41.444 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 39.109169925000003, 2024-09-21T15:01:41.444 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0221355759999999, 2024-09-21T15:01:41.444 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:41.444 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:41.444 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:41.444 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:41.444 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:41.445 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140348 2024-09-21T15:01:41.445 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.445 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:41.445 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.445 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:41.445 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.241762+0000", 2024-09-21T15:01:41.445 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:41.445 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.445 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.445 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:41.445 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.241762+0000", 2024-09-21T15:01:41.445 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:41.445 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.445 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.446 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:41.446 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.241761+0000", 2024-09-21T15:01:41.446 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:41.446 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.446 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.446 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:41.446 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.241767+0000", 2024-09-21T15:01:41.446 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4219999999999997e-06 2024-09-21T15:01:41.446 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.446 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.446 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:41.446 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.241783+0000", 2024-09-21T15:01:41.446 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5959000000000001e-05 2024-09-21T15:01:41.446 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.447 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.447 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:41.447 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.241793+0000", 2024-09-21T15:01:41.447 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3999999999999998e-06 2024-09-21T15:01:41.447 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.447 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.447 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:41.447 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.241894+0000", 2024-09-21T15:01:41.447 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000100976 2024-09-21T15:01:41.447 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.447 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.447 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:41.447 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.241907+0000", 2024-09-21T15:01:41.447 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3281e-05 2024-09-21T15:01:41.448 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.448 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.448 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:41.448 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.263430+0000", 2024-09-21T15:01:41.448 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0215229379999999 2024-09-21T15:01:41.448 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.448 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.448 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:41.448 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.263604+0000", 2024-09-21T15:01:41.448 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017368700000000001 2024-09-21T15:01:41.448 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:41.448 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:41.448 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:41.448 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.263898+0000", 2024-09-21T15:01:41.449 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00029406899999999999 2024-09-21T15:01:41.449 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:41.449 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:41.449 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:41.449 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:41.449 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:41.449 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:01:41.450 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T15:01:41.450 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T15:01:41.450 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T15:01:41.451 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T15:01:41.451 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:01:41.444+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:01:41.469 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T15:01:41.493 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T15:01:41.518 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T15:01:41.541 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T15:01:41.541 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:01:41.541 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:01:41.541 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:01:41.542 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:01:41.542 INFO:teuthology.orchestra.run.smithi032.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-21T15:01:41.542 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:01:41.542 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:01:41.542 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:01:41.546 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-21T15:01:41.547 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-21T15:01:41.551 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:01:41.548+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:01:41.611 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:41.611 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:41.621 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T15:01:41.652 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:01:41.650+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:01:41.743 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:41.743 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:41.753 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:01:41.750+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:01:41.755 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T15:01:41.853 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:01:41.848+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:01:41.858 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:41.858 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:41.868 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T15:01:41.954 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:01:41.948+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:01:41.967 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:41.967 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:41.977 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T15:01:42.055 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:01:42.052+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:01:42.084 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:42.085 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:42.095 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T15:01:42.156 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:01:42.152+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:01:42.254 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:42.255 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:42.257 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:01:42.254+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:01:42.265 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T15:01:42.358 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:01:42.352+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:01:42.391 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:42.391 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:42.402 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T15:01:42.459 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:01:42.456+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:01:42.461 INFO:tasks.thrashosds.thrasher:No PGs found for osd.4 2024-09-21T15:01:42.507 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:42.507 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:42.517 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T15:01:42.560 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:01:42.556+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:01:42.622 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:42.622 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:42.632 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T15:01:42.660 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 has been restored 2024-09-21T15:01:42.661 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:01:42.656+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:01:42.735 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:42.735 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:42.746 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T15:01:42.761 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:01:42.756+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:01:42.856 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:42.856 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:42.862 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:01:42.862+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:01:42.866 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T15:01:42.963 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:01:42.962+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:01:42.969 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:42.970 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:42.979 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T15:01:43.064 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:01:43.060+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:01:43.130 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:43.131 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:43.140 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T15:01:43.165 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:01:43.160+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:01:43.237 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T15:01:43.253 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:43.254 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:43.266 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:01:43.266+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:01:43.266 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T15:01:43.367 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:01:43.366+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:01:43.429 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:43.429 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:43.440 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T15:01:43.485 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:01:43.486+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:01:43.542 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:43.543 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:43.547 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T15:01:43.553 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T15:01:43.568 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:01:43.564+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:01:43.669 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:01:43.666+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:01:43.703 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:43.703 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:43.706 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T15:01:43.713 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T15:01:43.727 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T15:01:43.745 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T15:01:43.765 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T15:01:43.770 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:01:43.770+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:01:43.789 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T15:01:43.813 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:43.813 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T15:01:43.813 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T15:01:43.813 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T15:01:43.813 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.813 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139776 3.16 3:6c38d0e2:::rbd_data.113664780c85.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:43.814 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.223442+0000", 2024-09-21T15:01:43.814 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 41.58733514, 2024-09-21T15:01:43.814 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0315488150000001, 2024-09-21T15:01:43.814 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:43.814 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:43.814 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:43.814 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:43.814 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:43.814 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139776 2024-09-21T15:01:43.814 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.814 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:43.814 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.814 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:43.814 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223442+0000", 2024-09-21T15:01:43.815 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:43.815 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.815 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.815 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:43.815 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223442+0000", 2024-09-21T15:01:43.815 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:43.815 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.815 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.815 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:43.815 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223442+0000", 2024-09-21T15:01:43.815 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:43.815 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.815 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.815 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:43.816 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223446+0000", 2024-09-21T15:01:43.816 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2549999999999999e-06 2024-09-21T15:01:43.816 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.816 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.816 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:43.816 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223454+0000", 2024-09-21T15:01:43.816 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8199999999999997e-06 2024-09-21T15:01:43.816 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.816 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.816 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:43.816 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223460+0000", 2024-09-21T15:01:43.816 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6330000000000001e-06 2024-09-21T15:01:43.816 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.816 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.816 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:43.817 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223542+0000", 2024-09-21T15:01:43.817 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1929999999999997e-05 2024-09-21T15:01:43.817 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.817 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.817 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:43.817 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223552+0000", 2024-09-21T15:01:43.817 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0627e-05 2024-09-21T15:01:43.817 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.817 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.817 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:43.817 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.254634+0000", 2024-09-21T15:01:43.817 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.031081972 2024-09-21T15:01:43.817 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.817 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.817 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:43.818 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.254796+0000", 2024-09-21T15:01:43.818 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000161879 2024-09-21T15:01:43.818 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.818 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.818 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:43.818 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.254991+0000", 2024-09-21T15:01:43.818 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00019526499999999999 2024-09-21T15:01:43.818 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:43.818 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:43.818 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:43.818 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.818 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.818 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139785 3.11 3:8abbb109:::rbd_data.113664780c85.0000000000000070:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:43.818 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.223531+0000", 2024-09-21T15:01:43.818 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 41.587246399000001, 2024-09-21T15:01:43.819 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0187143510000001, 2024-09-21T15:01:43.819 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:43.819 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:43.819 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:43.819 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:43.819 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:43.819 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139785 2024-09-21T15:01:43.819 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.819 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:43.819 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.819 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:43.819 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223531+0000", 2024-09-21T15:01:43.819 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:43.819 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.819 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.820 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:43.820 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223531+0000", 2024-09-21T15:01:43.820 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:43.820 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.820 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.820 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:43.820 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223531+0000", 2024-09-21T15:01:43.820 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:43.820 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.820 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.820 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:43.820 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223534+0000", 2024-09-21T15:01:43.820 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2650000000000001e-06 2024-09-21T15:01:43.820 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.820 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.821 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:43.821 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223541+0000", 2024-09-21T15:01:43.821 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8990000000000004e-06 2024-09-21T15:01:43.821 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.821 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.821 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:43.821 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223546+0000", 2024-09-21T15:01:43.821 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3399999999999997e-06 2024-09-21T15:01:43.821 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.821 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.821 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:43.821 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223724+0000", 2024-09-21T15:01:43.821 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017745799999999999 2024-09-21T15:01:43.821 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.821 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.821 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:43.822 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223735+0000", 2024-09-21T15:01:43.822 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1610999999999999e-05 2024-09-21T15:01:43.822 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.822 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.822 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:43.822 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242113+0000", 2024-09-21T15:01:43.822 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.018377879 2024-09-21T15:01:43.822 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.822 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.822 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:43.822 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242246+0000", 2024-09-21T15:01:43.822 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013231500000000001 2024-09-21T15:01:43.822 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:43.822 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:43.822 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:43.823 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.823 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.823 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139790 3.11 3:8a18c7d8:::rbd_data.113664780c85.000000000000008d:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:43.823 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.223578+0000", 2024-09-21T15:01:43.823 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 41.587199404000003, 2024-09-21T15:01:43.823 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0187879719999999, 2024-09-21T15:01:43.823 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:43.823 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:43.823 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:43.823 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:43.823 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:43.823 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139790 2024-09-21T15:01:43.823 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.823 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:43.823 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.824 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:43.824 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223578+0000", 2024-09-21T15:01:43.824 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:43.824 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.824 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.824 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:43.824 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223578+0000", 2024-09-21T15:01:43.824 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:43.824 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.824 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.824 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:43.824 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223578+0000", 2024-09-21T15:01:43.824 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:43.824 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.824 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.825 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:43.825 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223581+0000", 2024-09-21T15:01:43.825 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6879999999999999e-06 2024-09-21T15:01:43.825 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.825 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.825 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:43.825 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223586+0000", 2024-09-21T15:01:43.825 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7080000000000002e-06 2024-09-21T15:01:43.825 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.825 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.825 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:43.825 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223591+0000", 2024-09-21T15:01:43.825 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5639999999999998e-06 2024-09-21T15:01:43.825 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.825 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.825 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:43.826 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223761+0000", 2024-09-21T15:01:43.826 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000170213 2024-09-21T15:01:43.826 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.826 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.826 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:43.826 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223769+0000", 2024-09-21T15:01:43.826 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7049999999999999e-06 2024-09-21T15:01:43.826 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.826 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.826 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:43.826 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242269+0000", 2024-09-21T15:01:43.826 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.018500288 2024-09-21T15:01:43.826 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.826 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.826 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:43.827 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242366+0000", 2024-09-21T15:01:43.827 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.7126000000000005e-05 2024-09-21T15:01:43.827 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:43.827 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:43.827 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:43.827 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.827 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.827 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139792 3.16 3:6ce5caf3:::rbd_data.113664780c85.000000000000009b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:43.827 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.223595+0000", 2024-09-21T15:01:43.827 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 41.587182450999997, 2024-09-21T15:01:43.827 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0317517899999999, 2024-09-21T15:01:43.827 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:43.827 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:43.827 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:43.827 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:43.828 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:43.828 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139792 2024-09-21T15:01:43.828 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.828 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:43.828 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.828 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:43.828 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223595+0000", 2024-09-21T15:01:43.828 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:43.828 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.828 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.828 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:43.828 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223595+0000", 2024-09-21T15:01:43.828 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:43.828 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.828 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.828 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:43.829 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223595+0000", 2024-09-21T15:01:43.829 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:43.829 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.829 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.829 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:43.829 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223598+0000", 2024-09-21T15:01:43.829 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4620000000000001e-06 2024-09-21T15:01:43.829 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.829 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.829 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:43.829 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223604+0000", 2024-09-21T15:01:43.829 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6879999999999996e-06 2024-09-21T15:01:43.829 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.829 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.829 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:43.829 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223609+0000", 2024-09-21T15:01:43.830 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9490000000000004e-06 2024-09-21T15:01:43.830 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.830 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.830 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:43.830 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224067+0000", 2024-09-21T15:01:43.830 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00045780700000000002 2024-09-21T15:01:43.830 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.830 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.830 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:43.830 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224075+0000", 2024-09-21T15:01:43.830 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6610000000000006e-06 2024-09-21T15:01:43.830 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.830 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.830 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:43.830 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255141+0000", 2024-09-21T15:01:43.831 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0310657480000001 2024-09-21T15:01:43.831 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.831 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.831 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:43.831 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255347+0000", 2024-09-21T15:01:43.831 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000205837 2024-09-21T15:01:43.831 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:43.831 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:43.831 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:43.831 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.831 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.831 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139794 3.11 3:8cf3a2f4:::rbd_data.113664780c85.00000000000000a9:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:43.831 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.224011+0000", 2024-09-21T15:01:43.831 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 41.586766423, 2024-09-21T15:01:43.831 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.018507791, 2024-09-21T15:01:43.832 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:43.832 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:43.832 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:43.832 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:43.832 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:43.832 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139794 2024-09-21T15:01:43.832 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.832 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:43.832 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.832 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:43.832 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224011+0000", 2024-09-21T15:01:43.832 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:43.832 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.833 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.833 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:43.833 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224011+0000", 2024-09-21T15:01:43.833 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:43.833 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.833 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.833 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:43.833 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224010+0000", 2024-09-21T15:01:43.833 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:43.833 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.833 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.833 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:43.833 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224016+0000", 2024-09-21T15:01:43.833 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6550000000000001e-06 2024-09-21T15:01:43.834 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.834 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.834 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:43.834 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224028+0000", 2024-09-21T15:01:43.834 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2422999999999999e-05 2024-09-21T15:01:43.834 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.834 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.834 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:43.834 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224036+0000", 2024-09-21T15:01:43.834 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7749999999999993e-06 2024-09-21T15:01:43.834 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.834 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.834 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:43.834 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225176+0000", 2024-09-21T15:01:43.834 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0011401740000000001 2024-09-21T15:01:43.835 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.835 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.835 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:43.835 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225187+0000", 2024-09-21T15:01:43.835 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0951999999999999e-05 2024-09-21T15:01:43.835 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.835 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.835 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:43.835 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242392+0000", 2024-09-21T15:01:43.835 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.017204905 2024-09-21T15:01:43.835 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.835 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.835 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:43.835 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242519+0000", 2024-09-21T15:01:43.835 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012680499999999999 2024-09-21T15:01:43.835 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:43.836 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:43.836 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:43.836 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.836 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.836 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139800 3.16 3:68b050ed:::rbd_data.113664780c85.00000000000000d3:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:43.836 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.224579+0000", 2024-09-21T15:01:43.836 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 41.586198664000001, 2024-09-21T15:01:43.836 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.030932637, 2024-09-21T15:01:43.836 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:43.836 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:43.836 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:43.836 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:43.836 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:43.836 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139800 2024-09-21T15:01:43.836 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.836 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:43.837 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.837 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:43.837 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224579+0000", 2024-09-21T15:01:43.837 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:43.837 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.837 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.837 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:43.837 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224579+0000", 2024-09-21T15:01:43.837 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:43.837 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.837 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.837 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:43.837 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224578+0000", 2024-09-21T15:01:43.837 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:43.838 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.838 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.838 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:43.838 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224582+0000", 2024-09-21T15:01:43.838 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0029999999999999e-06 2024-09-21T15:01:43.838 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.838 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.838 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:43.838 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224588+0000", 2024-09-21T15:01:43.838 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8120000000000002e-06 2024-09-21T15:01:43.838 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.838 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.838 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:43.839 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224593+0000", 2024-09-21T15:01:43.839 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7469999999999996e-06 2024-09-21T15:01:43.839 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.839 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.839 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:43.839 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224648+0000", 2024-09-21T15:01:43.839 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4693e-05 2024-09-21T15:01:43.839 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.839 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.839 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:43.839 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224656+0000", 2024-09-21T15:01:43.839 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8429999999999993e-06 2024-09-21T15:01:43.839 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.839 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.840 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:43.840 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255382+0000", 2024-09-21T15:01:43.840 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0307266070000001 2024-09-21T15:01:43.840 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.840 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.840 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:43.840 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255512+0000", 2024-09-21T15:01:43.840 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000129367 2024-09-21T15:01:43.840 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:43.840 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:43.840 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:43.840 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.840 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.840 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139804 3.11 3:88f9ddce:::rbd_data.113664780c85.00000000000000ef:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:43.840 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.224632+0000", 2024-09-21T15:01:43.841 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 41.586145682999998, 2024-09-21T15:01:43.841 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.018021302, 2024-09-21T15:01:43.841 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:43.841 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:43.841 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:43.841 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:43.841 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:43.841 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139804 2024-09-21T15:01:43.841 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.841 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:43.841 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.841 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:43.841 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224632+0000", 2024-09-21T15:01:43.841 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:43.841 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.842 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.842 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:43.842 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224632+0000", 2024-09-21T15:01:43.842 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:43.842 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.842 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.842 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:43.842 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224632+0000", 2024-09-21T15:01:43.842 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:43.842 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.842 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.842 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:43.842 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224635+0000", 2024-09-21T15:01:43.843 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2710000000000001e-06 2024-09-21T15:01:43.843 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.843 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.843 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:43.843 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224640+0000", 2024-09-21T15:01:43.843 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6779999999999997e-06 2024-09-21T15:01:43.843 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.843 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.843 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:43.843 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224645+0000", 2024-09-21T15:01:43.843 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9339999999999997e-06 2024-09-21T15:01:43.843 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.843 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.843 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:43.843 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225307+0000", 2024-09-21T15:01:43.844 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00066201699999999999 2024-09-21T15:01:43.844 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.844 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.844 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:43.844 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225312+0000", 2024-09-21T15:01:43.844 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9799999999999998e-06 2024-09-21T15:01:43.844 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.844 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.844 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:43.844 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242548+0000", 2024-09-21T15:01:43.844 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.017235603 2024-09-21T15:01:43.844 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.844 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.844 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:43.845 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242653+0000", 2024-09-21T15:01:43.845 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010522 2024-09-21T15:01:43.845 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:43.845 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:43.845 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:43.845 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.845 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.845 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139809 3.11 3:884da946:::rbd_data.113664780c85.0000000000000118:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:43.845 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.224667+0000", 2024-09-21T15:01:43.845 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 41.586110638999997, 2024-09-21T15:01:43.845 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0181101960000001, 2024-09-21T15:01:43.845 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:43.845 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:43.845 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:43.846 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:43.846 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:43.846 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139809 2024-09-21T15:01:43.846 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.846 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:43.846 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.846 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:43.846 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224667+0000", 2024-09-21T15:01:43.846 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:43.846 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.846 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.846 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:43.846 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224667+0000", 2024-09-21T15:01:43.846 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:43.847 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.847 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.847 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:43.847 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224667+0000", 2024-09-21T15:01:43.847 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:43.847 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.847 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.847 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:43.847 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224669+0000", 2024-09-21T15:01:43.847 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.452e-06 2024-09-21T15:01:43.847 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.847 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.847 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:43.847 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224675+0000", 2024-09-21T15:01:43.848 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.818e-06 2024-09-21T15:01:43.848 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.848 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.848 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:43.848 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224680+0000", 2024-09-21T15:01:43.848 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6319999999999999e-06 2024-09-21T15:01:43.848 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.848 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.848 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:43.848 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225400+0000", 2024-09-21T15:01:43.848 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00072012699999999999 2024-09-21T15:01:43.848 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.848 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.848 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:43.849 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225409+0000", 2024-09-21T15:01:43.849 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1039999999999996e-06 2024-09-21T15:01:43.849 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.849 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.849 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:43.849 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242675+0000", 2024-09-21T15:01:43.849 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0172663319999999 2024-09-21T15:01:43.849 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.849 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.849 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:43.849 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242777+0000", 2024-09-21T15:01:43.849 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010197899999999999 2024-09-21T15:01:43.849 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:43.849 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:43.849 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:43.849 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.850 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.850 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139868 3.16 3:69f82090:::rbd_data.113664780c85.00000000000000f1:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:43.850 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.225982+0000", 2024-09-21T15:01:43.850 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 41.584795139000001, 2024-09-21T15:01:43.850 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0296791729999999, 2024-09-21T15:01:43.850 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:43.850 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:43.850 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:43.850 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:43.850 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:43.850 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139868 2024-09-21T15:01:43.850 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.850 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:43.850 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.850 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:43.851 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225982+0000", 2024-09-21T15:01:43.851 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:43.851 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.851 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.851 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:43.851 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225982+0000", 2024-09-21T15:01:43.851 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:43.851 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.851 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.851 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:43.851 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225982+0000", 2024-09-21T15:01:43.851 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:43.851 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.851 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.851 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:43.852 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225985+0000", 2024-09-21T15:01:43.852 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.74e-06 2024-09-21T15:01:43.852 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.852 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.852 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:43.852 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225990+0000", 2024-09-21T15:01:43.852 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7039999999999999e-06 2024-09-21T15:01:43.852 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.852 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.852 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:43.852 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225995+0000", 2024-09-21T15:01:43.852 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7249999999999997e-06 2024-09-21T15:01:43.852 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.852 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.853 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:43.853 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226299+0000", 2024-09-21T15:01:43.853 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00030383199999999997 2024-09-21T15:01:43.853 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.853 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.853 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:43.853 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226308+0000", 2024-09-21T15:01:43.853 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.9740000000000008e-06 2024-09-21T15:01:43.853 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.853 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.853 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:43.853 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255539+0000", 2024-09-21T15:01:43.853 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0292305960000001 2024-09-21T15:01:43.853 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.854 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.854 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:43.854 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255662+0000", 2024-09-21T15:01:43.854 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012307400000000001 2024-09-21T15:01:43.854 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:43.854 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:43.854 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:43.854 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.854 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.854 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139869 3.16 3:6b6c90a4:::rbd_data.113664780c85.00000000000000fe:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:43.854 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.226006+0000", 2024-09-21T15:01:43.854 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 41.584771523000001, 2024-09-21T15:01:43.854 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.029790518, 2024-09-21T15:01:43.854 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:43.854 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:43.855 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:43.855 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:43.855 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:43.855 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139869 2024-09-21T15:01:43.855 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.855 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:43.855 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.855 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:43.855 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226006+0000", 2024-09-21T15:01:43.855 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:43.855 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.855 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.855 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:43.856 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226006+0000", 2024-09-21T15:01:43.856 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:43.856 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.856 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.856 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:43.856 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226006+0000", 2024-09-21T15:01:43.856 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:43.856 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.856 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.856 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:43.856 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226008+0000", 2024-09-21T15:01:43.856 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5569999999999998e-06 2024-09-21T15:01:43.856 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.856 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.856 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:43.857 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226014+0000", 2024-09-21T15:01:43.857 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2079999999999997e-06 2024-09-21T15:01:43.857 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.857 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.857 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:43.857 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226020+0000", 2024-09-21T15:01:43.857 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7429999999999999e-06 2024-09-21T15:01:43.857 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.857 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.857 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:43.857 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226330+0000", 2024-09-21T15:01:43.857 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00030963300000000002 2024-09-21T15:01:43.857 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.857 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.857 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:43.858 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226336+0000", 2024-09-21T15:01:43.858 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6340000000000003e-06 2024-09-21T15:01:43.858 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.858 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.858 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:43.858 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255687+0000", 2024-09-21T15:01:43.858 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0293507150000001 2024-09-21T15:01:43.858 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.858 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.858 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:43.858 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255797+0000", 2024-09-21T15:01:43.858 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000109471 2024-09-21T15:01:43.858 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:43.858 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:43.859 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:43.859 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.859 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.859 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139884 3.16 3:69f10324:::rbd_data.113664780c85.0000000000000161:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:43.859 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.227573+0000", 2024-09-21T15:01:43.859 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 41.583204881999997, 2024-09-21T15:01:43.859 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0283568759999999, 2024-09-21T15:01:43.859 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:43.859 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:43.859 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:43.859 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:43.859 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:43.859 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139884 2024-09-21T15:01:43.859 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.859 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:43.860 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.860 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:43.860 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227573+0000", 2024-09-21T15:01:43.860 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:43.860 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.860 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.860 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:43.860 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227573+0000", 2024-09-21T15:01:43.860 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:43.860 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.860 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.860 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:43.860 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227572+0000", 2024-09-21T15:01:43.860 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:43.860 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.860 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.860 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:43.861 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227575+0000", 2024-09-21T15:01:43.861 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7769999999999998e-06 2024-09-21T15:01:43.861 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.861 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.861 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:43.861 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227581+0000", 2024-09-21T15:01:43.861 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7359999999999998e-06 2024-09-21T15:01:43.861 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.861 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.861 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:43.861 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227585+0000", 2024-09-21T15:01:43.861 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5090000000000004e-06 2024-09-21T15:01:43.861 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.861 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.861 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:43.861 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227753+0000", 2024-09-21T15:01:43.862 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00016761599999999999 2024-09-21T15:01:43.862 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.862 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.862 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:43.862 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227761+0000", 2024-09-21T15:01:43.862 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0579999999999998e-06 2024-09-21T15:01:43.862 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.862 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.862 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:43.862 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255828+0000", 2024-09-21T15:01:43.862 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0280671349999999 2024-09-21T15:01:43.862 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.862 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.862 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:43.862 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255930+0000", 2024-09-21T15:01:43.863 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000101372 2024-09-21T15:01:43.863 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:43.863 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:43.863 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:43.863 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.863 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.863 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139913 3.16 3:6f81b46d:::rbd_data.113664780c85.0000000000000074:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:43.863 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.227685+0000", 2024-09-21T15:01:43.863 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 41.583092702999998, 2024-09-21T15:01:43.863 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0283644750000001, 2024-09-21T15:01:43.863 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:43.863 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:43.863 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:43.863 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:43.863 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:43.863 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139913 2024-09-21T15:01:43.864 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.864 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:43.864 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.864 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:43.864 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227685+0000", 2024-09-21T15:01:43.864 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:43.864 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.864 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.864 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:43.864 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227685+0000", 2024-09-21T15:01:43.864 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:43.864 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.864 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.864 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:43.865 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227685+0000", 2024-09-21T15:01:43.865 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:43.865 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.865 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.865 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:43.865 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227687+0000", 2024-09-21T15:01:43.865 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2929999999999999e-06 2024-09-21T15:01:43.865 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.865 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.865 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:43.865 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227693+0000", 2024-09-21T15:01:43.865 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1070000000000001e-06 2024-09-21T15:01:43.865 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.866 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.866 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:43.866 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227698+0000", 2024-09-21T15:01:43.866 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9339999999999997e-06 2024-09-21T15:01:43.866 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.866 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.866 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:43.866 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227784+0000", 2024-09-21T15:01:43.866 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6096999999999999e-05 2024-09-21T15:01:43.866 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.866 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.866 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:43.866 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227791+0000", 2024-09-21T15:01:43.866 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9090000000000003e-06 2024-09-21T15:01:43.867 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.867 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.867 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:43.867 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255955+0000", 2024-09-21T15:01:43.867 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0281643579999999 2024-09-21T15:01:43.867 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.867 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.867 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:43.867 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256049+0000", 2024-09-21T15:01:43.867 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4047000000000003e-05 2024-09-21T15:01:43.867 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:43.867 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:43.867 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:43.867 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.867 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.868 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139988 3.11 3:88a29fa4:::rbd_data.113664780c85.00000000000000bd:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:43.868 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.229772+0000", 2024-09-21T15:01:43.868 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 41.581005204, 2024-09-21T15:01:43.868 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.017967668, 2024-09-21T15:01:43.868 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:43.868 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:43.868 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:43.868 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:43.868 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:43.868 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139988 2024-09-21T15:01:43.868 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.868 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:43.868 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.868 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:43.868 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229772+0000", 2024-09-21T15:01:43.869 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:43.869 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.869 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.869 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:43.869 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229772+0000", 2024-09-21T15:01:43.869 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:43.869 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.869 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.869 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:43.869 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229772+0000", 2024-09-21T15:01:43.869 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:43.869 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.869 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.869 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:43.870 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229776+0000", 2024-09-21T15:01:43.870 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6890000000000002e-06 2024-09-21T15:01:43.870 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.870 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.870 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:43.870 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229783+0000", 2024-09-21T15:01:43.870 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2280000000000001e-06 2024-09-21T15:01:43.870 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.870 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.870 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:43.870 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229789+0000", 2024-09-21T15:01:43.870 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1700000000000002e-06 2024-09-21T15:01:43.870 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.870 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.871 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:43.871 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.230106+0000", 2024-09-21T15:01:43.871 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00031700800000000003 2024-09-21T15:01:43.871 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.871 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.871 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:43.871 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.230115+0000", 2024-09-21T15:01:43.871 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4190000000000008e-06 2024-09-21T15:01:43.871 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.871 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.871 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:43.871 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242935+0000", 2024-09-21T15:01:43.871 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0128191449999999 2024-09-21T15:01:43.871 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.872 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.872 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for missing object", 2024-09-21T15:01:43.872 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.243078+0000", 2024-09-21T15:01:43.872 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000142984 2024-09-21T15:01:43.872 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.872 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.872 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:43.872 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.247659+0000", 2024-09-21T15:01:43.872 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0045813030000000001 2024-09-21T15:01:43.872 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.872 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.872 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:43.872 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.247740+0000", 2024-09-21T15:01:43.872 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1216000000000003e-05 2024-09-21T15:01:43.872 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:43.873 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:43.873 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:43.873 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.873 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.873 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139994 3.16 3:6b9df8d1:::rbd_data.113664780c85.00000000000000e7:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:43.873 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.231370+0000", 2024-09-21T15:01:43.873 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 41.579407126, 2024-09-21T15:01:43.873 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.024795576, 2024-09-21T15:01:43.873 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:43.873 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:43.873 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:43.873 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:43.873 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:43.873 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139994 2024-09-21T15:01:43.873 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.874 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:43.874 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.874 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:43.874 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231370+0000", 2024-09-21T15:01:43.874 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:43.874 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.874 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.874 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:43.874 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231370+0000", 2024-09-21T15:01:43.874 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:43.874 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.874 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.874 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:43.874 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231369+0000", 2024-09-21T15:01:43.875 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:43.875 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.875 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.875 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:43.875 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231376+0000", 2024-09-21T15:01:43.875 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5509999999999996e-06 2024-09-21T15:01:43.875 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.875 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.875 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:43.875 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231387+0000", 2024-09-21T15:01:43.875 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.061e-05 2024-09-21T15:01:43.875 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.875 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.875 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:43.875 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231394+0000", 2024-09-21T15:01:43.876 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3799999999999996e-06 2024-09-21T15:01:43.876 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.876 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.876 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:43.876 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231459+0000", 2024-09-21T15:01:43.876 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4716999999999997e-05 2024-09-21T15:01:43.876 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.876 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.876 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:43.876 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231468+0000", 2024-09-21T15:01:43.876 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1479999999999995e-06 2024-09-21T15:01:43.876 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.876 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.876 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:43.877 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256074+0000", 2024-09-21T15:01:43.877 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0246064180000001 2024-09-21T15:01:43.877 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.877 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.877 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:43.877 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256166+0000", 2024-09-21T15:01:43.877 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1862999999999998e-05 2024-09-21T15:01:43.877 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:43.877 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:43.877 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:43.877 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.877 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.877 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140012 3.16 3:6bd6dfb6:::rbd_data.113664780c85.0000000000000165:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:43.877 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.231890+0000", 2024-09-21T15:01:43.878 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 41.578887367, 2024-09-21T15:01:43.878 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.024408881, 2024-09-21T15:01:43.878 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:43.878 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:43.878 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:43.878 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:43.878 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:43.878 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140012 2024-09-21T15:01:43.878 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.878 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:43.878 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.878 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:43.878 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231890+0000", 2024-09-21T15:01:43.878 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:43.879 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.879 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.879 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:43.879 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231890+0000", 2024-09-21T15:01:43.879 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:43.879 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.879 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.879 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:43.879 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231889+0000", 2024-09-21T15:01:43.879 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:43.879 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.879 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.879 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:43.879 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231895+0000", 2024-09-21T15:01:43.880 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9569999999999999e-06 2024-09-21T15:01:43.880 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.880 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.880 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:43.880 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231910+0000", 2024-09-21T15:01:43.880 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4549000000000001e-05 2024-09-21T15:01:43.880 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.880 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.880 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:43.880 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231917+0000", 2024-09-21T15:01:43.880 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5730000000000002e-06 2024-09-21T15:01:43.880 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.880 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.880 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:43.880 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231988+0000", 2024-09-21T15:01:43.881 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0852999999999994e-05 2024-09-21T15:01:43.881 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.881 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.881 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:43.881 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.232000+0000", 2024-09-21T15:01:43.881 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2159e-05 2024-09-21T15:01:43.881 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.881 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.881 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:43.881 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256192+0000", 2024-09-21T15:01:43.881 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.024192073 2024-09-21T15:01:43.881 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.881 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.881 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:43.882 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256299+0000", 2024-09-21T15:01:43.882 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000106733 2024-09-21T15:01:43.882 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:43.882 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:43.882 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:43.882 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.882 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.882 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140055 3.16 3:69bb9803:::rbd_data.113664780c85.00000000000000da:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:43.882 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.233922+0000", 2024-09-21T15:01:43.882 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 41.576855838, 2024-09-21T15:01:43.882 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.022511768, 2024-09-21T15:01:43.882 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:43.882 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:43.882 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:43.883 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:43.883 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:43.883 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140055 2024-09-21T15:01:43.883 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.883 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:43.883 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.883 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:43.883 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233922+0000", 2024-09-21T15:01:43.883 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:43.883 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.883 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.883 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:43.883 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233922+0000", 2024-09-21T15:01:43.883 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:43.884 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.884 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.884 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:43.884 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233921+0000", 2024-09-21T15:01:43.884 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:43.884 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.884 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.884 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:43.884 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233927+0000", 2024-09-21T15:01:43.884 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0360000000000003e-06 2024-09-21T15:01:43.884 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.884 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.884 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:43.884 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233937+0000", 2024-09-21T15:01:43.884 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6439999999999995e-06 2024-09-21T15:01:43.885 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.885 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.885 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:43.885 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233944+0000", 2024-09-21T15:01:43.885 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7710000000000006e-06 2024-09-21T15:01:43.885 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.885 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.885 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:43.885 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233996+0000", 2024-09-21T15:01:43.885 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1597000000000001e-05 2024-09-21T15:01:43.885 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.885 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.885 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:43.885 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.234005+0000", 2024-09-21T15:01:43.886 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5960000000000004e-06 2024-09-21T15:01:43.886 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.886 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.886 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:43.886 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256332+0000", 2024-09-21T15:01:43.886 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0223271949999999 2024-09-21T15:01:43.886 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.886 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.886 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:43.886 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256434+0000", 2024-09-21T15:01:43.886 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010180599999999999 2024-09-21T15:01:43.886 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:43.886 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:43.886 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:43.887 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.887 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.887 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140143 3.16 3:6d3ed596:::rbd_data.113664780c85.0000000000000184:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:43.887 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.239496+0000", 2024-09-21T15:01:43.887 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 41.571281309, 2024-09-21T15:01:43.887 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.017076576, 2024-09-21T15:01:43.887 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:43.887 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:43.887 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:43.887 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:43.887 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:43.887 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140143 2024-09-21T15:01:43.887 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.887 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:43.887 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.888 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:43.888 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239496+0000", 2024-09-21T15:01:43.888 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:43.888 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.888 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.888 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:43.888 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239496+0000", 2024-09-21T15:01:43.888 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:43.888 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.888 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.888 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:43.888 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239496+0000", 2024-09-21T15:01:43.888 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:43.888 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.889 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.889 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:43.889 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239498+0000", 2024-09-21T15:01:43.889 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.509e-06 2024-09-21T15:01:43.889 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.889 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.889 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:43.889 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239504+0000", 2024-09-21T15:01:43.889 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6869999999999999e-06 2024-09-21T15:01:43.889 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.889 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.889 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:43.889 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239509+0000", 2024-09-21T15:01:43.889 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8860000000000003e-06 2024-09-21T15:01:43.889 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.890 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.890 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:43.890 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239572+0000", 2024-09-21T15:01:43.890 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2908000000000001e-05 2024-09-21T15:01:43.890 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.890 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.890 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:43.890 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239580+0000", 2024-09-21T15:01:43.890 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4270000000000008e-06 2024-09-21T15:01:43.890 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.890 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.890 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:43.890 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256457+0000", 2024-09-21T15:01:43.891 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0168770979999999 2024-09-21T15:01:43.891 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.891 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.891 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:43.891 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256573+0000", 2024-09-21T15:01:43.891 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00011538400000000001 2024-09-21T15:01:43.891 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:43.891 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:43.891 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:43.891 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.891 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.891 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140163 3.16 3:694e82eb:::rbd_data.113664780c85.0000000000000052:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:43.891 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.240334+0000", 2024-09-21T15:01:43.891 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 41.570443597000001, 2024-09-21T15:01:43.891 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.016365537, 2024-09-21T15:01:43.892 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:43.892 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:43.892 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:43.892 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:43.892 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:43.892 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140163 2024-09-21T15:01:43.892 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.892 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:43.892 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.892 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:43.892 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240334+0000", 2024-09-21T15:01:43.892 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:43.892 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.892 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.893 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:43.893 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240334+0000", 2024-09-21T15:01:43.893 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:43.893 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.893 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.893 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:43.893 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240334+0000", 2024-09-21T15:01:43.893 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:43.893 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.893 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.893 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:43.893 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240337+0000", 2024-09-21T15:01:43.893 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.089e-06 2024-09-21T15:01:43.893 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.893 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.894 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:43.894 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240343+0000", 2024-09-21T15:01:43.894 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4420000000000003e-06 2024-09-21T15:01:43.894 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.894 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.894 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:43.894 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240348+0000", 2024-09-21T15:01:43.894 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1039999999999998e-06 2024-09-21T15:01:43.894 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.894 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.894 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:43.894 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240451+0000", 2024-09-21T15:01:43.894 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000102996 2024-09-21T15:01:43.894 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.895 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.895 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:43.895 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240460+0000", 2024-09-21T15:01:43.895 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5369999999999997e-06 2024-09-21T15:01:43.895 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.895 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.895 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:43.895 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256600+0000", 2024-09-21T15:01:43.895 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.016139884 2024-09-21T15:01:43.895 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.895 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.895 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:43.895 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256700+0000", 2024-09-21T15:01:43.895 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9844000000000002e-05 2024-09-21T15:01:43.896 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:43.896 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:43.896 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:43.896 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.896 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.896 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140164 3.16 3:68bfdc57:::rbd_data.113664780c85.0000000000000053:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:43.896 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.240358+0000", 2024-09-21T15:01:43.896 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 41.570419903000001, 2024-09-21T15:01:43.896 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.01646094, 2024-09-21T15:01:43.896 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:43.896 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:43.896 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:43.896 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:43.896 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:43.897 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140164 2024-09-21T15:01:43.897 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.897 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:43.897 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.897 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:43.897 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240358+0000", 2024-09-21T15:01:43.897 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:43.897 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.897 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.897 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:43.897 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240358+0000", 2024-09-21T15:01:43.897 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:43.897 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.897 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.898 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:43.898 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240357+0000", 2024-09-21T15:01:43.898 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:43.898 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.898 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.898 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:43.898 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240360+0000", 2024-09-21T15:01:43.898 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7089999999999998e-06 2024-09-21T15:01:43.898 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.898 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.898 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:43.898 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240368+0000", 2024-09-21T15:01:43.898 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7270000000000007e-06 2024-09-21T15:01:43.898 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.899 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.899 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:43.899 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240373+0000", 2024-09-21T15:01:43.899 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4060000000000004e-06 2024-09-21T15:01:43.899 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.899 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.899 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:43.899 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240473+0000", 2024-09-21T15:01:43.899 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000100132 2024-09-21T15:01:43.899 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.899 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.899 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:43.899 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240476+0000", 2024-09-21T15:01:43.899 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.332e-06 2024-09-21T15:01:43.899 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.900 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.900 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:43.900 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256724+0000", 2024-09-21T15:01:43.900 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0162471930000001 2024-09-21T15:01:43.900 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.900 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.900 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:43.900 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256819+0000", 2024-09-21T15:01:43.900 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5007999999999996e-05 2024-09-21T15:01:43.900 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:43.900 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:43.900 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:43.900 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.900 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.901 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140188 3.16 3:6f6217d4:::rbd_data.113664780c85.00000000000000fb:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:43.901 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.240978+0000", 2024-09-21T15:01:43.901 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 41.569799244000002, 2024-09-21T15:01:43.901 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.015983809, 2024-09-21T15:01:43.901 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:43.901 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:43.901 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:43.901 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:43.901 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:43.901 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140188 2024-09-21T15:01:43.901 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.901 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:43.901 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.901 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:43.902 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240978+0000", 2024-09-21T15:01:43.902 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:43.902 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.902 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.902 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:43.902 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240978+0000", 2024-09-21T15:01:43.902 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:43.902 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.902 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.902 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:43.902 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240978+0000", 2024-09-21T15:01:43.902 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:43.902 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.903 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.903 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:43.903 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240981+0000", 2024-09-21T15:01:43.903 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1559999999999999e-06 2024-09-21T15:01:43.903 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.903 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.903 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:43.903 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240987+0000", 2024-09-21T15:01:43.903 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.229e-06 2024-09-21T15:01:43.903 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.903 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.903 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:43.903 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240992+0000", 2024-09-21T15:01:43.903 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9710000000000003e-06 2024-09-21T15:01:43.904 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.904 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.904 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:43.904 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.241361+0000", 2024-09-21T15:01:43.904 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000369236 2024-09-21T15:01:43.904 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.904 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.904 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:43.904 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.241369+0000", 2024-09-21T15:01:43.904 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6199999999999999e-06 2024-09-21T15:01:43.904 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.904 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.904 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:43.904 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256844+0000", 2024-09-21T15:01:43.904 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0154751609999999 2024-09-21T15:01:43.905 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:43.905 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:43.905 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:43.905 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256962+0000", 2024-09-21T15:01:43.905 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000117964 2024-09-21T15:01:43.905 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:43.905 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:43.905 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:43.905 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:43.905 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:43.905 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:43.906 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T15:01:43.907 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T15:01:43.907 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T15:01:43.907 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-21T15:01:43.908 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:01:43.908 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:01:43.908 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:01:43.908 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:01:43.908 INFO:teuthology.orchestra.run.smithi032.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-21T15:01:43.908 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:01:43.908 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:01:43.908 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:01:43.908 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-21T15:01:43.910 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:01:43.904+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:01:44.010 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:01:44.004+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:01:44.026 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:01:44.027 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:01:44.027 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T15:01:44.027 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:01:44.037 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T15:01:44.112 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:01:44.110+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:01:44.145 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:01:44.145 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:01:44.145 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T15:01:44.145 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T15:01:44.145 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:01:44.156 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T15:01:44.212 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:01:44.210+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:01:44.251 INFO:teuthology.orchestra.run.smithi032.stderr:nodeep-scrub is set 2024-09-21T15:01:44.272 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:01:44.272 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T15:01:44.272 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T15:01:44.272 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T15:01:44.272 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.272 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45854 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:44.272 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159802+0000", 2024-09-21T15:01:44.272 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 154.108137825, 2024-09-21T15:01:44.272 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068358938999999994, 2024-09-21T15:01:44.272 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:44.273 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:44.273 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:44.273 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:44.273 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:44.273 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45854 2024-09-21T15:01:44.273 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.273 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:44.273 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.273 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:44.273 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159802+0000", 2024-09-21T15:01:44.273 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:44.273 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.273 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.273 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:44.273 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159802+0000", 2024-09-21T15:01:44.274 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:44.274 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.274 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.274 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:44.274 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159801+0000", 2024-09-21T15:01:44.274 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:44.274 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.274 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.274 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:44.274 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159804+0000", 2024-09-21T15:01:44.274 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0970000000000002e-06 2024-09-21T15:01:44.274 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.274 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.275 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:44.275 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159823+0000", 2024-09-21T15:01:44.275 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8895000000000001e-05 2024-09-21T15:01:44.275 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.275 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.275 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:44.275 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159837+0000", 2024-09-21T15:01:44.275 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3247999999999999e-05 2024-09-21T15:01:44.275 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.275 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.275 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:44.275 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228151+0000", 2024-09-21T15:01:44.275 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068314642999999994 2024-09-21T15:01:44.276 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.276 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.276 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:44.276 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228161+0000", 2024-09-21T15:01:44.276 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4250000000000004e-06 2024-09-21T15:01:44.276 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:44.276 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:44.276 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:44.276 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.276 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.276 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45985 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:44.276 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159897+0000", 2024-09-21T15:01:44.276 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 154.10804232300001, 2024-09-21T15:01:44.276 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068345664, 2024-09-21T15:01:44.276 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:44.277 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:44.277 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:44.277 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:44.277 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:44.277 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45985 2024-09-21T15:01:44.277 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.277 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:44.277 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.277 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:44.277 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159897+0000", 2024-09-21T15:01:44.277 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:44.277 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.277 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.277 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:44.277 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159897+0000", 2024-09-21T15:01:44.278 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:44.278 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.278 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.278 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:44.278 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159897+0000", 2024-09-21T15:01:44.278 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:44.278 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.278 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.278 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:44.278 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159900+0000", 2024-09-21T15:01:44.278 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.954e-06 2024-09-21T15:01:44.278 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.278 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.278 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:44.279 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159913+0000", 2024-09-21T15:01:44.279 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3106e-05 2024-09-21T15:01:44.279 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.279 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.279 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:44.279 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159918+0000", 2024-09-21T15:01:44.279 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5199999999999997e-06 2024-09-21T15:01:44.279 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.279 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.279 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:44.279 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228236+0000", 2024-09-21T15:01:44.279 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068317040999999995 2024-09-21T15:01:44.279 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.279 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.280 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:44.280 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228243+0000", 2024-09-21T15:01:44.280 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3880000000000002e-06 2024-09-21T15:01:44.280 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:44.280 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:44.280 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:44.280 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.280 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.280 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45986 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:44.280 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159924+0000", 2024-09-21T15:01:44.280 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 154.10801602800001, 2024-09-21T15:01:44.280 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068364362999999997, 2024-09-21T15:01:44.280 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:44.280 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:44.281 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:44.281 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:44.281 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:44.281 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45986 2024-09-21T15:01:44.281 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.281 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:44.281 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.281 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:44.281 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159924+0000", 2024-09-21T15:01:44.281 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:44.281 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.281 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.281 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:44.281 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159924+0000", 2024-09-21T15:01:44.281 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:44.281 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.282 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.282 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:44.282 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159923+0000", 2024-09-21T15:01:44.282 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:44.282 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.282 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.282 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:44.282 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159926+0000", 2024-09-21T15:01:44.282 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0510000000000001e-06 2024-09-21T15:01:44.282 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.282 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.282 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:44.282 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159932+0000", 2024-09-21T15:01:44.283 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4160000000000001e-06 2024-09-21T15:01:44.283 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.283 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.283 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:44.283 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159938+0000", 2024-09-21T15:01:44.283 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.383e-06 2024-09-21T15:01:44.283 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.283 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.283 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:44.283 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228275+0000", 2024-09-21T15:01:44.283 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068337007000000005 2024-09-21T15:01:44.283 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.283 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.283 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:44.283 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228288+0000", 2024-09-21T15:01:44.284 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3052e-05 2024-09-21T15:01:44.284 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:44.284 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:44.284 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:44.284 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.284 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.284 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45987 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:44.284 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159944+0000", 2024-09-21T15:01:44.284 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 154.10799580400001, 2024-09-21T15:01:44.284 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068387232000000006, 2024-09-21T15:01:44.284 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:44.284 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:44.284 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:44.284 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:44.284 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:44.285 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45987 2024-09-21T15:01:44.285 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.285 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:44.285 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.285 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:44.285 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159944+0000", 2024-09-21T15:01:44.285 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:44.285 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.285 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.285 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:44.285 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159944+0000", 2024-09-21T15:01:44.285 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:44.285 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.285 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.285 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:44.285 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159943+0000", 2024-09-21T15:01:44.286 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:44.286 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.286 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.286 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:44.286 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159946+0000", 2024-09-21T15:01:44.286 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6309999999999999e-06 2024-09-21T15:01:44.286 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.286 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.286 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:44.286 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159952+0000", 2024-09-21T15:01:44.286 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6540000000000004e-06 2024-09-21T15:01:44.286 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.286 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.286 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:44.286 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159957+0000", 2024-09-21T15:01:44.287 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9280000000000001e-06 2024-09-21T15:01:44.287 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.287 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.287 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:44.287 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228322+0000", 2024-09-21T15:01:44.287 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068365067000000002 2024-09-21T15:01:44.287 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.287 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.287 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:44.287 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228331+0000", 2024-09-21T15:01:44.287 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3270000000000007e-06 2024-09-21T15:01:44.287 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:44.287 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:44.287 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:44.287 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.288 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.288 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:97672 3.4 3:26046d29:::rbd_data.1136bfc79a70.0000000000000001:head [write 0~4718592 [fadvise_sequential+fadvise_nocache],write 5242880~131072 [fadvise_sequential+fadvise_nocache],write 5439488~147456 [fadvise_sequential+fadvise_nocache],write 5652480~229376 [fadvise_sequential+fadvise_nocache],write 5947392~147456 [fadvise_sequential+fadvise_nocache],write 6160384~229376 [fadvise_sequential+fadvise_nocache],write 6455296~147456 [fadvise_sequential+fadvise_nocache],write 6668288~229376 [fadvise_sequential+fadvise_nocache],write 6963200~147456 [fadvise_sequential+fadvise_nocache],write 7176192~147456 [fadvise_sequential+fadvise_nocache],write 7340032~65536 [fadvise_sequential+fadvise_nocache],write 7471104~147456 [fadvise_sequential+fadvise_nocache],write 7684096~147456 [fadvise_sequential+fadvise_nocache],write 7864320~49152 [fadvise_sequential+fadvise_nocache],write 7979008~147456 [fadvise_sequential+fadvise_nocache],write 8192000~147456 [fadvise_sequential+fadvise_nocache],write 8388608~32768 [fadvise_sequential+fadvise_nocache],write 8519680~114688 [fadvise_sequential+fadvise_nocache],write 8732672~114688 [fadvise_sequential+fadvise_nocache],write 8912896~16384 [fadvise_sequential+fadvise_nocache],write 9109504~32768 [fadvise_sequential+fadvise_nocache],write 9322496~32768 [fadvise_sequential+fadvise_nocache],write 9617408~32768 [fadvise_sequential+fadvise_nocache],write 9830400~32768 [fadvise_sequential+fadvise_nocache],write 9961472~98304 [fadvise_sequential+fadvise_nocache],write 10076160~196608 [fadvise_sequential+fadvise_nocache],write 10289152~278528 [fadvise_sequential+fadvise_nocache],write 10584064~196608 [fadvise_sequential+fadvise_nocache],write 10797056~196608 [fadvise_sequential+fadvise_nocache],write 11010048~65536 [fadvise_sequential+fadvise_nocache],write 11091968~196608 [fadvise_sequential+fadvise_nocache],write 11304960~196608 [fadvise_sequential+fadvise_nocache],write 11517952~65536 [fadvise_sequential+fadvise_nocache],write 11616256~98304 [fadvise_sequential+fadvise_nocache],write 11730944~65536 [fadvise_sequential+fadvise_nocache],write 11829248~98304 [fadvise_sequential+fadvise_nocache],write 11943936~65536 [fadvise_sequential+fadvise_nocache],write 12042240~49152 [fadvise_sequential+fadvise_nocache],write 12124160~98304 [fadvise_sequential+fadvise_nocache],write 12238848~65536 [fadvise_sequential+fadvise_nocache],write 12337152~98304 [fadvise_sequential+fadvise_nocache],write 12451840~65536 [fadvise_sequential+fadvise_nocache],write 12550144~32768 [fadvise_sequential+fadvise_nocache],write 12632064~98304 [fadvise_sequential+fadvise_nocache],write 12845056~98304 [fadvise_sequential+fadvise_nocache],write 13058048~49152 [fadvise_sequential+fadvise_nocache],write 13139968~98304 [fadvise_sequential+fadvise_nocache],write 13352960~98304 [fadvise_sequential+fadvise_nocache],write 13565952~65536 [fadvise_sequential+fadvise_nocache],write 13647872~98304 [fadvise_sequential+fadvise_nocache],write 13860864~98304 [fadvise_sequential+fadvise_nocache],write 14073856~81920 [fadvise_sequential+fadvise_nocache],write 15269888~81920 [fadvise_sequential+fadvise_nocache],write 15482880~81920 [fadvise_sequential+fadvise_nocache],write 15695872~32768 [fadvise_sequential+fadvise_nocache],write 15777792~81920 [fadvise_sequential+fadvise_nocache],write 15990784~81920 [fadvise_sequential+fadvise_nocache],write 16203776~49152 [fadvise_sequential+fadvise_nocache],write 16285696~81920 [fadvise_sequential+fadvise_nocache],write 16498688~81920 [fadvise_sequential+fadvise_nocache],write 16711680~65536 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:01:44.288 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.298669+0000", 2024-09-21T15:01:44.288 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 98.969270738000006, 2024-09-21T15:01:44.288 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.10282358900000001, 2024-09-21T15:01:44.288 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:44.288 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:44.288 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:44.288 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:44.288 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:44.288 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97672 2024-09-21T15:01:44.288 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.288 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:44.288 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.288 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:44.289 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.298669+0000", 2024-09-21T15:01:44.289 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:44.289 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.289 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.289 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:44.289 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.298669+0000", 2024-09-21T15:01:44.289 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:44.289 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.289 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.289 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:44.289 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.298667+0000", 2024-09-21T15:01:44.289 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:01:44.289 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.289 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.289 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:44.289 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.327458+0000", 2024-09-21T15:01:44.289 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.028790754000000002 2024-09-21T15:01:44.290 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.290 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.290 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:44.290 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.327492+0000", 2024-09-21T15:01:44.290 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3806999999999997e-05 2024-09-21T15:01:44.290 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.290 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.290 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:44.290 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.327507+0000", 2024-09-21T15:01:44.290 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5112e-05 2024-09-21T15:01:44.290 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.290 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.290 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:44.291 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.327597+0000", 2024-09-21T15:01:44.291 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9938999999999994e-05 2024-09-21T15:01:44.291 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.291 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.291 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:44.291 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.327927+0000", 2024-09-21T15:01:44.291 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000329766 2024-09-21T15:01:44.291 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.291 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.291 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T15:01:44.291 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.330735+0000", 2024-09-21T15:01:44.291 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0028086869999999998 2024-09-21T15:01:44.291 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.291 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.291 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:01:44.291 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.378299+0000", 2024-09-21T15:01:44.292 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.047563495999999997 2024-09-21T15:01:44.292 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.292 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.292 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:01:44.292 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.401248+0000", 2024-09-21T15:01:44.292 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.022949311 2024-09-21T15:01:44.292 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.292 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.292 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:01:44.292 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.401446+0000", 2024-09-21T15:01:44.292 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019831199999999999 2024-09-21T15:01:44.292 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.292 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.292 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:44.292 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.401492+0000", 2024-09-21T15:01:44.292 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6109000000000002e-05 2024-09-21T15:01:44.293 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:44.293 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:44.293 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:44.293 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.293 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.293 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139769 3.1e 3:793dcf44:::rbd_data.113664780c85.0000000000000000:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:44.293 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.221432+0000", 2024-09-21T15:01:44.293 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.046507869000003, 2024-09-21T15:01:44.293 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.035363362, 2024-09-21T15:01:44.293 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:44.293 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:44.293 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:44.293 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:44.293 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:44.293 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139769 2024-09-21T15:01:44.294 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.294 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:44.294 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.294 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:44.294 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221432+0000", 2024-09-21T15:01:44.294 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:44.294 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.294 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.294 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:44.294 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221432+0000", 2024-09-21T15:01:44.294 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:44.294 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.294 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.295 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:44.295 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221430+0000", 2024-09-21T15:01:44.295 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:01:44.295 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.295 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.295 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:44.295 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221438+0000", 2024-09-21T15:01:44.295 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2630000000000003e-06 2024-09-21T15:01:44.295 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.295 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.295 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:44.295 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221458+0000", 2024-09-21T15:01:44.295 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0071000000000001e-05 2024-09-21T15:01:44.295 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.295 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.296 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:44.296 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221467+0000", 2024-09-21T15:01:44.296 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.816e-06 2024-09-21T15:01:44.296 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.296 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.296 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:44.296 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221572+0000", 2024-09-21T15:01:44.296 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000105166 2024-09-21T15:01:44.296 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.296 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.296 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:44.296 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221589+0000", 2024-09-21T15:01:44.296 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7391e-05 2024-09-21T15:01:44.296 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.296 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.297 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:44.297 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.255741+0000", 2024-09-21T15:01:44.297 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0341519690000001 2024-09-21T15:01:44.297 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.297 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.297 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:44.297 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.256353+0000", 2024-09-21T15:01:44.297 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000611544 2024-09-21T15:01:44.297 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.297 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.297 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:44.297 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.256795+0000", 2024-09-21T15:01:44.297 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00044208700000000002 2024-09-21T15:01:44.297 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:44.297 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:44.297 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:44.298 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.298 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.298 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139773 3.1e 3:7fa3054f:::rbd_data.113664780c85.000000000000001c:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:44.298 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.221517+0000", 2024-09-21T15:01:44.298 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.046422825000001, 2024-09-21T15:01:44.298 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0367423140000001, 2024-09-21T15:01:44.298 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:44.298 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:44.298 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:44.298 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:44.298 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:44.298 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139773 2024-09-21T15:01:44.298 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.299 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:44.299 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.299 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:44.299 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221517+0000", 2024-09-21T15:01:44.299 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:44.299 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.299 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.299 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:44.299 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221517+0000", 2024-09-21T15:01:44.299 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:44.299 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.299 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.299 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:44.299 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221516+0000", 2024-09-21T15:01:44.300 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:44.300 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.300 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.300 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:44.300 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221520+0000", 2024-09-21T15:01:44.300 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1030000000000004e-06 2024-09-21T15:01:44.300 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.300 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.300 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:44.300 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221566+0000", 2024-09-21T15:01:44.300 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5512000000000001e-05 2024-09-21T15:01:44.300 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.300 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.300 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:44.300 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221575+0000", 2024-09-21T15:01:44.300 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9630000000000004e-06 2024-09-21T15:01:44.301 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.301 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.301 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:44.301 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221650+0000", 2024-09-21T15:01:44.301 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5531999999999994e-05 2024-09-21T15:01:44.301 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.301 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.301 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:44.301 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221661+0000", 2024-09-21T15:01:44.301 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0878e-05 2024-09-21T15:01:44.301 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.301 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.301 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:44.301 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.257513+0000", 2024-09-21T15:01:44.301 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0358513089999999 2024-09-21T15:01:44.302 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.302 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.302 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:44.302 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.257885+0000", 2024-09-21T15:01:44.302 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00037274299999999998 2024-09-21T15:01:44.302 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.302 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.302 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:44.302 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.258259+0000", 2024-09-21T15:01:44.302 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000373829 2024-09-21T15:01:44.302 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:44.302 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:44.302 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:44.302 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.303 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.303 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139789 3.1e 3:7c803367:::rbd_data.113664780c85.000000000000008c:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:44.303 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.221656+0000", 2024-09-21T15:01:44.303 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.046283193999997, 2024-09-21T15:01:44.303 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.037139595, 2024-09-21T15:01:44.303 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:44.303 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:44.303 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:44.303 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:44.303 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:44.303 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139789 2024-09-21T15:01:44.303 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.303 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:44.303 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.303 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:44.304 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221656+0000", 2024-09-21T15:01:44.304 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:44.304 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.304 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.304 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:44.304 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221656+0000", 2024-09-21T15:01:44.304 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:44.304 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.304 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.304 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:44.304 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221656+0000", 2024-09-21T15:01:44.304 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:44.304 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.304 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.304 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:44.305 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221660+0000", 2024-09-21T15:01:44.305 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0369999999999996e-06 2024-09-21T15:01:44.305 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.305 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.305 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:44.305 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221668+0000", 2024-09-21T15:01:44.305 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3440000000000001e-06 2024-09-21T15:01:44.305 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.305 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.305 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:44.305 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221674+0000", 2024-09-21T15:01:44.305 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0920000000000003e-06 2024-09-21T15:01:44.305 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.305 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.305 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:44.306 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221787+0000", 2024-09-21T15:01:44.306 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000112289 2024-09-21T15:01:44.306 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.306 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.306 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:44.306 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221798+0000", 2024-09-21T15:01:44.306 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1419e-05 2024-09-21T15:01:44.306 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.306 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.306 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:44.306 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.258642+0000", 2024-09-21T15:01:44.306 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.036843851 2024-09-21T15:01:44.306 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.307 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.307 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:44.307 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.258796+0000", 2024-09-21T15:01:44.307 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015414199999999999 2024-09-21T15:01:44.307 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:44.307 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:44.307 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:44.307 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.307 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.307 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139832 3.1e 3:7b675eeb:::rbd_data.113664780c85.00000000000001b3:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:44.307 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.223976+0000", 2024-09-21T15:01:44.307 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.043963460999997, 2024-09-21T15:01:44.307 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0350267980000001, 2024-09-21T15:01:44.307 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:44.308 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:44.308 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:44.308 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:44.308 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:44.308 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139832 2024-09-21T15:01:44.308 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.308 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:44.308 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.308 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:44.308 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223976+0000", 2024-09-21T15:01:44.308 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:44.308 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.308 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.308 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:44.309 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223976+0000", 2024-09-21T15:01:44.309 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:44.309 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.309 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.309 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:44.309 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223976+0000", 2024-09-21T15:01:44.309 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:44.309 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.309 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.309 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:44.309 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223981+0000", 2024-09-21T15:01:44.309 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1560000000000003e-06 2024-09-21T15:01:44.309 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.309 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.310 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:44.310 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223999+0000", 2024-09-21T15:01:44.310 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8808e-05 2024-09-21T15:01:44.310 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.310 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.310 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:44.310 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224008+0000", 2024-09-21T15:01:44.310 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7269999999999996e-06 2024-09-21T15:01:44.310 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.310 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.310 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:44.310 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224208+0000", 2024-09-21T15:01:44.310 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000199614 2024-09-21T15:01:44.310 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.311 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.311 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:44.311 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224218+0000", 2024-09-21T15:01:44.311 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0209e-05 2024-09-21T15:01:44.311 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.311 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.311 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:44.311 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.258839+0000", 2024-09-21T15:01:44.311 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.034621083 2024-09-21T15:01:44.311 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.311 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.311 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:44.311 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.259003+0000", 2024-09-21T15:01:44.311 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016379 2024-09-21T15:01:44.312 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:44.312 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:44.312 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:44.312 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.312 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.312 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139900 3.1e 3:7b440264:::rbd_data.113664780c85.0000000000000013:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:44.312 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.225748+0000", 2024-09-21T15:01:44.312 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.042191459999998, 2024-09-21T15:01:44.312 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0339966009999999, 2024-09-21T15:01:44.312 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:44.312 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:44.312 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:44.312 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:44.312 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:44.313 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139900 2024-09-21T15:01:44.313 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.313 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:44.313 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.313 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:44.313 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.225748+0000", 2024-09-21T15:01:44.313 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:44.313 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.313 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.313 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:44.313 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.225748+0000", 2024-09-21T15:01:44.313 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:44.313 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.313 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.314 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:44.314 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.225746+0000", 2024-09-21T15:01:44.314 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:44.314 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.314 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.314 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:44.314 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.225757+0000", 2024-09-21T15:01:44.314 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.066e-05 2024-09-21T15:01:44.314 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.314 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.314 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:44.314 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.225775+0000", 2024-09-21T15:01:44.314 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7631000000000001e-05 2024-09-21T15:01:44.314 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.315 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.315 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:44.315 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.225788+0000", 2024-09-21T15:01:44.315 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3046e-05 2024-09-21T15:01:44.315 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.315 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.315 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:44.315 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.225881+0000", 2024-09-21T15:01:44.315 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2987999999999998e-05 2024-09-21T15:01:44.315 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.315 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.315 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:44.315 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.225893+0000", 2024-09-21T15:01:44.315 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2398999999999999e-05 2024-09-21T15:01:44.316 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.316 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.316 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:44.316 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.259207+0000", 2024-09-21T15:01:44.316 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.033313696 2024-09-21T15:01:44.316 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.316 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.316 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:44.316 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.259452+0000", 2024-09-21T15:01:44.316 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00024495700000000002 2024-09-21T15:01:44.316 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.316 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.316 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:44.316 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.259745+0000", 2024-09-21T15:01:44.316 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000292868 2024-09-21T15:01:44.317 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:44.317 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:44.317 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:44.317 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.317 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.317 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140008 3.1e 3:7cd8b476:::rbd_data.113664780c85.0000000000000149:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:44.317 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.229555+0000", 2024-09-21T15:01:44.317 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.038384596, 2024-09-21T15:01:44.317 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0304914199999999, 2024-09-21T15:01:44.317 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:44.317 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:44.317 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:44.317 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:44.317 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:44.317 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140008 2024-09-21T15:01:44.318 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.318 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:44.318 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.318 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:44.318 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.229555+0000", 2024-09-21T15:01:44.318 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:44.318 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.318 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.318 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:44.318 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.229555+0000", 2024-09-21T15:01:44.318 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:44.318 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.318 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.318 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:44.318 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.229554+0000", 2024-09-21T15:01:44.319 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:44.319 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.319 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.319 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:44.319 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.229559+0000", 2024-09-21T15:01:44.319 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1059999999999999e-06 2024-09-21T15:01:44.319 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.319 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.319 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:44.319 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.229570+0000", 2024-09-21T15:01:44.319 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0465000000000001e-05 2024-09-21T15:01:44.319 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.319 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.319 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:44.320 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.229581+0000", 2024-09-21T15:01:44.320 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0958000000000001e-05 2024-09-21T15:01:44.320 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.320 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.320 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:44.320 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.229660+0000", 2024-09-21T15:01:44.320 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9103999999999993e-05 2024-09-21T15:01:44.320 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.320 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.320 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:44.320 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.229675+0000", 2024-09-21T15:01:44.320 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5622999999999999e-05 2024-09-21T15:01:44.320 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.320 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.320 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:44.321 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.259816+0000", 2024-09-21T15:01:44.321 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.030140595 2024-09-21T15:01:44.321 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.321 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.321 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:44.321 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.260046+0000", 2024-09-21T15:01:44.321 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00023045300000000001 2024-09-21T15:01:44.321 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:44.321 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:44.321 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:44.321 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.321 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.321 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140037 3.1e 3:7a4585d9:::rbd_data.113664780c85.000000000000005c:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:44.321 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.230439+0000", 2024-09-21T15:01:44.321 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.037500108000003, 2024-09-21T15:01:44.321 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.030277613, 2024-09-21T15:01:44.322 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:44.322 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:44.322 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:44.322 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:44.322 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:44.322 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140037 2024-09-21T15:01:44.322 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.322 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:44.322 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.322 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:44.322 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.230439+0000", 2024-09-21T15:01:44.322 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:44.322 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.323 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.323 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:44.323 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.230439+0000", 2024-09-21T15:01:44.323 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:44.323 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.323 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.323 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:44.323 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.230438+0000", 2024-09-21T15:01:44.323 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:44.323 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.323 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.323 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:44.323 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.230447+0000", 2024-09-21T15:01:44.324 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6270000000000006e-06 2024-09-21T15:01:44.324 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.324 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.324 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:44.324 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.230464+0000", 2024-09-21T15:01:44.324 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7394e-05 2024-09-21T15:01:44.324 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.324 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.324 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:44.324 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.230472+0000", 2024-09-21T15:01:44.324 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0360000000000006e-06 2024-09-21T15:01:44.324 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.324 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.324 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:44.325 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.230567+0000", 2024-09-21T15:01:44.325 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4648000000000006e-05 2024-09-21T15:01:44.325 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.325 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.325 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:44.325 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.230581+0000", 2024-09-21T15:01:44.325 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3799e-05 2024-09-21T15:01:44.325 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.325 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.325 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:44.325 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.260095+0000", 2024-09-21T15:01:44.325 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.029514153 2024-09-21T15:01:44.325 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.325 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.325 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:44.326 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.260356+0000", 2024-09-21T15:01:44.326 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00026118400000000003 2024-09-21T15:01:44.326 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.326 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.326 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:44.326 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.260717+0000", 2024-09-21T15:01:44.326 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00036124799999999999 2024-09-21T15:01:44.326 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:44.326 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:44.326 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:44.326 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.326 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.326 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140061 3.1e 3:7fdd08f8:::rbd_data.113664780c85.0000000000000104:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:44.326 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.232026+0000", 2024-09-21T15:01:44.326 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.035913778000001, 2024-09-21T15:01:44.326 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0290620450000001, 2024-09-21T15:01:44.327 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:44.327 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:44.327 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:44.327 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:44.327 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:44.327 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140061 2024-09-21T15:01:44.327 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.327 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:44.327 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.327 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:44.327 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232026+0000", 2024-09-21T15:01:44.327 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:44.327 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.327 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.327 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:44.328 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232026+0000", 2024-09-21T15:01:44.328 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:44.328 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.328 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.328 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:44.328 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232024+0000", 2024-09-21T15:01:44.328 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:01:44.328 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.328 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.328 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:44.328 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232034+0000", 2024-09-21T15:01:44.328 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0421000000000001e-05 2024-09-21T15:01:44.328 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.328 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.328 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:44.328 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232058+0000", 2024-09-21T15:01:44.328 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3944e-05 2024-09-21T15:01:44.329 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.329 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.329 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:44.329 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232071+0000", 2024-09-21T15:01:44.329 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2816999999999999e-05 2024-09-21T15:01:44.329 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.329 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.329 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:44.329 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232180+0000", 2024-09-21T15:01:44.329 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000109323 2024-09-21T15:01:44.329 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.329 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.329 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:44.329 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232199+0000", 2024-09-21T15:01:44.329 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8606000000000001e-05 2024-09-21T15:01:44.330 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.330 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.330 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:44.330 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.260821+0000", 2024-09-21T15:01:44.330 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0286222270000001 2024-09-21T15:01:44.330 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.330 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.330 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:44.330 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.261088+0000", 2024-09-21T15:01:44.330 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00026663799999999998 2024-09-21T15:01:44.330 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:44.330 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:44.330 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:44.330 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.331 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.331 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140090 3.1e 3:79f99893:::rbd_data.113664780c85.000000000000000b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:44.331 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.233668+0000", 2024-09-21T15:01:44.331 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.034271912999998, 2024-09-21T15:01:44.331 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0281047409999999, 2024-09-21T15:01:44.331 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:44.331 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:44.331 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:44.331 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:44.331 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:44.331 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140090 2024-09-21T15:01:44.331 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.331 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:44.331 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.331 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:44.332 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233668+0000", 2024-09-21T15:01:44.332 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:44.332 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.332 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.332 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:44.332 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233668+0000", 2024-09-21T15:01:44.332 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:44.332 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.332 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.332 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:44.332 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233666+0000", 2024-09-21T15:01:44.332 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:44.332 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.332 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.332 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:44.333 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233673+0000", 2024-09-21T15:01:44.333 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1840000000000002e-06 2024-09-21T15:01:44.333 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.333 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.333 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:44.333 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233689+0000", 2024-09-21T15:01:44.333 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5366000000000002e-05 2024-09-21T15:01:44.333 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.333 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.333 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:44.333 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233697+0000", 2024-09-21T15:01:44.333 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7679999999999997e-06 2024-09-21T15:01:44.333 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.333 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.334 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:44.334 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233772+0000", 2024-09-21T15:01:44.334 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4832000000000005e-05 2024-09-21T15:01:44.334 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.334 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.334 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:44.334 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233790+0000", 2024-09-21T15:01:44.334 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7297999999999999e-05 2024-09-21T15:01:44.334 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.334 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.334 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:44.334 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.261145+0000", 2024-09-21T15:01:44.334 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.027355381 2024-09-21T15:01:44.334 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.335 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.335 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:44.335 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.261466+0000", 2024-09-21T15:01:44.335 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00032105300000000002 2024-09-21T15:01:44.335 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.335 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.335 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:44.335 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.261772+0000", 2024-09-21T15:01:44.335 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00030635100000000001 2024-09-21T15:01:44.335 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:44.335 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:44.335 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:44.335 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.335 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.336 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140128 3.1e 3:78baf82e:::rbd_data.113664780c85.0000000000000115:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:44.336 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.235029+0000", 2024-09-21T15:01:44.336 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.032910659000002, 2024-09-21T15:01:44.336 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.02698942, 2024-09-21T15:01:44.336 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:44.336 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:44.336 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:44.336 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:44.336 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:44.336 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140128 2024-09-21T15:01:44.336 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.336 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:44.336 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.336 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:44.336 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235029+0000", 2024-09-21T15:01:44.337 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:44.337 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.337 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.337 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:44.337 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235029+0000", 2024-09-21T15:01:44.337 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:44.337 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.337 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.337 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:44.337 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235028+0000", 2024-09-21T15:01:44.337 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:44.337 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.337 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.337 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:44.337 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235032+0000", 2024-09-21T15:01:44.337 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0579999999999999e-06 2024-09-21T15:01:44.338 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.338 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.338 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:44.338 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235040+0000", 2024-09-21T15:01:44.338 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2099999999999993e-06 2024-09-21T15:01:44.338 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.338 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.338 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:44.338 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235047+0000", 2024-09-21T15:01:44.338 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1099999999999999e-06 2024-09-21T15:01:44.338 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.338 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.338 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:44.338 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235123+0000", 2024-09-21T15:01:44.339 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6835999999999994e-05 2024-09-21T15:01:44.339 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.339 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.339 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:44.339 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235136+0000", 2024-09-21T15:01:44.339 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2962000000000001e-05 2024-09-21T15:01:44.339 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.339 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.339 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:44.339 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.261827+0000", 2024-09-21T15:01:44.339 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0266906149999999 2024-09-21T15:01:44.339 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.339 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.339 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:44.339 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.262018+0000", 2024-09-21T15:01:44.340 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019132400000000001 2024-09-21T15:01:44.340 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:44.340 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:44.340 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:44.340 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.340 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.340 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140217 3.1e 3:793dcf44:::rbd_data.113664780c85.0000000000000000:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:44.340 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.235497+0000", 2024-09-21T15:01:44.340 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.032442359999997, 2024-09-21T15:01:44.340 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0269053020000001, 2024-09-21T15:01:44.340 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:44.340 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:44.340 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:44.340 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:44.340 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:44.340 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140217 2024-09-21T15:01:44.341 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.341 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:44.341 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.341 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:44.341 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235497+0000", 2024-09-21T15:01:44.341 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:44.341 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.341 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.341 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:44.341 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235497+0000", 2024-09-21T15:01:44.341 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:44.341 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.341 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.341 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:44.341 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235496+0000", 2024-09-21T15:01:44.341 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:44.342 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.342 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.342 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:44.342 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235503+0000", 2024-09-21T15:01:44.342 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8460000000000002e-06 2024-09-21T15:01:44.342 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.342 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.342 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:44.342 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235514+0000", 2024-09-21T15:01:44.342 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0576e-05 2024-09-21T15:01:44.342 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.342 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.342 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:44.342 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235522+0000", 2024-09-21T15:01:44.343 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2640000000000008e-06 2024-09-21T15:01:44.343 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.343 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.343 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:44.343 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235579+0000", 2024-09-21T15:01:44.343 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6842999999999998e-05 2024-09-21T15:01:44.343 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.343 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.343 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:44.343 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235590+0000", 2024-09-21T15:01:44.343 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1194e-05 2024-09-21T15:01:44.343 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.343 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.343 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:44.343 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.262119+0000", 2024-09-21T15:01:44.344 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0265294540000001 2024-09-21T15:01:44.344 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.344 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.344 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:44.344 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.262241+0000", 2024-09-21T15:01:44.344 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012182 2024-09-21T15:01:44.344 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.344 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.344 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:44.344 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.262403+0000", 2024-09-21T15:01:44.344 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000161442 2024-09-21T15:01:44.344 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:44.344 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:44.344 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:44.344 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.345 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.345 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140221 3.1e 3:7fa3054f:::rbd_data.113664780c85.000000000000001c:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:44.345 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.235592+0000", 2024-09-21T15:01:44.345 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.032347498999997, 2024-09-21T15:01:44.345 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.027322012, 2024-09-21T15:01:44.345 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:44.345 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:44.345 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:44.345 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:44.345 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:44.345 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140221 2024-09-21T15:01:44.345 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.345 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:44.346 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.346 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:44.346 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235592+0000", 2024-09-21T15:01:44.346 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:44.346 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.346 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.346 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:44.346 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235592+0000", 2024-09-21T15:01:44.346 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:44.346 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.346 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.346 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:44.346 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235591+0000", 2024-09-21T15:01:44.346 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:44.347 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.347 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.347 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:44.347 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235596+0000", 2024-09-21T15:01:44.347 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4209999999999997e-06 2024-09-21T15:01:44.347 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.347 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.347 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:44.347 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235603+0000", 2024-09-21T15:01:44.347 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.909e-06 2024-09-21T15:01:44.347 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.347 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.347 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:44.347 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235609+0000", 2024-09-21T15:01:44.347 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3539999999999999e-06 2024-09-21T15:01:44.348 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.348 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.348 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:44.348 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236013+0000", 2024-09-21T15:01:44.348 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00040399799999999998 2024-09-21T15:01:44.348 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.348 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.348 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:44.348 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236020+0000", 2024-09-21T15:01:44.348 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6699999999999994e-06 2024-09-21T15:01:44.348 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.348 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.348 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:44.348 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.262452+0000", 2024-09-21T15:01:44.348 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.02643145 2024-09-21T15:01:44.349 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.349 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.349 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:44.349 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.262624+0000", 2024-09-21T15:01:44.349 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000172206 2024-09-21T15:01:44.349 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.349 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.349 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:44.349 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.262914+0000", 2024-09-21T15:01:44.349 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00028994600000000002 2024-09-21T15:01:44.349 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:44.349 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:44.349 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:44.349 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.350 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.350 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140237 3.1e 3:7c803367:::rbd_data.113664780c85.000000000000008c:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:44.350 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.237113+0000", 2024-09-21T15:01:44.350 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.030826992000001, 2024-09-21T15:01:44.350 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0260336299999999, 2024-09-21T15:01:44.350 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:44.350 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:44.350 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:44.350 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:44.350 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:44.350 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140237 2024-09-21T15:01:44.350 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.350 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:44.350 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.351 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:44.351 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237113+0000", 2024-09-21T15:01:44.351 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:44.351 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.351 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.351 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:44.351 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237113+0000", 2024-09-21T15:01:44.351 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:44.351 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.351 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.351 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:44.351 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237112+0000", 2024-09-21T15:01:44.351 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:44.351 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.352 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.352 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:44.352 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237118+0000", 2024-09-21T15:01:44.352 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8930000000000002e-06 2024-09-21T15:01:44.352 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.352 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.352 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:44.352 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237126+0000", 2024-09-21T15:01:44.352 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6729999999999998e-06 2024-09-21T15:01:44.352 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.352 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.352 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:44.352 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237135+0000", 2024-09-21T15:01:44.352 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0929999999999992e-06 2024-09-21T15:01:44.353 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.353 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.353 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:44.353 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237205+0000", 2024-09-21T15:01:44.353 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9697999999999996e-05 2024-09-21T15:01:44.353 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.353 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.353 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:44.353 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237218+0000", 2024-09-21T15:01:44.353 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2524000000000001e-05 2024-09-21T15:01:44.353 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.353 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.353 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:44.353 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.262965+0000", 2024-09-21T15:01:44.353 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0257478630000001 2024-09-21T15:01:44.354 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.354 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.354 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:44.354 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.263146+0000", 2024-09-21T15:01:44.354 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018074599999999999 2024-09-21T15:01:44.354 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:44.354 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:44.354 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:44.354 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.354 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.354 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140280 3.1e 3:7b675eeb:::rbd_data.113664780c85.00000000000001b3:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:44.354 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.237477+0000", 2024-09-21T15:01:44.354 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.030462730000004, 2024-09-21T15:01:44.354 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.026437329, 2024-09-21T15:01:44.355 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:44.355 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:44.355 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:44.355 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:44.355 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:44.355 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140280 2024-09-21T15:01:44.355 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.355 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:44.355 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.355 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:44.355 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237477+0000", 2024-09-21T15:01:44.355 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:44.355 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.355 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.355 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:44.356 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237477+0000", 2024-09-21T15:01:44.356 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:44.356 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.356 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.356 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:44.356 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237476+0000", 2024-09-21T15:01:44.356 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:44.356 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.356 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.356 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:44.356 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237480+0000", 2024-09-21T15:01:44.356 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7570000000000002e-06 2024-09-21T15:01:44.356 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.356 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.356 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:44.357 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237490+0000", 2024-09-21T15:01:44.357 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3170000000000008e-06 2024-09-21T15:01:44.357 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.357 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.357 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:44.357 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237496+0000", 2024-09-21T15:01:44.357 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7150000000000001e-06 2024-09-21T15:01:44.357 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.357 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.357 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:44.357 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237559+0000", 2024-09-21T15:01:44.357 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2844999999999999e-05 2024-09-21T15:01:44.357 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.357 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.357 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:44.358 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237570+0000", 2024-09-21T15:01:44.358 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1253000000000001e-05 2024-09-21T15:01:44.358 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.358 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.358 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:44.358 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.263203+0000", 2024-09-21T15:01:44.358 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0256330339999999 2024-09-21T15:01:44.358 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.358 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.358 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:44.358 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.263914+0000", 2024-09-21T15:01:44.358 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00071080899999999996 2024-09-21T15:01:44.358 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:44.358 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:44.358 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:44.359 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.359 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.359 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140348 3.1e 3:7b440264:::rbd_data.113664780c85.0000000000000013:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:44.359 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.241762+0000", 2024-09-21T15:01:44.359 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.026177459000003, 2024-09-21T15:01:44.359 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0221355759999999, 2024-09-21T15:01:44.359 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:44.359 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:44.359 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:44.359 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:44.359 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:44.359 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140348 2024-09-21T15:01:44.359 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.360 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:44.360 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.360 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:44.360 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.241762+0000", 2024-09-21T15:01:44.360 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:44.360 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.360 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.360 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:44.360 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.241762+0000", 2024-09-21T15:01:44.360 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:44.360 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.360 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.360 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:44.360 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.241761+0000", 2024-09-21T15:01:44.360 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:44.361 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.361 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.361 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:44.361 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.241767+0000", 2024-09-21T15:01:44.361 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4219999999999997e-06 2024-09-21T15:01:44.361 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.361 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.361 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:44.361 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.241783+0000", 2024-09-21T15:01:44.361 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5959000000000001e-05 2024-09-21T15:01:44.361 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.361 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.361 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:44.361 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.241793+0000", 2024-09-21T15:01:44.362 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3999999999999998e-06 2024-09-21T15:01:44.362 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.362 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.362 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:44.362 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.241894+0000", 2024-09-21T15:01:44.362 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000100976 2024-09-21T15:01:44.362 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.362 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.362 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:44.362 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.241907+0000", 2024-09-21T15:01:44.362 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3281e-05 2024-09-21T15:01:44.362 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.362 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.362 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:44.363 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.263430+0000", 2024-09-21T15:01:44.363 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0215229379999999 2024-09-21T15:01:44.363 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.363 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.363 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:44.363 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.263604+0000", 2024-09-21T15:01:44.363 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017368700000000001 2024-09-21T15:01:44.363 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.363 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.363 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:44.363 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.263898+0000", 2024-09-21T15:01:44.363 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00029406899999999999 2024-09-21T15:01:44.363 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:44.363 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:44.363 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:44.364 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:44.364 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:44.364 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:01:44.365 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T15:01:44.366 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:01:44.360+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:01:44.466 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:01:44.466+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:01:44.488 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:01:44.489 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:01:44.489 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T15:01:44.489 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:01:44.499 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T15:01:44.567 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:01:44.560+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:01:44.606 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:01:44.606 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:01:44.606 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T15:01:44.606 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T15:01:44.606 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:01:44.616 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T15:01:44.668 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:01:44.664+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:01:44.729 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:01:44.730 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T15:01:44.730 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T15:01:44.730 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T15:01:44.730 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.730 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:97677 3.15 e122/24 3:a8e72133:::rbd_data.1136bfc79a70.0000000000000005:head v 122'567, mlcod=120'565)", 2024-09-21T15:01:44.730 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.320753+0000", 2024-09-21T15:01:44.730 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 99.404168083000002, 2024-09-21T15:01:44.730 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070639375000000004, 2024-09-21T15:01:44.730 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:44.730 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:44.730 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:44.730 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.730 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:44.730 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.320753+0000", 2024-09-21T15:01:44.730 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:44.731 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.731 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.731 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:44.731 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.320753+0000", 2024-09-21T15:01:44.731 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:44.731 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.731 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.731 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:44.731 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.320753+0000", 2024-09-21T15:01:44.731 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:44.731 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.731 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.731 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:44.731 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.375452+0000", 2024-09-21T15:01:44.731 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054698945999999998 2024-09-21T15:01:44.732 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.732 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.732 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:44.732 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.375476+0000", 2024-09-21T15:01:44.732 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4380999999999999e-05 2024-09-21T15:01:44.732 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.732 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.732 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:44.732 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.375485+0000", 2024-09-21T15:01:44.732 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0049999999999993e-06 2024-09-21T15:01:44.732 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.732 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.732 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:44.732 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.375552+0000", 2024-09-21T15:01:44.732 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6550999999999994e-05 2024-09-21T15:01:44.732 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.733 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.733 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:44.733 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.375578+0000", 2024-09-21T15:01:44.733 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6196000000000001e-05 2024-09-21T15:01:44.733 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.733 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.733 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:01:44.733 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.391319+0000", 2024-09-21T15:01:44.733 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015741271000000001 2024-09-21T15:01:44.733 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.733 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.733 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:44.733 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.391393+0000", 2024-09-21T15:01:44.733 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3287000000000005e-05 2024-09-21T15:01:44.733 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:44.734 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:44.734 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:44.734 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.734 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.734 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:97672 3.4 e122/51 3:26046d29:::rbd_data.1136bfc79a70.0000000000000001:head v 122'256, mlcod=120'254)", 2024-09-21T15:01:44.734 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.331696+0000", 2024-09-21T15:01:44.734 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 99.393224919999994, 2024-09-21T15:01:44.734 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041222507999999998, 2024-09-21T15:01:44.734 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:44.734 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:44.734 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:44.734 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.734 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:44.734 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.331696+0000", 2024-09-21T15:01:44.735 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:44.735 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.735 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.735 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:44.735 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.331696+0000", 2024-09-21T15:01:44.735 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:44.735 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.735 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.735 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:44.735 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.331695+0000", 2024-09-21T15:01:44.735 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:44.735 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.735 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.735 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:44.736 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.340175+0000", 2024-09-21T15:01:44.736 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0084799520000000007 2024-09-21T15:01:44.736 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.736 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.736 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:44.736 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.340208+0000", 2024-09-21T15:01:44.736 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2997999999999998e-05 2024-09-21T15:01:44.736 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.736 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.736 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:44.736 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.340223+0000", 2024-09-21T15:01:44.736 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5330999999999999e-05 2024-09-21T15:01:44.736 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.736 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.736 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:44.737 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.340305+0000", 2024-09-21T15:01:44.737 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2095999999999995e-05 2024-09-21T15:01:44.737 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.737 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.737 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:44.737 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.340337+0000", 2024-09-21T15:01:44.737 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2097000000000001e-05 2024-09-21T15:01:44.737 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.737 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.737 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:01:44.737 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.372863+0000", 2024-09-21T15:01:44.737 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.032525923999999998 2024-09-21T15:01:44.737 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.737 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.737 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:44.738 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.372919+0000", 2024-09-21T15:01:44.738 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5729000000000001e-05 2024-09-21T15:01:44.738 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:44.738 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:44.738 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:44.738 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.738 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.738 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:116182 3.12 3:4efbaa3f:::rbd_data.1136b1194d20.0000000000000006:58 [read 6242304~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:44.738 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.712538+0000", 2024-09-21T15:01:44.738 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 75.012382928999997, 2024-09-21T15:01:44.738 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034702127999999999, 2024-09-21T15:01:44.738 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:44.738 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:44.738 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:44.739 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:44.739 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:44.739 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116182 2024-09-21T15:01:44.739 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.739 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:44.739 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.739 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:44.739 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712538+0000", 2024-09-21T15:01:44.739 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:44.739 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.739 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.739 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:44.739 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712538+0000", 2024-09-21T15:01:44.739 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:44.740 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.740 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.740 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:44.740 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712538+0000", 2024-09-21T15:01:44.740 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:44.740 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.740 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.740 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:44.740 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712541+0000", 2024-09-21T15:01:44.740 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5330000000000001e-06 2024-09-21T15:01:44.740 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.740 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.740 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:44.740 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712547+0000", 2024-09-21T15:01:44.740 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1109999999999996e-06 2024-09-21T15:01:44.740 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.740 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.741 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:44.741 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712551+0000", 2024-09-21T15:01:44.741 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.685e-06 2024-09-21T15:01:44.741 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.741 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.741 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:44.741 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.747067+0000", 2024-09-21T15:01:44.741 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034516181 2024-09-21T15:01:44.741 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.741 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.741 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:44.741 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.747152+0000", 2024-09-21T15:01:44.741 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4075999999999999e-05 2024-09-21T15:01:44.741 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.741 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.741 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:44.742 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.747240+0000", 2024-09-21T15:01:44.742 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8924000000000004e-05 2024-09-21T15:01:44.742 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:44.742 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:44.742 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:44.742 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.742 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.742 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139784 3.1 3:862e0e65:::rbd_data.113664780c85.0000000000000063:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:44.742 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.222486+0000", 2024-09-21T15:01:44.742 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.502435062000004, 2024-09-21T15:01:44.742 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0141793059999999, 2024-09-21T15:01:44.742 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:44.742 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:44.742 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:44.742 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:44.743 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:44.743 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139784 2024-09-21T15:01:44.743 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.743 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:44.743 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.743 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:44.743 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222486+0000", 2024-09-21T15:01:44.743 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:44.743 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.743 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.743 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:44.743 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222486+0000", 2024-09-21T15:01:44.743 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:44.743 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.743 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.744 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:44.744 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222484+0000", 2024-09-21T15:01:44.744 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-21T15:01:44.744 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.744 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.744 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:44.744 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222494+0000", 2024-09-21T15:01:44.744 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0676000000000001e-05 2024-09-21T15:01:44.744 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.744 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.744 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:44.744 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222517+0000", 2024-09-21T15:01:44.744 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2371e-05 2024-09-21T15:01:44.744 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.744 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.745 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:44.745 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222527+0000", 2024-09-21T15:01:44.745 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0461e-05 2024-09-21T15:01:44.745 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.745 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.745 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:44.745 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222625+0000", 2024-09-21T15:01:44.745 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.8160999999999997e-05 2024-09-21T15:01:44.745 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.745 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.745 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:44.745 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222641+0000", 2024-09-21T15:01:44.745 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5899000000000001e-05 2024-09-21T15:01:44.746 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.746 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.746 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:44.746 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.236479+0000", 2024-09-21T15:01:44.746 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0138377709999999 2024-09-21T15:01:44.746 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.746 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.746 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:44.746 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.236666+0000", 2024-09-21T15:01:44.746 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018648199999999999 2024-09-21T15:01:44.746 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:44.746 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:44.746 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:44.746 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.747 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.747 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139812 3.1 3:8407fb9b:::rbd_data.113664780c85.0000000000000127:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:44.747 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.222547+0000", 2024-09-21T15:01:44.747 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.502374138, 2024-09-21T15:01:44.747 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.014796059, 2024-09-21T15:01:44.747 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:44.747 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:44.747 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:44.747 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:44.747 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:44.747 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139812 2024-09-21T15:01:44.747 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.747 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:44.747 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.747 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:44.748 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222547+0000", 2024-09-21T15:01:44.748 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:44.748 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.748 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.748 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:44.748 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222547+0000", 2024-09-21T15:01:44.748 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:44.748 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.748 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.748 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:44.748 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222546+0000", 2024-09-21T15:01:44.748 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:44.748 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.748 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.749 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:44.749 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222552+0000", 2024-09-21T15:01:44.749 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2739999999999998e-06 2024-09-21T15:01:44.749 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.749 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.749 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:44.749 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222578+0000", 2024-09-21T15:01:44.749 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6594999999999999e-05 2024-09-21T15:01:44.749 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.749 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.749 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:44.749 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222596+0000", 2024-09-21T15:01:44.749 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7838000000000001e-05 2024-09-21T15:01:44.749 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.750 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.750 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:44.750 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222701+0000", 2024-09-21T15:01:44.750 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000104714 2024-09-21T15:01:44.750 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.750 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.750 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:44.750 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222714+0000", 2024-09-21T15:01:44.750 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2998e-05 2024-09-21T15:01:44.750 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.750 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.750 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:44.750 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.237124+0000", 2024-09-21T15:01:44.750 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.014409806 2024-09-21T15:01:44.751 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.751 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.751 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:44.751 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.237343+0000", 2024-09-21T15:01:44.751 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021950499999999999 2024-09-21T15:01:44.751 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:44.751 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:44.751 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:44.751 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.751 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.751 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139826 3.1 3:854f6f9b:::rbd_data.113664780c85.0000000000000189:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:44.751 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.222611+0000", 2024-09-21T15:01:44.751 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.502310045999998, 2024-09-21T15:01:44.751 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.014916828, 2024-09-21T15:01:44.751 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:44.751 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:44.752 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:44.752 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:44.752 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:44.752 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139826 2024-09-21T15:01:44.752 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.752 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:44.752 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.752 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:44.752 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222611+0000", 2024-09-21T15:01:44.752 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:44.752 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.752 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.752 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:44.752 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222611+0000", 2024-09-21T15:01:44.753 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:44.753 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.753 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.753 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:44.753 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222610+0000", 2024-09-21T15:01:44.753 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:44.753 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.753 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.753 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:44.753 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222618+0000", 2024-09-21T15:01:44.753 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6690000000000006e-06 2024-09-21T15:01:44.753 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.753 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.753 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:44.754 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222632+0000", 2024-09-21T15:01:44.754 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4433999999999999e-05 2024-09-21T15:01:44.754 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.754 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.754 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:44.754 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222640+0000", 2024-09-21T15:01:44.754 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6260000000000003e-06 2024-09-21T15:01:44.754 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.754 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.754 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:44.754 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222758+0000", 2024-09-21T15:01:44.754 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011826499999999999 2024-09-21T15:01:44.754 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.754 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.755 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:44.755 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222776+0000", 2024-09-21T15:01:44.755 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7564999999999999e-05 2024-09-21T15:01:44.755 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.755 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.755 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:44.755 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.237402+0000", 2024-09-21T15:01:44.755 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0146261889999999 2024-09-21T15:01:44.755 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.755 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.755 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:44.755 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.237528+0000", 2024-09-21T15:01:44.755 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000126212 2024-09-21T15:01:44.755 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:44.755 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:44.756 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:44.756 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.756 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.756 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139864 3.1 3:86b14cb7:::rbd_data.113664780c85.00000000000000d5:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:44.756 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.223966+0000", 2024-09-21T15:01:44.756 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.500955310000002, 2024-09-21T15:01:44.756 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.013859007, 2024-09-21T15:01:44.756 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:44.756 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:44.756 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:44.756 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:44.756 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:44.756 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139864 2024-09-21T15:01:44.756 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.756 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:44.757 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.757 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:44.757 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223966+0000", 2024-09-21T15:01:44.757 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:44.757 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.757 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.757 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:44.757 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223966+0000", 2024-09-21T15:01:44.757 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:44.757 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.757 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.757 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:44.757 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223965+0000", 2024-09-21T15:01:44.757 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:44.757 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.758 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.758 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:44.758 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223970+0000", 2024-09-21T15:01:44.758 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3769999999999998e-06 2024-09-21T15:01:44.758 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.758 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.758 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:44.758 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223979+0000", 2024-09-21T15:01:44.758 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5899999999999997e-06 2024-09-21T15:01:44.758 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.758 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.758 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:44.758 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223985+0000", 2024-09-21T15:01:44.758 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5790000000000003e-06 2024-09-21T15:01:44.758 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.759 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.759 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:44.759 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224073+0000", 2024-09-21T15:01:44.759 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7781000000000005e-05 2024-09-21T15:01:44.759 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.759 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.759 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:44.759 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224110+0000", 2024-09-21T15:01:44.759 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7060000000000001e-05 2024-09-21T15:01:44.759 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.759 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.759 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:44.759 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.237599+0000", 2024-09-21T15:01:44.759 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.01348953 2024-09-21T15:01:44.759 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.760 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.760 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:44.760 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.237825+0000", 2024-09-21T15:01:44.760 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000225758 2024-09-21T15:01:44.760 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:44.760 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:44.760 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:44.760 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.760 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.760 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139877 3.1 3:87277f92:::rbd_data.113664780c85.0000000000000136:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:44.760 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.224489+0000", 2024-09-21T15:01:44.760 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.500432451999998, 2024-09-21T15:01:44.760 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0137916330000001, 2024-09-21T15:01:44.760 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:44.760 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:44.761 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:44.761 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:44.761 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:44.761 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139877 2024-09-21T15:01:44.761 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.761 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:44.761 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.761 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:44.761 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224489+0000", 2024-09-21T15:01:44.761 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:44.761 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.761 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.761 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:44.761 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224489+0000", 2024-09-21T15:01:44.761 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:44.761 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.762 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.762 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:44.762 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224488+0000", 2024-09-21T15:01:44.762 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:44.762 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.762 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.762 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:44.762 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224502+0000", 2024-09-21T15:01:44.762 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4194e-05 2024-09-21T15:01:44.762 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.762 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.762 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:44.762 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224522+0000", 2024-09-21T15:01:44.762 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0038e-05 2024-09-21T15:01:44.762 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.763 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.763 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:44.763 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224531+0000", 2024-09-21T15:01:44.763 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6900000000000004e-06 2024-09-21T15:01:44.763 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.763 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.763 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:44.763 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224621+0000", 2024-09-21T15:01:44.763 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9654000000000006e-05 2024-09-21T15:01:44.763 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.763 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.763 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:44.763 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224641+0000", 2024-09-21T15:01:44.763 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9375e-05 2024-09-21T15:01:44.763 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.763 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.764 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:44.764 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.238027+0000", 2024-09-21T15:01:44.764 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0133866840000001 2024-09-21T15:01:44.764 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.764 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.764 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:44.764 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.238280+0000", 2024-09-21T15:01:44.764 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000253281 2024-09-21T15:01:44.764 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:44.764 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:44.764 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:44.764 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.764 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.764 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139887 3.1 3:80f1e8f1:::rbd_data.113664780c85.000000000000017c:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:44.764 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.224549+0000", 2024-09-21T15:01:44.765 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.500372161999998, 2024-09-21T15:01:44.765 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0139776060000001, 2024-09-21T15:01:44.765 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:44.765 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:44.765 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:44.765 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:44.765 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:44.765 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139887 2024-09-21T15:01:44.765 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.765 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:44.765 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.765 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:44.765 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224549+0000", 2024-09-21T15:01:44.765 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:44.765 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.765 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.766 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:44.766 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224549+0000", 2024-09-21T15:01:44.766 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:44.766 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.766 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.766 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:44.766 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224548+0000", 2024-09-21T15:01:44.766 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:44.766 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.766 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.766 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:44.766 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224554+0000", 2024-09-21T15:01:44.766 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4049999999999999e-06 2024-09-21T15:01:44.766 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.767 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.767 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:44.767 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224565+0000", 2024-09-21T15:01:44.767 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0879e-05 2024-09-21T15:01:44.767 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.767 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.767 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:44.767 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224572+0000", 2024-09-21T15:01:44.767 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0380000000000002e-06 2024-09-21T15:01:44.767 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.767 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.767 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:44.767 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224684+0000", 2024-09-21T15:01:44.767 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011229 2024-09-21T15:01:44.767 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.767 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.768 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:44.768 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224698+0000", 2024-09-21T15:01:44.768 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4448e-05 2024-09-21T15:01:44.768 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.768 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.768 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:44.768 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.238359+0000", 2024-09-21T15:01:44.768 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0136610660000001 2024-09-21T15:01:44.768 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.768 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.768 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:44.768 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.238527+0000", 2024-09-21T15:01:44.768 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016731100000000001 2024-09-21T15:01:44.768 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:44.768 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:44.768 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:44.769 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.769 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.769 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140077 3.1 3:85642daf:::rbd_data.113664780c85.0000000000000174:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:44.769 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.232797+0000", 2024-09-21T15:01:44.769 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.492124447000002, 2024-09-21T15:01:44.769 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.006029955, 2024-09-21T15:01:44.769 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:44.769 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:44.769 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:44.769 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:44.769 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:44.769 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140077 2024-09-21T15:01:44.769 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.769 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:44.770 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.770 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:44.770 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232797+0000", 2024-09-21T15:01:44.770 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:44.770 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.770 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.770 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:44.770 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232797+0000", 2024-09-21T15:01:44.770 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:44.770 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.770 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.770 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:44.770 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232795+0000", 2024-09-21T15:01:44.770 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:44.771 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.771 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.771 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:44.771 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232804+0000", 2024-09-21T15:01:44.771 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9670000000000007e-06 2024-09-21T15:01:44.771 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.771 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.771 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:44.771 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232824+0000", 2024-09-21T15:01:44.771 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9238000000000001e-05 2024-09-21T15:01:44.771 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.771 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.771 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:44.771 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232843+0000", 2024-09-21T15:01:44.771 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9256e-05 2024-09-21T15:01:44.772 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.772 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.772 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:44.772 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232912+0000", 2024-09-21T15:01:44.772 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9589000000000001e-05 2024-09-21T15:01:44.772 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.772 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.772 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:44.772 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232925+0000", 2024-09-21T15:01:44.772 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2585999999999999e-05 2024-09-21T15:01:44.772 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.772 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.772 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:44.772 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.238591+0000", 2024-09-21T15:01:44.772 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0056659939999999 2024-09-21T15:01:44.773 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.773 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.773 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:44.773 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.238827+0000", 2024-09-21T15:01:44.773 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00023583999999999999 2024-09-21T15:01:44.773 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:44.773 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:44.773 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:44.773 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.773 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.773 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140091 3.1 3:8358f0db:::rbd_data.113664780c85.0000000000000018:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:44.773 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.233637+0000", 2024-09-21T15:01:44.773 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.491284151000002, 2024-09-21T15:01:44.773 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.006272847, 2024-09-21T15:01:44.773 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:44.774 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:44.774 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:44.774 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:44.774 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:44.774 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140091 2024-09-21T15:01:44.774 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.774 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:44.774 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.774 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:44.774 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233637+0000", 2024-09-21T15:01:44.774 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:44.774 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.774 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.775 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:44.775 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233637+0000", 2024-09-21T15:01:44.775 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:44.775 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.775 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.775 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:44.775 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233636+0000", 2024-09-21T15:01:44.775 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:44.775 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.775 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.775 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:44.775 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233644+0000", 2024-09-21T15:01:44.775 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1380000000000007e-06 2024-09-21T15:01:44.775 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.775 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.775 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:44.776 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233662+0000", 2024-09-21T15:01:44.776 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8224999999999999e-05 2024-09-21T15:01:44.776 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.776 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.776 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:44.776 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233674+0000", 2024-09-21T15:01:44.776 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2081e-05 2024-09-21T15:01:44.776 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.776 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.776 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:44.776 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233750+0000", 2024-09-21T15:01:44.776 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6106000000000006e-05 2024-09-21T15:01:44.776 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.776 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.777 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:44.777 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233768+0000", 2024-09-21T15:01:44.777 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7285000000000001e-05 2024-09-21T15:01:44.777 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.777 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.777 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:44.777 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.239196+0000", 2024-09-21T15:01:44.777 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.005428223 2024-09-21T15:01:44.777 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.777 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.777 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:44.777 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.239528+0000", 2024-09-21T15:01:44.777 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00033262199999999997 2024-09-21T15:01:44.777 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.777 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.778 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:44.778 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.239910+0000", 2024-09-21T15:01:44.778 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00038151100000000001 2024-09-21T15:01:44.778 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:44.778 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:44.778 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:44.778 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.778 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.778 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140109 3.1 3:8586e9d9:::rbd_data.113664780c85.0000000000000096:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:44.778 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.234171+0000", 2024-09-21T15:01:44.778 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.490750317, 2024-09-21T15:01:44.778 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.006019896, 2024-09-21T15:01:44.778 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:44.778 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:44.778 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:44.779 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:44.779 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:44.779 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140109 2024-09-21T15:01:44.779 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.779 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:44.779 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.779 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:44.779 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.234171+0000", 2024-09-21T15:01:44.779 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:44.779 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.779 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.779 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:44.779 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.234171+0000", 2024-09-21T15:01:44.779 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:44.779 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.780 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.780 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:44.780 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.234170+0000", 2024-09-21T15:01:44.780 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:44.780 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.780 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.780 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:44.780 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.234178+0000", 2024-09-21T15:01:44.780 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.464e-06 2024-09-21T15:01:44.780 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.780 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.780 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:44.780 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.234191+0000", 2024-09-21T15:01:44.780 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3489000000000001e-05 2024-09-21T15:01:44.780 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.781 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.781 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:44.781 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.234200+0000", 2024-09-21T15:01:44.781 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2779999999999999e-06 2024-09-21T15:01:44.781 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.781 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.781 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:44.781 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.234279+0000", 2024-09-21T15:01:44.781 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8813000000000006e-05 2024-09-21T15:01:44.781 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.781 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.781 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:44.781 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.234298+0000", 2024-09-21T15:01:44.781 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8947999999999999e-05 2024-09-21T15:01:44.781 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.781 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.782 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:44.782 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.239974+0000", 2024-09-21T15:01:44.782 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0056760259999999 2024-09-21T15:01:44.782 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.782 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.782 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:44.782 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.240191+0000", 2024-09-21T15:01:44.782 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000216708 2024-09-21T15:01:44.782 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:44.782 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:44.782 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:44.782 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.782 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.782 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140149 3.1 3:86b1e433:::rbd_data.113664780c85.00000000000001ae:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:44.783 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.235545+0000", 2024-09-21T15:01:44.783 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.489376032000003, 2024-09-21T15:01:44.783 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0049193430000001, 2024-09-21T15:01:44.783 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:44.783 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:44.783 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:44.783 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:44.783 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:44.783 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140149 2024-09-21T15:01:44.783 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.783 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:44.783 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.783 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:44.783 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235545+0000", 2024-09-21T15:01:44.784 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:44.784 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.784 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.784 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:44.784 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235545+0000", 2024-09-21T15:01:44.784 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:44.784 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.784 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.784 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:44.784 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235544+0000", 2024-09-21T15:01:44.784 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:44.784 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.784 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.784 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:44.784 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235554+0000", 2024-09-21T15:01:44.785 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.8870000000000004e-06 2024-09-21T15:01:44.785 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.785 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.785 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:44.785 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235577+0000", 2024-09-21T15:01:44.785 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2946999999999999e-05 2024-09-21T15:01:44.785 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.785 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.785 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:44.785 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235587+0000", 2024-09-21T15:01:44.785 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0326000000000001e-05 2024-09-21T15:01:44.785 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.785 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.785 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:44.785 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235658+0000", 2024-09-21T15:01:44.785 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1360999999999997e-05 2024-09-21T15:01:44.786 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.786 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.786 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:44.786 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235670+0000", 2024-09-21T15:01:44.786 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1735e-05 2024-09-21T15:01:44.786 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.786 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.786 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:44.786 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.240246+0000", 2024-09-21T15:01:44.786 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0045757790000001 2024-09-21T15:01:44.786 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.786 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.786 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:44.786 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.240465+0000", 2024-09-21T15:01:44.787 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021870799999999999 2024-09-21T15:01:44.787 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:44.787 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:44.787 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:44.787 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.787 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.787 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140232 3.1 3:862e0e65:::rbd_data.113664780c85.0000000000000063:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:44.787 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.235760+0000", 2024-09-21T15:01:44.787 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.489161314, 2024-09-21T15:01:44.787 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.004979812, 2024-09-21T15:01:44.787 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:44.787 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:44.787 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:44.787 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:44.787 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:44.788 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140232 2024-09-21T15:01:44.788 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.788 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:44.788 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.788 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:44.788 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235760+0000", 2024-09-21T15:01:44.788 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:44.788 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.788 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.788 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:44.788 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235760+0000", 2024-09-21T15:01:44.788 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:44.788 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.788 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.788 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:44.789 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235759+0000", 2024-09-21T15:01:44.789 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:44.789 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.789 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.789 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:44.789 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235777+0000", 2024-09-21T15:01:44.789 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8215000000000001e-05 2024-09-21T15:01:44.789 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.789 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.789 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:44.789 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235793+0000", 2024-09-21T15:01:44.789 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6461e-05 2024-09-21T15:01:44.789 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.789 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.789 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:44.790 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235803+0000", 2024-09-21T15:01:44.790 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5300000000000002e-06 2024-09-21T15:01:44.790 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.790 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.790 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:44.790 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235875+0000", 2024-09-21T15:01:44.790 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2113000000000006e-05 2024-09-21T15:01:44.790 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.790 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.790 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:44.790 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235887+0000", 2024-09-21T15:01:44.790 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2160999999999999e-05 2024-09-21T15:01:44.790 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.790 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.790 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:44.790 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.240521+0000", 2024-09-21T15:01:44.791 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.004633428 2024-09-21T15:01:44.791 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.791 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.791 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:44.791 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.240740+0000", 2024-09-21T15:01:44.791 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000219228 2024-09-21T15:01:44.791 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:44.791 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:44.791 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:44.791 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.791 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.791 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140260 3.1 3:8407fb9b:::rbd_data.113664780c85.0000000000000127:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:44.791 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.235839+0000", 2024-09-21T15:01:44.791 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.489082144999998, 2024-09-21T15:01:44.791 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.005157981, 2024-09-21T15:01:44.792 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:44.792 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:44.792 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:44.792 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:44.792 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:44.792 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140260 2024-09-21T15:01:44.792 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.792 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:44.792 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.792 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:44.792 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235839+0000", 2024-09-21T15:01:44.792 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:44.792 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.792 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.792 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:44.792 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235839+0000", 2024-09-21T15:01:44.793 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:44.793 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.793 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.793 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:44.793 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235838+0000", 2024-09-21T15:01:44.793 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:44.793 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.793 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.793 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:44.793 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235843+0000", 2024-09-21T15:01:44.793 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.481e-06 2024-09-21T15:01:44.793 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.793 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.793 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:44.793 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235849+0000", 2024-09-21T15:01:44.793 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4880000000000004e-06 2024-09-21T15:01:44.794 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.794 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.794 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:44.794 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235856+0000", 2024-09-21T15:01:44.794 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7649999999999996e-06 2024-09-21T15:01:44.794 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.794 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.794 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:44.794 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235933+0000", 2024-09-21T15:01:44.794 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6987e-05 2024-09-21T15:01:44.794 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.794 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.794 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:44.794 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235949+0000", 2024-09-21T15:01:44.795 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6696000000000001e-05 2024-09-21T15:01:44.795 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.795 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.795 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:44.795 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.240823+0000", 2024-09-21T15:01:44.795 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.004873259 2024-09-21T15:01:44.795 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.795 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.795 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:44.795 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.240997+0000", 2024-09-21T15:01:44.795 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000174365 2024-09-21T15:01:44.795 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:44.795 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:44.795 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:44.795 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.795 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.796 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140274 3.1 3:854f6f9b:::rbd_data.113664780c85.0000000000000189:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:44.796 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.235884+0000", 2024-09-21T15:01:44.796 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.489037197000002, 2024-09-21T15:01:44.796 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.005397366, 2024-09-21T15:01:44.796 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:44.796 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:44.796 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:44.796 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:44.796 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:44.796 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140274 2024-09-21T15:01:44.796 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.796 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:44.796 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.796 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:44.796 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235884+0000", 2024-09-21T15:01:44.797 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:44.797 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.797 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.797 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:44.797 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235884+0000", 2024-09-21T15:01:44.797 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:44.797 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.797 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.797 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:44.797 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235883+0000", 2024-09-21T15:01:44.797 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:44.797 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.797 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.798 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:44.798 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235891+0000", 2024-09-21T15:01:44.798 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9740000000000002e-06 2024-09-21T15:01:44.798 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.798 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.798 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:44.798 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235900+0000", 2024-09-21T15:01:44.798 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6659999999999997e-06 2024-09-21T15:01:44.798 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.798 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.798 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:44.798 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235909+0000", 2024-09-21T15:01:44.798 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3400000000000004e-06 2024-09-21T15:01:44.798 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.798 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.799 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:44.799 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236012+0000", 2024-09-21T15:01:44.799 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010240500000000001 2024-09-21T15:01:44.799 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.799 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.799 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:44.799 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236027+0000", 2024-09-21T15:01:44.799 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5418e-05 2024-09-21T15:01:44.799 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.799 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.799 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:44.799 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.241068+0000", 2024-09-21T15:01:44.799 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0050410729999999 2024-09-21T15:01:44.799 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.800 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.800 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:44.800 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.241281+0000", 2024-09-21T15:01:44.800 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021321900000000001 2024-09-21T15:01:44.800 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:44.800 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:44.800 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:44.800 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.800 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.800 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140312 3.1 3:86b14cb7:::rbd_data.113664780c85.00000000000000d5:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:44.800 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.236769+0000", 2024-09-21T15:01:44.800 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.488152733, 2024-09-21T15:01:44.800 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.004717198, 2024-09-21T15:01:44.800 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:44.800 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:44.800 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:44.801 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:44.801 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:44.801 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140312 2024-09-21T15:01:44.801 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.801 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:44.801 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.801 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:44.801 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236769+0000", 2024-09-21T15:01:44.801 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:44.801 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.801 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.801 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:44.801 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236769+0000", 2024-09-21T15:01:44.801 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:44.801 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.802 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.802 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:44.802 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236768+0000", 2024-09-21T15:01:44.802 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:44.802 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.802 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.802 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:44.802 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236771+0000", 2024-09-21T15:01:44.802 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.331e-06 2024-09-21T15:01:44.802 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.802 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.802 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:44.802 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236777+0000", 2024-09-21T15:01:44.802 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6949999999999997e-06 2024-09-21T15:01:44.802 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.802 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.803 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:44.803 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236787+0000", 2024-09-21T15:01:44.803 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6700000000000006e-06 2024-09-21T15:01:44.803 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.803 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.803 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:44.803 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236846+0000", 2024-09-21T15:01:44.803 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.944e-05 2024-09-21T15:01:44.803 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.803 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.803 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:44.803 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236861+0000", 2024-09-21T15:01:44.803 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4611e-05 2024-09-21T15:01:44.803 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.803 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.803 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:44.804 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.241349+0000", 2024-09-21T15:01:44.804 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0044879250000001 2024-09-21T15:01:44.804 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.804 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.804 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:44.804 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.241486+0000", 2024-09-21T15:01:44.804 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013713000000000001 2024-09-21T15:01:44.804 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:44.804 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:44.804 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:44.804 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.804 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.804 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140325 3.1 3:87277f92:::rbd_data.113664780c85.0000000000000136:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:44.805 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.239351+0000", 2024-09-21T15:01:44.805 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.485570594000002, 2024-09-21T15:01:44.805 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0024334690000001, 2024-09-21T15:01:44.805 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:44.805 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:44.805 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:44.805 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:44.805 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:44.805 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140325 2024-09-21T15:01:44.805 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.805 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:44.805 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.805 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:44.805 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239351+0000", 2024-09-21T15:01:44.805 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:44.806 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.806 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.806 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:44.806 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239351+0000", 2024-09-21T15:01:44.806 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:44.806 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.806 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.806 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:44.806 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239349+0000", 2024-09-21T15:01:44.806 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:44.806 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.806 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.806 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:44.806 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239358+0000", 2024-09-21T15:01:44.806 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2830000000000001e-06 2024-09-21T15:01:44.807 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.807 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.807 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:44.807 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239379+0000", 2024-09-21T15:01:44.807 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1780000000000002e-05 2024-09-21T15:01:44.807 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.807 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.807 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:44.807 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239389+0000", 2024-09-21T15:01:44.807 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0056e-05 2024-09-21T15:01:44.807 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.807 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.807 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:44.807 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239484+0000", 2024-09-21T15:01:44.807 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4611000000000006e-05 2024-09-21T15:01:44.808 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.808 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.808 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:44.808 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239497+0000", 2024-09-21T15:01:44.808 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2909e-05 2024-09-21T15:01:44.808 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.808 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.808 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:44.808 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.241559+0000", 2024-09-21T15:01:44.808 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0020622459999999 2024-09-21T15:01:44.808 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.808 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.808 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:44.808 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.241784+0000", 2024-09-21T15:01:44.808 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000225009 2024-09-21T15:01:44.808 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:44.809 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:44.809 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:44.809 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.809 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.809 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140335 3.1 3:80f1e8f1:::rbd_data.113664780c85.000000000000017c:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:44.809 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.239409+0000", 2024-09-21T15:01:44.809 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 42.485511875, 2024-09-21T15:01:44.809 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0025803150000001, 2024-09-21T15:01:44.809 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:44.809 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:44.809 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:44.809 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:44.809 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:44.809 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140335 2024-09-21T15:01:44.809 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.810 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:44.810 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.810 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:44.810 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239409+0000", 2024-09-21T15:01:44.810 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:44.810 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.810 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.810 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:44.810 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239409+0000", 2024-09-21T15:01:44.810 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:44.810 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.810 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.810 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:44.810 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239409+0000", 2024-09-21T15:01:44.810 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:44.811 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.811 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.811 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:44.811 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239413+0000", 2024-09-21T15:01:44.811 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7620000000000003e-06 2024-09-21T15:01:44.811 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.811 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.811 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:44.811 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239436+0000", 2024-09-21T15:01:44.811 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2889000000000001e-05 2024-09-21T15:01:44.811 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.811 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.811 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:44.811 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239444+0000", 2024-09-21T15:01:44.811 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6809999999999998e-06 2024-09-21T15:01:44.811 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.812 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.812 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:44.812 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239542+0000", 2024-09-21T15:01:44.812 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7868999999999994e-05 2024-09-21T15:01:44.812 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.812 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.812 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:44.812 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239558+0000", 2024-09-21T15:01:44.812 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6257e-05 2024-09-21T15:01:44.812 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.812 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.812 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:44.812 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.241847+0000", 2024-09-21T15:01:44.812 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.002288533 2024-09-21T15:01:44.812 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.812 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.813 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:44.813 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.241990+0000", 2024-09-21T15:01:44.813 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014308300000000001 2024-09-21T15:01:44.813 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:44.813 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:44.813 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:44.813 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.813 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.813 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:166488 3.4 3.7bcf3e84 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e189)", 2024-09-21T15:01:44.813 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:21.342023+0000", 2024-09-21T15:01:44.813 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 23.382898043000001, 2024-09-21T15:01:44.813 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0015098469999999, 2024-09-21T15:01:44.813 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:44.813 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:44.813 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:44.813 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:44.814 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:44.814 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 166488 2024-09-21T15:01:44.814 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.814 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:44.814 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.814 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:44.814 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:21.342023+0000", 2024-09-21T15:01:44.814 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:44.814 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.814 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.814 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:44.814 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:21.342023+0000", 2024-09-21T15:01:44.814 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:44.814 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.814 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.815 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:44.815 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:21.342022+0000", 2024-09-21T15:01:44.815 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:44.815 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.815 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.815 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:44.815 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:21.342029+0000", 2024-09-21T15:01:44.815 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1420000000000004e-06 2024-09-21T15:01:44.815 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.815 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.815 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:44.815 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:21.342051+0000", 2024-09-21T15:01:44.815 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2549999999999999e-05 2024-09-21T15:01:44.815 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.815 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.816 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:44.816 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:21.342059+0000", 2024-09-21T15:01:44.816 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8490000000000006e-06 2024-09-21T15:01:44.816 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.816 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.816 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:44.816 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:21.342105+0000", 2024-09-21T15:01:44.816 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5488999999999998e-05 2024-09-21T15:01:44.816 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.816 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.816 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:44.816 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:21.342115+0000", 2024-09-21T15:01:44.816 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0633e-05 2024-09-21T15:01:44.816 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.816 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.817 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:44.817 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:22.343521+0000", 2024-09-21T15:01:44.817 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0014057270000001 2024-09-21T15:01:44.817 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:44.817 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:44.817 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:44.817 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:22.343533+0000", 2024-09-21T15:01:44.817 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2097e-05 2024-09-21T15:01:44.817 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:44.817 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:44.817 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:44.817 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:44.817 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:44.817 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:01:44.819 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T15:01:44.819 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:01:44.818+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:01:44.919 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:01:44.918+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:01:44.934 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:01:44.934 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:01:44.934 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T15:01:44.934 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:01:44.944 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T15:01:45.021 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:01:45.018+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:01:45.049 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:01:45.049 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:01:45.049 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T15:01:45.049 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T15:01:45.049 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:01:45.059 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T15:01:45.121 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:01:45.122+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:01:45.162 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:01:45.162 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T15:01:45.162 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T15:01:45.162 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T15:01:45.162 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.162 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31476 3.c 3:35718b6b:::rbd_data.11367721c19.0000000000000014:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T15:01:45.162 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.505975+0000", 2024-09-21T15:01:45.162 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 166.653012993, 2024-09-21T15:01:45.162 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99515995899999998, 2024-09-21T15:01:45.162 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:45.163 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:45.163 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:45.163 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:45.163 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:45.163 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31476 2024-09-21T15:01:45.163 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.163 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:45.163 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.163 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:45.163 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.505975+0000", 2024-09-21T15:01:45.163 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:45.163 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.163 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.163 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:45.163 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.505975+0000", 2024-09-21T15:01:45.164 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:45.164 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.164 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.164 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:45.164 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.505971+0000", 2024-09-21T15:01:45.164 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999967 2024-09-21T15:01:45.164 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.164 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.164 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:45.164 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.505989+0000", 2024-09-21T15:01:45.164 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7985000000000001e-05 2024-09-21T15:01:45.164 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.164 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.164 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:45.164 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506050+0000", 2024-09-21T15:01:45.165 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0659000000000003e-05 2024-09-21T15:01:45.165 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.165 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.165 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:45.165 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506074+0000", 2024-09-21T15:01:45.165 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4156e-05 2024-09-21T15:01:45.165 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.165 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.165 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:45.165 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506225+0000", 2024-09-21T15:01:45.165 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015097 2024-09-21T15:01:45.165 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.165 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.165 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:45.166 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506256+0000", 2024-09-21T15:01:45.166 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0505999999999999e-05 2024-09-21T15:01:45.166 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.166 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.166 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:45.166 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.497974+0000", 2024-09-21T15:01:45.166 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99171825700000005 2024-09-21T15:01:45.166 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.166 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.166 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for missing object", 2024-09-21T15:01:45.166 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.498449+0000", 2024-09-21T15:01:45.166 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00047506399999999998 2024-09-21T15:01:45.166 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.166 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.167 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:45.167 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.501000+0000", 2024-09-21T15:01:45.167 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0025506919999999998 2024-09-21T15:01:45.167 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.167 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.167 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:45.167 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.501135+0000", 2024-09-21T15:01:45.167 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013524000000000001 2024-09-21T15:01:45.167 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:45.167 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:45.167 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:45.167 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.167 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.167 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31480 3.c 3:36ccd808:::rbd_data.11367721c19.0000000000000018:head [sparse-read 0~14821] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T15:01:45.168 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.506116+0000", 2024-09-21T15:01:45.168 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 166.65287165000001, 2024-09-21T15:01:45.168 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99316033599999998, 2024-09-21T15:01:45.168 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:45.168 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:45.168 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:45.168 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:45.168 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:45.168 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31480 2024-09-21T15:01:45.168 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.168 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:45.168 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.168 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:45.168 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506116+0000", 2024-09-21T15:01:45.168 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:45.169 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.169 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.169 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:45.169 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506116+0000", 2024-09-21T15:01:45.169 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:45.169 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.169 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.169 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:45.169 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506115+0000", 2024-09-21T15:01:45.169 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:45.169 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.169 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.169 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:45.169 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506124+0000", 2024-09-21T15:01:45.169 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0659999999999993e-06 2024-09-21T15:01:45.169 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.170 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.170 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:45.170 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506145+0000", 2024-09-21T15:01:45.170 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1447e-05 2024-09-21T15:01:45.170 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.170 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.170 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:45.170 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506161+0000", 2024-09-21T15:01:45.170 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5209e-05 2024-09-21T15:01:45.170 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.170 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.170 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:45.170 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506337+0000", 2024-09-21T15:01:45.170 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017642999999999999 2024-09-21T15:01:45.170 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.171 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.171 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:45.171 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506375+0000", 2024-09-21T15:01:45.171 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7536000000000001e-05 2024-09-21T15:01:45.171 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.171 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.171 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:45.171 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.498815+0000", 2024-09-21T15:01:45.171 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99244067999999996 2024-09-21T15:01:45.171 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.171 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.171 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:45.171 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.498950+0000", 2024-09-21T15:01:45.171 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013436700000000001 2024-09-21T15:01:45.171 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.172 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.172 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:45.172 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499277+0000", 2024-09-21T15:01:45.172 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00032696800000000002 2024-09-21T15:01:45.172 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:45.172 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:45.172 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:45.172 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.172 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.172 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31513 3.c 3:36de8523:::rbd_data.11367721c19.0000000000000039:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T15:01:45.172 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.509071+0000", 2024-09-21T15:01:45.172 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 166.64991745, 2024-09-21T15:01:45.172 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99038225099999999, 2024-09-21T15:01:45.172 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:45.173 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:45.173 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:45.173 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:45.173 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:45.173 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31513 2024-09-21T15:01:45.173 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.173 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:45.173 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.173 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:45.173 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509071+0000", 2024-09-21T15:01:45.173 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:45.173 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.173 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.173 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:45.173 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509071+0000", 2024-09-21T15:01:45.174 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:45.174 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.174 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.174 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:45.174 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509068+0000", 2024-09-21T15:01:45.174 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-21T15:01:45.174 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.174 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.174 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:45.174 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509082+0000", 2024-09-21T15:01:45.174 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3545e-05 2024-09-21T15:01:45.174 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.174 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.174 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:45.174 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509116+0000", 2024-09-21T15:01:45.175 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3942000000000001e-05 2024-09-21T15:01:45.175 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.175 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.175 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:45.175 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509134+0000", 2024-09-21T15:01:45.175 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8839999999999999e-05 2024-09-21T15:01:45.175 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.175 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.175 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:45.175 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509318+0000", 2024-09-21T15:01:45.175 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018326799999999999 2024-09-21T15:01:45.175 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.175 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.175 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:45.176 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509346+0000", 2024-09-21T15:01:45.176 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8019000000000001e-05 2024-09-21T15:01:45.176 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.176 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.176 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:45.176 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499330+0000", 2024-09-21T15:01:45.176 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.98998399299999995 2024-09-21T15:01:45.176 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.176 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.176 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:45.176 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499453+0000", 2024-09-21T15:01:45.176 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000123115 2024-09-21T15:01:45.176 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:45.176 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:45.177 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:45.177 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.177 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.177 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31515 3.c 3:35b36ddd:::rbd_data.11367721c19.000000000000003b:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T15:01:45.177 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.509177+0000", 2024-09-21T15:01:45.177 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 166.649811024, 2024-09-21T15:01:45.177 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99044870600000001, 2024-09-21T15:01:45.177 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:45.177 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:45.177 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:45.177 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:45.177 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:45.177 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31515 2024-09-21T15:01:45.177 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.177 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:45.178 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.178 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:45.178 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509177+0000", 2024-09-21T15:01:45.178 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:45.178 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.178 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.178 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:45.178 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509177+0000", 2024-09-21T15:01:45.178 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:45.178 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.178 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.178 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:45.178 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509176+0000", 2024-09-21T15:01:45.178 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:45.178 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.179 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.179 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:45.179 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509183+0000", 2024-09-21T15:01:45.179 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6720000000000004e-06 2024-09-21T15:01:45.179 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.179 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.179 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:45.179 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509269+0000", 2024-09-21T15:01:45.179 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5881e-05 2024-09-21T15:01:45.179 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.179 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.179 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:45.179 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509303+0000", 2024-09-21T15:01:45.179 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3367000000000001e-05 2024-09-21T15:01:45.179 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.180 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.180 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:45.180 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509425+0000", 2024-09-21T15:01:45.180 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000122199 2024-09-21T15:01:45.180 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.180 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.180 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:45.180 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509448+0000", 2024-09-21T15:01:45.180 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2688999999999999e-05 2024-09-21T15:01:45.180 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.180 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.180 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:45.180 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499501+0000", 2024-09-21T15:01:45.180 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99005306999999998 2024-09-21T15:01:45.180 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.181 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.181 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:45.181 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499626+0000", 2024-09-21T15:01:45.181 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012501400000000001 2024-09-21T15:01:45.181 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:45.181 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:45.181 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:45.181 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.181 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.181 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31531 3.c 3:314f0287:::rbd_data.11367721c19.000000000000004b:head [sparse-read 0~7106] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T15:01:45.181 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.509329+0000", 2024-09-21T15:01:45.181 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 166.649659004, 2024-09-21T15:01:45.181 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99103085300000004, 2024-09-21T15:01:45.181 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:45.181 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:45.182 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:45.182 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:45.182 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:45.182 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31531 2024-09-21T15:01:45.182 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.182 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:45.182 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.182 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:45.182 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509329+0000", 2024-09-21T15:01:45.182 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:45.182 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.182 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.182 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:45.183 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509329+0000", 2024-09-21T15:01:45.183 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:45.183 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.183 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.183 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:45.183 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509328+0000", 2024-09-21T15:01:45.183 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:45.183 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.183 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.183 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:45.183 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509336+0000", 2024-09-21T15:01:45.183 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3170000000000002e-06 2024-09-21T15:01:45.183 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.183 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.183 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:45.183 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509353+0000", 2024-09-21T15:01:45.184 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7214000000000002e-05 2024-09-21T15:01:45.184 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.184 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.184 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:45.184 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509366+0000", 2024-09-21T15:01:45.184 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.26e-05 2024-09-21T15:01:45.184 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.184 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.184 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:45.184 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509523+0000", 2024-09-21T15:01:45.184 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015708000000000001 2024-09-21T15:01:45.184 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.184 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.184 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:45.185 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509544+0000", 2024-09-21T15:01:45.185 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0783000000000001e-05 2024-09-21T15:01:45.185 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.185 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.185 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:45.185 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499727+0000", 2024-09-21T15:01:45.185 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99018345200000002 2024-09-21T15:01:45.185 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.185 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.185 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:45.185 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499949+0000", 2024-09-21T15:01:45.185 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000221358 2024-09-21T15:01:45.185 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.185 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.186 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:45.186 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.500360+0000", 2024-09-21T15:01:45.186 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00041130700000000002 2024-09-21T15:01:45.186 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:45.186 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:45.186 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:45.186 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.186 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.186 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:82545 3.b 3:d4eb1ee5:::rbd_data.11368296f791.0000000000000000:head [write 0~671744 [fadvise_sequential+fadvise_nocache],write 704512~65536 [fadvise_sequential+fadvise_nocache],write 786432~49152 [fadvise_sequential+fadvise_nocache],write 868352~49152 [fadvise_sequential+fadvise_nocache],write 933888~65536 [fadvise_sequential+fadvise_nocache],write 1032192~32768 [fadvise_sequential+fadvise_nocache],write 1097728~65536 [fadvise_sequential+fadvise_nocache],write 1179648~49152 [fadvise_sequential+fadvise_nocache],write 1261568~49152 [fadvise_sequential+fadvise_nocache],write 1327104~65536 [fadvise_sequential+fadvise_nocache],write 1425408~32768 [fadvise_sequential+fadvise_nocache],write 1490944~65536 [fadvise_sequential+fadvise_nocache],write 1572864~8192 [fadvise_sequential+fadvise_nocache],write 1630208~49152 [fadvise_sequential+fadvise_nocache],write 1703936~40960 [fadvise_sequential+fadvise_nocache],write 1794048~40960 [fadvise_sequential+fadvise_nocache],write 1859584~49152 [fadvise_sequential+fadvise_nocache],write 1957888~8192 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:01:45.186 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:47.416089+0000", 2024-09-21T15:01:45.186 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 117.742899279, 2024-09-21T15:01:45.186 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011896455, 2024-09-21T15:01:45.186 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:45.187 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:45.187 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:45.187 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:45.187 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:45.187 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82545 2024-09-21T15:01:45.187 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.187 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:45.187 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.187 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:45.187 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.416089+0000", 2024-09-21T15:01:45.187 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:45.187 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.187 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.187 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:45.188 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.416089+0000", 2024-09-21T15:01:45.188 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:45.188 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.188 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.188 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:45.188 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.416087+0000", 2024-09-21T15:01:45.188 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:01:45.188 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.188 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.188 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:45.188 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.419643+0000", 2024-09-21T15:01:45.188 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0035555550000000002 2024-09-21T15:01:45.188 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.189 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.189 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:45.189 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.419663+0000", 2024-09-21T15:01:45.189 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0202000000000001e-05 2024-09-21T15:01:45.189 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.189 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.189 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:45.189 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.419676+0000", 2024-09-21T15:01:45.189 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3207000000000001e-05 2024-09-21T15:01:45.189 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.189 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.189 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:45.189 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.419759+0000", 2024-09-21T15:01:45.189 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3521000000000002e-05 2024-09-21T15:01:45.190 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.190 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.190 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op must wait for map", 2024-09-21T15:01:45.190 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.419768+0000", 2024-09-21T15:01:45.190 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4479999999999994e-06 2024-09-21T15:01:45.190 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.190 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.190 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:45.190 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.422171+0000", 2024-09-21T15:01:45.190 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0024035469999999998 2024-09-21T15:01:45.190 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.190 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.190 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:45.190 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.422350+0000", 2024-09-21T15:01:45.190 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000179001 2024-09-21T15:01:45.191 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.191 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.191 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T15:01:45.191 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.422710+0000", 2024-09-21T15:01:45.191 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000359878 2024-09-21T15:01:45.191 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.191 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.191 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:01:45.191 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.426676+0000", 2024-09-21T15:01:45.191 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0039655790000000003 2024-09-21T15:01:45.191 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.191 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.191 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:01:45.191 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.427884+0000", 2024-09-21T15:01:45.191 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001208207 2024-09-21T15:01:45.192 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.192 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.192 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:01:45.192 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.427953+0000", 2024-09-21T15:01:45.192 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8533000000000003e-05 2024-09-21T15:01:45.192 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.192 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.192 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:45.192 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.427985+0000", 2024-09-21T15:01:45.192 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2481999999999998e-05 2024-09-21T15:01:45.192 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:45.192 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:45.192 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:45.192 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.192 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.193 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:97675 3.b 3:d321acb8:::rbd_data.1136bfc79a70.0000000000000003:head [write 0~4718592 [fadvise_sequential+fadvise_nocache],write 5242880~147456 [fadvise_sequential+fadvise_nocache],write 5455872~147456 [fadvise_sequential+fadvise_nocache],write 5668864~229376 [fadvise_sequential+fadvise_nocache],write 5963776~147456 [fadvise_sequential+fadvise_nocache],write 6176768~229376 [fadvise_sequential+fadvise_nocache],write 6471680~147456 [fadvise_sequential+fadvise_nocache],write 6684672~229376 [fadvise_sequential+fadvise_nocache],write 6979584~147456 [fadvise_sequential+fadvise_nocache],write 7192576~229376 [fadvise_sequential+fadvise_nocache],write 7487488~147456 [fadvise_sequential+fadvise_nocache],write 7700480~147456 [fadvise_sequential+fadvise_nocache],write 7864320~65536 [fadvise_sequential+fadvise_nocache],write 7995392~147456 [fadvise_sequential+fadvise_nocache],write 8208384~147456 [fadvise_sequential+fadvise_nocache],write 8388608~49152 [fadvise_sequential+fadvise_nocache],write 8536064~114688 [fadvise_sequential+fadvise_nocache],write 8749056~114688 [fadvise_sequential+fadvise_nocache],write 8912896~32768 [fadvise_sequential+fadvise_nocache],write 9125888~32768 [fadvise_sequential+fadvise_nocache],write 9338880~32768 [fadvise_sequential+fadvise_nocache],write 9437184~16384 [fadvise_sequential+fadvise_nocache],write 9633792~32768 [fadvise_sequential+fadvise_nocache],write 9846784~32768 [fadvise_sequential+fadvise_nocache],write 9961472~114688 [fadvise_sequential+fadvise_nocache],write 10092544~196608 [fadvise_sequential+fadvise_nocache],write 10305536~278528 [fadvise_sequential+fadvise_nocache],write 10600448~196608 [fadvise_sequential+fadvise_nocache],write 10813440~278528 [fadvise_sequential+fadvise_nocache],write 11108352~196608 [fadvise_sequential+fadvise_nocache],write 11321344~196608 [fadvise_sequential+fadvise_nocache],write 11534336~65536 [fadvise_sequential+fadvise_nocache],write 11632640~98304 [fadvise_sequential+fadvise_nocache],write 11747328~65536 [fadvise_sequential+fadvise_nocache],write 11845632~98304 [fadvise_sequential+fadvise_nocache],write 11960320~65536 [fadvise_sequential+fadvise_nocache],write 12058624~49152 [fadvise_sequential+fadvise_nocache],write 12140544~98304 [fadvise_sequential+fadvise_nocache],write 12255232~65536 [fadvise_sequential+fadvise_nocache],write 12353536~98304 [fadvise_sequential+fadvise_nocache],write 12468224~65536 [fadvise_sequential+fadvise_nocache],write 12566528~16384 [fadvise_sequential+fadvise_nocache],write 12648448~98304 [fadvise_sequential+fadvise_nocache],write 12861440~98304 [fadvise_sequential+fadvise_nocache],write 13074432~32768 [fadvise_sequential+fadvise_nocache],write 13156352~98304 [fadvise_sequential+fadvise_nocache],write 13369344~98304 [fadvise_sequential+fadvise_nocache],write 13582336~49152 [fadvise_sequential+fadvise_nocache],write 13664256~98304 [fadvise_sequential+fadvise_nocache],write 13877248~98304 [fadvise_sequential+fadvise_nocache],write 14090240~65536 [fadvise_sequential+fadvise_nocache],write 15286272~81920 [fadvise_sequential+fadvise_nocache],write 15499264~81920 [fadvise_sequential+fadvise_nocache],write 15712256~16384 [fadvise_sequential+fadvise_nocache],write 15794176~81920 [fadvise_sequential+fadvise_nocache],write 16007168~81920 [fadvise_sequential+fadvise_nocache],write 16220160~32768 [fadvise_sequential+fadvise_nocache],write 16302080~81920 [fadvise_sequential+fadvise_nocache],write 16515072~81920 [fadvise_sequential+fadvise_nocache],write 16728064~49152 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:01:45.193 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.302412+0000", 2024-09-21T15:01:45.193 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 99.856576489999995, 2024-09-21T15:01:45.193 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.067380532000000007, 2024-09-21T15:01:45.193 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:45.193 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:45.193 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:45.193 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:45.193 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:45.193 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97675 2024-09-21T15:01:45.193 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.193 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:45.193 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.193 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:45.194 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.302412+0000", 2024-09-21T15:01:45.194 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:45.194 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.194 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.194 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:45.194 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.302412+0000", 2024-09-21T15:01:45.194 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:45.194 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.194 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.194 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:45.194 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.302409+0000", 2024-09-21T15:01:45.194 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-21T15:01:45.194 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.194 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.195 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:45.195 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328550+0000", 2024-09-21T15:01:45.195 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.026140674999999999 2024-09-21T15:01:45.195 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.195 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.195 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:45.195 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328584+0000", 2024-09-21T15:01:45.195 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4165999999999998e-05 2024-09-21T15:01:45.195 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.195 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.195 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:45.195 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328599+0000", 2024-09-21T15:01:45.195 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5505000000000001e-05 2024-09-21T15:01:45.195 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.196 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.196 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:45.196 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328704+0000", 2024-09-21T15:01:45.196 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010487199999999999 2024-09-21T15:01:45.196 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.196 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.196 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:45.196 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.329082+0000", 2024-09-21T15:01:45.196 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00037804899999999998 2024-09-21T15:01:45.196 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.196 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.196 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T15:01:45.196 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.331158+0000", 2024-09-21T15:01:45.196 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0020755449999999998 2024-09-21T15:01:45.197 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.197 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.197 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:01:45.197 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.360553+0000", 2024-09-21T15:01:45.197 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.029395553000000001 2024-09-21T15:01:45.197 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.197 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.197 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:01:45.197 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.369496+0000", 2024-09-21T15:01:45.197 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0089428619999999993 2024-09-21T15:01:45.197 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.197 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.197 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:01:45.197 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.369686+0000", 2024-09-21T15:01:45.198 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019000399999999999 2024-09-21T15:01:45.198 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.198 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.198 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:45.198 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.369792+0000", 2024-09-21T15:01:45.198 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000105775 2024-09-21T15:01:45.198 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:45.198 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:45.198 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:45.198 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.198 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.198 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:97693 3.b 3:d7d7b39b:::rbd_data.1136b9c49b1c.000000000000000c:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:01:45.198 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.328643+0000", 2024-09-21T15:01:45.198 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 99.830344703999998, 2024-09-21T15:01:45.199 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.031062136000000001, 2024-09-21T15:01:45.199 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:45.199 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:45.199 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:45.199 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:45.199 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:45.199 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97693 2024-09-21T15:01:45.199 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.199 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:45.199 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.199 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:45.199 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328643+0000", 2024-09-21T15:01:45.199 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:45.199 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.200 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.200 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:45.200 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328643+0000", 2024-09-21T15:01:45.200 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:45.200 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.200 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.200 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:45.200 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328642+0000", 2024-09-21T15:01:45.200 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:01:45.200 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.200 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.200 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:45.200 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328650+0000", 2024-09-21T15:01:45.200 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7770000000000008e-06 2024-09-21T15:01:45.200 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.200 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.201 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:45.201 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328662+0000", 2024-09-21T15:01:45.201 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2026e-05 2024-09-21T15:01:45.201 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.201 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.201 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:45.201 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328671+0000", 2024-09-21T15:01:45.201 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.585e-06 2024-09-21T15:01:45.201 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.201 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.201 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:45.201 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.359498+0000", 2024-09-21T15:01:45.201 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.030827001 2024-09-21T15:01:45.201 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.202 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.202 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:45.202 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.359603+0000", 2024-09-21T15:01:45.202 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010512400000000001 2024-09-21T15:01:45.202 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.202 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.202 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:45.202 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.359705+0000", 2024-09-21T15:01:45.202 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000102327 2024-09-21T15:01:45.202 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:45.202 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:45.202 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:45.202 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.202 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.202 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:97724 3.b 3:d4bf6bcf:::rbd_data.1136b9c49b1c.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:01:45.202 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.328704+0000", 2024-09-21T15:01:45.203 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 99.830284431999999, 2024-09-21T15:01:45.203 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.031157957, 2024-09-21T15:01:45.203 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:45.203 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:45.203 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:45.203 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:45.203 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:45.203 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97724 2024-09-21T15:01:45.203 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.203 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:45.203 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.203 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:45.203 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328704+0000", 2024-09-21T15:01:45.203 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:45.203 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.204 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.204 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:45.204 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328704+0000", 2024-09-21T15:01:45.204 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:45.204 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.204 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.204 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:45.204 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328703+0000", 2024-09-21T15:01:45.204 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:45.204 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.204 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.204 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:45.204 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328708+0000", 2024-09-21T15:01:45.204 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0050000000000004e-06 2024-09-21T15:01:45.204 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.205 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.205 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:45.205 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328719+0000", 2024-09-21T15:01:45.205 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1052e-05 2024-09-21T15:01:45.205 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.205 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.205 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:45.205 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328727+0000", 2024-09-21T15:01:45.205 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4519999999999997e-06 2024-09-21T15:01:45.205 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.205 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.205 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:45.205 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.359737+0000", 2024-09-21T15:01:45.205 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.031009341999999999 2024-09-21T15:01:45.206 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.206 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.206 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:45.206 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.359806+0000", 2024-09-21T15:01:45.206 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9750000000000001e-05 2024-09-21T15:01:45.206 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.206 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.206 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:45.206 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.359862+0000", 2024-09-21T15:01:45.206 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5084999999999999e-05 2024-09-21T15:01:45.206 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:45.206 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:45.206 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:45.206 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.207 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.207 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:129970 3.c e158/87 3:33fa4d4e:::rbd_data.1136a12e4351.0000000000000000:head v 158'379, mlcod=156'377)", 2024-09-21T15:01:45.207 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:44.452766+0000", 2024-09-21T15:01:45.207 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 60.706221880000001, 2024-09-21T15:01:45.207 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011942062, 2024-09-21T15:01:45.207 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:45.207 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:45.207 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:45.207 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.207 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:45.207 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:44.452766+0000", 2024-09-21T15:01:45.207 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:45.207 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.207 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.207 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:45.207 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:44.452766+0000", 2024-09-21T15:01:45.208 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:45.208 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.208 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.208 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:45.208 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:44.452766+0000", 2024-09-21T15:01:45.208 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:45.208 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.208 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.208 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:45.208 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:44.458962+0000", 2024-09-21T15:01:45.208 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0061964799999999999 2024-09-21T15:01:45.208 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.208 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.208 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:45.208 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:44.459012+0000", 2024-09-21T15:01:45.209 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9427e-05 2024-09-21T15:01:45.209 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.209 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.209 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:45.209 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:44.459042+0000", 2024-09-21T15:01:45.209 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9838000000000001e-05 2024-09-21T15:01:45.209 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.209 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.209 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:45.209 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:44.459115+0000", 2024-09-21T15:01:45.209 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3430999999999995e-05 2024-09-21T15:01:45.209 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.209 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.209 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:45.209 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:44.459135+0000", 2024-09-21T15:01:45.210 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9701999999999999e-05 2024-09-21T15:01:45.210 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.210 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.210 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:01:45.210 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:44.464645+0000", 2024-09-21T15:01:45.210 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0055100820000000003 2024-09-21T15:01:45.210 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.210 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.210 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:45.210 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:44.464708+0000", 2024-09-21T15:01:45.210 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3399999999999996e-05 2024-09-21T15:01:45.210 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:45.210 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:45.211 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:45.211 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.211 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.211 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:135368 3.b 3:d3301c39:::rbd_data.11365ed419e3.0000000000000102:67 [read 335872~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:45.211 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:53.998901+0000", 2024-09-21T15:01:45.211 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 51.160087068000003, 2024-09-21T15:01:45.211 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011666174, 2024-09-21T15:01:45.211 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:45.211 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:45.211 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:45.211 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:45.211 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:45.211 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 135368 2024-09-21T15:01:45.211 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.212 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:45.212 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.212 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:45.212 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998901+0000", 2024-09-21T15:01:45.212 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:45.212 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.212 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.212 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:45.212 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998901+0000", 2024-09-21T15:01:45.212 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:45.212 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.212 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.212 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:45.213 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998901+0000", 2024-09-21T15:01:45.213 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:45.213 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.213 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.213 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:45.213 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998903+0000", 2024-09-21T15:01:45.213 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5500000000000001e-06 2024-09-21T15:01:45.213 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.213 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.213 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:45.213 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998913+0000", 2024-09-21T15:01:45.213 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6479999999999998e-06 2024-09-21T15:01:45.213 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.213 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.213 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:45.214 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998919+0000", 2024-09-21T15:01:45.214 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7930000000000003e-06 2024-09-21T15:01:45.214 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.214 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.214 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:45.214 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:54.010277+0000", 2024-09-21T15:01:45.214 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011358692 2024-09-21T15:01:45.214 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.214 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.214 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:45.214 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:54.010348+0000", 2024-09-21T15:01:45.214 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0247999999999996e-05 2024-09-21T15:01:45.214 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.214 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.215 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:45.215 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:54.010567+0000", 2024-09-21T15:01:45.215 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021953599999999999 2024-09-21T15:01:45.215 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:45.215 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:45.215 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:45.215 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.215 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.215 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:135369 3.b 3:d3301c39:::rbd_data.11365ed419e3.0000000000000102:67 [read 352256~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:45.215 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:53.998923+0000", 2024-09-21T15:01:45.215 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 51.160064767000001, 2024-09-21T15:01:45.215 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011990331999999999, 2024-09-21T15:01:45.215 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:45.215 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:45.216 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:45.216 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:45.216 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:45.216 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 135369 2024-09-21T15:01:45.216 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.216 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:45.216 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.216 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:45.216 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998923+0000", 2024-09-21T15:01:45.216 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:45.216 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.216 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.216 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:45.216 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998923+0000", 2024-09-21T15:01:45.216 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:45.217 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.217 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.217 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:45.217 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998923+0000", 2024-09-21T15:01:45.217 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:45.217 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.217 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.217 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:45.217 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998926+0000", 2024-09-21T15:01:45.217 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6749999999999998e-06 2024-09-21T15:01:45.217 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.217 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.217 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:45.217 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998935+0000", 2024-09-21T15:01:45.218 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3619999999999995e-06 2024-09-21T15:01:45.218 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.218 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.218 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:45.218 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998940+0000", 2024-09-21T15:01:45.218 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2379999999999997e-06 2024-09-21T15:01:45.218 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.218 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.218 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:45.218 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:54.010593+0000", 2024-09-21T15:01:45.218 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011652531000000001 2024-09-21T15:01:45.218 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.218 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.219 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:45.219 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:54.010679+0000", 2024-09-21T15:01:45.219 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6811000000000006e-05 2024-09-21T15:01:45.219 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.219 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.219 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:45.219 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:54.010914+0000", 2024-09-21T15:01:45.219 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000234071 2024-09-21T15:01:45.219 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:45.219 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:45.219 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:45.219 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.219 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.219 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:137706 3.b 3:d0cb966d:::rbd_data.1136f64ddcfe.0000000000000005:6a [read 1441792~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-21T15:01:45.219 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:58.376385+0000", 2024-09-21T15:01:45.220 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 46.782603180000002, 2024-09-21T15:01:45.220 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011783609, 2024-09-21T15:01:45.220 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:45.220 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:45.220 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:45.220 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:45.220 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:45.220 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 137706 2024-09-21T15:01:45.220 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.220 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:45.220 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.220 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:45.220 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376385+0000", 2024-09-21T15:01:45.220 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:45.221 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.221 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.221 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:45.221 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376385+0000", 2024-09-21T15:01:45.221 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:45.221 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.221 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.221 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:45.221 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376384+0000", 2024-09-21T15:01:45.221 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:45.221 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.221 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.221 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:45.221 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376387+0000", 2024-09-21T15:01:45.221 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9809999999999999e-06 2024-09-21T15:01:45.222 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.222 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.222 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:45.222 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376398+0000", 2024-09-21T15:01:45.222 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0319000000000001e-05 2024-09-21T15:01:45.222 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.222 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.222 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:45.222 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376404+0000", 2024-09-21T15:01:45.222 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5339999999999996e-06 2024-09-21T15:01:45.222 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.222 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.222 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:45.222 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.388025+0000", 2024-09-21T15:01:45.222 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011620449999999999 2024-09-21T15:01:45.223 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.223 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.223 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:45.223 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.388082+0000", 2024-09-21T15:01:45.223 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7065999999999999e-05 2024-09-21T15:01:45.223 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.223 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.223 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:45.223 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.388168+0000", 2024-09-21T15:01:45.223 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6689999999999998e-05 2024-09-21T15:01:45.223 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:45.223 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:45.223 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:45.223 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.223 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.223 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:137707 3.b 3:d0cb966d:::rbd_data.1136f64ddcfe.0000000000000005:6a [read 1556480~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-21T15:01:45.224 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:58.376428+0000", 2024-09-21T15:01:45.224 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 46.782559687999999, 2024-09-21T15:01:45.224 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012004865999999999, 2024-09-21T15:01:45.224 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:45.224 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:45.224 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:45.224 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:45.224 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:45.224 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 137707 2024-09-21T15:01:45.224 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.224 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:45.224 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.224 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:45.224 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376428+0000", 2024-09-21T15:01:45.224 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:45.224 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.225 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.225 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:45.225 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376428+0000", 2024-09-21T15:01:45.225 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:45.225 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.225 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.225 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:45.225 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376428+0000", 2024-09-21T15:01:45.225 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:45.225 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.225 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.225 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:45.225 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376432+0000", 2024-09-21T15:01:45.225 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2119999999999997e-06 2024-09-21T15:01:45.225 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.226 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.226 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:45.226 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376448+0000", 2024-09-21T15:01:45.226 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5913000000000002e-05 2024-09-21T15:01:45.226 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.226 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.226 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:45.226 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376455+0000", 2024-09-21T15:01:45.226 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6359999999999994e-06 2024-09-21T15:01:45.226 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.226 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.226 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:45.226 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.388197+0000", 2024-09-21T15:01:45.226 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011741843 2024-09-21T15:01:45.226 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.227 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.227 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:45.227 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.388249+0000", 2024-09-21T15:01:45.227 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1549999999999999e-05 2024-09-21T15:01:45.227 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.227 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.227 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:45.227 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.388433+0000", 2024-09-21T15:01:45.227 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018437999999999999 2024-09-21T15:01:45.227 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:45.227 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:45.227 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:45.227 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.227 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.227 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:137708 3.b 3:d0cb966d:::rbd_data.1136f64ddcfe.0000000000000005:6a [read 1671168~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-21T15:01:45.228 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:58.376462+0000", 2024-09-21T15:01:45.228 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 46.782525900000003, 2024-09-21T15:01:45.228 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012315491, 2024-09-21T15:01:45.228 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:45.228 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:45.228 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:45.228 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:45.228 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:45.228 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 137708 2024-09-21T15:01:45.228 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.228 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:45.228 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.228 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:45.228 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376462+0000", 2024-09-21T15:01:45.228 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:45.229 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.229 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.229 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:45.229 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376462+0000", 2024-09-21T15:01:45.229 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:45.229 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.229 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.229 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:45.229 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376461+0000", 2024-09-21T15:01:45.229 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:45.229 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.229 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.229 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:45.229 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376465+0000", 2024-09-21T15:01:45.229 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7069999999999998e-06 2024-09-21T15:01:45.229 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.230 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.230 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:45.230 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376473+0000", 2024-09-21T15:01:45.230 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6669999999999996e-06 2024-09-21T15:01:45.230 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.230 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.230 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:45.230 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376479+0000", 2024-09-21T15:01:45.230 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.263e-06 2024-09-21T15:01:45.230 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.230 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.230 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:45.230 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.388489+0000", 2024-09-21T15:01:45.230 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012009953 2024-09-21T15:01:45.230 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.231 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.231 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:45.231 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.388625+0000", 2024-09-21T15:01:45.231 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013548799999999999 2024-09-21T15:01:45.231 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.231 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.231 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:45.231 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.388778+0000", 2024-09-21T15:01:45.231 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015302299999999999 2024-09-21T15:01:45.231 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:45.231 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:45.231 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:45.231 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.231 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.232 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:137709 3.b 3:d0cb966d:::rbd_data.1136f64ddcfe.0000000000000005:6a [read 1785856~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-21T15:01:45.232 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:58.376485+0000", 2024-09-21T15:01:45.232 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 46.782502567999998, 2024-09-21T15:01:45.232 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012466549, 2024-09-21T15:01:45.232 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:45.232 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:45.232 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:45.232 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:45.232 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:45.232 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 137709 2024-09-21T15:01:45.232 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.232 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:45.232 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.232 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:45.233 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376485+0000", 2024-09-21T15:01:45.233 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:45.233 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.233 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.233 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:45.233 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376485+0000", 2024-09-21T15:01:45.233 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:45.233 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.233 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.233 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:45.233 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376485+0000", 2024-09-21T15:01:45.233 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:45.233 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.233 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.234 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:45.234 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376492+0000", 2024-09-21T15:01:45.234 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8929999999999999e-06 2024-09-21T15:01:45.234 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.234 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.234 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:45.234 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376500+0000", 2024-09-21T15:01:45.234 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7989999999999994e-06 2024-09-21T15:01:45.234 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.234 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.234 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:45.234 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376506+0000", 2024-09-21T15:01:45.234 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4799999999999998e-06 2024-09-21T15:01:45.235 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.235 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.235 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:45.235 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.388810+0000", 2024-09-21T15:01:45.235 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012303727 2024-09-21T15:01:45.235 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.235 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.235 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:45.235 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.388871+0000", 2024-09-21T15:01:45.235 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1278000000000005e-05 2024-09-21T15:01:45.235 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.235 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.235 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:45.235 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.388952+0000", 2024-09-21T15:01:45.236 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0770000000000001e-05 2024-09-21T15:01:45.236 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:45.236 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:45.236 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:45.236 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.236 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.236 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:137710 3.b 3:d0cb966d:::rbd_data.1136f64ddcfe.0000000000000005:6a [read 1900544~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-21T15:01:45.236 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:58.376512+0000", 2024-09-21T15:01:45.236 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 46.782476365000001, 2024-09-21T15:01:45.236 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012746351, 2024-09-21T15:01:45.236 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:45.236 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:45.236 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:45.236 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:45.236 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:45.236 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 137710 2024-09-21T15:01:45.237 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.237 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:45.237 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.237 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:45.237 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376512+0000", 2024-09-21T15:01:45.237 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:45.237 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.237 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.237 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:45.237 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376512+0000", 2024-09-21T15:01:45.237 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:45.237 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.237 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.237 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:45.238 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376511+0000", 2024-09-21T15:01:45.238 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:45.238 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.238 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.238 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:45.238 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376514+0000", 2024-09-21T15:01:45.238 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1719999999999999e-06 2024-09-21T15:01:45.238 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.238 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.238 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:45.238 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376522+0000", 2024-09-21T15:01:45.238 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1389999999999998e-06 2024-09-21T15:01:45.238 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.238 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.238 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:45.239 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376535+0000", 2024-09-21T15:01:45.239 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3658999999999999e-05 2024-09-21T15:01:45.239 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.239 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.239 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:45.239 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.389005+0000", 2024-09-21T15:01:45.239 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012470261 2024-09-21T15:01:45.239 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.239 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.239 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:45.239 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.389119+0000", 2024-09-21T15:01:45.239 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000113346 2024-09-21T15:01:45.239 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.239 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.239 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:45.240 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.389258+0000", 2024-09-21T15:01:45.240 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013920100000000001 2024-09-21T15:01:45.240 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:45.240 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:45.240 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:45.240 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.240 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.240 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:137711 3.b 3:d0cb966d:::rbd_data.1136f64ddcfe.0000000000000005:6a [read 2015232~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-21T15:01:45.240 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:58.376541+0000", 2024-09-21T15:01:45.240 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 46.782447003000001, 2024-09-21T15:01:45.240 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012849955999999999, 2024-09-21T15:01:45.240 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:45.240 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:45.240 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:45.240 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:45.241 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:45.241 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 137711 2024-09-21T15:01:45.241 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.241 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:45.241 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.241 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:45.241 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376541+0000", 2024-09-21T15:01:45.241 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:45.241 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.241 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.241 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:45.241 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376541+0000", 2024-09-21T15:01:45.241 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:45.241 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.241 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.241 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:45.242 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376540+0000", 2024-09-21T15:01:45.242 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:45.242 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.242 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.242 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:45.242 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376544+0000", 2024-09-21T15:01:45.242 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3249999999999999e-06 2024-09-21T15:01:45.242 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.242 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.242 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:45.242 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376551+0000", 2024-09-21T15:01:45.242 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2189999999999998e-06 2024-09-21T15:01:45.242 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.242 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.242 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:45.243 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376557+0000", 2024-09-21T15:01:45.243 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8869999999999997e-06 2024-09-21T15:01:45.243 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.243 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.243 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:45.243 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.389277+0000", 2024-09-21T15:01:45.243 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012720555999999999 2024-09-21T15:01:45.243 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.243 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.243 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:45.243 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.389325+0000", 2024-09-21T15:01:45.243 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7367000000000002e-05 2024-09-21T15:01:45.243 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.243 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.243 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:45.244 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.389391+0000", 2024-09-21T15:01:45.244 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6095999999999999e-05 2024-09-21T15:01:45.244 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:45.244 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:45.244 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:45.244 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.244 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.244 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:137723 3.b 3:d0cb966d:::rbd_data.1136f64ddcfe.0000000000000005:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-21T15:01:45.244 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:58.377091+0000", 2024-09-21T15:01:45.244 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 46.781896682000003, 2024-09-21T15:01:45.244 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012594424, 2024-09-21T15:01:45.244 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:45.244 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:45.244 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:45.244 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:45.245 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:45.245 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 137723 2024-09-21T15:01:45.245 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.245 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:45.245 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.245 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:45.245 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.377091+0000", 2024-09-21T15:01:45.245 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:45.245 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.245 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.245 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:45.245 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.377091+0000", 2024-09-21T15:01:45.245 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:45.245 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.245 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.246 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:45.246 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.377090+0000", 2024-09-21T15:01:45.246 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:45.246 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.246 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.246 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:45.246 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.377097+0000", 2024-09-21T15:01:45.246 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7100000000000001e-06 2024-09-21T15:01:45.246 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.246 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.246 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:45.246 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.377120+0000", 2024-09-21T15:01:45.246 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3189000000000001e-05 2024-09-21T15:01:45.246 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.246 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.247 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:45.247 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.377132+0000", 2024-09-21T15:01:45.247 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2041999999999999e-05 2024-09-21T15:01:45.247 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.247 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.247 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:45.247 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.389439+0000", 2024-09-21T15:01:45.247 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.01230698 2024-09-21T15:01:45.247 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.247 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.247 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:45.247 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.389556+0000", 2024-09-21T15:01:45.247 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000116446 2024-09-21T15:01:45.248 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.248 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.248 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:45.248 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.389686+0000", 2024-09-21T15:01:45.248 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000129966 2024-09-21T15:01:45.248 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:45.248 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:45.248 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:45.248 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.248 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.248 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:166795 3.8 3:1499e676:::rbd_id.image_client.0-clone28:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e193)", 2024-09-21T15:01:45.248 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:25.321589+0000", 2024-09-21T15:01:45.248 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 19.837398518000001, 2024-09-21T15:01:45.248 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.855281438, 2024-09-21T15:01:45.248 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:45.249 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:45.249 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:45.249 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:45.249 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:45.249 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 166795 2024-09-21T15:01:45.249 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.249 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:45.249 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.249 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:45.249 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:25.321589+0000", 2024-09-21T15:01:45.249 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:45.249 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.249 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.249 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:45.249 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:25.321589+0000", 2024-09-21T15:01:45.249 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:45.250 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.250 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.250 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:45.250 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:25.321588+0000", 2024-09-21T15:01:45.250 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:45.250 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.250 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.250 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:45.250 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:25.321598+0000", 2024-09-21T15:01:45.250 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7429999999999997e-06 2024-09-21T15:01:45.250 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.250 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.250 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:45.250 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:25.321624+0000", 2024-09-21T15:01:45.251 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6752000000000001e-05 2024-09-21T15:01:45.251 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.251 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.251 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:45.251 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:25.321637+0000", 2024-09-21T15:01:45.251 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2809999999999999e-05 2024-09-21T15:01:45.251 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.251 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.251 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:45.251 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:25.321721+0000", 2024-09-21T15:01:45.251 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3314999999999998e-05 2024-09-21T15:01:45.251 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.251 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.251 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:45.251 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:25.321736+0000", 2024-09-21T15:01:45.252 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5262000000000001e-05 2024-09-21T15:01:45.252 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.252 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.252 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:45.252 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:26.176475+0000", 2024-09-21T15:01:45.252 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.85473961200000004 2024-09-21T15:01:45.252 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.252 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.252 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:45.252 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:26.176655+0000", 2024-09-21T15:01:45.252 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017934400000000001 2024-09-21T15:01:45.252 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:45.252 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:45.252 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:45.252 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:26.176871+0000", 2024-09-21T15:01:45.253 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000216133 2024-09-21T15:01:45.253 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:45.253 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:45.253 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:45.253 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:45.253 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:45.253 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:01:45.254 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T15:01:45.255 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:01:45.254+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:01:45.355 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:01:45.354+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:01:45.384 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:45.384 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:01:45.384 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T15:01:45.384 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:45.394 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T15:01:45.455 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:01:45.454+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:01:45.497 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:45.498 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:01:45.498 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T15:01:45.498 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T15:01:45.498 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:45.507 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T15:01:45.556 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:01:45.552+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:01:45.622 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:45.622 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T15:01:45.622 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T15:01:45.622 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T15:01:45.623 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.623 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:97677 3.15 3:a8e72133:::rbd_data.1136bfc79a70.0000000000000005:head [write 0~4440027 [fadvise_sequential+fadvise_nocache],write 4751360~147456 [fadvise_sequential+fadvise_nocache],write 4964352~147456 [fadvise_sequential+fadvise_nocache],write 5177344~65536 [fadvise_sequential+fadvise_nocache],write 5259264~147456 [fadvise_sequential+fadvise_nocache],write 5472256~147456 [fadvise_sequential+fadvise_nocache],write 5685248~229376 [fadvise_sequential+fadvise_nocache],write 5980160~147456 [fadvise_sequential+fadvise_nocache],write 6193152~229376 [fadvise_sequential+fadvise_nocache],write 6488064~147456 [fadvise_sequential+fadvise_nocache],write 6701056~229376 [fadvise_sequential+fadvise_nocache],write 6995968~147456 [fadvise_sequential+fadvise_nocache],write 7208960~229376 [fadvise_sequential+fadvise_nocache],write 7503872~147456 [fadvise_sequential+fadvise_nocache],write 7716864~229376 [fadvise_sequential+fadvise_nocache],write 8044544~114688 [fadvise_sequential+fadvise_nocache],write 8257536~114688 [fadvise_sequential+fadvise_nocache],write 8388608~65536 [fadvise_sequential+fadvise_nocache],write 8552448~114688 [fadvise_sequential+fadvise_nocache],write 8765440~114688 [fadvise_sequential+fadvise_nocache],write 8929280~32768 [fadvise_sequential+fadvise_nocache],write 9142272~32768 [fadvise_sequential+fadvise_nocache],write 9355264~32768 [fadvise_sequential+fadvise_nocache],write 9437184~32768 [fadvise_sequential+fadvise_nocache],write 9650176~32768 [fadvise_sequential+fadvise_nocache],write 9863168~32768 [fadvise_sequential+fadvise_nocache],write 9961472~131072 [fadvise_sequential+fadvise_nocache],write 10108928~196608 [fadvise_sequential+fadvise_nocache],write 10321920~278528 [fadvise_sequential+fadvise_nocache],write 10616832~196608 [fadvise_sequential+fadvise_nocache],write 10829824~278528 [fadvise_sequential+fadvise_nocache],write 11124736~196608 [fadvise_sequential+fadvise_nocache],write 11337728~196608 [fadvise_sequential+fadvise_nocache],write 11550720~65536 [fadvise_sequential+fadvise_nocache],write 11649024~98304 [fadvise_sequential+fadvise_nocache],write 11763712~65536 [fadvise_sequential+fadvise_nocache],write 11862016~98304 [fadvise_sequential+fadvise_nocache],write 11976704~65536 [fadvise_sequential+fadvise_nocache],write 12058624~65536 [fadvise_sequential+fadvise_nocache],write 12156928~98304 [fadvise_sequential+fadvise_nocache],write 12271616~65536 [fadvise_sequential+fadvise_nocache],write 12369920~98304 [fadvise_sequential+fadvise_nocache],write 12484608~65536 [fadvise_sequential+fadvise_nocache],write 12664832~98304 [fadvise_sequential+fadvise_nocache],write 12877824~98304 [fadvise_sequential+fadvise_nocache],write 13090816~16384 [fadvise_sequential+fadvise_nocache],write 13172736~98304 [fadvise_sequential+fadvise_nocache],write 13385728~98304 [fadvise_sequential+fadvise_nocache],write 13598720~32768 [fadvise_sequential+fadvise_nocache],write 13680640~98304 [fadvise_sequential+fadvise_nocache],write 13893632~98304 [fadvise_sequential+fadvise_nocache],write 14106624~49152 [fadvise_sequential+fadvise_nocache],write 14794752~81920 [fadvise_sequential+fadvise_nocache],write 15007744~81920 [fadvise_sequential+fadvise_nocache],write 15302656~81920 [fadvise_sequential+fadvise_nocache],write 15515648~81920 [fadvise_sequential+fadvise_nocache],write 15810560~81920 [fadvise_sequential+fadvise_nocache],write 16023552~81920 [fadvise_sequential+fadvise_nocache],write 16236544~16384 [fadvise_sequential+fadvise_nocache],write 16318464~81920 [fadvise_sequential+fadvise_nocache],write 16531456~81920 [fadvise_sequential+fadvise_nocache],write 16744448~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:01:45.623 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.302946+0000", 2024-09-21T15:01:45.623 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 100.316593228, 2024-09-21T15:01:45.623 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.091231204999999996, 2024-09-21T15:01:45.623 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:45.623 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:45.623 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:45.623 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:45.623 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:45.623 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 97677 2024-09-21T15:01:45.623 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.623 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:45.624 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.624 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:45.624 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.302946+0000", 2024-09-21T15:01:45.624 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:45.624 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.624 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.624 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:45.624 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.302946+0000", 2024-09-21T15:01:45.624 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:45.624 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.624 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.624 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:45.624 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.302942+0000", 2024-09-21T15:01:45.624 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999962 2024-09-21T15:01:45.625 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.625 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.625 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:45.625 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311253+0000", 2024-09-21T15:01:45.625 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0083105360000000003 2024-09-21T15:01:45.625 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.625 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.625 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:45.625 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311283+0000", 2024-09-21T15:01:45.625 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0065000000000001e-05 2024-09-21T15:01:45.625 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.625 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.625 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:45.625 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311296+0000", 2024-09-21T15:01:45.625 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3206e-05 2024-09-21T15:01:45.626 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.626 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.626 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:45.626 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311404+0000", 2024-09-21T15:01:45.626 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000107514 2024-09-21T15:01:45.626 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.626 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.626 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:45.626 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311795+0000", 2024-09-21T15:01:45.626 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000390913 2024-09-21T15:01:45.626 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.626 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.626 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 6", 2024-09-21T15:01:45.626 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.314686+0000", 2024-09-21T15:01:45.626 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0028915659999999999 2024-09-21T15:01:45.627 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.627 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.627 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:01:45.627 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.368178+0000", 2024-09-21T15:01:45.627 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.053491706 2024-09-21T15:01:45.627 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.627 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.627 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:01:45.627 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.393925+0000", 2024-09-21T15:01:45.627 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025747289 2024-09-21T15:01:45.627 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.627 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.627 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:01:45.627 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.394127+0000", 2024-09-21T15:01:45.627 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000201444 2024-09-21T15:01:45.627 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.628 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.628 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:45.628 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.394177+0000", 2024-09-21T15:01:45.628 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0689000000000002e-05 2024-09-21T15:01:45.628 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:45.628 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:45.628 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:45.628 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.628 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.628 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:97673 3.1d e122/24 3:bd91946e:::rbd_data.1136bfc79a70.0000000000000002:head v 122'162, mlcod=120'160)", 2024-09-21T15:01:45.628 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.313730+0000", 2024-09-21T15:01:45.628 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 100.305808903, 2024-09-21T15:01:45.628 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055373462999999998, 2024-09-21T15:01:45.629 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:45.629 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:45.629 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:45.629 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.629 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:45.629 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.313730+0000", 2024-09-21T15:01:45.629 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:45.629 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.629 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.629 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:45.629 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.313730+0000", 2024-09-21T15:01:45.629 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:45.629 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.629 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.629 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:45.630 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.313730+0000", 2024-09-21T15:01:45.630 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:45.630 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.630 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.630 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:45.630 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.336113+0000", 2024-09-21T15:01:45.630 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022382962999999999 2024-09-21T15:01:45.630 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.630 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.630 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:45.630 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.336140+0000", 2024-09-21T15:01:45.630 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7248999999999999e-05 2024-09-21T15:01:45.630 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.631 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.631 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:45.631 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.336151+0000", 2024-09-21T15:01:45.631 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1015e-05 2024-09-21T15:01:45.631 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.631 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.631 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:45.631 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.355428+0000", 2024-09-21T15:01:45.631 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019276379999999999 2024-09-21T15:01:45.631 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.631 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.631 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:45.631 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.355484+0000", 2024-09-21T15:01:45.631 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6820999999999997e-05 2024-09-21T15:01:45.631 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.631 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.632 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:01:45.632 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.369040+0000", 2024-09-21T15:01:45.632 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013555466 2024-09-21T15:01:45.632 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.632 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.632 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:45.632 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.369104+0000", 2024-09-21T15:01:45.632 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3986999999999996e-05 2024-09-21T15:01:45.632 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:45.632 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:45.632 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:45.632 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.632 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.633 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139890 3.e 3:71dba096:::rbd_data.113664780c85.000000000000018b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:45.633 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.226444+0000", 2024-09-21T15:01:45.633 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 43.393095260999999, 2024-09-21T15:01:45.633 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.028209567, 2024-09-21T15:01:45.633 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:45.633 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:45.633 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:45.633 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:45.633 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:45.633 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139890 2024-09-21T15:01:45.633 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.633 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:45.633 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.633 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:45.633 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226444+0000", 2024-09-21T15:01:45.634 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:45.634 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.634 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.634 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:45.634 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226444+0000", 2024-09-21T15:01:45.634 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:45.634 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.634 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.634 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:45.634 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226443+0000", 2024-09-21T15:01:45.634 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:45.634 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.634 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.634 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:45.635 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226448+0000", 2024-09-21T15:01:45.635 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5499999999999996e-06 2024-09-21T15:01:45.635 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.635 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.635 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:45.635 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226456+0000", 2024-09-21T15:01:45.635 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8679999999999999e-06 2024-09-21T15:01:45.635 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.635 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.635 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:45.635 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226462+0000", 2024-09-21T15:01:45.635 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1800000000000001e-06 2024-09-21T15:01:45.635 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.635 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.635 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:45.636 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226545+0000", 2024-09-21T15:01:45.636 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2527000000000005e-05 2024-09-21T15:01:45.636 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.636 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.636 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:45.636 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226553+0000", 2024-09-21T15:01:45.636 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8019999999999998e-06 2024-09-21T15:01:45.636 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.636 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.636 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:45.636 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.254491+0000", 2024-09-21T15:01:45.636 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.02793785 2024-09-21T15:01:45.636 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.636 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.636 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:45.637 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.254654+0000", 2024-09-21T15:01:45.637 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00016245800000000001 2024-09-21T15:01:45.637 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:45.637 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:45.637 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:45.637 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.637 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.637 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139897 3.e 3:76ddeb1d:::rbd_data.113664780c85.0000000000000004:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:45.637 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.227198+0000", 2024-09-21T15:01:45.637 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 43.392341655999999, 2024-09-21T15:01:45.637 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.027799822, 2024-09-21T15:01:45.637 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:45.637 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:45.637 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:45.638 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:45.638 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:45.638 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139897 2024-09-21T15:01:45.638 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.638 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:45.638 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.638 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:45.638 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227198+0000", 2024-09-21T15:01:45.638 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:45.638 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.638 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.638 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:45.638 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227198+0000", 2024-09-21T15:01:45.638 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:45.638 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.639 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.639 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:45.639 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227197+0000", 2024-09-21T15:01:45.639 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:45.639 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.639 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.639 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:45.639 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227203+0000", 2024-09-21T15:01:45.639 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4819999999999999e-06 2024-09-21T15:01:45.639 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.639 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.639 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:45.639 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227220+0000", 2024-09-21T15:01:45.639 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6671e-05 2024-09-21T15:01:45.639 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.640 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.640 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:45.640 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227229+0000", 2024-09-21T15:01:45.640 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0499999999999997e-06 2024-09-21T15:01:45.640 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.640 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.640 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:45.640 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227293+0000", 2024-09-21T15:01:45.640 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4434e-05 2024-09-21T15:01:45.640 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.640 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.640 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:45.640 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227302+0000", 2024-09-21T15:01:45.640 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0929999999999992e-06 2024-09-21T15:01:45.640 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.641 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.641 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:45.641 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.254720+0000", 2024-09-21T15:01:45.641 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0274177550000001 2024-09-21T15:01:45.641 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.641 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.641 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:45.641 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.254854+0000", 2024-09-21T15:01:45.641 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000133416 2024-09-21T15:01:45.641 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.641 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.641 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:45.641 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.254998+0000", 2024-09-21T15:01:45.641 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014400099999999999 2024-09-21T15:01:45.641 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:45.642 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:45.642 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:45.642 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.642 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.642 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139902 3.e 3:76b9b799:::rbd_data.113664780c85.0000000000000021:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:45.642 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.227319+0000", 2024-09-21T15:01:45.642 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 43.392220275, 2024-09-21T15:01:45.642 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0279584639999999, 2024-09-21T15:01:45.642 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:45.642 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:45.642 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:45.642 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:45.642 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:45.642 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139902 2024-09-21T15:01:45.642 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.643 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:45.643 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.643 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:45.643 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227319+0000", 2024-09-21T15:01:45.643 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:45.643 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.643 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.643 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:45.643 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227319+0000", 2024-09-21T15:01:45.643 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:45.643 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.643 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.643 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:45.643 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227319+0000", 2024-09-21T15:01:45.644 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:45.644 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.644 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.644 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:45.644 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227322+0000", 2024-09-21T15:01:45.644 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1980000000000001e-06 2024-09-21T15:01:45.644 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.644 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.644 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:45.644 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227328+0000", 2024-09-21T15:01:45.644 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4230000000000002e-06 2024-09-21T15:01:45.644 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.644 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.644 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:45.645 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227333+0000", 2024-09-21T15:01:45.645 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0560000000000004e-06 2024-09-21T15:01:45.645 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.645 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.645 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:45.645 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227729+0000", 2024-09-21T15:01:45.645 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00039591799999999999 2024-09-21T15:01:45.645 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.645 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.645 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:45.645 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227738+0000", 2024-09-21T15:01:45.645 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8119999999999997e-06 2024-09-21T15:01:45.645 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.645 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.645 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:45.646 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255021+0000", 2024-09-21T15:01:45.646 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0272826779999999 2024-09-21T15:01:45.646 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.646 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.646 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:45.646 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255147+0000", 2024-09-21T15:01:45.646 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012595600000000001 2024-09-21T15:01:45.646 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.646 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.646 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:45.646 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255278+0000", 2024-09-21T15:01:45.646 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013081400000000001 2024-09-21T15:01:45.646 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:45.647 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:45.647 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:45.647 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.647 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.647 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139945 3.e 3:7756a55d:::rbd_data.113664780c85.0000000000000154:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:45.647 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.229015+0000", 2024-09-21T15:01:45.647 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 43.390524472000003, 2024-09-21T15:01:45.647 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.026468825, 2024-09-21T15:01:45.647 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:45.647 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:45.647 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:45.647 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:45.647 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:45.647 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139945 2024-09-21T15:01:45.647 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.648 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:45.648 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.648 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:45.648 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229015+0000", 2024-09-21T15:01:45.648 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:45.648 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.648 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.648 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:45.648 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229015+0000", 2024-09-21T15:01:45.648 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:45.648 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.648 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.648 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:45.649 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229014+0000", 2024-09-21T15:01:45.649 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:45.649 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.649 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.649 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:45.649 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229017+0000", 2024-09-21T15:01:45.649 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7310000000000002e-06 2024-09-21T15:01:45.649 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.649 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.649 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:45.649 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229023+0000", 2024-09-21T15:01:45.649 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8150000000000002e-06 2024-09-21T15:01:45.649 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.649 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.649 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:45.650 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229028+0000", 2024-09-21T15:01:45.650 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7650000000000001e-06 2024-09-21T15:01:45.650 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.650 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.650 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:45.650 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229082+0000", 2024-09-21T15:01:45.650 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4521999999999997e-05 2024-09-21T15:01:45.650 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.650 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.650 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:45.650 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229092+0000", 2024-09-21T15:01:45.650 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.3940000000000002e-06 2024-09-21T15:01:45.650 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.650 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.650 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:45.650 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255382+0000", 2024-09-21T15:01:45.651 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0262900930000001 2024-09-21T15:01:45.651 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.651 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.651 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:45.651 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255484+0000", 2024-09-21T15:01:45.651 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000101932 2024-09-21T15:01:45.651 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:45.651 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:45.651 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:45.651 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.651 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.651 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139967 3.e 3:74e90d85:::rbd_data.113664780c85.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:45.651 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.229124+0000", 2024-09-21T15:01:45.651 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 43.390415531999999, 2024-09-21T15:01:45.651 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.026724381, 2024-09-21T15:01:45.652 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:45.652 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:45.652 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:45.652 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:45.652 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:45.652 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139967 2024-09-21T15:01:45.652 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.652 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:45.652 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.652 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:45.652 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229124+0000", 2024-09-21T15:01:45.652 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:45.652 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.652 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.652 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:45.653 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229124+0000", 2024-09-21T15:01:45.653 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:45.653 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.653 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.653 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:45.653 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229123+0000", 2024-09-21T15:01:45.653 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:45.653 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.653 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.653 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:45.653 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229126+0000", 2024-09-21T15:01:45.653 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4779999999999998e-06 2024-09-21T15:01:45.653 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.653 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.653 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:45.653 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229131+0000", 2024-09-21T15:01:45.654 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.729e-06 2024-09-21T15:01:45.654 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.654 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.654 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:45.654 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229135+0000", 2024-09-21T15:01:45.654 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5859999999999998e-06 2024-09-21T15:01:45.654 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.654 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.654 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:45.654 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229558+0000", 2024-09-21T15:01:45.654 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00042262499999999999 2024-09-21T15:01:45.654 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.654 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.654 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:45.654 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229569+0000", 2024-09-21T15:01:45.655 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0985000000000001e-05 2024-09-21T15:01:45.655 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.655 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.655 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:45.655 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255509+0000", 2024-09-21T15:01:45.655 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.025940246 2024-09-21T15:01:45.655 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.655 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.655 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:45.655 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255662+0000", 2024-09-21T15:01:45.655 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015324199999999999 2024-09-21T15:01:45.655 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.655 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.656 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:45.656 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255848+0000", 2024-09-21T15:01:45.656 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00018583599999999999 2024-09-21T15:01:45.656 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:45.656 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:45.656 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:45.656 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.656 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.656 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140010 3.e 3:735c06f2:::rbd_data.113664780c85.0000000000000157:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:45.656 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.231531+0000", 2024-09-21T15:01:45.656 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 43.388008667000001, 2024-09-21T15:01:45.656 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0244773149999999, 2024-09-21T15:01:45.656 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:45.656 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:45.657 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:45.657 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:45.657 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:45.657 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140010 2024-09-21T15:01:45.657 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.657 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:45.657 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.657 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:45.657 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231531+0000", 2024-09-21T15:01:45.657 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:45.657 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.657 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.657 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:45.657 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231531+0000", 2024-09-21T15:01:45.657 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:45.658 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.658 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.658 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:45.658 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231530+0000", 2024-09-21T15:01:45.658 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:45.658 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.658 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.658 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:45.658 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231535+0000", 2024-09-21T15:01:45.658 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3970000000000002e-06 2024-09-21T15:01:45.658 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.658 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.658 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:45.658 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231544+0000", 2024-09-21T15:01:45.659 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5909999999999996e-06 2024-09-21T15:01:45.659 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.659 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.659 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:45.659 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231551+0000", 2024-09-21T15:01:45.659 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7129999999999999e-06 2024-09-21T15:01:45.659 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.659 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.659 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:45.659 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231632+0000", 2024-09-21T15:01:45.659 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1181999999999996e-05 2024-09-21T15:01:45.659 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.659 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.659 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:45.659 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231640+0000", 2024-09-21T15:01:45.659 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.551e-06 2024-09-21T15:01:45.660 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.660 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.660 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:45.660 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255884+0000", 2024-09-21T15:01:45.660 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.024244057 2024-09-21T15:01:45.660 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.660 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.660 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:45.660 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256008+0000", 2024-09-21T15:01:45.660 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012351699999999999 2024-09-21T15:01:45.660 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:45.660 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:45.660 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:45.660 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.660 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.660 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140043 3.e 3:74a37859:::rbd_data.113664780c85.0000000000000086:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:45.661 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.232623+0000", 2024-09-21T15:01:45.661 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 43.386916192999998, 2024-09-21T15:01:45.661 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.023518634, 2024-09-21T15:01:45.661 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:45.661 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:45.661 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:45.661 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:45.661 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:45.661 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140043 2024-09-21T15:01:45.661 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.661 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:45.661 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.661 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:45.661 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.232623+0000", 2024-09-21T15:01:45.662 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:45.662 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.662 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.662 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:45.662 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.232623+0000", 2024-09-21T15:01:45.662 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:45.662 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.662 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.662 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:45.662 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.232623+0000", 2024-09-21T15:01:45.662 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:45.662 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.662 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.662 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:45.663 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.232626+0000", 2024-09-21T15:01:45.663 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0019999999999998e-06 2024-09-21T15:01:45.663 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.663 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.663 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:45.663 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.232632+0000", 2024-09-21T15:01:45.663 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3260000000000001e-06 2024-09-21T15:01:45.663 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.663 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.663 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:45.663 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.232637+0000", 2024-09-21T15:01:45.663 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9899999999999997e-06 2024-09-21T15:01:45.663 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.663 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.664 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:45.664 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.232678+0000", 2024-09-21T15:01:45.664 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.0423999999999998e-05 2024-09-21T15:01:45.664 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.664 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.664 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:45.664 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.232686+0000", 2024-09-21T15:01:45.664 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1419999999999993e-06 2024-09-21T15:01:45.664 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.664 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.664 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:45.664 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256037+0000", 2024-09-21T15:01:45.664 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.023351736 2024-09-21T15:01:45.664 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.664 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.665 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:45.665 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256142+0000", 2024-09-21T15:01:45.665 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010443 2024-09-21T15:01:45.665 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:45.665 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:45.665 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:45.665 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.665 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.665 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140063 3.e 3:771cae23:::rbd_data.113664780c85.0000000000000112:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:45.665 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.233948+0000", 2024-09-21T15:01:45.665 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 43.385591126000001, 2024-09-21T15:01:45.665 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0223114710000001, 2024-09-21T15:01:45.665 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:45.665 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:45.665 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:45.666 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:45.666 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:45.666 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140063 2024-09-21T15:01:45.666 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.666 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:45.666 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.666 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:45.666 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233948+0000", 2024-09-21T15:01:45.666 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:45.666 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.666 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.666 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:45.666 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233948+0000", 2024-09-21T15:01:45.666 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:45.667 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.667 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.667 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:45.667 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233948+0000", 2024-09-21T15:01:45.667 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:45.667 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.667 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.667 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:45.667 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233951+0000", 2024-09-21T15:01:45.667 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3380000000000001e-06 2024-09-21T15:01:45.667 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.667 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.667 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:45.667 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233958+0000", 2024-09-21T15:01:45.668 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0990000000000001e-06 2024-09-21T15:01:45.668 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.668 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.668 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:45.668 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233964+0000", 2024-09-21T15:01:45.668 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.852e-06 2024-09-21T15:01:45.668 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.668 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.668 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:45.668 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.234056+0000", 2024-09-21T15:01:45.668 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2392000000000006e-05 2024-09-21T15:01:45.668 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.668 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.668 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:45.668 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.234066+0000", 2024-09-21T15:01:45.669 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1360000000000003e-06 2024-09-21T15:01:45.669 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.669 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.669 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:45.669 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256169+0000", 2024-09-21T15:01:45.669 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0221031949999999 2024-09-21T15:01:45.669 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.669 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.669 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:45.669 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256260+0000", 2024-09-21T15:01:45.669 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0939000000000005e-05 2024-09-21T15:01:45.669 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:45.669 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:45.669 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:45.669 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.670 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.670 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140086 3.e 3:758134b7:::rbd_data.113664780c85.00000000000001ad:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:45.670 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.235110+0000", 2024-09-21T15:01:45.670 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 43.384429427000001, 2024-09-21T15:01:45.670 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.021276217, 2024-09-21T15:01:45.670 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:45.670 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:45.670 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:45.670 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:45.670 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:45.670 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140086 2024-09-21T15:01:45.670 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.670 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:45.670 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.671 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:45.671 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.235110+0000", 2024-09-21T15:01:45.671 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:45.671 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.671 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.671 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:45.671 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.235110+0000", 2024-09-21T15:01:45.671 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:45.671 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.671 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.671 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:45.671 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.235109+0000", 2024-09-21T15:01:45.671 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:45.671 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.672 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.672 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:45.672 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.235114+0000", 2024-09-21T15:01:45.672 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1659999999999996e-06 2024-09-21T15:01:45.672 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.672 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.672 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:45.672 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.235122+0000", 2024-09-21T15:01:45.672 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5590000000000006e-06 2024-09-21T15:01:45.672 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.672 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.672 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:45.672 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.235128+0000", 2024-09-21T15:01:45.672 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8579999999999996e-06 2024-09-21T15:01:45.672 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.673 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.673 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:45.673 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.235169+0000", 2024-09-21T15:01:45.673 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1304000000000003e-05 2024-09-21T15:01:45.673 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.673 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.673 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:45.673 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.235178+0000", 2024-09-21T15:01:45.673 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8610000000000004e-06 2024-09-21T15:01:45.673 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.673 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.673 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:45.673 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256286+0000", 2024-09-21T15:01:45.673 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.021108162 2024-09-21T15:01:45.674 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.674 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.674 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:45.674 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256386+0000", 2024-09-21T15:01:45.674 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9871000000000007e-05 2024-09-21T15:01:45.674 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:45.674 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:45.674 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:45.674 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.674 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.674 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140168 3.e 3:76f4e81d:::rbd_data.113664780c85.000000000000006f:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:45.674 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.236992+0000", 2024-09-21T15:01:45.674 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 43.382547363999997, 2024-09-21T15:01:45.674 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.019513699, 2024-09-21T15:01:45.675 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:45.675 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:45.675 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:45.675 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:45.675 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:45.675 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140168 2024-09-21T15:01:45.675 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.675 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:45.675 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.675 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:45.675 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.236992+0000", 2024-09-21T15:01:45.675 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:45.675 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.676 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.676 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:45.676 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.236992+0000", 2024-09-21T15:01:45.676 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:45.676 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.676 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.676 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:45.676 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.236991+0000", 2024-09-21T15:01:45.676 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:45.676 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.676 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.676 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:45.676 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.236995+0000", 2024-09-21T15:01:45.677 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3100000000000001e-06 2024-09-21T15:01:45.677 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.677 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.677 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:45.677 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237002+0000", 2024-09-21T15:01:45.677 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9929999999999998e-06 2024-09-21T15:01:45.677 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.677 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.677 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:45.677 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237008+0000", 2024-09-21T15:01:45.677 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7559999999999996e-06 2024-09-21T15:01:45.677 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.677 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.677 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:45.677 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237050+0000", 2024-09-21T15:01:45.677 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2852999999999998e-05 2024-09-21T15:01:45.678 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.678 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.678 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:45.678 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237060+0000", 2024-09-21T15:01:45.678 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2399999999999996e-06 2024-09-21T15:01:45.678 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.678 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.678 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:45.678 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256413+0000", 2024-09-21T15:01:45.678 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0193528999999999 2024-09-21T15:01:45.678 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.678 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.678 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:45.678 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256506+0000", 2024-09-21T15:01:45.679 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.3154999999999998e-05 2024-09-21T15:01:45.679 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:45.679 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:45.679 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:45.679 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.679 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.679 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140186 3.e 3:75504345:::rbd_data.113664780c85.00000000000000ed:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:45.679 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.237050+0000", 2024-09-21T15:01:45.679 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 43.382489393, 2024-09-21T15:01:45.679 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0195720150000001, 2024-09-21T15:01:45.679 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:45.679 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:45.679 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:45.679 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:45.680 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:45.680 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140186 2024-09-21T15:01:45.680 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.680 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:45.680 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.680 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:45.680 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237050+0000", 2024-09-21T15:01:45.680 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:45.680 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.680 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.680 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:45.680 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237050+0000", 2024-09-21T15:01:45.680 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:45.680 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.680 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.681 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:45.681 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237049+0000", 2024-09-21T15:01:45.681 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:45.681 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.681 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.681 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:45.681 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237052+0000", 2024-09-21T15:01:45.681 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.016e-06 2024-09-21T15:01:45.681 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.681 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.681 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:45.681 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237070+0000", 2024-09-21T15:01:45.681 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7266999999999999e-05 2024-09-21T15:01:45.681 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.681 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.682 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:45.682 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237075+0000", 2024-09-21T15:01:45.682 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6910000000000002e-06 2024-09-21T15:01:45.682 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.682 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.682 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:45.682 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237232+0000", 2024-09-21T15:01:45.682 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015683599999999999 2024-09-21T15:01:45.682 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.682 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.682 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:45.682 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237237+0000", 2024-09-21T15:01:45.682 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6439999999999999e-06 2024-09-21T15:01:45.682 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.682 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.683 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:45.683 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256534+0000", 2024-09-21T15:01:45.683 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.019296736 2024-09-21T15:01:45.683 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.683 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.683 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:45.683 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256622+0000", 2024-09-21T15:01:45.683 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8386000000000004e-05 2024-09-21T15:01:45.683 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:45.683 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:45.683 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:45.683 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.683 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.683 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140197 3.e 3:71ef5b14:::rbd_data.113664780c85.0000000000000140:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:45.684 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.237251+0000", 2024-09-21T15:01:45.684 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 43.382288365000001, 2024-09-21T15:01:45.684 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.01968386, 2024-09-21T15:01:45.684 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:45.684 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:45.684 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:45.684 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:45.684 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:45.684 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140197 2024-09-21T15:01:45.684 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.684 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:45.684 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.684 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:45.684 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237251+0000", 2024-09-21T15:01:45.684 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:45.685 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.685 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.685 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:45.685 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237251+0000", 2024-09-21T15:01:45.685 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:45.685 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.685 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.685 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:45.685 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237250+0000", 2024-09-21T15:01:45.685 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:45.685 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.685 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.685 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:45.685 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237254+0000", 2024-09-21T15:01:45.686 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.0180000000000003e-06 2024-09-21T15:01:45.686 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.686 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.686 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:45.686 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237262+0000", 2024-09-21T15:01:45.686 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0669999999999992e-06 2024-09-21T15:01:45.686 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.686 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.686 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:45.686 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237269+0000", 2024-09-21T15:01:45.686 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6000000000000003e-06 2024-09-21T15:01:45.686 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.686 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.686 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:45.686 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237319+0000", 2024-09-21T15:01:45.687 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9932000000000002e-05 2024-09-21T15:01:45.687 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.687 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.687 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:45.687 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237326+0000", 2024-09-21T15:01:45.687 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5360000000000004e-06 2024-09-21T15:01:45.687 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.687 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.687 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:45.687 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256657+0000", 2024-09-21T15:01:45.687 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.019330756 2024-09-21T15:01:45.687 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.687 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.687 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:45.688 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256935+0000", 2024-09-21T15:01:45.688 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00027762099999999998 2024-09-21T15:01:45.688 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:45.688 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:45.688 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:45.688 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.688 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.688 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140201 3.e 3:732ea575:::rbd_data.113664780c85.000000000000015c:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:45.688 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.237281+0000", 2024-09-21T15:01:45.688 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 43.382258526999998, 2024-09-21T15:01:45.688 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0198284529999999, 2024-09-21T15:01:45.688 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:45.688 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:45.688 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:45.689 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:45.689 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:45.689 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140201 2024-09-21T15:01:45.689 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.689 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:45.689 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.689 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:45.689 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237281+0000", 2024-09-21T15:01:45.689 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:45.689 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.689 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.689 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:45.689 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237281+0000", 2024-09-21T15:01:45.689 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:45.689 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.690 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.690 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:45.690 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237280+0000", 2024-09-21T15:01:45.690 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:45.690 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.690 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.690 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:45.690 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237284+0000", 2024-09-21T15:01:45.690 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7400000000000002e-06 2024-09-21T15:01:45.690 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.690 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.690 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:45.690 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237292+0000", 2024-09-21T15:01:45.690 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8939999999999993e-06 2024-09-21T15:01:45.690 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.691 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.691 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:45.691 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237298+0000", 2024-09-21T15:01:45.691 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8000000000000004e-06 2024-09-21T15:01:45.691 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.691 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.691 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:45.691 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237521+0000", 2024-09-21T15:01:45.691 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00022326 2024-09-21T15:01:45.691 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.691 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.691 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:45.691 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237529+0000", 2024-09-21T15:01:45.691 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6170000000000001e-06 2024-09-21T15:01:45.691 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.692 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.692 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:45.692 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256968+0000", 2024-09-21T15:01:45.692 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.019439113 2024-09-21T15:01:45.692 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.692 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.692 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:45.692 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.257109+0000", 2024-09-21T15:01:45.692 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014148399999999999 2024-09-21T15:01:45.692 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:45.692 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:45.692 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:45.692 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.692 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.692 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140206 3.e 3:76d38029:::rbd_data.113664780c85.0000000000000179:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:45.692 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.237520+0000", 2024-09-21T15:01:45.693 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 43.382019413000002, 2024-09-21T15:01:45.693 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0210658699999999, 2024-09-21T15:01:45.693 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:45.693 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:45.693 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:45.693 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:45.693 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:45.693 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140206 2024-09-21T15:01:45.693 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.693 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:45.693 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.693 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:45.693 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237520+0000", 2024-09-21T15:01:45.693 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:45.693 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.693 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.694 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:45.694 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237520+0000", 2024-09-21T15:01:45.694 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:45.694 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.694 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.694 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:45.694 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237519+0000", 2024-09-21T15:01:45.694 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:45.694 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.694 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.694 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:45.694 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237523+0000", 2024-09-21T15:01:45.694 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.9009999999999996e-06 2024-09-21T15:01:45.694 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.695 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.695 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:45.695 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237531+0000", 2024-09-21T15:01:45.695 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7069999999999992e-06 2024-09-21T15:01:45.695 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.695 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.695 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:45.695 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237537+0000", 2024-09-21T15:01:45.695 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4559999999999997e-06 2024-09-21T15:01:45.695 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.695 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.695 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:45.695 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237583+0000", 2024-09-21T15:01:45.695 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5532999999999999e-05 2024-09-21T15:01:45.695 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.695 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.696 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:45.696 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237592+0000", 2024-09-21T15:01:45.696 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6009999999999995e-06 2024-09-21T15:01:45.696 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.696 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.696 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:45.696 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.257157+0000", 2024-09-21T15:01:45.696 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0195657279999999 2024-09-21T15:01:45.696 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.696 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.696 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:45.696 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.258586+0000", 2024-09-21T15:01:45.696 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00142847 2024-09-21T15:01:45.696 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:45.696 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:45.697 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:45.697 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.697 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.697 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140338 3.e 3:71dba096:::rbd_data.113664780c85.000000000000018b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:45.697 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.243856+0000", 2024-09-21T15:01:45.697 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 43.37568358, 2024-09-21T15:01:45.697 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0138226889999999, 2024-09-21T15:01:45.697 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:45.697 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:45.697 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:45.697 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:45.697 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:45.697 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140338 2024-09-21T15:01:45.697 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.698 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:45.698 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.698 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:45.698 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243856+0000", 2024-09-21T15:01:45.698 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:45.698 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.698 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.698 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:45.698 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243856+0000", 2024-09-21T15:01:45.698 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:45.698 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.698 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.698 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:45.698 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243854+0000", 2024-09-21T15:01:45.698 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:45.699 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.699 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.699 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:45.699 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243862+0000", 2024-09-21T15:01:45.699 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2570000000000001e-06 2024-09-21T15:01:45.699 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.699 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.699 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:45.699 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243877+0000", 2024-09-21T15:01:45.699 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.571e-05 2024-09-21T15:01:45.699 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.699 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.699 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:45.699 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243884+0000", 2024-09-21T15:01:45.699 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.1269999999999997e-06 2024-09-21T15:01:45.699 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.700 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.700 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:45.700 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243932+0000", 2024-09-21T15:01:45.700 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7655999999999999e-05 2024-09-21T15:01:45.700 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.700 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.700 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:45.700 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243942+0000", 2024-09-21T15:01:45.700 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6679999999999996e-06 2024-09-21T15:01:45.700 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.700 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.700 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:45.700 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.257568+0000", 2024-09-21T15:01:45.700 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0136267670000001 2024-09-21T15:01:45.701 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.701 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.701 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:45.701 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.257678+0000", 2024-09-21T15:01:45.701 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000110011 2024-09-21T15:01:45.701 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:45.701 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:45.701 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:45.701 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.701 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.701 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140345 3.e 3:76ddeb1d:::rbd_data.113664780c85.0000000000000004:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:45.701 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.243927+0000", 2024-09-21T15:01:45.701 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 43.375611996000004, 2024-09-21T15:01:45.702 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0140328439999999, 2024-09-21T15:01:45.702 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:45.702 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:45.702 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:45.702 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:45.702 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:45.702 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140345 2024-09-21T15:01:45.702 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.702 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:45.702 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.702 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:45.702 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243927+0000", 2024-09-21T15:01:45.702 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:45.702 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.702 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.703 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:45.703 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243927+0000", 2024-09-21T15:01:45.703 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:45.703 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.703 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.703 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:45.703 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243927+0000", 2024-09-21T15:01:45.703 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:45.703 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.703 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.703 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:45.703 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243930+0000", 2024-09-21T15:01:45.703 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1319999999999998e-06 2024-09-21T15:01:45.703 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.703 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.704 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:45.704 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243937+0000", 2024-09-21T15:01:45.704 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4330000000000001e-06 2024-09-21T15:01:45.704 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.704 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.704 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:45.704 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243942+0000", 2024-09-21T15:01:45.704 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5090000000000001e-06 2024-09-21T15:01:45.704 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.704 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.704 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:45.704 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243964+0000", 2024-09-21T15:01:45.704 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1838999999999999e-05 2024-09-21T15:01:45.704 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.704 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.705 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:45.705 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243971+0000", 2024-09-21T15:01:45.705 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.131e-06 2024-09-21T15:01:45.705 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.705 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.705 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:45.705 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.257704+0000", 2024-09-21T15:01:45.705 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0137329310000001 2024-09-21T15:01:45.705 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.705 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.705 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:45.705 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.257805+0000", 2024-09-21T15:01:45.705 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000101121 2024-09-21T15:01:45.705 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.705 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.706 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:45.706 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.257960+0000", 2024-09-21T15:01:45.706 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015518500000000001 2024-09-21T15:01:45.706 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:45.706 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:45.706 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:45.706 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.706 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.706 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140350 3.e 3:76b9b799:::rbd_data.113664780c85.0000000000000021:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:45.706 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.243989+0000", 2024-09-21T15:01:45.706 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 43.375550244999999, 2024-09-21T15:01:45.706 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0142022340000001, 2024-09-21T15:01:45.706 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:45.706 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:45.706 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:45.707 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:45.707 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:45.707 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140350 2024-09-21T15:01:45.707 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.707 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:45.707 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.707 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:45.707 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243989+0000", 2024-09-21T15:01:45.707 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:45.707 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.707 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.707 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:45.707 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243989+0000", 2024-09-21T15:01:45.708 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:45.708 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.708 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.708 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:45.708 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243989+0000", 2024-09-21T15:01:45.708 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:45.708 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.708 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.708 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:45.708 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243992+0000", 2024-09-21T15:01:45.708 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.236e-06 2024-09-21T15:01:45.708 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.708 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.708 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:45.708 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243998+0000", 2024-09-21T15:01:45.709 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2500000000000003e-06 2024-09-21T15:01:45.709 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.709 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.709 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:45.709 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.244003+0000", 2024-09-21T15:01:45.709 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7740000000000004e-06 2024-09-21T15:01:45.709 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.709 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.709 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:45.709 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.244047+0000", 2024-09-21T15:01:45.709 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3665999999999998e-05 2024-09-21T15:01:45.709 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.709 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.709 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:45.709 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.244055+0000", 2024-09-21T15:01:45.709 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0639999999999994e-06 2024-09-21T15:01:45.710 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.710 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.710 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:45.710 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.257985+0000", 2024-09-21T15:01:45.710 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0139306079999999 2024-09-21T15:01:45.710 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.710 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.710 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:45.710 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.258084+0000", 2024-09-21T15:01:45.710 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8610000000000006e-05 2024-09-21T15:01:45.710 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.710 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.710 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:45.710 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.258191+0000", 2024-09-21T15:01:45.710 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000107518 2024-09-21T15:01:45.710 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:45.711 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:45.711 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:45.711 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.711 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.711 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140394 3.e 3:7756a55d:::rbd_data.113664780c85.0000000000000154:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:45.711 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.245038+0000", 2024-09-21T15:01:45.711 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 43.374501027999997, 2024-09-21T15:01:45.711 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.013225182, 2024-09-21T15:01:45.711 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:45.711 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:45.711 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:45.711 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:45.711 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:45.711 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140394 2024-09-21T15:01:45.711 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.712 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:45.712 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.712 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:45.712 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.245038+0000", 2024-09-21T15:01:45.712 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:45.712 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.712 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.712 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:45.712 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.245038+0000", 2024-09-21T15:01:45.712 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:45.712 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.712 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.712 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:45.712 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.245037+0000", 2024-09-21T15:01:45.712 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:45.713 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.713 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.713 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:45.713 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.245044+0000", 2024-09-21T15:01:45.713 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3239999999999999e-06 2024-09-21T15:01:45.713 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.713 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.713 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:45.713 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.245061+0000", 2024-09-21T15:01:45.713 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7166000000000001e-05 2024-09-21T15:01:45.713 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.713 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.713 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:45.713 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.245069+0000", 2024-09-21T15:01:45.713 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0779999999999996e-06 2024-09-21T15:01:45.713 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.714 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.714 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:45.714 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.245118+0000", 2024-09-21T15:01:45.714 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8887000000000001e-05 2024-09-21T15:01:45.714 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.714 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.714 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:45.714 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.245126+0000", 2024-09-21T15:01:45.714 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0409999999999998e-06 2024-09-21T15:01:45.714 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.714 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.714 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:45.714 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.258206+0000", 2024-09-21T15:01:45.714 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.013080451 2024-09-21T15:01:45.714 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:45.715 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:45.715 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:45.715 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.258264+0000", 2024-09-21T15:01:45.715 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7294999999999999e-05 2024-09-21T15:01:45.715 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:45.715 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:45.715 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:45.715 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:45.715 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:45.715 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:45.716 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T15:01:45.717 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:01:45.714+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:01:45.817 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:01:45.812+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:01:45.872 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:45.873 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:01:45.873 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T15:01:45.873 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:45.883 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T15:01:45.910 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T15:01:45.918 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:01:45.912+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:01:45.991 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:45.991 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:01:45.991 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T15:01:45.991 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T15:01:45.991 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:46.001 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T15:01:46.045 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:01:46.046+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:01:46.059 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T15:01:46.080 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T15:01:46.105 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:46.105 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:46.115 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T15:01:46.119 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:01:46.118+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:01:46.220 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:01:46.217+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:01:46.224 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:46.224 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:01:46.224 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T15:01:46.224 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:46.234 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T15:01:46.266 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T15:01:46.309 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T15:01:46.320 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:01:46.316+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:01:46.333 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T15:01:46.360 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T15:01:46.385 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T15:01:46.396 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:46.396 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:46.409 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T15:01:46.418 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T15:01:46.418 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:01:46.418 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:01:46.418 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:01:46.418 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:01:46.419 INFO:teuthology.orchestra.run.smithi032.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-21T15:01:46.419 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:01:46.419 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:01:46.419 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:01:46.421 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:01:46.416+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:01:46.424 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-21T15:01:46.424 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-21T15:01:46.509 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:46.509 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:46.518 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T15:01:46.521 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:01:46.521+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:01:46.622 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:46.622 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:46.622 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:01:46.616+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:01:46.632 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T15:01:46.723 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:01:46.721+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:01:46.784 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:46.785 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:46.794 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T15:01:46.824 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:01:46.820+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:01:46.914 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:46.914 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:46.925 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T15:01:46.926 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:01:46.921+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:01:47.025 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:01:47.020+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:01:47.043 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:47.043 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:47.054 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T15:01:47.126 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:01:47.125+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:01:47.159 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:47.159 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:47.170 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T15:01:47.227 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:01:47.225+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:01:47.283 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:47.283 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:47.286 INFO:teuthology.orchestra.run.smithi032.stderr:noscrub is unset 2024-09-21T15:01:47.295 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T15:01:47.346 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:01:47.345+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:01:47.400 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:47.400 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:47.411 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T15:01:47.429 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:01:47.425+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:01:47.461 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 5, 6, 7] out_osds: [4] dead_osds: [4] live_osds: [0, 1, 2, 3, 5, 6, 7] 2024-09-21T15:01:47.461 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T15:01:47.461 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-21T15:01:47.461 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-21T15:01:47.524 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:47.524 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:47.529 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:01:47.524+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:01:47.535 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T15:01:47.630 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:01:47.629+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:01:47.692 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:47.692 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:47.702 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T15:01:47.730 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:01:47.729+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:01:47.777 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-21T15:01:47.777 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:47.777 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 215, 2024-09-21T15:01:47.777 INFO:teuthology.orchestra.run.smithi032.stdout: "fsid": "caf51e55-9234-4a26-bcd2-d60811962560", 2024-09-21T15:01:47.777 INFO:teuthology.orchestra.run.smithi032.stdout: "created": "2024-09-21T14:57:50.338822+0000", 2024-09-21T15:01:47.777 INFO:teuthology.orchestra.run.smithi032.stdout: "modified": "2024-09-21T15:01:47.266268+0000", 2024-09-21T15:01:47.777 INFO:teuthology.orchestra.run.smithi032.stdout: "last_up_change": "2024-09-21T15:01:34.172764+0000", 2024-09-21T15:01:47.777 INFO:teuthology.orchestra.run.smithi032.stdout: "last_in_change": "2024-09-21T15:01:21.319966+0000", 2024-09-21T15:01:47.777 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-21T15:01:47.777 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_num": 5804032, 2024-09-21T15:01:47.778 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_set": [ 2024-09-21T15:01:47.778 INFO:teuthology.orchestra.run.smithi032.stdout: "nodeep-scrub", 2024-09-21T15:01:47.778 INFO:teuthology.orchestra.run.smithi032.stdout: "pglog_hardlimit", 2024-09-21T15:01:47.778 INFO:teuthology.orchestra.run.smithi032.stdout: "purged_snapdirs", 2024-09-21T15:01:47.778 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_deletes", 2024-09-21T15:01:47.778 INFO:teuthology.orchestra.run.smithi032.stdout: "sortbitwise" 2024-09-21T15:01:47.778 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-21T15:01:47.778 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_version": 5, 2024-09-21T15:01:47.778 INFO:teuthology.orchestra.run.smithi032.stdout: "full_ratio": 0.94999998807907104, 2024-09-21T15:01:47.778 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-21T15:01:47.778 INFO:teuthology.orchestra.run.smithi032.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-21T15:01:47.778 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_snapshot": "", 2024-09-21T15:01:47.778 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_max": 3, 2024-09-21T15:01:47.778 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd": 8, 2024-09-21T15:01:47.779 INFO:teuthology.orchestra.run.smithi032.stdout: "require_min_compat_client": "luminous", 2024-09-21T15:01:47.779 INFO:teuthology.orchestra.run.smithi032.stdout: "min_compat_client": "luminous", 2024-09-21T15:01:47.779 INFO:teuthology.orchestra.run.smithi032.stdout: "require_osd_release": "reef", 2024-09-21T15:01:47.779 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_crimson": false, 2024-09-21T15:01:47.779 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2024-09-21T15:01:47.779 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.779 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 1, 2024-09-21T15:01:47.779 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": ".mgr", 2024-09-21T15:01:47.779 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-09-21T14:58:01.842810+0000", 2024-09-21T15:01:47.779 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-09-21T15:01:47.779 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2024-09-21T15:01:47.779 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-09-21T15:01:47.779 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-09-21T15:01:47.779 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-09-21T15:01:47.779 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-09-21T15:01:47.779 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-09-21T15:01:47.780 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-09-21T15:01:47.780 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T15:01:47.780 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T15:01:47.780 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-09-21T15:01:47.780 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-09-21T15:01:47.780 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 1, 2024-09-21T15:01:47.780 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 1, 2024-09-21T15:01:47.780 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 1, 2024-09-21T15:01:47.780 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 1, 2024-09-21T15:01:47.780 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 1, 2024-09-21T15:01:47.780 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-09-21T15:01:47.780 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-09-21T15:01:47.780 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-09-21T15:01:47.780 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-09-21T15:01:47.781 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-09-21T15:01:47.781 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-09-21T15:01:47.781 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-09-21T15:01:47.781 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.781 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "18", 2024-09-21T15:01:47.781 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-09-21T15:01:47.781 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T15:01:47.781 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T15:01:47.781 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-09-21T15:01:47.781 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-09-21T15:01:47.781 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-09-21T15:01:47.781 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-09-21T15:01:47.781 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-09-21T15:01:47.781 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-09-21T15:01:47.781 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-09-21T15:01:47.781 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-09-21T15:01:47.782 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-09-21T15:01:47.782 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-09-21T15:01:47.782 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-09-21T15:01:47.782 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-09-21T15:01:47.782 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-09-21T15:01:47.782 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-09-21T15:01:47.782 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-09-21T15:01:47.782 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T15:01:47.782 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T15:01:47.782 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T15:01:47.782 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-09-21T15:01:47.782 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-09-21T15:01:47.782 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-09-21T15:01:47.782 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-09-21T15:01:47.782 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-09-21T15:01:47.783 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.783 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-09-21T15:01:47.783 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-09-21T15:01:47.783 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-09-21T15:01:47.783 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-09-21T15:01:47.783 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-09-21T15:01:47.783 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T15:01:47.783 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-09-21T15:01:47.783 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-09-21T15:01:47.783 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-09-21T15:01:47.783 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-09-21T15:01:47.783 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-09-21T15:01:47.783 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-09-21T15:01:47.783 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_max": 32, 2024-09-21T15:01:47.783 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 1 2024-09-21T15:01:47.784 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.784 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-09-21T15:01:47.784 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": {} 2024-09-21T15:01:47.784 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.784 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-09-21T15:01:47.784 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 8, 2024-09-21T15:01:47.784 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 8, 2024-09-21T15:01:47.784 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 0.25, 2024-09-21T15:01:47.784 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 2, 2024-09-21T15:01:47.784 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 2, 2024-09-21T15:01:47.784 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 0.95859527587890625, 2024-09-21T15:01:47.784 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 0.95999997854232788, 2024-09-21T15:01:47.784 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 0.95999997854232788 2024-09-21T15:01:47.784 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:47.785 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.785 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.785 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 2, 2024-09-21T15:01:47.785 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "rbd", 2024-09-21T15:01:47.785 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-09-21T14:58:08.284925+0000", 2024-09-21T15:01:47.785 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 8193, 2024-09-21T15:01:47.785 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T15:01:47.785 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-09-21T15:01:47.785 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-09-21T15:01:47.785 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-09-21T15:01:47.785 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-09-21T15:01:47.785 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-09-21T15:01:47.785 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-09-21T15:01:47.785 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T15:01:47.785 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T15:01:47.786 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-09-21T15:01:47.786 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-09-21T15:01:47.786 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2024-09-21T15:01:47.786 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2024-09-21T15:01:47.786 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2024-09-21T15:01:47.786 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2024-09-21T15:01:47.786 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2024-09-21T15:01:47.786 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-09-21T15:01:47.786 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-09-21T15:01:47.786 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-09-21T15:01:47.786 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-09-21T15:01:47.786 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-09-21T15:01:47.786 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-09-21T15:01:47.786 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-09-21T15:01:47.786 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.786 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "22", 2024-09-21T15:01:47.787 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-09-21T15:01:47.787 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T15:01:47.787 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T15:01:47.787 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-09-21T15:01:47.787 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-09-21T15:01:47.787 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 3, 2024-09-21T15:01:47.787 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 22, 2024-09-21T15:01:47.787 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-09-21T15:01:47.787 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-09-21T15:01:47.787 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-09-21T15:01:47.787 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-09-21T15:01:47.788 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-09-21T15:01:47.788 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-09-21T15:01:47.788 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-09-21T15:01:47.788 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-09-21T15:01:47.788 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-09-21T15:01:47.788 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-09-21T15:01:47.788 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-09-21T15:01:47.788 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T15:01:47.788 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T15:01:47.788 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T15:01:47.788 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-09-21T15:01:47.788 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-09-21T15:01:47.788 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-09-21T15:01:47.788 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-09-21T15:01:47.788 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-09-21T15:01:47.788 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.789 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-09-21T15:01:47.789 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-09-21T15:01:47.789 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-09-21T15:01:47.789 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-09-21T15:01:47.789 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-09-21T15:01:47.789 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T15:01:47.789 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-09-21T15:01:47.789 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-09-21T15:01:47.789 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-09-21T15:01:47.789 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-09-21T15:01:47.789 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-09-21T15:01:47.789 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2024-09-21T15:01:47.789 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-09-21T15:01:47.790 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd": {} 2024-09-21T15:01:47.790 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.790 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-09-21T15:01:47.790 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 5, 2024-09-21T15:01:47.790 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 5, 2024-09-21T15:01:47.790 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 0.75, 2024-09-21T15:01:47.790 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 3.75, 2024-09-21T15:01:47.790 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 3.75, 2024-09-21T15:01:47.790 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 0.95859527587890625, 2024-09-21T15:01:47.790 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 0.95999997854232788, 2024-09-21T15:01:47.790 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 0.95999997854232788 2024-09-21T15:01:47.790 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:47.790 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.790 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.790 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 3, 2024-09-21T15:01:47.791 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "pool_client.0", 2024-09-21T15:01:47.791 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-09-21T14:58:17.510709+0000", 2024-09-21T15:01:47.791 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 8193, 2024-09-21T15:01:47.791 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T15:01:47.791 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-09-21T15:01:47.791 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-09-21T15:01:47.791 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-09-21T15:01:47.791 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-09-21T15:01:47.791 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-09-21T15:01:47.791 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-09-21T15:01:47.791 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T15:01:47.791 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T15:01:47.791 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-09-21T15:01:47.791 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-09-21T15:01:47.791 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 32, 2024-09-21T15:01:47.791 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 32, 2024-09-21T15:01:47.792 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 32, 2024-09-21T15:01:47.792 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 32, 2024-09-21T15:01:47.792 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 32, 2024-09-21T15:01:47.792 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-09-21T15:01:47.792 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-09-21T15:01:47.792 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-09-21T15:01:47.792 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-09-21T15:01:47.792 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-09-21T15:01:47.792 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-09-21T15:01:47.792 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-09-21T15:01:47.792 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.792 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "215", 2024-09-21T15:01:47.792 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-09-21T15:01:47.792 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T15:01:47.792 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T15:01:47.793 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-09-21T15:01:47.793 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-09-21T15:01:47.793 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 140, 2024-09-21T15:01:47.793 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 215, 2024-09-21T15:01:47.793 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-09-21T15:01:47.793 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-09-21T15:01:47.793 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-09-21T15:01:47.793 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-09-21T15:01:47.793 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-09-21T15:01:47.793 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-09-21T15:01:47.793 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-09-21T15:01:47.793 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-09-21T15:01:47.793 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-09-21T15:01:47.793 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-09-21T15:01:47.793 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-09-21T15:01:47.793 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T15:01:47.794 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T15:01:47.794 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T15:01:47.794 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-09-21T15:01:47.794 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-09-21T15:01:47.794 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-09-21T15:01:47.794 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-09-21T15:01:47.794 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-09-21T15:01:47.794 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.794 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-09-21T15:01:47.794 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-09-21T15:01:47.794 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-09-21T15:01:47.794 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-09-21T15:01:47.794 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-09-21T15:01:47.794 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T15:01:47.794 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-09-21T15:01:47.795 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-09-21T15:01:47.795 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-09-21T15:01:47.795 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-09-21T15:01:47.795 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-09-21T15:01:47.795 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2024-09-21T15:01:47.795 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-09-21T15:01:47.795 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd": {} 2024-09-21T15:01:47.795 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.795 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-09-21T15:01:47.795 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 1.4900000095367432, 2024-09-21T15:01:47.795 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 1.4900000095367432, 2024-09-21T15:01:47.795 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 0.87999999523162842, 2024-09-21T15:01:47.795 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 1.309999942779541, 2024-09-21T15:01:47.795 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 1.309999942779541, 2024-09-21T15:01:47.795 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 0.95859527587890625, 2024-09-21T15:01:47.796 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 0.95999997854232788, 2024-09-21T15:01:47.796 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 0.95999997854232788 2024-09-21T15:01:47.796 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:47.796 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:47.796 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-21T15:01:47.796 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2024-09-21T15:01:47.796 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.796 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-09-21T15:01:47.796 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "a4a73316-3a94-42e9-9a51-deb46a66bb06", 2024-09-21T15:01:47.796 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-21T15:01:47.796 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-21T15:01:47.796 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-21T15:01:47.796 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-21T15:01:47.797 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-21T15:01:47.797 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-21T15:01:47.797 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2024-09-21T15:01:47.797 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 192, 2024-09-21T15:01:47.797 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-21T15:01:47.797 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-21T15:01:47.797 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-21T15:01:47.797 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:01:47.797 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.797 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:01:47.797 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6808", 2024-09-21T15:01:47.797 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4016672243 2024-09-21T15:01:47.797 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.797 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.797 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:01:47.798 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6809", 2024-09-21T15:01:47.798 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4016672243 2024-09-21T15:01:47.798 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:47.798 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:47.798 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.798 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-21T15:01:47.798 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:01:47.798 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.798 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:01:47.798 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6810", 2024-09-21T15:01:47.798 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4016672243 2024-09-21T15:01:47.798 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.798 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.798 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:01:47.799 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6811", 2024-09-21T15:01:47.799 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4016672243 2024-09-21T15:01:47.799 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:47.799 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:47.799 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.799 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-21T15:01:47.799 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:01:47.799 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.799 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:01:47.799 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6814", 2024-09-21T15:01:47.799 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4016672243 2024-09-21T15:01:47.799 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.799 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.800 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:01:47.800 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6815", 2024-09-21T15:01:47.800 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4016672243 2024-09-21T15:01:47.800 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:47.800 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:47.800 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.800 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-21T15:01:47.800 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:01:47.800 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.800 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:01:47.800 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6812", 2024-09-21T15:01:47.800 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4016672243 2024-09-21T15:01:47.800 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.800 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.801 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:01:47.801 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6813", 2024-09-21T15:01:47.801 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4016672243 2024-09-21T15:01:47.801 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:47.801 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:47.801 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.801 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6809/4016672243", 2024-09-21T15:01:47.801 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6811/4016672243", 2024-09-21T15:01:47.801 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6815/4016672243", 2024-09-21T15:01:47.801 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6813/4016672243", 2024-09-21T15:01:47.801 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-21T15:01:47.801 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-21T15:01:47.801 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-21T15:01:47.801 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:47.801 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.801 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.802 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-09-21T15:01:47.802 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "006757a0-94d1-4a9b-aef8-73634cc5828e", 2024-09-21T15:01:47.802 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-21T15:01:47.802 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-21T15:01:47.802 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-21T15:01:47.802 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-21T15:01:47.802 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-21T15:01:47.802 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-21T15:01:47.802 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2024-09-21T15:01:47.802 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 192, 2024-09-21T15:01:47.802 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-21T15:01:47.802 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-21T15:01:47.802 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-21T15:01:47.802 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:01:47.802 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.803 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:01:47.803 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6824", 2024-09-21T15:01:47.803 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 641090577 2024-09-21T15:01:47.803 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.803 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.803 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:01:47.803 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6825", 2024-09-21T15:01:47.803 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 641090577 2024-09-21T15:01:47.803 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:47.803 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:47.803 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.803 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-21T15:01:47.803 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:01:47.803 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.803 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:01:47.804 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6826", 2024-09-21T15:01:47.804 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 641090577 2024-09-21T15:01:47.804 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.804 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.804 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:01:47.804 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6827", 2024-09-21T15:01:47.804 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 641090577 2024-09-21T15:01:47.804 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:47.804 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:47.804 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.804 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-21T15:01:47.804 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:01:47.804 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.804 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:01:47.804 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6830", 2024-09-21T15:01:47.805 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 641090577 2024-09-21T15:01:47.805 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.805 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.805 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:01:47.805 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6831", 2024-09-21T15:01:47.805 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 641090577 2024-09-21T15:01:47.805 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:47.805 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:47.805 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.805 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-21T15:01:47.805 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:01:47.805 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.805 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:01:47.805 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6828", 2024-09-21T15:01:47.805 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 641090577 2024-09-21T15:01:47.806 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.806 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.806 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:01:47.806 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6829", 2024-09-21T15:01:47.806 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 641090577 2024-09-21T15:01:47.806 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:47.806 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:47.806 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.806 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6825/641090577", 2024-09-21T15:01:47.806 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6827/641090577", 2024-09-21T15:01:47.806 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6831/641090577", 2024-09-21T15:01:47.806 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6829/641090577", 2024-09-21T15:01:47.806 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-21T15:01:47.806 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-21T15:01:47.806 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-21T15:01:47.807 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:47.807 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.807 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.807 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-09-21T15:01:47.807 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "0b499a23-820e-4e6c-a558-adbf88b7014f", 2024-09-21T15:01:47.807 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-21T15:01:47.807 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-21T15:01:47.807 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-21T15:01:47.807 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-21T15:01:47.807 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-21T15:01:47.807 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-21T15:01:47.807 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2024-09-21T15:01:47.807 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 198, 2024-09-21T15:01:47.807 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-21T15:01:47.807 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-21T15:01:47.808 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-21T15:01:47.808 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:01:47.808 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.808 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:01:47.808 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6800", 2024-09-21T15:01:47.808 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2609154414 2024-09-21T15:01:47.808 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.808 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.808 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:01:47.808 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6801", 2024-09-21T15:01:47.808 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2609154414 2024-09-21T15:01:47.808 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:47.808 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:47.808 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.809 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-21T15:01:47.809 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:01:47.809 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.809 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:01:47.809 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6802", 2024-09-21T15:01:47.809 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2609154414 2024-09-21T15:01:47.809 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.809 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.809 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:01:47.809 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6803", 2024-09-21T15:01:47.809 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2609154414 2024-09-21T15:01:47.809 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:47.809 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:47.809 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.809 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-21T15:01:47.810 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:01:47.810 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.810 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:01:47.810 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6806", 2024-09-21T15:01:47.810 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2609154414 2024-09-21T15:01:47.810 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.810 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.810 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:01:47.810 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6807", 2024-09-21T15:01:47.810 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2609154414 2024-09-21T15:01:47.810 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:47.810 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:47.810 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.811 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-21T15:01:47.811 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:01:47.811 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.811 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:01:47.811 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6804", 2024-09-21T15:01:47.811 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2609154414 2024-09-21T15:01:47.811 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.811 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.811 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:01:47.811 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6805", 2024-09-21T15:01:47.811 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2609154414 2024-09-21T15:01:47.811 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:47.811 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:47.811 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.811 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6801/2609154414", 2024-09-21T15:01:47.812 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6803/2609154414", 2024-09-21T15:01:47.812 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6807/2609154414", 2024-09-21T15:01:47.812 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6805/2609154414", 2024-09-21T15:01:47.812 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-21T15:01:47.812 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-21T15:01:47.812 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-21T15:01:47.812 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:47.812 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.812 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.812 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-09-21T15:01:47.812 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "5a57b3dd-4a07-45b7-b159-3cac4c7eb262", 2024-09-21T15:01:47.812 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-21T15:01:47.812 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-21T15:01:47.812 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-21T15:01:47.812 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 0.833831787109375, 2024-09-21T15:01:47.813 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-21T15:01:47.813 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-21T15:01:47.813 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2024-09-21T15:01:47.813 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 192, 2024-09-21T15:01:47.813 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-21T15:01:47.813 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-21T15:01:47.813 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-21T15:01:47.813 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:01:47.813 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.813 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:01:47.813 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6816", 2024-09-21T15:01:47.813 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1462563376 2024-09-21T15:01:47.813 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.813 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.813 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:01:47.813 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6817", 2024-09-21T15:01:47.814 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1462563376 2024-09-21T15:01:47.814 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:47.814 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:47.814 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.814 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-21T15:01:47.814 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:01:47.814 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.814 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:01:47.814 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6818", 2024-09-21T15:01:47.814 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1462563376 2024-09-21T15:01:47.814 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.814 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.814 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:01:47.814 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6819", 2024-09-21T15:01:47.814 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1462563376 2024-09-21T15:01:47.815 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:47.815 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:47.815 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.815 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-21T15:01:47.815 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:01:47.815 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.815 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:01:47.815 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6822", 2024-09-21T15:01:47.815 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1462563376 2024-09-21T15:01:47.815 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.815 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.815 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:01:47.815 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6823", 2024-09-21T15:01:47.815 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1462563376 2024-09-21T15:01:47.815 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:47.816 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:47.816 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.816 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-21T15:01:47.816 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:01:47.816 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.816 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:01:47.816 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6820", 2024-09-21T15:01:47.816 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1462563376 2024-09-21T15:01:47.816 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.816 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.816 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:01:47.816 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6821", 2024-09-21T15:01:47.816 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1462563376 2024-09-21T15:01:47.816 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:47.816 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:47.817 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.817 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6817/1462563376", 2024-09-21T15:01:47.817 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6819/1462563376", 2024-09-21T15:01:47.817 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6823/1462563376", 2024-09-21T15:01:47.817 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6821/1462563376", 2024-09-21T15:01:47.817 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-21T15:01:47.817 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-21T15:01:47.817 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-21T15:01:47.817 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:47.817 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.817 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.817 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-09-21T15:01:47.817 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "39a00807-7517-474e-8842-623dd7093cc5", 2024-09-21T15:01:47.817 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 0, 2024-09-21T15:01:47.817 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 0, 2024-09-21T15:01:47.818 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 0, 2024-09-21T15:01:47.818 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-21T15:01:47.818 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-21T15:01:47.818 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-21T15:01:47.818 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2024-09-21T15:01:47.818 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 147, 2024-09-21T15:01:47.818 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 202, 2024-09-21T15:01:47.818 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-21T15:01:47.818 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-21T15:01:47.818 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:01:47.818 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.818 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:01:47.818 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6801", 2024-09-21T15:01:47.818 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 144412042 2024-09-21T15:01:47.818 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.819 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.819 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:01:47.819 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6803", 2024-09-21T15:01:47.819 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 144412042 2024-09-21T15:01:47.819 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:47.819 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:47.819 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.819 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-21T15:01:47.819 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:01:47.819 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.819 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:01:47.819 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6805", 2024-09-21T15:01:47.819 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 144412042 2024-09-21T15:01:47.819 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.819 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.820 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:01:47.820 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6807", 2024-09-21T15:01:47.820 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 144412042 2024-09-21T15:01:47.820 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:47.820 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:47.820 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.820 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-21T15:01:47.820 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:01:47.820 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.820 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:01:47.820 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6812", 2024-09-21T15:01:47.820 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 144412042 2024-09-21T15:01:47.820 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.820 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.820 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:01:47.821 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6814", 2024-09-21T15:01:47.821 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 144412042 2024-09-21T15:01:47.821 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:47.821 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:47.821 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.821 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-21T15:01:47.821 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:01:47.821 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.821 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:01:47.821 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6809", 2024-09-21T15:01:47.821 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 144412042 2024-09-21T15:01:47.821 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.821 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.821 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:01:47.821 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6811", 2024-09-21T15:01:47.821 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 144412042 2024-09-21T15:01:47.822 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:47.822 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:47.822 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.822 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.46:6803/144412042", 2024-09-21T15:01:47.822 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.46:6807/144412042", 2024-09-21T15:01:47.822 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.46:6814/144412042", 2024-09-21T15:01:47.822 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.46:6811/144412042", 2024-09-21T15:01:47.822 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-21T15:01:47.822 INFO:teuthology.orchestra.run.smithi032.stdout: "exists" 2024-09-21T15:01:47.822 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:47.822 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.822 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.822 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-09-21T15:01:47.822 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "edf9f045-ddeb-4504-a0fc-1a0e4e853550", 2024-09-21T15:01:47.822 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-21T15:01:47.823 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-21T15:01:47.823 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-21T15:01:47.823 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-21T15:01:47.823 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-21T15:01:47.823 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-21T15:01:47.823 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2024-09-21T15:01:47.823 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 177, 2024-09-21T15:01:47.823 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-21T15:01:47.823 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-21T15:01:47.823 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-21T15:01:47.823 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:01:47.823 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.823 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:01:47.823 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6800", 2024-09-21T15:01:47.823 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 849101442 2024-09-21T15:01:47.823 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.824 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.824 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:01:47.824 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6802", 2024-09-21T15:01:47.824 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 849101442 2024-09-21T15:01:47.824 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:47.824 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:47.824 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.824 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-21T15:01:47.824 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:01:47.824 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.824 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:01:47.824 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6804", 2024-09-21T15:01:47.824 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 849101442 2024-09-21T15:01:47.824 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.824 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.824 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:01:47.825 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6806", 2024-09-21T15:01:47.825 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 849101442 2024-09-21T15:01:47.825 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:47.825 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:47.825 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.825 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-21T15:01:47.825 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:01:47.825 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.825 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:01:47.825 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6813", 2024-09-21T15:01:47.825 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 849101442 2024-09-21T15:01:47.825 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.825 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.825 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:01:47.825 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6815", 2024-09-21T15:01:47.825 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 849101442 2024-09-21T15:01:47.826 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:47.826 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:47.826 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.826 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-21T15:01:47.826 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:01:47.826 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.826 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:01:47.826 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6808", 2024-09-21T15:01:47.826 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 849101442 2024-09-21T15:01:47.826 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.826 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.826 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:01:47.826 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6810", 2024-09-21T15:01:47.826 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 849101442 2024-09-21T15:01:47.827 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:47.827 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:47.827 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.827 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.46:6802/849101442", 2024-09-21T15:01:47.827 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.46:6806/849101442", 2024-09-21T15:01:47.827 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.46:6815/849101442", 2024-09-21T15:01:47.827 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.46:6810/849101442", 2024-09-21T15:01:47.827 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-21T15:01:47.827 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-21T15:01:47.827 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-21T15:01:47.827 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:47.827 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.827 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.827 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-09-21T15:01:47.827 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "c820ea6b-88fe-4421-83ec-17c466691fea", 2024-09-21T15:01:47.828 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-21T15:01:47.828 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-21T15:01:47.828 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-21T15:01:47.828 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 0.834930419921875, 2024-09-21T15:01:47.828 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-21T15:01:47.828 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-21T15:01:47.828 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2024-09-21T15:01:47.828 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 199, 2024-09-21T15:01:47.828 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-21T15:01:47.828 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-21T15:01:47.828 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-21T15:01:47.828 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:01:47.828 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.828 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:01:47.828 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6817", 2024-09-21T15:01:47.828 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 847189172 2024-09-21T15:01:47.829 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.829 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.829 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:01:47.829 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6818", 2024-09-21T15:01:47.829 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 847189172 2024-09-21T15:01:47.829 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:47.829 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:47.829 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.829 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-21T15:01:47.829 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:01:47.829 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.829 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:01:47.829 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6820", 2024-09-21T15:01:47.829 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 847189172 2024-09-21T15:01:47.829 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.830 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.830 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:01:47.830 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6822", 2024-09-21T15:01:47.830 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 847189172 2024-09-21T15:01:47.830 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:47.830 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:47.830 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.830 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-21T15:01:47.830 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:01:47.830 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.830 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:01:47.830 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6827", 2024-09-21T15:01:47.830 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 847189172 2024-09-21T15:01:47.831 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.831 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.831 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:01:47.831 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6828", 2024-09-21T15:01:47.831 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 847189172 2024-09-21T15:01:47.831 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:47.831 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:47.831 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.831 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-21T15:01:47.831 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:01:47.831 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.831 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:01:47.831 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6823", 2024-09-21T15:01:47.831 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 847189172 2024-09-21T15:01:47.832 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.832 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.832 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:01:47.832 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6825", 2024-09-21T15:01:47.832 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 847189172 2024-09-21T15:01:47.832 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:47.832 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:47.832 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.832 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.46:6818/847189172", 2024-09-21T15:01:47.832 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.46:6822/847189172", 2024-09-21T15:01:47.832 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.46:6828/847189172", 2024-09-21T15:01:47.832 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.46:6825/847189172", 2024-09-21T15:01:47.832 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-21T15:01:47.832 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-21T15:01:47.832 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-21T15:01:47.832 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:47.833 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.833 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.833 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-09-21T15:01:47.833 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "3a2f82a4-28a0-4fcb-9d6b-2285fa4d07ac", 2024-09-21T15:01:47.833 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-21T15:01:47.833 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-21T15:01:47.833 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-21T15:01:47.833 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-21T15:01:47.833 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-21T15:01:47.833 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-21T15:01:47.833 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 15, 2024-09-21T15:01:47.833 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 193, 2024-09-21T15:01:47.833 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-21T15:01:47.833 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-21T15:01:47.833 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-21T15:01:47.834 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:01:47.834 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.834 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:01:47.834 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6816", 2024-09-21T15:01:47.834 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3894963553 2024-09-21T15:01:47.834 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.834 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.834 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:01:47.834 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6819", 2024-09-21T15:01:47.834 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3894963553 2024-09-21T15:01:47.834 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:47.834 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:47.834 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.835 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-21T15:01:47.835 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:01:47.835 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.835 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:01:47.835 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6821", 2024-09-21T15:01:47.835 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3894963553 2024-09-21T15:01:47.835 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.835 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.835 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:01:47.835 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6824", 2024-09-21T15:01:47.835 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3894963553 2024-09-21T15:01:47.835 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:47.835 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:47.835 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.836 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-21T15:01:47.836 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:01:47.836 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.836 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:01:47.836 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6830", 2024-09-21T15:01:47.836 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3894963553 2024-09-21T15:01:47.836 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.836 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.836 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:01:47.836 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6831", 2024-09-21T15:01:47.836 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3894963553 2024-09-21T15:01:47.836 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:47.836 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:47.837 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.837 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-21T15:01:47.837 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:01:47.837 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.837 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:01:47.837 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6826", 2024-09-21T15:01:47.837 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3894963553 2024-09-21T15:01:47.837 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.837 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.837 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:01:47.837 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6829", 2024-09-21T15:01:47.837 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3894963553 2024-09-21T15:01:47.837 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:47.837 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:47.838 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.838 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.46:6819/3894963553", 2024-09-21T15:01:47.838 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.46:6824/3894963553", 2024-09-21T15:01:47.838 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.46:6831/3894963553", 2024-09-21T15:01:47.838 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.46:6829/3894963553", 2024-09-21T15:01:47.838 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-21T15:01:47.838 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-21T15:01:47.838 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-21T15:01:47.838 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:47.838 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:47.838 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-21T15:01:47.838 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_xinfo": [ 2024-09-21T15:01:47.838 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.838 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-09-21T15:01:47.839 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-21T15:01:47.839 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-21T15:01:47.839 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-21T15:01:47.839 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-21T15:01:47.839 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-21T15:01:47.839 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-21T14:57:57.458317+0000", 2024-09-21T15:01:47.839 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-21T15:01:47.839 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.839 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.839 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-09-21T15:01:47.839 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-21T15:01:47.839 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-21T15:01:47.839 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-21T15:01:47.840 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-21T15:01:47.840 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-21T15:01:47.840 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-21T14:57:57.907624+0000", 2024-09-21T15:01:47.840 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-21T15:01:47.840 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.840 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.840 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-09-21T15:01:47.840 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-21T15:01:47.840 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-21T15:01:47.840 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-21T15:01:47.840 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-21T15:01:47.840 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-21T15:01:47.840 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-21T14:57:57.278817+0000", 2024-09-21T15:01:47.840 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-21T15:01:47.840 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.841 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.841 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-09-21T15:01:47.841 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-21T15:01:47.841 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-21T15:01:47.841 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-21T15:01:47.841 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-21T15:01:47.841 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-21T15:01:47.841 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-21T14:57:57.500449+0000", 2024-09-21T15:01:47.841 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-21T15:01:47.841 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.841 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.841 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-09-21T15:01:47.841 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-21T15:01:47.841 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-21T15:01:47.842 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-21T15:01:47.842 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-21T15:01:47.842 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 65536, 2024-09-21T15:01:47.842 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-21T14:57:57.504325+0000", 2024-09-21T15:01:47.842 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 202 2024-09-21T15:01:47.842 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.842 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.842 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-09-21T15:01:47.842 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-21T15:01:47.842 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-21T15:01:47.842 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-21T15:01:47.842 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-21T15:01:47.842 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-21T15:01:47.842 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-21T14:57:57.777525+0000", 2024-09-21T15:01:47.843 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-21T15:01:47.843 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.843 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.843 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-09-21T15:01:47.843 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-21T15:01:47.843 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-21T15:01:47.843 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-21T15:01:47.843 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-21T15:01:47.843 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-21T15:01:47.843 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-21T14:57:57.698961+0000", 2024-09-21T15:01:47.843 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-21T15:01:47.843 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.843 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.843 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-09-21T15:01:47.844 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-21T15:01:47.844 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-21T15:01:47.844 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-21T15:01:47.844 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-21T15:01:47.844 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-21T15:01:47.844 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-21T14:57:57.932625+0000", 2024-09-21T15:01:47.844 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-21T15:01:47.844 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:47.844 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-21T15:01:47.844 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap": [ 2024-09-21T15:01:47.844 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.844 INFO:teuthology.orchestra.run.smithi032.stdout: "pgid": "3.2", 2024-09-21T15:01:47.844 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2024-09-21T15:01:47.844 INFO:teuthology.orchestra.run.smithi032.stdout: 2, 2024-09-21T15:01:47.845 INFO:teuthology.orchestra.run.smithi032.stdout: 3 2024-09-21T15:01:47.845 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:47.845 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.845 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.845 INFO:teuthology.orchestra.run.smithi032.stdout: "pgid": "3.4", 2024-09-21T15:01:47.845 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2024-09-21T15:01:47.845 INFO:teuthology.orchestra.run.smithi032.stdout: 4, 2024-09-21T15:01:47.845 INFO:teuthology.orchestra.run.smithi032.stdout: 2 2024-09-21T15:01:47.845 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:47.845 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.845 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.845 INFO:teuthology.orchestra.run.smithi032.stdout: "pgid": "3.8", 2024-09-21T15:01:47.845 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2024-09-21T15:01:47.845 INFO:teuthology.orchestra.run.smithi032.stdout: 6, 2024-09-21T15:01:47.845 INFO:teuthology.orchestra.run.smithi032.stdout: 3 2024-09-21T15:01:47.845 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:47.846 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.846 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.846 INFO:teuthology.orchestra.run.smithi032.stdout: "pgid": "3.a", 2024-09-21T15:01:47.846 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2024-09-21T15:01:47.846 INFO:teuthology.orchestra.run.smithi032.stdout: 1, 2024-09-21T15:01:47.846 INFO:teuthology.orchestra.run.smithi032.stdout: 0 2024-09-21T15:01:47.846 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:47.846 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:47.846 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-21T15:01:47.846 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_items": [], 2024-09-21T15:01:47.846 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_primaries": [], 2024-09-21T15:01:47.846 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_temp": [], 2024-09-21T15:01:47.846 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_temp": [], 2024-09-21T15:01:47.847 INFO:teuthology.orchestra.run.smithi032.stdout: "blocklist": {}, 2024-09-21T15:01:47.847 INFO:teuthology.orchestra.run.smithi032.stdout: "range_blocklist": {}, 2024-09-21T15:01:47.847 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profiles": { 2024-09-21T15:01:47.847 INFO:teuthology.orchestra.run.smithi032.stdout: "default": { 2024-09-21T15:01:47.847 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2024-09-21T15:01:47.847 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2024-09-21T15:01:47.847 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "1", 2024-09-21T15:01:47.847 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2024-09-21T15:01:47.847 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van" 2024-09-21T15:01:47.847 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:47.847 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.847 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps_queue": [ 2024-09-21T15:01:47.847 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.847 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 3, 2024-09-21T15:01:47.848 INFO:teuthology.orchestra.run.smithi032.stdout: "snaps": [ 2024-09-21T15:01:47.848 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.848 INFO:teuthology.orchestra.run.smithi032.stdout: "begin": 37, 2024-09-21T15:01:47.848 INFO:teuthology.orchestra.run.smithi032.stdout: "length": 1 2024-09-21T15:01:47.848 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.848 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.848 INFO:teuthology.orchestra.run.smithi032.stdout: "begin": 40, 2024-09-21T15:01:47.848 INFO:teuthology.orchestra.run.smithi032.stdout: "length": 1 2024-09-21T15:01:47.848 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.848 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.848 INFO:teuthology.orchestra.run.smithi032.stdout: "begin": 43, 2024-09-21T15:01:47.848 INFO:teuthology.orchestra.run.smithi032.stdout: "length": 1 2024-09-21T15:01:47.848 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.848 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.848 INFO:teuthology.orchestra.run.smithi032.stdout: "begin": 46, 2024-09-21T15:01:47.849 INFO:teuthology.orchestra.run.smithi032.stdout: "length": 1 2024-09-21T15:01:47.849 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.849 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.849 INFO:teuthology.orchestra.run.smithi032.stdout: "begin": 49, 2024-09-21T15:01:47.849 INFO:teuthology.orchestra.run.smithi032.stdout: "length": 1 2024-09-21T15:01:47.849 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.849 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.849 INFO:teuthology.orchestra.run.smithi032.stdout: "begin": 52, 2024-09-21T15:01:47.849 INFO:teuthology.orchestra.run.smithi032.stdout: "length": 1 2024-09-21T15:01:47.849 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:47.849 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.849 INFO:teuthology.orchestra.run.smithi032.stdout: "begin": 55, 2024-09-21T15:01:47.849 INFO:teuthology.orchestra.run.smithi032.stdout: "length": 1 2024-09-21T15:01:47.849 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:47.849 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:47.849 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:47.850 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-21T15:01:47.850 INFO:teuthology.orchestra.run.smithi032.stdout: "new_removed_snaps": [ 2024-09-21T15:01:47.850 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.850 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 3, 2024-09-21T15:01:47.850 INFO:teuthology.orchestra.run.smithi032.stdout: "snaps": [ 2024-09-21T15:01:47.850 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:47.850 INFO:teuthology.orchestra.run.smithi032.stdout: "begin": 37, 2024-09-21T15:01:47.850 INFO:teuthology.orchestra.run.smithi032.stdout: "length": 1 2024-09-21T15:01:47.850 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:47.850 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:47.850 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:47.850 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-21T15:01:47.850 INFO:teuthology.orchestra.run.smithi032.stdout: "new_purged_snaps": [], 2024-09-21T15:01:47.850 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_node_flags": {}, 2024-09-21T15:01:47.851 INFO:teuthology.orchestra.run.smithi032.stdout: "device_class_flags": {}, 2024-09-21T15:01:47.851 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode": { 2024-09-21T15:01:47.851 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_enabled": false, 2024-09-21T15:01:47.851 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_bucket_count": 0, 2024-09-21T15:01:47.851 INFO:teuthology.orchestra.run.smithi032.stdout: "degraded_stretch_mode": 0, 2024-09-21T15:01:47.851 INFO:teuthology.orchestra.run.smithi032.stdout: "recovering_stretch_mode": 0, 2024-09-21T15:01:47.851 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_bucket": 0 2024-09-21T15:01:47.851 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:47.851 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:47.853 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:47.853 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:47.853 INFO:tasks.thrashosds.thrasher:j is {'epoch': 215, 'fsid': 'caf51e55-9234-4a26-bcd2-d60811962560', 'created': '2024-09-21T14:57:50.338822+0000', 'modified': '2024-09-21T15:01:47.266268+0000', 'last_up_change': '2024-09-21T15:01:34.172764+0000', 'last_in_change': '2024-09-21T15:01:21.319966+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T14:58:01.842810+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.9585952758789062, 'average_primary_affinity': 0.9599999785423279, 'average_primary_affinity_weighted': 0.9599999785423279}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-21T14:58:08.284925+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 22, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 5, 'score_stable': 5, 'optimal_score': 0.75, 'raw_score_acting': 3.75, 'raw_score_stable': 3.75, 'primary_affinity_weighted': 0.9585952758789062, 'average_primary_affinity': 0.9599999785423279, 'average_primary_affinity_weighted': 0.9599999785423279}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-21T14:58:17.510709+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '215', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 140, 'snap_epoch': 215, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.4900000095367432, 'score_stable': 1.4900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.309999942779541, 'raw_score_stable': 1.309999942779541, 'primary_affinity_weighted': 0.9585952758789062, 'average_primary_affinity': 0.9599999785423279, 'average_primary_affinity_weighted': 0.9599999785423279}}], 'osds': [{'osd': 0, 'uuid': 'a4a73316-3a94-42e9-9a51-deb46a66bb06', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 192, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6808', 'nonce': 4016672243}, {'type': 'v1', 'addr': '172.21.15.32:6809', 'nonce': 4016672243}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6810', 'nonce': 4016672243}, {'type': 'v1', 'addr': '172.21.15.32:6811', 'nonce': 4016672243}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6814', 'nonce': 4016672243}, {'type': 'v1', 'addr': '172.21.15.32:6815', 'nonce': 4016672243}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6812', 'nonce': 4016672243}, {'type': 'v1', 'addr': '172.21.15.32:6813', 'nonce': 4016672243}]}, 'public_addr': '172.21.15.32:6809/4016672243', 'cluster_addr': '172.21.15.32:6811/4016672243', 'heartbeat_back_addr': '172.21.15.32:6815/4016672243', 'heartbeat_front_addr': '172.21.15.32:6813/4016672243', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '006757a0-94d1-4a9b-aef8-73634cc5828e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 192, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6824', 'nonce': 641090577}, {'type': 'v1', 'addr': '172.21.15.32:6825', 'nonce': 641090577}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6826', 'nonce': 641090577}, {'type': 'v1', 'addr': '172.21.15.32:6827', 'nonce': 641090577}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6830', 'nonce': 641090577}, {'type': 'v1', 'addr': '172.21.15.32:6831', 'nonce': 641090577}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6828', 'nonce': 641090577}, {'type': 'v1', 'addr': '172.21.15.32:6829', 'nonce': 641090577}]}, 'public_addr': '172.21.15.32:6825/641090577', 'cluster_addr': '172.21.15.32:6827/641090577', 'heartbeat_back_addr': '172.21.15.32:6831/641090577', 'heartbeat_front_addr': '172.21.15.32:6829/641090577', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '0b499a23-820e-4e6c-a558-adbf88b7014f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 198, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6800', 'nonce': 2609154414}, {'type': 'v1', 'addr': '172.21.15.32:6801', 'nonce': 2609154414}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6802', 'nonce': 2609154414}, {'type': 'v1', 'addr': '172.21.15.32:6803', 'nonce': 2609154414}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6806', 'nonce': 2609154414}, {'type': 'v1', 'addr': '172.21.15.32:6807', 'nonce': 2609154414}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6804', 'nonce': 2609154414}, {'type': 'v1', 'addr': '172.21.15.32:6805', 'nonce': 2609154414}]}, 'public_addr': '172.21.15.32:6801/2609154414', 'cluster_addr': '172.21.15.32:6803/2609154414', 'heartbeat_back_addr': '172.21.15.32:6807/2609154414', 'heartbeat_front_addr': '172.21.15.32:6805/2609154414', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '5a57b3dd-4a07-45b7-b159-3cac4c7eb262', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.833831787109375, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 192, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6816', 'nonce': 1462563376}, {'type': 'v1', 'addr': '172.21.15.32:6817', 'nonce': 1462563376}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6818', 'nonce': 1462563376}, {'type': 'v1', 'addr': '172.21.15.32:6819', 'nonce': 1462563376}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6822', 'nonce': 1462563376}, {'type': 'v1', 'addr': '172.21.15.32:6823', 'nonce': 1462563376}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6820', 'nonce': 1462563376}, {'type': 'v1', 'addr': '172.21.15.32:6821', 'nonce': 1462563376}]}, 'public_addr': '172.21.15.32:6817/1462563376', 'cluster_addr': '172.21.15.32:6819/1462563376', 'heartbeat_back_addr': '172.21.15.32:6823/1462563376', 'heartbeat_front_addr': '172.21.15.32:6821/1462563376', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '39a00807-7517-474e-8842-623dd7093cc5', 'up': 0, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 147, 'down_at': 202, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6801', 'nonce': 144412042}, {'type': 'v1', 'addr': '172.21.15.46:6803', 'nonce': 144412042}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6805', 'nonce': 144412042}, {'type': 'v1', 'addr': '172.21.15.46:6807', 'nonce': 144412042}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6812', 'nonce': 144412042}, {'type': 'v1', 'addr': '172.21.15.46:6814', 'nonce': 144412042}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6809', 'nonce': 144412042}, {'type': 'v1', 'addr': '172.21.15.46:6811', 'nonce': 144412042}]}, 'public_addr': '172.21.15.46:6803/144412042', 'cluster_addr': '172.21.15.46:6807/144412042', 'heartbeat_back_addr': '172.21.15.46:6814/144412042', 'heartbeat_front_addr': '172.21.15.46:6811/144412042', 'state': ['exists']}, {'osd': 5, 'uuid': 'edf9f045-ddeb-4504-a0fc-1a0e4e853550', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 177, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6800', 'nonce': 849101442}, {'type': 'v1', 'addr': '172.21.15.46:6802', 'nonce': 849101442}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6804', 'nonce': 849101442}, {'type': 'v1', 'addr': '172.21.15.46:6806', 'nonce': 849101442}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6813', 'nonce': 849101442}, {'type': 'v1', 'addr': '172.21.15.46:6815', 'nonce': 849101442}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6808', 'nonce': 849101442}, {'type': 'v1', 'addr': '172.21.15.46:6810', 'nonce': 849101442}]}, 'public_addr': '172.21.15.46:6802/849101442', 'cluster_addr': '172.21.15.46:6806/849101442', 'heartbeat_back_addr': '172.21.15.46:6815/849101442', 'heartbeat_front_addr': '172.21.15.46:6810/849101442', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'c820ea6b-88fe-4421-83ec-17c466691fea', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.834930419921875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 199, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6817', 'nonce': 847189172}, {'type': 'v1', 'addr': '172.21.15.46:6818', 'nonce': 847189172}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6820', 'nonce': 847189172}, {'type': 'v1', 'addr': '172.21.15.46:6822', 'nonce': 847189172}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6827', 'nonce': 847189172}, {'type': 'v1', 'addr': '172.21.15.46:6828', 'nonce': 847189172}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6823', 'nonce': 847189172}, {'type': 'v1', 'addr': '172.21.15.46:6825', 'nonce': 847189172}]}, 'public_addr': '172.21.15.46:6818/847189172', 'cluster_addr': '172.21.15.46:6822/847189172', 'heartbeat_back_addr': '172.21.15.46:6828/847189172', 'heartbeat_front_addr': '172.21.15.46:6825/847189172', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '3a2f82a4-28a0-4fcb-9d6b-2285fa4d07ac', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 193, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6816', 'nonce': 3894963553}, {'type': 'v1', 'addr': '172.21.15.46:6819', 'nonce': 3894963553}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6821', 'nonce': 3894963553}, {'type': 'v1', 'addr': '172.21.15.46:6824', 'nonce': 3894963553}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6830', 'nonce': 3894963553}, {'type': 'v1', 'addr': '172.21.15.46:6831', 'nonce': 3894963553}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6826', 'nonce': 3894963553}, {'type': 'v1', 'addr': '172.21.15.46:6829', 'nonce': 3894963553}]}, 'public_addr': '172.21.15.46:6819/3894963553', 'cluster_addr': '172.21.15.46:6824/3894963553', 'heartbeat_back_addr': '172.21.15.46:6831/3894963553', 'heartbeat_front_addr': '172.21.15.46:6829/3894963553', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:57:57.458317+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:57:57.907624+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:57:57.278817+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:57:57.500449+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-21T14:57:57.504325+0000', 'dead_epoch': 202}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:57:57.777525+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:57:57.698961+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:57:57.932625+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.2', 'osds': [2, 3]}, {'pgid': '3.4', 'osds': [4, 2]}, {'pgid': '3.8', 'osds': [6, 3]}, {'pgid': '3.a', 'osds': [1, 0]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 37, 'length': 1}, {'begin': 40, 'length': 1}, {'begin': 43, 'length': 1}, {'begin': 46, 'length': 1}, {'begin': 49, 'length': 1}, {'begin': 52, 'length': 1}, {'begin': 55, 'length': 1}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 37, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-21T15:01:47.854 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T15:01:47.854 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:01:47.849+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:01:47.854 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T15:01:47.953 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:01:47.948+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:01:47.966 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:47.966 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:47.976 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T15:01:48.053 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:01:48.053+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:01:48.087 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:48.087 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:48.097 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T15:01:48.155 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:01:48.148+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:01:48.212 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-21T15:01:48.212 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:48.213 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:48.215 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-09-21T15:01:48.222 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T15:01:48.228 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":322,"stamp":"2024-09-21T15:01:47.833752+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":272757419,"num_objects":609,"num_object_clones":0,"num_object_copies":1218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":609,"num_whiteouts":0,"num_read":171314,"num_read_kb":1235892,"num_write":33417,"num_write_kb":943077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3029,"num_bytes_recovered":210630977,"num_keys_recovered":305,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21569,"ondisk_log_size":21569,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":415344,"kb_used_data":40736,"kb_used_omap":10,"kb_used_meta":374389,"kb_avail":660187536,"statfs":{"total":676457349120,"available":676032036864,"internally_reserved":0,"allocated":41713664,"data_stored":226780545,"data_compressed":3372067,"data_compressed_allocated":18436096,"data_compressed_original":214874764,"omap_allocated":11150,"internal_metadata":383374450},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-107470494,"num_objects":-190,"num_object_clones":0,"num_object_copies":-380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-190,"num_whiteouts":0,"num_read":1064,"num_read_kb":915,"num_write":359,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000987"},"pg_stats":[{"pgid":"3.1f","version":"213'564","reported_seq":3780,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-21T15:01:45.306314+0000","last_change":"2024-09-21T15:01:45.298990+0000","last_active":"2024-09-21T15:01:45.306314+0000","last_peered":"2024-09-21T15:01:45.306314+0000","last_clean":"2024-09-21T15:01:45.306314+0000","last_became_active":"2024-09-21T14:58:19.410055+0000","last_became_peered":"2024-09-21T14:58:19.410055+0000","last_unstale":"2024-09-21T15:01:45.306314+0000","last_undegraded":"2024-09-21T15:01:45.306314+0000","last_fullsized":"2024-09-21T15:01:45.306314+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":564,"log_dups_size":0,"ondisk_log_size":564,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:49:13.944219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041919920000000003,"stat_sum":{"num_bytes":3225319,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":3580,"num_read_kb":43067,"num_write":946,"num_write_kb":36430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2b","length":"1"},{"start":"2e","length":"1"},{"start":"31","length":"1"},{"start":"34","length":"1"},{"start":"37","length":"1"}]},{"pgid":"3.1e","version":"213'477","reported_seq":3384,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-21T15:01:45.304055+0000","last_change":"2024-09-21T15:01:45.299079+0000","last_active":"2024-09-21T15:01:45.304055+0000","last_peered":"2024-09-21T15:01:45.304055+0000","last_clean":"2024-09-21T15:01:45.304055+0000","last_became_active":"2024-09-21T15:01:06.168726+0000","last_became_peered":"2024-09-21T15:01:06.168726+0000","last_unstale":"2024-09-21T15:01:45.304055+0000","last_undegraded":"2024-09-21T15:01:45.304055+0000","last_fullsized":"2024-09-21T15:01:45.304055+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":477,"log_dups_size":0,"ondisk_log_size":477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:29:01.079772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041230379999999999,"stat_sum":{"num_bytes":9807682,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":3389,"num_read_kb":23383,"num_write":611,"num_write_kb":18440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":148,"num_bytes_recovered":11635695,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2b","length":"1"},{"start":"2e","length":"1"},{"start":"31","length":"1"},{"start":"34","length":"1"},{"start":"37","length":"1"}]},{"pgid":"3.1d","version":"213'325","reported_seq":4680,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-21T15:01:45.312158+0000","last_change":"2024-09-21T15:01:45.181735+0000","last_active":"2024-09-21T15:01:45.312158+0000","last_peered":"2024-09-21T15:01:45.312158+0000","last_clean":"2024-09-21T15:01:45.312158+0000","last_became_active":"2024-09-21T14:58:19.411387+0000","last_became_peered":"2024-09-21T14:58:19.411387+0000","last_unstale":"2024-09-21T15:01:45.312158+0000","last_undegraded":"2024-09-21T15:01:45.312158+0000","last_fullsized":"2024-09-21T15:01:45.312158+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":325,"log_dups_size":0,"ondisk_log_size":325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:37:57.597069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017711409999999999,"stat_sum":{"num_bytes":7777608,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":3878,"num_read_kb":28288,"num_write":519,"num_write_kb":26606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2b","length":"1"},{"start":"2e","length":"1"},{"start":"31","length":"1"},{"start":"34","length":"1"},{"start":"37","length":"1"}]},{"pgid":"3.1c","version":"213'520","reported_seq":4049,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-21T15:01:45.263176+0000","last_change":"2024-09-21T15:01:45.185856+0000","last_active":"2024-09-21T15:01:45.263176+0000","last_peered":"2024-09-21T15:01:45.263176+0000","last_clean":"2024-09-21T15:01:45.263176+0000","last_became_active":"2024-09-21T14:58:19.411306+0000","last_became_peered":"2024-09-21T14:58:19.411306+0000","last_unstale":"2024-09-21T15:01:45.263176+0000","last_undegraded":"2024-09-21T15:01:45.263176+0000","last_fullsized":"2024-09-21T15:01:45.263176+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":520,"log_dups_size":0,"ondisk_log_size":520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:20:27.079795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064745299999999998,"stat_sum":{"num_bytes":4963550,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":3810,"num_read_kb":24459,"num_write":803,"num_write_kb":25222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2b","length":"1"},{"start":"2e","length":"1"},{"start":"31","length":"1"},{"start":"34","length":"1"},{"start":"37","length":"1"}]},{"pgid":"3.1b","version":"213'504","reported_seq":4088,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-21T15:01:45.338213+0000","last_change":"2024-09-21T15:01:45.190307+0000","last_active":"2024-09-21T15:01:45.338213+0000","last_peered":"2024-09-21T15:01:45.338213+0000","last_clean":"2024-09-21T15:01:45.338213+0000","last_became_active":"2024-09-21T15:01:25.176460+0000","last_became_peered":"2024-09-21T15:01:25.176460+0000","last_unstale":"2024-09-21T15:01:45.338213+0000","last_undegraded":"2024-09-21T15:01:45.338213+0000","last_fullsized":"2024-09-21T15:01:45.338213+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":504,"log_dups_size":0,"ondisk_log_size":504,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:57.431201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001052486,"stat_sum":{"num_bytes":3457024,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":3499,"num_read_kb":32547,"num_write":679,"num_write_kb":22576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":248,"num_bytes_recovered":13616631,"num_keys_recovered":33,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2b","length":"1"},{"start":"2e","length":"1"},{"start":"31","length":"1"},{"start":"34","length":"1"},{"start":"37","length":"1"}]},{"pgid":"3.1a","version":"213'447","reported_seq":4573,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-21T15:01:45.352471+0000","last_change":"2024-09-21T15:01:45.187266+0000","last_active":"2024-09-21T15:01:45.352471+0000","last_peered":"2024-09-21T15:01:45.352471+0000","last_clean":"2024-09-21T15:01:45.352471+0000","last_became_active":"2024-09-21T14:58:19.409415+0000","last_became_peered":"2024-09-21T14:58:19.409415+0000","last_unstale":"2024-09-21T15:01:45.352471+0000","last_undegraded":"2024-09-21T15:01:45.352471+0000","last_fullsized":"2024-09-21T15:01:45.352471+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":447,"log_dups_size":0,"ondisk_log_size":447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:28:46.336301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016250629999999999,"stat_sum":{"num_bytes":5503682,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":4137,"num_read_kb":28121,"num_write":669,"num_write_kb":23870,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2b","length":"1"},{"start":"2e","length":"1"},{"start":"31","length":"1"},{"start":"34","length":"1"},{"start":"37","length":"1"}]},{"pgid":"3.19","version":"213'757","reported_seq":8421,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-21T15:01:45.320205+0000","last_change":"2024-09-21T15:01:45.186116+0000","last_active":"2024-09-21T15:01:45.320205+0000","last_peered":"2024-09-21T15:01:45.320205+0000","last_clean":"2024-09-21T15:01:45.320205+0000","last_became_active":"2024-09-21T15:01:25.179729+0000","last_became_peered":"2024-09-21T15:01:25.179729+0000","last_unstale":"2024-09-21T15:01:45.320205+0000","last_undegraded":"2024-09-21T15:01:45.320205+0000","last_fullsized":"2024-09-21T15:01:45.320205+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":757,"log_dups_size":0,"ondisk_log_size":757,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:15:24.357933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092075299999999996,"stat_sum":{"num_bytes":18981741,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":9033,"num_read_kb":67754,"num_write":1194,"num_write_kb":29909,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":261,"num_bytes_recovered":23509073,"num_keys_recovered":66,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2b","length":"1"},{"start":"2e","length":"1"},{"start":"31","length":"1"},{"start":"34","length":"1"},{"start":"37","length":"1"}]},{"pgid":"3.18","version":"213'611","reported_seq":8474,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-21T15:01:45.308589+0000","last_change":"2024-09-21T15:01:45.299569+0000","last_active":"2024-09-21T15:01:45.308589+0000","last_peered":"2024-09-21T15:01:45.308589+0000","last_clean":"2024-09-21T15:01:45.308589+0000","last_became_active":"2024-09-21T14:58:19.409371+0000","last_became_peered":"2024-09-21T14:58:19.409371+0000","last_unstale":"2024-09-21T15:01:45.308589+0000","last_undegraded":"2024-09-21T15:01:45.308589+0000","last_fullsized":"2024-09-21T15:01:45.308589+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":611,"log_dups_size":0,"ondisk_log_size":611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:35:27.143445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040557789999999998,"stat_sum":{"num_bytes":7549650,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":8857,"num_read_kb":44939,"num_write":929,"num_write_kb":30599,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2b","length":"1"},{"start":"2e","length":"1"},{"start":"31","length":"1"},{"start":"34","length":"1"},{"start":"37","length":"1"}]},{"pgid":"3.17","version":"212'401","reported_seq":3898,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-21T15:01:45.299628+0000","last_change":"2024-09-21T15:01:45.299839+0000","last_active":"2024-09-21T15:01:45.299628+0000","last_peered":"2024-09-21T15:01:45.299628+0000","last_clean":"2024-09-21T15:01:45.299628+0000","last_became_active":"2024-09-21T14:58:19.410306+0000","last_became_peered":"2024-09-21T14:58:19.410306+0000","last_unstale":"2024-09-21T15:01:45.299628+0000","last_undegraded":"2024-09-21T15:01:45.299628+0000","last_fullsized":"2024-09-21T15:01:45.299628+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":401,"log_dups_size":0,"ondisk_log_size":401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:33:50.089165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.004201977,"stat_sum":{"num_bytes":3805548,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":3380,"num_read_kb":28207,"num_write":757,"num_write_kb":34633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2b","length":"1"},{"start":"2e","length":"1"},{"start":"31","length":"1"},{"start":"34","length":"1"},{"start":"37","length":"1"}]},{"pgid":"3.16","version":"213'1002","reported_seq":5954,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-21T15:01:45.182893+0000","last_change":"2024-09-21T15:01:45.182893+0000","last_active":"2024-09-21T15:01:45.182893+0000","last_peered":"2024-09-21T15:01:45.182893+0000","last_clean":"2024-09-21T15:01:45.182893+0000","last_became_active":"2024-09-21T15:01:06.166756+0000","last_became_peered":"2024-09-21T15:01:06.166756+0000","last_unstale":"2024-09-21T15:01:45.182893+0000","last_undegraded":"2024-09-21T15:01:45.182893+0000","last_fullsized":"2024-09-21T15:01:45.182893+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":1002,"log_dups_size":0,"ondisk_log_size":1002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:19:04.988784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068605700000000001,"stat_sum":{"num_bytes":5201893,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":7869,"num_read_kb":43824,"num_write":1573,"num_write_kb":28363,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":271,"num_bytes_recovered":16821266,"num_keys_recovered":55,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2b","length":"1"},{"start":"2e","length":"1"},{"start":"31","length":"1"},{"start":"34","length":"1"},{"start":"37","length":"1"}]},{"pgid":"3.15","version":"213'791","reported_seq":3848,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-21T15:01:45.310040+0000","last_change":"2024-09-21T15:01:45.190609+0000","last_active":"2024-09-21T15:01:45.310040+0000","last_peered":"2024-09-21T15:01:45.310040+0000","last_clean":"2024-09-21T15:01:45.310040+0000","last_became_active":"2024-09-21T14:58:19.410687+0000","last_became_peered":"2024-09-21T14:58:19.410687+0000","last_unstale":"2024-09-21T15:01:45.310040+0000","last_undegraded":"2024-09-21T15:01:45.310040+0000","last_fullsized":"2024-09-21T15:01:45.310040+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":791,"log_dups_size":0,"ondisk_log_size":791,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:21:42.226982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010797789999999999,"stat_sum":{"num_bytes":8356180,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":4724,"num_read_kb":49745,"num_write":1314,"num_write_kb":33334,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2b","length":"1"},{"start":"2e","length":"1"},{"start":"31","length":"1"},{"start":"34","length":"1"},{"start":"37","length":"1"}]},{"pgid":"3.14","version":"209'503","reported_seq":4069,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-21T15:01:45.180914+0000","last_change":"2024-09-21T15:01:45.181383+0000","last_active":"2024-09-21T15:01:45.180914+0000","last_peered":"2024-09-21T15:01:45.180914+0000","last_clean":"2024-09-21T15:01:45.180914+0000","last_became_active":"2024-09-21T14:58:19.407310+0000","last_became_peered":"2024-09-21T14:58:19.407310+0000","last_unstale":"2024-09-21T15:01:45.180914+0000","last_undegraded":"2024-09-21T15:01:45.180914+0000","last_fullsized":"2024-09-21T15:01:45.180914+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":503,"log_dups_size":0,"ondisk_log_size":503,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:47:46.079115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018941100000000001,"stat_sum":{"num_bytes":17038365,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":3722,"num_read_kb":43297,"num_write":687,"num_write_kb":33466,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2b","length":"1"},{"start":"2e","length":"1"},{"start":"31","length":"1"},{"start":"34","length":"1"},{"start":"37","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":207,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-21T15:01:44.354549+0000","last_change":"2024-09-21T14:58:11.348177+0000","last_active":"2024-09-21T15:01:44.354549+0000","last_peered":"2024-09-21T15:01:44.354549+0000","last_clean":"2024-09-21T15:01:44.354549+0000","last_became_active":"2024-09-21T14:58:09.328417+0000","last_became_peered":"2024-09-21T14:58:09.328417+0000","last_unstale":"2024-09-21T15:01:44.354549+0000","last_undegraded":"2024-09-21T15:01:44.354549+0000","last_fullsized":"2024-09-21T15:01:44.354549+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:52:22.671185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018917490000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"214'546","reported_seq":6345,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-21T15:01:46.395705+0000","last_change":"2024-09-21T15:01:46.307383+0000","last_active":"2024-09-21T15:01:46.395705+0000","last_peered":"2024-09-21T15:01:46.395705+0000","last_clean":"2024-09-21T15:01:46.395705+0000","last_became_active":"2024-09-21T15:01:25.173568+0000","last_became_peered":"2024-09-21T15:01:25.173568+0000","last_unstale":"2024-09-21T15:01:46.395705+0000","last_undegraded":"2024-09-21T15:01:46.395705+0000","last_fullsized":"2024-09-21T15:01:46.395705+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":546,"log_dups_size":0,"ondisk_log_size":546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:53:08.599143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089645100000000004,"stat_sum":{"num_bytes":9707382,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":6109,"num_read_kb":52874,"num_write":648,"num_write_kb":27950,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":202,"num_bytes_recovered":11818795,"num_keys_recovered":18,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"28","length":"1"},{"start":"2b","length":"1"},{"start":"2e","length":"1"},{"start":"31","length":"1"},{"start":"34","length":"1"},{"start":"37","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":54,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-21T15:01:44.257823+0000","last_change":"2024-09-21T15:01:22.344040+0000","last_active":"2024-09-21T15:01:44.257823+0000","last_peered":"2024-09-21T15:01:44.257823+0000","last_clean":"2024-09-21T15:01:44.257823+0000","last_became_active":"2024-09-21T15:01:22.343688+0000","last_became_peered":"2024-09-21T15:01:22.343688+0000","last_unstale":"2024-09-21T15:01:44.257823+0000","last_undegraded":"2024-09-21T15:01:44.257823+0000","last_fullsized":"2024-09-21T15:01:44.257823+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:11:18.042935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"213'947","reported_seq":4143,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-21T15:01:45.305614+0000","last_change":"2024-09-21T15:01:45.255023+0000","last_active":"2024-09-21T15:01:45.305614+0000","last_peered":"2024-09-21T15:01:45.305614+0000","last_clean":"2024-09-21T15:01:45.305614+0000","last_became_active":"2024-09-21T15:01:25.173798+0000","last_became_peered":"2024-09-21T15:01:25.173798+0000","last_unstale":"2024-09-21T15:01:45.305614+0000","last_undegraded":"2024-09-21T15:01:45.305614+0000","last_fullsized":"2024-09-21T15:01:45.305614+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":947,"log_dups_size":0,"ondisk_log_size":947,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:03:31.905691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019521409999999999,"stat_sum":{"num_bytes":11021775,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":5418,"num_read_kb":31174,"num_write":1365,"num_write_kb":25276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":260,"num_bytes_recovered":12566413,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2b","length":"1"},{"start":"2e","length":"1"},{"start":"31","length":"1"},{"start":"34","length":"1"},{"start":"37","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":231,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-21T15:01:44.355798+0000","last_change":"2024-09-21T15:01:22.346021+0000","last_active":"2024-09-21T15:01:44.355798+0000","last_peered":"2024-09-21T15:01:44.355798+0000","last_clean":"2024-09-21T15:01:44.355798+0000","last_became_active":"2024-09-21T15:01:22.345643+0000","last_became_peered":"2024-09-21T15:01:22.345643+0000","last_unstale":"2024-09-21T15:01:44.355798+0000","last_undegraded":"2024-09-21T15:01:44.355798+0000","last_fullsized":"2024-09-21T15:01:44.355798+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:55:55.681542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019338770000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"213'646","reported_seq":5453,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-21T15:01:45.369374+0000","last_change":"2024-09-21T15:01:45.254643+0000","last_active":"2024-09-21T15:01:45.369374+0000","last_peered":"2024-09-21T15:01:45.369374+0000","last_clean":"2024-09-21T15:01:45.369374+0000","last_became_active":"2024-09-21T15:01:25.172697+0000","last_became_peered":"2024-09-21T15:01:25.172697+0000","last_unstale":"2024-09-21T15:01:45.369374+0000","last_undegraded":"2024-09-21T15:01:45.369374+0000","last_fullsized":"2024-09-21T15:01:45.369374+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":646,"log_dups_size":0,"ondisk_log_size":646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:34:57.331662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001947877,"stat_sum":{"num_bytes":7960709,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":5151,"num_read_kb":38438,"num_write":1102,"num_write_kb":41302,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":203,"num_bytes_recovered":25796920,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2b","length":"1"},{"start":"2e","length":"1"},{"start":"31","length":"1"},{"start":"34","length":"1"},{"start":"37","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":208,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-21T15:01:44.401206+0000","last_change":"2024-09-21T14:58:11.406176+0000","last_active":"2024-09-21T15:01:44.401206+0000","last_peered":"2024-09-21T15:01:44.401206+0000","last_clean":"2024-09-21T15:01:44.401206+0000","last_became_active":"2024-09-21T14:58:09.777465+0000","last_became_peered":"2024-09-21T14:58:09.777465+0000","last_unstale":"2024-09-21T15:01:44.401206+0000","last_undegraded":"2024-09-21T15:01:44.401206+0000","last_fullsized":"2024-09-21T15:01:44.401206+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:55:04.667816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095704000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"213'711","reported_seq":2739,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-21T15:01:45.306004+0000","last_change":"2024-09-21T15:01:45.179901+0000","last_active":"2024-09-21T15:01:45.306004+0000","last_peered":"2024-09-21T15:01:45.306004+0000","last_clean":"2024-09-21T15:01:45.306004+0000","last_became_active":"2024-09-21T14:58:19.408971+0000","last_became_peered":"2024-09-21T14:58:19.408971+0000","last_unstale":"2024-09-21T15:01:45.306004+0000","last_undegraded":"2024-09-21T15:01:45.306004+0000","last_fullsized":"2024-09-21T15:01:45.306004+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":711,"log_dups_size":0,"ondisk_log_size":711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:29:52.642121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012931869999999999,"stat_sum":{"num_bytes":6716071,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":3596,"num_read_kb":20653,"num_write":1209,"num_write_kb":20679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2b","length":"1"},{"start":"2e","length":"1"},{"start":"31","length":"1"},{"start":"34","length":"1"},{"start":"37","length":"1"}]},{"pgid":"3.3","version":"213'937","reported_seq":3471,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-21T15:01:45.342247+0000","last_change":"2024-09-21T15:01:45.191068+0000","last_active":"2024-09-21T15:01:45.342247+0000","last_peered":"2024-09-21T15:01:45.342247+0000","last_clean":"2024-09-21T15:01:45.342247+0000","last_became_active":"2024-09-21T14:58:19.417453+0000","last_became_peered":"2024-09-21T14:58:19.417453+0000","last_unstale":"2024-09-21T15:01:45.342247+0000","last_undegraded":"2024-09-21T15:01:45.342247+0000","last_fullsized":"2024-09-21T15:01:45.342247+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":937,"log_dups_size":0,"ondisk_log_size":937,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:19:27.087272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014493729999999999,"stat_sum":{"num_bytes":8229567,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":4767,"num_read_kb":25206,"num_write":1494,"num_write_kb":21696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2b","length":"1"},{"start":"2e","length":"1"},{"start":"31","length":"1"},{"start":"34","length":"1"},{"start":"37","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":209,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-21T15:01:44.354359+0000","last_change":"2024-09-21T14:58:11.347504+0000","last_active":"2024-09-21T15:01:44.354359+0000","last_peered":"2024-09-21T15:01:44.354359+0000","last_clean":"2024-09-21T15:01:44.354359+0000","last_became_active":"2024-09-21T14:58:09.775981+0000","last_became_peered":"2024-09-21T14:58:09.775981+0000","last_unstale":"2024-09-21T15:01:44.354359+0000","last_undegraded":"2024-09-21T15:01:44.354359+0000","last_fullsized":"2024-09-21T15:01:44.354359+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:47:16.286652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012568880000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"213'1053","reported_seq":3803,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-21T15:01:45.299798+0000","last_change":"2024-09-21T15:01:45.299798+0000","last_active":"2024-09-21T15:01:45.299798+0000","last_peered":"2024-09-21T15:01:45.299798+0000","last_clean":"2024-09-21T15:01:45.299798+0000","last_became_active":"2024-09-21T14:58:19.405811+0000","last_became_peered":"2024-09-21T14:58:19.405811+0000","last_unstale":"2024-09-21T15:01:45.299798+0000","last_undegraded":"2024-09-21T15:01:45.299798+0000","last_fullsized":"2024-09-21T15:01:45.299798+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":1053,"log_dups_size":0,"ondisk_log_size":1053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:36:25.551607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0042387789999999998,"stat_sum":{"num_bytes":6039659,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":6162,"num_read_kb":28643,"num_write":1643,"num_write_kb":26968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2b","length":"1"},{"start":"2e","length":"1"},{"start":"31","length":"1"},{"start":"34","length":"1"},{"start":"37","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":209,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-21T15:01:45.174851+0000","last_change":"2024-09-21T14:58:11.342210+0000","last_active":"2024-09-21T15:01:45.174851+0000","last_peered":"2024-09-21T15:01:45.174851+0000","last_clean":"2024-09-21T15:01:45.174851+0000","last_became_active":"2024-09-21T14:58:09.330624+0000","last_became_peered":"2024-09-21T14:58:09.330624+0000","last_unstale":"2024-09-21T15:01:45.174851+0000","last_undegraded":"2024-09-21T15:01:45.174851+0000","last_fullsized":"2024-09-21T15:01:45.174851+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:29:43.229538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001051151,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"212'322","reported_seq":2199,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-21T15:01:45.254025+0000","last_change":"2024-09-21T15:01:45.254413+0000","last_active":"2024-09-21T15:01:45.254025+0000","last_peered":"2024-09-21T15:01:45.254025+0000","last_clean":"2024-09-21T15:01:45.254025+0000","last_became_active":"2024-09-21T15:01:05.164862+0000","last_became_peered":"2024-09-21T15:01:05.164862+0000","last_unstale":"2024-09-21T15:01:45.254025+0000","last_undegraded":"2024-09-21T15:01:45.254025+0000","last_fullsized":"2024-09-21T15:01:45.254025+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":322,"log_dups_size":0,"ondisk_log_size":322,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:57:46.675335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001938061,"stat_sum":{"num_bytes":11688835,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":2369,"num_read_kb":40192,"num_write":512,"num_write_kb":39390,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":254,"num_bytes_recovered":20571253,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2b","length":"1"},{"start":"2e","length":"1"},{"start":"31","length":"1"},{"start":"34","length":"1"},{"start":"37","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":207,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-21T15:01:44.354589+0000","last_change":"2024-09-21T14:58:11.346766+0000","last_active":"2024-09-21T15:01:44.354589+0000","last_peered":"2024-09-21T15:01:44.354589+0000","last_clean":"2024-09-21T15:01:44.354589+0000","last_became_active":"2024-09-21T14:58:09.329748+0000","last_became_peered":"2024-09-21T14:58:09.329748+0000","last_unstale":"2024-09-21T15:01:44.354589+0000","last_undegraded":"2024-09-21T15:01:44.354589+0000","last_fullsized":"2024-09-21T15:01:44.354589+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:31:20.267675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052698000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"213'1254","reported_seq":8261,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-21T15:01:45.178836+0000","last_change":"2024-09-21T15:01:45.178836+0000","last_active":"2024-09-21T15:01:45.178836+0000","last_peered":"2024-09-21T15:01:45.178836+0000","last_clean":"2024-09-21T15:01:45.178836+0000","last_became_active":"2024-09-21T14:58:41.401003+0000","last_became_peered":"2024-09-21T14:58:41.401003+0000","last_unstale":"2024-09-21T15:01:45.178836+0000","last_undegraded":"2024-09-21T15:01:45.178836+0000","last_fullsized":"2024-09-21T15:01:45.178836+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":1254,"log_dups_size":0,"ondisk_log_size":1254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:05:37.644835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010952780000000001,"stat_sum":{"num_bytes":12763963,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":11705,"num_read_kb":50815,"num_write":1935,"num_write_kb":24913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1403348,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2b","length":"1"},{"start":"2e","length":"1"},{"start":"31","length":"1"},{"start":"34","length":"1"},{"start":"37","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":208,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-21T15:01:44.354674+0000","last_change":"2024-09-21T14:58:11.347700+0000","last_active":"2024-09-21T15:01:44.354674+0000","last_peered":"2024-09-21T15:01:44.354674+0000","last_clean":"2024-09-21T15:01:44.354674+0000","last_became_active":"2024-09-21T14:58:09.438842+0000","last_became_peered":"2024-09-21T14:58:09.438842+0000","last_unstale":"2024-09-21T15:01:44.354674+0000","last_undegraded":"2024-09-21T15:01:44.354674+0000","last_fullsized":"2024-09-21T15:01:44.354674+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:41:34.626249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001426194,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":356,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-21T15:01:44.242825+0000","last_change":"2024-09-21T14:58:03.616055+0000","last_active":"2024-09-21T15:01:44.242825+0000","last_peered":"2024-09-21T15:01:44.242825+0000","last_clean":"2024-09-21T15:01:44.242825+0000","last_became_active":"2024-09-21T14:58:03.613214+0000","last_became_peered":"2024-09-21T14:58:03.613214+0000","last_unstale":"2024-09-21T15:01:44.242825+0000","last_undegraded":"2024-09-21T15:01:44.242825+0000","last_fullsized":"2024-09-21T15:01:44.242825+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:02.180162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:02.180162+0000","last_clean_scrub_stamp":"2024-09-21T14:58:02.180162+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:29:05.860361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"213'666","reported_seq":4581,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-21T15:01:45.253770+0000","last_change":"2024-09-21T15:01:45.253770+0000","last_active":"2024-09-21T15:01:45.253770+0000","last_peered":"2024-09-21T15:01:45.253770+0000","last_clean":"2024-09-21T15:01:45.253770+0000","last_became_active":"2024-09-21T15:01:32.177913+0000","last_became_peered":"2024-09-21T15:01:32.177913+0000","last_unstale":"2024-09-21T15:01:45.253770+0000","last_undegraded":"2024-09-21T15:01:45.253770+0000","last_fullsized":"2024-09-21T15:01:45.253770+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":666,"log_dups_size":0,"ondisk_log_size":666,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:47:09.376828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013215309999999999,"stat_sum":{"num_bytes":11723388,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":4404,"num_read_kb":44537,"num_write":1145,"num_write_kb":30502,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":413,"num_bytes_recovered":20409497,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2b","length":"1"},{"start":"2e","length":"1"},{"start":"31","length":"1"},{"start":"34","length":"1"},{"start":"37","length":"1"}]},{"pgid":"3.9","version":"213'960","reported_seq":3711,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-21T15:01:45.180663+0000","last_change":"2024-09-21T15:01:45.180663+0000","last_active":"2024-09-21T15:01:45.180663+0000","last_peered":"2024-09-21T15:01:45.180663+0000","last_clean":"2024-09-21T15:01:45.180663+0000","last_became_active":"2024-09-21T15:01:25.176267+0000","last_became_peered":"2024-09-21T15:01:25.176267+0000","last_unstale":"2024-09-21T15:01:45.180663+0000","last_undegraded":"2024-09-21T15:01:45.180663+0000","last_fullsized":"2024-09-21T15:01:45.180663+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":960,"log_dups_size":0,"ondisk_log_size":960,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:21:28.802799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00227421,"stat_sum":{"num_bytes":3999342,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":4815,"num_read_kb":35700,"num_write":1582,"num_write_kb":33802,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":233,"num_bytes_recovered":14742839,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2b","length":"1"},{"start":"2e","length":"1"},{"start":"31","length":"1"},{"start":"34","length":"1"},{"start":"37","length":"1"}]},{"pgid":"3.a","version":"213'986","reported_seq":4843,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-21T15:01:45.374814+0000","last_change":"2024-09-21T15:01:45.187442+0000","last_active":"2024-09-21T15:01:45.374814+0000","last_peered":"2024-09-21T15:01:45.374814+0000","last_clean":"2024-09-21T15:01:45.374814+0000","last_became_active":"2024-09-21T15:00:27.280971+0000","last_became_peered":"2024-09-21T15:00:27.280971+0000","last_unstale":"2024-09-21T15:01:45.374814+0000","last_undegraded":"2024-09-21T15:01:45.374814+0000","last_fullsized":"2024-09-21T15:01:45.374814+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":986,"log_dups_size":0,"ondisk_log_size":986,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:26:07.617059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001770464,"stat_sum":{"num_bytes":11986518,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":7236,"num_read_kb":31048,"num_write":1387,"num_write_kb":33159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":9228123,"num_keys_recovered":33,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2b","length":"1"},{"start":"2e","length":"1"},{"start":"31","length":"1"},{"start":"34","length":"1"},{"start":"37","length":"1"}]},{"pgid":"3.b","version":"214'553","reported_seq":3311,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-21T15:01:46.307466+0000","last_change":"2024-09-21T15:01:46.307466+0000","last_active":"2024-09-21T15:01:46.307466+0000","last_peered":"2024-09-21T15:01:46.307466+0000","last_clean":"2024-09-21T15:01:46.307466+0000","last_became_active":"2024-09-21T14:58:19.410679+0000","last_became_peered":"2024-09-21T14:58:19.410679+0000","last_unstale":"2024-09-21T15:01:46.307466+0000","last_undegraded":"2024-09-21T15:01:46.307466+0000","last_fullsized":"2024-09-21T15:01:46.307466+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":553,"log_dups_size":0,"ondisk_log_size":553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:07:40.758932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097758099999999994,"stat_sum":{"num_bytes":7673320,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":3075,"num_read_kb":36736,"num_write":847,"num_write_kb":39644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"28","length":"1"},{"start":"2b","length":"1"},{"start":"2e","length":"1"},{"start":"31","length":"1"},{"start":"34","length":"1"},{"start":"37","length":"1"}]},{"pgid":"3.c","version":"213'468","reported_seq":4590,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-21T15:01:45.315116+0000","last_change":"2024-09-21T15:01:45.190013+0000","last_active":"2024-09-21T15:01:45.315116+0000","last_peered":"2024-09-21T15:01:45.315116+0000","last_clean":"2024-09-21T15:01:45.315116+0000","last_became_active":"2024-09-21T14:59:25.130057+0000","last_became_peered":"2024-09-21T14:59:25.130057+0000","last_unstale":"2024-09-21T15:01:45.315116+0000","last_undegraded":"2024-09-21T15:01:45.315116+0000","last_fullsized":"2024-09-21T15:01:45.315116+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":468,"log_dups_size":0,"ondisk_log_size":468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:53:33.752550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096798100000000003,"stat_sum":{"num_bytes":10756559,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":4513,"num_read_kb":28786,"num_write":704,"num_write_kb":24616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":9059600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2b","length":"1"},{"start":"2e","length":"1"},{"start":"31","length":"1"},{"start":"34","length":"1"},{"start":"37","length":"1"}]},{"pgid":"3.d","version":"213'523","reported_seq":5239,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-21T15:01:45.181486+0000","last_change":"2024-09-21T15:01:45.181486+0000","last_active":"2024-09-21T15:01:45.181486+0000","last_peered":"2024-09-21T15:01:45.181486+0000","last_clean":"2024-09-21T15:01:45.181486+0000","last_became_active":"2024-09-21T14:58:19.407884+0000","last_became_peered":"2024-09-21T14:58:19.407884+0000","last_unstale":"2024-09-21T15:01:45.181486+0000","last_undegraded":"2024-09-21T15:01:45.181486+0000","last_fullsized":"2024-09-21T15:01:45.181486+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":523,"log_dups_size":0,"ondisk_log_size":523,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:28:57.111615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016463859999999999,"stat_sum":{"num_bytes":20595995,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":4968,"num_read_kb":55828,"num_write":924,"num_write_kb":38650,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2b","length":"1"},{"start":"2e","length":"1"},{"start":"31","length":"1"},{"start":"34","length":"1"},{"start":"37","length":"1"}]},{"pgid":"3.e","version":"213'852","reported_seq":3091,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-21T15:01:45.294175+0000","last_change":"2024-09-21T15:01:45.182794+0000","last_active":"2024-09-21T15:01:45.294175+0000","last_peered":"2024-09-21T15:01:45.294175+0000","last_clean":"2024-09-21T15:01:45.294175+0000","last_became_active":"2024-09-21T15:01:05.172427+0000","last_became_peered":"2024-09-21T15:01:05.172427+0000","last_unstale":"2024-09-21T15:01:45.294175+0000","last_undegraded":"2024-09-21T15:01:45.294175+0000","last_fullsized":"2024-09-21T15:01:45.294175+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":852,"log_dups_size":0,"ondisk_log_size":852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:28:18.710621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037456329999999999,"stat_sum":{"num_bytes":6014655,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":4229,"num_read_kb":17263,"num_write":1480,"num_write_kb":17965,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":138,"num_bytes_recovered":7486496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2b","length":"1"},{"start":"2e","length":"1"},{"start":"31","length":"1"},{"start":"34","length":"1"},{"start":"37","length":"1"}]},{"pgid":"3.f","version":"213'500","reported_seq":6077,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-21T15:01:45.324860+0000","last_change":"2024-09-21T15:01:45.189973+0000","last_active":"2024-09-21T15:01:45.324860+0000","last_peered":"2024-09-21T15:01:45.324860+0000","last_clean":"2024-09-21T15:01:45.324860+0000","last_became_active":"2024-09-21T14:58:19.410406+0000","last_became_peered":"2024-09-21T14:58:19.410406+0000","last_unstale":"2024-09-21T15:01:45.324860+0000","last_undegraded":"2024-09-21T15:01:45.324860+0000","last_fullsized":"2024-09-21T15:01:45.324860+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":500,"log_dups_size":0,"ondisk_log_size":500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:59:16.026939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078822699999999996,"stat_sum":{"num_bytes":5345572,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":5773,"num_read_kb":56248,"num_write":783,"num_write_kb":36768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2b","length":"1"},{"start":"2e","length":"1"},{"start":"31","length":"1"},{"start":"34","length":"1"},{"start":"37","length":"1"}]},{"pgid":"3.10","version":"213'507","reported_seq":4536,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-21T15:01:45.375434+0000","last_change":"2024-09-21T15:01:45.185772+0000","last_active":"2024-09-21T15:01:45.375434+0000","last_peered":"2024-09-21T15:01:45.375434+0000","last_clean":"2024-09-21T15:01:45.375434+0000","last_became_active":"2024-09-21T14:58:19.409935+0000","last_became_peered":"2024-09-21T14:58:19.409935+0000","last_unstale":"2024-09-21T15:01:45.375434+0000","last_undegraded":"2024-09-21T15:01:45.375434+0000","last_fullsized":"2024-09-21T15:01:45.375434+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":507,"log_dups_size":0,"ondisk_log_size":507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:10:36.252629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057139199999999999,"stat_sum":{"num_bytes":10863077,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":3947,"num_read_kb":58692,"num_write":794,"num_write_kb":37279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2b","length":"1"},{"start":"2e","length":"1"},{"start":"31","length":"1"},{"start":"34","length":"1"},{"start":"37","length":"1"}]},{"pgid":"3.11","version":"213'604","reported_seq":5082,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-21T15:01:45.306283+0000","last_change":"2024-09-21T15:01:45.179984+0000","last_active":"2024-09-21T15:01:45.306283+0000","last_peered":"2024-09-21T15:01:45.306283+0000","last_clean":"2024-09-21T15:01:45.306283+0000","last_became_active":"2024-09-21T15:01:05.165955+0000","last_became_peered":"2024-09-21T15:01:05.165955+0000","last_unstale":"2024-09-21T15:01:45.306283+0000","last_undegraded":"2024-09-21T15:01:45.306283+0000","last_fullsized":"2024-09-21T15:01:45.306283+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":604,"log_dups_size":0,"ondisk_log_size":604,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:34:54.111119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024048870000000001,"stat_sum":{"num_bytes":4654958,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":5758,"num_read_kb":28485,"num_write":758,"num_write_kb":24185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":166,"num_bytes_recovered":11965028,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2b","length":"1"},{"start":"2e","length":"1"},{"start":"31","length":"1"},{"start":"34","length":"1"},{"start":"37","length":"1"}]},{"pgid":"3.12","version":"213'626","reported_seq":8421,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-21T15:01:45.307561+0000","last_change":"2024-09-21T15:01:45.253810+0000","last_active":"2024-09-21T15:01:45.307561+0000","last_peered":"2024-09-21T15:01:45.307561+0000","last_clean":"2024-09-21T15:01:45.307561+0000","last_became_active":"2024-09-21T15:00:07.155144+0000","last_became_peered":"2024-09-21T15:00:07.155144+0000","last_unstale":"2024-09-21T15:01:45.307561+0000","last_undegraded":"2024-09-21T15:01:45.307561+0000","last_fullsized":"2024-09-21T15:01:45.307561+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":626,"log_dups_size":0,"ondisk_log_size":626,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:58:00.133639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012600999999999999,"stat_sum":{"num_bytes":4983903,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":9098,"num_read_kb":37251,"num_write":996,"num_write_kb":20896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2b","length":"1"},{"start":"2e","length":"1"},{"start":"31","length":"1"},{"start":"34","length":"1"},{"start":"37","length":"1"}]},{"pgid":"3.13","version":"213'886","reported_seq":6585,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-21T15:01:45.322514+0000","last_change":"2024-09-21T15:01:45.180668+0000","last_active":"2024-09-21T15:01:45.322514+0000","last_peered":"2024-09-21T15:01:45.322514+0000","last_clean":"2024-09-21T15:01:45.322514+0000","last_became_active":"2024-09-21T14:58:19.412030+0000","last_became_peered":"2024-09-21T14:58:19.412030+0000","last_unstale":"2024-09-21T15:01:45.322514+0000","last_undegraded":"2024-09-21T15:01:45.322514+0000","last_fullsized":"2024-09-21T15:01:45.322514+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":886,"log_dups_size":0,"ondisk_log_size":886,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:44:29.282391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001518876,"stat_sum":{"num_bytes":3773542,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":8227,"num_read_kb":59592,"num_write":1267,"num_write_kb":31571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2b","length":"1"},{"start":"2e","length":"1"},{"start":"31","length":"1"},{"start":"34","length":"1"},{"start":"37","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":272167032,"num_objects":605,"num_object_clones":0,"num_object_copies":1210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":605,"num_whiteouts":0,"num_read":171198,"num_read_kb":1235792,"num_write":33256,"num_write_kb":940659,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3029,"num_bytes_recovered":210630977,"num_keys_recovered":305,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":21905408,"data_stored":213145158,"data_compressed":3323447,"data_compressed_allocated":17702912,"data_compressed_original":210676617,"omap_allocated":0,"internal_metadata":0},"log_size":21449,"ondisk_log_size":21449,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":10404,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509489,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34840,"kb_used_data":3716,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94337000,"statfs":{"total":96636764160,"available":96601088000,"internally_reserved":0,"allocated":3805184,"data_stored":14235151,"data_compressed":113620,"data_compressed_allocated":999424,"data_compressed_original":12854450,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.85499999999999998,"15min":0.84199999999999997},"min":{"1min":0.46500000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.872,"5min":0.93300000000000005,"15min":0.95199999999999996},"min":{"1min":0.54900000000000004,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.3839999999999999,"5min":1.917,"15min":1.917},"last":0.94199999999999995}]},{"osd":1,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.81000000000000005,"15min":0.77700000000000002},"min":{"1min":0.35499999999999998,"5min":0.312,"15min":0.312},"max":{"1min":1.2789999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":1.038},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.83199999999999996,"15min":0.82599999999999996},"min":{"1min":0.57399999999999995,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.208,"5min":2.2149999999999999,"15min":2.2149999999999999},"last":0.67500000000000004}]},{"osd":2,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.82199999999999995,"15min":0.81200000000000006},"min":{"1min":0.50900000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.169,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.93500000000000005,"15min":0.94299999999999995},"min":{"1min":0.55600000000000005,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.355,"5min":1.466,"15min":1.466},"last":0.97799999999999998}]},{"osd":3,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.88500000000000001,"15min":0.86099999999999999},"min":{"1min":0.60999999999999999,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.401,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.97799999999999998,"15min":0.96899999999999997},"min":{"1min":0.68200000000000005,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.3420000000000001,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.88800000000000001}]},{"osd":4,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.92200000000000004,"15min":0.93300000000000005},"min":{"1min":0.42199999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.3089999999999999,"5min":2.7130000000000001,"15min":2.7130000000000001},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.94499999999999995,"15min":0.92600000000000005},"min":{"1min":0.39000000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.425,"5min":2.125,"15min":2.125},"last":0.69999999999999996}]},{"osd":5,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":1.026,"15min":1.022},"min":{"1min":0.51300000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.5800000000000001,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":0.75},{"interface":"front","average":{"1min":0.94299999999999995,"5min":1.0409999999999999,"15min":1.05},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.3029999999999999,"5min":3.1880000000000002,"15min":3.1880000000000002},"last":0.80200000000000005}]},{"osd":6,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.133,"15min":1.149},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.5329999999999999,"5min":3.2730000000000001,"15min":3.2730000000000001},"last":1.1140000000000001},{"interface":"front","average":{"1min":1.095,"5min":1.0629999999999999,"15min":1.03},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.6990000000000001,"5min":2.2879999999999998,"15min":2.2879999999999998},"last":1.0920000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542192,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63816,"kb_used_data":6388,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94308024,"statfs":{"total":96636764160,"available":96571416576,"internally_reserved":0,"allocated":6541312,"data_stored":35357167,"data_compressed":603331,"data_compressed_allocated":3006464,"data_compressed_original":33545357,"omap_allocated":1593,"internal_metadata":58718663},"hb_peers":[0,1,2,3,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:01:08 2024","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.92500000000000004,"15min":0.95099999999999996},"min":{"1min":0.51900000000000002,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.2010000000000001,"5min":1.593,"15min":1.593},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.93300000000000005,"15min":0.94999999999999996},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.226,"5min":1.587,"15min":1.587},"last":1.1180000000000001}]},{"osd":1,"last update":"Sat Sep 21 15:01:08 2024","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.89400000000000002,"15min":0.91000000000000003},"min":{"1min":0.51700000000000002,"5min":0.502,"15min":0.502},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.90200000000000002,"15min":0.96199999999999997},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.57399999999999995}]},{"osd":2,"last update":"Sat Sep 21 15:01:08 2024","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.96199999999999997,"15min":0.98399999999999999},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.389,"5min":1.758,"15min":1.758},"last":1.0529999999999999},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.90900000000000003,"15min":0.91300000000000003},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.444,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.59099999999999997}]},{"osd":3,"last update":"Sat Sep 21 15:01:08 2024","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.94799999999999995,"15min":0.95899999999999996},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.5329999999999999,"5min":1.835,"15min":1.835},"last":0.65300000000000002},{"interface":"front","average":{"1min":1.016,"5min":1.0920000000000001,"15min":1.127},"min":{"1min":0.64600000000000002,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.4670000000000001,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.98499999999999999}]},{"osd":4,"last update":"Sat Sep 21 15:01:08 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.93500000000000005,"15min":0.99099999999999999},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":1.2,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.98499999999999999,"15min":1.0209999999999999},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.3759999999999999,"5min":1.897,"15min":1.897},"last":0.68300000000000005}]},{"osd":5,"last update":"Sat Sep 21 15:01:08 2024","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":1.0449999999999999,"15min":1.1100000000000001},"min":{"1min":0.63500000000000001,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.256,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.41799999999999998},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0660000000000001,"15min":1.087},"min":{"1min":0.65400000000000003,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.609,"5min":1.609,"15min":1.609},"last":0.54200000000000004}]},{"osd":7,"last update":"Sat Sep 21 15:01:08 2024","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.206,"15min":1.244},"min":{"1min":0.78900000000000003,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.3109999999999999,"5min":1.8660000000000001,"15min":1.8660000000000001},"last":0.68799999999999994},{"interface":"front","average":{"1min":1,"5min":1.1970000000000001,"15min":1.2749999999999999},"min":{"1min":0.69499999999999995,"5min":0.63,"15min":0.63},"max":{"1min":1.298,"5min":2.004,"15min":2.004},"last":0.52700000000000002}]}]},{"osd":5,"up_from":14,"seq":60129542193,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62564,"kb_used_data":5204,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94309276,"statfs":{"total":96636764160,"available":96572698624,"internally_reserved":0,"allocated":5328896,"data_stored":27165164,"data_compressed":362675,"data_compressed_allocated":2129920,"data_compressed_original":25524456,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,1,2,3,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:01:07 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.89900000000000002,"15min":0.98799999999999999},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.337,"5min":4.4800000000000004,"15min":4.4800000000000004},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.627,"5min":0.76800000000000002,"15min":0.80800000000000005},"min":{"1min":0.36099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.94999999999999996,"5min":1.181,"15min":1.181},"last":0.88}]},{"osd":1,"last update":"Sat Sep 21 15:01:02 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.89600000000000002,"15min":0.92400000000000004},"min":{"1min":0.47699999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":2.7709999999999999,"5min":2.7709999999999999,"15min":2.7709999999999999},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.91400000000000003,"15min":0.93400000000000005},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":2.823,"5min":2.823,"15min":2.823},"last":0.98799999999999999}]},{"osd":2,"last update":"Sat Sep 21 15:01:02 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.91200000000000003,"15min":0.97399999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.159,"5min":3.419,"15min":3.419},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.87,"15min":0.89600000000000002},"min":{"1min":0.53400000000000003,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0780000000000001,"5min":1.423,"15min":1.423},"last":0.66300000000000003}]},{"osd":3,"last update":"Sat Sep 21 15:01:02 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.80300000000000005,"15min":0.80700000000000005},"min":{"1min":0.58499999999999996,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.96699999999999997,"5min":1.054,"15min":1.0840000000000001},"min":{"1min":0.59899999999999998,"5min":0.435,"15min":0.435},"max":{"1min":1.272,"5min":2.6379999999999999,"15min":2.6379999999999999},"last":0.80100000000000005}]},{"osd":4,"last update":"Sat Sep 21 15:01:02 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.98899999999999999,"15min":1.016},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.5609999999999999,"5min":2.5470000000000002,"15min":2.5470000000000002},"last":1.351},{"interface":"front","average":{"1min":0.82099999999999995,"5min":1.0169999999999999,"15min":1.0780000000000001},"min":{"1min":0.39300000000000002,"5min":0.376,"15min":0.376},"max":{"1min":1.302,"5min":2.5099999999999998,"15min":2.5099999999999998},"last":0.86299999999999999}]},{"osd":6,"last update":"Sat Sep 21 15:01:02 2024","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.94299999999999995,"15min":0.96899999999999997},"min":{"1min":0.63,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.74199999999999999},{"interface":"front","average":{"1min":1.004,"5min":1.2450000000000001,"15min":1.3520000000000001},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.425,"5min":4.2469999999999999,"15min":4.2469999999999999},"last":0.69999999999999996}]},{"osd":7,"last update":"Sat Sep 21 15:01:07 2024","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.2569999999999999,"15min":1.3420000000000001},"min":{"1min":0.52500000000000002,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.383,"5min":3.3010000000000002,"15min":3.3010000000000002},"last":1.01},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.107,"15min":1.135},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.462,"5min":2.5880000000000001,"15min":2.5880000000000001},"last":1.038}]}]},{"osd":4,"up_from":0,"seq":60129542189,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":3,"up_from":14,"seq":60129542193,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78004,"kb_used_data":6696,"kb_used_omap":1,"kb_used_meta":71294,"kb_avail":94293836,"statfs":{"total":96636764160,"available":96556888064,"internally_reserved":0,"allocated":6856704,"data_stored":36847673,"data_compressed":602696,"data_compressed_allocated":3137536,"data_compressed_original":34901137,"omap_allocated":1592,"internal_metadata":73005512},"hb_peers":[0,1,2,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:01:09 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.625,"15min":0.63400000000000001},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.171,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.68200000000000005,"15min":0.70999999999999996},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.0700000000000001,"5min":1.919,"15min":1.919},"last":0.54400000000000004}]},{"osd":1,"last update":"Sat Sep 21 15:01:09 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.66400000000000003,"15min":0.67200000000000004},"min":{"1min":0.29499999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":1.722,"5min":1.722,"15min":1.722},"last":0.501},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.76500000000000001,"15min":0.80300000000000005},"min":{"1min":0.33300000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.89500000000000002,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.78100000000000003}]},{"osd":2,"last update":"Sat Sep 21 15:01:09 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.71999999999999997,"15min":0.72199999999999998},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.92400000000000004,"5min":1.157,"15min":1.157},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.81000000000000005,"15min":0.82699999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.1539999999999999,"5min":1.254,"15min":1.254},"last":0.92700000000000005}]},{"osd":4,"last update":"Sat Sep 21 15:01:09 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.93799999999999994,"15min":0.95599999999999996},"min":{"1min":0.54300000000000004,"5min":0.442,"15min":0.442},"max":{"1min":1.2629999999999999,"5min":1.7609999999999999,"15min":1.7609999999999999},"last":0.97899999999999998},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.98899999999999999,"15min":1.0269999999999999},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.234,"5min":2.028,"15min":2.028},"last":1.1459999999999999}]},{"osd":5,"last update":"Sat Sep 21 15:01:09 2024","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":1.0369999999999999,"15min":1.052},"min":{"1min":0.70499999999999996,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.3440000000000001,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":0.96699999999999997},{"interface":"front","average":{"1min":1.0249999999999999,"5min":1.056,"15min":1.0629999999999999},"min":{"1min":0.57999999999999996,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.548,"5min":1.548,"15min":1.548},"last":0.68799999999999994}]},{"osd":6,"last update":"Sat Sep 21 15:01:09 2024","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":1.0669999999999999,"15min":1.0920000000000001},"min":{"1min":0.59099999999999997,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.6419999999999999,"5min":2.1219999999999999,"15min":2.1219999999999999},"last":0.71799999999999997},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.091,"15min":1.103},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.6839999999999999,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":0.64900000000000002}]},{"osd":7,"last update":"Sat Sep 21 15:01:09 2024","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":1.103,"15min":1.131},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.498,"5min":1.706,"15min":1.706},"last":0.81999999999999995},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.147,"15min":1.177},"min":{"1min":0.59699999999999998,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.417,"5min":2.3380000000000001,"15min":2.3380000000000001},"last":0.85599999999999998}]}]},{"osd":2,"up_from":14,"seq":60129542191,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73048,"kb_used_data":5832,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94298792,"statfs":{"total":96636764160,"available":96561963008,"internally_reserved":0,"allocated":5971968,"data_stored":33723294,"data_compressed":608630,"data_compressed_allocated":2789376,"data_compressed_original":32092214,"omap_allocated":1594,"internal_metadata":68811206},"hb_peers":[0,1,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.62},"min":{"1min":0.26900000000000002,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.92800000000000005,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.69399999999999995,"15min":0.67000000000000004},"min":{"1min":0.34799999999999998,"5min":0.223,"15min":0.223},"max":{"1min":2.9169999999999998,"5min":2.9169999999999998,"15min":2.9169999999999998},"last":0.55500000000000005}]},{"osd":1,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.32700000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.0529999999999999,"5min":2.6520000000000001,"15min":2.6520000000000001},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.746,"5min":0.73499999999999999,"15min":0.75800000000000001},"min":{"1min":0.40500000000000003,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.349,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.70699999999999996}]},{"osd":3,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.73999999999999999,"15min":0.73499999999999999},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":3.0110000000000001,"5min":3.0110000000000001,"15min":3.0110000000000001},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.877,"15min":0.83899999999999997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.05,"5min":4.1440000000000001,"15min":4.1440000000000001},"last":0.747}]},{"osd":4,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.93200000000000005,"15min":0.95599999999999996},"min":{"1min":0.42399999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.222,"5min":2.899,"15min":2.899},"last":0.996},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.94399999999999995,"15min":0.95799999999999996},"min":{"1min":0.47499999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.4379999999999999,"5min":1.675,"15min":1.675},"last":1.1220000000000001}]},{"osd":5,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.0129999999999999,"15min":1.0089999999999999},"min":{"1min":0.45500000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":3.1080000000000001,"5min":3.1080000000000001,"15min":3.1080000000000001},"last":1.224},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.98199999999999998,"15min":0.99399999999999999},"min":{"1min":0.51700000000000002,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.268,"5min":1.546,"15min":1.546},"last":0.98899999999999999}]},{"osd":6,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":0.98299999999999998,"15min":0.98499999999999999},"min":{"1min":0.56200000000000006,"5min":0.378,"15min":0.378},"max":{"1min":1.345,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":0.86099999999999999},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.0649999999999999,"15min":1.0549999999999999},"min":{"1min":0.53200000000000003,"5min":0.502,"15min":0.502},"max":{"1min":3.0739999999999998,"5min":3.0739999999999998,"15min":3.0739999999999998},"last":1.04}]},{"osd":7,"last update":"Sat Sep 21 15:01:08 2024","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.0449999999999999,"15min":1.0089999999999999},"min":{"1min":0.59499999999999997,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":3.1589999999999998,"5min":3.1589999999999998,"15min":3.1589999999999998},"last":1.129},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.121,"15min":1.099},"min":{"1min":0.66000000000000003,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":3.1400000000000001,"5min":3.1400000000000001,"15min":3.1400000000000001},"last":1.177}]}]},{"osd":1,"up_from":14,"seq":60129542193,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52016,"kb_used_data":6908,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94319824,"statfs":{"total":96636764160,"available":96583499776,"internally_reserved":0,"allocated":7073792,"data_stored":42180549,"data_compressed":511838,"data_compressed_allocated":3411968,"data_compressed_original":40296191,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.59599999999999997,"15min":0.60899999999999999},"min":{"1min":0.36399999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.78200000000000003,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59699999999999998,"15min":0.60799999999999998},"min":{"1min":0.39500000000000002,"5min":0.216,"15min":0.216},"max":{"1min":0.80100000000000005,"5min":1.107,"15min":1.107},"last":0.83499999999999996}]},{"osd":2,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":0.79600000000000004,"15min":0.73999999999999999},"min":{"1min":0.38700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":11.303000000000001,"5min":11.303000000000001,"15min":11.303000000000001},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.71299999999999997,"15min":0.70299999999999996},"min":{"1min":0.47299999999999998,"5min":0.435,"15min":0.435},"max":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.64100000000000001}]},{"osd":3,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.81799999999999995,"15min":0.83899999999999997},"min":{"1min":0.27800000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.0049999999999999,"5min":3.1469999999999998,"15min":3.1469999999999998},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.86399999999999999,"15min":0.88600000000000001},"min":{"1min":0.52800000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.306,"5min":3.2320000000000002,"15min":3.2320000000000002},"last":0.93700000000000006}]},{"osd":4,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.89000000000000001,"15min":0.91500000000000004},"min":{"1min":0.59599999999999997,"5min":0.309,"15min":0.309},"max":{"1min":1.45,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":1.0209999999999999},{"interface":"front","average":{"1min":1.5600000000000001,"5min":1.069,"15min":0.96599999999999997},"min":{"1min":0.504,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":11.563000000000001,"5min":11.563000000000001,"15min":11.563000000000001},"last":0.84599999999999997}]},{"osd":5,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":1.625,"5min":1.0469999999999999,"15min":0.94899999999999995},"min":{"1min":0.73399999999999999,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":11.534000000000001,"5min":11.534000000000001,"15min":11.534000000000001},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95799999999999996,"15min":0.96099999999999997},"min":{"1min":0.63,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.256,"5min":1.484,"15min":1.484},"last":0.76000000000000001}]},{"osd":6,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":1.0109999999999999,"15min":1.0489999999999999},"min":{"1min":0.60399999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.4059999999999999,"5min":2.3519999999999999,"15min":2.3519999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":1.631,"5min":1.081,"15min":0.99299999999999999},"min":{"1min":0.57399999999999995,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":11.601000000000001,"5min":11.601000000000001,"15min":11.601000000000001},"last":0.88600000000000001}]},{"osd":7,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":1.046,"15min":1.0640000000000001},"min":{"1min":0.79300000000000004,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.5820000000000001,"5min":2.2360000000000002,"15min":2.2360000000000002},"last":0.67400000000000004},{"interface":"front","average":{"1min":1.76,"5min":1.171,"15min":1.087},"min":{"1min":0.877,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":11.489000000000001,"5min":11.489000000000001,"15min":11.489000000000001},"last":0.79500000000000004}]}]},{"osd":0,"up_from":14,"seq":60129542191,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51056,"kb_used_data":5992,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94320784,"statfs":{"total":96636764160,"available":96584482816,"internally_reserved":0,"allocated":6135808,"data_stored":37271547,"data_compressed":569277,"data_compressed_allocated":2961408,"data_compressed_original":35660959,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 21 15:01:04 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.67500000000000004,"15min":0.71699999999999997},"min":{"1min":0.40999999999999998,"5min":0.309,"15min":0.309},"max":{"1min":1.1299999999999999,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.68100000000000005,"15min":0.627},"min":{"1min":0.58999999999999997,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.81799999999999995}]},{"osd":2,"last update":"Sat Sep 21 15:01:04 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.65200000000000002,"15min":0.61799999999999999},"min":{"1min":0.59599999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.72499999999999998,"15min":0.75800000000000001},"min":{"1min":0.39300000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.86899999999999999}]},{"osd":3,"last update":"Sat Sep 21 15:01:04 2024","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.76600000000000001,"15min":0.754},"min":{"1min":0.50900000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.78700000000000003,"15min":0.81499999999999995},"min":{"1min":0.50900000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.70399999999999996}]},{"osd":4,"last update":"Sat Sep 21 15:01:04 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.98399999999999999,"15min":1.038},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.1140000000000001,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":0.85599999999999998},{"interface":"front","average":{"1min":1.036,"5min":1.0469999999999999,"15min":1.0620000000000001},"min":{"1min":0.67800000000000005,"5min":0.379,"15min":0.379},"max":{"1min":1.446,"5min":1.665,"15min":1.665},"last":0.98399999999999999}]},{"osd":5,"last update":"Sat Sep 21 15:01:04 2024","interfaces":[{"interface":"back","average":{"1min":1.0960000000000001,"5min":1.024,"15min":1.032},"min":{"1min":0.79300000000000004,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.4450000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.90400000000000003},{"interface":"front","average":{"1min":1.0589999999999999,"5min":1.0429999999999999,"15min":1.077},"min":{"1min":0.82399999999999995,"5min":0.435,"15min":0.435},"max":{"1min":1.2609999999999999,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":1.0149999999999999}]},{"osd":6,"last update":"Sat Sep 21 15:01:04 2024","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":1.0580000000000001,"15min":1.1120000000000001},"min":{"1min":0.68400000000000005,"5min":0.628,"15min":0.628},"max":{"1min":1.167,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":0.96799999999999997},{"interface":"front","average":{"1min":1.109,"5min":1.0629999999999999,"15min":1.0680000000000001},"min":{"1min":0.91000000000000003,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.5029999999999999,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.79500000000000004}]},{"osd":7,"last update":"Sat Sep 21 15:01:04 2024","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.0469999999999999,"15min":1.0620000000000001},"min":{"1min":0.78700000000000003,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.4770000000000001,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":0.92900000000000005},{"interface":"front","average":{"1min":1.02,"5min":1.085,"15min":1.131},"min":{"1min":0.84799999999999998,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.403,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":1.0740000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5202,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5202,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":36009702,"data_compressed":569277,"data_compressed_allocated":2961408,"data_compressed_original":35660959,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3407872,"data_stored":35037395,"data_compressed":409382,"data_compressed_allocated":2777088,"data_compressed_original":34660095,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3399680,"data_stored":32461449,"data_compressed":608630,"data_compressed_allocated":2789376,"data_compressed_original":32092214,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":3690496,"data_stored":33021699,"data_compressed":551820,"data_compressed_allocated":2822144,"data_compressed_original":32484497,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":675840,"data_stored":7155966,"data_compressed":134408,"data_compressed_allocated":589824,"data_compressed_original":7119101,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":2699264,"data_stored":25312951,"data_compressed":357473,"data_compressed_allocated":2093056,"data_compressed_original":24951016,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":3248128,"data_stored":31179113,"data_compressed":578837,"data_compressed_allocated":2670592,"data_compressed_original":30854285,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1220608,"data_stored":12966883,"data_compressed":113620,"data_compressed_allocated":999424,"data_compressed_original":12854450,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T15:01:48.230 INFO:tasks.thrashosds.thrasher:Setting 3.1 to [3, 6, 2, 0] 2024-09-21T15:01:48.230 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.1', '3', '6', '2', '0'] 2024-09-21T15:01:48.230 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.1 3 6 2 0 2024-09-21T15:01:48.255 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:01:48.252+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:01:48.334 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:48.334 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:48.344 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T15:01:48.357 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:01:48.352+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:01:48.425 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T15:01:48.489 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:48.489 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:48.490 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T15:01:48.490 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:01:48.484+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:01:48.590 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:01:48.584+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:01:48.639 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:48.639 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:48.650 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T15:01:48.679 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T15:01:48.690 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:01:48.684+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:01:48.700 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T15:01:48.721 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T15:01:48.740 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T15:01:48.750 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:48.750 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:01:48.750 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T15:01:48.750 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T15:01:48.750 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:48.760 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T15:01:48.765 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T15:01:48.787 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T15:01:48.791 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:01:48.789+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:01:48.810 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T15:01:48.810 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:01:48.810 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:01:48.810 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:01:48.810 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:01:48.810 INFO:teuthology.orchestra.run.smithi032.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-21T15:01:48.810 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:01:48.810 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:01:48.810 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:01:48.815 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-21T15:01:48.815 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-21T15:01:48.871 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:48.871 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T15:01:48.871 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T15:01:48.871 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T15:01:48.871 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.871 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T15:01:48.872 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.205754+0000", 2024-09-21T15:01:48.872 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 225.66313775, 2024-09-21T15:01:48.872 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.41117094199999998, 2024-09-21T15:01:48.872 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:48.872 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:48.872 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:48.872 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T15:01:48.872 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T15:01:48.872 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1 2024-09-21T15:01:48.872 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.872 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:48.872 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.872 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:48.872 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205754+0000", 2024-09-21T15:01:48.872 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:48.873 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.873 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.873 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:48.873 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205754+0000", 2024-09-21T15:01:48.873 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:48.873 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.873 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.873 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:48.873 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205752+0000", 2024-09-21T15:01:48.873 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T15:01:48.873 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.873 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.873 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:48.873 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205765+0000", 2024-09-21T15:01:48.874 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3230999999999999e-05 2024-09-21T15:01:48.874 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.874 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.874 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:48.874 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205782+0000", 2024-09-21T15:01:48.874 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6634e-05 2024-09-21T15:01:48.874 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.874 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.874 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:48.874 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205793+0000", 2024-09-21T15:01:48.874 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1297e-05 2024-09-21T15:01:48.874 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.874 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.874 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:48.875 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205867+0000", 2024-09-21T15:01:48.875 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4269000000000003e-05 2024-09-21T15:01:48.875 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.875 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.875 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:48.875 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205877+0000", 2024-09-21T15:01:48.875 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2690000000000005e-06 2024-09-21T15:01:48.875 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.875 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.875 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:48.875 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.613700+0000", 2024-09-21T15:01:48.875 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.407823455 2024-09-21T15:01:48.875 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.875 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.876 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:48.876 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.614203+0000", 2024-09-21T15:01:48.876 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00050311500000000005 2024-09-21T15:01:48.876 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.876 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.876 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T15:01:48.876 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.614750+0000", 2024-09-21T15:01:48.876 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00054738699999999998 2024-09-21T15:01:48.876 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.876 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.876 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:01:48.876 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616367+0000", 2024-09-21T15:01:48.876 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001616634 2024-09-21T15:01:48.876 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.876 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.877 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:01:48.877 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616574+0000", 2024-09-21T15:01:48.877 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00020668499999999999 2024-09-21T15:01:48.877 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.877 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.877 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:01:48.877 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616802+0000", 2024-09-21T15:01:48.877 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00022817900000000001 2024-09-21T15:01:48.877 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.877 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.877 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:48.877 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616925+0000", 2024-09-21T15:01:48.877 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012309699999999999 2024-09-21T15:01:48.877 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:48.878 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:48.878 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:48.878 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.878 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.878 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:61214 3.7 3:e28aec10:::rbd_header.1136862f872f:head [watch reconnect cookie 139864671932080 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-21T15:01:48.878 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:18.147172+0000", 2024-09-21T15:01:48.878 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 150.72172010700001, 2024-09-21T15:01:48.878 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.019264956, 2024-09-21T15:01:48.878 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:48.878 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:48.878 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:48.878 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:48.878 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:48.878 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 61214 2024-09-21T15:01:48.879 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.879 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:48.879 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.879 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:48.879 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147172+0000", 2024-09-21T15:01:48.879 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:48.879 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.879 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.879 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:48.879 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147172+0000", 2024-09-21T15:01:48.879 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:48.879 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.879 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.879 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:48.880 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147168+0000", 2024-09-21T15:01:48.880 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999962 2024-09-21T15:01:48.880 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.880 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.880 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:48.880 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147188+0000", 2024-09-21T15:01:48.880 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9717000000000001e-05 2024-09-21T15:01:48.880 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.880 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.880 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:48.880 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147239+0000", 2024-09-21T15:01:48.880 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1493999999999999e-05 2024-09-21T15:01:48.880 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.880 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.880 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:48.881 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147265+0000", 2024-09-21T15:01:48.881 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5746999999999999e-05 2024-09-21T15:01:48.881 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.881 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.881 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:48.881 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147372+0000", 2024-09-21T15:01:48.881 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010681200000000001 2024-09-21T15:01:48.881 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.881 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.881 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op must wait for map", 2024-09-21T15:01:48.881 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147380+0000", 2024-09-21T15:01:48.881 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7919999999999993e-06 2024-09-21T15:01:48.881 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.881 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.881 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:48.882 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.154025+0000", 2024-09-21T15:01:48.882 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0066454909999999999 2024-09-21T15:01:48.882 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.882 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.882 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:48.882 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.154041+0000", 2024-09-21T15:01:48.882 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5894000000000001e-05 2024-09-21T15:01:48.882 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.882 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.882 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:48.882 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.164345+0000", 2024-09-21T15:01:48.882 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0103040459999999 2024-09-21T15:01:48.882 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.882 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.883 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:48.883 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.164590+0000", 2024-09-21T15:01:48.883 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00024492699999999998 2024-09-21T15:01:48.883 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.883 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.883 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:48.883 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.166437+0000", 2024-09-21T15:01:48.883 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0018466330000000001 2024-09-21T15:01:48.883 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:48.883 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:48.883 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:48.883 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.883 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.883 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:61215 3.7 3:e28aec10:::rbd_header.1136862f872f:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-21T15:01:48.883 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:18.152008+0000", 2024-09-21T15:01:48.884 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 150.716883655, 2024-09-21T15:01:48.884 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.015644958, 2024-09-21T15:01:48.884 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:48.884 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:48.884 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:48.884 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:48.884 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:48.884 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 61215 2024-09-21T15:01:48.884 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.884 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:48.884 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.884 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:48.884 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152008+0000", 2024-09-21T15:01:48.884 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:48.884 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.885 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.885 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:48.885 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152008+0000", 2024-09-21T15:01:48.885 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:48.885 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.885 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.885 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:48.885 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152005+0000", 2024-09-21T15:01:48.885 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-21T15:01:48.885 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.885 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.885 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:48.885 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152025+0000", 2024-09-21T15:01:48.885 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9306000000000001e-05 2024-09-21T15:01:48.885 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.886 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.886 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:48.886 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152054+0000", 2024-09-21T15:01:48.886 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9189000000000001e-05 2024-09-21T15:01:48.886 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.886 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.886 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:48.886 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152078+0000", 2024-09-21T15:01:48.886 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4317000000000001e-05 2024-09-21T15:01:48.886 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.886 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.886 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:48.886 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152243+0000", 2024-09-21T15:01:48.886 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000164802 2024-09-21T15:01:48.886 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.887 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.887 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting_for_map not empty", 2024-09-21T15:01:48.887 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152251+0000", 2024-09-21T15:01:48.887 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0600000000000008e-06 2024-09-21T15:01:48.887 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.887 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.887 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:48.887 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.154741+0000", 2024-09-21T15:01:48.887 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0024900069999999998 2024-09-21T15:01:48.887 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.887 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.887 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:48.887 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.154753+0000", 2024-09-21T15:01:48.887 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1588e-05 2024-09-21T15:01:48.887 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.888 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.888 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:48.888 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.165380+0000", 2024-09-21T15:01:48.888 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0106272249999999 2024-09-21T15:01:48.888 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.888 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.888 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:48.888 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.165517+0000", 2024-09-21T15:01:48.888 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013737999999999999 2024-09-21T15:01:48.888 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.888 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.888 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 6", 2024-09-21T15:01:48.888 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.165938+0000", 2024-09-21T15:01:48.888 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00042082399999999998 2024-09-21T15:01:48.888 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.888 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.889 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:01:48.889 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.166795+0000", 2024-09-21T15:01:48.889 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00085674899999999997 2024-09-21T15:01:48.889 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.889 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.889 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:01:48.889 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.167598+0000", 2024-09-21T15:01:48.889 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00080335599999999997 2024-09-21T15:01:48.889 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.889 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.889 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:01:48.889 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.167632+0000", 2024-09-21T15:01:48.889 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4122000000000003e-05 2024-09-21T15:01:48.889 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.889 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.889 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:48.890 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.167653+0000", 2024-09-21T15:01:48.890 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0862000000000001e-05 2024-09-21T15:01:48.890 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:48.890 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:48.890 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:48.890 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.890 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.890 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:97678 3.d e122/24 3:b2f34c6d:::rbd_data.1136bfc79a70.0000000000000006:head v 122'234, mlcod=120'232)", 2024-09-21T15:01:48.890 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.321655+0000", 2024-09-21T15:01:48.890 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 103.547236738, 2024-09-21T15:01:48.890 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024509388999999999, 2024-09-21T15:01:48.890 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:48.890 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:48.890 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:48.890 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.891 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:48.891 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.321655+0000", 2024-09-21T15:01:48.891 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:48.891 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.891 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.891 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:48.891 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.321655+0000", 2024-09-21T15:01:48.891 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:48.891 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.891 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.891 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:48.891 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.321655+0000", 2024-09-21T15:01:48.891 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:48.891 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.891 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.892 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:48.892 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.329453+0000", 2024-09-21T15:01:48.892 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0077980560000000003 2024-09-21T15:01:48.892 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.892 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.892 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:48.892 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.329482+0000", 2024-09-21T15:01:48.892 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9153999999999999e-05 2024-09-21T15:01:48.892 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.892 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.892 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:48.892 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.329493+0000", 2024-09-21T15:01:48.892 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1103e-05 2024-09-21T15:01:48.892 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.892 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.892 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:48.893 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.329555+0000", 2024-09-21T15:01:48.893 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2345e-05 2024-09-21T15:01:48.893 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.893 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.893 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:48.893 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.329589+0000", 2024-09-21T15:01:48.893 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3908999999999997e-05 2024-09-21T15:01:48.893 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.893 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.893 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:01:48.893 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.346083+0000", 2024-09-21T15:01:48.893 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016493383 2024-09-21T15:01:48.893 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.893 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.893 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:48.894 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.346165+0000", 2024-09-21T15:01:48.894 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1796000000000001e-05 2024-09-21T15:01:48.894 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:48.894 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:48.894 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:48.894 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.894 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.894 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:97679 3.f e122/24 3:f2a362b4:::rbd_data.1136bfc79a70.0000000000000007:head v 122'331, mlcod=120'329)", 2024-09-21T15:01:48.894 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.337145+0000", 2024-09-21T15:01:48.894 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 103.53174672900001, 2024-09-21T15:01:48.894 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021912411, 2024-09-21T15:01:48.894 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:48.894 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:48.894 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:48.894 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.895 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:48.895 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.337145+0000", 2024-09-21T15:01:48.895 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:48.895 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.895 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.895 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:48.895 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.337145+0000", 2024-09-21T15:01:48.895 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:48.895 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.895 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.895 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:48.895 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.337145+0000", 2024-09-21T15:01:48.895 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:48.895 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.895 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.896 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:48.896 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.343899+0000", 2024-09-21T15:01:48.896 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0067541420000000003 2024-09-21T15:01:48.896 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.896 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.896 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:48.896 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.343921+0000", 2024-09-21T15:01:48.896 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2255e-05 2024-09-21T15:01:48.896 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.896 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.896 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:48.896 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.343933+0000", 2024-09-21T15:01:48.896 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1528e-05 2024-09-21T15:01:48.896 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.896 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.897 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:48.897 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.344001+0000", 2024-09-21T15:01:48.897 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8269999999999995e-05 2024-09-21T15:01:48.897 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.897 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.897 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:48.897 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.344029+0000", 2024-09-21T15:01:48.897 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8254000000000001e-05 2024-09-21T15:01:48.897 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.897 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.897 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:01:48.897 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.358901+0000", 2024-09-21T15:01:48.897 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0148719 2024-09-21T15:01:48.897 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.898 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.898 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:48.898 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.359058+0000", 2024-09-21T15:01:48.898 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000156446 2024-09-21T15:01:48.898 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:48.898 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:48.898 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:48.898 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.898 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.898 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:97676 3.1f e122/24 3:ffaa836a:::rbd_data.1136bfc79a70.0000000000000004:head v 122'312, mlcod=120'310)", 2024-09-21T15:01:48.898 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.348683+0000", 2024-09-21T15:01:48.898 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 103.520208564, 2024-09-21T15:01:48.898 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.033768802000000001, 2024-09-21T15:01:48.899 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:48.899 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:48.899 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:48.899 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.899 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:48.899 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.348683+0000", 2024-09-21T15:01:48.899 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:48.899 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.899 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.899 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:48.899 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.348683+0000", 2024-09-21T15:01:48.899 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:48.899 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.899 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.899 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:48.899 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.348683+0000", 2024-09-21T15:01:48.900 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:48.900 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.900 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.900 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:48.900 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.366937+0000", 2024-09-21T15:01:48.900 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018254415 2024-09-21T15:01:48.900 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.900 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.900 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:48.900 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.366959+0000", 2024-09-21T15:01:48.900 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1634e-05 2024-09-21T15:01:48.900 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.900 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.900 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:48.900 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.366971+0000", 2024-09-21T15:01:48.900 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2305e-05 2024-09-21T15:01:48.901 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.901 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.901 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:48.901 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.367034+0000", 2024-09-21T15:01:48.901 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3211000000000002e-05 2024-09-21T15:01:48.901 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.901 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.901 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:48.901 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.367056+0000", 2024-09-21T15:01:48.901 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2098000000000001e-05 2024-09-21T15:01:48.901 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.901 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.901 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:01:48.901 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.382400+0000", 2024-09-21T15:01:48.901 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.015343867000000001 2024-09-21T15:01:48.901 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.902 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.902 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:48.902 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.382452+0000", 2024-09-21T15:01:48.902 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1992e-05 2024-09-21T15:01:48.902 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:48.902 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:48.902 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:48.902 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.902 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.902 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:97734 3.19 e122/94 3:9fcf754f:::rbd_data.1136bfc79a70.0000000000000009:head v 122'314, mlcod=120'312)", 2024-09-21T15:01:48.902 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.378547+0000", 2024-09-21T15:01:48.902 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 103.49034458600001, 2024-09-21T15:01:48.902 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.063380675999999997, 2024-09-21T15:01:48.902 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:48.902 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:48.903 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:48.903 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.903 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:48.903 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.378547+0000", 2024-09-21T15:01:48.903 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:48.903 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.903 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.903 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:48.903 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.378547+0000", 2024-09-21T15:01:48.903 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:48.903 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.903 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.903 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:48.903 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.378547+0000", 2024-09-21T15:01:48.904 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:01:48.904 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.904 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.904 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:48.904 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.425118+0000", 2024-09-21T15:01:48.904 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046571166999999997 2024-09-21T15:01:48.904 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.904 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.904 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:48.904 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.425150+0000", 2024-09-21T15:01:48.904 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2150999999999998e-05 2024-09-21T15:01:48.904 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.904 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.904 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:48.904 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.425164+0000", 2024-09-21T15:01:48.904 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3302000000000001e-05 2024-09-21T15:01:48.905 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.905 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.905 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:48.905 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.425231+0000", 2024-09-21T15:01:48.905 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7106000000000004e-05 2024-09-21T15:01:48.905 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.905 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.905 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:48.905 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.425259+0000", 2024-09-21T15:01:48.905 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8087000000000001e-05 2024-09-21T15:01:48.905 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.905 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.905 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:01:48.905 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.441890+0000", 2024-09-21T15:01:48.905 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016631328000000001 2024-09-21T15:01:48.905 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.906 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.906 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:48.906 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.441928+0000", 2024-09-21T15:01:48.906 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7701999999999999e-05 2024-09-21T15:01:48.906 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:48.906 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:48.906 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:48.906 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.906 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.906 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:100187 3.10 3:08941a97:::rbd_data.1136ab4d6ff1.000000000000000e:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-21T15:01:48.906 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:10.869845+0000", 2024-09-21T15:01:48.906 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 97.99904669, 2024-09-21T15:01:48.906 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029345237, 2024-09-21T15:01:48.906 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:48.907 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:48.907 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:48.907 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:48.907 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:48.907 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 100187 2024-09-21T15:01:48.907 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.907 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:48.907 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.907 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:48.907 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869845+0000", 2024-09-21T15:01:48.907 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:48.907 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.907 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.907 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:48.908 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869845+0000", 2024-09-21T15:01:48.908 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:48.908 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.908 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.908 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:48.908 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869844+0000", 2024-09-21T15:01:48.908 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:48.908 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.908 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.908 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:48.908 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869853+0000", 2024-09-21T15:01:48.908 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5719999999999992e-06 2024-09-21T15:01:48.908 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.908 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.909 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:48.909 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869874+0000", 2024-09-21T15:01:48.909 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0483e-05 2024-09-21T15:01:48.909 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.909 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.909 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:48.909 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869883+0000", 2024-09-21T15:01:48.909 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6080000000000002e-06 2024-09-21T15:01:48.909 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.909 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.909 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:48.909 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869991+0000", 2024-09-21T15:01:48.909 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000108013 2024-09-21T15:01:48.909 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.909 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.909 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:48.910 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.870258+0000", 2024-09-21T15:01:48.910 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00026629099999999999 2024-09-21T15:01:48.910 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.910 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.910 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T15:01:48.910 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.878656+0000", 2024-09-21T15:01:48.910 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0083989000000000008 2024-09-21T15:01:48.910 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.910 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.910 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:01:48.910 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.899081+0000", 2024-09-21T15:01:48.910 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.020424437 2024-09-21T15:01:48.910 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.911 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.911 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:01:48.911 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.899125+0000", 2024-09-21T15:01:48.911 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4440999999999997e-05 2024-09-21T15:01:48.911 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.911 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.911 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:01:48.911 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.899160+0000", 2024-09-21T15:01:48.911 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4518999999999999e-05 2024-09-21T15:01:48.911 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.911 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.911 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:48.911 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.899190+0000", 2024-09-21T15:01:48.911 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0574999999999998e-05 2024-09-21T15:01:48.912 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:48.912 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:48.912 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:48.912 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.912 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.912 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:100263 3.10 3:0ece142e:::rbd_data.1136ab4d6ff1.0000000000000008:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-21T15:01:48.912 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:11.052257+0000", 2024-09-21T15:01:48.912 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 97.816634632000003, 2024-09-21T15:01:48.912 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042678025000000001, 2024-09-21T15:01:48.912 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:48.912 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:48.912 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:48.912 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:48.912 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:48.912 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 100263 2024-09-21T15:01:48.913 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.913 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:48.913 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.913 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:48.913 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052257+0000", 2024-09-21T15:01:48.913 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:48.913 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.913 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.913 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:48.913 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052257+0000", 2024-09-21T15:01:48.913 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:48.913 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.913 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.913 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:48.913 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052256+0000", 2024-09-21T15:01:48.914 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:48.914 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.914 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.914 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:48.914 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052263+0000", 2024-09-21T15:01:48.914 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4799999999999998e-06 2024-09-21T15:01:48.914 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.914 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.914 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:48.914 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052275+0000", 2024-09-21T15:01:48.914 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2386e-05 2024-09-21T15:01:48.914 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.914 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.914 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:48.915 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052281+0000", 2024-09-21T15:01:48.915 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6699999999999999e-06 2024-09-21T15:01:48.915 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.915 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.915 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:48.915 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052350+0000", 2024-09-21T15:01:48.915 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9099999999999999e-05 2024-09-21T15:01:48.915 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.915 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.915 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:48.915 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052456+0000", 2024-09-21T15:01:48.915 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000106669 2024-09-21T15:01:48.915 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.915 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.915 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T15:01:48.915 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.059694+0000", 2024-09-21T15:01:48.916 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.007237566 2024-09-21T15:01:48.916 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.916 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.916 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:01:48.916 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.094721+0000", 2024-09-21T15:01:48.916 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035026977000000001 2024-09-21T15:01:48.916 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.916 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.916 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:01:48.916 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.094746+0000", 2024-09-21T15:01:48.916 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5329000000000001e-05 2024-09-21T15:01:48.916 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.916 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.916 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:01:48.916 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.094850+0000", 2024-09-21T15:01:48.916 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000103452 2024-09-21T15:01:48.917 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.917 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.917 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:48.917 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.094935+0000", 2024-09-21T15:01:48.917 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5550000000000006e-05 2024-09-21T15:01:48.917 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:48.917 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:48.917 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:48.917 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.917 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.917 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:134708 3.a 3:51eaadc0:::rbd_data.11365ed419e3.0000000000000007:67 [read 221184~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:48.917 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.938945+0000", 2024-09-21T15:01:48.917 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 54.929947353999999, 2024-09-21T15:01:48.917 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021073784000000002, 2024-09-21T15:01:48.917 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:48.917 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:48.918 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:48.918 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:48.918 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:48.918 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 134708 2024-09-21T15:01:48.918 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.918 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:48.918 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.918 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:48.918 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938945+0000", 2024-09-21T15:01:48.918 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:48.918 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.918 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.918 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:48.918 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938945+0000", 2024-09-21T15:01:48.918 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:48.919 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.919 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.919 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:48.919 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938944+0000", 2024-09-21T15:01:48.919 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:48.919 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.919 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.919 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:48.919 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938946+0000", 2024-09-21T15:01:48.919 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.616e-06 2024-09-21T15:01:48.919 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.919 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.919 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:48.919 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938952+0000", 2024-09-21T15:01:48.919 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7309999999999998e-06 2024-09-21T15:01:48.920 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.920 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.920 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:48.920 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938957+0000", 2024-09-21T15:01:48.920 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8579999999999999e-06 2024-09-21T15:01:48.920 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.920 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.920 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:48.920 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.959730+0000", 2024-09-21T15:01:48.920 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.020773183000000001 2024-09-21T15:01:48.920 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.920 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.920 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:48.920 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.959810+0000", 2024-09-21T15:01:48.920 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0158000000000001e-05 2024-09-21T15:01:48.921 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.921 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.921 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:48.921 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.960018+0000", 2024-09-21T15:01:48.921 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000207951 2024-09-21T15:01:48.921 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:48.921 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:48.921 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:48.921 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.921 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.921 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:134709 3.a 3:51eaadc0:::rbd_data.11365ed419e3.0000000000000007:67 [read 237568~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:48.921 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.938996+0000", 2024-09-21T15:01:48.921 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 54.929896362000001, 2024-09-21T15:01:48.921 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021382577, 2024-09-21T15:01:48.922 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:48.922 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:48.922 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:48.922 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:48.922 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:48.922 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 134709 2024-09-21T15:01:48.922 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.922 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:48.922 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.922 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:48.922 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938996+0000", 2024-09-21T15:01:48.922 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:48.922 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.922 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.923 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:48.923 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938996+0000", 2024-09-21T15:01:48.923 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:48.923 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.923 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.923 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:48.923 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938995+0000", 2024-09-21T15:01:48.923 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:48.923 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.923 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.923 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:48.923 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938999+0000", 2024-09-21T15:01:48.923 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7979999999999999e-06 2024-09-21T15:01:48.923 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.924 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.924 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:48.924 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939006+0000", 2024-09-21T15:01:48.924 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6669999999999996e-06 2024-09-21T15:01:48.924 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.924 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.924 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:48.924 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939012+0000", 2024-09-21T15:01:48.924 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9329999999999998e-06 2024-09-21T15:01:48.924 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.924 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.924 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:48.924 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.960080+0000", 2024-09-21T15:01:48.924 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021068303 2024-09-21T15:01:48.924 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.924 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.925 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:48.925 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.960154+0000", 2024-09-21T15:01:48.925 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3819000000000006e-05 2024-09-21T15:01:48.925 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.925 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.925 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:48.925 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.960378+0000", 2024-09-21T15:01:48.925 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00022385800000000001 2024-09-21T15:01:48.925 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:48.925 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:48.925 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:48.925 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.925 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.925 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:134710 3.a 3:51eaadc0:::rbd_data.11365ed419e3.0000000000000007:67 [read 253952~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:48.925 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.939017+0000", 2024-09-21T15:01:48.926 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 54.929875351, 2024-09-21T15:01:48.926 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021721436, 2024-09-21T15:01:48.926 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:48.926 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:48.926 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:48.926 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:48.926 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:48.926 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 134710 2024-09-21T15:01:48.926 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.926 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:48.926 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.926 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:48.926 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939017+0000", 2024-09-21T15:01:48.926 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:48.927 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.927 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.927 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:48.927 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939017+0000", 2024-09-21T15:01:48.927 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:48.927 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.927 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.927 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:48.927 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939016+0000", 2024-09-21T15:01:48.927 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:48.927 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.927 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.927 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:48.927 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939019+0000", 2024-09-21T15:01:48.927 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.554e-06 2024-09-21T15:01:48.928 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.928 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.928 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:48.928 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939024+0000", 2024-09-21T15:01:48.928 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1569999999999999e-06 2024-09-21T15:01:48.928 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.928 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.928 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:48.928 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939034+0000", 2024-09-21T15:01:48.928 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9219999999999992e-06 2024-09-21T15:01:48.928 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.928 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.928 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:48.928 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.960408+0000", 2024-09-21T15:01:48.929 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021373792999999999 2024-09-21T15:01:48.929 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.929 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.929 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:48.929 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.960482+0000", 2024-09-21T15:01:48.929 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4331000000000004e-05 2024-09-21T15:01:48.929 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.929 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.929 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:48.929 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.960738+0000", 2024-09-21T15:01:48.929 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00025600500000000001 2024-09-21T15:01:48.929 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:48.929 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:48.929 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:48.929 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.930 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.930 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:134711 3.a 3:51eaadc0:::rbd_data.11365ed419e3.0000000000000007:67 [read 270336~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:48.930 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.939039+0000", 2024-09-21T15:01:48.930 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 54.929853301999998, 2024-09-21T15:01:48.930 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022064698000000001, 2024-09-21T15:01:48.930 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:48.930 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:48.930 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:48.930 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:48.930 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:48.930 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 134711 2024-09-21T15:01:48.930 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.930 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:48.930 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.931 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:48.931 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939039+0000", 2024-09-21T15:01:48.931 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:48.931 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.931 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.931 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:48.931 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939039+0000", 2024-09-21T15:01:48.931 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:48.931 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.931 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.931 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:48.931 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939038+0000", 2024-09-21T15:01:48.931 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:48.931 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.932 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.932 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:48.932 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939041+0000", 2024-09-21T15:01:48.932 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9440000000000001e-06 2024-09-21T15:01:48.932 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.932 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.932 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:48.932 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939047+0000", 2024-09-21T15:01:48.932 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1260000000000003e-06 2024-09-21T15:01:48.932 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.932 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.932 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:48.932 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939052+0000", 2024-09-21T15:01:48.932 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1619999999999999e-06 2024-09-21T15:01:48.933 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.933 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.933 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:48.933 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.960768+0000", 2024-09-21T15:01:48.933 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021715140000000001 2024-09-21T15:01:48.933 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.933 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.933 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:48.933 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.960842+0000", 2024-09-21T15:01:48.933 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4698999999999997e-05 2024-09-21T15:01:48.933 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.933 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.933 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:48.933 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.961103+0000", 2024-09-21T15:01:48.934 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00026098499999999998 2024-09-21T15:01:48.934 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:48.934 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:48.934 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:48.934 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.934 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.934 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:134712 3.a 3:51eaadc0:::rbd_data.11365ed419e3.0000000000000007:67 [read 286720~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:48.934 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.939057+0000", 2024-09-21T15:01:48.934 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 54.929834769000003, 2024-09-21T15:01:48.934 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022387067, 2024-09-21T15:01:48.934 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:48.934 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:48.934 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:48.934 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:48.934 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:48.935 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 134712 2024-09-21T15:01:48.935 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.935 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:48.935 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.935 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:48.935 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939057+0000", 2024-09-21T15:01:48.935 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:48.935 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.935 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.935 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:48.935 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939057+0000", 2024-09-21T15:01:48.935 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:48.935 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.935 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.935 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:48.936 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939057+0000", 2024-09-21T15:01:48.936 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:48.936 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.936 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.936 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:48.936 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939059+0000", 2024-09-21T15:01:48.936 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.447e-06 2024-09-21T15:01:48.936 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.936 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.936 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:48.936 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939065+0000", 2024-09-21T15:01:48.936 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4670000000000003e-06 2024-09-21T15:01:48.936 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.936 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.937 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:48.937 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939069+0000", 2024-09-21T15:01:48.937 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2119999999999997e-06 2024-09-21T15:01:48.937 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.937 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.937 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:48.937 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.961141+0000", 2024-09-21T15:01:48.937 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022072268999999999 2024-09-21T15:01:48.937 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.937 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.937 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:48.937 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.961240+0000", 2024-09-21T15:01:48.937 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9147999999999993e-05 2024-09-21T15:01:48.937 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.937 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.937 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:48.938 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.961444+0000", 2024-09-21T15:01:48.938 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00020381899999999999 2024-09-21T15:01:48.938 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:48.938 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:48.938 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:48.938 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.938 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.938 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:134713 3.a 3:51eaadc0:::rbd_data.11365ed419e3.0000000000000007:67 [read 303104~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:48.938 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.939073+0000", 2024-09-21T15:01:48.938 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 54.929818865999998, 2024-09-21T15:01:48.938 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022687394999999999, 2024-09-21T15:01:48.938 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:48.938 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:48.938 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:48.939 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:48.939 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:48.939 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 134713 2024-09-21T15:01:48.939 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.939 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:48.939 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.939 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:48.939 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939073+0000", 2024-09-21T15:01:48.939 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:48.939 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.939 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.939 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:48.939 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939073+0000", 2024-09-21T15:01:48.939 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:48.939 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.940 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.940 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:48.940 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939073+0000", 2024-09-21T15:01:48.940 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:48.940 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.940 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.940 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:48.940 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939075+0000", 2024-09-21T15:01:48.940 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5969999999999999e-06 2024-09-21T15:01:48.940 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.940 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.940 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:48.940 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939081+0000", 2024-09-21T15:01:48.940 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.609e-06 2024-09-21T15:01:48.940 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.940 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.941 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:48.941 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939086+0000", 2024-09-21T15:01:48.941 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.916e-06 2024-09-21T15:01:48.941 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.941 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.941 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:48.941 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.961474+0000", 2024-09-21T15:01:48.941 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022388512999999999 2024-09-21T15:01:48.941 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.941 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.941 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:48.941 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.961567+0000", 2024-09-21T15:01:48.941 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2267000000000003e-05 2024-09-21T15:01:48.941 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.941 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.941 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:48.942 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.961760+0000", 2024-09-21T15:01:48.942 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00019375999999999999 2024-09-21T15:01:48.942 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:48.942 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:48.942 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:48.942 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.942 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.942 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:134714 3.a 3:51eaadc0:::rbd_data.11365ed419e3.0000000000000007:67 [read 319488~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:48.942 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.939090+0000", 2024-09-21T15:01:48.942 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 54.929802039000002, 2024-09-21T15:01:48.942 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023043403, 2024-09-21T15:01:48.942 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:48.942 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:48.942 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:48.942 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:48.943 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:48.943 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 134714 2024-09-21T15:01:48.943 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.943 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:48.943 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.943 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:48.943 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939090+0000", 2024-09-21T15:01:48.943 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:48.943 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.943 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.943 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:48.943 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939090+0000", 2024-09-21T15:01:48.943 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:48.943 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.943 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.943 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:48.944 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939090+0000", 2024-09-21T15:01:48.944 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:48.944 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.944 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.944 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:48.944 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939092+0000", 2024-09-21T15:01:48.944 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2790000000000001e-06 2024-09-21T15:01:48.944 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.944 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.944 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:48.944 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939097+0000", 2024-09-21T15:01:48.944 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1410000000000004e-06 2024-09-21T15:01:48.944 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.944 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.944 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:48.944 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939102+0000", 2024-09-21T15:01:48.945 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7369999999999997e-06 2024-09-21T15:01:48.945 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.945 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.945 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:48.945 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.961791+0000", 2024-09-21T15:01:48.945 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022688974000000001 2024-09-21T15:01:48.945 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.945 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.945 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:48.945 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.961865+0000", 2024-09-21T15:01:48.945 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4651000000000001e-05 2024-09-21T15:01:48.945 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.945 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.945 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:48.945 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.962133+0000", 2024-09-21T15:01:48.945 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00026793999999999999 2024-09-21T15:01:48.946 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:48.946 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:48.946 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:48.946 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.946 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.946 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:134715 3.a 3:51eaadc0:::rbd_data.11365ed419e3.0000000000000007:67 [read 335872~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:48.946 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.939106+0000", 2024-09-21T15:01:48.946 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 54.929785428999999, 2024-09-21T15:01:48.946 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023401142999999999, 2024-09-21T15:01:48.946 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:48.946 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:48.946 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:48.946 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:48.946 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:48.947 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 134715 2024-09-21T15:01:48.947 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.947 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:48.947 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.947 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:48.947 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939106+0000", 2024-09-21T15:01:48.947 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:48.947 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.947 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.947 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:48.947 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939106+0000", 2024-09-21T15:01:48.947 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:48.947 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.947 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.947 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:48.948 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939106+0000", 2024-09-21T15:01:48.948 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:48.948 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.948 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.948 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:48.948 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939114+0000", 2024-09-21T15:01:48.948 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1009999999999992e-06 2024-09-21T15:01:48.948 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.948 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.948 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:48.948 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939121+0000", 2024-09-21T15:01:48.948 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7710000000000001e-06 2024-09-21T15:01:48.948 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.948 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.949 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:48.949 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939126+0000", 2024-09-21T15:01:48.949 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3600000000000004e-06 2024-09-21T15:01:48.949 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.949 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.949 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:48.949 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.962162+0000", 2024-09-21T15:01:48.949 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023036148999999999 2024-09-21T15:01:48.949 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.949 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.949 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:48.949 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.962239+0000", 2024-09-21T15:01:48.949 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6874999999999998e-05 2024-09-21T15:01:48.949 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.949 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.949 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:48.950 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.962508+0000", 2024-09-21T15:01:48.950 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00026842199999999999 2024-09-21T15:01:48.950 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:48.950 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:48.950 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:48.950 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.950 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.950 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:134716 3.a 3:51eaadc0:::rbd_data.11365ed419e3.0000000000000007:67 [read 352256~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:48.950 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.939131+0000", 2024-09-21T15:01:48.950 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 54.929761294999999, 2024-09-21T15:01:48.950 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023742441, 2024-09-21T15:01:48.950 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:48.950 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:48.950 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:48.950 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:48.951 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:48.951 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 134716 2024-09-21T15:01:48.951 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.951 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:48.951 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.951 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:48.951 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939131+0000", 2024-09-21T15:01:48.951 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:48.951 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.951 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.951 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:48.951 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939131+0000", 2024-09-21T15:01:48.951 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:48.951 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.951 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.952 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:48.952 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939130+0000", 2024-09-21T15:01:48.952 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:48.952 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.952 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.952 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:48.952 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939133+0000", 2024-09-21T15:01:48.952 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5620000000000002e-06 2024-09-21T15:01:48.952 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.952 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.952 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:48.952 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939138+0000", 2024-09-21T15:01:48.952 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.322e-06 2024-09-21T15:01:48.952 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.952 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.953 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:48.953 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939143+0000", 2024-09-21T15:01:48.953 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5510000000000001e-06 2024-09-21T15:01:48.953 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.953 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.953 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:48.953 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.962537+0000", 2024-09-21T15:01:48.953 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023394808 2024-09-21T15:01:48.953 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.953 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.953 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:48.953 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.962608+0000", 2024-09-21T15:01:48.953 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0699999999999997e-05 2024-09-21T15:01:48.953 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.953 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.953 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:48.954 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.962873+0000", 2024-09-21T15:01:48.954 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00026487 2024-09-21T15:01:48.954 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:48.954 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:48.954 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:48.954 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.954 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.954 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:135024 3.a 3:53261383:::rbd_data.11365ed419e3.0000000000000037:snapdir [list-snaps] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:48.954 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.940162+0000", 2024-09-21T15:01:48.954 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 54.928730332000001, 2024-09-21T15:01:48.954 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022852172, 2024-09-21T15:01:48.954 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:48.954 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:48.954 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:48.955 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:48.955 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:48.955 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 135024 2024-09-21T15:01:48.955 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.955 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:48.955 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.955 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:48.955 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940162+0000", 2024-09-21T15:01:48.955 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:48.955 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.955 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.955 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:48.955 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940162+0000", 2024-09-21T15:01:48.955 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:48.955 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.956 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.956 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:48.956 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940161+0000", 2024-09-21T15:01:48.956 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:48.956 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.956 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.956 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:48.956 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940164+0000", 2024-09-21T15:01:48.956 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4099999999999998e-06 2024-09-21T15:01:48.956 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.956 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.956 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:48.956 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940169+0000", 2024-09-21T15:01:48.956 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3720000000000004e-06 2024-09-21T15:01:48.956 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.957 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.957 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:48.957 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940174+0000", 2024-09-21T15:01:48.957 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.724e-06 2024-09-21T15:01:48.957 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.957 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.957 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:48.957 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.962902+0000", 2024-09-21T15:01:48.957 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022728719000000001 2024-09-21T15:01:48.957 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.957 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.957 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:48.957 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.963014+0000", 2024-09-21T15:01:48.957 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000111245 2024-09-21T15:01:48.957 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:48.957 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:48.958 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:48.958 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.958 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.958 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:135044 3.a 3:53261383:::rbd_data.11365ed419e3.0000000000000037:snapdir [list-snaps] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:48.958 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.940484+0000", 2024-09-21T15:01:48.958 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 54.9284076, 2024-09-21T15:01:48.958 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022619169000000001, 2024-09-21T15:01:48.958 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:48.958 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:48.958 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:48.958 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:48.958 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:48.958 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 135044 2024-09-21T15:01:48.958 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.959 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:48.959 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.959 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:48.959 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940484+0000", 2024-09-21T15:01:48.959 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:48.959 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.959 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.959 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:48.959 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940484+0000", 2024-09-21T15:01:48.959 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:48.959 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.959 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.959 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:48.959 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940484+0000", 2024-09-21T15:01:48.960 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:48.960 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.960 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.960 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:48.960 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940487+0000", 2024-09-21T15:01:48.960 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8449999999999999e-06 2024-09-21T15:01:48.960 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.960 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.960 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:48.960 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940493+0000", 2024-09-21T15:01:48.960 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9599999999999997e-06 2024-09-21T15:01:48.960 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.960 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.960 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:48.960 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940497+0000", 2024-09-21T15:01:48.961 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9620000000000001e-06 2024-09-21T15:01:48.961 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.961 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.961 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:48.961 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.963034+0000", 2024-09-21T15:01:48.961 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022536878 2024-09-21T15:01:48.961 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:48.961 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:48.961 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:48.961 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.963103+0000", 2024-09-21T15:01:48.961 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9073999999999996e-05 2024-09-21T15:01:48.961 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:48.961 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:48.961 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:48.962 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:48.962 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:48.962 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:48.963 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T15:01:48.964 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:01:48.960+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:01:49.064 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:01:49.061+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:01:49.074 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:49.074 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:01:49.074 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T15:01:49.074 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:49.084 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T15:01:49.165 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:01:49.160+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:01:49.195 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:49.195 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:01:49.195 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T15:01:49.195 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T15:01:49.195 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:49.205 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T15:01:49.266 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:01:49.260+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:01:49.299 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T15:01:49.302 INFO:teuthology.orchestra.run.smithi032.stderr:set 3.1 pg_upmap_items mapping to [3->6,2->0] 2024-09-21T15:01:49.323 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:49.323 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T15:01:49.323 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T15:01:49.323 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T15:01:49.323 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.323 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:15130 3.2 3:47f1bbdb:::rbd_data.11361bed1f78.0000000000000067:head [sparse-read 0~147456] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-21T15:01:49.323 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:40.288284+0000", 2024-09-21T15:01:49.323 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 189.03389008100001, 2024-09-21T15:01:49.323 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1142935039999999, 2024-09-21T15:01:49.323 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:49.323 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:49.324 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:49.324 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:49.324 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:49.324 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 15130 2024-09-21T15:01:49.324 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.324 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:49.324 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.324 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:49.324 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288284+0000", 2024-09-21T15:01:49.324 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:49.324 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.324 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.324 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:49.324 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288284+0000", 2024-09-21T15:01:49.325 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:49.325 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.325 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.325 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:49.325 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288284+0000", 2024-09-21T15:01:49.325 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:49.325 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.325 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.325 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:49.325 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288288+0000", 2024-09-21T15:01:49.325 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5829999999999998e-06 2024-09-21T15:01:49.325 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.325 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.325 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:49.326 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288297+0000", 2024-09-21T15:01:49.326 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0669999999999998e-06 2024-09-21T15:01:49.326 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.326 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.326 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:49.326 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288305+0000", 2024-09-21T15:01:49.326 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8860000000000004e-06 2024-09-21T15:01:49.326 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.326 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.326 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:49.326 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288417+0000", 2024-09-21T15:01:49.326 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000112952 2024-09-21T15:01:49.326 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.326 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.326 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:49.327 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288428+0000", 2024-09-21T15:01:49.327 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0631e-05 2024-09-21T15:01:49.327 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.327 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.327 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:49.327 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.401876+0000", 2024-09-21T15:01:49.327 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1134479610000001 2024-09-21T15:01:49.327 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.327 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.327 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:49.327 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.402076+0000", 2024-09-21T15:01:49.327 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00019944200000000001 2024-09-21T15:01:49.327 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.327 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.328 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:49.328 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.402578+0000", 2024-09-21T15:01:49.328 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00050242899999999998 2024-09-21T15:01:49.328 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:49.328 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:49.328 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:49.328 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.328 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.328 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45708 3.2 3:415db91a:::rbd_data.1136ecbe5879.0000000000000000:28 [read 140288~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:49.328 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141801+0000", 2024-09-21T15:01:49.328 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 159.180373887, 2024-09-21T15:01:49.328 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034274200999999997, 2024-09-21T15:01:49.328 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:49.328 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:49.329 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:49.329 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:49.329 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:49.329 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45708 2024-09-21T15:01:49.329 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.329 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:49.329 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.329 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:49.329 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141801+0000", 2024-09-21T15:01:49.329 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:49.329 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.329 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.329 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:49.329 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141801+0000", 2024-09-21T15:01:49.329 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:49.329 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.330 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.330 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:49.330 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141800+0000", 2024-09-21T15:01:49.330 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:49.330 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.330 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.330 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:49.330 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141803+0000", 2024-09-21T15:01:49.330 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4210000000000002e-06 2024-09-21T15:01:49.330 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.330 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.330 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:49.330 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141812+0000", 2024-09-21T15:01:49.330 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.9800000000000004e-06 2024-09-21T15:01:49.330 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.331 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.331 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:49.331 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141816+0000", 2024-09-21T15:01:49.331 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4769999999999997e-06 2024-09-21T15:01:49.331 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.331 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.331 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:49.331 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.175865+0000", 2024-09-21T15:01:49.331 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034048875999999999 2024-09-21T15:01:49.331 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.331 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.331 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:49.331 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.175943+0000", 2024-09-21T15:01:49.331 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7464000000000002e-05 2024-09-21T15:01:49.332 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.332 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.332 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:49.332 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176075+0000", 2024-09-21T15:01:49.332 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013230800000000001 2024-09-21T15:01:49.332 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:49.332 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:49.332 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:49.332 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.332 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.332 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45709 3.2 3:415db91a:::rbd_data.1136ecbe5879.0000000000000000:28 [read 143360~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:49.332 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141820+0000", 2024-09-21T15:01:49.332 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 159.18035404, 2024-09-21T15:01:49.332 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034537536000000001, 2024-09-21T15:01:49.332 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:49.333 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:49.333 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:49.333 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:49.333 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:49.333 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45709 2024-09-21T15:01:49.333 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.333 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:49.333 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.333 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:49.333 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141820+0000", 2024-09-21T15:01:49.333 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:49.333 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.333 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.333 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:49.333 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141820+0000", 2024-09-21T15:01:49.334 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:49.334 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.334 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.334 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:49.334 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141820+0000", 2024-09-21T15:01:49.334 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:49.334 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.334 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.334 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:49.334 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141823+0000", 2024-09-21T15:01:49.334 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5110000000000002e-06 2024-09-21T15:01:49.334 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.334 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.334 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:49.335 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141829+0000", 2024-09-21T15:01:49.335 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0850000000000002e-06 2024-09-21T15:01:49.335 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.335 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.335 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:49.335 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141833+0000", 2024-09-21T15:01:49.335 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7620000000000003e-06 2024-09-21T15:01:49.335 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.335 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.335 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:49.335 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176103+0000", 2024-09-21T15:01:49.335 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034269933000000002 2024-09-21T15:01:49.335 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.335 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.336 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:49.336 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176161+0000", 2024-09-21T15:01:49.336 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7658999999999998e-05 2024-09-21T15:01:49.336 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.336 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.336 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:49.336 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176358+0000", 2024-09-21T15:01:49.336 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000196937 2024-09-21T15:01:49.336 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:49.336 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:49.336 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:49.336 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.336 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.336 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45710 3.2 3:415db91a:::rbd_data.1136ecbe5879.0000000000000000:28 [read 146432~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:49.337 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141838+0000", 2024-09-21T15:01:49.337 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 159.180336877, 2024-09-21T15:01:49.337 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034953674999999997, 2024-09-21T15:01:49.337 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:49.337 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:49.337 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:49.337 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:49.337 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:49.337 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45710 2024-09-21T15:01:49.337 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.337 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:49.337 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.337 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:49.337 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141838+0000", 2024-09-21T15:01:49.337 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:49.337 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.338 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.338 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:49.338 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141838+0000", 2024-09-21T15:01:49.338 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:49.338 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.338 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.338 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:49.338 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141837+0000", 2024-09-21T15:01:49.338 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:49.338 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.338 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.338 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:49.338 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141843+0000", 2024-09-21T15:01:49.338 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1619999999999996e-06 2024-09-21T15:01:49.339 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.339 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.339 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:49.339 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141850+0000", 2024-09-21T15:01:49.339 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4799999999999998e-06 2024-09-21T15:01:49.339 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.339 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.339 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:49.339 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141853+0000", 2024-09-21T15:01:49.339 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3050000000000001e-06 2024-09-21T15:01:49.339 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.339 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.339 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:49.339 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176490+0000", 2024-09-21T15:01:49.339 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034636413999999997 2024-09-21T15:01:49.340 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.340 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.340 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:49.340 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176606+0000", 2024-09-21T15:01:49.340 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000115841 2024-09-21T15:01:49.340 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.340 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.340 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:49.340 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176791+0000", 2024-09-21T15:01:49.340 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00018581500000000001 2024-09-21T15:01:49.340 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:49.340 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:49.340 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:49.340 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.340 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.341 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45711 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:49.341 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141856+0000", 2024-09-21T15:01:49.341 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 159.18031854, 2024-09-21T15:01:49.341 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034986308000000001, 2024-09-21T15:01:49.341 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:49.341 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:49.341 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:49.341 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:49.341 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:49.341 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45711 2024-09-21T15:01:49.341 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.341 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:49.341 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.341 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:49.342 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141856+0000", 2024-09-21T15:01:49.342 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:49.342 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.342 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.342 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:49.342 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141856+0000", 2024-09-21T15:01:49.342 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:49.342 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.342 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.342 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:49.342 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141856+0000", 2024-09-21T15:01:49.342 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:49.342 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.342 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.343 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:49.343 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141862+0000", 2024-09-21T15:01:49.343 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.567e-06 2024-09-21T15:01:49.343 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.343 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.343 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:49.343 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141868+0000", 2024-09-21T15:01:49.343 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9569999999999999e-06 2024-09-21T15:01:49.343 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.343 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.343 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:49.343 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141871+0000", 2024-09-21T15:01:49.343 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.146e-06 2024-09-21T15:01:49.343 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.343 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.344 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:49.344 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176831+0000", 2024-09-21T15:01:49.344 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034959935999999997 2024-09-21T15:01:49.344 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.344 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.344 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:49.344 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176842+0000", 2024-09-21T15:01:49.344 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1008e-05 2024-09-21T15:01:49.344 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:49.344 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:49.344 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:49.344 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.344 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.344 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45712 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:49.345 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141874+0000", 2024-09-21T15:01:49.345 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 159.18030064499999, 2024-09-21T15:01:49.345 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034993511999999997, 2024-09-21T15:01:49.345 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:49.345 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:49.345 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:49.345 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:49.345 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:49.345 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45712 2024-09-21T15:01:49.345 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.345 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:49.345 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.345 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:49.345 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141874+0000", 2024-09-21T15:01:49.346 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:49.346 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.346 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.346 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:49.346 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141874+0000", 2024-09-21T15:01:49.346 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:49.346 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.346 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.346 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:49.346 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141874+0000", 2024-09-21T15:01:49.346 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:49.346 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.346 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.346 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:49.346 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141875+0000", 2024-09-21T15:01:49.347 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5430000000000001e-06 2024-09-21T15:01:49.347 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.347 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.347 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:49.347 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141878+0000", 2024-09-21T15:01:49.347 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1420000000000002e-06 2024-09-21T15:01:49.347 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.347 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.347 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:49.347 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141881+0000", 2024-09-21T15:01:49.347 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.807e-06 2024-09-21T15:01:49.347 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.347 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.347 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:49.347 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176860+0000", 2024-09-21T15:01:49.348 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034979243 2024-09-21T15:01:49.348 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.348 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.348 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:49.348 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176867+0000", 2024-09-21T15:01:49.348 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0169999999999999e-06 2024-09-21T15:01:49.348 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:49.348 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:49.348 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:49.348 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.348 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.348 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45713 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:49.348 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141883+0000", 2024-09-21T15:01:49.348 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 159.18029107000001, 2024-09-21T15:01:49.349 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035010516999999998, 2024-09-21T15:01:49.349 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:49.349 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:49.349 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:49.349 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:49.349 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:49.349 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45713 2024-09-21T15:01:49.349 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.349 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:49.349 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.349 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:49.349 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141883+0000", 2024-09-21T15:01:49.349 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:49.349 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.349 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.350 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:49.350 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141883+0000", 2024-09-21T15:01:49.350 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:49.350 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.350 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.350 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:49.350 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141883+0000", 2024-09-21T15:01:49.350 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:49.350 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.350 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.350 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:49.350 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141885+0000", 2024-09-21T15:01:49.350 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4649999999999999e-06 2024-09-21T15:01:49.351 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.351 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.351 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:49.351 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141888+0000", 2024-09-21T15:01:49.351 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3890000000000001e-06 2024-09-21T15:01:49.351 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.351 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.351 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:49.351 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141891+0000", 2024-09-21T15:01:49.351 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9239999999999999e-06 2024-09-21T15:01:49.351 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.351 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.351 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:49.351 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176887+0000", 2024-09-21T15:01:49.351 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034996370999999998 2024-09-21T15:01:49.352 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.352 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.352 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:49.352 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176894+0000", 2024-09-21T15:01:49.352 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6069999999999996e-06 2024-09-21T15:01:49.352 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:49.352 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:49.352 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:49.352 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.352 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.352 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45714 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:49.352 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141893+0000", 2024-09-21T15:01:49.352 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 159.18028115499999, 2024-09-21T15:01:49.352 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035048215000000001, 2024-09-21T15:01:49.352 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:49.353 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:49.353 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:49.353 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:49.353 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:49.353 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45714 2024-09-21T15:01:49.353 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.353 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:49.353 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.353 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:49.353 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141893+0000", 2024-09-21T15:01:49.353 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:49.353 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.353 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.353 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:49.354 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141893+0000", 2024-09-21T15:01:49.354 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:49.354 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.354 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.354 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:49.354 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141893+0000", 2024-09-21T15:01:49.354 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:49.354 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.354 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.354 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:49.354 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141895+0000", 2024-09-21T15:01:49.354 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.703e-06 2024-09-21T15:01:49.354 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.354 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.355 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:49.355 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141900+0000", 2024-09-21T15:01:49.355 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.242e-06 2024-09-21T15:01:49.355 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.355 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.355 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:49.355 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141903+0000", 2024-09-21T15:01:49.355 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.117e-06 2024-09-21T15:01:49.355 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.355 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.355 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:49.355 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176930+0000", 2024-09-21T15:01:49.355 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035026750000000002 2024-09-21T15:01:49.355 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.356 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.356 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:49.356 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176942+0000", 2024-09-21T15:01:49.356 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1668000000000001e-05 2024-09-21T15:01:49.356 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:49.356 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:49.356 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:49.356 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.356 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.356 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45715 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:49.356 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141906+0000", 2024-09-21T15:01:49.357 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 159.18026899700001, 2024-09-21T15:01:49.357 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035066057999999997, 2024-09-21T15:01:49.357 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:49.357 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:49.357 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:49.357 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:49.357 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:49.357 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45715 2024-09-21T15:01:49.357 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.357 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:49.357 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.357 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:49.357 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141906+0000", 2024-09-21T15:01:49.357 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:49.357 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.358 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.358 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:49.358 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141906+0000", 2024-09-21T15:01:49.358 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:49.358 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.358 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.358 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:49.358 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141905+0000", 2024-09-21T15:01:49.358 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:49.358 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.358 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.358 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:49.358 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141907+0000", 2024-09-21T15:01:49.358 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5400000000000001e-06 2024-09-21T15:01:49.358 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.359 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.359 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:49.359 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141912+0000", 2024-09-21T15:01:49.359 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.502e-06 2024-09-21T15:01:49.359 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.359 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.359 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:49.359 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141915+0000", 2024-09-21T15:01:49.359 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0170000000000001e-06 2024-09-21T15:01:49.359 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.359 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.359 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:49.359 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176964+0000", 2024-09-21T15:01:49.359 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035048587999999999 2024-09-21T15:01:49.360 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.360 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.360 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:49.360 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176972+0000", 2024-09-21T15:01:49.360 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6580000000000002e-06 2024-09-21T15:01:49.360 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:49.360 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:49.360 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:49.360 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.360 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.360 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45716 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:49.360 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141918+0000", 2024-09-21T15:01:49.360 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 159.180256821, 2024-09-21T15:01:49.360 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035086223, 2024-09-21T15:01:49.361 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:49.361 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:49.361 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:49.361 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:49.361 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:49.361 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45716 2024-09-21T15:01:49.361 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.361 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:49.361 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.361 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:49.361 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141918+0000", 2024-09-21T15:01:49.361 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:49.361 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.361 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.362 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:49.362 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141918+0000", 2024-09-21T15:01:49.362 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:49.362 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.362 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.362 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:49.362 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141917+0000", 2024-09-21T15:01:49.362 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:49.362 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.362 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.362 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:49.362 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141919+0000", 2024-09-21T15:01:49.362 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5680000000000001e-06 2024-09-21T15:01:49.362 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.363 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.363 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:49.363 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141922+0000", 2024-09-21T15:01:49.363 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2789999999999999e-06 2024-09-21T15:01:49.363 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.363 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.363 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:49.363 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141925+0000", 2024-09-21T15:01:49.363 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9469999999999999e-06 2024-09-21T15:01:49.363 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.363 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.363 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:49.363 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176997+0000", 2024-09-21T15:01:49.364 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035071366999999999 2024-09-21T15:01:49.364 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.364 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.364 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:49.364 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177004+0000", 2024-09-21T15:01:49.364 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3309999999999998e-06 2024-09-21T15:01:49.364 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:49.364 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:49.364 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:49.364 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.364 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.364 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45717 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:49.364 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141928+0000", 2024-09-21T15:01:49.364 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 159.18024687499999, 2024-09-21T15:01:49.365 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035102573999999998, 2024-09-21T15:01:49.365 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:49.365 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:49.365 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:49.365 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:49.365 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:49.365 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45717 2024-09-21T15:01:49.365 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.365 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:49.365 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.365 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:49.365 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141928+0000", 2024-09-21T15:01:49.365 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:49.365 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.366 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.366 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:49.366 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141928+0000", 2024-09-21T15:01:49.366 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:49.366 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.366 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.366 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:49.366 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141927+0000", 2024-09-21T15:01:49.366 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:01:49.366 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.366 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.366 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:49.366 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141929+0000", 2024-09-21T15:01:49.366 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.533e-06 2024-09-21T15:01:49.367 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.367 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.367 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:49.367 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141934+0000", 2024-09-21T15:01:49.367 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1660000000000002e-06 2024-09-21T15:01:49.367 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.367 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.367 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:49.367 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141937+0000", 2024-09-21T15:01:49.367 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0309999999999999e-06 2024-09-21T15:01:49.367 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.367 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.367 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:49.367 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177023+0000", 2024-09-21T15:01:49.368 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035085595999999997 2024-09-21T15:01:49.368 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.368 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.368 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:49.368 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177030+0000", 2024-09-21T15:01:49.368 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.486e-06 2024-09-21T15:01:49.368 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:49.368 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:49.368 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:49.368 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.368 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.368 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45718 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:49.368 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141940+0000", 2024-09-21T15:01:49.368 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 159.180235017, 2024-09-21T15:01:49.368 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035124956999999998, 2024-09-21T15:01:49.369 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:49.369 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:49.369 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:49.369 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:49.369 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:49.369 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45718 2024-09-21T15:01:49.369 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.369 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:49.369 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.369 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:49.369 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141940+0000", 2024-09-21T15:01:49.369 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:49.369 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.369 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.370 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:49.370 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141940+0000", 2024-09-21T15:01:49.370 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:49.370 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.370 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.370 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:49.370 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141939+0000", 2024-09-21T15:01:49.370 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:49.370 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.370 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.370 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:49.370 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141941+0000", 2024-09-21T15:01:49.370 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.502e-06 2024-09-21T15:01:49.370 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.371 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.371 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:49.371 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141946+0000", 2024-09-21T15:01:49.371 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3449999999999996e-06 2024-09-21T15:01:49.371 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.371 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.371 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:49.371 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141949+0000", 2024-09-21T15:01:49.371 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9749999999999999e-06 2024-09-21T15:01:49.371 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.371 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.371 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:49.371 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177057+0000", 2024-09-21T15:01:49.371 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035108385999999998 2024-09-21T15:01:49.371 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.372 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.372 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:49.372 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177064+0000", 2024-09-21T15:01:49.372 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0099999999999998e-06 2024-09-21T15:01:49.372 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:49.372 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:49.372 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:49.372 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.372 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.372 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45719 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:49.372 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141952+0000", 2024-09-21T15:01:49.372 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 159.18022274399999, 2024-09-21T15:01:49.372 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035139108000000002, 2024-09-21T15:01:49.372 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:49.373 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:49.373 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:49.373 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:49.373 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:49.373 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45719 2024-09-21T15:01:49.373 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.373 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:49.373 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.373 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:49.373 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141952+0000", 2024-09-21T15:01:49.373 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:49.373 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.373 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.373 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:49.374 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141952+0000", 2024-09-21T15:01:49.374 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:49.374 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.374 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.374 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:49.374 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141952+0000", 2024-09-21T15:01:49.374 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:49.374 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.374 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.374 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:49.374 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141953+0000", 2024-09-21T15:01:49.374 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.486e-06 2024-09-21T15:01:49.374 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.374 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.375 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:49.375 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141959+0000", 2024-09-21T15:01:49.375 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4940000000000003e-06 2024-09-21T15:01:49.375 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.375 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.375 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:49.375 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141963+0000", 2024-09-21T15:01:49.375 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3540000000000002e-06 2024-09-21T15:01:49.375 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.375 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.375 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:49.375 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177084+0000", 2024-09-21T15:01:49.375 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035120983000000001 2024-09-21T15:01:49.375 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.375 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.376 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:49.376 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177091+0000", 2024-09-21T15:01:49.376 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0350000000000004e-06 2024-09-21T15:01:49.376 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:49.376 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:49.376 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:49.376 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.376 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.376 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45720 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:49.376 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141965+0000", 2024-09-21T15:01:49.376 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 159.18020927699999, 2024-09-21T15:01:49.376 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035155268000000003, 2024-09-21T15:01:49.376 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:49.376 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:49.377 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:49.377 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:49.377 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:49.377 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45720 2024-09-21T15:01:49.377 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.377 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:49.377 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.377 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:49.377 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141965+0000", 2024-09-21T15:01:49.377 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:49.377 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.377 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.377 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:49.377 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141965+0000", 2024-09-21T15:01:49.377 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:49.378 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.378 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.378 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:49.378 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141965+0000", 2024-09-21T15:01:49.378 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:49.378 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.378 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.378 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:49.378 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141966+0000", 2024-09-21T15:01:49.378 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.46e-06 2024-09-21T15:01:49.378 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.378 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.378 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:49.378 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141972+0000", 2024-09-21T15:01:49.379 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4210000000000003e-06 2024-09-21T15:01:49.379 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.379 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.379 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:49.379 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141975+0000", 2024-09-21T15:01:49.379 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.948e-06 2024-09-21T15:01:49.379 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.379 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.379 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:49.379 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177113+0000", 2024-09-21T15:01:49.379 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035137855000000003 2024-09-21T15:01:49.379 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.379 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.379 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:49.380 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177121+0000", 2024-09-21T15:01:49.380 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8269999999999998e-06 2024-09-21T15:01:49.380 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:49.380 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:49.380 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:49.380 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.380 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.380 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45721 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:49.380 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141977+0000", 2024-09-21T15:01:49.380 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 159.18019725400001, 2024-09-21T15:01:49.380 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035169984000000001, 2024-09-21T15:01:49.380 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:49.381 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:49.381 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:49.381 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:49.381 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:49.381 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45721 2024-09-21T15:01:49.381 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.381 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:49.381 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.381 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:49.381 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141977+0000", 2024-09-21T15:01:49.381 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:49.381 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.381 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.381 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:49.382 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141977+0000", 2024-09-21T15:01:49.382 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:49.382 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.382 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.382 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:49.382 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141977+0000", 2024-09-21T15:01:49.382 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:49.382 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.382 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.382 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:49.382 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141979+0000", 2024-09-21T15:01:49.382 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.511e-06 2024-09-21T15:01:49.382 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.382 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.382 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:49.383 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141982+0000", 2024-09-21T15:01:49.383 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4180000000000001e-06 2024-09-21T15:01:49.383 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.383 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.383 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:49.383 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141985+0000", 2024-09-21T15:01:49.383 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9450000000000002e-06 2024-09-21T15:01:49.383 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.383 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.383 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:49.383 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177139+0000", 2024-09-21T15:01:49.383 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035154409999999997 2024-09-21T15:01:49.383 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.383 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.383 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:49.384 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177147+0000", 2024-09-21T15:01:49.384 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9570000000000002e-06 2024-09-21T15:01:49.384 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:49.384 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:49.384 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:49.384 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.384 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.384 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45722 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:49.384 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141987+0000", 2024-09-21T15:01:49.384 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 159.18018728300001, 2024-09-21T15:01:49.384 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035188492000000002, 2024-09-21T15:01:49.384 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:49.384 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:49.384 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:49.384 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:49.385 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:49.385 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45722 2024-09-21T15:01:49.385 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.385 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:49.385 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.385 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:49.385 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141987+0000", 2024-09-21T15:01:49.385 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:49.385 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.385 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.385 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:49.385 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141987+0000", 2024-09-21T15:01:49.385 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:49.385 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.385 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.385 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:49.386 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141987+0000", 2024-09-21T15:01:49.386 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:49.386 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.386 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.386 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:49.386 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141988+0000", 2024-09-21T15:01:49.386 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.463e-06 2024-09-21T15:01:49.386 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.386 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.386 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:49.386 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141994+0000", 2024-09-21T15:01:49.386 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.361e-06 2024-09-21T15:01:49.386 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.386 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.387 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:49.387 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141997+0000", 2024-09-21T15:01:49.387 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0359999999999998e-06 2024-09-21T15:01:49.387 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.387 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.387 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:49.387 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177169+0000", 2024-09-21T15:01:49.387 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035171651999999998 2024-09-21T15:01:49.387 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.387 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.387 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:49.387 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177176+0000", 2024-09-21T15:01:49.387 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2189999999999998e-06 2024-09-21T15:01:49.388 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:49.388 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:49.388 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:49.388 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.388 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.388 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45723 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:49.388 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141999+0000", 2024-09-21T15:01:49.388 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 159.18017511599999, 2024-09-21T15:01:49.388 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035197701999999997, 2024-09-21T15:01:49.388 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:49.388 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:49.388 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:49.388 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:49.388 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:49.389 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45723 2024-09-21T15:01:49.389 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.389 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:49.389 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.389 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:49.389 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141999+0000", 2024-09-21T15:01:49.389 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:49.389 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.389 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.389 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:49.389 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141999+0000", 2024-09-21T15:01:49.389 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:49.389 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.389 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.389 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:49.390 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141999+0000", 2024-09-21T15:01:49.390 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:49.390 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.390 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.390 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:49.390 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142003+0000", 2024-09-21T15:01:49.390 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5049999999999999e-06 2024-09-21T15:01:49.390 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.390 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.390 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:49.390 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142008+0000", 2024-09-21T15:01:49.390 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6579999999999999e-06 2024-09-21T15:01:49.390 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.391 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.391 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:49.391 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142011+0000", 2024-09-21T15:01:49.391 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0529999999999998e-06 2024-09-21T15:01:49.391 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.391 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.391 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:49.391 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177194+0000", 2024-09-21T15:01:49.391 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035182221999999999 2024-09-21T15:01:49.391 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.391 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.391 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:49.391 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177197+0000", 2024-09-21T15:01:49.391 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5030000000000002e-06 2024-09-21T15:01:49.391 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:49.392 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:49.392 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:49.392 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.392 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.392 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45724 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:49.392 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.142015+0000", 2024-09-21T15:01:49.392 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 159.18015914899999, 2024-09-21T15:01:49.392 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035195345000000003, 2024-09-21T15:01:49.392 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:49.392 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:49.392 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:49.392 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:49.392 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:49.392 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45724 2024-09-21T15:01:49.393 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.393 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:49.393 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.393 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:49.393 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142015+0000", 2024-09-21T15:01:49.393 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:49.393 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.393 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.393 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:49.393 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142015+0000", 2024-09-21T15:01:49.393 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:49.393 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.393 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.393 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:49.393 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142015+0000", 2024-09-21T15:01:49.394 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:49.394 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.394 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.394 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:49.394 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142017+0000", 2024-09-21T15:01:49.394 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5540000000000001e-06 2024-09-21T15:01:49.394 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.394 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.394 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:49.394 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142022+0000", 2024-09-21T15:01:49.394 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4229999999999996e-06 2024-09-21T15:01:49.394 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.394 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.394 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:49.394 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142025+0000", 2024-09-21T15:01:49.395 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.056e-06 2024-09-21T15:01:49.395 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.395 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.395 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:49.395 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177207+0000", 2024-09-21T15:01:49.395 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035182089 2024-09-21T15:01:49.395 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.395 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.395 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:49.395 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177211+0000", 2024-09-21T15:01:49.395 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4649999999999999e-06 2024-09-21T15:01:49.395 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:49.395 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:49.395 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:49.396 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.396 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.396 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45725 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:49.396 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.142028+0000", 2024-09-21T15:01:49.396 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 159.18014686999999, 2024-09-21T15:01:49.396 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035195731000000001, 2024-09-21T15:01:49.396 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:49.396 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:49.396 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:49.396 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:49.396 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:49.396 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45725 2024-09-21T15:01:49.396 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.397 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:49.397 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.397 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:49.397 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142028+0000", 2024-09-21T15:01:49.397 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:49.397 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.397 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.397 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:49.397 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142028+0000", 2024-09-21T15:01:49.397 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:49.397 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.397 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.397 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:49.397 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142027+0000", 2024-09-21T15:01:49.397 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:49.398 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.398 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.398 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:49.398 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142029+0000", 2024-09-21T15:01:49.398 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5269999999999999e-06 2024-09-21T15:01:49.398 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.398 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.398 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:49.398 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142032+0000", 2024-09-21T15:01:49.398 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3249999999999999e-06 2024-09-21T15:01:49.398 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.398 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.398 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:49.398 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142035+0000", 2024-09-21T15:01:49.399 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.897e-06 2024-09-21T15:01:49.399 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.399 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.399 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:49.399 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177220+0000", 2024-09-21T15:01:49.399 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035184871 2024-09-21T15:01:49.399 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.399 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.399 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:49.399 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177223+0000", 2024-09-21T15:01:49.399 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3680000000000002e-06 2024-09-21T15:01:49.399 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:49.399 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:49.400 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:49.400 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.400 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.400 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45726 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:49.400 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.142037+0000", 2024-09-21T15:01:49.400 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 159.18013715500001, 2024-09-21T15:01:49.400 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035199273000000003, 2024-09-21T15:01:49.400 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:49.400 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:49.400 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:49.400 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:49.400 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:49.400 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45726 2024-09-21T15:01:49.400 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.400 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:49.401 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.401 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:49.401 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142037+0000", 2024-09-21T15:01:49.401 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:49.401 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.401 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.401 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:49.401 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142037+0000", 2024-09-21T15:01:49.401 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:49.401 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.401 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.401 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:49.401 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142037+0000", 2024-09-21T15:01:49.401 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:49.401 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.402 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.402 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:49.402 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142038+0000", 2024-09-21T15:01:49.402 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.344e-06 2024-09-21T15:01:49.402 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.402 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.402 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:49.402 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142044+0000", 2024-09-21T15:01:49.402 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1930000000000001e-06 2024-09-21T15:01:49.402 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.402 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.402 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:49.402 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142047+0000", 2024-09-21T15:01:49.402 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0280000000000001e-06 2024-09-21T15:01:49.403 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.403 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.403 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:49.403 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177233+0000", 2024-09-21T15:01:49.403 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035186394000000003 2024-09-21T15:01:49.403 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.403 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.403 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:49.403 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177237+0000", 2024-09-21T15:01:49.403 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5530000000000001e-06 2024-09-21T15:01:49.403 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:49.403 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:49.403 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:49.403 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:49.404 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:49.404 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:49.405 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T15:01:49.405 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:01:49.401+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:01:49.505 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:01:49.500+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:01:49.560 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:49.561 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:01:49.561 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T15:01:49.561 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:49.570 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T15:01:49.606 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:01:49.605+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:01:49.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-09-21T15:01:49.617+0000 7fb20cbb0700 0 --2- 172.21.15.32:0/1752736846 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb20806fc40 0x7fb208140490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-21T15:01:49.680 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:49.680 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:01:49.680 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T15:01:49.680 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T15:01:49.680 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:49.690 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T15:01:49.707 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:01:49.704+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:01:49.795 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:49.795 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T15:01:49.795 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T15:01:49.795 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T15:01:49.795 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.795 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139776 3.16 3:6c38d0e2:::rbd_data.113664780c85.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:49.795 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.223442+0000", 2024-09-21T15:01:49.796 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 47.569510366000003, 2024-09-21T15:01:49.796 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0315488150000001, 2024-09-21T15:01:49.796 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:49.796 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:49.796 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:49.796 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:49.796 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:49.796 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139776 2024-09-21T15:01:49.796 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.796 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:49.796 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.796 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:49.796 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223442+0000", 2024-09-21T15:01:49.796 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:49.797 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.797 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.797 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:49.797 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223442+0000", 2024-09-21T15:01:49.797 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:49.797 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.797 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.797 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:49.797 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223442+0000", 2024-09-21T15:01:49.797 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:49.797 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.797 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.797 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:49.797 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223446+0000", 2024-09-21T15:01:49.798 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2549999999999999e-06 2024-09-21T15:01:49.798 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.798 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.798 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:49.798 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223454+0000", 2024-09-21T15:01:49.798 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8199999999999997e-06 2024-09-21T15:01:49.798 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.798 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.798 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:49.798 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223460+0000", 2024-09-21T15:01:49.798 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6330000000000001e-06 2024-09-21T15:01:49.798 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.798 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.799 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:49.799 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223542+0000", 2024-09-21T15:01:49.799 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1929999999999997e-05 2024-09-21T15:01:49.799 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.799 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.799 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:49.799 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223552+0000", 2024-09-21T15:01:49.799 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0627e-05 2024-09-21T15:01:49.799 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.799 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.799 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:49.799 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.254634+0000", 2024-09-21T15:01:49.799 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.031081972 2024-09-21T15:01:49.799 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.800 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.800 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:49.800 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.254796+0000", 2024-09-21T15:01:49.800 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000161879 2024-09-21T15:01:49.800 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.800 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.800 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:49.800 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.254991+0000", 2024-09-21T15:01:49.800 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00019526499999999999 2024-09-21T15:01:49.800 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:49.800 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:49.800 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:49.800 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.801 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.801 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139785 3.11 3:8abbb109:::rbd_data.113664780c85.0000000000000070:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:49.801 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.223531+0000", 2024-09-21T15:01:49.801 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 47.569421624999997, 2024-09-21T15:01:49.801 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0187143510000001, 2024-09-21T15:01:49.801 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:49.801 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:49.801 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:49.801 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:49.801 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:49.801 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139785 2024-09-21T15:01:49.801 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.801 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:49.801 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.801 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:49.802 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223531+0000", 2024-09-21T15:01:49.802 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:49.802 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.802 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.802 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:49.802 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223531+0000", 2024-09-21T15:01:49.802 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:49.802 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.802 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.802 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:49.802 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223531+0000", 2024-09-21T15:01:49.802 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:49.802 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.802 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.802 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:49.803 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223534+0000", 2024-09-21T15:01:49.803 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2650000000000001e-06 2024-09-21T15:01:49.803 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.803 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.803 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:49.803 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223541+0000", 2024-09-21T15:01:49.803 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8990000000000004e-06 2024-09-21T15:01:49.803 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.803 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.803 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:49.803 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223546+0000", 2024-09-21T15:01:49.803 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3399999999999997e-06 2024-09-21T15:01:49.803 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.803 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.803 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:49.804 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223724+0000", 2024-09-21T15:01:49.804 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017745799999999999 2024-09-21T15:01:49.804 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.804 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.804 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:49.804 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223735+0000", 2024-09-21T15:01:49.804 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1610999999999999e-05 2024-09-21T15:01:49.804 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.804 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.804 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:49.804 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242113+0000", 2024-09-21T15:01:49.804 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.018377879 2024-09-21T15:01:49.804 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.804 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.804 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:49.805 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242246+0000", 2024-09-21T15:01:49.805 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013231500000000001 2024-09-21T15:01:49.805 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:49.805 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:49.805 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:49.805 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.805 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.805 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139790 3.11 3:8a18c7d8:::rbd_data.113664780c85.000000000000008d:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:49.805 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.223578+0000", 2024-09-21T15:01:49.805 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 47.569374629999999, 2024-09-21T15:01:49.805 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0187879719999999, 2024-09-21T15:01:49.805 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:49.805 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:49.805 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:49.805 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:49.806 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:49.806 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139790 2024-09-21T15:01:49.806 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.806 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:49.806 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.806 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:49.806 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223578+0000", 2024-09-21T15:01:49.806 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:49.806 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.806 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.806 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:49.806 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223578+0000", 2024-09-21T15:01:49.806 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:49.806 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.807 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.807 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:49.807 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223578+0000", 2024-09-21T15:01:49.807 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:49.807 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.807 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.807 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:49.807 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223581+0000", 2024-09-21T15:01:49.807 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6879999999999999e-06 2024-09-21T15:01:49.807 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.807 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.807 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:49.807 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223586+0000", 2024-09-21T15:01:49.807 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7080000000000002e-06 2024-09-21T15:01:49.807 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.808 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.808 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:49.808 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223591+0000", 2024-09-21T15:01:49.808 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5639999999999998e-06 2024-09-21T15:01:49.808 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.808 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.808 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:49.808 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223761+0000", 2024-09-21T15:01:49.808 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000170213 2024-09-21T15:01:49.808 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.808 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.808 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:49.808 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223769+0000", 2024-09-21T15:01:49.808 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7049999999999999e-06 2024-09-21T15:01:49.809 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.809 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.809 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:49.809 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242269+0000", 2024-09-21T15:01:49.809 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.018500288 2024-09-21T15:01:49.809 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.809 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.809 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:49.809 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242366+0000", 2024-09-21T15:01:49.809 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.7126000000000005e-05 2024-09-21T15:01:49.809 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:49.809 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:49.809 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:49.809 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.809 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.810 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139792 3.16 3:6ce5caf3:::rbd_data.113664780c85.000000000000009b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:49.810 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.223595+0000", 2024-09-21T15:01:49.810 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 47.569357676999999, 2024-09-21T15:01:49.810 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0317517899999999, 2024-09-21T15:01:49.810 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:49.810 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:49.810 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:49.810 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:49.810 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:49.810 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139792 2024-09-21T15:01:49.810 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.810 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:49.810 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.810 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:49.810 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223595+0000", 2024-09-21T15:01:49.811 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:49.811 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.811 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.811 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:49.811 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223595+0000", 2024-09-21T15:01:49.811 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:49.811 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.811 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.811 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:49.811 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223595+0000", 2024-09-21T15:01:49.811 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:49.811 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.811 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.811 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:49.812 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223598+0000", 2024-09-21T15:01:49.812 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4620000000000001e-06 2024-09-21T15:01:49.812 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.812 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.812 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:49.812 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223604+0000", 2024-09-21T15:01:49.812 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6879999999999996e-06 2024-09-21T15:01:49.812 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.812 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.812 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:49.812 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223609+0000", 2024-09-21T15:01:49.812 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9490000000000004e-06 2024-09-21T15:01:49.812 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.812 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.812 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:49.812 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224067+0000", 2024-09-21T15:01:49.813 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00045780700000000002 2024-09-21T15:01:49.813 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.813 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.813 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:49.813 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224075+0000", 2024-09-21T15:01:49.813 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6610000000000006e-06 2024-09-21T15:01:49.813 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.813 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.813 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:49.813 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255141+0000", 2024-09-21T15:01:49.813 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0310657480000001 2024-09-21T15:01:49.813 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.813 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.813 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:49.813 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255347+0000", 2024-09-21T15:01:49.813 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000205837 2024-09-21T15:01:49.814 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:49.814 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:49.814 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:49.814 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.814 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.814 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139794 3.11 3:8cf3a2f4:::rbd_data.113664780c85.00000000000000a9:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:49.814 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.224011+0000", 2024-09-21T15:01:49.814 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 47.568941649000003, 2024-09-21T15:01:49.814 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.018507791, 2024-09-21T15:01:49.814 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:49.814 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:49.814 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:49.814 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:49.814 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:49.814 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139794 2024-09-21T15:01:49.815 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.815 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:49.815 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.815 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:49.815 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224011+0000", 2024-09-21T15:01:49.815 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:49.815 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.815 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.815 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:49.815 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224011+0000", 2024-09-21T15:01:49.815 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:49.815 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.815 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.815 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:49.815 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224010+0000", 2024-09-21T15:01:49.816 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:49.816 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.816 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.816 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:49.816 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224016+0000", 2024-09-21T15:01:49.816 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6550000000000001e-06 2024-09-21T15:01:49.816 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.816 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.816 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:49.816 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224028+0000", 2024-09-21T15:01:49.816 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2422999999999999e-05 2024-09-21T15:01:49.816 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.816 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.816 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:49.816 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224036+0000", 2024-09-21T15:01:49.816 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7749999999999993e-06 2024-09-21T15:01:49.817 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.817 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.817 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:49.817 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225176+0000", 2024-09-21T15:01:49.817 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0011401740000000001 2024-09-21T15:01:49.817 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.817 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.817 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:49.817 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225187+0000", 2024-09-21T15:01:49.817 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0951999999999999e-05 2024-09-21T15:01:49.817 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.817 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.817 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:49.818 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242392+0000", 2024-09-21T15:01:49.818 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.017204905 2024-09-21T15:01:49.818 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.818 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.818 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:49.818 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242519+0000", 2024-09-21T15:01:49.818 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012680499999999999 2024-09-21T15:01:49.818 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:49.818 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:49.818 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:49.818 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.818 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.818 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139800 3.16 3:68b050ed:::rbd_data.113664780c85.00000000000000d3:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:49.818 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.224579+0000", 2024-09-21T15:01:49.818 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 47.568373889999997, 2024-09-21T15:01:49.819 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.030932637, 2024-09-21T15:01:49.819 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:49.819 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:49.819 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:49.819 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:49.819 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:49.819 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139800 2024-09-21T15:01:49.819 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.819 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:49.819 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.819 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:49.819 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224579+0000", 2024-09-21T15:01:49.819 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:49.819 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.819 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.820 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:49.820 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224579+0000", 2024-09-21T15:01:49.820 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:49.820 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.820 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.820 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:49.820 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224578+0000", 2024-09-21T15:01:49.820 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:49.820 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.820 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.820 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:49.820 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224582+0000", 2024-09-21T15:01:49.820 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0029999999999999e-06 2024-09-21T15:01:49.820 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.820 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.820 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:49.821 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224588+0000", 2024-09-21T15:01:49.821 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8120000000000002e-06 2024-09-21T15:01:49.821 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.821 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.821 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:49.821 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224593+0000", 2024-09-21T15:01:49.821 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7469999999999996e-06 2024-09-21T15:01:49.821 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.821 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.821 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:49.821 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224648+0000", 2024-09-21T15:01:49.821 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4693e-05 2024-09-21T15:01:49.821 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.821 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.821 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:49.822 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224656+0000", 2024-09-21T15:01:49.822 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8429999999999993e-06 2024-09-21T15:01:49.822 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.822 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.822 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:49.822 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255382+0000", 2024-09-21T15:01:49.822 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0307266070000001 2024-09-21T15:01:49.822 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.822 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.822 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:49.822 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255512+0000", 2024-09-21T15:01:49.822 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000129367 2024-09-21T15:01:49.822 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:49.822 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:49.822 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:49.823 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.823 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.823 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139804 3.11 3:88f9ddce:::rbd_data.113664780c85.00000000000000ef:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:49.823 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.224632+0000", 2024-09-21T15:01:49.823 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 47.568320909000001, 2024-09-21T15:01:49.823 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.018021302, 2024-09-21T15:01:49.823 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:49.823 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:49.823 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:49.823 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:49.823 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:49.823 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139804 2024-09-21T15:01:49.823 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.824 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:49.824 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.824 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:49.824 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224632+0000", 2024-09-21T15:01:49.824 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:49.824 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.824 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.824 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:49.824 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224632+0000", 2024-09-21T15:01:49.824 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:49.824 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.824 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.824 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:49.824 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224632+0000", 2024-09-21T15:01:49.825 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:49.825 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.825 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.825 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:49.825 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224635+0000", 2024-09-21T15:01:49.825 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2710000000000001e-06 2024-09-21T15:01:49.825 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.825 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.825 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:49.825 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224640+0000", 2024-09-21T15:01:49.825 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6779999999999997e-06 2024-09-21T15:01:49.825 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.825 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.825 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:49.825 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224645+0000", 2024-09-21T15:01:49.826 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9339999999999997e-06 2024-09-21T15:01:49.826 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.826 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.826 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:49.826 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225307+0000", 2024-09-21T15:01:49.826 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00066201699999999999 2024-09-21T15:01:49.826 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.826 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.826 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:49.826 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225312+0000", 2024-09-21T15:01:49.826 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9799999999999998e-06 2024-09-21T15:01:49.826 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.826 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.826 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:49.827 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242548+0000", 2024-09-21T15:01:49.827 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.017235603 2024-09-21T15:01:49.827 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.827 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.827 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:49.827 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242653+0000", 2024-09-21T15:01:49.827 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010522 2024-09-21T15:01:49.827 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:49.827 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:49.827 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:49.827 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.827 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.827 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139809 3.11 3:884da946:::rbd_data.113664780c85.0000000000000118:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:49.828 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.224667+0000", 2024-09-21T15:01:49.828 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 47.568285865, 2024-09-21T15:01:49.828 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0181101960000001, 2024-09-21T15:01:49.828 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:49.828 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:49.828 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:49.828 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:49.828 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:49.828 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139809 2024-09-21T15:01:49.828 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.828 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:49.828 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.828 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:49.828 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224667+0000", 2024-09-21T15:01:49.828 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:49.828 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.829 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.829 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:49.829 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224667+0000", 2024-09-21T15:01:49.829 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:49.829 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.829 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.829 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:49.829 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224667+0000", 2024-09-21T15:01:49.829 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:49.829 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.829 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.829 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:49.829 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224669+0000", 2024-09-21T15:01:49.829 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.452e-06 2024-09-21T15:01:49.830 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.830 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.830 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:49.830 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224675+0000", 2024-09-21T15:01:49.830 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.818e-06 2024-09-21T15:01:49.830 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.830 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.830 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:49.830 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224680+0000", 2024-09-21T15:01:49.830 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6319999999999999e-06 2024-09-21T15:01:49.830 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.830 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.830 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:49.830 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225400+0000", 2024-09-21T15:01:49.831 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00072012699999999999 2024-09-21T15:01:49.831 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.831 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.831 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:49.831 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225409+0000", 2024-09-21T15:01:49.831 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1039999999999996e-06 2024-09-21T15:01:49.831 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.831 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.831 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:49.831 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242675+0000", 2024-09-21T15:01:49.831 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0172663319999999 2024-09-21T15:01:49.831 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.831 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.831 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:49.831 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242777+0000", 2024-09-21T15:01:49.832 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010197899999999999 2024-09-21T15:01:49.832 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:49.832 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:49.832 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:49.832 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.832 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.832 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139868 3.16 3:69f82090:::rbd_data.113664780c85.00000000000000f1:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:49.832 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.225982+0000", 2024-09-21T15:01:49.832 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 47.566970365000003, 2024-09-21T15:01:49.832 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0296791729999999, 2024-09-21T15:01:49.832 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:49.832 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:49.832 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:49.832 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:49.832 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:49.833 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139868 2024-09-21T15:01:49.833 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.833 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:49.833 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.833 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:49.833 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225982+0000", 2024-09-21T15:01:49.833 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:49.833 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.833 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.833 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:49.833 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225982+0000", 2024-09-21T15:01:49.833 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:49.833 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.833 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.834 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:49.834 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225982+0000", 2024-09-21T15:01:49.834 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:49.834 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.834 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.834 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:49.834 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225985+0000", 2024-09-21T15:01:49.834 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.74e-06 2024-09-21T15:01:49.834 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.834 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.834 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:49.834 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225990+0000", 2024-09-21T15:01:49.834 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7039999999999999e-06 2024-09-21T15:01:49.834 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.834 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.835 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:49.835 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225995+0000", 2024-09-21T15:01:49.835 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7249999999999997e-06 2024-09-21T15:01:49.835 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.835 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.835 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:49.835 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226299+0000", 2024-09-21T15:01:49.835 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00030383199999999997 2024-09-21T15:01:49.835 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.835 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.835 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:49.835 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226308+0000", 2024-09-21T15:01:49.835 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.9740000000000008e-06 2024-09-21T15:01:49.835 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.836 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.836 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:49.836 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255539+0000", 2024-09-21T15:01:49.836 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0292305960000001 2024-09-21T15:01:49.836 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.836 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.836 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:49.836 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255662+0000", 2024-09-21T15:01:49.836 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012307400000000001 2024-09-21T15:01:49.836 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:49.836 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:49.836 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:49.836 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.836 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.836 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139869 3.16 3:6b6c90a4:::rbd_data.113664780c85.00000000000000fe:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:49.837 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.226006+0000", 2024-09-21T15:01:49.837 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 47.566946749000003, 2024-09-21T15:01:49.837 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.029790518, 2024-09-21T15:01:49.837 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:49.837 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:49.837 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:49.837 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:49.837 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:49.837 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139869 2024-09-21T15:01:49.837 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.837 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:49.837 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.837 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:49.837 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226006+0000", 2024-09-21T15:01:49.837 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:49.838 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.838 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.838 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:49.838 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226006+0000", 2024-09-21T15:01:49.838 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:49.838 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.838 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.838 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:49.838 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226006+0000", 2024-09-21T15:01:49.838 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:49.838 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.838 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.838 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:49.839 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226008+0000", 2024-09-21T15:01:49.839 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5569999999999998e-06 2024-09-21T15:01:49.839 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.839 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.839 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:49.839 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226014+0000", 2024-09-21T15:01:49.839 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2079999999999997e-06 2024-09-21T15:01:49.839 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.839 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.839 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:49.839 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226020+0000", 2024-09-21T15:01:49.839 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7429999999999999e-06 2024-09-21T15:01:49.839 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.839 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.839 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:49.840 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226330+0000", 2024-09-21T15:01:49.840 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00030963300000000002 2024-09-21T15:01:49.840 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.840 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.840 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:49.840 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226336+0000", 2024-09-21T15:01:49.840 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6340000000000003e-06 2024-09-21T15:01:49.840 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.840 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.840 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:49.840 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255687+0000", 2024-09-21T15:01:49.840 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0293507150000001 2024-09-21T15:01:49.840 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.840 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.840 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:49.841 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255797+0000", 2024-09-21T15:01:49.841 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000109471 2024-09-21T15:01:49.841 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:49.841 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:49.841 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:49.841 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.841 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.841 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139884 3.16 3:69f10324:::rbd_data.113664780c85.0000000000000161:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:49.841 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.227573+0000", 2024-09-21T15:01:49.841 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 47.565380107999999, 2024-09-21T15:01:49.841 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0283568759999999, 2024-09-21T15:01:49.841 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:49.841 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:49.841 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:49.841 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:49.842 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:49.842 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139884 2024-09-21T15:01:49.842 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.842 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:49.842 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.842 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:49.842 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227573+0000", 2024-09-21T15:01:49.842 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:49.842 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.842 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.842 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:49.842 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227573+0000", 2024-09-21T15:01:49.842 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:49.842 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.843 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.843 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:49.843 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227572+0000", 2024-09-21T15:01:49.843 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:49.843 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.843 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.843 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:49.843 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227575+0000", 2024-09-21T15:01:49.843 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7769999999999998e-06 2024-09-21T15:01:49.843 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.843 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.843 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:49.843 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227581+0000", 2024-09-21T15:01:49.843 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7359999999999998e-06 2024-09-21T15:01:49.844 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.844 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.844 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:49.844 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227585+0000", 2024-09-21T15:01:49.844 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5090000000000004e-06 2024-09-21T15:01:49.844 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.844 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.844 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:49.844 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227753+0000", 2024-09-21T15:01:49.844 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00016761599999999999 2024-09-21T15:01:49.844 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.844 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.844 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:49.844 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227761+0000", 2024-09-21T15:01:49.844 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0579999999999998e-06 2024-09-21T15:01:49.844 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.845 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.845 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:49.845 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255828+0000", 2024-09-21T15:01:49.845 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0280671349999999 2024-09-21T15:01:49.845 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.845 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.845 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:49.845 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255930+0000", 2024-09-21T15:01:49.845 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000101372 2024-09-21T15:01:49.845 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:49.845 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:49.845 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:49.845 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.845 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.845 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139913 3.16 3:6f81b46d:::rbd_data.113664780c85.0000000000000074:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:49.846 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.227685+0000", 2024-09-21T15:01:49.846 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 47.565267929000001, 2024-09-21T15:01:49.846 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0283644750000001, 2024-09-21T15:01:49.846 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:49.846 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:49.846 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:49.846 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:49.846 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:49.846 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139913 2024-09-21T15:01:49.846 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.846 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:49.846 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.846 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:49.846 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227685+0000", 2024-09-21T15:01:49.847 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:49.847 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.847 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.847 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:49.847 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227685+0000", 2024-09-21T15:01:49.847 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:49.847 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.847 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.847 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:49.847 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227685+0000", 2024-09-21T15:01:49.847 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:49.847 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.847 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.847 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:49.848 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227687+0000", 2024-09-21T15:01:49.848 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2929999999999999e-06 2024-09-21T15:01:49.848 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.848 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.848 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:49.848 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227693+0000", 2024-09-21T15:01:49.848 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1070000000000001e-06 2024-09-21T15:01:49.848 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.848 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.848 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:49.848 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227698+0000", 2024-09-21T15:01:49.848 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9339999999999997e-06 2024-09-21T15:01:49.848 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.848 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.848 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:49.849 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227784+0000", 2024-09-21T15:01:49.849 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6096999999999999e-05 2024-09-21T15:01:49.849 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.849 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.849 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:49.849 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227791+0000", 2024-09-21T15:01:49.849 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9090000000000003e-06 2024-09-21T15:01:49.849 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.849 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.849 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:49.849 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255955+0000", 2024-09-21T15:01:49.849 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0281643579999999 2024-09-21T15:01:49.849 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.850 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.850 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:49.850 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256049+0000", 2024-09-21T15:01:49.850 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4047000000000003e-05 2024-09-21T15:01:49.850 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:49.850 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:49.850 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:49.850 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.850 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.850 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139988 3.11 3:88a29fa4:::rbd_data.113664780c85.00000000000000bd:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:49.850 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.229772+0000", 2024-09-21T15:01:49.850 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 47.563180430000003, 2024-09-21T15:01:49.850 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.017967668, 2024-09-21T15:01:49.851 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:49.851 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:49.851 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:49.851 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:49.851 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:49.851 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139988 2024-09-21T15:01:49.851 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.851 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:49.851 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.851 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:49.851 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229772+0000", 2024-09-21T15:01:49.851 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:49.851 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.851 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.852 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:49.852 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229772+0000", 2024-09-21T15:01:49.852 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:49.852 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.852 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.852 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:49.852 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229772+0000", 2024-09-21T15:01:49.852 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:49.852 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.852 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.852 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:49.852 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229776+0000", 2024-09-21T15:01:49.852 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6890000000000002e-06 2024-09-21T15:01:49.853 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.853 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.853 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:49.853 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229783+0000", 2024-09-21T15:01:49.853 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2280000000000001e-06 2024-09-21T15:01:49.853 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.853 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.853 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:49.853 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229789+0000", 2024-09-21T15:01:49.853 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1700000000000002e-06 2024-09-21T15:01:49.853 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.853 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.853 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:49.853 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.230106+0000", 2024-09-21T15:01:49.853 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00031700800000000003 2024-09-21T15:01:49.854 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.854 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.854 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:49.854 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.230115+0000", 2024-09-21T15:01:49.854 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4190000000000008e-06 2024-09-21T15:01:49.854 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.854 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.854 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:49.854 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242935+0000", 2024-09-21T15:01:49.854 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0128191449999999 2024-09-21T15:01:49.854 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.854 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.854 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for missing object", 2024-09-21T15:01:49.854 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.243078+0000", 2024-09-21T15:01:49.854 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000142984 2024-09-21T15:01:49.855 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.855 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.855 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:49.855 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.247659+0000", 2024-09-21T15:01:49.855 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0045813030000000001 2024-09-21T15:01:49.855 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.855 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.855 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:49.855 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.247740+0000", 2024-09-21T15:01:49.855 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1216000000000003e-05 2024-09-21T15:01:49.855 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:49.855 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:49.855 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:49.855 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.855 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.856 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139994 3.16 3:6b9df8d1:::rbd_data.113664780c85.00000000000000e7:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:49.856 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.231370+0000", 2024-09-21T15:01:49.856 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 47.561582352000002, 2024-09-21T15:01:49.856 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.024795576, 2024-09-21T15:01:49.856 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:49.856 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:49.856 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:49.856 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:49.856 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:49.856 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139994 2024-09-21T15:01:49.856 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.856 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:49.856 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.856 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:49.856 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231370+0000", 2024-09-21T15:01:49.857 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:49.857 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.857 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.857 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:49.857 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231370+0000", 2024-09-21T15:01:49.857 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:49.857 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.857 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.857 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:49.857 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231369+0000", 2024-09-21T15:01:49.857 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:49.857 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.857 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.857 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:49.858 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231376+0000", 2024-09-21T15:01:49.858 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5509999999999996e-06 2024-09-21T15:01:49.858 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.858 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.858 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:49.858 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231387+0000", 2024-09-21T15:01:49.858 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.061e-05 2024-09-21T15:01:49.858 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.858 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.858 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:49.858 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231394+0000", 2024-09-21T15:01:49.858 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3799999999999996e-06 2024-09-21T15:01:49.858 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.859 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.859 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:49.859 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231459+0000", 2024-09-21T15:01:49.859 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4716999999999997e-05 2024-09-21T15:01:49.859 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.859 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.859 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:49.859 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231468+0000", 2024-09-21T15:01:49.859 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1479999999999995e-06 2024-09-21T15:01:49.859 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.859 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.859 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:49.859 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256074+0000", 2024-09-21T15:01:49.859 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0246064180000001 2024-09-21T15:01:49.860 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.860 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.860 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:49.860 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256166+0000", 2024-09-21T15:01:49.860 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1862999999999998e-05 2024-09-21T15:01:49.860 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:49.860 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:49.860 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:49.860 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.860 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.860 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140012 3.16 3:6bd6dfb6:::rbd_data.113664780c85.0000000000000165:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:49.860 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.231890+0000", 2024-09-21T15:01:49.860 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 47.561062593000003, 2024-09-21T15:01:49.860 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.024408881, 2024-09-21T15:01:49.861 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:49.861 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:49.861 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:49.861 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:49.861 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:49.861 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140012 2024-09-21T15:01:49.861 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.861 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:49.861 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.861 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:49.861 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231890+0000", 2024-09-21T15:01:49.861 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:49.861 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.861 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.862 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:49.862 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231890+0000", 2024-09-21T15:01:49.862 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:49.862 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.862 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.862 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:49.862 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231889+0000", 2024-09-21T15:01:49.862 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:49.862 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.862 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.862 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:49.862 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231895+0000", 2024-09-21T15:01:49.862 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9569999999999999e-06 2024-09-21T15:01:49.862 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.863 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.863 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:49.863 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231910+0000", 2024-09-21T15:01:49.863 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4549000000000001e-05 2024-09-21T15:01:49.863 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.863 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.863 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:49.863 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231917+0000", 2024-09-21T15:01:49.863 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5730000000000002e-06 2024-09-21T15:01:49.863 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.863 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.863 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:49.863 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231988+0000", 2024-09-21T15:01:49.863 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0852999999999994e-05 2024-09-21T15:01:49.864 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.864 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.864 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:49.864 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.232000+0000", 2024-09-21T15:01:49.864 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2159e-05 2024-09-21T15:01:49.864 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.864 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.864 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:49.864 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256192+0000", 2024-09-21T15:01:49.864 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.024192073 2024-09-21T15:01:49.864 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.864 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.864 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:49.864 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256299+0000", 2024-09-21T15:01:49.865 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000106733 2024-09-21T15:01:49.865 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:49.865 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:49.865 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:49.865 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.865 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.865 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140055 3.16 3:69bb9803:::rbd_data.113664780c85.00000000000000da:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:49.865 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.233922+0000", 2024-09-21T15:01:49.865 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 47.559031064000003, 2024-09-21T15:01:49.865 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.022511768, 2024-09-21T15:01:49.865 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:49.865 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:49.865 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:49.865 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:49.866 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:49.866 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140055 2024-09-21T15:01:49.866 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.866 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:49.866 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.866 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:49.866 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233922+0000", 2024-09-21T15:01:49.866 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:49.866 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.866 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.866 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:49.866 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233922+0000", 2024-09-21T15:01:49.866 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:49.866 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.866 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.867 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:49.867 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233921+0000", 2024-09-21T15:01:49.867 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:49.867 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.867 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.867 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:49.867 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233927+0000", 2024-09-21T15:01:49.867 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0360000000000003e-06 2024-09-21T15:01:49.867 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.867 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.867 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:49.867 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233937+0000", 2024-09-21T15:01:49.867 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6439999999999995e-06 2024-09-21T15:01:49.867 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.867 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.868 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:49.868 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233944+0000", 2024-09-21T15:01:49.868 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7710000000000006e-06 2024-09-21T15:01:49.868 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.868 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.868 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:49.868 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233996+0000", 2024-09-21T15:01:49.868 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1597000000000001e-05 2024-09-21T15:01:49.868 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.868 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.868 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:49.868 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.234005+0000", 2024-09-21T15:01:49.868 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5960000000000004e-06 2024-09-21T15:01:49.868 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.869 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.869 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:49.869 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256332+0000", 2024-09-21T15:01:49.869 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0223271949999999 2024-09-21T15:01:49.869 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.869 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.869 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:49.869 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256434+0000", 2024-09-21T15:01:49.869 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010180599999999999 2024-09-21T15:01:49.869 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:49.869 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:49.869 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:49.869 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.869 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.869 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140143 3.16 3:6d3ed596:::rbd_data.113664780c85.0000000000000184:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:49.869 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.239496+0000", 2024-09-21T15:01:49.870 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 47.553456535000002, 2024-09-21T15:01:49.870 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.017076576, 2024-09-21T15:01:49.870 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:49.870 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:49.870 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:49.870 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:49.870 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:49.870 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140143 2024-09-21T15:01:49.870 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.870 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:49.870 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.870 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:49.870 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239496+0000", 2024-09-21T15:01:49.870 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:49.871 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.871 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.871 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:49.871 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239496+0000", 2024-09-21T15:01:49.871 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:49.871 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.871 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.871 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:49.871 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239496+0000", 2024-09-21T15:01:49.871 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:49.871 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.871 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.871 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:49.871 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239498+0000", 2024-09-21T15:01:49.872 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.509e-06 2024-09-21T15:01:49.872 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.872 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.872 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:49.872 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239504+0000", 2024-09-21T15:01:49.872 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6869999999999999e-06 2024-09-21T15:01:49.872 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.872 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.872 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:49.872 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239509+0000", 2024-09-21T15:01:49.872 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8860000000000003e-06 2024-09-21T15:01:49.872 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.872 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.873 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:49.873 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239572+0000", 2024-09-21T15:01:49.873 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2908000000000001e-05 2024-09-21T15:01:49.873 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.873 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.873 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:49.873 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239580+0000", 2024-09-21T15:01:49.873 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4270000000000008e-06 2024-09-21T15:01:49.873 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.873 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.873 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:49.873 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256457+0000", 2024-09-21T15:01:49.873 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0168770979999999 2024-09-21T15:01:49.873 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.873 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.873 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:49.874 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256573+0000", 2024-09-21T15:01:49.874 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00011538400000000001 2024-09-21T15:01:49.874 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:49.874 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:49.874 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:49.874 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.874 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.874 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140163 3.16 3:694e82eb:::rbd_data.113664780c85.0000000000000052:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:49.874 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.240334+0000", 2024-09-21T15:01:49.874 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 47.552618823000003, 2024-09-21T15:01:49.874 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.016365537, 2024-09-21T15:01:49.874 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:49.874 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:49.874 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:49.875 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:49.875 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:49.875 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140163 2024-09-21T15:01:49.875 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.875 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:49.875 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.875 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:49.875 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240334+0000", 2024-09-21T15:01:49.875 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:49.875 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.875 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.875 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:49.875 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240334+0000", 2024-09-21T15:01:49.875 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:49.875 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.876 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.876 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:49.876 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240334+0000", 2024-09-21T15:01:49.876 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:49.876 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.876 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.876 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:49.876 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240337+0000", 2024-09-21T15:01:49.876 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.089e-06 2024-09-21T15:01:49.876 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.876 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.876 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:49.876 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240343+0000", 2024-09-21T15:01:49.876 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4420000000000003e-06 2024-09-21T15:01:49.876 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.877 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.877 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:49.877 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240348+0000", 2024-09-21T15:01:49.877 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1039999999999998e-06 2024-09-21T15:01:49.877 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.877 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.877 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:49.877 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240451+0000", 2024-09-21T15:01:49.877 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000102996 2024-09-21T15:01:49.877 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.877 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.877 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:49.877 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240460+0000", 2024-09-21T15:01:49.877 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5369999999999997e-06 2024-09-21T15:01:49.877 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.878 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.878 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:49.878 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256600+0000", 2024-09-21T15:01:49.878 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.016139884 2024-09-21T15:01:49.878 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.878 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.878 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:49.878 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256700+0000", 2024-09-21T15:01:49.878 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9844000000000002e-05 2024-09-21T15:01:49.878 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:49.878 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:49.878 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:49.878 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.878 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.879 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140164 3.16 3:68bfdc57:::rbd_data.113664780c85.0000000000000053:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:49.879 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.240358+0000", 2024-09-21T15:01:49.879 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 47.552595128999997, 2024-09-21T15:01:49.879 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.01646094, 2024-09-21T15:01:49.879 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:49.879 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:49.879 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:49.879 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:49.879 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:49.879 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140164 2024-09-21T15:01:49.879 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.879 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:49.879 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.879 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:49.879 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240358+0000", 2024-09-21T15:01:49.880 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:49.880 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.880 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.880 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:49.880 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240358+0000", 2024-09-21T15:01:49.880 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:49.880 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.880 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.880 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:49.880 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240357+0000", 2024-09-21T15:01:49.880 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:49.880 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.880 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.880 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:49.880 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240360+0000", 2024-09-21T15:01:49.881 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7089999999999998e-06 2024-09-21T15:01:49.881 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.881 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.881 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:49.881 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240368+0000", 2024-09-21T15:01:49.881 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7270000000000007e-06 2024-09-21T15:01:49.881 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.881 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.881 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:49.881 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240373+0000", 2024-09-21T15:01:49.881 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4060000000000004e-06 2024-09-21T15:01:49.881 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.881 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.881 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:49.881 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240473+0000", 2024-09-21T15:01:49.881 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000100132 2024-09-21T15:01:49.882 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.882 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.882 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:49.882 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240476+0000", 2024-09-21T15:01:49.882 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.332e-06 2024-09-21T15:01:49.882 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.882 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.882 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:49.882 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256724+0000", 2024-09-21T15:01:49.882 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0162471930000001 2024-09-21T15:01:49.882 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.882 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.882 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:49.883 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256819+0000", 2024-09-21T15:01:49.883 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5007999999999996e-05 2024-09-21T15:01:49.883 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:49.883 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:49.883 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:49.883 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.883 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.883 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140188 3.16 3:6f6217d4:::rbd_data.113664780c85.00000000000000fb:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:49.883 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.240978+0000", 2024-09-21T15:01:49.883 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 47.551974469999998, 2024-09-21T15:01:49.883 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.015983809, 2024-09-21T15:01:49.883 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:49.883 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:49.883 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:49.884 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:49.884 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:49.884 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140188 2024-09-21T15:01:49.884 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.884 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:49.884 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.884 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:49.884 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240978+0000", 2024-09-21T15:01:49.884 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:49.884 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.884 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.884 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:49.884 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240978+0000", 2024-09-21T15:01:49.884 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:49.884 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.884 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.885 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:49.885 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240978+0000", 2024-09-21T15:01:49.885 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:49.885 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.885 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.885 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:49.885 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240981+0000", 2024-09-21T15:01:49.885 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1559999999999999e-06 2024-09-21T15:01:49.885 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.885 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.885 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:49.885 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240987+0000", 2024-09-21T15:01:49.885 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.229e-06 2024-09-21T15:01:49.885 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.885 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.886 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:49.886 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240992+0000", 2024-09-21T15:01:49.886 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9710000000000003e-06 2024-09-21T15:01:49.886 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.886 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.886 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:49.886 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.241361+0000", 2024-09-21T15:01:49.886 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000369236 2024-09-21T15:01:49.886 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.886 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.886 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:49.886 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.241369+0000", 2024-09-21T15:01:49.886 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6199999999999999e-06 2024-09-21T15:01:49.886 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.886 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.886 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:49.887 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256844+0000", 2024-09-21T15:01:49.887 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0154751609999999 2024-09-21T15:01:49.887 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:49.887 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:49.887 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:49.887 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256962+0000", 2024-09-21T15:01:49.887 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000117964 2024-09-21T15:01:49.887 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:49.887 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:49.887 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:49.887 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:49.887 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:49.887 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:49.889 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T15:01:49.889 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:01:49.885+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:01:49.989 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:01:49.984+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:01:50.007 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:01:50.007 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:01:50.007 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T15:01:50.008 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:01:50.017 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T15:01:50.090 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:01:50.084+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:01:50.123 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:01:50.123 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:01:50.123 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T15:01:50.123 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T15:01:50.123 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:01:50.133 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T15:01:50.188 INFO:teuthology.orchestra.run.smithi032.stderr:nodeep-scrub is unset 2024-09-21T15:01:50.190 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:01:50.184+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:01:50.250 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:01:50.250 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T15:01:50.250 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T15:01:50.250 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T15:01:50.250 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.250 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45854 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:50.250 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159802+0000", 2024-09-21T15:01:50.250 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 160.08540293199999, 2024-09-21T15:01:50.251 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068358938999999994, 2024-09-21T15:01:50.251 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:50.251 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:50.251 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:50.251 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:50.251 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:50.251 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45854 2024-09-21T15:01:50.251 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.251 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:50.251 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.251 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:50.251 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159802+0000", 2024-09-21T15:01:50.251 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:50.252 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.252 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.252 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:50.252 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159802+0000", 2024-09-21T15:01:50.252 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:50.252 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.252 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.252 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:50.252 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159801+0000", 2024-09-21T15:01:50.252 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:50.252 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.252 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.252 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:50.252 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159804+0000", 2024-09-21T15:01:50.252 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0970000000000002e-06 2024-09-21T15:01:50.253 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.253 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.253 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:50.253 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159823+0000", 2024-09-21T15:01:50.253 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8895000000000001e-05 2024-09-21T15:01:50.253 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.253 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.253 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:50.253 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159837+0000", 2024-09-21T15:01:50.253 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3247999999999999e-05 2024-09-21T15:01:50.253 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.253 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.253 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:50.253 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228151+0000", 2024-09-21T15:01:50.253 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068314642999999994 2024-09-21T15:01:50.254 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.254 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.254 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:50.254 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228161+0000", 2024-09-21T15:01:50.254 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4250000000000004e-06 2024-09-21T15:01:50.254 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:50.254 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:50.254 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:50.254 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.254 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.254 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45985 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:50.254 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159897+0000", 2024-09-21T15:01:50.254 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 160.08530743, 2024-09-21T15:01:50.255 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068345664, 2024-09-21T15:01:50.255 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:50.255 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:50.255 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:50.255 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:50.255 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:50.255 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45985 2024-09-21T15:01:50.255 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.255 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:50.255 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.255 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:50.255 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159897+0000", 2024-09-21T15:01:50.255 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:50.255 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.256 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.256 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:50.256 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159897+0000", 2024-09-21T15:01:50.256 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:50.256 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.256 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.256 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:50.256 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159897+0000", 2024-09-21T15:01:50.256 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:50.256 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.256 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.256 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:50.256 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159900+0000", 2024-09-21T15:01:50.256 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.954e-06 2024-09-21T15:01:50.257 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.257 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.257 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:50.257 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159913+0000", 2024-09-21T15:01:50.257 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3106e-05 2024-09-21T15:01:50.257 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.257 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.257 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:50.257 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159918+0000", 2024-09-21T15:01:50.257 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5199999999999997e-06 2024-09-21T15:01:50.257 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.257 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.257 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:50.257 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228236+0000", 2024-09-21T15:01:50.258 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068317040999999995 2024-09-21T15:01:50.258 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.258 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.258 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:50.258 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228243+0000", 2024-09-21T15:01:50.258 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3880000000000002e-06 2024-09-21T15:01:50.258 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:50.258 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:50.258 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:50.258 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.258 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.258 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45986 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:50.258 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159924+0000", 2024-09-21T15:01:50.259 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 160.085281135, 2024-09-21T15:01:50.259 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068364362999999997, 2024-09-21T15:01:50.259 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:50.259 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:50.259 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:50.259 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:50.259 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:50.259 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45986 2024-09-21T15:01:50.259 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.259 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:50.259 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.259 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:50.259 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159924+0000", 2024-09-21T15:01:50.259 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:50.259 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.260 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.260 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:50.260 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159924+0000", 2024-09-21T15:01:50.260 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:50.260 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.260 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.260 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:50.260 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159923+0000", 2024-09-21T15:01:50.260 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:50.260 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.260 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.260 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:50.260 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159926+0000", 2024-09-21T15:01:50.260 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0510000000000001e-06 2024-09-21T15:01:50.260 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.261 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.261 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:50.261 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159932+0000", 2024-09-21T15:01:50.261 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4160000000000001e-06 2024-09-21T15:01:50.261 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.261 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.261 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:50.261 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159938+0000", 2024-09-21T15:01:50.261 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.383e-06 2024-09-21T15:01:50.261 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.261 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.261 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:50.261 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228275+0000", 2024-09-21T15:01:50.261 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068337007000000005 2024-09-21T15:01:50.262 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.262 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.262 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:50.262 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228288+0000", 2024-09-21T15:01:50.262 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3052e-05 2024-09-21T15:01:50.262 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:50.262 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:50.262 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:50.262 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.262 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.262 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45987 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:50.262 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159944+0000", 2024-09-21T15:01:50.262 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 160.08526091100001, 2024-09-21T15:01:50.262 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068387232000000006, 2024-09-21T15:01:50.263 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:50.263 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:50.263 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:50.263 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:50.263 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:50.263 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45987 2024-09-21T15:01:50.263 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.263 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:50.263 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.263 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:50.263 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159944+0000", 2024-09-21T15:01:50.263 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:50.263 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.263 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.263 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:50.264 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159944+0000", 2024-09-21T15:01:50.264 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:50.264 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.264 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.264 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:50.264 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159943+0000", 2024-09-21T15:01:50.264 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:50.264 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.264 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.264 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:50.264 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159946+0000", 2024-09-21T15:01:50.264 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6309999999999999e-06 2024-09-21T15:01:50.264 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.264 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.265 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:50.265 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159952+0000", 2024-09-21T15:01:50.265 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6540000000000004e-06 2024-09-21T15:01:50.265 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.265 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.265 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:50.265 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159957+0000", 2024-09-21T15:01:50.265 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9280000000000001e-06 2024-09-21T15:01:50.265 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.265 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.265 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:50.265 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228322+0000", 2024-09-21T15:01:50.265 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068365067000000002 2024-09-21T15:01:50.265 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.266 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.266 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:50.266 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228331+0000", 2024-09-21T15:01:50.266 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3270000000000007e-06 2024-09-21T15:01:50.266 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:50.266 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:50.266 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:50.266 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.266 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.266 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:97672 3.4 3:26046d29:::rbd_data.1136bfc79a70.0000000000000001:head [write 0~4718592 [fadvise_sequential+fadvise_nocache],write 5242880~131072 [fadvise_sequential+fadvise_nocache],write 5439488~147456 [fadvise_sequential+fadvise_nocache],write 5652480~229376 [fadvise_sequential+fadvise_nocache],write 5947392~147456 [fadvise_sequential+fadvise_nocache],write 6160384~229376 [fadvise_sequential+fadvise_nocache],write 6455296~147456 [fadvise_sequential+fadvise_nocache],write 6668288~229376 [fadvise_sequential+fadvise_nocache],write 6963200~147456 [fadvise_sequential+fadvise_nocache],write 7176192~147456 [fadvise_sequential+fadvise_nocache],write 7340032~65536 [fadvise_sequential+fadvise_nocache],write 7471104~147456 [fadvise_sequential+fadvise_nocache],write 7684096~147456 [fadvise_sequential+fadvise_nocache],write 7864320~49152 [fadvise_sequential+fadvise_nocache],write 7979008~147456 [fadvise_sequential+fadvise_nocache],write 8192000~147456 [fadvise_sequential+fadvise_nocache],write 8388608~32768 [fadvise_sequential+fadvise_nocache],write 8519680~114688 [fadvise_sequential+fadvise_nocache],write 8732672~114688 [fadvise_sequential+fadvise_nocache],write 8912896~16384 [fadvise_sequential+fadvise_nocache],write 9109504~32768 [fadvise_sequential+fadvise_nocache],write 9322496~32768 [fadvise_sequential+fadvise_nocache],write 9617408~32768 [fadvise_sequential+fadvise_nocache],write 9830400~32768 [fadvise_sequential+fadvise_nocache],write 9961472~98304 [fadvise_sequential+fadvise_nocache],write 10076160~196608 [fadvise_sequential+fadvise_nocache],write 10289152~278528 [fadvise_sequential+fadvise_nocache],write 10584064~196608 [fadvise_sequential+fadvise_nocache],write 10797056~196608 [fadvise_sequential+fadvise_nocache],write 11010048~65536 [fadvise_sequential+fadvise_nocache],write 11091968~196608 [fadvise_sequential+fadvise_nocache],write 11304960~196608 [fadvise_sequential+fadvise_nocache],write 11517952~65536 [fadvise_sequential+fadvise_nocache],write 11616256~98304 [fadvise_sequential+fadvise_nocache],write 11730944~65536 [fadvise_sequential+fadvise_nocache],write 11829248~98304 [fadvise_sequential+fadvise_nocache],write 11943936~65536 [fadvise_sequential+fadvise_nocache],write 12042240~49152 [fadvise_sequential+fadvise_nocache],write 12124160~98304 [fadvise_sequential+fadvise_nocache],write 12238848~65536 [fadvise_sequential+fadvise_nocache],write 12337152~98304 [fadvise_sequential+fadvise_nocache],write 12451840~65536 [fadvise_sequential+fadvise_nocache],write 12550144~32768 [fadvise_sequential+fadvise_nocache],write 12632064~98304 [fadvise_sequential+fadvise_nocache],write 12845056~98304 [fadvise_sequential+fadvise_nocache],write 13058048~49152 [fadvise_sequential+fadvise_nocache],write 13139968~98304 [fadvise_sequential+fadvise_nocache],write 13352960~98304 [fadvise_sequential+fadvise_nocache],write 13565952~65536 [fadvise_sequential+fadvise_nocache],write 13647872~98304 [fadvise_sequential+fadvise_nocache],write 13860864~98304 [fadvise_sequential+fadvise_nocache],write 14073856~81920 [fadvise_sequential+fadvise_nocache],write 15269888~81920 [fadvise_sequential+fadvise_nocache],write 15482880~81920 [fadvise_sequential+fadvise_nocache],write 15695872~32768 [fadvise_sequential+fadvise_nocache],write 15777792~81920 [fadvise_sequential+fadvise_nocache],write 15990784~81920 [fadvise_sequential+fadvise_nocache],write 16203776~49152 [fadvise_sequential+fadvise_nocache],write 16285696~81920 [fadvise_sequential+fadvise_nocache],write 16498688~81920 [fadvise_sequential+fadvise_nocache],write 16711680~65536 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:01:50.266 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.298669+0000", 2024-09-21T15:01:50.266 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 104.946535845, 2024-09-21T15:01:50.266 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.10282358900000001, 2024-09-21T15:01:50.267 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:50.267 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:50.267 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:50.267 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:50.267 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:50.267 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97672 2024-09-21T15:01:50.267 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.267 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:50.267 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.267 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:50.267 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.298669+0000", 2024-09-21T15:01:50.267 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:50.267 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.267 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.268 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:50.268 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.298669+0000", 2024-09-21T15:01:50.268 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:50.268 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.268 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.268 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:50.268 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.298667+0000", 2024-09-21T15:01:50.268 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:01:50.268 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.268 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.268 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:50.268 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.327458+0000", 2024-09-21T15:01:50.268 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.028790754000000002 2024-09-21T15:01:50.268 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.268 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.269 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:50.269 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.327492+0000", 2024-09-21T15:01:50.269 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3806999999999997e-05 2024-09-21T15:01:50.269 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.269 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.269 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:50.269 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.327507+0000", 2024-09-21T15:01:50.269 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5112e-05 2024-09-21T15:01:50.269 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.269 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.269 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:50.269 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.327597+0000", 2024-09-21T15:01:50.269 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9938999999999994e-05 2024-09-21T15:01:50.269 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.269 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.269 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:50.270 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.327927+0000", 2024-09-21T15:01:50.270 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000329766 2024-09-21T15:01:50.270 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.270 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.270 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T15:01:50.270 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.330735+0000", 2024-09-21T15:01:50.270 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0028086869999999998 2024-09-21T15:01:50.270 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.270 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.270 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:01:50.270 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.378299+0000", 2024-09-21T15:01:50.270 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.047563495999999997 2024-09-21T15:01:50.270 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.270 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.270 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:01:50.271 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.401248+0000", 2024-09-21T15:01:50.271 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.022949311 2024-09-21T15:01:50.271 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.271 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.271 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:01:50.271 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.401446+0000", 2024-09-21T15:01:50.271 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019831199999999999 2024-09-21T15:01:50.271 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.271 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.271 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:50.271 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.401492+0000", 2024-09-21T15:01:50.271 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6109000000000002e-05 2024-09-21T15:01:50.271 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:50.271 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:50.272 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:50.272 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.272 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.272 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139769 3.1e 3:793dcf44:::rbd_data.113664780c85.0000000000000000:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:50.272 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.221432+0000", 2024-09-21T15:01:50.272 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 48.023772975999997, 2024-09-21T15:01:50.272 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.035363362, 2024-09-21T15:01:50.272 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:50.272 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:50.272 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:50.272 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:50.272 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:50.272 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139769 2024-09-21T15:01:50.272 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.273 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:50.273 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.273 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:50.273 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221432+0000", 2024-09-21T15:01:50.273 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:50.273 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.273 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.273 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:50.273 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221432+0000", 2024-09-21T15:01:50.273 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:50.273 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.273 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.273 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:50.273 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221430+0000", 2024-09-21T15:01:50.273 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:01:50.274 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.274 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.274 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:50.274 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221438+0000", 2024-09-21T15:01:50.274 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2630000000000003e-06 2024-09-21T15:01:50.274 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.274 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.274 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:50.274 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221458+0000", 2024-09-21T15:01:50.274 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0071000000000001e-05 2024-09-21T15:01:50.274 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.274 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.274 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:50.274 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221467+0000", 2024-09-21T15:01:50.274 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.816e-06 2024-09-21T15:01:50.275 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.275 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.275 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:50.275 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221572+0000", 2024-09-21T15:01:50.275 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000105166 2024-09-21T15:01:50.275 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.275 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.275 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:50.275 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221589+0000", 2024-09-21T15:01:50.275 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7391e-05 2024-09-21T15:01:50.275 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.275 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.275 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:50.275 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.255741+0000", 2024-09-21T15:01:50.275 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0341519690000001 2024-09-21T15:01:50.276 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.276 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.276 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:50.276 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.256353+0000", 2024-09-21T15:01:50.276 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000611544 2024-09-21T15:01:50.276 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.276 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.276 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:50.276 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.256795+0000", 2024-09-21T15:01:50.276 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00044208700000000002 2024-09-21T15:01:50.276 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:50.276 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:50.276 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:50.276 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.277 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.277 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139773 3.1e 3:7fa3054f:::rbd_data.113664780c85.000000000000001c:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:50.277 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.221517+0000", 2024-09-21T15:01:50.277 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 48.023687932000001, 2024-09-21T15:01:50.277 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0367423140000001, 2024-09-21T15:01:50.277 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:50.277 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:50.277 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:50.277 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:50.277 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:50.277 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139773 2024-09-21T15:01:50.277 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.277 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:50.277 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.277 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:50.277 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221517+0000", 2024-09-21T15:01:50.278 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:50.278 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.278 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.278 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:50.278 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221517+0000", 2024-09-21T15:01:50.278 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:50.278 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.278 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.278 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:50.278 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221516+0000", 2024-09-21T15:01:50.278 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:50.278 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.278 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.279 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:50.279 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221520+0000", 2024-09-21T15:01:50.279 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1030000000000004e-06 2024-09-21T15:01:50.279 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.279 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.279 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:50.279 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221566+0000", 2024-09-21T15:01:50.279 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5512000000000001e-05 2024-09-21T15:01:50.279 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.279 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.279 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:50.279 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221575+0000", 2024-09-21T15:01:50.279 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9630000000000004e-06 2024-09-21T15:01:50.279 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.280 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.280 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:50.280 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221650+0000", 2024-09-21T15:01:50.280 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5531999999999994e-05 2024-09-21T15:01:50.280 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.280 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.280 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:50.280 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221661+0000", 2024-09-21T15:01:50.280 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0878e-05 2024-09-21T15:01:50.280 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.280 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.280 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:50.280 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.257513+0000", 2024-09-21T15:01:50.280 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0358513089999999 2024-09-21T15:01:50.280 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.281 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.281 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:50.281 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.257885+0000", 2024-09-21T15:01:50.281 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00037274299999999998 2024-09-21T15:01:50.281 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.281 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.281 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:50.281 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.258259+0000", 2024-09-21T15:01:50.281 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000373829 2024-09-21T15:01:50.281 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:50.281 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:50.281 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:50.281 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.281 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.281 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139789 3.1e 3:7c803367:::rbd_data.113664780c85.000000000000008c:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:50.282 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.221656+0000", 2024-09-21T15:01:50.282 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 48.023548300999998, 2024-09-21T15:01:50.282 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.037139595, 2024-09-21T15:01:50.282 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:50.282 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:50.282 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:50.282 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:50.282 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:50.282 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139789 2024-09-21T15:01:50.282 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.282 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:50.282 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.282 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:50.283 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221656+0000", 2024-09-21T15:01:50.283 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:50.283 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.283 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.283 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:50.283 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221656+0000", 2024-09-21T15:01:50.283 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:50.283 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.283 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.283 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:50.283 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221656+0000", 2024-09-21T15:01:50.283 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:50.283 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.283 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.283 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:50.284 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221660+0000", 2024-09-21T15:01:50.284 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0369999999999996e-06 2024-09-21T15:01:50.284 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.284 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.284 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:50.284 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221668+0000", 2024-09-21T15:01:50.284 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3440000000000001e-06 2024-09-21T15:01:50.284 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.284 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.284 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:50.284 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221674+0000", 2024-09-21T15:01:50.284 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0920000000000003e-06 2024-09-21T15:01:50.284 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.284 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.285 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:50.285 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221787+0000", 2024-09-21T15:01:50.285 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000112289 2024-09-21T15:01:50.285 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.285 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.285 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:50.285 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221798+0000", 2024-09-21T15:01:50.285 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1419e-05 2024-09-21T15:01:50.285 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.285 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.285 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:50.285 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.258642+0000", 2024-09-21T15:01:50.285 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.036843851 2024-09-21T15:01:50.285 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.285 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.286 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:50.286 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.258796+0000", 2024-09-21T15:01:50.286 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015414199999999999 2024-09-21T15:01:50.286 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:50.286 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:50.286 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:50.286 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.286 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.286 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139832 3.1e 3:7b675eeb:::rbd_data.113664780c85.00000000000001b3:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:50.286 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.223976+0000", 2024-09-21T15:01:50.286 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 48.021228567999998, 2024-09-21T15:01:50.286 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0350267980000001, 2024-09-21T15:01:50.286 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:50.287 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:50.287 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:50.287 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:50.287 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:50.287 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139832 2024-09-21T15:01:50.287 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.287 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:50.287 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.287 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:50.287 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223976+0000", 2024-09-21T15:01:50.287 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:50.287 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.287 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.287 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:50.287 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223976+0000", 2024-09-21T15:01:50.288 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:50.288 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.288 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.288 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:50.288 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223976+0000", 2024-09-21T15:01:50.288 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:50.288 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.288 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.288 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:50.288 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223981+0000", 2024-09-21T15:01:50.288 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1560000000000003e-06 2024-09-21T15:01:50.288 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.288 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.288 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:50.289 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223999+0000", 2024-09-21T15:01:50.289 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8808e-05 2024-09-21T15:01:50.289 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.289 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.289 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:50.289 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224008+0000", 2024-09-21T15:01:50.289 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7269999999999996e-06 2024-09-21T15:01:50.289 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.289 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.289 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:50.289 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224208+0000", 2024-09-21T15:01:50.289 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000199614 2024-09-21T15:01:50.289 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.289 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.289 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:50.289 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224218+0000", 2024-09-21T15:01:50.290 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0209e-05 2024-09-21T15:01:50.290 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.290 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.290 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:50.290 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.258839+0000", 2024-09-21T15:01:50.290 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.034621083 2024-09-21T15:01:50.290 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.290 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.290 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:50.290 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.259003+0000", 2024-09-21T15:01:50.290 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016379 2024-09-21T15:01:50.290 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:50.290 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:50.290 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:50.291 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.291 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.291 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139900 3.1e 3:7b440264:::rbd_data.113664780c85.0000000000000013:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:50.291 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.225748+0000", 2024-09-21T15:01:50.291 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 48.019456566999999, 2024-09-21T15:01:50.291 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0339966009999999, 2024-09-21T15:01:50.291 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:50.291 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:50.291 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:50.291 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:50.291 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:50.291 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139900 2024-09-21T15:01:50.291 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.291 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:50.291 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.292 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:50.292 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.225748+0000", 2024-09-21T15:01:50.292 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:50.292 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.292 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.292 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:50.292 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.225748+0000", 2024-09-21T15:01:50.292 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:50.292 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.292 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.292 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:50.292 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.225746+0000", 2024-09-21T15:01:50.292 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:50.292 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.292 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.293 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:50.293 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.225757+0000", 2024-09-21T15:01:50.293 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.066e-05 2024-09-21T15:01:50.293 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.293 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.293 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:50.293 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.225775+0000", 2024-09-21T15:01:50.293 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7631000000000001e-05 2024-09-21T15:01:50.293 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.293 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.293 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:50.293 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.225788+0000", 2024-09-21T15:01:50.293 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3046e-05 2024-09-21T15:01:50.293 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.294 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.294 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:50.294 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.225881+0000", 2024-09-21T15:01:50.294 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2987999999999998e-05 2024-09-21T15:01:50.294 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.294 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.294 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:50.294 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.225893+0000", 2024-09-21T15:01:50.294 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2398999999999999e-05 2024-09-21T15:01:50.294 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.294 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.294 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:50.294 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.259207+0000", 2024-09-21T15:01:50.295 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.033313696 2024-09-21T15:01:50.295 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.295 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.295 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:50.295 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.259452+0000", 2024-09-21T15:01:50.295 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00024495700000000002 2024-09-21T15:01:50.295 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.295 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.295 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:50.295 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.259745+0000", 2024-09-21T15:01:50.295 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000292868 2024-09-21T15:01:50.295 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:50.295 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:50.295 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:50.295 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.296 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.296 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140008 3.1e 3:7cd8b476:::rbd_data.113664780c85.0000000000000149:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:50.296 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.229555+0000", 2024-09-21T15:01:50.296 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 48.015649703000001, 2024-09-21T15:01:50.296 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0304914199999999, 2024-09-21T15:01:50.296 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:50.296 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:50.296 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:50.296 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:50.296 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:50.296 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140008 2024-09-21T15:01:50.296 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.296 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:50.296 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.296 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:50.296 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.229555+0000", 2024-09-21T15:01:50.297 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:50.297 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.297 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.297 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:50.297 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.229555+0000", 2024-09-21T15:01:50.297 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:50.297 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.297 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.297 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:50.297 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.229554+0000", 2024-09-21T15:01:50.297 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:50.297 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.297 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.297 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:50.297 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.229559+0000", 2024-09-21T15:01:50.298 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1059999999999999e-06 2024-09-21T15:01:50.298 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.298 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.298 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:50.298 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.229570+0000", 2024-09-21T15:01:50.298 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0465000000000001e-05 2024-09-21T15:01:50.298 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.298 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.298 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:50.298 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.229581+0000", 2024-09-21T15:01:50.298 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0958000000000001e-05 2024-09-21T15:01:50.298 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.298 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.298 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:50.299 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.229660+0000", 2024-09-21T15:01:50.299 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9103999999999993e-05 2024-09-21T15:01:50.299 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.299 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.299 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:50.299 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.229675+0000", 2024-09-21T15:01:50.299 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5622999999999999e-05 2024-09-21T15:01:50.299 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.299 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.299 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:50.299 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.259816+0000", 2024-09-21T15:01:50.299 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.030140595 2024-09-21T15:01:50.299 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.300 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.300 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:50.300 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.260046+0000", 2024-09-21T15:01:50.300 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00023045300000000001 2024-09-21T15:01:50.300 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:50.300 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:50.300 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:50.300 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.300 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.300 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140037 3.1e 3:7a4585d9:::rbd_data.113664780c85.000000000000005c:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:50.300 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.230439+0000", 2024-09-21T15:01:50.300 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 48.014765214999997, 2024-09-21T15:01:50.300 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.030277613, 2024-09-21T15:01:50.300 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:50.301 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:50.301 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:50.301 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:50.301 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:50.301 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140037 2024-09-21T15:01:50.301 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.301 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:50.301 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.301 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:50.301 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.230439+0000", 2024-09-21T15:01:50.301 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:50.301 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.301 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.301 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:50.302 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.230439+0000", 2024-09-21T15:01:50.302 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:50.302 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.302 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.302 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:50.302 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.230438+0000", 2024-09-21T15:01:50.302 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:50.302 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.302 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.302 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:50.302 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.230447+0000", 2024-09-21T15:01:50.302 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6270000000000006e-06 2024-09-21T15:01:50.302 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.303 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.303 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:50.303 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.230464+0000", 2024-09-21T15:01:50.303 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7394e-05 2024-09-21T15:01:50.303 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.303 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.303 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:50.303 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.230472+0000", 2024-09-21T15:01:50.303 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0360000000000006e-06 2024-09-21T15:01:50.303 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.303 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.303 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:50.303 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.230567+0000", 2024-09-21T15:01:50.303 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4648000000000006e-05 2024-09-21T15:01:50.304 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.304 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.304 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:50.304 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.230581+0000", 2024-09-21T15:01:50.304 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3799e-05 2024-09-21T15:01:50.304 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.304 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.304 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:50.304 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.260095+0000", 2024-09-21T15:01:50.304 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.029514153 2024-09-21T15:01:50.304 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.304 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.304 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:50.304 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.260356+0000", 2024-09-21T15:01:50.304 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00026118400000000003 2024-09-21T15:01:50.305 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.305 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.305 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:50.305 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.260717+0000", 2024-09-21T15:01:50.305 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00036124799999999999 2024-09-21T15:01:50.305 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:50.305 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:50.305 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:50.305 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.305 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.305 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140061 3.1e 3:7fdd08f8:::rbd_data.113664780c85.0000000000000104:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:50.305 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.232026+0000", 2024-09-21T15:01:50.305 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 48.013178885000002, 2024-09-21T15:01:50.305 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0290620450000001, 2024-09-21T15:01:50.306 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:50.306 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:50.306 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:50.306 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:50.306 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:50.306 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140061 2024-09-21T15:01:50.306 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.306 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:50.306 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.306 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:50.306 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232026+0000", 2024-09-21T15:01:50.306 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:50.306 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.306 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.306 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:50.307 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232026+0000", 2024-09-21T15:01:50.307 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:50.307 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.307 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.307 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:50.307 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232024+0000", 2024-09-21T15:01:50.307 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:01:50.307 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.307 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.307 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:50.307 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232034+0000", 2024-09-21T15:01:50.307 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0421000000000001e-05 2024-09-21T15:01:50.307 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.307 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.307 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:50.308 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232058+0000", 2024-09-21T15:01:50.308 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3944e-05 2024-09-21T15:01:50.308 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.308 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.308 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:50.308 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232071+0000", 2024-09-21T15:01:50.308 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2816999999999999e-05 2024-09-21T15:01:50.308 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.308 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.308 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:50.308 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232180+0000", 2024-09-21T15:01:50.308 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000109323 2024-09-21T15:01:50.308 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.308 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.309 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:50.309 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232199+0000", 2024-09-21T15:01:50.309 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8606000000000001e-05 2024-09-21T15:01:50.309 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.309 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.309 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:50.309 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.260821+0000", 2024-09-21T15:01:50.309 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0286222270000001 2024-09-21T15:01:50.309 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.309 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.309 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:50.309 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.261088+0000", 2024-09-21T15:01:50.309 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00026663799999999998 2024-09-21T15:01:50.309 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:50.309 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:50.309 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:50.310 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.310 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.310 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140090 3.1e 3:79f99893:::rbd_data.113664780c85.000000000000000b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:50.310 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.233668+0000", 2024-09-21T15:01:50.310 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 48.011537019999999, 2024-09-21T15:01:50.310 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0281047409999999, 2024-09-21T15:01:50.310 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:50.310 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:50.310 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:50.310 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:50.310 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:50.310 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140090 2024-09-21T15:01:50.310 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.310 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:50.310 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.311 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:50.311 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233668+0000", 2024-09-21T15:01:50.311 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:50.311 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.311 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.311 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:50.311 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233668+0000", 2024-09-21T15:01:50.311 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:50.311 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.311 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.311 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:50.311 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233666+0000", 2024-09-21T15:01:50.311 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:50.311 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.311 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.312 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:50.312 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233673+0000", 2024-09-21T15:01:50.312 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1840000000000002e-06 2024-09-21T15:01:50.312 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.312 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.312 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:50.312 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233689+0000", 2024-09-21T15:01:50.312 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5366000000000002e-05 2024-09-21T15:01:50.312 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.312 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.312 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:50.312 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233697+0000", 2024-09-21T15:01:50.312 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7679999999999997e-06 2024-09-21T15:01:50.313 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.313 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.313 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:50.313 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233772+0000", 2024-09-21T15:01:50.313 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4832000000000005e-05 2024-09-21T15:01:50.313 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.313 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.313 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:50.313 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233790+0000", 2024-09-21T15:01:50.313 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7297999999999999e-05 2024-09-21T15:01:50.313 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.313 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.313 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:50.313 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.261145+0000", 2024-09-21T15:01:50.314 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.027355381 2024-09-21T15:01:50.314 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.314 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.314 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:50.314 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.261466+0000", 2024-09-21T15:01:50.314 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00032105300000000002 2024-09-21T15:01:50.314 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.314 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.314 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:50.314 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.261772+0000", 2024-09-21T15:01:50.314 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00030635100000000001 2024-09-21T15:01:50.314 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:50.314 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:50.314 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:50.314 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.315 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.315 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140128 3.1e 3:78baf82e:::rbd_data.113664780c85.0000000000000115:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:50.315 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.235029+0000", 2024-09-21T15:01:50.315 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 48.010175766000003, 2024-09-21T15:01:50.315 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.02698942, 2024-09-21T15:01:50.315 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:50.315 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:50.315 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:50.315 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:50.315 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:50.315 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140128 2024-09-21T15:01:50.315 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.315 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:50.315 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.315 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:50.315 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235029+0000", 2024-09-21T15:01:50.316 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:50.316 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.316 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.316 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:50.316 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235029+0000", 2024-09-21T15:01:50.316 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:50.316 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.316 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.316 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:50.316 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235028+0000", 2024-09-21T15:01:50.316 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:50.316 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.316 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.316 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:50.316 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235032+0000", 2024-09-21T15:01:50.317 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0579999999999999e-06 2024-09-21T15:01:50.317 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.317 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.317 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:50.317 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235040+0000", 2024-09-21T15:01:50.317 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2099999999999993e-06 2024-09-21T15:01:50.317 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.317 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.317 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:50.317 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235047+0000", 2024-09-21T15:01:50.317 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1099999999999999e-06 2024-09-21T15:01:50.317 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.317 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.317 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:50.317 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235123+0000", 2024-09-21T15:01:50.318 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6835999999999994e-05 2024-09-21T15:01:50.318 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.318 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.318 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:50.318 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235136+0000", 2024-09-21T15:01:50.318 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2962000000000001e-05 2024-09-21T15:01:50.318 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.318 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.318 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:50.318 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.261827+0000", 2024-09-21T15:01:50.318 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0266906149999999 2024-09-21T15:01:50.318 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.318 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.318 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:50.318 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.262018+0000", 2024-09-21T15:01:50.319 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019132400000000001 2024-09-21T15:01:50.319 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:50.319 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:50.319 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:50.319 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.319 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.319 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140217 3.1e 3:793dcf44:::rbd_data.113664780c85.0000000000000000:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:50.319 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.235497+0000", 2024-09-21T15:01:50.319 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 48.009707466999998, 2024-09-21T15:01:50.319 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0269053020000001, 2024-09-21T15:01:50.319 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:50.319 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:50.319 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:50.319 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:50.320 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:50.320 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140217 2024-09-21T15:01:50.320 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.320 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:50.320 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.320 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:50.320 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235497+0000", 2024-09-21T15:01:50.320 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:50.320 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.320 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.320 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:50.320 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235497+0000", 2024-09-21T15:01:50.320 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:50.320 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.320 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.321 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:50.321 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235496+0000", 2024-09-21T15:01:50.321 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:50.321 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.321 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.321 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:50.321 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235503+0000", 2024-09-21T15:01:50.321 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8460000000000002e-06 2024-09-21T15:01:50.321 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.321 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.321 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:50.321 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235514+0000", 2024-09-21T15:01:50.321 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0576e-05 2024-09-21T15:01:50.321 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.321 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.322 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:50.322 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235522+0000", 2024-09-21T15:01:50.322 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2640000000000008e-06 2024-09-21T15:01:50.322 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.322 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.322 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:50.322 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235579+0000", 2024-09-21T15:01:50.322 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6842999999999998e-05 2024-09-21T15:01:50.322 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.322 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.322 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:50.322 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235590+0000", 2024-09-21T15:01:50.322 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1194e-05 2024-09-21T15:01:50.322 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.322 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.323 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:50.323 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.262119+0000", 2024-09-21T15:01:50.323 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0265294540000001 2024-09-21T15:01:50.323 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.323 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.323 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:50.323 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.262241+0000", 2024-09-21T15:01:50.323 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012182 2024-09-21T15:01:50.323 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.323 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.323 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:50.323 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.262403+0000", 2024-09-21T15:01:50.323 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000161442 2024-09-21T15:01:50.323 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:50.323 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:50.324 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:50.324 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.324 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.324 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140221 3.1e 3:7fa3054f:::rbd_data.113664780c85.000000000000001c:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:50.324 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.235592+0000", 2024-09-21T15:01:50.324 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 48.009612605999997, 2024-09-21T15:01:50.324 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.027322012, 2024-09-21T15:01:50.324 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:50.324 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:50.324 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:50.324 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:50.324 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:50.324 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140221 2024-09-21T15:01:50.324 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.324 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:50.325 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.325 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:50.325 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235592+0000", 2024-09-21T15:01:50.325 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:50.325 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.325 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.325 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:50.325 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235592+0000", 2024-09-21T15:01:50.325 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:50.325 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.325 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.325 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:50.325 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235591+0000", 2024-09-21T15:01:50.325 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:50.325 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.325 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.326 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:50.326 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235596+0000", 2024-09-21T15:01:50.326 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4209999999999997e-06 2024-09-21T15:01:50.326 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.326 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.326 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:50.326 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235603+0000", 2024-09-21T15:01:50.326 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.909e-06 2024-09-21T15:01:50.326 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.326 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.326 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:50.326 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235609+0000", 2024-09-21T15:01:50.326 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3539999999999999e-06 2024-09-21T15:01:50.326 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.326 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.327 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:50.327 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236013+0000", 2024-09-21T15:01:50.327 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00040399799999999998 2024-09-21T15:01:50.327 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.327 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.327 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:50.327 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236020+0000", 2024-09-21T15:01:50.327 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6699999999999994e-06 2024-09-21T15:01:50.327 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.327 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.327 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:50.327 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.262452+0000", 2024-09-21T15:01:50.327 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.02643145 2024-09-21T15:01:50.327 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.328 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.328 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:50.328 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.262624+0000", 2024-09-21T15:01:50.328 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000172206 2024-09-21T15:01:50.328 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.328 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.328 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:50.328 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.262914+0000", 2024-09-21T15:01:50.328 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00028994600000000002 2024-09-21T15:01:50.328 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:50.328 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:50.328 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:50.328 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.328 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.328 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140237 3.1e 3:7c803367:::rbd_data.113664780c85.000000000000008c:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:50.329 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.237113+0000", 2024-09-21T15:01:50.329 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 48.008092099000002, 2024-09-21T15:01:50.329 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0260336299999999, 2024-09-21T15:01:50.329 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:50.329 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:50.329 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:50.329 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:50.329 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:50.329 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140237 2024-09-21T15:01:50.329 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.329 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:50.329 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.329 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:50.329 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237113+0000", 2024-09-21T15:01:50.329 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:50.330 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.330 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.330 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:50.330 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237113+0000", 2024-09-21T15:01:50.330 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:50.330 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.330 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.330 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:50.330 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237112+0000", 2024-09-21T15:01:50.330 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:50.330 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.330 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.330 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:50.330 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237118+0000", 2024-09-21T15:01:50.330 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8930000000000002e-06 2024-09-21T15:01:50.331 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.331 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.331 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:50.331 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237126+0000", 2024-09-21T15:01:50.331 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6729999999999998e-06 2024-09-21T15:01:50.331 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.331 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.331 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:50.331 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237135+0000", 2024-09-21T15:01:50.331 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0929999999999992e-06 2024-09-21T15:01:50.331 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.331 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.331 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:50.331 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237205+0000", 2024-09-21T15:01:50.332 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9697999999999996e-05 2024-09-21T15:01:50.332 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.332 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.332 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:50.332 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237218+0000", 2024-09-21T15:01:50.332 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2524000000000001e-05 2024-09-21T15:01:50.332 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.332 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.332 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:50.332 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.262965+0000", 2024-09-21T15:01:50.332 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0257478630000001 2024-09-21T15:01:50.332 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.332 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.332 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:50.332 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.263146+0000", 2024-09-21T15:01:50.333 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018074599999999999 2024-09-21T15:01:50.333 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:50.333 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:50.333 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:50.333 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.333 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.333 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140280 3.1e 3:7b675eeb:::rbd_data.113664780c85.00000000000001b3:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:50.333 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.237477+0000", 2024-09-21T15:01:50.333 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 48.007727836999997, 2024-09-21T15:01:50.333 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.026437329, 2024-09-21T15:01:50.333 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:50.333 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:50.333 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:50.333 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:50.334 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:50.334 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140280 2024-09-21T15:01:50.334 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.334 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:50.334 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.334 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:50.334 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237477+0000", 2024-09-21T15:01:50.334 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:50.334 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.334 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.334 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:50.334 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237477+0000", 2024-09-21T15:01:50.334 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:50.334 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.335 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.335 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:50.335 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237476+0000", 2024-09-21T15:01:50.335 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:50.335 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.335 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.335 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:50.335 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237480+0000", 2024-09-21T15:01:50.335 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7570000000000002e-06 2024-09-21T15:01:50.335 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.335 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.335 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:50.335 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237490+0000", 2024-09-21T15:01:50.336 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3170000000000008e-06 2024-09-21T15:01:50.336 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.336 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.336 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:50.336 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237496+0000", 2024-09-21T15:01:50.336 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7150000000000001e-06 2024-09-21T15:01:50.336 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.336 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.336 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:50.336 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237559+0000", 2024-09-21T15:01:50.336 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2844999999999999e-05 2024-09-21T15:01:50.336 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.336 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.336 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:50.337 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237570+0000", 2024-09-21T15:01:50.337 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1253000000000001e-05 2024-09-21T15:01:50.337 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.337 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.337 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:50.337 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.263203+0000", 2024-09-21T15:01:50.337 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0256330339999999 2024-09-21T15:01:50.337 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.337 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.337 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:50.337 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.263914+0000", 2024-09-21T15:01:50.337 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00071080899999999996 2024-09-21T15:01:50.337 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:50.337 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:50.337 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:50.338 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.338 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.338 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140348 3.1e 3:7b440264:::rbd_data.113664780c85.0000000000000013:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:50.338 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.241762+0000", 2024-09-21T15:01:50.338 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 48.003442565999997, 2024-09-21T15:01:50.338 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0221355759999999, 2024-09-21T15:01:50.338 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:50.338 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:50.338 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:50.338 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:50.338 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:50.338 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140348 2024-09-21T15:01:50.338 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.338 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:50.338 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.339 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:50.339 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.241762+0000", 2024-09-21T15:01:50.339 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:50.339 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.339 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.339 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:50.339 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.241762+0000", 2024-09-21T15:01:50.339 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:50.339 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.339 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.339 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:50.339 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.241761+0000", 2024-09-21T15:01:50.339 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:50.339 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.339 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.340 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:50.340 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.241767+0000", 2024-09-21T15:01:50.340 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4219999999999997e-06 2024-09-21T15:01:50.340 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.340 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.340 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:50.340 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.241783+0000", 2024-09-21T15:01:50.340 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5959000000000001e-05 2024-09-21T15:01:50.340 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.340 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.340 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:50.340 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.241793+0000", 2024-09-21T15:01:50.340 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3999999999999998e-06 2024-09-21T15:01:50.340 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.340 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.341 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:50.341 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.241894+0000", 2024-09-21T15:01:50.341 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000100976 2024-09-21T15:01:50.341 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.341 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.341 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:50.341 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.241907+0000", 2024-09-21T15:01:50.341 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3281e-05 2024-09-21T15:01:50.341 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.341 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.341 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:50.341 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.263430+0000", 2024-09-21T15:01:50.341 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0215229379999999 2024-09-21T15:01:50.341 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.341 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.341 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:50.342 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.263604+0000", 2024-09-21T15:01:50.342 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017368700000000001 2024-09-21T15:01:50.342 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.342 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.342 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:50.342 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.263898+0000", 2024-09-21T15:01:50.342 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00029406899999999999 2024-09-21T15:01:50.342 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:50.342 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:50.342 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:50.342 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:50.342 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:50.342 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:01:50.344 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T15:01:50.344 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:01:50.340+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:01:50.445 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:01:50.441+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:01:50.502 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:01:50.502 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T15:01:50.502 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.502 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:170180 3.1 3.43355e21 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e217)", 2024-09-21T15:01:50.502 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:49.417090+0000", 2024-09-21T15:01:50.502 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 1.083547152, 2024-09-21T15:01:50.503 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0835712310000001, 2024-09-21T15:01:50.503 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:50.503 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "waiting for peered", 2024-09-21T15:01:50.503 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:50.503 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:50.503 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:50.503 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 170180 2024-09-21T15:01:50.503 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.503 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:50.503 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.503 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:50.503 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:49.417090+0000", 2024-09-21T15:01:50.503 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:50.504 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.504 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.504 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:50.504 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:49.417090+0000", 2024-09-21T15:01:50.504 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:50.504 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.504 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.504 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:50.504 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:49.417089+0000", 2024-09-21T15:01:50.504 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:50.504 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.504 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.504 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:50.505 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:49.417094+0000", 2024-09-21T15:01:50.505 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9060000000000001e-06 2024-09-21T15:01:50.505 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.505 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.505 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:50.505 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:49.417109+0000", 2024-09-21T15:01:50.505 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.455e-05 2024-09-21T15:01:50.505 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.505 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.505 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:50.505 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:49.417116+0000", 2024-09-21T15:01:50.505 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5630000000000003e-06 2024-09-21T15:01:50.505 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.506 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.506 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:50.506 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:49.417244+0000", 2024-09-21T15:01:50.506 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000127904 2024-09-21T15:01:50.506 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.506 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.506 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:50.506 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:49.417275+0000", 2024-09-21T15:01:50.506 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0193e-05 2024-09-21T15:01:50.506 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.506 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.506 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:50.506 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:50.177149+0000", 2024-09-21T15:01:50.507 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.759874559 2024-09-21T15:01:50.507 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.507 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.507 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:50.507 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:50.177172+0000", 2024-09-21T15:01:50.507 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3167e-05 2024-09-21T15:01:50.507 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:50.507 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:50.507 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:50.507 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:50.507 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-21T15:01:50.507 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 1 2024-09-21T15:01:50.507 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:01:50.512 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T15:01:50.546 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:01:50.540+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:01:50.623 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:01:50.623 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:01:50.624 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T15:01:50.624 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T15:01:50.624 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:01:50.634 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T15:01:50.647 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:01:50.645+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:01:50.746 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:01:50.746 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T15:01:50.746 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T15:01:50.746 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T15:01:50.746 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.746 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:97677 3.15 e122/24 3:a8e72133:::rbd_data.1136bfc79a70.0000000000000005:head v 122'567, mlcod=120'565)", 2024-09-21T15:01:50.746 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.320753+0000", 2024-09-21T15:01:50.746 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 105.421945522, 2024-09-21T15:01:50.746 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070639375000000004, 2024-09-21T15:01:50.746 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:50.746 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:50.747 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:50.747 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.747 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:50.747 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.320753+0000", 2024-09-21T15:01:50.747 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:50.747 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.747 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.747 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:50.747 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.320753+0000", 2024-09-21T15:01:50.747 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:50.747 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.747 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.747 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:50.747 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.320753+0000", 2024-09-21T15:01:50.748 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:50.748 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.748 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.748 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:50.748 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.375452+0000", 2024-09-21T15:01:50.748 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054698945999999998 2024-09-21T15:01:50.748 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.748 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.748 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:50.748 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.375476+0000", 2024-09-21T15:01:50.748 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4380999999999999e-05 2024-09-21T15:01:50.748 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.748 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.748 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:50.749 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.375485+0000", 2024-09-21T15:01:50.749 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0049999999999993e-06 2024-09-21T15:01:50.749 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.749 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.749 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:50.749 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.375552+0000", 2024-09-21T15:01:50.749 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6550999999999994e-05 2024-09-21T15:01:50.749 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.749 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.749 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:50.749 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.375578+0000", 2024-09-21T15:01:50.749 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6196000000000001e-05 2024-09-21T15:01:50.749 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.749 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.750 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:01:50.750 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.391319+0000", 2024-09-21T15:01:50.750 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015741271000000001 2024-09-21T15:01:50.750 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.750 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.750 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:50.750 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.391393+0000", 2024-09-21T15:01:50.750 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3287000000000005e-05 2024-09-21T15:01:50.750 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:50.750 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:50.750 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:50.750 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.750 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.750 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:97672 3.4 e122/51 3:26046d29:::rbd_data.1136bfc79a70.0000000000000001:head v 122'256, mlcod=120'254)", 2024-09-21T15:01:50.751 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.331696+0000", 2024-09-21T15:01:50.751 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 105.41100235899999, 2024-09-21T15:01:50.751 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041222507999999998, 2024-09-21T15:01:50.751 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:50.751 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:50.751 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:50.751 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.751 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:50.751 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.331696+0000", 2024-09-21T15:01:50.751 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:50.751 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.751 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.751 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:50.751 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.331696+0000", 2024-09-21T15:01:50.752 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:50.752 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.752 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.752 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:50.752 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.331695+0000", 2024-09-21T15:01:50.752 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:50.752 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.752 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.752 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:50.752 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.340175+0000", 2024-09-21T15:01:50.752 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0084799520000000007 2024-09-21T15:01:50.752 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.752 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.752 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:50.753 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.340208+0000", 2024-09-21T15:01:50.753 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2997999999999998e-05 2024-09-21T15:01:50.753 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.753 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.753 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:50.753 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.340223+0000", 2024-09-21T15:01:50.753 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5330999999999999e-05 2024-09-21T15:01:50.753 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.753 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.753 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:50.753 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.340305+0000", 2024-09-21T15:01:50.753 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2095999999999995e-05 2024-09-21T15:01:50.753 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.753 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.753 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:50.754 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.340337+0000", 2024-09-21T15:01:50.754 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2097000000000001e-05 2024-09-21T15:01:50.754 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.754 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.754 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:01:50.754 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.372863+0000", 2024-09-21T15:01:50.754 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.032525923999999998 2024-09-21T15:01:50.754 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.754 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.754 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:50.754 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.372919+0000", 2024-09-21T15:01:50.754 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5729000000000001e-05 2024-09-21T15:01:50.754 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:50.754 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:50.755 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:50.755 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.755 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.755 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:116182 3.12 3:4efbaa3f:::rbd_data.1136b1194d20.0000000000000006:58 [read 6242304~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:50.755 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.712538+0000", 2024-09-21T15:01:50.755 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 81.030160367999997, 2024-09-21T15:01:50.755 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034702127999999999, 2024-09-21T15:01:50.755 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:50.755 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:50.755 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:50.755 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:50.755 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:50.755 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116182 2024-09-21T15:01:50.755 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.756 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:50.756 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.756 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:50.756 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712538+0000", 2024-09-21T15:01:50.756 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:50.756 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.756 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.756 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:50.756 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712538+0000", 2024-09-21T15:01:50.756 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:50.756 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.756 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.756 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:50.756 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712538+0000", 2024-09-21T15:01:50.757 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:50.757 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.757 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.757 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:50.757 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712541+0000", 2024-09-21T15:01:50.757 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5330000000000001e-06 2024-09-21T15:01:50.757 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.757 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.757 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:50.757 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712547+0000", 2024-09-21T15:01:50.757 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1109999999999996e-06 2024-09-21T15:01:50.757 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.757 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.757 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:50.757 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712551+0000", 2024-09-21T15:01:50.757 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.685e-06 2024-09-21T15:01:50.758 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.758 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.758 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:50.758 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.747067+0000", 2024-09-21T15:01:50.758 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034516181 2024-09-21T15:01:50.758 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.758 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.758 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:50.758 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.747152+0000", 2024-09-21T15:01:50.758 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4075999999999999e-05 2024-09-21T15:01:50.758 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.758 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.758 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:50.758 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.747240+0000", 2024-09-21T15:01:50.759 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8924000000000004e-05 2024-09-21T15:01:50.759 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:50.759 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:50.759 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:50.759 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.759 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.759 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139784 3.1 3:862e0e65:::rbd_data.113664780c85.0000000000000063:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:50.759 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.222486+0000", 2024-09-21T15:01:50.759 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 48.520212501000003, 2024-09-21T15:01:50.759 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0141793059999999, 2024-09-21T15:01:50.759 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:50.759 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:50.759 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:50.760 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:50.760 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:50.760 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139784 2024-09-21T15:01:50.760 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.760 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:50.760 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.760 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:50.760 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222486+0000", 2024-09-21T15:01:50.760 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:50.760 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.760 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.760 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:50.760 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222486+0000", 2024-09-21T15:01:50.760 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:50.760 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.761 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.761 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:50.761 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222484+0000", 2024-09-21T15:01:50.761 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-21T15:01:50.761 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.761 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.761 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:50.761 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222494+0000", 2024-09-21T15:01:50.761 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0676000000000001e-05 2024-09-21T15:01:50.761 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.761 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.761 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:50.761 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222517+0000", 2024-09-21T15:01:50.761 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2371e-05 2024-09-21T15:01:50.762 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.762 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.762 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:50.762 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222527+0000", 2024-09-21T15:01:50.762 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0461e-05 2024-09-21T15:01:50.762 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.762 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.762 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:50.762 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222625+0000", 2024-09-21T15:01:50.762 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.8160999999999997e-05 2024-09-21T15:01:50.762 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.763 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.763 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:50.763 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222641+0000", 2024-09-21T15:01:50.763 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5899000000000001e-05 2024-09-21T15:01:50.763 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.763 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.763 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:50.763 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.236479+0000", 2024-09-21T15:01:50.763 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0138377709999999 2024-09-21T15:01:50.763 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.763 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.763 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:50.763 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.236666+0000", 2024-09-21T15:01:50.763 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018648199999999999 2024-09-21T15:01:50.764 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:50.764 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:50.764 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:50.764 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.764 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.764 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139812 3.1 3:8407fb9b:::rbd_data.113664780c85.0000000000000127:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:50.764 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.222547+0000", 2024-09-21T15:01:50.764 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 48.520151577, 2024-09-21T15:01:50.764 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.014796059, 2024-09-21T15:01:50.764 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:50.764 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:50.764 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:50.764 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:50.764 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:50.765 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139812 2024-09-21T15:01:50.765 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.765 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:50.765 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.765 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:50.765 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222547+0000", 2024-09-21T15:01:50.765 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:50.765 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.765 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.765 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:50.765 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222547+0000", 2024-09-21T15:01:50.765 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:50.765 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.765 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.766 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:50.766 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222546+0000", 2024-09-21T15:01:50.766 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:50.766 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.766 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.766 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:50.766 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222552+0000", 2024-09-21T15:01:50.766 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2739999999999998e-06 2024-09-21T15:01:50.766 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.766 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.766 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:50.766 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222578+0000", 2024-09-21T15:01:50.766 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6594999999999999e-05 2024-09-21T15:01:50.766 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.767 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.767 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:50.767 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222596+0000", 2024-09-21T15:01:50.767 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7838000000000001e-05 2024-09-21T15:01:50.767 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.767 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.767 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:50.767 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222701+0000", 2024-09-21T15:01:50.767 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000104714 2024-09-21T15:01:50.767 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.767 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.767 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:50.767 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222714+0000", 2024-09-21T15:01:50.767 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2998e-05 2024-09-21T15:01:50.768 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.768 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.768 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:50.768 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.237124+0000", 2024-09-21T15:01:50.768 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.014409806 2024-09-21T15:01:50.768 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.768 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.768 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:50.768 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.237343+0000", 2024-09-21T15:01:50.768 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021950499999999999 2024-09-21T15:01:50.768 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:50.768 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:50.768 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:50.768 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.769 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.769 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139826 3.1 3:854f6f9b:::rbd_data.113664780c85.0000000000000189:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:50.769 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.222611+0000", 2024-09-21T15:01:50.769 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 48.520087484999998, 2024-09-21T15:01:50.769 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.014916828, 2024-09-21T15:01:50.769 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:50.769 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:50.769 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:50.769 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:50.769 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:50.769 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139826 2024-09-21T15:01:50.769 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.769 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:50.769 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.770 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:50.770 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222611+0000", 2024-09-21T15:01:50.770 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:50.770 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.770 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.770 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:50.770 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222611+0000", 2024-09-21T15:01:50.770 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:50.770 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.770 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.770 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:50.770 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222610+0000", 2024-09-21T15:01:50.770 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:50.770 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.771 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.771 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:50.771 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222618+0000", 2024-09-21T15:01:50.771 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6690000000000006e-06 2024-09-21T15:01:50.771 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.771 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.771 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:50.771 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222632+0000", 2024-09-21T15:01:50.771 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4433999999999999e-05 2024-09-21T15:01:50.771 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.771 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.771 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:50.771 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222640+0000", 2024-09-21T15:01:50.771 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6260000000000003e-06 2024-09-21T15:01:50.771 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.772 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.772 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:50.772 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222758+0000", 2024-09-21T15:01:50.772 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011826499999999999 2024-09-21T15:01:50.772 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.772 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.772 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:50.772 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222776+0000", 2024-09-21T15:01:50.772 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7564999999999999e-05 2024-09-21T15:01:50.772 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.772 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.772 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:50.772 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.237402+0000", 2024-09-21T15:01:50.772 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0146261889999999 2024-09-21T15:01:50.772 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.772 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.773 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:50.773 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.237528+0000", 2024-09-21T15:01:50.773 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000126212 2024-09-21T15:01:50.773 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:50.773 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:50.773 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:50.773 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.773 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.773 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139864 3.1 3:86b14cb7:::rbd_data.113664780c85.00000000000000d5:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:50.773 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.223966+0000", 2024-09-21T15:01:50.773 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 48.518732749000002, 2024-09-21T15:01:50.773 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.013859007, 2024-09-21T15:01:50.773 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:50.773 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:50.774 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:50.774 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:50.774 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:50.774 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139864 2024-09-21T15:01:50.774 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.774 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:50.774 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.774 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:50.774 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223966+0000", 2024-09-21T15:01:50.774 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:50.774 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.774 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.774 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:50.774 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223966+0000", 2024-09-21T15:01:50.774 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:50.775 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.775 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.775 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:50.775 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223965+0000", 2024-09-21T15:01:50.775 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:50.775 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.775 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.775 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:50.775 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223970+0000", 2024-09-21T15:01:50.775 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3769999999999998e-06 2024-09-21T15:01:50.775 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.775 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.775 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:50.775 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223979+0000", 2024-09-21T15:01:50.775 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5899999999999997e-06 2024-09-21T15:01:50.775 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.776 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.776 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:50.776 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223985+0000", 2024-09-21T15:01:50.776 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5790000000000003e-06 2024-09-21T15:01:50.776 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.776 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.776 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:50.776 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224073+0000", 2024-09-21T15:01:50.776 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7781000000000005e-05 2024-09-21T15:01:50.776 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.776 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.776 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:50.776 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224110+0000", 2024-09-21T15:01:50.776 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7060000000000001e-05 2024-09-21T15:01:50.777 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.777 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.777 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:50.777 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.237599+0000", 2024-09-21T15:01:50.777 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.01348953 2024-09-21T15:01:50.777 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.777 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.777 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:50.777 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.237825+0000", 2024-09-21T15:01:50.777 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000225758 2024-09-21T15:01:50.777 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:50.777 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:50.777 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:50.777 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.778 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.778 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139877 3.1 3:87277f92:::rbd_data.113664780c85.0000000000000136:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:50.778 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.224489+0000", 2024-09-21T15:01:50.778 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 48.518209890999998, 2024-09-21T15:01:50.778 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0137916330000001, 2024-09-21T15:01:50.778 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:50.778 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:50.778 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:50.778 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:50.778 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:50.778 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139877 2024-09-21T15:01:50.778 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.778 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:50.778 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.779 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:50.779 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224489+0000", 2024-09-21T15:01:50.779 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:50.779 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.779 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.779 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:50.779 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224489+0000", 2024-09-21T15:01:50.779 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:50.779 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.779 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.779 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:50.779 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224488+0000", 2024-09-21T15:01:50.779 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:50.780 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.780 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.780 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:50.780 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224502+0000", 2024-09-21T15:01:50.780 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4194e-05 2024-09-21T15:01:50.780 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.780 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.780 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:50.780 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224522+0000", 2024-09-21T15:01:50.780 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0038e-05 2024-09-21T15:01:50.780 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.780 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.780 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:50.780 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224531+0000", 2024-09-21T15:01:50.780 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6900000000000004e-06 2024-09-21T15:01:50.780 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.781 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.781 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:50.781 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224621+0000", 2024-09-21T15:01:50.781 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9654000000000006e-05 2024-09-21T15:01:50.781 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.781 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.781 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:50.781 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224641+0000", 2024-09-21T15:01:50.781 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9375e-05 2024-09-21T15:01:50.781 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.781 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.781 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:50.781 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.238027+0000", 2024-09-21T15:01:50.781 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0133866840000001 2024-09-21T15:01:50.781 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.782 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.782 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:50.782 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.238280+0000", 2024-09-21T15:01:50.782 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000253281 2024-09-21T15:01:50.782 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:50.782 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:50.782 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:50.782 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.782 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.782 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139887 3.1 3:80f1e8f1:::rbd_data.113664780c85.000000000000017c:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:50.782 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.224549+0000", 2024-09-21T15:01:50.782 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 48.518149600999998, 2024-09-21T15:01:50.782 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0139776060000001, 2024-09-21T15:01:50.782 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:50.782 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:50.783 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:50.783 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:50.783 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:50.783 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139887 2024-09-21T15:01:50.783 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.783 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:50.783 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.783 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:50.783 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224549+0000", 2024-09-21T15:01:50.783 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:50.783 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.783 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.783 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:50.783 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224549+0000", 2024-09-21T15:01:50.783 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:50.784 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.784 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.784 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:50.784 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224548+0000", 2024-09-21T15:01:50.784 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:50.784 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.784 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.784 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:50.784 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224554+0000", 2024-09-21T15:01:50.784 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4049999999999999e-06 2024-09-21T15:01:50.784 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.784 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.784 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:50.784 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224565+0000", 2024-09-21T15:01:50.784 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0879e-05 2024-09-21T15:01:50.784 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.785 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.785 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:50.785 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224572+0000", 2024-09-21T15:01:50.785 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0380000000000002e-06 2024-09-21T15:01:50.785 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.785 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.785 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:50.785 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224684+0000", 2024-09-21T15:01:50.785 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011229 2024-09-21T15:01:50.785 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.785 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.785 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:50.785 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224698+0000", 2024-09-21T15:01:50.785 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4448e-05 2024-09-21T15:01:50.785 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.786 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.786 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:50.786 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.238359+0000", 2024-09-21T15:01:50.786 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0136610660000001 2024-09-21T15:01:50.786 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.786 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.786 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:50.786 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.238527+0000", 2024-09-21T15:01:50.786 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016731100000000001 2024-09-21T15:01:50.786 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:50.786 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:50.786 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:50.786 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.787 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.787 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140077 3.1 3:85642daf:::rbd_data.113664780c85.0000000000000174:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:50.787 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.232797+0000", 2024-09-21T15:01:50.787 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 48.509901886000002, 2024-09-21T15:01:50.787 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.006029955, 2024-09-21T15:01:50.787 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:50.787 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:50.787 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:50.787 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:50.787 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:50.787 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140077 2024-09-21T15:01:50.787 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.787 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:50.788 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.788 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:50.788 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232797+0000", 2024-09-21T15:01:50.788 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:50.788 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.788 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.788 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:50.788 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232797+0000", 2024-09-21T15:01:50.788 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:50.788 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.788 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.788 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:50.788 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232795+0000", 2024-09-21T15:01:50.788 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:50.789 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.789 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.789 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:50.789 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232804+0000", 2024-09-21T15:01:50.789 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9670000000000007e-06 2024-09-21T15:01:50.789 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.789 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.789 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:50.789 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232824+0000", 2024-09-21T15:01:50.789 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9238000000000001e-05 2024-09-21T15:01:50.789 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.789 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.789 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:50.789 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232843+0000", 2024-09-21T15:01:50.790 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9256e-05 2024-09-21T15:01:50.790 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.790 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.790 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:50.790 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232912+0000", 2024-09-21T15:01:50.790 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9589000000000001e-05 2024-09-21T15:01:50.790 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.790 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.790 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:50.790 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232925+0000", 2024-09-21T15:01:50.790 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2585999999999999e-05 2024-09-21T15:01:50.790 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.790 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.790 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:50.791 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.238591+0000", 2024-09-21T15:01:50.791 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0056659939999999 2024-09-21T15:01:50.791 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.791 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.791 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:50.791 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.238827+0000", 2024-09-21T15:01:50.791 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00023583999999999999 2024-09-21T15:01:50.791 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:50.791 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:50.791 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:50.791 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.791 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.791 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140091 3.1 3:8358f0db:::rbd_data.113664780c85.0000000000000018:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:50.792 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.233637+0000", 2024-09-21T15:01:50.792 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 48.509061590000002, 2024-09-21T15:01:50.792 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.006272847, 2024-09-21T15:01:50.792 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:50.792 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:50.792 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:50.792 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:50.792 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:50.792 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140091 2024-09-21T15:01:50.792 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.792 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:50.792 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.792 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:50.792 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233637+0000", 2024-09-21T15:01:50.792 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:50.792 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.793 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.793 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:50.793 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233637+0000", 2024-09-21T15:01:50.793 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:50.793 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.793 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.793 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:50.793 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233636+0000", 2024-09-21T15:01:50.793 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:50.793 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.793 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.793 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:50.793 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233644+0000", 2024-09-21T15:01:50.793 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1380000000000007e-06 2024-09-21T15:01:50.794 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.794 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.794 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:50.794 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233662+0000", 2024-09-21T15:01:50.794 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8224999999999999e-05 2024-09-21T15:01:50.794 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.794 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.794 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:50.794 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233674+0000", 2024-09-21T15:01:50.794 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2081e-05 2024-09-21T15:01:50.794 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.794 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.794 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:50.794 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233750+0000", 2024-09-21T15:01:50.795 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6106000000000006e-05 2024-09-21T15:01:50.795 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.795 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.795 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:50.795 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233768+0000", 2024-09-21T15:01:50.795 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7285000000000001e-05 2024-09-21T15:01:50.795 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.795 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.795 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:50.795 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.239196+0000", 2024-09-21T15:01:50.795 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.005428223 2024-09-21T15:01:50.795 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.795 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.795 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:50.796 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.239528+0000", 2024-09-21T15:01:50.796 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00033262199999999997 2024-09-21T15:01:50.796 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.796 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.796 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:50.796 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.239910+0000", 2024-09-21T15:01:50.796 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00038151100000000001 2024-09-21T15:01:50.796 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:50.796 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:50.796 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:50.796 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.796 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.796 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140109 3.1 3:8586e9d9:::rbd_data.113664780c85.0000000000000096:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:50.796 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.234171+0000", 2024-09-21T15:01:50.797 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 48.508527755999999, 2024-09-21T15:01:50.797 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.006019896, 2024-09-21T15:01:50.797 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:50.797 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:50.797 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:50.797 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:50.797 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:50.797 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140109 2024-09-21T15:01:50.797 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.797 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:50.797 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.797 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:50.797 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.234171+0000", 2024-09-21T15:01:50.797 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:50.798 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.798 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.798 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:50.798 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.234171+0000", 2024-09-21T15:01:50.798 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:50.798 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.798 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.798 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:50.798 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.234170+0000", 2024-09-21T15:01:50.798 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:50.798 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.798 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.798 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:50.798 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.234178+0000", 2024-09-21T15:01:50.798 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.464e-06 2024-09-21T15:01:50.799 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.799 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.799 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:50.799 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.234191+0000", 2024-09-21T15:01:50.799 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3489000000000001e-05 2024-09-21T15:01:50.799 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.799 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.799 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:50.799 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.234200+0000", 2024-09-21T15:01:50.799 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2779999999999999e-06 2024-09-21T15:01:50.799 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.799 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.799 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:50.799 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.234279+0000", 2024-09-21T15:01:50.800 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8813000000000006e-05 2024-09-21T15:01:50.800 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.800 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.800 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:50.800 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.234298+0000", 2024-09-21T15:01:50.800 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8947999999999999e-05 2024-09-21T15:01:50.800 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.800 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.800 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:50.800 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.239974+0000", 2024-09-21T15:01:50.800 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0056760259999999 2024-09-21T15:01:50.800 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.800 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.800 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:50.800 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.240191+0000", 2024-09-21T15:01:50.801 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000216708 2024-09-21T15:01:50.801 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:50.801 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:50.801 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:50.801 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.801 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.801 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140149 3.1 3:86b1e433:::rbd_data.113664780c85.00000000000001ae:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:50.801 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.235545+0000", 2024-09-21T15:01:50.801 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 48.507153471000002, 2024-09-21T15:01:50.801 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0049193430000001, 2024-09-21T15:01:50.801 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:50.801 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:50.801 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:50.801 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:50.801 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:50.801 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140149 2024-09-21T15:01:50.802 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.802 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:50.802 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.802 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:50.802 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235545+0000", 2024-09-21T15:01:50.802 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:50.802 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.802 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.802 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:50.802 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235545+0000", 2024-09-21T15:01:50.802 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:50.802 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.802 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.802 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:50.803 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235544+0000", 2024-09-21T15:01:50.803 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:50.803 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.803 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.803 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:50.803 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235554+0000", 2024-09-21T15:01:50.803 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.8870000000000004e-06 2024-09-21T15:01:50.803 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.803 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.803 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:50.803 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235577+0000", 2024-09-21T15:01:50.803 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2946999999999999e-05 2024-09-21T15:01:50.803 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.803 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.804 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:50.804 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235587+0000", 2024-09-21T15:01:50.804 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0326000000000001e-05 2024-09-21T15:01:50.804 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.804 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.804 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:50.804 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235658+0000", 2024-09-21T15:01:50.804 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1360999999999997e-05 2024-09-21T15:01:50.804 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.804 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.804 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:50.804 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235670+0000", 2024-09-21T15:01:50.804 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1735e-05 2024-09-21T15:01:50.804 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.804 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.805 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:50.805 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.240246+0000", 2024-09-21T15:01:50.805 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0045757790000001 2024-09-21T15:01:50.805 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.805 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.805 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:50.805 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.240465+0000", 2024-09-21T15:01:50.805 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021870799999999999 2024-09-21T15:01:50.805 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:50.805 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:50.805 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:50.805 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.805 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.805 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140232 3.1 3:862e0e65:::rbd_data.113664780c85.0000000000000063:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:50.806 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.235760+0000", 2024-09-21T15:01:50.806 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 48.506938753, 2024-09-21T15:01:50.806 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.004979812, 2024-09-21T15:01:50.806 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:50.806 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:50.806 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:50.806 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:50.806 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:50.806 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140232 2024-09-21T15:01:50.806 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.806 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:50.806 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.806 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:50.806 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235760+0000", 2024-09-21T15:01:50.807 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:50.807 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.807 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.807 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:50.807 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235760+0000", 2024-09-21T15:01:50.807 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:50.807 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.807 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.807 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:50.807 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235759+0000", 2024-09-21T15:01:50.807 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:50.807 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.807 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.807 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:50.808 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235777+0000", 2024-09-21T15:01:50.808 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8215000000000001e-05 2024-09-21T15:01:50.808 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.808 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.808 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:50.808 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235793+0000", 2024-09-21T15:01:50.808 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6461e-05 2024-09-21T15:01:50.808 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.808 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.808 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:50.808 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235803+0000", 2024-09-21T15:01:50.808 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5300000000000002e-06 2024-09-21T15:01:50.808 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.809 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.809 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:50.809 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235875+0000", 2024-09-21T15:01:50.809 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2113000000000006e-05 2024-09-21T15:01:50.809 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.809 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.809 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:50.809 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235887+0000", 2024-09-21T15:01:50.809 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2160999999999999e-05 2024-09-21T15:01:50.809 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.809 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.809 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:50.809 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.240521+0000", 2024-09-21T15:01:50.809 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.004633428 2024-09-21T15:01:50.809 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.810 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.810 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:50.810 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.240740+0000", 2024-09-21T15:01:50.810 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000219228 2024-09-21T15:01:50.810 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:50.810 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:50.810 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:50.810 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.810 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.810 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140260 3.1 3:8407fb9b:::rbd_data.113664780c85.0000000000000127:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:50.810 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.235839+0000", 2024-09-21T15:01:50.810 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 48.506859583999997, 2024-09-21T15:01:50.810 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.005157981, 2024-09-21T15:01:50.811 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:50.811 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:50.811 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:50.811 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:50.811 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:50.811 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140260 2024-09-21T15:01:50.811 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.811 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:50.811 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.811 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:50.811 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235839+0000", 2024-09-21T15:01:50.811 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:50.811 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.811 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.811 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:50.812 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235839+0000", 2024-09-21T15:01:50.812 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:50.812 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.812 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.812 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:50.812 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235838+0000", 2024-09-21T15:01:50.812 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:50.812 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.812 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.812 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:50.812 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235843+0000", 2024-09-21T15:01:50.812 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.481e-06 2024-09-21T15:01:50.812 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.812 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.812 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:50.812 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235849+0000", 2024-09-21T15:01:50.813 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4880000000000004e-06 2024-09-21T15:01:50.813 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.813 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.813 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:50.813 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235856+0000", 2024-09-21T15:01:50.813 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7649999999999996e-06 2024-09-21T15:01:50.813 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.813 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.813 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:50.813 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235933+0000", 2024-09-21T15:01:50.813 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6987e-05 2024-09-21T15:01:50.813 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.813 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.813 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:50.813 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235949+0000", 2024-09-21T15:01:50.814 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6696000000000001e-05 2024-09-21T15:01:50.814 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.814 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.814 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:50.814 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.240823+0000", 2024-09-21T15:01:50.814 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.004873259 2024-09-21T15:01:50.814 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.814 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.814 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:50.814 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.240997+0000", 2024-09-21T15:01:50.814 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000174365 2024-09-21T15:01:50.814 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:50.814 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:50.814 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:50.815 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.815 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.815 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140274 3.1 3:854f6f9b:::rbd_data.113664780c85.0000000000000189:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:50.815 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.235884+0000", 2024-09-21T15:01:50.815 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 48.506814636000001, 2024-09-21T15:01:50.815 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.005397366, 2024-09-21T15:01:50.815 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:50.815 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:50.815 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:50.815 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:50.815 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:50.815 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140274 2024-09-21T15:01:50.815 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.815 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:50.816 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.816 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:50.816 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235884+0000", 2024-09-21T15:01:50.816 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:50.816 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.816 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.816 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:50.816 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235884+0000", 2024-09-21T15:01:50.816 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:50.816 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.816 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.816 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:50.816 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235883+0000", 2024-09-21T15:01:50.816 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:50.816 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.817 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.817 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:50.817 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235891+0000", 2024-09-21T15:01:50.817 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9740000000000002e-06 2024-09-21T15:01:50.817 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.817 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.817 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:50.817 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235900+0000", 2024-09-21T15:01:50.817 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6659999999999997e-06 2024-09-21T15:01:50.817 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.817 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.817 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:50.817 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235909+0000", 2024-09-21T15:01:50.817 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3400000000000004e-06 2024-09-21T15:01:50.818 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.818 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.818 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:50.818 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236012+0000", 2024-09-21T15:01:50.818 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010240500000000001 2024-09-21T15:01:50.818 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.818 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.818 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:50.818 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236027+0000", 2024-09-21T15:01:50.818 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5418e-05 2024-09-21T15:01:50.818 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.818 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.818 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:50.818 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.241068+0000", 2024-09-21T15:01:50.818 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0050410729999999 2024-09-21T15:01:50.819 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.819 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.819 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:50.819 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.241281+0000", 2024-09-21T15:01:50.819 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021321900000000001 2024-09-21T15:01:50.819 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:50.819 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:50.819 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:50.819 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.819 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.819 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140312 3.1 3:86b14cb7:::rbd_data.113664780c85.00000000000000d5:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:50.819 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.236769+0000", 2024-09-21T15:01:50.819 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 48.505930171999999, 2024-09-21T15:01:50.819 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.004717198, 2024-09-21T15:01:50.819 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:50.820 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:50.820 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:50.820 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:50.820 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:50.820 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140312 2024-09-21T15:01:50.820 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.820 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:50.820 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.820 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:50.820 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236769+0000", 2024-09-21T15:01:50.820 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:50.820 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.820 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.820 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:50.821 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236769+0000", 2024-09-21T15:01:50.821 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:50.821 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.821 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.821 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:50.821 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236768+0000", 2024-09-21T15:01:50.821 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:50.821 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.821 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.821 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:50.821 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236771+0000", 2024-09-21T15:01:50.821 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.331e-06 2024-09-21T15:01:50.821 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.821 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.821 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:50.822 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236777+0000", 2024-09-21T15:01:50.822 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6949999999999997e-06 2024-09-21T15:01:50.822 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.822 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.822 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:50.822 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236787+0000", 2024-09-21T15:01:50.822 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6700000000000006e-06 2024-09-21T15:01:50.822 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.822 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.822 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:50.822 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236846+0000", 2024-09-21T15:01:50.822 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.944e-05 2024-09-21T15:01:50.822 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.822 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.822 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:50.823 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236861+0000", 2024-09-21T15:01:50.823 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4611e-05 2024-09-21T15:01:50.823 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.823 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.823 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:50.823 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.241349+0000", 2024-09-21T15:01:50.823 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0044879250000001 2024-09-21T15:01:50.823 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.823 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.823 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:50.823 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.241486+0000", 2024-09-21T15:01:50.823 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013713000000000001 2024-09-21T15:01:50.823 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:50.823 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:50.823 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:50.823 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.824 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.824 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140325 3.1 3:87277f92:::rbd_data.113664780c85.0000000000000136:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:50.824 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.239351+0000", 2024-09-21T15:01:50.824 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 48.503348033000002, 2024-09-21T15:01:50.824 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0024334690000001, 2024-09-21T15:01:50.824 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:50.824 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:50.824 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:50.824 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:50.824 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:50.824 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140325 2024-09-21T15:01:50.824 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.824 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:50.824 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.824 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:50.825 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239351+0000", 2024-09-21T15:01:50.825 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:50.825 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.825 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.825 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:50.825 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239351+0000", 2024-09-21T15:01:50.825 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:50.825 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.825 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.825 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:50.825 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239349+0000", 2024-09-21T15:01:50.825 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:50.825 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.825 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.826 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:50.826 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239358+0000", 2024-09-21T15:01:50.826 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2830000000000001e-06 2024-09-21T15:01:50.826 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.826 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.826 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:50.826 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239379+0000", 2024-09-21T15:01:50.826 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1780000000000002e-05 2024-09-21T15:01:50.826 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.826 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.826 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:50.826 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239389+0000", 2024-09-21T15:01:50.826 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0056e-05 2024-09-21T15:01:50.826 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.827 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.827 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:50.827 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239484+0000", 2024-09-21T15:01:50.827 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4611000000000006e-05 2024-09-21T15:01:50.827 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.827 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.827 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:50.827 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239497+0000", 2024-09-21T15:01:50.827 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2909e-05 2024-09-21T15:01:50.827 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.827 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.827 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:50.827 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.241559+0000", 2024-09-21T15:01:50.827 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0020622459999999 2024-09-21T15:01:50.828 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.828 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.828 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:50.828 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.241784+0000", 2024-09-21T15:01:50.828 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000225009 2024-09-21T15:01:50.828 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:50.828 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:50.828 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:50.828 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.828 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.828 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140335 3.1 3:80f1e8f1:::rbd_data.113664780c85.000000000000017c:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:50.828 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.239409+0000", 2024-09-21T15:01:50.828 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 48.503289314, 2024-09-21T15:01:50.828 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0025803150000001, 2024-09-21T15:01:50.828 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:50.828 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:50.829 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:50.829 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:50.829 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:50.829 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140335 2024-09-21T15:01:50.829 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.829 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:50.829 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.829 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:50.829 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239409+0000", 2024-09-21T15:01:50.829 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:50.829 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.829 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.829 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:50.829 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239409+0000", 2024-09-21T15:01:50.829 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:50.830 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.830 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.830 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:50.830 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239409+0000", 2024-09-21T15:01:50.830 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:50.830 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.830 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.830 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:50.830 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239413+0000", 2024-09-21T15:01:50.830 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7620000000000003e-06 2024-09-21T15:01:50.830 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.830 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.830 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:50.830 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239436+0000", 2024-09-21T15:01:50.831 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2889000000000001e-05 2024-09-21T15:01:50.831 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.831 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.831 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:50.831 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239444+0000", 2024-09-21T15:01:50.831 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6809999999999998e-06 2024-09-21T15:01:50.831 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.831 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.831 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:50.831 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239542+0000", 2024-09-21T15:01:50.831 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7868999999999994e-05 2024-09-21T15:01:50.831 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.831 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.831 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:50.832 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239558+0000", 2024-09-21T15:01:50.832 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6257e-05 2024-09-21T15:01:50.832 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.832 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.832 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:50.832 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.241847+0000", 2024-09-21T15:01:50.832 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.002288533 2024-09-21T15:01:50.832 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.832 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.832 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:50.832 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.241990+0000", 2024-09-21T15:01:50.832 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014308300000000001 2024-09-21T15:01:50.832 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:50.832 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:50.832 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:50.833 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.833 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.833 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:166488 3.4 3.7bcf3e84 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e189)", 2024-09-21T15:01:50.833 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:21.342023+0000", 2024-09-21T15:01:50.833 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 29.400675482, 2024-09-21T15:01:50.833 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0015098469999999, 2024-09-21T15:01:50.833 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:50.833 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:50.833 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:50.833 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:50.833 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:50.833 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 166488 2024-09-21T15:01:50.833 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.833 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:50.834 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.834 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:50.834 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:21.342023+0000", 2024-09-21T15:01:50.834 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:50.834 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.834 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.834 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:50.834 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:21.342023+0000", 2024-09-21T15:01:50.834 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:50.834 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.834 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.834 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:50.834 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:21.342022+0000", 2024-09-21T15:01:50.834 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:50.835 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.835 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.835 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:50.835 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:21.342029+0000", 2024-09-21T15:01:50.835 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1420000000000004e-06 2024-09-21T15:01:50.835 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.835 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.835 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:50.835 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:21.342051+0000", 2024-09-21T15:01:50.835 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2549999999999999e-05 2024-09-21T15:01:50.835 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.835 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.835 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:50.835 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:21.342059+0000", 2024-09-21T15:01:50.836 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8490000000000006e-06 2024-09-21T15:01:50.836 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.836 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.836 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:50.836 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:21.342105+0000", 2024-09-21T15:01:50.836 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5488999999999998e-05 2024-09-21T15:01:50.836 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.836 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.836 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:50.836 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:21.342115+0000", 2024-09-21T15:01:50.836 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0633e-05 2024-09-21T15:01:50.836 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.836 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.836 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:50.836 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:22.343521+0000", 2024-09-21T15:01:50.836 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0014057270000001 2024-09-21T15:01:50.837 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:50.837 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:50.837 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:50.837 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:22.343533+0000", 2024-09-21T15:01:50.837 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2097e-05 2024-09-21T15:01:50.837 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:50.837 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:50.837 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:50.837 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:50.837 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:50.837 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:01:50.838 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T15:01:50.839 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T15:01:50.839 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:01:50.832+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:01:50.939 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:01:50.937+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:01:50.952 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:01:50.952 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:01:50.952 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T15:01:50.952 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:01:50.962 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T15:01:50.997 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T15:01:51.021 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T15:01:51.040 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:01:51.037+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:01:51.043 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T15:01:51.065 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T15:01:51.072 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:01:51.072 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:01:51.072 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T15:01:51.072 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T15:01:51.072 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:01:51.082 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T15:01:51.089 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T15:01:51.109 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T15:01:51.132 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T15:01:51.132 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:01:51.132 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:01:51.133 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:01:51.133 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:01:51.133 INFO:teuthology.orchestra.run.smithi032.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-21T15:01:51.133 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:01:51.133 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:01:51.133 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:01:51.137 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-21T15:01:51.137 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-21T15:01:51.140 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:01:51.136+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:01:51.195 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:51.195 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:51.208 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T15:01:51.253 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:01:51.253+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:01:51.330 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:51.330 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:51.341 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T15:01:51.342 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:01:51.341+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:01:51.443 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:01:51.441+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:01:51.452 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:51.452 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:51.462 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T15:01:51.544 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:01:51.540+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:01:51.620 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:51.620 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:51.630 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T15:01:51.645 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:01:51.640+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:01:51.740 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:51.740 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:51.745 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:01:51.741+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:01:51.750 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T15:01:51.846 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:01:51.845+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:01:51.860 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:51.860 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:51.871 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T15:01:51.946 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:01:51.945+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:01:52.024 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:52.024 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:52.034 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T15:01:52.047 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:01:52.045+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:01:52.139 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:52.139 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:52.148 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T15:01:52.149 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:01:52.144+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:01:52.201 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T15:01:52.250 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:01:52.244+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:01:52.310 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:52.310 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:52.322 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T15:01:52.351 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:01:52.349+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:01:52.424 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:52.424 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:52.434 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T15:01:52.453 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:01:52.448+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:01:52.552 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:01:52.548+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:01:52.588 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:52.588 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:52.598 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T15:01:52.652 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:01:52.649+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:01:52.706 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:52.706 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:52.716 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T15:01:52.762 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:01:52.761+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:01:52.819 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:52.819 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:52.829 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T15:01:52.855 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:01:52.853+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:01:52.947 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:52.947 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:52.955 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:01:52.953+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:01:52.957 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T15:01:53.057 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:01:53.057+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:01:53.066 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:53.066 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:53.076 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T15:01:53.139 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T15:01:53.157 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:01:53.152+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:01:53.187 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:53.187 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:53.196 INFO:teuthology.orchestra.run.smithi032.stderr:noscrub is set 2024-09-21T15:01:53.198 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T15:01:53.258 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:01:53.257+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:01:53.298 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:53.298 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:53.300 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T15:01:53.308 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T15:01:53.321 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T15:01:53.344 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T15:01:53.360 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:01:53.356+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:01:53.367 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T15:01:53.394 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T15:01:53.417 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T15:01:53.418 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:01:53.418 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:01:53.418 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T15:01:53.418 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T15:01:53.418 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:01:53.428 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T15:01:53.441 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T15:01:53.442 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:01:53.442 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:01:53.442 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:01:53.442 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:01:53.442 INFO:teuthology.orchestra.run.smithi032.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-21T15:01:53.442 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:01:53.442 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:01:53.442 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:01:53.446 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-21T15:01:53.446 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-21T15:01:53.471 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:01:53.468+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:01:53.561 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:01:53.561+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:01:53.584 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:01:53.584 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T15:01:53.584 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T15:01:53.584 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T15:01:53.585 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.585 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:97677 3.15 e122/24 3:a8e72133:::rbd_data.1136bfc79a70.0000000000000005:head v 122'567, mlcod=120'565)", 2024-09-21T15:01:53.585 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.320753+0000", 2024-09-21T15:01:53.585 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 108.26031919, 2024-09-21T15:01:53.585 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070639375000000004, 2024-09-21T15:01:53.585 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:53.585 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:53.585 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:53.585 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.585 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:53.585 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.320753+0000", 2024-09-21T15:01:53.585 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:53.585 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.586 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.586 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:53.586 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.320753+0000", 2024-09-21T15:01:53.586 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:53.586 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.586 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.586 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:53.586 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.320753+0000", 2024-09-21T15:01:53.586 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:53.586 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.586 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.586 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:53.586 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.375452+0000", 2024-09-21T15:01:53.586 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054698945999999998 2024-09-21T15:01:53.587 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.587 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.587 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:53.587 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.375476+0000", 2024-09-21T15:01:53.587 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4380999999999999e-05 2024-09-21T15:01:53.587 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.587 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.587 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:53.587 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.375485+0000", 2024-09-21T15:01:53.587 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0049999999999993e-06 2024-09-21T15:01:53.587 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.587 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.587 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:53.588 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.375552+0000", 2024-09-21T15:01:53.588 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6550999999999994e-05 2024-09-21T15:01:53.588 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.588 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.588 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:53.588 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.375578+0000", 2024-09-21T15:01:53.588 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6196000000000001e-05 2024-09-21T15:01:53.588 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.588 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.588 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:01:53.588 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.391319+0000", 2024-09-21T15:01:53.588 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015741271000000001 2024-09-21T15:01:53.588 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.588 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.589 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:53.589 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.391393+0000", 2024-09-21T15:01:53.589 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3287000000000005e-05 2024-09-21T15:01:53.589 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:53.589 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:53.589 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:53.589 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.589 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.589 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:97672 3.4 e122/51 3:26046d29:::rbd_data.1136bfc79a70.0000000000000001:head v 122'256, mlcod=120'254)", 2024-09-21T15:01:53.589 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.331696+0000", 2024-09-21T15:01:53.589 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 108.249376027, 2024-09-21T15:01:53.589 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041222507999999998, 2024-09-21T15:01:53.589 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:53.589 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:53.590 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:53.590 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.590 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:53.590 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.331696+0000", 2024-09-21T15:01:53.590 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:53.590 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.590 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.590 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:53.590 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.331696+0000", 2024-09-21T15:01:53.590 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:53.590 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.590 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.590 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:53.590 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.331695+0000", 2024-09-21T15:01:53.591 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:53.591 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.591 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.591 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:53.591 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.340175+0000", 2024-09-21T15:01:53.591 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0084799520000000007 2024-09-21T15:01:53.591 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.591 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.591 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:53.591 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.340208+0000", 2024-09-21T15:01:53.591 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2997999999999998e-05 2024-09-21T15:01:53.591 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.591 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.591 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:53.592 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.340223+0000", 2024-09-21T15:01:53.592 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5330999999999999e-05 2024-09-21T15:01:53.592 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.592 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.592 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:53.592 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.340305+0000", 2024-09-21T15:01:53.592 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2095999999999995e-05 2024-09-21T15:01:53.592 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.592 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.592 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:53.592 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.340337+0000", 2024-09-21T15:01:53.592 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2097000000000001e-05 2024-09-21T15:01:53.592 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.592 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.592 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:01:53.593 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.372863+0000", 2024-09-21T15:01:53.593 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.032525923999999998 2024-09-21T15:01:53.593 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.593 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.593 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:53.593 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.372919+0000", 2024-09-21T15:01:53.593 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5729000000000001e-05 2024-09-21T15:01:53.593 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:53.593 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:53.593 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:53.593 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.593 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.593 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:116182 3.12 3:4efbaa3f:::rbd_data.1136b1194d20.0000000000000006:58 [read 6242304~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:53.593 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.712538+0000", 2024-09-21T15:01:53.594 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 83.868534036, 2024-09-21T15:01:53.594 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034702127999999999, 2024-09-21T15:01:53.594 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:53.594 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:53.594 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:53.594 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:53.594 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:53.594 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116182 2024-09-21T15:01:53.594 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.594 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:53.594 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.594 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:53.594 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712538+0000", 2024-09-21T15:01:53.594 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:53.595 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.595 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.595 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:53.595 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712538+0000", 2024-09-21T15:01:53.595 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:53.595 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.595 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.595 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:53.595 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712538+0000", 2024-09-21T15:01:53.595 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:53.595 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.595 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.595 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:53.595 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712541+0000", 2024-09-21T15:01:53.596 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5330000000000001e-06 2024-09-21T15:01:53.596 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.596 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.596 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:53.596 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712547+0000", 2024-09-21T15:01:53.596 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1109999999999996e-06 2024-09-21T15:01:53.596 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.596 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.596 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:53.596 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712551+0000", 2024-09-21T15:01:53.596 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.685e-06 2024-09-21T15:01:53.596 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.596 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.596 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:53.597 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.747067+0000", 2024-09-21T15:01:53.597 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034516181 2024-09-21T15:01:53.597 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.597 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.597 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:53.597 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.747152+0000", 2024-09-21T15:01:53.597 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4075999999999999e-05 2024-09-21T15:01:53.597 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.597 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.597 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:53.597 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.747240+0000", 2024-09-21T15:01:53.597 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8924000000000004e-05 2024-09-21T15:01:53.597 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:53.597 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:53.597 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:53.598 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.598 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.598 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139784 3.1 3:862e0e65:::rbd_data.113664780c85.0000000000000063:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:53.598 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.222486+0000", 2024-09-21T15:01:53.598 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 51.358586168999999, 2024-09-21T15:01:53.598 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0141793059999999, 2024-09-21T15:01:53.598 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:53.598 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:53.598 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:53.598 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:53.598 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:53.598 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139784 2024-09-21T15:01:53.598 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.599 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:53.599 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.599 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:53.599 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222486+0000", 2024-09-21T15:01:53.599 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:53.599 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.599 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.599 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:53.599 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222486+0000", 2024-09-21T15:01:53.599 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:53.599 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.599 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.599 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:53.599 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222484+0000", 2024-09-21T15:01:53.599 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-21T15:01:53.600 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.600 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.600 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:53.600 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222494+0000", 2024-09-21T15:01:53.600 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0676000000000001e-05 2024-09-21T15:01:53.600 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.600 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.600 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:53.600 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222517+0000", 2024-09-21T15:01:53.600 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2371e-05 2024-09-21T15:01:53.600 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.600 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.600 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:53.600 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222527+0000", 2024-09-21T15:01:53.601 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0461e-05 2024-09-21T15:01:53.601 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.601 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.601 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:53.601 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222625+0000", 2024-09-21T15:01:53.601 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.8160999999999997e-05 2024-09-21T15:01:53.601 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.601 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.601 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:53.601 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222641+0000", 2024-09-21T15:01:53.601 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5899000000000001e-05 2024-09-21T15:01:53.601 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.601 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.601 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:53.601 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.236479+0000", 2024-09-21T15:01:53.602 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0138377709999999 2024-09-21T15:01:53.602 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.602 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.602 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:53.602 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.236666+0000", 2024-09-21T15:01:53.602 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018648199999999999 2024-09-21T15:01:53.602 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:53.602 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:53.602 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:53.602 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.602 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.602 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139812 3.1 3:8407fb9b:::rbd_data.113664780c85.0000000000000127:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:53.602 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.222547+0000", 2024-09-21T15:01:53.603 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 51.358525245000003, 2024-09-21T15:01:53.603 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.014796059, 2024-09-21T15:01:53.603 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:53.603 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:53.603 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:53.603 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:53.603 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:53.603 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139812 2024-09-21T15:01:53.603 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.603 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:53.603 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.603 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:53.603 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222547+0000", 2024-09-21T15:01:53.603 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:53.603 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.604 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.604 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:53.604 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222547+0000", 2024-09-21T15:01:53.604 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:53.604 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.604 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.604 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:53.604 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222546+0000", 2024-09-21T15:01:53.604 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:53.604 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.604 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.604 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:53.604 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222552+0000", 2024-09-21T15:01:53.604 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2739999999999998e-06 2024-09-21T15:01:53.605 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.605 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.605 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:53.605 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222578+0000", 2024-09-21T15:01:53.605 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6594999999999999e-05 2024-09-21T15:01:53.605 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.605 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.605 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:53.605 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222596+0000", 2024-09-21T15:01:53.605 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7838000000000001e-05 2024-09-21T15:01:53.605 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.605 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.605 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:53.605 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222701+0000", 2024-09-21T15:01:53.605 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000104714 2024-09-21T15:01:53.606 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.606 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.606 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:53.606 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222714+0000", 2024-09-21T15:01:53.606 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2998e-05 2024-09-21T15:01:53.606 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.606 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.606 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:53.606 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.237124+0000", 2024-09-21T15:01:53.606 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.014409806 2024-09-21T15:01:53.606 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.606 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.606 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:53.606 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.237343+0000", 2024-09-21T15:01:53.607 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021950499999999999 2024-09-21T15:01:53.607 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:53.607 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:53.607 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:53.607 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.607 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.607 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139826 3.1 3:854f6f9b:::rbd_data.113664780c85.0000000000000189:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:53.607 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.222611+0000", 2024-09-21T15:01:53.607 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 51.358461153, 2024-09-21T15:01:53.607 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.014916828, 2024-09-21T15:01:53.607 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:53.607 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:53.607 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:53.607 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:53.608 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:53.608 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139826 2024-09-21T15:01:53.608 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.608 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:53.608 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.608 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:53.608 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222611+0000", 2024-09-21T15:01:53.608 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:53.608 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.608 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.608 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:53.608 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222611+0000", 2024-09-21T15:01:53.608 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:53.608 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.609 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.609 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:53.609 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222610+0000", 2024-09-21T15:01:53.609 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:53.609 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.609 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.609 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:53.609 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222618+0000", 2024-09-21T15:01:53.609 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6690000000000006e-06 2024-09-21T15:01:53.609 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.609 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.609 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:53.609 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222632+0000", 2024-09-21T15:01:53.609 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4433999999999999e-05 2024-09-21T15:01:53.609 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.610 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.610 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:53.610 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222640+0000", 2024-09-21T15:01:53.610 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6260000000000003e-06 2024-09-21T15:01:53.610 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.610 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.610 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:53.610 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222758+0000", 2024-09-21T15:01:53.610 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011826499999999999 2024-09-21T15:01:53.610 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.610 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.610 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:53.610 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222776+0000", 2024-09-21T15:01:53.610 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7564999999999999e-05 2024-09-21T15:01:53.611 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.611 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.611 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:53.611 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.237402+0000", 2024-09-21T15:01:53.611 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0146261889999999 2024-09-21T15:01:53.611 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.611 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.611 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:53.611 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.237528+0000", 2024-09-21T15:01:53.611 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000126212 2024-09-21T15:01:53.611 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:53.611 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:53.611 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:53.611 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.611 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.612 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139864 3.1 3:86b14cb7:::rbd_data.113664780c85.00000000000000d5:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:53.612 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.223966+0000", 2024-09-21T15:01:53.612 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 51.357106416999997, 2024-09-21T15:01:53.612 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.013859007, 2024-09-21T15:01:53.612 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:53.612 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:53.612 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:53.612 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:53.612 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:53.612 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139864 2024-09-21T15:01:53.612 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.612 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:53.612 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.612 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:53.613 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223966+0000", 2024-09-21T15:01:53.613 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:53.613 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.613 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.613 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:53.613 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223966+0000", 2024-09-21T15:01:53.613 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:53.613 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.613 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.613 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:53.613 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223965+0000", 2024-09-21T15:01:53.613 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:53.613 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.613 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.614 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:53.614 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223970+0000", 2024-09-21T15:01:53.614 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3769999999999998e-06 2024-09-21T15:01:53.614 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.614 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.614 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:53.614 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223979+0000", 2024-09-21T15:01:53.614 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5899999999999997e-06 2024-09-21T15:01:53.614 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.614 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.614 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:53.614 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223985+0000", 2024-09-21T15:01:53.614 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5790000000000003e-06 2024-09-21T15:01:53.614 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.615 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.615 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:53.615 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224073+0000", 2024-09-21T15:01:53.615 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7781000000000005e-05 2024-09-21T15:01:53.615 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.615 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.615 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:53.615 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224110+0000", 2024-09-21T15:01:53.615 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7060000000000001e-05 2024-09-21T15:01:53.615 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.615 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.615 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:53.615 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.237599+0000", 2024-09-21T15:01:53.615 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.01348953 2024-09-21T15:01:53.616 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.616 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.616 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:53.616 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.237825+0000", 2024-09-21T15:01:53.616 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000225758 2024-09-21T15:01:53.616 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:53.616 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:53.616 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:53.616 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.616 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.616 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139877 3.1 3:87277f92:::rbd_data.113664780c85.0000000000000136:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:53.616 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.224489+0000", 2024-09-21T15:01:53.616 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 51.356583559000001, 2024-09-21T15:01:53.616 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0137916330000001, 2024-09-21T15:01:53.617 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:53.617 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:53.617 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:53.617 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:53.617 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:53.617 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139877 2024-09-21T15:01:53.617 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.617 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:53.617 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.617 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:53.617 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224489+0000", 2024-09-21T15:01:53.617 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:53.617 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.617 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.618 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:53.618 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224489+0000", 2024-09-21T15:01:53.618 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:53.618 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.618 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.618 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:53.618 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224488+0000", 2024-09-21T15:01:53.618 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:53.618 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.618 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.618 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:53.618 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224502+0000", 2024-09-21T15:01:53.618 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4194e-05 2024-09-21T15:01:53.618 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.619 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.619 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:53.619 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224522+0000", 2024-09-21T15:01:53.619 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0038e-05 2024-09-21T15:01:53.619 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.619 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.619 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:53.619 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224531+0000", 2024-09-21T15:01:53.619 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6900000000000004e-06 2024-09-21T15:01:53.619 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.619 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.619 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:53.619 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224621+0000", 2024-09-21T15:01:53.619 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9654000000000006e-05 2024-09-21T15:01:53.620 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.620 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.620 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:53.620 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224641+0000", 2024-09-21T15:01:53.620 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9375e-05 2024-09-21T15:01:53.620 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.620 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.620 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:53.620 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.238027+0000", 2024-09-21T15:01:53.620 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0133866840000001 2024-09-21T15:01:53.620 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.620 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.620 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:53.620 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.238280+0000", 2024-09-21T15:01:53.621 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000253281 2024-09-21T15:01:53.621 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:53.621 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:53.621 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:53.621 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.621 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.621 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139887 3.1 3:80f1e8f1:::rbd_data.113664780c85.000000000000017c:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:53.621 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.224549+0000", 2024-09-21T15:01:53.621 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 51.356523269, 2024-09-21T15:01:53.621 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0139776060000001, 2024-09-21T15:01:53.621 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:53.621 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:53.621 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:53.621 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:53.621 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:53.622 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139887 2024-09-21T15:01:53.622 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.622 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:53.622 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.622 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:53.622 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224549+0000", 2024-09-21T15:01:53.622 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:53.622 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.622 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.622 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:53.622 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224549+0000", 2024-09-21T15:01:53.622 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:53.622 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.622 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.623 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:53.623 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224548+0000", 2024-09-21T15:01:53.623 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:53.623 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.623 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.623 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:53.623 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224554+0000", 2024-09-21T15:01:53.623 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4049999999999999e-06 2024-09-21T15:01:53.623 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.623 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.623 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:53.623 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224565+0000", 2024-09-21T15:01:53.623 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0879e-05 2024-09-21T15:01:53.623 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.624 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.624 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:53.624 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224572+0000", 2024-09-21T15:01:53.624 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0380000000000002e-06 2024-09-21T15:01:53.624 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.624 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.624 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:53.624 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224684+0000", 2024-09-21T15:01:53.624 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011229 2024-09-21T15:01:53.624 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.624 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.624 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:53.624 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224698+0000", 2024-09-21T15:01:53.624 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4448e-05 2024-09-21T15:01:53.625 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.625 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.625 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:53.625 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.238359+0000", 2024-09-21T15:01:53.625 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0136610660000001 2024-09-21T15:01:53.625 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.625 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.625 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:53.625 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.238527+0000", 2024-09-21T15:01:53.625 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016731100000000001 2024-09-21T15:01:53.625 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:53.625 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:53.625 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:53.626 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.626 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.626 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140077 3.1 3:85642daf:::rbd_data.113664780c85.0000000000000174:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:53.626 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.232797+0000", 2024-09-21T15:01:53.626 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 51.348275553999997, 2024-09-21T15:01:53.626 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.006029955, 2024-09-21T15:01:53.626 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:53.626 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:53.626 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:53.626 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:53.626 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:53.626 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140077 2024-09-21T15:01:53.626 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.626 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:53.627 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.627 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:53.627 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232797+0000", 2024-09-21T15:01:53.627 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:53.627 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.627 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.627 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:53.627 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232797+0000", 2024-09-21T15:01:53.627 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:53.627 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.627 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.627 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:53.627 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232795+0000", 2024-09-21T15:01:53.627 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:53.628 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.628 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.628 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:53.628 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232804+0000", 2024-09-21T15:01:53.628 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9670000000000007e-06 2024-09-21T15:01:53.628 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.628 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.628 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:53.628 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232824+0000", 2024-09-21T15:01:53.628 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9238000000000001e-05 2024-09-21T15:01:53.628 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.628 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.628 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:53.628 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232843+0000", 2024-09-21T15:01:53.629 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9256e-05 2024-09-21T15:01:53.629 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.629 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.629 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:53.629 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232912+0000", 2024-09-21T15:01:53.629 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9589000000000001e-05 2024-09-21T15:01:53.629 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.629 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.629 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:53.629 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232925+0000", 2024-09-21T15:01:53.629 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2585999999999999e-05 2024-09-21T15:01:53.629 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.629 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.629 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:53.630 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.238591+0000", 2024-09-21T15:01:53.630 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0056659939999999 2024-09-21T15:01:53.630 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.630 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.630 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:53.630 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.238827+0000", 2024-09-21T15:01:53.630 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00023583999999999999 2024-09-21T15:01:53.630 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:53.630 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:53.630 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:53.630 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.630 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.630 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140091 3.1 3:8358f0db:::rbd_data.113664780c85.0000000000000018:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:53.631 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.233637+0000", 2024-09-21T15:01:53.631 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 51.347435257999997, 2024-09-21T15:01:53.631 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.006272847, 2024-09-21T15:01:53.631 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:53.631 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:53.631 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:53.631 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:53.631 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:53.631 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140091 2024-09-21T15:01:53.631 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.631 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:53.631 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.631 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:53.631 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233637+0000", 2024-09-21T15:01:53.632 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:53.632 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.632 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.632 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:53.632 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233637+0000", 2024-09-21T15:01:53.632 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:53.632 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.632 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.632 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:53.632 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233636+0000", 2024-09-21T15:01:53.632 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:53.632 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.632 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.632 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:53.633 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233644+0000", 2024-09-21T15:01:53.633 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1380000000000007e-06 2024-09-21T15:01:53.633 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.633 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.633 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:53.633 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233662+0000", 2024-09-21T15:01:53.633 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8224999999999999e-05 2024-09-21T15:01:53.633 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.633 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.633 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:53.633 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233674+0000", 2024-09-21T15:01:53.633 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2081e-05 2024-09-21T15:01:53.633 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.633 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.633 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:53.634 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233750+0000", 2024-09-21T15:01:53.634 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6106000000000006e-05 2024-09-21T15:01:53.634 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.634 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.634 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:53.634 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233768+0000", 2024-09-21T15:01:53.634 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7285000000000001e-05 2024-09-21T15:01:53.634 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.634 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.634 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:53.634 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.239196+0000", 2024-09-21T15:01:53.634 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.005428223 2024-09-21T15:01:53.634 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.634 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.635 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:53.635 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.239528+0000", 2024-09-21T15:01:53.635 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00033262199999999997 2024-09-21T15:01:53.635 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.635 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.635 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:53.635 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.239910+0000", 2024-09-21T15:01:53.635 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00038151100000000001 2024-09-21T15:01:53.635 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:53.635 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:53.635 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:53.635 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.635 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.636 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140109 3.1 3:8586e9d9:::rbd_data.113664780c85.0000000000000096:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:53.636 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.234171+0000", 2024-09-21T15:01:53.636 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 51.346901424000002, 2024-09-21T15:01:53.636 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.006019896, 2024-09-21T15:01:53.636 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:53.636 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:53.636 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:53.636 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:53.636 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:53.636 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140109 2024-09-21T15:01:53.636 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.636 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:53.636 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.636 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:53.636 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.234171+0000", 2024-09-21T15:01:53.637 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:53.637 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.637 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.637 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:53.637 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.234171+0000", 2024-09-21T15:01:53.637 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:53.637 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.637 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.637 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:53.637 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.234170+0000", 2024-09-21T15:01:53.637 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:53.637 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.637 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.637 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:53.638 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.234178+0000", 2024-09-21T15:01:53.638 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.464e-06 2024-09-21T15:01:53.638 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.638 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.638 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:53.638 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.234191+0000", 2024-09-21T15:01:53.638 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3489000000000001e-05 2024-09-21T15:01:53.638 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.638 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.638 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:53.638 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.234200+0000", 2024-09-21T15:01:53.638 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2779999999999999e-06 2024-09-21T15:01:53.638 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.638 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.639 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:53.639 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.234279+0000", 2024-09-21T15:01:53.639 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8813000000000006e-05 2024-09-21T15:01:53.639 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.639 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.639 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:53.639 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.234298+0000", 2024-09-21T15:01:53.639 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8947999999999999e-05 2024-09-21T15:01:53.639 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.639 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.639 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:53.639 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.239974+0000", 2024-09-21T15:01:53.639 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0056760259999999 2024-09-21T15:01:53.640 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.640 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.640 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:53.640 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.240191+0000", 2024-09-21T15:01:53.640 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000216708 2024-09-21T15:01:53.640 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:53.640 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:53.640 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:53.640 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.640 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.640 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140149 3.1 3:86b1e433:::rbd_data.113664780c85.00000000000001ae:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:53.640 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.235545+0000", 2024-09-21T15:01:53.640 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 51.345527138999998, 2024-09-21T15:01:53.640 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0049193430000001, 2024-09-21T15:01:53.641 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:53.641 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:53.641 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:53.641 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:53.641 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:53.641 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140149 2024-09-21T15:01:53.641 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.641 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:53.641 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.641 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:53.641 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235545+0000", 2024-09-21T15:01:53.641 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:53.641 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.641 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.642 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:53.642 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235545+0000", 2024-09-21T15:01:53.642 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:53.642 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.642 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.642 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:53.642 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235544+0000", 2024-09-21T15:01:53.642 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:53.642 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.642 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.642 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:53.642 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235554+0000", 2024-09-21T15:01:53.642 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.8870000000000004e-06 2024-09-21T15:01:53.642 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.643 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.643 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:53.643 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235577+0000", 2024-09-21T15:01:53.643 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2946999999999999e-05 2024-09-21T15:01:53.643 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.643 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.643 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:53.643 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235587+0000", 2024-09-21T15:01:53.643 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0326000000000001e-05 2024-09-21T15:01:53.643 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.643 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.643 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:53.643 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235658+0000", 2024-09-21T15:01:53.643 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1360999999999997e-05 2024-09-21T15:01:53.644 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.644 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.644 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:53.644 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235670+0000", 2024-09-21T15:01:53.644 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1735e-05 2024-09-21T15:01:53.644 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.644 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.644 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:53.644 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.240246+0000", 2024-09-21T15:01:53.644 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0045757790000001 2024-09-21T15:01:53.644 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.644 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.644 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:53.644 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.240465+0000", 2024-09-21T15:01:53.645 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021870799999999999 2024-09-21T15:01:53.645 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:53.645 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:53.645 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:53.645 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.645 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.645 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140232 3.1 3:862e0e65:::rbd_data.113664780c85.0000000000000063:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:53.645 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.235760+0000", 2024-09-21T15:01:53.645 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 51.345312421000003, 2024-09-21T15:01:53.645 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.004979812, 2024-09-21T15:01:53.645 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:53.645 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:53.645 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:53.645 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:53.646 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:53.646 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140232 2024-09-21T15:01:53.646 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.646 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:53.646 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.646 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:53.646 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235760+0000", 2024-09-21T15:01:53.646 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:53.646 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.646 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.646 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:53.646 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235760+0000", 2024-09-21T15:01:53.646 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:53.647 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.647 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.647 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:53.647 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235759+0000", 2024-09-21T15:01:53.647 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:53.647 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.647 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.647 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:53.647 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235777+0000", 2024-09-21T15:01:53.647 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8215000000000001e-05 2024-09-21T15:01:53.647 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.647 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.647 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:53.647 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235793+0000", 2024-09-21T15:01:53.647 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6461e-05 2024-09-21T15:01:53.648 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.648 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.648 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:53.648 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235803+0000", 2024-09-21T15:01:53.648 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5300000000000002e-06 2024-09-21T15:01:53.648 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.648 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.648 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:53.648 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235875+0000", 2024-09-21T15:01:53.648 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2113000000000006e-05 2024-09-21T15:01:53.648 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.648 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.648 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:53.648 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235887+0000", 2024-09-21T15:01:53.649 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2160999999999999e-05 2024-09-21T15:01:53.649 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.649 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.649 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:53.649 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.240521+0000", 2024-09-21T15:01:53.649 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.004633428 2024-09-21T15:01:53.649 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.649 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.649 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:53.649 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.240740+0000", 2024-09-21T15:01:53.649 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000219228 2024-09-21T15:01:53.649 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:53.649 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:53.649 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:53.650 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.650 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.650 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140260 3.1 3:8407fb9b:::rbd_data.113664780c85.0000000000000127:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:53.650 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.235839+0000", 2024-09-21T15:01:53.650 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 51.345233252, 2024-09-21T15:01:53.650 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.005157981, 2024-09-21T15:01:53.650 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:53.650 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:53.650 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:53.650 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:53.650 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:53.650 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140260 2024-09-21T15:01:53.650 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.650 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:53.651 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.651 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:53.651 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235839+0000", 2024-09-21T15:01:53.651 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:53.651 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.651 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.651 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:53.651 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235839+0000", 2024-09-21T15:01:53.651 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:53.651 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.651 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.651 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:53.651 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235838+0000", 2024-09-21T15:01:53.651 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:53.652 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.652 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.652 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:53.652 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235843+0000", 2024-09-21T15:01:53.652 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.481e-06 2024-09-21T15:01:53.652 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.652 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.652 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:53.652 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235849+0000", 2024-09-21T15:01:53.652 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4880000000000004e-06 2024-09-21T15:01:53.652 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.652 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.652 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:53.652 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235856+0000", 2024-09-21T15:01:53.653 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7649999999999996e-06 2024-09-21T15:01:53.653 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.653 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.653 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:53.653 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235933+0000", 2024-09-21T15:01:53.653 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6987e-05 2024-09-21T15:01:53.653 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.653 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.653 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:53.653 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235949+0000", 2024-09-21T15:01:53.653 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6696000000000001e-05 2024-09-21T15:01:53.653 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.653 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.653 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:53.654 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.240823+0000", 2024-09-21T15:01:53.654 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.004873259 2024-09-21T15:01:53.654 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.654 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.654 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:53.654 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.240997+0000", 2024-09-21T15:01:53.654 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000174365 2024-09-21T15:01:53.654 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:53.654 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:53.654 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:53.654 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.654 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.654 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140274 3.1 3:854f6f9b:::rbd_data.113664780c85.0000000000000189:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:53.654 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.235884+0000", 2024-09-21T15:01:53.655 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 51.345188303999997, 2024-09-21T15:01:53.655 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.005397366, 2024-09-21T15:01:53.655 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:53.655 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:53.655 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:53.655 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:53.655 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:53.655 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140274 2024-09-21T15:01:53.655 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.655 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:53.655 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.655 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:53.655 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235884+0000", 2024-09-21T15:01:53.655 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:53.656 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.656 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.656 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:53.656 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235884+0000", 2024-09-21T15:01:53.656 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:53.656 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.656 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.656 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:53.656 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235883+0000", 2024-09-21T15:01:53.656 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:53.656 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.656 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.656 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:53.656 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235891+0000", 2024-09-21T15:01:53.657 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9740000000000002e-06 2024-09-21T15:01:53.657 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.657 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.657 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:53.657 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235900+0000", 2024-09-21T15:01:53.657 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6659999999999997e-06 2024-09-21T15:01:53.657 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.657 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.657 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:53.657 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235909+0000", 2024-09-21T15:01:53.657 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3400000000000004e-06 2024-09-21T15:01:53.657 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.657 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.657 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:53.658 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236012+0000", 2024-09-21T15:01:53.658 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010240500000000001 2024-09-21T15:01:53.658 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.658 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.658 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:53.658 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236027+0000", 2024-09-21T15:01:53.658 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5418e-05 2024-09-21T15:01:53.658 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.658 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.658 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:53.658 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.241068+0000", 2024-09-21T15:01:53.658 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0050410729999999 2024-09-21T15:01:53.658 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.658 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.659 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:53.659 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.241281+0000", 2024-09-21T15:01:53.659 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021321900000000001 2024-09-21T15:01:53.659 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:53.659 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:53.659 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:53.659 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.659 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.659 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140312 3.1 3:86b14cb7:::rbd_data.113664780c85.00000000000000d5:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:53.659 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.236769+0000", 2024-09-21T15:01:53.659 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 51.344303840000002, 2024-09-21T15:01:53.659 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.004717198, 2024-09-21T15:01:53.659 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:53.659 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:53.660 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:53.660 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:53.660 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:53.660 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140312 2024-09-21T15:01:53.660 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.660 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:53.660 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.660 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:53.660 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236769+0000", 2024-09-21T15:01:53.660 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:53.660 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.660 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.660 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:53.660 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236769+0000", 2024-09-21T15:01:53.661 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:53.661 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.661 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.661 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:53.661 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236768+0000", 2024-09-21T15:01:53.661 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:53.661 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.661 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.661 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:53.661 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236771+0000", 2024-09-21T15:01:53.661 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.331e-06 2024-09-21T15:01:53.661 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.661 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.661 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:53.662 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236777+0000", 2024-09-21T15:01:53.662 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6949999999999997e-06 2024-09-21T15:01:53.662 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.662 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.662 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:53.662 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236787+0000", 2024-09-21T15:01:53.662 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6700000000000006e-06 2024-09-21T15:01:53.662 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.662 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.662 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:53.662 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236846+0000", 2024-09-21T15:01:53.662 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.944e-05 2024-09-21T15:01:53.662 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.662 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.663 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:53.663 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236861+0000", 2024-09-21T15:01:53.663 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4611e-05 2024-09-21T15:01:53.663 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.663 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.663 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:53.663 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.241349+0000", 2024-09-21T15:01:53.663 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0044879250000001 2024-09-21T15:01:53.663 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.663 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.663 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:53.663 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.241486+0000", 2024-09-21T15:01:53.663 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013713000000000001 2024-09-21T15:01:53.663 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:53.664 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:53.664 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:53.664 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.664 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.664 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140325 3.1 3:87277f92:::rbd_data.113664780c85.0000000000000136:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:53.664 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.239351+0000", 2024-09-21T15:01:53.664 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 51.341721700999997, 2024-09-21T15:01:53.664 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0024334690000001, 2024-09-21T15:01:53.664 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:53.664 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:53.664 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:53.664 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:53.664 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:53.664 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140325 2024-09-21T15:01:53.665 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.665 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:53.665 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.665 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:53.665 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239351+0000", 2024-09-21T15:01:53.665 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:53.665 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.665 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.665 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:53.665 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239351+0000", 2024-09-21T15:01:53.665 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:53.665 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.665 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.665 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:53.666 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239349+0000", 2024-09-21T15:01:53.666 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:53.666 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.666 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.666 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:53.666 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239358+0000", 2024-09-21T15:01:53.666 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2830000000000001e-06 2024-09-21T15:01:53.666 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.666 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.666 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:53.666 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239379+0000", 2024-09-21T15:01:53.666 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1780000000000002e-05 2024-09-21T15:01:53.666 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.666 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.667 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:53.667 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239389+0000", 2024-09-21T15:01:53.667 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0056e-05 2024-09-21T15:01:53.667 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.667 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.667 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:53.667 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239484+0000", 2024-09-21T15:01:53.667 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4611000000000006e-05 2024-09-21T15:01:53.667 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.667 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.667 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:53.667 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239497+0000", 2024-09-21T15:01:53.667 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2909e-05 2024-09-21T15:01:53.667 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.668 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.668 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:53.668 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.241559+0000", 2024-09-21T15:01:53.668 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0020622459999999 2024-09-21T15:01:53.668 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.668 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.668 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:53.668 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.241784+0000", 2024-09-21T15:01:53.668 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000225009 2024-09-21T15:01:53.668 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:53.668 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:53.668 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:53.668 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.668 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.668 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140335 3.1 3:80f1e8f1:::rbd_data.113664780c85.000000000000017c:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:53.669 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.239409+0000", 2024-09-21T15:01:53.669 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 51.341662982000003, 2024-09-21T15:01:53.669 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0025803150000001, 2024-09-21T15:01:53.669 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:53.669 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:53.669 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:53.669 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:53.669 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:53.669 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140335 2024-09-21T15:01:53.669 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.669 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:53.669 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.669 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:53.669 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239409+0000", 2024-09-21T15:01:53.670 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:53.670 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.670 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.670 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:53.670 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239409+0000", 2024-09-21T15:01:53.670 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:53.670 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.670 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.670 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:53.670 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239409+0000", 2024-09-21T15:01:53.670 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:53.670 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.670 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.670 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:53.671 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239413+0000", 2024-09-21T15:01:53.671 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7620000000000003e-06 2024-09-21T15:01:53.671 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.671 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.671 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:53.671 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239436+0000", 2024-09-21T15:01:53.671 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2889000000000001e-05 2024-09-21T15:01:53.671 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.671 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.671 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:53.671 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239444+0000", 2024-09-21T15:01:53.671 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6809999999999998e-06 2024-09-21T15:01:53.671 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.671 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.672 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:53.672 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239542+0000", 2024-09-21T15:01:53.672 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7868999999999994e-05 2024-09-21T15:01:53.672 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.672 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.672 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:53.672 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239558+0000", 2024-09-21T15:01:53.672 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6257e-05 2024-09-21T15:01:53.672 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.672 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.672 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:53.672 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.241847+0000", 2024-09-21T15:01:53.672 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.002288533 2024-09-21T15:01:53.672 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.673 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.673 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:53.673 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.241990+0000", 2024-09-21T15:01:53.673 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014308300000000001 2024-09-21T15:01:53.673 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:53.673 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:53.673 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:53.673 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.673 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.673 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:166488 3.4 3.7bcf3e84 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e189)", 2024-09-21T15:01:53.673 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:21.342023+0000", 2024-09-21T15:01:53.673 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 32.23904915, 2024-09-21T15:01:53.673 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0015098469999999, 2024-09-21T15:01:53.673 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:53.674 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:53.674 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:53.674 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:53.674 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:53.674 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 166488 2024-09-21T15:01:53.674 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.674 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:53.674 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.674 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:53.674 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:21.342023+0000", 2024-09-21T15:01:53.674 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:53.674 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.674 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.675 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:53.675 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:21.342023+0000", 2024-09-21T15:01:53.675 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:53.675 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.675 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.675 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:53.675 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:21.342022+0000", 2024-09-21T15:01:53.675 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:53.675 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.675 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.675 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:53.675 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:21.342029+0000", 2024-09-21T15:01:53.675 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1420000000000004e-06 2024-09-21T15:01:53.675 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.675 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.676 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:53.676 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:21.342051+0000", 2024-09-21T15:01:53.676 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2549999999999999e-05 2024-09-21T15:01:53.676 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.676 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.676 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:53.676 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:21.342059+0000", 2024-09-21T15:01:53.676 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8490000000000006e-06 2024-09-21T15:01:53.676 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.676 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.676 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:53.676 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:21.342105+0000", 2024-09-21T15:01:53.676 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5488999999999998e-05 2024-09-21T15:01:53.676 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.677 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.677 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:53.677 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:21.342115+0000", 2024-09-21T15:01:53.677 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0633e-05 2024-09-21T15:01:53.677 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.677 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.677 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:53.677 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:22.343521+0000", 2024-09-21T15:01:53.677 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0014057270000001 2024-09-21T15:01:53.677 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:53.677 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:53.677 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:53.677 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:22.343533+0000", 2024-09-21T15:01:53.677 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2097e-05 2024-09-21T15:01:53.677 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:53.678 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:53.678 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:53.678 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:53.678 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:53.678 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:01:53.679 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T15:01:53.680 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:01:53.677+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:01:53.780 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:01:53.777+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:01:53.795 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:01:53.795 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:01:53.795 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T15:01:53.795 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:01:53.805 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T15:01:53.881 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:01:53.876+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:01:53.913 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:01:53.914 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:01:53.914 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T15:01:53.914 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T15:01:53.914 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:01:53.923 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T15:01:53.982 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:01:53.976+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:01:54.080 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:01:54.081 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T15:01:54.081 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T15:01:54.081 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T15:01:54.081 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.081 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31476 3.c 3:35718b6b:::rbd_data.11367721c19.0000000000000014:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T15:01:54.081 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.505975+0000", 2024-09-21T15:01:54.081 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 175.56991774599999, 2024-09-21T15:01:54.081 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99515995899999998, 2024-09-21T15:01:54.081 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:54.081 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:54.081 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:54.081 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:54.082 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:54.082 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31476 2024-09-21T15:01:54.082 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.082 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:54.082 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.082 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:54.082 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.505975+0000", 2024-09-21T15:01:54.082 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:54.082 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.082 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.082 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:54.082 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.505975+0000", 2024-09-21T15:01:54.082 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:54.082 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.083 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.083 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:54.083 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.505971+0000", 2024-09-21T15:01:54.083 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999967 2024-09-21T15:01:54.083 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.083 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.083 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:54.083 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.505989+0000", 2024-09-21T15:01:54.083 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7985000000000001e-05 2024-09-21T15:01:54.083 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.083 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.083 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:54.083 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506050+0000", 2024-09-21T15:01:54.084 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0659000000000003e-05 2024-09-21T15:01:54.084 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.084 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.084 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:54.084 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506074+0000", 2024-09-21T15:01:54.084 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4156e-05 2024-09-21T15:01:54.084 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.084 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.084 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:54.084 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506225+0000", 2024-09-21T15:01:54.084 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015097 2024-09-21T15:01:54.084 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.084 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.084 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:54.085 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506256+0000", 2024-09-21T15:01:54.085 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0505999999999999e-05 2024-09-21T15:01:54.085 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.085 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.085 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:54.085 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.497974+0000", 2024-09-21T15:01:54.085 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99171825700000005 2024-09-21T15:01:54.085 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.085 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.085 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for missing object", 2024-09-21T15:01:54.085 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.498449+0000", 2024-09-21T15:01:54.085 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00047506399999999998 2024-09-21T15:01:54.085 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.085 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.086 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:54.086 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.501000+0000", 2024-09-21T15:01:54.086 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0025506919999999998 2024-09-21T15:01:54.086 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.086 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.086 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:54.086 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.501135+0000", 2024-09-21T15:01:54.086 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013524000000000001 2024-09-21T15:01:54.086 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:54.086 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:54.086 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:54.086 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.086 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.086 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31480 3.c 3:36ccd808:::rbd_data.11367721c19.0000000000000018:head [sparse-read 0~14821] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T15:01:54.087 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.506116+0000", 2024-09-21T15:01:54.087 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 175.56977640299999, 2024-09-21T15:01:54.087 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99316033599999998, 2024-09-21T15:01:54.087 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:54.087 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:54.087 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:54.087 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:54.087 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:54.087 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31480 2024-09-21T15:01:54.087 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.087 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:54.087 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.087 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:54.088 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506116+0000", 2024-09-21T15:01:54.088 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:54.088 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.088 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.088 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:54.088 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506116+0000", 2024-09-21T15:01:54.088 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:54.088 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.088 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.088 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:54.088 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506115+0000", 2024-09-21T15:01:54.088 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:54.088 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.088 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.088 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:54.089 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506124+0000", 2024-09-21T15:01:54.089 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0659999999999993e-06 2024-09-21T15:01:54.089 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.089 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.089 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:54.089 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506145+0000", 2024-09-21T15:01:54.089 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1447e-05 2024-09-21T15:01:54.089 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.089 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.089 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:54.089 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506161+0000", 2024-09-21T15:01:54.089 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5209e-05 2024-09-21T15:01:54.089 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.089 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.090 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:54.090 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506337+0000", 2024-09-21T15:01:54.090 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017642999999999999 2024-09-21T15:01:54.090 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.090 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.090 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:54.090 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506375+0000", 2024-09-21T15:01:54.090 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7536000000000001e-05 2024-09-21T15:01:54.090 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.090 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.090 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:54.090 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.498815+0000", 2024-09-21T15:01:54.090 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99244067999999996 2024-09-21T15:01:54.090 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.091 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.091 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:54.091 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.498950+0000", 2024-09-21T15:01:54.091 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013436700000000001 2024-09-21T15:01:54.091 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.091 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.091 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:54.091 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499277+0000", 2024-09-21T15:01:54.091 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00032696800000000002 2024-09-21T15:01:54.091 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:54.091 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:54.091 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:54.091 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.091 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.092 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31513 3.c 3:36de8523:::rbd_data.11367721c19.0000000000000039:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T15:01:54.092 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.509071+0000", 2024-09-21T15:01:54.092 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 175.56682220299999, 2024-09-21T15:01:54.092 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99038225099999999, 2024-09-21T15:01:54.093 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:54.093 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:54.094 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:54.094 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:54.094 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:54.094 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31513 2024-09-21T15:01:54.094 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.094 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:54.094 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.094 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:54.094 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509071+0000", 2024-09-21T15:01:54.094 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:54.094 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.094 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.094 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:54.095 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509071+0000", 2024-09-21T15:01:54.095 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:54.095 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.095 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.095 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:54.095 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509068+0000", 2024-09-21T15:01:54.095 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-21T15:01:54.095 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.095 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.095 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:54.095 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509082+0000", 2024-09-21T15:01:54.095 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3545e-05 2024-09-21T15:01:54.095 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.096 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.096 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:54.096 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509116+0000", 2024-09-21T15:01:54.096 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3942000000000001e-05 2024-09-21T15:01:54.096 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.096 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.096 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:54.096 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509134+0000", 2024-09-21T15:01:54.096 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8839999999999999e-05 2024-09-21T15:01:54.096 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.096 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.096 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:54.096 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509318+0000", 2024-09-21T15:01:54.097 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018326799999999999 2024-09-21T15:01:54.097 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.097 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.097 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:54.097 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509346+0000", 2024-09-21T15:01:54.097 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8019000000000001e-05 2024-09-21T15:01:54.097 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.097 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.097 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:54.097 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499330+0000", 2024-09-21T15:01:54.097 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.98998399299999995 2024-09-21T15:01:54.097 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.097 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.098 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:54.098 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499453+0000", 2024-09-21T15:01:54.098 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000123115 2024-09-21T15:01:54.098 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:54.098 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:54.098 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:54.098 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.098 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.098 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31515 3.c 3:35b36ddd:::rbd_data.11367721c19.000000000000003b:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T15:01:54.098 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.509177+0000", 2024-09-21T15:01:54.098 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 175.56671577700001, 2024-09-21T15:01:54.099 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99044870600000001, 2024-09-21T15:01:54.099 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:54.099 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:54.099 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:54.099 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:54.099 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:54.099 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31515 2024-09-21T15:01:54.099 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.099 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:54.099 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.099 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:54.099 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509177+0000", 2024-09-21T15:01:54.099 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:54.100 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.100 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.100 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:54.100 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509177+0000", 2024-09-21T15:01:54.100 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:54.100 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.100 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.100 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:54.100 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509176+0000", 2024-09-21T15:01:54.100 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:54.100 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.100 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.100 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:54.100 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509183+0000", 2024-09-21T15:01:54.101 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6720000000000004e-06 2024-09-21T15:01:54.101 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.101 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.101 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:54.101 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509269+0000", 2024-09-21T15:01:54.101 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5881e-05 2024-09-21T15:01:54.101 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.101 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.101 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:54.101 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509303+0000", 2024-09-21T15:01:54.101 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3367000000000001e-05 2024-09-21T15:01:54.101 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.101 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.101 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:54.102 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509425+0000", 2024-09-21T15:01:54.102 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000122199 2024-09-21T15:01:54.102 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.102 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.102 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:54.102 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509448+0000", 2024-09-21T15:01:54.102 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2688999999999999e-05 2024-09-21T15:01:54.102 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.102 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.102 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:54.102 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499501+0000", 2024-09-21T15:01:54.102 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99005306999999998 2024-09-21T15:01:54.102 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.103 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.103 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:54.103 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499626+0000", 2024-09-21T15:01:54.103 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012501400000000001 2024-09-21T15:01:54.103 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:54.103 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:54.103 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:54.103 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.103 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.103 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31531 3.c 3:314f0287:::rbd_data.11367721c19.000000000000004b:head [sparse-read 0~7106] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T15:01:54.103 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.509329+0000", 2024-09-21T15:01:54.103 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 175.56656375700001, 2024-09-21T15:01:54.103 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99103085300000004, 2024-09-21T15:01:54.104 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:54.104 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:54.104 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:54.104 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:54.104 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:54.104 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31531 2024-09-21T15:01:54.104 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.104 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:54.104 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.104 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:54.104 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509329+0000", 2024-09-21T15:01:54.104 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:54.104 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.105 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.105 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:54.105 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509329+0000", 2024-09-21T15:01:54.105 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:54.105 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.105 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.105 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:54.105 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509328+0000", 2024-09-21T15:01:54.105 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:54.105 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.105 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.105 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:54.105 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509336+0000", 2024-09-21T15:01:54.105 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3170000000000002e-06 2024-09-21T15:01:54.106 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.106 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.106 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:54.106 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509353+0000", 2024-09-21T15:01:54.106 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7214000000000002e-05 2024-09-21T15:01:54.106 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.106 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.106 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:54.106 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509366+0000", 2024-09-21T15:01:54.106 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.26e-05 2024-09-21T15:01:54.106 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.106 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.106 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:54.107 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509523+0000", 2024-09-21T15:01:54.107 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015708000000000001 2024-09-21T15:01:54.107 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.107 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.107 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:54.107 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509544+0000", 2024-09-21T15:01:54.107 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0783000000000001e-05 2024-09-21T15:01:54.107 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.107 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.107 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:54.107 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499727+0000", 2024-09-21T15:01:54.107 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99018345200000002 2024-09-21T15:01:54.107 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.107 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.108 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:54.108 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499949+0000", 2024-09-21T15:01:54.108 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000221358 2024-09-21T15:01:54.108 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.108 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.108 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:54.108 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.500360+0000", 2024-09-21T15:01:54.108 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00041130700000000002 2024-09-21T15:01:54.108 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:54.108 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:54.108 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:54.108 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.108 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.108 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:82545 3.b 3:d4eb1ee5:::rbd_data.11368296f791.0000000000000000:head [write 0~671744 [fadvise_sequential+fadvise_nocache],write 704512~65536 [fadvise_sequential+fadvise_nocache],write 786432~49152 [fadvise_sequential+fadvise_nocache],write 868352~49152 [fadvise_sequential+fadvise_nocache],write 933888~65536 [fadvise_sequential+fadvise_nocache],write 1032192~32768 [fadvise_sequential+fadvise_nocache],write 1097728~65536 [fadvise_sequential+fadvise_nocache],write 1179648~49152 [fadvise_sequential+fadvise_nocache],write 1261568~49152 [fadvise_sequential+fadvise_nocache],write 1327104~65536 [fadvise_sequential+fadvise_nocache],write 1425408~32768 [fadvise_sequential+fadvise_nocache],write 1490944~65536 [fadvise_sequential+fadvise_nocache],write 1572864~8192 [fadvise_sequential+fadvise_nocache],write 1630208~49152 [fadvise_sequential+fadvise_nocache],write 1703936~40960 [fadvise_sequential+fadvise_nocache],write 1794048~40960 [fadvise_sequential+fadvise_nocache],write 1859584~49152 [fadvise_sequential+fadvise_nocache],write 1957888~8192 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:01:54.108 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:47.416089+0000", 2024-09-21T15:01:54.109 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 126.659804032, 2024-09-21T15:01:54.109 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011896455, 2024-09-21T15:01:54.109 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:54.109 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:54.109 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:54.109 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:54.109 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:54.109 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82545 2024-09-21T15:01:54.109 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.109 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:54.109 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.109 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:54.109 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.416089+0000", 2024-09-21T15:01:54.109 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:54.110 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.110 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.110 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:54.110 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.416089+0000", 2024-09-21T15:01:54.110 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:54.110 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.110 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.110 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:54.110 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.416087+0000", 2024-09-21T15:01:54.110 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:01:54.110 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.110 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.110 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:54.110 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.419643+0000", 2024-09-21T15:01:54.111 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0035555550000000002 2024-09-21T15:01:54.111 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.111 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.111 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:54.111 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.419663+0000", 2024-09-21T15:01:54.111 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0202000000000001e-05 2024-09-21T15:01:54.111 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.111 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.111 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:54.111 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.419676+0000", 2024-09-21T15:01:54.111 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3207000000000001e-05 2024-09-21T15:01:54.111 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.111 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.111 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:54.112 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.419759+0000", 2024-09-21T15:01:54.112 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3521000000000002e-05 2024-09-21T15:01:54.112 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.112 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.112 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op must wait for map", 2024-09-21T15:01:54.112 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.419768+0000", 2024-09-21T15:01:54.112 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4479999999999994e-06 2024-09-21T15:01:54.112 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.112 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.112 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:54.112 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.422171+0000", 2024-09-21T15:01:54.112 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0024035469999999998 2024-09-21T15:01:54.112 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.112 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.112 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:54.113 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.422350+0000", 2024-09-21T15:01:54.113 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000179001 2024-09-21T15:01:54.113 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.113 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.113 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T15:01:54.113 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.422710+0000", 2024-09-21T15:01:54.113 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000359878 2024-09-21T15:01:54.113 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.113 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.113 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:01:54.113 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.426676+0000", 2024-09-21T15:01:54.113 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0039655790000000003 2024-09-21T15:01:54.113 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.113 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.113 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:01:54.114 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.427884+0000", 2024-09-21T15:01:54.114 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001208207 2024-09-21T15:01:54.114 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.114 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.114 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:01:54.114 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.427953+0000", 2024-09-21T15:01:54.114 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8533000000000003e-05 2024-09-21T15:01:54.114 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.114 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.114 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:54.114 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.427985+0000", 2024-09-21T15:01:54.114 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2481999999999998e-05 2024-09-21T15:01:54.114 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:54.114 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:54.115 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:54.115 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.115 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.115 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:97675 3.b 3:d321acb8:::rbd_data.1136bfc79a70.0000000000000003:head [write 0~4718592 [fadvise_sequential+fadvise_nocache],write 5242880~147456 [fadvise_sequential+fadvise_nocache],write 5455872~147456 [fadvise_sequential+fadvise_nocache],write 5668864~229376 [fadvise_sequential+fadvise_nocache],write 5963776~147456 [fadvise_sequential+fadvise_nocache],write 6176768~229376 [fadvise_sequential+fadvise_nocache],write 6471680~147456 [fadvise_sequential+fadvise_nocache],write 6684672~229376 [fadvise_sequential+fadvise_nocache],write 6979584~147456 [fadvise_sequential+fadvise_nocache],write 7192576~229376 [fadvise_sequential+fadvise_nocache],write 7487488~147456 [fadvise_sequential+fadvise_nocache],write 7700480~147456 [fadvise_sequential+fadvise_nocache],write 7864320~65536 [fadvise_sequential+fadvise_nocache],write 7995392~147456 [fadvise_sequential+fadvise_nocache],write 8208384~147456 [fadvise_sequential+fadvise_nocache],write 8388608~49152 [fadvise_sequential+fadvise_nocache],write 8536064~114688 [fadvise_sequential+fadvise_nocache],write 8749056~114688 [fadvise_sequential+fadvise_nocache],write 8912896~32768 [fadvise_sequential+fadvise_nocache],write 9125888~32768 [fadvise_sequential+fadvise_nocache],write 9338880~32768 [fadvise_sequential+fadvise_nocache],write 9437184~16384 [fadvise_sequential+fadvise_nocache],write 9633792~32768 [fadvise_sequential+fadvise_nocache],write 9846784~32768 [fadvise_sequential+fadvise_nocache],write 9961472~114688 [fadvise_sequential+fadvise_nocache],write 10092544~196608 [fadvise_sequential+fadvise_nocache],write 10305536~278528 [fadvise_sequential+fadvise_nocache],write 10600448~196608 [fadvise_sequential+fadvise_nocache],write 10813440~278528 [fadvise_sequential+fadvise_nocache],write 11108352~196608 [fadvise_sequential+fadvise_nocache],write 11321344~196608 [fadvise_sequential+fadvise_nocache],write 11534336~65536 [fadvise_sequential+fadvise_nocache],write 11632640~98304 [fadvise_sequential+fadvise_nocache],write 11747328~65536 [fadvise_sequential+fadvise_nocache],write 11845632~98304 [fadvise_sequential+fadvise_nocache],write 11960320~65536 [fadvise_sequential+fadvise_nocache],write 12058624~49152 [fadvise_sequential+fadvise_nocache],write 12140544~98304 [fadvise_sequential+fadvise_nocache],write 12255232~65536 [fadvise_sequential+fadvise_nocache],write 12353536~98304 [fadvise_sequential+fadvise_nocache],write 12468224~65536 [fadvise_sequential+fadvise_nocache],write 12566528~16384 [fadvise_sequential+fadvise_nocache],write 12648448~98304 [fadvise_sequential+fadvise_nocache],write 12861440~98304 [fadvise_sequential+fadvise_nocache],write 13074432~32768 [fadvise_sequential+fadvise_nocache],write 13156352~98304 [fadvise_sequential+fadvise_nocache],write 13369344~98304 [fadvise_sequential+fadvise_nocache],write 13582336~49152 [fadvise_sequential+fadvise_nocache],write 13664256~98304 [fadvise_sequential+fadvise_nocache],write 13877248~98304 [fadvise_sequential+fadvise_nocache],write 14090240~65536 [fadvise_sequential+fadvise_nocache],write 15286272~81920 [fadvise_sequential+fadvise_nocache],write 15499264~81920 [fadvise_sequential+fadvise_nocache],write 15712256~16384 [fadvise_sequential+fadvise_nocache],write 15794176~81920 [fadvise_sequential+fadvise_nocache],write 16007168~81920 [fadvise_sequential+fadvise_nocache],write 16220160~32768 [fadvise_sequential+fadvise_nocache],write 16302080~81920 [fadvise_sequential+fadvise_nocache],write 16515072~81920 [fadvise_sequential+fadvise_nocache],write 16728064~49152 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:01:54.115 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.302412+0000", 2024-09-21T15:01:54.115 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 108.77348124300001, 2024-09-21T15:01:54.115 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.067380532000000007, 2024-09-21T15:01:54.115 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:54.115 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:54.115 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:54.115 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:54.115 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:54.115 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97675 2024-09-21T15:01:54.115 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.116 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:54.116 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.116 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:54.116 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.302412+0000", 2024-09-21T15:01:54.116 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:54.116 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.116 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.116 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:54.116 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.302412+0000", 2024-09-21T15:01:54.116 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:54.116 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.116 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.116 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:54.116 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.302409+0000", 2024-09-21T15:01:54.116 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-21T15:01:54.117 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.117 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.117 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:54.117 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328550+0000", 2024-09-21T15:01:54.117 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.026140674999999999 2024-09-21T15:01:54.117 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.117 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.117 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:54.117 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328584+0000", 2024-09-21T15:01:54.117 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4165999999999998e-05 2024-09-21T15:01:54.117 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.117 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.117 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:54.117 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328599+0000", 2024-09-21T15:01:54.117 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5505000000000001e-05 2024-09-21T15:01:54.118 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.118 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.118 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:54.118 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328704+0000", 2024-09-21T15:01:54.118 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010487199999999999 2024-09-21T15:01:54.118 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.118 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.118 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:54.118 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.329082+0000", 2024-09-21T15:01:54.118 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00037804899999999998 2024-09-21T15:01:54.118 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.118 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.118 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T15:01:54.118 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.331158+0000", 2024-09-21T15:01:54.119 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0020755449999999998 2024-09-21T15:01:54.119 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.119 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.119 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:01:54.119 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.360553+0000", 2024-09-21T15:01:54.119 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.029395553000000001 2024-09-21T15:01:54.119 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.119 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.119 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:01:54.119 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.369496+0000", 2024-09-21T15:01:54.119 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0089428619999999993 2024-09-21T15:01:54.119 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.119 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.119 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:01:54.120 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.369686+0000", 2024-09-21T15:01:54.120 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019000399999999999 2024-09-21T15:01:54.120 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.120 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.120 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:54.120 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.369792+0000", 2024-09-21T15:01:54.120 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000105775 2024-09-21T15:01:54.120 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:54.120 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:54.120 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:54.120 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.120 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.120 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:97693 3.b 3:d7d7b39b:::rbd_data.1136b9c49b1c.000000000000000c:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:01:54.120 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.328643+0000", 2024-09-21T15:01:54.120 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 108.747249457, 2024-09-21T15:01:54.121 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.031062136000000001, 2024-09-21T15:01:54.121 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:54.121 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:54.121 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:54.121 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:54.121 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:54.121 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97693 2024-09-21T15:01:54.121 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.121 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:54.121 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.121 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:54.121 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328643+0000", 2024-09-21T15:01:54.121 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:54.121 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.121 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.121 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:54.122 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328643+0000", 2024-09-21T15:01:54.122 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:54.122 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.122 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.122 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:54.122 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328642+0000", 2024-09-21T15:01:54.122 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:01:54.122 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.122 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.122 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:54.122 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328650+0000", 2024-09-21T15:01:54.122 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7770000000000008e-06 2024-09-21T15:01:54.122 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.122 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.123 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:54.123 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328662+0000", 2024-09-21T15:01:54.123 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2026e-05 2024-09-21T15:01:54.123 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.123 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.123 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:54.123 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328671+0000", 2024-09-21T15:01:54.123 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.585e-06 2024-09-21T15:01:54.123 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.123 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.123 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:54.123 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.359498+0000", 2024-09-21T15:01:54.123 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.030827001 2024-09-21T15:01:54.124 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.124 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.124 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:54.124 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.359603+0000", 2024-09-21T15:01:54.124 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010512400000000001 2024-09-21T15:01:54.124 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.124 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.124 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:54.124 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.359705+0000", 2024-09-21T15:01:54.124 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000102327 2024-09-21T15:01:54.124 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:54.124 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:54.124 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:54.124 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.125 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.125 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:97724 3.b 3:d4bf6bcf:::rbd_data.1136b9c49b1c.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:01:54.125 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.328704+0000", 2024-09-21T15:01:54.125 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 108.747189185, 2024-09-21T15:01:54.125 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.031157957, 2024-09-21T15:01:54.125 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:54.125 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:54.125 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:54.125 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:54.125 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:54.125 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97724 2024-09-21T15:01:54.125 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.125 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:54.125 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.126 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:54.126 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328704+0000", 2024-09-21T15:01:54.126 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:54.126 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.126 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.126 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:54.126 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328704+0000", 2024-09-21T15:01:54.126 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:54.126 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.126 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.126 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:54.126 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328703+0000", 2024-09-21T15:01:54.126 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:54.126 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.126 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.127 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:54.127 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328708+0000", 2024-09-21T15:01:54.127 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0050000000000004e-06 2024-09-21T15:01:54.127 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.127 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.127 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:54.127 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328719+0000", 2024-09-21T15:01:54.127 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1052e-05 2024-09-21T15:01:54.127 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.127 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.127 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:54.127 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328727+0000", 2024-09-21T15:01:54.127 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4519999999999997e-06 2024-09-21T15:01:54.127 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.128 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.128 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:54.128 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.359737+0000", 2024-09-21T15:01:54.128 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.031009341999999999 2024-09-21T15:01:54.128 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.128 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.128 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:54.128 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.359806+0000", 2024-09-21T15:01:54.128 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9750000000000001e-05 2024-09-21T15:01:54.128 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.128 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.128 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:54.128 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.359862+0000", 2024-09-21T15:01:54.128 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5084999999999999e-05 2024-09-21T15:01:54.128 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:54.129 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:54.129 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:54.129 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.129 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.129 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:129970 3.c e158/87 3:33fa4d4e:::rbd_data.1136a12e4351.0000000000000000:head v 158'379, mlcod=156'377)", 2024-09-21T15:01:54.129 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:44.452766+0000", 2024-09-21T15:01:54.129 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 69.623126632999998, 2024-09-21T15:01:54.129 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011942062, 2024-09-21T15:01:54.129 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:54.129 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:54.129 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:54.129 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.129 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:54.129 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:44.452766+0000", 2024-09-21T15:01:54.129 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:54.130 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.130 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.130 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:54.130 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:44.452766+0000", 2024-09-21T15:01:54.130 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:54.130 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.130 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.130 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:54.130 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:44.452766+0000", 2024-09-21T15:01:54.130 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:54.130 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.130 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.130 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:54.131 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:44.458962+0000", 2024-09-21T15:01:54.131 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0061964799999999999 2024-09-21T15:01:54.131 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.131 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.131 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:54.131 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:44.459012+0000", 2024-09-21T15:01:54.131 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9427e-05 2024-09-21T15:01:54.131 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.131 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.131 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:54.131 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:44.459042+0000", 2024-09-21T15:01:54.131 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9838000000000001e-05 2024-09-21T15:01:54.131 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.131 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.132 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:54.132 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:44.459115+0000", 2024-09-21T15:01:54.132 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3430999999999995e-05 2024-09-21T15:01:54.132 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.132 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.132 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:54.132 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:44.459135+0000", 2024-09-21T15:01:54.132 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9701999999999999e-05 2024-09-21T15:01:54.132 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.132 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.132 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:01:54.132 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:44.464645+0000", 2024-09-21T15:01:54.132 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0055100820000000003 2024-09-21T15:01:54.132 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.132 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.132 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:54.133 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:44.464708+0000", 2024-09-21T15:01:54.133 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3399999999999996e-05 2024-09-21T15:01:54.133 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:54.133 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:54.133 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:54.133 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.133 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.133 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:135368 3.b 3:d3301c39:::rbd_data.11365ed419e3.0000000000000102:67 [read 335872~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:54.133 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:53.998901+0000", 2024-09-21T15:01:54.133 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 60.076991821, 2024-09-21T15:01:54.133 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011666174, 2024-09-21T15:01:54.133 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:54.133 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:54.133 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:54.134 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:54.134 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:54.134 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 135368 2024-09-21T15:01:54.134 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.134 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:54.134 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.134 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:54.134 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998901+0000", 2024-09-21T15:01:54.134 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:54.134 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.134 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.134 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:54.134 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998901+0000", 2024-09-21T15:01:54.135 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:54.135 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.135 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.135 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:54.135 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998901+0000", 2024-09-21T15:01:54.135 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:54.135 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.135 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.135 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:54.135 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998903+0000", 2024-09-21T15:01:54.135 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5500000000000001e-06 2024-09-21T15:01:54.135 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.135 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.135 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:54.136 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998913+0000", 2024-09-21T15:01:54.136 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6479999999999998e-06 2024-09-21T15:01:54.136 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.136 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.136 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:54.136 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998919+0000", 2024-09-21T15:01:54.136 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7930000000000003e-06 2024-09-21T15:01:54.136 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.136 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.136 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:54.136 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:54.010277+0000", 2024-09-21T15:01:54.136 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011358692 2024-09-21T15:01:54.136 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.136 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.136 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:54.137 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:54.010348+0000", 2024-09-21T15:01:54.137 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0247999999999996e-05 2024-09-21T15:01:54.137 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.137 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.137 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:54.137 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:54.010567+0000", 2024-09-21T15:01:54.137 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021953599999999999 2024-09-21T15:01:54.137 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:54.137 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:54.137 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:54.137 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.137 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.137 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:135369 3.b 3:d3301c39:::rbd_data.11365ed419e3.0000000000000102:67 [read 352256~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:54.137 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:53.998923+0000", 2024-09-21T15:01:54.138 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 60.076969519999999, 2024-09-21T15:01:54.138 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011990331999999999, 2024-09-21T15:01:54.138 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:54.138 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:54.138 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:54.138 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:54.138 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:54.138 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 135369 2024-09-21T15:01:54.138 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.138 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:54.138 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.138 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:54.138 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998923+0000", 2024-09-21T15:01:54.138 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:54.139 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.139 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.139 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:54.139 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998923+0000", 2024-09-21T15:01:54.139 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:54.139 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.139 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.139 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:54.139 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998923+0000", 2024-09-21T15:01:54.139 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:54.139 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.139 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.139 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:54.139 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998926+0000", 2024-09-21T15:01:54.139 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6749999999999998e-06 2024-09-21T15:01:54.139 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.140 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.140 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:54.140 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998935+0000", 2024-09-21T15:01:54.140 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3619999999999995e-06 2024-09-21T15:01:54.140 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.140 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.140 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:54.140 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998940+0000", 2024-09-21T15:01:54.140 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2379999999999997e-06 2024-09-21T15:01:54.140 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.140 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.140 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:54.140 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:54.010593+0000", 2024-09-21T15:01:54.140 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011652531000000001 2024-09-21T15:01:54.141 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.141 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.141 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:54.141 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:54.010679+0000", 2024-09-21T15:01:54.141 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6811000000000006e-05 2024-09-21T15:01:54.141 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.141 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.141 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:54.141 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:54.010914+0000", 2024-09-21T15:01:54.141 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000234071 2024-09-21T15:01:54.141 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:54.141 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:54.141 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:54.141 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.141 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.142 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:137706 3.b 3:d0cb966d:::rbd_data.1136f64ddcfe.0000000000000005:6a [read 1441792~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-21T15:01:54.142 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:58.376385+0000", 2024-09-21T15:01:54.142 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 55.699507933, 2024-09-21T15:01:54.142 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011783609, 2024-09-21T15:01:54.142 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:54.142 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:54.142 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:54.142 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:54.142 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:54.142 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 137706 2024-09-21T15:01:54.142 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.142 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:54.142 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.142 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:54.143 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376385+0000", 2024-09-21T15:01:54.143 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:54.143 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.143 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.143 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:54.143 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376385+0000", 2024-09-21T15:01:54.143 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:54.143 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.143 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.143 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:54.143 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376384+0000", 2024-09-21T15:01:54.143 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:54.143 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.143 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.143 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:54.144 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376387+0000", 2024-09-21T15:01:54.144 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9809999999999999e-06 2024-09-21T15:01:54.144 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.144 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.144 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:54.144 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376398+0000", 2024-09-21T15:01:54.144 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0319000000000001e-05 2024-09-21T15:01:54.144 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.144 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.144 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:54.144 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376404+0000", 2024-09-21T15:01:54.144 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5339999999999996e-06 2024-09-21T15:01:54.144 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.144 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.145 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:54.145 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.388025+0000", 2024-09-21T15:01:54.145 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011620449999999999 2024-09-21T15:01:54.145 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.145 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.145 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:54.145 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.388082+0000", 2024-09-21T15:01:54.145 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7065999999999999e-05 2024-09-21T15:01:54.145 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.145 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.145 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:54.145 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.388168+0000", 2024-09-21T15:01:54.145 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6689999999999998e-05 2024-09-21T15:01:54.145 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:54.145 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:54.145 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:54.146 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.146 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.146 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:137707 3.b 3:d0cb966d:::rbd_data.1136f64ddcfe.0000000000000005:6a [read 1556480~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-21T15:01:54.146 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:58.376428+0000", 2024-09-21T15:01:54.146 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 55.699464441000003, 2024-09-21T15:01:54.146 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012004865999999999, 2024-09-21T15:01:54.146 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:54.146 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:54.146 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:54.146 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:54.146 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:54.146 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 137707 2024-09-21T15:01:54.146 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.146 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:54.146 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.147 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:54.147 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376428+0000", 2024-09-21T15:01:54.147 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:54.147 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.147 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.147 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:54.147 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376428+0000", 2024-09-21T15:01:54.147 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:54.147 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.147 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.147 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:54.147 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376428+0000", 2024-09-21T15:01:54.147 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:54.147 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.147 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.148 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:54.148 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376432+0000", 2024-09-21T15:01:54.148 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2119999999999997e-06 2024-09-21T15:01:54.148 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.148 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.148 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:54.148 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376448+0000", 2024-09-21T15:01:54.148 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5913000000000002e-05 2024-09-21T15:01:54.148 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.148 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.148 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:54.148 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376455+0000", 2024-09-21T15:01:54.148 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6359999999999994e-06 2024-09-21T15:01:54.148 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.148 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.149 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:54.149 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.388197+0000", 2024-09-21T15:01:54.149 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011741843 2024-09-21T15:01:54.149 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.149 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.149 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:54.149 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.388249+0000", 2024-09-21T15:01:54.149 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1549999999999999e-05 2024-09-21T15:01:54.149 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.149 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.149 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:54.149 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.388433+0000", 2024-09-21T15:01:54.149 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018437999999999999 2024-09-21T15:01:54.149 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:54.149 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:54.150 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:54.150 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.150 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.150 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:137708 3.b 3:d0cb966d:::rbd_data.1136f64ddcfe.0000000000000005:6a [read 1671168~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-21T15:01:54.150 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:58.376462+0000", 2024-09-21T15:01:54.150 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 55.699430653, 2024-09-21T15:01:54.150 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012315491, 2024-09-21T15:01:54.150 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:54.150 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:54.150 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:54.150 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:54.150 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:54.150 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 137708 2024-09-21T15:01:54.150 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.150 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:54.150 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.151 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:54.151 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376462+0000", 2024-09-21T15:01:54.151 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:54.151 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.151 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.151 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:54.151 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376462+0000", 2024-09-21T15:01:54.151 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:54.151 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.151 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.151 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:54.151 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376461+0000", 2024-09-21T15:01:54.151 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:54.151 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.151 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.152 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:54.152 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376465+0000", 2024-09-21T15:01:54.152 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7069999999999998e-06 2024-09-21T15:01:54.152 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.152 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.152 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:54.152 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376473+0000", 2024-09-21T15:01:54.152 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6669999999999996e-06 2024-09-21T15:01:54.152 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.152 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.152 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:54.152 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376479+0000", 2024-09-21T15:01:54.152 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.263e-06 2024-09-21T15:01:54.152 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.152 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.152 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:54.153 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.388489+0000", 2024-09-21T15:01:54.153 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012009953 2024-09-21T15:01:54.153 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.153 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.153 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:54.153 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.388625+0000", 2024-09-21T15:01:54.153 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013548799999999999 2024-09-21T15:01:54.153 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.153 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.153 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:54.153 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.388778+0000", 2024-09-21T15:01:54.153 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015302299999999999 2024-09-21T15:01:54.153 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:54.153 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:54.153 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:54.153 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.154 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.154 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:137709 3.b 3:d0cb966d:::rbd_data.1136f64ddcfe.0000000000000005:6a [read 1785856~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-21T15:01:54.154 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:58.376485+0000", 2024-09-21T15:01:54.154 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 55.699407321000002, 2024-09-21T15:01:54.154 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012466549, 2024-09-21T15:01:54.154 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:54.154 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:54.154 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:54.154 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:54.154 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:54.154 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 137709 2024-09-21T15:01:54.154 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.154 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:54.154 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.154 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:54.155 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376485+0000", 2024-09-21T15:01:54.155 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:54.155 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.155 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.155 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:54.155 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376485+0000", 2024-09-21T15:01:54.155 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:54.155 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.155 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.155 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:54.155 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376485+0000", 2024-09-21T15:01:54.155 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:54.155 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.155 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.156 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:54.156 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376492+0000", 2024-09-21T15:01:54.156 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8929999999999999e-06 2024-09-21T15:01:54.156 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.156 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.156 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:54.156 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376500+0000", 2024-09-21T15:01:54.156 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7989999999999994e-06 2024-09-21T15:01:54.156 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.156 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.156 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:54.156 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376506+0000", 2024-09-21T15:01:54.156 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4799999999999998e-06 2024-09-21T15:01:54.156 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.157 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.157 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:54.157 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.388810+0000", 2024-09-21T15:01:54.157 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012303727 2024-09-21T15:01:54.157 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.157 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.157 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:54.157 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.388871+0000", 2024-09-21T15:01:54.157 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1278000000000005e-05 2024-09-21T15:01:54.157 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.157 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.157 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:54.157 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.388952+0000", 2024-09-21T15:01:54.157 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0770000000000001e-05 2024-09-21T15:01:54.157 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:54.157 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:54.158 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:54.158 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.158 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.158 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:137710 3.b 3:d0cb966d:::rbd_data.1136f64ddcfe.0000000000000005:6a [read 1900544~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-21T15:01:54.158 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:58.376512+0000", 2024-09-21T15:01:54.158 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 55.699381117999998, 2024-09-21T15:01:54.158 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012746351, 2024-09-21T15:01:54.158 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:54.158 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:54.158 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:54.158 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:54.158 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:54.158 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 137710 2024-09-21T15:01:54.159 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.159 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:54.159 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.159 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:54.159 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376512+0000", 2024-09-21T15:01:54.159 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:54.159 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.159 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.159 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:54.159 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376512+0000", 2024-09-21T15:01:54.159 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:54.159 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.159 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.159 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:54.160 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376511+0000", 2024-09-21T15:01:54.160 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:54.160 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.160 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.160 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:54.160 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376514+0000", 2024-09-21T15:01:54.160 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1719999999999999e-06 2024-09-21T15:01:54.160 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.160 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.160 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:54.160 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376522+0000", 2024-09-21T15:01:54.160 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1389999999999998e-06 2024-09-21T15:01:54.160 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.161 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.161 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:54.161 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376535+0000", 2024-09-21T15:01:54.161 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3658999999999999e-05 2024-09-21T15:01:54.161 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.161 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.161 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:54.161 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.389005+0000", 2024-09-21T15:01:54.161 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012470261 2024-09-21T15:01:54.161 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.161 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.161 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:54.161 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.389119+0000", 2024-09-21T15:01:54.161 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000113346 2024-09-21T15:01:54.162 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.162 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.162 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:54.162 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.389258+0000", 2024-09-21T15:01:54.162 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013920100000000001 2024-09-21T15:01:54.162 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:54.162 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:54.162 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:54.162 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.162 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.162 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:137711 3.b 3:d0cb966d:::rbd_data.1136f64ddcfe.0000000000000005:6a [read 2015232~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-21T15:01:54.162 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:58.376541+0000", 2024-09-21T15:01:54.162 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 55.699351755999999, 2024-09-21T15:01:54.162 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012849955999999999, 2024-09-21T15:01:54.163 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:54.163 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:54.163 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:54.163 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:54.163 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:54.163 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 137711 2024-09-21T15:01:54.163 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.163 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:54.163 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.163 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:54.163 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376541+0000", 2024-09-21T15:01:54.163 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:54.163 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.163 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.164 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:54.164 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376541+0000", 2024-09-21T15:01:54.164 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:54.164 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.164 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.164 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:54.164 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376540+0000", 2024-09-21T15:01:54.164 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:54.164 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.164 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.164 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:54.164 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376544+0000", 2024-09-21T15:01:54.164 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3249999999999999e-06 2024-09-21T15:01:54.164 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.165 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.165 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:54.165 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376551+0000", 2024-09-21T15:01:54.165 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2189999999999998e-06 2024-09-21T15:01:54.165 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.165 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.165 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:54.165 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376557+0000", 2024-09-21T15:01:54.165 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8869999999999997e-06 2024-09-21T15:01:54.165 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.165 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.165 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:54.165 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.389277+0000", 2024-09-21T15:01:54.165 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012720555999999999 2024-09-21T15:01:54.165 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.165 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.166 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:54.166 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.389325+0000", 2024-09-21T15:01:54.166 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7367000000000002e-05 2024-09-21T15:01:54.166 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.166 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.166 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:54.166 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.389391+0000", 2024-09-21T15:01:54.166 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6095999999999999e-05 2024-09-21T15:01:54.166 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:54.166 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:54.166 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:54.166 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.166 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.167 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:137723 3.b 3:d0cb966d:::rbd_data.1136f64ddcfe.0000000000000005:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-21T15:01:54.167 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:58.377091+0000", 2024-09-21T15:01:54.167 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 55.698801435, 2024-09-21T15:01:54.167 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012594424, 2024-09-21T15:01:54.167 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:54.167 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:54.167 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:54.167 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:54.167 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:54.167 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 137723 2024-09-21T15:01:54.167 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.167 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:54.167 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.167 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:54.167 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.377091+0000", 2024-09-21T15:01:54.167 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:54.168 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.168 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.168 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:54.168 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.377091+0000", 2024-09-21T15:01:54.168 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:54.168 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.168 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.168 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:54.168 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.377090+0000", 2024-09-21T15:01:54.168 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:54.168 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.168 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.168 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:54.168 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.377097+0000", 2024-09-21T15:01:54.168 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7100000000000001e-06 2024-09-21T15:01:54.168 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.169 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.169 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:54.169 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.377120+0000", 2024-09-21T15:01:54.169 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3189000000000001e-05 2024-09-21T15:01:54.169 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.169 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.169 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:54.169 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.377132+0000", 2024-09-21T15:01:54.169 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2041999999999999e-05 2024-09-21T15:01:54.169 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.169 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.169 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:54.169 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.389439+0000", 2024-09-21T15:01:54.169 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.01230698 2024-09-21T15:01:54.169 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.169 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.170 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:54.170 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.389556+0000", 2024-09-21T15:01:54.170 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000116446 2024-09-21T15:01:54.170 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.170 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.170 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:54.170 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.389686+0000", 2024-09-21T15:01:54.170 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000129966 2024-09-21T15:01:54.170 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:54.170 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:54.170 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:54.170 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.170 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.170 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:166795 3.8 3:1499e676:::rbd_id.image_client.0-clone28:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e193)", 2024-09-21T15:01:54.171 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:25.321589+0000", 2024-09-21T15:01:54.171 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 28.754303271000001, 2024-09-21T15:01:54.171 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.855281438, 2024-09-21T15:01:54.171 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:54.171 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:54.171 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:54.171 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:54.171 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:54.171 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 166795 2024-09-21T15:01:54.171 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.171 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:54.171 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.171 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:54.171 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:25.321589+0000", 2024-09-21T15:01:54.171 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:54.171 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.172 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.172 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:54.172 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:25.321589+0000", 2024-09-21T15:01:54.172 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:54.172 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.172 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.172 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:54.172 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:25.321588+0000", 2024-09-21T15:01:54.172 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:54.172 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.172 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.172 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:54.172 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:25.321598+0000", 2024-09-21T15:01:54.172 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7429999999999997e-06 2024-09-21T15:01:54.172 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.172 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.173 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:54.173 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:25.321624+0000", 2024-09-21T15:01:54.173 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6752000000000001e-05 2024-09-21T15:01:54.173 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.173 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.173 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:54.173 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:25.321637+0000", 2024-09-21T15:01:54.173 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2809999999999999e-05 2024-09-21T15:01:54.173 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.173 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.173 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:54.173 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:25.321721+0000", 2024-09-21T15:01:54.173 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3314999999999998e-05 2024-09-21T15:01:54.173 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.173 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.173 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:54.174 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:25.321736+0000", 2024-09-21T15:01:54.174 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5262000000000001e-05 2024-09-21T15:01:54.174 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.174 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.174 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:54.174 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:26.176475+0000", 2024-09-21T15:01:54.174 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.85473961200000004 2024-09-21T15:01:54.174 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.174 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.174 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:54.174 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:26.176655+0000", 2024-09-21T15:01:54.174 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017934400000000001 2024-09-21T15:01:54.174 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:54.174 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:54.175 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:54.175 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:26.176871+0000", 2024-09-21T15:01:54.175 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000216133 2024-09-21T15:01:54.175 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:54.175 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:54.175 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:54.175 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:54.175 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:54.175 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:01:54.177 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T15:01:54.177 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:01:54.173+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:01:54.277 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:01:54.277+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:01:54.303 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:54.303 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:01:54.304 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T15:01:54.304 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:54.315 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T15:01:54.318 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 5, 6, 7] out_osds: [4] dead_osds: [4] live_osds: [0, 1, 2, 3, 5, 6, 7] 2024-09-21T15:01:54.318 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T15:01:54.319 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-21T15:01:54.319 INFO:tasks.thrashosds.thrasher:Setting osd 5 primary_affinity to 0.493130 2024-09-21T15:01:54.319 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 0.49313021494151243 2024-09-21T15:01:54.378 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:01:54.372+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:01:54.431 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:54.431 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:01:54.431 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T15:01:54.432 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T15:01:54.432 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:54.441 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T15:01:54.485 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:01:54.485+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:01:54.544 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:54.544 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T15:01:54.544 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T15:01:54.544 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T15:01:54.544 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.544 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:97677 3.15 3:a8e72133:::rbd_data.1136bfc79a70.0000000000000005:head [write 0~4440027 [fadvise_sequential+fadvise_nocache],write 4751360~147456 [fadvise_sequential+fadvise_nocache],write 4964352~147456 [fadvise_sequential+fadvise_nocache],write 5177344~65536 [fadvise_sequential+fadvise_nocache],write 5259264~147456 [fadvise_sequential+fadvise_nocache],write 5472256~147456 [fadvise_sequential+fadvise_nocache],write 5685248~229376 [fadvise_sequential+fadvise_nocache],write 5980160~147456 [fadvise_sequential+fadvise_nocache],write 6193152~229376 [fadvise_sequential+fadvise_nocache],write 6488064~147456 [fadvise_sequential+fadvise_nocache],write 6701056~229376 [fadvise_sequential+fadvise_nocache],write 6995968~147456 [fadvise_sequential+fadvise_nocache],write 7208960~229376 [fadvise_sequential+fadvise_nocache],write 7503872~147456 [fadvise_sequential+fadvise_nocache],write 7716864~229376 [fadvise_sequential+fadvise_nocache],write 8044544~114688 [fadvise_sequential+fadvise_nocache],write 8257536~114688 [fadvise_sequential+fadvise_nocache],write 8388608~65536 [fadvise_sequential+fadvise_nocache],write 8552448~114688 [fadvise_sequential+fadvise_nocache],write 8765440~114688 [fadvise_sequential+fadvise_nocache],write 8929280~32768 [fadvise_sequential+fadvise_nocache],write 9142272~32768 [fadvise_sequential+fadvise_nocache],write 9355264~32768 [fadvise_sequential+fadvise_nocache],write 9437184~32768 [fadvise_sequential+fadvise_nocache],write 9650176~32768 [fadvise_sequential+fadvise_nocache],write 9863168~32768 [fadvise_sequential+fadvise_nocache],write 9961472~131072 [fadvise_sequential+fadvise_nocache],write 10108928~196608 [fadvise_sequential+fadvise_nocache],write 10321920~278528 [fadvise_sequential+fadvise_nocache],write 10616832~196608 [fadvise_sequential+fadvise_nocache],write 10829824~278528 [fadvise_sequential+fadvise_nocache],write 11124736~196608 [fadvise_sequential+fadvise_nocache],write 11337728~196608 [fadvise_sequential+fadvise_nocache],write 11550720~65536 [fadvise_sequential+fadvise_nocache],write 11649024~98304 [fadvise_sequential+fadvise_nocache],write 11763712~65536 [fadvise_sequential+fadvise_nocache],write 11862016~98304 [fadvise_sequential+fadvise_nocache],write 11976704~65536 [fadvise_sequential+fadvise_nocache],write 12058624~65536 [fadvise_sequential+fadvise_nocache],write 12156928~98304 [fadvise_sequential+fadvise_nocache],write 12271616~65536 [fadvise_sequential+fadvise_nocache],write 12369920~98304 [fadvise_sequential+fadvise_nocache],write 12484608~65536 [fadvise_sequential+fadvise_nocache],write 12664832~98304 [fadvise_sequential+fadvise_nocache],write 12877824~98304 [fadvise_sequential+fadvise_nocache],write 13090816~16384 [fadvise_sequential+fadvise_nocache],write 13172736~98304 [fadvise_sequential+fadvise_nocache],write 13385728~98304 [fadvise_sequential+fadvise_nocache],write 13598720~32768 [fadvise_sequential+fadvise_nocache],write 13680640~98304 [fadvise_sequential+fadvise_nocache],write 13893632~98304 [fadvise_sequential+fadvise_nocache],write 14106624~49152 [fadvise_sequential+fadvise_nocache],write 14794752~81920 [fadvise_sequential+fadvise_nocache],write 15007744~81920 [fadvise_sequential+fadvise_nocache],write 15302656~81920 [fadvise_sequential+fadvise_nocache],write 15515648~81920 [fadvise_sequential+fadvise_nocache],write 15810560~81920 [fadvise_sequential+fadvise_nocache],write 16023552~81920 [fadvise_sequential+fadvise_nocache],write 16236544~16384 [fadvise_sequential+fadvise_nocache],write 16318464~81920 [fadvise_sequential+fadvise_nocache],write 16531456~81920 [fadvise_sequential+fadvise_nocache],write 16744448~32768 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:01:54.544 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.302946+0000", 2024-09-21T15:01:54.545 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 109.24002193699999, 2024-09-21T15:01:54.545 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.091231204999999996, 2024-09-21T15:01:54.545 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:54.545 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:54.545 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:54.545 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:54.545 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:54.545 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 97677 2024-09-21T15:01:54.545 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.545 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:54.545 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.545 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:54.545 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.302946+0000", 2024-09-21T15:01:54.545 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:54.546 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.546 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.546 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:54.546 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.302946+0000", 2024-09-21T15:01:54.546 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:54.546 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.546 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.546 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:54.546 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.302942+0000", 2024-09-21T15:01:54.546 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999962 2024-09-21T15:01:54.546 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.546 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.546 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:54.546 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311253+0000", 2024-09-21T15:01:54.547 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0083105360000000003 2024-09-21T15:01:54.547 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.547 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.547 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:54.547 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311283+0000", 2024-09-21T15:01:54.547 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0065000000000001e-05 2024-09-21T15:01:54.547 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.547 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.547 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:54.547 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311296+0000", 2024-09-21T15:01:54.547 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3206e-05 2024-09-21T15:01:54.547 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.547 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.547 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:54.547 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311404+0000", 2024-09-21T15:01:54.548 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000107514 2024-09-21T15:01:54.548 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.548 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.548 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:54.548 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.311795+0000", 2024-09-21T15:01:54.548 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000390913 2024-09-21T15:01:54.548 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.548 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.548 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 6", 2024-09-21T15:01:54.548 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.314686+0000", 2024-09-21T15:01:54.548 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0028915659999999999 2024-09-21T15:01:54.548 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.548 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.548 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:01:54.549 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.368178+0000", 2024-09-21T15:01:54.549 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.053491706 2024-09-21T15:01:54.549 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.549 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.549 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:01:54.549 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.393925+0000", 2024-09-21T15:01:54.549 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.025747289 2024-09-21T15:01:54.549 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.549 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.549 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:01:54.549 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.394127+0000", 2024-09-21T15:01:54.549 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000201444 2024-09-21T15:01:54.549 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.549 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.549 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:54.549 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.394177+0000", 2024-09-21T15:01:54.550 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0689000000000002e-05 2024-09-21T15:01:54.550 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:54.550 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:54.550 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:54.550 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.550 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.550 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:97673 3.1d e122/24 3:bd91946e:::rbd_data.1136bfc79a70.0000000000000002:head v 122'162, mlcod=120'160)", 2024-09-21T15:01:54.550 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.313730+0000", 2024-09-21T15:01:54.550 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 109.22923761200001, 2024-09-21T15:01:54.550 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.055373462999999998, 2024-09-21T15:01:54.550 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:54.550 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:54.550 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:54.550 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.550 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:54.551 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.313730+0000", 2024-09-21T15:01:54.551 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:54.551 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.551 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.551 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:54.551 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.313730+0000", 2024-09-21T15:01:54.551 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:54.551 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.551 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.551 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:54.551 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.313730+0000", 2024-09-21T15:01:54.551 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:54.551 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.551 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.551 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:54.552 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.336113+0000", 2024-09-21T15:01:54.552 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022382962999999999 2024-09-21T15:01:54.552 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.552 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.552 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:54.552 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.336140+0000", 2024-09-21T15:01:54.552 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7248999999999999e-05 2024-09-21T15:01:54.552 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.552 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.552 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:54.552 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.336151+0000", 2024-09-21T15:01:54.552 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1015e-05 2024-09-21T15:01:54.552 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.552 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.552 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:54.553 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.355428+0000", 2024-09-21T15:01:54.553 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.019276379999999999 2024-09-21T15:01:54.553 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.553 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.553 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:54.553 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.355484+0000", 2024-09-21T15:01:54.553 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6820999999999997e-05 2024-09-21T15:01:54.553 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.553 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.553 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:01:54.553 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.369040+0000", 2024-09-21T15:01:54.553 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.013555466 2024-09-21T15:01:54.553 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.553 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.554 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:54.554 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.369104+0000", 2024-09-21T15:01:54.554 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3986999999999996e-05 2024-09-21T15:01:54.554 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:54.554 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:54.554 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:54.554 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.554 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.554 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139890 3.e 3:71dba096:::rbd_data.113664780c85.000000000000018b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:54.554 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.226444+0000", 2024-09-21T15:01:54.554 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 52.316523969999999, 2024-09-21T15:01:54.554 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.028209567, 2024-09-21T15:01:54.554 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:54.555 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:54.555 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:54.555 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:54.555 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:54.555 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139890 2024-09-21T15:01:54.555 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.555 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:54.555 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.555 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:54.555 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226444+0000", 2024-09-21T15:01:54.555 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:54.555 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.555 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.555 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:54.555 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226444+0000", 2024-09-21T15:01:54.556 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:54.556 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.556 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.556 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:54.556 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226443+0000", 2024-09-21T15:01:54.556 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:54.556 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.556 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.556 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:54.556 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226448+0000", 2024-09-21T15:01:54.556 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5499999999999996e-06 2024-09-21T15:01:54.556 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.556 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.556 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:54.556 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226456+0000", 2024-09-21T15:01:54.556 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8679999999999999e-06 2024-09-21T15:01:54.557 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.557 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.557 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:54.557 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226462+0000", 2024-09-21T15:01:54.557 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1800000000000001e-06 2024-09-21T15:01:54.557 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.557 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.557 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:54.557 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226545+0000", 2024-09-21T15:01:54.557 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.2527000000000005e-05 2024-09-21T15:01:54.557 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.557 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.557 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:54.557 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226553+0000", 2024-09-21T15:01:54.557 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8019999999999998e-06 2024-09-21T15:01:54.557 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.558 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.558 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:54.558 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.254491+0000", 2024-09-21T15:01:54.558 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.02793785 2024-09-21T15:01:54.558 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.558 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.558 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:54.558 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.254654+0000", 2024-09-21T15:01:54.558 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00016245800000000001 2024-09-21T15:01:54.558 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:54.558 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:54.558 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:54.558 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.558 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.559 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139897 3.e 3:76ddeb1d:::rbd_data.113664780c85.0000000000000004:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:54.559 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.227198+0000", 2024-09-21T15:01:54.559 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 52.315770364999999, 2024-09-21T15:01:54.559 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.027799822, 2024-09-21T15:01:54.559 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:54.559 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:54.559 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:54.559 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:54.559 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:54.559 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139897 2024-09-21T15:01:54.559 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.559 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:54.559 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.559 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:54.559 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227198+0000", 2024-09-21T15:01:54.559 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:54.560 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.560 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.560 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:54.560 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227198+0000", 2024-09-21T15:01:54.560 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:54.560 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.560 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.560 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:54.560 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227197+0000", 2024-09-21T15:01:54.560 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:54.560 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.560 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.560 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:54.560 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227203+0000", 2024-09-21T15:01:54.560 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4819999999999999e-06 2024-09-21T15:01:54.561 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.561 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.561 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:54.561 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227220+0000", 2024-09-21T15:01:54.561 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6671e-05 2024-09-21T15:01:54.561 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.561 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.561 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:54.561 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227229+0000", 2024-09-21T15:01:54.561 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0499999999999997e-06 2024-09-21T15:01:54.561 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.561 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.561 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:54.561 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227293+0000", 2024-09-21T15:01:54.561 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4434e-05 2024-09-21T15:01:54.561 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.562 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.562 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:54.562 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227302+0000", 2024-09-21T15:01:54.562 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0929999999999992e-06 2024-09-21T15:01:54.562 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.562 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.562 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:54.562 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.254720+0000", 2024-09-21T15:01:54.562 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0274177550000001 2024-09-21T15:01:54.562 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.562 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.562 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:54.562 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.254854+0000", 2024-09-21T15:01:54.562 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000133416 2024-09-21T15:01:54.563 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.563 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.563 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:54.563 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.254998+0000", 2024-09-21T15:01:54.563 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014400099999999999 2024-09-21T15:01:54.563 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:54.563 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:54.563 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:54.563 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.563 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.563 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139902 3.e 3:76b9b799:::rbd_data.113664780c85.0000000000000021:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:54.563 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.227319+0000", 2024-09-21T15:01:54.563 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 52.315648983999999, 2024-09-21T15:01:54.563 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0279584639999999, 2024-09-21T15:01:54.563 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:54.564 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:54.564 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:54.564 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:54.564 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:54.564 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139902 2024-09-21T15:01:54.564 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.564 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:54.564 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.564 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:54.564 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227319+0000", 2024-09-21T15:01:54.564 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:54.564 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.564 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.564 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:54.564 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227319+0000", 2024-09-21T15:01:54.565 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:54.565 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.565 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.565 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:54.565 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227319+0000", 2024-09-21T15:01:54.565 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:54.565 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.565 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.565 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:54.565 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227322+0000", 2024-09-21T15:01:54.565 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1980000000000001e-06 2024-09-21T15:01:54.565 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.565 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.565 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:54.565 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227328+0000", 2024-09-21T15:01:54.566 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4230000000000002e-06 2024-09-21T15:01:54.566 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.566 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.566 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:54.566 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227333+0000", 2024-09-21T15:01:54.566 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.0560000000000004e-06 2024-09-21T15:01:54.566 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.566 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.566 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:54.566 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227729+0000", 2024-09-21T15:01:54.566 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00039591799999999999 2024-09-21T15:01:54.566 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.566 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.566 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:54.567 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227738+0000", 2024-09-21T15:01:54.567 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8119999999999997e-06 2024-09-21T15:01:54.567 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.567 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.567 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:54.567 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255021+0000", 2024-09-21T15:01:54.567 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0272826779999999 2024-09-21T15:01:54.567 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.567 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.567 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:54.567 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255147+0000", 2024-09-21T15:01:54.567 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012595600000000001 2024-09-21T15:01:54.567 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.567 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.567 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:54.568 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255278+0000", 2024-09-21T15:01:54.568 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013081400000000001 2024-09-21T15:01:54.568 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:54.568 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:54.568 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:54.568 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.568 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.568 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139945 3.e 3:7756a55d:::rbd_data.113664780c85.0000000000000154:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:54.568 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.229015+0000", 2024-09-21T15:01:54.568 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 52.313953181000002, 2024-09-21T15:01:54.568 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.026468825, 2024-09-21T15:01:54.568 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:54.568 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:54.568 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:54.568 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:54.569 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:54.569 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139945 2024-09-21T15:01:54.569 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.569 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:54.569 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.569 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:54.569 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229015+0000", 2024-09-21T15:01:54.569 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:54.569 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.569 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.569 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:54.569 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229015+0000", 2024-09-21T15:01:54.569 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:54.569 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.569 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.570 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:54.570 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229014+0000", 2024-09-21T15:01:54.570 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:54.570 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.570 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.570 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:54.570 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229017+0000", 2024-09-21T15:01:54.570 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7310000000000002e-06 2024-09-21T15:01:54.570 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.570 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.570 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:54.570 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229023+0000", 2024-09-21T15:01:54.570 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8150000000000002e-06 2024-09-21T15:01:54.570 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.570 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.571 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:54.571 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229028+0000", 2024-09-21T15:01:54.571 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7650000000000001e-06 2024-09-21T15:01:54.571 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.571 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.571 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:54.571 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229082+0000", 2024-09-21T15:01:54.571 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4521999999999997e-05 2024-09-21T15:01:54.571 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.571 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.571 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:54.571 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229092+0000", 2024-09-21T15:01:54.571 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.3940000000000002e-06 2024-09-21T15:01:54.571 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.572 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.572 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:54.572 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255382+0000", 2024-09-21T15:01:54.572 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0262900930000001 2024-09-21T15:01:54.572 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.572 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.572 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:54.572 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255484+0000", 2024-09-21T15:01:54.572 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000101932 2024-09-21T15:01:54.572 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:54.572 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:54.572 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:54.572 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.572 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.572 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139967 3.e 3:74e90d85:::rbd_data.113664780c85.0000000000000030:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:54.572 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.229124+0000", 2024-09-21T15:01:54.573 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 52.313844240999998, 2024-09-21T15:01:54.573 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.026724381, 2024-09-21T15:01:54.573 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:54.573 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:54.573 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:54.573 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:54.573 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:54.573 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139967 2024-09-21T15:01:54.573 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.573 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:54.573 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.573 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:54.573 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229124+0000", 2024-09-21T15:01:54.573 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:54.574 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.574 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.574 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:54.574 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229124+0000", 2024-09-21T15:01:54.574 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:54.574 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.574 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.574 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:54.574 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229123+0000", 2024-09-21T15:01:54.574 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:54.574 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.574 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.574 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:54.574 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229126+0000", 2024-09-21T15:01:54.575 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4779999999999998e-06 2024-09-21T15:01:54.575 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.575 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.575 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:54.575 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229131+0000", 2024-09-21T15:01:54.575 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.729e-06 2024-09-21T15:01:54.575 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.575 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.575 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:54.575 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229135+0000", 2024-09-21T15:01:54.575 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5859999999999998e-06 2024-09-21T15:01:54.575 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.575 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.575 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:54.576 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229558+0000", 2024-09-21T15:01:54.576 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00042262499999999999 2024-09-21T15:01:54.576 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.576 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.576 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:54.576 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229569+0000", 2024-09-21T15:01:54.576 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0985000000000001e-05 2024-09-21T15:01:54.576 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.576 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.576 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:54.576 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255509+0000", 2024-09-21T15:01:54.576 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.025940246 2024-09-21T15:01:54.576 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.576 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.577 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:54.577 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255662+0000", 2024-09-21T15:01:54.577 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015324199999999999 2024-09-21T15:01:54.577 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.577 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.577 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:54.577 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255848+0000", 2024-09-21T15:01:54.577 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00018583599999999999 2024-09-21T15:01:54.577 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:54.577 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:54.577 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:54.577 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.577 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.577 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140010 3.e 3:735c06f2:::rbd_data.113664780c85.0000000000000157:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:54.577 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.231531+0000", 2024-09-21T15:01:54.578 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 52.311437376000001, 2024-09-21T15:01:54.578 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0244773149999999, 2024-09-21T15:01:54.578 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:54.578 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:54.578 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:54.578 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:54.578 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:54.578 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140010 2024-09-21T15:01:54.578 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.578 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:54.578 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.578 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:54.578 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231531+0000", 2024-09-21T15:01:54.578 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:54.579 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.579 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.579 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:54.579 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231531+0000", 2024-09-21T15:01:54.579 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:54.579 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.579 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.579 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:54.579 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231530+0000", 2024-09-21T15:01:54.579 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:54.579 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.579 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.579 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:54.579 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231535+0000", 2024-09-21T15:01:54.580 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3970000000000002e-06 2024-09-21T15:01:54.580 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.580 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.580 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:54.580 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231544+0000", 2024-09-21T15:01:54.580 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5909999999999996e-06 2024-09-21T15:01:54.580 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.580 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.580 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:54.580 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231551+0000", 2024-09-21T15:01:54.580 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7129999999999999e-06 2024-09-21T15:01:54.580 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.580 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.580 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:54.580 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231632+0000", 2024-09-21T15:01:54.581 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1181999999999996e-05 2024-09-21T15:01:54.581 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.581 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.581 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:54.581 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231640+0000", 2024-09-21T15:01:54.581 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.551e-06 2024-09-21T15:01:54.581 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.581 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.581 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:54.581 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255884+0000", 2024-09-21T15:01:54.581 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.024244057 2024-09-21T15:01:54.581 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.581 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.581 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:54.581 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256008+0000", 2024-09-21T15:01:54.582 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012351699999999999 2024-09-21T15:01:54.582 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:54.582 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:54.582 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:54.582 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.582 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.582 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140043 3.e 3:74a37859:::rbd_data.113664780c85.0000000000000086:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:54.582 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.232623+0000", 2024-09-21T15:01:54.582 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 52.310344901999997, 2024-09-21T15:01:54.582 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.023518634, 2024-09-21T15:01:54.582 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:54.582 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:54.582 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:54.582 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:54.582 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:54.582 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140043 2024-09-21T15:01:54.583 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.583 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:54.583 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.583 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:54.583 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.232623+0000", 2024-09-21T15:01:54.583 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:54.583 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.583 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.583 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:54.583 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.232623+0000", 2024-09-21T15:01:54.583 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:54.583 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.583 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.583 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:54.583 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.232623+0000", 2024-09-21T15:01:54.583 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:54.584 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.584 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.584 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:54.584 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.232626+0000", 2024-09-21T15:01:54.584 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0019999999999998e-06 2024-09-21T15:01:54.584 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.584 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.584 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:54.584 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.232632+0000", 2024-09-21T15:01:54.584 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3260000000000001e-06 2024-09-21T15:01:54.584 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.584 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.584 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:54.584 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.232637+0000", 2024-09-21T15:01:54.585 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9899999999999997e-06 2024-09-21T15:01:54.585 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.585 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.585 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:54.585 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.232678+0000", 2024-09-21T15:01:54.585 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.0423999999999998e-05 2024-09-21T15:01:54.585 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.585 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.585 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:54.585 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.232686+0000", 2024-09-21T15:01:54.585 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1419999999999993e-06 2024-09-21T15:01:54.585 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.585 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.586 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:54.586 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256037+0000", 2024-09-21T15:01:54.586 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.023351736 2024-09-21T15:01:54.586 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.586 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.586 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:54.586 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256142+0000", 2024-09-21T15:01:54.586 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010443 2024-09-21T15:01:54.586 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:54.586 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:54.586 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:54.586 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.586 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.586 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140063 3.e 3:771cae23:::rbd_data.113664780c85.0000000000000112:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:54.586 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.233948+0000", 2024-09-21T15:01:54.587 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 52.309019835000001, 2024-09-21T15:01:54.587 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0223114710000001, 2024-09-21T15:01:54.587 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:54.587 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:54.587 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:54.587 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:54.587 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:54.587 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140063 2024-09-21T15:01:54.587 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.587 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:54.587 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.587 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:54.587 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233948+0000", 2024-09-21T15:01:54.587 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:54.588 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.588 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.588 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:54.588 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233948+0000", 2024-09-21T15:01:54.588 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:54.588 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.588 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.588 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:54.588 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233948+0000", 2024-09-21T15:01:54.588 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:54.588 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.588 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.588 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:54.588 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233951+0000", 2024-09-21T15:01:54.588 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3380000000000001e-06 2024-09-21T15:01:54.589 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.589 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.589 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:54.589 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233958+0000", 2024-09-21T15:01:54.589 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0990000000000001e-06 2024-09-21T15:01:54.589 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.589 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.589 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:54.589 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233964+0000", 2024-09-21T15:01:54.589 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.852e-06 2024-09-21T15:01:54.589 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.589 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.589 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:54.589 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.234056+0000", 2024-09-21T15:01:54.590 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2392000000000006e-05 2024-09-21T15:01:54.590 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.590 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.590 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:54.590 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.234066+0000", 2024-09-21T15:01:54.590 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1360000000000003e-06 2024-09-21T15:01:54.590 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.590 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.590 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:54.590 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256169+0000", 2024-09-21T15:01:54.590 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0221031949999999 2024-09-21T15:01:54.590 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.590 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.590 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:54.590 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256260+0000", 2024-09-21T15:01:54.591 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0939000000000005e-05 2024-09-21T15:01:54.591 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:54.591 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:54.591 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:54.591 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.591 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.591 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140086 3.e 3:758134b7:::rbd_data.113664780c85.00000000000001ad:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:54.591 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.235110+0000", 2024-09-21T15:01:54.591 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 52.307858136, 2024-09-21T15:01:54.591 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.021276217, 2024-09-21T15:01:54.591 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:54.591 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:54.591 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:54.591 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:54.591 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:54.592 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140086 2024-09-21T15:01:54.592 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.592 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:54.592 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.592 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:54.592 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.235110+0000", 2024-09-21T15:01:54.592 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:54.592 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.592 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.592 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:54.592 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.235110+0000", 2024-09-21T15:01:54.592 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:54.592 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.592 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.593 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:54.593 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.235109+0000", 2024-09-21T15:01:54.593 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:54.593 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.593 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.593 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:54.593 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.235114+0000", 2024-09-21T15:01:54.593 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1659999999999996e-06 2024-09-21T15:01:54.593 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.593 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.593 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:54.593 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.235122+0000", 2024-09-21T15:01:54.593 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5590000000000006e-06 2024-09-21T15:01:54.594 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.594 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.594 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:54.594 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.235128+0000", 2024-09-21T15:01:54.594 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8579999999999996e-06 2024-09-21T15:01:54.594 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.594 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.594 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:54.594 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.235169+0000", 2024-09-21T15:01:54.594 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.1304000000000003e-05 2024-09-21T15:01:54.594 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.594 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.594 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:54.595 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.235178+0000", 2024-09-21T15:01:54.595 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8610000000000004e-06 2024-09-21T15:01:54.595 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.595 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.595 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:54.595 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256286+0000", 2024-09-21T15:01:54.595 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.021108162 2024-09-21T15:01:54.595 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.595 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.595 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:54.595 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256386+0000", 2024-09-21T15:01:54.595 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9871000000000007e-05 2024-09-21T15:01:54.595 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:54.595 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:54.595 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:54.596 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.596 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.596 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140168 3.e 3:76f4e81d:::rbd_data.113664780c85.000000000000006f:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:54.596 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.236992+0000", 2024-09-21T15:01:54.596 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 52.305976072999997, 2024-09-21T15:01:54.596 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.019513699, 2024-09-21T15:01:54.596 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:54.596 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:54.596 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:54.596 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:54.596 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:54.596 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140168 2024-09-21T15:01:54.596 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.596 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:54.596 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.596 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:54.597 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.236992+0000", 2024-09-21T15:01:54.597 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:54.597 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.597 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.597 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:54.597 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.236992+0000", 2024-09-21T15:01:54.597 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:54.597 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.597 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.597 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:54.597 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.236991+0000", 2024-09-21T15:01:54.597 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:54.597 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.597 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.597 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:54.598 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.236995+0000", 2024-09-21T15:01:54.598 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3100000000000001e-06 2024-09-21T15:01:54.598 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.598 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.598 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:54.598 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237002+0000", 2024-09-21T15:01:54.598 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9929999999999998e-06 2024-09-21T15:01:54.598 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.598 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.598 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:54.598 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237008+0000", 2024-09-21T15:01:54.598 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7559999999999996e-06 2024-09-21T15:01:54.598 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.598 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.598 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:54.598 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237050+0000", 2024-09-21T15:01:54.599 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2852999999999998e-05 2024-09-21T15:01:54.599 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.599 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.599 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:54.599 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237060+0000", 2024-09-21T15:01:54.599 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2399999999999996e-06 2024-09-21T15:01:54.599 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.599 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.599 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:54.599 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256413+0000", 2024-09-21T15:01:54.599 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0193528999999999 2024-09-21T15:01:54.599 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.599 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.599 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:54.600 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256506+0000", 2024-09-21T15:01:54.600 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.3154999999999998e-05 2024-09-21T15:01:54.600 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:54.600 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:54.600 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:54.600 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.600 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.600 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140186 3.e 3:75504345:::rbd_data.113664780c85.00000000000000ed:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:54.600 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.237050+0000", 2024-09-21T15:01:54.600 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 52.305918102, 2024-09-21T15:01:54.600 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0195720150000001, 2024-09-21T15:01:54.600 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:54.600 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:54.600 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:54.600 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:54.600 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:54.601 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140186 2024-09-21T15:01:54.601 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.601 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:54.601 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.601 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:54.601 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237050+0000", 2024-09-21T15:01:54.601 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:54.601 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.601 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.601 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:54.601 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237050+0000", 2024-09-21T15:01:54.601 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:54.601 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.601 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.601 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:54.602 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237049+0000", 2024-09-21T15:01:54.602 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:54.602 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.602 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.602 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:54.602 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237052+0000", 2024-09-21T15:01:54.602 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.016e-06 2024-09-21T15:01:54.602 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.602 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.602 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:54.602 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237070+0000", 2024-09-21T15:01:54.602 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7266999999999999e-05 2024-09-21T15:01:54.602 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.602 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.603 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:54.603 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237075+0000", 2024-09-21T15:01:54.603 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6910000000000002e-06 2024-09-21T15:01:54.603 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.603 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.603 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:54.603 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237232+0000", 2024-09-21T15:01:54.603 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015683599999999999 2024-09-21T15:01:54.603 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.603 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.603 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:54.603 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237237+0000", 2024-09-21T15:01:54.603 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6439999999999999e-06 2024-09-21T15:01:54.603 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.603 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.604 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:54.604 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256534+0000", 2024-09-21T15:01:54.604 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.019296736 2024-09-21T15:01:54.604 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.604 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.604 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:54.604 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256622+0000", 2024-09-21T15:01:54.604 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.8386000000000004e-05 2024-09-21T15:01:54.604 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:54.604 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:54.604 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:54.604 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.604 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.604 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140197 3.e 3:71ef5b14:::rbd_data.113664780c85.0000000000000140:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:54.605 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.237251+0000", 2024-09-21T15:01:54.605 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 52.305717074, 2024-09-21T15:01:54.605 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.01968386, 2024-09-21T15:01:54.605 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:54.605 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:54.605 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:54.605 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:54.605 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:54.605 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140197 2024-09-21T15:01:54.605 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.605 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:54.605 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.605 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:54.605 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237251+0000", 2024-09-21T15:01:54.606 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:54.606 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.606 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.606 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:54.606 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237251+0000", 2024-09-21T15:01:54.606 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:54.606 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.606 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.606 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:54.606 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237250+0000", 2024-09-21T15:01:54.606 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:54.606 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.606 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.607 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:54.607 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237254+0000", 2024-09-21T15:01:54.607 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.0180000000000003e-06 2024-09-21T15:01:54.607 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.607 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.607 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:54.607 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237262+0000", 2024-09-21T15:01:54.607 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0669999999999992e-06 2024-09-21T15:01:54.607 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.607 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.607 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:54.607 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237269+0000", 2024-09-21T15:01:54.607 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6000000000000003e-06 2024-09-21T15:01:54.607 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.607 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.608 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:54.608 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237319+0000", 2024-09-21T15:01:54.608 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9932000000000002e-05 2024-09-21T15:01:54.608 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.608 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.608 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:54.608 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237326+0000", 2024-09-21T15:01:54.608 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5360000000000004e-06 2024-09-21T15:01:54.608 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.608 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.608 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:54.608 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256657+0000", 2024-09-21T15:01:54.608 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.019330756 2024-09-21T15:01:54.608 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.608 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.609 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:54.609 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256935+0000", 2024-09-21T15:01:54.609 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00027762099999999998 2024-09-21T15:01:54.609 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:54.609 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:54.609 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:54.609 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.609 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.609 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140201 3.e 3:732ea575:::rbd_data.113664780c85.000000000000015c:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:54.609 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.237281+0000", 2024-09-21T15:01:54.609 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 52.305687235999997, 2024-09-21T15:01:54.609 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0198284529999999, 2024-09-21T15:01:54.609 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:54.609 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:54.610 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:54.610 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:54.610 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:54.610 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140201 2024-09-21T15:01:54.610 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.610 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:54.610 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.610 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:54.610 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237281+0000", 2024-09-21T15:01:54.610 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:54.610 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.610 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.610 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:54.610 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237281+0000", 2024-09-21T15:01:54.610 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:54.611 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.611 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.611 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:54.611 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237280+0000", 2024-09-21T15:01:54.611 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:54.611 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.611 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.611 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:54.611 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237284+0000", 2024-09-21T15:01:54.611 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7400000000000002e-06 2024-09-21T15:01:54.611 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.611 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.611 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:54.611 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237292+0000", 2024-09-21T15:01:54.611 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8939999999999993e-06 2024-09-21T15:01:54.612 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.612 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.612 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:54.612 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237298+0000", 2024-09-21T15:01:54.612 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.8000000000000004e-06 2024-09-21T15:01:54.612 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.612 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.612 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:54.612 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237521+0000", 2024-09-21T15:01:54.612 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00022326 2024-09-21T15:01:54.612 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.612 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.612 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:54.612 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237529+0000", 2024-09-21T15:01:54.612 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6170000000000001e-06 2024-09-21T15:01:54.613 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.613 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.613 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:54.613 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256968+0000", 2024-09-21T15:01:54.613 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.019439113 2024-09-21T15:01:54.613 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.613 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.613 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:54.613 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.257109+0000", 2024-09-21T15:01:54.613 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00014148399999999999 2024-09-21T15:01:54.613 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:54.613 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:54.613 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:54.613 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.614 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.614 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140206 3.e 3:76d38029:::rbd_data.113664780c85.0000000000000179:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:54.614 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.237520+0000", 2024-09-21T15:01:54.614 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 52.305448122000001, 2024-09-21T15:01:54.614 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0210658699999999, 2024-09-21T15:01:54.614 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:54.614 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:54.614 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:54.614 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:54.614 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:54.614 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140206 2024-09-21T15:01:54.614 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.614 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:54.614 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.615 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:54.615 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237520+0000", 2024-09-21T15:01:54.615 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:54.615 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.615 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.615 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:54.615 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237520+0000", 2024-09-21T15:01:54.615 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:54.615 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.615 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.615 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:54.615 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237519+0000", 2024-09-21T15:01:54.615 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:54.615 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.615 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.616 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:54.616 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237523+0000", 2024-09-21T15:01:54.616 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.9009999999999996e-06 2024-09-21T15:01:54.616 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.616 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.616 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:54.616 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237531+0000", 2024-09-21T15:01:54.616 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7069999999999992e-06 2024-09-21T15:01:54.616 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.616 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.616 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:54.616 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237537+0000", 2024-09-21T15:01:54.616 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4559999999999997e-06 2024-09-21T15:01:54.616 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.616 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.617 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:54.617 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237583+0000", 2024-09-21T15:01:54.617 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5532999999999999e-05 2024-09-21T15:01:54.617 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.617 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.617 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:54.617 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.237592+0000", 2024-09-21T15:01:54.617 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6009999999999995e-06 2024-09-21T15:01:54.617 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.617 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.617 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:54.617 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.257157+0000", 2024-09-21T15:01:54.617 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0195657279999999 2024-09-21T15:01:54.617 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.617 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.617 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:54.618 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.258586+0000", 2024-09-21T15:01:54.618 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00142847 2024-09-21T15:01:54.618 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:54.618 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:54.618 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:54.618 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.618 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.618 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140338 3.e 3:71dba096:::rbd_data.113664780c85.000000000000018b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:54.618 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.243856+0000", 2024-09-21T15:01:54.618 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 52.299112289, 2024-09-21T15:01:54.618 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0138226889999999, 2024-09-21T15:01:54.618 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:54.618 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:54.618 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:54.618 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:54.619 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:54.619 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140338 2024-09-21T15:01:54.619 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.619 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:54.619 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.619 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:54.619 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243856+0000", 2024-09-21T15:01:54.619 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:54.619 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.619 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.619 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:54.619 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243856+0000", 2024-09-21T15:01:54.619 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:54.619 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.619 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.620 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:54.620 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243854+0000", 2024-09-21T15:01:54.620 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:54.620 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.620 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.620 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:54.620 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243862+0000", 2024-09-21T15:01:54.620 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2570000000000001e-06 2024-09-21T15:01:54.620 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.620 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.620 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:54.620 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243877+0000", 2024-09-21T15:01:54.620 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.571e-05 2024-09-21T15:01:54.620 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.620 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.620 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:54.621 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243884+0000", 2024-09-21T15:01:54.621 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.1269999999999997e-06 2024-09-21T15:01:54.621 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.621 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.621 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:54.621 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243932+0000", 2024-09-21T15:01:54.621 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7655999999999999e-05 2024-09-21T15:01:54.621 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.621 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.621 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:54.621 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243942+0000", 2024-09-21T15:01:54.621 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6679999999999996e-06 2024-09-21T15:01:54.621 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.621 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.621 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:54.622 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.257568+0000", 2024-09-21T15:01:54.622 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0136267670000001 2024-09-21T15:01:54.622 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.622 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.622 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:54.622 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.257678+0000", 2024-09-21T15:01:54.622 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000110011 2024-09-21T15:01:54.622 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:54.622 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:54.622 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:54.622 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.622 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.622 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140345 3.e 3:76ddeb1d:::rbd_data.113664780c85.0000000000000004:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:54.622 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.243927+0000", 2024-09-21T15:01:54.622 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 52.299040705000003, 2024-09-21T15:01:54.622 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0140328439999999, 2024-09-21T15:01:54.623 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:54.623 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:54.623 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:54.623 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:54.623 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:54.623 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140345 2024-09-21T15:01:54.623 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.623 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:54.623 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.623 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:54.623 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243927+0000", 2024-09-21T15:01:54.623 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:54.623 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.623 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.623 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:54.624 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243927+0000", 2024-09-21T15:01:54.624 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:54.624 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.624 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.624 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:54.624 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243927+0000", 2024-09-21T15:01:54.624 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:54.624 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.624 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.624 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:54.624 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243930+0000", 2024-09-21T15:01:54.624 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1319999999999998e-06 2024-09-21T15:01:54.624 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.624 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.624 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:54.625 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243937+0000", 2024-09-21T15:01:54.625 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4330000000000001e-06 2024-09-21T15:01:54.625 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.625 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.625 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:54.625 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243942+0000", 2024-09-21T15:01:54.625 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.5090000000000001e-06 2024-09-21T15:01:54.625 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.625 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.625 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:54.625 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243964+0000", 2024-09-21T15:01:54.625 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1838999999999999e-05 2024-09-21T15:01:54.625 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.625 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.625 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:54.625 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243971+0000", 2024-09-21T15:01:54.626 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.131e-06 2024-09-21T15:01:54.626 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.626 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.626 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:54.626 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.257704+0000", 2024-09-21T15:01:54.626 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0137329310000001 2024-09-21T15:01:54.626 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.626 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.626 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:54.626 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.257805+0000", 2024-09-21T15:01:54.626 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000101121 2024-09-21T15:01:54.626 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.626 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.626 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:54.626 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.257960+0000", 2024-09-21T15:01:54.626 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00015518500000000001 2024-09-21T15:01:54.627 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:54.627 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:54.627 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:54.627 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.627 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.627 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140350 3.e 3:76b9b799:::rbd_data.113664780c85.0000000000000021:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:54.627 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.243989+0000", 2024-09-21T15:01:54.627 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 52.298978953999999, 2024-09-21T15:01:54.627 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0142022340000001, 2024-09-21T15:01:54.627 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:54.627 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:54.627 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:54.627 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:54.627 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:54.627 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140350 2024-09-21T15:01:54.628 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.628 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:54.628 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.628 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:54.628 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243989+0000", 2024-09-21T15:01:54.628 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:54.628 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.628 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.628 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:54.628 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243989+0000", 2024-09-21T15:01:54.628 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:54.628 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.628 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.628 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:54.628 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243989+0000", 2024-09-21T15:01:54.628 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:54.629 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.629 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.629 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:54.629 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243992+0000", 2024-09-21T15:01:54.629 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.236e-06 2024-09-21T15:01:54.629 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.629 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.629 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:54.629 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.243998+0000", 2024-09-21T15:01:54.629 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2500000000000003e-06 2024-09-21T15:01:54.629 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.629 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.629 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:54.629 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.244003+0000", 2024-09-21T15:01:54.629 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7740000000000004e-06 2024-09-21T15:01:54.630 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.630 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.630 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:54.630 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.244047+0000", 2024-09-21T15:01:54.630 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3665999999999998e-05 2024-09-21T15:01:54.630 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.630 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.630 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:54.630 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.244055+0000", 2024-09-21T15:01:54.630 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0639999999999994e-06 2024-09-21T15:01:54.630 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.630 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.630 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:54.630 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.257985+0000", 2024-09-21T15:01:54.631 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0139306079999999 2024-09-21T15:01:54.631 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.631 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.631 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:54.631 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.258084+0000", 2024-09-21T15:01:54.631 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.8610000000000006e-05 2024-09-21T15:01:54.631 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.631 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.631 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:54.631 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.258191+0000", 2024-09-21T15:01:54.631 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000107518 2024-09-21T15:01:54.631 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:54.631 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:54.631 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:54.631 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.631 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.632 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140394 3.e 3:7756a55d:::rbd_data.113664780c85.0000000000000154:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:54.632 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.245038+0000", 2024-09-21T15:01:54.632 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 52.297929736999997, 2024-09-21T15:01:54.632 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.013225182, 2024-09-21T15:01:54.632 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:54.632 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:54.632 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:54.632 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:54.632 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:54.632 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140394 2024-09-21T15:01:54.632 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.632 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:54.632 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.632 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:54.632 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.245038+0000", 2024-09-21T15:01:54.633 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:54.633 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.633 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.633 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:54.633 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.245038+0000", 2024-09-21T15:01:54.633 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:54.633 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.633 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.633 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:54.633 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.245037+0000", 2024-09-21T15:01:54.633 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:54.633 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.633 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.633 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:54.633 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.245044+0000", 2024-09-21T15:01:54.633 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3239999999999999e-06 2024-09-21T15:01:54.634 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.634 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.634 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:54.634 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.245061+0000", 2024-09-21T15:01:54.634 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.7166000000000001e-05 2024-09-21T15:01:54.634 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.634 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.634 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:54.634 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.245069+0000", 2024-09-21T15:01:54.634 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0779999999999996e-06 2024-09-21T15:01:54.634 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.634 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.634 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:54.634 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.245118+0000", 2024-09-21T15:01:54.634 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8887000000000001e-05 2024-09-21T15:01:54.635 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.635 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.635 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:54.635 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.245126+0000", 2024-09-21T15:01:54.635 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0409999999999998e-06 2024-09-21T15:01:54.635 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.635 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.635 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:54.635 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.258206+0000", 2024-09-21T15:01:54.635 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.013080451 2024-09-21T15:01:54.635 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:54.635 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:54.635 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:54.635 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.258264+0000", 2024-09-21T15:01:54.635 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7294999999999999e-05 2024-09-21T15:01:54.635 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:54.636 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:54.636 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:54.636 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:54.636 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:54.636 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:54.637 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T15:01:54.638 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:01:54.632+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:01:54.738 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:01:54.737+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:01:54.744 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:54.744 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:01:54.745 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T15:01:54.745 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:54.754 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T15:01:54.839 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:01:54.837+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:01:54.910 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:54.910 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:01:54.910 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T15:01:54.910 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T15:01:54.910 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:54.921 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T15:01:54.940 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:01:54.936+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:01:55.030 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:55.030 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T15:01:55.030 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T15:01:55.030 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T15:01:55.030 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.030 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T15:01:55.030 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.205754+0000", 2024-09-21T15:01:55.030 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 231.821635394, 2024-09-21T15:01:55.030 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.41117094199999998, 2024-09-21T15:01:55.030 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:55.030 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:55.031 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:55.031 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T15:01:55.031 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T15:01:55.031 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1 2024-09-21T15:01:55.031 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.031 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:55.031 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.031 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:55.031 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205754+0000", 2024-09-21T15:01:55.031 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:55.031 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.031 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.031 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:55.032 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205754+0000", 2024-09-21T15:01:55.032 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:55.032 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.032 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.032 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:55.032 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205752+0000", 2024-09-21T15:01:55.032 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T15:01:55.032 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.032 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.032 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:55.032 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205765+0000", 2024-09-21T15:01:55.032 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3230999999999999e-05 2024-09-21T15:01:55.032 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.032 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.033 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:55.033 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205782+0000", 2024-09-21T15:01:55.033 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6634e-05 2024-09-21T15:01:55.033 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.033 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.033 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:55.033 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205793+0000", 2024-09-21T15:01:55.033 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1297e-05 2024-09-21T15:01:55.033 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.033 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.033 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:55.033 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205867+0000", 2024-09-21T15:01:55.033 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4269000000000003e-05 2024-09-21T15:01:55.033 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.034 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.034 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:55.034 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205877+0000", 2024-09-21T15:01:55.034 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2690000000000005e-06 2024-09-21T15:01:55.034 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.034 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.034 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:55.034 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.613700+0000", 2024-09-21T15:01:55.034 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.407823455 2024-09-21T15:01:55.034 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.034 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.034 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:55.034 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.614203+0000", 2024-09-21T15:01:55.035 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00050311500000000005 2024-09-21T15:01:55.035 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.035 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.035 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T15:01:55.035 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.614750+0000", 2024-09-21T15:01:55.035 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00054738699999999998 2024-09-21T15:01:55.035 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.035 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.035 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:01:55.035 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616367+0000", 2024-09-21T15:01:55.035 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001616634 2024-09-21T15:01:55.035 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.035 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.035 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:01:55.035 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616574+0000", 2024-09-21T15:01:55.036 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00020668499999999999 2024-09-21T15:01:55.036 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.036 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.036 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:01:55.036 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616802+0000", 2024-09-21T15:01:55.036 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00022817900000000001 2024-09-21T15:01:55.036 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.036 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.036 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:55.036 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616925+0000", 2024-09-21T15:01:55.036 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012309699999999999 2024-09-21T15:01:55.036 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:55.036 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:55.037 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:55.037 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.037 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.037 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:61214 3.7 3:e28aec10:::rbd_header.1136862f872f:head [watch reconnect cookie 139864671932080 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-21T15:01:55.037 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:18.147172+0000", 2024-09-21T15:01:55.037 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 156.880217751, 2024-09-21T15:01:55.037 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.019264956, 2024-09-21T15:01:55.037 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:55.037 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:55.037 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:55.037 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:55.037 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:55.037 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 61214 2024-09-21T15:01:55.038 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.038 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:55.038 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.038 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:55.038 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147172+0000", 2024-09-21T15:01:55.038 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:55.038 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.038 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.038 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:55.038 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147172+0000", 2024-09-21T15:01:55.038 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:55.038 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.038 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.038 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:55.039 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147168+0000", 2024-09-21T15:01:55.039 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999962 2024-09-21T15:01:55.039 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.039 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.039 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:55.039 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147188+0000", 2024-09-21T15:01:55.039 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9717000000000001e-05 2024-09-21T15:01:55.039 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.039 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.039 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:55.039 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147239+0000", 2024-09-21T15:01:55.039 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1493999999999999e-05 2024-09-21T15:01:55.039 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.039 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.040 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:55.040 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147265+0000", 2024-09-21T15:01:55.040 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5746999999999999e-05 2024-09-21T15:01:55.040 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.040 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.040 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:55.040 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147372+0000", 2024-09-21T15:01:55.040 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010681200000000001 2024-09-21T15:01:55.040 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.040 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.040 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op must wait for map", 2024-09-21T15:01:55.040 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147380+0000", 2024-09-21T15:01:55.040 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7919999999999993e-06 2024-09-21T15:01:55.040 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.040 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.040 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:55.041 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.154025+0000", 2024-09-21T15:01:55.041 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0066454909999999999 2024-09-21T15:01:55.041 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.041 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.041 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:55.041 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.154041+0000", 2024-09-21T15:01:55.041 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5894000000000001e-05 2024-09-21T15:01:55.041 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.041 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.041 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:55.041 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.164345+0000", 2024-09-21T15:01:55.041 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0103040459999999 2024-09-21T15:01:55.041 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.042 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.042 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:55.042 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.164590+0000", 2024-09-21T15:01:55.042 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00024492699999999998 2024-09-21T15:01:55.042 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.042 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.042 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:55.042 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.166437+0000", 2024-09-21T15:01:55.042 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0018466330000000001 2024-09-21T15:01:55.042 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:55.042 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:55.042 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:55.042 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.043 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.043 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:61215 3.7 3:e28aec10:::rbd_header.1136862f872f:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-21T15:01:55.043 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:18.152008+0000", 2024-09-21T15:01:55.043 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 156.875381299, 2024-09-21T15:01:55.043 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.015644958, 2024-09-21T15:01:55.043 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:55.043 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:55.043 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:55.043 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:55.043 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:55.043 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 61215 2024-09-21T15:01:55.043 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.043 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:55.043 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.044 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:55.044 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152008+0000", 2024-09-21T15:01:55.044 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:55.044 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.044 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.044 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:55.044 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152008+0000", 2024-09-21T15:01:55.044 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:55.044 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.044 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.044 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:55.044 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152005+0000", 2024-09-21T15:01:55.044 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-21T15:01:55.044 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.045 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.045 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:55.045 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152025+0000", 2024-09-21T15:01:55.045 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9306000000000001e-05 2024-09-21T15:01:55.045 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.045 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.045 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:55.045 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152054+0000", 2024-09-21T15:01:55.045 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9189000000000001e-05 2024-09-21T15:01:55.045 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.045 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.045 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:55.045 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152078+0000", 2024-09-21T15:01:55.045 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4317000000000001e-05 2024-09-21T15:01:55.045 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.045 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.046 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:55.046 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152243+0000", 2024-09-21T15:01:55.046 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000164802 2024-09-21T15:01:55.046 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.046 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.046 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting_for_map not empty", 2024-09-21T15:01:55.046 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152251+0000", 2024-09-21T15:01:55.046 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0600000000000008e-06 2024-09-21T15:01:55.046 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.046 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.046 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:55.046 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.154741+0000", 2024-09-21T15:01:55.046 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0024900069999999998 2024-09-21T15:01:55.046 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.046 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.047 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:55.047 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.154753+0000", 2024-09-21T15:01:55.047 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1588e-05 2024-09-21T15:01:55.047 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.047 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.047 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:55.047 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.165380+0000", 2024-09-21T15:01:55.047 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0106272249999999 2024-09-21T15:01:55.047 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.047 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.047 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:55.047 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.165517+0000", 2024-09-21T15:01:55.047 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013737999999999999 2024-09-21T15:01:55.047 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.047 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.047 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 6", 2024-09-21T15:01:55.048 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.165938+0000", 2024-09-21T15:01:55.048 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00042082399999999998 2024-09-21T15:01:55.048 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.048 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.048 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:01:55.048 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.166795+0000", 2024-09-21T15:01:55.048 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00085674899999999997 2024-09-21T15:01:55.048 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.048 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.048 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:01:55.048 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.167598+0000", 2024-09-21T15:01:55.048 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00080335599999999997 2024-09-21T15:01:55.048 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.048 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.048 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:01:55.049 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.167632+0000", 2024-09-21T15:01:55.049 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4122000000000003e-05 2024-09-21T15:01:55.049 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.049 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.049 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:55.049 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.167653+0000", 2024-09-21T15:01:55.049 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0862000000000001e-05 2024-09-21T15:01:55.049 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:55.049 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:55.049 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:55.049 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.049 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.049 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:97678 3.d e122/24 3:b2f34c6d:::rbd_data.1136bfc79a70.0000000000000006:head v 122'234, mlcod=120'232)", 2024-09-21T15:01:55.049 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.321655+0000", 2024-09-21T15:01:55.050 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 109.705734382, 2024-09-21T15:01:55.050 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024509388999999999, 2024-09-21T15:01:55.050 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:55.050 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:55.050 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:55.050 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.050 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:55.050 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.321655+0000", 2024-09-21T15:01:55.050 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:55.050 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.050 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.050 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:55.050 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.321655+0000", 2024-09-21T15:01:55.051 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:55.051 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.051 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.051 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:55.051 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.321655+0000", 2024-09-21T15:01:55.051 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:55.051 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.051 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.051 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:55.051 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.329453+0000", 2024-09-21T15:01:55.051 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0077980560000000003 2024-09-21T15:01:55.051 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.051 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.052 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:55.052 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.329482+0000", 2024-09-21T15:01:55.052 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9153999999999999e-05 2024-09-21T15:01:55.052 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.052 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.052 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:55.052 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.329493+0000", 2024-09-21T15:01:55.052 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1103e-05 2024-09-21T15:01:55.052 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.052 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.052 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:55.052 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.329555+0000", 2024-09-21T15:01:55.052 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2345e-05 2024-09-21T15:01:55.052 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.052 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.052 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:55.052 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.329589+0000", 2024-09-21T15:01:55.053 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3908999999999997e-05 2024-09-21T15:01:55.053 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.053 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.053 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:01:55.053 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.346083+0000", 2024-09-21T15:01:55.053 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016493383 2024-09-21T15:01:55.053 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.053 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.053 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:55.053 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.346165+0000", 2024-09-21T15:01:55.053 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1796000000000001e-05 2024-09-21T15:01:55.053 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:55.053 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:55.053 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:55.053 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.053 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.054 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:97679 3.f e122/24 3:f2a362b4:::rbd_data.1136bfc79a70.0000000000000007:head v 122'331, mlcod=120'329)", 2024-09-21T15:01:55.054 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.337145+0000", 2024-09-21T15:01:55.054 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 109.690244373, 2024-09-21T15:01:55.054 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021912411, 2024-09-21T15:01:55.054 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:55.054 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:55.054 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:55.054 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.054 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:55.054 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.337145+0000", 2024-09-21T15:01:55.054 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:55.054 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.054 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.055 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:55.055 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.337145+0000", 2024-09-21T15:01:55.055 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:55.055 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.055 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.055 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:55.055 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.337145+0000", 2024-09-21T15:01:55.055 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:55.055 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.055 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.055 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:55.056 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.343899+0000", 2024-09-21T15:01:55.056 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0067541420000000003 2024-09-21T15:01:55.056 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.056 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.056 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:55.056 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.343921+0000", 2024-09-21T15:01:55.056 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2255e-05 2024-09-21T15:01:55.056 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.056 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.056 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:55.056 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.343933+0000", 2024-09-21T15:01:55.056 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1528e-05 2024-09-21T15:01:55.056 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.056 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.056 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:55.057 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.344001+0000", 2024-09-21T15:01:55.057 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8269999999999995e-05 2024-09-21T15:01:55.057 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.057 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.057 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:55.057 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.344029+0000", 2024-09-21T15:01:55.057 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8254000000000001e-05 2024-09-21T15:01:55.057 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.057 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.057 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:01:55.057 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.358901+0000", 2024-09-21T15:01:55.057 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0148719 2024-09-21T15:01:55.057 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.057 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.058 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:55.058 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.359058+0000", 2024-09-21T15:01:55.058 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000156446 2024-09-21T15:01:55.058 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:55.058 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:55.058 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:55.058 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.058 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.058 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:97676 3.1f e122/24 3:ffaa836a:::rbd_data.1136bfc79a70.0000000000000004:head v 122'312, mlcod=120'310)", 2024-09-21T15:01:55.058 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.348683+0000", 2024-09-21T15:01:55.058 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 109.67870620799999, 2024-09-21T15:01:55.058 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.033768802000000001, 2024-09-21T15:01:55.058 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:55.058 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:55.059 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:55.059 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.059 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:55.059 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.348683+0000", 2024-09-21T15:01:55.059 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:55.059 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.059 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.059 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:55.059 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.348683+0000", 2024-09-21T15:01:55.059 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:55.059 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.059 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.059 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:55.059 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.348683+0000", 2024-09-21T15:01:55.059 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:55.060 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.060 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.060 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:55.060 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.366937+0000", 2024-09-21T15:01:55.060 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018254415 2024-09-21T15:01:55.060 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.060 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.060 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:55.060 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.366959+0000", 2024-09-21T15:01:55.060 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1634e-05 2024-09-21T15:01:55.060 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.060 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.060 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:55.060 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.366971+0000", 2024-09-21T15:01:55.060 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2305e-05 2024-09-21T15:01:55.061 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.061 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.061 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:55.061 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.367034+0000", 2024-09-21T15:01:55.061 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3211000000000002e-05 2024-09-21T15:01:55.061 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.061 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.061 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:55.061 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.367056+0000", 2024-09-21T15:01:55.061 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2098000000000001e-05 2024-09-21T15:01:55.061 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.061 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.061 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:01:55.061 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.382400+0000", 2024-09-21T15:01:55.061 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.015343867000000001 2024-09-21T15:01:55.062 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.062 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.062 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:55.062 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.382452+0000", 2024-09-21T15:01:55.062 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1992e-05 2024-09-21T15:01:55.062 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:55.062 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:55.062 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:55.062 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.062 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.062 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:97734 3.19 e122/94 3:9fcf754f:::rbd_data.1136bfc79a70.0000000000000009:head v 122'314, mlcod=120'312)", 2024-09-21T15:01:55.062 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.378547+0000", 2024-09-21T15:01:55.062 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 109.64884223, 2024-09-21T15:01:55.063 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.063380675999999997, 2024-09-21T15:01:55.063 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:55.063 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:55.063 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:55.063 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.063 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:55.063 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.378547+0000", 2024-09-21T15:01:55.063 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:55.063 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.063 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.063 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:55.063 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.378547+0000", 2024-09-21T15:01:55.063 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:55.063 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.063 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.064 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:55.064 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.378547+0000", 2024-09-21T15:01:55.064 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:01:55.064 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.064 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.064 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:55.064 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.425118+0000", 2024-09-21T15:01:55.064 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046571166999999997 2024-09-21T15:01:55.064 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.064 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.064 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:55.064 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.425150+0000", 2024-09-21T15:01:55.064 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2150999999999998e-05 2024-09-21T15:01:55.064 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.065 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.065 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:55.065 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.425164+0000", 2024-09-21T15:01:55.065 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3302000000000001e-05 2024-09-21T15:01:55.065 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.065 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.065 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:55.065 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.425231+0000", 2024-09-21T15:01:55.065 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7106000000000004e-05 2024-09-21T15:01:55.065 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.065 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.065 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:55.065 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.425259+0000", 2024-09-21T15:01:55.065 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8087000000000001e-05 2024-09-21T15:01:55.065 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.065 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.066 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:01:55.066 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.441890+0000", 2024-09-21T15:01:55.066 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016631328000000001 2024-09-21T15:01:55.066 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.066 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.066 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:55.066 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.441928+0000", 2024-09-21T15:01:55.066 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7701999999999999e-05 2024-09-21T15:01:55.066 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:55.066 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:55.066 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:55.066 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.066 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.066 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:100187 3.10 3:08941a97:::rbd_data.1136ab4d6ff1.000000000000000e:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-21T15:01:55.067 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:10.869845+0000", 2024-09-21T15:01:55.067 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 104.15754433399999, 2024-09-21T15:01:55.067 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029345237, 2024-09-21T15:01:55.067 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:55.067 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:55.067 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:55.067 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:55.067 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:55.067 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 100187 2024-09-21T15:01:55.067 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.067 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:55.067 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.067 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:55.067 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869845+0000", 2024-09-21T15:01:55.067 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:55.068 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.068 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.068 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:55.068 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869845+0000", 2024-09-21T15:01:55.068 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:55.068 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.068 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.068 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:55.068 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869844+0000", 2024-09-21T15:01:55.068 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:55.068 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.068 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.068 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:55.068 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869853+0000", 2024-09-21T15:01:55.069 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5719999999999992e-06 2024-09-21T15:01:55.069 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.069 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.069 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:55.069 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869874+0000", 2024-09-21T15:01:55.069 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0483e-05 2024-09-21T15:01:55.069 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.069 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.069 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:55.069 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869883+0000", 2024-09-21T15:01:55.069 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6080000000000002e-06 2024-09-21T15:01:55.069 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.069 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.069 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:55.069 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869991+0000", 2024-09-21T15:01:55.070 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000108013 2024-09-21T15:01:55.070 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.070 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.070 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:55.070 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.870258+0000", 2024-09-21T15:01:55.070 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00026629099999999999 2024-09-21T15:01:55.070 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.070 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.070 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T15:01:55.070 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.878656+0000", 2024-09-21T15:01:55.070 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0083989000000000008 2024-09-21T15:01:55.070 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.070 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.071 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:01:55.071 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.899081+0000", 2024-09-21T15:01:55.071 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.020424437 2024-09-21T15:01:55.071 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.071 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.071 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:01:55.071 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.899125+0000", 2024-09-21T15:01:55.071 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4440999999999997e-05 2024-09-21T15:01:55.071 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.071 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.071 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:01:55.071 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.899160+0000", 2024-09-21T15:01:55.071 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4518999999999999e-05 2024-09-21T15:01:55.071 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.071 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.071 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:55.072 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.899190+0000", 2024-09-21T15:01:55.072 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0574999999999998e-05 2024-09-21T15:01:55.072 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:55.072 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:55.072 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:55.072 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.072 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.072 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:100263 3.10 3:0ece142e:::rbd_data.1136ab4d6ff1.0000000000000008:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-21T15:01:55.072 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:11.052257+0000", 2024-09-21T15:01:55.072 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 103.975132276, 2024-09-21T15:01:55.072 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042678025000000001, 2024-09-21T15:01:55.072 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:55.072 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:55.072 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:55.072 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:55.073 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:55.073 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 100263 2024-09-21T15:01:55.073 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.073 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:55.073 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.073 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:55.073 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052257+0000", 2024-09-21T15:01:55.073 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:55.073 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.073 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.073 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:55.073 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052257+0000", 2024-09-21T15:01:55.073 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:55.073 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.074 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.074 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:55.074 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052256+0000", 2024-09-21T15:01:55.074 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:55.074 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.074 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.074 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:55.074 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052263+0000", 2024-09-21T15:01:55.074 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4799999999999998e-06 2024-09-21T15:01:55.074 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.074 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.074 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:55.074 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052275+0000", 2024-09-21T15:01:55.074 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2386e-05 2024-09-21T15:01:55.075 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.075 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.075 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:55.075 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052281+0000", 2024-09-21T15:01:55.075 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6699999999999999e-06 2024-09-21T15:01:55.075 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.075 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.075 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:55.075 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052350+0000", 2024-09-21T15:01:55.075 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9099999999999999e-05 2024-09-21T15:01:55.075 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.075 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.075 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:55.075 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052456+0000", 2024-09-21T15:01:55.075 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000106669 2024-09-21T15:01:55.076 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.076 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.076 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T15:01:55.076 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.059694+0000", 2024-09-21T15:01:55.076 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.007237566 2024-09-21T15:01:55.076 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.076 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.076 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:01:55.076 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.094721+0000", 2024-09-21T15:01:55.076 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035026977000000001 2024-09-21T15:01:55.076 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.076 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.076 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:01:55.076 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.094746+0000", 2024-09-21T15:01:55.076 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5329000000000001e-05 2024-09-21T15:01:55.076 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.077 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.077 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:01:55.077 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.094850+0000", 2024-09-21T15:01:55.077 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000103452 2024-09-21T15:01:55.077 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.077 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.077 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:55.077 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.094935+0000", 2024-09-21T15:01:55.077 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5550000000000006e-05 2024-09-21T15:01:55.077 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:55.077 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:55.077 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:55.077 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.077 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.078 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:134708 3.a 3:51eaadc0:::rbd_data.11365ed419e3.0000000000000007:67 [read 221184~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:55.078 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.938945+0000", 2024-09-21T15:01:55.078 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 61.088444998, 2024-09-21T15:01:55.078 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021073784000000002, 2024-09-21T15:01:55.078 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:55.078 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:55.078 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:55.078 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:55.078 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:55.078 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 134708 2024-09-21T15:01:55.078 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.078 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:55.078 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.078 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:55.078 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938945+0000", 2024-09-21T15:01:55.079 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:55.079 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.079 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.079 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:55.079 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938945+0000", 2024-09-21T15:01:55.079 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:55.079 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.079 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.079 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:55.079 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938944+0000", 2024-09-21T15:01:55.079 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:55.079 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.079 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.079 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:55.079 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938946+0000", 2024-09-21T15:01:55.079 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.616e-06 2024-09-21T15:01:55.080 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.080 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.080 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:55.080 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938952+0000", 2024-09-21T15:01:55.080 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7309999999999998e-06 2024-09-21T15:01:55.080 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.080 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.080 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:55.080 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938957+0000", 2024-09-21T15:01:55.080 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8579999999999999e-06 2024-09-21T15:01:55.080 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.080 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.080 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:55.080 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.959730+0000", 2024-09-21T15:01:55.080 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.020773183000000001 2024-09-21T15:01:55.080 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.081 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.081 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:55.081 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.959810+0000", 2024-09-21T15:01:55.081 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0158000000000001e-05 2024-09-21T15:01:55.081 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.081 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.081 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:55.081 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.960018+0000", 2024-09-21T15:01:55.081 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000207951 2024-09-21T15:01:55.081 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:55.081 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:55.081 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:55.081 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.081 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.082 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:134709 3.a 3:51eaadc0:::rbd_data.11365ed419e3.0000000000000007:67 [read 237568~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:55.082 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.938996+0000", 2024-09-21T15:01:55.082 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 61.088394006000001, 2024-09-21T15:01:55.082 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021382577, 2024-09-21T15:01:55.082 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:55.082 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:55.082 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:55.082 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:55.082 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:55.082 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 134709 2024-09-21T15:01:55.082 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.082 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:55.082 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.082 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:55.082 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938996+0000", 2024-09-21T15:01:55.083 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:55.083 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.083 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.083 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:55.083 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938996+0000", 2024-09-21T15:01:55.083 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:55.083 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.083 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.083 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:55.083 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938995+0000", 2024-09-21T15:01:55.083 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:55.083 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.083 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.083 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:55.084 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938999+0000", 2024-09-21T15:01:55.084 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7979999999999999e-06 2024-09-21T15:01:55.084 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.084 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.084 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:55.084 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939006+0000", 2024-09-21T15:01:55.084 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6669999999999996e-06 2024-09-21T15:01:55.084 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.084 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.084 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:55.084 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939012+0000", 2024-09-21T15:01:55.084 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9329999999999998e-06 2024-09-21T15:01:55.084 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.085 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.085 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:55.085 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.960080+0000", 2024-09-21T15:01:55.085 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021068303 2024-09-21T15:01:55.085 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.085 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.085 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:55.085 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.960154+0000", 2024-09-21T15:01:55.085 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3819000000000006e-05 2024-09-21T15:01:55.085 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.085 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.085 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:55.085 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.960378+0000", 2024-09-21T15:01:55.085 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00022385800000000001 2024-09-21T15:01:55.085 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:55.086 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:55.086 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:55.086 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.086 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.086 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:134710 3.a 3:51eaadc0:::rbd_data.11365ed419e3.0000000000000007:67 [read 253952~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:55.086 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.939017+0000", 2024-09-21T15:01:55.086 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 61.088372995, 2024-09-21T15:01:55.086 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021721436, 2024-09-21T15:01:55.086 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:55.086 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:55.086 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:55.086 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:55.086 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:55.086 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 134710 2024-09-21T15:01:55.087 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.087 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:55.087 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.087 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:55.087 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939017+0000", 2024-09-21T15:01:55.087 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:55.087 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.087 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.087 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:55.087 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939017+0000", 2024-09-21T15:01:55.087 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:55.087 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.087 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.087 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:55.087 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939016+0000", 2024-09-21T15:01:55.088 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:55.088 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.088 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.088 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:55.088 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939019+0000", 2024-09-21T15:01:55.088 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.554e-06 2024-09-21T15:01:55.088 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.088 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.088 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:55.088 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939024+0000", 2024-09-21T15:01:55.088 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1569999999999999e-06 2024-09-21T15:01:55.088 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.088 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.088 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:55.089 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939034+0000", 2024-09-21T15:01:55.089 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9219999999999992e-06 2024-09-21T15:01:55.089 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.089 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.089 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:55.089 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.960408+0000", 2024-09-21T15:01:55.089 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021373792999999999 2024-09-21T15:01:55.089 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.089 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.089 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:55.089 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.960482+0000", 2024-09-21T15:01:55.089 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4331000000000004e-05 2024-09-21T15:01:55.089 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.089 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.089 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:55.090 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.960738+0000", 2024-09-21T15:01:55.090 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00025600500000000001 2024-09-21T15:01:55.090 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:55.090 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:55.090 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:55.090 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.090 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.090 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:134711 3.a 3:51eaadc0:::rbd_data.11365ed419e3.0000000000000007:67 [read 270336~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:55.090 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.939039+0000", 2024-09-21T15:01:55.090 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 61.088350945999998, 2024-09-21T15:01:55.090 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022064698000000001, 2024-09-21T15:01:55.090 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:55.090 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:55.090 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:55.091 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:55.091 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:55.091 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 134711 2024-09-21T15:01:55.091 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.091 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:55.091 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.091 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:55.091 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939039+0000", 2024-09-21T15:01:55.091 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:55.091 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.091 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.091 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:55.091 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939039+0000", 2024-09-21T15:01:55.091 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:55.091 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.092 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.092 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:55.092 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939038+0000", 2024-09-21T15:01:55.092 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:55.092 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.092 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.092 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:55.092 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939041+0000", 2024-09-21T15:01:55.092 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9440000000000001e-06 2024-09-21T15:01:55.092 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.092 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.092 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:55.092 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939047+0000", 2024-09-21T15:01:55.092 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1260000000000003e-06 2024-09-21T15:01:55.092 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.093 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.093 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:55.093 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939052+0000", 2024-09-21T15:01:55.093 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1619999999999999e-06 2024-09-21T15:01:55.093 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.093 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.093 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:55.093 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.960768+0000", 2024-09-21T15:01:55.093 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021715140000000001 2024-09-21T15:01:55.093 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.093 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.093 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:55.093 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.960842+0000", 2024-09-21T15:01:55.093 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4698999999999997e-05 2024-09-21T15:01:55.093 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.094 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.094 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:55.094 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.961103+0000", 2024-09-21T15:01:55.094 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00026098499999999998 2024-09-21T15:01:55.094 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:55.094 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:55.094 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:55.094 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.094 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.094 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:134712 3.a 3:51eaadc0:::rbd_data.11365ed419e3.0000000000000007:67 [read 286720~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:55.094 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.939057+0000", 2024-09-21T15:01:55.094 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 61.088332413000003, 2024-09-21T15:01:55.094 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022387067, 2024-09-21T15:01:55.094 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:55.095 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:55.095 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:55.095 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:55.095 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:55.095 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 134712 2024-09-21T15:01:55.095 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.095 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:55.095 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.095 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:55.095 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939057+0000", 2024-09-21T15:01:55.095 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:55.095 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.095 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.095 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:55.095 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939057+0000", 2024-09-21T15:01:55.096 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:55.096 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.096 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.096 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:55.096 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939057+0000", 2024-09-21T15:01:55.096 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:55.096 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.096 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.096 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:55.096 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939059+0000", 2024-09-21T15:01:55.096 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.447e-06 2024-09-21T15:01:55.096 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.096 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.096 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:55.096 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939065+0000", 2024-09-21T15:01:55.097 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4670000000000003e-06 2024-09-21T15:01:55.097 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.097 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.097 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:55.097 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939069+0000", 2024-09-21T15:01:55.097 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2119999999999997e-06 2024-09-21T15:01:55.097 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.097 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.097 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:55.097 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.961141+0000", 2024-09-21T15:01:55.097 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022072268999999999 2024-09-21T15:01:55.097 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.097 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.097 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:55.097 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.961240+0000", 2024-09-21T15:01:55.098 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9147999999999993e-05 2024-09-21T15:01:55.098 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.098 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.098 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:55.098 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.961444+0000", 2024-09-21T15:01:55.098 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00020381899999999999 2024-09-21T15:01:55.098 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:55.098 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:55.098 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:55.098 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.098 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.098 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:134713 3.a 3:51eaadc0:::rbd_data.11365ed419e3.0000000000000007:67 [read 303104~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:55.098 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.939073+0000", 2024-09-21T15:01:55.098 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 61.088316509999999, 2024-09-21T15:01:55.098 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022687394999999999, 2024-09-21T15:01:55.099 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:55.099 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:55.099 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:55.099 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:55.099 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:55.099 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 134713 2024-09-21T15:01:55.099 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.099 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:55.099 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.099 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:55.099 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939073+0000", 2024-09-21T15:01:55.099 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:55.099 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.099 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.099 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:55.099 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939073+0000", 2024-09-21T15:01:55.100 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:55.100 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.100 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.100 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:55.100 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939073+0000", 2024-09-21T15:01:55.100 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:55.100 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.100 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.100 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:55.100 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939075+0000", 2024-09-21T15:01:55.100 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5969999999999999e-06 2024-09-21T15:01:55.100 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.100 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.100 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:55.100 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939081+0000", 2024-09-21T15:01:55.101 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.609e-06 2024-09-21T15:01:55.101 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.101 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.101 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:55.101 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939086+0000", 2024-09-21T15:01:55.101 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.916e-06 2024-09-21T15:01:55.101 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.101 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.101 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:55.101 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.961474+0000", 2024-09-21T15:01:55.101 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022388512999999999 2024-09-21T15:01:55.101 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.101 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.101 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:55.101 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.961567+0000", 2024-09-21T15:01:55.102 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2267000000000003e-05 2024-09-21T15:01:55.102 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.102 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.102 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:55.102 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.961760+0000", 2024-09-21T15:01:55.102 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00019375999999999999 2024-09-21T15:01:55.102 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:55.102 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:55.102 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:55.102 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.102 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.102 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:134714 3.a 3:51eaadc0:::rbd_data.11365ed419e3.0000000000000007:67 [read 319488~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:55.102 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.939090+0000", 2024-09-21T15:01:55.102 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 61.088299683000002, 2024-09-21T15:01:55.102 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023043403, 2024-09-21T15:01:55.103 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:55.103 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:55.103 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:55.103 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:55.103 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:55.103 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 134714 2024-09-21T15:01:55.103 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.103 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:55.103 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.103 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:55.103 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939090+0000", 2024-09-21T15:01:55.103 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:55.103 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.103 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.103 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:55.104 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939090+0000", 2024-09-21T15:01:55.104 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:55.104 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.104 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.104 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:55.104 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939090+0000", 2024-09-21T15:01:55.104 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:55.104 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.104 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.104 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:55.104 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939092+0000", 2024-09-21T15:01:55.104 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2790000000000001e-06 2024-09-21T15:01:55.104 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.104 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.104 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:55.104 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939097+0000", 2024-09-21T15:01:55.105 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1410000000000004e-06 2024-09-21T15:01:55.105 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.105 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.105 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:55.105 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939102+0000", 2024-09-21T15:01:55.105 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7369999999999997e-06 2024-09-21T15:01:55.105 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.105 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.105 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:55.105 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.961791+0000", 2024-09-21T15:01:55.105 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022688974000000001 2024-09-21T15:01:55.105 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.105 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.105 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:55.105 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.961865+0000", 2024-09-21T15:01:55.105 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4651000000000001e-05 2024-09-21T15:01:55.106 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.106 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.106 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:55.106 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.962133+0000", 2024-09-21T15:01:55.106 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00026793999999999999 2024-09-21T15:01:55.106 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:55.106 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:55.106 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:55.106 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.106 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.106 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:134715 3.a 3:51eaadc0:::rbd_data.11365ed419e3.0000000000000007:67 [read 335872~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:55.106 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.939106+0000", 2024-09-21T15:01:55.106 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 61.088283072999999, 2024-09-21T15:01:55.106 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023401142999999999, 2024-09-21T15:01:55.106 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:55.107 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:55.107 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:55.107 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:55.107 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:55.107 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 134715 2024-09-21T15:01:55.107 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.107 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:55.107 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.107 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:55.107 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939106+0000", 2024-09-21T15:01:55.107 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:55.107 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.107 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.107 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:55.107 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939106+0000", 2024-09-21T15:01:55.108 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:55.108 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.108 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.108 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:55.108 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939106+0000", 2024-09-21T15:01:55.108 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:55.108 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.108 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.108 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:55.108 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939114+0000", 2024-09-21T15:01:55.108 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1009999999999992e-06 2024-09-21T15:01:55.108 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.108 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.108 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:55.109 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939121+0000", 2024-09-21T15:01:55.109 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7710000000000001e-06 2024-09-21T15:01:55.109 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.109 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.109 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:55.109 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939126+0000", 2024-09-21T15:01:55.109 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3600000000000004e-06 2024-09-21T15:01:55.109 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.109 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.109 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:55.109 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.962162+0000", 2024-09-21T15:01:55.109 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023036148999999999 2024-09-21T15:01:55.109 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.109 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.109 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:55.109 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.962239+0000", 2024-09-21T15:01:55.110 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6874999999999998e-05 2024-09-21T15:01:55.110 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.110 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.110 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:55.110 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.962508+0000", 2024-09-21T15:01:55.110 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00026842199999999999 2024-09-21T15:01:55.110 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:55.110 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:55.110 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:55.110 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.110 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.110 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:134716 3.a 3:51eaadc0:::rbd_data.11365ed419e3.0000000000000007:67 [read 352256~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:55.110 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.939131+0000", 2024-09-21T15:01:55.110 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 61.088258938999999, 2024-09-21T15:01:55.110 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023742441, 2024-09-21T15:01:55.111 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:55.111 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:55.111 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:55.111 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:55.111 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:55.111 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 134716 2024-09-21T15:01:55.111 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.111 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:55.111 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.111 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:55.111 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939131+0000", 2024-09-21T15:01:55.111 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:55.111 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.111 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.111 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:55.112 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939131+0000", 2024-09-21T15:01:55.112 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:55.112 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.112 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.112 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:55.113 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939130+0000", 2024-09-21T15:01:55.113 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:55.113 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.113 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.113 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:55.113 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939133+0000", 2024-09-21T15:01:55.113 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5620000000000002e-06 2024-09-21T15:01:55.113 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.114 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.114 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:55.114 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939138+0000", 2024-09-21T15:01:55.114 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.322e-06 2024-09-21T15:01:55.114 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.114 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.114 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:55.114 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939143+0000", 2024-09-21T15:01:55.114 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5510000000000001e-06 2024-09-21T15:01:55.114 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.114 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.114 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:55.114 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.962537+0000", 2024-09-21T15:01:55.114 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023394808 2024-09-21T15:01:55.114 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.115 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.115 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:55.115 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.962608+0000", 2024-09-21T15:01:55.115 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0699999999999997e-05 2024-09-21T15:01:55.115 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.115 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.115 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:55.115 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.962873+0000", 2024-09-21T15:01:55.115 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00026487 2024-09-21T15:01:55.115 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:55.115 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:55.115 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:55.115 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.115 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.115 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:135024 3.a 3:53261383:::rbd_data.11365ed419e3.0000000000000037:snapdir [list-snaps] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:55.115 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.940162+0000", 2024-09-21T15:01:55.116 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 61.087227976000001, 2024-09-21T15:01:55.116 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022852172, 2024-09-21T15:01:55.116 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:55.116 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:55.116 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:55.116 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:55.116 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:55.116 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 135024 2024-09-21T15:01:55.116 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.116 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:55.116 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.116 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:55.116 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940162+0000", 2024-09-21T15:01:55.116 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:55.116 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.116 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.117 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:55.117 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940162+0000", 2024-09-21T15:01:55.117 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:55.117 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.117 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.117 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:55.117 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940161+0000", 2024-09-21T15:01:55.117 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:55.117 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.117 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.117 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:55.117 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940164+0000", 2024-09-21T15:01:55.117 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4099999999999998e-06 2024-09-21T15:01:55.117 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.118 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.118 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:55.118 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940169+0000", 2024-09-21T15:01:55.118 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3720000000000004e-06 2024-09-21T15:01:55.118 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.118 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.118 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:55.118 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940174+0000", 2024-09-21T15:01:55.118 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.724e-06 2024-09-21T15:01:55.118 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.118 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.118 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:55.118 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.962902+0000", 2024-09-21T15:01:55.118 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022728719000000001 2024-09-21T15:01:55.118 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.118 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.119 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:55.119 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.963014+0000", 2024-09-21T15:01:55.119 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000111245 2024-09-21T15:01:55.119 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:55.119 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:55.119 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:55.119 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.119 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.119 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:135044 3.a 3:53261383:::rbd_data.11365ed419e3.0000000000000037:snapdir [list-snaps] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:55.119 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.940484+0000", 2024-09-21T15:01:55.119 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 61.086905244, 2024-09-21T15:01:55.119 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022619169000000001, 2024-09-21T15:01:55.119 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:55.119 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:55.119 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:55.119 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:55.120 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:55.120 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 135044 2024-09-21T15:01:55.120 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.120 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:55.120 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.120 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:55.120 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940484+0000", 2024-09-21T15:01:55.120 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:55.120 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.120 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.120 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:55.120 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940484+0000", 2024-09-21T15:01:55.120 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:55.120 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.120 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.120 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:55.121 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940484+0000", 2024-09-21T15:01:55.121 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:55.121 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.121 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.121 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:55.121 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940487+0000", 2024-09-21T15:01:55.121 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8449999999999999e-06 2024-09-21T15:01:55.121 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.121 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.121 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:55.121 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940493+0000", 2024-09-21T15:01:55.121 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9599999999999997e-06 2024-09-21T15:01:55.121 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.121 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.121 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:55.121 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940497+0000", 2024-09-21T15:01:55.121 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9620000000000001e-06 2024-09-21T15:01:55.122 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.122 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.122 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:55.122 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.963034+0000", 2024-09-21T15:01:55.122 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022536878 2024-09-21T15:01:55.122 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.122 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.122 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:55.122 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.963103+0000", 2024-09-21T15:01:55.122 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9073999999999996e-05 2024-09-21T15:01:55.122 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:55.122 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:55.122 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:55.122 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:55.122 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:55.123 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:55.124 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T15:01:55.124 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:01:55.121+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:01:55.195 INFO:teuthology.orchestra.run.smithi032.stderr:set osd.5 primary-affinity to 0.49313 (8323172) 2024-09-21T15:01:55.212 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T15:01:55.260 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:55.261 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T15:01:55.261 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.261 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:170642 3.5 3:a51c63e5:::rbd_id.image_client.0-clone6:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e223)", 2024-09-21T15:01:55.261 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:55.259838+0000", 2024-09-21T15:01:55.261 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 0.0016343320000000001, 2024-09-21T15:01:55.261 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0016612969999999999, 2024-09-21T15:01:55.261 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:55.261 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "waiting for sub ops", 2024-09-21T15:01:55.261 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:55.261 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:55.261 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:55.261 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 170642 2024-09-21T15:01:55.261 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.262 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:55.262 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.262 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:55.262 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:55.259838+0000", 2024-09-21T15:01:55.262 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:55.262 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.262 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.262 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:55.262 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:55.259838+0000", 2024-09-21T15:01:55.262 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:55.262 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.262 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.262 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:55.263 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:55.259836+0000", 2024-09-21T15:01:55.263 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999981 2024-09-21T15:01:55.263 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.263 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.263 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:55.263 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:55.259845+0000", 2024-09-21T15:01:55.263 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4860000000000003e-06 2024-09-21T15:01:55.263 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.263 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.263 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:55.263 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:55.259865+0000", 2024-09-21T15:01:55.263 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9924000000000001e-05 2024-09-21T15:01:55.263 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.263 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.264 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:55.264 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:55.259874+0000", 2024-09-21T15:01:55.264 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.3989999999999993e-06 2024-09-21T15:01:55.264 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.264 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.264 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:55.264 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:55.259947+0000", 2024-09-21T15:01:55.264 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2464000000000003e-05 2024-09-21T15:01:55.264 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.264 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.264 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:55.264 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:55.260057+0000", 2024-09-21T15:01:55.264 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000109894 2024-09-21T15:01:55.264 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.265 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.265 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 6", 2024-09-21T15:01:55.265 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:55.260188+0000", 2024-09-21T15:01:55.265 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013095399999999999 2024-09-21T15:01:55.265 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.265 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.265 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:01:55.265 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:55.261312+0000", 2024-09-21T15:01:55.265 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0011248269999999999 2024-09-21T15:01:55.265 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:55.265 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:55.265 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:55.265 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:55.265 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-21T15:01:55.265 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 1 2024-09-21T15:01:55.266 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:55.266 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:01:55.257+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:01:55.273 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T15:01:55.325 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:01:55.320+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:01:55.412 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:55.412 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:01:55.412 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T15:01:55.412 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T15:01:55.412 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:55.423 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T15:01:55.426 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:01:55.425+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:01:55.447 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T15:01:55.526 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:01:55.525+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:01:55.527 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:55.528 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T15:01:55.528 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T15:01:55.528 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T15:01:55.528 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.528 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:15130 3.2 3:47f1bbdb:::rbd_data.11361bed1f78.0000000000000067:head [sparse-read 0~147456] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-21T15:01:55.528 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:40.288284+0000", 2024-09-21T15:01:55.528 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 195.23896722200001, 2024-09-21T15:01:55.528 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1142935039999999, 2024-09-21T15:01:55.528 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:55.528 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:55.528 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:55.528 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:55.528 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:55.529 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 15130 2024-09-21T15:01:55.529 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.529 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:55.529 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.529 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:55.529 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288284+0000", 2024-09-21T15:01:55.529 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:55.529 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.529 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.529 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:55.529 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288284+0000", 2024-09-21T15:01:55.529 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:55.529 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.529 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.529 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:55.529 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288284+0000", 2024-09-21T15:01:55.530 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:55.530 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.530 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.530 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:55.530 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288288+0000", 2024-09-21T15:01:55.530 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5829999999999998e-06 2024-09-21T15:01:55.530 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.530 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.530 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:55.530 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288297+0000", 2024-09-21T15:01:55.530 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0669999999999998e-06 2024-09-21T15:01:55.530 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.530 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.530 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:55.530 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288305+0000", 2024-09-21T15:01:55.531 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8860000000000004e-06 2024-09-21T15:01:55.531 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.531 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.531 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:55.531 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288417+0000", 2024-09-21T15:01:55.531 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000112952 2024-09-21T15:01:55.531 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.531 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.531 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:55.531 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288428+0000", 2024-09-21T15:01:55.531 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0631e-05 2024-09-21T15:01:55.531 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.531 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.531 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:55.531 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.401876+0000", 2024-09-21T15:01:55.532 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1134479610000001 2024-09-21T15:01:55.532 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.532 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.532 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:55.532 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.402076+0000", 2024-09-21T15:01:55.532 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00019944200000000001 2024-09-21T15:01:55.532 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.532 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.532 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:55.532 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.402578+0000", 2024-09-21T15:01:55.532 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00050242899999999998 2024-09-21T15:01:55.532 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:55.532 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:55.532 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:55.533 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.533 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.533 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45708 3.2 3:415db91a:::rbd_data.1136ecbe5879.0000000000000000:28 [read 140288~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:55.533 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141801+0000", 2024-09-21T15:01:55.533 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 165.38545102800001, 2024-09-21T15:01:55.533 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034274200999999997, 2024-09-21T15:01:55.533 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:55.533 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:55.533 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:55.533 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:55.533 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:55.533 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45708 2024-09-21T15:01:55.533 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.533 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:55.533 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.534 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:55.534 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141801+0000", 2024-09-21T15:01:55.534 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:55.534 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.534 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.534 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:55.534 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141801+0000", 2024-09-21T15:01:55.534 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:55.534 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.534 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.534 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:55.534 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141800+0000", 2024-09-21T15:01:55.534 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:55.534 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.535 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.535 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:55.535 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141803+0000", 2024-09-21T15:01:55.535 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4210000000000002e-06 2024-09-21T15:01:55.535 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.535 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.535 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:55.535 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141812+0000", 2024-09-21T15:01:55.535 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.9800000000000004e-06 2024-09-21T15:01:55.535 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.535 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.535 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:55.535 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141816+0000", 2024-09-21T15:01:55.535 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4769999999999997e-06 2024-09-21T15:01:55.535 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.536 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.536 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:55.536 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.175865+0000", 2024-09-21T15:01:55.536 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034048875999999999 2024-09-21T15:01:55.536 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.536 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.536 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:55.536 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.175943+0000", 2024-09-21T15:01:55.536 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7464000000000002e-05 2024-09-21T15:01:55.536 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.536 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.536 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:55.536 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176075+0000", 2024-09-21T15:01:55.536 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013230800000000001 2024-09-21T15:01:55.536 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:55.536 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:55.537 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:55.537 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.537 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.537 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45709 3.2 3:415db91a:::rbd_data.1136ecbe5879.0000000000000000:28 [read 143360~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:55.537 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141820+0000", 2024-09-21T15:01:55.537 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 165.385431181, 2024-09-21T15:01:55.537 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034537536000000001, 2024-09-21T15:01:55.537 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:55.537 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:55.537 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:55.537 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:55.537 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:55.537 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45709 2024-09-21T15:01:55.537 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.537 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:55.538 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.538 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:55.538 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141820+0000", 2024-09-21T15:01:55.538 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:55.538 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.538 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.538 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:55.538 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141820+0000", 2024-09-21T15:01:55.538 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:55.538 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.538 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.538 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:55.538 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141820+0000", 2024-09-21T15:01:55.538 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:55.538 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.539 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.539 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:55.539 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141823+0000", 2024-09-21T15:01:55.539 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5110000000000002e-06 2024-09-21T15:01:55.539 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.539 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.539 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:55.539 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141829+0000", 2024-09-21T15:01:55.539 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0850000000000002e-06 2024-09-21T15:01:55.539 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.539 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.539 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:55.539 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141833+0000", 2024-09-21T15:01:55.539 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7620000000000003e-06 2024-09-21T15:01:55.539 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.539 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.540 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:55.540 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176103+0000", 2024-09-21T15:01:55.540 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034269933000000002 2024-09-21T15:01:55.540 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.540 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.540 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:55.540 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176161+0000", 2024-09-21T15:01:55.540 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7658999999999998e-05 2024-09-21T15:01:55.540 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.540 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.540 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:55.540 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176358+0000", 2024-09-21T15:01:55.540 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000196937 2024-09-21T15:01:55.541 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:55.541 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:55.541 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:55.541 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.541 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.541 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45710 3.2 3:415db91a:::rbd_data.1136ecbe5879.0000000000000000:28 [read 146432~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:55.541 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141838+0000", 2024-09-21T15:01:55.541 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 165.38541401800001, 2024-09-21T15:01:55.541 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034953674999999997, 2024-09-21T15:01:55.541 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:55.541 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:55.541 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:55.541 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:55.541 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:55.541 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45710 2024-09-21T15:01:55.541 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.542 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:55.542 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.542 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:55.542 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141838+0000", 2024-09-21T15:01:55.542 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:55.542 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.542 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.542 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:55.542 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141838+0000", 2024-09-21T15:01:55.542 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:55.542 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.542 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.542 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:55.542 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141837+0000", 2024-09-21T15:01:55.542 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:55.543 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.543 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.543 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:55.543 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141843+0000", 2024-09-21T15:01:55.543 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1619999999999996e-06 2024-09-21T15:01:55.543 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.543 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.543 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:55.543 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141850+0000", 2024-09-21T15:01:55.543 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4799999999999998e-06 2024-09-21T15:01:55.543 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.543 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.543 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:55.543 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141853+0000", 2024-09-21T15:01:55.543 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3050000000000001e-06 2024-09-21T15:01:55.544 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.544 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.544 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:55.544 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176490+0000", 2024-09-21T15:01:55.544 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034636413999999997 2024-09-21T15:01:55.544 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.544 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.544 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:55.544 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176606+0000", 2024-09-21T15:01:55.544 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000115841 2024-09-21T15:01:55.544 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.544 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.544 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:55.544 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176791+0000", 2024-09-21T15:01:55.544 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00018581500000000001 2024-09-21T15:01:55.545 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:55.545 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:55.545 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:55.545 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.545 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.545 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45711 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:55.545 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141856+0000", 2024-09-21T15:01:55.545 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 165.38539568100001, 2024-09-21T15:01:55.545 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034986308000000001, 2024-09-21T15:01:55.545 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:55.545 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:55.545 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:55.545 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:55.545 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:55.545 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45711 2024-09-21T15:01:55.545 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.546 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:55.546 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.546 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:55.546 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141856+0000", 2024-09-21T15:01:55.546 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:55.546 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.546 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.546 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:55.546 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141856+0000", 2024-09-21T15:01:55.546 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:55.546 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.546 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.546 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:55.546 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141856+0000", 2024-09-21T15:01:55.547 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:55.547 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.547 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.547 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:55.547 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141862+0000", 2024-09-21T15:01:55.547 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.567e-06 2024-09-21T15:01:55.547 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.547 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.547 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:55.547 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141868+0000", 2024-09-21T15:01:55.547 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9569999999999999e-06 2024-09-21T15:01:55.547 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.547 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.547 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:55.547 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141871+0000", 2024-09-21T15:01:55.547 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.146e-06 2024-09-21T15:01:55.548 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.548 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.548 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:55.548 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176831+0000", 2024-09-21T15:01:55.548 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034959935999999997 2024-09-21T15:01:55.548 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.548 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.548 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:55.548 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176842+0000", 2024-09-21T15:01:55.548 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1008e-05 2024-09-21T15:01:55.548 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:55.548 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:55.548 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:55.548 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.548 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.549 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45712 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:55.549 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141874+0000", 2024-09-21T15:01:55.549 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 165.38537778599999, 2024-09-21T15:01:55.549 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034993511999999997, 2024-09-21T15:01:55.549 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:55.549 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:55.549 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:55.549 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:55.549 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:55.549 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45712 2024-09-21T15:01:55.549 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.549 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:55.549 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.549 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:55.549 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141874+0000", 2024-09-21T15:01:55.550 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:55.550 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.550 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.550 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:55.550 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141874+0000", 2024-09-21T15:01:55.550 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:55.550 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.550 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.550 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:55.550 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141874+0000", 2024-09-21T15:01:55.550 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:55.550 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.550 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.550 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:55.550 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141875+0000", 2024-09-21T15:01:55.551 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5430000000000001e-06 2024-09-21T15:01:55.551 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.551 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.551 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:55.551 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141878+0000", 2024-09-21T15:01:55.551 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1420000000000002e-06 2024-09-21T15:01:55.551 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.551 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.551 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:55.551 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141881+0000", 2024-09-21T15:01:55.551 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.807e-06 2024-09-21T15:01:55.551 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.551 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.551 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:55.551 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176860+0000", 2024-09-21T15:01:55.551 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034979243 2024-09-21T15:01:55.552 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.552 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.552 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:55.552 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176867+0000", 2024-09-21T15:01:55.552 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0169999999999999e-06 2024-09-21T15:01:55.552 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:55.552 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:55.552 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:55.552 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.552 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.552 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45713 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:55.552 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141883+0000", 2024-09-21T15:01:55.552 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 165.38536821100001, 2024-09-21T15:01:55.552 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035010516999999998, 2024-09-21T15:01:55.552 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:55.552 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:55.553 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:55.553 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:55.553 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:55.553 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45713 2024-09-21T15:01:55.553 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.553 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:55.553 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.553 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:55.553 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141883+0000", 2024-09-21T15:01:55.553 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:55.553 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.553 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.553 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:55.553 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141883+0000", 2024-09-21T15:01:55.553 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:55.554 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.554 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.554 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:55.554 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141883+0000", 2024-09-21T15:01:55.554 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:55.554 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.554 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.554 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:55.554 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141885+0000", 2024-09-21T15:01:55.554 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4649999999999999e-06 2024-09-21T15:01:55.554 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.554 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.554 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:55.554 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141888+0000", 2024-09-21T15:01:55.555 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3890000000000001e-06 2024-09-21T15:01:55.555 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.555 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.555 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:55.555 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141891+0000", 2024-09-21T15:01:55.555 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9239999999999999e-06 2024-09-21T15:01:55.555 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.555 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.555 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:55.555 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176887+0000", 2024-09-21T15:01:55.555 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034996370999999998 2024-09-21T15:01:55.555 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.555 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.555 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:55.555 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176894+0000", 2024-09-21T15:01:55.556 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6069999999999996e-06 2024-09-21T15:01:55.556 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:55.556 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:55.556 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:55.556 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.556 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.556 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45714 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:55.556 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141893+0000", 2024-09-21T15:01:55.556 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 165.38535829599999, 2024-09-21T15:01:55.556 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035048215000000001, 2024-09-21T15:01:55.556 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:55.556 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:55.556 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:55.557 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:55.557 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:55.557 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45714 2024-09-21T15:01:55.557 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.557 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:55.557 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.557 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:55.557 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141893+0000", 2024-09-21T15:01:55.557 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:55.557 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.557 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.557 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:55.557 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141893+0000", 2024-09-21T15:01:55.557 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:55.557 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.557 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.558 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:55.558 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141893+0000", 2024-09-21T15:01:55.558 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:55.558 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.558 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.558 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:55.558 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141895+0000", 2024-09-21T15:01:55.558 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.703e-06 2024-09-21T15:01:55.558 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.558 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.558 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:55.558 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141900+0000", 2024-09-21T15:01:55.558 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.242e-06 2024-09-21T15:01:55.558 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.558 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.559 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:55.559 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141903+0000", 2024-09-21T15:01:55.559 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.117e-06 2024-09-21T15:01:55.559 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.559 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.559 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:55.559 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176930+0000", 2024-09-21T15:01:55.559 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035026750000000002 2024-09-21T15:01:55.559 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.559 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.559 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:55.559 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176942+0000", 2024-09-21T15:01:55.559 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1668000000000001e-05 2024-09-21T15:01:55.559 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:55.559 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:55.559 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:55.560 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.560 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.560 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45715 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:55.560 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141906+0000", 2024-09-21T15:01:55.560 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 165.38534613799999, 2024-09-21T15:01:55.560 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035066057999999997, 2024-09-21T15:01:55.560 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:55.560 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:55.560 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:55.560 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:55.560 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:55.560 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45715 2024-09-21T15:01:55.560 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.560 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:55.560 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.560 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:55.561 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141906+0000", 2024-09-21T15:01:55.561 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:55.561 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.561 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.561 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:55.561 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141906+0000", 2024-09-21T15:01:55.561 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:55.561 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.561 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.561 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:55.561 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141905+0000", 2024-09-21T15:01:55.561 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:55.561 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.561 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.561 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:55.561 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141907+0000", 2024-09-21T15:01:55.562 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5400000000000001e-06 2024-09-21T15:01:55.562 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.562 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.562 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:55.562 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141912+0000", 2024-09-21T15:01:55.562 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.502e-06 2024-09-21T15:01:55.562 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.562 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.562 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:55.562 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141915+0000", 2024-09-21T15:01:55.562 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0170000000000001e-06 2024-09-21T15:01:55.562 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.562 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.562 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:55.562 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176964+0000", 2024-09-21T15:01:55.563 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035048587999999999 2024-09-21T15:01:55.563 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.563 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.563 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:55.563 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176972+0000", 2024-09-21T15:01:55.563 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6580000000000002e-06 2024-09-21T15:01:55.563 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:55.563 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:55.563 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:55.563 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.563 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.563 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45716 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:55.563 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141918+0000", 2024-09-21T15:01:55.563 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 165.385333962, 2024-09-21T15:01:55.563 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035086223, 2024-09-21T15:01:55.563 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:55.564 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:55.564 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:55.564 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:55.564 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:55.564 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45716 2024-09-21T15:01:55.564 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.564 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:55.564 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.564 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:55.564 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141918+0000", 2024-09-21T15:01:55.564 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:55.564 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.564 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.564 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:55.564 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141918+0000", 2024-09-21T15:01:55.565 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:55.565 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.565 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.565 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:55.565 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141917+0000", 2024-09-21T15:01:55.565 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:55.565 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.565 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.565 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:55.565 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141919+0000", 2024-09-21T15:01:55.565 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5680000000000001e-06 2024-09-21T15:01:55.565 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.565 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.565 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:55.565 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141922+0000", 2024-09-21T15:01:55.565 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2789999999999999e-06 2024-09-21T15:01:55.566 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.566 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.566 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:55.566 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141925+0000", 2024-09-21T15:01:55.566 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9469999999999999e-06 2024-09-21T15:01:55.566 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.566 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.566 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:55.566 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176997+0000", 2024-09-21T15:01:55.566 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035071366999999999 2024-09-21T15:01:55.566 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.566 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.566 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:55.566 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177004+0000", 2024-09-21T15:01:55.566 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3309999999999998e-06 2024-09-21T15:01:55.567 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:55.567 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:55.567 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:55.567 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.567 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.567 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45717 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:55.567 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141928+0000", 2024-09-21T15:01:55.567 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 165.385324016, 2024-09-21T15:01:55.567 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035102573999999998, 2024-09-21T15:01:55.567 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:55.567 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:55.567 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:55.567 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:55.567 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:55.567 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45717 2024-09-21T15:01:55.567 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.568 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:55.568 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.568 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:55.568 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141928+0000", 2024-09-21T15:01:55.568 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:55.568 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.568 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.568 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:55.568 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141928+0000", 2024-09-21T15:01:55.568 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:55.568 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.568 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.568 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:55.568 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141927+0000", 2024-09-21T15:01:55.569 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:01:55.569 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.569 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.569 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:55.569 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141929+0000", 2024-09-21T15:01:55.569 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.533e-06 2024-09-21T15:01:55.569 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.569 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.569 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:55.569 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141934+0000", 2024-09-21T15:01:55.569 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1660000000000002e-06 2024-09-21T15:01:55.569 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.569 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.569 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:55.569 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141937+0000", 2024-09-21T15:01:55.570 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0309999999999999e-06 2024-09-21T15:01:55.570 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.570 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.570 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:55.570 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177023+0000", 2024-09-21T15:01:55.570 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035085595999999997 2024-09-21T15:01:55.570 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.570 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.570 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:55.570 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177030+0000", 2024-09-21T15:01:55.570 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.486e-06 2024-09-21T15:01:55.570 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:55.570 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:55.570 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:55.570 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.571 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.571 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45718 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:55.571 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141940+0000", 2024-09-21T15:01:55.571 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 165.385312158, 2024-09-21T15:01:55.571 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035124956999999998, 2024-09-21T15:01:55.571 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:55.571 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:55.571 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:55.571 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:55.571 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:55.571 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45718 2024-09-21T15:01:55.571 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.571 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:55.571 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.571 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:55.572 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141940+0000", 2024-09-21T15:01:55.572 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:55.572 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.572 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.572 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:55.572 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141940+0000", 2024-09-21T15:01:55.572 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:55.572 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.572 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.572 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:55.572 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141939+0000", 2024-09-21T15:01:55.572 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:55.572 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.572 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.572 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:55.573 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141941+0000", 2024-09-21T15:01:55.573 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.502e-06 2024-09-21T15:01:55.573 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.573 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.573 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:55.573 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141946+0000", 2024-09-21T15:01:55.573 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3449999999999996e-06 2024-09-21T15:01:55.573 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.573 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.573 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:55.573 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141949+0000", 2024-09-21T15:01:55.573 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9749999999999999e-06 2024-09-21T15:01:55.574 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.574 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.574 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:55.574 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177057+0000", 2024-09-21T15:01:55.574 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035108385999999998 2024-09-21T15:01:55.574 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.574 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.574 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:55.574 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177064+0000", 2024-09-21T15:01:55.574 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0099999999999998e-06 2024-09-21T15:01:55.574 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:55.574 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:55.574 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:55.574 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.574 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.575 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45719 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:55.575 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141952+0000", 2024-09-21T15:01:55.575 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 165.38529988499999, 2024-09-21T15:01:55.575 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035139108000000002, 2024-09-21T15:01:55.575 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:55.575 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:55.575 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:55.575 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:55.575 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:55.575 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45719 2024-09-21T15:01:55.575 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.575 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:55.575 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.575 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:55.575 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141952+0000", 2024-09-21T15:01:55.576 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:55.576 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.576 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.576 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:55.576 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141952+0000", 2024-09-21T15:01:55.576 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:55.576 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.576 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.576 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:55.576 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141952+0000", 2024-09-21T15:01:55.576 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:55.576 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.576 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.576 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:55.576 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141953+0000", 2024-09-21T15:01:55.576 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.486e-06 2024-09-21T15:01:55.577 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.577 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.577 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:55.577 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141959+0000", 2024-09-21T15:01:55.577 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4940000000000003e-06 2024-09-21T15:01:55.577 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.577 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.577 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:55.577 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141963+0000", 2024-09-21T15:01:55.577 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3540000000000002e-06 2024-09-21T15:01:55.577 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.577 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.577 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:55.577 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177084+0000", 2024-09-21T15:01:55.577 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035120983000000001 2024-09-21T15:01:55.577 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.578 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.578 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:55.578 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177091+0000", 2024-09-21T15:01:55.578 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0350000000000004e-06 2024-09-21T15:01:55.578 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:55.578 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:55.578 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:55.578 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.578 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.578 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45720 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:55.578 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141965+0000", 2024-09-21T15:01:55.578 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 165.38528641799999, 2024-09-21T15:01:55.578 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035155268000000003, 2024-09-21T15:01:55.578 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:55.578 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:55.578 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:55.579 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:55.579 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:55.579 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45720 2024-09-21T15:01:55.579 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.579 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:55.579 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.579 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:55.579 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141965+0000", 2024-09-21T15:01:55.579 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:55.579 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.579 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.579 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:55.579 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141965+0000", 2024-09-21T15:01:55.579 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:55.579 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.579 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.580 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:55.580 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141965+0000", 2024-09-21T15:01:55.580 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:55.580 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.580 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.580 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:55.580 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141966+0000", 2024-09-21T15:01:55.580 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.46e-06 2024-09-21T15:01:55.580 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.580 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.580 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:55.580 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141972+0000", 2024-09-21T15:01:55.580 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4210000000000003e-06 2024-09-21T15:01:55.580 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.581 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.581 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:55.581 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141975+0000", 2024-09-21T15:01:55.581 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.948e-06 2024-09-21T15:01:55.581 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.581 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.581 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:55.581 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177113+0000", 2024-09-21T15:01:55.581 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035137855000000003 2024-09-21T15:01:55.581 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.581 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.581 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:55.581 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177121+0000", 2024-09-21T15:01:55.581 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8269999999999998e-06 2024-09-21T15:01:55.581 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:55.581 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:55.582 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:55.582 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.582 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.582 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45721 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:55.582 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141977+0000", 2024-09-21T15:01:55.582 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 165.38527439500001, 2024-09-21T15:01:55.582 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035169984000000001, 2024-09-21T15:01:55.582 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:55.582 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:55.582 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:55.582 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:55.582 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:55.582 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45721 2024-09-21T15:01:55.582 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.582 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:55.583 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.583 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:55.583 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141977+0000", 2024-09-21T15:01:55.583 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:55.583 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.583 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.583 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:55.583 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141977+0000", 2024-09-21T15:01:55.583 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:55.583 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.583 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.583 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:55.583 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141977+0000", 2024-09-21T15:01:55.583 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:55.584 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.584 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.584 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:55.584 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141979+0000", 2024-09-21T15:01:55.584 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.511e-06 2024-09-21T15:01:55.584 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.584 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.584 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:55.584 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141982+0000", 2024-09-21T15:01:55.584 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4180000000000001e-06 2024-09-21T15:01:55.584 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.584 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.584 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:55.584 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141985+0000", 2024-09-21T15:01:55.584 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9450000000000002e-06 2024-09-21T15:01:55.585 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.585 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.585 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:55.585 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177139+0000", 2024-09-21T15:01:55.585 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035154409999999997 2024-09-21T15:01:55.585 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.585 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.585 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:55.585 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177147+0000", 2024-09-21T15:01:55.585 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9570000000000002e-06 2024-09-21T15:01:55.585 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:55.585 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:55.585 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:55.585 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.585 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.586 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45722 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:55.586 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141987+0000", 2024-09-21T15:01:55.586 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 165.38526442400001, 2024-09-21T15:01:55.586 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035188492000000002, 2024-09-21T15:01:55.586 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:55.586 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:55.586 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:55.586 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:55.586 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:55.586 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45722 2024-09-21T15:01:55.586 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.586 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:55.586 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.586 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:55.587 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141987+0000", 2024-09-21T15:01:55.587 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:55.587 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.587 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.587 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:55.587 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141987+0000", 2024-09-21T15:01:55.587 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:55.587 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.587 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.587 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:55.587 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141987+0000", 2024-09-21T15:01:55.587 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:55.587 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.587 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.587 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:55.587 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141988+0000", 2024-09-21T15:01:55.588 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.463e-06 2024-09-21T15:01:55.588 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.588 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.588 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:55.588 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141994+0000", 2024-09-21T15:01:55.588 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.361e-06 2024-09-21T15:01:55.588 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.588 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.588 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:55.588 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141997+0000", 2024-09-21T15:01:55.588 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0359999999999998e-06 2024-09-21T15:01:55.588 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.588 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.588 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:55.588 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177169+0000", 2024-09-21T15:01:55.589 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035171651999999998 2024-09-21T15:01:55.589 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.589 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.589 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:55.589 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177176+0000", 2024-09-21T15:01:55.589 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2189999999999998e-06 2024-09-21T15:01:55.589 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:55.589 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:55.589 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:55.589 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.589 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.589 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45723 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:55.589 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141999+0000", 2024-09-21T15:01:55.589 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 165.38525225699999, 2024-09-21T15:01:55.589 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035197701999999997, 2024-09-21T15:01:55.589 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:55.590 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:55.590 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:55.590 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:55.590 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:55.590 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45723 2024-09-21T15:01:55.590 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.590 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:55.590 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.590 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:55.590 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141999+0000", 2024-09-21T15:01:55.590 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:55.591 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.591 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.591 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:55.591 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141999+0000", 2024-09-21T15:01:55.591 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:55.591 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.591 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.591 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:55.591 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141999+0000", 2024-09-21T15:01:55.591 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:55.591 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.591 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.592 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:55.592 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142003+0000", 2024-09-21T15:01:55.592 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5049999999999999e-06 2024-09-21T15:01:55.592 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.592 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.592 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:55.592 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142008+0000", 2024-09-21T15:01:55.592 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6579999999999999e-06 2024-09-21T15:01:55.592 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.592 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.592 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:55.592 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142011+0000", 2024-09-21T15:01:55.593 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0529999999999998e-06 2024-09-21T15:01:55.593 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.593 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.593 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:55.593 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177194+0000", 2024-09-21T15:01:55.593 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035182221999999999 2024-09-21T15:01:55.593 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.593 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.593 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:55.593 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177197+0000", 2024-09-21T15:01:55.593 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5030000000000002e-06 2024-09-21T15:01:55.593 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:55.593 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:55.593 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:55.594 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.594 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.594 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45724 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:55.594 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.142015+0000", 2024-09-21T15:01:55.594 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 165.38523628999999, 2024-09-21T15:01:55.594 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035195345000000003, 2024-09-21T15:01:55.594 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:55.594 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:55.594 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:55.594 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:55.594 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:55.594 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45724 2024-09-21T15:01:55.594 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.595 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:55.595 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.595 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:55.595 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142015+0000", 2024-09-21T15:01:55.595 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:55.595 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.595 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.595 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:55.595 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142015+0000", 2024-09-21T15:01:55.595 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:55.595 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.595 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.595 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:55.596 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142015+0000", 2024-09-21T15:01:55.596 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:55.596 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.596 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.596 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:55.596 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142017+0000", 2024-09-21T15:01:55.596 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5540000000000001e-06 2024-09-21T15:01:55.596 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.596 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.596 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:55.596 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142022+0000", 2024-09-21T15:01:55.596 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4229999999999996e-06 2024-09-21T15:01:55.596 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.596 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.597 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:55.597 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142025+0000", 2024-09-21T15:01:55.597 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.056e-06 2024-09-21T15:01:55.597 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.597 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.597 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:55.597 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177207+0000", 2024-09-21T15:01:55.597 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035182089 2024-09-21T15:01:55.597 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.597 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.597 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:55.597 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177211+0000", 2024-09-21T15:01:55.598 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4649999999999999e-06 2024-09-21T15:01:55.598 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:55.598 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:55.598 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:55.598 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.598 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.598 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45725 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:55.598 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.142028+0000", 2024-09-21T15:01:55.598 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 165.38522401099999, 2024-09-21T15:01:55.598 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035195731000000001, 2024-09-21T15:01:55.598 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:55.598 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:55.598 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:55.599 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:55.599 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:55.599 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45725 2024-09-21T15:01:55.599 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.599 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:55.599 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.599 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:55.599 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142028+0000", 2024-09-21T15:01:55.599 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:55.599 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.599 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.599 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:55.599 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142028+0000", 2024-09-21T15:01:55.600 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:55.600 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.600 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.600 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:55.600 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142027+0000", 2024-09-21T15:01:55.600 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:55.600 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.600 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.600 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:55.600 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142029+0000", 2024-09-21T15:01:55.600 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5269999999999999e-06 2024-09-21T15:01:55.600 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.600 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.600 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:55.601 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142032+0000", 2024-09-21T15:01:55.601 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3249999999999999e-06 2024-09-21T15:01:55.601 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.601 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.601 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:55.601 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142035+0000", 2024-09-21T15:01:55.601 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.897e-06 2024-09-21T15:01:55.601 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.601 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.601 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:55.601 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177220+0000", 2024-09-21T15:01:55.601 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035184871 2024-09-21T15:01:55.601 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.601 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.602 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:55.602 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177223+0000", 2024-09-21T15:01:55.602 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3680000000000002e-06 2024-09-21T15:01:55.602 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:55.602 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:55.602 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:55.602 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.602 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.602 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45726 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:55.602 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.142037+0000", 2024-09-21T15:01:55.602 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 165.38521429599999, 2024-09-21T15:01:55.602 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035199273000000003, 2024-09-21T15:01:55.602 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:55.602 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:55.603 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:55.603 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:55.603 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:55.603 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45726 2024-09-21T15:01:55.603 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.603 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:55.603 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.603 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:55.603 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142037+0000", 2024-09-21T15:01:55.603 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:55.603 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.603 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.603 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:55.603 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142037+0000", 2024-09-21T15:01:55.604 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:55.604 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.604 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.604 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:55.604 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142037+0000", 2024-09-21T15:01:55.604 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:55.604 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.604 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.604 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:55.604 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142038+0000", 2024-09-21T15:01:55.604 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.344e-06 2024-09-21T15:01:55.604 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.604 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.604 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:55.605 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142044+0000", 2024-09-21T15:01:55.605 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1930000000000001e-06 2024-09-21T15:01:55.605 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.605 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.605 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:55.605 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142047+0000", 2024-09-21T15:01:55.605 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0280000000000001e-06 2024-09-21T15:01:55.605 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.605 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.605 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:55.605 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177233+0000", 2024-09-21T15:01:55.605 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035186394000000003 2024-09-21T15:01:55.605 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:55.605 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:55.606 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:55.606 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177237+0000", 2024-09-21T15:01:55.606 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5530000000000001e-06 2024-09-21T15:01:55.606 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:55.606 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:55.606 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:55.606 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:55.606 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:55.606 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:55.607 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T15:01:55.608 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T15:01:55.608 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T15:01:55.626 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T15:01:55.627 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:01:55.620+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:01:55.663 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T15:01:55.700 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T15:01:55.728 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:01:55.724+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:01:55.735 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T15:01:55.748 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:55.748 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:55.761 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T15:01:55.767 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T15:01:55.767 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:01:55.767 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:01:55.767 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:01:55.768 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:01:55.768 INFO:teuthology.orchestra.run.smithi032.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-21T15:01:55.768 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:01:55.768 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:01:55.768 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:01:55.771 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-21T15:01:55.772 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-21T15:01:55.829 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:01:55.824+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:01:55.872 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:55.872 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:55.883 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T15:01:55.930 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:01:55.924+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:01:55.995 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:55.995 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:56.005 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T15:01:56.052 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:01:56.048+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:01:56.121 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:56.121 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:56.131 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T15:01:56.132 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:01:56.129+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:01:56.192 INFO:teuthology.orchestra.run.smithi032.stderr:nodeep-scrub is set 2024-09-21T15:01:56.232 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:01:56.228+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:01:56.248 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:56.248 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:56.259 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T15:01:56.333 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:01:56.333+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:01:56.417 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:56.417 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:56.427 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T15:01:56.434 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:01:56.433+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:01:56.532 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:56.532 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:56.534 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:01:56.528+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:01:56.542 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T15:01:56.635 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:01:56.633+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:01:56.651 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:56.651 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:56.662 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T15:01:56.736 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:01:56.732+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:01:56.767 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:56.768 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:56.777 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T15:01:56.837 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:01:56.832+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:01:56.885 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:56.886 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:56.896 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T15:01:56.940 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:01:56.936+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:01:57.038 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:01:57.032+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:01:57.049 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:57.049 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:57.059 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T15:01:57.139 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:01:57.137+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:01:57.165 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:57.165 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:57.176 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T15:01:57.240 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:01:57.237+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:01:57.304 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:57.304 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:57.316 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T15:01:57.340 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:01:57.337+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:01:57.433 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:57.433 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:57.442 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:01:57.441+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:01:57.444 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T15:01:57.543 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:01:57.540+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:01:57.601 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:57.601 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:57.611 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T15:01:57.644 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:01:57.641+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:01:57.716 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:57.716 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:57.726 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T15:01:57.745 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:01:57.741+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:01:57.772 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T15:01:57.846 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:01:57.845+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:01:57.850 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:01:57.850 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:01:57.861 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T15:01:57.914 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T15:01:57.933 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T15:01:57.946 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:01:57.945+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:01:57.951 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T15:01:57.974 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:57.974 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T15:01:57.974 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T15:01:57.975 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T15:01:57.975 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:57.975 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4209.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T15:01:57.975 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:03.205754+0000", 2024-09-21T15:01:57.975 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 234.76558140500001, 2024-09-21T15:01:57.975 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.41117094199999998, 2024-09-21T15:01:57.975 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:57.975 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:57.975 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:57.976 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4209", 2024-09-21T15:01:57.976 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/3411998640", 2024-09-21T15:01:57.976 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 1 2024-09-21T15:01:57.976 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:57.976 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:57.976 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:57.976 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:57.976 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205754+0000", 2024-09-21T15:01:57.976 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:57.976 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:57.976 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:57.976 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:57.976 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205754+0000", 2024-09-21T15:01:57.976 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:57.976 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:57.977 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:57.977 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:57.977 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205752+0000", 2024-09-21T15:01:57.977 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999976 2024-09-21T15:01:57.977 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:57.977 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:57.977 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:57.977 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205765+0000", 2024-09-21T15:01:57.977 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3230999999999999e-05 2024-09-21T15:01:57.977 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:57.977 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:57.977 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:57.977 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205782+0000", 2024-09-21T15:01:57.978 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.6634e-05 2024-09-21T15:01:57.978 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:57.978 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:57.978 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:57.978 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205793+0000", 2024-09-21T15:01:57.978 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1297e-05 2024-09-21T15:01:57.978 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:57.978 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:57.978 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:57.978 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205867+0000", 2024-09-21T15:01:57.978 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4269000000000003e-05 2024-09-21T15:01:57.978 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:57.978 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:57.978 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:57.979 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.205877+0000", 2024-09-21T15:01:57.979 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2690000000000005e-06 2024-09-21T15:01:57.979 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:57.979 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:57.979 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:57.979 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.613700+0000", 2024-09-21T15:01:57.979 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.407823455 2024-09-21T15:01:57.979 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:57.979 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:57.979 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:57.979 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.614203+0000", 2024-09-21T15:01:57.979 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00050311500000000005 2024-09-21T15:01:57.979 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:57.979 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:57.980 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T15:01:57.980 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.614750+0000", 2024-09-21T15:01:57.980 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00054738699999999998 2024-09-21T15:01:57.980 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:57.980 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:57.980 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:01:57.980 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616367+0000", 2024-09-21T15:01:57.980 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.001616634 2024-09-21T15:01:57.980 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:57.980 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:57.980 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:01:57.980 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616574+0000", 2024-09-21T15:01:57.980 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00020668499999999999 2024-09-21T15:01:57.980 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:57.981 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:57.981 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:01:57.981 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616802+0000", 2024-09-21T15:01:57.981 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00022817900000000001 2024-09-21T15:01:57.981 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:57.981 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:57.981 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:57.981 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:03.616925+0000", 2024-09-21T15:01:57.981 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012309699999999999 2024-09-21T15:01:57.981 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:57.981 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:57.981 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:57.981 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:57.981 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:57.982 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:61214 3.7 3:e28aec10:::rbd_header.1136862f872f:head [watch reconnect cookie 139864671932080 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-21T15:01:57.982 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:18.147172+0000", 2024-09-21T15:01:57.982 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 159.82416376200001, 2024-09-21T15:01:57.982 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.019264956, 2024-09-21T15:01:57.982 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:57.982 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:57.982 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:57.982 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:57.982 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:57.982 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 61214 2024-09-21T15:01:57.982 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:57.982 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:57.982 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:57.983 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:57.983 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147172+0000", 2024-09-21T15:01:57.983 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:57.983 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:57.983 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:57.983 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:57.983 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147172+0000", 2024-09-21T15:01:57.983 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:57.983 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:57.983 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:57.983 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:57.983 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147168+0000", 2024-09-21T15:01:57.983 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999962 2024-09-21T15:01:57.983 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:57.984 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:57.984 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:57.984 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147188+0000", 2024-09-21T15:01:57.984 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9717000000000001e-05 2024-09-21T15:01:57.984 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:57.984 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:57.984 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:57.984 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147239+0000", 2024-09-21T15:01:57.984 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1493999999999999e-05 2024-09-21T15:01:57.984 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:57.984 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:57.984 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:57.984 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147265+0000", 2024-09-21T15:01:57.984 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5746999999999999e-05 2024-09-21T15:01:57.984 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:57.985 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:57.985 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:57.985 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147372+0000", 2024-09-21T15:01:57.985 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010681200000000001 2024-09-21T15:01:57.985 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:57.985 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:57.985 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op must wait for map", 2024-09-21T15:01:57.985 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.147380+0000", 2024-09-21T15:01:57.985 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7919999999999993e-06 2024-09-21T15:01:57.985 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:57.985 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:57.985 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:57.985 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.154025+0000", 2024-09-21T15:01:57.985 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0066454909999999999 2024-09-21T15:01:57.986 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:57.986 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:57.986 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:57.986 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.154041+0000", 2024-09-21T15:01:57.986 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5894000000000001e-05 2024-09-21T15:01:57.986 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:57.986 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:57.986 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:57.986 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.164345+0000", 2024-09-21T15:01:57.986 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0103040459999999 2024-09-21T15:01:57.986 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:57.986 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:57.986 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:57.986 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.164590+0000", 2024-09-21T15:01:57.986 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00024492699999999998 2024-09-21T15:01:57.987 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:57.987 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:57.987 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:57.987 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.166437+0000", 2024-09-21T15:01:57.987 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0018466330000000001 2024-09-21T15:01:57.987 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:57.987 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:57.987 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:57.987 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:57.987 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:57.987 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:61215 3.7 3:e28aec10:::rbd_header.1136862f872f:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-21T15:01:57.987 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:18.152008+0000", 2024-09-21T15:01:57.987 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 159.81932731000001, 2024-09-21T15:01:57.987 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.015644958, 2024-09-21T15:01:57.987 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:57.987 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:57.988 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:57.988 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:57.988 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:57.988 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 61215 2024-09-21T15:01:57.988 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:57.988 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:57.988 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:57.988 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:57.988 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152008+0000", 2024-09-21T15:01:57.988 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:57.988 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:57.988 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:57.988 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:57.988 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152008+0000", 2024-09-21T15:01:57.989 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:57.989 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:57.989 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:57.989 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:57.989 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152005+0000", 2024-09-21T15:01:57.989 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999971 2024-09-21T15:01:57.989 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:57.989 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:57.989 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:57.989 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152025+0000", 2024-09-21T15:01:57.989 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.9306000000000001e-05 2024-09-21T15:01:57.989 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:57.989 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:57.989 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:57.989 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152054+0000", 2024-09-21T15:01:57.989 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9189000000000001e-05 2024-09-21T15:01:57.990 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:57.990 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:57.990 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:57.990 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152078+0000", 2024-09-21T15:01:57.990 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4317000000000001e-05 2024-09-21T15:01:57.990 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:57.990 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:57.990 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:57.990 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152243+0000", 2024-09-21T15:01:57.990 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000164802 2024-09-21T15:01:57.990 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:57.990 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:57.990 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting_for_map not empty", 2024-09-21T15:01:57.990 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.152251+0000", 2024-09-21T15:01:57.991 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0600000000000008e-06 2024-09-21T15:01:57.991 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:57.991 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:57.991 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:57.991 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.154741+0000", 2024-09-21T15:01:57.991 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0024900069999999998 2024-09-21T15:01:57.991 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:57.991 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:57.991 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:57.991 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:18.154753+0000", 2024-09-21T15:01:57.991 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1588e-05 2024-09-21T15:01:57.991 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:57.991 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:57.991 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:57.991 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.165380+0000", 2024-09-21T15:01:57.991 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0106272249999999 2024-09-21T15:01:57.992 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:57.992 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:57.992 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:57.992 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.165517+0000", 2024-09-21T15:01:57.992 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013737999999999999 2024-09-21T15:01:57.992 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:57.992 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:57.992 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 6", 2024-09-21T15:01:57.992 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.165938+0000", 2024-09-21T15:01:57.992 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00042082399999999998 2024-09-21T15:01:57.992 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:57.992 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:57.992 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:01:57.992 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.166795+0000", 2024-09-21T15:01:57.993 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00085674899999999997 2024-09-21T15:01:57.993 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:57.993 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:57.993 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:01:57.993 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.167598+0000", 2024-09-21T15:01:57.993 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00080335599999999997 2024-09-21T15:01:57.993 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:57.993 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:57.993 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:01:57.993 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.167632+0000", 2024-09-21T15:01:57.993 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4122000000000003e-05 2024-09-21T15:01:57.993 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:57.993 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:57.993 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:57.993 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:19.167653+0000", 2024-09-21T15:01:57.994 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0862000000000001e-05 2024-09-21T15:01:57.994 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:57.994 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:57.994 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:57.994 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:57.994 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:57.994 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:97678 3.d e122/24 3:b2f34c6d:::rbd_data.1136bfc79a70.0000000000000006:head v 122'234, mlcod=120'232)", 2024-09-21T15:01:57.994 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.321655+0000", 2024-09-21T15:01:57.994 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 112.649680393, 2024-09-21T15:01:57.994 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.024509388999999999, 2024-09-21T15:01:57.994 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:57.994 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:57.994 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:57.995 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:57.995 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:57.995 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.321655+0000", 2024-09-21T15:01:57.995 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:57.995 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:57.995 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:57.995 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:57.995 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.321655+0000", 2024-09-21T15:01:57.995 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:57.995 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:57.995 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:57.995 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:57.995 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.321655+0000", 2024-09-21T15:01:57.995 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:57.995 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:57.996 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:57.996 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:57.996 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.329453+0000", 2024-09-21T15:01:57.996 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0077980560000000003 2024-09-21T15:01:57.996 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:57.996 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:57.996 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:57.996 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.329482+0000", 2024-09-21T15:01:57.996 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9153999999999999e-05 2024-09-21T15:01:57.996 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:57.996 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:57.996 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:57.996 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.329493+0000", 2024-09-21T15:01:57.996 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1103e-05 2024-09-21T15:01:57.996 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:57.996 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:57.997 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:57.997 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.329555+0000", 2024-09-21T15:01:57.997 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2345e-05 2024-09-21T15:01:57.997 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:57.997 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:57.997 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:57.997 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.329589+0000", 2024-09-21T15:01:57.997 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3908999999999997e-05 2024-09-21T15:01:57.997 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:57.997 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:57.997 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:01:57.997 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.346083+0000", 2024-09-21T15:01:57.998 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016493383 2024-09-21T15:01:57.998 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:57.998 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:57.998 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:57.998 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.346165+0000", 2024-09-21T15:01:57.998 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1796000000000001e-05 2024-09-21T15:01:57.998 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:57.998 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:57.998 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:57.998 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:57.998 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:57.998 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:97679 3.f e122/24 3:f2a362b4:::rbd_data.1136bfc79a70.0000000000000007:head v 122'331, mlcod=120'329)", 2024-09-21T15:01:57.998 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.337145+0000", 2024-09-21T15:01:57.999 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 112.63419038399999, 2024-09-21T15:01:57.999 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021912411, 2024-09-21T15:01:57.999 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:57.999 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:57.999 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:57.999 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:57.999 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:57.999 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.337145+0000", 2024-09-21T15:01:57.999 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:57.999 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:57.999 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:57.999 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:57.999 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.337145+0000", 2024-09-21T15:01:57.999 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:57.999 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.000 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.000 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:58.000 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.337145+0000", 2024-09-21T15:01:58.000 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:58.000 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.000 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.000 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:58.000 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.343899+0000", 2024-09-21T15:01:58.000 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0067541420000000003 2024-09-21T15:01:58.000 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.000 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.000 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:58.000 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.343921+0000", 2024-09-21T15:01:58.000 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2255e-05 2024-09-21T15:01:58.001 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.001 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.001 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:58.001 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.343933+0000", 2024-09-21T15:01:58.001 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1528e-05 2024-09-21T15:01:58.001 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.001 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.001 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:58.001 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.344001+0000", 2024-09-21T15:01:58.001 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8269999999999995e-05 2024-09-21T15:01:58.001 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.001 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.001 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:58.001 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.344029+0000", 2024-09-21T15:01:58.002 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8254000000000001e-05 2024-09-21T15:01:58.002 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.002 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.002 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:01:58.002 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.358901+0000", 2024-09-21T15:01:58.002 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0148719 2024-09-21T15:01:58.002 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.002 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.002 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:58.002 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.359058+0000", 2024-09-21T15:01:58.002 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000156446 2024-09-21T15:01:58.002 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:58.002 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:58.003 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:58.003 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.003 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.003 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:97676 3.1f e122/24 3:ffaa836a:::rbd_data.1136bfc79a70.0000000000000004:head v 122'312, mlcod=120'310)", 2024-09-21T15:01:58.003 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.348683+0000", 2024-09-21T15:01:58.003 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 112.622652219, 2024-09-21T15:01:58.003 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.033768802000000001, 2024-09-21T15:01:58.003 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:58.003 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:58.003 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:58.003 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.003 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:58.003 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.348683+0000", 2024-09-21T15:01:58.003 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:58.003 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.003 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.004 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:58.004 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.348683+0000", 2024-09-21T15:01:58.004 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:58.004 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.004 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.004 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:58.004 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.348683+0000", 2024-09-21T15:01:58.004 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:58.004 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.004 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.004 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:58.004 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.366937+0000", 2024-09-21T15:01:58.004 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.018254415 2024-09-21T15:01:58.004 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.005 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.005 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:58.005 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.366959+0000", 2024-09-21T15:01:58.005 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.1634e-05 2024-09-21T15:01:58.005 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.005 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.005 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:58.005 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.366971+0000", 2024-09-21T15:01:58.005 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2305e-05 2024-09-21T15:01:58.005 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.005 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.005 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:58.005 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.367034+0000", 2024-09-21T15:01:58.005 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.3211000000000002e-05 2024-09-21T15:01:58.005 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.005 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.006 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:58.006 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.367056+0000", 2024-09-21T15:01:58.006 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2098000000000001e-05 2024-09-21T15:01:58.006 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.006 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.006 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:01:58.006 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.382400+0000", 2024-09-21T15:01:58.006 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.015343867000000001 2024-09-21T15:01:58.006 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.006 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.006 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:58.006 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.382452+0000", 2024-09-21T15:01:58.006 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1992e-05 2024-09-21T15:01:58.006 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:58.007 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:58.007 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:58.007 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.007 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.007 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_repop(client.4406.0:97734 3.19 e122/94 3:9fcf754f:::rbd_data.1136bfc79a70.0000000000000009:head v 122'314, mlcod=120'312)", 2024-09-21T15:01:58.007 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:05.378547+0000", 2024-09-21T15:01:58.007 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 112.59278824099999, 2024-09-21T15:01:58.007 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.063380675999999997, 2024-09-21T15:01:58.007 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:58.007 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:58.007 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:58.007 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.007 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:58.008 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.378547+0000", 2024-09-21T15:01:58.008 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:58.008 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.008 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.008 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:58.008 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.378547+0000", 2024-09-21T15:01:58.008 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:58.008 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.008 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.008 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:58.008 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.378547+0000", 2024-09-21T15:01:58.008 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:01:58.008 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.008 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.009 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:58.009 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.425118+0000", 2024-09-21T15:01:58.009 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.046571166999999997 2024-09-21T15:01:58.009 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.009 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.009 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:58.009 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.425150+0000", 2024-09-21T15:01:58.009 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2150999999999998e-05 2024-09-21T15:01:58.009 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.009 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.009 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:58.009 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.425164+0000", 2024-09-21T15:01:58.009 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.3302000000000001e-05 2024-09-21T15:01:58.009 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.009 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.010 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:58.010 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.425231+0000", 2024-09-21T15:01:58.010 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7106000000000004e-05 2024-09-21T15:01:58.010 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.010 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.010 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:58.010 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.425259+0000", 2024-09-21T15:01:58.010 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8087000000000001e-05 2024-09-21T15:01:58.010 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.010 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.010 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:01:58.010 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.441890+0000", 2024-09-21T15:01:58.011 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.016631328000000001 2024-09-21T15:01:58.011 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.011 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.011 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:58.011 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:05.441928+0000", 2024-09-21T15:01:58.011 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7701999999999999e-05 2024-09-21T15:01:58.011 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:58.011 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:58.011 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:58.011 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.011 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.011 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:100187 3.10 3:08941a97:::rbd_data.1136ab4d6ff1.000000000000000e:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-21T15:01:58.012 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:10.869845+0000", 2024-09-21T15:01:58.012 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 107.101490345, 2024-09-21T15:01:58.012 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.029345237, 2024-09-21T15:01:58.012 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:58.012 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:58.012 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:58.012 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:58.012 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:58.012 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 100187 2024-09-21T15:01:58.012 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.012 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:58.012 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.012 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:58.013 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869845+0000", 2024-09-21T15:01:58.013 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:58.013 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.013 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.013 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:58.013 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869845+0000", 2024-09-21T15:01:58.013 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:58.013 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.013 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.013 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:58.013 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869844+0000", 2024-09-21T15:01:58.013 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:58.013 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.013 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.014 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:58.014 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869853+0000", 2024-09-21T15:01:58.014 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5719999999999992e-06 2024-09-21T15:01:58.014 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.014 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.014 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:58.014 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869874+0000", 2024-09-21T15:01:58.014 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.0483e-05 2024-09-21T15:01:58.014 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.014 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.014 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:58.014 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869883+0000", 2024-09-21T15:01:58.014 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6080000000000002e-06 2024-09-21T15:01:58.014 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.015 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.015 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:58.015 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.869991+0000", 2024-09-21T15:01:58.015 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000108013 2024-09-21T15:01:58.015 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.015 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.015 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:58.015 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.870258+0000", 2024-09-21T15:01:58.015 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00026629099999999999 2024-09-21T15:01:58.015 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.015 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.015 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T15:01:58.015 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.878656+0000", 2024-09-21T15:01:58.016 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0083989000000000008 2024-09-21T15:01:58.016 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.016 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.016 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:01:58.016 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.899081+0000", 2024-09-21T15:01:58.016 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.020424437 2024-09-21T15:01:58.016 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.016 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.016 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:01:58.016 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.899125+0000", 2024-09-21T15:01:58.016 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4440999999999997e-05 2024-09-21T15:01:58.016 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.016 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.017 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:01:58.017 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.899160+0000", 2024-09-21T15:01:58.017 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4518999999999999e-05 2024-09-21T15:01:58.017 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.017 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.017 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:58.017 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:10.899190+0000", 2024-09-21T15:01:58.017 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0574999999999998e-05 2024-09-21T15:01:58.017 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:58.017 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:58.017 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:58.017 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.017 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.017 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:100263 3.10 3:0ece142e:::rbd_data.1136ab4d6ff1.0000000000000008:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-21T15:01:58.018 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:11.052257+0000", 2024-09-21T15:01:58.018 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 106.919078287, 2024-09-21T15:01:58.018 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.042678025000000001, 2024-09-21T15:01:58.018 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:58.018 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:58.018 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:58.018 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:58.018 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:58.018 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 100263 2024-09-21T15:01:58.018 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.018 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:58.018 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.018 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:58.018 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052257+0000", 2024-09-21T15:01:58.019 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:58.019 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.019 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.019 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:58.019 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052257+0000", 2024-09-21T15:01:58.019 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:58.019 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.019 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.019 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:58.019 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052256+0000", 2024-09-21T15:01:58.019 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:58.019 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.019 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.019 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:58.020 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052263+0000", 2024-09-21T15:01:58.020 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4799999999999998e-06 2024-09-21T15:01:58.020 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.020 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.020 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:58.020 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052275+0000", 2024-09-21T15:01:58.020 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2386e-05 2024-09-21T15:01:58.020 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.020 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.020 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:58.020 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052281+0000", 2024-09-21T15:01:58.020 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6699999999999999e-06 2024-09-21T15:01:58.020 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.020 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.021 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:58.021 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052350+0000", 2024-09-21T15:01:58.021 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9099999999999999e-05 2024-09-21T15:01:58.021 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.021 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.021 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:58.021 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.052456+0000", 2024-09-21T15:01:58.021 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000106669 2024-09-21T15:01:58.021 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.021 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.021 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for subops from 5", 2024-09-21T15:01:58.021 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.059694+0000", 2024-09-21T15:01:58.021 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.007237566 2024-09-21T15:01:58.021 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.021 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.022 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:01:58.022 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.094721+0000", 2024-09-21T15:01:58.022 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035026977000000001 2024-09-21T15:01:58.022 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.022 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.022 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "op_commit", 2024-09-21T15:01:58.022 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.094746+0000", 2024-09-21T15:01:58.022 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5329000000000001e-05 2024-09-21T15:01:58.022 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.022 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.022 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "commit_sent", 2024-09-21T15:01:58.022 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.094850+0000", 2024-09-21T15:01:58.022 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000103452 2024-09-21T15:01:58.022 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.023 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.023 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:58.023 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:11.094935+0000", 2024-09-21T15:01:58.023 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5550000000000006e-05 2024-09-21T15:01:58.023 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:58.023 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:58.023 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:58.023 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.023 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.023 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:134708 3.a 3:51eaadc0:::rbd_data.11365ed419e3.0000000000000007:67 [read 221184~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:58.023 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.938945+0000", 2024-09-21T15:01:58.023 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 64.032391008999994, 2024-09-21T15:01:58.023 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021073784000000002, 2024-09-21T15:01:58.024 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:58.024 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:58.024 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:58.024 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:58.024 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:58.024 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 134708 2024-09-21T15:01:58.024 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.024 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:58.024 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.024 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:58.024 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938945+0000", 2024-09-21T15:01:58.024 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:58.024 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.024 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.025 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:58.025 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938945+0000", 2024-09-21T15:01:58.025 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:58.025 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.025 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.025 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:58.025 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938944+0000", 2024-09-21T15:01:58.025 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:58.025 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.025 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.025 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:58.025 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938946+0000", 2024-09-21T15:01:58.025 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.616e-06 2024-09-21T15:01:58.026 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.026 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.026 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:58.026 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938952+0000", 2024-09-21T15:01:58.026 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7309999999999998e-06 2024-09-21T15:01:58.026 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.026 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.026 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:58.026 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938957+0000", 2024-09-21T15:01:58.026 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8579999999999999e-06 2024-09-21T15:01:58.026 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.026 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.026 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:58.027 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.959730+0000", 2024-09-21T15:01:58.027 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.020773183000000001 2024-09-21T15:01:58.027 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.027 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.027 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:58.027 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.959810+0000", 2024-09-21T15:01:58.027 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0158000000000001e-05 2024-09-21T15:01:58.027 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.027 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.027 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:58.027 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.960018+0000", 2024-09-21T15:01:58.027 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000207951 2024-09-21T15:01:58.027 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:58.027 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:58.027 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:58.028 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.028 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.028 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:134709 3.a 3:51eaadc0:::rbd_data.11365ed419e3.0000000000000007:67 [read 237568~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:58.028 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.938996+0000", 2024-09-21T15:01:58.028 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 64.032340016999996, 2024-09-21T15:01:58.028 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021382577, 2024-09-21T15:01:58.028 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:58.028 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:58.028 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:58.028 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:58.028 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:58.028 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 134709 2024-09-21T15:01:58.028 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.028 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:58.029 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.029 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:58.029 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938996+0000", 2024-09-21T15:01:58.029 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:58.029 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.029 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.029 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:58.029 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938996+0000", 2024-09-21T15:01:58.029 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:58.029 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.029 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.029 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:58.029 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938995+0000", 2024-09-21T15:01:58.030 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:58.030 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.030 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.030 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:58.030 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.938999+0000", 2024-09-21T15:01:58.030 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.7979999999999999e-06 2024-09-21T15:01:58.030 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.030 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.030 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:58.030 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939006+0000", 2024-09-21T15:01:58.030 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6669999999999996e-06 2024-09-21T15:01:58.030 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.030 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.031 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:58.031 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939012+0000", 2024-09-21T15:01:58.031 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9329999999999998e-06 2024-09-21T15:01:58.031 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.031 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.031 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:58.031 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.960080+0000", 2024-09-21T15:01:58.031 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021068303 2024-09-21T15:01:58.031 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.031 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.031 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:58.031 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.960154+0000", 2024-09-21T15:01:58.031 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3819000000000006e-05 2024-09-21T15:01:58.031 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.032 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.032 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:58.032 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.960378+0000", 2024-09-21T15:01:58.032 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00022385800000000001 2024-09-21T15:01:58.032 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:58.032 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:58.032 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:58.032 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.032 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.032 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:134710 3.a 3:51eaadc0:::rbd_data.11365ed419e3.0000000000000007:67 [read 253952~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:58.032 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.939017+0000", 2024-09-21T15:01:58.032 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 64.032319005999994, 2024-09-21T15:01:58.032 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021721436, 2024-09-21T15:01:58.033 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:58.033 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:58.033 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:58.033 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:58.033 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:58.033 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 134710 2024-09-21T15:01:58.033 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.033 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:58.033 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.033 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:58.033 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939017+0000", 2024-09-21T15:01:58.033 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:58.033 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.033 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.033 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:58.034 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939017+0000", 2024-09-21T15:01:58.034 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:58.034 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.034 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.034 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:58.034 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939016+0000", 2024-09-21T15:01:58.034 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:58.034 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.034 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.034 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:58.034 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939019+0000", 2024-09-21T15:01:58.034 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.554e-06 2024-09-21T15:01:58.034 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.035 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.035 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:58.035 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939024+0000", 2024-09-21T15:01:58.035 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1569999999999999e-06 2024-09-21T15:01:58.035 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.035 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.035 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:58.035 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939034+0000", 2024-09-21T15:01:58.035 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9219999999999992e-06 2024-09-21T15:01:58.035 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.035 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.035 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:58.035 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.960408+0000", 2024-09-21T15:01:58.035 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021373792999999999 2024-09-21T15:01:58.036 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.036 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.036 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:58.036 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.960482+0000", 2024-09-21T15:01:58.036 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4331000000000004e-05 2024-09-21T15:01:58.036 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.036 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.036 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:58.036 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.960738+0000", 2024-09-21T15:01:58.036 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00025600500000000001 2024-09-21T15:01:58.036 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:58.036 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:58.036 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:58.037 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.037 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.037 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:134711 3.a 3:51eaadc0:::rbd_data.11365ed419e3.0000000000000007:67 [read 270336~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:58.037 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.939039+0000", 2024-09-21T15:01:58.037 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 64.032296957, 2024-09-21T15:01:58.037 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022064698000000001, 2024-09-21T15:01:58.037 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:58.037 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:58.037 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:58.037 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:58.037 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:58.037 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 134711 2024-09-21T15:01:58.037 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.037 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:58.038 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.038 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:58.038 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939039+0000", 2024-09-21T15:01:58.038 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:58.038 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.038 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.038 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:58.038 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939039+0000", 2024-09-21T15:01:58.038 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:58.038 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.038 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.038 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:58.038 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939038+0000", 2024-09-21T15:01:58.038 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:58.039 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.039 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.039 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:58.039 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939041+0000", 2024-09-21T15:01:58.039 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9440000000000001e-06 2024-09-21T15:01:58.039 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.039 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.039 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:58.039 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939047+0000", 2024-09-21T15:01:58.039 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1260000000000003e-06 2024-09-21T15:01:58.039 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.039 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.039 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:58.039 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939052+0000", 2024-09-21T15:01:58.040 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1619999999999999e-06 2024-09-21T15:01:58.040 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.040 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.040 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:58.040 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.960768+0000", 2024-09-21T15:01:58.040 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.021715140000000001 2024-09-21T15:01:58.040 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.040 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.040 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:58.040 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.960842+0000", 2024-09-21T15:01:58.040 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4698999999999997e-05 2024-09-21T15:01:58.040 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.040 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.040 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:58.041 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.961103+0000", 2024-09-21T15:01:58.041 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00026098499999999998 2024-09-21T15:01:58.041 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:58.041 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:58.041 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:58.041 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.041 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.041 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:134712 3.a 3:51eaadc0:::rbd_data.11365ed419e3.0000000000000007:67 [read 286720~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:58.041 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.939057+0000", 2024-09-21T15:01:58.041 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 64.032278423999998, 2024-09-21T15:01:58.041 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022387067, 2024-09-21T15:01:58.041 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:58.041 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:58.041 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:58.042 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:58.042 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:58.042 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 134712 2024-09-21T15:01:58.042 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.042 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:58.042 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.042 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:58.042 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939057+0000", 2024-09-21T15:01:58.042 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:58.042 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.042 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.042 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:58.042 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939057+0000", 2024-09-21T15:01:58.043 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:58.043 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.043 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.043 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:58.043 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939057+0000", 2024-09-21T15:01:58.043 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:58.043 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.043 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.043 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:58.043 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939059+0000", 2024-09-21T15:01:58.043 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.447e-06 2024-09-21T15:01:58.043 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.043 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.044 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:58.044 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939065+0000", 2024-09-21T15:01:58.044 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4670000000000003e-06 2024-09-21T15:01:58.044 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.044 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.044 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:58.044 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939069+0000", 2024-09-21T15:01:58.044 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2119999999999997e-06 2024-09-21T15:01:58.044 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.044 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.044 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:58.044 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.961141+0000", 2024-09-21T15:01:58.044 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022072268999999999 2024-09-21T15:01:58.044 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.045 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.045 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:58.045 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.961240+0000", 2024-09-21T15:01:58.045 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9147999999999993e-05 2024-09-21T15:01:58.045 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.045 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.045 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:58.045 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.961444+0000", 2024-09-21T15:01:58.045 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00020381899999999999 2024-09-21T15:01:58.045 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:58.045 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:58.045 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:58.045 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.045 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.046 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:134713 3.a 3:51eaadc0:::rbd_data.11365ed419e3.0000000000000007:67 [read 303104~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:58.046 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.939073+0000", 2024-09-21T15:01:58.046 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 64.032262521000007, 2024-09-21T15:01:58.046 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022687394999999999, 2024-09-21T15:01:58.046 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:58.046 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:58.046 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:58.046 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:58.046 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:58.046 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 134713 2024-09-21T15:01:58.046 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.046 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:58.047 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.047 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:58.047 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939073+0000", 2024-09-21T15:01:58.047 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:58.047 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.047 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.047 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:58.047 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939073+0000", 2024-09-21T15:01:58.047 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:58.047 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.047 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.047 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:58.047 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939073+0000", 2024-09-21T15:01:58.047 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:58.047 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.048 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.048 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:58.048 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939075+0000", 2024-09-21T15:01:58.048 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5969999999999999e-06 2024-09-21T15:01:58.048 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.048 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.048 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:58.048 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939081+0000", 2024-09-21T15:01:58.048 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.609e-06 2024-09-21T15:01:58.048 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.048 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.048 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:58.048 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939086+0000", 2024-09-21T15:01:58.048 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.916e-06 2024-09-21T15:01:58.048 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.049 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.049 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:58.049 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.961474+0000", 2024-09-21T15:01:58.049 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022388512999999999 2024-09-21T15:01:58.049 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.049 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.049 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:58.049 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.961567+0000", 2024-09-21T15:01:58.049 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.2267000000000003e-05 2024-09-21T15:01:58.049 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.049 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.049 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:58.049 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.961760+0000", 2024-09-21T15:01:58.050 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00019375999999999999 2024-09-21T15:01:58.050 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:58.050 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:58.050 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:58.050 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.050 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.050 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:134714 3.a 3:51eaadc0:::rbd_data.11365ed419e3.0000000000000007:67 [read 319488~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:58.050 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.939090+0000", 2024-09-21T15:01:58.050 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 64.032245693999997, 2024-09-21T15:01:58.050 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023043403, 2024-09-21T15:01:58.050 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:58.050 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:58.050 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:58.050 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:58.051 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:58.051 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 134714 2024-09-21T15:01:58.051 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.051 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:58.051 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.051 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:58.051 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939090+0000", 2024-09-21T15:01:58.051 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:58.051 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.051 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.051 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:58.051 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939090+0000", 2024-09-21T15:01:58.051 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:58.052 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.052 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.052 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:58.052 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939090+0000", 2024-09-21T15:01:58.052 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:58.052 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.052 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.052 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:58.052 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939092+0000", 2024-09-21T15:01:58.052 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2790000000000001e-06 2024-09-21T15:01:58.052 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.052 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.052 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:58.052 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939097+0000", 2024-09-21T15:01:58.052 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1410000000000004e-06 2024-09-21T15:01:58.052 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.053 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.053 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:58.053 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939102+0000", 2024-09-21T15:01:58.053 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7369999999999997e-06 2024-09-21T15:01:58.053 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.053 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.053 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:58.053 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.961791+0000", 2024-09-21T15:01:58.053 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022688974000000001 2024-09-21T15:01:58.053 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.053 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.053 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:58.053 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.961865+0000", 2024-09-21T15:01:58.053 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.4651000000000001e-05 2024-09-21T15:01:58.053 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.054 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.054 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:58.054 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.962133+0000", 2024-09-21T15:01:58.054 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00026793999999999999 2024-09-21T15:01:58.054 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:58.054 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:58.054 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:58.054 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.054 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.054 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:134715 3.a 3:51eaadc0:::rbd_data.11365ed419e3.0000000000000007:67 [read 335872~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:58.054 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.939106+0000", 2024-09-21T15:01:58.054 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 64.032229083999994, 2024-09-21T15:01:58.055 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023401142999999999, 2024-09-21T15:01:58.055 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:58.055 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:58.055 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:58.055 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:58.055 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:58.055 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 134715 2024-09-21T15:01:58.055 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.055 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:58.055 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.055 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:58.055 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939106+0000", 2024-09-21T15:01:58.055 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:58.055 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.056 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.056 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:58.056 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939106+0000", 2024-09-21T15:01:58.056 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:58.056 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.056 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.056 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:58.056 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939106+0000", 2024-09-21T15:01:58.056 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:58.056 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.056 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.056 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:58.056 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939114+0000", 2024-09-21T15:01:58.056 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1009999999999992e-06 2024-09-21T15:01:58.057 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.057 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.057 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:58.057 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939121+0000", 2024-09-21T15:01:58.057 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.7710000000000001e-06 2024-09-21T15:01:58.057 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.057 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.057 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:58.057 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939126+0000", 2024-09-21T15:01:58.057 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3600000000000004e-06 2024-09-21T15:01:58.057 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.057 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.057 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:58.057 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.962162+0000", 2024-09-21T15:01:58.058 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023036148999999999 2024-09-21T15:01:58.058 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.058 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.058 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:58.058 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.962239+0000", 2024-09-21T15:01:58.058 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6874999999999998e-05 2024-09-21T15:01:58.058 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.058 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.058 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:58.058 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.962508+0000", 2024-09-21T15:01:58.058 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00026842199999999999 2024-09-21T15:01:58.058 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:58.058 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:58.058 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:58.058 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.059 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.059 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:134716 3.a 3:51eaadc0:::rbd_data.11365ed419e3.0000000000000007:67 [read 352256~8192 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:58.059 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.939131+0000", 2024-09-21T15:01:58.059 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 64.032204949999993, 2024-09-21T15:01:58.059 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023742441, 2024-09-21T15:01:58.059 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:58.059 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:58.059 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:58.059 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:58.059 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:58.059 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 134716 2024-09-21T15:01:58.059 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.059 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:58.059 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.060 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:58.060 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939131+0000", 2024-09-21T15:01:58.060 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:58.060 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.060 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.060 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:58.060 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939131+0000", 2024-09-21T15:01:58.060 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:58.060 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.060 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.060 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:58.060 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939130+0000", 2024-09-21T15:01:58.060 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:58.060 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.060 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.061 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:58.061 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939133+0000", 2024-09-21T15:01:58.061 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5620000000000002e-06 2024-09-21T15:01:58.061 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.061 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.061 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:58.061 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939138+0000", 2024-09-21T15:01:58.061 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.322e-06 2024-09-21T15:01:58.061 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.061 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.061 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:58.061 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.939143+0000", 2024-09-21T15:01:58.061 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5510000000000001e-06 2024-09-21T15:01:58.061 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.061 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.062 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:58.062 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.962537+0000", 2024-09-21T15:01:58.062 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.023394808 2024-09-21T15:01:58.062 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.062 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.062 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:58.062 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.962608+0000", 2024-09-21T15:01:58.062 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0699999999999997e-05 2024-09-21T15:01:58.062 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.062 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.062 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:58.062 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.962873+0000", 2024-09-21T15:01:58.063 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00026487 2024-09-21T15:01:58.063 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:58.063 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:58.063 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:58.063 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.063 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.063 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:135024 3.a 3:53261383:::rbd_data.11365ed419e3.0000000000000037:snapdir [list-snaps] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:58.063 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.940162+0000", 2024-09-21T15:01:58.063 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 64.031173987000003, 2024-09-21T15:01:58.063 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022852172, 2024-09-21T15:01:58.063 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:58.063 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:58.063 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:58.063 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:58.063 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:58.064 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 135024 2024-09-21T15:01:58.064 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.064 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:58.064 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.064 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:58.064 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940162+0000", 2024-09-21T15:01:58.064 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:58.064 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.064 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.064 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:58.064 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940162+0000", 2024-09-21T15:01:58.064 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:58.064 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.064 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.064 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:58.065 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940161+0000", 2024-09-21T15:01:58.065 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:58.065 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.065 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.065 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:58.065 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940164+0000", 2024-09-21T15:01:58.065 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4099999999999998e-06 2024-09-21T15:01:58.065 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.065 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.065 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:58.065 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940169+0000", 2024-09-21T15:01:58.065 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3720000000000004e-06 2024-09-21T15:01:58.066 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.066 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.066 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:58.066 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940174+0000", 2024-09-21T15:01:58.066 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.724e-06 2024-09-21T15:01:58.066 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.066 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.066 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:58.066 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.962902+0000", 2024-09-21T15:01:58.066 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022728719000000001 2024-09-21T15:01:58.066 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.066 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.066 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:58.067 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.963014+0000", 2024-09-21T15:01:58.067 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000111245 2024-09-21T15:01:58.067 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:58.067 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:58.067 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:58.067 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.067 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.067 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:135044 3.a 3:53261383:::rbd_data.11365ed419e3.0000000000000037:snapdir [list-snaps] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:01:58.067 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:00:53.940484+0000", 2024-09-21T15:01:58.068 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 64.030851255000002, 2024-09-21T15:01:58.068 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022619169000000001, 2024-09-21T15:01:58.068 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:58.068 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:58.068 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:58.068 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:58.068 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:58.068 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 135044 2024-09-21T15:01:58.068 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.068 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:58.068 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.068 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:58.068 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940484+0000", 2024-09-21T15:01:58.068 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:58.069 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.069 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.069 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:58.069 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940484+0000", 2024-09-21T15:01:58.069 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:58.069 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.069 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.069 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:58.069 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940484+0000", 2024-09-21T15:01:58.069 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:58.069 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.069 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.069 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:58.069 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940487+0000", 2024-09-21T15:01:58.069 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.8449999999999999e-06 2024-09-21T15:01:58.070 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.070 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.070 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:58.070 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940493+0000", 2024-09-21T15:01:58.070 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9599999999999997e-06 2024-09-21T15:01:58.070 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.070 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.070 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:58.070 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.940497+0000", 2024-09-21T15:01:58.070 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9620000000000001e-06 2024-09-21T15:01:58.070 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.070 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.070 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:58.071 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.963034+0000", 2024-09-21T15:01:58.071 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.022536878 2024-09-21T15:01:58.071 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.071 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.071 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:58.071 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:00:53.963103+0000", 2024-09-21T15:01:58.071 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9073999999999996e-05 2024-09-21T15:01:58.071 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:58.071 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:58.071 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:58.071 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:58.071 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:58.071 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:58.072 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T15:01:58.073 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T15:01:58.074 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T15:01:58.074 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T15:01:58.074 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T15:01:58.074 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-21T15:01:58.075 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:01:58.075 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:01:58.075 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:01:58.075 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:01:58.075 INFO:teuthology.orchestra.run.smithi032.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-21T15:01:58.075 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:01:58.075 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:01:58.075 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:01:58.075 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-21T15:01:58.076 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:01:58.068+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:01:58.173 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:01:58.168+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:01:58.207 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T15:01:58.251 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:58.251 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:01:58.251 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T15:01:58.251 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:58.263 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T15:01:58.274 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:01:58.268+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:01:58.376 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:01:58.373+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:01:58.429 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:58.429 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:01:58.429 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T15:01:58.429 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T15:01:58.429 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:58.440 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T15:01:58.476 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:01:58.473+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:01:58.537 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:58.538 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T15:01:58.538 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T15:01:58.538 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T15:01:58.538 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.538 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:15130 3.2 3:47f1bbdb:::rbd_data.11361bed1f78.0000000000000067:head [sparse-read 0~147456] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-21T15:01:58.538 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:58:40.288284+0000", 2024-09-21T15:01:58.538 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 198.24794044399999, 2024-09-21T15:01:58.538 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1142935039999999, 2024-09-21T15:01:58.538 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:58.538 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:58.538 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:58.538 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:58.538 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:58.539 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 15130 2024-09-21T15:01:58.539 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.539 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:58.539 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.539 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:58.539 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288284+0000", 2024-09-21T15:01:58.539 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:58.539 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.539 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.539 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:58.539 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288284+0000", 2024-09-21T15:01:58.539 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:58.539 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.539 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.539 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:58.540 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288284+0000", 2024-09-21T15:01:58.540 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:58.540 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.540 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.540 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:58.540 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288288+0000", 2024-09-21T15:01:58.540 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5829999999999998e-06 2024-09-21T15:01:58.540 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.540 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.540 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:58.540 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288297+0000", 2024-09-21T15:01:58.540 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.0669999999999998e-06 2024-09-21T15:01:58.540 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.541 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.541 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:58.541 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288305+0000", 2024-09-21T15:01:58.541 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8860000000000004e-06 2024-09-21T15:01:58.541 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.541 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.541 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:58.541 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288417+0000", 2024-09-21T15:01:58.541 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000112952 2024-09-21T15:01:58.541 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.541 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.541 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:58.541 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:40.288428+0000", 2024-09-21T15:01:58.541 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0631e-05 2024-09-21T15:01:58.542 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.542 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.542 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:58.542 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.401876+0000", 2024-09-21T15:01:58.542 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1134479610000001 2024-09-21T15:01:58.542 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.542 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.542 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:58.542 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.402076+0000", 2024-09-21T15:01:58.542 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00019944200000000001 2024-09-21T15:01:58.542 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.542 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.542 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:58.543 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:58:41.402578+0000", 2024-09-21T15:01:58.543 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00050242899999999998 2024-09-21T15:01:58.543 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:58.543 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:58.543 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:58.543 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.543 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.543 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45708 3.2 3:415db91a:::rbd_data.1136ecbe5879.0000000000000000:28 [read 140288~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:58.543 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141801+0000", 2024-09-21T15:01:58.543 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 168.39442424999999, 2024-09-21T15:01:58.543 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034274200999999997, 2024-09-21T15:01:58.543 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:58.543 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:58.543 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:58.544 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:58.544 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:58.544 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45708 2024-09-21T15:01:58.544 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.544 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:58.544 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.544 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:58.544 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141801+0000", 2024-09-21T15:01:58.544 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:58.544 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.544 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.544 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:58.544 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141801+0000", 2024-09-21T15:01:58.544 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:58.544 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.545 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.545 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:58.545 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141800+0000", 2024-09-21T15:01:58.545 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:58.545 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.545 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.545 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:58.545 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141803+0000", 2024-09-21T15:01:58.545 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.4210000000000002e-06 2024-09-21T15:01:58.545 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.545 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.545 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:58.545 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141812+0000", 2024-09-21T15:01:58.545 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.9800000000000004e-06 2024-09-21T15:01:58.545 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.546 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.546 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:58.546 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141816+0000", 2024-09-21T15:01:58.546 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.4769999999999997e-06 2024-09-21T15:01:58.546 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.546 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.546 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:58.546 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.175865+0000", 2024-09-21T15:01:58.546 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034048875999999999 2024-09-21T15:01:58.546 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.546 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.546 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:58.546 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.175943+0000", 2024-09-21T15:01:58.546 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7464000000000002e-05 2024-09-21T15:01:58.546 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.547 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.547 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:58.547 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176075+0000", 2024-09-21T15:01:58.547 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013230800000000001 2024-09-21T15:01:58.547 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:58.547 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:58.547 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:58.547 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.547 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.547 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45709 3.2 3:415db91a:::rbd_data.1136ecbe5879.0000000000000000:28 [read 143360~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:58.547 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141820+0000", 2024-09-21T15:01:58.547 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 168.39440440300001, 2024-09-21T15:01:58.547 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034537536000000001, 2024-09-21T15:01:58.547 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:58.547 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:58.548 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:58.548 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:58.548 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:58.548 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45709 2024-09-21T15:01:58.548 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.548 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:58.548 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.548 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:58.548 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141820+0000", 2024-09-21T15:01:58.548 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:58.548 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.548 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.548 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:58.548 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141820+0000", 2024-09-21T15:01:58.549 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:58.549 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.549 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.549 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:58.549 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141820+0000", 2024-09-21T15:01:58.549 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:58.549 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.549 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.549 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:58.549 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141823+0000", 2024-09-21T15:01:58.549 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5110000000000002e-06 2024-09-21T15:01:58.549 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.549 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.549 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:58.549 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141829+0000", 2024-09-21T15:01:58.550 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0850000000000002e-06 2024-09-21T15:01:58.550 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.550 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.550 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:58.550 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141833+0000", 2024-09-21T15:01:58.550 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7620000000000003e-06 2024-09-21T15:01:58.550 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.550 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.550 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:58.550 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176103+0000", 2024-09-21T15:01:58.550 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034269933000000002 2024-09-21T15:01:58.550 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.550 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.550 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:58.550 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176161+0000", 2024-09-21T15:01:58.551 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7658999999999998e-05 2024-09-21T15:01:58.551 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.551 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.551 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:58.551 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176358+0000", 2024-09-21T15:01:58.551 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000196937 2024-09-21T15:01:58.551 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:58.551 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:58.551 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:58.551 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.551 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.551 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45710 3.2 3:415db91a:::rbd_data.1136ecbe5879.0000000000000000:28 [read 146432~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:58.551 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141838+0000", 2024-09-21T15:01:58.551 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 168.39438723999999, 2024-09-21T15:01:58.552 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034953674999999997, 2024-09-21T15:01:58.552 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:58.552 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:58.552 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:58.552 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:58.552 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:58.552 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45710 2024-09-21T15:01:58.552 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.552 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:58.552 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.552 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:58.552 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141838+0000", 2024-09-21T15:01:58.552 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:58.553 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.553 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.553 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:58.553 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141838+0000", 2024-09-21T15:01:58.553 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:58.553 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.553 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.553 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:58.553 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141837+0000", 2024-09-21T15:01:58.553 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:58.553 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.553 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.553 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:58.553 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141843+0000", 2024-09-21T15:01:58.554 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1619999999999996e-06 2024-09-21T15:01:58.554 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.554 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.554 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:58.554 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141850+0000", 2024-09-21T15:01:58.554 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4799999999999998e-06 2024-09-21T15:01:58.554 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.554 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.554 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:58.554 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141853+0000", 2024-09-21T15:01:58.554 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3050000000000001e-06 2024-09-21T15:01:58.554 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.554 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.554 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:58.554 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176490+0000", 2024-09-21T15:01:58.554 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034636413999999997 2024-09-21T15:01:58.555 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.555 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.555 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:58.555 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176606+0000", 2024-09-21T15:01:58.555 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000115841 2024-09-21T15:01:58.555 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.555 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.555 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:58.555 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176791+0000", 2024-09-21T15:01:58.555 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00018581500000000001 2024-09-21T15:01:58.555 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:58.555 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:58.555 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:58.555 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.555 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.556 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45711 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:58.556 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141856+0000", 2024-09-21T15:01:58.556 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 168.39436890299999, 2024-09-21T15:01:58.556 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034986308000000001, 2024-09-21T15:01:58.556 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:58.556 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:58.556 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:58.556 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:58.556 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:58.556 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45711 2024-09-21T15:01:58.556 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.556 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:58.556 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.556 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:58.557 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141856+0000", 2024-09-21T15:01:58.557 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:58.557 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.557 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.557 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:58.557 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141856+0000", 2024-09-21T15:01:58.557 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:58.557 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.557 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.557 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:58.557 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141856+0000", 2024-09-21T15:01:58.557 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:58.557 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.557 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.558 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:58.558 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141862+0000", 2024-09-21T15:01:58.558 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.567e-06 2024-09-21T15:01:58.558 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.558 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.558 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:58.558 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141868+0000", 2024-09-21T15:01:58.558 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9569999999999999e-06 2024-09-21T15:01:58.558 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.558 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.558 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:58.558 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141871+0000", 2024-09-21T15:01:58.558 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.146e-06 2024-09-21T15:01:58.558 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.559 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.559 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:58.559 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176831+0000", 2024-09-21T15:01:58.559 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034959935999999997 2024-09-21T15:01:58.559 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.559 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.559 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:58.559 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176842+0000", 2024-09-21T15:01:58.559 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1008e-05 2024-09-21T15:01:58.559 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:58.559 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:58.559 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:58.559 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.560 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.560 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45712 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:58.560 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141874+0000", 2024-09-21T15:01:58.560 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 168.394351008, 2024-09-21T15:01:58.560 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034993511999999997, 2024-09-21T15:01:58.560 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:58.560 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:58.560 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:58.560 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:58.560 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:58.560 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45712 2024-09-21T15:01:58.560 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.560 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:58.561 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.561 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:58.561 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141874+0000", 2024-09-21T15:01:58.561 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:58.561 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.561 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.561 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:58.561 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141874+0000", 2024-09-21T15:01:58.561 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:58.561 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.561 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.561 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:58.561 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141874+0000", 2024-09-21T15:01:58.561 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:58.561 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.562 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.562 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:58.562 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141875+0000", 2024-09-21T15:01:58.562 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5430000000000001e-06 2024-09-21T15:01:58.562 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.562 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.562 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:58.562 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141878+0000", 2024-09-21T15:01:58.562 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1420000000000002e-06 2024-09-21T15:01:58.562 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.562 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.562 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:58.562 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141881+0000", 2024-09-21T15:01:58.562 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.807e-06 2024-09-21T15:01:58.562 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.563 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.563 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:58.563 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176860+0000", 2024-09-21T15:01:58.563 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034979243 2024-09-21T15:01:58.563 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.563 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.563 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:58.563 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176867+0000", 2024-09-21T15:01:58.563 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0169999999999999e-06 2024-09-21T15:01:58.563 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:58.563 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:58.563 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:58.563 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.563 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.563 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45713 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:58.564 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141883+0000", 2024-09-21T15:01:58.564 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 168.39434143299999, 2024-09-21T15:01:58.564 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035010516999999998, 2024-09-21T15:01:58.564 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:58.564 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:58.564 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:58.564 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:58.564 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:58.564 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45713 2024-09-21T15:01:58.564 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.564 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:58.564 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.564 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:58.564 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141883+0000", 2024-09-21T15:01:58.564 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:58.565 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.565 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.565 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:58.565 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141883+0000", 2024-09-21T15:01:58.565 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:58.565 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.565 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.565 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:58.565 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141883+0000", 2024-09-21T15:01:58.565 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:58.565 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.565 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.565 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:58.565 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141885+0000", 2024-09-21T15:01:58.566 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4649999999999999e-06 2024-09-21T15:01:58.566 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.566 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.566 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:58.566 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141888+0000", 2024-09-21T15:01:58.566 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3890000000000001e-06 2024-09-21T15:01:58.566 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.566 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.566 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:58.566 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141891+0000", 2024-09-21T15:01:58.566 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9239999999999999e-06 2024-09-21T15:01:58.566 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.566 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.566 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:58.567 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176887+0000", 2024-09-21T15:01:58.567 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.034996370999999998 2024-09-21T15:01:58.567 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.567 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.567 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:58.567 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176894+0000", 2024-09-21T15:01:58.567 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6069999999999996e-06 2024-09-21T15:01:58.567 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:58.567 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:58.567 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:58.567 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.567 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.567 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45714 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:58.567 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141893+0000", 2024-09-21T15:01:58.568 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 168.394331518, 2024-09-21T15:01:58.568 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035048215000000001, 2024-09-21T15:01:58.568 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:58.568 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:58.568 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:58.568 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:58.568 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:58.568 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45714 2024-09-21T15:01:58.568 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.568 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:58.568 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.568 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:58.568 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141893+0000", 2024-09-21T15:01:58.568 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:58.568 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.569 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.569 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:58.569 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141893+0000", 2024-09-21T15:01:58.569 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:58.569 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.569 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.569 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:58.569 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141893+0000", 2024-09-21T15:01:58.569 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:58.569 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.569 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.569 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:58.570 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141895+0000", 2024-09-21T15:01:58.570 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.703e-06 2024-09-21T15:01:58.570 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.570 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.570 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:58.570 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141900+0000", 2024-09-21T15:01:58.570 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.242e-06 2024-09-21T15:01:58.570 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.570 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.570 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:58.570 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141903+0000", 2024-09-21T15:01:58.570 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.117e-06 2024-09-21T15:01:58.571 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.571 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.571 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:58.571 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176930+0000", 2024-09-21T15:01:58.571 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035026750000000002 2024-09-21T15:01:58.571 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.571 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.571 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:58.571 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176942+0000", 2024-09-21T15:01:58.571 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1668000000000001e-05 2024-09-21T15:01:58.571 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:58.571 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:58.571 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:58.571 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.571 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.572 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45715 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:58.572 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141906+0000", 2024-09-21T15:01:58.572 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 168.39431936, 2024-09-21T15:01:58.572 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035066057999999997, 2024-09-21T15:01:58.572 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:58.572 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:58.572 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:58.572 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:58.572 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:58.572 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45715 2024-09-21T15:01:58.572 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.572 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:58.572 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.572 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:58.572 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141906+0000", 2024-09-21T15:01:58.572 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:58.573 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.573 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.573 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:58.573 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141906+0000", 2024-09-21T15:01:58.573 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:58.573 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.573 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.573 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:58.573 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141905+0000", 2024-09-21T15:01:58.573 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:58.573 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.573 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.573 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:58.573 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141907+0000", 2024-09-21T15:01:58.573 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5400000000000001e-06 2024-09-21T15:01:58.574 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.574 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.574 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:58.574 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141912+0000", 2024-09-21T15:01:58.574 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.502e-06 2024-09-21T15:01:58.574 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.574 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.574 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:58.574 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141915+0000", 2024-09-21T15:01:58.574 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0170000000000001e-06 2024-09-21T15:01:58.574 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.574 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.574 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:58.574 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176964+0000", 2024-09-21T15:01:58.574 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035048587999999999 2024-09-21T15:01:58.575 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.575 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.575 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:58.575 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176972+0000", 2024-09-21T15:01:58.575 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6580000000000002e-06 2024-09-21T15:01:58.575 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:58.575 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:58.575 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:58.575 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.575 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.575 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45716 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:58.575 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141918+0000", 2024-09-21T15:01:58.575 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 168.39430718400001, 2024-09-21T15:01:58.575 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035086223, 2024-09-21T15:01:58.576 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:58.576 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:58.576 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:58.576 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:58.576 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:58.576 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45716 2024-09-21T15:01:58.576 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.576 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:58.576 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.576 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:58.576 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141918+0000", 2024-09-21T15:01:58.576 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:58.576 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.576 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.577 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:58.577 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141918+0000", 2024-09-21T15:01:58.577 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:58.577 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.577 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.577 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:58.577 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141917+0000", 2024-09-21T15:01:58.577 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:58.577 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.577 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.577 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:58.577 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141919+0000", 2024-09-21T15:01:58.577 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5680000000000001e-06 2024-09-21T15:01:58.577 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.578 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.578 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:58.578 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141922+0000", 2024-09-21T15:01:58.578 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2789999999999999e-06 2024-09-21T15:01:58.578 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.578 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.578 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:58.578 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141925+0000", 2024-09-21T15:01:58.578 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9469999999999999e-06 2024-09-21T15:01:58.578 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.578 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.579 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:58.579 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.176997+0000", 2024-09-21T15:01:58.579 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035071366999999999 2024-09-21T15:01:58.579 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.579 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.579 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:58.579 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177004+0000", 2024-09-21T15:01:58.579 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3309999999999998e-06 2024-09-21T15:01:58.579 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:58.579 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:58.579 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:58.579 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.579 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.579 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45717 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:58.580 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141928+0000", 2024-09-21T15:01:58.580 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 168.39429723800001, 2024-09-21T15:01:58.580 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035102573999999998, 2024-09-21T15:01:58.580 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:58.580 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:58.580 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:58.580 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:58.580 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:58.580 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45717 2024-09-21T15:01:58.580 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.580 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:58.581 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.581 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:58.581 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141928+0000", 2024-09-21T15:01:58.581 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:58.581 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.581 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.581 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:58.581 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141928+0000", 2024-09-21T15:01:58.581 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:58.581 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.581 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.581 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:58.581 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141927+0000", 2024-09-21T15:01:58.581 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967296 2024-09-21T15:01:58.581 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.582 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.582 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:58.582 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141929+0000", 2024-09-21T15:01:58.582 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.533e-06 2024-09-21T15:01:58.582 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.582 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.582 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:58.582 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141934+0000", 2024-09-21T15:01:58.582 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1660000000000002e-06 2024-09-21T15:01:58.582 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.582 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.582 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:58.582 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141937+0000", 2024-09-21T15:01:58.583 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0309999999999999e-06 2024-09-21T15:01:58.583 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.583 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.583 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:58.583 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177023+0000", 2024-09-21T15:01:58.583 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035085595999999997 2024-09-21T15:01:58.583 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.583 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.583 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:58.583 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177030+0000", 2024-09-21T15:01:58.583 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.486e-06 2024-09-21T15:01:58.583 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:58.583 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:58.583 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:58.584 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.584 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.584 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45718 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:58.584 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141940+0000", 2024-09-21T15:01:58.584 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 168.39428538000001, 2024-09-21T15:01:58.584 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035124956999999998, 2024-09-21T15:01:58.584 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:58.584 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:58.584 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:58.584 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:58.584 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:58.584 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45718 2024-09-21T15:01:58.584 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.584 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:58.585 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.585 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:58.585 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141940+0000", 2024-09-21T15:01:58.585 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:58.585 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.585 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.585 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:58.585 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141940+0000", 2024-09-21T15:01:58.585 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:58.585 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.585 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.585 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:58.585 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141939+0000", 2024-09-21T15:01:58.585 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:58.585 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.586 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.586 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:58.586 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141941+0000", 2024-09-21T15:01:58.586 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.502e-06 2024-09-21T15:01:58.586 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.586 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.586 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:58.586 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141946+0000", 2024-09-21T15:01:58.586 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3449999999999996e-06 2024-09-21T15:01:58.586 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.586 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.586 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:58.586 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141949+0000", 2024-09-21T15:01:58.586 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9749999999999999e-06 2024-09-21T15:01:58.587 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.587 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.587 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:58.587 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177057+0000", 2024-09-21T15:01:58.587 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035108385999999998 2024-09-21T15:01:58.587 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.587 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.587 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:58.587 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177064+0000", 2024-09-21T15:01:58.587 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0099999999999998e-06 2024-09-21T15:01:58.587 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:58.587 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:58.587 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:58.587 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.587 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.588 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45719 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:58.588 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141952+0000", 2024-09-21T15:01:58.588 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 168.394273107, 2024-09-21T15:01:58.588 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035139108000000002, 2024-09-21T15:01:58.588 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:58.588 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:58.588 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:58.588 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:58.588 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:58.588 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45719 2024-09-21T15:01:58.588 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.588 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:58.588 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.588 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:58.589 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141952+0000", 2024-09-21T15:01:58.589 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:58.589 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.589 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.589 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:58.589 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141952+0000", 2024-09-21T15:01:58.589 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:58.589 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.589 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.589 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:58.589 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141952+0000", 2024-09-21T15:01:58.589 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:58.590 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.590 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.590 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:58.590 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141953+0000", 2024-09-21T15:01:58.590 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.486e-06 2024-09-21T15:01:58.590 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.590 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.590 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:58.590 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141959+0000", 2024-09-21T15:01:58.590 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4940000000000003e-06 2024-09-21T15:01:58.590 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.590 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.590 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:58.590 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141963+0000", 2024-09-21T15:01:58.590 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.3540000000000002e-06 2024-09-21T15:01:58.591 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.591 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.591 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:58.591 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177084+0000", 2024-09-21T15:01:58.591 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035120983000000001 2024-09-21T15:01:58.591 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.591 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.591 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:58.591 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177091+0000", 2024-09-21T15:01:58.591 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0350000000000004e-06 2024-09-21T15:01:58.591 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:58.591 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:58.591 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:58.591 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.591 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.592 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45720 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:58.592 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141965+0000", 2024-09-21T15:01:58.592 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 168.39425964, 2024-09-21T15:01:58.592 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035155268000000003, 2024-09-21T15:01:58.592 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:58.592 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:58.592 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:58.592 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:58.592 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:58.592 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45720 2024-09-21T15:01:58.592 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.592 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:58.592 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.592 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:58.592 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141965+0000", 2024-09-21T15:01:58.592 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:58.593 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.593 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.593 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:58.593 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141965+0000", 2024-09-21T15:01:58.593 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:58.593 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.593 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.593 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:58.593 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141965+0000", 2024-09-21T15:01:58.593 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:58.593 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.593 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.593 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:58.593 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141966+0000", 2024-09-21T15:01:58.593 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.46e-06 2024-09-21T15:01:58.594 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.594 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.594 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:58.594 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141972+0000", 2024-09-21T15:01:58.594 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4210000000000003e-06 2024-09-21T15:01:58.594 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.594 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.594 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:58.594 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141975+0000", 2024-09-21T15:01:58.594 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.948e-06 2024-09-21T15:01:58.594 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.594 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.594 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:58.594 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177113+0000", 2024-09-21T15:01:58.594 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035137855000000003 2024-09-21T15:01:58.595 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.595 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.595 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:58.595 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177121+0000", 2024-09-21T15:01:58.595 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8269999999999998e-06 2024-09-21T15:01:58.595 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:58.595 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:58.595 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:58.595 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.595 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.595 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45721 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:58.595 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141977+0000", 2024-09-21T15:01:58.595 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 168.39424761699999, 2024-09-21T15:01:58.595 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035169984000000001, 2024-09-21T15:01:58.595 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:58.596 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:58.596 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:58.596 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:58.596 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:58.596 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45721 2024-09-21T15:01:58.596 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.596 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:58.596 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.596 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:58.596 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141977+0000", 2024-09-21T15:01:58.596 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:58.596 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.596 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.596 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:58.597 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141977+0000", 2024-09-21T15:01:58.597 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:58.597 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.597 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.597 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:58.597 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141977+0000", 2024-09-21T15:01:58.597 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:58.597 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.597 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.597 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:58.597 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141979+0000", 2024-09-21T15:01:58.597 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.511e-06 2024-09-21T15:01:58.597 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.597 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.597 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:58.597 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141982+0000", 2024-09-21T15:01:58.598 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4180000000000001e-06 2024-09-21T15:01:58.598 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.598 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.598 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:58.598 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141985+0000", 2024-09-21T15:01:58.598 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.9450000000000002e-06 2024-09-21T15:01:58.598 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.598 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.598 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:58.598 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177139+0000", 2024-09-21T15:01:58.598 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035154409999999997 2024-09-21T15:01:58.599 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.599 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.599 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:58.599 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177147+0000", 2024-09-21T15:01:58.599 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.9570000000000002e-06 2024-09-21T15:01:58.599 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:58.599 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:58.599 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:58.599 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.599 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.599 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45722 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:58.599 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141987+0000", 2024-09-21T15:01:58.600 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 168.39423764599999, 2024-09-21T15:01:58.600 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035188492000000002, 2024-09-21T15:01:58.600 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:58.600 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:58.600 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:58.600 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:58.600 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:58.600 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45722 2024-09-21T15:01:58.600 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.600 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:58.600 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.600 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:58.600 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141987+0000", 2024-09-21T15:01:58.601 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:58.601 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.601 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.601 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:58.601 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141987+0000", 2024-09-21T15:01:58.601 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:58.601 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.601 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.601 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:58.601 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141987+0000", 2024-09-21T15:01:58.601 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:58.601 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.601 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.601 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:58.602 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141988+0000", 2024-09-21T15:01:58.602 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.463e-06 2024-09-21T15:01:58.602 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.602 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.602 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:58.602 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141994+0000", 2024-09-21T15:01:58.602 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.361e-06 2024-09-21T15:01:58.602 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.602 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.602 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:58.602 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141997+0000", 2024-09-21T15:01:58.602 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0359999999999998e-06 2024-09-21T15:01:58.602 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.602 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.603 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:58.603 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177169+0000", 2024-09-21T15:01:58.603 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035171651999999998 2024-09-21T15:01:58.603 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.603 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.603 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:58.603 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177176+0000", 2024-09-21T15:01:58.603 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2189999999999998e-06 2024-09-21T15:01:58.603 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:58.603 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:58.603 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:58.603 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.603 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.603 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45723 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:58.604 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.141999+0000", 2024-09-21T15:01:58.604 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 168.394225479, 2024-09-21T15:01:58.604 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035197701999999997, 2024-09-21T15:01:58.604 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:58.604 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:58.604 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:58.604 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:58.604 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:58.604 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45723 2024-09-21T15:01:58.604 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.604 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:58.604 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.604 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:58.604 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141999+0000", 2024-09-21T15:01:58.604 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:58.604 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.605 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.605 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:58.605 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141999+0000", 2024-09-21T15:01:58.605 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:58.605 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.605 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.605 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:58.605 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.141999+0000", 2024-09-21T15:01:58.605 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:58.605 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.605 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.605 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:58.605 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142003+0000", 2024-09-21T15:01:58.605 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5049999999999999e-06 2024-09-21T15:01:58.606 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.606 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.606 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:58.606 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142008+0000", 2024-09-21T15:01:58.606 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6579999999999999e-06 2024-09-21T15:01:58.606 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.606 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.606 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:58.606 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142011+0000", 2024-09-21T15:01:58.606 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0529999999999998e-06 2024-09-21T15:01:58.606 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.606 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.606 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:58.606 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177194+0000", 2024-09-21T15:01:58.606 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035182221999999999 2024-09-21T15:01:58.607 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.607 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.607 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:58.607 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177197+0000", 2024-09-21T15:01:58.607 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5030000000000002e-06 2024-09-21T15:01:58.607 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:58.607 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:58.607 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:58.607 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.607 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.607 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45724 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:58.607 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.142015+0000", 2024-09-21T15:01:58.607 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 168.394209512, 2024-09-21T15:01:58.607 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035195345000000003, 2024-09-21T15:01:58.607 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:58.608 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:58.608 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:58.608 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:58.608 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:58.608 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45724 2024-09-21T15:01:58.608 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.608 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:58.608 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.608 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:58.608 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142015+0000", 2024-09-21T15:01:58.608 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:58.608 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.608 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.608 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:58.609 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142015+0000", 2024-09-21T15:01:58.609 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:58.609 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.609 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.609 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:58.609 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142015+0000", 2024-09-21T15:01:58.609 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:58.609 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.609 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.609 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:58.609 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142017+0000", 2024-09-21T15:01:58.609 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5540000000000001e-06 2024-09-21T15:01:58.609 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.609 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.609 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:58.610 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142022+0000", 2024-09-21T15:01:58.610 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4229999999999996e-06 2024-09-21T15:01:58.610 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.610 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.610 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:58.610 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142025+0000", 2024-09-21T15:01:58.610 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.056e-06 2024-09-21T15:01:58.610 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.610 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.610 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:58.610 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177207+0000", 2024-09-21T15:01:58.610 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035182089 2024-09-21T15:01:58.610 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.611 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.611 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:58.611 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177211+0000", 2024-09-21T15:01:58.611 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4649999999999999e-06 2024-09-21T15:01:58.611 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:58.611 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:58.611 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:58.611 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.611 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.611 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45725 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:58.611 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.142028+0000", 2024-09-21T15:01:58.611 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 168.394197233, 2024-09-21T15:01:58.611 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035195731000000001, 2024-09-21T15:01:58.611 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:58.611 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:58.612 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:58.612 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:58.612 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:58.612 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45725 2024-09-21T15:01:58.612 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.612 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:58.612 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.612 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:58.612 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142028+0000", 2024-09-21T15:01:58.612 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:58.612 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.612 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.612 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:58.612 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142028+0000", 2024-09-21T15:01:58.612 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:58.613 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.613 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.613 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:58.613 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142027+0000", 2024-09-21T15:01:58.613 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:58.613 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.613 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.613 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:58.613 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142029+0000", 2024-09-21T15:01:58.613 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.5269999999999999e-06 2024-09-21T15:01:58.613 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.613 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.613 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:58.613 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142032+0000", 2024-09-21T15:01:58.613 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3249999999999999e-06 2024-09-21T15:01:58.614 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.614 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.614 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:58.614 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142035+0000", 2024-09-21T15:01:58.614 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.897e-06 2024-09-21T15:01:58.614 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.614 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.614 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:58.614 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177220+0000", 2024-09-21T15:01:58.614 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035184871 2024-09-21T15:01:58.614 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.614 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.614 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:58.614 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177223+0000", 2024-09-21T15:01:58.615 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.3680000000000002e-06 2024-09-21T15:01:58.615 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:58.615 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:58.615 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:58.615 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.615 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.615 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:45726 3.2 3.589dba82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:58.615 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T14:59:10.142037+0000", 2024-09-21T15:01:58.615 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 168.394187518, 2024-09-21T15:01:58.615 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035199273000000003, 2024-09-21T15:01:58.615 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:58.615 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:58.616 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:58.616 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:58.616 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:58.616 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 45726 2024-09-21T15:01:58.616 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.616 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:58.616 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.616 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:58.616 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142037+0000", 2024-09-21T15:01:58.616 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:58.616 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.616 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.616 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:58.616 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142037+0000", 2024-09-21T15:01:58.616 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:58.616 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.617 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.617 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:58.617 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142037+0000", 2024-09-21T15:01:58.617 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:58.617 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.617 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.617 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:58.617 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142038+0000", 2024-09-21T15:01:58.617 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.344e-06 2024-09-21T15:01:58.617 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.617 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.617 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:58.617 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142044+0000", 2024-09-21T15:01:58.617 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1930000000000001e-06 2024-09-21T15:01:58.617 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.618 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.618 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:58.618 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.142047+0000", 2024-09-21T15:01:58.618 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0280000000000001e-06 2024-09-21T15:01:58.618 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.618 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.618 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:58.618 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177233+0000", 2024-09-21T15:01:58.618 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.035186394000000003 2024-09-21T15:01:58.618 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:58.618 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:58.618 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:58.619 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T14:59:10.177237+0000", 2024-09-21T15:01:58.619 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.5530000000000001e-06 2024-09-21T15:01:58.619 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:58.619 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:58.619 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:58.619 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:58.619 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:58.619 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:58.620 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T15:01:58.621 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:01:58.617+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:01:58.721 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:01:58.721+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:01:58.778 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:58.779 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:01:58.779 INFO:teuthology.orchestra.run.smithi032.stdout: "num_ops": 0 2024-09-21T15:01:58.779 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:58.789 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T15:01:58.822 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:01:58.816+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:01:58.894 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:58.895 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [], 2024-09-21T15:01:58.895 INFO:teuthology.orchestra.run.smithi032.stdout: "complaint_time": 30, 2024-09-21T15:01:58.895 INFO:teuthology.orchestra.run.smithi032.stdout: "num_blocked_ops": 0 2024-09-21T15:01:58.895 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:58.904 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T15:01:58.923 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:01:58.920+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:01:59.020 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:01:59.020 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 20, 2024-09-21T15:01:59.020 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 600, 2024-09-21T15:01:59.020 INFO:teuthology.orchestra.run.smithi032.stdout: "ops": [ 2024-09-21T15:01:59.020 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.020 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139776 3.16 3:6c38d0e2:::rbd_data.113664780c85.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:59.020 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.223442+0000", 2024-09-21T15:01:59.020 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 56.793810874000002, 2024-09-21T15:01:59.020 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0315488150000001, 2024-09-21T15:01:59.020 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:59.021 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "started", 2024-09-21T15:01:59.021 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:59.021 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:59.021 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:59.021 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139776 2024-09-21T15:01:59.021 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.021 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:59.021 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.021 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:59.021 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223442+0000", 2024-09-21T15:01:59.021 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:59.021 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.021 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.021 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:59.022 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223442+0000", 2024-09-21T15:01:59.022 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:59.022 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.022 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.022 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:59.022 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223442+0000", 2024-09-21T15:01:59.022 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:59.022 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.022 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.022 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:59.022 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223446+0000", 2024-09-21T15:01:59.022 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.2549999999999999e-06 2024-09-21T15:01:59.022 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.022 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.023 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:59.023 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223454+0000", 2024-09-21T15:01:59.023 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8199999999999997e-06 2024-09-21T15:01:59.023 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.023 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.023 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:59.023 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223460+0000", 2024-09-21T15:01:59.023 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6330000000000001e-06 2024-09-21T15:01:59.023 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.023 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.023 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:59.023 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223542+0000", 2024-09-21T15:01:59.023 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1929999999999997e-05 2024-09-21T15:01:59.023 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.024 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.024 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:59.024 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223552+0000", 2024-09-21T15:01:59.024 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0627e-05 2024-09-21T15:01:59.024 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.024 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.024 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:59.024 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.254634+0000", 2024-09-21T15:01:59.024 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.031081972 2024-09-21T15:01:59.024 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.024 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.024 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "started", 2024-09-21T15:01:59.024 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.254796+0000", 2024-09-21T15:01:59.025 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000161879 2024-09-21T15:01:59.025 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.025 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.025 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:59.025 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.254991+0000", 2024-09-21T15:01:59.025 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00019526499999999999 2024-09-21T15:01:59.025 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:59.025 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:59.025 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:59.025 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.025 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.025 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139785 3.11 3:8abbb109:::rbd_data.113664780c85.0000000000000070:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:59.025 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.223531+0000", 2024-09-21T15:01:59.025 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 56.793722133000003, 2024-09-21T15:01:59.026 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0187143510000001, 2024-09-21T15:01:59.026 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:59.026 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:59.026 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:59.026 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:59.026 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:59.026 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139785 2024-09-21T15:01:59.026 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.026 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:59.026 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.026 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:59.026 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223531+0000", 2024-09-21T15:01:59.026 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:59.026 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.027 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.027 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:59.027 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223531+0000", 2024-09-21T15:01:59.027 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:59.027 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.027 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.027 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:59.027 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223531+0000", 2024-09-21T15:01:59.027 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:59.027 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.027 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.027 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:59.027 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223534+0000", 2024-09-21T15:01:59.027 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2650000000000001e-06 2024-09-21T15:01:59.028 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.028 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.028 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:59.028 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223541+0000", 2024-09-21T15:01:59.028 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8990000000000004e-06 2024-09-21T15:01:59.028 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.028 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.028 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:59.028 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223546+0000", 2024-09-21T15:01:59.028 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.3399999999999997e-06 2024-09-21T15:01:59.028 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.028 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.028 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:59.028 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223724+0000", 2024-09-21T15:01:59.029 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00017745799999999999 2024-09-21T15:01:59.029 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.029 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.029 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:59.029 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223735+0000", 2024-09-21T15:01:59.029 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.1610999999999999e-05 2024-09-21T15:01:59.029 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.029 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.029 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:59.029 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242113+0000", 2024-09-21T15:01:59.029 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.018377879 2024-09-21T15:01:59.029 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.029 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.029 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:59.030 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242246+0000", 2024-09-21T15:01:59.030 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00013231500000000001 2024-09-21T15:01:59.030 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:59.030 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:59.030 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:59.030 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.030 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.030 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139790 3.11 3:8a18c7d8:::rbd_data.113664780c85.000000000000008d:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:59.030 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.223578+0000", 2024-09-21T15:01:59.030 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 56.793675137999998, 2024-09-21T15:01:59.030 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0187879719999999, 2024-09-21T15:01:59.030 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:59.030 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:59.030 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:59.031 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:59.031 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:59.031 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139790 2024-09-21T15:01:59.031 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.031 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:59.031 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.031 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:59.031 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223578+0000", 2024-09-21T15:01:59.031 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:59.031 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.031 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.031 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:59.031 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223578+0000", 2024-09-21T15:01:59.031 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:59.031 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.032 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.032 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:59.032 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223578+0000", 2024-09-21T15:01:59.032 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:59.032 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.032 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.032 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:59.032 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223581+0000", 2024-09-21T15:01:59.032 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.6879999999999999e-06 2024-09-21T15:01:59.032 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.032 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.032 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:59.032 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223586+0000", 2024-09-21T15:01:59.032 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7080000000000002e-06 2024-09-21T15:01:59.033 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.033 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.033 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:59.033 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223591+0000", 2024-09-21T15:01:59.033 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5639999999999998e-06 2024-09-21T15:01:59.033 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.033 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.033 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:59.033 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223761+0000", 2024-09-21T15:01:59.033 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000170213 2024-09-21T15:01:59.033 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.033 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.033 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:59.033 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223769+0000", 2024-09-21T15:01:59.034 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7049999999999999e-06 2024-09-21T15:01:59.034 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.034 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.034 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:59.034 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242269+0000", 2024-09-21T15:01:59.034 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.018500288 2024-09-21T15:01:59.034 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.034 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.034 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:59.034 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242366+0000", 2024-09-21T15:01:59.034 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.7126000000000005e-05 2024-09-21T15:01:59.034 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:59.034 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:59.034 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:59.035 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.035 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.035 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139792 3.16 3:6ce5caf3:::rbd_data.113664780c85.000000000000009b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:59.035 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.223595+0000", 2024-09-21T15:01:59.035 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 56.793658184999998, 2024-09-21T15:01:59.035 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0317517899999999, 2024-09-21T15:01:59.035 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:59.035 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:59.035 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:59.035 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:59.035 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:59.035 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139792 2024-09-21T15:01:59.035 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.036 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:59.036 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.036 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:59.036 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223595+0000", 2024-09-21T15:01:59.036 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:59.036 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.036 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.036 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:59.036 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223595+0000", 2024-09-21T15:01:59.036 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:59.036 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.036 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.036 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:59.036 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223595+0000", 2024-09-21T15:01:59.037 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:59.037 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.037 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.037 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:59.037 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223598+0000", 2024-09-21T15:01:59.037 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.4620000000000001e-06 2024-09-21T15:01:59.037 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.037 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.037 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:59.037 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223604+0000", 2024-09-21T15:01:59.037 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6879999999999996e-06 2024-09-21T15:01:59.037 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.037 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.037 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:59.037 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.223609+0000", 2024-09-21T15:01:59.038 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9490000000000004e-06 2024-09-21T15:01:59.038 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.038 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.038 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:59.038 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224067+0000", 2024-09-21T15:01:59.038 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00045780700000000002 2024-09-21T15:01:59.038 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.038 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.038 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:59.038 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224075+0000", 2024-09-21T15:01:59.038 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6610000000000006e-06 2024-09-21T15:01:59.038 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.039 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.039 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:59.039 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255141+0000", 2024-09-21T15:01:59.039 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0310657480000001 2024-09-21T15:01:59.039 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.039 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.039 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:59.039 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255347+0000", 2024-09-21T15:01:59.039 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000205837 2024-09-21T15:01:59.039 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:59.039 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:59.039 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:59.039 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.039 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.040 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139794 3.11 3:8cf3a2f4:::rbd_data.113664780c85.00000000000000a9:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:59.040 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.224011+0000", 2024-09-21T15:01:59.040 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 56.793242157000002, 2024-09-21T15:01:59.040 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.018507791, 2024-09-21T15:01:59.040 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:59.040 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:59.040 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:59.040 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:59.040 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:59.040 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139794 2024-09-21T15:01:59.040 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.040 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:59.040 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.040 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:59.041 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224011+0000", 2024-09-21T15:01:59.041 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:59.041 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.041 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.041 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:59.041 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224011+0000", 2024-09-21T15:01:59.041 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:59.041 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.041 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.041 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:59.041 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224010+0000", 2024-09-21T15:01:59.041 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:59.041 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.041 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.042 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:59.042 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224016+0000", 2024-09-21T15:01:59.042 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6550000000000001e-06 2024-09-21T15:01:59.042 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.042 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.042 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:59.042 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224028+0000", 2024-09-21T15:01:59.042 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2422999999999999e-05 2024-09-21T15:01:59.042 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.042 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.042 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:59.042 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224036+0000", 2024-09-21T15:01:59.042 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7749999999999993e-06 2024-09-21T15:01:59.042 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.042 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.043 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:59.043 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225176+0000", 2024-09-21T15:01:59.043 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0011401740000000001 2024-09-21T15:01:59.043 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.043 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.043 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:59.043 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225187+0000", 2024-09-21T15:01:59.043 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0951999999999999e-05 2024-09-21T15:01:59.043 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.043 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.043 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:59.043 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242392+0000", 2024-09-21T15:01:59.043 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.017204905 2024-09-21T15:01:59.043 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.044 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.044 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:59.044 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242519+0000", 2024-09-21T15:01:59.044 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012680499999999999 2024-09-21T15:01:59.044 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:59.044 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:59.044 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:59.044 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.044 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.044 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139800 3.16 3:68b050ed:::rbd_data.113664780c85.00000000000000d3:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:59.044 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.224579+0000", 2024-09-21T15:01:59.044 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 56.792674398000003, 2024-09-21T15:01:59.044 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.030932637, 2024-09-21T15:01:59.044 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:59.045 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:59.045 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:59.045 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:59.045 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:59.045 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139800 2024-09-21T15:01:59.045 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.045 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:59.045 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.045 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:59.045 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224579+0000", 2024-09-21T15:01:59.045 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:59.045 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.045 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.046 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:59.046 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224579+0000", 2024-09-21T15:01:59.046 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:59.046 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.046 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.046 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:59.046 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224578+0000", 2024-09-21T15:01:59.046 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:59.046 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.046 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.046 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:59.046 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224582+0000", 2024-09-21T15:01:59.046 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.0029999999999999e-06 2024-09-21T15:01:59.046 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.047 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.047 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:59.047 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224588+0000", 2024-09-21T15:01:59.047 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.8120000000000002e-06 2024-09-21T15:01:59.047 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.047 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.047 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:59.047 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224593+0000", 2024-09-21T15:01:59.047 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7469999999999996e-06 2024-09-21T15:01:59.047 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.047 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.047 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:59.047 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224648+0000", 2024-09-21T15:01:59.047 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4693e-05 2024-09-21T15:01:59.048 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.048 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.048 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:59.048 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224656+0000", 2024-09-21T15:01:59.048 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.8429999999999993e-06 2024-09-21T15:01:59.048 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.048 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.048 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:59.048 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255382+0000", 2024-09-21T15:01:59.048 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0307266070000001 2024-09-21T15:01:59.048 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.048 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.048 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:59.048 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255512+0000", 2024-09-21T15:01:59.049 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000129367 2024-09-21T15:01:59.049 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:59.049 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:59.049 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:59.049 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.049 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.049 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139804 3.11 3:88f9ddce:::rbd_data.113664780c85.00000000000000ef:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:59.049 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.224632+0000", 2024-09-21T15:01:59.049 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 56.792621416999999, 2024-09-21T15:01:59.049 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.018021302, 2024-09-21T15:01:59.049 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:59.049 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:59.049 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:59.050 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:59.050 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:59.050 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139804 2024-09-21T15:01:59.050 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.050 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:59.050 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.050 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:59.050 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224632+0000", 2024-09-21T15:01:59.050 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:59.050 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.051 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.051 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:59.051 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224632+0000", 2024-09-21T15:01:59.051 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:59.051 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.051 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.051 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:59.051 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224632+0000", 2024-09-21T15:01:59.051 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:59.051 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.051 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.051 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:59.051 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224635+0000", 2024-09-21T15:01:59.052 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.2710000000000001e-06 2024-09-21T15:01:59.052 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.052 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.052 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:59.052 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224640+0000", 2024-09-21T15:01:59.052 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6779999999999997e-06 2024-09-21T15:01:59.052 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.052 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.052 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:59.052 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224645+0000", 2024-09-21T15:01:59.052 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9339999999999997e-06 2024-09-21T15:01:59.052 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.052 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.052 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:59.053 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225307+0000", 2024-09-21T15:01:59.053 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00066201699999999999 2024-09-21T15:01:59.053 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.053 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.053 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:59.053 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225312+0000", 2024-09-21T15:01:59.053 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9799999999999998e-06 2024-09-21T15:01:59.053 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.053 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.053 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:59.053 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242548+0000", 2024-09-21T15:01:59.053 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.017235603 2024-09-21T15:01:59.053 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.053 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.054 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:59.054 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242653+0000", 2024-09-21T15:01:59.054 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010522 2024-09-21T15:01:59.054 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:59.054 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:59.054 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:59.054 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.054 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.054 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139809 3.11 3:884da946:::rbd_data.113664780c85.0000000000000118:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:59.054 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.224667+0000", 2024-09-21T15:01:59.054 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 56.792586372999999, 2024-09-21T15:01:59.054 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0181101960000001, 2024-09-21T15:01:59.054 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:59.054 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:59.055 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:59.055 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:59.055 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:59.055 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139809 2024-09-21T15:01:59.055 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.055 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:59.055 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.055 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:59.055 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224667+0000", 2024-09-21T15:01:59.055 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:59.055 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.055 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.055 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:59.055 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224667+0000", 2024-09-21T15:01:59.055 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:59.056 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.056 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.056 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:59.056 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224667+0000", 2024-09-21T15:01:59.056 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:59.056 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.056 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.056 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:59.056 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224669+0000", 2024-09-21T15:01:59.056 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.452e-06 2024-09-21T15:01:59.056 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.056 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.056 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:59.056 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224675+0000", 2024-09-21T15:01:59.057 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.818e-06 2024-09-21T15:01:59.057 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.057 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.057 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:59.057 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.224680+0000", 2024-09-21T15:01:59.057 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.6319999999999999e-06 2024-09-21T15:01:59.057 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.057 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.057 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:59.057 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225400+0000", 2024-09-21T15:01:59.057 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00072012699999999999 2024-09-21T15:01:59.057 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.057 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.057 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:59.058 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225409+0000", 2024-09-21T15:01:59.058 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1039999999999996e-06 2024-09-21T15:01:59.058 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.058 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.058 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:59.058 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242675+0000", 2024-09-21T15:01:59.058 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0172663319999999 2024-09-21T15:01:59.058 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.058 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.058 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:59.058 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242777+0000", 2024-09-21T15:01:59.058 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010197899999999999 2024-09-21T15:01:59.058 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:59.058 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:59.058 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:59.059 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.059 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.059 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139868 3.16 3:69f82090:::rbd_data.113664780c85.00000000000000f1:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:59.059 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.225982+0000", 2024-09-21T15:01:59.059 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 56.791270873000002, 2024-09-21T15:01:59.059 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0296791729999999, 2024-09-21T15:01:59.059 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:59.059 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:59.059 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:59.059 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:59.059 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:59.059 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139868 2024-09-21T15:01:59.059 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.060 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:59.060 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.060 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:59.060 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225982+0000", 2024-09-21T15:01:59.060 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:59.060 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.060 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.060 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:59.060 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225982+0000", 2024-09-21T15:01:59.060 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:59.060 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.060 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.060 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:59.061 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225982+0000", 2024-09-21T15:01:59.061 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:59.061 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.061 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.061 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:59.061 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225985+0000", 2024-09-21T15:01:59.061 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.74e-06 2024-09-21T15:01:59.061 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.061 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.061 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:59.061 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225990+0000", 2024-09-21T15:01:59.061 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7039999999999999e-06 2024-09-21T15:01:59.061 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.061 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.062 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:59.062 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.225995+0000", 2024-09-21T15:01:59.062 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.7249999999999997e-06 2024-09-21T15:01:59.062 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.062 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.062 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:59.062 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226299+0000", 2024-09-21T15:01:59.062 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00030383199999999997 2024-09-21T15:01:59.062 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.062 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.062 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:59.062 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226308+0000", 2024-09-21T15:01:59.062 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.9740000000000008e-06 2024-09-21T15:01:59.063 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.063 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.063 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:59.063 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255539+0000", 2024-09-21T15:01:59.063 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0292305960000001 2024-09-21T15:01:59.063 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.063 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.063 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:59.063 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255662+0000", 2024-09-21T15:01:59.063 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00012307400000000001 2024-09-21T15:01:59.063 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:59.063 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:59.064 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:59.064 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.064 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.064 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139869 3.16 3:6b6c90a4:::rbd_data.113664780c85.00000000000000fe:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:59.064 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.226006+0000", 2024-09-21T15:01:59.064 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 56.791247257000002, 2024-09-21T15:01:59.064 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.029790518, 2024-09-21T15:01:59.065 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:59.065 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:59.065 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:59.065 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:59.065 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:59.065 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139869 2024-09-21T15:01:59.065 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.066 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:59.066 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.066 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:59.066 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226006+0000", 2024-09-21T15:01:59.066 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:59.066 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.066 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.066 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:59.066 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226006+0000", 2024-09-21T15:01:59.066 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:59.066 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.066 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.066 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:59.067 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226006+0000", 2024-09-21T15:01:59.067 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:59.067 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.067 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.067 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:59.067 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226008+0000", 2024-09-21T15:01:59.067 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.5569999999999998e-06 2024-09-21T15:01:59.067 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.067 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.067 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:59.067 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226014+0000", 2024-09-21T15:01:59.067 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2079999999999997e-06 2024-09-21T15:01:59.067 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.067 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.067 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:59.068 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226020+0000", 2024-09-21T15:01:59.068 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7429999999999999e-06 2024-09-21T15:01:59.068 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.068 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.068 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:59.068 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226330+0000", 2024-09-21T15:01:59.068 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00030963300000000002 2024-09-21T15:01:59.068 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.068 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.068 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:59.068 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.226336+0000", 2024-09-21T15:01:59.068 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.6340000000000003e-06 2024-09-21T15:01:59.068 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.068 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.069 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:59.069 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255687+0000", 2024-09-21T15:01:59.069 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0293507150000001 2024-09-21T15:01:59.069 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.069 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.069 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:59.069 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255797+0000", 2024-09-21T15:01:59.069 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000109471 2024-09-21T15:01:59.069 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:59.069 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:59.069 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:59.069 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.069 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.069 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139884 3.16 3:69f10324:::rbd_data.113664780c85.0000000000000161:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:59.070 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.227573+0000", 2024-09-21T15:01:59.070 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 56.789680615999998, 2024-09-21T15:01:59.070 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0283568759999999, 2024-09-21T15:01:59.070 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:59.070 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:59.070 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:59.070 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:59.070 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:59.070 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139884 2024-09-21T15:01:59.070 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.070 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:59.070 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.070 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:59.071 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227573+0000", 2024-09-21T15:01:59.071 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:59.071 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.071 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.071 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:59.071 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227573+0000", 2024-09-21T15:01:59.071 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:59.071 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.071 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.071 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:59.071 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227572+0000", 2024-09-21T15:01:59.071 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:59.071 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.071 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.072 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:59.072 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227575+0000", 2024-09-21T15:01:59.072 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7769999999999998e-06 2024-09-21T15:01:59.072 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.072 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.072 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:59.072 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227581+0000", 2024-09-21T15:01:59.072 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.7359999999999998e-06 2024-09-21T15:01:59.072 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.072 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.073 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:59.073 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227585+0000", 2024-09-21T15:01:59.073 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.5090000000000004e-06 2024-09-21T15:01:59.073 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.073 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.073 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:59.073 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227753+0000", 2024-09-21T15:01:59.073 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00016761599999999999 2024-09-21T15:01:59.073 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.073 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.073 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:59.073 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227761+0000", 2024-09-21T15:01:59.073 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.0579999999999998e-06 2024-09-21T15:01:59.073 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.074 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.074 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:59.074 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255828+0000", 2024-09-21T15:01:59.074 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0280671349999999 2024-09-21T15:01:59.074 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.074 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.074 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:59.074 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255930+0000", 2024-09-21T15:01:59.074 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000101372 2024-09-21T15:01:59.074 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:59.074 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:59.074 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:59.074 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.075 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.075 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139913 3.16 3:6f81b46d:::rbd_data.113664780c85.0000000000000074:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:59.075 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.227685+0000", 2024-09-21T15:01:59.075 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 56.789568437, 2024-09-21T15:01:59.075 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0283644750000001, 2024-09-21T15:01:59.075 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:59.075 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:59.075 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:59.075 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:59.075 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:59.075 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139913 2024-09-21T15:01:59.075 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.075 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:59.075 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.076 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:59.076 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227685+0000", 2024-09-21T15:01:59.076 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:59.076 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.076 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.076 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:59.076 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227685+0000", 2024-09-21T15:01:59.076 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:59.076 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.076 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.076 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:59.076 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227685+0000", 2024-09-21T15:01:59.076 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:59.076 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.076 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.077 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:59.077 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227687+0000", 2024-09-21T15:01:59.077 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.2929999999999999e-06 2024-09-21T15:01:59.077 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.077 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.077 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:59.077 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227693+0000", 2024-09-21T15:01:59.077 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1070000000000001e-06 2024-09-21T15:01:59.077 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.077 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.077 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:59.077 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227698+0000", 2024-09-21T15:01:59.077 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9339999999999997e-06 2024-09-21T15:01:59.077 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.078 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.078 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:59.078 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227784+0000", 2024-09-21T15:01:59.078 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.6096999999999999e-05 2024-09-21T15:01:59.078 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.078 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.078 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:59.078 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.227791+0000", 2024-09-21T15:01:59.078 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.9090000000000003e-06 2024-09-21T15:01:59.078 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.078 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.078 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:59.078 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.255955+0000", 2024-09-21T15:01:59.078 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0281643579999999 2024-09-21T15:01:59.079 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.079 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.079 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:59.079 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256049+0000", 2024-09-21T15:01:59.079 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4047000000000003e-05 2024-09-21T15:01:59.079 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:59.079 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:59.079 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:59.079 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.079 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.079 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139988 3.11 3:88a29fa4:::rbd_data.113664780c85.00000000000000bd:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:59.079 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.229772+0000", 2024-09-21T15:01:59.079 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 56.787480938000002, 2024-09-21T15:01:59.079 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.017967668, 2024-09-21T15:01:59.080 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:59.080 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:59.080 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:59.080 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:59.080 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:59.080 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139988 2024-09-21T15:01:59.080 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.080 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:59.080 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.080 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:59.080 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229772+0000", 2024-09-21T15:01:59.080 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:59.080 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.080 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.081 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:59.081 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229772+0000", 2024-09-21T15:01:59.081 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:59.081 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.081 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.081 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:59.081 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229772+0000", 2024-09-21T15:01:59.081 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:59.081 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.082 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.082 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:59.082 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229776+0000", 2024-09-21T15:01:59.082 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.6890000000000002e-06 2024-09-21T15:01:59.082 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.082 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.082 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:59.082 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229783+0000", 2024-09-21T15:01:59.082 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.2280000000000001e-06 2024-09-21T15:01:59.082 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.082 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.082 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:59.082 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.229789+0000", 2024-09-21T15:01:59.082 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.1700000000000002e-06 2024-09-21T15:01:59.083 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.083 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.083 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:59.083 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.230106+0000", 2024-09-21T15:01:59.083 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00031700800000000003 2024-09-21T15:01:59.083 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.083 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.083 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:59.083 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.230115+0000", 2024-09-21T15:01:59.083 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.4190000000000008e-06 2024-09-21T15:01:59.083 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.083 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.083 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:59.084 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.242935+0000", 2024-09-21T15:01:59.084 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0128191449999999 2024-09-21T15:01:59.084 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.084 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.084 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for missing object", 2024-09-21T15:01:59.084 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.243078+0000", 2024-09-21T15:01:59.084 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000142984 2024-09-21T15:01:59.084 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.084 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.084 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:59.084 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.247659+0000", 2024-09-21T15:01:59.084 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.0045813030000000001 2024-09-21T15:01:59.085 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.085 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.085 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:59.085 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.247740+0000", 2024-09-21T15:01:59.085 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.1216000000000003e-05 2024-09-21T15:01:59.085 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:59.085 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:59.085 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:59.085 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.085 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.085 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:139994 3.16 3:6b9df8d1:::rbd_data.113664780c85.00000000000000e7:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:59.085 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.231370+0000", 2024-09-21T15:01:59.085 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 56.785882860000001, 2024-09-21T15:01:59.085 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.024795576, 2024-09-21T15:01:59.086 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:59.086 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:59.086 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:59.086 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:59.086 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:59.086 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 139994 2024-09-21T15:01:59.086 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.086 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:59.086 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.086 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:59.086 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231370+0000", 2024-09-21T15:01:59.086 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:59.086 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.087 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.087 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:59.087 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231370+0000", 2024-09-21T15:01:59.087 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:59.087 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.087 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.087 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:59.087 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231369+0000", 2024-09-21T15:01:59.087 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:59.087 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.087 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.087 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:59.087 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231376+0000", 2024-09-21T15:01:59.087 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.5509999999999996e-06 2024-09-21T15:01:59.087 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.088 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.088 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:59.088 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231387+0000", 2024-09-21T15:01:59.088 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.061e-05 2024-09-21T15:01:59.088 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.088 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.088 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:59.088 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231394+0000", 2024-09-21T15:01:59.088 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.3799999999999996e-06 2024-09-21T15:01:59.088 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.088 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.088 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:59.088 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231459+0000", 2024-09-21T15:01:59.088 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4716999999999997e-05 2024-09-21T15:01:59.088 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.089 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.089 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:59.089 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231468+0000", 2024-09-21T15:01:59.089 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1479999999999995e-06 2024-09-21T15:01:59.089 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.089 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.089 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:59.089 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256074+0000", 2024-09-21T15:01:59.089 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0246064180000001 2024-09-21T15:01:59.089 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.089 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.089 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:59.089 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256166+0000", 2024-09-21T15:01:59.089 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.1862999999999998e-05 2024-09-21T15:01:59.090 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:59.090 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:59.090 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:59.090 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.090 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.090 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140012 3.16 3:6bd6dfb6:::rbd_data.113664780c85.0000000000000165:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:59.090 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.231890+0000", 2024-09-21T15:01:59.090 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 56.785363101000002, 2024-09-21T15:01:59.090 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.024408881, 2024-09-21T15:01:59.090 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:59.090 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:59.090 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:59.090 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:59.090 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:59.090 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140012 2024-09-21T15:01:59.091 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.091 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:59.091 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.091 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:59.091 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231890+0000", 2024-09-21T15:01:59.091 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:59.091 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.091 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.091 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:59.091 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231890+0000", 2024-09-21T15:01:59.091 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:59.091 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.091 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.091 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:59.092 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231889+0000", 2024-09-21T15:01:59.092 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:59.092 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.092 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.092 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:59.092 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231895+0000", 2024-09-21T15:01:59.092 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.9569999999999999e-06 2024-09-21T15:01:59.092 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.092 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.092 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:59.092 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231910+0000", 2024-09-21T15:01:59.092 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.4549000000000001e-05 2024-09-21T15:01:59.092 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.092 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.093 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:59.093 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231917+0000", 2024-09-21T15:01:59.093 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.5730000000000002e-06 2024-09-21T15:01:59.093 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.093 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.093 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:59.093 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.231988+0000", 2024-09-21T15:01:59.093 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.0852999999999994e-05 2024-09-21T15:01:59.093 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.093 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.093 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:59.093 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.232000+0000", 2024-09-21T15:01:59.093 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.2159e-05 2024-09-21T15:01:59.093 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.094 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.094 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:59.094 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256192+0000", 2024-09-21T15:01:59.094 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.024192073 2024-09-21T15:01:59.094 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.094 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.094 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:59.094 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256299+0000", 2024-09-21T15:01:59.094 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000106733 2024-09-21T15:01:59.094 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:59.094 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:59.094 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:59.094 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.095 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.095 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140055 3.16 3:69bb9803:::rbd_data.113664780c85.00000000000000da:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:59.095 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.233922+0000", 2024-09-21T15:01:59.095 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 56.783331572000002, 2024-09-21T15:01:59.095 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.022511768, 2024-09-21T15:01:59.095 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:59.095 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:59.095 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:59.095 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:59.095 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:59.095 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140055 2024-09-21T15:01:59.095 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.095 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:59.095 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.095 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:59.096 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233922+0000", 2024-09-21T15:01:59.096 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:59.096 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.096 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.096 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:59.096 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233922+0000", 2024-09-21T15:01:59.096 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:59.096 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.096 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.096 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:59.096 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233921+0000", 2024-09-21T15:01:59.096 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.999999 2024-09-21T15:01:59.096 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.096 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.096 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:59.097 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233927+0000", 2024-09-21T15:01:59.097 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.0360000000000003e-06 2024-09-21T15:01:59.097 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.097 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.097 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:59.097 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233937+0000", 2024-09-21T15:01:59.097 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.6439999999999995e-06 2024-09-21T15:01:59.097 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.097 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.097 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:59.097 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233944+0000", 2024-09-21T15:01:59.097 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7710000000000006e-06 2024-09-21T15:01:59.097 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.097 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.098 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:59.098 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.233996+0000", 2024-09-21T15:01:59.098 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1597000000000001e-05 2024-09-21T15:01:59.098 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.098 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.098 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:59.098 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.234005+0000", 2024-09-21T15:01:59.098 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5960000000000004e-06 2024-09-21T15:01:59.098 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.098 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.098 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:59.098 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256332+0000", 2024-09-21T15:01:59.098 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0223271949999999 2024-09-21T15:01:59.098 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.099 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.099 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:59.099 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256434+0000", 2024-09-21T15:01:59.099 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00010180599999999999 2024-09-21T15:01:59.099 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:59.099 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:59.099 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:59.099 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.099 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.099 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140143 3.16 3:6d3ed596:::rbd_data.113664780c85.0000000000000184:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:59.099 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.239496+0000", 2024-09-21T15:01:59.099 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 56.777757043000001, 2024-09-21T15:01:59.099 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.017076576, 2024-09-21T15:01:59.099 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:59.100 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:59.100 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:59.100 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:59.100 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:59.100 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140143 2024-09-21T15:01:59.100 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.100 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:59.100 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.100 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:59.100 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239496+0000", 2024-09-21T15:01:59.100 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:59.100 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.100 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.101 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:59.101 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239496+0000", 2024-09-21T15:01:59.101 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:59.101 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.101 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.101 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:59.101 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239496+0000", 2024-09-21T15:01:59.101 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:59.101 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.101 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.101 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:59.101 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239498+0000", 2024-09-21T15:01:59.101 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.509e-06 2024-09-21T15:01:59.102 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.102 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.102 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:59.102 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239504+0000", 2024-09-21T15:01:59.102 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.6869999999999999e-06 2024-09-21T15:01:59.102 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.102 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.102 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:59.102 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239509+0000", 2024-09-21T15:01:59.102 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.8860000000000003e-06 2024-09-21T15:01:59.102 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.102 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.102 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:59.102 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239572+0000", 2024-09-21T15:01:59.103 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.2908000000000001e-05 2024-09-21T15:01:59.103 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.103 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.103 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:59.103 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.239580+0000", 2024-09-21T15:01:59.103 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.4270000000000008e-06 2024-09-21T15:01:59.103 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.103 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.103 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:59.103 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256457+0000", 2024-09-21T15:01:59.103 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0168770979999999 2024-09-21T15:01:59.103 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.103 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.103 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:59.104 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256573+0000", 2024-09-21T15:01:59.104 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.00011538400000000001 2024-09-21T15:01:59.104 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:59.104 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:59.104 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:59.104 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.104 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.104 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140163 3.16 3:694e82eb:::rbd_data.113664780c85.0000000000000052:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:59.104 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.240334+0000", 2024-09-21T15:01:59.104 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 56.776919331000002, 2024-09-21T15:01:59.104 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.016365537, 2024-09-21T15:01:59.104 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:59.104 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:59.104 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:59.104 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:59.105 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:59.105 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140163 2024-09-21T15:01:59.105 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.105 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:59.105 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.105 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:59.105 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240334+0000", 2024-09-21T15:01:59.105 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:59.105 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.105 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.105 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:59.105 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240334+0000", 2024-09-21T15:01:59.105 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:59.105 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.106 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.106 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:59.106 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240334+0000", 2024-09-21T15:01:59.106 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:59.106 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.106 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.106 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:59.106 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240337+0000", 2024-09-21T15:01:59.106 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.089e-06 2024-09-21T15:01:59.106 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.106 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.107 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:59.107 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240343+0000", 2024-09-21T15:01:59.107 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.4420000000000003e-06 2024-09-21T15:01:59.107 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.107 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.107 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:59.107 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240348+0000", 2024-09-21T15:01:59.108 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.1039999999999998e-06 2024-09-21T15:01:59.108 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.108 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.108 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:59.108 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240451+0000", 2024-09-21T15:01:59.108 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000102996 2024-09-21T15:01:59.108 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.108 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.108 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:59.108 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240460+0000", 2024-09-21T15:01:59.108 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 8.5369999999999997e-06 2024-09-21T15:01:59.108 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.108 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.109 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:59.109 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256600+0000", 2024-09-21T15:01:59.109 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.016139884 2024-09-21T15:01:59.109 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.109 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.109 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:59.109 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256700+0000", 2024-09-21T15:01:59.109 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.9844000000000002e-05 2024-09-21T15:01:59.109 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:59.109 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:59.109 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:59.109 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.109 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.110 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140164 3.16 3:68bfdc57:::rbd_data.113664780c85.0000000000000053:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:59.110 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.240358+0000", 2024-09-21T15:01:59.110 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 56.776895637000003, 2024-09-21T15:01:59.110 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.01646094, 2024-09-21T15:01:59.110 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:59.110 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:59.110 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:59.110 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:59.110 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:59.110 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140164 2024-09-21T15:01:59.110 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.110 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:59.110 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.111 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:59.111 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240358+0000", 2024-09-21T15:01:59.111 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:59.111 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.111 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.111 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:59.111 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240358+0000", 2024-09-21T15:01:59.111 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:59.111 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.111 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.111 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:59.111 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240357+0000", 2024-09-21T15:01:59.111 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:59.112 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.112 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.112 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:59.112 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240360+0000", 2024-09-21T15:01:59.112 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 2.7089999999999998e-06 2024-09-21T15:01:59.112 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.112 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.112 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:59.112 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240368+0000", 2024-09-21T15:01:59.112 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.7270000000000007e-06 2024-09-21T15:01:59.112 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.112 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.112 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:59.113 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240373+0000", 2024-09-21T15:01:59.113 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 5.4060000000000004e-06 2024-09-21T15:01:59.113 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.113 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.113 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:59.113 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240473+0000", 2024-09-21T15:01:59.113 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000100132 2024-09-21T15:01:59.113 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.113 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.113 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:59.113 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240476+0000", 2024-09-21T15:01:59.113 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.332e-06 2024-09-21T15:01:59.113 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.114 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.114 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:59.114 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256724+0000", 2024-09-21T15:01:59.114 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0162471930000001 2024-09-21T15:01:59.114 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.114 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.114 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:59.114 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256819+0000", 2024-09-21T15:01:59.114 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 9.5007999999999996e-05 2024-09-21T15:01:59.114 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:59.114 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:59.114 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:59.114 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.115 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.115 INFO:teuthology.orchestra.run.smithi032.stdout: "description": "osd_op(client.4406.0:140188 3.16 3:6f6217d4:::rbd_data.113664780c85.00000000000000fb:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:59.115 INFO:teuthology.orchestra.run.smithi032.stdout: "initiated_at": "2024-09-21T15:01:02.240978+0000", 2024-09-21T15:01:59.115 INFO:teuthology.orchestra.run.smithi032.stdout: "age": 56.776274978000004, 2024-09-21T15:01:59.115 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.015983809, 2024-09-21T15:01:59.115 INFO:teuthology.orchestra.run.smithi032.stdout: "type_data": { 2024-09-21T15:01:59.115 INFO:teuthology.orchestra.run.smithi032.stdout: "flag_point": "reached pg", 2024-09-21T15:01:59.115 INFO:teuthology.orchestra.run.smithi032.stdout: "client_info": { 2024-09-21T15:01:59.115 INFO:teuthology.orchestra.run.smithi032.stdout: "client": "client.4406", 2024-09-21T15:01:59.115 INFO:teuthology.orchestra.run.smithi032.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:59.115 INFO:teuthology.orchestra.run.smithi032.stdout: "tid": 140188 2024-09-21T15:01:59.115 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.115 INFO:teuthology.orchestra.run.smithi032.stdout: "events": [ 2024-09-21T15:01:59.115 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.115 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "initiated", 2024-09-21T15:01:59.116 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240978+0000", 2024-09-21T15:01:59.116 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:59.116 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.116 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.116 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "throttled", 2024-09-21T15:01:59.116 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240978+0000", 2024-09-21T15:01:59.116 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0 2024-09-21T15:01:59.116 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.116 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.116 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "header_read", 2024-09-21T15:01:59.116 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240978+0000", 2024-09-21T15:01:59.116 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:59.116 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.116 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.117 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "all_read", 2024-09-21T15:01:59.117 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240981+0000", 2024-09-21T15:01:59.117 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 3.1559999999999999e-06 2024-09-21T15:01:59.117 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.117 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.117 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "dispatched", 2024-09-21T15:01:59.117 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240987+0000", 2024-09-21T15:01:59.117 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 6.229e-06 2024-09-21T15:01:59.117 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.117 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.117 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "queued_for_pg", 2024-09-21T15:01:59.117 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.240992+0000", 2024-09-21T15:01:59.117 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 4.9710000000000003e-06 2024-09-21T15:01:59.118 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.118 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.118 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:59.118 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.241361+0000", 2024-09-21T15:01:59.118 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000369236 2024-09-21T15:01:59.118 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.118 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.118 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "waiting for peered", 2024-09-21T15:01:59.118 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:02.241369+0000", 2024-09-21T15:01:59.118 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 7.6199999999999999e-06 2024-09-21T15:01:59.118 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.118 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.119 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "reached_pg", 2024-09-21T15:01:59.119 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256844+0000", 2024-09-21T15:01:59.119 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 1.0154751609999999 2024-09-21T15:01:59.119 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:01:59.119 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:01:59.119 INFO:teuthology.orchestra.run.smithi032.stdout: "event": "done", 2024-09-21T15:01:59.119 INFO:teuthology.orchestra.run.smithi032.stdout: "time": "2024-09-21T15:01:03.256962+0000", 2024-09-21T15:01:59.119 INFO:teuthology.orchestra.run.smithi032.stdout: "duration": 0.000117964 2024-09-21T15:01:59.119 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:59.119 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:59.119 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:59.119 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:01:59.119 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:01:59.119 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:01:59.121 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T15:01:59.121 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:01:59.116+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:01:59.222 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:01:59.221+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:01:59.244 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:01:59.244 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:01:59.244 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T15:01:59.244 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:01:59.246 INFO:teuthology.orchestra.run.smithi032.stderr:noscrub is unset 2024-09-21T15:01:59.257 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T15:01:59.322 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:01:59.321+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:01:59.356 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:01:59.356 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:01:59.356 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T15:01:59.356 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T15:01:59.357 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:01:59.366 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T15:01:59.423 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:01:59.421+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:01:59.481 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:01:59.481 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T15:01:59.481 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T15:01:59.481 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T15:01:59.481 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.482 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45854 3.18 3.e315c3d8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:59.482 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159802+0000", 2024-09-21T15:01:59.482 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 169.316794198, 2024-09-21T15:01:59.482 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068358938999999994, 2024-09-21T15:01:59.482 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:59.482 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:59.482 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:59.482 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:59.482 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:59.482 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45854 2024-09-21T15:01:59.482 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.482 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:59.482 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.483 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:59.483 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159802+0000", 2024-09-21T15:01:59.483 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:59.483 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.483 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.483 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:59.483 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159802+0000", 2024-09-21T15:01:59.483 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:59.483 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.483 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.484 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:59.484 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159801+0000", 2024-09-21T15:01:59.484 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:59.484 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.484 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.484 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:59.484 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159804+0000", 2024-09-21T15:01:59.484 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0970000000000002e-06 2024-09-21T15:01:59.484 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.484 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.484 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:59.484 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159823+0000", 2024-09-21T15:01:59.484 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8895000000000001e-05 2024-09-21T15:01:59.485 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.485 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.485 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:59.485 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159837+0000", 2024-09-21T15:01:59.485 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3247999999999999e-05 2024-09-21T15:01:59.485 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.485 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.485 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:59.485 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228151+0000", 2024-09-21T15:01:59.485 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068314642999999994 2024-09-21T15:01:59.485 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.485 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.485 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:59.486 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228161+0000", 2024-09-21T15:01:59.486 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4250000000000004e-06 2024-09-21T15:01:59.486 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:59.486 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:59.486 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:59.486 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.486 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.486 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45985 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:59.486 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159897+0000", 2024-09-21T15:01:59.486 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 169.316698696, 2024-09-21T15:01:59.486 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068345664, 2024-09-21T15:01:59.486 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:59.486 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:59.487 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:59.487 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:59.487 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:59.487 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45985 2024-09-21T15:01:59.487 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.487 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:59.487 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.487 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:59.487 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159897+0000", 2024-09-21T15:01:59.487 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:59.487 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.488 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.488 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:59.488 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159897+0000", 2024-09-21T15:01:59.488 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:59.488 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.488 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.488 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:59.488 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159897+0000", 2024-09-21T15:01:59.488 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:59.488 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.488 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.488 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:59.488 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159900+0000", 2024-09-21T15:01:59.488 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.954e-06 2024-09-21T15:01:59.489 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.489 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.489 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:59.489 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159913+0000", 2024-09-21T15:01:59.489 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3106e-05 2024-09-21T15:01:59.489 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.489 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.489 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:59.489 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159918+0000", 2024-09-21T15:01:59.489 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5199999999999997e-06 2024-09-21T15:01:59.489 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.489 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.489 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:59.489 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228236+0000", 2024-09-21T15:01:59.490 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068317040999999995 2024-09-21T15:01:59.490 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.490 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.490 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:59.490 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228243+0000", 2024-09-21T15:01:59.490 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3880000000000002e-06 2024-09-21T15:01:59.490 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:59.490 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:59.490 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:59.490 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.490 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.490 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45986 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:59.490 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159924+0000", 2024-09-21T15:01:59.491 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 169.31667240100001, 2024-09-21T15:01:59.491 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068364362999999997, 2024-09-21T15:01:59.491 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:59.491 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:59.491 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:59.491 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:59.491 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:59.491 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45986 2024-09-21T15:01:59.491 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.491 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:59.491 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.491 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:59.491 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159924+0000", 2024-09-21T15:01:59.491 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:59.492 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.492 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.492 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:59.492 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159924+0000", 2024-09-21T15:01:59.492 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:59.492 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.492 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.492 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:59.492 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159923+0000", 2024-09-21T15:01:59.492 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:59.492 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.492 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.492 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:59.492 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159926+0000", 2024-09-21T15:01:59.492 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0510000000000001e-06 2024-09-21T15:01:59.493 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.493 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.493 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:59.493 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159932+0000", 2024-09-21T15:01:59.493 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4160000000000001e-06 2024-09-21T15:01:59.493 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.493 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.493 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:59.493 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159938+0000", 2024-09-21T15:01:59.493 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.383e-06 2024-09-21T15:01:59.493 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.493 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.493 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:59.493 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228275+0000", 2024-09-21T15:01:59.494 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068337007000000005 2024-09-21T15:01:59.494 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.494 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.494 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:59.494 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228288+0000", 2024-09-21T15:01:59.494 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3052e-05 2024-09-21T15:01:59.494 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:59.494 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:59.494 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:59.494 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.494 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.494 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:45987 3.18 3.43c3f7d8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:01:59.494 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:10.159944+0000", 2024-09-21T15:01:59.494 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 169.31665217700001, 2024-09-21T15:01:59.495 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068387232000000006, 2024-09-21T15:01:59.495 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:59.495 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:59.495 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:59.495 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:59.495 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:59.495 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 45987 2024-09-21T15:01:59.495 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.495 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:59.495 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.495 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:59.495 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159944+0000", 2024-09-21T15:01:59.495 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:59.495 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.496 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.496 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:59.496 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159944+0000", 2024-09-21T15:01:59.496 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:59.496 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.496 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.496 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:59.496 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159943+0000", 2024-09-21T15:01:59.496 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:59.496 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.496 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.496 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:59.496 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159946+0000", 2024-09-21T15:01:59.497 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6309999999999999e-06 2024-09-21T15:01:59.497 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.497 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.497 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:59.497 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159952+0000", 2024-09-21T15:01:59.497 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6540000000000004e-06 2024-09-21T15:01:59.497 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.497 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.497 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:59.497 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.159957+0000", 2024-09-21T15:01:59.497 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9280000000000001e-06 2024-09-21T15:01:59.497 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.497 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.497 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:59.498 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228322+0000", 2024-09-21T15:01:59.498 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.068365067000000002 2024-09-21T15:01:59.498 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.498 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.498 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:59.498 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:10.228331+0000", 2024-09-21T15:01:59.498 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3270000000000007e-06 2024-09-21T15:01:59.498 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:59.498 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:59.498 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:59.498 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.498 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.498 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:97672 3.4 3:26046d29:::rbd_data.1136bfc79a70.0000000000000001:head [write 0~4718592 [fadvise_sequential+fadvise_nocache],write 5242880~131072 [fadvise_sequential+fadvise_nocache],write 5439488~147456 [fadvise_sequential+fadvise_nocache],write 5652480~229376 [fadvise_sequential+fadvise_nocache],write 5947392~147456 [fadvise_sequential+fadvise_nocache],write 6160384~229376 [fadvise_sequential+fadvise_nocache],write 6455296~147456 [fadvise_sequential+fadvise_nocache],write 6668288~229376 [fadvise_sequential+fadvise_nocache],write 6963200~147456 [fadvise_sequential+fadvise_nocache],write 7176192~147456 [fadvise_sequential+fadvise_nocache],write 7340032~65536 [fadvise_sequential+fadvise_nocache],write 7471104~147456 [fadvise_sequential+fadvise_nocache],write 7684096~147456 [fadvise_sequential+fadvise_nocache],write 7864320~49152 [fadvise_sequential+fadvise_nocache],write 7979008~147456 [fadvise_sequential+fadvise_nocache],write 8192000~147456 [fadvise_sequential+fadvise_nocache],write 8388608~32768 [fadvise_sequential+fadvise_nocache],write 8519680~114688 [fadvise_sequential+fadvise_nocache],write 8732672~114688 [fadvise_sequential+fadvise_nocache],write 8912896~16384 [fadvise_sequential+fadvise_nocache],write 9109504~32768 [fadvise_sequential+fadvise_nocache],write 9322496~32768 [fadvise_sequential+fadvise_nocache],write 9617408~32768 [fadvise_sequential+fadvise_nocache],write 9830400~32768 [fadvise_sequential+fadvise_nocache],write 9961472~98304 [fadvise_sequential+fadvise_nocache],write 10076160~196608 [fadvise_sequential+fadvise_nocache],write 10289152~278528 [fadvise_sequential+fadvise_nocache],write 10584064~196608 [fadvise_sequential+fadvise_nocache],write 10797056~196608 [fadvise_sequential+fadvise_nocache],write 11010048~65536 [fadvise_sequential+fadvise_nocache],write 11091968~196608 [fadvise_sequential+fadvise_nocache],write 11304960~196608 [fadvise_sequential+fadvise_nocache],write 11517952~65536 [fadvise_sequential+fadvise_nocache],write 11616256~98304 [fadvise_sequential+fadvise_nocache],write 11730944~65536 [fadvise_sequential+fadvise_nocache],write 11829248~98304 [fadvise_sequential+fadvise_nocache],write 11943936~65536 [fadvise_sequential+fadvise_nocache],write 12042240~49152 [fadvise_sequential+fadvise_nocache],write 12124160~98304 [fadvise_sequential+fadvise_nocache],write 12238848~65536 [fadvise_sequential+fadvise_nocache],write 12337152~98304 [fadvise_sequential+fadvise_nocache],write 12451840~65536 [fadvise_sequential+fadvise_nocache],write 12550144~32768 [fadvise_sequential+fadvise_nocache],write 12632064~98304 [fadvise_sequential+fadvise_nocache],write 12845056~98304 [fadvise_sequential+fadvise_nocache],write 13058048~49152 [fadvise_sequential+fadvise_nocache],write 13139968~98304 [fadvise_sequential+fadvise_nocache],write 13352960~98304 [fadvise_sequential+fadvise_nocache],write 13565952~65536 [fadvise_sequential+fadvise_nocache],write 13647872~98304 [fadvise_sequential+fadvise_nocache],write 13860864~98304 [fadvise_sequential+fadvise_nocache],write 14073856~81920 [fadvise_sequential+fadvise_nocache],write 15269888~81920 [fadvise_sequential+fadvise_nocache],write 15482880~81920 [fadvise_sequential+fadvise_nocache],write 15695872~32768 [fadvise_sequential+fadvise_nocache],write 15777792~81920 [fadvise_sequential+fadvise_nocache],write 15990784~81920 [fadvise_sequential+fadvise_nocache],write 16203776~49152 [fadvise_sequential+fadvise_nocache],write 16285696~81920 [fadvise_sequential+fadvise_nocache],write 16498688~81920 [fadvise_sequential+fadvise_nocache],write 16711680~65536 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:01:59.499 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.298669+0000", 2024-09-21T15:01:59.499 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 114.177927111, 2024-09-21T15:01:59.499 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.10282358900000001, 2024-09-21T15:01:59.499 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:59.499 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:59.499 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:59.499 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:59.499 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:59.499 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97672 2024-09-21T15:01:59.499 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.499 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:59.499 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.499 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:59.499 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.298669+0000", 2024-09-21T15:01:59.499 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:59.500 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.500 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.500 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:59.500 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.298669+0000", 2024-09-21T15:01:59.500 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:59.500 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.500 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.500 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:59.500 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.298667+0000", 2024-09-21T15:01:59.500 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:01:59.500 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.500 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.500 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:59.500 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.327458+0000", 2024-09-21T15:01:59.500 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.028790754000000002 2024-09-21T15:01:59.501 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.501 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.501 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:59.501 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.327492+0000", 2024-09-21T15:01:59.501 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3806999999999997e-05 2024-09-21T15:01:59.501 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.501 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.501 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:59.501 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.327507+0000", 2024-09-21T15:01:59.501 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5112e-05 2024-09-21T15:01:59.501 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.501 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.501 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:59.501 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.327597+0000", 2024-09-21T15:01:59.501 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9938999999999994e-05 2024-09-21T15:01:59.501 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.502 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.502 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:59.502 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.327927+0000", 2024-09-21T15:01:59.502 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000329766 2024-09-21T15:01:59.502 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.502 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.502 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T15:01:59.502 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.330735+0000", 2024-09-21T15:01:59.502 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0028086869999999998 2024-09-21T15:01:59.502 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.502 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.503 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:01:59.503 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.378299+0000", 2024-09-21T15:01:59.503 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.047563495999999997 2024-09-21T15:01:59.503 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.503 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.503 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:01:59.503 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.401248+0000", 2024-09-21T15:01:59.503 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.022949311 2024-09-21T15:01:59.503 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.503 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.503 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:01:59.503 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.401446+0000", 2024-09-21T15:01:59.503 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019831199999999999 2024-09-21T15:01:59.503 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.504 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.504 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:59.504 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.401492+0000", 2024-09-21T15:01:59.504 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.6109000000000002e-05 2024-09-21T15:01:59.504 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:59.504 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:59.504 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:59.504 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.504 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.504 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139769 3.1e 3:793dcf44:::rbd_data.113664780c85.0000000000000000:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:59.504 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.221432+0000", 2024-09-21T15:01:59.504 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 57.255164241999999, 2024-09-21T15:01:59.504 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.035363362, 2024-09-21T15:01:59.505 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:59.505 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:59.505 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:59.505 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:59.505 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:59.505 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139769 2024-09-21T15:01:59.505 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.505 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:59.505 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.505 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:59.505 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221432+0000", 2024-09-21T15:01:59.505 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:59.505 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.505 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.505 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:59.506 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221432+0000", 2024-09-21T15:01:59.506 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:59.506 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.506 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.506 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:59.506 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221430+0000", 2024-09-21T15:01:59.506 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:01:59.506 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.506 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.506 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:59.506 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221438+0000", 2024-09-21T15:01:59.506 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2630000000000003e-06 2024-09-21T15:01:59.506 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.506 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.507 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:59.507 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221458+0000", 2024-09-21T15:01:59.507 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0071000000000001e-05 2024-09-21T15:01:59.507 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.507 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.507 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:59.507 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221467+0000", 2024-09-21T15:01:59.507 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.816e-06 2024-09-21T15:01:59.507 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.507 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.507 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:59.507 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221572+0000", 2024-09-21T15:01:59.508 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000105166 2024-09-21T15:01:59.508 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.508 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.508 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:59.508 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221589+0000", 2024-09-21T15:01:59.508 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7391e-05 2024-09-21T15:01:59.508 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.508 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.508 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:59.508 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.255741+0000", 2024-09-21T15:01:59.508 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0341519690000001 2024-09-21T15:01:59.508 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.508 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.508 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:59.509 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.256353+0000", 2024-09-21T15:01:59.509 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000611544 2024-09-21T15:01:59.509 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.509 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.509 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:59.509 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.256795+0000", 2024-09-21T15:01:59.509 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00044208700000000002 2024-09-21T15:01:59.509 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:59.509 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:59.509 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:59.509 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.509 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.510 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139773 3.1e 3:7fa3054f:::rbd_data.113664780c85.000000000000001c:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:59.510 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.221517+0000", 2024-09-21T15:01:59.510 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 57.255079197999997, 2024-09-21T15:01:59.510 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0367423140000001, 2024-09-21T15:01:59.510 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:59.510 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:59.510 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:59.510 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:59.510 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:59.510 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139773 2024-09-21T15:01:59.510 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.510 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:59.510 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.511 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:59.511 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221517+0000", 2024-09-21T15:01:59.511 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:59.511 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.511 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.511 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:59.511 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221517+0000", 2024-09-21T15:01:59.511 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:59.511 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.511 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.511 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:59.511 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221516+0000", 2024-09-21T15:01:59.511 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:59.511 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.512 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.512 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:59.512 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221520+0000", 2024-09-21T15:01:59.512 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.1030000000000004e-06 2024-09-21T15:01:59.512 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.512 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.512 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:59.512 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221566+0000", 2024-09-21T15:01:59.512 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5512000000000001e-05 2024-09-21T15:01:59.512 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.512 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.512 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:59.512 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221575+0000", 2024-09-21T15:01:59.513 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9630000000000004e-06 2024-09-21T15:01:59.513 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.513 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.513 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:59.513 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221650+0000", 2024-09-21T15:01:59.513 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.5531999999999994e-05 2024-09-21T15:01:59.513 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.513 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.513 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:59.513 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221661+0000", 2024-09-21T15:01:59.513 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0878e-05 2024-09-21T15:01:59.513 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.513 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.514 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:59.514 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.257513+0000", 2024-09-21T15:01:59.514 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0358513089999999 2024-09-21T15:01:59.514 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.514 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.514 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:59.514 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.257885+0000", 2024-09-21T15:01:59.514 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00037274299999999998 2024-09-21T15:01:59.514 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.514 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.514 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:59.514 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.258259+0000", 2024-09-21T15:01:59.514 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000373829 2024-09-21T15:01:59.514 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:59.515 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:59.515 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:59.515 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.515 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.515 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139789 3.1e 3:7c803367:::rbd_data.113664780c85.000000000000008c:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:59.515 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.221656+0000", 2024-09-21T15:01:59.515 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 57.254939567000001, 2024-09-21T15:01:59.515 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.037139595, 2024-09-21T15:01:59.515 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:59.515 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:59.515 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:59.515 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:59.515 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:59.516 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139789 2024-09-21T15:01:59.516 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.516 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:59.516 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.516 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:59.516 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221656+0000", 2024-09-21T15:01:59.516 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:59.516 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.516 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.516 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:59.516 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221656+0000", 2024-09-21T15:01:59.516 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:59.516 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.517 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.517 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:59.517 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221656+0000", 2024-09-21T15:01:59.517 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:59.517 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.517 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.517 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:59.517 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221660+0000", 2024-09-21T15:01:59.517 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0369999999999996e-06 2024-09-21T15:01:59.517 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.517 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.517 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:59.517 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221668+0000", 2024-09-21T15:01:59.517 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3440000000000001e-06 2024-09-21T15:01:59.518 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.518 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.518 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:59.518 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221674+0000", 2024-09-21T15:01:59.518 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0920000000000003e-06 2024-09-21T15:01:59.518 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.518 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.518 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:59.518 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221787+0000", 2024-09-21T15:01:59.518 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000112289 2024-09-21T15:01:59.518 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.518 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.519 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:59.519 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.221798+0000", 2024-09-21T15:01:59.519 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1419e-05 2024-09-21T15:01:59.519 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.519 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.519 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:59.519 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.258642+0000", 2024-09-21T15:01:59.519 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.036843851 2024-09-21T15:01:59.519 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.519 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.519 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:59.519 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.258796+0000", 2024-09-21T15:01:59.520 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015414199999999999 2024-09-21T15:01:59.520 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:59.520 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:59.520 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:59.520 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.520 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.520 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139832 3.1e 3:7b675eeb:::rbd_data.113664780c85.00000000000001b3:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:59.520 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.223976+0000", 2024-09-21T15:01:59.520 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 57.252619834000001, 2024-09-21T15:01:59.520 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0350267980000001, 2024-09-21T15:01:59.520 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:59.520 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:59.520 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:59.520 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:59.520 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:59.520 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139832 2024-09-21T15:01:59.521 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.521 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:59.521 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.521 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:59.521 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223976+0000", 2024-09-21T15:01:59.521 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:59.521 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.521 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.521 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:59.521 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223976+0000", 2024-09-21T15:01:59.521 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:59.521 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.521 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.521 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:59.522 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223976+0000", 2024-09-21T15:01:59.522 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:59.522 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.522 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.522 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:59.522 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223981+0000", 2024-09-21T15:01:59.522 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1560000000000003e-06 2024-09-21T15:01:59.522 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.522 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.522 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:59.522 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223999+0000", 2024-09-21T15:01:59.522 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8808e-05 2024-09-21T15:01:59.522 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.522 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.522 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:59.523 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224008+0000", 2024-09-21T15:01:59.523 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7269999999999996e-06 2024-09-21T15:01:59.523 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.523 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.523 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:59.523 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224208+0000", 2024-09-21T15:01:59.523 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000199614 2024-09-21T15:01:59.523 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.523 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.523 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:59.523 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224218+0000", 2024-09-21T15:01:59.523 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0209e-05 2024-09-21T15:01:59.523 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.523 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.524 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:59.524 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.258839+0000", 2024-09-21T15:01:59.524 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.034621083 2024-09-21T15:01:59.524 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.524 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.524 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:59.524 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.259003+0000", 2024-09-21T15:01:59.524 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016379 2024-09-21T15:01:59.524 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:59.524 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:59.524 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:59.525 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.525 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.525 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139900 3.1e 3:7b440264:::rbd_data.113664780c85.0000000000000013:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:59.525 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.225748+0000", 2024-09-21T15:01:59.525 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 57.250847833000002, 2024-09-21T15:01:59.525 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0339966009999999, 2024-09-21T15:01:59.525 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:59.525 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:59.525 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:59.525 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:59.525 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:59.525 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139900 2024-09-21T15:01:59.526 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.526 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:59.526 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.526 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:59.526 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.225748+0000", 2024-09-21T15:01:59.526 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:59.526 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.526 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.526 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:59.526 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.225748+0000", 2024-09-21T15:01:59.526 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:59.526 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.527 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.527 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:59.527 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.225746+0000", 2024-09-21T15:01:59.527 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:59.527 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.527 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.527 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:59.527 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.225757+0000", 2024-09-21T15:01:59.527 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.066e-05 2024-09-21T15:01:59.527 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.527 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.527 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:59.527 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.225775+0000", 2024-09-21T15:01:59.527 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7631000000000001e-05 2024-09-21T15:01:59.528 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.528 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.528 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:59.528 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.225788+0000", 2024-09-21T15:01:59.528 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3046e-05 2024-09-21T15:01:59.528 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.528 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.528 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:59.528 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.225881+0000", 2024-09-21T15:01:59.528 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2987999999999998e-05 2024-09-21T15:01:59.528 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.528 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.528 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:59.528 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.225893+0000", 2024-09-21T15:01:59.528 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2398999999999999e-05 2024-09-21T15:01:59.529 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.529 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.529 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:59.529 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.259207+0000", 2024-09-21T15:01:59.529 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.033313696 2024-09-21T15:01:59.529 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.529 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.529 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:59.529 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.259452+0000", 2024-09-21T15:01:59.529 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00024495700000000002 2024-09-21T15:01:59.529 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.529 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.529 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:59.529 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.259745+0000", 2024-09-21T15:01:59.530 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000292868 2024-09-21T15:01:59.530 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:59.530 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:59.530 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:59.530 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.530 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.530 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140008 3.1e 3:7cd8b476:::rbd_data.113664780c85.0000000000000149:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:59.530 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.229555+0000", 2024-09-21T15:01:59.530 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 57.247040968999997, 2024-09-21T15:01:59.530 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0304914199999999, 2024-09-21T15:01:59.530 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:59.530 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:59.531 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:59.531 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:59.531 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:59.531 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140008 2024-09-21T15:01:59.531 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.531 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:59.531 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.531 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:59.531 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.229555+0000", 2024-09-21T15:01:59.531 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:59.531 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.531 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.531 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:59.531 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.229555+0000", 2024-09-21T15:01:59.531 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:59.532 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.532 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.532 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:59.532 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.229554+0000", 2024-09-21T15:01:59.532 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:59.532 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.532 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.532 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:59.532 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.229559+0000", 2024-09-21T15:01:59.532 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1059999999999999e-06 2024-09-21T15:01:59.532 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.532 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.532 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:59.532 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.229570+0000", 2024-09-21T15:01:59.532 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0465000000000001e-05 2024-09-21T15:01:59.532 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.533 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.533 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:59.533 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.229581+0000", 2024-09-21T15:01:59.533 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0958000000000001e-05 2024-09-21T15:01:59.533 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.533 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.533 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:59.533 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.229660+0000", 2024-09-21T15:01:59.533 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9103999999999993e-05 2024-09-21T15:01:59.533 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.533 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.533 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:59.533 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.229675+0000", 2024-09-21T15:01:59.533 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5622999999999999e-05 2024-09-21T15:01:59.533 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.534 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.534 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:59.534 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.259816+0000", 2024-09-21T15:01:59.534 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.030140595 2024-09-21T15:01:59.534 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.534 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.534 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:59.534 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.260046+0000", 2024-09-21T15:01:59.534 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00023045300000000001 2024-09-21T15:01:59.534 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:59.534 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:59.534 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:59.534 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.534 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.535 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140037 3.1e 3:7a4585d9:::rbd_data.113664780c85.000000000000005c:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:59.535 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.230439+0000", 2024-09-21T15:01:59.535 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 57.246156481, 2024-09-21T15:01:59.535 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.030277613, 2024-09-21T15:01:59.535 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:59.535 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:59.535 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:59.535 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:59.535 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:59.535 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140037 2024-09-21T15:01:59.535 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.535 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:59.535 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.535 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:59.536 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.230439+0000", 2024-09-21T15:01:59.536 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:59.536 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.536 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.536 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:59.536 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.230439+0000", 2024-09-21T15:01:59.536 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:59.536 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.536 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.536 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:59.536 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.230438+0000", 2024-09-21T15:01:59.536 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:59.536 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.536 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.536 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:59.536 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.230447+0000", 2024-09-21T15:01:59.537 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6270000000000006e-06 2024-09-21T15:01:59.537 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.537 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.537 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:59.537 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.230464+0000", 2024-09-21T15:01:59.537 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7394e-05 2024-09-21T15:01:59.537 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.537 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.537 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:59.537 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.230472+0000", 2024-09-21T15:01:59.537 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0360000000000006e-06 2024-09-21T15:01:59.537 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.537 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.537 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:59.537 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.230567+0000", 2024-09-21T15:01:59.538 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4648000000000006e-05 2024-09-21T15:01:59.538 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.538 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.538 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:59.538 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.230581+0000", 2024-09-21T15:01:59.538 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3799e-05 2024-09-21T15:01:59.538 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.538 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.538 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:59.538 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.260095+0000", 2024-09-21T15:01:59.538 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.029514153 2024-09-21T15:01:59.538 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.538 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.538 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:59.539 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.260356+0000", 2024-09-21T15:01:59.539 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00026118400000000003 2024-09-21T15:01:59.539 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.539 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.539 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:59.539 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.260717+0000", 2024-09-21T15:01:59.539 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00036124799999999999 2024-09-21T15:01:59.539 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:59.539 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:59.539 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:59.539 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.539 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.539 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140061 3.1e 3:7fdd08f8:::rbd_data.113664780c85.0000000000000104:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:59.539 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.232026+0000", 2024-09-21T15:01:59.540 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 57.244570150999998, 2024-09-21T15:01:59.540 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0290620450000001, 2024-09-21T15:01:59.540 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:59.540 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:59.540 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:59.540 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:59.540 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:59.540 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140061 2024-09-21T15:01:59.540 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.540 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:59.540 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.540 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:59.540 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232026+0000", 2024-09-21T15:01:59.540 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:59.541 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.541 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.541 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:59.541 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232026+0000", 2024-09-21T15:01:59.541 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:59.541 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.541 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.541 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:59.541 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232024+0000", 2024-09-21T15:01:59.541 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:01:59.541 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.541 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.541 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:59.541 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232034+0000", 2024-09-21T15:01:59.541 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0421000000000001e-05 2024-09-21T15:01:59.542 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.542 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.542 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:59.542 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232058+0000", 2024-09-21T15:01:59.542 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3944e-05 2024-09-21T15:01:59.542 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.542 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.542 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:59.542 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232071+0000", 2024-09-21T15:01:59.542 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2816999999999999e-05 2024-09-21T15:01:59.542 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.542 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.542 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:59.542 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232180+0000", 2024-09-21T15:01:59.543 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000109323 2024-09-21T15:01:59.543 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.543 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.543 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:59.543 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232199+0000", 2024-09-21T15:01:59.543 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8606000000000001e-05 2024-09-21T15:01:59.543 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.543 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.543 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:59.543 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.260821+0000", 2024-09-21T15:01:59.543 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0286222270000001 2024-09-21T15:01:59.543 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.543 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.543 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:59.544 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.261088+0000", 2024-09-21T15:01:59.544 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00026663799999999998 2024-09-21T15:01:59.544 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:59.544 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:59.544 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:59.544 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.544 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.544 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140090 3.1e 3:79f99893:::rbd_data.113664780c85.000000000000000b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:59.544 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.233668+0000", 2024-09-21T15:01:59.544 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 57.242928286000001, 2024-09-21T15:01:59.544 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0281047409999999, 2024-09-21T15:01:59.544 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:59.544 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:59.545 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:59.545 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:59.545 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:59.545 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140090 2024-09-21T15:01:59.545 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.545 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:59.545 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.545 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:59.545 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233668+0000", 2024-09-21T15:01:59.545 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:59.545 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.545 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.545 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:59.545 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233668+0000", 2024-09-21T15:01:59.545 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:59.546 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.546 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.546 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:59.546 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233666+0000", 2024-09-21T15:01:59.546 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:59.546 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.546 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.546 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:59.546 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233673+0000", 2024-09-21T15:01:59.546 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1840000000000002e-06 2024-09-21T15:01:59.546 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.546 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.547 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:59.547 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233689+0000", 2024-09-21T15:01:59.547 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5366000000000002e-05 2024-09-21T15:01:59.547 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.547 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.547 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:59.547 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233697+0000", 2024-09-21T15:01:59.547 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7679999999999997e-06 2024-09-21T15:01:59.547 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.547 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.547 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:59.547 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233772+0000", 2024-09-21T15:01:59.547 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.4832000000000005e-05 2024-09-21T15:01:59.547 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.547 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.548 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:59.548 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233790+0000", 2024-09-21T15:01:59.548 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7297999999999999e-05 2024-09-21T15:01:59.548 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.548 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.548 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:59.548 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.261145+0000", 2024-09-21T15:01:59.548 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.027355381 2024-09-21T15:01:59.548 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.548 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.548 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:59.548 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.261466+0000", 2024-09-21T15:01:59.548 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00032105300000000002 2024-09-21T15:01:59.549 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.549 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.549 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:59.549 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.261772+0000", 2024-09-21T15:01:59.549 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00030635100000000001 2024-09-21T15:01:59.549 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:59.549 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:59.549 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:59.549 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.549 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.549 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140128 3.1e 3:78baf82e:::rbd_data.113664780c85.0000000000000115:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:59.549 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.235029+0000", 2024-09-21T15:01:59.549 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 57.241567031999999, 2024-09-21T15:01:59.549 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.02698942, 2024-09-21T15:01:59.550 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:59.550 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:59.550 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:59.550 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:59.550 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:59.550 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140128 2024-09-21T15:01:59.550 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.550 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:59.550 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.550 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:59.550 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235029+0000", 2024-09-21T15:01:59.550 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:59.550 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.550 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.551 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:59.551 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235029+0000", 2024-09-21T15:01:59.551 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:59.551 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.551 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.551 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:59.551 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235028+0000", 2024-09-21T15:01:59.551 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:59.551 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.551 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.551 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:59.551 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235032+0000", 2024-09-21T15:01:59.551 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.0579999999999999e-06 2024-09-21T15:01:59.551 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.551 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.552 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:59.552 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235040+0000", 2024-09-21T15:01:59.552 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2099999999999993e-06 2024-09-21T15:01:59.552 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.552 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.552 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:59.552 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235047+0000", 2024-09-21T15:01:59.552 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1099999999999999e-06 2024-09-21T15:01:59.552 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.552 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.552 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:59.552 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235123+0000", 2024-09-21T15:01:59.552 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6835999999999994e-05 2024-09-21T15:01:59.552 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.552 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.553 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:59.553 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235136+0000", 2024-09-21T15:01:59.553 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2962000000000001e-05 2024-09-21T15:01:59.553 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.553 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.553 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:59.553 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.261827+0000", 2024-09-21T15:01:59.553 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0266906149999999 2024-09-21T15:01:59.553 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.553 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.553 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:59.553 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.262018+0000", 2024-09-21T15:01:59.553 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019132400000000001 2024-09-21T15:01:59.554 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:59.554 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:59.554 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:59.554 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.554 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.554 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140217 3.1e 3:793dcf44:::rbd_data.113664780c85.0000000000000000:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:59.554 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.235497+0000", 2024-09-21T15:01:59.554 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 57.241098733000001, 2024-09-21T15:01:59.554 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0269053020000001, 2024-09-21T15:01:59.554 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:59.554 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:59.554 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:59.554 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:59.555 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:59.555 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140217 2024-09-21T15:01:59.555 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.555 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:59.555 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.555 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:59.555 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235497+0000", 2024-09-21T15:01:59.555 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:59.555 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.555 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.555 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:59.555 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235497+0000", 2024-09-21T15:01:59.555 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:59.555 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.556 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.556 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:59.556 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235496+0000", 2024-09-21T15:01:59.556 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:59.556 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.556 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.556 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:59.556 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235503+0000", 2024-09-21T15:01:59.556 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8460000000000002e-06 2024-09-21T15:01:59.556 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.556 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.556 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:59.556 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235514+0000", 2024-09-21T15:01:59.556 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0576e-05 2024-09-21T15:01:59.557 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.557 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.557 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:59.557 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235522+0000", 2024-09-21T15:01:59.557 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2640000000000008e-06 2024-09-21T15:01:59.557 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.557 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.557 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:59.557 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235579+0000", 2024-09-21T15:01:59.557 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6842999999999998e-05 2024-09-21T15:01:59.557 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.557 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.557 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:59.558 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235590+0000", 2024-09-21T15:01:59.558 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1194e-05 2024-09-21T15:01:59.558 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.558 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.558 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:59.558 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.262119+0000", 2024-09-21T15:01:59.558 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0265294540000001 2024-09-21T15:01:59.558 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.558 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.558 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:59.558 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.262241+0000", 2024-09-21T15:01:59.558 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012182 2024-09-21T15:01:59.558 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.558 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.559 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:59.559 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.262403+0000", 2024-09-21T15:01:59.559 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000161442 2024-09-21T15:01:59.559 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:59.559 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:59.559 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:59.559 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.559 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.559 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140221 3.1e 3:7fa3054f:::rbd_data.113664780c85.000000000000001c:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:59.559 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.235592+0000", 2024-09-21T15:01:59.559 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 57.241003872, 2024-09-21T15:01:59.559 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.027322012, 2024-09-21T15:01:59.559 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:59.559 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:59.560 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:59.560 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:59.560 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:59.560 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140221 2024-09-21T15:01:59.560 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.560 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:59.560 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.560 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:59.560 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235592+0000", 2024-09-21T15:01:59.560 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:59.560 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.560 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.560 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:59.560 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235592+0000", 2024-09-21T15:01:59.561 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:59.561 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.561 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.561 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:59.561 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235591+0000", 2024-09-21T15:01:59.561 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:59.561 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.561 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.561 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:59.561 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235596+0000", 2024-09-21T15:01:59.561 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.4209999999999997e-06 2024-09-21T15:01:59.561 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.561 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.561 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:59.561 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235603+0000", 2024-09-21T15:01:59.562 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.909e-06 2024-09-21T15:01:59.562 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.562 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.562 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:59.562 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235609+0000", 2024-09-21T15:01:59.562 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.3539999999999999e-06 2024-09-21T15:01:59.562 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.562 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.562 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:59.562 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236013+0000", 2024-09-21T15:01:59.562 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00040399799999999998 2024-09-21T15:01:59.562 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.562 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.562 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:59.563 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236020+0000", 2024-09-21T15:01:59.563 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6699999999999994e-06 2024-09-21T15:01:59.563 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.563 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.563 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:59.563 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.262452+0000", 2024-09-21T15:01:59.563 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.02643145 2024-09-21T15:01:59.563 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.563 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.563 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:59.563 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.262624+0000", 2024-09-21T15:01:59.563 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000172206 2024-09-21T15:01:59.563 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.563 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.564 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:59.564 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.262914+0000", 2024-09-21T15:01:59.564 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00028994600000000002 2024-09-21T15:01:59.564 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:59.564 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:59.564 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:59.564 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.564 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.564 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140237 3.1e 3:7c803367:::rbd_data.113664780c85.000000000000008c:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:59.564 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.237113+0000", 2024-09-21T15:01:59.564 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 57.239483364999998, 2024-09-21T15:01:59.564 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0260336299999999, 2024-09-21T15:01:59.564 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:59.565 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:59.565 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:59.565 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:59.565 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:59.565 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140237 2024-09-21T15:01:59.565 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.565 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:59.565 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.565 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:59.565 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237113+0000", 2024-09-21T15:01:59.565 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:59.565 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.565 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.565 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:59.566 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237113+0000", 2024-09-21T15:01:59.566 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:59.566 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.566 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.566 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:59.566 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237112+0000", 2024-09-21T15:01:59.566 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:59.566 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.566 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.566 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:59.566 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237118+0000", 2024-09-21T15:01:59.566 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8930000000000002e-06 2024-09-21T15:01:59.566 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.566 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.567 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:59.567 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237126+0000", 2024-09-21T15:01:59.567 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6729999999999998e-06 2024-09-21T15:01:59.567 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.567 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.567 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:59.567 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237135+0000", 2024-09-21T15:01:59.567 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0929999999999992e-06 2024-09-21T15:01:59.567 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.567 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.567 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:59.567 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237205+0000", 2024-09-21T15:01:59.567 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9697999999999996e-05 2024-09-21T15:01:59.567 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.567 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.568 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:59.568 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237218+0000", 2024-09-21T15:01:59.568 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2524000000000001e-05 2024-09-21T15:01:59.568 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.568 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.568 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:59.568 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.262965+0000", 2024-09-21T15:01:59.568 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0257478630000001 2024-09-21T15:01:59.568 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.568 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.568 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:59.568 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.263146+0000", 2024-09-21T15:01:59.568 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018074599999999999 2024-09-21T15:01:59.568 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:59.568 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:59.569 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:59.569 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.569 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.569 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140280 3.1e 3:7b675eeb:::rbd_data.113664780c85.00000000000001b3:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:59.569 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.237477+0000", 2024-09-21T15:01:59.569 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 57.239119103, 2024-09-21T15:01:59.569 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.026437329, 2024-09-21T15:01:59.569 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:59.569 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:59.569 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:59.569 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:59.569 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:59.569 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140280 2024-09-21T15:01:59.569 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.569 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:59.570 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.570 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:59.570 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237477+0000", 2024-09-21T15:01:59.570 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:59.570 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.570 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.570 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:59.570 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237477+0000", 2024-09-21T15:01:59.570 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:59.570 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.570 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.570 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:59.570 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237476+0000", 2024-09-21T15:01:59.570 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:59.571 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.571 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.571 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:59.571 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237480+0000", 2024-09-21T15:01:59.571 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7570000000000002e-06 2024-09-21T15:01:59.571 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.571 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.571 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:59.571 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237490+0000", 2024-09-21T15:01:59.571 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3170000000000008e-06 2024-09-21T15:01:59.571 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.571 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.571 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:59.572 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237496+0000", 2024-09-21T15:01:59.572 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7150000000000001e-06 2024-09-21T15:01:59.572 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.572 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.572 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:59.572 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237559+0000", 2024-09-21T15:01:59.572 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.2844999999999999e-05 2024-09-21T15:01:59.572 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.572 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.572 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:59.572 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.237570+0000", 2024-09-21T15:01:59.572 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1253000000000001e-05 2024-09-21T15:01:59.572 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.572 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.572 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:59.573 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.263203+0000", 2024-09-21T15:01:59.573 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0256330339999999 2024-09-21T15:01:59.573 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.573 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.573 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:59.573 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.263914+0000", 2024-09-21T15:01:59.573 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00071080899999999996 2024-09-21T15:01:59.573 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:59.573 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:59.573 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:59.573 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.573 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.573 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140348 3.1e 3:7b440264:::rbd_data.113664780c85.0000000000000013:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:59.573 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.241762+0000", 2024-09-21T15:01:59.574 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 57.234833832, 2024-09-21T15:01:59.574 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0221355759999999, 2024-09-21T15:01:59.574 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:59.574 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:59.574 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:59.574 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:59.574 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:59.574 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140348 2024-09-21T15:01:59.574 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.575 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:59.575 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.575 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:59.575 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.241762+0000", 2024-09-21T15:01:59.575 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:59.575 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.575 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.575 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:59.575 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.241762+0000", 2024-09-21T15:01:59.575 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:59.576 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.576 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.576 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:59.576 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.241761+0000", 2024-09-21T15:01:59.576 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:59.576 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.576 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.576 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:59.576 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.241767+0000", 2024-09-21T15:01:59.576 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4219999999999997e-06 2024-09-21T15:01:59.576 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.577 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.577 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:59.577 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.241783+0000", 2024-09-21T15:01:59.577 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5959000000000001e-05 2024-09-21T15:01:59.577 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.577 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.577 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:59.577 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.241793+0000", 2024-09-21T15:01:59.577 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3999999999999998e-06 2024-09-21T15:01:59.577 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.577 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.578 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:59.578 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.241894+0000", 2024-09-21T15:01:59.578 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000100976 2024-09-21T15:01:59.578 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.578 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.578 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:59.578 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.241907+0000", 2024-09-21T15:01:59.578 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3281e-05 2024-09-21T15:01:59.578 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.578 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.578 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:59.578 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.263430+0000", 2024-09-21T15:01:59.578 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0215229379999999 2024-09-21T15:01:59.578 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.578 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.579 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:59.579 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.263604+0000", 2024-09-21T15:01:59.579 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017368700000000001 2024-09-21T15:01:59.579 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.579 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.579 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:59.579 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.263898+0000", 2024-09-21T15:01:59.579 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00029406899999999999 2024-09-21T15:01:59.579 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:59.579 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:59.579 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:59.579 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:59.580 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:59.580 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:01:59.581 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T15:01:59.582 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:01:59.577+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:01:59.682 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:01:59.681+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:01:59.693 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:01:59.693 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:01:59.693 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T15:01:59.693 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:01:59.703 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T15:01:59.783 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:01:59.781+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:01:59.808 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:01:59.808 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:01:59.808 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T15:01:59.808 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T15:01:59.808 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:01:59.819 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T15:01:59.883 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:01:59.880+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:01:59.923 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:01:59.923 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T15:01:59.923 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T15:01:59.923 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T15:01:59.923 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.923 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:97677 3.15 e122/24 3:a8e72133:::rbd_data.1136bfc79a70.0000000000000005:head v 122'567, mlcod=120'565)", 2024-09-21T15:01:59.923 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.320753+0000", 2024-09-21T15:01:59.923 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 114.598607089, 2024-09-21T15:01:59.923 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070639375000000004, 2024-09-21T15:01:59.924 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:59.924 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:59.924 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:59.924 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.924 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:59.924 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.320753+0000", 2024-09-21T15:01:59.924 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:59.924 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.924 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.924 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:59.924 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.320753+0000", 2024-09-21T15:01:59.924 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:59.924 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.925 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.925 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:59.925 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.320753+0000", 2024-09-21T15:01:59.925 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:59.925 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.925 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.925 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:59.925 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.375452+0000", 2024-09-21T15:01:59.925 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054698945999999998 2024-09-21T15:01:59.925 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.925 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.925 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:59.925 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.375476+0000", 2024-09-21T15:01:59.926 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4380999999999999e-05 2024-09-21T15:01:59.926 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.926 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.926 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:59.926 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.375485+0000", 2024-09-21T15:01:59.926 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0049999999999993e-06 2024-09-21T15:01:59.926 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.926 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.926 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:59.926 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.375552+0000", 2024-09-21T15:01:59.926 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6550999999999994e-05 2024-09-21T15:01:59.926 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.926 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.926 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:59.927 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.375578+0000", 2024-09-21T15:01:59.927 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6196000000000001e-05 2024-09-21T15:01:59.927 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.927 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.927 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:01:59.927 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.391319+0000", 2024-09-21T15:01:59.927 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015741271000000001 2024-09-21T15:01:59.927 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.927 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.927 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:59.927 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.391393+0000", 2024-09-21T15:01:59.927 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3287000000000005e-05 2024-09-21T15:01:59.927 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:59.927 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:59.928 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:59.928 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.928 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.928 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:97672 3.4 e122/51 3:26046d29:::rbd_data.1136bfc79a70.0000000000000001:head v 122'256, mlcod=120'254)", 2024-09-21T15:01:59.928 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.331696+0000", 2024-09-21T15:01:59.928 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 114.587663926, 2024-09-21T15:01:59.928 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041222507999999998, 2024-09-21T15:01:59.928 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:59.928 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:01:59.928 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:59.928 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.928 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:59.928 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.331696+0000", 2024-09-21T15:01:59.928 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:59.929 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.929 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.929 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:59.929 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.331696+0000", 2024-09-21T15:01:59.929 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:59.929 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.929 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.929 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:59.929 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.331695+0000", 2024-09-21T15:01:59.929 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:59.929 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.929 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.929 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:59.929 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.340175+0000", 2024-09-21T15:01:59.930 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0084799520000000007 2024-09-21T15:01:59.930 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.930 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.930 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:59.930 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.340208+0000", 2024-09-21T15:01:59.930 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2997999999999998e-05 2024-09-21T15:01:59.930 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.930 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.930 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:59.930 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.340223+0000", 2024-09-21T15:01:59.930 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5330999999999999e-05 2024-09-21T15:01:59.930 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.930 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.931 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:59.931 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.340305+0000", 2024-09-21T15:01:59.931 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2095999999999995e-05 2024-09-21T15:01:59.931 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.931 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.931 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:59.931 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.340337+0000", 2024-09-21T15:01:59.931 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2097000000000001e-05 2024-09-21T15:01:59.931 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.931 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.931 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:01:59.931 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.372863+0000", 2024-09-21T15:01:59.931 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.032525923999999998 2024-09-21T15:01:59.931 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.932 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.932 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:59.932 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.372919+0000", 2024-09-21T15:01:59.932 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5729000000000001e-05 2024-09-21T15:01:59.932 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:59.932 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:59.932 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:59.932 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.932 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.932 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:116182 3.12 3:4efbaa3f:::rbd_data.1136b1194d20.0000000000000006:58 [read 6242304~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:01:59.932 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.712538+0000", 2024-09-21T15:01:59.932 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 90.206821934999994, 2024-09-21T15:01:59.932 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034702127999999999, 2024-09-21T15:01:59.932 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:59.932 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:59.933 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:59.933 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:59.933 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:59.933 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116182 2024-09-21T15:01:59.933 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.933 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:59.933 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.933 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:59.933 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712538+0000", 2024-09-21T15:01:59.933 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:59.933 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.933 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.933 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:59.934 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712538+0000", 2024-09-21T15:01:59.934 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:59.934 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.934 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.934 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:59.934 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712538+0000", 2024-09-21T15:01:59.934 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:59.934 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.934 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.934 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:59.934 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712541+0000", 2024-09-21T15:01:59.934 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5330000000000001e-06 2024-09-21T15:01:59.934 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.935 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.935 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:59.935 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712547+0000", 2024-09-21T15:01:59.935 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1109999999999996e-06 2024-09-21T15:01:59.935 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.935 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.935 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:59.935 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712551+0000", 2024-09-21T15:01:59.935 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.685e-06 2024-09-21T15:01:59.935 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.935 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.935 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:59.935 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.747067+0000", 2024-09-21T15:01:59.935 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034516181 2024-09-21T15:01:59.936 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.936 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.936 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:59.936 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.747152+0000", 2024-09-21T15:01:59.936 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4075999999999999e-05 2024-09-21T15:01:59.936 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.936 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.936 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:59.936 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.747240+0000", 2024-09-21T15:01:59.936 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8924000000000004e-05 2024-09-21T15:01:59.936 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:59.936 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:59.936 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:59.936 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.937 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.937 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139784 3.1 3:862e0e65:::rbd_data.113664780c85.0000000000000063:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:59.937 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.222486+0000", 2024-09-21T15:01:59.937 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 57.696874068, 2024-09-21T15:01:59.937 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0141793059999999, 2024-09-21T15:01:59.937 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:59.937 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:59.937 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:59.937 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:59.937 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:59.937 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139784 2024-09-21T15:01:59.937 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.937 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:59.937 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.938 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:59.938 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222486+0000", 2024-09-21T15:01:59.938 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:59.938 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.938 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.938 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:59.938 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222486+0000", 2024-09-21T15:01:59.938 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:59.938 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.938 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.938 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:59.938 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222484+0000", 2024-09-21T15:01:59.938 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-21T15:01:59.938 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.939 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.939 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:59.939 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222494+0000", 2024-09-21T15:01:59.939 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0676000000000001e-05 2024-09-21T15:01:59.939 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.939 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.939 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:59.939 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222517+0000", 2024-09-21T15:01:59.939 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2371e-05 2024-09-21T15:01:59.939 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.939 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.939 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:59.939 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222527+0000", 2024-09-21T15:01:59.939 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0461e-05 2024-09-21T15:01:59.940 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.940 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.940 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:59.940 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222625+0000", 2024-09-21T15:01:59.940 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.8160999999999997e-05 2024-09-21T15:01:59.940 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.940 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.940 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:59.940 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222641+0000", 2024-09-21T15:01:59.940 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5899000000000001e-05 2024-09-21T15:01:59.940 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.940 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.940 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:59.940 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.236479+0000", 2024-09-21T15:01:59.940 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0138377709999999 2024-09-21T15:01:59.941 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.941 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.941 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:59.941 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.236666+0000", 2024-09-21T15:01:59.941 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018648199999999999 2024-09-21T15:01:59.941 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:59.941 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:59.941 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:59.941 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.941 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.941 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139812 3.1 3:8407fb9b:::rbd_data.113664780c85.0000000000000127:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:59.941 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.222547+0000", 2024-09-21T15:01:59.941 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 57.696813144000004, 2024-09-21T15:01:59.941 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.014796059, 2024-09-21T15:01:59.942 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:59.942 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:59.942 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:59.942 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:59.942 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:59.942 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139812 2024-09-21T15:01:59.942 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.942 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:59.942 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.942 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:59.942 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222547+0000", 2024-09-21T15:01:59.942 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:59.942 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.942 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.943 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:59.943 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222547+0000", 2024-09-21T15:01:59.943 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:59.943 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.943 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.943 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:59.943 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222546+0000", 2024-09-21T15:01:59.943 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:59.943 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.943 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.943 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:59.943 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222552+0000", 2024-09-21T15:01:59.943 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2739999999999998e-06 2024-09-21T15:01:59.944 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.944 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.944 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:59.944 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222578+0000", 2024-09-21T15:01:59.944 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6594999999999999e-05 2024-09-21T15:01:59.944 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.944 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.944 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:59.944 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222596+0000", 2024-09-21T15:01:59.944 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7838000000000001e-05 2024-09-21T15:01:59.944 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.944 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.944 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:59.944 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222701+0000", 2024-09-21T15:01:59.944 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000104714 2024-09-21T15:01:59.945 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.945 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.945 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:59.945 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222714+0000", 2024-09-21T15:01:59.945 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2998e-05 2024-09-21T15:01:59.945 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.945 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.945 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:59.945 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.237124+0000", 2024-09-21T15:01:59.945 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.014409806 2024-09-21T15:01:59.945 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.945 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.945 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:59.945 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.237343+0000", 2024-09-21T15:01:59.945 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021950499999999999 2024-09-21T15:01:59.946 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:59.946 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:59.946 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:59.946 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.946 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.946 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139826 3.1 3:854f6f9b:::rbd_data.113664780c85.0000000000000189:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:59.946 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.222611+0000", 2024-09-21T15:01:59.946 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 57.696749052000001, 2024-09-21T15:01:59.946 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.014916828, 2024-09-21T15:01:59.946 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:59.946 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:59.946 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:59.946 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:59.946 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:59.947 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139826 2024-09-21T15:01:59.947 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.947 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:59.947 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.947 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:59.947 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222611+0000", 2024-09-21T15:01:59.947 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:59.947 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.947 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.947 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:59.947 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222611+0000", 2024-09-21T15:01:59.947 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:59.947 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.947 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.948 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:59.948 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222610+0000", 2024-09-21T15:01:59.948 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:59.948 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.948 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.948 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:59.948 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222618+0000", 2024-09-21T15:01:59.948 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6690000000000006e-06 2024-09-21T15:01:59.948 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.948 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.948 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:59.948 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222632+0000", 2024-09-21T15:01:59.948 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4433999999999999e-05 2024-09-21T15:01:59.948 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.948 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.949 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:59.949 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222640+0000", 2024-09-21T15:01:59.949 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6260000000000003e-06 2024-09-21T15:01:59.949 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.949 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.949 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:59.949 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222758+0000", 2024-09-21T15:01:59.949 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011826499999999999 2024-09-21T15:01:59.949 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.949 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.949 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:59.949 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222776+0000", 2024-09-21T15:01:59.949 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7564999999999999e-05 2024-09-21T15:01:59.949 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.950 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.950 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:59.950 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.237402+0000", 2024-09-21T15:01:59.950 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0146261889999999 2024-09-21T15:01:59.950 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.950 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.950 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:59.950 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.237528+0000", 2024-09-21T15:01:59.950 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000126212 2024-09-21T15:01:59.950 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:59.950 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:59.950 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:59.950 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.950 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.950 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139864 3.1 3:86b14cb7:::rbd_data.113664780c85.00000000000000d5:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:59.951 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.223966+0000", 2024-09-21T15:01:59.951 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 57.695394315999998, 2024-09-21T15:01:59.951 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.013859007, 2024-09-21T15:01:59.951 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:59.951 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:59.951 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:59.951 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:59.951 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:59.951 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139864 2024-09-21T15:01:59.951 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.951 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:59.951 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.951 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:59.951 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223966+0000", 2024-09-21T15:01:59.952 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:59.952 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.952 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.952 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:59.952 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223966+0000", 2024-09-21T15:01:59.952 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:59.952 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.952 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.952 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:59.952 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223965+0000", 2024-09-21T15:01:59.952 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:59.952 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.952 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.952 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:59.952 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223970+0000", 2024-09-21T15:01:59.952 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3769999999999998e-06 2024-09-21T15:01:59.953 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.953 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.953 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:59.953 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223979+0000", 2024-09-21T15:01:59.953 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5899999999999997e-06 2024-09-21T15:01:59.953 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.953 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.953 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:59.953 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223985+0000", 2024-09-21T15:01:59.953 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5790000000000003e-06 2024-09-21T15:01:59.953 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.953 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.953 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:59.953 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224073+0000", 2024-09-21T15:01:59.953 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7781000000000005e-05 2024-09-21T15:01:59.954 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.954 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.954 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:59.954 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224110+0000", 2024-09-21T15:01:59.954 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7060000000000001e-05 2024-09-21T15:01:59.954 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.954 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.954 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:59.954 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.237599+0000", 2024-09-21T15:01:59.954 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.01348953 2024-09-21T15:01:59.954 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.954 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.954 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:59.954 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.237825+0000", 2024-09-21T15:01:59.955 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000225758 2024-09-21T15:01:59.955 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:59.955 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:59.955 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:59.955 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.955 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.955 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139877 3.1 3:87277f92:::rbd_data.113664780c85.0000000000000136:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:59.955 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.224489+0000", 2024-09-21T15:01:59.955 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 57.694871458000001, 2024-09-21T15:01:59.955 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0137916330000001, 2024-09-21T15:01:59.955 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:59.955 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:59.955 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:59.955 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:59.956 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:59.956 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139877 2024-09-21T15:01:59.956 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.956 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:59.956 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.956 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:59.956 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224489+0000", 2024-09-21T15:01:59.956 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:59.956 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.956 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.956 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:59.956 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224489+0000", 2024-09-21T15:01:59.956 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:59.956 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.956 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.957 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:59.957 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224488+0000", 2024-09-21T15:01:59.957 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:59.957 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.957 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.957 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:59.957 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224502+0000", 2024-09-21T15:01:59.957 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4194e-05 2024-09-21T15:01:59.957 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.957 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.957 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:59.957 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224522+0000", 2024-09-21T15:01:59.957 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0038e-05 2024-09-21T15:01:59.957 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.958 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.958 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:59.958 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224531+0000", 2024-09-21T15:01:59.958 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6900000000000004e-06 2024-09-21T15:01:59.958 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.958 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.958 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:59.958 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224621+0000", 2024-09-21T15:01:59.958 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9654000000000006e-05 2024-09-21T15:01:59.958 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.958 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.958 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:59.958 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224641+0000", 2024-09-21T15:01:59.959 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9375e-05 2024-09-21T15:01:59.959 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.959 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.959 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:59.959 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.238027+0000", 2024-09-21T15:01:59.959 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0133866840000001 2024-09-21T15:01:59.959 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.959 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.959 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:59.959 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.238280+0000", 2024-09-21T15:01:59.959 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000253281 2024-09-21T15:01:59.959 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:59.959 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:59.959 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:59.960 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.960 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.960 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139887 3.1 3:80f1e8f1:::rbd_data.113664780c85.000000000000017c:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:59.960 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.224549+0000", 2024-09-21T15:01:59.960 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 57.694811168000001, 2024-09-21T15:01:59.960 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0139776060000001, 2024-09-21T15:01:59.960 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:59.960 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:59.960 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:59.960 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:59.960 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:59.960 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139887 2024-09-21T15:01:59.960 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.960 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:59.960 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.961 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:59.961 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224549+0000", 2024-09-21T15:01:59.961 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:59.961 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.961 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.961 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:59.961 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224549+0000", 2024-09-21T15:01:59.961 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:59.961 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.961 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.961 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:59.961 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224548+0000", 2024-09-21T15:01:59.961 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:59.961 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.962 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.962 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:59.962 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224554+0000", 2024-09-21T15:01:59.962 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4049999999999999e-06 2024-09-21T15:01:59.962 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.962 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.962 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:59.962 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224565+0000", 2024-09-21T15:01:59.962 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0879e-05 2024-09-21T15:01:59.962 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.962 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.962 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:59.962 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224572+0000", 2024-09-21T15:01:59.962 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0380000000000002e-06 2024-09-21T15:01:59.963 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.963 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.963 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:59.963 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224684+0000", 2024-09-21T15:01:59.963 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011229 2024-09-21T15:01:59.963 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.963 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.963 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:59.963 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224698+0000", 2024-09-21T15:01:59.963 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4448e-05 2024-09-21T15:01:59.963 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.963 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.963 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:59.963 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.238359+0000", 2024-09-21T15:01:59.963 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0136610660000001 2024-09-21T15:01:59.964 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.964 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.964 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:59.964 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.238527+0000", 2024-09-21T15:01:59.964 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016731100000000001 2024-09-21T15:01:59.964 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:59.964 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:59.964 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:59.964 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.964 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.964 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140077 3.1 3:85642daf:::rbd_data.113664780c85.0000000000000174:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:59.964 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.232797+0000", 2024-09-21T15:01:59.964 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 57.686563452999998, 2024-09-21T15:01:59.964 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.006029955, 2024-09-21T15:01:59.965 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:59.965 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:59.965 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:59.965 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:59.965 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:59.965 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140077 2024-09-21T15:01:59.965 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.965 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:59.965 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.965 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:59.965 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232797+0000", 2024-09-21T15:01:59.965 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:59.965 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.965 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.966 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:59.966 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232797+0000", 2024-09-21T15:01:59.966 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:59.966 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.966 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.966 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:59.966 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232795+0000", 2024-09-21T15:01:59.966 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:59.966 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.966 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.966 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:59.966 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232804+0000", 2024-09-21T15:01:59.966 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9670000000000007e-06 2024-09-21T15:01:59.966 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.966 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.967 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:59.967 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232824+0000", 2024-09-21T15:01:59.967 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9238000000000001e-05 2024-09-21T15:01:59.967 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.967 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.967 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:59.967 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232843+0000", 2024-09-21T15:01:59.967 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9256e-05 2024-09-21T15:01:59.967 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.967 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.967 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:59.967 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232912+0000", 2024-09-21T15:01:59.967 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9589000000000001e-05 2024-09-21T15:01:59.967 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.968 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.968 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:59.968 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232925+0000", 2024-09-21T15:01:59.968 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2585999999999999e-05 2024-09-21T15:01:59.968 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.968 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.968 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:59.968 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.238591+0000", 2024-09-21T15:01:59.968 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0056659939999999 2024-09-21T15:01:59.968 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.968 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.968 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:59.968 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.238827+0000", 2024-09-21T15:01:59.968 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00023583999999999999 2024-09-21T15:01:59.968 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:59.968 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:59.969 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:59.969 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.969 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.969 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140091 3.1 3:8358f0db:::rbd_data.113664780c85.0000000000000018:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:59.969 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.233637+0000", 2024-09-21T15:01:59.969 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 57.685723156999998, 2024-09-21T15:01:59.969 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.006272847, 2024-09-21T15:01:59.969 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:59.969 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:01:59.969 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:59.969 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:59.969 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:59.969 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140091 2024-09-21T15:01:59.969 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.970 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:59.970 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.970 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:59.970 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233637+0000", 2024-09-21T15:01:59.970 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:59.970 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.970 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.970 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:59.970 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233637+0000", 2024-09-21T15:01:59.970 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:59.970 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.970 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.970 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:59.970 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233636+0000", 2024-09-21T15:01:59.971 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:59.971 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.971 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.971 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:59.971 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233644+0000", 2024-09-21T15:01:59.971 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1380000000000007e-06 2024-09-21T15:01:59.971 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.971 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.971 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:59.971 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233662+0000", 2024-09-21T15:01:59.971 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8224999999999999e-05 2024-09-21T15:01:59.971 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.971 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.971 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:59.971 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233674+0000", 2024-09-21T15:01:59.972 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2081e-05 2024-09-21T15:01:59.972 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.972 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.972 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:59.972 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233750+0000", 2024-09-21T15:01:59.972 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6106000000000006e-05 2024-09-21T15:01:59.972 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.972 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.972 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:59.972 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233768+0000", 2024-09-21T15:01:59.972 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7285000000000001e-05 2024-09-21T15:01:59.972 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.972 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.972 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:59.973 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.239196+0000", 2024-09-21T15:01:59.973 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.005428223 2024-09-21T15:01:59.973 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.973 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.973 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:01:59.973 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.239528+0000", 2024-09-21T15:01:59.973 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00033262199999999997 2024-09-21T15:01:59.973 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.973 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.973 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:59.973 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.239910+0000", 2024-09-21T15:01:59.973 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00038151100000000001 2024-09-21T15:01:59.973 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:59.973 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:59.973 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:59.973 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.974 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.974 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140109 3.1 3:8586e9d9:::rbd_data.113664780c85.0000000000000096:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:59.974 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.234171+0000", 2024-09-21T15:01:59.974 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 57.685189323000003, 2024-09-21T15:01:59.974 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.006019896, 2024-09-21T15:01:59.974 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:59.974 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:59.974 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:59.974 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:59.974 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:59.974 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140109 2024-09-21T15:01:59.974 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.974 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:59.974 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.975 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:59.975 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.234171+0000", 2024-09-21T15:01:59.975 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:59.975 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.975 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.975 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:59.975 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.234171+0000", 2024-09-21T15:01:59.975 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:59.975 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.975 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.975 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:59.975 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.234170+0000", 2024-09-21T15:01:59.975 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:59.975 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.976 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.976 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:59.976 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.234178+0000", 2024-09-21T15:01:59.976 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.464e-06 2024-09-21T15:01:59.976 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.976 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.976 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:59.976 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.234191+0000", 2024-09-21T15:01:59.976 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3489000000000001e-05 2024-09-21T15:01:59.976 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.976 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.976 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:59.976 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.234200+0000", 2024-09-21T15:01:59.976 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2779999999999999e-06 2024-09-21T15:01:59.976 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.977 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.977 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:59.977 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.234279+0000", 2024-09-21T15:01:59.977 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8813000000000006e-05 2024-09-21T15:01:59.977 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.977 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.977 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:59.977 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.234298+0000", 2024-09-21T15:01:59.977 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8947999999999999e-05 2024-09-21T15:01:59.977 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.977 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.977 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:59.977 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.239974+0000", 2024-09-21T15:01:59.977 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0056760259999999 2024-09-21T15:01:59.978 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.978 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.978 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:59.978 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.240191+0000", 2024-09-21T15:01:59.978 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000216708 2024-09-21T15:01:59.978 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:59.978 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:59.978 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:59.978 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.978 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.978 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140149 3.1 3:86b1e433:::rbd_data.113664780c85.00000000000001ae:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:59.978 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.235545+0000", 2024-09-21T15:01:59.978 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 57.683815037999999, 2024-09-21T15:01:59.978 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0049193430000001, 2024-09-21T15:01:59.979 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:59.979 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:59.979 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:59.979 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:59.979 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:59.979 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140149 2024-09-21T15:01:59.979 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.979 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:59.979 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.979 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:59.979 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235545+0000", 2024-09-21T15:01:59.979 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:59.979 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.979 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.979 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:59.980 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235545+0000", 2024-09-21T15:01:59.980 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:59.980 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.980 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.980 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:59.980 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235544+0000", 2024-09-21T15:01:59.980 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:59.980 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.980 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.980 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:59.980 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235554+0000", 2024-09-21T15:01:59.980 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.8870000000000004e-06 2024-09-21T15:01:59.980 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.980 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.981 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:59.981 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235577+0000", 2024-09-21T15:01:59.981 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2946999999999999e-05 2024-09-21T15:01:59.981 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.981 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.981 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:59.981 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235587+0000", 2024-09-21T15:01:59.981 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0326000000000001e-05 2024-09-21T15:01:59.981 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.981 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.981 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:59.981 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235658+0000", 2024-09-21T15:01:59.981 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1360999999999997e-05 2024-09-21T15:01:59.981 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.981 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.982 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:59.982 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235670+0000", 2024-09-21T15:01:59.982 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1735e-05 2024-09-21T15:01:59.982 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.982 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.982 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:59.982 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.240246+0000", 2024-09-21T15:01:59.982 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0045757790000001 2024-09-21T15:01:59.982 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.982 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.982 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:59.982 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.240465+0000", 2024-09-21T15:01:59.982 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021870799999999999 2024-09-21T15:01:59.982 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:59.983 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:59.983 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:59.983 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.983 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.983 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140232 3.1 3:862e0e65:::rbd_data.113664780c85.0000000000000063:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:59.983 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.235760+0000", 2024-09-21T15:01:59.983 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 57.683600319999996, 2024-09-21T15:01:59.983 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.004979812, 2024-09-21T15:01:59.983 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:59.983 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:59.983 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:59.983 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:59.983 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:59.983 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140232 2024-09-21T15:01:59.983 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.984 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:59.984 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.984 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:59.984 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235760+0000", 2024-09-21T15:01:59.984 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:59.984 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.984 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.984 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:59.984 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235760+0000", 2024-09-21T15:01:59.984 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:59.984 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.984 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.984 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:59.984 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235759+0000", 2024-09-21T15:01:59.984 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:01:59.985 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.985 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.985 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:59.985 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235777+0000", 2024-09-21T15:01:59.985 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8215000000000001e-05 2024-09-21T15:01:59.985 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.985 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.985 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:59.985 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235793+0000", 2024-09-21T15:01:59.985 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6461e-05 2024-09-21T15:01:59.985 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.985 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.985 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:59.985 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235803+0000", 2024-09-21T15:01:59.986 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5300000000000002e-06 2024-09-21T15:01:59.986 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.986 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.986 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:59.986 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235875+0000", 2024-09-21T15:01:59.986 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2113000000000006e-05 2024-09-21T15:01:59.986 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.986 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.986 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:59.986 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235887+0000", 2024-09-21T15:01:59.986 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2160999999999999e-05 2024-09-21T15:01:59.986 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.986 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.987 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:59.987 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.240521+0000", 2024-09-21T15:01:59.987 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.004633428 2024-09-21T15:01:59.987 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.987 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.987 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:59.987 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.240740+0000", 2024-09-21T15:01:59.987 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000219228 2024-09-21T15:01:59.987 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:59.987 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:59.987 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:59.987 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.987 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.987 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140260 3.1 3:8407fb9b:::rbd_data.113664780c85.0000000000000127:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:59.987 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.235839+0000", 2024-09-21T15:01:59.988 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 57.683521151000001, 2024-09-21T15:01:59.988 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.005157981, 2024-09-21T15:01:59.988 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:59.988 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:59.988 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:59.988 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:59.988 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:59.988 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140260 2024-09-21T15:01:59.988 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.988 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:59.988 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.988 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:59.988 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235839+0000", 2024-09-21T15:01:59.988 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:59.988 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.988 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.989 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:59.989 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235839+0000", 2024-09-21T15:01:59.989 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:59.989 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.989 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.989 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:59.989 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235838+0000", 2024-09-21T15:01:59.989 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:59.989 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.989 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.989 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:59.989 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235843+0000", 2024-09-21T15:01:59.989 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.481e-06 2024-09-21T15:01:59.989 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.989 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.989 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:59.990 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235849+0000", 2024-09-21T15:01:59.990 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4880000000000004e-06 2024-09-21T15:01:59.990 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.990 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.990 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:59.990 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235856+0000", 2024-09-21T15:01:59.990 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7649999999999996e-06 2024-09-21T15:01:59.990 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.990 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.990 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:59.990 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235933+0000", 2024-09-21T15:01:59.990 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6987e-05 2024-09-21T15:01:59.990 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.990 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.990 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:59.991 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235949+0000", 2024-09-21T15:01:59.991 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6696000000000001e-05 2024-09-21T15:01:59.991 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.991 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.991 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:59.991 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.240823+0000", 2024-09-21T15:01:59.991 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.004873259 2024-09-21T15:01:59.991 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.991 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.991 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:59.991 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.240997+0000", 2024-09-21T15:01:59.991 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000174365 2024-09-21T15:01:59.991 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:59.991 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:59.991 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:59.992 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.992 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.992 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140274 3.1 3:854f6f9b:::rbd_data.113664780c85.0000000000000189:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:59.992 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.235884+0000", 2024-09-21T15:01:59.992 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 57.683476202999998, 2024-09-21T15:01:59.992 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.005397366, 2024-09-21T15:01:59.992 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:59.992 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:59.992 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:59.992 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:59.992 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:59.992 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140274 2024-09-21T15:01:59.992 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.992 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:59.992 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.993 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:59.993 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235884+0000", 2024-09-21T15:01:59.993 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:59.993 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.993 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.993 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:59.993 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235884+0000", 2024-09-21T15:01:59.993 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:59.993 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.993 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.993 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:59.993 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235883+0000", 2024-09-21T15:01:59.993 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:01:59.993 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.993 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.993 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:59.994 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235891+0000", 2024-09-21T15:01:59.994 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9740000000000002e-06 2024-09-21T15:01:59.994 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.994 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.994 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:59.994 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235900+0000", 2024-09-21T15:01:59.994 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6659999999999997e-06 2024-09-21T15:01:59.994 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.994 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.994 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:59.994 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235909+0000", 2024-09-21T15:01:59.994 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3400000000000004e-06 2024-09-21T15:01:59.994 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.994 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.994 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:59.995 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236012+0000", 2024-09-21T15:01:59.995 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010240500000000001 2024-09-21T15:01:59.995 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.995 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.995 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:59.995 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236027+0000", 2024-09-21T15:01:59.995 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5418e-05 2024-09-21T15:01:59.995 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.995 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.995 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:59.995 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.241068+0000", 2024-09-21T15:01:59.995 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0050410729999999 2024-09-21T15:01:59.995 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.995 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.995 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:01:59.995 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.241281+0000", 2024-09-21T15:01:59.996 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021321900000000001 2024-09-21T15:01:59.996 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:59.996 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:01:59.996 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:01:59.996 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.996 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.996 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140312 3.1 3:86b14cb7:::rbd_data.113664780c85.00000000000000d5:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:01:59.996 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.236769+0000", 2024-09-21T15:01:59.996 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 57.682591739000003, 2024-09-21T15:01:59.996 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.004717198, 2024-09-21T15:01:59.996 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:01:59.996 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:01:59.996 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:01:59.996 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:01:59.996 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:01:59.996 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140312 2024-09-21T15:01:59.997 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.997 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:01:59.997 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.997 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:01:59.997 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236769+0000", 2024-09-21T15:01:59.997 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:59.997 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.997 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.997 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:01:59.997 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236769+0000", 2024-09-21T15:01:59.997 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:01:59.997 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.997 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.997 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:01:59.997 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236768+0000", 2024-09-21T15:01:59.997 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:01:59.998 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.998 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.998 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:01:59.998 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236771+0000", 2024-09-21T15:01:59.998 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.331e-06 2024-09-21T15:01:59.998 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.998 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.998 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:01:59.998 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236777+0000", 2024-09-21T15:01:59.998 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6949999999999997e-06 2024-09-21T15:01:59.998 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.998 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.998 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:01:59.998 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236787+0000", 2024-09-21T15:01:59.999 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6700000000000006e-06 2024-09-21T15:01:59.999 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.999 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.999 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:59.999 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236846+0000", 2024-09-21T15:01:59.999 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.944e-05 2024-09-21T15:01:59.999 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.999 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.999 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:01:59.999 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236861+0000", 2024-09-21T15:01:59.999 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4611e-05 2024-09-21T15:01:59.999 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:01:59.999 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:01:59.999 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:01:59.999 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.241349+0000", 2024-09-21T15:01:59.999 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0044879250000001 2024-09-21T15:02:00.000 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:00.000 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:00.000 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:02:00.000 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.241486+0000", 2024-09-21T15:02:00.000 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013713000000000001 2024-09-21T15:02:00.000 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:00.000 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:02:00.000 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:00.000 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:00.000 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:00.000 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140325 3.1 3:87277f92:::rbd_data.113664780c85.0000000000000136:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:02:00.000 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.239351+0000", 2024-09-21T15:02:00.000 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 57.680009599999998, 2024-09-21T15:02:00.000 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0024334690000001, 2024-09-21T15:02:00.001 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:02:00.001 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:02:00.001 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:02:00.001 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:02:00.001 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:02:00.001 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140325 2024-09-21T15:02:00.001 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:00.001 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:02:00.001 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:00.001 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:02:00.001 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239351+0000", 2024-09-21T15:02:00.001 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:02:00.001 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:00.001 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:00.001 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:02:00.002 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239351+0000", 2024-09-21T15:02:00.002 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:02:00.002 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:00.002 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:00.002 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:02:00.002 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239349+0000", 2024-09-21T15:02:00.002 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:02:00.002 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:00.002 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:00.002 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:02:00.002 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239358+0000", 2024-09-21T15:02:00.002 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2830000000000001e-06 2024-09-21T15:02:00.002 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:00.002 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:00.002 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:02:00.002 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239379+0000", 2024-09-21T15:02:00.003 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1780000000000002e-05 2024-09-21T15:02:00.003 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:00.003 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:00.003 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:02:00.003 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239389+0000", 2024-09-21T15:02:00.003 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0056e-05 2024-09-21T15:02:00.003 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:00.003 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:00.003 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:02:00.003 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239484+0000", 2024-09-21T15:02:00.003 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4611000000000006e-05 2024-09-21T15:02:00.003 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:00.003 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:00.003 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:02:00.004 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239497+0000", 2024-09-21T15:02:00.004 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2909e-05 2024-09-21T15:02:00.004 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:00.004 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:00.004 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:02:00.004 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.241559+0000", 2024-09-21T15:02:00.004 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0020622459999999 2024-09-21T15:02:00.004 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:00.004 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:00.004 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:02:00.004 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.241784+0000", 2024-09-21T15:02:00.004 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000225009 2024-09-21T15:02:00.004 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:00.004 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:02:00.004 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:00.005 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:00.005 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:00.005 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140335 3.1 3:80f1e8f1:::rbd_data.113664780c85.000000000000017c:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:02:00.005 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.239409+0000", 2024-09-21T15:02:00.005 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 57.679950881000003, 2024-09-21T15:02:00.005 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0025803150000001, 2024-09-21T15:02:00.005 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:02:00.005 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:02:00.005 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:02:00.005 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:02:00.005 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:02:00.005 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140335 2024-09-21T15:02:00.005 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:00.005 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:02:00.005 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:00.006 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:02:00.006 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239409+0000", 2024-09-21T15:02:00.006 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:02:00.006 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:00.006 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:00.006 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:02:00.006 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239409+0000", 2024-09-21T15:02:00.006 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:02:00.006 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:00.006 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:00.006 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:02:00.006 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239409+0000", 2024-09-21T15:02:00.006 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:02:00.007 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:00.007 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:00.007 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:02:00.007 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239413+0000", 2024-09-21T15:02:00.007 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7620000000000003e-06 2024-09-21T15:02:00.007 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:00.007 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:00.007 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:02:00.007 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239436+0000", 2024-09-21T15:02:00.007 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2889000000000001e-05 2024-09-21T15:02:00.007 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:00.007 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:00.007 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:02:00.007 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239444+0000", 2024-09-21T15:02:00.008 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6809999999999998e-06 2024-09-21T15:02:00.008 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:00.008 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:00.008 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:02:00.008 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239542+0000", 2024-09-21T15:02:00.008 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7868999999999994e-05 2024-09-21T15:02:00.008 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:00.008 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:00.008 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:02:00.008 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239558+0000", 2024-09-21T15:02:00.008 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6257e-05 2024-09-21T15:02:00.008 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:00.008 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:00.008 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:02:00.009 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.241847+0000", 2024-09-21T15:02:00.009 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.002288533 2024-09-21T15:02:00.009 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:00.009 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:00.009 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:02:00.009 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.241990+0000", 2024-09-21T15:02:00.009 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014308300000000001 2024-09-21T15:02:00.009 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:00.009 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:02:00.009 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:00.009 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:00.009 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:00.009 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:166488 3.4 3.7bcf3e84 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e189)", 2024-09-21T15:02:00.009 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:21.342023+0000", 2024-09-21T15:02:00.009 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 38.577337049, 2024-09-21T15:02:00.010 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0015098469999999, 2024-09-21T15:02:00.010 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:02:00.010 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:02:00.010 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:02:00.010 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:02:00.010 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:02:00.010 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 166488 2024-09-21T15:02:00.010 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:00.010 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:02:00.010 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:00.010 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:02:00.010 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:21.342023+0000", 2024-09-21T15:02:00.010 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:02:00.010 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:00.011 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:00.011 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:02:00.011 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:21.342023+0000", 2024-09-21T15:02:00.011 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:02:00.011 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:00.011 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:00.011 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:02:00.011 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:21.342022+0000", 2024-09-21T15:02:00.011 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:02:00.011 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:00.011 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:00.011 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:02:00.011 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:21.342029+0000", 2024-09-21T15:02:00.011 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1420000000000004e-06 2024-09-21T15:02:00.012 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:00.012 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:00.012 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:02:00.012 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:21.342051+0000", 2024-09-21T15:02:00.012 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2549999999999999e-05 2024-09-21T15:02:00.012 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:00.012 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:00.012 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:02:00.012 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:21.342059+0000", 2024-09-21T15:02:00.012 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8490000000000006e-06 2024-09-21T15:02:00.012 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:00.012 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:00.012 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:02:00.012 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:21.342105+0000", 2024-09-21T15:02:00.012 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5488999999999998e-05 2024-09-21T15:02:00.013 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:00.013 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:00.013 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:02:00.013 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:21.342115+0000", 2024-09-21T15:02:00.013 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0633e-05 2024-09-21T15:02:00.013 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:00.013 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:00.013 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:02:00.013 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:22.343521+0000", 2024-09-21T15:02:00.013 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0014057270000001 2024-09-21T15:02:00.013 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:00.013 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:00.013 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:02:00.013 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:22.343533+0000", 2024-09-21T15:02:00.014 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2097e-05 2024-09-21T15:02:00.014 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:00.014 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:02:00.014 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:00.014 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:00.014 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:02:00.014 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:02:00.015 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T15:02:00.016 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:02:00.013+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:02:00.076 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T15:02:00.117 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:02:00.112+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:02:00.125 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:02:00.125 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:02:00.125 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T15:02:00.126 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:02:00.135 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T15:02:00.208 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 5, 6, 7] out_osds: [4] dead_osds: [4] live_osds: [0, 1, 2, 3, 5, 6, 7] 2024-09-21T15:02:00.208 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T15:02:00.208 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-21T15:02:00.208 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-21T15:02:00.217 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:02:00.212+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:02:00.219 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T15:02:00.247 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T15:02:00.253 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:02:00.253 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:02:00.253 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T15:02:00.254 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T15:02:00.254 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:02:00.265 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T15:02:00.271 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T15:02:00.304 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T15:02:00.317 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:02:00.317+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:02:00.326 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T15:02:00.350 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T15:02:00.379 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T15:02:00.379 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:02:00.379 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:02:00.379 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:02:00.379 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:02:00.379 INFO:teuthology.orchestra.run.smithi032.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-21T15:02:00.379 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:02:00.379 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:02:00.379 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:02:00.382 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-21T15:02:00.383 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-21T15:02:00.417 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:02:00.417 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:02:00.418 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:02:00.412+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:02:00.427 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T15:02:00.518 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:02:00.517+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:02:00.523 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-21T15:02:00.524 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:02:00.524 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 228, 2024-09-21T15:02:00.524 INFO:teuthology.orchestra.run.smithi032.stdout: "fsid": "caf51e55-9234-4a26-bcd2-d60811962560", 2024-09-21T15:02:00.524 INFO:teuthology.orchestra.run.smithi032.stdout: "created": "2024-09-21T14:57:50.338822+0000", 2024-09-21T15:02:00.524 INFO:teuthology.orchestra.run.smithi032.stdout: "modified": "2024-09-21T15:02:00.176499+0000", 2024-09-21T15:02:00.524 INFO:teuthology.orchestra.run.smithi032.stdout: "last_up_change": "2024-09-21T15:01:34.172764+0000", 2024-09-21T15:02:00.524 INFO:teuthology.orchestra.run.smithi032.stdout: "last_in_change": "2024-09-21T15:01:21.319966+0000", 2024-09-21T15:02:00.524 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-21T15:02:00.524 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_num": 5804032, 2024-09-21T15:02:00.524 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_set": [ 2024-09-21T15:02:00.524 INFO:teuthology.orchestra.run.smithi032.stdout: "nodeep-scrub", 2024-09-21T15:02:00.524 INFO:teuthology.orchestra.run.smithi032.stdout: "pglog_hardlimit", 2024-09-21T15:02:00.524 INFO:teuthology.orchestra.run.smithi032.stdout: "purged_snapdirs", 2024-09-21T15:02:00.524 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_deletes", 2024-09-21T15:02:00.525 INFO:teuthology.orchestra.run.smithi032.stdout: "sortbitwise" 2024-09-21T15:02:00.525 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-21T15:02:00.525 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_version": 5, 2024-09-21T15:02:00.525 INFO:teuthology.orchestra.run.smithi032.stdout: "full_ratio": 0.94999998807907104, 2024-09-21T15:02:00.525 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-21T15:02:00.525 INFO:teuthology.orchestra.run.smithi032.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-21T15:02:00.525 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_snapshot": "", 2024-09-21T15:02:00.525 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_max": 3, 2024-09-21T15:02:00.525 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd": 8, 2024-09-21T15:02:00.525 INFO:teuthology.orchestra.run.smithi032.stdout: "require_min_compat_client": "luminous", 2024-09-21T15:02:00.525 INFO:teuthology.orchestra.run.smithi032.stdout: "min_compat_client": "luminous", 2024-09-21T15:02:00.525 INFO:teuthology.orchestra.run.smithi032.stdout: "require_osd_release": "reef", 2024-09-21T15:02:00.525 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_crimson": false, 2024-09-21T15:02:00.526 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2024-09-21T15:02:00.526 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.526 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 1, 2024-09-21T15:02:00.526 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": ".mgr", 2024-09-21T15:02:00.526 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-09-21T14:58:01.842810+0000", 2024-09-21T15:02:00.526 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2024-09-21T15:02:00.526 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2024-09-21T15:02:00.526 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-09-21T15:02:00.526 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-09-21T15:02:00.526 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-09-21T15:02:00.526 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-09-21T15:02:00.526 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-09-21T15:02:00.526 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-09-21T15:02:00.526 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T15:02:00.527 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T15:02:00.527 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-09-21T15:02:00.527 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-09-21T15:02:00.527 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 1, 2024-09-21T15:02:00.527 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 1, 2024-09-21T15:02:00.527 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 1, 2024-09-21T15:02:00.527 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 1, 2024-09-21T15:02:00.527 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 1, 2024-09-21T15:02:00.527 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-09-21T15:02:00.527 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-09-21T15:02:00.527 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-09-21T15:02:00.527 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-09-21T15:02:00.527 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-09-21T15:02:00.527 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-09-21T15:02:00.528 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-09-21T15:02:00.528 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.528 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "18", 2024-09-21T15:02:00.528 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-09-21T15:02:00.528 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T15:02:00.528 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T15:02:00.528 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-09-21T15:02:00.528 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-09-21T15:02:00.528 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2024-09-21T15:02:00.528 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2024-09-21T15:02:00.528 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-09-21T15:02:00.528 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-09-21T15:02:00.528 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-09-21T15:02:00.529 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-09-21T15:02:00.529 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-09-21T15:02:00.529 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-09-21T15:02:00.529 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-09-21T15:02:00.529 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-09-21T15:02:00.529 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-09-21T15:02:00.529 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-09-21T15:02:00.529 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-09-21T15:02:00.529 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T15:02:00.529 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T15:02:00.529 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T15:02:00.529 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-09-21T15:02:00.529 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-09-21T15:02:00.529 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-09-21T15:02:00.529 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-09-21T15:02:00.530 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-09-21T15:02:00.530 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.530 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-09-21T15:02:00.530 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-09-21T15:02:00.530 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-09-21T15:02:00.530 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-09-21T15:02:00.530 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-09-21T15:02:00.530 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T15:02:00.530 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-09-21T15:02:00.530 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-09-21T15:02:00.530 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-09-21T15:02:00.530 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-09-21T15:02:00.530 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-09-21T15:02:00.531 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2024-09-21T15:02:00.531 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_max": 32, 2024-09-21T15:02:00.531 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 1 2024-09-21T15:02:00.531 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.531 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-09-21T15:02:00.531 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": {} 2024-09-21T15:02:00.531 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.531 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-09-21T15:02:00.531 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 8, 2024-09-21T15:02:00.531 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 8, 2024-09-21T15:02:00.531 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 0.25, 2024-09-21T15:02:00.531 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 2, 2024-09-21T15:02:00.531 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 2, 2024-09-21T15:02:00.531 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 0.89523506164550781, 2024-09-21T15:02:00.532 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 0.89999997615814209, 2024-09-21T15:02:00.532 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 0.89999997615814209 2024-09-21T15:02:00.532 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:02:00.532 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.532 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.532 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 2, 2024-09-21T15:02:00.532 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "rbd", 2024-09-21T15:02:00.532 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-09-21T14:58:08.284925+0000", 2024-09-21T15:02:00.532 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 8193, 2024-09-21T15:02:00.532 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T15:02:00.532 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-09-21T15:02:00.532 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-09-21T15:02:00.532 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-09-21T15:02:00.532 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-09-21T15:02:00.533 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-09-21T15:02:00.533 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-09-21T15:02:00.533 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T15:02:00.533 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T15:02:00.533 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-09-21T15:02:00.533 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-09-21T15:02:00.533 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2024-09-21T15:02:00.533 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2024-09-21T15:02:00.533 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2024-09-21T15:02:00.533 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2024-09-21T15:02:00.533 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2024-09-21T15:02:00.533 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-09-21T15:02:00.533 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-09-21T15:02:00.533 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-09-21T15:02:00.534 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-09-21T15:02:00.534 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-09-21T15:02:00.534 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-09-21T15:02:00.534 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-09-21T15:02:00.534 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.534 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "22", 2024-09-21T15:02:00.534 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-09-21T15:02:00.534 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T15:02:00.534 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T15:02:00.534 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-09-21T15:02:00.534 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-09-21T15:02:00.534 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 3, 2024-09-21T15:02:00.534 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 22, 2024-09-21T15:02:00.534 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-09-21T15:02:00.535 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-09-21T15:02:00.535 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-09-21T15:02:00.535 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-09-21T15:02:00.535 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-09-21T15:02:00.535 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-09-21T15:02:00.535 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-09-21T15:02:00.535 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-09-21T15:02:00.535 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-09-21T15:02:00.535 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-09-21T15:02:00.535 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-09-21T15:02:00.535 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T15:02:00.535 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T15:02:00.535 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T15:02:00.535 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-09-21T15:02:00.536 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-09-21T15:02:00.536 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-09-21T15:02:00.536 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-09-21T15:02:00.536 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-09-21T15:02:00.536 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.536 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-09-21T15:02:00.536 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-09-21T15:02:00.536 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-09-21T15:02:00.536 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-09-21T15:02:00.536 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-09-21T15:02:00.536 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T15:02:00.536 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-09-21T15:02:00.536 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-09-21T15:02:00.536 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-09-21T15:02:00.536 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-09-21T15:02:00.537 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-09-21T15:02:00.537 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2024-09-21T15:02:00.537 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-09-21T15:02:00.537 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd": {} 2024-09-21T15:02:00.537 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.537 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-09-21T15:02:00.537 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 5, 2024-09-21T15:02:00.537 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 5, 2024-09-21T15:02:00.537 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 0.75, 2024-09-21T15:02:00.537 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 3.75, 2024-09-21T15:02:00.537 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 3.75, 2024-09-21T15:02:00.537 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 0.89523506164550781, 2024-09-21T15:02:00.537 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 0.89999997615814209, 2024-09-21T15:02:00.537 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 0.89999997615814209 2024-09-21T15:02:00.538 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:02:00.538 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.538 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.538 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 3, 2024-09-21T15:02:00.538 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "pool_client.0", 2024-09-21T15:02:00.538 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2024-09-21T14:58:17.510709+0000", 2024-09-21T15:02:00.538 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 8193, 2024-09-21T15:02:00.538 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T15:02:00.538 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2024-09-21T15:02:00.538 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2024-09-21T15:02:00.538 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2024-09-21T15:02:00.538 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2024-09-21T15:02:00.538 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2024-09-21T15:02:00.538 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2024-09-21T15:02:00.539 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T15:02:00.539 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T15:02:00.539 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2024-09-21T15:02:00.539 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2024-09-21T15:02:00.539 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 32, 2024-09-21T15:02:00.539 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 32, 2024-09-21T15:02:00.539 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 32, 2024-09-21T15:02:00.539 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 32, 2024-09-21T15:02:00.539 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 32, 2024-09-21T15:02:00.539 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2024-09-21T15:02:00.539 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2024-09-21T15:02:00.539 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2024-09-21T15:02:00.539 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2024-09-21T15:02:00.539 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2024-09-21T15:02:00.540 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2024-09-21T15:02:00.540 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2024-09-21T15:02:00.540 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.540 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "228", 2024-09-21T15:02:00.540 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2024-09-21T15:02:00.540 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T15:02:00.540 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T15:02:00.540 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2024-09-21T15:02:00.540 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2024-09-21T15:02:00.540 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 149, 2024-09-21T15:02:00.540 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 228, 2024-09-21T15:02:00.540 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2024-09-21T15:02:00.540 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2024-09-21T15:02:00.540 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2024-09-21T15:02:00.540 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2024-09-21T15:02:00.541 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2024-09-21T15:02:00.541 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2024-09-21T15:02:00.541 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2024-09-21T15:02:00.541 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2024-09-21T15:02:00.541 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2024-09-21T15:02:00.541 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2024-09-21T15:02:00.541 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2024-09-21T15:02:00.541 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T15:02:00.541 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T15:02:00.541 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T15:02:00.541 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2024-09-21T15:02:00.541 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2024-09-21T15:02:00.541 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2024-09-21T15:02:00.541 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2024-09-21T15:02:00.541 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2024-09-21T15:02:00.542 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.542 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2024-09-21T15:02:00.542 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2024-09-21T15:02:00.542 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2024-09-21T15:02:00.542 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2024-09-21T15:02:00.542 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2024-09-21T15:02:00.542 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T15:02:00.542 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2024-09-21T15:02:00.542 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2024-09-21T15:02:00.542 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2024-09-21T15:02:00.542 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2024-09-21T15:02:00.542 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2024-09-21T15:02:00.542 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2024-09-21T15:02:00.542 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2024-09-21T15:02:00.542 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd": {} 2024-09-21T15:02:00.543 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.543 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2024-09-21T15:02:00.543 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 1.4900000095367432, 2024-09-21T15:02:00.543 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 1.4900000095367432, 2024-09-21T15:02:00.543 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 0.87999999523162842, 2024-09-21T15:02:00.543 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 1.309999942779541, 2024-09-21T15:02:00.543 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 1.309999942779541, 2024-09-21T15:02:00.543 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 0.89523506164550781, 2024-09-21T15:02:00.543 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 0.89999997615814209, 2024-09-21T15:02:00.543 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 0.89999997615814209 2024-09-21T15:02:00.543 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:02:00.543 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:02:00.543 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-21T15:02:00.543 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2024-09-21T15:02:00.544 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.544 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-09-21T15:02:00.544 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "a4a73316-3a94-42e9-9a51-deb46a66bb06", 2024-09-21T15:02:00.544 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-21T15:02:00.544 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-21T15:02:00.544 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-21T15:02:00.544 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-21T15:02:00.544 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-21T15:02:00.544 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-21T15:02:00.544 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2024-09-21T15:02:00.544 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 192, 2024-09-21T15:02:00.544 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-21T15:02:00.544 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-21T15:02:00.544 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-21T15:02:00.544 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:02:00.545 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.545 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:02:00.545 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6808", 2024-09-21T15:02:00.545 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4016672243 2024-09-21T15:02:00.545 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.545 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.545 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:02:00.545 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6809", 2024-09-21T15:02:00.545 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4016672243 2024-09-21T15:02:00.545 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:02:00.545 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:02:00.545 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.545 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-21T15:02:00.545 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:02:00.545 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.546 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:02:00.546 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6810", 2024-09-21T15:02:00.546 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4016672243 2024-09-21T15:02:00.546 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.546 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.546 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:02:00.546 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6811", 2024-09-21T15:02:00.546 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4016672243 2024-09-21T15:02:00.546 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:02:00.546 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:02:00.546 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.546 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-21T15:02:00.546 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:02:00.546 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.547 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:02:00.547 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6814", 2024-09-21T15:02:00.547 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4016672243 2024-09-21T15:02:00.547 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.547 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.547 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:02:00.547 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6815", 2024-09-21T15:02:00.547 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4016672243 2024-09-21T15:02:00.547 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:02:00.547 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:02:00.547 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.547 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-21T15:02:00.547 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:02:00.547 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.548 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:02:00.548 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6812", 2024-09-21T15:02:00.548 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4016672243 2024-09-21T15:02:00.548 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.548 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.548 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:02:00.548 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6813", 2024-09-21T15:02:00.548 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 4016672243 2024-09-21T15:02:00.548 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:02:00.548 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:02:00.548 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.548 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6809/4016672243", 2024-09-21T15:02:00.548 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6811/4016672243", 2024-09-21T15:02:00.548 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6815/4016672243", 2024-09-21T15:02:00.549 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6813/4016672243", 2024-09-21T15:02:00.549 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-21T15:02:00.549 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-21T15:02:00.549 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-21T15:02:00.549 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:02:00.549 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.549 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.549 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-09-21T15:02:00.549 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "006757a0-94d1-4a9b-aef8-73634cc5828e", 2024-09-21T15:02:00.549 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-21T15:02:00.549 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-21T15:02:00.549 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-21T15:02:00.549 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-21T15:02:00.549 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-21T15:02:00.549 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-21T15:02:00.550 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2024-09-21T15:02:00.550 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 223, 2024-09-21T15:02:00.550 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-21T15:02:00.550 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-21T15:02:00.550 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-21T15:02:00.550 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:02:00.550 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.550 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:02:00.550 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6824", 2024-09-21T15:02:00.550 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 641090577 2024-09-21T15:02:00.550 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.550 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.550 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:02:00.550 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6825", 2024-09-21T15:02:00.551 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 641090577 2024-09-21T15:02:00.551 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:02:00.551 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:02:00.551 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.551 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-21T15:02:00.551 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:02:00.551 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.551 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:02:00.551 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6826", 2024-09-21T15:02:00.551 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 641090577 2024-09-21T15:02:00.551 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.551 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.551 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:02:00.551 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6827", 2024-09-21T15:02:00.552 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 641090577 2024-09-21T15:02:00.552 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:02:00.552 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:02:00.552 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.552 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-21T15:02:00.552 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:02:00.552 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.552 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:02:00.552 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6830", 2024-09-21T15:02:00.552 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 641090577 2024-09-21T15:02:00.552 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.552 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.552 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:02:00.552 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6831", 2024-09-21T15:02:00.553 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 641090577 2024-09-21T15:02:00.553 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:02:00.553 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:02:00.553 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.553 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-21T15:02:00.553 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:02:00.553 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.553 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:02:00.553 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6828", 2024-09-21T15:02:00.553 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 641090577 2024-09-21T15:02:00.553 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.553 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.553 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:02:00.553 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6829", 2024-09-21T15:02:00.554 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 641090577 2024-09-21T15:02:00.554 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:02:00.554 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:02:00.554 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.554 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6825/641090577", 2024-09-21T15:02:00.554 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6827/641090577", 2024-09-21T15:02:00.554 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6831/641090577", 2024-09-21T15:02:00.554 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6829/641090577", 2024-09-21T15:02:00.554 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-21T15:02:00.554 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-21T15:02:00.554 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-21T15:02:00.554 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:02:00.554 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.554 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.555 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-09-21T15:02:00.555 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "0b499a23-820e-4e6c-a558-adbf88b7014f", 2024-09-21T15:02:00.555 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-21T15:02:00.555 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-21T15:02:00.555 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-21T15:02:00.555 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-21T15:02:00.555 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-21T15:02:00.555 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-21T15:02:00.555 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2024-09-21T15:02:00.555 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 223, 2024-09-21T15:02:00.555 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-21T15:02:00.555 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-21T15:02:00.555 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-21T15:02:00.555 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:02:00.555 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.556 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:02:00.556 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6800", 2024-09-21T15:02:00.556 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2609154414 2024-09-21T15:02:00.556 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.556 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.556 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:02:00.556 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6801", 2024-09-21T15:02:00.556 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2609154414 2024-09-21T15:02:00.556 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:02:00.556 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:02:00.556 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.556 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-21T15:02:00.556 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:02:00.556 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.557 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:02:00.557 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6802", 2024-09-21T15:02:00.557 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2609154414 2024-09-21T15:02:00.557 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.557 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.557 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:02:00.557 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6803", 2024-09-21T15:02:00.557 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2609154414 2024-09-21T15:02:00.557 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:02:00.557 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:02:00.557 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.557 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-21T15:02:00.557 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:02:00.557 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.557 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:02:00.558 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6806", 2024-09-21T15:02:00.558 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2609154414 2024-09-21T15:02:00.558 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.558 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.558 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:02:00.558 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6807", 2024-09-21T15:02:00.558 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2609154414 2024-09-21T15:02:00.558 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:02:00.558 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:02:00.558 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.558 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-21T15:02:00.558 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:02:00.558 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.559 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:02:00.559 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6804", 2024-09-21T15:02:00.559 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2609154414 2024-09-21T15:02:00.559 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.559 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.559 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:02:00.559 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6805", 2024-09-21T15:02:00.559 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2609154414 2024-09-21T15:02:00.559 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:02:00.559 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:02:00.559 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.559 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6801/2609154414", 2024-09-21T15:02:00.559 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6803/2609154414", 2024-09-21T15:02:00.559 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6807/2609154414", 2024-09-21T15:02:00.560 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6805/2609154414", 2024-09-21T15:02:00.560 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-21T15:02:00.560 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-21T15:02:00.560 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-21T15:02:00.560 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:02:00.560 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.560 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.560 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-09-21T15:02:00.560 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "5a57b3dd-4a07-45b7-b159-3cac4c7eb262", 2024-09-21T15:02:00.560 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-21T15:02:00.560 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-21T15:02:00.560 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-21T15:02:00.560 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 0.833831787109375, 2024-09-21T15:02:00.560 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-21T15:02:00.560 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-21T15:02:00.561 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2024-09-21T15:02:00.561 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 192, 2024-09-21T15:02:00.561 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-21T15:02:00.561 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-21T15:02:00.561 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-21T15:02:00.561 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:02:00.561 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.561 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:02:00.561 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6816", 2024-09-21T15:02:00.561 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1462563376 2024-09-21T15:02:00.561 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.561 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.561 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:02:00.561 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6817", 2024-09-21T15:02:00.561 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1462563376 2024-09-21T15:02:00.562 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:02:00.562 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:02:00.562 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.562 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-21T15:02:00.562 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:02:00.562 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.562 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:02:00.562 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6818", 2024-09-21T15:02:00.562 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1462563376 2024-09-21T15:02:00.562 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.562 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.562 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:02:00.562 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6819", 2024-09-21T15:02:00.563 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1462563376 2024-09-21T15:02:00.563 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:02:00.563 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:02:00.563 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.563 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-21T15:02:00.563 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:02:00.563 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.563 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:02:00.563 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6822", 2024-09-21T15:02:00.563 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1462563376 2024-09-21T15:02:00.563 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.563 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.563 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:02:00.563 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6823", 2024-09-21T15:02:00.564 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1462563376 2024-09-21T15:02:00.564 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:02:00.564 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:02:00.564 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.564 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-21T15:02:00.564 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:02:00.564 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.564 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:02:00.564 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6820", 2024-09-21T15:02:00.564 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1462563376 2024-09-21T15:02:00.564 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.564 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.564 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:02:00.564 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6821", 2024-09-21T15:02:00.564 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1462563376 2024-09-21T15:02:00.565 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:02:00.565 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:02:00.565 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.565 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6817/1462563376", 2024-09-21T15:02:00.565 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6819/1462563376", 2024-09-21T15:02:00.565 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6823/1462563376", 2024-09-21T15:02:00.565 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6821/1462563376", 2024-09-21T15:02:00.565 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-21T15:02:00.565 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-21T15:02:00.565 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-21T15:02:00.565 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:02:00.565 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.565 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.565 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-09-21T15:02:00.566 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "39a00807-7517-474e-8842-623dd7093cc5", 2024-09-21T15:02:00.566 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 0, 2024-09-21T15:02:00.566 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 0, 2024-09-21T15:02:00.566 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 0, 2024-09-21T15:02:00.566 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-21T15:02:00.566 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-21T15:02:00.566 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-21T15:02:00.566 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2024-09-21T15:02:00.566 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 147, 2024-09-21T15:02:00.566 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 202, 2024-09-21T15:02:00.566 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-21T15:02:00.566 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-21T15:02:00.566 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:02:00.566 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.567 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:02:00.567 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6801", 2024-09-21T15:02:00.567 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 144412042 2024-09-21T15:02:00.567 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.567 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.567 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:02:00.567 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6803", 2024-09-21T15:02:00.567 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 144412042 2024-09-21T15:02:00.567 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:02:00.567 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:02:00.567 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.567 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-21T15:02:00.567 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:02:00.567 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.567 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:02:00.568 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6805", 2024-09-21T15:02:00.568 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 144412042 2024-09-21T15:02:00.568 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.568 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.568 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:02:00.568 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6807", 2024-09-21T15:02:00.568 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 144412042 2024-09-21T15:02:00.568 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:02:00.568 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:02:00.568 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.568 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-21T15:02:00.568 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:02:00.568 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.568 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:02:00.568 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6812", 2024-09-21T15:02:00.569 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 144412042 2024-09-21T15:02:00.569 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.569 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.569 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:02:00.569 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6814", 2024-09-21T15:02:00.569 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 144412042 2024-09-21T15:02:00.569 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:02:00.569 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:02:00.569 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.569 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-21T15:02:00.569 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:02:00.569 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.569 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:02:00.569 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6809", 2024-09-21T15:02:00.569 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 144412042 2024-09-21T15:02:00.569 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.570 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.570 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:02:00.570 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6811", 2024-09-21T15:02:00.570 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 144412042 2024-09-21T15:02:00.570 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:02:00.570 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:02:00.570 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.570 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.46:6803/144412042", 2024-09-21T15:02:00.570 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.46:6807/144412042", 2024-09-21T15:02:00.570 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.46:6814/144412042", 2024-09-21T15:02:00.570 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.46:6811/144412042", 2024-09-21T15:02:00.570 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-21T15:02:00.570 INFO:teuthology.orchestra.run.smithi032.stdout: "exists" 2024-09-21T15:02:00.570 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:02:00.571 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.571 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.571 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-09-21T15:02:00.571 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "edf9f045-ddeb-4504-a0fc-1a0e4e853550", 2024-09-21T15:02:00.571 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-21T15:02:00.571 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-21T15:02:00.571 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-21T15:02:00.571 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 0.4931182861328125, 2024-09-21T15:02:00.571 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-21T15:02:00.571 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-21T15:02:00.571 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2024-09-21T15:02:00.571 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 177, 2024-09-21T15:02:00.571 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-21T15:02:00.571 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-21T15:02:00.572 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-21T15:02:00.572 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:02:00.572 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.572 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:02:00.572 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6800", 2024-09-21T15:02:00.572 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 849101442 2024-09-21T15:02:00.572 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.572 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.572 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:02:00.572 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6802", 2024-09-21T15:02:00.572 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 849101442 2024-09-21T15:02:00.572 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:02:00.572 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:02:00.572 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.572 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-21T15:02:00.573 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:02:00.573 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.573 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:02:00.573 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6804", 2024-09-21T15:02:00.573 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 849101442 2024-09-21T15:02:00.573 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.573 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.573 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:02:00.573 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6806", 2024-09-21T15:02:00.573 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 849101442 2024-09-21T15:02:00.573 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:02:00.573 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:02:00.573 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.573 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-21T15:02:00.573 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:02:00.574 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.574 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:02:00.574 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6813", 2024-09-21T15:02:00.574 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 849101442 2024-09-21T15:02:00.574 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.574 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.574 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:02:00.574 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6815", 2024-09-21T15:02:00.574 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 849101442 2024-09-21T15:02:00.574 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:02:00.574 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:02:00.574 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.574 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-21T15:02:00.574 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:02:00.575 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.575 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:02:00.575 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6808", 2024-09-21T15:02:00.575 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 849101442 2024-09-21T15:02:00.575 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.575 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.575 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:02:00.575 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6810", 2024-09-21T15:02:00.575 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 849101442 2024-09-21T15:02:00.575 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:02:00.575 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:02:00.575 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.575 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.46:6802/849101442", 2024-09-21T15:02:00.575 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.46:6806/849101442", 2024-09-21T15:02:00.576 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.46:6815/849101442", 2024-09-21T15:02:00.576 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.46:6810/849101442", 2024-09-21T15:02:00.576 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-21T15:02:00.576 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-21T15:02:00.576 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-21T15:02:00.576 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:02:00.576 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.576 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.576 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-09-21T15:02:00.576 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "c820ea6b-88fe-4421-83ec-17c466691fea", 2024-09-21T15:02:00.576 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-21T15:02:00.576 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-21T15:02:00.576 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-21T15:02:00.576 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 0.834930419921875, 2024-09-21T15:02:00.577 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-21T15:02:00.577 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-21T15:02:00.577 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 14, 2024-09-21T15:02:00.577 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 223, 2024-09-21T15:02:00.577 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-21T15:02:00.577 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-21T15:02:00.577 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-21T15:02:00.577 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:02:00.577 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.577 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:02:00.577 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6817", 2024-09-21T15:02:00.577 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 847189172 2024-09-21T15:02:00.577 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.577 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.577 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:02:00.577 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6818", 2024-09-21T15:02:00.578 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 847189172 2024-09-21T15:02:00.578 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:02:00.578 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:02:00.578 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.578 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-21T15:02:00.578 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:02:00.578 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.578 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:02:00.578 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6820", 2024-09-21T15:02:00.578 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 847189172 2024-09-21T15:02:00.578 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.578 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.578 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:02:00.578 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6822", 2024-09-21T15:02:00.579 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 847189172 2024-09-21T15:02:00.579 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:02:00.579 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:02:00.579 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.579 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-21T15:02:00.579 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:02:00.579 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.579 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:02:00.579 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6827", 2024-09-21T15:02:00.579 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 847189172 2024-09-21T15:02:00.579 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.579 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.579 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:02:00.579 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6828", 2024-09-21T15:02:00.580 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 847189172 2024-09-21T15:02:00.580 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:02:00.580 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:02:00.580 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.580 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-21T15:02:00.580 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:02:00.580 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.580 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:02:00.580 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6823", 2024-09-21T15:02:00.580 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 847189172 2024-09-21T15:02:00.580 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.580 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.580 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:02:00.580 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6825", 2024-09-21T15:02:00.581 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 847189172 2024-09-21T15:02:00.581 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:02:00.581 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:02:00.581 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.581 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.46:6818/847189172", 2024-09-21T15:02:00.581 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.46:6822/847189172", 2024-09-21T15:02:00.581 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.46:6828/847189172", 2024-09-21T15:02:00.581 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.46:6825/847189172", 2024-09-21T15:02:00.581 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-21T15:02:00.581 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-21T15:02:00.581 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-21T15:02:00.581 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:02:00.581 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.581 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.582 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-09-21T15:02:00.582 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "3a2f82a4-28a0-4fcb-9d6b-2285fa4d07ac", 2024-09-21T15:02:00.582 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2024-09-21T15:02:00.582 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2024-09-21T15:02:00.582 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2024-09-21T15:02:00.582 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2024-09-21T15:02:00.582 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 0, 2024-09-21T15:02:00.582 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 0, 2024-09-21T15:02:00.582 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 15, 2024-09-21T15:02:00.582 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 223, 2024-09-21T15:02:00.582 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 0, 2024-09-21T15:02:00.582 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2024-09-21T15:02:00.582 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2024-09-21T15:02:00.583 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:02:00.583 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.583 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:02:00.583 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6816", 2024-09-21T15:02:00.583 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3894963553 2024-09-21T15:02:00.583 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.583 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.583 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:02:00.583 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6819", 2024-09-21T15:02:00.583 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3894963553 2024-09-21T15:02:00.583 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:02:00.583 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:02:00.583 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.583 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2024-09-21T15:02:00.584 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:02:00.584 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.584 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:02:00.584 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6821", 2024-09-21T15:02:00.584 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3894963553 2024-09-21T15:02:00.584 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.584 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.584 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:02:00.584 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6824", 2024-09-21T15:02:00.584 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3894963553 2024-09-21T15:02:00.584 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:02:00.584 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:02:00.584 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.584 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2024-09-21T15:02:00.585 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:02:00.585 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.585 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:02:00.585 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6830", 2024-09-21T15:02:00.585 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3894963553 2024-09-21T15:02:00.585 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.585 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.585 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:02:00.585 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6831", 2024-09-21T15:02:00.585 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3894963553 2024-09-21T15:02:00.585 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:02:00.585 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:02:00.585 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.585 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2024-09-21T15:02:00.586 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2024-09-21T15:02:00.586 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.586 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2024-09-21T15:02:00.586 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6826", 2024-09-21T15:02:00.586 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3894963553 2024-09-21T15:02:00.586 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.586 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.586 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-09-21T15:02:00.586 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.46:6829", 2024-09-21T15:02:00.586 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3894963553 2024-09-21T15:02:00.586 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:02:00.586 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:02:00.586 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.586 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.46:6819/3894963553", 2024-09-21T15:02:00.587 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.46:6824/3894963553", 2024-09-21T15:02:00.587 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.46:6831/3894963553", 2024-09-21T15:02:00.587 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.46:6829/3894963553", 2024-09-21T15:02:00.587 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2024-09-21T15:02:00.587 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2024-09-21T15:02:00.587 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2024-09-21T15:02:00.587 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:02:00.587 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:02:00.587 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-21T15:02:00.587 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_xinfo": [ 2024-09-21T15:02:00.587 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.587 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2024-09-21T15:02:00.587 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-21T15:02:00.587 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-21T15:02:00.588 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-21T15:02:00.588 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-21T15:02:00.588 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-21T15:02:00.588 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-21T14:57:57.458317+0000", 2024-09-21T15:02:00.588 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-21T15:02:00.588 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.588 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.588 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2024-09-21T15:02:00.588 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-21T15:02:00.588 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-21T15:02:00.588 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-21T15:02:00.588 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-21T15:02:00.588 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-21T15:02:00.588 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-21T14:57:57.907624+0000", 2024-09-21T15:02:00.588 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-21T15:02:00.589 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.589 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.589 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2024-09-21T15:02:00.589 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-21T15:02:00.589 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-21T15:02:00.589 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-21T15:02:00.589 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-21T15:02:00.589 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-21T15:02:00.589 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-21T14:57:57.278817+0000", 2024-09-21T15:02:00.589 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-21T15:02:00.589 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.589 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.589 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2024-09-21T15:02:00.589 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-21T15:02:00.589 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-21T15:02:00.590 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-21T15:02:00.590 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-21T15:02:00.590 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-21T15:02:00.590 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-21T14:57:57.500449+0000", 2024-09-21T15:02:00.590 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-21T15:02:00.590 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.590 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.590 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 4, 2024-09-21T15:02:00.590 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-21T15:02:00.590 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-21T15:02:00.590 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-21T15:02:00.590 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-21T15:02:00.590 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 65536, 2024-09-21T15:02:00.590 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-21T14:57:57.504325+0000", 2024-09-21T15:02:00.591 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 202 2024-09-21T15:02:00.591 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.591 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.591 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 5, 2024-09-21T15:02:00.591 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-21T15:02:00.591 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-21T15:02:00.591 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-21T15:02:00.591 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-21T15:02:00.591 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-21T15:02:00.591 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-21T14:57:57.777525+0000", 2024-09-21T15:02:00.591 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-21T15:02:00.591 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.591 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.591 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 6, 2024-09-21T15:02:00.592 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-21T15:02:00.592 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-21T15:02:00.592 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-21T15:02:00.592 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-21T15:02:00.592 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-21T15:02:00.592 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-21T14:57:57.698961+0000", 2024-09-21T15:02:00.592 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-21T15:02:00.592 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.592 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.592 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 7, 2024-09-21T15:02:00.592 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2024-09-21T15:02:00.592 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2024-09-21T15:02:00.592 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2024-09-21T15:02:00.592 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2024-09-21T15:02:00.593 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2024-09-21T15:02:00.593 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2024-09-21T14:57:57.932625+0000", 2024-09-21T15:02:00.593 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2024-09-21T15:02:00.593 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:02:00.593 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-21T15:02:00.593 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap": [ 2024-09-21T15:02:00.593 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.593 INFO:teuthology.orchestra.run.smithi032.stdout: "pgid": "3.2", 2024-09-21T15:02:00.593 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2024-09-21T15:02:00.593 INFO:teuthology.orchestra.run.smithi032.stdout: 2, 2024-09-21T15:02:00.593 INFO:teuthology.orchestra.run.smithi032.stdout: 3 2024-09-21T15:02:00.593 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:02:00.593 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.593 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.593 INFO:teuthology.orchestra.run.smithi032.stdout: "pgid": "3.4", 2024-09-21T15:02:00.594 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2024-09-21T15:02:00.594 INFO:teuthology.orchestra.run.smithi032.stdout: 4, 2024-09-21T15:02:00.594 INFO:teuthology.orchestra.run.smithi032.stdout: 2 2024-09-21T15:02:00.594 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:02:00.594 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.594 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.594 INFO:teuthology.orchestra.run.smithi032.stdout: "pgid": "3.8", 2024-09-21T15:02:00.594 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2024-09-21T15:02:00.594 INFO:teuthology.orchestra.run.smithi032.stdout: 6, 2024-09-21T15:02:00.594 INFO:teuthology.orchestra.run.smithi032.stdout: 3 2024-09-21T15:02:00.594 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:02:00.594 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.594 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.594 INFO:teuthology.orchestra.run.smithi032.stdout: "pgid": "3.a", 2024-09-21T15:02:00.594 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2024-09-21T15:02:00.595 INFO:teuthology.orchestra.run.smithi032.stdout: 1, 2024-09-21T15:02:00.595 INFO:teuthology.orchestra.run.smithi032.stdout: 0 2024-09-21T15:02:00.595 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:02:00.595 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:02:00.595 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-21T15:02:00.595 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_items": [ 2024-09-21T15:02:00.595 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.595 INFO:teuthology.orchestra.run.smithi032.stdout: "pgid": "3.1", 2024-09-21T15:02:00.595 INFO:teuthology.orchestra.run.smithi032.stdout: "mappings": [ 2024-09-21T15:02:00.595 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.595 INFO:teuthology.orchestra.run.smithi032.stdout: "from": 2, 2024-09-21T15:02:00.595 INFO:teuthology.orchestra.run.smithi032.stdout: "to": 0 2024-09-21T15:02:00.595 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:02:00.595 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:02:00.595 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:02:00.596 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-21T15:02:00.596 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_primaries": [], 2024-09-21T15:02:00.596 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_temp": [], 2024-09-21T15:02:00.596 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_temp": [], 2024-09-21T15:02:00.596 INFO:teuthology.orchestra.run.smithi032.stdout: "blocklist": {}, 2024-09-21T15:02:00.596 INFO:teuthology.orchestra.run.smithi032.stdout: "range_blocklist": {}, 2024-09-21T15:02:00.596 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profiles": { 2024-09-21T15:02:00.596 INFO:teuthology.orchestra.run.smithi032.stdout: "default": { 2024-09-21T15:02:00.596 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2024-09-21T15:02:00.596 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2024-09-21T15:02:00.596 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "1", 2024-09-21T15:02:00.596 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2024-09-21T15:02:00.596 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van" 2024-09-21T15:02:00.596 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:02:00.597 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.597 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps_queue": [ 2024-09-21T15:02:00.597 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.597 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 3, 2024-09-21T15:02:00.597 INFO:teuthology.orchestra.run.smithi032.stdout: "snaps": [ 2024-09-21T15:02:00.597 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.597 INFO:teuthology.orchestra.run.smithi032.stdout: "begin": 10, 2024-09-21T15:02:00.597 INFO:teuthology.orchestra.run.smithi032.stdout: "length": 1 2024-09-21T15:02:00.597 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.597 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.597 INFO:teuthology.orchestra.run.smithi032.stdout: "begin": 13, 2024-09-21T15:02:00.597 INFO:teuthology.orchestra.run.smithi032.stdout: "length": 1 2024-09-21T15:02:00.598 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.598 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.598 INFO:teuthology.orchestra.run.smithi032.stdout: "begin": 16, 2024-09-21T15:02:00.598 INFO:teuthology.orchestra.run.smithi032.stdout: "length": 1 2024-09-21T15:02:00.598 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.598 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.598 INFO:teuthology.orchestra.run.smithi032.stdout: "begin": 19, 2024-09-21T15:02:00.598 INFO:teuthology.orchestra.run.smithi032.stdout: "length": 1 2024-09-21T15:02:00.598 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.598 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.598 INFO:teuthology.orchestra.run.smithi032.stdout: "begin": 22, 2024-09-21T15:02:00.598 INFO:teuthology.orchestra.run.smithi032.stdout: "length": 1 2024-09-21T15:02:00.598 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:02:00.598 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:02:00.599 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:02:00.599 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-21T15:02:00.599 INFO:teuthology.orchestra.run.smithi032.stdout: "new_removed_snaps": [ 2024-09-21T15:02:00.599 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.599 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 3, 2024-09-21T15:02:00.599 INFO:teuthology.orchestra.run.smithi032.stdout: "snaps": [ 2024-09-21T15:02:00.599 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.599 INFO:teuthology.orchestra.run.smithi032.stdout: "begin": 10, 2024-09-21T15:02:00.599 INFO:teuthology.orchestra.run.smithi032.stdout: "length": 1 2024-09-21T15:02:00.599 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:02:00.599 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:02:00.599 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:02:00.599 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-21T15:02:00.599 INFO:teuthology.orchestra.run.smithi032.stdout: "new_purged_snaps": [ 2024-09-21T15:02:00.599 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.600 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 3, 2024-09-21T15:02:00.600 INFO:teuthology.orchestra.run.smithi032.stdout: "snaps": [ 2024-09-21T15:02:00.600 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.600 INFO:teuthology.orchestra.run.smithi032.stdout: "begin": 25, 2024-09-21T15:02:00.600 INFO:teuthology.orchestra.run.smithi032.stdout: "length": 1 2024-09-21T15:02:00.600 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.600 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.600 INFO:teuthology.orchestra.run.smithi032.stdout: "begin": 28, 2024-09-21T15:02:00.600 INFO:teuthology.orchestra.run.smithi032.stdout: "length": 1 2024-09-21T15:02:00.600 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-09-21T15:02:00.600 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-09-21T15:02:00.600 INFO:teuthology.orchestra.run.smithi032.stdout: "begin": 31, 2024-09-21T15:02:00.600 INFO:teuthology.orchestra.run.smithi032.stdout: "length": 1 2024-09-21T15:02:00.600 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:02:00.601 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-09-21T15:02:00.601 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:02:00.601 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-09-21T15:02:00.601 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_node_flags": {}, 2024-09-21T15:02:00.601 INFO:teuthology.orchestra.run.smithi032.stdout: "device_class_flags": {}, 2024-09-21T15:02:00.601 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode": { 2024-09-21T15:02:00.601 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_enabled": false, 2024-09-21T15:02:00.601 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_bucket_count": 0, 2024-09-21T15:02:00.601 INFO:teuthology.orchestra.run.smithi032.stdout: "degraded_stretch_mode": 0, 2024-09-21T15:02:00.601 INFO:teuthology.orchestra.run.smithi032.stdout: "recovering_stretch_mode": 0, 2024-09-21T15:02:00.601 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_bucket": 0 2024-09-21T15:02:00.601 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-09-21T15:02:00.601 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:02:00.603 INFO:tasks.thrashosds.thrasher:j is {'epoch': 228, 'fsid': 'caf51e55-9234-4a26-bcd2-d60811962560', 'created': '2024-09-21T14:57:50.338822+0000', 'modified': '2024-09-21T15:02:00.176499+0000', 'last_up_change': '2024-09-21T15:01:34.172764+0000', 'last_in_change': '2024-09-21T15:01:21.319966+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T14:58:01.842810+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.8952350616455078, 'average_primary_affinity': 0.8999999761581421, 'average_primary_affinity_weighted': 0.8999999761581421}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-21T14:58:08.284925+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 22, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 5, 'score_stable': 5, 'optimal_score': 0.75, 'raw_score_acting': 3.75, 'raw_score_stable': 3.75, 'primary_affinity_weighted': 0.8952350616455078, 'average_primary_affinity': 0.8999999761581421, 'average_primary_affinity_weighted': 0.8999999761581421}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-21T14:58:17.510709+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '228', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 149, 'snap_epoch': 228, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.4900000095367432, 'score_stable': 1.4900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.309999942779541, 'raw_score_stable': 1.309999942779541, 'primary_affinity_weighted': 0.8952350616455078, 'average_primary_affinity': 0.8999999761581421, 'average_primary_affinity_weighted': 0.8999999761581421}}], 'osds': [{'osd': 0, 'uuid': 'a4a73316-3a94-42e9-9a51-deb46a66bb06', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 192, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6808', 'nonce': 4016672243}, {'type': 'v1', 'addr': '172.21.15.32:6809', 'nonce': 4016672243}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6810', 'nonce': 4016672243}, {'type': 'v1', 'addr': '172.21.15.32:6811', 'nonce': 4016672243}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6814', 'nonce': 4016672243}, {'type': 'v1', 'addr': '172.21.15.32:6815', 'nonce': 4016672243}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6812', 'nonce': 4016672243}, {'type': 'v1', 'addr': '172.21.15.32:6813', 'nonce': 4016672243}]}, 'public_addr': '172.21.15.32:6809/4016672243', 'cluster_addr': '172.21.15.32:6811/4016672243', 'heartbeat_back_addr': '172.21.15.32:6815/4016672243', 'heartbeat_front_addr': '172.21.15.32:6813/4016672243', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '006757a0-94d1-4a9b-aef8-73634cc5828e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 223, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6824', 'nonce': 641090577}, {'type': 'v1', 'addr': '172.21.15.32:6825', 'nonce': 641090577}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6826', 'nonce': 641090577}, {'type': 'v1', 'addr': '172.21.15.32:6827', 'nonce': 641090577}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6830', 'nonce': 641090577}, {'type': 'v1', 'addr': '172.21.15.32:6831', 'nonce': 641090577}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6828', 'nonce': 641090577}, {'type': 'v1', 'addr': '172.21.15.32:6829', 'nonce': 641090577}]}, 'public_addr': '172.21.15.32:6825/641090577', 'cluster_addr': '172.21.15.32:6827/641090577', 'heartbeat_back_addr': '172.21.15.32:6831/641090577', 'heartbeat_front_addr': '172.21.15.32:6829/641090577', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '0b499a23-820e-4e6c-a558-adbf88b7014f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 223, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6800', 'nonce': 2609154414}, {'type': 'v1', 'addr': '172.21.15.32:6801', 'nonce': 2609154414}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6802', 'nonce': 2609154414}, {'type': 'v1', 'addr': '172.21.15.32:6803', 'nonce': 2609154414}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6806', 'nonce': 2609154414}, {'type': 'v1', 'addr': '172.21.15.32:6807', 'nonce': 2609154414}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6804', 'nonce': 2609154414}, {'type': 'v1', 'addr': '172.21.15.32:6805', 'nonce': 2609154414}]}, 'public_addr': '172.21.15.32:6801/2609154414', 'cluster_addr': '172.21.15.32:6803/2609154414', 'heartbeat_back_addr': '172.21.15.32:6807/2609154414', 'heartbeat_front_addr': '172.21.15.32:6805/2609154414', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '5a57b3dd-4a07-45b7-b159-3cac4c7eb262', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.833831787109375, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 192, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6816', 'nonce': 1462563376}, {'type': 'v1', 'addr': '172.21.15.32:6817', 'nonce': 1462563376}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6818', 'nonce': 1462563376}, {'type': 'v1', 'addr': '172.21.15.32:6819', 'nonce': 1462563376}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6822', 'nonce': 1462563376}, {'type': 'v1', 'addr': '172.21.15.32:6823', 'nonce': 1462563376}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6820', 'nonce': 1462563376}, {'type': 'v1', 'addr': '172.21.15.32:6821', 'nonce': 1462563376}]}, 'public_addr': '172.21.15.32:6817/1462563376', 'cluster_addr': '172.21.15.32:6819/1462563376', 'heartbeat_back_addr': '172.21.15.32:6823/1462563376', 'heartbeat_front_addr': '172.21.15.32:6821/1462563376', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '39a00807-7517-474e-8842-623dd7093cc5', 'up': 0, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 147, 'down_at': 202, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6801', 'nonce': 144412042}, {'type': 'v1', 'addr': '172.21.15.46:6803', 'nonce': 144412042}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6805', 'nonce': 144412042}, {'type': 'v1', 'addr': '172.21.15.46:6807', 'nonce': 144412042}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6812', 'nonce': 144412042}, {'type': 'v1', 'addr': '172.21.15.46:6814', 'nonce': 144412042}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6809', 'nonce': 144412042}, {'type': 'v1', 'addr': '172.21.15.46:6811', 'nonce': 144412042}]}, 'public_addr': '172.21.15.46:6803/144412042', 'cluster_addr': '172.21.15.46:6807/144412042', 'heartbeat_back_addr': '172.21.15.46:6814/144412042', 'heartbeat_front_addr': '172.21.15.46:6811/144412042', 'state': ['exists']}, {'osd': 5, 'uuid': 'edf9f045-ddeb-4504-a0fc-1a0e4e853550', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.4931182861328125, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 177, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6800', 'nonce': 849101442}, {'type': 'v1', 'addr': '172.21.15.46:6802', 'nonce': 849101442}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6804', 'nonce': 849101442}, {'type': 'v1', 'addr': '172.21.15.46:6806', 'nonce': 849101442}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6813', 'nonce': 849101442}, {'type': 'v1', 'addr': '172.21.15.46:6815', 'nonce': 849101442}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6808', 'nonce': 849101442}, {'type': 'v1', 'addr': '172.21.15.46:6810', 'nonce': 849101442}]}, 'public_addr': '172.21.15.46:6802/849101442', 'cluster_addr': '172.21.15.46:6806/849101442', 'heartbeat_back_addr': '172.21.15.46:6815/849101442', 'heartbeat_front_addr': '172.21.15.46:6810/849101442', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'c820ea6b-88fe-4421-83ec-17c466691fea', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.834930419921875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 223, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6817', 'nonce': 847189172}, {'type': 'v1', 'addr': '172.21.15.46:6818', 'nonce': 847189172}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6820', 'nonce': 847189172}, {'type': 'v1', 'addr': '172.21.15.46:6822', 'nonce': 847189172}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6827', 'nonce': 847189172}, {'type': 'v1', 'addr': '172.21.15.46:6828', 'nonce': 847189172}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6823', 'nonce': 847189172}, {'type': 'v1', 'addr': '172.21.15.46:6825', 'nonce': 847189172}]}, 'public_addr': '172.21.15.46:6818/847189172', 'cluster_addr': '172.21.15.46:6822/847189172', 'heartbeat_back_addr': '172.21.15.46:6828/847189172', 'heartbeat_front_addr': '172.21.15.46:6825/847189172', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '3a2f82a4-28a0-4fcb-9d6b-2285fa4d07ac', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 223, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6816', 'nonce': 3894963553}, {'type': 'v1', 'addr': '172.21.15.46:6819', 'nonce': 3894963553}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6821', 'nonce': 3894963553}, {'type': 'v1', 'addr': '172.21.15.46:6824', 'nonce': 3894963553}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6830', 'nonce': 3894963553}, {'type': 'v1', 'addr': '172.21.15.46:6831', 'nonce': 3894963553}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6826', 'nonce': 3894963553}, {'type': 'v1', 'addr': '172.21.15.46:6829', 'nonce': 3894963553}]}, 'public_addr': '172.21.15.46:6819/3894963553', 'cluster_addr': '172.21.15.46:6824/3894963553', 'heartbeat_back_addr': '172.21.15.46:6831/3894963553', 'heartbeat_front_addr': '172.21.15.46:6829/3894963553', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:57:57.458317+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:57:57.907624+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:57:57.278817+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:57:57.500449+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-21T14:57:57.504325+0000', 'dead_epoch': 202}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:57:57.777525+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:57:57.698961+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T14:57:57.932625+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.2', 'osds': [2, 3]}, {'pgid': '3.4', 'osds': [4, 2]}, {'pgid': '3.8', 'osds': [6, 3]}, {'pgid': '3.a', 'osds': [1, 0]}], 'pg_upmap_items': [{'pgid': '3.1', 'mappings': [{'from': 2, 'to': 0}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 10, 'length': 1}, {'begin': 13, 'length': 1}, {'begin': 16, 'length': 1}, {'begin': 19, 'length': 1}, {'begin': 22, 'length': 1}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 10, 'length': 1}]}], 'new_purged_snaps': [{'pool': 3, 'snaps': [{'begin': 25, 'length': 1}, {'begin': 28, 'length': 1}, {'begin': 31, 'length': 1}]}], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-21T15:02:00.603 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T15:02:00.604 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:02:00.604 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:02:00.605 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T15:02:00.650 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:02:00.649+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:02:00.715 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:02:00.715 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:02:00.720 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:02:00.715+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:02:00.725 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T15:02:00.822 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:02:00.821+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:02:00.828 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:02:00.828 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:02:00.838 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T15:02:00.919 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-21T15:02:00.922 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-09-21T15:02:00.923 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:02:00.915+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:02:00.934 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":341,"stamp":"2024-09-21T15:02:00.179238+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":112382722,"num_objects":347,"num_object_clones":0,"num_object_copies":694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":347,"num_whiteouts":0,"num_read":172813,"num_read_kb":1237182,"num_write":33914,"num_write_kb":943169,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3141,"num_bytes_recovered":212783659,"num_keys_recovered":305,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22087,"ondisk_log_size":22087,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":404300,"kb_used_data":29600,"kb_used_omap":10,"kb_used_meta":374389,"kb_avail":660198580,"statfs":{"total":676457349120,"available":676043345920,"internally_reserved":0,"allocated":30310400,"data_stored":130719761,"data_compressed":1501764,"data_compressed_allocated":10035200,"data_compressed_original":120678111,"omap_allocated":11150,"internal_metadata":383374450},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8365044,"num_objects":-14,"num_object_clones":0,"num_object_copies":-28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-14,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":27,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.992092"},"pg_stats":[{"pgid":"3.1f","version":"222'570","reported_seq":3821,"reported_epoch":223,"state":"peering","last_fresh":"2024-09-21T15:01:55.193473+0000","last_change":"2024-09-21T15:01:55.193473+0000","last_active":"2024-09-21T15:01:55.193473+0000","last_peered":"2024-09-21T15:01:54.203118+0000","last_clean":"2024-09-21T15:01:54.203118+0000","last_became_active":"2024-09-21T14:58:19.410055+0000","last_became_peered":"2024-09-21T14:58:19.410055+0000","last_unstale":"2024-09-21T15:01:55.193473+0000","last_undegraded":"2024-09-21T15:01:55.193473+0000","last_fullsized":"2024-09-21T15:01:55.193473+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":570,"log_dups_size":0,"ondisk_log_size":570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:24:43.708649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021268839999999999,"stat_sum":{"num_bytes":1212432,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":3580,"num_read_kb":43067,"num_write":952,"num_write_kb":36430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"19","length":"1"},{"start":"1c","length":"1"},{"start":"1f","length":"1"},{"start":"22","length":"1"},{"start":"25","length":"1"},{"start":"28","length":"1"}]},{"pgid":"3.1e","version":"222'496","reported_seq":3442,"reported_epoch":223,"state":"peering","last_fresh":"2024-09-21T15:01:55.186299+0000","last_change":"2024-09-21T15:01:55.186299+0000","last_active":"2024-09-21T15:01:55.186299+0000","last_peered":"2024-09-21T15:01:54.332650+0000","last_clean":"2024-09-21T15:01:54.332650+0000","last_became_active":"2024-09-21T15:01:06.168726+0000","last_became_peered":"2024-09-21T15:01:06.168726+0000","last_unstale":"2024-09-21T15:01:55.186299+0000","last_undegraded":"2024-09-21T15:01:55.186299+0000","last_fullsized":"2024-09-21T15:01:55.186299+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":496,"log_dups_size":0,"ondisk_log_size":496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:16:09.174437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018681150000000001,"stat_sum":{"num_bytes":5032958,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":3420,"num_read_kb":23407,"num_write":632,"num_write_kb":18445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":148,"num_bytes_recovered":11635695,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"19","length":"1"},{"start":"1c","length":"1"},{"start":"1f","length":"1"},{"start":"22","length":"1"},{"start":"25","length":"1"},{"start":"28","length":"1"}]},{"pgid":"3.1d","version":"223'330","reported_seq":4725,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-09-21T15:01:55.324233+0000","last_change":"2024-09-21T15:01:55.187816+0000","last_active":"2024-09-21T15:01:55.324233+0000","last_peered":"2024-09-21T15:01:55.324233+0000","last_clean":"2024-09-21T15:01:55.324233+0000","last_became_active":"2024-09-21T14:58:19.411387+0000","last_became_peered":"2024-09-21T14:58:19.411387+0000","last_unstale":"2024-09-21T15:01:55.324233+0000","last_undegraded":"2024-09-21T15:01:55.324233+0000","last_fullsized":"2024-09-21T15:01:55.324233+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":330,"log_dups_size":0,"ondisk_log_size":330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:37:57.597069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040585070000000003,"stat_sum":{"num_bytes":5539144,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":3878,"num_read_kb":28288,"num_write":524,"num_write_kb":26606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"16","length":"1"},{"start":"19","length":"1"},{"start":"1c","length":"1"},{"start":"1f","length":"1"}]},{"pgid":"3.1c","version":"223'556","reported_seq":4217,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-09-21T15:01:55.316631+0000","last_change":"2024-09-21T15:01:55.195667+0000","last_active":"2024-09-21T15:01:55.316631+0000","last_peered":"2024-09-21T15:01:55.316631+0000","last_clean":"2024-09-21T15:01:55.316631+0000","last_became_active":"2024-09-21T14:58:19.411306+0000","last_became_peered":"2024-09-21T14:58:19.411306+0000","last_unstale":"2024-09-21T15:01:55.316631+0000","last_undegraded":"2024-09-21T15:01:55.316631+0000","last_fullsized":"2024-09-21T15:01:55.316631+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":556,"log_dups_size":0,"ondisk_log_size":556,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:20:27.079795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020831280000000001,"stat_sum":{"num_bytes":2012585,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":4018,"num_read_kb":24645,"num_write":841,"num_write_kb":25228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"16","length":"1"},{"start":"19","length":"1"},{"start":"1c","length":"1"},{"start":"1f","length":"1"}]},{"pgid":"3.1b","version":"223'527","reported_seq":4220,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-09-21T15:01:55.305923+0000","last_change":"2024-09-21T15:01:55.193692+0000","last_active":"2024-09-21T15:01:55.305923+0000","last_peered":"2024-09-21T15:01:55.305923+0000","last_clean":"2024-09-21T15:01:55.305923+0000","last_became_active":"2024-09-21T15:01:25.176460+0000","last_became_peered":"2024-09-21T15:01:25.176460+0000","last_unstale":"2024-09-21T15:01:55.305923+0000","last_undegraded":"2024-09-21T15:01:55.305923+0000","last_fullsized":"2024-09-21T15:01:55.305923+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":527,"log_dups_size":0,"ondisk_log_size":527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:57.431201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001624499,"stat_sum":{"num_bytes":1228800,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":3671,"num_read_kb":32697,"num_write":699,"num_write_kb":22582,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":248,"num_bytes_recovered":13616631,"num_keys_recovered":33,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"16","length":"1"},{"start":"19","length":"1"},{"start":"1c","length":"1"},{"start":"1f","length":"1"}]},{"pgid":"3.1a","version":"223'456","reported_seq":4664,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-09-21T15:01:55.316987+0000","last_change":"2024-09-21T15:01:55.195374+0000","last_active":"2024-09-21T15:01:55.316987+0000","last_peered":"2024-09-21T15:01:55.316987+0000","last_clean":"2024-09-21T15:01:55.316987+0000","last_became_active":"2024-09-21T14:58:19.409415+0000","last_became_peered":"2024-09-21T14:58:19.409415+0000","last_unstale":"2024-09-21T15:01:55.316987+0000","last_undegraded":"2024-09-21T15:01:55.316987+0000","last_fullsized":"2024-09-21T15:01:55.316987+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":456,"log_dups_size":0,"ondisk_log_size":456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:28:46.336301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00192158,"stat_sum":{"num_bytes":2195475,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":4180,"num_read_kb":28162,"num_write":678,"num_write_kb":23870,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"16","length":"1"},{"start":"19","length":"1"},{"start":"1c","length":"1"},{"start":"1f","length":"1"}]},{"pgid":"3.19","version":"223'787","reported_seq":8564,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-09-21T15:01:55.301691+0000","last_change":"2024-09-21T15:01:55.194463+0000","last_active":"2024-09-21T15:01:55.301691+0000","last_peered":"2024-09-21T15:01:55.301691+0000","last_clean":"2024-09-21T15:01:55.301691+0000","last_became_active":"2024-09-21T15:01:25.179729+0000","last_became_peered":"2024-09-21T15:01:25.179729+0000","last_unstale":"2024-09-21T15:01:55.301691+0000","last_undegraded":"2024-09-21T15:01:55.301691+0000","last_fullsized":"2024-09-21T15:01:55.301691+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":787,"log_dups_size":0,"ondisk_log_size":787,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:15:24.357933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010778210000000001,"stat_sum":{"num_bytes":3491217,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":9210,"num_read_kb":67907,"num_write":1219,"num_write_kb":29915,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":261,"num_bytes_recovered":23509073,"num_keys_recovered":66,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"16","length":"1"},{"start":"19","length":"1"},{"start":"1c","length":"1"},{"start":"1f","length":"1"}]},{"pgid":"3.18","version":"222'656","reported_seq":8635,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-09-21T15:01:55.186574+0000","last_change":"2024-09-21T15:01:55.187602+0000","last_active":"2024-09-21T15:01:55.186574+0000","last_peered":"2024-09-21T15:01:55.186574+0000","last_clean":"2024-09-21T15:01:55.186574+0000","last_became_active":"2024-09-21T14:58:19.409371+0000","last_became_peered":"2024-09-21T14:58:19.409371+0000","last_unstale":"2024-09-21T15:01:55.186574+0000","last_undegraded":"2024-09-21T15:01:55.186574+0000","last_fullsized":"2024-09-21T15:01:55.186574+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":656,"log_dups_size":0,"ondisk_log_size":656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:35:27.143445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001243638,"stat_sum":{"num_bytes":2115111,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":9063,"num_read_kb":45115,"num_write":971,"num_write_kb":30610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"16","length":"1"},{"start":"19","length":"1"},{"start":"1c","length":"1"},{"start":"1f","length":"1"}]},{"pgid":"3.17","version":"223'407","reported_seq":3944,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-09-21T15:01:55.187086+0000","last_change":"2024-09-21T15:01:55.187086+0000","last_active":"2024-09-21T15:01:55.187086+0000","last_peered":"2024-09-21T15:01:55.187086+0000","last_clean":"2024-09-21T15:01:55.187086+0000","last_became_active":"2024-09-21T14:58:19.410306+0000","last_became_peered":"2024-09-21T14:58:19.410306+0000","last_unstale":"2024-09-21T15:01:55.187086+0000","last_undegraded":"2024-09-21T15:01:55.187086+0000","last_fullsized":"2024-09-21T15:01:55.187086+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":407,"log_dups_size":0,"ondisk_log_size":407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:33:50.089165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058904300000000003,"stat_sum":{"num_bytes":1569422,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":3380,"num_read_kb":28207,"num_write":763,"num_write_kb":34633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"16","length":"1"},{"start":"19","length":"1"},{"start":"1c","length":"1"},{"start":"1f","length":"1"}]},{"pgid":"3.16","version":"223'1023","reported_seq":6019,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-09-21T15:01:55.318773+0000","last_change":"2024-09-21T15:01:55.185994+0000","last_active":"2024-09-21T15:01:55.318773+0000","last_peered":"2024-09-21T15:01:55.318773+0000","last_clean":"2024-09-21T15:01:55.318773+0000","last_became_active":"2024-09-21T15:01:06.166756+0000","last_became_peered":"2024-09-21T15:01:06.166756+0000","last_unstale":"2024-09-21T15:01:55.318773+0000","last_undegraded":"2024-09-21T15:01:55.318773+0000","last_fullsized":"2024-09-21T15:01:55.318773+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":1023,"log_dups_size":0,"ondisk_log_size":1023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:19:04.988784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048923300000000005,"stat_sum":{"num_bytes":2089346,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":7900,"num_read_kb":43848,"num_write":1596,"num_write_kb":28368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":271,"num_bytes_recovered":16821266,"num_keys_recovered":55,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"16","length":"1"},{"start":"19","length":"1"},{"start":"1c","length":"1"},{"start":"1f","length":"1"}]},{"pgid":"3.15","version":"223'800","reported_seq":3897,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-09-21T15:01:55.304129+0000","last_change":"2024-09-21T15:01:55.192374+0000","last_active":"2024-09-21T15:01:55.304129+0000","last_peered":"2024-09-21T15:01:55.304129+0000","last_clean":"2024-09-21T15:01:55.304129+0000","last_became_active":"2024-09-21T14:58:19.410687+0000","last_became_peered":"2024-09-21T14:58:19.410687+0000","last_unstale":"2024-09-21T15:01:55.304129+0000","last_undegraded":"2024-09-21T15:01:55.304129+0000","last_fullsized":"2024-09-21T15:01:55.304129+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":800,"log_dups_size":0,"ondisk_log_size":800,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:21:42.226982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047389200000000001,"stat_sum":{"num_bytes":2927408,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":4724,"num_read_kb":49745,"num_write":1323,"num_write_kb":33334,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"16","length":"1"},{"start":"19","length":"1"},{"start":"1c","length":"1"},{"start":"1f","length":"1"}]},{"pgid":"3.14","version":"220'509","reported_seq":4115,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-09-21T15:01:55.186097+0000","last_change":"2024-09-21T15:01:55.186849+0000","last_active":"2024-09-21T15:01:55.186097+0000","last_peered":"2024-09-21T15:01:55.186097+0000","last_clean":"2024-09-21T15:01:55.186097+0000","last_became_active":"2024-09-21T14:58:19.407310+0000","last_became_peered":"2024-09-21T14:58:19.407310+0000","last_unstale":"2024-09-21T15:01:55.186097+0000","last_undegraded":"2024-09-21T15:01:55.186097+0000","last_fullsized":"2024-09-21T15:01:55.186097+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":509,"log_dups_size":0,"ondisk_log_size":509,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:47:46.079115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038495080000000002,"stat_sum":{"num_bytes":677874,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":3722,"num_read_kb":43297,"num_write":693,"num_write_kb":33466,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"16","length":"1"},{"start":"19","length":"1"},{"start":"1c","length":"1"},{"start":"1f","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":217,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-21T15:01:54.329131+0000","last_change":"2024-09-21T14:58:11.348177+0000","last_active":"2024-09-21T15:01:54.329131+0000","last_peered":"2024-09-21T15:01:54.329131+0000","last_clean":"2024-09-21T15:01:54.329131+0000","last_became_active":"2024-09-21T14:58:09.328417+0000","last_became_peered":"2024-09-21T14:58:09.328417+0000","last_unstale":"2024-09-21T15:01:54.329131+0000","last_undegraded":"2024-09-21T15:01:54.329131+0000","last_fullsized":"2024-09-21T15:01:54.329131+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:52:22.671185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018917490000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"224'559","reported_seq":6394,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T15:01:56.184931+0000","last_change":"2024-09-21T15:01:55.192217+0000","last_active":"2024-09-21T15:01:56.184931+0000","last_peered":"2024-09-21T15:01:56.184931+0000","last_clean":"2024-09-21T15:01:56.184931+0000","last_became_active":"2024-09-21T15:01:25.173568+0000","last_became_peered":"2024-09-21T15:01:25.173568+0000","last_unstale":"2024-09-21T15:01:56.184931+0000","last_undegraded":"2024-09-21T15:01:56.184931+0000","last_fullsized":"2024-09-21T15:01:56.184931+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":559,"log_dups_size":0,"ondisk_log_size":559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:53:08.599143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001317311,"stat_sum":{"num_bytes":7095361,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":6109,"num_read_kb":52874,"num_write":655,"num_write_kb":27950,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":202,"num_bytes_recovered":11818795,"num_keys_recovered":18,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"16","length":"1"},{"start":"19","length":"1"},{"start":"1c","length":"1"},{"start":"1f","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":64,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-21T15:01:54.212182+0000","last_change":"2024-09-21T15:01:22.344040+0000","last_active":"2024-09-21T15:01:54.212182+0000","last_peered":"2024-09-21T15:01:54.212182+0000","last_clean":"2024-09-21T15:01:54.212182+0000","last_became_active":"2024-09-21T15:01:22.343688+0000","last_became_peered":"2024-09-21T15:01:22.343688+0000","last_unstale":"2024-09-21T15:01:54.212182+0000","last_undegraded":"2024-09-21T15:01:54.212182+0000","last_fullsized":"2024-09-21T15:01:54.212182+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:11:18.042935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"223'956","reported_seq":4192,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-09-21T15:01:55.314328+0000","last_change":"2024-09-21T15:01:55.183138+0000","last_active":"2024-09-21T15:01:55.314328+0000","last_peered":"2024-09-21T15:01:55.314328+0000","last_clean":"2024-09-21T15:01:55.314328+0000","last_became_active":"2024-09-21T15:01:25.173798+0000","last_became_peered":"2024-09-21T15:01:25.173798+0000","last_unstale":"2024-09-21T15:01:55.314328+0000","last_undegraded":"2024-09-21T15:01:55.314328+0000","last_fullsized":"2024-09-21T15:01:55.314328+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":956,"log_dups_size":0,"ondisk_log_size":956,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:03:31.905691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020184999999999999,"stat_sum":{"num_bytes":4150994,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":5418,"num_read_kb":31174,"num_write":1374,"num_write_kb":25276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":260,"num_bytes_recovered":12566413,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"16","length":"1"},{"start":"19","length":"1"},{"start":"1c","length":"1"},{"start":"1f","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":241,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-21T15:01:54.329592+0000","last_change":"2024-09-21T15:01:22.346021+0000","last_active":"2024-09-21T15:01:54.329592+0000","last_peered":"2024-09-21T15:01:54.329592+0000","last_clean":"2024-09-21T15:01:54.329592+0000","last_became_active":"2024-09-21T15:01:22.345643+0000","last_became_peered":"2024-09-21T15:01:22.345643+0000","last_unstale":"2024-09-21T15:01:54.329592+0000","last_undegraded":"2024-09-21T15:01:54.329592+0000","last_fullsized":"2024-09-21T15:01:54.329592+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:55:55.681542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019338770000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"222'665","reported_seq":5516,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-09-21T15:01:55.181859+0000","last_change":"2024-09-21T15:01:55.182536+0000","last_active":"2024-09-21T15:01:55.181859+0000","last_peered":"2024-09-21T15:01:55.181859+0000","last_clean":"2024-09-21T15:01:55.181859+0000","last_became_active":"2024-09-21T15:01:25.172697+0000","last_became_peered":"2024-09-21T15:01:25.172697+0000","last_unstale":"2024-09-21T15:01:55.181859+0000","last_undegraded":"2024-09-21T15:01:55.181859+0000","last_fullsized":"2024-09-21T15:01:55.181859+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":665,"log_dups_size":0,"ondisk_log_size":665,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:34:57.331662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019702069999999999,"stat_sum":{"num_bytes":2441216,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":5178,"num_read_kb":38459,"num_write":1123,"num_write_kb":41306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":203,"num_bytes_recovered":25796920,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"16","length":"1"},{"start":"19","length":"1"},{"start":"1c","length":"1"},{"start":"1f","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":3,"reported_epoch":223,"state":"peering","last_fresh":"2024-09-21T15:01:55.181835+0000","last_change":"2024-09-21T15:01:55.181835+0000","last_active":"2024-09-21T14:58:08.305870+0000","last_peered":"2024-09-21T14:58:08.305870+0000","last_clean":"2024-09-21T14:58:08.305870+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T15:01:55.181835+0000","last_undegraded":"2024-09-21T15:01:55.181835+0000","last_fullsized":"2024-09-21T15:01:55.181835+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:51:39.223781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"223'720","reported_seq":2790,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-09-21T15:01:55.262670+0000","last_change":"2024-09-21T15:01:55.184561+0000","last_active":"2024-09-21T15:01:55.262670+0000","last_peered":"2024-09-21T15:01:55.262670+0000","last_clean":"2024-09-21T15:01:55.262670+0000","last_became_active":"2024-09-21T14:58:19.408971+0000","last_became_peered":"2024-09-21T14:58:19.408971+0000","last_unstale":"2024-09-21T15:01:55.262670+0000","last_undegraded":"2024-09-21T15:01:55.262670+0000","last_fullsized":"2024-09-21T15:01:55.262670+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":720,"log_dups_size":0,"ondisk_log_size":720,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:29:52.642121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001635435,"stat_sum":{"num_bytes":2719760,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":3599,"num_read_kb":20654,"num_write":1218,"num_write_kb":20679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"16","length":"1"},{"start":"19","length":"1"},{"start":"1c","length":"1"},{"start":"1f","length":"1"}]},{"pgid":"3.3","version":"223'973","reported_seq":3574,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-09-21T15:01:55.275938+0000","last_change":"2024-09-21T15:01:55.194363+0000","last_active":"2024-09-21T15:01:55.275938+0000","last_peered":"2024-09-21T15:01:55.275938+0000","last_clean":"2024-09-21T15:01:55.275938+0000","last_became_active":"2024-09-21T14:58:19.417453+0000","last_became_peered":"2024-09-21T14:58:19.417453+0000","last_unstale":"2024-09-21T15:01:55.275938+0000","last_undegraded":"2024-09-21T15:01:55.275938+0000","last_fullsized":"2024-09-21T15:01:55.275938+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":973,"log_dups_size":0,"ondisk_log_size":973,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:19:27.087272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020344830000000001,"stat_sum":{"num_bytes":5670640,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":4808,"num_read_kb":25247,"num_write":1530,"num_write_kb":21717,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"16","length":"1"},{"start":"19","length":"1"},{"start":"1c","length":"1"},{"start":"1f","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":219,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-21T15:01:54.332656+0000","last_change":"2024-09-21T14:58:11.347504+0000","last_active":"2024-09-21T15:01:54.332656+0000","last_peered":"2024-09-21T15:01:54.332656+0000","last_clean":"2024-09-21T15:01:54.332656+0000","last_became_active":"2024-09-21T14:58:09.775981+0000","last_became_peered":"2024-09-21T14:58:09.775981+0000","last_unstale":"2024-09-21T15:01:54.332656+0000","last_undegraded":"2024-09-21T15:01:54.332656+0000","last_fullsized":"2024-09-21T15:01:54.332656+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:47:16.286652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012568880000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"224'1082","reported_seq":3953,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T15:01:56.192507+0000","last_change":"2024-09-21T15:01:56.192004+0000","last_active":"2024-09-21T15:01:56.192507+0000","last_peered":"2024-09-21T15:01:56.192507+0000","last_clean":"2024-09-21T15:01:56.192507+0000","last_became_active":"2024-09-21T15:01:56.187944+0000","last_became_peered":"2024-09-21T15:01:56.187944+0000","last_unstale":"2024-09-21T15:01:56.192507+0000","last_undegraded":"2024-09-21T15:01:56.192507+0000","last_fullsized":"2024-09-21T15:01:56.192507+0000","mapping_epoch":223,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":1082,"log_dups_size":0,"ondisk_log_size":1082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:07:01.286162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038572600000000003,"stat_sum":{"num_bytes":2064475,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":6338,"num_read_kb":28795,"num_write":1667,"num_write_kb":26974,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"16","length":"1"},{"start":"19","length":"1"},{"start":"1c","length":"1"},{"start":"1f","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":219,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-09-21T15:01:55.190686+0000","last_change":"2024-09-21T14:58:11.342210+0000","last_active":"2024-09-21T15:01:55.190686+0000","last_peered":"2024-09-21T15:01:55.190686+0000","last_clean":"2024-09-21T15:01:55.190686+0000","last_became_active":"2024-09-21T14:58:09.330624+0000","last_became_peered":"2024-09-21T14:58:09.330624+0000","last_unstale":"2024-09-21T15:01:55.190686+0000","last_undegraded":"2024-09-21T15:01:55.190686+0000","last_fullsized":"2024-09-21T15:01:55.190686+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:29:43.229538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001051151,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"223'328","reported_seq":2486,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-09-21T15:01:55.181432+0000","last_change":"2024-09-21T15:01:55.181432+0000","last_active":"2024-09-21T15:01:55.181432+0000","last_peered":"2024-09-21T15:01:55.181432+0000","last_clean":"2024-09-21T15:01:55.181432+0000","last_became_active":"2024-09-21T15:01:53.193166+0000","last_became_peered":"2024-09-21T15:01:53.193166+0000","last_unstale":"2024-09-21T15:01:55.181432+0000","last_undegraded":"2024-09-21T15:01:55.181432+0000","last_fullsized":"2024-09-21T15:01:55.181432+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":328,"log_dups_size":0,"ondisk_log_size":328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:37:52.546654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097239500000000003,"stat_sum":{"num_bytes":2532166,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":2371,"num_read_kb":40193,"num_write":518,"num_write_kb":39390,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":366,"num_bytes_recovered":22723935,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"16","length":"1"},{"start":"19","length":"1"},{"start":"1c","length":"1"},{"start":"1f","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":217,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-21T15:01:54.328227+0000","last_change":"2024-09-21T14:58:11.346766+0000","last_active":"2024-09-21T15:01:54.328227+0000","last_peered":"2024-09-21T15:01:54.328227+0000","last_clean":"2024-09-21T15:01:54.328227+0000","last_became_active":"2024-09-21T14:58:09.329748+0000","last_became_peered":"2024-09-21T14:58:09.329748+0000","last_unstale":"2024-09-21T15:01:54.328227+0000","last_undegraded":"2024-09-21T15:01:54.328227+0000","last_fullsized":"2024-09-21T15:01:54.328227+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:31:20.267675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052698000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"223'1262","reported_seq":8309,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-09-21T15:01:55.185316+0000","last_change":"2024-09-21T15:01:55.185316+0000","last_active":"2024-09-21T15:01:55.185316+0000","last_peered":"2024-09-21T15:01:55.185316+0000","last_clean":"2024-09-21T15:01:55.185316+0000","last_became_active":"2024-09-21T14:58:41.401003+0000","last_became_peered":"2024-09-21T14:58:41.401003+0000","last_unstale":"2024-09-21T15:01:55.185316+0000","last_undegraded":"2024-09-21T15:01:55.185316+0000","last_fullsized":"2024-09-21T15:01:55.185316+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":1262,"log_dups_size":0,"ondisk_log_size":1262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:05:37.644835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036915599999999999,"stat_sum":{"num_bytes":9791509,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":11705,"num_read_kb":50815,"num_write":1943,"num_write_kb":24913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1403348,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"16","length":"1"},{"start":"19","length":"1"},{"start":"1c","length":"1"},{"start":"1f","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":218,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-21T15:01:54.328746+0000","last_change":"2024-09-21T14:58:11.347700+0000","last_active":"2024-09-21T15:01:54.328746+0000","last_peered":"2024-09-21T15:01:54.328746+0000","last_clean":"2024-09-21T15:01:54.328746+0000","last_became_active":"2024-09-21T14:58:09.438842+0000","last_became_peered":"2024-09-21T14:58:09.438842+0000","last_unstale":"2024-09-21T15:01:54.328746+0000","last_undegraded":"2024-09-21T15:01:54.328746+0000","last_fullsized":"2024-09-21T15:01:54.328746+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:41:34.626249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001426194,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":366,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-21T15:01:54.207341+0000","last_change":"2024-09-21T14:58:03.616055+0000","last_active":"2024-09-21T15:01:54.207341+0000","last_peered":"2024-09-21T15:01:54.207341+0000","last_clean":"2024-09-21T15:01:54.207341+0000","last_became_active":"2024-09-21T14:58:03.613214+0000","last_became_peered":"2024-09-21T14:58:03.613214+0000","last_unstale":"2024-09-21T15:01:54.207341+0000","last_undegraded":"2024-09-21T15:01:54.207341+0000","last_fullsized":"2024-09-21T15:01:54.207341+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:02.180162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:02.180162+0000","last_clean_scrub_stamp":"2024-09-21T14:58:02.180162+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:29:05.860361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"223'674","reported_seq":4629,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-09-21T15:01:55.310670+0000","last_change":"2024-09-21T15:01:55.182128+0000","last_active":"2024-09-21T15:01:55.310670+0000","last_peered":"2024-09-21T15:01:55.310670+0000","last_clean":"2024-09-21T15:01:55.310670+0000","last_became_active":"2024-09-21T15:01:32.177913+0000","last_became_peered":"2024-09-21T15:01:32.177913+0000","last_unstale":"2024-09-21T15:01:55.310670+0000","last_undegraded":"2024-09-21T15:01:55.310670+0000","last_fullsized":"2024-09-21T15:01:55.310670+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":674,"log_dups_size":0,"ondisk_log_size":674,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:47:09.376828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015716529999999999,"stat_sum":{"num_bytes":8067081,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":4404,"num_read_kb":44537,"num_write":1153,"num_write_kb":30502,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":413,"num_bytes_recovered":20409497,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"16","length":"1"},{"start":"19","length":"1"},{"start":"1c","length":"1"},{"start":"1f","length":"1"}]},{"pgid":"3.9","version":"223'965","reported_seq":3756,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-09-21T15:01:55.316429+0000","last_change":"2024-09-21T15:01:55.187409+0000","last_active":"2024-09-21T15:01:55.316429+0000","last_peered":"2024-09-21T15:01:55.316429+0000","last_clean":"2024-09-21T15:01:55.316429+0000","last_became_active":"2024-09-21T15:01:25.176267+0000","last_became_peered":"2024-09-21T15:01:25.176267+0000","last_unstale":"2024-09-21T15:01:55.316429+0000","last_undegraded":"2024-09-21T15:01:55.316429+0000","last_fullsized":"2024-09-21T15:01:55.316429+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":965,"log_dups_size":0,"ondisk_log_size":965,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:21:28.802799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039454529999999998,"stat_sum":{"num_bytes":2997870,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":4815,"num_read_kb":35700,"num_write":1587,"num_write_kb":33802,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":233,"num_bytes_recovered":14742839,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"16","length":"1"},{"start":"19","length":"1"},{"start":"1c","length":"1"},{"start":"1f","length":"1"}]},{"pgid":"3.a","version":"223'1013","reported_seq":4961,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-09-21T15:01:55.312998+0000","last_change":"2024-09-21T15:01:55.194046+0000","last_active":"2024-09-21T15:01:55.312998+0000","last_peered":"2024-09-21T15:01:55.312998+0000","last_clean":"2024-09-21T15:01:55.312998+0000","last_became_active":"2024-09-21T15:00:27.280971+0000","last_became_peered":"2024-09-21T15:00:27.280971+0000","last_unstale":"2024-09-21T15:01:55.312998+0000","last_undegraded":"2024-09-21T15:01:55.312998+0000","last_fullsized":"2024-09-21T15:01:55.312998+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":1013,"log_dups_size":0,"ondisk_log_size":1013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:26:07.617059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000860862,"stat_sum":{"num_bytes":7346712,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":7341,"num_read_kb":31141,"num_write":1407,"num_write_kb":33160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":9228123,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"16","length":"1"},{"start":"19","length":"1"},{"start":"1c","length":"1"},{"start":"1f","length":"1"}]},{"pgid":"3.b","version":"224'573","reported_seq":3373,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T15:01:56.198273+0000","last_change":"2024-09-21T15:01:55.191489+0000","last_active":"2024-09-21T15:01:56.198273+0000","last_peered":"2024-09-21T15:01:56.198273+0000","last_clean":"2024-09-21T15:01:56.198273+0000","last_became_active":"2024-09-21T14:58:19.410679+0000","last_became_peered":"2024-09-21T14:58:19.410679+0000","last_unstale":"2024-09-21T15:01:56.198273+0000","last_undegraded":"2024-09-21T15:01:56.198273+0000","last_fullsized":"2024-09-21T15:01:56.198273+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":573,"log_dups_size":0,"ondisk_log_size":573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:07:40.758932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070079599999999995,"stat_sum":{"num_bytes":4492585,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":3109,"num_read_kb":36761,"num_write":869,"num_write_kb":39649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"16","length":"1"},{"start":"19","length":"1"},{"start":"1c","length":"1"},{"start":"1f","length":"1"}]},{"pgid":"3.c","version":"223'481","reported_seq":4643,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-09-21T15:01:55.310283+0000","last_change":"2024-09-21T15:01:55.191792+0000","last_active":"2024-09-21T15:01:55.310283+0000","last_peered":"2024-09-21T15:01:55.310283+0000","last_clean":"2024-09-21T15:01:55.310283+0000","last_became_active":"2024-09-21T14:59:25.130057+0000","last_became_peered":"2024-09-21T14:59:25.130057+0000","last_unstale":"2024-09-21T15:01:55.310283+0000","last_undegraded":"2024-09-21T15:01:55.310283+0000","last_fullsized":"2024-09-21T15:01:55.310283+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":481,"log_dups_size":0,"ondisk_log_size":481,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:53:33.752550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051520400000000001,"stat_sum":{"num_bytes":6499853,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":4513,"num_read_kb":28786,"num_write":717,"num_write_kb":24616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":9059600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"16","length":"1"},{"start":"19","length":"1"},{"start":"1c","length":"1"},{"start":"1f","length":"1"}]},{"pgid":"3.d","version":"223'535","reported_seq":5291,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-09-21T15:01:55.312850+0000","last_change":"2024-09-21T15:01:55.188016+0000","last_active":"2024-09-21T15:01:55.312850+0000","last_peered":"2024-09-21T15:01:55.312850+0000","last_clean":"2024-09-21T15:01:55.312850+0000","last_became_active":"2024-09-21T14:58:19.407884+0000","last_became_peered":"2024-09-21T14:58:19.407884+0000","last_unstale":"2024-09-21T15:01:55.312850+0000","last_undegraded":"2024-09-21T15:01:55.312850+0000","last_fullsized":"2024-09-21T15:01:55.312850+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":535,"log_dups_size":0,"ondisk_log_size":535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:28:57.111615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037571969999999999,"stat_sum":{"num_bytes":1540096,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":4968,"num_read_kb":55828,"num_write":936,"num_write_kb":38650,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"16","length":"1"},{"start":"19","length":"1"},{"start":"1c","length":"1"},{"start":"1f","length":"1"}]},{"pgid":"3.e","version":"222'869","reported_seq":3153,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-09-21T15:01:55.185396+0000","last_change":"2024-09-21T15:01:55.185787+0000","last_active":"2024-09-21T15:01:55.185396+0000","last_peered":"2024-09-21T15:01:55.185396+0000","last_clean":"2024-09-21T15:01:55.185396+0000","last_became_active":"2024-09-21T15:01:05.172427+0000","last_became_peered":"2024-09-21T15:01:05.172427+0000","last_unstale":"2024-09-21T15:01:55.185396+0000","last_undegraded":"2024-09-21T15:01:55.185396+0000","last_fullsized":"2024-09-21T15:01:55.185396+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":869,"log_dups_size":0,"ondisk_log_size":869,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:28:18.710621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058347100000000001,"stat_sum":{"num_bytes":2803249,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":4262,"num_read_kb":17288,"num_write":1499,"num_write_kb":17970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":138,"num_bytes_recovered":7486496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"16","length":"1"},{"start":"19","length":"1"},{"start":"1c","length":"1"},{"start":"1f","length":"1"}]},{"pgid":"3.f","version":"222'506","reported_seq":6125,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-09-21T15:01:55.191888+0000","last_change":"2024-09-21T15:01:55.192082+0000","last_active":"2024-09-21T15:01:55.191888+0000","last_peered":"2024-09-21T15:01:55.191888+0000","last_clean":"2024-09-21T15:01:55.191888+0000","last_became_active":"2024-09-21T14:58:19.410406+0000","last_became_peered":"2024-09-21T14:58:19.410406+0000","last_unstale":"2024-09-21T15:01:55.191888+0000","last_undegraded":"2024-09-21T15:01:55.191888+0000","last_fullsized":"2024-09-21T15:01:55.191888+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:59:16.026939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048090399999999999,"stat_sum":{"num_bytes":2891939,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":5776,"num_read_kb":56249,"num_write":789,"num_write_kb":36768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"16","length":"1"},{"start":"19","length":"1"},{"start":"1c","length":"1"},{"start":"1f","length":"1"}]},{"pgid":"3.10","version":"223'512","reported_seq":4581,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-09-21T15:01:55.194052+0000","last_change":"2024-09-21T15:01:55.194052+0000","last_active":"2024-09-21T15:01:55.194052+0000","last_peered":"2024-09-21T15:01:55.194052+0000","last_clean":"2024-09-21T15:01:55.194052+0000","last_became_active":"2024-09-21T14:58:19.409935+0000","last_became_peered":"2024-09-21T14:58:19.409935+0000","last_unstale":"2024-09-21T15:01:55.194052+0000","last_undegraded":"2024-09-21T15:01:55.194052+0000","last_fullsized":"2024-09-21T15:01:55.194052+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":512,"log_dups_size":0,"ondisk_log_size":512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:10:36.252629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095973399999999996,"stat_sum":{"num_bytes":2612825,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":3947,"num_read_kb":58692,"num_write":799,"num_write_kb":37279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"16","length":"1"},{"start":"19","length":"1"},{"start":"1c","length":"1"},{"start":"1f","length":"1"}]},{"pgid":"3.11","version":"223'613","reported_seq":5131,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-09-21T15:01:55.314831+0000","last_change":"2024-09-21T15:01:55.188512+0000","last_active":"2024-09-21T15:01:55.314831+0000","last_peered":"2024-09-21T15:01:55.314831+0000","last_clean":"2024-09-21T15:01:55.314831+0000","last_became_active":"2024-09-21T15:01:05.165955+0000","last_became_peered":"2024-09-21T15:01:05.165955+0000","last_unstale":"2024-09-21T15:01:55.314831+0000","last_undegraded":"2024-09-21T15:01:55.314831+0000","last_fullsized":"2024-09-21T15:01:55.314831+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":613,"log_dups_size":0,"ondisk_log_size":613,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:34:54.111119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029686159999999999,"stat_sum":{"num_bytes":2609500,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":5758,"num_read_kb":28485,"num_write":767,"num_write_kb":24185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":166,"num_bytes_recovered":11965028,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"16","length":"1"},{"start":"19","length":"1"},{"start":"1c","length":"1"},{"start":"1f","length":"1"}]},{"pgid":"3.12","version":"222'635","reported_seq":8470,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-09-21T15:01:55.181154+0000","last_change":"2024-09-21T15:01:55.181762+0000","last_active":"2024-09-21T15:01:55.181154+0000","last_peered":"2024-09-21T15:01:55.181154+0000","last_clean":"2024-09-21T15:01:55.181154+0000","last_became_active":"2024-09-21T15:00:07.155144+0000","last_became_peered":"2024-09-21T15:00:07.155144+0000","last_unstale":"2024-09-21T15:01:55.181154+0000","last_undegraded":"2024-09-21T15:01:55.181154+0000","last_fullsized":"2024-09-21T15:01:55.181154+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":635,"log_dups_size":0,"ondisk_log_size":635,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:58:00.133639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012993150000000001,"stat_sum":{"num_bytes":1963605,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":9098,"num_read_kb":37251,"num_write":1004,"num_write_kb":20896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"16","length":"1"},{"start":"19","length":"1"},{"start":"1c","length":"1"},{"start":"1f","length":"1"}]},{"pgid":"3.13","version":"223'929","reported_seq":6746,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-09-21T15:01:55.306165+0000","last_change":"2024-09-21T15:01:55.187414+0000","last_active":"2024-09-21T15:01:55.306165+0000","last_peered":"2024-09-21T15:01:55.306165+0000","last_clean":"2024-09-21T15:01:55.306165+0000","last_became_active":"2024-09-21T14:58:19.412030+0000","last_became_peered":"2024-09-21T14:58:19.412030+0000","last_unstale":"2024-09-21T15:01:55.306165+0000","last_undegraded":"2024-09-21T15:01:55.306165+0000","last_fullsized":"2024-09-21T15:01:55.306165+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":929,"log_dups_size":0,"ondisk_log_size":929,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:44:29.282391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0042060049999999996,"stat_sum":{"num_bytes":1409127,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":8434,"num_read_kb":59768,"num_write":1307,"num_write_kb":31582,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"16","length":"1"},{"start":"19","length":"1"},{"start":"1c","length":"1"},{"start":"1f","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":111792335,"num_objects":343,"num_object_clones":0,"num_object_copies":686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":343,"num_whiteouts":0,"num_read":172697,"num_read_kb":1237082,"num_write":33753,"num_write_kb":940751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3141,"num_bytes_recovered":212783659,"num_keys_recovered":305,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":11149312,"data_stored":116644300,"data_compressed":1501419,"data_compressed_allocated":9748480,"data_compressed_original":115899486,"omap_allocated":0,"internal_metadata":0},"log_size":21967,"ondisk_log_size":21967,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":10404,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509491,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34512,"kb_used_data":3392,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94337328,"statfs":{"total":96636764160,"available":96601423872,"internally_reserved":0,"allocated":3473408,"data_stored":9940680,"data_compressed":56002,"data_compressed_allocated":663552,"data_compressed_original":8558134,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.85499999999999998,"15min":0.84199999999999997},"min":{"1min":0.46500000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.872,"5min":0.93300000000000005,"15min":0.95199999999999996},"min":{"1min":0.54900000000000004,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.3839999999999999,"5min":1.917,"15min":1.917},"last":0.60499999999999998}]},{"osd":1,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.81000000000000005,"15min":0.77700000000000002},"min":{"1min":0.35499999999999998,"5min":0.312,"15min":0.312},"max":{"1min":1.2789999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.83199999999999996,"15min":0.82599999999999996},"min":{"1min":0.57399999999999995,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.208,"5min":2.2149999999999999,"15min":2.2149999999999999},"last":0.70299999999999996}]},{"osd":2,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.82199999999999995,"15min":0.81200000000000006},"min":{"1min":0.50900000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.169,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":0.872},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.93500000000000005,"15min":0.94299999999999995},"min":{"1min":0.55600000000000005,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.355,"5min":1.466,"15min":1.466},"last":1.05}]},{"osd":3,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.88500000000000001,"15min":0.86099999999999999},"min":{"1min":0.60999999999999999,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.401,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.97799999999999998,"15min":0.96899999999999997},"min":{"1min":0.68200000000000005,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.3420000000000001,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.91100000000000003}]},{"osd":4,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.92200000000000004,"15min":0.93300000000000005},"min":{"1min":0.42199999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.3089999999999999,"5min":2.7130000000000001,"15min":2.7130000000000001},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.94499999999999995,"15min":0.92600000000000005},"min":{"1min":0.39000000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.425,"5min":2.125,"15min":2.125},"last":0.69999999999999996}]},{"osd":5,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":1.026,"15min":1.022},"min":{"1min":0.51300000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.5800000000000001,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.94299999999999995,"5min":1.0409999999999999,"15min":1.05},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.3029999999999999,"5min":3.1880000000000002,"15min":3.1880000000000002},"last":0.65900000000000003}]},{"osd":6,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.133,"15min":1.149},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.5329999999999999,"5min":3.2730000000000001,"15min":3.2730000000000001},"last":0.84799999999999998},{"interface":"front","average":{"1min":1.095,"5min":1.0629999999999999,"15min":1.03},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.6990000000000001,"5min":2.2879999999999998,"15min":2.2879999999999998},"last":0.998}]}]},{"osd":6,"up_from":14,"seq":60129542194,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61644,"kb_used_data":4224,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94310196,"statfs":{"total":96636764160,"available":96573640704,"internally_reserved":0,"allocated":4325376,"data_stored":18461829,"data_compressed":246916,"data_compressed_allocated":1466368,"data_compressed_original":17069622,"omap_allocated":1593,"internal_metadata":58718663},"hb_peers":[0,1,2,3,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:01:08 2024","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.92500000000000004,"15min":0.95099999999999996},"min":{"1min":0.51900000000000002,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.2010000000000001,"5min":1.593,"15min":1.593},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.93300000000000005,"15min":0.94999999999999996},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.226,"5min":1.587,"15min":1.587},"last":0.93400000000000005}]},{"osd":1,"last update":"Sat Sep 21 15:01:08 2024","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.89400000000000002,"15min":0.91000000000000003},"min":{"1min":0.51700000000000002,"5min":0.502,"15min":0.502},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.90200000000000002,"15min":0.96199999999999997},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":1.036}]},{"osd":2,"last update":"Sat Sep 21 15:01:08 2024","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.96199999999999997,"15min":0.98399999999999999},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.389,"5min":1.758,"15min":1.758},"last":1.1220000000000001},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.90900000000000003,"15min":0.91300000000000003},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.444,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.73899999999999999}]},{"osd":3,"last update":"Sat Sep 21 15:01:08 2024","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.94799999999999995,"15min":0.95899999999999996},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.5329999999999999,"5min":1.835,"15min":1.835},"last":0.57399999999999995},{"interface":"front","average":{"1min":1.016,"5min":1.0920000000000001,"15min":1.127},"min":{"1min":0.64600000000000002,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.4670000000000001,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":1.002}]},{"osd":4,"last update":"Sat Sep 21 15:01:08 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.93500000000000005,"15min":0.99099999999999999},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":1.2,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.98499999999999999,"15min":1.0209999999999999},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.3759999999999999,"5min":1.897,"15min":1.897},"last":0.68300000000000005}]},{"osd":5,"last update":"Sat Sep 21 15:01:08 2024","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":1.0449999999999999,"15min":1.1100000000000001},"min":{"1min":0.63500000000000001,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.256,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.96599999999999997},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0660000000000001,"15min":1.087},"min":{"1min":0.65400000000000003,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.609,"5min":1.609,"15min":1.609},"last":0.88500000000000001}]},{"osd":7,"last update":"Sat Sep 21 15:01:08 2024","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.206,"15min":1.244},"min":{"1min":0.78900000000000003,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.3109999999999999,"5min":1.8660000000000001,"15min":1.8660000000000001},"last":1.0640000000000001},{"interface":"front","average":{"1min":1,"5min":1.1970000000000001,"15min":1.2749999999999999},"min":{"1min":0.69499999999999995,"5min":0.63,"15min":0.63},"max":{"1min":1.298,"5min":2.004,"15min":2.004},"last":0.85699999999999998}]}]},{"osd":5,"up_from":14,"seq":60129542195,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61232,"kb_used_data":3868,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94310608,"statfs":{"total":96636764160,"available":96574062592,"internally_reserved":0,"allocated":3960832,"data_stored":15148498,"data_compressed":159322,"data_compressed_allocated":1142784,"data_compressed_original":13753768,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,1,2,3,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:01:07 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.89900000000000002,"15min":0.98799999999999999},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.337,"5min":4.4800000000000004,"15min":4.4800000000000004},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.627,"5min":0.76800000000000002,"15min":0.80800000000000005},"min":{"1min":0.36099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.94999999999999996,"5min":1.181,"15min":1.181},"last":0.95299999999999996}]},{"osd":1,"last update":"Sat Sep 21 15:01:02 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.89600000000000002,"15min":0.92400000000000004},"min":{"1min":0.47699999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":2.7709999999999999,"5min":2.7709999999999999,"15min":2.7709999999999999},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.91400000000000003,"15min":0.93400000000000005},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":2.823,"5min":2.823,"15min":2.823},"last":0.82299999999999995}]},{"osd":2,"last update":"Sat Sep 21 15:01:02 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.91200000000000003,"15min":0.97399999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.159,"5min":3.419,"15min":3.419},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.87,"15min":0.89600000000000002},"min":{"1min":0.53400000000000003,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0780000000000001,"5min":1.423,"15min":1.423},"last":0.876}]},{"osd":3,"last update":"Sat Sep 21 15:01:02 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.80300000000000005,"15min":0.80700000000000005},"min":{"1min":0.58499999999999996,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.96699999999999997,"5min":1.054,"15min":1.0840000000000001},"min":{"1min":0.59899999999999998,"5min":0.435,"15min":0.435},"max":{"1min":1.272,"5min":2.6379999999999999,"15min":2.6379999999999999},"last":0.78900000000000003}]},{"osd":4,"last update":"Sat Sep 21 15:01:02 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.98899999999999999,"15min":1.016},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.5609999999999999,"5min":2.5470000000000002,"15min":2.5470000000000002},"last":1.351},{"interface":"front","average":{"1min":0.82099999999999995,"5min":1.0169999999999999,"15min":1.0780000000000001},"min":{"1min":0.39300000000000002,"5min":0.376,"15min":0.376},"max":{"1min":1.302,"5min":2.5099999999999998,"15min":2.5099999999999998},"last":0.86299999999999999}]},{"osd":6,"last update":"Sat Sep 21 15:01:02 2024","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.94299999999999995,"15min":0.96899999999999997},"min":{"1min":0.63,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.627},{"interface":"front","average":{"1min":1.004,"5min":1.2450000000000001,"15min":1.3520000000000001},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.425,"5min":4.2469999999999999,"15min":4.2469999999999999},"last":0.68999999999999995}]},{"osd":7,"last update":"Sat Sep 21 15:01:07 2024","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.2569999999999999,"15min":1.3420000000000001},"min":{"1min":0.52500000000000002,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.383,"5min":3.3010000000000002,"15min":3.3010000000000002},"last":0.92100000000000004},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.107,"15min":1.135},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.462,"5min":2.5880000000000001,"15min":2.5880000000000001},"last":1.0269999999999999}]}]},{"osd":4,"up_from":0,"seq":60129542189,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":3,"up_from":14,"seq":60129542195,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":75960,"kb_used_data":4652,"kb_used_omap":1,"kb_used_meta":71294,"kb_avail":94295880,"statfs":{"total":96636764160,"available":96558981120,"internally_reserved":0,"allocated":4763648,"data_stored":22222237,"data_compressed":302718,"data_compressed_allocated":1785856,"data_compressed_original":20748905,"omap_allocated":1592,"internal_metadata":73005512},"hb_peers":[0,1,2,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:01:09 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.625,"15min":0.63400000000000001},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.171,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.68200000000000005,"15min":0.70999999999999996},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.0700000000000001,"5min":1.919,"15min":1.919},"last":0.61899999999999999}]},{"osd":1,"last update":"Sat Sep 21 15:01:09 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.66400000000000003,"15min":0.67200000000000004},"min":{"1min":0.29499999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":1.722,"5min":1.722,"15min":1.722},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.76500000000000001,"15min":0.80300000000000005},"min":{"1min":0.33300000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.89500000000000002,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.80900000000000005}]},{"osd":2,"last update":"Sat Sep 21 15:01:09 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.71999999999999997,"15min":0.72199999999999998},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.92400000000000004,"5min":1.157,"15min":1.157},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.81000000000000005,"15min":0.82699999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.1539999999999999,"5min":1.254,"15min":1.254},"last":0.59199999999999997}]},{"osd":4,"last update":"Sat Sep 21 15:01:09 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.93799999999999994,"15min":0.95599999999999996},"min":{"1min":0.54300000000000004,"5min":0.442,"15min":0.442},"max":{"1min":1.2629999999999999,"5min":1.7609999999999999,"15min":1.7609999999999999},"last":0.97899999999999998},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.98899999999999999,"15min":1.0269999999999999},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.234,"5min":2.028,"15min":2.028},"last":1.1459999999999999}]},{"osd":5,"last update":"Sat Sep 21 15:01:09 2024","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":1.0369999999999999,"15min":1.052},"min":{"1min":0.70499999999999996,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.3440000000000001,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.077},{"interface":"front","average":{"1min":1.0249999999999999,"5min":1.056,"15min":1.0629999999999999},"min":{"1min":0.57999999999999996,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.548,"5min":1.548,"15min":1.548},"last":1.129}]},{"osd":6,"last update":"Sat Sep 21 15:01:09 2024","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":1.0669999999999999,"15min":1.0920000000000001},"min":{"1min":0.59099999999999997,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.6419999999999999,"5min":2.1219999999999999,"15min":2.1219999999999999},"last":1.0940000000000001},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.091,"15min":1.103},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.6839999999999999,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":0.93700000000000006}]},{"osd":7,"last update":"Sat Sep 21 15:01:09 2024","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":1.103,"15min":1.131},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.498,"5min":1.706,"15min":1.706},"last":1},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.147,"15min":1.177},"min":{"1min":0.59699999999999998,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.417,"5min":2.3380000000000001,"15min":2.3380000000000001},"last":1.1930000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542193,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":71592,"kb_used_data":4248,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94300248,"statfs":{"total":96636764160,"available":96563453952,"internally_reserved":0,"allocated":4349952,"data_stored":17216160,"data_compressed":272864,"data_compressed_allocated":1404928,"data_compressed_original":15737856,"omap_allocated":1594,"internal_metadata":68811206},"hb_peers":[0,1,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.62},"min":{"1min":0.26900000000000002,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.92800000000000005,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.69399999999999995,"15min":0.67000000000000004},"min":{"1min":0.34799999999999998,"5min":0.223,"15min":0.223},"max":{"1min":2.9169999999999998,"5min":2.9169999999999998,"15min":2.9169999999999998},"last":0.754}]},{"osd":1,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.32700000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.0529999999999999,"5min":2.6520000000000001,"15min":2.6520000000000001},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.746,"5min":0.73499999999999999,"15min":0.75800000000000001},"min":{"1min":0.40500000000000003,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.349,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.873}]},{"osd":3,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.73999999999999999,"15min":0.73499999999999999},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":3.0110000000000001,"5min":3.0110000000000001,"15min":3.0110000000000001},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.877,"15min":0.83899999999999997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.05,"5min":4.1440000000000001,"15min":4.1440000000000001},"last":0.81200000000000006}]},{"osd":4,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.93200000000000005,"15min":0.95599999999999996},"min":{"1min":0.42399999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.222,"5min":2.899,"15min":2.899},"last":0.996},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.94399999999999995,"15min":0.95799999999999996},"min":{"1min":0.47499999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.4379999999999999,"5min":1.675,"15min":1.675},"last":1.1220000000000001}]},{"osd":5,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.0129999999999999,"15min":1.0089999999999999},"min":{"1min":0.45500000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":3.1080000000000001,"5min":3.1080000000000001,"15min":3.1080000000000001},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.98199999999999998,"15min":0.99399999999999999},"min":{"1min":0.51700000000000002,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.268,"5min":1.546,"15min":1.546},"last":0.92500000000000004}]},{"osd":6,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":0.98299999999999998,"15min":0.98499999999999999},"min":{"1min":0.56200000000000006,"5min":0.378,"15min":0.378},"max":{"1min":1.345,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":1.2509999999999999},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.0649999999999999,"15min":1.0549999999999999},"min":{"1min":0.53200000000000003,"5min":0.502,"15min":0.502},"max":{"1min":3.0739999999999998,"5min":3.0739999999999998,"15min":3.0739999999999998},"last":0.90900000000000003}]},{"osd":7,"last update":"Sat Sep 21 15:01:08 2024","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.0449999999999999,"15min":1.0089999999999999},"min":{"1min":0.59499999999999997,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":3.1589999999999998,"5min":3.1589999999999998,"15min":3.1589999999999998},"last":0.96499999999999997},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.121,"15min":1.099},"min":{"1min":0.66000000000000003,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":3.1400000000000001,"5min":3.1400000000000001,"15min":3.1400000000000001},"last":1.012}]}]},{"osd":1,"up_from":14,"seq":60129542195,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49828,"kb_used_data":4752,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94322012,"statfs":{"total":96636764160,"available":96585740288,"internally_reserved":0,"allocated":4866048,"data_stored":26300906,"data_compressed":220037,"data_compressed_allocated":1925120,"data_compressed_original":24829947,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.59599999999999997,"15min":0.60899999999999999},"min":{"1min":0.36399999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.78200000000000003,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.95499999999999996},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59699999999999998,"15min":0.60799999999999998},"min":{"1min":0.39500000000000002,"5min":0.216,"15min":0.216},"max":{"1min":0.80100000000000005,"5min":1.107,"15min":1.107},"last":0.68300000000000005}]},{"osd":2,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":0.79600000000000004,"15min":0.73999999999999999},"min":{"1min":0.38700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":11.303000000000001,"5min":11.303000000000001,"15min":11.303000000000001},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.71299999999999997,"15min":0.70299999999999996},"min":{"1min":0.47299999999999998,"5min":0.435,"15min":0.435},"max":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.70699999999999996}]},{"osd":3,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.81799999999999995,"15min":0.83899999999999997},"min":{"1min":0.27800000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.0049999999999999,"5min":3.1469999999999998,"15min":3.1469999999999998},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.86399999999999999,"15min":0.88600000000000001},"min":{"1min":0.52800000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.306,"5min":3.2320000000000002,"15min":3.2320000000000002},"last":0.61499999999999999}]},{"osd":4,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.89000000000000001,"15min":0.91500000000000004},"min":{"1min":0.59599999999999997,"5min":0.309,"15min":0.309},"max":{"1min":1.45,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":1.0209999999999999},{"interface":"front","average":{"1min":1.5600000000000001,"5min":1.069,"15min":0.96599999999999997},"min":{"1min":0.504,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":11.563000000000001,"5min":11.563000000000001,"15min":11.563000000000001},"last":0.84599999999999997}]},{"osd":5,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":1.625,"5min":1.0469999999999999,"15min":0.94899999999999995},"min":{"1min":0.73399999999999999,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":11.534000000000001,"5min":11.534000000000001,"15min":11.534000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95799999999999996,"15min":0.96099999999999997},"min":{"1min":0.63,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.256,"5min":1.484,"15min":1.484},"last":0.82199999999999995}]},{"osd":6,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":1.0109999999999999,"15min":1.0489999999999999},"min":{"1min":0.60399999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.4059999999999999,"5min":2.3519999999999999,"15min":2.3519999999999999},"last":0.91700000000000004},{"interface":"front","average":{"1min":1.631,"5min":1.081,"15min":0.99299999999999999},"min":{"1min":0.57399999999999995,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":11.601000000000001,"5min":11.601000000000001,"15min":11.601000000000001},"last":1.081}]},{"osd":7,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":1.046,"15min":1.0640000000000001},"min":{"1min":0.79300000000000004,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.5820000000000001,"5min":2.2360000000000002,"15min":2.2360000000000002},"last":1.022},{"interface":"front","average":{"1min":1.76,"5min":1.171,"15min":1.087},"min":{"1min":0.877,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":11.489000000000001,"5min":11.489000000000001,"15min":11.489000000000001},"last":0.99099999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542193,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49532,"kb_used_data":4464,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94322308,"statfs":{"total":96636764160,"available":96586043392,"internally_reserved":0,"allocated":4571136,"data_stored":21429451,"data_compressed":243905,"data_compressed_allocated":1646592,"data_compressed_original":19979879,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 21 15:01:04 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.67500000000000004,"15min":0.71699999999999997},"min":{"1min":0.40999999999999998,"5min":0.309,"15min":0.309},"max":{"1min":1.1299999999999999,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.68100000000000005,"15min":0.627},"min":{"1min":0.58999999999999997,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.52700000000000002}]},{"osd":2,"last update":"Sat Sep 21 15:01:04 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.65200000000000002,"15min":0.61799999999999999},"min":{"1min":0.59599999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.502},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.72499999999999998,"15min":0.75800000000000001},"min":{"1min":0.39300000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.622}]},{"osd":3,"last update":"Sat Sep 21 15:01:04 2024","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.76600000000000001,"15min":0.754},"min":{"1min":0.50900000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.78700000000000003,"15min":0.81499999999999995},"min":{"1min":0.50900000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.97599999999999998}]},{"osd":4,"last update":"Sat Sep 21 15:01:04 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.98399999999999999,"15min":1.038},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.1140000000000001,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":0.85599999999999998},{"interface":"front","average":{"1min":1.036,"5min":1.0469999999999999,"15min":1.0620000000000001},"min":{"1min":0.67800000000000005,"5min":0.379,"15min":0.379},"max":{"1min":1.446,"5min":1.665,"15min":1.665},"last":0.98399999999999999}]},{"osd":5,"last update":"Sat Sep 21 15:01:04 2024","interfaces":[{"interface":"back","average":{"1min":1.0960000000000001,"5min":1.024,"15min":1.032},"min":{"1min":0.79300000000000004,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.4450000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.91600000000000004},{"interface":"front","average":{"1min":1.0589999999999999,"5min":1.0429999999999999,"15min":1.077},"min":{"1min":0.82399999999999995,"5min":0.435,"15min":0.435},"max":{"1min":1.2609999999999999,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":1.024}]},{"osd":6,"last update":"Sat Sep 21 15:01:04 2024","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":1.0580000000000001,"15min":1.1120000000000001},"min":{"1min":0.68400000000000005,"5min":0.628,"15min":0.628},"max":{"1min":1.167,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":0.76600000000000001},{"interface":"front","average":{"1min":1.109,"5min":1.0629999999999999,"15min":1.0680000000000001},"min":{"1min":0.91000000000000003,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.5029999999999999,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.88100000000000001}]},{"osd":7,"last update":"Sat Sep 21 15:01:04 2024","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.0469999999999999,"15min":1.0620000000000001},"min":{"1min":0.78700000000000003,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.4770000000000001,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":1.0600000000000001},{"interface":"front","average":{"1min":1.02,"5min":1.085,"15min":1.131},"min":{"1min":0.84799999999999998,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.403,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":0.84099999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5202,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5202,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1875968,"data_stored":20103918,"data_compressed":243905,"data_compressed_allocated":1646592,"data_compressed_original":19979879,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1716224,"data_stored":19551256,"data_compressed":127120,"data_compressed_allocated":1503232,"data_compressed_original":19429142,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1654784,"data_stored":15890627,"data_compressed":272864,"data_compressed_allocated":1404928,"data_compressed_original":15737856,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1626112,"data_stored":14964570,"data_compressed":266084,"data_compressed_allocated":1368064,"data_compressed_original":14825424,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":675840,"data_stored":7155966,"data_compressed":134408,"data_compressed_allocated":589824,"data_compressed_original":7119101,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1208320,"data_stored":13232597,"data_compressed":154120,"data_compressed_allocated":1105920,"data_compressed_original":13180328,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1626112,"data_stored":17136277,"data_compressed":246916,"data_compressed_allocated":1466368,"data_compressed_original":17069622,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":765952,"data_stored":8609089,"data_compressed":56002,"data_compressed_allocated":663552,"data_compressed_original":8558134,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T15:02:00.937 INFO:tasks.thrashosds.thrasher:Setting 2.3 to [6, 1, 4, 0] 2024-09-21T15:02:00.937 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '2.3', '6', '1', '4', '0'] 2024-09-21T15:02:00.937 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 2.3 6 1 4 0 2024-09-21T15:02:00.994 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:02:00.994 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:02:01.004 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T15:02:01.049 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:02:01.049+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:02:01.107 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:02:01.107 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:02:01.117 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T15:02:01.124 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:02:01.121+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:02:01.223 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:02:01.223 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:02:01.225 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:02:01.221+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:02:01.233 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T15:02:01.260 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T15:02:01.326 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:02:01.319+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:02:01.401 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:02:01.401 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:02:01.412 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T15:02:01.427 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:02:01.419+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:02:01.513 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:02:01.513 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:02:01.523 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T15:02:01.527 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:02:01.525+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:02:01.628 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-09-21T15:02:01.625+0000 7fa865845700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22154) UID: 0 2024-09-21T15:02:01.628 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:02:01.629 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:02:01.638 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T15:02:01.729 INFO:tasks.ceph.osd.7.smithi046.stderr:2024-09-21T15:02:01.727+0000 7f60512d7700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22242) UID: 0 2024-09-21T15:02:01.742 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:02:01.742 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:02:01.752 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T15:02:01.830 INFO:tasks.ceph.osd.5.smithi046.stderr:2024-09-21T15:02:01.827+0000 7fe95bd28700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22239) UID: 0 2024-09-21T15:02:01.859 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:02:01.860 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:02:01.869 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T15:02:01.931 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:02:01.929+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:02:01.980 INFO:teuthology.orchestra.run.smithi032.stdout:ERROR: (22) Invalid argument 2024-09-21T15:02:01.980 INFO:teuthology.orchestra.run.smithi032.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:02:01.990 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T15:02:02.036 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:02:02.031+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:02:02.110 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:02:02.110 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:02:02.121 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T15:02:02.132 INFO:tasks.ceph.osd.6.smithi046.stderr:2024-09-21T15:02:02.127+0000 7efc09e6c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22244) UID: 0 2024-09-21T15:02:02.231 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:02:02.231 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:02:02.233 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-09-21T15:02:02.233+0000 7f016bcd0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22157) UID: 0 2024-09-21T15:02:02.241 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T15:02:02.266 INFO:teuthology.orchestra.run.smithi032.stderr:nodeep-scrub is unset 2024-09-21T15:02:02.267 INFO:teuthology.orchestra.run.smithi032.stderr:set 2.3 pg_upmap_items mapping to [6->1,4->0] 2024-09-21T15:02:02.334 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-09-21T15:02:02.333+0000 7fd8ce700700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22152) UID: 0 2024-09-21T15:02:02.366 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:02:02.366 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:02:02.377 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T15:02:02.383 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T15:02:02.433 INFO:teuthology.orchestra.run.smithi032.stdout:All operations completed A-OK! 2024-09-21T15:02:02.435 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-09-21T15:02:02.433+0000 7f50a5b09700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22150) UID: 0 2024-09-21T15:02:02.467 DEBUG:teuthology.parallel:result is None 2024-09-21T15:02:02.467 DEBUG:teuthology.run_tasks:Unwinding manager rbd_fsx 2024-09-21T15:02:02.476 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-09-21T15:02:02.487 INFO:tasks.thrashosds:joining thrashosds 2024-09-21T15:02:02.487 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:02:02.487 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:02:02.493 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T15:02:02.534 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0: {} 2024-09-21T15:02:02.557 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1: {} 2024-09-21T15:02:02.580 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2: {} 2024-09-21T15:02:02.599 INFO:teuthology.orchestra.run.smithi046.stdout:ERROR: (22) Invalid argument 2024-09-21T15:02:02.599 INFO:teuthology.orchestra.run.smithi046.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:02:02.607 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3: {} 2024-09-21T15:02:02.609 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T15:02:02.628 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5: {} 2024-09-21T15:02:02.651 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6: {} 2024-09-21T15:02:02.673 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7: {} 2024-09-21T15:02:02.674 INFO:teuthology.orchestra.run.smithi032.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:02:02.674 INFO:teuthology.orchestra.run.smithi032.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:02:02.674 INFO:teuthology.orchestra.run.smithi032.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:02:02.674 INFO:teuthology.orchestra.run.smithi032.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:02:02.674 INFO:teuthology.orchestra.run.smithi032.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-21T15:02:02.674 INFO:teuthology.orchestra.run.smithi032.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:02:02.674 INFO:teuthology.orchestra.run.smithi032.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:02:02.674 INFO:teuthology.orchestra.run.smithi032.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:02:02.677 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-21T15:02:02.677 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-21T15:02:02.718 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:02:02.718 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T15:02:02.718 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T15:02:02.718 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T15:02:02.719 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.719 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:97677 3.15 e122/24 3:a8e72133:::rbd_data.1136bfc79a70.0000000000000005:head v 122'567, mlcod=120'565)", 2024-09-21T15:02:02.719 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.320753+0000", 2024-09-21T15:02:02.719 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 117.392795303, 2024-09-21T15:02:02.719 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.070639375000000004, 2024-09-21T15:02:02.719 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:02:02.719 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:02:02.719 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:02:02.719 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.719 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:02:02.719 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.320753+0000", 2024-09-21T15:02:02.719 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:02:02.719 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.720 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.720 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:02:02.720 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.320753+0000", 2024-09-21T15:02:02.720 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:02:02.720 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.720 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.720 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:02:02.720 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.320753+0000", 2024-09-21T15:02:02.720 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:02:02.720 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.720 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.720 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:02:02.720 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.375452+0000", 2024-09-21T15:02:02.721 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.054698945999999998 2024-09-21T15:02:02.721 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.721 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.721 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:02:02.721 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.375476+0000", 2024-09-21T15:02:02.721 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4380999999999999e-05 2024-09-21T15:02:02.721 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.721 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.721 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:02:02.721 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.375485+0000", 2024-09-21T15:02:02.721 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0049999999999993e-06 2024-09-21T15:02:02.721 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.721 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.722 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:02:02.722 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.375552+0000", 2024-09-21T15:02:02.722 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6550999999999994e-05 2024-09-21T15:02:02.722 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.722 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.722 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:02:02.722 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.375578+0000", 2024-09-21T15:02:02.722 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6196000000000001e-05 2024-09-21T15:02:02.722 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.722 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.722 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:02:02.722 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.391319+0000", 2024-09-21T15:02:02.722 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.015741271000000001 2024-09-21T15:02:02.723 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.723 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.723 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:02:02.723 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.391393+0000", 2024-09-21T15:02:02.723 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3287000000000005e-05 2024-09-21T15:02:02.723 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:02.723 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:02:02.723 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:02.723 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.723 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.723 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:97672 3.4 e122/51 3:26046d29:::rbd_data.1136bfc79a70.0000000000000001:head v 122'256, mlcod=120'254)", 2024-09-21T15:02:02.723 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.331696+0000", 2024-09-21T15:02:02.723 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 117.38185214000001, 2024-09-21T15:02:02.723 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.041222507999999998, 2024-09-21T15:02:02.724 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:02:02.724 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:02:02.724 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:02:02.724 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.724 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:02:02.724 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.331696+0000", 2024-09-21T15:02:02.724 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:02:02.724 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.724 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.724 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:02:02.724 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.331696+0000", 2024-09-21T15:02:02.724 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:02:02.724 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.724 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.724 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:02:02.725 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.331695+0000", 2024-09-21T15:02:02.725 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:02:02.725 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.725 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.725 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:02:02.725 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.340175+0000", 2024-09-21T15:02:02.725 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0084799520000000007 2024-09-21T15:02:02.725 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.725 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.725 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:02:02.725 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.340208+0000", 2024-09-21T15:02:02.725 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2997999999999998e-05 2024-09-21T15:02:02.725 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.725 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.726 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:02:02.726 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.340223+0000", 2024-09-21T15:02:02.726 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5330999999999999e-05 2024-09-21T15:02:02.726 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.726 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.726 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:02:02.726 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.340305+0000", 2024-09-21T15:02:02.726 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2095999999999995e-05 2024-09-21T15:02:02.726 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.726 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.726 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:02:02.726 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.340337+0000", 2024-09-21T15:02:02.726 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2097000000000001e-05 2024-09-21T15:02:02.726 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.727 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.727 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:02:02.727 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.372863+0000", 2024-09-21T15:02:02.727 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.032525923999999998 2024-09-21T15:02:02.727 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.727 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.727 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:02:02.727 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.372919+0000", 2024-09-21T15:02:02.727 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5729000000000001e-05 2024-09-21T15:02:02.727 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:02.727 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:02:02.727 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:02.727 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.727 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.728 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:116182 3.12 3:4efbaa3f:::rbd_data.1136b1194d20.0000000000000006:58 [read 6242304~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-21T15:02:02.728 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:29.712538+0000", 2024-09-21T15:02:02.728 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 93.001010148999995, 2024-09-21T15:02:02.728 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034702127999999999, 2024-09-21T15:02:02.728 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:02:02.728 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:02:02.728 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:02:02.728 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:02:02.728 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:02:02.728 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 116182 2024-09-21T15:02:02.728 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.728 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:02:02.728 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.728 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:02:02.729 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712538+0000", 2024-09-21T15:02:02.729 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:02:02.729 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.729 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.729 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:02:02.729 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712538+0000", 2024-09-21T15:02:02.729 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:02:02.729 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.729 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.729 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:02:02.729 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712538+0000", 2024-09-21T15:02:02.729 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:02:02.729 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.729 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.729 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:02:02.730 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712541+0000", 2024-09-21T15:02:02.730 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5330000000000001e-06 2024-09-21T15:02:02.730 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.730 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.730 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:02:02.730 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712547+0000", 2024-09-21T15:02:02.730 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1109999999999996e-06 2024-09-21T15:02:02.730 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.730 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.730 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:02:02.730 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.712551+0000", 2024-09-21T15:02:02.730 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.685e-06 2024-09-21T15:02:02.730 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.730 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.731 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:02:02.731 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.747067+0000", 2024-09-21T15:02:02.731 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.034516181 2024-09-21T15:02:02.731 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.731 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.731 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:02:02.731 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.747152+0000", 2024-09-21T15:02:02.731 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4075999999999999e-05 2024-09-21T15:02:02.731 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.731 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.731 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:02:02.731 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:29.747240+0000", 2024-09-21T15:02:02.731 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.8924000000000004e-05 2024-09-21T15:02:02.731 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:02.732 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:02:02.732 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:02.732 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.732 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.732 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139784 3.1 3:862e0e65:::rbd_data.113664780c85.0000000000000063:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:02:02.732 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.222486+0000", 2024-09-21T15:02:02.732 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 60.491062282000001, 2024-09-21T15:02:02.732 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0141793059999999, 2024-09-21T15:02:02.732 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:02:02.732 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:02:02.732 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:02:02.732 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:02:02.732 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:02:02.732 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139784 2024-09-21T15:02:02.733 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.733 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:02:02.733 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.733 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:02:02.733 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222486+0000", 2024-09-21T15:02:02.733 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:02:02.733 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.733 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.733 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:02:02.733 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222486+0000", 2024-09-21T15:02:02.733 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:02:02.733 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.733 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.733 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:02:02.733 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222484+0000", 2024-09-21T15:02:02.733 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-21T15:02:02.734 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.734 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.734 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:02:02.734 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222494+0000", 2024-09-21T15:02:02.734 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0676000000000001e-05 2024-09-21T15:02:02.734 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.734 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.734 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:02:02.734 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222517+0000", 2024-09-21T15:02:02.734 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2371e-05 2024-09-21T15:02:02.734 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.734 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.734 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:02:02.735 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222527+0000", 2024-09-21T15:02:02.735 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0461e-05 2024-09-21T15:02:02.735 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.735 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.735 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:02:02.735 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222625+0000", 2024-09-21T15:02:02.735 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.8160999999999997e-05 2024-09-21T15:02:02.735 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.735 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.735 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:02:02.735 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222641+0000", 2024-09-21T15:02:02.735 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5899000000000001e-05 2024-09-21T15:02:02.735 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.735 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.735 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:02:02.736 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.236479+0000", 2024-09-21T15:02:02.736 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0138377709999999 2024-09-21T15:02:02.736 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.736 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.736 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:02:02.736 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.236666+0000", 2024-09-21T15:02:02.736 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018648199999999999 2024-09-21T15:02:02.736 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:02.736 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:02:02.736 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:02.736 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.736 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.736 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139812 3.1 3:8407fb9b:::rbd_data.113664780c85.0000000000000127:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:02:02.736 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.222547+0000", 2024-09-21T15:02:02.737 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 60.491001357999998, 2024-09-21T15:02:02.737 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.014796059, 2024-09-21T15:02:02.737 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:02:02.737 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:02:02.737 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:02:02.737 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:02:02.737 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:02:02.737 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139812 2024-09-21T15:02:02.737 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.737 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:02:02.737 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.737 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:02:02.737 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222547+0000", 2024-09-21T15:02:02.737 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:02:02.737 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.738 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.738 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:02:02.738 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222547+0000", 2024-09-21T15:02:02.738 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:02:02.738 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.738 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.738 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:02:02.738 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222546+0000", 2024-09-21T15:02:02.738 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:02:02.738 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.738 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.738 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:02:02.738 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222552+0000", 2024-09-21T15:02:02.739 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2739999999999998e-06 2024-09-21T15:02:02.739 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.739 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.739 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:02:02.739 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222578+0000", 2024-09-21T15:02:02.739 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6594999999999999e-05 2024-09-21T15:02:02.739 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.739 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.739 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:02:02.739 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222596+0000", 2024-09-21T15:02:02.739 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7838000000000001e-05 2024-09-21T15:02:02.739 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.739 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.740 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:02:02.740 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222701+0000", 2024-09-21T15:02:02.740 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000104714 2024-09-21T15:02:02.740 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.740 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.740 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:02:02.740 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222714+0000", 2024-09-21T15:02:02.740 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2998e-05 2024-09-21T15:02:02.740 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.740 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.740 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:02:02.740 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.237124+0000", 2024-09-21T15:02:02.740 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.014409806 2024-09-21T15:02:02.740 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.740 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.741 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:02:02.741 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.237343+0000", 2024-09-21T15:02:02.741 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021950499999999999 2024-09-21T15:02:02.741 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:02.741 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:02:02.741 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:02.741 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.741 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.741 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139826 3.1 3:854f6f9b:::rbd_data.113664780c85.0000000000000189:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:02:02.741 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.222611+0000", 2024-09-21T15:02:02.741 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 60.490937266000003, 2024-09-21T15:02:02.741 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.014916828, 2024-09-21T15:02:02.741 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:02:02.741 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:02:02.741 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:02:02.742 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:02:02.742 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:02:02.742 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139826 2024-09-21T15:02:02.742 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.742 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:02:02.742 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.742 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:02:02.742 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222611+0000", 2024-09-21T15:02:02.742 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:02:02.742 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.742 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.742 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:02:02.742 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222611+0000", 2024-09-21T15:02:02.742 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:02:02.743 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.743 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.743 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:02:02.743 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222610+0000", 2024-09-21T15:02:02.743 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:02:02.743 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.743 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.743 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:02:02.743 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222618+0000", 2024-09-21T15:02:02.743 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6690000000000006e-06 2024-09-21T15:02:02.743 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.743 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.743 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:02:02.743 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222632+0000", 2024-09-21T15:02:02.744 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4433999999999999e-05 2024-09-21T15:02:02.744 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.744 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.744 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:02:02.744 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222640+0000", 2024-09-21T15:02:02.744 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6260000000000003e-06 2024-09-21T15:02:02.744 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.744 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.744 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:02:02.744 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222758+0000", 2024-09-21T15:02:02.744 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011826499999999999 2024-09-21T15:02:02.745 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.745 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.745 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:02:02.745 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.222776+0000", 2024-09-21T15:02:02.745 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7564999999999999e-05 2024-09-21T15:02:02.745 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.745 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.745 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:02:02.745 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.237402+0000", 2024-09-21T15:02:02.745 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0146261889999999 2024-09-21T15:02:02.745 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.745 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.745 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:02:02.745 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.237528+0000", 2024-09-21T15:02:02.745 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000126212 2024-09-21T15:02:02.746 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:02.746 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:02:02.746 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:02.746 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.746 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.746 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139864 3.1 3:86b14cb7:::rbd_data.113664780c85.00000000000000d5:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:02:02.746 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.223966+0000", 2024-09-21T15:02:02.746 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 60.48958253, 2024-09-21T15:02:02.746 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.013859007, 2024-09-21T15:02:02.746 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:02:02.746 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:02:02.746 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:02:02.746 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:02:02.747 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:02:02.747 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139864 2024-09-21T15:02:02.747 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.747 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:02:02.747 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.747 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:02:02.747 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223966+0000", 2024-09-21T15:02:02.747 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:02:02.747 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.747 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.747 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:02:02.747 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223966+0000", 2024-09-21T15:02:02.747 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:02:02.748 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.748 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.748 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:02:02.748 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223965+0000", 2024-09-21T15:02:02.748 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:02:02.748 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.748 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.748 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:02:02.748 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223970+0000", 2024-09-21T15:02:02.748 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.3769999999999998e-06 2024-09-21T15:02:02.748 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.748 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.748 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:02:02.749 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223979+0000", 2024-09-21T15:02:02.749 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5899999999999997e-06 2024-09-21T15:02:02.749 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.749 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.749 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:02:02.749 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.223985+0000", 2024-09-21T15:02:02.749 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5790000000000003e-06 2024-09-21T15:02:02.749 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.749 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.749 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:02:02.749 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224073+0000", 2024-09-21T15:02:02.749 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7781000000000005e-05 2024-09-21T15:02:02.749 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.749 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.750 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:02:02.750 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224110+0000", 2024-09-21T15:02:02.750 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7060000000000001e-05 2024-09-21T15:02:02.750 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.750 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.750 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:02:02.750 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.237599+0000", 2024-09-21T15:02:02.750 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.01348953 2024-09-21T15:02:02.750 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.750 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.750 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:02:02.750 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.237825+0000", 2024-09-21T15:02:02.750 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000225758 2024-09-21T15:02:02.751 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:02.751 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:02:02.751 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:02.751 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.751 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.751 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139877 3.1 3:87277f92:::rbd_data.113664780c85.0000000000000136:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:02:02.751 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.224489+0000", 2024-09-21T15:02:02.751 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 60.489059672000003, 2024-09-21T15:02:02.751 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0137916330000001, 2024-09-21T15:02:02.751 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:02:02.751 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:02:02.751 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:02:02.751 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:02:02.751 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:02:02.752 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139877 2024-09-21T15:02:02.752 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.752 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:02:02.752 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.752 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:02:02.752 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224489+0000", 2024-09-21T15:02:02.752 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:02:02.752 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.752 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.752 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:02:02.752 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224489+0000", 2024-09-21T15:02:02.752 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:02:02.752 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.752 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.753 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:02:02.753 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224488+0000", 2024-09-21T15:02:02.753 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:02:02.753 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.753 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.753 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:02:02.753 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224502+0000", 2024-09-21T15:02:02.753 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4194e-05 2024-09-21T15:02:02.753 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.753 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.753 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:02:02.753 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224522+0000", 2024-09-21T15:02:02.753 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0038e-05 2024-09-21T15:02:02.753 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.754 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.754 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:02:02.754 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224531+0000", 2024-09-21T15:02:02.754 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6900000000000004e-06 2024-09-21T15:02:02.754 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.754 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.754 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:02:02.754 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224621+0000", 2024-09-21T15:02:02.754 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9654000000000006e-05 2024-09-21T15:02:02.754 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.754 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.754 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:02:02.754 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224641+0000", 2024-09-21T15:02:02.754 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9375e-05 2024-09-21T15:02:02.755 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.755 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.755 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:02:02.755 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.238027+0000", 2024-09-21T15:02:02.755 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0133866840000001 2024-09-21T15:02:02.755 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.755 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.755 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:02:02.755 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.238280+0000", 2024-09-21T15:02:02.755 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000253281 2024-09-21T15:02:02.755 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:02.755 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:02:02.755 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:02.755 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.756 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.756 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:139887 3.1 3:80f1e8f1:::rbd_data.113664780c85.000000000000017c:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:02:02.756 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.224549+0000", 2024-09-21T15:02:02.756 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 60.488999382000003, 2024-09-21T15:02:02.756 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0139776060000001, 2024-09-21T15:02:02.756 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:02:02.756 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:02:02.756 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:02:02.756 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:02:02.756 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:02:02.756 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 139887 2024-09-21T15:02:02.756 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.756 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:02:02.756 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.756 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:02:02.756 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224549+0000", 2024-09-21T15:02:02.757 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:02:02.757 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.757 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.757 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:02:02.757 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224549+0000", 2024-09-21T15:02:02.757 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:02:02.757 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.757 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.757 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:02:02.757 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224548+0000", 2024-09-21T15:02:02.757 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:02:02.757 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.757 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.757 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:02:02.758 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224554+0000", 2024-09-21T15:02:02.758 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.4049999999999999e-06 2024-09-21T15:02:02.758 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.758 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.758 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:02:02.758 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224565+0000", 2024-09-21T15:02:02.758 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0879e-05 2024-09-21T15:02:02.758 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.758 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.758 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:02:02.758 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224572+0000", 2024-09-21T15:02:02.758 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0380000000000002e-06 2024-09-21T15:02:02.758 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.758 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.758 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:02:02.759 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224684+0000", 2024-09-21T15:02:02.759 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00011229 2024-09-21T15:02:02.759 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.759 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.759 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:02:02.759 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.224698+0000", 2024-09-21T15:02:02.759 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4448e-05 2024-09-21T15:02:02.759 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.759 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.759 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:02:02.759 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.238359+0000", 2024-09-21T15:02:02.759 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0136610660000001 2024-09-21T15:02:02.759 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.759 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.759 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:02:02.760 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.238527+0000", 2024-09-21T15:02:02.760 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00016731100000000001 2024-09-21T15:02:02.760 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:02.760 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:02:02.760 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:02.760 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.760 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.760 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140077 3.1 3:85642daf:::rbd_data.113664780c85.0000000000000174:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:02:02.760 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.232797+0000", 2024-09-21T15:02:02.760 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 60.480751667, 2024-09-21T15:02:02.760 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.006029955, 2024-09-21T15:02:02.760 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:02:02.760 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:02:02.760 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:02:02.761 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:02:02.761 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:02:02.761 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140077 2024-09-21T15:02:02.761 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.761 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:02:02.761 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.761 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:02:02.761 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232797+0000", 2024-09-21T15:02:02.761 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:02:02.761 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.761 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.761 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:02:02.761 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232797+0000", 2024-09-21T15:02:02.761 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:02:02.761 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.761 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.762 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:02:02.762 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232795+0000", 2024-09-21T15:02:02.762 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:02:02.762 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.762 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.762 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:02:02.762 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232804+0000", 2024-09-21T15:02:02.762 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.9670000000000007e-06 2024-09-21T15:02:02.762 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.762 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.762 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:02:02.762 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232824+0000", 2024-09-21T15:02:02.762 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9238000000000001e-05 2024-09-21T15:02:02.762 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.763 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.763 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:02:02.763 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232843+0000", 2024-09-21T15:02:02.763 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9256e-05 2024-09-21T15:02:02.763 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.763 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.763 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:02:02.763 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232912+0000", 2024-09-21T15:02:02.763 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9589000000000001e-05 2024-09-21T15:02:02.763 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.763 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.763 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:02:02.763 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.232925+0000", 2024-09-21T15:02:02.763 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2585999999999999e-05 2024-09-21T15:02:02.764 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.764 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.764 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:02:02.764 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.238591+0000", 2024-09-21T15:02:02.764 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0056659939999999 2024-09-21T15:02:02.764 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.764 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.764 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:02:02.764 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.238827+0000", 2024-09-21T15:02:02.764 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00023583999999999999 2024-09-21T15:02:02.764 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:02.764 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:02:02.764 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:02.764 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.765 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.765 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140091 3.1 3:8358f0db:::rbd_data.113664780c85.0000000000000018:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:02:02.765 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.233637+0000", 2024-09-21T15:02:02.765 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 60.479911371, 2024-09-21T15:02:02.765 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.006272847, 2024-09-21T15:02:02.765 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:02:02.765 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:02:02.765 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:02:02.765 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:02:02.765 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:02:02.765 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140091 2024-09-21T15:02:02.765 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.765 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:02:02.765 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.765 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:02:02.766 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233637+0000", 2024-09-21T15:02:02.766 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:02:02.766 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.766 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.766 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:02:02.766 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233637+0000", 2024-09-21T15:02:02.766 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:02:02.766 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.766 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.766 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:02:02.766 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233636+0000", 2024-09-21T15:02:02.766 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:02:02.766 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.766 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.767 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:02:02.767 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233644+0000", 2024-09-21T15:02:02.767 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.1380000000000007e-06 2024-09-21T15:02:02.767 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.767 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.767 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:02:02.767 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233662+0000", 2024-09-21T15:02:02.767 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8224999999999999e-05 2024-09-21T15:02:02.767 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.767 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.767 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:02:02.767 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233674+0000", 2024-09-21T15:02:02.767 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2081e-05 2024-09-21T15:02:02.767 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.767 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.767 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:02:02.768 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233750+0000", 2024-09-21T15:02:02.768 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6106000000000006e-05 2024-09-21T15:02:02.768 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.768 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.768 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:02:02.768 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.233768+0000", 2024-09-21T15:02:02.768 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7285000000000001e-05 2024-09-21T15:02:02.768 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.768 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.768 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:02:02.768 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.239196+0000", 2024-09-21T15:02:02.768 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.005428223 2024-09-21T15:02:02.768 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.768 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.768 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:02:02.769 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.239528+0000", 2024-09-21T15:02:02.769 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00033262199999999997 2024-09-21T15:02:02.769 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.769 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.769 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:02:02.769 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.239910+0000", 2024-09-21T15:02:02.769 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00038151100000000001 2024-09-21T15:02:02.769 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:02.769 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:02:02.769 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:02.769 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.769 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.769 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140109 3.1 3:8586e9d9:::rbd_data.113664780c85.0000000000000096:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:02:02.769 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.234171+0000", 2024-09-21T15:02:02.769 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 60.479377536999998, 2024-09-21T15:02:02.769 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.006019896, 2024-09-21T15:02:02.770 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:02:02.770 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:02:02.770 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:02:02.770 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:02:02.770 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:02:02.770 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140109 2024-09-21T15:02:02.770 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.770 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:02:02.770 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.770 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:02:02.770 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.234171+0000", 2024-09-21T15:02:02.770 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:02:02.770 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.770 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.770 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:02:02.771 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.234171+0000", 2024-09-21T15:02:02.771 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:02:02.771 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.771 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.771 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:02:02.771 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.234170+0000", 2024-09-21T15:02:02.771 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:02:02.771 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.771 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.771 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:02:02.771 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.234178+0000", 2024-09-21T15:02:02.771 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.464e-06 2024-09-21T15:02:02.771 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.771 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.771 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:02:02.771 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.234191+0000", 2024-09-21T15:02:02.772 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3489000000000001e-05 2024-09-21T15:02:02.772 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.772 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.772 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:02:02.772 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.234200+0000", 2024-09-21T15:02:02.772 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.2779999999999999e-06 2024-09-21T15:02:02.772 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.772 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.772 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:02:02.772 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.234279+0000", 2024-09-21T15:02:02.772 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8813000000000006e-05 2024-09-21T15:02:02.772 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.772 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.772 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:02:02.772 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.234298+0000", 2024-09-21T15:02:02.772 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8947999999999999e-05 2024-09-21T15:02:02.773 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.773 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.773 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:02:02.773 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.239974+0000", 2024-09-21T15:02:02.773 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0056760259999999 2024-09-21T15:02:02.773 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.773 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.773 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:02:02.773 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.240191+0000", 2024-09-21T15:02:02.773 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000216708 2024-09-21T15:02:02.773 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:02.773 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:02:02.773 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:02.773 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.774 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.774 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140149 3.1 3:86b1e433:::rbd_data.113664780c85.00000000000001ae:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:02:02.774 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.235545+0000", 2024-09-21T15:02:02.774 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 60.478003252000001, 2024-09-21T15:02:02.774 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0049193430000001, 2024-09-21T15:02:02.774 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:02:02.774 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:02:02.774 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:02:02.774 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:02:02.774 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:02:02.774 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140149 2024-09-21T15:02:02.774 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.774 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:02:02.775 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.775 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:02:02.775 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235545+0000", 2024-09-21T15:02:02.775 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:02:02.775 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.775 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.775 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:02:02.775 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235545+0000", 2024-09-21T15:02:02.775 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:02:02.775 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.775 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.775 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:02:02.775 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235544+0000", 2024-09-21T15:02:02.775 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:02:02.775 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.776 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.776 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:02:02.776 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235554+0000", 2024-09-21T15:02:02.776 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.8870000000000004e-06 2024-09-21T15:02:02.776 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.776 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.776 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:02:02.776 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235577+0000", 2024-09-21T15:02:02.776 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2946999999999999e-05 2024-09-21T15:02:02.776 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.776 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.776 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:02:02.776 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235587+0000", 2024-09-21T15:02:02.776 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0326000000000001e-05 2024-09-21T15:02:02.776 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.776 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.776 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:02:02.777 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235658+0000", 2024-09-21T15:02:02.777 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1360999999999997e-05 2024-09-21T15:02:02.777 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.777 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.777 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:02:02.777 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235670+0000", 2024-09-21T15:02:02.777 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1735e-05 2024-09-21T15:02:02.777 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.777 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.777 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:02:02.777 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.240246+0000", 2024-09-21T15:02:02.777 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0045757790000001 2024-09-21T15:02:02.777 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.777 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.778 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:02:02.778 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.240465+0000", 2024-09-21T15:02:02.778 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021870799999999999 2024-09-21T15:02:02.778 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:02.778 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:02:02.778 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:02.778 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.778 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.778 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140232 3.1 3:862e0e65:::rbd_data.113664780c85.0000000000000063:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:02:02.778 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.235760+0000", 2024-09-21T15:02:02.778 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 60.477788533999998, 2024-09-21T15:02:02.778 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.004979812, 2024-09-21T15:02:02.778 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:02:02.778 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:02:02.778 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:02:02.779 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:02:02.779 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:02:02.779 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140232 2024-09-21T15:02:02.779 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.779 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:02:02.779 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.779 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:02:02.779 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235760+0000", 2024-09-21T15:02:02.779 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:02:02.779 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.779 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.779 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:02:02.779 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235760+0000", 2024-09-21T15:02:02.779 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:02:02.779 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.780 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.780 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:02:02.780 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235759+0000", 2024-09-21T15:02:02.780 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:02:02.780 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.780 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.780 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:02:02.780 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235777+0000", 2024-09-21T15:02:02.780 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8215000000000001e-05 2024-09-21T15:02:02.780 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.780 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.780 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:02:02.780 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235793+0000", 2024-09-21T15:02:02.780 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6461e-05 2024-09-21T15:02:02.780 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.780 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.781 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:02:02.781 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235803+0000", 2024-09-21T15:02:02.781 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.5300000000000002e-06 2024-09-21T15:02:02.781 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.781 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.781 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:02:02.781 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235875+0000", 2024-09-21T15:02:02.781 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2113000000000006e-05 2024-09-21T15:02:02.781 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.781 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.781 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:02:02.781 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235887+0000", 2024-09-21T15:02:02.781 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2160999999999999e-05 2024-09-21T15:02:02.781 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.781 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.781 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:02:02.782 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.240521+0000", 2024-09-21T15:02:02.782 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.004633428 2024-09-21T15:02:02.782 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.782 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.782 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:02:02.782 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.240740+0000", 2024-09-21T15:02:02.782 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000219228 2024-09-21T15:02:02.782 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:02.782 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:02:02.782 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:02.782 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.782 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.782 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140260 3.1 3:8407fb9b:::rbd_data.113664780c85.0000000000000127:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:02:02.782 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.235839+0000", 2024-09-21T15:02:02.782 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 60.477709365000003, 2024-09-21T15:02:02.783 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.005157981, 2024-09-21T15:02:02.783 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:02:02.783 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:02:02.783 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:02:02.783 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:02:02.783 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:02:02.783 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140260 2024-09-21T15:02:02.783 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.783 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:02:02.783 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.783 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:02:02.783 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235839+0000", 2024-09-21T15:02:02.783 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:02:02.784 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.784 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.784 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:02:02.784 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235839+0000", 2024-09-21T15:02:02.784 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:02:02.784 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.784 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.784 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:02:02.784 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235838+0000", 2024-09-21T15:02:02.784 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:02:02.784 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.784 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.784 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:02:02.784 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235843+0000", 2024-09-21T15:02:02.784 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.481e-06 2024-09-21T15:02:02.785 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.785 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.785 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:02:02.785 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235849+0000", 2024-09-21T15:02:02.785 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4880000000000004e-06 2024-09-21T15:02:02.785 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.785 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.785 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:02:02.785 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235856+0000", 2024-09-21T15:02:02.785 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7649999999999996e-06 2024-09-21T15:02:02.785 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.785 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.785 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:02:02.785 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235933+0000", 2024-09-21T15:02:02.786 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6987e-05 2024-09-21T15:02:02.786 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.786 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.786 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:02:02.786 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235949+0000", 2024-09-21T15:02:02.786 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6696000000000001e-05 2024-09-21T15:02:02.786 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.786 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.786 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:02:02.786 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.240823+0000", 2024-09-21T15:02:02.786 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.004873259 2024-09-21T15:02:02.786 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.786 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.786 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:02:02.787 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.240997+0000", 2024-09-21T15:02:02.787 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000174365 2024-09-21T15:02:02.787 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:02.787 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:02:02.787 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:02.787 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.787 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.787 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140274 3.1 3:854f6f9b:::rbd_data.113664780c85.0000000000000189:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:02:02.787 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.235884+0000", 2024-09-21T15:02:02.787 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 60.477664417, 2024-09-21T15:02:02.787 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.005397366, 2024-09-21T15:02:02.787 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:02:02.787 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:02:02.787 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:02:02.788 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:02:02.788 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:02:02.788 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140274 2024-09-21T15:02:02.788 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.788 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:02:02.788 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.788 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:02:02.788 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235884+0000", 2024-09-21T15:02:02.788 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:02:02.788 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.788 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.788 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:02:02.788 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235884+0000", 2024-09-21T15:02:02.788 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:02:02.788 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.789 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.789 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:02:02.789 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235883+0000", 2024-09-21T15:02:02.789 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:02:02.789 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.789 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.789 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:02:02.789 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235891+0000", 2024-09-21T15:02:02.789 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.9740000000000002e-06 2024-09-21T15:02:02.789 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.789 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.789 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:02:02.789 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235900+0000", 2024-09-21T15:02:02.789 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6659999999999997e-06 2024-09-21T15:02:02.790 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.790 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.790 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:02:02.790 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.235909+0000", 2024-09-21T15:02:02.790 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3400000000000004e-06 2024-09-21T15:02:02.790 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.790 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.790 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:02:02.790 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236012+0000", 2024-09-21T15:02:02.790 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010240500000000001 2024-09-21T15:02:02.790 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.790 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.790 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:02:02.790 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236027+0000", 2024-09-21T15:02:02.790 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5418e-05 2024-09-21T15:02:02.791 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.791 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.791 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:02:02.791 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.241068+0000", 2024-09-21T15:02:02.791 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0050410729999999 2024-09-21T15:02:02.791 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.791 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.791 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:02:02.791 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.241281+0000", 2024-09-21T15:02:02.791 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021321900000000001 2024-09-21T15:02:02.791 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:02.791 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:02:02.791 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:02.791 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.791 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.792 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140312 3.1 3:86b14cb7:::rbd_data.113664780c85.00000000000000d5:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:02:02.792 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.236769+0000", 2024-09-21T15:02:02.792 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 60.476779952999998, 2024-09-21T15:02:02.792 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.004717198, 2024-09-21T15:02:02.792 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:02:02.792 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:02:02.792 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:02:02.792 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:02:02.792 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:02:02.792 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140312 2024-09-21T15:02:02.792 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.792 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:02:02.792 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.793 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:02:02.793 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236769+0000", 2024-09-21T15:02:02.793 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:02:02.793 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.793 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.793 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:02:02.793 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236769+0000", 2024-09-21T15:02:02.793 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:02:02.793 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.793 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.793 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:02:02.793 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236768+0000", 2024-09-21T15:02:02.793 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:02:02.793 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.793 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.794 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:02:02.794 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236771+0000", 2024-09-21T15:02:02.794 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.331e-06 2024-09-21T15:02:02.794 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.794 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.794 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:02:02.794 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236777+0000", 2024-09-21T15:02:02.794 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.6949999999999997e-06 2024-09-21T15:02:02.794 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.794 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.794 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:02:02.794 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236787+0000", 2024-09-21T15:02:02.794 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6700000000000006e-06 2024-09-21T15:02:02.795 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.795 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.795 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:02:02.795 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236846+0000", 2024-09-21T15:02:02.795 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.944e-05 2024-09-21T15:02:02.795 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.795 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.795 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:02:02.795 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.236861+0000", 2024-09-21T15:02:02.795 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.4611e-05 2024-09-21T15:02:02.795 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.795 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.795 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:02:02.795 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.241349+0000", 2024-09-21T15:02:02.796 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0044879250000001 2024-09-21T15:02:02.796 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.796 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.796 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:02:02.796 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.241486+0000", 2024-09-21T15:02:02.796 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013713000000000001 2024-09-21T15:02:02.796 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:02.796 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:02:02.796 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:02.796 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.796 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.796 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140325 3.1 3:87277f92:::rbd_data.113664780c85.0000000000000136:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:02:02.796 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.239351+0000", 2024-09-21T15:02:02.796 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 60.474197814, 2024-09-21T15:02:02.796 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0024334690000001, 2024-09-21T15:02:02.797 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:02:02.797 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:02:02.797 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:02:02.797 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:02:02.797 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:02:02.797 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140325 2024-09-21T15:02:02.797 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.797 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:02:02.797 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.797 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:02:02.797 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239351+0000", 2024-09-21T15:02:02.797 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:02:02.797 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.797 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.797 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:02:02.798 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239351+0000", 2024-09-21T15:02:02.798 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:02:02.798 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.798 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.798 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:02:02.798 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239349+0000", 2024-09-21T15:02:02.798 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:02:02.798 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.798 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.798 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:02:02.798 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239358+0000", 2024-09-21T15:02:02.798 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.2830000000000001e-06 2024-09-21T15:02:02.798 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.798 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.799 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:02:02.799 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239379+0000", 2024-09-21T15:02:02.799 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1780000000000002e-05 2024-09-21T15:02:02.799 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.799 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.799 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:02:02.799 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239389+0000", 2024-09-21T15:02:02.799 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0056e-05 2024-09-21T15:02:02.799 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.799 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.799 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:02:02.799 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239484+0000", 2024-09-21T15:02:02.799 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.4611000000000006e-05 2024-09-21T15:02:02.800 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.800 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.800 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:02:02.800 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239497+0000", 2024-09-21T15:02:02.800 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2909e-05 2024-09-21T15:02:02.800 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.800 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.800 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:02:02.800 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.241559+0000", 2024-09-21T15:02:02.800 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0020622459999999 2024-09-21T15:02:02.800 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.800 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.800 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:02:02.800 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.241784+0000", 2024-09-21T15:02:02.800 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000225009 2024-09-21T15:02:02.800 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:02.801 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:02:02.801 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:02.801 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.801 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.801 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:140335 3.1 3:80f1e8f1:::rbd_data.113664780c85.000000000000017c:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e174)", 2024-09-21T15:02:02.801 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:02.239409+0000", 2024-09-21T15:02:02.801 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 60.474139094999998, 2024-09-21T15:02:02.801 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0025803150000001, 2024-09-21T15:02:02.801 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:02:02.801 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:02:02.801 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:02:02.801 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:02:02.801 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:02:02.801 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 140335 2024-09-21T15:02:02.801 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.802 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:02:02.802 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.802 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:02:02.802 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239409+0000", 2024-09-21T15:02:02.802 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:02:02.802 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.802 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.802 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:02:02.802 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239409+0000", 2024-09-21T15:02:02.802 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:02:02.802 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.802 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.802 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:02:02.803 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239409+0000", 2024-09-21T15:02:02.803 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:02:02.803 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.803 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.803 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:02:02.803 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239413+0000", 2024-09-21T15:02:02.803 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7620000000000003e-06 2024-09-21T15:02:02.803 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.803 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.803 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:02:02.803 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239436+0000", 2024-09-21T15:02:02.803 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2889000000000001e-05 2024-09-21T15:02:02.803 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.803 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.804 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:02:02.804 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239444+0000", 2024-09-21T15:02:02.804 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6809999999999998e-06 2024-09-21T15:02:02.804 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.804 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.804 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:02:02.804 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239542+0000", 2024-09-21T15:02:02.804 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7868999999999994e-05 2024-09-21T15:02:02.804 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.804 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.804 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:02:02.804 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:02.239558+0000", 2024-09-21T15:02:02.804 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.6257e-05 2024-09-21T15:02:02.804 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.805 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.805 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:02:02.805 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.241847+0000", 2024-09-21T15:02:02.805 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.002288533 2024-09-21T15:02:02.805 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.805 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.805 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:02:02.805 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:03.241990+0000", 2024-09-21T15:02:02.805 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00014308300000000001 2024-09-21T15:02:02.805 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:02.805 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:02:02.805 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:02.805 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.805 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.805 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:166488 3.4 3.7bcf3e84 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e189)", 2024-09-21T15:02:02.805 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:21.342023+0000", 2024-09-21T15:02:02.806 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 41.371525263000002, 2024-09-21T15:02:02.806 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0015098469999999, 2024-09-21T15:02:02.806 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:02:02.806 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:02:02.806 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:02:02.806 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:02:02.806 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:02:02.806 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 166488 2024-09-21T15:02:02.806 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.806 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:02:02.806 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.806 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:02:02.806 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:21.342023+0000", 2024-09-21T15:02:02.806 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:02:02.807 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.807 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.807 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:02:02.807 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:21.342023+0000", 2024-09-21T15:02:02.807 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:02:02.807 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.807 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.807 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:02:02.807 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:21.342022+0000", 2024-09-21T15:02:02.807 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:02:02.807 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.807 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.807 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:02:02.807 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:21.342029+0000", 2024-09-21T15:02:02.807 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1420000000000004e-06 2024-09-21T15:02:02.808 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.808 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.808 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:02:02.808 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:21.342051+0000", 2024-09-21T15:02:02.808 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2549999999999999e-05 2024-09-21T15:02:02.808 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.808 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.808 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:02:02.808 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:21.342059+0000", 2024-09-21T15:02:02.808 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.8490000000000006e-06 2024-09-21T15:02:02.808 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.808 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.808 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:02:02.808 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:21.342105+0000", 2024-09-21T15:02:02.808 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.5488999999999998e-05 2024-09-21T15:02:02.809 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.809 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.809 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:02:02.809 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:21.342115+0000", 2024-09-21T15:02:02.809 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0633e-05 2024-09-21T15:02:02.809 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.809 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.809 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:02:02.809 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:22.343521+0000", 2024-09-21T15:02:02.809 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0014057270000001 2024-09-21T15:02:02.809 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:02.809 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:02.809 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:02:02.809 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:22.343533+0000", 2024-09-21T15:02:02.809 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2097e-05 2024-09-21T15:02:02.810 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:02.810 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:02:02.810 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:02.810 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:02.810 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:02:02.810 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:02:02.811 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T15:02:02.924 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:02:02.924 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:02:02.925 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T15:02:02.925 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:02:02.934 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T15:02:03.040 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:02:03.041 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:02:03.041 INFO:teuthology.orchestra.run.smithi046.stdout: "complaint_time": 30, 2024-09-21T15:02:03.041 INFO:teuthology.orchestra.run.smithi046.stdout: "num_blocked_ops": 0 2024-09-21T15:02:03.041 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:02:03.051 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T15:02:03.162 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:02:03.162 INFO:teuthology.orchestra.run.smithi046.stdout: "size": 20, 2024-09-21T15:02:03.162 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 600, 2024-09-21T15:02:03.162 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [ 2024-09-21T15:02:03.162 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.162 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31476 3.c 3:35718b6b:::rbd_data.11367721c19.0000000000000014:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T15:02:03.162 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.505975+0000", 2024-09-21T15:02:03.162 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 184.65081022300001, 2024-09-21T15:02:03.162 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99515995899999998, 2024-09-21T15:02:03.162 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:02:03.163 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:02:03.163 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:02:03.163 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:02:03.163 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:02:03.163 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31476 2024-09-21T15:02:03.163 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.163 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:02:03.163 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.163 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:02:03.163 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.505975+0000", 2024-09-21T15:02:03.163 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:02:03.163 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.163 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.163 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:02:03.164 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.505975+0000", 2024-09-21T15:02:03.164 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:02:03.164 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.164 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.164 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:02:03.164 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.505971+0000", 2024-09-21T15:02:03.164 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999967 2024-09-21T15:02:03.164 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.164 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.164 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:02:03.164 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.505989+0000", 2024-09-21T15:02:03.164 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7985000000000001e-05 2024-09-21T15:02:03.164 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.165 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.165 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:02:03.165 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506050+0000", 2024-09-21T15:02:03.165 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.0659000000000003e-05 2024-09-21T15:02:03.165 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.165 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.165 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:02:03.165 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506074+0000", 2024-09-21T15:02:03.165 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.4156e-05 2024-09-21T15:02:03.165 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.165 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.165 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:02:03.165 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506225+0000", 2024-09-21T15:02:03.165 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015097 2024-09-21T15:02:03.166 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.166 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.166 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:02:03.166 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506256+0000", 2024-09-21T15:02:03.166 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.0505999999999999e-05 2024-09-21T15:02:03.166 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.166 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.166 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:02:03.166 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.497974+0000", 2024-09-21T15:02:03.166 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99171825700000005 2024-09-21T15:02:03.166 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.166 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.166 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for missing object", 2024-09-21T15:02:03.167 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.498449+0000", 2024-09-21T15:02:03.167 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00047506399999999998 2024-09-21T15:02:03.167 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.167 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.167 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:02:03.167 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.501000+0000", 2024-09-21T15:02:03.167 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0025506919999999998 2024-09-21T15:02:03.167 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.167 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.167 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:02:03.167 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.501135+0000", 2024-09-21T15:02:03.167 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013524000000000001 2024-09-21T15:02:03.167 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:03.167 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:02:03.167 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:03.168 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.168 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.168 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31480 3.c 3:36ccd808:::rbd_data.11367721c19.0000000000000018:head [sparse-read 0~14821] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T15:02:03.168 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.506116+0000", 2024-09-21T15:02:03.168 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 184.65066888000001, 2024-09-21T15:02:03.168 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99316033599999998, 2024-09-21T15:02:03.168 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:02:03.168 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:02:03.168 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:02:03.168 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:02:03.168 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:02:03.168 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31480 2024-09-21T15:02:03.168 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.168 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:02:03.168 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.169 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:02:03.169 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506116+0000", 2024-09-21T15:02:03.169 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:02:03.169 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.169 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.169 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:02:03.169 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506116+0000", 2024-09-21T15:02:03.169 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:02:03.169 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.169 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.169 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:02:03.169 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506115+0000", 2024-09-21T15:02:03.169 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:02:03.169 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.170 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.170 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:02:03.170 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506124+0000", 2024-09-21T15:02:03.170 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.0659999999999993e-06 2024-09-21T15:02:03.170 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.170 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.170 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:02:03.170 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506145+0000", 2024-09-21T15:02:03.170 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.1447e-05 2024-09-21T15:02:03.170 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.170 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.170 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:02:03.170 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506161+0000", 2024-09-21T15:02:03.170 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5209e-05 2024-09-21T15:02:03.170 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.171 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.171 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:02:03.171 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506337+0000", 2024-09-21T15:02:03.171 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017642999999999999 2024-09-21T15:02:03.171 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.171 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.171 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:02:03.171 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.506375+0000", 2024-09-21T15:02:03.171 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7536000000000001e-05 2024-09-21T15:02:03.171 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.171 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.171 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:02:03.171 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.498815+0000", 2024-09-21T15:02:03.171 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99244067999999996 2024-09-21T15:02:03.172 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.172 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.172 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:02:03.172 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.498950+0000", 2024-09-21T15:02:03.172 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013436700000000001 2024-09-21T15:02:03.172 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.172 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.172 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:02:03.172 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499277+0000", 2024-09-21T15:02:03.172 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00032696800000000002 2024-09-21T15:02:03.172 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:03.172 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:02:03.172 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:03.172 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.172 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.173 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31513 3.c 3:36de8523:::rbd_data.11367721c19.0000000000000039:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T15:02:03.173 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.509071+0000", 2024-09-21T15:02:03.173 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 184.64771468000001, 2024-09-21T15:02:03.173 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99038225099999999, 2024-09-21T15:02:03.173 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:02:03.173 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:02:03.173 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:02:03.173 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:02:03.173 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:02:03.173 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31513 2024-09-21T15:02:03.173 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.173 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:02:03.173 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.173 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:02:03.173 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509071+0000", 2024-09-21T15:02:03.174 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:02:03.174 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.174 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.174 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:02:03.174 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509071+0000", 2024-09-21T15:02:03.174 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:02:03.174 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.174 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.174 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:02:03.174 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509068+0000", 2024-09-21T15:02:03.174 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-21T15:02:03.174 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.174 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.174 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:02:03.174 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509082+0000", 2024-09-21T15:02:03.175 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3545e-05 2024-09-21T15:02:03.175 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.175 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.175 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:02:03.175 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509116+0000", 2024-09-21T15:02:03.175 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3942000000000001e-05 2024-09-21T15:02:03.175 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.175 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.175 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:02:03.175 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509134+0000", 2024-09-21T15:02:03.175 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.8839999999999999e-05 2024-09-21T15:02:03.175 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.175 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.176 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:02:03.176 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509318+0000", 2024-09-21T15:02:03.176 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018326799999999999 2024-09-21T15:02:03.176 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.176 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.176 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:02:03.176 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509346+0000", 2024-09-21T15:02:03.176 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.8019000000000001e-05 2024-09-21T15:02:03.176 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.176 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.176 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:02:03.176 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499330+0000", 2024-09-21T15:02:03.176 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.98998399299999995 2024-09-21T15:02:03.176 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.177 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.177 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:02:03.177 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499453+0000", 2024-09-21T15:02:03.177 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000123115 2024-09-21T15:02:03.177 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:03.177 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:02:03.177 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:03.177 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.177 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.177 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31515 3.c 3:35b36ddd:::rbd_data.11367721c19.000000000000003b:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T15:02:03.177 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.509177+0000", 2024-09-21T15:02:03.177 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 184.647608254, 2024-09-21T15:02:03.177 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99044870600000001, 2024-09-21T15:02:03.177 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:02:03.177 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "reached pg", 2024-09-21T15:02:03.178 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:02:03.178 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:02:03.178 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:02:03.178 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31515 2024-09-21T15:02:03.178 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.178 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:02:03.178 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.178 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:02:03.178 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509177+0000", 2024-09-21T15:02:03.178 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:02:03.178 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.178 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.178 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:02:03.179 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509177+0000", 2024-09-21T15:02:03.179 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:02:03.179 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.179 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.179 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:02:03.179 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509176+0000", 2024-09-21T15:02:03.179 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:02:03.179 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.179 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.179 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:02:03.179 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509183+0000", 2024-09-21T15:02:03.179 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6720000000000004e-06 2024-09-21T15:02:03.179 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.179 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.180 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:02:03.180 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509269+0000", 2024-09-21T15:02:03.180 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.5881e-05 2024-09-21T15:02:03.180 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.180 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.180 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:02:03.180 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509303+0000", 2024-09-21T15:02:03.180 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3367000000000001e-05 2024-09-21T15:02:03.180 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.180 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.180 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:02:03.180 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509425+0000", 2024-09-21T15:02:03.180 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000122199 2024-09-21T15:02:03.180 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.180 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.181 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:02:03.181 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509448+0000", 2024-09-21T15:02:03.181 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.2688999999999999e-05 2024-09-21T15:02:03.181 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.181 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.181 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:02:03.181 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499501+0000", 2024-09-21T15:02:03.181 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99005306999999998 2024-09-21T15:02:03.181 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.181 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.181 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:02:03.181 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499626+0000", 2024-09-21T15:02:03.181 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00012501400000000001 2024-09-21T15:02:03.181 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:03.181 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:02:03.182 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:03.182 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.182 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.182 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:31531 3.c 3:314f0287:::rbd_data.11367721c19.000000000000004b:head [sparse-read 0~7106] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-21T15:02:03.182 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:58:58.509329+0000", 2024-09-21T15:02:03.182 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 184.647456234, 2024-09-21T15:02:03.182 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99103085300000004, 2024-09-21T15:02:03.182 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:02:03.182 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:02:03.182 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:02:03.182 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:02:03.182 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:02:03.182 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 31531 2024-09-21T15:02:03.182 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.183 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:02:03.183 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.183 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:02:03.183 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509329+0000", 2024-09-21T15:02:03.183 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:02:03.183 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.183 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.183 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:02:03.183 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509329+0000", 2024-09-21T15:02:03.183 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:02:03.183 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.183 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.183 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:02:03.183 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509328+0000", 2024-09-21T15:02:03.184 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:02:03.184 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.184 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.184 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:02:03.184 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509336+0000", 2024-09-21T15:02:03.184 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3170000000000002e-06 2024-09-21T15:02:03.184 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.184 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.184 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:02:03.184 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509353+0000", 2024-09-21T15:02:03.184 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.7214000000000002e-05 2024-09-21T15:02:03.184 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.184 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.184 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:02:03.185 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509366+0000", 2024-09-21T15:02:03.185 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.26e-05 2024-09-21T15:02:03.185 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.185 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.185 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:02:03.185 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509523+0000", 2024-09-21T15:02:03.185 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015708000000000001 2024-09-21T15:02:03.185 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.185 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.185 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:02:03.185 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:58.509544+0000", 2024-09-21T15:02:03.185 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0783000000000001e-05 2024-09-21T15:02:03.185 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.185 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.185 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:02:03.186 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499727+0000", 2024-09-21T15:02:03.186 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.99018345200000002 2024-09-21T15:02:03.186 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.186 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.186 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:02:03.186 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.499949+0000", 2024-09-21T15:02:03.186 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000221358 2024-09-21T15:02:03.186 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.186 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.186 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:02:03.186 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:58:59.500360+0000", 2024-09-21T15:02:03.186 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00041130700000000002 2024-09-21T15:02:03.186 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:03.186 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:02:03.187 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:03.187 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.187 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.187 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:82545 3.b 3:d4eb1ee5:::rbd_data.11368296f791.0000000000000000:head [write 0~671744 [fadvise_sequential+fadvise_nocache],write 704512~65536 [fadvise_sequential+fadvise_nocache],write 786432~49152 [fadvise_sequential+fadvise_nocache],write 868352~49152 [fadvise_sequential+fadvise_nocache],write 933888~65536 [fadvise_sequential+fadvise_nocache],write 1032192~32768 [fadvise_sequential+fadvise_nocache],write 1097728~65536 [fadvise_sequential+fadvise_nocache],write 1179648~49152 [fadvise_sequential+fadvise_nocache],write 1261568~49152 [fadvise_sequential+fadvise_nocache],write 1327104~65536 [fadvise_sequential+fadvise_nocache],write 1425408~32768 [fadvise_sequential+fadvise_nocache],write 1490944~65536 [fadvise_sequential+fadvise_nocache],write 1572864~8192 [fadvise_sequential+fadvise_nocache],write 1630208~49152 [fadvise_sequential+fadvise_nocache],write 1703936~40960 [fadvise_sequential+fadvise_nocache],write 1794048~40960 [fadvise_sequential+fadvise_nocache],write 1859584~49152 [fadvise_sequential+fadvise_nocache],write 1957888~8192 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:02:03.187 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T14:59:47.416089+0000", 2024-09-21T15:02:03.187 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 135.740696509, 2024-09-21T15:02:03.187 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011896455, 2024-09-21T15:02:03.187 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:02:03.187 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:02:03.187 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:02:03.187 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:02:03.187 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:02:03.188 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 82545 2024-09-21T15:02:03.188 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.188 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:02:03.188 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.188 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:02:03.188 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.416089+0000", 2024-09-21T15:02:03.188 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:02:03.188 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.188 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.188 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:02:03.188 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.416089+0000", 2024-09-21T15:02:03.188 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:02:03.188 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.188 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.189 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:02:03.189 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.416087+0000", 2024-09-21T15:02:03.189 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:02:03.189 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.189 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.189 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:02:03.189 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.419643+0000", 2024-09-21T15:02:03.189 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0035555550000000002 2024-09-21T15:02:03.189 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.189 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.189 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:02:03.189 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.419663+0000", 2024-09-21T15:02:03.189 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.0202000000000001e-05 2024-09-21T15:02:03.189 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.189 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.190 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:02:03.190 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.419676+0000", 2024-09-21T15:02:03.190 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3207000000000001e-05 2024-09-21T15:02:03.190 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.190 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.190 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:02:03.190 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.419759+0000", 2024-09-21T15:02:03.190 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3521000000000002e-05 2024-09-21T15:02:03.190 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.190 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.190 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op must wait for map", 2024-09-21T15:02:03.190 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.419768+0000", 2024-09-21T15:02:03.190 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4479999999999994e-06 2024-09-21T15:02:03.190 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.191 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.191 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:02:03.191 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.422171+0000", 2024-09-21T15:02:03.191 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0024035469999999998 2024-09-21T15:02:03.191 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.191 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.191 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:02:03.191 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.422350+0000", 2024-09-21T15:02:03.191 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000179001 2024-09-21T15:02:03.191 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.191 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.191 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T15:02:03.191 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.422710+0000", 2024-09-21T15:02:03.191 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000359878 2024-09-21T15:02:03.192 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.192 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.192 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:02:03.192 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.426676+0000", 2024-09-21T15:02:03.192 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0039655790000000003 2024-09-21T15:02:03.192 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.192 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.192 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:02:03.192 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.427884+0000", 2024-09-21T15:02:03.192 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.001208207 2024-09-21T15:02:03.192 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.192 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.192 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:02:03.193 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.427953+0000", 2024-09-21T15:02:03.193 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8533000000000003e-05 2024-09-21T15:02:03.193 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.193 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.193 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:02:03.193 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T14:59:47.427985+0000", 2024-09-21T15:02:03.193 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.2481999999999998e-05 2024-09-21T15:02:03.193 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:03.193 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:02:03.194 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:03.194 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.194 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.194 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:97675 3.b 3:d321acb8:::rbd_data.1136bfc79a70.0000000000000003:head [write 0~4718592 [fadvise_sequential+fadvise_nocache],write 5242880~147456 [fadvise_sequential+fadvise_nocache],write 5455872~147456 [fadvise_sequential+fadvise_nocache],write 5668864~229376 [fadvise_sequential+fadvise_nocache],write 5963776~147456 [fadvise_sequential+fadvise_nocache],write 6176768~229376 [fadvise_sequential+fadvise_nocache],write 6471680~147456 [fadvise_sequential+fadvise_nocache],write 6684672~229376 [fadvise_sequential+fadvise_nocache],write 6979584~147456 [fadvise_sequential+fadvise_nocache],write 7192576~229376 [fadvise_sequential+fadvise_nocache],write 7487488~147456 [fadvise_sequential+fadvise_nocache],write 7700480~147456 [fadvise_sequential+fadvise_nocache],write 7864320~65536 [fadvise_sequential+fadvise_nocache],write 7995392~147456 [fadvise_sequential+fadvise_nocache],write 8208384~147456 [fadvise_sequential+fadvise_nocache],write 8388608~49152 [fadvise_sequential+fadvise_nocache],write 8536064~114688 [fadvise_sequential+fadvise_nocache],write 8749056~114688 [fadvise_sequential+fadvise_nocache],write 8912896~32768 [fadvise_sequential+fadvise_nocache],write 9125888~32768 [fadvise_sequential+fadvise_nocache],write 9338880~32768 [fadvise_sequential+fadvise_nocache],write 9437184~16384 [fadvise_sequential+fadvise_nocache],write 9633792~32768 [fadvise_sequential+fadvise_nocache],write 9846784~32768 [fadvise_sequential+fadvise_nocache],write 9961472~114688 [fadvise_sequential+fadvise_nocache],write 10092544~196608 [fadvise_sequential+fadvise_nocache],write 10305536~278528 [fadvise_sequential+fadvise_nocache],write 10600448~196608 [fadvise_sequential+fadvise_nocache],write 10813440~278528 [fadvise_sequential+fadvise_nocache],write 11108352~196608 [fadvise_sequential+fadvise_nocache],write 11321344~196608 [fadvise_sequential+fadvise_nocache],write 11534336~65536 [fadvise_sequential+fadvise_nocache],write 11632640~98304 [fadvise_sequential+fadvise_nocache],write 11747328~65536 [fadvise_sequential+fadvise_nocache],write 11845632~98304 [fadvise_sequential+fadvise_nocache],write 11960320~65536 [fadvise_sequential+fadvise_nocache],write 12058624~49152 [fadvise_sequential+fadvise_nocache],write 12140544~98304 [fadvise_sequential+fadvise_nocache],write 12255232~65536 [fadvise_sequential+fadvise_nocache],write 12353536~98304 [fadvise_sequential+fadvise_nocache],write 12468224~65536 [fadvise_sequential+fadvise_nocache],write 12566528~16384 [fadvise_sequential+fadvise_nocache],write 12648448~98304 [fadvise_sequential+fadvise_nocache],write 12861440~98304 [fadvise_sequential+fadvise_nocache],write 13074432~32768 [fadvise_sequential+fadvise_nocache],write 13156352~98304 [fadvise_sequential+fadvise_nocache],write 13369344~98304 [fadvise_sequential+fadvise_nocache],write 13582336~49152 [fadvise_sequential+fadvise_nocache],write 13664256~98304 [fadvise_sequential+fadvise_nocache],write 13877248~98304 [fadvise_sequential+fadvise_nocache],write 14090240~65536 [fadvise_sequential+fadvise_nocache],write 15286272~81920 [fadvise_sequential+fadvise_nocache],write 15499264~81920 [fadvise_sequential+fadvise_nocache],write 15712256~16384 [fadvise_sequential+fadvise_nocache],write 15794176~81920 [fadvise_sequential+fadvise_nocache],write 16007168~81920 [fadvise_sequential+fadvise_nocache],write 16220160~32768 [fadvise_sequential+fadvise_nocache],write 16302080~81920 [fadvise_sequential+fadvise_nocache],write 16515072~81920 [fadvise_sequential+fadvise_nocache],write 16728064~49152 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:02:03.194 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.302412+0000", 2024-09-21T15:02:03.194 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 117.85437372, 2024-09-21T15:02:03.194 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.067380532000000007, 2024-09-21T15:02:03.194 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:02:03.194 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:02:03.194 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:02:03.194 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:02:03.194 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:02:03.195 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97675 2024-09-21T15:02:03.195 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.195 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:02:03.195 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.195 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:02:03.195 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.302412+0000", 2024-09-21T15:02:03.195 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:02:03.195 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.195 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.195 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:02:03.195 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.302412+0000", 2024-09-21T15:02:03.196 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:02:03.196 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.196 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.196 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:02:03.196 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.302409+0000", 2024-09-21T15:02:03.196 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999976 2024-09-21T15:02:03.196 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.196 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.196 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:02:03.196 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328550+0000", 2024-09-21T15:02:03.196 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.026140674999999999 2024-09-21T15:02:03.196 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.197 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.197 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:02:03.197 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328584+0000", 2024-09-21T15:02:03.197 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.4165999999999998e-05 2024-09-21T15:02:03.197 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.197 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.197 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:02:03.197 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328599+0000", 2024-09-21T15:02:03.197 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5505000000000001e-05 2024-09-21T15:02:03.197 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.197 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.197 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:02:03.198 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328704+0000", 2024-09-21T15:02:03.198 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010487199999999999 2024-09-21T15:02:03.198 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.198 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.198 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:02:03.198 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.329082+0000", 2024-09-21T15:02:03.198 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00037804899999999998 2024-09-21T15:02:03.198 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.198 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.198 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for subops from 6", 2024-09-21T15:02:03.198 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.331158+0000", 2024-09-21T15:02:03.198 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0020755449999999998 2024-09-21T15:02:03.198 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.199 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.199 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "op_commit", 2024-09-21T15:02:03.199 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.360553+0000", 2024-09-21T15:02:03.199 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.029395553000000001 2024-09-21T15:02:03.199 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.199 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.199 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:02:03.199 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.369496+0000", 2024-09-21T15:02:03.199 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0089428619999999993 2024-09-21T15:02:03.199 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.199 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.200 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:02:03.200 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.369686+0000", 2024-09-21T15:02:03.200 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00019000399999999999 2024-09-21T15:02:03.200 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.200 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.200 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:02:03.200 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.369792+0000", 2024-09-21T15:02:03.200 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000105775 2024-09-21T15:02:03.200 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:03.200 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:02:03.200 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:03.200 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.200 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.200 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:97693 3.b 3:d7d7b39b:::rbd_data.1136b9c49b1c.000000000000000c:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:02:03.201 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.328643+0000", 2024-09-21T15:02:03.201 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 117.828141934, 2024-09-21T15:02:03.201 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.031062136000000001, 2024-09-21T15:02:03.201 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:02:03.201 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:02:03.201 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:02:03.201 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:02:03.201 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:02:03.201 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97693 2024-09-21T15:02:03.201 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.201 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:02:03.202 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.202 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:02:03.202 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328643+0000", 2024-09-21T15:02:03.202 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:02:03.202 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.202 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.202 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:02:03.202 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328643+0000", 2024-09-21T15:02:03.202 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:02:03.202 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.202 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.202 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:02:03.202 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328642+0000", 2024-09-21T15:02:03.203 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999981 2024-09-21T15:02:03.203 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.203 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.203 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:02:03.203 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328650+0000", 2024-09-21T15:02:03.203 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.7770000000000008e-06 2024-09-21T15:02:03.203 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.203 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.203 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:02:03.203 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328662+0000", 2024-09-21T15:02:03.203 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2026e-05 2024-09-21T15:02:03.203 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.203 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.204 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:02:03.204 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328671+0000", 2024-09-21T15:02:03.204 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.585e-06 2024-09-21T15:02:03.204 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.204 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.204 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:02:03.204 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.359498+0000", 2024-09-21T15:02:03.204 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.030827001 2024-09-21T15:02:03.204 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.204 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.204 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:02:03.204 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.359603+0000", 2024-09-21T15:02:03.205 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00010512400000000001 2024-09-21T15:02:03.205 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.205 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.205 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:02:03.205 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.359705+0000", 2024-09-21T15:02:03.205 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000102327 2024-09-21T15:02:03.205 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:03.205 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:02:03.205 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:03.205 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.205 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.205 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:97724 3.b 3:d4bf6bcf:::rbd_data.1136b9c49b1c.000000000000002b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-21T15:02:03.205 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:05.328704+0000", 2024-09-21T15:02:03.205 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 117.828081662, 2024-09-21T15:02:03.206 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.031157957, 2024-09-21T15:02:03.206 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:02:03.206 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:02:03.206 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:02:03.206 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:02:03.206 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:02:03.206 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 97724 2024-09-21T15:02:03.206 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.206 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:02:03.206 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.206 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:02:03.206 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328704+0000", 2024-09-21T15:02:03.207 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:02:03.207 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.207 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.207 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:02:03.207 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328704+0000", 2024-09-21T15:02:03.207 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:02:03.207 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.207 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.207 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:02:03.207 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328703+0000", 2024-09-21T15:02:03.207 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:02:03.207 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.207 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.207 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:02:03.208 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328708+0000", 2024-09-21T15:02:03.208 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.0050000000000004e-06 2024-09-21T15:02:03.208 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.208 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.208 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:02:03.208 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328719+0000", 2024-09-21T15:02:03.208 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.1052e-05 2024-09-21T15:02:03.208 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.208 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.208 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:02:03.208 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.328727+0000", 2024-09-21T15:02:03.208 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.4519999999999997e-06 2024-09-21T15:02:03.208 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.208 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.209 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:02:03.209 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.359737+0000", 2024-09-21T15:02:03.209 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.031009341999999999 2024-09-21T15:02:03.209 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.209 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.209 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:02:03.209 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.359806+0000", 2024-09-21T15:02:03.209 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.9750000000000001e-05 2024-09-21T15:02:03.209 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.209 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.209 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:02:03.209 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:05.359862+0000", 2024-09-21T15:02:03.209 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.5084999999999999e-05 2024-09-21T15:02:03.210 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:03.210 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:02:03.210 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:03.210 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.210 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.210 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_repop(client.4406.0:129970 3.c e158/87 3:33fa4d4e:::rbd_data.1136a12e4351.0000000000000000:head v 158'379, mlcod=156'377)", 2024-09-21T15:02:03.210 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:44.452766+0000", 2024-09-21T15:02:03.210 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 78.704019110000004, 2024-09-21T15:02:03.210 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011942062, 2024-09-21T15:02:03.210 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:02:03.210 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:02:03.210 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:02:03.210 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.211 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:02:03.211 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:44.452766+0000", 2024-09-21T15:02:03.211 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:02:03.211 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.211 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.211 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:02:03.211 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:44.452766+0000", 2024-09-21T15:02:03.211 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:02:03.211 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.211 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.211 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:02:03.212 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:44.452766+0000", 2024-09-21T15:02:03.212 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:02:03.212 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.212 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.212 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:02:03.212 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:44.458962+0000", 2024-09-21T15:02:03.212 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0061964799999999999 2024-09-21T15:02:03.212 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.212 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.212 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:02:03.212 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:44.459012+0000", 2024-09-21T15:02:03.212 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.9427e-05 2024-09-21T15:02:03.212 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.213 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.213 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:02:03.213 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:44.459042+0000", 2024-09-21T15:02:03.213 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9838000000000001e-05 2024-09-21T15:02:03.213 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.213 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.213 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:02:03.213 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:44.459115+0000", 2024-09-21T15:02:03.213 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.3430999999999995e-05 2024-09-21T15:02:03.213 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.213 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.213 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:02:03.213 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:44.459135+0000", 2024-09-21T15:02:03.213 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.9701999999999999e-05 2024-09-21T15:02:03.214 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.214 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.214 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "commit_sent", 2024-09-21T15:02:03.214 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:44.464645+0000", 2024-09-21T15:02:03.214 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.0055100820000000003 2024-09-21T15:02:03.214 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.214 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.214 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:02:03.214 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:44.464708+0000", 2024-09-21T15:02:03.214 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.3399999999999996e-05 2024-09-21T15:02:03.214 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:03.214 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:02:03.214 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:03.215 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.215 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.215 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:135368 3.b 3:d3301c39:::rbd_data.11365ed419e3.0000000000000102:67 [read 335872~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:02:03.215 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:53.998901+0000", 2024-09-21T15:02:03.215 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 69.157884297999999, 2024-09-21T15:02:03.215 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011666174, 2024-09-21T15:02:03.215 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:02:03.215 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:02:03.215 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:02:03.215 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:02:03.215 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:02:03.215 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 135368 2024-09-21T15:02:03.215 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.216 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:02:03.216 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.216 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:02:03.216 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998901+0000", 2024-09-21T15:02:03.216 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:02:03.216 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.216 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.216 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:02:03.216 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998901+0000", 2024-09-21T15:02:03.216 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:02:03.216 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.216 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.216 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:02:03.217 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998901+0000", 2024-09-21T15:02:03.217 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:02:03.217 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.217 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.217 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:02:03.217 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998903+0000", 2024-09-21T15:02:03.217 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.5500000000000001e-06 2024-09-21T15:02:03.217 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.217 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.217 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:02:03.217 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998913+0000", 2024-09-21T15:02:03.217 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.6479999999999998e-06 2024-09-21T15:02:03.217 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.218 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.218 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:02:03.218 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998919+0000", 2024-09-21T15:02:03.218 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7930000000000003e-06 2024-09-21T15:02:03.218 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.218 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.218 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:02:03.218 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:54.010277+0000", 2024-09-21T15:02:03.218 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011358692 2024-09-21T15:02:03.218 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.218 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.218 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:02:03.219 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:54.010348+0000", 2024-09-21T15:02:03.219 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.0247999999999996e-05 2024-09-21T15:02:03.219 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.219 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.219 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:02:03.219 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:54.010567+0000", 2024-09-21T15:02:03.219 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00021953599999999999 2024-09-21T15:02:03.219 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:03.219 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:02:03.219 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:03.219 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.219 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.219 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:135369 3.b 3:d3301c39:::rbd_data.11365ed419e3.0000000000000102:67 [read 352256~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e165)", 2024-09-21T15:02:03.220 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:53.998923+0000", 2024-09-21T15:02:03.220 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 69.157861996999998, 2024-09-21T15:02:03.220 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011990331999999999, 2024-09-21T15:02:03.220 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:02:03.220 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:02:03.220 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:02:03.220 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:02:03.220 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:02:03.220 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 135369 2024-09-21T15:02:03.220 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.220 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:02:03.220 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.220 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:02:03.221 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998923+0000", 2024-09-21T15:02:03.221 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:02:03.221 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.221 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.221 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:02:03.221 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998923+0000", 2024-09-21T15:02:03.221 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:02:03.221 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.221 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.221 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:02:03.221 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998923+0000", 2024-09-21T15:02:03.222 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:02:03.222 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.222 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.222 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:02:03.222 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998926+0000", 2024-09-21T15:02:03.222 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6749999999999998e-06 2024-09-21T15:02:03.222 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.222 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.222 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:02:03.222 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998935+0000", 2024-09-21T15:02:03.223 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.3619999999999995e-06 2024-09-21T15:02:03.223 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.223 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.223 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:02:03.223 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:53.998940+0000", 2024-09-21T15:02:03.223 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.2379999999999997e-06 2024-09-21T15:02:03.223 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.223 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.223 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:02:03.223 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:54.010593+0000", 2024-09-21T15:02:03.223 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011652531000000001 2024-09-21T15:02:03.224 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.224 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.224 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:02:03.224 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:54.010679+0000", 2024-09-21T15:02:03.224 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6811000000000006e-05 2024-09-21T15:02:03.224 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.224 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.224 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:02:03.224 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:54.010914+0000", 2024-09-21T15:02:03.224 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000234071 2024-09-21T15:02:03.224 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:03.224 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:02:03.225 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:03.225 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.225 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.225 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:137706 3.b 3:d0cb966d:::rbd_data.1136f64ddcfe.0000000000000005:6a [read 1441792~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-21T15:02:03.225 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:58.376385+0000", 2024-09-21T15:02:03.225 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 64.780400409999999, 2024-09-21T15:02:03.225 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011783609, 2024-09-21T15:02:03.225 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:02:03.225 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:02:03.225 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:02:03.225 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:02:03.225 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:02:03.226 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 137706 2024-09-21T15:02:03.226 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.226 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:02:03.226 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.226 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:02:03.226 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376385+0000", 2024-09-21T15:02:03.226 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:02:03.226 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.226 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.226 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:02:03.226 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376385+0000", 2024-09-21T15:02:03.226 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:02:03.226 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.227 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.227 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:02:03.227 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376384+0000", 2024-09-21T15:02:03.227 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:02:03.227 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.227 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.227 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:02:03.227 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376387+0000", 2024-09-21T15:02:03.227 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.9809999999999999e-06 2024-09-21T15:02:03.227 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.227 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.228 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:02:03.228 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376398+0000", 2024-09-21T15:02:03.228 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.0319000000000001e-05 2024-09-21T15:02:03.228 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.228 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.228 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:02:03.228 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376404+0000", 2024-09-21T15:02:03.228 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.5339999999999996e-06 2024-09-21T15:02:03.228 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.228 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.228 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:02:03.228 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.388025+0000", 2024-09-21T15:02:03.228 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011620449999999999 2024-09-21T15:02:03.229 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.229 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.229 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:02:03.229 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.388082+0000", 2024-09-21T15:02:03.229 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.7065999999999999e-05 2024-09-21T15:02:03.229 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.229 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.229 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:02:03.229 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.388168+0000", 2024-09-21T15:02:03.229 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.6689999999999998e-05 2024-09-21T15:02:03.229 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:03.229 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:02:03.229 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:03.230 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.230 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.230 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:137707 3.b 3:d0cb966d:::rbd_data.1136f64ddcfe.0000000000000005:6a [read 1556480~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-21T15:02:03.230 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:58.376428+0000", 2024-09-21T15:02:03.230 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 64.780356917999995, 2024-09-21T15:02:03.230 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012004865999999999, 2024-09-21T15:02:03.230 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:02:03.230 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:02:03.230 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:02:03.230 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:02:03.230 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:02:03.230 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 137707 2024-09-21T15:02:03.231 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.231 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:02:03.231 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.231 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:02:03.231 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376428+0000", 2024-09-21T15:02:03.231 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:02:03.231 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.231 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.231 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:02:03.231 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376428+0000", 2024-09-21T15:02:03.231 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:02:03.231 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.232 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.232 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:02:03.232 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376428+0000", 2024-09-21T15:02:03.232 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:02:03.232 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.232 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.232 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:02:03.232 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376432+0000", 2024-09-21T15:02:03.232 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.2119999999999997e-06 2024-09-21T15:02:03.232 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.232 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.232 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:02:03.233 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376448+0000", 2024-09-21T15:02:03.233 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5913000000000002e-05 2024-09-21T15:02:03.233 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.233 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.233 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:02:03.233 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376455+0000", 2024-09-21T15:02:03.233 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6359999999999994e-06 2024-09-21T15:02:03.233 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.233 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.233 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:02:03.233 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.388197+0000", 2024-09-21T15:02:03.233 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.011741843 2024-09-21T15:02:03.234 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.234 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.234 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:02:03.234 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.388249+0000", 2024-09-21T15:02:03.234 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.1549999999999999e-05 2024-09-21T15:02:03.234 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.234 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.234 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:02:03.234 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.388433+0000", 2024-09-21T15:02:03.234 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00018437999999999999 2024-09-21T15:02:03.234 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:03.234 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:02:03.235 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:03.235 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.235 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.235 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:137708 3.b 3:d0cb966d:::rbd_data.1136f64ddcfe.0000000000000005:6a [read 1671168~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-21T15:02:03.235 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:58.376462+0000", 2024-09-21T15:02:03.235 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 64.780323129999999, 2024-09-21T15:02:03.235 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012315491, 2024-09-21T15:02:03.235 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:02:03.235 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:02:03.235 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:02:03.235 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:02:03.235 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:02:03.235 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 137708 2024-09-21T15:02:03.235 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.236 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:02:03.236 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.236 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:02:03.236 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376462+0000", 2024-09-21T15:02:03.236 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:02:03.236 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.236 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.236 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:02:03.236 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376462+0000", 2024-09-21T15:02:03.236 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:02:03.236 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.237 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.237 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:02:03.237 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376461+0000", 2024-09-21T15:02:03.237 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:02:03.237 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.237 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.237 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:02:03.237 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376465+0000", 2024-09-21T15:02:03.237 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.7069999999999998e-06 2024-09-21T15:02:03.237 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.237 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.237 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:02:03.237 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376473+0000", 2024-09-21T15:02:03.238 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.6669999999999996e-06 2024-09-21T15:02:03.238 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.238 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.238 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:02:03.238 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376479+0000", 2024-09-21T15:02:03.238 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.263e-06 2024-09-21T15:02:03.238 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.238 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.238 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:02:03.238 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.388489+0000", 2024-09-21T15:02:03.238 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012009953 2024-09-21T15:02:03.238 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.239 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.239 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:02:03.239 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.388625+0000", 2024-09-21T15:02:03.239 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013548799999999999 2024-09-21T15:02:03.239 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.239 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.239 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:02:03.239 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.388778+0000", 2024-09-21T15:02:03.239 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00015302299999999999 2024-09-21T15:02:03.239 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:03.239 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:02:03.239 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:03.240 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.240 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.240 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:137709 3.b 3:d0cb966d:::rbd_data.1136f64ddcfe.0000000000000005:6a [read 1785856~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-21T15:02:03.240 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:58.376485+0000", 2024-09-21T15:02:03.240 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 64.780299798000001, 2024-09-21T15:02:03.240 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012466549, 2024-09-21T15:02:03.240 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:02:03.240 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:02:03.240 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:02:03.240 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:02:03.240 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:02:03.240 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 137709 2024-09-21T15:02:03.240 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.241 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:02:03.241 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.241 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:02:03.241 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376485+0000", 2024-09-21T15:02:03.241 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:02:03.241 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.241 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.241 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:02:03.241 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376485+0000", 2024-09-21T15:02:03.241 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:02:03.241 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.241 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.241 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:02:03.242 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376485+0000", 2024-09-21T15:02:03.242 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:02:03.242 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.242 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.242 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:02:03.242 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376492+0000", 2024-09-21T15:02:03.242 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.8929999999999999e-06 2024-09-21T15:02:03.242 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.242 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.242 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:02:03.242 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376500+0000", 2024-09-21T15:02:03.242 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.7989999999999994e-06 2024-09-21T15:02:03.243 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.243 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.243 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:02:03.243 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376506+0000", 2024-09-21T15:02:03.243 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.4799999999999998e-06 2024-09-21T15:02:03.243 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.243 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.243 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:02:03.243 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.388810+0000", 2024-09-21T15:02:03.243 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012303727 2024-09-21T15:02:03.243 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.243 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.243 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:02:03.244 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.388871+0000", 2024-09-21T15:02:03.244 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.1278000000000005e-05 2024-09-21T15:02:03.244 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.244 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.244 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:02:03.244 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.388952+0000", 2024-09-21T15:02:03.244 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.0770000000000001e-05 2024-09-21T15:02:03.244 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:03.244 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:02:03.244 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:03.244 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.244 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.245 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:137710 3.b 3:d0cb966d:::rbd_data.1136f64ddcfe.0000000000000005:6a [read 1900544~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-21T15:02:03.245 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:58.376512+0000", 2024-09-21T15:02:03.245 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 64.780273594999997, 2024-09-21T15:02:03.245 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012746351, 2024-09-21T15:02:03.245 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:02:03.245 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:02:03.245 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:02:03.245 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:02:03.245 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:02:03.245 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 137710 2024-09-21T15:02:03.245 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.245 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:02:03.245 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.246 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:02:03.246 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376512+0000", 2024-09-21T15:02:03.246 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:02:03.246 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.246 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.246 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:02:03.246 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376512+0000", 2024-09-21T15:02:03.246 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:02:03.246 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.246 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.246 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:02:03.246 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376511+0000", 2024-09-21T15:02:03.247 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:02:03.247 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.247 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.247 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:02:03.247 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376514+0000", 2024-09-21T15:02:03.247 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.1719999999999999e-06 2024-09-21T15:02:03.247 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.247 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.247 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:02:03.247 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376522+0000", 2024-09-21T15:02:03.247 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.1389999999999998e-06 2024-09-21T15:02:03.248 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.248 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.248 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:02:03.248 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376535+0000", 2024-09-21T15:02:03.248 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.3658999999999999e-05 2024-09-21T15:02:03.248 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.248 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.248 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:02:03.248 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.389005+0000", 2024-09-21T15:02:03.248 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012470261 2024-09-21T15:02:03.248 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.249 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.249 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:02:03.249 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.389119+0000", 2024-09-21T15:02:03.249 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000113346 2024-09-21T15:02:03.249 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.249 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.249 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:02:03.249 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.389258+0000", 2024-09-21T15:02:03.249 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00013920100000000001 2024-09-21T15:02:03.249 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:03.249 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:02:03.250 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:03.250 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.250 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.250 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:137711 3.b 3:d0cb966d:::rbd_data.1136f64ddcfe.0000000000000005:6a [read 2015232~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-21T15:02:03.250 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:58.376541+0000", 2024-09-21T15:02:03.250 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 64.780244233000005, 2024-09-21T15:02:03.250 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012849955999999999, 2024-09-21T15:02:03.250 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:02:03.250 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:02:03.250 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:02:03.251 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:02:03.251 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:02:03.251 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 137711 2024-09-21T15:02:03.251 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.251 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:02:03.251 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.251 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:02:03.251 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376541+0000", 2024-09-21T15:02:03.251 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:02:03.251 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.251 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.251 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:02:03.252 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376541+0000", 2024-09-21T15:02:03.252 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:02:03.252 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.252 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.252 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:02:03.252 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376540+0000", 2024-09-21T15:02:03.252 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999995 2024-09-21T15:02:03.252 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.252 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.252 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:02:03.252 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376544+0000", 2024-09-21T15:02:03.252 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 3.3249999999999999e-06 2024-09-21T15:02:03.253 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.253 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.253 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:02:03.253 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376551+0000", 2024-09-21T15:02:03.253 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 7.2189999999999998e-06 2024-09-21T15:02:03.253 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.253 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.253 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:02:03.253 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.376557+0000", 2024-09-21T15:02:03.253 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 5.8869999999999997e-06 2024-09-21T15:02:03.253 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.254 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.254 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:02:03.254 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.389277+0000", 2024-09-21T15:02:03.254 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012720555999999999 2024-09-21T15:02:03.254 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.254 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.254 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:02:03.254 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.389325+0000", 2024-09-21T15:02:03.254 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4.7367000000000002e-05 2024-09-21T15:02:03.254 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.254 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.254 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:02:03.255 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.389391+0000", 2024-09-21T15:02:03.255 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.6095999999999999e-05 2024-09-21T15:02:03.255 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:03.255 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:02:03.255 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:03.255 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.255 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.255 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:137723 3.b 3:d0cb966d:::rbd_data.1136f64ddcfe.0000000000000005:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-21T15:02:03.255 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:00:58.377091+0000", 2024-09-21T15:02:03.255 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 64.779693911999999, 2024-09-21T15:02:03.255 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.012594424, 2024-09-21T15:02:03.256 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:02:03.256 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:02:03.256 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:02:03.256 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:02:03.256 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:02:03.256 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 137723 2024-09-21T15:02:03.256 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.256 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:02:03.256 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.256 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:02:03.256 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.377091+0000", 2024-09-21T15:02:03.256 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:02:03.257 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.257 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.257 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:02:03.257 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.377091+0000", 2024-09-21T15:02:03.257 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:02:03.257 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.257 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.257 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:02:03.257 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.377090+0000", 2024-09-21T15:02:03.257 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.999999 2024-09-21T15:02:03.257 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.258 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.258 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:02:03.258 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.377097+0000", 2024-09-21T15:02:03.258 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 6.7100000000000001e-06 2024-09-21T15:02:03.258 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.258 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.258 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:02:03.258 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.377120+0000", 2024-09-21T15:02:03.258 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.3189000000000001e-05 2024-09-21T15:02:03.258 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.258 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.258 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:02:03.259 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.377132+0000", 2024-09-21T15:02:03.259 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2041999999999999e-05 2024-09-21T15:02:03.259 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.259 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.259 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:02:03.259 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.389439+0000", 2024-09-21T15:02:03.259 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.01230698 2024-09-21T15:02:03.259 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.259 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.259 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:02:03.259 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.389556+0000", 2024-09-21T15:02:03.260 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000116446 2024-09-21T15:02:03.260 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.260 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.260 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:02:03.260 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:00:58.389686+0000", 2024-09-21T15:02:03.260 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000129966 2024-09-21T15:02:03.260 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:03.260 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:02:03.260 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:03.260 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.260 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.260 INFO:teuthology.orchestra.run.smithi046.stdout: "description": "osd_op(client.4406.0:166795 3.8 3:1499e676:::rbd_id.image_client.0-clone28:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e193)", 2024-09-21T15:02:03.261 INFO:teuthology.orchestra.run.smithi046.stdout: "initiated_at": "2024-09-21T15:01:25.321589+0000", 2024-09-21T15:02:03.261 INFO:teuthology.orchestra.run.smithi046.stdout: "age": 37.835195747999997, 2024-09-21T15:02:03.261 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.855281438, 2024-09-21T15:02:03.261 INFO:teuthology.orchestra.run.smithi046.stdout: "type_data": { 2024-09-21T15:02:03.261 INFO:teuthology.orchestra.run.smithi046.stdout: "flag_point": "started", 2024-09-21T15:02:03.261 INFO:teuthology.orchestra.run.smithi046.stdout: "client_info": { 2024-09-21T15:02:03.261 INFO:teuthology.orchestra.run.smithi046.stdout: "client": "client.4406", 2024-09-21T15:02:03.261 INFO:teuthology.orchestra.run.smithi046.stdout: "client_addr": "172.21.15.32:0/621097851", 2024-09-21T15:02:03.261 INFO:teuthology.orchestra.run.smithi046.stdout: "tid": 166795 2024-09-21T15:02:03.261 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.261 INFO:teuthology.orchestra.run.smithi046.stdout: "events": [ 2024-09-21T15:02:03.261 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.261 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "initiated", 2024-09-21T15:02:03.262 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:25.321589+0000", 2024-09-21T15:02:03.262 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:02:03.262 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.262 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.262 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "throttled", 2024-09-21T15:02:03.262 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:25.321589+0000", 2024-09-21T15:02:03.262 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0 2024-09-21T15:02:03.262 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.262 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.262 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "header_read", 2024-09-21T15:02:03.262 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:25.321588+0000", 2024-09-21T15:02:03.262 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 4294967295.9999986 2024-09-21T15:02:03.263 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.263 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.263 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "all_read", 2024-09-21T15:02:03.263 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:25.321598+0000", 2024-09-21T15:02:03.263 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 9.7429999999999997e-06 2024-09-21T15:02:03.263 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.263 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.263 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "dispatched", 2024-09-21T15:02:03.263 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:25.321624+0000", 2024-09-21T15:02:03.263 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 2.6752000000000001e-05 2024-09-21T15:02:03.263 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.263 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.264 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "queued_for_pg", 2024-09-21T15:02:03.264 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:25.321637+0000", 2024-09-21T15:02:03.264 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.2809999999999999e-05 2024-09-21T15:02:03.264 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.264 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.264 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:02:03.264 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:25.321721+0000", 2024-09-21T15:02:03.264 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 8.3314999999999998e-05 2024-09-21T15:02:03.264 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.264 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.264 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "waiting for peered", 2024-09-21T15:02:03.264 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:25.321736+0000", 2024-09-21T15:02:03.265 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 1.5262000000000001e-05 2024-09-21T15:02:03.265 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.265 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.265 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "reached_pg", 2024-09-21T15:02:03.265 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:26.176475+0000", 2024-09-21T15:02:03.265 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.85473961200000004 2024-09-21T15:02:03.265 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.265 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.265 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "started", 2024-09-21T15:02:03.265 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:26.176655+0000", 2024-09-21T15:02:03.265 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.00017934400000000001 2024-09-21T15:02:03.265 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-21T15:02:03.265 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-21T15:02:03.266 INFO:teuthology.orchestra.run.smithi046.stdout: "event": "done", 2024-09-21T15:02:03.266 INFO:teuthology.orchestra.run.smithi046.stdout: "time": "2024-09-21T15:01:26.176871+0000", 2024-09-21T15:02:03.266 INFO:teuthology.orchestra.run.smithi046.stdout: "duration": 0.000216133 2024-09-21T15:02:03.266 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:03.266 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:02:03.266 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:03.266 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-21T15:02:03.266 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-21T15:02:03.266 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:02:04.283 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T15:02:05.194 INFO:teuthology.orchestra.run.smithi032.stderr:noscrub is unset 2024-09-21T15:02:05.206 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T15:02:06.273 INFO:teuthology.orchestra.run.smithi032.stderr:nodeep-scrub is unset 2024-09-21T15:02:07.283 INFO:tasks.thrashosds.thrasher:reviving osd 2024-09-21T15:02:07.283 INFO:tasks.thrashosds.thrasher:Reviving osd 4 2024-09-21T15:02:07.283 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-21T15:02:07.284 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-21T15:02:07.286 INFO:tasks.ceph.osd.4:Started 2024-09-21T15:02:07.286 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T15:02:07.396 INFO:teuthology.orchestra.run.smithi046.stderr:no valid command found; 10 closest matches: 2024-09-21T15:02:07.397 INFO:teuthology.orchestra.run.smithi046.stderr:0 2024-09-21T15:02:07.397 INFO:teuthology.orchestra.run.smithi046.stderr:1 2024-09-21T15:02:07.397 INFO:teuthology.orchestra.run.smithi046.stderr:2 2024-09-21T15:02:07.397 INFO:teuthology.orchestra.run.smithi046.stderr:abort 2024-09-21T15:02:07.397 INFO:teuthology.orchestra.run.smithi046.stderr:assert 2024-09-21T15:02:07.397 INFO:teuthology.orchestra.run.smithi046.stderr:config diff 2024-09-21T15:02:07.397 INFO:teuthology.orchestra.run.smithi046.stderr:config diff get 2024-09-21T15:02:07.397 INFO:teuthology.orchestra.run.smithi046.stderr:config get 2024-09-21T15:02:07.397 INFO:teuthology.orchestra.run.smithi046.stderr:config help [] 2024-09-21T15:02:07.397 INFO:teuthology.orchestra.run.smithi046.stderr:config set ... 2024-09-21T15:02:07.397 INFO:teuthology.orchestra.run.smithi046.stderr:admin_socket: invalid command 2024-09-21T15:02:07.399 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-21T15:02:07.399 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-4, ['dump_ops_in_flight'] 2024-09-21T15:02:07.909 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:02:07.903+0000 7fda0786b880 -1 Falling back to public interface 2024-09-21T15:02:11.661 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:02:11.655+0000 7fda0786b880 -1 osd.4 200 log_to_monitors true 2024-09-21T15:02:12.228 INFO:tasks.ceph.osd.4.smithi046.stderr:2024-09-21T15:02:12.223+0000 7fd9fb159700 -1 osd.4 200 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T15:02:12.400 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T15:02:12.523 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-21T15:02:12.523 INFO:teuthology.orchestra.run.smithi046.stdout: "ops": [], 2024-09-21T15:02:12.523 INFO:teuthology.orchestra.run.smithi046.stdout: "num_ops": 0 2024-09-21T15:02:12.523 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-21T15:02:12.533 INFO:tasks.thrashosds.thrasher:inning osd 2024-09-21T15:02:12.533 INFO:tasks.thrashosds.thrasher:Adding osd 4 2024-09-21T15:02:12.533 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 4 2024-09-21T15:02:12.859 INFO:tasks.ceph.mon.a.smithi032.stderr:2024-09-21T15:02:12.857+0000 7fdb7159c700 -1 mon.a@0(leader).osd e234 definitely_dead 0 2024-09-21T15:02:13.216 INFO:teuthology.orchestra.run.smithi032.stderr:marked in osd.4. 2024-09-21T15:02:13.228 INFO:tasks.thrashosds.thrasher:Added osd 4 2024-09-21T15:02:13.228 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-09-21T15:02:13.363 INFO:teuthology.orchestra.run.smithi032.stdout:mon.a: {} 2024-09-21T15:02:13.379 INFO:teuthology.orchestra.run.smithi032.stdout:mon.b: {} 2024-09-21T15:02:13.392 INFO:teuthology.orchestra.run.smithi032.stdout:mon.c: {} 2024-09-21T15:02:13.395 INFO:teuthology.orchestra.run.smithi032.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-21T15:02:13.395 INFO:teuthology.orchestra.run.smithi032.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-21T15:02:13.395 INFO:teuthology.orchestra.run.smithi032.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-21T15:02:13.407 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-21T15:02:14.231 INFO:teuthology.orchestra.run.smithi032.stderr:reweighted osd.0 to 1 (10000) 2024-09-21T15:02:14.243 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-09-21T15:02:15.237 INFO:teuthology.orchestra.run.smithi032.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-09-21T15:02:15.250 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-21T15:02:16.249 INFO:teuthology.orchestra.run.smithi032.stderr:reweighted osd.1 to 1 (10000) 2024-09-21T15:02:16.263 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-09-21T15:02:17.257 INFO:teuthology.orchestra.run.smithi032.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-09-21T15:02:17.264 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-21T15:02:18.254 INFO:teuthology.orchestra.run.smithi032.stderr:reweighted osd.2 to 1 (10000) 2024-09-21T15:02:18.269 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-09-21T15:02:19.253 INFO:teuthology.orchestra.run.smithi032.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-21T15:02:19.267 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-21T15:02:20.268 INFO:teuthology.orchestra.run.smithi032.stderr:reweighted osd.3 to 1 (10000) 2024-09-21T15:02:20.281 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-09-21T15:02:21.282 INFO:teuthology.orchestra.run.smithi032.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-21T15:02:21.298 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-21T15:02:22.294 INFO:teuthology.orchestra.run.smithi032.stderr:reweighted osd.5 to 1 (10000) 2024-09-21T15:02:22.309 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-09-21T15:02:23.302 INFO:teuthology.orchestra.run.smithi032.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-09-21T15:02:23.318 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-21T15:02:24.310 INFO:teuthology.orchestra.run.smithi032.stderr:reweighted osd.6 to 1 (10000) 2024-09-21T15:02:24.323 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 1 2024-09-21T15:02:25.315 INFO:teuthology.orchestra.run.smithi032.stderr:set osd.6 primary-affinity to 1 (8655362) 2024-09-21T15:02:25.327 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-21T15:02:26.329 INFO:teuthology.orchestra.run.smithi032.stderr:reweighted osd.7 to 1 (10000) 2024-09-21T15:02:26.341 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 7 1 2024-09-21T15:02:27.332 INFO:teuthology.orchestra.run.smithi032.stderr:set osd.7 primary-affinity to 1 (8655362) 2024-09-21T15:02:27.345 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-21T15:02:28.345 INFO:teuthology.orchestra.run.smithi032.stderr:reweighted osd.4 to 1 (10000) 2024-09-21T15:02:28.359 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-09-21T15:02:29.359 INFO:teuthology.orchestra.run.smithi032.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-09-21T15:02:29.374 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-09-21T15:02:29.374 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-09-21T15:02:29.374 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-09-21T15:02:29.374 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-09-21T15:02:29.374 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-21T15:02:29.374 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T15:02:29.689 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-21T15:02:29.690 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":251,"fsid":"caf51e55-9234-4a26-bcd2-d60811962560","created":"2024-09-21T14:57:50.338822+0000","modified":"2024-09-21T15:02:29.341630+0000","last_up_change":"2024-09-21T15:02:13.195875+0000","last_in_change":"2024-09-21T15:02:13.195875+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T14:58:01.842810+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-21T14:58:08.284925+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":22,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-21T14:58:17.510709+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"230","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":151,"snap_epoch":230,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a4a73316-3a94-42e9-9a51-deb46a66bb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":4016672243},{"type":"v1","addr":"172.21.15.32:6809","nonce":4016672243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":4016672243},{"type":"v1","addr":"172.21.15.32:6811","nonce":4016672243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":4016672243},{"type":"v1","addr":"172.21.15.32:6815","nonce":4016672243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":4016672243},{"type":"v1","addr":"172.21.15.32:6813","nonce":4016672243}]},"public_addr":"172.21.15.32:6809/4016672243","cluster_addr":"172.21.15.32:6811/4016672243","heartbeat_back_addr":"172.21.15.32:6815/4016672243","heartbeat_front_addr":"172.21.15.32:6813/4016672243","state":["exists","up"]},{"osd":1,"uuid":"006757a0-94d1-4a9b-aef8-73634cc5828e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":641090577},{"type":"v1","addr":"172.21.15.32:6825","nonce":641090577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":641090577},{"type":"v1","addr":"172.21.15.32:6827","nonce":641090577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":641090577},{"type":"v1","addr":"172.21.15.32:6831","nonce":641090577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":641090577},{"type":"v1","addr":"172.21.15.32:6829","nonce":641090577}]},"public_addr":"172.21.15.32:6825/641090577","cluster_addr":"172.21.15.32:6827/641090577","heartbeat_back_addr":"172.21.15.32:6831/641090577","heartbeat_front_addr":"172.21.15.32:6829/641090577","state":["exists","up"]},{"osd":2,"uuid":"0b499a23-820e-4e6c-a558-adbf88b7014f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2609154414},{"type":"v1","addr":"172.21.15.32:6801","nonce":2609154414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2609154414},{"type":"v1","addr":"172.21.15.32:6803","nonce":2609154414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2609154414},{"type":"v1","addr":"172.21.15.32:6807","nonce":2609154414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2609154414},{"type":"v1","addr":"172.21.15.32:6805","nonce":2609154414}]},"public_addr":"172.21.15.32:6801/2609154414","cluster_addr":"172.21.15.32:6803/2609154414","heartbeat_back_addr":"172.21.15.32:6807/2609154414","heartbeat_front_addr":"172.21.15.32:6805/2609154414","state":["exists","up"]},{"osd":3,"uuid":"5a57b3dd-4a07-45b7-b159-3cac4c7eb262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":1462563376},{"type":"v1","addr":"172.21.15.32:6817","nonce":1462563376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1462563376},{"type":"v1","addr":"172.21.15.32:6819","nonce":1462563376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1462563376},{"type":"v1","addr":"172.21.15.32:6823","nonce":1462563376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1462563376},{"type":"v1","addr":"172.21.15.32:6821","nonce":1462563376}]},"public_addr":"172.21.15.32:6817/1462563376","cluster_addr":"172.21.15.32:6819/1462563376","heartbeat_back_addr":"172.21.15.32:6823/1462563376","heartbeat_front_addr":"172.21.15.32:6821/1462563376","state":["exists","up"]},{"osd":4,"uuid":"39a00807-7517-474e-8842-623dd7093cc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":200,"up_from":235,"up_thru":240,"down_at":202,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6801","nonce":2900482440},{"type":"v1","addr":"172.21.15.46:6803","nonce":2900482440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6805","nonce":2900482440},{"type":"v1","addr":"172.21.15.46:6807","nonce":2900482440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2900482440},{"type":"v1","addr":"172.21.15.46:6814","nonce":2900482440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2900482440},{"type":"v1","addr":"172.21.15.46:6811","nonce":2900482440}]},"public_addr":"172.21.15.46:6803/2900482440","cluster_addr":"172.21.15.46:6807/2900482440","heartbeat_back_addr":"172.21.15.46:6814/2900482440","heartbeat_front_addr":"172.21.15.46:6811/2900482440","state":["exists","up"]},{"osd":5,"uuid":"edf9f045-ddeb-4504-a0fc-1a0e4e853550","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":245,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":849101442},{"type":"v1","addr":"172.21.15.46:6802","nonce":849101442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":849101442},{"type":"v1","addr":"172.21.15.46:6806","nonce":849101442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":849101442},{"type":"v1","addr":"172.21.15.46:6815","nonce":849101442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":849101442},{"type":"v1","addr":"172.21.15.46:6810","nonce":849101442}]},"public_addr":"172.21.15.46:6802/849101442","cluster_addr":"172.21.15.46:6806/849101442","heartbeat_back_addr":"172.21.15.46:6815/849101442","heartbeat_front_addr":"172.21.15.46:6810/849101442","state":["exists","up"]},{"osd":6,"uuid":"c820ea6b-88fe-4421-83ec-17c466691fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":847189172},{"type":"v1","addr":"172.21.15.46:6818","nonce":847189172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":847189172},{"type":"v1","addr":"172.21.15.46:6822","nonce":847189172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6827","nonce":847189172},{"type":"v1","addr":"172.21.15.46:6828","nonce":847189172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6823","nonce":847189172},{"type":"v1","addr":"172.21.15.46:6825","nonce":847189172}]},"public_addr":"172.21.15.46:6818/847189172","cluster_addr":"172.21.15.46:6822/847189172","heartbeat_back_addr":"172.21.15.46:6828/847189172","heartbeat_front_addr":"172.21.15.46:6825/847189172","state":["exists","up"]},{"osd":7,"uuid":"3a2f82a4-28a0-4fcb-9d6b-2285fa4d07ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3894963553},{"type":"v1","addr":"172.21.15.46:6819","nonce":3894963553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":3894963553},{"type":"v1","addr":"172.21.15.46:6824","nonce":3894963553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":3894963553},{"type":"v1","addr":"172.21.15.46:6831","nonce":3894963553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":3894963553},{"type":"v1","addr":"172.21.15.46:6829","nonce":3894963553}]},"public_addr":"172.21.15.46:6819/3894963553","cluster_addr":"172.21.15.46:6824/3894963553","heartbeat_back_addr":"172.21.15.46:6831/3894963553","heartbeat_front_addr":"172.21.15.46:6829/3894963553","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T14:57:57.458317+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T14:57:57.907624+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T14:57:57.278817+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T14:57:57.500449+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T14:57:57.504325+0000","dead_epoch":202},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T14:57:57.777525+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T14:57:57.698961+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T14:57:57.932625+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.2","osds":[2,3]},{"pgid":"3.4","osds":[4,2]},{"pgid":"3.8","osds":[6,3]},{"pgid":"3.a","osds":[1,0]}],"pg_upmap_items":[{"pgid":"3.1","mappings":[{"from":2,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T15:02:29.709 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-21T15:02:29.709 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T15:02:30.025 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-21T15:02:30.025 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":251,"fsid":"caf51e55-9234-4a26-bcd2-d60811962560","created":"2024-09-21T14:57:50.338822+0000","modified":"2024-09-21T15:02:29.341630+0000","last_up_change":"2024-09-21T15:02:13.195875+0000","last_in_change":"2024-09-21T15:02:13.195875+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T14:58:01.842810+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-21T14:58:08.284925+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":22,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-21T14:58:17.510709+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"230","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":151,"snap_epoch":230,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a4a73316-3a94-42e9-9a51-deb46a66bb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":4016672243},{"type":"v1","addr":"172.21.15.32:6809","nonce":4016672243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":4016672243},{"type":"v1","addr":"172.21.15.32:6811","nonce":4016672243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":4016672243},{"type":"v1","addr":"172.21.15.32:6815","nonce":4016672243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":4016672243},{"type":"v1","addr":"172.21.15.32:6813","nonce":4016672243}]},"public_addr":"172.21.15.32:6809/4016672243","cluster_addr":"172.21.15.32:6811/4016672243","heartbeat_back_addr":"172.21.15.32:6815/4016672243","heartbeat_front_addr":"172.21.15.32:6813/4016672243","state":["exists","up"]},{"osd":1,"uuid":"006757a0-94d1-4a9b-aef8-73634cc5828e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":641090577},{"type":"v1","addr":"172.21.15.32:6825","nonce":641090577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":641090577},{"type":"v1","addr":"172.21.15.32:6827","nonce":641090577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":641090577},{"type":"v1","addr":"172.21.15.32:6831","nonce":641090577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":641090577},{"type":"v1","addr":"172.21.15.32:6829","nonce":641090577}]},"public_addr":"172.21.15.32:6825/641090577","cluster_addr":"172.21.15.32:6827/641090577","heartbeat_back_addr":"172.21.15.32:6831/641090577","heartbeat_front_addr":"172.21.15.32:6829/641090577","state":["exists","up"]},{"osd":2,"uuid":"0b499a23-820e-4e6c-a558-adbf88b7014f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2609154414},{"type":"v1","addr":"172.21.15.32:6801","nonce":2609154414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2609154414},{"type":"v1","addr":"172.21.15.32:6803","nonce":2609154414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2609154414},{"type":"v1","addr":"172.21.15.32:6807","nonce":2609154414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2609154414},{"type":"v1","addr":"172.21.15.32:6805","nonce":2609154414}]},"public_addr":"172.21.15.32:6801/2609154414","cluster_addr":"172.21.15.32:6803/2609154414","heartbeat_back_addr":"172.21.15.32:6807/2609154414","heartbeat_front_addr":"172.21.15.32:6805/2609154414","state":["exists","up"]},{"osd":3,"uuid":"5a57b3dd-4a07-45b7-b159-3cac4c7eb262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":1462563376},{"type":"v1","addr":"172.21.15.32:6817","nonce":1462563376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1462563376},{"type":"v1","addr":"172.21.15.32:6819","nonce":1462563376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1462563376},{"type":"v1","addr":"172.21.15.32:6823","nonce":1462563376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1462563376},{"type":"v1","addr":"172.21.15.32:6821","nonce":1462563376}]},"public_addr":"172.21.15.32:6817/1462563376","cluster_addr":"172.21.15.32:6819/1462563376","heartbeat_back_addr":"172.21.15.32:6823/1462563376","heartbeat_front_addr":"172.21.15.32:6821/1462563376","state":["exists","up"]},{"osd":4,"uuid":"39a00807-7517-474e-8842-623dd7093cc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":200,"up_from":235,"up_thru":240,"down_at":202,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6801","nonce":2900482440},{"type":"v1","addr":"172.21.15.46:6803","nonce":2900482440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6805","nonce":2900482440},{"type":"v1","addr":"172.21.15.46:6807","nonce":2900482440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2900482440},{"type":"v1","addr":"172.21.15.46:6814","nonce":2900482440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2900482440},{"type":"v1","addr":"172.21.15.46:6811","nonce":2900482440}]},"public_addr":"172.21.15.46:6803/2900482440","cluster_addr":"172.21.15.46:6807/2900482440","heartbeat_back_addr":"172.21.15.46:6814/2900482440","heartbeat_front_addr":"172.21.15.46:6811/2900482440","state":["exists","up"]},{"osd":5,"uuid":"edf9f045-ddeb-4504-a0fc-1a0e4e853550","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":245,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":849101442},{"type":"v1","addr":"172.21.15.46:6802","nonce":849101442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":849101442},{"type":"v1","addr":"172.21.15.46:6806","nonce":849101442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":849101442},{"type":"v1","addr":"172.21.15.46:6815","nonce":849101442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":849101442},{"type":"v1","addr":"172.21.15.46:6810","nonce":849101442}]},"public_addr":"172.21.15.46:6802/849101442","cluster_addr":"172.21.15.46:6806/849101442","heartbeat_back_addr":"172.21.15.46:6815/849101442","heartbeat_front_addr":"172.21.15.46:6810/849101442","state":["exists","up"]},{"osd":6,"uuid":"c820ea6b-88fe-4421-83ec-17c466691fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":847189172},{"type":"v1","addr":"172.21.15.46:6818","nonce":847189172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":847189172},{"type":"v1","addr":"172.21.15.46:6822","nonce":847189172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6827","nonce":847189172},{"type":"v1","addr":"172.21.15.46:6828","nonce":847189172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6823","nonce":847189172},{"type":"v1","addr":"172.21.15.46:6825","nonce":847189172}]},"public_addr":"172.21.15.46:6818/847189172","cluster_addr":"172.21.15.46:6822/847189172","heartbeat_back_addr":"172.21.15.46:6828/847189172","heartbeat_front_addr":"172.21.15.46:6825/847189172","state":["exists","up"]},{"osd":7,"uuid":"3a2f82a4-28a0-4fcb-9d6b-2285fa4d07ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3894963553},{"type":"v1","addr":"172.21.15.46:6819","nonce":3894963553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":3894963553},{"type":"v1","addr":"172.21.15.46:6824","nonce":3894963553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":3894963553},{"type":"v1","addr":"172.21.15.46:6831","nonce":3894963553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":3894963553},{"type":"v1","addr":"172.21.15.46:6829","nonce":3894963553}]},"public_addr":"172.21.15.46:6819/3894963553","cluster_addr":"172.21.15.46:6824/3894963553","heartbeat_back_addr":"172.21.15.46:6831/3894963553","heartbeat_front_addr":"172.21.15.46:6829/3894963553","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T14:57:57.458317+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T14:57:57.907624+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T14:57:57.278817+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T14:57:57.500449+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T14:57:57.504325+0000","dead_epoch":202},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T14:57:57.777525+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T14:57:57.698961+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T14:57:57.932625+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.2","osds":[2,3]},{"pgid":"3.4","osds":[4,2]},{"pgid":"3.8","osds":[6,3]},{"pgid":"3.a","osds":[1,0]}],"pg_upmap_items":[{"pgid":"3.1","mappings":[{"from":2,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T15:02:30.046 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-21T15:02:30.046 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-21T15:02:30.047 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-21T15:02:30.047 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-21T15:02:30.048 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-21T15:02:30.048 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-21T15:02:30.048 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-21T15:02:30.049 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-21T15:02:30.254 INFO:teuthology.orchestra.run.smithi032.stdout:60129542200 2024-09-21T15:02:30.255 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-21T15:02:30.267 INFO:teuthology.orchestra.run.smithi032.stdout:60129542202 2024-09-21T15:02:30.267 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-21T15:02:30.270 INFO:teuthology.orchestra.run.smithi032.stdout:60129542201 2024-09-21T15:02:30.271 INFO:teuthology.orchestra.run.smithi032.stdout:64424509498 2024-09-21T15:02:30.271 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-21T15:02:30.272 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-21T15:02:30.279 INFO:teuthology.orchestra.run.smithi032.stdout:60129542203 2024-09-21T15:02:30.279 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-21T15:02:30.287 INFO:teuthology.orchestra.run.smithi032.stdout:60129542202 2024-09-21T15:02:30.288 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-21T15:02:30.289 INFO:teuthology.orchestra.run.smithi032.stdout:1009317314566 2024-09-21T15:02:30.290 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-21T15:02:30.294 INFO:teuthology.orchestra.run.smithi032.stdout:60129542200 2024-09-21T15:02:30.294 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-21T15:02:30.753 INFO:teuthology.orchestra.run.smithi032.stdout:60129542199 2024-09-21T15:02:30.769 INFO:teuthology.orchestra.run.smithi032.stdout:60129542201 2024-09-21T15:02:30.773 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542200 got 60129542199 for osd.2 2024-09-21T15:02:30.775 INFO:teuthology.orchestra.run.smithi032.stdout:60129542200 2024-09-21T15:02:30.780 INFO:teuthology.orchestra.run.smithi032.stdout:64424509497 2024-09-21T15:02:30.783 INFO:teuthology.orchestra.run.smithi032.stdout:60129542202 2024-09-21T15:02:30.796 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509498 got 64424509497 for osd.7 2024-09-21T15:02:30.797 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542201 got 60129542200 for osd.6 2024-09-21T15:02:30.798 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542202 got 60129542201 for osd.3 2024-09-21T15:02:30.798 INFO:teuthology.orchestra.run.smithi032.stdout:60129542201 2024-09-21T15:02:30.803 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542203 got 60129542202 for osd.1 2024-09-21T15:02:30.810 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542202 got 60129542201 for osd.5 2024-09-21T15:02:30.824 INFO:teuthology.orchestra.run.smithi032.stdout:1009317314565 2024-09-21T15:02:30.831 INFO:teuthology.orchestra.run.smithi032.stdout:60129542199 2024-09-21T15:02:30.838 INFO:tasks.ceph.ceph_manager.ceph:need seq 1009317314566 got 1009317314565 for osd.4 2024-09-21T15:02:30.844 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542200 got 60129542199 for osd.0 2024-09-21T15:02:31.774 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-21T15:02:31.797 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-21T15:02:31.797 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-21T15:02:31.798 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-21T15:02:31.804 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-21T15:02:31.810 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-21T15:02:31.839 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-21T15:02:31.844 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-21T15:02:32.268 INFO:teuthology.orchestra.run.smithi032.stdout:60129542199 2024-09-21T15:02:32.281 INFO:teuthology.orchestra.run.smithi032.stdout:64424509497 2024-09-21T15:02:32.289 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542200 got 60129542199 for osd.2 2024-09-21T15:02:32.292 INFO:teuthology.orchestra.run.smithi032.stdout:60129542200 2024-09-21T15:02:32.301 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509498 got 64424509497 for osd.7 2024-09-21T15:02:32.310 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542201 got 60129542200 for osd.6 2024-09-21T15:02:32.325 INFO:teuthology.orchestra.run.smithi032.stdout:60129542201 2024-09-21T15:02:32.333 INFO:teuthology.orchestra.run.smithi032.stdout:60129542202 2024-09-21T15:02:32.338 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542202 got 60129542201 for osd.3 2024-09-21T15:02:32.339 INFO:teuthology.orchestra.run.smithi032.stdout:1009317314565 2024-09-21T15:02:32.347 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542203 got 60129542202 for osd.1 2024-09-21T15:02:32.350 INFO:teuthology.orchestra.run.smithi032.stdout:60129542201 2024-09-21T15:02:32.352 INFO:tasks.ceph.ceph_manager.ceph:need seq 1009317314566 got 1009317314565 for osd.4 2024-09-21T15:02:32.365 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542202 got 60129542201 for osd.5 2024-09-21T15:02:32.368 INFO:teuthology.orchestra.run.smithi032.stdout:60129542201 2024-09-21T15:02:32.380 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542200 got 60129542201 for osd.0 2024-09-21T15:02:32.380 DEBUG:teuthology.parallel:result is None 2024-09-21T15:02:33.290 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-21T15:02:33.302 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-21T15:02:33.310 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-21T15:02:33.339 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-21T15:02:33.347 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-21T15:02:33.353 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-21T15:02:33.365 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-21T15:02:33.705 INFO:teuthology.orchestra.run.smithi032.stdout:60129542203 2024-09-21T15:02:33.718 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542202 got 60129542203 for osd.3 2024-09-21T15:02:33.718 DEBUG:teuthology.parallel:result is None 2024-09-21T15:02:33.751 INFO:teuthology.orchestra.run.smithi032.stdout:60129542201 2024-09-21T15:02:33.768 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542200 got 60129542201 for osd.2 2024-09-21T15:02:33.768 DEBUG:teuthology.parallel:result is None 2024-09-21T15:02:33.780 INFO:teuthology.orchestra.run.smithi032.stdout:60129542202 2024-09-21T15:02:33.790 INFO:teuthology.orchestra.run.smithi032.stdout:64424509498 2024-09-21T15:02:33.797 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542201 got 60129542202 for osd.6 2024-09-21T15:02:33.798 DEBUG:teuthology.parallel:result is None 2024-09-21T15:02:33.807 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509498 got 64424509498 for osd.7 2024-09-21T15:02:33.807 DEBUG:teuthology.parallel:result is None 2024-09-21T15:02:33.815 INFO:teuthology.orchestra.run.smithi032.stdout:60129542203 2024-09-21T15:02:33.818 INFO:teuthology.orchestra.run.smithi032.stdout:60129542204 2024-09-21T15:02:33.828 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542202 got 60129542203 for osd.5 2024-09-21T15:02:33.828 DEBUG:teuthology.parallel:result is None 2024-09-21T15:02:33.830 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542203 got 60129542204 for osd.1 2024-09-21T15:02:33.831 DEBUG:teuthology.parallel:result is None 2024-09-21T15:02:33.832 INFO:teuthology.orchestra.run.smithi032.stdout:1009317314566 2024-09-21T15:02:33.844 INFO:tasks.ceph.ceph_manager.ceph:need seq 1009317314566 got 1009317314566 for osd.4 2024-09-21T15:02:33.845 DEBUG:teuthology.parallel:result is None 2024-09-21T15:02:33.845 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-21T15:02:33.845 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T15:02:34.198 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-21T15:02:34.200 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-09-21T15:02:34.209 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":381,"stamp":"2024-09-21T15:02:33.847574+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":174155,"num_read_kb":1238336,"num_write":34466,"num_write_kb":943253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3956,"num_bytes_recovered":212783659,"num_keys_recovered":305,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22661,"ondisk_log_size":22661,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":460608,"kb_used_data":23872,"kb_used_omap":19,"kb_used_meta":436588,"kb_avail":754514112,"statfs":{"total":773094113280,"available":772622450688,"internally_reserved":0,"allocated":24444928,"data_stored":13166724,"data_compressed":10404,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":20129,"internal_metadata":447066463},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.524348"},"pg_stats":[{"pgid":"3.1f","version":"230'598","reported_seq":3979,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:29.452925+0000","last_change":"2024-09-21T15:02:24.311888+0000","last_active":"2024-09-21T15:02:29.452925+0000","last_peered":"2024-09-21T15:02:29.452925+0000","last_clean":"2024-09-21T15:02:29.452925+0000","last_became_active":"2024-09-21T15:02:24.308451+0000","last_became_peered":"2024-09-21T15:02:24.308451+0000","last_unstale":"2024-09-21T15:02:29.452925+0000","last_undegraded":"2024-09-21T15:02:29.452925+0000","last_fullsized":"2024-09-21T15:02:29.452925+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":598,"log_dups_size":0,"ondisk_log_size":598,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:25:13.708671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010410479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3756,"num_read_kb":43219,"num_write":975,"num_write_kb":36436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"230'507","reported_seq":3510,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.335752+0000","last_change":"2024-09-21T15:02:24.312475+0000","last_active":"2024-09-21T15:02:28.335752+0000","last_peered":"2024-09-21T15:02:28.335752+0000","last_clean":"2024-09-21T15:02:28.335752+0000","last_became_active":"2024-09-21T15:02:24.309506+0000","last_became_peered":"2024-09-21T15:02:24.309506+0000","last_unstale":"2024-09-21T15:02:28.335752+0000","last_undegraded":"2024-09-21T15:02:28.335752+0000","last_fullsized":"2024-09-21T15:02:28.335752+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":507,"log_dups_size":0,"ondisk_log_size":507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:49:24.984436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094912299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3420,"num_read_kb":23407,"num_write":643,"num_write_kb":18445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":148,"num_bytes_recovered":11635695,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"228'341","reported_seq":4790,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.957133+0000","last_change":"2024-09-21T15:02:02.409182+0000","last_active":"2024-09-21T15:02:28.957133+0000","last_peered":"2024-09-21T15:02:28.957133+0000","last_clean":"2024-09-21T15:02:28.957133+0000","last_became_active":"2024-09-21T14:58:19.411387+0000","last_became_peered":"2024-09-21T14:58:19.411387+0000","last_unstale":"2024-09-21T15:02:28.957133+0000","last_undegraded":"2024-09-21T15:02:28.957133+0000","last_fullsized":"2024-09-21T15:02:28.957133+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":341,"log_dups_size":0,"ondisk_log_size":341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:37:57.597069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045388299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3878,"num_read_kb":28288,"num_write":535,"num_write_kb":26606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"230'572","reported_seq":4306,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.956805+0000","last_change":"2024-09-21T15:02:02.261134+0000","last_active":"2024-09-21T15:02:28.956805+0000","last_peered":"2024-09-21T15:02:28.956805+0000","last_clean":"2024-09-21T15:02:28.956805+0000","last_became_active":"2024-09-21T14:58:19.411306+0000","last_became_peered":"2024-09-21T14:58:19.411306+0000","last_unstale":"2024-09-21T15:02:28.956805+0000","last_undegraded":"2024-09-21T15:02:28.956805+0000","last_fullsized":"2024-09-21T15:02:28.956805+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":572,"log_dups_size":0,"ondisk_log_size":572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:20:27.079795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011721469999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4050,"num_read_kb":24676,"num_write":863,"num_write_kb":25228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"227'536","reported_seq":4293,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.335194+0000","last_change":"2024-09-21T15:02:17.248373+0000","last_active":"2024-09-21T15:02:28.335194+0000","last_peered":"2024-09-21T15:02:28.335194+0000","last_clean":"2024-09-21T15:02:28.335194+0000","last_became_active":"2024-09-21T15:02:17.242801+0000","last_became_peered":"2024-09-21T15:02:17.242801+0000","last_unstale":"2024-09-21T15:02:28.335194+0000","last_undegraded":"2024-09-21T15:02:28.335194+0000","last_fullsized":"2024-09-21T15:02:28.335194+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":536,"log_dups_size":0,"ondisk_log_size":536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:04:28.325046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032550800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3672,"num_read_kb":32698,"num_write":706,"num_write_kb":22582,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":379,"num_bytes_recovered":13616631,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"227'487","reported_seq":4857,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.956810+0000","last_change":"2024-09-21T15:02:02.262402+0000","last_active":"2024-09-21T15:02:28.956810+0000","last_peered":"2024-09-21T15:02:28.956810+0000","last_clean":"2024-09-21T15:02:28.956810+0000","last_became_active":"2024-09-21T14:58:19.409415+0000","last_became_peered":"2024-09-21T14:58:19.409415+0000","last_unstale":"2024-09-21T15:02:28.956810+0000","last_undegraded":"2024-09-21T15:02:28.956810+0000","last_fullsized":"2024-09-21T15:02:28.956810+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:28:46.336301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019572090000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4389,"num_read_kb":28349,"num_write":704,"num_write_kb":23876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"228'824","reported_seq":8731,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.335092+0000","last_change":"2024-09-21T15:02:18.252228+0000","last_active":"2024-09-21T15:02:28.335092+0000","last_peered":"2024-09-21T15:02:28.335092+0000","last_clean":"2024-09-21T15:02:28.335092+0000","last_became_active":"2024-09-21T15:02:18.246987+0000","last_became_peered":"2024-09-21T15:02:18.246987+0000","last_unstale":"2024-09-21T15:02:28.335092+0000","last_undegraded":"2024-09-21T15:02:28.335092+0000","last_fullsized":"2024-09-21T15:02:28.335092+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":824,"log_dups_size":0,"ondisk_log_size":824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:42:11.072452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014776850000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9383,"num_read_kb":68058,"num_write":1251,"num_write_kb":29921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":394,"num_bytes_recovered":23509073,"num_keys_recovered":66,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"227'688","reported_seq":8793,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.335591+0000","last_change":"2024-09-21T15:02:02.359228+0000","last_active":"2024-09-21T15:02:28.335591+0000","last_peered":"2024-09-21T15:02:28.335591+0000","last_clean":"2024-09-21T15:02:28.335591+0000","last_became_active":"2024-09-21T14:58:19.409371+0000","last_became_peered":"2024-09-21T14:58:19.409371+0000","last_unstale":"2024-09-21T15:02:28.335591+0000","last_undegraded":"2024-09-21T15:02:28.335591+0000","last_fullsized":"2024-09-21T15:02:28.335591+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":688,"log_dups_size":0,"ondisk_log_size":688,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:35:27.143445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052048100000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9236,"num_read_kb":45266,"num_write":998,"num_write_kb":30616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"227'413","reported_seq":4005,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.335834+0000","last_change":"2024-09-21T15:02:02.358947+0000","last_active":"2024-09-21T15:02:28.335834+0000","last_peered":"2024-09-21T15:02:28.335834+0000","last_clean":"2024-09-21T15:02:28.335834+0000","last_became_active":"2024-09-21T14:58:19.410306+0000","last_became_peered":"2024-09-21T14:58:19.410306+0000","last_unstale":"2024-09-21T15:02:28.335834+0000","last_undegraded":"2024-09-21T15:02:28.335834+0000","last_fullsized":"2024-09-21T15:02:28.335834+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":413,"log_dups_size":0,"ondisk_log_size":413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:33:50.089165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065724399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3380,"num_read_kb":28207,"num_write":769,"num_write_kb":34633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"229'1053","reported_seq":6175,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.956985+0000","last_change":"2024-09-21T15:02:02.263447+0000","last_active":"2024-09-21T15:02:28.956985+0000","last_peered":"2024-09-21T15:02:28.956985+0000","last_clean":"2024-09-21T15:02:28.956985+0000","last_became_active":"2024-09-21T15:01:06.166756+0000","last_became_peered":"2024-09-21T15:01:06.166756+0000","last_unstale":"2024-09-21T15:02:28.956985+0000","last_undegraded":"2024-09-21T15:02:28.956985+0000","last_fullsized":"2024-09-21T15:02:28.956985+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":1053,"log_dups_size":0,"ondisk_log_size":1053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:19:04.988784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001157057,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8073,"num_read_kb":43999,"num_write":1621,"num_write_kb":28374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":271,"num_bytes_recovered":16821266,"num_keys_recovered":55,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"230'823","reported_seq":3979,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.339198+0000","last_change":"2024-09-21T15:02:02.276055+0000","last_active":"2024-09-21T15:02:28.339198+0000","last_peered":"2024-09-21T15:02:28.339198+0000","last_clean":"2024-09-21T15:02:28.339198+0000","last_became_active":"2024-09-21T14:58:19.410687+0000","last_became_peered":"2024-09-21T14:58:19.410687+0000","last_unstale":"2024-09-21T15:02:28.339198+0000","last_undegraded":"2024-09-21T15:02:28.339198+0000","last_fullsized":"2024-09-21T15:02:28.339198+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":823,"log_dups_size":0,"ondisk_log_size":823,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:21:42.226982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017043010000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4757,"num_read_kb":49770,"num_write":1348,"num_write_kb":33339,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"228'527","reported_seq":4193,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.955479+0000","last_change":"2024-09-21T15:02:02.408062+0000","last_active":"2024-09-21T15:02:28.955479+0000","last_peered":"2024-09-21T15:02:28.955479+0000","last_clean":"2024-09-21T15:02:28.955479+0000","last_became_active":"2024-09-21T14:58:19.407310+0000","last_became_peered":"2024-09-21T14:58:19.407310+0000","last_unstale":"2024-09-21T15:02:28.955479+0000","last_undegraded":"2024-09-21T15:02:28.955479+0000","last_fullsized":"2024-09-21T15:02:28.955479+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":527,"log_dups_size":0,"ondisk_log_size":527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:47:46.079115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016534340000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3756,"num_read_kb":43322,"num_write":713,"num_write_kb":33471,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":245,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:29.771991+0000","last_change":"2024-09-21T14:58:11.348177+0000","last_active":"2024-09-21T15:02:29.771991+0000","last_peered":"2024-09-21T15:02:29.771991+0000","last_clean":"2024-09-21T15:02:29.771991+0000","last_became_active":"2024-09-21T14:58:09.328417+0000","last_became_peered":"2024-09-21T14:58:09.328417+0000","last_unstale":"2024-09-21T15:02:29.771991+0000","last_undegraded":"2024-09-21T15:02:29.771991+0000","last_fullsized":"2024-09-21T15:02:29.771991+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:52:22.671185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018917490000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"230'610","reported_seq":6567,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.335492+0000","last_change":"2024-09-21T15:02:17.251834+0000","last_active":"2024-09-21T15:02:28.335492+0000","last_peered":"2024-09-21T15:02:28.335492+0000","last_clean":"2024-09-21T15:02:28.335492+0000","last_became_active":"2024-09-21T15:02:17.247583+0000","last_became_peered":"2024-09-21T15:02:17.247583+0000","last_unstale":"2024-09-21T15:02:28.335492+0000","last_undegraded":"2024-09-21T15:02:28.335492+0000","last_fullsized":"2024-09-21T15:02:28.335492+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":610,"log_dups_size":0,"ondisk_log_size":610,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:54:31.703168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013409660000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6297,"num_read_kb":53037,"num_write":697,"num_write_kb":27961,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":11818795,"num_keys_recovered":18,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":24,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.335350+0000","last_change":"2024-09-21T15:02:14.225503+0000","last_active":"2024-09-21T15:02:28.335350+0000","last_peered":"2024-09-21T15:02:28.335350+0000","last_clean":"2024-09-21T15:02:28.335350+0000","last_became_active":"2024-09-21T15:02:14.224646+0000","last_became_peered":"2024-09-21T15:02:14.224646+0000","last_unstale":"2024-09-21T15:02:28.335350+0000","last_undegraded":"2024-09-21T15:02:28.335350+0000","last_fullsized":"2024-09-21T15:02:28.335350+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:57:35.948252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"228'980","reported_seq":4262,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.335626+0000","last_change":"2024-09-21T15:02:19.255988+0000","last_active":"2024-09-21T15:02:28.335626+0000","last_peered":"2024-09-21T15:02:28.335626+0000","last_clean":"2024-09-21T15:02:28.335626+0000","last_became_active":"2024-09-21T15:02:19.251008+0000","last_became_peered":"2024-09-21T15:02:19.251008+0000","last_unstale":"2024-09-21T15:02:28.335626+0000","last_undegraded":"2024-09-21T15:02:28.335626+0000","last_fullsized":"2024-09-21T15:02:28.335626+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":980,"log_dups_size":0,"ondisk_log_size":980,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:13:15.145504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015726799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5449,"num_read_kb":31198,"num_write":1400,"num_write_kb":25281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":401,"num_bytes_recovered":12566413,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":277,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.957358+0000","last_change":"2024-09-21T15:02:14.227927+0000","last_active":"2024-09-21T15:02:28.957358+0000","last_peered":"2024-09-21T15:02:28.957358+0000","last_clean":"2024-09-21T15:02:28.957358+0000","last_became_active":"2024-09-21T15:02:14.227223+0000","last_became_peered":"2024-09-21T15:02:14.227223+0000","last_unstale":"2024-09-21T15:02:28.957358+0000","last_undegraded":"2024-09-21T15:02:28.957358+0000","last_fullsized":"2024-09-21T15:02:28.957358+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:43:37.059225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019338770000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"227'672","reported_seq":5560,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.335126+0000","last_change":"2024-09-21T15:02:18.251535+0000","last_active":"2024-09-21T15:02:28.335126+0000","last_peered":"2024-09-21T15:02:28.335126+0000","last_clean":"2024-09-21T15:02:28.335126+0000","last_became_active":"2024-09-21T15:02:18.246450+0000","last_became_peered":"2024-09-21T15:02:18.246450+0000","last_unstale":"2024-09-21T15:02:28.335126+0000","last_undegraded":"2024-09-21T15:02:28.335126+0000","last_fullsized":"2024-09-21T15:02:28.335126+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":672,"log_dups_size":0,"ondisk_log_size":672,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:04:40.306321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071861500000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5178,"num_read_kb":38459,"num_write":1130,"num_write_kb":41306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":331,"num_bytes_recovered":25796920,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":232,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.335618+0000","last_change":"2024-09-21T15:02:24.351816+0000","last_active":"2024-09-21T15:02:28.335618+0000","last_peered":"2024-09-21T15:02:28.335618+0000","last_clean":"2024-09-21T15:02:28.335618+0000","last_became_active":"2024-09-21T15:02:24.351245+0000","last_became_peered":"2024-09-21T15:02:24.351245+0000","last_unstale":"2024-09-21T15:02:28.335618+0000","last_undegraded":"2024-09-21T15:02:28.335618+0000","last_fullsized":"2024-09-21T15:02:28.335618+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:43:32.690041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"227'732","reported_seq":2860,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:29.354087+0000","last_change":"2024-09-21T15:02:02.262648+0000","last_active":"2024-09-21T15:02:29.354087+0000","last_peered":"2024-09-21T15:02:29.354087+0000","last_clean":"2024-09-21T15:02:29.354087+0000","last_became_active":"2024-09-21T14:58:19.408971+0000","last_became_peered":"2024-09-21T14:58:19.408971+0000","last_unstale":"2024-09-21T15:02:29.354087+0000","last_undegraded":"2024-09-21T15:02:29.354087+0000","last_fullsized":"2024-09-21T15:02:29.354087+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":732,"log_dups_size":0,"ondisk_log_size":732,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:29:52.642121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073528299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3602,"num_read_kb":20655,"num_write":1230,"num_write_kb":20679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"230'1009","reported_seq":3690,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:29.738404+0000","last_change":"2024-09-21T15:02:02.276284+0000","last_active":"2024-09-21T15:02:29.738404+0000","last_peered":"2024-09-21T15:02:29.738404+0000","last_clean":"2024-09-21T15:02:29.738404+0000","last_became_active":"2024-09-21T14:58:19.417453+0000","last_became_peered":"2024-09-21T14:58:19.417453+0000","last_unstale":"2024-09-21T15:02:29.738404+0000","last_undegraded":"2024-09-21T15:02:29.738404+0000","last_fullsized":"2024-09-21T15:02:29.738404+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":1009,"log_dups_size":0,"ondisk_log_size":1009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:19:27.087272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044463100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4845,"num_read_kb":25284,"num_write":1566,"num_write_kb":21735,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":247,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.956823+0000","last_change":"2024-09-21T14:58:11.347504+0000","last_active":"2024-09-21T15:02:28.956823+0000","last_peered":"2024-09-21T15:02:28.956823+0000","last_clean":"2024-09-21T15:02:28.956823+0000","last_became_active":"2024-09-21T14:58:09.775981+0000","last_became_peered":"2024-09-21T14:58:09.775981+0000","last_unstale":"2024-09-21T15:02:28.956823+0000","last_undegraded":"2024-09-21T15:02:28.956823+0000","last_fullsized":"2024-09-21T15:02:28.956823+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:47:16.286652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012568880000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"228'1092","reported_seq":4001,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.335364+0000","last_change":"2024-09-21T15:02:24.314768+0000","last_active":"2024-09-21T15:02:28.335364+0000","last_peered":"2024-09-21T15:02:28.335364+0000","last_clean":"2024-09-21T15:02:28.335364+0000","last_became_active":"2024-09-21T15:02:24.311531+0000","last_became_peered":"2024-09-21T15:02:24.311531+0000","last_unstale":"2024-09-21T15:02:28.335364+0000","last_undegraded":"2024-09-21T15:02:28.335364+0000","last_fullsized":"2024-09-21T15:02:28.335364+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":1092,"log_dups_size":0,"ondisk_log_size":1092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:22:48.528933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00183417,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6343,"num_read_kb":28797,"num_write":1677,"num_write_kb":26974,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":246,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.953692+0000","last_change":"2024-09-21T14:58:11.342210+0000","last_active":"2024-09-21T15:02:28.953692+0000","last_peered":"2024-09-21T15:02:28.953692+0000","last_clean":"2024-09-21T15:02:28.953692+0000","last_became_active":"2024-09-21T14:58:09.330624+0000","last_became_peered":"2024-09-21T14:58:09.330624+0000","last_unstale":"2024-09-21T15:02:28.953692+0000","last_undegraded":"2024-09-21T15:02:28.953692+0000","last_fullsized":"2024-09-21T15:02:28.953692+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:29:43.229538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001051151,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"227'339","reported_seq":2553,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:29.769648+0000","last_change":"2024-09-21T15:02:02.257475+0000","last_active":"2024-09-21T15:02:29.769648+0000","last_peered":"2024-09-21T15:02:29.769648+0000","last_clean":"2024-09-21T15:02:29.769648+0000","last_became_active":"2024-09-21T15:01:53.193166+0000","last_became_peered":"2024-09-21T15:01:53.193166+0000","last_unstale":"2024-09-21T15:02:29.769648+0000","last_undegraded":"2024-09-21T15:02:29.769648+0000","last_fullsized":"2024-09-21T15:02:29.769648+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":339,"log_dups_size":0,"ondisk_log_size":339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:37:52.546654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014983959999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2371,"num_read_kb":40193,"num_write":529,"num_write_kb":39390,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":366,"num_bytes_recovered":22723935,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":245,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.955327+0000","last_change":"2024-09-21T14:58:11.346766+0000","last_active":"2024-09-21T15:02:28.955327+0000","last_peered":"2024-09-21T15:02:28.955327+0000","last_clean":"2024-09-21T15:02:28.955327+0000","last_became_active":"2024-09-21T14:58:09.329748+0000","last_became_peered":"2024-09-21T14:58:09.329748+0000","last_unstale":"2024-09-21T15:02:28.955327+0000","last_undegraded":"2024-09-21T15:02:28.955327+0000","last_fullsized":"2024-09-21T15:02:28.955327+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:31:20.267675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052698000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"230'1275","reported_seq":8379,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.955421+0000","last_change":"2024-09-21T15:02:02.261569+0000","last_active":"2024-09-21T15:02:28.955421+0000","last_peered":"2024-09-21T15:02:28.955421+0000","last_clean":"2024-09-21T15:02:28.955421+0000","last_became_active":"2024-09-21T14:58:41.401003+0000","last_became_peered":"2024-09-21T14:58:41.401003+0000","last_unstale":"2024-09-21T15:02:28.955421+0000","last_undegraded":"2024-09-21T15:02:28.955421+0000","last_fullsized":"2024-09-21T15:02:28.955421+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":1275,"log_dups_size":0,"ondisk_log_size":1275,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:05:37.644835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000746033,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11708,"num_read_kb":50816,"num_write":1955,"num_write_kb":24913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1403348,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":246,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.955718+0000","last_change":"2024-09-21T14:58:11.347700+0000","last_active":"2024-09-21T15:02:28.955718+0000","last_peered":"2024-09-21T15:02:28.955718+0000","last_clean":"2024-09-21T15:02:28.955718+0000","last_became_active":"2024-09-21T14:58:09.438842+0000","last_became_peered":"2024-09-21T14:58:09.438842+0000","last_unstale":"2024-09-21T15:02:28.955718+0000","last_undegraded":"2024-09-21T15:02:28.955718+0000","last_fullsized":"2024-09-21T15:02:28.955718+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:41:34.626249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001426194,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":394,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.957058+0000","last_change":"2024-09-21T14:58:03.616055+0000","last_active":"2024-09-21T15:02:28.957058+0000","last_peered":"2024-09-21T15:02:28.957058+0000","last_clean":"2024-09-21T15:02:28.957058+0000","last_became_active":"2024-09-21T14:58:03.613214+0000","last_became_peered":"2024-09-21T14:58:03.613214+0000","last_unstale":"2024-09-21T15:02:28.957058+0000","last_undegraded":"2024-09-21T15:02:28.957058+0000","last_fullsized":"2024-09-21T15:02:28.957058+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:02.180162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:02.180162+0000","last_clean_scrub_stamp":"2024-09-21T14:58:02.180162+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:29:05.860361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"228'685","reported_seq":4697,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.953055+0000","last_change":"2024-09-21T15:02:02.257763+0000","last_active":"2024-09-21T15:02:28.953055+0000","last_peered":"2024-09-21T15:02:28.953055+0000","last_clean":"2024-09-21T15:02:28.953055+0000","last_became_active":"2024-09-21T15:01:32.177913+0000","last_became_peered":"2024-09-21T15:01:32.177913+0000","last_unstale":"2024-09-21T15:02:28.953055+0000","last_undegraded":"2024-09-21T15:02:28.953055+0000","last_fullsized":"2024-09-21T15:02:28.953055+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":685,"log_dups_size":0,"ondisk_log_size":685,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:47:09.376828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015618730000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4407,"num_read_kb":44538,"num_write":1164,"num_write_kb":30502,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":413,"num_bytes_recovered":20409497,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.9","version":"229'973","reported_seq":4119,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.955874+0000","last_change":"2024-09-21T15:02:18.257627+0000","last_active":"2024-09-21T15:02:28.955874+0000","last_peered":"2024-09-21T15:02:28.955874+0000","last_clean":"2024-09-21T15:02:28.955874+0000","last_became_active":"2024-09-21T15:02:18.255482+0000","last_became_peered":"2024-09-21T15:02:18.255482+0000","last_unstale":"2024-09-21T15:02:28.955874+0000","last_undegraded":"2024-09-21T15:02:28.955874+0000","last_fullsized":"2024-09-21T15:02:28.955874+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":973,"log_dups_size":0,"ondisk_log_size":973,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:15:56.076204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001545105,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4815,"num_read_kb":35700,"num_write":1595,"num_write_kb":33802,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":375,"num_bytes_recovered":14742839,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"229'1027","reported_seq":5029,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.956693+0000","last_change":"2024-09-21T15:02:02.261576+0000","last_active":"2024-09-21T15:02:28.956693+0000","last_peered":"2024-09-21T15:02:28.956693+0000","last_clean":"2024-09-21T15:02:28.956693+0000","last_became_active":"2024-09-21T15:00:27.280971+0000","last_became_peered":"2024-09-21T15:00:27.280971+0000","last_unstale":"2024-09-21T15:02:28.956693+0000","last_undegraded":"2024-09-21T15:02:28.956693+0000","last_fullsized":"2024-09-21T15:02:28.956693+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":1027,"log_dups_size":0,"ondisk_log_size":1027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:26:07.617059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013613779999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7341,"num_read_kb":31141,"num_write":1421,"num_write_kb":33160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":9228123,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"227'593","reported_seq":3447,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.953752+0000","last_change":"2024-09-21T15:02:02.265620+0000","last_active":"2024-09-21T15:02:28.953752+0000","last_peered":"2024-09-21T15:02:28.953752+0000","last_clean":"2024-09-21T15:02:28.953752+0000","last_became_active":"2024-09-21T14:58:19.410679+0000","last_became_peered":"2024-09-21T14:58:19.410679+0000","last_unstale":"2024-09-21T15:02:28.953752+0000","last_undegraded":"2024-09-21T15:02:28.953752+0000","last_fullsized":"2024-09-21T15:02:28.953752+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":593,"log_dups_size":0,"ondisk_log_size":593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:07:40.758932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001466743,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3111,"num_read_kb":36762,"num_write":889,"num_write_kb":39649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"228'493","reported_seq":4709,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.338915+0000","last_change":"2024-09-21T15:02:02.275784+0000","last_active":"2024-09-21T15:02:28.338915+0000","last_peered":"2024-09-21T15:02:28.338915+0000","last_clean":"2024-09-21T15:02:28.338915+0000","last_became_active":"2024-09-21T14:59:25.130057+0000","last_became_peered":"2024-09-21T14:59:25.130057+0000","last_unstale":"2024-09-21T15:02:28.338915+0000","last_undegraded":"2024-09-21T15:02:28.338915+0000","last_fullsized":"2024-09-21T15:02:28.338915+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:53:33.752550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014500649999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4513,"num_read_kb":28786,"num_write":729,"num_write_kb":24616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":9059600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"227'544","reported_seq":5354,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.957258+0000","last_change":"2024-09-21T15:02:02.409235+0000","last_active":"2024-09-21T15:02:28.957258+0000","last_peered":"2024-09-21T15:02:28.957258+0000","last_clean":"2024-09-21T15:02:28.957258+0000","last_became_active":"2024-09-21T14:58:19.407884+0000","last_became_peered":"2024-09-21T14:58:19.407884+0000","last_unstale":"2024-09-21T15:02:28.957258+0000","last_undegraded":"2024-09-21T15:02:28.957258+0000","last_fullsized":"2024-09-21T15:02:28.957258+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":544,"log_dups_size":0,"ondisk_log_size":544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:28:57.111615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045666499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4968,"num_read_kb":55828,"num_write":945,"num_write_kb":38650,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"230'879","reported_seq":3218,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.956772+0000","last_change":"2024-09-21T15:02:02.263156+0000","last_active":"2024-09-21T15:02:28.956772+0000","last_peered":"2024-09-21T15:02:28.956772+0000","last_clean":"2024-09-21T15:02:28.956772+0000","last_became_active":"2024-09-21T15:01:05.172427+0000","last_became_peered":"2024-09-21T15:01:05.172427+0000","last_unstale":"2024-09-21T15:02:28.956772+0000","last_undegraded":"2024-09-21T15:02:28.956772+0000","last_fullsized":"2024-09-21T15:02:28.956772+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":879,"log_dups_size":0,"ondisk_log_size":879,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:28:18.710621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001142789,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4262,"num_read_kb":17288,"num_write":1508,"num_write_kb":17970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":138,"num_bytes_recovered":7486496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"227'532","reported_seq":6209,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.339180+0000","last_change":"2024-09-21T15:02:02.275716+0000","last_active":"2024-09-21T15:02:28.339180+0000","last_peered":"2024-09-21T15:02:28.339180+0000","last_clean":"2024-09-21T15:02:28.339180+0000","last_became_active":"2024-09-21T14:58:19.410406+0000","last_became_peered":"2024-09-21T14:58:19.410406+0000","last_unstale":"2024-09-21T15:02:28.339180+0000","last_undegraded":"2024-09-21T15:02:28.339180+0000","last_fullsized":"2024-09-21T15:02:28.339180+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:59:16.026939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014059350000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5807,"num_read_kb":56273,"num_write":817,"num_write_kb":36773,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"228'523","reported_seq":4647,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.956536+0000","last_change":"2024-09-21T15:02:02.260617+0000","last_active":"2024-09-21T15:02:28.956536+0000","last_peered":"2024-09-21T15:02:28.956536+0000","last_clean":"2024-09-21T15:02:28.956536+0000","last_became_active":"2024-09-21T14:58:19.409935+0000","last_became_peered":"2024-09-21T14:58:19.409935+0000","last_unstale":"2024-09-21T15:02:28.956536+0000","last_undegraded":"2024-09-21T15:02:28.956536+0000","last_fullsized":"2024-09-21T15:02:28.956536+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":523,"log_dups_size":0,"ondisk_log_size":523,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:10:36.252629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068183199999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3949,"num_read_kb":58693,"num_write":810,"num_write_kb":37279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"230'623","reported_seq":5195,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.955624+0000","last_change":"2024-09-21T15:02:02.408072+0000","last_active":"2024-09-21T15:02:28.955624+0000","last_peered":"2024-09-21T15:02:28.955624+0000","last_clean":"2024-09-21T15:02:28.955624+0000","last_became_active":"2024-09-21T15:01:05.165955+0000","last_became_peered":"2024-09-21T15:01:05.165955+0000","last_unstale":"2024-09-21T15:02:28.955624+0000","last_undegraded":"2024-09-21T15:02:28.955624+0000","last_fullsized":"2024-09-21T15:02:28.955624+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":623,"log_dups_size":0,"ondisk_log_size":623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:34:54.111119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021463580000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5758,"num_read_kb":28485,"num_write":777,"num_write_kb":24185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":166,"num_bytes_recovered":11965028,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"229'644","reported_seq":8535,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.953028+0000","last_change":"2024-09-21T15:02:02.256934+0000","last_active":"2024-09-21T15:02:28.953028+0000","last_peered":"2024-09-21T15:02:28.953028+0000","last_clean":"2024-09-21T15:02:28.953028+0000","last_became_active":"2024-09-21T15:00:07.155144+0000","last_became_peered":"2024-09-21T15:00:07.155144+0000","last_unstale":"2024-09-21T15:02:28.953028+0000","last_undegraded":"2024-09-21T15:02:28.953028+0000","last_fullsized":"2024-09-21T15:02:28.953028+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":644,"log_dups_size":0,"ondisk_log_size":644,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:58:00.133639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010987029999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9100,"num_read_kb":37252,"num_write":1013,"num_write_kb":20896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"227'947","reported_seq":6822,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.955476+0000","last_change":"2024-09-21T15:02:02.406594+0000","last_active":"2024-09-21T15:02:28.955476+0000","last_peered":"2024-09-21T15:02:28.955476+0000","last_clean":"2024-09-21T15:02:28.955476+0000","last_became_active":"2024-09-21T14:58:19.412030+0000","last_became_peered":"2024-09-21T14:58:19.412030+0000","last_unstale":"2024-09-21T15:02:28.955476+0000","last_undegraded":"2024-09-21T15:02:28.955476+0000","last_fullsized":"2024-09-21T15:02:28.955476+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":947,"log_dups_size":0,"ondisk_log_size":947,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:44:29.282391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067099300000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8465,"num_read_kb":59792,"num_write":1327,"num_write_kb":31587,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":174039,"num_read_kb":1238236,"num_write":34305,"num_write_kb":940835,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3956,"num_bytes_recovered":212783659,"num_keys_recovered":305,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7385,"internal_metadata":0},"log_size":22541,"ondisk_log_size":22541,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":10404,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509499,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34088,"kb_used_data":2968,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94337752,"statfs":{"total":96636764160,"available":96601858048,"internally_reserved":0,"allocated":3039232,"data_stored":1498239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:02:06 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.83899999999999997,"15min":0.83699999999999997},"min":{"1min":0.29199999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.044,"5min":1.351,"15min":1.351},"last":1.135},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.91400000000000003,"15min":0.94599999999999995},"min":{"1min":0.48099999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.0589999999999999,"5min":1.917,"15min":1.917},"last":1.0009999999999999}]},{"osd":1,"last update":"Sat Sep 21 15:02:06 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.80800000000000005,"15min":0.77600000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":1.0409999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.88},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.81599999999999995,"15min":0.82099999999999995},"min":{"1min":0.40500000000000003,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.002,"5min":2.2149999999999999,"15min":2.2149999999999999},"last":0.47599999999999998}]},{"osd":2,"last update":"Sat Sep 21 15:02:06 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.82899999999999996,"15min":0.81499999999999995},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.161,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.92600000000000005,"15min":0.93999999999999995},"min":{"1min":0.56499999999999995,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.0920000000000001,"5min":1.466,"15min":1.466},"last":0.96799999999999997}]},{"osd":3,"last update":"Sat Sep 21 15:02:06 2024","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.89100000000000001,"15min":0.86299999999999999},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.24,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":1.0669999999999999},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.96799999999999997,"15min":0.96599999999999997},"min":{"1min":0.54200000000000004,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.159,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":1.1919999999999999}]},{"osd":4,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.92200000000000004,"15min":0.93300000000000005},"min":{"1min":0.42199999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.3089999999999999,"5min":2.7130000000000001,"15min":2.7130000000000001},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.94499999999999995,"15min":0.92600000000000005},"min":{"1min":0.39000000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.425,"5min":2.125,"15min":2.125},"last":0.66000000000000003}]},{"osd":5,"last update":"Sat Sep 21 15:02:06 2024","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":1.006,"15min":1.016},"min":{"1min":0.505,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.375,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":1.0389999999999999},{"interface":"front","average":{"1min":0.97699999999999998,"5min":1.026,"15min":1.0449999999999999},"min":{"1min":0.61599999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.425,"5min":3.1880000000000002,"15min":3.1880000000000002},"last":1.1599999999999999}]},{"osd":6,"last update":"Sat Sep 21 15:02:06 2024","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.115,"15min":1.143},"min":{"1min":0.52900000000000003,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.679,"5min":3.2730000000000001,"15min":3.2730000000000001},"last":1.0840000000000001},{"interface":"front","average":{"1min":1.099,"5min":1.0800000000000001,"15min":1.036},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.6719999999999999,"5min":2.2879999999999998,"15min":2.2879999999999998},"last":1.113}]}]},{"osd":6,"up_from":14,"seq":60129542202,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64432,"kb_used_data":2972,"kb_used_omap":1,"kb_used_meta":61438,"kb_avail":94307408,"statfs":{"total":96636764160,"available":96570785792,"internally_reserved":0,"allocated":3043328,"data_stored":1498258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":62912967},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:02:09 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.87,"15min":0.93200000000000005},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.089,"5min":1.593,"15min":1.593},"last":1.222},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.90500000000000003,"15min":0.94099999999999995},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.1180000000000001,"5min":1.587,"15min":1.587},"last":0.70599999999999996}]},{"osd":1,"last update":"Sat Sep 21 15:02:09 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.876,"15min":0.90400000000000003},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.145,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":1.0940000000000001},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.85099999999999998,"15min":0.94499999999999995},"min":{"1min":0.40600000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.036,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":1.121}]},{"osd":2,"last update":"Sat Sep 21 15:02:09 2024","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.93600000000000005,"15min":0.97499999999999998},"min":{"1min":0.53400000000000003,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.194,"5min":1.758,"15min":1.758},"last":1.266},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.878,"15min":0.90200000000000002},"min":{"1min":0.51700000000000002,"5min":0.496,"15min":0.496},"max":{"1min":1.002,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.73599999999999999}]},{"osd":3,"last update":"Sat Sep 21 15:02:09 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.91600000000000004,"15min":0.94799999999999995},"min":{"1min":0.57399999999999995,"5min":0.434,"15min":0.434},"max":{"1min":0.98699999999999999,"5min":1.835,"15min":1.835},"last":1.137},{"interface":"front","average":{"1min":0.90100000000000002,"5min":1.0429999999999999,"15min":1.111},"min":{"1min":0.65900000000000003,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.296,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.92700000000000005}]},{"osd":4,"last update":"Sat Sep 21 15:01:08 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.93500000000000005,"15min":0.99099999999999999},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":1.2,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.98499999999999999,"15min":1.0209999999999999},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.3759999999999999,"5min":1.897,"15min":1.897},"last":0.96999999999999997}]},{"osd":5,"last update":"Sat Sep 21 15:02:09 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.97799999999999998,"15min":1.0880000000000001},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.222,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.85699999999999998,"5min":1.018,"15min":1.071},"min":{"1min":0.54200000000000004,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.3600000000000001,"5min":1.609,"15min":1.609},"last":1.0209999999999999}]},{"osd":7,"last update":"Sat Sep 21 15:02:09 2024","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":1.131,"15min":1.2190000000000001},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.256,"5min":1.8660000000000001,"15min":1.8660000000000001},"last":1.1619999999999999},{"interface":"front","average":{"1min":0.91300000000000003,"5min":1.117,"15min":1.248},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.413,"5min":2.004,"15min":2.004},"last":1.071}]}]},{"osd":5,"up_from":14,"seq":60129542203,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60384,"kb_used_data":3028,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94311456,"statfs":{"total":96636764160,"available":96574930944,"internally_reserved":0,"allocated":3100672,"data_stored":2088626,"data_compressed":5202,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:02:09 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.84199999999999997,"15min":0.96899999999999997},"min":{"1min":0.40100000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.98099999999999998,"5min":4.4800000000000004,"15min":4.4800000000000004},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.78400000000000003,"15min":0.81299999999999994},"min":{"1min":0.44500000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.179,"5min":1.181,"15min":1.181},"last":1.173}]},{"osd":1,"last update":"Sat Sep 21 15:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.86399999999999999,"15min":0.91300000000000003},"min":{"1min":0.41899999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.036,"5min":2.7709999999999999,"15min":2.7709999999999999},"last":1.0049999999999999},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.89900000000000002,"15min":0.92900000000000005},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.1439999999999999,"5min":2.823,"15min":2.823},"last":1.3420000000000001}]},{"osd":2,"last update":"Sat Sep 21 15:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.86599999999999999,"15min":0.95799999999999996},"min":{"1min":0.53600000000000003,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.2090000000000001,"5min":3.419,"15min":3.419},"last":0.754},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.84499999999999997,"15min":0.88800000000000001},"min":{"1min":0.58999999999999997,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.1699999999999999,"5min":1.423,"15min":1.423},"last":0.91600000000000004}]},{"osd":3,"last update":"Sat Sep 21 15:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.81100000000000005,"15min":0.81000000000000005},"min":{"1min":0.51000000000000001,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.105,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.97299999999999998},{"interface":"front","average":{"1min":0.91700000000000004,"5min":1.018,"15min":1.0720000000000001},"min":{"1min":0.66600000000000004,"5min":0.435,"15min":0.435},"max":{"1min":1.4239999999999999,"5min":2.6379999999999999,"15min":2.6379999999999999},"last":1.038}]},{"osd":4,"last update":"Sat Sep 21 15:01:02 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.98899999999999999,"15min":1.016},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.5609999999999999,"5min":2.5470000000000002,"15min":2.5470000000000002},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.82099999999999995,"5min":1.0169999999999999,"15min":1.0780000000000001},"min":{"1min":0.39300000000000002,"5min":0.376,"15min":0.376},"max":{"1min":1.302,"5min":2.5099999999999998,"15min":2.5099999999999998},"last":1.145}]},{"osd":6,"last update":"Sat Sep 21 15:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.91700000000000004,"15min":0.95999999999999996},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.1499999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.879,"5min":1.1399999999999999,"15min":1.3169999999999999},"min":{"1min":0.60199999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.498,"5min":4.2469999999999999,"15min":4.2469999999999999},"last":0.88200000000000001}]},{"osd":7,"last update":"Sat Sep 21 15:02:09 2024","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":1.167,"15min":1.3120000000000001},"min":{"1min":0.61199999999999999,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.5700000000000001,"5min":3.3010000000000002,"15min":3.3010000000000002},"last":1.117},{"interface":"front","average":{"1min":0.99399999999999999,"5min":1.075,"15min":1.125},"min":{"1min":0.64300000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.25,"5min":2.5880000000000001,"15min":2.5880000000000001},"last":1.4430000000000001}]}]},{"osd":4,"up_from":235,"seq":1009317314567,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34860,"kb_used_data":2968,"kb_used_omap":8,"kb_used_meta":31863,"kb_avail":94336980,"statfs":{"total":96636764160,"available":96601067520,"internally_reserved":0,"allocated":3039232,"data_stored":1498239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8979,"internal_metadata":32627949},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0269999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.091}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86899999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92000000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2390000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542203,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78376,"kb_used_data":2972,"kb_used_omap":1,"kb_used_meta":75390,"kb_avail":94293464,"statfs":{"total":96636764160,"available":96556507136,"internally_reserved":0,"allocated":3043328,"data_stored":1498258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":77199816},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.61799999999999999,"15min":0.63200000000000001},"min":{"1min":0.34599999999999997,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.92700000000000005,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.68200000000000005,"15min":0.70999999999999996},"min":{"1min":0.40500000000000003,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.3029999999999999,"5min":1.919,"15min":1.919},"last":0.29999999999999999}]},{"osd":1,"last update":"Sat Sep 21 15:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.65800000000000003,"15min":0.67000000000000004},"min":{"1min":0.44400000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":1.115,"5min":1.722,"15min":1.722},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.746,"15min":0.79700000000000004},"min":{"1min":0.41199999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.2509999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.28499999999999998}]},{"osd":2,"last update":"Sat Sep 21 15:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.746,"15min":0.72999999999999998},"min":{"1min":0.38200000000000001,"5min":0.311,"15min":0.311},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.746,"5min":0.79200000000000004,"15min":0.82099999999999995},"min":{"1min":0.40300000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.94399999999999995,"5min":1.254,"15min":1.254},"last":0.49199999999999999}]},{"osd":4,"last update":"Sat Sep 21 15:01:09 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.93799999999999994,"15min":0.95599999999999996},"min":{"1min":0.54300000000000004,"5min":0.442,"15min":0.442},"max":{"1min":1.2629999999999999,"5min":1.7609999999999999,"15min":1.7609999999999999},"last":1.0600000000000001},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.98899999999999999,"15min":1.0269999999999999},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.234,"5min":2.028,"15min":2.028},"last":0.95399999999999996}]},{"osd":5,"last update":"Sat Sep 21 15:02:13 2024","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.036,"15min":1.052},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.4670000000000001,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":0.76500000000000001},{"interface":"front","average":{"1min":1.081,"5min":1.0589999999999999,"15min":1.0640000000000001},"min":{"1min":0.68799999999999994,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.395,"5min":1.548,"15min":1.548},"last":0.86199999999999999}]},{"osd":6,"last update":"Sat Sep 21 15:02:13 2024","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":1.046,"15min":1.085},"min":{"1min":0.70099999999999996,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.391,"5min":2.1219999999999999,"15min":2.1219999999999999},"last":0.78100000000000003},{"interface":"front","average":{"1min":1.03,"5min":1.075,"15min":1.0980000000000001},"min":{"1min":0.64900000000000002,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.448,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":0.747}]},{"osd":7,"last update":"Sat Sep 21 15:02:13 2024","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.0800000000000001,"15min":1.123},"min":{"1min":0.746,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.339,"5min":1.706,"15min":1.706},"last":0.84899999999999998},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.1279999999999999,"15min":1.171},"min":{"1min":0.67700000000000005,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.421,"5min":2.3380000000000001,"15min":2.3380000000000001},"last":0.89900000000000002}]}]},{"osd":2,"up_from":14,"seq":60129542201,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70180,"kb_used_data":2968,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94301660,"statfs":{"total":96636764160,"available":96564899840,"internally_reserved":0,"allocated":3039232,"data_stored":1498239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68811206},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:02:07 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.63500000000000001,"15min":0.625},"min":{"1min":0.32100000000000001,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.383,"5min":1.383,"15min":1.383},"last":0.439},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.69099999999999995,"15min":0.66900000000000004},"min":{"1min":0.30099999999999999,"5min":0.223,"15min":0.223},"max":{"1min":1.1739999999999999,"5min":2.9169999999999998,"15min":2.9169999999999998},"last":0.28399999999999997}]},{"osd":1,"last update":"Sat Sep 21 15:02:07 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.67500000000000004,"15min":0.68100000000000005},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.129,"5min":2.6520000000000001,"15min":2.6520000000000001},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.73399999999999999,"15min":0.75800000000000001},"min":{"1min":0.36199999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.48899999999999999}]},{"osd":3,"last update":"Sat Sep 21 15:02:07 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.71699999999999997,"15min":0.72699999999999998},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.996,"5min":3.0110000000000001,"15min":3.0110000000000001},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.85999999999999999,"15min":0.83399999999999996},"min":{"1min":0.41099999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0369999999999999,"5min":4.1440000000000001,"15min":4.1440000000000001},"last":0.50600000000000001}]},{"osd":4,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.93200000000000005,"15min":0.95599999999999996},"min":{"1min":0.42399999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.222,"5min":2.899,"15min":2.899},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.94399999999999995,"15min":0.95799999999999996},"min":{"1min":0.47499999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.4379999999999999,"5min":1.675,"15min":1.675},"last":0.58899999999999997}]},{"osd":5,"last update":"Sat Sep 21 15:02:07 2024","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.98699999999999999,"15min":1.0009999999999999},"min":{"1min":0.45500000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.224,"5min":3.1080000000000001,"15min":3.1080000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.96399999999999997,"15min":0.98799999999999999},"min":{"1min":0.55200000000000005,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.228,"5min":1.546,"15min":1.546},"last":0.91300000000000003}]},{"osd":6,"last update":"Sat Sep 21 15:02:07 2024","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":0.98899999999999999,"15min":0.98699999999999999},"min":{"1min":0.52100000000000002,"5min":0.378,"15min":0.378},"max":{"1min":1.516,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.93999999999999995,"5min":1.0429999999999999,"15min":1.048},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.403,"5min":3.0739999999999998,"15min":3.0739999999999998},"last":0.625}]},{"osd":7,"last update":"Sat Sep 21 15:02:09 2024","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.0489999999999999,"15min":1.01},"min":{"1min":0.61399999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.415,"5min":3.1589999999999998,"15min":3.1589999999999998},"last":0.84499999999999997},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.111,"15min":1.095},"min":{"1min":0.58799999999999997,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.4610000000000001,"5min":3.1400000000000001,"15min":3.1400000000000001},"last":0.69799999999999995}]}]},{"osd":1,"up_from":14,"seq":60129542204,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48104,"kb_used_data":3028,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323736,"statfs":{"total":96636764160,"available":96587505664,"internally_reserved":0,"allocated":3100672,"data_stored":2088626,"data_compressed":5202,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.58899999999999997,"15min":0.60699999999999998},"min":{"1min":0.25600000000000001,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.60299999999999998,"15min":0.60999999999999999},"min":{"1min":0.26300000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.83499999999999996,"5min":1.107,"15min":1.107},"last":0.47199999999999998}]},{"osd":2,"last update":"Sat Sep 21 15:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.78000000000000003,"15min":0.73499999999999999},"min":{"1min":0.38800000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.92100000000000004,"5min":11.303000000000001,"15min":11.303000000000001},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.71399999999999997,"15min":0.70299999999999996},"min":{"1min":0.439,"5min":0.435,"15min":0.435},"max":{"1min":1.0840000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.373}]},{"osd":3,"last update":"Sat Sep 21 15:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.80200000000000005,"15min":0.83399999999999996},"min":{"1min":0.36899999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.4850000000000001,"5min":3.1469999999999998,"15min":3.1469999999999998},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.83699999999999997,"15min":0.877},"min":{"1min":0.42499999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.284,"5min":3.2320000000000002,"15min":3.2320000000000002},"last":0.41199999999999998}]},{"osd":4,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.89000000000000001,"15min":0.91500000000000004},"min":{"1min":0.59599999999999997,"5min":0.309,"15min":0.309},"max":{"1min":1.45,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":1.5600000000000001,"5min":1.069,"15min":0.96599999999999997},"min":{"1min":0.504,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":11.563000000000001,"5min":11.563000000000001,"15min":11.563000000000001},"last":0.70299999999999996}]},{"osd":5,"last update":"Sat Sep 21 15:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":1.0620000000000001,"15min":0.95399999999999996},"min":{"1min":0.61199999999999999,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.3799999999999999,"5min":11.534000000000001,"15min":11.534000000000001},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.88,"5min":0.94199999999999995,"15min":0.95599999999999996},"min":{"1min":0.58599999999999997,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.1319999999999999,"5min":1.484,"15min":1.484},"last":0.78300000000000003}]},{"osd":6,"last update":"Sat Sep 21 15:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.97999999999999998,"15min":1.038},"min":{"1min":0.53100000000000003,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.254,"5min":2.3519999999999999,"15min":2.3519999999999999},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.97499999999999998,"5min":1.087,"15min":0.995},"min":{"1min":0.66400000000000003,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.4259999999999999,"5min":11.601000000000001,"15min":11.601000000000001},"last":0.61299999999999999}]},{"osd":7,"last update":"Sat Sep 21 15:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":1.02,"15min":1.056},"min":{"1min":0.58899999999999997,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.4039999999999999,"5min":2.2360000000000002,"15min":2.2360000000000002},"last":0.79600000000000004},{"interface":"front","average":{"1min":1.044,"5min":1.171,"15min":1.087},"min":{"1min":0.63100000000000001,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.3799999999999999,"5min":11.489000000000001,"15min":11.489000000000001},"last":0.66800000000000004}]}]},{"osd":0,"up_from":14,"seq":60129542201,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70184,"kb_used_data":2968,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94301656,"statfs":{"total":96636764160,"available":96564895744,"internally_reserved":0,"allocated":3039232,"data_stored":1498239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":68811207},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 21 15:02:07 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.64000000000000001,"15min":0.70599999999999996},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.002,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.69499999999999995,"15min":0.63200000000000001},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.98699999999999999,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.78100000000000003}]},{"osd":2,"last update":"Sat Sep 21 15:02:07 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.66100000000000003,"15min":0.621},"min":{"1min":0.33900000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.0960000000000001,"5min":1.153,"15min":1.153},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.70099999999999996,"15min":0.75},"min":{"1min":0.28799999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.0329999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.748}]},{"osd":3,"last update":"Sat Sep 21 15:02:07 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.76400000000000001,"15min":0.754},"min":{"1min":0.37,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.77400000000000002,"15min":0.81100000000000005},"min":{"1min":0.42799999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.04,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.54900000000000004}]},{"osd":4,"last update":"Sat Sep 21 15:01:04 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.98399999999999999,"15min":1.038},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.1140000000000001,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":1.069},{"interface":"front","average":{"1min":1.036,"5min":1.0469999999999999,"15min":1.0620000000000001},"min":{"1min":0.67800000000000005,"5min":0.379,"15min":0.379},"max":{"1min":1.446,"5min":1.665,"15min":1.665},"last":1.05}]},{"osd":5,"last update":"Sat Sep 21 15:02:07 2024","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.997,"15min":1.0229999999999999},"min":{"1min":0.45800000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.296,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.98699999999999999},{"interface":"front","average":{"1min":0.94299999999999995,"5min":1.0129999999999999,"15min":1.0669999999999999},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.3919999999999999,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":1.1240000000000001}]},{"osd":6,"last update":"Sat Sep 21 15:02:07 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.98799999999999999,"15min":1.089},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.119,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.94799999999999995,"5min":1.038,"15min":1.0600000000000001},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.393,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":1.214}]},{"osd":7,"last update":"Sat Sep 21 15:02:07 2024","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":1.012,"15min":1.05},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.1719999999999999,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":1.0289999999999999},{"interface":"front","average":{"1min":0.88300000000000001,"5min":1.0309999999999999,"15min":1.113},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.28,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":0.95999999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5202,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5202,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7385,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T15:02:34.212 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T15:02:34.503 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-21T15:02:34.530 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-09-21T15:02:34.530 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":381,"stamp":"2024-09-21T15:02:33.847574+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":174155,"num_read_kb":1238336,"num_write":34466,"num_write_kb":943253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3956,"num_bytes_recovered":212783659,"num_keys_recovered":305,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22661,"ondisk_log_size":22661,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":460608,"kb_used_data":23872,"kb_used_omap":19,"kb_used_meta":436588,"kb_avail":754514112,"statfs":{"total":773094113280,"available":772622450688,"internally_reserved":0,"allocated":24444928,"data_stored":13166724,"data_compressed":10404,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":20129,"internal_metadata":447066463},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.524348"},"pg_stats":[{"pgid":"3.1f","version":"230'598","reported_seq":3979,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:29.452925+0000","last_change":"2024-09-21T15:02:24.311888+0000","last_active":"2024-09-21T15:02:29.452925+0000","last_peered":"2024-09-21T15:02:29.452925+0000","last_clean":"2024-09-21T15:02:29.452925+0000","last_became_active":"2024-09-21T15:02:24.308451+0000","last_became_peered":"2024-09-21T15:02:24.308451+0000","last_unstale":"2024-09-21T15:02:29.452925+0000","last_undegraded":"2024-09-21T15:02:29.452925+0000","last_fullsized":"2024-09-21T15:02:29.452925+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":598,"log_dups_size":0,"ondisk_log_size":598,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:25:13.708671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010410479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3756,"num_read_kb":43219,"num_write":975,"num_write_kb":36436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"230'507","reported_seq":3510,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.335752+0000","last_change":"2024-09-21T15:02:24.312475+0000","last_active":"2024-09-21T15:02:28.335752+0000","last_peered":"2024-09-21T15:02:28.335752+0000","last_clean":"2024-09-21T15:02:28.335752+0000","last_became_active":"2024-09-21T15:02:24.309506+0000","last_became_peered":"2024-09-21T15:02:24.309506+0000","last_unstale":"2024-09-21T15:02:28.335752+0000","last_undegraded":"2024-09-21T15:02:28.335752+0000","last_fullsized":"2024-09-21T15:02:28.335752+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":507,"log_dups_size":0,"ondisk_log_size":507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:49:24.984436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094912299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3420,"num_read_kb":23407,"num_write":643,"num_write_kb":18445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":148,"num_bytes_recovered":11635695,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"228'341","reported_seq":4790,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.957133+0000","last_change":"2024-09-21T15:02:02.409182+0000","last_active":"2024-09-21T15:02:28.957133+0000","last_peered":"2024-09-21T15:02:28.957133+0000","last_clean":"2024-09-21T15:02:28.957133+0000","last_became_active":"2024-09-21T14:58:19.411387+0000","last_became_peered":"2024-09-21T14:58:19.411387+0000","last_unstale":"2024-09-21T15:02:28.957133+0000","last_undegraded":"2024-09-21T15:02:28.957133+0000","last_fullsized":"2024-09-21T15:02:28.957133+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":341,"log_dups_size":0,"ondisk_log_size":341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:37:57.597069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045388299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3878,"num_read_kb":28288,"num_write":535,"num_write_kb":26606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"230'572","reported_seq":4306,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.956805+0000","last_change":"2024-09-21T15:02:02.261134+0000","last_active":"2024-09-21T15:02:28.956805+0000","last_peered":"2024-09-21T15:02:28.956805+0000","last_clean":"2024-09-21T15:02:28.956805+0000","last_became_active":"2024-09-21T14:58:19.411306+0000","last_became_peered":"2024-09-21T14:58:19.411306+0000","last_unstale":"2024-09-21T15:02:28.956805+0000","last_undegraded":"2024-09-21T15:02:28.956805+0000","last_fullsized":"2024-09-21T15:02:28.956805+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":572,"log_dups_size":0,"ondisk_log_size":572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:20:27.079795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011721469999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4050,"num_read_kb":24676,"num_write":863,"num_write_kb":25228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"227'536","reported_seq":4293,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.335194+0000","last_change":"2024-09-21T15:02:17.248373+0000","last_active":"2024-09-21T15:02:28.335194+0000","last_peered":"2024-09-21T15:02:28.335194+0000","last_clean":"2024-09-21T15:02:28.335194+0000","last_became_active":"2024-09-21T15:02:17.242801+0000","last_became_peered":"2024-09-21T15:02:17.242801+0000","last_unstale":"2024-09-21T15:02:28.335194+0000","last_undegraded":"2024-09-21T15:02:28.335194+0000","last_fullsized":"2024-09-21T15:02:28.335194+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":536,"log_dups_size":0,"ondisk_log_size":536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:04:28.325046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032550800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3672,"num_read_kb":32698,"num_write":706,"num_write_kb":22582,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":379,"num_bytes_recovered":13616631,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"227'487","reported_seq":4857,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.956810+0000","last_change":"2024-09-21T15:02:02.262402+0000","last_active":"2024-09-21T15:02:28.956810+0000","last_peered":"2024-09-21T15:02:28.956810+0000","last_clean":"2024-09-21T15:02:28.956810+0000","last_became_active":"2024-09-21T14:58:19.409415+0000","last_became_peered":"2024-09-21T14:58:19.409415+0000","last_unstale":"2024-09-21T15:02:28.956810+0000","last_undegraded":"2024-09-21T15:02:28.956810+0000","last_fullsized":"2024-09-21T15:02:28.956810+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:28:46.336301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019572090000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4389,"num_read_kb":28349,"num_write":704,"num_write_kb":23876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"228'824","reported_seq":8731,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.335092+0000","last_change":"2024-09-21T15:02:18.252228+0000","last_active":"2024-09-21T15:02:28.335092+0000","last_peered":"2024-09-21T15:02:28.335092+0000","last_clean":"2024-09-21T15:02:28.335092+0000","last_became_active":"2024-09-21T15:02:18.246987+0000","last_became_peered":"2024-09-21T15:02:18.246987+0000","last_unstale":"2024-09-21T15:02:28.335092+0000","last_undegraded":"2024-09-21T15:02:28.335092+0000","last_fullsized":"2024-09-21T15:02:28.335092+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":824,"log_dups_size":0,"ondisk_log_size":824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:42:11.072452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014776850000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9383,"num_read_kb":68058,"num_write":1251,"num_write_kb":29921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":394,"num_bytes_recovered":23509073,"num_keys_recovered":66,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"227'688","reported_seq":8793,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.335591+0000","last_change":"2024-09-21T15:02:02.359228+0000","last_active":"2024-09-21T15:02:28.335591+0000","last_peered":"2024-09-21T15:02:28.335591+0000","last_clean":"2024-09-21T15:02:28.335591+0000","last_became_active":"2024-09-21T14:58:19.409371+0000","last_became_peered":"2024-09-21T14:58:19.409371+0000","last_unstale":"2024-09-21T15:02:28.335591+0000","last_undegraded":"2024-09-21T15:02:28.335591+0000","last_fullsized":"2024-09-21T15:02:28.335591+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":688,"log_dups_size":0,"ondisk_log_size":688,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:35:27.143445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052048100000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9236,"num_read_kb":45266,"num_write":998,"num_write_kb":30616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"227'413","reported_seq":4005,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.335834+0000","last_change":"2024-09-21T15:02:02.358947+0000","last_active":"2024-09-21T15:02:28.335834+0000","last_peered":"2024-09-21T15:02:28.335834+0000","last_clean":"2024-09-21T15:02:28.335834+0000","last_became_active":"2024-09-21T14:58:19.410306+0000","last_became_peered":"2024-09-21T14:58:19.410306+0000","last_unstale":"2024-09-21T15:02:28.335834+0000","last_undegraded":"2024-09-21T15:02:28.335834+0000","last_fullsized":"2024-09-21T15:02:28.335834+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":413,"log_dups_size":0,"ondisk_log_size":413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:33:50.089165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065724399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3380,"num_read_kb":28207,"num_write":769,"num_write_kb":34633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"229'1053","reported_seq":6175,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.956985+0000","last_change":"2024-09-21T15:02:02.263447+0000","last_active":"2024-09-21T15:02:28.956985+0000","last_peered":"2024-09-21T15:02:28.956985+0000","last_clean":"2024-09-21T15:02:28.956985+0000","last_became_active":"2024-09-21T15:01:06.166756+0000","last_became_peered":"2024-09-21T15:01:06.166756+0000","last_unstale":"2024-09-21T15:02:28.956985+0000","last_undegraded":"2024-09-21T15:02:28.956985+0000","last_fullsized":"2024-09-21T15:02:28.956985+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":1053,"log_dups_size":0,"ondisk_log_size":1053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:19:04.988784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001157057,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8073,"num_read_kb":43999,"num_write":1621,"num_write_kb":28374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":271,"num_bytes_recovered":16821266,"num_keys_recovered":55,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"230'823","reported_seq":3979,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.339198+0000","last_change":"2024-09-21T15:02:02.276055+0000","last_active":"2024-09-21T15:02:28.339198+0000","last_peered":"2024-09-21T15:02:28.339198+0000","last_clean":"2024-09-21T15:02:28.339198+0000","last_became_active":"2024-09-21T14:58:19.410687+0000","last_became_peered":"2024-09-21T14:58:19.410687+0000","last_unstale":"2024-09-21T15:02:28.339198+0000","last_undegraded":"2024-09-21T15:02:28.339198+0000","last_fullsized":"2024-09-21T15:02:28.339198+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":823,"log_dups_size":0,"ondisk_log_size":823,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:21:42.226982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017043010000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4757,"num_read_kb":49770,"num_write":1348,"num_write_kb":33339,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"228'527","reported_seq":4193,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.955479+0000","last_change":"2024-09-21T15:02:02.408062+0000","last_active":"2024-09-21T15:02:28.955479+0000","last_peered":"2024-09-21T15:02:28.955479+0000","last_clean":"2024-09-21T15:02:28.955479+0000","last_became_active":"2024-09-21T14:58:19.407310+0000","last_became_peered":"2024-09-21T14:58:19.407310+0000","last_unstale":"2024-09-21T15:02:28.955479+0000","last_undegraded":"2024-09-21T15:02:28.955479+0000","last_fullsized":"2024-09-21T15:02:28.955479+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":527,"log_dups_size":0,"ondisk_log_size":527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:47:46.079115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016534340000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3756,"num_read_kb":43322,"num_write":713,"num_write_kb":33471,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":245,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:29.771991+0000","last_change":"2024-09-21T14:58:11.348177+0000","last_active":"2024-09-21T15:02:29.771991+0000","last_peered":"2024-09-21T15:02:29.771991+0000","last_clean":"2024-09-21T15:02:29.771991+0000","last_became_active":"2024-09-21T14:58:09.328417+0000","last_became_peered":"2024-09-21T14:58:09.328417+0000","last_unstale":"2024-09-21T15:02:29.771991+0000","last_undegraded":"2024-09-21T15:02:29.771991+0000","last_fullsized":"2024-09-21T15:02:29.771991+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:52:22.671185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018917490000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"230'610","reported_seq":6567,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.335492+0000","last_change":"2024-09-21T15:02:17.251834+0000","last_active":"2024-09-21T15:02:28.335492+0000","last_peered":"2024-09-21T15:02:28.335492+0000","last_clean":"2024-09-21T15:02:28.335492+0000","last_became_active":"2024-09-21T15:02:17.247583+0000","last_became_peered":"2024-09-21T15:02:17.247583+0000","last_unstale":"2024-09-21T15:02:28.335492+0000","last_undegraded":"2024-09-21T15:02:28.335492+0000","last_fullsized":"2024-09-21T15:02:28.335492+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":610,"log_dups_size":0,"ondisk_log_size":610,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:54:31.703168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013409660000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6297,"num_read_kb":53037,"num_write":697,"num_write_kb":27961,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":11818795,"num_keys_recovered":18,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":24,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.335350+0000","last_change":"2024-09-21T15:02:14.225503+0000","last_active":"2024-09-21T15:02:28.335350+0000","last_peered":"2024-09-21T15:02:28.335350+0000","last_clean":"2024-09-21T15:02:28.335350+0000","last_became_active":"2024-09-21T15:02:14.224646+0000","last_became_peered":"2024-09-21T15:02:14.224646+0000","last_unstale":"2024-09-21T15:02:28.335350+0000","last_undegraded":"2024-09-21T15:02:28.335350+0000","last_fullsized":"2024-09-21T15:02:28.335350+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:57:35.948252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"228'980","reported_seq":4262,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.335626+0000","last_change":"2024-09-21T15:02:19.255988+0000","last_active":"2024-09-21T15:02:28.335626+0000","last_peered":"2024-09-21T15:02:28.335626+0000","last_clean":"2024-09-21T15:02:28.335626+0000","last_became_active":"2024-09-21T15:02:19.251008+0000","last_became_peered":"2024-09-21T15:02:19.251008+0000","last_unstale":"2024-09-21T15:02:28.335626+0000","last_undegraded":"2024-09-21T15:02:28.335626+0000","last_fullsized":"2024-09-21T15:02:28.335626+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":980,"log_dups_size":0,"ondisk_log_size":980,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:13:15.145504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015726799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5449,"num_read_kb":31198,"num_write":1400,"num_write_kb":25281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":401,"num_bytes_recovered":12566413,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":277,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.957358+0000","last_change":"2024-09-21T15:02:14.227927+0000","last_active":"2024-09-21T15:02:28.957358+0000","last_peered":"2024-09-21T15:02:28.957358+0000","last_clean":"2024-09-21T15:02:28.957358+0000","last_became_active":"2024-09-21T15:02:14.227223+0000","last_became_peered":"2024-09-21T15:02:14.227223+0000","last_unstale":"2024-09-21T15:02:28.957358+0000","last_undegraded":"2024-09-21T15:02:28.957358+0000","last_fullsized":"2024-09-21T15:02:28.957358+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:43:37.059225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019338770000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"227'672","reported_seq":5560,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.335126+0000","last_change":"2024-09-21T15:02:18.251535+0000","last_active":"2024-09-21T15:02:28.335126+0000","last_peered":"2024-09-21T15:02:28.335126+0000","last_clean":"2024-09-21T15:02:28.335126+0000","last_became_active":"2024-09-21T15:02:18.246450+0000","last_became_peered":"2024-09-21T15:02:18.246450+0000","last_unstale":"2024-09-21T15:02:28.335126+0000","last_undegraded":"2024-09-21T15:02:28.335126+0000","last_fullsized":"2024-09-21T15:02:28.335126+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":672,"log_dups_size":0,"ondisk_log_size":672,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:04:40.306321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071861500000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5178,"num_read_kb":38459,"num_write":1130,"num_write_kb":41306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":331,"num_bytes_recovered":25796920,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":232,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.335618+0000","last_change":"2024-09-21T15:02:24.351816+0000","last_active":"2024-09-21T15:02:28.335618+0000","last_peered":"2024-09-21T15:02:28.335618+0000","last_clean":"2024-09-21T15:02:28.335618+0000","last_became_active":"2024-09-21T15:02:24.351245+0000","last_became_peered":"2024-09-21T15:02:24.351245+0000","last_unstale":"2024-09-21T15:02:28.335618+0000","last_undegraded":"2024-09-21T15:02:28.335618+0000","last_fullsized":"2024-09-21T15:02:28.335618+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:43:32.690041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"227'732","reported_seq":2860,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:29.354087+0000","last_change":"2024-09-21T15:02:02.262648+0000","last_active":"2024-09-21T15:02:29.354087+0000","last_peered":"2024-09-21T15:02:29.354087+0000","last_clean":"2024-09-21T15:02:29.354087+0000","last_became_active":"2024-09-21T14:58:19.408971+0000","last_became_peered":"2024-09-21T14:58:19.408971+0000","last_unstale":"2024-09-21T15:02:29.354087+0000","last_undegraded":"2024-09-21T15:02:29.354087+0000","last_fullsized":"2024-09-21T15:02:29.354087+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":732,"log_dups_size":0,"ondisk_log_size":732,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:29:52.642121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073528299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3602,"num_read_kb":20655,"num_write":1230,"num_write_kb":20679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"230'1009","reported_seq":3690,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:29.738404+0000","last_change":"2024-09-21T15:02:02.276284+0000","last_active":"2024-09-21T15:02:29.738404+0000","last_peered":"2024-09-21T15:02:29.738404+0000","last_clean":"2024-09-21T15:02:29.738404+0000","last_became_active":"2024-09-21T14:58:19.417453+0000","last_became_peered":"2024-09-21T14:58:19.417453+0000","last_unstale":"2024-09-21T15:02:29.738404+0000","last_undegraded":"2024-09-21T15:02:29.738404+0000","last_fullsized":"2024-09-21T15:02:29.738404+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":1009,"log_dups_size":0,"ondisk_log_size":1009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:19:27.087272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044463100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4845,"num_read_kb":25284,"num_write":1566,"num_write_kb":21735,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":247,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.956823+0000","last_change":"2024-09-21T14:58:11.347504+0000","last_active":"2024-09-21T15:02:28.956823+0000","last_peered":"2024-09-21T15:02:28.956823+0000","last_clean":"2024-09-21T15:02:28.956823+0000","last_became_active":"2024-09-21T14:58:09.775981+0000","last_became_peered":"2024-09-21T14:58:09.775981+0000","last_unstale":"2024-09-21T15:02:28.956823+0000","last_undegraded":"2024-09-21T15:02:28.956823+0000","last_fullsized":"2024-09-21T15:02:28.956823+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:47:16.286652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012568880000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"228'1092","reported_seq":4001,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.335364+0000","last_change":"2024-09-21T15:02:24.314768+0000","last_active":"2024-09-21T15:02:28.335364+0000","last_peered":"2024-09-21T15:02:28.335364+0000","last_clean":"2024-09-21T15:02:28.335364+0000","last_became_active":"2024-09-21T15:02:24.311531+0000","last_became_peered":"2024-09-21T15:02:24.311531+0000","last_unstale":"2024-09-21T15:02:28.335364+0000","last_undegraded":"2024-09-21T15:02:28.335364+0000","last_fullsized":"2024-09-21T15:02:28.335364+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":1092,"log_dups_size":0,"ondisk_log_size":1092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:22:48.528933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00183417,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6343,"num_read_kb":28797,"num_write":1677,"num_write_kb":26974,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":246,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.953692+0000","last_change":"2024-09-21T14:58:11.342210+0000","last_active":"2024-09-21T15:02:28.953692+0000","last_peered":"2024-09-21T15:02:28.953692+0000","last_clean":"2024-09-21T15:02:28.953692+0000","last_became_active":"2024-09-21T14:58:09.330624+0000","last_became_peered":"2024-09-21T14:58:09.330624+0000","last_unstale":"2024-09-21T15:02:28.953692+0000","last_undegraded":"2024-09-21T15:02:28.953692+0000","last_fullsized":"2024-09-21T15:02:28.953692+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:29:43.229538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001051151,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"227'339","reported_seq":2553,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:29.769648+0000","last_change":"2024-09-21T15:02:02.257475+0000","last_active":"2024-09-21T15:02:29.769648+0000","last_peered":"2024-09-21T15:02:29.769648+0000","last_clean":"2024-09-21T15:02:29.769648+0000","last_became_active":"2024-09-21T15:01:53.193166+0000","last_became_peered":"2024-09-21T15:01:53.193166+0000","last_unstale":"2024-09-21T15:02:29.769648+0000","last_undegraded":"2024-09-21T15:02:29.769648+0000","last_fullsized":"2024-09-21T15:02:29.769648+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":339,"log_dups_size":0,"ondisk_log_size":339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:37:52.546654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014983959999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2371,"num_read_kb":40193,"num_write":529,"num_write_kb":39390,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":366,"num_bytes_recovered":22723935,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":245,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.955327+0000","last_change":"2024-09-21T14:58:11.346766+0000","last_active":"2024-09-21T15:02:28.955327+0000","last_peered":"2024-09-21T15:02:28.955327+0000","last_clean":"2024-09-21T15:02:28.955327+0000","last_became_active":"2024-09-21T14:58:09.329748+0000","last_became_peered":"2024-09-21T14:58:09.329748+0000","last_unstale":"2024-09-21T15:02:28.955327+0000","last_undegraded":"2024-09-21T15:02:28.955327+0000","last_fullsized":"2024-09-21T15:02:28.955327+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:31:20.267675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052698000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"230'1275","reported_seq":8379,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.955421+0000","last_change":"2024-09-21T15:02:02.261569+0000","last_active":"2024-09-21T15:02:28.955421+0000","last_peered":"2024-09-21T15:02:28.955421+0000","last_clean":"2024-09-21T15:02:28.955421+0000","last_became_active":"2024-09-21T14:58:41.401003+0000","last_became_peered":"2024-09-21T14:58:41.401003+0000","last_unstale":"2024-09-21T15:02:28.955421+0000","last_undegraded":"2024-09-21T15:02:28.955421+0000","last_fullsized":"2024-09-21T15:02:28.955421+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":1275,"log_dups_size":0,"ondisk_log_size":1275,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:05:37.644835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000746033,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11708,"num_read_kb":50816,"num_write":1955,"num_write_kb":24913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1403348,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":246,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.955718+0000","last_change":"2024-09-21T14:58:11.347700+0000","last_active":"2024-09-21T15:02:28.955718+0000","last_peered":"2024-09-21T15:02:28.955718+0000","last_clean":"2024-09-21T15:02:28.955718+0000","last_became_active":"2024-09-21T14:58:09.438842+0000","last_became_peered":"2024-09-21T14:58:09.438842+0000","last_unstale":"2024-09-21T15:02:28.955718+0000","last_undegraded":"2024-09-21T15:02:28.955718+0000","last_fullsized":"2024-09-21T15:02:28.955718+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:41:34.626249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001426194,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":394,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.957058+0000","last_change":"2024-09-21T14:58:03.616055+0000","last_active":"2024-09-21T15:02:28.957058+0000","last_peered":"2024-09-21T15:02:28.957058+0000","last_clean":"2024-09-21T15:02:28.957058+0000","last_became_active":"2024-09-21T14:58:03.613214+0000","last_became_peered":"2024-09-21T14:58:03.613214+0000","last_unstale":"2024-09-21T15:02:28.957058+0000","last_undegraded":"2024-09-21T15:02:28.957058+0000","last_fullsized":"2024-09-21T15:02:28.957058+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:02.180162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:02.180162+0000","last_clean_scrub_stamp":"2024-09-21T14:58:02.180162+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:29:05.860361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"228'685","reported_seq":4697,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.953055+0000","last_change":"2024-09-21T15:02:02.257763+0000","last_active":"2024-09-21T15:02:28.953055+0000","last_peered":"2024-09-21T15:02:28.953055+0000","last_clean":"2024-09-21T15:02:28.953055+0000","last_became_active":"2024-09-21T15:01:32.177913+0000","last_became_peered":"2024-09-21T15:01:32.177913+0000","last_unstale":"2024-09-21T15:02:28.953055+0000","last_undegraded":"2024-09-21T15:02:28.953055+0000","last_fullsized":"2024-09-21T15:02:28.953055+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":685,"log_dups_size":0,"ondisk_log_size":685,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:47:09.376828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015618730000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4407,"num_read_kb":44538,"num_write":1164,"num_write_kb":30502,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":413,"num_bytes_recovered":20409497,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.9","version":"229'973","reported_seq":4119,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.955874+0000","last_change":"2024-09-21T15:02:18.257627+0000","last_active":"2024-09-21T15:02:28.955874+0000","last_peered":"2024-09-21T15:02:28.955874+0000","last_clean":"2024-09-21T15:02:28.955874+0000","last_became_active":"2024-09-21T15:02:18.255482+0000","last_became_peered":"2024-09-21T15:02:18.255482+0000","last_unstale":"2024-09-21T15:02:28.955874+0000","last_undegraded":"2024-09-21T15:02:28.955874+0000","last_fullsized":"2024-09-21T15:02:28.955874+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":973,"log_dups_size":0,"ondisk_log_size":973,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:15:56.076204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001545105,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4815,"num_read_kb":35700,"num_write":1595,"num_write_kb":33802,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":375,"num_bytes_recovered":14742839,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"229'1027","reported_seq":5029,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.956693+0000","last_change":"2024-09-21T15:02:02.261576+0000","last_active":"2024-09-21T15:02:28.956693+0000","last_peered":"2024-09-21T15:02:28.956693+0000","last_clean":"2024-09-21T15:02:28.956693+0000","last_became_active":"2024-09-21T15:00:27.280971+0000","last_became_peered":"2024-09-21T15:00:27.280971+0000","last_unstale":"2024-09-21T15:02:28.956693+0000","last_undegraded":"2024-09-21T15:02:28.956693+0000","last_fullsized":"2024-09-21T15:02:28.956693+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":1027,"log_dups_size":0,"ondisk_log_size":1027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:26:07.617059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013613779999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7341,"num_read_kb":31141,"num_write":1421,"num_write_kb":33160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":9228123,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"227'593","reported_seq":3447,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.953752+0000","last_change":"2024-09-21T15:02:02.265620+0000","last_active":"2024-09-21T15:02:28.953752+0000","last_peered":"2024-09-21T15:02:28.953752+0000","last_clean":"2024-09-21T15:02:28.953752+0000","last_became_active":"2024-09-21T14:58:19.410679+0000","last_became_peered":"2024-09-21T14:58:19.410679+0000","last_unstale":"2024-09-21T15:02:28.953752+0000","last_undegraded":"2024-09-21T15:02:28.953752+0000","last_fullsized":"2024-09-21T15:02:28.953752+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":593,"log_dups_size":0,"ondisk_log_size":593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:07:40.758932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001466743,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3111,"num_read_kb":36762,"num_write":889,"num_write_kb":39649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"228'493","reported_seq":4709,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.338915+0000","last_change":"2024-09-21T15:02:02.275784+0000","last_active":"2024-09-21T15:02:28.338915+0000","last_peered":"2024-09-21T15:02:28.338915+0000","last_clean":"2024-09-21T15:02:28.338915+0000","last_became_active":"2024-09-21T14:59:25.130057+0000","last_became_peered":"2024-09-21T14:59:25.130057+0000","last_unstale":"2024-09-21T15:02:28.338915+0000","last_undegraded":"2024-09-21T15:02:28.338915+0000","last_fullsized":"2024-09-21T15:02:28.338915+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:53:33.752550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014500649999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4513,"num_read_kb":28786,"num_write":729,"num_write_kb":24616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":9059600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"227'544","reported_seq":5354,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.957258+0000","last_change":"2024-09-21T15:02:02.409235+0000","last_active":"2024-09-21T15:02:28.957258+0000","last_peered":"2024-09-21T15:02:28.957258+0000","last_clean":"2024-09-21T15:02:28.957258+0000","last_became_active":"2024-09-21T14:58:19.407884+0000","last_became_peered":"2024-09-21T14:58:19.407884+0000","last_unstale":"2024-09-21T15:02:28.957258+0000","last_undegraded":"2024-09-21T15:02:28.957258+0000","last_fullsized":"2024-09-21T15:02:28.957258+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":544,"log_dups_size":0,"ondisk_log_size":544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:28:57.111615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045666499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4968,"num_read_kb":55828,"num_write":945,"num_write_kb":38650,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"230'879","reported_seq":3218,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.956772+0000","last_change":"2024-09-21T15:02:02.263156+0000","last_active":"2024-09-21T15:02:28.956772+0000","last_peered":"2024-09-21T15:02:28.956772+0000","last_clean":"2024-09-21T15:02:28.956772+0000","last_became_active":"2024-09-21T15:01:05.172427+0000","last_became_peered":"2024-09-21T15:01:05.172427+0000","last_unstale":"2024-09-21T15:02:28.956772+0000","last_undegraded":"2024-09-21T15:02:28.956772+0000","last_fullsized":"2024-09-21T15:02:28.956772+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":879,"log_dups_size":0,"ondisk_log_size":879,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:28:18.710621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001142789,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4262,"num_read_kb":17288,"num_write":1508,"num_write_kb":17970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":138,"num_bytes_recovered":7486496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"227'532","reported_seq":6209,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.339180+0000","last_change":"2024-09-21T15:02:02.275716+0000","last_active":"2024-09-21T15:02:28.339180+0000","last_peered":"2024-09-21T15:02:28.339180+0000","last_clean":"2024-09-21T15:02:28.339180+0000","last_became_active":"2024-09-21T14:58:19.410406+0000","last_became_peered":"2024-09-21T14:58:19.410406+0000","last_unstale":"2024-09-21T15:02:28.339180+0000","last_undegraded":"2024-09-21T15:02:28.339180+0000","last_fullsized":"2024-09-21T15:02:28.339180+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:59:16.026939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014059350000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5807,"num_read_kb":56273,"num_write":817,"num_write_kb":36773,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"228'523","reported_seq":4647,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.956536+0000","last_change":"2024-09-21T15:02:02.260617+0000","last_active":"2024-09-21T15:02:28.956536+0000","last_peered":"2024-09-21T15:02:28.956536+0000","last_clean":"2024-09-21T15:02:28.956536+0000","last_became_active":"2024-09-21T14:58:19.409935+0000","last_became_peered":"2024-09-21T14:58:19.409935+0000","last_unstale":"2024-09-21T15:02:28.956536+0000","last_undegraded":"2024-09-21T15:02:28.956536+0000","last_fullsized":"2024-09-21T15:02:28.956536+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":523,"log_dups_size":0,"ondisk_log_size":523,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:10:36.252629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068183199999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3949,"num_read_kb":58693,"num_write":810,"num_write_kb":37279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"230'623","reported_seq":5195,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.955624+0000","last_change":"2024-09-21T15:02:02.408072+0000","last_active":"2024-09-21T15:02:28.955624+0000","last_peered":"2024-09-21T15:02:28.955624+0000","last_clean":"2024-09-21T15:02:28.955624+0000","last_became_active":"2024-09-21T15:01:05.165955+0000","last_became_peered":"2024-09-21T15:01:05.165955+0000","last_unstale":"2024-09-21T15:02:28.955624+0000","last_undegraded":"2024-09-21T15:02:28.955624+0000","last_fullsized":"2024-09-21T15:02:28.955624+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":623,"log_dups_size":0,"ondisk_log_size":623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:34:54.111119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021463580000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5758,"num_read_kb":28485,"num_write":777,"num_write_kb":24185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":166,"num_bytes_recovered":11965028,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"229'644","reported_seq":8535,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.953028+0000","last_change":"2024-09-21T15:02:02.256934+0000","last_active":"2024-09-21T15:02:28.953028+0000","last_peered":"2024-09-21T15:02:28.953028+0000","last_clean":"2024-09-21T15:02:28.953028+0000","last_became_active":"2024-09-21T15:00:07.155144+0000","last_became_peered":"2024-09-21T15:00:07.155144+0000","last_unstale":"2024-09-21T15:02:28.953028+0000","last_undegraded":"2024-09-21T15:02:28.953028+0000","last_fullsized":"2024-09-21T15:02:28.953028+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":644,"log_dups_size":0,"ondisk_log_size":644,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:58:00.133639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010987029999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9100,"num_read_kb":37252,"num_write":1013,"num_write_kb":20896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"227'947","reported_seq":6822,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.955476+0000","last_change":"2024-09-21T15:02:02.406594+0000","last_active":"2024-09-21T15:02:28.955476+0000","last_peered":"2024-09-21T15:02:28.955476+0000","last_clean":"2024-09-21T15:02:28.955476+0000","last_became_active":"2024-09-21T14:58:19.412030+0000","last_became_peered":"2024-09-21T14:58:19.412030+0000","last_unstale":"2024-09-21T15:02:28.955476+0000","last_undegraded":"2024-09-21T15:02:28.955476+0000","last_fullsized":"2024-09-21T15:02:28.955476+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":947,"log_dups_size":0,"ondisk_log_size":947,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:44:29.282391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067099300000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8465,"num_read_kb":59792,"num_write":1327,"num_write_kb":31587,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":174039,"num_read_kb":1238236,"num_write":34305,"num_write_kb":940835,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3956,"num_bytes_recovered":212783659,"num_keys_recovered":305,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7385,"internal_metadata":0},"log_size":22541,"ondisk_log_size":22541,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":10404,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509499,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34088,"kb_used_data":2968,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94337752,"statfs":{"total":96636764160,"available":96601858048,"internally_reserved":0,"allocated":3039232,"data_stored":1498239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:02:06 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.83899999999999997,"15min":0.83699999999999997},"min":{"1min":0.29199999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.044,"5min":1.351,"15min":1.351},"last":1.135},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.91400000000000003,"15min":0.94599999999999995},"min":{"1min":0.48099999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.0589999999999999,"5min":1.917,"15min":1.917},"last":1.0009999999999999}]},{"osd":1,"last update":"Sat Sep 21 15:02:06 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.80800000000000005,"15min":0.77600000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":1.0409999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.88},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.81599999999999995,"15min":0.82099999999999995},"min":{"1min":0.40500000000000003,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.002,"5min":2.2149999999999999,"15min":2.2149999999999999},"last":0.47599999999999998}]},{"osd":2,"last update":"Sat Sep 21 15:02:06 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.82899999999999996,"15min":0.81499999999999995},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.161,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.92600000000000005,"15min":0.93999999999999995},"min":{"1min":0.56499999999999995,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.0920000000000001,"5min":1.466,"15min":1.466},"last":0.96799999999999997}]},{"osd":3,"last update":"Sat Sep 21 15:02:06 2024","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.89100000000000001,"15min":0.86299999999999999},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.24,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":1.0669999999999999},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.96799999999999997,"15min":0.96599999999999997},"min":{"1min":0.54200000000000004,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.159,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":1.1919999999999999}]},{"osd":4,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.92200000000000004,"15min":0.93300000000000005},"min":{"1min":0.42199999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.3089999999999999,"5min":2.7130000000000001,"15min":2.7130000000000001},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.94499999999999995,"15min":0.92600000000000005},"min":{"1min":0.39000000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.425,"5min":2.125,"15min":2.125},"last":0.66000000000000003}]},{"osd":5,"last update":"Sat Sep 21 15:02:06 2024","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":1.006,"15min":1.016},"min":{"1min":0.505,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.375,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":1.0389999999999999},{"interface":"front","average":{"1min":0.97699999999999998,"5min":1.026,"15min":1.0449999999999999},"min":{"1min":0.61599999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.425,"5min":3.1880000000000002,"15min":3.1880000000000002},"last":1.1599999999999999}]},{"osd":6,"last update":"Sat Sep 21 15:02:06 2024","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.115,"15min":1.143},"min":{"1min":0.52900000000000003,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.679,"5min":3.2730000000000001,"15min":3.2730000000000001},"last":1.0840000000000001},{"interface":"front","average":{"1min":1.099,"5min":1.0800000000000001,"15min":1.036},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.6719999999999999,"5min":2.2879999999999998,"15min":2.2879999999999998},"last":1.113}]}]},{"osd":6,"up_from":14,"seq":60129542202,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64432,"kb_used_data":2972,"kb_used_omap":1,"kb_used_meta":61438,"kb_avail":94307408,"statfs":{"total":96636764160,"available":96570785792,"internally_reserved":0,"allocated":3043328,"data_stored":1498258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":62912967},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:02:09 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.87,"15min":0.93200000000000005},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.089,"5min":1.593,"15min":1.593},"last":1.222},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.90500000000000003,"15min":0.94099999999999995},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.1180000000000001,"5min":1.587,"15min":1.587},"last":0.70599999999999996}]},{"osd":1,"last update":"Sat Sep 21 15:02:09 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.876,"15min":0.90400000000000003},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.145,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":1.0940000000000001},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.85099999999999998,"15min":0.94499999999999995},"min":{"1min":0.40600000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.036,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":1.121}]},{"osd":2,"last update":"Sat Sep 21 15:02:09 2024","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.93600000000000005,"15min":0.97499999999999998},"min":{"1min":0.53400000000000003,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.194,"5min":1.758,"15min":1.758},"last":1.266},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.878,"15min":0.90200000000000002},"min":{"1min":0.51700000000000002,"5min":0.496,"15min":0.496},"max":{"1min":1.002,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.73599999999999999}]},{"osd":3,"last update":"Sat Sep 21 15:02:09 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.91600000000000004,"15min":0.94799999999999995},"min":{"1min":0.57399999999999995,"5min":0.434,"15min":0.434},"max":{"1min":0.98699999999999999,"5min":1.835,"15min":1.835},"last":1.137},{"interface":"front","average":{"1min":0.90100000000000002,"5min":1.0429999999999999,"15min":1.111},"min":{"1min":0.65900000000000003,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.296,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.92700000000000005}]},{"osd":4,"last update":"Sat Sep 21 15:01:08 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.93500000000000005,"15min":0.99099999999999999},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":1.2,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.98499999999999999,"15min":1.0209999999999999},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.3759999999999999,"5min":1.897,"15min":1.897},"last":0.96999999999999997}]},{"osd":5,"last update":"Sat Sep 21 15:02:09 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.97799999999999998,"15min":1.0880000000000001},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.222,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.85699999999999998,"5min":1.018,"15min":1.071},"min":{"1min":0.54200000000000004,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.3600000000000001,"5min":1.609,"15min":1.609},"last":1.0209999999999999}]},{"osd":7,"last update":"Sat Sep 21 15:02:09 2024","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":1.131,"15min":1.2190000000000001},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.256,"5min":1.8660000000000001,"15min":1.8660000000000001},"last":1.1619999999999999},{"interface":"front","average":{"1min":0.91300000000000003,"5min":1.117,"15min":1.248},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.413,"5min":2.004,"15min":2.004},"last":1.071}]}]},{"osd":5,"up_from":14,"seq":60129542203,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60384,"kb_used_data":3028,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94311456,"statfs":{"total":96636764160,"available":96574930944,"internally_reserved":0,"allocated":3100672,"data_stored":2088626,"data_compressed":5202,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:02:09 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.84199999999999997,"15min":0.96899999999999997},"min":{"1min":0.40100000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.98099999999999998,"5min":4.4800000000000004,"15min":4.4800000000000004},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.78400000000000003,"15min":0.81299999999999994},"min":{"1min":0.44500000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.179,"5min":1.181,"15min":1.181},"last":1.173}]},{"osd":1,"last update":"Sat Sep 21 15:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.86399999999999999,"15min":0.91300000000000003},"min":{"1min":0.41899999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.036,"5min":2.7709999999999999,"15min":2.7709999999999999},"last":1.0049999999999999},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.89900000000000002,"15min":0.92900000000000005},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.1439999999999999,"5min":2.823,"15min":2.823},"last":1.3420000000000001}]},{"osd":2,"last update":"Sat Sep 21 15:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.86599999999999999,"15min":0.95799999999999996},"min":{"1min":0.53600000000000003,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.2090000000000001,"5min":3.419,"15min":3.419},"last":0.754},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.84499999999999997,"15min":0.88800000000000001},"min":{"1min":0.58999999999999997,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.1699999999999999,"5min":1.423,"15min":1.423},"last":0.91600000000000004}]},{"osd":3,"last update":"Sat Sep 21 15:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.81100000000000005,"15min":0.81000000000000005},"min":{"1min":0.51000000000000001,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.105,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.97299999999999998},{"interface":"front","average":{"1min":0.91700000000000004,"5min":1.018,"15min":1.0720000000000001},"min":{"1min":0.66600000000000004,"5min":0.435,"15min":0.435},"max":{"1min":1.4239999999999999,"5min":2.6379999999999999,"15min":2.6379999999999999},"last":1.038}]},{"osd":4,"last update":"Sat Sep 21 15:01:02 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.98899999999999999,"15min":1.016},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.5609999999999999,"5min":2.5470000000000002,"15min":2.5470000000000002},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.82099999999999995,"5min":1.0169999999999999,"15min":1.0780000000000001},"min":{"1min":0.39300000000000002,"5min":0.376,"15min":0.376},"max":{"1min":1.302,"5min":2.5099999999999998,"15min":2.5099999999999998},"last":1.145}]},{"osd":6,"last update":"Sat Sep 21 15:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.91700000000000004,"15min":0.95999999999999996},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.1499999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.879,"5min":1.1399999999999999,"15min":1.3169999999999999},"min":{"1min":0.60199999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.498,"5min":4.2469999999999999,"15min":4.2469999999999999},"last":0.88200000000000001}]},{"osd":7,"last update":"Sat Sep 21 15:02:09 2024","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":1.167,"15min":1.3120000000000001},"min":{"1min":0.61199999999999999,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.5700000000000001,"5min":3.3010000000000002,"15min":3.3010000000000002},"last":1.117},{"interface":"front","average":{"1min":0.99399999999999999,"5min":1.075,"15min":1.125},"min":{"1min":0.64300000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.25,"5min":2.5880000000000001,"15min":2.5880000000000001},"last":1.4430000000000001}]}]},{"osd":4,"up_from":235,"seq":1009317314567,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34860,"kb_used_data":2968,"kb_used_omap":8,"kb_used_meta":31863,"kb_avail":94336980,"statfs":{"total":96636764160,"available":96601067520,"internally_reserved":0,"allocated":3039232,"data_stored":1498239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8979,"internal_metadata":32627949},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0269999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.091}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86899999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92000000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2390000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542203,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78376,"kb_used_data":2972,"kb_used_omap":1,"kb_used_meta":75390,"kb_avail":94293464,"statfs":{"total":96636764160,"available":96556507136,"internally_reserved":0,"allocated":3043328,"data_stored":1498258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":77199816},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.61799999999999999,"15min":0.63200000000000001},"min":{"1min":0.34599999999999997,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.92700000000000005,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.68200000000000005,"15min":0.70999999999999996},"min":{"1min":0.40500000000000003,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.3029999999999999,"5min":1.919,"15min":1.919},"last":0.29999999999999999}]},{"osd":1,"last update":"Sat Sep 21 15:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.65800000000000003,"15min":0.67000000000000004},"min":{"1min":0.44400000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":1.115,"5min":1.722,"15min":1.722},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.746,"15min":0.79700000000000004},"min":{"1min":0.41199999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.2509999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.28499999999999998}]},{"osd":2,"last update":"Sat Sep 21 15:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.746,"15min":0.72999999999999998},"min":{"1min":0.38200000000000001,"5min":0.311,"15min":0.311},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.746,"5min":0.79200000000000004,"15min":0.82099999999999995},"min":{"1min":0.40300000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.94399999999999995,"5min":1.254,"15min":1.254},"last":0.49199999999999999}]},{"osd":4,"last update":"Sat Sep 21 15:01:09 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.93799999999999994,"15min":0.95599999999999996},"min":{"1min":0.54300000000000004,"5min":0.442,"15min":0.442},"max":{"1min":1.2629999999999999,"5min":1.7609999999999999,"15min":1.7609999999999999},"last":1.0600000000000001},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.98899999999999999,"15min":1.0269999999999999},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.234,"5min":2.028,"15min":2.028},"last":0.95399999999999996}]},{"osd":5,"last update":"Sat Sep 21 15:02:13 2024","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.036,"15min":1.052},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.4670000000000001,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":0.76500000000000001},{"interface":"front","average":{"1min":1.081,"5min":1.0589999999999999,"15min":1.0640000000000001},"min":{"1min":0.68799999999999994,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.395,"5min":1.548,"15min":1.548},"last":0.86199999999999999}]},{"osd":6,"last update":"Sat Sep 21 15:02:13 2024","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":1.046,"15min":1.085},"min":{"1min":0.70099999999999996,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.391,"5min":2.1219999999999999,"15min":2.1219999999999999},"last":0.78100000000000003},{"interface":"front","average":{"1min":1.03,"5min":1.075,"15min":1.0980000000000001},"min":{"1min":0.64900000000000002,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.448,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":0.747}]},{"osd":7,"last update":"Sat Sep 21 15:02:13 2024","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.0800000000000001,"15min":1.123},"min":{"1min":0.746,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.339,"5min":1.706,"15min":1.706},"last":0.84899999999999998},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.1279999999999999,"15min":1.171},"min":{"1min":0.67700000000000005,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.421,"5min":2.3380000000000001,"15min":2.3380000000000001},"last":0.89900000000000002}]}]},{"osd":2,"up_from":14,"seq":60129542201,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70180,"kb_used_data":2968,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94301660,"statfs":{"total":96636764160,"available":96564899840,"internally_reserved":0,"allocated":3039232,"data_stored":1498239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68811206},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:02:07 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.63500000000000001,"15min":0.625},"min":{"1min":0.32100000000000001,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.383,"5min":1.383,"15min":1.383},"last":0.439},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.69099999999999995,"15min":0.66900000000000004},"min":{"1min":0.30099999999999999,"5min":0.223,"15min":0.223},"max":{"1min":1.1739999999999999,"5min":2.9169999999999998,"15min":2.9169999999999998},"last":0.28399999999999997}]},{"osd":1,"last update":"Sat Sep 21 15:02:07 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.67500000000000004,"15min":0.68100000000000005},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.129,"5min":2.6520000000000001,"15min":2.6520000000000001},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.73399999999999999,"15min":0.75800000000000001},"min":{"1min":0.36199999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.48899999999999999}]},{"osd":3,"last update":"Sat Sep 21 15:02:07 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.71699999999999997,"15min":0.72699999999999998},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.996,"5min":3.0110000000000001,"15min":3.0110000000000001},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.85999999999999999,"15min":0.83399999999999996},"min":{"1min":0.41099999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0369999999999999,"5min":4.1440000000000001,"15min":4.1440000000000001},"last":0.50600000000000001}]},{"osd":4,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.93200000000000005,"15min":0.95599999999999996},"min":{"1min":0.42399999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.222,"5min":2.899,"15min":2.899},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.94399999999999995,"15min":0.95799999999999996},"min":{"1min":0.47499999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.4379999999999999,"5min":1.675,"15min":1.675},"last":0.58899999999999997}]},{"osd":5,"last update":"Sat Sep 21 15:02:07 2024","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.98699999999999999,"15min":1.0009999999999999},"min":{"1min":0.45500000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.224,"5min":3.1080000000000001,"15min":3.1080000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.96399999999999997,"15min":0.98799999999999999},"min":{"1min":0.55200000000000005,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.228,"5min":1.546,"15min":1.546},"last":0.91300000000000003}]},{"osd":6,"last update":"Sat Sep 21 15:02:07 2024","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":0.98899999999999999,"15min":0.98699999999999999},"min":{"1min":0.52100000000000002,"5min":0.378,"15min":0.378},"max":{"1min":1.516,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.93999999999999995,"5min":1.0429999999999999,"15min":1.048},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.403,"5min":3.0739999999999998,"15min":3.0739999999999998},"last":0.625}]},{"osd":7,"last update":"Sat Sep 21 15:02:09 2024","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.0489999999999999,"15min":1.01},"min":{"1min":0.61399999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.415,"5min":3.1589999999999998,"15min":3.1589999999999998},"last":0.84499999999999997},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.111,"15min":1.095},"min":{"1min":0.58799999999999997,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.4610000000000001,"5min":3.1400000000000001,"15min":3.1400000000000001},"last":0.69799999999999995}]}]},{"osd":1,"up_from":14,"seq":60129542204,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48104,"kb_used_data":3028,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323736,"statfs":{"total":96636764160,"available":96587505664,"internally_reserved":0,"allocated":3100672,"data_stored":2088626,"data_compressed":5202,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.58899999999999997,"15min":0.60699999999999998},"min":{"1min":0.25600000000000001,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.60299999999999998,"15min":0.60999999999999999},"min":{"1min":0.26300000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.83499999999999996,"5min":1.107,"15min":1.107},"last":0.47199999999999998}]},{"osd":2,"last update":"Sat Sep 21 15:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.78000000000000003,"15min":0.73499999999999999},"min":{"1min":0.38800000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.92100000000000004,"5min":11.303000000000001,"15min":11.303000000000001},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.71399999999999997,"15min":0.70299999999999996},"min":{"1min":0.439,"5min":0.435,"15min":0.435},"max":{"1min":1.0840000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.373}]},{"osd":3,"last update":"Sat Sep 21 15:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.80200000000000005,"15min":0.83399999999999996},"min":{"1min":0.36899999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.4850000000000001,"5min":3.1469999999999998,"15min":3.1469999999999998},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.83699999999999997,"15min":0.877},"min":{"1min":0.42499999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.284,"5min":3.2320000000000002,"15min":3.2320000000000002},"last":0.41199999999999998}]},{"osd":4,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.89000000000000001,"15min":0.91500000000000004},"min":{"1min":0.59599999999999997,"5min":0.309,"15min":0.309},"max":{"1min":1.45,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":1.5600000000000001,"5min":1.069,"15min":0.96599999999999997},"min":{"1min":0.504,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":11.563000000000001,"5min":11.563000000000001,"15min":11.563000000000001},"last":0.70299999999999996}]},{"osd":5,"last update":"Sat Sep 21 15:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":1.0620000000000001,"15min":0.95399999999999996},"min":{"1min":0.61199999999999999,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.3799999999999999,"5min":11.534000000000001,"15min":11.534000000000001},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.88,"5min":0.94199999999999995,"15min":0.95599999999999996},"min":{"1min":0.58599999999999997,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.1319999999999999,"5min":1.484,"15min":1.484},"last":0.78300000000000003}]},{"osd":6,"last update":"Sat Sep 21 15:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.97999999999999998,"15min":1.038},"min":{"1min":0.53100000000000003,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.254,"5min":2.3519999999999999,"15min":2.3519999999999999},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.97499999999999998,"5min":1.087,"15min":0.995},"min":{"1min":0.66400000000000003,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.4259999999999999,"5min":11.601000000000001,"15min":11.601000000000001},"last":0.61299999999999999}]},{"osd":7,"last update":"Sat Sep 21 15:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":1.02,"15min":1.056},"min":{"1min":0.58899999999999997,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.4039999999999999,"5min":2.2360000000000002,"15min":2.2360000000000002},"last":0.79600000000000004},{"interface":"front","average":{"1min":1.044,"5min":1.171,"15min":1.087},"min":{"1min":0.63100000000000001,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.3799999999999999,"5min":11.489000000000001,"15min":11.489000000000001},"last":0.66800000000000004}]}]},{"osd":0,"up_from":14,"seq":60129542201,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70184,"kb_used_data":2968,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94301656,"statfs":{"total":96636764160,"available":96564895744,"internally_reserved":0,"allocated":3039232,"data_stored":1498239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":68811207},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 21 15:02:07 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.64000000000000001,"15min":0.70599999999999996},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.002,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.69499999999999995,"15min":0.63200000000000001},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.98699999999999999,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.78100000000000003}]},{"osd":2,"last update":"Sat Sep 21 15:02:07 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.66100000000000003,"15min":0.621},"min":{"1min":0.33900000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.0960000000000001,"5min":1.153,"15min":1.153},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.70099999999999996,"15min":0.75},"min":{"1min":0.28799999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.0329999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.748}]},{"osd":3,"last update":"Sat Sep 21 15:02:07 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.76400000000000001,"15min":0.754},"min":{"1min":0.37,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.77400000000000002,"15min":0.81100000000000005},"min":{"1min":0.42799999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.04,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.54900000000000004}]},{"osd":4,"last update":"Sat Sep 21 15:01:04 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.98399999999999999,"15min":1.038},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.1140000000000001,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":1.069},{"interface":"front","average":{"1min":1.036,"5min":1.0469999999999999,"15min":1.0620000000000001},"min":{"1min":0.67800000000000005,"5min":0.379,"15min":0.379},"max":{"1min":1.446,"5min":1.665,"15min":1.665},"last":1.05}]},{"osd":5,"last update":"Sat Sep 21 15:02:07 2024","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.997,"15min":1.0229999999999999},"min":{"1min":0.45800000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.296,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.98699999999999999},{"interface":"front","average":{"1min":0.94299999999999995,"5min":1.0129999999999999,"15min":1.0669999999999999},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.3919999999999999,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":1.1240000000000001}]},{"osd":6,"last update":"Sat Sep 21 15:02:07 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.98799999999999999,"15min":1.089},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.119,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.94799999999999995,"5min":1.038,"15min":1.0600000000000001},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.393,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":1.214}]},{"osd":7,"last update":"Sat Sep 21 15:02:07 2024","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":1.012,"15min":1.05},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.1719999999999999,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":1.0289999999999999},{"interface":"front","average":{"1min":0.88300000000000001,"5min":1.0309999999999999,"15min":1.113},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.28,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":0.95999999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5202,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5202,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7385,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T15:02:34.533 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-21T15:02:34.533 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-09-21T15:02:34.849 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-21T15:02:34.858 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T15:02:34.858 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T15:02:35.158 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-21T15:02:35.162 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-09-21T15:02:35.174 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":381,"stamp":"2024-09-21T15:02:33.847574+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":174155,"num_read_kb":1238336,"num_write":34466,"num_write_kb":943253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3956,"num_bytes_recovered":212783659,"num_keys_recovered":305,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22661,"ondisk_log_size":22661,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":460608,"kb_used_data":23872,"kb_used_omap":19,"kb_used_meta":436588,"kb_avail":754514112,"statfs":{"total":773094113280,"available":772622450688,"internally_reserved":0,"allocated":24444928,"data_stored":13166724,"data_compressed":10404,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":20129,"internal_metadata":447066463},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.524348"},"pg_stats":[{"pgid":"3.1f","version":"230'598","reported_seq":3979,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:29.452925+0000","last_change":"2024-09-21T15:02:24.311888+0000","last_active":"2024-09-21T15:02:29.452925+0000","last_peered":"2024-09-21T15:02:29.452925+0000","last_clean":"2024-09-21T15:02:29.452925+0000","last_became_active":"2024-09-21T15:02:24.308451+0000","last_became_peered":"2024-09-21T15:02:24.308451+0000","last_unstale":"2024-09-21T15:02:29.452925+0000","last_undegraded":"2024-09-21T15:02:29.452925+0000","last_fullsized":"2024-09-21T15:02:29.452925+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":598,"log_dups_size":0,"ondisk_log_size":598,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:25:13.708671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010410479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3756,"num_read_kb":43219,"num_write":975,"num_write_kb":36436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"230'507","reported_seq":3510,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.335752+0000","last_change":"2024-09-21T15:02:24.312475+0000","last_active":"2024-09-21T15:02:28.335752+0000","last_peered":"2024-09-21T15:02:28.335752+0000","last_clean":"2024-09-21T15:02:28.335752+0000","last_became_active":"2024-09-21T15:02:24.309506+0000","last_became_peered":"2024-09-21T15:02:24.309506+0000","last_unstale":"2024-09-21T15:02:28.335752+0000","last_undegraded":"2024-09-21T15:02:28.335752+0000","last_fullsized":"2024-09-21T15:02:28.335752+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":507,"log_dups_size":0,"ondisk_log_size":507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:49:24.984436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094912299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3420,"num_read_kb":23407,"num_write":643,"num_write_kb":18445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":148,"num_bytes_recovered":11635695,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"228'341","reported_seq":4790,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.957133+0000","last_change":"2024-09-21T15:02:02.409182+0000","last_active":"2024-09-21T15:02:28.957133+0000","last_peered":"2024-09-21T15:02:28.957133+0000","last_clean":"2024-09-21T15:02:28.957133+0000","last_became_active":"2024-09-21T14:58:19.411387+0000","last_became_peered":"2024-09-21T14:58:19.411387+0000","last_unstale":"2024-09-21T15:02:28.957133+0000","last_undegraded":"2024-09-21T15:02:28.957133+0000","last_fullsized":"2024-09-21T15:02:28.957133+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":341,"log_dups_size":0,"ondisk_log_size":341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:37:57.597069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045388299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3878,"num_read_kb":28288,"num_write":535,"num_write_kb":26606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"230'572","reported_seq":4306,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.956805+0000","last_change":"2024-09-21T15:02:02.261134+0000","last_active":"2024-09-21T15:02:28.956805+0000","last_peered":"2024-09-21T15:02:28.956805+0000","last_clean":"2024-09-21T15:02:28.956805+0000","last_became_active":"2024-09-21T14:58:19.411306+0000","last_became_peered":"2024-09-21T14:58:19.411306+0000","last_unstale":"2024-09-21T15:02:28.956805+0000","last_undegraded":"2024-09-21T15:02:28.956805+0000","last_fullsized":"2024-09-21T15:02:28.956805+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":572,"log_dups_size":0,"ondisk_log_size":572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:20:27.079795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011721469999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4050,"num_read_kb":24676,"num_write":863,"num_write_kb":25228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"227'536","reported_seq":4293,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.335194+0000","last_change":"2024-09-21T15:02:17.248373+0000","last_active":"2024-09-21T15:02:28.335194+0000","last_peered":"2024-09-21T15:02:28.335194+0000","last_clean":"2024-09-21T15:02:28.335194+0000","last_became_active":"2024-09-21T15:02:17.242801+0000","last_became_peered":"2024-09-21T15:02:17.242801+0000","last_unstale":"2024-09-21T15:02:28.335194+0000","last_undegraded":"2024-09-21T15:02:28.335194+0000","last_fullsized":"2024-09-21T15:02:28.335194+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":536,"log_dups_size":0,"ondisk_log_size":536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:04:28.325046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032550800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3672,"num_read_kb":32698,"num_write":706,"num_write_kb":22582,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":379,"num_bytes_recovered":13616631,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"227'487","reported_seq":4857,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.956810+0000","last_change":"2024-09-21T15:02:02.262402+0000","last_active":"2024-09-21T15:02:28.956810+0000","last_peered":"2024-09-21T15:02:28.956810+0000","last_clean":"2024-09-21T15:02:28.956810+0000","last_became_active":"2024-09-21T14:58:19.409415+0000","last_became_peered":"2024-09-21T14:58:19.409415+0000","last_unstale":"2024-09-21T15:02:28.956810+0000","last_undegraded":"2024-09-21T15:02:28.956810+0000","last_fullsized":"2024-09-21T15:02:28.956810+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:28:46.336301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019572090000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4389,"num_read_kb":28349,"num_write":704,"num_write_kb":23876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"228'824","reported_seq":8731,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.335092+0000","last_change":"2024-09-21T15:02:18.252228+0000","last_active":"2024-09-21T15:02:28.335092+0000","last_peered":"2024-09-21T15:02:28.335092+0000","last_clean":"2024-09-21T15:02:28.335092+0000","last_became_active":"2024-09-21T15:02:18.246987+0000","last_became_peered":"2024-09-21T15:02:18.246987+0000","last_unstale":"2024-09-21T15:02:28.335092+0000","last_undegraded":"2024-09-21T15:02:28.335092+0000","last_fullsized":"2024-09-21T15:02:28.335092+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":824,"log_dups_size":0,"ondisk_log_size":824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:42:11.072452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014776850000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9383,"num_read_kb":68058,"num_write":1251,"num_write_kb":29921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":394,"num_bytes_recovered":23509073,"num_keys_recovered":66,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"227'688","reported_seq":8793,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.335591+0000","last_change":"2024-09-21T15:02:02.359228+0000","last_active":"2024-09-21T15:02:28.335591+0000","last_peered":"2024-09-21T15:02:28.335591+0000","last_clean":"2024-09-21T15:02:28.335591+0000","last_became_active":"2024-09-21T14:58:19.409371+0000","last_became_peered":"2024-09-21T14:58:19.409371+0000","last_unstale":"2024-09-21T15:02:28.335591+0000","last_undegraded":"2024-09-21T15:02:28.335591+0000","last_fullsized":"2024-09-21T15:02:28.335591+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":688,"log_dups_size":0,"ondisk_log_size":688,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:35:27.143445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052048100000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9236,"num_read_kb":45266,"num_write":998,"num_write_kb":30616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"227'413","reported_seq":4005,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.335834+0000","last_change":"2024-09-21T15:02:02.358947+0000","last_active":"2024-09-21T15:02:28.335834+0000","last_peered":"2024-09-21T15:02:28.335834+0000","last_clean":"2024-09-21T15:02:28.335834+0000","last_became_active":"2024-09-21T14:58:19.410306+0000","last_became_peered":"2024-09-21T14:58:19.410306+0000","last_unstale":"2024-09-21T15:02:28.335834+0000","last_undegraded":"2024-09-21T15:02:28.335834+0000","last_fullsized":"2024-09-21T15:02:28.335834+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":413,"log_dups_size":0,"ondisk_log_size":413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:33:50.089165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065724399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3380,"num_read_kb":28207,"num_write":769,"num_write_kb":34633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"229'1053","reported_seq":6175,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.956985+0000","last_change":"2024-09-21T15:02:02.263447+0000","last_active":"2024-09-21T15:02:28.956985+0000","last_peered":"2024-09-21T15:02:28.956985+0000","last_clean":"2024-09-21T15:02:28.956985+0000","last_became_active":"2024-09-21T15:01:06.166756+0000","last_became_peered":"2024-09-21T15:01:06.166756+0000","last_unstale":"2024-09-21T15:02:28.956985+0000","last_undegraded":"2024-09-21T15:02:28.956985+0000","last_fullsized":"2024-09-21T15:02:28.956985+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":1053,"log_dups_size":0,"ondisk_log_size":1053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:19:04.988784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001157057,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8073,"num_read_kb":43999,"num_write":1621,"num_write_kb":28374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":271,"num_bytes_recovered":16821266,"num_keys_recovered":55,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"230'823","reported_seq":3979,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.339198+0000","last_change":"2024-09-21T15:02:02.276055+0000","last_active":"2024-09-21T15:02:28.339198+0000","last_peered":"2024-09-21T15:02:28.339198+0000","last_clean":"2024-09-21T15:02:28.339198+0000","last_became_active":"2024-09-21T14:58:19.410687+0000","last_became_peered":"2024-09-21T14:58:19.410687+0000","last_unstale":"2024-09-21T15:02:28.339198+0000","last_undegraded":"2024-09-21T15:02:28.339198+0000","last_fullsized":"2024-09-21T15:02:28.339198+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":823,"log_dups_size":0,"ondisk_log_size":823,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:21:42.226982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017043010000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4757,"num_read_kb":49770,"num_write":1348,"num_write_kb":33339,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"228'527","reported_seq":4193,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.955479+0000","last_change":"2024-09-21T15:02:02.408062+0000","last_active":"2024-09-21T15:02:28.955479+0000","last_peered":"2024-09-21T15:02:28.955479+0000","last_clean":"2024-09-21T15:02:28.955479+0000","last_became_active":"2024-09-21T14:58:19.407310+0000","last_became_peered":"2024-09-21T14:58:19.407310+0000","last_unstale":"2024-09-21T15:02:28.955479+0000","last_undegraded":"2024-09-21T15:02:28.955479+0000","last_fullsized":"2024-09-21T15:02:28.955479+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":527,"log_dups_size":0,"ondisk_log_size":527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:47:46.079115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016534340000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3756,"num_read_kb":43322,"num_write":713,"num_write_kb":33471,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":245,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:29.771991+0000","last_change":"2024-09-21T14:58:11.348177+0000","last_active":"2024-09-21T15:02:29.771991+0000","last_peered":"2024-09-21T15:02:29.771991+0000","last_clean":"2024-09-21T15:02:29.771991+0000","last_became_active":"2024-09-21T14:58:09.328417+0000","last_became_peered":"2024-09-21T14:58:09.328417+0000","last_unstale":"2024-09-21T15:02:29.771991+0000","last_undegraded":"2024-09-21T15:02:29.771991+0000","last_fullsized":"2024-09-21T15:02:29.771991+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:52:22.671185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018917490000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"230'610","reported_seq":6567,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.335492+0000","last_change":"2024-09-21T15:02:17.251834+0000","last_active":"2024-09-21T15:02:28.335492+0000","last_peered":"2024-09-21T15:02:28.335492+0000","last_clean":"2024-09-21T15:02:28.335492+0000","last_became_active":"2024-09-21T15:02:17.247583+0000","last_became_peered":"2024-09-21T15:02:17.247583+0000","last_unstale":"2024-09-21T15:02:28.335492+0000","last_undegraded":"2024-09-21T15:02:28.335492+0000","last_fullsized":"2024-09-21T15:02:28.335492+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":610,"log_dups_size":0,"ondisk_log_size":610,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:54:31.703168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013409660000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6297,"num_read_kb":53037,"num_write":697,"num_write_kb":27961,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":11818795,"num_keys_recovered":18,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":24,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.335350+0000","last_change":"2024-09-21T15:02:14.225503+0000","last_active":"2024-09-21T15:02:28.335350+0000","last_peered":"2024-09-21T15:02:28.335350+0000","last_clean":"2024-09-21T15:02:28.335350+0000","last_became_active":"2024-09-21T15:02:14.224646+0000","last_became_peered":"2024-09-21T15:02:14.224646+0000","last_unstale":"2024-09-21T15:02:28.335350+0000","last_undegraded":"2024-09-21T15:02:28.335350+0000","last_fullsized":"2024-09-21T15:02:28.335350+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:57:35.948252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"228'980","reported_seq":4262,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.335626+0000","last_change":"2024-09-21T15:02:19.255988+0000","last_active":"2024-09-21T15:02:28.335626+0000","last_peered":"2024-09-21T15:02:28.335626+0000","last_clean":"2024-09-21T15:02:28.335626+0000","last_became_active":"2024-09-21T15:02:19.251008+0000","last_became_peered":"2024-09-21T15:02:19.251008+0000","last_unstale":"2024-09-21T15:02:28.335626+0000","last_undegraded":"2024-09-21T15:02:28.335626+0000","last_fullsized":"2024-09-21T15:02:28.335626+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":980,"log_dups_size":0,"ondisk_log_size":980,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:13:15.145504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015726799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5449,"num_read_kb":31198,"num_write":1400,"num_write_kb":25281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":401,"num_bytes_recovered":12566413,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":277,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.957358+0000","last_change":"2024-09-21T15:02:14.227927+0000","last_active":"2024-09-21T15:02:28.957358+0000","last_peered":"2024-09-21T15:02:28.957358+0000","last_clean":"2024-09-21T15:02:28.957358+0000","last_became_active":"2024-09-21T15:02:14.227223+0000","last_became_peered":"2024-09-21T15:02:14.227223+0000","last_unstale":"2024-09-21T15:02:28.957358+0000","last_undegraded":"2024-09-21T15:02:28.957358+0000","last_fullsized":"2024-09-21T15:02:28.957358+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:43:37.059225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019338770000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"227'672","reported_seq":5560,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.335126+0000","last_change":"2024-09-21T15:02:18.251535+0000","last_active":"2024-09-21T15:02:28.335126+0000","last_peered":"2024-09-21T15:02:28.335126+0000","last_clean":"2024-09-21T15:02:28.335126+0000","last_became_active":"2024-09-21T15:02:18.246450+0000","last_became_peered":"2024-09-21T15:02:18.246450+0000","last_unstale":"2024-09-21T15:02:28.335126+0000","last_undegraded":"2024-09-21T15:02:28.335126+0000","last_fullsized":"2024-09-21T15:02:28.335126+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":672,"log_dups_size":0,"ondisk_log_size":672,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:04:40.306321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071861500000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5178,"num_read_kb":38459,"num_write":1130,"num_write_kb":41306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":331,"num_bytes_recovered":25796920,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":232,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.335618+0000","last_change":"2024-09-21T15:02:24.351816+0000","last_active":"2024-09-21T15:02:28.335618+0000","last_peered":"2024-09-21T15:02:28.335618+0000","last_clean":"2024-09-21T15:02:28.335618+0000","last_became_active":"2024-09-21T15:02:24.351245+0000","last_became_peered":"2024-09-21T15:02:24.351245+0000","last_unstale":"2024-09-21T15:02:28.335618+0000","last_undegraded":"2024-09-21T15:02:28.335618+0000","last_fullsized":"2024-09-21T15:02:28.335618+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:43:32.690041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"227'732","reported_seq":2860,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:29.354087+0000","last_change":"2024-09-21T15:02:02.262648+0000","last_active":"2024-09-21T15:02:29.354087+0000","last_peered":"2024-09-21T15:02:29.354087+0000","last_clean":"2024-09-21T15:02:29.354087+0000","last_became_active":"2024-09-21T14:58:19.408971+0000","last_became_peered":"2024-09-21T14:58:19.408971+0000","last_unstale":"2024-09-21T15:02:29.354087+0000","last_undegraded":"2024-09-21T15:02:29.354087+0000","last_fullsized":"2024-09-21T15:02:29.354087+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":732,"log_dups_size":0,"ondisk_log_size":732,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:29:52.642121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073528299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3602,"num_read_kb":20655,"num_write":1230,"num_write_kb":20679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"230'1009","reported_seq":3690,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:29.738404+0000","last_change":"2024-09-21T15:02:02.276284+0000","last_active":"2024-09-21T15:02:29.738404+0000","last_peered":"2024-09-21T15:02:29.738404+0000","last_clean":"2024-09-21T15:02:29.738404+0000","last_became_active":"2024-09-21T14:58:19.417453+0000","last_became_peered":"2024-09-21T14:58:19.417453+0000","last_unstale":"2024-09-21T15:02:29.738404+0000","last_undegraded":"2024-09-21T15:02:29.738404+0000","last_fullsized":"2024-09-21T15:02:29.738404+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":1009,"log_dups_size":0,"ondisk_log_size":1009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:19:27.087272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044463100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4845,"num_read_kb":25284,"num_write":1566,"num_write_kb":21735,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":247,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.956823+0000","last_change":"2024-09-21T14:58:11.347504+0000","last_active":"2024-09-21T15:02:28.956823+0000","last_peered":"2024-09-21T15:02:28.956823+0000","last_clean":"2024-09-21T15:02:28.956823+0000","last_became_active":"2024-09-21T14:58:09.775981+0000","last_became_peered":"2024-09-21T14:58:09.775981+0000","last_unstale":"2024-09-21T15:02:28.956823+0000","last_undegraded":"2024-09-21T15:02:28.956823+0000","last_fullsized":"2024-09-21T15:02:28.956823+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:47:16.286652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012568880000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"228'1092","reported_seq":4001,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.335364+0000","last_change":"2024-09-21T15:02:24.314768+0000","last_active":"2024-09-21T15:02:28.335364+0000","last_peered":"2024-09-21T15:02:28.335364+0000","last_clean":"2024-09-21T15:02:28.335364+0000","last_became_active":"2024-09-21T15:02:24.311531+0000","last_became_peered":"2024-09-21T15:02:24.311531+0000","last_unstale":"2024-09-21T15:02:28.335364+0000","last_undegraded":"2024-09-21T15:02:28.335364+0000","last_fullsized":"2024-09-21T15:02:28.335364+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":1092,"log_dups_size":0,"ondisk_log_size":1092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:22:48.528933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00183417,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6343,"num_read_kb":28797,"num_write":1677,"num_write_kb":26974,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":246,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.953692+0000","last_change":"2024-09-21T14:58:11.342210+0000","last_active":"2024-09-21T15:02:28.953692+0000","last_peered":"2024-09-21T15:02:28.953692+0000","last_clean":"2024-09-21T15:02:28.953692+0000","last_became_active":"2024-09-21T14:58:09.330624+0000","last_became_peered":"2024-09-21T14:58:09.330624+0000","last_unstale":"2024-09-21T15:02:28.953692+0000","last_undegraded":"2024-09-21T15:02:28.953692+0000","last_fullsized":"2024-09-21T15:02:28.953692+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:29:43.229538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001051151,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"227'339","reported_seq":2553,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:29.769648+0000","last_change":"2024-09-21T15:02:02.257475+0000","last_active":"2024-09-21T15:02:29.769648+0000","last_peered":"2024-09-21T15:02:29.769648+0000","last_clean":"2024-09-21T15:02:29.769648+0000","last_became_active":"2024-09-21T15:01:53.193166+0000","last_became_peered":"2024-09-21T15:01:53.193166+0000","last_unstale":"2024-09-21T15:02:29.769648+0000","last_undegraded":"2024-09-21T15:02:29.769648+0000","last_fullsized":"2024-09-21T15:02:29.769648+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":339,"log_dups_size":0,"ondisk_log_size":339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:37:52.546654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014983959999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2371,"num_read_kb":40193,"num_write":529,"num_write_kb":39390,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":366,"num_bytes_recovered":22723935,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":245,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.955327+0000","last_change":"2024-09-21T14:58:11.346766+0000","last_active":"2024-09-21T15:02:28.955327+0000","last_peered":"2024-09-21T15:02:28.955327+0000","last_clean":"2024-09-21T15:02:28.955327+0000","last_became_active":"2024-09-21T14:58:09.329748+0000","last_became_peered":"2024-09-21T14:58:09.329748+0000","last_unstale":"2024-09-21T15:02:28.955327+0000","last_undegraded":"2024-09-21T15:02:28.955327+0000","last_fullsized":"2024-09-21T15:02:28.955327+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:31:20.267675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052698000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"230'1275","reported_seq":8379,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.955421+0000","last_change":"2024-09-21T15:02:02.261569+0000","last_active":"2024-09-21T15:02:28.955421+0000","last_peered":"2024-09-21T15:02:28.955421+0000","last_clean":"2024-09-21T15:02:28.955421+0000","last_became_active":"2024-09-21T14:58:41.401003+0000","last_became_peered":"2024-09-21T14:58:41.401003+0000","last_unstale":"2024-09-21T15:02:28.955421+0000","last_undegraded":"2024-09-21T15:02:28.955421+0000","last_fullsized":"2024-09-21T15:02:28.955421+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":1275,"log_dups_size":0,"ondisk_log_size":1275,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:05:37.644835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000746033,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11708,"num_read_kb":50816,"num_write":1955,"num_write_kb":24913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1403348,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":246,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.955718+0000","last_change":"2024-09-21T14:58:11.347700+0000","last_active":"2024-09-21T15:02:28.955718+0000","last_peered":"2024-09-21T15:02:28.955718+0000","last_clean":"2024-09-21T15:02:28.955718+0000","last_became_active":"2024-09-21T14:58:09.438842+0000","last_became_peered":"2024-09-21T14:58:09.438842+0000","last_unstale":"2024-09-21T15:02:28.955718+0000","last_undegraded":"2024-09-21T15:02:28.955718+0000","last_fullsized":"2024-09-21T15:02:28.955718+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:41:34.626249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001426194,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":394,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.957058+0000","last_change":"2024-09-21T14:58:03.616055+0000","last_active":"2024-09-21T15:02:28.957058+0000","last_peered":"2024-09-21T15:02:28.957058+0000","last_clean":"2024-09-21T15:02:28.957058+0000","last_became_active":"2024-09-21T14:58:03.613214+0000","last_became_peered":"2024-09-21T14:58:03.613214+0000","last_unstale":"2024-09-21T15:02:28.957058+0000","last_undegraded":"2024-09-21T15:02:28.957058+0000","last_fullsized":"2024-09-21T15:02:28.957058+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:02.180162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:02.180162+0000","last_clean_scrub_stamp":"2024-09-21T14:58:02.180162+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:29:05.860361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"228'685","reported_seq":4697,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.953055+0000","last_change":"2024-09-21T15:02:02.257763+0000","last_active":"2024-09-21T15:02:28.953055+0000","last_peered":"2024-09-21T15:02:28.953055+0000","last_clean":"2024-09-21T15:02:28.953055+0000","last_became_active":"2024-09-21T15:01:32.177913+0000","last_became_peered":"2024-09-21T15:01:32.177913+0000","last_unstale":"2024-09-21T15:02:28.953055+0000","last_undegraded":"2024-09-21T15:02:28.953055+0000","last_fullsized":"2024-09-21T15:02:28.953055+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":685,"log_dups_size":0,"ondisk_log_size":685,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:47:09.376828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015618730000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4407,"num_read_kb":44538,"num_write":1164,"num_write_kb":30502,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":413,"num_bytes_recovered":20409497,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.9","version":"229'973","reported_seq":4119,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.955874+0000","last_change":"2024-09-21T15:02:18.257627+0000","last_active":"2024-09-21T15:02:28.955874+0000","last_peered":"2024-09-21T15:02:28.955874+0000","last_clean":"2024-09-21T15:02:28.955874+0000","last_became_active":"2024-09-21T15:02:18.255482+0000","last_became_peered":"2024-09-21T15:02:18.255482+0000","last_unstale":"2024-09-21T15:02:28.955874+0000","last_undegraded":"2024-09-21T15:02:28.955874+0000","last_fullsized":"2024-09-21T15:02:28.955874+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":973,"log_dups_size":0,"ondisk_log_size":973,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:15:56.076204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001545105,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4815,"num_read_kb":35700,"num_write":1595,"num_write_kb":33802,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":375,"num_bytes_recovered":14742839,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"229'1027","reported_seq":5029,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.956693+0000","last_change":"2024-09-21T15:02:02.261576+0000","last_active":"2024-09-21T15:02:28.956693+0000","last_peered":"2024-09-21T15:02:28.956693+0000","last_clean":"2024-09-21T15:02:28.956693+0000","last_became_active":"2024-09-21T15:00:27.280971+0000","last_became_peered":"2024-09-21T15:00:27.280971+0000","last_unstale":"2024-09-21T15:02:28.956693+0000","last_undegraded":"2024-09-21T15:02:28.956693+0000","last_fullsized":"2024-09-21T15:02:28.956693+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":1027,"log_dups_size":0,"ondisk_log_size":1027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:26:07.617059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013613779999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7341,"num_read_kb":31141,"num_write":1421,"num_write_kb":33160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":9228123,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"227'593","reported_seq":3447,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.953752+0000","last_change":"2024-09-21T15:02:02.265620+0000","last_active":"2024-09-21T15:02:28.953752+0000","last_peered":"2024-09-21T15:02:28.953752+0000","last_clean":"2024-09-21T15:02:28.953752+0000","last_became_active":"2024-09-21T14:58:19.410679+0000","last_became_peered":"2024-09-21T14:58:19.410679+0000","last_unstale":"2024-09-21T15:02:28.953752+0000","last_undegraded":"2024-09-21T15:02:28.953752+0000","last_fullsized":"2024-09-21T15:02:28.953752+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":593,"log_dups_size":0,"ondisk_log_size":593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:07:40.758932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001466743,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3111,"num_read_kb":36762,"num_write":889,"num_write_kb":39649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"228'493","reported_seq":4709,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.338915+0000","last_change":"2024-09-21T15:02:02.275784+0000","last_active":"2024-09-21T15:02:28.338915+0000","last_peered":"2024-09-21T15:02:28.338915+0000","last_clean":"2024-09-21T15:02:28.338915+0000","last_became_active":"2024-09-21T14:59:25.130057+0000","last_became_peered":"2024-09-21T14:59:25.130057+0000","last_unstale":"2024-09-21T15:02:28.338915+0000","last_undegraded":"2024-09-21T15:02:28.338915+0000","last_fullsized":"2024-09-21T15:02:28.338915+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:53:33.752550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014500649999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4513,"num_read_kb":28786,"num_write":729,"num_write_kb":24616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":9059600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"227'544","reported_seq":5354,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.957258+0000","last_change":"2024-09-21T15:02:02.409235+0000","last_active":"2024-09-21T15:02:28.957258+0000","last_peered":"2024-09-21T15:02:28.957258+0000","last_clean":"2024-09-21T15:02:28.957258+0000","last_became_active":"2024-09-21T14:58:19.407884+0000","last_became_peered":"2024-09-21T14:58:19.407884+0000","last_unstale":"2024-09-21T15:02:28.957258+0000","last_undegraded":"2024-09-21T15:02:28.957258+0000","last_fullsized":"2024-09-21T15:02:28.957258+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":544,"log_dups_size":0,"ondisk_log_size":544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:28:57.111615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045666499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4968,"num_read_kb":55828,"num_write":945,"num_write_kb":38650,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"230'879","reported_seq":3218,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.956772+0000","last_change":"2024-09-21T15:02:02.263156+0000","last_active":"2024-09-21T15:02:28.956772+0000","last_peered":"2024-09-21T15:02:28.956772+0000","last_clean":"2024-09-21T15:02:28.956772+0000","last_became_active":"2024-09-21T15:01:05.172427+0000","last_became_peered":"2024-09-21T15:01:05.172427+0000","last_unstale":"2024-09-21T15:02:28.956772+0000","last_undegraded":"2024-09-21T15:02:28.956772+0000","last_fullsized":"2024-09-21T15:02:28.956772+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":879,"log_dups_size":0,"ondisk_log_size":879,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:28:18.710621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001142789,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4262,"num_read_kb":17288,"num_write":1508,"num_write_kb":17970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":138,"num_bytes_recovered":7486496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"227'532","reported_seq":6209,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.339180+0000","last_change":"2024-09-21T15:02:02.275716+0000","last_active":"2024-09-21T15:02:28.339180+0000","last_peered":"2024-09-21T15:02:28.339180+0000","last_clean":"2024-09-21T15:02:28.339180+0000","last_became_active":"2024-09-21T14:58:19.410406+0000","last_became_peered":"2024-09-21T14:58:19.410406+0000","last_unstale":"2024-09-21T15:02:28.339180+0000","last_undegraded":"2024-09-21T15:02:28.339180+0000","last_fullsized":"2024-09-21T15:02:28.339180+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:59:16.026939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014059350000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5807,"num_read_kb":56273,"num_write":817,"num_write_kb":36773,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"228'523","reported_seq":4647,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.956536+0000","last_change":"2024-09-21T15:02:02.260617+0000","last_active":"2024-09-21T15:02:28.956536+0000","last_peered":"2024-09-21T15:02:28.956536+0000","last_clean":"2024-09-21T15:02:28.956536+0000","last_became_active":"2024-09-21T14:58:19.409935+0000","last_became_peered":"2024-09-21T14:58:19.409935+0000","last_unstale":"2024-09-21T15:02:28.956536+0000","last_undegraded":"2024-09-21T15:02:28.956536+0000","last_fullsized":"2024-09-21T15:02:28.956536+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":523,"log_dups_size":0,"ondisk_log_size":523,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:10:36.252629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068183199999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3949,"num_read_kb":58693,"num_write":810,"num_write_kb":37279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"230'623","reported_seq":5195,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.955624+0000","last_change":"2024-09-21T15:02:02.408072+0000","last_active":"2024-09-21T15:02:28.955624+0000","last_peered":"2024-09-21T15:02:28.955624+0000","last_clean":"2024-09-21T15:02:28.955624+0000","last_became_active":"2024-09-21T15:01:05.165955+0000","last_became_peered":"2024-09-21T15:01:05.165955+0000","last_unstale":"2024-09-21T15:02:28.955624+0000","last_undegraded":"2024-09-21T15:02:28.955624+0000","last_fullsized":"2024-09-21T15:02:28.955624+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":623,"log_dups_size":0,"ondisk_log_size":623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:34:54.111119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021463580000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5758,"num_read_kb":28485,"num_write":777,"num_write_kb":24185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":166,"num_bytes_recovered":11965028,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"229'644","reported_seq":8535,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.953028+0000","last_change":"2024-09-21T15:02:02.256934+0000","last_active":"2024-09-21T15:02:28.953028+0000","last_peered":"2024-09-21T15:02:28.953028+0000","last_clean":"2024-09-21T15:02:28.953028+0000","last_became_active":"2024-09-21T15:00:07.155144+0000","last_became_peered":"2024-09-21T15:00:07.155144+0000","last_unstale":"2024-09-21T15:02:28.953028+0000","last_undegraded":"2024-09-21T15:02:28.953028+0000","last_fullsized":"2024-09-21T15:02:28.953028+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":644,"log_dups_size":0,"ondisk_log_size":644,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:58:00.133639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010987029999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9100,"num_read_kb":37252,"num_write":1013,"num_write_kb":20896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"227'947","reported_seq":6822,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.955476+0000","last_change":"2024-09-21T15:02:02.406594+0000","last_active":"2024-09-21T15:02:28.955476+0000","last_peered":"2024-09-21T15:02:28.955476+0000","last_clean":"2024-09-21T15:02:28.955476+0000","last_became_active":"2024-09-21T14:58:19.412030+0000","last_became_peered":"2024-09-21T14:58:19.412030+0000","last_unstale":"2024-09-21T15:02:28.955476+0000","last_undegraded":"2024-09-21T15:02:28.955476+0000","last_fullsized":"2024-09-21T15:02:28.955476+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":947,"log_dups_size":0,"ondisk_log_size":947,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:44:29.282391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067099300000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8465,"num_read_kb":59792,"num_write":1327,"num_write_kb":31587,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":174039,"num_read_kb":1238236,"num_write":34305,"num_write_kb":940835,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3956,"num_bytes_recovered":212783659,"num_keys_recovered":305,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7385,"internal_metadata":0},"log_size":22541,"ondisk_log_size":22541,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":10404,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509499,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34088,"kb_used_data":2968,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94337752,"statfs":{"total":96636764160,"available":96601858048,"internally_reserved":0,"allocated":3039232,"data_stored":1498239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:02:06 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.83899999999999997,"15min":0.83699999999999997},"min":{"1min":0.29199999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.044,"5min":1.351,"15min":1.351},"last":1.135},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.91400000000000003,"15min":0.94599999999999995},"min":{"1min":0.48099999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.0589999999999999,"5min":1.917,"15min":1.917},"last":1.0009999999999999}]},{"osd":1,"last update":"Sat Sep 21 15:02:06 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.80800000000000005,"15min":0.77600000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":1.0409999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.88},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.81599999999999995,"15min":0.82099999999999995},"min":{"1min":0.40500000000000003,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.002,"5min":2.2149999999999999,"15min":2.2149999999999999},"last":0.47599999999999998}]},{"osd":2,"last update":"Sat Sep 21 15:02:06 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.82899999999999996,"15min":0.81499999999999995},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.161,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.92600000000000005,"15min":0.93999999999999995},"min":{"1min":0.56499999999999995,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.0920000000000001,"5min":1.466,"15min":1.466},"last":0.96799999999999997}]},{"osd":3,"last update":"Sat Sep 21 15:02:06 2024","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.89100000000000001,"15min":0.86299999999999999},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.24,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":1.0669999999999999},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.96799999999999997,"15min":0.96599999999999997},"min":{"1min":0.54200000000000004,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.159,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":1.1919999999999999}]},{"osd":4,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.92200000000000004,"15min":0.93300000000000005},"min":{"1min":0.42199999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.3089999999999999,"5min":2.7130000000000001,"15min":2.7130000000000001},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.94499999999999995,"15min":0.92600000000000005},"min":{"1min":0.39000000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.425,"5min":2.125,"15min":2.125},"last":0.66000000000000003}]},{"osd":5,"last update":"Sat Sep 21 15:02:06 2024","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":1.006,"15min":1.016},"min":{"1min":0.505,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.375,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":1.0389999999999999},{"interface":"front","average":{"1min":0.97699999999999998,"5min":1.026,"15min":1.0449999999999999},"min":{"1min":0.61599999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.425,"5min":3.1880000000000002,"15min":3.1880000000000002},"last":1.1599999999999999}]},{"osd":6,"last update":"Sat Sep 21 15:02:06 2024","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.115,"15min":1.143},"min":{"1min":0.52900000000000003,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.679,"5min":3.2730000000000001,"15min":3.2730000000000001},"last":1.0840000000000001},{"interface":"front","average":{"1min":1.099,"5min":1.0800000000000001,"15min":1.036},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.6719999999999999,"5min":2.2879999999999998,"15min":2.2879999999999998},"last":1.113}]}]},{"osd":6,"up_from":14,"seq":60129542202,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64432,"kb_used_data":2972,"kb_used_omap":1,"kb_used_meta":61438,"kb_avail":94307408,"statfs":{"total":96636764160,"available":96570785792,"internally_reserved":0,"allocated":3043328,"data_stored":1498258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":62912967},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:02:09 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.87,"15min":0.93200000000000005},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.089,"5min":1.593,"15min":1.593},"last":1.222},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.90500000000000003,"15min":0.94099999999999995},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.1180000000000001,"5min":1.587,"15min":1.587},"last":0.70599999999999996}]},{"osd":1,"last update":"Sat Sep 21 15:02:09 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.876,"15min":0.90400000000000003},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.145,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":1.0940000000000001},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.85099999999999998,"15min":0.94499999999999995},"min":{"1min":0.40600000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.036,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":1.121}]},{"osd":2,"last update":"Sat Sep 21 15:02:09 2024","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.93600000000000005,"15min":0.97499999999999998},"min":{"1min":0.53400000000000003,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.194,"5min":1.758,"15min":1.758},"last":1.266},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.878,"15min":0.90200000000000002},"min":{"1min":0.51700000000000002,"5min":0.496,"15min":0.496},"max":{"1min":1.002,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.73599999999999999}]},{"osd":3,"last update":"Sat Sep 21 15:02:09 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.91600000000000004,"15min":0.94799999999999995},"min":{"1min":0.57399999999999995,"5min":0.434,"15min":0.434},"max":{"1min":0.98699999999999999,"5min":1.835,"15min":1.835},"last":1.137},{"interface":"front","average":{"1min":0.90100000000000002,"5min":1.0429999999999999,"15min":1.111},"min":{"1min":0.65900000000000003,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.296,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.92700000000000005}]},{"osd":4,"last update":"Sat Sep 21 15:01:08 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.93500000000000005,"15min":0.99099999999999999},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":1.2,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.98499999999999999,"15min":1.0209999999999999},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.3759999999999999,"5min":1.897,"15min":1.897},"last":0.96999999999999997}]},{"osd":5,"last update":"Sat Sep 21 15:02:09 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.97799999999999998,"15min":1.0880000000000001},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.222,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.85699999999999998,"5min":1.018,"15min":1.071},"min":{"1min":0.54200000000000004,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.3600000000000001,"5min":1.609,"15min":1.609},"last":1.0209999999999999}]},{"osd":7,"last update":"Sat Sep 21 15:02:09 2024","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":1.131,"15min":1.2190000000000001},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.256,"5min":1.8660000000000001,"15min":1.8660000000000001},"last":1.1619999999999999},{"interface":"front","average":{"1min":0.91300000000000003,"5min":1.117,"15min":1.248},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.413,"5min":2.004,"15min":2.004},"last":1.071}]}]},{"osd":5,"up_from":14,"seq":60129542203,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60384,"kb_used_data":3028,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94311456,"statfs":{"total":96636764160,"available":96574930944,"internally_reserved":0,"allocated":3100672,"data_stored":2088626,"data_compressed":5202,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:02:09 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.84199999999999997,"15min":0.96899999999999997},"min":{"1min":0.40100000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.98099999999999998,"5min":4.4800000000000004,"15min":4.4800000000000004},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.78400000000000003,"15min":0.81299999999999994},"min":{"1min":0.44500000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.179,"5min":1.181,"15min":1.181},"last":1.173}]},{"osd":1,"last update":"Sat Sep 21 15:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.86399999999999999,"15min":0.91300000000000003},"min":{"1min":0.41899999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.036,"5min":2.7709999999999999,"15min":2.7709999999999999},"last":1.0049999999999999},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.89900000000000002,"15min":0.92900000000000005},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.1439999999999999,"5min":2.823,"15min":2.823},"last":1.3420000000000001}]},{"osd":2,"last update":"Sat Sep 21 15:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.86599999999999999,"15min":0.95799999999999996},"min":{"1min":0.53600000000000003,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.2090000000000001,"5min":3.419,"15min":3.419},"last":0.754},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.84499999999999997,"15min":0.88800000000000001},"min":{"1min":0.58999999999999997,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.1699999999999999,"5min":1.423,"15min":1.423},"last":0.91600000000000004}]},{"osd":3,"last update":"Sat Sep 21 15:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.81100000000000005,"15min":0.81000000000000005},"min":{"1min":0.51000000000000001,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.105,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.97299999999999998},{"interface":"front","average":{"1min":0.91700000000000004,"5min":1.018,"15min":1.0720000000000001},"min":{"1min":0.66600000000000004,"5min":0.435,"15min":0.435},"max":{"1min":1.4239999999999999,"5min":2.6379999999999999,"15min":2.6379999999999999},"last":1.038}]},{"osd":4,"last update":"Sat Sep 21 15:01:02 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.98899999999999999,"15min":1.016},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.5609999999999999,"5min":2.5470000000000002,"15min":2.5470000000000002},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.82099999999999995,"5min":1.0169999999999999,"15min":1.0780000000000001},"min":{"1min":0.39300000000000002,"5min":0.376,"15min":0.376},"max":{"1min":1.302,"5min":2.5099999999999998,"15min":2.5099999999999998},"last":1.145}]},{"osd":6,"last update":"Sat Sep 21 15:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.91700000000000004,"15min":0.95999999999999996},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.1499999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.879,"5min":1.1399999999999999,"15min":1.3169999999999999},"min":{"1min":0.60199999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.498,"5min":4.2469999999999999,"15min":4.2469999999999999},"last":0.88200000000000001}]},{"osd":7,"last update":"Sat Sep 21 15:02:09 2024","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":1.167,"15min":1.3120000000000001},"min":{"1min":0.61199999999999999,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.5700000000000001,"5min":3.3010000000000002,"15min":3.3010000000000002},"last":1.117},{"interface":"front","average":{"1min":0.99399999999999999,"5min":1.075,"15min":1.125},"min":{"1min":0.64300000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.25,"5min":2.5880000000000001,"15min":2.5880000000000001},"last":1.4430000000000001}]}]},{"osd":4,"up_from":235,"seq":1009317314567,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34860,"kb_used_data":2968,"kb_used_omap":8,"kb_used_meta":31863,"kb_avail":94336980,"statfs":{"total":96636764160,"available":96601067520,"internally_reserved":0,"allocated":3039232,"data_stored":1498239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8979,"internal_metadata":32627949},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0269999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.091}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86899999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92000000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2390000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542203,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78376,"kb_used_data":2972,"kb_used_omap":1,"kb_used_meta":75390,"kb_avail":94293464,"statfs":{"total":96636764160,"available":96556507136,"internally_reserved":0,"allocated":3043328,"data_stored":1498258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":77199816},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.61799999999999999,"15min":0.63200000000000001},"min":{"1min":0.34599999999999997,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.92700000000000005,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.68200000000000005,"15min":0.70999999999999996},"min":{"1min":0.40500000000000003,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.3029999999999999,"5min":1.919,"15min":1.919},"last":0.29999999999999999}]},{"osd":1,"last update":"Sat Sep 21 15:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.65800000000000003,"15min":0.67000000000000004},"min":{"1min":0.44400000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":1.115,"5min":1.722,"15min":1.722},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.746,"15min":0.79700000000000004},"min":{"1min":0.41199999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.2509999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.28499999999999998}]},{"osd":2,"last update":"Sat Sep 21 15:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.746,"15min":0.72999999999999998},"min":{"1min":0.38200000000000001,"5min":0.311,"15min":0.311},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.746,"5min":0.79200000000000004,"15min":0.82099999999999995},"min":{"1min":0.40300000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.94399999999999995,"5min":1.254,"15min":1.254},"last":0.49199999999999999}]},{"osd":4,"last update":"Sat Sep 21 15:01:09 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.93799999999999994,"15min":0.95599999999999996},"min":{"1min":0.54300000000000004,"5min":0.442,"15min":0.442},"max":{"1min":1.2629999999999999,"5min":1.7609999999999999,"15min":1.7609999999999999},"last":1.0600000000000001},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.98899999999999999,"15min":1.0269999999999999},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.234,"5min":2.028,"15min":2.028},"last":0.95399999999999996}]},{"osd":5,"last update":"Sat Sep 21 15:02:13 2024","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.036,"15min":1.052},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.4670000000000001,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":0.76500000000000001},{"interface":"front","average":{"1min":1.081,"5min":1.0589999999999999,"15min":1.0640000000000001},"min":{"1min":0.68799999999999994,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.395,"5min":1.548,"15min":1.548},"last":0.86199999999999999}]},{"osd":6,"last update":"Sat Sep 21 15:02:13 2024","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":1.046,"15min":1.085},"min":{"1min":0.70099999999999996,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.391,"5min":2.1219999999999999,"15min":2.1219999999999999},"last":0.78100000000000003},{"interface":"front","average":{"1min":1.03,"5min":1.075,"15min":1.0980000000000001},"min":{"1min":0.64900000000000002,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.448,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":0.747}]},{"osd":7,"last update":"Sat Sep 21 15:02:13 2024","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.0800000000000001,"15min":1.123},"min":{"1min":0.746,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.339,"5min":1.706,"15min":1.706},"last":0.84899999999999998},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.1279999999999999,"15min":1.171},"min":{"1min":0.67700000000000005,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.421,"5min":2.3380000000000001,"15min":2.3380000000000001},"last":0.89900000000000002}]}]},{"osd":2,"up_from":14,"seq":60129542201,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70180,"kb_used_data":2968,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94301660,"statfs":{"total":96636764160,"available":96564899840,"internally_reserved":0,"allocated":3039232,"data_stored":1498239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68811206},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:02:07 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.63500000000000001,"15min":0.625},"min":{"1min":0.32100000000000001,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.383,"5min":1.383,"15min":1.383},"last":0.439},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.69099999999999995,"15min":0.66900000000000004},"min":{"1min":0.30099999999999999,"5min":0.223,"15min":0.223},"max":{"1min":1.1739999999999999,"5min":2.9169999999999998,"15min":2.9169999999999998},"last":0.28399999999999997}]},{"osd":1,"last update":"Sat Sep 21 15:02:07 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.67500000000000004,"15min":0.68100000000000005},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.129,"5min":2.6520000000000001,"15min":2.6520000000000001},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.73399999999999999,"15min":0.75800000000000001},"min":{"1min":0.36199999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.48899999999999999}]},{"osd":3,"last update":"Sat Sep 21 15:02:07 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.71699999999999997,"15min":0.72699999999999998},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.996,"5min":3.0110000000000001,"15min":3.0110000000000001},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.85999999999999999,"15min":0.83399999999999996},"min":{"1min":0.41099999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0369999999999999,"5min":4.1440000000000001,"15min":4.1440000000000001},"last":0.50600000000000001}]},{"osd":4,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.93200000000000005,"15min":0.95599999999999996},"min":{"1min":0.42399999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.222,"5min":2.899,"15min":2.899},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.94399999999999995,"15min":0.95799999999999996},"min":{"1min":0.47499999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.4379999999999999,"5min":1.675,"15min":1.675},"last":0.58899999999999997}]},{"osd":5,"last update":"Sat Sep 21 15:02:07 2024","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.98699999999999999,"15min":1.0009999999999999},"min":{"1min":0.45500000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.224,"5min":3.1080000000000001,"15min":3.1080000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.96399999999999997,"15min":0.98799999999999999},"min":{"1min":0.55200000000000005,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.228,"5min":1.546,"15min":1.546},"last":0.91300000000000003}]},{"osd":6,"last update":"Sat Sep 21 15:02:07 2024","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":0.98899999999999999,"15min":0.98699999999999999},"min":{"1min":0.52100000000000002,"5min":0.378,"15min":0.378},"max":{"1min":1.516,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.93999999999999995,"5min":1.0429999999999999,"15min":1.048},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.403,"5min":3.0739999999999998,"15min":3.0739999999999998},"last":0.625}]},{"osd":7,"last update":"Sat Sep 21 15:02:09 2024","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.0489999999999999,"15min":1.01},"min":{"1min":0.61399999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.415,"5min":3.1589999999999998,"15min":3.1589999999999998},"last":0.84499999999999997},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.111,"15min":1.095},"min":{"1min":0.58799999999999997,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.4610000000000001,"5min":3.1400000000000001,"15min":3.1400000000000001},"last":0.69799999999999995}]}]},{"osd":1,"up_from":14,"seq":60129542204,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48104,"kb_used_data":3028,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323736,"statfs":{"total":96636764160,"available":96587505664,"internally_reserved":0,"allocated":3100672,"data_stored":2088626,"data_compressed":5202,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.58899999999999997,"15min":0.60699999999999998},"min":{"1min":0.25600000000000001,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.60299999999999998,"15min":0.60999999999999999},"min":{"1min":0.26300000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.83499999999999996,"5min":1.107,"15min":1.107},"last":0.47199999999999998}]},{"osd":2,"last update":"Sat Sep 21 15:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.78000000000000003,"15min":0.73499999999999999},"min":{"1min":0.38800000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.92100000000000004,"5min":11.303000000000001,"15min":11.303000000000001},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.71399999999999997,"15min":0.70299999999999996},"min":{"1min":0.439,"5min":0.435,"15min":0.435},"max":{"1min":1.0840000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.373}]},{"osd":3,"last update":"Sat Sep 21 15:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.80200000000000005,"15min":0.83399999999999996},"min":{"1min":0.36899999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.4850000000000001,"5min":3.1469999999999998,"15min":3.1469999999999998},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.83699999999999997,"15min":0.877},"min":{"1min":0.42499999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.284,"5min":3.2320000000000002,"15min":3.2320000000000002},"last":0.41199999999999998}]},{"osd":4,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.89000000000000001,"15min":0.91500000000000004},"min":{"1min":0.59599999999999997,"5min":0.309,"15min":0.309},"max":{"1min":1.45,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":1.5600000000000001,"5min":1.069,"15min":0.96599999999999997},"min":{"1min":0.504,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":11.563000000000001,"5min":11.563000000000001,"15min":11.563000000000001},"last":0.70299999999999996}]},{"osd":5,"last update":"Sat Sep 21 15:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":1.0620000000000001,"15min":0.95399999999999996},"min":{"1min":0.61199999999999999,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.3799999999999999,"5min":11.534000000000001,"15min":11.534000000000001},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.88,"5min":0.94199999999999995,"15min":0.95599999999999996},"min":{"1min":0.58599999999999997,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.1319999999999999,"5min":1.484,"15min":1.484},"last":0.78300000000000003}]},{"osd":6,"last update":"Sat Sep 21 15:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.97999999999999998,"15min":1.038},"min":{"1min":0.53100000000000003,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.254,"5min":2.3519999999999999,"15min":2.3519999999999999},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.97499999999999998,"5min":1.087,"15min":0.995},"min":{"1min":0.66400000000000003,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.4259999999999999,"5min":11.601000000000001,"15min":11.601000000000001},"last":0.61299999999999999}]},{"osd":7,"last update":"Sat Sep 21 15:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":1.02,"15min":1.056},"min":{"1min":0.58899999999999997,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.4039999999999999,"5min":2.2360000000000002,"15min":2.2360000000000002},"last":0.79600000000000004},{"interface":"front","average":{"1min":1.044,"5min":1.171,"15min":1.087},"min":{"1min":0.63100000000000001,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.3799999999999999,"5min":11.489000000000001,"15min":11.489000000000001},"last":0.66800000000000004}]}]},{"osd":0,"up_from":14,"seq":60129542201,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70184,"kb_used_data":2968,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94301656,"statfs":{"total":96636764160,"available":96564895744,"internally_reserved":0,"allocated":3039232,"data_stored":1498239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":68811207},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 21 15:02:07 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.64000000000000001,"15min":0.70599999999999996},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.002,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.69499999999999995,"15min":0.63200000000000001},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.98699999999999999,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.78100000000000003}]},{"osd":2,"last update":"Sat Sep 21 15:02:07 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.66100000000000003,"15min":0.621},"min":{"1min":0.33900000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.0960000000000001,"5min":1.153,"15min":1.153},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.70099999999999996,"15min":0.75},"min":{"1min":0.28799999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.0329999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.748}]},{"osd":3,"last update":"Sat Sep 21 15:02:07 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.76400000000000001,"15min":0.754},"min":{"1min":0.37,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.77400000000000002,"15min":0.81100000000000005},"min":{"1min":0.42799999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.04,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.54900000000000004}]},{"osd":4,"last update":"Sat Sep 21 15:01:04 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.98399999999999999,"15min":1.038},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.1140000000000001,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":1.069},{"interface":"front","average":{"1min":1.036,"5min":1.0469999999999999,"15min":1.0620000000000001},"min":{"1min":0.67800000000000005,"5min":0.379,"15min":0.379},"max":{"1min":1.446,"5min":1.665,"15min":1.665},"last":1.05}]},{"osd":5,"last update":"Sat Sep 21 15:02:07 2024","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.997,"15min":1.0229999999999999},"min":{"1min":0.45800000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.296,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.98699999999999999},{"interface":"front","average":{"1min":0.94299999999999995,"5min":1.0129999999999999,"15min":1.0669999999999999},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.3919999999999999,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":1.1240000000000001}]},{"osd":6,"last update":"Sat Sep 21 15:02:07 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.98799999999999999,"15min":1.089},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.119,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.94799999999999995,"5min":1.038,"15min":1.0600000000000001},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.393,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":1.214}]},{"osd":7,"last update":"Sat Sep 21 15:02:07 2024","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":1.012,"15min":1.05},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.1719999999999999,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":1.0289999999999999},{"interface":"front","average":{"1min":0.88300000000000001,"5min":1.0309999999999999,"15min":1.113},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.28,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":0.95999999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5202,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5202,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7385,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T15:02:35.177 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T15:02:35.489 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-21T15:02:35.492 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-09-21T15:02:35.504 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":381,"stamp":"2024-09-21T15:02:33.847574+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":174155,"num_read_kb":1238336,"num_write":34466,"num_write_kb":943253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3956,"num_bytes_recovered":212783659,"num_keys_recovered":305,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22661,"ondisk_log_size":22661,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":460608,"kb_used_data":23872,"kb_used_omap":19,"kb_used_meta":436588,"kb_avail":754514112,"statfs":{"total":773094113280,"available":772622450688,"internally_reserved":0,"allocated":24444928,"data_stored":13166724,"data_compressed":10404,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":20129,"internal_metadata":447066463},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.524348"},"pg_stats":[{"pgid":"3.1f","version":"230'598","reported_seq":3979,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:29.452925+0000","last_change":"2024-09-21T15:02:24.311888+0000","last_active":"2024-09-21T15:02:29.452925+0000","last_peered":"2024-09-21T15:02:29.452925+0000","last_clean":"2024-09-21T15:02:29.452925+0000","last_became_active":"2024-09-21T15:02:24.308451+0000","last_became_peered":"2024-09-21T15:02:24.308451+0000","last_unstale":"2024-09-21T15:02:29.452925+0000","last_undegraded":"2024-09-21T15:02:29.452925+0000","last_fullsized":"2024-09-21T15:02:29.452925+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":598,"log_dups_size":0,"ondisk_log_size":598,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:25:13.708671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010410479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3756,"num_read_kb":43219,"num_write":975,"num_write_kb":36436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"230'507","reported_seq":3510,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.335752+0000","last_change":"2024-09-21T15:02:24.312475+0000","last_active":"2024-09-21T15:02:28.335752+0000","last_peered":"2024-09-21T15:02:28.335752+0000","last_clean":"2024-09-21T15:02:28.335752+0000","last_became_active":"2024-09-21T15:02:24.309506+0000","last_became_peered":"2024-09-21T15:02:24.309506+0000","last_unstale":"2024-09-21T15:02:28.335752+0000","last_undegraded":"2024-09-21T15:02:28.335752+0000","last_fullsized":"2024-09-21T15:02:28.335752+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":507,"log_dups_size":0,"ondisk_log_size":507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:49:24.984436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094912299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3420,"num_read_kb":23407,"num_write":643,"num_write_kb":18445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":148,"num_bytes_recovered":11635695,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"228'341","reported_seq":4790,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.957133+0000","last_change":"2024-09-21T15:02:02.409182+0000","last_active":"2024-09-21T15:02:28.957133+0000","last_peered":"2024-09-21T15:02:28.957133+0000","last_clean":"2024-09-21T15:02:28.957133+0000","last_became_active":"2024-09-21T14:58:19.411387+0000","last_became_peered":"2024-09-21T14:58:19.411387+0000","last_unstale":"2024-09-21T15:02:28.957133+0000","last_undegraded":"2024-09-21T15:02:28.957133+0000","last_fullsized":"2024-09-21T15:02:28.957133+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":341,"log_dups_size":0,"ondisk_log_size":341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:37:57.597069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045388299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3878,"num_read_kb":28288,"num_write":535,"num_write_kb":26606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"230'572","reported_seq":4306,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.956805+0000","last_change":"2024-09-21T15:02:02.261134+0000","last_active":"2024-09-21T15:02:28.956805+0000","last_peered":"2024-09-21T15:02:28.956805+0000","last_clean":"2024-09-21T15:02:28.956805+0000","last_became_active":"2024-09-21T14:58:19.411306+0000","last_became_peered":"2024-09-21T14:58:19.411306+0000","last_unstale":"2024-09-21T15:02:28.956805+0000","last_undegraded":"2024-09-21T15:02:28.956805+0000","last_fullsized":"2024-09-21T15:02:28.956805+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":572,"log_dups_size":0,"ondisk_log_size":572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:20:27.079795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011721469999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4050,"num_read_kb":24676,"num_write":863,"num_write_kb":25228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"227'536","reported_seq":4293,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.335194+0000","last_change":"2024-09-21T15:02:17.248373+0000","last_active":"2024-09-21T15:02:28.335194+0000","last_peered":"2024-09-21T15:02:28.335194+0000","last_clean":"2024-09-21T15:02:28.335194+0000","last_became_active":"2024-09-21T15:02:17.242801+0000","last_became_peered":"2024-09-21T15:02:17.242801+0000","last_unstale":"2024-09-21T15:02:28.335194+0000","last_undegraded":"2024-09-21T15:02:28.335194+0000","last_fullsized":"2024-09-21T15:02:28.335194+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":536,"log_dups_size":0,"ondisk_log_size":536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:04:28.325046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032550800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3672,"num_read_kb":32698,"num_write":706,"num_write_kb":22582,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":379,"num_bytes_recovered":13616631,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"227'487","reported_seq":4857,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.956810+0000","last_change":"2024-09-21T15:02:02.262402+0000","last_active":"2024-09-21T15:02:28.956810+0000","last_peered":"2024-09-21T15:02:28.956810+0000","last_clean":"2024-09-21T15:02:28.956810+0000","last_became_active":"2024-09-21T14:58:19.409415+0000","last_became_peered":"2024-09-21T14:58:19.409415+0000","last_unstale":"2024-09-21T15:02:28.956810+0000","last_undegraded":"2024-09-21T15:02:28.956810+0000","last_fullsized":"2024-09-21T15:02:28.956810+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:28:46.336301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019572090000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4389,"num_read_kb":28349,"num_write":704,"num_write_kb":23876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"228'824","reported_seq":8731,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.335092+0000","last_change":"2024-09-21T15:02:18.252228+0000","last_active":"2024-09-21T15:02:28.335092+0000","last_peered":"2024-09-21T15:02:28.335092+0000","last_clean":"2024-09-21T15:02:28.335092+0000","last_became_active":"2024-09-21T15:02:18.246987+0000","last_became_peered":"2024-09-21T15:02:18.246987+0000","last_unstale":"2024-09-21T15:02:28.335092+0000","last_undegraded":"2024-09-21T15:02:28.335092+0000","last_fullsized":"2024-09-21T15:02:28.335092+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":824,"log_dups_size":0,"ondisk_log_size":824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:42:11.072452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014776850000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9383,"num_read_kb":68058,"num_write":1251,"num_write_kb":29921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":394,"num_bytes_recovered":23509073,"num_keys_recovered":66,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"227'688","reported_seq":8793,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.335591+0000","last_change":"2024-09-21T15:02:02.359228+0000","last_active":"2024-09-21T15:02:28.335591+0000","last_peered":"2024-09-21T15:02:28.335591+0000","last_clean":"2024-09-21T15:02:28.335591+0000","last_became_active":"2024-09-21T14:58:19.409371+0000","last_became_peered":"2024-09-21T14:58:19.409371+0000","last_unstale":"2024-09-21T15:02:28.335591+0000","last_undegraded":"2024-09-21T15:02:28.335591+0000","last_fullsized":"2024-09-21T15:02:28.335591+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":688,"log_dups_size":0,"ondisk_log_size":688,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:35:27.143445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052048100000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9236,"num_read_kb":45266,"num_write":998,"num_write_kb":30616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"227'413","reported_seq":4005,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.335834+0000","last_change":"2024-09-21T15:02:02.358947+0000","last_active":"2024-09-21T15:02:28.335834+0000","last_peered":"2024-09-21T15:02:28.335834+0000","last_clean":"2024-09-21T15:02:28.335834+0000","last_became_active":"2024-09-21T14:58:19.410306+0000","last_became_peered":"2024-09-21T14:58:19.410306+0000","last_unstale":"2024-09-21T15:02:28.335834+0000","last_undegraded":"2024-09-21T15:02:28.335834+0000","last_fullsized":"2024-09-21T15:02:28.335834+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":413,"log_dups_size":0,"ondisk_log_size":413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:33:50.089165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065724399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3380,"num_read_kb":28207,"num_write":769,"num_write_kb":34633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"229'1053","reported_seq":6175,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.956985+0000","last_change":"2024-09-21T15:02:02.263447+0000","last_active":"2024-09-21T15:02:28.956985+0000","last_peered":"2024-09-21T15:02:28.956985+0000","last_clean":"2024-09-21T15:02:28.956985+0000","last_became_active":"2024-09-21T15:01:06.166756+0000","last_became_peered":"2024-09-21T15:01:06.166756+0000","last_unstale":"2024-09-21T15:02:28.956985+0000","last_undegraded":"2024-09-21T15:02:28.956985+0000","last_fullsized":"2024-09-21T15:02:28.956985+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":1053,"log_dups_size":0,"ondisk_log_size":1053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:19:04.988784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001157057,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8073,"num_read_kb":43999,"num_write":1621,"num_write_kb":28374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":271,"num_bytes_recovered":16821266,"num_keys_recovered":55,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"230'823","reported_seq":3979,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.339198+0000","last_change":"2024-09-21T15:02:02.276055+0000","last_active":"2024-09-21T15:02:28.339198+0000","last_peered":"2024-09-21T15:02:28.339198+0000","last_clean":"2024-09-21T15:02:28.339198+0000","last_became_active":"2024-09-21T14:58:19.410687+0000","last_became_peered":"2024-09-21T14:58:19.410687+0000","last_unstale":"2024-09-21T15:02:28.339198+0000","last_undegraded":"2024-09-21T15:02:28.339198+0000","last_fullsized":"2024-09-21T15:02:28.339198+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":823,"log_dups_size":0,"ondisk_log_size":823,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:21:42.226982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017043010000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4757,"num_read_kb":49770,"num_write":1348,"num_write_kb":33339,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"228'527","reported_seq":4193,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.955479+0000","last_change":"2024-09-21T15:02:02.408062+0000","last_active":"2024-09-21T15:02:28.955479+0000","last_peered":"2024-09-21T15:02:28.955479+0000","last_clean":"2024-09-21T15:02:28.955479+0000","last_became_active":"2024-09-21T14:58:19.407310+0000","last_became_peered":"2024-09-21T14:58:19.407310+0000","last_unstale":"2024-09-21T15:02:28.955479+0000","last_undegraded":"2024-09-21T15:02:28.955479+0000","last_fullsized":"2024-09-21T15:02:28.955479+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":527,"log_dups_size":0,"ondisk_log_size":527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:47:46.079115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016534340000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3756,"num_read_kb":43322,"num_write":713,"num_write_kb":33471,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":245,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:29.771991+0000","last_change":"2024-09-21T14:58:11.348177+0000","last_active":"2024-09-21T15:02:29.771991+0000","last_peered":"2024-09-21T15:02:29.771991+0000","last_clean":"2024-09-21T15:02:29.771991+0000","last_became_active":"2024-09-21T14:58:09.328417+0000","last_became_peered":"2024-09-21T14:58:09.328417+0000","last_unstale":"2024-09-21T15:02:29.771991+0000","last_undegraded":"2024-09-21T15:02:29.771991+0000","last_fullsized":"2024-09-21T15:02:29.771991+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:52:22.671185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018917490000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"230'610","reported_seq":6567,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.335492+0000","last_change":"2024-09-21T15:02:17.251834+0000","last_active":"2024-09-21T15:02:28.335492+0000","last_peered":"2024-09-21T15:02:28.335492+0000","last_clean":"2024-09-21T15:02:28.335492+0000","last_became_active":"2024-09-21T15:02:17.247583+0000","last_became_peered":"2024-09-21T15:02:17.247583+0000","last_unstale":"2024-09-21T15:02:28.335492+0000","last_undegraded":"2024-09-21T15:02:28.335492+0000","last_fullsized":"2024-09-21T15:02:28.335492+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":610,"log_dups_size":0,"ondisk_log_size":610,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:54:31.703168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013409660000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6297,"num_read_kb":53037,"num_write":697,"num_write_kb":27961,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":11818795,"num_keys_recovered":18,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":24,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.335350+0000","last_change":"2024-09-21T15:02:14.225503+0000","last_active":"2024-09-21T15:02:28.335350+0000","last_peered":"2024-09-21T15:02:28.335350+0000","last_clean":"2024-09-21T15:02:28.335350+0000","last_became_active":"2024-09-21T15:02:14.224646+0000","last_became_peered":"2024-09-21T15:02:14.224646+0000","last_unstale":"2024-09-21T15:02:28.335350+0000","last_undegraded":"2024-09-21T15:02:28.335350+0000","last_fullsized":"2024-09-21T15:02:28.335350+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:57:35.948252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"228'980","reported_seq":4262,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.335626+0000","last_change":"2024-09-21T15:02:19.255988+0000","last_active":"2024-09-21T15:02:28.335626+0000","last_peered":"2024-09-21T15:02:28.335626+0000","last_clean":"2024-09-21T15:02:28.335626+0000","last_became_active":"2024-09-21T15:02:19.251008+0000","last_became_peered":"2024-09-21T15:02:19.251008+0000","last_unstale":"2024-09-21T15:02:28.335626+0000","last_undegraded":"2024-09-21T15:02:28.335626+0000","last_fullsized":"2024-09-21T15:02:28.335626+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":980,"log_dups_size":0,"ondisk_log_size":980,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:13:15.145504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015726799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5449,"num_read_kb":31198,"num_write":1400,"num_write_kb":25281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":401,"num_bytes_recovered":12566413,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":277,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.957358+0000","last_change":"2024-09-21T15:02:14.227927+0000","last_active":"2024-09-21T15:02:28.957358+0000","last_peered":"2024-09-21T15:02:28.957358+0000","last_clean":"2024-09-21T15:02:28.957358+0000","last_became_active":"2024-09-21T15:02:14.227223+0000","last_became_peered":"2024-09-21T15:02:14.227223+0000","last_unstale":"2024-09-21T15:02:28.957358+0000","last_undegraded":"2024-09-21T15:02:28.957358+0000","last_fullsized":"2024-09-21T15:02:28.957358+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:43:37.059225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019338770000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"227'672","reported_seq":5560,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.335126+0000","last_change":"2024-09-21T15:02:18.251535+0000","last_active":"2024-09-21T15:02:28.335126+0000","last_peered":"2024-09-21T15:02:28.335126+0000","last_clean":"2024-09-21T15:02:28.335126+0000","last_became_active":"2024-09-21T15:02:18.246450+0000","last_became_peered":"2024-09-21T15:02:18.246450+0000","last_unstale":"2024-09-21T15:02:28.335126+0000","last_undegraded":"2024-09-21T15:02:28.335126+0000","last_fullsized":"2024-09-21T15:02:28.335126+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":672,"log_dups_size":0,"ondisk_log_size":672,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:04:40.306321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071861500000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5178,"num_read_kb":38459,"num_write":1130,"num_write_kb":41306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":331,"num_bytes_recovered":25796920,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":232,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.335618+0000","last_change":"2024-09-21T15:02:24.351816+0000","last_active":"2024-09-21T15:02:28.335618+0000","last_peered":"2024-09-21T15:02:28.335618+0000","last_clean":"2024-09-21T15:02:28.335618+0000","last_became_active":"2024-09-21T15:02:24.351245+0000","last_became_peered":"2024-09-21T15:02:24.351245+0000","last_unstale":"2024-09-21T15:02:28.335618+0000","last_undegraded":"2024-09-21T15:02:28.335618+0000","last_fullsized":"2024-09-21T15:02:28.335618+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:43:32.690041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"227'732","reported_seq":2860,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:29.354087+0000","last_change":"2024-09-21T15:02:02.262648+0000","last_active":"2024-09-21T15:02:29.354087+0000","last_peered":"2024-09-21T15:02:29.354087+0000","last_clean":"2024-09-21T15:02:29.354087+0000","last_became_active":"2024-09-21T14:58:19.408971+0000","last_became_peered":"2024-09-21T14:58:19.408971+0000","last_unstale":"2024-09-21T15:02:29.354087+0000","last_undegraded":"2024-09-21T15:02:29.354087+0000","last_fullsized":"2024-09-21T15:02:29.354087+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":732,"log_dups_size":0,"ondisk_log_size":732,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:29:52.642121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073528299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3602,"num_read_kb":20655,"num_write":1230,"num_write_kb":20679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"230'1009","reported_seq":3690,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:29.738404+0000","last_change":"2024-09-21T15:02:02.276284+0000","last_active":"2024-09-21T15:02:29.738404+0000","last_peered":"2024-09-21T15:02:29.738404+0000","last_clean":"2024-09-21T15:02:29.738404+0000","last_became_active":"2024-09-21T14:58:19.417453+0000","last_became_peered":"2024-09-21T14:58:19.417453+0000","last_unstale":"2024-09-21T15:02:29.738404+0000","last_undegraded":"2024-09-21T15:02:29.738404+0000","last_fullsized":"2024-09-21T15:02:29.738404+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":1009,"log_dups_size":0,"ondisk_log_size":1009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:19:27.087272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044463100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4845,"num_read_kb":25284,"num_write":1566,"num_write_kb":21735,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":247,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.956823+0000","last_change":"2024-09-21T14:58:11.347504+0000","last_active":"2024-09-21T15:02:28.956823+0000","last_peered":"2024-09-21T15:02:28.956823+0000","last_clean":"2024-09-21T15:02:28.956823+0000","last_became_active":"2024-09-21T14:58:09.775981+0000","last_became_peered":"2024-09-21T14:58:09.775981+0000","last_unstale":"2024-09-21T15:02:28.956823+0000","last_undegraded":"2024-09-21T15:02:28.956823+0000","last_fullsized":"2024-09-21T15:02:28.956823+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:47:16.286652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012568880000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"228'1092","reported_seq":4001,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.335364+0000","last_change":"2024-09-21T15:02:24.314768+0000","last_active":"2024-09-21T15:02:28.335364+0000","last_peered":"2024-09-21T15:02:28.335364+0000","last_clean":"2024-09-21T15:02:28.335364+0000","last_became_active":"2024-09-21T15:02:24.311531+0000","last_became_peered":"2024-09-21T15:02:24.311531+0000","last_unstale":"2024-09-21T15:02:28.335364+0000","last_undegraded":"2024-09-21T15:02:28.335364+0000","last_fullsized":"2024-09-21T15:02:28.335364+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":1092,"log_dups_size":0,"ondisk_log_size":1092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:22:48.528933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00183417,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6343,"num_read_kb":28797,"num_write":1677,"num_write_kb":26974,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":246,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.953692+0000","last_change":"2024-09-21T14:58:11.342210+0000","last_active":"2024-09-21T15:02:28.953692+0000","last_peered":"2024-09-21T15:02:28.953692+0000","last_clean":"2024-09-21T15:02:28.953692+0000","last_became_active":"2024-09-21T14:58:09.330624+0000","last_became_peered":"2024-09-21T14:58:09.330624+0000","last_unstale":"2024-09-21T15:02:28.953692+0000","last_undegraded":"2024-09-21T15:02:28.953692+0000","last_fullsized":"2024-09-21T15:02:28.953692+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:29:43.229538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001051151,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"227'339","reported_seq":2553,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:29.769648+0000","last_change":"2024-09-21T15:02:02.257475+0000","last_active":"2024-09-21T15:02:29.769648+0000","last_peered":"2024-09-21T15:02:29.769648+0000","last_clean":"2024-09-21T15:02:29.769648+0000","last_became_active":"2024-09-21T15:01:53.193166+0000","last_became_peered":"2024-09-21T15:01:53.193166+0000","last_unstale":"2024-09-21T15:02:29.769648+0000","last_undegraded":"2024-09-21T15:02:29.769648+0000","last_fullsized":"2024-09-21T15:02:29.769648+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":339,"log_dups_size":0,"ondisk_log_size":339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:37:52.546654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014983959999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2371,"num_read_kb":40193,"num_write":529,"num_write_kb":39390,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":366,"num_bytes_recovered":22723935,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":245,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.955327+0000","last_change":"2024-09-21T14:58:11.346766+0000","last_active":"2024-09-21T15:02:28.955327+0000","last_peered":"2024-09-21T15:02:28.955327+0000","last_clean":"2024-09-21T15:02:28.955327+0000","last_became_active":"2024-09-21T14:58:09.329748+0000","last_became_peered":"2024-09-21T14:58:09.329748+0000","last_unstale":"2024-09-21T15:02:28.955327+0000","last_undegraded":"2024-09-21T15:02:28.955327+0000","last_fullsized":"2024-09-21T15:02:28.955327+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:31:20.267675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052698000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"230'1275","reported_seq":8379,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.955421+0000","last_change":"2024-09-21T15:02:02.261569+0000","last_active":"2024-09-21T15:02:28.955421+0000","last_peered":"2024-09-21T15:02:28.955421+0000","last_clean":"2024-09-21T15:02:28.955421+0000","last_became_active":"2024-09-21T14:58:41.401003+0000","last_became_peered":"2024-09-21T14:58:41.401003+0000","last_unstale":"2024-09-21T15:02:28.955421+0000","last_undegraded":"2024-09-21T15:02:28.955421+0000","last_fullsized":"2024-09-21T15:02:28.955421+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":1275,"log_dups_size":0,"ondisk_log_size":1275,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:05:37.644835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000746033,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11708,"num_read_kb":50816,"num_write":1955,"num_write_kb":24913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1403348,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":246,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.955718+0000","last_change":"2024-09-21T14:58:11.347700+0000","last_active":"2024-09-21T15:02:28.955718+0000","last_peered":"2024-09-21T15:02:28.955718+0000","last_clean":"2024-09-21T15:02:28.955718+0000","last_became_active":"2024-09-21T14:58:09.438842+0000","last_became_peered":"2024-09-21T14:58:09.438842+0000","last_unstale":"2024-09-21T15:02:28.955718+0000","last_undegraded":"2024-09-21T15:02:28.955718+0000","last_fullsized":"2024-09-21T15:02:28.955718+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:41:34.626249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001426194,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":394,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.957058+0000","last_change":"2024-09-21T14:58:03.616055+0000","last_active":"2024-09-21T15:02:28.957058+0000","last_peered":"2024-09-21T15:02:28.957058+0000","last_clean":"2024-09-21T15:02:28.957058+0000","last_became_active":"2024-09-21T14:58:03.613214+0000","last_became_peered":"2024-09-21T14:58:03.613214+0000","last_unstale":"2024-09-21T15:02:28.957058+0000","last_undegraded":"2024-09-21T15:02:28.957058+0000","last_fullsized":"2024-09-21T15:02:28.957058+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:02.180162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:02.180162+0000","last_clean_scrub_stamp":"2024-09-21T14:58:02.180162+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:29:05.860361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"228'685","reported_seq":4697,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.953055+0000","last_change":"2024-09-21T15:02:02.257763+0000","last_active":"2024-09-21T15:02:28.953055+0000","last_peered":"2024-09-21T15:02:28.953055+0000","last_clean":"2024-09-21T15:02:28.953055+0000","last_became_active":"2024-09-21T15:01:32.177913+0000","last_became_peered":"2024-09-21T15:01:32.177913+0000","last_unstale":"2024-09-21T15:02:28.953055+0000","last_undegraded":"2024-09-21T15:02:28.953055+0000","last_fullsized":"2024-09-21T15:02:28.953055+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":685,"log_dups_size":0,"ondisk_log_size":685,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:47:09.376828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015618730000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4407,"num_read_kb":44538,"num_write":1164,"num_write_kb":30502,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":413,"num_bytes_recovered":20409497,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.9","version":"229'973","reported_seq":4119,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.955874+0000","last_change":"2024-09-21T15:02:18.257627+0000","last_active":"2024-09-21T15:02:28.955874+0000","last_peered":"2024-09-21T15:02:28.955874+0000","last_clean":"2024-09-21T15:02:28.955874+0000","last_became_active":"2024-09-21T15:02:18.255482+0000","last_became_peered":"2024-09-21T15:02:18.255482+0000","last_unstale":"2024-09-21T15:02:28.955874+0000","last_undegraded":"2024-09-21T15:02:28.955874+0000","last_fullsized":"2024-09-21T15:02:28.955874+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":973,"log_dups_size":0,"ondisk_log_size":973,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:15:56.076204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001545105,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4815,"num_read_kb":35700,"num_write":1595,"num_write_kb":33802,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":375,"num_bytes_recovered":14742839,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"229'1027","reported_seq":5029,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.956693+0000","last_change":"2024-09-21T15:02:02.261576+0000","last_active":"2024-09-21T15:02:28.956693+0000","last_peered":"2024-09-21T15:02:28.956693+0000","last_clean":"2024-09-21T15:02:28.956693+0000","last_became_active":"2024-09-21T15:00:27.280971+0000","last_became_peered":"2024-09-21T15:00:27.280971+0000","last_unstale":"2024-09-21T15:02:28.956693+0000","last_undegraded":"2024-09-21T15:02:28.956693+0000","last_fullsized":"2024-09-21T15:02:28.956693+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":1027,"log_dups_size":0,"ondisk_log_size":1027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:26:07.617059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013613779999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7341,"num_read_kb":31141,"num_write":1421,"num_write_kb":33160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":9228123,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"227'593","reported_seq":3447,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.953752+0000","last_change":"2024-09-21T15:02:02.265620+0000","last_active":"2024-09-21T15:02:28.953752+0000","last_peered":"2024-09-21T15:02:28.953752+0000","last_clean":"2024-09-21T15:02:28.953752+0000","last_became_active":"2024-09-21T14:58:19.410679+0000","last_became_peered":"2024-09-21T14:58:19.410679+0000","last_unstale":"2024-09-21T15:02:28.953752+0000","last_undegraded":"2024-09-21T15:02:28.953752+0000","last_fullsized":"2024-09-21T15:02:28.953752+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":593,"log_dups_size":0,"ondisk_log_size":593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:07:40.758932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001466743,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3111,"num_read_kb":36762,"num_write":889,"num_write_kb":39649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"228'493","reported_seq":4709,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.338915+0000","last_change":"2024-09-21T15:02:02.275784+0000","last_active":"2024-09-21T15:02:28.338915+0000","last_peered":"2024-09-21T15:02:28.338915+0000","last_clean":"2024-09-21T15:02:28.338915+0000","last_became_active":"2024-09-21T14:59:25.130057+0000","last_became_peered":"2024-09-21T14:59:25.130057+0000","last_unstale":"2024-09-21T15:02:28.338915+0000","last_undegraded":"2024-09-21T15:02:28.338915+0000","last_fullsized":"2024-09-21T15:02:28.338915+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:53:33.752550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014500649999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4513,"num_read_kb":28786,"num_write":729,"num_write_kb":24616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":9059600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"227'544","reported_seq":5354,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.957258+0000","last_change":"2024-09-21T15:02:02.409235+0000","last_active":"2024-09-21T15:02:28.957258+0000","last_peered":"2024-09-21T15:02:28.957258+0000","last_clean":"2024-09-21T15:02:28.957258+0000","last_became_active":"2024-09-21T14:58:19.407884+0000","last_became_peered":"2024-09-21T14:58:19.407884+0000","last_unstale":"2024-09-21T15:02:28.957258+0000","last_undegraded":"2024-09-21T15:02:28.957258+0000","last_fullsized":"2024-09-21T15:02:28.957258+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":544,"log_dups_size":0,"ondisk_log_size":544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:28:57.111615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045666499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4968,"num_read_kb":55828,"num_write":945,"num_write_kb":38650,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"230'879","reported_seq":3218,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.956772+0000","last_change":"2024-09-21T15:02:02.263156+0000","last_active":"2024-09-21T15:02:28.956772+0000","last_peered":"2024-09-21T15:02:28.956772+0000","last_clean":"2024-09-21T15:02:28.956772+0000","last_became_active":"2024-09-21T15:01:05.172427+0000","last_became_peered":"2024-09-21T15:01:05.172427+0000","last_unstale":"2024-09-21T15:02:28.956772+0000","last_undegraded":"2024-09-21T15:02:28.956772+0000","last_fullsized":"2024-09-21T15:02:28.956772+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":879,"log_dups_size":0,"ondisk_log_size":879,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:28:18.710621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001142789,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4262,"num_read_kb":17288,"num_write":1508,"num_write_kb":17970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":138,"num_bytes_recovered":7486496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"227'532","reported_seq":6209,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.339180+0000","last_change":"2024-09-21T15:02:02.275716+0000","last_active":"2024-09-21T15:02:28.339180+0000","last_peered":"2024-09-21T15:02:28.339180+0000","last_clean":"2024-09-21T15:02:28.339180+0000","last_became_active":"2024-09-21T14:58:19.410406+0000","last_became_peered":"2024-09-21T14:58:19.410406+0000","last_unstale":"2024-09-21T15:02:28.339180+0000","last_undegraded":"2024-09-21T15:02:28.339180+0000","last_fullsized":"2024-09-21T15:02:28.339180+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:59:16.026939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014059350000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5807,"num_read_kb":56273,"num_write":817,"num_write_kb":36773,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"228'523","reported_seq":4647,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.956536+0000","last_change":"2024-09-21T15:02:02.260617+0000","last_active":"2024-09-21T15:02:28.956536+0000","last_peered":"2024-09-21T15:02:28.956536+0000","last_clean":"2024-09-21T15:02:28.956536+0000","last_became_active":"2024-09-21T14:58:19.409935+0000","last_became_peered":"2024-09-21T14:58:19.409935+0000","last_unstale":"2024-09-21T15:02:28.956536+0000","last_undegraded":"2024-09-21T15:02:28.956536+0000","last_fullsized":"2024-09-21T15:02:28.956536+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":523,"log_dups_size":0,"ondisk_log_size":523,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:10:36.252629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068183199999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3949,"num_read_kb":58693,"num_write":810,"num_write_kb":37279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"230'623","reported_seq":5195,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.955624+0000","last_change":"2024-09-21T15:02:02.408072+0000","last_active":"2024-09-21T15:02:28.955624+0000","last_peered":"2024-09-21T15:02:28.955624+0000","last_clean":"2024-09-21T15:02:28.955624+0000","last_became_active":"2024-09-21T15:01:05.165955+0000","last_became_peered":"2024-09-21T15:01:05.165955+0000","last_unstale":"2024-09-21T15:02:28.955624+0000","last_undegraded":"2024-09-21T15:02:28.955624+0000","last_fullsized":"2024-09-21T15:02:28.955624+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":623,"log_dups_size":0,"ondisk_log_size":623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:34:54.111119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021463580000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5758,"num_read_kb":28485,"num_write":777,"num_write_kb":24185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":166,"num_bytes_recovered":11965028,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"229'644","reported_seq":8535,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.953028+0000","last_change":"2024-09-21T15:02:02.256934+0000","last_active":"2024-09-21T15:02:28.953028+0000","last_peered":"2024-09-21T15:02:28.953028+0000","last_clean":"2024-09-21T15:02:28.953028+0000","last_became_active":"2024-09-21T15:00:07.155144+0000","last_became_peered":"2024-09-21T15:00:07.155144+0000","last_unstale":"2024-09-21T15:02:28.953028+0000","last_undegraded":"2024-09-21T15:02:28.953028+0000","last_fullsized":"2024-09-21T15:02:28.953028+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":644,"log_dups_size":0,"ondisk_log_size":644,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:58:00.133639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010987029999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9100,"num_read_kb":37252,"num_write":1013,"num_write_kb":20896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"227'947","reported_seq":6822,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.955476+0000","last_change":"2024-09-21T15:02:02.406594+0000","last_active":"2024-09-21T15:02:28.955476+0000","last_peered":"2024-09-21T15:02:28.955476+0000","last_clean":"2024-09-21T15:02:28.955476+0000","last_became_active":"2024-09-21T14:58:19.412030+0000","last_became_peered":"2024-09-21T14:58:19.412030+0000","last_unstale":"2024-09-21T15:02:28.955476+0000","last_undegraded":"2024-09-21T15:02:28.955476+0000","last_fullsized":"2024-09-21T15:02:28.955476+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":947,"log_dups_size":0,"ondisk_log_size":947,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:44:29.282391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067099300000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8465,"num_read_kb":59792,"num_write":1327,"num_write_kb":31587,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":174039,"num_read_kb":1238236,"num_write":34305,"num_write_kb":940835,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3956,"num_bytes_recovered":212783659,"num_keys_recovered":305,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7385,"internal_metadata":0},"log_size":22541,"ondisk_log_size":22541,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":10404,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509499,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34088,"kb_used_data":2968,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94337752,"statfs":{"total":96636764160,"available":96601858048,"internally_reserved":0,"allocated":3039232,"data_stored":1498239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:02:06 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.83899999999999997,"15min":0.83699999999999997},"min":{"1min":0.29199999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.044,"5min":1.351,"15min":1.351},"last":1.135},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.91400000000000003,"15min":0.94599999999999995},"min":{"1min":0.48099999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.0589999999999999,"5min":1.917,"15min":1.917},"last":1.0009999999999999}]},{"osd":1,"last update":"Sat Sep 21 15:02:06 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.80800000000000005,"15min":0.77600000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":1.0409999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.88},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.81599999999999995,"15min":0.82099999999999995},"min":{"1min":0.40500000000000003,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.002,"5min":2.2149999999999999,"15min":2.2149999999999999},"last":0.47599999999999998}]},{"osd":2,"last update":"Sat Sep 21 15:02:06 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.82899999999999996,"15min":0.81499999999999995},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.161,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.92600000000000005,"15min":0.93999999999999995},"min":{"1min":0.56499999999999995,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.0920000000000001,"5min":1.466,"15min":1.466},"last":0.96799999999999997}]},{"osd":3,"last update":"Sat Sep 21 15:02:06 2024","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.89100000000000001,"15min":0.86299999999999999},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.24,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":1.0669999999999999},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.96799999999999997,"15min":0.96599999999999997},"min":{"1min":0.54200000000000004,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.159,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":1.1919999999999999}]},{"osd":4,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.92200000000000004,"15min":0.93300000000000005},"min":{"1min":0.42199999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.3089999999999999,"5min":2.7130000000000001,"15min":2.7130000000000001},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.94499999999999995,"15min":0.92600000000000005},"min":{"1min":0.39000000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.425,"5min":2.125,"15min":2.125},"last":0.66000000000000003}]},{"osd":5,"last update":"Sat Sep 21 15:02:06 2024","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":1.006,"15min":1.016},"min":{"1min":0.505,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.375,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":1.0389999999999999},{"interface":"front","average":{"1min":0.97699999999999998,"5min":1.026,"15min":1.0449999999999999},"min":{"1min":0.61599999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.425,"5min":3.1880000000000002,"15min":3.1880000000000002},"last":1.1599999999999999}]},{"osd":6,"last update":"Sat Sep 21 15:02:06 2024","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.115,"15min":1.143},"min":{"1min":0.52900000000000003,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.679,"5min":3.2730000000000001,"15min":3.2730000000000001},"last":1.0840000000000001},{"interface":"front","average":{"1min":1.099,"5min":1.0800000000000001,"15min":1.036},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.6719999999999999,"5min":2.2879999999999998,"15min":2.2879999999999998},"last":1.113}]}]},{"osd":6,"up_from":14,"seq":60129542202,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64432,"kb_used_data":2972,"kb_used_omap":1,"kb_used_meta":61438,"kb_avail":94307408,"statfs":{"total":96636764160,"available":96570785792,"internally_reserved":0,"allocated":3043328,"data_stored":1498258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":62912967},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:02:09 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.87,"15min":0.93200000000000005},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.089,"5min":1.593,"15min":1.593},"last":1.222},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.90500000000000003,"15min":0.94099999999999995},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.1180000000000001,"5min":1.587,"15min":1.587},"last":0.70599999999999996}]},{"osd":1,"last update":"Sat Sep 21 15:02:09 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.876,"15min":0.90400000000000003},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.145,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":1.0940000000000001},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.85099999999999998,"15min":0.94499999999999995},"min":{"1min":0.40600000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.036,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":1.121}]},{"osd":2,"last update":"Sat Sep 21 15:02:09 2024","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.93600000000000005,"15min":0.97499999999999998},"min":{"1min":0.53400000000000003,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.194,"5min":1.758,"15min":1.758},"last":1.266},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.878,"15min":0.90200000000000002},"min":{"1min":0.51700000000000002,"5min":0.496,"15min":0.496},"max":{"1min":1.002,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.73599999999999999}]},{"osd":3,"last update":"Sat Sep 21 15:02:09 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.91600000000000004,"15min":0.94799999999999995},"min":{"1min":0.57399999999999995,"5min":0.434,"15min":0.434},"max":{"1min":0.98699999999999999,"5min":1.835,"15min":1.835},"last":1.137},{"interface":"front","average":{"1min":0.90100000000000002,"5min":1.0429999999999999,"15min":1.111},"min":{"1min":0.65900000000000003,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.296,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.92700000000000005}]},{"osd":4,"last update":"Sat Sep 21 15:01:08 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.93500000000000005,"15min":0.99099999999999999},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":1.2,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.98499999999999999,"15min":1.0209999999999999},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.3759999999999999,"5min":1.897,"15min":1.897},"last":0.96999999999999997}]},{"osd":5,"last update":"Sat Sep 21 15:02:09 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.97799999999999998,"15min":1.0880000000000001},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.222,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.85699999999999998,"5min":1.018,"15min":1.071},"min":{"1min":0.54200000000000004,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.3600000000000001,"5min":1.609,"15min":1.609},"last":1.0209999999999999}]},{"osd":7,"last update":"Sat Sep 21 15:02:09 2024","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":1.131,"15min":1.2190000000000001},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.256,"5min":1.8660000000000001,"15min":1.8660000000000001},"last":1.1619999999999999},{"interface":"front","average":{"1min":0.91300000000000003,"5min":1.117,"15min":1.248},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.413,"5min":2.004,"15min":2.004},"last":1.071}]}]},{"osd":5,"up_from":14,"seq":60129542203,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60384,"kb_used_data":3028,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94311456,"statfs":{"total":96636764160,"available":96574930944,"internally_reserved":0,"allocated":3100672,"data_stored":2088626,"data_compressed":5202,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:02:09 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.84199999999999997,"15min":0.96899999999999997},"min":{"1min":0.40100000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.98099999999999998,"5min":4.4800000000000004,"15min":4.4800000000000004},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.78400000000000003,"15min":0.81299999999999994},"min":{"1min":0.44500000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.179,"5min":1.181,"15min":1.181},"last":1.173}]},{"osd":1,"last update":"Sat Sep 21 15:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.86399999999999999,"15min":0.91300000000000003},"min":{"1min":0.41899999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.036,"5min":2.7709999999999999,"15min":2.7709999999999999},"last":1.0049999999999999},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.89900000000000002,"15min":0.92900000000000005},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.1439999999999999,"5min":2.823,"15min":2.823},"last":1.3420000000000001}]},{"osd":2,"last update":"Sat Sep 21 15:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.86599999999999999,"15min":0.95799999999999996},"min":{"1min":0.53600000000000003,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.2090000000000001,"5min":3.419,"15min":3.419},"last":0.754},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.84499999999999997,"15min":0.88800000000000001},"min":{"1min":0.58999999999999997,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.1699999999999999,"5min":1.423,"15min":1.423},"last":0.91600000000000004}]},{"osd":3,"last update":"Sat Sep 21 15:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.81100000000000005,"15min":0.81000000000000005},"min":{"1min":0.51000000000000001,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.105,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.97299999999999998},{"interface":"front","average":{"1min":0.91700000000000004,"5min":1.018,"15min":1.0720000000000001},"min":{"1min":0.66600000000000004,"5min":0.435,"15min":0.435},"max":{"1min":1.4239999999999999,"5min":2.6379999999999999,"15min":2.6379999999999999},"last":1.038}]},{"osd":4,"last update":"Sat Sep 21 15:01:02 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.98899999999999999,"15min":1.016},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.5609999999999999,"5min":2.5470000000000002,"15min":2.5470000000000002},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.82099999999999995,"5min":1.0169999999999999,"15min":1.0780000000000001},"min":{"1min":0.39300000000000002,"5min":0.376,"15min":0.376},"max":{"1min":1.302,"5min":2.5099999999999998,"15min":2.5099999999999998},"last":1.145}]},{"osd":6,"last update":"Sat Sep 21 15:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.91700000000000004,"15min":0.95999999999999996},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.1499999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.879,"5min":1.1399999999999999,"15min":1.3169999999999999},"min":{"1min":0.60199999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.498,"5min":4.2469999999999999,"15min":4.2469999999999999},"last":0.88200000000000001}]},{"osd":7,"last update":"Sat Sep 21 15:02:09 2024","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":1.167,"15min":1.3120000000000001},"min":{"1min":0.61199999999999999,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.5700000000000001,"5min":3.3010000000000002,"15min":3.3010000000000002},"last":1.117},{"interface":"front","average":{"1min":0.99399999999999999,"5min":1.075,"15min":1.125},"min":{"1min":0.64300000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.25,"5min":2.5880000000000001,"15min":2.5880000000000001},"last":1.4430000000000001}]}]},{"osd":4,"up_from":235,"seq":1009317314567,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34860,"kb_used_data":2968,"kb_used_omap":8,"kb_used_meta":31863,"kb_avail":94336980,"statfs":{"total":96636764160,"available":96601067520,"internally_reserved":0,"allocated":3039232,"data_stored":1498239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8979,"internal_metadata":32627949},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0269999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.091}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86899999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92000000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2390000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542203,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78376,"kb_used_data":2972,"kb_used_omap":1,"kb_used_meta":75390,"kb_avail":94293464,"statfs":{"total":96636764160,"available":96556507136,"internally_reserved":0,"allocated":3043328,"data_stored":1498258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":77199816},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.61799999999999999,"15min":0.63200000000000001},"min":{"1min":0.34599999999999997,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.92700000000000005,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.68200000000000005,"15min":0.70999999999999996},"min":{"1min":0.40500000000000003,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.3029999999999999,"5min":1.919,"15min":1.919},"last":0.29999999999999999}]},{"osd":1,"last update":"Sat Sep 21 15:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.65800000000000003,"15min":0.67000000000000004},"min":{"1min":0.44400000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":1.115,"5min":1.722,"15min":1.722},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.746,"15min":0.79700000000000004},"min":{"1min":0.41199999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.2509999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.28499999999999998}]},{"osd":2,"last update":"Sat Sep 21 15:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.746,"15min":0.72999999999999998},"min":{"1min":0.38200000000000001,"5min":0.311,"15min":0.311},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.746,"5min":0.79200000000000004,"15min":0.82099999999999995},"min":{"1min":0.40300000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.94399999999999995,"5min":1.254,"15min":1.254},"last":0.49199999999999999}]},{"osd":4,"last update":"Sat Sep 21 15:01:09 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.93799999999999994,"15min":0.95599999999999996},"min":{"1min":0.54300000000000004,"5min":0.442,"15min":0.442},"max":{"1min":1.2629999999999999,"5min":1.7609999999999999,"15min":1.7609999999999999},"last":1.0600000000000001},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.98899999999999999,"15min":1.0269999999999999},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.234,"5min":2.028,"15min":2.028},"last":0.95399999999999996}]},{"osd":5,"last update":"Sat Sep 21 15:02:13 2024","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.036,"15min":1.052},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.4670000000000001,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":0.76500000000000001},{"interface":"front","average":{"1min":1.081,"5min":1.0589999999999999,"15min":1.0640000000000001},"min":{"1min":0.68799999999999994,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.395,"5min":1.548,"15min":1.548},"last":0.86199999999999999}]},{"osd":6,"last update":"Sat Sep 21 15:02:13 2024","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":1.046,"15min":1.085},"min":{"1min":0.70099999999999996,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.391,"5min":2.1219999999999999,"15min":2.1219999999999999},"last":0.78100000000000003},{"interface":"front","average":{"1min":1.03,"5min":1.075,"15min":1.0980000000000001},"min":{"1min":0.64900000000000002,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.448,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":0.747}]},{"osd":7,"last update":"Sat Sep 21 15:02:13 2024","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.0800000000000001,"15min":1.123},"min":{"1min":0.746,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.339,"5min":1.706,"15min":1.706},"last":0.84899999999999998},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.1279999999999999,"15min":1.171},"min":{"1min":0.67700000000000005,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.421,"5min":2.3380000000000001,"15min":2.3380000000000001},"last":0.89900000000000002}]}]},{"osd":2,"up_from":14,"seq":60129542201,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70180,"kb_used_data":2968,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94301660,"statfs":{"total":96636764160,"available":96564899840,"internally_reserved":0,"allocated":3039232,"data_stored":1498239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68811206},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:02:07 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.63500000000000001,"15min":0.625},"min":{"1min":0.32100000000000001,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.383,"5min":1.383,"15min":1.383},"last":0.439},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.69099999999999995,"15min":0.66900000000000004},"min":{"1min":0.30099999999999999,"5min":0.223,"15min":0.223},"max":{"1min":1.1739999999999999,"5min":2.9169999999999998,"15min":2.9169999999999998},"last":0.28399999999999997}]},{"osd":1,"last update":"Sat Sep 21 15:02:07 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.67500000000000004,"15min":0.68100000000000005},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.129,"5min":2.6520000000000001,"15min":2.6520000000000001},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.73399999999999999,"15min":0.75800000000000001},"min":{"1min":0.36199999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.48899999999999999}]},{"osd":3,"last update":"Sat Sep 21 15:02:07 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.71699999999999997,"15min":0.72699999999999998},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.996,"5min":3.0110000000000001,"15min":3.0110000000000001},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.85999999999999999,"15min":0.83399999999999996},"min":{"1min":0.41099999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0369999999999999,"5min":4.1440000000000001,"15min":4.1440000000000001},"last":0.50600000000000001}]},{"osd":4,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.93200000000000005,"15min":0.95599999999999996},"min":{"1min":0.42399999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.222,"5min":2.899,"15min":2.899},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.94399999999999995,"15min":0.95799999999999996},"min":{"1min":0.47499999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.4379999999999999,"5min":1.675,"15min":1.675},"last":0.58899999999999997}]},{"osd":5,"last update":"Sat Sep 21 15:02:07 2024","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.98699999999999999,"15min":1.0009999999999999},"min":{"1min":0.45500000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.224,"5min":3.1080000000000001,"15min":3.1080000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.96399999999999997,"15min":0.98799999999999999},"min":{"1min":0.55200000000000005,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.228,"5min":1.546,"15min":1.546},"last":0.91300000000000003}]},{"osd":6,"last update":"Sat Sep 21 15:02:07 2024","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":0.98899999999999999,"15min":0.98699999999999999},"min":{"1min":0.52100000000000002,"5min":0.378,"15min":0.378},"max":{"1min":1.516,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.93999999999999995,"5min":1.0429999999999999,"15min":1.048},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.403,"5min":3.0739999999999998,"15min":3.0739999999999998},"last":0.625}]},{"osd":7,"last update":"Sat Sep 21 15:02:09 2024","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.0489999999999999,"15min":1.01},"min":{"1min":0.61399999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.415,"5min":3.1589999999999998,"15min":3.1589999999999998},"last":0.84499999999999997},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.111,"15min":1.095},"min":{"1min":0.58799999999999997,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.4610000000000001,"5min":3.1400000000000001,"15min":3.1400000000000001},"last":0.69799999999999995}]}]},{"osd":1,"up_from":14,"seq":60129542204,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48104,"kb_used_data":3028,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323736,"statfs":{"total":96636764160,"available":96587505664,"internally_reserved":0,"allocated":3100672,"data_stored":2088626,"data_compressed":5202,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.58899999999999997,"15min":0.60699999999999998},"min":{"1min":0.25600000000000001,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.60299999999999998,"15min":0.60999999999999999},"min":{"1min":0.26300000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.83499999999999996,"5min":1.107,"15min":1.107},"last":0.47199999999999998}]},{"osd":2,"last update":"Sat Sep 21 15:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.78000000000000003,"15min":0.73499999999999999},"min":{"1min":0.38800000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.92100000000000004,"5min":11.303000000000001,"15min":11.303000000000001},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.71399999999999997,"15min":0.70299999999999996},"min":{"1min":0.439,"5min":0.435,"15min":0.435},"max":{"1min":1.0840000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.373}]},{"osd":3,"last update":"Sat Sep 21 15:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.80200000000000005,"15min":0.83399999999999996},"min":{"1min":0.36899999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.4850000000000001,"5min":3.1469999999999998,"15min":3.1469999999999998},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.83699999999999997,"15min":0.877},"min":{"1min":0.42499999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.284,"5min":3.2320000000000002,"15min":3.2320000000000002},"last":0.41199999999999998}]},{"osd":4,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.89000000000000001,"15min":0.91500000000000004},"min":{"1min":0.59599999999999997,"5min":0.309,"15min":0.309},"max":{"1min":1.45,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":1.5600000000000001,"5min":1.069,"15min":0.96599999999999997},"min":{"1min":0.504,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":11.563000000000001,"5min":11.563000000000001,"15min":11.563000000000001},"last":0.70299999999999996}]},{"osd":5,"last update":"Sat Sep 21 15:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":1.0620000000000001,"15min":0.95399999999999996},"min":{"1min":0.61199999999999999,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.3799999999999999,"5min":11.534000000000001,"15min":11.534000000000001},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.88,"5min":0.94199999999999995,"15min":0.95599999999999996},"min":{"1min":0.58599999999999997,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.1319999999999999,"5min":1.484,"15min":1.484},"last":0.78300000000000003}]},{"osd":6,"last update":"Sat Sep 21 15:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.97999999999999998,"15min":1.038},"min":{"1min":0.53100000000000003,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.254,"5min":2.3519999999999999,"15min":2.3519999999999999},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.97499999999999998,"5min":1.087,"15min":0.995},"min":{"1min":0.66400000000000003,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.4259999999999999,"5min":11.601000000000001,"15min":11.601000000000001},"last":0.61299999999999999}]},{"osd":7,"last update":"Sat Sep 21 15:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":1.02,"15min":1.056},"min":{"1min":0.58899999999999997,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.4039999999999999,"5min":2.2360000000000002,"15min":2.2360000000000002},"last":0.79600000000000004},{"interface":"front","average":{"1min":1.044,"5min":1.171,"15min":1.087},"min":{"1min":0.63100000000000001,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.3799999999999999,"5min":11.489000000000001,"15min":11.489000000000001},"last":0.66800000000000004}]}]},{"osd":0,"up_from":14,"seq":60129542201,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70184,"kb_used_data":2968,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94301656,"statfs":{"total":96636764160,"available":96564895744,"internally_reserved":0,"allocated":3039232,"data_stored":1498239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":68811207},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 21 15:02:07 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.64000000000000001,"15min":0.70599999999999996},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.002,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.69499999999999995,"15min":0.63200000000000001},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.98699999999999999,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.78100000000000003}]},{"osd":2,"last update":"Sat Sep 21 15:02:07 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.66100000000000003,"15min":0.621},"min":{"1min":0.33900000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.0960000000000001,"5min":1.153,"15min":1.153},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.70099999999999996,"15min":0.75},"min":{"1min":0.28799999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.0329999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.748}]},{"osd":3,"last update":"Sat Sep 21 15:02:07 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.76400000000000001,"15min":0.754},"min":{"1min":0.37,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.77400000000000002,"15min":0.81100000000000005},"min":{"1min":0.42799999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.04,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.54900000000000004}]},{"osd":4,"last update":"Sat Sep 21 15:01:04 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.98399999999999999,"15min":1.038},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.1140000000000001,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":1.069},{"interface":"front","average":{"1min":1.036,"5min":1.0469999999999999,"15min":1.0620000000000001},"min":{"1min":0.67800000000000005,"5min":0.379,"15min":0.379},"max":{"1min":1.446,"5min":1.665,"15min":1.665},"last":1.05}]},{"osd":5,"last update":"Sat Sep 21 15:02:07 2024","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.997,"15min":1.0229999999999999},"min":{"1min":0.45800000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.296,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.98699999999999999},{"interface":"front","average":{"1min":0.94299999999999995,"5min":1.0129999999999999,"15min":1.0669999999999999},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.3919999999999999,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":1.1240000000000001}]},{"osd":6,"last update":"Sat Sep 21 15:02:07 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.98799999999999999,"15min":1.089},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.119,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.94799999999999995,"5min":1.038,"15min":1.0600000000000001},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.393,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":1.214}]},{"osd":7,"last update":"Sat Sep 21 15:02:07 2024","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":1.012,"15min":1.05},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.1719999999999999,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":1.0289999999999999},{"interface":"front","average":{"1min":0.88300000000000001,"5min":1.0309999999999999,"15min":1.113},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.28,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":0.95999999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5202,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5202,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7385,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T15:02:35.507 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T15:02:35.507 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T15:02:35.826 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-21T15:02:35.829 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-09-21T15:02:35.840 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":381,"stamp":"2024-09-21T15:02:33.847574+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":174155,"num_read_kb":1238336,"num_write":34466,"num_write_kb":943253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3956,"num_bytes_recovered":212783659,"num_keys_recovered":305,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22661,"ondisk_log_size":22661,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":460608,"kb_used_data":23872,"kb_used_omap":19,"kb_used_meta":436588,"kb_avail":754514112,"statfs":{"total":773094113280,"available":772622450688,"internally_reserved":0,"allocated":24444928,"data_stored":13166724,"data_compressed":10404,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":20129,"internal_metadata":447066463},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.524348"},"pg_stats":[{"pgid":"3.1f","version":"230'598","reported_seq":3979,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:29.452925+0000","last_change":"2024-09-21T15:02:24.311888+0000","last_active":"2024-09-21T15:02:29.452925+0000","last_peered":"2024-09-21T15:02:29.452925+0000","last_clean":"2024-09-21T15:02:29.452925+0000","last_became_active":"2024-09-21T15:02:24.308451+0000","last_became_peered":"2024-09-21T15:02:24.308451+0000","last_unstale":"2024-09-21T15:02:29.452925+0000","last_undegraded":"2024-09-21T15:02:29.452925+0000","last_fullsized":"2024-09-21T15:02:29.452925+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":598,"log_dups_size":0,"ondisk_log_size":598,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:25:13.708671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010410479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3756,"num_read_kb":43219,"num_write":975,"num_write_kb":36436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"230'507","reported_seq":3510,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.335752+0000","last_change":"2024-09-21T15:02:24.312475+0000","last_active":"2024-09-21T15:02:28.335752+0000","last_peered":"2024-09-21T15:02:28.335752+0000","last_clean":"2024-09-21T15:02:28.335752+0000","last_became_active":"2024-09-21T15:02:24.309506+0000","last_became_peered":"2024-09-21T15:02:24.309506+0000","last_unstale":"2024-09-21T15:02:28.335752+0000","last_undegraded":"2024-09-21T15:02:28.335752+0000","last_fullsized":"2024-09-21T15:02:28.335752+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":507,"log_dups_size":0,"ondisk_log_size":507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:49:24.984436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094912299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3420,"num_read_kb":23407,"num_write":643,"num_write_kb":18445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":148,"num_bytes_recovered":11635695,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"228'341","reported_seq":4790,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.957133+0000","last_change":"2024-09-21T15:02:02.409182+0000","last_active":"2024-09-21T15:02:28.957133+0000","last_peered":"2024-09-21T15:02:28.957133+0000","last_clean":"2024-09-21T15:02:28.957133+0000","last_became_active":"2024-09-21T14:58:19.411387+0000","last_became_peered":"2024-09-21T14:58:19.411387+0000","last_unstale":"2024-09-21T15:02:28.957133+0000","last_undegraded":"2024-09-21T15:02:28.957133+0000","last_fullsized":"2024-09-21T15:02:28.957133+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":341,"log_dups_size":0,"ondisk_log_size":341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:37:57.597069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045388299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3878,"num_read_kb":28288,"num_write":535,"num_write_kb":26606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"230'572","reported_seq":4306,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.956805+0000","last_change":"2024-09-21T15:02:02.261134+0000","last_active":"2024-09-21T15:02:28.956805+0000","last_peered":"2024-09-21T15:02:28.956805+0000","last_clean":"2024-09-21T15:02:28.956805+0000","last_became_active":"2024-09-21T14:58:19.411306+0000","last_became_peered":"2024-09-21T14:58:19.411306+0000","last_unstale":"2024-09-21T15:02:28.956805+0000","last_undegraded":"2024-09-21T15:02:28.956805+0000","last_fullsized":"2024-09-21T15:02:28.956805+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":572,"log_dups_size":0,"ondisk_log_size":572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:20:27.079795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011721469999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4050,"num_read_kb":24676,"num_write":863,"num_write_kb":25228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"227'536","reported_seq":4293,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.335194+0000","last_change":"2024-09-21T15:02:17.248373+0000","last_active":"2024-09-21T15:02:28.335194+0000","last_peered":"2024-09-21T15:02:28.335194+0000","last_clean":"2024-09-21T15:02:28.335194+0000","last_became_active":"2024-09-21T15:02:17.242801+0000","last_became_peered":"2024-09-21T15:02:17.242801+0000","last_unstale":"2024-09-21T15:02:28.335194+0000","last_undegraded":"2024-09-21T15:02:28.335194+0000","last_fullsized":"2024-09-21T15:02:28.335194+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":536,"log_dups_size":0,"ondisk_log_size":536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:04:28.325046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032550800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3672,"num_read_kb":32698,"num_write":706,"num_write_kb":22582,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":379,"num_bytes_recovered":13616631,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"227'487","reported_seq":4857,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.956810+0000","last_change":"2024-09-21T15:02:02.262402+0000","last_active":"2024-09-21T15:02:28.956810+0000","last_peered":"2024-09-21T15:02:28.956810+0000","last_clean":"2024-09-21T15:02:28.956810+0000","last_became_active":"2024-09-21T14:58:19.409415+0000","last_became_peered":"2024-09-21T14:58:19.409415+0000","last_unstale":"2024-09-21T15:02:28.956810+0000","last_undegraded":"2024-09-21T15:02:28.956810+0000","last_fullsized":"2024-09-21T15:02:28.956810+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:28:46.336301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019572090000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4389,"num_read_kb":28349,"num_write":704,"num_write_kb":23876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"228'824","reported_seq":8731,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.335092+0000","last_change":"2024-09-21T15:02:18.252228+0000","last_active":"2024-09-21T15:02:28.335092+0000","last_peered":"2024-09-21T15:02:28.335092+0000","last_clean":"2024-09-21T15:02:28.335092+0000","last_became_active":"2024-09-21T15:02:18.246987+0000","last_became_peered":"2024-09-21T15:02:18.246987+0000","last_unstale":"2024-09-21T15:02:28.335092+0000","last_undegraded":"2024-09-21T15:02:28.335092+0000","last_fullsized":"2024-09-21T15:02:28.335092+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":824,"log_dups_size":0,"ondisk_log_size":824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:42:11.072452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014776850000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9383,"num_read_kb":68058,"num_write":1251,"num_write_kb":29921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":394,"num_bytes_recovered":23509073,"num_keys_recovered":66,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"227'688","reported_seq":8793,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.335591+0000","last_change":"2024-09-21T15:02:02.359228+0000","last_active":"2024-09-21T15:02:28.335591+0000","last_peered":"2024-09-21T15:02:28.335591+0000","last_clean":"2024-09-21T15:02:28.335591+0000","last_became_active":"2024-09-21T14:58:19.409371+0000","last_became_peered":"2024-09-21T14:58:19.409371+0000","last_unstale":"2024-09-21T15:02:28.335591+0000","last_undegraded":"2024-09-21T15:02:28.335591+0000","last_fullsized":"2024-09-21T15:02:28.335591+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":688,"log_dups_size":0,"ondisk_log_size":688,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:35:27.143445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052048100000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9236,"num_read_kb":45266,"num_write":998,"num_write_kb":30616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"227'413","reported_seq":4005,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.335834+0000","last_change":"2024-09-21T15:02:02.358947+0000","last_active":"2024-09-21T15:02:28.335834+0000","last_peered":"2024-09-21T15:02:28.335834+0000","last_clean":"2024-09-21T15:02:28.335834+0000","last_became_active":"2024-09-21T14:58:19.410306+0000","last_became_peered":"2024-09-21T14:58:19.410306+0000","last_unstale":"2024-09-21T15:02:28.335834+0000","last_undegraded":"2024-09-21T15:02:28.335834+0000","last_fullsized":"2024-09-21T15:02:28.335834+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":413,"log_dups_size":0,"ondisk_log_size":413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:33:50.089165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065724399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3380,"num_read_kb":28207,"num_write":769,"num_write_kb":34633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"229'1053","reported_seq":6175,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.956985+0000","last_change":"2024-09-21T15:02:02.263447+0000","last_active":"2024-09-21T15:02:28.956985+0000","last_peered":"2024-09-21T15:02:28.956985+0000","last_clean":"2024-09-21T15:02:28.956985+0000","last_became_active":"2024-09-21T15:01:06.166756+0000","last_became_peered":"2024-09-21T15:01:06.166756+0000","last_unstale":"2024-09-21T15:02:28.956985+0000","last_undegraded":"2024-09-21T15:02:28.956985+0000","last_fullsized":"2024-09-21T15:02:28.956985+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":1053,"log_dups_size":0,"ondisk_log_size":1053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:19:04.988784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001157057,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8073,"num_read_kb":43999,"num_write":1621,"num_write_kb":28374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":271,"num_bytes_recovered":16821266,"num_keys_recovered":55,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"230'823","reported_seq":3979,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.339198+0000","last_change":"2024-09-21T15:02:02.276055+0000","last_active":"2024-09-21T15:02:28.339198+0000","last_peered":"2024-09-21T15:02:28.339198+0000","last_clean":"2024-09-21T15:02:28.339198+0000","last_became_active":"2024-09-21T14:58:19.410687+0000","last_became_peered":"2024-09-21T14:58:19.410687+0000","last_unstale":"2024-09-21T15:02:28.339198+0000","last_undegraded":"2024-09-21T15:02:28.339198+0000","last_fullsized":"2024-09-21T15:02:28.339198+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":823,"log_dups_size":0,"ondisk_log_size":823,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:21:42.226982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017043010000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4757,"num_read_kb":49770,"num_write":1348,"num_write_kb":33339,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"228'527","reported_seq":4193,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.955479+0000","last_change":"2024-09-21T15:02:02.408062+0000","last_active":"2024-09-21T15:02:28.955479+0000","last_peered":"2024-09-21T15:02:28.955479+0000","last_clean":"2024-09-21T15:02:28.955479+0000","last_became_active":"2024-09-21T14:58:19.407310+0000","last_became_peered":"2024-09-21T14:58:19.407310+0000","last_unstale":"2024-09-21T15:02:28.955479+0000","last_undegraded":"2024-09-21T15:02:28.955479+0000","last_fullsized":"2024-09-21T15:02:28.955479+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":527,"log_dups_size":0,"ondisk_log_size":527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:47:46.079115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016534340000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3756,"num_read_kb":43322,"num_write":713,"num_write_kb":33471,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":245,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:29.771991+0000","last_change":"2024-09-21T14:58:11.348177+0000","last_active":"2024-09-21T15:02:29.771991+0000","last_peered":"2024-09-21T15:02:29.771991+0000","last_clean":"2024-09-21T15:02:29.771991+0000","last_became_active":"2024-09-21T14:58:09.328417+0000","last_became_peered":"2024-09-21T14:58:09.328417+0000","last_unstale":"2024-09-21T15:02:29.771991+0000","last_undegraded":"2024-09-21T15:02:29.771991+0000","last_fullsized":"2024-09-21T15:02:29.771991+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:52:22.671185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018917490000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"230'610","reported_seq":6567,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.335492+0000","last_change":"2024-09-21T15:02:17.251834+0000","last_active":"2024-09-21T15:02:28.335492+0000","last_peered":"2024-09-21T15:02:28.335492+0000","last_clean":"2024-09-21T15:02:28.335492+0000","last_became_active":"2024-09-21T15:02:17.247583+0000","last_became_peered":"2024-09-21T15:02:17.247583+0000","last_unstale":"2024-09-21T15:02:28.335492+0000","last_undegraded":"2024-09-21T15:02:28.335492+0000","last_fullsized":"2024-09-21T15:02:28.335492+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":610,"log_dups_size":0,"ondisk_log_size":610,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:54:31.703168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013409660000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6297,"num_read_kb":53037,"num_write":697,"num_write_kb":27961,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":11818795,"num_keys_recovered":18,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":24,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.335350+0000","last_change":"2024-09-21T15:02:14.225503+0000","last_active":"2024-09-21T15:02:28.335350+0000","last_peered":"2024-09-21T15:02:28.335350+0000","last_clean":"2024-09-21T15:02:28.335350+0000","last_became_active":"2024-09-21T15:02:14.224646+0000","last_became_peered":"2024-09-21T15:02:14.224646+0000","last_unstale":"2024-09-21T15:02:28.335350+0000","last_undegraded":"2024-09-21T15:02:28.335350+0000","last_fullsized":"2024-09-21T15:02:28.335350+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:57:35.948252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"228'980","reported_seq":4262,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.335626+0000","last_change":"2024-09-21T15:02:19.255988+0000","last_active":"2024-09-21T15:02:28.335626+0000","last_peered":"2024-09-21T15:02:28.335626+0000","last_clean":"2024-09-21T15:02:28.335626+0000","last_became_active":"2024-09-21T15:02:19.251008+0000","last_became_peered":"2024-09-21T15:02:19.251008+0000","last_unstale":"2024-09-21T15:02:28.335626+0000","last_undegraded":"2024-09-21T15:02:28.335626+0000","last_fullsized":"2024-09-21T15:02:28.335626+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":980,"log_dups_size":0,"ondisk_log_size":980,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:13:15.145504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015726799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5449,"num_read_kb":31198,"num_write":1400,"num_write_kb":25281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":401,"num_bytes_recovered":12566413,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":277,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.957358+0000","last_change":"2024-09-21T15:02:14.227927+0000","last_active":"2024-09-21T15:02:28.957358+0000","last_peered":"2024-09-21T15:02:28.957358+0000","last_clean":"2024-09-21T15:02:28.957358+0000","last_became_active":"2024-09-21T15:02:14.227223+0000","last_became_peered":"2024-09-21T15:02:14.227223+0000","last_unstale":"2024-09-21T15:02:28.957358+0000","last_undegraded":"2024-09-21T15:02:28.957358+0000","last_fullsized":"2024-09-21T15:02:28.957358+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:43:37.059225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019338770000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"227'672","reported_seq":5560,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.335126+0000","last_change":"2024-09-21T15:02:18.251535+0000","last_active":"2024-09-21T15:02:28.335126+0000","last_peered":"2024-09-21T15:02:28.335126+0000","last_clean":"2024-09-21T15:02:28.335126+0000","last_became_active":"2024-09-21T15:02:18.246450+0000","last_became_peered":"2024-09-21T15:02:18.246450+0000","last_unstale":"2024-09-21T15:02:28.335126+0000","last_undegraded":"2024-09-21T15:02:28.335126+0000","last_fullsized":"2024-09-21T15:02:28.335126+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":672,"log_dups_size":0,"ondisk_log_size":672,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:04:40.306321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071861500000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5178,"num_read_kb":38459,"num_write":1130,"num_write_kb":41306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":331,"num_bytes_recovered":25796920,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":232,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.335618+0000","last_change":"2024-09-21T15:02:24.351816+0000","last_active":"2024-09-21T15:02:28.335618+0000","last_peered":"2024-09-21T15:02:28.335618+0000","last_clean":"2024-09-21T15:02:28.335618+0000","last_became_active":"2024-09-21T15:02:24.351245+0000","last_became_peered":"2024-09-21T15:02:24.351245+0000","last_unstale":"2024-09-21T15:02:28.335618+0000","last_undegraded":"2024-09-21T15:02:28.335618+0000","last_fullsized":"2024-09-21T15:02:28.335618+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:43:32.690041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"227'732","reported_seq":2860,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:29.354087+0000","last_change":"2024-09-21T15:02:02.262648+0000","last_active":"2024-09-21T15:02:29.354087+0000","last_peered":"2024-09-21T15:02:29.354087+0000","last_clean":"2024-09-21T15:02:29.354087+0000","last_became_active":"2024-09-21T14:58:19.408971+0000","last_became_peered":"2024-09-21T14:58:19.408971+0000","last_unstale":"2024-09-21T15:02:29.354087+0000","last_undegraded":"2024-09-21T15:02:29.354087+0000","last_fullsized":"2024-09-21T15:02:29.354087+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":732,"log_dups_size":0,"ondisk_log_size":732,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:29:52.642121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073528299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3602,"num_read_kb":20655,"num_write":1230,"num_write_kb":20679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"230'1009","reported_seq":3690,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:29.738404+0000","last_change":"2024-09-21T15:02:02.276284+0000","last_active":"2024-09-21T15:02:29.738404+0000","last_peered":"2024-09-21T15:02:29.738404+0000","last_clean":"2024-09-21T15:02:29.738404+0000","last_became_active":"2024-09-21T14:58:19.417453+0000","last_became_peered":"2024-09-21T14:58:19.417453+0000","last_unstale":"2024-09-21T15:02:29.738404+0000","last_undegraded":"2024-09-21T15:02:29.738404+0000","last_fullsized":"2024-09-21T15:02:29.738404+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":1009,"log_dups_size":0,"ondisk_log_size":1009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:19:27.087272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044463100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4845,"num_read_kb":25284,"num_write":1566,"num_write_kb":21735,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":247,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.956823+0000","last_change":"2024-09-21T14:58:11.347504+0000","last_active":"2024-09-21T15:02:28.956823+0000","last_peered":"2024-09-21T15:02:28.956823+0000","last_clean":"2024-09-21T15:02:28.956823+0000","last_became_active":"2024-09-21T14:58:09.775981+0000","last_became_peered":"2024-09-21T14:58:09.775981+0000","last_unstale":"2024-09-21T15:02:28.956823+0000","last_undegraded":"2024-09-21T15:02:28.956823+0000","last_fullsized":"2024-09-21T15:02:28.956823+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:47:16.286652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012568880000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"228'1092","reported_seq":4001,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.335364+0000","last_change":"2024-09-21T15:02:24.314768+0000","last_active":"2024-09-21T15:02:28.335364+0000","last_peered":"2024-09-21T15:02:28.335364+0000","last_clean":"2024-09-21T15:02:28.335364+0000","last_became_active":"2024-09-21T15:02:24.311531+0000","last_became_peered":"2024-09-21T15:02:24.311531+0000","last_unstale":"2024-09-21T15:02:28.335364+0000","last_undegraded":"2024-09-21T15:02:28.335364+0000","last_fullsized":"2024-09-21T15:02:28.335364+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":1092,"log_dups_size":0,"ondisk_log_size":1092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:22:48.528933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00183417,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6343,"num_read_kb":28797,"num_write":1677,"num_write_kb":26974,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":246,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.953692+0000","last_change":"2024-09-21T14:58:11.342210+0000","last_active":"2024-09-21T15:02:28.953692+0000","last_peered":"2024-09-21T15:02:28.953692+0000","last_clean":"2024-09-21T15:02:28.953692+0000","last_became_active":"2024-09-21T14:58:09.330624+0000","last_became_peered":"2024-09-21T14:58:09.330624+0000","last_unstale":"2024-09-21T15:02:28.953692+0000","last_undegraded":"2024-09-21T15:02:28.953692+0000","last_fullsized":"2024-09-21T15:02:28.953692+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:29:43.229538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001051151,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"227'339","reported_seq":2553,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:29.769648+0000","last_change":"2024-09-21T15:02:02.257475+0000","last_active":"2024-09-21T15:02:29.769648+0000","last_peered":"2024-09-21T15:02:29.769648+0000","last_clean":"2024-09-21T15:02:29.769648+0000","last_became_active":"2024-09-21T15:01:53.193166+0000","last_became_peered":"2024-09-21T15:01:53.193166+0000","last_unstale":"2024-09-21T15:02:29.769648+0000","last_undegraded":"2024-09-21T15:02:29.769648+0000","last_fullsized":"2024-09-21T15:02:29.769648+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":339,"log_dups_size":0,"ondisk_log_size":339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:37:52.546654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014983959999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2371,"num_read_kb":40193,"num_write":529,"num_write_kb":39390,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":366,"num_bytes_recovered":22723935,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":245,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.955327+0000","last_change":"2024-09-21T14:58:11.346766+0000","last_active":"2024-09-21T15:02:28.955327+0000","last_peered":"2024-09-21T15:02:28.955327+0000","last_clean":"2024-09-21T15:02:28.955327+0000","last_became_active":"2024-09-21T14:58:09.329748+0000","last_became_peered":"2024-09-21T14:58:09.329748+0000","last_unstale":"2024-09-21T15:02:28.955327+0000","last_undegraded":"2024-09-21T15:02:28.955327+0000","last_fullsized":"2024-09-21T15:02:28.955327+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:31:20.267675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052698000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"230'1275","reported_seq":8379,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.955421+0000","last_change":"2024-09-21T15:02:02.261569+0000","last_active":"2024-09-21T15:02:28.955421+0000","last_peered":"2024-09-21T15:02:28.955421+0000","last_clean":"2024-09-21T15:02:28.955421+0000","last_became_active":"2024-09-21T14:58:41.401003+0000","last_became_peered":"2024-09-21T14:58:41.401003+0000","last_unstale":"2024-09-21T15:02:28.955421+0000","last_undegraded":"2024-09-21T15:02:28.955421+0000","last_fullsized":"2024-09-21T15:02:28.955421+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":1275,"log_dups_size":0,"ondisk_log_size":1275,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:05:37.644835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000746033,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11708,"num_read_kb":50816,"num_write":1955,"num_write_kb":24913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1403348,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":246,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.955718+0000","last_change":"2024-09-21T14:58:11.347700+0000","last_active":"2024-09-21T15:02:28.955718+0000","last_peered":"2024-09-21T15:02:28.955718+0000","last_clean":"2024-09-21T15:02:28.955718+0000","last_became_active":"2024-09-21T14:58:09.438842+0000","last_became_peered":"2024-09-21T14:58:09.438842+0000","last_unstale":"2024-09-21T15:02:28.955718+0000","last_undegraded":"2024-09-21T15:02:28.955718+0000","last_fullsized":"2024-09-21T15:02:28.955718+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:41:34.626249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001426194,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":394,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.957058+0000","last_change":"2024-09-21T14:58:03.616055+0000","last_active":"2024-09-21T15:02:28.957058+0000","last_peered":"2024-09-21T15:02:28.957058+0000","last_clean":"2024-09-21T15:02:28.957058+0000","last_became_active":"2024-09-21T14:58:03.613214+0000","last_became_peered":"2024-09-21T14:58:03.613214+0000","last_unstale":"2024-09-21T15:02:28.957058+0000","last_undegraded":"2024-09-21T15:02:28.957058+0000","last_fullsized":"2024-09-21T15:02:28.957058+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:02.180162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:02.180162+0000","last_clean_scrub_stamp":"2024-09-21T14:58:02.180162+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:29:05.860361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"228'685","reported_seq":4697,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.953055+0000","last_change":"2024-09-21T15:02:02.257763+0000","last_active":"2024-09-21T15:02:28.953055+0000","last_peered":"2024-09-21T15:02:28.953055+0000","last_clean":"2024-09-21T15:02:28.953055+0000","last_became_active":"2024-09-21T15:01:32.177913+0000","last_became_peered":"2024-09-21T15:01:32.177913+0000","last_unstale":"2024-09-21T15:02:28.953055+0000","last_undegraded":"2024-09-21T15:02:28.953055+0000","last_fullsized":"2024-09-21T15:02:28.953055+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":685,"log_dups_size":0,"ondisk_log_size":685,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:47:09.376828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015618730000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4407,"num_read_kb":44538,"num_write":1164,"num_write_kb":30502,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":413,"num_bytes_recovered":20409497,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.9","version":"229'973","reported_seq":4119,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.955874+0000","last_change":"2024-09-21T15:02:18.257627+0000","last_active":"2024-09-21T15:02:28.955874+0000","last_peered":"2024-09-21T15:02:28.955874+0000","last_clean":"2024-09-21T15:02:28.955874+0000","last_became_active":"2024-09-21T15:02:18.255482+0000","last_became_peered":"2024-09-21T15:02:18.255482+0000","last_unstale":"2024-09-21T15:02:28.955874+0000","last_undegraded":"2024-09-21T15:02:28.955874+0000","last_fullsized":"2024-09-21T15:02:28.955874+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":973,"log_dups_size":0,"ondisk_log_size":973,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:15:56.076204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001545105,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4815,"num_read_kb":35700,"num_write":1595,"num_write_kb":33802,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":375,"num_bytes_recovered":14742839,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"229'1027","reported_seq":5029,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.956693+0000","last_change":"2024-09-21T15:02:02.261576+0000","last_active":"2024-09-21T15:02:28.956693+0000","last_peered":"2024-09-21T15:02:28.956693+0000","last_clean":"2024-09-21T15:02:28.956693+0000","last_became_active":"2024-09-21T15:00:27.280971+0000","last_became_peered":"2024-09-21T15:00:27.280971+0000","last_unstale":"2024-09-21T15:02:28.956693+0000","last_undegraded":"2024-09-21T15:02:28.956693+0000","last_fullsized":"2024-09-21T15:02:28.956693+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":1027,"log_dups_size":0,"ondisk_log_size":1027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:26:07.617059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013613779999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7341,"num_read_kb":31141,"num_write":1421,"num_write_kb":33160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":9228123,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"227'593","reported_seq":3447,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.953752+0000","last_change":"2024-09-21T15:02:02.265620+0000","last_active":"2024-09-21T15:02:28.953752+0000","last_peered":"2024-09-21T15:02:28.953752+0000","last_clean":"2024-09-21T15:02:28.953752+0000","last_became_active":"2024-09-21T14:58:19.410679+0000","last_became_peered":"2024-09-21T14:58:19.410679+0000","last_unstale":"2024-09-21T15:02:28.953752+0000","last_undegraded":"2024-09-21T15:02:28.953752+0000","last_fullsized":"2024-09-21T15:02:28.953752+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":593,"log_dups_size":0,"ondisk_log_size":593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:07:40.758932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001466743,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3111,"num_read_kb":36762,"num_write":889,"num_write_kb":39649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"228'493","reported_seq":4709,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.338915+0000","last_change":"2024-09-21T15:02:02.275784+0000","last_active":"2024-09-21T15:02:28.338915+0000","last_peered":"2024-09-21T15:02:28.338915+0000","last_clean":"2024-09-21T15:02:28.338915+0000","last_became_active":"2024-09-21T14:59:25.130057+0000","last_became_peered":"2024-09-21T14:59:25.130057+0000","last_unstale":"2024-09-21T15:02:28.338915+0000","last_undegraded":"2024-09-21T15:02:28.338915+0000","last_fullsized":"2024-09-21T15:02:28.338915+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:53:33.752550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014500649999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4513,"num_read_kb":28786,"num_write":729,"num_write_kb":24616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":9059600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"227'544","reported_seq":5354,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.957258+0000","last_change":"2024-09-21T15:02:02.409235+0000","last_active":"2024-09-21T15:02:28.957258+0000","last_peered":"2024-09-21T15:02:28.957258+0000","last_clean":"2024-09-21T15:02:28.957258+0000","last_became_active":"2024-09-21T14:58:19.407884+0000","last_became_peered":"2024-09-21T14:58:19.407884+0000","last_unstale":"2024-09-21T15:02:28.957258+0000","last_undegraded":"2024-09-21T15:02:28.957258+0000","last_fullsized":"2024-09-21T15:02:28.957258+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":544,"log_dups_size":0,"ondisk_log_size":544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:28:57.111615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045666499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4968,"num_read_kb":55828,"num_write":945,"num_write_kb":38650,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"230'879","reported_seq":3218,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.956772+0000","last_change":"2024-09-21T15:02:02.263156+0000","last_active":"2024-09-21T15:02:28.956772+0000","last_peered":"2024-09-21T15:02:28.956772+0000","last_clean":"2024-09-21T15:02:28.956772+0000","last_became_active":"2024-09-21T15:01:05.172427+0000","last_became_peered":"2024-09-21T15:01:05.172427+0000","last_unstale":"2024-09-21T15:02:28.956772+0000","last_undegraded":"2024-09-21T15:02:28.956772+0000","last_fullsized":"2024-09-21T15:02:28.956772+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":879,"log_dups_size":0,"ondisk_log_size":879,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:28:18.710621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001142789,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4262,"num_read_kb":17288,"num_write":1508,"num_write_kb":17970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":138,"num_bytes_recovered":7486496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"227'532","reported_seq":6209,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.339180+0000","last_change":"2024-09-21T15:02:02.275716+0000","last_active":"2024-09-21T15:02:28.339180+0000","last_peered":"2024-09-21T15:02:28.339180+0000","last_clean":"2024-09-21T15:02:28.339180+0000","last_became_active":"2024-09-21T14:58:19.410406+0000","last_became_peered":"2024-09-21T14:58:19.410406+0000","last_unstale":"2024-09-21T15:02:28.339180+0000","last_undegraded":"2024-09-21T15:02:28.339180+0000","last_fullsized":"2024-09-21T15:02:28.339180+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:59:16.026939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014059350000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5807,"num_read_kb":56273,"num_write":817,"num_write_kb":36773,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"228'523","reported_seq":4647,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.956536+0000","last_change":"2024-09-21T15:02:02.260617+0000","last_active":"2024-09-21T15:02:28.956536+0000","last_peered":"2024-09-21T15:02:28.956536+0000","last_clean":"2024-09-21T15:02:28.956536+0000","last_became_active":"2024-09-21T14:58:19.409935+0000","last_became_peered":"2024-09-21T14:58:19.409935+0000","last_unstale":"2024-09-21T15:02:28.956536+0000","last_undegraded":"2024-09-21T15:02:28.956536+0000","last_fullsized":"2024-09-21T15:02:28.956536+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":523,"log_dups_size":0,"ondisk_log_size":523,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:10:36.252629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068183199999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3949,"num_read_kb":58693,"num_write":810,"num_write_kb":37279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"230'623","reported_seq":5195,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.955624+0000","last_change":"2024-09-21T15:02:02.408072+0000","last_active":"2024-09-21T15:02:28.955624+0000","last_peered":"2024-09-21T15:02:28.955624+0000","last_clean":"2024-09-21T15:02:28.955624+0000","last_became_active":"2024-09-21T15:01:05.165955+0000","last_became_peered":"2024-09-21T15:01:05.165955+0000","last_unstale":"2024-09-21T15:02:28.955624+0000","last_undegraded":"2024-09-21T15:02:28.955624+0000","last_fullsized":"2024-09-21T15:02:28.955624+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":623,"log_dups_size":0,"ondisk_log_size":623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:34:54.111119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021463580000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5758,"num_read_kb":28485,"num_write":777,"num_write_kb":24185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":166,"num_bytes_recovered":11965028,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"229'644","reported_seq":8535,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.953028+0000","last_change":"2024-09-21T15:02:02.256934+0000","last_active":"2024-09-21T15:02:28.953028+0000","last_peered":"2024-09-21T15:02:28.953028+0000","last_clean":"2024-09-21T15:02:28.953028+0000","last_became_active":"2024-09-21T15:00:07.155144+0000","last_became_peered":"2024-09-21T15:00:07.155144+0000","last_unstale":"2024-09-21T15:02:28.953028+0000","last_undegraded":"2024-09-21T15:02:28.953028+0000","last_fullsized":"2024-09-21T15:02:28.953028+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":644,"log_dups_size":0,"ondisk_log_size":644,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:58:00.133639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010987029999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9100,"num_read_kb":37252,"num_write":1013,"num_write_kb":20896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"227'947","reported_seq":6822,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.955476+0000","last_change":"2024-09-21T15:02:02.406594+0000","last_active":"2024-09-21T15:02:28.955476+0000","last_peered":"2024-09-21T15:02:28.955476+0000","last_clean":"2024-09-21T15:02:28.955476+0000","last_became_active":"2024-09-21T14:58:19.412030+0000","last_became_peered":"2024-09-21T14:58:19.412030+0000","last_unstale":"2024-09-21T15:02:28.955476+0000","last_undegraded":"2024-09-21T15:02:28.955476+0000","last_fullsized":"2024-09-21T15:02:28.955476+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":947,"log_dups_size":0,"ondisk_log_size":947,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:44:29.282391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067099300000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8465,"num_read_kb":59792,"num_write":1327,"num_write_kb":31587,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":174039,"num_read_kb":1238236,"num_write":34305,"num_write_kb":940835,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3956,"num_bytes_recovered":212783659,"num_keys_recovered":305,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7385,"internal_metadata":0},"log_size":22541,"ondisk_log_size":22541,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":10404,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509499,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34088,"kb_used_data":2968,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94337752,"statfs":{"total":96636764160,"available":96601858048,"internally_reserved":0,"allocated":3039232,"data_stored":1498239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:02:06 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.83899999999999997,"15min":0.83699999999999997},"min":{"1min":0.29199999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.044,"5min":1.351,"15min":1.351},"last":1.135},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.91400000000000003,"15min":0.94599999999999995},"min":{"1min":0.48099999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.0589999999999999,"5min":1.917,"15min":1.917},"last":1.0009999999999999}]},{"osd":1,"last update":"Sat Sep 21 15:02:06 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.80800000000000005,"15min":0.77600000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":1.0409999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.88},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.81599999999999995,"15min":0.82099999999999995},"min":{"1min":0.40500000000000003,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.002,"5min":2.2149999999999999,"15min":2.2149999999999999},"last":0.47599999999999998}]},{"osd":2,"last update":"Sat Sep 21 15:02:06 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.82899999999999996,"15min":0.81499999999999995},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.161,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.92600000000000005,"15min":0.93999999999999995},"min":{"1min":0.56499999999999995,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.0920000000000001,"5min":1.466,"15min":1.466},"last":0.96799999999999997}]},{"osd":3,"last update":"Sat Sep 21 15:02:06 2024","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.89100000000000001,"15min":0.86299999999999999},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.24,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":1.0669999999999999},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.96799999999999997,"15min":0.96599999999999997},"min":{"1min":0.54200000000000004,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.159,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":1.1919999999999999}]},{"osd":4,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.92200000000000004,"15min":0.93300000000000005},"min":{"1min":0.42199999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.3089999999999999,"5min":2.7130000000000001,"15min":2.7130000000000001},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.94499999999999995,"15min":0.92600000000000005},"min":{"1min":0.39000000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.425,"5min":2.125,"15min":2.125},"last":0.66000000000000003}]},{"osd":5,"last update":"Sat Sep 21 15:02:06 2024","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":1.006,"15min":1.016},"min":{"1min":0.505,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.375,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":1.0389999999999999},{"interface":"front","average":{"1min":0.97699999999999998,"5min":1.026,"15min":1.0449999999999999},"min":{"1min":0.61599999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.425,"5min":3.1880000000000002,"15min":3.1880000000000002},"last":1.1599999999999999}]},{"osd":6,"last update":"Sat Sep 21 15:02:06 2024","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.115,"15min":1.143},"min":{"1min":0.52900000000000003,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.679,"5min":3.2730000000000001,"15min":3.2730000000000001},"last":1.0840000000000001},{"interface":"front","average":{"1min":1.099,"5min":1.0800000000000001,"15min":1.036},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.6719999999999999,"5min":2.2879999999999998,"15min":2.2879999999999998},"last":1.113}]}]},{"osd":6,"up_from":14,"seq":60129542202,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64432,"kb_used_data":2972,"kb_used_omap":1,"kb_used_meta":61438,"kb_avail":94307408,"statfs":{"total":96636764160,"available":96570785792,"internally_reserved":0,"allocated":3043328,"data_stored":1498258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":62912967},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:02:09 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.87,"15min":0.93200000000000005},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.089,"5min":1.593,"15min":1.593},"last":1.222},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.90500000000000003,"15min":0.94099999999999995},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.1180000000000001,"5min":1.587,"15min":1.587},"last":0.70599999999999996}]},{"osd":1,"last update":"Sat Sep 21 15:02:09 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.876,"15min":0.90400000000000003},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.145,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":1.0940000000000001},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.85099999999999998,"15min":0.94499999999999995},"min":{"1min":0.40600000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.036,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":1.121}]},{"osd":2,"last update":"Sat Sep 21 15:02:09 2024","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.93600000000000005,"15min":0.97499999999999998},"min":{"1min":0.53400000000000003,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.194,"5min":1.758,"15min":1.758},"last":1.266},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.878,"15min":0.90200000000000002},"min":{"1min":0.51700000000000002,"5min":0.496,"15min":0.496},"max":{"1min":1.002,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.73599999999999999}]},{"osd":3,"last update":"Sat Sep 21 15:02:09 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.91600000000000004,"15min":0.94799999999999995},"min":{"1min":0.57399999999999995,"5min":0.434,"15min":0.434},"max":{"1min":0.98699999999999999,"5min":1.835,"15min":1.835},"last":1.137},{"interface":"front","average":{"1min":0.90100000000000002,"5min":1.0429999999999999,"15min":1.111},"min":{"1min":0.65900000000000003,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.296,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.92700000000000005}]},{"osd":4,"last update":"Sat Sep 21 15:01:08 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.93500000000000005,"15min":0.99099999999999999},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":1.2,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.98499999999999999,"15min":1.0209999999999999},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.3759999999999999,"5min":1.897,"15min":1.897},"last":0.96999999999999997}]},{"osd":5,"last update":"Sat Sep 21 15:02:09 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.97799999999999998,"15min":1.0880000000000001},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.222,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.85699999999999998,"5min":1.018,"15min":1.071},"min":{"1min":0.54200000000000004,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.3600000000000001,"5min":1.609,"15min":1.609},"last":1.0209999999999999}]},{"osd":7,"last update":"Sat Sep 21 15:02:09 2024","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":1.131,"15min":1.2190000000000001},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.256,"5min":1.8660000000000001,"15min":1.8660000000000001},"last":1.1619999999999999},{"interface":"front","average":{"1min":0.91300000000000003,"5min":1.117,"15min":1.248},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.413,"5min":2.004,"15min":2.004},"last":1.071}]}]},{"osd":5,"up_from":14,"seq":60129542203,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60384,"kb_used_data":3028,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94311456,"statfs":{"total":96636764160,"available":96574930944,"internally_reserved":0,"allocated":3100672,"data_stored":2088626,"data_compressed":5202,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:02:09 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.84199999999999997,"15min":0.96899999999999997},"min":{"1min":0.40100000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.98099999999999998,"5min":4.4800000000000004,"15min":4.4800000000000004},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.78400000000000003,"15min":0.81299999999999994},"min":{"1min":0.44500000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.179,"5min":1.181,"15min":1.181},"last":1.173}]},{"osd":1,"last update":"Sat Sep 21 15:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.86399999999999999,"15min":0.91300000000000003},"min":{"1min":0.41899999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.036,"5min":2.7709999999999999,"15min":2.7709999999999999},"last":1.0049999999999999},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.89900000000000002,"15min":0.92900000000000005},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.1439999999999999,"5min":2.823,"15min":2.823},"last":1.3420000000000001}]},{"osd":2,"last update":"Sat Sep 21 15:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.86599999999999999,"15min":0.95799999999999996},"min":{"1min":0.53600000000000003,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.2090000000000001,"5min":3.419,"15min":3.419},"last":0.754},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.84499999999999997,"15min":0.88800000000000001},"min":{"1min":0.58999999999999997,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.1699999999999999,"5min":1.423,"15min":1.423},"last":0.91600000000000004}]},{"osd":3,"last update":"Sat Sep 21 15:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.81100000000000005,"15min":0.81000000000000005},"min":{"1min":0.51000000000000001,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.105,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.97299999999999998},{"interface":"front","average":{"1min":0.91700000000000004,"5min":1.018,"15min":1.0720000000000001},"min":{"1min":0.66600000000000004,"5min":0.435,"15min":0.435},"max":{"1min":1.4239999999999999,"5min":2.6379999999999999,"15min":2.6379999999999999},"last":1.038}]},{"osd":4,"last update":"Sat Sep 21 15:01:02 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.98899999999999999,"15min":1.016},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.5609999999999999,"5min":2.5470000000000002,"15min":2.5470000000000002},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.82099999999999995,"5min":1.0169999999999999,"15min":1.0780000000000001},"min":{"1min":0.39300000000000002,"5min":0.376,"15min":0.376},"max":{"1min":1.302,"5min":2.5099999999999998,"15min":2.5099999999999998},"last":1.145}]},{"osd":6,"last update":"Sat Sep 21 15:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.91700000000000004,"15min":0.95999999999999996},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.1499999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.879,"5min":1.1399999999999999,"15min":1.3169999999999999},"min":{"1min":0.60199999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.498,"5min":4.2469999999999999,"15min":4.2469999999999999},"last":0.88200000000000001}]},{"osd":7,"last update":"Sat Sep 21 15:02:09 2024","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":1.167,"15min":1.3120000000000001},"min":{"1min":0.61199999999999999,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.5700000000000001,"5min":3.3010000000000002,"15min":3.3010000000000002},"last":1.117},{"interface":"front","average":{"1min":0.99399999999999999,"5min":1.075,"15min":1.125},"min":{"1min":0.64300000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.25,"5min":2.5880000000000001,"15min":2.5880000000000001},"last":1.4430000000000001}]}]},{"osd":4,"up_from":235,"seq":1009317314567,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34860,"kb_used_data":2968,"kb_used_omap":8,"kb_used_meta":31863,"kb_avail":94336980,"statfs":{"total":96636764160,"available":96601067520,"internally_reserved":0,"allocated":3039232,"data_stored":1498239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8979,"internal_metadata":32627949},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0269999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.091}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86899999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92000000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2390000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542203,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78376,"kb_used_data":2972,"kb_used_omap":1,"kb_used_meta":75390,"kb_avail":94293464,"statfs":{"total":96636764160,"available":96556507136,"internally_reserved":0,"allocated":3043328,"data_stored":1498258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":77199816},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.61799999999999999,"15min":0.63200000000000001},"min":{"1min":0.34599999999999997,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.92700000000000005,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.68200000000000005,"15min":0.70999999999999996},"min":{"1min":0.40500000000000003,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.3029999999999999,"5min":1.919,"15min":1.919},"last":0.29999999999999999}]},{"osd":1,"last update":"Sat Sep 21 15:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.65800000000000003,"15min":0.67000000000000004},"min":{"1min":0.44400000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":1.115,"5min":1.722,"15min":1.722},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.746,"15min":0.79700000000000004},"min":{"1min":0.41199999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.2509999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.28499999999999998}]},{"osd":2,"last update":"Sat Sep 21 15:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.746,"15min":0.72999999999999998},"min":{"1min":0.38200000000000001,"5min":0.311,"15min":0.311},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.746,"5min":0.79200000000000004,"15min":0.82099999999999995},"min":{"1min":0.40300000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.94399999999999995,"5min":1.254,"15min":1.254},"last":0.49199999999999999}]},{"osd":4,"last update":"Sat Sep 21 15:01:09 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.93799999999999994,"15min":0.95599999999999996},"min":{"1min":0.54300000000000004,"5min":0.442,"15min":0.442},"max":{"1min":1.2629999999999999,"5min":1.7609999999999999,"15min":1.7609999999999999},"last":1.0600000000000001},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.98899999999999999,"15min":1.0269999999999999},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.234,"5min":2.028,"15min":2.028},"last":0.95399999999999996}]},{"osd":5,"last update":"Sat Sep 21 15:02:13 2024","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.036,"15min":1.052},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.4670000000000001,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":0.76500000000000001},{"interface":"front","average":{"1min":1.081,"5min":1.0589999999999999,"15min":1.0640000000000001},"min":{"1min":0.68799999999999994,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.395,"5min":1.548,"15min":1.548},"last":0.86199999999999999}]},{"osd":6,"last update":"Sat Sep 21 15:02:13 2024","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":1.046,"15min":1.085},"min":{"1min":0.70099999999999996,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.391,"5min":2.1219999999999999,"15min":2.1219999999999999},"last":0.78100000000000003},{"interface":"front","average":{"1min":1.03,"5min":1.075,"15min":1.0980000000000001},"min":{"1min":0.64900000000000002,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.448,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":0.747}]},{"osd":7,"last update":"Sat Sep 21 15:02:13 2024","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.0800000000000001,"15min":1.123},"min":{"1min":0.746,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.339,"5min":1.706,"15min":1.706},"last":0.84899999999999998},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.1279999999999999,"15min":1.171},"min":{"1min":0.67700000000000005,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.421,"5min":2.3380000000000001,"15min":2.3380000000000001},"last":0.89900000000000002}]}]},{"osd":2,"up_from":14,"seq":60129542201,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70180,"kb_used_data":2968,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94301660,"statfs":{"total":96636764160,"available":96564899840,"internally_reserved":0,"allocated":3039232,"data_stored":1498239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68811206},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:02:07 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.63500000000000001,"15min":0.625},"min":{"1min":0.32100000000000001,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.383,"5min":1.383,"15min":1.383},"last":0.439},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.69099999999999995,"15min":0.66900000000000004},"min":{"1min":0.30099999999999999,"5min":0.223,"15min":0.223},"max":{"1min":1.1739999999999999,"5min":2.9169999999999998,"15min":2.9169999999999998},"last":0.28399999999999997}]},{"osd":1,"last update":"Sat Sep 21 15:02:07 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.67500000000000004,"15min":0.68100000000000005},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.129,"5min":2.6520000000000001,"15min":2.6520000000000001},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.73399999999999999,"15min":0.75800000000000001},"min":{"1min":0.36199999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.48899999999999999}]},{"osd":3,"last update":"Sat Sep 21 15:02:07 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.71699999999999997,"15min":0.72699999999999998},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.996,"5min":3.0110000000000001,"15min":3.0110000000000001},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.85999999999999999,"15min":0.83399999999999996},"min":{"1min":0.41099999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0369999999999999,"5min":4.1440000000000001,"15min":4.1440000000000001},"last":0.50600000000000001}]},{"osd":4,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.93200000000000005,"15min":0.95599999999999996},"min":{"1min":0.42399999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.222,"5min":2.899,"15min":2.899},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.94399999999999995,"15min":0.95799999999999996},"min":{"1min":0.47499999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.4379999999999999,"5min":1.675,"15min":1.675},"last":0.58899999999999997}]},{"osd":5,"last update":"Sat Sep 21 15:02:07 2024","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.98699999999999999,"15min":1.0009999999999999},"min":{"1min":0.45500000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.224,"5min":3.1080000000000001,"15min":3.1080000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.96399999999999997,"15min":0.98799999999999999},"min":{"1min":0.55200000000000005,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.228,"5min":1.546,"15min":1.546},"last":0.91300000000000003}]},{"osd":6,"last update":"Sat Sep 21 15:02:07 2024","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":0.98899999999999999,"15min":0.98699999999999999},"min":{"1min":0.52100000000000002,"5min":0.378,"15min":0.378},"max":{"1min":1.516,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.93999999999999995,"5min":1.0429999999999999,"15min":1.048},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.403,"5min":3.0739999999999998,"15min":3.0739999999999998},"last":0.625}]},{"osd":7,"last update":"Sat Sep 21 15:02:09 2024","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.0489999999999999,"15min":1.01},"min":{"1min":0.61399999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.415,"5min":3.1589999999999998,"15min":3.1589999999999998},"last":0.84499999999999997},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.111,"15min":1.095},"min":{"1min":0.58799999999999997,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.4610000000000001,"5min":3.1400000000000001,"15min":3.1400000000000001},"last":0.69799999999999995}]}]},{"osd":1,"up_from":14,"seq":60129542204,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48104,"kb_used_data":3028,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323736,"statfs":{"total":96636764160,"available":96587505664,"internally_reserved":0,"allocated":3100672,"data_stored":2088626,"data_compressed":5202,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.58899999999999997,"15min":0.60699999999999998},"min":{"1min":0.25600000000000001,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.60299999999999998,"15min":0.60999999999999999},"min":{"1min":0.26300000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.83499999999999996,"5min":1.107,"15min":1.107},"last":0.47199999999999998}]},{"osd":2,"last update":"Sat Sep 21 15:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.78000000000000003,"15min":0.73499999999999999},"min":{"1min":0.38800000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.92100000000000004,"5min":11.303000000000001,"15min":11.303000000000001},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.71399999999999997,"15min":0.70299999999999996},"min":{"1min":0.439,"5min":0.435,"15min":0.435},"max":{"1min":1.0840000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.373}]},{"osd":3,"last update":"Sat Sep 21 15:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.80200000000000005,"15min":0.83399999999999996},"min":{"1min":0.36899999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.4850000000000001,"5min":3.1469999999999998,"15min":3.1469999999999998},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.83699999999999997,"15min":0.877},"min":{"1min":0.42499999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.284,"5min":3.2320000000000002,"15min":3.2320000000000002},"last":0.41199999999999998}]},{"osd":4,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.89000000000000001,"15min":0.91500000000000004},"min":{"1min":0.59599999999999997,"5min":0.309,"15min":0.309},"max":{"1min":1.45,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":1.5600000000000001,"5min":1.069,"15min":0.96599999999999997},"min":{"1min":0.504,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":11.563000000000001,"5min":11.563000000000001,"15min":11.563000000000001},"last":0.70299999999999996}]},{"osd":5,"last update":"Sat Sep 21 15:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":1.0620000000000001,"15min":0.95399999999999996},"min":{"1min":0.61199999999999999,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.3799999999999999,"5min":11.534000000000001,"15min":11.534000000000001},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.88,"5min":0.94199999999999995,"15min":0.95599999999999996},"min":{"1min":0.58599999999999997,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.1319999999999999,"5min":1.484,"15min":1.484},"last":0.78300000000000003}]},{"osd":6,"last update":"Sat Sep 21 15:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.97999999999999998,"15min":1.038},"min":{"1min":0.53100000000000003,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.254,"5min":2.3519999999999999,"15min":2.3519999999999999},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.97499999999999998,"5min":1.087,"15min":0.995},"min":{"1min":0.66400000000000003,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.4259999999999999,"5min":11.601000000000001,"15min":11.601000000000001},"last":0.61299999999999999}]},{"osd":7,"last update":"Sat Sep 21 15:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":1.02,"15min":1.056},"min":{"1min":0.58899999999999997,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.4039999999999999,"5min":2.2360000000000002,"15min":2.2360000000000002},"last":0.79600000000000004},{"interface":"front","average":{"1min":1.044,"5min":1.171,"15min":1.087},"min":{"1min":0.63100000000000001,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.3799999999999999,"5min":11.489000000000001,"15min":11.489000000000001},"last":0.66800000000000004}]}]},{"osd":0,"up_from":14,"seq":60129542201,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70184,"kb_used_data":2968,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94301656,"statfs":{"total":96636764160,"available":96564895744,"internally_reserved":0,"allocated":3039232,"data_stored":1498239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":68811207},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 21 15:02:07 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.64000000000000001,"15min":0.70599999999999996},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.002,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.69499999999999995,"15min":0.63200000000000001},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.98699999999999999,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.78100000000000003}]},{"osd":2,"last update":"Sat Sep 21 15:02:07 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.66100000000000003,"15min":0.621},"min":{"1min":0.33900000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.0960000000000001,"5min":1.153,"15min":1.153},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.70099999999999996,"15min":0.75},"min":{"1min":0.28799999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.0329999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.748}]},{"osd":3,"last update":"Sat Sep 21 15:02:07 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.76400000000000001,"15min":0.754},"min":{"1min":0.37,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.77400000000000002,"15min":0.81100000000000005},"min":{"1min":0.42799999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.04,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.54900000000000004}]},{"osd":4,"last update":"Sat Sep 21 15:01:04 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.98399999999999999,"15min":1.038},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.1140000000000001,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":1.069},{"interface":"front","average":{"1min":1.036,"5min":1.0469999999999999,"15min":1.0620000000000001},"min":{"1min":0.67800000000000005,"5min":0.379,"15min":0.379},"max":{"1min":1.446,"5min":1.665,"15min":1.665},"last":1.05}]},{"osd":5,"last update":"Sat Sep 21 15:02:07 2024","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.997,"15min":1.0229999999999999},"min":{"1min":0.45800000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.296,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.98699999999999999},{"interface":"front","average":{"1min":0.94299999999999995,"5min":1.0129999999999999,"15min":1.0669999999999999},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.3919999999999999,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":1.1240000000000001}]},{"osd":6,"last update":"Sat Sep 21 15:02:07 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.98799999999999999,"15min":1.089},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.119,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.94799999999999995,"5min":1.038,"15min":1.0600000000000001},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.393,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":1.214}]},{"osd":7,"last update":"Sat Sep 21 15:02:07 2024","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":1.012,"15min":1.05},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.1719999999999999,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":1.0289999999999999},{"interface":"front","average":{"1min":0.88300000000000001,"5min":1.0309999999999999,"15min":1.113},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.28,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":0.95999999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5202,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5202,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7385,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T15:02:35.843 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T15:02:36.150 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-21T15:02:36.150 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":251,"fsid":"caf51e55-9234-4a26-bcd2-d60811962560","created":"2024-09-21T14:57:50.338822+0000","modified":"2024-09-21T15:02:29.341630+0000","last_up_change":"2024-09-21T15:02:13.195875+0000","last_in_change":"2024-09-21T15:02:13.195875+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T14:58:01.842810+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-21T14:58:08.284925+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":22,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-21T14:58:17.510709+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"230","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":151,"snap_epoch":230,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a4a73316-3a94-42e9-9a51-deb46a66bb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":4016672243},{"type":"v1","addr":"172.21.15.32:6809","nonce":4016672243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":4016672243},{"type":"v1","addr":"172.21.15.32:6811","nonce":4016672243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":4016672243},{"type":"v1","addr":"172.21.15.32:6815","nonce":4016672243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":4016672243},{"type":"v1","addr":"172.21.15.32:6813","nonce":4016672243}]},"public_addr":"172.21.15.32:6809/4016672243","cluster_addr":"172.21.15.32:6811/4016672243","heartbeat_back_addr":"172.21.15.32:6815/4016672243","heartbeat_front_addr":"172.21.15.32:6813/4016672243","state":["exists","up"]},{"osd":1,"uuid":"006757a0-94d1-4a9b-aef8-73634cc5828e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":641090577},{"type":"v1","addr":"172.21.15.32:6825","nonce":641090577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":641090577},{"type":"v1","addr":"172.21.15.32:6827","nonce":641090577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":641090577},{"type":"v1","addr":"172.21.15.32:6831","nonce":641090577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":641090577},{"type":"v1","addr":"172.21.15.32:6829","nonce":641090577}]},"public_addr":"172.21.15.32:6825/641090577","cluster_addr":"172.21.15.32:6827/641090577","heartbeat_back_addr":"172.21.15.32:6831/641090577","heartbeat_front_addr":"172.21.15.32:6829/641090577","state":["exists","up"]},{"osd":2,"uuid":"0b499a23-820e-4e6c-a558-adbf88b7014f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2609154414},{"type":"v1","addr":"172.21.15.32:6801","nonce":2609154414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2609154414},{"type":"v1","addr":"172.21.15.32:6803","nonce":2609154414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2609154414},{"type":"v1","addr":"172.21.15.32:6807","nonce":2609154414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2609154414},{"type":"v1","addr":"172.21.15.32:6805","nonce":2609154414}]},"public_addr":"172.21.15.32:6801/2609154414","cluster_addr":"172.21.15.32:6803/2609154414","heartbeat_back_addr":"172.21.15.32:6807/2609154414","heartbeat_front_addr":"172.21.15.32:6805/2609154414","state":["exists","up"]},{"osd":3,"uuid":"5a57b3dd-4a07-45b7-b159-3cac4c7eb262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":239,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":1462563376},{"type":"v1","addr":"172.21.15.32:6817","nonce":1462563376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":1462563376},{"type":"v1","addr":"172.21.15.32:6819","nonce":1462563376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":1462563376},{"type":"v1","addr":"172.21.15.32:6823","nonce":1462563376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":1462563376},{"type":"v1","addr":"172.21.15.32:6821","nonce":1462563376}]},"public_addr":"172.21.15.32:6817/1462563376","cluster_addr":"172.21.15.32:6819/1462563376","heartbeat_back_addr":"172.21.15.32:6823/1462563376","heartbeat_front_addr":"172.21.15.32:6821/1462563376","state":["exists","up"]},{"osd":4,"uuid":"39a00807-7517-474e-8842-623dd7093cc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":200,"up_from":235,"up_thru":240,"down_at":202,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6801","nonce":2900482440},{"type":"v1","addr":"172.21.15.46:6803","nonce":2900482440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6805","nonce":2900482440},{"type":"v1","addr":"172.21.15.46:6807","nonce":2900482440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2900482440},{"type":"v1","addr":"172.21.15.46:6814","nonce":2900482440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2900482440},{"type":"v1","addr":"172.21.15.46:6811","nonce":2900482440}]},"public_addr":"172.21.15.46:6803/2900482440","cluster_addr":"172.21.15.46:6807/2900482440","heartbeat_back_addr":"172.21.15.46:6814/2900482440","heartbeat_front_addr":"172.21.15.46:6811/2900482440","state":["exists","up"]},{"osd":5,"uuid":"edf9f045-ddeb-4504-a0fc-1a0e4e853550","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":245,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":849101442},{"type":"v1","addr":"172.21.15.46:6802","nonce":849101442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":849101442},{"type":"v1","addr":"172.21.15.46:6806","nonce":849101442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":849101442},{"type":"v1","addr":"172.21.15.46:6815","nonce":849101442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":849101442},{"type":"v1","addr":"172.21.15.46:6810","nonce":849101442}]},"public_addr":"172.21.15.46:6802/849101442","cluster_addr":"172.21.15.46:6806/849101442","heartbeat_back_addr":"172.21.15.46:6815/849101442","heartbeat_front_addr":"172.21.15.46:6810/849101442","state":["exists","up"]},{"osd":6,"uuid":"c820ea6b-88fe-4421-83ec-17c466691fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":847189172},{"type":"v1","addr":"172.21.15.46:6818","nonce":847189172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":847189172},{"type":"v1","addr":"172.21.15.46:6822","nonce":847189172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6827","nonce":847189172},{"type":"v1","addr":"172.21.15.46:6828","nonce":847189172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6823","nonce":847189172},{"type":"v1","addr":"172.21.15.46:6825","nonce":847189172}]},"public_addr":"172.21.15.46:6818/847189172","cluster_addr":"172.21.15.46:6822/847189172","heartbeat_back_addr":"172.21.15.46:6828/847189172","heartbeat_front_addr":"172.21.15.46:6825/847189172","state":["exists","up"]},{"osd":7,"uuid":"3a2f82a4-28a0-4fcb-9d6b-2285fa4d07ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":223,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3894963553},{"type":"v1","addr":"172.21.15.46:6819","nonce":3894963553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":3894963553},{"type":"v1","addr":"172.21.15.46:6824","nonce":3894963553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":3894963553},{"type":"v1","addr":"172.21.15.46:6831","nonce":3894963553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":3894963553},{"type":"v1","addr":"172.21.15.46:6829","nonce":3894963553}]},"public_addr":"172.21.15.46:6819/3894963553","cluster_addr":"172.21.15.46:6824/3894963553","heartbeat_back_addr":"172.21.15.46:6831/3894963553","heartbeat_front_addr":"172.21.15.46:6829/3894963553","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T14:57:57.458317+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T14:57:57.907624+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T14:57:57.278817+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T14:57:57.500449+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T14:57:57.504325+0000","dead_epoch":202},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T14:57:57.777525+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T14:57:57.698961+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T14:57:57.932625+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.2","osds":[2,3]},{"pgid":"3.4","osds":[4,2]},{"pgid":"3.8","osds":[6,3]},{"pgid":"3.a","osds":[1,0]}],"pg_upmap_items":[{"pgid":"3.1","mappings":[{"from":2,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T15:02:37.165 INFO:tasks.ceph:Scrubbing osd.0 2024-09-21T15:02:37.166 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-21T15:02:37.332 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:02:37.332 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T15:02:37.332 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:02:37.350 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-21T15:02:37.675 INFO:teuthology.orchestra.run.smithi032.stderr:instructed osd(s) 0 to deep-scrub 2024-09-21T15:02:37.692 INFO:tasks.ceph:Scrubbing osd.1 2024-09-21T15:02:37.692 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-21T15:02:37.847 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:02:37.848 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T15:02:37.848 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:02:37.866 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-21T15:02:38.179 INFO:teuthology.orchestra.run.smithi032.stderr:instructed osd(s) 1 to deep-scrub 2024-09-21T15:02:38.190 INFO:tasks.ceph:Scrubbing osd.2 2024-09-21T15:02:38.190 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-21T15:02:38.344 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:02:38.345 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T15:02:38.345 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:02:38.359 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-21T15:02:38.696 INFO:teuthology.orchestra.run.smithi032.stderr:instructed osd(s) 2 to deep-scrub 2024-09-21T15:02:38.709 INFO:tasks.ceph:Scrubbing osd.3 2024-09-21T15:02:38.710 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-21T15:02:38.869 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:02:38.870 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T15:02:38.870 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:02:38.883 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-21T15:02:39.198 INFO:teuthology.orchestra.run.smithi032.stderr:instructed osd(s) 3 to deep-scrub 2024-09-21T15:02:39.214 INFO:tasks.ceph:Scrubbing osd.4 2024-09-21T15:02:39.215 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-21T15:02:39.374 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:02:39.374 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_pg_delete_cost = '' (not observed, change may require restart) osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2024-09-21T15:02:39.374 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:02:39.387 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-21T15:02:39.701 INFO:teuthology.orchestra.run.smithi032.stderr:instructed osd(s) 4 to deep-scrub 2024-09-21T15:02:39.712 INFO:tasks.ceph:Scrubbing osd.5 2024-09-21T15:02:39.712 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-21T15:02:39.873 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:02:39.873 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T15:02:39.873 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:02:39.886 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-21T15:02:40.199 INFO:teuthology.orchestra.run.smithi032.stderr:instructed osd(s) 5 to deep-scrub 2024-09-21T15:02:40.211 INFO:tasks.ceph:Scrubbing osd.6 2024-09-21T15:02:40.211 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-21T15:02:40.354 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:02:40.354 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T15:02:40.354 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:02:40.367 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-21T15:02:40.675 INFO:teuthology.orchestra.run.smithi032.stderr:instructed osd(s) 6 to deep-scrub 2024-09-21T15:02:40.686 INFO:tasks.ceph:Scrubbing osd.7 2024-09-21T15:02:40.687 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-21T15:02:40.840 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-09-21T15:02:40.840 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T15:02:40.840 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-09-21T15:02:40.854 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-21T15:02:41.169 INFO:teuthology.orchestra.run.smithi032.stderr:instructed osd(s) 7 to deep-scrub 2024-09-21T15:02:41.180 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T15:02:41.492 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-21T15:02:41.496 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-09-21T15:02:41.512 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":384,"stamp":"2024-09-21T15:02:39.849374+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":174155,"num_read_kb":1238336,"num_write":34466,"num_write_kb":943253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3956,"num_bytes_recovered":212783659,"num_keys_recovered":305,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22661,"ondisk_log_size":22661,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":460608,"kb_used_data":23872,"kb_used_omap":19,"kb_used_meta":436588,"kb_avail":754514112,"statfs":{"total":773094113280,"available":772622450688,"internally_reserved":0,"allocated":24444928,"data_stored":13166724,"data_compressed":10404,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":20129,"internal_metadata":447066463},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.498422"},"pg_stats":[{"pgid":"3.1f","version":"230'598","reported_seq":3979,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:29.452925+0000","last_change":"2024-09-21T15:02:24.311888+0000","last_active":"2024-09-21T15:02:29.452925+0000","last_peered":"2024-09-21T15:02:29.452925+0000","last_clean":"2024-09-21T15:02:29.452925+0000","last_became_active":"2024-09-21T15:02:24.308451+0000","last_became_peered":"2024-09-21T15:02:24.308451+0000","last_unstale":"2024-09-21T15:02:29.452925+0000","last_undegraded":"2024-09-21T15:02:29.452925+0000","last_fullsized":"2024-09-21T15:02:29.452925+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":598,"log_dups_size":0,"ondisk_log_size":598,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:25:13.708671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010410479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3756,"num_read_kb":43219,"num_write":975,"num_write_kb":36436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"230'507","reported_seq":3510,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.335752+0000","last_change":"2024-09-21T15:02:24.312475+0000","last_active":"2024-09-21T15:02:28.335752+0000","last_peered":"2024-09-21T15:02:28.335752+0000","last_clean":"2024-09-21T15:02:28.335752+0000","last_became_active":"2024-09-21T15:02:24.309506+0000","last_became_peered":"2024-09-21T15:02:24.309506+0000","last_unstale":"2024-09-21T15:02:28.335752+0000","last_undegraded":"2024-09-21T15:02:28.335752+0000","last_fullsized":"2024-09-21T15:02:28.335752+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":507,"log_dups_size":0,"ondisk_log_size":507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:49:24.984436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094912299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3420,"num_read_kb":23407,"num_write":643,"num_write_kb":18445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":148,"num_bytes_recovered":11635695,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"228'341","reported_seq":4790,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.957133+0000","last_change":"2024-09-21T15:02:02.409182+0000","last_active":"2024-09-21T15:02:28.957133+0000","last_peered":"2024-09-21T15:02:28.957133+0000","last_clean":"2024-09-21T15:02:28.957133+0000","last_became_active":"2024-09-21T14:58:19.411387+0000","last_became_peered":"2024-09-21T14:58:19.411387+0000","last_unstale":"2024-09-21T15:02:28.957133+0000","last_undegraded":"2024-09-21T15:02:28.957133+0000","last_fullsized":"2024-09-21T15:02:28.957133+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":341,"log_dups_size":0,"ondisk_log_size":341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:37:57.597069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045388299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3878,"num_read_kb":28288,"num_write":535,"num_write_kb":26606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"230'572","reported_seq":4306,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.956805+0000","last_change":"2024-09-21T15:02:02.261134+0000","last_active":"2024-09-21T15:02:28.956805+0000","last_peered":"2024-09-21T15:02:28.956805+0000","last_clean":"2024-09-21T15:02:28.956805+0000","last_became_active":"2024-09-21T14:58:19.411306+0000","last_became_peered":"2024-09-21T14:58:19.411306+0000","last_unstale":"2024-09-21T15:02:28.956805+0000","last_undegraded":"2024-09-21T15:02:28.956805+0000","last_fullsized":"2024-09-21T15:02:28.956805+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":572,"log_dups_size":0,"ondisk_log_size":572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:20:27.079795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011721469999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4050,"num_read_kb":24676,"num_write":863,"num_write_kb":25228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"227'536","reported_seq":4293,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.335194+0000","last_change":"2024-09-21T15:02:17.248373+0000","last_active":"2024-09-21T15:02:28.335194+0000","last_peered":"2024-09-21T15:02:28.335194+0000","last_clean":"2024-09-21T15:02:28.335194+0000","last_became_active":"2024-09-21T15:02:17.242801+0000","last_became_peered":"2024-09-21T15:02:17.242801+0000","last_unstale":"2024-09-21T15:02:28.335194+0000","last_undegraded":"2024-09-21T15:02:28.335194+0000","last_fullsized":"2024-09-21T15:02:28.335194+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":536,"log_dups_size":0,"ondisk_log_size":536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:04:28.325046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032550800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3672,"num_read_kb":32698,"num_write":706,"num_write_kb":22582,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":379,"num_bytes_recovered":13616631,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"227'487","reported_seq":4857,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.956810+0000","last_change":"2024-09-21T15:02:02.262402+0000","last_active":"2024-09-21T15:02:28.956810+0000","last_peered":"2024-09-21T15:02:28.956810+0000","last_clean":"2024-09-21T15:02:28.956810+0000","last_became_active":"2024-09-21T14:58:19.409415+0000","last_became_peered":"2024-09-21T14:58:19.409415+0000","last_unstale":"2024-09-21T15:02:28.956810+0000","last_undegraded":"2024-09-21T15:02:28.956810+0000","last_fullsized":"2024-09-21T15:02:28.956810+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:28:46.336301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019572090000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4389,"num_read_kb":28349,"num_write":704,"num_write_kb":23876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"228'824","reported_seq":8731,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.335092+0000","last_change":"2024-09-21T15:02:18.252228+0000","last_active":"2024-09-21T15:02:28.335092+0000","last_peered":"2024-09-21T15:02:28.335092+0000","last_clean":"2024-09-21T15:02:28.335092+0000","last_became_active":"2024-09-21T15:02:18.246987+0000","last_became_peered":"2024-09-21T15:02:18.246987+0000","last_unstale":"2024-09-21T15:02:28.335092+0000","last_undegraded":"2024-09-21T15:02:28.335092+0000","last_fullsized":"2024-09-21T15:02:28.335092+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":824,"log_dups_size":0,"ondisk_log_size":824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:42:11.072452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014776850000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9383,"num_read_kb":68058,"num_write":1251,"num_write_kb":29921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":394,"num_bytes_recovered":23509073,"num_keys_recovered":66,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"227'688","reported_seq":8793,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.335591+0000","last_change":"2024-09-21T15:02:02.359228+0000","last_active":"2024-09-21T15:02:28.335591+0000","last_peered":"2024-09-21T15:02:28.335591+0000","last_clean":"2024-09-21T15:02:28.335591+0000","last_became_active":"2024-09-21T14:58:19.409371+0000","last_became_peered":"2024-09-21T14:58:19.409371+0000","last_unstale":"2024-09-21T15:02:28.335591+0000","last_undegraded":"2024-09-21T15:02:28.335591+0000","last_fullsized":"2024-09-21T15:02:28.335591+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":688,"log_dups_size":0,"ondisk_log_size":688,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:35:27.143445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052048100000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9236,"num_read_kb":45266,"num_write":998,"num_write_kb":30616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"227'413","reported_seq":4005,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.335834+0000","last_change":"2024-09-21T15:02:02.358947+0000","last_active":"2024-09-21T15:02:28.335834+0000","last_peered":"2024-09-21T15:02:28.335834+0000","last_clean":"2024-09-21T15:02:28.335834+0000","last_became_active":"2024-09-21T14:58:19.410306+0000","last_became_peered":"2024-09-21T14:58:19.410306+0000","last_unstale":"2024-09-21T15:02:28.335834+0000","last_undegraded":"2024-09-21T15:02:28.335834+0000","last_fullsized":"2024-09-21T15:02:28.335834+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":413,"log_dups_size":0,"ondisk_log_size":413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:33:50.089165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065724399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3380,"num_read_kb":28207,"num_write":769,"num_write_kb":34633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"229'1053","reported_seq":6175,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.956985+0000","last_change":"2024-09-21T15:02:02.263447+0000","last_active":"2024-09-21T15:02:28.956985+0000","last_peered":"2024-09-21T15:02:28.956985+0000","last_clean":"2024-09-21T15:02:28.956985+0000","last_became_active":"2024-09-21T15:01:06.166756+0000","last_became_peered":"2024-09-21T15:01:06.166756+0000","last_unstale":"2024-09-21T15:02:28.956985+0000","last_undegraded":"2024-09-21T15:02:28.956985+0000","last_fullsized":"2024-09-21T15:02:28.956985+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":1053,"log_dups_size":0,"ondisk_log_size":1053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:19:04.988784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001157057,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8073,"num_read_kb":43999,"num_write":1621,"num_write_kb":28374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":271,"num_bytes_recovered":16821266,"num_keys_recovered":55,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"230'823","reported_seq":3979,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.339198+0000","last_change":"2024-09-21T15:02:02.276055+0000","last_active":"2024-09-21T15:02:28.339198+0000","last_peered":"2024-09-21T15:02:28.339198+0000","last_clean":"2024-09-21T15:02:28.339198+0000","last_became_active":"2024-09-21T14:58:19.410687+0000","last_became_peered":"2024-09-21T14:58:19.410687+0000","last_unstale":"2024-09-21T15:02:28.339198+0000","last_undegraded":"2024-09-21T15:02:28.339198+0000","last_fullsized":"2024-09-21T15:02:28.339198+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":823,"log_dups_size":0,"ondisk_log_size":823,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:21:42.226982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017043010000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4757,"num_read_kb":49770,"num_write":1348,"num_write_kb":33339,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"228'527","reported_seq":4193,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.955479+0000","last_change":"2024-09-21T15:02:02.408062+0000","last_active":"2024-09-21T15:02:28.955479+0000","last_peered":"2024-09-21T15:02:28.955479+0000","last_clean":"2024-09-21T15:02:28.955479+0000","last_became_active":"2024-09-21T14:58:19.407310+0000","last_became_peered":"2024-09-21T14:58:19.407310+0000","last_unstale":"2024-09-21T15:02:28.955479+0000","last_undegraded":"2024-09-21T15:02:28.955479+0000","last_fullsized":"2024-09-21T15:02:28.955479+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":527,"log_dups_size":0,"ondisk_log_size":527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:47:46.079115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016534340000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3756,"num_read_kb":43322,"num_write":713,"num_write_kb":33471,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":245,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:29.771991+0000","last_change":"2024-09-21T14:58:11.348177+0000","last_active":"2024-09-21T15:02:29.771991+0000","last_peered":"2024-09-21T15:02:29.771991+0000","last_clean":"2024-09-21T15:02:29.771991+0000","last_became_active":"2024-09-21T14:58:09.328417+0000","last_became_peered":"2024-09-21T14:58:09.328417+0000","last_unstale":"2024-09-21T15:02:29.771991+0000","last_undegraded":"2024-09-21T15:02:29.771991+0000","last_fullsized":"2024-09-21T15:02:29.771991+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:52:22.671185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018917490000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"230'610","reported_seq":6567,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.335492+0000","last_change":"2024-09-21T15:02:17.251834+0000","last_active":"2024-09-21T15:02:28.335492+0000","last_peered":"2024-09-21T15:02:28.335492+0000","last_clean":"2024-09-21T15:02:28.335492+0000","last_became_active":"2024-09-21T15:02:17.247583+0000","last_became_peered":"2024-09-21T15:02:17.247583+0000","last_unstale":"2024-09-21T15:02:28.335492+0000","last_undegraded":"2024-09-21T15:02:28.335492+0000","last_fullsized":"2024-09-21T15:02:28.335492+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":610,"log_dups_size":0,"ondisk_log_size":610,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:54:31.703168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013409660000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6297,"num_read_kb":53037,"num_write":697,"num_write_kb":27961,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":11818795,"num_keys_recovered":18,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":24,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.335350+0000","last_change":"2024-09-21T15:02:14.225503+0000","last_active":"2024-09-21T15:02:28.335350+0000","last_peered":"2024-09-21T15:02:28.335350+0000","last_clean":"2024-09-21T15:02:28.335350+0000","last_became_active":"2024-09-21T15:02:14.224646+0000","last_became_peered":"2024-09-21T15:02:14.224646+0000","last_unstale":"2024-09-21T15:02:28.335350+0000","last_undegraded":"2024-09-21T15:02:28.335350+0000","last_fullsized":"2024-09-21T15:02:28.335350+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:57:35.948252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"228'980","reported_seq":4262,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.335626+0000","last_change":"2024-09-21T15:02:19.255988+0000","last_active":"2024-09-21T15:02:28.335626+0000","last_peered":"2024-09-21T15:02:28.335626+0000","last_clean":"2024-09-21T15:02:28.335626+0000","last_became_active":"2024-09-21T15:02:19.251008+0000","last_became_peered":"2024-09-21T15:02:19.251008+0000","last_unstale":"2024-09-21T15:02:28.335626+0000","last_undegraded":"2024-09-21T15:02:28.335626+0000","last_fullsized":"2024-09-21T15:02:28.335626+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":980,"log_dups_size":0,"ondisk_log_size":980,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:13:15.145504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015726799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5449,"num_read_kb":31198,"num_write":1400,"num_write_kb":25281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":401,"num_bytes_recovered":12566413,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":277,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.957358+0000","last_change":"2024-09-21T15:02:14.227927+0000","last_active":"2024-09-21T15:02:28.957358+0000","last_peered":"2024-09-21T15:02:28.957358+0000","last_clean":"2024-09-21T15:02:28.957358+0000","last_became_active":"2024-09-21T15:02:14.227223+0000","last_became_peered":"2024-09-21T15:02:14.227223+0000","last_unstale":"2024-09-21T15:02:28.957358+0000","last_undegraded":"2024-09-21T15:02:28.957358+0000","last_fullsized":"2024-09-21T15:02:28.957358+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:43:37.059225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019338770000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"227'672","reported_seq":5560,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.335126+0000","last_change":"2024-09-21T15:02:18.251535+0000","last_active":"2024-09-21T15:02:28.335126+0000","last_peered":"2024-09-21T15:02:28.335126+0000","last_clean":"2024-09-21T15:02:28.335126+0000","last_became_active":"2024-09-21T15:02:18.246450+0000","last_became_peered":"2024-09-21T15:02:18.246450+0000","last_unstale":"2024-09-21T15:02:28.335126+0000","last_undegraded":"2024-09-21T15:02:28.335126+0000","last_fullsized":"2024-09-21T15:02:28.335126+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":672,"log_dups_size":0,"ondisk_log_size":672,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:04:40.306321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071861500000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5178,"num_read_kb":38459,"num_write":1130,"num_write_kb":41306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":331,"num_bytes_recovered":25796920,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":232,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.335618+0000","last_change":"2024-09-21T15:02:24.351816+0000","last_active":"2024-09-21T15:02:28.335618+0000","last_peered":"2024-09-21T15:02:28.335618+0000","last_clean":"2024-09-21T15:02:28.335618+0000","last_became_active":"2024-09-21T15:02:24.351245+0000","last_became_peered":"2024-09-21T15:02:24.351245+0000","last_unstale":"2024-09-21T15:02:28.335618+0000","last_undegraded":"2024-09-21T15:02:28.335618+0000","last_fullsized":"2024-09-21T15:02:28.335618+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:43:32.690041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"227'732","reported_seq":2860,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:29.354087+0000","last_change":"2024-09-21T15:02:02.262648+0000","last_active":"2024-09-21T15:02:29.354087+0000","last_peered":"2024-09-21T15:02:29.354087+0000","last_clean":"2024-09-21T15:02:29.354087+0000","last_became_active":"2024-09-21T14:58:19.408971+0000","last_became_peered":"2024-09-21T14:58:19.408971+0000","last_unstale":"2024-09-21T15:02:29.354087+0000","last_undegraded":"2024-09-21T15:02:29.354087+0000","last_fullsized":"2024-09-21T15:02:29.354087+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":732,"log_dups_size":0,"ondisk_log_size":732,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:29:52.642121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073528299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3602,"num_read_kb":20655,"num_write":1230,"num_write_kb":20679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"230'1009","reported_seq":3690,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:29.738404+0000","last_change":"2024-09-21T15:02:02.276284+0000","last_active":"2024-09-21T15:02:29.738404+0000","last_peered":"2024-09-21T15:02:29.738404+0000","last_clean":"2024-09-21T15:02:29.738404+0000","last_became_active":"2024-09-21T14:58:19.417453+0000","last_became_peered":"2024-09-21T14:58:19.417453+0000","last_unstale":"2024-09-21T15:02:29.738404+0000","last_undegraded":"2024-09-21T15:02:29.738404+0000","last_fullsized":"2024-09-21T15:02:29.738404+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":1009,"log_dups_size":0,"ondisk_log_size":1009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:19:27.087272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044463100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4845,"num_read_kb":25284,"num_write":1566,"num_write_kb":21735,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":247,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.956823+0000","last_change":"2024-09-21T14:58:11.347504+0000","last_active":"2024-09-21T15:02:28.956823+0000","last_peered":"2024-09-21T15:02:28.956823+0000","last_clean":"2024-09-21T15:02:28.956823+0000","last_became_active":"2024-09-21T14:58:09.775981+0000","last_became_peered":"2024-09-21T14:58:09.775981+0000","last_unstale":"2024-09-21T15:02:28.956823+0000","last_undegraded":"2024-09-21T15:02:28.956823+0000","last_fullsized":"2024-09-21T15:02:28.956823+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:47:16.286652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012568880000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"228'1092","reported_seq":4001,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.335364+0000","last_change":"2024-09-21T15:02:24.314768+0000","last_active":"2024-09-21T15:02:28.335364+0000","last_peered":"2024-09-21T15:02:28.335364+0000","last_clean":"2024-09-21T15:02:28.335364+0000","last_became_active":"2024-09-21T15:02:24.311531+0000","last_became_peered":"2024-09-21T15:02:24.311531+0000","last_unstale":"2024-09-21T15:02:28.335364+0000","last_undegraded":"2024-09-21T15:02:28.335364+0000","last_fullsized":"2024-09-21T15:02:28.335364+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":1092,"log_dups_size":0,"ondisk_log_size":1092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:22:48.528933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00183417,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6343,"num_read_kb":28797,"num_write":1677,"num_write_kb":26974,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":246,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.953692+0000","last_change":"2024-09-21T14:58:11.342210+0000","last_active":"2024-09-21T15:02:28.953692+0000","last_peered":"2024-09-21T15:02:28.953692+0000","last_clean":"2024-09-21T15:02:28.953692+0000","last_became_active":"2024-09-21T14:58:09.330624+0000","last_became_peered":"2024-09-21T14:58:09.330624+0000","last_unstale":"2024-09-21T15:02:28.953692+0000","last_undegraded":"2024-09-21T15:02:28.953692+0000","last_fullsized":"2024-09-21T15:02:28.953692+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:29:43.229538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001051151,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"227'339","reported_seq":2553,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:29.769648+0000","last_change":"2024-09-21T15:02:02.257475+0000","last_active":"2024-09-21T15:02:29.769648+0000","last_peered":"2024-09-21T15:02:29.769648+0000","last_clean":"2024-09-21T15:02:29.769648+0000","last_became_active":"2024-09-21T15:01:53.193166+0000","last_became_peered":"2024-09-21T15:01:53.193166+0000","last_unstale":"2024-09-21T15:02:29.769648+0000","last_undegraded":"2024-09-21T15:02:29.769648+0000","last_fullsized":"2024-09-21T15:02:29.769648+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":339,"log_dups_size":0,"ondisk_log_size":339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:37:52.546654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014983959999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2371,"num_read_kb":40193,"num_write":529,"num_write_kb":39390,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":366,"num_bytes_recovered":22723935,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":245,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.955327+0000","last_change":"2024-09-21T14:58:11.346766+0000","last_active":"2024-09-21T15:02:28.955327+0000","last_peered":"2024-09-21T15:02:28.955327+0000","last_clean":"2024-09-21T15:02:28.955327+0000","last_became_active":"2024-09-21T14:58:09.329748+0000","last_became_peered":"2024-09-21T14:58:09.329748+0000","last_unstale":"2024-09-21T15:02:28.955327+0000","last_undegraded":"2024-09-21T15:02:28.955327+0000","last_fullsized":"2024-09-21T15:02:28.955327+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:31:20.267675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052698000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"230'1275","reported_seq":8379,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.955421+0000","last_change":"2024-09-21T15:02:02.261569+0000","last_active":"2024-09-21T15:02:28.955421+0000","last_peered":"2024-09-21T15:02:28.955421+0000","last_clean":"2024-09-21T15:02:28.955421+0000","last_became_active":"2024-09-21T14:58:41.401003+0000","last_became_peered":"2024-09-21T14:58:41.401003+0000","last_unstale":"2024-09-21T15:02:28.955421+0000","last_undegraded":"2024-09-21T15:02:28.955421+0000","last_fullsized":"2024-09-21T15:02:28.955421+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":1275,"log_dups_size":0,"ondisk_log_size":1275,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:05:37.644835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000746033,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11708,"num_read_kb":50816,"num_write":1955,"num_write_kb":24913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1403348,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":246,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.955718+0000","last_change":"2024-09-21T14:58:11.347700+0000","last_active":"2024-09-21T15:02:28.955718+0000","last_peered":"2024-09-21T15:02:28.955718+0000","last_clean":"2024-09-21T15:02:28.955718+0000","last_became_active":"2024-09-21T14:58:09.438842+0000","last_became_peered":"2024-09-21T14:58:09.438842+0000","last_unstale":"2024-09-21T15:02:28.955718+0000","last_undegraded":"2024-09-21T15:02:28.955718+0000","last_fullsized":"2024-09-21T15:02:28.955718+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:41:34.626249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001426194,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":394,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.957058+0000","last_change":"2024-09-21T14:58:03.616055+0000","last_active":"2024-09-21T15:02:28.957058+0000","last_peered":"2024-09-21T15:02:28.957058+0000","last_clean":"2024-09-21T15:02:28.957058+0000","last_became_active":"2024-09-21T14:58:03.613214+0000","last_became_peered":"2024-09-21T14:58:03.613214+0000","last_unstale":"2024-09-21T15:02:28.957058+0000","last_undegraded":"2024-09-21T15:02:28.957058+0000","last_fullsized":"2024-09-21T15:02:28.957058+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:02.180162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:02.180162+0000","last_clean_scrub_stamp":"2024-09-21T14:58:02.180162+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:29:05.860361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"228'685","reported_seq":4697,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.953055+0000","last_change":"2024-09-21T15:02:02.257763+0000","last_active":"2024-09-21T15:02:28.953055+0000","last_peered":"2024-09-21T15:02:28.953055+0000","last_clean":"2024-09-21T15:02:28.953055+0000","last_became_active":"2024-09-21T15:01:32.177913+0000","last_became_peered":"2024-09-21T15:01:32.177913+0000","last_unstale":"2024-09-21T15:02:28.953055+0000","last_undegraded":"2024-09-21T15:02:28.953055+0000","last_fullsized":"2024-09-21T15:02:28.953055+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":685,"log_dups_size":0,"ondisk_log_size":685,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:47:09.376828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015618730000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4407,"num_read_kb":44538,"num_write":1164,"num_write_kb":30502,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":413,"num_bytes_recovered":20409497,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.9","version":"229'973","reported_seq":4119,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.955874+0000","last_change":"2024-09-21T15:02:18.257627+0000","last_active":"2024-09-21T15:02:28.955874+0000","last_peered":"2024-09-21T15:02:28.955874+0000","last_clean":"2024-09-21T15:02:28.955874+0000","last_became_active":"2024-09-21T15:02:18.255482+0000","last_became_peered":"2024-09-21T15:02:18.255482+0000","last_unstale":"2024-09-21T15:02:28.955874+0000","last_undegraded":"2024-09-21T15:02:28.955874+0000","last_fullsized":"2024-09-21T15:02:28.955874+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":973,"log_dups_size":0,"ondisk_log_size":973,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:15:56.076204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001545105,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4815,"num_read_kb":35700,"num_write":1595,"num_write_kb":33802,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":375,"num_bytes_recovered":14742839,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"229'1027","reported_seq":5030,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:29.355100+0000","last_change":"2024-09-21T15:02:02.261576+0000","last_active":"2024-09-21T15:02:29.355100+0000","last_peered":"2024-09-21T15:02:29.355100+0000","last_clean":"2024-09-21T15:02:29.355100+0000","last_became_active":"2024-09-21T15:00:27.280971+0000","last_became_peered":"2024-09-21T15:00:27.280971+0000","last_unstale":"2024-09-21T15:02:29.355100+0000","last_undegraded":"2024-09-21T15:02:29.355100+0000","last_fullsized":"2024-09-21T15:02:29.355100+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":1027,"log_dups_size":0,"ondisk_log_size":1027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:26:07.617059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013613779999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7341,"num_read_kb":31141,"num_write":1421,"num_write_kb":33160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":9228123,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"227'593","reported_seq":3447,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.953752+0000","last_change":"2024-09-21T15:02:02.265620+0000","last_active":"2024-09-21T15:02:28.953752+0000","last_peered":"2024-09-21T15:02:28.953752+0000","last_clean":"2024-09-21T15:02:28.953752+0000","last_became_active":"2024-09-21T14:58:19.410679+0000","last_became_peered":"2024-09-21T14:58:19.410679+0000","last_unstale":"2024-09-21T15:02:28.953752+0000","last_undegraded":"2024-09-21T15:02:28.953752+0000","last_fullsized":"2024-09-21T15:02:28.953752+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":593,"log_dups_size":0,"ondisk_log_size":593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:07:40.758932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001466743,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3111,"num_read_kb":36762,"num_write":889,"num_write_kb":39649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"228'493","reported_seq":4709,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.338915+0000","last_change":"2024-09-21T15:02:02.275784+0000","last_active":"2024-09-21T15:02:28.338915+0000","last_peered":"2024-09-21T15:02:28.338915+0000","last_clean":"2024-09-21T15:02:28.338915+0000","last_became_active":"2024-09-21T14:59:25.130057+0000","last_became_peered":"2024-09-21T14:59:25.130057+0000","last_unstale":"2024-09-21T15:02:28.338915+0000","last_undegraded":"2024-09-21T15:02:28.338915+0000","last_fullsized":"2024-09-21T15:02:28.338915+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:53:33.752550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014500649999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4513,"num_read_kb":28786,"num_write":729,"num_write_kb":24616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":9059600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"227'544","reported_seq":5354,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.957258+0000","last_change":"2024-09-21T15:02:02.409235+0000","last_active":"2024-09-21T15:02:28.957258+0000","last_peered":"2024-09-21T15:02:28.957258+0000","last_clean":"2024-09-21T15:02:28.957258+0000","last_became_active":"2024-09-21T14:58:19.407884+0000","last_became_peered":"2024-09-21T14:58:19.407884+0000","last_unstale":"2024-09-21T15:02:28.957258+0000","last_undegraded":"2024-09-21T15:02:28.957258+0000","last_fullsized":"2024-09-21T15:02:28.957258+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":544,"log_dups_size":0,"ondisk_log_size":544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:28:57.111615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045666499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4968,"num_read_kb":55828,"num_write":945,"num_write_kb":38650,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"230'879","reported_seq":3218,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.956772+0000","last_change":"2024-09-21T15:02:02.263156+0000","last_active":"2024-09-21T15:02:28.956772+0000","last_peered":"2024-09-21T15:02:28.956772+0000","last_clean":"2024-09-21T15:02:28.956772+0000","last_became_active":"2024-09-21T15:01:05.172427+0000","last_became_peered":"2024-09-21T15:01:05.172427+0000","last_unstale":"2024-09-21T15:02:28.956772+0000","last_undegraded":"2024-09-21T15:02:28.956772+0000","last_fullsized":"2024-09-21T15:02:28.956772+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":879,"log_dups_size":0,"ondisk_log_size":879,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:28:18.710621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001142789,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4262,"num_read_kb":17288,"num_write":1508,"num_write_kb":17970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":138,"num_bytes_recovered":7486496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"227'532","reported_seq":6209,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.339180+0000","last_change":"2024-09-21T15:02:02.275716+0000","last_active":"2024-09-21T15:02:28.339180+0000","last_peered":"2024-09-21T15:02:28.339180+0000","last_clean":"2024-09-21T15:02:28.339180+0000","last_became_active":"2024-09-21T14:58:19.410406+0000","last_became_peered":"2024-09-21T14:58:19.410406+0000","last_unstale":"2024-09-21T15:02:28.339180+0000","last_undegraded":"2024-09-21T15:02:28.339180+0000","last_fullsized":"2024-09-21T15:02:28.339180+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:59:16.026939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014059350000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5807,"num_read_kb":56273,"num_write":817,"num_write_kb":36773,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"228'523","reported_seq":4647,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.956536+0000","last_change":"2024-09-21T15:02:02.260617+0000","last_active":"2024-09-21T15:02:28.956536+0000","last_peered":"2024-09-21T15:02:28.956536+0000","last_clean":"2024-09-21T15:02:28.956536+0000","last_became_active":"2024-09-21T14:58:19.409935+0000","last_became_peered":"2024-09-21T14:58:19.409935+0000","last_unstale":"2024-09-21T15:02:28.956536+0000","last_undegraded":"2024-09-21T15:02:28.956536+0000","last_fullsized":"2024-09-21T15:02:28.956536+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":523,"log_dups_size":0,"ondisk_log_size":523,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:10:36.252629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068183199999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3949,"num_read_kb":58693,"num_write":810,"num_write_kb":37279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"230'623","reported_seq":5195,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.955624+0000","last_change":"2024-09-21T15:02:02.408072+0000","last_active":"2024-09-21T15:02:28.955624+0000","last_peered":"2024-09-21T15:02:28.955624+0000","last_clean":"2024-09-21T15:02:28.955624+0000","last_became_active":"2024-09-21T15:01:05.165955+0000","last_became_peered":"2024-09-21T15:01:05.165955+0000","last_unstale":"2024-09-21T15:02:28.955624+0000","last_undegraded":"2024-09-21T15:02:28.955624+0000","last_fullsized":"2024-09-21T15:02:28.955624+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":623,"log_dups_size":0,"ondisk_log_size":623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:34:54.111119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021463580000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5758,"num_read_kb":28485,"num_write":777,"num_write_kb":24185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":166,"num_bytes_recovered":11965028,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"229'644","reported_seq":8535,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.953028+0000","last_change":"2024-09-21T15:02:02.256934+0000","last_active":"2024-09-21T15:02:28.953028+0000","last_peered":"2024-09-21T15:02:28.953028+0000","last_clean":"2024-09-21T15:02:28.953028+0000","last_became_active":"2024-09-21T15:00:07.155144+0000","last_became_peered":"2024-09-21T15:00:07.155144+0000","last_unstale":"2024-09-21T15:02:28.953028+0000","last_undegraded":"2024-09-21T15:02:28.953028+0000","last_fullsized":"2024-09-21T15:02:28.953028+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":644,"log_dups_size":0,"ondisk_log_size":644,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:58:00.133639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010987029999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9100,"num_read_kb":37252,"num_write":1013,"num_write_kb":20896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"227'947","reported_seq":6822,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T15:02:28.955476+0000","last_change":"2024-09-21T15:02:02.406594+0000","last_active":"2024-09-21T15:02:28.955476+0000","last_peered":"2024-09-21T15:02:28.955476+0000","last_clean":"2024-09-21T15:02:28.955476+0000","last_became_active":"2024-09-21T14:58:19.412030+0000","last_became_peered":"2024-09-21T14:58:19.412030+0000","last_unstale":"2024-09-21T15:02:28.955476+0000","last_undegraded":"2024-09-21T15:02:28.955476+0000","last_fullsized":"2024-09-21T15:02:28.955476+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":947,"log_dups_size":0,"ondisk_log_size":947,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:44:29.282391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067099300000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8465,"num_read_kb":59792,"num_write":1327,"num_write_kb":31587,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":174039,"num_read_kb":1238236,"num_write":34305,"num_write_kb":940835,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3956,"num_bytes_recovered":212783659,"num_keys_recovered":305,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7385,"internal_metadata":0},"log_size":22541,"ondisk_log_size":22541,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":10404,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509500,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34088,"kb_used_data":2968,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94337752,"statfs":{"total":96636764160,"available":96601858048,"internally_reserved":0,"allocated":3039232,"data_stored":1498239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:02:06 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.83899999999999997,"15min":0.83699999999999997},"min":{"1min":0.29199999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.044,"5min":1.351,"15min":1.351},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.91400000000000003,"15min":0.94599999999999995},"min":{"1min":0.48099999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.0589999999999999,"5min":1.917,"15min":1.917},"last":1.21}]},{"osd":1,"last update":"Sat Sep 21 15:02:06 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.80800000000000005,"15min":0.77600000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":1.0409999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":1.3720000000000001},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.81599999999999995,"15min":0.82099999999999995},"min":{"1min":0.40500000000000003,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.002,"5min":2.2149999999999999,"15min":2.2149999999999999},"last":1.0409999999999999}]},{"osd":2,"last update":"Sat Sep 21 15:02:06 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.82899999999999996,"15min":0.81499999999999995},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.161,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":0.626},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.92600000000000005,"15min":0.93999999999999995},"min":{"1min":0.56499999999999995,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.0920000000000001,"5min":1.466,"15min":1.466},"last":0.78700000000000003}]},{"osd":3,"last update":"Sat Sep 21 15:02:06 2024","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.89100000000000001,"15min":0.86299999999999999},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.24,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.96799999999999997,"15min":0.96599999999999997},"min":{"1min":0.54200000000000004,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.159,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.95299999999999996}]},{"osd":4,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.92200000000000004,"15min":0.93300000000000005},"min":{"1min":0.42199999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.3089999999999999,"5min":2.7130000000000001,"15min":2.7130000000000001},"last":1.125},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.94499999999999995,"15min":0.92600000000000005},"min":{"1min":0.39000000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.425,"5min":2.125,"15min":2.125},"last":1.3280000000000001}]},{"osd":5,"last update":"Sat Sep 21 15:02:06 2024","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":1.006,"15min":1.016},"min":{"1min":0.505,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.375,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":1.069},{"interface":"front","average":{"1min":0.97699999999999998,"5min":1.026,"15min":1.0449999999999999},"min":{"1min":0.61599999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.425,"5min":3.1880000000000002,"15min":3.1880000000000002},"last":1.409}]},{"osd":6,"last update":"Sat Sep 21 15:02:06 2024","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.115,"15min":1.143},"min":{"1min":0.52900000000000003,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.679,"5min":3.2730000000000001,"15min":3.2730000000000001},"last":1.4530000000000001},{"interface":"front","average":{"1min":1.099,"5min":1.0800000000000001,"15min":1.036},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.6719999999999999,"5min":2.2879999999999998,"15min":2.2879999999999998},"last":1.1830000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542203,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64432,"kb_used_data":2972,"kb_used_omap":1,"kb_used_meta":61438,"kb_avail":94307408,"statfs":{"total":96636764160,"available":96570785792,"internally_reserved":0,"allocated":3043328,"data_stored":1498258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":62912967},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:02:09 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.87,"15min":0.93200000000000005},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.089,"5min":1.593,"15min":1.593},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.90500000000000003,"15min":0.94099999999999995},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.1180000000000001,"5min":1.587,"15min":1.587},"last":0.629}]},{"osd":1,"last update":"Sat Sep 21 15:02:09 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.876,"15min":0.90400000000000003},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.145,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.85099999999999998,"15min":0.94499999999999995},"min":{"1min":0.40600000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.036,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.69599999999999995}]},{"osd":2,"last update":"Sat Sep 21 15:02:09 2024","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.93600000000000005,"15min":0.97499999999999998},"min":{"1min":0.53400000000000003,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.194,"5min":1.758,"15min":1.758},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.878,"15min":0.90200000000000002},"min":{"1min":0.51700000000000002,"5min":0.496,"15min":0.496},"max":{"1min":1.002,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.89300000000000002}]},{"osd":3,"last update":"Sat Sep 21 15:02:09 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.91600000000000004,"15min":0.94799999999999995},"min":{"1min":0.57399999999999995,"5min":0.434,"15min":0.434},"max":{"1min":0.98699999999999999,"5min":1.835,"15min":1.835},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.90100000000000002,"5min":1.0429999999999999,"15min":1.111},"min":{"1min":0.65900000000000003,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.296,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.85999999999999999}]},{"osd":4,"last update":"Sat Sep 21 15:01:08 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.93500000000000005,"15min":0.99099999999999999},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":1.2,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.97799999999999998},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.98499999999999999,"15min":1.0209999999999999},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.3759999999999999,"5min":1.897,"15min":1.897},"last":0.93300000000000005}]},{"osd":5,"last update":"Sat Sep 21 15:02:09 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.97799999999999998,"15min":1.0880000000000001},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.222,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.85699999999999998,"5min":1.018,"15min":1.071},"min":{"1min":0.54200000000000004,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.3600000000000001,"5min":1.609,"15min":1.609},"last":1.0489999999999999}]},{"osd":7,"last update":"Sat Sep 21 15:02:09 2024","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":1.131,"15min":1.2190000000000001},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.256,"5min":1.8660000000000001,"15min":1.8660000000000001},"last":1.079},{"interface":"front","average":{"1min":0.91300000000000003,"5min":1.117,"15min":1.248},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.413,"5min":2.004,"15min":2.004},"last":1.004}]}]},{"osd":5,"up_from":14,"seq":60129542204,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60384,"kb_used_data":3028,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94311456,"statfs":{"total":96636764160,"available":96574930944,"internally_reserved":0,"allocated":3100672,"data_stored":2088626,"data_compressed":5202,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:02:09 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.84199999999999997,"15min":0.96899999999999997},"min":{"1min":0.40100000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.98099999999999998,"5min":4.4800000000000004,"15min":4.4800000000000004},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.78400000000000003,"15min":0.81299999999999994},"min":{"1min":0.44500000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.179,"5min":1.181,"15min":1.181},"last":0.70399999999999996}]},{"osd":1,"last update":"Sat Sep 21 15:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.86399999999999999,"15min":0.91300000000000003},"min":{"1min":0.41899999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.036,"5min":2.7709999999999999,"15min":2.7709999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.89900000000000002,"15min":0.92900000000000005},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.1439999999999999,"5min":2.823,"15min":2.823},"last":0.78800000000000003}]},{"osd":2,"last update":"Sat Sep 21 15:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.86599999999999999,"15min":0.95799999999999996},"min":{"1min":0.53600000000000003,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.2090000000000001,"5min":3.419,"15min":3.419},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.84499999999999997,"15min":0.88800000000000001},"min":{"1min":0.58999999999999997,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.1699999999999999,"5min":1.423,"15min":1.423},"last":1.0569999999999999}]},{"osd":3,"last update":"Sat Sep 21 15:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.81100000000000005,"15min":0.81000000000000005},"min":{"1min":0.51000000000000001,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.105,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.91700000000000004,"5min":1.018,"15min":1.0720000000000001},"min":{"1min":0.66600000000000004,"5min":0.435,"15min":0.435},"max":{"1min":1.4239999999999999,"5min":2.6379999999999999,"15min":2.6379999999999999},"last":0.89600000000000002}]},{"osd":4,"last update":"Sat Sep 21 15:01:02 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.98899999999999999,"15min":1.016},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.5609999999999999,"5min":2.5470000000000002,"15min":2.5470000000000002},"last":1.0229999999999999},{"interface":"front","average":{"1min":0.82099999999999995,"5min":1.0169999999999999,"15min":1.0780000000000001},"min":{"1min":0.39300000000000002,"5min":0.376,"15min":0.376},"max":{"1min":1.302,"5min":2.5099999999999998,"15min":2.5099999999999998},"last":1.1779999999999999}]},{"osd":6,"last update":"Sat Sep 21 15:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.91700000000000004,"15min":0.95999999999999996},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.1499999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":1.256},{"interface":"front","average":{"1min":0.879,"5min":1.1399999999999999,"15min":1.3169999999999999},"min":{"1min":0.60199999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.498,"5min":4.2469999999999999,"15min":4.2469999999999999},"last":1.1020000000000001}]},{"osd":7,"last update":"Sat Sep 21 15:02:09 2024","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":1.167,"15min":1.3120000000000001},"min":{"1min":0.61199999999999999,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.5700000000000001,"5min":3.3010000000000002,"15min":3.3010000000000002},"last":1.2110000000000001},{"interface":"front","average":{"1min":0.99399999999999999,"5min":1.075,"15min":1.125},"min":{"1min":0.64300000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.25,"5min":2.5880000000000001,"15min":2.5880000000000001},"last":1.323}]}]},{"osd":4,"up_from":235,"seq":1009317314568,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34860,"kb_used_data":2968,"kb_used_omap":8,"kb_used_meta":31863,"kb_avail":94336980,"statfs":{"total":96636764160,"available":96601067520,"internally_reserved":0,"allocated":3039232,"data_stored":1498239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8979,"internal_metadata":32627949},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95899999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86499999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57399999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.749}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2130000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.024}]}]},{"osd":3,"up_from":14,"seq":60129542204,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78376,"kb_used_data":2972,"kb_used_omap":1,"kb_used_meta":75390,"kb_avail":94293464,"statfs":{"total":96636764160,"available":96556507136,"internally_reserved":0,"allocated":3043328,"data_stored":1498258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":77199816},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.61799999999999999,"15min":0.63200000000000001},"min":{"1min":0.34599999999999997,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.92700000000000005,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.68200000000000005,"15min":0.70999999999999996},"min":{"1min":0.40500000000000003,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.3029999999999999,"5min":1.919,"15min":1.919},"last":0.628}]},{"osd":1,"last update":"Sat Sep 21 15:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.65800000000000003,"15min":0.67000000000000004},"min":{"1min":0.44400000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":1.115,"5min":1.722,"15min":1.722},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.746,"15min":0.79700000000000004},"min":{"1min":0.41199999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.2509999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.65600000000000003}]},{"osd":2,"last update":"Sat Sep 21 15:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.746,"15min":0.72999999999999998},"min":{"1min":0.38200000000000001,"5min":0.311,"15min":0.311},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.746,"5min":0.79200000000000004,"15min":0.82099999999999995},"min":{"1min":0.40300000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.94399999999999995,"5min":1.254,"15min":1.254},"last":0.46400000000000002}]},{"osd":4,"last update":"Sat Sep 21 15:01:09 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.93799999999999994,"15min":0.95599999999999996},"min":{"1min":0.54300000000000004,"5min":0.442,"15min":0.442},"max":{"1min":1.2629999999999999,"5min":1.7609999999999999,"15min":1.7609999999999999},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.98899999999999999,"15min":1.0269999999999999},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.234,"5min":2.028,"15min":2.028},"last":0.86599999999999999}]},{"osd":5,"last update":"Sat Sep 21 15:02:13 2024","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.036,"15min":1.052},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.4670000000000001,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":0.77000000000000002},{"interface":"front","average":{"1min":1.081,"5min":1.0589999999999999,"15min":1.0640000000000001},"min":{"1min":0.68799999999999994,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.395,"5min":1.548,"15min":1.548},"last":0.90000000000000002}]},{"osd":6,"last update":"Sat Sep 21 15:02:13 2024","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":1.046,"15min":1.085},"min":{"1min":0.70099999999999996,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.391,"5min":2.1219999999999999,"15min":2.1219999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":1.03,"5min":1.075,"15min":1.0980000000000001},"min":{"1min":0.64900000000000002,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.448,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":0.67400000000000004}]},{"osd":7,"last update":"Sat Sep 21 15:02:13 2024","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.0800000000000001,"15min":1.123},"min":{"1min":0.746,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.339,"5min":1.706,"15min":1.706},"last":0.93000000000000005},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.1279999999999999,"15min":1.171},"min":{"1min":0.67700000000000005,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.421,"5min":2.3380000000000001,"15min":2.3380000000000001},"last":0.96899999999999997}]}]},{"osd":2,"up_from":14,"seq":60129542202,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70180,"kb_used_data":2968,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94301660,"statfs":{"total":96636764160,"available":96564899840,"internally_reserved":0,"allocated":3039232,"data_stored":1498239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68811206},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:02:07 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.63500000000000001,"15min":0.625},"min":{"1min":0.32100000000000001,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.383,"5min":1.383,"15min":1.383},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.69099999999999995,"15min":0.66900000000000004},"min":{"1min":0.30099999999999999,"5min":0.223,"15min":0.223},"max":{"1min":1.1739999999999999,"5min":2.9169999999999998,"15min":2.9169999999999998},"last":0.64400000000000002}]},{"osd":1,"last update":"Sat Sep 21 15:02:07 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.67500000000000004,"15min":0.68100000000000005},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.129,"5min":2.6520000000000001,"15min":2.6520000000000001},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.73399999999999999,"15min":0.75800000000000001},"min":{"1min":0.36199999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.46800000000000003}]},{"osd":3,"last update":"Sat Sep 21 15:02:07 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.71699999999999997,"15min":0.72699999999999998},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.996,"5min":3.0110000000000001,"15min":3.0110000000000001},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.85999999999999999,"15min":0.83399999999999996},"min":{"1min":0.41099999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0369999999999999,"5min":4.1440000000000001,"15min":4.1440000000000001},"last":0.83899999999999997}]},{"osd":4,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.93200000000000005,"15min":0.95599999999999996},"min":{"1min":0.42399999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.222,"5min":2.899,"15min":2.899},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.94399999999999995,"15min":0.95799999999999996},"min":{"1min":0.47499999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.4379999999999999,"5min":1.675,"15min":1.675},"last":0.97199999999999998}]},{"osd":5,"last update":"Sat Sep 21 15:02:07 2024","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.98699999999999999,"15min":1.0009999999999999},"min":{"1min":0.45500000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.224,"5min":3.1080000000000001,"15min":3.1080000000000001},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.96399999999999997,"15min":0.98799999999999999},"min":{"1min":0.55200000000000005,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.228,"5min":1.546,"15min":1.546},"last":1.2130000000000001}]},{"osd":6,"last update":"Sat Sep 21 15:02:07 2024","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":0.98899999999999999,"15min":0.98699999999999999},"min":{"1min":0.52100000000000002,"5min":0.378,"15min":0.378},"max":{"1min":1.516,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.93999999999999995,"5min":1.0429999999999999,"15min":1.048},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.403,"5min":3.0739999999999998,"15min":3.0739999999999998},"last":1.02}]},{"osd":7,"last update":"Sat Sep 21 15:02:09 2024","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.0489999999999999,"15min":1.01},"min":{"1min":0.61399999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.415,"5min":3.1589999999999998,"15min":3.1589999999999998},"last":1.0649999999999999},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.111,"15min":1.095},"min":{"1min":0.58799999999999997,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.4610000000000001,"5min":3.1400000000000001,"15min":3.1400000000000001},"last":1.103}]}]},{"osd":1,"up_from":14,"seq":60129542205,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48104,"kb_used_data":3028,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323736,"statfs":{"total":96636764160,"available":96587505664,"internally_reserved":0,"allocated":3100672,"data_stored":2088626,"data_compressed":5202,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.58899999999999997,"15min":0.60699999999999998},"min":{"1min":0.25600000000000001,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.60299999999999998,"15min":0.60999999999999999},"min":{"1min":0.26300000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.83499999999999996,"5min":1.107,"15min":1.107},"last":0.88}]},{"osd":2,"last update":"Sat Sep 21 15:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.78000000000000003,"15min":0.73499999999999999},"min":{"1min":0.38800000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.92100000000000004,"5min":11.303000000000001,"15min":11.303000000000001},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.71399999999999997,"15min":0.70299999999999996},"min":{"1min":0.439,"5min":0.435,"15min":0.435},"max":{"1min":1.0840000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.60299999999999998}]},{"osd":3,"last update":"Sat Sep 21 15:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.80200000000000005,"15min":0.83399999999999996},"min":{"1min":0.36899999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.4850000000000001,"5min":3.1469999999999998,"15min":3.1469999999999998},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.83699999999999997,"15min":0.877},"min":{"1min":0.42499999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.284,"5min":3.2320000000000002,"15min":3.2320000000000002},"last":0.73599999999999999}]},{"osd":4,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.89000000000000001,"15min":0.91500000000000004},"min":{"1min":0.59599999999999997,"5min":0.309,"15min":0.309},"max":{"1min":1.45,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":1.0149999999999999},{"interface":"front","average":{"1min":1.5600000000000001,"5min":1.069,"15min":0.96599999999999997},"min":{"1min":0.504,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":11.563000000000001,"5min":11.563000000000001,"15min":11.563000000000001},"last":0.85399999999999998}]},{"osd":5,"last update":"Sat Sep 21 15:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":1.0620000000000001,"15min":0.95399999999999996},"min":{"1min":0.61199999999999999,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.3799999999999999,"5min":11.534000000000001,"15min":11.534000000000001},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.88,"5min":0.94199999999999995,"15min":0.95599999999999996},"min":{"1min":0.58599999999999997,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.1319999999999999,"5min":1.484,"15min":1.484},"last":0.93300000000000005}]},{"osd":6,"last update":"Sat Sep 21 15:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.97999999999999998,"15min":1.038},"min":{"1min":0.53100000000000003,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.254,"5min":2.3519999999999999,"15min":2.3519999999999999},"last":1.2769999999999999},{"interface":"front","average":{"1min":0.97499999999999998,"5min":1.087,"15min":0.995},"min":{"1min":0.66400000000000003,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.4259999999999999,"5min":11.601000000000001,"15min":11.601000000000001},"last":1.0880000000000001}]},{"osd":7,"last update":"Sat Sep 21 15:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":1.02,"15min":1.056},"min":{"1min":0.58899999999999997,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.4039999999999999,"5min":2.2360000000000002,"15min":2.2360000000000002},"last":1.337},{"interface":"front","average":{"1min":1.044,"5min":1.171,"15min":1.087},"min":{"1min":0.63100000000000001,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.3799999999999999,"5min":11.489000000000001,"15min":11.489000000000001},"last":1.129}]}]},{"osd":0,"up_from":14,"seq":60129542202,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70184,"kb_used_data":2968,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94301656,"statfs":{"total":96636764160,"available":96564895744,"internally_reserved":0,"allocated":3039232,"data_stored":1498239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":68811207},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 21 15:02:07 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.64000000000000001,"15min":0.70599999999999996},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.002,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.69499999999999995,"15min":0.63200000000000001},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.98699999999999999,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.745}]},{"osd":2,"last update":"Sat Sep 21 15:02:07 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.66100000000000003,"15min":0.621},"min":{"1min":0.33900000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.0960000000000001,"5min":1.153,"15min":1.153},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.70099999999999996,"15min":0.75},"min":{"1min":0.28799999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.0329999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.58299999999999996}]},{"osd":3,"last update":"Sat Sep 21 15:02:07 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.76400000000000001,"15min":0.754},"min":{"1min":0.37,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.77400000000000002,"15min":0.81100000000000005},"min":{"1min":0.42799999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.04,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.88300000000000001}]},{"osd":4,"last update":"Sat Sep 21 15:01:04 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.98399999999999999,"15min":1.038},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.1140000000000001,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":0.71899999999999997},{"interface":"front","average":{"1min":1.036,"5min":1.0469999999999999,"15min":1.0620000000000001},"min":{"1min":0.67800000000000005,"5min":0.379,"15min":0.379},"max":{"1min":1.446,"5min":1.665,"15min":1.665},"last":0.69899999999999995}]},{"osd":5,"last update":"Sat Sep 21 15:02:07 2024","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.997,"15min":1.0229999999999999},"min":{"1min":0.45800000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.296,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.96099999999999997},{"interface":"front","average":{"1min":0.94299999999999995,"5min":1.0129999999999999,"15min":1.0669999999999999},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.3919999999999999,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":1.0409999999999999}]},{"osd":6,"last update":"Sat Sep 21 15:02:07 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.98799999999999999,"15min":1.089},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.119,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.94799999999999995,"5min":1.038,"15min":1.0600000000000001},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.393,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.79300000000000004}]},{"osd":7,"last update":"Sat Sep 21 15:02:07 2024","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":1.012,"15min":1.05},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.1719999999999999,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":0.99399999999999999},{"interface":"front","average":{"1min":0.88300000000000001,"5min":1.0309999999999999,"15min":1.113},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.28,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":1.0189999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5202,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5202,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7385,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T15:02:41.516 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-09-21T14:58:18.379818+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=58, tm_sec=18, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=2, tm_sec=36, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:02:41.516 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-09-21T14:58:18.379818+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=58, tm_sec=18, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=2, tm_sec=36, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:02:41.516 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-09-21T14:58:18.379818+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=58, tm_sec=18, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=2, tm_sec=36, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:02:41.517 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-09-21T14:58:18.379818+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=58, tm_sec=18, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=2, tm_sec=36, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:02:41.517 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-09-21T14:58:18.379818+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=58, tm_sec=18, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=2, tm_sec=36, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:02:41.517 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-09-21T14:58:18.379818+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=58, tm_sec=18, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=2, tm_sec=36, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:02:41.517 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-09-21T14:58:18.379818+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=58, tm_sec=18, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=2, tm_sec=36, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:02:41.517 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-09-21T14:58:18.379818+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=58, tm_sec=18, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=2, tm_sec=36, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:02:41.517 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-09-21T14:58:18.379818+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=58, tm_sec=18, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=2, tm_sec=36, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:02:41.517 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-09-21T14:58:18.379818+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=58, tm_sec=18, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=2, tm_sec=36, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:02:41.517 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-09-21T14:58:18.379818+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=58, tm_sec=18, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=2, tm_sec=36, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:02:41.518 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-09-21T14:58:18.379818+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=58, tm_sec=18, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=2, tm_sec=36, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:02:41.518 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-21T14:58:08.305870+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=58, tm_sec=8, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=2, tm_sec=36, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:02:41.518 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-09-21T14:58:18.379818+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=58, tm_sec=18, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=2, tm_sec=36, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:02:41.518 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-21T14:58:08.305870+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=58, tm_sec=8, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=2, tm_sec=36, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:02:41.518 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-09-21T14:58:18.379818+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=58, tm_sec=18, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=2, tm_sec=36, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:02:41.518 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-21T14:58:08.305870+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=58, tm_sec=8, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=2, tm_sec=36, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:02:41.518 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-09-21T14:58:18.379818+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=58, tm_sec=18, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=2, tm_sec=36, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:02:41.518 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-21T14:58:08.305870+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=58, tm_sec=8, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=2, tm_sec=36, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:02:41.518 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-09-21T14:58:18.379818+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=58, tm_sec=18, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=2, tm_sec=36, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:02:41.518 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-09-21T14:58:18.379818+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=58, tm_sec=18, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=2, tm_sec=36, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:02:41.519 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-21T14:58:08.305870+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=58, tm_sec=8, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=2, tm_sec=36, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:02:41.519 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-09-21T14:58:18.379818+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=58, tm_sec=18, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=2, tm_sec=36, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:02:41.519 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-09-21T14:58:08.305870+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=58, tm_sec=8, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=2, tm_sec=36, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:02:41.519 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-09-21T14:58:18.379818+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=58, tm_sec=18, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=2, tm_sec=36, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:02:41.519 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-21T14:58:08.305870+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=58, tm_sec=8, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=2, tm_sec=36, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:02:41.519 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-09-21T14:58:18.379818+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=58, tm_sec=18, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=2, tm_sec=36, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:02:41.519 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-21T14:58:08.305870+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=58, tm_sec=8, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=2, tm_sec=36, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:02:41.519 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-21T14:58:02.180162+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=58, tm_sec=2, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=2, tm_sec=36, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:02:41.519 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-09-21T14:58:18.379818+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=58, tm_sec=18, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=2, tm_sec=36, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:02:41.520 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-09-21T14:58:18.379818+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=58, tm_sec=18, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=2, tm_sec=36, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:02:41.520 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-09-21T14:58:18.379818+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=58, tm_sec=18, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=2, tm_sec=36, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:02:41.520 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-09-21T14:58:18.379818+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=58, tm_sec=18, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=2, tm_sec=36, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:02:41.520 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-09-21T14:58:18.379818+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=58, tm_sec=18, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=2, tm_sec=36, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:02:41.520 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-09-21T14:58:18.379818+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=58, tm_sec=18, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=2, tm_sec=36, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:02:41.520 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-09-21T14:58:18.379818+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=58, tm_sec=18, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=2, tm_sec=36, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:02:41.520 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-09-21T14:58:18.379818+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=58, tm_sec=18, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=2, tm_sec=36, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:02:41.520 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-09-21T14:58:18.379818+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=58, tm_sec=18, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=2, tm_sec=36, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:02:41.520 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-09-21T14:58:18.379818+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=58, tm_sec=18, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=2, tm_sec=36, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:02:41.521 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-09-21T14:58:18.379818+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=58, tm_sec=18, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=2, tm_sec=36, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:02:41.521 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-09-21T14:58:18.379818+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=58, tm_sec=18, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=2, tm_sec=36, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:02:41.521 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-21T15:03:01.521 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T15:03:01.846 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-21T15:03:01.849 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-09-21T15:03:01.861 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":394,"stamp":"2024-09-21T15:02:59.856001+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":174155,"num_read_kb":1238336,"num_write":34466,"num_write_kb":943253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3956,"num_bytes_recovered":212783659,"num_keys_recovered":305,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22668,"ondisk_log_size":22668,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":464640,"kb_used_data":23872,"kb_used_omap":19,"kb_used_meta":440684,"kb_avail":754510080,"statfs":{"total":773094113280,"available":772618321920,"internally_reserved":0,"allocated":24444928,"data_stored":13166724,"data_compressed":10404,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":20129,"internal_metadata":451260767},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003907"},"pg_stats":[{"pgid":"3.1f","version":"230'598","reported_seq":3991,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:50.506670+0000","last_change":"2024-09-21T15:02:50.506935+0000","last_active":"2024-09-21T15:02:50.506670+0000","last_peered":"2024-09-21T15:02:50.506670+0000","last_clean":"2024-09-21T15:02:50.506670+0000","last_became_active":"2024-09-21T15:02:24.308451+0000","last_became_peered":"2024-09-21T15:02:24.308451+0000","last_unstale":"2024-09-21T15:02:50.506670+0000","last_undegraded":"2024-09-21T15:02:50.506670+0000","last_fullsized":"2024-09-21T15:02:50.506670+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"230'598","last_scrub_stamp":"2024-09-21T15:02:50.506616+0000","last_deep_scrub":"230'598","last_deep_scrub_stamp":"2024-09-21T15:02:50.506616+0000","last_clean_scrub_stamp":"2024-09-21T15:02:50.506616+0000","objects_scrubbed":0,"log_size":598,"log_dups_size":0,"ondisk_log_size":598,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:11:45.796964+0000","scrub_duration":0.0038622230000000001,"objects_trimmed":0,"snaptrim_duration":0.0010410479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3756,"num_read_kb":43219,"num_write":975,"num_write_kb":36436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"230'507","reported_seq":3522,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:47.523706+0000","last_change":"2024-09-21T15:02:47.524034+0000","last_active":"2024-09-21T15:02:47.523706+0000","last_peered":"2024-09-21T15:02:47.523706+0000","last_clean":"2024-09-21T15:02:47.523706+0000","last_became_active":"2024-09-21T15:02:24.309506+0000","last_became_peered":"2024-09-21T15:02:24.309506+0000","last_unstale":"2024-09-21T15:02:47.523706+0000","last_undegraded":"2024-09-21T15:02:47.523706+0000","last_fullsized":"2024-09-21T15:02:47.523706+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"230'507","last_scrub_stamp":"2024-09-21T15:02:47.523655+0000","last_deep_scrub":"230'507","last_deep_scrub_stamp":"2024-09-21T15:02:47.523655+0000","last_clean_scrub_stamp":"2024-09-21T15:02:47.523655+0000","objects_scrubbed":0,"log_size":507,"log_dups_size":0,"ondisk_log_size":507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:56:11.657252+0000","scrub_duration":0.0038311500000000002,"objects_trimmed":0,"snaptrim_duration":0.00094912299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3420,"num_read_kb":23407,"num_write":643,"num_write_kb":18445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":148,"num_bytes_recovered":11635695,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"228'341","reported_seq":4802,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:55.751846+0000","last_change":"2024-09-21T15:02:55.752127+0000","last_active":"2024-09-21T15:02:55.751846+0000","last_peered":"2024-09-21T15:02:55.751846+0000","last_clean":"2024-09-21T15:02:55.751846+0000","last_became_active":"2024-09-21T14:58:19.411387+0000","last_became_peered":"2024-09-21T14:58:19.411387+0000","last_unstale":"2024-09-21T15:02:55.751846+0000","last_undegraded":"2024-09-21T15:02:55.751846+0000","last_fullsized":"2024-09-21T15:02:55.751846+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"228'341","last_scrub_stamp":"2024-09-21T15:02:55.751791+0000","last_deep_scrub":"228'341","last_deep_scrub_stamp":"2024-09-21T15:02:55.751791+0000","last_clean_scrub_stamp":"2024-09-21T15:02:55.751791+0000","objects_scrubbed":0,"log_size":341,"log_dups_size":0,"ondisk_log_size":341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:43:49.694451+0000","scrub_duration":0.0036333149999999998,"objects_trimmed":0,"snaptrim_duration":0.00045388299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3878,"num_read_kb":28288,"num_write":535,"num_write_kb":26606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"251'573","reported_seq":4319,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:53.920933+0000","last_change":"2024-09-21T15:02:53.921166+0000","last_active":"2024-09-21T15:02:53.920933+0000","last_peered":"2024-09-21T15:02:53.920933+0000","last_clean":"2024-09-21T15:02:53.920933+0000","last_became_active":"2024-09-21T14:58:19.411306+0000","last_became_peered":"2024-09-21T14:58:19.411306+0000","last_unstale":"2024-09-21T15:02:53.920933+0000","last_undegraded":"2024-09-21T15:02:53.920933+0000","last_fullsized":"2024-09-21T15:02:53.920933+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"251'573","last_scrub_stamp":"2024-09-21T15:02:53.920881+0000","last_deep_scrub":"251'573","last_deep_scrub_stamp":"2024-09-21T15:02:53.920881+0000","last_clean_scrub_stamp":"2024-09-21T15:02:53.920881+0000","objects_scrubbed":1,"log_size":573,"log_dups_size":0,"ondisk_log_size":573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:24:58.256436+0000","scrub_duration":0.0067313260000000001,"objects_trimmed":0,"snaptrim_duration":0.0011721469999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4050,"num_read_kb":24676,"num_write":863,"num_write_kb":25228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"227'536","reported_seq":4305,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:41.892311+0000","last_change":"2024-09-21T15:02:41.892553+0000","last_active":"2024-09-21T15:02:41.892311+0000","last_peered":"2024-09-21T15:02:41.892311+0000","last_clean":"2024-09-21T15:02:41.892311+0000","last_became_active":"2024-09-21T15:02:17.242801+0000","last_became_peered":"2024-09-21T15:02:17.242801+0000","last_unstale":"2024-09-21T15:02:41.892311+0000","last_undegraded":"2024-09-21T15:02:41.892311+0000","last_fullsized":"2024-09-21T15:02:41.892311+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"227'536","last_scrub_stamp":"2024-09-21T15:02:41.892260+0000","last_deep_scrub":"227'536","last_deep_scrub_stamp":"2024-09-21T15:02:41.892260+0000","last_clean_scrub_stamp":"2024-09-21T15:02:41.892260+0000","objects_scrubbed":0,"log_size":536,"log_dups_size":0,"ondisk_log_size":536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:02:55.660386+0000","scrub_duration":0.0035449259999999999,"objects_trimmed":0,"snaptrim_duration":0.00032550800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3672,"num_read_kb":32698,"num_write":706,"num_write_kb":22582,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":379,"num_bytes_recovered":13616631,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"251'488","reported_seq":4870,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:45.888094+0000","last_change":"2024-09-21T15:02:45.888473+0000","last_active":"2024-09-21T15:02:45.888094+0000","last_peered":"2024-09-21T15:02:45.888094+0000","last_clean":"2024-09-21T15:02:45.888094+0000","last_became_active":"2024-09-21T14:58:19.409415+0000","last_became_peered":"2024-09-21T14:58:19.409415+0000","last_unstale":"2024-09-21T15:02:45.888094+0000","last_undegraded":"2024-09-21T15:02:45.888094+0000","last_fullsized":"2024-09-21T15:02:45.888094+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"251'488","last_scrub_stamp":"2024-09-21T15:02:45.888038+0000","last_deep_scrub":"251'488","last_deep_scrub_stamp":"2024-09-21T15:02:45.888038+0000","last_clean_scrub_stamp":"2024-09-21T15:02:45.888038+0000","objects_scrubbed":1,"log_size":488,"log_dups_size":0,"ondisk_log_size":488,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:11:04.964634+0000","scrub_duration":0.0079002050000000004,"objects_trimmed":0,"snaptrim_duration":0.0019572090000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4389,"num_read_kb":28349,"num_write":704,"num_write_kb":23876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"228'824","reported_seq":8743,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:44.910392+0000","last_change":"2024-09-21T15:02:44.910761+0000","last_active":"2024-09-21T15:02:44.910392+0000","last_peered":"2024-09-21T15:02:44.910392+0000","last_clean":"2024-09-21T15:02:44.910392+0000","last_became_active":"2024-09-21T15:02:18.246987+0000","last_became_peered":"2024-09-21T15:02:18.246987+0000","last_unstale":"2024-09-21T15:02:44.910392+0000","last_undegraded":"2024-09-21T15:02:44.910392+0000","last_fullsized":"2024-09-21T15:02:44.910392+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"228'824","last_scrub_stamp":"2024-09-21T15:02:44.910267+0000","last_deep_scrub":"228'824","last_deep_scrub_stamp":"2024-09-21T15:02:44.910267+0000","last_clean_scrub_stamp":"2024-09-21T15:02:44.910267+0000","objects_scrubbed":0,"log_size":824,"log_dups_size":0,"ondisk_log_size":824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:59:01.552553+0000","scrub_duration":0.0041170110000000003,"objects_trimmed":0,"snaptrim_duration":0.0014776850000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9383,"num_read_kb":68058,"num_write":1251,"num_write_kb":29921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":394,"num_bytes_recovered":23509073,"num_keys_recovered":66,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"227'688","reported_seq":8805,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:43.600624+0000","last_change":"2024-09-21T15:02:43.600925+0000","last_active":"2024-09-21T15:02:43.600624+0000","last_peered":"2024-09-21T15:02:43.600624+0000","last_clean":"2024-09-21T15:02:43.600624+0000","last_became_active":"2024-09-21T14:58:19.409371+0000","last_became_peered":"2024-09-21T14:58:19.409371+0000","last_unstale":"2024-09-21T15:02:43.600624+0000","last_undegraded":"2024-09-21T15:02:43.600624+0000","last_fullsized":"2024-09-21T15:02:43.600624+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"227'688","last_scrub_stamp":"2024-09-21T15:02:43.600574+0000","last_deep_scrub":"227'688","last_deep_scrub_stamp":"2024-09-21T15:02:43.600574+0000","last_clean_scrub_stamp":"2024-09-21T15:02:43.600574+0000","objects_scrubbed":0,"log_size":688,"log_dups_size":0,"ondisk_log_size":688,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:50:52.626013+0000","scrub_duration":0.0037969309999999999,"objects_trimmed":0,"snaptrim_duration":0.00052048100000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9236,"num_read_kb":45266,"num_write":998,"num_write_kb":30616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"227'413","reported_seq":4017,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:41.571195+0000","last_change":"2024-09-21T15:02:41.571486+0000","last_active":"2024-09-21T15:02:41.571195+0000","last_peered":"2024-09-21T15:02:41.571195+0000","last_clean":"2024-09-21T15:02:41.571195+0000","last_became_active":"2024-09-21T14:58:19.410306+0000","last_became_peered":"2024-09-21T14:58:19.410306+0000","last_unstale":"2024-09-21T15:02:41.571195+0000","last_undegraded":"2024-09-21T15:02:41.571195+0000","last_fullsized":"2024-09-21T15:02:41.571195+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"227'413","last_scrub_stamp":"2024-09-21T15:02:41.571143+0000","last_deep_scrub":"227'413","last_deep_scrub_stamp":"2024-09-21T15:02:41.571143+0000","last_clean_scrub_stamp":"2024-09-21T15:02:41.571143+0000","objects_scrubbed":0,"log_size":413,"log_dups_size":0,"ondisk_log_size":413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:56:07.400850+0000","scrub_duration":0.0040871240000000001,"objects_trimmed":0,"snaptrim_duration":0.00065724399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3380,"num_read_kb":28207,"num_write":769,"num_write_kb":34633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"229'1053","reported_seq":6187,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:52.403113+0000","last_change":"2024-09-21T15:02:52.403417+0000","last_active":"2024-09-21T15:02:52.403113+0000","last_peered":"2024-09-21T15:02:52.403113+0000","last_clean":"2024-09-21T15:02:52.403113+0000","last_became_active":"2024-09-21T15:01:06.166756+0000","last_became_peered":"2024-09-21T15:01:06.166756+0000","last_unstale":"2024-09-21T15:02:52.403113+0000","last_undegraded":"2024-09-21T15:02:52.403113+0000","last_fullsized":"2024-09-21T15:02:52.403113+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"229'1053","last_scrub_stamp":"2024-09-21T15:02:52.403056+0000","last_deep_scrub":"229'1053","last_deep_scrub_stamp":"2024-09-21T15:02:52.403056+0000","last_clean_scrub_stamp":"2024-09-21T15:02:52.403056+0000","objects_scrubbed":0,"log_size":1053,"log_dups_size":0,"ondisk_log_size":1053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:20:25.468011+0000","scrub_duration":0.003478456,"objects_trimmed":0,"snaptrim_duration":0.001157057,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8073,"num_read_kb":43999,"num_write":1621,"num_write_kb":28374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":271,"num_bytes_recovered":16821266,"num_keys_recovered":55,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"230'823","reported_seq":3991,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:43.000477+0000","last_change":"2024-09-21T15:02:43.000714+0000","last_active":"2024-09-21T15:02:43.000477+0000","last_peered":"2024-09-21T15:02:43.000477+0000","last_clean":"2024-09-21T15:02:43.000477+0000","last_became_active":"2024-09-21T14:58:19.410687+0000","last_became_peered":"2024-09-21T14:58:19.410687+0000","last_unstale":"2024-09-21T15:02:43.000477+0000","last_undegraded":"2024-09-21T15:02:43.000477+0000","last_fullsized":"2024-09-21T15:02:43.000477+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"230'823","last_scrub_stamp":"2024-09-21T15:02:43.000429+0000","last_deep_scrub":"230'823","last_deep_scrub_stamp":"2024-09-21T15:02:43.000429+0000","last_clean_scrub_stamp":"2024-09-21T15:02:43.000429+0000","objects_scrubbed":0,"log_size":823,"log_dups_size":0,"ondisk_log_size":823,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:26:50.871886+0000","scrub_duration":0.003893999,"objects_trimmed":0,"snaptrim_duration":0.0017043010000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4757,"num_read_kb":49770,"num_write":1348,"num_write_kb":33339,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"228'527","reported_seq":4194,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:29.771625+0000","last_change":"2024-09-21T15:02:02.408062+0000","last_active":"2024-09-21T15:02:29.771625+0000","last_peered":"2024-09-21T15:02:29.771625+0000","last_clean":"2024-09-21T15:02:29.771625+0000","last_became_active":"2024-09-21T14:58:19.407310+0000","last_became_peered":"2024-09-21T14:58:19.407310+0000","last_unstale":"2024-09-21T15:02:29.771625+0000","last_undegraded":"2024-09-21T15:02:29.771625+0000","last_fullsized":"2024-09-21T15:02:29.771625+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":527,"log_dups_size":0,"ondisk_log_size":527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016534340000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3756,"num_read_kb":43322,"num_write":713,"num_write_kb":33471,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":257,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:42.857085+0000","last_change":"2024-09-21T15:02:42.857348+0000","last_active":"2024-09-21T15:02:42.857085+0000","last_peered":"2024-09-21T15:02:42.857085+0000","last_clean":"2024-09-21T15:02:42.857085+0000","last_became_active":"2024-09-21T14:58:09.328417+0000","last_became_peered":"2024-09-21T14:58:09.328417+0000","last_unstale":"2024-09-21T15:02:42.857085+0000","last_undegraded":"2024-09-21T15:02:42.857085+0000","last_fullsized":"2024-09-21T15:02:42.857085+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:02:42.857039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:02:42.857039+0000","last_clean_scrub_stamp":"2024-09-21T15:02:42.857039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:46:28.753658+0000","scrub_duration":0.003187479,"objects_trimmed":0,"snaptrim_duration":0.0018917490000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"230'610","reported_seq":6579,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:42.879847+0000","last_change":"2024-09-21T15:02:42.880263+0000","last_active":"2024-09-21T15:02:42.879847+0000","last_peered":"2024-09-21T15:02:42.879847+0000","last_clean":"2024-09-21T15:02:42.879847+0000","last_became_active":"2024-09-21T15:02:17.247583+0000","last_became_peered":"2024-09-21T15:02:17.247583+0000","last_unstale":"2024-09-21T15:02:42.879847+0000","last_undegraded":"2024-09-21T15:02:42.879847+0000","last_fullsized":"2024-09-21T15:02:42.879847+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"230'610","last_scrub_stamp":"2024-09-21T15:02:42.879794+0000","last_deep_scrub":"230'610","last_deep_scrub_stamp":"2024-09-21T15:02:42.879794+0000","last_clean_scrub_stamp":"2024-09-21T15:02:42.879794+0000","objects_scrubbed":0,"log_size":610,"log_dups_size":0,"ondisk_log_size":610,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:34:53.361589+0000","scrub_duration":0.0033718060000000002,"objects_trimmed":0,"snaptrim_duration":0.0013409660000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6297,"num_read_kb":53037,"num_write":697,"num_write_kb":27961,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":11818795,"num_keys_recovered":18,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":36,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:39.875135+0000","last_change":"2024-09-21T15:02:39.875518+0000","last_active":"2024-09-21T15:02:39.875135+0000","last_peered":"2024-09-21T15:02:39.875135+0000","last_clean":"2024-09-21T15:02:39.875135+0000","last_became_active":"2024-09-21T15:02:14.224646+0000","last_became_peered":"2024-09-21T15:02:14.224646+0000","last_unstale":"2024-09-21T15:02:39.875135+0000","last_undegraded":"2024-09-21T15:02:39.875135+0000","last_fullsized":"2024-09-21T15:02:39.875135+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:02:39.875043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:02:39.875043+0000","last_clean_scrub_stamp":"2024-09-21T15:02:39.875043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:21:49.722904+0000","scrub_duration":0.0029364460000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"228'980","reported_seq":4274,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:50.951315+0000","last_change":"2024-09-21T15:02:50.951684+0000","last_active":"2024-09-21T15:02:50.951315+0000","last_peered":"2024-09-21T15:02:50.951315+0000","last_clean":"2024-09-21T15:02:50.951315+0000","last_became_active":"2024-09-21T15:02:19.251008+0000","last_became_peered":"2024-09-21T15:02:19.251008+0000","last_unstale":"2024-09-21T15:02:50.951315+0000","last_undegraded":"2024-09-21T15:02:50.951315+0000","last_fullsized":"2024-09-21T15:02:50.951315+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"228'980","last_scrub_stamp":"2024-09-21T15:02:50.951209+0000","last_deep_scrub":"228'980","last_deep_scrub_stamp":"2024-09-21T15:02:50.951209+0000","last_clean_scrub_stamp":"2024-09-21T15:02:50.951209+0000","objects_scrubbed":0,"log_size":980,"log_dups_size":0,"ondisk_log_size":980,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:05:58.139787+0000","scrub_duration":0.0036781090000000002,"objects_trimmed":0,"snaptrim_duration":0.0015726799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5449,"num_read_kb":31198,"num_write":1400,"num_write_kb":25281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":401,"num_bytes_recovered":12566413,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":289,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:44.807178+0000","last_change":"2024-09-21T15:02:44.807466+0000","last_active":"2024-09-21T15:02:44.807178+0000","last_peered":"2024-09-21T15:02:44.807178+0000","last_clean":"2024-09-21T15:02:44.807178+0000","last_became_active":"2024-09-21T15:02:14.227223+0000","last_became_peered":"2024-09-21T15:02:14.227223+0000","last_unstale":"2024-09-21T15:02:44.807178+0000","last_undegraded":"2024-09-21T15:02:44.807178+0000","last_fullsized":"2024-09-21T15:02:44.807178+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:02:44.807130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:02:44.807130+0000","last_clean_scrub_stamp":"2024-09-21T15:02:44.807130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:25:47.959507+0000","scrub_duration":0.0033867839999999999,"objects_trimmed":0,"snaptrim_duration":0.0019338770000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"227'672","reported_seq":5572,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:43.908800+0000","last_change":"2024-09-21T15:02:43.909036+0000","last_active":"2024-09-21T15:02:43.908800+0000","last_peered":"2024-09-21T15:02:43.908800+0000","last_clean":"2024-09-21T15:02:43.908800+0000","last_became_active":"2024-09-21T15:02:18.246450+0000","last_became_peered":"2024-09-21T15:02:18.246450+0000","last_unstale":"2024-09-21T15:02:43.908800+0000","last_undegraded":"2024-09-21T15:02:43.908800+0000","last_fullsized":"2024-09-21T15:02:43.908800+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"227'672","last_scrub_stamp":"2024-09-21T15:02:43.908749+0000","last_deep_scrub":"227'672","last_deep_scrub_stamp":"2024-09-21T15:02:43.908749+0000","last_clean_scrub_stamp":"2024-09-21T15:02:43.908749+0000","objects_scrubbed":0,"log_size":672,"log_dups_size":0,"ondisk_log_size":672,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:25:22.438634+0000","scrub_duration":0.0034825709999999998,"objects_trimmed":0,"snaptrim_duration":0.00071861500000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5178,"num_read_kb":38459,"num_write":1130,"num_write_kb":41306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":331,"num_bytes_recovered":25796920,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":233,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:29.452803+0000","last_change":"2024-09-21T15:02:24.351816+0000","last_active":"2024-09-21T15:02:29.452803+0000","last_peered":"2024-09-21T15:02:29.452803+0000","last_clean":"2024-09-21T15:02:29.452803+0000","last_became_active":"2024-09-21T15:02:24.351245+0000","last_became_peered":"2024-09-21T15:02:24.351245+0000","last_unstale":"2024-09-21T15:02:29.452803+0000","last_undegraded":"2024-09-21T15:02:29.452803+0000","last_fullsized":"2024-09-21T15:02:29.452803+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:08.305870+0000","last_clean_scrub_stamp":"2024-09-21T14:58:08.305870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"227'732","reported_seq":2872,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:46.413573+0000","last_change":"2024-09-21T15:02:46.414002+0000","last_active":"2024-09-21T15:02:46.413573+0000","last_peered":"2024-09-21T15:02:46.413573+0000","last_clean":"2024-09-21T15:02:46.413573+0000","last_became_active":"2024-09-21T14:58:19.408971+0000","last_became_peered":"2024-09-21T14:58:19.408971+0000","last_unstale":"2024-09-21T15:02:46.413573+0000","last_undegraded":"2024-09-21T15:02:46.413573+0000","last_fullsized":"2024-09-21T15:02:46.413573+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"227'732","last_scrub_stamp":"2024-09-21T15:02:46.413448+0000","last_deep_scrub":"227'732","last_deep_scrub_stamp":"2024-09-21T15:02:46.413448+0000","last_clean_scrub_stamp":"2024-09-21T15:02:46.413448+0000","objects_scrubbed":0,"log_size":732,"log_dups_size":0,"ondisk_log_size":732,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:40:10.883458+0000","scrub_duration":0.0039337310000000002,"objects_trimmed":0,"snaptrim_duration":0.00073528299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3602,"num_read_kb":20655,"num_write":1230,"num_write_kb":20679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"251'1010","reported_seq":3703,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:43.964566+0000","last_change":"2024-09-21T15:02:43.964798+0000","last_active":"2024-09-21T15:02:43.964566+0000","last_peered":"2024-09-21T15:02:43.964566+0000","last_clean":"2024-09-21T15:02:43.964566+0000","last_became_active":"2024-09-21T14:58:19.417453+0000","last_became_peered":"2024-09-21T14:58:19.417453+0000","last_unstale":"2024-09-21T15:02:43.964566+0000","last_undegraded":"2024-09-21T15:02:43.964566+0000","last_fullsized":"2024-09-21T15:02:43.964566+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"251'1010","last_scrub_stamp":"2024-09-21T15:02:43.964517+0000","last_deep_scrub":"251'1010","last_deep_scrub_stamp":"2024-09-21T15:02:43.964517+0000","last_clean_scrub_stamp":"2024-09-21T15:02:43.964517+0000","objects_scrubbed":1,"log_size":1010,"log_dups_size":0,"ondisk_log_size":1010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:19:16.788592+0000","scrub_duration":0.0075472940000000004,"objects_trimmed":0,"snaptrim_duration":0.00044463100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4845,"num_read_kb":25284,"num_write":1566,"num_write_kb":21735,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"251'3","reported_seq":260,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:46.842916+0000","last_change":"2024-09-21T15:02:46.843145+0000","last_active":"2024-09-21T15:02:46.842916+0000","last_peered":"2024-09-21T15:02:46.842916+0000","last_clean":"2024-09-21T15:02:46.842916+0000","last_became_active":"2024-09-21T14:58:09.775981+0000","last_became_peered":"2024-09-21T14:58:09.775981+0000","last_unstale":"2024-09-21T15:02:46.842916+0000","last_undegraded":"2024-09-21T15:02:46.842916+0000","last_fullsized":"2024-09-21T15:02:46.842916+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"251'3","last_scrub_stamp":"2024-09-21T15:02:46.842863+0000","last_deep_scrub":"251'3","last_deep_scrub_stamp":"2024-09-21T15:02:46.842863+0000","last_clean_scrub_stamp":"2024-09-21T15:02:46.842863+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:13:15.037486+0000","scrub_duration":0.0065196350000000002,"objects_trimmed":0,"snaptrim_duration":0.0012568880000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"228'1092","reported_seq":4002,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:29.452664+0000","last_change":"2024-09-21T15:02:24.314768+0000","last_active":"2024-09-21T15:02:29.452664+0000","last_peered":"2024-09-21T15:02:29.452664+0000","last_clean":"2024-09-21T15:02:29.452664+0000","last_became_active":"2024-09-21T15:02:24.311531+0000","last_became_peered":"2024-09-21T15:02:24.311531+0000","last_unstale":"2024-09-21T15:02:29.452664+0000","last_undegraded":"2024-09-21T15:02:29.452664+0000","last_fullsized":"2024-09-21T15:02:29.452664+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":1092,"log_dups_size":0,"ondisk_log_size":1092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00183417,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6343,"num_read_kb":28797,"num_write":1677,"num_write_kb":26974,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":258,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:42.562315+0000","last_change":"2024-09-21T15:02:42.562594+0000","last_active":"2024-09-21T15:02:42.562315+0000","last_peered":"2024-09-21T15:02:42.562315+0000","last_clean":"2024-09-21T15:02:42.562315+0000","last_became_active":"2024-09-21T14:58:09.330624+0000","last_became_peered":"2024-09-21T14:58:09.330624+0000","last_unstale":"2024-09-21T15:02:42.562315+0000","last_undegraded":"2024-09-21T15:02:42.562315+0000","last_fullsized":"2024-09-21T15:02:42.562315+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:02:42.562269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:02:42.562269+0000","last_clean_scrub_stamp":"2024-09-21T15:02:42.562269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:33:48.021528+0000","scrub_duration":0.0030134150000000002,"objects_trimmed":0,"snaptrim_duration":0.001051151,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"227'339","reported_seq":2565,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:51.132726+0000","last_change":"2024-09-21T15:02:51.133110+0000","last_active":"2024-09-21T15:02:51.132726+0000","last_peered":"2024-09-21T15:02:51.132726+0000","last_clean":"2024-09-21T15:02:51.132726+0000","last_became_active":"2024-09-21T15:01:53.193166+0000","last_became_peered":"2024-09-21T15:01:53.193166+0000","last_unstale":"2024-09-21T15:02:51.132726+0000","last_undegraded":"2024-09-21T15:02:51.132726+0000","last_fullsized":"2024-09-21T15:02:51.132726+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"227'339","last_scrub_stamp":"2024-09-21T15:02:51.132605+0000","last_deep_scrub":"227'339","last_deep_scrub_stamp":"2024-09-21T15:02:51.132605+0000","last_clean_scrub_stamp":"2024-09-21T15:02:51.132605+0000","objects_scrubbed":0,"log_size":339,"log_dups_size":0,"ondisk_log_size":339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:29:13.184215+0000","scrub_duration":0.0039362850000000003,"objects_trimmed":0,"snaptrim_duration":0.0014983959999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2371,"num_read_kb":40193,"num_write":529,"num_write_kb":39390,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":366,"num_bytes_recovered":22723935,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":257,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:52.720296+0000","last_change":"2024-09-21T15:02:52.720548+0000","last_active":"2024-09-21T15:02:52.720296+0000","last_peered":"2024-09-21T15:02:52.720296+0000","last_clean":"2024-09-21T15:02:52.720296+0000","last_became_active":"2024-09-21T14:58:09.329748+0000","last_became_peered":"2024-09-21T14:58:09.329748+0000","last_unstale":"2024-09-21T15:02:52.720296+0000","last_undegraded":"2024-09-21T15:02:52.720296+0000","last_fullsized":"2024-09-21T15:02:52.720296+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:02:52.720244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:02:52.720244+0000","last_clean_scrub_stamp":"2024-09-21T15:02:52.720244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:31:35.905577+0000","scrub_duration":0.0035768499999999999,"objects_trimmed":0,"snaptrim_duration":0.00052698000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"230'1275","reported_seq":8392,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:48.427816+0000","last_change":"2024-09-21T15:02:48.427816+0000","last_active":"2024-09-21T15:02:48.427816+0000","last_peered":"2024-09-21T15:02:48.427816+0000","last_clean":"2024-09-21T15:02:48.427816+0000","last_became_active":"2024-09-21T14:58:41.401003+0000","last_became_peered":"2024-09-21T14:58:41.401003+0000","last_unstale":"2024-09-21T15:02:48.427816+0000","last_undegraded":"2024-09-21T15:02:48.427816+0000","last_fullsized":"2024-09-21T15:02:48.427816+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"230'1275","last_scrub_stamp":"2024-09-21T15:02:48.427488+0000","last_deep_scrub":"230'1275","last_deep_scrub_stamp":"2024-09-21T15:02:48.427488+0000","last_clean_scrub_stamp":"2024-09-21T15:02:48.427488+0000","objects_scrubbed":0,"log_size":1275,"log_dups_size":0,"ondisk_log_size":1275,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:10:31.281504+0000","scrub_duration":0.003581196,"objects_trimmed":0,"snaptrim_duration":0.000746033,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11708,"num_read_kb":50816,"num_write":1955,"num_write_kb":24913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1403348,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"251'2","reported_seq":259,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:48.778737+0000","last_change":"2024-09-21T15:02:48.779105+0000","last_active":"2024-09-21T15:02:48.778737+0000","last_peered":"2024-09-21T15:02:48.778737+0000","last_clean":"2024-09-21T15:02:48.778737+0000","last_became_active":"2024-09-21T14:58:09.438842+0000","last_became_peered":"2024-09-21T14:58:09.438842+0000","last_unstale":"2024-09-21T15:02:48.778737+0000","last_undegraded":"2024-09-21T15:02:48.778737+0000","last_fullsized":"2024-09-21T15:02:48.778737+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"251'2","last_scrub_stamp":"2024-09-21T15:02:48.778686+0000","last_deep_scrub":"251'2","last_deep_scrub_stamp":"2024-09-21T15:02:48.778686+0000","last_clean_scrub_stamp":"2024-09-21T15:02:48.778686+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:57:10.255068+0000","scrub_duration":0.0065253000000000004,"objects_trimmed":0,"snaptrim_duration":0.001426194,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"251'119","reported_seq":408,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:41.838807+0000","last_change":"2024-09-21T15:02:41.839094+0000","last_active":"2024-09-21T15:02:41.838807+0000","last_peered":"2024-09-21T15:02:41.838807+0000","last_clean":"2024-09-21T15:02:41.838807+0000","last_became_active":"2024-09-21T14:58:03.613214+0000","last_became_peered":"2024-09-21T14:58:03.613214+0000","last_unstale":"2024-09-21T15:02:41.838807+0000","last_undegraded":"2024-09-21T15:02:41.838807+0000","last_fullsized":"2024-09-21T15:02:41.838807+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"251'119","last_scrub_stamp":"2024-09-21T15:02:41.838753+0000","last_deep_scrub":"251'119","last_deep_scrub_stamp":"2024-09-21T15:02:41.838753+0000","last_clean_scrub_stamp":"2024-09-21T15:02:41.838753+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:59:31.901927+0000","scrub_duration":0.0092177409999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"228'685","reported_seq":4709,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:52.105924+0000","last_change":"2024-09-21T15:02:52.106241+0000","last_active":"2024-09-21T15:02:52.105924+0000","last_peered":"2024-09-21T15:02:52.105924+0000","last_clean":"2024-09-21T15:02:52.105924+0000","last_became_active":"2024-09-21T15:01:32.177913+0000","last_became_peered":"2024-09-21T15:01:32.177913+0000","last_unstale":"2024-09-21T15:02:52.105924+0000","last_undegraded":"2024-09-21T15:02:52.105924+0000","last_fullsized":"2024-09-21T15:02:52.105924+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"228'685","last_scrub_stamp":"2024-09-21T15:02:52.105873+0000","last_deep_scrub":"228'685","last_deep_scrub_stamp":"2024-09-21T15:02:52.105873+0000","last_clean_scrub_stamp":"2024-09-21T15:02:52.105873+0000","objects_scrubbed":0,"log_size":685,"log_dups_size":0,"ondisk_log_size":685,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:49:28.346429+0000","scrub_duration":0.0036573,"objects_trimmed":0,"snaptrim_duration":0.0015618730000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4407,"num_read_kb":44538,"num_write":1164,"num_write_kb":30502,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":413,"num_bytes_recovered":20409497,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.9","version":"229'973","reported_seq":4121,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:39.194618+0000","last_change":"2024-09-21T15:02:18.257627+0000","last_active":"2024-09-21T15:02:39.194618+0000","last_peered":"2024-09-21T15:02:39.194618+0000","last_clean":"2024-09-21T15:02:39.194618+0000","last_became_active":"2024-09-21T15:02:18.255482+0000","last_became_peered":"2024-09-21T15:02:18.255482+0000","last_unstale":"2024-09-21T15:02:39.194618+0000","last_undegraded":"2024-09-21T15:02:39.194618+0000","last_fullsized":"2024-09-21T15:02:39.194618+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":973,"log_dups_size":0,"ondisk_log_size":973,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001545105,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4815,"num_read_kb":35700,"num_write":1595,"num_write_kb":33802,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":375,"num_bytes_recovered":14742839,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"229'1027","reported_seq":5031,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:35.931016+0000","last_change":"2024-09-21T15:02:02.261576+0000","last_active":"2024-09-21T15:02:35.931016+0000","last_peered":"2024-09-21T15:02:35.931016+0000","last_clean":"2024-09-21T15:02:35.931016+0000","last_became_active":"2024-09-21T15:00:27.280971+0000","last_became_peered":"2024-09-21T15:00:27.280971+0000","last_unstale":"2024-09-21T15:02:35.931016+0000","last_undegraded":"2024-09-21T15:02:35.931016+0000","last_fullsized":"2024-09-21T15:02:35.931016+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":1027,"log_dups_size":0,"ondisk_log_size":1027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013613779999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7341,"num_read_kb":31141,"num_write":1421,"num_write_kb":33160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":9228123,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"227'593","reported_seq":3460,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:43.548064+0000","last_change":"2024-09-21T15:02:43.548064+0000","last_active":"2024-09-21T15:02:43.548064+0000","last_peered":"2024-09-21T15:02:43.548064+0000","last_clean":"2024-09-21T15:02:43.548064+0000","last_became_active":"2024-09-21T14:58:19.410679+0000","last_became_peered":"2024-09-21T14:58:19.410679+0000","last_unstale":"2024-09-21T15:02:43.548064+0000","last_undegraded":"2024-09-21T15:02:43.548064+0000","last_fullsized":"2024-09-21T15:02:43.548064+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"227'593","last_scrub_stamp":"2024-09-21T15:02:43.547754+0000","last_deep_scrub":"227'593","last_deep_scrub_stamp":"2024-09-21T15:02:43.547754+0000","last_clean_scrub_stamp":"2024-09-21T15:02:43.547754+0000","objects_scrubbed":0,"log_size":593,"log_dups_size":0,"ondisk_log_size":593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:10:36.426327+0000","scrub_duration":0.0039169069999999999,"objects_trimmed":0,"snaptrim_duration":0.001466743,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3111,"num_read_kb":36762,"num_write":889,"num_write_kb":39649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"228'493","reported_seq":4721,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:54.095577+0000","last_change":"2024-09-21T15:02:54.095817+0000","last_active":"2024-09-21T15:02:54.095577+0000","last_peered":"2024-09-21T15:02:54.095577+0000","last_clean":"2024-09-21T15:02:54.095577+0000","last_became_active":"2024-09-21T14:59:25.130057+0000","last_became_peered":"2024-09-21T14:59:25.130057+0000","last_unstale":"2024-09-21T15:02:54.095577+0000","last_undegraded":"2024-09-21T15:02:54.095577+0000","last_fullsized":"2024-09-21T15:02:54.095577+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"228'493","last_scrub_stamp":"2024-09-21T15:02:54.095523+0000","last_deep_scrub":"228'493","last_deep_scrub_stamp":"2024-09-21T15:02:54.095523+0000","last_clean_scrub_stamp":"2024-09-21T15:02:54.095523+0000","objects_scrubbed":0,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:12:06.451414+0000","scrub_duration":0.0037916920000000002,"objects_trimmed":0,"snaptrim_duration":0.0014500649999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4513,"num_read_kb":28786,"num_write":729,"num_write_kb":24616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":9059600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"227'544","reported_seq":5367,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:53.718875+0000","last_change":"2024-09-21T15:02:53.719159+0000","last_active":"2024-09-21T15:02:53.718875+0000","last_peered":"2024-09-21T15:02:53.718875+0000","last_clean":"2024-09-21T15:02:53.718875+0000","last_became_active":"2024-09-21T14:58:19.407884+0000","last_became_peered":"2024-09-21T14:58:19.407884+0000","last_unstale":"2024-09-21T15:02:53.718875+0000","last_undegraded":"2024-09-21T15:02:53.718875+0000","last_fullsized":"2024-09-21T15:02:53.718875+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"227'544","last_scrub_stamp":"2024-09-21T15:02:53.718820+0000","last_deep_scrub":"227'544","last_deep_scrub_stamp":"2024-09-21T15:02:53.718820+0000","last_clean_scrub_stamp":"2024-09-21T15:02:53.718820+0000","objects_scrubbed":0,"log_size":544,"log_dups_size":0,"ondisk_log_size":544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:17:32.825677+0000","scrub_duration":0.0037810449999999998,"objects_trimmed":0,"snaptrim_duration":0.00045666499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4968,"num_read_kb":55828,"num_write":945,"num_write_kb":38650,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"230'879","reported_seq":3230,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:49.427198+0000","last_change":"2024-09-21T15:02:49.427437+0000","last_active":"2024-09-21T15:02:49.427198+0000","last_peered":"2024-09-21T15:02:49.427198+0000","last_clean":"2024-09-21T15:02:49.427198+0000","last_became_active":"2024-09-21T15:01:05.172427+0000","last_became_peered":"2024-09-21T15:01:05.172427+0000","last_unstale":"2024-09-21T15:02:49.427198+0000","last_undegraded":"2024-09-21T15:02:49.427198+0000","last_fullsized":"2024-09-21T15:02:49.427198+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"230'879","last_scrub_stamp":"2024-09-21T15:02:49.427146+0000","last_deep_scrub":"230'879","last_deep_scrub_stamp":"2024-09-21T15:02:49.427146+0000","last_clean_scrub_stamp":"2024-09-21T15:02:49.427146+0000","objects_scrubbed":0,"log_size":879,"log_dups_size":0,"ondisk_log_size":879,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:23:50.808175+0000","scrub_duration":0.0033179379999999999,"objects_trimmed":0,"snaptrim_duration":0.001142789,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4262,"num_read_kb":17288,"num_write":1508,"num_write_kb":17970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":138,"num_bytes_recovered":7486496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"227'532","reported_seq":6222,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:45.994244+0000","last_change":"2024-09-21T15:02:45.994244+0000","last_active":"2024-09-21T15:02:45.994244+0000","last_peered":"2024-09-21T15:02:45.994244+0000","last_clean":"2024-09-21T15:02:45.994244+0000","last_became_active":"2024-09-21T14:58:19.410406+0000","last_became_peered":"2024-09-21T14:58:19.410406+0000","last_unstale":"2024-09-21T15:02:45.994244+0000","last_undegraded":"2024-09-21T15:02:45.994244+0000","last_fullsized":"2024-09-21T15:02:45.994244+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"227'532","last_scrub_stamp":"2024-09-21T15:02:45.993899+0000","last_deep_scrub":"227'532","last_deep_scrub_stamp":"2024-09-21T15:02:45.993899+0000","last_clean_scrub_stamp":"2024-09-21T15:02:45.993899+0000","objects_scrubbed":0,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:50:55.037554+0000","scrub_duration":0.0038182770000000001,"objects_trimmed":0,"snaptrim_duration":0.0014059350000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5807,"num_read_kb":56273,"num_write":817,"num_write_kb":36773,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"228'523","reported_seq":4648,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:29.355009+0000","last_change":"2024-09-21T15:02:02.260617+0000","last_active":"2024-09-21T15:02:29.355009+0000","last_peered":"2024-09-21T15:02:29.355009+0000","last_clean":"2024-09-21T15:02:29.355009+0000","last_became_active":"2024-09-21T14:58:19.409935+0000","last_became_peered":"2024-09-21T14:58:19.409935+0000","last_unstale":"2024-09-21T15:02:29.355009+0000","last_undegraded":"2024-09-21T15:02:29.355009+0000","last_fullsized":"2024-09-21T15:02:29.355009+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":523,"log_dups_size":0,"ondisk_log_size":523,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068183199999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3949,"num_read_kb":58693,"num_write":810,"num_write_kb":37279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"230'623","reported_seq":5196,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:29.771602+0000","last_change":"2024-09-21T15:02:02.408072+0000","last_active":"2024-09-21T15:02:29.771602+0000","last_peered":"2024-09-21T15:02:29.771602+0000","last_clean":"2024-09-21T15:02:29.771602+0000","last_became_active":"2024-09-21T15:01:05.165955+0000","last_became_peered":"2024-09-21T15:01:05.165955+0000","last_unstale":"2024-09-21T15:02:29.771602+0000","last_undegraded":"2024-09-21T15:02:29.771602+0000","last_fullsized":"2024-09-21T15:02:29.771602+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":623,"log_dups_size":0,"ondisk_log_size":623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021463580000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5758,"num_read_kb":28485,"num_write":777,"num_write_kb":24185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":166,"num_bytes_recovered":11965028,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"229'644","reported_seq":8547,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:41.163032+0000","last_change":"2024-09-21T15:02:41.163435+0000","last_active":"2024-09-21T15:02:41.163032+0000","last_peered":"2024-09-21T15:02:41.163032+0000","last_clean":"2024-09-21T15:02:41.163032+0000","last_became_active":"2024-09-21T15:00:07.155144+0000","last_became_peered":"2024-09-21T15:00:07.155144+0000","last_unstale":"2024-09-21T15:02:41.163032+0000","last_undegraded":"2024-09-21T15:02:41.163032+0000","last_fullsized":"2024-09-21T15:02:41.163032+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"229'644","last_scrub_stamp":"2024-09-21T15:02:41.162917+0000","last_deep_scrub":"229'644","last_deep_scrub_stamp":"2024-09-21T15:02:41.162917+0000","last_clean_scrub_stamp":"2024-09-21T15:02:41.162917+0000","objects_scrubbed":0,"log_size":644,"log_dups_size":0,"ondisk_log_size":644,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:05:13.736243+0000","scrub_duration":0.0040251990000000001,"objects_trimmed":0,"snaptrim_duration":0.0010987029999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9100,"num_read_kb":37252,"num_write":1013,"num_write_kb":20896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"227'947","reported_seq":6823,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:29.771775+0000","last_change":"2024-09-21T15:02:02.406594+0000","last_active":"2024-09-21T15:02:29.771775+0000","last_peered":"2024-09-21T15:02:29.771775+0000","last_clean":"2024-09-21T15:02:29.771775+0000","last_became_active":"2024-09-21T14:58:19.412030+0000","last_became_peered":"2024-09-21T14:58:19.412030+0000","last_unstale":"2024-09-21T15:02:29.771775+0000","last_undegraded":"2024-09-21T15:02:29.771775+0000","last_fullsized":"2024-09-21T15:02:29.771775+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T14:58:18.379818+0000","last_clean_scrub_stamp":"2024-09-21T14:58:18.379818+0000","objects_scrubbed":0,"log_size":947,"log_dups_size":0,"ondisk_log_size":947,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067099300000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8465,"num_read_kb":59792,"num_write":1327,"num_write_kb":31587,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":174039,"num_read_kb":1238236,"num_write":34305,"num_write_kb":940835,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3956,"num_bytes_recovered":212783659,"num_keys_recovered":305,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7385,"internal_metadata":0},"log_size":22544,"ondisk_log_size":22544,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":10404,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509504,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34080,"kb_used_data":2968,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94337760,"statfs":{"total":96636764160,"available":96601866240,"internally_reserved":0,"allocated":3039232,"data_stored":1498239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:02:06 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.83899999999999997,"15min":0.83699999999999997},"min":{"1min":0.29199999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.044,"5min":1.351,"15min":1.351},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.91400000000000003,"15min":0.94599999999999995},"min":{"1min":0.48099999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.0589999999999999,"5min":1.917,"15min":1.917},"last":0.81399999999999995}]},{"osd":1,"last update":"Sat Sep 21 15:02:06 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.80800000000000005,"15min":0.77600000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":1.0409999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.81599999999999995,"15min":0.82099999999999995},"min":{"1min":0.40500000000000003,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.002,"5min":2.2149999999999999,"15min":2.2149999999999999},"last":1.1319999999999999}]},{"osd":2,"last update":"Sat Sep 21 15:02:06 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.82899999999999996,"15min":0.81499999999999995},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.161,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":1.002},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.92600000000000005,"15min":0.93999999999999995},"min":{"1min":0.56499999999999995,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.0920000000000001,"5min":1.466,"15min":1.466},"last":1.028}]},{"osd":3,"last update":"Sat Sep 21 15:02:06 2024","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.89100000000000001,"15min":0.86299999999999999},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.24,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":1.091},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.96799999999999997,"15min":0.96599999999999997},"min":{"1min":0.54200000000000004,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.159,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":1.224}]},{"osd":4,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.92200000000000004,"15min":0.93300000000000005},"min":{"1min":0.42199999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.3089999999999999,"5min":2.7130000000000001,"15min":2.7130000000000001},"last":1.177},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.94499999999999995,"15min":0.92600000000000005},"min":{"1min":0.39000000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.425,"5min":2.125,"15min":2.125},"last":0.75}]},{"osd":5,"last update":"Sat Sep 21 15:02:06 2024","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":1.006,"15min":1.016},"min":{"1min":0.505,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.375,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.97699999999999998,"5min":1.026,"15min":1.0449999999999999},"min":{"1min":0.61599999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.425,"5min":3.1880000000000002,"15min":3.1880000000000002},"last":0.86399999999999999}]},{"osd":6,"last update":"Sat Sep 21 15:02:06 2024","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.115,"15min":1.143},"min":{"1min":0.52900000000000003,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.679,"5min":3.2730000000000001,"15min":3.2730000000000001},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.099,"5min":1.0800000000000001,"15min":1.036},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.6719999999999999,"5min":2.2879999999999998,"15min":2.2879999999999998},"last":1.27}]}]},{"osd":6,"up_from":14,"seq":60129542207,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64420,"kb_used_data":2972,"kb_used_omap":1,"kb_used_meta":61438,"kb_avail":94307420,"statfs":{"total":96636764160,"available":96570798080,"internally_reserved":0,"allocated":3043328,"data_stored":1498258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":62912967},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:02:09 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.87,"15min":0.93200000000000005},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.089,"5min":1.593,"15min":1.593},"last":0.62},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.90500000000000003,"15min":0.94099999999999995},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.1180000000000001,"5min":1.587,"15min":1.587},"last":0.95899999999999996}]},{"osd":1,"last update":"Sat Sep 21 15:02:09 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.876,"15min":0.90400000000000003},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.145,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.85099999999999998,"15min":0.94499999999999995},"min":{"1min":0.40600000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.036,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.98799999999999999}]},{"osd":2,"last update":"Sat Sep 21 15:02:09 2024","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.93600000000000005,"15min":0.97499999999999998},"min":{"1min":0.53400000000000003,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.194,"5min":1.758,"15min":1.758},"last":1.1259999999999999},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.878,"15min":0.90200000000000002},"min":{"1min":0.51700000000000002,"5min":0.496,"15min":0.496},"max":{"1min":1.002,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.88900000000000001}]},{"osd":3,"last update":"Sat Sep 21 15:02:09 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.91600000000000004,"15min":0.94799999999999995},"min":{"1min":0.57399999999999995,"5min":0.434,"15min":0.434},"max":{"1min":0.98699999999999999,"5min":1.835,"15min":1.835},"last":1.091},{"interface":"front","average":{"1min":0.90100000000000002,"5min":1.0429999999999999,"15min":1.111},"min":{"1min":0.65900000000000003,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.296,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":1.0349999999999999}]},{"osd":4,"last update":"Sat Sep 21 15:01:08 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.93500000000000005,"15min":0.99099999999999999},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":1.2,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.98499999999999999,"15min":1.0209999999999999},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.3759999999999999,"5min":1.897,"15min":1.897},"last":0.81899999999999995}]},{"osd":5,"last update":"Sat Sep 21 15:02:09 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.97799999999999998,"15min":1.0880000000000001},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.222,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":1.0640000000000001},{"interface":"front","average":{"1min":0.85699999999999998,"5min":1.018,"15min":1.071},"min":{"1min":0.54200000000000004,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.3600000000000001,"5min":1.609,"15min":1.609},"last":1.01}]},{"osd":7,"last update":"Sat Sep 21 15:02:09 2024","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":1.131,"15min":1.2190000000000001},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.256,"5min":1.8660000000000001,"15min":1.8660000000000001},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.91300000000000003,"5min":1.117,"15min":1.248},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.413,"5min":2.004,"15min":2.004},"last":0.92800000000000005}]}]},{"osd":5,"up_from":14,"seq":60129542208,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60380,"kb_used_data":3028,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94311460,"statfs":{"total":96636764160,"available":96574935040,"internally_reserved":0,"allocated":3100672,"data_stored":2088626,"data_compressed":5202,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:02:09 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.84199999999999997,"15min":0.96899999999999997},"min":{"1min":0.40100000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.98099999999999998,"5min":4.4800000000000004,"15min":4.4800000000000004},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.78400000000000003,"15min":0.81299999999999994},"min":{"1min":0.44500000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.179,"5min":1.181,"15min":1.181},"last":0.78200000000000003}]},{"osd":1,"last update":"Sat Sep 21 15:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.86399999999999999,"15min":0.91300000000000003},"min":{"1min":0.41899999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.036,"5min":2.7709999999999999,"15min":2.7709999999999999},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.89900000000000002,"15min":0.92900000000000005},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.1439999999999999,"5min":2.823,"15min":2.823},"last":0.98399999999999999}]},{"osd":2,"last update":"Sat Sep 21 15:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.86599999999999999,"15min":0.95799999999999996},"min":{"1min":0.53600000000000003,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.2090000000000001,"5min":3.419,"15min":3.419},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.84499999999999997,"15min":0.88800000000000001},"min":{"1min":0.58999999999999997,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.1699999999999999,"5min":1.423,"15min":1.423},"last":1.034}]},{"osd":3,"last update":"Sat Sep 21 15:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.81100000000000005,"15min":0.81000000000000005},"min":{"1min":0.51000000000000001,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.105,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":1.0569999999999999},{"interface":"front","average":{"1min":0.91700000000000004,"5min":1.018,"15min":1.0720000000000001},"min":{"1min":0.66600000000000004,"5min":0.435,"15min":0.435},"max":{"1min":1.4239999999999999,"5min":2.6379999999999999,"15min":2.6379999999999999},"last":1.0109999999999999}]},{"osd":4,"last update":"Sat Sep 21 15:01:02 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.98899999999999999,"15min":1.016},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.5609999999999999,"5min":2.5470000000000002,"15min":2.5470000000000002},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.82099999999999995,"5min":1.0169999999999999,"15min":1.0780000000000001},"min":{"1min":0.39300000000000002,"5min":0.376,"15min":0.376},"max":{"1min":1.302,"5min":2.5099999999999998,"15min":2.5099999999999998},"last":1.0940000000000001}]},{"osd":6,"last update":"Sat Sep 21 15:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.91700000000000004,"15min":0.95999999999999996},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.1499999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.879,"5min":1.1399999999999999,"15min":1.3169999999999999},"min":{"1min":0.60199999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.498,"5min":4.2469999999999999,"15min":4.2469999999999999},"last":0.82699999999999996}]},{"osd":7,"last update":"Sat Sep 21 15:02:09 2024","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":1.167,"15min":1.3120000000000001},"min":{"1min":0.61199999999999999,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.5700000000000001,"5min":3.3010000000000002,"15min":3.3010000000000002},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.99399999999999999,"5min":1.075,"15min":1.125},"min":{"1min":0.64300000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.25,"5min":2.5880000000000001,"15min":2.5880000000000001},"last":1.119}]}]},{"osd":4,"up_from":235,"seq":1009317314572,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34848,"kb_used_data":2968,"kb_used_omap":8,"kb_used_meta":31863,"kb_avail":94336992,"statfs":{"total":96636764160,"available":96601079808,"internally_reserved":0,"allocated":3039232,"data_stored":1498239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8979,"internal_metadata":32627949},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2589999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0600000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91100000000000003}]}]},{"osd":3,"up_from":14,"seq":60129542208,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78372,"kb_used_data":2972,"kb_used_omap":1,"kb_used_meta":75390,"kb_avail":94293468,"statfs":{"total":96636764160,"available":96556511232,"internally_reserved":0,"allocated":3043328,"data_stored":1498258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":77199816},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.61799999999999999,"15min":0.63200000000000001},"min":{"1min":0.34599999999999997,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.92700000000000005,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.68200000000000005,"15min":0.70999999999999996},"min":{"1min":0.40500000000000003,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.3029999999999999,"5min":1.919,"15min":1.919},"last":0.94999999999999996}]},{"osd":1,"last update":"Sat Sep 21 15:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.65800000000000003,"15min":0.67000000000000004},"min":{"1min":0.44400000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":1.115,"5min":1.722,"15min":1.722},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.746,"15min":0.79700000000000004},"min":{"1min":0.41199999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.2509999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.65300000000000002}]},{"osd":2,"last update":"Sat Sep 21 15:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.746,"15min":0.72999999999999998},"min":{"1min":0.38200000000000001,"5min":0.311,"15min":0.311},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.746,"5min":0.79200000000000004,"15min":0.82099999999999995},"min":{"1min":0.40300000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.94399999999999995,"5min":1.254,"15min":1.254},"last":0.67900000000000005}]},{"osd":4,"last update":"Sat Sep 21 15:01:09 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.93799999999999994,"15min":0.95599999999999996},"min":{"1min":0.54300000000000004,"5min":0.442,"15min":0.442},"max":{"1min":1.2629999999999999,"5min":1.7609999999999999,"15min":1.7609999999999999},"last":1.077},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.98899999999999999,"15min":1.0269999999999999},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.234,"5min":2.028,"15min":2.028},"last":1.1080000000000001}]},{"osd":5,"last update":"Sat Sep 21 15:02:13 2024","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.036,"15min":1.052},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.4670000000000001,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":0.80500000000000005},{"interface":"front","average":{"1min":1.081,"5min":1.0589999999999999,"15min":1.0640000000000001},"min":{"1min":0.68799999999999994,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.395,"5min":1.548,"15min":1.548},"last":1.0289999999999999}]},{"osd":6,"last update":"Sat Sep 21 15:02:13 2024","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":1.046,"15min":1.085},"min":{"1min":0.70099999999999996,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.391,"5min":2.1219999999999999,"15min":2.1219999999999999},"last":0.91900000000000004},{"interface":"front","average":{"1min":1.03,"5min":1.075,"15min":1.0980000000000001},"min":{"1min":0.64900000000000002,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.448,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":1.052}]},{"osd":7,"last update":"Sat Sep 21 15:02:13 2024","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.0800000000000001,"15min":1.123},"min":{"1min":0.746,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.339,"5min":1.706,"15min":1.706},"last":0.99199999999999999},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.1279999999999999,"15min":1.171},"min":{"1min":0.67700000000000005,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.421,"5min":2.3380000000000001,"15min":2.3380000000000001},"last":1.129}]}]},{"osd":2,"up_from":14,"seq":60129542206,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70176,"kb_used_data":2968,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94301664,"statfs":{"total":96636764160,"available":96564903936,"internally_reserved":0,"allocated":3039232,"data_stored":1498239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68811206},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:02:07 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.63500000000000001,"15min":0.625},"min":{"1min":0.32100000000000001,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.383,"5min":1.383,"15min":1.383},"last":1.3009999999999999},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.69099999999999995,"15min":0.66900000000000004},"min":{"1min":0.30099999999999999,"5min":0.223,"15min":0.223},"max":{"1min":1.1739999999999999,"5min":2.9169999999999998,"15min":2.9169999999999998},"last":0.80700000000000005}]},{"osd":1,"last update":"Sat Sep 21 15:02:07 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.67500000000000004,"15min":0.68100000000000005},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.129,"5min":2.6520000000000001,"15min":2.6520000000000001},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.73399999999999999,"15min":0.75800000000000001},"min":{"1min":0.36199999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":1.351}]},{"osd":3,"last update":"Sat Sep 21 15:02:07 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.71699999999999997,"15min":0.72699999999999998},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.996,"5min":3.0110000000000001,"15min":3.0110000000000001},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.85999999999999999,"15min":0.83399999999999996},"min":{"1min":0.41099999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0369999999999999,"5min":4.1440000000000001,"15min":4.1440000000000001},"last":0.70699999999999996}]},{"osd":4,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.93200000000000005,"15min":0.95599999999999996},"min":{"1min":0.42399999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.222,"5min":2.899,"15min":2.899},"last":1.5369999999999999},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.94399999999999995,"15min":0.95799999999999996},"min":{"1min":0.47499999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.4379999999999999,"5min":1.675,"15min":1.675},"last":0.90900000000000003}]},{"osd":5,"last update":"Sat Sep 21 15:02:07 2024","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.98699999999999999,"15min":1.0009999999999999},"min":{"1min":0.45500000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.224,"5min":3.1080000000000001,"15min":3.1080000000000001},"last":0.99299999999999999},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.96399999999999997,"15min":0.98799999999999999},"min":{"1min":0.55200000000000005,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.228,"5min":1.546,"15min":1.546},"last":1.179}]},{"osd":6,"last update":"Sat Sep 21 15:02:07 2024","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":0.98899999999999999,"15min":0.98699999999999999},"min":{"1min":0.52100000000000002,"5min":0.378,"15min":0.378},"max":{"1min":1.516,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":1.677},{"interface":"front","average":{"1min":0.93999999999999995,"5min":1.0429999999999999,"15min":1.048},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.403,"5min":3.0739999999999998,"15min":3.0739999999999998},"last":1.0680000000000001}]},{"osd":7,"last update":"Sat Sep 21 15:02:09 2024","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.0489999999999999,"15min":1.01},"min":{"1min":0.61399999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.415,"5min":3.1589999999999998,"15min":3.1589999999999998},"last":1.157},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.111,"15min":1.095},"min":{"1min":0.58799999999999997,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.4610000000000001,"5min":3.1400000000000001,"15min":3.1400000000000001},"last":1.2}]}]},{"osd":1,"up_from":14,"seq":60129542209,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52188,"kb_used_data":3028,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94319652,"statfs":{"total":96636764160,"available":96583323648,"internally_reserved":0,"allocated":3100672,"data_stored":2088626,"data_compressed":5202,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":50330053},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.58899999999999997,"15min":0.60699999999999998},"min":{"1min":0.25600000000000001,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.60299999999999998,"15min":0.60999999999999999},"min":{"1min":0.26300000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.83499999999999996,"5min":1.107,"15min":1.107},"last":0.79400000000000004}]},{"osd":2,"last update":"Sat Sep 21 15:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.78000000000000003,"15min":0.73499999999999999},"min":{"1min":0.38800000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.92100000000000004,"5min":11.303000000000001,"15min":11.303000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.71399999999999997,"15min":0.70299999999999996},"min":{"1min":0.439,"5min":0.435,"15min":0.435},"max":{"1min":1.0840000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.72899999999999998}]},{"osd":3,"last update":"Sat Sep 21 15:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.80200000000000005,"15min":0.83399999999999996},"min":{"1min":0.36899999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.4850000000000001,"5min":3.1469999999999998,"15min":3.1469999999999998},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.83699999999999997,"15min":0.877},"min":{"1min":0.42499999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.284,"5min":3.2320000000000002,"15min":3.2320000000000002},"last":0.86499999999999999}]},{"osd":4,"last update":"Sat Sep 21 15:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.89000000000000001,"15min":0.91500000000000004},"min":{"1min":0.59599999999999997,"5min":0.309,"15min":0.309},"max":{"1min":1.45,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":0.88900000000000001},{"interface":"front","average":{"1min":1.5600000000000001,"5min":1.069,"15min":0.96599999999999997},"min":{"1min":0.504,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":11.563000000000001,"5min":11.563000000000001,"15min":11.563000000000001},"last":0.98899999999999999}]},{"osd":5,"last update":"Sat Sep 21 15:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":1.0620000000000001,"15min":0.95399999999999996},"min":{"1min":0.61199999999999999,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.3799999999999999,"5min":11.534000000000001,"15min":11.534000000000001},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.88,"5min":0.94199999999999995,"15min":0.95599999999999996},"min":{"1min":0.58599999999999997,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.1319999999999999,"5min":1.484,"15min":1.484},"last":1.0489999999999999}]},{"osd":6,"last update":"Sat Sep 21 15:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.97999999999999998,"15min":1.038},"min":{"1min":0.53100000000000003,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.254,"5min":2.3519999999999999,"15min":2.3519999999999999},"last":1.0169999999999999},{"interface":"front","average":{"1min":0.97499999999999998,"5min":1.087,"15min":0.995},"min":{"1min":0.66400000000000003,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.4259999999999999,"5min":11.601000000000001,"15min":11.601000000000001},"last":1.1140000000000001}]},{"osd":7,"last update":"Sat Sep 21 15:02:05 2024","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":1.02,"15min":1.056},"min":{"1min":0.58899999999999997,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.4039999999999999,"5min":2.2360000000000002,"15min":2.2360000000000002},"last":0.94199999999999995},{"interface":"front","average":{"1min":1.044,"5min":1.171,"15min":1.087},"min":{"1min":0.63100000000000001,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.3799999999999999,"5min":11.489000000000001,"15min":11.489000000000001},"last":0.95899999999999996}]}]},{"osd":0,"up_from":14,"seq":60129542206,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70176,"kb_used_data":2968,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94301664,"statfs":{"total":96636764160,"available":96564903936,"internally_reserved":0,"allocated":3039232,"data_stored":1498239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":68811207},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 21 15:02:07 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.64000000000000001,"15min":0.70599999999999996},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.002,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.69499999999999995,"15min":0.63200000000000001},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.98699999999999999,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.65800000000000003}]},{"osd":2,"last update":"Sat Sep 21 15:02:07 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.66100000000000003,"15min":0.621},"min":{"1min":0.33900000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.0960000000000001,"5min":1.153,"15min":1.153},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.70099999999999996,"15min":0.75},"min":{"1min":0.28799999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.0329999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.59699999999999998}]},{"osd":3,"last update":"Sat Sep 21 15:02:07 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.76400000000000001,"15min":0.754},"min":{"1min":0.37,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":1.101},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.77400000000000002,"15min":0.81100000000000005},"min":{"1min":0.42799999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.04,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.71999999999999997}]},{"osd":4,"last update":"Sat Sep 21 15:01:04 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.98399999999999999,"15min":1.038},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.1140000000000001,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":0.82699999999999996},{"interface":"front","average":{"1min":1.036,"5min":1.0469999999999999,"15min":1.0620000000000001},"min":{"1min":0.67800000000000005,"5min":0.379,"15min":0.379},"max":{"1min":1.446,"5min":1.665,"15min":1.665},"last":1.177}]},{"osd":5,"last update":"Sat Sep 21 15:02:07 2024","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.997,"15min":1.0229999999999999},"min":{"1min":0.45800000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.296,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":1.25},{"interface":"front","average":{"1min":0.94299999999999995,"5min":1.0129999999999999,"15min":1.0669999999999999},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.3919999999999999,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":0.95599999999999996}]},{"osd":6,"last update":"Sat Sep 21 15:02:07 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.98799999999999999,"15min":1.089},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.119,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.94799999999999995,"5min":1.038,"15min":1.0600000000000001},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.393,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":1.0069999999999999}]},{"osd":7,"last update":"Sat Sep 21 15:02:07 2024","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":1.012,"15min":1.05},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.1719999999999999,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.88300000000000001,"5min":1.0309999999999999,"15min":1.113},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.28,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":0.98399999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5202,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5202,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7385,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T15:03:01.865 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-09-21T14:58:18.379818+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=58, tm_sec=18, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=2, tm_sec=36, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:03:01.865 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-21T14:58:08.305870+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=58, tm_sec=8, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=2, tm_sec=36, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:03:01.866 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-09-21T14:58:18.379818+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=58, tm_sec=18, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=2, tm_sec=36, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:03:01.866 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-09-21T14:58:18.379818+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=58, tm_sec=18, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=2, tm_sec=36, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:03:01.866 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-09-21T14:58:18.379818+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=58, tm_sec=18, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=2, tm_sec=36, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:03:01.866 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-09-21T14:58:18.379818+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=58, tm_sec=18, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=2, tm_sec=36, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:03:01.866 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-09-21T14:58:18.379818+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=58, tm_sec=18, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=2, tm_sec=36, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:03:01.866 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-09-21T14:58:18.379818+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=58, tm_sec=18, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=2, tm_sec=36, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:03:01.866 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-21T15:03:21.867 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T15:03:22.185 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-09-21T15:03:22.187 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-09-21T15:03:22.203 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":405,"stamp":"2024-09-21T15:03:21.863216+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":174155,"num_read_kb":1238336,"num_write":34466,"num_write_kb":943253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3956,"num_bytes_recovered":212783659,"num_keys_recovered":305,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22668,"ondisk_log_size":22668,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":464640,"kb_used_data":23872,"kb_used_omap":19,"kb_used_meta":440684,"kb_avail":754510080,"statfs":{"total":773094113280,"available":772618321920,"internally_reserved":0,"allocated":24444928,"data_stored":13166724,"data_compressed":10404,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":20129,"internal_metadata":451260767},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003912"},"pg_stats":[{"pgid":"3.1f","version":"230'598","reported_seq":3991,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:50.506670+0000","last_change":"2024-09-21T15:02:50.506935+0000","last_active":"2024-09-21T15:02:50.506670+0000","last_peered":"2024-09-21T15:02:50.506670+0000","last_clean":"2024-09-21T15:02:50.506670+0000","last_became_active":"2024-09-21T15:02:24.308451+0000","last_became_peered":"2024-09-21T15:02:24.308451+0000","last_unstale":"2024-09-21T15:02:50.506670+0000","last_undegraded":"2024-09-21T15:02:50.506670+0000","last_fullsized":"2024-09-21T15:02:50.506670+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"230'598","last_scrub_stamp":"2024-09-21T15:02:50.506616+0000","last_deep_scrub":"230'598","last_deep_scrub_stamp":"2024-09-21T15:02:50.506616+0000","last_clean_scrub_stamp":"2024-09-21T15:02:50.506616+0000","objects_scrubbed":0,"log_size":598,"log_dups_size":0,"ondisk_log_size":598,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:11:45.796964+0000","scrub_duration":0.0038622230000000001,"objects_trimmed":0,"snaptrim_duration":0.0010410479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3756,"num_read_kb":43219,"num_write":975,"num_write_kb":36436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"230'507","reported_seq":3522,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:47.523706+0000","last_change":"2024-09-21T15:02:47.524034+0000","last_active":"2024-09-21T15:02:47.523706+0000","last_peered":"2024-09-21T15:02:47.523706+0000","last_clean":"2024-09-21T15:02:47.523706+0000","last_became_active":"2024-09-21T15:02:24.309506+0000","last_became_peered":"2024-09-21T15:02:24.309506+0000","last_unstale":"2024-09-21T15:02:47.523706+0000","last_undegraded":"2024-09-21T15:02:47.523706+0000","last_fullsized":"2024-09-21T15:02:47.523706+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"230'507","last_scrub_stamp":"2024-09-21T15:02:47.523655+0000","last_deep_scrub":"230'507","last_deep_scrub_stamp":"2024-09-21T15:02:47.523655+0000","last_clean_scrub_stamp":"2024-09-21T15:02:47.523655+0000","objects_scrubbed":0,"log_size":507,"log_dups_size":0,"ondisk_log_size":507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:56:11.657252+0000","scrub_duration":0.0038311500000000002,"objects_trimmed":0,"snaptrim_duration":0.00094912299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3420,"num_read_kb":23407,"num_write":643,"num_write_kb":18445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":148,"num_bytes_recovered":11635695,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"228'341","reported_seq":4803,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:55.752127+0000","last_change":"2024-09-21T15:02:55.752127+0000","last_active":"2024-09-21T15:02:55.752127+0000","last_peered":"2024-09-21T15:02:55.752127+0000","last_clean":"2024-09-21T15:02:55.752127+0000","last_became_active":"2024-09-21T14:58:19.411387+0000","last_became_peered":"2024-09-21T14:58:19.411387+0000","last_unstale":"2024-09-21T15:02:55.752127+0000","last_undegraded":"2024-09-21T15:02:55.752127+0000","last_fullsized":"2024-09-21T15:02:55.752127+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"228'341","last_scrub_stamp":"2024-09-21T15:02:55.751791+0000","last_deep_scrub":"228'341","last_deep_scrub_stamp":"2024-09-21T15:02:55.751791+0000","last_clean_scrub_stamp":"2024-09-21T15:02:55.751791+0000","objects_scrubbed":0,"log_size":341,"log_dups_size":0,"ondisk_log_size":341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:43:49.694451+0000","scrub_duration":0.0036333149999999998,"objects_trimmed":0,"snaptrim_duration":0.00045388299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3878,"num_read_kb":28288,"num_write":535,"num_write_kb":26606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"251'573","reported_seq":4320,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:53.921166+0000","last_change":"2024-09-21T15:02:53.921166+0000","last_active":"2024-09-21T15:02:53.921166+0000","last_peered":"2024-09-21T15:02:53.921166+0000","last_clean":"2024-09-21T15:02:53.921166+0000","last_became_active":"2024-09-21T14:58:19.411306+0000","last_became_peered":"2024-09-21T14:58:19.411306+0000","last_unstale":"2024-09-21T15:02:53.921166+0000","last_undegraded":"2024-09-21T15:02:53.921166+0000","last_fullsized":"2024-09-21T15:02:53.921166+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"251'573","last_scrub_stamp":"2024-09-21T15:02:53.920881+0000","last_deep_scrub":"251'573","last_deep_scrub_stamp":"2024-09-21T15:02:53.920881+0000","last_clean_scrub_stamp":"2024-09-21T15:02:53.920881+0000","objects_scrubbed":1,"log_size":573,"log_dups_size":0,"ondisk_log_size":573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:24:58.256436+0000","scrub_duration":0.0067313260000000001,"objects_trimmed":0,"snaptrim_duration":0.0011721469999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4050,"num_read_kb":24676,"num_write":863,"num_write_kb":25228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"227'536","reported_seq":4305,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:41.892311+0000","last_change":"2024-09-21T15:02:41.892553+0000","last_active":"2024-09-21T15:02:41.892311+0000","last_peered":"2024-09-21T15:02:41.892311+0000","last_clean":"2024-09-21T15:02:41.892311+0000","last_became_active":"2024-09-21T15:02:17.242801+0000","last_became_peered":"2024-09-21T15:02:17.242801+0000","last_unstale":"2024-09-21T15:02:41.892311+0000","last_undegraded":"2024-09-21T15:02:41.892311+0000","last_fullsized":"2024-09-21T15:02:41.892311+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"227'536","last_scrub_stamp":"2024-09-21T15:02:41.892260+0000","last_deep_scrub":"227'536","last_deep_scrub_stamp":"2024-09-21T15:02:41.892260+0000","last_clean_scrub_stamp":"2024-09-21T15:02:41.892260+0000","objects_scrubbed":0,"log_size":536,"log_dups_size":0,"ondisk_log_size":536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:02:55.660386+0000","scrub_duration":0.0035449259999999999,"objects_trimmed":0,"snaptrim_duration":0.00032550800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3672,"num_read_kb":32698,"num_write":706,"num_write_kb":22582,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":379,"num_bytes_recovered":13616631,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"251'488","reported_seq":4871,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:45.888473+0000","last_change":"2024-09-21T15:02:45.888473+0000","last_active":"2024-09-21T15:02:45.888473+0000","last_peered":"2024-09-21T15:02:45.888473+0000","last_clean":"2024-09-21T15:02:45.888473+0000","last_became_active":"2024-09-21T14:58:19.409415+0000","last_became_peered":"2024-09-21T14:58:19.409415+0000","last_unstale":"2024-09-21T15:02:45.888473+0000","last_undegraded":"2024-09-21T15:02:45.888473+0000","last_fullsized":"2024-09-21T15:02:45.888473+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"251'488","last_scrub_stamp":"2024-09-21T15:02:45.888038+0000","last_deep_scrub":"251'488","last_deep_scrub_stamp":"2024-09-21T15:02:45.888038+0000","last_clean_scrub_stamp":"2024-09-21T15:02:45.888038+0000","objects_scrubbed":1,"log_size":488,"log_dups_size":0,"ondisk_log_size":488,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:11:04.964634+0000","scrub_duration":0.0079002050000000004,"objects_trimmed":0,"snaptrim_duration":0.0019572090000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4389,"num_read_kb":28349,"num_write":704,"num_write_kb":23876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"228'824","reported_seq":8743,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:44.910392+0000","last_change":"2024-09-21T15:02:44.910761+0000","last_active":"2024-09-21T15:02:44.910392+0000","last_peered":"2024-09-21T15:02:44.910392+0000","last_clean":"2024-09-21T15:02:44.910392+0000","last_became_active":"2024-09-21T15:02:18.246987+0000","last_became_peered":"2024-09-21T15:02:18.246987+0000","last_unstale":"2024-09-21T15:02:44.910392+0000","last_undegraded":"2024-09-21T15:02:44.910392+0000","last_fullsized":"2024-09-21T15:02:44.910392+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"228'824","last_scrub_stamp":"2024-09-21T15:02:44.910267+0000","last_deep_scrub":"228'824","last_deep_scrub_stamp":"2024-09-21T15:02:44.910267+0000","last_clean_scrub_stamp":"2024-09-21T15:02:44.910267+0000","objects_scrubbed":0,"log_size":824,"log_dups_size":0,"ondisk_log_size":824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:59:01.552553+0000","scrub_duration":0.0041170110000000003,"objects_trimmed":0,"snaptrim_duration":0.0014776850000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9383,"num_read_kb":68058,"num_write":1251,"num_write_kb":29921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":394,"num_bytes_recovered":23509073,"num_keys_recovered":66,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"227'688","reported_seq":8806,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:43.600925+0000","last_change":"2024-09-21T15:02:43.600925+0000","last_active":"2024-09-21T15:02:43.600925+0000","last_peered":"2024-09-21T15:02:43.600925+0000","last_clean":"2024-09-21T15:02:43.600925+0000","last_became_active":"2024-09-21T14:58:19.409371+0000","last_became_peered":"2024-09-21T14:58:19.409371+0000","last_unstale":"2024-09-21T15:02:43.600925+0000","last_undegraded":"2024-09-21T15:02:43.600925+0000","last_fullsized":"2024-09-21T15:02:43.600925+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"227'688","last_scrub_stamp":"2024-09-21T15:02:43.600574+0000","last_deep_scrub":"227'688","last_deep_scrub_stamp":"2024-09-21T15:02:43.600574+0000","last_clean_scrub_stamp":"2024-09-21T15:02:43.600574+0000","objects_scrubbed":0,"log_size":688,"log_dups_size":0,"ondisk_log_size":688,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:50:52.626013+0000","scrub_duration":0.0037969309999999999,"objects_trimmed":0,"snaptrim_duration":0.00052048100000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9236,"num_read_kb":45266,"num_write":998,"num_write_kb":30616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"227'413","reported_seq":4018,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:41.571486+0000","last_change":"2024-09-21T15:02:41.571486+0000","last_active":"2024-09-21T15:02:41.571486+0000","last_peered":"2024-09-21T15:02:41.571486+0000","last_clean":"2024-09-21T15:02:41.571486+0000","last_became_active":"2024-09-21T14:58:19.410306+0000","last_became_peered":"2024-09-21T14:58:19.410306+0000","last_unstale":"2024-09-21T15:02:41.571486+0000","last_undegraded":"2024-09-21T15:02:41.571486+0000","last_fullsized":"2024-09-21T15:02:41.571486+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"227'413","last_scrub_stamp":"2024-09-21T15:02:41.571143+0000","last_deep_scrub":"227'413","last_deep_scrub_stamp":"2024-09-21T15:02:41.571143+0000","last_clean_scrub_stamp":"2024-09-21T15:02:41.571143+0000","objects_scrubbed":0,"log_size":413,"log_dups_size":0,"ondisk_log_size":413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:56:07.400850+0000","scrub_duration":0.0040871240000000001,"objects_trimmed":0,"snaptrim_duration":0.00065724399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3380,"num_read_kb":28207,"num_write":769,"num_write_kb":34633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"229'1053","reported_seq":6187,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:52.403113+0000","last_change":"2024-09-21T15:02:52.403417+0000","last_active":"2024-09-21T15:02:52.403113+0000","last_peered":"2024-09-21T15:02:52.403113+0000","last_clean":"2024-09-21T15:02:52.403113+0000","last_became_active":"2024-09-21T15:01:06.166756+0000","last_became_peered":"2024-09-21T15:01:06.166756+0000","last_unstale":"2024-09-21T15:02:52.403113+0000","last_undegraded":"2024-09-21T15:02:52.403113+0000","last_fullsized":"2024-09-21T15:02:52.403113+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"229'1053","last_scrub_stamp":"2024-09-21T15:02:52.403056+0000","last_deep_scrub":"229'1053","last_deep_scrub_stamp":"2024-09-21T15:02:52.403056+0000","last_clean_scrub_stamp":"2024-09-21T15:02:52.403056+0000","objects_scrubbed":0,"log_size":1053,"log_dups_size":0,"ondisk_log_size":1053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:20:25.468011+0000","scrub_duration":0.003478456,"objects_trimmed":0,"snaptrim_duration":0.001157057,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8073,"num_read_kb":43999,"num_write":1621,"num_write_kb":28374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":271,"num_bytes_recovered":16821266,"num_keys_recovered":55,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"230'823","reported_seq":3992,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:43.000714+0000","last_change":"2024-09-21T15:02:43.000714+0000","last_active":"2024-09-21T15:02:43.000714+0000","last_peered":"2024-09-21T15:02:43.000714+0000","last_clean":"2024-09-21T15:02:43.000714+0000","last_became_active":"2024-09-21T14:58:19.410687+0000","last_became_peered":"2024-09-21T14:58:19.410687+0000","last_unstale":"2024-09-21T15:02:43.000714+0000","last_undegraded":"2024-09-21T15:02:43.000714+0000","last_fullsized":"2024-09-21T15:02:43.000714+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"230'823","last_scrub_stamp":"2024-09-21T15:02:43.000429+0000","last_deep_scrub":"230'823","last_deep_scrub_stamp":"2024-09-21T15:02:43.000429+0000","last_clean_scrub_stamp":"2024-09-21T15:02:43.000429+0000","objects_scrubbed":0,"log_size":823,"log_dups_size":0,"ondisk_log_size":823,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:26:50.871886+0000","scrub_duration":0.003893999,"objects_trimmed":0,"snaptrim_duration":0.0017043010000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4757,"num_read_kb":49770,"num_write":1348,"num_write_kb":33339,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"228'527","reported_seq":4206,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:03:01.708564+0000","last_change":"2024-09-21T15:03:01.708952+0000","last_active":"2024-09-21T15:03:01.708564+0000","last_peered":"2024-09-21T15:03:01.708564+0000","last_clean":"2024-09-21T15:03:01.708564+0000","last_became_active":"2024-09-21T14:58:19.407310+0000","last_became_peered":"2024-09-21T14:58:19.407310+0000","last_unstale":"2024-09-21T15:03:01.708564+0000","last_undegraded":"2024-09-21T15:03:01.708564+0000","last_fullsized":"2024-09-21T15:03:01.708564+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"228'527","last_scrub_stamp":"2024-09-21T15:03:01.708443+0000","last_deep_scrub":"228'527","last_deep_scrub_stamp":"2024-09-21T15:03:01.708443+0000","last_clean_scrub_stamp":"2024-09-21T15:03:01.708443+0000","objects_scrubbed":0,"log_size":527,"log_dups_size":0,"ondisk_log_size":527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:39:26.050722+0000","scrub_duration":0.0040227989999999996,"objects_trimmed":0,"snaptrim_duration":0.0016534340000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3756,"num_read_kb":43322,"num_write":713,"num_write_kb":33471,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":257,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:42.857085+0000","last_change":"2024-09-21T15:02:42.857348+0000","last_active":"2024-09-21T15:02:42.857085+0000","last_peered":"2024-09-21T15:02:42.857085+0000","last_clean":"2024-09-21T15:02:42.857085+0000","last_became_active":"2024-09-21T14:58:09.328417+0000","last_became_peered":"2024-09-21T14:58:09.328417+0000","last_unstale":"2024-09-21T15:02:42.857085+0000","last_undegraded":"2024-09-21T15:02:42.857085+0000","last_fullsized":"2024-09-21T15:02:42.857085+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:02:42.857039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:02:42.857039+0000","last_clean_scrub_stamp":"2024-09-21T15:02:42.857039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:46:28.753658+0000","scrub_duration":0.003187479,"objects_trimmed":0,"snaptrim_duration":0.0018917490000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"230'610","reported_seq":6579,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:42.879847+0000","last_change":"2024-09-21T15:02:42.880263+0000","last_active":"2024-09-21T15:02:42.879847+0000","last_peered":"2024-09-21T15:02:42.879847+0000","last_clean":"2024-09-21T15:02:42.879847+0000","last_became_active":"2024-09-21T15:02:17.247583+0000","last_became_peered":"2024-09-21T15:02:17.247583+0000","last_unstale":"2024-09-21T15:02:42.879847+0000","last_undegraded":"2024-09-21T15:02:42.879847+0000","last_fullsized":"2024-09-21T15:02:42.879847+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"230'610","last_scrub_stamp":"2024-09-21T15:02:42.879794+0000","last_deep_scrub":"230'610","last_deep_scrub_stamp":"2024-09-21T15:02:42.879794+0000","last_clean_scrub_stamp":"2024-09-21T15:02:42.879794+0000","objects_scrubbed":0,"log_size":610,"log_dups_size":0,"ondisk_log_size":610,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:34:53.361589+0000","scrub_duration":0.0033718060000000002,"objects_trimmed":0,"snaptrim_duration":0.0013409660000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6297,"num_read_kb":53037,"num_write":697,"num_write_kb":27961,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":11818795,"num_keys_recovered":18,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":36,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:39.875135+0000","last_change":"2024-09-21T15:02:39.875518+0000","last_active":"2024-09-21T15:02:39.875135+0000","last_peered":"2024-09-21T15:02:39.875135+0000","last_clean":"2024-09-21T15:02:39.875135+0000","last_became_active":"2024-09-21T15:02:14.224646+0000","last_became_peered":"2024-09-21T15:02:14.224646+0000","last_unstale":"2024-09-21T15:02:39.875135+0000","last_undegraded":"2024-09-21T15:02:39.875135+0000","last_fullsized":"2024-09-21T15:02:39.875135+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:02:39.875043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:02:39.875043+0000","last_clean_scrub_stamp":"2024-09-21T15:02:39.875043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:21:49.722904+0000","scrub_duration":0.0029364460000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"228'980","reported_seq":4274,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:50.951315+0000","last_change":"2024-09-21T15:02:50.951684+0000","last_active":"2024-09-21T15:02:50.951315+0000","last_peered":"2024-09-21T15:02:50.951315+0000","last_clean":"2024-09-21T15:02:50.951315+0000","last_became_active":"2024-09-21T15:02:19.251008+0000","last_became_peered":"2024-09-21T15:02:19.251008+0000","last_unstale":"2024-09-21T15:02:50.951315+0000","last_undegraded":"2024-09-21T15:02:50.951315+0000","last_fullsized":"2024-09-21T15:02:50.951315+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"228'980","last_scrub_stamp":"2024-09-21T15:02:50.951209+0000","last_deep_scrub":"228'980","last_deep_scrub_stamp":"2024-09-21T15:02:50.951209+0000","last_clean_scrub_stamp":"2024-09-21T15:02:50.951209+0000","objects_scrubbed":0,"log_size":980,"log_dups_size":0,"ondisk_log_size":980,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:05:58.139787+0000","scrub_duration":0.0036781090000000002,"objects_trimmed":0,"snaptrim_duration":0.0015726799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5449,"num_read_kb":31198,"num_write":1400,"num_write_kb":25281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":401,"num_bytes_recovered":12566413,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":289,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:44.807178+0000","last_change":"2024-09-21T15:02:44.807466+0000","last_active":"2024-09-21T15:02:44.807178+0000","last_peered":"2024-09-21T15:02:44.807178+0000","last_clean":"2024-09-21T15:02:44.807178+0000","last_became_active":"2024-09-21T15:02:14.227223+0000","last_became_peered":"2024-09-21T15:02:14.227223+0000","last_unstale":"2024-09-21T15:02:44.807178+0000","last_undegraded":"2024-09-21T15:02:44.807178+0000","last_fullsized":"2024-09-21T15:02:44.807178+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:02:44.807130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:02:44.807130+0000","last_clean_scrub_stamp":"2024-09-21T15:02:44.807130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:25:47.959507+0000","scrub_duration":0.0033867839999999999,"objects_trimmed":0,"snaptrim_duration":0.0019338770000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"227'672","reported_seq":5572,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:43.908800+0000","last_change":"2024-09-21T15:02:43.909036+0000","last_active":"2024-09-21T15:02:43.908800+0000","last_peered":"2024-09-21T15:02:43.908800+0000","last_clean":"2024-09-21T15:02:43.908800+0000","last_became_active":"2024-09-21T15:02:18.246450+0000","last_became_peered":"2024-09-21T15:02:18.246450+0000","last_unstale":"2024-09-21T15:02:43.908800+0000","last_undegraded":"2024-09-21T15:02:43.908800+0000","last_fullsized":"2024-09-21T15:02:43.908800+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"227'672","last_scrub_stamp":"2024-09-21T15:02:43.908749+0000","last_deep_scrub":"227'672","last_deep_scrub_stamp":"2024-09-21T15:02:43.908749+0000","last_clean_scrub_stamp":"2024-09-21T15:02:43.908749+0000","objects_scrubbed":0,"log_size":672,"log_dups_size":0,"ondisk_log_size":672,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:25:22.438634+0000","scrub_duration":0.0034825709999999998,"objects_trimmed":0,"snaptrim_duration":0.00071861500000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5178,"num_read_kb":38459,"num_write":1130,"num_write_kb":41306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":331,"num_bytes_recovered":25796920,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":244,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:03:00.505593+0000","last_change":"2024-09-21T15:03:00.505825+0000","last_active":"2024-09-21T15:03:00.505593+0000","last_peered":"2024-09-21T15:03:00.505593+0000","last_clean":"2024-09-21T15:03:00.505593+0000","last_became_active":"2024-09-21T15:02:24.351245+0000","last_became_peered":"2024-09-21T15:02:24.351245+0000","last_unstale":"2024-09-21T15:03:00.505593+0000","last_undegraded":"2024-09-21T15:03:00.505593+0000","last_fullsized":"2024-09-21T15:03:00.505593+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:03:00.505548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:03:00.505548+0000","last_clean_scrub_stamp":"2024-09-21T15:03:00.505548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:17:44.464666+0000","scrub_duration":0.0029177600000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"227'732","reported_seq":2872,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:46.413573+0000","last_change":"2024-09-21T15:02:46.414002+0000","last_active":"2024-09-21T15:02:46.413573+0000","last_peered":"2024-09-21T15:02:46.413573+0000","last_clean":"2024-09-21T15:02:46.413573+0000","last_became_active":"2024-09-21T14:58:19.408971+0000","last_became_peered":"2024-09-21T14:58:19.408971+0000","last_unstale":"2024-09-21T15:02:46.413573+0000","last_undegraded":"2024-09-21T15:02:46.413573+0000","last_fullsized":"2024-09-21T15:02:46.413573+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"227'732","last_scrub_stamp":"2024-09-21T15:02:46.413448+0000","last_deep_scrub":"227'732","last_deep_scrub_stamp":"2024-09-21T15:02:46.413448+0000","last_clean_scrub_stamp":"2024-09-21T15:02:46.413448+0000","objects_scrubbed":0,"log_size":732,"log_dups_size":0,"ondisk_log_size":732,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:40:10.883458+0000","scrub_duration":0.0039337310000000002,"objects_trimmed":0,"snaptrim_duration":0.00073528299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3602,"num_read_kb":20655,"num_write":1230,"num_write_kb":20679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"251'1010","reported_seq":3703,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:43.964566+0000","last_change":"2024-09-21T15:02:43.964798+0000","last_active":"2024-09-21T15:02:43.964566+0000","last_peered":"2024-09-21T15:02:43.964566+0000","last_clean":"2024-09-21T15:02:43.964566+0000","last_became_active":"2024-09-21T14:58:19.417453+0000","last_became_peered":"2024-09-21T14:58:19.417453+0000","last_unstale":"2024-09-21T15:02:43.964566+0000","last_undegraded":"2024-09-21T15:02:43.964566+0000","last_fullsized":"2024-09-21T15:02:43.964566+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"251'1010","last_scrub_stamp":"2024-09-21T15:02:43.964517+0000","last_deep_scrub":"251'1010","last_deep_scrub_stamp":"2024-09-21T15:02:43.964517+0000","last_clean_scrub_stamp":"2024-09-21T15:02:43.964517+0000","objects_scrubbed":1,"log_size":1010,"log_dups_size":0,"ondisk_log_size":1010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:19:16.788592+0000","scrub_duration":0.0075472940000000004,"objects_trimmed":0,"snaptrim_duration":0.00044463100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4845,"num_read_kb":25284,"num_write":1566,"num_write_kb":21735,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"251'3","reported_seq":260,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:46.842916+0000","last_change":"2024-09-21T15:02:46.843145+0000","last_active":"2024-09-21T15:02:46.842916+0000","last_peered":"2024-09-21T15:02:46.842916+0000","last_clean":"2024-09-21T15:02:46.842916+0000","last_became_active":"2024-09-21T14:58:09.775981+0000","last_became_peered":"2024-09-21T14:58:09.775981+0000","last_unstale":"2024-09-21T15:02:46.842916+0000","last_undegraded":"2024-09-21T15:02:46.842916+0000","last_fullsized":"2024-09-21T15:02:46.842916+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"251'3","last_scrub_stamp":"2024-09-21T15:02:46.842863+0000","last_deep_scrub":"251'3","last_deep_scrub_stamp":"2024-09-21T15:02:46.842863+0000","last_clean_scrub_stamp":"2024-09-21T15:02:46.842863+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:13:15.037486+0000","scrub_duration":0.0065196350000000002,"objects_trimmed":0,"snaptrim_duration":0.0012568880000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"228'1092","reported_seq":4013,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:58.474454+0000","last_change":"2024-09-21T15:02:58.474808+0000","last_active":"2024-09-21T15:02:58.474454+0000","last_peered":"2024-09-21T15:02:58.474454+0000","last_clean":"2024-09-21T15:02:58.474454+0000","last_became_active":"2024-09-21T15:02:24.311531+0000","last_became_peered":"2024-09-21T15:02:24.311531+0000","last_unstale":"2024-09-21T15:02:58.474454+0000","last_undegraded":"2024-09-21T15:02:58.474454+0000","last_fullsized":"2024-09-21T15:02:58.474454+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"228'1092","last_scrub_stamp":"2024-09-21T15:02:58.474400+0000","last_deep_scrub":"228'1092","last_deep_scrub_stamp":"2024-09-21T15:02:58.474400+0000","last_clean_scrub_stamp":"2024-09-21T15:02:58.474400+0000","objects_scrubbed":0,"log_size":1092,"log_dups_size":0,"ondisk_log_size":1092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:27:02.458597+0000","scrub_duration":0.003525673,"objects_trimmed":0,"snaptrim_duration":0.00183417,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6343,"num_read_kb":28797,"num_write":1677,"num_write_kb":26974,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":258,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:42.562315+0000","last_change":"2024-09-21T15:02:42.562594+0000","last_active":"2024-09-21T15:02:42.562315+0000","last_peered":"2024-09-21T15:02:42.562315+0000","last_clean":"2024-09-21T15:02:42.562315+0000","last_became_active":"2024-09-21T14:58:09.330624+0000","last_became_peered":"2024-09-21T14:58:09.330624+0000","last_unstale":"2024-09-21T15:02:42.562315+0000","last_undegraded":"2024-09-21T15:02:42.562315+0000","last_fullsized":"2024-09-21T15:02:42.562315+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:02:42.562269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:02:42.562269+0000","last_clean_scrub_stamp":"2024-09-21T15:02:42.562269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:33:48.021528+0000","scrub_duration":0.0030134150000000002,"objects_trimmed":0,"snaptrim_duration":0.001051151,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"227'339","reported_seq":2565,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:51.132726+0000","last_change":"2024-09-21T15:02:51.133110+0000","last_active":"2024-09-21T15:02:51.132726+0000","last_peered":"2024-09-21T15:02:51.132726+0000","last_clean":"2024-09-21T15:02:51.132726+0000","last_became_active":"2024-09-21T15:01:53.193166+0000","last_became_peered":"2024-09-21T15:01:53.193166+0000","last_unstale":"2024-09-21T15:02:51.132726+0000","last_undegraded":"2024-09-21T15:02:51.132726+0000","last_fullsized":"2024-09-21T15:02:51.132726+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"227'339","last_scrub_stamp":"2024-09-21T15:02:51.132605+0000","last_deep_scrub":"227'339","last_deep_scrub_stamp":"2024-09-21T15:02:51.132605+0000","last_clean_scrub_stamp":"2024-09-21T15:02:51.132605+0000","objects_scrubbed":0,"log_size":339,"log_dups_size":0,"ondisk_log_size":339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:29:13.184215+0000","scrub_duration":0.0039362850000000003,"objects_trimmed":0,"snaptrim_duration":0.0014983959999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2371,"num_read_kb":40193,"num_write":529,"num_write_kb":39390,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":366,"num_bytes_recovered":22723935,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":257,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:52.720296+0000","last_change":"2024-09-21T15:02:52.720548+0000","last_active":"2024-09-21T15:02:52.720296+0000","last_peered":"2024-09-21T15:02:52.720296+0000","last_clean":"2024-09-21T15:02:52.720296+0000","last_became_active":"2024-09-21T14:58:09.329748+0000","last_became_peered":"2024-09-21T14:58:09.329748+0000","last_unstale":"2024-09-21T15:02:52.720296+0000","last_undegraded":"2024-09-21T15:02:52.720296+0000","last_fullsized":"2024-09-21T15:02:52.720296+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:02:52.720244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:02:52.720244+0000","last_clean_scrub_stamp":"2024-09-21T15:02:52.720244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:31:35.905577+0000","scrub_duration":0.0035768499999999999,"objects_trimmed":0,"snaptrim_duration":0.00052698000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"230'1275","reported_seq":8392,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:48.427816+0000","last_change":"2024-09-21T15:02:48.427816+0000","last_active":"2024-09-21T15:02:48.427816+0000","last_peered":"2024-09-21T15:02:48.427816+0000","last_clean":"2024-09-21T15:02:48.427816+0000","last_became_active":"2024-09-21T14:58:41.401003+0000","last_became_peered":"2024-09-21T14:58:41.401003+0000","last_unstale":"2024-09-21T15:02:48.427816+0000","last_undegraded":"2024-09-21T15:02:48.427816+0000","last_fullsized":"2024-09-21T15:02:48.427816+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"230'1275","last_scrub_stamp":"2024-09-21T15:02:48.427488+0000","last_deep_scrub":"230'1275","last_deep_scrub_stamp":"2024-09-21T15:02:48.427488+0000","last_clean_scrub_stamp":"2024-09-21T15:02:48.427488+0000","objects_scrubbed":0,"log_size":1275,"log_dups_size":0,"ondisk_log_size":1275,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:10:31.281504+0000","scrub_duration":0.003581196,"objects_trimmed":0,"snaptrim_duration":0.000746033,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11708,"num_read_kb":50816,"num_write":1955,"num_write_kb":24913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1403348,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"251'2","reported_seq":259,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:48.778737+0000","last_change":"2024-09-21T15:02:48.779105+0000","last_active":"2024-09-21T15:02:48.778737+0000","last_peered":"2024-09-21T15:02:48.778737+0000","last_clean":"2024-09-21T15:02:48.778737+0000","last_became_active":"2024-09-21T14:58:09.438842+0000","last_became_peered":"2024-09-21T14:58:09.438842+0000","last_unstale":"2024-09-21T15:02:48.778737+0000","last_undegraded":"2024-09-21T15:02:48.778737+0000","last_fullsized":"2024-09-21T15:02:48.778737+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"251'2","last_scrub_stamp":"2024-09-21T15:02:48.778686+0000","last_deep_scrub":"251'2","last_deep_scrub_stamp":"2024-09-21T15:02:48.778686+0000","last_clean_scrub_stamp":"2024-09-21T15:02:48.778686+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:57:10.255068+0000","scrub_duration":0.0065253000000000004,"objects_trimmed":0,"snaptrim_duration":0.001426194,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"251'119","reported_seq":408,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:41.838807+0000","last_change":"2024-09-21T15:02:41.839094+0000","last_active":"2024-09-21T15:02:41.838807+0000","last_peered":"2024-09-21T15:02:41.838807+0000","last_clean":"2024-09-21T15:02:41.838807+0000","last_became_active":"2024-09-21T14:58:03.613214+0000","last_became_peered":"2024-09-21T14:58:03.613214+0000","last_unstale":"2024-09-21T15:02:41.838807+0000","last_undegraded":"2024-09-21T15:02:41.838807+0000","last_fullsized":"2024-09-21T15:02:41.838807+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"251'119","last_scrub_stamp":"2024-09-21T15:02:41.838753+0000","last_deep_scrub":"251'119","last_deep_scrub_stamp":"2024-09-21T15:02:41.838753+0000","last_clean_scrub_stamp":"2024-09-21T15:02:41.838753+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:59:31.901927+0000","scrub_duration":0.0092177409999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"228'685","reported_seq":4709,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:52.105924+0000","last_change":"2024-09-21T15:02:52.106241+0000","last_active":"2024-09-21T15:02:52.105924+0000","last_peered":"2024-09-21T15:02:52.105924+0000","last_clean":"2024-09-21T15:02:52.105924+0000","last_became_active":"2024-09-21T15:01:32.177913+0000","last_became_peered":"2024-09-21T15:01:32.177913+0000","last_unstale":"2024-09-21T15:02:52.105924+0000","last_undegraded":"2024-09-21T15:02:52.105924+0000","last_fullsized":"2024-09-21T15:02:52.105924+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"228'685","last_scrub_stamp":"2024-09-21T15:02:52.105873+0000","last_deep_scrub":"228'685","last_deep_scrub_stamp":"2024-09-21T15:02:52.105873+0000","last_clean_scrub_stamp":"2024-09-21T15:02:52.105873+0000","objects_scrubbed":0,"log_size":685,"log_dups_size":0,"ondisk_log_size":685,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:49:28.346429+0000","scrub_duration":0.0036573,"objects_trimmed":0,"snaptrim_duration":0.0015618730000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4407,"num_read_kb":44538,"num_write":1164,"num_write_kb":30502,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":413,"num_bytes_recovered":20409497,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.9","version":"229'973","reported_seq":4132,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:59.778689+0000","last_change":"2024-09-21T15:02:59.779059+0000","last_active":"2024-09-21T15:02:59.778689+0000","last_peered":"2024-09-21T15:02:59.778689+0000","last_clean":"2024-09-21T15:02:59.778689+0000","last_became_active":"2024-09-21T15:02:18.255482+0000","last_became_peered":"2024-09-21T15:02:18.255482+0000","last_unstale":"2024-09-21T15:02:59.778689+0000","last_undegraded":"2024-09-21T15:02:59.778689+0000","last_fullsized":"2024-09-21T15:02:59.778689+0000","mapping_epoch":239,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":240,"parent":"0.0","parent_split_bits":0,"last_scrub":"229'973","last_scrub_stamp":"2024-09-21T15:02:59.778565+0000","last_deep_scrub":"229'973","last_deep_scrub_stamp":"2024-09-21T15:02:59.778565+0000","last_clean_scrub_stamp":"2024-09-21T15:02:59.778565+0000","objects_scrubbed":0,"log_size":973,"log_dups_size":0,"ondisk_log_size":973,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:51:50.383808+0000","scrub_duration":0.0047539829999999998,"objects_trimmed":0,"snaptrim_duration":0.001545105,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4815,"num_read_kb":35700,"num_write":1595,"num_write_kb":33802,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":375,"num_bytes_recovered":14742839,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"229'1027","reported_seq":5042,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:03:00.905364+0000","last_change":"2024-09-21T15:03:00.905605+0000","last_active":"2024-09-21T15:03:00.905364+0000","last_peered":"2024-09-21T15:03:00.905364+0000","last_clean":"2024-09-21T15:03:00.905364+0000","last_became_active":"2024-09-21T15:00:27.280971+0000","last_became_peered":"2024-09-21T15:00:27.280971+0000","last_unstale":"2024-09-21T15:03:00.905364+0000","last_undegraded":"2024-09-21T15:03:00.905364+0000","last_fullsized":"2024-09-21T15:03:00.905364+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"229'1027","last_scrub_stamp":"2024-09-21T15:03:00.905312+0000","last_deep_scrub":"229'1027","last_deep_scrub_stamp":"2024-09-21T15:03:00.905312+0000","last_clean_scrub_stamp":"2024-09-21T15:03:00.905312+0000","objects_scrubbed":0,"log_size":1027,"log_dups_size":0,"ondisk_log_size":1027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:59:30.180721+0000","scrub_duration":0.0036421980000000001,"objects_trimmed":0,"snaptrim_duration":0.0013613779999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7341,"num_read_kb":31141,"num_write":1421,"num_write_kb":33160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":9228123,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"227'593","reported_seq":3460,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:43.548064+0000","last_change":"2024-09-21T15:02:43.548064+0000","last_active":"2024-09-21T15:02:43.548064+0000","last_peered":"2024-09-21T15:02:43.548064+0000","last_clean":"2024-09-21T15:02:43.548064+0000","last_became_active":"2024-09-21T14:58:19.410679+0000","last_became_peered":"2024-09-21T14:58:19.410679+0000","last_unstale":"2024-09-21T15:02:43.548064+0000","last_undegraded":"2024-09-21T15:02:43.548064+0000","last_fullsized":"2024-09-21T15:02:43.548064+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"227'593","last_scrub_stamp":"2024-09-21T15:02:43.547754+0000","last_deep_scrub":"227'593","last_deep_scrub_stamp":"2024-09-21T15:02:43.547754+0000","last_clean_scrub_stamp":"2024-09-21T15:02:43.547754+0000","objects_scrubbed":0,"log_size":593,"log_dups_size":0,"ondisk_log_size":593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:10:36.426327+0000","scrub_duration":0.0039169069999999999,"objects_trimmed":0,"snaptrim_duration":0.001466743,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3111,"num_read_kb":36762,"num_write":889,"num_write_kb":39649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"228'493","reported_seq":4721,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:54.095577+0000","last_change":"2024-09-21T15:02:54.095817+0000","last_active":"2024-09-21T15:02:54.095577+0000","last_peered":"2024-09-21T15:02:54.095577+0000","last_clean":"2024-09-21T15:02:54.095577+0000","last_became_active":"2024-09-21T14:59:25.130057+0000","last_became_peered":"2024-09-21T14:59:25.130057+0000","last_unstale":"2024-09-21T15:02:54.095577+0000","last_undegraded":"2024-09-21T15:02:54.095577+0000","last_fullsized":"2024-09-21T15:02:54.095577+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"228'493","last_scrub_stamp":"2024-09-21T15:02:54.095523+0000","last_deep_scrub":"228'493","last_deep_scrub_stamp":"2024-09-21T15:02:54.095523+0000","last_clean_scrub_stamp":"2024-09-21T15:02:54.095523+0000","objects_scrubbed":0,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:12:06.451414+0000","scrub_duration":0.0037916920000000002,"objects_trimmed":0,"snaptrim_duration":0.0014500649999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4513,"num_read_kb":28786,"num_write":729,"num_write_kb":24616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":9059600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"227'544","reported_seq":5367,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:53.718875+0000","last_change":"2024-09-21T15:02:53.719159+0000","last_active":"2024-09-21T15:02:53.718875+0000","last_peered":"2024-09-21T15:02:53.718875+0000","last_clean":"2024-09-21T15:02:53.718875+0000","last_became_active":"2024-09-21T14:58:19.407884+0000","last_became_peered":"2024-09-21T14:58:19.407884+0000","last_unstale":"2024-09-21T15:02:53.718875+0000","last_undegraded":"2024-09-21T15:02:53.718875+0000","last_fullsized":"2024-09-21T15:02:53.718875+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"227'544","last_scrub_stamp":"2024-09-21T15:02:53.718820+0000","last_deep_scrub":"227'544","last_deep_scrub_stamp":"2024-09-21T15:02:53.718820+0000","last_clean_scrub_stamp":"2024-09-21T15:02:53.718820+0000","objects_scrubbed":0,"log_size":544,"log_dups_size":0,"ondisk_log_size":544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:17:32.825677+0000","scrub_duration":0.0037810449999999998,"objects_trimmed":0,"snaptrim_duration":0.00045666499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4968,"num_read_kb":55828,"num_write":945,"num_write_kb":38650,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"230'879","reported_seq":3231,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:49.427437+0000","last_change":"2024-09-21T15:02:49.427437+0000","last_active":"2024-09-21T15:02:49.427437+0000","last_peered":"2024-09-21T15:02:49.427437+0000","last_clean":"2024-09-21T15:02:49.427437+0000","last_became_active":"2024-09-21T15:01:05.172427+0000","last_became_peered":"2024-09-21T15:01:05.172427+0000","last_unstale":"2024-09-21T15:02:49.427437+0000","last_undegraded":"2024-09-21T15:02:49.427437+0000","last_fullsized":"2024-09-21T15:02:49.427437+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"230'879","last_scrub_stamp":"2024-09-21T15:02:49.427146+0000","last_deep_scrub":"230'879","last_deep_scrub_stamp":"2024-09-21T15:02:49.427146+0000","last_clean_scrub_stamp":"2024-09-21T15:02:49.427146+0000","objects_scrubbed":0,"log_size":879,"log_dups_size":0,"ondisk_log_size":879,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:23:50.808175+0000","scrub_duration":0.0033179379999999999,"objects_trimmed":0,"snaptrim_duration":0.001142789,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4262,"num_read_kb":17288,"num_write":1508,"num_write_kb":17970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":138,"num_bytes_recovered":7486496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"227'532","reported_seq":6222,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:45.994244+0000","last_change":"2024-09-21T15:02:45.994244+0000","last_active":"2024-09-21T15:02:45.994244+0000","last_peered":"2024-09-21T15:02:45.994244+0000","last_clean":"2024-09-21T15:02:45.994244+0000","last_became_active":"2024-09-21T14:58:19.410406+0000","last_became_peered":"2024-09-21T14:58:19.410406+0000","last_unstale":"2024-09-21T15:02:45.994244+0000","last_undegraded":"2024-09-21T15:02:45.994244+0000","last_fullsized":"2024-09-21T15:02:45.994244+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"227'532","last_scrub_stamp":"2024-09-21T15:02:45.993899+0000","last_deep_scrub":"227'532","last_deep_scrub_stamp":"2024-09-21T15:02:45.993899+0000","last_clean_scrub_stamp":"2024-09-21T15:02:45.993899+0000","objects_scrubbed":0,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:50:55.037554+0000","scrub_duration":0.0038182770000000001,"objects_trimmed":0,"snaptrim_duration":0.0014059350000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5807,"num_read_kb":56273,"num_write":817,"num_write_kb":36773,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"228'523","reported_seq":4660,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:55.968699+0000","last_change":"2024-09-21T15:02:55.968699+0000","last_active":"2024-09-21T15:02:55.968699+0000","last_peered":"2024-09-21T15:02:55.968699+0000","last_clean":"2024-09-21T15:02:55.968699+0000","last_became_active":"2024-09-21T14:58:19.409935+0000","last_became_peered":"2024-09-21T14:58:19.409935+0000","last_unstale":"2024-09-21T15:02:55.968699+0000","last_undegraded":"2024-09-21T15:02:55.968699+0000","last_fullsized":"2024-09-21T15:02:55.968699+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"228'523","last_scrub_stamp":"2024-09-21T15:02:55.968343+0000","last_deep_scrub":"228'523","last_deep_scrub_stamp":"2024-09-21T15:02:55.968343+0000","last_clean_scrub_stamp":"2024-09-21T15:02:55.968343+0000","objects_scrubbed":0,"log_size":523,"log_dups_size":0,"ondisk_log_size":523,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:29:33.545250+0000","scrub_duration":0.0039904129999999999,"objects_trimmed":0,"snaptrim_duration":0.00068183199999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3949,"num_read_kb":58693,"num_write":810,"num_write_kb":37279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"230'623","reported_seq":5208,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:03:00.745733+0000","last_change":"2024-09-21T15:03:00.745733+0000","last_active":"2024-09-21T15:03:00.745733+0000","last_peered":"2024-09-21T15:03:00.745733+0000","last_clean":"2024-09-21T15:03:00.745733+0000","last_became_active":"2024-09-21T15:01:05.165955+0000","last_became_peered":"2024-09-21T15:01:05.165955+0000","last_unstale":"2024-09-21T15:03:00.745733+0000","last_undegraded":"2024-09-21T15:03:00.745733+0000","last_fullsized":"2024-09-21T15:03:00.745733+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"230'623","last_scrub_stamp":"2024-09-21T15:03:00.745391+0000","last_deep_scrub":"230'623","last_deep_scrub_stamp":"2024-09-21T15:03:00.745391+0000","last_clean_scrub_stamp":"2024-09-21T15:03:00.745391+0000","objects_scrubbed":0,"log_size":623,"log_dups_size":0,"ondisk_log_size":623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:25:13.568172+0000","scrub_duration":0.0035154880000000002,"objects_trimmed":0,"snaptrim_duration":0.0021463580000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5758,"num_read_kb":28485,"num_write":777,"num_write_kb":24185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":166,"num_bytes_recovered":11965028,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"229'644","reported_seq":8548,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:41.163435+0000","last_change":"2024-09-21T15:02:41.163435+0000","last_active":"2024-09-21T15:02:41.163435+0000","last_peered":"2024-09-21T15:02:41.163435+0000","last_clean":"2024-09-21T15:02:41.163435+0000","last_became_active":"2024-09-21T15:00:07.155144+0000","last_became_peered":"2024-09-21T15:00:07.155144+0000","last_unstale":"2024-09-21T15:02:41.163435+0000","last_undegraded":"2024-09-21T15:02:41.163435+0000","last_fullsized":"2024-09-21T15:02:41.163435+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"229'644","last_scrub_stamp":"2024-09-21T15:02:41.162917+0000","last_deep_scrub":"229'644","last_deep_scrub_stamp":"2024-09-21T15:02:41.162917+0000","last_clean_scrub_stamp":"2024-09-21T15:02:41.162917+0000","objects_scrubbed":0,"log_size":644,"log_dups_size":0,"ondisk_log_size":644,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:05:13.736243+0000","scrub_duration":0.0040251990000000001,"objects_trimmed":0,"snaptrim_duration":0.0010987029999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9100,"num_read_kb":37252,"num_write":1013,"num_write_kb":20896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"227'947","reported_seq":6835,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-09-21T15:02:56.712819+0000","last_change":"2024-09-21T15:02:56.712819+0000","last_active":"2024-09-21T15:02:56.712819+0000","last_peered":"2024-09-21T15:02:56.712819+0000","last_clean":"2024-09-21T15:02:56.712819+0000","last_became_active":"2024-09-21T14:58:19.412030+0000","last_became_peered":"2024-09-21T14:58:19.412030+0000","last_unstale":"2024-09-21T15:02:56.712819+0000","last_undegraded":"2024-09-21T15:02:56.712819+0000","last_fullsized":"2024-09-21T15:02:56.712819+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"227'947","last_scrub_stamp":"2024-09-21T15:02:56.712523+0000","last_deep_scrub":"227'947","last_deep_scrub_stamp":"2024-09-21T15:02:56.712523+0000","last_clean_scrub_stamp":"2024-09-21T15:02:56.712523+0000","objects_scrubbed":0,"log_size":947,"log_dups_size":0,"ondisk_log_size":947,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:45:25.972445+0000","scrub_duration":0.0036850429999999998,"objects_trimmed":0,"snaptrim_duration":0.00067099300000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8465,"num_read_kb":59792,"num_write":1327,"num_write_kb":31587,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":174039,"num_read_kb":1238236,"num_write":34305,"num_write_kb":940835,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3956,"num_bytes_recovered":212783659,"num_keys_recovered":305,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7385,"internal_metadata":0},"log_size":22544,"ondisk_log_size":22544,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":10404,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509508,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34080,"kb_used_data":2968,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94337760,"statfs":{"total":96636764160,"available":96601866240,"internally_reserved":0,"allocated":3039232,"data_stored":1498239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.85899999999999999,"15min":0.84299999999999997},"min":{"1min":0.52600000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.302,"5min":1.351,"15min":1.351},"last":0.72399999999999998},{"interface":"front","average":{"1min":1.0309999999999999,"5min":0.92700000000000005,"15min":0.94999999999999996},"min":{"1min":0.81399999999999995,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.4510000000000001,"5min":1.917,"15min":1.917},"last":1.0620000000000001}]},{"osd":1,"last update":"Sat Sep 21 15:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.84499999999999997,"15min":0.78900000000000003},"min":{"1min":0.66000000000000003,"5min":0.307,"15min":0.307},"max":{"1min":1.3720000000000001,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.94199999999999995},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.83499999999999996,"15min":0.82699999999999996},"min":{"1min":0.47599999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.3149999999999999,"5min":2.2149999999999999,"15min":2.2149999999999999},"last":0.87}]},{"osd":2,"last update":"Sat Sep 21 15:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.85599999999999998,"15min":0.82399999999999995},"min":{"1min":0.623,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.46,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":0.90500000000000003},{"interface":"front","average":{"1min":1.099,"5min":0.95599999999999996,"15min":0.94999999999999996},"min":{"1min":0.67300000000000004,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":1.113}]},{"osd":3,"last update":"Sat Sep 21 15:03:06 2024","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":0.92700000000000005,"15min":0.875},"min":{"1min":0.66700000000000004,"5min":0.37,"15min":0.37},"max":{"1min":1.5,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":0.97699999999999998},{"interface":"front","average":{"1min":1.0700000000000001,"5min":0.98899999999999999,"15min":0.97299999999999998},"min":{"1min":0.81799999999999995,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.5109999999999999,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":1.1990000000000001}]},{"osd":4,"last update":"Sat Sep 21 15:03:14 2024","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.595,"5min":1.595,"15min":1.595},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.81799999999999995}]},{"osd":5,"last update":"Sat Sep 21 15:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.98999999999999999,"15min":1.01},"min":{"1min":0.72199999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.548,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":1.016},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.0189999999999999,"15min":1.0429999999999999},"min":{"1min":0.70399999999999996,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.409,"5min":3.1880000000000002,"15min":3.1880000000000002},"last":1.1659999999999999}]},{"osd":6,"last update":"Sat Sep 21 15:03:06 2024","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.1100000000000001,"15min":1.1419999999999999},"min":{"1min":0.65400000000000003,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.4530000000000001,"5min":3.2730000000000001,"15min":3.2730000000000001},"last":1.254},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.121,"15min":1.0489999999999999},"min":{"1min":0.76300000000000001,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.696,"5min":2.2879999999999998,"15min":2.2879999999999998},"last":1.282}]}]},{"osd":6,"up_from":14,"seq":60129542212,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64420,"kb_used_data":2972,"kb_used_omap":1,"kb_used_meta":61438,"kb_avail":94307420,"statfs":{"total":96636764160,"available":96570798080,"internally_reserved":0,"allocated":3043328,"data_stored":1498258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":62912967},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:03:11 2024","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.85599999999999998,"15min":0.92800000000000005},"min":{"1min":0.40799999999999997,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.276,"5min":1.593,"15min":1.593},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.90000000000000002,"15min":0.93899999999999995},"min":{"1min":0.629,"5min":0.379,"15min":0.379},"max":{"1min":1.337,"5min":1.587,"15min":1.587},"last":0.68899999999999995}]},{"osd":1,"last update":"Sat Sep 21 15:03:11 2024","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.88100000000000001,"15min":0.90600000000000003},"min":{"1min":0.55700000000000005,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":1.2230000000000001},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.82199999999999995,"15min":0.93500000000000005},"min":{"1min":0.59399999999999997,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.1659999999999999,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.89800000000000002}]},{"osd":2,"last update":"Sat Sep 21 15:03:11 2024","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.93700000000000006,"15min":0.97499999999999998},"min":{"1min":0.56399999999999995,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.2789999999999999,"5min":1.758,"15min":1.758},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.86699999999999999,"15min":0.89900000000000002},"min":{"1min":0.70499999999999996,"5min":0.496,"15min":0.496},"max":{"1min":1.1539999999999999,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.92800000000000005}]},{"osd":3,"last update":"Sat Sep 21 15:03:11 2024","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.91400000000000003,"15min":0.94799999999999995},"min":{"1min":0.68600000000000005,"5min":0.434,"15min":0.434},"max":{"1min":1.2969999999999999,"5min":1.835,"15min":1.835},"last":0.99299999999999999},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.0229999999999999,"15min":1.1040000000000001},"min":{"1min":0.80500000000000005,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.2749999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":1.1240000000000001}]},{"osd":4,"last update":"Sat Sep 21 15:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.72599999999999998}]},{"osd":5,"last update":"Sat Sep 21 15:03:11 2024","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":0.94599999999999995,"15min":1.077},"min":{"1min":0.747,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.1890000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":1.077},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.0109999999999999,"15min":1.069},"min":{"1min":0.68100000000000005,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.3380000000000001,"5min":1.609,"15min":1.609},"last":0.877}]},{"osd":7,"last update":"Sat Sep 21 15:03:11 2024","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":1.1020000000000001,"15min":1.2090000000000001},"min":{"1min":0.63600000000000001,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.4259999999999999,"5min":1.8660000000000001,"15min":1.8660000000000001},"last":1.0449999999999999},{"interface":"front","average":{"1min":1.089,"5min":1.0720000000000001,"15min":1.2330000000000001},"min":{"1min":0.89700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.4399999999999999,"5min":2.004,"15min":2.004},"last":1.0149999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542213,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60380,"kb_used_data":3028,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94311460,"statfs":{"total":96636764160,"available":96574935040,"internally_reserved":0,"allocated":3100672,"data_stored":2088626,"data_compressed":5202,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:03:09 2024","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.82099999999999995,"15min":0.96199999999999997},"min":{"1min":0.54000000000000004,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.3400000000000001,"5min":4.4800000000000004,"15min":4.4800000000000004},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.81299999999999994,"15min":0.82299999999999995},"min":{"1min":0.66700000000000004,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.92400000000000004}]},{"osd":1,"last update":"Sat Sep 21 15:03:08 2024","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.85599999999999998,"15min":0.91000000000000003},"min":{"1min":0.60399999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.224,"5min":2.7709999999999999,"15min":2.7709999999999999},"last":0.95899999999999996},{"interface":"front","average":{"1min":1.006,"5min":0.91100000000000003,"15min":0.93300000000000005},"min":{"1min":0.78800000000000003,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.3420000000000001,"5min":2.823,"15min":2.823},"last":0.99099999999999999}]},{"osd":2,"last update":"Sat Sep 21 15:03:08 2024","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.83299999999999996,"15min":0.94699999999999995},"min":{"1min":0.59799999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.1799999999999999,"5min":3.419,"15min":3.419},"last":1.016},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.85299999999999998,"15min":0.89100000000000001},"min":{"1min":0.58699999999999997,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.59999999999999998}]},{"osd":3,"last update":"Sat Sep 21 15:03:08 2024","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":0.84599999999999997,"15min":0.82199999999999995},"min":{"1min":0.73099999999999998,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.381,"5min":1.381,"15min":1.381},"last":1.0860000000000001},{"interface":"front","average":{"1min":1.036,"5min":1.0049999999999999,"15min":1.0680000000000001},"min":{"1min":0.628,"5min":0.435,"15min":0.435},"max":{"1min":1.4910000000000001,"5min":2.6379999999999999,"15min":2.6379999999999999},"last":1.181}]},{"osd":4,"last update":"Sat Sep 21 15:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":0.73699999999999999},{"interface":"front","average":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":1.0580000000000001}]},{"osd":6,"last update":"Sat Sep 21 15:03:08 2024","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":0.92300000000000004,"15min":0.96199999999999997},"min":{"1min":0.57299999999999995,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.571,"5min":1.571,"15min":1.571},"last":0.82599999999999996},{"interface":"front","average":{"1min":1.097,"5min":1.0780000000000001,"15min":1.296},"min":{"1min":0.70799999999999996,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.6679999999999999,"5min":4.2469999999999999,"15min":4.2469999999999999},"last":1.109}]},{"osd":7,"last update":"Sat Sep 21 15:03:09 2024","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.131,"15min":1.3},"min":{"1min":0.83299999999999996,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.7290000000000001,"5min":3.3010000000000002,"15min":3.3010000000000002},"last":0.89300000000000002},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.093,"15min":1.131},"min":{"1min":0.67800000000000005,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.4630000000000001,"5min":2.5880000000000001,"15min":2.5880000000000001},"last":1.137}]}]},{"osd":4,"up_from":235,"seq":1009317314576,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34848,"kb_used_data":2968,"kb_used_omap":8,"kb_used_meta":31863,"kb_avail":94336992,"statfs":{"total":96636764160,"available":96601079808,"internally_reserved":0,"allocated":3039232,"data_stored":1498239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8979,"internal_metadata":32627949},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:03:13 2024","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":1.248},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.71399999999999997}]},{"osd":1,"last update":"Sat Sep 21 15:03:13 2024","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.88}]},{"osd":2,"last update":"Sat Sep 21 15:03:13 2024","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":1.0449999999999999},{"interface":"front","average":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.397,"5min":1.397,"15min":1.397},"last":0.96799999999999997}]},{"osd":3,"last update":"Sat Sep 21 15:03:13 2024","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":1.1539999999999999},{"interface":"front","average":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":1.2090000000000001}]},{"osd":5,"last update":"Sat Sep 21 15:03:13 2024","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":1.004,"15min":1.004},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.4339999999999999,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":0.83399999999999996},{"interface":"front","average":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":1.0740000000000001}]},{"osd":6,"last update":"Sat Sep 21 15:03:13 2024","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.597,"5min":1.597,"15min":1.597},"last":1.282},{"interface":"front","average":{"1min":1.034,"5min":1.034,"15min":1.034},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.411,"5min":1.411,"15min":1.411},"last":1.1339999999999999}]},{"osd":7,"last update":"Sat Sep 21 15:03:13 2024","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"min":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":1.1870000000000001},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"min":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"max":{"1min":1.5229999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":1.306}]}]},{"osd":3,"up_from":14,"seq":60129542213,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78372,"kb_used_data":2972,"kb_used_omap":1,"kb_used_meta":75390,"kb_avail":94293468,"statfs":{"total":96636764160,"available":96556511232,"internally_reserved":0,"allocated":3043328,"data_stored":1498258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":77199816},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.61599999999999999,"15min":0.63100000000000001},"min":{"1min":0.31,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.99299999999999999,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66900000000000004,"15min":0.70499999999999996},"min":{"1min":0.29999999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.94999999999999996,"5min":1.919,"15min":1.919},"last":0.61299999999999999}]},{"osd":1,"last update":"Sat Sep 21 15:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.65700000000000003,"15min":0.67000000000000004},"min":{"1min":0.51700000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.96299999999999997,"5min":1.722,"15min":1.722},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.72299999999999998,"15min":0.78900000000000003},"min":{"1min":0.28499999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.96399999999999997,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.56999999999999995}]},{"osd":2,"last update":"Sat Sep 21 15:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.75700000000000001,"15min":0.73399999999999999},"min":{"1min":0.35699999999999998,"5min":0.311,"15min":0.311},"max":{"1min":1.1499999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.77700000000000002,"15min":0.81599999999999995},"min":{"1min":0.46400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.0109999999999999,"5min":1.254,"15min":1.254},"last":1.046}]},{"osd":4,"last update":"Sat Sep 21 15:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.94499999999999995}]},{"osd":5,"last update":"Sat Sep 21 15:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":1.022,"15min":1.0469999999999999},"min":{"1min":0.64300000000000002,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.4419999999999999,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":0.85899999999999999},{"interface":"front","average":{"1min":1.0740000000000001,"5min":1.0609999999999999,"15min":1.0649999999999999},"min":{"1min":0.86199999999999999,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.5169999999999999,"5min":1.548,"15min":1.548},"last":0.97999999999999998}]},{"osd":6,"last update":"Sat Sep 21 15:03:15 2024","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":1.034,"15min":1.081},"min":{"1min":0.70999999999999996,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.302,"5min":2.1219999999999999,"15min":2.1219999999999999},"last":1.153},{"interface":"front","average":{"1min":1.075,"5min":1.0680000000000001,"15min":1.0960000000000001},"min":{"1min":0.67400000000000004,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.399,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":1.272}]},{"osd":7,"last update":"Sat Sep 21 15:03:15 2024","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.073,"15min":1.121},"min":{"1min":0.84899999999999998,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.4910000000000001,"5min":1.706,"15min":1.706},"last":1.2250000000000001},{"interface":"front","average":{"1min":1.1579999999999999,"5min":1.121,"15min":1.169},"min":{"1min":0.88200000000000001,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.575,"5min":2.3380000000000001,"15min":2.3380000000000001},"last":1.0680000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542211,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70176,"kb_used_data":2968,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94301664,"statfs":{"total":96636764160,"available":96564903936,"internally_reserved":0,"allocated":3039232,"data_stored":1498239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68811206},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:03:07 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64100000000000001,"15min":0.627},"min":{"1min":0.42599999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.3009999999999999,"5min":1.383,"15min":1.383},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.71999999999999997,"15min":0.67800000000000005},"min":{"1min":0.28399999999999997,"5min":0.223,"15min":0.223},"max":{"1min":1.0900000000000001,"5min":2.9169999999999998,"15min":2.9169999999999998},"last":0.50900000000000001}]},{"osd":1,"last update":"Sat Sep 21 15:03:07 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.67700000000000005,"15min":0.68100000000000005},"min":{"1min":0.51100000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.91700000000000004,"5min":2.6520000000000001,"15min":2.6520000000000001},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.74199999999999999,"15min":0.76000000000000001},"min":{"1min":0.46800000000000003,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.351,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.68100000000000005}]},{"osd":3,"last update":"Sat Sep 21 15:03:07 2024","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.40200000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.254,"5min":3.0110000000000001,"15min":3.0110000000000001},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.86299999999999999,"15min":0.83499999999999996},"min":{"1min":0.50600000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.5169999999999999,"5min":4.1440000000000001,"15min":4.1440000000000001},"last":0.86499999999999999}]},{"osd":4,"last update":"Sat Sep 21 15:03:14 2024","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":1.026,"15min":1.026},"min":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":1.5369999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":1.01},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.512,"5min":1.512,"15min":1.512},"last":1.079}]},{"osd":5,"last update":"Sat Sep 21 15:03:07 2024","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.016,"15min":1.01},"min":{"1min":0.73799999999999999,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.6060000000000001,"5min":3.1080000000000001,"15min":3.1080000000000001},"last":0.96199999999999997},{"interface":"front","average":{"1min":1.0780000000000001,"5min":0.97899999999999998,"15min":0.99299999999999999},"min":{"1min":0.79400000000000004,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.286,"5min":1.546,"15min":1.546},"last":1.0529999999999999}]},{"osd":6,"last update":"Sat Sep 21 15:03:07 2024","interfaces":[{"interface":"back","average":{"1min":1.091,"5min":1.01,"15min":0.99399999999999999},"min":{"1min":0.72499999999999998,"5min":0.378,"15min":0.378},"max":{"1min":1.677,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":0.91100000000000003},{"interface":"front","average":{"1min":1.155,"5min":1.0640000000000001,"15min":1.0549999999999999},"min":{"1min":0.625,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.4410000000000001,"5min":3.0739999999999998,"15min":3.0739999999999998},"last":0.83899999999999997}]},{"osd":7,"last update":"Sat Sep 21 15:03:10 2024","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.099,"15min":1.0269999999999999},"min":{"1min":0.84499999999999997,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.526,"5min":3.1589999999999998,"15min":3.1589999999999998},"last":1.1739999999999999},{"interface":"front","average":{"1min":1.3049999999999999,"5min":1.155,"15min":1.1100000000000001},"min":{"1min":0.69799999999999995,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.667,"5min":3.1400000000000001,"15min":3.1400000000000001},"last":1.222}]}]},{"osd":1,"up_from":14,"seq":60129542214,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52188,"kb_used_data":3028,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94319652,"statfs":{"total":96636764160,"available":96583323648,"internally_reserved":0,"allocated":3100672,"data_stored":2088626,"data_compressed":5202,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":50330053},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.59199999999999997,"15min":0.60799999999999998},"min":{"1min":0.33600000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.93899999999999995,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.871},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.626,"15min":0.61799999999999999},"min":{"1min":0.47199999999999998,"5min":0.216,"15min":0.216},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.66800000000000004}]},{"osd":2,"last update":"Sat Sep 21 15:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.77400000000000002,"15min":0.73299999999999998},"min":{"1min":0.28299999999999997,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":1.0700000000000001,"5min":11.303000000000001,"15min":11.303000000000001},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.72699999999999998,"15min":0.70699999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.214,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":1.359}]},{"osd":3,"last update":"Sat Sep 21 15:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.78600000000000003,"15min":0.82799999999999996},"min":{"1min":0.43099999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.1879999999999999,"5min":3.1469999999999998,"15min":3.1469999999999998},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.83899999999999997,"15min":0.877},"min":{"1min":0.41199999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.286,"5min":3.2320000000000002,"15min":3.2320000000000002},"last":1.425}]},{"osd":4,"last update":"Sat Sep 21 15:03:17 2024","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.97399999999999998},{"interface":"front","average":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":1.5569999999999999}]},{"osd":5,"last update":"Sat Sep 21 15:03:06 2024","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":1.0760000000000001,"15min":0.95899999999999996},"min":{"1min":0.64300000000000002,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.6220000000000001,"5min":11.534000000000001,"15min":11.534000000000001},"last":0.91100000000000003},{"interface":"front","average":{"1min":1.0609999999999999,"5min":0.96099999999999997,"15min":0.96199999999999997},"min":{"1min":0.78300000000000003,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.3520000000000001,"5min":1.484,"15min":1.484},"last":1.488}]},{"osd":6,"last update":"Sat Sep 21 15:03:06 2024","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":0.96699999999999997,"15min":1.034},"min":{"1min":0.78500000000000003,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.357,"5min":2.3519999999999999,"15min":2.3519999999999999},"last":1.3},{"interface":"front","average":{"1min":1.129,"5min":1.123,"15min":1.0069999999999999},"min":{"1min":0.61299999999999999,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.7250000000000001,"5min":11.601000000000001,"15min":11.601000000000001},"last":1.0349999999999999}]},{"osd":7,"last update":"Sat Sep 21 15:03:06 2024","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.0229999999999999,"15min":1.0569999999999999},"min":{"1min":0.79600000000000004,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.4159999999999999,"5min":2.2360000000000002,"15min":2.2360000000000002},"last":1.379},{"interface":"front","average":{"1min":1.181,"5min":1.198,"15min":1.0960000000000001},"min":{"1min":0.66800000000000004,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.5149999999999999,"5min":11.489000000000001,"15min":11.489000000000001},"last":1.0469999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542211,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70176,"kb_used_data":2968,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94301664,"statfs":{"total":96636764160,"available":96564903936,"internally_reserved":0,"allocated":3039232,"data_stored":1498239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":68811207},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 21 15:03:07 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.623,"15min":0.69999999999999996},"min":{"1min":0.35599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.90000000000000002,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.73699999999999999,"15min":0.64600000000000002},"min":{"1min":0.61599999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.167,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.76500000000000001}]},{"osd":2,"last update":"Sat Sep 21 15:03:07 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.68700000000000006,"15min":0.63},"min":{"1min":0.49099999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.044,"5min":1.153,"15min":1.153},"last":1.0740000000000001},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.70799999999999996,"15min":0.752},"min":{"1min":0.58299999999999996,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.107,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.70099999999999996}]},{"osd":3,"last update":"Sat Sep 21 15:03:07 2024","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.78800000000000003,"15min":0.76200000000000001},"min":{"1min":0.46100000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.77400000000000002,"15min":0.81100000000000005},"min":{"1min":0.54900000000000004,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.3109999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":1.0289999999999999}]},{"osd":4,"last update":"Sat Sep 21 15:03:17 2024","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.65900000000000003},{"interface":"front","average":{"1min":1.163,"5min":1.163,"15min":1.163},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.6990000000000001,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":0.99099999999999999}]},{"osd":5,"last update":"Sat Sep 21 15:03:07 2024","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.0209999999999999,"15min":1.0309999999999999},"min":{"1min":0.80200000000000005,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.7150000000000001,"5min":1.7150000000000001,"15min":1.7150000000000001},"last":1.103},{"interface":"front","average":{"1min":1.129,"5min":1.02,"15min":1.069},"min":{"1min":0.92300000000000004,"5min":0.378,"15min":0.378},"max":{"1min":1.504,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":0.95599999999999996}]},{"osd":6,"last update":"Sat Sep 21 15:03:11 2024","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.94999999999999996,"15min":1.0760000000000001},"min":{"1min":0.80500000000000005,"5min":0.441,"15min":0.441},"max":{"1min":1.145,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":0.85399999999999998},{"interface":"front","average":{"1min":1.1180000000000001,"5min":1.048,"15min":1.0629999999999999},"min":{"1min":0.79300000000000004,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.675,"5min":1.675,"15min":1.675},"last":1.194}]},{"osd":7,"last update":"Sat Sep 21 15:03:07 2024","interfaces":[{"interface":"back","average":{"1min":1.107,"5min":1.0189999999999999,"15min":1.0529999999999999},"min":{"1min":0.79400000000000004,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.3799999999999999,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":1.1479999999999999},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.0069999999999999,"15min":1.105},"min":{"1min":0.81000000000000005,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.4299999999999999,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":0.92300000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5202,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5202,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7385,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T15:03:22.207 DEBUG:teuthology.orchestra.run.smithi032:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-21T15:03:22.548 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-21T15:03:22.548 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-21T15:03:22.548 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-21T15:03:22.548 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T15:03:28.650 INFO:tasks.ceph.osd.0:Stopped 2024-09-21T15:03:28.650 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-21T15:03:28.650 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T15:03:28.738 INFO:tasks.ceph.osd.1:Stopped 2024-09-21T15:03:28.738 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-21T15:03:28.739 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T15:03:28.831 INFO:tasks.ceph.osd.2:Stopped 2024-09-21T15:03:28.832 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-21T15:03:28.832 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T15:03:29.737 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2024-09-21T15:03:34.933 INFO:tasks.ceph.osd.3:Stopped 2024-09-21T15:03:34.934 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-21T15:03:34.934 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T15:03:35.645 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2024-09-21T15:03:35.645 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2024-09-21T15:03:41.036 INFO:tasks.ceph.osd.4:Stopped 2024-09-21T15:03:41.036 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-21T15:03:41.036 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T15:03:41.452 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~0s 2024-09-21T15:03:41.452 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2024-09-21T15:03:47.138 INFO:tasks.ceph.osd.5:Stopped 2024-09-21T15:03:47.139 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-21T15:03:47.139 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T15:03:47.157 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2024-09-21T15:03:47.157 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 has been restored 2024-09-21T15:03:52.761 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2024-09-21T15:03:52.762 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 has been restored 2024-09-21T15:03:53.241 INFO:tasks.ceph.osd.6:Stopped 2024-09-21T15:03:53.241 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-21T15:03:53.241 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T15:03:53.340 INFO:tasks.ceph.osd.7:Stopped 2024-09-21T15:03:53.340 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-21T15:03:53.340 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-21T15:03:53.340 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T15:03:53.367 INFO:tasks.ceph.mgr.y:Stopped 2024-09-21T15:03:53.368 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-21T15:03:53.368 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T15:03:53.399 INFO:tasks.ceph.mgr.x:Stopped 2024-09-21T15:03:53.399 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-21T15:03:53.400 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-21T15:03:53.400 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T15:03:53.452 INFO:tasks.ceph.mon.a:Stopped 2024-09-21T15:03:53.452 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-21T15:03:53.452 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T15:03:53.540 INFO:tasks.ceph.mon.c:Stopped 2024-09-21T15:03:53.540 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-21T15:03:53.540 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T15:03:53.590 INFO:tasks.ceph.mon.b:Stopped 2024-09-21T15:03:53.590 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-21T15:03:53.590 DEBUG:teuthology.orchestra.run.smithi032:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | head -n 1 2024-09-21T15:03:53.611 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi032.front.sepia.ceph.com 2024-09-21T15:03:53.611 DEBUG:teuthology.orchestra.run.smithi032:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-21T15:03:53.834 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi032.front.sepia.ceph.com 2024-09-21T15:03:53.834 DEBUG:teuthology.orchestra.run.smithi032:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-21T15:03:54.004 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi032.front.sepia.ceph.com 2024-09-21T15:03:54.004 DEBUG:teuthology.orchestra.run.smithi032:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-21T15:03:54.199 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi032.front.sepia.ceph.com 2024-09-21T15:03:54.199 DEBUG:teuthology.orchestra.run.smithi032:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-21T15:03:54.394 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi046.front.sepia.ceph.com 2024-09-21T15:03:54.394 DEBUG:teuthology.orchestra.run.smithi046:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-21T15:03:54.592 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi046.front.sepia.ceph.com 2024-09-21T15:03:54.592 DEBUG:teuthology.orchestra.run.smithi046:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-21T15:03:54.835 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi046.front.sepia.ceph.com 2024-09-21T15:03:54.835 DEBUG:teuthology.orchestra.run.smithi046:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-21T15:03:55.012 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi046.front.sepia.ceph.com 2024-09-21T15:03:55.012 DEBUG:teuthology.orchestra.run.smithi046:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-21T15:03:55.170 INFO:tasks.ceph:Archiving mon data... 2024-09-21T15:03:55.171 DEBUG:teuthology.misc:Transferring archived files from smithi032:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-09-17_22:16:03-rbd-reef-distro-default-smithi/7909956/data/mon.a.tgz 2024-09-21T15:03:55.171 DEBUG:teuthology.orchestra.run.smithi032:> mktemp 2024-09-21T15:03:55.178 INFO:teuthology.orchestra.run.smithi032.stdout:/tmp/tmp.nwIAYgkFIB 2024-09-21T15:03:55.178 DEBUG:teuthology.orchestra.run.smithi032:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.nwIAYgkFIB 2024-09-21T15:03:55.353 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 0666 /tmp/tmp.nwIAYgkFIB 2024-09-21T15:03:55.379 DEBUG:teuthology.orchestra.remote:smithi032:/tmp/tmp.nwIAYgkFIB is 521KB 2024-09-21T15:03:55.444 DEBUG:teuthology.orchestra.run.smithi032:> rm -fr /tmp/tmp.nwIAYgkFIB 2024-09-21T15:03:55.450 DEBUG:teuthology.misc:Transferring archived files from smithi032:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-09-17_22:16:03-rbd-reef-distro-default-smithi/7909956/data/mon.c.tgz 2024-09-21T15:03:55.451 DEBUG:teuthology.orchestra.run.smithi032:> mktemp 2024-09-21T15:03:55.457 INFO:teuthology.orchestra.run.smithi032.stdout:/tmp/tmp.SZ8RoXndfI 2024-09-21T15:03:55.457 DEBUG:teuthology.orchestra.run.smithi032:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.SZ8RoXndfI 2024-09-21T15:03:55.625 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 0666 /tmp/tmp.SZ8RoXndfI 2024-09-21T15:03:55.692 DEBUG:teuthology.orchestra.remote:smithi032:/tmp/tmp.SZ8RoXndfI is 698KB 2024-09-21T15:03:55.760 DEBUG:teuthology.orchestra.run.smithi032:> rm -fr /tmp/tmp.SZ8RoXndfI 2024-09-21T15:03:55.767 DEBUG:teuthology.misc:Transferring archived files from smithi046:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-09-17_22:16:03-rbd-reef-distro-default-smithi/7909956/data/mon.b.tgz 2024-09-21T15:03:55.768 DEBUG:teuthology.orchestra.run.smithi046:> mktemp 2024-09-21T15:03:55.775 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/tmp.SfbPOs1wKf 2024-09-21T15:03:55.775 DEBUG:teuthology.orchestra.run.smithi046:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.SfbPOs1wKf 2024-09-21T15:03:55.951 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 0666 /tmp/tmp.SfbPOs1wKf 2024-09-21T15:03:55.971 DEBUG:teuthology.orchestra.remote:smithi046:/tmp/tmp.SfbPOs1wKf is 698KB 2024-09-21T15:03:56.043 DEBUG:teuthology.orchestra.run.smithi046:> rm -fr /tmp/tmp.SfbPOs1wKf 2024-09-21T15:03:56.049 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-21T15:03:56.049 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-21T15:03:56.051 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-21T15:03:56.066 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2024-09-21T15:03:56.066 DEBUG:teuthology.orchestra.run.smithi032:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-09-21T15:03:56.068 DEBUG:teuthology.orchestra.run.smithi046:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-09-21T15:03:56.096 INFO:teuthology.orchestra.run.smithi046.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-09-21T15:03:56.100 INFO:teuthology.orchestra.run.smithi032.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-09-21T15:03:56.107 INFO:tasks.ceph:Archiving crash dumps... 2024-09-21T15:03:56.108 DEBUG:teuthology.misc:Transferring archived files from smithi032:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-17_22:16:03-rbd-reef-distro-default-smithi/7909956/remote/smithi032/crash 2024-09-21T15:03:56.109 DEBUG:teuthology.orchestra.run.smithi032:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-21T15:03:56.166 DEBUG:teuthology.misc:Transferring archived files from smithi046:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-17_22:16:03-rbd-reef-distro-default-smithi/7909956/remote/smithi046/crash 2024-09-21T15:03:56.167 DEBUG:teuthology.orchestra.run.smithi046:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-21T15:03:56.184 INFO:tasks.ceph:Compressing logs... 2024-09-21T15:03:56.184 DEBUG:teuthology.orchestra.run.smithi032:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T15:03:56.187 DEBUG:teuthology.orchestra.run.smithi046:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T15:03:57.762 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 has been restored 2024-09-21T15:04:10.612 INFO:tasks.ceph:Archiving logs... 2024-09-21T15:04:10.612 DEBUG:teuthology.misc:Transferring archived files from smithi032:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-17_22:16:03-rbd-reef-distro-default-smithi/7909956/remote/smithi032/log 2024-09-21T15:04:10.613 DEBUG:teuthology.orchestra.run.smithi032:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T15:04:12.320 DEBUG:teuthology.misc:Transferring archived files from smithi046:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-17_22:16:03-rbd-reef-distro-default-smithi/7909956/remote/smithi046/log 2024-09-21T15:04:12.321 DEBUG:teuthology.orchestra.run.smithi046:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T15:04:13.290 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-21T15:04:13.298 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-21T15:04:13.299 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T15:04:13.301 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T15:04:13.376 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-21T15:04:13.384 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-21T15:04:13.384 DEBUG:teuthology.orchestra.run.smithi032:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T15:04:13.386 DEBUG:teuthology.orchestra.run.smithi046:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T15:04:13.404 INFO:teuthology.orchestra.run.smithi032.stdout: remote refid st t when poll reach delay offset jitter 2024-09-21T15:04:13.404 INFO:teuthology.orchestra.run.smithi032.stdout:============================================================================== 2024-09-21T15:04:13.404 INFO:teuthology.orchestra.run.smithi032.stdout:+hv01.front.sepi 67.205.162.81 3 u 13 64 377 0.074 1.465 1.785 2024-09-21T15:04:13.404 INFO:teuthology.orchestra.run.smithi032.stdout:*hv02.front.sepi 96.245.170.99 3 u 12 64 377 0.090 2.080 1.274 2024-09-21T15:04:13.404 INFO:teuthology.orchestra.run.smithi032.stdout:+hv03.front.sepi 74.6.168.72 3 u 2 64 377 0.083 0.192 1.418 2024-09-21T15:04:13.405 INFO:teuthology.orchestra.run.smithi032.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-21T15:04:13.407 INFO:teuthology.orchestra.run.smithi046.stdout: remote refid st t when poll reach delay offset jitter 2024-09-21T15:04:13.407 INFO:teuthology.orchestra.run.smithi046.stdout:============================================================================== 2024-09-21T15:04:13.407 INFO:teuthology.orchestra.run.smithi046.stdout:*hv01.front.sepi 67.205.162.81 3 u 4 64 377 0.069 0.419 2.008 2024-09-21T15:04:13.407 INFO:teuthology.orchestra.run.smithi046.stdout:+hv02.front.sepi 96.245.170.99 3 u - 64 377 0.082 4.527 2.241 2024-09-21T15:04:13.407 INFO:teuthology.orchestra.run.smithi046.stdout:+hv03.front.sepi 74.6.168.72 3 u 8 64 377 0.100 -1.206 2.334 2024-09-21T15:04:13.407 INFO:teuthology.orchestra.run.smithi046.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-21T15:04:13.408 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-21T15:04:13.416 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-21T15:04:13.416 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-21T15:04:13.427 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-21T15:04:13.437 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-21T15:04:13.448 INFO:teuthology.task.internal:Duration was 879.479253 seconds 2024-09-21T15:04:13.448 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-21T15:04:13.457 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-21T15:04:13.457 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T15:04:13.460 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T15:04:13.513 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-21T15:04:13.513 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi032.front.sepia.ceph.com 2024-09-21T15:04:13.514 DEBUG:teuthology.orchestra.run.smithi032:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T15:04:13.526 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi046.front.sepia.ceph.com 2024-09-21T15:04:13.527 DEBUG:teuthology.orchestra.run.smithi046:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T15:04:13.540 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-21T15:04:13.540 DEBUG:teuthology.orchestra.run.smithi032:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T15:04:13.542 DEBUG:teuthology.orchestra.run.smithi046:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T15:04:13.621 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-21T15:04:13.622 DEBUG:teuthology.orchestra.run.smithi032:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T15:04:13.631 DEBUG:teuthology.orchestra.run.smithi046:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T15:04:13.730 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-21T15:04:13.740 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-21T15:04:13.740 DEBUG:teuthology.orchestra.run.smithi032:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T15:04:13.757 DEBUG:teuthology.orchestra.run.smithi046:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T15:04:13.773 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-21T15:04:13.790 DEBUG:teuthology.orchestra.run.smithi032:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T15:04:13.803 DEBUG:teuthology.orchestra.run.smithi046:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T15:04:13.816 INFO:teuthology.orchestra.run.smithi032.stdout:kernel.core_pattern = core 2024-09-21T15:04:13.831 INFO:teuthology.orchestra.run.smithi046.stdout:kernel.core_pattern = core 2024-09-21T15:04:13.858 DEBUG:teuthology.orchestra.run.smithi032:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T15:04:13.864 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T15:04:13.864 DEBUG:teuthology.orchestra.run.smithi046:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T15:04:13.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T15:04:13.870 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-21T15:04:13.879 INFO:teuthology.task.internal:Transferring archived files... 2024-09-21T15:04:13.880 DEBUG:teuthology.misc:Transferring archived files from smithi032:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-17_22:16:03-rbd-reef-distro-default-smithi/7909956/remote/smithi032 2024-09-21T15:04:13.880 DEBUG:teuthology.orchestra.run.smithi032:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T15:04:13.912 DEBUG:teuthology.misc:Transferring archived files from smithi046:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-17_22:16:03-rbd-reef-distro-default-smithi/7909956/remote/smithi046 2024-09-21T15:04:13.912 DEBUG:teuthology.orchestra.run.smithi046:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T15:04:13.938 INFO:teuthology.task.internal:Removing archive directory... 2024-09-21T15:04:13.939 DEBUG:teuthology.orchestra.run.smithi032:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T15:04:13.947 DEBUG:teuthology.orchestra.run.smithi046:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T15:04:13.987 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-21T15:04:13.997 INFO:teuthology.task.internal:Not uploading archives. 2024-09-21T15:04:13.997 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-21T15:04:14.007 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-21T15:04:14.007 DEBUG:teuthology.orchestra.run.smithi032:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T15:04:14.010 DEBUG:teuthology.orchestra.run.smithi046:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T15:04:14.015 INFO:teuthology.orchestra.run.smithi032.stdout: 1048593 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 15:04 /home/ubuntu/cephtest 2024-09-21T15:04:14.017 INFO:teuthology.orchestra.run.smithi046.stdout: 1048601 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 15:04 /home/ubuntu/cephtest 2024-09-21T15:04:14.019 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-21T15:04:14.029 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-21T15:04:14.053 INFO:teuthology.run:Summary data: description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-zstd supported-random-distro$/{ubuntu_20.04} thrashers/default thrashosds-health workloads/rbd_fsx_deep_copy} duration: 879.4792528152466 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-21T15:04:14.053 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-21T15:04:14.126 INFO:teuthology.run:pass